2024-09-13T18:02:03.137 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-13T18:02:03.164 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-13T18:02:03.273 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903964 branch: wip-rishabh-testing-20240910.100730-reef description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/iogen}} email: null first_in_suite: false job_id: '7903964' kernel: kdb: 1 sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240910.100730-reef ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 kclient: client.0: mount_subvol_num: 1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240910.100730-reef sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 owner: scheduled_rishabh@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 6439 sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 sleep_before_teardown: 0 subset: 964/1024 suite: fs suite_branch: wip-rishabh-testing-20240910.100730-reef suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi114.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMJKY32fiomyR3WDWT8DzoPslpduDrSNNoTgVxN5bHCVzqr2gUAe8F+djYIBlPLVmERLjU/mtqsFfeLXBO3Ox2o= smithi134.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE/RmZciVyy9wO/z/52/9R1h2vrZxHFGP7TC1z+mp8O7/KyWJtcxv4/sADeXKZl3/K+esQ6FXXb8YM6BfoA8aDc= smithi204.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHEH1cnhvNA60SF4JO8Cv+Wqi4lmw7zMCZ2ymPGLUeMu85e+CCp/CKSvX0uPrGL3QshQcqsaWALSlNltIzfh+Yc= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/iogen.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-13_14:42:16 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-13T18:02:03.273 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/qa; will attempt to use it 2024-09-13T18:02:03.274 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/qa/tasks 2024-09-13T18:02:03.274 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-13T18:02:03.276 INFO:teuthology.task.internal:Checking packages... 2024-09-13T18:02:03.296 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3' 2024-09-13T18:02:03.296 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-13T18:02:03.296 INFO:teuthology.packaging:ref: None 2024-09-13T18:02:03.296 INFO:teuthology.packaging:tag: None 2024-09-13T18:02:03.296 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240910.100730-reef 2024-09-13T18:02:03.296 INFO:teuthology.packaging:sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T18:02:03.297 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240910.100730-reef 2024-09-13T18:02:03.449 INFO:teuthology.task.internal:Found packages for ceph version 18.2.4-860.g6c6b083f 2024-09-13T18:02:03.451 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-13T18:02:03.457 INFO:teuthology.task.internal:no buildpackages task found 2024-09-13T18:02:03.457 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-13T18:02:03.481 INFO:teuthology.task.internal:Saving configuration 2024-09-13T18:02:03.501 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-13T18:02:03.543 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-13T18:02:03.576 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi114.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903964', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 17:56:48.882869', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMJKY32fiomyR3WDWT8DzoPslpduDrSNNoTgVxN5bHCVzqr2gUAe8F+djYIBlPLVmERLjU/mtqsFfeLXBO3Ox2o='} 2024-09-13T18:02:03.607 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi134.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903964', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 17:56:48.884631', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE/RmZciVyy9wO/z/52/9R1h2vrZxHFGP7TC1z+mp8O7/KyWJtcxv4/sADeXKZl3/K+esQ6FXXb8YM6BfoA8aDc='} 2024-09-13T18:02:03.636 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi204.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903964', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 17:56:48.886165', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHEH1cnhvNA60SF4JO8Cv+Wqi4lmw7zMCZ2ymPGLUeMu85e+CCp/CKSvX0uPrGL3QshQcqsaWALSlNltIzfh+Yc='} 2024-09-13T18:02:03.636 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-13T18:02:03.646 INFO:teuthology.task.internal:roles: ubuntu@smithi114.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-13T18:02:03.646 INFO:teuthology.task.internal:roles: ubuntu@smithi134.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-13T18:02:03.646 INFO:teuthology.task.internal:roles: ubuntu@smithi204.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-13T18:02:03.646 INFO:teuthology.run_tasks:Running task console_log... 2024-09-13T18:02:03.824 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f208d47fbe0>, signals=[15]) 2024-09-13T18:02:03.824 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-13T18:02:04.039 INFO:teuthology.task.internal:Opening connections... 2024-09-13T18:02:04.039 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi114.front.sepia.ceph.com 2024-09-13T18:02:04.040 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T18:02:04.126 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi134.front.sepia.ceph.com 2024-09-13T18:02:04.127 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T18:02:04.208 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi204.front.sepia.ceph.com 2024-09-13T18:02:04.208 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T18:02:04.292 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-13T18:02:04.323 DEBUG:teuthology.orchestra.run.smithi114:> uname -m 2024-09-13T18:02:04.342 INFO:teuthology.orchestra.run.smithi114.stdout:x86_64 2024-09-13T18:02:04.342 DEBUG:teuthology.orchestra.run.smithi114:> cat /etc/os-release 2024-09-13T18:02:04.399 INFO:teuthology.orchestra.run.smithi114.stdout:NAME="CentOS Stream" 2024-09-13T18:02:04.400 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION="9" 2024-09-13T18:02:04.400 INFO:teuthology.orchestra.run.smithi114.stdout:ID="centos" 2024-09-13T18:02:04.400 INFO:teuthology.orchestra.run.smithi114.stdout:ID_LIKE="rhel fedora" 2024-09-13T18:02:04.400 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION_ID="9" 2024-09-13T18:02:04.401 INFO:teuthology.orchestra.run.smithi114.stdout:PLATFORM_ID="platform:el9" 2024-09-13T18:02:04.401 INFO:teuthology.orchestra.run.smithi114.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-13T18:02:04.401 INFO:teuthology.orchestra.run.smithi114.stdout:ANSI_COLOR="0;31" 2024-09-13T18:02:04.401 INFO:teuthology.orchestra.run.smithi114.stdout:LOGO="fedora-logo-icon" 2024-09-13T18:02:04.401 INFO:teuthology.orchestra.run.smithi114.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-13T18:02:04.401 INFO:teuthology.orchestra.run.smithi114.stdout:HOME_URL="https://centos.org/" 2024-09-13T18:02:04.402 INFO:teuthology.orchestra.run.smithi114.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-13T18:02:04.402 INFO:teuthology.orchestra.run.smithi114.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-13T18:02:04.402 INFO:teuthology.orchestra.run.smithi114.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-13T18:02:04.403 INFO:teuthology.lock.ops:Updating smithi114.front.sepia.ceph.com on lock server 2024-09-13T18:02:04.433 DEBUG:teuthology.orchestra.run.smithi134:> uname -m 2024-09-13T18:02:04.451 INFO:teuthology.orchestra.run.smithi134.stdout:x86_64 2024-09-13T18:02:04.452 DEBUG:teuthology.orchestra.run.smithi134:> cat /etc/os-release 2024-09-13T18:02:04.509 INFO:teuthology.orchestra.run.smithi134.stdout:NAME="CentOS Stream" 2024-09-13T18:02:04.509 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION="9" 2024-09-13T18:02:04.509 INFO:teuthology.orchestra.run.smithi134.stdout:ID="centos" 2024-09-13T18:02:04.510 INFO:teuthology.orchestra.run.smithi134.stdout:ID_LIKE="rhel fedora" 2024-09-13T18:02:04.510 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION_ID="9" 2024-09-13T18:02:04.510 INFO:teuthology.orchestra.run.smithi134.stdout:PLATFORM_ID="platform:el9" 2024-09-13T18:02:04.510 INFO:teuthology.orchestra.run.smithi134.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-13T18:02:04.510 INFO:teuthology.orchestra.run.smithi134.stdout:ANSI_COLOR="0;31" 2024-09-13T18:02:04.510 INFO:teuthology.orchestra.run.smithi134.stdout:LOGO="fedora-logo-icon" 2024-09-13T18:02:04.510 INFO:teuthology.orchestra.run.smithi134.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-13T18:02:04.510 INFO:teuthology.orchestra.run.smithi134.stdout:HOME_URL="https://centos.org/" 2024-09-13T18:02:04.510 INFO:teuthology.orchestra.run.smithi134.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-13T18:02:04.510 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-13T18:02:04.510 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-13T18:02:04.511 INFO:teuthology.lock.ops:Updating smithi134.front.sepia.ceph.com on lock server 2024-09-13T18:02:04.541 DEBUG:teuthology.orchestra.run.smithi204:> uname -m 2024-09-13T18:02:04.559 INFO:teuthology.orchestra.run.smithi204.stdout:x86_64 2024-09-13T18:02:04.560 DEBUG:teuthology.orchestra.run.smithi204:> cat /etc/os-release 2024-09-13T18:02:04.618 INFO:teuthology.orchestra.run.smithi204.stdout:NAME="CentOS Stream" 2024-09-13T18:02:04.618 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION="9" 2024-09-13T18:02:04.618 INFO:teuthology.orchestra.run.smithi204.stdout:ID="centos" 2024-09-13T18:02:04.619 INFO:teuthology.orchestra.run.smithi204.stdout:ID_LIKE="rhel fedora" 2024-09-13T18:02:04.619 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION_ID="9" 2024-09-13T18:02:04.619 INFO:teuthology.orchestra.run.smithi204.stdout:PLATFORM_ID="platform:el9" 2024-09-13T18:02:04.619 INFO:teuthology.orchestra.run.smithi204.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-13T18:02:04.619 INFO:teuthology.orchestra.run.smithi204.stdout:ANSI_COLOR="0;31" 2024-09-13T18:02:04.619 INFO:teuthology.orchestra.run.smithi204.stdout:LOGO="fedora-logo-icon" 2024-09-13T18:02:04.619 INFO:teuthology.orchestra.run.smithi204.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-13T18:02:04.619 INFO:teuthology.orchestra.run.smithi204.stdout:HOME_URL="https://centos.org/" 2024-09-13T18:02:04.619 INFO:teuthology.orchestra.run.smithi204.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-13T18:02:04.619 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-13T18:02:04.619 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-13T18:02:04.620 INFO:teuthology.lock.ops:Updating smithi204.front.sepia.ceph.com on lock server 2024-09-13T18:02:04.649 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-13T18:02:04.658 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-13T18:02:04.675 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-13T18:02:04.675 DEBUG:teuthology.orchestra.run.smithi114:> test '!' -e /home/ubuntu/cephtest 2024-09-13T18:02:04.678 DEBUG:teuthology.orchestra.run.smithi134:> test '!' -e /home/ubuntu/cephtest 2024-09-13T18:02:04.680 DEBUG:teuthology.orchestra.run.smithi204:> test '!' -e /home/ubuntu/cephtest 2024-09-13T18:02:04.698 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-13T18:02:04.734 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-13T18:02:04.735 DEBUG:teuthology.orchestra.run.smithi114:> test -z $(ls -A /var/lib/ceph) 2024-09-13T18:02:04.737 DEBUG:teuthology.orchestra.run.smithi134:> test -z $(ls -A /var/lib/ceph) 2024-09-13T18:02:04.740 DEBUG:teuthology.orchestra.run.smithi204:> test -z $(ls -A /var/lib/ceph) 2024-09-13T18:02:04.839 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-13T18:02:04.935 INFO:teuthology.run_tasks:Running task kernel... 2024-09-13T18:02:04.958 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-13T18:02:04.958 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'mon.c': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-13T18:02:04.958 DEBUG:teuthology.orchestra.run.smithi114:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T18:02:04.959 DEBUG:teuthology.orchestra.run.smithi134:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T18:02:04.960 DEBUG:teuthology.orchestra.run.smithi204:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T18:02:04.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:02:04.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:02:04.979 DEBUG:teuthology.orchestra.run.smithi114:> uname -r 2024-09-13T18:02:04.979 DEBUG:teuthology.orchestra.run.smithi134:> uname -r 2024-09-13T18:02:04.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:02:04.980 DEBUG:teuthology.orchestra.run.smithi204:> uname -r 2024-09-13T18:02:05.036 INFO:teuthology.orchestra.run.smithi114.stdout:5.14.0-503.el9.x86_64 2024-09-13T18:02:05.036 INFO:teuthology.task.kernel:Running kernel on smithi114: 5.14.0-503.el9.x86_64 2024-09-13T18:02:05.036 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum install -y kernel 2024-09-13T18:02:05.038 INFO:teuthology.orchestra.run.smithi134.stdout:5.14.0-503.el9.x86_64 2024-09-13T18:02:05.038 INFO:teuthology.task.kernel:Running kernel on smithi134: 5.14.0-503.el9.x86_64 2024-09-13T18:02:05.038 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum install -y kernel 2024-09-13T18:02:05.039 INFO:teuthology.orchestra.run.smithi204.stdout:5.14.0-503.el9.x86_64 2024-09-13T18:02:05.039 INFO:teuthology.task.kernel:Running kernel on smithi204: 5.14.0-503.el9.x86_64 2024-09-13T18:02:05.039 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum install -y kernel 2024-09-13T18:02:07.754 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - BaseOS 8.7 MB/s | 8.2 MB 00:00 2024-09-13T18:02:08.564 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - BaseOS 4.5 MB/s | 8.2 MB 00:01 2024-09-13T18:02:09.624 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - BaseOS 2.9 MB/s | 8.2 MB 00:02 2024-09-13T18:02:09.664 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - AppStream 26 MB/s | 20 MB 00:00 2024-09-13T18:02:13.990 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.5 MB 00:00 2024-09-13T18:02:14.065 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - AppStream 4.6 MB/s | 20 MB 00:04 2024-09-13T18:02:15.457 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - Extras packages 70 kB/s | 19 kB 00:00 2024-09-13T18:02:16.861 INFO:teuthology.orchestra.run.smithi114.stdout:Extra Packages for Enterprise Linux 18 MB/s | 23 MB 00:01 2024-09-13T18:02:17.600 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - AppStream 3.0 MB/s | 20 MB 00:06 2024-09-13T18:02:19.640 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.5 MB 00:01 2024-09-13T18:02:21.174 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - Extras packages 55 kB/s | 19 kB 00:00 2024-09-13T18:02:21.884 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.5 MB 00:00 2024-09-13T18:02:22.300 INFO:teuthology.orchestra.run.smithi114.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-13T18:02:22.378 INFO:teuthology.orchestra.run.smithi134.stdout:Extra Packages for Enterprise Linux 21 MB/s | 23 MB 00:01 2024-09-13T18:02:23.285 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - Extras packages 106 kB/s | 19 kB 00:00 2024-09-13T18:02:24.298 INFO:teuthology.orchestra.run.smithi114.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-13T18:02:24.298 INFO:teuthology.orchestra.run.smithi114.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-13T18:02:24.391 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-13T18:02:24.396 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-13T18:02:24.396 INFO:teuthology.orchestra.run.smithi114.stdout: Package Architecture Version Repository Size 2024-09-13T18:02:24.396 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-13T18:02:24.396 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-09-13T18:02:24.396 INFO:teuthology.orchestra.run.smithi114.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-13T18:02:24.396 INFO:teuthology.orchestra.run.smithi114.stdout:Installing dependencies: 2024-09-13T18:02:24.397 INFO:teuthology.orchestra.run.smithi114.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-13T18:02:24.397 INFO:teuthology.orchestra.run.smithi114.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-13T18:02:24.397 INFO:teuthology.orchestra.run.smithi114.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-13T18:02:24.397 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:02:24.397 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-09-13T18:02:24.397 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-13T18:02:24.397 INFO:teuthology.orchestra.run.smithi114.stdout:Install 4 Packages 2024-09-13T18:02:24.397 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:02:24.398 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 79 M 2024-09-13T18:02:24.398 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 126 M 2024-09-13T18:02:24.398 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-09-13T18:02:24.456 INFO:teuthology.orchestra.run.smithi204.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-13T18:02:24.535 INFO:teuthology.orchestra.run.smithi114.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 249 kB/s | 23 kB 00:00 2024-09-13T18:02:26.943 INFO:teuthology.orchestra.run.smithi114.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 6.3 MB/s | 16 MB 00:02 2024-09-13T18:02:27.867 INFO:teuthology.orchestra.run.smithi134.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-13T18:02:29.001 INFO:teuthology.orchestra.run.smithi114.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 6.4 MB/s | 29 MB 00:04 2024-09-13T18:02:29.710 INFO:teuthology.orchestra.run.smithi114.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 6.6 MB/s | 35 MB 00:05 2024-09-13T18:02:29.710 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-09-13T18:02:29.710 INFO:teuthology.orchestra.run.smithi114.stdout:Total 15 MB/s | 79 MB 00:05 2024-09-13T18:02:29.841 INFO:teuthology.orchestra.run.smithi134.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-13T18:02:29.841 INFO:teuthology.orchestra.run.smithi134.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-13T18:02:29.853 INFO:teuthology.orchestra.run.smithi204.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-09-13T18:02:29.926 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2024-09-13T18:02:29.931 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-09-13T18:02:29.931 INFO:teuthology.orchestra.run.smithi134.stdout: Package Architecture Version Repository Size 2024-09-13T18:02:29.931 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-09-13T18:02:29.931 INFO:teuthology.orchestra.run.smithi134.stdout:Installing: 2024-09-13T18:02:29.931 INFO:teuthology.orchestra.run.smithi134.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-13T18:02:29.932 INFO:teuthology.orchestra.run.smithi134.stdout:Installing dependencies: 2024-09-13T18:02:29.932 INFO:teuthology.orchestra.run.smithi134.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-13T18:02:29.932 INFO:teuthology.orchestra.run.smithi134.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-13T18:02:29.932 INFO:teuthology.orchestra.run.smithi134.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-13T18:02:29.932 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-13T18:02:29.932 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2024-09-13T18:02:29.932 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-09-13T18:02:29.932 INFO:teuthology.orchestra.run.smithi134.stdout:Install 4 Packages 2024-09-13T18:02:29.932 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-13T18:02:29.932 INFO:teuthology.orchestra.run.smithi134.stdout:Total download size: 79 M 2024-09-13T18:02:29.932 INFO:teuthology.orchestra.run.smithi134.stdout:Installed size: 126 M 2024-09-13T18:02:29.932 INFO:teuthology.orchestra.run.smithi134.stdout:Downloading Packages: 2024-09-13T18:02:30.110 INFO:teuthology.orchestra.run.smithi134.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 299 kB/s | 23 kB 00:00 2024-09-13T18:02:30.391 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-09-13T18:02:30.730 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-09-13T18:02:30.730 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-09-13T18:02:30.936 INFO:teuthology.orchestra.run.smithi134.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 17 MB/s | 16 MB 00:00 2024-09-13T18:02:31.536 INFO:teuthology.orchestra.run.smithi134.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 20 MB/s | 29 MB 00:01 2024-09-13T18:02:31.695 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-09-13T18:02:31.695 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-09-13T18:02:31.877 INFO:teuthology.orchestra.run.smithi204.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-13T18:02:31.877 INFO:teuthology.orchestra.run.smithi204.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-13T18:02:31.960 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-09-13T18:02:31.966 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-09-13T18:02:31.967 INFO:teuthology.orchestra.run.smithi204.stdout: Package Architecture Version Repository Size 2024-09-13T18:02:31.967 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-09-13T18:02:31.967 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2024-09-13T18:02:31.967 INFO:teuthology.orchestra.run.smithi204.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-13T18:02:31.967 INFO:teuthology.orchestra.run.smithi204.stdout:Installing dependencies: 2024-09-13T18:02:31.967 INFO:teuthology.orchestra.run.smithi204.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-13T18:02:31.967 INFO:teuthology.orchestra.run.smithi204.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-13T18:02:31.967 INFO:teuthology.orchestra.run.smithi204.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-13T18:02:31.967 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-13T18:02:31.967 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-09-13T18:02:31.967 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-09-13T18:02:31.968 INFO:teuthology.orchestra.run.smithi204.stdout:Install 4 Packages 2024-09-13T18:02:31.968 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-13T18:02:31.968 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 79 M 2024-09-13T18:02:31.968 INFO:teuthology.orchestra.run.smithi204.stdout:Installed size: 126 M 2024-09-13T18:02:31.968 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-09-13T18:02:32.094 INFO:teuthology.orchestra.run.smithi134.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 17 MB/s | 35 MB 00:02 2024-09-13T18:02:32.095 INFO:teuthology.orchestra.run.smithi134.stdout:-------------------------------------------------------------------------------- 2024-09-13T18:02:32.095 INFO:teuthology.orchestra.run.smithi134.stdout:Total 36 MB/s | 79 MB 00:02 2024-09-13T18:02:32.296 INFO:teuthology.orchestra.run.smithi204.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 280 kB/s | 23 kB 00:00 2024-09-13T18:02:32.771 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction check 2024-09-13T18:02:33.101 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction check succeeded. 2024-09-13T18:02:33.101 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction test 2024-09-13T18:02:33.122 INFO:teuthology.orchestra.run.smithi204.stdout:(2/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 38 MB/s | 35 MB 00:00 2024-09-13T18:02:33.289 INFO:teuthology.orchestra.run.smithi204.stdout:(3/4): kernel-core-5.14.0-505.el9.x86_64.rpm 15 MB/s | 16 MB 00:01 2024-09-13T18:02:33.939 INFO:teuthology.orchestra.run.smithi204.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 17 MB/s | 29 MB 00:01 2024-09-13T18:02:33.939 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-09-13T18:02:33.939 INFO:teuthology.orchestra.run.smithi204.stdout:Total 40 MB/s | 79 MB 00:01 2024-09-13T18:02:34.053 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction test succeeded. 2024-09-13T18:02:34.053 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction 2024-09-13T18:02:34.158 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-09-13T18:02:34.616 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-09-13T18:02:34.812 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-13T18:02:34.894 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T18:02:34.969 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-09-13T18:02:34.970 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-09-13T18:02:35.956 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-09-13T18:02:35.956 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-09-13T18:02:36.485 INFO:teuthology.orchestra.run.smithi134.stdout: Preparing : 1/1 2024-09-13T18:02:37.173 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-13T18:02:37.255 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T18:02:37.707 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T18:02:37.854 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T18:02:38.409 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-09-13T18:02:39.039 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-13T18:02:39.133 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T18:02:40.051 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T18:02:40.183 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T18:02:41.939 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T18:02:42.071 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T18:02:42.663 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T18:02:42.789 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-13T18:02:45.023 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T18:02:45.059 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-13T18:02:46.928 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T18:02:47.188 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-13T18:02:47.657 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T18:02:49.901 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T18:02:52.047 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T18:03:25.780 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T18:03:25.819 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-13T18:03:26.167 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-13T18:03:26.167 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-13T18:03:26.167 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T18:03:26.167 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T18:03:27.090 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T18:03:27.090 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:03:27.090 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-09-13T18:03:27.090 INFO:teuthology.orchestra.run.smithi114.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-13T18:03:27.090 INFO:teuthology.orchestra.run.smithi114.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-13T18:03:27.090 INFO:teuthology.orchestra.run.smithi114.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-13T18:03:27.090 INFO:teuthology.orchestra.run.smithi114.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-13T18:03:27.091 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:03:27.091 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-13T18:03:27.444 DEBUG:teuthology.orchestra.run.smithi114:> rpm -q kernel | sort -rV | head -n 1 2024-09-13T18:03:27.533 INFO:teuthology.orchestra.run.smithi114.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-13T18:03:27.533 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-13T18:03:27.534 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-13T18:03:27.534 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-13T18:03:27.534 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-13T18:03:27.534 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-13T18:03:27.534 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi114.front.sepia.ceph.com, path=None, version=distro) 2024-09-13T18:03:27.534 DEBUG:teuthology.orchestra.run.smithi114:> rpm -q kernel | sort -rV | head -n 1 2024-09-13T18:03:27.592 INFO:teuthology.orchestra.run.smithi114.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-13T18:03:27.592 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-13T18:03:27.592 DEBUG:teuthology.orchestra.run.smithi114:> sudo rpm -qi grub2-tools 2024-09-13T18:03:27.624 INFO:teuthology.orchestra.run.smithi114.stdout:Name : grub2-tools 2024-09-13T18:03:27.624 INFO:teuthology.orchestra.run.smithi114.stdout:Epoch : 1 2024-09-13T18:03:27.624 INFO:teuthology.orchestra.run.smithi114.stdout:Version : 2.06 2024-09-13T18:03:27.624 INFO:teuthology.orchestra.run.smithi114.stdout:Release : 82.el9 2024-09-13T18:03:27.624 INFO:teuthology.orchestra.run.smithi114.stdout:Architecture: x86_64 2024-09-13T18:03:27.624 INFO:teuthology.orchestra.run.smithi114.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-13T18:03:27.625 INFO:teuthology.orchestra.run.smithi114.stdout:Group : Unspecified 2024-09-13T18:03:27.625 INFO:teuthology.orchestra.run.smithi114.stdout:Size : 8274670 2024-09-13T18:03:27.625 INFO:teuthology.orchestra.run.smithi114.stdout:License : GPLv3+ 2024-09-13T18:03:27.625 INFO:teuthology.orchestra.run.smithi114.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-13T18:03:27.625 INFO:teuthology.orchestra.run.smithi114.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-13T18:03:27.625 INFO:teuthology.orchestra.run.smithi114.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-13T18:03:27.625 INFO:teuthology.orchestra.run.smithi114.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-13T18:03:27.625 INFO:teuthology.orchestra.run.smithi114.stdout:Packager : builder@centos.org 2024-09-13T18:03:27.625 INFO:teuthology.orchestra.run.smithi114.stdout:Vendor : CentOS 2024-09-13T18:03:27.625 INFO:teuthology.orchestra.run.smithi114.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-13T18:03:27.625 INFO:teuthology.orchestra.run.smithi114.stdout:Summary : Support tools for GRUB. 2024-09-13T18:03:27.625 INFO:teuthology.orchestra.run.smithi114.stdout:Description : 2024-09-13T18:03:27.625 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:03:27.625 INFO:teuthology.orchestra.run.smithi114.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-13T18:03:27.626 INFO:teuthology.orchestra.run.smithi114.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-13T18:03:27.626 INFO:teuthology.orchestra.run.smithi114.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-13T18:03:27.626 INFO:teuthology.orchestra.run.smithi114.stdout:hardware devices. 2024-09-13T18:03:27.626 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:03:27.626 INFO:teuthology.orchestra.run.smithi114.stdout:This subpackage provides tools for support of all platforms. 2024-09-13T18:03:27.627 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-13T18:03:27.627 INFO:teuthology.task.kernel:Updating grub on smithi114 to boot 5.14.0-505.el9.x86_64 2024-09-13T18:03:27.627 DEBUG:teuthology.orchestra.run.smithi114:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-13T18:03:28.029 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T18:03:28.069 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-13T18:03:28.257 INFO:teuthology.orchestra.run.smithi114.stderr:Generating grub configuration file ... 2024-09-13T18:03:28.372 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-13T18:03:28.373 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-13T18:03:28.373 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T18:03:28.373 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T18:03:29.198 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T18:03:29.198 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-13T18:03:29.198 INFO:teuthology.orchestra.run.smithi134.stdout:Installed: 2024-09-13T18:03:29.198 INFO:teuthology.orchestra.run.smithi134.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-13T18:03:29.198 INFO:teuthology.orchestra.run.smithi134.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-13T18:03:29.198 INFO:teuthology.orchestra.run.smithi134.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-13T18:03:29.198 INFO:teuthology.orchestra.run.smithi134.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-13T18:03:29.199 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-13T18:03:29.199 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2024-09-13T18:03:29.572 DEBUG:teuthology.orchestra.run.smithi134:> rpm -q kernel | sort -rV | head -n 1 2024-09-13T18:03:29.621 INFO:teuthology.orchestra.run.smithi114.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-13T18:03:29.652 INFO:teuthology.orchestra.run.smithi114.stderr:done 2024-09-13T18:03:29.655 DEBUG:teuthology.orchestra.run.smithi114:> mktemp 2024-09-13T18:03:29.664 INFO:teuthology.orchestra.run.smithi134.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-13T18:03:29.665 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-13T18:03:29.665 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-13T18:03:29.665 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-13T18:03:29.665 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-13T18:03:29.665 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-13T18:03:29.666 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi134.front.sepia.ceph.com, path=None, version=distro) 2024-09-13T18:03:29.666 DEBUG:teuthology.orchestra.run.smithi134:> rpm -q kernel | sort -rV | head -n 1 2024-09-13T18:03:29.670 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/tmp.aAZEXsRSKz 2024-09-13T18:03:29.671 DEBUG:teuthology.orchestra.run.smithi114:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.aAZEXsRSKz 2024-09-13T18:03:29.725 INFO:teuthology.orchestra.run.smithi134.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-13T18:03:29.725 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-13T18:03:29.726 DEBUG:teuthology.orchestra.run.smithi134:> sudo rpm -qi grub2-tools 2024-09-13T18:03:29.744 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 0666 /tmp/tmp.aAZEXsRSKz 2024-09-13T18:03:29.766 INFO:teuthology.orchestra.run.smithi134.stdout:Name : grub2-tools 2024-09-13T18:03:29.766 INFO:teuthology.orchestra.run.smithi134.stdout:Epoch : 1 2024-09-13T18:03:29.766 INFO:teuthology.orchestra.run.smithi134.stdout:Version : 2.06 2024-09-13T18:03:29.766 INFO:teuthology.orchestra.run.smithi134.stdout:Release : 82.el9 2024-09-13T18:03:29.766 INFO:teuthology.orchestra.run.smithi134.stdout:Architecture: x86_64 2024-09-13T18:03:29.766 INFO:teuthology.orchestra.run.smithi134.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-13T18:03:29.766 INFO:teuthology.orchestra.run.smithi134.stdout:Group : Unspecified 2024-09-13T18:03:29.766 INFO:teuthology.orchestra.run.smithi134.stdout:Size : 8274670 2024-09-13T18:03:29.766 INFO:teuthology.orchestra.run.smithi134.stdout:License : GPLv3+ 2024-09-13T18:03:29.766 INFO:teuthology.orchestra.run.smithi134.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-13T18:03:29.766 INFO:teuthology.orchestra.run.smithi134.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-13T18:03:29.766 INFO:teuthology.orchestra.run.smithi134.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-13T18:03:29.767 INFO:teuthology.orchestra.run.smithi134.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-13T18:03:29.767 INFO:teuthology.orchestra.run.smithi134.stdout:Packager : builder@centos.org 2024-09-13T18:03:29.767 INFO:teuthology.orchestra.run.smithi134.stdout:Vendor : CentOS 2024-09-13T18:03:29.767 INFO:teuthology.orchestra.run.smithi134.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-13T18:03:29.767 INFO:teuthology.orchestra.run.smithi134.stdout:Summary : Support tools for GRUB. 2024-09-13T18:03:29.767 INFO:teuthology.orchestra.run.smithi134.stdout:Description : 2024-09-13T18:03:29.767 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-13T18:03:29.767 INFO:teuthology.orchestra.run.smithi134.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-13T18:03:29.767 INFO:teuthology.orchestra.run.smithi134.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-13T18:03:29.767 INFO:teuthology.orchestra.run.smithi134.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-13T18:03:29.767 INFO:teuthology.orchestra.run.smithi134.stdout:hardware devices. 2024-09-13T18:03:29.767 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-13T18:03:29.767 INFO:teuthology.orchestra.run.smithi134.stdout:This subpackage provides tools for support of all platforms. 2024-09-13T18:03:29.768 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-13T18:03:29.768 INFO:teuthology.task.kernel:Updating grub on smithi134 to boot 5.14.0-505.el9.x86_64 2024-09-13T18:03:29.768 DEBUG:teuthology.orchestra.run.smithi134:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-13T18:03:29.887 DEBUG:teuthology.orchestra.remote:smithi114:/tmp/tmp.aAZEXsRSKz is 6KB 2024-09-13T18:03:29.938 DEBUG:teuthology.orchestra.run.smithi114:> rm -fr /tmp/tmp.aAZEXsRSKz 2024-09-13T18:03:29.953 DEBUG:teuthology.orchestra.run.smithi114:> sudo /bin/ls /boot/loader/entries || true 2024-09-13T18:03:30.020 INFO:teuthology.orchestra.run.smithi114.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-13T18:03:30.020 INFO:teuthology.orchestra.run.smithi114.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-13T18:03:30.020 INFO:teuthology.orchestra.run.smithi114.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-13T18:03:30.020 INFO:teuthology.orchestra.run.smithi114.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-13T18:03:30.021 DEBUG:teuthology.orchestra.run.smithi114:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-13T18:03:30.165 DEBUG:teuthology.orchestra.run.smithi114:> sudo shutdown -r now 2024-09-13T18:03:30.328 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T18:03:30.358 INFO:teuthology.orchestra.run.smithi134.stderr:Generating grub configuration file ... 2024-09-13T18:03:30.368 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-13T18:03:30.674 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-13T18:03:30.674 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-13T18:03:30.674 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T18:03:30.674 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T18:03:31.554 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T18:03:31.554 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-13T18:03:31.554 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-09-13T18:03:31.555 INFO:teuthology.orchestra.run.smithi204.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-13T18:03:31.555 INFO:teuthology.orchestra.run.smithi204.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-13T18:03:31.555 INFO:teuthology.orchestra.run.smithi204.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-13T18:03:31.555 INFO:teuthology.orchestra.run.smithi204.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-13T18:03:31.555 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-13T18:03:31.555 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-09-13T18:03:31.717 INFO:teuthology.orchestra.run.smithi134.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-13T18:03:31.752 INFO:teuthology.orchestra.run.smithi134.stderr:done 2024-09-13T18:03:31.754 DEBUG:teuthology.orchestra.run.smithi134:> mktemp 2024-09-13T18:03:31.770 INFO:teuthology.orchestra.run.smithi134.stdout:/tmp/tmp.jfa6fhxbyR 2024-09-13T18:03:31.770 DEBUG:teuthology.orchestra.run.smithi134:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.jfa6fhxbyR 2024-09-13T18:03:31.837 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 0666 /tmp/tmp.jfa6fhxbyR 2024-09-13T18:03:31.896 DEBUG:teuthology.orchestra.run.smithi204:> rpm -q kernel | sort -rV | head -n 1 2024-09-13T18:03:31.982 INFO:teuthology.orchestra.run.smithi204.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-13T18:03:31.983 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-13T18:03:31.983 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-13T18:03:31.983 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-13T18:03:31.983 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-13T18:03:31.983 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2024-09-13T18:03:31.983 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi204.front.sepia.ceph.com, path=None, version=distro) 2024-09-13T18:03:31.983 DEBUG:teuthology.orchestra.run.smithi204:> rpm -q kernel | sort -rV | head -n 1 2024-09-13T18:03:31.987 DEBUG:teuthology.orchestra.remote:smithi134:/tmp/tmp.jfa6fhxbyR is 6KB 2024-09-13T18:03:32.036 DEBUG:teuthology.orchestra.run.smithi134:> rm -fr /tmp/tmp.jfa6fhxbyR 2024-09-13T18:03:32.043 INFO:teuthology.orchestra.run.smithi204.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-13T18:03:32.043 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-13T18:03:32.043 DEBUG:teuthology.orchestra.run.smithi204:> sudo rpm -qi grub2-tools 2024-09-13T18:03:32.052 DEBUG:teuthology.orchestra.run.smithi134:> sudo /bin/ls /boot/loader/entries || true 2024-09-13T18:03:32.081 INFO:teuthology.orchestra.run.smithi204.stdout:Name : grub2-tools 2024-09-13T18:03:32.081 INFO:teuthology.orchestra.run.smithi204.stdout:Epoch : 1 2024-09-13T18:03:32.082 INFO:teuthology.orchestra.run.smithi204.stdout:Version : 2.06 2024-09-13T18:03:32.082 INFO:teuthology.orchestra.run.smithi204.stdout:Release : 82.el9 2024-09-13T18:03:32.082 INFO:teuthology.orchestra.run.smithi204.stdout:Architecture: x86_64 2024-09-13T18:03:32.082 INFO:teuthology.orchestra.run.smithi204.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-13T18:03:32.082 INFO:teuthology.orchestra.run.smithi204.stdout:Group : Unspecified 2024-09-13T18:03:32.082 INFO:teuthology.orchestra.run.smithi204.stdout:Size : 8274670 2024-09-13T18:03:32.083 INFO:teuthology.orchestra.run.smithi204.stdout:License : GPLv3+ 2024-09-13T18:03:32.083 INFO:teuthology.orchestra.run.smithi204.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-13T18:03:32.083 INFO:teuthology.orchestra.run.smithi204.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-13T18:03:32.083 INFO:teuthology.orchestra.run.smithi204.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-13T18:03:32.083 INFO:teuthology.orchestra.run.smithi204.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-13T18:03:32.084 INFO:teuthology.orchestra.run.smithi204.stdout:Packager : builder@centos.org 2024-09-13T18:03:32.084 INFO:teuthology.orchestra.run.smithi204.stdout:Vendor : CentOS 2024-09-13T18:03:32.084 INFO:teuthology.orchestra.run.smithi204.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-13T18:03:32.084 INFO:teuthology.orchestra.run.smithi204.stdout:Summary : Support tools for GRUB. 2024-09-13T18:03:32.084 INFO:teuthology.orchestra.run.smithi204.stdout:Description : 2024-09-13T18:03:32.084 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-13T18:03:32.085 INFO:teuthology.orchestra.run.smithi204.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-13T18:03:32.085 INFO:teuthology.orchestra.run.smithi204.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-13T18:03:32.085 INFO:teuthology.orchestra.run.smithi204.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-13T18:03:32.085 INFO:teuthology.orchestra.run.smithi204.stdout:hardware devices. 2024-09-13T18:03:32.085 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-13T18:03:32.085 INFO:teuthology.orchestra.run.smithi204.stdout:This subpackage provides tools for support of all platforms. 2024-09-13T18:03:32.087 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-13T18:03:32.087 INFO:teuthology.task.kernel:Updating grub on smithi204 to boot 5.14.0-505.el9.x86_64 2024-09-13T18:03:32.087 DEBUG:teuthology.orchestra.run.smithi204:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-13T18:03:32.123 INFO:teuthology.orchestra.run.smithi134.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-13T18:03:32.123 INFO:teuthology.orchestra.run.smithi134.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-13T18:03:32.123 INFO:teuthology.orchestra.run.smithi134.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-13T18:03:32.123 INFO:teuthology.orchestra.run.smithi134.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-13T18:03:32.124 DEBUG:teuthology.orchestra.run.smithi134:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-13T18:03:32.277 DEBUG:teuthology.orchestra.run.smithi134:> sudo shutdown -r now 2024-09-13T18:03:32.747 INFO:teuthology.orchestra.run.smithi204.stderr:Generating grub configuration file ... 2024-09-13T18:03:34.069 INFO:teuthology.orchestra.run.smithi204.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-13T18:03:34.086 INFO:teuthology.orchestra.run.smithi204.stderr:done 2024-09-13T18:03:34.089 DEBUG:teuthology.orchestra.run.smithi204:> mktemp 2024-09-13T18:03:34.105 INFO:teuthology.orchestra.run.smithi204.stdout:/tmp/tmp.RCeRnYBNUi 2024-09-13T18:03:34.105 DEBUG:teuthology.orchestra.run.smithi204:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.RCeRnYBNUi 2024-09-13T18:03:34.172 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 0666 /tmp/tmp.RCeRnYBNUi 2024-09-13T18:03:34.314 DEBUG:teuthology.orchestra.remote:smithi204:/tmp/tmp.RCeRnYBNUi is 6KB 2024-09-13T18:03:34.362 DEBUG:teuthology.orchestra.run.smithi204:> rm -fr /tmp/tmp.RCeRnYBNUi 2024-09-13T18:03:34.377 DEBUG:teuthology.orchestra.run.smithi204:> sudo /bin/ls /boot/loader/entries || true 2024-09-13T18:03:34.440 INFO:teuthology.orchestra.run.smithi204.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-13T18:03:34.440 INFO:teuthology.orchestra.run.smithi204.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-13T18:03:34.440 INFO:teuthology.orchestra.run.smithi204.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-13T18:03:34.440 INFO:teuthology.orchestra.run.smithi204.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-13T18:03:34.441 DEBUG:teuthology.orchestra.run.smithi204:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-13T18:03:34.599 DEBUG:teuthology.orchestra.run.smithi204:> sudo shutdown -r now 2024-09-13T18:04:00.186 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-13T18:04:00.186 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-09-13T18:04:00.187 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T18:04:02.283 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-13T18:04:02.283 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-09-13T18:04:02.284 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T18:04:04.603 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-09-13T18:04:04.603 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-09-13T18:04:04.603 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T18:04:18.621 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.114 2024-09-13T18:04:20.669 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.134 2024-09-13T18:04:22.973 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.204 2024-09-13T18:04:27.625 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-09-13T18:04:27.626 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T18:04:29.672 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-09-13T18:04:29.673 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T18:04:30.717 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.114 2024-09-13T18:04:31.975 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-09-13T18:04:31.976 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T18:04:32.737 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.134 2024-09-13T18:04:35.037 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.204 2024-09-13T18:04:42.725 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-09-13T18:04:42.725 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T18:04:44.739 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-09-13T18:04:44.740 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T18:04:47.040 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-09-13T18:04:47.040 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T18:05:03.165 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.134 2024-09-13T18:05:14.577 DEBUG:teuthology.orchestra.run.smithi114:> true 2024-09-13T18:05:14.977 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-09-13T18:05:14.978 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-13T18:05:14.978 DEBUG:teuthology.orchestra.run.smithi114:> uname -r 2024-09-13T18:05:14.992 INFO:teuthology.orchestra.run.smithi114.stdout:5.14.0-505.el9.x86_64 2024-09-13T18:05:14.993 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-13T18:05:14.993 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T18:05:14.993 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-13T18:05:15.994 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-13T18:05:15.994 DEBUG:teuthology.orchestra.run.smithi114:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-13T18:05:16.090 INFO:teuthology.orchestra.run.smithi114.stdout:ttyS1 2024-09-13T18:05:16.123 DEBUG:teuthology.parallel:result is None 2024-09-13T18:05:18.166 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-09-13T18:05:18.167 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T18:05:18.701 DEBUG:teuthology.orchestra.run.smithi134:> true 2024-09-13T18:05:19.232 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-09-13T18:05:19.232 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-13T18:05:19.232 DEBUG:teuthology.orchestra.run.smithi134:> uname -r 2024-09-13T18:05:19.282 INFO:teuthology.orchestra.run.smithi134.stdout:5.14.0-505.el9.x86_64 2024-09-13T18:05:19.282 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-13T18:05:19.282 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T18:05:19.282 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-13T18:05:20.283 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-13T18:05:20.283 DEBUG:teuthology.orchestra.run.smithi134:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-13T18:05:20.356 INFO:teuthology.orchestra.run.smithi134.stdout:ttyS1 2024-09-13T18:05:20.563 DEBUG:teuthology.parallel:result is None 2024-09-13T18:05:47.051 DEBUG:teuthology.orchestra.remote:timed out 2024-09-13T18:06:02.052 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-09-13T18:06:02.053 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T18:06:02.433 DEBUG:teuthology.orchestra.run.smithi204:> true 2024-09-13T18:06:02.804 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-09-13T18:06:02.805 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.14.0-505.el9.x86_64"... 2024-09-13T18:06:02.805 DEBUG:teuthology.orchestra.run.smithi204:> uname -r 2024-09-13T18:06:02.822 INFO:teuthology.orchestra.run.smithi204.stdout:5.14.0-505.el9.x86_64 2024-09-13T18:06:02.822 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-13T18:06:02.822 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T18:06:02.822 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-13T18:06:03.823 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-09-13T18:06:03.823 DEBUG:teuthology.orchestra.run.smithi204:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-13T18:06:03.911 INFO:teuthology.orchestra.run.smithi204.stdout:ttyS1 2024-09-13T18:06:03.934 DEBUG:teuthology.parallel:result is None 2024-09-13T18:06:03.934 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-13T18:06:03.942 INFO:teuthology.task.internal:Creating test directory... 2024-09-13T18:06:03.942 DEBUG:teuthology.orchestra.run.smithi114:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T18:06:03.945 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T18:06:03.948 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T18:06:03.967 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-13T18:06:03.973 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-13T18:06:04.011 INFO:teuthology.task.internal:Creating archive directory... 2024-09-13T18:06:04.012 DEBUG:teuthology.orchestra.run.smithi114:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T18:06:04.014 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T18:06:04.017 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T18:06:04.104 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-13T18:06:04.110 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-13T18:06:04.111 DEBUG:teuthology.orchestra.run.smithi114:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T18:06:04.113 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T18:06:04.115 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T18:06:04.149 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T18:06:04.152 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T18:06:04.161 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T18:06:04.162 INFO:teuthology.orchestra.run.smithi114.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T18:06:04.163 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T18:06:04.174 INFO:teuthology.orchestra.run.smithi114.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T18:06:04.175 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-13T18:06:04.183 INFO:teuthology.task.internal:Configuring sudo... 2024-09-13T18:06:04.183 DEBUG:teuthology.orchestra.run.smithi114:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T18:06:04.185 DEBUG:teuthology.orchestra.run.smithi134:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T18:06:04.207 DEBUG:teuthology.orchestra.run.smithi204:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T18:06:04.235 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-13T18:06:04.245 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-13T18:06:04.245 DEBUG:teuthology.orchestra.run.smithi114:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T18:06:04.253 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T18:06:04.275 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T18:06:04.293 DEBUG:teuthology.orchestra.run.smithi114:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T18:06:04.360 DEBUG:teuthology.orchestra.run.smithi114:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T18:06:04.402 DEBUG:teuthology.orchestra.run.smithi114:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T18:06:04.490 DEBUG:teuthology.orchestra.run.smithi114:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T18:06:04.527 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-13T18:06:04.527 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T18:06:04.594 DEBUG:teuthology.orchestra.run.smithi134:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T18:06:04.648 DEBUG:teuthology.orchestra.run.smithi134:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T18:06:04.696 DEBUG:teuthology.orchestra.run.smithi134:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T18:06:04.792 DEBUG:teuthology.orchestra.run.smithi134:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T18:06:04.825 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-13T18:06:04.825 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T18:06:04.892 DEBUG:teuthology.orchestra.run.smithi204:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T18:06:04.945 DEBUG:teuthology.orchestra.run.smithi204:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T18:06:04.987 DEBUG:teuthology.orchestra.run.smithi204:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T18:06:05.079 DEBUG:teuthology.orchestra.run.smithi204:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T18:06:05.114 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-13T18:06:05.114 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T18:06:05.183 DEBUG:teuthology.orchestra.run.smithi114:> sudo service rsyslog restart 2024-09-13T18:06:05.185 DEBUG:teuthology.orchestra.run.smithi134:> sudo service rsyslog restart 2024-09-13T18:06:05.187 DEBUG:teuthology.orchestra.run.smithi204:> sudo service rsyslog restart 2024-09-13T18:06:05.235 INFO:teuthology.orchestra.run.smithi134.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-13T18:06:05.250 INFO:teuthology.orchestra.run.smithi114.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-13T18:06:05.267 INFO:teuthology.orchestra.run.smithi204.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-13T18:06:05.731 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-13T18:06:05.739 INFO:teuthology.task.internal:Starting timer... 2024-09-13T18:06:05.739 INFO:teuthology.run_tasks:Running task pcp... 2024-09-13T18:06:05.770 INFO:teuthology.run_tasks:Running task selinux... 2024-09-13T18:06:05.779 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-13T18:06:05.780 DEBUG:teuthology.orchestra.run.smithi114:> sudo service auditd rotate 2024-09-13T18:06:05.842 INFO:teuthology.orchestra.run.smithi114.stdout:Rotating logs: 2024-09-13T18:06:05.844 DEBUG:teuthology.orchestra.run.smithi134:> sudo service auditd rotate 2024-09-13T18:06:05.895 INFO:teuthology.orchestra.run.smithi134.stdout:Rotating logs: 2024-09-13T18:06:05.896 DEBUG:teuthology.orchestra.run.smithi204:> sudo service auditd rotate 2024-09-13T18:06:05.963 INFO:teuthology.orchestra.run.smithi204.stdout:Rotating logs: 2024-09-13T18:06:05.965 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-13T18:06:05.965 DEBUG:teuthology.orchestra.run.smithi114:> /usr/sbin/getenforce 2024-09-13T18:06:05.993 INFO:teuthology.orchestra.run.smithi114.stdout:Permissive 2024-09-13T18:06:05.993 DEBUG:teuthology.orchestra.run.smithi134:> /usr/sbin/getenforce 2024-09-13T18:06:06.021 INFO:teuthology.orchestra.run.smithi134.stdout:Permissive 2024-09-13T18:06:06.022 DEBUG:teuthology.orchestra.run.smithi204:> /usr/sbin/getenforce 2024-09-13T18:06:06.043 INFO:teuthology.orchestra.run.smithi204.stdout:Permissive 2024-09-13T18:06:06.043 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi114.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi134.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi204.front.sepia.ceph.com': 'permissive'} 2024-09-13T18:06:06.043 DEBUG:teuthology.orchestra.run.smithi114:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-13T18:06:06.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:06:06.072 DEBUG:teuthology.orchestra.run.smithi134:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-13T18:06:06.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:06:06.103 DEBUG:teuthology.orchestra.run.smithi204:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-13T18:06:06.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:06:06.129 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-13T18:06:06.130 DEBUG:teuthology.orchestra.run.smithi114:> sudo /usr/sbin/setenforce permissive 2024-09-13T18:06:06.168 DEBUG:teuthology.orchestra.run.smithi134:> sudo /usr/sbin/setenforce permissive 2024-09-13T18:06:06.207 DEBUG:teuthology.orchestra.run.smithi204:> sudo /usr/sbin/setenforce permissive 2024-09-13T18:06:06.244 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-13T18:06:06.254 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-13T18:06:06.266 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-13T18:06:06.267 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi114.front.sepia.ceph.com,smithi134.front.sepia.ceph.com,smithi204.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-13T18:09:56.281 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi114.front.sepia.ceph.com'), Remote(name='ubuntu@smithi134.front.sepia.ceph.com'), Remote(name='ubuntu@smithi204.front.sepia.ceph.com')] 2024-09-13T18:09:56.283 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-09-13T18:09:56.284 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T18:09:56.393 DEBUG:teuthology.orchestra.run.smithi114:> true 2024-09-13T18:09:56.479 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-09-13T18:09:56.479 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-09-13T18:09:56.479 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T18:09:56.563 DEBUG:teuthology.orchestra.run.smithi134:> true 2024-09-13T18:09:56.640 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-09-13T18:09:56.641 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-09-13T18:09:56.641 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T18:09:56.727 DEBUG:teuthology.orchestra.run.smithi204:> true 2024-09-13T18:09:56.810 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-09-13T18:09:56.811 INFO:teuthology.run_tasks:Running task clock... 2024-09-13T18:09:56.823 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-13T18:09:56.823 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T18:09:56.823 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T18:09:56.826 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T18:09:56.826 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T18:09:56.829 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T18:09:56.829 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T18:09:56.857 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-13T18:09:56.858 INFO:teuthology.orchestra.run.smithi114.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-13T18:09:56.872 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-13T18:09:56.874 INFO:teuthology.orchestra.run.smithi114.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-13T18:09:56.885 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-13T18:09:56.896 INFO:teuthology.orchestra.run.smithi134.stderr:sudo: ntpd: command not found 2024-09-13T18:09:56.900 INFO:teuthology.orchestra.run.smithi114.stderr:sudo: ntpd: command not found 2024-09-13T18:09:56.901 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-13T18:09:56.907 INFO:teuthology.orchestra.run.smithi134.stdout:506 Cannot talk to daemon 2024-09-13T18:09:56.912 INFO:teuthology.orchestra.run.smithi114.stdout:506 Cannot talk to daemon 2024-09-13T18:09:56.921 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-13T18:09:56.927 INFO:teuthology.orchestra.run.smithi114.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-13T18:09:56.928 INFO:teuthology.orchestra.run.smithi204.stderr:sudo: ntpd: command not found 2024-09-13T18:09:56.935 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-13T18:09:56.940 INFO:teuthology.orchestra.run.smithi204.stdout:506 Cannot talk to daemon 2024-09-13T18:09:56.941 INFO:teuthology.orchestra.run.smithi114.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-13T18:09:56.955 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-13T18:09:56.970 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-13T18:09:56.989 INFO:teuthology.orchestra.run.smithi134.stderr:bash: line 1: ntpq: command not found 2024-09-13T18:09:56.993 INFO:teuthology.orchestra.run.smithi134.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-13T18:09:56.993 INFO:teuthology.orchestra.run.smithi134.stdout:=============================================================================== 2024-09-13T18:09:56.993 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T18:09:56.993 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T18:09:56.993 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T18:09:56.994 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T18:09:56.997 INFO:teuthology.orchestra.run.smithi114.stderr:bash: line 1: ntpq: command not found 2024-09-13T18:09:57.000 INFO:teuthology.orchestra.run.smithi114.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-13T18:09:57.001 INFO:teuthology.orchestra.run.smithi114.stdout:=============================================================================== 2024-09-13T18:09:57.001 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T18:09:57.001 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T18:09:57.001 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T18:09:57.001 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T18:09:57.038 INFO:teuthology.orchestra.run.smithi204.stderr:bash: line 1: ntpq: command not found 2024-09-13T18:09:57.041 INFO:teuthology.orchestra.run.smithi204.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-13T18:09:57.042 INFO:teuthology.orchestra.run.smithi204.stdout:=============================================================================== 2024-09-13T18:09:57.042 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T18:09:57.042 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T18:09:57.042 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T18:09:57.042 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T18:09:57.043 INFO:teuthology.run_tasks:Running task install... 2024-09-13T18:09:57.053 DEBUG:teuthology.task.install:project ceph 2024-09-13T18:09:57.053 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3'}} 2024-09-13T18:09:57.053 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3'} 2024-09-13T18:09:57.053 INFO:teuthology.task.install:Using flavor: default 2024-09-13T18:09:57.064 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-13T18:09:57.065 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-09-13T18:09:57.065 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3', 'tag': None, 'wait_for_package': False} 2024-09-13T18:09:57.066 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T18:09:57.070 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3', 'tag': None, 'wait_for_package': False} 2024-09-13T18:09:57.070 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T18:09:57.072 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3', 'tag': None, 'wait_for_package': False} 2024-09-13T18:09:57.072 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T18:09:57.231 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/flavors/default/ 2024-09-13T18:09:57.231 INFO:teuthology.task.install.rpm:Package version is 18.2.4-860.g6c6b083f 2024-09-13T18:09:57.249 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/flavors/default/ 2024-09-13T18:09:57.249 INFO:teuthology.task.install.rpm:Package version is 18.2.4-860.g6c6b083f 2024-09-13T18:09:57.250 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/flavors/default/ 2024-09-13T18:09:57.250 INFO:teuthology.task.install.rpm:Package version is 18.2.4-860.g6c6b083f 2024-09-13T18:09:57.359 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-13T18:09:57.359 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-13T18:09:57.359 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-13T18:09:57.369 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-13T18:09:57.369 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-13T18:09:57.369 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-13T18:09:57.377 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-13T18:09:57.377 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-13T18:09:57.377 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-13T18:09:57.386 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-13T18:09:57.387 DEBUG:teuthology.orchestra.run.smithi114:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-13T18:09:57.397 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-13T18:09:57.397 DEBUG:teuthology.orchestra.run.smithi204:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-13T18:09:57.403 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-13T18:09:57.403 DEBUG:teuthology.orchestra.run.smithi134:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-13T18:09:57.454 DEBUG:teuthology.orchestra.run.smithi114:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-13T18:09:57.464 DEBUG:teuthology.orchestra.run.smithi204:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-13T18:09:57.469 DEBUG:teuthology.orchestra.run.smithi134:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-13T18:09:57.531 DEBUG:teuthology.orchestra.run.smithi114:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-13T18:09:57.541 DEBUG:teuthology.orchestra.run.smithi204:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-13T18:09:57.544 DEBUG:teuthology.orchestra.run.smithi134:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-13T18:09:57.597 INFO:teuthology.orchestra.run.smithi114.stdout:check_obsoletes = 1 2024-09-13T18:09:57.599 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum clean all 2024-09-13T18:09:57.606 INFO:teuthology.orchestra.run.smithi204.stdout:check_obsoletes = 1 2024-09-13T18:09:57.608 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum clean all 2024-09-13T18:09:57.611 INFO:teuthology.orchestra.run.smithi134.stdout:check_obsoletes = 1 2024-09-13T18:09:57.611 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum clean all 2024-09-13T18:09:57.832 INFO:teuthology.orchestra.run.smithi204.stdout:45 files removed 2024-09-13T18:09:57.839 INFO:teuthology.orchestra.run.smithi134.stdout:45 files removed 2024-09-13T18:09:57.859 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-13T18:09:57.867 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-13T18:09:57.877 INFO:teuthology.orchestra.run.smithi114.stdout:45 files removed 2024-09-13T18:09:57.904 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-13T18:09:58.428 INFO:teuthology.orchestra.run.smithi204.stdout:ceph packages for x86_64 277 kB/s | 78 kB 00:00 2024-09-13T18:09:58.428 INFO:teuthology.orchestra.run.smithi134.stdout:ceph packages for x86_64 284 kB/s | 78 kB 00:00 2024-09-13T18:09:58.448 INFO:teuthology.orchestra.run.smithi114.stdout:ceph packages for x86_64 296 kB/s | 78 kB 00:00 2024-09-13T18:09:58.712 INFO:teuthology.orchestra.run.smithi134.stdout:ceph noarch packages 48 kB/s | 11 kB 00:00 2024-09-13T18:09:58.721 INFO:teuthology.orchestra.run.smithi204.stdout:ceph noarch packages 46 kB/s | 11 kB 00:00 2024-09-13T18:09:58.732 INFO:teuthology.orchestra.run.smithi114.stdout:ceph noarch packages 48 kB/s | 11 kB 00:00 2024-09-13T18:09:58.987 INFO:teuthology.orchestra.run.smithi134.stdout:ceph source packages 7.8 kB/s | 1.9 kB 00:00 2024-09-13T18:09:58.996 INFO:teuthology.orchestra.run.smithi204.stdout:ceph source packages 7.8 kB/s | 1.9 kB 00:00 2024-09-13T18:09:59.008 INFO:teuthology.orchestra.run.smithi114.stdout:ceph source packages 7.7 kB/s | 1.9 kB 00:00 2024-09-13T18:09:59.396 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - BaseOS 22 MB/s | 8.2 MB 00:00 2024-09-13T18:10:00.291 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - BaseOS 6.6 MB/s | 8.2 MB 00:01 2024-09-13T18:10:01.431 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - AppStream 22 MB/s | 20 MB 00:00 2024-09-13T18:10:02.088 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - BaseOS 2.7 MB/s | 8.2 MB 00:03 2024-09-13T18:10:03.964 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - AppStream 27 MB/s | 20 MB 00:00 2024-09-13T18:10:03.984 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - AppStream 7.8 MB/s | 20 MB 00:02 2024-09-13T18:10:07.006 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.5 MB 00:01 2024-09-13T18:10:08.377 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.5 MB 00:00 2024-09-13T18:10:08.490 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - CRB 8.9 MB/s | 6.5 MB 00:00 2024-09-13T18:10:08.581 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - Extras packages 75 kB/s | 19 kB 00:00 2024-09-13T18:10:09.619 INFO:teuthology.orchestra.run.smithi134.stdout:Extra Packages for Enterprise Linux 25 MB/s | 23 MB 00:00 2024-09-13T18:10:10.003 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - Extras packages 80 kB/s | 19 kB 00:00 2024-09-13T18:10:10.341 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 19 kB 00:00 2024-09-13T18:10:11.423 INFO:teuthology.orchestra.run.smithi114.stdout:Extra Packages for Enterprise Linux 17 MB/s | 23 MB 00:01 2024-09-13T18:10:11.545 INFO:teuthology.orchestra.run.smithi204.stdout:Extra Packages for Enterprise Linux 21 MB/s | 23 MB 00:01 2024-09-13T18:10:15.083 INFO:teuthology.orchestra.run.smithi134.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-13T18:10:16.838 INFO:teuthology.orchestra.run.smithi114.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-09-13T18:10:16.943 INFO:teuthology.orchestra.run.smithi204.stdout:lab-extras 36 kB/s | 1.7 kB 00:00 2024-09-13T18:10:17.458 INFO:teuthology.orchestra.run.smithi134.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-13T18:10:17.459 INFO:teuthology.orchestra.run.smithi134.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-13T18:10:17.472 INFO:teuthology.orchestra.run.smithi134.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-13T18:10:17.473 INFO:teuthology.orchestra.run.smithi134.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-13T18:10:17.474 INFO:teuthology.orchestra.run.smithi134.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-13T18:10:17.475 INFO:teuthology.orchestra.run.smithi134.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-13T18:10:17.477 INFO:teuthology.orchestra.run.smithi134.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-13T18:10:17.480 INFO:teuthology.orchestra.run.smithi134.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-13T18:10:17.481 INFO:teuthology.orchestra.run.smithi134.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-13T18:10:17.541 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2024-09-13T18:10:17.559 INFO:teuthology.orchestra.run.smithi134.stdout:====================================================================================== 2024-09-13T18:10:17.559 INFO:teuthology.orchestra.run.smithi134.stdout: Package Arch Version Repository Size 2024-09-13T18:10:17.559 INFO:teuthology.orchestra.run.smithi134.stdout:====================================================================================== 2024-09-13T18:10:17.559 INFO:teuthology.orchestra.run.smithi134.stdout:Installing: 2024-09-13T18:10:17.559 INFO:teuthology.orchestra.run.smithi134.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-13T18:10:17.559 INFO:teuthology.orchestra.run.smithi134.stdout: ceph x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 6.5 k 2024-09-13T18:10:17.560 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-base x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 5.1 M 2024-09-13T18:10:17.560 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-fuse x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 852 k 2024-09-13T18:10:17.560 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-immutable-object-cache x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 142 k 2024-09-13T18:10:17.560 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 1.5 M 2024-09-13T18:10:17.560 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-cephadm noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 139 k 2024-09-13T18:10:17.560 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-dashboard noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 3.5 M 2024-09-13T18:10:17.560 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 7.4 M 2024-09-13T18:10:17.560 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-rook noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 49 k 2024-09-13T18:10:17.560 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-radosgw x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 7.7 M 2024-09-13T18:10:17.560 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-test x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 40 M 2024-09-13T18:10:17.560 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-volume noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 264 k 2024-09-13T18:10:17.560 INFO:teuthology.orchestra.run.smithi134.stdout: cephadm noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 226 k 2024-09-13T18:10:17.561 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-mirror x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 224 k 2024-09-13T18:10:17.561 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-top noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 18 k 2024-09-13T18:10:17.561 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-13T18:10:17.561 INFO:teuthology.orchestra.run.smithi134.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-13T18:10:17.561 INFO:teuthology.orchestra.run.smithi134.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-13T18:10:17.561 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs-devel x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 31 k 2024-09-13T18:10:17.561 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs2 x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 712 k 2024-09-13T18:10:17.561 INFO:teuthology.orchestra.run.smithi134.stdout: librados-devel x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 127 k 2024-09-13T18:10:17.561 INFO:teuthology.orchestra.run.smithi134.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-13T18:10:17.561 INFO:teuthology.orchestra.run.smithi134.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-13T18:10:17.561 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-13T18:10:17.564 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-13T18:10:17.564 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-13T18:10:17.564 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-13T18:10:17.564 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cephfs x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 162 k 2024-09-13T18:10:17.564 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rados x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 321 k 2024-09-13T18:10:17.565 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rbd x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 301 k 2024-09-13T18:10:17.565 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rgw x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 99 k 2024-09-13T18:10:17.565 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-fuse x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 87 k 2024-09-13T18:10:17.565 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-mirror x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 3.0 M 2024-09-13T18:10:17.565 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-nbd x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 172 k 2024-09-13T18:10:17.565 INFO:teuthology.orchestra.run.smithi134.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-13T18:10:17.565 INFO:teuthology.orchestra.run.smithi134.stdout:Upgrading: 2024-09-13T18:10:17.565 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-13T18:10:17.565 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-13T18:10:17.565 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-13T18:10:17.565 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-13T18:10:17.565 INFO:teuthology.orchestra.run.smithi134.stdout: librados2 x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 3.3 M 2024-09-13T18:10:17.565 INFO:teuthology.orchestra.run.smithi134.stdout: librbd1 x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 3.0 M 2024-09-13T18:10:17.566 INFO:teuthology.orchestra.run.smithi134.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-13T18:10:17.566 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-13T18:10:17.566 INFO:teuthology.orchestra.run.smithi134.stdout:Installing dependencies: 2024-09-13T18:10:17.566 INFO:teuthology.orchestra.run.smithi134.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-13T18:10:17.566 INFO:teuthology.orchestra.run.smithi134.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-13T18:10:17.566 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-common x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 18 M 2024-09-13T18:10:17.566 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-grafana-dashboards noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 24 k 2024-09-13T18:10:17.566 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mds x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 2.1 M 2024-09-13T18:10:17.566 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-modules-core noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 246 k 2024-09-13T18:10:17.566 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mon x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 4.7 M 2024-09-13T18:10:17.566 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-osd x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 17 M 2024-09-13T18:10:17.566 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-prometheus-alerts noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 16 k 2024-09-13T18:10:17.566 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-selinux x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 25 k 2024-09-13T18:10:17.567 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-13T18:10:17.567 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-13T18:10:17.567 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-13T18:10:17.567 INFO:teuthology.orchestra.run.smithi134.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-09-13T18:10:17.567 INFO:teuthology.orchestra.run.smithi134.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-13T18:10:17.567 INFO:teuthology.orchestra.run.smithi134.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-13T18:10:17.567 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-13T18:10:17.567 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-13T18:10:17.567 INFO:teuthology.orchestra.run.smithi134.stdout: libcephsqlite x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 166 k 2024-09-13T18:10:17.567 INFO:teuthology.orchestra.run.smithi134.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-13T18:10:17.567 INFO:teuthology.orchestra.run.smithi134.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-13T18:10:17.567 INFO:teuthology.orchestra.run.smithi134.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-13T18:10:17.567 INFO:teuthology.orchestra.run.smithi134.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-13T18:10:17.567 INFO:teuthology.orchestra.run.smithi134.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-13T18:10:17.568 INFO:teuthology.orchestra.run.smithi134.stdout: libradosstriper1 x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 478 k 2024-09-13T18:10:17.568 INFO:teuthology.orchestra.run.smithi134.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-13T18:10:17.568 INFO:teuthology.orchestra.run.smithi134.stdout: librgw2 x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 4.5 M 2024-09-13T18:10:17.568 INFO:teuthology.orchestra.run.smithi134.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-13T18:10:17.568 INFO:teuthology.orchestra.run.smithi134.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-13T18:10:17.568 INFO:teuthology.orchestra.run.smithi134.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-13T18:10:17.568 INFO:teuthology.orchestra.run.smithi134.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-13T18:10:17.568 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-13T18:10:17.568 INFO:teuthology.orchestra.run.smithi134.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-13T18:10:17.568 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-13T18:10:17.568 INFO:teuthology.orchestra.run.smithi134.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-13T18:10:17.568 INFO:teuthology.orchestra.run.smithi134.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-13T18:10:17.569 INFO:teuthology.orchestra.run.smithi134.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-13T18:10:17.569 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-13T18:10:17.569 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-argparse x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 45 k 2024-09-13T18:10:17.569 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-common x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 129 k 2024-09-13T18:10:17.569 INFO:teuthology.orchestra.run.smithi134.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-13T18:10:17.569 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-13T18:10:17.569 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-13T18:10:17.569 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-13T18:10:17.569 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-13T18:10:17.569 INFO:teuthology.orchestra.run.smithi134.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-13T18:10:17.569 INFO:teuthology.orchestra.run.smithi134.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-13T18:10:17.569 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-13T18:10:17.569 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-13T18:10:17.570 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-13T18:10:17.570 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-13T18:10:17.570 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-13T18:10:17.570 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-13T18:10:17.570 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-13T18:10:17.570 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-13T18:10:17.570 INFO:teuthology.orchestra.run.smithi134.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-13T18:10:17.570 INFO:teuthology.orchestra.run.smithi134.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-13T18:10:17.570 INFO:teuthology.orchestra.run.smithi134.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-13T18:10:17.570 INFO:teuthology.orchestra.run.smithi134.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-13T18:10:17.570 INFO:teuthology.orchestra.run.smithi134.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-13T18:10:17.570 INFO:teuthology.orchestra.run.smithi134.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-13T18:10:17.570 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-13T18:10:17.570 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-13T18:10:17.571 INFO:teuthology.orchestra.run.smithi134.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-13T18:10:17.571 INFO:teuthology.orchestra.run.smithi134.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-13T18:10:17.571 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-13T18:10:17.571 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-13T18:10:17.571 INFO:teuthology.orchestra.run.smithi134.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-13T18:10:17.571 INFO:teuthology.orchestra.run.smithi134.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-13T18:10:17.571 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-13T18:10:17.571 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-13T18:10:17.571 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-13T18:10:17.571 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-13T18:10:17.571 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-13T18:10:17.571 INFO:teuthology.orchestra.run.smithi134.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-13T18:10:17.571 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-13T18:10:17.571 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-13T18:10:17.572 INFO:teuthology.orchestra.run.smithi134.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-13T18:10:17.572 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-13T18:10:17.572 INFO:teuthology.orchestra.run.smithi134.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-13T18:10:17.572 INFO:teuthology.orchestra.run.smithi134.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-13T18:10:17.572 INFO:teuthology.orchestra.run.smithi134.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-13T18:10:17.572 INFO:teuthology.orchestra.run.smithi134.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-13T18:10:17.572 INFO:teuthology.orchestra.run.smithi134.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-13T18:10:17.572 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-13T18:10:17.572 INFO:teuthology.orchestra.run.smithi134.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-13T18:10:17.572 INFO:teuthology.orchestra.run.smithi134.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-13T18:10:17.572 INFO:teuthology.orchestra.run.smithi134.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-13T18:10:17.572 INFO:teuthology.orchestra.run.smithi134.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-13T18:10:17.572 INFO:teuthology.orchestra.run.smithi134.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-13T18:10:17.572 INFO:teuthology.orchestra.run.smithi134.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-13T18:10:17.573 INFO:teuthology.orchestra.run.smithi134.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-13T18:10:17.573 INFO:teuthology.orchestra.run.smithi134.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-13T18:10:17.573 INFO:teuthology.orchestra.run.smithi134.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-13T18:10:17.573 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-13T18:10:17.573 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2024-09-13T18:10:17.573 INFO:teuthology.orchestra.run.smithi134.stdout:====================================================================================== 2024-09-13T18:10:17.573 INFO:teuthology.orchestra.run.smithi134.stdout:Install 130 Packages 2024-09-13T18:10:17.573 INFO:teuthology.orchestra.run.smithi134.stdout:Upgrade 8 Packages 2024-09-13T18:10:17.573 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-13T18:10:17.574 INFO:teuthology.orchestra.run.smithi134.stdout:Total download size: 203 M 2024-09-13T18:10:17.574 INFO:teuthology.orchestra.run.smithi134.stdout:Downloading Packages: 2024-09-13T18:10:17.818 INFO:teuthology.orchestra.run.smithi134.stdout:(1/138): ceph-18.2.4-860.g6c6b083f.el9.x86_64.r 54 kB/s | 6.5 kB 00:00 2024-09-13T18:10:18.043 INFO:teuthology.orchestra.run.smithi134.stdout:(2/138): ceph-fuse-18.2.4-860.g6c6b083f.el9.x86 3.7 MB/s | 852 kB 00:00 2024-09-13T18:10:18.143 INFO:teuthology.orchestra.run.smithi134.stdout:(3/138): ceph-immutable-object-cache-18.2.4-860 1.4 MB/s | 142 kB 00:00 2024-09-13T18:10:18.210 INFO:teuthology.orchestra.run.smithi134.stdout:(4/138): ceph-base-18.2.4-860.g6c6b083f.el9.x86 9.9 MB/s | 5.1 MB 00:00 2024-09-13T18:10:18.302 INFO:teuthology.orchestra.run.smithi134.stdout:(5/138): ceph-mgr-18.2.4-860.g6c6b083f.el9.x86_ 16 MB/s | 1.5 MB 00:00 2024-09-13T18:10:18.352 INFO:teuthology.orchestra.run.smithi134.stdout:(6/138): ceph-mds-18.2.4-860.g6c6b083f.el9.x86_ 10 MB/s | 2.1 MB 00:00 2024-09-13T18:10:18.561 INFO:teuthology.orchestra.run.smithi134.stdout:(7/138): ceph-mon-18.2.4-860.g6c6b083f.el9.x86_ 18 MB/s | 4.7 MB 00:00 2024-09-13T18:10:18.736 INFO:teuthology.orchestra.run.smithi134.stdout:(8/138): ceph-common-18.2.4-860.g6c6b083f.el9.x 18 MB/s | 18 MB 00:01 2024-09-13T18:10:18.778 INFO:teuthology.orchestra.run.smithi134.stdout:(9/138): ceph-selinux-18.2.4-860.g6c6b083f.el9. 607 kB/s | 25 kB 00:00 2024-09-13T18:10:18.978 INFO:teuthology.orchestra.run.smithi134.stdout:(10/138): ceph-radosgw-18.2.4-860.g6c6b083f.el9 19 MB/s | 7.7 MB 00:00 2024-09-13T18:10:19.045 INFO:teuthology.orchestra.run.smithi134.stdout:(11/138): cephfs-mirror-18.2.4-860.g6c6b083f.el 3.3 MB/s | 224 kB 00:00 2024-09-13T18:10:19.129 INFO:teuthology.orchestra.run.smithi134.stdout:(12/138): libcephfs-devel-18.2.4-860.g6c6b083f. 373 kB/s | 31 kB 00:00 2024-09-13T18:10:19.164 INFO:teuthology.orchestra.run.smithi114.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-13T18:10:19.165 INFO:teuthology.orchestra.run.smithi114.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-13T18:10:19.175 INFO:teuthology.orchestra.run.smithi114.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-13T18:10:19.176 INFO:teuthology.orchestra.run.smithi114.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-13T18:10:19.177 INFO:teuthology.orchestra.run.smithi114.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-13T18:10:19.178 INFO:teuthology.orchestra.run.smithi114.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-13T18:10:19.179 INFO:teuthology.orchestra.run.smithi114.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-13T18:10:19.182 INFO:teuthology.orchestra.run.smithi114.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-13T18:10:19.182 INFO:teuthology.orchestra.run.smithi114.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-13T18:10:19.241 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-13T18:10:19.258 INFO:teuthology.orchestra.run.smithi114.stdout:====================================================================================== 2024-09-13T18:10:19.258 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repository Size 2024-09-13T18:10:19.259 INFO:teuthology.orchestra.run.smithi114.stdout:====================================================================================== 2024-09-13T18:10:19.259 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-09-13T18:10:19.259 INFO:teuthology.orchestra.run.smithi114.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-13T18:10:19.259 INFO:teuthology.orchestra.run.smithi114.stdout: ceph x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 6.5 k 2024-09-13T18:10:19.259 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-base x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 5.1 M 2024-09-13T18:10:19.259 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-fuse x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 852 k 2024-09-13T18:10:19.259 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-immutable-object-cache x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 142 k 2024-09-13T18:10:19.259 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 1.5 M 2024-09-13T18:10:19.259 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-cephadm noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 139 k 2024-09-13T18:10:19.259 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-dashboard noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 3.5 M 2024-09-13T18:10:19.259 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 7.4 M 2024-09-13T18:10:19.259 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-rook noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 49 k 2024-09-13T18:10:19.260 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-radosgw x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 7.7 M 2024-09-13T18:10:19.260 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-test x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 40 M 2024-09-13T18:10:19.260 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-volume noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 264 k 2024-09-13T18:10:19.260 INFO:teuthology.orchestra.run.smithi114.stdout: cephadm noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 226 k 2024-09-13T18:10:19.260 INFO:teuthology.orchestra.run.smithi114.stdout: cephfs-mirror x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 224 k 2024-09-13T18:10:19.260 INFO:teuthology.orchestra.run.smithi114.stdout: cephfs-top noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 18 k 2024-09-13T18:10:19.260 INFO:teuthology.orchestra.run.smithi114.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-13T18:10:19.260 INFO:teuthology.orchestra.run.smithi114.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-13T18:10:19.260 INFO:teuthology.orchestra.run.smithi114.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-13T18:10:19.260 INFO:teuthology.orchestra.run.smithi114.stdout: libcephfs-devel x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 31 k 2024-09-13T18:10:19.260 INFO:teuthology.orchestra.run.smithi114.stdout: libcephfs2 x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 712 k 2024-09-13T18:10:19.260 INFO:teuthology.orchestra.run.smithi114.stdout: librados-devel x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 127 k 2024-09-13T18:10:19.261 INFO:teuthology.orchestra.run.smithi114.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-13T18:10:19.261 INFO:teuthology.orchestra.run.smithi114.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-13T18:10:19.261 INFO:teuthology.orchestra.run.smithi114.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-13T18:10:19.261 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-13T18:10:19.261 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-13T18:10:19.261 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-13T18:10:19.261 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cephfs x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 162 k 2024-09-13T18:10:19.261 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rados x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 321 k 2024-09-13T18:10:19.261 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rbd x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 301 k 2024-09-13T18:10:19.261 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rgw x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 99 k 2024-09-13T18:10:19.261 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-fuse x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 87 k 2024-09-13T18:10:19.261 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-mirror x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 3.0 M 2024-09-13T18:10:19.261 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-nbd x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 172 k 2024-09-13T18:10:19.262 INFO:teuthology.orchestra.run.smithi114.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-13T18:10:19.262 INFO:teuthology.orchestra.run.smithi114.stdout:Upgrading: 2024-09-13T18:10:19.262 INFO:teuthology.orchestra.run.smithi114.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-13T18:10:19.262 INFO:teuthology.orchestra.run.smithi114.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-13T18:10:19.262 INFO:teuthology.orchestra.run.smithi114.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-13T18:10:19.262 INFO:teuthology.orchestra.run.smithi114.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-13T18:10:19.262 INFO:teuthology.orchestra.run.smithi114.stdout: librados2 x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 3.3 M 2024-09-13T18:10:19.262 INFO:teuthology.orchestra.run.smithi114.stdout: librbd1 x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 3.0 M 2024-09-13T18:10:19.262 INFO:teuthology.orchestra.run.smithi114.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-13T18:10:19.262 INFO:teuthology.orchestra.run.smithi114.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-13T18:10:19.262 INFO:teuthology.orchestra.run.smithi114.stdout:Installing dependencies: 2024-09-13T18:10:19.262 INFO:teuthology.orchestra.run.smithi114.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-13T18:10:19.262 INFO:teuthology.orchestra.run.smithi114.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-13T18:10:19.263 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-common x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 18 M 2024-09-13T18:10:19.263 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-grafana-dashboards noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 24 k 2024-09-13T18:10:19.263 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mds x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 2.1 M 2024-09-13T18:10:19.263 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-modules-core noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 246 k 2024-09-13T18:10:19.263 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mon x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 4.7 M 2024-09-13T18:10:19.263 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-osd x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 17 M 2024-09-13T18:10:19.263 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-prometheus-alerts noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 16 k 2024-09-13T18:10:19.263 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-selinux x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 25 k 2024-09-13T18:10:19.263 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-13T18:10:19.263 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-13T18:10:19.263 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-13T18:10:19.263 INFO:teuthology.orchestra.run.smithi114.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-09-13T18:10:19.263 INFO:teuthology.orchestra.run.smithi114.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-13T18:10:19.264 INFO:teuthology.orchestra.run.smithi114.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-13T18:10:19.264 INFO:teuthology.orchestra.run.smithi114.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-13T18:10:19.264 INFO:teuthology.orchestra.run.smithi114.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-13T18:10:19.264 INFO:teuthology.orchestra.run.smithi114.stdout: libcephsqlite x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 166 k 2024-09-13T18:10:19.264 INFO:teuthology.orchestra.run.smithi114.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-13T18:10:19.264 INFO:teuthology.orchestra.run.smithi114.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-13T18:10:19.264 INFO:teuthology.orchestra.run.smithi114.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-13T18:10:19.264 INFO:teuthology.orchestra.run.smithi114.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-13T18:10:19.264 INFO:teuthology.orchestra.run.smithi114.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-13T18:10:19.264 INFO:teuthology.orchestra.run.smithi114.stdout: libradosstriper1 x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 478 k 2024-09-13T18:10:19.264 INFO:teuthology.orchestra.run.smithi114.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-13T18:10:19.264 INFO:teuthology.orchestra.run.smithi114.stdout: librgw2 x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 4.5 M 2024-09-13T18:10:19.264 INFO:teuthology.orchestra.run.smithi114.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-13T18:10:19.265 INFO:teuthology.orchestra.run.smithi114.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-13T18:10:19.265 INFO:teuthology.orchestra.run.smithi114.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-13T18:10:19.265 INFO:teuthology.orchestra.run.smithi114.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-13T18:10:19.265 INFO:teuthology.orchestra.run.smithi114.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-13T18:10:19.265 INFO:teuthology.orchestra.run.smithi114.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-13T18:10:19.265 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-13T18:10:19.265 INFO:teuthology.orchestra.run.smithi114.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-13T18:10:19.265 INFO:teuthology.orchestra.run.smithi114.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-13T18:10:19.265 INFO:teuthology.orchestra.run.smithi114.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-13T18:10:19.265 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-13T18:10:19.265 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ceph-argparse x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 45 k 2024-09-13T18:10:19.265 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ceph-common x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 129 k 2024-09-13T18:10:19.265 INFO:teuthology.orchestra.run.smithi114.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-13T18:10:19.265 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-13T18:10:19.266 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-13T18:10:19.266 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-13T18:10:19.266 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-13T18:10:19.266 INFO:teuthology.orchestra.run.smithi114.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-13T18:10:19.266 INFO:teuthology.orchestra.run.smithi114.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-13T18:10:19.266 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-13T18:10:19.266 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-13T18:10:19.266 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-13T18:10:19.266 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-13T18:10:19.266 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-13T18:10:19.266 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-13T18:10:19.266 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-13T18:10:19.266 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-13T18:10:19.267 INFO:teuthology.orchestra.run.smithi114.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-13T18:10:19.267 INFO:teuthology.orchestra.run.smithi114.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-13T18:10:19.267 INFO:teuthology.orchestra.run.smithi114.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-13T18:10:19.267 INFO:teuthology.orchestra.run.smithi114.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-13T18:10:19.267 INFO:teuthology.orchestra.run.smithi114.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-13T18:10:19.267 INFO:teuthology.orchestra.run.smithi114.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-13T18:10:19.267 INFO:teuthology.orchestra.run.smithi114.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-13T18:10:19.267 INFO:teuthology.orchestra.run.smithi114.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-13T18:10:19.267 INFO:teuthology.orchestra.run.smithi114.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-13T18:10:19.267 INFO:teuthology.orchestra.run.smithi114.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-13T18:10:19.267 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-13T18:10:19.267 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-13T18:10:19.267 INFO:teuthology.orchestra.run.smithi114.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-13T18:10:19.267 INFO:teuthology.orchestra.run.smithi114.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-13T18:10:19.268 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-13T18:10:19.268 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-13T18:10:19.268 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-13T18:10:19.268 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-13T18:10:19.268 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-13T18:10:19.268 INFO:teuthology.orchestra.run.smithi114.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-13T18:10:19.268 INFO:teuthology.orchestra.run.smithi114.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-13T18:10:19.268 INFO:teuthology.orchestra.run.smithi114.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-13T18:10:19.268 INFO:teuthology.orchestra.run.smithi114.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-13T18:10:19.268 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-13T18:10:19.268 INFO:teuthology.orchestra.run.smithi114.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-13T18:10:19.268 INFO:teuthology.orchestra.run.smithi114.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-13T18:10:19.269 INFO:teuthology.orchestra.run.smithi114.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-13T18:10:19.269 INFO:teuthology.orchestra.run.smithi114.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-13T18:10:19.269 INFO:teuthology.orchestra.run.smithi114.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-13T18:10:19.269 INFO:teuthology.orchestra.run.smithi114.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-13T18:10:19.269 INFO:teuthology.orchestra.run.smithi114.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-13T18:10:19.269 INFO:teuthology.orchestra.run.smithi114.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-13T18:10:19.269 INFO:teuthology.orchestra.run.smithi114.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-13T18:10:19.269 INFO:teuthology.orchestra.run.smithi114.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-13T18:10:19.269 INFO:teuthology.orchestra.run.smithi114.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-13T18:10:19.269 INFO:teuthology.orchestra.run.smithi114.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-13T18:10:19.269 INFO:teuthology.orchestra.run.smithi114.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-13T18:10:19.269 INFO:teuthology.orchestra.run.smithi114.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-13T18:10:19.269 INFO:teuthology.orchestra.run.smithi114.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-13T18:10:19.269 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:10:19.270 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-09-13T18:10:19.270 INFO:teuthology.orchestra.run.smithi114.stdout:====================================================================================== 2024-09-13T18:10:19.270 INFO:teuthology.orchestra.run.smithi114.stdout:Install 130 Packages 2024-09-13T18:10:19.270 INFO:teuthology.orchestra.run.smithi114.stdout:Upgrade 8 Packages 2024-09-13T18:10:19.270 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:10:19.271 INFO:teuthology.orchestra.run.smithi204.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-13T18:10:19.271 INFO:teuthology.orchestra.run.smithi204.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-13T18:10:19.271 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 203 M 2024-09-13T18:10:19.271 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-09-13T18:10:19.277 INFO:teuthology.orchestra.run.smithi204.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-13T18:10:19.278 INFO:teuthology.orchestra.run.smithi204.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-13T18:10:19.279 INFO:teuthology.orchestra.run.smithi204.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-13T18:10:19.280 INFO:teuthology.orchestra.run.smithi204.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-13T18:10:19.282 INFO:teuthology.orchestra.run.smithi204.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-13T18:10:19.284 INFO:teuthology.orchestra.run.smithi204.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-13T18:10:19.285 INFO:teuthology.orchestra.run.smithi204.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-13T18:10:19.321 INFO:teuthology.orchestra.run.smithi134.stdout:(13/138): ceph-osd-18.2.4-860.g6c6b083f.el9.x86 17 MB/s | 17 MB 00:00 2024-09-13T18:10:19.345 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-09-13T18:10:19.354 INFO:teuthology.orchestra.run.smithi134.stdout:(14/138): libcephfs2-18.2.4-860.g6c6b083f.el9.x 3.1 MB/s | 712 kB 00:00 2024-09-13T18:10:19.364 INFO:teuthology.orchestra.run.smithi204.stdout:====================================================================================== 2024-09-13T18:10:19.365 INFO:teuthology.orchestra.run.smithi204.stdout: Package Arch Version Repository Size 2024-09-13T18:10:19.365 INFO:teuthology.orchestra.run.smithi204.stdout:====================================================================================== 2024-09-13T18:10:19.365 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2024-09-13T18:10:19.365 INFO:teuthology.orchestra.run.smithi204.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-13T18:10:19.365 INFO:teuthology.orchestra.run.smithi204.stdout: ceph x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 6.5 k 2024-09-13T18:10:19.365 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-base x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 5.1 M 2024-09-13T18:10:19.365 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-fuse x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 852 k 2024-09-13T18:10:19.365 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-immutable-object-cache x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 142 k 2024-09-13T18:10:19.365 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 1.5 M 2024-09-13T18:10:19.365 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-cephadm noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 139 k 2024-09-13T18:10:19.365 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-dashboard noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 3.5 M 2024-09-13T18:10:19.365 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 7.4 M 2024-09-13T18:10:19.365 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-rook noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 49 k 2024-09-13T18:10:19.366 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-radosgw x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 7.7 M 2024-09-13T18:10:19.366 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-test x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 40 M 2024-09-13T18:10:19.366 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-volume noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 264 k 2024-09-13T18:10:19.366 INFO:teuthology.orchestra.run.smithi204.stdout: cephadm noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 226 k 2024-09-13T18:10:19.366 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-mirror x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 224 k 2024-09-13T18:10:19.366 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-top noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 18 k 2024-09-13T18:10:19.366 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-13T18:10:19.366 INFO:teuthology.orchestra.run.smithi204.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-13T18:10:19.366 INFO:teuthology.orchestra.run.smithi204.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-13T18:10:19.366 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs-devel x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 31 k 2024-09-13T18:10:19.366 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs2 x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 712 k 2024-09-13T18:10:19.366 INFO:teuthology.orchestra.run.smithi204.stdout: librados-devel x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 127 k 2024-09-13T18:10:19.366 INFO:teuthology.orchestra.run.smithi204.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-13T18:10:19.367 INFO:teuthology.orchestra.run.smithi204.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-13T18:10:19.367 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-13T18:10:19.367 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-13T18:10:19.367 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-13T18:10:19.367 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-13T18:10:19.367 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cephfs x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 162 k 2024-09-13T18:10:19.367 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rados x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 321 k 2024-09-13T18:10:19.367 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rbd x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 301 k 2024-09-13T18:10:19.367 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rgw x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 99 k 2024-09-13T18:10:19.367 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-fuse x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 87 k 2024-09-13T18:10:19.367 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-mirror x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 3.0 M 2024-09-13T18:10:19.367 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-nbd x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 172 k 2024-09-13T18:10:19.367 INFO:teuthology.orchestra.run.smithi204.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-13T18:10:19.367 INFO:teuthology.orchestra.run.smithi204.stdout:Upgrading: 2024-09-13T18:10:19.368 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-13T18:10:19.368 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-13T18:10:19.368 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-13T18:10:19.368 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-13T18:10:19.368 INFO:teuthology.orchestra.run.smithi204.stdout: librados2 x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 3.3 M 2024-09-13T18:10:19.368 INFO:teuthology.orchestra.run.smithi204.stdout: librbd1 x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 3.0 M 2024-09-13T18:10:19.368 INFO:teuthology.orchestra.run.smithi204.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-13T18:10:19.368 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-13T18:10:19.368 INFO:teuthology.orchestra.run.smithi204.stdout:Installing dependencies: 2024-09-13T18:10:19.368 INFO:teuthology.orchestra.run.smithi204.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-13T18:10:19.368 INFO:teuthology.orchestra.run.smithi204.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-13T18:10:19.368 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-common x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 18 M 2024-09-13T18:10:19.368 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-grafana-dashboards noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 24 k 2024-09-13T18:10:19.369 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mds x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 2.1 M 2024-09-13T18:10:19.369 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-modules-core noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 246 k 2024-09-13T18:10:19.369 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mon x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 4.7 M 2024-09-13T18:10:19.369 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-osd x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 17 M 2024-09-13T18:10:19.369 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-prometheus-alerts noarch 2:18.2.4-860.g6c6b083f.el9 ceph-noarch 16 k 2024-09-13T18:10:19.369 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-selinux x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 25 k 2024-09-13T18:10:19.369 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-13T18:10:19.369 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-13T18:10:19.369 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-13T18:10:19.369 INFO:teuthology.orchestra.run.smithi204.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-09-13T18:10:19.369 INFO:teuthology.orchestra.run.smithi204.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-13T18:10:19.369 INFO:teuthology.orchestra.run.smithi204.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-13T18:10:19.369 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-13T18:10:19.369 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-13T18:10:19.370 INFO:teuthology.orchestra.run.smithi204.stdout: libcephsqlite x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 166 k 2024-09-13T18:10:19.370 INFO:teuthology.orchestra.run.smithi204.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-13T18:10:19.370 INFO:teuthology.orchestra.run.smithi204.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-13T18:10:19.370 INFO:teuthology.orchestra.run.smithi204.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-13T18:10:19.370 INFO:teuthology.orchestra.run.smithi204.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-13T18:10:19.370 INFO:teuthology.orchestra.run.smithi204.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-13T18:10:19.370 INFO:teuthology.orchestra.run.smithi204.stdout: libradosstriper1 x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 478 k 2024-09-13T18:10:19.370 INFO:teuthology.orchestra.run.smithi204.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-13T18:10:19.370 INFO:teuthology.orchestra.run.smithi204.stdout: librgw2 x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 4.5 M 2024-09-13T18:10:19.370 INFO:teuthology.orchestra.run.smithi204.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-13T18:10:19.370 INFO:teuthology.orchestra.run.smithi204.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-13T18:10:19.370 INFO:teuthology.orchestra.run.smithi204.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-13T18:10:19.370 INFO:teuthology.orchestra.run.smithi204.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-13T18:10:19.370 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-13T18:10:19.371 INFO:teuthology.orchestra.run.smithi204.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-13T18:10:19.371 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-13T18:10:19.371 INFO:teuthology.orchestra.run.smithi204.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-13T18:10:19.371 INFO:teuthology.orchestra.run.smithi204.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-13T18:10:19.371 INFO:teuthology.orchestra.run.smithi204.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-13T18:10:19.371 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-13T18:10:19.371 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-argparse x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 45 k 2024-09-13T18:10:19.371 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-common x86_64 2:18.2.4-860.g6c6b083f.el9 ceph 129 k 2024-09-13T18:10:19.371 INFO:teuthology.orchestra.run.smithi204.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-13T18:10:19.371 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-13T18:10:19.372 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-13T18:10:19.372 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-13T18:10:19.372 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-13T18:10:19.372 INFO:teuthology.orchestra.run.smithi204.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-13T18:10:19.372 INFO:teuthology.orchestra.run.smithi204.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-13T18:10:19.372 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-13T18:10:19.372 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-13T18:10:19.373 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-13T18:10:19.373 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-13T18:10:19.373 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-13T18:10:19.373 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-13T18:10:19.373 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-13T18:10:19.373 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-13T18:10:19.373 INFO:teuthology.orchestra.run.smithi204.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-13T18:10:19.373 INFO:teuthology.orchestra.run.smithi204.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-13T18:10:19.373 INFO:teuthology.orchestra.run.smithi204.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-13T18:10:19.373 INFO:teuthology.orchestra.run.smithi204.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-13T18:10:19.373 INFO:teuthology.orchestra.run.smithi204.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-13T18:10:19.373 INFO:teuthology.orchestra.run.smithi204.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-13T18:10:19.373 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-13T18:10:19.373 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-13T18:10:19.374 INFO:teuthology.orchestra.run.smithi204.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-13T18:10:19.374 INFO:teuthology.orchestra.run.smithi204.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-13T18:10:19.374 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-13T18:10:19.374 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-13T18:10:19.374 INFO:teuthology.orchestra.run.smithi204.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-13T18:10:19.374 INFO:teuthology.orchestra.run.smithi204.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-13T18:10:19.374 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-13T18:10:19.374 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-13T18:10:19.374 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-13T18:10:19.374 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-13T18:10:19.374 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-13T18:10:19.374 INFO:teuthology.orchestra.run.smithi204.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-13T18:10:19.374 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-13T18:10:19.374 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-13T18:10:19.375 INFO:teuthology.orchestra.run.smithi204.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-13T18:10:19.375 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-13T18:10:19.375 INFO:teuthology.orchestra.run.smithi204.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-13T18:10:19.375 INFO:teuthology.orchestra.run.smithi204.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-13T18:10:19.375 INFO:teuthology.orchestra.run.smithi204.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-13T18:10:19.375 INFO:teuthology.orchestra.run.smithi204.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-13T18:10:19.375 INFO:teuthology.orchestra.run.smithi204.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-13T18:10:19.375 INFO:teuthology.orchestra.run.smithi204.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-13T18:10:19.375 INFO:teuthology.orchestra.run.smithi204.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-13T18:10:19.375 INFO:teuthology.orchestra.run.smithi204.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-13T18:10:19.375 INFO:teuthology.orchestra.run.smithi204.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-13T18:10:19.375 INFO:teuthology.orchestra.run.smithi204.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-13T18:10:19.375 INFO:teuthology.orchestra.run.smithi204.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-13T18:10:19.375 INFO:teuthology.orchestra.run.smithi204.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-13T18:10:19.376 INFO:teuthology.orchestra.run.smithi204.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-13T18:10:19.376 INFO:teuthology.orchestra.run.smithi204.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-13T18:10:19.376 INFO:teuthology.orchestra.run.smithi204.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-13T18:10:19.376 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-13T18:10:19.376 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-09-13T18:10:19.376 INFO:teuthology.orchestra.run.smithi204.stdout:====================================================================================== 2024-09-13T18:10:19.376 INFO:teuthology.orchestra.run.smithi204.stdout:Install 130 Packages 2024-09-13T18:10:19.376 INFO:teuthology.orchestra.run.smithi204.stdout:Upgrade 8 Packages 2024-09-13T18:10:19.376 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-13T18:10:19.377 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 203 M 2024-09-13T18:10:19.377 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-09-13T18:10:19.379 INFO:teuthology.orchestra.run.smithi134.stdout:(15/138): libcephsqlite-18.2.4-860.g6c6b083f.el 2.8 MB/s | 166 kB 00:00 2024-09-13T18:10:19.405 INFO:teuthology.orchestra.run.smithi134.stdout:(16/138): librados-devel-18.2.4-860.g6c6b083f.e 2.5 MB/s | 127 kB 00:00 2024-09-13T18:10:19.430 INFO:teuthology.orchestra.run.smithi134.stdout:(17/138): libradosstriper1-18.2.4-860.g6c6b083f 9.4 MB/s | 478 kB 00:00 2024-09-13T18:10:19.505 INFO:teuthology.orchestra.run.smithi134.stdout:(18/138): python3-ceph-argparse-18.2.4-860.g6c6 599 kB/s | 45 kB 00:00 2024-09-13T18:10:19.597 INFO:teuthology.orchestra.run.smithi134.stdout:(19/138): librgw2-18.2.4-860.g6c6b083f.el9.x86_ 23 MB/s | 4.5 MB 00:00 2024-09-13T18:10:19.623 INFO:teuthology.orchestra.run.smithi134.stdout:(20/138): python3-ceph-common-18.2.4-860.g6c6b0 1.1 MB/s | 129 kB 00:00 2024-09-13T18:10:19.656 INFO:teuthology.orchestra.run.smithi134.stdout:(21/138): python3-cephfs-18.2.4-860.g6c6b083f.e 2.7 MB/s | 162 kB 00:00 2024-09-13T18:10:19.681 INFO:teuthology.orchestra.run.smithi134.stdout:(22/138): python3-rados-18.2.4-860.g6c6b083f.el 5.4 MB/s | 321 kB 00:00 2024-09-13T18:10:19.703 INFO:teuthology.orchestra.run.smithi204.stdout:(1/138): ceph-18.2.4-860.g6c6b083f.el9.x86_64.r 54 kB/s | 6.5 kB 00:00 2024-09-13T18:10:19.707 INFO:teuthology.orchestra.run.smithi134.stdout:(23/138): python3-rbd-18.2.4-860.g6c6b083f.el9. 5.8 MB/s | 301 kB 00:00 2024-09-13T18:10:19.722 INFO:teuthology.orchestra.run.smithi114.stdout:(1/138): ceph-18.2.4-860.g6c6b083f.el9.x86_64.r 46 kB/s | 6.5 kB 00:00 2024-09-13T18:10:19.732 INFO:teuthology.orchestra.run.smithi134.stdout:(24/138): python3-rgw-18.2.4-860.g6c6b083f.el9. 1.9 MB/s | 99 kB 00:00 2024-09-13T18:10:19.757 INFO:teuthology.orchestra.run.smithi134.stdout:(25/138): rbd-fuse-18.2.4-860.g6c6b083f.el9.x86 1.7 MB/s | 87 kB 00:00 2024-09-13T18:10:19.824 INFO:teuthology.orchestra.run.smithi134.stdout:(26/138): rbd-nbd-18.2.4-860.g6c6b083f.el9.x86_ 2.5 MB/s | 172 kB 00:00 2024-09-13T18:10:19.874 INFO:teuthology.orchestra.run.smithi134.stdout:(27/138): rbd-mirror-18.2.4-860.g6c6b083f.el9.x 21 MB/s | 3.0 MB 00:00 2024-09-13T18:10:19.899 INFO:teuthology.orchestra.run.smithi134.stdout:(28/138): ceph-grafana-dashboards-18.2.4-860.g6 325 kB/s | 24 kB 00:00 2024-09-13T18:10:19.942 INFO:teuthology.orchestra.run.smithi134.stdout:(29/138): ceph-mgr-cephadm-18.2.4-860.g6c6b083f 2.0 MB/s | 139 kB 00:00 2024-09-13T18:10:20.142 INFO:teuthology.orchestra.run.smithi134.stdout:(30/138): ceph-mgr-dashboard-18.2.4-860.g6c6b08 15 MB/s | 3.5 MB 00:00 2024-09-13T18:10:20.178 INFO:teuthology.orchestra.run.smithi204.stdout:(2/138): ceph-fuse-18.2.4-860.g6c6b083f.el9.x86 1.8 MB/s | 852 kB 00:00 2024-09-13T18:10:20.225 INFO:teuthology.orchestra.run.smithi134.stdout:(31/138): ceph-mgr-modules-core-18.2.4-860.g6c6 2.9 MB/s | 246 kB 00:00 2024-09-13T18:10:20.309 INFO:teuthology.orchestra.run.smithi134.stdout:(32/138): ceph-mgr-rook-18.2.4-860.g6c6b083f.el 591 kB/s | 49 kB 00:00 2024-09-13T18:10:20.320 INFO:teuthology.orchestra.run.smithi204.stdout:(3/138): ceph-immutable-object-cache-18.2.4-860 1.0 MB/s | 142 kB 00:00 2024-09-13T18:10:20.348 INFO:teuthology.orchestra.run.smithi114.stdout:(2/138): ceph-fuse-18.2.4-860.g6c6b083f.el9.x86 1.3 MB/s | 852 kB 00:00 2024-09-13T18:10:20.384 INFO:teuthology.orchestra.run.smithi134.stdout:(33/138): ceph-prometheus-alerts-18.2.4-860.g6c 219 kB/s | 16 kB 00:00 2024-09-13T18:10:20.485 INFO:teuthology.orchestra.run.smithi134.stdout:(34/138): ceph-volume-18.2.4-860.g6c6b083f.el9. 2.6 MB/s | 264 kB 00:00 2024-09-13T18:10:20.506 INFO:teuthology.orchestra.run.smithi114.stdout:(3/138): ceph-immutable-object-cache-18.2.4-860 901 kB/s | 142 kB 00:00 2024-09-13T18:10:20.712 INFO:teuthology.orchestra.run.smithi204.stdout:(4/138): ceph-mds-18.2.4-860.g6c6b083f.el9.x86_ 5.4 MB/s | 2.1 MB 00:00 2024-09-13T18:10:20.782 INFO:teuthology.orchestra.run.smithi114.stdout:(4/138): ceph-base-18.2.4-860.g6c6b083f.el9.x86 4.2 MB/s | 5.1 MB 00:01 2024-09-13T18:10:20.852 INFO:teuthology.orchestra.run.smithi134.stdout:(35/138): ceph-test-18.2.4-860.g6c6b083f.el9.x8 19 MB/s | 40 MB 00:02 2024-09-13T18:10:20.877 INFO:teuthology.orchestra.run.smithi134.stdout:(36/138): cephadm-18.2.4-860.g6c6b083f.el9.noar 577 kB/s | 226 kB 00:00 2024-09-13T18:10:20.896 INFO:teuthology.orchestra.run.smithi204.stdout:(5/138): ceph-mgr-18.2.4-860.g6c6b083f.el9.x86_ 8.0 MB/s | 1.5 MB 00:00 2024-09-13T18:10:20.932 INFO:teuthology.orchestra.run.smithi114.stdout:(5/138): ceph-mds-18.2.4-860.g6c6b083f.el9.x86_ 4.9 MB/s | 2.1 MB 00:00 2024-09-13T18:10:20.960 INFO:teuthology.orchestra.run.smithi134.stdout:(37/138): ceph-mgr-diskprediction-local-18.2.4- 7.3 MB/s | 7.4 MB 00:01 2024-09-13T18:10:20.973 INFO:teuthology.orchestra.run.smithi114.stdout:(6/138): ceph-mgr-18.2.4-860.g6c6b083f.el9.x86_ 7.6 MB/s | 1.5 MB 00:00 2024-09-13T18:10:20.986 INFO:teuthology.orchestra.run.smithi134.stdout:(38/138): cephfs-top-18.2.4-860.g6c6b083f.el9.n 137 kB/s | 18 kB 00:00 2024-09-13T18:10:21.094 INFO:teuthology.orchestra.run.smithi134.stdout:(39/138): fuse-2.9.9-16.el9.x86_64.rpm 368 kB/s | 80 kB 00:00 2024-09-13T18:10:21.144 INFO:teuthology.orchestra.run.smithi134.stdout:(40/138): libquadmath-11.5.0-2.el9.x86_64.rpm 1.2 MB/s | 191 kB 00:00 2024-09-13T18:10:21.186 INFO:teuthology.orchestra.run.smithi134.stdout:(41/138): libgfortran-11.5.0-2.el9.x86_64.rpm 3.5 MB/s | 801 kB 00:00 2024-09-13T18:10:21.211 INFO:teuthology.orchestra.run.smithi134.stdout:(42/138): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.1 MB/s | 253 kB 00:00 2024-09-13T18:10:21.237 INFO:teuthology.orchestra.run.smithi134.stdout:(43/138): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-13T18:10:21.270 INFO:teuthology.orchestra.run.smithi134.stdout:(44/138): python3-cryptography-36.0.1-4.el9.x86 9.7 MB/s | 1.2 MB 00:00 2024-09-13T18:10:21.295 INFO:teuthology.orchestra.run.smithi134.stdout:(45/138): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-09-13T18:10:21.321 INFO:teuthology.orchestra.run.smithi134.stdout:(46/138): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-09-13T18:10:21.346 INFO:teuthology.orchestra.run.smithi134.stdout:(47/138): python3-urllib3-1.26.5-6.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-09-13T18:10:21.396 INFO:teuthology.orchestra.run.smithi134.stdout:(48/138): bison-3.7.4-5.el9.x86_64.rpm 12 MB/s | 943 kB 00:00 2024-09-13T18:10:21.422 INFO:teuthology.orchestra.run.smithi134.stdout:(49/138): automake-1.16.2-8.el9.noarch.rpm 5.4 MB/s | 689 kB 00:00 2024-09-13T18:10:21.429 INFO:teuthology.orchestra.run.smithi204.stdout:(6/138): ceph-mon-18.2.4-860.g6c6b083f.el9.x86_ 8.8 MB/s | 4.7 MB 00:00 2024-09-13T18:10:21.447 INFO:teuthology.orchestra.run.smithi134.stdout:(50/138): elfutils-libelf-devel-0.191-4.el9.x86 434 kB/s | 22 kB 00:00 2024-09-13T18:10:21.464 INFO:teuthology.orchestra.run.smithi134.stdout:(51/138): boost-program-options-1.75.0-8.el9.x8 910 kB/s | 107 kB 00:00 2024-09-13T18:10:21.489 INFO:teuthology.orchestra.run.smithi134.stdout:(52/138): flex-2.6.4-9.el9.x86_64.rpm 4.6 MB/s | 313 kB 00:00 2024-09-13T18:10:21.496 INFO:teuthology.orchestra.run.smithi204.stdout:(7/138): ceph-base-18.2.4-860.g6c6b083f.el9.x86 2.7 MB/s | 5.1 MB 00:01 2024-09-13T18:10:21.514 INFO:teuthology.orchestra.run.smithi134.stdout:(53/138): flexiblas-3.0.4-8.el9.x86_64.rpm 485 kB/s | 33 kB 00:00 2024-09-13T18:10:21.540 INFO:teuthology.orchestra.run.smithi134.stdout:(54/138): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-09-13T18:10:21.565 INFO:teuthology.orchestra.run.smithi134.stdout:(55/138): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-13T18:10:21.590 INFO:teuthology.orchestra.run.smithi134.stdout:(56/138): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-13T18:10:21.615 INFO:teuthology.orchestra.run.smithi134.stdout:(57/138): librabbitmq-0.11.0-7.el9.x86_64.rpm 902 kB/s | 45 kB 00:00 2024-09-13T18:10:21.658 INFO:teuthology.orchestra.run.smithi134.stdout:(58/138): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-09-13T18:10:21.699 INFO:teuthology.orchestra.run.smithi134.stdout:(59/138): flexiblas-netlib-3.0.4-8.el9.x86_64.r 13 MB/s | 3.0 MB 00:00 2024-09-13T18:10:21.725 INFO:teuthology.orchestra.run.smithi134.stdout:(60/138): libuuid-devel-2.37.4-20.el9.x86_64.rp 366 kB/s | 24 kB 00:00 2024-09-13T18:10:21.741 INFO:teuthology.orchestra.run.smithi114.stdout:(7/138): ceph-common-18.2.4-860.g6c6b083f.el9.x 8.5 MB/s | 18 MB 00:02 2024-09-13T18:10:21.750 INFO:teuthology.orchestra.run.smithi134.stdout:(61/138): libzstd-devel-1.5.1-2.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-09-13T18:10:21.775 INFO:teuthology.orchestra.run.smithi134.stdout:(62/138): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-09-13T18:10:21.799 INFO:teuthology.orchestra.run.smithi114.stdout:(8/138): ceph-mon-18.2.4-860.g6c6b083f.el9.x86_ 5.4 MB/s | 4.7 MB 00:00 2024-09-13T18:10:21.800 INFO:teuthology.orchestra.run.smithi134.stdout:(63/138): openblas-0.3.26-2.el9.x86_64.rpm 770 kB/s | 38 kB 00:00 2024-09-13T18:10:21.813 INFO:teuthology.orchestra.run.smithi204.stdout:(8/138): ceph-common-18.2.4-860.g6c6b083f.el9.x 8.2 MB/s | 18 MB 00:02 2024-09-13T18:10:21.834 INFO:teuthology.orchestra.run.smithi134.stdout:(64/138): libtool-2.4.6-46.el9.x86_64.rpm 2.6 MB/s | 581 kB 00:00 2024-09-13T18:10:21.866 INFO:teuthology.orchestra.run.smithi114.stdout:(9/138): ceph-selinux-18.2.4-860.g6c6b083f.el9. 378 kB/s | 25 kB 00:00 2024-09-13T18:10:21.871 INFO:teuthology.orchestra.run.smithi204.stdout:(9/138): ceph-selinux-18.2.4-860.g6c6b083f.el9. 433 kB/s | 25 kB 00:00 2024-09-13T18:10:21.901 INFO:teuthology.orchestra.run.smithi134.stdout:(65/138): postgresql-13.16-1.el9.x86_64.rpm 24 MB/s | 1.6 MB 00:00 2024-09-13T18:10:21.976 INFO:teuthology.orchestra.run.smithi134.stdout:(66/138): openblas-openmp-0.3.26-2.el9.x86_64.r 25 MB/s | 4.9 MB 00:00 2024-09-13T18:10:22.035 INFO:teuthology.orchestra.run.smithi134.stdout:(67/138): openssl-devel-3.2.2-6.el9.x86_64.rpm 19 MB/s | 4.4 MB 00:00 2024-09-13T18:10:22.060 INFO:teuthology.orchestra.run.smithi134.stdout:(68/138): postgresql-private-libs-13.16-1.el9.x 1.6 MB/s | 137 kB 00:00 2024-09-13T18:10:22.093 INFO:teuthology.orchestra.run.smithi134.stdout:(69/138): postgresql-contrib-13.16-1.el9.x86_64 4.6 MB/s | 910 kB 00:00 2024-09-13T18:10:22.144 INFO:teuthology.orchestra.run.smithi134.stdout:(70/138): python3-devel-3.9.19-8.el9.x86_64.rpm 4.8 MB/s | 245 kB 00:00 2024-09-13T18:10:22.219 INFO:teuthology.orchestra.run.smithi134.stdout:(71/138): postgresql-server-13.16-1.el9.x86_64. 31 MB/s | 5.8 MB 00:00 2024-09-13T18:10:22.244 INFO:teuthology.orchestra.run.smithi134.stdout:(72/138): python3-jinja2-2.11.3-6.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-09-13T18:10:22.261 INFO:teuthology.orchestra.run.smithi134.stdout:(73/138): python3-jsonpatch-1.21-16.el9.noarch. 616 kB/s | 26 kB 00:00 2024-09-13T18:10:22.283 INFO:teuthology.orchestra.run.smithi114.stdout:(10/138): ceph-radosgw-18.2.4-860.g6c6b083f.el9 14 MB/s | 7.7 MB 00:00 2024-09-13T18:10:22.337 INFO:teuthology.orchestra.run.smithi134.stdout:(74/138): python3-babel-2.9.1-2.el9.noarch.rpm 22 MB/s | 6.0 MB 00:00 2024-09-13T18:10:22.353 INFO:teuthology.orchestra.run.smithi134.stdout:(75/138): python3-jsonpointer-2.0-4.el9.noarch. 176 kB/s | 19 kB 00:00 2024-09-13T18:10:22.378 INFO:teuthology.orchestra.run.smithi134.stdout:(76/138): python3-mako-1.1.4-6.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-09-13T18:10:22.395 INFO:teuthology.orchestra.run.smithi134.stdout:(77/138): python3-markupsafe-1.1.1-12.el9.x86_6 592 kB/s | 35 kB 00:00 2024-09-13T18:10:22.432 INFO:teuthology.orchestra.run.smithi134.stdout:(78/138): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-09-13T18:10:22.441 INFO:teuthology.orchestra.run.smithi114.stdout:(11/138): ceph-osd-18.2.4-860.g6c6b083f.el9.x86 11 MB/s | 17 MB 00:01 2024-09-13T18:10:22.454 INFO:teuthology.orchestra.run.smithi134.stdout:(79/138): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-09-13T18:10:22.467 INFO:teuthology.orchestra.run.smithi114.stdout:(12/138): cephfs-mirror-18.2.4-860.g6c6b083f.el 1.2 MB/s | 224 kB 00:00 2024-09-13T18:10:22.479 INFO:teuthology.orchestra.run.smithi134.stdout:(80/138): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-13T18:10:22.492 INFO:teuthology.orchestra.run.smithi114.stdout:(13/138): libcephfs-devel-18.2.4-860.g6c6b083f. 620 kB/s | 31 kB 00:00 2024-09-13T18:10:22.496 INFO:teuthology.orchestra.run.smithi134.stdout:(81/138): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-09-13T18:10:22.530 INFO:teuthology.orchestra.run.smithi134.stdout:(82/138): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-13T18:10:22.542 INFO:teuthology.orchestra.run.smithi114.stdout:(14/138): libcephfs2-18.2.4-860.g6c6b083f.el9.x 9.2 MB/s | 712 kB 00:00 2024-09-13T18:10:22.555 INFO:teuthology.orchestra.run.smithi134.stdout:(83/138): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-09-13T18:10:22.567 INFO:teuthology.orchestra.run.smithi114.stdout:(15/138): libcephsqlite-18.2.4-860.g6c6b083f.el 2.2 MB/s | 166 kB 00:00 2024-09-13T18:10:22.572 INFO:teuthology.orchestra.run.smithi134.stdout:(84/138): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-09-13T18:10:22.598 INFO:teuthology.orchestra.run.smithi134.stdout:(85/138): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 54 kB 00:00 2024-09-13T18:10:22.601 INFO:teuthology.orchestra.run.smithi114.stdout:(16/138): librados-devel-18.2.4-860.g6c6b083f.e 2.1 MB/s | 127 kB 00:00 2024-09-13T18:10:22.665 INFO:teuthology.orchestra.run.smithi134.stdout:(86/138): python3-numpy-1.20.1-5.el9.x86_64.rpm 16 MB/s | 5.1 MB 00:00 2024-09-13T18:10:22.685 INFO:teuthology.orchestra.run.smithi114.stdout:(17/138): libradosstriper1-18.2.4-860.g6c6b083f 4.0 MB/s | 478 kB 00:00 2024-09-13T18:10:22.690 INFO:teuthology.orchestra.run.smithi134.stdout:(87/138): python3-toml-0.10.2-6.el9.noarch.rpm 453 kB/s | 42 kB 00:00 2024-09-13T18:10:22.715 INFO:teuthology.orchestra.run.smithi134.stdout:(88/138): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-13T18:10:22.732 INFO:teuthology.orchestra.run.smithi134.stdout:(89/138): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2024-09-13T18:10:22.743 INFO:teuthology.orchestra.run.smithi114.stdout:(18/138): python3-ceph-argparse-18.2.4-860.g6c6 771 kB/s | 45 kB 00:00 2024-09-13T18:10:22.757 INFO:teuthology.orchestra.run.smithi134.stdout:(90/138): xfsprogs-devel-6.4.0-4.el9.x86_64.rpm 2.2 MB/s | 96 kB 00:00 2024-09-13T18:10:22.782 INFO:teuthology.orchestra.run.smithi134.stdout:(91/138): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-13T18:10:22.802 INFO:teuthology.orchestra.run.smithi114.stdout:(19/138): python3-ceph-common-18.2.4-860.g6c6b0 2.2 MB/s | 129 kB 00:00 2024-09-13T18:10:22.808 INFO:teuthology.orchestra.run.smithi134.stdout:(92/138): zlib-devel-1.2.11-41.el9.x86_64.rpm 894 kB/s | 45 kB 00:00 2024-09-13T18:10:22.847 INFO:teuthology.orchestra.run.smithi204.stdout:(10/138): ceph-osd-18.2.4-860.g6c6b083f.el9.x86 12 MB/s | 17 MB 00:01 2024-09-13T18:10:22.858 INFO:teuthology.orchestra.run.smithi134.stdout:(93/138): fmt-8.1.1-5.el9.x86_64.rpm 1.4 MB/s | 111 kB 00:00 2024-09-13T18:10:22.883 INFO:teuthology.orchestra.run.smithi134.stdout:(94/138): gperftools-libs-2.9.1-3.el9.x86_64.rp 4.0 MB/s | 308 kB 00:00 2024-09-13T18:10:22.894 INFO:teuthology.orchestra.run.smithi114.stdout:(20/138): librgw2-18.2.4-860.g6c6b083f.el9.x86_ 15 MB/s | 4.5 MB 00:00 2024-09-13T18:10:22.897 INFO:teuthology.orchestra.run.smithi204.stdout:(11/138): cephfs-mirror-18.2.4-860.g6c6b083f.el 4.4 MB/s | 224 kB 00:00 2024-09-13T18:10:22.908 INFO:teuthology.orchestra.run.smithi134.stdout:(95/138): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-13T18:10:22.919 INFO:teuthology.orchestra.run.smithi114.stdout:(21/138): python3-cephfs-18.2.4-860.g6c6b083f.e 1.4 MB/s | 162 kB 00:00 2024-09-13T18:10:22.934 INFO:teuthology.orchestra.run.smithi134.stdout:(96/138): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-13T18:10:22.944 INFO:teuthology.orchestra.run.smithi114.stdout:(22/138): python3-rados-18.2.4-860.g6c6b083f.el 6.2 MB/s | 321 kB 00:00 2024-09-13T18:10:22.956 INFO:teuthology.orchestra.run.smithi204.stdout:(12/138): libcephfs-devel-18.2.4-860.g6c6b083f. 534 kB/s | 31 kB 00:00 2024-09-13T18:10:22.959 INFO:teuthology.orchestra.run.smithi134.stdout:(97/138): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-13T18:10:22.978 INFO:teuthology.orchestra.run.smithi114.stdout:(23/138): python3-rbd-18.2.4-860.g6c6b083f.el9. 5.0 MB/s | 301 kB 00:00 2024-09-13T18:10:23.003 INFO:teuthology.orchestra.run.smithi114.stdout:(24/138): python3-rgw-18.2.4-860.g6c6b083f.el9. 1.7 MB/s | 99 kB 00:00 2024-09-13T18:10:23.029 INFO:teuthology.orchestra.run.smithi114.stdout:(25/138): rbd-fuse-18.2.4-860.g6c6b083f.el9.x86 1.7 MB/s | 87 kB 00:00 2024-09-13T18:10:23.043 INFO:teuthology.orchestra.run.smithi134.stdout:(98/138): parquet-libs-9.0.0-11.el9.x86_64.rpm 9.8 MB/s | 839 kB 00:00 2024-09-13T18:10:23.056 INFO:teuthology.orchestra.run.smithi204.stdout:(13/138): libcephfs2-18.2.4-860.g6c6b083f.el9.x 7.0 MB/s | 712 kB 00:00 2024-09-13T18:10:23.088 INFO:teuthology.orchestra.run.smithi114.stdout:(26/138): rbd-nbd-18.2.4-860.g6c6b083f.el9.x86_ 2.9 MB/s | 172 kB 00:00 2024-09-13T18:10:23.093 INFO:teuthology.orchestra.run.smithi134.stdout:(99/138): python3-asyncssh-2.13.2-5.el9.noarch. 11 MB/s | 548 kB 00:00 2024-09-13T18:10:23.118 INFO:teuthology.orchestra.run.smithi134.stdout:(100/138): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-13T18:10:23.140 INFO:teuthology.orchestra.run.smithi204.stdout:(14/138): libcephsqlite-18.2.4-860.g6c6b083f.el 1.9 MB/s | 166 kB 00:00 2024-09-13T18:10:23.143 INFO:teuthology.orchestra.run.smithi134.stdout:(101/138): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-13T18:10:23.154 INFO:teuthology.orchestra.run.smithi114.stdout:(27/138): ceph-grafana-dashboards-18.2.4-860.g6 367 kB/s | 24 kB 00:00 2024-09-13T18:10:23.169 INFO:teuthology.orchestra.run.smithi134.stdout:(102/138): python3-certifi-2023.05.07-4.el9.noa 562 kB/s | 14 kB 00:00 2024-09-13T18:10:23.190 INFO:teuthology.orchestra.run.smithi204.stdout:(15/138): librados-devel-18.2.4-860.g6c6b083f.e 2.5 MB/s | 127 kB 00:00 2024-09-13T18:10:23.213 INFO:teuthology.orchestra.run.smithi114.stdout:(28/138): rbd-mirror-18.2.4-860.g6c6b083f.el9.x 14 MB/s | 3.0 MB 00:00 2024-09-13T18:10:23.227 INFO:teuthology.orchestra.run.smithi134.stdout:(103/138): libarrow-9.0.0-11.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-09-13T18:10:23.238 INFO:teuthology.orchestra.run.smithi114.stdout:(29/138): ceph-mgr-cephadm-18.2.4-860.g6c6b083f 1.6 MB/s | 139 kB 00:00 2024-09-13T18:10:23.248 INFO:teuthology.orchestra.run.smithi204.stdout:(16/138): libradosstriper1-18.2.4-860.g6c6b083f 8.0 MB/s | 478 kB 00:00 2024-09-13T18:10:23.252 INFO:teuthology.orchestra.run.smithi134.stdout:(104/138): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-09-13T18:10:23.420 INFO:teuthology.orchestra.run.smithi134.stdout:(105/138): python3-scipy-1.6.2-8.el9.x86_64.rpm 19 MB/s | 16 MB 00:00 2024-09-13T18:10:23.445 INFO:teuthology.orchestra.run.smithi134.stdout:(106/138): python3-cherrypy-18.6.1-2.el9.noarch 1.6 MB/s | 358 kB 00:00 2024-09-13T18:10:23.478 INFO:teuthology.orchestra.run.smithi134.stdout:(107/138): python3-jaraco-classes-3.2.1-5.el9.n 685 kB/s | 18 kB 00:00 2024-09-13T18:10:23.504 INFO:teuthology.orchestra.run.smithi134.stdout:(108/138): python3-google-auth-2.34.0-1.el9.noa 877 kB/s | 220 kB 00:00 2024-09-13T18:10:23.522 INFO:teuthology.orchestra.run.smithi114.stdout:(30/138): ceph-mgr-dashboard-18.2.4-860.g6c6b08 11 MB/s | 3.5 MB 00:00 2024-09-13T18:10:23.529 INFO:teuthology.orchestra.run.smithi134.stdout:(109/138): python3-jaraco-8.2.1-3.el9.noarch.rp 98 kB/s | 11 kB 00:00 2024-09-13T18:10:23.554 INFO:teuthology.orchestra.run.smithi134.stdout:(110/138): python3-jaraco-collections-3.0.0-8.e 314 kB/s | 23 kB 00:00 2024-09-13T18:10:23.579 INFO:teuthology.orchestra.run.smithi134.stdout:(111/138): python3-jaraco-functools-3.5.0-2.el9 258 kB/s | 19 kB 00:00 2024-09-13T18:10:23.596 INFO:teuthology.orchestra.run.smithi134.stdout:(112/138): python3-jaraco-text-3.2.0-6.el9.noar 293 kB/s | 20 kB 00:00 2024-09-13T18:10:23.613 INFO:teuthology.orchestra.run.smithi134.stdout:(113/138): python3-logutils-0.3.5-21.el9.noarch 1.4 MB/s | 46 kB 00:00 2024-09-13T18:10:23.622 INFO:teuthology.orchestra.run.smithi114.stdout:(31/138): ceph-mgr-modules-core-18.2.4-860.g6c6 2.4 MB/s | 246 kB 00:00 2024-09-13T18:10:23.630 INFO:teuthology.orchestra.run.smithi134.stdout:(114/138): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-13T18:10:23.647 INFO:teuthology.orchestra.run.smithi134.stdout:(115/138): python3-natsort-7.1.1-5.el9.noarch.r 1.7 MB/s | 58 kB 00:00 2024-09-13T18:10:23.664 INFO:teuthology.orchestra.run.smithi134.stdout:(116/138): python3-portend-3.1.0-2.el9.noarch.r 995 kB/s | 16 kB 00:00 2024-09-13T18:10:23.689 INFO:teuthology.orchestra.run.smithi134.stdout:(117/138): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-13T18:10:23.699 INFO:teuthology.orchestra.run.smithi204.stdout:(17/138): librgw2-18.2.4-860.g6c6b083f.el9.x86_ 9.9 MB/s | 4.5 MB 00:00 2024-09-13T18:10:23.707 INFO:teuthology.orchestra.run.smithi114.stdout:(32/138): ceph-mgr-rook-18.2.4-860.g6c6b083f.el 593 kB/s | 49 kB 00:00 2024-09-13T18:10:23.715 INFO:teuthology.orchestra.run.smithi134.stdout:(118/138): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-09-13T18:10:23.732 INFO:teuthology.orchestra.run.smithi134.stdout:(119/138): python3-repoze-lru-0.7-16.el9.noarch 735 kB/s | 31 kB 00:00 2024-09-13T18:10:23.748 INFO:teuthology.orchestra.run.smithi134.stdout:(120/138): python3-rsa-4.9-2.el9.noarch.rpm 3.5 MB/s | 59 kB 00:00 2024-09-13T18:10:23.749 INFO:teuthology.orchestra.run.smithi204.stdout:(18/138): python3-ceph-argparse-18.2.4-860.g6c6 901 kB/s | 45 kB 00:00 2024-09-13T18:10:23.774 INFO:teuthology.orchestra.run.smithi134.stdout:(121/138): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-09-13T18:10:23.799 INFO:teuthology.orchestra.run.smithi134.stdout:(122/138): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-09-13T18:10:23.799 INFO:teuthology.orchestra.run.smithi204.stdout:(19/138): python3-ceph-common-18.2.4-860.g6c6b0 2.5 MB/s | 129 kB 00:00 2024-09-13T18:10:23.824 INFO:teuthology.orchestra.run.smithi134.stdout:(123/138): python3-typing-extensions-4.12.2-2.e 1.5 MB/s | 74 kB 00:00 2024-09-13T18:10:23.840 INFO:teuthology.orchestra.run.smithi114.stdout:(33/138): ceph-mgr-diskprediction-local-18.2.4- 12 MB/s | 7.4 MB 00:00 2024-09-13T18:10:23.858 INFO:teuthology.orchestra.run.smithi134.stdout:(124/138): python3-kubernetes-26.1.0-2.el9.noar 3.4 MB/s | 1.0 MB 00:00 2024-09-13T18:10:23.858 INFO:teuthology.orchestra.run.smithi204.stdout:(20/138): python3-cephfs-18.2.4-860.g6c6b083f.e 2.7 MB/s | 162 kB 00:00 2024-09-13T18:10:23.865 INFO:teuthology.orchestra.run.smithi114.stdout:(34/138): ceph-prometheus-alerts-18.2.4-860.g6c 103 kB/s | 16 kB 00:00 2024-09-13T18:10:23.883 INFO:teuthology.orchestra.run.smithi134.stdout:(125/138): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-09-13T18:10:23.890 INFO:teuthology.orchestra.run.smithi114.stdout:(35/138): ceph-volume-18.2.4-860.g6c6b083f.el9. 5.2 MB/s | 264 kB 00:00 2024-09-13T18:10:23.908 INFO:teuthology.orchestra.run.smithi134.stdout:(126/138): python3-webob-1.8.8-2.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-09-13T18:10:23.916 INFO:teuthology.orchestra.run.smithi114.stdout:(36/138): cephadm-18.2.4-860.g6c6b083f.el9.noar 4.4 MB/s | 226 kB 00:00 2024-09-13T18:10:23.933 INFO:teuthology.orchestra.run.smithi134.stdout:(127/138): python3-zc-lockfile-2.0-10.el9.noarc 399 kB/s | 20 kB 00:00 2024-09-13T18:10:23.941 INFO:teuthology.orchestra.run.smithi114.stdout:(37/138): cephfs-top-18.2.4-860.g6c6b083f.el9.n 364 kB/s | 18 kB 00:00 2024-09-13T18:10:23.942 INFO:teuthology.orchestra.run.smithi204.stdout:(21/138): ceph-radosgw-18.2.4-860.g6c6b083f.el9 3.2 MB/s | 7.7 MB 00:02 2024-09-13T18:10:23.959 INFO:teuthology.orchestra.run.smithi134.stdout:(128/138): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-13T18:10:23.967 INFO:teuthology.orchestra.run.smithi204.stdout:(22/138): python3-rados-18.2.4-860.g6c6b083f.el 2.9 MB/s | 321 kB 00:00 2024-09-13T18:10:23.984 INFO:teuthology.orchestra.run.smithi134.stdout:(129/138): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-13T18:10:24.043 INFO:teuthology.orchestra.run.smithi134.stdout:(130/138): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-13T18:10:24.083 INFO:teuthology.orchestra.run.smithi114.stdout:(38/138): fuse-2.9.9-16.el9.x86_64.rpm 478 kB/s | 80 kB 00:00 2024-09-13T18:10:24.118 INFO:teuthology.orchestra.run.smithi134.stdout:(131/138): NetworkManager-1.49.90-1.el9.x86_64. 31 MB/s | 2.3 MB 00:00 2024-09-13T18:10:24.166 INFO:teuthology.orchestra.run.smithi114.stdout:(39/138): libquadmath-11.5.0-2.el9.x86_64.rpm 2.2 MB/s | 191 kB 00:00 2024-09-13T18:10:24.185 INFO:teuthology.orchestra.run.smithi134.stdout:(132/138): NetworkManager-libnm-1.49.90-1.el9.x 27 MB/s | 1.8 MB 00:00 2024-09-13T18:10:24.227 INFO:teuthology.orchestra.run.smithi134.stdout:(133/138): NetworkManager-team-1.49.90-1.el9.x8 983 kB/s | 41 kB 00:00 2024-09-13T18:10:24.250 INFO:teuthology.orchestra.run.smithi114.stdout:(40/138): libgfortran-11.5.0-2.el9.x86_64.rpm 2.5 MB/s | 801 kB 00:00 2024-09-13T18:10:24.275 INFO:teuthology.orchestra.run.smithi114.stdout:(41/138): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.3 MB/s | 253 kB 00:00 2024-09-13T18:10:24.285 INFO:teuthology.orchestra.run.smithi134.stdout:(134/138): librbd1-18.2.4-860.g6c6b083f.el9.x86 10 MB/s | 3.0 MB 00:00 2024-09-13T18:10:24.311 INFO:teuthology.orchestra.run.smithi134.stdout:(135/138): NetworkManager-tui-1.49.90-1.el9.x86 2.9 MB/s | 249 kB 00:00 2024-09-13T18:10:24.320 INFO:teuthology.orchestra.run.smithi204.stdout:(23/138): ceph-test-18.2.4-860.g6c6b083f.el9.x8 16 MB/s | 40 MB 00:02 2024-09-13T18:10:24.325 INFO:teuthology.orchestra.run.smithi114.stdout:(42/138): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-13T18:10:24.351 INFO:teuthology.orchestra.run.smithi204.stdout:(24/138): python3-rgw-18.2.4-860.g6c6b083f.el9. 259 kB/s | 99 kB 00:00 2024-09-13T18:10:24.369 INFO:teuthology.orchestra.run.smithi134.stdout:(136/138): librados2-18.2.4-860.g6c6b083f.el9.x 8.1 MB/s | 3.3 MB 00:00 2024-09-13T18:10:24.376 INFO:teuthology.orchestra.run.smithi114.stdout:(43/138): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-13T18:10:24.376 INFO:teuthology.orchestra.run.smithi204.stdout:(25/138): python3-rbd-18.2.4-860.g6c6b083f.el9. 692 kB/s | 301 kB 00:00 2024-09-13T18:10:24.403 INFO:teuthology.orchestra.run.smithi134.stdout:(137/138): openssl-3.2.2-6.el9.x86_64.rpm 12 MB/s | 1.4 MB 00:00 2024-09-13T18:10:24.410 INFO:teuthology.orchestra.run.smithi204.stdout:(26/138): rbd-fuse-18.2.4-860.g6c6b083f.el9.x86 945 kB/s | 87 kB 00:00 2024-09-13T18:10:24.426 INFO:teuthology.orchestra.run.smithi114.stdout:(44/138): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-13T18:10:24.444 INFO:teuthology.orchestra.run.smithi204.stdout:(27/138): rbd-nbd-18.2.4-860.g6c6b083f.el9.x86_ 2.5 MB/s | 172 kB 00:00 2024-09-13T18:10:24.453 INFO:teuthology.orchestra.run.smithi134.stdout:(138/138): openssl-libs-3.2.2-6.el9.x86_64.rpm 17 MB/s | 2.4 MB 00:00 2024-09-13T18:10:24.459 INFO:teuthology.orchestra.run.smithi134.stdout:-------------------------------------------------------------------------------- 2024-09-13T18:10:24.459 INFO:teuthology.orchestra.run.smithi134.stdout:Total 29 MB/s | 203 MB 00:06 2024-09-13T18:10:24.468 INFO:teuthology.orchestra.run.smithi114.stdout:(45/138): python3-cryptography-36.0.1-4.el9.x86 5.6 MB/s | 1.2 MB 00:00 2024-09-13T18:10:24.469 INFO:teuthology.orchestra.run.smithi204.stdout:(28/138): ceph-grafana-dashboards-18.2.4-860.g6 418 kB/s | 24 kB 00:00 2024-09-13T18:10:24.493 INFO:teuthology.orchestra.run.smithi114.stdout:(46/138): python3-urllib3-1.26.5-6.el9.noarch.r 3.1 MB/s | 215 kB 00:00 2024-09-13T18:10:24.494 INFO:teuthology.orchestra.run.smithi204.stdout:(29/138): ceph-mgr-cephadm-18.2.4-860.g6c6b083f 2.7 MB/s | 139 kB 00:00 2024-09-13T18:10:24.570 INFO:teuthology.orchestra.run.smithi204.stdout:(30/138): rbd-mirror-18.2.4-860.g6c6b083f.el9.x 14 MB/s | 3.0 MB 00:00 2024-09-13T18:10:24.620 INFO:teuthology.orchestra.run.smithi204.stdout:(31/138): ceph-mgr-modules-core-18.2.4-860.g6c6 4.8 MB/s | 246 kB 00:00 2024-09-13T18:10:24.670 INFO:teuthology.orchestra.run.smithi204.stdout:(32/138): ceph-mgr-rook-18.2.4-860.g6c6b083f.el 985 kB/s | 49 kB 00:00 2024-09-13T18:10:24.732 INFO:teuthology.orchestra.run.smithi204.stdout:(33/138): ceph-mgr-dashboard-18.2.4-860.g6c6b08 14 MB/s | 3.5 MB 00:00 2024-09-13T18:10:24.735 INFO:teuthology.orchestra.run.smithi114.stdout:(47/138): automake-1.16.2-8.el9.noarch.rpm 2.5 MB/s | 689 kB 00:00 2024-09-13T18:10:24.754 INFO:teuthology.orchestra.run.smithi204.stdout:(34/138): ceph-prometheus-alerts-18.2.4-860.g6c 197 kB/s | 16 kB 00:00 2024-09-13T18:10:24.777 INFO:teuthology.orchestra.run.smithi114.stdout:(48/138): bison-3.7.4-5.el9.x86_64.rpm 3.2 MB/s | 943 kB 00:00 2024-09-13T18:10:24.779 INFO:teuthology.orchestra.run.smithi204.stdout:(35/138): ceph-volume-18.2.4-860.g6c6b083f.el9. 5.2 MB/s | 264 kB 00:00 2024-09-13T18:10:24.802 INFO:teuthology.orchestra.run.smithi114.stdout:(49/138): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-09-13T18:10:24.805 INFO:teuthology.orchestra.run.smithi204.stdout:(36/138): cephadm-18.2.4-860.g6c6b083f.el9.noar 4.4 MB/s | 226 kB 00:00 2024-09-13T18:10:24.830 INFO:teuthology.orchestra.run.smithi204.stdout:(37/138): cephfs-top-18.2.4-860.g6c6b083f.el9.n 363 kB/s | 18 kB 00:00 2024-09-13T18:10:24.836 INFO:teuthology.orchestra.run.smithi114.stdout:(50/138): elfutils-libelf-devel-0.191-4.el9.x86 370 kB/s | 22 kB 00:00 2024-09-13T18:10:24.886 INFO:teuthology.orchestra.run.smithi114.stdout:(51/138): flexiblas-3.0.4-8.el9.x86_64.rpm 654 kB/s | 33 kB 00:00 2024-09-13T18:10:24.920 INFO:teuthology.orchestra.run.smithi114.stdout:(52/138): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-09-13T18:10:24.930 INFO:teuthology.orchestra.run.smithi204.stdout:(38/138): fuse-2.9.9-16.el9.x86_64.rpm 637 kB/s | 80 kB 00:00 2024-09-13T18:10:24.970 INFO:teuthology.orchestra.run.smithi114.stdout:(53/138): flexiblas-openblas-openmp-3.0.4-8.el9 366 kB/s | 18 kB 00:00 2024-09-13T18:10:25.006 INFO:teuthology.orchestra.run.smithi204.stdout:(39/138): libquadmath-11.5.0-2.el9.x86_64.rpm 2.5 MB/s | 191 kB 00:00 2024-09-13T18:10:25.029 INFO:teuthology.orchestra.run.smithi114.stdout:(54/138): libaio-devel-0.3.111-13.el9.x86_64.rp 191 kB/s | 11 kB 00:00 2024-09-13T18:10:25.039 INFO:teuthology.orchestra.run.smithi204.stdout:(40/138): libgfortran-11.5.0-2.el9.x86_64.rpm 3.8 MB/s | 801 kB 00:00 2024-09-13T18:10:25.065 INFO:teuthology.orchestra.run.smithi204.stdout:(41/138): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 253 kB 00:00 2024-09-13T18:10:25.088 INFO:teuthology.orchestra.run.smithi114.stdout:(55/138): flexiblas-netlib-3.0.4-8.el9.x86_64.r 15 MB/s | 3.0 MB 00:00 2024-09-13T18:10:25.107 INFO:teuthology.orchestra.run.smithi204.stdout:(42/138): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-13T18:10:25.113 INFO:teuthology.orchestra.run.smithi114.stdout:(56/138): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-09-13T18:10:25.139 INFO:teuthology.orchestra.run.smithi114.stdout:(57/138): librabbitmq-0.11.0-7.el9.x86_64.rpm 899 kB/s | 45 kB 00:00 2024-09-13T18:10:25.174 INFO:teuthology.orchestra.run.smithi204.stdout:(43/138): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-09-13T18:10:25.197 INFO:teuthology.orchestra.run.smithi114.stdout:(58/138): libtool-2.4.6-46.el9.x86_64.rpm 9.7 MB/s | 581 kB 00:00 2024-09-13T18:10:25.215 INFO:teuthology.orchestra.run.smithi204.stdout:(44/138): python3-requests-2.25.1-8.el9.noarch. 2.9 MB/s | 125 kB 00:00 2024-09-13T18:10:25.231 INFO:teuthology.orchestra.run.smithi114.stdout:(59/138): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-09-13T18:10:25.264 INFO:teuthology.orchestra.run.smithi114.stdout:(60/138): libuuid-devel-2.37.4-20.el9.x86_64.rp 366 kB/s | 24 kB 00:00 2024-09-13T18:10:25.282 INFO:teuthology.orchestra.run.smithi204.stdout:(45/138): python3-cryptography-36.0.1-4.el9.x86 5.1 MB/s | 1.2 MB 00:00 2024-09-13T18:10:25.308 INFO:teuthology.orchestra.run.smithi204.stdout:(46/138): python3-urllib3-1.26.5-6.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-09-13T18:10:25.466 INFO:teuthology.orchestra.run.smithi204.stdout:(47/138): automake-1.16.2-8.el9.noarch.rpm 3.7 MB/s | 689 kB 00:00 2024-09-13T18:10:25.508 INFO:teuthology.orchestra.run.smithi204.stdout:(48/138): bison-3.7.4-5.el9.x86_64.rpm 4.6 MB/s | 943 kB 00:00 2024-09-13T18:10:25.534 INFO:teuthology.orchestra.run.smithi204.stdout:(49/138): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-09-13T18:10:25.576 INFO:teuthology.orchestra.run.smithi204.stdout:(50/138): elfutils-libelf-devel-0.191-4.el9.x86 323 kB/s | 22 kB 00:00 2024-09-13T18:10:25.579 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction check 2024-09-13T18:10:25.601 INFO:teuthology.orchestra.run.smithi204.stdout:(51/138): flex-2.6.4-9.el9.x86_64.rpm 4.6 MB/s | 313 kB 00:00 2024-09-13T18:10:25.623 INFO:teuthology.orchestra.run.smithi114.stdout:(61/138): ceph-test-18.2.4-860.g6c6b083f.el9.x8 11 MB/s | 40 MB 00:03 2024-09-13T18:10:25.635 INFO:teuthology.orchestra.run.smithi204.stdout:(52/138): flexiblas-3.0.4-8.el9.x86_64.rpm 558 kB/s | 33 kB 00:00 2024-09-13T18:10:25.648 INFO:teuthology.orchestra.run.smithi114.stdout:(62/138): libzstd-devel-1.5.1-2.el9.x86_64.rpm 114 kB/s | 47 kB 00:00 2024-09-13T18:10:25.673 INFO:teuthology.orchestra.run.smithi114.stdout:(63/138): lttng-ust-2.12.0-6.el9.x86_64.rpm 715 kB/s | 292 kB 00:00 2024-09-13T18:10:25.685 INFO:teuthology.orchestra.run.smithi204.stdout:(53/138): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-09-13T18:10:25.723 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction check succeeded. 2024-09-13T18:10:25.724 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction test 2024-09-13T18:10:25.732 INFO:teuthology.orchestra.run.smithi114.stdout:(64/138): openblas-0.3.26-2.el9.x86_64.rpm 462 kB/s | 38 kB 00:00 2024-09-13T18:10:25.769 INFO:teuthology.orchestra.run.smithi204.stdout:(54/138): ceph-mgr-diskprediction-local-18.2.4- 5.8 MB/s | 7.4 MB 00:01 2024-09-13T18:10:25.794 INFO:teuthology.orchestra.run.smithi204.stdout:(55/138): libaio-devel-0.3.111-13.el9.x86_64.rp 103 kB/s | 11 kB 00:00 2024-09-13T18:10:25.853 INFO:teuthology.orchestra.run.smithi204.stdout:(56/138): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2024-09-13T18:10:25.878 INFO:teuthology.orchestra.run.smithi204.stdout:(57/138): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-09-13T18:10:25.937 INFO:teuthology.orchestra.run.smithi204.stdout:(58/138): librdkafka-1.6.1-102.el9.x86_64.rpm 7.8 MB/s | 662 kB 00:00 2024-09-13T18:10:25.962 INFO:teuthology.orchestra.run.smithi204.stdout:(59/138): librabbitmq-0.11.0-7.el9.x86_64.rpm 270 kB/s | 45 kB 00:00 2024-09-13T18:10:25.987 INFO:teuthology.orchestra.run.smithi204.stdout:(60/138): libtool-2.4.6-46.el9.x86_64.rpm 5.2 MB/s | 581 kB 00:00 2024-09-13T18:10:26.012 INFO:teuthology.orchestra.run.smithi204.stdout:(61/138): libuuid-devel-2.37.4-20.el9.x86_64.rp 325 kB/s | 24 kB 00:00 2024-09-13T18:10:26.041 INFO:teuthology.orchestra.run.smithi114.stdout:(65/138): openssl-devel-3.2.2-6.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-09-13T18:10:26.055 INFO:teuthology.orchestra.run.smithi204.stdout:(62/138): libzstd-devel-1.5.1-2.el9.x86_64.rpm 514 kB/s | 47 kB 00:00 2024-09-13T18:10:26.080 INFO:teuthology.orchestra.run.smithi204.stdout:(63/138): openblas-0.3.26-2.el9.x86_64.rpm 575 kB/s | 38 kB 00:00 2024-09-13T18:10:26.082 INFO:teuthology.orchestra.run.smithi114.stdout:(66/138): postgresql-13.16-1.el9.x86_64.rpm 4.5 MB/s | 1.6 MB 00:00 2024-09-13T18:10:26.105 INFO:teuthology.orchestra.run.smithi204.stdout:(64/138): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-09-13T18:10:26.141 INFO:teuthology.orchestra.run.smithi114.stdout:(67/138): postgresql-contrib-13.16-1.el9.x86_64 8.9 MB/s | 910 kB 00:00 2024-09-13T18:10:26.166 INFO:teuthology.orchestra.run.smithi114.stdout:(68/138): postgresql-private-libs-13.16-1.el9.x 1.6 MB/s | 137 kB 00:00 2024-09-13T18:10:26.230 INFO:teuthology.orchestra.run.smithi204.stdout:(65/138): openssl-devel-3.2.2-6.el9.x86_64.rpm 29 MB/s | 4.4 MB 00:00 2024-09-13T18:10:26.264 INFO:teuthology.orchestra.run.smithi204.stdout:(66/138): postgresql-13.16-1.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-09-13T18:10:26.275 INFO:teuthology.orchestra.run.smithi114.stdout:(69/138): openblas-openmp-0.3.26-2.el9.x86_64.r 7.9 MB/s | 4.9 MB 00:00 2024-09-13T18:10:26.297 INFO:teuthology.orchestra.run.smithi204.stdout:(67/138): postgresql-contrib-13.16-1.el9.x86_64 13 MB/s | 910 kB 00:00 2024-09-13T18:10:26.334 INFO:teuthology.orchestra.run.smithi114.stdout:(70/138): python3-devel-3.9.19-8.el9.x86_64.rpm 4.1 MB/s | 245 kB 00:00 2024-09-13T18:10:26.365 INFO:teuthology.orchestra.run.smithi204.stdout:(68/138): openblas-openmp-0.3.26-2.el9.x86_64.r 16 MB/s | 4.9 MB 00:00 2024-09-13T18:10:26.390 INFO:teuthology.orchestra.run.smithi204.stdout:(69/138): postgresql-private-libs-13.16-1.el9.x 1.1 MB/s | 137 kB 00:00 2024-09-13T18:10:26.434 INFO:teuthology.orchestra.run.smithi114.stdout:(71/138): postgresql-server-13.16-1.el9.x86_64. 20 MB/s | 5.8 MB 00:00 2024-09-13T18:10:26.459 INFO:teuthology.orchestra.run.smithi114.stdout:(72/138): python3-jinja2-2.11.3-6.el9.noarch.rp 1.9 MB/s | 248 kB 00:00 2024-09-13T18:10:26.493 INFO:teuthology.orchestra.run.smithi114.stdout:(73/138): python3-jsonpatch-1.21-16.el9.noarch. 439 kB/s | 26 kB 00:00 2024-09-13T18:10:26.498 INFO:teuthology.orchestra.run.smithi204.stdout:(70/138): postgresql-server-13.16-1.el9.x86_64. 29 MB/s | 5.8 MB 00:00 2024-09-13T18:10:26.518 INFO:teuthology.orchestra.run.smithi114.stdout:(74/138): python3-jsonpointer-2.0-4.el9.noarch. 326 kB/s | 19 kB 00:00 2024-09-13T18:10:26.524 INFO:teuthology.orchestra.run.smithi204.stdout:(71/138): python3-devel-3.9.19-8.el9.x86_64.rpm 1.8 MB/s | 245 kB 00:00 2024-09-13T18:10:26.551 INFO:teuthology.orchestra.run.smithi114.stdout:(75/138): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-09-13T18:10:26.577 INFO:teuthology.orchestra.run.smithi114.stdout:(76/138): python3-markupsafe-1.1.1-12.el9.x86_6 593 kB/s | 35 kB 00:00 2024-09-13T18:10:26.632 INFO:teuthology.orchestra.run.smithi204.stdout:(72/138): python3-babel-2.9.1-2.el9.noarch.rpm 22 MB/s | 6.0 MB 00:00 2024-09-13T18:10:26.635 INFO:teuthology.orchestra.run.smithi114.stdout:(77/138): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-09-13T18:10:26.658 INFO:teuthology.orchestra.run.smithi204.stdout:(73/138): python3-jinja2-2.11.3-6.el9.noarch.rp 1.5 MB/s | 248 kB 00:00 2024-09-13T18:10:26.683 INFO:teuthology.orchestra.run.smithi204.stdout:(74/138): python3-jsonpatch-1.21-16.el9.noarch. 161 kB/s | 26 kB 00:00 2024-09-13T18:10:26.708 INFO:teuthology.orchestra.run.smithi204.stdout:(75/138): python3-jsonpointer-2.0-4.el9.noarch. 254 kB/s | 19 kB 00:00 2024-09-13T18:10:26.733 INFO:teuthology.orchestra.run.smithi204.stdout:(76/138): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-09-13T18:10:26.735 INFO:teuthology.orchestra.run.smithi114.stdout:(78/138): python3-babel-2.9.1-2.el9.noarch.rpm 10 MB/s | 6.0 MB 00:00 2024-09-13T18:10:26.758 INFO:teuthology.orchestra.run.smithi204.stdout:(77/138): python3-markupsafe-1.1.1-12.el9.x86_6 460 kB/s | 35 kB 00:00 2024-09-13T18:10:26.761 INFO:teuthology.orchestra.run.smithi114.stdout:(79/138): python3-oauthlib-3.1.1-5.el9.noarch.r 1.7 MB/s | 222 kB 00:00 2024-09-13T18:10:26.801 INFO:teuthology.orchestra.run.smithi204.stdout:(78/138): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.7 MB/s | 250 kB 00:00 2024-09-13T18:10:26.836 INFO:teuthology.orchestra.run.smithi114.stdout:(80/138): python3-numpy-1.20.1-5.el9.x86_64.rpm 18 MB/s | 5.1 MB 00:00 2024-09-13T18:10:26.862 INFO:teuthology.orchestra.run.smithi114.stdout:(81/138): python3-prettytable-0.7.2-27.el9.noar 421 kB/s | 42 kB 00:00 2024-09-13T18:10:26.887 INFO:teuthology.orchestra.run.smithi114.stdout:(82/138): python3-packaging-20.9-5.el9.noarch.r 511 kB/s | 77 kB 00:00 2024-09-13T18:10:26.901 INFO:teuthology.orchestra.run.smithi204.stdout:(79/138): python3-numpy-1.20.1-5.el9.x86_64.rpm 26 MB/s | 5.1 MB 00:00 2024-09-13T18:10:26.912 INFO:teuthology.orchestra.run.smithi114.stdout:(83/138): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-09-13T18:10:26.926 INFO:teuthology.orchestra.run.smithi204.stdout:(80/138): python3-packaging-20.9-5.el9.noarch.r 616 kB/s | 77 kB 00:00 2024-09-13T18:10:26.937 INFO:teuthology.orchestra.run.smithi114.stdout:(84/138): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-09-13T18:10:26.952 INFO:teuthology.orchestra.run.smithi204.stdout:(81/138): python3-oauthlib-3.1.1-5.el9.noarch.r 1.1 MB/s | 222 kB 00:00 2024-09-13T18:10:26.962 INFO:teuthology.orchestra.run.smithi114.stdout:(85/138): python3-pytz-2021.1-5.el9.noarch.rpm 677 kB/s | 51 kB 00:00 2024-09-13T18:10:26.977 INFO:teuthology.orchestra.run.smithi204.stdout:(82/138): python3-prettytable-0.7.2-27.el9.noar 560 kB/s | 42 kB 00:00 2024-09-13T18:10:26.988 INFO:teuthology.orchestra.run.smithi114.stdout:(86/138): python3-requests-oauthlib-1.3.0-12.el 712 kB/s | 54 kB 00:00 2024-09-13T18:10:27.002 INFO:teuthology.orchestra.run.smithi204.stdout:(83/138): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-09-13T18:10:27.021 INFO:teuthology.orchestra.run.smithi114.stdout:(87/138): python3-toml-0.10.2-6.el9.noarch.rpm 711 kB/s | 42 kB 00:00 2024-09-13T18:10:27.028 INFO:teuthology.orchestra.run.smithi204.stdout:(84/138): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-09-13T18:10:27.044 INFO:teuthology.orchestra.run.smithi204.stdout:(85/138): python3-pytz-2021.1-5.el9.noarch.rpm 759 kB/s | 51 kB 00:00 2024-09-13T18:10:27.046 INFO:teuthology.orchestra.run.smithi114.stdout:(88/138): socat-1.7.4.1-6.el9.x86_64.rpm 5.1 MB/s | 304 kB 00:00 2024-09-13T18:10:27.061 INFO:teuthology.orchestra.run.smithi204.stdout:(86/138): python3-requests-oauthlib-1.3.0-12.el 911 kB/s | 54 kB 00:00 2024-09-13T18:10:27.080 INFO:teuthology.orchestra.run.smithi114.stdout:(89/138): uuid-1.6.2-55.el9.x86_64.rpm 990 kB/s | 58 kB 00:00 2024-09-13T18:10:27.103 INFO:teuthology.orchestra.run.smithi204.stdout:(87/138): python3-toml-0.10.2-6.el9.noarch.rpm 711 kB/s | 42 kB 00:00 2024-09-13T18:10:27.106 INFO:teuthology.orchestra.run.smithi114.stdout:(90/138): xfsprogs-devel-6.4.0-4.el9.x86_64.rpm 1.6 MB/s | 96 kB 00:00 2024-09-13T18:10:27.129 INFO:teuthology.orchestra.run.smithi204.stdout:(88/138): socat-1.7.4.1-6.el9.x86_64.rpm 4.4 MB/s | 304 kB 00:00 2024-09-13T18:10:27.139 INFO:teuthology.orchestra.run.smithi114.stdout:(91/138): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-13T18:10:27.167 INFO:teuthology.orchestra.run.smithi114.stdout:(92/138): zlib-devel-1.2.11-41.el9.x86_64.rpm 739 kB/s | 45 kB 00:00 2024-09-13T18:10:27.179 INFO:teuthology.orchestra.run.smithi204.stdout:(89/138): uuid-1.6.2-55.el9.x86_64.rpm 767 kB/s | 58 kB 00:00 2024-09-13T18:10:27.204 INFO:teuthology.orchestra.run.smithi204.stdout:(90/138): xfsprogs-devel-6.4.0-4.el9.x86_64.rpm 1.2 MB/s | 96 kB 00:00 2024-09-13T18:10:27.225 INFO:teuthology.orchestra.run.smithi114.stdout:(93/138): gperftools-libs-2.9.1-3.el9.x86_64.rp 5.9 MB/s | 308 kB 00:00 2024-09-13T18:10:27.238 INFO:teuthology.orchestra.run.smithi204.stdout:(91/138): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-13T18:10:27.251 INFO:teuthology.orchestra.run.smithi114.stdout:(94/138): fmt-8.1.1-5.el9.x86_64.rpm 996 kB/s | 111 kB 00:00 2024-09-13T18:10:27.263 INFO:teuthology.orchestra.run.smithi204.stdout:(92/138): zlib-devel-1.2.11-41.el9.x86_64.rpm 765 kB/s | 45 kB 00:00 2024-09-13T18:10:27.306 INFO:teuthology.orchestra.run.smithi204.stdout:(93/138): fmt-8.1.1-5.el9.x86_64.rpm 1.9 MB/s | 111 kB 00:00 2024-09-13T18:10:27.306 INFO:teuthology.orchestra.run.smithi114.stdout:(95/138): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-13T18:10:27.306 INFO:teuthology.orchestra.run.smithi114.stdout:(96/138): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-13T18:10:27.335 INFO:teuthology.orchestra.run.smithi114.stdout:(97/138): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-09-13T18:10:27.356 INFO:teuthology.orchestra.run.smithi204.stdout:(94/138): gperftools-libs-2.9.1-3.el9.x86_64.rp 3.3 MB/s | 308 kB 00:00 2024-09-13T18:10:27.381 INFO:teuthology.orchestra.run.smithi204.stdout:(95/138): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-13T18:10:27.402 INFO:teuthology.orchestra.run.smithi114.stdout:(98/138): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-09-13T18:10:27.406 INFO:teuthology.orchestra.run.smithi204.stdout:(96/138): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-13T18:10:27.440 INFO:teuthology.orchestra.run.smithi204.stdout:(97/138): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-09-13T18:10:27.452 INFO:teuthology.orchestra.run.smithi114.stdout:(99/138): python3-asyncssh-2.13.2-5.el9.noarch. 11 MB/s | 548 kB 00:00 2024-09-13T18:10:27.469 INFO:teuthology.orchestra.run.smithi114.stdout:(100/138): python3-bcrypt-3.2.2-1.el9.x86_64.rp 2.6 MB/s | 43 kB 00:00 2024-09-13T18:10:27.527 INFO:teuthology.orchestra.run.smithi114.stdout:(101/138): libarrow-9.0.0-11.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-09-13T18:10:27.544 INFO:teuthology.orchestra.run.smithi114.stdout:(102/138): python3-cachetools-4.2.4-1.el9.noarc 427 kB/s | 32 kB 00:00 2024-09-13T18:10:27.561 INFO:teuthology.orchestra.run.smithi114.stdout:(103/138): python3-certifi-2023.05.07-4.el9.noa 421 kB/s | 14 kB 00:00 2024-09-13T18:10:27.597 INFO:teuthology.orchestra.run.smithi114.stdout:(104/138): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-13T18:10:27.615 INFO:teuthology.orchestra.run.smithi204.stdout:(98/138): python3-scipy-1.6.2-8.el9.x86_64.rpm 28 MB/s | 16 MB 00:00 2024-09-13T18:10:27.620 INFO:teuthology.orchestra.run.smithi114.stdout:(105/138): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-13T18:10:27.646 INFO:teuthology.orchestra.run.smithi114.stdout:(106/138): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-13T18:10:27.665 INFO:teuthology.orchestra.run.smithi204.stdout:(99/138): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.6 MB/s | 839 kB 00:00 2024-09-13T18:10:27.670 INFO:teuthology.orchestra.run.smithi114.stdout:(107/138): python3-jaraco-8.2.1-3.el9.noarch.rp 214 kB/s | 11 kB 00:00 2024-09-13T18:10:27.691 INFO:teuthology.orchestra.run.smithi204.stdout:(100/138): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.8 MB/s | 43 kB 00:00 2024-09-13T18:10:27.696 INFO:teuthology.orchestra.run.smithi114.stdout:(108/138): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-09-13T18:10:27.716 INFO:teuthology.orchestra.run.smithi204.stdout:(101/138): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-13T18:10:27.721 INFO:teuthology.orchestra.run.smithi114.stdout:(109/138): python3-jaraco-collections-3.0.0-8.e 459 kB/s | 23 kB 00:00 2024-09-13T18:10:27.741 INFO:teuthology.orchestra.run.smithi204.stdout:(102/138): python3-certifi-2023.05.07-4.el9.noa 566 kB/s | 14 kB 00:00 2024-09-13T18:10:27.746 INFO:teuthology.orchestra.run.smithi114.stdout:(110/138): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-09-13T18:10:27.752 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction test succeeded. 2024-09-13T18:10:27.752 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction 2024-09-13T18:10:27.766 INFO:teuthology.orchestra.run.smithi204.stdout:(103/138): python3-cheroot-8.6.0-4.el9.noarch.r 6.7 MB/s | 172 kB 00:00 2024-09-13T18:10:27.772 INFO:teuthology.orchestra.run.smithi114.stdout:(111/138): python3-jaraco-text-3.2.0-6.el9.noar 393 kB/s | 20 kB 00:00 2024-09-13T18:10:27.800 INFO:teuthology.orchestra.run.smithi204.stdout:(104/138): python3-asyncssh-2.13.2-5.el9.noarch 2.9 MB/s | 548 kB 00:00 2024-09-13T18:10:27.875 INFO:teuthology.orchestra.run.smithi204.stdout:(105/138): libarrow-9.0.0-11.el9.x86_64.rpm 7.7 MB/s | 4.4 MB 00:00 2024-09-13T18:10:27.901 INFO:teuthology.orchestra.run.smithi204.stdout:(106/138): python3-jaraco-8.2.1-3.el9.noarch.rp 430 kB/s | 11 kB 00:00 2024-09-13T18:10:27.926 INFO:teuthology.orchestra.run.smithi204.stdout:(107/138): python3-jaraco-classes-3.2.1-5.el9.n 703 kB/s | 18 kB 00:00 2024-09-13T18:10:27.930 INFO:teuthology.orchestra.run.smithi114.stdout:(112/138): python3-scipy-1.6.2-8.el9.x86_64.rpm 16 MB/s | 16 MB 00:00 2024-09-13T18:10:27.952 INFO:teuthology.orchestra.run.smithi204.stdout:(108/138): python3-google-auth-2.34.0-1.el9.noa 1.4 MB/s | 220 kB 00:00 2024-09-13T18:10:27.955 INFO:teuthology.orchestra.run.smithi114.stdout:(113/138): python3-logutils-0.3.5-21.el9.noarch 252 kB/s | 46 kB 00:00 2024-09-13T18:10:27.977 INFO:teuthology.orchestra.run.smithi204.stdout:(109/138): python3-jaraco-collections-3.0.0-8.e 460 kB/s | 23 kB 00:00 2024-09-13T18:10:27.989 INFO:teuthology.orchestra.run.smithi114.stdout:(114/138): python3-natsort-7.1.1-5.el9.noarch.r 2.1 MB/s | 58 kB 00:00 2024-09-13T18:10:28.005 INFO:teuthology.orchestra.run.smithi204.stdout:(110/138): python3-cherrypy-18.6.1-2.el9.noarch 1.5 MB/s | 358 kB 00:00 2024-09-13T18:10:28.023 INFO:teuthology.orchestra.run.smithi114.stdout:(115/138): python3-pecan-1.4.2-3.el9.noarch.rpm 8.5 MB/s | 272 kB 00:00 2024-09-13T18:10:28.027 INFO:teuthology.orchestra.run.smithi204.stdout:(111/138): python3-jaraco-functools-3.5.0-2.el9 258 kB/s | 19 kB 00:00 2024-09-13T18:10:28.048 INFO:teuthology.orchestra.run.smithi114.stdout:(116/138): python3-more-itertools-8.12.0-2.el9. 673 kB/s | 79 kB 00:00 2024-09-13T18:10:28.053 INFO:teuthology.orchestra.run.smithi204.stdout:(112/138): python3-jaraco-text-3.2.0-6.el9.noar 260 kB/s | 20 kB 00:00 2024-09-13T18:10:28.073 INFO:teuthology.orchestra.run.smithi114.stdout:(117/138): python3-portend-3.1.0-2.el9.noarch.r 326 kB/s | 16 kB 00:00 2024-09-13T18:10:28.078 INFO:teuthology.orchestra.run.smithi204.stdout:(113/138): python3-logutils-0.3.5-21.el9.noarch 922 kB/s | 46 kB 00:00 2024-09-13T18:10:28.099 INFO:teuthology.orchestra.run.smithi114.stdout:(118/138): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-09-13T18:10:28.103 INFO:teuthology.orchestra.run.smithi204.stdout:(114/138): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-09-13T18:10:28.124 INFO:teuthology.orchestra.run.smithi114.stdout:(119/138): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-09-13T18:10:28.128 INFO:teuthology.orchestra.run.smithi204.stdout:(115/138): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-09-13T18:10:28.154 INFO:teuthology.orchestra.run.smithi204.stdout:(116/138): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-09-13T18:10:28.157 INFO:teuthology.orchestra.run.smithi114.stdout:(120/138): python3-kubernetes-26.1.0-2.el9.noar 2.5 MB/s | 1.0 MB 00:00 2024-09-13T18:10:28.179 INFO:teuthology.orchestra.run.smithi204.stdout:(117/138): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-09-13T18:10:28.183 INFO:teuthology.orchestra.run.smithi114.stdout:(121/138): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-09-13T18:10:28.204 INFO:teuthology.orchestra.run.smithi204.stdout:(118/138): python3-repoze-lru-0.7-16.el9.noarch 1.2 MB/s | 31 kB 00:00 2024-09-13T18:10:28.208 INFO:teuthology.orchestra.run.smithi114.stdout:(122/138): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-09-13T18:10:28.229 INFO:teuthology.orchestra.run.smithi204.stdout:(119/138): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-09-13T18:10:28.242 INFO:teuthology.orchestra.run.smithi114.stdout:(123/138): python3-routes-2.5.1-5.el9.noarch.rp 1.3 MB/s | 188 kB 00:00 2024-09-13T18:10:28.255 INFO:teuthology.orchestra.run.smithi204.stdout:(120/138): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-13T18:10:28.267 INFO:teuthology.orchestra.run.smithi114.stdout:(124/138): python3-typing-extensions-4.12.2-2.e 887 kB/s | 74 kB 00:00 2024-09-13T18:10:28.280 INFO:teuthology.orchestra.run.smithi204.stdout:(121/138): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-13T18:10:28.292 INFO:teuthology.orchestra.run.smithi114.stdout:(125/138): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-13T18:10:28.305 INFO:teuthology.orchestra.run.smithi204.stdout:(122/138): python3-tempora-5.0.0-2.el9.noarch.r 715 kB/s | 36 kB 00:00 2024-09-13T18:10:28.318 INFO:teuthology.orchestra.run.smithi114.stdout:(126/138): python3-zc-lockfile-2.0-10.el9.noarc 803 kB/s | 20 kB 00:00 2024-09-13T18:10:28.343 INFO:teuthology.orchestra.run.smithi114.stdout:(127/138): python3-webob-1.8.8-2.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-09-13T18:10:28.368 INFO:teuthology.orchestra.run.smithi114.stdout:(128/138): python3-werkzeug-2.0.3-3.el9.1.noarc 4.1 MB/s | 427 kB 00:00 2024-09-13T18:10:28.390 INFO:teuthology.orchestra.run.smithi204.stdout:(123/138): python3-typing-extensions-4.12.2-2.e 881 kB/s | 74 kB 00:00 2024-09-13T18:10:28.394 INFO:teuthology.orchestra.run.smithi114.stdout:(129/138): re2-20211101-20.el9.x86_64.rpm 2.5 MB/s | 191 kB 00:00 2024-09-13T18:10:28.431 INFO:teuthology.orchestra.run.smithi204.stdout:(124/138): python3-kubernetes-26.1.0-2.el9.noar 2.4 MB/s | 1.0 MB 00:00 2024-09-13T18:10:28.457 INFO:teuthology.orchestra.run.smithi204.stdout:(125/138): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-09-13T18:10:28.470 INFO:teuthology.orchestra.run.smithi114.stdout:(130/138): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-09-13T18:10:28.490 INFO:teuthology.orchestra.run.smithi204.stdout:(126/138): python3-webob-1.8.8-2.el9.noarch.rpm 1.2 MB/s | 230 kB 00:00 2024-09-13T18:10:28.515 INFO:teuthology.orchestra.run.smithi204.stdout:(127/138): python3-zc-lockfile-2.0-10.el9.noarc 341 kB/s | 20 kB 00:00 2024-09-13T18:10:28.541 INFO:teuthology.orchestra.run.smithi204.stdout:(128/138): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-13T18:10:28.574 INFO:teuthology.orchestra.run.smithi204.stdout:(129/138): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-09-13T18:10:28.624 INFO:teuthology.orchestra.run.smithi204.stdout:(130/138): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-13T18:10:28.678 INFO:teuthology.orchestra.run.smithi114.stdout:(131/138): librados2-18.2.4-860.g6c6b083f.el9.x 11 MB/s | 3.3 MB 00:00 2024-09-13T18:10:28.728 INFO:teuthology.orchestra.run.smithi114.stdout:(132/138): librbd1-18.2.4-860.g6c6b083f.el9.x86 9.0 MB/s | 3.0 MB 00:00 2024-09-13T18:10:28.825 INFO:teuthology.orchestra.run.smithi204.stdout:(131/138): librados2-18.2.4-860.g6c6b083f.el9.x 12 MB/s | 3.3 MB 00:00 2024-09-13T18:10:28.845 INFO:teuthology.orchestra.run.smithi114.stdout:(133/138): NetworkManager-team-1.49.90-1.el9.x8 350 kB/s | 41 kB 00:00 2024-09-13T18:10:28.962 INFO:teuthology.orchestra.run.smithi114.stdout:(134/138): NetworkManager-libnm-1.49.90-1.el9.x 6.4 MB/s | 1.8 MB 00:00 2024-09-13T18:10:28.995 INFO:teuthology.orchestra.run.smithi114.stdout:(135/138): NetworkManager-tui-1.49.90-1.el9.x86 1.6 MB/s | 249 kB 00:00 2024-09-13T18:10:29.037 INFO:teuthology.orchestra.run.smithi114.stdout:(136/138): openssl-3.2.2-6.el9.x86_64.rpm 18 MB/s | 1.4 MB 00:00 2024-09-13T18:10:29.067 INFO:teuthology.orchestra.run.smithi204.stdout:(132/138): librbd1-18.2.4-860.g6c6b083f.el9.x86 6.1 MB/s | 3.0 MB 00:00 2024-09-13T18:10:29.117 INFO:teuthology.orchestra.run.smithi204.stdout:(133/138): NetworkManager-1.49.90-1.el9.x86_64. 4.7 MB/s | 2.3 MB 00:00 2024-09-13T18:10:29.175 INFO:teuthology.orchestra.run.smithi204.stdout:(134/138): NetworkManager-team-1.49.90-1.el9.x8 375 kB/s | 41 kB 00:00 2024-09-13T18:10:29.209 INFO:teuthology.orchestra.run.smithi204.stdout:(135/138): NetworkManager-tui-1.49.90-1.el9.x86 2.6 MB/s | 249 kB 00:00 2024-09-13T18:10:29.221 INFO:teuthology.orchestra.run.smithi114.stdout:(137/138): NetworkManager-1.49.90-1.el9.x86_64. 3.1 MB/s | 2.3 MB 00:00 2024-09-13T18:10:29.277 INFO:teuthology.orchestra.run.smithi204.stdout:(136/138): NetworkManager-libnm-1.49.90-1.el9.x 4.0 MB/s | 1.8 MB 00:00 2024-09-13T18:10:29.312 INFO:teuthology.orchestra.run.smithi114.stdout:(138/138): openssl-libs-3.2.2-6.el9.x86_64.rpm 7.5 MB/s | 2.4 MB 00:00 2024-09-13T18:10:29.318 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-09-13T18:10:29.318 INFO:teuthology.orchestra.run.smithi114.stdout:Total 20 MB/s | 203 MB 00:10 2024-09-13T18:10:29.379 INFO:teuthology.orchestra.run.smithi204.stdout:(137/138): openssl-3.2.2-6.el9.x86_64.rpm 6.6 MB/s | 1.4 MB 00:00 2024-09-13T18:10:29.555 INFO:teuthology.orchestra.run.smithi204.stdout:(138/138): openssl-libs-3.2.2-6.el9.x86_64.rpm 6.9 MB/s | 2.4 MB 00:00 2024-09-13T18:10:29.560 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-09-13T18:10:29.560 INFO:teuthology.orchestra.run.smithi204.stdout:Total 20 MB/s | 203 MB 00:10 2024-09-13T18:10:30.395 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-09-13T18:10:30.539 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-09-13T18:10:30.539 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-09-13T18:10:30.656 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-09-13T18:10:30.803 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-09-13T18:10:30.803 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-09-13T18:10:32.586 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-09-13T18:10:32.586 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-09-13T18:10:32.872 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-09-13T18:10:32.873 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-09-13T18:10:33.400 INFO:teuthology.orchestra.run.smithi134.stdout: Preparing : 1/1 2024-09-13T18:10:33.482 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/146 2024-09-13T18:10:33.529 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/146 2024-09-13T18:10:33.581 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/146 2024-09-13T18:10:33.615 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/146 2024-09-13T18:10:33.641 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/146 2024-09-13T18:10:33.697 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/146 2024-09-13T18:10:34.012 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/146 2024-09-13T18:10:34.141 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : librados2-2:18.2.4-860.g6c6b083f.el9.x86_64 8/146 2024-09-13T18:10:34.261 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librados2-2:18.2.4-860.g6c6b083f.el9.x86_64 8/146 2024-09-13T18:10:34.292 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephfs2-2:18.2.4-860.g6c6b083f.el9.x86_64 9/146 2024-09-13T18:10:34.384 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libcephfs2-2:18.2.4-860.g6c6b083f.el9.x86_64 9/146 2024-09-13T18:10:34.413 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rados-2:18.2.4-860.g6c6b083f.el9.x86_64 10/146 2024-09-13T18:10:34.572 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/146 2024-09-13T18:10:34.626 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/146 2024-09-13T18:10:34.653 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephsqlite-2:18.2.4-860.g6c6b083f.el9.x86_64 13/146 2024-09-13T18:10:34.755 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libcephsqlite-2:18.2.4-860.g6c6b083f.el9.x86_64 13/146 2024-09-13T18:10:34.779 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libradosstriper1-2:18.2.4-860.g6c6b083f.el9.x86_ 14/146 2024-09-13T18:10:34.984 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libradosstriper1-2:18.2.4-860.g6c6b083f.el9.x86_ 14/146 2024-09-13T18:10:35.126 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 15/146 2024-09-13T18:10:35.243 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 16/146 2024-09-13T18:10:35.318 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 16/146 2024-09-13T18:10:35.588 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 16/146 2024-09-13T18:10:35.623 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 17/146 2024-09-13T18:10:35.658 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : liboath-2.6.7-2.el9.x86_64 18/146 2024-09-13T18:10:35.731 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/146 2024-09-13T18:10:35.765 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 20/146 2024-09-13T18:10:35.792 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 21/146 2024-09-13T18:10:35.870 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 22/146 2024-09-13T18:10:35.923 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 23/146 2024-09-13T18:10:35.954 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 24/146 2024-09-13T18:10:36.000 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 25/146 2024-09-13T18:10:36.073 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 26/146 2024-09-13T18:10:36.108 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ceph-common-2:18.2.4-860.g6c6b083f.el9.x 27/146 2024-09-13T18:10:36.143 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ceph-argparse-2:18.2.4-860.g6c6b083f.el9 28/146 2024-09-13T18:10:36.216 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cephfs-2:18.2.4-860.g6c6b083f.el9.x86_64 29/146 2024-09-13T18:10:36.359 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 30/146 2024-09-13T18:10:36.422 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 31/146 2024-09-13T18:10:36.952 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rsa-4.9-2.el9.noarch 32/146 2024-09-13T18:10:37.039 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 33/146 2024-09-13T18:10:37.079 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 34/146 2024-09-13T18:10:37.116 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librados-devel-2:18.2.4-860.g6c6b083f.el9.x86_64 35/146 2024-09-13T18:10:37.157 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 36/146 2024-09-13T18:10:37.322 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 37/146 2024-09-13T18:10:37.383 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 38/146 2024-09-13T18:10:37.491 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : re2-1:20211101-20.el9.x86_64 39/146 2024-09-13T18:10:37.528 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 40/146 2024-09-13T18:10:37.582 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 41/146 2024-09-13T18:10:37.654 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 42/146 2024-09-13T18:10:37.687 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 43/146 2024-09-13T18:10:37.723 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 44/146 2024-09-13T18:10:37.770 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 45/146 2024-09-13T18:10:37.815 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 46/146 2024-09-13T18:10:37.857 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 47/146 2024-09-13T18:10:37.992 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 48/146 2024-09-13T18:10:38.034 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 49/146 2024-09-13T18:10:38.075 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 50/146 2024-09-13T18:10:38.176 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 51/146 2024-09-13T18:10:38.209 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 52/146 2024-09-13T18:10:38.249 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 53/146 2024-09-13T18:10:38.282 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 54/146 2024-09-13T18:10:38.285 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-09-13T18:10:38.309 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 55/146 2024-09-13T18:10:38.343 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : fmt-8.1.1-5.el9.x86_64 56/146 2024-09-13T18:10:38.371 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 57/146 2024-09-13T18:10:38.375 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/146 2024-09-13T18:10:38.406 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 58/146 2024-09-13T18:10:38.418 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/146 2024-09-13T18:10:38.442 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : uuid-1.6.2-55.el9.x86_64 59/146 2024-09-13T18:10:38.472 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 60/146 2024-09-13T18:10:38.484 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/146 2024-09-13T18:10:38.506 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 61/146 2024-09-13T18:10:38.516 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/146 2024-09-13T18:10:38.539 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 62/146 2024-09-13T18:10:38.540 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-09-13T18:10:38.542 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/146 2024-09-13T18:10:38.581 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 63/146 2024-09-13T18:10:38.607 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 64/146 2024-09-13T18:10:38.609 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/146 2024-09-13T18:10:38.643 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 65/146 2024-09-13T18:10:38.676 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/146 2024-09-13T18:10:38.689 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 66/146 2024-09-13T18:10:38.787 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-packaging-20.9-5.el9.noarch 67/146 2024-09-13T18:10:38.822 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/146 2024-09-13T18:10:38.835 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 68/146 2024-09-13T18:10:38.940 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/146 2024-09-13T18:10:38.960 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/146 2024-09-13T18:10:39.001 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/146 2024-09-13T18:10:39.027 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/146 2024-09-13T18:10:39.085 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : librados2-2:18.2.4-860.g6c6b083f.el9.x86_64 8/146 2024-09-13T18:10:39.089 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/146 2024-09-13T18:10:39.204 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librados2-2:18.2.4-860.g6c6b083f.el9.x86_64 8/146 2024-09-13T18:10:39.236 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libcephfs2-2:18.2.4-860.g6c6b083f.el9.x86_64 9/146 2024-09-13T18:10:39.310 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: libcephfs2-2:18.2.4-860.g6c6b083f.el9.x86_64 9/146 2024-09-13T18:10:39.341 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-rados-2:18.2.4-860.g6c6b083f.el9.x86_64 10/146 2024-09-13T18:10:39.466 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/146 2024-09-13T18:10:39.504 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/146 2024-09-13T18:10:39.536 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : librados2-2:18.2.4-860.g6c6b083f.el9.x86_64 8/146 2024-09-13T18:10:39.547 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 69/146 2024-09-13T18:10:39.571 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/146 2024-09-13T18:10:39.599 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libcephsqlite-2:18.2.4-860.g6c6b083f.el9.x86_64 13/146 2024-09-13T18:10:39.660 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librados2-2:18.2.4-860.g6c6b083f.el9.x86_64 8/146 2024-09-13T18:10:39.696 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephfs2-2:18.2.4-860.g6c6b083f.el9.x86_64 9/146 2024-09-13T18:10:39.700 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: libcephsqlite-2:18.2.4-860.g6c6b083f.el9.x86_64 13/146 2024-09-13T18:10:39.725 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libradosstriper1-2:18.2.4-860.g6c6b083f.el9.x86_ 14/146 2024-09-13T18:10:39.732 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 70/146 2024-09-13T18:10:39.766 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 71/146 2024-09-13T18:10:39.786 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libcephfs2-2:18.2.4-860.g6c6b083f.el9.x86_64 9/146 2024-09-13T18:10:39.817 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rados-2:18.2.4-860.g6c6b083f.el9.x86_64 10/146 2024-09-13T18:10:39.864 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 72/146 2024-09-13T18:10:39.900 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 73/146 2024-09-13T18:10:39.959 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: libradosstriper1-2:18.2.4-860.g6c6b083f.el9.x86_ 14/146 2024-09-13T18:10:39.987 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/146 2024-09-13T18:10:40.047 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/146 2024-09-13T18:10:40.071 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 15/146 2024-09-13T18:10:40.075 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephsqlite-2:18.2.4-860.g6c6b083f.el9.x86_64 13/146 2024-09-13T18:10:40.083 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openblas-0.3.26-2.el9.x86_64 74/146 2024-09-13T18:10:40.114 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 75/146 2024-09-13T18:10:40.160 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libcephsqlite-2:18.2.4-860.g6c6b083f.el9.x86_64 13/146 2024-09-13T18:10:40.184 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libradosstriper1-2:18.2.4-860.g6c6b083f.el9.x86_ 14/146 2024-09-13T18:10:40.196 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 16/146 2024-09-13T18:10:40.199 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 76/146 2024-09-13T18:10:40.264 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 16/146 2024-09-13T18:10:40.391 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libradosstriper1-2:18.2.4-860.g6c6b083f.el9.x86_ 14/146 2024-09-13T18:10:40.531 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 15/146 2024-09-13T18:10:40.540 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 16/146 2024-09-13T18:10:40.577 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 17/146 2024-09-13T18:10:40.612 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : liboath-2.6.7-2.el9.x86_64 18/146 2024-09-13T18:10:40.654 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 16/146 2024-09-13T18:10:40.676 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/146 2024-09-13T18:10:40.711 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 20/146 2024-09-13T18:10:40.724 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 16/146 2024-09-13T18:10:40.738 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 21/146 2024-09-13T18:10:40.806 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 22/146 2024-09-13T18:10:40.845 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 77/146 2024-09-13T18:10:40.860 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 23/146 2024-09-13T18:10:40.891 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 24/146 2024-09-13T18:10:40.935 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 25/146 2024-09-13T18:10:40.964 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 78/146 2024-09-13T18:10:41.007 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 16/146 2024-09-13T18:10:41.010 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 26/146 2024-09-13T18:10:41.045 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 17/146 2024-09-13T18:10:41.045 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-ceph-common-2:18.2.4-860.g6c6b083f.el9.x 27/146 2024-09-13T18:10:41.079 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : liboath-2.6.7-2.el9.x86_64 18/146 2024-09-13T18:10:41.089 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-ceph-argparse-2:18.2.4-860.g6c6b083f.el9 28/146 2024-09-13T18:10:41.144 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/146 2024-09-13T18:10:41.164 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cephfs-2:18.2.4-860.g6c6b083f.el9.x86_64 29/146 2024-09-13T18:10:41.187 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 20/146 2024-09-13T18:10:41.215 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 21/146 2024-09-13T18:10:41.284 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 22/146 2024-09-13T18:10:41.305 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 30/146 2024-09-13T18:10:41.345 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 23/146 2024-09-13T18:10:41.358 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 31/146 2024-09-13T18:10:41.377 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 24/146 2024-09-13T18:10:41.421 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 25/146 2024-09-13T18:10:41.495 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 26/146 2024-09-13T18:10:41.533 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ceph-common-2:18.2.4-860.g6c6b083f.el9.x 27/146 2024-09-13T18:10:41.575 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ceph-argparse-2:18.2.4-860.g6c6b083f.el9 28/146 2024-09-13T18:10:41.647 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cephfs-2:18.2.4-860.g6c6b083f.el9.x86_64 29/146 2024-09-13T18:10:41.781 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 30/146 2024-09-13T18:10:41.843 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 31/146 2024-09-13T18:10:41.886 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-rsa-4.9-2.el9.noarch 32/146 2024-09-13T18:10:41.970 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 33/146 2024-09-13T18:10:42.017 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 34/146 2024-09-13T18:10:42.054 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : librados-devel-2:18.2.4-860.g6c6b083f.el9.x86_64 35/146 2024-09-13T18:10:42.096 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 36/146 2024-09-13T18:10:42.215 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 79/146 2024-09-13T18:10:42.255 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 37/146 2024-09-13T18:10:42.304 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 80/146 2024-09-13T18:10:42.321 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 38/146 2024-09-13T18:10:42.331 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 81/146 2024-09-13T18:10:42.364 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 82/146 2024-09-13T18:10:42.369 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rsa-4.9-2.el9.noarch 32/146 2024-09-13T18:10:42.436 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : re2-1:20211101-20.el9.x86_64 39/146 2024-09-13T18:10:42.447 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 33/146 2024-09-13T18:10:42.483 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 40/146 2024-09-13T18:10:42.486 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 34/146 2024-09-13T18:10:42.531 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librados-devel-2:18.2.4-860.g6c6b083f.el9.x86_64 35/146 2024-09-13T18:10:42.538 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 41/146 2024-09-13T18:10:42.582 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 36/146 2024-09-13T18:10:42.612 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 42/146 2024-09-13T18:10:42.651 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 43/146 2024-09-13T18:10:42.673 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 83/146 2024-09-13T18:10:42.687 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 44/146 2024-09-13T18:10:42.711 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : librbd1-2:18.2.4-860.g6c6b083f.el9.x86_64 84/146 2024-09-13T18:10:42.733 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 37/146 2024-09-13T18:10:42.734 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 45/146 2024-09-13T18:10:42.779 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 46/146 2024-09-13T18:10:42.790 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 38/146 2024-09-13T18:10:42.818 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librbd1-2:18.2.4-860.g6c6b083f.el9.x86_64 84/146 2024-09-13T18:10:42.821 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 47/146 2024-09-13T18:10:42.848 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rbd-2:18.2.4-860.g6c6b083f.el9.x86_64 85/146 2024-09-13T18:10:42.893 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 86/146 2024-09-13T18:10:42.898 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : re2-1:20211101-20.el9.x86_64 39/146 2024-09-13T18:10:42.936 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 40/146 2024-09-13T18:10:42.956 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 48/146 2024-09-13T18:10:42.990 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 41/146 2024-09-13T18:10:43.005 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 49/146 2024-09-13T18:10:43.055 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 50/146 2024-09-13T18:10:43.063 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 42/146 2024-09-13T18:10:43.104 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 43/146 2024-09-13T18:10:43.148 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 44/146 2024-09-13T18:10:43.161 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 51/146 2024-09-13T18:10:43.195 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 45/146 2024-09-13T18:10:43.198 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 52/146 2024-09-13T18:10:43.239 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 53/146 2024-09-13T18:10:43.240 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 46/146 2024-09-13T18:10:43.271 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 54/146 2024-09-13T18:10:43.282 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 47/146 2024-09-13T18:10:43.298 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 87/146 2024-09-13T18:10:43.298 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 55/146 2024-09-13T18:10:43.330 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librgw2-2:18.2.4-860.g6c6b083f.el9.x86_64 88/146 2024-09-13T18:10:43.331 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : fmt-8.1.1-5.el9.x86_64 56/146 2024-09-13T18:10:43.360 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 57/146 2024-09-13T18:10:43.395 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 58/146 2024-09-13T18:10:43.401 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librgw2-2:18.2.4-860.g6c6b083f.el9.x86_64 88/146 2024-09-13T18:10:43.417 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 48/146 2024-09-13T18:10:43.430 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rgw-2:18.2.4-860.g6c6b083f.el9.x86_64 89/146 2024-09-13T18:10:43.430 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : uuid-1.6.2-55.el9.x86_64 59/146 2024-09-13T18:10:43.459 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 49/146 2024-09-13T18:10:43.460 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 60/146 2024-09-13T18:10:43.487 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 61/146 2024-09-13T18:10:43.500 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 50/146 2024-09-13T18:10:43.520 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 62/146 2024-09-13T18:10:43.562 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 63/146 2024-09-13T18:10:43.596 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 64/146 2024-09-13T18:10:43.602 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 51/146 2024-09-13T18:10:43.631 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 65/146 2024-09-13T18:10:43.643 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 52/146 2024-09-13T18:10:43.678 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 66/146 2024-09-13T18:10:43.683 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 53/146 2024-09-13T18:10:43.716 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 54/146 2024-09-13T18:10:43.744 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 55/146 2024-09-13T18:10:43.777 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : fmt-8.1.1-5.el9.x86_64 56/146 2024-09-13T18:10:43.778 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-packaging-20.9-5.el9.noarch 67/146 2024-09-13T18:10:43.806 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 57/146 2024-09-13T18:10:43.833 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 68/146 2024-09-13T18:10:43.840 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 58/146 2024-09-13T18:10:43.876 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : uuid-1.6.2-55.el9.x86_64 59/146 2024-09-13T18:10:43.906 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 60/146 2024-09-13T18:10:43.940 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 61/146 2024-09-13T18:10:43.974 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 62/146 2024-09-13T18:10:44.024 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 63/146 2024-09-13T18:10:44.067 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 64/146 2024-09-13T18:10:44.102 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 65/146 2024-09-13T18:10:44.149 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 66/146 2024-09-13T18:10:44.248 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-packaging-20.9-5.el9.noarch 67/146 2024-09-13T18:10:44.329 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 68/146 2024-09-13T18:10:44.555 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 69/146 2024-09-13T18:10:44.605 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 70/146 2024-09-13T18:10:44.638 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 71/146 2024-09-13T18:10:44.739 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 72/146 2024-09-13T18:10:44.780 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 73/146 2024-09-13T18:10:44.963 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : openblas-0.3.26-2.el9.x86_64 74/146 2024-09-13T18:10:45.003 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 75/146 2024-09-13T18:10:45.042 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 69/146 2024-09-13T18:10:45.081 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 76/146 2024-09-13T18:10:45.096 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 70/146 2024-09-13T18:10:45.138 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 71/146 2024-09-13T18:10:45.235 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 72/146 2024-09-13T18:10:45.272 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 73/146 2024-09-13T18:10:45.454 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 90/146 2024-09-13T18:10:45.456 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openblas-0.3.26-2.el9.x86_64 74/146 2024-09-13T18:10:45.486 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 75/146 2024-09-13T18:10:45.493 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 90/146 2024-09-13T18:10:45.565 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 76/146 2024-09-13T18:10:45.728 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 77/146 2024-09-13T18:10:45.858 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 78/146 2024-09-13T18:10:46.019 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 90/146 2024-09-13T18:10:46.054 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-base-2:18.2.4-860.g6c6b083f.el9.x86_64 91/146 2024-09-13T18:10:46.143 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-base-2:18.2.4-860.g6c6b083f.el9.x86_64 91/146 2024-09-13T18:10:46.143 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-13T18:10:46.143 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-13T18:10:46.144 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-13T18:10:46.180 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-selinux-2:18.2.4-860.g6c6b083f.el9.x86_64 92/146 2024-09-13T18:10:46.211 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 77/146 2024-09-13T18:10:46.325 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 78/146 2024-09-13T18:10:47.114 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 79/146 2024-09-13T18:10:47.210 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 80/146 2024-09-13T18:10:47.238 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 81/146 2024-09-13T18:10:47.271 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 82/146 2024-09-13T18:10:47.568 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 79/146 2024-09-13T18:10:47.571 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 83/146 2024-09-13T18:10:47.608 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : librbd1-2:18.2.4-860.g6c6b083f.el9.x86_64 84/146 2024-09-13T18:10:47.662 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 80/146 2024-09-13T18:10:47.682 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librbd1-2:18.2.4-860.g6c6b083f.el9.x86_64 84/146 2024-09-13T18:10:47.696 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 81/146 2024-09-13T18:10:47.713 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-rbd-2:18.2.4-860.g6c6b083f.el9.x86_64 85/146 2024-09-13T18:10:47.730 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 82/146 2024-09-13T18:10:47.750 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 86/146 2024-09-13T18:10:48.029 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 83/146 2024-09-13T18:10:48.068 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : librbd1-2:18.2.4-860.g6c6b083f.el9.x86_64 84/146 2024-09-13T18:10:48.142 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librbd1-2:18.2.4-860.g6c6b083f.el9.x86_64 84/146 2024-09-13T18:10:48.153 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 87/146 2024-09-13T18:10:48.172 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rbd-2:18.2.4-860.g6c6b083f.el9.x86_64 85/146 2024-09-13T18:10:48.178 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : librgw2-2:18.2.4-860.g6c6b083f.el9.x86_64 88/146 2024-09-13T18:10:48.209 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 86/146 2024-09-13T18:10:48.249 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librgw2-2:18.2.4-860.g6c6b083f.el9.x86_64 88/146 2024-09-13T18:10:48.271 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-rgw-2:18.2.4-860.g6c6b083f.el9.x86_64 89/146 2024-09-13T18:10:48.612 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 87/146 2024-09-13T18:10:48.637 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librgw2-2:18.2.4-860.g6c6b083f.el9.x86_64 88/146 2024-09-13T18:10:48.727 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librgw2-2:18.2.4-860.g6c6b083f.el9.x86_64 88/146 2024-09-13T18:10:48.749 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rgw-2:18.2.4-860.g6c6b083f.el9.x86_64 89/146 2024-09-13T18:10:50.310 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 90/146 2024-09-13T18:10:50.343 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 90/146 2024-09-13T18:10:50.765 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 90/146 2024-09-13T18:10:50.796 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 90/146 2024-09-13T18:10:50.869 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 90/146 2024-09-13T18:10:50.905 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-base-2:18.2.4-860.g6c6b083f.el9.x86_64 91/146 2024-09-13T18:10:51.003 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-base-2:18.2.4-860.g6c6b083f.el9.x86_64 91/146 2024-09-13T18:10:51.003 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-13T18:10:51.003 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-13T18:10:51.003 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:10:51.031 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-selinux-2:18.2.4-860.g6c6b083f.el9.x86_64 92/146 2024-09-13T18:10:51.327 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 90/146 2024-09-13T18:10:51.358 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-base-2:18.2.4-860.g6c6b083f.el9.x86_64 91/146 2024-09-13T18:10:51.438 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-base-2:18.2.4-860.g6c6b083f.el9.x86_64 91/146 2024-09-13T18:10:51.438 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-13T18:10:51.438 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-13T18:10:51.438 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-13T18:10:51.476 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-selinux-2:18.2.4-860.g6c6b083f.el9.x86_64 92/146 2024-09-13T18:10:57.433 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-selinux-2:18.2.4-860.g6c6b083f.el9.x86_64 92/146 2024-09-13T18:10:57.433 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /sys 2024-09-13T18:10:57.433 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /proc 2024-09-13T18:10:57.433 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /mnt 2024-09-13T18:10:57.433 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /var/tmp 2024-09-13T18:10:57.434 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /home 2024-09-13T18:10:57.434 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /root 2024-09-13T18:10:57.434 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /tmp 2024-09-13T18:10:57.434 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-13T18:10:57.599 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-volume-2:18.2.4-860.g6c6b083f.el9.noarch 93/146 2024-09-13T18:10:57.615 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-volume-2:18.2.4-860.g6c6b083f.el9.noarch 93/146 2024-09-13T18:10:57.615 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T18:10:57.615 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-13T18:10:57.615 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-13T18:10:59.077 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-osd-2:18.2.4-860.g6c6b083f.el9.x86_64 94/146 2024-09-13T18:10:59.104 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-osd-2:18.2.4-860.g6c6b083f.el9.x86_64 94/146 2024-09-13T18:10:59.105 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T18:10:59.105 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-13T18:10:59.105 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-13T18:10:59.105 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-13T18:10:59.105 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-13T18:10:59.302 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mds-2:18.2.4-860.g6c6b083f.el9.x86_64 95/146 2024-09-13T18:10:59.326 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mds-2:18.2.4-860.g6c6b083f.el9.x86_64 95/146 2024-09-13T18:10:59.326 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T18:10:59.326 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-13T18:10:59.326 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-13T18:10:59.326 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-13T18:10:59.326 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-13T18:10:59.761 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mon-2:18.2.4-860.g6c6b083f.el9.x86_64 96/146 2024-09-13T18:10:59.785 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mon-2:18.2.4-860.g6c6b083f.el9.x86_64 96/146 2024-09-13T18:10:59.785 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T18:10:59.785 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-13T18:10:59.785 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-13T18:10:59.785 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-13T18:10:59.786 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-13T18:10:59.883 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : automake-1.16.2-8.el9.noarch 97/146 2024-09-13T18:10:59.940 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ply-3.11-14.el9.noarch 98/146 2024-09-13T18:10:59.995 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 99/146 2024-09-13T18:11:00.180 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 100/146 2024-09-13T18:11:00.228 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 101/146 2024-09-13T18:11:00.306 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 102/146 2024-09-13T18:11:00.394 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 103/146 2024-09-13T18:11:00.534 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 104/146 2024-09-13T18:11:00.578 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 105/146 2024-09-13T18:11:00.610 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 106/146 2024-09-13T18:11:00.638 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : fuse-2.9.9-16.el9.x86_64 107/146 2024-09-13T18:11:01.102 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: cephadm-2:18.2.4-860.g6c6b083f.el9.noarch 108/146 2024-09-13T18:11:01.128 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephadm-2:18.2.4-860.g6c6b083f.el9.noarch 108/146 2024-09-13T18:11:01.157 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-prometheus-alerts-2:18.2.4-860.g6c6b083f.el 109/146 2024-09-13T18:11:01.228 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-grafana-dashboards-2:18.2.4-860.g6c6b083f.e 110/146 2024-09-13T18:11:01.259 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-cephadm-2:18.2.4-860.g6c6b083f.el9.noar 111/146 2024-09-13T18:11:02.235 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-selinux-2:18.2.4-860.g6c6b083f.el9.x86_64 92/146 2024-09-13T18:11:02.235 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /sys 2024-09-13T18:11:02.235 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /proc 2024-09-13T18:11:02.235 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /mnt 2024-09-13T18:11:02.235 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /var/tmp 2024-09-13T18:11:02.235 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /home 2024-09-13T18:11:02.235 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /root 2024-09-13T18:11:02.235 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /tmp 2024-09-13T18:11:02.235 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:11:02.291 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.4-860.g6c6b083f.el9.noar 111/146 2024-09-13T18:11:02.327 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-dashboard-2:18.2.4-860.g6c6b083f.el9.no 112/146 2024-09-13T18:11:02.400 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-volume-2:18.2.4-860.g6c6b083f.el9.noarch 93/146 2024-09-13T18:11:02.416 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-volume-2:18.2.4-860.g6c6b083f.el9.noarch 93/146 2024-09-13T18:11:02.416 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T18:11:02.416 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-13T18:11:02.416 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:11:02.930 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-selinux-2:18.2.4-860.g6c6b083f.el9.x86_64 92/146 2024-09-13T18:11:02.930 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /sys 2024-09-13T18:11:02.930 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /proc 2024-09-13T18:11:02.930 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /mnt 2024-09-13T18:11:02.930 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /var/tmp 2024-09-13T18:11:02.930 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /home 2024-09-13T18:11:02.931 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /root 2024-09-13T18:11:02.931 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /tmp 2024-09-13T18:11:02.931 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-13T18:11:03.112 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-volume-2:18.2.4-860.g6c6b083f.el9.noarch 93/146 2024-09-13T18:11:03.130 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-volume-2:18.2.4-860.g6c6b083f.el9.noarch 93/146 2024-09-13T18:11:03.130 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T18:11:03.130 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-13T18:11:03.130 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-13T18:11:03.285 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.4-860.g6c6b083f.el9.no 112/146 2024-09-13T18:11:03.311 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.4-860.g6c6b 113/146 2024-09-13T18:11:03.412 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.4-860.g6c6b 113/146 2024-09-13T18:11:03.571 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-modules-core-2:18.2.4-860.g6c6b083f.el9 114/146 2024-09-13T18:11:03.596 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-2:18.2.4-860.g6c6b083f.el9.x86_64 115/146 2024-09-13T18:11:03.621 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-2:18.2.4-860.g6c6b083f.el9.x86_64 115/146 2024-09-13T18:11:03.621 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T18:11:03.621 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-13T18:11:03.621 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-13T18:11:03.621 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-13T18:11:03.621 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-13T18:11:03.672 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-rook-2:18.2.4-860.g6c6b083f.el9.noarch 116/146 2024-09-13T18:11:03.684 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.4-860.g6c6b083f.el9.noarch 116/146 2024-09-13T18:11:03.777 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-2:18.2.4-860.g6c6b083f.el9.x86_64 117/146 2024-09-13T18:11:03.832 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 118/146 2024-09-13T18:11:03.895 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-osd-2:18.2.4-860.g6c6b083f.el9.x86_64 94/146 2024-09-13T18:11:03.921 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-osd-2:18.2.4-860.g6c6b083f.el9.x86_64 94/146 2024-09-13T18:11:03.921 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T18:11:03.922 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-13T18:11:03.922 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-13T18:11:03.922 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-13T18:11:03.922 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:11:04.120 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mds-2:18.2.4-860.g6c6b083f.el9.x86_64 95/146 2024-09-13T18:11:04.143 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mds-2:18.2.4-860.g6c6b083f.el9.x86_64 95/146 2024-09-13T18:11:04.143 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T18:11:04.143 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-13T18:11:04.143 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-13T18:11:04.143 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-13T18:11:04.143 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:11:04.545 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libtool-2.4.6-46.el9.x86_64 119/146 2024-09-13T18:11:04.574 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-radosgw-2:18.2.4-860.g6c6b083f.el9.x86_64 120/146 2024-09-13T18:11:04.588 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mon-2:18.2.4-860.g6c6b083f.el9.x86_64 96/146 2024-09-13T18:11:04.591 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-osd-2:18.2.4-860.g6c6b083f.el9.x86_64 94/146 2024-09-13T18:11:04.598 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-radosgw-2:18.2.4-860.g6c6b083f.el9.x86_64 120/146 2024-09-13T18:11:04.598 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T18:11:04.598 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-13T18:11:04.598 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-13T18:11:04.598 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-13T18:11:04.598 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-13T18:11:04.610 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mon-2:18.2.4-860.g6c6b083f.el9.x86_64 96/146 2024-09-13T18:11:04.611 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T18:11:04.611 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-13T18:11:04.611 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-13T18:11:04.611 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-13T18:11:04.611 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:11:04.618 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-osd-2:18.2.4-860.g6c6b083f.el9.x86_64 94/146 2024-09-13T18:11:04.618 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T18:11:04.618 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-13T18:11:04.618 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-13T18:11:04.618 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-13T18:11:04.618 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-13T18:11:04.650 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-immutable-object-cache-2:18.2.4-860.g6c6b08 121/146 2024-09-13T18:11:04.673 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.4-860.g6c6b08 121/146 2024-09-13T18:11:04.673 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T18:11:04.673 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-13T18:11:04.673 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-13T18:11:04.718 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : automake-1.16.2-8.el9.noarch 97/146 2024-09-13T18:11:04.734 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephfs-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 122/146 2024-09-13T18:11:04.757 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: cephfs-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 122/146 2024-09-13T18:11:04.757 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T18:11:04.758 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-13T18:11:04.758 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-13T18:11:04.758 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-13T18:11:04.758 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-13T18:11:04.775 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-ply-3.11-14.el9.noarch 98/146 2024-09-13T18:11:04.825 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mds-2:18.2.4-860.g6c6b083f.el9.x86_64 95/146 2024-09-13T18:11:04.830 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 99/146 2024-09-13T18:11:04.848 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mds-2:18.2.4-860.g6c6b083f.el9.x86_64 95/146 2024-09-13T18:11:04.848 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T18:11:04.848 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-13T18:11:04.849 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-13T18:11:04.849 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-13T18:11:04.849 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-13T18:11:05.017 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 100/146 2024-09-13T18:11:05.059 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 123/146 2024-09-13T18:11:05.080 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 101/146 2024-09-13T18:11:05.083 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: rbd-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 123/146 2024-09-13T18:11:05.083 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T18:11:05.083 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-13T18:11:05.083 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-13T18:11:05.083 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-13T18:11:05.084 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-13T18:11:05.158 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 102/146 2024-09-13T18:11:05.272 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 103/146 2024-09-13T18:11:05.292 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mon-2:18.2.4-860.g6c6b083f.el9.x86_64 96/146 2024-09-13T18:11:05.315 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mon-2:18.2.4-860.g6c6b083f.el9.x86_64 96/146 2024-09-13T18:11:05.315 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T18:11:05.315 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-13T18:11:05.316 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-13T18:11:05.316 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-13T18:11:05.316 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-13T18:11:05.414 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : automake-1.16.2-8.el9.noarch 97/146 2024-09-13T18:11:05.421 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 104/146 2024-09-13T18:11:05.463 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 105/146 2024-09-13T18:11:05.470 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ply-3.11-14.el9.noarch 98/146 2024-09-13T18:11:05.495 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 106/146 2024-09-13T18:11:05.522 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : fuse-2.9.9-16.el9.x86_64 107/146 2024-09-13T18:11:05.534 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 99/146 2024-09-13T18:11:05.721 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 100/146 2024-09-13T18:11:05.776 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 101/146 2024-09-13T18:11:05.836 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: cephadm-2:18.2.4-860.g6c6b083f.el9.noarch 108/146 2024-09-13T18:11:05.845 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 102/146 2024-09-13T18:11:05.862 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : cephadm-2:18.2.4-860.g6c6b083f.el9.noarch 108/146 2024-09-13T18:11:05.891 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-prometheus-alerts-2:18.2.4-860.g6c6b083f.el 109/146 2024-09-13T18:11:05.934 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 103/146 2024-09-13T18:11:05.955 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-grafana-dashboards-2:18.2.4-860.g6c6b083f.e 110/146 2024-09-13T18:11:05.985 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-cephadm-2:18.2.4-860.g6c6b083f.el9.noar 111/146 2024-09-13T18:11:06.073 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 104/146 2024-09-13T18:11:06.110 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 105/146 2024-09-13T18:11:06.142 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 106/146 2024-09-13T18:11:06.169 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : fuse-2.9.9-16.el9.x86_64 107/146 2024-09-13T18:11:06.609 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: cephadm-2:18.2.4-860.g6c6b083f.el9.noarch 108/146 2024-09-13T18:11:06.643 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephadm-2:18.2.4-860.g6c6b083f.el9.noarch 108/146 2024-09-13T18:11:06.672 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-prometheus-alerts-2:18.2.4-860.g6c6b083f.el 109/146 2024-09-13T18:11:06.735 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-grafana-dashboards-2:18.2.4-860.g6c6b083f.e 110/146 2024-09-13T18:11:06.766 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-cephadm-2:18.2.4-860.g6c6b083f.el9.noar 111/146 2024-09-13T18:11:07.030 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.4-860.g6c6b083f.el9.noar 111/146 2024-09-13T18:11:07.062 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-dashboard-2:18.2.4-860.g6c6b083f.el9.no 112/146 2024-09-13T18:11:07.807 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.4-860.g6c6b083f.el9.noar 111/146 2024-09-13T18:11:07.843 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-dashboard-2:18.2.4-860.g6c6b083f.el9.no 112/146 2024-09-13T18:11:08.028 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.4-860.g6c6b083f.el9.no 112/146 2024-09-13T18:11:08.054 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.4-860.g6c6b 113/146 2024-09-13T18:11:08.158 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.4-860.g6c6b 113/146 2024-09-13T18:11:08.339 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-modules-core-2:18.2.4-860.g6c6b083f.el9 114/146 2024-09-13T18:11:08.382 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-2:18.2.4-860.g6c6b083f.el9.x86_64 115/146 2024-09-13T18:11:08.407 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-2:18.2.4-860.g6c6b083f.el9.x86_64 115/146 2024-09-13T18:11:08.407 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T18:11:08.407 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-13T18:11:08.407 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-13T18:11:08.407 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-13T18:11:08.407 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:11:08.457 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-rook-2:18.2.4-860.g6c6b083f.el9.noarch 116/146 2024-09-13T18:11:08.469 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.4-860.g6c6b083f.el9.noarch 116/146 2024-09-13T18:11:08.564 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-2:18.2.4-860.g6c6b083f.el9.x86_64 117/146 2024-09-13T18:11:08.623 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 118/146 2024-09-13T18:11:08.746 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-test-2:18.2.4-860.g6c6b083f.el9.x86_64 124/146 2024-09-13T18:11:08.787 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 125/146 2024-09-13T18:11:08.805 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.4-860.g6c6b083f.el9.no 112/146 2024-09-13T18:11:08.827 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.4-860.g6c6b 113/146 2024-09-13T18:11:08.827 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-nbd-2:18.2.4-860.g6c6b083f.el9.x86_64 126/146 2024-09-13T18:11:08.860 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 127/146 2024-09-13T18:11:08.927 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.4-860.g6c6b 113/146 2024-09-13T18:11:09.056 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 128/146 2024-09-13T18:11:09.090 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-modules-core-2:18.2.4-860.g6c6b083f.el9 114/146 2024-09-13T18:11:09.099 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 129/146 2024-09-13T18:11:09.120 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-2:18.2.4-860.g6c6b083f.el9.x86_64 115/146 2024-09-13T18:11:09.130 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephfs-devel-2:18.2.4-860.g6c6b083f.el9.x86_6 130/146 2024-09-13T18:11:09.144 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-2:18.2.4-860.g6c6b083f.el9.x86_64 115/146 2024-09-13T18:11:09.144 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T18:11:09.144 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-13T18:11:09.144 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-13T18:11:09.144 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-13T18:11:09.145 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-13T18:11:09.159 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 131/146 2024-09-13T18:11:09.188 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-rook-2:18.2.4-860.g6c6b083f.el9.noarch 116/146 2024-09-13T18:11:09.197 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 132/146 2024-09-13T18:11:09.199 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.4-860.g6c6b083f.el9.noarch 116/146 2024-09-13T18:11:09.293 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-2:18.2.4-860.g6c6b083f.el9.x86_64 117/146 2024-09-13T18:11:09.343 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libtool-2.4.6-46.el9.x86_64 119/146 2024-09-13T18:11:09.352 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 118/146 2024-09-13T18:11:09.376 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-radosgw-2:18.2.4-860.g6c6b083f.el9.x86_64 120/146 2024-09-13T18:11:09.400 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-radosgw-2:18.2.4-860.g6c6b083f.el9.x86_64 120/146 2024-09-13T18:11:09.400 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T18:11:09.400 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-13T18:11:09.400 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-13T18:11:09.400 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-13T18:11:09.400 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:11:09.443 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-immutable-object-cache-2:18.2.4-860.g6c6b08 121/146 2024-09-13T18:11:09.466 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.4-860.g6c6b08 121/146 2024-09-13T18:11:09.466 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T18:11:09.466 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-13T18:11:09.466 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:11:09.511 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : cephfs-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 122/146 2024-09-13T18:11:09.534 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: cephfs-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 122/146 2024-09-13T18:11:09.534 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T18:11:09.534 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-13T18:11:09.534 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-13T18:11:09.534 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-13T18:11:09.534 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:11:09.820 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : rbd-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 123/146 2024-09-13T18:11:09.823 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 133/146 2024-09-13T18:11:09.842 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: rbd-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 123/146 2024-09-13T18:11:09.843 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T18:11:09.843 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-13T18:11:09.843 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-13T18:11:09.843 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-13T18:11:09.843 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:11:09.870 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 133/146 2024-09-13T18:11:09.892 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 133/146 2024-09-13T18:11:10.068 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libtool-2.4.6-46.el9.x86_64 119/146 2024-09-13T18:11:10.099 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-radosgw-2:18.2.4-860.g6c6b083f.el9.x86_64 120/146 2024-09-13T18:11:10.122 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-radosgw-2:18.2.4-860.g6c6b083f.el9.x86_64 120/146 2024-09-13T18:11:10.122 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T18:11:10.122 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-13T18:11:10.122 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-13T18:11:10.123 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-13T18:11:10.123 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-13T18:11:10.175 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-immutable-object-cache-2:18.2.4-860.g6c6b08 121/146 2024-09-13T18:11:10.197 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.4-860.g6c6b08 121/146 2024-09-13T18:11:10.197 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T18:11:10.198 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-13T18:11:10.198 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-13T18:11:10.242 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephfs-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 122/146 2024-09-13T18:11:10.265 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: cephfs-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 122/146 2024-09-13T18:11:10.265 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T18:11:10.265 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-13T18:11:10.265 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-13T18:11:10.265 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-13T18:11:10.265 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-13T18:11:10.559 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 123/146 2024-09-13T18:11:10.582 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: rbd-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 123/146 2024-09-13T18:11:10.582 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T18:11:10.582 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-13T18:11:10.582 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-13T18:11:10.583 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-13T18:11:10.583 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-13T18:11:10.723 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephfs-top-2:18.2.4-860.g6c6b083f.el9.noarch 134/146 2024-09-13T18:11:10.800 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 135/146 2024-09-13T18:11:10.849 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 136/146 2024-09-13T18:11:10.934 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flex-2.6.4-9.el9.x86_64 137/146 2024-09-13T18:11:10.969 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : bison-3.7.4-5.el9.x86_64 138/146 2024-09-13T18:11:10.994 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 139/146 2024-09-13T18:11:11.020 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 140/146 2024-09-13T18:11:11.020 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 141/146 2024-09-13T18:11:11.104 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 141/146 2024-09-13T18:11:11.104 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 142/146 2024-09-13T18:11:11.188 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 142/146 2024-09-13T18:11:11.213 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 143/146 2024-09-13T18:11:11.219 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 144/146 2024-09-13T18:11:11.219 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 144/146 2024-09-13T18:11:11.398 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 144/146 2024-09-13T18:11:11.398 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 145/146 2024-09-13T18:11:11.524 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 145/146 2024-09-13T18:11:11.589 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 146/146 2024-09-13T18:11:13.549 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-test-2:18.2.4-860.g6c6b083f.el9.x86_64 124/146 2024-09-13T18:11:13.606 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : rbd-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 125/146 2024-09-13T18:11:13.647 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : rbd-nbd-2:18.2.4-860.g6c6b083f.el9.x86_64 126/146 2024-09-13T18:11:13.679 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 127/146 2024-09-13T18:11:13.870 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 128/146 2024-09-13T18:11:13.912 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 129/146 2024-09-13T18:11:13.943 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libcephfs-devel-2:18.2.4-860.g6c6b083f.el9.x86_6 130/146 2024-09-13T18:11:13.972 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 131/146 2024-09-13T18:11:14.010 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 132/146 2024-09-13T18:11:14.279 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-test-2:18.2.4-860.g6c6b083f.el9.x86_64 124/146 2024-09-13T18:11:14.321 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 125/146 2024-09-13T18:11:14.370 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-nbd-2:18.2.4-860.g6c6b083f.el9.x86_64 126/146 2024-09-13T18:11:14.402 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 127/146 2024-09-13T18:11:14.601 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 128/146 2024-09-13T18:11:14.610 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 133/146 2024-09-13T18:11:14.650 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 133/146 2024-09-13T18:11:14.651 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 129/146 2024-09-13T18:11:14.670 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 133/146 2024-09-13T18:11:14.681 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephfs-devel-2:18.2.4-860.g6c6b083f.el9.x86_6 130/146 2024-09-13T18:11:14.710 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 131/146 2024-09-13T18:11:14.765 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 132/146 2024-09-13T18:11:15.420 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 133/146 2024-09-13T18:11:15.455 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 133/146 2024-09-13T18:11:15.475 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 133/146 2024-09-13T18:11:15.509 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : cephfs-top-2:18.2.4-860.g6c6b083f.el9.noarch 134/146 2024-09-13T18:11:15.596 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 135/146 2024-09-13T18:11:15.646 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 136/146 2024-09-13T18:11:15.741 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : flex-2.6.4-9.el9.x86_64 137/146 2024-09-13T18:11:15.774 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : bison-3.7.4-5.el9.x86_64 138/146 2024-09-13T18:11:15.799 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 139/146 2024-09-13T18:11:15.826 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 140/146 2024-09-13T18:11:15.826 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 141/146 2024-09-13T18:11:15.901 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 141/146 2024-09-13T18:11:15.901 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 142/146 2024-09-13T18:11:15.985 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 142/146 2024-09-13T18:11:16.010 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 143/146 2024-09-13T18:11:16.016 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 144/146 2024-09-13T18:11:16.016 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 144/146 2024-09-13T18:11:16.204 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 144/146 2024-09-13T18:11:16.204 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 145/146 2024-09-13T18:11:16.297 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 145/146 2024-09-13T18:11:16.318 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephfs-top-2:18.2.4-860.g6c6b083f.el9.noarch 134/146 2024-09-13T18:11:16.371 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 146/146 2024-09-13T18:11:16.411 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 135/146 2024-09-13T18:11:16.455 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 136/146 2024-09-13T18:11:16.546 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flex-2.6.4-9.el9.x86_64 137/146 2024-09-13T18:11:16.580 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : bison-3.7.4-5.el9.x86_64 138/146 2024-09-13T18:11:16.605 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 139/146 2024-09-13T18:11:16.632 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 140/146 2024-09-13T18:11:16.632 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 141/146 2024-09-13T18:11:16.708 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 141/146 2024-09-13T18:11:16.708 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 142/146 2024-09-13T18:11:16.792 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 142/146 2024-09-13T18:11:16.817 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 143/146 2024-09-13T18:11:16.822 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 144/146 2024-09-13T18:11:16.823 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 144/146 2024-09-13T18:11:16.876 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 146/146 2024-09-13T18:11:16.876 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-2:18.2.4-860.g6c6b083f.el9.x86_64 1/146 2024-09-13T18:11:16.876 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-base-2:18.2.4-860.g6c6b083f.el9.x86_64 2/146 2024-09-13T18:11:16.876 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 3/146 2024-09-13T18:11:16.877 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 4/146 2024-09-13T18:11:16.877 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-immutable-object-cache-2:18.2.4-860.g6c6b08 5/146 2024-09-13T18:11:16.877 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mds-2:18.2.4-860.g6c6b083f.el9.x86_64 6/146 2024-09-13T18:11:16.877 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-2:18.2.4-860.g6c6b083f.el9.x86_64 7/146 2024-09-13T18:11:16.877 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mon-2:18.2.4-860.g6c6b083f.el9.x86_64 8/146 2024-09-13T18:11:16.877 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-osd-2:18.2.4-860.g6c6b083f.el9.x86_64 9/146 2024-09-13T18:11:16.877 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-radosgw-2:18.2.4-860.g6c6b083f.el9.x86_64 10/146 2024-09-13T18:11:16.877 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-selinux-2:18.2.4-860.g6c6b083f.el9.x86_64 11/146 2024-09-13T18:11:16.877 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-test-2:18.2.4-860.g6c6b083f.el9.x86_64 12/146 2024-09-13T18:11:16.877 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephfs-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 13/146 2024-09-13T18:11:16.878 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephfs-devel-2:18.2.4-860.g6c6b083f.el9.x86_6 14/146 2024-09-13T18:11:16.878 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephfs2-2:18.2.4-860.g6c6b083f.el9.x86_64 15/146 2024-09-13T18:11:16.878 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephsqlite-2:18.2.4-860.g6c6b083f.el9.x86_64 16/146 2024-09-13T18:11:16.878 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados-devel-2:18.2.4-860.g6c6b083f.el9.x86_64 17/146 2024-09-13T18:11:16.878 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libradosstriper1-2:18.2.4-860.g6c6b083f.el9.x86_ 18/146 2024-09-13T18:11:16.878 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librgw2-2:18.2.4-860.g6c6b083f.el9.x86_64 19/146 2024-09-13T18:11:16.878 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ceph-argparse-2:18.2.4-860.g6c6b083f.el9 20/146 2024-09-13T18:11:16.878 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ceph-common-2:18.2.4-860.g6c6b083f.el9.x 21/146 2024-09-13T18:11:16.878 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cephfs-2:18.2.4-860.g6c6b083f.el9.x86_64 22/146 2024-09-13T18:11:16.878 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rados-2:18.2.4-860.g6c6b083f.el9.x86_64 23/146 2024-09-13T18:11:16.878 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rbd-2:18.2.4-860.g6c6b083f.el9.x86_64 24/146 2024-09-13T18:11:16.878 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rgw-2:18.2.4-860.g6c6b083f.el9.x86_64 25/146 2024-09-13T18:11:16.878 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 26/146 2024-09-13T18:11:16.879 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 27/146 2024-09-13T18:11:16.879 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-nbd-2:18.2.4-860.g6c6b083f.el9.x86_64 28/146 2024-09-13T18:11:16.879 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-grafana-dashboards-2:18.2.4-860.g6c6b083f.e 29/146 2024-09-13T18:11:16.879 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-cephadm-2:18.2.4-860.g6c6b083f.el9.noar 30/146 2024-09-13T18:11:16.889 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-dashboard-2:18.2.4-860.g6c6b083f.el9.no 31/146 2024-09-13T18:11:16.889 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.4-860.g6c6b 32/146 2024-09-13T18:11:16.889 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-modules-core-2:18.2.4-860.g6c6b083f.el9 33/146 2024-09-13T18:11:16.890 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-rook-2:18.2.4-860.g6c6b083f.el9.noarch 34/146 2024-09-13T18:11:16.890 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-prometheus-alerts-2:18.2.4-860.g6c6b083f.el 35/146 2024-09-13T18:11:16.890 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-volume-2:18.2.4-860.g6c6b083f.el9.noarch 36/146 2024-09-13T18:11:16.890 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephadm-2:18.2.4-860.g6c6b083f.el9.noarch 37/146 2024-09-13T18:11:16.890 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephfs-top-2:18.2.4-860.g6c6b083f.el9.noarch 38/146 2024-09-13T18:11:16.890 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 39/146 2024-09-13T18:11:16.890 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 40/146 2024-09-13T18:11:16.890 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 41/146 2024-09-13T18:11:16.890 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 42/146 2024-09-13T18:11:16.890 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 43/146 2024-09-13T18:11:16.890 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ply-3.11-14.el9.noarch 44/146 2024-09-13T18:11:16.890 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 45/146 2024-09-13T18:11:16.891 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 46/146 2024-09-13T18:11:16.891 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 47/146 2024-09-13T18:11:16.891 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : automake-1.16.2-8.el9.noarch 48/146 2024-09-13T18:11:16.891 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : bison-3.7.4-5.el9.x86_64 49/146 2024-09-13T18:11:16.891 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 50/146 2024-09-13T18:11:16.891 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 51/146 2024-09-13T18:11:16.891 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flex-2.6.4-9.el9.x86_64 52/146 2024-09-13T18:11:16.891 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 53/146 2024-09-13T18:11:16.891 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 54/146 2024-09-13T18:11:16.891 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 55/146 2024-09-13T18:11:16.891 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 56/146 2024-09-13T18:11:16.891 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 57/146 2024-09-13T18:11:16.891 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 58/146 2024-09-13T18:11:16.892 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 59/146 2024-09-13T18:11:16.892 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 60/146 2024-09-13T18:11:16.892 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 61/146 2024-09-13T18:11:16.892 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 62/146 2024-09-13T18:11:16.892 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 63/146 2024-09-13T18:11:16.892 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 64/146 2024-09-13T18:11:16.892 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 65/146 2024-09-13T18:11:16.892 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 66/146 2024-09-13T18:11:16.892 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-13.16-1.el9.x86_64 67/146 2024-09-13T18:11:16.892 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 68/146 2024-09-13T18:11:16.892 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 69/146 2024-09-13T18:11:16.892 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 70/146 2024-09-13T18:11:16.892 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 71/146 2024-09-13T18:11:16.892 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 72/146 2024-09-13T18:11:16.893 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 73/146 2024-09-13T18:11:16.893 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 74/146 2024-09-13T18:11:16.893 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 75/146 2024-09-13T18:11:16.893 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 76/146 2024-09-13T18:11:16.893 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 77/146 2024-09-13T18:11:16.893 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 78/146 2024-09-13T18:11:16.893 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 79/146 2024-09-13T18:11:16.894 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 80/146 2024-09-13T18:11:16.894 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 81/146 2024-09-13T18:11:16.894 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 82/146 2024-09-13T18:11:16.894 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 83/146 2024-09-13T18:11:16.894 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 84/146 2024-09-13T18:11:16.894 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 85/146 2024-09-13T18:11:16.894 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 86/146 2024-09-13T18:11:16.894 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 87/146 2024-09-13T18:11:16.894 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 88/146 2024-09-13T18:11:16.894 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 89/146 2024-09-13T18:11:16.894 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 90/146 2024-09-13T18:11:16.894 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 91/146 2024-09-13T18:11:16.895 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 92/146 2024-09-13T18:11:16.895 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 93/146 2024-09-13T18:11:16.895 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 94/146 2024-09-13T18:11:16.895 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 95/146 2024-09-13T18:11:16.895 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 96/146 2024-09-13T18:11:16.895 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 97/146 2024-09-13T18:11:16.895 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 98/146 2024-09-13T18:11:16.895 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 99/146 2024-09-13T18:11:16.895 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 100/146 2024-09-13T18:11:16.895 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 101/146 2024-09-13T18:11:16.895 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 102/146 2024-09-13T18:11:16.895 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 103/146 2024-09-13T18:11:16.895 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 104/146 2024-09-13T18:11:16.896 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 105/146 2024-09-13T18:11:16.896 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 106/146 2024-09-13T18:11:16.896 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 107/146 2024-09-13T18:11:16.896 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 108/146 2024-09-13T18:11:16.896 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 109/146 2024-09-13T18:11:16.896 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 110/146 2024-09-13T18:11:16.896 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 111/146 2024-09-13T18:11:16.896 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 112/146 2024-09-13T18:11:16.896 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 113/146 2024-09-13T18:11:16.896 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 114/146 2024-09-13T18:11:16.896 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 115/146 2024-09-13T18:11:16.896 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 116/146 2024-09-13T18:11:16.896 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 117/146 2024-09-13T18:11:16.896 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 118/146 2024-09-13T18:11:16.897 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/146 2024-09-13T18:11:16.897 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 120/146 2024-09-13T18:11:16.897 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 121/146 2024-09-13T18:11:16.897 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 122/146 2024-09-13T18:11:16.897 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 123/146 2024-09-13T18:11:16.897 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 124/146 2024-09-13T18:11:16.897 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 125/146 2024-09-13T18:11:16.897 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 126/146 2024-09-13T18:11:16.897 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 127/146 2024-09-13T18:11:16.897 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 128/146 2024-09-13T18:11:16.897 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : re2-1:20211101-20.el9.x86_64 129/146 2024-09-13T18:11:16.898 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 130/146 2024-09-13T18:11:16.898 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados2-2:18.2.4-860.g6c6b083f.el9.x86_64 131/146 2024-09-13T18:11:16.898 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 132/146 2024-09-13T18:11:16.898 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librbd1-2:18.2.4-860.g6c6b083f.el9.x86_64 133/146 2024-09-13T18:11:16.898 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 134/146 2024-09-13T18:11:16.898 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 135/146 2024-09-13T18:11:16.898 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 136/146 2024-09-13T18:11:16.898 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 137/146 2024-09-13T18:11:16.898 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 138/146 2024-09-13T18:11:16.898 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 139/146 2024-09-13T18:11:16.898 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 140/146 2024-09-13T18:11:16.898 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 141/146 2024-09-13T18:11:16.899 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 142/146 2024-09-13T18:11:16.899 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 143/146 2024-09-13T18:11:16.899 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 144/146 2024-09-13T18:11:16.899 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 145/146 2024-09-13T18:11:17.010 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 144/146 2024-09-13T18:11:17.011 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 145/146 2024-09-13T18:11:17.095 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 145/146 2024-09-13T18:11:17.159 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 146/146 2024-09-13T18:11:19.604 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 146/146 2024-09-13T18:11:19.605 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-13T18:11:19.605 INFO:teuthology.orchestra.run.smithi134.stdout:Upgraded: 2024-09-13T18:11:19.605 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-13T18:11:19.605 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-13T18:11:19.605 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-13T18:11:19.605 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-13T18:11:19.605 INFO:teuthology.orchestra.run.smithi134.stdout: librados2-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:19.605 INFO:teuthology.orchestra.run.smithi134.stdout: librbd1-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:19.605 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-13T18:11:19.605 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-13T18:11:19.605 INFO:teuthology.orchestra.run.smithi134.stdout:Installed: 2024-09-13T18:11:19.606 INFO:teuthology.orchestra.run.smithi134.stdout: automake-1.16.2-8.el9.noarch 2024-09-13T18:11:19.606 INFO:teuthology.orchestra.run.smithi134.stdout: bison-3.7.4-5.el9.x86_64 2024-09-13T18:11:19.606 INFO:teuthology.orchestra.run.smithi134.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-13T18:11:19.606 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:19.606 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-base-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:19.606 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:19.606 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:19.606 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-grafana-dashboards-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T18:11:19.606 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-immutable-object-cache-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:19.606 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mds-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:19.606 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:19.606 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-cephadm-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T18:11:19.607 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-dashboard-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T18:11:19.607 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-diskprediction-local-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T18:11:19.607 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-modules-core-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T18:11:19.607 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-rook-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T18:11:19.607 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mon-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:19.607 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-osd-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:19.607 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-prometheus-alerts-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T18:11:19.607 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-radosgw-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:19.607 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-selinux-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:19.607 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-test-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:19.607 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-volume-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T18:11:19.607 INFO:teuthology.orchestra.run.smithi134.stdout: cephadm-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T18:11:19.607 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:19.607 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-top-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T18:11:19.608 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-13T18:11:19.608 INFO:teuthology.orchestra.run.smithi134.stdout: flex-2.6.4-9.el9.x86_64 2024-09-13T18:11:19.608 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-13T18:11:19.608 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-13T18:11:19.608 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-13T18:11:19.608 INFO:teuthology.orchestra.run.smithi134.stdout: fmt-8.1.1-5.el9.x86_64 2024-09-13T18:11:19.608 INFO:teuthology.orchestra.run.smithi134.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-13T18:11:19.608 INFO:teuthology.orchestra.run.smithi134.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-13T18:11:19.608 INFO:teuthology.orchestra.run.smithi134.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-13T18:11:19.608 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-13T18:11:19.608 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-13T18:11:19.608 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs-devel-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:19.608 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs2-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:19.609 INFO:teuthology.orchestra.run.smithi134.stdout: libcephsqlite-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:19.609 INFO:teuthology.orchestra.run.smithi134.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-13T18:11:19.609 INFO:teuthology.orchestra.run.smithi134.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-13T18:11:19.609 INFO:teuthology.orchestra.run.smithi134.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-13T18:11:19.609 INFO:teuthology.orchestra.run.smithi134.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-13T18:11:19.609 INFO:teuthology.orchestra.run.smithi134.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-13T18:11:19.609 INFO:teuthology.orchestra.run.smithi134.stdout: librados-devel-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:19.609 INFO:teuthology.orchestra.run.smithi134.stdout: libradosstriper1-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:19.609 INFO:teuthology.orchestra.run.smithi134.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-13T18:11:19.609 INFO:teuthology.orchestra.run.smithi134.stdout: librgw2-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:19.609 INFO:teuthology.orchestra.run.smithi134.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-13T18:11:19.609 INFO:teuthology.orchestra.run.smithi134.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-13T18:11:19.609 INFO:teuthology.orchestra.run.smithi134.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-13T18:11:19.610 INFO:teuthology.orchestra.run.smithi134.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-13T18:11:19.610 INFO:teuthology.orchestra.run.smithi134.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-13T18:11:19.610 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-13T18:11:19.610 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-13T18:11:19.610 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-13T18:11:19.610 INFO:teuthology.orchestra.run.smithi134.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-13T18:11:19.610 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-13.16-1.el9.x86_64 2024-09-13T18:11:19.610 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-13T18:11:19.610 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-13T18:11:19.610 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-13T18:11:19.610 INFO:teuthology.orchestra.run.smithi134.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-13T18:11:19.610 INFO:teuthology.orchestra.run.smithi134.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-13T18:11:19.610 INFO:teuthology.orchestra.run.smithi134.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-13T18:11:19.610 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-13T18:11:19.611 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-argparse-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:19.611 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:19.611 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cephfs-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:19.611 INFO:teuthology.orchestra.run.smithi134.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-13T18:11:19.611 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-13T18:11:19.611 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-13T18:11:19.611 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-13T18:11:19.611 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-13T18:11:19.611 INFO:teuthology.orchestra.run.smithi134.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-13T18:11:19.611 INFO:teuthology.orchestra.run.smithi134.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-13T18:11:19.611 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-13T18:11:19.611 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-13T18:11:19.611 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-13T18:11:19.611 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-13T18:11:19.612 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-13T18:11:19.612 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-13T18:11:19.612 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-13T18:11:19.612 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-13T18:11:19.612 INFO:teuthology.orchestra.run.smithi134.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-13T18:11:19.612 INFO:teuthology.orchestra.run.smithi134.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-13T18:11:19.612 INFO:teuthology.orchestra.run.smithi134.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-13T18:11:19.612 INFO:teuthology.orchestra.run.smithi134.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-13T18:11:19.612 INFO:teuthology.orchestra.run.smithi134.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-13T18:11:19.612 INFO:teuthology.orchestra.run.smithi134.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-13T18:11:19.612 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-13T18:11:19.612 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-13T18:11:19.612 INFO:teuthology.orchestra.run.smithi134.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-13T18:11:19.612 INFO:teuthology.orchestra.run.smithi134.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-13T18:11:19.613 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-13T18:11:19.613 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ply-3.11-14.el9.noarch 2024-09-13T18:11:19.613 INFO:teuthology.orchestra.run.smithi134.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-13T18:11:19.613 INFO:teuthology.orchestra.run.smithi134.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-13T18:11:19.613 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-13T18:11:19.613 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-13T18:11:19.613 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-13T18:11:19.613 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-13T18:11:19.613 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-13T18:11:19.613 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rados-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:19.613 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rbd-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:19.613 INFO:teuthology.orchestra.run.smithi134.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-13T18:11:19.613 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-13T18:11:19.614 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-13T18:11:19.614 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rgw-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:19.614 INFO:teuthology.orchestra.run.smithi134.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-13T18:11:19.614 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-13T18:11:19.614 INFO:teuthology.orchestra.run.smithi134.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-13T18:11:19.614 INFO:teuthology.orchestra.run.smithi134.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-13T18:11:19.614 INFO:teuthology.orchestra.run.smithi134.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-13T18:11:19.614 INFO:teuthology.orchestra.run.smithi134.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-13T18:11:19.614 INFO:teuthology.orchestra.run.smithi134.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-13T18:11:19.614 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-13T18:11:19.614 INFO:teuthology.orchestra.run.smithi134.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-13T18:11:19.614 INFO:teuthology.orchestra.run.smithi134.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-13T18:11:19.614 INFO:teuthology.orchestra.run.smithi134.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-13T18:11:19.615 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:19.615 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:19.615 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-nbd-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:19.615 INFO:teuthology.orchestra.run.smithi134.stdout: re2-1:20211101-20.el9.x86_64 2024-09-13T18:11:19.615 INFO:teuthology.orchestra.run.smithi134.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-13T18:11:19.615 INFO:teuthology.orchestra.run.smithi134.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-13T18:11:19.615 INFO:teuthology.orchestra.run.smithi134.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-13T18:11:19.615 INFO:teuthology.orchestra.run.smithi134.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-13T18:11:19.615 INFO:teuthology.orchestra.run.smithi134.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-13T18:11:19.615 INFO:teuthology.orchestra.run.smithi134.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-13T18:11:19.615 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-13T18:11:19.615 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2024-09-13T18:11:19.970 DEBUG:teuthology.parallel:result is None 2024-09-13T18:11:21.453 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 146/146 2024-09-13T18:11:21.453 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-2:18.2.4-860.g6c6b083f.el9.x86_64 1/146 2024-09-13T18:11:21.453 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-base-2:18.2.4-860.g6c6b083f.el9.x86_64 2/146 2024-09-13T18:11:21.453 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 3/146 2024-09-13T18:11:21.453 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 4/146 2024-09-13T18:11:21.453 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-immutable-object-cache-2:18.2.4-860.g6c6b08 5/146 2024-09-13T18:11:21.453 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mds-2:18.2.4-860.g6c6b083f.el9.x86_64 6/146 2024-09-13T18:11:21.453 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-2:18.2.4-860.g6c6b083f.el9.x86_64 7/146 2024-09-13T18:11:21.453 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mon-2:18.2.4-860.g6c6b083f.el9.x86_64 8/146 2024-09-13T18:11:21.453 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-osd-2:18.2.4-860.g6c6b083f.el9.x86_64 9/146 2024-09-13T18:11:21.453 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-radosgw-2:18.2.4-860.g6c6b083f.el9.x86_64 10/146 2024-09-13T18:11:21.454 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-selinux-2:18.2.4-860.g6c6b083f.el9.x86_64 11/146 2024-09-13T18:11:21.454 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-test-2:18.2.4-860.g6c6b083f.el9.x86_64 12/146 2024-09-13T18:11:21.454 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : cephfs-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 13/146 2024-09-13T18:11:21.454 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libcephfs-devel-2:18.2.4-860.g6c6b083f.el9.x86_6 14/146 2024-09-13T18:11:21.454 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libcephfs2-2:18.2.4-860.g6c6b083f.el9.x86_64 15/146 2024-09-13T18:11:21.454 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libcephsqlite-2:18.2.4-860.g6c6b083f.el9.x86_64 16/146 2024-09-13T18:11:21.454 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librados-devel-2:18.2.4-860.g6c6b083f.el9.x86_64 17/146 2024-09-13T18:11:21.454 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libradosstriper1-2:18.2.4-860.g6c6b083f.el9.x86_ 18/146 2024-09-13T18:11:21.454 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librgw2-2:18.2.4-860.g6c6b083f.el9.x86_64 19/146 2024-09-13T18:11:21.454 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-ceph-argparse-2:18.2.4-860.g6c6b083f.el9 20/146 2024-09-13T18:11:21.454 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-ceph-common-2:18.2.4-860.g6c6b083f.el9.x 21/146 2024-09-13T18:11:21.454 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cephfs-2:18.2.4-860.g6c6b083f.el9.x86_64 22/146 2024-09-13T18:11:21.454 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-rados-2:18.2.4-860.g6c6b083f.el9.x86_64 23/146 2024-09-13T18:11:21.455 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-rbd-2:18.2.4-860.g6c6b083f.el9.x86_64 24/146 2024-09-13T18:11:21.455 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-rgw-2:18.2.4-860.g6c6b083f.el9.x86_64 25/146 2024-09-13T18:11:21.455 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : rbd-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 26/146 2024-09-13T18:11:21.455 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : rbd-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 27/146 2024-09-13T18:11:21.455 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : rbd-nbd-2:18.2.4-860.g6c6b083f.el9.x86_64 28/146 2024-09-13T18:11:21.455 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-grafana-dashboards-2:18.2.4-860.g6c6b083f.e 29/146 2024-09-13T18:11:21.455 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-cephadm-2:18.2.4-860.g6c6b083f.el9.noar 30/146 2024-09-13T18:11:21.465 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-dashboard-2:18.2.4-860.g6c6b083f.el9.no 31/146 2024-09-13T18:11:21.466 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.4-860.g6c6b 32/146 2024-09-13T18:11:21.466 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-modules-core-2:18.2.4-860.g6c6b083f.el9 33/146 2024-09-13T18:11:21.466 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-rook-2:18.2.4-860.g6c6b083f.el9.noarch 34/146 2024-09-13T18:11:21.466 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-prometheus-alerts-2:18.2.4-860.g6c6b083f.el 35/146 2024-09-13T18:11:21.466 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-volume-2:18.2.4-860.g6c6b083f.el9.noarch 36/146 2024-09-13T18:11:21.466 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : cephadm-2:18.2.4-860.g6c6b083f.el9.noarch 37/146 2024-09-13T18:11:21.466 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : cephfs-top-2:18.2.4-860.g6c6b083f.el9.noarch 38/146 2024-09-13T18:11:21.466 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 39/146 2024-09-13T18:11:21.466 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 40/146 2024-09-13T18:11:21.466 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 41/146 2024-09-13T18:11:21.466 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 42/146 2024-09-13T18:11:21.466 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 43/146 2024-09-13T18:11:21.466 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-ply-3.11-14.el9.noarch 44/146 2024-09-13T18:11:21.467 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 45/146 2024-09-13T18:11:21.467 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 46/146 2024-09-13T18:11:21.467 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 47/146 2024-09-13T18:11:21.467 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : automake-1.16.2-8.el9.noarch 48/146 2024-09-13T18:11:21.467 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : bison-3.7.4-5.el9.x86_64 49/146 2024-09-13T18:11:21.467 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 50/146 2024-09-13T18:11:21.467 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 51/146 2024-09-13T18:11:21.467 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : flex-2.6.4-9.el9.x86_64 52/146 2024-09-13T18:11:21.467 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 53/146 2024-09-13T18:11:21.467 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 54/146 2024-09-13T18:11:21.467 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 55/146 2024-09-13T18:11:21.467 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 56/146 2024-09-13T18:11:21.467 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 57/146 2024-09-13T18:11:21.468 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 58/146 2024-09-13T18:11:21.468 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 59/146 2024-09-13T18:11:21.468 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 60/146 2024-09-13T18:11:21.468 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 61/146 2024-09-13T18:11:21.468 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 62/146 2024-09-13T18:11:21.468 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 63/146 2024-09-13T18:11:21.468 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 64/146 2024-09-13T18:11:21.468 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 65/146 2024-09-13T18:11:21.468 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 66/146 2024-09-13T18:11:21.468 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : postgresql-13.16-1.el9.x86_64 67/146 2024-09-13T18:11:21.468 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 68/146 2024-09-13T18:11:21.468 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 69/146 2024-09-13T18:11:21.468 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 70/146 2024-09-13T18:11:21.468 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 71/146 2024-09-13T18:11:21.469 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 72/146 2024-09-13T18:11:21.469 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 73/146 2024-09-13T18:11:21.469 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 74/146 2024-09-13T18:11:21.469 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 75/146 2024-09-13T18:11:21.469 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 76/146 2024-09-13T18:11:21.469 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 77/146 2024-09-13T18:11:21.469 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 78/146 2024-09-13T18:11:21.469 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 79/146 2024-09-13T18:11:21.469 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 80/146 2024-09-13T18:11:21.469 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 81/146 2024-09-13T18:11:21.469 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 82/146 2024-09-13T18:11:21.469 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 83/146 2024-09-13T18:11:21.469 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 84/146 2024-09-13T18:11:21.470 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 85/146 2024-09-13T18:11:21.470 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 86/146 2024-09-13T18:11:21.470 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 87/146 2024-09-13T18:11:21.470 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 88/146 2024-09-13T18:11:21.470 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 89/146 2024-09-13T18:11:21.470 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 90/146 2024-09-13T18:11:21.470 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 91/146 2024-09-13T18:11:21.470 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 92/146 2024-09-13T18:11:21.470 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 93/146 2024-09-13T18:11:21.470 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 94/146 2024-09-13T18:11:21.470 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 95/146 2024-09-13T18:11:21.470 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 96/146 2024-09-13T18:11:21.470 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 97/146 2024-09-13T18:11:21.470 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 98/146 2024-09-13T18:11:21.470 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 99/146 2024-09-13T18:11:21.471 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 100/146 2024-09-13T18:11:21.471 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 101/146 2024-09-13T18:11:21.471 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 102/146 2024-09-13T18:11:21.471 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 103/146 2024-09-13T18:11:21.471 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 104/146 2024-09-13T18:11:21.471 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 105/146 2024-09-13T18:11:21.471 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 106/146 2024-09-13T18:11:21.471 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 107/146 2024-09-13T18:11:21.471 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 108/146 2024-09-13T18:11:21.471 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 109/146 2024-09-13T18:11:21.471 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 110/146 2024-09-13T18:11:21.471 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 111/146 2024-09-13T18:11:21.472 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 112/146 2024-09-13T18:11:21.472 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 113/146 2024-09-13T18:11:21.472 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 114/146 2024-09-13T18:11:21.472 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 115/146 2024-09-13T18:11:21.472 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 116/146 2024-09-13T18:11:21.472 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 117/146 2024-09-13T18:11:21.472 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 118/146 2024-09-13T18:11:21.472 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/146 2024-09-13T18:11:21.472 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 120/146 2024-09-13T18:11:21.472 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 121/146 2024-09-13T18:11:21.472 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 122/146 2024-09-13T18:11:21.472 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 123/146 2024-09-13T18:11:21.472 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 124/146 2024-09-13T18:11:21.472 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 125/146 2024-09-13T18:11:21.473 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 126/146 2024-09-13T18:11:21.473 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 127/146 2024-09-13T18:11:21.473 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 128/146 2024-09-13T18:11:21.473 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : re2-1:20211101-20.el9.x86_64 129/146 2024-09-13T18:11:21.473 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 130/146 2024-09-13T18:11:21.473 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librados2-2:18.2.4-860.g6c6b083f.el9.x86_64 131/146 2024-09-13T18:11:21.473 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 132/146 2024-09-13T18:11:21.473 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librbd1-2:18.2.4-860.g6c6b083f.el9.x86_64 133/146 2024-09-13T18:11:21.473 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 134/146 2024-09-13T18:11:21.473 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 135/146 2024-09-13T18:11:21.473 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 136/146 2024-09-13T18:11:21.473 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 137/146 2024-09-13T18:11:21.473 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 138/146 2024-09-13T18:11:21.474 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 139/146 2024-09-13T18:11:21.474 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 140/146 2024-09-13T18:11:21.474 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 141/146 2024-09-13T18:11:21.474 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 142/146 2024-09-13T18:11:21.474 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 143/146 2024-09-13T18:11:21.474 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 144/146 2024-09-13T18:11:21.474 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 145/146 2024-09-13T18:11:22.587 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 146/146 2024-09-13T18:11:22.588 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-2:18.2.4-860.g6c6b083f.el9.x86_64 1/146 2024-09-13T18:11:22.588 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-base-2:18.2.4-860.g6c6b083f.el9.x86_64 2/146 2024-09-13T18:11:22.588 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 3/146 2024-09-13T18:11:22.588 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 4/146 2024-09-13T18:11:22.588 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-immutable-object-cache-2:18.2.4-860.g6c6b08 5/146 2024-09-13T18:11:22.588 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mds-2:18.2.4-860.g6c6b083f.el9.x86_64 6/146 2024-09-13T18:11:22.588 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-2:18.2.4-860.g6c6b083f.el9.x86_64 7/146 2024-09-13T18:11:22.588 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mon-2:18.2.4-860.g6c6b083f.el9.x86_64 8/146 2024-09-13T18:11:22.588 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-osd-2:18.2.4-860.g6c6b083f.el9.x86_64 9/146 2024-09-13T18:11:22.588 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-radosgw-2:18.2.4-860.g6c6b083f.el9.x86_64 10/146 2024-09-13T18:11:22.588 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-selinux-2:18.2.4-860.g6c6b083f.el9.x86_64 11/146 2024-09-13T18:11:22.588 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-test-2:18.2.4-860.g6c6b083f.el9.x86_64 12/146 2024-09-13T18:11:22.588 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephfs-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 13/146 2024-09-13T18:11:22.589 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephfs-devel-2:18.2.4-860.g6c6b083f.el9.x86_6 14/146 2024-09-13T18:11:22.589 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephfs2-2:18.2.4-860.g6c6b083f.el9.x86_64 15/146 2024-09-13T18:11:22.589 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephsqlite-2:18.2.4-860.g6c6b083f.el9.x86_64 16/146 2024-09-13T18:11:22.589 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados-devel-2:18.2.4-860.g6c6b083f.el9.x86_64 17/146 2024-09-13T18:11:22.589 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libradosstriper1-2:18.2.4-860.g6c6b083f.el9.x86_ 18/146 2024-09-13T18:11:22.589 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librgw2-2:18.2.4-860.g6c6b083f.el9.x86_64 19/146 2024-09-13T18:11:22.589 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ceph-argparse-2:18.2.4-860.g6c6b083f.el9 20/146 2024-09-13T18:11:22.589 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ceph-common-2:18.2.4-860.g6c6b083f.el9.x 21/146 2024-09-13T18:11:22.589 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cephfs-2:18.2.4-860.g6c6b083f.el9.x86_64 22/146 2024-09-13T18:11:22.589 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rados-2:18.2.4-860.g6c6b083f.el9.x86_64 23/146 2024-09-13T18:11:22.589 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rbd-2:18.2.4-860.g6c6b083f.el9.x86_64 24/146 2024-09-13T18:11:22.589 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rgw-2:18.2.4-860.g6c6b083f.el9.x86_64 25/146 2024-09-13T18:11:22.589 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 26/146 2024-09-13T18:11:22.589 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 27/146 2024-09-13T18:11:22.590 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-nbd-2:18.2.4-860.g6c6b083f.el9.x86_64 28/146 2024-09-13T18:11:22.590 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-grafana-dashboards-2:18.2.4-860.g6c6b083f.e 29/146 2024-09-13T18:11:22.590 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-cephadm-2:18.2.4-860.g6c6b083f.el9.noar 30/146 2024-09-13T18:11:22.600 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-dashboard-2:18.2.4-860.g6c6b083f.el9.no 31/146 2024-09-13T18:11:22.600 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.4-860.g6c6b 32/146 2024-09-13T18:11:22.600 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-modules-core-2:18.2.4-860.g6c6b083f.el9 33/146 2024-09-13T18:11:22.600 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-rook-2:18.2.4-860.g6c6b083f.el9.noarch 34/146 2024-09-13T18:11:22.600 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-prometheus-alerts-2:18.2.4-860.g6c6b083f.el 35/146 2024-09-13T18:11:22.600 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-volume-2:18.2.4-860.g6c6b083f.el9.noarch 36/146 2024-09-13T18:11:22.600 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephadm-2:18.2.4-860.g6c6b083f.el9.noarch 37/146 2024-09-13T18:11:22.600 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephfs-top-2:18.2.4-860.g6c6b083f.el9.noarch 38/146 2024-09-13T18:11:22.600 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 39/146 2024-09-13T18:11:22.600 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 40/146 2024-09-13T18:11:22.600 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 41/146 2024-09-13T18:11:22.600 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 42/146 2024-09-13T18:11:22.601 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 43/146 2024-09-13T18:11:22.601 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ply-3.11-14.el9.noarch 44/146 2024-09-13T18:11:22.601 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 45/146 2024-09-13T18:11:22.601 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 46/146 2024-09-13T18:11:22.601 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 47/146 2024-09-13T18:11:22.601 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : automake-1.16.2-8.el9.noarch 48/146 2024-09-13T18:11:22.601 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : bison-3.7.4-5.el9.x86_64 49/146 2024-09-13T18:11:22.601 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 50/146 2024-09-13T18:11:22.601 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 51/146 2024-09-13T18:11:22.601 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flex-2.6.4-9.el9.x86_64 52/146 2024-09-13T18:11:22.601 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 53/146 2024-09-13T18:11:22.601 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 54/146 2024-09-13T18:11:22.601 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 55/146 2024-09-13T18:11:22.602 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 56/146 2024-09-13T18:11:22.602 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 57/146 2024-09-13T18:11:22.602 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 58/146 2024-09-13T18:11:22.602 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 59/146 2024-09-13T18:11:22.602 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 60/146 2024-09-13T18:11:22.602 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 61/146 2024-09-13T18:11:22.602 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 62/146 2024-09-13T18:11:22.602 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 63/146 2024-09-13T18:11:22.602 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 64/146 2024-09-13T18:11:22.602 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 65/146 2024-09-13T18:11:22.602 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 66/146 2024-09-13T18:11:22.602 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-13.16-1.el9.x86_64 67/146 2024-09-13T18:11:22.602 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 68/146 2024-09-13T18:11:22.602 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 69/146 2024-09-13T18:11:22.603 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 70/146 2024-09-13T18:11:22.603 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 71/146 2024-09-13T18:11:22.603 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 72/146 2024-09-13T18:11:22.603 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 73/146 2024-09-13T18:11:22.603 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 74/146 2024-09-13T18:11:22.603 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 75/146 2024-09-13T18:11:22.603 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 76/146 2024-09-13T18:11:22.603 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 77/146 2024-09-13T18:11:22.603 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 78/146 2024-09-13T18:11:22.603 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 79/146 2024-09-13T18:11:22.603 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 80/146 2024-09-13T18:11:22.603 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 81/146 2024-09-13T18:11:22.603 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 82/146 2024-09-13T18:11:22.603 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 83/146 2024-09-13T18:11:22.604 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 84/146 2024-09-13T18:11:22.604 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 85/146 2024-09-13T18:11:22.604 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 86/146 2024-09-13T18:11:22.604 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 87/146 2024-09-13T18:11:22.604 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 88/146 2024-09-13T18:11:22.604 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 89/146 2024-09-13T18:11:22.604 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 90/146 2024-09-13T18:11:22.604 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 91/146 2024-09-13T18:11:22.604 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 92/146 2024-09-13T18:11:22.604 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 93/146 2024-09-13T18:11:22.604 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 94/146 2024-09-13T18:11:22.604 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 95/146 2024-09-13T18:11:22.604 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 96/146 2024-09-13T18:11:22.604 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 97/146 2024-09-13T18:11:22.605 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 98/146 2024-09-13T18:11:22.605 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 99/146 2024-09-13T18:11:22.605 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 100/146 2024-09-13T18:11:22.605 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 101/146 2024-09-13T18:11:22.605 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 102/146 2024-09-13T18:11:22.605 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 103/146 2024-09-13T18:11:22.605 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 104/146 2024-09-13T18:11:22.605 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 105/146 2024-09-13T18:11:22.605 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 106/146 2024-09-13T18:11:22.605 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 107/146 2024-09-13T18:11:22.605 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 108/146 2024-09-13T18:11:22.605 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 109/146 2024-09-13T18:11:22.605 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 110/146 2024-09-13T18:11:22.605 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 111/146 2024-09-13T18:11:22.606 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 112/146 2024-09-13T18:11:22.606 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 113/146 2024-09-13T18:11:22.606 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 114/146 2024-09-13T18:11:22.606 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 115/146 2024-09-13T18:11:22.606 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 116/146 2024-09-13T18:11:22.606 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 117/146 2024-09-13T18:11:22.606 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 118/146 2024-09-13T18:11:22.606 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/146 2024-09-13T18:11:22.606 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 120/146 2024-09-13T18:11:22.606 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 121/146 2024-09-13T18:11:22.606 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 122/146 2024-09-13T18:11:22.606 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 123/146 2024-09-13T18:11:22.606 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 124/146 2024-09-13T18:11:22.606 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 125/146 2024-09-13T18:11:22.607 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 126/146 2024-09-13T18:11:22.607 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 127/146 2024-09-13T18:11:22.607 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 128/146 2024-09-13T18:11:22.607 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : re2-1:20211101-20.el9.x86_64 129/146 2024-09-13T18:11:22.607 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 130/146 2024-09-13T18:11:22.607 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados2-2:18.2.4-860.g6c6b083f.el9.x86_64 131/146 2024-09-13T18:11:22.607 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 132/146 2024-09-13T18:11:22.607 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librbd1-2:18.2.4-860.g6c6b083f.el9.x86_64 133/146 2024-09-13T18:11:22.607 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 134/146 2024-09-13T18:11:22.607 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 135/146 2024-09-13T18:11:22.607 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 136/146 2024-09-13T18:11:22.607 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 137/146 2024-09-13T18:11:22.607 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 138/146 2024-09-13T18:11:22.607 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 139/146 2024-09-13T18:11:22.607 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 140/146 2024-09-13T18:11:22.608 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 141/146 2024-09-13T18:11:22.608 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 142/146 2024-09-13T18:11:22.608 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 143/146 2024-09-13T18:11:22.608 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 144/146 2024-09-13T18:11:22.608 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 145/146 2024-09-13T18:11:24.290 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 146/146 2024-09-13T18:11:24.290 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:11:24.290 INFO:teuthology.orchestra.run.smithi114.stdout:Upgraded: 2024-09-13T18:11:24.290 INFO:teuthology.orchestra.run.smithi114.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-13T18:11:24.290 INFO:teuthology.orchestra.run.smithi114.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-13T18:11:24.291 INFO:teuthology.orchestra.run.smithi114.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-13T18:11:24.291 INFO:teuthology.orchestra.run.smithi114.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-13T18:11:24.291 INFO:teuthology.orchestra.run.smithi114.stdout: librados2-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:24.291 INFO:teuthology.orchestra.run.smithi114.stdout: librbd1-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:24.291 INFO:teuthology.orchestra.run.smithi114.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-13T18:11:24.291 INFO:teuthology.orchestra.run.smithi114.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-13T18:11:24.291 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-09-13T18:11:24.291 INFO:teuthology.orchestra.run.smithi114.stdout: automake-1.16.2-8.el9.noarch 2024-09-13T18:11:24.291 INFO:teuthology.orchestra.run.smithi114.stdout: bison-3.7.4-5.el9.x86_64 2024-09-13T18:11:24.291 INFO:teuthology.orchestra.run.smithi114.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-13T18:11:24.291 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:24.291 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-base-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:24.291 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:24.292 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:24.292 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-grafana-dashboards-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T18:11:24.292 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-immutable-object-cache-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:24.292 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mds-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:24.292 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:24.292 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-cephadm-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T18:11:24.292 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-dashboard-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T18:11:24.292 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-diskprediction-local-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T18:11:24.292 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-modules-core-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T18:11:24.292 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-rook-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T18:11:24.292 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mon-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:24.292 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-osd-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:24.292 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-prometheus-alerts-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T18:11:24.293 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-radosgw-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:24.293 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-selinux-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:24.293 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-test-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:24.293 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-volume-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T18:11:24.293 INFO:teuthology.orchestra.run.smithi114.stdout: cephadm-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T18:11:24.293 INFO:teuthology.orchestra.run.smithi114.stdout: cephfs-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:24.293 INFO:teuthology.orchestra.run.smithi114.stdout: cephfs-top-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T18:11:24.293 INFO:teuthology.orchestra.run.smithi114.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-13T18:11:24.293 INFO:teuthology.orchestra.run.smithi114.stdout: flex-2.6.4-9.el9.x86_64 2024-09-13T18:11:24.293 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-13T18:11:24.293 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-13T18:11:24.293 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-13T18:11:24.294 INFO:teuthology.orchestra.run.smithi114.stdout: fmt-8.1.1-5.el9.x86_64 2024-09-13T18:11:24.294 INFO:teuthology.orchestra.run.smithi114.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-13T18:11:24.294 INFO:teuthology.orchestra.run.smithi114.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-13T18:11:24.294 INFO:teuthology.orchestra.run.smithi114.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-13T18:11:24.294 INFO:teuthology.orchestra.run.smithi114.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-13T18:11:24.294 INFO:teuthology.orchestra.run.smithi114.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-13T18:11:24.294 INFO:teuthology.orchestra.run.smithi114.stdout: libcephfs-devel-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:24.294 INFO:teuthology.orchestra.run.smithi114.stdout: libcephfs2-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:24.294 INFO:teuthology.orchestra.run.smithi114.stdout: libcephsqlite-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:24.294 INFO:teuthology.orchestra.run.smithi114.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-13T18:11:24.294 INFO:teuthology.orchestra.run.smithi114.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-13T18:11:24.294 INFO:teuthology.orchestra.run.smithi114.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-13T18:11:24.294 INFO:teuthology.orchestra.run.smithi114.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-13T18:11:24.295 INFO:teuthology.orchestra.run.smithi114.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-13T18:11:24.295 INFO:teuthology.orchestra.run.smithi114.stdout: librados-devel-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:24.295 INFO:teuthology.orchestra.run.smithi114.stdout: libradosstriper1-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:24.295 INFO:teuthology.orchestra.run.smithi114.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-13T18:11:24.295 INFO:teuthology.orchestra.run.smithi114.stdout: librgw2-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:24.295 INFO:teuthology.orchestra.run.smithi114.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-13T18:11:24.295 INFO:teuthology.orchestra.run.smithi114.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-13T18:11:24.295 INFO:teuthology.orchestra.run.smithi114.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-13T18:11:24.295 INFO:teuthology.orchestra.run.smithi114.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-13T18:11:24.295 INFO:teuthology.orchestra.run.smithi114.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-13T18:11:24.295 INFO:teuthology.orchestra.run.smithi114.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-13T18:11:24.295 INFO:teuthology.orchestra.run.smithi114.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-13T18:11:24.295 INFO:teuthology.orchestra.run.smithi114.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-13T18:11:24.295 INFO:teuthology.orchestra.run.smithi114.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-13T18:11:24.296 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql-13.16-1.el9.x86_64 2024-09-13T18:11:24.296 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-13T18:11:24.296 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-13T18:11:24.296 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-13T18:11:24.296 INFO:teuthology.orchestra.run.smithi114.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-13T18:11:24.296 INFO:teuthology.orchestra.run.smithi114.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-13T18:11:24.296 INFO:teuthology.orchestra.run.smithi114.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-13T18:11:24.296 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-13T18:11:24.296 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ceph-argparse-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:24.296 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:24.296 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cephfs-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:24.296 INFO:teuthology.orchestra.run.smithi114.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-13T18:11:24.296 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-13T18:11:24.297 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-13T18:11:24.297 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-13T18:11:24.297 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-13T18:11:24.297 INFO:teuthology.orchestra.run.smithi114.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-13T18:11:24.297 INFO:teuthology.orchestra.run.smithi114.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-13T18:11:24.297 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-13T18:11:24.297 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-13T18:11:24.297 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-13T18:11:24.297 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-13T18:11:24.297 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-13T18:11:24.297 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-13T18:11:24.297 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-13T18:11:24.297 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-13T18:11:24.297 INFO:teuthology.orchestra.run.smithi114.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-13T18:11:24.298 INFO:teuthology.orchestra.run.smithi114.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-13T18:11:24.298 INFO:teuthology.orchestra.run.smithi114.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-13T18:11:24.298 INFO:teuthology.orchestra.run.smithi114.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-13T18:11:24.298 INFO:teuthology.orchestra.run.smithi114.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-13T18:11:24.298 INFO:teuthology.orchestra.run.smithi114.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-13T18:11:24.298 INFO:teuthology.orchestra.run.smithi114.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-13T18:11:24.298 INFO:teuthology.orchestra.run.smithi114.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-13T18:11:24.298 INFO:teuthology.orchestra.run.smithi114.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-13T18:11:24.298 INFO:teuthology.orchestra.run.smithi114.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-13T18:11:24.298 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-13T18:11:24.298 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ply-3.11-14.el9.noarch 2024-09-13T18:11:24.298 INFO:teuthology.orchestra.run.smithi114.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-13T18:11:24.298 INFO:teuthology.orchestra.run.smithi114.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-13T18:11:24.299 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-13T18:11:24.299 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-13T18:11:24.299 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-13T18:11:24.299 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-13T18:11:24.299 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-13T18:11:24.299 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rados-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:24.299 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rbd-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:24.299 INFO:teuthology.orchestra.run.smithi114.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-13T18:11:24.299 INFO:teuthology.orchestra.run.smithi114.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-13T18:11:24.299 INFO:teuthology.orchestra.run.smithi114.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-13T18:11:24.299 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rgw-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:24.299 INFO:teuthology.orchestra.run.smithi114.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-13T18:11:24.299 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-13T18:11:24.299 INFO:teuthology.orchestra.run.smithi114.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-13T18:11:24.300 INFO:teuthology.orchestra.run.smithi114.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-13T18:11:24.300 INFO:teuthology.orchestra.run.smithi114.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-13T18:11:24.300 INFO:teuthology.orchestra.run.smithi114.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-13T18:11:24.300 INFO:teuthology.orchestra.run.smithi114.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-13T18:11:24.300 INFO:teuthology.orchestra.run.smithi114.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-13T18:11:24.300 INFO:teuthology.orchestra.run.smithi114.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-13T18:11:24.300 INFO:teuthology.orchestra.run.smithi114.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-13T18:11:24.300 INFO:teuthology.orchestra.run.smithi114.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-13T18:11:24.300 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:24.300 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:24.300 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-nbd-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:24.300 INFO:teuthology.orchestra.run.smithi114.stdout: re2-1:20211101-20.el9.x86_64 2024-09-13T18:11:24.300 INFO:teuthology.orchestra.run.smithi114.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-13T18:11:24.300 INFO:teuthology.orchestra.run.smithi114.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-13T18:11:24.301 INFO:teuthology.orchestra.run.smithi114.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-13T18:11:24.301 INFO:teuthology.orchestra.run.smithi114.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-13T18:11:24.301 INFO:teuthology.orchestra.run.smithi114.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-13T18:11:24.301 INFO:teuthology.orchestra.run.smithi114.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-13T18:11:24.301 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:11:24.301 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-13T18:11:24.676 DEBUG:teuthology.parallel:result is None 2024-09-13T18:11:25.097 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 146/146 2024-09-13T18:11:25.097 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-13T18:11:25.097 INFO:teuthology.orchestra.run.smithi204.stdout:Upgraded: 2024-09-13T18:11:25.097 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-13T18:11:25.097 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-13T18:11:25.097 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-13T18:11:25.097 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-13T18:11:25.098 INFO:teuthology.orchestra.run.smithi204.stdout: librados2-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:25.098 INFO:teuthology.orchestra.run.smithi204.stdout: librbd1-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:25.098 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-13T18:11:25.098 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-13T18:11:25.098 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-09-13T18:11:25.098 INFO:teuthology.orchestra.run.smithi204.stdout: automake-1.16.2-8.el9.noarch 2024-09-13T18:11:25.098 INFO:teuthology.orchestra.run.smithi204.stdout: bison-3.7.4-5.el9.x86_64 2024-09-13T18:11:25.098 INFO:teuthology.orchestra.run.smithi204.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-13T18:11:25.098 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:25.098 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-base-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:25.098 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:25.098 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:25.098 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-grafana-dashboards-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T18:11:25.099 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-immutable-object-cache-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:25.099 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mds-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:25.099 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:25.099 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-cephadm-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T18:11:25.099 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-dashboard-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T18:11:25.099 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-diskprediction-local-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T18:11:25.099 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-modules-core-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T18:11:25.099 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-rook-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T18:11:25.099 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mon-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:25.099 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-osd-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:25.099 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-prometheus-alerts-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T18:11:25.099 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-radosgw-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:25.099 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-selinux-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:25.100 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-test-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:25.100 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-volume-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T18:11:25.100 INFO:teuthology.orchestra.run.smithi204.stdout: cephadm-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T18:11:25.100 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:25.100 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-top-2:18.2.4-860.g6c6b083f.el9.noarch 2024-09-13T18:11:25.100 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-13T18:11:25.100 INFO:teuthology.orchestra.run.smithi204.stdout: flex-2.6.4-9.el9.x86_64 2024-09-13T18:11:25.100 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-13T18:11:25.100 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-13T18:11:25.100 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-13T18:11:25.100 INFO:teuthology.orchestra.run.smithi204.stdout: fmt-8.1.1-5.el9.x86_64 2024-09-13T18:11:25.100 INFO:teuthology.orchestra.run.smithi204.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-13T18:11:25.100 INFO:teuthology.orchestra.run.smithi204.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-13T18:11:25.101 INFO:teuthology.orchestra.run.smithi204.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-13T18:11:25.101 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-13T18:11:25.101 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-13T18:11:25.101 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs-devel-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:25.101 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs2-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:25.101 INFO:teuthology.orchestra.run.smithi204.stdout: libcephsqlite-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:25.101 INFO:teuthology.orchestra.run.smithi204.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-13T18:11:25.101 INFO:teuthology.orchestra.run.smithi204.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-13T18:11:25.101 INFO:teuthology.orchestra.run.smithi204.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-13T18:11:25.101 INFO:teuthology.orchestra.run.smithi204.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-13T18:11:25.101 INFO:teuthology.orchestra.run.smithi204.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-13T18:11:25.101 INFO:teuthology.orchestra.run.smithi204.stdout: librados-devel-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:25.101 INFO:teuthology.orchestra.run.smithi204.stdout: libradosstriper1-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:25.102 INFO:teuthology.orchestra.run.smithi204.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-13T18:11:25.102 INFO:teuthology.orchestra.run.smithi204.stdout: librgw2-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:25.102 INFO:teuthology.orchestra.run.smithi204.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-13T18:11:25.102 INFO:teuthology.orchestra.run.smithi204.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-13T18:11:25.103 INFO:teuthology.orchestra.run.smithi204.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-13T18:11:25.103 INFO:teuthology.orchestra.run.smithi204.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-13T18:11:25.103 INFO:teuthology.orchestra.run.smithi204.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-13T18:11:25.103 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-13T18:11:25.103 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-13T18:11:25.103 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-13T18:11:25.103 INFO:teuthology.orchestra.run.smithi204.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-13T18:11:25.103 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-13.16-1.el9.x86_64 2024-09-13T18:11:25.103 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-13T18:11:25.103 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-13T18:11:25.103 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-13T18:11:25.103 INFO:teuthology.orchestra.run.smithi204.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-13T18:11:25.104 INFO:teuthology.orchestra.run.smithi204.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-13T18:11:25.104 INFO:teuthology.orchestra.run.smithi204.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-13T18:11:25.104 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-13T18:11:25.104 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-argparse-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:25.104 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-common-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:25.104 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cephfs-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:25.104 INFO:teuthology.orchestra.run.smithi204.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-13T18:11:25.104 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-13T18:11:25.104 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-13T18:11:25.105 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-13T18:11:25.105 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-13T18:11:25.105 INFO:teuthology.orchestra.run.smithi204.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-13T18:11:25.105 INFO:teuthology.orchestra.run.smithi204.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-13T18:11:25.105 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-13T18:11:25.105 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-13T18:11:25.105 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-13T18:11:25.105 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-13T18:11:25.105 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-13T18:11:25.105 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-13T18:11:25.105 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-13T18:11:25.105 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-13T18:11:25.105 INFO:teuthology.orchestra.run.smithi204.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-13T18:11:25.105 INFO:teuthology.orchestra.run.smithi204.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-13T18:11:25.106 INFO:teuthology.orchestra.run.smithi204.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-13T18:11:25.106 INFO:teuthology.orchestra.run.smithi204.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-13T18:11:25.106 INFO:teuthology.orchestra.run.smithi204.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-13T18:11:25.106 INFO:teuthology.orchestra.run.smithi204.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-13T18:11:25.106 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-13T18:11:25.106 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-13T18:11:25.106 INFO:teuthology.orchestra.run.smithi204.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-13T18:11:25.106 INFO:teuthology.orchestra.run.smithi204.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-13T18:11:25.106 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-13T18:11:25.106 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ply-3.11-14.el9.noarch 2024-09-13T18:11:25.106 INFO:teuthology.orchestra.run.smithi204.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-13T18:11:25.106 INFO:teuthology.orchestra.run.smithi204.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-13T18:11:25.106 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-13T18:11:25.106 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-13T18:11:25.107 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-13T18:11:25.107 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-13T18:11:25.107 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-13T18:11:25.107 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rados-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:25.107 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rbd-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:25.107 INFO:teuthology.orchestra.run.smithi204.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-13T18:11:25.107 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-13T18:11:25.107 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-13T18:11:25.107 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rgw-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:25.107 INFO:teuthology.orchestra.run.smithi204.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-13T18:11:25.107 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-13T18:11:25.107 INFO:teuthology.orchestra.run.smithi204.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-13T18:11:25.107 INFO:teuthology.orchestra.run.smithi204.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-13T18:11:25.107 INFO:teuthology.orchestra.run.smithi204.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-13T18:11:25.108 INFO:teuthology.orchestra.run.smithi204.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-13T18:11:25.108 INFO:teuthology.orchestra.run.smithi204.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-13T18:11:25.108 INFO:teuthology.orchestra.run.smithi204.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-13T18:11:25.108 INFO:teuthology.orchestra.run.smithi204.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-13T18:11:25.108 INFO:teuthology.orchestra.run.smithi204.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-13T18:11:25.108 INFO:teuthology.orchestra.run.smithi204.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-13T18:11:25.108 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-fuse-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:25.108 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-mirror-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:25.108 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-nbd-2:18.2.4-860.g6c6b083f.el9.x86_64 2024-09-13T18:11:25.108 INFO:teuthology.orchestra.run.smithi204.stdout: re2-1:20211101-20.el9.x86_64 2024-09-13T18:11:25.108 INFO:teuthology.orchestra.run.smithi204.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-13T18:11:25.108 INFO:teuthology.orchestra.run.smithi204.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-13T18:11:25.108 INFO:teuthology.orchestra.run.smithi204.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-13T18:11:25.109 INFO:teuthology.orchestra.run.smithi204.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-13T18:11:25.109 INFO:teuthology.orchestra.run.smithi204.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-13T18:11:25.109 INFO:teuthology.orchestra.run.smithi204.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-13T18:11:25.109 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-13T18:11:25.109 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-09-13T18:11:25.485 DEBUG:teuthology.parallel:result is None 2024-09-13T18:11:25.485 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T18:11:25.645 DEBUG:teuthology.orchestra.run.smithi114:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-13T18:11:25.672 INFO:teuthology.orchestra.run.smithi114.stdout:18.2.4-860.g6c6b083f.el9 2024-09-13T18:11:25.673 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-860.g6c6b083f.el9 2024-09-13T18:11:25.673 INFO:teuthology.task.install:The correct ceph version 18.2.4-860.g6c6b083f is installed. 2024-09-13T18:11:25.675 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T18:11:25.819 DEBUG:teuthology.orchestra.run.smithi134:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-13T18:11:25.845 INFO:teuthology.orchestra.run.smithi134.stdout:18.2.4-860.g6c6b083f.el9 2024-09-13T18:11:25.846 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-860.g6c6b083f.el9 2024-09-13T18:11:25.846 INFO:teuthology.task.install:The correct ceph version 18.2.4-860.g6c6b083f is installed. 2024-09-13T18:11:25.848 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T18:11:26.002 DEBUG:teuthology.orchestra.run.smithi204:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-13T18:11:26.027 INFO:teuthology.orchestra.run.smithi204.stdout:18.2.4-860.g6c6b083f.el9 2024-09-13T18:11:26.027 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-860.g6c6b083f.el9 2024-09-13T18:11:26.027 INFO:teuthology.task.install:The correct ceph version 18.2.4-860.g6c6b083f is installed. 2024-09-13T18:11:26.029 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-13T18:11:26.029 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-13T18:11:26.029 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T18:11:26.062 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-13T18:11:26.062 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T18:11:26.095 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-13T18:11:26.095 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T18:11:26.124 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-13T18:11:26.125 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-13T18:11:26.125 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T18:11:26.157 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T18:11:26.226 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-13T18:11:26.226 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T18:11:26.255 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T18:11:26.324 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-13T18:11:26.324 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T18:11:26.354 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T18:11:26.423 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-13T18:11:26.423 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-13T18:11:26.423 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T18:11:26.452 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T18:11:26.521 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-13T18:11:26.522 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T18:11:26.550 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T18:11:26.617 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-13T18:11:26.617 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T18:11:26.647 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T18:11:26.716 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-13T18:11:26.716 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-13T18:11:26.716 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T18:11:26.746 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T18:11:26.815 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-13T18:11:26.815 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T18:11:26.843 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T18:11:26.910 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-13T18:11:26.910 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T18:11:26.940 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T18:11:27.009 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-13T18:11:27.105 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-09-13T18:11:27.106 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T18:11:27.106 INFO:tasks.cephadm:Cluster fsid is 97e7edf2-71fb-11ef-bceb-c7b262605968 2024-09-13T18:11:27.106 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-13T18:11:27.106 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.114', 'mon.b': '172.21.15.134', 'mon.c': '172.21.15.204'} 2024-09-13T18:11:27.107 INFO:tasks.cephadm:First mon is mon.a on smithi114 2024-09-13T18:11:27.107 INFO:tasks.cephadm:First mgr is x 2024-09-13T18:11:27.107 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-13T18:11:27.107 DEBUG:teuthology.orchestra.run.smithi114:> sudo hostname $(hostname -s) 2024-09-13T18:11:27.137 DEBUG:teuthology.orchestra.run.smithi134:> sudo hostname $(hostname -s) 2024-09-13T18:11:27.167 DEBUG:teuthology.orchestra.run.smithi204:> sudo hostname $(hostname -s) 2024-09-13T18:11:27.197 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-13T18:11:27.197 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T18:11:27.353 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/82954/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.2.4-860-g6c6b083f', 'node_name': '172.21.4.58+omani008', 'job_name': 'ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '18.2.4-860.g6c6b083f'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-10 17:57:22.919627', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240910.100730-reef', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/flavors/default/', 'archs': ['arm64', 'x86_64', 'source'], 'distro': 'centos'}, {'status': 'ready', 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/82954/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.2.4-860-g6c6b083f', 'node_name': '172.21.5.40+adami10', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '18.2.4-860.g6c6b083f'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-10 13:55:29.568955', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240910.100730-reef', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-13T18:11:27.505 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-rishabh-testing-20240910.100730-reef, sha1 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T18:11:27.507 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/x86_64/flavors/default/cephadm 2024-09-13T18:11:27.507 DEBUG:teuthology.orchestra.run.smithi114:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-13T18:11:27.853 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-r--r--. 1 ubuntu ubuntu 217223 Sep 13 18:11 /home/ubuntu/cephtest/cephadm 2024-09-13T18:11:27.853 DEBUG:teuthology.orchestra.run.smithi134:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-13T18:11:28.198 INFO:teuthology.orchestra.run.smithi134.stdout:-rw-r--r--. 1 ubuntu ubuntu 217223 Sep 13 18:11 /home/ubuntu/cephtest/cephadm 2024-09-13T18:11:28.198 DEBUG:teuthology.orchestra.run.smithi204:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-13T18:11:28.542 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r--. 1 ubuntu ubuntu 217223 Sep 13 18:11 /home/ubuntu/cephtest/cephadm 2024-09-13T18:11:28.542 DEBUG:teuthology.orchestra.run.smithi114:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-13T18:11:28.562 DEBUG:teuthology.orchestra.run.smithi134:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-13T18:11:28.582 DEBUG:teuthology.orchestra.run.smithi204:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-13T18:11:28.617 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 on all hosts... 2024-09-13T18:11:28.618 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 pull 2024-09-13T18:11:28.621 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 pull 2024-09-13T18:11:28.625 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 pull 2024-09-13T18:11:30.191 INFO:teuthology.orchestra.run.smithi134.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3... 2024-09-13T18:11:30.253 INFO:teuthology.orchestra.run.smithi114.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3... 2024-09-13T18:11:30.305 INFO:teuthology.orchestra.run.smithi204.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3... 2024-09-13T18:11:57.822 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-09-13T18:11:57.822 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_version": "ceph version 18.2.4-860-g6c6b083f (6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3) reef (stable)", 2024-09-13T18:11:57.822 INFO:teuthology.orchestra.run.smithi204.stdout: "image_id": "8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61", 2024-09-13T18:11:57.822 INFO:teuthology.orchestra.run.smithi204.stdout: "repo_digests": [ 2024-09-13T18:11:57.822 INFO:teuthology.orchestra.run.smithi204.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256" 2024-09-13T18:11:57.823 INFO:teuthology.orchestra.run.smithi204.stdout: ] 2024-09-13T18:11:57.823 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-09-13T18:12:27.317 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:12:27.317 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_version": "ceph version 18.2.4-860-g6c6b083f (6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3) reef (stable)", 2024-09-13T18:12:27.317 INFO:teuthology.orchestra.run.smithi114.stdout: "image_id": "8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61", 2024-09-13T18:12:27.318 INFO:teuthology.orchestra.run.smithi114.stdout: "repo_digests": [ 2024-09-13T18:12:27.318 INFO:teuthology.orchestra.run.smithi114.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256" 2024-09-13T18:12:27.318 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-13T18:12:27.318 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:12:27.480 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-13T18:12:27.480 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_version": "ceph version 18.2.4-860-g6c6b083f (6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3) reef (stable)", 2024-09-13T18:12:27.480 INFO:teuthology.orchestra.run.smithi134.stdout: "image_id": "8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61", 2024-09-13T18:12:27.480 INFO:teuthology.orchestra.run.smithi134.stdout: "repo_digests": [ 2024-09-13T18:12:27.480 INFO:teuthology.orchestra.run.smithi134.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256" 2024-09-13T18:12:27.481 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-13T18:12:27.481 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-13T18:12:27.499 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /etc/ceph 2024-09-13T18:12:27.527 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /etc/ceph 2024-09-13T18:12:27.555 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /etc/ceph 2024-09-13T18:12:27.590 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 777 /etc/ceph 2024-09-13T18:12:27.616 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 777 /etc/ceph 2024-09-13T18:12:27.641 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 777 /etc/ceph 2024-09-13T18:12:27.670 INFO:tasks.cephadm:Writing seed config... 2024-09-13T18:12:27.671 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-13T18:12:27.671 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-13T18:12:27.671 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-13T18:12:27.672 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-13T18:12:27.672 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-13T18:12:27.672 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-13T18:12:27.672 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-13T18:12:27.672 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-13T18:12:27.672 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-13T18:12:27.672 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-13T18:12:27.672 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-13T18:12:27.672 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-13T18:12:27.672 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-13T18:12:27.672 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-13T18:12:27.672 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-13T18:12:27.673 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-13T18:12:27.673 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-09-13T18:12:27.673 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-13T18:12:27.673 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-13T18:12:27.673 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-13T18:12:27.673 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-13T18:12:27.673 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-09-13T18:12:27.673 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-13T18:12:27.673 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-13T18:12:27.673 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-13T18:12:27.673 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-13T18:12:27.673 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-13T18:12:27.673 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-13T18:12:27.674 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-13T18:12:27.674 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-13T18:12:27.674 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-13T18:12:27.674 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-09-13T18:12:27.674 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-13T18:12:27.674 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-13T18:12:27.674 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-13T18:12:27.674 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-13T18:12:27.674 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-13T18:12:27.674 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-13T18:12:27.674 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-13T18:12:27.674 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-13T18:12:27.675 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-13T18:12:27.675 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-13T18:12:27.675 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-13T18:12:27.675 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-13T18:12:27.675 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-13T18:12:27.675 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-09-13T18:12:27.676 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-13T18:12:27.676 DEBUG:teuthology.orchestra.run.smithi114:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-13T18:12:27.691 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 97e7edf2-71fb-11ef-bceb-c7b262605968 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-13T18:12:27.692 DEBUG:teuthology.orchestra.run.smithi114:mon.a> sudo journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mon.a.service 2024-09-13T18:12:27.734 DEBUG:teuthology.orchestra.run.smithi114:mgr.x> sudo journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mgr.x.service 2024-09-13T18:12:27.777 INFO:tasks.cephadm:Bootstrapping... 2024-09-13T18:12:27.777 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 -v bootstrap --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.114 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-13T18:12:27.928 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-09-13T18:12:27.928 INFO:teuthology.orchestra.run.smithi114.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3', '-v', 'bootstrap', '--fsid', '97e7edf2-71fb-11ef-bceb-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.114', '--skip-admin-label'] 2024-09-13T18:12:27.961 INFO:teuthology.orchestra.run.smithi114.stdout:/bin/podman: stdout 5.2.2 2024-09-13T18:12:27.962 INFO:teuthology.orchestra.run.smithi114.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-13T18:12:27.962 INFO:teuthology.orchestra.run.smithi114.stdout:Verifying podman|docker is present... 2024-09-13T18:12:27.983 INFO:teuthology.orchestra.run.smithi114.stdout:/bin/podman: stdout 5.2.2 2024-09-13T18:12:27.983 INFO:teuthology.orchestra.run.smithi114.stdout:Verifying lvm2 is present... 2024-09-13T18:12:27.983 INFO:teuthology.orchestra.run.smithi114.stdout:Verifying time synchronization is in place... 2024-09-13T18:12:27.991 INFO:teuthology.orchestra.run.smithi114.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-13T18:12:27.991 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-13T18:12:27.997 INFO:teuthology.orchestra.run.smithi114.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-13T18:12:27.997 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stdout inactive 2024-09-13T18:12:28.004 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stdout enabled 2024-09-13T18:12:28.010 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stdout active 2024-09-13T18:12:28.010 INFO:teuthology.orchestra.run.smithi114.stdout:Unit chronyd.service is enabled and running 2024-09-13T18:12:28.010 INFO:teuthology.orchestra.run.smithi114.stdout:Repeating the final host check... 2024-09-13T18:12:28.032 INFO:teuthology.orchestra.run.smithi114.stdout:/bin/podman: stdout 5.2.2 2024-09-13T18:12:28.032 INFO:teuthology.orchestra.run.smithi114.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-13T18:12:28.032 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl is present 2024-09-13T18:12:28.032 INFO:teuthology.orchestra.run.smithi114.stdout:lvcreate is present 2024-09-13T18:12:28.038 INFO:teuthology.orchestra.run.smithi114.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-13T18:12:28.038 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-13T18:12:28.044 INFO:teuthology.orchestra.run.smithi114.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-13T18:12:28.044 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stdout inactive 2024-09-13T18:12:28.051 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stdout enabled 2024-09-13T18:12:28.057 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stdout active 2024-09-13T18:12:28.057 INFO:teuthology.orchestra.run.smithi114.stdout:Unit chronyd.service is enabled and running 2024-09-13T18:12:28.057 INFO:teuthology.orchestra.run.smithi114.stdout:Host looks OK 2024-09-13T18:12:28.058 INFO:teuthology.orchestra.run.smithi114.stdout:Cluster fsid: 97e7edf2-71fb-11ef-bceb-c7b262605968 2024-09-13T18:12:28.058 INFO:teuthology.orchestra.run.smithi114.stdout:Acquiring lock 139962081074768 on /run/cephadm/97e7edf2-71fb-11ef-bceb-c7b262605968.lock 2024-09-13T18:12:28.058 INFO:teuthology.orchestra.run.smithi114.stdout:Lock 139962081074768 acquired on /run/cephadm/97e7edf2-71fb-11ef-bceb-c7b262605968.lock 2024-09-13T18:12:28.058 INFO:teuthology.orchestra.run.smithi114.stdout:Verifying IP 172.21.15.114 port 3300 ... 2024-09-13T18:12:28.058 INFO:teuthology.orchestra.run.smithi114.stdout:Verifying IP 172.21.15.114 port 6789 ... 2024-09-13T18:12:28.058 INFO:teuthology.orchestra.run.smithi114.stdout:Base mon IP(s) is [172.21.15.114:3300, 172.21.15.114:6789], mon addrv is [v2:172.21.15.114:3300,v1:172.21.15.114:6789] 2024-09-13T18:12:28.061 INFO:teuthology.orchestra.run.smithi114.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.114 metric 100 2024-09-13T18:12:28.061 INFO:teuthology.orchestra.run.smithi114.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.114 metric 100 2024-09-13T18:12:28.064 INFO:teuthology.orchestra.run.smithi114.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-13T18:12:28.064 INFO:teuthology.orchestra.run.smithi114.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-13T18:12:28.064 INFO:teuthology.orchestra.run.smithi114.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-13T18:12:28.066 INFO:teuthology.orchestra.run.smithi114.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-13T18:12:28.066 INFO:teuthology.orchestra.run.smithi114.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-13T18:12:28.067 INFO:teuthology.orchestra.run.smithi114.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-13T18:12:28.067 INFO:teuthology.orchestra.run.smithi114.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-13T18:12:28.067 INFO:teuthology.orchestra.run.smithi114.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:cb63/64 scope link noprefixroute 2024-09-13T18:12:28.067 INFO:teuthology.orchestra.run.smithi114.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-13T18:12:28.067 INFO:teuthology.orchestra.run.smithi114.stdout:Mon IP `172.21.15.114` is in CIDR network `172.21.0.0/20` 2024-09-13T18:12:28.068 INFO:teuthology.orchestra.run.smithi114.stdout:Mon IP `172.21.15.114` is in CIDR network `172.21.0.0/20` 2024-09-13T18:12:28.068 INFO:teuthology.orchestra.run.smithi114.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-13T18:12:28.068 INFO:teuthology.orchestra.run.smithi114.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-13T18:12:28.068 INFO:teuthology.orchestra.run.smithi114.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3... 2024-09-13T18:12:28.527 INFO:teuthology.orchestra.run.smithi114.stdout:/bin/podman: stdout 8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61 2024-09-13T18:12:28.528 INFO:teuthology.orchestra.run.smithi114.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3... 2024-09-13T18:12:28.528 INFO:teuthology.orchestra.run.smithi114.stdout:/bin/podman: stderr Getting image source signatures 2024-09-13T18:12:28.528 INFO:teuthology.orchestra.run.smithi114.stdout:/bin/podman: stderr Copying blob sha256:cdfdf789b01393b98d7733e02a8d993977e2aa05d922747269d27c845cd4920c 2024-09-13T18:12:28.528 INFO:teuthology.orchestra.run.smithi114.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-13T18:12:28.528 INFO:teuthology.orchestra.run.smithi114.stdout:/bin/podman: stderr Copying config sha256:8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61 2024-09-13T18:12:28.528 INFO:teuthology.orchestra.run.smithi114.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-13T18:12:29.424 INFO:teuthology.orchestra.run.smithi114.stdout:ceph: stdout ceph version 18.2.4-860-g6c6b083f (6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3) reef (stable) 2024-09-13T18:12:29.424 INFO:teuthology.orchestra.run.smithi114.stdout:Ceph version: ceph version 18.2.4-860-g6c6b083f (6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3) reef (stable) 2024-09-13T18:12:29.424 INFO:teuthology.orchestra.run.smithi114.stdout:Extracting ceph user uid/gid from container image... 2024-09-13T18:12:30.397 INFO:teuthology.orchestra.run.smithi114.stdout:stat: stdout 167 167 2024-09-13T18:12:30.397 INFO:teuthology.orchestra.run.smithi114.stdout:Creating initial keys... 2024-09-13T18:12:31.352 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph-authtool: stdout AQCOgORmmqVQKBAAQOMmMyL5EKlKaKGpMzUhzg== 2024-09-13T18:12:32.216 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph-authtool: stdout AQCPgORmKjKmJxAAcb5FmuVTWWiNKJCdCRCFtw== 2024-09-13T18:12:33.063 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph-authtool: stdout AQCQgORmzo3KHRAAjHbgjB/Lp0u/bFnsF70m3A== 2024-09-13T18:12:33.064 INFO:teuthology.orchestra.run.smithi114.stdout:Creating initial monmap... 2024-09-13T18:12:33.961 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-13T18:12:33.961 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-09-13T18:12:33.961 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 97e7edf2-71fb-11ef-bceb-c7b262605968 2024-09-13T18:12:33.961 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-13T18:12:33.962 INFO:teuthology.orchestra.run.smithi114.stdout:monmaptool for a [v2:172.21.15.114:3300,v1:172.21.15.114:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-13T18:12:33.962 INFO:teuthology.orchestra.run.smithi114.stdout:setting min_mon_release = pacific 2024-09-13T18:12:33.962 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/monmaptool: set fsid to 97e7edf2-71fb-11ef-bceb-c7b262605968 2024-09-13T18:12:33.962 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-13T18:12:33.962 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:12:33.962 INFO:teuthology.orchestra.run.smithi114.stdout:Creating mon... 2024-09-13T18:12:34.975 INFO:teuthology.orchestra.run.smithi114.stdout:create mon.a on 2024-09-13T18:12:35.188 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-13T18:12:35.333 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-13T18:12:35.515 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-97e7edf2-71fb-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-97e7edf2-71fb-11ef-bceb-c7b262605968.target. 2024-09-13T18:12:35.515 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-97e7edf2-71fb-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-97e7edf2-71fb-11ef-bceb-c7b262605968.target. 2024-09-13T18:12:35.737 INFO:teuthology.orchestra.run.smithi114.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mon.a 2024-09-13T18:12:35.737 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stderr Failed to reset failed state of unit ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mon.a.service: Unit ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mon.a.service not loaded. 2024-09-13T18:12:35.899 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-97e7edf2-71fb-11ef-bceb-c7b262605968.target.wants/ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mon.a.service → /etc/systemd/system/ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@.service. 2024-09-13T18:12:36.393 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:36 smithi114 ceph-mon[30524]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-13T18:12:36.393 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:36 smithi114 bash[30504]: 0da18cea228bdbde9392e50ae4a9429653f2e9584ff9444ac8f34bc8c6f98e86 2024-09-13T18:12:36.402 INFO:teuthology.orchestra.run.smithi114.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-13T18:12:36.403 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stdout disabled 2024-09-13T18:12:36.410 INFO:teuthology.orchestra.run.smithi114.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-13T18:12:36.410 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stdout inactive 2024-09-13T18:12:36.410 INFO:teuthology.orchestra.run.smithi114.stdout:firewalld.service is not enabled 2024-09-13T18:12:36.410 INFO:teuthology.orchestra.run.smithi114.stdout:Not possible to enable service . firewalld.service is not available 2024-09-13T18:12:36.411 INFO:teuthology.orchestra.run.smithi114.stdout:Waiting for mon to start... 2024-09-13T18:12:36.411 INFO:teuthology.orchestra.run.smithi114.stdout:Waiting for mon... 2024-09-13T18:12:36.645 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:36 smithi114 systemd[1]: Started Ceph mon.a for 97e7edf2-71fb-11ef-bceb-c7b262605968. 2024-09-13T18:12:37.563 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:37 smithi114 ceph-mon[30524]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-13T18:12:37.563 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:37 smithi114 ceph-mon[30524]: monmap e1: 1 mons at {a=[v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-13T18:12:37.563 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:37 smithi114 ceph-mon[30524]: fsmap 2024-09-13T18:12:37.563 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:37 smithi114 ceph-mon[30524]: osdmap e1: 0 total, 0 up, 0 in 2024-09-13T18:12:37.563 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:37 smithi114 ceph-mon[30524]: mgrmap e1: no daemons active 2024-09-13T18:12:37.563 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:37 smithi114 ceph-mon[30524]: from='client.? 172.21.15.114:0/491140085' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-13T18:12:37.570 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout cluster: 2024-09-13T18:12:37.571 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout id: 97e7edf2-71fb-11ef-bceb-c7b262605968 2024-09-13T18:12:37.571 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-13T18:12:37.571 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout 2024-09-13T18:12:37.571 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout services: 2024-09-13T18:12:37.571 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.527507s) 2024-09-13T18:12:37.571 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-13T18:12:37.571 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-13T18:12:37.571 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout 2024-09-13T18:12:37.571 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout data: 2024-09-13T18:12:37.571 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-13T18:12:37.571 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-13T18:12:37.572 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-13T18:12:37.572 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout pgs: 2024-09-13T18:12:37.572 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout 2024-09-13T18:12:37.572 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:36.817+0000 7f7fbecfb640 1 Processor -- start 2024-09-13T18:12:37.572 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:36.817+0000 7f7fbecfb640 1 -- start start 2024-09-13T18:12:37.572 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:36.817+0000 7f7fbecfb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7fb8106030 0x7f7fb8108420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:37.572 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:36.817+0000 7f7fbecfb640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb8079e90 con 0x7f7fb8106030 2024-09-13T18:12:37.572 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:36.818+0000 7f7fbca97640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7fb8106030 0x7f7fb8108420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:37.572 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:36.818+0000 7f7fbca97640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7fb8106030 0x7f7fb8108420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47552/0 (socket says 172.21.15.114:47552) 2024-09-13T18:12:37.572 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:36.818+0000 7f7fbca97640 1 -- 172.21.15.114:0/3762440246 learned_addr learned my addr 172.21.15.114:0/3762440246 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:12:37.572 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:36.818+0000 7f7fbca97640 1 -- 172.21.15.114:0/3762440246 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb8108960 con 0x7f7fb8106030 2024-09-13T18:12:37.572 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:36.821+0000 7f7fbca97640 1 --2- 172.21.15.114:0/3762440246 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7fb8106030 0x7f7fb8108420 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7fac009820 tx=0x7f7fac02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df4675029841d162 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:37.573 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:36.822+0000 7f7fb77fe640 1 -- 172.21.15.114:0/3762440246 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7fac02ae70 con 0x7f7fb8106030 2024-09-13T18:12:37.573 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:36.822+0000 7f7fb77fe640 1 -- 172.21.15.114:0/3762440246 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fac032440 con 0x7f7fb8106030 2024-09-13T18:12:37.573 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:36.822+0000 7f7fb77fe640 1 -- 172.21.15.114:0/3762440246 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7fac032700 con 0x7f7fb8106030 2024-09-13T18:12:37.573 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:36.822+0000 7f7fbecfb640 1 -- 172.21.15.114:0/3762440246 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7fb8106030 msgr2=0x7f7fb8108420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:37.573 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:36.822+0000 7f7fbecfb640 1 --2- 172.21.15.114:0/3762440246 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7fb8106030 0x7f7fb8108420 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7fac009820 tx=0x7f7fac02a270 comp rx=0 tx=0).stop 2024-09-13T18:12:37.573 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:36.822+0000 7f7fbecfb640 1 -- 172.21.15.114:0/3762440246 shutdown_connections 2024-09-13T18:12:37.573 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:36.822+0000 7f7fbecfb640 1 --2- 172.21.15.114:0/3762440246 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7fb8106030 0x7f7fb8108420 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:37.573 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:36.822+0000 7f7fbecfb640 1 -- 172.21.15.114:0/3762440246 >> 172.21.15.114:0/3762440246 conn(0x7f7fb80ffc30 msgr2=0x7f7fb8102050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:12:37.573 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:36.822+0000 7f7fbecfb640 1 -- 172.21.15.114:0/3762440246 shutdown_connections 2024-09-13T18:12:37.573 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:36.822+0000 7f7fbecfb640 1 -- 172.21.15.114:0/3762440246 wait complete. 2024-09-13T18:12:37.573 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:36.823+0000 7f7fbecfb640 1 Processor -- start 2024-09-13T18:12:37.573 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:36.823+0000 7f7fbecfb640 1 -- start start 2024-09-13T18:12:37.574 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:36.823+0000 7f7fbecfb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7fb8106030 0x7f7fb8105860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:37.574 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:36.823+0000 7f7fbecfb640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb8105da0 con 0x7f7fb8106030 2024-09-13T18:12:37.574 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:36.823+0000 7f7fbca97640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7fb8106030 0x7f7fb8105860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:37.574 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:36.823+0000 7f7fbca97640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7fb8106030 0x7f7fb8105860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47560/0 (socket says 172.21.15.114:47560) 2024-09-13T18:12:37.574 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:36.823+0000 7f7fbca97640 1 -- 172.21.15.114:0/491140085 learned_addr learned my addr 172.21.15.114:0/491140085 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:12:37.574 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:36.824+0000 7f7fbca97640 1 -- 172.21.15.114:0/491140085 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fac0094d0 con 0x7f7fb8106030 2024-09-13T18:12:37.574 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:36.824+0000 7f7fbca97640 1 --2- 172.21.15.114:0/491140085 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7fb8106030 0x7f7fb8105860 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7fac0090b0 tx=0x7f7fac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:37.574 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:36.824+0000 7f7fb5ffb640 1 -- 172.21.15.114:0/491140085 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7fac0308d0 con 0x7f7fb8106030 2024-09-13T18:12:37.574 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:36.824+0000 7f7fb5ffb640 1 -- 172.21.15.114:0/491140085 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fac030a30 con 0x7f7fb8106030 2024-09-13T18:12:37.574 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:36.824+0000 7f7fb5ffb640 1 -- 172.21.15.114:0/491140085 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7fac030cf0 con 0x7f7fb8106030 2024-09-13T18:12:37.574 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:36.824+0000 7f7fbecfb640 1 -- 172.21.15.114:0/491140085 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fb8103fe0 con 0x7f7fb8106030 2024-09-13T18:12:37.574 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:36.824+0000 7f7fbecfb640 1 -- 172.21.15.114:0/491140085 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fb8104480 con 0x7f7fb8106030 2024-09-13T18:12:37.574 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:36.824+0000 7f7fb5ffb640 1 -- 172.21.15.114:0/491140085 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f7fac039040 con 0x7f7fb8106030 2024-09-13T18:12:37.575 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:36.825+0000 7f7fb5ffb640 1 -- 172.21.15.114:0/491140085 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7fac0468c0 con 0x7f7fb8106030 2024-09-13T18:12:37.575 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:36.825+0000 7f7fbecfb640 1 -- 172.21.15.114:0/491140085 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fb8106490 con 0x7f7fb8106030 2024-09-13T18:12:37.575 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:36.827+0000 7f7fb5ffb640 1 -- 172.21.15.114:0/491140085 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f7fac037070 con 0x7f7fb8106030 2024-09-13T18:12:37.575 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:36.885+0000 7f7fbecfb640 1 -- 172.21.15.114:0/491140085 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f7fb8108df0 con 0x7f7fb8106030 2024-09-13T18:12:37.575 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:36.886+0000 7f7fb5ffb640 1 -- 172.21.15.114:0/491140085 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f7fac046b20 con 0x7f7fb8106030 2024-09-13T18:12:37.575 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:36.886+0000 7f7fbecfb640 1 -- 172.21.15.114:0/491140085 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7fb8106030 msgr2=0x7f7fb8105860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:37.575 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:36.886+0000 7f7fbecfb640 1 --2- 172.21.15.114:0/491140085 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7fb8106030 0x7f7fb8105860 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7fac0090b0 tx=0x7f7fac0046c0 comp rx=0 tx=0).stop 2024-09-13T18:12:37.575 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:36.887+0000 7f7fbecfb640 1 -- 172.21.15.114:0/491140085 shutdown_connections 2024-09-13T18:12:37.575 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:36.887+0000 7f7fbecfb640 1 --2- 172.21.15.114:0/491140085 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7fb8106030 0x7f7fb8105860 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:37.575 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:36.887+0000 7f7fbecfb640 1 -- 172.21.15.114:0/491140085 >> 172.21.15.114:0/491140085 conn(0x7f7fb80ffc30 msgr2=0x7f7fb8100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:12:37.575 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:36.887+0000 7f7fbecfb640 1 -- 172.21.15.114:0/491140085 shutdown_connections 2024-09-13T18:12:37.576 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:36.887+0000 7f7fbecfb640 1 -- 172.21.15.114:0/491140085 wait complete. 2024-09-13T18:12:37.576 INFO:teuthology.orchestra.run.smithi114.stdout:mon is available 2024-09-13T18:12:37.576 INFO:teuthology.orchestra.run.smithi114.stdout:Assimilating anything we can from ceph.conf... 2024-09-13T18:12:38.561 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout 2024-09-13T18:12:38.561 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout [global] 2024-09-13T18:12:38.561 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout fsid = 97e7edf2-71fb-11ef-bceb-c7b262605968 2024-09-13T18:12:38.561 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.114:3300,v1:172.21.15.114:6789] 2024-09-13T18:12:38.561 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-13T18:12:38.562 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-13T18:12:38.562 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-13T18:12:38.562 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-13T18:12:38.562 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout 2024-09-13T18:12:38.562 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-13T18:12:38.562 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-13T18:12:38.562 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout 2024-09-13T18:12:38.562 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout [osd] 2024-09-13T18:12:38.562 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-13T18:12:38.562 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-13T18:12:38.562 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-13T18:12:38.562 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-13T18:12:38.562 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-13T18:12:38.562 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:37.955+0000 7f9061904640 1 Processor -- start 2024-09-13T18:12:38.563 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:37.955+0000 7f9061904640 1 -- start start 2024-09-13T18:12:38.563 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:37.956+0000 7f9061904640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f905c103840 0x7f905c103c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:38.563 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:37.956+0000 7f9061904640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905c104210 con 0x7f905c103840 2024-09-13T18:12:38.563 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:37.956+0000 7f905affd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f905c103840 0x7f905c103c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:38.563 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:37.956+0000 7f905affd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f905c103840 0x7f905c103c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47574/0 (socket says 172.21.15.114:47574) 2024-09-13T18:12:38.563 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:37.956+0000 7f905affd640 1 -- 172.21.15.114:0/394393833 learned_addr learned my addr 172.21.15.114:0/394393833 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:12:38.563 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:37.956+0000 7f905affd640 1 -- 172.21.15.114:0/394393833 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f905c104350 con 0x7f905c103840 2024-09-13T18:12:38.563 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:37.957+0000 7f905affd640 1 --2- 172.21.15.114:0/394393833 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f905c103840 0x7f905c103c40 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f904c009820 tx=0x7f904c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84e8098b27f7370d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:38.563 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:37.957+0000 7f905a7fc640 1 -- 172.21.15.114:0/394393833 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f904c02ae70 con 0x7f905c103840 2024-09-13T18:12:38.563 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:37.957+0000 7f905a7fc640 1 -- 172.21.15.114:0/394393833 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f904c032440 con 0x7f905c103840 2024-09-13T18:12:38.563 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:37.957+0000 7f905a7fc640 1 -- 172.21.15.114:0/394393833 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f904c032700 con 0x7f905c103840 2024-09-13T18:12:38.563 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:37.958+0000 7f9061904640 1 -- 172.21.15.114:0/394393833 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f905c103840 msgr2=0x7f905c103c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:38.563 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:37.958+0000 7f9061904640 1 --2- 172.21.15.114:0/394393833 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f905c103840 0x7f905c103c40 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f904c009820 tx=0x7f904c02a270 comp rx=0 tx=0).stop 2024-09-13T18:12:38.564 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:37.958+0000 7f9061904640 1 -- 172.21.15.114:0/394393833 shutdown_connections 2024-09-13T18:12:38.564 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:37.958+0000 7f9061904640 1 --2- 172.21.15.114:0/394393833 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f905c103840 0x7f905c103c40 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:38.564 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:37.958+0000 7f9061904640 1 -- 172.21.15.114:0/394393833 >> 172.21.15.114:0/394393833 conn(0x7f905c0ff390 msgr2=0x7f905c1017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:12:38.564 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:37.958+0000 7f9061904640 1 -- 172.21.15.114:0/394393833 shutdown_connections 2024-09-13T18:12:38.564 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:37.958+0000 7f9061904640 1 -- 172.21.15.114:0/394393833 wait complete. 2024-09-13T18:12:38.564 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:37.958+0000 7f9061904640 1 Processor -- start 2024-09-13T18:12:38.564 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:37.958+0000 7f9061904640 1 -- start start 2024-09-13T18:12:38.564 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:37.959+0000 7f9061904640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f905c103840 0x7f905c198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:38.564 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:37.959+0000 7f9061904640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905c199000 con 0x7f905c103840 2024-09-13T18:12:38.564 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:37.959+0000 7f905affd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f905c103840 0x7f905c198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:38.564 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:37.959+0000 7f905affd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f905c103840 0x7f905c198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47582/0 (socket says 172.21.15.114:47582) 2024-09-13T18:12:38.564 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:37.959+0000 7f905affd640 1 -- 172.21.15.114:0/1073943279 learned_addr learned my addr 172.21.15.114:0/1073943279 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:12:38.564 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:37.959+0000 7f905affd640 1 -- 172.21.15.114:0/1073943279 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f904c0094d0 con 0x7f905c103840 2024-09-13T18:12:38.565 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:37.959+0000 7f905affd640 1 --2- 172.21.15.114:0/1073943279 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f905c103840 0x7f905c198ac0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f904c0090b0 tx=0x7f904c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:38.565 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:37.959+0000 7f9060902640 1 -- 172.21.15.114:0/1073943279 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f904c0308d0 con 0x7f905c103840 2024-09-13T18:12:38.565 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:37.959+0000 7f9060902640 1 -- 172.21.15.114:0/1073943279 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f904c030a30 con 0x7f905c103840 2024-09-13T18:12:38.565 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:37.959+0000 7f9060902640 1 -- 172.21.15.114:0/1073943279 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f904c030cf0 con 0x7f905c103840 2024-09-13T18:12:38.565 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:37.959+0000 7f9061904640 1 -- 172.21.15.114:0/1073943279 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f905c199200 con 0x7f905c103840 2024-09-13T18:12:38.565 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:37.960+0000 7f9061904640 1 -- 172.21.15.114:0/1073943279 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f905c1996a0 con 0x7f905c103840 2024-09-13T18:12:38.565 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:37.960+0000 7f9060902640 1 -- 172.21.15.114:0/1073943279 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f904c039040 con 0x7f905c103840 2024-09-13T18:12:38.565 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:37.960+0000 7f9060902640 1 -- 172.21.15.114:0/1073943279 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f904c0468c0 con 0x7f905c103840 2024-09-13T18:12:38.565 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:37.961+0000 7f9061904640 1 -- 172.21.15.114:0/1073943279 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f905c103cc0 con 0x7f905c103840 2024-09-13T18:12:38.565 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:37.963+0000 7f9060902640 1 -- 172.21.15.114:0/1073943279 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f904c037070 con 0x7f905c103840 2024-09-13T18:12:38.565 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.019+0000 7f9061904640 1 -- 172.21.15.114:0/1073943279 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f905c19bf80 con 0x7f905c103840 2024-09-13T18:12:38.565 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.022+0000 7f9060902640 1 -- 172.21.15.114:0/1073943279 <== mon.0 v2:172.21.15.114:3300/0 7 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f904c04e7f0 con 0x7f905c103840 2024-09-13T18:12:38.566 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.022+0000 7f9060902640 1 -- 172.21.15.114:0/1073943279 <== mon.0 v2:172.21.15.114:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+431 (secure 0 0 0) 0x7f904c046b80 con 0x7f905c103840 2024-09-13T18:12:38.566 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.023+0000 7f9061904640 1 -- 172.21.15.114:0/1073943279 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f905c103840 msgr2=0x7f905c198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:38.566 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.023+0000 7f9061904640 1 --2- 172.21.15.114:0/1073943279 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f905c103840 0x7f905c198ac0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f904c0090b0 tx=0x7f904c0046c0 comp rx=0 tx=0).stop 2024-09-13T18:12:38.566 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.023+0000 7f9061904640 1 -- 172.21.15.114:0/1073943279 shutdown_connections 2024-09-13T18:12:38.566 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.023+0000 7f9061904640 1 --2- 172.21.15.114:0/1073943279 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f905c103840 0x7f905c198ac0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:38.566 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.023+0000 7f9061904640 1 -- 172.21.15.114:0/1073943279 >> 172.21.15.114:0/1073943279 conn(0x7f905c0ff390 msgr2=0x7f905c18d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:12:38.566 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.023+0000 7f9061904640 1 -- 172.21.15.114:0/1073943279 shutdown_connections 2024-09-13T18:12:38.566 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.023+0000 7f9061904640 1 -- 172.21.15.114:0/1073943279 wait complete. 2024-09-13T18:12:38.566 INFO:teuthology.orchestra.run.smithi114.stdout:Generating new minimal ceph.conf... 2024-09-13T18:12:38.653 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:38 smithi114 ceph-mon[30524]: from='client.? 172.21.15.114:0/1073943279' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-13T18:12:38.653 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:38 smithi114 ceph-mon[30524]: from='client.? 172.21.15.114:0/1073943279' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-13T18:12:39.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:39 smithi114 ceph-mon[30524]: from='client.? 172.21.15.114:0/3268299526' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:12:39.782 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.920+0000 7fa31f587640 1 Processor -- start 2024-09-13T18:12:39.782 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.920+0000 7fa31f587640 1 -- start start 2024-09-13T18:12:39.782 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.921+0000 7fa31f587640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa318105ff0 0x7fa3181083e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:39.782 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.921+0000 7fa31f587640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa318079ed0 con 0x7fa318105ff0 2024-09-13T18:12:39.782 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.921+0000 7fa31d323640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa318105ff0 0x7fa3181083e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:39.782 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.921+0000 7fa31d323640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa318105ff0 0x7fa3181083e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47594/0 (socket says 172.21.15.114:47594) 2024-09-13T18:12:39.783 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.921+0000 7fa31d323640 1 -- 172.21.15.114:0/1762944249 learned_addr learned my addr 172.21.15.114:0/1762944249 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:12:39.783 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.921+0000 7fa31d323640 1 -- 172.21.15.114:0/1762944249 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa318108920 con 0x7fa318105ff0 2024-09-13T18:12:39.783 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.922+0000 7fa31d323640 1 --2- 172.21.15.114:0/1762944249 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa318105ff0 0x7fa3181083e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa314009820 tx=0x7fa31402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d2863cbf8c0994e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:39.783 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.922+0000 7fa30ffff640 1 -- 172.21.15.114:0/1762944249 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa31402ae70 con 0x7fa318105ff0 2024-09-13T18:12:39.783 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.922+0000 7fa30ffff640 1 -- 172.21.15.114:0/1762944249 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fa3140328b0 con 0x7fa318105ff0 2024-09-13T18:12:39.783 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.922+0000 7fa30ffff640 1 -- 172.21.15.114:0/1762944249 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa314030850 con 0x7fa318105ff0 2024-09-13T18:12:39.783 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.922+0000 7fa31f587640 1 -- 172.21.15.114:0/1762944249 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa318105ff0 msgr2=0x7fa3181083e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:39.783 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.922+0000 7fa31f587640 1 --2- 172.21.15.114:0/1762944249 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa318105ff0 0x7fa3181083e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa314009820 tx=0x7fa31402a270 comp rx=0 tx=0).stop 2024-09-13T18:12:39.783 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.922+0000 7fa31f587640 1 -- 172.21.15.114:0/1762944249 shutdown_connections 2024-09-13T18:12:39.783 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.922+0000 7fa31f587640 1 --2- 172.21.15.114:0/1762944249 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa318105ff0 0x7fa3181083e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:39.783 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.922+0000 7fa31f587640 1 -- 172.21.15.114:0/1762944249 >> 172.21.15.114:0/1762944249 conn(0x7fa3180ffc30 msgr2=0x7fa318102050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:12:39.783 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.923+0000 7fa31f587640 1 -- 172.21.15.114:0/1762944249 shutdown_connections 2024-09-13T18:12:39.783 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.923+0000 7fa31f587640 1 -- 172.21.15.114:0/1762944249 wait complete. 2024-09-13T18:12:39.784 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.923+0000 7fa31f587640 1 Processor -- start 2024-09-13T18:12:39.784 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.923+0000 7fa31f587640 1 -- start start 2024-09-13T18:12:39.784 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.923+0000 7fa31f587640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa318105ff0 0x7fa318105870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:39.784 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.923+0000 7fa31f587640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa318105db0 con 0x7fa318105ff0 2024-09-13T18:12:39.784 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.923+0000 7fa31d323640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa318105ff0 0x7fa318105870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:39.784 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.923+0000 7fa31d323640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa318105ff0 0x7fa318105870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47600/0 (socket says 172.21.15.114:47600) 2024-09-13T18:12:39.784 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.923+0000 7fa31d323640 1 -- 172.21.15.114:0/3268299526 learned_addr learned my addr 172.21.15.114:0/3268299526 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:12:39.784 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.924+0000 7fa31d323640 1 -- 172.21.15.114:0/3268299526 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3140094d0 con 0x7fa318105ff0 2024-09-13T18:12:39.784 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.924+0000 7fa31d323640 1 --2- 172.21.15.114:0/3268299526 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa318105ff0 0x7fa318105870 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa314000c00 tx=0x7fa314030db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:39.784 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.924+0000 7fa30e7fc640 1 -- 172.21.15.114:0/3268299526 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa314032d60 con 0x7fa318105ff0 2024-09-13T18:12:39.784 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.924+0000 7fa31f587640 1 -- 172.21.15.114:0/3268299526 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa318103fa0 con 0x7fa318105ff0 2024-09-13T18:12:39.784 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.924+0000 7fa30e7fc640 1 -- 172.21.15.114:0/3268299526 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fa31403b8a0 con 0x7fa318105ff0 2024-09-13T18:12:39.784 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.924+0000 7fa30e7fc640 1 -- 172.21.15.114:0/3268299526 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa31403a640 con 0x7fa318105ff0 2024-09-13T18:12:39.785 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.924+0000 7fa31f587640 1 -- 172.21.15.114:0/3268299526 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa318104440 con 0x7fa318105ff0 2024-09-13T18:12:39.785 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.924+0000 7fa30e7fc640 1 -- 172.21.15.114:0/3268299526 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fa314039040 con 0x7fa318105ff0 2024-09-13T18:12:39.785 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.924+0000 7fa30e7fc640 1 -- 172.21.15.114:0/3268299526 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa314044bc0 con 0x7fa318105ff0 2024-09-13T18:12:39.785 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.925+0000 7fa31f587640 1 -- 172.21.15.114:0/3268299526 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa318106450 con 0x7fa318105ff0 2024-09-13T18:12:39.785 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.927+0000 7fa30e7fc640 1 -- 172.21.15.114:0/3268299526 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7fa31403f020 con 0x7fa318105ff0 2024-09-13T18:12:39.785 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.983+0000 7fa31f587640 1 -- 172.21.15.114:0/3268299526 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fa318109970 con 0x7fa318105ff0 2024-09-13T18:12:39.785 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.983+0000 7fa30e7fc640 1 -- 172.21.15.114:0/3268299526 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+177 (secure 0 0 0) 0x7fa314037070 con 0x7fa318105ff0 2024-09-13T18:12:39.785 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.984+0000 7fa31f587640 1 -- 172.21.15.114:0/3268299526 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa318105ff0 msgr2=0x7fa318105870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:39.785 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.984+0000 7fa31f587640 1 --2- 172.21.15.114:0/3268299526 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa318105ff0 0x7fa318105870 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa314000c00 tx=0x7fa314030db0 comp rx=0 tx=0).stop 2024-09-13T18:12:39.785 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.984+0000 7fa31f587640 1 -- 172.21.15.114:0/3268299526 shutdown_connections 2024-09-13T18:12:39.785 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.984+0000 7fa31f587640 1 --2- 172.21.15.114:0/3268299526 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa318105ff0 0x7fa318105870 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:39.785 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.984+0000 7fa31f587640 1 -- 172.21.15.114:0/3268299526 >> 172.21.15.114:0/3268299526 conn(0x7fa3180ffc30 msgr2=0x7fa318100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:12:39.785 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.984+0000 7fa31f587640 1 -- 172.21.15.114:0/3268299526 shutdown_connections 2024-09-13T18:12:39.786 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:38.984+0000 7fa31f587640 1 -- 172.21.15.114:0/3268299526 wait complete. 2024-09-13T18:12:39.786 INFO:teuthology.orchestra.run.smithi114.stdout:Restarting the monitor... 2024-09-13T18:12:40.219 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:39 smithi114 systemd[1]: Stopping Ceph mon.a for 97e7edf2-71fb-11ef-bceb-c7b262605968... 2024-09-13T18:12:41.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:40 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mon-a[30519]: 2024-09-13T18:12:40.950+0000 7f115aa7c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T18:12:41.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:40 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mon-a[30519]: 2024-09-13T18:12:40.950+0000 7f115aa7c640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-13T18:12:41.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:41 smithi114 podman[30775]: 2024-09-13 18:12:41.083281405 +0000 UTC m=+1.139709910 container died 0da18cea228bdbde9392e50ae4a9429653f2e9584ff9444ac8f34bc8c6f98e86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mon-a, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-13T18:12:44.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:44 smithi114 podman[30775]: 2024-09-13 18:12:44.598894135 +0000 UTC m=+4.655322631 container cleanup 0da18cea228bdbde9392e50ae4a9429653f2e9584ff9444ac8f34bc8c6f98e86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-13T18:12:44.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:44 smithi114 bash[30775]: ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mon-a 2024-09-13T18:12:46.968 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:46 smithi114 podman[30787]: 2024-09-13 18:12:46.652693334 +0000 UTC m=+5.571622642 container remove 0da18cea228bdbde9392e50ae4a9429653f2e9584ff9444ac8f34bc8c6f98e86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-13T18:12:46.968 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:46 smithi114 systemd[1]: ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-13T18:12:46.968 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:46 smithi114 systemd[1]: Stopped Ceph mon.a for 97e7edf2-71fb-11ef-bceb-c7b262605968. 2024-09-13T18:12:46.969 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:46 smithi114 systemd[1]: Starting Ceph mon.a for 97e7edf2-71fb-11ef-bceb-c7b262605968... 2024-09-13T18:12:47.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:47 smithi114 podman[30877]: 2024-09-13 18:12:46.987336509 +0000 UTC m=+0.015883811 image pull 8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T18:12:48.190 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:47 smithi114 podman[30877]: 2024-09-13 18:12:47.91121229 +0000 UTC m=+0.939759579 container create cc359b54bfa5ebeb9d577333f6cb9bd13d4d122c686bca9cc0880d2710b4e368 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-13T18:12:48.190 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 podman[30877]: 2024-09-13 18:12:48.121956509 +0000 UTC m=+1.150503798 container init cc359b54bfa5ebeb9d577333f6cb9bd13d4d122c686bca9cc0880d2710b4e368 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-13T18:12:48.191 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 podman[30877]: 2024-09-13 18:12:48.125347322 +0000 UTC m=+1.153894614 container start cc359b54bfa5ebeb9d577333f6cb9bd13d4d122c686bca9cc0880d2710b4e368 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-09-13T18:12:48.191 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: set uid:gid to 167:167 (ceph:ceph) 2024-09-13T18:12:48.191 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: ceph version 18.2.4-860-g6c6b083f (6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3) reef (stable), process ceph-mon, pid 2 2024-09-13T18:12:48.191 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: pidfile_write: ignore empty --pid-file 2024-09-13T18:12:48.191 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: load: jerasure load: lrc 2024-09-13T18:12:48.192 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: RocksDB version: 7.9.2 2024-09-13T18:12:48.192 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Git sha 0 2024-09-13T18:12:48.192 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Compile date 2024-09-10 12:59:06 2024-09-13T18:12:48.192 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: DB SUMMARY 2024-09-13T18:12:48.192 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: DB Session ID: I70SZP38KDWSEAJCEJ9U 2024-09-13T18:12:48.192 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: CURRENT file: CURRENT 2024-09-13T18:12:48.193 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: IDENTITY file: IDENTITY 2024-09-13T18:12:48.193 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-13T18:12:48.193 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-13T18:12:48.193 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 99781 ; 2024-09-13T18:12:48.193 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.error_if_exists: 0 2024-09-13T18:12:48.193 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.create_if_missing: 0 2024-09-13T18:12:48.194 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.paranoid_checks: 1 2024-09-13T18:12:48.196 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-13T18:12:48.196 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-13T18:12:48.196 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-13T18:12:48.196 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.env: 0x556e1f789bc0 2024-09-13T18:12:48.196 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.fs: PosixFileSystem 2024-09-13T18:12:48.196 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.info_log: 0x556e20e78be0 2024-09-13T18:12:48.197 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.max_file_opening_threads: 16 2024-09-13T18:12:48.197 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.statistics: (nil) 2024-09-13T18:12:48.197 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.use_fsync: 0 2024-09-13T18:12:48.197 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.max_log_file_size: 0 2024-09-13T18:12:48.197 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-13T18:12:48.197 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-13T18:12:48.197 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.keep_log_file_num: 1000 2024-09-13T18:12:48.197 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.recycle_log_file_num: 0 2024-09-13T18:12:48.197 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.allow_fallocate: 1 2024-09-13T18:12:48.197 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.allow_mmap_reads: 0 2024-09-13T18:12:48.197 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.allow_mmap_writes: 0 2024-09-13T18:12:48.198 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.use_direct_reads: 0 2024-09-13T18:12:48.198 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-13T18:12:48.198 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.create_missing_column_families: 0 2024-09-13T18:12:48.198 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.db_log_dir: 2024-09-13T18:12:48.198 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.wal_dir: 2024-09-13T18:12:48.198 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-13T18:12:48.198 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-13T18:12:48.198 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-13T18:12:48.198 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-13T18:12:48.198 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-13T18:12:48.198 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-13T18:12:48.198 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.advise_random_on_open: 1 2024-09-13T18:12:48.198 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.db_write_buffer_size: 0 2024-09-13T18:12:48.199 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.write_buffer_manager: 0x556e20e86320 2024-09-13T18:12:48.199 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-13T18:12:48.199 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-13T18:12:48.199 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-13T18:12:48.199 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.rate_limiter: (nil) 2024-09-13T18:12:48.199 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-13T18:12:48.199 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.wal_recovery_mode: 2 2024-09-13T18:12:48.199 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.enable_thread_tracking: 0 2024-09-13T18:12:48.199 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.enable_pipelined_write: 0 2024-09-13T18:12:48.199 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.unordered_write: 0 2024-09-13T18:12:48.199 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-13T18:12:48.199 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-13T18:12:48.200 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-13T18:12:48.200 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-13T18:12:48.200 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.row_cache: None 2024-09-13T18:12:48.200 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.wal_filter: None 2024-09-13T18:12:48.200 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-13T18:12:48.200 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.allow_ingest_behind: 0 2024-09-13T18:12:48.200 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.two_write_queues: 0 2024-09-13T18:12:48.200 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.manual_wal_flush: 0 2024-09-13T18:12:48.200 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.wal_compression: 0 2024-09-13T18:12:48.200 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.atomic_flush: 0 2024-09-13T18:12:48.200 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-13T18:12:48.201 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-13T18:12:48.201 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-13T18:12:48.201 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.log_readahead_size: 0 2024-09-13T18:12:48.201 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-13T18:12:48.201 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.best_efforts_recovery: 0 2024-09-13T18:12:48.201 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-13T18:12:48.201 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-13T18:12:48.201 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.allow_data_in_errors: 0 2024-09-13T18:12:48.201 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.db_host_id: __hostname__ 2024-09-13T18:12:48.201 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-13T18:12:48.201 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.max_background_jobs: 2 2024-09-13T18:12:48.201 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.max_background_compactions: -1 2024-09-13T18:12:48.201 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.max_subcompactions: 1 2024-09-13T18:12:48.202 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-13T18:12:48.202 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-13T18:12:48.202 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-13T18:12:48.202 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.max_total_wal_size: 0 2024-09-13T18:12:48.202 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-13T18:12:48.202 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-13T18:12:48.202 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-13T18:12:48.202 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-13T18:12:48.202 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.max_open_files: -1 2024-09-13T18:12:48.202 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.bytes_per_sync: 0 2024-09-13T18:12:48.202 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-13T18:12:48.202 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-13T18:12:48.202 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.compaction_readahead_size: 0 2024-09-13T18:12:48.202 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.max_background_flushes: -1 2024-09-13T18:12:48.203 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Compression algorithms supported: 2024-09-13T18:12:48.203 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: kZSTD supported: 0 2024-09-13T18:12:48.203 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: kXpressCompression supported: 0 2024-09-13T18:12:48.203 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: kBZip2Compression supported: 0 2024-09-13T18:12:48.203 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-13T18:12:48.203 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: kLZ4Compression supported: 1 2024-09-13T18:12:48.203 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: kZlibCompression supported: 1 2024-09-13T18:12:48.203 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: kLZ4HCCompression supported: 1 2024-09-13T18:12:48.203 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: kSnappyCompression supported: 1 2024-09-13T18:12:48.203 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-13T18:12:48.203 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-13T18:12:48.203 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-13T18:12:48.203 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-13T18:12:48.203 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-13T18:12:48.204 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.merge_operator: 2024-09-13T18:12:48.204 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.compaction_filter: None 2024-09-13T18:12:48.204 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.compaction_filter_factory: None 2024-09-13T18:12:48.204 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.sst_partitioner_factory: None 2024-09-13T18:12:48.204 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-13T18:12:48.204 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-13T18:12:48.204 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x556e20e787e0) 2024-09-13T18:12:48.204 INFO:journalctl@ceph.mon.a.smithi114.stdout: cache_index_and_filter_blocks: 1 2024-09-13T18:12:48.204 INFO:journalctl@ceph.mon.a.smithi114.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-13T18:12:48.204 INFO:journalctl@ceph.mon.a.smithi114.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-13T18:12:48.204 INFO:journalctl@ceph.mon.a.smithi114.stdout: pin_top_level_index_and_filter: 1 2024-09-13T18:12:48.204 INFO:journalctl@ceph.mon.a.smithi114.stdout: index_type: 0 2024-09-13T18:12:48.204 INFO:journalctl@ceph.mon.a.smithi114.stdout: data_block_index_type: 0 2024-09-13T18:12:48.205 INFO:journalctl@ceph.mon.a.smithi114.stdout: index_shortening: 1 2024-09-13T18:12:48.205 INFO:journalctl@ceph.mon.a.smithi114.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-13T18:12:48.205 INFO:journalctl@ceph.mon.a.smithi114.stdout: checksum: 4 2024-09-13T18:12:48.205 INFO:journalctl@ceph.mon.a.smithi114.stdout: no_block_cache: 0 2024-09-13T18:12:48.205 INFO:journalctl@ceph.mon.a.smithi114.stdout: block_cache: 0x556e20e6d350 2024-09-13T18:12:48.205 INFO:journalctl@ceph.mon.a.smithi114.stdout: block_cache_name: BinnedLRUCache 2024-09-13T18:12:48.205 INFO:journalctl@ceph.mon.a.smithi114.stdout: block_cache_options: 2024-09-13T18:12:48.205 INFO:journalctl@ceph.mon.a.smithi114.stdout: capacity : 536870912 2024-09-13T18:12:48.205 INFO:journalctl@ceph.mon.a.smithi114.stdout: num_shard_bits : 4 2024-09-13T18:12:48.205 INFO:journalctl@ceph.mon.a.smithi114.stdout: strict_capacity_limit : 0 2024-09-13T18:12:48.205 INFO:journalctl@ceph.mon.a.smithi114.stdout: high_pri_pool_ratio: 0.000 2024-09-13T18:12:48.205 INFO:journalctl@ceph.mon.a.smithi114.stdout: block_cache_compressed: (nil) 2024-09-13T18:12:48.206 INFO:journalctl@ceph.mon.a.smithi114.stdout: persistent_cache: (nil) 2024-09-13T18:12:48.206 INFO:journalctl@ceph.mon.a.smithi114.stdout: block_size: 4096 2024-09-13T18:12:48.206 INFO:journalctl@ceph.mon.a.smithi114.stdout: block_size_deviation: 10 2024-09-13T18:12:48.206 INFO:journalctl@ceph.mon.a.smithi114.stdout: block_restart_interval: 16 2024-09-13T18:12:48.206 INFO:journalctl@ceph.mon.a.smithi114.stdout: index_block_restart_interval: 1 2024-09-13T18:12:48.206 INFO:journalctl@ceph.mon.a.smithi114.stdout: metadata_block_size: 4096 2024-09-13T18:12:48.206 INFO:journalctl@ceph.mon.a.smithi114.stdout: partition_filters: 0 2024-09-13T18:12:48.206 INFO:journalctl@ceph.mon.a.smithi114.stdout: use_delta_encoding: 1 2024-09-13T18:12:48.206 INFO:journalctl@ceph.mon.a.smithi114.stdout: filter_policy: bloomfilter 2024-09-13T18:12:48.206 INFO:journalctl@ceph.mon.a.smithi114.stdout: whole_key_filtering: 1 2024-09-13T18:12:48.206 INFO:journalctl@ceph.mon.a.smithi114.stdout: verify_compression: 0 2024-09-13T18:12:48.206 INFO:journalctl@ceph.mon.a.smithi114.stdout: read_amp_bytes_per_bit: 0 2024-09-13T18:12:48.206 INFO:journalctl@ceph.mon.a.smithi114.stdout: format_version: 5 2024-09-13T18:12:48.206 INFO:journalctl@ceph.mon.a.smithi114.stdout: enable_index_compression: 1 2024-09-13T18:12:48.206 INFO:journalctl@ceph.mon.a.smithi114.stdout: block_align: 0 2024-09-13T18:12:48.207 INFO:journalctl@ceph.mon.a.smithi114.stdout: max_auto_readahead_size: 262144 2024-09-13T18:12:48.207 INFO:journalctl@ceph.mon.a.smithi114.stdout: prepopulate_block_cache: 0 2024-09-13T18:12:48.207 INFO:journalctl@ceph.mon.a.smithi114.stdout: initial_auto_readahead_size: 8192 2024-09-13T18:12:48.207 INFO:journalctl@ceph.mon.a.smithi114.stdout: num_file_reads_for_auto_readahead: 2 2024-09-13T18:12:48.207 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.write_buffer_size: 33554432 2024-09-13T18:12:48.207 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.max_write_buffer_number: 2 2024-09-13T18:12:48.207 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.compression: NoCompression 2024-09-13T18:12:48.207 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.bottommost_compression: Disabled 2024-09-13T18:12:48.207 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.prefix_extractor: nullptr 2024-09-13T18:12:48.208 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-13T18:12:48.208 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.num_levels: 7 2024-09-13T18:12:48.208 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-13T18:12:48.208 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-13T18:12:48.208 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-13T18:12:48.208 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-13T18:12:48.208 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-13T18:12:48.208 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-13T18:12:48.208 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-13T18:12:48.208 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-13T18:12:48.208 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-13T18:12:48.208 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-13T18:12:48.208 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-13T18:12:48.208 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-13T18:12:48.209 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-13T18:12:48.209 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.compression_opts.level: 32767 2024-09-13T18:12:48.209 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.compression_opts.strategy: 0 2024-09-13T18:12:48.209 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-13T18:12:48.209 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-13T18:12:48.209 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-13T18:12:48.209 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-13T18:12:48.209 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.compression_opts.enabled: false 2024-09-13T18:12:48.209 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-13T18:12:48.209 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-13T18:12:48.209 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-13T18:12:48.209 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-13T18:12:48.210 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.target_file_size_base: 67108864 2024-09-13T18:12:48.210 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-13T18:12:48.210 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-13T18:12:48.210 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-13T18:12:48.210 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-13T18:12:48.210 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-13T18:12:48.211 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-13T18:12:48.211 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-13T18:12:48.211 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-13T18:12:48.211 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-13T18:12:48.211 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-13T18:12:48.212 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-13T18:12:48.212 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-13T18:12:48.212 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-13T18:12:48.212 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-13T18:12:48.212 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.arena_block_size: 1048576 2024-09-13T18:12:48.212 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-13T18:12:48.213 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-13T18:12:48.213 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.disable_auto_compactions: 0 2024-09-13T18:12:48.213 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-13T18:12:48.213 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-13T18:12:48.213 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-13T18:12:48.214 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-13T18:12:48.214 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-13T18:12:48.214 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-13T18:12:48.214 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-13T18:12:48.214 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-13T18:12:48.214 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-13T18:12:48.214 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-13T18:12:48.214 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-13T18:12:48.215 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.inplace_update_support: 0 2024-09-13T18:12:48.215 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-13T18:12:48.215 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-13T18:12:48.215 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-13T18:12:48.215 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-13T18:12:48.215 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.bloom_locality: 0 2024-09-13T18:12:48.215 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.max_successive_merges: 0 2024-09-13T18:12:48.215 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-13T18:12:48.215 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.paranoid_file_checks: 0 2024-09-13T18:12:48.215 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.force_consistency_checks: 1 2024-09-13T18:12:48.215 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.report_bg_io_stats: 0 2024-09-13T18:12:48.216 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.ttl: 2592000 2024-09-13T18:12:48.216 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-13T18:12:48.216 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-13T18:12:48.216 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-13T18:12:48.216 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.enable_blob_files: false 2024-09-13T18:12:48.216 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.min_blob_size: 0 2024-09-13T18:12:48.217 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.blob_file_size: 268435456 2024-09-13T18:12:48.217 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-13T18:12:48.217 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-13T18:12:48.217 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-13T18:12:48.217 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-13T18:12:48.217 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-13T18:12:48.218 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.blob_file_starting_level: 0 2024-09-13T18:12:48.218 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-13T18:12:48.218 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-13T18:12:48.218 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-13T18:12:48.218 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 69b601a9-3cd7-4872-8dce-6913a8e153dc 2024-09-13T18:12:48.218 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726251168154249, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-13T18:12:48.219 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-13T18:12:48.219 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726251168155304, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 94962, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 318, "table_properties": {"data_size": 92927, "index_size": 291, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13866, "raw_average_key_size": 50, "raw_value_size": 85386, "raw_average_value_size": 308, "num_data_blocks": 11, "num_entries": 277, "num_filter_entries": 277, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726251168, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "69b601a9-3cd7-4872-8dce-6913a8e153dc", "db_session_id": "I70SZP38KDWSEAJCEJ9U", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-13T18:12:48.219 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726251168155408, "job": 1, "event": "recovery_finished"} 2024-09-13T18:12:48.219 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-13T18:12:48.219 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-13T18:12:48.219 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x556e20e98e00 2024-09-13T18:12:48.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: DB pointer 0x556e20f8c000 2024-09-13T18:12:48.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-13T18:12:48.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-13T18:12:48.220 INFO:journalctl@ceph.mon.a.smithi114.stdout: ** DB Stats ** 2024-09-13T18:12:48.220 INFO:journalctl@ceph.mon.a.smithi114.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-13T18:12:48.220 INFO:journalctl@ceph.mon.a.smithi114.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-13T18:12:48.220 INFO:journalctl@ceph.mon.a.smithi114.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-13T18:12:48.220 INFO:journalctl@ceph.mon.a.smithi114.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-13T18:12:48.220 INFO:journalctl@ceph.mon.a.smithi114.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-13T18:12:48.221 INFO:journalctl@ceph.mon.a.smithi114.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-13T18:12:48.221 INFO:journalctl@ceph.mon.a.smithi114.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-13T18:12:48.221 INFO:journalctl@ceph.mon.a.smithi114.stdout: 2024-09-13T18:12:48.221 INFO:journalctl@ceph.mon.a.smithi114.stdout: ** Compaction Stats [default] ** 2024-09-13T18:12:48.221 INFO:journalctl@ceph.mon.a.smithi114.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-13T18:12:48.221 INFO:journalctl@ceph.mon.a.smithi114.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-13T18:12:48.221 INFO:journalctl@ceph.mon.a.smithi114.stdout: L0 2/0 94.60 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 131.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-13T18:12:48.221 INFO:journalctl@ceph.mon.a.smithi114.stdout: Sum 2/0 94.60 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 131.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-13T18:12:48.221 INFO:journalctl@ceph.mon.a.smithi114.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 131.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-13T18:12:48.221 INFO:journalctl@ceph.mon.a.smithi114.stdout: 2024-09-13T18:12:48.221 INFO:journalctl@ceph.mon.a.smithi114.stdout: ** Compaction Stats [default] ** 2024-09-13T18:12:48.222 INFO:journalctl@ceph.mon.a.smithi114.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-13T18:12:48.222 INFO:journalctl@ceph.mon.a.smithi114.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-13T18:12:48.222 INFO:journalctl@ceph.mon.a.smithi114.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 131.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-13T18:12:48.222 INFO:journalctl@ceph.mon.a.smithi114.stdout: 2024-09-13T18:12:48.222 INFO:journalctl@ceph.mon.a.smithi114.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-13T18:12:48.222 INFO:journalctl@ceph.mon.a.smithi114.stdout: 2024-09-13T18:12:48.222 INFO:journalctl@ceph.mon.a.smithi114.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-13T18:12:48.222 INFO:journalctl@ceph.mon.a.smithi114.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-13T18:12:48.222 INFO:journalctl@ceph.mon.a.smithi114.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-13T18:12:48.222 INFO:journalctl@ceph.mon.a.smithi114.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-13T18:12:48.222 INFO:journalctl@ceph.mon.a.smithi114.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-13T18:12:48.222 INFO:journalctl@ceph.mon.a.smithi114.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-13T18:12:48.222 INFO:journalctl@ceph.mon.a.smithi114.stdout: Cumulative compaction: 0.00 GB write, 27.14 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-13T18:12:48.222 INFO:journalctl@ceph.mon.a.smithi114.stdout: Interval compaction: 0.00 GB write, 27.14 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-13T18:12:48.223 INFO:journalctl@ceph.mon.a.smithi114.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-13T18:12:48.223 INFO:journalctl@ceph.mon.a.smithi114.stdout: Block cache BinnedLRUCache@0x556e20e6d350#2 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-13T18:12:48.223 INFO:journalctl@ceph.mon.a.smithi114.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-13T18:12:48.223 INFO:journalctl@ceph.mon.a.smithi114.stdout: 2024-09-13T18:12:48.223 INFO:journalctl@ceph.mon.a.smithi114.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-13T18:12:48.223 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: starting mon.a rank 0 at public addrs [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] at bind addrs [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 2024-09-13T18:12:48.224 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: mon.a@-1(???) e1 preinit fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 2024-09-13T18:12:48.224 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: mon.a@-1(???).mds e1 new map 2024-09-13T18:12:48.224 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: mon.a@-1(???).mds e1 print_map 2024-09-13T18:12:48.224 INFO:journalctl@ceph.mon.a.smithi114.stdout: e1 2024-09-13T18:12:48.224 INFO:journalctl@ceph.mon.a.smithi114.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-13T18:12:48.224 INFO:journalctl@ceph.mon.a.smithi114.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-13T18:12:48.224 INFO:journalctl@ceph.mon.a.smithi114.stdout: legacy client fscid: -1 2024-09-13T18:12:48.225 INFO:journalctl@ceph.mon.a.smithi114.stdout: 2024-09-13T18:12:48.225 INFO:journalctl@ceph.mon.a.smithi114.stdout: No filesystems configured 2024-09-13T18:12:48.225 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-13T18:12:48.225 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-13T18:12:48.225 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-13T18:12:48.225 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-13T18:12:48.226 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-13T18:12:48.226 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 bash[30877]: cc359b54bfa5ebeb9d577333f6cb9bd13d4d122c686bca9cc0880d2710b4e368 2024-09-13T18:12:48.226 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 systemd[1]: Started Ceph mon.a for 97e7edf2-71fb-11ef-bceb-c7b262605968. 2024-09-13T18:12:48.226 INFO:teuthology.orchestra.run.smithi114.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-13T18:12:48.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-13T18:12:48.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: monmap e1: 1 mons at {a=[v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-13T18:12:48.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: fsmap 2024-09-13T18:12:48.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: osdmap e1: 0 total, 0 up, 0 in 2024-09-13T18:12:48.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:48 smithi114 ceph-mon[30897]: mgrmap e1: no daemons active 2024-09-13T18:12:49.211 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:48.588+0000 7ff8bb12b640 1 Processor -- start 2024-09-13T18:12:49.212 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:48.588+0000 7ff8bb12b640 1 -- start start 2024-09-13T18:12:49.212 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:48.588+0000 7ff8bb12b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8b407bee0 0x7ff8b407a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:49.212 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:48.588+0000 7ff8bb12b640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b407a910 con 0x7ff8b407bee0 2024-09-13T18:12:49.212 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:48.588+0000 7ff8b8ec7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8b407bee0 0x7ff8b407a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:49.212 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:48.588+0000 7ff8b8ec7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8b407bee0 0x7ff8b407a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33062/0 (socket says 172.21.15.114:33062) 2024-09-13T18:12:49.212 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:48.588+0000 7ff8b8ec7640 1 -- 172.21.15.114:0/1155018157 learned_addr learned my addr 172.21.15.114:0/1155018157 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:12:49.212 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:48.589+0000 7ff8b8ec7640 1 -- 172.21.15.114:0/1155018157 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b407aa50 con 0x7ff8b407bee0 2024-09-13T18:12:49.212 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:48.590+0000 7ff8b8ec7640 1 --2- 172.21.15.114:0/1155018157 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8b407bee0 0x7ff8b407a340 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8009820 tx=0x7ff8a802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d007e4c02ce2b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:49.212 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:48.591+0000 7ff8b37fe640 1 -- 172.21.15.114:0/1155018157 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff8a802ae70 con 0x7ff8b407bee0 2024-09-13T18:12:49.212 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:48.591+0000 7ff8b37fe640 1 -- 172.21.15.114:0/1155018157 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7ff8a80328b0 con 0x7ff8b407bee0 2024-09-13T18:12:49.213 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:48.591+0000 7ff8b37fe640 1 -- 172.21.15.114:0/1155018157 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff8a8030850 con 0x7ff8b407bee0 2024-09-13T18:12:49.213 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:48.591+0000 7ff8bb12b640 1 -- 172.21.15.114:0/1155018157 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8b407bee0 msgr2=0x7ff8b407a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:49.213 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:48.591+0000 7ff8bb12b640 1 --2- 172.21.15.114:0/1155018157 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8b407bee0 0x7ff8b407a340 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8009820 tx=0x7ff8a802a270 comp rx=0 tx=0).stop 2024-09-13T18:12:49.213 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:48.591+0000 7ff8bb12b640 1 -- 172.21.15.114:0/1155018157 shutdown_connections 2024-09-13T18:12:49.213 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:48.591+0000 7ff8bb12b640 1 --2- 172.21.15.114:0/1155018157 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8b407bee0 0x7ff8b407a340 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:49.213 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:48.591+0000 7ff8bb12b640 1 -- 172.21.15.114:0/1155018157 >> 172.21.15.114:0/1155018157 conn(0x7ff8b4100de0 msgr2=0x7ff8b4103200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:12:49.213 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:48.591+0000 7ff8bb12b640 1 -- 172.21.15.114:0/1155018157 shutdown_connections 2024-09-13T18:12:49.213 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:48.591+0000 7ff8bb12b640 1 -- 172.21.15.114:0/1155018157 wait complete. 2024-09-13T18:12:49.213 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:48.592+0000 7ff8bb12b640 1 Processor -- start 2024-09-13T18:12:49.213 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:48.592+0000 7ff8bb12b640 1 -- start start 2024-09-13T18:12:49.213 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:48.592+0000 7ff8bb12b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8b407bee0 0x7ff8b419b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:49.213 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:48.592+0000 7ff8bb12b640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b419ba30 con 0x7ff8b407bee0 2024-09-13T18:12:49.213 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:48.592+0000 7ff8b8ec7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8b407bee0 0x7ff8b419b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:49.214 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:48.592+0000 7ff8b8ec7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8b407bee0 0x7ff8b419b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33064/0 (socket says 172.21.15.114:33064) 2024-09-13T18:12:49.214 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:48.592+0000 7ff8b8ec7640 1 -- 172.21.15.114:0/750416534 learned_addr learned my addr 172.21.15.114:0/750416534 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:12:49.214 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:48.592+0000 7ff8b8ec7640 1 -- 172.21.15.114:0/750416534 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8a80094d0 con 0x7ff8b407bee0 2024-09-13T18:12:49.214 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:48.593+0000 7ff8b8ec7640 1 --2- 172.21.15.114:0/750416534 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8b407bee0 0x7ff8b419b4f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff8a80097f0 tx=0x7ff8a8030d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:49.214 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:48.593+0000 7ff8b1ffb640 1 -- 172.21.15.114:0/750416534 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff8a8032c30 con 0x7ff8b407bee0 2024-09-13T18:12:49.214 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:48.593+0000 7ff8b1ffb640 1 -- 172.21.15.114:0/750416534 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7ff8a8032d90 con 0x7ff8b407bee0 2024-09-13T18:12:49.214 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:48.593+0000 7ff8b1ffb640 1 -- 172.21.15.114:0/750416534 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff8a803a5b0 con 0x7ff8b407bee0 2024-09-13T18:12:49.214 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:48.593+0000 7ff8bb12b640 1 -- 172.21.15.114:0/750416534 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8b419bc30 con 0x7ff8b407bee0 2024-09-13T18:12:49.214 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:48.593+0000 7ff8bb12b640 1 -- 172.21.15.114:0/750416534 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8b419c0d0 con 0x7ff8b407bee0 2024-09-13T18:12:49.214 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:48.593+0000 7ff8b1ffb640 1 -- 172.21.15.114:0/750416534 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7ff8a8039040 con 0x7ff8b407bee0 2024-09-13T18:12:49.214 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:48.593+0000 7ff8b1ffb640 1 -- 172.21.15.114:0/750416534 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff8a8044b20 con 0x7ff8b407bee0 2024-09-13T18:12:49.214 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:48.594+0000 7ff8bb12b640 1 -- 172.21.15.114:0/750416534 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8b410b980 con 0x7ff8b407bee0 2024-09-13T18:12:49.215 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:48.596+0000 7ff8b1ffb640 1 -- 172.21.15.114:0/750416534 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7ff8a803f020 con 0x7ff8b407bee0 2024-09-13T18:12:49.215 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:48.652+0000 7ff8bb12b640 1 -- 172.21.15.114:0/750416534 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7ff8b419e9b0 con 0x7ff8b407bee0 2024-09-13T18:12:49.215 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:48.655+0000 7ff8b1ffb640 1 -- 172.21.15.114:0/750416534 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7ff8a8037070 con 0x7ff8b407bee0 2024-09-13T18:12:49.215 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:48.656+0000 7ff8bb12b640 1 -- 172.21.15.114:0/750416534 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8b407bee0 msgr2=0x7ff8b419b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:49.215 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:48.656+0000 7ff8bb12b640 1 --2- 172.21.15.114:0/750416534 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8b407bee0 0x7ff8b419b4f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff8a80097f0 tx=0x7ff8a8030d40 comp rx=0 tx=0).stop 2024-09-13T18:12:49.215 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:48.656+0000 7ff8bb12b640 1 -- 172.21.15.114:0/750416534 shutdown_connections 2024-09-13T18:12:49.215 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:48.656+0000 7ff8bb12b640 1 --2- 172.21.15.114:0/750416534 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8b407bee0 0x7ff8b419b4f0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:49.215 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:48.656+0000 7ff8bb12b640 1 -- 172.21.15.114:0/750416534 >> 172.21.15.114:0/750416534 conn(0x7ff8b4100de0 msgr2=0x7ff8b4101a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:12:49.215 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:48.656+0000 7ff8bb12b640 1 -- 172.21.15.114:0/750416534 shutdown_connections 2024-09-13T18:12:49.215 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:48.656+0000 7ff8bb12b640 1 -- 172.21.15.114:0/750416534 wait complete. 2024-09-13T18:12:49.215 INFO:teuthology.orchestra.run.smithi114.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-13T18:12:49.221 INFO:teuthology.orchestra.run.smithi114.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-13T18:12:49.222 INFO:teuthology.orchestra.run.smithi114.stdout:Creating mgr... 2024-09-13T18:12:49.222 INFO:teuthology.orchestra.run.smithi114.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-13T18:12:49.222 INFO:teuthology.orchestra.run.smithi114.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-13T18:12:49.461 INFO:teuthology.orchestra.run.smithi114.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mgr.x 2024-09-13T18:12:49.461 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stderr Failed to reset failed state of unit ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mgr.x.service: Unit ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mgr.x.service not loaded. 2024-09-13T18:12:49.614 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-97e7edf2-71fb-11ef-bceb-c7b262605968.target.wants/ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@.service. 2024-09-13T18:12:49.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:49 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/750416534' entity='client.admin' 2024-09-13T18:12:50.127 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:12:50 smithi114 podman[31111]: 2024-09-13 18:12:50.03561476 +0000 UTC m=+0.304480870 container start 254a2146bf84fe453b2ded5f75cabb1b97370b7d4fdb9f869001b71d0888a226 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-13T18:12:50.127 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:12:50 smithi114 bash[31111]: 254a2146bf84fe453b2ded5f75cabb1b97370b7d4fdb9f869001b71d0888a226 2024-09-13T18:12:50.140 INFO:teuthology.orchestra.run.smithi114.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-13T18:12:50.140 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stdout disabled 2024-09-13T18:12:50.147 INFO:teuthology.orchestra.run.smithi114.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-13T18:12:50.147 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stdout inactive 2024-09-13T18:12:50.147 INFO:teuthology.orchestra.run.smithi114.stdout:firewalld.service is not enabled 2024-09-13T18:12:50.147 INFO:teuthology.orchestra.run.smithi114.stdout:Not possible to enable service . firewalld.service is not available 2024-09-13T18:12:50.156 INFO:teuthology.orchestra.run.smithi114.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-13T18:12:50.156 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stdout disabled 2024-09-13T18:12:50.163 INFO:teuthology.orchestra.run.smithi114.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-13T18:12:50.163 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stdout inactive 2024-09-13T18:12:50.164 INFO:teuthology.orchestra.run.smithi114.stdout:firewalld.service is not enabled 2024-09-13T18:12:50.164 INFO:teuthology.orchestra.run.smithi114.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-13T18:12:50.164 INFO:teuthology.orchestra.run.smithi114.stdout:Waiting for mgr to start... 2024-09-13T18:12:50.164 INFO:teuthology.orchestra.run.smithi114.stdout:Waiting for mgr... 2024-09-13T18:12:50.412 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:12:50 smithi114 systemd[1]: Started Ceph mgr.x for 97e7edf2-71fb-11ef-bceb-c7b262605968. 2024-09-13T18:12:50.412 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:12:50 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:12:50.153+0000 7f8bf772a040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T18:12:50.412 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:12:50 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:12:50.218+0000 7f8bf772a040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T18:12:50.413 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:12:50 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:12:50.366+0000 7f8bf772a040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T18:12:50.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:50 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/3060861704' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-13T18:12:50.720 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:12:50 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:12:50.436+0000 7f8bf772a040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T18:12:50.720 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:12:50 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:12:50.587+0000 7f8bf772a040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T18:12:51.049 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:12:50 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:12:50.731+0000 7f8bf772a040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T18:12:51.049 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:12:51 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:12:51.046+0000 7f8bf772a040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T18:12:51.343 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:12:51 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:12:51.108+0000 7f8bf772a040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T18:12:51.343 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:12:51 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:12:51.291+0000 7f8bf772a040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T18:12:51.345 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout 2024-09-13T18:12:51.357 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout { 2024-09-13T18:12:51.357 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "fsid": "97e7edf2-71fb-11ef-bceb-c7b262605968", 2024-09-13T18:12:51.357 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "health": { 2024-09-13T18:12:51.357 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-13T18:12:51.357 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-13T18:12:51.357 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-13T18:12:51.357 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-09-13T18:12:51.357 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-13T18:12:51.357 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-13T18:12:51.357 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout 0 2024-09-13T18:12:51.358 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout ], 2024-09-13T18:12:51.358 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-13T18:12:51.358 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "a" 2024-09-13T18:12:51.358 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout ], 2024-09-13T18:12:51.358 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-13T18:12:51.358 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-13T18:12:51.358 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T18:12:51.358 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-09-13T18:12:51.358 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-13T18:12:51.358 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-09-13T18:12:51.358 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-13T18:12:51.358 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T18:12:51.358 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-13T18:12:51.358 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-13T18:12:51.359 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-13T18:12:51.359 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-13T18:12:51.359 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-13T18:12:51.359 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-13T18:12:51.359 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-09-13T18:12:51.359 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-13T18:12:51.359 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-13T18:12:51.359 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-13T18:12:51.359 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-13T18:12:51.359 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-13T18:12:51.359 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-13T18:12:51.359 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-13T18:12:51.359 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-13T18:12:51.360 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-13T18:12:51.360 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-09-13T18:12:51.360 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-13T18:12:51.360 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T18:12:51.360 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-13T18:12:51.360 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-13T18:12:51.360 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-09-13T18:12:51.360 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-13T18:12:51.360 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-13T18:12:51.360 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-13T18:12:51.360 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-13T18:12:51.360 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-13T18:12:51.360 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-13T18:12:51.361 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "restful" 2024-09-13T18:12:51.361 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout ], 2024-09-13T18:12:51.361 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-13T18:12:51.361 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-09-13T18:12:51.361 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-13T18:12:51.361 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T18:12:51.361 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "modified": "2024-09-13T18:12:36.361064+0000", 2024-09-13T18:12:51.361 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-13T18:12:51.361 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-09-13T18:12:51.361 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-13T18:12:51.361 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout } 2024-09-13T18:12:51.361 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:50.557+0000 7ffa01a2f640 1 Processor -- start 2024-09-13T18:12:51.361 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:50.557+0000 7ffa01a2f640 1 -- start start 2024-09-13T18:12:51.362 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:50.557+0000 7ffa01a2f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9fc103de0 0x7ff9fc1041e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:51.362 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:50.557+0000 7ffa01a2f640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9fc079ed0 con 0x7ff9fc103de0 2024-09-13T18:12:51.362 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:50.558+0000 7ff9faffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9fc103de0 0x7ff9fc1041e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:51.362 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:50.558+0000 7ff9faffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9fc103de0 0x7ff9fc1041e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33098/0 (socket says 172.21.15.114:33098) 2024-09-13T18:12:51.362 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:50.558+0000 7ff9faffd640 1 -- 172.21.15.114:0/553630519 learned_addr learned my addr 172.21.15.114:0/553630519 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:12:51.362 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:50.558+0000 7ff9faffd640 1 -- 172.21.15.114:0/553630519 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9fc104720 con 0x7ff9fc103de0 2024-09-13T18:12:51.362 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:50.559+0000 7ff9faffd640 1 --2- 172.21.15.114:0/553630519 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9fc103de0 0x7ff9fc1041e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec009820 tx=0x7ff9ec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68c4cfb37f2e9819 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:51.362 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:50.559+0000 7ff9fa7fc640 1 -- 172.21.15.114:0/553630519 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff9ec02ae70 con 0x7ff9fc103de0 2024-09-13T18:12:51.362 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:50.559+0000 7ff9fa7fc640 1 -- 172.21.15.114:0/553630519 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7ff9ec0328b0 con 0x7ff9fc103de0 2024-09-13T18:12:51.362 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:50.559+0000 7ff9fa7fc640 1 -- 172.21.15.114:0/553630519 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff9ec030850 con 0x7ff9fc103de0 2024-09-13T18:12:51.362 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:50.559+0000 7ffa01a2f640 1 -- 172.21.15.114:0/553630519 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9fc103de0 msgr2=0x7ff9fc1041e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:51.362 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:50.559+0000 7ffa01a2f640 1 --2- 172.21.15.114:0/553630519 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9fc103de0 0x7ff9fc1041e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec009820 tx=0x7ff9ec02a270 comp rx=0 tx=0).stop 2024-09-13T18:12:51.362 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:50.559+0000 7ffa01a2f640 1 -- 172.21.15.114:0/553630519 shutdown_connections 2024-09-13T18:12:51.363 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:50.559+0000 7ffa01a2f640 1 --2- 172.21.15.114:0/553630519 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9fc103de0 0x7ff9fc1041e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:51.363 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:50.559+0000 7ffa01a2f640 1 -- 172.21.15.114:0/553630519 >> 172.21.15.114:0/553630519 conn(0x7ff9fc0ffc30 msgr2=0x7ff9fc102050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:12:51.363 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:50.560+0000 7ffa01a2f640 1 -- 172.21.15.114:0/553630519 shutdown_connections 2024-09-13T18:12:51.363 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:50.560+0000 7ffa01a2f640 1 -- 172.21.15.114:0/553630519 wait complete. 2024-09-13T18:12:51.363 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:50.560+0000 7ffa01a2f640 1 Processor -- start 2024-09-13T18:12:51.363 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:50.560+0000 7ffa01a2f640 1 -- start start 2024-09-13T18:12:51.363 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:50.560+0000 7ffa01a2f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9fc103de0 0x7ff9fc19b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:51.363 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:50.560+0000 7ffa01a2f640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9fc19b950 con 0x7ff9fc103de0 2024-09-13T18:12:51.363 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:50.561+0000 7ff9faffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9fc103de0 0x7ff9fc19b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:51.363 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:50.561+0000 7ff9faffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9fc103de0 0x7ff9fc19b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33104/0 (socket says 172.21.15.114:33104) 2024-09-13T18:12:51.363 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:50.561+0000 7ff9faffd640 1 -- 172.21.15.114:0/3060861704 learned_addr learned my addr 172.21.15.114:0/3060861704 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:12:51.363 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:50.561+0000 7ff9faffd640 1 -- 172.21.15.114:0/3060861704 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9ec0094d0 con 0x7ff9fc103de0 2024-09-13T18:12:51.363 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:50.561+0000 7ff9faffd640 1 --2- 172.21.15.114:0/3060861704 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9fc103de0 0x7ff9fc19b410 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec030560 tx=0x7ff9ec030d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:51.364 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:50.561+0000 7ffa00a2d640 1 -- 172.21.15.114:0/3060861704 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff9ec032d60 con 0x7ff9fc103de0 2024-09-13T18:12:51.364 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:50.561+0000 7ffa00a2d640 1 -- 172.21.15.114:0/3060861704 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7ff9ec03b8a0 con 0x7ff9fc103de0 2024-09-13T18:12:51.364 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:50.561+0000 7ffa00a2d640 1 -- 172.21.15.114:0/3060861704 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff9ec03a740 con 0x7ff9fc103de0 2024-09-13T18:12:51.364 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:50.561+0000 7ffa01a2f640 1 -- 172.21.15.114:0/3060861704 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9fc19bb50 con 0x7ff9fc103de0 2024-09-13T18:12:51.364 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:50.561+0000 7ffa01a2f640 1 -- 172.21.15.114:0/3060861704 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9fc19bff0 con 0x7ff9fc103de0 2024-09-13T18:12:51.364 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:50.561+0000 7ffa00a2d640 1 -- 172.21.15.114:0/3060861704 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7ff9ec039040 con 0x7ff9fc103de0 2024-09-13T18:12:51.364 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:50.562+0000 7ffa00a2d640 1 -- 172.21.15.114:0/3060861704 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff9ec044c50 con 0x7ff9fc103de0 2024-09-13T18:12:51.364 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:50.562+0000 7ffa01a2f640 1 -- 172.21.15.114:0/3060861704 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9fc105d70 con 0x7ff9fc103de0 2024-09-13T18:12:51.364 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:50.564+0000 7ffa00a2d640 1 -- 172.21.15.114:0/3060861704 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7ff9ec03f020 con 0x7ff9fc103de0 2024-09-13T18:12:51.364 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:50.624+0000 7ffa01a2f640 1 -- 172.21.15.114:0/3060861704 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7ff9fc1057f0 con 0x7ff9fc103de0 2024-09-13T18:12:51.364 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:50.624+0000 7ffa00a2d640 1 -- 172.21.15.114:0/3060861704 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7ff9ec037070 con 0x7ff9fc103de0 2024-09-13T18:12:51.364 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:50.625+0000 7ffa01a2f640 1 -- 172.21.15.114:0/3060861704 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9fc103de0 msgr2=0x7ff9fc19b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:51.364 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:50.625+0000 7ffa01a2f640 1 --2- 172.21.15.114:0/3060861704 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9fc103de0 0x7ff9fc19b410 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec030560 tx=0x7ff9ec030d90 comp rx=0 tx=0).stop 2024-09-13T18:12:51.365 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:50.625+0000 7ffa01a2f640 1 -- 172.21.15.114:0/3060861704 shutdown_connections 2024-09-13T18:12:51.365 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:50.625+0000 7ffa01a2f640 1 --2- 172.21.15.114:0/3060861704 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9fc103de0 0x7ff9fc19b410 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:51.365 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:50.625+0000 7ffa01a2f640 1 -- 172.21.15.114:0/3060861704 >> 172.21.15.114:0/3060861704 conn(0x7ff9fc0ffc30 msgr2=0x7ff9fc18fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:12:51.365 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:50.625+0000 7ffa01a2f640 1 -- 172.21.15.114:0/3060861704 shutdown_connections 2024-09-13T18:12:51.365 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:50.625+0000 7ffa01a2f640 1 -- 172.21.15.114:0/3060861704 wait complete. 2024-09-13T18:12:51.365 INFO:teuthology.orchestra.run.smithi114.stdout:mgr not available, waiting (1/15)... 2024-09-13T18:12:52.470 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:12:52 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:12:52.151+0000 7f8bf772a040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T18:12:52.470 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:12:52 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:12:52.215+0000 7f8bf772a040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T18:12:53.172 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:12:52 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:12:52.920+0000 7f8bf772a040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-13T18:12:53.172 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:12:53 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:12:53.031+0000 7f8bf772a040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T18:12:53.173 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:12:53 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:12:53.170+0000 7f8bf772a040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T18:12:53.451 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:12:53 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:12:53.230+0000 7f8bf772a040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T18:12:53.451 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:12:53 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:12:53.376+0000 7f8bf772a040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-13T18:12:53.451 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:12:53 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:12:53.449+0000 7f8bf772a040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T18:12:53.719 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:12:53 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:12:53.509+0000 7f8bf772a040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T18:12:53.720 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:12:53 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:12:53.667+0000 7f8bf772a040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T18:12:54.000 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:12:53 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:12:53.733+0000 7f8bf772a040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T18:12:54.001 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:12:53 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:12:53.798+0000 7f8bf772a040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T18:12:54.001 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:12:53 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:12:53.880+0000 7f8bf772a040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T18:12:54.001 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:12:53 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:12:53.939+0000 7f8bf772a040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T18:12:54.001 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:12:53 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:12:53.998+0000 7f8bf772a040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T18:12:55.166 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:12:54 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:12:54.775+0000 7f8bf772a040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T18:12:55.167 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:54 smithi114 ceph-mon[30897]: Activating manager daemon x 2024-09-13T18:12:55.167 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:54 smithi114 ceph-mon[30897]: mgrmap e2: x(active, starting, since 0.00304973s) 2024-09-13T18:12:55.167 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:54 smithi114 ceph-mon[30897]: from='mgr.14100 172.21.15.114:0/3247087317' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-13T18:12:55.167 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:54 smithi114 ceph-mon[30897]: from='mgr.14100 172.21.15.114:0/3247087317' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-13T18:12:55.167 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:54 smithi114 ceph-mon[30897]: from='mgr.14100 172.21.15.114:0/3247087317' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-13T18:12:55.167 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:54 smithi114 ceph-mon[30897]: from='mgr.14100 172.21.15.114:0/3247087317' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T18:12:55.167 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:54 smithi114 ceph-mon[30897]: from='mgr.14100 172.21.15.114:0/3247087317' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-13T18:12:55.167 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:54 smithi114 ceph-mon[30897]: Manager daemon x is now available 2024-09-13T18:12:55.167 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:54 smithi114 ceph-mon[30897]: from='mgr.14100 172.21.15.114:0/3247087317' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-13T18:12:55.167 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:54 smithi114 ceph-mon[30897]: from='mgr.14100 172.21.15.114:0/3247087317' entity='mgr.x' 2024-09-13T18:12:55.167 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:54 smithi114 ceph-mon[30897]: from='mgr.14100 172.21.15.114:0/3247087317' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-13T18:12:55.168 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:54 smithi114 ceph-mon[30897]: from='mgr.14100 172.21.15.114:0/3247087317' entity='mgr.x' 2024-09-13T18:12:55.168 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:54 smithi114 ceph-mon[30897]: from='mgr.14100 172.21.15.114:0/3247087317' entity='mgr.x' 2024-09-13T18:12:56.934 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:56 smithi114 ceph-mon[30897]: mgrmap e3: x(active, since 1.00716s) 2024-09-13T18:12:56.934 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:56 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/657868039' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-13T18:12:56.938 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout 2024-09-13T18:12:56.938 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout { 2024-09-13T18:12:56.938 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "fsid": "97e7edf2-71fb-11ef-bceb-c7b262605968", 2024-09-13T18:12:56.939 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "health": { 2024-09-13T18:12:56.939 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-13T18:12:56.939 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-13T18:12:56.939 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-13T18:12:56.939 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-09-13T18:12:56.939 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-13T18:12:56.939 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-13T18:12:56.939 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout 0 2024-09-13T18:12:56.939 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout ], 2024-09-13T18:12:56.939 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-13T18:12:56.939 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "a" 2024-09-13T18:12:56.939 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout ], 2024-09-13T18:12:56.939 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-09-13T18:12:56.939 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-13T18:12:56.940 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T18:12:56.940 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-09-13T18:12:56.940 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-13T18:12:56.940 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-09-13T18:12:56.940 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-13T18:12:56.940 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T18:12:56.940 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-13T18:12:56.940 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-13T18:12:56.940 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-13T18:12:56.948 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-13T18:12:56.948 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-13T18:12:56.948 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-13T18:12:56.949 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-09-13T18:12:56.949 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-13T18:12:56.949 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-13T18:12:56.949 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-13T18:12:56.949 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-13T18:12:56.949 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-13T18:12:56.949 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-13T18:12:56.949 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-13T18:12:56.949 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-13T18:12:56.949 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-13T18:12:56.949 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-09-13T18:12:56.949 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-13T18:12:56.950 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T18:12:56.950 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-13T18:12:56.950 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-13T18:12:56.950 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-09-13T18:12:56.950 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-13T18:12:56.950 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-13T18:12:56.950 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-13T18:12:56.950 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-13T18:12:56.950 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-13T18:12:56.950 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-13T18:12:56.950 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "restful" 2024-09-13T18:12:56.950 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout ], 2024-09-13T18:12:56.950 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-13T18:12:56.950 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-09-13T18:12:56.951 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-13T18:12:56.951 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-13T18:12:56.951 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "modified": "2024-09-13T18:12:36.361064+0000", 2024-09-13T18:12:56.951 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-13T18:12:56.951 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-09-13T18:12:56.951 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-13T18:12:56.951 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout } 2024-09-13T18:12:56.951 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:56.122+0000 7fd66e386640 1 Processor -- start 2024-09-13T18:12:56.951 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:56.122+0000 7fd66e386640 1 -- start start 2024-09-13T18:12:56.951 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:56.122+0000 7fd66e386640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd66807bec0 0x7fd66807a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:56.951 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:56.122+0000 7fd66e386640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd66807a8f0 con 0x7fd66807bec0 2024-09-13T18:12:56.951 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:56.122+0000 7fd667fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd66807bec0 0x7fd66807a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:56.951 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:56.122+0000 7fd667fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd66807bec0 0x7fd66807a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50782/0 (socket says 172.21.15.114:50782) 2024-09-13T18:12:56.952 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:56.122+0000 7fd667fff640 1 -- 172.21.15.114:0/1274451600 learned_addr learned my addr 172.21.15.114:0/1274451600 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:12:56.952 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:56.122+0000 7fd667fff640 1 -- 172.21.15.114:0/1274451600 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd66807aa30 con 0x7fd66807bec0 2024-09-13T18:12:56.952 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:56.123+0000 7fd667fff640 1 --2- 172.21.15.114:0/1274451600 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd66807bec0 0x7fd66807a320 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd658009820 tx=0x7fd65802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9411e16b73878f00 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:56.952 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:56.124+0000 7fd666ffd640 1 -- 172.21.15.114:0/1274451600 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd65802ae70 con 0x7fd66807bec0 2024-09-13T18:12:56.952 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:56.124+0000 7fd666ffd640 1 -- 172.21.15.114:0/1274451600 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fd6580328b0 con 0x7fd66807bec0 2024-09-13T18:12:56.952 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:56.124+0000 7fd666ffd640 1 -- 172.21.15.114:0/1274451600 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd658030850 con 0x7fd66807bec0 2024-09-13T18:12:56.952 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:56.124+0000 7fd66e386640 1 -- 172.21.15.114:0/1274451600 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd66807bec0 msgr2=0x7fd66807a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:56.952 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:56.124+0000 7fd66e386640 1 --2- 172.21.15.114:0/1274451600 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd66807bec0 0x7fd66807a320 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd658009820 tx=0x7fd65802a270 comp rx=0 tx=0).stop 2024-09-13T18:12:56.952 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:56.124+0000 7fd66e386640 1 -- 172.21.15.114:0/1274451600 shutdown_connections 2024-09-13T18:12:56.952 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:56.124+0000 7fd66e386640 1 --2- 172.21.15.114:0/1274451600 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd66807bec0 0x7fd66807a320 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:56.952 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:56.124+0000 7fd66e386640 1 -- 172.21.15.114:0/1274451600 >> 172.21.15.114:0/1274451600 conn(0x7fd668100db0 msgr2=0x7fd6681031d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:12:56.952 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:56.124+0000 7fd66e386640 1 -- 172.21.15.114:0/1274451600 shutdown_connections 2024-09-13T18:12:56.952 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:56.124+0000 7fd66e386640 1 -- 172.21.15.114:0/1274451600 wait complete. 2024-09-13T18:12:56.953 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:56.125+0000 7fd66e386640 1 Processor -- start 2024-09-13T18:12:56.953 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:56.125+0000 7fd66e386640 1 -- start start 2024-09-13T18:12:56.953 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:56.125+0000 7fd66e386640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd66807bec0 0x7fd66819b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:56.953 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:56.125+0000 7fd66e386640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd66819b9f0 con 0x7fd66807bec0 2024-09-13T18:12:56.953 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:56.126+0000 7fd667fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd66807bec0 0x7fd66819b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:56.953 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:56.126+0000 7fd667fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd66807bec0 0x7fd66819b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50790/0 (socket says 172.21.15.114:50790) 2024-09-13T18:12:56.953 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:56.126+0000 7fd667fff640 1 -- 172.21.15.114:0/657868039 learned_addr learned my addr 172.21.15.114:0/657868039 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:12:56.953 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:56.126+0000 7fd667fff640 1 -- 172.21.15.114:0/657868039 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6580094d0 con 0x7fd66807bec0 2024-09-13T18:12:56.953 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:56.126+0000 7fd667fff640 1 --2- 172.21.15.114:0/657868039 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd66807bec0 0x7fd66819b4b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd6580097f0 tx=0x7fd658030d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:56.954 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:56.126+0000 7fd6657fa640 1 -- 172.21.15.114:0/657868039 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd658032c30 con 0x7fd66807bec0 2024-09-13T18:12:56.954 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:56.126+0000 7fd6657fa640 1 -- 172.21.15.114:0/657868039 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fd658032d90 con 0x7fd66807bec0 2024-09-13T18:12:56.954 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:56.126+0000 7fd66e386640 1 -- 172.21.15.114:0/657868039 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd66819bbf0 con 0x7fd66807bec0 2024-09-13T18:12:56.954 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:56.126+0000 7fd6657fa640 1 -- 172.21.15.114:0/657868039 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd65803a5b0 con 0x7fd66807bec0 2024-09-13T18:12:56.954 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:56.126+0000 7fd66e386640 1 -- 172.21.15.114:0/657868039 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd66819c090 con 0x7fd66807bec0 2024-09-13T18:12:56.954 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:56.127+0000 7fd6657fa640 1 -- 172.21.15.114:0/657868039 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 3) v1 ==== 45424+0+0 (secure 0 0 0) 0x7fd658039040 con 0x7fd66807bec0 2024-09-13T18:12:56.954 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:56.127+0000 7fd6657fa640 1 --2- 172.21.15.114:0/657868039 >> [v2:172.21.15.114:6800/830738794,v1:172.21.15.114:6801/830738794] conn(0x7fd6500384b0 0x7fd65003a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:56.954 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:56.127+0000 7fd6657fa640 1 -- 172.21.15.114:0/657868039 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd65806c1e0 con 0x7fd66807bec0 2024-09-13T18:12:56.954 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:56.127+0000 7fd6677fe640 1 --2- 172.21.15.114:0/657868039 >> [v2:172.21.15.114:6800/830738794,v1:172.21.15.114:6801/830738794] conn(0x7fd6500384b0 0x7fd65003a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:56.954 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:56.128+0000 7fd66e386640 1 -- 172.21.15.114:0/657868039 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd66807a3a0 con 0x7fd66807bec0 2024-09-13T18:12:56.954 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:56.128+0000 7fd6677fe640 1 --2- 172.21.15.114:0/657868039 >> [v2:172.21.15.114:6800/830738794,v1:172.21.15.114:6801/830738794] conn(0x7fd6500384b0 0x7fd65003a970 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd65c0098c0 tx=0x7fd65c006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:56.954 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:56.133+0000 7fd6657fa640 1 -- 172.21.15.114:0/657868039 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fd658037070 con 0x7fd66807bec0 2024-09-13T18:12:56.954 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:56.353+0000 7fd66e386640 1 -- 172.21.15.114:0/657868039 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fd66819ebe0 con 0x7fd66807bec0 2024-09-13T18:12:56.955 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:56.353+0000 7fd6657fa640 1 -- 172.21.15.114:0/657868039 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1237 (secure 0 0 0) 0x7fd65803f030 con 0x7fd66807bec0 2024-09-13T18:12:56.955 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:56.356+0000 7fd66e386640 1 -- 172.21.15.114:0/657868039 >> [v2:172.21.15.114:6800/830738794,v1:172.21.15.114:6801/830738794] conn(0x7fd6500384b0 msgr2=0x7fd65003a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:56.955 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:56.356+0000 7fd66e386640 1 --2- 172.21.15.114:0/657868039 >> [v2:172.21.15.114:6800/830738794,v1:172.21.15.114:6801/830738794] conn(0x7fd6500384b0 0x7fd65003a970 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd65c0098c0 tx=0x7fd65c006db0 comp rx=0 tx=0).stop 2024-09-13T18:12:56.955 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:56.356+0000 7fd66e386640 1 -- 172.21.15.114:0/657868039 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd66807bec0 msgr2=0x7fd66819b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:56.955 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:56.356+0000 7fd66e386640 1 --2- 172.21.15.114:0/657868039 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd66807bec0 0x7fd66819b4b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd6580097f0 tx=0x7fd658030d40 comp rx=0 tx=0).stop 2024-09-13T18:12:56.955 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:56.356+0000 7fd66e386640 1 -- 172.21.15.114:0/657868039 shutdown_connections 2024-09-13T18:12:56.955 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:56.356+0000 7fd66e386640 1 --2- 172.21.15.114:0/657868039 >> [v2:172.21.15.114:6800/830738794,v1:172.21.15.114:6801/830738794] conn(0x7fd6500384b0 0x7fd65003a970 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:56.955 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:56.356+0000 7fd66e386640 1 --2- 172.21.15.114:0/657868039 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd66807bec0 0x7fd66819b4b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:56.955 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:56.356+0000 7fd66e386640 1 -- 172.21.15.114:0/657868039 >> 172.21.15.114:0/657868039 conn(0x7fd668100db0 msgr2=0x7fd6681019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:12:56.955 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:56.356+0000 7fd66e386640 1 -- 172.21.15.114:0/657868039 shutdown_connections 2024-09-13T18:12:56.955 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:56.356+0000 7fd66e386640 1 -- 172.21.15.114:0/657868039 wait complete. 2024-09-13T18:12:56.955 INFO:teuthology.orchestra.run.smithi114.stdout:mgr is available 2024-09-13T18:12:58.041 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:57 smithi114 ceph-mon[30897]: mgrmap e4: x(active, since 2s) 2024-09-13T18:12:58.041 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:57 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/1630295216' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-13T18:12:58.043 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout 2024-09-13T18:12:58.050 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout [global] 2024-09-13T18:12:58.050 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout fsid = 97e7edf2-71fb-11ef-bceb-c7b262605968 2024-09-13T18:12:58.050 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-13T18:12:58.051 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-13T18:12:58.051 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-13T18:12:58.051 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-13T18:12:58.051 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout 2024-09-13T18:12:58.051 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-13T18:12:58.051 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-13T18:12:58.051 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout 2024-09-13T18:12:58.051 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout [osd] 2024-09-13T18:12:58.051 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-13T18:12:58.051 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-13T18:12:58.051 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-13T18:12:58.051 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-13T18:12:58.051 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-13T18:12:58.052 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:57.336+0000 7fbd1bcf3640 1 Processor -- start 2024-09-13T18:12:58.052 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:57.336+0000 7fbd1bcf3640 1 -- start start 2024-09-13T18:12:58.052 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:57.336+0000 7fbd1bcf3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbd14106090 0x7fbd14108480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:58.052 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:57.336+0000 7fbd1bcf3640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd14079e30 con 0x7fbd14106090 2024-09-13T18:12:58.052 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:57.337+0000 7fbd19a8f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbd14106090 0x7fbd14108480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:58.052 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:57.337+0000 7fbd19a8f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbd14106090 0x7fbd14108480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50806/0 (socket says 172.21.15.114:50806) 2024-09-13T18:12:58.052 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:57.337+0000 7fbd19a8f640 1 -- 172.21.15.114:0/1975756916 learned_addr learned my addr 172.21.15.114:0/1975756916 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:12:58.052 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:57.337+0000 7fbd19a8f640 1 -- 172.21.15.114:0/1975756916 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd14079f70 con 0x7fbd14106090 2024-09-13T18:12:58.052 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:57.338+0000 7fbd19a8f640 1 --2- 172.21.15.114:0/1975756916 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbd14106090 0x7fbd14108480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbd04009820 tx=0x7fbd0402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1709eafcbc46004 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:58.053 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:57.338+0000 7fbd18a8d640 1 -- 172.21.15.114:0/1975756916 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbd0402ae70 con 0x7fbd14106090 2024-09-13T18:12:58.053 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:57.338+0000 7fbd18a8d640 1 -- 172.21.15.114:0/1975756916 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fbd040328b0 con 0x7fbd14106090 2024-09-13T18:12:58.053 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:57.338+0000 7fbd18a8d640 1 -- 172.21.15.114:0/1975756916 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbd04030850 con 0x7fbd14106090 2024-09-13T18:12:58.053 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:57.339+0000 7fbd1bcf3640 1 -- 172.21.15.114:0/1975756916 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbd14106090 msgr2=0x7fbd14108480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:58.053 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:57.339+0000 7fbd1bcf3640 1 --2- 172.21.15.114:0/1975756916 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbd14106090 0x7fbd14108480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbd04009820 tx=0x7fbd0402a270 comp rx=0 tx=0).stop 2024-09-13T18:12:58.053 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:57.339+0000 7fbd1bcf3640 1 -- 172.21.15.114:0/1975756916 shutdown_connections 2024-09-13T18:12:58.053 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:57.339+0000 7fbd1bcf3640 1 --2- 172.21.15.114:0/1975756916 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbd14106090 0x7fbd14108480 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:58.053 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:57.339+0000 7fbd1bcf3640 1 -- 172.21.15.114:0/1975756916 >> 172.21.15.114:0/1975756916 conn(0x7fbd140ffc30 msgr2=0x7fbd14102050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:12:58.053 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:57.339+0000 7fbd1bcf3640 1 -- 172.21.15.114:0/1975756916 shutdown_connections 2024-09-13T18:12:58.053 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:57.339+0000 7fbd1bcf3640 1 -- 172.21.15.114:0/1975756916 wait complete. 2024-09-13T18:12:58.053 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:57.339+0000 7fbd1bcf3640 1 Processor -- start 2024-09-13T18:12:58.053 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:57.339+0000 7fbd1bcf3640 1 -- start start 2024-09-13T18:12:58.053 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:57.340+0000 7fbd1bcf3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbd14106090 0x7fbd14199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:58.054 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:57.340+0000 7fbd1bcf3640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd14199770 con 0x7fbd14106090 2024-09-13T18:12:58.054 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:57.340+0000 7fbd19a8f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbd14106090 0x7fbd14199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:58.054 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:57.340+0000 7fbd19a8f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbd14106090 0x7fbd14199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50816/0 (socket says 172.21.15.114:50816) 2024-09-13T18:12:58.054 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:57.340+0000 7fbd19a8f640 1 -- 172.21.15.114:0/1630295216 learned_addr learned my addr 172.21.15.114:0/1630295216 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:12:58.054 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:57.340+0000 7fbd19a8f640 1 -- 172.21.15.114:0/1630295216 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd040094d0 con 0x7fbd14106090 2024-09-13T18:12:58.054 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:57.340+0000 7fbd19a8f640 1 --2- 172.21.15.114:0/1630295216 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbd14106090 0x7fbd14199230 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fbd04000c00 tx=0x7fbd04030db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:58.054 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:57.340+0000 7fbd0affd640 1 -- 172.21.15.114:0/1630295216 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbd04032d60 con 0x7fbd14106090 2024-09-13T18:12:58.054 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:57.340+0000 7fbd0affd640 1 -- 172.21.15.114:0/1630295216 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fbd0403b8a0 con 0x7fbd14106090 2024-09-13T18:12:58.054 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:57.340+0000 7fbd1bcf3640 1 -- 172.21.15.114:0/1630295216 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd14199970 con 0x7fbd14106090 2024-09-13T18:12:58.054 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:57.340+0000 7fbd0affd640 1 -- 172.21.15.114:0/1630295216 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbd0403a640 con 0x7fbd14106090 2024-09-13T18:12:58.054 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:57.340+0000 7fbd1bcf3640 1 -- 172.21.15.114:0/1630295216 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd14199e10 con 0x7fbd14106090 2024-09-13T18:12:58.054 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:57.341+0000 7fbd0affd640 1 -- 172.21.15.114:0/1630295216 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 4) v1 ==== 45530+0+0 (secure 0 0 0) 0x7fbd04039040 con 0x7fbd14106090 2024-09-13T18:12:58.055 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:57.341+0000 7fbd1bcf3640 1 -- 172.21.15.114:0/1630295216 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd141096b0 con 0x7fbd14106090 2024-09-13T18:12:58.055 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:57.341+0000 7fbd0affd640 1 --2- 172.21.15.114:0/1630295216 >> [v2:172.21.15.114:6800/830738794,v1:172.21.15.114:6801/830738794] conn(0x7fbd00038620 0x7fbd0003aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:58.055 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:57.342+0000 7fbd0affd640 1 -- 172.21.15.114:0/1630295216 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fbd0406b6c0 con 0x7fbd14106090 2024-09-13T18:12:58.055 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:57.342+0000 7fbd1928e640 1 --2- 172.21.15.114:0/1630295216 >> [v2:172.21.15.114:6800/830738794,v1:172.21.15.114:6801/830738794] conn(0x7fbd00038620 0x7fbd0003aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:58.055 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:57.342+0000 7fbd1928e640 1 --2- 172.21.15.114:0/1630295216 >> [v2:172.21.15.114:6800/830738794,v1:172.21.15.114:6801/830738794] conn(0x7fbd00038620 0x7fbd0003aae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbd100098c0 tx=0x7fbd10006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:58.055 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:57.346+0000 7fbd0affd640 1 -- 172.21.15.114:0/1630295216 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fbd04031b50 con 0x7fbd14106090 2024-09-13T18:12:58.055 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:57.509+0000 7fbd1bcf3640 1 -- 172.21.15.114:0/1630295216 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fbd141098f0 con 0x7fbd14106090 2024-09-13T18:12:58.055 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:57.510+0000 7fbd0affd640 1 -- 172.21.15.114:0/1630295216 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7fbd0403f020 con 0x7fbd14106090 2024-09-13T18:12:58.055 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:57.512+0000 7fbd1bcf3640 1 -- 172.21.15.114:0/1630295216 >> [v2:172.21.15.114:6800/830738794,v1:172.21.15.114:6801/830738794] conn(0x7fbd00038620 msgr2=0x7fbd0003aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:58.055 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:57.512+0000 7fbd1bcf3640 1 --2- 172.21.15.114:0/1630295216 >> [v2:172.21.15.114:6800/830738794,v1:172.21.15.114:6801/830738794] conn(0x7fbd00038620 0x7fbd0003aae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbd100098c0 tx=0x7fbd10006db0 comp rx=0 tx=0).stop 2024-09-13T18:12:58.055 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:57.512+0000 7fbd1bcf3640 1 -- 172.21.15.114:0/1630295216 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbd14106090 msgr2=0x7fbd14199230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:58.055 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:57.512+0000 7fbd1bcf3640 1 --2- 172.21.15.114:0/1630295216 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbd14106090 0x7fbd14199230 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fbd04000c00 tx=0x7fbd04030db0 comp rx=0 tx=0).stop 2024-09-13T18:12:58.055 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:57.512+0000 7fbd1bcf3640 1 -- 172.21.15.114:0/1630295216 shutdown_connections 2024-09-13T18:12:58.056 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:57.512+0000 7fbd1bcf3640 1 --2- 172.21.15.114:0/1630295216 >> [v2:172.21.15.114:6800/830738794,v1:172.21.15.114:6801/830738794] conn(0x7fbd00038620 0x7fbd0003aae0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:58.056 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:57.512+0000 7fbd1bcf3640 1 --2- 172.21.15.114:0/1630295216 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbd14106090 0x7fbd14199230 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:58.056 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:57.512+0000 7fbd1bcf3640 1 -- 172.21.15.114:0/1630295216 >> 172.21.15.114:0/1630295216 conn(0x7fbd140ffc30 msgr2=0x7fbd14100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:12:58.056 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:57.512+0000 7fbd1bcf3640 1 -- 172.21.15.114:0/1630295216 shutdown_connections 2024-09-13T18:12:58.056 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:57.512+0000 7fbd1bcf3640 1 -- 172.21.15.114:0/1630295216 wait complete. 2024-09-13T18:12:58.056 INFO:teuthology.orchestra.run.smithi114.stdout:Enabling cephadm module... 2024-09-13T18:12:58.937 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:58 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/1104503796' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-13T18:12:58.937 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:12:58 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: ignoring --setuser ceph since I am not root 2024-09-13T18:12:58.938 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:12:58 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: ignoring --setgroup ceph since I am not root 2024-09-13T18:12:58.938 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:12:58 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:12:58.935+0000 7f1d9303b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T18:12:59.209 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:12:58 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:12:58.991+0000 7f1d9303b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T18:12:59.209 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:12:59 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:12:59.141+0000 7f1d9303b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T18:12:59.210 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:12:59 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:12:59.207+0000 7f1d9303b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T18:12:59.325 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:58.428+0000 7f6ff0204640 1 Processor -- start 2024-09-13T18:12:59.325 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:58.428+0000 7f6ff0204640 1 -- start start 2024-09-13T18:12:59.325 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:58.429+0000 7f6ff0204640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6fe8105590 0x7f6fe8105990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:59.332 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:58.429+0000 7f6ff0204640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe8105f60 con 0x7f6fe8105590 2024-09-13T18:12:59.333 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:58.429+0000 7f6fedfa0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6fe8105590 0x7f6fe8105990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:59.333 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:58.429+0000 7f6fedfa0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6fe8105590 0x7f6fe8105990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50822/0 (socket says 172.21.15.114:50822) 2024-09-13T18:12:59.333 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:58.429+0000 7f6fedfa0640 1 -- 172.21.15.114:0/2391427971 learned_addr learned my addr 172.21.15.114:0/2391427971 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:12:59.333 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:58.429+0000 7f6fedfa0640 1 -- 172.21.15.114:0/2391427971 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe8106790 con 0x7f6fe8105590 2024-09-13T18:12:59.333 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:58.430+0000 7f6fedfa0640 1 --2- 172.21.15.114:0/2391427971 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6fe8105590 0x7f6fe8105990 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4009820 tx=0x7f6fe402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66718ee2b8f2e64c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:59.333 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:58.430+0000 7f6fecf9e640 1 -- 172.21.15.114:0/2391427971 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6fe402ae70 con 0x7f6fe8105590 2024-09-13T18:12:59.334 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:58.430+0000 7f6fecf9e640 1 -- 172.21.15.114:0/2391427971 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f6fe40328b0 con 0x7f6fe8105590 2024-09-13T18:12:59.334 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:58.430+0000 7f6fecf9e640 1 -- 172.21.15.114:0/2391427971 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6fe4030850 con 0x7f6fe8105590 2024-09-13T18:12:59.334 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:58.430+0000 7f6ff0204640 1 -- 172.21.15.114:0/2391427971 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6fe8105590 msgr2=0x7f6fe8105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:59.334 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:58.430+0000 7f6ff0204640 1 --2- 172.21.15.114:0/2391427971 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6fe8105590 0x7f6fe8105990 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4009820 tx=0x7f6fe402a270 comp rx=0 tx=0).stop 2024-09-13T18:12:59.334 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:58.430+0000 7f6ff0204640 1 -- 172.21.15.114:0/2391427971 shutdown_connections 2024-09-13T18:12:59.334 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:58.430+0000 7f6ff0204640 1 --2- 172.21.15.114:0/2391427971 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6fe8105590 0x7f6fe8105990 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:59.334 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:58.430+0000 7f6ff0204640 1 -- 172.21.15.114:0/2391427971 >> 172.21.15.114:0/2391427971 conn(0x7f6fe8100d60 msgr2=0x7f6fe8103180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:12:59.334 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:58.431+0000 7f6ff0204640 1 -- 172.21.15.114:0/2391427971 shutdown_connections 2024-09-13T18:12:59.334 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:58.431+0000 7f6ff0204640 1 -- 172.21.15.114:0/2391427971 wait complete. 2024-09-13T18:12:59.334 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:58.431+0000 7f6ff0204640 1 Processor -- start 2024-09-13T18:12:59.334 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:58.431+0000 7f6ff0204640 1 -- start start 2024-09-13T18:12:59.334 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:58.431+0000 7f6ff0204640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6fe8105590 0x7f6fe81993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:59.334 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:58.431+0000 7f6ff0204640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe8199920 con 0x7f6fe8105590 2024-09-13T18:12:59.334 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:58.431+0000 7f6fedfa0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6fe8105590 0x7f6fe81993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:59.335 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:58.431+0000 7f6fedfa0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6fe8105590 0x7f6fe81993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50830/0 (socket says 172.21.15.114:50830) 2024-09-13T18:12:59.335 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:58.431+0000 7f6fedfa0640 1 -- 172.21.15.114:0/1104503796 learned_addr learned my addr 172.21.15.114:0/1104503796 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:12:59.335 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:58.432+0000 7f6fedfa0640 1 -- 172.21.15.114:0/1104503796 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe40094d0 con 0x7f6fe8105590 2024-09-13T18:12:59.335 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:58.432+0000 7f6fedfa0640 1 --2- 172.21.15.114:0/1104503796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6fe8105590 0x7f6fe81993e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4030560 tx=0x7f6fe4030d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:59.335 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:58.432+0000 7f6fdeffd640 1 -- 172.21.15.114:0/1104503796 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6fe4032d60 con 0x7f6fe8105590 2024-09-13T18:12:59.335 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:58.432+0000 7f6ff0204640 1 -- 172.21.15.114:0/1104503796 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fe8107610 con 0x7f6fe8105590 2024-09-13T18:12:59.335 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:58.432+0000 7f6fdeffd640 1 -- 172.21.15.114:0/1104503796 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f6fe403b8a0 con 0x7f6fe8105590 2024-09-13T18:12:59.335 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:58.432+0000 7f6fdeffd640 1 -- 172.21.15.114:0/1104503796 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6fe403a680 con 0x7f6fe8105590 2024-09-13T18:12:59.335 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:58.432+0000 7f6ff0204640 1 -- 172.21.15.114:0/1104503796 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fe8199e30 con 0x7f6fe8105590 2024-09-13T18:12:59.335 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:58.433+0000 7f6fdeffd640 1 -- 172.21.15.114:0/1104503796 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 4) v1 ==== 45530+0+0 (secure 0 0 0) 0x7f6fe4039040 con 0x7f6fe8105590 2024-09-13T18:12:59.335 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:58.433+0000 7f6ff0204640 1 -- 172.21.15.114:0/1104503796 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fe8105a10 con 0x7f6fe8105590 2024-09-13T18:12:59.335 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:58.433+0000 7f6fdeffd640 1 --2- 172.21.15.114:0/1104503796 >> [v2:172.21.15.114:6800/830738794,v1:172.21.15.114:6801/830738794] conn(0x7f6fd40385d0 0x7f6fd403aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:59.335 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:58.433+0000 7f6fdeffd640 1 -- 172.21.15.114:0/1104503796 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6fe406b5e0 con 0x7f6fe8105590 2024-09-13T18:12:59.336 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:58.433+0000 7f6fed79f640 1 --2- 172.21.15.114:0/1104503796 >> [v2:172.21.15.114:6800/830738794,v1:172.21.15.114:6801/830738794] conn(0x7f6fd40385d0 0x7f6fd403aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:59.336 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:58.433+0000 7f6fed79f640 1 --2- 172.21.15.114:0/1104503796 >> [v2:172.21.15.114:6800/830738794,v1:172.21.15.114:6801/830738794] conn(0x7f6fd40385d0 0x7f6fd403aa90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6fd80098c0 tx=0x7f6fd8006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:59.336 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:58.438+0000 7f6fdeffd640 1 -- 172.21.15.114:0/1104503796 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f6fe403b430 con 0x7f6fe8105590 2024-09-13T18:12:59.336 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:58.640+0000 7f6ff0204640 1 -- 172.21.15.114:0/1104503796 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f6fe819cb10 con 0x7f6fe8105590 2024-09-13T18:12:59.336 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:58.788+0000 7f6fdeffd640 1 -- 172.21.15.114:0/1104503796 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mgrmap(e 5) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f6fe404b300 con 0x7f6fe8105590 2024-09-13T18:12:59.336 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:58.789+0000 7f6fdeffd640 1 -- 172.21.15.114:0/1104503796 <== mon.0 v2:172.21.15.114:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f6fe406b340 con 0x7f6fe8105590 2024-09-13T18:12:59.336 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:58.791+0000 7f6ff0204640 1 -- 172.21.15.114:0/1104503796 >> [v2:172.21.15.114:6800/830738794,v1:172.21.15.114:6801/830738794] conn(0x7f6fd40385d0 msgr2=0x7f6fd403aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:59.336 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:58.791+0000 7f6ff0204640 1 --2- 172.21.15.114:0/1104503796 >> [v2:172.21.15.114:6800/830738794,v1:172.21.15.114:6801/830738794] conn(0x7f6fd40385d0 0x7f6fd403aa90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6fd80098c0 tx=0x7f6fd8006db0 comp rx=0 tx=0).stop 2024-09-13T18:12:59.336 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:58.791+0000 7f6ff0204640 1 -- 172.21.15.114:0/1104503796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6fe8105590 msgr2=0x7f6fe81993e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:59.336 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:58.791+0000 7f6ff0204640 1 --2- 172.21.15.114:0/1104503796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6fe8105590 0x7f6fe81993e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4030560 tx=0x7f6fe4030d90 comp rx=0 tx=0).stop 2024-09-13T18:12:59.336 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:58.791+0000 7f6ff0204640 1 -- 172.21.15.114:0/1104503796 shutdown_connections 2024-09-13T18:12:59.336 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:58.791+0000 7f6ff0204640 1 --2- 172.21.15.114:0/1104503796 >> [v2:172.21.15.114:6800/830738794,v1:172.21.15.114:6801/830738794] conn(0x7f6fd40385d0 0x7f6fd403aa90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:59.337 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:58.791+0000 7f6ff0204640 1 --2- 172.21.15.114:0/1104503796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6fe8105590 0x7f6fe81993e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:59.337 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:58.791+0000 7f6ff0204640 1 -- 172.21.15.114:0/1104503796 >> 172.21.15.114:0/1104503796 conn(0x7f6fe8100d60 msgr2=0x7f6fe8101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:12:59.337 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:58.791+0000 7f6ff0204640 1 -- 172.21.15.114:0/1104503796 shutdown_connections 2024-09-13T18:12:59.337 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:58.791+0000 7f6ff0204640 1 -- 172.21.15.114:0/1104503796 wait complete. 2024-09-13T18:12:59.469 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:12:59 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:12:59.365+0000 7f1d9303b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T18:12:59.742 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:12:59 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:12:59.507+0000 7f1d9303b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T18:13:00.074 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:59 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/1104503796' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-13T18:13:00.074 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:12:59 smithi114 ceph-mon[30897]: mgrmap e5: x(active, since 4s) 2024-09-13T18:13:00.074 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:12:59 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:12:59.822+0000 7f1d9303b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T18:13:00.075 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:12:59 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:12:59.884+0000 7f1d9303b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T18:13:00.075 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:00 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:13:00.071+0000 7f1d9303b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T18:13:00.630 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout { 2024-09-13T18:13:00.631 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-13T18:13:00.631 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-13T18:13:00.631 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-13T18:13:00.631 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-13T18:13:00.631 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout } 2024-09-13T18:13:00.631 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:59.748+0000 7f5847e32640 1 Processor -- start 2024-09-13T18:13:00.631 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:59.749+0000 7f5847e32640 1 -- start start 2024-09-13T18:13:00.631 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:59.749+0000 7f5847e32640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5840104480 0x7f5840104880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:00.631 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:59.749+0000 7f5847e32640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5840104e50 con 0x7f5840104480 2024-09-13T18:13:00.632 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:59.749+0000 7f5845bce640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5840104480 0x7f5840104880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:00.632 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:59.749+0000 7f5845bce640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5840104480 0x7f5840104880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50850/0 (socket says 172.21.15.114:50850) 2024-09-13T18:13:00.632 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:59.749+0000 7f5845bce640 1 -- 172.21.15.114:0/2778764931 learned_addr learned my addr 172.21.15.114:0/2778764931 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:13:00.632 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:59.749+0000 7f5845bce640 1 -- 172.21.15.114:0/2778764931 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5840105680 con 0x7f5840104480 2024-09-13T18:13:00.632 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:59.750+0000 7f5845bce640 1 --2- 172.21.15.114:0/2778764931 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5840104480 0x7f5840104880 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f583c009820 tx=0x7f583c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=227c5a329a1a90c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:00.632 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:59.750+0000 7f5844bcc640 1 -- 172.21.15.114:0/2778764931 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f583c02ae70 con 0x7f5840104480 2024-09-13T18:13:00.633 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:59.750+0000 7f5844bcc640 1 -- 172.21.15.114:0/2778764931 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f583c0328b0 con 0x7f5840104480 2024-09-13T18:13:00.633 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:59.750+0000 7f5844bcc640 1 -- 172.21.15.114:0/2778764931 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f583c030850 con 0x7f5840104480 2024-09-13T18:13:00.633 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:59.751+0000 7f5847e32640 1 -- 172.21.15.114:0/2778764931 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5840104480 msgr2=0x7f5840104880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:00.633 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:59.751+0000 7f5847e32640 1 --2- 172.21.15.114:0/2778764931 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5840104480 0x7f5840104880 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f583c009820 tx=0x7f583c02a270 comp rx=0 tx=0).stop 2024-09-13T18:13:00.633 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:59.751+0000 7f5847e32640 1 -- 172.21.15.114:0/2778764931 shutdown_connections 2024-09-13T18:13:00.633 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:59.751+0000 7f5847e32640 1 --2- 172.21.15.114:0/2778764931 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5840104480 0x7f5840104880 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:00.633 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:59.751+0000 7f5847e32640 1 -- 172.21.15.114:0/2778764931 >> 172.21.15.114:0/2778764931 conn(0x7f58400ffc30 msgr2=0x7f5840102050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:00.633 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:59.751+0000 7f5847e32640 1 -- 172.21.15.114:0/2778764931 shutdown_connections 2024-09-13T18:13:00.633 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:59.751+0000 7f5847e32640 1 -- 172.21.15.114:0/2778764931 wait complete. 2024-09-13T18:13:00.633 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:59.751+0000 7f5847e32640 1 Processor -- start 2024-09-13T18:13:00.633 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:59.751+0000 7f5847e32640 1 -- start start 2024-09-13T18:13:00.633 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:59.752+0000 7f5847e32640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5840104480 0x7f5840199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:00.633 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:59.752+0000 7f5847e32640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5840199970 con 0x7f5840104480 2024-09-13T18:13:00.634 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:59.752+0000 7f5845bce640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5840104480 0x7f5840199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:00.634 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:59.752+0000 7f5845bce640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5840104480 0x7f5840199430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50862/0 (socket says 172.21.15.114:50862) 2024-09-13T18:13:00.634 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:59.752+0000 7f5845bce640 1 -- 172.21.15.114:0/3132925980 learned_addr learned my addr 172.21.15.114:0/3132925980 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:13:00.634 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:59.752+0000 7f5845bce640 1 -- 172.21.15.114:0/3132925980 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f583c0094d0 con 0x7f5840104480 2024-09-13T18:13:00.634 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:59.752+0000 7f5845bce640 1 --2- 172.21.15.114:0/3132925980 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5840104480 0x7f5840199430 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f583c02a790 tx=0x7f583c030db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:00.634 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:59.752+0000 7f5836ffd640 1 -- 172.21.15.114:0/3132925980 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f583c032d60 con 0x7f5840104480 2024-09-13T18:13:00.634 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:59.752+0000 7f5836ffd640 1 -- 172.21.15.114:0/3132925980 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f583c03b8a0 con 0x7f5840104480 2024-09-13T18:13:00.634 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:59.752+0000 7f5836ffd640 1 -- 172.21.15.114:0/3132925980 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f583c03a640 con 0x7f5840104480 2024-09-13T18:13:00.634 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:59.752+0000 7f5847e32640 1 -- 172.21.15.114:0/3132925980 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5840106500 con 0x7f5840104480 2024-09-13T18:13:00.634 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:59.752+0000 7f5847e32640 1 -- 172.21.15.114:0/3132925980 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5840199e80 con 0x7f5840104480 2024-09-13T18:13:00.634 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:59.753+0000 7f5836ffd640 1 -- 172.21.15.114:0/3132925980 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f583c039040 con 0x7f5840104480 2024-09-13T18:13:00.634 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:59.753+0000 7f5836ffd640 1 --2- 172.21.15.114:0/3132925980 >> [v2:172.21.15.114:6800/830738794,v1:172.21.15.114:6801/830738794] conn(0x7f582c038670 0x7f582c03ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:00.634 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:59.753+0000 7f5836ffd640 1 -- 172.21.15.114:0/3132925980 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f583c06b6e0 con 0x7f5840104480 2024-09-13T18:13:00.635 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:59.753+0000 7f5847e32640 1 -- 172.21.15.114:0/3132925980 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5840104900 con 0x7f5840104480 2024-09-13T18:13:00.635 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:59.753+0000 7f58453cd640 1 -- 172.21.15.114:0/3132925980 >> [v2:172.21.15.114:6800/830738794,v1:172.21.15.114:6801/830738794] conn(0x7f582c038670 msgr2=0x7f582c03ab30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.114:6800/830738794 2024-09-13T18:13:00.635 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:59.753+0000 7f58453cd640 1 --2- 172.21.15.114:0/3132925980 >> [v2:172.21.15.114:6800/830738794,v1:172.21.15.114:6801/830738794] conn(0x7f582c038670 0x7f582c03ab30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:13:00.635 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:59.758+0000 7f5836ffd640 1 -- 172.21.15.114:0/3132925980 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f583c03f070 con 0x7f5840104480 2024-09-13T18:13:00.635 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:59.954+0000 7f58453cd640 1 -- 172.21.15.114:0/3132925980 >> [v2:172.21.15.114:6800/830738794,v1:172.21.15.114:6801/830738794] conn(0x7f582c038670 msgr2=0x7f582c03ab30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.114:6800/830738794 2024-09-13T18:13:00.635 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:59.954+0000 7f58453cd640 1 --2- 172.21.15.114:0/3132925980 >> [v2:172.21.15.114:6800/830738794,v1:172.21.15.114:6801/830738794] conn(0x7f582c038670 0x7f582c03ab30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:13:00.635 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:59.971+0000 7f5847e32640 1 -- 172.21.15.114:0/3132925980 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f584002c480 con 0x7f5840104480 2024-09-13T18:13:00.635 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:59.971+0000 7f5836ffd640 1 -- 172.21.15.114:0/3132925980 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f583c037030 con 0x7f5840104480 2024-09-13T18:13:00.636 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:59.973+0000 7f5847e32640 1 -- 172.21.15.114:0/3132925980 >> [v2:172.21.15.114:6800/830738794,v1:172.21.15.114:6801/830738794] conn(0x7f582c038670 msgr2=0x7f582c03ab30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:13:00.636 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:59.973+0000 7f5847e32640 1 --2- 172.21.15.114:0/3132925980 >> [v2:172.21.15.114:6800/830738794,v1:172.21.15.114:6801/830738794] conn(0x7f582c038670 0x7f582c03ab30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:00.636 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:59.973+0000 7f5847e32640 1 -- 172.21.15.114:0/3132925980 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5840104480 msgr2=0x7f5840199430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:00.636 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:59.973+0000 7f5847e32640 1 --2- 172.21.15.114:0/3132925980 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5840104480 0x7f5840199430 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f583c02a790 tx=0x7f583c030db0 comp rx=0 tx=0).stop 2024-09-13T18:13:00.636 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:59.973+0000 7f5847e32640 1 -- 172.21.15.114:0/3132925980 shutdown_connections 2024-09-13T18:13:00.636 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:59.973+0000 7f5847e32640 1 --2- 172.21.15.114:0/3132925980 >> [v2:172.21.15.114:6800/830738794,v1:172.21.15.114:6801/830738794] conn(0x7f582c038670 0x7f582c03ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:00.636 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:59.973+0000 7f5847e32640 1 --2- 172.21.15.114:0/3132925980 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5840104480 0x7f5840199430 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:00.636 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:59.973+0000 7f5847e32640 1 -- 172.21.15.114:0/3132925980 >> 172.21.15.114:0/3132925980 conn(0x7f58400ffc30 msgr2=0x7f5840100b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:00.636 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:59.973+0000 7f5847e32640 1 -- 172.21.15.114:0/3132925980 shutdown_connections 2024-09-13T18:13:00.636 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:12:59.974+0000 7f5847e32640 1 -- 172.21.15.114:0/3132925980 wait complete. 2024-09-13T18:13:00.636 INFO:teuthology.orchestra.run.smithi114.stdout:Waiting for the mgr to restart... 2024-09-13T18:13:00.636 INFO:teuthology.orchestra.run.smithi114.stdout:Waiting for mgr epoch 5... 2024-09-13T18:13:00.915 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:00 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/3132925980' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-13T18:13:01.198 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:00 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:13:00.947+0000 7f1d9303b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T18:13:01.198 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:01 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:13:01.016+0000 7f1d9303b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T18:13:01.965 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:01 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:13:01.713+0000 7f1d9303b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-13T18:13:01.965 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:01 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:13:01.824+0000 7f1d9303b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T18:13:02.220 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:01 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:13:01.961+0000 7f1d9303b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T18:13:02.222 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:02 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:13:02.021+0000 7f1d9303b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T18:13:02.223 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:02 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:13:02.164+0000 7f1d9303b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-13T18:13:02.505 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:02 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:13:02.229+0000 7f1d9303b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T18:13:02.505 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:02 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:13:02.287+0000 7f1d9303b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T18:13:02.505 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:02 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:13:02.439+0000 7f1d9303b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T18:13:02.505 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:02 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:13:02.501+0000 7f1d9303b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T18:13:02.764 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:02 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:13:02.566+0000 7f1d9303b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T18:13:02.764 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:02 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:13:02.646+0000 7f1d9303b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T18:13:02.764 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:02 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:13:02.703+0000 7f1d9303b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T18:13:02.764 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:02 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:13:02.760+0000 7f1d9303b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T18:13:03.969 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:03 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:13:03.518+0000 7f1d9303b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T18:13:03.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:03 smithi114 ceph-mon[30897]: Active manager daemon x restarted 2024-09-13T18:13:03.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:03 smithi114 ceph-mon[30897]: Activating manager daemon x 2024-09-13T18:13:03.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:03 smithi114 ceph-mon[30897]: osdmap e2: 0 total, 0 up, 0 in 2024-09-13T18:13:03.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:03 smithi114 ceph-mon[30897]: mgrmap e6: x(active, starting, since 0.00367042s) 2024-09-13T18:13:03.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:03 smithi114 ceph-mon[30897]: from='mgr.14116 172.21.15.114:0/3845812472' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T18:13:03.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:03 smithi114 ceph-mon[30897]: from='mgr.14116 172.21.15.114:0/3845812472' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-13T18:13:03.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:03 smithi114 ceph-mon[30897]: from='mgr.14116 172.21.15.114:0/3845812472' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-13T18:13:03.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:03 smithi114 ceph-mon[30897]: from='mgr.14116 172.21.15.114:0/3845812472' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-13T18:13:03.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:03 smithi114 ceph-mon[30897]: from='mgr.14116 172.21.15.114:0/3845812472' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-13T18:13:03.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:03 smithi114 ceph-mon[30897]: Manager daemon x is now available 2024-09-13T18:13:03.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:03 smithi114 ceph-mon[30897]: from='mgr.14116 172.21.15.114:0/3845812472' entity='mgr.x' 2024-09-13T18:13:03.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:03 smithi114 ceph-mon[30897]: from='mgr.14116 172.21.15.114:0/3845812472' entity='mgr.x' 2024-09-13T18:13:03.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:03 smithi114 ceph-mon[30897]: from='mgr.14116 172.21.15.114:0/3845812472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:13:03.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:03 smithi114 ceph-mon[30897]: from='mgr.14116 172.21.15.114:0/3845812472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:13:03.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:03 smithi114 ceph-mon[30897]: from='mgr.14116 172.21.15.114:0/3845812472' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-13T18:13:03.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:03 smithi114 ceph-mon[30897]: from='mgr.14116 172.21.15.114:0/3845812472' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-13T18:13:04.818 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:04 smithi114 ceph-mon[30897]: Found migration_current of "None". Setting to last migration. 2024-09-13T18:13:04.818 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:04 smithi114 ceph-mon[30897]: from='mgr.14116 172.21.15.114:0/3845812472' entity='mgr.x' 2024-09-13T18:13:04.819 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:04 smithi114 ceph-mon[30897]: from='mgr.14116 172.21.15.114:0/3845812472' entity='mgr.x' 2024-09-13T18:13:04.819 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:04 smithi114 ceph-mon[30897]: mgrmap e7: x(active, since 1.0051s) 2024-09-13T18:13:05.086 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout { 2024-09-13T18:13:05.086 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-13T18:13:05.086 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-13T18:13:05.086 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout } 2024-09-13T18:13:05.086 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:00.990+0000 7f8d29507640 1 Processor -- start 2024-09-13T18:13:05.086 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:00.990+0000 7f8d29507640 1 -- start start 2024-09-13T18:13:05.087 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:00.990+0000 7f8d29507640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d241055a0 0x7f8d241059a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:05.087 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:00.990+0000 7f8d29507640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d24105f70 con 0x7f8d241055a0 2024-09-13T18:13:05.087 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:00.990+0000 7f8d22ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d241055a0 0x7f8d241059a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:05.087 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:00.990+0000 7f8d22ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d241055a0 0x7f8d241059a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50874/0 (socket says 172.21.15.114:50874) 2024-09-13T18:13:05.087 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:00.990+0000 7f8d22ffd640 1 -- 172.21.15.114:0/1738738768 learned_addr learned my addr 172.21.15.114:0/1738738768 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:13:05.087 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:00.990+0000 7f8d22ffd640 1 -- 172.21.15.114:0/1738738768 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d241067a0 con 0x7f8d241055a0 2024-09-13T18:13:05.087 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:00.991+0000 7f8d22ffd640 1 --2- 172.21.15.114:0/1738738768 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d241055a0 0x7f8d241059a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8d14009820 tx=0x7f8d1402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bef85e514c11c923 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:05.087 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:00.991+0000 7f8d227fc640 1 -- 172.21.15.114:0/1738738768 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d1402ae70 con 0x7f8d241055a0 2024-09-13T18:13:05.087 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:00.991+0000 7f8d227fc640 1 -- 172.21.15.114:0/1738738768 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f8d140328b0 con 0x7f8d241055a0 2024-09-13T18:13:05.087 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:00.991+0000 7f8d227fc640 1 -- 172.21.15.114:0/1738738768 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d14030850 con 0x7f8d241055a0 2024-09-13T18:13:05.087 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:00.992+0000 7f8d29507640 1 -- 172.21.15.114:0/1738738768 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d241055a0 msgr2=0x7f8d241059a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:05.087 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:00.992+0000 7f8d29507640 1 --2- 172.21.15.114:0/1738738768 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d241055a0 0x7f8d241059a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8d14009820 tx=0x7f8d1402a270 comp rx=0 tx=0).stop 2024-09-13T18:13:05.087 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:00.992+0000 7f8d29507640 1 -- 172.21.15.114:0/1738738768 shutdown_connections 2024-09-13T18:13:05.088 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:00.992+0000 7f8d29507640 1 --2- 172.21.15.114:0/1738738768 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d241055a0 0x7f8d241059a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:05.088 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:00.992+0000 7f8d29507640 1 -- 172.21.15.114:0/1738738768 >> 172.21.15.114:0/1738738768 conn(0x7f8d24100db0 msgr2=0x7f8d241031d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:05.088 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:00.992+0000 7f8d29507640 1 -- 172.21.15.114:0/1738738768 shutdown_connections 2024-09-13T18:13:05.088 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:00.992+0000 7f8d29507640 1 -- 172.21.15.114:0/1738738768 wait complete. 2024-09-13T18:13:05.088 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:00.992+0000 7f8d29507640 1 Processor -- start 2024-09-13T18:13:05.088 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:00.993+0000 7f8d29507640 1 -- start start 2024-09-13T18:13:05.088 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:00.993+0000 7f8d29507640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d241055a0 0x7f8d2407bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:05.088 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:00.993+0000 7f8d29507640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2407c0d0 con 0x7f8d241055a0 2024-09-13T18:13:05.088 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:00.993+0000 7f8d22ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d241055a0 0x7f8d2407bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:05.088 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:00.993+0000 7f8d22ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d241055a0 0x7f8d2407bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50888/0 (socket says 172.21.15.114:50888) 2024-09-13T18:13:05.088 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:00.993+0000 7f8d22ffd640 1 -- 172.21.15.114:0/3593932645 learned_addr learned my addr 172.21.15.114:0/3593932645 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:13:05.088 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:00.993+0000 7f8d22ffd640 1 -- 172.21.15.114:0/3593932645 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d140094d0 con 0x7f8d241055a0 2024-09-13T18:13:05.088 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:00.993+0000 7f8d22ffd640 1 --2- 172.21.15.114:0/3593932645 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d241055a0 0x7f8d2407bb90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8d14030560 tx=0x7f8d14030d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:05.089 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:00.993+0000 7f8d20ff9640 1 -- 172.21.15.114:0/3593932645 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d14032d60 con 0x7f8d241055a0 2024-09-13T18:13:05.089 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:00.993+0000 7f8d20ff9640 1 -- 172.21.15.114:0/3593932645 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f8d1403b8a0 con 0x7f8d241055a0 2024-09-13T18:13:05.089 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:00.993+0000 7f8d29507640 1 -- 172.21.15.114:0/3593932645 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d24107620 con 0x7f8d241055a0 2024-09-13T18:13:05.089 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:00.993+0000 7f8d20ff9640 1 -- 172.21.15.114:0/3593932645 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d1403a5f0 con 0x7f8d241055a0 2024-09-13T18:13:05.089 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:00.993+0000 7f8d29507640 1 -- 172.21.15.114:0/3593932645 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d2407a510 con 0x7f8d241055a0 2024-09-13T18:13:05.089 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:00.994+0000 7f8d20ff9640 1 -- 172.21.15.114:0/3593932645 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f8d14039040 con 0x7f8d241055a0 2024-09-13T18:13:05.089 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:00.994+0000 7f8d20ff9640 1 --2- 172.21.15.114:0/3593932645 >> [v2:172.21.15.114:6800/830738794,v1:172.21.15.114:6801/830738794] conn(0x7f8d10038620 0x7f8d1003aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:05.089 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:00.994+0000 7f8d20ff9640 1 -- 172.21.15.114:0/3593932645 --> [v2:172.21.15.114:6800/830738794,v1:172.21.15.114:6801/830738794] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8d1003b1f0 con 0x7f8d10038620 2024-09-13T18:13:05.089 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:00.994+0000 7f8d20ff9640 1 -- 172.21.15.114:0/3593932645 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8d1406b510 con 0x7f8d241055a0 2024-09-13T18:13:05.089 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:00.994+0000 7f8d1bfff640 1 -- 172.21.15.114:0/3593932645 >> [v2:172.21.15.114:6800/830738794,v1:172.21.15.114:6801/830738794] conn(0x7f8d10038620 msgr2=0x7f8d1003aae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.114:6800/830738794 2024-09-13T18:13:05.089 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:00.994+0000 7f8d1bfff640 1 --2- 172.21.15.114:0/3593932645 >> [v2:172.21.15.114:6800/830738794,v1:172.21.15.114:6801/830738794] conn(0x7f8d10038620 0x7f8d1003aae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:13:05.089 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:01.195+0000 7f8d1bfff640 1 -- 172.21.15.114:0/3593932645 >> [v2:172.21.15.114:6800/830738794,v1:172.21.15.114:6801/830738794] conn(0x7f8d10038620 msgr2=0x7f8d1003aae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.114:6800/830738794 2024-09-13T18:13:05.089 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:01.195+0000 7f8d1bfff640 1 --2- 172.21.15.114:0/3593932645 >> [v2:172.21.15.114:6800/830738794,v1:172.21.15.114:6801/830738794] conn(0x7f8d10038620 0x7f8d1003aae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:13:05.090 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:01.596+0000 7f8d1bfff640 1 -- 172.21.15.114:0/3593932645 >> [v2:172.21.15.114:6800/830738794,v1:172.21.15.114:6801/830738794] conn(0x7f8d10038620 msgr2=0x7f8d1003aae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.114:6800/830738794 2024-09-13T18:13:05.090 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:01.596+0000 7f8d1bfff640 1 --2- 172.21.15.114:0/3593932645 >> [v2:172.21.15.114:6800/830738794,v1:172.21.15.114:6801/830738794] conn(0x7f8d10038620 0x7f8d1003aae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:13:05.090 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:02.397+0000 7f8d1bfff640 1 -- 172.21.15.114:0/3593932645 >> [v2:172.21.15.114:6800/830738794,v1:172.21.15.114:6801/830738794] conn(0x7f8d10038620 msgr2=0x7f8d1003aae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.114:6800/830738794 2024-09-13T18:13:05.090 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:02.397+0000 7f8d1bfff640 1 --2- 172.21.15.114:0/3593932645 >> [v2:172.21.15.114:6800/830738794,v1:172.21.15.114:6801/830738794] conn(0x7f8d10038620 0x7f8d1003aae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:13:05.090 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:03.522+0000 7f8d20ff9640 1 -- 172.21.15.114:0/3593932645 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mgrmap(e 6) v1 ==== 45308+0+0 (secure 0 0 0) 0x7f8d1406a710 con 0x7f8d241055a0 2024-09-13T18:13:05.090 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:03.522+0000 7f8d20ff9640 1 -- 172.21.15.114:0/3593932645 >> [v2:172.21.15.114:6800/830738794,v1:172.21.15.114:6801/830738794] conn(0x7f8d10038620 msgr2=0x7f8d1003aae0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:13:05.090 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:03.522+0000 7f8d20ff9640 1 --2- 172.21.15.114:0/3593932645 >> [v2:172.21.15.114:6800/830738794,v1:172.21.15.114:6801/830738794] conn(0x7f8d10038620 0x7f8d1003aae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:05.090 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:04.522+0000 7f8d20ff9640 1 -- 172.21.15.114:0/3593932645 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mgrmap(e 7) v1 ==== 45435+0+0 (secure 0 0 0) 0x7f8d1404ad30 con 0x7f8d241055a0 2024-09-13T18:13:05.090 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:04.522+0000 7f8d20ff9640 1 --2- 172.21.15.114:0/3593932645 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7f8d1003bf50 0x7f8d1003e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:05.090 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:04.523+0000 7f8d20ff9640 1 -- 172.21.15.114:0/3593932645 --> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8d1003b1f0 con 0x7f8d1003bf50 2024-09-13T18:13:05.090 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:04.523+0000 7f8d1bfff640 1 --2- 172.21.15.114:0/3593932645 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7f8d1003bf50 0x7f8d1003e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:05.090 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:04.524+0000 7f8d1bfff640 1 --2- 172.21.15.114:0/3593932645 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7f8d1003bf50 0x7f8d1003e340 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c003980 tx=0x7f8d0c006e10 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:05.090 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:04.524+0000 7f8d20ff9640 1 -- 172.21.15.114:0/3593932645 <== mgr.14116 v2:172.21.15.114:6800/4281866858 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f8d1003b1f0 con 0x7f8d1003bf50 2024-09-13T18:13:05.091 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:04.530+0000 7f8d29507640 1 -- 172.21.15.114:0/3593932645 --> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f8d241099d0 con 0x7f8d1003bf50 2024-09-13T18:13:05.091 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:04.530+0000 7f8d20ff9640 1 -- 172.21.15.114:0/3593932645 <== mgr.14116 v2:172.21.15.114:6800/4281866858 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f8d241099d0 con 0x7f8d1003bf50 2024-09-13T18:13:05.091 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:04.530+0000 7f8d29507640 1 -- 172.21.15.114:0/3593932645 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7f8d1003bf50 msgr2=0x7f8d1003e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:05.091 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:04.530+0000 7f8d29507640 1 --2- 172.21.15.114:0/3593932645 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7f8d1003bf50 0x7f8d1003e340 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c003980 tx=0x7f8d0c006e10 comp rx=0 tx=0).stop 2024-09-13T18:13:05.091 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:04.530+0000 7f8d29507640 1 -- 172.21.15.114:0/3593932645 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d241055a0 msgr2=0x7f8d2407bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:05.091 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:04.530+0000 7f8d29507640 1 --2- 172.21.15.114:0/3593932645 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d241055a0 0x7f8d2407bb90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8d14030560 tx=0x7f8d14030d90 comp rx=0 tx=0).stop 2024-09-13T18:13:05.091 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:04.530+0000 7f8d29507640 1 -- 172.21.15.114:0/3593932645 shutdown_connections 2024-09-13T18:13:05.091 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:04.530+0000 7f8d29507640 1 --2- 172.21.15.114:0/3593932645 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7f8d1003bf50 0x7f8d1003e340 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:05.091 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:04.530+0000 7f8d29507640 1 --2- 172.21.15.114:0/3593932645 >> [v2:172.21.15.114:6800/830738794,v1:172.21.15.114:6801/830738794] conn(0x7f8d10038620 0x7f8d1003aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:05.091 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:04.530+0000 7f8d29507640 1 --2- 172.21.15.114:0/3593932645 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d241055a0 0x7f8d2407bb90 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:05.091 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:04.530+0000 7f8d29507640 1 -- 172.21.15.114:0/3593932645 >> 172.21.15.114:0/3593932645 conn(0x7f8d24100db0 msgr2=0x7f8d241019b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:05.091 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:04.530+0000 7f8d29507640 1 -- 172.21.15.114:0/3593932645 shutdown_connections 2024-09-13T18:13:05.091 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:04.530+0000 7f8d29507640 1 -- 172.21.15.114:0/3593932645 wait complete. 2024-09-13T18:13:05.092 INFO:teuthology.orchestra.run.smithi114.stdout:mgr epoch 5 is available 2024-09-13T18:13:05.092 INFO:teuthology.orchestra.run.smithi114.stdout:Setting orchestrator backend to cephadm... 2024-09-13T18:13:05.381 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:05 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:13:05.086+0000 7f1cfad1b640 -1 log_channel(cephadm) log [ERR] : [13/Sep/2024:18:13:05] ENGINE Error in HTTPServer.serve 2024-09-13T18:13:05.381 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:05 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: Traceback (most recent call last): 2024-09-13T18:13:05.381 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:05 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-13T18:13:05.381 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:05 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: self._connections.run(self.expiration_interval) 2024-09-13T18:13:05.381 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:05 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-13T18:13:05.381 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:05 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: self._run(expiration_interval) 2024-09-13T18:13:05.381 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:05 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-13T18:13:05.381 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:05 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: new_conn = self._from_server_socket(self.server.socket) 2024-09-13T18:13:05.381 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:05 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-13T18:13:05.381 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:05 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-13T18:13:05.381 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:05 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-13T18:13:05.382 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:05 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: s = self.context.wrap_socket( 2024-09-13T18:13:05.382 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:05 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-13T18:13:05.382 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:05 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: return self.sslsocket_class._create( 2024-09-13T18:13:05.382 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:05 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-13T18:13:05.382 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:05 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: self.do_handshake() 2024-09-13T18:13:05.382 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:05 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-13T18:13:05.382 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:05 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: self._sslobj.do_handshake() 2024-09-13T18:13:05.382 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:05 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-13T18:13:05.382 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:05 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:13:05.646 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:05 smithi114 ceph-mon[30897]: from='client.14120 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-13T18:13:05.646 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:05 smithi114 ceph-mon[30897]: from='client.14120 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-13T18:13:05.646 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:05 smithi114 ceph-mon[30897]: from='mgr.14116 172.21.15.114:0/3845812472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:13:06.233 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:05.455+0000 7f5ba22a6640 1 Processor -- start 2024-09-13T18:13:06.233 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:05.455+0000 7f5ba22a6640 1 -- start start 2024-09-13T18:13:06.233 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:05.456+0000 7f5ba22a6640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b9c106080 0x7f5b9c108470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:06.233 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:05.456+0000 7f5ba22a6640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9c079e90 con 0x7f5b9c106080 2024-09-13T18:13:06.233 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:05.456+0000 7f5b9bfff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b9c106080 0x7f5b9c108470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:06.233 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:05.456+0000 7f5b9bfff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b9c106080 0x7f5b9c108470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60142/0 (socket says 172.21.15.114:60142) 2024-09-13T18:13:06.234 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:05.456+0000 7f5b9bfff640 1 -- 172.21.15.114:0/432488339 learned_addr learned my addr 172.21.15.114:0/432488339 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:13:06.234 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:05.456+0000 7f5b9bfff640 1 -- 172.21.15.114:0/432488339 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b9c1089b0 con 0x7f5b9c106080 2024-09-13T18:13:06.234 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:05.457+0000 7f5b9bfff640 1 --2- 172.21.15.114:0/432488339 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b9c106080 0x7f5b9c108470 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c009820 tx=0x7f5b8c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4513d6731ea7c92 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:06.234 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:05.457+0000 7f5b9b7fe640 1 -- 172.21.15.114:0/432488339 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5b8c02ae70 con 0x7f5b9c106080 2024-09-13T18:13:06.234 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:05.457+0000 7f5b9b7fe640 1 -- 172.21.15.114:0/432488339 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f5b8c0328b0 con 0x7f5b9c106080 2024-09-13T18:13:06.234 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:05.457+0000 7f5b9b7fe640 1 -- 172.21.15.114:0/432488339 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5b8c030850 con 0x7f5b9c106080 2024-09-13T18:13:06.234 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:05.458+0000 7f5ba22a6640 1 -- 172.21.15.114:0/432488339 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b9c106080 msgr2=0x7f5b9c108470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:06.234 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:05.458+0000 7f5ba22a6640 1 --2- 172.21.15.114:0/432488339 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b9c106080 0x7f5b9c108470 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c009820 tx=0x7f5b8c02a270 comp rx=0 tx=0).stop 2024-09-13T18:13:06.234 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:05.458+0000 7f5ba22a6640 1 -- 172.21.15.114:0/432488339 shutdown_connections 2024-09-13T18:13:06.234 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:05.458+0000 7f5ba22a6640 1 --2- 172.21.15.114:0/432488339 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b9c106080 0x7f5b9c108470 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:06.234 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:05.458+0000 7f5ba22a6640 1 -- 172.21.15.114:0/432488339 >> 172.21.15.114:0/432488339 conn(0x7f5b9c0ffc60 msgr2=0x7f5b9c1020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:06.234 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:05.458+0000 7f5ba22a6640 1 -- 172.21.15.114:0/432488339 shutdown_connections 2024-09-13T18:13:06.234 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:05.458+0000 7f5ba22a6640 1 -- 172.21.15.114:0/432488339 wait complete. 2024-09-13T18:13:06.235 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:05.458+0000 7f5ba22a6640 1 Processor -- start 2024-09-13T18:13:06.235 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:05.458+0000 7f5ba22a6640 1 -- start start 2024-09-13T18:13:06.235 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:05.458+0000 7f5ba22a6640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b9c106080 0x7f5b9c199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:06.235 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:05.458+0000 7f5ba22a6640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9c199740 con 0x7f5b9c106080 2024-09-13T18:13:06.235 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:05.459+0000 7f5b9bfff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b9c106080 0x7f5b9c199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:06.235 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:05.459+0000 7f5b9bfff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b9c106080 0x7f5b9c199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60154/0 (socket says 172.21.15.114:60154) 2024-09-13T18:13:06.235 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:05.459+0000 7f5b9bfff640 1 -- 172.21.15.114:0/433915870 learned_addr learned my addr 172.21.15.114:0/433915870 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:13:06.235 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:05.459+0000 7f5b9bfff640 1 -- 172.21.15.114:0/433915870 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b8c0094d0 con 0x7f5b9c106080 2024-09-13T18:13:06.235 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:05.459+0000 7f5b9bfff640 1 --2- 172.21.15.114:0/433915870 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b9c106080 0x7f5b9c199200 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c000c00 tx=0x7f5b8c030db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:06.235 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:05.459+0000 7f5b99ffb640 1 -- 172.21.15.114:0/433915870 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5b8c032d60 con 0x7f5b9c106080 2024-09-13T18:13:06.235 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:05.459+0000 7f5ba22a6640 1 -- 172.21.15.114:0/433915870 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b9c199940 con 0x7f5b9c106080 2024-09-13T18:13:06.235 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:05.459+0000 7f5b99ffb640 1 -- 172.21.15.114:0/433915870 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f5b8c03b8a0 con 0x7f5b9c106080 2024-09-13T18:13:06.235 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:05.459+0000 7f5b99ffb640 1 -- 172.21.15.114:0/433915870 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5b8c03a640 con 0x7f5b9c106080 2024-09-13T18:13:06.235 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:05.459+0000 7f5ba22a6640 1 -- 172.21.15.114:0/433915870 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b9c199de0 con 0x7f5b9c106080 2024-09-13T18:13:06.236 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:05.460+0000 7f5b99ffb640 1 -- 172.21.15.114:0/433915870 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 7) v1 ==== 45435+0+0 (secure 0 0 0) 0x7f5b8c039040 con 0x7f5b9c106080 2024-09-13T18:13:06.236 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:05.460+0000 7f5ba22a6640 1 -- 172.21.15.114:0/433915870 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b9c1096a0 con 0x7f5b9c106080 2024-09-13T18:13:06.236 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:05.460+0000 7f5b99ffb640 1 --2- 172.21.15.114:0/433915870 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7f5b88038500 0x7f5b8803a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:06.236 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:05.460+0000 7f5b99ffb640 1 -- 172.21.15.114:0/433915870 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f5b8c06b780 con 0x7f5b9c106080 2024-09-13T18:13:06.236 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:05.460+0000 7f5b93fff640 1 --2- 172.21.15.114:0/433915870 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7f5b88038500 0x7f5b8803a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:06.236 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:05.461+0000 7f5b93fff640 1 --2- 172.21.15.114:0/433915870 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7f5b88038500 0x7f5b8803a9c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5b840098c0 tx=0x7f5b84006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:06.236 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:05.465+0000 7f5b99ffb640 1 -- 172.21.15.114:0/433915870 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f5b8c0319f0 con 0x7f5b9c106080 2024-09-13T18:13:06.236 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:05.579+0000 7f5b99ffb640 1 -- 172.21.15.114:0/433915870 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f5b8c038d20 con 0x7f5b9c106080 2024-09-13T18:13:06.237 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:05.642+0000 7f5ba22a6640 1 -- 172.21.15.114:0/433915870 --> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f5b9c100f70 con 0x7f5b88038500 2024-09-13T18:13:06.237 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:05.653+0000 7f5b99ffb640 1 -- 172.21.15.114:0/433915870 <== mgr.14116 v2:172.21.15.114:6800/4281866858 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f5b9c100f70 con 0x7f5b88038500 2024-09-13T18:13:06.237 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:05.656+0000 7f5ba22a6640 1 -- 172.21.15.114:0/433915870 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7f5b88038500 msgr2=0x7f5b8803a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:06.237 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:05.656+0000 7f5ba22a6640 1 --2- 172.21.15.114:0/433915870 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7f5b88038500 0x7f5b8803a9c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5b840098c0 tx=0x7f5b84006db0 comp rx=0 tx=0).stop 2024-09-13T18:13:06.237 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:05.656+0000 7f5ba22a6640 1 -- 172.21.15.114:0/433915870 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b9c106080 msgr2=0x7f5b9c199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:06.237 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:05.656+0000 7f5ba22a6640 1 --2- 172.21.15.114:0/433915870 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b9c106080 0x7f5b9c199200 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c000c00 tx=0x7f5b8c030db0 comp rx=0 tx=0).stop 2024-09-13T18:13:06.237 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:05.656+0000 7f5ba22a6640 1 -- 172.21.15.114:0/433915870 shutdown_connections 2024-09-13T18:13:06.237 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:05.656+0000 7f5ba22a6640 1 --2- 172.21.15.114:0/433915870 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7f5b88038500 0x7f5b8803a9c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:06.237 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:05.656+0000 7f5ba22a6640 1 --2- 172.21.15.114:0/433915870 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b9c106080 0x7f5b9c199200 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:06.237 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:05.656+0000 7f5ba22a6640 1 -- 172.21.15.114:0/433915870 >> 172.21.15.114:0/433915870 conn(0x7f5b9c0ffc60 msgr2=0x7f5b9c100860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:06.238 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:05.656+0000 7f5ba22a6640 1 -- 172.21.15.114:0/433915870 shutdown_connections 2024-09-13T18:13:06.238 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:05.657+0000 7f5ba22a6640 1 -- 172.21.15.114:0/433915870 wait complete. 2024-09-13T18:13:06.773 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:06 smithi114 ceph-mon[30897]: [13/Sep/2024:18:13:04] ENGINE Bus STARTING 2024-09-13T18:13:06.773 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:06 smithi114 ceph-mon[30897]: [13/Sep/2024:18:13:04] ENGINE Serving on http://172.21.15.114:8765 2024-09-13T18:13:06.773 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:06 smithi114 ceph-mon[30897]: [13/Sep/2024:18:13:05] ENGINE Serving on https://172.21.15.114:7150 2024-09-13T18:13:06.773 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:06 smithi114 ceph-mon[30897]: [13/Sep/2024:18:13:05] ENGINE Bus STARTED 2024-09-13T18:13:06.773 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:06 smithi114 ceph-mon[30897]: [13/Sep/2024:18:13:05] ENGINE Error in HTTPServer.serve 2024-09-13T18:13:06.773 INFO:journalctl@ceph.mon.a.smithi114.stdout: Traceback (most recent call last): 2024-09-13T18:13:06.773 INFO:journalctl@ceph.mon.a.smithi114.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-13T18:13:06.773 INFO:journalctl@ceph.mon.a.smithi114.stdout: self._connections.run(self.expiration_interval) 2024-09-13T18:13:06.773 INFO:journalctl@ceph.mon.a.smithi114.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-13T18:13:06.773 INFO:journalctl@ceph.mon.a.smithi114.stdout: self._run(expiration_interval) 2024-09-13T18:13:06.774 INFO:journalctl@ceph.mon.a.smithi114.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-13T18:13:06.774 INFO:journalctl@ceph.mon.a.smithi114.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-13T18:13:06.774 INFO:journalctl@ceph.mon.a.smithi114.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-13T18:13:06.774 INFO:journalctl@ceph.mon.a.smithi114.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-13T18:13:06.774 INFO:journalctl@ceph.mon.a.smithi114.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-13T18:13:06.774 INFO:journalctl@ceph.mon.a.smithi114.stdout: s = self.context.wrap_socket( 2024-09-13T18:13:06.774 INFO:journalctl@ceph.mon.a.smithi114.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-13T18:13:06.774 INFO:journalctl@ceph.mon.a.smithi114.stdout: return self.sslsocket_class._create( 2024-09-13T18:13:06.774 INFO:journalctl@ceph.mon.a.smithi114.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-13T18:13:06.774 INFO:journalctl@ceph.mon.a.smithi114.stdout: self.do_handshake() 2024-09-13T18:13:06.774 INFO:journalctl@ceph.mon.a.smithi114.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-13T18:13:06.774 INFO:journalctl@ceph.mon.a.smithi114.stdout: self._sslobj.do_handshake() 2024-09-13T18:13:06.774 INFO:journalctl@ceph.mon.a.smithi114.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-13T18:13:06.774 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:06 smithi114 ceph-mon[30897]: mgrmap e8: x(active, since 2s) 2024-09-13T18:13:06.775 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:06 smithi114 ceph-mon[30897]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:13:06.775 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:06 smithi114 ceph-mon[30897]: from='mgr.14116 172.21.15.114:0/3845812472' entity='mgr.x' 2024-09-13T18:13:06.775 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:06 smithi114 ceph-mon[30897]: from='mgr.14116 172.21.15.114:0/3845812472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:13:07.306 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-13T18:13:07.306 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:06.596+0000 7f0a06fd9640 1 Processor -- start 2024-09-13T18:13:07.307 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:06.596+0000 7f0a06fd9640 1 -- start start 2024-09-13T18:13:07.307 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:06.596+0000 7f0a06fd9640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a00105510 0x7f0a00105910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:07.307 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:06.596+0000 7f0a06fd9640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a00105ee0 con 0x7f0a00105510 2024-09-13T18:13:07.307 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:06.596+0000 7f0a04d75640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a00105510 0x7f0a00105910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:07.307 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:06.596+0000 7f0a04d75640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a00105510 0x7f0a00105910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60166/0 (socket says 172.21.15.114:60166) 2024-09-13T18:13:07.307 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:06.596+0000 7f0a04d75640 1 -- 172.21.15.114:0/920923230 learned_addr learned my addr 172.21.15.114:0/920923230 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:13:07.307 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:06.596+0000 7f0a04d75640 1 -- 172.21.15.114:0/920923230 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a00106710 con 0x7f0a00105510 2024-09-13T18:13:07.307 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:06.597+0000 7f0a04d75640 1 --2- 172.21.15.114:0/920923230 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a00105510 0x7f0a00105910 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f09f4009820 tx=0x7f09f402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24a198d1c7ea09ee server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:07.307 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:06.597+0000 7f09ff7fe640 1 -- 172.21.15.114:0/920923230 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09f402ae70 con 0x7f0a00105510 2024-09-13T18:13:07.307 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:06.597+0000 7f09ff7fe640 1 -- 172.21.15.114:0/920923230 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f09f40328b0 con 0x7f0a00105510 2024-09-13T18:13:07.308 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:06.597+0000 7f09ff7fe640 1 -- 172.21.15.114:0/920923230 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09f4030850 con 0x7f0a00105510 2024-09-13T18:13:07.308 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:06.598+0000 7f0a06fd9640 1 -- 172.21.15.114:0/920923230 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a00105510 msgr2=0x7f0a00105910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:07.308 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:06.598+0000 7f0a06fd9640 1 --2- 172.21.15.114:0/920923230 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a00105510 0x7f0a00105910 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f09f4009820 tx=0x7f09f402a270 comp rx=0 tx=0).stop 2024-09-13T18:13:07.308 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:06.598+0000 7f0a06fd9640 1 -- 172.21.15.114:0/920923230 shutdown_connections 2024-09-13T18:13:07.308 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:06.598+0000 7f0a06fd9640 1 --2- 172.21.15.114:0/920923230 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a00105510 0x7f0a00105910 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:07.308 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:06.598+0000 7f0a06fd9640 1 -- 172.21.15.114:0/920923230 >> 172.21.15.114:0/920923230 conn(0x7f0a00100d00 msgr2=0x7f0a00103120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:07.308 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:06.598+0000 7f0a06fd9640 1 -- 172.21.15.114:0/920923230 shutdown_connections 2024-09-13T18:13:07.308 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:06.598+0000 7f0a06fd9640 1 -- 172.21.15.114:0/920923230 wait complete. 2024-09-13T18:13:07.308 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:06.598+0000 7f0a06fd9640 1 Processor -- start 2024-09-13T18:13:07.308 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:06.599+0000 7f0a06fd9640 1 -- start start 2024-09-13T18:13:07.308 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:06.599+0000 7f0a06fd9640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a00105510 0x7f0a00199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:07.308 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:06.599+0000 7f0a06fd9640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a001998a0 con 0x7f0a00105510 2024-09-13T18:13:07.309 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:06.599+0000 7f0a04d75640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a00105510 0x7f0a00199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:07.309 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:06.599+0000 7f0a04d75640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a00105510 0x7f0a00199360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60182/0 (socket says 172.21.15.114:60182) 2024-09-13T18:13:07.309 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:06.599+0000 7f0a04d75640 1 -- 172.21.15.114:0/4018608313 learned_addr learned my addr 172.21.15.114:0/4018608313 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:13:07.309 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:06.599+0000 7f0a04d75640 1 -- 172.21.15.114:0/4018608313 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09f40094d0 con 0x7f0a00105510 2024-09-13T18:13:07.309 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:06.599+0000 7f0a04d75640 1 --2- 172.21.15.114:0/4018608313 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a00105510 0x7f0a00199360 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f09f4030560 tx=0x7f09f4030d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:07.309 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:06.599+0000 7f09fdffb640 1 -- 172.21.15.114:0/4018608313 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09f4032d60 con 0x7f0a00105510 2024-09-13T18:13:07.309 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:06.599+0000 7f09fdffb640 1 -- 172.21.15.114:0/4018608313 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f09f403b8a0 con 0x7f0a00105510 2024-09-13T18:13:07.309 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:06.599+0000 7f0a06fd9640 1 -- 172.21.15.114:0/4018608313 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a00107590 con 0x7f0a00105510 2024-09-13T18:13:07.309 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:06.599+0000 7f09fdffb640 1 -- 172.21.15.114:0/4018608313 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09f403a680 con 0x7f0a00105510 2024-09-13T18:13:07.309 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:06.600+0000 7f0a06fd9640 1 -- 172.21.15.114:0/4018608313 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a00199db0 con 0x7f0a00105510 2024-09-13T18:13:07.309 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:06.600+0000 7f09fdffb640 1 -- 172.21.15.114:0/4018608313 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f09f4039040 con 0x7f0a00105510 2024-09-13T18:13:07.309 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:06.601+0000 7f09fdffb640 1 --2- 172.21.15.114:0/4018608313 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7f09f0038670 0x7f09f003ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:07.310 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:06.601+0000 7f09fdffb640 1 -- 172.21.15.114:0/4018608313 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f09f406b9c0 con 0x7f0a00105510 2024-09-13T18:13:07.310 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:06.601+0000 7f09fffff640 1 --2- 172.21.15.114:0/4018608313 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7f09f0038670 0x7f09f003ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:07.310 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:06.601+0000 7f0a06fd9640 1 -- 172.21.15.114:0/4018608313 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a00199ff0 con 0x7f0a00105510 2024-09-13T18:13:07.311 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:06.601+0000 7f09fffff640 1 --2- 172.21.15.114:0/4018608313 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7f09f0038670 0x7f09f003ab30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f09ec0098c0 tx=0x7f09ec006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:07.312 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:06.605+0000 7f09fdffb640 1 -- 172.21.15.114:0/4018608313 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f09f4037070 con 0x7f0a00105510 2024-09-13T18:13:07.312 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:06.769+0000 7f0a06fd9640 1 -- 172.21.15.114:0/4018608313 --> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f0a00101fc0 con 0x7f09f0038670 2024-09-13T18:13:07.312 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:06.770+0000 7f09fdffb640 1 -- 172.21.15.114:0/4018608313 <== mgr.14116 v2:172.21.15.114:6800/4281866858 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f0a00101fc0 con 0x7f09f0038670 2024-09-13T18:13:07.312 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:06.772+0000 7f0a06fd9640 1 -- 172.21.15.114:0/4018608313 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7f09f0038670 msgr2=0x7f09f003ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:07.312 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:06.772+0000 7f0a06fd9640 1 --2- 172.21.15.114:0/4018608313 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7f09f0038670 0x7f09f003ab30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f09ec0098c0 tx=0x7f09ec006db0 comp rx=0 tx=0).stop 2024-09-13T18:13:07.312 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:06.772+0000 7f0a06fd9640 1 -- 172.21.15.114:0/4018608313 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a00105510 msgr2=0x7f0a00199360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:07.312 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:06.772+0000 7f0a06fd9640 1 --2- 172.21.15.114:0/4018608313 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a00105510 0x7f0a00199360 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f09f4030560 tx=0x7f09f4030d90 comp rx=0 tx=0).stop 2024-09-13T18:13:07.312 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:06.772+0000 7f0a06fd9640 1 -- 172.21.15.114:0/4018608313 shutdown_connections 2024-09-13T18:13:07.312 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:06.772+0000 7f0a06fd9640 1 --2- 172.21.15.114:0/4018608313 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7f09f0038670 0x7f09f003ab30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:07.313 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:06.772+0000 7f0a06fd9640 1 --2- 172.21.15.114:0/4018608313 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a00105510 0x7f0a00199360 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:07.313 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:06.772+0000 7f0a06fd9640 1 -- 172.21.15.114:0/4018608313 >> 172.21.15.114:0/4018608313 conn(0x7f0a00100d00 msgr2=0x7f0a001018b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:07.313 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:06.772+0000 7f0a06fd9640 1 -- 172.21.15.114:0/4018608313 shutdown_connections 2024-09-13T18:13:07.313 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:06.772+0000 7f0a06fd9640 1 -- 172.21.15.114:0/4018608313 wait complete. 2024-09-13T18:13:07.313 INFO:teuthology.orchestra.run.smithi114.stdout:Generating ssh key... 2024-09-13T18:13:07.890 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:07 smithi114 ceph-mon[30897]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:13:08.220 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:07 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: Generating public/private rsa key pair. 2024-09-13T18:13:08.220 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:07 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: Your identification has been saved in /tmp/tmp1t9n3r6h/key 2024-09-13T18:13:08.220 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:07 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: Your public key has been saved in /tmp/tmp1t9n3r6h/key.pub 2024-09-13T18:13:08.220 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:07 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: The key fingerprint is: 2024-09-13T18:13:08.221 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:07 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: SHA256:7UU9Xo84ZsFxi4RDSoeXvxG14H3zPHGTTZayE80lU6Y ceph-97e7edf2-71fb-11ef-bceb-c7b262605968 2024-09-13T18:13:08.221 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:07 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: The key's randomart image is: 2024-09-13T18:13:08.221 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:07 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: +---[RSA 3072]----+ 2024-09-13T18:13:08.221 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:07 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: | .ooo+.B.O| 2024-09-13T18:13:08.221 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:07 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: | ..o=+.X &o| 2024-09-13T18:13:08.221 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:07 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: | .. o*.E==| 2024-09-13T18:13:08.221 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:07 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: | . .o* *B| 2024-09-13T18:13:08.221 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:07 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: | S . *o+.+| 2024-09-13T18:13:08.221 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:07 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: | . +.. .| 2024-09-13T18:13:08.221 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:07 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: | . | 2024-09-13T18:13:08.221 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:07 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: | | 2024-09-13T18:13:08.221 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:07 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: | | 2024-09-13T18:13:08.221 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:07 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: +----[SHA256]-----+ 2024-09-13T18:13:08.554 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:07.716+0000 7fa191b9b640 1 Processor -- start 2024-09-13T18:13:08.554 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:07.716+0000 7fa191b9b640 1 -- start start 2024-09-13T18:13:08.554 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:07.716+0000 7fa191b9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa18c07bec0 0x7fa18c07a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:08.555 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:07.716+0000 7fa191b9b640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18c07a8f0 con 0x7fa18c07bec0 2024-09-13T18:13:08.555 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:07.716+0000 7fa18b7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa18c07bec0 0x7fa18c07a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:08.555 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:07.716+0000 7fa18b7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa18c07bec0 0x7fa18c07a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60194/0 (socket says 172.21.15.114:60194) 2024-09-13T18:13:08.555 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:07.716+0000 7fa18b7fe640 1 -- 172.21.15.114:0/2510265754 learned_addr learned my addr 172.21.15.114:0/2510265754 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:13:08.555 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:07.716+0000 7fa18b7fe640 1 -- 172.21.15.114:0/2510265754 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa18c07aa30 con 0x7fa18c07bec0 2024-09-13T18:13:08.555 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:07.717+0000 7fa18b7fe640 1 --2- 172.21.15.114:0/2510265754 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa18c07bec0 0x7fa18c07a320 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa17c009820 tx=0x7fa17c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5a9fa71fe4e8914 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:08.555 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:07.717+0000 7fa18a7fc640 1 -- 172.21.15.114:0/2510265754 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa17c02ae70 con 0x7fa18c07bec0 2024-09-13T18:13:08.555 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:07.717+0000 7fa18a7fc640 1 -- 172.21.15.114:0/2510265754 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fa17c0328b0 con 0x7fa18c07bec0 2024-09-13T18:13:08.555 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:07.717+0000 7fa18a7fc640 1 -- 172.21.15.114:0/2510265754 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa17c030850 con 0x7fa18c07bec0 2024-09-13T18:13:08.555 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:07.718+0000 7fa191b9b640 1 -- 172.21.15.114:0/2510265754 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa18c07bec0 msgr2=0x7fa18c07a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:08.555 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:07.718+0000 7fa191b9b640 1 --2- 172.21.15.114:0/2510265754 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa18c07bec0 0x7fa18c07a320 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa17c009820 tx=0x7fa17c02a270 comp rx=0 tx=0).stop 2024-09-13T18:13:08.555 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:07.718+0000 7fa191b9b640 1 -- 172.21.15.114:0/2510265754 shutdown_connections 2024-09-13T18:13:08.555 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:07.718+0000 7fa191b9b640 1 --2- 172.21.15.114:0/2510265754 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa18c07bec0 0x7fa18c07a320 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:08.555 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:07.718+0000 7fa191b9b640 1 -- 172.21.15.114:0/2510265754 >> 172.21.15.114:0/2510265754 conn(0x7fa18c100db0 msgr2=0x7fa18c1031d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:08.556 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:07.718+0000 7fa191b9b640 1 -- 172.21.15.114:0/2510265754 shutdown_connections 2024-09-13T18:13:08.556 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:07.718+0000 7fa191b9b640 1 -- 172.21.15.114:0/2510265754 wait complete. 2024-09-13T18:13:08.556 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:07.718+0000 7fa191b9b640 1 Processor -- start 2024-09-13T18:13:08.556 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:07.719+0000 7fa191b9b640 1 -- start start 2024-09-13T18:13:08.556 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:07.719+0000 7fa191b9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa18c07bec0 0x7fa18c19b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:08.556 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:07.719+0000 7fa191b9b640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18c19b9f0 con 0x7fa18c07bec0 2024-09-13T18:13:08.556 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:07.719+0000 7fa18b7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa18c07bec0 0x7fa18c19b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:08.556 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:07.719+0000 7fa18b7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa18c07bec0 0x7fa18c19b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60202/0 (socket says 172.21.15.114:60202) 2024-09-13T18:13:08.556 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:07.719+0000 7fa18b7fe640 1 -- 172.21.15.114:0/1196343319 learned_addr learned my addr 172.21.15.114:0/1196343319 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:13:08.556 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:07.719+0000 7fa18b7fe640 1 -- 172.21.15.114:0/1196343319 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa17c0094d0 con 0x7fa18c07bec0 2024-09-13T18:13:08.556 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:07.719+0000 7fa18b7fe640 1 --2- 172.21.15.114:0/1196343319 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa18c07bec0 0x7fa18c19b4b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa17c030560 tx=0x7fa17c030d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:08.556 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:07.719+0000 7fa188ff9640 1 -- 172.21.15.114:0/1196343319 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa17c032d60 con 0x7fa18c07bec0 2024-09-13T18:13:08.556 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:07.719+0000 7fa191b9b640 1 -- 172.21.15.114:0/1196343319 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa18c19bbf0 con 0x7fa18c07bec0 2024-09-13T18:13:08.557 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:07.719+0000 7fa188ff9640 1 -- 172.21.15.114:0/1196343319 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fa17c03b8a0 con 0x7fa18c07bec0 2024-09-13T18:13:08.557 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:07.719+0000 7fa188ff9640 1 -- 172.21.15.114:0/1196343319 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa17c03a5b0 con 0x7fa18c07bec0 2024-09-13T18:13:08.557 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:07.719+0000 7fa191b9b640 1 -- 172.21.15.114:0/1196343319 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa18c19c090 con 0x7fa18c07bec0 2024-09-13T18:13:08.557 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:07.720+0000 7fa188ff9640 1 -- 172.21.15.114:0/1196343319 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7fa17c039040 con 0x7fa18c07bec0 2024-09-13T18:13:08.557 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:07.720+0000 7fa191b9b640 1 -- 172.21.15.114:0/1196343319 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa18c10b970 con 0x7fa18c07bec0 2024-09-13T18:13:08.557 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:07.720+0000 7fa188ff9640 1 --2- 172.21.15.114:0/1196343319 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fa174038620 0x7fa17403aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:08.557 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:07.720+0000 7fa188ff9640 1 -- 172.21.15.114:0/1196343319 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa17c06b880 con 0x7fa18c07bec0 2024-09-13T18:13:08.557 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:07.721+0000 7fa18affd640 1 --2- 172.21.15.114:0/1196343319 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fa174038620 0x7fa17403aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:08.557 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:07.721+0000 7fa18affd640 1 --2- 172.21.15.114:0/1196343319 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fa174038620 0x7fa17403aae0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa1800098c0 tx=0x7fa180006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:08.557 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:07.725+0000 7fa188ff9640 1 -- 172.21.15.114:0/1196343319 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fa17c03a870 con 0x7fa18c07bec0 2024-09-13T18:13:08.557 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:07.887+0000 7fa191b9b640 1 -- 172.21.15.114:0/1196343319 --> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fa18c1020e0 con 0x7fa174038620 2024-09-13T18:13:08.557 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:08.008+0000 7fa188ff9640 1 -- 172.21.15.114:0/1196343319 <== mgr.14116 v2:172.21.15.114:6800/4281866858 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fa18c1020e0 con 0x7fa174038620 2024-09-13T18:13:08.557 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:08.010+0000 7fa191b9b640 1 -- 172.21.15.114:0/1196343319 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fa174038620 msgr2=0x7fa17403aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:08.557 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:08.010+0000 7fa191b9b640 1 --2- 172.21.15.114:0/1196343319 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fa174038620 0x7fa17403aae0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa1800098c0 tx=0x7fa180006db0 comp rx=0 tx=0).stop 2024-09-13T18:13:08.558 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:08.011+0000 7fa191b9b640 1 -- 172.21.15.114:0/1196343319 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa18c07bec0 msgr2=0x7fa18c19b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:08.558 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:08.011+0000 7fa191b9b640 1 --2- 172.21.15.114:0/1196343319 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa18c07bec0 0x7fa18c19b4b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa17c030560 tx=0x7fa17c030d90 comp rx=0 tx=0).stop 2024-09-13T18:13:08.558 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:08.011+0000 7fa191b9b640 1 -- 172.21.15.114:0/1196343319 shutdown_connections 2024-09-13T18:13:08.558 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:08.011+0000 7fa191b9b640 1 --2- 172.21.15.114:0/1196343319 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fa174038620 0x7fa17403aae0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:08.558 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:08.011+0000 7fa191b9b640 1 --2- 172.21.15.114:0/1196343319 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa18c07bec0 0x7fa18c19b4b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:08.558 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:08.011+0000 7fa191b9b640 1 -- 172.21.15.114:0/1196343319 >> 172.21.15.114:0/1196343319 conn(0x7fa18c100db0 msgr2=0x7fa18c1019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:08.558 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:08.011+0000 7fa191b9b640 1 -- 172.21.15.114:0/1196343319 shutdown_connections 2024-09-13T18:13:08.558 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:08.011+0000 7fa191b9b640 1 -- 172.21.15.114:0/1196343319 wait complete. 2024-09-13T18:13:09.219 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:09 smithi114 ceph-mon[30897]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:13:09.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:09 smithi114 ceph-mon[30897]: Generating ssh key... 2024-09-13T18:13:09.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:09 smithi114 ceph-mon[30897]: from='mgr.14116 172.21.15.114:0/3845812472' entity='mgr.x' 2024-09-13T18:13:09.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:09 smithi114 ceph-mon[30897]: from='mgr.14116 172.21.15.114:0/3845812472' entity='mgr.x' 2024-09-13T18:13:09.745 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3XTyIttOV4ZdNWkAxkfStaCiCLQupZwaIq1ERyZHL8prVEvdx10ZBmTzOVb48lZFK0xK+EedWHZQv4SaQc+FfKW7mMyMji9FXRo45SJzmXp59iNUoHqfMEZPRN2otgRDj8KdroMELAwUtx4loSUFrZNsf7xlOzzbE96TmNam7FzXYlXqEL1cuq1vhTuCzTkpKw/qxF4JDX4oGlv54dCpYeEoydMhykK92WZP3D6ZCvjRqv07alXomjSSblBYTnDdLzVDZt+5fIueLZNqvGRKaSHqctV+HzG1zfyMpGfFDDQhMnNs3l0C8Tk235KdPirUbJoHlLs6VgsGPzhkE5hg9Sk4vqldmx0J8/nDkVklOCEzCuuRhUwgc4vtPedSgLeidCfUo2mqo4FJ6p+7C2gpONkxrBCX3T1CvFRzZhxzImtWcpiefzgm/bDUaLzn1OHcnHDWLr2l82hP271ucUV/J86URXmCMI1Q/qdpYkBNJrihanvmFliUXgkSbB9vuiE8= ceph-97e7edf2-71fb-11ef-bceb-c7b262605968 2024-09-13T18:13:09.745 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:08.931+0000 7fae244ce640 1 Processor -- start 2024-09-13T18:13:09.746 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:08.931+0000 7fae244ce640 1 -- start start 2024-09-13T18:13:09.746 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:08.931+0000 7fae244ce640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fae1c1055a0 0x7fae1c1059a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:09.746 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:08.931+0000 7fae244ce640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1c105f70 con 0x7fae1c1055a0 2024-09-13T18:13:09.746 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:08.931+0000 7fae2226a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fae1c1055a0 0x7fae1c1059a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:09.746 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:08.931+0000 7fae2226a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fae1c1055a0 0x7fae1c1059a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60204/0 (socket says 172.21.15.114:60204) 2024-09-13T18:13:09.746 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:08.931+0000 7fae2226a640 1 -- 172.21.15.114:0/2727590827 learned_addr learned my addr 172.21.15.114:0/2727590827 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:13:09.746 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:08.931+0000 7fae2226a640 1 -- 172.21.15.114:0/2727590827 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae1c1067a0 con 0x7fae1c1055a0 2024-09-13T18:13:09.746 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:08.932+0000 7fae2226a640 1 --2- 172.21.15.114:0/2727590827 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fae1c1055a0 0x7fae1c1059a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fae18009820 tx=0x7fae1802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecf831e4814a1ec2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:09.746 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:08.932+0000 7fae21268640 1 -- 172.21.15.114:0/2727590827 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fae1802ae70 con 0x7fae1c1055a0 2024-09-13T18:13:09.746 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:08.932+0000 7fae21268640 1 -- 172.21.15.114:0/2727590827 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fae180328b0 con 0x7fae1c1055a0 2024-09-13T18:13:09.746 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:08.932+0000 7fae21268640 1 -- 172.21.15.114:0/2727590827 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fae18030850 con 0x7fae1c1055a0 2024-09-13T18:13:09.746 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:08.933+0000 7fae244ce640 1 -- 172.21.15.114:0/2727590827 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fae1c1055a0 msgr2=0x7fae1c1059a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:09.747 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:08.933+0000 7fae244ce640 1 --2- 172.21.15.114:0/2727590827 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fae1c1055a0 0x7fae1c1059a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fae18009820 tx=0x7fae1802a270 comp rx=0 tx=0).stop 2024-09-13T18:13:09.747 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:08.933+0000 7fae244ce640 1 -- 172.21.15.114:0/2727590827 shutdown_connections 2024-09-13T18:13:09.747 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:08.933+0000 7fae244ce640 1 --2- 172.21.15.114:0/2727590827 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fae1c1055a0 0x7fae1c1059a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:09.747 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:08.933+0000 7fae244ce640 1 -- 172.21.15.114:0/2727590827 >> 172.21.15.114:0/2727590827 conn(0x7fae1c100db0 msgr2=0x7fae1c1031d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:09.747 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:08.933+0000 7fae244ce640 1 -- 172.21.15.114:0/2727590827 shutdown_connections 2024-09-13T18:13:09.747 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:08.933+0000 7fae244ce640 1 -- 172.21.15.114:0/2727590827 wait complete. 2024-09-13T18:13:09.747 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:08.933+0000 7fae244ce640 1 Processor -- start 2024-09-13T18:13:09.747 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:08.934+0000 7fae244ce640 1 -- start start 2024-09-13T18:13:09.747 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:08.934+0000 7fae244ce640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fae1c1055a0 0x7fae1c07bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:09.747 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:08.934+0000 7fae244ce640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1c07c0d0 con 0x7fae1c1055a0 2024-09-13T18:13:09.747 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:08.934+0000 7fae2226a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fae1c1055a0 0x7fae1c07bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:09.747 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:08.934+0000 7fae2226a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fae1c1055a0 0x7fae1c07bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60216/0 (socket says 172.21.15.114:60216) 2024-09-13T18:13:09.747 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:08.934+0000 7fae2226a640 1 -- 172.21.15.114:0/781899064 learned_addr learned my addr 172.21.15.114:0/781899064 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:13:09.747 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:08.934+0000 7fae2226a640 1 -- 172.21.15.114:0/781899064 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae180094d0 con 0x7fae1c1055a0 2024-09-13T18:13:09.747 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:08.934+0000 7fae2226a640 1 --2- 172.21.15.114:0/781899064 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fae1c1055a0 0x7fae1c07bb90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fae18030560 tx=0x7fae18030d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:09.747 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:08.934+0000 7fae137fe640 1 -- 172.21.15.114:0/781899064 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fae18032d60 con 0x7fae1c1055a0 2024-09-13T18:13:09.748 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:08.934+0000 7fae137fe640 1 -- 172.21.15.114:0/781899064 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fae1803b8a0 con 0x7fae1c1055a0 2024-09-13T18:13:09.748 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:08.934+0000 7fae244ce640 1 -- 172.21.15.114:0/781899064 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae1c107620 con 0x7fae1c1055a0 2024-09-13T18:13:09.748 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:08.934+0000 7fae137fe640 1 -- 172.21.15.114:0/781899064 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fae1803a5f0 con 0x7fae1c1055a0 2024-09-13T18:13:09.748 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:08.934+0000 7fae244ce640 1 -- 172.21.15.114:0/781899064 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae1c07a510 con 0x7fae1c1055a0 2024-09-13T18:13:09.748 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:08.935+0000 7fae137fe640 1 -- 172.21.15.114:0/781899064 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7fae18039040 con 0x7fae1c1055a0 2024-09-13T18:13:09.748 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:08.935+0000 7fae244ce640 1 -- 172.21.15.114:0/781899064 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae1c1099d0 con 0x7fae1c1055a0 2024-09-13T18:13:09.748 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:08.935+0000 7fae137fe640 1 --2- 172.21.15.114:0/781899064 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fae08038620 0x7fae0803aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:09.748 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:08.935+0000 7fae137fe640 1 -- 172.21.15.114:0/781899064 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fae1806bab0 con 0x7fae1c1055a0 2024-09-13T18:13:09.748 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:08.936+0000 7fae21a69640 1 --2- 172.21.15.114:0/781899064 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fae08038620 0x7fae0803aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:09.748 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:08.936+0000 7fae21a69640 1 --2- 172.21.15.114:0/781899064 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fae08038620 0x7fae0803aae0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fae0c0098c0 tx=0x7fae0c006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:09.748 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:08.940+0000 7fae137fe640 1 -- 172.21.15.114:0/781899064 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fae1804b6c0 con 0x7fae1c1055a0 2024-09-13T18:13:09.748 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:09.103+0000 7fae244ce640 1 -- 172.21.15.114:0/781899064 --> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fae1c07ac70 con 0x7fae08038620 2024-09-13T18:13:09.748 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:09.104+0000 7fae137fe640 1 -- 172.21.15.114:0/781899064 <== mgr.14116 v2:172.21.15.114:6800/4281866858 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fae1c07ac70 con 0x7fae08038620 2024-09-13T18:13:09.749 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:09.106+0000 7fae244ce640 1 -- 172.21.15.114:0/781899064 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fae08038620 msgr2=0x7fae0803aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:09.749 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:09.106+0000 7fae244ce640 1 --2- 172.21.15.114:0/781899064 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fae08038620 0x7fae0803aae0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fae0c0098c0 tx=0x7fae0c006db0 comp rx=0 tx=0).stop 2024-09-13T18:13:09.749 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:09.106+0000 7fae244ce640 1 -- 172.21.15.114:0/781899064 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fae1c1055a0 msgr2=0x7fae1c07bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:09.749 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:09.106+0000 7fae244ce640 1 --2- 172.21.15.114:0/781899064 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fae1c1055a0 0x7fae1c07bb90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fae18030560 tx=0x7fae18030d90 comp rx=0 tx=0).stop 2024-09-13T18:13:09.749 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:09.106+0000 7fae244ce640 1 -- 172.21.15.114:0/781899064 shutdown_connections 2024-09-13T18:13:09.749 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:09.106+0000 7fae244ce640 1 --2- 172.21.15.114:0/781899064 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fae08038620 0x7fae0803aae0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:09.749 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:09.106+0000 7fae244ce640 1 --2- 172.21.15.114:0/781899064 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fae1c1055a0 0x7fae1c07bb90 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:09.749 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:09.106+0000 7fae244ce640 1 -- 172.21.15.114:0/781899064 >> 172.21.15.114:0/781899064 conn(0x7fae1c100db0 msgr2=0x7fae1c1019b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:09.749 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:09.106+0000 7fae244ce640 1 -- 172.21.15.114:0/781899064 shutdown_connections 2024-09-13T18:13:09.749 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:09.106+0000 7fae244ce640 1 -- 172.21.15.114:0/781899064 wait complete. 2024-09-13T18:13:09.749 INFO:teuthology.orchestra.run.smithi114.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-13T18:13:09.749 INFO:teuthology.orchestra.run.smithi114.stdout:Adding key to root@localhost authorized_keys... 2024-09-13T18:13:09.750 INFO:teuthology.orchestra.run.smithi114.stdout:Adding host smithi114... 2024-09-13T18:13:10.288 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:10 smithi114 ceph-mon[30897]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:13:11.078 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:11 smithi114 ceph-mon[30897]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi114", "addr": "172.21.15.114", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:13:12.398 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:12 smithi114 ceph-mon[30897]: Deploying cephadm binary to smithi114 2024-09-13T18:13:13.486 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout Added host 'smithi114' with addr '172.21.15.114' 2024-09-13T18:13:13.486 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:10.108+0000 7fda00437640 1 Processor -- start 2024-09-13T18:13:13.486 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:10.108+0000 7fda00437640 1 -- start start 2024-09-13T18:13:13.486 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:10.108+0000 7fda00437640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9f8104470 0x7fd9f8104870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:13.486 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:10.108+0000 7fda00437640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f8104e40 con 0x7fd9f8104470 2024-09-13T18:13:13.486 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:10.108+0000 7fd9fe1d3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9f8104470 0x7fd9f8104870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:13.486 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:10.108+0000 7fd9fe1d3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9f8104470 0x7fd9f8104870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60226/0 (socket says 172.21.15.114:60226) 2024-09-13T18:13:13.486 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:10.108+0000 7fd9fe1d3640 1 -- 172.21.15.114:0/744284056 learned_addr learned my addr 172.21.15.114:0/744284056 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:13:13.486 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:10.108+0000 7fd9fe1d3640 1 -- 172.21.15.114:0/744284056 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f8105670 con 0x7fd9f8104470 2024-09-13T18:13:13.486 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:10.109+0000 7fd9fe1d3640 1 --2- 172.21.15.114:0/744284056 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9f8104470 0x7fd9f8104870 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4009820 tx=0x7fd9f402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ce57df569cbbad1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:13.487 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:10.109+0000 7fd9fd1d1640 1 -- 172.21.15.114:0/744284056 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd9f402ae70 con 0x7fd9f8104470 2024-09-13T18:13:13.487 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:10.109+0000 7fd9fd1d1640 1 -- 172.21.15.114:0/744284056 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fd9f40328b0 con 0x7fd9f8104470 2024-09-13T18:13:13.487 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:10.109+0000 7fd9fd1d1640 1 -- 172.21.15.114:0/744284056 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd9f4030850 con 0x7fd9f8104470 2024-09-13T18:13:13.487 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:10.110+0000 7fda00437640 1 -- 172.21.15.114:0/744284056 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9f8104470 msgr2=0x7fd9f8104870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:13.487 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:10.110+0000 7fda00437640 1 --2- 172.21.15.114:0/744284056 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9f8104470 0x7fd9f8104870 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4009820 tx=0x7fd9f402a270 comp rx=0 tx=0).stop 2024-09-13T18:13:13.487 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:10.110+0000 7fda00437640 1 -- 172.21.15.114:0/744284056 shutdown_connections 2024-09-13T18:13:13.487 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:10.110+0000 7fda00437640 1 --2- 172.21.15.114:0/744284056 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9f8104470 0x7fd9f8104870 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:13.487 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:10.110+0000 7fda00437640 1 -- 172.21.15.114:0/744284056 >> 172.21.15.114:0/744284056 conn(0x7fd9f80ffc60 msgr2=0x7fd9f8102080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:13.487 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:10.110+0000 7fda00437640 1 -- 172.21.15.114:0/744284056 shutdown_connections 2024-09-13T18:13:13.487 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:10.110+0000 7fda00437640 1 -- 172.21.15.114:0/744284056 wait complete. 2024-09-13T18:13:13.487 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:10.111+0000 7fda00437640 1 Processor -- start 2024-09-13T18:13:13.487 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:10.111+0000 7fda00437640 1 -- start start 2024-09-13T18:13:13.487 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:10.111+0000 7fda00437640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9f8104470 0x7fd9f8079a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:13.488 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:10.111+0000 7fda00437640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f8079fb0 con 0x7fd9f8104470 2024-09-13T18:13:13.488 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:10.111+0000 7fd9fe1d3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9f8104470 0x7fd9f8079a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:13.488 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:10.111+0000 7fd9fe1d3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9f8104470 0x7fd9f8079a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60230/0 (socket says 172.21.15.114:60230) 2024-09-13T18:13:13.488 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:10.111+0000 7fd9fe1d3640 1 -- 172.21.15.114:0/2443961504 learned_addr learned my addr 172.21.15.114:0/2443961504 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:13:13.488 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:10.111+0000 7fd9fe1d3640 1 -- 172.21.15.114:0/2443961504 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f40094d0 con 0x7fd9f8104470 2024-09-13T18:13:13.488 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:10.111+0000 7fd9fe1d3640 1 --2- 172.21.15.114:0/2443961504 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9f8104470 0x7fd9f8079a70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd9f402a790 tx=0x7fd9f4030db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:13.488 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:10.111+0000 7fd9ef7fe640 1 -- 172.21.15.114:0/2443961504 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd9f4032d60 con 0x7fd9f8104470 2024-09-13T18:13:13.488 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:10.112+0000 7fd9ef7fe640 1 -- 172.21.15.114:0/2443961504 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fd9f403b8a0 con 0x7fd9f8104470 2024-09-13T18:13:13.488 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:10.112+0000 7fda00437640 1 -- 172.21.15.114:0/2443961504 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9f81064f0 con 0x7fd9f8104470 2024-09-13T18:13:13.488 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:10.112+0000 7fd9ef7fe640 1 -- 172.21.15.114:0/2443961504 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd9f403a640 con 0x7fd9f8104470 2024-09-13T18:13:13.488 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:10.112+0000 7fda00437640 1 -- 172.21.15.114:0/2443961504 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9f819c3f0 con 0x7fd9f8104470 2024-09-13T18:13:13.488 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:10.112+0000 7fd9ef7fe640 1 -- 172.21.15.114:0/2443961504 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7fd9f4039040 con 0x7fd9f8104470 2024-09-13T18:13:13.488 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:10.113+0000 7fd9ef7fe640 1 --2- 172.21.15.114:0/2443961504 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fd9e4038670 0x7fd9e403ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:13.489 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:10.113+0000 7fd9ef7fe640 1 -- 172.21.15.114:0/2443961504 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd9f406ba30 con 0x7fd9f8104470 2024-09-13T18:13:13.489 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:10.113+0000 7fda00437640 1 -- 172.21.15.114:0/2443961504 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9f81088a0 con 0x7fd9f8104470 2024-09-13T18:13:13.489 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:10.113+0000 7fd9fd9d2640 1 --2- 172.21.15.114:0/2443961504 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fd9e4038670 0x7fd9e403ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:13.489 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:10.113+0000 7fd9fd9d2640 1 --2- 172.21.15.114:0/2443961504 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fd9e4038670 0x7fd9e403ab30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd9e80098c0 tx=0x7fd9e8006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:13.489 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:10.117+0000 7fd9ef7fe640 1 -- 172.21.15.114:0/2443961504 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fd9f403f070 con 0x7fd9f8104470 2024-09-13T18:13:13.489 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:10.284+0000 7fda00437640 1 -- 172.21.15.114:0/2443961504 --> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi114", "addr": "172.21.15.114", "target": ["mon-mgr", ""]}) v1 -- 0x7fd9f8100e80 con 0x7fd9e4038670 2024-09-13T18:13:13.489 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:12.906+0000 7fd9ef7fe640 1 -- 172.21.15.114:0/2443961504 <== mgr.14116 v2:172.21.15.114:6800/4281866858 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fd9f8100e80 con 0x7fd9e4038670 2024-09-13T18:13:13.489 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:12.908+0000 7fda00437640 1 -- 172.21.15.114:0/2443961504 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fd9e4038670 msgr2=0x7fd9e403ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:13.489 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:12.908+0000 7fda00437640 1 --2- 172.21.15.114:0/2443961504 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fd9e4038670 0x7fd9e403ab30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd9e80098c0 tx=0x7fd9e8006db0 comp rx=0 tx=0).stop 2024-09-13T18:13:13.489 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:12.908+0000 7fda00437640 1 -- 172.21.15.114:0/2443961504 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9f8104470 msgr2=0x7fd9f8079a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:13.489 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:12.908+0000 7fda00437640 1 --2- 172.21.15.114:0/2443961504 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9f8104470 0x7fd9f8079a70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd9f402a790 tx=0x7fd9f4030db0 comp rx=0 tx=0).stop 2024-09-13T18:13:13.489 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:12.908+0000 7fda00437640 1 -- 172.21.15.114:0/2443961504 shutdown_connections 2024-09-13T18:13:13.489 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:12.908+0000 7fda00437640 1 --2- 172.21.15.114:0/2443961504 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fd9e4038670 0x7fd9e403ab30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:13.489 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:12.908+0000 7fda00437640 1 --2- 172.21.15.114:0/2443961504 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9f8104470 0x7fd9f8079a70 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:13.490 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:12.908+0000 7fda00437640 1 -- 172.21.15.114:0/2443961504 >> 172.21.15.114:0/2443961504 conn(0x7fd9f80ffc60 msgr2=0x7fd9f8100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:13.490 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:12.909+0000 7fda00437640 1 -- 172.21.15.114:0/2443961504 shutdown_connections 2024-09-13T18:13:13.490 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:12.909+0000 7fda00437640 1 -- 172.21.15.114:0/2443961504 wait complete. 2024-09-13T18:13:13.490 INFO:teuthology.orchestra.run.smithi114.stdout:Deploying unmanaged mon service... 2024-09-13T18:13:14.073 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:13 smithi114 ceph-mon[30897]: from='mgr.14116 172.21.15.114:0/3845812472' entity='mgr.x' 2024-09-13T18:13:14.073 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:13 smithi114 ceph-mon[30897]: Added host smithi114 2024-09-13T18:13:14.073 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:13 smithi114 ceph-mon[30897]: from='mgr.14116 172.21.15.114:0/3845812472' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:13:15.469 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:15 smithi114 ceph-mon[30897]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:13:15.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:15 smithi114 ceph-mon[30897]: Saving service mon spec with placement count:5 2024-09-13T18:13:15.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:15 smithi114 ceph-mon[30897]: from='mgr.14116 172.21.15.114:0/3845812472' entity='mgr.x' 2024-09-13T18:13:15.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:15 smithi114 ceph-mon[30897]: from='mgr.14116 172.21.15.114:0/3845812472' entity='mgr.x' 2024-09-13T18:13:15.491 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-13T18:13:15.491 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:13.884+0000 7fd30cad2640 1 Processor -- start 2024-09-13T18:13:15.491 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:13.884+0000 7fd30cad2640 1 -- start start 2024-09-13T18:13:15.491 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:13.884+0000 7fd30cad2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd308106080 0x7fd308108470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:15.491 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:13.884+0000 7fd30cad2640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd308079e50 con 0x7fd308106080 2024-09-13T18:13:15.491 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:13.884+0000 7fd30659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd308106080 0x7fd308108470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:15.491 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:13.884+0000 7fd30659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd308106080 0x7fd308108470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60232/0 (socket says 172.21.15.114:60232) 2024-09-13T18:13:15.491 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:13.884+0000 7fd30659c640 1 -- 172.21.15.114:0/403241278 learned_addr learned my addr 172.21.15.114:0/403241278 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:13:15.491 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:13.884+0000 7fd30659c640 1 -- 172.21.15.114:0/403241278 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3081089b0 con 0x7fd308106080 2024-09-13T18:13:15.492 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:13.885+0000 7fd30659c640 1 --2- 172.21.15.114:0/403241278 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd308106080 0x7fd308108470 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd300009820 tx=0x7fd30002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fc86036b92a6457 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:15.492 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:13.885+0000 7fd30559a640 1 -- 172.21.15.114:0/403241278 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd30002ae70 con 0x7fd308106080 2024-09-13T18:13:15.492 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:13.885+0000 7fd30559a640 1 -- 172.21.15.114:0/403241278 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fd3000328b0 con 0x7fd308106080 2024-09-13T18:13:15.492 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:13.885+0000 7fd30559a640 1 -- 172.21.15.114:0/403241278 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd300030850 con 0x7fd308106080 2024-09-13T18:13:15.492 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:13.886+0000 7fd30cad2640 1 -- 172.21.15.114:0/403241278 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd308106080 msgr2=0x7fd308108470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:15.492 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:13.886+0000 7fd30cad2640 1 --2- 172.21.15.114:0/403241278 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd308106080 0x7fd308108470 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd300009820 tx=0x7fd30002a270 comp rx=0 tx=0).stop 2024-09-13T18:13:15.492 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:13.886+0000 7fd30cad2640 1 -- 172.21.15.114:0/403241278 shutdown_connections 2024-09-13T18:13:15.492 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:13.886+0000 7fd30cad2640 1 --2- 172.21.15.114:0/403241278 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd308106080 0x7fd308108470 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:15.492 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:13.886+0000 7fd30cad2640 1 -- 172.21.15.114:0/403241278 >> 172.21.15.114:0/403241278 conn(0x7fd3080ffc60 msgr2=0x7fd308102080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:15.492 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:13.886+0000 7fd30cad2640 1 -- 172.21.15.114:0/403241278 shutdown_connections 2024-09-13T18:13:15.492 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:13.886+0000 7fd30cad2640 1 -- 172.21.15.114:0/403241278 wait complete. 2024-09-13T18:13:15.492 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:13.887+0000 7fd30cad2640 1 Processor -- start 2024-09-13T18:13:15.492 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:13.887+0000 7fd30cad2640 1 -- start start 2024-09-13T18:13:15.493 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:13.887+0000 7fd30cad2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd308106080 0x7fd308199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:15.493 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:13.887+0000 7fd30cad2640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd308199760 con 0x7fd308106080 2024-09-13T18:13:15.493 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:13.887+0000 7fd30659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd308106080 0x7fd308199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:15.493 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:13.887+0000 7fd30659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd308106080 0x7fd308199220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60246/0 (socket says 172.21.15.114:60246) 2024-09-13T18:13:15.493 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:13.887+0000 7fd30659c640 1 -- 172.21.15.114:0/1060876479 learned_addr learned my addr 172.21.15.114:0/1060876479 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:13:15.493 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:13.887+0000 7fd30659c640 1 -- 172.21.15.114:0/1060876479 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3000094d0 con 0x7fd308106080 2024-09-13T18:13:15.493 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:13.888+0000 7fd30659c640 1 --2- 172.21.15.114:0/1060876479 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd308106080 0x7fd308199220 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd300030560 tx=0x7fd300030d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:15.493 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:13.888+0000 7fd2ff7fe640 1 -- 172.21.15.114:0/1060876479 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd300032d60 con 0x7fd308106080 2024-09-13T18:13:15.493 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:13.888+0000 7fd2ff7fe640 1 -- 172.21.15.114:0/1060876479 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fd30003b8a0 con 0x7fd308106080 2024-09-13T18:13:15.493 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:13.888+0000 7fd30cad2640 1 -- 172.21.15.114:0/1060876479 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd308199960 con 0x7fd308106080 2024-09-13T18:13:15.493 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:13.888+0000 7fd2ff7fe640 1 -- 172.21.15.114:0/1060876479 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd30003a740 con 0x7fd308106080 2024-09-13T18:13:15.493 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:13.888+0000 7fd30cad2640 1 -- 172.21.15.114:0/1060876479 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd308199e00 con 0x7fd308106080 2024-09-13T18:13:15.494 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:13.889+0000 7fd2ff7fe640 1 -- 172.21.15.114:0/1060876479 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7fd300039040 con 0x7fd308106080 2024-09-13T18:13:15.494 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:13.889+0000 7fd2ff7fe640 1 --2- 172.21.15.114:0/1060876479 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fd2f4038670 0x7fd2f403ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:15.494 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:13.889+0000 7fd30cad2640 1 -- 172.21.15.114:0/1060876479 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd308109710 con 0x7fd308106080 2024-09-13T18:13:15.494 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:13.889+0000 7fd2ff7fe640 1 -- 172.21.15.114:0/1060876479 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd30006bac0 con 0x7fd308106080 2024-09-13T18:13:15.494 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:13.889+0000 7fd305d9b640 1 --2- 172.21.15.114:0/1060876479 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fd2f4038670 0x7fd2f403ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:15.494 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:13.889+0000 7fd305d9b640 1 --2- 172.21.15.114:0/1060876479 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fd2f4038670 0x7fd2f403ab30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd2f00098c0 tx=0x7fd2f0006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:15.494 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:13.894+0000 7fd2ff7fe640 1 -- 172.21.15.114:0/1060876479 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fd300037070 con 0x7fd308106080 2024-09-13T18:13:15.494 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:14.070+0000 7fd30cad2640 1 -- 172.21.15.114:0/1060876479 --> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fd308100f50 con 0x7fd2f4038670 2024-09-13T18:13:15.494 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:14.074+0000 7fd2ff7fe640 1 -- 172.21.15.114:0/1060876479 <== mgr.14116 v2:172.21.15.114:6800/4281866858 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fd308100f50 con 0x7fd2f4038670 2024-09-13T18:13:15.494 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:14.076+0000 7fd30cad2640 1 -- 172.21.15.114:0/1060876479 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fd2f4038670 msgr2=0x7fd2f403ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:15.494 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:14.076+0000 7fd30cad2640 1 --2- 172.21.15.114:0/1060876479 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fd2f4038670 0x7fd2f403ab30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd2f00098c0 tx=0x7fd2f0006db0 comp rx=0 tx=0).stop 2024-09-13T18:13:15.494 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:14.076+0000 7fd30cad2640 1 -- 172.21.15.114:0/1060876479 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd308106080 msgr2=0x7fd308199220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:15.495 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:14.076+0000 7fd30cad2640 1 --2- 172.21.15.114:0/1060876479 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd308106080 0x7fd308199220 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd300030560 tx=0x7fd300030d90 comp rx=0 tx=0).stop 2024-09-13T18:13:15.495 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:14.076+0000 7fd30cad2640 1 -- 172.21.15.114:0/1060876479 shutdown_connections 2024-09-13T18:13:15.495 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:14.076+0000 7fd30cad2640 1 --2- 172.21.15.114:0/1060876479 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fd2f4038670 0x7fd2f403ab30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:15.495 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:14.076+0000 7fd30cad2640 1 --2- 172.21.15.114:0/1060876479 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd308106080 0x7fd308199220 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:15.495 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:14.076+0000 7fd30cad2640 1 -- 172.21.15.114:0/1060876479 >> 172.21.15.114:0/1060876479 conn(0x7fd3080ffc60 msgr2=0x7fd308100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:15.495 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:14.076+0000 7fd30cad2640 1 -- 172.21.15.114:0/1060876479 shutdown_connections 2024-09-13T18:13:15.495 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:14.076+0000 7fd30cad2640 1 -- 172.21.15.114:0/1060876479 wait complete. 2024-09-13T18:13:15.495 INFO:teuthology.orchestra.run.smithi114.stdout:Deploying unmanaged mgr service... 2024-09-13T18:13:16.706 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-13T18:13:16.706 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:15.877+0000 7fa650ea3640 1 Processor -- start 2024-09-13T18:13:16.706 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:15.877+0000 7fa650ea3640 1 -- start start 2024-09-13T18:13:16.706 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:15.877+0000 7fa650ea3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa64c07bec0 0x7fa64c07a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:16.706 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:15.877+0000 7fa650ea3640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c07a8f0 con 0x7fa64c07bec0 2024-09-13T18:13:16.706 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:15.877+0000 7fa64a59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa64c07bec0 0x7fa64c07a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:16.706 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:15.877+0000 7fa64a59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa64c07bec0 0x7fa64c07a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60256/0 (socket says 172.21.15.114:60256) 2024-09-13T18:13:16.707 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:15.877+0000 7fa64a59c640 1 -- 172.21.15.114:0/386750876 learned_addr learned my addr 172.21.15.114:0/386750876 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:13:16.707 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:15.877+0000 7fa64a59c640 1 -- 172.21.15.114:0/386750876 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa64c07aa30 con 0x7fa64c07bec0 2024-09-13T18:13:16.707 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:15.878+0000 7fa64a59c640 1 --2- 172.21.15.114:0/386750876 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa64c07bec0 0x7fa64c07a320 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa63c009820 tx=0x7fa63c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a4dd8d1f0747f56 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:16.707 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:15.878+0000 7fa64959a640 1 -- 172.21.15.114:0/386750876 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa63c02ae70 con 0x7fa64c07bec0 2024-09-13T18:13:16.707 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:15.878+0000 7fa64959a640 1 -- 172.21.15.114:0/386750876 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa63c0328d0 con 0x7fa64c07bec0 2024-09-13T18:13:16.707 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:15.878+0000 7fa64959a640 1 -- 172.21.15.114:0/386750876 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa63c0307a0 con 0x7fa64c07bec0 2024-09-13T18:13:16.707 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:15.879+0000 7fa650ea3640 1 -- 172.21.15.114:0/386750876 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa64c07bec0 msgr2=0x7fa64c07a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:16.707 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:15.879+0000 7fa650ea3640 1 --2- 172.21.15.114:0/386750876 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa64c07bec0 0x7fa64c07a320 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa63c009820 tx=0x7fa63c02a270 comp rx=0 tx=0).stop 2024-09-13T18:13:16.707 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:15.879+0000 7fa650ea3640 1 -- 172.21.15.114:0/386750876 shutdown_connections 2024-09-13T18:13:16.707 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:15.879+0000 7fa650ea3640 1 --2- 172.21.15.114:0/386750876 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa64c07bec0 0x7fa64c07a320 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:16.707 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:15.879+0000 7fa650ea3640 1 -- 172.21.15.114:0/386750876 >> 172.21.15.114:0/386750876 conn(0x7fa64c100de0 msgr2=0x7fa64c103200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:16.707 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:15.879+0000 7fa650ea3640 1 -- 172.21.15.114:0/386750876 shutdown_connections 2024-09-13T18:13:16.707 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:15.879+0000 7fa650ea3640 1 -- 172.21.15.114:0/386750876 wait complete. 2024-09-13T18:13:16.708 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:15.880+0000 7fa650ea3640 1 Processor -- start 2024-09-13T18:13:16.708 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:15.880+0000 7fa650ea3640 1 -- start start 2024-09-13T18:13:16.708 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:15.880+0000 7fa650ea3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa64c07bec0 0x7fa64c19b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:16.708 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:15.880+0000 7fa650ea3640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c19ba90 con 0x7fa64c07bec0 2024-09-13T18:13:16.708 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:15.880+0000 7fa64a59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa64c07bec0 0x7fa64c19b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:16.708 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:15.880+0000 7fa64a59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa64c07bec0 0x7fa64c19b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60272/0 (socket says 172.21.15.114:60272) 2024-09-13T18:13:16.708 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:15.880+0000 7fa64a59c640 1 -- 172.21.15.114:0/1054768245 learned_addr learned my addr 172.21.15.114:0/1054768245 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:13:16.708 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:15.880+0000 7fa64a59c640 1 -- 172.21.15.114:0/1054768245 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa63c0094d0 con 0x7fa64c07bec0 2024-09-13T18:13:16.708 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:15.880+0000 7fa64a59c640 1 --2- 172.21.15.114:0/1054768245 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa64c07bec0 0x7fa64c19b550 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa63c035040 tx=0x7fa63c030ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:16.708 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:15.880+0000 7fa63b7fe640 1 -- 172.21.15.114:0/1054768245 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa63c032c50 con 0x7fa64c07bec0 2024-09-13T18:13:16.708 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:15.880+0000 7fa63b7fe640 1 -- 172.21.15.114:0/1054768245 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa63c032db0 con 0x7fa64c07bec0 2024-09-13T18:13:16.708 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:15.880+0000 7fa63b7fe640 1 -- 172.21.15.114:0/1054768245 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa63c03a5d0 con 0x7fa64c07bec0 2024-09-13T18:13:16.708 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:15.880+0000 7fa650ea3640 1 -- 172.21.15.114:0/1054768245 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa64c19bc90 con 0x7fa64c07bec0 2024-09-13T18:13:16.709 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:15.881+0000 7fa650ea3640 1 -- 172.21.15.114:0/1054768245 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa64c19c130 con 0x7fa64c07bec0 2024-09-13T18:13:16.709 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:15.881+0000 7fa63b7fe640 1 -- 172.21.15.114:0/1054768245 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7fa63c03a780 con 0x7fa64c07bec0 2024-09-13T18:13:16.709 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:15.881+0000 7fa63b7fe640 1 --2- 172.21.15.114:0/1054768245 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fa634038670 0x7fa63403ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:16.709 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:15.881+0000 7fa63b7fe640 1 -- 172.21.15.114:0/1054768245 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa63c06b9f0 con 0x7fa64c07bec0 2024-09-13T18:13:16.709 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:15.882+0000 7fa650ea3640 1 -- 172.21.15.114:0/1054768245 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa64c10ba00 con 0x7fa64c07bec0 2024-09-13T18:13:16.709 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:15.882+0000 7fa649d9b640 1 --2- 172.21.15.114:0/1054768245 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fa634038670 0x7fa63403ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:16.709 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:15.882+0000 7fa649d9b640 1 --2- 172.21.15.114:0/1054768245 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fa634038670 0x7fa63403ab30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa6400098c0 tx=0x7fa640006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:16.709 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:15.886+0000 7fa63b7fe640 1 -- 172.21.15.114:0/1054768245 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fa63c037070 con 0x7fa64c07bec0 2024-09-13T18:13:16.709 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:16.064+0000 7fa650ea3640 1 -- 172.21.15.114:0/1054768245 --> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fa64c102120 con 0x7fa634038670 2024-09-13T18:13:16.709 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:16.069+0000 7fa63b7fe640 1 -- 172.21.15.114:0/1054768245 <== mgr.14116 v2:172.21.15.114:6800/4281866858 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fa64c102120 con 0x7fa634038670 2024-09-13T18:13:16.709 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:16.072+0000 7fa650ea3640 1 -- 172.21.15.114:0/1054768245 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fa634038670 msgr2=0x7fa63403ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:16.709 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:16.072+0000 7fa650ea3640 1 --2- 172.21.15.114:0/1054768245 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fa634038670 0x7fa63403ab30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa6400098c0 tx=0x7fa640006db0 comp rx=0 tx=0).stop 2024-09-13T18:13:16.710 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:16.072+0000 7fa650ea3640 1 -- 172.21.15.114:0/1054768245 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa64c07bec0 msgr2=0x7fa64c19b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:16.710 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:16.072+0000 7fa650ea3640 1 --2- 172.21.15.114:0/1054768245 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa64c07bec0 0x7fa64c19b550 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa63c035040 tx=0x7fa63c030ce0 comp rx=0 tx=0).stop 2024-09-13T18:13:16.710 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:16.072+0000 7fa650ea3640 1 -- 172.21.15.114:0/1054768245 shutdown_connections 2024-09-13T18:13:16.710 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:16.072+0000 7fa650ea3640 1 --2- 172.21.15.114:0/1054768245 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fa634038670 0x7fa63403ab30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:16.710 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:16.072+0000 7fa650ea3640 1 --2- 172.21.15.114:0/1054768245 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa64c07bec0 0x7fa64c19b550 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:16.710 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:16.072+0000 7fa650ea3640 1 -- 172.21.15.114:0/1054768245 >> 172.21.15.114:0/1054768245 conn(0x7fa64c100de0 msgr2=0x7fa64c101a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:16.710 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:16.072+0000 7fa650ea3640 1 -- 172.21.15.114:0/1054768245 shutdown_connections 2024-09-13T18:13:16.710 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:16.072+0000 7fa650ea3640 1 -- 172.21.15.114:0/1054768245 wait complete. 2024-09-13T18:13:17.265 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:17 smithi114 ceph-mon[30897]: from='mgr.14116 172.21.15.114:0/3845812472' entity='mgr.x' 2024-09-13T18:13:17.266 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:17 smithi114 ceph-mon[30897]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:13:17.266 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:17 smithi114 ceph-mon[30897]: Saving service mgr spec with placement count:2 2024-09-13T18:13:17.266 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:17 smithi114 ceph-mon[30897]: from='mgr.14116 172.21.15.114:0/3845812472' entity='mgr.x' 2024-09-13T18:13:18.378 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:18 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2734618112' entity='client.admin' 2024-09-13T18:13:18.383 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:17.073+0000 7f0334b4a640 1 Processor -- start 2024-09-13T18:13:18.383 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:17.073+0000 7f0334b4a640 1 -- start start 2024-09-13T18:13:18.383 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:17.074+0000 7f0334b4a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0330104480 0x7f0330104880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:18.383 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:17.074+0000 7f0334b4a640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0330104e50 con 0x7f0330104480 2024-09-13T18:13:18.383 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:17.074+0000 7f032e59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0330104480 0x7f0330104880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:18.383 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:17.074+0000 7f032e59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0330104480 0x7f0330104880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60284/0 (socket says 172.21.15.114:60284) 2024-09-13T18:13:18.384 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:17.074+0000 7f032e59c640 1 -- 172.21.15.114:0/3754420144 learned_addr learned my addr 172.21.15.114:0/3754420144 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:13:18.384 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:17.074+0000 7f032e59c640 1 -- 172.21.15.114:0/3754420144 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0330105680 con 0x7f0330104480 2024-09-13T18:13:18.384 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:17.075+0000 7f032e59c640 1 --2- 172.21.15.114:0/3754420144 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0330104480 0x7f0330104880 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0320009820 tx=0x7f032002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89892161edbb24f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:18.384 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:17.075+0000 7f032d59a640 1 -- 172.21.15.114:0/3754420144 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f032002ae70 con 0x7f0330104480 2024-09-13T18:13:18.384 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:17.075+0000 7f032d59a640 1 -- 172.21.15.114:0/3754420144 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f03200328d0 con 0x7f0330104480 2024-09-13T18:13:18.384 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:17.075+0000 7f032d59a640 1 -- 172.21.15.114:0/3754420144 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f03200307a0 con 0x7f0330104480 2024-09-13T18:13:18.384 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:17.076+0000 7f0334b4a640 1 -- 172.21.15.114:0/3754420144 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0330104480 msgr2=0x7f0330104880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:18.384 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:17.076+0000 7f0334b4a640 1 --2- 172.21.15.114:0/3754420144 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0330104480 0x7f0330104880 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0320009820 tx=0x7f032002a270 comp rx=0 tx=0).stop 2024-09-13T18:13:18.384 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:17.076+0000 7f0334b4a640 1 -- 172.21.15.114:0/3754420144 shutdown_connections 2024-09-13T18:13:18.384 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:17.076+0000 7f0334b4a640 1 --2- 172.21.15.114:0/3754420144 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0330104480 0x7f0330104880 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:18.384 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:17.076+0000 7f0334b4a640 1 -- 172.21.15.114:0/3754420144 >> 172.21.15.114:0/3754420144 conn(0x7f03300ffc30 msgr2=0x7f0330102050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:18.384 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:17.076+0000 7f0334b4a640 1 -- 172.21.15.114:0/3754420144 shutdown_connections 2024-09-13T18:13:18.384 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:17.076+0000 7f0334b4a640 1 -- 172.21.15.114:0/3754420144 wait complete. 2024-09-13T18:13:18.385 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:17.077+0000 7f0334b4a640 1 Processor -- start 2024-09-13T18:13:18.385 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:17.077+0000 7f0334b4a640 1 -- start start 2024-09-13T18:13:18.385 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:17.077+0000 7f0334b4a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0330104480 0x7f0330079a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:18.385 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:17.077+0000 7f0334b4a640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0330079fb0 con 0x7f0330104480 2024-09-13T18:13:18.385 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:17.077+0000 7f032e59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0330104480 0x7f0330079a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:18.385 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:17.077+0000 7f032e59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0330104480 0x7f0330079a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60292/0 (socket says 172.21.15.114:60292) 2024-09-13T18:13:18.385 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:17.077+0000 7f032e59c640 1 -- 172.21.15.114:0/2734618112 learned_addr learned my addr 172.21.15.114:0/2734618112 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:13:18.385 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:17.078+0000 7f032e59c640 1 -- 172.21.15.114:0/2734618112 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03200094d0 con 0x7f0330104480 2024-09-13T18:13:18.385 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:17.078+0000 7f032e59c640 1 --2- 172.21.15.114:0/2734618112 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0330104480 0x7f0330079a70 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0320030380 tx=0x7f0320032d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:18.385 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:17.078+0000 7f031f7fe640 1 -- 172.21.15.114:0/2734618112 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0320030c70 con 0x7f0330104480 2024-09-13T18:13:18.385 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:17.078+0000 7f0334b4a640 1 -- 172.21.15.114:0/2734618112 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f033019bfd0 con 0x7f0330104480 2024-09-13T18:13:18.385 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:17.078+0000 7f031f7fe640 1 -- 172.21.15.114:0/2734618112 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0320030dd0 con 0x7f0330104480 2024-09-13T18:13:18.386 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:17.078+0000 7f031f7fe640 1 -- 172.21.15.114:0/2734618112 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f032003a5b0 con 0x7f0330104480 2024-09-13T18:13:18.386 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:17.078+0000 7f0334b4a640 1 -- 172.21.15.114:0/2734618112 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f033019c470 con 0x7f0330104480 2024-09-13T18:13:18.386 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:17.079+0000 7f031f7fe640 1 -- 172.21.15.114:0/2734618112 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f0320039040 con 0x7f0330104480 2024-09-13T18:13:18.386 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:17.079+0000 7f0334b4a640 1 -- 172.21.15.114:0/2734618112 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03301088b0 con 0x7f0330104480 2024-09-13T18:13:18.386 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:17.079+0000 7f031f7fe640 1 --2- 172.21.15.114:0/2734618112 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7f0318038620 0x7f031803aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:18.386 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:17.079+0000 7f031f7fe640 1 -- 172.21.15.114:0/2734618112 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f032006b940 con 0x7f0330104480 2024-09-13T18:13:18.386 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:17.079+0000 7f032dd9b640 1 --2- 172.21.15.114:0/2734618112 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7f0318038620 0x7f031803aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:18.386 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:17.080+0000 7f032dd9b640 1 --2- 172.21.15.114:0/2734618112 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7f0318038620 0x7f031803aae0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f03240098c0 tx=0x7f0324006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:18.386 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:17.084+0000 7f031f7fe640 1 -- 172.21.15.114:0/2734618112 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f032004b710 con 0x7f0330104480 2024-09-13T18:13:18.386 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:17.262+0000 7f0334b4a640 1 -- 172.21.15.114:0/2734618112 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f0330108ac0 con 0x7f0330104480 2024-09-13T18:13:18.386 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:17.266+0000 7f031f7fe640 1 -- 172.21.15.114:0/2734618112 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f032003f030 con 0x7f0330104480 2024-09-13T18:13:18.386 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:17.268+0000 7f0334b4a640 1 -- 172.21.15.114:0/2734618112 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7f0318038620 msgr2=0x7f031803aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:18.386 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:17.268+0000 7f0334b4a640 1 --2- 172.21.15.114:0/2734618112 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7f0318038620 0x7f031803aae0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f03240098c0 tx=0x7f0324006db0 comp rx=0 tx=0).stop 2024-09-13T18:13:18.386 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:17.268+0000 7f0334b4a640 1 -- 172.21.15.114:0/2734618112 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0330104480 msgr2=0x7f0330079a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:18.387 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:17.268+0000 7f0334b4a640 1 --2- 172.21.15.114:0/2734618112 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0330104480 0x7f0330079a70 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0320030380 tx=0x7f0320032d00 comp rx=0 tx=0).stop 2024-09-13T18:13:18.387 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:17.268+0000 7f0334b4a640 1 -- 172.21.15.114:0/2734618112 shutdown_connections 2024-09-13T18:13:18.387 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:17.268+0000 7f0334b4a640 1 --2- 172.21.15.114:0/2734618112 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7f0318038620 0x7f031803aae0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:18.387 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:17.268+0000 7f0334b4a640 1 --2- 172.21.15.114:0/2734618112 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0330104480 0x7f0330079a70 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:18.387 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:17.268+0000 7f0334b4a640 1 -- 172.21.15.114:0/2734618112 >> 172.21.15.114:0/2734618112 conn(0x7f03300ffc30 msgr2=0x7f03301007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:18.387 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:17.268+0000 7f0334b4a640 1 -- 172.21.15.114:0/2734618112 shutdown_connections 2024-09-13T18:13:18.387 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:17.268+0000 7f0334b4a640 1 -- 172.21.15.114:0/2734618112 wait complete. 2024-09-13T18:13:19.497 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:18.760+0000 7fa8d8689640 1 Processor -- start 2024-09-13T18:13:19.497 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:18.760+0000 7fa8d8689640 1 -- start start 2024-09-13T18:13:19.498 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:18.761+0000 7fa8d8689640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa8d0105540 0x7fa8d0105940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:19.498 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:18.761+0000 7fa8d8689640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d0105f10 con 0x7fa8d0105540 2024-09-13T18:13:19.498 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:18.761+0000 7fa8d6425640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa8d0105540 0x7fa8d0105940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:19.498 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:18.761+0000 7fa8d6425640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa8d0105540 0x7fa8d0105940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60306/0 (socket says 172.21.15.114:60306) 2024-09-13T18:13:19.498 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:18.761+0000 7fa8d6425640 1 -- 172.21.15.114:0/4136396133 learned_addr learned my addr 172.21.15.114:0/4136396133 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:13:19.498 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:18.761+0000 7fa8d6425640 1 -- 172.21.15.114:0/4136396133 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d0106740 con 0x7fa8d0105540 2024-09-13T18:13:19.498 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:18.762+0000 7fa8d6425640 1 --2- 172.21.15.114:0/4136396133 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa8d0105540 0x7fa8d0105940 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc009820 tx=0x7fa8cc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9252055a44753d4b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:19.498 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:18.762+0000 7fa8d5423640 1 -- 172.21.15.114:0/4136396133 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa8cc02ae70 con 0x7fa8d0105540 2024-09-13T18:13:19.498 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:18.762+0000 7fa8d5423640 1 -- 172.21.15.114:0/4136396133 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa8cc0328d0 con 0x7fa8d0105540 2024-09-13T18:13:19.498 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:18.762+0000 7fa8d5423640 1 -- 172.21.15.114:0/4136396133 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa8cc0307a0 con 0x7fa8d0105540 2024-09-13T18:13:19.498 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:18.763+0000 7fa8d8689640 1 -- 172.21.15.114:0/4136396133 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa8d0105540 msgr2=0x7fa8d0105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:19.499 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:18.763+0000 7fa8d8689640 1 --2- 172.21.15.114:0/4136396133 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa8d0105540 0x7fa8d0105940 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc009820 tx=0x7fa8cc02a270 comp rx=0 tx=0).stop 2024-09-13T18:13:19.499 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:18.763+0000 7fa8d8689640 1 -- 172.21.15.114:0/4136396133 shutdown_connections 2024-09-13T18:13:19.499 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:18.763+0000 7fa8d8689640 1 --2- 172.21.15.114:0/4136396133 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa8d0105540 0x7fa8d0105940 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:19.499 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:18.763+0000 7fa8d8689640 1 -- 172.21.15.114:0/4136396133 >> 172.21.15.114:0/4136396133 conn(0x7fa8d0100d30 msgr2=0x7fa8d0103150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:19.499 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:18.763+0000 7fa8d8689640 1 -- 172.21.15.114:0/4136396133 shutdown_connections 2024-09-13T18:13:19.499 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:18.763+0000 7fa8d8689640 1 -- 172.21.15.114:0/4136396133 wait complete. 2024-09-13T18:13:19.499 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:18.764+0000 7fa8d8689640 1 Processor -- start 2024-09-13T18:13:19.499 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:18.764+0000 7fa8d8689640 1 -- start start 2024-09-13T18:13:19.499 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:18.764+0000 7fa8d8689640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa8d0105540 0x7fa8d0199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:19.499 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:18.764+0000 7fa8d8689640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d0199780 con 0x7fa8d0105540 2024-09-13T18:13:19.499 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:18.764+0000 7fa8d6425640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa8d0105540 0x7fa8d0199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:19.499 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:18.764+0000 7fa8d6425640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa8d0105540 0x7fa8d0199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60308/0 (socket says 172.21.15.114:60308) 2024-09-13T18:13:19.499 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:18.764+0000 7fa8d6425640 1 -- 172.21.15.114:0/1632839373 learned_addr learned my addr 172.21.15.114:0/1632839373 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:13:19.500 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:18.765+0000 7fa8d6425640 1 -- 172.21.15.114:0/1632839373 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8cc0094d0 con 0x7fa8d0105540 2024-09-13T18:13:19.500 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:18.765+0000 7fa8d6425640 1 --2- 172.21.15.114:0/1632839373 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa8d0105540 0x7fa8d0199240 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc0304f0 tx=0x7fa8cc030cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:19.500 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:18.765+0000 7fa8c77fe640 1 -- 172.21.15.114:0/1632839373 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa8cc032c50 con 0x7fa8d0105540 2024-09-13T18:13:19.500 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:18.765+0000 7fa8c77fe640 1 -- 172.21.15.114:0/1632839373 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa8cc032db0 con 0x7fa8d0105540 2024-09-13T18:13:19.500 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:18.765+0000 7fa8d8689640 1 -- 172.21.15.114:0/1632839373 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8d0199980 con 0x7fa8d0105540 2024-09-13T18:13:19.500 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:18.765+0000 7fa8c77fe640 1 -- 172.21.15.114:0/1632839373 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa8cc03a5b0 con 0x7fa8d0105540 2024-09-13T18:13:19.500 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:18.765+0000 7fa8d8689640 1 -- 172.21.15.114:0/1632839373 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8d0199e20 con 0x7fa8d0105540 2024-09-13T18:13:19.500 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:18.766+0000 7fa8c77fe640 1 -- 172.21.15.114:0/1632839373 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7fa8cc039040 con 0x7fa8d0105540 2024-09-13T18:13:19.500 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:18.766+0000 7fa8d8689640 1 -- 172.21.15.114:0/1632839373 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8d0109970 con 0x7fa8d0105540 2024-09-13T18:13:19.500 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:18.766+0000 7fa8c77fe640 1 --2- 172.21.15.114:0/1632839373 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fa8bc038620 0x7fa8bc03aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:19.500 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:18.766+0000 7fa8c77fe640 1 -- 172.21.15.114:0/1632839373 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa8cc06b940 con 0x7fa8d0105540 2024-09-13T18:13:19.501 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:18.767+0000 7fa8d5c24640 1 --2- 172.21.15.114:0/1632839373 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fa8bc038620 0x7fa8bc03aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:19.501 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:18.767+0000 7fa8d5c24640 1 --2- 172.21.15.114:0/1632839373 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fa8bc038620 0x7fa8bc03aae0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa8c00098c0 tx=0x7fa8c0006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:19.501 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:18.775+0000 7fa8c77fe640 1 -- 172.21.15.114:0/1632839373 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fa8cc04b6c0 con 0x7fa8d0105540 2024-09-13T18:13:19.501 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:18.943+0000 7fa8d8689640 1 -- 172.21.15.114:0/1632839373 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7fa8d0109bb0 con 0x7fa8d0105540 2024-09-13T18:13:19.501 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:18.947+0000 7fa8c77fe640 1 -- 172.21.15.114:0/1632839373 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7fa8cc037030 con 0x7fa8d0105540 2024-09-13T18:13:19.501 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:18.949+0000 7fa8d8689640 1 -- 172.21.15.114:0/1632839373 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fa8bc038620 msgr2=0x7fa8bc03aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:19.501 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:18.949+0000 7fa8d8689640 1 --2- 172.21.15.114:0/1632839373 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fa8bc038620 0x7fa8bc03aae0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa8c00098c0 tx=0x7fa8c0006db0 comp rx=0 tx=0).stop 2024-09-13T18:13:19.501 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:18.949+0000 7fa8d8689640 1 -- 172.21.15.114:0/1632839373 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa8d0105540 msgr2=0x7fa8d0199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:19.501 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:18.949+0000 7fa8d8689640 1 --2- 172.21.15.114:0/1632839373 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa8d0105540 0x7fa8d0199240 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc0304f0 tx=0x7fa8cc030cc0 comp rx=0 tx=0).stop 2024-09-13T18:13:19.501 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:18.949+0000 7fa8d8689640 1 -- 172.21.15.114:0/1632839373 shutdown_connections 2024-09-13T18:13:19.501 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:18.949+0000 7fa8d8689640 1 --2- 172.21.15.114:0/1632839373 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fa8bc038620 0x7fa8bc03aae0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:19.501 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:18.949+0000 7fa8d8689640 1 --2- 172.21.15.114:0/1632839373 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa8d0105540 0x7fa8d0199240 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:19.502 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:18.949+0000 7fa8d8689640 1 -- 172.21.15.114:0/1632839373 >> 172.21.15.114:0/1632839373 conn(0x7fa8d0100d30 msgr2=0x7fa8d01018e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:19.502 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:18.949+0000 7fa8d8689640 1 -- 172.21.15.114:0/1632839373 shutdown_connections 2024-09-13T18:13:19.502 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:18.949+0000 7fa8d8689640 1 -- 172.21.15.114:0/1632839373 wait complete. 2024-09-13T18:13:19.502 INFO:teuthology.orchestra.run.smithi114.stdout:Enabling the dashboard module... 2024-09-13T18:13:20.200 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:19 smithi114 ceph-mon[30897]: from='mgr.14116 172.21.15.114:0/3845812472' entity='mgr.x' 2024-09-13T18:13:20.200 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:19 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/1632839373' entity='client.admin' 2024-09-13T18:13:21.399 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:21 smithi114 ceph-mon[30897]: from='mgr.14116 172.21.15.114:0/3845812472' entity='mgr.x' 2024-09-13T18:13:21.399 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:21 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/3917133571' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-13T18:13:21.399 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:21 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: ignoring --setuser ceph since I am not root 2024-09-13T18:13:21.400 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:21 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: ignoring --setgroup ceph since I am not root 2024-09-13T18:13:21.400 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:21 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:13:21.190+0000 7fb1496a9040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T18:13:21.400 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:21 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:13:21.248+0000 7fb1496a9040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T18:13:21.400 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:21 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:13:21.397+0000 7fb1496a9040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T18:13:21.720 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:21 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:13:21.463+0000 7fb1496a9040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T18:13:21.720 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:21 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:13:21.612+0000 7fb1496a9040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T18:13:21.819 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:19.969+0000 7f5f9ab3e640 1 Processor -- start 2024-09-13T18:13:21.820 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:19.969+0000 7f5f9ab3e640 1 -- start start 2024-09-13T18:13:21.820 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:19.969+0000 7f5f9ab3e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f941055d0 0x7f5f941059d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:21.820 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:19.969+0000 7f5f9ab3e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f94105fa0 con 0x7f5f941055d0 2024-09-13T18:13:21.820 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:19.970+0000 7f5f988da640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f941055d0 0x7f5f941059d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:21.820 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:19.970+0000 7f5f988da640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f941055d0 0x7f5f941059d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60322/0 (socket says 172.21.15.114:60322) 2024-09-13T18:13:21.820 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:19.970+0000 7f5f988da640 1 -- 172.21.15.114:0/574596471 learned_addr learned my addr 172.21.15.114:0/574596471 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:13:21.820 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:19.970+0000 7f5f988da640 1 -- 172.21.15.114:0/574596471 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f941067d0 con 0x7f5f941055d0 2024-09-13T18:13:21.820 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:19.970+0000 7f5f988da640 1 --2- 172.21.15.114:0/574596471 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f941055d0 0x7f5f941059d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5f88009820 tx=0x7f5f8802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=808fd95245cc171e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:21.820 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:19.971+0000 7f5f937fe640 1 -- 172.21.15.114:0/574596471 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5f8802ae70 con 0x7f5f941055d0 2024-09-13T18:13:21.820 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:19.971+0000 7f5f937fe640 1 -- 172.21.15.114:0/574596471 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5f880328d0 con 0x7f5f941055d0 2024-09-13T18:13:21.820 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:19.971+0000 7f5f937fe640 1 -- 172.21.15.114:0/574596471 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5f880307a0 con 0x7f5f941055d0 2024-09-13T18:13:21.821 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:19.971+0000 7f5f9ab3e640 1 -- 172.21.15.114:0/574596471 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f941055d0 msgr2=0x7f5f941059d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:21.821 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:19.971+0000 7f5f9ab3e640 1 --2- 172.21.15.114:0/574596471 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f941055d0 0x7f5f941059d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5f88009820 tx=0x7f5f8802a270 comp rx=0 tx=0).stop 2024-09-13T18:13:21.821 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:19.971+0000 7f5f9ab3e640 1 -- 172.21.15.114:0/574596471 shutdown_connections 2024-09-13T18:13:21.821 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:19.971+0000 7f5f9ab3e640 1 --2- 172.21.15.114:0/574596471 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f941055d0 0x7f5f941059d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:21.821 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:19.971+0000 7f5f9ab3e640 1 -- 172.21.15.114:0/574596471 >> 172.21.15.114:0/574596471 conn(0x7f5f94100de0 msgr2=0x7f5f94103200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:21.821 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:19.971+0000 7f5f9ab3e640 1 -- 172.21.15.114:0/574596471 shutdown_connections 2024-09-13T18:13:21.821 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:19.971+0000 7f5f9ab3e640 1 -- 172.21.15.114:0/574596471 wait complete. 2024-09-13T18:13:21.821 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:19.972+0000 7f5f9ab3e640 1 Processor -- start 2024-09-13T18:13:21.821 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:19.972+0000 7f5f9ab3e640 1 -- start start 2024-09-13T18:13:21.821 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:19.972+0000 7f5f9ab3e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f941055d0 0x7f5f9407bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:21.821 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:19.972+0000 7f5f9ab3e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9407c0d0 con 0x7f5f941055d0 2024-09-13T18:13:21.821 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:19.972+0000 7f5f988da640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f941055d0 0x7f5f9407bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:21.821 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:19.972+0000 7f5f988da640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f941055d0 0x7f5f9407bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60334/0 (socket says 172.21.15.114:60334) 2024-09-13T18:13:21.822 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:19.972+0000 7f5f988da640 1 -- 172.21.15.114:0/3917133571 learned_addr learned my addr 172.21.15.114:0/3917133571 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:13:21.822 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:19.972+0000 7f5f988da640 1 -- 172.21.15.114:0/3917133571 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f880094d0 con 0x7f5f941055d0 2024-09-13T18:13:21.822 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:19.973+0000 7f5f988da640 1 --2- 172.21.15.114:0/3917133571 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f941055d0 0x7f5f9407bb90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5f88030410 tx=0x7f5f88030d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:21.822 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:19.973+0000 7f5f91ffb640 1 -- 172.21.15.114:0/3917133571 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5f88032c50 con 0x7f5f941055d0 2024-09-13T18:13:21.822 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:19.973+0000 7f5f91ffb640 1 -- 172.21.15.114:0/3917133571 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5f88032db0 con 0x7f5f941055d0 2024-09-13T18:13:21.822 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:19.973+0000 7f5f9ab3e640 1 -- 172.21.15.114:0/3917133571 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f9407a2c0 con 0x7f5f941055d0 2024-09-13T18:13:21.822 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:19.973+0000 7f5f91ffb640 1 -- 172.21.15.114:0/3917133571 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5f8803a520 con 0x7f5f941055d0 2024-09-13T18:13:21.822 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:19.973+0000 7f5f9ab3e640 1 -- 172.21.15.114:0/3917133571 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f9407a760 con 0x7f5f941055d0 2024-09-13T18:13:21.822 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:19.974+0000 7f5f91ffb640 1 -- 172.21.15.114:0/3917133571 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f5f88039040 con 0x7f5f941055d0 2024-09-13T18:13:21.822 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:19.974+0000 7f5f91ffb640 1 --2- 172.21.15.114:0/3917133571 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7f5f84038620 0x7f5f8403aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:21.822 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:19.974+0000 7f5f91ffb640 1 -- 172.21.15.114:0/3917133571 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f5f8803b840 con 0x7f5f941055d0 2024-09-13T18:13:21.822 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:19.974+0000 7f5f93fff640 1 --2- 172.21.15.114:0/3917133571 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7f5f84038620 0x7f5f8403aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:21.822 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:19.974+0000 7f5f9ab3e640 1 -- 172.21.15.114:0/3917133571 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f94105a50 con 0x7f5f941055d0 2024-09-13T18:13:21.823 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:19.974+0000 7f5f93fff640 1 --2- 172.21.15.114:0/3917133571 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7f5f84038620 0x7f5f8403aae0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5f800098c0 tx=0x7f5f80006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:21.823 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:19.979+0000 7f5f91ffb640 1 -- 172.21.15.114:0/3917133571 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f5f88037070 con 0x7f5f941055d0 2024-09-13T18:13:21.823 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:20.197+0000 7f5f9ab3e640 1 -- 172.21.15.114:0/3917133571 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f5f94109a00 con 0x7f5f941055d0 2024-09-13T18:13:21.823 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:21.045+0000 7f5f91ffb640 1 -- 172.21.15.114:0/3917133571 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mgrmap(e 9) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f5f8803ba50 con 0x7f5f941055d0 2024-09-13T18:13:21.823 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:21.046+0000 7f5f91ffb640 1 -- 172.21.15.114:0/3917133571 <== mon.0 v2:172.21.15.114:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f5f8803add0 con 0x7f5f941055d0 2024-09-13T18:13:21.823 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:21.048+0000 7f5f9ab3e640 1 -- 172.21.15.114:0/3917133571 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7f5f84038620 msgr2=0x7f5f8403aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:21.823 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:21.048+0000 7f5f9ab3e640 1 --2- 172.21.15.114:0/3917133571 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7f5f84038620 0x7f5f8403aae0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5f800098c0 tx=0x7f5f80006db0 comp rx=0 tx=0).stop 2024-09-13T18:13:21.823 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:21.048+0000 7f5f9ab3e640 1 -- 172.21.15.114:0/3917133571 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f941055d0 msgr2=0x7f5f9407bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:21.823 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:21.048+0000 7f5f9ab3e640 1 --2- 172.21.15.114:0/3917133571 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f941055d0 0x7f5f9407bb90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5f88030410 tx=0x7f5f88030d50 comp rx=0 tx=0).stop 2024-09-13T18:13:21.823 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:21.048+0000 7f5f9ab3e640 1 -- 172.21.15.114:0/3917133571 shutdown_connections 2024-09-13T18:13:21.823 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:21.048+0000 7f5f9ab3e640 1 --2- 172.21.15.114:0/3917133571 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7f5f84038620 0x7f5f8403aae0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:21.823 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:21.048+0000 7f5f9ab3e640 1 --2- 172.21.15.114:0/3917133571 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f941055d0 0x7f5f9407bb90 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:21.823 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:21.048+0000 7f5f9ab3e640 1 -- 172.21.15.114:0/3917133571 >> 172.21.15.114:0/3917133571 conn(0x7f5f94100de0 msgr2=0x7f5f94101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:21.823 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:21.049+0000 7f5f9ab3e640 1 -- 172.21.15.114:0/3917133571 shutdown_connections 2024-09-13T18:13:21.824 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:21.049+0000 7f5f9ab3e640 1 -- 172.21.15.114:0/3917133571 wait complete. 2024-09-13T18:13:22.047 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:21 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:13:21.754+0000 7fb1496a9040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T18:13:22.324 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:22 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/3917133571' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-13T18:13:22.324 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:22 smithi114 ceph-mon[30897]: mgrmap e9: x(active, since 17s) 2024-09-13T18:13:22.324 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:22 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:13:22.070+0000 7fb1496a9040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T18:13:22.324 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:22 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:13:22.132+0000 7fb1496a9040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T18:13:22.324 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:22 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:13:22.321+0000 7fb1496a9040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T18:13:23.026 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout { 2024-09-13T18:13:23.026 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-13T18:13:23.027 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-13T18:13:23.027 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-13T18:13:23.027 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-13T18:13:23.027 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout } 2024-09-13T18:13:23.027 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:22.240+0000 7fa94eaf9640 1 Processor -- start 2024-09-13T18:13:23.027 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:22.240+0000 7fa94eaf9640 1 -- start start 2024-09-13T18:13:23.027 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:22.240+0000 7fa94eaf9640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa948104480 0x7fa948104880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:23.027 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:22.240+0000 7fa94eaf9640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa948104e50 con 0x7fa948104480 2024-09-13T18:13:23.027 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:22.240+0000 7fa94c895640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa948104480 0x7fa948104880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:23.027 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:22.240+0000 7fa94c895640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa948104480 0x7fa948104880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60364/0 (socket says 172.21.15.114:60364) 2024-09-13T18:13:23.027 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:22.240+0000 7fa94c895640 1 -- 172.21.15.114:0/1895109512 learned_addr learned my addr 172.21.15.114:0/1895109512 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:13:23.027 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:22.241+0000 7fa94c895640 1 -- 172.21.15.114:0/1895109512 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa948105680 con 0x7fa948104480 2024-09-13T18:13:23.027 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:22.241+0000 7fa94c895640 1 --2- 172.21.15.114:0/1895109512 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa948104480 0x7fa948104880 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa93c009820 tx=0x7fa93c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71a59511152686bb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:23.028 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:22.241+0000 7fa9477fe640 1 -- 172.21.15.114:0/1895109512 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa93c02ae70 con 0x7fa948104480 2024-09-13T18:13:23.028 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:22.241+0000 7fa9477fe640 1 -- 172.21.15.114:0/1895109512 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa93c0328d0 con 0x7fa948104480 2024-09-13T18:13:23.028 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:22.241+0000 7fa9477fe640 1 -- 172.21.15.114:0/1895109512 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa93c0307a0 con 0x7fa948104480 2024-09-13T18:13:23.028 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:22.242+0000 7fa94eaf9640 1 -- 172.21.15.114:0/1895109512 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa948104480 msgr2=0x7fa948104880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:23.028 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:22.242+0000 7fa94eaf9640 1 --2- 172.21.15.114:0/1895109512 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa948104480 0x7fa948104880 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa93c009820 tx=0x7fa93c02a270 comp rx=0 tx=0).stop 2024-09-13T18:13:23.028 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:22.242+0000 7fa94eaf9640 1 -- 172.21.15.114:0/1895109512 shutdown_connections 2024-09-13T18:13:23.028 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:22.242+0000 7fa94eaf9640 1 --2- 172.21.15.114:0/1895109512 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa948104480 0x7fa948104880 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:23.028 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:22.242+0000 7fa94eaf9640 1 -- 172.21.15.114:0/1895109512 >> 172.21.15.114:0/1895109512 conn(0x7fa9480ffc30 msgr2=0x7fa948102050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:23.028 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:22.242+0000 7fa94eaf9640 1 -- 172.21.15.114:0/1895109512 shutdown_connections 2024-09-13T18:13:23.028 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:22.242+0000 7fa94eaf9640 1 -- 172.21.15.114:0/1895109512 wait complete. 2024-09-13T18:13:23.028 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:22.243+0000 7fa94eaf9640 1 Processor -- start 2024-09-13T18:13:23.028 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:22.243+0000 7fa94eaf9640 1 -- start start 2024-09-13T18:13:23.028 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:22.243+0000 7fa94eaf9640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa948104480 0x7fa94818f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:23.029 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:22.243+0000 7fa94eaf9640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94818fb30 con 0x7fa948104480 2024-09-13T18:13:23.029 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:22.243+0000 7fa94c895640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa948104480 0x7fa94818f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:23.029 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:22.243+0000 7fa94c895640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa948104480 0x7fa94818f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60372/0 (socket says 172.21.15.114:60372) 2024-09-13T18:13:23.029 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:22.243+0000 7fa94c895640 1 -- 172.21.15.114:0/1784897526 learned_addr learned my addr 172.21.15.114:0/1784897526 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:13:23.029 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:22.243+0000 7fa94c895640 1 -- 172.21.15.114:0/1784897526 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa93c0094d0 con 0x7fa948104480 2024-09-13T18:13:23.029 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:22.243+0000 7fa94c895640 1 --2- 172.21.15.114:0/1784897526 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa948104480 0x7fa94818f5f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa93c0304f0 tx=0x7fa93c030cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:23.029 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:22.243+0000 7fa945ffb640 1 -- 172.21.15.114:0/1784897526 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa93c032c50 con 0x7fa948104480 2024-09-13T18:13:23.029 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:22.244+0000 7fa945ffb640 1 -- 172.21.15.114:0/1784897526 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa93c032db0 con 0x7fa948104480 2024-09-13T18:13:23.029 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:22.244+0000 7fa94eaf9640 1 -- 172.21.15.114:0/1784897526 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa94818dd20 con 0x7fa948104480 2024-09-13T18:13:23.029 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:22.244+0000 7fa945ffb640 1 -- 172.21.15.114:0/1784897526 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa93c03a5b0 con 0x7fa948104480 2024-09-13T18:13:23.029 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:22.244+0000 7fa94eaf9640 1 -- 172.21.15.114:0/1784897526 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa94818e1c0 con 0x7fa948104480 2024-09-13T18:13:23.029 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:22.244+0000 7fa945ffb640 1 -- 172.21.15.114:0/1784897526 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 9) v1 ==== 45554+0+0 (secure 0 0 0) 0x7fa93c039040 con 0x7fa948104480 2024-09-13T18:13:23.029 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:22.245+0000 7fa945ffb640 1 --2- 172.21.15.114:0/1784897526 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fa9380386c0 0x7fa93803ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:23.030 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:22.245+0000 7fa945ffb640 1 -- 172.21.15.114:0/1784897526 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa93c06c790 con 0x7fa948104480 2024-09-13T18:13:23.030 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:22.245+0000 7fa947fff640 1 -- 172.21.15.114:0/1784897526 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fa9380386c0 msgr2=0x7fa93803ab80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.114:6800/4281866858 2024-09-13T18:13:23.030 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:22.245+0000 7fa947fff640 1 --2- 172.21.15.114:0/1784897526 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fa9380386c0 0x7fa93803ab80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:13:23.030 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:22.245+0000 7fa94eaf9640 1 -- 172.21.15.114:0/1784897526 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa948104900 con 0x7fa948104480 2024-09-13T18:13:23.030 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:22.250+0000 7fa945ffb640 1 -- 172.21.15.114:0/1784897526 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fa93c037070 con 0x7fa948104480 2024-09-13T18:13:23.030 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:22.446+0000 7fa947fff640 1 -- 172.21.15.114:0/1784897526 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fa9380386c0 msgr2=0x7fa93803ab80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.114:6800/4281866858 2024-09-13T18:13:23.030 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:22.446+0000 7fa947fff640 1 --2- 172.21.15.114:0/1784897526 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fa9380386c0 0x7fa93803ab80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:13:23.030 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:22.462+0000 7fa94eaf9640 1 -- 172.21.15.114:0/1784897526 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fa948192fa0 con 0x7fa948104480 2024-09-13T18:13:23.030 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:22.463+0000 7fa945ffb640 1 -- 172.21.15.114:0/1784897526 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7fa93c03f030 con 0x7fa948104480 2024-09-13T18:13:23.030 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:22.465+0000 7fa94eaf9640 1 -- 172.21.15.114:0/1784897526 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fa9380386c0 msgr2=0x7fa93803ab80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:13:23.030 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:22.465+0000 7fa94eaf9640 1 --2- 172.21.15.114:0/1784897526 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fa9380386c0 0x7fa93803ab80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:23.030 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:22.465+0000 7fa94eaf9640 1 -- 172.21.15.114:0/1784897526 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa948104480 msgr2=0x7fa94818f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:23.030 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:22.465+0000 7fa94eaf9640 1 --2- 172.21.15.114:0/1784897526 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa948104480 0x7fa94818f5f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa93c0304f0 tx=0x7fa93c030cc0 comp rx=0 tx=0).stop 2024-09-13T18:13:23.030 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:22.465+0000 7fa94eaf9640 1 -- 172.21.15.114:0/1784897526 shutdown_connections 2024-09-13T18:13:23.031 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:22.465+0000 7fa94eaf9640 1 --2- 172.21.15.114:0/1784897526 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fa9380386c0 0x7fa93803ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:23.031 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:22.465+0000 7fa94eaf9640 1 --2- 172.21.15.114:0/1784897526 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa948104480 0x7fa94818f5f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:23.031 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:22.465+0000 7fa94eaf9640 1 -- 172.21.15.114:0/1784897526 >> 172.21.15.114:0/1784897526 conn(0x7fa9480ffc30 msgr2=0x7fa948100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:23.031 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:22.465+0000 7fa94eaf9640 1 -- 172.21.15.114:0/1784897526 shutdown_connections 2024-09-13T18:13:23.031 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:22.465+0000 7fa94eaf9640 1 -- 172.21.15.114:0/1784897526 wait complete. 2024-09-13T18:13:23.031 INFO:teuthology.orchestra.run.smithi114.stdout:Waiting for the mgr to restart... 2024-09-13T18:13:23.031 INFO:teuthology.orchestra.run.smithi114.stdout:Waiting for mgr epoch 9... 2024-09-13T18:13:23.298 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:23 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/1784897526' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-13T18:13:23.298 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:23 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:13:23.218+0000 7fb1496a9040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T18:13:23.298 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:23 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:13:23.284+0000 7fb1496a9040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T18:13:24.253 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:23 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:13:23.997+0000 7fb1496a9040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-13T18:13:24.253 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:24 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:13:24.111+0000 7fb1496a9040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T18:13:24.523 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:24 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:13:24.251+0000 7fb1496a9040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T18:13:24.523 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:24 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:13:24.311+0000 7fb1496a9040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T18:13:24.523 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:24 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:13:24.455+0000 7fb1496a9040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-13T18:13:24.523 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:24 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:13:24.521+0000 7fb1496a9040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T18:13:24.797 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:24 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:13:24.579+0000 7fb1496a9040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T18:13:24.798 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:24 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:13:24.732+0000 7fb1496a9040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T18:13:24.798 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:24 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:13:24.795+0000 7fb1496a9040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T18:13:25.059 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:24 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:13:24.859+0000 7fb1496a9040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T18:13:25.059 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:24 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:13:24.941+0000 7fb1496a9040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T18:13:25.059 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:24 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:13:24.999+0000 7fb1496a9040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T18:13:25.060 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:25 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:13:25.057+0000 7fb1496a9040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T18:13:26.220 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:25 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:13:25.821+0000 7fb1496a9040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T18:13:26.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:25 smithi114 ceph-mon[30897]: Active manager daemon x restarted 2024-09-13T18:13:26.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:25 smithi114 ceph-mon[30897]: Activating manager daemon x 2024-09-13T18:13:26.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:25 smithi114 ceph-mon[30897]: osdmap e3: 0 total, 0 up, 0 in 2024-09-13T18:13:26.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:25 smithi114 ceph-mon[30897]: mgrmap e10: x(active, starting, since 0.00322557s) 2024-09-13T18:13:26.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:25 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T18:13:26.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:25 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-13T18:13:26.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:25 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-13T18:13:26.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:25 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-13T18:13:26.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:25 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-13T18:13:26.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:25 smithi114 ceph-mon[30897]: Manager daemon x is now available 2024-09-13T18:13:26.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:25 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:13:26.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:25 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-13T18:13:26.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:25 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-13T18:13:27.605 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:27 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:13:27.472+0000 7fb0b533e640 -1 log_channel(cephadm) log [ERR] : [13/Sep/2024:18:13:27] ENGINE Error in HTTPServer.serve 2024-09-13T18:13:27.605 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:27 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: Traceback (most recent call last): 2024-09-13T18:13:27.605 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:27 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-13T18:13:27.605 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:27 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: self._connections.run(self.expiration_interval) 2024-09-13T18:13:27.605 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:27 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-13T18:13:27.605 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:27 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: self._run(expiration_interval) 2024-09-13T18:13:27.605 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:27 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-13T18:13:27.606 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:27 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: new_conn = self._from_server_socket(self.server.socket) 2024-09-13T18:13:27.606 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:27 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-13T18:13:27.606 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:27 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-13T18:13:27.606 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:27 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-13T18:13:27.606 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:27 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: s = self.context.wrap_socket( 2024-09-13T18:13:27.606 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:27 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-13T18:13:27.606 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:27 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: return self.sslsocket_class._create( 2024-09-13T18:13:27.606 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:27 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-13T18:13:27.606 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:27 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: self.do_handshake() 2024-09-13T18:13:27.606 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:27 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-13T18:13:27.606 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:27 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: self._sslobj.do_handshake() 2024-09-13T18:13:27.606 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:27 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-13T18:13:27.606 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:13:27 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:13:27.614 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout { 2024-09-13T18:13:27.614 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-13T18:13:27.614 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-13T18:13:27.614 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout } 2024-09-13T18:13:27.614 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:23.481+0000 7fb5714b0640 1 Processor -- start 2024-09-13T18:13:27.614 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:23.481+0000 7fb5714b0640 1 -- start start 2024-09-13T18:13:27.614 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:23.481+0000 7fb5714b0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb56c1055a0 0x7fb56c1059a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:27.614 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:23.481+0000 7fb5714b0640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb56c105f70 con 0x7fb56c1055a0 2024-09-13T18:13:27.615 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:23.481+0000 7fb56affd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb56c1055a0 0x7fb56c1059a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:27.615 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:23.481+0000 7fb56affd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb56c1055a0 0x7fb56c1059a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55114/0 (socket says 172.21.15.114:55114) 2024-09-13T18:13:27.615 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:23.481+0000 7fb56affd640 1 -- 172.21.15.114:0/4147872399 learned_addr learned my addr 172.21.15.114:0/4147872399 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:13:27.615 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:23.481+0000 7fb56affd640 1 -- 172.21.15.114:0/4147872399 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb56c1067a0 con 0x7fb56c1055a0 2024-09-13T18:13:27.615 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:23.482+0000 7fb56affd640 1 --2- 172.21.15.114:0/4147872399 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb56c1055a0 0x7fb56c1059a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb55c009820 tx=0x7fb55c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e7a1b7b2b5e403e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:27.615 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:23.482+0000 7fb569ffb640 1 -- 172.21.15.114:0/4147872399 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb55c02ae70 con 0x7fb56c1055a0 2024-09-13T18:13:27.615 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:23.482+0000 7fb569ffb640 1 -- 172.21.15.114:0/4147872399 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb55c0328d0 con 0x7fb56c1055a0 2024-09-13T18:13:27.615 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:23.482+0000 7fb569ffb640 1 -- 172.21.15.114:0/4147872399 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb55c0307a0 con 0x7fb56c1055a0 2024-09-13T18:13:27.615 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:23.483+0000 7fb5714b0640 1 -- 172.21.15.114:0/4147872399 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb56c1055a0 msgr2=0x7fb56c1059a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:27.615 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:23.483+0000 7fb5714b0640 1 --2- 172.21.15.114:0/4147872399 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb56c1055a0 0x7fb56c1059a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb55c009820 tx=0x7fb55c02a270 comp rx=0 tx=0).stop 2024-09-13T18:13:27.615 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:23.483+0000 7fb5714b0640 1 -- 172.21.15.114:0/4147872399 shutdown_connections 2024-09-13T18:13:27.615 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:23.483+0000 7fb5714b0640 1 --2- 172.21.15.114:0/4147872399 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb56c1055a0 0x7fb56c1059a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:27.615 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:23.483+0000 7fb5714b0640 1 -- 172.21.15.114:0/4147872399 >> 172.21.15.114:0/4147872399 conn(0x7fb56c100db0 msgr2=0x7fb56c1031d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:27.615 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:23.483+0000 7fb5714b0640 1 -- 172.21.15.114:0/4147872399 shutdown_connections 2024-09-13T18:13:27.616 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:23.483+0000 7fb5714b0640 1 -- 172.21.15.114:0/4147872399 wait complete. 2024-09-13T18:13:27.616 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:23.484+0000 7fb5714b0640 1 Processor -- start 2024-09-13T18:13:27.616 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:23.484+0000 7fb5714b0640 1 -- start start 2024-09-13T18:13:27.616 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:23.484+0000 7fb5714b0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb56c1055a0 0x7fb56c1992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:27.616 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:23.484+0000 7fb5714b0640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb56c199810 con 0x7fb56c1055a0 2024-09-13T18:13:27.616 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:23.484+0000 7fb56affd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb56c1055a0 0x7fb56c1992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:27.616 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:23.485+0000 7fb56affd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb56c1055a0 0x7fb56c1992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55130/0 (socket says 172.21.15.114:55130) 2024-09-13T18:13:27.616 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:23.485+0000 7fb56affd640 1 -- 172.21.15.114:0/2776647591 learned_addr learned my addr 172.21.15.114:0/2776647591 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:13:27.616 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:23.485+0000 7fb56affd640 1 -- 172.21.15.114:0/2776647591 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb55c0094d0 con 0x7fb56c1055a0 2024-09-13T18:13:27.616 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:23.485+0000 7fb56affd640 1 --2- 172.21.15.114:0/2776647591 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb56c1055a0 0x7fb56c1992d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb55c030410 tx=0x7fb55c030d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:27.616 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:23.485+0000 7fb553fff640 1 -- 172.21.15.114:0/2776647591 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb55c032c50 con 0x7fb56c1055a0 2024-09-13T18:13:27.616 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:23.485+0000 7fb553fff640 1 -- 172.21.15.114:0/2776647591 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb55c032db0 con 0x7fb56c1055a0 2024-09-13T18:13:27.616 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:23.485+0000 7fb5714b0640 1 -- 172.21.15.114:0/2776647591 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb56c199a10 con 0x7fb56c1055a0 2024-09-13T18:13:27.616 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:23.485+0000 7fb553fff640 1 -- 172.21.15.114:0/2776647591 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb55c03a520 con 0x7fb56c1055a0 2024-09-13T18:13:27.616 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:23.485+0000 7fb5714b0640 1 -- 172.21.15.114:0/2776647591 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb56c199eb0 con 0x7fb56c1055a0 2024-09-13T18:13:27.617 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:23.486+0000 7fb553fff640 1 -- 172.21.15.114:0/2776647591 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 9) v1 ==== 45554+0+0 (secure 0 0 0) 0x7fb55c039040 con 0x7fb56c1055a0 2024-09-13T18:13:27.617 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:23.486+0000 7fb553fff640 1 --2- 172.21.15.114:0/2776647591 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fb554038670 0x7fb55403ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:27.617 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:23.486+0000 7fb553fff640 1 -- 172.21.15.114:0/2776647591 --> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb55403b240 con 0x7fb554038670 2024-09-13T18:13:27.617 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:23.486+0000 7fb553fff640 1 -- 172.21.15.114:0/2776647591 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb55c03b840 con 0x7fb56c1055a0 2024-09-13T18:13:27.617 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:23.486+0000 7fb56a7fc640 1 -- 172.21.15.114:0/2776647591 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fb554038670 msgr2=0x7fb55403ab30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.114:6800/4281866858 2024-09-13T18:13:27.617 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:23.486+0000 7fb56a7fc640 1 --2- 172.21.15.114:0/2776647591 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fb554038670 0x7fb55403ab30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:13:27.617 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:23.687+0000 7fb56a7fc640 1 -- 172.21.15.114:0/2776647591 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fb554038670 msgr2=0x7fb55403ab30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.114:6800/4281866858 2024-09-13T18:13:27.617 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:23.687+0000 7fb56a7fc640 1 --2- 172.21.15.114:0/2776647591 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fb554038670 0x7fb55403ab30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:13:27.617 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:24.088+0000 7fb56a7fc640 1 -- 172.21.15.114:0/2776647591 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fb554038670 msgr2=0x7fb55403ab30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.114:6800/4281866858 2024-09-13T18:13:27.617 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:24.088+0000 7fb56a7fc640 1 --2- 172.21.15.114:0/2776647591 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fb554038670 0x7fb55403ab30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:13:27.617 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:24.889+0000 7fb56a7fc640 1 -- 172.21.15.114:0/2776647591 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fb554038670 msgr2=0x7fb55403ab30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.114:6800/4281866858 2024-09-13T18:13:27.617 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:24.889+0000 7fb56a7fc640 1 --2- 172.21.15.114:0/2776647591 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fb554038670 0x7fb55403ab30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:13:27.617 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:25.824+0000 7fb553fff640 1 -- 172.21.15.114:0/2776647591 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mgrmap(e 10) v1 ==== 45321+0+0 (secure 0 0 0) 0x7fb55c06b560 con 0x7fb56c1055a0 2024-09-13T18:13:27.617 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:25.824+0000 7fb553fff640 1 -- 172.21.15.114:0/2776647591 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fb554038670 msgr2=0x7fb55403ab30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:13:27.618 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:25.824+0000 7fb553fff640 1 --2- 172.21.15.114:0/2776647591 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fb554038670 0x7fb55403ab30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:27.618 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:26.827+0000 7fb553fff640 1 -- 172.21.15.114:0/2776647591 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mgrmap(e 11) v1 ==== 45448+0+0 (secure 0 0 0) 0x7fb55c02fd90 con 0x7fb56c1055a0 2024-09-13T18:13:27.618 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:26.827+0000 7fb553fff640 1 --2- 172.21.15.114:0/2776647591 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb55403bfb0 0x7fb55403e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:27.618 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:26.827+0000 7fb553fff640 1 -- 172.21.15.114:0/2776647591 --> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb55403b240 con 0x7fb55403bfb0 2024-09-13T18:13:27.618 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:26.827+0000 7fb56a7fc640 1 --2- 172.21.15.114:0/2776647591 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb55403bfb0 0x7fb55403e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:27.618 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:26.828+0000 7fb56a7fc640 1 --2- 172.21.15.114:0/2776647591 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb55403bfb0 0x7fb55403e3a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb560003980 tx=0x7fb560006e10 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:27.618 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:26.829+0000 7fb553fff640 1 -- 172.21.15.114:0/2776647591 <== mgr.14148 v2:172.21.15.114:6800/3396185541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7fb55403b240 con 0x7fb55403bfb0 2024-09-13T18:13:27.618 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:26.834+0000 7fb5714b0640 1 -- 172.21.15.114:0/2776647591 --> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fb56c1099d0 con 0x7fb55403bfb0 2024-09-13T18:13:27.618 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:26.835+0000 7fb553fff640 1 -- 172.21.15.114:0/2776647591 <== mgr.14148 v2:172.21.15.114:6800/3396185541 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fb56c1099d0 con 0x7fb55403bfb0 2024-09-13T18:13:27.618 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:26.835+0000 7fb5714b0640 1 -- 172.21.15.114:0/2776647591 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb55403bfb0 msgr2=0x7fb55403e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:27.618 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:26.835+0000 7fb5714b0640 1 --2- 172.21.15.114:0/2776647591 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb55403bfb0 0x7fb55403e3a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb560003980 tx=0x7fb560006e10 comp rx=0 tx=0).stop 2024-09-13T18:13:27.618 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:26.835+0000 7fb5714b0640 1 -- 172.21.15.114:0/2776647591 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb56c1055a0 msgr2=0x7fb56c1992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:27.618 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:26.835+0000 7fb5714b0640 1 --2- 172.21.15.114:0/2776647591 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb56c1055a0 0x7fb56c1992d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb55c030410 tx=0x7fb55c030d50 comp rx=0 tx=0).stop 2024-09-13T18:13:27.618 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:26.835+0000 7fb5714b0640 1 -- 172.21.15.114:0/2776647591 shutdown_connections 2024-09-13T18:13:27.618 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:26.835+0000 7fb5714b0640 1 --2- 172.21.15.114:0/2776647591 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb55403bfb0 0x7fb55403e3a0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:27.619 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:26.835+0000 7fb5714b0640 1 --2- 172.21.15.114:0/2776647591 >> [v2:172.21.15.114:6800/4281866858,v1:172.21.15.114:6801/4281866858] conn(0x7fb554038670 0x7fb55403ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:27.619 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:26.835+0000 7fb5714b0640 1 --2- 172.21.15.114:0/2776647591 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb56c1055a0 0x7fb56c1992d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:27.619 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:26.835+0000 7fb5714b0640 1 -- 172.21.15.114:0/2776647591 >> 172.21.15.114:0/2776647591 conn(0x7fb56c100db0 msgr2=0x7fb56c101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:27.619 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:26.835+0000 7fb5714b0640 1 -- 172.21.15.114:0/2776647591 shutdown_connections 2024-09-13T18:13:27.619 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:26.835+0000 7fb5714b0640 1 -- 172.21.15.114:0/2776647591 wait complete. 2024-09-13T18:13:27.619 INFO:teuthology.orchestra.run.smithi114.stdout:mgr epoch 9 is available 2024-09-13T18:13:27.619 INFO:teuthology.orchestra.run.smithi114.stdout:Generating a dashboard self-signed certificate... 2024-09-13T18:13:27.860 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:27 smithi114 ceph-mon[30897]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-13T18:13:27.860 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:27 smithi114 ceph-mon[30897]: mgrmap e11: x(active, since 1.0073s) 2024-09-13T18:13:27.860 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:27 smithi114 ceph-mon[30897]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-13T18:13:27.861 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:27 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:13:29.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:28 smithi114 ceph-mon[30897]: [13/Sep/2024:18:13:27] ENGINE Bus STARTING 2024-09-13T18:13:29.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:28 smithi114 ceph-mon[30897]: [13/Sep/2024:18:13:27] ENGINE Serving on https://172.21.15.114:7150 2024-09-13T18:13:29.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:28 smithi114 ceph-mon[30897]: [13/Sep/2024:18:13:27] ENGINE Error in HTTPServer.serve 2024-09-13T18:13:29.220 INFO:journalctl@ceph.mon.a.smithi114.stdout: Traceback (most recent call last): 2024-09-13T18:13:29.220 INFO:journalctl@ceph.mon.a.smithi114.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-13T18:13:29.221 INFO:journalctl@ceph.mon.a.smithi114.stdout: self._connections.run(self.expiration_interval) 2024-09-13T18:13:29.221 INFO:journalctl@ceph.mon.a.smithi114.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-13T18:13:29.221 INFO:journalctl@ceph.mon.a.smithi114.stdout: self._run(expiration_interval) 2024-09-13T18:13:29.221 INFO:journalctl@ceph.mon.a.smithi114.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-13T18:13:29.221 INFO:journalctl@ceph.mon.a.smithi114.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-13T18:13:29.221 INFO:journalctl@ceph.mon.a.smithi114.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-13T18:13:29.221 INFO:journalctl@ceph.mon.a.smithi114.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-13T18:13:29.221 INFO:journalctl@ceph.mon.a.smithi114.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-13T18:13:29.221 INFO:journalctl@ceph.mon.a.smithi114.stdout: s = self.context.wrap_socket( 2024-09-13T18:13:29.221 INFO:journalctl@ceph.mon.a.smithi114.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-13T18:13:29.221 INFO:journalctl@ceph.mon.a.smithi114.stdout: return self.sslsocket_class._create( 2024-09-13T18:13:29.221 INFO:journalctl@ceph.mon.a.smithi114.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-13T18:13:29.221 INFO:journalctl@ceph.mon.a.smithi114.stdout: self.do_handshake() 2024-09-13T18:13:29.222 INFO:journalctl@ceph.mon.a.smithi114.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-13T18:13:29.222 INFO:journalctl@ceph.mon.a.smithi114.stdout: self._sslobj.do_handshake() 2024-09-13T18:13:29.222 INFO:journalctl@ceph.mon.a.smithi114.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-13T18:13:29.222 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:28 smithi114 ceph-mon[30897]: [13/Sep/2024:18:13:27] ENGINE Serving on http://172.21.15.114:8765 2024-09-13T18:13:29.222 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:28 smithi114 ceph-mon[30897]: [13/Sep/2024:18:13:27] ENGINE Bus STARTED 2024-09-13T18:13:29.222 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:28 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:13:29.222 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:28 smithi114 ceph-mon[30897]: mgrmap e12: x(active, since 2s) 2024-09-13T18:13:29.222 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:28 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:13:29.222 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:28 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:13:29.352 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-13T18:13:29.358 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:28.061+0000 7f2771642640 1 Processor -- start 2024-09-13T18:13:29.358 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:28.061+0000 7f2771642640 1 -- start start 2024-09-13T18:13:29.358 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:28.061+0000 7f2771642640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f276c104480 0x7f276c104880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:29.358 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:28.061+0000 7f2771642640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f276c104e50 con 0x7f276c104480 2024-09-13T18:13:29.358 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:28.061+0000 7f276affd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f276c104480 0x7f276c104880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:29.358 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:28.061+0000 7f276affd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f276c104480 0x7f276c104880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55228/0 (socket says 172.21.15.114:55228) 2024-09-13T18:13:29.358 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:28.061+0000 7f276affd640 1 -- 172.21.15.114:0/837998918 learned_addr learned my addr 172.21.15.114:0/837998918 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:13:29.358 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:28.062+0000 7f276affd640 1 -- 172.21.15.114:0/837998918 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f276c105680 con 0x7f276c104480 2024-09-13T18:13:29.358 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:28.062+0000 7f276affd640 1 --2- 172.21.15.114:0/837998918 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f276c104480 0x7f276c104880 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f275c009820 tx=0x7f275c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7c37552c38cc07 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:29.358 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:28.062+0000 7f276a7fc640 1 -- 172.21.15.114:0/837998918 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f275c02ae70 con 0x7f276c104480 2024-09-13T18:13:29.358 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:28.063+0000 7f276a7fc640 1 -- 172.21.15.114:0/837998918 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f275c0328d0 con 0x7f276c104480 2024-09-13T18:13:29.358 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:28.063+0000 7f276a7fc640 1 -- 172.21.15.114:0/837998918 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f275c0307a0 con 0x7f276c104480 2024-09-13T18:13:29.359 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:28.063+0000 7f2771642640 1 -- 172.21.15.114:0/837998918 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f276c104480 msgr2=0x7f276c104880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:29.359 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:28.063+0000 7f2771642640 1 --2- 172.21.15.114:0/837998918 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f276c104480 0x7f276c104880 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f275c009820 tx=0x7f275c02a270 comp rx=0 tx=0).stop 2024-09-13T18:13:29.359 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:28.063+0000 7f2771642640 1 -- 172.21.15.114:0/837998918 shutdown_connections 2024-09-13T18:13:29.359 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:28.063+0000 7f2771642640 1 --2- 172.21.15.114:0/837998918 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f276c104480 0x7f276c104880 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:29.359 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:28.063+0000 7f2771642640 1 -- 172.21.15.114:0/837998918 >> 172.21.15.114:0/837998918 conn(0x7f276c0ffc30 msgr2=0x7f276c102050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:29.359 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:28.063+0000 7f2771642640 1 -- 172.21.15.114:0/837998918 shutdown_connections 2024-09-13T18:13:29.359 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:28.063+0000 7f2771642640 1 -- 172.21.15.114:0/837998918 wait complete. 2024-09-13T18:13:29.359 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:28.064+0000 7f2771642640 1 Processor -- start 2024-09-13T18:13:29.359 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:28.064+0000 7f2771642640 1 -- start start 2024-09-13T18:13:29.359 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:28.064+0000 7f2771642640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f276c104480 0x7f276c18f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:29.359 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:28.064+0000 7f2771642640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f276c18fb30 con 0x7f276c104480 2024-09-13T18:13:29.359 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:28.064+0000 7f276affd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f276c104480 0x7f276c18f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:29.360 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:28.064+0000 7f276affd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f276c104480 0x7f276c18f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55238/0 (socket says 172.21.15.114:55238) 2024-09-13T18:13:29.360 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:28.064+0000 7f276affd640 1 -- 172.21.15.114:0/645627015 learned_addr learned my addr 172.21.15.114:0/645627015 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:13:29.360 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:28.065+0000 7f276affd640 1 -- 172.21.15.114:0/645627015 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f275c0094d0 con 0x7f276c104480 2024-09-13T18:13:29.360 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:28.065+0000 7f276affd640 1 --2- 172.21.15.114:0/645627015 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f276c104480 0x7f276c18f5f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f275c030380 tx=0x7f275c032d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:29.360 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:28.065+0000 7f2768ff9640 1 -- 172.21.15.114:0/645627015 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f275c030c70 con 0x7f276c104480 2024-09-13T18:13:29.360 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:28.065+0000 7f2768ff9640 1 -- 172.21.15.114:0/645627015 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f275c030dd0 con 0x7f276c104480 2024-09-13T18:13:29.360 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:28.065+0000 7f2768ff9640 1 -- 172.21.15.114:0/645627015 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f275c03a5b0 con 0x7f276c104480 2024-09-13T18:13:29.360 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:28.065+0000 7f2771642640 1 -- 172.21.15.114:0/645627015 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f276c18dd20 con 0x7f276c104480 2024-09-13T18:13:29.360 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:28.065+0000 7f2771642640 1 -- 172.21.15.114:0/645627015 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f276c18e1c0 con 0x7f276c104480 2024-09-13T18:13:29.360 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:28.066+0000 7f2768ff9640 1 -- 172.21.15.114:0/645627015 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 11) v1 ==== 45448+0+0 (secure 0 0 0) 0x7f275c039040 con 0x7f276c104480 2024-09-13T18:13:29.360 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:28.066+0000 7f2768ff9640 1 --2- 172.21.15.114:0/645627015 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f27580385a0 0x7f275803aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:29.360 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:28.066+0000 7f2768ff9640 1 -- 172.21.15.114:0/645627015 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f275c06b8c0 con 0x7f276c104480 2024-09-13T18:13:29.361 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:28.066+0000 7f2763fff640 1 --2- 172.21.15.114:0/645627015 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f27580385a0 0x7f275803aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:29.361 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:28.066+0000 7f2771642640 1 -- 172.21.15.114:0/645627015 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f276c18ebc0 con 0x7f276c104480 2024-09-13T18:13:29.361 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:28.067+0000 7f2763fff640 1 --2- 172.21.15.114:0/645627015 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f27580385a0 0x7f275803aa60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f27540098c0 tx=0x7f2754006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:29.361 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:28.072+0000 7f2768ff9640 1 -- 172.21.15.114:0/645627015 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f275c03f070 con 0x7f276c104480 2024-09-13T18:13:29.361 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:28.225+0000 7f2768ff9640 1 -- 172.21.15.114:0/645627015 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mgrmap(e 12) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f275c03bac0 con 0x7f276c104480 2024-09-13T18:13:29.361 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:28.254+0000 7f2771642640 1 -- 172.21.15.114:0/645627015 --> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f276c192fa0 con 0x7f27580385a0 2024-09-13T18:13:29.361 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:28.323+0000 7f2768ff9640 1 -- 172.21.15.114:0/645627015 <== mgr.14148 v2:172.21.15.114:6800/3396185541 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f276c192fa0 con 0x7f27580385a0 2024-09-13T18:13:29.361 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:28.325+0000 7f2771642640 1 -- 172.21.15.114:0/645627015 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f27580385a0 msgr2=0x7f275803aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:29.361 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:28.325+0000 7f2771642640 1 --2- 172.21.15.114:0/645627015 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f27580385a0 0x7f275803aa60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f27540098c0 tx=0x7f2754006db0 comp rx=0 tx=0).stop 2024-09-13T18:13:29.361 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:28.325+0000 7f2771642640 1 -- 172.21.15.114:0/645627015 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f276c104480 msgr2=0x7f276c18f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:29.361 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:28.325+0000 7f2771642640 1 --2- 172.21.15.114:0/645627015 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f276c104480 0x7f276c18f5f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f275c030380 tx=0x7f275c032d00 comp rx=0 tx=0).stop 2024-09-13T18:13:29.361 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:28.325+0000 7f2771642640 1 -- 172.21.15.114:0/645627015 shutdown_connections 2024-09-13T18:13:29.361 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:28.325+0000 7f2771642640 1 --2- 172.21.15.114:0/645627015 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f27580385a0 0x7f275803aa60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:29.362 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:28.325+0000 7f2771642640 1 --2- 172.21.15.114:0/645627015 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f276c104480 0x7f276c18f5f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:29.362 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:28.325+0000 7f2771642640 1 -- 172.21.15.114:0/645627015 >> 172.21.15.114:0/645627015 conn(0x7f276c0ffc30 msgr2=0x7f276c100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:29.362 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:28.325+0000 7f2771642640 1 -- 172.21.15.114:0/645627015 shutdown_connections 2024-09-13T18:13:29.362 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:28.325+0000 7f2771642640 1 -- 172.21.15.114:0/645627015 wait complete. 2024-09-13T18:13:29.362 INFO:teuthology.orchestra.run.smithi114.stdout:Creating initial admin user... 2024-09-13T18:13:30.072 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:29 smithi114 ceph-mon[30897]: from='client.14160 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:13:31.167 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$DCO2JgLanLklxwAoq.8GDeLhjVf702cYoLSXL/fuKQdupKhVOOzoS", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726251210, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-13T18:13:31.167 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:30.159+0000 7f6585878640 1 Processor -- start 2024-09-13T18:13:31.167 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:30.159+0000 7f6585878640 1 -- start start 2024-09-13T18:13:31.167 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:30.160+0000 7f6585878640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f65801044a0 0x7f65801048a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:31.167 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:30.160+0000 7f6585878640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6580104e70 con 0x7f65801044a0 2024-09-13T18:13:31.168 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:30.160+0000 7f657effd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f65801044a0 0x7f65801048a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:31.168 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:30.160+0000 7f657effd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f65801044a0 0x7f65801048a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55242/0 (socket says 172.21.15.114:55242) 2024-09-13T18:13:31.168 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:30.160+0000 7f657effd640 1 -- 172.21.15.114:0/3055761969 learned_addr learned my addr 172.21.15.114:0/3055761969 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:13:31.168 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:30.160+0000 7f657effd640 1 -- 172.21.15.114:0/3055761969 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65801056a0 con 0x7f65801044a0 2024-09-13T18:13:31.168 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:30.161+0000 7f657effd640 1 --2- 172.21.15.114:0/3055761969 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f65801044a0 0x7f65801048a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6570009e80 tx=0x7f657002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c96e9aa3531d7b39 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:31.168 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:30.161+0000 7f657dffb640 1 -- 172.21.15.114:0/3055761969 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6570031400 con 0x7f65801044a0 2024-09-13T18:13:31.168 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:30.161+0000 7f657dffb640 1 -- 172.21.15.114:0/3055761969 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f657002d050 con 0x7f65801044a0 2024-09-13T18:13:31.168 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:30.161+0000 7f657dffb640 1 -- 172.21.15.114:0/3055761969 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6570007830 con 0x7f65801044a0 2024-09-13T18:13:31.168 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:30.162+0000 7f6585878640 1 -- 172.21.15.114:0/3055761969 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f65801044a0 msgr2=0x7f65801048a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:31.168 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:30.162+0000 7f6585878640 1 --2- 172.21.15.114:0/3055761969 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f65801044a0 0x7f65801048a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6570009e80 tx=0x7f657002a7a0 comp rx=0 tx=0).stop 2024-09-13T18:13:31.168 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:30.162+0000 7f6585878640 1 -- 172.21.15.114:0/3055761969 shutdown_connections 2024-09-13T18:13:31.169 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:30.162+0000 7f6585878640 1 --2- 172.21.15.114:0/3055761969 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f65801044a0 0x7f65801048a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:31.169 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:30.162+0000 7f6585878640 1 -- 172.21.15.114:0/3055761969 >> 172.21.15.114:0/3055761969 conn(0x7f65800ffcd0 msgr2=0x7f65801020f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:31.169 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:30.162+0000 7f6585878640 1 -- 172.21.15.114:0/3055761969 shutdown_connections 2024-09-13T18:13:31.169 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:30.162+0000 7f6585878640 1 -- 172.21.15.114:0/3055761969 wait complete. 2024-09-13T18:13:31.169 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:30.162+0000 7f6585878640 1 Processor -- start 2024-09-13T18:13:31.169 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:30.163+0000 7f6585878640 1 -- start start 2024-09-13T18:13:31.169 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:30.163+0000 7f6585878640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f65801044a0 0x7f658018f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:31.169 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:30.163+0000 7f6585878640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f658018fb00 con 0x7f65801044a0 2024-09-13T18:13:31.169 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:30.163+0000 7f657effd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f65801044a0 0x7f658018f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:31.169 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:30.163+0000 7f657effd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f65801044a0 0x7f658018f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55248/0 (socket says 172.21.15.114:55248) 2024-09-13T18:13:31.169 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:30.163+0000 7f657effd640 1 -- 172.21.15.114:0/150990168 learned_addr learned my addr 172.21.15.114:0/150990168 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:13:31.169 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:30.163+0000 7f657effd640 1 -- 172.21.15.114:0/150990168 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6570009b30 con 0x7f65801044a0 2024-09-13T18:13:31.170 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:30.163+0000 7f657effd640 1 --2- 172.21.15.114:0/150990168 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f65801044a0 0x7f658018f5c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f657002acc0 tx=0x7f6570031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:31.170 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:30.164+0000 7f6584876640 1 -- 172.21.15.114:0/150990168 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6570031b30 con 0x7f65801044a0 2024-09-13T18:13:31.170 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:30.164+0000 7f6584876640 1 -- 172.21.15.114:0/150990168 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6570031c90 con 0x7f65801044a0 2024-09-13T18:13:31.170 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:30.164+0000 7f6585878640 1 -- 172.21.15.114:0/150990168 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f658018dd40 con 0x7f65801044a0 2024-09-13T18:13:31.170 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:30.164+0000 7f6584876640 1 -- 172.21.15.114:0/150990168 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f657003a500 con 0x7f65801044a0 2024-09-13T18:13:31.170 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:30.164+0000 7f6585878640 1 -- 172.21.15.114:0/150990168 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f658018e1e0 con 0x7f65801044a0 2024-09-13T18:13:31.170 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:30.165+0000 7f6584876640 1 -- 172.21.15.114:0/150990168 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 12) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f6570038050 con 0x7f65801044a0 2024-09-13T18:13:31.170 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:30.165+0000 7f6585878640 1 -- 172.21.15.114:0/150990168 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6580104920 con 0x7f65801044a0 2024-09-13T18:13:31.170 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:30.165+0000 7f6584876640 1 --2- 172.21.15.114:0/150990168 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f65680386c0 0x7f656803ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:31.170 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:30.165+0000 7f6584876640 1 -- 172.21.15.114:0/150990168 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f657006b730 con 0x7f65801044a0 2024-09-13T18:13:31.170 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:30.165+0000 7f657e7fc640 1 --2- 172.21.15.114:0/150990168 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f65680386c0 0x7f656803ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:31.171 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:30.166+0000 7f657e7fc640 1 --2- 172.21.15.114:0/150990168 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f65680386c0 0x7f656803ab80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f65740098c0 tx=0x7f6574006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:31.171 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:30.170+0000 7f6584876640 1 -- 172.21.15.114:0/150990168 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f6570044360 con 0x7f65801044a0 2024-09-13T18:13:31.171 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:30.345+0000 7f6585878640 1 -- 172.21.15.114:0/150990168 --> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f6580061900 con 0x7f65680386c0 2024-09-13T18:13:31.171 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:30.566+0000 7f6584876640 1 -- 172.21.15.114:0/150990168 <== mgr.14148 v2:172.21.15.114:6800/3396185541 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f6580061900 con 0x7f65680386c0 2024-09-13T18:13:31.171 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:30.568+0000 7f6585878640 1 -- 172.21.15.114:0/150990168 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f65680386c0 msgr2=0x7f656803ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:31.171 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:30.568+0000 7f6585878640 1 --2- 172.21.15.114:0/150990168 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f65680386c0 0x7f656803ab80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f65740098c0 tx=0x7f6574006db0 comp rx=0 tx=0).stop 2024-09-13T18:13:31.171 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:30.568+0000 7f6585878640 1 -- 172.21.15.114:0/150990168 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f65801044a0 msgr2=0x7f658018f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:31.171 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:30.568+0000 7f6585878640 1 --2- 172.21.15.114:0/150990168 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f65801044a0 0x7f658018f5c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f657002acc0 tx=0x7f6570031630 comp rx=0 tx=0).stop 2024-09-13T18:13:31.171 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:30.568+0000 7f6585878640 1 -- 172.21.15.114:0/150990168 shutdown_connections 2024-09-13T18:13:31.171 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:30.568+0000 7f6585878640 1 --2- 172.21.15.114:0/150990168 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f65680386c0 0x7f656803ab80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:31.171 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:30.568+0000 7f6585878640 1 --2- 172.21.15.114:0/150990168 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f65801044a0 0x7f658018f5c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:31.172 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:30.568+0000 7f6585878640 1 -- 172.21.15.114:0/150990168 >> 172.21.15.114:0/150990168 conn(0x7f65800ffcd0 msgr2=0x7f6580100bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:31.172 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:30.568+0000 7f6585878640 1 -- 172.21.15.114:0/150990168 shutdown_connections 2024-09-13T18:13:31.172 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:30.568+0000 7f6585878640 1 -- 172.21.15.114:0/150990168 wait complete. 2024-09-13T18:13:31.172 INFO:teuthology.orchestra.run.smithi114.stdout:Fetching dashboard port number... 2024-09-13T18:13:31.689 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:31 smithi114 ceph-mon[30897]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:13:31.689 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:31 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:13:32.305 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout 8443 2024-09-13T18:13:32.306 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:31.512+0000 7f8c21ae7640 1 Processor -- start 2024-09-13T18:13:32.306 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:31.512+0000 7f8c21ae7640 1 -- start start 2024-09-13T18:13:32.306 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:31.512+0000 7f8c21ae7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c1c105590 0x7f8c1c105990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:32.306 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:31.512+0000 7f8c21ae7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c105f60 con 0x7f8c1c105590 2024-09-13T18:13:32.306 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:31.512+0000 7f8c1b7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c1c105590 0x7f8c1c105990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:32.306 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:31.513+0000 7f8c1b7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c1c105590 0x7f8c1c105990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55250/0 (socket says 172.21.15.114:55250) 2024-09-13T18:13:32.306 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:31.513+0000 7f8c1b7fe640 1 -- 172.21.15.114:0/1328295120 learned_addr learned my addr 172.21.15.114:0/1328295120 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:13:32.306 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:31.513+0000 7f8c1b7fe640 1 -- 172.21.15.114:0/1328295120 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c1c106790 con 0x7f8c1c105590 2024-09-13T18:13:32.306 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:31.513+0000 7f8c1b7fe640 1 --2- 172.21.15.114:0/1328295120 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c1c105590 0x7f8c1c105990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c009820 tx=0x7f8c0c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8c4d6b0e2c61101 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:32.306 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:31.513+0000 7f8c1a7fc640 1 -- 172.21.15.114:0/1328295120 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c0c02ae70 con 0x7f8c1c105590 2024-09-13T18:13:32.306 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:31.514+0000 7f8c1a7fc640 1 -- 172.21.15.114:0/1328295120 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8c0c0328d0 con 0x7f8c1c105590 2024-09-13T18:13:32.307 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:31.514+0000 7f8c1a7fc640 1 -- 172.21.15.114:0/1328295120 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c0c0307a0 con 0x7f8c1c105590 2024-09-13T18:13:32.307 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:31.514+0000 7f8c21ae7640 1 -- 172.21.15.114:0/1328295120 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c1c105590 msgr2=0x7f8c1c105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:32.307 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:31.514+0000 7f8c21ae7640 1 --2- 172.21.15.114:0/1328295120 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c1c105590 0x7f8c1c105990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c009820 tx=0x7f8c0c02a270 comp rx=0 tx=0).stop 2024-09-13T18:13:32.307 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:31.514+0000 7f8c21ae7640 1 -- 172.21.15.114:0/1328295120 shutdown_connections 2024-09-13T18:13:32.307 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:31.514+0000 7f8c21ae7640 1 --2- 172.21.15.114:0/1328295120 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c1c105590 0x7f8c1c105990 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:32.307 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:31.514+0000 7f8c21ae7640 1 -- 172.21.15.114:0/1328295120 >> 172.21.15.114:0/1328295120 conn(0x7f8c1c100d60 msgr2=0x7f8c1c103180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:32.307 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:31.514+0000 7f8c21ae7640 1 -- 172.21.15.114:0/1328295120 shutdown_connections 2024-09-13T18:13:32.307 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:31.514+0000 7f8c21ae7640 1 -- 172.21.15.114:0/1328295120 wait complete. 2024-09-13T18:13:32.307 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:31.515+0000 7f8c21ae7640 1 Processor -- start 2024-09-13T18:13:32.307 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:31.515+0000 7f8c21ae7640 1 -- start start 2024-09-13T18:13:32.307 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:31.515+0000 7f8c21ae7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c1c105590 0x7f8c1c199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:32.307 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:31.515+0000 7f8c21ae7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c1997a0 con 0x7f8c1c105590 2024-09-13T18:13:32.307 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:31.515+0000 7f8c1b7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c1c105590 0x7f8c1c199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:32.308 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:31.515+0000 7f8c1b7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c1c105590 0x7f8c1c199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55262/0 (socket says 172.21.15.114:55262) 2024-09-13T18:13:32.308 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:31.515+0000 7f8c1b7fe640 1 -- 172.21.15.114:0/3878423728 learned_addr learned my addr 172.21.15.114:0/3878423728 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:13:32.308 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:31.515+0000 7f8c1b7fe640 1 -- 172.21.15.114:0/3878423728 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c0c0094d0 con 0x7f8c1c105590 2024-09-13T18:13:32.308 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:31.516+0000 7f8c1b7fe640 1 --2- 172.21.15.114:0/3878423728 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c1c105590 0x7f8c1c199260 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c0304f0 tx=0x7f8c0c030cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:32.308 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:31.516+0000 7f8c18ff9640 1 -- 172.21.15.114:0/3878423728 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c0c032c50 con 0x7f8c1c105590 2024-09-13T18:13:32.308 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:31.516+0000 7f8c18ff9640 1 -- 172.21.15.114:0/3878423728 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8c0c032db0 con 0x7f8c1c105590 2024-09-13T18:13:32.308 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:31.516+0000 7f8c21ae7640 1 -- 172.21.15.114:0/3878423728 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c1c1999a0 con 0x7f8c1c105590 2024-09-13T18:13:32.308 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:31.516+0000 7f8c18ff9640 1 -- 172.21.15.114:0/3878423728 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c0c03a5b0 con 0x7f8c1c105590 2024-09-13T18:13:32.308 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:31.516+0000 7f8c21ae7640 1 -- 172.21.15.114:0/3878423728 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c1c199e40 con 0x7f8c1c105590 2024-09-13T18:13:32.308 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:31.517+0000 7f8c18ff9640 1 -- 172.21.15.114:0/3878423728 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 12) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f8c0c039040 con 0x7f8c1c105590 2024-09-13T18:13:32.308 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:31.517+0000 7f8c21ae7640 1 -- 172.21.15.114:0/3878423728 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c1c1099c0 con 0x7f8c1c105590 2024-09-13T18:13:32.308 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:31.517+0000 7f8c18ff9640 1 --2- 172.21.15.114:0/3878423728 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f8c040386c0 0x7f8c0403ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:32.308 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:31.517+0000 7f8c18ff9640 1 -- 172.21.15.114:0/3878423728 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8c0c06ba10 con 0x7f8c1c105590 2024-09-13T18:13:32.309 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:31.517+0000 7f8c1affd640 1 --2- 172.21.15.114:0/3878423728 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f8c040386c0 0x7f8c0403ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:32.309 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:31.517+0000 7f8c1affd640 1 --2- 172.21.15.114:0/3878423728 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f8c040386c0 0x7f8c0403ab80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8c100098c0 tx=0x7f8c10006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:32.309 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:31.522+0000 7f8c18ff9640 1 -- 172.21.15.114:0/3878423728 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f8c0c031580 con 0x7f8c1c105590 2024-09-13T18:13:32.309 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:31.686+0000 7f8c21ae7640 1 -- 172.21.15.114:0/3878423728 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f8c1c109c00 con 0x7f8c1c105590 2024-09-13T18:13:32.309 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:31.687+0000 7f8c18ff9640 1 -- 172.21.15.114:0/3878423728 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f8c0c03f030 con 0x7f8c1c105590 2024-09-13T18:13:32.309 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:31.689+0000 7f8c21ae7640 1 -- 172.21.15.114:0/3878423728 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f8c040386c0 msgr2=0x7f8c0403ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:32.309 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:31.689+0000 7f8c21ae7640 1 --2- 172.21.15.114:0/3878423728 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f8c040386c0 0x7f8c0403ab80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8c100098c0 tx=0x7f8c10006db0 comp rx=0 tx=0).stop 2024-09-13T18:13:32.309 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:31.689+0000 7f8c21ae7640 1 -- 172.21.15.114:0/3878423728 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c1c105590 msgr2=0x7f8c1c199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:32.309 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:31.689+0000 7f8c21ae7640 1 --2- 172.21.15.114:0/3878423728 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c1c105590 0x7f8c1c199260 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c0304f0 tx=0x7f8c0c030cc0 comp rx=0 tx=0).stop 2024-09-13T18:13:32.309 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:31.689+0000 7f8c21ae7640 1 -- 172.21.15.114:0/3878423728 shutdown_connections 2024-09-13T18:13:32.309 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:31.689+0000 7f8c21ae7640 1 --2- 172.21.15.114:0/3878423728 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f8c040386c0 0x7f8c0403ab80 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:32.309 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:31.689+0000 7f8c21ae7640 1 --2- 172.21.15.114:0/3878423728 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c1c105590 0x7f8c1c199260 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:32.309 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:31.689+0000 7f8c21ae7640 1 -- 172.21.15.114:0/3878423728 >> 172.21.15.114:0/3878423728 conn(0x7f8c1c100d60 msgr2=0x7f8c1c101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:32.310 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:31.689+0000 7f8c21ae7640 1 -- 172.21.15.114:0/3878423728 shutdown_connections 2024-09-13T18:13:32.310 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:31.689+0000 7f8c21ae7640 1 -- 172.21.15.114:0/3878423728 wait complete. 2024-09-13T18:13:32.313 INFO:teuthology.orchestra.run.smithi114.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-13T18:13:32.313 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stdout disabled 2024-09-13T18:13:32.320 INFO:teuthology.orchestra.run.smithi114.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-13T18:13:32.320 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stdout inactive 2024-09-13T18:13:32.320 INFO:teuthology.orchestra.run.smithi114.stdout:firewalld.service is not enabled 2024-09-13T18:13:32.320 INFO:teuthology.orchestra.run.smithi114.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-13T18:13:32.322 INFO:teuthology.orchestra.run.smithi114.stdout:Ceph Dashboard is now available at: 2024-09-13T18:13:32.322 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:13:32.322 INFO:teuthology.orchestra.run.smithi114.stdout: URL: https://smithi114.front.sepia.ceph.com:8443/ 2024-09-13T18:13:32.322 INFO:teuthology.orchestra.run.smithi114.stdout: User: admin 2024-09-13T18:13:32.322 INFO:teuthology.orchestra.run.smithi114.stdout: Password: 7ocfzvtitd 2024-09-13T18:13:32.323 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:13:32.323 INFO:teuthology.orchestra.run.smithi114.stdout:Saving cluster configuration to /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/config directory 2024-09-13T18:13:32.323 INFO:teuthology.orchestra.run.smithi114.stdout:Enabling autotune for osd_memory_target 2024-09-13T18:13:32.911 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:32 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/3878423728' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-13T18:13:33.463 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:32.737+0000 7fdbdc013640 1 Processor -- start 2024-09-13T18:13:33.463 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:32.737+0000 7fdbdc013640 1 -- start start 2024-09-13T18:13:33.463 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:32.738+0000 7fdbdc013640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbd4106050 0x7fdbd4108440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:33.463 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:32.738+0000 7fdbdc013640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd4079e70 con 0x7fdbd4106050 2024-09-13T18:13:33.463 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:32.738+0000 7fdbd9daf640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbd4106050 0x7fdbd4108440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:33.463 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:32.738+0000 7fdbd9daf640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbd4106050 0x7fdbd4108440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55266/0 (socket says 172.21.15.114:55266) 2024-09-13T18:13:33.463 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:32.738+0000 7fdbd9daf640 1 -- 172.21.15.114:0/1013360724 learned_addr learned my addr 172.21.15.114:0/1013360724 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:13:33.463 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:32.738+0000 7fdbd9daf640 1 -- 172.21.15.114:0/1013360724 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd4108980 con 0x7fdbd4106050 2024-09-13T18:13:33.464 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:32.739+0000 7fdbd9daf640 1 --2- 172.21.15.114:0/1013360724 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbd4106050 0x7fdbd4108440 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0009820 tx=0x7fdbd002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46eb4b10e76eb542 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:33.464 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:32.739+0000 7fdbd8dad640 1 -- 172.21.15.114:0/1013360724 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdbd002ae70 con 0x7fdbd4106050 2024-09-13T18:13:33.464 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:32.739+0000 7fdbd8dad640 1 -- 172.21.15.114:0/1013360724 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdbd00328d0 con 0x7fdbd4106050 2024-09-13T18:13:33.464 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:32.739+0000 7fdbd8dad640 1 -- 172.21.15.114:0/1013360724 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdbd00307a0 con 0x7fdbd4106050 2024-09-13T18:13:33.464 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:32.739+0000 7fdbdc013640 1 -- 172.21.15.114:0/1013360724 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbd4106050 msgr2=0x7fdbd4108440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:33.464 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:32.739+0000 7fdbdc013640 1 --2- 172.21.15.114:0/1013360724 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbd4106050 0x7fdbd4108440 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0009820 tx=0x7fdbd002a270 comp rx=0 tx=0).stop 2024-09-13T18:13:33.464 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:32.739+0000 7fdbdc013640 1 -- 172.21.15.114:0/1013360724 shutdown_connections 2024-09-13T18:13:33.464 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:32.739+0000 7fdbdc013640 1 --2- 172.21.15.114:0/1013360724 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbd4106050 0x7fdbd4108440 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:33.464 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:32.739+0000 7fdbdc013640 1 -- 172.21.15.114:0/1013360724 >> 172.21.15.114:0/1013360724 conn(0x7fdbd40ffc30 msgr2=0x7fdbd4102050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:33.464 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:32.739+0000 7fdbdc013640 1 -- 172.21.15.114:0/1013360724 shutdown_connections 2024-09-13T18:13:33.464 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:32.739+0000 7fdbdc013640 1 -- 172.21.15.114:0/1013360724 wait complete. 2024-09-13T18:13:33.464 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:32.740+0000 7fdbdc013640 1 Processor -- start 2024-09-13T18:13:33.464 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:32.740+0000 7fdbdc013640 1 -- start start 2024-09-13T18:13:33.465 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:32.740+0000 7fdbdc013640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbd4106050 0x7fdbd4199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:33.465 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:32.740+0000 7fdbdc013640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd41997a0 con 0x7fdbd4106050 2024-09-13T18:13:33.465 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:32.740+0000 7fdbd9daf640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbd4106050 0x7fdbd4199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:33.465 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:32.740+0000 7fdbd9daf640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbd4106050 0x7fdbd4199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55276/0 (socket says 172.21.15.114:55276) 2024-09-13T18:13:33.465 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:32.740+0000 7fdbd9daf640 1 -- 172.21.15.114:0/3392563601 learned_addr learned my addr 172.21.15.114:0/3392563601 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:13:33.465 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:32.740+0000 7fdbd9daf640 1 -- 172.21.15.114:0/3392563601 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd00094d0 con 0x7fdbd4106050 2024-09-13T18:13:33.465 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:32.741+0000 7fdbd9daf640 1 --2- 172.21.15.114:0/3392563601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbd4106050 0x7fdbd4199260 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdbd00304f0 tx=0x7fdbd0030cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:33.465 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:32.741+0000 7fdbcaffd640 1 -- 172.21.15.114:0/3392563601 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdbd0032c50 con 0x7fdbd4106050 2024-09-13T18:13:33.465 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:32.741+0000 7fdbcaffd640 1 -- 172.21.15.114:0/3392563601 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdbd0032db0 con 0x7fdbd4106050 2024-09-13T18:13:33.465 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:32.741+0000 7fdbdc013640 1 -- 172.21.15.114:0/3392563601 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbd41999a0 con 0x7fdbd4106050 2024-09-13T18:13:33.465 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:32.741+0000 7fdbcaffd640 1 -- 172.21.15.114:0/3392563601 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdbd003a640 con 0x7fdbd4106050 2024-09-13T18:13:33.465 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:32.741+0000 7fdbdc013640 1 -- 172.21.15.114:0/3392563601 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbd4199e40 con 0x7fdbd4106050 2024-09-13T18:13:33.465 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:32.742+0000 7fdbcaffd640 1 -- 172.21.15.114:0/3392563601 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7fdbd0039040 con 0x7fdbd4106050 2024-09-13T18:13:33.465 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:32.742+0000 7fdbcaffd640 1 --2- 172.21.15.114:0/3392563601 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fdbc0038760 0x7fdbc003ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:33.466 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:32.742+0000 7fdbdc013640 1 -- 172.21.15.114:0/3392563601 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbd4109720 con 0x7fdbd4106050 2024-09-13T18:13:33.466 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:32.742+0000 7fdbcaffd640 1 -- 172.21.15.114:0/3392563601 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fdbd006baa0 con 0x7fdbd4106050 2024-09-13T18:13:33.466 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:32.742+0000 7fdbd95ae640 1 --2- 172.21.15.114:0/3392563601 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fdbc0038760 0x7fdbc003ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:33.466 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:32.742+0000 7fdbd95ae640 1 --2- 172.21.15.114:0/3392563601 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fdbc0038760 0x7fdbc003ac20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdbc40098c0 tx=0x7fdbc4006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:33.466 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:32.747+0000 7fdbcaffd640 1 -- 172.21.15.114:0/3392563601 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fdbd003a7a0 con 0x7fdbd4106050 2024-09-13T18:13:33.466 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:32.908+0000 7fdbdc013640 1 -- 172.21.15.114:0/3392563601 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7fdbd4109960 con 0x7fdbd4106050 2024-09-13T18:13:33.466 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:32.909+0000 7fdbcaffd640 1 -- 172.21.15.114:0/3392563601 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7fdbd003f070 con 0x7fdbd4106050 2024-09-13T18:13:33.466 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:32.911+0000 7fdbdc013640 1 -- 172.21.15.114:0/3392563601 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fdbc0038760 msgr2=0x7fdbc003ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:33.466 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:32.911+0000 7fdbdc013640 1 --2- 172.21.15.114:0/3392563601 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fdbc0038760 0x7fdbc003ac20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdbc40098c0 tx=0x7fdbc4006db0 comp rx=0 tx=0).stop 2024-09-13T18:13:33.466 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:32.911+0000 7fdbdc013640 1 -- 172.21.15.114:0/3392563601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbd4106050 msgr2=0x7fdbd4199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:33.466 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:32.911+0000 7fdbdc013640 1 --2- 172.21.15.114:0/3392563601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbd4106050 0x7fdbd4199260 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdbd00304f0 tx=0x7fdbd0030cc0 comp rx=0 tx=0).stop 2024-09-13T18:13:33.466 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:32.911+0000 7fdbdc013640 1 -- 172.21.15.114:0/3392563601 shutdown_connections 2024-09-13T18:13:33.467 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:32.911+0000 7fdbdc013640 1 --2- 172.21.15.114:0/3392563601 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fdbc0038760 0x7fdbc003ac20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:33.467 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:32.911+0000 7fdbdc013640 1 --2- 172.21.15.114:0/3392563601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbd4106050 0x7fdbd4199260 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:33.467 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:32.911+0000 7fdbdc013640 1 -- 172.21.15.114:0/3392563601 >> 172.21.15.114:0/3392563601 conn(0x7fdbd40ffc30 msgr2=0x7fdbd4100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:33.467 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:32.911+0000 7fdbdc013640 1 -- 172.21.15.114:0/3392563601 shutdown_connections 2024-09-13T18:13:33.467 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:32.911+0000 7fdbdc013640 1 -- 172.21.15.114:0/3392563601 wait complete. 2024-09-13T18:13:33.833 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:33 smithi114 ceph-mon[30897]: mgrmap e13: x(active, since 6s) 2024-09-13T18:13:34.612 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:33.850+0000 7fe11f9f1640 1 Processor -- start 2024-09-13T18:13:34.612 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:33.850+0000 7fe11f9f1640 1 -- start start 2024-09-13T18:13:34.612 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:33.850+0000 7fe11f9f1640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe11807bee0 0x7fe11807a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:34.612 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:33.850+0000 7fe11f9f1640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11807a910 con 0x7fe11807bee0 2024-09-13T18:13:34.613 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:33.850+0000 7fe11d78d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe11807bee0 0x7fe11807a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:34.613 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:33.850+0000 7fe11d78d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe11807bee0 0x7fe11807a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36314/0 (socket says 172.21.15.114:36314) 2024-09-13T18:13:34.613 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:33.850+0000 7fe11d78d640 1 -- 172.21.15.114:0/1959250031 learned_addr learned my addr 172.21.15.114:0/1959250031 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:13:34.613 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:33.851+0000 7fe11d78d640 1 -- 172.21.15.114:0/1959250031 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe11807aa50 con 0x7fe11807bee0 2024-09-13T18:13:34.613 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:33.851+0000 7fe11d78d640 1 --2- 172.21.15.114:0/1959250031 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe11807bee0 0x7fe11807a340 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe114009820 tx=0x7fe11402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b9332212b34b4a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:34.613 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:33.851+0000 7fe10ffff640 1 -- 172.21.15.114:0/1959250031 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe11402ae70 con 0x7fe11807bee0 2024-09-13T18:13:34.613 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:33.852+0000 7fe10ffff640 1 -- 172.21.15.114:0/1959250031 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe1140328d0 con 0x7fe11807bee0 2024-09-13T18:13:34.613 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:33.852+0000 7fe10ffff640 1 -- 172.21.15.114:0/1959250031 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe1140307a0 con 0x7fe11807bee0 2024-09-13T18:13:34.613 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:33.852+0000 7fe11f9f1640 1 -- 172.21.15.114:0/1959250031 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe11807bee0 msgr2=0x7fe11807a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:34.613 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:33.852+0000 7fe11f9f1640 1 --2- 172.21.15.114:0/1959250031 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe11807bee0 0x7fe11807a340 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe114009820 tx=0x7fe11402a270 comp rx=0 tx=0).stop 2024-09-13T18:13:34.613 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:33.852+0000 7fe11f9f1640 1 -- 172.21.15.114:0/1959250031 shutdown_connections 2024-09-13T18:13:34.613 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:33.852+0000 7fe11f9f1640 1 --2- 172.21.15.114:0/1959250031 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe11807bee0 0x7fe11807a340 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:34.614 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:33.852+0000 7fe11f9f1640 1 -- 172.21.15.114:0/1959250031 >> 172.21.15.114:0/1959250031 conn(0x7fe118100de0 msgr2=0x7fe118103200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:34.614 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:33.852+0000 7fe11f9f1640 1 -- 172.21.15.114:0/1959250031 shutdown_connections 2024-09-13T18:13:34.614 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:33.852+0000 7fe11f9f1640 1 -- 172.21.15.114:0/1959250031 wait complete. 2024-09-13T18:13:34.614 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:33.853+0000 7fe11f9f1640 1 Processor -- start 2024-09-13T18:13:34.614 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:33.853+0000 7fe11f9f1640 1 -- start start 2024-09-13T18:13:34.614 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:33.853+0000 7fe11f9f1640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe11807bee0 0x7fe11819b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:34.614 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:33.853+0000 7fe11f9f1640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11819ba50 con 0x7fe11807bee0 2024-09-13T18:13:34.614 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:33.853+0000 7fe11d78d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe11807bee0 0x7fe11819b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:34.614 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:33.853+0000 7fe11d78d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe11807bee0 0x7fe11819b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36322/0 (socket says 172.21.15.114:36322) 2024-09-13T18:13:34.614 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:33.853+0000 7fe11d78d640 1 -- 172.21.15.114:0/2959942469 learned_addr learned my addr 172.21.15.114:0/2959942469 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:13:34.614 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:33.853+0000 7fe11d78d640 1 -- 172.21.15.114:0/2959942469 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1140094d0 con 0x7fe11807bee0 2024-09-13T18:13:34.614 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:33.853+0000 7fe11d78d640 1 --2- 172.21.15.114:0/2959942469 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe11807bee0 0x7fe11819b510 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe1140097f0 tx=0x7fe114032c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:34.614 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:33.854+0000 7fe10e7fc640 1 -- 172.21.15.114:0/2959942469 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe114030c70 con 0x7fe11807bee0 2024-09-13T18:13:34.614 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:33.854+0000 7fe10e7fc640 1 -- 172.21.15.114:0/2959942469 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe114030dd0 con 0x7fe11807bee0 2024-09-13T18:13:34.615 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:33.854+0000 7fe10e7fc640 1 -- 172.21.15.114:0/2959942469 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe11403a5b0 con 0x7fe11807bee0 2024-09-13T18:13:34.615 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:33.854+0000 7fe11f9f1640 1 -- 172.21.15.114:0/2959942469 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe11819bc50 con 0x7fe11807bee0 2024-09-13T18:13:34.615 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:33.854+0000 7fe11f9f1640 1 -- 172.21.15.114:0/2959942469 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe11819c0f0 con 0x7fe11807bee0 2024-09-13T18:13:34.615 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:33.854+0000 7fe10e7fc640 1 -- 172.21.15.114:0/2959942469 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7fe114039040 con 0x7fe11807bee0 2024-09-13T18:13:34.615 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:33.855+0000 7fe10e7fc640 1 --2- 172.21.15.114:0/2959942469 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fe104038760 0x7fe10403ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:34.615 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:33.855+0000 7fe10e7fc640 1 -- 172.21.15.114:0/2959942469 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe11406ba90 con 0x7fe11807bee0 2024-09-13T18:13:34.615 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:33.855+0000 7fe11f9f1640 1 -- 172.21.15.114:0/2959942469 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe11810b9e0 con 0x7fe11807bee0 2024-09-13T18:13:34.615 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:33.855+0000 7fe11cf8c640 1 --2- 172.21.15.114:0/2959942469 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fe104038760 0x7fe10403ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:34.615 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:33.855+0000 7fe11cf8c640 1 --2- 172.21.15.114:0/2959942469 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fe104038760 0x7fe10403ac20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe1080098c0 tx=0x7fe108006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:34.615 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:33.859+0000 7fe10e7fc640 1 -- 172.21.15.114:0/2959942469 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fe11403f070 con 0x7fe11807bee0 2024-09-13T18:13:34.615 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:34.105+0000 7fe11f9f1640 1 -- 172.21.15.114:0/2959942469 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7fe11819e9d0 con 0x7fe11807bee0 2024-09-13T18:13:34.615 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:34.107+0000 7fe10e7fc640 1 -- 172.21.15.114:0/2959942469 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7fe1140530c0 con 0x7fe11807bee0 2024-09-13T18:13:34.615 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-13T18:13:34.615 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:34.109+0000 7fe11f9f1640 1 -- 172.21.15.114:0/2959942469 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fe104038760 msgr2=0x7fe10403ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:34.616 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:34.109+0000 7fe11f9f1640 1 --2- 172.21.15.114:0/2959942469 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fe104038760 0x7fe10403ac20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe1080098c0 tx=0x7fe108006db0 comp rx=0 tx=0).stop 2024-09-13T18:13:34.616 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:34.109+0000 7fe11f9f1640 1 -- 172.21.15.114:0/2959942469 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe11807bee0 msgr2=0x7fe11819b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:34.616 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:34.109+0000 7fe11f9f1640 1 --2- 172.21.15.114:0/2959942469 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe11807bee0 0x7fe11819b510 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe1140097f0 tx=0x7fe114032c70 comp rx=0 tx=0).stop 2024-09-13T18:13:34.616 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:34.110+0000 7fe11f9f1640 1 -- 172.21.15.114:0/2959942469 shutdown_connections 2024-09-13T18:13:34.616 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:34.110+0000 7fe11f9f1640 1 --2- 172.21.15.114:0/2959942469 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fe104038760 0x7fe10403ac20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:34.616 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:34.110+0000 7fe11f9f1640 1 --2- 172.21.15.114:0/2959942469 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe11807bee0 0x7fe11819b510 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:34.616 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:34.110+0000 7fe11f9f1640 1 -- 172.21.15.114:0/2959942469 >> 172.21.15.114:0/2959942469 conn(0x7fe118100de0 msgr2=0x7fe118101a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:34.616 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:34.110+0000 7fe11f9f1640 1 -- 172.21.15.114:0/2959942469 shutdown_connections 2024-09-13T18:13:34.616 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr 2024-09-13T18:13:34.110+0000 7fe11f9f1640 1 -- 172.21.15.114:0/2959942469 wait complete. 2024-09-13T18:13:34.616 INFO:teuthology.orchestra.run.smithi114.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-13T18:13:34.616 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:13:34.616 INFO:teuthology.orchestra.run.smithi114.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-13T18:13:34.616 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:13:34.617 INFO:teuthology.orchestra.run.smithi114.stdout:Or, if you are only running a single cluster on this host: 2024-09-13T18:13:34.617 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:13:34.617 INFO:teuthology.orchestra.run.smithi114.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-13T18:13:34.617 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:13:34.617 INFO:teuthology.orchestra.run.smithi114.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-13T18:13:34.617 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:13:34.617 INFO:teuthology.orchestra.run.smithi114.stdout: ceph telemetry on 2024-09-13T18:13:34.617 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:13:34.617 INFO:teuthology.orchestra.run.smithi114.stdout:For more information see: 2024-09-13T18:13:34.617 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:13:34.617 INFO:teuthology.orchestra.run.smithi114.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-13T18:13:34.617 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:13:34.617 INFO:teuthology.orchestra.run.smithi114.stdout:Bootstrap complete. 2024-09-13T18:13:34.643 INFO:tasks.cephadm:Fetching config... 2024-09-13T18:13:34.644 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-13T18:13:34.644 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-13T18:13:34.660 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-13T18:13:34.660 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-13T18:13:34.660 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-13T18:13:34.715 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-13T18:13:34.715 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-13T18:13:34.715 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-13T18:13:34.780 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-13T18:13:34.780 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-13T18:13:34.780 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-13T18:13:34.835 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-13T18:13:34.836 DEBUG:teuthology.orchestra.run.smithi114:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3XTyIttOV4ZdNWkAxkfStaCiCLQupZwaIq1ERyZHL8prVEvdx10ZBmTzOVb48lZFK0xK+EedWHZQv4SaQc+FfKW7mMyMji9FXRo45SJzmXp59iNUoHqfMEZPRN2otgRDj8KdroMELAwUtx4loSUFrZNsf7xlOzzbE96TmNam7FzXYlXqEL1cuq1vhTuCzTkpKw/qxF4JDX4oGlv54dCpYeEoydMhykK92WZP3D6ZCvjRqv07alXomjSSblBYTnDdLzVDZt+5fIueLZNqvGRKaSHqctV+HzG1zfyMpGfFDDQhMnNs3l0C8Tk235KdPirUbJoHlLs6VgsGPzhkE5hg9Sk4vqldmx0J8/nDkVklOCEzCuuRhUwgc4vtPedSgLeidCfUo2mqo4FJ6p+7C2gpONkxrBCX3T1CvFRzZhxzImtWcpiefzgm/bDUaLzn1OHcnHDWLr2l82hP271ucUV/J86URXmCMI1Q/qdpYkBNJrihanvmFliUXgkSbB9vuiE8= ceph-97e7edf2-71fb-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-13T18:13:34.926 INFO:teuthology.orchestra.run.smithi114.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3XTyIttOV4ZdNWkAxkfStaCiCLQupZwaIq1ERyZHL8prVEvdx10ZBmTzOVb48lZFK0xK+EedWHZQv4SaQc+FfKW7mMyMji9FXRo45SJzmXp59iNUoHqfMEZPRN2otgRDj8KdroMELAwUtx4loSUFrZNsf7xlOzzbE96TmNam7FzXYlXqEL1cuq1vhTuCzTkpKw/qxF4JDX4oGlv54dCpYeEoydMhykK92WZP3D6ZCvjRqv07alXomjSSblBYTnDdLzVDZt+5fIueLZNqvGRKaSHqctV+HzG1zfyMpGfFDDQhMnNs3l0C8Tk235KdPirUbJoHlLs6VgsGPzhkE5hg9Sk4vqldmx0J8/nDkVklOCEzCuuRhUwgc4vtPedSgLeidCfUo2mqo4FJ6p+7C2gpONkxrBCX3T1CvFRzZhxzImtWcpiefzgm/bDUaLzn1OHcnHDWLr2l82hP271ucUV/J86URXmCMI1Q/qdpYkBNJrihanvmFliUXgkSbB9vuiE8= ceph-97e7edf2-71fb-11ef-bceb-c7b262605968 2024-09-13T18:13:34.938 DEBUG:teuthology.orchestra.run.smithi134:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3XTyIttOV4ZdNWkAxkfStaCiCLQupZwaIq1ERyZHL8prVEvdx10ZBmTzOVb48lZFK0xK+EedWHZQv4SaQc+FfKW7mMyMji9FXRo45SJzmXp59iNUoHqfMEZPRN2otgRDj8KdroMELAwUtx4loSUFrZNsf7xlOzzbE96TmNam7FzXYlXqEL1cuq1vhTuCzTkpKw/qxF4JDX4oGlv54dCpYeEoydMhykK92WZP3D6ZCvjRqv07alXomjSSblBYTnDdLzVDZt+5fIueLZNqvGRKaSHqctV+HzG1zfyMpGfFDDQhMnNs3l0C8Tk235KdPirUbJoHlLs6VgsGPzhkE5hg9Sk4vqldmx0J8/nDkVklOCEzCuuRhUwgc4vtPedSgLeidCfUo2mqo4FJ6p+7C2gpONkxrBCX3T1CvFRzZhxzImtWcpiefzgm/bDUaLzn1OHcnHDWLr2l82hP271ucUV/J86URXmCMI1Q/qdpYkBNJrihanvmFliUXgkSbB9vuiE8= ceph-97e7edf2-71fb-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-13T18:13:35.056 INFO:teuthology.orchestra.run.smithi134.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3XTyIttOV4ZdNWkAxkfStaCiCLQupZwaIq1ERyZHL8prVEvdx10ZBmTzOVb48lZFK0xK+EedWHZQv4SaQc+FfKW7mMyMji9FXRo45SJzmXp59iNUoHqfMEZPRN2otgRDj8KdroMELAwUtx4loSUFrZNsf7xlOzzbE96TmNam7FzXYlXqEL1cuq1vhTuCzTkpKw/qxF4JDX4oGlv54dCpYeEoydMhykK92WZP3D6ZCvjRqv07alXomjSSblBYTnDdLzVDZt+5fIueLZNqvGRKaSHqctV+HzG1zfyMpGfFDDQhMnNs3l0C8Tk235KdPirUbJoHlLs6VgsGPzhkE5hg9Sk4vqldmx0J8/nDkVklOCEzCuuRhUwgc4vtPedSgLeidCfUo2mqo4FJ6p+7C2gpONkxrBCX3T1CvFRzZhxzImtWcpiefzgm/bDUaLzn1OHcnHDWLr2l82hP271ucUV/J86URXmCMI1Q/qdpYkBNJrihanvmFliUXgkSbB9vuiE8= ceph-97e7edf2-71fb-11ef-bceb-c7b262605968 2024-09-13T18:13:35.070 DEBUG:teuthology.orchestra.run.smithi204:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3XTyIttOV4ZdNWkAxkfStaCiCLQupZwaIq1ERyZHL8prVEvdx10ZBmTzOVb48lZFK0xK+EedWHZQv4SaQc+FfKW7mMyMji9FXRo45SJzmXp59iNUoHqfMEZPRN2otgRDj8KdroMELAwUtx4loSUFrZNsf7xlOzzbE96TmNam7FzXYlXqEL1cuq1vhTuCzTkpKw/qxF4JDX4oGlv54dCpYeEoydMhykK92WZP3D6ZCvjRqv07alXomjSSblBYTnDdLzVDZt+5fIueLZNqvGRKaSHqctV+HzG1zfyMpGfFDDQhMnNs3l0C8Tk235KdPirUbJoHlLs6VgsGPzhkE5hg9Sk4vqldmx0J8/nDkVklOCEzCuuRhUwgc4vtPedSgLeidCfUo2mqo4FJ6p+7C2gpONkxrBCX3T1CvFRzZhxzImtWcpiefzgm/bDUaLzn1OHcnHDWLr2l82hP271ucUV/J86URXmCMI1Q/qdpYkBNJrihanvmFliUXgkSbB9vuiE8= ceph-97e7edf2-71fb-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-13T18:13:35.197 INFO:teuthology.orchestra.run.smithi204.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3XTyIttOV4ZdNWkAxkfStaCiCLQupZwaIq1ERyZHL8prVEvdx10ZBmTzOVb48lZFK0xK+EedWHZQv4SaQc+FfKW7mMyMji9FXRo45SJzmXp59iNUoHqfMEZPRN2otgRDj8KdroMELAwUtx4loSUFrZNsf7xlOzzbE96TmNam7FzXYlXqEL1cuq1vhTuCzTkpKw/qxF4JDX4oGlv54dCpYeEoydMhykK92WZP3D6ZCvjRqv07alXomjSSblBYTnDdLzVDZt+5fIueLZNqvGRKaSHqctV+HzG1zfyMpGfFDDQhMnNs3l0C8Tk235KdPirUbJoHlLs6VgsGPzhkE5hg9Sk4vqldmx0J8/nDkVklOCEzCuuRhUwgc4vtPedSgLeidCfUo2mqo4FJ6p+7C2gpONkxrBCX3T1CvFRzZhxzImtWcpiefzgm/bDUaLzn1OHcnHDWLr2l82hP271ucUV/J86URXmCMI1Q/qdpYkBNJrihanvmFliUXgkSbB9vuiE8= ceph-97e7edf2-71fb-11ef-bceb-c7b262605968 2024-09-13T18:13:35.210 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-13T18:13:35.219 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:35 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2959942469' entity='client.admin' 2024-09-13T18:13:35.417 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:13:37.877 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:37 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:13:37.877 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:37 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:13:37.877 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:37 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi114", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:13:37.877 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:37 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:13:37.877 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:37 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:13:37.878 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:37 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:13:37.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:37.874+0000 7ff047c62640 1 -- 172.21.15.114:0/1986102719 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff04006ae50 msgr2=0x7ff04006b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:37.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:37.874+0000 7ff047c62640 1 --2- 172.21.15.114:0/1986102719 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff04006ae50 0x7ff04006b250 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff030009880 tx=0x7ff03002a5d0 comp rx=0 tx=0).stop 2024-09-13T18:13:37.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:37.874+0000 7ff047c62640 1 -- 172.21.15.114:0/1986102719 shutdown_connections 2024-09-13T18:13:37.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:37.874+0000 7ff047c62640 1 --2- 172.21.15.114:0/1986102719 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff04006ae50 0x7ff04006b250 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:37.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:37.874+0000 7ff047c62640 1 -- 172.21.15.114:0/1986102719 >> 172.21.15.114:0/1986102719 conn(0x7ff040069650 msgr2=0x7ff040069a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:37.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:37.874+0000 7ff047c62640 1 -- 172.21.15.114:0/1986102719 shutdown_connections 2024-09-13T18:13:37.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:37.874+0000 7ff047c62640 1 -- 172.21.15.114:0/1986102719 wait complete. 2024-09-13T18:13:37.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:37.875+0000 7ff047c62640 1 Processor -- start 2024-09-13T18:13:37.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:37.875+0000 7ff047c62640 1 -- start start 2024-09-13T18:13:37.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:37.875+0000 7ff047c62640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff04006ae50 0x7ff0401a39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:37.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:37.875+0000 7ff047c62640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0401a3ee0 con 0x7ff04006ae50 2024-09-13T18:13:37.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:37.875+0000 7ff0459fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff04006ae50 0x7ff0401a39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:37.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:37.875+0000 7ff0459fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff04006ae50 0x7ff0401a39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36342/0 (socket says 172.21.15.114:36342) 2024-09-13T18:13:37.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:37.875+0000 7ff0459fe640 1 -- 172.21.15.114:0/840129750 learned_addr learned my addr 172.21.15.114:0/840129750 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:13:37.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:37.876+0000 7ff0459fe640 1 -- 172.21.15.114:0/840129750 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff030009560 con 0x7ff04006ae50 2024-09-13T18:13:37.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:37.876+0000 7ff0459fe640 1 --2- 172.21.15.114:0/840129750 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff04006ae50 0x7ff0401a39a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff030006010 tx=0x7ff030005230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:37.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:37.876+0000 7ff03effd640 1 -- 172.21.15.114:0/840129750 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff0300035f0 con 0x7ff04006ae50 2024-09-13T18:13:37.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:37.876+0000 7ff03effd640 1 -- 172.21.15.114:0/840129750 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff030003be0 con 0x7ff04006ae50 2024-09-13T18:13:37.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:37.876+0000 7ff047c62640 1 -- 172.21.15.114:0/840129750 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0401a40e0 con 0x7ff04006ae50 2024-09-13T18:13:37.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:37.876+0000 7ff03effd640 1 -- 172.21.15.114:0/840129750 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff03003c970 con 0x7ff04006ae50 2024-09-13T18:13:37.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:37.876+0000 7ff047c62640 1 -- 172.21.15.114:0/840129750 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0401a4580 con 0x7ff04006ae50 2024-09-13T18:13:37.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:37.877+0000 7ff03effd640 1 -- 172.21.15.114:0/840129750 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7ff03003cad0 con 0x7ff04006ae50 2024-09-13T18:13:37.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:37.877+0000 7ff03effd640 1 --2- 172.21.15.114:0/840129750 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7ff028038760 0x7ff02803ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:37.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:37.877+0000 7ff03effd640 1 -- 172.21.15.114:0/840129750 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff03006d2a0 con 0x7ff04006ae50 2024-09-13T18:13:37.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:37.877+0000 7ff047c62640 1 -- 172.21.15.114:0/840129750 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff04006b2d0 con 0x7ff04006ae50 2024-09-13T18:13:37.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:37.878+0000 7ff0451fd640 1 --2- 172.21.15.114:0/840129750 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7ff028038760 0x7ff02803ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:37.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:37.878+0000 7ff0451fd640 1 --2- 172.21.15.114:0/840129750 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7ff028038760 0x7ff02803ac20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff03400ad30 tx=0x7ff0340093f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:37.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:37.882+0000 7ff03effd640 1 -- 172.21.15.114:0/840129750 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7ff030035080 con 0x7ff04006ae50 2024-09-13T18:13:38.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:38.052+0000 7ff047c62640 1 -- 172.21.15.114:0/840129750 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7ff04019d190 con 0x7ff04006ae50 2024-09-13T18:13:38.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:38.055+0000 7ff03effd640 1 -- 172.21.15.114:0/840129750 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7ff030040020 con 0x7ff04006ae50 2024-09-13T18:13:38.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:38.057+0000 7ff047c62640 1 -- 172.21.15.114:0/840129750 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7ff028038760 msgr2=0x7ff02803ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:38.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:38.057+0000 7ff047c62640 1 --2- 172.21.15.114:0/840129750 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7ff028038760 0x7ff02803ac20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff03400ad30 tx=0x7ff0340093f0 comp rx=0 tx=0).stop 2024-09-13T18:13:38.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:38.057+0000 7ff047c62640 1 -- 172.21.15.114:0/840129750 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff04006ae50 msgr2=0x7ff0401a39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:38.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:38.057+0000 7ff047c62640 1 --2- 172.21.15.114:0/840129750 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff04006ae50 0x7ff0401a39a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff030006010 tx=0x7ff030005230 comp rx=0 tx=0).stop 2024-09-13T18:13:38.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:38.057+0000 7ff047c62640 1 -- 172.21.15.114:0/840129750 shutdown_connections 2024-09-13T18:13:38.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:38.057+0000 7ff047c62640 1 --2- 172.21.15.114:0/840129750 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7ff028038760 0x7ff02803ac20 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:38.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:38.057+0000 7ff047c62640 1 --2- 172.21.15.114:0/840129750 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff04006ae50 0x7ff0401a39a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:38.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:38.057+0000 7ff047c62640 1 -- 172.21.15.114:0/840129750 >> 172.21.15.114:0/840129750 conn(0x7ff040069650 msgr2=0x7ff040108630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:38.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:38.057+0000 7ff047c62640 1 -- 172.21.15.114:0/840129750 shutdown_connections 2024-09-13T18:13:38.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:38.057+0000 7ff047c62640 1 -- 172.21.15.114:0/840129750 wait complete. 2024-09-13T18:13:38.701 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-13T18:13:38.701 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-13T18:13:39.299 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:13:39.419 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:39 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/840129750' entity='client.admin' 2024-09-13T18:13:39.419 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:39 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:13:39.419 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:39 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:13:40.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:40.789+0000 7f956508e640 1 -- 172.21.15.114:0/3704781109 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95600fe6f0 msgr2=0x7f95600feaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:40.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:40.789+0000 7f956508e640 1 --2- 172.21.15.114:0/3704781109 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95600fe6f0 0x7f95600feaf0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9550009880 tx=0x7f955002a5d0 comp rx=0 tx=0).stop 2024-09-13T18:13:40.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:40.789+0000 7f956508e640 1 -- 172.21.15.114:0/3704781109 shutdown_connections 2024-09-13T18:13:40.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:40.789+0000 7f956508e640 1 --2- 172.21.15.114:0/3704781109 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95600fe6f0 0x7f95600feaf0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:40.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:40.789+0000 7f956508e640 1 -- 172.21.15.114:0/3704781109 >> 172.21.15.114:0/3704781109 conn(0x7f95600f9e80 msgr2=0x7f95600fc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:40.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:40.789+0000 7f956508e640 1 -- 172.21.15.114:0/3704781109 shutdown_connections 2024-09-13T18:13:40.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:40.789+0000 7f956508e640 1 -- 172.21.15.114:0/3704781109 wait complete. 2024-09-13T18:13:40.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:40.790+0000 7f956508e640 1 Processor -- start 2024-09-13T18:13:40.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:40.790+0000 7f956508e640 1 -- start start 2024-09-13T18:13:40.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:40.790+0000 7f956508e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95600fe6f0 0x7f95600688f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:40.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:40.790+0000 7f956508e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9560068e30 con 0x7f95600fe6f0 2024-09-13T18:13:40.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:40.790+0000 7f955ed9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95600fe6f0 0x7f95600688f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:40.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:40.790+0000 7f955ed9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95600fe6f0 0x7f95600688f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36364/0 (socket says 172.21.15.114:36364) 2024-09-13T18:13:40.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:40.790+0000 7f955ed9d640 1 -- 172.21.15.114:0/570190691 learned_addr learned my addr 172.21.15.114:0/570190691 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:13:40.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:40.791+0000 7f955ed9d640 1 -- 172.21.15.114:0/570190691 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9550009560 con 0x7f95600fe6f0 2024-09-13T18:13:40.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:40.791+0000 7f955ed9d640 1 --2- 172.21.15.114:0/570190691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95600fe6f0 0x7f95600688f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9550005230 tx=0x7f9550005260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:40.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:40.791+0000 7f9547fff640 1 -- 172.21.15.114:0/570190691 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9550004b70 con 0x7f95600fe6f0 2024-09-13T18:13:40.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:40.791+0000 7f9547fff640 1 -- 172.21.15.114:0/570190691 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9550003a80 con 0x7f95600fe6f0 2024-09-13T18:13:40.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:40.791+0000 7f956508e640 1 -- 172.21.15.114:0/570190691 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95601979a0 con 0x7f95600fe6f0 2024-09-13T18:13:40.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:40.791+0000 7f9547fff640 1 -- 172.21.15.114:0/570190691 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f955003c970 con 0x7f95600fe6f0 2024-09-13T18:13:40.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:40.791+0000 7f956508e640 1 -- 172.21.15.114:0/570190691 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9560197e40 con 0x7f95600fe6f0 2024-09-13T18:13:40.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:40.792+0000 7f9547fff640 1 -- 172.21.15.114:0/570190691 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7f955003cc30 con 0x7f95600fe6f0 2024-09-13T18:13:40.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:40.792+0000 7f9547fff640 1 --2- 172.21.15.114:0/570190691 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f9548038710 0x7f954803abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:40.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:40.792+0000 7f9547fff640 1 -- 172.21.15.114:0/570190691 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f955006d370 con 0x7f95600fe6f0 2024-09-13T18:13:40.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:40.792+0000 7f955e59c640 1 --2- 172.21.15.114:0/570190691 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f9548038710 0x7f954803abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:40.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:40.792+0000 7f956508e640 1 -- 172.21.15.114:0/570190691 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95600feb70 con 0x7f95600fe6f0 2024-09-13T18:13:40.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:40.792+0000 7f955e59c640 1 --2- 172.21.15.114:0/570190691 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f9548038710 0x7f954803abd0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f95540098c0 tx=0x7f9554006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:40.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:40.797+0000 7f9547fff640 1 -- 172.21.15.114:0/570190691 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f9550035080 con 0x7f95600fe6f0 2024-09-13T18:13:40.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:40.961+0000 7f956508e640 1 -- 172.21.15.114:0/570190691 --> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f9560104620 con 0x7f9548038710 2024-09-13T18:13:40.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:40.965+0000 7f9547fff640 1 -- 172.21.15.114:0/570190691 <== mgr.14148 v2:172.21.15.114:6800/3396185541 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f9560104620 con 0x7f9548038710 2024-09-13T18:13:40.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:40.967+0000 7f956508e640 1 -- 172.21.15.114:0/570190691 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f9548038710 msgr2=0x7f954803abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:40.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:40.967+0000 7f956508e640 1 --2- 172.21.15.114:0/570190691 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f9548038710 0x7f954803abd0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f95540098c0 tx=0x7f9554006db0 comp rx=0 tx=0).stop 2024-09-13T18:13:40.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:40.967+0000 7f956508e640 1 -- 172.21.15.114:0/570190691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95600fe6f0 msgr2=0x7f95600688f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:40.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:40.967+0000 7f956508e640 1 --2- 172.21.15.114:0/570190691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95600fe6f0 0x7f95600688f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9550005230 tx=0x7f9550005260 comp rx=0 tx=0).stop 2024-09-13T18:13:40.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:40.967+0000 7f956508e640 1 -- 172.21.15.114:0/570190691 shutdown_connections 2024-09-13T18:13:40.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:40.967+0000 7f956508e640 1 --2- 172.21.15.114:0/570190691 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f9548038710 0x7f954803abd0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:40.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:40.967+0000 7f956508e640 1 --2- 172.21.15.114:0/570190691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95600fe6f0 0x7f95600688f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:40.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:40.967+0000 7f956508e640 1 -- 172.21.15.114:0/570190691 >> 172.21.15.114:0/570190691 conn(0x7f95600f9e80 msgr2=0x7f95600faa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:40.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:40.967+0000 7f956508e640 1 -- 172.21.15.114:0/570190691 shutdown_connections 2024-09-13T18:13:40.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:40.967+0000 7f956508e640 1 -- 172.21.15.114:0/570190691 wait complete. 2024-09-13T18:13:41.672 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi134 2024-09-13T18:13:41.672 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-13T18:13:41.672 DEBUG:teuthology.orchestra.run.smithi134:> dd of=/etc/ceph/ceph.conf 2024-09-13T18:13:41.692 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-13T18:13:41.692 DEBUG:teuthology.orchestra.run.smithi134:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-13T18:13:41.750 INFO:tasks.cephadm:Adding host smithi134 to orchestrator... 2024-09-13T18:13:41.750 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph orch host add smithi134 2024-09-13T18:13:41.946 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:13:42.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:41 smithi114 ceph-mon[30897]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:13:42.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:41 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:13:42.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:41 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:13:42.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:41 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:13:42.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:41 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:13:42.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:41 smithi114 ceph-mon[30897]: Updating smithi114:/etc/ceph/ceph.conf 2024-09-13T18:13:42.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:41 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:13:42.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:41 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:13:42.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:41 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:13:43.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:42 smithi114 ceph-mon[30897]: Updating smithi114:/var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T18:13:43.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:42 smithi114 ceph-mon[30897]: Updating smithi114:/etc/ceph/ceph.client.admin.keyring 2024-09-13T18:13:43.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:42 smithi114 ceph-mon[30897]: Updating smithi114:/var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-13T18:13:43.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:43.420+0000 7f7504bf9640 1 -- 172.21.15.114:0/3517524086 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7500104670 msgr2=0x7f7500104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:43.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:43.420+0000 7f7504bf9640 1 --2- 172.21.15.114:0/3517524086 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7500104670 0x7f7500104a50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f74f0009c20 tx=0x7f74f002aa90 comp rx=0 tx=0).stop 2024-09-13T18:13:43.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:43.420+0000 7f7504bf9640 1 -- 172.21.15.114:0/3517524086 shutdown_connections 2024-09-13T18:13:43.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:43.420+0000 7f7504bf9640 1 --2- 172.21.15.114:0/3517524086 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7500104670 0x7f7500104a50 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:43.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:43.420+0000 7f7504bf9640 1 -- 172.21.15.114:0/3517524086 >> 172.21.15.114:0/3517524086 conn(0x7f7500075460 msgr2=0x7f75000fc820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:43.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:43.420+0000 7f7504bf9640 1 -- 172.21.15.114:0/3517524086 shutdown_connections 2024-09-13T18:13:43.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:43.420+0000 7f7504bf9640 1 -- 172.21.15.114:0/3517524086 wait complete. 2024-09-13T18:13:43.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:43.421+0000 7f7504bf9640 1 Processor -- start 2024-09-13T18:13:43.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:43.421+0000 7f7504bf9640 1 -- start start 2024-09-13T18:13:43.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:43.421+0000 7f7504bf9640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7500104670 0x7f7500195d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:43.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:43.421+0000 7f7504bf9640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7500196250 con 0x7f7500104670 2024-09-13T18:13:43.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:43.421+0000 7f74fe59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7500104670 0x7f7500195d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:43.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:43.421+0000 7f74fe59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7500104670 0x7f7500195d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:53142/0 (socket says 172.21.15.114:53142) 2024-09-13T18:13:43.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:43.421+0000 7f74fe59c640 1 -- 172.21.15.114:0/2284930774 learned_addr learned my addr 172.21.15.114:0/2284930774 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:13:43.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:43.421+0000 7f74fe59c640 1 -- 172.21.15.114:0/2284930774 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f0009900 con 0x7f7500104670 2024-09-13T18:13:43.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:43.421+0000 7f74fe59c640 1 --2- 172.21.15.114:0/2284930774 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7500104670 0x7f7500195d10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f74f0005e30 tx=0x7f74f002e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:43.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:43.422+0000 7f74ef7fe640 1 -- 172.21.15.114:0/2284930774 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74f003e070 con 0x7f7500104670 2024-09-13T18:13:43.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:43.422+0000 7f74ef7fe640 1 -- 172.21.15.114:0/2284930774 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f74f0003dc0 con 0x7f7500104670 2024-09-13T18:13:43.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:43.422+0000 7f7504bf9640 1 -- 172.21.15.114:0/2284930774 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7500196450 con 0x7f7500104670 2024-09-13T18:13:43.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:43.422+0000 7f74ef7fe640 1 -- 172.21.15.114:0/2284930774 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74f0004700 con 0x7f7500104670 2024-09-13T18:13:43.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:43.422+0000 7f7504bf9640 1 -- 172.21.15.114:0/2284930774 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75001968d0 con 0x7f7500104670 2024-09-13T18:13:43.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:43.423+0000 7f74ef7fe640 1 -- 172.21.15.114:0/2284930774 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7f74f000a040 con 0x7f7500104670 2024-09-13T18:13:43.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:43.423+0000 7f7504bf9640 1 -- 172.21.15.114:0/2284930774 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75000ffdb0 con 0x7f7500104670 2024-09-13T18:13:43.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:43.423+0000 7f74ef7fe640 1 --2- 172.21.15.114:0/2284930774 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f74e8038710 0x7f74e803abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:43.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:43.423+0000 7f74ef7fe640 1 -- 172.21.15.114:0/2284930774 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f74f006c800 con 0x7f7500104670 2024-09-13T18:13:43.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:43.423+0000 7f74fdd9b640 1 --2- 172.21.15.114:0/2284930774 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f74e8038710 0x7f74e803abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:43.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:43.423+0000 7f74fdd9b640 1 --2- 172.21.15.114:0/2284930774 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f74e8038710 0x7f74e803abd0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f74f40098c0 tx=0x7f74f4006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:43.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:43.427+0000 7f74ef7fe640 1 -- 172.21.15.114:0/2284930774 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f74f00423d0 con 0x7f7500104670 2024-09-13T18:13:43.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:43.594+0000 7f7504bf9640 1 -- 172.21.15.114:0/2284930774 --> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi134", "target": ["mon-mgr", ""]}) v1 -- 0x7f7500067c80 con 0x7f74e8038710 2024-09-13T18:13:44.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:44 smithi114 ceph-mon[30897]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi134", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:13:45.469 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:45 smithi114 ceph-mon[30897]: Deploying cephadm binary to smithi134 2024-09-13T18:13:46.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:46.280+0000 7f74ef7fe640 1 -- 172.21.15.114:0/2284930774 <== mgr.14148 v2:172.21.15.114:6800/3396185541 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f7500067c80 con 0x7f74e8038710 2024-09-13T18:13:46.284 INFO:teuthology.orchestra.run.smithi114.stdout:Added host 'smithi134' with addr '172.21.15.134' 2024-09-13T18:13:46.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:46.283+0000 7f7504bf9640 1 -- 172.21.15.114:0/2284930774 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f74e8038710 msgr2=0x7f74e803abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:46.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:46.283+0000 7f7504bf9640 1 --2- 172.21.15.114:0/2284930774 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f74e8038710 0x7f74e803abd0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f74f40098c0 tx=0x7f74f4006db0 comp rx=0 tx=0).stop 2024-09-13T18:13:46.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:46.283+0000 7f7504bf9640 1 -- 172.21.15.114:0/2284930774 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7500104670 msgr2=0x7f7500195d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:46.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:46.283+0000 7f7504bf9640 1 --2- 172.21.15.114:0/2284930774 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7500104670 0x7f7500195d10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f74f0005e30 tx=0x7f74f002e960 comp rx=0 tx=0).stop 2024-09-13T18:13:46.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:46.283+0000 7f7504bf9640 1 -- 172.21.15.114:0/2284930774 shutdown_connections 2024-09-13T18:13:46.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:46.283+0000 7f7504bf9640 1 --2- 172.21.15.114:0/2284930774 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f74e8038710 0x7f74e803abd0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:46.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:46.283+0000 7f7504bf9640 1 --2- 172.21.15.114:0/2284930774 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7500104670 0x7f7500195d10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:46.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:46.283+0000 7f7504bf9640 1 -- 172.21.15.114:0/2284930774 >> 172.21.15.114:0/2284930774 conn(0x7f7500075460 msgr2=0x7f7500102180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:46.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:46.283+0000 7f7504bf9640 1 -- 172.21.15.114:0/2284930774 shutdown_connections 2024-09-13T18:13:46.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:46.283+0000 7f7504bf9640 1 -- 172.21.15.114:0/2284930774 wait complete. 2024-09-13T18:13:46.862 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:46 smithi114 ceph-mon[30897]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:13:46.862 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:46 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:13:46.862 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:46 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:13:46.862 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:46 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:13:46.862 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:46 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:13:46.883 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-13T18:13:47.072 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:13:48.118 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:47 smithi114 ceph-mon[30897]: Added host smithi134 2024-09-13T18:13:48.118 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:47 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:13:48.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:48.547+0000 7f199ecc2640 1 -- 172.21.15.114:0/2591742527 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1998104660 msgr2=0x7f1998104a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:48.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:48.547+0000 7f199ecc2640 1 --2- 172.21.15.114:0/2591742527 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1998104660 0x7f1998104a40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f198c009880 tx=0x7f198c02a5d0 comp rx=0 tx=0).stop 2024-09-13T18:13:48.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:48.547+0000 7f199ecc2640 1 -- 172.21.15.114:0/2591742527 shutdown_connections 2024-09-13T18:13:48.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:48.547+0000 7f199ecc2640 1 --2- 172.21.15.114:0/2591742527 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1998104660 0x7f1998104a40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:48.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:48.547+0000 7f199ecc2640 1 -- 172.21.15.114:0/2591742527 >> 172.21.15.114:0/2591742527 conn(0x7f19980fa360 msgr2=0x7f19980fc780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:48.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:48.547+0000 7f199ecc2640 1 -- 172.21.15.114:0/2591742527 shutdown_connections 2024-09-13T18:13:48.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:48.547+0000 7f199ecc2640 1 -- 172.21.15.114:0/2591742527 wait complete. 2024-09-13T18:13:48.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:48.548+0000 7f199ecc2640 1 Processor -- start 2024-09-13T18:13:48.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:48.548+0000 7f199ecc2640 1 -- start start 2024-09-13T18:13:48.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:48.548+0000 7f199ecc2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1998104660 0x7f1998193e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:48.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:48.548+0000 7f199ecc2640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1998194340 con 0x7f1998104660 2024-09-13T18:13:48.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:48.548+0000 7f199ca5e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1998104660 0x7f1998193e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:48.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:48.548+0000 7f199ca5e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1998104660 0x7f1998193e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:53156/0 (socket says 172.21.15.114:53156) 2024-09-13T18:13:48.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:48.548+0000 7f199ca5e640 1 -- 172.21.15.114:0/3445318362 learned_addr learned my addr 172.21.15.114:0/3445318362 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:13:48.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:48.549+0000 7f199ca5e640 1 -- 172.21.15.114:0/3445318362 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f198c009560 con 0x7f1998104660 2024-09-13T18:13:48.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:48.549+0000 7f199ca5e640 1 --2- 172.21.15.114:0/3445318362 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1998104660 0x7f1998193e00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f198c005ab0 tx=0x7f198c005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:48.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:48.549+0000 7f1995ffb640 1 -- 172.21.15.114:0/3445318362 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f198c005440 con 0x7f1998104660 2024-09-13T18:13:48.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:48.549+0000 7f1995ffb640 1 -- 172.21.15.114:0/3445318362 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f198c0055a0 con 0x7f1998104660 2024-09-13T18:13:48.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:48.549+0000 7f199ecc2640 1 -- 172.21.15.114:0/3445318362 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19981945a0 con 0x7f1998104660 2024-09-13T18:13:48.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:48.549+0000 7f1995ffb640 1 -- 172.21.15.114:0/3445318362 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f198c03c740 con 0x7f1998104660 2024-09-13T18:13:48.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:48.549+0000 7f199ecc2640 1 -- 172.21.15.114:0/3445318362 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1998194a20 con 0x7f1998104660 2024-09-13T18:13:48.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:48.550+0000 7f1995ffb640 1 -- 172.21.15.114:0/3445318362 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7f198c03c9b0 con 0x7f1998104660 2024-09-13T18:13:48.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:48.550+0000 7f1995ffb640 1 --2- 172.21.15.114:0/3445318362 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f1988038760 0x7f198803ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:48.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:48.550+0000 7f1995ffb640 1 -- 172.21.15.114:0/3445318362 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f198c06c290 con 0x7f1998104660 2024-09-13T18:13:48.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:48.550+0000 7f199ecc2640 1 -- 172.21.15.114:0/3445318362 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19980ffda0 con 0x7f1998104660 2024-09-13T18:13:48.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:48.550+0000 7f1997fff640 1 --2- 172.21.15.114:0/3445318362 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f1988038760 0x7f198803ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:48.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:48.551+0000 7f1997fff640 1 --2- 172.21.15.114:0/3445318362 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f1988038760 0x7f198803ac20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f19840098c0 tx=0x7f1984006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:48.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:48.555+0000 7f1995ffb640 1 -- 172.21.15.114:0/3445318362 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f198c035080 con 0x7f1998104660 2024-09-13T18:13:48.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:48.723+0000 7f199ecc2640 1 -- 172.21.15.114:0/3445318362 --> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f19981088b0 con 0x7f1988038760 2024-09-13T18:13:48.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:48.724+0000 7f1995ffb640 1 -- 172.21.15.114:0/3445318362 <== mgr.14148 v2:172.21.15.114:6800/3396185541 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+161 (secure 0 0 0) 0x7f19981088b0 con 0x7f1988038760 2024-09-13T18:13:48.727 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:13:48.727 INFO:teuthology.orchestra.run.smithi114.stdout:[{"addr": "172.21.15.114", "hostname": "smithi114", "labels": [], "status": ""}, {"addr": "172.21.15.134", "hostname": "smithi134", "labels": [], "status": ""}] 2024-09-13T18:13:48.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:48.726+0000 7f199ecc2640 1 -- 172.21.15.114:0/3445318362 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f1988038760 msgr2=0x7f198803ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:48.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:48.726+0000 7f199ecc2640 1 --2- 172.21.15.114:0/3445318362 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f1988038760 0x7f198803ac20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f19840098c0 tx=0x7f1984006db0 comp rx=0 tx=0).stop 2024-09-13T18:13:48.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:48.726+0000 7f199ecc2640 1 -- 172.21.15.114:0/3445318362 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1998104660 msgr2=0x7f1998193e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:48.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:48.726+0000 7f199ecc2640 1 --2- 172.21.15.114:0/3445318362 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1998104660 0x7f1998193e00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f198c005ab0 tx=0x7f198c005e70 comp rx=0 tx=0).stop 2024-09-13T18:13:48.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:48.726+0000 7f199ecc2640 1 -- 172.21.15.114:0/3445318362 shutdown_connections 2024-09-13T18:13:48.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:48.726+0000 7f199ecc2640 1 --2- 172.21.15.114:0/3445318362 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f1988038760 0x7f198803ac20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:48.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:48.726+0000 7f199ecc2640 1 --2- 172.21.15.114:0/3445318362 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1998104660 0x7f1998193e00 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:48.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:48.726+0000 7f199ecc2640 1 -- 172.21.15.114:0/3445318362 >> 172.21.15.114:0/3445318362 conn(0x7f19980fa360 msgr2=0x7f19980fa740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:48.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:48.726+0000 7f199ecc2640 1 -- 172.21.15.114:0/3445318362 shutdown_connections 2024-09-13T18:13:48.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:48.726+0000 7f199ecc2640 1 -- 172.21.15.114:0/3445318362 wait complete. 2024-09-13T18:13:49.024 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:48 smithi114 ceph-mon[30897]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:13:49.306 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi204 2024-09-13T18:13:49.307 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-13T18:13:49.307 DEBUG:teuthology.orchestra.run.smithi204:> dd of=/etc/ceph/ceph.conf 2024-09-13T18:13:49.327 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-13T18:13:49.327 DEBUG:teuthology.orchestra.run.smithi204:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-13T18:13:49.385 INFO:tasks.cephadm:Adding host smithi204 to orchestrator... 2024-09-13T18:13:49.385 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph orch host add smithi204 2024-09-13T18:13:49.579 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:13:49.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:49 smithi114 ceph-mon[30897]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T18:13:50.942 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:50 smithi114 ceph-mon[30897]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:13:50.942 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:50 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:13:50.942 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:50 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:13:50.942 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:50 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:13:50.942 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:50 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:13:50.943 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:50 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi134", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:13:50.943 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:50 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:13:50.943 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:50 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:13:51.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:51.138+0000 7fa4bab5e640 1 -- 172.21.15.114:0/3941216413 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4b4104660 msgr2=0x7fa4b4104a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:51.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:51.138+0000 7fa4bab5e640 1 --2- 172.21.15.114:0/3941216413 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4b4104660 0x7fa4b4104a40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac009880 tx=0x7fa4ac02a5d0 comp rx=0 tx=0).stop 2024-09-13T18:13:51.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:51.138+0000 7fa4bab5e640 1 -- 172.21.15.114:0/3941216413 shutdown_connections 2024-09-13T18:13:51.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:51.138+0000 7fa4bab5e640 1 --2- 172.21.15.114:0/3941216413 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4b4104660 0x7fa4b4104a40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:51.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:51.138+0000 7fa4bab5e640 1 -- 172.21.15.114:0/3941216413 >> 172.21.15.114:0/3941216413 conn(0x7fa4b40fa360 msgr2=0x7fa4b40fc780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:51.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:51.138+0000 7fa4bab5e640 1 -- 172.21.15.114:0/3941216413 shutdown_connections 2024-09-13T18:13:51.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:51.138+0000 7fa4bab5e640 1 -- 172.21.15.114:0/3941216413 wait complete. 2024-09-13T18:13:51.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:51.138+0000 7fa4bab5e640 1 Processor -- start 2024-09-13T18:13:51.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:51.138+0000 7fa4bab5e640 1 -- start start 2024-09-13T18:13:51.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:51.138+0000 7fa4bab5e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4b4104660 0x7fa4b4193e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:51.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:51.138+0000 7fa4bab5e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b4194340 con 0x7fa4b4104660 2024-09-13T18:13:51.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:51.139+0000 7fa4b88fa640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4b4104660 0x7fa4b4193e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:51.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:51.139+0000 7fa4b88fa640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4b4104660 0x7fa4b4193e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:53170/0 (socket says 172.21.15.114:53170) 2024-09-13T18:13:51.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:51.139+0000 7fa4b88fa640 1 -- 172.21.15.114:0/635614119 learned_addr learned my addr 172.21.15.114:0/635614119 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:13:51.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:51.139+0000 7fa4b88fa640 1 -- 172.21.15.114:0/635614119 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4ac009560 con 0x7fa4b4104660 2024-09-13T18:13:51.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:51.139+0000 7fa4b88fa640 1 --2- 172.21.15.114:0/635614119 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4b4104660 0x7fa4b4193e00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac005ab0 tx=0x7fa4ac005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:51.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:51.139+0000 7fa4b1ffb640 1 -- 172.21.15.114:0/635614119 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa4ac005440 con 0x7fa4b4104660 2024-09-13T18:13:51.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:51.139+0000 7fa4b1ffb640 1 -- 172.21.15.114:0/635614119 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa4ac0055a0 con 0x7fa4b4104660 2024-09-13T18:13:51.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:51.139+0000 7fa4bab5e640 1 -- 172.21.15.114:0/635614119 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4b41945a0 con 0x7fa4b4104660 2024-09-13T18:13:51.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:51.139+0000 7fa4b1ffb640 1 -- 172.21.15.114:0/635614119 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa4ac03c740 con 0x7fa4b4104660 2024-09-13T18:13:51.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:51.139+0000 7fa4bab5e640 1 -- 172.21.15.114:0/635614119 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4b4194a20 con 0x7fa4b4104660 2024-09-13T18:13:51.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:51.140+0000 7fa4b1ffb640 1 -- 172.21.15.114:0/635614119 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7fa4ac03c9b0 con 0x7fa4b4104660 2024-09-13T18:13:51.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:51.140+0000 7fa4bab5e640 1 -- 172.21.15.114:0/635614119 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4b40ffda0 con 0x7fa4b4104660 2024-09-13T18:13:51.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:51.140+0000 7fa4b1ffb640 1 --2- 172.21.15.114:0/635614119 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fa4a0038760 0x7fa4a003ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:51.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:51.140+0000 7fa4b1ffb640 1 -- 172.21.15.114:0/635614119 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa4ac06c390 con 0x7fa4b4104660 2024-09-13T18:13:51.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:51.140+0000 7fa4b3fff640 1 --2- 172.21.15.114:0/635614119 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fa4a0038760 0x7fa4a003ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:51.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:51.141+0000 7fa4b3fff640 1 --2- 172.21.15.114:0/635614119 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fa4a0038760 0x7fa4a003ac20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa4a400ad30 tx=0x7fa4a40093f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:51.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:51.145+0000 7fa4b1ffb640 1 -- 172.21.15.114:0/635614119 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fa4ac04d920 con 0x7fa4b4104660 2024-09-13T18:13:51.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:51.316+0000 7fa4bab5e640 1 -- 172.21.15.114:0/635614119 --> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi204", "target": ["mon-mgr", ""]}) v1 -- 0x7fa4b41088e0 con 0x7fa4a0038760 2024-09-13T18:13:52.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:51 smithi114 ceph-mon[30897]: Updating smithi134:/etc/ceph/ceph.conf 2024-09-13T18:13:52.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:51 smithi114 ceph-mon[30897]: Updating smithi134:/var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T18:13:52.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:51 smithi114 ceph-mon[30897]: Updating smithi134:/etc/ceph/ceph.client.admin.keyring 2024-09-13T18:13:52.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:51 smithi114 ceph-mon[30897]: Updating smithi134:/var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-13T18:13:52.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:51 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:13:52.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:51 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:13:52.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:51 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:13:53.219 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:52 smithi114 ceph-mon[30897]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi204", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:13:53.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:52 smithi114 ceph-mon[30897]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:13:54.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:54.124+0000 7fa4b1ffb640 1 -- 172.21.15.114:0/635614119 <== mgr.14148 v2:172.21.15.114:6800/3396185541 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fa4b41088e0 con 0x7fa4a0038760 2024-09-13T18:13:54.128 INFO:teuthology.orchestra.run.smithi114.stdout:Added host 'smithi204' with addr '172.21.15.204' 2024-09-13T18:13:54.128 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:53 smithi114 ceph-mon[30897]: Deploying cephadm binary to smithi204 2024-09-13T18:13:54.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:54.127+0000 7fa4bab5e640 1 -- 172.21.15.114:0/635614119 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fa4a0038760 msgr2=0x7fa4a003ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:54.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:54.127+0000 7fa4bab5e640 1 --2- 172.21.15.114:0/635614119 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fa4a0038760 0x7fa4a003ac20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa4a400ad30 tx=0x7fa4a40093f0 comp rx=0 tx=0).stop 2024-09-13T18:13:54.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:54.127+0000 7fa4bab5e640 1 -- 172.21.15.114:0/635614119 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4b4104660 msgr2=0x7fa4b4193e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:54.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:54.127+0000 7fa4bab5e640 1 --2- 172.21.15.114:0/635614119 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4b4104660 0x7fa4b4193e00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac005ab0 tx=0x7fa4ac005e70 comp rx=0 tx=0).stop 2024-09-13T18:13:54.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:54.127+0000 7fa4bab5e640 1 -- 172.21.15.114:0/635614119 shutdown_connections 2024-09-13T18:13:54.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:54.127+0000 7fa4bab5e640 1 --2- 172.21.15.114:0/635614119 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fa4a0038760 0x7fa4a003ac20 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:54.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:54.127+0000 7fa4bab5e640 1 --2- 172.21.15.114:0/635614119 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4b4104660 0x7fa4b4193e00 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:54.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:54.127+0000 7fa4bab5e640 1 -- 172.21.15.114:0/635614119 >> 172.21.15.114:0/635614119 conn(0x7fa4b40fa360 msgr2=0x7fa4b40fa740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:54.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:54.127+0000 7fa4bab5e640 1 -- 172.21.15.114:0/635614119 shutdown_connections 2024-09-13T18:13:54.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:54.127+0000 7fa4bab5e640 1 -- 172.21.15.114:0/635614119 wait complete. 2024-09-13T18:13:54.710 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-13T18:13:54.899 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:13:54.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:54 smithi114 ceph-mon[30897]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:13:54.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:54 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:13:54.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:54 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:13:54.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:54 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:13:54.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:54 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:13:55.912 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:55 smithi114 ceph-mon[30897]: Added host smithi204 2024-09-13T18:13:55.912 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:55 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:13:56.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:56.407+0000 7fef0a087640 1 -- 172.21.15.114:0/1949669412 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fef04105770 msgr2=0x7fef04105b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:56.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:56.407+0000 7fef0a087640 1 --2- 172.21.15.114:0/1949669412 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fef04105770 0x7fef04105b50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7feef4009880 tx=0x7feef402a5d0 comp rx=0 tx=0).stop 2024-09-13T18:13:56.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:56.407+0000 7fef0a087640 1 -- 172.21.15.114:0/1949669412 shutdown_connections 2024-09-13T18:13:56.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:56.407+0000 7fef0a087640 1 --2- 172.21.15.114:0/1949669412 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fef04105770 0x7fef04105b50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:56.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:56.407+0000 7fef0a087640 1 -- 172.21.15.114:0/1949669412 >> 172.21.15.114:0/1949669412 conn(0x7fef04076660 msgr2=0x7fef040fd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:56.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:56.407+0000 7fef0a087640 1 -- 172.21.15.114:0/1949669412 shutdown_connections 2024-09-13T18:13:56.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:56.407+0000 7fef0a087640 1 -- 172.21.15.114:0/1949669412 wait complete. 2024-09-13T18:13:56.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:56.408+0000 7fef0a087640 1 Processor -- start 2024-09-13T18:13:56.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:56.408+0000 7fef0a087640 1 -- start start 2024-09-13T18:13:56.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:56.408+0000 7fef0a087640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fef04105770 0x7fef04194f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:56.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:56.408+0000 7fef0a087640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef04195440 con 0x7fef04105770 2024-09-13T18:13:56.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:56.408+0000 7fef037fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fef04105770 0x7fef04194f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:56.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:56.408+0000 7fef037fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fef04105770 0x7fef04194f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38152/0 (socket says 172.21.15.114:38152) 2024-09-13T18:13:56.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:56.408+0000 7fef037fe640 1 -- 172.21.15.114:0/2016108614 learned_addr learned my addr 172.21.15.114:0/2016108614 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:13:56.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:56.409+0000 7fef037fe640 1 -- 172.21.15.114:0/2016108614 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feef4009560 con 0x7fef04105770 2024-09-13T18:13:56.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:56.409+0000 7fef037fe640 1 --2- 172.21.15.114:0/2016108614 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fef04105770 0x7fef04194f00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7feef40099a0 tx=0x7feef4004e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:56.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:56.409+0000 7fef00ff9640 1 -- 172.21.15.114:0/2016108614 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feef40053a0 con 0x7fef04105770 2024-09-13T18:13:56.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:56.409+0000 7fef00ff9640 1 -- 172.21.15.114:0/2016108614 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feef4005500 con 0x7fef04105770 2024-09-13T18:13:56.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:56.409+0000 7fef0a087640 1 -- 172.21.15.114:0/2016108614 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef041956a0 con 0x7fef04105770 2024-09-13T18:13:56.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:56.409+0000 7fef00ff9640 1 -- 172.21.15.114:0/2016108614 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feef403c7c0 con 0x7fef04105770 2024-09-13T18:13:56.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:56.409+0000 7fef0a087640 1 -- 172.21.15.114:0/2016108614 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef04195b20 con 0x7fef04105770 2024-09-13T18:13:56.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:56.410+0000 7fef00ff9640 1 -- 172.21.15.114:0/2016108614 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7feef40035f0 con 0x7fef04105770 2024-09-13T18:13:56.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:56.410+0000 7fef00ff9640 1 --2- 172.21.15.114:0/2016108614 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7feef0038710 0x7feef003abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:56.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:56.410+0000 7fef00ff9640 1 -- 172.21.15.114:0/2016108614 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7feef406c2b0 con 0x7fef04105770 2024-09-13T18:13:56.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:56.410+0000 7fef0a087640 1 -- 172.21.15.114:0/2016108614 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef04100eb0 con 0x7fef04105770 2024-09-13T18:13:56.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:56.410+0000 7fef02ffd640 1 --2- 172.21.15.114:0/2016108614 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7feef0038710 0x7feef003abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:56.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:56.410+0000 7fef02ffd640 1 --2- 172.21.15.114:0/2016108614 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7feef0038710 0x7feef003abd0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7feeec0098c0 tx=0x7feeec006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:56.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:56.415+0000 7fef00ff9640 1 -- 172.21.15.114:0/2016108614 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7feef4040080 con 0x7fef04105770 2024-09-13T18:13:56.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:56.584+0000 7fef0a087640 1 -- 172.21.15.114:0/2016108614 --> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fef041099c0 con 0x7feef0038710 2024-09-13T18:13:56.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:56.585+0000 7fef00ff9640 1 -- 172.21.15.114:0/2016108614 <== mgr.14148 v2:172.21.15.114:6800/3396185541 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+241 (secure 0 0 0) 0x7fef041099c0 con 0x7feef0038710 2024-09-13T18:13:56.589 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:13:56.589 INFO:teuthology.orchestra.run.smithi114.stdout:[{"addr": "172.21.15.114", "hostname": "smithi114", "labels": [], "status": ""}, {"addr": "172.21.15.134", "hostname": "smithi134", "labels": [], "status": ""}, {"addr": "172.21.15.204", "hostname": "smithi204", "labels": [], "status": ""}] 2024-09-13T18:13:56.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:56.588+0000 7fef0a087640 1 -- 172.21.15.114:0/2016108614 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7feef0038710 msgr2=0x7feef003abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:56.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:56.588+0000 7fef0a087640 1 --2- 172.21.15.114:0/2016108614 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7feef0038710 0x7feef003abd0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7feeec0098c0 tx=0x7feeec006db0 comp rx=0 tx=0).stop 2024-09-13T18:13:56.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:56.588+0000 7fef0a087640 1 -- 172.21.15.114:0/2016108614 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fef04105770 msgr2=0x7fef04194f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:56.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:56.588+0000 7fef0a087640 1 --2- 172.21.15.114:0/2016108614 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fef04105770 0x7fef04194f00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7feef40099a0 tx=0x7feef4004e50 comp rx=0 tx=0).stop 2024-09-13T18:13:56.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:56.588+0000 7fef0a087640 1 -- 172.21.15.114:0/2016108614 shutdown_connections 2024-09-13T18:13:56.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:56.588+0000 7fef0a087640 1 --2- 172.21.15.114:0/2016108614 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7feef0038710 0x7feef003abd0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:56.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:56.588+0000 7fef0a087640 1 --2- 172.21.15.114:0/2016108614 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fef04105770 0x7fef04194f00 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:56.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:56.588+0000 7fef0a087640 1 -- 172.21.15.114:0/2016108614 >> 172.21.15.114:0/2016108614 conn(0x7fef04076660 msgr2=0x7fef040fb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:56.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:56.588+0000 7fef0a087640 1 -- 172.21.15.114:0/2016108614 shutdown_connections 2024-09-13T18:13:56.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:56.588+0000 7fef0a087640 1 -- 172.21.15.114:0/2016108614 wait complete. 2024-09-13T18:13:57.135 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:56 smithi114 ceph-mon[30897]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:13:57.156 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-13T18:13:57.156 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph osd crush tunables default 2024-09-13T18:13:57.357 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:13:58.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:57 smithi114 ceph-mon[30897]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T18:13:58.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:58.925+0000 7f3a97a44640 1 -- 172.21.15.114:0/337323291 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a90104660 msgr2=0x7f3a90104a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:58.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:58.925+0000 7f3a97a44640 1 --2- 172.21.15.114:0/337323291 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a90104660 0x7f3a90104a40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c009880 tx=0x7f3a8c02a5d0 comp rx=0 tx=0).stop 2024-09-13T18:13:58.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:58.925+0000 7f3a97a44640 1 -- 172.21.15.114:0/337323291 shutdown_connections 2024-09-13T18:13:58.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:58.925+0000 7f3a97a44640 1 --2- 172.21.15.114:0/337323291 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a90104660 0x7f3a90104a40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:58.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:58.925+0000 7f3a97a44640 1 -- 172.21.15.114:0/337323291 >> 172.21.15.114:0/337323291 conn(0x7f3a900fa360 msgr2=0x7f3a900fc780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:58.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:58.925+0000 7f3a97a44640 1 -- 172.21.15.114:0/337323291 shutdown_connections 2024-09-13T18:13:58.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:58.925+0000 7f3a97a44640 1 -- 172.21.15.114:0/337323291 wait complete. 2024-09-13T18:13:58.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:58.926+0000 7f3a97a44640 1 Processor -- start 2024-09-13T18:13:58.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:58.926+0000 7f3a97a44640 1 -- start start 2024-09-13T18:13:58.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:58.926+0000 7f3a97a44640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a90104660 0x7f3a90193b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:58.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:58.926+0000 7f3a97a44640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a90068ce0 con 0x7f3a90104660 2024-09-13T18:13:58.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:58.926+0000 7f3a957e0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a90104660 0x7f3a90193b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:58.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:58.926+0000 7f3a957e0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a90104660 0x7f3a90193b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38168/0 (socket says 172.21.15.114:38168) 2024-09-13T18:13:58.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:58.926+0000 7f3a957e0640 1 -- 172.21.15.114:0/910298425 learned_addr learned my addr 172.21.15.114:0/910298425 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:13:58.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:58.926+0000 7f3a957e0640 1 -- 172.21.15.114:0/910298425 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a8c009560 con 0x7f3a90104660 2024-09-13T18:13:58.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:58.927+0000 7f3a957e0640 1 --2- 172.21.15.114:0/910298425 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a90104660 0x7f3a90193b60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c005ab0 tx=0x7f3a8c005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:58.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:58.927+0000 7f3a867fc640 1 -- 172.21.15.114:0/910298425 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a8c005440 con 0x7f3a90104660 2024-09-13T18:13:58.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:58.927+0000 7f3a97a44640 1 -- 172.21.15.114:0/910298425 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a901940a0 con 0x7f3a90104660 2024-09-13T18:13:58.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:58.927+0000 7f3a867fc640 1 -- 172.21.15.114:0/910298425 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3a8c0055a0 con 0x7f3a90104660 2024-09-13T18:13:58.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:58.927+0000 7f3a867fc640 1 -- 172.21.15.114:0/910298425 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a8c03c740 con 0x7f3a90104660 2024-09-13T18:13:58.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:58.927+0000 7f3a97a44640 1 -- 172.21.15.114:0/910298425 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a90194520 con 0x7f3a90104660 2024-09-13T18:13:58.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:58.928+0000 7f3a97a44640 1 -- 172.21.15.114:0/910298425 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a900ffda0 con 0x7f3a90104660 2024-09-13T18:13:58.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:58.928+0000 7f3a867fc640 1 -- 172.21.15.114:0/910298425 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7f3a8c03ca00 con 0x7f3a90104660 2024-09-13T18:13:58.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:58.928+0000 7f3a867fc640 1 --2- 172.21.15.114:0/910298425 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f3a7c038710 0x7f3a7c03abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:58.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:58.928+0000 7f3a867fc640 1 -- 172.21.15.114:0/910298425 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3a8c06c2e0 con 0x7f3a90104660 2024-09-13T18:13:58.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:58.928+0000 7f3a94fdf640 1 --2- 172.21.15.114:0/910298425 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f3a7c038710 0x7f3a7c03abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:58.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:58.929+0000 7f3a94fdf640 1 --2- 172.21.15.114:0/910298425 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f3a7c038710 0x7f3a7c03abd0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3a800098c0 tx=0x7f3a80006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:58.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:58.934+0000 7f3a867fc640 1 -- 172.21.15.114:0/910298425 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f3a8c04d940 con 0x7f3a90104660 2024-09-13T18:13:59.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:13:59.108+0000 7f3a97a44640 1 -- 172.21.15.114:0/910298425 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f3a90194710 con 0x7f3a90104660 2024-09-13T18:13:59.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:59 smithi114 ceph-mon[30897]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:13:59.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:59 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:13:59.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:59 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:13:59.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:59 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:13:59.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:59 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:13:59.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:59 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi204", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:13:59.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:59 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:13:59.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:59 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:13:59.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:59 smithi114 ceph-mon[30897]: Updating smithi204:/etc/ceph/ceph.conf 2024-09-13T18:13:59.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:59 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:13:59.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:59 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:13:59.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:13:59 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:00.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:00.068+0000 7f3a867fc640 1 -- 172.21.15.114:0/910298425 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f3a8c040090 con 0x7f3a90104660 2024-09-13T18:14:00.071 INFO:teuthology.orchestra.run.smithi114.stderr:adjusted tunables profile to default 2024-09-13T18:14:00.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:00.070+0000 7f3a97a44640 1 -- 172.21.15.114:0/910298425 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f3a7c038710 msgr2=0x7f3a7c03abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:00.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:00.070+0000 7f3a97a44640 1 --2- 172.21.15.114:0/910298425 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f3a7c038710 0x7f3a7c03abd0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3a800098c0 tx=0x7f3a80006db0 comp rx=0 tx=0).stop 2024-09-13T18:14:00.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:00.070+0000 7f3a97a44640 1 -- 172.21.15.114:0/910298425 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a90104660 msgr2=0x7f3a90193b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:00.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:00.070+0000 7f3a97a44640 1 --2- 172.21.15.114:0/910298425 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a90104660 0x7f3a90193b60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c005ab0 tx=0x7f3a8c005e70 comp rx=0 tx=0).stop 2024-09-13T18:14:00.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:00.070+0000 7f3a97a44640 1 -- 172.21.15.114:0/910298425 shutdown_connections 2024-09-13T18:14:00.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:00.070+0000 7f3a97a44640 1 --2- 172.21.15.114:0/910298425 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f3a7c038710 0x7f3a7c03abd0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:00.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:00.070+0000 7f3a97a44640 1 --2- 172.21.15.114:0/910298425 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a90104660 0x7f3a90193b60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:00.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:00.070+0000 7f3a97a44640 1 -- 172.21.15.114:0/910298425 >> 172.21.15.114:0/910298425 conn(0x7f3a900fa360 msgr2=0x7f3a900fa740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:14:00.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:00.070+0000 7f3a97a44640 1 -- 172.21.15.114:0/910298425 shutdown_connections 2024-09-13T18:14:00.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:00.070+0000 7f3a97a44640 1 -- 172.21.15.114:0/910298425 wait complete. 2024-09-13T18:14:00.373 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:00 smithi114 ceph-mon[30897]: Updating smithi204:/var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T18:14:00.373 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:00 smithi114 ceph-mon[30897]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-09-13T18:14:00.373 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:00 smithi114 ceph-mon[30897]: Updating smithi204:/var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-13T18:14:00.374 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:00 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/910298425' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-13T18:14:00.638 INFO:tasks.cephadm:Adding mon.a on smithi114 2024-09-13T18:14:00.638 INFO:tasks.cephadm:Adding mon.b on smithi134 2024-09-13T18:14:00.638 INFO:tasks.cephadm:Adding mon.c on smithi204 2024-09-13T18:14:00.638 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph orch apply mon '3;smithi114:172.21.15.114=a;smithi134:172.21.15.134=b;smithi204:172.21.15.204=c' 2024-09-13T18:14:00.930 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T18:14:01.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:01 smithi114 ceph-mon[30897]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:01.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:01 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/910298425' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-13T18:14:01.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:01 smithi114 ceph-mon[30897]: osdmap e4: 0 total, 0 up, 0 in 2024-09-13T18:14:02.259 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:02.255+0000 7f09b7736640 1 -- 172.21.15.204:0/3778801813 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09b0104710 msgr2=0x7f09b0104af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:02.259 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:02.255+0000 7f09b7736640 1 --2- 172.21.15.204:0/3778801813 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09b0104710 0x7f09b0104af0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f09ac009880 tx=0x7f09ac02a670 comp rx=0 tx=0).stop 2024-09-13T18:14:02.259 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:02.255+0000 7f09b7736640 1 -- 172.21.15.204:0/3778801813 shutdown_connections 2024-09-13T18:14:02.259 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:02.255+0000 7f09b7736640 1 --2- 172.21.15.204:0/3778801813 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09b0104710 0x7f09b0104af0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:02.259 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:02.255+0000 7f09b7736640 1 -- 172.21.15.204:0/3778801813 >> 172.21.15.204:0/3778801813 conn(0x7f09b00f9b10 msgr2=0x7f09b00fbf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:14:02.260 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:02.255+0000 7f09b7736640 1 -- 172.21.15.204:0/3778801813 shutdown_connections 2024-09-13T18:14:02.260 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:02.255+0000 7f09b7736640 1 -- 172.21.15.204:0/3778801813 wait complete. 2024-09-13T18:14:02.260 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:02.255+0000 7f09b7736640 1 Processor -- start 2024-09-13T18:14:02.260 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:02.256+0000 7f09b7736640 1 -- start start 2024-09-13T18:14:02.260 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:02.256+0000 7f09b7736640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09b0104710 0x7f09b019b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:02.260 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:02.256+0000 7f09b7736640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b019ba30 con 0x7f09b0104710 2024-09-13T18:14:02.260 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:02.256+0000 7f09b54d2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09b0104710 0x7f09b019b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:02.260 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:02.256+0000 7f09b54d2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09b0104710 0x7f09b019b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.204:51384/0 (socket says 172.21.15.204:51384) 2024-09-13T18:14:02.260 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:02.256+0000 7f09b54d2640 1 -- 172.21.15.204:0/248744494 learned_addr learned my addr 172.21.15.204:0/248744494 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-13T18:14:02.262 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:02.256+0000 7f09b54d2640 1 -- 172.21.15.204:0/248744494 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09ac009560 con 0x7f09b0104710 2024-09-13T18:14:02.262 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:02.257+0000 7f09b54d2640 1 --2- 172.21.15.204:0/248744494 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09b0104710 0x7f09b019b4f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f09ac005ab0 tx=0x7f09ac005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:02.262 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:02.257+0000 7f09a67fc640 1 -- 172.21.15.204:0/248744494 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09ac005440 con 0x7f09b0104710 2024-09-13T18:14:02.262 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:02.257+0000 7f09a67fc640 1 -- 172.21.15.204:0/248744494 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f09ac0055a0 con 0x7f09b0104710 2024-09-13T18:14:02.262 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:02.257+0000 7f09b7736640 1 -- 172.21.15.204:0/248744494 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09b019bc90 con 0x7f09b0104710 2024-09-13T18:14:02.262 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:02.257+0000 7f09a67fc640 1 -- 172.21.15.204:0/248744494 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09ac03c740 con 0x7f09b0104710 2024-09-13T18:14:02.262 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:02.257+0000 7f09b7736640 1 -- 172.21.15.204:0/248744494 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09b019c190 con 0x7f09b0104710 2024-09-13T18:14:02.262 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:02.258+0000 7f09a67fc640 1 -- 172.21.15.204:0/248744494 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7f09ac0035f0 con 0x7f09b0104710 2024-09-13T18:14:02.262 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:02.258+0000 7f09a67fc640 1 --2- 172.21.15.204:0/248744494 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f099c038760 0x7f099c03ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:02.262 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:02.258+0000 7f09b7736640 1 -- 172.21.15.204:0/248744494 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09b00fee00 con 0x7f09b0104710 2024-09-13T18:14:02.262 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:02.258+0000 7f09a67fc640 1 -- 172.21.15.204:0/248744494 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f09ac06bff0 con 0x7f09b0104710 2024-09-13T18:14:02.262 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:02.258+0000 7f09b4cd1640 1 --2- 172.21.15.204:0/248744494 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f099c038760 0x7f099c03ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:02.263 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:02.259+0000 7f09b4cd1640 1 --2- 172.21.15.204:0/248744494 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f099c038760 0x7f099c03ac20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f09a00098c0 tx=0x7f09a0006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:02.266 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:02.264+0000 7f09a67fc640 1 -- 172.21.15.204:0/248744494 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f09ac040080 con 0x7f09b0104710 2024-09-13T18:14:02.442 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:02.439+0000 7f09b7736640 1 -- 172.21.15.204:0/248744494 --> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi114:172.21.15.114=a;smithi134:172.21.15.134=b;smithi204:172.21.15.204=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f09b019c850 con 0x7f099c038760 2024-09-13T18:14:02.448 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:02.445+0000 7f09a67fc640 1 -- 172.21.15.204:0/248744494 <== mgr.14148 v2:172.21.15.114:6800/3396185541 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f09b019c850 con 0x7f099c038760 2024-09-13T18:14:02.449 INFO:teuthology.orchestra.run.smithi204.stdout:Scheduled mon update... 2024-09-13T18:14:02.451 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:02.448+0000 7f09b7736640 1 -- 172.21.15.204:0/248744494 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f099c038760 msgr2=0x7f099c03ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:02.451 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:02.448+0000 7f09b7736640 1 --2- 172.21.15.204:0/248744494 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f099c038760 0x7f099c03ac20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f09a00098c0 tx=0x7f09a0006db0 comp rx=0 tx=0).stop 2024-09-13T18:14:02.452 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:02.448+0000 7f09b7736640 1 -- 172.21.15.204:0/248744494 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09b0104710 msgr2=0x7f09b019b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:02.452 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:02.448+0000 7f09b7736640 1 --2- 172.21.15.204:0/248744494 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09b0104710 0x7f09b019b4f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f09ac005ab0 tx=0x7f09ac005e70 comp rx=0 tx=0).stop 2024-09-13T18:14:02.452 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:02.448+0000 7f09b7736640 1 -- 172.21.15.204:0/248744494 shutdown_connections 2024-09-13T18:14:02.452 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:02.448+0000 7f09b7736640 1 --2- 172.21.15.204:0/248744494 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f099c038760 0x7f099c03ac20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:02.452 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:02.448+0000 7f09b7736640 1 --2- 172.21.15.204:0/248744494 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09b0104710 0x7f09b019b4f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:02.452 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:02.448+0000 7f09b7736640 1 -- 172.21.15.204:0/248744494 >> 172.21.15.204:0/248744494 conn(0x7f09b00f9b10 msgr2=0x7f09b00f9ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:14:02.452 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:02.448+0000 7f09b7736640 1 -- 172.21.15.204:0/248744494 shutdown_connections 2024-09-13T18:14:02.452 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:02.448+0000 7f09b7736640 1 -- 172.21.15.204:0/248744494 wait complete. 2024-09-13T18:14:03.062 DEBUG:teuthology.orchestra.run.smithi134:mon.b> sudo journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mon.b.service 2024-09-13T18:14:03.065 DEBUG:teuthology.orchestra.run.smithi204:mon.c> sudo journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mon.c.service 2024-09-13T18:14:03.067 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-13T18:14:03.067 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-13T18:14:03.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:03 smithi114 ceph-mon[30897]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:03.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:03 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:03.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:03 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:14:03.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:03 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:03.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:03 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:14:03.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:03 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:03.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:03 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-13T18:14:03.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:03 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:04.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:04 smithi114 ceph-mon[30897]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi114:172.21.15.114=a;smithi134:172.21.15.134=b;smithi204:172.21.15.204=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:14:04.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:04 smithi114 ceph-mon[30897]: Saving service mon spec with placement smithi114:172.21.15.114=a;smithi134:172.21.15.134=b;smithi204:172.21.15.204=c;count:3 2024-09-13T18:14:04.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:04 smithi114 ceph-mon[30897]: Deploying daemon mon.c on smithi204 2024-09-13T18:14:04.594 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T18:14:05.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:05 smithi114 ceph-mon[30897]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:06.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:06 smithi204 podman[34733]: 2024-09-13 18:14:06.636088868 +0000 UTC m=+0.121577096 container create ce3d7d936a949f59620f5e0ac63781549c06310e8326c99f7857708a26b51387 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mon-c, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-13T18:14:06.915 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:06.911+0000 7f7ae1a23640 1 -- 172.21.15.204:0/3544866657 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7adc104670 msgr2=0x7f7adc104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:06.915 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:06.911+0000 7f7ae1a23640 1 --2- 172.21.15.204:0/3544866657 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7adc104670 0x7f7adc104a50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7acc009880 tx=0x7f7acc02a5d0 comp rx=0 tx=0).stop 2024-09-13T18:14:06.915 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:06.912+0000 7f7ae1a23640 1 -- 172.21.15.204:0/3544866657 shutdown_connections 2024-09-13T18:14:06.915 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:06.912+0000 7f7ae1a23640 1 --2- 172.21.15.204:0/3544866657 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7adc104670 0x7f7adc104a50 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:06.915 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:06.912+0000 7f7ae1a23640 1 -- 172.21.15.204:0/3544866657 >> 172.21.15.204:0/3544866657 conn(0x7f7adc075460 msgr2=0x7f7adc0fc820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:14:06.915 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:06.912+0000 7f7ae1a23640 1 -- 172.21.15.204:0/3544866657 shutdown_connections 2024-09-13T18:14:06.915 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:06.912+0000 7f7ae1a23640 1 -- 172.21.15.204:0/3544866657 wait complete. 2024-09-13T18:14:06.915 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:06.912+0000 7f7ae1a23640 1 Processor -- start 2024-09-13T18:14:06.915 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:06.912+0000 7f7ae1a23640 1 -- start start 2024-09-13T18:14:06.916 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:06.912+0000 7f7ae1a23640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7adc104670 0x7f7adc194f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:06.916 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:06.912+0000 7f7ae1a23640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7adc195460 con 0x7f7adc104670 2024-09-13T18:14:06.916 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:06.913+0000 7f7adaffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7adc104670 0x7f7adc194f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:06.916 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:06.913+0000 7f7adaffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7adc104670 0x7f7adc194f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.204:51402/0 (socket says 172.21.15.204:51402) 2024-09-13T18:14:06.916 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:06.913+0000 7f7adaffd640 1 -- 172.21.15.204:0/2602058883 learned_addr learned my addr 172.21.15.204:0/2602058883 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-13T18:14:06.917 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:06.913+0000 7f7adaffd640 1 -- 172.21.15.204:0/2602058883 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7acc009560 con 0x7f7adc104670 2024-09-13T18:14:06.917 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:06.913+0000 7f7adaffd640 1 --2- 172.21.15.204:0/2602058883 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7adc104670 0x7f7adc194f20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7acc0099a0 tx=0x7f7acc004e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:06.917 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:06.914+0000 7f7ae0a21640 1 -- 172.21.15.204:0/2602058883 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7acc0053a0 con 0x7f7adc104670 2024-09-13T18:14:06.917 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:06.914+0000 7f7ae0a21640 1 -- 172.21.15.204:0/2602058883 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7acc005500 con 0x7f7adc104670 2024-09-13T18:14:06.917 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:06.914+0000 7f7ae1a23640 1 -- 172.21.15.204:0/2602058883 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7adc1956c0 con 0x7f7adc104670 2024-09-13T18:14:06.918 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:06.914+0000 7f7ae0a21640 1 -- 172.21.15.204:0/2602058883 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7acc03c7c0 con 0x7f7adc104670 2024-09-13T18:14:06.918 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:06.914+0000 7f7ae1a23640 1 -- 172.21.15.204:0/2602058883 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7adc195b40 con 0x7f7adc104670 2024-09-13T18:14:06.918 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:06.914+0000 7f7ae0a21640 1 -- 172.21.15.204:0/2602058883 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7f7acc03ca00 con 0x7f7adc104670 2024-09-13T18:14:06.918 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:06.915+0000 7f7ae0a21640 1 --2- 172.21.15.204:0/2602058883 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f7ac8038710 0x7f7ac803abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:06.918 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:06.915+0000 7f7ae0a21640 1 -- 172.21.15.204:0/2602058883 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7acc06c610 con 0x7f7adc104670 2024-09-13T18:14:06.918 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:06.915+0000 7f7ae1a23640 1 -- 172.21.15.204:0/2602058883 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7adc0ffdb0 con 0x7f7adc104670 2024-09-13T18:14:06.918 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:06.915+0000 7f7ad3fff640 1 --2- 172.21.15.204:0/2602058883 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f7ac8038710 0x7f7ac803abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:06.918 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:06.915+0000 7f7ad3fff640 1 --2- 172.21.15.204:0/2602058883 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f7ac8038710 0x7f7ac803abd0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7ac40098c0 tx=0x7f7ac4006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:06.922 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:06.920+0000 7f7ae0a21640 1 -- 172.21.15.204:0/2602058883 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f7acc035020 con 0x7f7adc104670 2024-09-13T18:14:07.068 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:07.066+0000 7f7ae0a21640 1 -- 172.21.15.204:0/2602058883 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f7acc005670 con 0x7f7adc104670 2024-09-13T18:14:07.155 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:07.152+0000 7f7ae1a23640 1 -- 172.21.15.204:0/2602058883 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f7adc108890 con 0x7f7adc104670 2024-09-13T18:14:07.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 podman[34733]: 2024-09-13 18:14:07.006503822 +0000 UTC m=+0.491992051 container init ce3d7d936a949f59620f5e0ac63781549c06310e8326c99f7857708a26b51387 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mon-c, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-13T18:14:07.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 podman[34733]: 2024-09-13 18:14:07.011389752 +0000 UTC m=+0.496877980 container start ce3d7d936a949f59620f5e0ac63781549c06310e8326c99f7857708a26b51387 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mon-c, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-13T18:14:07.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: set uid:gid to 167:167 (ceph:ceph) 2024-09-13T18:14:07.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: ceph version 18.2.4-860-g6c6b083f (6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3) reef (stable), process ceph-mon, pid 2 2024-09-13T18:14:07.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: pidfile_write: ignore empty --pid-file 2024-09-13T18:14:07.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: load: jerasure load: lrc 2024-09-13T18:14:07.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: RocksDB version: 7.9.2 2024-09-13T18:14:07.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Git sha 0 2024-09-13T18:14:07.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Compile date 2024-09-10 12:59:06 2024-09-13T18:14:07.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: DB SUMMARY 2024-09-13T18:14:07.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: DB Session ID: US35B7AB067F18EZHI3J 2024-09-13T18:14:07.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: CURRENT file: CURRENT 2024-09-13T18:14:07.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: IDENTITY file: IDENTITY 2024-09-13T18:14:07.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-13T18:14:07.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-13T18:14:07.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-09-13T18:14:07.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.error_if_exists: 0 2024-09-13T18:14:07.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.create_if_missing: 0 2024-09-13T18:14:07.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.paranoid_checks: 1 2024-09-13T18:14:07.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-13T18:14:07.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-13T18:14:07.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-13T18:14:07.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.env: 0x55bb25fb9bc0 2024-09-13T18:14:07.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.fs: PosixFileSystem 2024-09-13T18:14:07.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.info_log: 0x55bb27678be0 2024-09-13T18:14:07.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.max_file_opening_threads: 16 2024-09-13T18:14:07.448 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.statistics: (nil) 2024-09-13T18:14:07.448 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.use_fsync: 0 2024-09-13T18:14:07.448 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.max_log_file_size: 0 2024-09-13T18:14:07.448 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-13T18:14:07.448 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-13T18:14:07.448 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.keep_log_file_num: 1000 2024-09-13T18:14:07.448 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.recycle_log_file_num: 0 2024-09-13T18:14:07.448 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.allow_fallocate: 1 2024-09-13T18:14:07.448 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.allow_mmap_reads: 0 2024-09-13T18:14:07.449 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.allow_mmap_writes: 0 2024-09-13T18:14:07.449 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.use_direct_reads: 0 2024-09-13T18:14:07.449 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-13T18:14:07.449 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.create_missing_column_families: 0 2024-09-13T18:14:07.449 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.db_log_dir: 2024-09-13T18:14:07.449 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.wal_dir: 2024-09-13T18:14:07.449 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-13T18:14:07.449 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-13T18:14:07.449 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-13T18:14:07.449 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-13T18:14:07.449 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-13T18:14:07.449 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-13T18:14:07.449 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.advise_random_on_open: 1 2024-09-13T18:14:07.449 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.db_write_buffer_size: 0 2024-09-13T18:14:07.450 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.write_buffer_manager: 0x55bb27686320 2024-09-13T18:14:07.450 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-13T18:14:07.450 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-13T18:14:07.450 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-13T18:14:07.450 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.rate_limiter: (nil) 2024-09-13T18:14:07.450 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-13T18:14:07.450 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.wal_recovery_mode: 2 2024-09-13T18:14:07.450 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.enable_thread_tracking: 0 2024-09-13T18:14:07.450 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.enable_pipelined_write: 0 2024-09-13T18:14:07.450 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.unordered_write: 0 2024-09-13T18:14:07.450 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-13T18:14:07.450 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-13T18:14:07.450 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-13T18:14:07.451 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-13T18:14:07.451 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.row_cache: None 2024-09-13T18:14:07.451 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.wal_filter: None 2024-09-13T18:14:07.451 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-13T18:14:07.451 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.allow_ingest_behind: 0 2024-09-13T18:14:07.451 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.two_write_queues: 0 2024-09-13T18:14:07.451 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.manual_wal_flush: 0 2024-09-13T18:14:07.451 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.wal_compression: 0 2024-09-13T18:14:07.451 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.atomic_flush: 0 2024-09-13T18:14:07.451 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-13T18:14:07.451 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-13T18:14:07.451 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-13T18:14:07.451 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.log_readahead_size: 0 2024-09-13T18:14:07.451 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-13T18:14:07.451 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.best_efforts_recovery: 0 2024-09-13T18:14:07.452 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-13T18:14:07.452 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-13T18:14:07.452 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.allow_data_in_errors: 0 2024-09-13T18:14:07.452 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.db_host_id: __hostname__ 2024-09-13T18:14:07.452 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-13T18:14:07.452 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.max_background_jobs: 2 2024-09-13T18:14:07.452 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.max_background_compactions: -1 2024-09-13T18:14:07.452 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.max_subcompactions: 1 2024-09-13T18:14:07.452 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-13T18:14:07.452 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-13T18:14:07.452 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-13T18:14:07.452 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.max_total_wal_size: 0 2024-09-13T18:14:07.452 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-13T18:14:07.453 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-13T18:14:07.453 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-13T18:14:07.453 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-13T18:14:07.453 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.max_open_files: -1 2024-09-13T18:14:07.453 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.bytes_per_sync: 0 2024-09-13T18:14:07.453 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-13T18:14:07.453 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-13T18:14:07.453 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.compaction_readahead_size: 0 2024-09-13T18:14:07.453 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.max_background_flushes: -1 2024-09-13T18:14:07.453 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Compression algorithms supported: 2024-09-13T18:14:07.454 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: kZSTD supported: 0 2024-09-13T18:14:07.454 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: kXpressCompression supported: 0 2024-09-13T18:14:07.454 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: kBZip2Compression supported: 0 2024-09-13T18:14:07.454 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-13T18:14:07.454 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: kLZ4Compression supported: 1 2024-09-13T18:14:07.454 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: kZlibCompression supported: 1 2024-09-13T18:14:07.454 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: kLZ4HCCompression supported: 1 2024-09-13T18:14:07.454 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: kSnappyCompression supported: 1 2024-09-13T18:14:07.454 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-13T18:14:07.454 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-13T18:14:07.454 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-13T18:14:07.454 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-13T18:14:07.455 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-13T18:14:07.455 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.merge_operator: 2024-09-13T18:14:07.455 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.compaction_filter: None 2024-09-13T18:14:07.455 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.compaction_filter_factory: None 2024-09-13T18:14:07.455 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.sst_partitioner_factory: None 2024-09-13T18:14:07.455 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-13T18:14:07.455 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-13T18:14:07.455 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55bb276787e0) 2024-09-13T18:14:07.455 INFO:journalctl@ceph.mon.c.smithi204.stdout: cache_index_and_filter_blocks: 1 2024-09-13T18:14:07.455 INFO:journalctl@ceph.mon.c.smithi204.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-13T18:14:07.455 INFO:journalctl@ceph.mon.c.smithi204.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-13T18:14:07.455 INFO:journalctl@ceph.mon.c.smithi204.stdout: pin_top_level_index_and_filter: 1 2024-09-13T18:14:07.455 INFO:journalctl@ceph.mon.c.smithi204.stdout: index_type: 0 2024-09-13T18:14:07.456 INFO:journalctl@ceph.mon.c.smithi204.stdout: data_block_index_type: 0 2024-09-13T18:14:07.456 INFO:journalctl@ceph.mon.c.smithi204.stdout: index_shortening: 1 2024-09-13T18:14:07.456 INFO:journalctl@ceph.mon.c.smithi204.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-13T18:14:07.456 INFO:journalctl@ceph.mon.c.smithi204.stdout: checksum: 4 2024-09-13T18:14:07.456 INFO:journalctl@ceph.mon.c.smithi204.stdout: no_block_cache: 0 2024-09-13T18:14:07.456 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_cache: 0x55bb2766d350 2024-09-13T18:14:07.456 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_cache_name: BinnedLRUCache 2024-09-13T18:14:07.456 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_cache_options: 2024-09-13T18:14:07.456 INFO:journalctl@ceph.mon.c.smithi204.stdout: capacity : 536870912 2024-09-13T18:14:07.456 INFO:journalctl@ceph.mon.c.smithi204.stdout: num_shard_bits : 4 2024-09-13T18:14:07.456 INFO:journalctl@ceph.mon.c.smithi204.stdout: strict_capacity_limit : 0 2024-09-13T18:14:07.456 INFO:journalctl@ceph.mon.c.smithi204.stdout: high_pri_pool_ratio: 0.000 2024-09-13T18:14:07.456 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_cache_compressed: (nil) 2024-09-13T18:14:07.457 INFO:journalctl@ceph.mon.c.smithi204.stdout: persistent_cache: (nil) 2024-09-13T18:14:07.457 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_size: 4096 2024-09-13T18:14:07.457 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_size_deviation: 10 2024-09-13T18:14:07.457 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_restart_interval: 16 2024-09-13T18:14:07.457 INFO:journalctl@ceph.mon.c.smithi204.stdout: index_block_restart_interval: 1 2024-09-13T18:14:07.457 INFO:journalctl@ceph.mon.c.smithi204.stdout: metadata_block_size: 4096 2024-09-13T18:14:07.457 INFO:journalctl@ceph.mon.c.smithi204.stdout: partition_filters: 0 2024-09-13T18:14:07.457 INFO:journalctl@ceph.mon.c.smithi204.stdout: use_delta_encoding: 1 2024-09-13T18:14:07.457 INFO:journalctl@ceph.mon.c.smithi204.stdout: filter_policy: bloomfilter 2024-09-13T18:14:07.457 INFO:journalctl@ceph.mon.c.smithi204.stdout: whole_key_filtering: 1 2024-09-13T18:14:07.457 INFO:journalctl@ceph.mon.c.smithi204.stdout: verify_compression: 0 2024-09-13T18:14:07.457 INFO:journalctl@ceph.mon.c.smithi204.stdout: read_amp_bytes_per_bit: 0 2024-09-13T18:14:07.457 INFO:journalctl@ceph.mon.c.smithi204.stdout: format_version: 5 2024-09-13T18:14:07.457 INFO:journalctl@ceph.mon.c.smithi204.stdout: enable_index_compression: 1 2024-09-13T18:14:07.457 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_align: 0 2024-09-13T18:14:07.458 INFO:journalctl@ceph.mon.c.smithi204.stdout: max_auto_readahead_size: 262144 2024-09-13T18:14:07.458 INFO:journalctl@ceph.mon.c.smithi204.stdout: prepopulate_block_cache: 0 2024-09-13T18:14:07.458 INFO:journalctl@ceph.mon.c.smithi204.stdout: initial_auto_readahead_size: 8192 2024-09-13T18:14:07.458 INFO:journalctl@ceph.mon.c.smithi204.stdout: num_file_reads_for_auto_readahead: 2 2024-09-13T18:14:07.458 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.write_buffer_size: 33554432 2024-09-13T18:14:07.458 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.max_write_buffer_number: 2 2024-09-13T18:14:07.458 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.compression: NoCompression 2024-09-13T18:14:07.458 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.bottommost_compression: Disabled 2024-09-13T18:14:07.458 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.prefix_extractor: nullptr 2024-09-13T18:14:07.458 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-13T18:14:07.458 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.num_levels: 7 2024-09-13T18:14:07.458 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-13T18:14:07.458 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-13T18:14:07.458 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-13T18:14:07.458 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-13T18:14:07.459 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-13T18:14:07.459 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-13T18:14:07.459 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-13T18:14:07.459 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-13T18:14:07.459 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-13T18:14:07.459 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-13T18:14:07.459 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-13T18:14:07.459 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-13T18:14:07.459 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-13T18:14:07.459 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.compression_opts.level: 32767 2024-09-13T18:14:07.459 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.compression_opts.strategy: 0 2024-09-13T18:14:07.459 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-13T18:14:07.459 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-13T18:14:07.459 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-13T18:14:07.460 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-13T18:14:07.460 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.compression_opts.enabled: false 2024-09-13T18:14:07.460 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-13T18:14:07.460 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-13T18:14:07.460 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-13T18:14:07.460 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-13T18:14:07.460 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.target_file_size_base: 67108864 2024-09-13T18:14:07.460 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-13T18:14:07.460 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-13T18:14:07.460 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-13T18:14:07.460 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-13T18:14:07.460 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-13T18:14:07.460 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-13T18:14:07.460 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-13T18:14:07.461 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-13T18:14:07.461 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-13T18:14:07.461 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-13T18:14:07.461 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-13T18:14:07.461 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-13T18:14:07.461 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-13T18:14:07.461 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-13T18:14:07.461 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.arena_block_size: 1048576 2024-09-13T18:14:07.461 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-13T18:14:07.461 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-13T18:14:07.461 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.disable_auto_compactions: 0 2024-09-13T18:14:07.461 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-13T18:14:07.461 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-13T18:14:07.461 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-13T18:14:07.462 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-13T18:14:07.462 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-13T18:14:07.462 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-13T18:14:07.462 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-13T18:14:07.462 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-13T18:14:07.462 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-13T18:14:07.462 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-13T18:14:07.462 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-13T18:14:07.462 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.inplace_update_support: 0 2024-09-13T18:14:07.462 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-13T18:14:07.462 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-13T18:14:07.462 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-13T18:14:07.462 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-13T18:14:07.462 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.bloom_locality: 0 2024-09-13T18:14:07.463 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.max_successive_merges: 0 2024-09-13T18:14:07.463 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-13T18:14:07.463 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.paranoid_file_checks: 0 2024-09-13T18:14:07.463 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.force_consistency_checks: 1 2024-09-13T18:14:07.463 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.report_bg_io_stats: 0 2024-09-13T18:14:07.463 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.ttl: 2592000 2024-09-13T18:14:07.463 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-13T18:14:07.463 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-13T18:14:07.463 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-13T18:14:07.463 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.enable_blob_files: false 2024-09-13T18:14:07.463 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.min_blob_size: 0 2024-09-13T18:14:07.463 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.blob_file_size: 268435456 2024-09-13T18:14:07.463 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-13T18:14:07.463 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-13T18:14:07.464 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-13T18:14:07.464 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-13T18:14:07.464 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-13T18:14:07.464 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.blob_file_starting_level: 0 2024-09-13T18:14:07.464 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-13T18:14:07.464 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-13T18:14:07.464 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-13T18:14:07.464 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 49e5814d-f6f3-4305-8067-e9ea8f0f0031 2024-09-13T18:14:07.464 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726251247038249, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-13T18:14:07.464 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-13T18:14:07.464 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726251247038675, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726251247, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "49e5814d-f6f3-4305-8067-e9ea8f0f0031", "db_session_id": "US35B7AB067F18EZHI3J", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-13T18:14:07.465 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726251247038771, "job": 1, "event": "recovery_finished"} 2024-09-13T18:14:07.465 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-13T18:14:07.465 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-13T18:14:07.465 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55bb27698e00 2024-09-13T18:14:07.465 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: DB pointer 0x55bb2778c000 2024-09-13T18:14:07.465 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-13T18:14:07.465 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-13T18:14:07.465 INFO:journalctl@ceph.mon.c.smithi204.stdout: ** DB Stats ** 2024-09-13T18:14:07.465 INFO:journalctl@ceph.mon.c.smithi204.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-13T18:14:07.465 INFO:journalctl@ceph.mon.c.smithi204.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-13T18:14:07.465 INFO:journalctl@ceph.mon.c.smithi204.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-13T18:14:07.465 INFO:journalctl@ceph.mon.c.smithi204.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-13T18:14:07.465 INFO:journalctl@ceph.mon.c.smithi204.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-13T18:14:07.466 INFO:journalctl@ceph.mon.c.smithi204.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-13T18:14:07.466 INFO:journalctl@ceph.mon.c.smithi204.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-13T18:14:07.466 INFO:journalctl@ceph.mon.c.smithi204.stdout: 2024-09-13T18:14:07.466 INFO:journalctl@ceph.mon.c.smithi204.stdout: ** Compaction Stats [default] ** 2024-09-13T18:14:07.466 INFO:journalctl@ceph.mon.c.smithi204.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-13T18:14:07.466 INFO:journalctl@ceph.mon.c.smithi204.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-13T18:14:07.466 INFO:journalctl@ceph.mon.c.smithi204.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-13T18:14:07.466 INFO:journalctl@ceph.mon.c.smithi204.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-13T18:14:07.466 INFO:journalctl@ceph.mon.c.smithi204.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-13T18:14:07.466 INFO:journalctl@ceph.mon.c.smithi204.stdout: 2024-09-13T18:14:07.466 INFO:journalctl@ceph.mon.c.smithi204.stdout: ** Compaction Stats [default] ** 2024-09-13T18:14:07.466 INFO:journalctl@ceph.mon.c.smithi204.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-13T18:14:07.466 INFO:journalctl@ceph.mon.c.smithi204.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-13T18:14:07.466 INFO:journalctl@ceph.mon.c.smithi204.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-13T18:14:07.467 INFO:journalctl@ceph.mon.c.smithi204.stdout: 2024-09-13T18:14:07.467 INFO:journalctl@ceph.mon.c.smithi204.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-13T18:14:07.467 INFO:journalctl@ceph.mon.c.smithi204.stdout: 2024-09-13T18:14:07.467 INFO:journalctl@ceph.mon.c.smithi204.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-13T18:14:07.467 INFO:journalctl@ceph.mon.c.smithi204.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-13T18:14:07.467 INFO:journalctl@ceph.mon.c.smithi204.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-13T18:14:07.467 INFO:journalctl@ceph.mon.c.smithi204.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-13T18:14:07.467 INFO:journalctl@ceph.mon.c.smithi204.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-13T18:14:07.467 INFO:journalctl@ceph.mon.c.smithi204.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-13T18:14:07.467 INFO:journalctl@ceph.mon.c.smithi204.stdout: Cumulative compaction: 0.00 GB write, 0.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-13T18:14:07.467 INFO:journalctl@ceph.mon.c.smithi204.stdout: Interval compaction: 0.00 GB write, 0.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-13T18:14:07.467 INFO:journalctl@ceph.mon.c.smithi204.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-13T18:14:07.467 INFO:journalctl@ceph.mon.c.smithi204.stdout: Block cache BinnedLRUCache@0x55bb2766d350#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-13T18:14:07.467 INFO:journalctl@ceph.mon.c.smithi204.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-13T18:14:07.468 INFO:journalctl@ceph.mon.c.smithi204.stdout: 2024-09-13T18:14:07.468 INFO:journalctl@ceph.mon.c.smithi204.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-13T18:14:07.468 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-13T18:14:07.468 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: using public_addr v2:172.21.15.204:0/0 -> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-13T18:14:07.468 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: starting mon.c rank -1 at public addrs [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] at bind addrs [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 2024-09-13T18:14:07.468 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: mon.c@-1(???) e0 preinit fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 2024-09-13T18:14:07.468 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: mon.c@-1(synchronizing).mds e1 new map 2024-09-13T18:14:07.468 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-13T18:14:07.468 INFO:journalctl@ceph.mon.c.smithi204.stdout: e1 2024-09-13T18:14:07.468 INFO:journalctl@ceph.mon.c.smithi204.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-13T18:14:07.468 INFO:journalctl@ceph.mon.c.smithi204.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-13T18:14:07.468 INFO:journalctl@ceph.mon.c.smithi204.stdout: legacy client fscid: -1 2024-09-13T18:14:07.468 INFO:journalctl@ceph.mon.c.smithi204.stdout: 2024-09-13T18:14:07.469 INFO:journalctl@ceph.mon.c.smithi204.stdout: No filesystems configured 2024-09-13T18:14:07.469 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-13T18:14:07.469 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-13T18:14:07.469 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-13T18:14:07.469 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-13T18:14:07.469 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-13T18:14:07.469 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-13T18:14:07.469 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-13T18:14:07.469 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-13T18:14:07.469 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-13T18:14:07.469 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-13T18:14:07.469 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi204", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:14:07.469 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:07.469 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: Deploying cephadm binary to smithi204 2024-09-13T18:14:07.470 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:07.470 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:07.470 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:14:07.470 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:07.470 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:07.470 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: Added host smithi204 2024-09-13T18:14:07.470 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:07.470 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:07.470 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T18:14:07.470 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:07.470 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:07.470 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:07.470 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:07.470 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:07.471 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi204", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:14:07.471 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:07.471 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:14:07.471 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: Updating smithi204:/etc/ceph/ceph.conf 2024-09-13T18:14:07.471 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:07.471 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:07.471 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:07.471 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: Updating smithi204:/var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T18:14:07.471 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-09-13T18:14:07.471 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: Updating smithi204:/var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-13T18:14:07.471 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/910298425' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-13T18:14:07.471 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:07.471 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/910298425' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-13T18:14:07.471 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: osdmap e4: 0 total, 0 up, 0 in 2024-09-13T18:14:07.472 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:07.472 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:07.472 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:14:07.472 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:07.472 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:14:07.472 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:07.472 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-13T18:14:07.472 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:07.472 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi114:172.21.15.114=a;smithi134:172.21.15.134=b;smithi204:172.21.15.204=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:14:07.472 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: Saving service mon spec with placement smithi114:172.21.15.114=a;smithi134:172.21.15.134=b;smithi204:172.21.15.204=c;count:3 2024-09-13T18:14:07.472 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: Deploying daemon mon.c on smithi204 2024-09-13T18:14:07.472 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:07.472 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 ceph-mon[34777]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-13T18:14:07.472 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 bash[34733]: ce3d7d936a949f59620f5e0ac63781549c06310e8326c99f7857708a26b51387 2024-09-13T18:14:07.472 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:07 smithi204 systemd[1]: Started Ceph mon.c for 97e7edf2-71fb-11ef-bceb-c7b262605968. 2024-09-13T18:14:12.081 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:12.078+0000 7f7ae0a21640 1 -- 172.21.15.204:0/2602058883 <== mon.0 v2:172.21.15.114:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1010 (secure 0 0 0) 0x7f7acc035070 con 0x7f7adc104670 2024-09-13T18:14:12.081 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-13T18:14:12.081 INFO:teuthology.orchestra.run.smithi204.stdout:{"epoch":2,"fsid":"97e7edf2-71fb-11ef-bceb-c7b262605968","modified":"2024-09-13T18:14:07.066533Z","created":"2024-09-13T18:12:33.374504Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:3300","nonce":0},{"type":"v1","addr":"172.21.15.114:6789","nonce":0}]},"addr":"172.21.15.114:6789/0","public_addr":"172.21.15.114:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-13T18:14:12.082 INFO:teuthology.orchestra.run.smithi204.stderr:dumped monmap epoch 2 2024-09-13T18:14:12.084 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:12.081+0000 7f7ae1a23640 1 -- 172.21.15.204:0/2602058883 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f7ac8038710 msgr2=0x7f7ac803abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:12.084 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:12.081+0000 7f7ae1a23640 1 --2- 172.21.15.204:0/2602058883 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f7ac8038710 0x7f7ac803abd0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7ac40098c0 tx=0x7f7ac4006db0 comp rx=0 tx=0).stop 2024-09-13T18:14:12.084 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:12.081+0000 7f7ae1a23640 1 -- 172.21.15.204:0/2602058883 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7adc104670 msgr2=0x7f7adc194f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:12.084 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:12.081+0000 7f7ae1a23640 1 --2- 172.21.15.204:0/2602058883 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7adc104670 0x7f7adc194f20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7acc0099a0 tx=0x7f7acc004e50 comp rx=0 tx=0).stop 2024-09-13T18:14:12.084 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:12.082+0000 7f7ae1a23640 1 -- 172.21.15.204:0/2602058883 shutdown_connections 2024-09-13T18:14:12.084 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:12.082+0000 7f7ae1a23640 1 --2- 172.21.15.204:0/2602058883 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f7ac8038710 0x7f7ac803abd0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:12.084 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:12.082+0000 7f7ae1a23640 1 --2- 172.21.15.204:0/2602058883 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7adc104670 0x7f7adc194f20 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:12.085 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:12.082+0000 7f7ae1a23640 1 -- 172.21.15.204:0/2602058883 >> 172.21.15.204:0/2602058883 conn(0x7f7adc075460 msgr2=0x7f7adc067650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:14:12.085 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:12.082+0000 7f7ae1a23640 1 -- 172.21.15.204:0/2602058883 shutdown_connections 2024-09-13T18:14:12.085 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:12.082+0000 7f7ae1a23640 1 -- 172.21.15.204:0/2602058883 wait complete. 2024-09-13T18:14:12.387 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:12 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T18:14:12.387 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:12 smithi204 ceph-mon[34777]: mon.a calling monitor election 2024-09-13T18:14:12.387 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:12 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T18:14:12.387 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:12 smithi204 ceph-mon[34777]: from='client.? 172.21.15.204:0/2602058883' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-13T18:14:12.387 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:12 smithi204 ceph-mon[34777]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:12.387 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:12 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T18:14:12.387 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:12 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T18:14:12.387 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:12 smithi204 ceph-mon[34777]: mon.c calling monitor election 2024-09-13T18:14:12.387 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:12 smithi204 ceph-mon[34777]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:12.387 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:12 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T18:14:12.387 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:12 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T18:14:12.388 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:12 smithi204 ceph-mon[34777]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:12.388 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:12 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T18:14:12.388 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:12 smithi204 ceph-mon[34777]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-13T18:14:12.388 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:12 smithi204 ceph-mon[34777]: monmap e2: 2 mons at {a=[v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0],c=[v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-13T18:14:12.388 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:12 smithi204 ceph-mon[34777]: fsmap 2024-09-13T18:14:12.388 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:12 smithi204 ceph-mon[34777]: osdmap e4: 0 total, 0 up, 0 in 2024-09-13T18:14:12.388 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:12 smithi204 ceph-mon[34777]: mgrmap e13: x(active, since 46s) 2024-09-13T18:14:12.388 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:12 smithi204 ceph-mon[34777]: overall HEALTH_OK 2024-09-13T18:14:12.388 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:12 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:12.388 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:12 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:12.388 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:12 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:12.388 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:12 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-13T18:14:12.388 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:12 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:12.388 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:12 smithi204 ceph-mon[34777]: Deploying daemon mon.b on smithi134 2024-09-13T18:14:12.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:12 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T18:14:12.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:12 smithi114 ceph-mon[30897]: mon.a calling monitor election 2024-09-13T18:14:12.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:12 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T18:14:12.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:12 smithi114 ceph-mon[30897]: from='client.? 172.21.15.204:0/2602058883' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-13T18:14:12.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:12 smithi114 ceph-mon[30897]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:12.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:12 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T18:14:12.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:12 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T18:14:12.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:12 smithi114 ceph-mon[30897]: mon.c calling monitor election 2024-09-13T18:14:12.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:12 smithi114 ceph-mon[30897]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:12.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:12 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T18:14:12.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:12 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T18:14:12.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:12 smithi114 ceph-mon[30897]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:12.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:12 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T18:14:12.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:12 smithi114 ceph-mon[30897]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-13T18:14:12.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:12 smithi114 ceph-mon[30897]: monmap e2: 2 mons at {a=[v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0],c=[v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-13T18:14:12.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:12 smithi114 ceph-mon[30897]: fsmap 2024-09-13T18:14:12.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:12 smithi114 ceph-mon[30897]: osdmap e4: 0 total, 0 up, 0 in 2024-09-13T18:14:12.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:12 smithi114 ceph-mon[30897]: mgrmap e13: x(active, since 46s) 2024-09-13T18:14:12.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:12 smithi114 ceph-mon[30897]: overall HEALTH_OK 2024-09-13T18:14:12.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:12 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:12.472 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:12 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:12.472 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:12 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:12.472 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:12 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-13T18:14:12.472 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:12 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:12.472 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:12 smithi114 ceph-mon[30897]: Deploying daemon mon.b on smithi134 2024-09-13T18:14:13.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:13 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T18:14:13.469 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:13 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T18:14:13.665 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-13T18:14:13.665 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-13T18:14:13.883 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T18:14:14.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:14 smithi204 ceph-mon[34777]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:14.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:14 smithi114 ceph-mon[30897]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:14.470 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:14:14 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:14:14.067+0000 7fb0e8d78640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-13T18:14:15.254 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:15.247+0000 7f9f25a5b640 1 -- 172.21.15.204:0/1590286132 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f20105890 msgr2=0x7f9f20105c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:15.254 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:15.247+0000 7f9f25a5b640 1 --2- 172.21.15.204:0/1590286132 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f20105890 0x7f9f20105c70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9f10009880 tx=0x7f9f1002a5d0 comp rx=0 tx=0).stop 2024-09-13T18:14:15.254 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:15.248+0000 7f9f25a5b640 1 -- 172.21.15.204:0/1590286132 shutdown_connections 2024-09-13T18:14:15.254 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:15.248+0000 7f9f25a5b640 1 --2- 172.21.15.204:0/1590286132 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f20105890 0x7f9f20105c70 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:15.254 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:15.248+0000 7f9f25a5b640 1 -- 172.21.15.204:0/1590286132 >> 172.21.15.204:0/1590286132 conn(0x7f9f200fb550 msgr2=0x7f9f200fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:14:15.254 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:15.248+0000 7f9f25a5b640 1 -- 172.21.15.204:0/1590286132 shutdown_connections 2024-09-13T18:14:15.255 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:15.248+0000 7f9f25a5b640 1 -- 172.21.15.204:0/1590286132 wait complete. 2024-09-13T18:14:15.255 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:15.248+0000 7f9f25a5b640 1 Processor -- start 2024-09-13T18:14:15.255 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:15.248+0000 7f9f25a5b640 1 -- start start 2024-09-13T18:14:15.255 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:15.248+0000 7f9f25a5b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f20105890 0x7f9f20195150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:15.255 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:15.248+0000 7f9f25a5b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f20195690 0x7f9f20199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:15.255 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:15.248+0000 7f9f25a5b640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f20195c00 con 0x7f9f20105890 2024-09-13T18:14:15.255 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:15.248+0000 7f9f25a5b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f20195d70 con 0x7f9f20195690 2024-09-13T18:14:15.255 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:15.249+0000 7f9f17fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f20195690 0x7f9f20199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:15.255 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:15.249+0000 7f9f17fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f20195690 0x7f9f20199a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:53206/0 (socket says 172.21.15.204:53206) 2024-09-13T18:14:15.255 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:15.249+0000 7f9f17fff640 1 -- 172.21.15.204:0/1238428250 learned_addr learned my addr 172.21.15.204:0/1238428250 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-13T18:14:15.255 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:15.249+0000 7f9f1effd640 1 --2- 172.21.15.204:0/1238428250 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f20105890 0x7f9f20195150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:15.255 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:15.249+0000 7f9f17fff640 1 -- 172.21.15.204:0/1238428250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f20195690 msgr2=0x7f9f20199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-13T18:14:15.255 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:15.249+0000 7f9f17fff640 1 -- 172.21.15.204:0/1238428250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f20195690 msgr2=0x7f9f20199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:14:15.256 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:15.249+0000 7f9f17fff640 1 --2- 172.21.15.204:0/1238428250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f20195690 0x7f9f20199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:14:15.256 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:15.249+0000 7f9f17fff640 1 --2- 172.21.15.204:0/1238428250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f20195690 0x7f9f20199a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:14:15.256 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:15.249+0000 7f9f1effd640 1 -- 172.21.15.204:0/1238428250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f20195690 msgr2=0x7f9f20199a90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:14:15.256 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:15.249+0000 7f9f1effd640 1 --2- 172.21.15.204:0/1238428250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f20195690 0x7f9f20199a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:15.256 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:15.249+0000 7f9f1effd640 1 -- 172.21.15.204:0/1238428250 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f10009560 con 0x7f9f20105890 2024-09-13T18:14:15.256 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:15.249+0000 7f9f1effd640 1 --2- 172.21.15.204:0/1238428250 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f20105890 0x7f9f20195150 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9f10006010 tx=0x7f9f100050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:15.256 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:15.249+0000 7f9f24a59640 1 -- 172.21.15.204:0/1238428250 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f9f100044d0 con 0x7f9f20105890 2024-09-13T18:14:15.256 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:15.249+0000 7f9f25a5b640 1 -- 172.21.15.204:0/1238428250 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f20199fd0 con 0x7f9f20105890 2024-09-13T18:14:15.256 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:15.249+0000 7f9f24a59640 1 -- 172.21.15.204:0/1238428250 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9f10005430 con 0x7f9f20105890 2024-09-13T18:14:15.256 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:15.249+0000 7f9f25a5b640 1 -- 172.21.15.204:0/1238428250 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f2019a520 con 0x7f9f20105890 2024-09-13T18:14:15.256 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:15.249+0000 7f9f24a59640 1 -- 172.21.15.204:0/1238428250 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f9f1003b4f0 con 0x7f9f20105890 2024-09-13T18:14:15.256 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:15.250+0000 7f9f24a59640 1 -- 172.21.15.204:0/1238428250 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7f9f100035f0 con 0x7f9f20105890 2024-09-13T18:14:15.256 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:15.250+0000 7f9f25a5b640 1 -- 172.21.15.204:0/1238428250 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f20100fd0 con 0x7f9f20105890 2024-09-13T18:14:15.257 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:15.250+0000 7f9f24a59640 1 --2- 172.21.15.204:0/1238428250 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f9f0c038550 0x7f9f0c03aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:15.257 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:15.251+0000 7f9f24a59640 1 -- 172.21.15.204:0/1238428250 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9f1006c3b0 con 0x7f9f20105890 2024-09-13T18:14:15.257 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:15.251+0000 7f9f17fff640 1 --2- 172.21.15.204:0/1238428250 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f9f0c038550 0x7f9f0c03aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:15.257 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:15.251+0000 7f9f17fff640 1 --2- 172.21.15.204:0/1238428250 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f9f0c038550 0x7f9f0c03aa10 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9f08009100 tx=0x7f9f08008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:15.258 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:15.256+0000 7f9f24a59640 1 -- 172.21.15.204:0/1238428250 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f9f10042540 con 0x7f9f20105890 2024-09-13T18:14:15.478 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:15.476+0000 7f9f25a5b640 1 -- 172.21.15.204:0/1238428250 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9f20105cf0 con 0x7f9f20105890 2024-09-13T18:14:15.479 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:15.477+0000 7f9f24a59640 1 -- 172.21.15.204:0/1238428250 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1010 (secure 0 0 0) 0x7f9f10042360 con 0x7f9f20105890 2024-09-13T18:14:15.480 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-13T18:14:15.480 INFO:teuthology.orchestra.run.smithi204.stdout:{"epoch":2,"fsid":"97e7edf2-71fb-11ef-bceb-c7b262605968","modified":"2024-09-13T18:14:07.066533Z","created":"2024-09-13T18:12:33.374504Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:3300","nonce":0},{"type":"v1","addr":"172.21.15.114:6789","nonce":0}]},"addr":"172.21.15.114:6789/0","public_addr":"172.21.15.114:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-13T18:14:15.480 INFO:teuthology.orchestra.run.smithi204.stderr:dumped monmap epoch 2 2024-09-13T18:14:15.482 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:15.479+0000 7f9f25a5b640 1 -- 172.21.15.204:0/1238428250 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f9f0c038550 msgr2=0x7f9f0c03aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:15.482 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:15.479+0000 7f9f25a5b640 1 --2- 172.21.15.204:0/1238428250 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f9f0c038550 0x7f9f0c03aa10 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9f08009100 tx=0x7f9f08008040 comp rx=0 tx=0).stop 2024-09-13T18:14:15.482 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:15.479+0000 7f9f25a5b640 1 -- 172.21.15.204:0/1238428250 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f20105890 msgr2=0x7f9f20195150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:15.482 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:15.479+0000 7f9f25a5b640 1 --2- 172.21.15.204:0/1238428250 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f20105890 0x7f9f20195150 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9f10006010 tx=0x7f9f100050b0 comp rx=0 tx=0).stop 2024-09-13T18:14:15.482 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:15.480+0000 7f9f25a5b640 1 -- 172.21.15.204:0/1238428250 shutdown_connections 2024-09-13T18:14:15.482 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:15.480+0000 7f9f25a5b640 1 --2- 172.21.15.204:0/1238428250 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f9f0c038550 0x7f9f0c03aa10 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:15.482 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:15.480+0000 7f9f25a5b640 1 --2- 172.21.15.204:0/1238428250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f20195690 0x7f9f20199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:15.482 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:15.480+0000 7f9f25a5b640 1 --2- 172.21.15.204:0/1238428250 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f20105890 0x7f9f20195150 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:15.482 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:15.480+0000 7f9f25a5b640 1 -- 172.21.15.204:0/1238428250 >> 172.21.15.204:0/1238428250 conn(0x7f9f200fb550 msgr2=0x7f9f200fb930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:14:15.482 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:15.480+0000 7f9f25a5b640 1 -- 172.21.15.204:0/1238428250 shutdown_connections 2024-09-13T18:14:15.483 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:15.480+0000 7f9f25a5b640 1 -- 172.21.15.204:0/1238428250 wait complete. 2024-09-13T18:14:15.690 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: load: jerasure load: lrc 2024-09-13T18:14:15.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:15 smithi204 ceph-mon[34777]: from='client.? 172.21.15.204:0/1238428250' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-13T18:14:15.969 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:15 smithi114 ceph-mon[30897]: from='client.? 172.21.15.204:0/1238428250' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-13T18:14:16.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: RocksDB version: 7.9.2 2024-09-13T18:14:16.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Git sha 0 2024-09-13T18:14:16.120 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Compile date 2024-09-10 12:59:06 2024-09-13T18:14:16.120 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: DB SUMMARY 2024-09-13T18:14:16.120 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: DB Session ID: 01RILYUL8ZI2DAKS6FYZ 2024-09-13T18:14:16.120 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: CURRENT file: CURRENT 2024-09-13T18:14:16.120 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: IDENTITY file: IDENTITY 2024-09-13T18:14:16.120 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-13T18:14:16.120 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-13T18:14:16.120 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-09-13T18:14:16.120 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.error_if_exists: 0 2024-09-13T18:14:16.120 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.create_if_missing: 0 2024-09-13T18:14:16.120 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.paranoid_checks: 1 2024-09-13T18:14:16.120 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-13T18:14:16.121 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-13T18:14:16.121 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-13T18:14:16.121 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.env: 0x55920dbc2bc0 2024-09-13T18:14:16.121 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.fs: PosixFileSystem 2024-09-13T18:14:16.121 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.info_log: 0x55920fb72be0 2024-09-13T18:14:16.121 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.max_file_opening_threads: 16 2024-09-13T18:14:16.121 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.statistics: (nil) 2024-09-13T18:14:16.121 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.use_fsync: 0 2024-09-13T18:14:16.121 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.max_log_file_size: 0 2024-09-13T18:14:16.121 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-13T18:14:16.121 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-13T18:14:16.121 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.keep_log_file_num: 1000 2024-09-13T18:14:16.121 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.recycle_log_file_num: 0 2024-09-13T18:14:16.122 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.allow_fallocate: 1 2024-09-13T18:14:16.122 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.allow_mmap_reads: 0 2024-09-13T18:14:16.122 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.allow_mmap_writes: 0 2024-09-13T18:14:16.122 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.use_direct_reads: 0 2024-09-13T18:14:16.122 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-13T18:14:16.122 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.create_missing_column_families: 0 2024-09-13T18:14:16.122 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.db_log_dir: 2024-09-13T18:14:16.122 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.wal_dir: 2024-09-13T18:14:16.122 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-13T18:14:16.122 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-13T18:14:16.122 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-13T18:14:16.122 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-13T18:14:16.124 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-13T18:14:16.124 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-13T18:14:16.124 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.advise_random_on_open: 1 2024-09-13T18:14:16.124 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.db_write_buffer_size: 0 2024-09-13T18:14:16.124 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.write_buffer_manager: 0x55920fb80320 2024-09-13T18:14:16.124 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-13T18:14:16.124 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-13T18:14:16.124 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-13T18:14:16.125 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.rate_limiter: (nil) 2024-09-13T18:14:16.125 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-13T18:14:16.125 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.wal_recovery_mode: 2 2024-09-13T18:14:16.125 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.enable_thread_tracking: 0 2024-09-13T18:14:16.125 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.enable_pipelined_write: 0 2024-09-13T18:14:16.125 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.unordered_write: 0 2024-09-13T18:14:16.125 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-13T18:14:16.125 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-13T18:14:16.125 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-13T18:14:16.125 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-13T18:14:16.125 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.row_cache: None 2024-09-13T18:14:16.125 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.wal_filter: None 2024-09-13T18:14:16.125 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-13T18:14:16.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.allow_ingest_behind: 0 2024-09-13T18:14:16.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.two_write_queues: 0 2024-09-13T18:14:16.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.manual_wal_flush: 0 2024-09-13T18:14:16.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.wal_compression: 0 2024-09-13T18:14:16.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.atomic_flush: 0 2024-09-13T18:14:16.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-13T18:14:16.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-13T18:14:16.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-13T18:14:16.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.log_readahead_size: 0 2024-09-13T18:14:16.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-13T18:14:16.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.best_efforts_recovery: 0 2024-09-13T18:14:16.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-13T18:14:16.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-13T18:14:16.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.allow_data_in_errors: 0 2024-09-13T18:14:16.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.db_host_id: __hostname__ 2024-09-13T18:14:16.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-13T18:14:16.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.max_background_jobs: 2 2024-09-13T18:14:16.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.max_background_compactions: -1 2024-09-13T18:14:16.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.max_subcompactions: 1 2024-09-13T18:14:16.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-13T18:14:16.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-13T18:14:16.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-13T18:14:16.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.max_total_wal_size: 0 2024-09-13T18:14:16.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-13T18:14:16.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-13T18:14:16.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-13T18:14:16.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-13T18:14:16.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.max_open_files: -1 2024-09-13T18:14:16.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.bytes_per_sync: 0 2024-09-13T18:14:16.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-13T18:14:16.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-13T18:14:16.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.compaction_readahead_size: 0 2024-09-13T18:14:16.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.max_background_flushes: -1 2024-09-13T18:14:16.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Compression algorithms supported: 2024-09-13T18:14:16.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: kZSTD supported: 0 2024-09-13T18:14:16.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: kXpressCompression supported: 0 2024-09-13T18:14:16.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: kBZip2Compression supported: 0 2024-09-13T18:14:16.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-13T18:14:16.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: kLZ4Compression supported: 1 2024-09-13T18:14:16.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: kZlibCompression supported: 1 2024-09-13T18:14:16.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: kLZ4HCCompression supported: 1 2024-09-13T18:14:16.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: kSnappyCompression supported: 1 2024-09-13T18:14:16.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-13T18:14:16.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-13T18:14:16.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-09-13T18:14:16.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-13T18:14:16.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-13T18:14:16.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.merge_operator: 2024-09-13T18:14:16.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.compaction_filter: None 2024-09-13T18:14:16.130 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.compaction_filter_factory: None 2024-09-13T18:14:16.130 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.sst_partitioner_factory: None 2024-09-13T18:14:16.130 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-13T18:14:16.130 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-13T18:14:16.130 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55920fb727e0) 2024-09-13T18:14:16.130 INFO:journalctl@ceph.mon.b.smithi134.stdout: cache_index_and_filter_blocks: 1 2024-09-13T18:14:16.130 INFO:journalctl@ceph.mon.b.smithi134.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-13T18:14:16.130 INFO:journalctl@ceph.mon.b.smithi134.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-13T18:14:16.130 INFO:journalctl@ceph.mon.b.smithi134.stdout: pin_top_level_index_and_filter: 1 2024-09-13T18:14:16.130 INFO:journalctl@ceph.mon.b.smithi134.stdout: index_type: 0 2024-09-13T18:14:16.130 INFO:journalctl@ceph.mon.b.smithi134.stdout: data_block_index_type: 0 2024-09-13T18:14:16.130 INFO:journalctl@ceph.mon.b.smithi134.stdout: index_shortening: 1 2024-09-13T18:14:16.130 INFO:journalctl@ceph.mon.b.smithi134.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-13T18:14:16.130 INFO:journalctl@ceph.mon.b.smithi134.stdout: checksum: 4 2024-09-13T18:14:16.131 INFO:journalctl@ceph.mon.b.smithi134.stdout: no_block_cache: 0 2024-09-13T18:14:16.131 INFO:journalctl@ceph.mon.b.smithi134.stdout: block_cache: 0x55920fb671f0 2024-09-13T18:14:16.131 INFO:journalctl@ceph.mon.b.smithi134.stdout: block_cache_name: BinnedLRUCache 2024-09-13T18:14:16.131 INFO:journalctl@ceph.mon.b.smithi134.stdout: block_cache_options: 2024-09-13T18:14:16.131 INFO:journalctl@ceph.mon.b.smithi134.stdout: capacity : 536870912 2024-09-13T18:14:16.131 INFO:journalctl@ceph.mon.b.smithi134.stdout: num_shard_bits : 4 2024-09-13T18:14:16.131 INFO:journalctl@ceph.mon.b.smithi134.stdout: strict_capacity_limit : 0 2024-09-13T18:14:16.131 INFO:journalctl@ceph.mon.b.smithi134.stdout: high_pri_pool_ratio: 0.000 2024-09-13T18:14:16.131 INFO:journalctl@ceph.mon.b.smithi134.stdout: block_cache_compressed: (nil) 2024-09-13T18:14:16.131 INFO:journalctl@ceph.mon.b.smithi134.stdout: persistent_cache: (nil) 2024-09-13T18:14:16.131 INFO:journalctl@ceph.mon.b.smithi134.stdout: block_size: 4096 2024-09-13T18:14:16.131 INFO:journalctl@ceph.mon.b.smithi134.stdout: block_size_deviation: 10 2024-09-13T18:14:16.131 INFO:journalctl@ceph.mon.b.smithi134.stdout: block_restart_interval: 16 2024-09-13T18:14:16.132 INFO:journalctl@ceph.mon.b.smithi134.stdout: index_block_restart_interval: 1 2024-09-13T18:14:16.132 INFO:journalctl@ceph.mon.b.smithi134.stdout: metadata_block_size: 4096 2024-09-13T18:14:16.132 INFO:journalctl@ceph.mon.b.smithi134.stdout: partition_filters: 0 2024-09-13T18:14:16.132 INFO:journalctl@ceph.mon.b.smithi134.stdout: use_delta_encoding: 1 2024-09-13T18:14:16.132 INFO:journalctl@ceph.mon.b.smithi134.stdout: filter_policy: bloomfilter 2024-09-13T18:14:16.132 INFO:journalctl@ceph.mon.b.smithi134.stdout: whole_key_filtering: 1 2024-09-13T18:14:16.132 INFO:journalctl@ceph.mon.b.smithi134.stdout: verify_compression: 0 2024-09-13T18:14:16.132 INFO:journalctl@ceph.mon.b.smithi134.stdout: read_amp_bytes_per_bit: 0 2024-09-13T18:14:16.132 INFO:journalctl@ceph.mon.b.smithi134.stdout: format_version: 5 2024-09-13T18:14:16.132 INFO:journalctl@ceph.mon.b.smithi134.stdout: enable_index_compression: 1 2024-09-13T18:14:16.132 INFO:journalctl@ceph.mon.b.smithi134.stdout: block_align: 0 2024-09-13T18:14:16.132 INFO:journalctl@ceph.mon.b.smithi134.stdout: max_auto_readahead_size: 262144 2024-09-13T18:14:16.132 INFO:journalctl@ceph.mon.b.smithi134.stdout: prepopulate_block_cache: 0 2024-09-13T18:14:16.132 INFO:journalctl@ceph.mon.b.smithi134.stdout: initial_auto_readahead_size: 8192 2024-09-13T18:14:16.133 INFO:journalctl@ceph.mon.b.smithi134.stdout: num_file_reads_for_auto_readahead: 2 2024-09-13T18:14:16.133 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.write_buffer_size: 33554432 2024-09-13T18:14:16.133 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.max_write_buffer_number: 2 2024-09-13T18:14:16.133 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.compression: NoCompression 2024-09-13T18:14:16.133 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.bottommost_compression: Disabled 2024-09-13T18:14:16.133 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.prefix_extractor: nullptr 2024-09-13T18:14:16.133 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-13T18:14:16.133 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.num_levels: 7 2024-09-13T18:14:16.133 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-13T18:14:16.133 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-13T18:14:16.133 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-13T18:14:16.133 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-13T18:14:16.133 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-13T18:14:16.134 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-13T18:14:16.134 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-13T18:14:16.134 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-13T18:14:16.134 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-13T18:14:16.134 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-13T18:14:16.134 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-13T18:14:16.134 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-13T18:14:16.134 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-13T18:14:16.134 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.compression_opts.level: 32767 2024-09-13T18:14:16.134 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.compression_opts.strategy: 0 2024-09-13T18:14:16.134 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-13T18:14:16.134 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-13T18:14:16.134 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-13T18:14:16.134 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-13T18:14:16.135 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.compression_opts.enabled: false 2024-09-13T18:14:16.135 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-13T18:14:16.135 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-13T18:14:16.135 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-13T18:14:16.135 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-13T18:14:16.135 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.target_file_size_base: 67108864 2024-09-13T18:14:16.135 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-13T18:14:16.135 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-13T18:14:16.135 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-13T18:14:16.135 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-13T18:14:16.135 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-13T18:14:16.135 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-13T18:14:16.135 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-13T18:14:16.135 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-13T18:14:16.135 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-13T18:14:16.136 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-13T18:14:16.136 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-13T18:14:16.136 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-13T18:14:16.136 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-13T18:14:16.136 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-13T18:14:16.136 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.arena_block_size: 1048576 2024-09-13T18:14:16.136 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-13T18:14:16.136 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-13T18:14:16.136 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.disable_auto_compactions: 0 2024-09-13T18:14:16.136 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-13T18:14:16.136 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-13T18:14:16.136 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-13T18:14:16.136 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-13T18:14:16.137 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-13T18:14:16.137 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-13T18:14:16.137 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-13T18:14:16.137 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-13T18:14:16.137 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-13T18:14:16.137 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-13T18:14:16.137 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-13T18:14:16.137 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.inplace_update_support: 0 2024-09-13T18:14:16.137 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-13T18:14:16.137 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-13T18:14:16.137 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-13T18:14:16.137 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-13T18:14:16.137 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.bloom_locality: 0 2024-09-13T18:14:16.137 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.max_successive_merges: 0 2024-09-13T18:14:16.138 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-13T18:14:16.138 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.paranoid_file_checks: 0 2024-09-13T18:14:16.138 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.force_consistency_checks: 1 2024-09-13T18:14:16.138 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.report_bg_io_stats: 0 2024-09-13T18:14:16.138 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.ttl: 2592000 2024-09-13T18:14:16.138 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-13T18:14:16.138 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-13T18:14:16.138 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-13T18:14:16.138 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.enable_blob_files: false 2024-09-13T18:14:16.138 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.min_blob_size: 0 2024-09-13T18:14:16.138 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.blob_file_size: 268435456 2024-09-13T18:14:16.138 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-13T18:14:16.138 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-13T18:14:16.138 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-13T18:14:16.139 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-13T18:14:16.139 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-13T18:14:16.139 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.blob_file_starting_level: 0 2024-09-13T18:14:16.139 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-13T18:14:16.139 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-13T18:14:16.139 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-13T18:14:16.139 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 45d7afb7-262b-4847-8aa6-2885aa11de65 2024-09-13T18:14:16.139 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726251255690447, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-13T18:14:16.139 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-13T18:14:16.139 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726251255690873, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726251255, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "45d7afb7-262b-4847-8aa6-2885aa11de65", "db_session_id": "01RILYUL8ZI2DAKS6FYZ", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-13T18:14:16.139 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726251255690974, "job": 1, "event": "recovery_finished"} 2024-09-13T18:14:16.139 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-13T18:14:16.139 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-13T18:14:16.139 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55920fb92e00 2024-09-13T18:14:16.140 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: DB pointer 0x55920fc86000 2024-09-13T18:14:16.140 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-13T18:14:16.140 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-13T18:14:16.140 INFO:journalctl@ceph.mon.b.smithi134.stdout: ** DB Stats ** 2024-09-13T18:14:16.140 INFO:journalctl@ceph.mon.b.smithi134.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-13T18:14:16.140 INFO:journalctl@ceph.mon.b.smithi134.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-13T18:14:16.140 INFO:journalctl@ceph.mon.b.smithi134.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-13T18:14:16.140 INFO:journalctl@ceph.mon.b.smithi134.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-13T18:14:16.140 INFO:journalctl@ceph.mon.b.smithi134.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-13T18:14:16.140 INFO:journalctl@ceph.mon.b.smithi134.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-13T18:14:16.140 INFO:journalctl@ceph.mon.b.smithi134.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-13T18:14:16.140 INFO:journalctl@ceph.mon.b.smithi134.stdout: 2024-09-13T18:14:16.140 INFO:journalctl@ceph.mon.b.smithi134.stdout: ** Compaction Stats [default] ** 2024-09-13T18:14:16.140 INFO:journalctl@ceph.mon.b.smithi134.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-13T18:14:16.141 INFO:journalctl@ceph.mon.b.smithi134.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-13T18:14:16.141 INFO:journalctl@ceph.mon.b.smithi134.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-13T18:14:16.141 INFO:journalctl@ceph.mon.b.smithi134.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-13T18:14:16.141 INFO:journalctl@ceph.mon.b.smithi134.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-13T18:14:16.141 INFO:journalctl@ceph.mon.b.smithi134.stdout: 2024-09-13T18:14:16.141 INFO:journalctl@ceph.mon.b.smithi134.stdout: ** Compaction Stats [default] ** 2024-09-13T18:14:16.141 INFO:journalctl@ceph.mon.b.smithi134.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-13T18:14:16.141 INFO:journalctl@ceph.mon.b.smithi134.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-13T18:14:16.141 INFO:journalctl@ceph.mon.b.smithi134.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-13T18:14:16.141 INFO:journalctl@ceph.mon.b.smithi134.stdout: 2024-09-13T18:14:16.141 INFO:journalctl@ceph.mon.b.smithi134.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-13T18:14:16.141 INFO:journalctl@ceph.mon.b.smithi134.stdout: 2024-09-13T18:14:16.141 INFO:journalctl@ceph.mon.b.smithi134.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-13T18:14:16.142 INFO:journalctl@ceph.mon.b.smithi134.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-13T18:14:16.142 INFO:journalctl@ceph.mon.b.smithi134.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-13T18:14:16.142 INFO:journalctl@ceph.mon.b.smithi134.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-13T18:14:16.142 INFO:journalctl@ceph.mon.b.smithi134.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-13T18:14:16.142 INFO:journalctl@ceph.mon.b.smithi134.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-13T18:14:16.142 INFO:journalctl@ceph.mon.b.smithi134.stdout: Cumulative compaction: 0.00 GB write, 0.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-13T18:14:16.142 INFO:journalctl@ceph.mon.b.smithi134.stdout: Interval compaction: 0.00 GB write, 0.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-13T18:14:16.142 INFO:journalctl@ceph.mon.b.smithi134.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-13T18:14:16.142 INFO:journalctl@ceph.mon.b.smithi134.stdout: Block cache BinnedLRUCache@0x55920fb671f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-13T18:14:16.142 INFO:journalctl@ceph.mon.b.smithi134.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-13T18:14:16.142 INFO:journalctl@ceph.mon.b.smithi134.stdout: 2024-09-13T18:14:16.142 INFO:journalctl@ceph.mon.b.smithi134.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-13T18:14:16.142 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-13T18:14:16.143 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: using public_addr v2:172.21.15.134:0/0 -> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-13T18:14:16.143 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: starting mon.b rank -1 at public addrs [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] at bind addrs [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 2024-09-13T18:14:16.143 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: mon.b@-1(???) e0 preinit fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 2024-09-13T18:14:16.143 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: mon.b@-1(synchronizing).mds e1 new map 2024-09-13T18:14:16.143 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-13T18:14:16.143 INFO:journalctl@ceph.mon.b.smithi134.stdout: e1 2024-09-13T18:14:16.143 INFO:journalctl@ceph.mon.b.smithi134.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-13T18:14:16.143 INFO:journalctl@ceph.mon.b.smithi134.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-13T18:14:16.143 INFO:journalctl@ceph.mon.b.smithi134.stdout: legacy client fscid: -1 2024-09-13T18:14:16.143 INFO:journalctl@ceph.mon.b.smithi134.stdout: 2024-09-13T18:14:16.143 INFO:journalctl@ceph.mon.b.smithi134.stdout: No filesystems configured 2024-09-13T18:14:16.143 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-13T18:14:16.143 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-13T18:14:16.143 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-13T18:14:16.144 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-13T18:14:16.144 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-13T18:14:16.144 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-13T18:14:16.144 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-13T18:14:16.144 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-13T18:14:16.144 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-13T18:14:16.144 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-13T18:14:16.144 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi204", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:14:16.144 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:16.144 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: Deploying cephadm binary to smithi204 2024-09-13T18:14:16.144 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:16.144 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:16.144 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:14:16.145 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:16.145 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:16.145 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: Added host smithi204 2024-09-13T18:14:16.145 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:16.145 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:16.145 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T18:14:16.145 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:16.145 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:16.145 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:16.145 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:16.145 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:16.145 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi204", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:14:16.145 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:16.145 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:14:16.146 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: Updating smithi204:/etc/ceph/ceph.conf 2024-09-13T18:14:16.146 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:16.146 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:16.146 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:16.146 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: Updating smithi204:/var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T18:14:16.146 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-09-13T18:14:16.146 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: Updating smithi204:/var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-13T18:14:16.146 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/910298425' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-13T18:14:16.146 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:16.146 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/910298425' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-13T18:14:16.146 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: osdmap e4: 0 total, 0 up, 0 in 2024-09-13T18:14:16.146 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:16.146 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:16.146 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:14:16.147 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:16.147 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:14:16.147 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:16.147 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-13T18:14:16.147 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:16.147 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi114:172.21.15.114=a;smithi134:172.21.15.134=b;smithi204:172.21.15.204=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:14:16.147 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: Saving service mon spec with placement smithi114:172.21.15.114=a;smithi134:172.21.15.134=b;smithi204:172.21.15.204=c;count:3 2024-09-13T18:14:16.147 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: Deploying daemon mon.c on smithi204 2024-09-13T18:14:16.147 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:16.147 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T18:14:16.147 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: mon.a calling monitor election 2024-09-13T18:14:16.147 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T18:14:16.147 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: from='client.? 172.21.15.204:0/2602058883' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-13T18:14:16.148 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:16.148 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T18:14:16.148 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T18:14:16.148 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: mon.c calling monitor election 2024-09-13T18:14:16.148 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:16.148 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T18:14:16.148 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T18:14:16.148 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:16.148 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T18:14:16.148 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-13T18:14:16.148 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: monmap e2: 2 mons at {a=[v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0],c=[v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-13T18:14:16.148 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: fsmap 2024-09-13T18:14:16.148 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: osdmap e4: 0 total, 0 up, 0 in 2024-09-13T18:14:16.148 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: mgrmap e13: x(active, since 46s) 2024-09-13T18:14:16.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: overall HEALTH_OK 2024-09-13T18:14:16.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:16.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:16.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:16.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-13T18:14:16.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:16.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: Deploying daemon mon.b on smithi134 2024-09-13T18:14:16.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T18:14:16.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:16.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: from='client.? 172.21.15.204:0/1238428250' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-13T18:14:16.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 ceph-mon[34283]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-13T18:14:16.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 bash[34264]: ba2043c577c27758566f2427edc1c77e6dfc8196941be0adfffb0d4563a22d83 2024-09-13T18:14:16.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:15 smithi134 systemd[1]: Started Ceph mon.b for 97e7edf2-71fb-11ef-bceb-c7b262605968. 2024-09-13T18:14:17.055 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-13T18:14:17.055 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-13T18:14:17.261 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T18:14:21.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:20 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T18:14:21.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:20 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T18:14:21.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:20 smithi204 ceph-mon[34777]: mon.a calling monitor election 2024-09-13T18:14:21.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:20 smithi204 ceph-mon[34777]: mon.c calling monitor election 2024-09-13T18:14:21.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:20 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T18:14:21.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:20 smithi204 ceph-mon[34777]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:21.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:20 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T18:14:21.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:20 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T18:14:21.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:20 smithi204 ceph-mon[34777]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:21.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:20 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T18:14:21.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:20 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T18:14:21.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:20 smithi204 ceph-mon[34777]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:21.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:20 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T18:14:21.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:20 smithi204 ceph-mon[34777]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-13T18:14:21.196 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:20 smithi204 ceph-mon[34777]: monmap e3: 3 mons at {a=[v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0],b=[v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0],c=[v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-13T18:14:21.196 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:20 smithi204 ceph-mon[34777]: fsmap 2024-09-13T18:14:21.196 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:20 smithi204 ceph-mon[34777]: osdmap e4: 0 total, 0 up, 0 in 2024-09-13T18:14:21.196 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:20 smithi204 ceph-mon[34777]: mgrmap e13: x(active, since 54s) 2024-09-13T18:14:21.196 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:20 smithi204 ceph-mon[34777]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-09-13T18:14:21.196 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:20 smithi204 ceph-mon[34777]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-09-13T18:14:21.196 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:20 smithi204 ceph-mon[34777]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-09-13T18:14:21.196 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:20 smithi204 ceph-mon[34777]: mon.b (rank 2) addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] is down (out of quorum) 2024-09-13T18:14:21.196 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:20 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:21.196 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:20 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:21.196 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:20 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:21.196 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:20 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:21.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:20 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T18:14:21.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:20 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T18:14:21.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:20 smithi114 ceph-mon[30897]: mon.a calling monitor election 2024-09-13T18:14:21.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:20 smithi114 ceph-mon[30897]: mon.c calling monitor election 2024-09-13T18:14:21.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:20 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T18:14:21.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:20 smithi114 ceph-mon[30897]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:21.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:20 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T18:14:21.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:20 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T18:14:21.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:20 smithi114 ceph-mon[30897]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:21.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:20 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T18:14:21.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:20 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T18:14:21.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:20 smithi114 ceph-mon[30897]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:21.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:20 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T18:14:21.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:20 smithi114 ceph-mon[30897]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-13T18:14:21.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:20 smithi114 ceph-mon[30897]: monmap e3: 3 mons at {a=[v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0],b=[v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0],c=[v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-13T18:14:21.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:20 smithi114 ceph-mon[30897]: fsmap 2024-09-13T18:14:21.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:20 smithi114 ceph-mon[30897]: osdmap e4: 0 total, 0 up, 0 in 2024-09-13T18:14:21.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:20 smithi114 ceph-mon[30897]: mgrmap e13: x(active, since 54s) 2024-09-13T18:14:21.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:20 smithi114 ceph-mon[30897]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-09-13T18:14:21.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:20 smithi114 ceph-mon[30897]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-09-13T18:14:21.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:20 smithi114 ceph-mon[30897]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-09-13T18:14:21.222 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:20 smithi114 ceph-mon[30897]: mon.b (rank 2) addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] is down (out of quorum) 2024-09-13T18:14:21.222 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:20 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:21.222 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:20 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:21.222 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:20 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:21.222 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:20 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:21.734 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.732+0000 7f9a6365c640 1 -- 172.21.15.204:0/515534235 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a48003530 msgr2=0x7f9a480059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:21.734 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.732+0000 7f9a6365c640 1 --2- 172.21.15.204:0/515534235 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a48003530 0x7f9a480059c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c100d40 tx=0x7f9a5802bce0 comp rx=0 tx=0).stop 2024-09-13T18:14:21.734 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.732+0000 7f9a6365c640 1 -- 172.21.15.204:0/515534235 shutdown_connections 2024-09-13T18:14:21.735 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.732+0000 7f9a6365c640 1 --2- 172.21.15.204:0/515534235 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a48003530 0x7f9a480059c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:21.735 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.732+0000 7f9a6365c640 1 -- 172.21.15.204:0/515534235 >> 172.21.15.204:0/515534235 conn(0x7f9a5c0fb590 msgr2=0x7f9a5c0fd9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:14:21.735 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.732+0000 7f9a6365c640 1 -- 172.21.15.204:0/515534235 shutdown_connections 2024-09-13T18:14:21.735 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.732+0000 7f9a6365c640 1 -- 172.21.15.204:0/515534235 wait complete. 2024-09-13T18:14:21.737 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.732+0000 7f9a6365c640 1 Processor -- start 2024-09-13T18:14:21.737 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.733+0000 7f9a6365c640 1 -- start start 2024-09-13T18:14:21.738 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.733+0000 7f9a6365c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a48003530 0x7f9a5c197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:21.738 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.733+0000 7f9a6365c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a5c1978c0 0x7f9a5c19bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:21.738 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.733+0000 7f9a6365c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a5c19c200 0x7f9a5c19c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:21.738 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.733+0000 7f9a6365c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5c197f90 con 0x7f9a5c19c200 2024-09-13T18:14:21.738 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.733+0000 7f9a6365c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5c1a3f70 con 0x7f9a5c1978c0 2024-09-13T18:14:21.738 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.733+0000 7f9a6365c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5c1a40b0 con 0x7f9a48003530 2024-09-13T18:14:21.738 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.733+0000 7f9a613f8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a48003530 0x7f9a5c197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:21.738 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.733+0000 7f9a613f8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a48003530 0x7f9a5c197380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:39078/0 (socket says 172.21.15.204:39078) 2024-09-13T18:14:21.738 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.733+0000 7f9a613f8640 1 -- 172.21.15.204:0/490773752 learned_addr learned my addr 172.21.15.204:0/490773752 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-13T18:14:21.738 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.733+0000 7f9a60bf7640 1 --2- 172.21.15.204:0/490773752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a5c1978c0 0x7f9a5c19bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:21.738 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.733+0000 7f9a613f8640 1 -- 172.21.15.204:0/490773752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a48003530 msgr2=0x7f9a5c197380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-13T18:14:21.738 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.733+0000 7f9a613f8640 1 -- 172.21.15.204:0/490773752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a48003530 msgr2=0x7f9a5c197380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:14:21.739 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.733+0000 7f9a613f8640 1 --2- 172.21.15.204:0/490773752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a48003530 0x7f9a5c197380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:14:21.739 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.733+0000 7f9a61bf9640 1 --2- 172.21.15.204:0/490773752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a5c19c200 0x7f9a5c19c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:21.739 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.733+0000 7f9a613f8640 1 --2- 172.21.15.204:0/490773752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a48003530 0x7f9a5c197380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:14:21.739 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.733+0000 7f9a60bf7640 1 -- 172.21.15.204:0/490773752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a5c1978c0 msgr2=0x7f9a5c19bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-13T18:14:21.739 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.733+0000 7f9a60bf7640 1 -- 172.21.15.204:0/490773752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a5c1978c0 msgr2=0x7f9a5c19bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:14:21.739 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.733+0000 7f9a60bf7640 1 --2- 172.21.15.204:0/490773752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a5c1978c0 0x7f9a5c19bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:14:21.739 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.733+0000 7f9a60bf7640 1 --2- 172.21.15.204:0/490773752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a5c1978c0 0x7f9a5c19bcc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:14:21.739 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.733+0000 7f9a61bf9640 1 -- 172.21.15.204:0/490773752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a48003530 msgr2=0x7f9a5c197380 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:14:21.739 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.733+0000 7f9a61bf9640 1 --2- 172.21.15.204:0/490773752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a48003530 0x7f9a5c197380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:21.739 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.733+0000 7f9a61bf9640 1 -- 172.21.15.204:0/490773752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a5c1978c0 msgr2=0x7f9a5c19bcc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:14:21.739 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.733+0000 7f9a61bf9640 1 --2- 172.21.15.204:0/490773752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a5c1978c0 0x7f9a5c19bcc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:21.739 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.733+0000 7f9a61bf9640 1 -- 172.21.15.204:0/490773752 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a58011870 con 0x7f9a5c19c200 2024-09-13T18:14:21.740 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.734+0000 7f9a61bf9640 1 --2- 172.21.15.204:0/490773752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a5c19c200 0x7f9a5c19c6b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9a5400bd80 tx=0x7f9a5400be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:21.740 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.734+0000 7f9a527fc640 1 -- 172.21.15.204:0/490773752 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a54014440 con 0x7f9a5c19c200 2024-09-13T18:14:21.740 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.734+0000 7f9a527fc640 1 -- 172.21.15.204:0/490773752 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9a54014a30 con 0x7f9a5c19c200 2024-09-13T18:14:21.740 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.734+0000 7f9a527fc640 1 -- 172.21.15.204:0/490773752 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a54012910 con 0x7f9a5c19c200 2024-09-13T18:14:21.740 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.734+0000 7f9a6365c640 1 -- 172.21.15.204:0/490773752 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a5c1a4360 con 0x7f9a5c19c200 2024-09-13T18:14:21.740 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.734+0000 7f9a6365c640 1 -- 172.21.15.204:0/490773752 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a5c1a47e0 con 0x7f9a5c19c200 2024-09-13T18:14:21.740 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.735+0000 7f9a527fc640 1 -- 172.21.15.204:0/490773752 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7f9a54014ba0 con 0x7f9a5c19c200 2024-09-13T18:14:21.740 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.735+0000 7f9a6365c640 1 -- 172.21.15.204:0/490773752 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a5c100ff0 con 0x7f9a5c19c200 2024-09-13T18:14:21.740 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.735+0000 7f9a527fc640 1 --2- 172.21.15.204:0/490773752 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f9a40038b90 0x7f9a4003b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:21.740 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.735+0000 7f9a527fc640 1 -- 172.21.15.204:0/490773752 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9a5404b080 con 0x7f9a5c19c200 2024-09-13T18:14:21.740 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.735+0000 7f9a613f8640 1 --2- 172.21.15.204:0/490773752 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f9a40038b90 0x7f9a4003b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:21.741 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.736+0000 7f9a613f8640 1 --2- 172.21.15.204:0/490773752 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f9a40038b90 0x7f9a4003b050 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9a58005c40 tx=0x7f9a58004050 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:21.742 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.740+0000 7f9a527fc640 1 -- 172.21.15.204:0/490773752 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f9a5407ee20 con 0x7f9a5c19c200 2024-09-13T18:14:21.977 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.974+0000 7f9a6365c640 1 -- 172.21.15.204:0/490773752 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9a5c104b70 con 0x7f9a5c19c200 2024-09-13T18:14:21.977 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.975+0000 7f9a527fc640 1 -- 172.21.15.204:0/490773752 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1278 (secure 0 0 0) 0x7f9a5401b8d0 con 0x7f9a5c19c200 2024-09-13T18:14:21.978 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-13T18:14:21.978 INFO:teuthology.orchestra.run.smithi204.stdout:{"epoch":3,"fsid":"97e7edf2-71fb-11ef-bceb-c7b262605968","modified":"2024-09-13T18:14:15.715955Z","created":"2024-09-13T18:12:33.374504Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:3300","nonce":0},{"type":"v1","addr":"172.21.15.114:6789","nonce":0}]},"addr":"172.21.15.114:6789/0","public_addr":"172.21.15.114:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-13T18:14:21.978 INFO:teuthology.orchestra.run.smithi204.stderr:dumped monmap epoch 3 2024-09-13T18:14:21.979 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.977+0000 7f9a6365c640 1 -- 172.21.15.204:0/490773752 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f9a40038b90 msgr2=0x7f9a4003b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:21.979 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.977+0000 7f9a6365c640 1 --2- 172.21.15.204:0/490773752 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f9a40038b90 0x7f9a4003b050 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9a58005c40 tx=0x7f9a58004050 comp rx=0 tx=0).stop 2024-09-13T18:14:21.980 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.977+0000 7f9a6365c640 1 -- 172.21.15.204:0/490773752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a5c19c200 msgr2=0x7f9a5c19c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:21.980 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.977+0000 7f9a6365c640 1 --2- 172.21.15.204:0/490773752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a5c19c200 0x7f9a5c19c6b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9a5400bd80 tx=0x7f9a5400be70 comp rx=0 tx=0).stop 2024-09-13T18:14:21.980 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.977+0000 7f9a6365c640 1 -- 172.21.15.204:0/490773752 shutdown_connections 2024-09-13T18:14:21.980 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.977+0000 7f9a6365c640 1 --2- 172.21.15.204:0/490773752 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f9a40038b90 0x7f9a4003b050 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:21.980 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.977+0000 7f9a6365c640 1 --2- 172.21.15.204:0/490773752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a5c19c200 0x7f9a5c19c6b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:21.980 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.977+0000 7f9a6365c640 1 --2- 172.21.15.204:0/490773752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a5c1978c0 0x7f9a5c19bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:21.980 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.977+0000 7f9a6365c640 1 --2- 172.21.15.204:0/490773752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a48003530 0x7f9a5c197380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:21.980 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.977+0000 7f9a6365c640 1 -- 172.21.15.204:0/490773752 >> 172.21.15.204:0/490773752 conn(0x7f9a5c0fb590 msgr2=0x7f9a5c073ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:14:21.980 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.977+0000 7f9a6365c640 1 -- 172.21.15.204:0/490773752 shutdown_connections 2024-09-13T18:14:21.980 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:21.977+0000 7f9a6365c640 1 -- 172.21.15.204:0/490773752 wait complete. 2024-09-13T18:14:21.989 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:21 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:21.989 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:21 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:14:21.989 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:21 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T18:14:21.989 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:21 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:21.990 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:21 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:21.990 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:21 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:21.990 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:21 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:21.990 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:21 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:21.990 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:21 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:14:22.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:21 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:22.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:21 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:14:22.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:21 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T18:14:22.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:21 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:22.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:21 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:22.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:21 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:22.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:21 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:22.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:21 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:22.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:21 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:14:22.638 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-13T18:14:22.639 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph config generate-minimal-conf 2024-09-13T18:14:23.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:22 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T18:14:23.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:22 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T18:14:23.120 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:22 smithi134 ceph-mon[34283]: mon.a calling monitor election 2024-09-13T18:14:23.120 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:22 smithi134 ceph-mon[34283]: mon.c calling monitor election 2024-09-13T18:14:23.120 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:22 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T18:14:23.120 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:22 smithi134 ceph-mon[34283]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:23.120 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:22 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T18:14:23.120 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:22 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T18:14:23.120 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:22 smithi134 ceph-mon[34283]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:23.120 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:22 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T18:14:23.120 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:22 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T18:14:23.120 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:22 smithi134 ceph-mon[34283]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:23.120 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:22 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T18:14:23.120 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:22 smithi134 ceph-mon[34283]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-13T18:14:23.120 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:22 smithi134 ceph-mon[34283]: monmap e3: 3 mons at {a=[v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0],b=[v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0],c=[v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-13T18:14:23.121 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:22 smithi134 ceph-mon[34283]: fsmap 2024-09-13T18:14:23.121 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:22 smithi134 ceph-mon[34283]: osdmap e4: 0 total, 0 up, 0 in 2024-09-13T18:14:23.121 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:22 smithi134 ceph-mon[34283]: mgrmap e13: x(active, since 54s) 2024-09-13T18:14:23.121 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:22 smithi134 ceph-mon[34283]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-09-13T18:14:23.121 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:22 smithi134 ceph-mon[34283]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-09-13T18:14:23.121 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:22 smithi134 ceph-mon[34283]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-09-13T18:14:23.121 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:22 smithi134 ceph-mon[34283]: mon.b (rank 2) addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] is down (out of quorum) 2024-09-13T18:14:23.121 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:22 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:23.121 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:22 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:23.121 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:22 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:23.121 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:22 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:23.121 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:22 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:23.121 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:22 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:14:23.121 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:22 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T18:14:23.122 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:22 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:23.122 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:22 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:23.122 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:22 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:23.122 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:22 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:23.122 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:22 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:23.122 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:22 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:14:23.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:23 smithi204 ceph-mon[34777]: mon.b calling monitor election 2024-09-13T18:14:23.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:23 smithi204 ceph-mon[34777]: Reconfiguring mon.a (unknown last config time)... 2024-09-13T18:14:23.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:23 smithi204 ceph-mon[34777]: Reconfiguring daemon mon.a on smithi114 2024-09-13T18:14:23.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:23 smithi204 ceph-mon[34777]: mon.b calling monitor election 2024-09-13T18:14:23.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:23 smithi204 ceph-mon[34777]: mon.c calling monitor election 2024-09-13T18:14:23.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:23 smithi204 ceph-mon[34777]: mon.a calling monitor election 2024-09-13T18:14:23.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:23 smithi204 ceph-mon[34777]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-13T18:14:23.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:23 smithi204 ceph-mon[34777]: monmap e3: 3 mons at {a=[v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0],b=[v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0],c=[v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-13T18:14:23.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:23 smithi204 ceph-mon[34777]: fsmap 2024-09-13T18:14:23.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:23 smithi204 ceph-mon[34777]: osdmap e4: 0 total, 0 up, 0 in 2024-09-13T18:14:23.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:23 smithi204 ceph-mon[34777]: mgrmap e13: x(active, since 56s) 2024-09-13T18:14:23.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:23 smithi204 ceph-mon[34777]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-09-13T18:14:23.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:23 smithi204 ceph-mon[34777]: Cluster is now healthy 2024-09-13T18:14:23.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:23 smithi204 ceph-mon[34777]: overall HEALTH_OK 2024-09-13T18:14:23.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:23 smithi114 ceph-mon[30897]: mon.b calling monitor election 2024-09-13T18:14:23.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:23 smithi114 ceph-mon[30897]: Reconfiguring mon.a (unknown last config time)... 2024-09-13T18:14:23.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:23 smithi114 ceph-mon[30897]: Reconfiguring daemon mon.a on smithi114 2024-09-13T18:14:23.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:23 smithi114 ceph-mon[30897]: mon.b calling monitor election 2024-09-13T18:14:23.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:23 smithi114 ceph-mon[30897]: mon.c calling monitor election 2024-09-13T18:14:23.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:23 smithi114 ceph-mon[30897]: mon.a calling monitor election 2024-09-13T18:14:23.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:23 smithi114 ceph-mon[30897]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-13T18:14:23.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:23 smithi114 ceph-mon[30897]: monmap e3: 3 mons at {a=[v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0],b=[v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0],c=[v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-13T18:14:23.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:23 smithi114 ceph-mon[30897]: fsmap 2024-09-13T18:14:23.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:23 smithi114 ceph-mon[30897]: osdmap e4: 0 total, 0 up, 0 in 2024-09-13T18:14:23.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:23 smithi114 ceph-mon[30897]: mgrmap e13: x(active, since 56s) 2024-09-13T18:14:23.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:23 smithi114 ceph-mon[30897]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-09-13T18:14:23.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:23 smithi114 ceph-mon[30897]: Cluster is now healthy 2024-09-13T18:14:23.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:23 smithi114 ceph-mon[30897]: overall HEALTH_OK 2024-09-13T18:14:23.786 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:14:23.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:23 smithi134 ceph-mon[34283]: mon.b calling monitor election 2024-09-13T18:14:23.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:23 smithi134 ceph-mon[34283]: Reconfiguring mon.a (unknown last config time)... 2024-09-13T18:14:23.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:23 smithi134 ceph-mon[34283]: Reconfiguring daemon mon.a on smithi114 2024-09-13T18:14:23.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:23 smithi134 ceph-mon[34283]: mon.b calling monitor election 2024-09-13T18:14:23.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:23 smithi134 ceph-mon[34283]: mon.c calling monitor election 2024-09-13T18:14:23.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:23 smithi134 ceph-mon[34283]: mon.a calling monitor election 2024-09-13T18:14:23.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:23 smithi134 ceph-mon[34283]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-13T18:14:23.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:23 smithi134 ceph-mon[34283]: monmap e3: 3 mons at {a=[v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0],b=[v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0],c=[v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-13T18:14:23.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:23 smithi134 ceph-mon[34283]: fsmap 2024-09-13T18:14:23.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:23 smithi134 ceph-mon[34283]: osdmap e4: 0 total, 0 up, 0 in 2024-09-13T18:14:23.870 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:23 smithi134 ceph-mon[34283]: mgrmap e13: x(active, since 56s) 2024-09-13T18:14:23.870 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:23 smithi134 ceph-mon[34283]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-09-13T18:14:23.870 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:23 smithi134 ceph-mon[34283]: Cluster is now healthy 2024-09-13T18:14:23.870 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:23 smithi134 ceph-mon[34283]: overall HEALTH_OK 2024-09-13T18:14:24.924 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:24 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:24.924 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:24 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:24.925 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:24 smithi114 ceph-mon[30897]: Reconfiguring mon.b (monmap changed)... 2024-09-13T18:14:24.925 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:24 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-13T18:14:24.925 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:24 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-13T18:14:24.925 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:24 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:24.925 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:24 smithi114 ceph-mon[30897]: Reconfiguring daemon mon.b on smithi134 2024-09-13T18:14:24.925 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:24 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T18:14:24.925 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:24 smithi114 ceph-mon[30897]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:25.045 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:24 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:25.045 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:24 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:25.045 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:24 smithi134 ceph-mon[34283]: Reconfiguring mon.b (monmap changed)... 2024-09-13T18:14:25.045 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:24 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-13T18:14:25.045 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:24 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-13T18:14:25.045 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:24 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:25.045 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:24 smithi134 ceph-mon[34283]: Reconfiguring daemon mon.b on smithi134 2024-09-13T18:14:25.045 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:24 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T18:14:25.045 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:24 smithi134 ceph-mon[34283]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:25.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.132+0000 7fa518810640 1 -- 172.21.15.114:0/2370330618 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa5140ff540 msgr2=0x7fa514109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:25.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.132+0000 7fa518810640 1 --2- 172.21.15.114:0/2370330618 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa5140ff540 0x7fa514109cb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc009900 tx=0x7fa4fc02a410 comp rx=0 tx=0).stop 2024-09-13T18:14:25.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.132+0000 7fa518810640 1 -- 172.21.15.114:0/2370330618 shutdown_connections 2024-09-13T18:14:25.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.132+0000 7fa518810640 1 --2- 172.21.15.114:0/2370330618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa51410a3a0 0x7fa51410c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:25.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.132+0000 7fa518810640 1 --2- 172.21.15.114:0/2370330618 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa5140ff540 0x7fa514109cb0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:25.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.132+0000 7fa518810640 1 --2- 172.21.15.114:0/2370330618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5140feb90 0x7fa5140fef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:25.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.132+0000 7fa518810640 1 -- 172.21.15.114:0/2370330618 >> 172.21.15.114:0/2370330618 conn(0x7fa514074130 msgr2=0x7fa514074540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:14:25.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.132+0000 7fa518810640 1 -- 172.21.15.114:0/2370330618 shutdown_connections 2024-09-13T18:14:25.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.132+0000 7fa518810640 1 -- 172.21.15.114:0/2370330618 wait complete. 2024-09-13T18:14:25.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.133+0000 7fa518810640 1 Processor -- start 2024-09-13T18:14:25.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.133+0000 7fa518810640 1 -- start start 2024-09-13T18:14:25.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.133+0000 7fa518810640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5140feb90 0x7fa514196220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:25.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.133+0000 7fa518810640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa5140ff540 0x7fa514196760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:25.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.133+0000 7fa518810640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa51410a3a0 0x7fa51419ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:25.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.133+0000 7fa518810640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51419b160 con 0x7fa5140ff540 2024-09-13T18:14:25.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.133+0000 7fa518810640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51419b2d0 con 0x7fa5140feb90 2024-09-13T18:14:25.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.133+0000 7fa518810640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51419b440 con 0x7fa51410a3a0 2024-09-13T18:14:25.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.133+0000 7fa509d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa5140ff540 0x7fa514196760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:25.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.133+0000 7fa509d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa5140ff540 0x7fa514196760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59278/0 (socket says 172.21.15.114:59278) 2024-09-13T18:14:25.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.133+0000 7fa509d9b640 1 -- 172.21.15.114:0/23587286 learned_addr learned my addr 172.21.15.114:0/23587286 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:14:25.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.133+0000 7fa51259c640 1 --2- 172.21.15.114:0/23587286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5140feb90 0x7fa514196220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:25.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.133+0000 7fa512d9d640 1 --2- 172.21.15.114:0/23587286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa51410a3a0 0x7fa51419ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:25.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.134+0000 7fa509d9b640 1 -- 172.21.15.114:0/23587286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa51410a3a0 msgr2=0x7fa51419ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:25.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.134+0000 7fa509d9b640 1 --2- 172.21.15.114:0/23587286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa51410a3a0 0x7fa51419ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:25.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.134+0000 7fa509d9b640 1 -- 172.21.15.114:0/23587286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5140feb90 msgr2=0x7fa514196220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:25.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.134+0000 7fa509d9b640 1 --2- 172.21.15.114:0/23587286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5140feb90 0x7fa514196220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:25.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.134+0000 7fa509d9b640 1 -- 172.21.15.114:0/23587286 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4fc009560 con 0x7fa5140ff540 2024-09-13T18:14:25.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.134+0000 7fa51259c640 1 --2- 172.21.15.114:0/23587286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5140feb90 0x7fa514196220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:14:25.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.134+0000 7fa509d9b640 1 --2- 172.21.15.114:0/23587286 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa5140ff540 0x7fa514196760 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc005ab0 tx=0x7fa4fc0329f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:25.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.134+0000 7fa50bfff640 1 -- 172.21.15.114:0/23587286 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc032df0 con 0x7fa5140ff540 2024-09-13T18:14:25.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.134+0000 7fa518810640 1 -- 172.21.15.114:0/23587286 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa51419b6c0 con 0x7fa5140ff540 2024-09-13T18:14:25.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.134+0000 7fa50bfff640 1 -- 172.21.15.114:0/23587286 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa4fc005490 con 0x7fa5140ff540 2024-09-13T18:14:25.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.134+0000 7fa50bfff640 1 -- 172.21.15.114:0/23587286 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc0035f0 con 0x7fa5140ff540 2024-09-13T18:14:25.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.134+0000 7fa518810640 1 -- 172.21.15.114:0/23587286 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa51419b9e0 con 0x7fa5140ff540 2024-09-13T18:14:25.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.135+0000 7fa50bfff640 1 -- 172.21.15.114:0/23587286 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7fa4fc0037f0 con 0x7fa5140ff540 2024-09-13T18:14:25.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.135+0000 7fa50bfff640 1 --2- 172.21.15.114:0/23587286 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fa500038930 0x7fa50003adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:25.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.135+0000 7fa50bfff640 1 -- 172.21.15.114:0/23587286 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa4fc06bea0 con 0x7fa5140ff540 2024-09-13T18:14:25.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.135+0000 7fa518810640 1 -- 172.21.15.114:0/23587286 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5140680a0 con 0x7fa5140ff540 2024-09-13T18:14:25.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.135+0000 7fa51259c640 1 --2- 172.21.15.114:0/23587286 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fa500038930 0x7fa50003adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:25.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.136+0000 7fa51259c640 1 --2- 172.21.15.114:0/23587286 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fa500038930 0x7fa50003adf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa504009fc0 tx=0x7fa504005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:25.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.140+0000 7fa50bfff640 1 -- 172.21.15.114:0/23587286 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fa4fc040020 con 0x7fa5140ff540 2024-09-13T18:14:25.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:24 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:25.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:24 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:25.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:24 smithi204 ceph-mon[34777]: Reconfiguring mon.b (monmap changed)... 2024-09-13T18:14:25.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:24 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-13T18:14:25.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:24 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-13T18:14:25.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:24 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:25.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:24 smithi204 ceph-mon[34777]: Reconfiguring daemon mon.b on smithi134 2024-09-13T18:14:25.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:24 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T18:14:25.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:24 smithi204 ceph-mon[34777]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:25.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.305+0000 7fa518810640 1 -- 172.21.15.114:0/23587286 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fa5140fef70 con 0x7fa5140ff540 2024-09-13T18:14:25.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.306+0000 7fa50bfff640 1 -- 172.21.15.114:0/23587286 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+277 (secure 0 0 0) 0x7fa4fc035070 con 0x7fa5140ff540 2024-09-13T18:14:25.308 INFO:teuthology.orchestra.run.smithi114.stdout:# minimal ceph.conf for 97e7edf2-71fb-11ef-bceb-c7b262605968 2024-09-13T18:14:25.308 INFO:teuthology.orchestra.run.smithi114.stdout:[global] 2024-09-13T18:14:25.308 INFO:teuthology.orchestra.run.smithi114.stdout: fsid = 97e7edf2-71fb-11ef-bceb-c7b262605968 2024-09-13T18:14:25.308 INFO:teuthology.orchestra.run.smithi114.stdout: mon_host = [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-13T18:14:25.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.308+0000 7fa518810640 1 -- 172.21.15.114:0/23587286 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fa500038930 msgr2=0x7fa50003adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:25.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.308+0000 7fa518810640 1 --2- 172.21.15.114:0/23587286 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fa500038930 0x7fa50003adf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa504009fc0 tx=0x7fa504005f90 comp rx=0 tx=0).stop 2024-09-13T18:14:25.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.308+0000 7fa518810640 1 -- 172.21.15.114:0/23587286 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa5140ff540 msgr2=0x7fa514196760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:25.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.308+0000 7fa518810640 1 --2- 172.21.15.114:0/23587286 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa5140ff540 0x7fa514196760 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc005ab0 tx=0x7fa4fc0329f0 comp rx=0 tx=0).stop 2024-09-13T18:14:25.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.308+0000 7fa518810640 1 -- 172.21.15.114:0/23587286 shutdown_connections 2024-09-13T18:14:25.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.308+0000 7fa518810640 1 --2- 172.21.15.114:0/23587286 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fa500038930 0x7fa50003adf0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:25.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.308+0000 7fa518810640 1 --2- 172.21.15.114:0/23587286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa51410a3a0 0x7fa51419ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:25.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.308+0000 7fa518810640 1 --2- 172.21.15.114:0/23587286 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa5140ff540 0x7fa514196760 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:25.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.308+0000 7fa518810640 1 --2- 172.21.15.114:0/23587286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5140feb90 0x7fa514196220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:25.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.308+0000 7fa518810640 1 -- 172.21.15.114:0/23587286 >> 172.21.15.114:0/23587286 conn(0x7fa514074130 msgr2=0x7fa5140fbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:14:25.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.308+0000 7fa518810640 1 -- 172.21.15.114:0/23587286 shutdown_connections 2024-09-13T18:14:25.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:25.308+0000 7fa518810640 1 -- 172.21.15.114:0/23587286 wait complete. 2024-09-13T18:14:25.923 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-13T18:14:25.923 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-13T18:14:25.923 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/ceph/ceph.conf 2024-09-13T18:14:25.952 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-13T18:14:25.952 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-13T18:14:26.019 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-13T18:14:26.019 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/ceph/ceph.conf 2024-09-13T18:14:26.032 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:25 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:26.032 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:25 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:26.032 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:25 smithi114 ceph-mon[30897]: Reconfiguring mon.c (monmap changed)... 2024-09-13T18:14:26.032 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:25 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-13T18:14:26.032 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:25 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-13T18:14:26.032 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:25 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:26.032 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:25 smithi114 ceph-mon[30897]: Reconfiguring daemon mon.c on smithi204 2024-09-13T18:14:26.033 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:25 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/23587286' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:26.052 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-13T18:14:26.052 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-13T18:14:26.121 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-13T18:14:26.121 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/ceph/ceph.conf 2024-09-13T18:14:26.153 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-13T18:14:26.154 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-13T18:14:26.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:25 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:26.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:25 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:26.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:25 smithi204 ceph-mon[34777]: Reconfiguring mon.c (monmap changed)... 2024-09-13T18:14:26.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:25 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-13T18:14:26.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:25 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-13T18:14:26.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:25 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:26.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:25 smithi204 ceph-mon[34777]: Reconfiguring daemon mon.c on smithi204 2024-09-13T18:14:26.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:25 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/23587286' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:26.219 INFO:tasks.cephadm:Adding mgr.x on smithi114 2024-09-13T18:14:26.219 INFO:tasks.cephadm:Adding mgr.y on smithi134 2024-09-13T18:14:26.220 INFO:tasks.cephadm:Adding mgr.z on smithi204 2024-09-13T18:14:26.220 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph orch apply mgr '3;smithi114=x;smithi134=y;smithi204=z' 2024-09-13T18:14:26.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:25 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:26.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:25 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:26.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:25 smithi134 ceph-mon[34283]: Reconfiguring mon.c (monmap changed)... 2024-09-13T18:14:26.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:25 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-13T18:14:26.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:25 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-13T18:14:26.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:25 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:26.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:25 smithi134 ceph-mon[34283]: Reconfiguring daemon mon.c on smithi204 2024-09-13T18:14:26.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:25 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/23587286' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:26.448 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T18:14:27.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:26 smithi204 ceph-mon[34777]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:27.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:26 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:27.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:26 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:27.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:26 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:14:27.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:26 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:27.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:26 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:14:27.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:26 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:27.230 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:26 smithi114 ceph-mon[30897]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:27.231 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:26 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:27.231 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:26 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:27.231 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:26 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:14:27.231 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:26 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:27.231 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:26 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:14:27.231 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:26 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:27.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:26 smithi134 ceph-mon[34283]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:27.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:26 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:27.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:26 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:27.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:26 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:14:27.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:26 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:27.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:26 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:14:27.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:26 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:27.901 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:27.894+0000 7f58da356640 1 -- 172.21.15.204:0/1688873321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58d40fed00 msgr2=0x7f58d40ff100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:27.901 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:27.894+0000 7f58da356640 1 --2- 172.21.15.204:0/1688873321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58d40fed00 0x7f58d40ff100 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f58c4009900 tx=0x7f58c402a410 comp rx=0 tx=0).stop 2024-09-13T18:14:27.901 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:27.894+0000 7f58da356640 1 -- 172.21.15.204:0/1688873321 shutdown_connections 2024-09-13T18:14:27.901 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:27.894+0000 7f58da356640 1 --2- 172.21.15.204:0/1688873321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58d41008c0 0x7f58d4107150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:27.902 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:27.894+0000 7f58da356640 1 --2- 172.21.15.204:0/1688873321 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58d40fff00 0x7f58d4100380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:27.902 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:27.894+0000 7f58da356640 1 --2- 172.21.15.204:0/1688873321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58d40fed00 0x7f58d40ff100 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:27.902 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:27.894+0000 7f58da356640 1 -- 172.21.15.204:0/1688873321 >> 172.21.15.204:0/1688873321 conn(0x7f58d40fa490 msgr2=0x7f58d40fc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:14:27.902 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:27.894+0000 7f58da356640 1 -- 172.21.15.204:0/1688873321 shutdown_connections 2024-09-13T18:14:27.902 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:27.894+0000 7f58da356640 1 -- 172.21.15.204:0/1688873321 wait complete. 2024-09-13T18:14:27.903 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:27.895+0000 7f58da356640 1 Processor -- start 2024-09-13T18:14:27.903 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:27.895+0000 7f58da356640 1 -- start start 2024-09-13T18:14:27.903 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:27.895+0000 7f58da356640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58d40fed00 0x7f58d4195c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:27.903 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:27.895+0000 7f58da356640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58d40fff00 0x7f58d41961a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:27.903 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:27.895+0000 7f58da356640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58d41008c0 0x7f58d419d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:27.903 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:27.895+0000 7f58da356640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d419d750 con 0x7f58d40fff00 2024-09-13T18:14:27.903 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:27.895+0000 7f58da356640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d419d8c0 con 0x7f58d40fed00 2024-09-13T18:14:27.904 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:27.895+0000 7f58da356640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d419da80 con 0x7f58d41008c0 2024-09-13T18:14:27.904 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:27.895+0000 7f58d88f3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58d41008c0 0x7f58d419d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:27.904 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:27.895+0000 7f58d88f3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58d41008c0 0x7f58d419d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:39086/0 (socket says 172.21.15.204:39086) 2024-09-13T18:14:27.904 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:27.895+0000 7f58d88f3640 1 -- 172.21.15.204:0/2850073393 learned_addr learned my addr 172.21.15.204:0/2850073393 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-13T18:14:27.904 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:27.895+0000 7f58d3fff640 1 --2- 172.21.15.204:0/2850073393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58d40fed00 0x7f58d4195c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:27.904 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:27.895+0000 7f58cbfff640 1 --2- 172.21.15.204:0/2850073393 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58d40fff00 0x7f58d41961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:27.904 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:27.895+0000 7f58d88f3640 1 -- 172.21.15.204:0/2850073393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58d40fed00 msgr2=0x7f58d4195c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:27.905 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:27.895+0000 7f58d88f3640 1 --2- 172.21.15.204:0/2850073393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58d40fed00 0x7f58d4195c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:27.905 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:27.895+0000 7f58d88f3640 1 -- 172.21.15.204:0/2850073393 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58d40fff00 msgr2=0x7f58d41961a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:27.905 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:27.896+0000 7f58d88f3640 1 --2- 172.21.15.204:0/2850073393 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58d40fff00 0x7f58d41961a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:27.905 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:27.896+0000 7f58d88f3640 1 -- 172.21.15.204:0/2850073393 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c4009560 con 0x7f58d41008c0 2024-09-13T18:14:27.905 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:27.896+0000 7f58cbfff640 1 --2- 172.21.15.204:0/2850073393 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58d40fff00 0x7f58d41961a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:14:27.905 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:27.896+0000 7f58d88f3640 1 --2- 172.21.15.204:0/2850073393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58d41008c0 0x7f58d419d210 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f58cc00eaa0 tx=0x7f58cc00ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:27.906 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:27.896+0000 7f58d1ffb640 1 -- 172.21.15.204:0/2850073393 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58cc00cdb0 con 0x7f58d41008c0 2024-09-13T18:14:27.906 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:27.896+0000 7f58d1ffb640 1 -- 172.21.15.204:0/2850073393 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f58cc005020 con 0x7f58d41008c0 2024-09-13T18:14:27.906 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:27.896+0000 7f58da356640 1 -- 172.21.15.204:0/2850073393 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58d419dd60 con 0x7f58d41008c0 2024-09-13T18:14:27.906 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:27.896+0000 7f58d1ffb640 1 -- 172.21.15.204:0/2850073393 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58cc0116d0 con 0x7f58d41008c0 2024-09-13T18:14:27.906 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:27.896+0000 7f58da356640 1 -- 172.21.15.204:0/2850073393 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58d419e220 con 0x7f58d41008c0 2024-09-13T18:14:27.906 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:27.897+0000 7f58d1ffb640 1 -- 172.21.15.204:0/2850073393 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7f58cc005190 con 0x7f58d41008c0 2024-09-13T18:14:27.906 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:27.897+0000 7f58da356640 1 -- 172.21.15.204:0/2850073393 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58d410a480 con 0x7f58d41008c0 2024-09-13T18:14:27.906 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:27.897+0000 7f58d1ffb640 1 --2- 172.21.15.204:0/2850073393 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f58c0038930 0x7f58c003adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:27.906 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:27.897+0000 7f58d1ffb640 1 -- 172.21.15.204:0/2850073393 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f58cc04b470 con 0x7f58d41008c0 2024-09-13T18:14:27.906 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:27.897+0000 7f58d3fff640 1 --2- 172.21.15.204:0/2850073393 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f58c0038930 0x7f58c003adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:27.907 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:27.898+0000 7f58d3fff640 1 --2- 172.21.15.204:0/2850073393 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f58c0038930 0x7f58c003adf0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f58c40057c0 tx=0x7f58c4005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:27.907 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:27.902+0000 7f58d1ffb640 1 -- 172.21.15.204:0/2850073393 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f58cc005490 con 0x7f58d41008c0 2024-09-13T18:14:28.084 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:28.082+0000 7f58da356640 1 -- 172.21.15.204:0/2850073393 --> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi114=x;smithi134=y;smithi204=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f58d4068e80 con 0x7f58c0038930 2024-09-13T18:14:28.090 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:28.088+0000 7f58d1ffb640 1 -- 172.21.15.204:0/2850073393 <== mgr.14148 v2:172.21.15.114:6800/3396185541 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f58d4068e80 con 0x7f58c0038930 2024-09-13T18:14:28.090 INFO:teuthology.orchestra.run.smithi204.stdout:Scheduled mgr update... 2024-09-13T18:14:28.093 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:28.091+0000 7f58da356640 1 -- 172.21.15.204:0/2850073393 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f58c0038930 msgr2=0x7f58c003adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:28.093 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:28.091+0000 7f58da356640 1 --2- 172.21.15.204:0/2850073393 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f58c0038930 0x7f58c003adf0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f58c40057c0 tx=0x7f58c4005a20 comp rx=0 tx=0).stop 2024-09-13T18:14:28.093 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:28.091+0000 7f58da356640 1 -- 172.21.15.204:0/2850073393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58d41008c0 msgr2=0x7f58d419d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:28.094 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:28.091+0000 7f58da356640 1 --2- 172.21.15.204:0/2850073393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58d41008c0 0x7f58d419d210 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f58cc00eaa0 tx=0x7f58cc00ef60 comp rx=0 tx=0).stop 2024-09-13T18:14:28.094 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:28.091+0000 7f58da356640 1 -- 172.21.15.204:0/2850073393 shutdown_connections 2024-09-13T18:14:28.094 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:28.091+0000 7f58da356640 1 --2- 172.21.15.204:0/2850073393 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f58c0038930 0x7f58c003adf0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:28.094 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:28.091+0000 7f58da356640 1 --2- 172.21.15.204:0/2850073393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58d41008c0 0x7f58d419d210 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:28.094 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:28.091+0000 7f58da356640 1 --2- 172.21.15.204:0/2850073393 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58d40fff00 0x7f58d41961a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:28.094 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:28.091+0000 7f58da356640 1 --2- 172.21.15.204:0/2850073393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58d40fed00 0x7f58d4195c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:28.094 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:28.091+0000 7f58da356640 1 -- 172.21.15.204:0/2850073393 >> 172.21.15.204:0/2850073393 conn(0x7f58d40fa490 msgr2=0x7f58d4068770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:14:28.094 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:28.091+0000 7f58da356640 1 -- 172.21.15.204:0/2850073393 shutdown_connections 2024-09-13T18:14:28.094 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:14:28.091+0000 7f58da356640 1 -- 172.21.15.204:0/2850073393 wait complete. 2024-09-13T18:14:28.700 DEBUG:teuthology.orchestra.run.smithi134:mgr.y> sudo journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mgr.y.service 2024-09-13T18:14:28.703 DEBUG:teuthology.orchestra.run.smithi204:mgr.z> sudo journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mgr.z.service 2024-09-13T18:14:28.705 INFO:tasks.cephadm:Deploying OSDs... 2024-09-13T18:14:28.706 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-13T18:14:28.706 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/scratch_devs of=/dev/stdout 2024-09-13T18:14:28.722 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-13T18:14:28.722 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_1 2024-09-13T18:14:28.779 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-13T18:14:28.779 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T18:14:28.779 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-13T18:14:28.779 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T18:14:28.779 INFO:teuthology.orchestra.run.smithi114.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T18:14:28.779 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2024-09-13 18:13:37.961125142 +0000 2024-09-13T18:14:28.779 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-09-13 18:09:36.417450383 +0000 2024-09-13T18:14:28.779 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-09-13 18:09:36.417450383 +0000 2024-09-13T18:14:28.779 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: 2024-09-13 18:09:36.417450383 +0000 2024-09-13T18:14:28.779 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-13T18:14:28.844 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2024-09-13T18:14:28.844 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2024-09-13T18:14:28.844 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.000254087 s, 2.0 MB/s 2024-09-13T18:14:28.845 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-13T18:14:28.902 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_2 2024-09-13T18:14:28.958 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-13T18:14:28.958 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T18:14:28.959 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-13T18:14:28.959 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T18:14:28.959 INFO:teuthology.orchestra.run.smithi114.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T18:14:28.959 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2024-09-13 18:13:37.990124474 +0000 2024-09-13T18:14:28.959 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-09-13 18:09:36.755442592 +0000 2024-09-13T18:14:28.959 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-09-13 18:09:36.755442592 +0000 2024-09-13T18:14:28.959 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: 2024-09-13 18:09:36.755442592 +0000 2024-09-13T18:14:28.959 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-13T18:14:29.023 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2024-09-13T18:14:29.023 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2024-09-13T18:14:29.023 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.000188702 s, 2.7 MB/s 2024-09-13T18:14:29.024 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-13T18:14:29.081 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_3 2024-09-13T18:14:29.136 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-13T18:14:29.136 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T18:14:29.136 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-13T18:14:29.136 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T18:14:29.136 INFO:teuthology.orchestra.run.smithi114.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T18:14:29.137 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2024-09-13 18:13:38.017123852 +0000 2024-09-13T18:14:29.137 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-09-13 18:09:37.089434894 +0000 2024-09-13T18:14:29.137 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-09-13 18:09:37.089434894 +0000 2024-09-13T18:14:29.137 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: 2024-09-13 18:09:37.089434894 +0000 2024-09-13T18:14:29.137 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-13T18:14:29.198 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:29 smithi114 ceph-mon[30897]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:29.198 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:29 smithi114 ceph-mon[30897]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi114=x;smithi134=y;smithi204=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:14:29.198 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:29 smithi114 ceph-mon[30897]: Saving service mgr spec with placement smithi114=x;smithi134=y;smithi204=z;count:3 2024-09-13T18:14:29.198 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:29 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:29.198 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:29 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:14:29.198 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:29 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:29.199 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:29 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:14:29.199 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:29 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:29.199 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:29 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-13T18:14:29.199 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:29 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-13T18:14:29.199 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:29 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-13T18:14:29.199 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:29 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:29.200 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2024-09-13T18:14:29.200 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2024-09-13T18:14:29.200 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.000213214 s, 2.4 MB/s 2024-09-13T18:14:29.201 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-13T18:14:29.258 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_4 2024-09-13T18:14:29.313 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-13T18:14:29.314 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T18:14:29.314 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-13T18:14:29.314 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T18:14:29.314 INFO:teuthology.orchestra.run.smithi114.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T18:14:29.314 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2024-09-13 18:13:38.044123230 +0000 2024-09-13T18:14:29.315 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-09-13 18:09:37.428427080 +0000 2024-09-13T18:14:29.315 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-09-13 18:09:37.428427080 +0000 2024-09-13T18:14:29.315 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: 2024-09-13 18:09:37.428427080 +0000 2024-09-13T18:14:29.315 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-13T18:14:29.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:29 smithi134 ceph-mon[34283]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:29.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:29 smithi134 ceph-mon[34283]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi114=x;smithi134=y;smithi204=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:14:29.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:29 smithi134 ceph-mon[34283]: Saving service mgr spec with placement smithi114=x;smithi134=y;smithi204=z;count:3 2024-09-13T18:14:29.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:29 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:29.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:29 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:14:29.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:29 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:29.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:29 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:14:29.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:29 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:29.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:29 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-13T18:14:29.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:29 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-13T18:14:29.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:29 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-13T18:14:29.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:29 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:29.377 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2024-09-13T18:14:29.377 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2024-09-13T18:14:29.377 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.000262468 s, 2.0 MB/s 2024-09-13T18:14:29.378 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-13T18:14:29.435 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-13T18:14:29.436 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/scratch_devs of=/dev/stdout 2024-09-13T18:14:29.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:29 smithi204 ceph-mon[34777]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:29.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:29 smithi204 ceph-mon[34777]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi114=x;smithi134=y;smithi204=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:14:29.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:29 smithi204 ceph-mon[34777]: Saving service mgr spec with placement smithi114=x;smithi134=y;smithi204=z;count:3 2024-09-13T18:14:29.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:29 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:29.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:29 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:14:29.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:29 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:29.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:29 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:14:29.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:29 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:29.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:29 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-13T18:14:29.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:29 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-13T18:14:29.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:29 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-13T18:14:29.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:29 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:29.455 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-13T18:14:29.456 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_1 2024-09-13T18:14:29.513 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-13T18:14:29.514 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T18:14:29.514 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-13T18:14:29.514 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T18:14:29.514 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T18:14:29.514 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-09-13 18:14:15.144830335 +0000 2024-09-13T18:14:29.514 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-09-13 18:09:30.606842925 +0000 2024-09-13T18:14:29.514 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-09-13 18:09:30.606842925 +0000 2024-09-13T18:14:29.514 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-09-13 18:09:30.606842925 +0000 2024-09-13T18:14:29.514 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-13T18:14:29.580 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-09-13T18:14:29.580 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-09-13T18:14:29.581 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.00022021 s, 2.3 MB/s 2024-09-13T18:14:29.581 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-13T18:14:29.640 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_2 2024-09-13T18:14:29.698 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-13T18:14:29.698 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T18:14:29.698 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-13T18:14:29.698 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T18:14:29.698 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T18:14:29.698 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-09-13 18:14:15.144830335 +0000 2024-09-13T18:14:29.698 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-09-13 18:09:30.985833430 +0000 2024-09-13T18:14:29.698 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-09-13 18:09:30.985833430 +0000 2024-09-13T18:14:29.699 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-09-13 18:09:30.985833430 +0000 2024-09-13T18:14:29.699 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-13T18:14:29.764 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-09-13T18:14:29.764 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-09-13T18:14:29.764 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000238708 s, 2.1 MB/s 2024-09-13T18:14:29.765 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-13T18:14:29.823 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_3 2024-09-13T18:14:29.879 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-13T18:14:29.880 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T18:14:29.880 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-13T18:14:29.880 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T18:14:29.880 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T18:14:29.880 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-09-13 18:14:15.145830310 +0000 2024-09-13T18:14:29.880 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-09-13 18:09:31.304825438 +0000 2024-09-13T18:14:29.880 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-09-13 18:09:31.304825438 +0000 2024-09-13T18:14:29.880 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-09-13 18:09:31.304825438 +0000 2024-09-13T18:14:29.880 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-13T18:14:29.944 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-09-13T18:14:29.945 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-09-13T18:14:29.945 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000218152 s, 2.3 MB/s 2024-09-13T18:14:29.946 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-13T18:14:30.003 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_4 2024-09-13T18:14:30.059 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-13T18:14:30.059 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T18:14:30.059 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-13T18:14:30.059 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T18:14:30.059 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T18:14:30.059 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-09-13 18:14:15.145830310 +0000 2024-09-13T18:14:30.059 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-09-13 18:09:31.623817447 +0000 2024-09-13T18:14:30.060 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-09-13 18:09:31.623817447 +0000 2024-09-13T18:14:30.060 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-09-13 18:09:31.623817447 +0000 2024-09-13T18:14:30.060 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-13T18:14:30.123 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-09-13T18:14:30.124 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-09-13T18:14:30.124 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000243852 s, 2.1 MB/s 2024-09-13T18:14:30.125 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-13T18:14:30.181 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-13T18:14:30.181 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/scratch_devs of=/dev/stdout 2024-09-13T18:14:30.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:30 smithi204 ceph-mon[34777]: Deploying daemon mgr.z on smithi204 2024-09-13T18:14:30.199 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-13T18:14:30.199 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_1 2024-09-13T18:14:30.254 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-13T18:14:30.254 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T18:14:30.255 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-13T18:14:30.255 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T18:14:30.255 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T18:14:30.255 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-09-13 18:14:30.000458464 +0000 2024-09-13T18:14:30.255 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-09-13 18:09:30.612408971 +0000 2024-09-13T18:14:30.255 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-09-13 18:09:30.612408971 +0000 2024-09-13T18:14:30.255 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-09-13 18:09:30.612408971 +0000 2024-09-13T18:14:30.255 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-13T18:14:30.321 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-09-13T18:14:30.321 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-09-13T18:14:30.321 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000267605 s, 1.9 MB/s 2024-09-13T18:14:30.322 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-13T18:14:30.368 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:30 smithi134 ceph-mon[34283]: Deploying daemon mgr.z on smithi204 2024-09-13T18:14:30.378 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_2 2024-09-13T18:14:30.436 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-13T18:14:30.436 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T18:14:30.436 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-13T18:14:30.436 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T18:14:30.436 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T18:14:30.436 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-09-13 18:14:30.000458464 +0000 2024-09-13T18:14:30.436 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-09-13 18:09:30.948404361 +0000 2024-09-13T18:14:30.437 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-09-13 18:09:30.948404361 +0000 2024-09-13T18:14:30.437 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-09-13 18:09:30.948404361 +0000 2024-09-13T18:14:30.437 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-13T18:14:30.469 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:30 smithi114 ceph-mon[30897]: Deploying daemon mgr.z on smithi204 2024-09-13T18:14:30.498 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-09-13T18:14:30.498 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-09-13T18:14:30.498 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000178308 s, 2.9 MB/s 2024-09-13T18:14:30.499 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-13T18:14:30.551 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:14:30 smithi204 podman[36401]: 2024-09-13 18:14:30.524423526 +0000 UTC m=+0.360906948 container start 130ab02f9a47adf7781653f394871faf0546b2c75d91c63fa44287cbcdf8acc1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-13T18:14:30.556 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_3 2024-09-13T18:14:30.612 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-13T18:14:30.612 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T18:14:30.612 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-13T18:14:30.612 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T18:14:30.612 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T18:14:30.612 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-09-13 18:14:30.000458464 +0000 2024-09-13T18:14:30.612 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-09-13 18:09:31.273399902 +0000 2024-09-13T18:14:30.612 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-09-13 18:09:31.273399902 +0000 2024-09-13T18:14:30.612 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-09-13 18:09:31.273399902 +0000 2024-09-13T18:14:30.613 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-13T18:14:30.677 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-09-13T18:14:30.677 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-09-13T18:14:30.677 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000208829 s, 2.5 MB/s 2024-09-13T18:14:30.678 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-13T18:14:30.734 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_4 2024-09-13T18:14:30.789 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-13T18:14:30.789 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T18:14:30.790 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-13T18:14:30.790 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T18:14:30.790 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T18:14:30.790 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-09-13 18:14:30.001458450 +0000 2024-09-13T18:14:30.790 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-09-13 18:09:31.611395263 +0000 2024-09-13T18:14:30.790 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-09-13 18:09:31.611395263 +0000 2024-09-13T18:14:30.790 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-09-13 18:09:31.611395263 +0000 2024-09-13T18:14:30.790 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-13T18:14:30.852 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:14:30 smithi204 ceph-mgr[36523]: -- 172.21.15.204:0/1037061957 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558fb1a04d00 con 0x558fb19d8800 2024-09-13T18:14:30.852 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:14:30 smithi204 bash[36401]: 130ab02f9a47adf7781653f394871faf0546b2c75d91c63fa44287cbcdf8acc1 2024-09-13T18:14:30.852 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:14:30 smithi204 systemd[1]: Started Ceph mgr.z for 97e7edf2-71fb-11ef-bceb-c7b262605968. 2024-09-13T18:14:30.852 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:14:30 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z[36485]: 2024-09-13T18:14:30.646+0000 7f52f0fcf040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T18:14:30.852 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:14:30 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z[36485]: 2024-09-13T18:14:30.707+0000 7f52f0fcf040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T18:14:30.854 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-09-13T18:14:30.854 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-09-13T18:14:30.854 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000203157 s, 2.5 MB/s 2024-09-13T18:14:30.856 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-13T18:14:30.912 INFO:tasks.cephadm:Deploying osd.0 on smithi114 with /dev/vg_nvme/lv_4... 2024-09-13T18:14:30.912 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-09-13T18:14:31.114 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:14:31.194 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:14:30 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z[36485]: 2024-09-13T18:14:30.862+0000 7f52f0fcf040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T18:14:31.194 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:14:30 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z[36485]: 2024-09-13T18:14:30.930+0000 7f52f0fcf040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T18:14:31.194 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:14:31 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z[36485]: 2024-09-13T18:14:31.075+0000 7f52f0fcf040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T18:14:31.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:31 smithi204 ceph-mon[34777]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:31.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:31 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:31.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:31 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:31.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:31 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:31.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:31 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-13T18:14:31.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:31 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-13T18:14:31.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:31 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-13T18:14:31.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:31 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:31.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:31 smithi114 ceph-mon[30897]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:31.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:31 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:31.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:31 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:31.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:31 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:31.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:31 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-13T18:14:31.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:31 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-13T18:14:31.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:31 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-13T18:14:31.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:31 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:31.311 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:31 smithi134 ceph-mon[34283]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:31.311 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:31 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:31.311 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:31 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:31.311 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:31 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:31.311 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:31 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-13T18:14:31.311 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:31 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-13T18:14:31.311 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:31 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-13T18:14:31.311 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:31 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:31.521 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:14:31 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z[36485]: 2024-09-13T18:14:31.214+0000 7f52f0fcf040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T18:14:31.521 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:14:31 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z[36485]: 2024-09-13T18:14:31.518+0000 7f52f0fcf040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T18:14:31.944 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:14:31 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z[36485]: 2024-09-13T18:14:31.578+0000 7f52f0fcf040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T18:14:31.944 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:14:31 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z[36485]: 2024-09-13T18:14:31.757+0000 7f52f0fcf040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T18:14:32.181 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:32 smithi134 ceph-mon[34283]: Deploying daemon mgr.y on smithi134 2024-09-13T18:14:32.376 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:32 smithi114 ceph-mon[30897]: Deploying daemon mgr.y on smithi134 2024-09-13T18:14:32.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:32 smithi204 ceph-mon[34777]: Deploying daemon mgr.y on smithi134 2024-09-13T18:14:32.561 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:14:32 smithi134 systemd[1]: Starting Ceph mgr.y for 97e7edf2-71fb-11ef-bceb-c7b262605968... 2024-09-13T18:14:32.812 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:14:32 smithi134 podman[35670]: 2024-09-13 18:14:32.462238435 +0000 UTC m=+0.015696653 image pull 8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256 2024-09-13T18:14:32.812 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:14:32 smithi134 podman[35670]: 2024-09-13 18:14:32.57181433 +0000 UTC m=+0.125272535 container create 2a4536a41e9315cedf95afec38051ff4fc8abde24038b769de9cd0222ee4aa4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-13T18:14:32.812 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:14:32 smithi134 podman[35670]: 2024-09-13 18:14:32.807593857 +0000 UTC m=+0.361052072 container init 2a4536a41e9315cedf95afec38051ff4fc8abde24038b769de9cd0222ee4aa4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True) 2024-09-13T18:14:32.812 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:14:32 smithi134 podman[35670]: 2024-09-13 18:14:32.810343108 +0000 UTC m=+0.363801314 container start 2a4536a41e9315cedf95afec38051ff4fc8abde24038b769de9cd0222ee4aa4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-13T18:14:32.841 INFO:teuthology.orchestra.run.smithi114.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-09-13T18:14:32.917 INFO:teuthology.orchestra.run.smithi114.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-09-13T18:14:32.930 INFO:teuthology.orchestra.run.smithi114.stderr: stderr: 10+0 records in 2024-09-13T18:14:32.931 INFO:teuthology.orchestra.run.smithi114.stderr:10+0 records out 2024-09-13T18:14:32.931 INFO:teuthology.orchestra.run.smithi114.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0117116 s, 895 MB/s 2024-09-13T18:14:32.931 INFO:teuthology.orchestra.run.smithi114.stderr:--> Zapping successful for: 2024-09-13T18:14:32.944 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:14:32 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z[36485]: 2024-09-13T18:14:32.623+0000 7f52f0fcf040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T18:14:32.944 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:14:32 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z[36485]: 2024-09-13T18:14:32.688+0000 7f52f0fcf040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T18:14:33.103 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:14:32 smithi134 ceph-mgr[35689]: -- 172.21.15.134:0/3504390202 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5569765ccd00 con 0x5569765a1000 2024-09-13T18:14:33.103 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:14:32 smithi134 bash[35670]: 2a4536a41e9315cedf95afec38051ff4fc8abde24038b769de9cd0222ee4aa4b 2024-09-13T18:14:33.103 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:14:32 smithi134 systemd[1]: Started Ceph mgr.y for 97e7edf2-71fb-11ef-bceb-c7b262605968. 2024-09-13T18:14:33.103 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:14:32 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y[35685]: 2024-09-13T18:14:32.930+0000 7f2764163040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T18:14:33.103 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:14:32 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y[35685]: 2024-09-13T18:14:32.988+0000 7f2764163040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T18:14:33.369 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:14:33 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y[35685]: 2024-09-13T18:14:33.144+0000 7f2764163040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T18:14:33.369 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:14:33 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y[35685]: 2024-09-13T18:14:33.210+0000 7f2764163040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T18:14:33.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:33 smithi134 ceph-mon[34283]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:33.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:33 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:33.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:33 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:33.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:33 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:33.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:33 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:33.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:33 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:14:33.435 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:33 smithi204 ceph-mon[34777]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:33.435 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:33 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:33.435 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:33 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:33.435 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:33 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:33.435 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:33 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:33.435 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:33 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:14:33.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:33 smithi114 ceph-mon[30897]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:33.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:33 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:33.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:33 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:33.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:33 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:33.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:33 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:33.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:33 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:14:33.552 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi114:vg_nvme/lv_4 2024-09-13T18:14:33.694 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:14:33 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z[36485]: 2024-09-13T18:14:33.432+0000 7f52f0fcf040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-13T18:14:33.694 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:14:33 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z[36485]: 2024-09-13T18:14:33.546+0000 7f52f0fcf040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T18:14:33.739 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:14:33 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y[35685]: 2024-09-13T18:14:33.371+0000 7f2764163040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T18:14:33.739 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:14:33 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y[35685]: 2024-09-13T18:14:33.512+0000 7f2764163040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T18:14:33.742 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:14:33.978 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:14:33 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z[36485]: 2024-09-13T18:14:33.699+0000 7f52f0fcf040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T18:14:33.978 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:14:33 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z[36485]: 2024-09-13T18:14:33.760+0000 7f52f0fcf040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T18:14:33.978 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:14:33 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z[36485]: 2024-09-13T18:14:33.908+0000 7f52f0fcf040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-13T18:14:33.978 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:14:33 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z[36485]: 2024-09-13T18:14:33.975+0000 7f52f0fcf040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T18:14:34.078 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:14:33 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y[35685]: 2024-09-13T18:14:33.831+0000 7f2764163040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T18:14:34.078 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:14:33 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y[35685]: 2024-09-13T18:14:33.890+0000 7f2764163040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T18:14:34.273 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:14:34 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z[36485]: 2024-09-13T18:14:34.039+0000 7f52f0fcf040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T18:14:34.273 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:14:34 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z[36485]: 2024-09-13T18:14:34.205+0000 7f52f0fcf040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T18:14:34.273 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:14:34 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z[36485]: 2024-09-13T18:14:34.271+0000 7f52f0fcf040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T18:14:34.369 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:14:34 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y[35685]: 2024-09-13T18:14:34.075+0000 7f2764163040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T18:14:34.536 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:14:34 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z[36485]: 2024-09-13T18:14:34.335+0000 7f52f0fcf040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T18:14:34.536 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:14:34 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z[36485]: 2024-09-13T18:14:34.417+0000 7f52f0fcf040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T18:14:34.536 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:14:34 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z[36485]: 2024-09-13T18:14:34.475+0000 7f52f0fcf040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T18:14:34.536 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:14:34 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z[36485]: 2024-09-13T18:14:34.533+0000 7f52f0fcf040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T18:14:35.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:35.269+0000 7fd81cc4d640 1 -- 172.21.15.114:0/3606975901 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8180fff00 msgr2=0x7fd818100380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:35.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:35.269+0000 7fd81cc4d640 1 --2- 172.21.15.114:0/3606975901 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8180fff00 0x7fd818100380 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd800009900 tx=0x7fd80002a410 comp rx=0 tx=0).stop 2024-09-13T18:14:35.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:35.269+0000 7fd81cc4d640 1 -- 172.21.15.114:0/3606975901 shutdown_connections 2024-09-13T18:14:35.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:35.269+0000 7fd81cc4d640 1 --2- 172.21.15.114:0/3606975901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8181008c0 0x7fd818107150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:35.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:35.269+0000 7fd81cc4d640 1 --2- 172.21.15.114:0/3606975901 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8180fff00 0x7fd818100380 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:35.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:35.269+0000 7fd81cc4d640 1 --2- 172.21.15.114:0/3606975901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8180fed00 0x7fd8180ff100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:35.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:35.269+0000 7fd81cc4d640 1 -- 172.21.15.114:0/3606975901 >> 172.21.15.114:0/3606975901 conn(0x7fd8180fa490 msgr2=0x7fd8180fc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:14:35.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:35.269+0000 7fd81cc4d640 1 -- 172.21.15.114:0/3606975901 shutdown_connections 2024-09-13T18:14:35.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:35.269+0000 7fd81cc4d640 1 -- 172.21.15.114:0/3606975901 wait complete. 2024-09-13T18:14:35.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:35.270+0000 7fd81cc4d640 1 Processor -- start 2024-09-13T18:14:35.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:35.270+0000 7fd81cc4d640 1 -- start start 2024-09-13T18:14:35.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:35.270+0000 7fd81cc4d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8180fed00 0x7fd818069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:35.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:35.270+0000 7fd81cc4d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8180fff00 0x7fd81819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:35.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:35.270+0000 7fd81cc4d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8181008c0 0x7fd81819d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:35.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:35.270+0000 7fd81cc4d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd81819d860 con 0x7fd8180fed00 2024-09-13T18:14:35.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:35.270+0000 7fd81cc4d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd81819d9d0 con 0x7fd8180fff00 2024-09-13T18:14:35.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:35.270+0000 7fd81cc4d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd81819db90 con 0x7fd8181008c0 2024-09-13T18:14:35.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:35.270+0000 7fd81659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8180fed00 0x7fd818069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:35.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:35.270+0000 7fd81659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8180fed00 0x7fd818069280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:52722/0 (socket says 172.21.15.114:52722) 2024-09-13T18:14:35.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:35.270+0000 7fd81659c640 1 -- 172.21.15.114:0/1723149435 learned_addr learned my addr 172.21.15.114:0/1723149435 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:14:35.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:35.270+0000 7fd80ffff640 1 --2- 172.21.15.114:0/1723149435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8180fff00 0x7fd81819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:35.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:35.270+0000 7fd816d9d640 1 --2- 172.21.15.114:0/1723149435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8181008c0 0x7fd81819d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:35.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:35.270+0000 7fd81659c640 1 -- 172.21.15.114:0/1723149435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8181008c0 msgr2=0x7fd81819d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:35.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:35.270+0000 7fd81659c640 1 --2- 172.21.15.114:0/1723149435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8181008c0 0x7fd81819d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:35.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:35.270+0000 7fd81659c640 1 -- 172.21.15.114:0/1723149435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8180fff00 msgr2=0x7fd81819aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:35.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:35.270+0000 7fd81659c640 1 --2- 172.21.15.114:0/1723149435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8180fff00 0x7fd81819aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:35.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:35.270+0000 7fd81659c640 1 -- 172.21.15.114:0/1723149435 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd800009560 con 0x7fd8180fed00 2024-09-13T18:14:35.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:35.270+0000 7fd816d9d640 1 --2- 172.21.15.114:0/1723149435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8181008c0 0x7fd81819d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:14:35.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:35.270+0000 7fd80ffff640 1 --2- 172.21.15.114:0/1723149435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8180fff00 0x7fd81819aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:14:35.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:35.271+0000 7fd81659c640 1 --2- 172.21.15.114:0/1723149435 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8180fed00 0x7fd818069280 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd80800d950 tx=0x7fd80800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:35.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:35.271+0000 7fd80f7fe640 1 -- 172.21.15.114:0/1723149435 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd808009c30 con 0x7fd8180fed00 2024-09-13T18:14:35.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:35.271+0000 7fd80f7fe640 1 -- 172.21.15.114:0/1723149435 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd808009d90 con 0x7fd8180fed00 2024-09-13T18:14:35.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:35.271+0000 7fd81cc4d640 1 -- 172.21.15.114:0/1723149435 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd81819de10 con 0x7fd8180fed00 2024-09-13T18:14:35.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:35.271+0000 7fd80f7fe640 1 -- 172.21.15.114:0/1723149435 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80800f6b0 con 0x7fd8180fed00 2024-09-13T18:14:35.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:35.271+0000 7fd81cc4d640 1 -- 172.21.15.114:0/1723149435 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd81819e220 con 0x7fd8180fed00 2024-09-13T18:14:35.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:35.272+0000 7fd80f7fe640 1 -- 172.21.15.114:0/1723149435 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7fd80800f810 con 0x7fd8180fed00 2024-09-13T18:14:35.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:35.272+0000 7fd80f7fe640 1 --2- 172.21.15.114:0/1723149435 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fd804038930 0x7fd80403adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:35.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:35.272+0000 7fd81cc4d640 1 -- 172.21.15.114:0/1723149435 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd818106240 con 0x7fd8180fed00 2024-09-13T18:14:35.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:35.272+0000 7fd80f7fe640 1 -- 172.21.15.114:0/1723149435 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd80804b4f0 con 0x7fd8180fed00 2024-09-13T18:14:35.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:35.272+0000 7fd80ffff640 1 --2- 172.21.15.114:0/1723149435 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fd804038930 0x7fd80403adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:35.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:35.272+0000 7fd80ffff640 1 --2- 172.21.15.114:0/1723149435 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fd804038930 0x7fd80403adf0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd800000c00 tx=0x7fd800005a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:35.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:35.277+0000 7fd80f7fe640 1 -- 172.21.15.114:0/1723149435 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fd808023950 con 0x7fd8180fed00 2024-09-13T18:14:35.366 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:35 smithi114 ceph-mon[30897]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:35.366 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:35 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:35.366 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:35 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:35.366 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:35 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:35.366 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:35 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:35.366 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:35 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:35.366 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:35 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:14:35.366 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:35 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:35.366 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:35 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-13T18:14:35.367 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:35 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-13T18:14:35.367 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:35 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:35.369 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:14:34 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y[35685]: 2024-09-13T18:14:34.941+0000 7f2764163040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T18:14:35.369 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:14:35 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y[35685]: 2024-09-13T18:14:35.005+0000 7f2764163040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T18:14:35.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:35 smithi134 ceph-mon[34283]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:35.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:35 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:35.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:35 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:35.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:35 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:35.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:35 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:35.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:35 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:35.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:35 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:14:35.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:35 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:35.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:35 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-13T18:14:35.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:35 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-13T18:14:35.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:35 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:35.444 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:14:35 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z[36485]: 2024-09-13T18:14:35.305+0000 7f52f0fcf040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T18:14:35.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:35 smithi204 ceph-mon[34777]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:35.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:35 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:35.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:35 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:35.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:35 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:35.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:35 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:35.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:35 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:35.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:35 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:14:35.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:35 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:35.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:35 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-13T18:14:35.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:35 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-13T18:14:35.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:35 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:35.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:35.452+0000 7fd81cc4d640 1 -- 172.21.15.114:0/1723149435 --> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fd81802c520 con 0x7fd804038930 2024-09-13T18:14:35.961 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:14:35 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y[35685]: 2024-09-13T18:14:35.708+0000 7f2764163040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-13T18:14:35.962 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:14:35 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y[35685]: 2024-09-13T18:14:35.820+0000 7f2764163040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T18:14:35.962 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:14:35 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y[35685]: 2024-09-13T18:14:35.959+0000 7f2764163040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T18:14:36.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:36.103+0000 7fd80f7fe640 1 -- 172.21.15.114:0/1723149435 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mgrmap(e 14) v1 ==== 90708+0+0 (secure 0 0 0) 0x7fd80801c920 con 0x7fd8180fed00 2024-09-13T18:14:36.232 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:14:36 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y[35685]: 2024-09-13T18:14:36.018+0000 7f2764163040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T18:14:36.232 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:14:36 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y[35685]: 2024-09-13T18:14:36.163+0000 7f2764163040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-13T18:14:36.232 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:14:36 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y[35685]: 2024-09-13T18:14:36.229+0000 7f2764163040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T18:14:36.232 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:36 smithi134 ceph-mon[34283]: Reconfiguring mgr.x (unknown last config time)... 2024-09-13T18:14:36.232 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:36 smithi134 ceph-mon[34283]: Reconfiguring daemon mgr.x on smithi114 2024-09-13T18:14:36.233 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:36 smithi134 ceph-mon[34283]: Standby manager daemon z started 2024-09-13T18:14:36.233 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:36 smithi134 ceph-mon[34283]: from='mgr.? 172.21.15.204:0/1997532904' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-13T18:14:36.233 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:36 smithi134 ceph-mon[34283]: from='mgr.? 172.21.15.204:0/1997532904' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-13T18:14:36.233 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:36 smithi134 ceph-mon[34283]: from='mgr.? 172.21.15.204:0/1997532904' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-13T18:14:36.233 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:36 smithi134 ceph-mon[34283]: from='mgr.? 172.21.15.204:0/1997532904' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-13T18:14:36.233 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:36 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T18:14:36.233 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:36 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T18:14:36.233 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:36 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:36.233 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:36 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:36.418 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:36 smithi114 ceph-mon[30897]: Reconfiguring mgr.x (unknown last config time)... 2024-09-13T18:14:36.418 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:36 smithi114 ceph-mon[30897]: Reconfiguring daemon mgr.x on smithi114 2024-09-13T18:14:36.418 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:36 smithi114 ceph-mon[30897]: Standby manager daemon z started 2024-09-13T18:14:36.418 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:36 smithi114 ceph-mon[30897]: from='mgr.? 172.21.15.204:0/1997532904' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-13T18:14:36.418 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:36 smithi114 ceph-mon[30897]: from='mgr.? 172.21.15.204:0/1997532904' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-13T18:14:36.418 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:36 smithi114 ceph-mon[30897]: from='mgr.? 172.21.15.204:0/1997532904' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-13T18:14:36.418 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:36 smithi114 ceph-mon[30897]: from='mgr.? 172.21.15.204:0/1997532904' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-13T18:14:36.418 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:36 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T18:14:36.418 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:36 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T18:14:36.418 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:36 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:36.418 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:36 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:36.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:36 smithi204 ceph-mon[34777]: Reconfiguring mgr.x (unknown last config time)... 2024-09-13T18:14:36.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:36 smithi204 ceph-mon[34777]: Reconfiguring daemon mgr.x on smithi114 2024-09-13T18:14:36.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:36 smithi204 ceph-mon[34777]: Standby manager daemon z started 2024-09-13T18:14:36.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:36 smithi204 ceph-mon[34777]: from='mgr.? 172.21.15.204:0/1997532904' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-13T18:14:36.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:36 smithi204 ceph-mon[34777]: from='mgr.? 172.21.15.204:0/1997532904' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-13T18:14:36.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:36 smithi204 ceph-mon[34777]: from='mgr.? 172.21.15.204:0/1997532904' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-13T18:14:36.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:36 smithi204 ceph-mon[34777]: from='mgr.? 172.21.15.204:0/1997532904' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-13T18:14:36.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:36 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T18:14:36.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:36 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T18:14:36.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:36 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:36.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:36 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:36.510 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:14:36 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y[35685]: 2024-09-13T18:14:36.287+0000 7f2764163040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T18:14:36.510 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:14:36 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y[35685]: 2024-09-13T18:14:36.441+0000 7f2764163040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T18:14:36.510 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:14:36 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y[35685]: 2024-09-13T18:14:36.507+0000 7f2764163040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T18:14:36.771 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:14:36 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y[35685]: 2024-09-13T18:14:36.572+0000 7f2764163040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T18:14:36.771 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:14:36 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y[35685]: 2024-09-13T18:14:36.653+0000 7f2764163040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T18:14:36.771 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:14:36 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y[35685]: 2024-09-13T18:14:36.710+0000 7f2764163040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T18:14:36.771 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:14:36 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y[35685]: 2024-09-13T18:14:36.768+0000 7f2764163040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T18:14:37.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:37 smithi134 ceph-mon[34283]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:14:37.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:37 smithi134 ceph-mon[34283]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:37.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:37 smithi134 ceph-mon[34283]: mgrmap e14: x(active, since 70s), standbys: z 2024-09-13T18:14:37.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:37 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-13T18:14:37.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:37 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:37.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:37 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:37.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:37 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:14:37.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:37 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:37.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:37 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:14:37.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:37 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:37.418 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:37 smithi114 ceph-mon[30897]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:14:37.418 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:37 smithi114 ceph-mon[30897]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:37.418 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:37 smithi114 ceph-mon[30897]: mgrmap e14: x(active, since 70s), standbys: z 2024-09-13T18:14:37.418 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:37 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-13T18:14:37.418 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:37 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:37.418 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:37 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:37.418 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:37 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:14:37.418 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:37 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:37.418 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:37 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:14:37.419 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:37 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:37.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:37 smithi204 ceph-mon[34777]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:14:37.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:37 smithi204 ceph-mon[34777]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:37.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:37 smithi204 ceph-mon[34777]: mgrmap e14: x(active, since 70s), standbys: z 2024-09-13T18:14:37.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:37 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-13T18:14:37.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:37 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:37.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:37 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:37.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:37 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:14:37.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:37 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:37.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:37 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:14:37.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:37 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:37.869 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:14:37 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y[35685]: 2024-09-13T18:14:37.530+0000 7f2764163040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T18:14:38.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:38.109+0000 7fd80f7fe640 1 -- 172.21.15.114:0/1723149435 <== mon.0 v2:172.21.15.114:3300/0 8 ==== mgrmap(e 15) v1 ==== 135818+0+0 (secure 0 0 0) 0x7fd80801c400 con 0x7fd8180fed00 2024-09-13T18:14:38.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:38 smithi134 ceph-mon[34283]: Standby manager daemon y started 2024-09-13T18:14:38.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:38 smithi134 ceph-mon[34283]: from='mgr.? 172.21.15.134:0/3558179159' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-13T18:14:38.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:38 smithi134 ceph-mon[34283]: from='mgr.? 172.21.15.134:0/3558179159' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-13T18:14:38.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:38 smithi134 ceph-mon[34283]: from='mgr.? 172.21.15.134:0/3558179159' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-13T18:14:38.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:38 smithi134 ceph-mon[34283]: from='mgr.? 172.21.15.134:0/3558179159' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-13T18:14:38.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:38 smithi204 ceph-mon[34777]: Standby manager daemon y started 2024-09-13T18:14:38.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:38 smithi204 ceph-mon[34777]: from='mgr.? 172.21.15.134:0/3558179159' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-13T18:14:38.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:38 smithi204 ceph-mon[34777]: from='mgr.? 172.21.15.134:0/3558179159' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-13T18:14:38.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:38 smithi204 ceph-mon[34777]: from='mgr.? 172.21.15.134:0/3558179159' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-13T18:14:38.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:38 smithi204 ceph-mon[34777]: from='mgr.? 172.21.15.134:0/3558179159' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-13T18:14:38.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:38 smithi114 ceph-mon[30897]: Standby manager daemon y started 2024-09-13T18:14:38.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:38 smithi114 ceph-mon[30897]: from='mgr.? 172.21.15.134:0/3558179159' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-13T18:14:38.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:38 smithi114 ceph-mon[30897]: from='mgr.? 172.21.15.134:0/3558179159' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-13T18:14:38.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:38 smithi114 ceph-mon[30897]: from='mgr.? 172.21.15.134:0/3558179159' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-13T18:14:38.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:38 smithi114 ceph-mon[30897]: from='mgr.? 172.21.15.134:0/3558179159' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-13T18:14:39.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:39 smithi134 ceph-mon[34283]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:39.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:39 smithi134 ceph-mon[34283]: mgrmap e15: x(active, since 72s), standbys: z, y 2024-09-13T18:14:39.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:39 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-13T18:14:39.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:39 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/1299277222' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9914f01a-9f6f-4878-b955-83570199c354"}]: dispatch 2024-09-13T18:14:39.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:39 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/1299277222' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9914f01a-9f6f-4878-b955-83570199c354"}]': finished 2024-09-13T18:14:39.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:39 smithi134 ceph-mon[34283]: osdmap e5: 1 total, 0 up, 1 in 2024-09-13T18:14:39.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:39 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T18:14:39.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:39 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/1593845217' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T18:14:39.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:39 smithi204 ceph-mon[34777]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:39.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:39 smithi204 ceph-mon[34777]: mgrmap e15: x(active, since 72s), standbys: z, y 2024-09-13T18:14:39.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:39 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-13T18:14:39.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:39 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/1299277222' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9914f01a-9f6f-4878-b955-83570199c354"}]: dispatch 2024-09-13T18:14:39.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:39 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/1299277222' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9914f01a-9f6f-4878-b955-83570199c354"}]': finished 2024-09-13T18:14:39.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:39 smithi204 ceph-mon[34777]: osdmap e5: 1 total, 0 up, 1 in 2024-09-13T18:14:39.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:39 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T18:14:39.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:39 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/1593845217' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T18:14:39.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:39 smithi114 ceph-mon[30897]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:39.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:39 smithi114 ceph-mon[30897]: mgrmap e15: x(active, since 72s), standbys: z, y 2024-09-13T18:14:39.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:39 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-13T18:14:39.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:39 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/1299277222' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9914f01a-9f6f-4878-b955-83570199c354"}]: dispatch 2024-09-13T18:14:39.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:39 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/1299277222' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9914f01a-9f6f-4878-b955-83570199c354"}]': finished 2024-09-13T18:14:39.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:39 smithi114 ceph-mon[30897]: osdmap e5: 1 total, 0 up, 1 in 2024-09-13T18:14:39.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:39 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T18:14:39.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:39 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/1593845217' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T18:14:41.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:41 smithi134 ceph-mon[34283]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:41.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:41 smithi204 ceph-mon[34777]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:41.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:41 smithi114 ceph-mon[30897]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:43.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:43 smithi134 ceph-mon[34283]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:43.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:43 smithi204 ceph-mon[34777]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:43.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:43 smithi114 ceph-mon[30897]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:44.197 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:44 smithi114 ceph-mon[30897]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:44.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:44 smithi134 ceph-mon[34283]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:44.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:44 smithi204 ceph-mon[34777]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:45.411 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:45 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-13T18:14:45.411 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:45 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:45.411 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:45 smithi114 ceph-mon[30897]: Deploying daemon osd.0 on smithi114 2024-09-13T18:14:45.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:45 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-13T18:14:45.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:45 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:45.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:45 smithi204 ceph-mon[34777]: Deploying daemon osd.0 on smithi114 2024-09-13T18:14:45.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:45 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-13T18:14:45.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:45 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:45.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:45 smithi134 ceph-mon[34283]: Deploying daemon osd.0 on smithi114 2024-09-13T18:14:46.325 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:46 smithi114 ceph-mon[30897]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:46.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:46 smithi204 ceph-mon[34777]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:46.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:46 smithi134 ceph-mon[34283]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:49.054 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:48 smithi114 ceph-mon[30897]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:49.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:48 smithi204 ceph-mon[34777]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:49.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:48 smithi134 ceph-mon[34283]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:50.194 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:49 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:14:50.194 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:49 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:50.194 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:49 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:50.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:49 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:14:50.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:49 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:50.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:49 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:50.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:49 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:14:50.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:49 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:50.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:49 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:51.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:50 smithi204 ceph-mon[34777]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:51.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:50 smithi114 ceph-mon[30897]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:51.368 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:50 smithi134 ceph-mon[34283]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:52.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:52.223+0000 7fd80f7fe640 1 -- 172.21.15.114:0/1723149435 <== mgr.14148 v2:172.21.15.114:6800/3396185541 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd81802c520 con 0x7fd804038930 2024-09-13T18:14:52.225 INFO:teuthology.orchestra.run.smithi114.stdout:Created osd(s) 0 on host 'smithi114' 2024-09-13T18:14:52.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:52.225+0000 7fd81cc4d640 1 -- 172.21.15.114:0/1723149435 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fd804038930 msgr2=0x7fd80403adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:52.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:52.225+0000 7fd81cc4d640 1 --2- 172.21.15.114:0/1723149435 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fd804038930 0x7fd80403adf0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd800000c00 tx=0x7fd800005a40 comp rx=0 tx=0).stop 2024-09-13T18:14:52.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:52.225+0000 7fd81cc4d640 1 -- 172.21.15.114:0/1723149435 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8180fed00 msgr2=0x7fd818069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:52.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:52.225+0000 7fd81cc4d640 1 --2- 172.21.15.114:0/1723149435 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8180fed00 0x7fd818069280 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd80800d950 tx=0x7fd80800de10 comp rx=0 tx=0).stop 2024-09-13T18:14:52.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:52.225+0000 7fd81cc4d640 1 -- 172.21.15.114:0/1723149435 shutdown_connections 2024-09-13T18:14:52.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:52.225+0000 7fd81cc4d640 1 --2- 172.21.15.114:0/1723149435 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fd804038930 0x7fd80403adf0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:52.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:52.225+0000 7fd81cc4d640 1 --2- 172.21.15.114:0/1723149435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8181008c0 0x7fd81819d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:52.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:52.225+0000 7fd81cc4d640 1 --2- 172.21.15.114:0/1723149435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8180fff00 0x7fd81819aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:52.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:52.225+0000 7fd81cc4d640 1 --2- 172.21.15.114:0/1723149435 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8180fed00 0x7fd818069280 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:52.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:52.225+0000 7fd81cc4d640 1 -- 172.21.15.114:0/1723149435 >> 172.21.15.114:0/1723149435 conn(0x7fd8180fa490 msgr2=0x7fd818103410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:14:52.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:52.225+0000 7fd81cc4d640 1 -- 172.21.15.114:0/1723149435 shutdown_connections 2024-09-13T18:14:52.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:14:52.225+0000 7fd81cc4d640 1 -- 172.21.15.114:0/1723149435 wait complete. 2024-09-13T18:14:52.855 DEBUG:teuthology.orchestra.run.smithi114:osd.0> sudo journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@osd.0.service 2024-09-13T18:14:52.859 INFO:tasks.cephadm:Deploying osd.1 on smithi134 with /dev/vg_nvme/lv_4... 2024-09-13T18:14:52.859 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-09-13T18:14:53.064 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T18:14:53.180 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:52 smithi134 ceph-mon[34283]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:53.180 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:52 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:53.180 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:52 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:53.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:52 smithi204 ceph-mon[34777]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:53.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:52 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:53.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:52 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:53.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:52 smithi114 ceph-mon[30897]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:53.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:52 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:53.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:52 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:53.831 INFO:journalctl@ceph.osd.0.smithi114.stdout:Sep 13 18:14:53 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-0[44930]: 2024-09-13T18:14:53.514+0000 7fde31c61640 -1 osd.0 0 log_to_monitors true 2024-09-13T18:14:54.440 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:54 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:54.440 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:54 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:54.440 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:54 smithi134 ceph-mon[34283]: from='osd.0 [v2:172.21.15.114:6802/3499525826,v1:172.21.15.114:6803/3499525826]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-13T18:14:54.440 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:54 smithi134 ceph-mon[34283]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:54.559 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:54 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:54.559 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:54 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:54.559 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:54 smithi114 ceph-mon[30897]: from='osd.0 [v2:172.21.15.114:6802/3499525826,v1:172.21.15.114:6803/3499525826]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-13T18:14:54.559 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:54 smithi114 ceph-mon[30897]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:54.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:54 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:54.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:54 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:54.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:54 smithi204 ceph-mon[34777]: from='osd.0 [v2:172.21.15.114:6802/3499525826,v1:172.21.15.114:6803/3499525826]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-13T18:14:54.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:54 smithi204 ceph-mon[34777]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:54.707 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-09-13T18:14:54.797 INFO:teuthology.orchestra.run.smithi134.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-09-13T18:14:54.808 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10+0 records in 2024-09-13T18:14:54.808 INFO:teuthology.orchestra.run.smithi134.stderr:10+0 records out 2024-09-13T18:14:54.808 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114754 s, 914 MB/s 2024-09-13T18:14:54.809 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping successful for: 2024-09-13T18:14:55.391 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi134:vg_nvme/lv_4 2024-09-13T18:14:55.505 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:55 smithi114 ceph-mon[30897]: from='osd.0 [v2:172.21.15.114:6802/3499525826,v1:172.21.15.114:6803/3499525826]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-13T18:14:55.505 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:55 smithi114 ceph-mon[30897]: osdmap e6: 1 total, 0 up, 1 in 2024-09-13T18:14:55.505 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:55 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T18:14:55.505 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:55 smithi114 ceph-mon[30897]: from='osd.0 [v2:172.21.15.114:6802/3499525826,v1:172.21.15.114:6803/3499525826]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-09-13T18:14:55.591 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T18:14:55.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:55 smithi134 ceph-mon[34283]: from='osd.0 [v2:172.21.15.114:6802/3499525826,v1:172.21.15.114:6803/3499525826]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-13T18:14:55.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:55 smithi134 ceph-mon[34283]: osdmap e6: 1 total, 0 up, 1 in 2024-09-13T18:14:55.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:55 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T18:14:55.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:55 smithi134 ceph-mon[34283]: from='osd.0 [v2:172.21.15.114:6802/3499525826,v1:172.21.15.114:6803/3499525826]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-09-13T18:14:55.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:55 smithi204 ceph-mon[34777]: from='osd.0 [v2:172.21.15.114:6802/3499525826,v1:172.21.15.114:6803/3499525826]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-13T18:14:55.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:55 smithi204 ceph-mon[34777]: osdmap e6: 1 total, 0 up, 1 in 2024-09-13T18:14:55.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:55 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T18:14:55.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:55 smithi204 ceph-mon[34777]: from='osd.0 [v2:172.21.15.114:6802/3499525826,v1:172.21.15.114:6803/3499525826]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-09-13T18:14:55.810 INFO:journalctl@ceph.osd.0.smithi114.stdout:Sep 13 18:14:55 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-0[44930]: 2024-09-13T18:14:55.657+0000 7fde2e3f9640 -1 osd.0 0 waiting for initial osdmap 2024-09-13T18:14:55.810 INFO:journalctl@ceph.osd.0.smithi114.stdout:Sep 13 18:14:55 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-0[44930]: 2024-09-13T18:14:55.663+0000 7fde25a03640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T18:14:56.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:56 smithi114 ceph-mon[30897]: from='osd.0 [v2:172.21.15.114:6802/3499525826,v1:172.21.15.114:6803/3499525826]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-09-13T18:14:56.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:56 smithi114 ceph-mon[30897]: osdmap e7: 1 total, 0 up, 1 in 2024-09-13T18:14:56.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:56 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T18:14:56.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:56 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T18:14:56.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:56 smithi114 ceph-mon[30897]: from='osd.0 [v2:172.21.15.114:6802/3499525826,v1:172.21.15.114:6803/3499525826]' entity='osd.0' 2024-09-13T18:14:56.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:56 smithi114 ceph-mon[30897]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:56.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:56 smithi114 ceph-mon[30897]: Detected new or changed devices on smithi114 2024-09-13T18:14:56.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:56 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:56.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:56 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:56.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:56 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:14:56.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:56 smithi114 ceph-mon[30897]: Adjusting osd_memory_target on smithi114 to 17088M 2024-09-13T18:14:56.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:56 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:56.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:56 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:56.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:56 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:14:56.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:56 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:56.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:56 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:14:56.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:56 smithi134 ceph-mon[34283]: from='osd.0 [v2:172.21.15.114:6802/3499525826,v1:172.21.15.114:6803/3499525826]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-09-13T18:14:56.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:56 smithi134 ceph-mon[34283]: osdmap e7: 1 total, 0 up, 1 in 2024-09-13T18:14:56.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:56 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T18:14:56.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:56 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T18:14:56.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:56 smithi134 ceph-mon[34283]: from='osd.0 [v2:172.21.15.114:6802/3499525826,v1:172.21.15.114:6803/3499525826]' entity='osd.0' 2024-09-13T18:14:56.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:56 smithi134 ceph-mon[34283]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:56.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:56 smithi134 ceph-mon[34283]: Detected new or changed devices on smithi114 2024-09-13T18:14:56.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:56 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:56.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:56 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:56.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:56 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:14:56.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:56 smithi134 ceph-mon[34283]: Adjusting osd_memory_target on smithi114 to 17088M 2024-09-13T18:14:56.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:56 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:56.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:56 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:56.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:56 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:14:56.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:56 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:56.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:56 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:14:56.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:56 smithi204 ceph-mon[34777]: from='osd.0 [v2:172.21.15.114:6802/3499525826,v1:172.21.15.114:6803/3499525826]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-09-13T18:14:56.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:56 smithi204 ceph-mon[34777]: osdmap e7: 1 total, 0 up, 1 in 2024-09-13T18:14:56.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:56 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T18:14:56.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:56 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T18:14:56.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:56 smithi204 ceph-mon[34777]: from='osd.0 [v2:172.21.15.114:6802/3499525826,v1:172.21.15.114:6803/3499525826]' entity='osd.0' 2024-09-13T18:14:56.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:56 smithi204 ceph-mon[34777]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-13T18:14:56.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:56 smithi204 ceph-mon[34777]: Detected new or changed devices on smithi114 2024-09-13T18:14:56.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:56 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:56.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:56 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:56.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:56 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:14:56.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:56 smithi204 ceph-mon[34777]: Adjusting osd_memory_target on smithi114 to 17088M 2024-09-13T18:14:56.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:56 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:56.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:56 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:56.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:56 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:14:56.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:56 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:56.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:56 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:14:57.136 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:14:57.131+0000 7fd9354a7640 1 -- 172.21.15.134:0/1913631574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd930100890 msgr2=0x7fd930102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:57.136 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:14:57.131+0000 7fd9354a7640 1 --2- 172.21.15.134:0/1913631574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd930100890 0x7fd930102c80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd920009900 tx=0x7fd92002a410 comp rx=0 tx=0).stop 2024-09-13T18:14:57.136 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:14:57.131+0000 7fd9354a7640 1 -- 172.21.15.134:0/1913631574 shutdown_connections 2024-09-13T18:14:57.136 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:14:57.131+0000 7fd9354a7640 1 --2- 172.21.15.134:0/1913631574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd930105af0 0x7fd930107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:57.136 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:14:57.131+0000 7fd9354a7640 1 --2- 172.21.15.134:0/1913631574 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9301031c0 0x7fd9301055b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:57.137 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:14:57.131+0000 7fd9354a7640 1 --2- 172.21.15.134:0/1913631574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd930100890 0x7fd930102c80 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:57.137 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:14:57.131+0000 7fd9354a7640 1 -- 172.21.15.134:0/1913631574 >> 172.21.15.134:0/1913631574 conn(0x7fd9300fa490 msgr2=0x7fd9300fc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:14:57.137 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:14:57.131+0000 7fd9354a7640 1 -- 172.21.15.134:0/1913631574 shutdown_connections 2024-09-13T18:14:57.137 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:14:57.131+0000 7fd9354a7640 1 -- 172.21.15.134:0/1913631574 wait complete. 2024-09-13T18:14:57.137 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:14:57.132+0000 7fd9354a7640 1 Processor -- start 2024-09-13T18:14:57.137 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:14:57.132+0000 7fd9354a7640 1 -- start start 2024-09-13T18:14:57.137 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:14:57.132+0000 7fd9354a7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd930100890 0x7fd930195d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:57.137 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:14:57.132+0000 7fd9354a7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9301031c0 0x7fd930196260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:57.137 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:14:57.132+0000 7fd9354a7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd930105af0 0x7fd93019d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:57.137 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:14:57.132+0000 7fd9354a7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd93019d810 con 0x7fd9301031c0 2024-09-13T18:14:57.137 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:14:57.132+0000 7fd9354a7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd93019d980 con 0x7fd930105af0 2024-09-13T18:14:57.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:14:57.132+0000 7fd9354a7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd93019db40 con 0x7fd930100890 2024-09-13T18:14:57.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:14:57.132+0000 7fd92effd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd930100890 0x7fd930195d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:57.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:14:57.132+0000 7fd92f7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd930105af0 0x7fd93019d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:57.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:14:57.132+0000 7fd92effd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd930100890 0x7fd930195d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.134:41232/0 (socket says 172.21.15.134:41232) 2024-09-13T18:14:57.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:14:57.132+0000 7fd92effd640 1 -- 172.21.15.134:0/2390519774 learned_addr learned my addr 172.21.15.134:0/2390519774 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-13T18:14:57.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:14:57.133+0000 7fd92e7fc640 1 --2- 172.21.15.134:0/2390519774 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9301031c0 0x7fd930196260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:57.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:14:57.133+0000 7fd92f7fe640 1 -- 172.21.15.134:0/2390519774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd930100890 msgr2=0x7fd930195d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:57.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:14:57.133+0000 7fd92f7fe640 1 --2- 172.21.15.134:0/2390519774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd930100890 0x7fd930195d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:57.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:14:57.133+0000 7fd92f7fe640 1 -- 172.21.15.134:0/2390519774 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9301031c0 msgr2=0x7fd930196260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:57.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:14:57.133+0000 7fd92f7fe640 1 --2- 172.21.15.134:0/2390519774 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9301031c0 0x7fd930196260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:57.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:14:57.133+0000 7fd92f7fe640 1 -- 172.21.15.134:0/2390519774 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd920009560 con 0x7fd930105af0 2024-09-13T18:14:57.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:14:57.133+0000 7fd92f7fe640 1 --2- 172.21.15.134:0/2390519774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd930105af0 0x7fd93019d2a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd928009fd0 tx=0x7fd92800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:57.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:14:57.133+0000 7fd917fff640 1 -- 172.21.15.134:0/2390519774 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92800ce20 con 0x7fd930105af0 2024-09-13T18:14:57.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:14:57.133+0000 7fd9354a7640 1 -- 172.21.15.134:0/2390519774 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd93019de20 con 0x7fd930105af0 2024-09-13T18:14:57.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:14:57.133+0000 7fd917fff640 1 -- 172.21.15.134:0/2390519774 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd928005020 con 0x7fd930105af0 2024-09-13T18:14:57.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:14:57.133+0000 7fd917fff640 1 -- 172.21.15.134:0/2390519774 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd928010690 con 0x7fd930105af0 2024-09-13T18:14:57.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:14:57.133+0000 7fd9354a7640 1 -- 172.21.15.134:0/2390519774 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd93019e230 con 0x7fd930105af0 2024-09-13T18:14:57.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:14:57.134+0000 7fd9354a7640 1 -- 172.21.15.134:0/2390519774 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd930103610 con 0x7fd930105af0 2024-09-13T18:14:57.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:14:57.135+0000 7fd917fff640 1 -- 172.21.15.134:0/2390519774 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 15) v1 ==== 135818+0+0 (secure 0 0 0) 0x7fd928005190 con 0x7fd930105af0 2024-09-13T18:14:57.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:14:57.136+0000 7fd917fff640 1 --2- 172.21.15.134:0/2390519774 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fd9180a0ff0 0x7fd9180a34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:57.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:14:57.136+0000 7fd917fff640 1 -- 172.21.15.134:0/2390519774 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1767+0+0 (secure 0 0 0) 0x7fd928014070 con 0x7fd930105af0 2024-09-13T18:14:57.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:14:57.136+0000 7fd92effd640 1 --2- 172.21.15.134:0/2390519774 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fd9180a0ff0 0x7fd9180a34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:57.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:14:57.137+0000 7fd92effd640 1 --2- 172.21.15.134:0/2390519774 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fd9180a0ff0 0x7fd9180a34b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd920005ec0 tx=0x7fd920005720 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:57.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:14:57.140+0000 7fd917fff640 1 -- 172.21.15.134:0/2390519774 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fd930103610 con 0x7fd930105af0 2024-09-13T18:14:57.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:14:57.312+0000 7fd9354a7640 1 -- 172.21.15.134:0/2390519774 --> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fd9301035a0 con 0x7fd9180a0ff0 2024-09-13T18:14:57.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:57 smithi134 ceph-mon[34283]: purged_snaps scrub starts 2024-09-13T18:14:57.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:57 smithi134 ceph-mon[34283]: purged_snaps scrub ok 2024-09-13T18:14:57.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:57 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T18:14:57.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:57 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:57.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:57 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:14:57.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:57 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:57.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:57 smithi134 ceph-mon[34283]: osd.0 [v2:172.21.15.114:6802/3499525826,v1:172.21.15.114:6803/3499525826] boot 2024-09-13T18:14:57.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:57 smithi134 ceph-mon[34283]: osdmap e8: 1 total, 1 up, 1 in 2024-09-13T18:14:57.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:57 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T18:14:57.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:57 smithi204 ceph-mon[34777]: purged_snaps scrub starts 2024-09-13T18:14:57.696 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:57 smithi204 ceph-mon[34777]: purged_snaps scrub ok 2024-09-13T18:14:57.696 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:57 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T18:14:57.696 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:57 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:57.696 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:57 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:14:57.696 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:57 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:57.696 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:57 smithi204 ceph-mon[34777]: osd.0 [v2:172.21.15.114:6802/3499525826,v1:172.21.15.114:6803/3499525826] boot 2024-09-13T18:14:57.697 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:57 smithi204 ceph-mon[34777]: osdmap e8: 1 total, 1 up, 1 in 2024-09-13T18:14:57.697 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:57 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T18:14:57.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:57 smithi114 ceph-mon[30897]: purged_snaps scrub starts 2024-09-13T18:14:57.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:57 smithi114 ceph-mon[30897]: purged_snaps scrub ok 2024-09-13T18:14:57.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:57 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T18:14:57.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:57 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:57.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:57 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:14:57.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:57 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:14:57.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:57 smithi114 ceph-mon[30897]: osd.0 [v2:172.21.15.114:6802/3499525826,v1:172.21.15.114:6803/3499525826] boot 2024-09-13T18:14:57.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:57 smithi114 ceph-mon[30897]: osdmap e8: 1 total, 1 up, 1 in 2024-09-13T18:14:57.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:57 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T18:14:58.585 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:58 smithi134 ceph-mon[34283]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:14:58.585 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:58 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T18:14:58.585 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:58 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T18:14:58.586 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:58 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:58.586 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:58 smithi134 ceph-mon[34283]: pgmap v44: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-13T18:14:58.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:58 smithi204 ceph-mon[34777]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:14:58.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:58 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T18:14:58.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:58 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T18:14:58.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:58 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:58.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:58 smithi204 ceph-mon[34777]: pgmap v44: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-13T18:14:58.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:58 smithi114 ceph-mon[30897]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:14:58.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:58 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T18:14:58.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:58 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T18:14:58.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:58 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:14:58.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:58 smithi114 ceph-mon[30897]: pgmap v44: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-13T18:14:59.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:14:59 smithi134 ceph-mon[34283]: osdmap e9: 1 total, 1 up, 1 in 2024-09-13T18:14:59.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:14:59 smithi204 ceph-mon[34777]: osdmap e9: 1 total, 1 up, 1 in 2024-09-13T18:14:59.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:14:59 smithi114 ceph-mon[30897]: osdmap e9: 1 total, 1 up, 1 in 2024-09-13T18:15:00.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:00 smithi134 ceph-mon[34283]: from='client.? 172.21.15.134:0/3211087787' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef65e012-7469-4c20-b354-17022bca63c5"}]: dispatch 2024-09-13T18:15:00.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:00 smithi134 ceph-mon[34283]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef65e012-7469-4c20-b354-17022bca63c5"}]: dispatch 2024-09-13T18:15:00.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:00 smithi134 ceph-mon[34283]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ef65e012-7469-4c20-b354-17022bca63c5"}]': finished 2024-09-13T18:15:00.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:00 smithi134 ceph-mon[34283]: osdmap e10: 2 total, 1 up, 2 in 2024-09-13T18:15:00.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:00 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T18:15:00.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:00 smithi134 ceph-mon[34283]: pgmap v47: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-13T18:15:00.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:00 smithi134 ceph-mon[34283]: from='client.? 172.21.15.134:0/3050480528' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T18:15:00.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:00 smithi204 ceph-mon[34777]: from='client.? 172.21.15.134:0/3211087787' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef65e012-7469-4c20-b354-17022bca63c5"}]: dispatch 2024-09-13T18:15:00.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:00 smithi204 ceph-mon[34777]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef65e012-7469-4c20-b354-17022bca63c5"}]: dispatch 2024-09-13T18:15:00.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:00 smithi204 ceph-mon[34777]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ef65e012-7469-4c20-b354-17022bca63c5"}]': finished 2024-09-13T18:15:00.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:00 smithi204 ceph-mon[34777]: osdmap e10: 2 total, 1 up, 2 in 2024-09-13T18:15:00.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:00 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T18:15:00.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:00 smithi204 ceph-mon[34777]: pgmap v47: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-13T18:15:00.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:00 smithi204 ceph-mon[34777]: from='client.? 172.21.15.134:0/3050480528' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T18:15:00.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:00 smithi114 ceph-mon[30897]: from='client.? 172.21.15.134:0/3211087787' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef65e012-7469-4c20-b354-17022bca63c5"}]: dispatch 2024-09-13T18:15:00.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:00 smithi114 ceph-mon[30897]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef65e012-7469-4c20-b354-17022bca63c5"}]: dispatch 2024-09-13T18:15:00.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:00 smithi114 ceph-mon[30897]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ef65e012-7469-4c20-b354-17022bca63c5"}]': finished 2024-09-13T18:15:00.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:00 smithi114 ceph-mon[30897]: osdmap e10: 2 total, 1 up, 2 in 2024-09-13T18:15:00.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:00 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T18:15:00.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:00 smithi114 ceph-mon[30897]: pgmap v47: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-13T18:15:00.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:00 smithi114 ceph-mon[30897]: from='client.? 172.21.15.134:0/3050480528' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T18:15:03.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:02 smithi204 ceph-mon[34777]: pgmap v48: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-13T18:15:03.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:02 smithi114 ceph-mon[30897]: pgmap v48: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-13T18:15:03.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:02 smithi134 ceph-mon[34283]: pgmap v48: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-13T18:15:05.109 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:04 smithi134 ceph-mon[34283]: pgmap v49: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-13T18:15:05.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:04 smithi204 ceph-mon[34777]: pgmap v49: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-13T18:15:05.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:04 smithi114 ceph-mon[30897]: pgmap v49: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-13T18:15:07.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:06 smithi134 ceph-mon[34283]: pgmap v50: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-13T18:15:07.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:06 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-13T18:15:07.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:06 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:15:07.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:06 smithi134 ceph-mon[34283]: Deploying daemon osd.1 on smithi134 2024-09-13T18:15:07.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:06 smithi204 ceph-mon[34777]: pgmap v50: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-13T18:15:07.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:06 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-13T18:15:07.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:06 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:15:07.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:06 smithi204 ceph-mon[34777]: Deploying daemon osd.1 on smithi134 2024-09-13T18:15:07.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:06 smithi114 ceph-mon[30897]: pgmap v50: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-13T18:15:07.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:06 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-13T18:15:07.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:06 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:15:07.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:06 smithi114 ceph-mon[30897]: Deploying daemon osd.1 on smithi134 2024-09-13T18:15:09.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:08 smithi134 ceph-mon[34283]: pgmap v51: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-13T18:15:09.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:08 smithi204 ceph-mon[34777]: pgmap v51: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-13T18:15:09.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:08 smithi114 ceph-mon[30897]: pgmap v51: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-13T18:15:11.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:10 smithi204 ceph-mon[34777]: pgmap v52: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-13T18:15:11.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:10 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:15:11.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:10 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:11.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:10 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:11.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:10 smithi114 ceph-mon[30897]: pgmap v52: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-13T18:15:11.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:10 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:15:11.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:10 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:11.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:10 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:11.281 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:10 smithi134 ceph-mon[34283]: pgmap v52: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-13T18:15:11.281 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:10 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:15:11.281 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:10 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:11.281 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:10 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:13.114 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:12 smithi134 ceph-mon[34283]: pgmap v53: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-13T18:15:13.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:12 smithi204 ceph-mon[34777]: pgmap v53: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-13T18:15:13.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:12 smithi114 ceph-mon[30897]: pgmap v53: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-13T18:15:14.183 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:13 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:14.183 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:13 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:14.183 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:13 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:15:14.183 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:13 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:15:14.183 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:13 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:14.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:13 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:14.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:13 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:14.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:13 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:15:14.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:13 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:15:14.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:13 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:14.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:15:14.212+0000 7fd917fff640 1 -- 172.21.15.134:0/2390519774 <== mgr.14148 v2:172.21.15.114:6800/3396185541 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd9301035a0 con 0x7fd9180a0ff0 2024-09-13T18:15:14.215 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 1 on host 'smithi134' 2024-09-13T18:15:14.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:15:14.216+0000 7fd9354a7640 1 -- 172.21.15.134:0/2390519774 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fd9180a0ff0 msgr2=0x7fd9180a34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:14.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:15:14.216+0000 7fd9354a7640 1 --2- 172.21.15.134:0/2390519774 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fd9180a0ff0 0x7fd9180a34b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd920005ec0 tx=0x7fd920005720 comp rx=0 tx=0).stop 2024-09-13T18:15:14.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:15:14.216+0000 7fd9354a7640 1 -- 172.21.15.134:0/2390519774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd930105af0 msgr2=0x7fd93019d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:14.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:15:14.216+0000 7fd9354a7640 1 --2- 172.21.15.134:0/2390519774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd930105af0 0x7fd93019d2a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd928009fd0 tx=0x7fd92800ef10 comp rx=0 tx=0).stop 2024-09-13T18:15:14.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:15:14.216+0000 7fd9354a7640 1 -- 172.21.15.134:0/2390519774 shutdown_connections 2024-09-13T18:15:14.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:15:14.216+0000 7fd9354a7640 1 --2- 172.21.15.134:0/2390519774 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fd9180a0ff0 0x7fd9180a34b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:14.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:15:14.216+0000 7fd9354a7640 1 --2- 172.21.15.134:0/2390519774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd930105af0 0x7fd93019d2a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:14.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:15:14.216+0000 7fd9354a7640 1 --2- 172.21.15.134:0/2390519774 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9301031c0 0x7fd930196260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:14.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:15:14.216+0000 7fd9354a7640 1 --2- 172.21.15.134:0/2390519774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd930100890 0x7fd930195d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:14.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:15:14.216+0000 7fd9354a7640 1 -- 172.21.15.134:0/2390519774 >> 172.21.15.134:0/2390519774 conn(0x7fd9300fa490 msgr2=0x7fd9301010f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:15:14.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:15:14.216+0000 7fd9354a7640 1 -- 172.21.15.134:0/2390519774 shutdown_connections 2024-09-13T18:15:14.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:15:14.216+0000 7fd9354a7640 1 -- 172.21.15.134:0/2390519774 wait complete. 2024-09-13T18:15:14.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:13 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:14.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:13 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:14.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:13 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:15:14.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:13 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:15:14.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:13 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:14.882 DEBUG:teuthology.orchestra.run.smithi134:osd.1> sudo journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@osd.1.service 2024-09-13T18:15:14.885 INFO:tasks.cephadm:Deploying osd.2 on smithi204 with /dev/vg_nvme/lv_4... 2024-09-13T18:15:14.885 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-09-13T18:15:15.091 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T18:15:15.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:14 smithi134 ceph-mon[34283]: pgmap v54: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-13T18:15:15.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:14 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:15:15.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:14 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:15.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:14 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:15.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:14 smithi134 ceph-mon[34283]: from='osd.1 [v2:172.21.15.134:6800/4264264866,v1:172.21.15.134:6801/4264264866]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-13T18:15:15.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:14 smithi134 ceph-mon[34283]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-13T18:15:15.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:14 smithi204 ceph-mon[34777]: pgmap v54: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-13T18:15:15.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:14 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:15:15.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:14 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:15.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:14 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:15.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:14 smithi204 ceph-mon[34777]: from='osd.1 [v2:172.21.15.134:6800/4264264866,v1:172.21.15.134:6801/4264264866]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-13T18:15:15.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:14 smithi204 ceph-mon[34777]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-13T18:15:15.219 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:14 smithi114 ceph-mon[30897]: pgmap v54: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-13T18:15:15.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:14 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:15:15.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:14 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:15.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:14 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:15.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:14 smithi114 ceph-mon[30897]: from='osd.1 [v2:172.21.15.134:6800/4264264866,v1:172.21.15.134:6801/4264264866]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-13T18:15:15.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:14 smithi114 ceph-mon[30897]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-13T18:15:16.428 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:16 smithi204 ceph-mon[34777]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-13T18:15:16.429 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:16 smithi204 ceph-mon[34777]: osdmap e11: 2 total, 1 up, 2 in 2024-09-13T18:15:16.429 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:16 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T18:15:16.429 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:16 smithi204 ceph-mon[34777]: from='osd.1 [v2:172.21.15.134:6800/4264264866,v1:172.21.15.134:6801/4264264866]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-13T18:15:16.429 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:16 smithi204 ceph-mon[34777]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-13T18:15:16.429 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:16 smithi204 ceph-mon[34777]: pgmap v56: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-13T18:15:16.531 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:16 smithi134 ceph-mon[34283]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-13T18:15:16.531 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:16 smithi134 ceph-mon[34283]: osdmap e11: 2 total, 1 up, 2 in 2024-09-13T18:15:16.532 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:16 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T18:15:16.532 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:16 smithi134 ceph-mon[34283]: from='osd.1 [v2:172.21.15.134:6800/4264264866,v1:172.21.15.134:6801/4264264866]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-13T18:15:16.532 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:16 smithi134 ceph-mon[34283]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-13T18:15:16.532 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:16 smithi134 ceph-mon[34283]: pgmap v56: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-13T18:15:16.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:16 smithi114 ceph-mon[30897]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-13T18:15:16.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:16 smithi114 ceph-mon[30897]: osdmap e11: 2 total, 1 up, 2 in 2024-09-13T18:15:16.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:16 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T18:15:16.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:16 smithi114 ceph-mon[30897]: from='osd.1 [v2:172.21.15.134:6800/4264264866,v1:172.21.15.134:6801/4264264866]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-13T18:15:16.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:16 smithi114 ceph-mon[30897]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-13T18:15:16.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:16 smithi114 ceph-mon[30897]: pgmap v56: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-13T18:15:16.797 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-09-13T18:15:16.869 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 13 18:15:16 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-1[38780]: 2024-09-13T18:15:16.618+0000 7fa920207640 -1 osd.1 0 waiting for initial osdmap 2024-09-13T18:15:16.869 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 13 18:15:16 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-1[38780]: 2024-09-13T18:15:16.623+0000 7fa918028640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T18:15:16.881 INFO:teuthology.orchestra.run.smithi204.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-09-13T18:15:16.894 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10+0 records in 2024-09-13T18:15:16.894 INFO:teuthology.orchestra.run.smithi204.stderr:10+0 records out 2024-09-13T18:15:16.894 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111785 s, 938 MB/s 2024-09-13T18:15:16.894 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping successful for: 2024-09-13T18:15:17.343 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:17 smithi134 ceph-mon[34283]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-13T18:15:17.343 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:17 smithi134 ceph-mon[34283]: osdmap e12: 2 total, 1 up, 2 in 2024-09-13T18:15:17.343 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:17 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T18:15:17.343 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:17 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T18:15:17.343 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:17 smithi134 ceph-mon[34283]: from='osd.1 ' entity='osd.1' 2024-09-13T18:15:17.513 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:17 smithi204 ceph-mon[34777]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-13T18:15:17.513 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:17 smithi204 ceph-mon[34777]: osdmap e12: 2 total, 1 up, 2 in 2024-09-13T18:15:17.513 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:17 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T18:15:17.513 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:17 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T18:15:17.513 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:17 smithi204 ceph-mon[34777]: from='osd.1 ' entity='osd.1' 2024-09-13T18:15:17.534 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi204:vg_nvme/lv_4 2024-09-13T18:15:17.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:17 smithi114 ceph-mon[30897]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-13T18:15:17.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:17 smithi114 ceph-mon[30897]: osdmap e12: 2 total, 1 up, 2 in 2024-09-13T18:15:17.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:17 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T18:15:17.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:17 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T18:15:17.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:17 smithi114 ceph-mon[30897]: from='osd.1 ' entity='osd.1' 2024-09-13T18:15:17.737 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T18:15:18.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:18 smithi204 ceph-mon[34777]: purged_snaps scrub starts 2024-09-13T18:15:18.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:18 smithi204 ceph-mon[34777]: purged_snaps scrub ok 2024-09-13T18:15:18.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:18 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T18:15:18.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:18 smithi204 ceph-mon[34777]: osd.1 [v2:172.21.15.134:6800/4264264866,v1:172.21.15.134:6801/4264264866] boot 2024-09-13T18:15:18.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:18 smithi204 ceph-mon[34777]: osdmap e13: 2 total, 2 up, 2 in 2024-09-13T18:15:18.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:18 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T18:15:18.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:18 smithi204 ceph-mon[34777]: Detected new or changed devices on smithi134 2024-09-13T18:15:18.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:18 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:18.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:18 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:18.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:18 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:15:18.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:18 smithi204 ceph-mon[34777]: Adjusting osd_memory_target on smithi134 to 17088M 2024-09-13T18:15:18.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:18 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:18.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:18 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:15:18.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:18 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:15:18.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:18 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:18.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:18 smithi204 ceph-mon[34777]: pgmap v59: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-13T18:15:18.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:18 smithi134 ceph-mon[34283]: purged_snaps scrub starts 2024-09-13T18:15:18.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:18 smithi134 ceph-mon[34283]: purged_snaps scrub ok 2024-09-13T18:15:18.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:18 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T18:15:18.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:18 smithi134 ceph-mon[34283]: osd.1 [v2:172.21.15.134:6800/4264264866,v1:172.21.15.134:6801/4264264866] boot 2024-09-13T18:15:18.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:18 smithi134 ceph-mon[34283]: osdmap e13: 2 total, 2 up, 2 in 2024-09-13T18:15:18.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:18 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T18:15:18.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:18 smithi134 ceph-mon[34283]: Detected new or changed devices on smithi134 2024-09-13T18:15:18.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:18 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:18.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:18 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:18.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:18 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:15:18.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:18 smithi134 ceph-mon[34283]: Adjusting osd_memory_target on smithi134 to 17088M 2024-09-13T18:15:18.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:18 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:18.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:18 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:15:18.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:18 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:15:18.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:18 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:18.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:18 smithi134 ceph-mon[34283]: pgmap v59: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-13T18:15:18.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:18 smithi114 ceph-mon[30897]: purged_snaps scrub starts 2024-09-13T18:15:18.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:18 smithi114 ceph-mon[30897]: purged_snaps scrub ok 2024-09-13T18:15:18.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:18 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T18:15:18.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:18 smithi114 ceph-mon[30897]: osd.1 [v2:172.21.15.134:6800/4264264866,v1:172.21.15.134:6801/4264264866] boot 2024-09-13T18:15:18.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:18 smithi114 ceph-mon[30897]: osdmap e13: 2 total, 2 up, 2 in 2024-09-13T18:15:18.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:18 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T18:15:18.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:18 smithi114 ceph-mon[30897]: Detected new or changed devices on smithi134 2024-09-13T18:15:18.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:18 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:18.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:18 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:18.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:18 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:15:18.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:18 smithi114 ceph-mon[30897]: Adjusting osd_memory_target on smithi134 to 17088M 2024-09-13T18:15:18.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:18 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:18.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:18 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:15:18.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:18 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:15:18.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:18 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:18.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:18 smithi114 ceph-mon[30897]: pgmap v59: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-09-13T18:15:19.241 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:15:19.237+0000 7f21ea2f1640 1 -- 172.21.15.204:0/1351689912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21e4100890 msgr2=0x7f21e4102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:19.242 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:15:19.237+0000 7f21ea2f1640 1 --2- 172.21.15.204:0/1351689912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21e4100890 0x7f21e4102c80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f21d4009900 tx=0x7f21d402a410 comp rx=0 tx=0).stop 2024-09-13T18:15:19.242 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:15:19.237+0000 7f21ea2f1640 1 -- 172.21.15.204:0/1351689912 shutdown_connections 2024-09-13T18:15:19.242 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:15:19.237+0000 7f21ea2f1640 1 --2- 172.21.15.204:0/1351689912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21e4105af0 0x7f21e4107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:19.242 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:15:19.237+0000 7f21ea2f1640 1 --2- 172.21.15.204:0/1351689912 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f21e41031c0 0x7f21e41055b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:19.242 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:15:19.237+0000 7f21ea2f1640 1 --2- 172.21.15.204:0/1351689912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21e4100890 0x7f21e4102c80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:19.242 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:15:19.237+0000 7f21ea2f1640 1 -- 172.21.15.204:0/1351689912 >> 172.21.15.204:0/1351689912 conn(0x7f21e40fa490 msgr2=0x7f21e40fc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:15:19.242 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:15:19.237+0000 7f21ea2f1640 1 -- 172.21.15.204:0/1351689912 shutdown_connections 2024-09-13T18:15:19.242 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:15:19.237+0000 7f21ea2f1640 1 -- 172.21.15.204:0/1351689912 wait complete. 2024-09-13T18:15:19.242 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:15:19.238+0000 7f21ea2f1640 1 Processor -- start 2024-09-13T18:15:19.242 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:15:19.238+0000 7f21ea2f1640 1 -- start start 2024-09-13T18:15:19.242 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:15:19.238+0000 7f21ea2f1640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f21e4100890 0x7f21e4195d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:19.243 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:15:19.238+0000 7f21ea2f1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21e41031c0 0x7f21e4196240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:19.243 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:15:19.238+0000 7f21ea2f1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21e4105af0 0x7f21e419d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:19.243 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:15:19.238+0000 7f21ea2f1640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e419d7f0 con 0x7f21e4100890 2024-09-13T18:15:19.243 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:15:19.238+0000 7f21ea2f1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e419d960 con 0x7f21e41031c0 2024-09-13T18:15:19.243 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:15:19.238+0000 7f21ea2f1640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e419db20 con 0x7f21e4105af0 2024-09-13T18:15:19.243 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:15:19.238+0000 7f21e888e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21e4105af0 0x7f21e419d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:19.243 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:15:19.238+0000 7f21e888e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21e4105af0 0x7f21e419d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:34820/0 (socket says 172.21.15.204:34820) 2024-09-13T18:15:19.243 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:15:19.238+0000 7f21e888e640 1 -- 172.21.15.204:0/1902802804 learned_addr learned my addr 172.21.15.204:0/1902802804 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-13T18:15:19.243 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:15:19.238+0000 7f21e3fff640 1 --2- 172.21.15.204:0/1902802804 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f21e4100890 0x7f21e4195d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:19.243 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:15:19.239+0000 7f21e37fe640 1 --2- 172.21.15.204:0/1902802804 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21e41031c0 0x7f21e4196240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:19.243 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:15:19.239+0000 7f21e888e640 1 -- 172.21.15.204:0/1902802804 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21e41031c0 msgr2=0x7f21e4196240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:19.243 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:15:19.239+0000 7f21e888e640 1 --2- 172.21.15.204:0/1902802804 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21e41031c0 0x7f21e4196240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:19.243 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:15:19.239+0000 7f21e888e640 1 -- 172.21.15.204:0/1902802804 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f21e4100890 msgr2=0x7f21e4195d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:19.244 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:15:19.239+0000 7f21e888e640 1 --2- 172.21.15.204:0/1902802804 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f21e4100890 0x7f21e4195d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:19.244 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:15:19.239+0000 7f21e888e640 1 -- 172.21.15.204:0/1902802804 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d4009560 con 0x7f21e4105af0 2024-09-13T18:15:19.244 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:15:19.239+0000 7f21e888e640 1 --2- 172.21.15.204:0/1902802804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21e4105af0 0x7f21e419d280 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f21dc00cad0 tx=0x7f21dc00cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:19.244 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:15:19.239+0000 7f21e17fa640 1 -- 172.21.15.204:0/1902802804 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21dc00ae20 con 0x7f21e4105af0 2024-09-13T18:15:19.244 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:15:19.239+0000 7f21e17fa640 1 -- 172.21.15.204:0/1902802804 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f21dc005020 con 0x7f21e4105af0 2024-09-13T18:15:19.244 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:15:19.239+0000 7f21ea2f1640 1 -- 172.21.15.204:0/1902802804 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21e419de00 con 0x7f21e4105af0 2024-09-13T18:15:19.244 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:15:19.239+0000 7f21e17fa640 1 -- 172.21.15.204:0/1902802804 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21dc00f6b0 con 0x7f21e4105af0 2024-09-13T18:15:19.244 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:15:19.239+0000 7f21ea2f1640 1 -- 172.21.15.204:0/1902802804 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21e419e270 con 0x7f21e4105af0 2024-09-13T18:15:19.245 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:15:19.240+0000 7f21ea2f1640 1 -- 172.21.15.204:0/1902802804 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21e40fbf80 con 0x7f21e4105af0 2024-09-13T18:15:19.245 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:15:19.241+0000 7f21e17fa640 1 -- 172.21.15.204:0/1902802804 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 15) v1 ==== 135818+0+0 (secure 0 0 0) 0x7f21dc005190 con 0x7f21e4105af0 2024-09-13T18:15:19.245 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:15:19.242+0000 7f21e17fa640 1 --2- 172.21.15.204:0/1902802804 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f21cc0a0ff0 0x7f21cc0a34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:19.245 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:15:19.242+0000 7f21e17fa640 1 -- 172.21.15.204:0/1902802804 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f21dc0cb4c0 con 0x7f21e4105af0 2024-09-13T18:15:19.245 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:15:19.242+0000 7f21e3fff640 1 --2- 172.21.15.204:0/1902802804 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f21cc0a0ff0 0x7f21cc0a34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:19.245 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:15:19.243+0000 7f21e3fff640 1 --2- 172.21.15.204:0/1902802804 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f21cc0a0ff0 0x7f21cc0a34b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f21d402aa00 tx=0x7f21d4005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:19.247 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:15:19.245+0000 7f21e17fa640 1 -- 172.21.15.204:0/1902802804 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f21dc096550 con 0x7f21e4105af0 2024-09-13T18:15:19.417 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:15:19.415+0000 7f21ea2f1640 1 -- 172.21.15.204:0/1902802804 --> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f21e402c450 con 0x7f21cc0a0ff0 2024-09-13T18:15:19.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:19 smithi114 ceph-mon[30897]: osdmap e14: 2 total, 2 up, 2 in 2024-09-13T18:15:19.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:19 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T18:15:19.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:19 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T18:15:19.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:19 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:15:20.051 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:19 smithi204 ceph-mon[34777]: osdmap e14: 2 total, 2 up, 2 in 2024-09-13T18:15:20.051 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:19 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T18:15:20.051 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:19 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T18:15:20.051 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:19 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:15:20.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:19 smithi134 ceph-mon[34283]: osdmap e14: 2 total, 2 up, 2 in 2024-09-13T18:15:20.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:19 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T18:15:20.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:19 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T18:15:20.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:19 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:15:20.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:20 smithi204 ceph-mon[34777]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:15:20.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:20 smithi204 ceph-mon[34777]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T18:15:21.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:20 smithi134 ceph-mon[34283]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:15:21.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:20 smithi134 ceph-mon[34283]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T18:15:21.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:20 smithi114 ceph-mon[30897]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:15:21.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:20 smithi114 ceph-mon[30897]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T18:15:21.982 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:21 smithi204 ceph-mon[34777]: from='client.? 172.21.15.204:0/3237423257' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "578213fd-715f-4be7-91fa-254f3d0e3022"}]: dispatch 2024-09-13T18:15:21.982 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:21 smithi204 ceph-mon[34777]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "578213fd-715f-4be7-91fa-254f3d0e3022"}]: dispatch 2024-09-13T18:15:21.983 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:21 smithi204 ceph-mon[34777]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "578213fd-715f-4be7-91fa-254f3d0e3022"}]': finished 2024-09-13T18:15:21.983 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:21 smithi204 ceph-mon[34777]: osdmap e15: 3 total, 2 up, 3 in 2024-09-13T18:15:21.983 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:21 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T18:15:22.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:21 smithi134 ceph-mon[34283]: from='client.? 172.21.15.204:0/3237423257' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "578213fd-715f-4be7-91fa-254f3d0e3022"}]: dispatch 2024-09-13T18:15:22.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:21 smithi134 ceph-mon[34283]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "578213fd-715f-4be7-91fa-254f3d0e3022"}]: dispatch 2024-09-13T18:15:22.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:21 smithi134 ceph-mon[34283]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "578213fd-715f-4be7-91fa-254f3d0e3022"}]': finished 2024-09-13T18:15:22.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:21 smithi134 ceph-mon[34283]: osdmap e15: 3 total, 2 up, 3 in 2024-09-13T18:15:22.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:21 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T18:15:22.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:21 smithi114 ceph-mon[30897]: from='client.? 172.21.15.204:0/3237423257' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "578213fd-715f-4be7-91fa-254f3d0e3022"}]: dispatch 2024-09-13T18:15:22.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:21 smithi114 ceph-mon[30897]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "578213fd-715f-4be7-91fa-254f3d0e3022"}]: dispatch 2024-09-13T18:15:22.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:21 smithi114 ceph-mon[30897]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "578213fd-715f-4be7-91fa-254f3d0e3022"}]': finished 2024-09-13T18:15:22.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:21 smithi114 ceph-mon[30897]: osdmap e15: 3 total, 2 up, 3 in 2024-09-13T18:15:22.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:21 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T18:15:23.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:22 smithi134 ceph-mon[34283]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T18:15:23.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:22 smithi134 ceph-mon[34283]: from='client.? 172.21.15.204:0/3493536170' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T18:15:23.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:22 smithi204 ceph-mon[34777]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T18:15:23.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:22 smithi204 ceph-mon[34777]: from='client.? 172.21.15.204:0/3493536170' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T18:15:23.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:22 smithi114 ceph-mon[30897]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T18:15:23.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:22 smithi114 ceph-mon[30897]: from='client.? 172.21.15.204:0/3493536170' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T18:15:25.183 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:24 smithi204 ceph-mon[34777]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T18:15:25.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:24 smithi114 ceph-mon[30897]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T18:15:25.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:24 smithi134 ceph-mon[34283]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T18:15:27.185 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:26 smithi204 ceph-mon[34777]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T18:15:27.219 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:26 smithi114 ceph-mon[30897]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T18:15:27.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:26 smithi134 ceph-mon[34283]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T18:15:29.186 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:28 smithi204 ceph-mon[34777]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T18:15:29.187 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:28 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-13T18:15:29.187 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:28 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:15:29.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:28 smithi114 ceph-mon[30897]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T18:15:29.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:28 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-13T18:15:29.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:28 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:15:29.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:28 smithi134 ceph-mon[34283]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T18:15:29.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:28 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-13T18:15:29.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:28 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:15:30.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:29 smithi204 ceph-mon[34777]: Deploying daemon osd.2 on smithi204 2024-09-13T18:15:30.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:29 smithi114 ceph-mon[30897]: Deploying daemon osd.2 on smithi204 2024-09-13T18:15:30.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:29 smithi134 ceph-mon[34283]: Deploying daemon osd.2 on smithi204 2024-09-13T18:15:31.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:30 smithi204 ceph-mon[34777]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T18:15:31.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:30 smithi114 ceph-mon[30897]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T18:15:31.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:30 smithi134 ceph-mon[34283]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T18:15:33.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:32 smithi204 ceph-mon[34777]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T18:15:33.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:32 smithi114 ceph-mon[30897]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T18:15:33.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:32 smithi134 ceph-mon[34283]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T18:15:34.147 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:33 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:15:34.147 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:33 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:34.147 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:33 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:34.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:33 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:15:34.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:33 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:34.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:33 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:34.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:33 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:15:34.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:33 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:34.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:33 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:35.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:34 smithi204 ceph-mon[34777]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T18:15:35.219 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:34 smithi114 ceph-mon[30897]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T18:15:35.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:34 smithi134 ceph-mon[34283]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T18:15:36.749 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:15:36.745+0000 7f21e17fa640 1 -- 172.21.15.204:0/1902802804 <== mgr.14148 v2:172.21.15.114:6800/3396185541 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f21e402c450 con 0x7f21cc0a0ff0 2024-09-13T18:15:36.749 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 2 on host 'smithi204' 2024-09-13T18:15:36.750 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:15:36.748+0000 7f21ea2f1640 1 -- 172.21.15.204:0/1902802804 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f21cc0a0ff0 msgr2=0x7f21cc0a34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:36.751 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:15:36.748+0000 7f21ea2f1640 1 --2- 172.21.15.204:0/1902802804 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f21cc0a0ff0 0x7f21cc0a34b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f21d402aa00 tx=0x7f21d4005a20 comp rx=0 tx=0).stop 2024-09-13T18:15:36.751 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:15:36.748+0000 7f21ea2f1640 1 -- 172.21.15.204:0/1902802804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21e4105af0 msgr2=0x7f21e419d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:36.751 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:15:36.748+0000 7f21ea2f1640 1 --2- 172.21.15.204:0/1902802804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21e4105af0 0x7f21e419d280 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f21dc00cad0 tx=0x7f21dc00cf90 comp rx=0 tx=0).stop 2024-09-13T18:15:36.751 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:15:36.748+0000 7f21ea2f1640 1 -- 172.21.15.204:0/1902802804 shutdown_connections 2024-09-13T18:15:36.751 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:15:36.748+0000 7f21ea2f1640 1 --2- 172.21.15.204:0/1902802804 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f21cc0a0ff0 0x7f21cc0a34b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:36.751 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:15:36.748+0000 7f21ea2f1640 1 --2- 172.21.15.204:0/1902802804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21e4105af0 0x7f21e419d280 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:36.751 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:15:36.748+0000 7f21ea2f1640 1 --2- 172.21.15.204:0/1902802804 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21e41031c0 0x7f21e4196240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:36.751 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:15:36.748+0000 7f21ea2f1640 1 --2- 172.21.15.204:0/1902802804 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f21e4100890 0x7f21e4195d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:36.751 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:15:36.748+0000 7f21ea2f1640 1 -- 172.21.15.204:0/1902802804 >> 172.21.15.204:0/1902802804 conn(0x7f21e40fa490 msgr2=0x7f21e4101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:15:36.751 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:15:36.748+0000 7f21ea2f1640 1 -- 172.21.15.204:0/1902802804 shutdown_connections 2024-09-13T18:15:36.751 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:15:36.748+0000 7f21ea2f1640 1 -- 172.21.15.204:0/1902802804 wait complete. 2024-09-13T18:15:36.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:36 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:36.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:36 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:36.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:36 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:15:36.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:36 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:15:36.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:36 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:36.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:36 smithi114 ceph-mon[30897]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T18:15:37.055 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:36 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:37.055 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:36 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:37.056 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:36 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:15:37.056 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:36 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:15:37.056 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:36 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:37.056 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:36 smithi204 ceph-mon[34777]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T18:15:37.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:36 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:37.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:36 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:37.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:36 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:15:37.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:36 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:15:37.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:36 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:37.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:36 smithi134 ceph-mon[34283]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T18:15:37.339 DEBUG:teuthology.orchestra.run.smithi204:osd.2> sudo journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@osd.2.service 2024-09-13T18:15:37.342 INFO:tasks.cephadm:Deploying osd.3 on smithi114 with /dev/vg_nvme/lv_3... 2024-09-13T18:15:37.342 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-09-13T18:15:37.545 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:15:37.815 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:37 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:15:37.815 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:37 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:37.816 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:37 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:37.816 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:37 smithi204 ceph-mon[34777]: from='osd.2 [v2:172.21.15.204:6800/2288807707,v1:172.21.15.204:6801/2288807707]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-13T18:15:37.816 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:37 smithi204 ceph-mon[34777]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-13T18:15:37.917 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:37 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:15:37.917 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:37 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:37.917 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:37 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:37.918 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:37 smithi114 ceph-mon[30897]: from='osd.2 [v2:172.21.15.204:6800/2288807707,v1:172.21.15.204:6801/2288807707]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-13T18:15:37.918 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:37 smithi114 ceph-mon[30897]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-13T18:15:38.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:37 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:15:38.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:37 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:38.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:37 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:38.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:37 smithi134 ceph-mon[34283]: from='osd.2 [v2:172.21.15.204:6800/2288807707,v1:172.21.15.204:6801/2288807707]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-13T18:15:38.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:37 smithi134 ceph-mon[34283]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-13T18:15:38.892 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:38 smithi204 ceph-mon[34777]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-13T18:15:38.892 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:38 smithi204 ceph-mon[34777]: osdmap e16: 3 total, 2 up, 3 in 2024-09-13T18:15:38.892 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:38 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T18:15:38.892 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:38 smithi204 ceph-mon[34777]: from='osd.2 [v2:172.21.15.204:6800/2288807707,v1:172.21.15.204:6801/2288807707]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-13T18:15:38.892 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:38 smithi204 ceph-mon[34777]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-13T18:15:38.892 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:38 smithi204 ceph-mon[34777]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T18:15:38.892 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 13 18:15:38 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-2[39815]: 2024-09-13T18:15:38.752+0000 7f7efc42a640 -1 osd.2 0 waiting for initial osdmap 2024-09-13T18:15:38.893 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 13 18:15:38 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-2[39815]: 2024-09-13T18:15:38.756+0000 7f7ef8253640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T18:15:38.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:38 smithi114 ceph-mon[30897]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-13T18:15:38.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:38 smithi114 ceph-mon[30897]: osdmap e16: 3 total, 2 up, 3 in 2024-09-13T18:15:38.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:38 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T18:15:38.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:38 smithi114 ceph-mon[30897]: from='osd.2 [v2:172.21.15.204:6800/2288807707,v1:172.21.15.204:6801/2288807707]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-13T18:15:38.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:38 smithi114 ceph-mon[30897]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-13T18:15:38.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:38 smithi114 ceph-mon[30897]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T18:15:39.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:38 smithi134 ceph-mon[34283]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-13T18:15:39.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:38 smithi134 ceph-mon[34283]: osdmap e16: 3 total, 2 up, 3 in 2024-09-13T18:15:39.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:38 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T18:15:39.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:38 smithi134 ceph-mon[34283]: from='osd.2 [v2:172.21.15.204:6800/2288807707,v1:172.21.15.204:6801/2288807707]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-13T18:15:39.120 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:38 smithi134 ceph-mon[34283]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-13T18:15:39.120 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:38 smithi134 ceph-mon[34283]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T18:15:39.358 INFO:teuthology.orchestra.run.smithi114.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-09-13T18:15:39.445 INFO:teuthology.orchestra.run.smithi114.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-09-13T18:15:39.458 INFO:teuthology.orchestra.run.smithi114.stderr: stderr: 10+0 records in 2024-09-13T18:15:39.458 INFO:teuthology.orchestra.run.smithi114.stderr:10+0 records out 2024-09-13T18:15:39.458 INFO:teuthology.orchestra.run.smithi114.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115335 s, 909 MB/s 2024-09-13T18:15:39.458 INFO:teuthology.orchestra.run.smithi114.stderr:--> Zapping successful for: 2024-09-13T18:15:39.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:39 smithi204 ceph-mon[34777]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-09-13T18:15:39.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:39 smithi204 ceph-mon[34777]: osdmap e17: 3 total, 2 up, 3 in 2024-09-13T18:15:39.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:39 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T18:15:39.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:39 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T18:15:40.105 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:39 smithi114 ceph-mon[30897]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-09-13T18:15:40.105 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:39 smithi114 ceph-mon[30897]: osdmap e17: 3 total, 2 up, 3 in 2024-09-13T18:15:40.105 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:39 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T18:15:40.105 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:39 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T18:15:40.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:39 smithi134 ceph-mon[34283]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-09-13T18:15:40.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:39 smithi134 ceph-mon[34283]: osdmap e17: 3 total, 2 up, 3 in 2024-09-13T18:15:40.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:39 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T18:15:40.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:39 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T18:15:40.126 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi114:vg_nvme/lv_3 2024-09-13T18:15:40.331 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:15:41.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:40 smithi134 ceph-mon[34283]: purged_snaps scrub starts 2024-09-13T18:15:41.154 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:40 smithi134 ceph-mon[34283]: purged_snaps scrub ok 2024-09-13T18:15:41.154 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:40 smithi134 ceph-mon[34283]: osd.2 [v2:172.21.15.204:6800/2288807707,v1:172.21.15.204:6801/2288807707] boot 2024-09-13T18:15:41.154 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:40 smithi134 ceph-mon[34283]: osdmap e18: 3 total, 3 up, 3 in 2024-09-13T18:15:41.154 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:40 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T18:15:41.154 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:40 smithi134 ceph-mon[34283]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T18:15:41.155 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:40 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-13T18:15:41.155 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:40 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:41.155 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:40 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:41.155 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:40 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:15:41.155 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:40 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:41.155 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:40 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:15:41.155 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:40 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:15:41.155 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:40 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:41.156 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:40 smithi114 ceph-mon[30897]: purged_snaps scrub starts 2024-09-13T18:15:41.156 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:40 smithi114 ceph-mon[30897]: purged_snaps scrub ok 2024-09-13T18:15:41.156 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:40 smithi114 ceph-mon[30897]: osd.2 [v2:172.21.15.204:6800/2288807707,v1:172.21.15.204:6801/2288807707] boot 2024-09-13T18:15:41.156 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:40 smithi114 ceph-mon[30897]: osdmap e18: 3 total, 3 up, 3 in 2024-09-13T18:15:41.156 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:40 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T18:15:41.156 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:40 smithi114 ceph-mon[30897]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T18:15:41.156 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:40 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-13T18:15:41.156 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:40 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:41.156 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:40 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:41.156 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:40 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:15:41.156 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:40 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:41.157 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:40 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:15:41.157 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:40 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:15:41.157 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:40 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:41.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:40 smithi204 ceph-mon[34777]: purged_snaps scrub starts 2024-09-13T18:15:41.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:40 smithi204 ceph-mon[34777]: purged_snaps scrub ok 2024-09-13T18:15:41.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:40 smithi204 ceph-mon[34777]: osd.2 [v2:172.21.15.204:6800/2288807707,v1:172.21.15.204:6801/2288807707] boot 2024-09-13T18:15:41.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:40 smithi204 ceph-mon[34777]: osdmap e18: 3 total, 3 up, 3 in 2024-09-13T18:15:41.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:40 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T18:15:41.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:40 smithi204 ceph-mon[34777]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-13T18:15:41.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:40 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-13T18:15:41.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:40 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:41.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:40 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:41.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:40 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:15:41.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:40 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:41.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:40 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:15:41.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:40 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:15:41.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:40 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:41.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:41.809+0000 7fca04f50640 1 -- 172.21.15.114:0/1444557797 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fca00101930 msgr2=0x7fca00108250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:41.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:41.809+0000 7fca04f50640 1 --2- 172.21.15.114:0/1444557797 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fca00101930 0x7fca00108250 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc9f8009ca0 tx=0x7fc9f802a8c0 comp rx=0 tx=0).stop 2024-09-13T18:15:41.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:41.809+0000 7fca04f50640 1 -- 172.21.15.114:0/1444557797 shutdown_connections 2024-09-13T18:15:41.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:41.809+0000 7fca04f50640 1 --2- 172.21.15.114:0/1444557797 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fca00101930 0x7fca00108250 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:41.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:41.809+0000 7fca04f50640 1 --2- 172.21.15.114:0/1444557797 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca00100f70 0x7fca001013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:41.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:41.809+0000 7fca04f50640 1 --2- 172.21.15.114:0/1444557797 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca000ffe10 0x7fca00100210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:41.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:41.809+0000 7fca04f50640 1 -- 172.21.15.114:0/1444557797 >> 172.21.15.114:0/1444557797 conn(0x7fca000fb5a0 msgr2=0x7fca000fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:15:41.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:41.809+0000 7fca04f50640 1 -- 172.21.15.114:0/1444557797 shutdown_connections 2024-09-13T18:15:41.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:41.809+0000 7fca04f50640 1 -- 172.21.15.114:0/1444557797 wait complete. 2024-09-13T18:15:41.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:41.809+0000 7fca04f50640 1 Processor -- start 2024-09-13T18:15:41.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:41.809+0000 7fca04f50640 1 -- start start 2024-09-13T18:15:41.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:41.810+0000 7fca04f50640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fca000ffe10 0x7fca00074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:41.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:41.810+0000 7fca04f50640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca00100f70 0x7fca000728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:41.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:41.810+0000 7fca04f50640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca00101930 0x7fca00072de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:41.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:41.810+0000 7fca04f50640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca00073570 con 0x7fca000ffe10 2024-09-13T18:15:41.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:41.810+0000 7fca04f50640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca000736e0 con 0x7fca00100f70 2024-09-13T18:15:41.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:41.810+0000 7fca04f50640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0019fc60 con 0x7fca00101930 2024-09-13T18:15:41.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:41.810+0000 7fc9fe59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fca000ffe10 0x7fca00074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:41.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:41.810+0000 7fc9fdd9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca00100f70 0x7fca000728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:41.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:41.810+0000 7fc9fe59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fca000ffe10 0x7fca00074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38208/0 (socket says 172.21.15.114:38208) 2024-09-13T18:15:41.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:41.810+0000 7fc9fe59c640 1 -- 172.21.15.114:0/711156030 learned_addr learned my addr 172.21.15.114:0/711156030 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:15:41.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:41.810+0000 7fc9fed9d640 1 --2- 172.21.15.114:0/711156030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca00101930 0x7fca00072de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:41.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:41.810+0000 7fc9fdd9b640 1 -- 172.21.15.114:0/711156030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca00101930 msgr2=0x7fca00072de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:41.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:41.810+0000 7fc9fdd9b640 1 --2- 172.21.15.114:0/711156030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca00101930 0x7fca00072de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:41.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:41.810+0000 7fc9fdd9b640 1 -- 172.21.15.114:0/711156030 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fca000ffe10 msgr2=0x7fca00074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:41.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:41.810+0000 7fc9fdd9b640 1 --2- 172.21.15.114:0/711156030 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fca000ffe10 0x7fca00074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:41.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:41.810+0000 7fc9fdd9b640 1 -- 172.21.15.114:0/711156030 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f8009900 con 0x7fca00100f70 2024-09-13T18:15:41.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:41.810+0000 7fc9fe59c640 1 --2- 172.21.15.114:0/711156030 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fca000ffe10 0x7fca00074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:15:41.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:41.810+0000 7fc9fdd9b640 1 --2- 172.21.15.114:0/711156030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca00100f70 0x7fca000728a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc9f400e9f0 tx=0x7fc9f400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:41.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:41.811+0000 7fc9ef7fe640 1 -- 172.21.15.114:0/711156030 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f400ccc0 con 0x7fca00100f70 2024-09-13T18:15:41.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:41.811+0000 7fca04f50640 1 -- 172.21.15.114:0/711156030 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca0019ff10 con 0x7fca00100f70 2024-09-13T18:15:41.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:41.811+0000 7fc9ef7fe640 1 -- 172.21.15.114:0/711156030 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc9f40044b0 con 0x7fca00100f70 2024-09-13T18:15:41.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:41.811+0000 7fc9ef7fe640 1 -- 172.21.15.114:0/711156030 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f4003e50 con 0x7fca00100f70 2024-09-13T18:15:41.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:41.811+0000 7fca04f50640 1 -- 172.21.15.114:0/711156030 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca001a03d0 con 0x7fca00100f70 2024-09-13T18:15:41.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:41.812+0000 7fca04f50640 1 -- 172.21.15.114:0/711156030 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca00107250 con 0x7fca00100f70 2024-09-13T18:15:41.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:41.813+0000 7fc9ef7fe640 1 -- 172.21.15.114:0/711156030 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 15) v1 ==== 135818+0+0 (secure 0 0 0) 0x7fc9f4004000 con 0x7fca00100f70 2024-09-13T18:15:41.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:41.814+0000 7fc9ef7fe640 1 --2- 172.21.15.114:0/711156030 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fc9e80a0ff0 0x7fc9e80a34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:41.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:41.814+0000 7fc9fe59c640 1 --2- 172.21.15.114:0/711156030 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fc9e80a0ff0 0x7fc9e80a34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:41.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:41.814+0000 7fc9ef7fe640 1 -- 172.21.15.114:0/711156030 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3237+0+0 (secure 0 0 0) 0x7fc9f4014070 con 0x7fca00100f70 2024-09-13T18:15:41.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:41.814+0000 7fc9fe59c640 1 --2- 172.21.15.114:0/711156030 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fc9e80a0ff0 0x7fc9e80a34b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0011500 tx=0x7fc9f0005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:41.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:41.818+0000 7fc9ef7fe640 1 -- 172.21.15.114:0/711156030 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fc9f4096a70 con 0x7fca00100f70 2024-09-13T18:15:42.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:42.001+0000 7fca04f50640 1 -- 172.21.15.114:0/711156030 --> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fca0002c4f0 con 0x7fc9e80a0ff0 2024-09-13T18:15:42.118 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:41 smithi114 ceph-mon[30897]: Detected new or changed devices on smithi204 2024-09-13T18:15:42.118 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:41 smithi114 ceph-mon[30897]: Adjusting osd_memory_target on smithi204 to 17088M 2024-09-13T18:15:42.118 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:41 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-13T18:15:42.118 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:41 smithi114 ceph-mon[30897]: osdmap e19: 3 total, 3 up, 3 in 2024-09-13T18:15:42.118 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:41 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-13T18:15:42.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:41 smithi134 ceph-mon[34283]: Detected new or changed devices on smithi204 2024-09-13T18:15:42.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:41 smithi134 ceph-mon[34283]: Adjusting osd_memory_target on smithi204 to 17088M 2024-09-13T18:15:42.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:41 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-13T18:15:42.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:41 smithi134 ceph-mon[34283]: osdmap e19: 3 total, 3 up, 3 in 2024-09-13T18:15:42.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:41 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-13T18:15:42.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:41 smithi204 ceph-mon[34777]: Detected new or changed devices on smithi204 2024-09-13T18:15:42.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:41 smithi204 ceph-mon[34777]: Adjusting osd_memory_target on smithi204 to 17088M 2024-09-13T18:15:42.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:41 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-13T18:15:42.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:41 smithi204 ceph-mon[34777]: osdmap e19: 3 total, 3 up, 3 in 2024-09-13T18:15:42.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:41 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-13T18:15:42.385 INFO:journalctl@ceph.osd.0.smithi114.stdout:Sep 13 18:15:42 smithi114 sudo[49203]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-13T18:15:42.386 INFO:journalctl@ceph.osd.0.smithi114.stdout:Sep 13 18:15:42 smithi114 sudo[49203]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T18:15:42.386 INFO:journalctl@ceph.osd.0.smithi114.stdout:Sep 13 18:15:42 smithi114 sudo[49203]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T18:15:42.386 INFO:journalctl@ceph.osd.0.smithi114.stdout:Sep 13 18:15:42 smithi114 sudo[49203]: pam_unix(sudo:session): session closed for user root 2024-09-13T18:15:42.639 INFO:journalctl@ceph.osd.0.smithi114.stdout:Sep 13 18:15:42 smithi114 sudo[49259]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-13T18:15:42.639 INFO:journalctl@ceph.osd.0.smithi114.stdout:Sep 13 18:15:42 smithi114 sudo[49259]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T18:15:42.639 INFO:journalctl@ceph.osd.0.smithi114.stdout:Sep 13 18:15:42 smithi114 sudo[49259]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T18:15:42.639 INFO:journalctl@ceph.osd.0.smithi114.stdout:Sep 13 18:15:42 smithi114 sudo[49259]: pam_unix(sudo:session): session closed for user root 2024-09-13T18:15:42.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:42.789+0000 7fc9ef7fe640 1 -- 172.21.15.114:0/711156030 <== mon.2 v2:172.21.15.134:3300/0 7 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fc9f4096e40 con 0x7fca00100f70 2024-09-13T18:15:42.919 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 13 18:15:42 smithi134 sudo[42558]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-13T18:15:42.919 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 13 18:15:42 smithi134 sudo[42558]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T18:15:42.919 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 13 18:15:42 smithi134 sudo[42558]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T18:15:42.919 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 13 18:15:42 smithi134 sudo[42558]: pam_unix(sudo:session): session closed for user root 2024-09-13T18:15:42.920 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:42 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-13T18:15:42.920 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:42 smithi134 ceph-mon[34283]: osdmap e20: 3 total, 3 up, 3 in 2024-09-13T18:15:42.920 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:42 smithi134 ceph-mon[34283]: pgmap v78: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-13T18:15:42.920 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:42 smithi134 ceph-mon[34283]: from='client.24185 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:15:42.920 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:42 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T18:15:42.920 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:42 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T18:15:42.920 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:42 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:15:42.920 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:42 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:42.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:42 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-13T18:15:42.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:42 smithi114 ceph-mon[30897]: osdmap e20: 3 total, 3 up, 3 in 2024-09-13T18:15:42.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:42 smithi114 ceph-mon[30897]: pgmap v78: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-13T18:15:42.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:42 smithi114 ceph-mon[30897]: from='client.24185 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:15:42.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:42 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T18:15:42.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:42 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T18:15:42.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:42 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:15:42.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:42 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:43.193 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:42 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-13T18:15:43.193 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:42 smithi204 ceph-mon[34777]: osdmap e20: 3 total, 3 up, 3 in 2024-09-13T18:15:43.193 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:42 smithi204 ceph-mon[34777]: pgmap v78: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-13T18:15:43.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:42 smithi204 ceph-mon[34777]: from='client.24185 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:15:43.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:42 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T18:15:43.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:42 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T18:15:43.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:42 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:15:43.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:42 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:43.369 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 13 18:15:42 smithi134 sudo[42561]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-13T18:15:43.369 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 13 18:15:42 smithi134 sudo[42561]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T18:15:43.369 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 13 18:15:42 smithi134 sudo[42561]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T18:15:43.369 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 13 18:15:42 smithi134 sudo[42561]: pam_unix(sudo:session): session closed for user root 2024-09-13T18:15:43.444 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 13 18:15:43 smithi204 sudo[43383]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-13T18:15:43.444 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 13 18:15:43 smithi204 sudo[43383]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T18:15:43.444 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 13 18:15:43 smithi204 sudo[43383]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T18:15:43.444 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 13 18:15:43 smithi204 sudo[43383]: pam_unix(sudo:session): session closed for user root 2024-09-13T18:15:43.944 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 13 18:15:43 smithi204 sudo[43386]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-13T18:15:43.944 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 13 18:15:43 smithi204 sudo[43386]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T18:15:43.944 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 13 18:15:43 smithi204 sudo[43386]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T18:15:43.945 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 13 18:15:43 smithi204 sudo[43386]: pam_unix(sudo:session): session closed for user root 2024-09-13T18:15:43.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:43 smithi114 sudo[49310]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-13T18:15:43.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:43 smithi114 sudo[49310]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T18:15:43.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:43 smithi114 sudo[49310]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T18:15:43.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:43 smithi114 sudo[49310]: pam_unix(sudo:session): session closed for user root 2024-09-13T18:15:44.253 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:43 smithi204 ceph-mon[34777]: osdmap e21: 3 total, 3 up, 3 in 2024-09-13T18:15:44.253 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:43 smithi204 ceph-mon[34777]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-13T18:15:44.253 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:43 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:44.254 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:43 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:44.254 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:43 smithi204 ceph-mon[34777]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-13T18:15:44.254 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:44 smithi204 sudo[43389]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-13T18:15:44.254 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:44 smithi204 sudo[43389]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T18:15:44.254 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:44 smithi204 sudo[43389]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T18:15:44.254 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:44 smithi204 sudo[43389]: pam_unix(sudo:session): session closed for user root 2024-09-13T18:15:44.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:43 smithi134 ceph-mon[34283]: osdmap e21: 3 total, 3 up, 3 in 2024-09-13T18:15:44.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:43 smithi134 ceph-mon[34283]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-13T18:15:44.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:43 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:44.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:43 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:44.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:43 smithi134 ceph-mon[34283]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-13T18:15:44.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:43 smithi114 sudo[49325]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-13T18:15:44.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:43 smithi114 sudo[49325]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T18:15:44.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:43 smithi114 sudo[49325]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T18:15:44.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:43 smithi114 sudo[49325]: pam_unix(sudo:session): session closed for user root 2024-09-13T18:15:44.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:43 smithi114 ceph-mon[30897]: osdmap e21: 3 total, 3 up, 3 in 2024-09-13T18:15:44.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:43 smithi114 ceph-mon[30897]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-13T18:15:44.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:43 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:44.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:43 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:44.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:43 smithi114 ceph-mon[30897]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-13T18:15:44.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:44 smithi204 sudo[43392]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-13T18:15:44.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:44 smithi204 sudo[43392]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T18:15:44.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:44 smithi204 sudo[43392]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T18:15:44.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:44 smithi204 sudo[43392]: pam_unix(sudo:session): session closed for user root 2024-09-13T18:15:44.868 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:44 smithi134 sudo[42576]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-13T18:15:44.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:44 smithi134 sudo[42576]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T18:15:44.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:44 smithi134 sudo[42576]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T18:15:44.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:44 smithi134 sudo[42576]: pam_unix(sudo:session): session closed for user root 2024-09-13T18:15:45.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:44 smithi134 sudo[42579]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-13T18:15:45.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:44 smithi134 sudo[42579]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-13T18:15:45.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:44 smithi134 sudo[42579]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-13T18:15:45.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:45 smithi134 sudo[42579]: pam_unix(sudo:session): session closed for user root 2024-09-13T18:15:45.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:45 smithi134 ceph-mon[34283]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-13T18:15:45.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:45 smithi134 ceph-mon[34283]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-13T18:15:45.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:45 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T18:15:45.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:45 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T18:15:45.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:45 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T18:15:45.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:45 smithi134 ceph-mon[34283]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-13T18:15:45.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:45 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T18:15:45.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:45 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T18:15:45.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:45 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T18:15:45.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:45 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/4224500325' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b64aab3f-1e52-4828-8397-74af5e05f2cf"}]: dispatch 2024-09-13T18:15:45.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:45 smithi134 ceph-mon[34283]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-13T18:15:45.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:45 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/4224500325' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b64aab3f-1e52-4828-8397-74af5e05f2cf"}]': finished 2024-09-13T18:15:45.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:45 smithi134 ceph-mon[34283]: osdmap e22: 4 total, 3 up, 4 in 2024-09-13T18:15:45.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:45 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-13T18:15:45.371 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:45 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T18:15:45.371 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:45 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T18:15:45.371 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:45 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T18:15:45.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:45 smithi204 ceph-mon[34777]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-13T18:15:45.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:45 smithi204 ceph-mon[34777]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-13T18:15:45.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:45 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T18:15:45.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:45 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T18:15:45.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:45 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T18:15:45.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:45 smithi204 ceph-mon[34777]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-13T18:15:45.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:45 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T18:15:45.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:45 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T18:15:45.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:45 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T18:15:45.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:45 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/4224500325' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b64aab3f-1e52-4828-8397-74af5e05f2cf"}]: dispatch 2024-09-13T18:15:45.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:45 smithi204 ceph-mon[34777]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-13T18:15:45.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:45 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/4224500325' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b64aab3f-1e52-4828-8397-74af5e05f2cf"}]': finished 2024-09-13T18:15:45.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:45 smithi204 ceph-mon[34777]: osdmap e22: 4 total, 3 up, 4 in 2024-09-13T18:15:45.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:45 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-13T18:15:45.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:45 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T18:15:45.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:45 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T18:15:45.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:45 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T18:15:45.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:45 smithi114 ceph-mon[30897]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-13T18:15:45.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:45 smithi114 ceph-mon[30897]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-13T18:15:45.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:45 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T18:15:45.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:45 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T18:15:45.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:45 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T18:15:45.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:45 smithi114 ceph-mon[30897]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-13T18:15:45.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:45 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T18:15:45.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:45 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T18:15:45.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:45 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T18:15:45.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:45 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/4224500325' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b64aab3f-1e52-4828-8397-74af5e05f2cf"}]: dispatch 2024-09-13T18:15:45.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:45 smithi114 ceph-mon[30897]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-13T18:15:45.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:45 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/4224500325' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b64aab3f-1e52-4828-8397-74af5e05f2cf"}]': finished 2024-09-13T18:15:45.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:45 smithi114 ceph-mon[30897]: osdmap e22: 4 total, 3 up, 4 in 2024-09-13T18:15:45.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:45 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-13T18:15:45.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:45 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T18:15:45.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:45 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T18:15:45.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:45 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T18:15:46.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:46 smithi134 ceph-mon[34283]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-13T18:15:46.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:46 smithi134 ceph-mon[34283]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-13T18:15:46.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:46 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/4279374638' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T18:15:46.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:46 smithi204 ceph-mon[34777]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-13T18:15:46.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:46 smithi204 ceph-mon[34777]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-13T18:15:46.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:46 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/4279374638' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T18:15:46.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:46 smithi114 ceph-mon[30897]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-13T18:15:46.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:46 smithi114 ceph-mon[30897]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-13T18:15:46.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:46 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/4279374638' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T18:15:47.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:47 smithi204 ceph-mon[34777]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-13T18:15:47.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:47 smithi134 ceph-mon[34283]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-13T18:15:47.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:47 smithi114 ceph-mon[30897]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-13T18:15:49.317 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:49 smithi114 ceph-mon[30897]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-13T18:15:49.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:49 smithi134 ceph-mon[34283]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-13T18:15:49.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:49 smithi204 ceph-mon[34777]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-13T18:15:51.272 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:51 smithi114 ceph-mon[30897]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-09-13T18:15:51.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:51 smithi134 ceph-mon[34283]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-09-13T18:15:51.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:51 smithi204 ceph-mon[34777]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-09-13T18:15:52.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:52 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-13T18:15:52.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:52 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:15:52.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:52 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-13T18:15:52.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:52 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:15:52.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:52 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-13T18:15:52.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:52 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:15:53.261 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:53 smithi114 ceph-mon[30897]: Deploying daemon osd.3 on smithi114 2024-09-13T18:15:53.261 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:53 smithi114 ceph-mon[30897]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 98 KiB/s wr, 7 op/s 2024-09-13T18:15:53.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:53 smithi134 ceph-mon[34283]: Deploying daemon osd.3 on smithi114 2024-09-13T18:15:53.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:53 smithi134 ceph-mon[34283]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 98 KiB/s wr, 7 op/s 2024-09-13T18:15:53.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:53 smithi204 ceph-mon[34777]: Deploying daemon osd.3 on smithi114 2024-09-13T18:15:53.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:53 smithi204 ceph-mon[34777]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 98 KiB/s wr, 7 op/s 2024-09-13T18:15:55.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:55 smithi114 ceph-mon[30897]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 2.0 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-09-13T18:15:55.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:55 smithi134 ceph-mon[34283]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 2.0 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-09-13T18:15:55.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:55 smithi204 ceph-mon[34777]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 2.0 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-09-13T18:15:57.174 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:57 smithi114 ceph-mon[30897]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 69 KiB/s wr, 5 op/s 2024-09-13T18:15:57.174 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:57 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:15:57.174 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:57 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:57.174 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:57 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:57.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:57 smithi134 ceph-mon[34283]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 69 KiB/s wr, 5 op/s 2024-09-13T18:15:57.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:57 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:15:57.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:57 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:57.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:57 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:57.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:57 smithi204 ceph-mon[34777]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 69 KiB/s wr, 5 op/s 2024-09-13T18:15:57.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:57 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:15:57.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:57 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:57.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:57 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:15:59.130 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:15:59 smithi114 ceph-mon[30897]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-13T18:15:59.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:15:59 smithi134 ceph-mon[34283]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-13T18:15:59.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:15:59 smithi204 ceph-mon[34777]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-13T18:15:59.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:59.781+0000 7fc9ef7fe640 1 -- 172.21.15.114:0/711156030 <== mgr.14148 v2:172.21.15.114:6800/3396185541 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fca0002c4f0 con 0x7fc9e80a0ff0 2024-09-13T18:15:59.784 INFO:teuthology.orchestra.run.smithi114.stdout:Created osd(s) 3 on host 'smithi114' 2024-09-13T18:15:59.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:59.783+0000 7fca04f50640 1 -- 172.21.15.114:0/711156030 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fc9e80a0ff0 msgr2=0x7fc9e80a34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:59.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:59.783+0000 7fca04f50640 1 --2- 172.21.15.114:0/711156030 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fc9e80a0ff0 0x7fc9e80a34b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0011500 tx=0x7fc9f0005f90 comp rx=0 tx=0).stop 2024-09-13T18:15:59.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:59.783+0000 7fca04f50640 1 -- 172.21.15.114:0/711156030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca00100f70 msgr2=0x7fca000728a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:59.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:59.783+0000 7fca04f50640 1 --2- 172.21.15.114:0/711156030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca00100f70 0x7fca000728a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc9f400e9f0 tx=0x7fc9f400eeb0 comp rx=0 tx=0).stop 2024-09-13T18:15:59.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:59.783+0000 7fca04f50640 1 -- 172.21.15.114:0/711156030 shutdown_connections 2024-09-13T18:15:59.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:59.783+0000 7fca04f50640 1 --2- 172.21.15.114:0/711156030 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fc9e80a0ff0 0x7fc9e80a34b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:59.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:59.783+0000 7fca04f50640 1 --2- 172.21.15.114:0/711156030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca00101930 0x7fca00072de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:59.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:59.783+0000 7fca04f50640 1 --2- 172.21.15.114:0/711156030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca00100f70 0x7fca000728a0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:59.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:59.783+0000 7fca04f50640 1 --2- 172.21.15.114:0/711156030 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fca000ffe10 0x7fca00074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:59.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:59.783+0000 7fca04f50640 1 -- 172.21.15.114:0/711156030 >> 172.21.15.114:0/711156030 conn(0x7fca000fb5a0 msgr2=0x7fca00104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:15:59.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:59.783+0000 7fca04f50640 1 -- 172.21.15.114:0/711156030 shutdown_connections 2024-09-13T18:15:59.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:15:59.783+0000 7fca04f50640 1 -- 172.21.15.114:0/711156030 wait complete. 2024-09-13T18:16:00.361 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:00 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:00.361 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:00 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:00.361 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:00 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:16:00.361 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:00 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:16:00.361 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:00 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:00.361 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:00 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:16:00.362 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:00 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:00.362 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:00 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:00.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:00 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:00.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:00 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:00.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:00 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:16:00.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:00 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:16:00.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:00 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:00.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:00 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:16:00.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:00 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:00.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:00 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:00.422 DEBUG:teuthology.orchestra.run.smithi114:osd.3> sudo journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@osd.3.service 2024-09-13T18:16:00.425 INFO:tasks.cephadm:Deploying osd.4 on smithi134 with /dev/vg_nvme/lv_3... 2024-09-13T18:16:00.425 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-09-13T18:16:00.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:00 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:00.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:00 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:00.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:00 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:16:00.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:00 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:16:00.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:00 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:00.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:00 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:16:00.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:00 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:00.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:00 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:00.629 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T18:16:01.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:01 smithi114 ceph-mon[30897]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-13T18:16:01.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:01 smithi114 ceph-mon[30897]: from='osd.3 [v2:172.21.15.114:6810/2904204254,v1:172.21.15.114:6811/2904204254]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-13T18:16:01.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:01 smithi134 ceph-mon[34283]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-13T18:16:01.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:01 smithi134 ceph-mon[34283]: from='osd.3 [v2:172.21.15.114:6810/2904204254,v1:172.21.15.114:6811/2904204254]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-13T18:16:01.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:01 smithi204 ceph-mon[34777]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-13T18:16:01.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:01 smithi204 ceph-mon[34777]: from='osd.3 [v2:172.21.15.114:6810/2904204254,v1:172.21.15.114:6811/2904204254]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-13T18:16:02.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:02 smithi114 ceph-mon[30897]: from='osd.3 [v2:172.21.15.114:6810/2904204254,v1:172.21.15.114:6811/2904204254]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-13T18:16:02.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:02 smithi114 ceph-mon[30897]: osdmap e23: 4 total, 3 up, 4 in 2024-09-13T18:16:02.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:02 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-13T18:16:02.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:02 smithi114 ceph-mon[30897]: from='osd.3 [v2:172.21.15.114:6810/2904204254,v1:172.21.15.114:6811/2904204254]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-09-13T18:16:02.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:02 smithi114 ceph-mon[30897]: from='osd.3 [v2:172.21.15.114:6810/2904204254,v1:172.21.15.114:6811/2904204254]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-09-13T18:16:02.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:02 smithi114 ceph-mon[30897]: osdmap e24: 4 total, 3 up, 4 in 2024-09-13T18:16:02.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:02 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-13T18:16:02.221 INFO:journalctl@ceph.osd.3.smithi114.stdout:Sep 13 18:16:02 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-3[51352]: 2024-09-13T18:16:02.055+0000 7ff22fdf9640 -1 osd.3 0 waiting for initial osdmap 2024-09-13T18:16:02.221 INFO:journalctl@ceph.osd.3.smithi114.stdout:Sep 13 18:16:02 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-3[51352]: 2024-09-13T18:16:02.062+0000 7ff22bc22640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T18:16:02.368 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:02 smithi134 ceph-mon[34283]: from='osd.3 [v2:172.21.15.114:6810/2904204254,v1:172.21.15.114:6811/2904204254]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-13T18:16:02.368 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:02 smithi134 ceph-mon[34283]: osdmap e23: 4 total, 3 up, 4 in 2024-09-13T18:16:02.368 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:02 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-13T18:16:02.368 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:02 smithi134 ceph-mon[34283]: from='osd.3 [v2:172.21.15.114:6810/2904204254,v1:172.21.15.114:6811/2904204254]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-09-13T18:16:02.368 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:02 smithi134 ceph-mon[34283]: from='osd.3 [v2:172.21.15.114:6810/2904204254,v1:172.21.15.114:6811/2904204254]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-09-13T18:16:02.368 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:02 smithi134 ceph-mon[34283]: osdmap e24: 4 total, 3 up, 4 in 2024-09-13T18:16:02.368 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:02 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-13T18:16:02.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:02 smithi204 ceph-mon[34777]: from='osd.3 [v2:172.21.15.114:6810/2904204254,v1:172.21.15.114:6811/2904204254]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-13T18:16:02.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:02 smithi204 ceph-mon[34777]: osdmap e23: 4 total, 3 up, 4 in 2024-09-13T18:16:02.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:02 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-13T18:16:02.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:02 smithi204 ceph-mon[34777]: from='osd.3 [v2:172.21.15.114:6810/2904204254,v1:172.21.15.114:6811/2904204254]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-09-13T18:16:02.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:02 smithi204 ceph-mon[34777]: from='osd.3 [v2:172.21.15.114:6810/2904204254,v1:172.21.15.114:6811/2904204254]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-09-13T18:16:02.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:02 smithi204 ceph-mon[34777]: osdmap e24: 4 total, 3 up, 4 in 2024-09-13T18:16:02.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:02 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-13T18:16:02.512 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-09-13T18:16:02.590 INFO:teuthology.orchestra.run.smithi134.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-09-13T18:16:02.603 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10+0 records in 2024-09-13T18:16:02.603 INFO:teuthology.orchestra.run.smithi134.stderr:10+0 records out 2024-09-13T18:16:02.603 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011389 s, 921 MB/s 2024-09-13T18:16:02.603 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping successful for: 2024-09-13T18:16:03.297 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:03 smithi134 ceph-mon[34283]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-13T18:16:03.298 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:03 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-13T18:16:03.298 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:03 smithi134 ceph-mon[34283]: osd.3 [v2:172.21.15.114:6810/2904204254,v1:172.21.15.114:6811/2904204254] boot 2024-09-13T18:16:03.298 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:03 smithi134 ceph-mon[34283]: osdmap e25: 4 total, 4 up, 4 in 2024-09-13T18:16:03.298 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:03 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-13T18:16:03.320 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi134:vg_nvme/lv_3 2024-09-13T18:16:03.358 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:03 smithi114 ceph-mon[30897]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-13T18:16:03.359 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:03 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-13T18:16:03.359 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:03 smithi114 ceph-mon[30897]: osd.3 [v2:172.21.15.114:6810/2904204254,v1:172.21.15.114:6811/2904204254] boot 2024-09-13T18:16:03.359 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:03 smithi114 ceph-mon[30897]: osdmap e25: 4 total, 4 up, 4 in 2024-09-13T18:16:03.359 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:03 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-13T18:16:03.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:03 smithi204 ceph-mon[34777]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-13T18:16:03.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:03 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-13T18:16:03.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:03 smithi204 ceph-mon[34777]: osd.3 [v2:172.21.15.114:6810/2904204254,v1:172.21.15.114:6811/2904204254] boot 2024-09-13T18:16:03.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:03 smithi204 ceph-mon[34777]: osdmap e25: 4 total, 4 up, 4 in 2024-09-13T18:16:03.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:03 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-13T18:16:03.523 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T18:16:04.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:04 smithi114 ceph-mon[30897]: purged_snaps scrub starts 2024-09-13T18:16:04.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:04 smithi114 ceph-mon[30897]: purged_snaps scrub ok 2024-09-13T18:16:04.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:04 smithi114 ceph-mon[30897]: osdmap e26: 4 total, 4 up, 4 in 2024-09-13T18:16:04.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:04 smithi114 ceph-mon[30897]: Detected new or changed devices on smithi114 2024-09-13T18:16:04.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:04 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:04.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:04 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:04.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:04 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:16:04.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:04 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:16:04.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:04 smithi114 ceph-mon[30897]: Adjusting osd_memory_target on smithi114 to 8544M 2024-09-13T18:16:04.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:04 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:04.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:04 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:16:04.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:04 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:16:04.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:04 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:04.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:04 smithi114 ceph-mon[30897]: pgmap v95: 1 pgs: 1 remapped+peering; 577 KiB data, 114 MiB used, 357 GiB / 358 GiB avail 2024-09-13T18:16:04.561 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:04 smithi134 ceph-mon[34283]: purged_snaps scrub starts 2024-09-13T18:16:04.561 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:04 smithi134 ceph-mon[34283]: purged_snaps scrub ok 2024-09-13T18:16:04.561 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:04 smithi134 ceph-mon[34283]: osdmap e26: 4 total, 4 up, 4 in 2024-09-13T18:16:04.561 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:04 smithi134 ceph-mon[34283]: Detected new or changed devices on smithi114 2024-09-13T18:16:04.562 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:04 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:04.562 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:04 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:04.562 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:04 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:16:04.562 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:04 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:16:04.562 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:04 smithi134 ceph-mon[34283]: Adjusting osd_memory_target on smithi114 to 8544M 2024-09-13T18:16:04.562 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:04 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:04.562 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:04 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:16:04.562 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:04 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:16:04.562 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:04 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:04.562 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:04 smithi134 ceph-mon[34283]: pgmap v95: 1 pgs: 1 remapped+peering; 577 KiB data, 114 MiB used, 357 GiB / 358 GiB avail 2024-09-13T18:16:04.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:04 smithi204 ceph-mon[34777]: purged_snaps scrub starts 2024-09-13T18:16:04.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:04 smithi204 ceph-mon[34777]: purged_snaps scrub ok 2024-09-13T18:16:04.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:04 smithi204 ceph-mon[34777]: osdmap e26: 4 total, 4 up, 4 in 2024-09-13T18:16:04.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:04 smithi204 ceph-mon[34777]: Detected new or changed devices on smithi114 2024-09-13T18:16:04.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:04 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:04.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:04 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:04.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:04 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:16:04.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:04 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:16:04.696 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:04 smithi204 ceph-mon[34777]: Adjusting osd_memory_target on smithi114 to 8544M 2024-09-13T18:16:04.696 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:04 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:04.696 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:04 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:16:04.696 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:04 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:16:04.696 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:04 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:04.696 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:04 smithi204 ceph-mon[34777]: pgmap v95: 1 pgs: 1 remapped+peering; 577 KiB data, 114 MiB used, 357 GiB / 358 GiB avail 2024-09-13T18:16:05.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:05.155+0000 7f4fbe7f1640 1 -- 172.21.15.134:0/632784319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fb80fece0 msgr2=0x7f4fb80ff0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:05.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:05.155+0000 7f4fbe7f1640 1 --2- 172.21.15.134:0/632784319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fb80fece0 0x7f4fb80ff0e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4fa8009900 tx=0x7f4fa802a410 comp rx=0 tx=0).stop 2024-09-13T18:16:05.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:05.155+0000 7f4fbe7f1640 1 -- 172.21.15.134:0/632784319 shutdown_connections 2024-09-13T18:16:05.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:05.155+0000 7f4fbe7f1640 1 --2- 172.21.15.134:0/632784319 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fb81008a0 0x7f4fb8107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:05.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:05.155+0000 7f4fbe7f1640 1 --2- 172.21.15.134:0/632784319 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4fb80ffee0 0x7f4fb8100360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:05.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:05.155+0000 7f4fbe7f1640 1 --2- 172.21.15.134:0/632784319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fb80fece0 0x7f4fb80ff0e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:05.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:05.155+0000 7f4fbe7f1640 1 -- 172.21.15.134:0/632784319 >> 172.21.15.134:0/632784319 conn(0x7f4fb80fa490 msgr2=0x7f4fb80fc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:16:05.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:05.155+0000 7f4fbe7f1640 1 -- 172.21.15.134:0/632784319 shutdown_connections 2024-09-13T18:16:05.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:05.155+0000 7f4fbe7f1640 1 -- 172.21.15.134:0/632784319 wait complete. 2024-09-13T18:16:05.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:05.156+0000 7f4fbe7f1640 1 Processor -- start 2024-09-13T18:16:05.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:05.156+0000 7f4fbe7f1640 1 -- start start 2024-09-13T18:16:05.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:05.156+0000 7f4fbe7f1640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4fb80fece0 0x7f4fb8068a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:05.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:05.156+0000 7f4fbe7f1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fb80ffee0 0x7f4fb819aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:05.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:05.156+0000 7f4fbe7f1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fb81008a0 0x7f4fb819cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:05.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:05.156+0000 7f4fbe7f1640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb819d760 con 0x7f4fb80fece0 2024-09-13T18:16:05.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:05.156+0000 7f4fbe7f1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb819d8d0 con 0x7f4fb81008a0 2024-09-13T18:16:05.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:05.156+0000 7f4fbe7f1640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb819da90 con 0x7f4fb80ffee0 2024-09-13T18:16:05.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:05.156+0000 7f4fbcd8e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fb81008a0 0x7f4fb819cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:05.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:05.156+0000 7f4fbcd8e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fb81008a0 0x7f4fb819cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:53770/0 (socket says 172.21.15.134:53770) 2024-09-13T18:16:05.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:05.156+0000 7f4fbcd8e640 1 -- 172.21.15.134:0/1464382990 learned_addr learned my addr 172.21.15.134:0/1464382990 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-13T18:16:05.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:05.156+0000 7f4fb7fff640 1 --2- 172.21.15.134:0/1464382990 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4fb80fece0 0x7f4fb8068a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:05.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:05.156+0000 7f4faffff640 1 --2- 172.21.15.134:0/1464382990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fb80ffee0 0x7f4fb819aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:05.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:05.156+0000 7f4fbcd8e640 1 -- 172.21.15.134:0/1464382990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fb80ffee0 msgr2=0x7f4fb819aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:05.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:05.156+0000 7f4fbcd8e640 1 --2- 172.21.15.134:0/1464382990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fb80ffee0 0x7f4fb819aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:05.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:05.156+0000 7f4fbcd8e640 1 -- 172.21.15.134:0/1464382990 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4fb80fece0 msgr2=0x7f4fb8068a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:05.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:05.156+0000 7f4fbcd8e640 1 --2- 172.21.15.134:0/1464382990 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4fb80fece0 0x7f4fb8068a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:05.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:05.156+0000 7f4fbcd8e640 1 -- 172.21.15.134:0/1464382990 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa8009560 con 0x7f4fb81008a0 2024-09-13T18:16:05.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:05.156+0000 7f4fb7fff640 1 --2- 172.21.15.134:0/1464382990 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4fb80fece0 0x7f4fb8068a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:16:05.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:05.156+0000 7f4fbcd8e640 1 --2- 172.21.15.134:0/1464382990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fb81008a0 0x7f4fb819cfd0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4fb000b970 tx=0x7f4fb000be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:05.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:05.157+0000 7f4fb5ffb640 1 -- 172.21.15.134:0/1464382990 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb000cb40 con 0x7f4fb81008a0 2024-09-13T18:16:05.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:05.157+0000 7f4fb5ffb640 1 -- 172.21.15.134:0/1464382990 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4fb000cca0 con 0x7f4fb81008a0 2024-09-13T18:16:05.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:05.157+0000 7f4fbe7f1640 1 -- 172.21.15.134:0/1464382990 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fb819dd70 con 0x7f4fb81008a0 2024-09-13T18:16:05.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:05.157+0000 7f4fb5ffb640 1 -- 172.21.15.134:0/1464382990 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0007610 con 0x7f4fb81008a0 2024-09-13T18:16:05.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:05.157+0000 7f4fbe7f1640 1 -- 172.21.15.134:0/1464382990 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fb819e1c0 con 0x7f4fb81008a0 2024-09-13T18:16:05.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:05.158+0000 7f4fbe7f1640 1 -- 172.21.15.134:0/1464382990 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fb8106180 con 0x7f4fb81008a0 2024-09-13T18:16:05.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:05.159+0000 7f4fb5ffb640 1 -- 172.21.15.134:0/1464382990 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f4fb0014440 con 0x7f4fb81008a0 2024-09-13T18:16:05.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:05.160+0000 7f4fb5ffb640 1 --2- 172.21.15.134:0/1464382990 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f4fa40a1080 0x7f4fa40a3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:05.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:05.160+0000 7f4fb5ffb640 1 -- 172.21.15.134:0/1464382990 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 3698+0+0 (secure 0 0 0) 0x7f4fb00cb060 con 0x7f4fb81008a0 2024-09-13T18:16:05.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:05.160+0000 7f4fb7fff640 1 --2- 172.21.15.134:0/1464382990 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f4fa40a1080 0x7f4fa40a3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:05.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:05.160+0000 7f4fb7fff640 1 --2- 172.21.15.134:0/1464382990 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f4fa40a1080 0x7f4fa40a3540 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4fa8005ab0 tx=0x7f4fa8005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:05.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:05.164+0000 7f4fb5ffb640 1 -- 172.21.15.134:0/1464382990 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f4fb00959f0 con 0x7f4fb81008a0 2024-09-13T18:16:05.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:05.339+0000 7f4fbe7f1640 1 -- 172.21.15.134:0/1464382990 --> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f4fb802c4f0 con 0x7f4fa40a1080 2024-09-13T18:16:05.342 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:05 smithi134 ceph-mon[34283]: osdmap e27: 4 total, 4 up, 4 in 2024-09-13T18:16:05.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:05 smithi114 ceph-mon[30897]: osdmap e27: 4 total, 4 up, 4 in 2024-09-13T18:16:05.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:05 smithi204 ceph-mon[34777]: osdmap e27: 4 total, 4 up, 4 in 2024-09-13T18:16:06.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:06 smithi134 ceph-mon[34283]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:16:06.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:06 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T18:16:06.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:06 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T18:16:06.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:06 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:16:06.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:06 smithi134 ceph-mon[34283]: osdmap e28: 4 total, 4 up, 4 in 2024-09-13T18:16:06.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:06 smithi134 ceph-mon[34283]: pgmap v98: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-13T18:16:06.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:06 smithi114 ceph-mon[30897]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:16:06.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:06 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T18:16:06.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:06 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T18:16:06.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:06 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:16:06.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:06 smithi114 ceph-mon[30897]: osdmap e28: 4 total, 4 up, 4 in 2024-09-13T18:16:06.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:06 smithi114 ceph-mon[30897]: pgmap v98: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-13T18:16:06.596 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:06 smithi204 ceph-mon[34777]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:16:06.596 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:06 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T18:16:06.596 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:06 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T18:16:06.596 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:06 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:16:06.596 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:06 smithi204 ceph-mon[34777]: osdmap e28: 4 total, 4 up, 4 in 2024-09-13T18:16:06.596 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:06 smithi204 ceph-mon[34777]: pgmap v98: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-13T18:16:07.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:07 smithi204 ceph-mon[34777]: osdmap e29: 4 total, 4 up, 4 in 2024-09-13T18:16:07.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:07 smithi114 ceph-mon[30897]: osdmap e29: 4 total, 4 up, 4 in 2024-09-13T18:16:07.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:07 smithi134 ceph-mon[34283]: osdmap e29: 4 total, 4 up, 4 in 2024-09-13T18:16:08.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:08 smithi134 ceph-mon[34283]: from='client.? 172.21.15.134:0/3537025808' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f38943d6-39c0-42e1-b427-cb6bb90ac38a"}]: dispatch 2024-09-13T18:16:08.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:08 smithi134 ceph-mon[34283]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f38943d6-39c0-42e1-b427-cb6bb90ac38a"}]: dispatch 2024-09-13T18:16:08.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:08 smithi134 ceph-mon[34283]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f38943d6-39c0-42e1-b427-cb6bb90ac38a"}]': finished 2024-09-13T18:16:08.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:08 smithi134 ceph-mon[34283]: osdmap e30: 5 total, 4 up, 5 in 2024-09-13T18:16:08.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:08 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-13T18:16:08.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:08 smithi134 ceph-mon[34283]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-13T18:16:08.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:08 smithi134 ceph-mon[34283]: from='client.? 172.21.15.134:0/354584445' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T18:16:08.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:08 smithi204 ceph-mon[34777]: from='client.? 172.21.15.134:0/3537025808' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f38943d6-39c0-42e1-b427-cb6bb90ac38a"}]: dispatch 2024-09-13T18:16:08.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:08 smithi204 ceph-mon[34777]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f38943d6-39c0-42e1-b427-cb6bb90ac38a"}]: dispatch 2024-09-13T18:16:08.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:08 smithi204 ceph-mon[34777]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f38943d6-39c0-42e1-b427-cb6bb90ac38a"}]': finished 2024-09-13T18:16:08.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:08 smithi204 ceph-mon[34777]: osdmap e30: 5 total, 4 up, 5 in 2024-09-13T18:16:08.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:08 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-13T18:16:08.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:08 smithi204 ceph-mon[34777]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-13T18:16:08.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:08 smithi204 ceph-mon[34777]: from='client.? 172.21.15.134:0/354584445' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T18:16:08.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:08 smithi114 ceph-mon[30897]: from='client.? 172.21.15.134:0/3537025808' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f38943d6-39c0-42e1-b427-cb6bb90ac38a"}]: dispatch 2024-09-13T18:16:08.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:08 smithi114 ceph-mon[30897]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f38943d6-39c0-42e1-b427-cb6bb90ac38a"}]: dispatch 2024-09-13T18:16:08.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:08 smithi114 ceph-mon[30897]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f38943d6-39c0-42e1-b427-cb6bb90ac38a"}]': finished 2024-09-13T18:16:08.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:08 smithi114 ceph-mon[30897]: osdmap e30: 5 total, 4 up, 5 in 2024-09-13T18:16:08.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:08 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-13T18:16:08.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:08 smithi114 ceph-mon[30897]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-13T18:16:08.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:08 smithi114 ceph-mon[30897]: from='client.? 172.21.15.134:0/354584445' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T18:16:11.182 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:10 smithi134 ceph-mon[34283]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-13T18:16:11.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:10 smithi204 ceph-mon[34777]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-13T18:16:11.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:10 smithi114 ceph-mon[30897]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-13T18:16:13.109 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:12 smithi134 ceph-mon[34283]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-13T18:16:13.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:12 smithi204 ceph-mon[34777]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-13T18:16:13.219 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:12 smithi114 ceph-mon[30897]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-13T18:16:15.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:14 smithi134 ceph-mon[34283]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-13T18:16:15.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:14 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-13T18:16:15.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:14 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:16:15.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:14 smithi204 ceph-mon[34777]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-13T18:16:15.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:14 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-13T18:16:15.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:14 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:16:15.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:14 smithi114 ceph-mon[30897]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-13T18:16:15.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:14 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-13T18:16:15.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:14 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:16:16.169 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:15 smithi134 ceph-mon[34283]: Deploying daemon osd.4 on smithi134 2024-09-13T18:16:16.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:15 smithi204 ceph-mon[34777]: Deploying daemon osd.4 on smithi134 2024-09-13T18:16:16.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:15 smithi114 ceph-mon[30897]: Deploying daemon osd.4 on smithi134 2024-09-13T18:16:17.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:16 smithi134 ceph-mon[34283]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-13T18:16:17.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:16 smithi204 ceph-mon[34777]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-13T18:16:17.219 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:16 smithi114 ceph-mon[30897]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-13T18:16:19.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:18 smithi204 ceph-mon[34777]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-13T18:16:19.205 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:18 smithi134 ceph-mon[34283]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-13T18:16:19.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:18 smithi114 ceph-mon[30897]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-13T18:16:20.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:19 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:16:20.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:19 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:20.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:19 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:20.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:19 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:16:20.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:19 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:20.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:19 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:20.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:19 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:16:20.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:19 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:20.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:19 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:21.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:20 smithi204 ceph-mon[34777]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-13T18:16:21.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:20 smithi114 ceph-mon[30897]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-13T18:16:21.232 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:20 smithi134 ceph-mon[34283]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-13T18:16:23.013 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 4 on host 'smithi134' 2024-09-13T18:16:23.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:23.010+0000 7f4fb5ffb640 1 -- 172.21.15.134:0/1464382990 <== mgr.14148 v2:172.21.15.114:6800/3396185541 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f4fb802c4f0 con 0x7f4fa40a1080 2024-09-13T18:16:23.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:23.013+0000 7f4fbe7f1640 1 -- 172.21.15.134:0/1464382990 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f4fa40a1080 msgr2=0x7f4fa40a3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:23.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:23.013+0000 7f4fbe7f1640 1 --2- 172.21.15.134:0/1464382990 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f4fa40a1080 0x7f4fa40a3540 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4fa8005ab0 tx=0x7f4fa8005a20 comp rx=0 tx=0).stop 2024-09-13T18:16:23.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:23.014+0000 7f4fbe7f1640 1 -- 172.21.15.134:0/1464382990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fb81008a0 msgr2=0x7f4fb819cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:23.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:23.014+0000 7f4fbe7f1640 1 --2- 172.21.15.134:0/1464382990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fb81008a0 0x7f4fb819cfd0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4fb000b970 tx=0x7f4fb000be30 comp rx=0 tx=0).stop 2024-09-13T18:16:23.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:23.014+0000 7f4fbe7f1640 1 -- 172.21.15.134:0/1464382990 shutdown_connections 2024-09-13T18:16:23.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:23.014+0000 7f4fbe7f1640 1 --2- 172.21.15.134:0/1464382990 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f4fa40a1080 0x7f4fa40a3540 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:23.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:23.014+0000 7f4fbe7f1640 1 --2- 172.21.15.134:0/1464382990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fb81008a0 0x7f4fb819cfd0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:23.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:23.014+0000 7f4fbe7f1640 1 --2- 172.21.15.134:0/1464382990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fb80ffee0 0x7f4fb819aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:23.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:23.014+0000 7f4fbe7f1640 1 --2- 172.21.15.134:0/1464382990 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4fb80fece0 0x7f4fb8068a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:23.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:23.014+0000 7f4fbe7f1640 1 -- 172.21.15.134:0/1464382990 >> 172.21.15.134:0/1464382990 conn(0x7f4fb80fa490 msgr2=0x7f4fb8103350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:16:23.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:23.014+0000 7f4fbe7f1640 1 -- 172.21.15.134:0/1464382990 shutdown_connections 2024-09-13T18:16:23.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:16:23.014+0000 7f4fbe7f1640 1 -- 172.21.15.134:0/1464382990 wait complete. 2024-09-13T18:16:23.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:22 smithi204 ceph-mon[34777]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-13T18:16:23.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:22 smithi114 ceph-mon[30897]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-13T18:16:23.340 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:22 smithi134 ceph-mon[34283]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-13T18:16:23.631 DEBUG:teuthology.orchestra.run.smithi134:osd.4> sudo journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@osd.4.service 2024-09-13T18:16:23.633 INFO:tasks.cephadm:Deploying osd.5 on smithi204 with /dev/vg_nvme/lv_3... 2024-09-13T18:16:23.633 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-09-13T18:16:23.844 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T18:16:23.868 INFO:journalctl@ceph.osd.4.smithi134.stdout:Sep 13 18:16:23 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-4[45342]: 2024-09-13T18:16:23.761+0000 7fc0c3b1b640 -1 osd.4 0 log_to_monitors true 2024-09-13T18:16:24.221 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:24 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:24.222 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:24 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:24.222 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:24 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:24.222 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:24 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:24.222 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:24 smithi204 ceph-mon[34777]: from='osd.4 [v2:172.21.15.134:6808/1773945333,v1:172.21.15.134:6809/1773945333]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-13T18:16:24.222 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:24 smithi204 ceph-mon[34777]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-13T18:16:24.309 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:24 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:24.309 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:24 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:24.309 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:24 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:24.309 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:24 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:24.309 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:24 smithi134 ceph-mon[34283]: from='osd.4 [v2:172.21.15.134:6808/1773945333,v1:172.21.15.134:6809/1773945333]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-13T18:16:24.309 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:24 smithi134 ceph-mon[34283]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-13T18:16:24.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:24 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:24.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:24 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:24.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:24 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:24.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:24 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:24.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:24 smithi114 ceph-mon[30897]: from='osd.4 [v2:172.21.15.134:6808/1773945333,v1:172.21.15.134:6809/1773945333]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-13T18:16:24.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:24 smithi114 ceph-mon[30897]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-13T18:16:25.286 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:25 smithi134 ceph-mon[34283]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-13T18:16:25.286 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:25 smithi134 ceph-mon[34283]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-13T18:16:25.286 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:25 smithi134 ceph-mon[34283]: osdmap e31: 5 total, 4 up, 5 in 2024-09-13T18:16:25.286 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:25 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-13T18:16:25.286 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:25 smithi134 ceph-mon[34283]: from='osd.4 [v2:172.21.15.134:6808/1773945333,v1:172.21.15.134:6809/1773945333]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-13T18:16:25.286 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:25 smithi134 ceph-mon[34283]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-13T18:16:25.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:25 smithi204 ceph-mon[34777]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-13T18:16:25.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:25 smithi204 ceph-mon[34777]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-13T18:16:25.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:25 smithi204 ceph-mon[34777]: osdmap e31: 5 total, 4 up, 5 in 2024-09-13T18:16:25.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:25 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-13T18:16:25.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:25 smithi204 ceph-mon[34777]: from='osd.4 [v2:172.21.15.134:6808/1773945333,v1:172.21.15.134:6809/1773945333]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-13T18:16:25.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:25 smithi204 ceph-mon[34777]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-13T18:16:25.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:25 smithi114 ceph-mon[30897]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-13T18:16:25.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:25 smithi114 ceph-mon[30897]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-13T18:16:25.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:25 smithi114 ceph-mon[30897]: osdmap e31: 5 total, 4 up, 5 in 2024-09-13T18:16:25.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:25 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-13T18:16:25.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:25 smithi114 ceph-mon[30897]: from='osd.4 [v2:172.21.15.134:6808/1773945333,v1:172.21.15.134:6809/1773945333]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-13T18:16:25.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:25 smithi114 ceph-mon[30897]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-13T18:16:25.697 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-09-13T18:16:25.783 INFO:teuthology.orchestra.run.smithi204.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-09-13T18:16:25.797 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10+0 records in 2024-09-13T18:16:25.797 INFO:teuthology.orchestra.run.smithi204.stderr:10+0 records out 2024-09-13T18:16:25.797 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0125491 s, 836 MB/s 2024-09-13T18:16:25.797 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping successful for: 2024-09-13T18:16:25.959 INFO:journalctl@ceph.osd.4.smithi134.stdout:Sep 13 18:16:25 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-4[45342]: 2024-09-13T18:16:25.700+0000 7fc0bfa9c640 -1 osd.4 0 waiting for initial osdmap 2024-09-13T18:16:25.959 INFO:journalctl@ceph.osd.4.smithi134.stdout:Sep 13 18:16:25 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-4[45342]: 2024-09-13T18:16:25.706+0000 7fc0bb8c5640 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T18:16:26.431 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi204:vg_nvme/lv_3 2024-09-13T18:16:26.640 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T18:16:26.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:26 smithi134 ceph-mon[34283]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-13T18:16:26.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:26 smithi134 ceph-mon[34283]: osdmap e32: 5 total, 4 up, 5 in 2024-09-13T18:16:26.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:26 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-13T18:16:26.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:26 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-13T18:16:26.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:26 smithi134 ceph-mon[34283]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-13T18:16:26.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:26 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:26.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:26 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:26.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:26 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:16:26.870 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:26 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:16:26.870 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:26 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:26.870 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:26 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:16:26.870 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:26 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:16:26.870 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:26 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:26.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:26 smithi114 ceph-mon[30897]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-13T18:16:26.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:26 smithi114 ceph-mon[30897]: osdmap e32: 5 total, 4 up, 5 in 2024-09-13T18:16:26.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:26 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-13T18:16:26.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:26 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-13T18:16:26.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:26 smithi114 ceph-mon[30897]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-13T18:16:26.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:26 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:26.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:26 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:26.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:26 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:16:26.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:26 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:16:26.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:26 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:26.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:26 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:16:26.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:26 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:16:26.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:26 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:27.028 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:26 smithi204 ceph-mon[34777]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-13T18:16:27.028 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:26 smithi204 ceph-mon[34777]: osdmap e32: 5 total, 4 up, 5 in 2024-09-13T18:16:27.028 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:26 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-13T18:16:27.028 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:26 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-13T18:16:27.028 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:26 smithi204 ceph-mon[34777]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-13T18:16:27.028 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:26 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:27.028 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:26 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:27.028 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:26 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:16:27.029 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:26 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:16:27.029 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:26 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:27.029 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:26 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:16:27.029 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:26 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:16:27.029 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:26 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:27.823 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:27 smithi204 ceph-mon[34777]: purged_snaps scrub starts 2024-09-13T18:16:27.823 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:27 smithi204 ceph-mon[34777]: purged_snaps scrub ok 2024-09-13T18:16:27.823 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:27 smithi204 ceph-mon[34777]: Detected new or changed devices on smithi134 2024-09-13T18:16:27.824 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:27 smithi204 ceph-mon[34777]: Adjusting osd_memory_target on smithi134 to 8544M 2024-09-13T18:16:27.824 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:27 smithi204 ceph-mon[34777]: osd.4 [v2:172.21.15.134:6808/1773945333,v1:172.21.15.134:6809/1773945333] boot 2024-09-13T18:16:27.824 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:27 smithi204 ceph-mon[34777]: osdmap e33: 5 total, 5 up, 5 in 2024-09-13T18:16:27.824 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:27 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-13T18:16:27.824 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:27 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:16:27.824 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:27 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:16:27.824 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:27 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:16:27.824 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:27 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:27.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:27 smithi114 ceph-mon[30897]: purged_snaps scrub starts 2024-09-13T18:16:27.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:27 smithi114 ceph-mon[30897]: purged_snaps scrub ok 2024-09-13T18:16:27.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:27 smithi114 ceph-mon[30897]: Detected new or changed devices on smithi134 2024-09-13T18:16:27.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:27 smithi114 ceph-mon[30897]: Adjusting osd_memory_target on smithi134 to 8544M 2024-09-13T18:16:27.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:27 smithi114 ceph-mon[30897]: osd.4 [v2:172.21.15.134:6808/1773945333,v1:172.21.15.134:6809/1773945333] boot 2024-09-13T18:16:27.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:27 smithi114 ceph-mon[30897]: osdmap e33: 5 total, 5 up, 5 in 2024-09-13T18:16:27.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:27 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-13T18:16:27.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:27 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:16:27.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:27 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:16:27.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:27 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:16:27.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:27 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:28.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:27 smithi134 ceph-mon[34283]: purged_snaps scrub starts 2024-09-13T18:16:28.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:27 smithi134 ceph-mon[34283]: purged_snaps scrub ok 2024-09-13T18:16:28.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:27 smithi134 ceph-mon[34283]: Detected new or changed devices on smithi134 2024-09-13T18:16:28.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:27 smithi134 ceph-mon[34283]: Adjusting osd_memory_target on smithi134 to 8544M 2024-09-13T18:16:28.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:27 smithi134 ceph-mon[34283]: osd.4 [v2:172.21.15.134:6808/1773945333,v1:172.21.15.134:6809/1773945333] boot 2024-09-13T18:16:28.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:27 smithi134 ceph-mon[34283]: osdmap e33: 5 total, 5 up, 5 in 2024-09-13T18:16:28.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:27 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-13T18:16:28.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:27 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:16:28.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:27 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:16:28.120 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:27 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:16:28.120 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:27 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:28.351 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:16:28.332+0000 7f99d9d82640 1 -- 172.21.15.204:0/2520511874 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99d41002c0 msgr2=0x7f99d41006c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:28.352 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:16:28.332+0000 7f99d9d82640 1 --2- 172.21.15.204:0/2520511874 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99d41002c0 0x7f99d41006c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f99bc009900 tx=0x7f99bc02a410 comp rx=0 tx=0).stop 2024-09-13T18:16:28.352 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:16:28.332+0000 7f99d9d82640 1 -- 172.21.15.204:0/2520511874 shutdown_connections 2024-09-13T18:16:28.352 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:16:28.332+0000 7f99d9d82640 1 --2- 172.21.15.204:0/2520511874 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99d40ff930 0x7f99d41071d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:28.352 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:16:28.332+0000 7f99d9d82640 1 --2- 172.21.15.204:0/2520511874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99d40fef70 0x7f99d40ff3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:28.352 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:16:28.332+0000 7f99d9d82640 1 --2- 172.21.15.204:0/2520511874 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99d41002c0 0x7f99d41006c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:28.352 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:16:28.332+0000 7f99d9d82640 1 -- 172.21.15.204:0/2520511874 >> 172.21.15.204:0/2520511874 conn(0x7f99d40f9c10 msgr2=0x7f99d40fc030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:16:28.352 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:16:28.332+0000 7f99d9d82640 1 -- 172.21.15.204:0/2520511874 shutdown_connections 2024-09-13T18:16:28.352 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:16:28.332+0000 7f99d9d82640 1 -- 172.21.15.204:0/2520511874 wait complete. 2024-09-13T18:16:28.352 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:16:28.332+0000 7f99d9d82640 1 Processor -- start 2024-09-13T18:16:28.352 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:16:28.333+0000 7f99d9d82640 1 -- start start 2024-09-13T18:16:28.352 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:16:28.333+0000 7f99d9d82640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99d40fef70 0x7f99d40fe9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:28.353 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:16:28.333+0000 7f99d9d82640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99d40ff930 0x7f99d419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:28.353 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:16:28.333+0000 7f99d9d82640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99d41002c0 0x7f99d419d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:28.353 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:16:28.333+0000 7f99d9d82640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d419d860 con 0x7f99d41002c0 2024-09-13T18:16:28.353 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:16:28.333+0000 7f99d9d82640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d419d9d0 con 0x7f99d40ff930 2024-09-13T18:16:28.353 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:16:28.333+0000 7f99d9d82640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d419db90 con 0x7f99d40fef70 2024-09-13T18:16:28.353 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:16:28.333+0000 7f99d37fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99d40fef70 0x7f99d40fe9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:28.353 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:16:28.333+0000 7f99d37fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99d40fef70 0x7f99d40fe9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:34502/0 (socket says 172.21.15.204:34502) 2024-09-13T18:16:28.353 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:16:28.333+0000 7f99d37fe640 1 -- 172.21.15.204:0/678813495 learned_addr learned my addr 172.21.15.204:0/678813495 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-13T18:16:28.353 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:16:28.333+0000 7f99d3fff640 1 --2- 172.21.15.204:0/678813495 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99d41002c0 0x7f99d419d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:28.353 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:16:28.333+0000 7f99d2ffd640 1 --2- 172.21.15.204:0/678813495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99d40ff930 0x7f99d419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:28.354 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:16:28.333+0000 7f99d37fe640 1 -- 172.21.15.204:0/678813495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99d40ff930 msgr2=0x7f99d419aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:28.354 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:16:28.333+0000 7f99d37fe640 1 --2- 172.21.15.204:0/678813495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99d40ff930 0x7f99d419aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:28.354 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:16:28.333+0000 7f99d37fe640 1 -- 172.21.15.204:0/678813495 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99d41002c0 msgr2=0x7f99d419d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:28.354 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:16:28.333+0000 7f99d37fe640 1 --2- 172.21.15.204:0/678813495 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99d41002c0 0x7f99d419d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:28.354 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:16:28.333+0000 7f99d37fe640 1 -- 172.21.15.204:0/678813495 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99bc009560 con 0x7f99d40fef70 2024-09-13T18:16:28.354 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:16:28.333+0000 7f99d37fe640 1 --2- 172.21.15.204:0/678813495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99d40fef70 0x7f99d40fe9c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f99bc005ab0 tx=0x7f99bc0042d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:28.354 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:16:28.334+0000 7f99d0ff9640 1 -- 172.21.15.204:0/678813495 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99bc031d30 con 0x7f99d40fef70 2024-09-13T18:16:28.354 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:16:28.334+0000 7f99d9d82640 1 -- 172.21.15.204:0/678813495 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99d419ddb0 con 0x7f99d40fef70 2024-09-13T18:16:28.354 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:16:28.334+0000 7f99d0ff9640 1 -- 172.21.15.204:0/678813495 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f99bc031e90 con 0x7f99d40fef70 2024-09-13T18:16:28.354 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:16:28.334+0000 7f99d0ff9640 1 -- 172.21.15.204:0/678813495 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99bc003af0 con 0x7f99d40fef70 2024-09-13T18:16:28.354 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:16:28.334+0000 7f99d9d82640 1 -- 172.21.15.204:0/678813495 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99d419e240 con 0x7f99d40fef70 2024-09-13T18:16:28.354 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:16:28.335+0000 7f99d9d82640 1 -- 172.21.15.204:0/678813495 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99d40ff3f0 con 0x7f99d40fef70 2024-09-13T18:16:28.355 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:16:28.336+0000 7f99d0ff9640 1 -- 172.21.15.204:0/678813495 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f99bc004be0 con 0x7f99d40fef70 2024-09-13T18:16:28.355 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:16:28.336+0000 7f99d0ff9640 1 --2- 172.21.15.204:0/678813495 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f99c00a1080 0x7f99c00a3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:28.355 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:16:28.337+0000 7f99d0ff9640 1 -- 172.21.15.204:0/678813495 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f99bc0ecb30 con 0x7f99d40fef70 2024-09-13T18:16:28.355 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:16:28.337+0000 7f99d2ffd640 1 --2- 172.21.15.204:0/678813495 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f99c00a1080 0x7f99c00a3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:28.355 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:16:28.337+0000 7f99d2ffd640 1 --2- 172.21.15.204:0/678813495 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f99c00a1080 0x7f99c00a3540 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f99d40fef40 tx=0x7f99c400b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:28.355 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:16:28.340+0000 7f99d0ff9640 1 -- 172.21.15.204:0/678813495 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f99bc0b7500 con 0x7f99d40fef70 2024-09-13T18:16:28.515 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:16:28.513+0000 7f99d9d82640 1 -- 172.21.15.204:0/678813495 --> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f99d402c4c0 con 0x7f99c00a1080 2024-09-13T18:16:28.873 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:28 smithi204 ceph-mon[34777]: osdmap e34: 5 total, 5 up, 5 in 2024-09-13T18:16:28.873 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:28 smithi204 ceph-mon[34777]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-09-13T18:16:28.873 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:28 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T18:16:28.873 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:28 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T18:16:28.873 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:28 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:16:28.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:28 smithi114 ceph-mon[30897]: osdmap e34: 5 total, 5 up, 5 in 2024-09-13T18:16:28.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:28 smithi114 ceph-mon[30897]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-09-13T18:16:28.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:28 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T18:16:28.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:28 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T18:16:28.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:28 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:16:29.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:28 smithi134 ceph-mon[34283]: osdmap e34: 5 total, 5 up, 5 in 2024-09-13T18:16:29.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:28 smithi134 ceph-mon[34283]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-09-13T18:16:29.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:28 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T18:16:29.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:28 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T18:16:29.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:28 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:16:29.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:29 smithi114 ceph-mon[30897]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:16:30.017 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:29 smithi204 ceph-mon[34777]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:16:30.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:29 smithi134 ceph-mon[34283]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:16:30.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:30 smithi114 ceph-mon[30897]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-09-13T18:16:31.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:30 smithi134 ceph-mon[34283]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-09-13T18:16:31.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:30 smithi204 ceph-mon[34777]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-09-13T18:16:31.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:31 smithi204 ceph-mon[34777]: from='client.? 172.21.15.204:0/2602025729' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb38f361-8e00-4178-9752-5ddbf3af20e1"}]: dispatch 2024-09-13T18:16:31.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:31 smithi204 ceph-mon[34777]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb38f361-8e00-4178-9752-5ddbf3af20e1"}]: dispatch 2024-09-13T18:16:31.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:31 smithi204 ceph-mon[34777]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bb38f361-8e00-4178-9752-5ddbf3af20e1"}]': finished 2024-09-13T18:16:31.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:31 smithi204 ceph-mon[34777]: osdmap e35: 6 total, 5 up, 6 in 2024-09-13T18:16:31.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:31 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T18:16:31.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:31 smithi204 ceph-mon[34777]: from='client.? 172.21.15.204:0/1212491293' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T18:16:32.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:31 smithi134 ceph-mon[34283]: from='client.? 172.21.15.204:0/2602025729' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb38f361-8e00-4178-9752-5ddbf3af20e1"}]: dispatch 2024-09-13T18:16:32.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:31 smithi134 ceph-mon[34283]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb38f361-8e00-4178-9752-5ddbf3af20e1"}]: dispatch 2024-09-13T18:16:32.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:31 smithi134 ceph-mon[34283]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bb38f361-8e00-4178-9752-5ddbf3af20e1"}]': finished 2024-09-13T18:16:32.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:31 smithi134 ceph-mon[34283]: osdmap e35: 6 total, 5 up, 6 in 2024-09-13T18:16:32.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:31 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T18:16:32.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:31 smithi134 ceph-mon[34283]: from='client.? 172.21.15.204:0/1212491293' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T18:16:32.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:31 smithi114 ceph-mon[30897]: from='client.? 172.21.15.204:0/2602025729' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb38f361-8e00-4178-9752-5ddbf3af20e1"}]: dispatch 2024-09-13T18:16:32.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:31 smithi114 ceph-mon[30897]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb38f361-8e00-4178-9752-5ddbf3af20e1"}]: dispatch 2024-09-13T18:16:32.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:31 smithi114 ceph-mon[30897]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bb38f361-8e00-4178-9752-5ddbf3af20e1"}]': finished 2024-09-13T18:16:32.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:31 smithi114 ceph-mon[30897]: osdmap e35: 6 total, 5 up, 6 in 2024-09-13T18:16:32.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:31 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T18:16:32.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:31 smithi114 ceph-mon[30897]: from='client.? 172.21.15.204:0/1212491293' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T18:16:33.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:32 smithi134 ceph-mon[34283]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-09-13T18:16:33.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:32 smithi204 ceph-mon[34777]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-09-13T18:16:33.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:32 smithi114 ceph-mon[30897]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-09-13T18:16:35.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:34 smithi204 ceph-mon[34777]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-13T18:16:35.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:34 smithi114 ceph-mon[30897]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-13T18:16:35.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:34 smithi134 ceph-mon[34283]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-13T18:16:37.137 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:36 smithi204 ceph-mon[34777]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-13T18:16:37.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:36 smithi114 ceph-mon[30897]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-13T18:16:37.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:36 smithi134 ceph-mon[34283]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-13T18:16:39.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:38 smithi204 ceph-mon[34777]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-13T18:16:39.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:38 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-13T18:16:39.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:38 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:16:39.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:38 smithi114 ceph-mon[30897]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-13T18:16:39.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:38 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-13T18:16:39.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:38 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:16:39.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:38 smithi134 ceph-mon[34283]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-13T18:16:39.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:38 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-13T18:16:39.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:38 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:16:40.110 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:39 smithi204 ceph-mon[34777]: Deploying daemon osd.5 on smithi204 2024-09-13T18:16:40.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:39 smithi114 ceph-mon[30897]: Deploying daemon osd.5 on smithi204 2024-09-13T18:16:40.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:39 smithi134 ceph-mon[34283]: Deploying daemon osd.5 on smithi204 2024-09-13T18:16:41.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:40 smithi204 ceph-mon[34777]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-13T18:16:41.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:40 smithi114 ceph-mon[30897]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-13T18:16:41.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:40 smithi134 ceph-mon[34283]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-13T18:16:43.150 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:42 smithi204 ceph-mon[34777]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-13T18:16:43.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:42 smithi114 ceph-mon[30897]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-13T18:16:43.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:42 smithi134 ceph-mon[34283]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-13T18:16:44.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:43 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:16:44.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:43 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:44.187 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:43 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:44.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:43 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:16:44.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:43 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:44.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:43 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:44.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:43 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:16:44.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:43 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:44.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:43 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:45.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:44 smithi204 ceph-mon[34777]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-13T18:16:45.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:44 smithi114 ceph-mon[30897]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-13T18:16:45.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:44 smithi134 ceph-mon[34283]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-13T18:16:46.988 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 5 on host 'smithi204' 2024-09-13T18:16:46.989 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:16:46.986+0000 7f99d0ff9640 1 -- 172.21.15.204:0/678813495 <== mgr.14148 v2:172.21.15.114:6800/3396185541 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f99d402c4c0 con 0x7f99c00a1080 2024-09-13T18:16:46.991 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:16:46.988+0000 7f99d9d82640 1 -- 172.21.15.204:0/678813495 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f99c00a1080 msgr2=0x7f99c00a3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:46.991 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:16:46.988+0000 7f99d9d82640 1 --2- 172.21.15.204:0/678813495 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f99c00a1080 0x7f99c00a3540 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f99d40fef40 tx=0x7f99c400b390 comp rx=0 tx=0).stop 2024-09-13T18:16:46.991 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:16:46.988+0000 7f99d9d82640 1 -- 172.21.15.204:0/678813495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99d40fef70 msgr2=0x7f99d40fe9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:46.991 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:16:46.988+0000 7f99d9d82640 1 --2- 172.21.15.204:0/678813495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99d40fef70 0x7f99d40fe9c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f99bc005ab0 tx=0x7f99bc0042d0 comp rx=0 tx=0).stop 2024-09-13T18:16:46.991 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:16:46.988+0000 7f99d9d82640 1 -- 172.21.15.204:0/678813495 shutdown_connections 2024-09-13T18:16:46.991 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:16:46.988+0000 7f99d9d82640 1 --2- 172.21.15.204:0/678813495 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f99c00a1080 0x7f99c00a3540 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:46.991 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:16:46.988+0000 7f99d9d82640 1 --2- 172.21.15.204:0/678813495 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99d41002c0 0x7f99d419d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:46.991 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:16:46.988+0000 7f99d9d82640 1 --2- 172.21.15.204:0/678813495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99d40ff930 0x7f99d419aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:46.991 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:16:46.988+0000 7f99d9d82640 1 --2- 172.21.15.204:0/678813495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99d40fef70 0x7f99d40fe9c0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:46.992 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:16:46.988+0000 7f99d9d82640 1 -- 172.21.15.204:0/678813495 >> 172.21.15.204:0/678813495 conn(0x7f99d40f9c10 msgr2=0x7f99d4103490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:16:46.992 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:16:46.989+0000 7f99d9d82640 1 -- 172.21.15.204:0/678813495 shutdown_connections 2024-09-13T18:16:46.992 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:16:46.989+0000 7f99d9d82640 1 -- 172.21.15.204:0/678813495 wait complete. 2024-09-13T18:16:47.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:46 smithi204 ceph-mon[34777]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-13T18:16:47.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:46 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:47.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:46 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:47.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:46 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:16:47.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:46 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:16:47.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:46 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:47.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:46 smithi114 ceph-mon[30897]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-13T18:16:47.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:46 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:47.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:46 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:47.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:46 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:16:47.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:46 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:16:47.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:46 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:47.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:46 smithi134 ceph-mon[34283]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-13T18:16:47.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:46 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:47.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:46 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:47.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:46 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:16:47.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:46 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:16:47.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:46 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:47.668 DEBUG:teuthology.orchestra.run.smithi204:osd.5> sudo journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@osd.5.service 2024-09-13T18:16:47.670 INFO:tasks.cephadm:Deploying osd.6 on smithi114 with /dev/vg_nvme/lv_2... 2024-09-13T18:16:47.670 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-09-13T18:16:47.876 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:16:47.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:47 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:16:47.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:47 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:47.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:47 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:47.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:47 smithi114 ceph-mon[30897]: from='osd.5 [v2:172.21.15.204:6808/2371778055,v1:172.21.15.204:6809/2371778055]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-13T18:16:47.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:47 smithi114 ceph-mon[30897]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-13T18:16:48.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:47 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:16:48.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:47 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:48.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:47 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:48.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:47 smithi204 ceph-mon[34777]: from='osd.5 [v2:172.21.15.204:6808/2371778055,v1:172.21.15.204:6809/2371778055]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-13T18:16:48.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:47 smithi204 ceph-mon[34777]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-13T18:16:48.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:47 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:16:48.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:47 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:48.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:47 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:48.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:47 smithi134 ceph-mon[34283]: from='osd.5 [v2:172.21.15.204:6808/2371778055,v1:172.21.15.204:6809/2371778055]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-13T18:16:48.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:47 smithi134 ceph-mon[34283]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-13T18:16:49.169 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:49 smithi204 ceph-mon[34777]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-13T18:16:49.170 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:49 smithi204 ceph-mon[34777]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-13T18:16:49.170 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:49 smithi204 ceph-mon[34777]: osdmap e36: 6 total, 5 up, 6 in 2024-09-13T18:16:49.170 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:49 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T18:16:49.170 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:49 smithi204 ceph-mon[34777]: from='osd.5 [v2:172.21.15.204:6808/2371778055,v1:172.21.15.204:6809/2371778055]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-13T18:16:49.170 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:49 smithi204 ceph-mon[34777]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-13T18:16:49.170 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 13 18:16:49 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-5[46242]: 2024-09-13T18:16:49.005+0000 7f0e18c25640 -1 osd.5 0 waiting for initial osdmap 2024-09-13T18:16:49.170 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 13 18:16:49 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-5[46242]: 2024-09-13T18:16:49.014+0000 7f0e14a4e640 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T18:16:49.288 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:49 smithi114 ceph-mon[30897]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-13T18:16:49.288 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:49 smithi114 ceph-mon[30897]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-13T18:16:49.288 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:49 smithi114 ceph-mon[30897]: osdmap e36: 6 total, 5 up, 6 in 2024-09-13T18:16:49.288 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:49 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T18:16:49.289 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:49 smithi114 ceph-mon[30897]: from='osd.5 [v2:172.21.15.204:6808/2371778055,v1:172.21.15.204:6809/2371778055]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-13T18:16:49.289 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:49 smithi114 ceph-mon[30897]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-13T18:16:49.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:49 smithi134 ceph-mon[34283]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-13T18:16:49.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:49 smithi134 ceph-mon[34283]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-13T18:16:49.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:49 smithi134 ceph-mon[34283]: osdmap e36: 6 total, 5 up, 6 in 2024-09-13T18:16:49.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:49 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T18:16:49.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:49 smithi134 ceph-mon[34283]: from='osd.5 [v2:172.21.15.204:6808/2371778055,v1:172.21.15.204:6809/2371778055]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-13T18:16:49.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:49 smithi134 ceph-mon[34283]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-13T18:16:49.797 INFO:teuthology.orchestra.run.smithi114.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-09-13T18:16:49.880 INFO:teuthology.orchestra.run.smithi114.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-09-13T18:16:49.896 INFO:teuthology.orchestra.run.smithi114.stderr: stderr: 10+0 records in 2024-09-13T18:16:49.896 INFO:teuthology.orchestra.run.smithi114.stderr:10+0 records out 2024-09-13T18:16:49.896 INFO:teuthology.orchestra.run.smithi114.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0131302 s, 799 MB/s 2024-09-13T18:16:49.896 INFO:teuthology.orchestra.run.smithi114.stderr:--> Zapping successful for: 2024-09-13T18:16:50.170 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:50 smithi204 ceph-mon[34777]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-09-13T18:16:50.170 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:50 smithi204 ceph-mon[34777]: osdmap e37: 6 total, 5 up, 6 in 2024-09-13T18:16:50.170 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:50 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T18:16:50.170 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:50 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T18:16:50.213 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:50 smithi114 ceph-mon[30897]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-09-13T18:16:50.213 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:50 smithi114 ceph-mon[30897]: osdmap e37: 6 total, 5 up, 6 in 2024-09-13T18:16:50.213 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:50 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T18:16:50.213 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:50 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T18:16:50.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:50 smithi134 ceph-mon[34283]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-09-13T18:16:50.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:50 smithi134 ceph-mon[34283]: osdmap e37: 6 total, 5 up, 6 in 2024-09-13T18:16:50.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:50 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T18:16:50.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:50 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T18:16:50.537 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi114:vg_nvme/lv_2 2024-09-13T18:16:50.749 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:16:51.115 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:51 smithi114 ceph-mon[30897]: purged_snaps scrub starts 2024-09-13T18:16:51.115 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:51 smithi114 ceph-mon[30897]: purged_snaps scrub ok 2024-09-13T18:16:51.115 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:51 smithi114 ceph-mon[30897]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-13T18:16:51.115 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:51 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T18:16:51.115 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:51 smithi114 ceph-mon[30897]: osd.5 [v2:172.21.15.204:6808/2371778055,v1:172.21.15.204:6809/2371778055] boot 2024-09-13T18:16:51.115 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:51 smithi114 ceph-mon[30897]: osdmap e38: 6 total, 6 up, 6 in 2024-09-13T18:16:51.115 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:51 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T18:16:51.115 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:51 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:51.116 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:51 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:51.116 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:51 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:16:51.116 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:51 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:16:51.116 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:51 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:51.116 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:51 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:16:51.116 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:51 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:16:51.116 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:51 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:51.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:51 smithi134 ceph-mon[34283]: purged_snaps scrub starts 2024-09-13T18:16:51.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:51 smithi134 ceph-mon[34283]: purged_snaps scrub ok 2024-09-13T18:16:51.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:51 smithi134 ceph-mon[34283]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-13T18:16:51.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:51 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T18:16:51.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:51 smithi134 ceph-mon[34283]: osd.5 [v2:172.21.15.204:6808/2371778055,v1:172.21.15.204:6809/2371778055] boot 2024-09-13T18:16:51.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:51 smithi134 ceph-mon[34283]: osdmap e38: 6 total, 6 up, 6 in 2024-09-13T18:16:51.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:51 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T18:16:51.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:51 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:51.371 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:51 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:51.371 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:51 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:16:51.371 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:51 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:16:51.371 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:51 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:51.371 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:51 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:16:51.371 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:51 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:16:51.372 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:51 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:51.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:51 smithi204 ceph-mon[34777]: purged_snaps scrub starts 2024-09-13T18:16:51.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:51 smithi204 ceph-mon[34777]: purged_snaps scrub ok 2024-09-13T18:16:51.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:51 smithi204 ceph-mon[34777]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-13T18:16:51.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:51 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T18:16:51.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:51 smithi204 ceph-mon[34777]: osd.5 [v2:172.21.15.204:6808/2371778055,v1:172.21.15.204:6809/2371778055] boot 2024-09-13T18:16:51.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:51 smithi204 ceph-mon[34777]: osdmap e38: 6 total, 6 up, 6 in 2024-09-13T18:16:51.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:51 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T18:16:51.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:51 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:51.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:51 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:51.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:51 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:16:51.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:51 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:16:51.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:51 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:51.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:51 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:16:51.447 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:51 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:16:51.448 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:51 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:16:52.114 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:52 smithi114 ceph-mon[30897]: Detected new or changed devices on smithi204 2024-09-13T18:16:52.114 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:52 smithi114 ceph-mon[30897]: Adjusting osd_memory_target on smithi204 to 8544M 2024-09-13T18:16:52.114 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:52 smithi114 ceph-mon[30897]: osdmap e39: 6 total, 6 up, 6 in 2024-09-13T18:16:52.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:16:52.309+0000 7f2674468640 1 -- 172.21.15.114:0/368675990 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f266c100f60 msgr2=0x7f266c1013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:52.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:16:52.309+0000 7f2674468640 1 --2- 172.21.15.114:0/368675990 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f266c100f60 0x7f266c1013e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2660009900 tx=0x7f266002a410 comp rx=0 tx=0).stop 2024-09-13T18:16:52.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:16:52.309+0000 7f2674468640 1 -- 172.21.15.114:0/368675990 shutdown_connections 2024-09-13T18:16:52.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:16:52.309+0000 7f2674468640 1 --2- 172.21.15.114:0/368675990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f266c101920 0x7f266c108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:52.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:16:52.309+0000 7f2674468640 1 --2- 172.21.15.114:0/368675990 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f266c100f60 0x7f266c1013e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:52.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:16:52.309+0000 7f2674468640 1 --2- 172.21.15.114:0/368675990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f266c0ffe00 0x7f266c100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:52.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:16:52.309+0000 7f2674468640 1 -- 172.21.15.114:0/368675990 >> 172.21.15.114:0/368675990 conn(0x7f266c0fb5d0 msgr2=0x7f266c0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:16:52.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:16:52.309+0000 7f2674468640 1 -- 172.21.15.114:0/368675990 shutdown_connections 2024-09-13T18:16:52.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:16:52.309+0000 7f2674468640 1 -- 172.21.15.114:0/368675990 wait complete. 2024-09-13T18:16:52.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:16:52.309+0000 7f2674468640 1 Processor -- start 2024-09-13T18:16:52.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:16:52.309+0000 7f2674468640 1 -- start start 2024-09-13T18:16:52.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:16:52.310+0000 7f2674468640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f266c0ffe00 0x7f266c195cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:52.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:16:52.310+0000 7f2674468640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f266c100f60 0x7f266c196210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:52.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:16:52.310+0000 7f2674468640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f266c101920 0x7f266c19d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:52.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:16:52.310+0000 7f2674468640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f266c19d770 con 0x7f266c101920 2024-09-13T18:16:52.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:16:52.310+0000 7f2674468640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f266c19d8e0 con 0x7f266c0ffe00 2024-09-13T18:16:52.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:16:52.310+0000 7f2674468640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f266c19daa0 con 0x7f266c100f60 2024-09-13T18:16:52.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:16:52.310+0000 7f2672a05640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f266c101920 0x7f266c19d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:52.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:16:52.310+0000 7f2672a05640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f266c101920 0x7f266c19d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40154/0 (socket says 172.21.15.114:40154) 2024-09-13T18:16:52.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:16:52.310+0000 7f2672a05640 1 -- 172.21.15.114:0/3647097700 learned_addr learned my addr 172.21.15.114:0/3647097700 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:16:52.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:16:52.310+0000 7f2671a03640 1 --2- 172.21.15.114:0/3647097700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f266c100f60 0x7f266c196210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:52.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:16:52.310+0000 7f2672204640 1 --2- 172.21.15.114:0/3647097700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f266c0ffe00 0x7f266c195cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:52.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:16:52.310+0000 7f2672a05640 1 -- 172.21.15.114:0/3647097700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f266c100f60 msgr2=0x7f266c196210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:52.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:16:52.310+0000 7f2672a05640 1 --2- 172.21.15.114:0/3647097700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f266c100f60 0x7f266c196210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:52.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:16:52.310+0000 7f2672a05640 1 -- 172.21.15.114:0/3647097700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f266c0ffe00 msgr2=0x7f266c195cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:52.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:16:52.310+0000 7f2672a05640 1 --2- 172.21.15.114:0/3647097700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f266c0ffe00 0x7f266c195cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:52.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:16:52.310+0000 7f2672a05640 1 -- 172.21.15.114:0/3647097700 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2660009560 con 0x7f266c101920 2024-09-13T18:16:52.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:16:52.310+0000 7f2672a05640 1 --2- 172.21.15.114:0/3647097700 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f266c101920 0x7f266c19d230 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f266400eaa0 tx=0x7f266400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:52.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:16:52.310+0000 7f266b7fe640 1 -- 172.21.15.114:0/3647097700 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266400cdb0 con 0x7f266c101920 2024-09-13T18:16:52.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:16:52.310+0000 7f266b7fe640 1 -- 172.21.15.114:0/3647097700 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2664005020 con 0x7f266c101920 2024-09-13T18:16:52.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:16:52.310+0000 7f2674468640 1 -- 172.21.15.114:0/3647097700 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f266c19dd80 con 0x7f266c101920 2024-09-13T18:16:52.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:16:52.310+0000 7f266b7fe640 1 -- 172.21.15.114:0/3647097700 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26640116d0 con 0x7f266c101920 2024-09-13T18:16:52.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:16:52.310+0000 7f2674468640 1 -- 172.21.15.114:0/3647097700 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f266c19e1a0 con 0x7f266c101920 2024-09-13T18:16:52.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:16:52.311+0000 7f2674468640 1 -- 172.21.15.114:0/3647097700 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f266c107350 con 0x7f266c101920 2024-09-13T18:16:52.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:16:52.312+0000 7f266b7fe640 1 -- 172.21.15.114:0/3647097700 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f2664005190 con 0x7f266c101920 2024-09-13T18:16:52.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:16:52.313+0000 7f266b7fe640 1 --2- 172.21.15.114:0/3647097700 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f26540a1080 0x7f26540a3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:52.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:16:52.313+0000 7f2672204640 1 --2- 172.21.15.114:0/3647097700 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f26540a1080 0x7f26540a3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:52.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:16:52.313+0000 7f266b7fe640 1 -- 172.21.15.114:0/3647097700 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4562+0+0 (secure 0 0 0) 0x7f26640cbcd0 con 0x7f266c101920 2024-09-13T18:16:52.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:16:52.313+0000 7f2672204640 1 --2- 172.21.15.114:0/3647097700 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f26540a1080 0x7f26540a3540 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f265c005fd0 tx=0x7f265c005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:52.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:16:52.317+0000 7f266b7fe640 1 -- 172.21.15.114:0/3647097700 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f2664096300 con 0x7f266c101920 2024-09-13T18:16:52.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:52 smithi134 ceph-mon[34283]: Detected new or changed devices on smithi204 2024-09-13T18:16:52.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:52 smithi134 ceph-mon[34283]: Adjusting osd_memory_target on smithi204 to 8544M 2024-09-13T18:16:52.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:52 smithi134 ceph-mon[34283]: osdmap e39: 6 total, 6 up, 6 in 2024-09-13T18:16:52.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:52 smithi204 ceph-mon[34777]: Detected new or changed devices on smithi204 2024-09-13T18:16:52.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:52 smithi204 ceph-mon[34777]: Adjusting osd_memory_target on smithi204 to 8544M 2024-09-13T18:16:52.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:52 smithi204 ceph-mon[34777]: osdmap e39: 6 total, 6 up, 6 in 2024-09-13T18:16:52.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:16:52.489+0000 7f2674468640 1 -- 172.21.15.114:0/3647097700 --> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f266c02c4f0 con 0x7f26540a1080 2024-09-13T18:16:53.125 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:53 smithi114 ceph-mon[30897]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-09-13T18:16:53.126 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:53 smithi114 ceph-mon[30897]: osdmap e40: 6 total, 6 up, 6 in 2024-09-13T18:16:53.126 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:53 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T18:16:53.126 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:53 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T18:16:53.126 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:53 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:16:53.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:53 smithi134 ceph-mon[34283]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-09-13T18:16:53.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:53 smithi134 ceph-mon[34283]: osdmap e40: 6 total, 6 up, 6 in 2024-09-13T18:16:53.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:53 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T18:16:53.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:53 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T18:16:53.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:53 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:16:53.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:53 smithi204 ceph-mon[34777]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-09-13T18:16:53.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:53 smithi204 ceph-mon[34777]: osdmap e40: 6 total, 6 up, 6 in 2024-09-13T18:16:53.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:53 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T18:16:53.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:53 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T18:16:53.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:53 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:16:54.043 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:54 smithi114 ceph-mon[30897]: from='client.14343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:16:54.043 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:54 smithi114 ceph-mon[30897]: osdmap e41: 6 total, 6 up, 6 in 2024-09-13T18:16:54.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:54 smithi134 ceph-mon[34283]: from='client.14343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:16:54.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:54 smithi134 ceph-mon[34283]: osdmap e41: 6 total, 6 up, 6 in 2024-09-13T18:16:54.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:54 smithi204 ceph-mon[34777]: from='client.14343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:16:54.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:54 smithi204 ceph-mon[34777]: osdmap e41: 6 total, 6 up, 6 in 2024-09-13T18:16:55.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:55 smithi114 ceph-mon[30897]: pgmap v135: 1 pgs: 1 peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-13T18:16:55.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:55 smithi114 ceph-mon[30897]: osdmap e42: 6 total, 6 up, 6 in 2024-09-13T18:16:55.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:55 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/188825088' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93d40a03-e7ff-4a3f-9efa-5c5403c9983d"}]: dispatch 2024-09-13T18:16:55.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:55 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/188825088' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "93d40a03-e7ff-4a3f-9efa-5c5403c9983d"}]': finished 2024-09-13T18:16:55.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:55 smithi114 ceph-mon[30897]: osdmap e43: 7 total, 6 up, 7 in 2024-09-13T18:16:55.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:55 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T18:16:55.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:55 smithi134 ceph-mon[34283]: pgmap v135: 1 pgs: 1 peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-13T18:16:55.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:55 smithi134 ceph-mon[34283]: osdmap e42: 6 total, 6 up, 6 in 2024-09-13T18:16:55.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:55 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/188825088' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93d40a03-e7ff-4a3f-9efa-5c5403c9983d"}]: dispatch 2024-09-13T18:16:55.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:55 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/188825088' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "93d40a03-e7ff-4a3f-9efa-5c5403c9983d"}]': finished 2024-09-13T18:16:55.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:55 smithi134 ceph-mon[34283]: osdmap e43: 7 total, 6 up, 7 in 2024-09-13T18:16:55.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:55 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T18:16:55.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:55 smithi204 ceph-mon[34777]: pgmap v135: 1 pgs: 1 peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-13T18:16:55.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:55 smithi204 ceph-mon[34777]: osdmap e42: 6 total, 6 up, 6 in 2024-09-13T18:16:55.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:55 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/188825088' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93d40a03-e7ff-4a3f-9efa-5c5403c9983d"}]: dispatch 2024-09-13T18:16:55.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:55 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/188825088' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "93d40a03-e7ff-4a3f-9efa-5c5403c9983d"}]': finished 2024-09-13T18:16:55.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:55 smithi204 ceph-mon[34777]: osdmap e43: 7 total, 6 up, 7 in 2024-09-13T18:16:55.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:55 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T18:16:56.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:56 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/27527132' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T18:16:56.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:56 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/27527132' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T18:16:56.469 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:56 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/27527132' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T18:16:57.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:57 smithi134 ceph-mon[34283]: pgmap v138: 1 pgs: 1 peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-13T18:16:57.371 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:57 smithi114 ceph-mon[30897]: pgmap v138: 1 pgs: 1 peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-13T18:16:57.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:57 smithi204 ceph-mon[34777]: pgmap v138: 1 pgs: 1 peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-13T18:16:59.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:16:59 smithi134 ceph-mon[34283]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-09-13T18:16:59.405 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:16:59 smithi114 ceph-mon[30897]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-09-13T18:16:59.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:16:59 smithi204 ceph-mon[34777]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-09-13T18:17:01.211 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:01 smithi114 ceph-mon[30897]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-09-13T18:17:01.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:01 smithi134 ceph-mon[34283]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-09-13T18:17:01.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:01 smithi204 ceph-mon[34777]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-09-13T18:17:02.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:02 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-13T18:17:02.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:02 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:17:02.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:02 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-13T18:17:02.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:02 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:17:02.461 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:02 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-13T18:17:02.461 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:02 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:17:03.164 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:03 smithi114 ceph-mon[30897]: Deploying daemon osd.6 on smithi114 2024-09-13T18:17:03.164 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:03 smithi114 ceph-mon[30897]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-13T18:17:03.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:03 smithi134 ceph-mon[34283]: Deploying daemon osd.6 on smithi114 2024-09-13T18:17:03.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:03 smithi134 ceph-mon[34283]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-13T18:17:03.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:03 smithi204 ceph-mon[34777]: Deploying daemon osd.6 on smithi114 2024-09-13T18:17:03.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:03 smithi204 ceph-mon[34777]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-13T18:17:05.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:05 smithi114 ceph-mon[30897]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-13T18:17:05.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:05 smithi134 ceph-mon[34283]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-13T18:17:05.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:05 smithi204 ceph-mon[34777]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-13T18:17:07.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:07 smithi134 ceph-mon[34283]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-13T18:17:07.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:07 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:17:07.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:07 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:07.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:07 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:07.411 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:07 smithi114 ceph-mon[30897]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-13T18:17:07.411 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:07 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:17:07.412 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:07 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:07.412 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:07 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:07.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:07 smithi204 ceph-mon[34777]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-13T18:17:07.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:07 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:17:07.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:07 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:07.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:07 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:09.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:09 smithi134 ceph-mon[34283]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-13T18:17:09.416 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:09 smithi114 ceph-mon[30897]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-13T18:17:09.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:09 smithi204 ceph-mon[34777]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-13T18:17:09.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:17:09.445+0000 7f266b7fe640 1 -- 172.21.15.114:0/3647097700 <== mgr.14148 v2:172.21.15.114:6800/3396185541 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f266c02c4f0 con 0x7f26540a1080 2024-09-13T18:17:09.448 INFO:teuthology.orchestra.run.smithi114.stdout:Created osd(s) 6 on host 'smithi114' 2024-09-13T18:17:09.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:17:09.448+0000 7f2674468640 1 -- 172.21.15.114:0/3647097700 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f26540a1080 msgr2=0x7f26540a3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:09.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:17:09.448+0000 7f2674468640 1 --2- 172.21.15.114:0/3647097700 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f26540a1080 0x7f26540a3540 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f265c005fd0 tx=0x7f265c005850 comp rx=0 tx=0).stop 2024-09-13T18:17:09.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:17:09.448+0000 7f2674468640 1 -- 172.21.15.114:0/3647097700 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f266c101920 msgr2=0x7f266c19d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:09.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:17:09.448+0000 7f2674468640 1 --2- 172.21.15.114:0/3647097700 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f266c101920 0x7f266c19d230 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f266400eaa0 tx=0x7f266400ef60 comp rx=0 tx=0).stop 2024-09-13T18:17:09.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:17:09.448+0000 7f2674468640 1 -- 172.21.15.114:0/3647097700 shutdown_connections 2024-09-13T18:17:09.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:17:09.448+0000 7f2674468640 1 --2- 172.21.15.114:0/3647097700 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f26540a1080 0x7f26540a3540 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:09.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:17:09.448+0000 7f2674468640 1 --2- 172.21.15.114:0/3647097700 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f266c101920 0x7f266c19d230 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:09.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:17:09.448+0000 7f2674468640 1 --2- 172.21.15.114:0/3647097700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f266c100f60 0x7f266c196210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:09.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:17:09.448+0000 7f2674468640 1 --2- 172.21.15.114:0/3647097700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f266c0ffe00 0x7f266c195cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:09.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:17:09.448+0000 7f2674468640 1 -- 172.21.15.114:0/3647097700 >> 172.21.15.114:0/3647097700 conn(0x7f266c0fb5d0 msgr2=0x7f266c104520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:17:09.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:17:09.448+0000 7f2674468640 1 -- 172.21.15.114:0/3647097700 shutdown_connections 2024-09-13T18:17:09.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:17:09.448+0000 7f2674468640 1 -- 172.21.15.114:0/3647097700 wait complete. 2024-09-13T18:17:10.141 DEBUG:teuthology.orchestra.run.smithi114:osd.6> sudo journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@osd.6.service 2024-09-13T18:17:10.144 INFO:tasks.cephadm:Deploying osd.7 on smithi134 with /dev/vg_nvme/lv_2... 2024-09-13T18:17:10.144 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-09-13T18:17:10.347 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T18:17:10.443 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 13 18:17:10 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-6[58262]: 2024-09-13T18:17:10.257+0000 7f9071ab2640 -1 osd.6 0 log_to_monitors true 2024-09-13T18:17:10.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:10 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:10.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:10 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:10.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:10 smithi204 ceph-mon[34777]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-13T18:17:10.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:10 smithi204 ceph-mon[34777]: from='osd.6 [v2:172.21.15.114:6818/1484325044,v1:172.21.15.114:6819/1484325044]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-13T18:17:10.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:10 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:10.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:10 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:10.715 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:10 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:10.715 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:10 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:10.715 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:10 smithi134 ceph-mon[34283]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-13T18:17:10.715 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:10 smithi134 ceph-mon[34283]: from='osd.6 [v2:172.21.15.114:6818/1484325044,v1:172.21.15.114:6819/1484325044]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-13T18:17:10.715 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:10 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:10.715 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:10 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:10.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:10 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:10.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:10 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:10.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:10 smithi114 ceph-mon[30897]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-13T18:17:10.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:10 smithi114 ceph-mon[30897]: from='osd.6 [v2:172.21.15.114:6818/1484325044,v1:172.21.15.114:6819/1484325044]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-13T18:17:10.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:10 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:10.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:10 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:11.622 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:11 smithi114 ceph-mon[30897]: from='osd.6 [v2:172.21.15.114:6818/1484325044,v1:172.21.15.114:6819/1484325044]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-13T18:17:11.622 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:11 smithi114 ceph-mon[30897]: osdmap e44: 7 total, 6 up, 7 in 2024-09-13T18:17:11.622 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:11 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T18:17:11.622 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:11 smithi114 ceph-mon[30897]: from='osd.6 [v2:172.21.15.114:6818/1484325044,v1:172.21.15.114:6819/1484325044]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-09-13T18:17:11.723 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:11 smithi134 ceph-mon[34283]: from='osd.6 [v2:172.21.15.114:6818/1484325044,v1:172.21.15.114:6819/1484325044]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-13T18:17:11.724 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:11 smithi134 ceph-mon[34283]: osdmap e44: 7 total, 6 up, 7 in 2024-09-13T18:17:11.724 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:11 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T18:17:11.724 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:11 smithi134 ceph-mon[34283]: from='osd.6 [v2:172.21.15.114:6818/1484325044,v1:172.21.15.114:6819/1484325044]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-09-13T18:17:11.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:11 smithi204 ceph-mon[34777]: from='osd.6 [v2:172.21.15.114:6818/1484325044,v1:172.21.15.114:6819/1484325044]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-13T18:17:11.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:11 smithi204 ceph-mon[34777]: osdmap e44: 7 total, 6 up, 7 in 2024-09-13T18:17:11.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:11 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T18:17:11.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:11 smithi204 ceph-mon[34777]: from='osd.6 [v2:172.21.15.114:6818/1484325044,v1:172.21.15.114:6819/1484325044]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-09-13T18:17:12.212 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-09-13T18:17:12.220 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 13 18:17:11 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-6[58262]: 2024-09-13T18:17:11.907+0000 7f906da33640 -1 osd.6 0 waiting for initial osdmap 2024-09-13T18:17:12.220 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 13 18:17:11 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-6[58262]: 2024-09-13T18:17:11.917+0000 7f9065854640 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T18:17:12.298 INFO:teuthology.orchestra.run.smithi134.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-09-13T18:17:12.311 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10+0 records in 2024-09-13T18:17:12.312 INFO:teuthology.orchestra.run.smithi134.stderr:10+0 records out 2024-09-13T18:17:12.312 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113742 s, 922 MB/s 2024-09-13T18:17:12.312 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping successful for: 2024-09-13T18:17:12.596 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:12 smithi134 ceph-mon[34283]: from='osd.6 [v2:172.21.15.114:6818/1484325044,v1:172.21.15.114:6819/1484325044]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-09-13T18:17:12.596 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:12 smithi134 ceph-mon[34283]: osdmap e45: 7 total, 6 up, 7 in 2024-09-13T18:17:12.596 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:12 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T18:17:12.596 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:12 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T18:17:12.596 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:12 smithi134 ceph-mon[34283]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-13T18:17:12.596 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:12 smithi134 ceph-mon[34283]: from='osd.6 [v2:172.21.15.114:6818/1484325044,v1:172.21.15.114:6819/1484325044]' entity='osd.6' 2024-09-13T18:17:12.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:12 smithi114 ceph-mon[30897]: from='osd.6 [v2:172.21.15.114:6818/1484325044,v1:172.21.15.114:6819/1484325044]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-09-13T18:17:12.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:12 smithi114 ceph-mon[30897]: osdmap e45: 7 total, 6 up, 7 in 2024-09-13T18:17:12.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:12 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T18:17:12.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:12 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T18:17:12.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:12 smithi114 ceph-mon[30897]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-13T18:17:12.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:12 smithi114 ceph-mon[30897]: from='osd.6 [v2:172.21.15.114:6818/1484325044,v1:172.21.15.114:6819/1484325044]' entity='osd.6' 2024-09-13T18:17:12.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:12 smithi204 ceph-mon[34777]: from='osd.6 [v2:172.21.15.114:6818/1484325044,v1:172.21.15.114:6819/1484325044]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-09-13T18:17:12.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:12 smithi204 ceph-mon[34777]: osdmap e45: 7 total, 6 up, 7 in 2024-09-13T18:17:12.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:12 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T18:17:12.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:12 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T18:17:12.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:12 smithi204 ceph-mon[34777]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-13T18:17:12.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:12 smithi204 ceph-mon[34777]: from='osd.6 [v2:172.21.15.114:6818/1484325044,v1:172.21.15.114:6819/1484325044]' entity='osd.6' 2024-09-13T18:17:12.988 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi134:vg_nvme/lv_2 2024-09-13T18:17:13.199 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T18:17:13.577 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:13 smithi134 ceph-mon[34283]: purged_snaps scrub starts 2024-09-13T18:17:13.577 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:13 smithi134 ceph-mon[34283]: purged_snaps scrub ok 2024-09-13T18:17:13.577 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:13 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T18:17:13.577 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:13 smithi134 ceph-mon[34283]: osd.6 [v2:172.21.15.114:6818/1484325044,v1:172.21.15.114:6819/1484325044] boot 2024-09-13T18:17:13.578 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:13 smithi134 ceph-mon[34283]: osdmap e46: 7 total, 7 up, 7 in 2024-09-13T18:17:13.578 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:13 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T18:17:13.578 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:13 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:13.578 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:13 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:13.578 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:13 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:17:13.578 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:13 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:17:13.578 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:13 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:17:13.578 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:13 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:13.578 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:13 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:17:13.578 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:13 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:17:13.578 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:13 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:13.578 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:13 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:17:13.578 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:13 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:17:13.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:13 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:17:13.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:13 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:13.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:13 smithi114 ceph-mon[30897]: purged_snaps scrub starts 2024-09-13T18:17:13.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:13 smithi114 ceph-mon[30897]: purged_snaps scrub ok 2024-09-13T18:17:13.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:13 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T18:17:13.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:13 smithi114 ceph-mon[30897]: osd.6 [v2:172.21.15.114:6818/1484325044,v1:172.21.15.114:6819/1484325044] boot 2024-09-13T18:17:13.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:13 smithi114 ceph-mon[30897]: osdmap e46: 7 total, 7 up, 7 in 2024-09-13T18:17:13.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:13 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T18:17:13.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:13 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:13.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:13 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:13.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:13 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:17:13.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:13 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:17:13.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:13 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:17:13.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:13 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:13.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:13 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:17:13.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:13 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:17:13.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:13 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:13.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:13 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:17:13.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:13 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:17:13.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:13 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:17:13.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:13 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:13.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:13 smithi204 ceph-mon[34777]: purged_snaps scrub starts 2024-09-13T18:17:13.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:13 smithi204 ceph-mon[34777]: purged_snaps scrub ok 2024-09-13T18:17:13.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:13 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T18:17:13.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:13 smithi204 ceph-mon[34777]: osd.6 [v2:172.21.15.114:6818/1484325044,v1:172.21.15.114:6819/1484325044] boot 2024-09-13T18:17:13.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:13 smithi204 ceph-mon[34777]: osdmap e46: 7 total, 7 up, 7 in 2024-09-13T18:17:13.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:13 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T18:17:13.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:13 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:13.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:13 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:13.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:13 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:17:13.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:13 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:17:13.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:13 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:17:13.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:13 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:13.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:13 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:17:13.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:13 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:17:13.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:13 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:13.946 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:13 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:17:13.946 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:13 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:17:13.946 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:13 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:17:13.946 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:13 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:14.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:14 smithi134 ceph-mon[34283]: Detected new or changed devices on smithi114 2024-09-13T18:17:14.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:14 smithi134 ceph-mon[34283]: Adjusting osd_memory_target on smithi114 to 5696M 2024-09-13T18:17:14.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:14 smithi134 ceph-mon[34283]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-13T18:17:14.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:14 smithi134 ceph-mon[34283]: osdmap e47: 7 total, 7 up, 7 in 2024-09-13T18:17:14.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:14 smithi114 ceph-mon[30897]: Detected new or changed devices on smithi114 2024-09-13T18:17:14.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:14 smithi114 ceph-mon[30897]: Adjusting osd_memory_target on smithi114 to 5696M 2024-09-13T18:17:14.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:14 smithi114 ceph-mon[30897]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-13T18:17:14.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:14 smithi114 ceph-mon[30897]: osdmap e47: 7 total, 7 up, 7 in 2024-09-13T18:17:14.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:17:14.907+0000 7f4a0ee96640 1 -- 172.21.15.134:0/236929651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a08105b50 msgr2=0x7f4a08107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:14.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:17:14.907+0000 7f4a0ee96640 1 --2- 172.21.15.134:0/236929651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a08105b50 0x7f4a08107f40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4a00009900 tx=0x7f4a0002a410 comp rx=0 tx=0).stop 2024-09-13T18:17:14.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:17:14.907+0000 7f4a0ee96640 1 -- 172.21.15.134:0/236929651 shutdown_connections 2024-09-13T18:17:14.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:17:14.907+0000 7f4a0ee96640 1 --2- 172.21.15.134:0/236929651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a08105b50 0x7f4a08107f40 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:14.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:17:14.907+0000 7f4a0ee96640 1 --2- 172.21.15.134:0/236929651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a08103220 0x7f4a08105610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:14.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:17:14.907+0000 7f4a0ee96640 1 --2- 172.21.15.134:0/236929651 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4a081008f0 0x7f4a08102ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:14.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:17:14.907+0000 7f4a0ee96640 1 -- 172.21.15.134:0/236929651 >> 172.21.15.134:0/236929651 conn(0x7f4a080fa490 msgr2=0x7f4a080fc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:17:14.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:17:14.907+0000 7f4a0ee96640 1 -- 172.21.15.134:0/236929651 shutdown_connections 2024-09-13T18:17:14.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:17:14.907+0000 7f4a0ee96640 1 -- 172.21.15.134:0/236929651 wait complete. 2024-09-13T18:17:14.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:17:14.907+0000 7f4a0ee96640 1 Processor -- start 2024-09-13T18:17:14.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:17:14.907+0000 7f4a0ee96640 1 -- start start 2024-09-13T18:17:14.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:17:14.907+0000 7f4a0ee96640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4a081008f0 0x7f4a08195b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:14.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:17:14.908+0000 7f4a0ee96640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a08103220 0x7f4a081960d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:14.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:17:14.908+0000 7f4a0ee96640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a08105b50 0x7f4a0819d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:14.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:17:14.908+0000 7f4a0ee96640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0819d690 con 0x7f4a081008f0 2024-09-13T18:17:14.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:17:14.908+0000 7f4a0ee96640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0819d800 con 0x7f4a08105b50 2024-09-13T18:17:14.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:17:14.908+0000 7f4a0ee96640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0819d9c0 con 0x7f4a08103220 2024-09-13T18:17:14.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:17:14.908+0000 7f4a0d433640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a08105b50 0x7f4a0819d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:14.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:17:14.908+0000 7f4a0d433640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a08105b50 0x7f4a0819d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:48106/0 (socket says 172.21.15.134:48106) 2024-09-13T18:17:14.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:17:14.908+0000 7f4a0d433640 1 -- 172.21.15.134:0/3180345710 learned_addr learned my addr 172.21.15.134:0/3180345710 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-13T18:17:14.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:17:14.908+0000 7f4a07fff640 1 --2- 172.21.15.134:0/3180345710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a08103220 0x7f4a081960d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:14.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:17:14.908+0000 7f4a0cc32640 1 --2- 172.21.15.134:0/3180345710 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4a081008f0 0x7f4a08195b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:14.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:17:14.908+0000 7f4a0d433640 1 -- 172.21.15.134:0/3180345710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a08103220 msgr2=0x7f4a081960d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:14.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:17:14.908+0000 7f4a0d433640 1 --2- 172.21.15.134:0/3180345710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a08103220 0x7f4a081960d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:14.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:17:14.908+0000 7f4a0d433640 1 -- 172.21.15.134:0/3180345710 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4a081008f0 msgr2=0x7f4a08195b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:14.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:17:14.908+0000 7f4a0d433640 1 --2- 172.21.15.134:0/3180345710 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4a081008f0 0x7f4a08195b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:14.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:17:14.908+0000 7f4a0d433640 1 -- 172.21.15.134:0/3180345710 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a00009560 con 0x7f4a08105b50 2024-09-13T18:17:14.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:17:14.908+0000 7f4a0d433640 1 --2- 172.21.15.134:0/3180345710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a08105b50 0x7f4a0819d150 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4a0002a930 tx=0x7f4a000048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:17:14.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:17:14.908+0000 7f4a05ffb640 1 -- 172.21.15.134:0/3180345710 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a000035f0 con 0x7f4a08105b50 2024-09-13T18:17:14.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:17:14.908+0000 7f4a05ffb640 1 -- 172.21.15.134:0/3180345710 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4a00003be0 con 0x7f4a08105b50 2024-09-13T18:17:14.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:17:14.908+0000 7f4a0ee96640 1 -- 172.21.15.134:0/3180345710 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a0819dc40 con 0x7f4a08105b50 2024-09-13T18:17:14.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:17:14.908+0000 7f4a05ffb640 1 -- 172.21.15.134:0/3180345710 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0003ca80 con 0x7f4a08105b50 2024-09-13T18:17:14.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:17:14.909+0000 7f4a0ee96640 1 -- 172.21.15.134:0/3180345710 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a0819e130 con 0x7f4a08105b50 2024-09-13T18:17:14.915 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:17:14.910+0000 7f4a0ee96640 1 -- 172.21.15.134:0/3180345710 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a080fbf80 con 0x7f4a08105b50 2024-09-13T18:17:14.915 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:17:14.911+0000 7f4a05ffb640 1 -- 172.21.15.134:0/3180345710 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f4a00003750 con 0x7f4a08105b50 2024-09-13T18:17:14.915 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:17:14.911+0000 7f4a05ffb640 1 --2- 172.21.15.134:0/3180345710 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f49f40a1130 0x7f49f40a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:14.915 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:17:14.911+0000 7f4a05ffb640 1 -- 172.21.15.134:0/3180345710 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4965+0+0 (secure 0 0 0) 0x7f4a000ecbf0 con 0x7f4a08105b50 2024-09-13T18:17:14.915 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:17:14.912+0000 7f4a0cc32640 1 --2- 172.21.15.134:0/3180345710 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f49f40a1130 0x7f49f40a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:14.915 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:17:14.912+0000 7f4a0cc32640 1 --2- 172.21.15.134:0/3180345710 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f49f40a1130 0x7f49f40a35f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f49f8007ec0 tx=0x7f49f80075b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:17:14.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:17:14.916+0000 7f4a05ffb640 1 -- 172.21.15.134:0/3180345710 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f4a000b7090 con 0x7f4a08105b50 2024-09-13T18:17:14.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:14 smithi204 ceph-mon[34777]: Detected new or changed devices on smithi114 2024-09-13T18:17:14.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:14 smithi204 ceph-mon[34777]: Adjusting osd_memory_target on smithi114 to 5696M 2024-09-13T18:17:14.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:14 smithi204 ceph-mon[34777]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-13T18:17:14.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:14 smithi204 ceph-mon[34777]: osdmap e47: 7 total, 7 up, 7 in 2024-09-13T18:17:15.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:17:15.087+0000 7f4a0ee96640 1 -- 172.21.15.134:0/3180345710 --> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a0802c450 con 0x7f49f40a1130 2024-09-13T18:17:15.704 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:15 smithi134 ceph-mon[34283]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:17:15.704 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:15 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T18:17:15.704 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:15 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T18:17:15.705 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:15 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:17:15.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:15 smithi114 ceph-mon[30897]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:17:15.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:15 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T18:17:15.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:15 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T18:17:15.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:15 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:17:15.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:15 smithi204 ceph-mon[34777]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:17:15.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:15 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T18:17:15.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:15 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T18:17:15.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:15 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:17:16.803 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:16 smithi134 ceph-mon[34283]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-13T18:17:16.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:16 smithi204 ceph-mon[34777]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-13T18:17:16.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:16 smithi114 ceph-mon[30897]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-13T18:17:17.800 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:17 smithi134 ceph-mon[34283]: from='client.? 172.21.15.134:0/8192266' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "982f629a-1b95-4be1-8e75-5ddce1ad77b4"}]: dispatch 2024-09-13T18:17:17.800 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:17 smithi134 ceph-mon[34283]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "982f629a-1b95-4be1-8e75-5ddce1ad77b4"}]: dispatch 2024-09-13T18:17:17.800 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:17 smithi134 ceph-mon[34283]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "982f629a-1b95-4be1-8e75-5ddce1ad77b4"}]': finished 2024-09-13T18:17:17.800 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:17 smithi134 ceph-mon[34283]: osdmap e48: 8 total, 7 up, 8 in 2024-09-13T18:17:17.800 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:17 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-13T18:17:17.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:17 smithi204 ceph-mon[34777]: from='client.? 172.21.15.134:0/8192266' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "982f629a-1b95-4be1-8e75-5ddce1ad77b4"}]: dispatch 2024-09-13T18:17:17.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:17 smithi204 ceph-mon[34777]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "982f629a-1b95-4be1-8e75-5ddce1ad77b4"}]: dispatch 2024-09-13T18:17:17.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:17 smithi204 ceph-mon[34777]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "982f629a-1b95-4be1-8e75-5ddce1ad77b4"}]': finished 2024-09-13T18:17:17.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:17 smithi204 ceph-mon[34777]: osdmap e48: 8 total, 7 up, 8 in 2024-09-13T18:17:17.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:17 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-13T18:17:17.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:17 smithi114 ceph-mon[30897]: from='client.? 172.21.15.134:0/8192266' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "982f629a-1b95-4be1-8e75-5ddce1ad77b4"}]: dispatch 2024-09-13T18:17:17.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:17 smithi114 ceph-mon[30897]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "982f629a-1b95-4be1-8e75-5ddce1ad77b4"}]: dispatch 2024-09-13T18:17:17.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:17 smithi114 ceph-mon[30897]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "982f629a-1b95-4be1-8e75-5ddce1ad77b4"}]': finished 2024-09-13T18:17:17.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:17 smithi114 ceph-mon[30897]: osdmap e48: 8 total, 7 up, 8 in 2024-09-13T18:17:17.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:17 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-13T18:17:18.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:18 smithi134 ceph-mon[34283]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-13T18:17:18.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:18 smithi134 ceph-mon[34283]: from='client.? 172.21.15.134:0/3948425721' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T18:17:18.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:18 smithi204 ceph-mon[34777]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-13T18:17:18.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:18 smithi204 ceph-mon[34777]: from='client.? 172.21.15.134:0/3948425721' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T18:17:18.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:18 smithi114 ceph-mon[30897]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-13T18:17:18.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:18 smithi114 ceph-mon[30897]: from='client.? 172.21.15.134:0/3948425721' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T18:17:21.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:20 smithi204 ceph-mon[34777]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-13T18:17:21.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:20 smithi114 ceph-mon[30897]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-13T18:17:21.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:20 smithi134 ceph-mon[34283]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-13T18:17:23.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:22 smithi134 ceph-mon[34283]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-13T18:17:23.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:22 smithi204 ceph-mon[34777]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-13T18:17:23.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:22 smithi114 ceph-mon[30897]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-13T18:17:24.942 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:24 smithi134 ceph-mon[34283]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-13T18:17:24.942 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:24 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-13T18:17:24.942 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:24 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:17:25.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:24 smithi204 ceph-mon[34777]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-13T18:17:25.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:24 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-13T18:17:25.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:24 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:17:25.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:24 smithi114 ceph-mon[30897]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-13T18:17:25.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:24 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-13T18:17:25.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:24 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:17:26.086 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:25 smithi134 ceph-mon[34283]: Deploying daemon osd.7 on smithi134 2024-09-13T18:17:26.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:25 smithi204 ceph-mon[34777]: Deploying daemon osd.7 on smithi134 2024-09-13T18:17:26.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:25 smithi114 ceph-mon[30897]: Deploying daemon osd.7 on smithi134 2024-09-13T18:17:27.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:26 smithi134 ceph-mon[34283]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-13T18:17:27.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:26 smithi204 ceph-mon[34777]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-13T18:17:27.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:26 smithi114 ceph-mon[30897]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-13T18:17:29.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:28 smithi134 ceph-mon[34283]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-13T18:17:29.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:28 smithi204 ceph-mon[34777]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-13T18:17:29.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:28 smithi114 ceph-mon[30897]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-13T18:17:30.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:29 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:17:30.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:29 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:30.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:29 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:30.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:29 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:17:30.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:29 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:30.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:29 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:30.237 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:29 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:17:30.237 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:29 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:30.237 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:29 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:31.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:30 smithi204 ceph-mon[34777]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-13T18:17:31.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:30 smithi114 ceph-mon[30897]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-13T18:17:31.280 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:30 smithi134 ceph-mon[34283]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-13T18:17:33.105 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:32 smithi134 ceph-mon[34283]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-13T18:17:33.106 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:32 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:33.106 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:32 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:33.106 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:32 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:17:33.106 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:32 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:17:33.106 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:32 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:33.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:17:33.135+0000 7f4a05ffb640 1 -- 172.21.15.134:0/3180345710 <== mgr.14148 v2:172.21.15.114:6800/3396185541 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f4a0802c450 con 0x7f49f40a1130 2024-09-13T18:17:33.139 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 7 on host 'smithi134' 2024-09-13T18:17:33.141 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:17:33.138+0000 7f4a0ee96640 1 -- 172.21.15.134:0/3180345710 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f49f40a1130 msgr2=0x7f49f40a35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:33.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:17:33.138+0000 7f4a0ee96640 1 --2- 172.21.15.134:0/3180345710 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f49f40a1130 0x7f49f40a35f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f49f8007ec0 tx=0x7f49f80075b0 comp rx=0 tx=0).stop 2024-09-13T18:17:33.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:17:33.138+0000 7f4a0ee96640 1 -- 172.21.15.134:0/3180345710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a08105b50 msgr2=0x7f4a0819d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:33.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:17:33.138+0000 7f4a0ee96640 1 --2- 172.21.15.134:0/3180345710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a08105b50 0x7f4a0819d150 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4a0002a930 tx=0x7f4a000048f0 comp rx=0 tx=0).stop 2024-09-13T18:17:33.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:17:33.138+0000 7f4a0ee96640 1 -- 172.21.15.134:0/3180345710 shutdown_connections 2024-09-13T18:17:33.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:17:33.138+0000 7f4a0ee96640 1 --2- 172.21.15.134:0/3180345710 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f49f40a1130 0x7f49f40a35f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:33.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:17:33.138+0000 7f4a0ee96640 1 --2- 172.21.15.134:0/3180345710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a08105b50 0x7f4a0819d150 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:33.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:17:33.138+0000 7f4a0ee96640 1 --2- 172.21.15.134:0/3180345710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a08103220 0x7f4a081960d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:33.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:17:33.138+0000 7f4a0ee96640 1 --2- 172.21.15.134:0/3180345710 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4a081008f0 0x7f4a08195b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:33.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:17:33.138+0000 7f4a0ee96640 1 -- 172.21.15.134:0/3180345710 >> 172.21.15.134:0/3180345710 conn(0x7f4a080fa490 msgr2=0x7f4a08101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:17:33.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:17:33.138+0000 7f4a0ee96640 1 -- 172.21.15.134:0/3180345710 shutdown_connections 2024-09-13T18:17:33.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:17:33.138+0000 7f4a0ee96640 1 -- 172.21.15.134:0/3180345710 wait complete. 2024-09-13T18:17:33.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:32 smithi204 ceph-mon[34777]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-13T18:17:33.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:32 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:33.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:32 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:33.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:32 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:17:33.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:32 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:17:33.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:32 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:33.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:32 smithi114 ceph-mon[30897]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-13T18:17:33.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:32 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:33.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:32 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:33.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:32 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:17:33.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:32 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:17:33.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:32 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:33.799 DEBUG:teuthology.orchestra.run.smithi134:osd.7> sudo journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@osd.7.service 2024-09-13T18:17:33.801 INFO:tasks.cephadm:Deploying osd.8 on smithi204 with /dev/vg_nvme/lv_2... 2024-09-13T18:17:33.801 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-09-13T18:17:34.011 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T18:17:34.088 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:33 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:17:34.088 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:33 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:34.089 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:33 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:34.089 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:33 smithi134 ceph-mon[34283]: from='osd.7 [v2:172.21.15.134:6816/799246614,v1:172.21.15.134:6817/799246614]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-13T18:17:34.089 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:33 smithi134 ceph-mon[34283]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-13T18:17:34.132 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:33 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:17:34.132 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:33 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:34.132 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:33 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:34.133 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:33 smithi204 ceph-mon[34777]: from='osd.7 [v2:172.21.15.134:6816/799246614,v1:172.21.15.134:6817/799246614]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-13T18:17:34.133 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:33 smithi204 ceph-mon[34777]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-13T18:17:34.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:33 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:17:34.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:33 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:34.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:33 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:34.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:33 smithi114 ceph-mon[30897]: from='osd.7 [v2:172.21.15.134:6816/799246614,v1:172.21.15.134:6817/799246614]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-13T18:17:34.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:33 smithi114 ceph-mon[30897]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-13T18:17:35.257 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:35 smithi134 ceph-mon[34283]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-13T18:17:35.257 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:35 smithi134 ceph-mon[34283]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-13T18:17:35.257 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:35 smithi134 ceph-mon[34283]: osdmap e49: 8 total, 7 up, 8 in 2024-09-13T18:17:35.257 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:35 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-13T18:17:35.257 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:35 smithi134 ceph-mon[34283]: from='osd.7 [v2:172.21.15.134:6816/799246614,v1:172.21.15.134:6817/799246614]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-13T18:17:35.257 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:35 smithi134 ceph-mon[34283]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-13T18:17:35.411 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:35 smithi204 ceph-mon[34777]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-13T18:17:35.411 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:35 smithi204 ceph-mon[34777]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-13T18:17:35.411 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:35 smithi204 ceph-mon[34777]: osdmap e49: 8 total, 7 up, 8 in 2024-09-13T18:17:35.411 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:35 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-13T18:17:35.411 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:35 smithi204 ceph-mon[34777]: from='osd.7 [v2:172.21.15.134:6816/799246614,v1:172.21.15.134:6817/799246614]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-13T18:17:35.411 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:35 smithi204 ceph-mon[34777]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-13T18:17:35.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:35 smithi114 ceph-mon[30897]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-13T18:17:35.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:35 smithi114 ceph-mon[30897]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-13T18:17:35.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:35 smithi114 ceph-mon[30897]: osdmap e49: 8 total, 7 up, 8 in 2024-09-13T18:17:35.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:35 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-13T18:17:35.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:35 smithi114 ceph-mon[30897]: from='osd.7 [v2:172.21.15.134:6816/799246614,v1:172.21.15.134:6817/799246614]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-13T18:17:35.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:35 smithi114 ceph-mon[30897]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-13T18:17:35.873 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:17:35 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:17:35.550+0000 7f789cfd0640 -1 osd.7 0 waiting for initial osdmap 2024-09-13T18:17:35.873 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:17:35 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:17:35.559+0000 7f78945da640 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T18:17:35.929 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-09-13T18:17:36.024 INFO:teuthology.orchestra.run.smithi204.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-09-13T18:17:36.037 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10+0 records in 2024-09-13T18:17:36.037 INFO:teuthology.orchestra.run.smithi204.stderr:10+0 records out 2024-09-13T18:17:36.037 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112181 s, 935 MB/s 2024-09-13T18:17:36.037 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping successful for: 2024-09-13T18:17:36.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:36 smithi204 ceph-mon[34777]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-13T18:17:36.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:36 smithi204 ceph-mon[34777]: osdmap e50: 8 total, 7 up, 8 in 2024-09-13T18:17:36.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:36 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-13T18:17:36.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:36 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-13T18:17:36.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:36 smithi204 ceph-mon[34777]: from='osd.7 ' entity='osd.7' 2024-09-13T18:17:36.365 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:36 smithi134 ceph-mon[34283]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-13T18:17:36.365 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:36 smithi134 ceph-mon[34283]: osdmap e50: 8 total, 7 up, 8 in 2024-09-13T18:17:36.365 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:36 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-13T18:17:36.365 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:36 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-13T18:17:36.365 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:36 smithi134 ceph-mon[34283]: from='osd.7 ' entity='osd.7' 2024-09-13T18:17:36.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:36 smithi114 ceph-mon[30897]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-13T18:17:36.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:36 smithi114 ceph-mon[30897]: osdmap e50: 8 total, 7 up, 8 in 2024-09-13T18:17:36.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:36 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-13T18:17:36.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:36 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-13T18:17:36.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:36 smithi114 ceph-mon[30897]: from='osd.7 ' entity='osd.7' 2024-09-13T18:17:36.783 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi204:vg_nvme/lv_2 2024-09-13T18:17:36.992 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T18:17:37.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:37 smithi204 ceph-mon[34777]: purged_snaps scrub starts 2024-09-13T18:17:37.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:37 smithi204 ceph-mon[34777]: purged_snaps scrub ok 2024-09-13T18:17:37.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:37 smithi204 ceph-mon[34777]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-13T18:17:37.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:37 smithi204 ceph-mon[34777]: osd.7 [v2:172.21.15.134:6816/799246614,v1:172.21.15.134:6817/799246614] boot 2024-09-13T18:17:37.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:37 smithi204 ceph-mon[34777]: osdmap e51: 8 total, 8 up, 8 in 2024-09-13T18:17:37.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:37 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-13T18:17:37.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:37 smithi204 ceph-mon[34777]: Detected new or changed devices on smithi134 2024-09-13T18:17:37.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:37 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:37.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:37 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:37.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:37 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:17:37.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:37 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:17:37.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:37 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:17:37.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:37 smithi204 ceph-mon[34777]: Adjusting osd_memory_target on smithi134 to 5696M 2024-09-13T18:17:37.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:37 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:37.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:37 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:17:37.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:37 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:17:37.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:37 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:37.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:37 smithi114 ceph-mon[30897]: purged_snaps scrub starts 2024-09-13T18:17:37.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:37 smithi114 ceph-mon[30897]: purged_snaps scrub ok 2024-09-13T18:17:37.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:37 smithi114 ceph-mon[30897]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-13T18:17:37.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:37 smithi114 ceph-mon[30897]: osd.7 [v2:172.21.15.134:6816/799246614,v1:172.21.15.134:6817/799246614] boot 2024-09-13T18:17:37.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:37 smithi114 ceph-mon[30897]: osdmap e51: 8 total, 8 up, 8 in 2024-09-13T18:17:37.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:37 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-13T18:17:37.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:37 smithi114 ceph-mon[30897]: Detected new or changed devices on smithi134 2024-09-13T18:17:37.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:37 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:37.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:37 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:37.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:37 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:17:37.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:37 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:17:37.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:37 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:17:37.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:37 smithi114 ceph-mon[30897]: Adjusting osd_memory_target on smithi134 to 5696M 2024-09-13T18:17:37.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:37 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:37.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:37 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:17:37.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:37 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:17:37.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:37 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:37.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:37 smithi134 ceph-mon[34283]: purged_snaps scrub starts 2024-09-13T18:17:37.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:37 smithi134 ceph-mon[34283]: purged_snaps scrub ok 2024-09-13T18:17:37.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:37 smithi134 ceph-mon[34283]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-13T18:17:37.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:37 smithi134 ceph-mon[34283]: osd.7 [v2:172.21.15.134:6816/799246614,v1:172.21.15.134:6817/799246614] boot 2024-09-13T18:17:37.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:37 smithi134 ceph-mon[34283]: osdmap e51: 8 total, 8 up, 8 in 2024-09-13T18:17:37.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:37 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-13T18:17:37.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:37 smithi134 ceph-mon[34283]: Detected new or changed devices on smithi134 2024-09-13T18:17:37.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:37 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:37.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:37 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:37.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:37 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:17:37.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:37 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:17:37.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:37 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:17:37.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:37 smithi134 ceph-mon[34283]: Adjusting osd_memory_target on smithi134 to 5696M 2024-09-13T18:17:37.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:37 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:37.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:37 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:17:37.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:37 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:17:37.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:37 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:38.366 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:38 smithi204 ceph-mon[34777]: osdmap e52: 8 total, 8 up, 8 in 2024-09-13T18:17:38.366 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:38 smithi204 ceph-mon[34777]: pgmap v168: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-13T18:17:38.422 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:38 smithi114 ceph-mon[30897]: osdmap e52: 8 total, 8 up, 8 in 2024-09-13T18:17:38.422 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:38 smithi114 ceph-mon[30897]: pgmap v168: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-13T18:17:38.596 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:17:38.574+0000 7f121c7b3640 1 -- 172.21.15.204:0/3184547507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1214101920 msgr2=0x7f1214108240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:38.596 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:17:38.574+0000 7f121c7b3640 1 --2- 172.21.15.204:0/3184547507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1214101920 0x7f1214108240 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f120c009ca0 tx=0x7f120c02a8c0 comp rx=0 tx=0).stop 2024-09-13T18:17:38.596 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:17:38.574+0000 7f121c7b3640 1 -- 172.21.15.204:0/3184547507 shutdown_connections 2024-09-13T18:17:38.596 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:17:38.574+0000 7f121c7b3640 1 --2- 172.21.15.204:0/3184547507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1214101920 0x7f1214108240 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:38.596 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:17:38.574+0000 7f121c7b3640 1 --2- 172.21.15.204:0/3184547507 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1214100f60 0x7f12141013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:38.596 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:17:38.574+0000 7f121c7b3640 1 --2- 172.21.15.204:0/3184547507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12140ffe00 0x7f1214100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:38.597 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:17:38.574+0000 7f121c7b3640 1 -- 172.21.15.204:0/3184547507 >> 172.21.15.204:0/3184547507 conn(0x7f12140fb5d0 msgr2=0x7f12140fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:17:38.597 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:17:38.574+0000 7f121c7b3640 1 -- 172.21.15.204:0/3184547507 shutdown_connections 2024-09-13T18:17:38.597 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:17:38.574+0000 7f121c7b3640 1 -- 172.21.15.204:0/3184547507 wait complete. 2024-09-13T18:17:38.597 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:17:38.575+0000 7f121c7b3640 1 Processor -- start 2024-09-13T18:17:38.597 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:17:38.575+0000 7f121c7b3640 1 -- start start 2024-09-13T18:17:38.597 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:17:38.575+0000 7f121c7b3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12140ffe00 0x7f1214195c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:38.597 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:17:38.575+0000 7f121c7b3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1214100f60 0x7f1214196170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:38.597 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:17:38.575+0000 7f121c7b3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1214101920 0x7f121419d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:38.597 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:17:38.575+0000 7f121c7b3640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f121419d720 con 0x7f12140ffe00 2024-09-13T18:17:38.597 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:17:38.575+0000 7f121c7b3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f121419d890 con 0x7f1214101920 2024-09-13T18:17:38.597 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:17:38.575+0000 7f121c7b3640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f121419da50 con 0x7f1214100f60 2024-09-13T18:17:38.597 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:17:38.575+0000 7f1219d4e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1214100f60 0x7f1214196170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:38.598 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:17:38.575+0000 7f1219d4e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1214100f60 0x7f1214196170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:40576/0 (socket says 172.21.15.204:40576) 2024-09-13T18:17:38.598 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:17:38.575+0000 7f1219d4e640 1 -- 172.21.15.204:0/1060106006 learned_addr learned my addr 172.21.15.204:0/1060106006 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-13T18:17:38.598 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:17:38.575+0000 7f121a54f640 1 --2- 172.21.15.204:0/1060106006 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12140ffe00 0x7f1214195c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:38.598 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:17:38.576+0000 7f121ad50640 1 --2- 172.21.15.204:0/1060106006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1214101920 0x7f121419d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:38.598 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:17:38.576+0000 7f1219d4e640 1 -- 172.21.15.204:0/1060106006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1214101920 msgr2=0x7f121419d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:38.598 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:17:38.576+0000 7f1219d4e640 1 --2- 172.21.15.204:0/1060106006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1214101920 0x7f121419d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:38.598 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:17:38.576+0000 7f1219d4e640 1 -- 172.21.15.204:0/1060106006 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12140ffe00 msgr2=0x7f1214195c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:38.598 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:17:38.576+0000 7f1219d4e640 1 --2- 172.21.15.204:0/1060106006 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12140ffe00 0x7f1214195c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:38.598 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:17:38.576+0000 7f1219d4e640 1 -- 172.21.15.204:0/1060106006 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f120c009900 con 0x7f1214100f60 2024-09-13T18:17:38.598 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:17:38.576+0000 7f1219d4e640 1 --2- 172.21.15.204:0/1060106006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1214100f60 0x7f1214196170 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f120400d770 tx=0x7f120400dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:17:38.598 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:17:38.576+0000 7f120b7fe640 1 -- 172.21.15.204:0/1060106006 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1204009c30 con 0x7f1214100f60 2024-09-13T18:17:38.598 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:17:38.576+0000 7f120b7fe640 1 -- 172.21.15.204:0/1060106006 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f120400de80 con 0x7f1214100f60 2024-09-13T18:17:38.598 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:17:38.576+0000 7f121c7b3640 1 -- 172.21.15.204:0/1060106006 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f121419dd30 con 0x7f1214100f60 2024-09-13T18:17:38.598 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:17:38.576+0000 7f120b7fe640 1 -- 172.21.15.204:0/1060106006 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1204003d40 con 0x7f1214100f60 2024-09-13T18:17:38.599 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:17:38.576+0000 7f121c7b3640 1 -- 172.21.15.204:0/1060106006 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f121419e180 con 0x7f1214100f60 2024-09-13T18:17:38.599 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:17:38.577+0000 7f121c7b3640 1 -- 172.21.15.204:0/1060106006 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1214107290 con 0x7f1214100f60 2024-09-13T18:17:38.599 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:17:38.578+0000 7f120b7fe640 1 -- 172.21.15.204:0/1060106006 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f1204003ea0 con 0x7f1214100f60 2024-09-13T18:17:38.599 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:17:38.579+0000 7f120b7fe640 1 --2- 172.21.15.204:0/1060106006 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f12000a1080 0x7f12000a3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:38.599 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:17:38.579+0000 7f120b7fe640 1 -- 172.21.15.204:0/1060106006 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5426+0+0 (secure 0 0 0) 0x7f12040cbb50 con 0x7f1214100f60 2024-09-13T18:17:38.599 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:17:38.579+0000 7f121a54f640 1 --2- 172.21.15.204:0/1060106006 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f12000a1080 0x7f12000a3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:38.599 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:17:38.579+0000 7f121a54f640 1 --2- 172.21.15.204:0/1060106006 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f12000a1080 0x7f12000a3540 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1214197110 tx=0x7f121000b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:17:38.599 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:17:38.582+0000 7f120b7fe640 1 -- 172.21.15.204:0/1060106006 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f1204095e20 con 0x7f1214100f60 2024-09-13T18:17:38.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:38 smithi134 ceph-mon[34283]: osdmap e52: 8 total, 8 up, 8 in 2024-09-13T18:17:38.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:38 smithi134 ceph-mon[34283]: pgmap v168: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-13T18:17:38.757 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:17:38.754+0000 7f121c7b3640 1 -- 172.21.15.204:0/1060106006 --> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f121402c4f0 con 0x7f12000a1080 2024-09-13T18:17:39.382 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:39 smithi204 ceph-mon[34777]: osdmap e53: 8 total, 8 up, 8 in 2024-09-13T18:17:39.383 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:39 smithi204 ceph-mon[34777]: from='client.24286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:17:39.383 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:39 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T18:17:39.383 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:39 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T18:17:39.383 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:39 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:17:39.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:39 smithi114 ceph-mon[30897]: osdmap e53: 8 total, 8 up, 8 in 2024-09-13T18:17:39.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:39 smithi114 ceph-mon[30897]: from='client.24286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:17:39.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:39 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T18:17:39.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:39 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T18:17:39.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:39 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:17:39.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:39 smithi134 ceph-mon[34283]: osdmap e53: 8 total, 8 up, 8 in 2024-09-13T18:17:39.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:39 smithi134 ceph-mon[34283]: from='client.24286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:17:39.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:39 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T18:17:39.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:39 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T18:17:39.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:39 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:17:40.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:40 smithi204 ceph-mon[34777]: osdmap e54: 8 total, 8 up, 8 in 2024-09-13T18:17:40.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:40 smithi204 ceph-mon[34777]: pgmap v171: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-13T18:17:40.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:40 smithi114 ceph-mon[30897]: osdmap e54: 8 total, 8 up, 8 in 2024-09-13T18:17:40.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:40 smithi114 ceph-mon[30897]: pgmap v171: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-13T18:17:40.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:40 smithi134 ceph-mon[34283]: osdmap e54: 8 total, 8 up, 8 in 2024-09-13T18:17:40.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:40 smithi134 ceph-mon[34283]: pgmap v171: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-13T18:17:41.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:41 smithi204 ceph-mon[34777]: osdmap e55: 8 total, 8 up, 8 in 2024-09-13T18:17:41.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:41 smithi204 ceph-mon[34777]: from='client.? 172.21.15.204:0/1092889175' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "409beb9b-8227-4dad-a4a0-f0e5ce70dc5b"}]: dispatch 2024-09-13T18:17:41.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:41 smithi204 ceph-mon[34777]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "409beb9b-8227-4dad-a4a0-f0e5ce70dc5b"}]: dispatch 2024-09-13T18:17:41.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:41 smithi204 ceph-mon[34777]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "409beb9b-8227-4dad-a4a0-f0e5ce70dc5b"}]': finished 2024-09-13T18:17:41.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:41 smithi204 ceph-mon[34777]: osdmap e56: 9 total, 8 up, 9 in 2024-09-13T18:17:41.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:41 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T18:17:41.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:41 smithi114 ceph-mon[30897]: osdmap e55: 8 total, 8 up, 8 in 2024-09-13T18:17:41.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:41 smithi114 ceph-mon[30897]: from='client.? 172.21.15.204:0/1092889175' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "409beb9b-8227-4dad-a4a0-f0e5ce70dc5b"}]: dispatch 2024-09-13T18:17:41.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:41 smithi114 ceph-mon[30897]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "409beb9b-8227-4dad-a4a0-f0e5ce70dc5b"}]: dispatch 2024-09-13T18:17:41.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:41 smithi114 ceph-mon[30897]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "409beb9b-8227-4dad-a4a0-f0e5ce70dc5b"}]': finished 2024-09-13T18:17:41.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:41 smithi114 ceph-mon[30897]: osdmap e56: 9 total, 8 up, 9 in 2024-09-13T18:17:41.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:41 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T18:17:41.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:41 smithi134 ceph-mon[34283]: osdmap e55: 8 total, 8 up, 8 in 2024-09-13T18:17:41.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:41 smithi134 ceph-mon[34283]: from='client.? 172.21.15.204:0/1092889175' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "409beb9b-8227-4dad-a4a0-f0e5ce70dc5b"}]: dispatch 2024-09-13T18:17:41.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:41 smithi134 ceph-mon[34283]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "409beb9b-8227-4dad-a4a0-f0e5ce70dc5b"}]: dispatch 2024-09-13T18:17:41.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:41 smithi134 ceph-mon[34283]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "409beb9b-8227-4dad-a4a0-f0e5ce70dc5b"}]': finished 2024-09-13T18:17:41.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:41 smithi134 ceph-mon[34283]: osdmap e56: 9 total, 8 up, 9 in 2024-09-13T18:17:41.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:41 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T18:17:42.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:42 smithi204 ceph-mon[34777]: from='client.? 172.21.15.204:0/2821568651' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T18:17:42.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:42 smithi204 ceph-mon[34777]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-13T18:17:42.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:42 smithi114 ceph-mon[30897]: from='client.? 172.21.15.204:0/2821568651' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T18:17:42.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:42 smithi114 ceph-mon[30897]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-13T18:17:42.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:42 smithi134 ceph-mon[34283]: from='client.? 172.21.15.204:0/2821568651' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T18:17:42.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:42 smithi134 ceph-mon[34283]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-13T18:17:45.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:44 smithi204 ceph-mon[34777]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-13T18:17:45.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:44 smithi114 ceph-mon[30897]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-13T18:17:45.321 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:44 smithi134 ceph-mon[34283]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-13T18:17:47.030 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:46 smithi204 ceph-mon[34777]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-13T18:17:47.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:46 smithi114 ceph-mon[30897]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-13T18:17:47.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:46 smithi134 ceph-mon[34283]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-13T18:17:49.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:48 smithi204 ceph-mon[34777]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-13T18:17:49.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:48 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-13T18:17:49.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:48 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:17:49.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:48 smithi204 ceph-mon[34777]: Deploying daemon osd.8 on smithi204 2024-09-13T18:17:49.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:48 smithi114 ceph-mon[30897]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-13T18:17:49.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:48 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-13T18:17:49.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:48 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:17:49.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:48 smithi114 ceph-mon[30897]: Deploying daemon osd.8 on smithi204 2024-09-13T18:17:49.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:48 smithi134 ceph-mon[34283]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-13T18:17:49.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:48 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-13T18:17:49.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:48 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:17:49.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:48 smithi134 ceph-mon[34283]: Deploying daemon osd.8 on smithi204 2024-09-13T18:17:51.071 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:50 smithi204 ceph-mon[34777]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-13T18:17:51.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:50 smithi114 ceph-mon[30897]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-13T18:17:51.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:50 smithi134 ceph-mon[34283]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-13T18:17:53.164 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:52 smithi204 ceph-mon[34777]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-13T18:17:53.219 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:52 smithi114 ceph-mon[30897]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-13T18:17:53.368 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:52 smithi134 ceph-mon[34283]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-13T18:17:54.026 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:53 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:17:54.027 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:53 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:54.027 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:53 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:54.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:53 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:17:54.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:53 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:54.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:53 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:54.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:53 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:17:54.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:53 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:54.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:53 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:55.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:54 smithi204 ceph-mon[34777]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-13T18:17:55.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:54 smithi114 ceph-mon[30897]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-13T18:17:55.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:54 smithi134 ceph-mon[34283]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-13T18:17:56.754 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:17:56.751+0000 7f120b7fe640 1 -- 172.21.15.204:0/1060106006 <== mgr.14148 v2:172.21.15.114:6800/3396185541 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f121402c4f0 con 0x7f12000a1080 2024-09-13T18:17:56.754 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 8 on host 'smithi204' 2024-09-13T18:17:56.756 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:17:56.753+0000 7f121c7b3640 1 -- 172.21.15.204:0/1060106006 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f12000a1080 msgr2=0x7f12000a3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:56.756 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:17:56.753+0000 7f121c7b3640 1 --2- 172.21.15.204:0/1060106006 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f12000a1080 0x7f12000a3540 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1214197110 tx=0x7f121000b390 comp rx=0 tx=0).stop 2024-09-13T18:17:56.756 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:17:56.753+0000 7f121c7b3640 1 -- 172.21.15.204:0/1060106006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1214100f60 msgr2=0x7f1214196170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:56.756 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:17:56.753+0000 7f121c7b3640 1 --2- 172.21.15.204:0/1060106006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1214100f60 0x7f1214196170 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f120400d770 tx=0x7f120400dc30 comp rx=0 tx=0).stop 2024-09-13T18:17:56.756 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:17:56.754+0000 7f121c7b3640 1 -- 172.21.15.204:0/1060106006 shutdown_connections 2024-09-13T18:17:56.756 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:17:56.754+0000 7f121c7b3640 1 --2- 172.21.15.204:0/1060106006 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f12000a1080 0x7f12000a3540 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:56.756 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:17:56.754+0000 7f121c7b3640 1 --2- 172.21.15.204:0/1060106006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1214101920 0x7f121419d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:56.756 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:17:56.754+0000 7f121c7b3640 1 --2- 172.21.15.204:0/1060106006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1214100f60 0x7f1214196170 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:56.756 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:17:56.754+0000 7f121c7b3640 1 --2- 172.21.15.204:0/1060106006 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12140ffe00 0x7f1214195c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:56.757 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:17:56.754+0000 7f121c7b3640 1 -- 172.21.15.204:0/1060106006 >> 172.21.15.204:0/1060106006 conn(0x7f12140fb5d0 msgr2=0x7f1214104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:17:56.757 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:17:56.754+0000 7f121c7b3640 1 -- 172.21.15.204:0/1060106006 shutdown_connections 2024-09-13T18:17:56.757 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:17:56.754+0000 7f121c7b3640 1 -- 172.21.15.204:0/1060106006 wait complete. 2024-09-13T18:17:57.116 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:56 smithi204 ceph-mon[34777]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-13T18:17:57.116 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:56 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:57.116 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:56 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:57.219 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:56 smithi114 ceph-mon[30897]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-13T18:17:57.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:56 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:57.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:56 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:57.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:56 smithi134 ceph-mon[34283]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-13T18:17:57.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:56 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:57.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:56 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:57.397 DEBUG:teuthology.orchestra.run.smithi204:osd.8> sudo journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@osd.8.service 2024-09-13T18:17:57.399 INFO:tasks.cephadm:Deploying osd.9 on smithi114 with /dev/vg_nvme/lv_1... 2024-09-13T18:17:57.400 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-09-13T18:17:57.609 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:17:57.981 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:57 smithi114 ceph-mon[30897]: from='osd.8 [v2:172.21.15.204:6816/858331805,v1:172.21.15.204:6817/858331805]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-13T18:17:57.981 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:57 smithi114 ceph-mon[30897]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-13T18:17:57.981 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:57 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:57.981 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:57 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:58.127 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:57 smithi204 ceph-mon[34777]: from='osd.8 [v2:172.21.15.204:6816/858331805,v1:172.21.15.204:6817/858331805]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-13T18:17:58.128 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:57 smithi204 ceph-mon[34777]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-13T18:17:58.128 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:57 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:58.128 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:57 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:58.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:57 smithi134 ceph-mon[34283]: from='osd.8 [v2:172.21.15.204:6816/858331805,v1:172.21.15.204:6817/858331805]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-13T18:17:58.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:57 smithi134 ceph-mon[34283]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-13T18:17:58.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:57 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:58.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:57 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:17:59.074 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:58 smithi114 ceph-mon[30897]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-13T18:17:59.074 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:58 smithi114 ceph-mon[30897]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-13T18:17:59.075 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:58 smithi114 ceph-mon[30897]: osdmap e57: 9 total, 8 up, 9 in 2024-09-13T18:17:59.075 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:58 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T18:17:59.075 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:58 smithi114 ceph-mon[30897]: from='osd.8 [v2:172.21.15.204:6816/858331805,v1:172.21.15.204:6817/858331805]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-13T18:17:59.075 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:58 smithi114 ceph-mon[30897]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-13T18:17:59.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:58 smithi204 ceph-mon[34777]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-13T18:17:59.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:58 smithi204 ceph-mon[34777]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-13T18:17:59.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:58 smithi204 ceph-mon[34777]: osdmap e57: 9 total, 8 up, 9 in 2024-09-13T18:17:59.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:58 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T18:17:59.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:58 smithi204 ceph-mon[34777]: from='osd.8 [v2:172.21.15.204:6816/858331805,v1:172.21.15.204:6817/858331805]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-13T18:17:59.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:58 smithi204 ceph-mon[34777]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-13T18:17:59.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:58 smithi134 ceph-mon[34283]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-13T18:17:59.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:58 smithi134 ceph-mon[34283]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-13T18:17:59.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:58 smithi134 ceph-mon[34283]: osdmap e57: 9 total, 8 up, 9 in 2024-09-13T18:17:59.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:58 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T18:17:59.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:58 smithi134 ceph-mon[34283]: from='osd.8 [v2:172.21.15.204:6816/858331805,v1:172.21.15.204:6817/858331805]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-13T18:17:59.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:58 smithi134 ceph-mon[34283]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-13T18:17:59.449 INFO:teuthology.orchestra.run.smithi114.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-09-13T18:17:59.547 INFO:teuthology.orchestra.run.smithi114.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-09-13T18:17:59.562 INFO:teuthology.orchestra.run.smithi114.stderr: stderr: 10+0 records in 2024-09-13T18:17:59.562 INFO:teuthology.orchestra.run.smithi114.stderr:10+0 records out 2024-09-13T18:17:59.562 INFO:teuthology.orchestra.run.smithi114.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0133787 s, 784 MB/s 2024-09-13T18:17:59.562 INFO:teuthology.orchestra.run.smithi114.stderr:--> Zapping successful for: 2024-09-13T18:17:59.695 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:17:59 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:17:59.395+0000 7fd48789a640 -1 osd.8 0 waiting for initial osdmap 2024-09-13T18:17:59.695 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:17:59 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:17:59.405+0000 7fd47eea4640 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T18:18:00.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:59 smithi114 ceph-mon[30897]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-09-13T18:18:00.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:59 smithi114 ceph-mon[30897]: osdmap e58: 9 total, 8 up, 9 in 2024-09-13T18:18:00.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:59 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T18:18:00.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:59 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T18:18:00.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:17:59 smithi114 ceph-mon[30897]: from='osd.8 ' entity='osd.8' 2024-09-13T18:18:00.274 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi114:vg_nvme/lv_1 2024-09-13T18:18:00.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:59 smithi204 ceph-mon[34777]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-09-13T18:18:00.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:59 smithi204 ceph-mon[34777]: osdmap e58: 9 total, 8 up, 9 in 2024-09-13T18:18:00.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:59 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T18:18:00.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:59 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T18:18:00.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:17:59 smithi204 ceph-mon[34777]: from='osd.8 ' entity='osd.8' 2024-09-13T18:18:00.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:59 smithi134 ceph-mon[34283]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-09-13T18:18:00.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:59 smithi134 ceph-mon[34283]: osdmap e58: 9 total, 8 up, 9 in 2024-09-13T18:18:00.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:59 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T18:18:00.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:59 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T18:18:00.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:17:59 smithi134 ceph-mon[34283]: from='osd.8 ' entity='osd.8' 2024-09-13T18:18:00.471 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:18:01.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:00 smithi114 ceph-mon[30897]: purged_snaps scrub starts 2024-09-13T18:18:01.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:00 smithi114 ceph-mon[30897]: purged_snaps scrub ok 2024-09-13T18:18:01.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:00 smithi114 ceph-mon[30897]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-13T18:18:01.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:00 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T18:18:01.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:00 smithi114 ceph-mon[30897]: osd.8 [v2:172.21.15.204:6816/858331805,v1:172.21.15.204:6817/858331805] boot 2024-09-13T18:18:01.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:00 smithi114 ceph-mon[30897]: osdmap e59: 9 total, 9 up, 9 in 2024-09-13T18:18:01.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:00 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T18:18:01.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:00 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:01.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:00 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:01.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:00 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:18:01.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:00 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:18:01.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:00 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:18:01.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:00 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:01.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:00 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:18:01.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:00 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:18:01.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:00 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:01.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:00 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:18:01.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:00 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:18:01.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:00 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:18:01.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:00 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:01.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:00 smithi134 ceph-mon[34283]: purged_snaps scrub starts 2024-09-13T18:18:01.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:00 smithi134 ceph-mon[34283]: purged_snaps scrub ok 2024-09-13T18:18:01.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:00 smithi134 ceph-mon[34283]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-13T18:18:01.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:00 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T18:18:01.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:00 smithi134 ceph-mon[34283]: osd.8 [v2:172.21.15.204:6816/858331805,v1:172.21.15.204:6817/858331805] boot 2024-09-13T18:18:01.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:00 smithi134 ceph-mon[34283]: osdmap e59: 9 total, 9 up, 9 in 2024-09-13T18:18:01.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:00 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T18:18:01.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:00 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:01.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:00 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:01.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:00 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:18:01.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:00 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:18:01.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:00 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:18:01.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:00 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:01.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:00 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:18:01.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:00 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:18:01.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:00 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:01.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:00 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:18:01.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:00 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:18:01.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:00 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:18:01.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:00 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:01.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:00 smithi204 ceph-mon[34777]: purged_snaps scrub starts 2024-09-13T18:18:01.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:00 smithi204 ceph-mon[34777]: purged_snaps scrub ok 2024-09-13T18:18:01.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:00 smithi204 ceph-mon[34777]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-13T18:18:01.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:00 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T18:18:01.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:00 smithi204 ceph-mon[34777]: osd.8 [v2:172.21.15.204:6816/858331805,v1:172.21.15.204:6817/858331805] boot 2024-09-13T18:18:01.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:00 smithi204 ceph-mon[34777]: osdmap e59: 9 total, 9 up, 9 in 2024-09-13T18:18:01.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:00 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T18:18:01.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:00 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:01.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:00 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:01.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:00 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:18:01.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:00 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:18:01.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:00 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:18:01.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:00 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:01.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:00 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:18:01.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:00 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:18:01.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:00 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:01.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:00 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:18:01.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:00 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:18:01.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:00 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:18:01.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:00 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:02.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:02.128+0000 7fbee75bc640 1 -- 172.21.15.114:0/1575067490 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbee01008a0 msgr2=0x7fbee0107130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:02.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:02.128+0000 7fbee75bc640 1 --2- 172.21.15.114:0/1575067490 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbee01008a0 0x7fbee0107130 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbed8009ca0 tx=0x7fbed802a8c0 comp rx=0 tx=0).stop 2024-09-13T18:18:02.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:02.128+0000 7fbee75bc640 1 -- 172.21.15.114:0/1575067490 shutdown_connections 2024-09-13T18:18:02.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:02.128+0000 7fbee75bc640 1 --2- 172.21.15.114:0/1575067490 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbee01008a0 0x7fbee0107130 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:02.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:02.128+0000 7fbee75bc640 1 --2- 172.21.15.114:0/1575067490 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbee00ffee0 0x7fbee0100360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:02.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:02.128+0000 7fbee75bc640 1 --2- 172.21.15.114:0/1575067490 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbee00fece0 0x7fbee00ff0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:02.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:02.128+0000 7fbee75bc640 1 -- 172.21.15.114:0/1575067490 >> 172.21.15.114:0/1575067490 conn(0x7fbee00fa490 msgr2=0x7fbee00fc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:18:02.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:02.128+0000 7fbee75bc640 1 -- 172.21.15.114:0/1575067490 shutdown_connections 2024-09-13T18:18:02.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:02.128+0000 7fbee75bc640 1 -- 172.21.15.114:0/1575067490 wait complete. 2024-09-13T18:18:02.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:02.128+0000 7fbee75bc640 1 Processor -- start 2024-09-13T18:18:02.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:02.128+0000 7fbee75bc640 1 -- start start 2024-09-13T18:18:02.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:02.129+0000 7fbee75bc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbee00fece0 0x7fbee0102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:02.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:02.129+0000 7fbee75bc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbee00ffee0 0x7fbee01012c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:02.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:02.129+0000 7fbee75bc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbee01008a0 0x7fbee0101800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:02.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:02.129+0000 7fbee75bc640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee0101f90 con 0x7fbee00fece0 2024-09-13T18:18:02.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:02.129+0000 7fbee75bc640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee0102100 con 0x7fbee00ffee0 2024-09-13T18:18:02.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:02.129+0000 7fbee75bc640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee019fd70 con 0x7fbee01008a0 2024-09-13T18:18:02.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:02.129+0000 7fbee5358640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbee00fece0 0x7fbee0102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:02.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:02.129+0000 7fbee5358640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbee00fece0 0x7fbee0102c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39992/0 (socket says 172.21.15.114:39992) 2024-09-13T18:18:02.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:02.129+0000 7fbee5358640 1 -- 172.21.15.114:0/1560663012 learned_addr learned my addr 172.21.15.114:0/1560663012 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:18:02.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:02.129+0000 7fbee4b57640 1 --2- 172.21.15.114:0/1560663012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbee00ffee0 0x7fbee01012c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:02.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:02.129+0000 7fbee5b59640 1 --2- 172.21.15.114:0/1560663012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbee01008a0 0x7fbee0101800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:02.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:02.129+0000 7fbee5358640 1 -- 172.21.15.114:0/1560663012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbee01008a0 msgr2=0x7fbee0101800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:02.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:02.129+0000 7fbee5358640 1 --2- 172.21.15.114:0/1560663012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbee01008a0 0x7fbee0101800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:02.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:02.129+0000 7fbee5358640 1 -- 172.21.15.114:0/1560663012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbee00ffee0 msgr2=0x7fbee01012c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:02.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:02.129+0000 7fbee5358640 1 --2- 172.21.15.114:0/1560663012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbee00ffee0 0x7fbee01012c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:02.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:02.129+0000 7fbee5358640 1 -- 172.21.15.114:0/1560663012 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed8009900 con 0x7fbee00fece0 2024-09-13T18:18:02.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:02.129+0000 7fbee5b59640 1 --2- 172.21.15.114:0/1560663012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbee01008a0 0x7fbee0101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:18:02.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:02.129+0000 7fbee5358640 1 --2- 172.21.15.114:0/1560663012 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbee00fece0 0x7fbee0102c70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbed400b4b0 tx=0x7fbed400b970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:02.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:02.129+0000 7fbed27fc640 1 -- 172.21.15.114:0/1560663012 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed4010610 con 0x7fbee00fece0 2024-09-13T18:18:02.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:02.129+0000 7fbed27fc640 1 -- 172.21.15.114:0/1560663012 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbed4010c00 con 0x7fbee00fece0 2024-09-13T18:18:02.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:02.129+0000 7fbee75bc640 1 -- 172.21.15.114:0/1560663012 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbee019ffc0 con 0x7fbee00fece0 2024-09-13T18:18:02.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:02.129+0000 7fbed27fc640 1 -- 172.21.15.114:0/1560663012 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed4017bb0 con 0x7fbee00fece0 2024-09-13T18:18:02.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:02.129+0000 7fbee75bc640 1 -- 172.21.15.114:0/1560663012 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbee01a0460 con 0x7fbee00fece0 2024-09-13T18:18:02.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:02.130+0000 7fbee75bc640 1 -- 172.21.15.114:0/1560663012 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbee010a4d0 con 0x7fbee00fece0 2024-09-13T18:18:02.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:02.132+0000 7fbed27fc640 1 -- 172.21.15.114:0/1560663012 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fbed4010770 con 0x7fbee00fece0 2024-09-13T18:18:02.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:02.132+0000 7fbed27fc640 1 --2- 172.21.15.114:0/1560663012 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fbecc0a1080 0x7fbecc0a3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:02.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:02.132+0000 7fbee4b57640 1 --2- 172.21.15.114:0/1560663012 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fbecc0a1080 0x7fbecc0a3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:02.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:02.132+0000 7fbed27fc640 1 -- 172.21.15.114:0/1560663012 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5858+0+0 (secure 0 0 0) 0x7fbed40cbaa0 con 0x7fbee00fece0 2024-09-13T18:18:02.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:02.133+0000 7fbee4b57640 1 --2- 172.21.15.114:0/1560663012 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fbecc0a1080 0x7fbecc0a3540 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbedc005fd0 tx=0x7fbedc005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:02.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:02.135+0000 7fbed27fc640 1 -- 172.21.15.114:0/1560663012 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fbed4095bc0 con 0x7fbee00fece0 2024-09-13T18:18:02.168 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:01 smithi114 ceph-mon[30897]: Detected new or changed devices on smithi204 2024-09-13T18:18:02.168 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:01 smithi114 ceph-mon[30897]: Adjusting osd_memory_target on smithi204 to 5696M 2024-09-13T18:18:02.168 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:01 smithi114 ceph-mon[30897]: osdmap e60: 9 total, 9 up, 9 in 2024-09-13T18:18:02.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:02.307+0000 7fbee75bc640 1 -- 172.21.15.114:0/1560663012 --> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fbee0068700 con 0x7fbecc0a1080 2024-09-13T18:18:02.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:01 smithi134 ceph-mon[34283]: Detected new or changed devices on smithi204 2024-09-13T18:18:02.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:01 smithi134 ceph-mon[34283]: Adjusting osd_memory_target on smithi204 to 5696M 2024-09-13T18:18:02.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:01 smithi134 ceph-mon[34283]: osdmap e60: 9 total, 9 up, 9 in 2024-09-13T18:18:02.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:01 smithi204 ceph-mon[34777]: Detected new or changed devices on smithi204 2024-09-13T18:18:02.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:01 smithi204 ceph-mon[34777]: Adjusting osd_memory_target on smithi204 to 5696M 2024-09-13T18:18:02.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:01 smithi204 ceph-mon[34777]: osdmap e60: 9 total, 9 up, 9 in 2024-09-13T18:18:03.338 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:02 smithi114 ceph-mon[30897]: pgmap v188: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-13T18:18:03.339 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:02 smithi114 ceph-mon[30897]: osdmap e61: 9 total, 9 up, 9 in 2024-09-13T18:18:03.339 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:02 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T18:18:03.339 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:02 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T18:18:03.339 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:02 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:18:03.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:02 smithi134 ceph-mon[34283]: pgmap v188: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-13T18:18:03.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:02 smithi134 ceph-mon[34283]: osdmap e61: 9 total, 9 up, 9 in 2024-09-13T18:18:03.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:02 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T18:18:03.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:02 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T18:18:03.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:02 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:18:03.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:02 smithi204 ceph-mon[34777]: pgmap v188: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-13T18:18:03.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:02 smithi204 ceph-mon[34777]: osdmap e61: 9 total, 9 up, 9 in 2024-09-13T18:18:03.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:02 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T18:18:03.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:02 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T18:18:03.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:02 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:18:04.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:03 smithi114 ceph-mon[30897]: from='client.14409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:18:04.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:03 smithi114 ceph-mon[30897]: osdmap e62: 9 total, 9 up, 9 in 2024-09-13T18:18:04.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:03 smithi134 ceph-mon[34283]: from='client.14409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:18:04.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:03 smithi134 ceph-mon[34283]: osdmap e62: 9 total, 9 up, 9 in 2024-09-13T18:18:04.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:03 smithi204 ceph-mon[34777]: from='client.14409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:18:04.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:03 smithi204 ceph-mon[34777]: osdmap e62: 9 total, 9 up, 9 in 2024-09-13T18:18:05.144 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:04 smithi114 ceph-mon[30897]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-13T18:18:05.144 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:04 smithi114 ceph-mon[30897]: osdmap e63: 9 total, 9 up, 9 in 2024-09-13T18:18:05.144 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:04 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/3988426420' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a345a15c-b1fc-4877-b445-418eebf0d0ce"}]: dispatch 2024-09-13T18:18:05.144 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:04 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/3988426420' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a345a15c-b1fc-4877-b445-418eebf0d0ce"}]': finished 2024-09-13T18:18:05.144 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:04 smithi114 ceph-mon[30897]: osdmap e64: 10 total, 9 up, 10 in 2024-09-13T18:18:05.144 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:04 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-13T18:18:05.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:04 smithi134 ceph-mon[34283]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-13T18:18:05.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:04 smithi134 ceph-mon[34283]: osdmap e63: 9 total, 9 up, 9 in 2024-09-13T18:18:05.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:04 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/3988426420' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a345a15c-b1fc-4877-b445-418eebf0d0ce"}]: dispatch 2024-09-13T18:18:05.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:04 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/3988426420' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a345a15c-b1fc-4877-b445-418eebf0d0ce"}]': finished 2024-09-13T18:18:05.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:04 smithi134 ceph-mon[34283]: osdmap e64: 10 total, 9 up, 10 in 2024-09-13T18:18:05.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:04 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-13T18:18:05.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:04 smithi204 ceph-mon[34777]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-13T18:18:05.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:04 smithi204 ceph-mon[34777]: osdmap e63: 9 total, 9 up, 9 in 2024-09-13T18:18:05.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:04 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/3988426420' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a345a15c-b1fc-4877-b445-418eebf0d0ce"}]: dispatch 2024-09-13T18:18:05.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:04 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/3988426420' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a345a15c-b1fc-4877-b445-418eebf0d0ce"}]': finished 2024-09-13T18:18:05.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:04 smithi204 ceph-mon[34777]: osdmap e64: 10 total, 9 up, 10 in 2024-09-13T18:18:05.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:04 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-13T18:18:06.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:05 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/3465375507' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T18:18:06.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:05 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/3465375507' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T18:18:06.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:05 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/3465375507' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T18:18:07.345 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:06 smithi204 ceph-mon[34777]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-13T18:18:07.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:06 smithi134 ceph-mon[34283]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-13T18:18:07.469 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:06 smithi114 ceph-mon[30897]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-13T18:18:09.174 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:08 smithi114 ceph-mon[30897]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-09-13T18:18:09.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:08 smithi134 ceph-mon[34283]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-09-13T18:18:09.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:08 smithi204 ceph-mon[34777]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-09-13T18:18:11.141 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:10 smithi114 ceph-mon[30897]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-09-13T18:18:11.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:10 smithi134 ceph-mon[34283]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-09-13T18:18:11.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:10 smithi204 ceph-mon[34777]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-09-13T18:18:12.203 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:12 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-13T18:18:12.203 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:12 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:18:12.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:12 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-13T18:18:12.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:12 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:18:12.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:12 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-13T18:18:12.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:12 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:18:13.219 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:13 smithi114 ceph-mon[30897]: Deploying daemon osd.9 on smithi114 2024-09-13T18:18:13.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:13 smithi114 ceph-mon[30897]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-13T18:18:13.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:13 smithi134 ceph-mon[34283]: Deploying daemon osd.9 on smithi114 2024-09-13T18:18:13.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:13 smithi134 ceph-mon[34283]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-13T18:18:13.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:13 smithi204 ceph-mon[34777]: Deploying daemon osd.9 on smithi114 2024-09-13T18:18:13.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:13 smithi204 ceph-mon[34777]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-13T18:18:15.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:15 smithi134 ceph-mon[34283]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-13T18:18:15.443 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:15 smithi114 ceph-mon[30897]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-13T18:18:15.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:15 smithi204 ceph-mon[34777]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-13T18:18:17.300 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:17 smithi114 ceph-mon[30897]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-13T18:18:17.300 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:17 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:18:17.300 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:17 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:17.301 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:17 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:17.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:17 smithi134 ceph-mon[34283]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-13T18:18:17.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:17 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:18:17.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:17 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:17.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:17 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:17.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:17 smithi204 ceph-mon[34777]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-13T18:18:17.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:17 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:18:17.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:17 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:17.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:17 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:19.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:19 smithi114 ceph-mon[30897]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-13T18:18:19.368 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:19 smithi134 ceph-mon[34283]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-13T18:18:19.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:19 smithi204 ceph-mon[34777]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-13T18:18:20.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:20.347+0000 7fbed27fc640 1 -- 172.21.15.114:0/1560663012 <== mgr.14148 v2:172.21.15.114:6800/3396185541 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fbee0068700 con 0x7fbecc0a1080 2024-09-13T18:18:20.351 INFO:teuthology.orchestra.run.smithi114.stdout:Created osd(s) 9 on host 'smithi114' 2024-09-13T18:18:20.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:20.349+0000 7fbee75bc640 1 -- 172.21.15.114:0/1560663012 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fbecc0a1080 msgr2=0x7fbecc0a3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:20.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:20.349+0000 7fbee75bc640 1 --2- 172.21.15.114:0/1560663012 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fbecc0a1080 0x7fbecc0a3540 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbedc005fd0 tx=0x7fbedc005850 comp rx=0 tx=0).stop 2024-09-13T18:18:20.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:20.349+0000 7fbee75bc640 1 -- 172.21.15.114:0/1560663012 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbee00fece0 msgr2=0x7fbee0102c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:20.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:20.349+0000 7fbee75bc640 1 --2- 172.21.15.114:0/1560663012 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbee00fece0 0x7fbee0102c70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbed400b4b0 tx=0x7fbed400b970 comp rx=0 tx=0).stop 2024-09-13T18:18:20.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:20.349+0000 7fbee75bc640 1 -- 172.21.15.114:0/1560663012 shutdown_connections 2024-09-13T18:18:20.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:20.349+0000 7fbee75bc640 1 --2- 172.21.15.114:0/1560663012 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fbecc0a1080 0x7fbecc0a3540 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:20.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:20.349+0000 7fbee75bc640 1 --2- 172.21.15.114:0/1560663012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbee01008a0 0x7fbee0101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:20.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:20.349+0000 7fbee75bc640 1 --2- 172.21.15.114:0/1560663012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbee00ffee0 0x7fbee01012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:20.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:20.349+0000 7fbee75bc640 1 --2- 172.21.15.114:0/1560663012 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbee00fece0 0x7fbee0102c70 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:20.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:20.349+0000 7fbee75bc640 1 -- 172.21.15.114:0/1560663012 >> 172.21.15.114:0/1560663012 conn(0x7fbee00fa490 msgr2=0x7fbee0067ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:18:20.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:20.349+0000 7fbee75bc640 1 -- 172.21.15.114:0/1560663012 shutdown_connections 2024-09-13T18:18:20.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:18:20.350+0000 7fbee75bc640 1 -- 172.21.15.114:0/1560663012 wait complete. 2024-09-13T18:18:20.603 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:20 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:20.603 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:20 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:20.604 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:20 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:18:20.604 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:20 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:18:20.604 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:20 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:20.604 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:20 smithi114 ceph-mon[30897]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-13T18:18:20.604 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:20 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:18:20.604 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:20 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:20.604 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:20 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:20.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:20 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:20.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:20 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:20.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:20 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:18:20.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:20 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:18:20.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:20 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:20.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:20 smithi134 ceph-mon[34283]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-13T18:18:20.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:20 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:18:20.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:20 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:20.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:20 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:20.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:20 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:20.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:20 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:20.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:20 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:18:20.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:20 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:18:20.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:20 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:20.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:20 smithi204 ceph-mon[34777]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-13T18:18:20.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:20 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:18:20.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:20 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:20.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:20 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:20.995 DEBUG:teuthology.orchestra.run.smithi114:osd.9> sudo journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@osd.9.service 2024-09-13T18:18:20.999 INFO:tasks.cephadm:Deploying osd.10 on smithi134 with /dev/vg_nvme/lv_1... 2024-09-13T18:18:20.999 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-09-13T18:18:21.207 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T18:18:21.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:21 smithi134 ceph-mon[34283]: from='osd.9 [v2:172.21.15.114:6826/2172392854,v1:172.21.15.114:6827/2172392854]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-13T18:18:21.864 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:21 smithi114 ceph-mon[30897]: from='osd.9 [v2:172.21.15.114:6826/2172392854,v1:172.21.15.114:6827/2172392854]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-13T18:18:21.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:21 smithi204 ceph-mon[34777]: from='osd.9 [v2:172.21.15.114:6826/2172392854,v1:172.21.15.114:6827/2172392854]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-13T18:18:22.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:22 smithi134 ceph-mon[34283]: from='osd.9 [v2:172.21.15.114:6826/2172392854,v1:172.21.15.114:6827/2172392854]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-13T18:18:22.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:22 smithi134 ceph-mon[34283]: osdmap e65: 10 total, 9 up, 10 in 2024-09-13T18:18:22.870 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:22 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-13T18:18:22.870 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:22 smithi134 ceph-mon[34283]: from='osd.9 [v2:172.21.15.114:6826/2172392854,v1:172.21.15.114:6827/2172392854]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-09-13T18:18:22.870 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:22 smithi134 ceph-mon[34283]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-13T18:18:22.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:22 smithi204 ceph-mon[34777]: from='osd.9 [v2:172.21.15.114:6826/2172392854,v1:172.21.15.114:6827/2172392854]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-13T18:18:22.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:22 smithi204 ceph-mon[34777]: osdmap e65: 10 total, 9 up, 10 in 2024-09-13T18:18:22.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:22 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-13T18:18:22.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:22 smithi204 ceph-mon[34777]: from='osd.9 [v2:172.21.15.114:6826/2172392854,v1:172.21.15.114:6827/2172392854]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-09-13T18:18:22.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:22 smithi204 ceph-mon[34777]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-13T18:18:22.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:22 smithi114 ceph-mon[30897]: from='osd.9 [v2:172.21.15.114:6826/2172392854,v1:172.21.15.114:6827/2172392854]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-13T18:18:22.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:22 smithi114 ceph-mon[30897]: osdmap e65: 10 total, 9 up, 10 in 2024-09-13T18:18:22.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:22 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-13T18:18:22.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:22 smithi114 ceph-mon[30897]: from='osd.9 [v2:172.21.15.114:6826/2172392854,v1:172.21.15.114:6827/2172392854]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-09-13T18:18:22.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:22 smithi114 ceph-mon[30897]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-13T18:18:22.971 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:18:22 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:18:22.571+0000 7fe9bbb93640 -1 osd.9 0 waiting for initial osdmap 2024-09-13T18:18:22.971 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:18:22 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:18:22.585+0000 7fe9b79bc640 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T18:18:23.022 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-09-13T18:18:23.103 INFO:teuthology.orchestra.run.smithi134.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-09-13T18:18:23.118 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10+0 records in 2024-09-13T18:18:23.118 INFO:teuthology.orchestra.run.smithi134.stderr:10+0 records out 2024-09-13T18:18:23.118 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0130432 s, 804 MB/s 2024-09-13T18:18:23.118 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping successful for: 2024-09-13T18:18:23.795 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:23 smithi134 ceph-mon[34283]: from='osd.9 [v2:172.21.15.114:6826/2172392854,v1:172.21.15.114:6827/2172392854]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-09-13T18:18:23.796 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:23 smithi134 ceph-mon[34283]: osdmap e66: 10 total, 9 up, 10 in 2024-09-13T18:18:23.796 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:23 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-13T18:18:23.796 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:23 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-13T18:18:23.817 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi134:vg_nvme/lv_1 2024-09-13T18:18:23.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:23 smithi204 ceph-mon[34777]: from='osd.9 [v2:172.21.15.114:6826/2172392854,v1:172.21.15.114:6827/2172392854]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-09-13T18:18:23.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:23 smithi204 ceph-mon[34777]: osdmap e66: 10 total, 9 up, 10 in 2024-09-13T18:18:23.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:23 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-13T18:18:23.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:23 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-13T18:18:23.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:23 smithi114 ceph-mon[30897]: from='osd.9 [v2:172.21.15.114:6826/2172392854,v1:172.21.15.114:6827/2172392854]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-09-13T18:18:23.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:23 smithi114 ceph-mon[30897]: osdmap e66: 10 total, 9 up, 10 in 2024-09-13T18:18:23.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:23 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-13T18:18:23.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:23 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-13T18:18:24.012 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T18:18:24.835 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:24 smithi134 ceph-mon[34283]: purged_snaps scrub starts 2024-09-13T18:18:24.835 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:24 smithi134 ceph-mon[34283]: purged_snaps scrub ok 2024-09-13T18:18:24.835 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:24 smithi134 ceph-mon[34283]: osd.9 [v2:172.21.15.114:6826/2172392854,v1:172.21.15.114:6827/2172392854] boot 2024-09-13T18:18:24.835 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:24 smithi134 ceph-mon[34283]: osdmap e67: 10 total, 10 up, 10 in 2024-09-13T18:18:24.835 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:24 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-13T18:18:24.835 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:24 smithi134 ceph-mon[34283]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-13T18:18:24.835 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:24 smithi134 ceph-mon[34283]: Detected new or changed devices on smithi114 2024-09-13T18:18:24.835 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:24 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:24.835 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:24 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:24.836 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:24 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:18:24.836 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:24 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:18:24.836 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:24 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:18:24.836 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:24 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:18:24.836 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:24 smithi134 ceph-mon[34283]: Adjusting osd_memory_target on smithi114 to 4272M 2024-09-13T18:18:24.836 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:24 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:24.836 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:24 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:18:24.836 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:24 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:18:24.836 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:24 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:24.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:24 smithi204 ceph-mon[34777]: purged_snaps scrub starts 2024-09-13T18:18:24.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:24 smithi204 ceph-mon[34777]: purged_snaps scrub ok 2024-09-13T18:18:24.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:24 smithi204 ceph-mon[34777]: osd.9 [v2:172.21.15.114:6826/2172392854,v1:172.21.15.114:6827/2172392854] boot 2024-09-13T18:18:24.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:24 smithi204 ceph-mon[34777]: osdmap e67: 10 total, 10 up, 10 in 2024-09-13T18:18:24.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:24 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-13T18:18:24.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:24 smithi204 ceph-mon[34777]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-13T18:18:24.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:24 smithi204 ceph-mon[34777]: Detected new or changed devices on smithi114 2024-09-13T18:18:24.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:24 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:24.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:24 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:24.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:24 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:18:24.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:24 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:18:24.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:24 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:18:24.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:24 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:18:24.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:24 smithi204 ceph-mon[34777]: Adjusting osd_memory_target on smithi114 to 4272M 2024-09-13T18:18:24.946 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:24 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:24.946 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:24 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:18:24.946 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:24 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:18:24.946 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:24 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:24.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:24 smithi114 ceph-mon[30897]: purged_snaps scrub starts 2024-09-13T18:18:24.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:24 smithi114 ceph-mon[30897]: purged_snaps scrub ok 2024-09-13T18:18:24.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:24 smithi114 ceph-mon[30897]: osd.9 [v2:172.21.15.114:6826/2172392854,v1:172.21.15.114:6827/2172392854] boot 2024-09-13T18:18:24.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:24 smithi114 ceph-mon[30897]: osdmap e67: 10 total, 10 up, 10 in 2024-09-13T18:18:24.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:24 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-13T18:18:24.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:24 smithi114 ceph-mon[30897]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-13T18:18:24.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:24 smithi114 ceph-mon[30897]: Detected new or changed devices on smithi114 2024-09-13T18:18:24.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:24 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:24.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:24 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:24.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:24 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:18:24.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:24 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:18:24.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:24 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:18:24.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:24 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:18:24.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:24 smithi114 ceph-mon[30897]: Adjusting osd_memory_target on smithi114 to 4272M 2024-09-13T18:18:24.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:24 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:24.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:24 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:18:24.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:24 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:18:24.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:24 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:25.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:25.668+0000 7f572f9db640 1 -- 172.21.15.134:0/3365910628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5728100fb0 msgr2=0x7f5728103420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:25.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:25.668+0000 7f572f9db640 1 --2- 172.21.15.134:0/3365910628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5728100fb0 0x7f5728103420 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5718009900 tx=0x7f571802a410 comp rx=0 tx=0).stop 2024-09-13T18:18:25.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:25.668+0000 7f572f9db640 1 -- 172.21.15.134:0/3365910628 shutdown_connections 2024-09-13T18:18:25.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:25.668+0000 7f572f9db640 1 --2- 172.21.15.134:0/3365910628 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5728103960 0x7f5728103e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:25.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:25.668+0000 7f572f9db640 1 --2- 172.21.15.134:0/3365910628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5728100fb0 0x7f5728103420 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:25.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:25.668+0000 7f572f9db640 1 --2- 172.21.15.134:0/3365910628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57280fe660 0x7f5728100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:25.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:25.668+0000 7f572f9db640 1 -- 172.21.15.134:0/3365910628 >> 172.21.15.134:0/3365910628 conn(0x7f57280fa470 msgr2=0x7f57280fc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:18:25.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:25.668+0000 7f572f9db640 1 -- 172.21.15.134:0/3365910628 shutdown_connections 2024-09-13T18:18:25.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:25.668+0000 7f572f9db640 1 -- 172.21.15.134:0/3365910628 wait complete. 2024-09-13T18:18:25.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:25.669+0000 7f572f9db640 1 Processor -- start 2024-09-13T18:18:25.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:25.669+0000 7f572f9db640 1 -- start start 2024-09-13T18:18:25.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:25.669+0000 7f572f9db640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57280fe660 0x7f572819c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:25.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:25.669+0000 7f572f9db640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5728100fb0 0x7f572819c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:25.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:25.669+0000 7f572f9db640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5728103960 0x7f57281a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:25.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:25.669+0000 7f572f9db640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57281a3d10 con 0x7f5728100fb0 2024-09-13T18:18:25.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:25.669+0000 7f572f9db640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57281a3e80 con 0x7f5728103960 2024-09-13T18:18:25.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:25.669+0000 7f572f9db640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57281a4040 con 0x7f57280fe660 2024-09-13T18:18:25.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:25.669+0000 7f572df78640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5728103960 0x7f57281a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:25.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:25.669+0000 7f572df78640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5728103960 0x7f57281a37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:44680/0 (socket says 172.21.15.134:44680) 2024-09-13T18:18:25.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:25.669+0000 7f572df78640 1 -- 172.21.15.134:0/2658516223 learned_addr learned my addr 172.21.15.134:0/2658516223 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-13T18:18:25.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:25.669+0000 7f572d777640 1 --2- 172.21.15.134:0/2658516223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57280fe660 0x7f572819c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:25.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:25.669+0000 7f572cf76640 1 --2- 172.21.15.134:0/2658516223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5728100fb0 0x7f572819c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:25.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:25.670+0000 7f572df78640 1 -- 172.21.15.134:0/2658516223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57280fe660 msgr2=0x7f572819c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:25.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:25.670+0000 7f572df78640 1 --2- 172.21.15.134:0/2658516223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57280fe660 0x7f572819c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:25.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:25.670+0000 7f572df78640 1 -- 172.21.15.134:0/2658516223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5728100fb0 msgr2=0x7f572819c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:25.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:25.670+0000 7f572df78640 1 --2- 172.21.15.134:0/2658516223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5728100fb0 0x7f572819c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:25.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:25.670+0000 7f572df78640 1 -- 172.21.15.134:0/2658516223 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5718009560 con 0x7f5728103960 2024-09-13T18:18:25.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:25.670+0000 7f572cf76640 1 --2- 172.21.15.134:0/2658516223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5728100fb0 0x7f572819c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:18:25.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:25.670+0000 7f572df78640 1 --2- 172.21.15.134:0/2658516223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5728103960 0x7f57281a37d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f572000cce0 tx=0x7f572000a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:25.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:25.670+0000 7f571e7fc640 1 -- 172.21.15.134:0/2658516223 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572000f450 con 0x7f5728103960 2024-09-13T18:18:25.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:25.670+0000 7f571e7fc640 1 -- 172.21.15.134:0/2658516223 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f572000fa40 con 0x7f5728103960 2024-09-13T18:18:25.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:25.670+0000 7f572f9db640 1 -- 172.21.15.134:0/2658516223 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57281a4320 con 0x7f5728103960 2024-09-13T18:18:25.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:25.670+0000 7f571e7fc640 1 -- 172.21.15.134:0/2658516223 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57200189a0 con 0x7f5728103960 2024-09-13T18:18:25.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:25.670+0000 7f572f9db640 1 -- 172.21.15.134:0/2658516223 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57281a47f0 con 0x7f5728103960 2024-09-13T18:18:25.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:25.671+0000 7f572f9db640 1 -- 172.21.15.134:0/2658516223 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57280fbf60 con 0x7f5728103960 2024-09-13T18:18:25.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:25.672+0000 7f571e7fc640 1 -- 172.21.15.134:0/2658516223 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f572000fbb0 con 0x7f5728103960 2024-09-13T18:18:25.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:25.673+0000 7f571e7fc640 1 --2- 172.21.15.134:0/2658516223 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f57140a1130 0x7f57140a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:25.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:25.673+0000 7f571e7fc640 1 -- 172.21.15.134:0/2658516223 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f57200cc500 con 0x7f5728103960 2024-09-13T18:18:25.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:25.673+0000 7f572d777640 1 --2- 172.21.15.134:0/2658516223 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f57140a1130 0x7f57140a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:25.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:25.673+0000 7f572d777640 1 --2- 172.21.15.134:0/2658516223 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f57140a1130 0x7f57140a35f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5724005fd0 tx=0x7f572400b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:25.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:25.677+0000 7f571e7fc640 1 -- 172.21.15.134:0/2658516223 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f5720096490 con 0x7f5728103960 2024-09-13T18:18:25.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:25.850+0000 7f572f9db640 1 -- 172.21.15.134:0/2658516223 --> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f57280608f0 con 0x7f57140a1130 2024-09-13T18:18:25.853 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:25 smithi134 ceph-mon[34283]: osdmap e68: 10 total, 10 up, 10 in 2024-09-13T18:18:25.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:25 smithi204 ceph-mon[34777]: osdmap e68: 10 total, 10 up, 10 in 2024-09-13T18:18:25.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:25 smithi114 ceph-mon[30897]: osdmap e68: 10 total, 10 up, 10 in 2024-09-13T18:18:26.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:26 smithi134 ceph-mon[34283]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:18:26.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:26 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T18:18:26.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:26 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T18:18:26.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:26 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:18:26.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:26 smithi134 ceph-mon[34283]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-13T18:18:26.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:26 smithi204 ceph-mon[34777]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:18:26.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:26 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T18:18:26.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:26 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T18:18:26.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:26 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:18:26.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:26 smithi204 ceph-mon[34777]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-13T18:18:26.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:26 smithi114 ceph-mon[30897]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:18:26.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:26 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T18:18:26.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:26 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T18:18:26.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:26 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:18:26.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:26 smithi114 ceph-mon[30897]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-13T18:18:29.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:28 smithi134 ceph-mon[34283]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T18:18:29.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:28 smithi134 ceph-mon[34283]: from='client.? 172.21.15.134:0/827611222' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "464d995f-d90b-4caa-addb-1f7562bd74b1"}]: dispatch 2024-09-13T18:18:29.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:28 smithi134 ceph-mon[34283]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "464d995f-d90b-4caa-addb-1f7562bd74b1"}]: dispatch 2024-09-13T18:18:29.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:28 smithi134 ceph-mon[34283]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "464d995f-d90b-4caa-addb-1f7562bd74b1"}]': finished 2024-09-13T18:18:29.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:28 smithi134 ceph-mon[34283]: osdmap e69: 11 total, 10 up, 11 in 2024-09-13T18:18:29.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:28 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T18:18:29.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:28 smithi134 ceph-mon[34283]: from='client.? 172.21.15.134:0/2729791158' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T18:18:29.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:28 smithi204 ceph-mon[34777]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T18:18:29.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:28 smithi204 ceph-mon[34777]: from='client.? 172.21.15.134:0/827611222' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "464d995f-d90b-4caa-addb-1f7562bd74b1"}]: dispatch 2024-09-13T18:18:29.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:28 smithi204 ceph-mon[34777]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "464d995f-d90b-4caa-addb-1f7562bd74b1"}]: dispatch 2024-09-13T18:18:29.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:28 smithi204 ceph-mon[34777]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "464d995f-d90b-4caa-addb-1f7562bd74b1"}]': finished 2024-09-13T18:18:29.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:28 smithi204 ceph-mon[34777]: osdmap e69: 11 total, 10 up, 11 in 2024-09-13T18:18:29.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:28 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T18:18:29.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:28 smithi204 ceph-mon[34777]: from='client.? 172.21.15.134:0/2729791158' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T18:18:29.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:28 smithi114 ceph-mon[30897]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T18:18:29.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:28 smithi114 ceph-mon[30897]: from='client.? 172.21.15.134:0/827611222' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "464d995f-d90b-4caa-addb-1f7562bd74b1"}]: dispatch 2024-09-13T18:18:29.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:28 smithi114 ceph-mon[30897]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "464d995f-d90b-4caa-addb-1f7562bd74b1"}]: dispatch 2024-09-13T18:18:29.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:28 smithi114 ceph-mon[30897]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "464d995f-d90b-4caa-addb-1f7562bd74b1"}]': finished 2024-09-13T18:18:29.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:28 smithi114 ceph-mon[30897]: osdmap e69: 11 total, 10 up, 11 in 2024-09-13T18:18:29.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:28 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T18:18:29.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:28 smithi114 ceph-mon[30897]: from='client.? 172.21.15.134:0/2729791158' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T18:18:31.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:30 smithi204 ceph-mon[34777]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T18:18:31.219 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:30 smithi114 ceph-mon[30897]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T18:18:31.368 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:30 smithi134 ceph-mon[34283]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T18:18:33.192 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:32 smithi134 ceph-mon[34283]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T18:18:33.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:32 smithi204 ceph-mon[34777]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T18:18:33.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:32 smithi114 ceph-mon[30897]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T18:18:35.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:34 smithi204 ceph-mon[34777]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T18:18:35.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:34 smithi114 ceph-mon[30897]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T18:18:35.324 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:34 smithi134 ceph-mon[34283]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T18:18:35.968 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:35 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-13T18:18:35.969 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:35 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:18:36.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:35 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-13T18:18:36.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:35 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:18:36.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:35 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-13T18:18:36.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:35 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:18:37.093 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:36 smithi134 ceph-mon[34283]: Deploying daemon osd.10 on smithi134 2024-09-13T18:18:37.093 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:36 smithi134 ceph-mon[34283]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T18:18:37.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:36 smithi114 ceph-mon[30897]: Deploying daemon osd.10 on smithi134 2024-09-13T18:18:37.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:36 smithi114 ceph-mon[30897]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T18:18:37.345 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:36 smithi204 ceph-mon[34777]: Deploying daemon osd.10 on smithi134 2024-09-13T18:18:37.345 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:36 smithi204 ceph-mon[34777]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T18:18:39.037 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:38 smithi134 ceph-mon[34283]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T18:18:39.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:38 smithi114 ceph-mon[30897]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T18:18:39.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:38 smithi204 ceph-mon[34777]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T18:18:41.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:40 smithi114 ceph-mon[30897]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T18:18:41.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:40 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:18:41.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:40 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:41.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:40 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:41.264 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:40 smithi134 ceph-mon[34283]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T18:18:41.264 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:40 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:18:41.264 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:40 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:41.264 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:40 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:41.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:40 smithi204 ceph-mon[34777]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T18:18:41.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:40 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:18:41.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:40 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:41.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:40 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:43.183 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:42 smithi134 ceph-mon[34283]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T18:18:43.219 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:42 smithi114 ceph-mon[30897]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T18:18:43.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:42 smithi204 ceph-mon[34777]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T18:18:44.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:44.219+0000 7f571e7fc640 1 -- 172.21.15.134:0/2658516223 <== mgr.14148 v2:172.21.15.114:6800/3396185541 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f57280608f0 con 0x7f57140a1130 2024-09-13T18:18:44.222 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 10 on host 'smithi134' 2024-09-13T18:18:44.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:44.221+0000 7f572f9db640 1 -- 172.21.15.134:0/2658516223 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f57140a1130 msgr2=0x7f57140a35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:44.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:44.221+0000 7f572f9db640 1 --2- 172.21.15.134:0/2658516223 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f57140a1130 0x7f57140a35f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5724005fd0 tx=0x7f572400b390 comp rx=0 tx=0).stop 2024-09-13T18:18:44.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:44.221+0000 7f572f9db640 1 -- 172.21.15.134:0/2658516223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5728103960 msgr2=0x7f57281a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:44.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:44.221+0000 7f572f9db640 1 --2- 172.21.15.134:0/2658516223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5728103960 0x7f57281a37d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f572000cce0 tx=0x7f572000a620 comp rx=0 tx=0).stop 2024-09-13T18:18:44.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:44.221+0000 7f572f9db640 1 -- 172.21.15.134:0/2658516223 shutdown_connections 2024-09-13T18:18:44.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:44.221+0000 7f572f9db640 1 --2- 172.21.15.134:0/2658516223 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f57140a1130 0x7f57140a35f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:44.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:44.221+0000 7f572f9db640 1 --2- 172.21.15.134:0/2658516223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5728103960 0x7f57281a37d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:44.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:44.221+0000 7f572f9db640 1 --2- 172.21.15.134:0/2658516223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5728100fb0 0x7f572819c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:44.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:44.221+0000 7f572f9db640 1 --2- 172.21.15.134:0/2658516223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57280fe660 0x7f572819c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:44.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:44.221+0000 7f572f9db640 1 -- 172.21.15.134:0/2658516223 >> 172.21.15.134:0/2658516223 conn(0x7f57280fa470 msgr2=0x7f57280ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:18:44.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:44.221+0000 7f572f9db640 1 -- 172.21.15.134:0/2658516223 shutdown_connections 2024-09-13T18:18:44.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-13T18:18:44.221+0000 7f572f9db640 1 -- 172.21.15.134:0/2658516223 wait complete. 2024-09-13T18:18:44.522 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:44 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:44.522 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:44 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:44.522 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:44 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:18:44.523 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:44 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:18:44.523 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:44 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:44.523 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:44 smithi134 ceph-mon[34283]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T18:18:44.523 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:44 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:18:44.523 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:44 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:44.523 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:44 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:44.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:44 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:44.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:44 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:44.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:44 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:18:44.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:44 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:18:44.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:44 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:44.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:44 smithi204 ceph-mon[34777]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T18:18:44.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:44 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:18:44.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:44 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:44.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:44 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:44.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:44 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:44.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:44 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:44.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:44 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:18:44.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:44 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:18:44.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:44 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:44.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:44 smithi114 ceph-mon[30897]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T18:18:44.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:44 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:18:44.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:44 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:44.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:44 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:44.941 DEBUG:teuthology.orchestra.run.smithi134:osd.10> sudo journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@osd.10.service 2024-09-13T18:18:44.943 INFO:tasks.cephadm:Deploying osd.11 on smithi204 with /dev/vg_nvme/lv_1... 2024-09-13T18:18:44.943 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-09-13T18:18:45.151 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T18:18:45.525 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:45 smithi204 ceph-mon[34777]: from='osd.10 [v2:172.21.15.134:6824/3073513468,v1:172.21.15.134:6825/3073513468]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-13T18:18:45.525 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:45 smithi204 ceph-mon[34777]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-13T18:18:45.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:45 smithi134 ceph-mon[34283]: from='osd.10 [v2:172.21.15.134:6824/3073513468,v1:172.21.15.134:6825/3073513468]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-13T18:18:45.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:45 smithi134 ceph-mon[34283]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-13T18:18:45.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:45 smithi114 ceph-mon[30897]: from='osd.10 [v2:172.21.15.134:6824/3073513468,v1:172.21.15.134:6825/3073513468]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-13T18:18:45.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:45 smithi114 ceph-mon[30897]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-13T18:18:46.575 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:46 smithi204 ceph-mon[34777]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-13T18:18:46.576 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:46 smithi204 ceph-mon[34777]: osdmap e70: 11 total, 10 up, 11 in 2024-09-13T18:18:46.576 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:46 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T18:18:46.576 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:46 smithi204 ceph-mon[34777]: from='osd.10 [v2:172.21.15.134:6824/3073513468,v1:172.21.15.134:6825/3073513468]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-13T18:18:46.576 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:46 smithi204 ceph-mon[34777]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-13T18:18:46.576 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:46 smithi204 ceph-mon[34777]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T18:18:46.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:46 smithi114 ceph-mon[30897]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-13T18:18:46.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:46 smithi114 ceph-mon[30897]: osdmap e70: 11 total, 10 up, 11 in 2024-09-13T18:18:46.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:46 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T18:18:46.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:46 smithi114 ceph-mon[30897]: from='osd.10 [v2:172.21.15.134:6824/3073513468,v1:172.21.15.134:6825/3073513468]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-13T18:18:46.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:46 smithi114 ceph-mon[30897]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-13T18:18:46.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:46 smithi114 ceph-mon[30897]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T18:18:46.824 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:46 smithi134 ceph-mon[34283]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-13T18:18:46.824 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:46 smithi134 ceph-mon[34283]: osdmap e70: 11 total, 10 up, 11 in 2024-09-13T18:18:46.824 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:46 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T18:18:46.824 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:46 smithi134 ceph-mon[34283]: from='osd.10 [v2:172.21.15.134:6824/3073513468,v1:172.21.15.134:6825/3073513468]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-13T18:18:46.824 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:46 smithi134 ceph-mon[34283]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-13T18:18:46.824 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:46 smithi134 ceph-mon[34283]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T18:18:46.825 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:18:46 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:18:46.460+0000 7fe0b48b1640 -1 osd.10 0 waiting for initial osdmap 2024-09-13T18:18:46.825 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:18:46 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:18:46.473+0000 7fe0b06da640 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T18:18:46.954 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-09-13T18:18:47.043 INFO:teuthology.orchestra.run.smithi204.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-09-13T18:18:47.056 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10+0 records in 2024-09-13T18:18:47.057 INFO:teuthology.orchestra.run.smithi204.stderr:10+0 records out 2024-09-13T18:18:47.057 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114471 s, 916 MB/s 2024-09-13T18:18:47.057 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping successful for: 2024-09-13T18:18:47.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:47 smithi204 ceph-mon[34777]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-13T18:18:47.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:47 smithi204 ceph-mon[34777]: osdmap e71: 11 total, 10 up, 11 in 2024-09-13T18:18:47.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:47 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T18:18:47.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:47 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T18:18:47.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:47 smithi114 ceph-mon[30897]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-13T18:18:47.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:47 smithi114 ceph-mon[30897]: osdmap e71: 11 total, 10 up, 11 in 2024-09-13T18:18:47.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:47 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T18:18:47.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:47 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T18:18:47.795 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi204:vg_nvme/lv_1 2024-09-13T18:18:47.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:47 smithi134 ceph-mon[34283]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-13T18:18:47.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:47 smithi134 ceph-mon[34283]: osdmap e71: 11 total, 10 up, 11 in 2024-09-13T18:18:47.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:47 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T18:18:47.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:47 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T18:18:47.996 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T18:18:48.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:48 smithi204 ceph-mon[34777]: purged_snaps scrub starts 2024-09-13T18:18:48.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:48 smithi204 ceph-mon[34777]: purged_snaps scrub ok 2024-09-13T18:18:48.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:48 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T18:18:48.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:48 smithi204 ceph-mon[34777]: osd.10 [v2:172.21.15.134:6824/3073513468,v1:172.21.15.134:6825/3073513468] boot 2024-09-13T18:18:48.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:48 smithi204 ceph-mon[34777]: osdmap e72: 11 total, 11 up, 11 in 2024-09-13T18:18:48.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:48 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T18:18:48.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:48 smithi204 ceph-mon[34777]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T18:18:48.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:48 smithi204 ceph-mon[34777]: Detected new or changed devices on smithi134 2024-09-13T18:18:48.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:48 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:48.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:48 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:48.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:48 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:18:48.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:48 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:18:48.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:48 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:18:48.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:48 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:18:48.696 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:48 smithi204 ceph-mon[34777]: Adjusting osd_memory_target on smithi134 to 4272M 2024-09-13T18:18:48.696 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:48 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:48.696 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:48 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:18:48.696 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:48 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:18:48.696 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:48 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:48.696 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:48 smithi204 ceph-mon[34777]: osdmap e73: 11 total, 11 up, 11 in 2024-09-13T18:18:48.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:48 smithi114 ceph-mon[30897]: purged_snaps scrub starts 2024-09-13T18:18:48.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:48 smithi114 ceph-mon[30897]: purged_snaps scrub ok 2024-09-13T18:18:48.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:48 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T18:18:48.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:48 smithi114 ceph-mon[30897]: osd.10 [v2:172.21.15.134:6824/3073513468,v1:172.21.15.134:6825/3073513468] boot 2024-09-13T18:18:48.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:48 smithi114 ceph-mon[30897]: osdmap e72: 11 total, 11 up, 11 in 2024-09-13T18:18:48.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:48 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T18:18:48.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:48 smithi114 ceph-mon[30897]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T18:18:48.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:48 smithi114 ceph-mon[30897]: Detected new or changed devices on smithi134 2024-09-13T18:18:48.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:48 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:48.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:48 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:48.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:48 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:18:48.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:48 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:18:48.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:48 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:18:48.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:48 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:18:48.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:48 smithi114 ceph-mon[30897]: Adjusting osd_memory_target on smithi134 to 4272M 2024-09-13T18:18:48.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:48 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:48.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:48 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:18:48.722 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:48 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:18:48.722 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:48 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:48.722 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:48 smithi114 ceph-mon[30897]: osdmap e73: 11 total, 11 up, 11 in 2024-09-13T18:18:48.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:48 smithi134 ceph-mon[34283]: purged_snaps scrub starts 2024-09-13T18:18:48.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:48 smithi134 ceph-mon[34283]: purged_snaps scrub ok 2024-09-13T18:18:48.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:48 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T18:18:48.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:48 smithi134 ceph-mon[34283]: osd.10 [v2:172.21.15.134:6824/3073513468,v1:172.21.15.134:6825/3073513468] boot 2024-09-13T18:18:48.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:48 smithi134 ceph-mon[34283]: osdmap e72: 11 total, 11 up, 11 in 2024-09-13T18:18:48.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:48 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T18:18:48.870 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:48 smithi134 ceph-mon[34283]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-13T18:18:48.870 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:48 smithi134 ceph-mon[34283]: Detected new or changed devices on smithi134 2024-09-13T18:18:48.870 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:48 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:48.870 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:48 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:48.870 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:48 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:18:48.870 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:48 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:18:48.870 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:48 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:18:48.870 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:48 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:18:48.870 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:48 smithi134 ceph-mon[34283]: Adjusting osd_memory_target on smithi134 to 4272M 2024-09-13T18:18:48.870 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:48 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:48.870 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:48 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:18:48.870 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:48 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:18:48.870 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:48 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:18:48.870 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:48 smithi134 ceph-mon[34283]: osdmap e73: 11 total, 11 up, 11 in 2024-09-13T18:18:49.693 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:18:49.672+0000 7f6401aa4640 1 -- 172.21.15.204:0/4253186035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63fc0fe480 msgr2=0x7f63fc0fe880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:49.693 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:18:49.672+0000 7f6401aa4640 1 --2- 172.21.15.204:0/4253186035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63fc0fe480 0x7f63fc0fe880 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f63ec009900 tx=0x7f63ec02a410 comp rx=0 tx=0).stop 2024-09-13T18:18:49.693 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:18:49.672+0000 7f6401aa4640 1 -- 172.21.15.204:0/4253186035 shutdown_connections 2024-09-13T18:18:49.693 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:18:49.672+0000 7f6401aa4640 1 --2- 172.21.15.204:0/4253186035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63fc100040 0x7f63fc1068d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:49.694 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:18:49.672+0000 7f6401aa4640 1 --2- 172.21.15.204:0/4253186035 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f63fc0ff680 0x7f63fc0ffb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:49.694 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:18:49.672+0000 7f6401aa4640 1 --2- 172.21.15.204:0/4253186035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63fc0fe480 0x7f63fc0fe880 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:49.694 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:18:49.672+0000 7f6401aa4640 1 -- 172.21.15.204:0/4253186035 >> 172.21.15.204:0/4253186035 conn(0x7f63fc0f9c10 msgr2=0x7f63fc0fc050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:18:49.694 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:18:49.672+0000 7f6401aa4640 1 -- 172.21.15.204:0/4253186035 shutdown_connections 2024-09-13T18:18:49.694 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:18:49.672+0000 7f6401aa4640 1 -- 172.21.15.204:0/4253186035 wait complete. 2024-09-13T18:18:49.694 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:18:49.672+0000 7f6401aa4640 1 Processor -- start 2024-09-13T18:18:49.694 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:18:49.672+0000 7f6401aa4640 1 -- start start 2024-09-13T18:18:49.694 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:18:49.673+0000 7f6401aa4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63fc0fe480 0x7f63fc195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:49.695 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:18:49.673+0000 7f6401aa4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f63fc0ff680 0x7f63fc196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:49.695 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:18:49.673+0000 7f6401aa4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63fc100040 0x7f63fc19d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:49.695 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:18:49.673+0000 7f6401aa4640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63fc19d7d0 con 0x7f63fc0ff680 2024-09-13T18:18:49.695 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:18:49.673+0000 7f6401aa4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63fc19d940 con 0x7f63fc0fe480 2024-09-13T18:18:49.695 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:18:49.673+0000 7f6401aa4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63fc19db00 con 0x7f63fc100040 2024-09-13T18:18:49.695 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:18:49.673+0000 7f63fbfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63fc100040 0x7f63fc19d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:49.695 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:18:49.673+0000 7f63fbfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63fc100040 0x7f63fc19d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:51774/0 (socket says 172.21.15.204:51774) 2024-09-13T18:18:49.695 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:18:49.673+0000 7f63fbfff640 1 -- 172.21.15.204:0/2666972221 learned_addr learned my addr 172.21.15.204:0/2666972221 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-13T18:18:49.695 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:18:49.673+0000 7f63f3fff640 1 --2- 172.21.15.204:0/2666972221 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f63fc0ff680 0x7f63fc196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:49.695 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:18:49.673+0000 7f63fb7fe640 1 --2- 172.21.15.204:0/2666972221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63fc0fe480 0x7f63fc195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:49.695 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:18:49.673+0000 7f63fbfff640 1 -- 172.21.15.204:0/2666972221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63fc0fe480 msgr2=0x7f63fc195d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:49.695 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:18:49.673+0000 7f63fbfff640 1 --2- 172.21.15.204:0/2666972221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63fc0fe480 0x7f63fc195d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:49.696 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:18:49.673+0000 7f63fbfff640 1 -- 172.21.15.204:0/2666972221 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f63fc0ff680 msgr2=0x7f63fc196270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:49.696 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:18:49.673+0000 7f63fbfff640 1 --2- 172.21.15.204:0/2666972221 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f63fc0ff680 0x7f63fc196270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:49.696 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:18:49.673+0000 7f63fbfff640 1 -- 172.21.15.204:0/2666972221 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63ec009560 con 0x7f63fc100040 2024-09-13T18:18:49.696 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:18:49.673+0000 7f63fbfff640 1 --2- 172.21.15.204:0/2666972221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63fc100040 0x7f63fc19d260 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f63f400cce0 tx=0x7f63f400a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:49.696 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:18:49.673+0000 7f63f97fa640 1 -- 172.21.15.204:0/2666972221 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f400f450 con 0x7f63fc100040 2024-09-13T18:18:49.696 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:18:49.673+0000 7f6401aa4640 1 -- 172.21.15.204:0/2666972221 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63fc19dde0 con 0x7f63fc100040 2024-09-13T18:18:49.696 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:18:49.673+0000 7f63f97fa640 1 -- 172.21.15.204:0/2666972221 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f63f400fa40 con 0x7f63fc100040 2024-09-13T18:18:49.696 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:18:49.673+0000 7f63f97fa640 1 -- 172.21.15.204:0/2666972221 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f40189a0 con 0x7f63fc100040 2024-09-13T18:18:49.696 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:18:49.673+0000 7f6401aa4640 1 -- 172.21.15.204:0/2666972221 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63fc19e250 con 0x7f63fc100040 2024-09-13T18:18:49.696 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:18:49.674+0000 7f6401aa4640 1 -- 172.21.15.204:0/2666972221 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63fc1093c0 con 0x7f63fc100040 2024-09-13T18:18:49.696 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:18:49.676+0000 7f63f97fa640 1 -- 172.21.15.204:0/2666972221 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f63f400fbb0 con 0x7f63fc100040 2024-09-13T18:18:49.696 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:18:49.676+0000 7f63f97fa640 1 --2- 172.21.15.204:0/2666972221 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f63e80a1080 0x7f63e80a3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:49.696 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:18:49.676+0000 7f63f97fa640 1 -- 172.21.15.204:0/2666972221 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6723+0+0 (secure 0 0 0) 0x7f63f40cc710 con 0x7f63fc100040 2024-09-13T18:18:49.697 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:18:49.677+0000 7f63fb7fe640 1 --2- 172.21.15.204:0/2666972221 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f63e80a1080 0x7f63e80a3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:49.697 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:18:49.677+0000 7f63fb7fe640 1 --2- 172.21.15.204:0/2666972221 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f63e80a1080 0x7f63e80a3540 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f63ec0057c0 tx=0x7f63ec000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:49.697 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:18:49.679+0000 7f63f97fa640 1 -- 172.21.15.204:0/2666972221 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f63f4096560 con 0x7f63fc100040 2024-09-13T18:18:49.853 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:18:49.850+0000 7f6401aa4640 1 -- 172.21.15.204:0/2666972221 --> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f63fc109550 con 0x7f63e80a1080 2024-09-13T18:18:50.466 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:50 smithi204 ceph-mon[34777]: osdmap e74: 11 total, 11 up, 11 in 2024-09-13T18:18:50.466 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:50 smithi204 ceph-mon[34777]: from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:18:50.466 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:50 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T18:18:50.466 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:50 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T18:18:50.467 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:50 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:18:50.467 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:50 smithi204 ceph-mon[34777]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-13T18:18:50.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:50 smithi134 ceph-mon[34283]: osdmap e74: 11 total, 11 up, 11 in 2024-09-13T18:18:50.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:50 smithi134 ceph-mon[34283]: from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:18:50.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:50 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T18:18:50.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:50 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T18:18:50.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:50 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:18:50.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:50 smithi134 ceph-mon[34283]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-13T18:18:50.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:50 smithi114 ceph-mon[30897]: osdmap e74: 11 total, 11 up, 11 in 2024-09-13T18:18:50.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:50 smithi114 ceph-mon[30897]: from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:18:50.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:50 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-13T18:18:50.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:50 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-13T18:18:50.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:50 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:18:50.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:50 smithi114 ceph-mon[30897]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-13T18:18:51.327 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:51 smithi204 ceph-mon[34777]: osdmap e75: 11 total, 11 up, 11 in 2024-09-13T18:18:51.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:51 smithi134 ceph-mon[34283]: osdmap e75: 11 total, 11 up, 11 in 2024-09-13T18:18:51.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:51 smithi114 ceph-mon[30897]: osdmap e75: 11 total, 11 up, 11 in 2024-09-13T18:18:52.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:52 smithi204 ceph-mon[34777]: osdmap e76: 11 total, 11 up, 11 in 2024-09-13T18:18:52.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:52 smithi204 ceph-mon[34777]: pgmap v229: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-13T18:18:52.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:52 smithi134 ceph-mon[34283]: osdmap e76: 11 total, 11 up, 11 in 2024-09-13T18:18:52.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:52 smithi134 ceph-mon[34283]: pgmap v229: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-13T18:18:52.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:52 smithi114 ceph-mon[30897]: osdmap e76: 11 total, 11 up, 11 in 2024-09-13T18:18:52.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:52 smithi114 ceph-mon[30897]: pgmap v229: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-13T18:18:53.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:53 smithi134 ceph-mon[34283]: from='client.? 172.21.15.204:0/906582410' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71dd794d-326c-477f-8d02-ee32bff1350d"}]: dispatch 2024-09-13T18:18:53.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:53 smithi134 ceph-mon[34283]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71dd794d-326c-477f-8d02-ee32bff1350d"}]: dispatch 2024-09-13T18:18:53.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:53 smithi134 ceph-mon[34283]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "71dd794d-326c-477f-8d02-ee32bff1350d"}]': finished 2024-09-13T18:18:53.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:53 smithi134 ceph-mon[34283]: osdmap e77: 12 total, 11 up, 12 in 2024-09-13T18:18:53.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:53 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T18:18:53.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:53 smithi134 ceph-mon[34283]: from='client.? 172.21.15.204:0/3883909327' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T18:18:53.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:53 smithi204 ceph-mon[34777]: from='client.? 172.21.15.204:0/906582410' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71dd794d-326c-477f-8d02-ee32bff1350d"}]: dispatch 2024-09-13T18:18:53.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:53 smithi204 ceph-mon[34777]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71dd794d-326c-477f-8d02-ee32bff1350d"}]: dispatch 2024-09-13T18:18:53.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:53 smithi204 ceph-mon[34777]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "71dd794d-326c-477f-8d02-ee32bff1350d"}]': finished 2024-09-13T18:18:53.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:53 smithi204 ceph-mon[34777]: osdmap e77: 12 total, 11 up, 12 in 2024-09-13T18:18:53.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:53 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T18:18:53.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:53 smithi204 ceph-mon[34777]: from='client.? 172.21.15.204:0/3883909327' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T18:18:53.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:53 smithi114 ceph-mon[30897]: from='client.? 172.21.15.204:0/906582410' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71dd794d-326c-477f-8d02-ee32bff1350d"}]: dispatch 2024-09-13T18:18:53.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:53 smithi114 ceph-mon[30897]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71dd794d-326c-477f-8d02-ee32bff1350d"}]: dispatch 2024-09-13T18:18:53.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:53 smithi114 ceph-mon[30897]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "71dd794d-326c-477f-8d02-ee32bff1350d"}]': finished 2024-09-13T18:18:53.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:53 smithi114 ceph-mon[30897]: osdmap e77: 12 total, 11 up, 12 in 2024-09-13T18:18:53.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:53 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T18:18:53.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:53 smithi114 ceph-mon[30897]: from='client.? 172.21.15.204:0/3883909327' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-13T18:18:54.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:54 smithi134 ceph-mon[34283]: pgmap v231: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-09-13T18:18:54.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:54 smithi204 ceph-mon[34777]: pgmap v231: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-09-13T18:18:54.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:54 smithi114 ceph-mon[30897]: pgmap v231: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-09-13T18:18:57.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:56 smithi204 ceph-mon[34777]: pgmap v232: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-13T18:18:57.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:56 smithi114 ceph-mon[30897]: pgmap v232: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-13T18:18:57.368 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:56 smithi134 ceph-mon[34283]: pgmap v232: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-13T18:18:59.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:58 smithi204 ceph-mon[34777]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-09-13T18:18:59.219 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:58 smithi114 ceph-mon[30897]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-09-13T18:18:59.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:58 smithi134 ceph-mon[34283]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-09-13T18:19:00.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:59 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-13T18:19:00.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:18:59 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:19:00.250 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:59 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-13T18:19:00.250 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:18:59 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:19:00.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:59 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-13T18:19:00.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:18:59 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:19:01.029 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:00 smithi204 ceph-mon[34777]: Deploying daemon osd.11 on smithi204 2024-09-13T18:19:01.029 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:00 smithi204 ceph-mon[34777]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-13T18:19:01.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:00 smithi114 ceph-mon[30897]: Deploying daemon osd.11 on smithi204 2024-09-13T18:19:01.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:00 smithi114 ceph-mon[30897]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-13T18:19:01.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:00 smithi134 ceph-mon[34283]: Deploying daemon osd.11 on smithi204 2024-09-13T18:19:01.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:00 smithi134 ceph-mon[34283]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-13T18:19:03.137 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:02 smithi204 ceph-mon[34777]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-13T18:19:03.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:02 smithi114 ceph-mon[30897]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-13T18:19:03.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:02 smithi134 ceph-mon[34283]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-13T18:19:05.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:04 smithi114 ceph-mon[30897]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-13T18:19:05.368 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:04 smithi134 ceph-mon[34283]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-13T18:19:05.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:04 smithi204 ceph-mon[34777]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-13T18:19:06.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:05 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:19:06.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:05 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:06.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:05 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:06.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:05 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:19:06.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:05 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:06.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:05 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:06.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:05 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:19:06.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:05 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:06.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:05 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:07.174 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:07 smithi204 ceph-mon[34777]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-13T18:19:07.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:07 smithi114 ceph-mon[30897]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-13T18:19:07.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:07 smithi134 ceph-mon[34283]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-13T18:19:08.933 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 11 on host 'smithi204' 2024-09-13T18:19:08.934 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:08.931+0000 7f63f97fa640 1 -- 172.21.15.204:0/2666972221 <== mgr.14148 v2:172.21.15.114:6800/3396185541 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f63fc109550 con 0x7f63e80a1080 2024-09-13T18:19:08.936 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:08.933+0000 7f6401aa4640 1 -- 172.21.15.204:0/2666972221 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f63e80a1080 msgr2=0x7f63e80a3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:08.936 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:08.933+0000 7f6401aa4640 1 --2- 172.21.15.204:0/2666972221 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f63e80a1080 0x7f63e80a3540 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f63ec0057c0 tx=0x7f63ec000bc0 comp rx=0 tx=0).stop 2024-09-13T18:19:08.936 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:08.933+0000 7f6401aa4640 1 -- 172.21.15.204:0/2666972221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63fc100040 msgr2=0x7f63fc19d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:08.936 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:08.933+0000 7f6401aa4640 1 --2- 172.21.15.204:0/2666972221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63fc100040 0x7f63fc19d260 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f63f400cce0 tx=0x7f63f400a620 comp rx=0 tx=0).stop 2024-09-13T18:19:08.936 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:08.934+0000 7f6401aa4640 1 -- 172.21.15.204:0/2666972221 shutdown_connections 2024-09-13T18:19:08.936 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:08.934+0000 7f6401aa4640 1 --2- 172.21.15.204:0/2666972221 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f63e80a1080 0x7f63e80a3540 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:08.936 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:08.934+0000 7f6401aa4640 1 --2- 172.21.15.204:0/2666972221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63fc100040 0x7f63fc19d260 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:08.936 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:08.934+0000 7f6401aa4640 1 --2- 172.21.15.204:0/2666972221 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f63fc0ff680 0x7f63fc196270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:08.937 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:08.934+0000 7f6401aa4640 1 --2- 172.21.15.204:0/2666972221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63fc0fe480 0x7f63fc195d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:08.937 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:08.934+0000 7f6401aa4640 1 -- 172.21.15.204:0/2666972221 >> 172.21.15.204:0/2666972221 conn(0x7f63fc0f9c10 msgr2=0x7f63fc104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:08.937 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:08.934+0000 7f6401aa4640 1 -- 172.21.15.204:0/2666972221 shutdown_connections 2024-09-13T18:19:08.937 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:08.934+0000 7f6401aa4640 1 -- 172.21.15.204:0/2666972221 wait complete. 2024-09-13T18:19:09.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:09 smithi204 ceph-mon[34777]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-13T18:19:09.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:09 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:09.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:09 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:09.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:09 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:19:09.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:09 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:19:09.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:09 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:09.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:09 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:19:09.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:09 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:09.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:09 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:09.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:09 smithi204 ceph-mon[34777]: from='osd.11 [v2:172.21.15.204:6824/3239521765,v1:172.21.15.204:6825/3239521765]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-13T18:19:09.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:09 smithi204 ceph-mon[34777]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-13T18:19:09.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:09 smithi114 ceph-mon[30897]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-13T18:19:09.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:09 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:09.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:09 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:09.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:09 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:19:09.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:09 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:19:09.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:09 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:09.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:09 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:19:09.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:09 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:09.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:09 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:09.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:09 smithi114 ceph-mon[30897]: from='osd.11 [v2:172.21.15.204:6824/3239521765,v1:172.21.15.204:6825/3239521765]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-13T18:19:09.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:09 smithi114 ceph-mon[30897]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-13T18:19:09.584 DEBUG:teuthology.orchestra.run.smithi204:osd.11> sudo journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@osd.11.service 2024-09-13T18:19:09.587 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-13T18:19:09.588 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-13T18:19:09.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:09 smithi134 ceph-mon[34283]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-13T18:19:09.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:09 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:09.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:09 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:09.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:09 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:19:09.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:09 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:19:09.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:09 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:09.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:09 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:19:09.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:09 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:09.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:09 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:09.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:09 smithi134 ceph-mon[34283]: from='osd.11 [v2:172.21.15.204:6824/3239521765,v1:172.21.15.204:6825/3239521765]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-13T18:19:09.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:09 smithi134 ceph-mon[34283]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-13T18:19:09.797 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:19:11.056 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:10 smithi204 ceph-mon[34777]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-13T18:19:11.056 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:10 smithi204 ceph-mon[34777]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-13T18:19:11.056 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:10 smithi204 ceph-mon[34777]: osdmap e78: 12 total, 11 up, 12 in 2024-09-13T18:19:11.057 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:10 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T18:19:11.057 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:10 smithi204 ceph-mon[34777]: from='osd.11 [v2:172.21.15.204:6824/3239521765,v1:172.21.15.204:6825/3239521765]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-13T18:19:11.057 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:10 smithi204 ceph-mon[34777]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-13T18:19:11.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:10 smithi114 ceph-mon[30897]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-13T18:19:11.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:10 smithi114 ceph-mon[30897]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-13T18:19:11.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:10 smithi114 ceph-mon[30897]: osdmap e78: 12 total, 11 up, 12 in 2024-09-13T18:19:11.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:10 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T18:19:11.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:10 smithi114 ceph-mon[30897]: from='osd.11 [v2:172.21.15.204:6824/3239521765,v1:172.21.15.204:6825/3239521765]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-13T18:19:11.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:10 smithi114 ceph-mon[30897]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-13T18:19:11.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:10 smithi134 ceph-mon[34283]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-13T18:19:11.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:10 smithi134 ceph-mon[34283]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-13T18:19:11.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:10 smithi134 ceph-mon[34283]: osdmap e78: 12 total, 11 up, 12 in 2024-09-13T18:19:11.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:10 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T18:19:11.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:10 smithi134 ceph-mon[34283]: from='osd.11 [v2:172.21.15.204:6824/3239521765,v1:172.21.15.204:6825/3239521765]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-13T18:19:11.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:10 smithi134 ceph-mon[34283]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-13T18:19:11.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:11.686+0000 7fc7198b5640 1 -- 172.21.15.114:0/1304031056 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc714105ad0 msgr2=0x7fc714107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:11.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:11.686+0000 7fc7198b5640 1 --2- 172.21.15.114:0/1304031056 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc714105ad0 0x7fc714107f20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc70c009900 tx=0x7fc70c02a410 comp rx=0 tx=0).stop 2024-09-13T18:19:11.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:11.686+0000 7fc7198b5640 1 -- 172.21.15.114:0/1304031056 shutdown_connections 2024-09-13T18:19:11.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:11.686+0000 7fc7198b5640 1 --2- 172.21.15.114:0/1304031056 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc714105ad0 0x7fc714107f20 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:11.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:11.686+0000 7fc7198b5640 1 --2- 172.21.15.114:0/1304031056 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7141031a0 0x7fc714105590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:11.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:11.686+0000 7fc7198b5640 1 --2- 172.21.15.114:0/1304031056 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc714100870 0x7fc714102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:11.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:11.686+0000 7fc7198b5640 1 -- 172.21.15.114:0/1304031056 >> 172.21.15.114:0/1304031056 conn(0x7fc7140fa490 msgr2=0x7fc7140fc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:11.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:11.687+0000 7fc7198b5640 1 -- 172.21.15.114:0/1304031056 shutdown_connections 2024-09-13T18:19:11.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:11.687+0000 7fc7198b5640 1 -- 172.21.15.114:0/1304031056 wait complete. 2024-09-13T18:19:11.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:11.687+0000 7fc7198b5640 1 Processor -- start 2024-09-13T18:19:11.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:11.687+0000 7fc7198b5640 1 -- start start 2024-09-13T18:19:11.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:11.688+0000 7fc7198b5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc714100870 0x7fc7141001d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:11.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:11.688+0000 7fc7198b5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7141031a0 0x7fc7140fe820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:11.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:11.688+0000 7fc7198b5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc714105ad0 0x7fc7140fed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:11.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:11.688+0000 7fc7198b5640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7140ff4f0 con 0x7fc7141031a0 2024-09-13T18:19:11.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:11.688+0000 7fc7198b5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7140ff660 con 0x7fc714105ad0 2024-09-13T18:19:11.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:11.688+0000 7fc7198b5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc71419fc40 con 0x7fc714100870 2024-09-13T18:19:11.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:11.688+0000 7fc7127fc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7141031a0 0x7fc7140fe820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:11.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:11.688+0000 7fc712ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc714100870 0x7fc7141001d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:11.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:11.688+0000 7fc7127fc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7141031a0 0x7fc7140fe820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60792/0 (socket says 172.21.15.114:60792) 2024-09-13T18:19:11.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:11.688+0000 7fc7127fc640 1 -- 172.21.15.114:0/999877193 learned_addr learned my addr 172.21.15.114:0/999877193 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:19:11.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:11.688+0000 7fc7137fe640 1 --2- 172.21.15.114:0/999877193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc714105ad0 0x7fc7140fed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:11.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:11.688+0000 7fc7127fc640 1 -- 172.21.15.114:0/999877193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc714100870 msgr2=0x7fc7141001d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:11.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:11.688+0000 7fc7127fc640 1 --2- 172.21.15.114:0/999877193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc714100870 0x7fc7141001d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:11.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:11.688+0000 7fc7127fc640 1 -- 172.21.15.114:0/999877193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc714105ad0 msgr2=0x7fc7140fed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:11.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:11.688+0000 7fc7127fc640 1 --2- 172.21.15.114:0/999877193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc714105ad0 0x7fc7140fed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:11.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:11.688+0000 7fc7127fc640 1 -- 172.21.15.114:0/999877193 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc70c009560 con 0x7fc7141031a0 2024-09-13T18:19:11.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:11.688+0000 7fc7127fc640 1 --2- 172.21.15.114:0/999877193 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7141031a0 0x7fc7140fe820 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc70800c9d0 tx=0x7fc70800ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:11.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:11.688+0000 7fc7188b3640 1 -- 172.21.15.114:0/999877193 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc708004440 con 0x7fc7141031a0 2024-09-13T18:19:11.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:11.688+0000 7fc7188b3640 1 -- 172.21.15.114:0/999877193 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc708007cf0 con 0x7fc7141031a0 2024-09-13T18:19:11.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:11.688+0000 7fc7198b5640 1 -- 172.21.15.114:0/999877193 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc71419fef0 con 0x7fc7141031a0 2024-09-13T18:19:11.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:11.688+0000 7fc7188b3640 1 -- 172.21.15.114:0/999877193 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc708003e50 con 0x7fc7141031a0 2024-09-13T18:19:11.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:11.689+0000 7fc7198b5640 1 -- 172.21.15.114:0/999877193 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7141a03b0 con 0x7fc7141031a0 2024-09-13T18:19:11.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:11.689+0000 7fc7198b5640 1 -- 172.21.15.114:0/999877193 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7140fbf80 con 0x7fc7141031a0 2024-09-13T18:19:11.695 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:19:11 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:19:11.347+0000 7f4301452640 -1 osd.11 0 waiting for initial osdmap 2024-09-13T18:19:11.695 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:19:11 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:19:11.364+0000 7f42f9273640 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T18:19:11.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:11.691+0000 7fc7188b3640 1 -- 172.21.15.114:0/999877193 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fc708007860 con 0x7fc7141031a0 2024-09-13T18:19:11.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:11.691+0000 7fc7188b3640 1 --2- 172.21.15.114:0/999877193 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fc6fc0a1100 0x7fc6fc0a35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:11.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:11.691+0000 7fc712ffd640 1 --2- 172.21.15.114:0/999877193 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fc6fc0a1100 0x7fc6fc0a35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:11.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:11.691+0000 7fc7188b3640 1 -- 172.21.15.114:0/999877193 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6847+0+0 (secure 0 0 0) 0x7fc7080cc3b0 con 0x7fc7141031a0 2024-09-13T18:19:11.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:11.692+0000 7fc712ffd640 1 --2- 172.21.15.114:0/999877193 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fc6fc0a1100 0x7fc6fc0a35c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc704005fd0 tx=0x7fc704009680 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:11.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:11.695+0000 7fc7188b3640 1 -- 172.21.15.114:0/999877193 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fc708096100 con 0x7fc7141031a0 2024-09-13T18:19:11.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:11.858+0000 7fc7198b5640 1 -- 172.21.15.114:0/999877193 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fc71410b760 con 0x7fc7141031a0 2024-09-13T18:19:11.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:11.858+0000 7fc7188b3640 1 -- 172.21.15.114:0/999877193 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v79) v1 ==== 74+0+133 (secure 0 0 0) 0x7fc708007e60 con 0x7fc7141031a0 2024-09-13T18:19:11.861 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:19:11.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:11.860+0000 7fc7198b5640 1 -- 172.21.15.114:0/999877193 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fc6fc0a1100 msgr2=0x7fc6fc0a35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:11.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:11.860+0000 7fc7198b5640 1 --2- 172.21.15.114:0/999877193 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fc6fc0a1100 0x7fc6fc0a35c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc704005fd0 tx=0x7fc704009680 comp rx=0 tx=0).stop 2024-09-13T18:19:11.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:11.860+0000 7fc7198b5640 1 -- 172.21.15.114:0/999877193 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7141031a0 msgr2=0x7fc7140fe820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:11.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:11.860+0000 7fc7198b5640 1 --2- 172.21.15.114:0/999877193 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7141031a0 0x7fc7140fe820 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc70800c9d0 tx=0x7fc70800ce90 comp rx=0 tx=0).stop 2024-09-13T18:19:11.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:11.860+0000 7fc7198b5640 1 -- 172.21.15.114:0/999877193 shutdown_connections 2024-09-13T18:19:11.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:11.860+0000 7fc7198b5640 1 --2- 172.21.15.114:0/999877193 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fc6fc0a1100 0x7fc6fc0a35c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:11.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:11.860+0000 7fc7198b5640 1 --2- 172.21.15.114:0/999877193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc714105ad0 0x7fc7140fed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:11.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:11.860+0000 7fc7198b5640 1 --2- 172.21.15.114:0/999877193 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7141031a0 0x7fc7140fe820 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:11.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:11.860+0000 7fc7198b5640 1 --2- 172.21.15.114:0/999877193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc714100870 0x7fc7141001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:11.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:11.860+0000 7fc7198b5640 1 -- 172.21.15.114:0/999877193 >> 172.21.15.114:0/999877193 conn(0x7fc7140fa490 msgr2=0x7fc7141011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:11.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:11.860+0000 7fc7198b5640 1 -- 172.21.15.114:0/999877193 shutdown_connections 2024-09-13T18:19:11.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:11.860+0000 7fc7198b5640 1 -- 172.21.15.114:0/999877193 wait complete. 2024-09-13T18:19:12.180 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:11 smithi114 ceph-mon[30897]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-09-13T18:19:12.180 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:11 smithi114 ceph-mon[30897]: osdmap e79: 12 total, 11 up, 12 in 2024-09-13T18:19:12.180 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:11 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T18:19:12.181 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:11 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T18:19:12.181 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:11 smithi114 ceph-mon[30897]: from='osd.11 ' entity='osd.11' 2024-09-13T18:19:12.181 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:11 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/999877193' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-13T18:19:12.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:11 smithi204 ceph-mon[34777]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-09-13T18:19:12.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:11 smithi204 ceph-mon[34777]: osdmap e79: 12 total, 11 up, 12 in 2024-09-13T18:19:12.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:11 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T18:19:12.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:11 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T18:19:12.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:11 smithi204 ceph-mon[34777]: from='osd.11 ' entity='osd.11' 2024-09-13T18:19:12.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:11 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/999877193' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-13T18:19:12.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:11 smithi134 ceph-mon[34283]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-09-13T18:19:12.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:11 smithi134 ceph-mon[34283]: osdmap e79: 12 total, 11 up, 12 in 2024-09-13T18:19:12.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:11 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T18:19:12.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:11 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T18:19:12.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:11 smithi134 ceph-mon[34283]: from='osd.11 ' entity='osd.11' 2024-09-13T18:19:12.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:11 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/999877193' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-13T18:19:12.502 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":79,"num_osds":12,"num_up_osds":11,"osd_up_since":1726251527,"num_in_osds":12,"osd_in_since":1726251532,"num_remapped_pgs":0} 2024-09-13T18:19:13.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:12 smithi204 ceph-mon[34777]: purged_snaps scrub starts 2024-09-13T18:19:13.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:12 smithi204 ceph-mon[34777]: purged_snaps scrub ok 2024-09-13T18:19:13.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:12 smithi204 ceph-mon[34777]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-13T18:19:13.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:12 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T18:19:13.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:12 smithi204 ceph-mon[34777]: osd.11 [v2:172.21.15.204:6824/3239521765,v1:172.21.15.204:6825/3239521765] boot 2024-09-13T18:19:13.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:12 smithi204 ceph-mon[34777]: osdmap e80: 12 total, 12 up, 12 in 2024-09-13T18:19:13.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:12 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T18:19:13.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:12 smithi114 ceph-mon[30897]: purged_snaps scrub starts 2024-09-13T18:19:13.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:12 smithi114 ceph-mon[30897]: purged_snaps scrub ok 2024-09-13T18:19:13.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:12 smithi114 ceph-mon[30897]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-13T18:19:13.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:12 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T18:19:13.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:12 smithi114 ceph-mon[30897]: osd.11 [v2:172.21.15.204:6824/3239521765,v1:172.21.15.204:6825/3239521765] boot 2024-09-13T18:19:13.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:12 smithi114 ceph-mon[30897]: osdmap e80: 12 total, 12 up, 12 in 2024-09-13T18:19:13.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:12 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T18:19:13.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:12 smithi134 ceph-mon[34283]: purged_snaps scrub starts 2024-09-13T18:19:13.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:12 smithi134 ceph-mon[34283]: purged_snaps scrub ok 2024-09-13T18:19:13.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:12 smithi134 ceph-mon[34283]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-13T18:19:13.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:12 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T18:19:13.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:12 smithi134 ceph-mon[34283]: osd.11 [v2:172.21.15.204:6824/3239521765,v1:172.21.15.204:6825/3239521765] boot 2024-09-13T18:19:13.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:12 smithi134 ceph-mon[34283]: osdmap e80: 12 total, 12 up, 12 in 2024-09-13T18:19:13.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:12 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T18:19:13.504 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-13T18:19:13.711 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:19:14.102 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:14 smithi114 ceph-mon[30897]: Detected new or changed devices on smithi204 2024-09-13T18:19:14.102 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:14 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:14.103 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:14 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:14.103 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:14 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:19:14.103 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:14 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:19:14.103 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:14 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:19:14.103 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:14 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:19:14.103 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:14 smithi114 ceph-mon[30897]: Adjusting osd_memory_target on smithi204 to 4272M 2024-09-13T18:19:14.103 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:14 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:14.104 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:14 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:19:14.104 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:14 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:19:14.104 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:14 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:14.104 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:14 smithi114 ceph-mon[30897]: osdmap e81: 12 total, 12 up, 12 in 2024-09-13T18:19:14.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:14 smithi134 ceph-mon[34283]: Detected new or changed devices on smithi204 2024-09-13T18:19:14.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:14 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:14.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:14 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:14.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:14 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:19:14.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:14 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:19:14.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:14 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:19:14.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:14 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:19:14.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:14 smithi134 ceph-mon[34283]: Adjusting osd_memory_target on smithi204 to 4272M 2024-09-13T18:19:14.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:14 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:14.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:14 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:19:14.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:14 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:19:14.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:14 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:14.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:14 smithi134 ceph-mon[34283]: osdmap e81: 12 total, 12 up, 12 in 2024-09-13T18:19:14.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:14 smithi204 ceph-mon[34777]: Detected new or changed devices on smithi204 2024-09-13T18:19:14.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:14 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:14.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:14 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:14.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:14 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:19:14.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:14 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:19:14.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:14 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:19:14.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:14 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:19:14.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:14 smithi204 ceph-mon[34777]: Adjusting osd_memory_target on smithi204 to 4272M 2024-09-13T18:19:14.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:14 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:14.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:14 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:19:14.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:14 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:19:14.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:14 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:14.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:14 smithi204 ceph-mon[34777]: osdmap e81: 12 total, 12 up, 12 in 2024-09-13T18:19:15.160 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:15 smithi114 ceph-mon[30897]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:15.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:15 smithi134 ceph-mon[34283]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:15.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:15.378+0000 7efc5974f640 1 -- 172.21.15.114:0/1454950588 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc540ffe00 msgr2=0x7efc54100200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:15.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:15.378+0000 7efc5974f640 1 --2- 172.21.15.114:0/1454950588 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc540ffe00 0x7efc54100200 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7efc44009900 tx=0x7efc4402a410 comp rx=0 tx=0).stop 2024-09-13T18:19:15.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:15.378+0000 7efc5974f640 1 -- 172.21.15.114:0/1454950588 shutdown_connections 2024-09-13T18:19:15.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:15.378+0000 7efc5974f640 1 --2- 172.21.15.114:0/1454950588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc54101920 0x7efc54108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:15.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:15.378+0000 7efc5974f640 1 --2- 172.21.15.114:0/1454950588 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc54100f60 0x7efc541013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:15.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:15.378+0000 7efc5974f640 1 --2- 172.21.15.114:0/1454950588 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc540ffe00 0x7efc54100200 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:15.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:15.378+0000 7efc5974f640 1 -- 172.21.15.114:0/1454950588 >> 172.21.15.114:0/1454950588 conn(0x7efc540fb5d0 msgr2=0x7efc540fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:15.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:15.378+0000 7efc5974f640 1 -- 172.21.15.114:0/1454950588 shutdown_connections 2024-09-13T18:19:15.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:15.378+0000 7efc5974f640 1 -- 172.21.15.114:0/1454950588 wait complete. 2024-09-13T18:19:15.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:15.379+0000 7efc5974f640 1 Processor -- start 2024-09-13T18:19:15.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:15.379+0000 7efc5974f640 1 -- start start 2024-09-13T18:19:15.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:15.379+0000 7efc5974f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc540ffe00 0x7efc54074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:15.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:15.379+0000 7efc5974f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc54100f60 0x7efc540728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:15.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:15.379+0000 7efc5974f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc54101920 0x7efc54072de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:15.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:15.379+0000 7efc5974f640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc54073570 con 0x7efc54100f60 2024-09-13T18:19:15.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:15.379+0000 7efc5974f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc540736e0 con 0x7efc54101920 2024-09-13T18:19:15.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:15.379+0000 7efc5974f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5419fcf0 con 0x7efc540ffe00 2024-09-13T18:19:15.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:15.379+0000 7efc4bfff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc54100f60 0x7efc540728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:15.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:15.379+0000 7efc4bfff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc54100f60 0x7efc540728a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41946/0 (socket says 172.21.15.114:41946) 2024-09-13T18:19:15.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:15.379+0000 7efc4bfff640 1 -- 172.21.15.114:0/2245336832 learned_addr learned my addr 172.21.15.114:0/2245336832 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:19:15.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:15.379+0000 7efc537fe640 1 --2- 172.21.15.114:0/2245336832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc54101920 0x7efc54072de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:15.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:15.379+0000 7efc4bfff640 1 -- 172.21.15.114:0/2245336832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc540ffe00 msgr2=0x7efc54074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:15.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:15.379+0000 7efc4bfff640 1 --2- 172.21.15.114:0/2245336832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc540ffe00 0x7efc54074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:15.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:15.379+0000 7efc4bfff640 1 -- 172.21.15.114:0/2245336832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc54101920 msgr2=0x7efc54072de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:15.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:15.379+0000 7efc4bfff640 1 --2- 172.21.15.114:0/2245336832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc54101920 0x7efc54072de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:15.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:15.379+0000 7efc4bfff640 1 -- 172.21.15.114:0/2245336832 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc44009560 con 0x7efc54100f60 2024-09-13T18:19:15.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:15.379+0000 7efc4bfff640 1 --2- 172.21.15.114:0/2245336832 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc54100f60 0x7efc540728a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efc3c00d7d0 tx=0x7efc3c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:15.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:15.380+0000 7efc50ff9640 1 -- 172.21.15.114:0/2245336832 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3c00dee0 con 0x7efc54100f60 2024-09-13T18:19:15.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:15.380+0000 7efc50ff9640 1 -- 172.21.15.114:0/2245336832 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efc3c0044b0 con 0x7efc54100f60 2024-09-13T18:19:15.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:15.380+0000 7efc5974f640 1 -- 172.21.15.114:0/2245336832 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc5419ffa0 con 0x7efc54100f60 2024-09-13T18:19:15.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:15.380+0000 7efc50ff9640 1 -- 172.21.15.114:0/2245336832 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3c003d00 con 0x7efc54100f60 2024-09-13T18:19:15.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:15.380+0000 7efc5974f640 1 -- 172.21.15.114:0/2245336832 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc541a0410 con 0x7efc54100f60 2024-09-13T18:19:15.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:15.381+0000 7efc5974f640 1 -- 172.21.15.114:0/2245336832 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc54107280 con 0x7efc54100f60 2024-09-13T18:19:15.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:15.382+0000 7efc50ff9640 1 -- 172.21.15.114:0/2245336832 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7efc3c003e60 con 0x7efc54100f60 2024-09-13T18:19:15.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:15.382+0000 7efc50ff9640 1 --2- 172.21.15.114:0/2245336832 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7efc400a1080 0x7efc400a3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:15.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:15.383+0000 7efc52ffd640 1 --2- 172.21.15.114:0/2245336832 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7efc400a1080 0x7efc400a3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:15.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:15.383+0000 7efc50ff9640 1 -- 172.21.15.114:0/2245336832 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7efc3c0cbe60 con 0x7efc54100f60 2024-09-13T18:19:15.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:15.383+0000 7efc52ffd640 1 --2- 172.21.15.114:0/2245336832 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7efc400a1080 0x7efc400a3540 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7efc4402a930 tx=0x7efc44000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:15.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:15.387+0000 7efc50ff9640 1 -- 172.21.15.114:0/2245336832 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7efc3c095a90 con 0x7efc54100f60 2024-09-13T18:19:15.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:15 smithi204 ceph-mon[34777]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:15.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:15.550+0000 7efc5974f640 1 -- 172.21.15.114:0/2245336832 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7efc5410b7e0 con 0x7efc54100f60 2024-09-13T18:19:15.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:15.550+0000 7efc50ff9640 1 -- 172.21.15.114:0/2245336832 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) v1 ==== 74+0+133 (secure 0 0 0) 0x7efc3c0990b0 con 0x7efc54100f60 2024-09-13T18:19:15.553 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:19:15.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:15.552+0000 7efc5974f640 1 -- 172.21.15.114:0/2245336832 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7efc400a1080 msgr2=0x7efc400a3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:15.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:15.552+0000 7efc5974f640 1 --2- 172.21.15.114:0/2245336832 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7efc400a1080 0x7efc400a3540 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7efc4402a930 tx=0x7efc44000bc0 comp rx=0 tx=0).stop 2024-09-13T18:19:15.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:15.552+0000 7efc5974f640 1 -- 172.21.15.114:0/2245336832 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc54100f60 msgr2=0x7efc540728a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:15.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:15.552+0000 7efc5974f640 1 --2- 172.21.15.114:0/2245336832 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc54100f60 0x7efc540728a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efc3c00d7d0 tx=0x7efc3c00dc90 comp rx=0 tx=0).stop 2024-09-13T18:19:15.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:15.552+0000 7efc5974f640 1 -- 172.21.15.114:0/2245336832 shutdown_connections 2024-09-13T18:19:15.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:15.552+0000 7efc5974f640 1 --2- 172.21.15.114:0/2245336832 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7efc400a1080 0x7efc400a3540 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:15.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:15.552+0000 7efc5974f640 1 --2- 172.21.15.114:0/2245336832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc54101920 0x7efc54072de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:15.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:15.552+0000 7efc5974f640 1 --2- 172.21.15.114:0/2245336832 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc54100f60 0x7efc540728a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:15.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:15.552+0000 7efc5974f640 1 --2- 172.21.15.114:0/2245336832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc540ffe00 0x7efc54074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:15.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:15.552+0000 7efc5974f640 1 -- 172.21.15.114:0/2245336832 >> 172.21.15.114:0/2245336832 conn(0x7efc540fb5d0 msgr2=0x7efc54104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:15.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:15.552+0000 7efc5974f640 1 -- 172.21.15.114:0/2245336832 shutdown_connections 2024-09-13T18:19:15.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:15.552+0000 7efc5974f640 1 -- 172.21.15.114:0/2245336832 wait complete. 2024-09-13T18:19:16.173 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:16 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2245336832' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-13T18:19:16.194 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1726251552,"num_in_osds":12,"osd_in_since":1726251532,"num_remapped_pgs":0} 2024-09-13T18:19:16.195 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-13T18:19:16.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:16 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/2245336832' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-13T18:19:16.389 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:19:16.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:16 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/2245336832' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-13T18:19:17.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:17 smithi134 ceph-mon[34283]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:17.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:17 smithi204 ceph-mon[34777]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:17.469 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:17 smithi114 ceph-mon[30897]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:18.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.044+0000 7f477eae3640 1 -- 172.21.15.114:0/3992189912 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f47780feed0 msgr2=0x7f47780ff2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:18.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.044+0000 7f477eae3640 1 --2- 172.21.15.114:0/3992189912 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f47780feed0 0x7f47780ff2b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f476c009930 tx=0x7f476c02a420 comp rx=0 tx=0).stop 2024-09-13T18:19:18.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.044+0000 7f477eae3640 1 -- 172.21.15.114:0/3992189912 shutdown_connections 2024-09-13T18:19:18.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.044+0000 7f477eae3640 1 --2- 172.21.15.114:0/3992189912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4778104000 0x7f477810bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:18.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.044+0000 7f477eae3640 1 --2- 172.21.15.114:0/3992189912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47780ff880 0x7f47781038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:18.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.045+0000 7f477eae3640 1 --2- 172.21.15.114:0/3992189912 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f47780feed0 0x7f47780ff2b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:18.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.045+0000 7f477eae3640 1 -- 172.21.15.114:0/3992189912 >> 172.21.15.114:0/3992189912 conn(0x7f47780fa9b0 msgr2=0x7f47780fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:18.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.045+0000 7f477eae3640 1 -- 172.21.15.114:0/3992189912 shutdown_connections 2024-09-13T18:19:18.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.045+0000 7f477eae3640 1 -- 172.21.15.114:0/3992189912 wait complete. 2024-09-13T18:19:18.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.045+0000 7f477eae3640 1 Processor -- start 2024-09-13T18:19:18.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.045+0000 7f477eae3640 1 -- start start 2024-09-13T18:19:18.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.046+0000 7f477eae3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f47780feed0 0x7f4778196240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:18.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.046+0000 7f477eae3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47780ff880 0x7f4778196780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:18.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.046+0000 7f477eae3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4778104000 0x7f477819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:18.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.046+0000 7f477eae3640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477819b130 con 0x7f47780feed0 2024-09-13T18:19:18.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.046+0000 7f477eae3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477819b2a0 con 0x7f4778104000 2024-09-13T18:19:18.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.046+0000 7f477eae3640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477819b410 con 0x7f47780ff880 2024-09-13T18:19:18.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.046+0000 7f477c87f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f47780feed0 0x7f4778196240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:18.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.046+0000 7f477c87f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f47780feed0 0x7f4778196240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41962/0 (socket says 172.21.15.114:41962) 2024-09-13T18:19:18.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.046+0000 7f477c87f640 1 -- 172.21.15.114:0/3073659189 learned_addr learned my addr 172.21.15.114:0/3073659189 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:19:18.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.046+0000 7f4777fff640 1 --2- 172.21.15.114:0/3073659189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47780ff880 0x7f4778196780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:18.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.046+0000 7f477d080640 1 --2- 172.21.15.114:0/3073659189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4778104000 0x7f477819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:18.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.046+0000 7f477c87f640 1 -- 172.21.15.114:0/3073659189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47780ff880 msgr2=0x7f4778196780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:18.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.046+0000 7f477c87f640 1 --2- 172.21.15.114:0/3073659189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47780ff880 0x7f4778196780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:18.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.046+0000 7f477c87f640 1 -- 172.21.15.114:0/3073659189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4778104000 msgr2=0x7f477819ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:18.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.046+0000 7f477c87f640 1 --2- 172.21.15.114:0/3073659189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4778104000 0x7f477819ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:18.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.046+0000 7f477c87f640 1 -- 172.21.15.114:0/3073659189 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f476c009560 con 0x7f47780feed0 2024-09-13T18:19:18.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.046+0000 7f4777fff640 1 --2- 172.21.15.114:0/3073659189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47780ff880 0x7f4778196780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:19:18.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.046+0000 7f477c87f640 1 --2- 172.21.15.114:0/3073659189 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f47780feed0 0x7f4778196240 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f476c02aa10 tx=0x7f476c032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:18.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.046+0000 7f4775ffb640 1 -- 172.21.15.114:0/3073659189 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476c005210 con 0x7f47780feed0 2024-09-13T18:19:18.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.046+0000 7f4775ffb640 1 -- 172.21.15.114:0/3073659189 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f476c005370 con 0x7f47780feed0 2024-09-13T18:19:18.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.046+0000 7f477eae3640 1 -- 172.21.15.114:0/3073659189 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f477819b690 con 0x7f47780feed0 2024-09-13T18:19:18.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.046+0000 7f4775ffb640 1 -- 172.21.15.114:0/3073659189 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476c0038e0 con 0x7f47780feed0 2024-09-13T18:19:18.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.047+0000 7f477eae3640 1 -- 172.21.15.114:0/3073659189 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47781a2de0 con 0x7f47780feed0 2024-09-13T18:19:18.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.047+0000 7f477eae3640 1 -- 172.21.15.114:0/3073659189 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47781003b0 con 0x7f47780feed0 2024-09-13T18:19:18.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.048+0000 7f4775ffb640 1 -- 172.21.15.114:0/3073659189 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f476c003a40 con 0x7f47780feed0 2024-09-13T18:19:18.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.049+0000 7f4775ffb640 1 --2- 172.21.15.114:0/3073659189 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f47680a1080 0x7f47680a3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:18.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.049+0000 7f4775ffb640 1 -- 172.21.15.114:0/3073659189 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f476c0ed1c0 con 0x7f47780feed0 2024-09-13T18:19:18.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.049+0000 7f4777fff640 1 --2- 172.21.15.114:0/3073659189 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f47680a1080 0x7f47680a3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:18.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.050+0000 7f4777fff640 1 --2- 172.21.15.114:0/3073659189 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f47680a1080 0x7f47680a3540 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f47781977a0 tx=0x7f4764008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:18.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.053+0000 7f4775ffb640 1 -- 172.21.15.114:0/3073659189 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f476c0b6df0 con 0x7f47780feed0 2024-09-13T18:19:18.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.219+0000 7f477eae3640 1 -- 172.21.15.114:0/3073659189 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f47780ff2b0 con 0x7f47780feed0 2024-09-13T18:19:18.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.220+0000 7f4775ffb640 1 -- 172.21.15.114:0/3073659189 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18491 (secure 0 0 0) 0x7f476c0ba410 con 0x7f47780feed0 2024-09-13T18:19:18.223 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:19:18.223 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":81,"fsid":"97e7edf2-71fb-11ef-bceb-c7b262605968","created":"2024-09-13T18:12:36.360625+0000","modified":"2024-09-13T18:19:13.938715+0000","last_up_change":"2024-09-13T18:19:12.350370+0000","last_in_change":"2024-09-13T18:18:52.321596+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:15:40.129659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9914f01a-9f6f-4878-b955-83570199c354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6803","nonce":3499525826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6805","nonce":3499525826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6809","nonce":3499525826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6807","nonce":3499525826}]},"public_addr":"172.21.15.114:6803/3499525826","cluster_addr":"172.21.15.114:6805/3499525826","heartbeat_back_addr":"172.21.15.114:6809/3499525826","heartbeat_front_addr":"172.21.15.114:6807/3499525826","state":["exists","up"]},{"osd":1,"uuid":"ef65e012-7469-4c20-b354-17022bca63c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6801","nonce":4264264866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6803","nonce":4264264866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6807","nonce":4264264866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6805","nonce":4264264866}]},"public_addr":"172.21.15.134:6801/4264264866","cluster_addr":"172.21.15.134:6803/4264264866","heartbeat_back_addr":"172.21.15.134:6807/4264264866","heartbeat_front_addr":"172.21.15.134:6805/4264264866","state":["exists","up"]},{"osd":2,"uuid":"578213fd-715f-4be7-91fa-254f3d0e3022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6801","nonce":2288807707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6803","nonce":2288807707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6807","nonce":2288807707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6805","nonce":2288807707}]},"public_addr":"172.21.15.204:6801/2288807707","cluster_addr":"172.21.15.204:6803/2288807707","heartbeat_back_addr":"172.21.15.204:6807/2288807707","heartbeat_front_addr":"172.21.15.204:6805/2288807707","state":["exists","up"]},{"osd":3,"uuid":"b64aab3f-1e52-4828-8397-74af5e05f2cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6811","nonce":2904204254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6813","nonce":2904204254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6817","nonce":2904204254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6815","nonce":2904204254}]},"public_addr":"172.21.15.114:6811/2904204254","cluster_addr":"172.21.15.114:6813/2904204254","heartbeat_back_addr":"172.21.15.114:6817/2904204254","heartbeat_front_addr":"172.21.15.114:6815/2904204254","state":["exists","up"]},{"osd":4,"uuid":"f38943d6-39c0-42e1-b427-cb6bb90ac38a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6809","nonce":1773945333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6811","nonce":1773945333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6815","nonce":1773945333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6813","nonce":1773945333}]},"public_addr":"172.21.15.134:6809/1773945333","cluster_addr":"172.21.15.134:6811/1773945333","heartbeat_back_addr":"172.21.15.134:6815/1773945333","heartbeat_front_addr":"172.21.15.134:6813/1773945333","state":["exists","up"]},{"osd":5,"uuid":"bb38f361-8e00-4178-9752-5ddbf3af20e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6809","nonce":2371778055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6811","nonce":2371778055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6815","nonce":2371778055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6813","nonce":2371778055}]},"public_addr":"172.21.15.204:6809/2371778055","cluster_addr":"172.21.15.204:6811/2371778055","heartbeat_back_addr":"172.21.15.204:6815/2371778055","heartbeat_front_addr":"172.21.15.204:6813/2371778055","state":["exists","up"]},{"osd":6,"uuid":"93d40a03-e7ff-4a3f-9efa-5c5403c9983d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6819","nonce":1484325044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6821","nonce":1484325044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6825","nonce":1484325044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6823","nonce":1484325044}]},"public_addr":"172.21.15.114:6819/1484325044","cluster_addr":"172.21.15.114:6821/1484325044","heartbeat_back_addr":"172.21.15.114:6825/1484325044","heartbeat_front_addr":"172.21.15.114:6823/1484325044","state":["exists","up"]},{"osd":7,"uuid":"982f629a-1b95-4be1-8e75-5ddce1ad77b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6817","nonce":799246614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6819","nonce":799246614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6823","nonce":799246614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6821","nonce":799246614}]},"public_addr":"172.21.15.134:6817/799246614","cluster_addr":"172.21.15.134:6819/799246614","heartbeat_back_addr":"172.21.15.134:6823/799246614","heartbeat_front_addr":"172.21.15.134:6821/799246614","state":["exists","up"]},{"osd":8,"uuid":"409beb9b-8227-4dad-a4a0-f0e5ce70dc5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6817","nonce":858331805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6819","nonce":858331805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6823","nonce":858331805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6821","nonce":858331805}]},"public_addr":"172.21.15.204:6817/858331805","cluster_addr":"172.21.15.204:6819/858331805","heartbeat_back_addr":"172.21.15.204:6823/858331805","heartbeat_front_addr":"172.21.15.204:6821/858331805","state":["exists","up"]},{"osd":9,"uuid":"a345a15c-b1fc-4877-b445-418eebf0d0ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6827","nonce":2172392854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6829","nonce":2172392854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6833","nonce":2172392854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6831","nonce":2172392854}]},"public_addr":"172.21.15.114:6827/2172392854","cluster_addr":"172.21.15.114:6829/2172392854","heartbeat_back_addr":"172.21.15.114:6833/2172392854","heartbeat_front_addr":"172.21.15.114:6831/2172392854","state":["exists","up"]},{"osd":10,"uuid":"464d995f-d90b-4caa-addb-1f7562bd74b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6825","nonce":3073513468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6827","nonce":3073513468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6831","nonce":3073513468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6829","nonce":3073513468}]},"public_addr":"172.21.15.134:6825/3073513468","cluster_addr":"172.21.15.134:6827/3073513468","heartbeat_back_addr":"172.21.15.134:6831/3073513468","heartbeat_front_addr":"172.21.15.134:6829/3073513468","state":["exists","up"]},{"osd":11,"uuid":"71dd794d-326c-477f-8d02-ee32bff1350d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6825","nonce":3239521765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6827","nonce":3239521765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6831","nonce":3239521765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6829","nonce":3239521765}]},"public_addr":"172.21.15.204:6825/3239521765","cluster_addr":"172.21.15.204:6827/3239521765","heartbeat_back_addr":"172.21.15.204:6831/3239521765","heartbeat_front_addr":"172.21.15.204:6829/3239521765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:14:54.542687+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:15:15.746055+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:15:38.371131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:01.383932+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:24.796237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:48.422214+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:11.284323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:34.570365+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:58.232855+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:18:21.609596+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:18:45.565859+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:19:10.017969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:6801/4281866858":"2024-09-14T18:13:25.823027+0000","172.21.15.114:6800/4281866858":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2949422849":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2034742251":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/722571999":"2024-09-14T18:13:03.521116+0000","172.21.15.114:6801/830738794":"2024-09-14T18:13:03.521116+0000","172.21.15.114:6800/830738794":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/2828364892":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/4257828906":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/3544951322":"2024-09-14T18:13:03.521116+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:19:18.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.222+0000 7f477eae3640 1 -- 172.21.15.114:0/3073659189 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f47680a1080 msgr2=0x7f47680a3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:18.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.222+0000 7f477eae3640 1 --2- 172.21.15.114:0/3073659189 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f47680a1080 0x7f47680a3540 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f47781977a0 tx=0x7f4764008040 comp rx=0 tx=0).stop 2024-09-13T18:19:18.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.222+0000 7f477eae3640 1 -- 172.21.15.114:0/3073659189 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f47780feed0 msgr2=0x7f4778196240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:18.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.222+0000 7f477eae3640 1 --2- 172.21.15.114:0/3073659189 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f47780feed0 0x7f4778196240 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f476c02aa10 tx=0x7f476c032c20 comp rx=0 tx=0).stop 2024-09-13T18:19:18.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.223+0000 7f477eae3640 1 -- 172.21.15.114:0/3073659189 shutdown_connections 2024-09-13T18:19:18.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.223+0000 7f477eae3640 1 --2- 172.21.15.114:0/3073659189 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f47680a1080 0x7f47680a3540 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:18.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.223+0000 7f477eae3640 1 --2- 172.21.15.114:0/3073659189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4778104000 0x7f477819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:18.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.223+0000 7f477eae3640 1 --2- 172.21.15.114:0/3073659189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47780ff880 0x7f4778196780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:18.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.223+0000 7f477eae3640 1 --2- 172.21.15.114:0/3073659189 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f47780feed0 0x7f4778196240 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:18.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.223+0000 7f477eae3640 1 -- 172.21.15.114:0/3073659189 >> 172.21.15.114:0/3073659189 conn(0x7f47780fa9b0 msgr2=0x7f47780fbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:18.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.223+0000 7f477eae3640 1 -- 172.21.15.114:0/3073659189 shutdown_connections 2024-09-13T18:19:18.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:18.223+0000 7f477eae3640 1 -- 172.21.15.114:0/3073659189 wait complete. 2024-09-13T18:19:18.929 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T18:15:40.129659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T18:19:18.929 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-13T18:19:19.125 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:19:19.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:19 smithi114 ceph-mon[30897]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:19.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:19 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/3073659189' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:19:19.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:19 smithi134 ceph-mon[34283]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:19.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:19 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/3073659189' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:19:19.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:19 smithi204 ceph-mon[34777]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:19.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:19 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/3073659189' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:19:20.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:20.747+0000 7fce7bf1d640 1 -- 172.21.15.114:0/2983596332 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce741044f0 msgr2=0x7fce741048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:20.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:20.747+0000 7fce7bf1d640 1 --2- 172.21.15.114:0/2983596332 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce741044f0 0x7fce741048d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fce68009900 tx=0x7fce6802a480 comp rx=0 tx=0).stop 2024-09-13T18:19:20.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:20.747+0000 7fce7bf1d640 1 -- 172.21.15.114:0/2983596332 shutdown_connections 2024-09-13T18:19:20.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:20.747+0000 7fce7bf1d640 1 --2- 172.21.15.114:0/2983596332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce740ff2b0 0x7fce7410b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:20.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:20.747+0000 7fce7bf1d640 1 --2- 172.21.15.114:0/2983596332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce740fe930 0x7fce740fed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:20.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:20.747+0000 7fce7bf1d640 1 --2- 172.21.15.114:0/2983596332 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce741044f0 0x7fce741048d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:20.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:20.747+0000 7fce7bf1d640 1 -- 172.21.15.114:0/2983596332 >> 172.21.15.114:0/2983596332 conn(0x7fce74074130 msgr2=0x7fce74074540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:20.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:20.747+0000 7fce7bf1d640 1 -- 172.21.15.114:0/2983596332 shutdown_connections 2024-09-13T18:19:20.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:20.747+0000 7fce7bf1d640 1 -- 172.21.15.114:0/2983596332 wait complete. 2024-09-13T18:19:20.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:20.748+0000 7fce7bf1d640 1 Processor -- start 2024-09-13T18:19:20.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:20.748+0000 7fce7bf1d640 1 -- start start 2024-09-13T18:19:20.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:20.748+0000 7fce7bf1d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce740fe930 0x7fce74196180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:20.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:20.748+0000 7fce7bf1d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce740ff2b0 0x7fce741966c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:20.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:20.748+0000 7fce7bf1d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce741044f0 0x7fce7419aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:20.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:20.748+0000 7fce7bf1d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7419b0c0 con 0x7fce741044f0 2024-09-13T18:19:20.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:20.748+0000 7fce7bf1d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7419b230 con 0x7fce740fe930 2024-09-13T18:19:20.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:20.748+0000 7fce7bf1d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7419b3a0 con 0x7fce740ff2b0 2024-09-13T18:19:20.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:20.748+0000 7fce7a4ba640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce741044f0 0x7fce7419aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:20.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:20.748+0000 7fce7a4ba640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce741044f0 0x7fce7419aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41990/0 (socket says 172.21.15.114:41990) 2024-09-13T18:19:20.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:20.748+0000 7fce7a4ba640 1 -- 172.21.15.114:0/3165312223 learned_addr learned my addr 172.21.15.114:0/3165312223 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:19:20.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:20.748+0000 7fce79cb9640 1 --2- 172.21.15.114:0/3165312223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce740fe930 0x7fce74196180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:20.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:20.748+0000 7fce794b8640 1 --2- 172.21.15.114:0/3165312223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce740ff2b0 0x7fce741966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:20.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:20.748+0000 7fce7a4ba640 1 -- 172.21.15.114:0/3165312223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce740ff2b0 msgr2=0x7fce741966c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:20.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:20.748+0000 7fce7a4ba640 1 --2- 172.21.15.114:0/3165312223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce740ff2b0 0x7fce741966c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:20.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:20.748+0000 7fce7a4ba640 1 -- 172.21.15.114:0/3165312223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce740fe930 msgr2=0x7fce74196180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:20.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:20.748+0000 7fce7a4ba640 1 --2- 172.21.15.114:0/3165312223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce740fe930 0x7fce74196180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:20.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:20.748+0000 7fce7a4ba640 1 -- 172.21.15.114:0/3165312223 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce68009560 con 0x7fce741044f0 2024-09-13T18:19:20.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:20.749+0000 7fce7a4ba640 1 --2- 172.21.15.114:0/3165312223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce741044f0 0x7fce7419aac0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fce6c00b970 tx=0x7fce6c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:20.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:20.749+0000 7fce66ffd640 1 -- 172.21.15.114:0/3165312223 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c00cb40 con 0x7fce741044f0 2024-09-13T18:19:20.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:20.749+0000 7fce66ffd640 1 -- 172.21.15.114:0/3165312223 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fce6c00cca0 con 0x7fce741044f0 2024-09-13T18:19:20.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:20.749+0000 7fce7bf1d640 1 -- 172.21.15.114:0/3165312223 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce7419b680 con 0x7fce741044f0 2024-09-13T18:19:20.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:20.749+0000 7fce66ffd640 1 -- 172.21.15.114:0/3165312223 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c007610 con 0x7fce741044f0 2024-09-13T18:19:20.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:20.749+0000 7fce7bf1d640 1 -- 172.21.15.114:0/3165312223 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce741a2f40 con 0x7fce741044f0 2024-09-13T18:19:20.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:20.750+0000 7fce7bf1d640 1 -- 172.21.15.114:0/3165312223 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce740ffb90 con 0x7fce741044f0 2024-09-13T18:19:20.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:20.751+0000 7fce66ffd640 1 -- 172.21.15.114:0/3165312223 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fce6c014440 con 0x7fce741044f0 2024-09-13T18:19:20.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:20.752+0000 7fce66ffd640 1 --2- 172.21.15.114:0/3165312223 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fce600a1080 0x7fce600a3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:20.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:20.752+0000 7fce79cb9640 1 --2- 172.21.15.114:0/3165312223 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fce600a1080 0x7fce600a3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:20.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:20.752+0000 7fce66ffd640 1 -- 172.21.15.114:0/3165312223 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fce6c0cbd10 con 0x7fce741044f0 2024-09-13T18:19:20.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:20.752+0000 7fce79cb9640 1 --2- 172.21.15.114:0/3165312223 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fce600a1080 0x7fce600a3540 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fce68006010 tx=0x7fce68005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:20.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:20.756+0000 7fce66ffd640 1 -- 172.21.15.114:0/3165312223 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fce6c0959f0 con 0x7fce741044f0 2024-09-13T18:19:20.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:20.917+0000 7fce7bf1d640 1 -- 172.21.15.114:0/3165312223 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fce7410ebe0 con 0x7fce741044f0 2024-09-13T18:19:20.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:20.918+0000 7fce66ffd640 1 -- 172.21.15.114:0/3165312223 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7fce6c099010 con 0x7fce741044f0 2024-09-13T18:19:20.920 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 1 2024-09-13T18:19:20.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:20.920+0000 7fce7bf1d640 1 -- 172.21.15.114:0/3165312223 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fce600a1080 msgr2=0x7fce600a3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:20.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:20.920+0000 7fce7bf1d640 1 --2- 172.21.15.114:0/3165312223 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fce600a1080 0x7fce600a3540 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fce68006010 tx=0x7fce68005a20 comp rx=0 tx=0).stop 2024-09-13T18:19:20.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:20.920+0000 7fce7bf1d640 1 -- 172.21.15.114:0/3165312223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce741044f0 msgr2=0x7fce7419aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:20.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:20.920+0000 7fce7bf1d640 1 --2- 172.21.15.114:0/3165312223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce741044f0 0x7fce7419aac0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fce6c00b970 tx=0x7fce6c00be30 comp rx=0 tx=0).stop 2024-09-13T18:19:20.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:20.920+0000 7fce7bf1d640 1 -- 172.21.15.114:0/3165312223 shutdown_connections 2024-09-13T18:19:20.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:20.920+0000 7fce7bf1d640 1 --2- 172.21.15.114:0/3165312223 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fce600a1080 0x7fce600a3540 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:20.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:20.920+0000 7fce7bf1d640 1 --2- 172.21.15.114:0/3165312223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce741044f0 0x7fce7419aac0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:20.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:20.920+0000 7fce7bf1d640 1 --2- 172.21.15.114:0/3165312223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce740ff2b0 0x7fce741966c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:20.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:20.920+0000 7fce7bf1d640 1 --2- 172.21.15.114:0/3165312223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce740fe930 0x7fce74196180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:20.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:20.920+0000 7fce7bf1d640 1 -- 172.21.15.114:0/3165312223 >> 172.21.15.114:0/3165312223 conn(0x7fce74074130 msgr2=0x7fce7410ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:20.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:20.920+0000 7fce7bf1d640 1 -- 172.21.15.114:0/3165312223 shutdown_connections 2024-09-13T18:19:20.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:20.920+0000 7fce7bf1d640 1 -- 172.21.15.114:0/3165312223 wait complete. 2024-09-13T18:19:21.219 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:21 smithi114 ceph-mon[30897]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:21.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:21 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/3165312223' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T18:19:21.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:21 smithi134 ceph-mon[34283]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:21.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:21 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/3165312223' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T18:19:21.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:21 smithi204 ceph-mon[34777]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:21.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:21 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/3165312223' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T18:19:21.613 INFO:tasks.cephadm:Adding mds.a on smithi114 2024-09-13T18:19:21.613 INFO:tasks.cephadm:Adding mds.d on smithi114 2024-09-13T18:19:21.613 INFO:tasks.cephadm:Adding mds.g on smithi114 2024-09-13T18:19:21.613 INFO:tasks.cephadm:Adding mds.j on smithi114 2024-09-13T18:19:21.613 INFO:tasks.cephadm:Adding mds.b on smithi134 2024-09-13T18:19:21.613 INFO:tasks.cephadm:Adding mds.e on smithi134 2024-09-13T18:19:21.613 INFO:tasks.cephadm:Adding mds.h on smithi134 2024-09-13T18:19:21.613 INFO:tasks.cephadm:Adding mds.k on smithi134 2024-09-13T18:19:21.614 INFO:tasks.cephadm:Adding mds.c on smithi204 2024-09-13T18:19:21.614 INFO:tasks.cephadm:Adding mds.f on smithi204 2024-09-13T18:19:21.614 INFO:tasks.cephadm:Adding mds.i on smithi204 2024-09-13T18:19:21.614 INFO:tasks.cephadm:Adding mds.l on smithi204 2024-09-13T18:19:21.614 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph orch apply mds all '12;smithi114=a;smithi114=d;smithi114=g;smithi114=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l' 2024-09-13T18:19:21.826 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T18:19:23.265 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:23 smithi204 ceph-mon[34777]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:23.368 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:23 smithi134 ceph-mon[34283]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:23.469 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:23 smithi114 ceph-mon[30897]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:23.695 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:23.681+0000 7f283e4b2640 1 -- 172.21.15.204:0/2331281990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2838101920 msgr2=0x7f2838108240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:23.695 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:23.681+0000 7f283e4b2640 1 --2- 172.21.15.204:0/2331281990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2838101920 0x7f2838108240 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2830009ca0 tx=0x7f283002a8c0 comp rx=0 tx=0).stop 2024-09-13T18:19:23.695 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:23.681+0000 7f283e4b2640 1 -- 172.21.15.204:0/2331281990 shutdown_connections 2024-09-13T18:19:23.695 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:23.681+0000 7f283e4b2640 1 --2- 172.21.15.204:0/2331281990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2838101920 0x7f2838108240 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:23.695 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:23.681+0000 7f283e4b2640 1 --2- 172.21.15.204:0/2331281990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2838100f60 0x7f28381013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:23.695 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:23.681+0000 7f283e4b2640 1 --2- 172.21.15.204:0/2331281990 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f28380ffe00 0x7f2838100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:23.695 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:23.681+0000 7f283e4b2640 1 -- 172.21.15.204:0/2331281990 >> 172.21.15.204:0/2331281990 conn(0x7f28380fb590 msgr2=0x7f28380fd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:23.695 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:23.681+0000 7f283e4b2640 1 -- 172.21.15.204:0/2331281990 shutdown_connections 2024-09-13T18:19:23.695 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:23.681+0000 7f283e4b2640 1 -- 172.21.15.204:0/2331281990 wait complete. 2024-09-13T18:19:23.695 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:23.681+0000 7f283e4b2640 1 Processor -- start 2024-09-13T18:19:23.696 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:23.681+0000 7f283e4b2640 1 -- start start 2024-09-13T18:19:23.696 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:23.681+0000 7f283e4b2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f28380ffe00 0x7f2838195c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:23.696 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:23.681+0000 7f283e4b2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2838100f60 0x7f28381961a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:23.696 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:23.681+0000 7f283e4b2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2838101920 0x7f283819d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:23.696 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:23.681+0000 7f283e4b2640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283819d750 con 0x7f28380ffe00 2024-09-13T18:19:23.696 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:23.681+0000 7f283e4b2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283819d8c0 con 0x7f2838101920 2024-09-13T18:19:23.696 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:23.681+0000 7f283e4b2640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283819da80 con 0x7f2838100f60 2024-09-13T18:19:23.696 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:23.682+0000 7f282ffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2838100f60 0x7f28381961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:23.696 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:23.682+0000 7f282ffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2838100f60 0x7f28381961a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:54100/0 (socket says 172.21.15.204:54100) 2024-09-13T18:19:23.696 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:23.682+0000 7f282ffff640 1 -- 172.21.15.204:0/4111598542 learned_addr learned my addr 172.21.15.204:0/4111598542 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-13T18:19:23.697 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:23.682+0000 7f2837fff640 1 --2- 172.21.15.204:0/4111598542 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f28380ffe00 0x7f2838195c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:23.697 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:23.682+0000 7f283ca4f640 1 --2- 172.21.15.204:0/4111598542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2838101920 0x7f283819d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:23.697 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:23.682+0000 7f282ffff640 1 -- 172.21.15.204:0/4111598542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2838101920 msgr2=0x7f283819d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:23.697 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:23.682+0000 7f282ffff640 1 --2- 172.21.15.204:0/4111598542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2838101920 0x7f283819d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:23.697 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:23.682+0000 7f282ffff640 1 -- 172.21.15.204:0/4111598542 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f28380ffe00 msgr2=0x7f2838195c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:23.697 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:23.682+0000 7f282ffff640 1 --2- 172.21.15.204:0/4111598542 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f28380ffe00 0x7f2838195c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:23.697 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:23.682+0000 7f282ffff640 1 -- 172.21.15.204:0/4111598542 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2830009900 con 0x7f2838100f60 2024-09-13T18:19:23.697 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:23.682+0000 7f282ffff640 1 --2- 172.21.15.204:0/4111598542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2838100f60 0x7f28381961a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f282000d770 tx=0x7f282000dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:23.697 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:23.682+0000 7f2835ffb640 1 -- 172.21.15.204:0/4111598542 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2820009c30 con 0x7f2838100f60 2024-09-13T18:19:23.697 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:23.682+0000 7f2835ffb640 1 -- 172.21.15.204:0/4111598542 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f282000de80 con 0x7f2838100f60 2024-09-13T18:19:23.698 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:23.682+0000 7f283e4b2640 1 -- 172.21.15.204:0/4111598542 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f283819dd60 con 0x7f2838100f60 2024-09-13T18:19:23.698 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:23.682+0000 7f2835ffb640 1 -- 172.21.15.204:0/4111598542 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2820003d40 con 0x7f2838100f60 2024-09-13T18:19:23.698 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:23.682+0000 7f283e4b2640 1 -- 172.21.15.204:0/4111598542 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f283819e200 con 0x7f2838100f60 2024-09-13T18:19:23.698 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:23.683+0000 7f283e4b2640 1 -- 172.21.15.204:0/4111598542 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2838107230 con 0x7f2838100f60 2024-09-13T18:19:23.698 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:23.684+0000 7f2835ffb640 1 -- 172.21.15.204:0/4111598542 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f2820003ea0 con 0x7f2838100f60 2024-09-13T18:19:23.698 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:23.685+0000 7f2835ffb640 1 --2- 172.21.15.204:0/4111598542 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f28240a1080 0x7f28240a3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:23.698 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:23.685+0000 7f2835ffb640 1 -- 172.21.15.204:0/4111598542 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f28200cbd10 con 0x7f2838100f60 2024-09-13T18:19:23.698 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:23.686+0000 7f2837fff640 1 --2- 172.21.15.204:0/4111598542 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f28240a1080 0x7f28240a3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:23.698 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:23.686+0000 7f2837fff640 1 --2- 172.21.15.204:0/4111598542 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f28240a1080 0x7f28240a3540 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2838196b40 tx=0x7f2828005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:23.698 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:23.690+0000 7f2835ffb640 1 -- 172.21.15.204:0/4111598542 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f28200959f0 con 0x7f2838100f60 2024-09-13T18:19:23.864 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:23.861+0000 7f283e4b2640 1 -- 172.21.15.204:0/4111598542 --> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi114=a;smithi114=d;smithi114=g;smithi114=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f283802c5f0 con 0x7f28240a1080 2024-09-13T18:19:23.871 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:23.869+0000 7f2835ffb640 1 -- 172.21.15.204:0/4111598542 <== mgr.14148 v2:172.21.15.114:6800/3396185541 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f283802c5f0 con 0x7f28240a1080 2024-09-13T18:19:23.871 INFO:teuthology.orchestra.run.smithi204.stdout:Scheduled mds.all update... 2024-09-13T18:19:23.874 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:23.871+0000 7f283e4b2640 1 -- 172.21.15.204:0/4111598542 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f28240a1080 msgr2=0x7f28240a3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:23.874 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:23.871+0000 7f283e4b2640 1 --2- 172.21.15.204:0/4111598542 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f28240a1080 0x7f28240a3540 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2838196b40 tx=0x7f2828005f90 comp rx=0 tx=0).stop 2024-09-13T18:19:23.874 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:23.871+0000 7f283e4b2640 1 -- 172.21.15.204:0/4111598542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2838100f60 msgr2=0x7f28381961a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:23.874 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:23.871+0000 7f283e4b2640 1 --2- 172.21.15.204:0/4111598542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2838100f60 0x7f28381961a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f282000d770 tx=0x7f282000dc30 comp rx=0 tx=0).stop 2024-09-13T18:19:23.874 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:23.871+0000 7f283e4b2640 1 -- 172.21.15.204:0/4111598542 shutdown_connections 2024-09-13T18:19:23.874 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:23.871+0000 7f283e4b2640 1 --2- 172.21.15.204:0/4111598542 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f28240a1080 0x7f28240a3540 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:23.874 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:23.871+0000 7f283e4b2640 1 --2- 172.21.15.204:0/4111598542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2838101920 0x7f283819d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:23.874 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:23.871+0000 7f283e4b2640 1 --2- 172.21.15.204:0/4111598542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2838100f60 0x7f28381961a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:23.875 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:23.871+0000 7f283e4b2640 1 --2- 172.21.15.204:0/4111598542 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f28380ffe00 0x7f2838195c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:23.875 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:23.871+0000 7f283e4b2640 1 -- 172.21.15.204:0/4111598542 >> 172.21.15.204:0/4111598542 conn(0x7f28380fb590 msgr2=0x7f2838104400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:23.875 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:23.871+0000 7f283e4b2640 1 -- 172.21.15.204:0/4111598542 shutdown_connections 2024-09-13T18:19:23.875 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-13T18:19:23.871+0000 7f283e4b2640 1 -- 172.21.15.204:0/4111598542 wait complete. 2024-09-13T18:19:24.624 DEBUG:teuthology.orchestra.run.smithi114:mds.a> sudo journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mds.a.service 2024-09-13T18:19:24.627 DEBUG:teuthology.orchestra.run.smithi114:mds.d> sudo journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mds.d.service 2024-09-13T18:19:24.671 DEBUG:teuthology.orchestra.run.smithi114:mds.g> sudo journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mds.g.service 2024-09-13T18:19:24.714 DEBUG:teuthology.orchestra.run.smithi114:mds.j> sudo journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mds.j.service 2024-09-13T18:19:24.757 DEBUG:teuthology.orchestra.run.smithi134:mds.b> sudo journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mds.b.service 2024-09-13T18:19:24.760 DEBUG:teuthology.orchestra.run.smithi134:mds.e> sudo journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mds.e.service 2024-09-13T18:19:24.802 DEBUG:teuthology.orchestra.run.smithi134:mds.h> sudo journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mds.h.service 2024-09-13T18:19:24.845 DEBUG:teuthology.orchestra.run.smithi134:mds.k> sudo journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mds.k.service 2024-09-13T18:19:24.888 DEBUG:teuthology.orchestra.run.smithi204:mds.c> sudo journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mds.c.service 2024-09-13T18:19:24.890 DEBUG:teuthology.orchestra.run.smithi204:mds.f> sudo journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mds.f.service 2024-09-13T18:19:24.933 DEBUG:teuthology.orchestra.run.smithi204:mds.i> sudo journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mds.i.service 2024-09-13T18:19:24.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:24 smithi114 ceph-mon[30897]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi114=a;smithi114=d;smithi114=g;smithi114=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:19:24.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:24 smithi114 ceph-mon[30897]: Saving service mds.all spec with placement smithi114=a;smithi114=d;smithi114=g;smithi114=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2024-09-13T18:19:24.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:24 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:24.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:24 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:19:24.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:24 smithi114 ceph-mon[30897]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:24.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:24 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:19:24.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:24 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:19:24.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:24 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:24.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:24 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T18:19:24.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:24 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T18:19:24.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:24 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:19:24.976 DEBUG:teuthology.orchestra.run.smithi204:mds.l> sudo journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mds.l.service 2024-09-13T18:19:25.019 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-13T18:19:25.019 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:19:25.040 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:24 smithi204 ceph-mon[34777]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi114=a;smithi114=d;smithi114=g;smithi114=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:19:25.040 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:24 smithi204 ceph-mon[34777]: Saving service mds.all spec with placement smithi114=a;smithi114=d;smithi114=g;smithi114=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2024-09-13T18:19:25.041 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:24 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:25.041 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:24 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:19:25.041 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:24 smithi204 ceph-mon[34777]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:25.041 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:24 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:19:25.041 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:24 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:19:25.041 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:24 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:25.041 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:24 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T18:19:25.041 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:24 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T18:19:25.041 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:24 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:19:25.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:24 smithi134 ceph-mon[34283]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi114=a;smithi114=d;smithi114=g;smithi114=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:19:25.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:24 smithi134 ceph-mon[34283]: Saving service mds.all spec with placement smithi114=a;smithi114=d;smithi114=g;smithi114=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2024-09-13T18:19:25.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:24 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:25.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:24 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:19:25.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:24 smithi134 ceph-mon[34283]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:25.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:24 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:19:25.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:24 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:19:25.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:24 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:25.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:24 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T18:19:25.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:24 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T18:19:25.120 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:24 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:19:25.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.127+0000 7f3a9d533640 1 -- 172.21.15.114:0/3149362054 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a98072e90 msgr2=0x7f3a980732f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:25.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.127+0000 7f3a9d533640 1 --2- 172.21.15.114:0/3149362054 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a98072e90 0x7f3a980732f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3a80009900 tx=0x7f3a8002a480 comp rx=0 tx=0).stop 2024-09-13T18:19:25.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.128+0000 7f3a9d533640 1 -- 172.21.15.114:0/3149362054 shutdown_connections 2024-09-13T18:19:25.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.128+0000 7f3a9d533640 1 --2- 172.21.15.114:0/3149362054 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a98107c50 0x7f3a9810a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:25.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.128+0000 7f3a9d533640 1 --2- 172.21.15.114:0/3149362054 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a98072e90 0x7f3a980732f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:25.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.128+0000 7f3a9d533640 1 --2- 172.21.15.114:0/3149362054 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a98074470 0x7f3a980728c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:25.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.128+0000 7f3a9d533640 1 -- 172.21.15.114:0/3149362054 >> 172.21.15.114:0/3149362054 conn(0x7f3a980fb520 msgr2=0x7f3a980fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:25.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.128+0000 7f3a9d533640 1 -- 172.21.15.114:0/3149362054 shutdown_connections 2024-09-13T18:19:25.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.128+0000 7f3a9d533640 1 -- 172.21.15.114:0/3149362054 wait complete. 2024-09-13T18:19:25.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.128+0000 7f3a9d533640 1 Processor -- start 2024-09-13T18:19:25.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.128+0000 7f3a9d533640 1 -- start start 2024-09-13T18:19:25.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.128+0000 7f3a9d533640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a98072e90 0x7f3a98197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:25.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.129+0000 7f3a9d533640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a98074470 0x7f3a98198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:25.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.129+0000 7f3a9d533640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a98107c50 0x7f3a9819f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:25.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.129+0000 7f3a9d533640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9819fa20 con 0x7f3a98107c50 2024-09-13T18:19:25.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.129+0000 7f3a9d533640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9819fb90 con 0x7f3a98074470 2024-09-13T18:19:25.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.129+0000 7f3a9d533640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9819fd50 con 0x7f3a98072e90 2024-09-13T18:19:25.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.129+0000 7f3a977fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a98107c50 0x7f3a9819f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:25.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.129+0000 7f3a977fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a98107c50 0x7f3a9819f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42416/0 (socket says 172.21.15.114:42416) 2024-09-13T18:19:25.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.129+0000 7f3a977fe640 1 -- 172.21.15.114:0/4268874734 learned_addr learned my addr 172.21.15.114:0/4268874734 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:19:25.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.129+0000 7f3a96ffd640 1 --2- 172.21.15.114:0/4268874734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a98072e90 0x7f3a98197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:25.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.129+0000 7f3a8e7fc640 1 --2- 172.21.15.114:0/4268874734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a98074470 0x7f3a98198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:25.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.129+0000 7f3a977fe640 1 -- 172.21.15.114:0/4268874734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a98072e90 msgr2=0x7f3a98197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:25.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.129+0000 7f3a977fe640 1 --2- 172.21.15.114:0/4268874734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a98072e90 0x7f3a98197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:25.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.129+0000 7f3a977fe640 1 -- 172.21.15.114:0/4268874734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a98074470 msgr2=0x7f3a98198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:25.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.129+0000 7f3a977fe640 1 --2- 172.21.15.114:0/4268874734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a98074470 0x7f3a98198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:25.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.129+0000 7f3a977fe640 1 -- 172.21.15.114:0/4268874734 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a80009560 con 0x7f3a98107c50 2024-09-13T18:19:25.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.129+0000 7f3a977fe640 1 --2- 172.21.15.114:0/4268874734 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a98107c50 0x7f3a9819f4e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3a90009fd0 tx=0x7f3a9000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:25.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.129+0000 7f3a94ff9640 1 -- 172.21.15.114:0/4268874734 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9000bea0 con 0x7f3a98107c50 2024-09-13T18:19:25.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.129+0000 7f3a94ff9640 1 -- 172.21.15.114:0/4268874734 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3a90009940 con 0x7f3a98107c50 2024-09-13T18:19:25.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.129+0000 7f3a9d533640 1 -- 172.21.15.114:0/4268874734 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a9819ffd0 con 0x7f3a98107c50 2024-09-13T18:19:25.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.129+0000 7f3a94ff9640 1 -- 172.21.15.114:0/4268874734 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a90010480 con 0x7f3a98107c50 2024-09-13T18:19:25.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.129+0000 7f3a9d533640 1 -- 172.21.15.114:0/4268874734 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a981a0460 con 0x7f3a98107c50 2024-09-13T18:19:25.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.130+0000 7f3a9d533640 1 -- 172.21.15.114:0/4268874734 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a98106300 con 0x7f3a98107c50 2024-09-13T18:19:25.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.131+0000 7f3a94ff9640 1 -- 172.21.15.114:0/4268874734 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f3a90003970 con 0x7f3a98107c50 2024-09-13T18:19:25.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.132+0000 7f3a94ff9640 1 --2- 172.21.15.114:0/4268874734 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f3a840a1090 0x7f3a840a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:25.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.132+0000 7f3a96ffd640 1 --2- 172.21.15.114:0/4268874734 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f3a840a1090 0x7f3a840a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:25.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.132+0000 7f3a94ff9640 1 -- 172.21.15.114:0/4268874734 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3a900cc140 con 0x7f3a98107c50 2024-09-13T18:19:25.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.133+0000 7f3a96ffd640 1 --2- 172.21.15.114:0/4268874734 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f3a840a1090 0x7f3a840a3550 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3a88009bd0 tx=0x7f3a88005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:25.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.136+0000 7f3a94ff9640 1 -- 172.21.15.114:0/4268874734 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f3a90095d70 con 0x7f3a98107c50 2024-09-13T18:19:25.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.301+0000 7f3a9d533640 1 -- 172.21.15.114:0/4268874734 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3a98105d70 con 0x7f3a98107c50 2024-09-13T18:19:25.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.302+0000 7f3a94ff9640 1 -- 172.21.15.114:0/4268874734 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18491 (secure 0 0 0) 0x7f3a90099390 con 0x7f3a98107c50 2024-09-13T18:19:25.305 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:19:25.305 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":81,"fsid":"97e7edf2-71fb-11ef-bceb-c7b262605968","created":"2024-09-13T18:12:36.360625+0000","modified":"2024-09-13T18:19:13.938715+0000","last_up_change":"2024-09-13T18:19:12.350370+0000","last_in_change":"2024-09-13T18:18:52.321596+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:15:40.129659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9914f01a-9f6f-4878-b955-83570199c354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6803","nonce":3499525826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6805","nonce":3499525826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6809","nonce":3499525826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6807","nonce":3499525826}]},"public_addr":"172.21.15.114:6803/3499525826","cluster_addr":"172.21.15.114:6805/3499525826","heartbeat_back_addr":"172.21.15.114:6809/3499525826","heartbeat_front_addr":"172.21.15.114:6807/3499525826","state":["exists","up"]},{"osd":1,"uuid":"ef65e012-7469-4c20-b354-17022bca63c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6801","nonce":4264264866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6803","nonce":4264264866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6807","nonce":4264264866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6805","nonce":4264264866}]},"public_addr":"172.21.15.134:6801/4264264866","cluster_addr":"172.21.15.134:6803/4264264866","heartbeat_back_addr":"172.21.15.134:6807/4264264866","heartbeat_front_addr":"172.21.15.134:6805/4264264866","state":["exists","up"]},{"osd":2,"uuid":"578213fd-715f-4be7-91fa-254f3d0e3022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6801","nonce":2288807707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6803","nonce":2288807707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6807","nonce":2288807707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6805","nonce":2288807707}]},"public_addr":"172.21.15.204:6801/2288807707","cluster_addr":"172.21.15.204:6803/2288807707","heartbeat_back_addr":"172.21.15.204:6807/2288807707","heartbeat_front_addr":"172.21.15.204:6805/2288807707","state":["exists","up"]},{"osd":3,"uuid":"b64aab3f-1e52-4828-8397-74af5e05f2cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6811","nonce":2904204254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6813","nonce":2904204254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6817","nonce":2904204254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6815","nonce":2904204254}]},"public_addr":"172.21.15.114:6811/2904204254","cluster_addr":"172.21.15.114:6813/2904204254","heartbeat_back_addr":"172.21.15.114:6817/2904204254","heartbeat_front_addr":"172.21.15.114:6815/2904204254","state":["exists","up"]},{"osd":4,"uuid":"f38943d6-39c0-42e1-b427-cb6bb90ac38a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6809","nonce":1773945333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6811","nonce":1773945333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6815","nonce":1773945333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6813","nonce":1773945333}]},"public_addr":"172.21.15.134:6809/1773945333","cluster_addr":"172.21.15.134:6811/1773945333","heartbeat_back_addr":"172.21.15.134:6815/1773945333","heartbeat_front_addr":"172.21.15.134:6813/1773945333","state":["exists","up"]},{"osd":5,"uuid":"bb38f361-8e00-4178-9752-5ddbf3af20e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6809","nonce":2371778055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6811","nonce":2371778055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6815","nonce":2371778055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6813","nonce":2371778055}]},"public_addr":"172.21.15.204:6809/2371778055","cluster_addr":"172.21.15.204:6811/2371778055","heartbeat_back_addr":"172.21.15.204:6815/2371778055","heartbeat_front_addr":"172.21.15.204:6813/2371778055","state":["exists","up"]},{"osd":6,"uuid":"93d40a03-e7ff-4a3f-9efa-5c5403c9983d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6819","nonce":1484325044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6821","nonce":1484325044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6825","nonce":1484325044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6823","nonce":1484325044}]},"public_addr":"172.21.15.114:6819/1484325044","cluster_addr":"172.21.15.114:6821/1484325044","heartbeat_back_addr":"172.21.15.114:6825/1484325044","heartbeat_front_addr":"172.21.15.114:6823/1484325044","state":["exists","up"]},{"osd":7,"uuid":"982f629a-1b95-4be1-8e75-5ddce1ad77b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6817","nonce":799246614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6819","nonce":799246614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6823","nonce":799246614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6821","nonce":799246614}]},"public_addr":"172.21.15.134:6817/799246614","cluster_addr":"172.21.15.134:6819/799246614","heartbeat_back_addr":"172.21.15.134:6823/799246614","heartbeat_front_addr":"172.21.15.134:6821/799246614","state":["exists","up"]},{"osd":8,"uuid":"409beb9b-8227-4dad-a4a0-f0e5ce70dc5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6817","nonce":858331805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6819","nonce":858331805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6823","nonce":858331805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6821","nonce":858331805}]},"public_addr":"172.21.15.204:6817/858331805","cluster_addr":"172.21.15.204:6819/858331805","heartbeat_back_addr":"172.21.15.204:6823/858331805","heartbeat_front_addr":"172.21.15.204:6821/858331805","state":["exists","up"]},{"osd":9,"uuid":"a345a15c-b1fc-4877-b445-418eebf0d0ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6827","nonce":2172392854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6829","nonce":2172392854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6833","nonce":2172392854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6831","nonce":2172392854}]},"public_addr":"172.21.15.114:6827/2172392854","cluster_addr":"172.21.15.114:6829/2172392854","heartbeat_back_addr":"172.21.15.114:6833/2172392854","heartbeat_front_addr":"172.21.15.114:6831/2172392854","state":["exists","up"]},{"osd":10,"uuid":"464d995f-d90b-4caa-addb-1f7562bd74b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6825","nonce":3073513468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6827","nonce":3073513468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6831","nonce":3073513468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6829","nonce":3073513468}]},"public_addr":"172.21.15.134:6825/3073513468","cluster_addr":"172.21.15.134:6827/3073513468","heartbeat_back_addr":"172.21.15.134:6831/3073513468","heartbeat_front_addr":"172.21.15.134:6829/3073513468","state":["exists","up"]},{"osd":11,"uuid":"71dd794d-326c-477f-8d02-ee32bff1350d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6825","nonce":3239521765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6827","nonce":3239521765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6831","nonce":3239521765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6829","nonce":3239521765}]},"public_addr":"172.21.15.204:6825/3239521765","cluster_addr":"172.21.15.204:6827/3239521765","heartbeat_back_addr":"172.21.15.204:6831/3239521765","heartbeat_front_addr":"172.21.15.204:6829/3239521765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:14:54.542687+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:15:15.746055+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:15:38.371131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:01.383932+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:24.796237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:48.422214+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:11.284323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:34.570365+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:58.232855+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:18:21.609596+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:18:45.565859+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:19:10.017969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:6801/4281866858":"2024-09-14T18:13:25.823027+0000","172.21.15.114:6800/4281866858":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2949422849":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2034742251":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/722571999":"2024-09-14T18:13:03.521116+0000","172.21.15.114:6801/830738794":"2024-09-14T18:13:03.521116+0000","172.21.15.114:6800/830738794":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/2828364892":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/4257828906":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/3544951322":"2024-09-14T18:13:03.521116+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:19:25.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.304+0000 7f3a9d533640 1 -- 172.21.15.114:0/4268874734 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f3a840a1090 msgr2=0x7f3a840a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:25.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.304+0000 7f3a9d533640 1 --2- 172.21.15.114:0/4268874734 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f3a840a1090 0x7f3a840a3550 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3a88009bd0 tx=0x7f3a88005f90 comp rx=0 tx=0).stop 2024-09-13T18:19:25.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.304+0000 7f3a9d533640 1 -- 172.21.15.114:0/4268874734 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a98107c50 msgr2=0x7f3a9819f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:25.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.304+0000 7f3a9d533640 1 --2- 172.21.15.114:0/4268874734 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a98107c50 0x7f3a9819f4e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3a90009fd0 tx=0x7f3a9000b260 comp rx=0 tx=0).stop 2024-09-13T18:19:25.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.304+0000 7f3a9d533640 1 -- 172.21.15.114:0/4268874734 shutdown_connections 2024-09-13T18:19:25.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.304+0000 7f3a9d533640 1 --2- 172.21.15.114:0/4268874734 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f3a840a1090 0x7f3a840a3550 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:25.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.304+0000 7f3a9d533640 1 --2- 172.21.15.114:0/4268874734 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a98107c50 0x7f3a9819f4e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:25.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.304+0000 7f3a9d533640 1 --2- 172.21.15.114:0/4268874734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a98074470 0x7f3a98198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:25.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.304+0000 7f3a9d533640 1 --2- 172.21.15.114:0/4268874734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a98072e90 0x7f3a98197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:25.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.304+0000 7f3a9d533640 1 -- 172.21.15.114:0/4268874734 >> 172.21.15.114:0/4268874734 conn(0x7f3a980fb520 msgr2=0x7f3a980fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:25.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.304+0000 7f3a9d533640 1 -- 172.21.15.114:0/4268874734 shutdown_connections 2024-09-13T18:19:25.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.304+0000 7f3a9d533640 1 -- 172.21.15.114:0/4268874734 wait complete. 2024-09-13T18:19:25.317 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T18:15:40.129659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T18:19:25.318 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T18:19:25.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.417+0000 7f566a75e640 1 -- 172.21.15.114:0/3330907600 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56640ffe50 msgr2=0x7f56641002d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:25.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.417+0000 7f566a75e640 1 --2- 172.21.15.114:0/3330907600 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56640ffe50 0x7f56641002d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5658009900 tx=0x7f565802a410 comp rx=0 tx=0).stop 2024-09-13T18:19:25.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.417+0000 7f566a75e640 1 -- 172.21.15.114:0/3330907600 shutdown_connections 2024-09-13T18:19:25.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.417+0000 7f566a75e640 1 --2- 172.21.15.114:0/3330907600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5664100810 0x7f56641070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:25.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.417+0000 7f566a75e640 1 --2- 172.21.15.114:0/3330907600 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56640ffe50 0x7f56641002d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:25.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.417+0000 7f566a75e640 1 --2- 172.21.15.114:0/3330907600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56640fec60 0x7f56640ff060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:25.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.417+0000 7f566a75e640 1 -- 172.21.15.114:0/3330907600 >> 172.21.15.114:0/3330907600 conn(0x7f56640fa3e0 msgr2=0x7f56640fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:25.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.417+0000 7f566a75e640 1 -- 172.21.15.114:0/3330907600 shutdown_connections 2024-09-13T18:19:25.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.417+0000 7f566a75e640 1 -- 172.21.15.114:0/3330907600 wait complete. 2024-09-13T18:19:25.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.417+0000 7f566a75e640 1 Processor -- start 2024-09-13T18:19:25.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.418+0000 7f566a75e640 1 -- start start 2024-09-13T18:19:25.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.418+0000 7f566a75e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56640fec60 0x7f5664068110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:25.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.418+0000 7f566a75e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56640ffe50 0x7f566419a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:25.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.418+0000 7f566a75e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5664100810 0x7f566419d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:25.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.418+0000 7f566a75e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f566419d7f0 con 0x7f5664100810 2024-09-13T18:19:25.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.418+0000 7f566a75e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f566419d960 con 0x7f56640ffe50 2024-09-13T18:19:25.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.418+0000 7f566a75e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f566419db20 con 0x7f56640fec60 2024-09-13T18:19:25.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.418+0000 7f5668cfb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5664100810 0x7f566419d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:25.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.418+0000 7f5668cfb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5664100810 0x7f566419d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42426/0 (socket says 172.21.15.114:42426) 2024-09-13T18:19:25.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.418+0000 7f5668cfb640 1 -- 172.21.15.114:0/71014906 learned_addr learned my addr 172.21.15.114:0/71014906 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:19:25.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.418+0000 7f5663fff640 1 --2- 172.21.15.114:0/71014906 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56640fec60 0x7f5664068110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:25.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.418+0000 7f56637fe640 1 --2- 172.21.15.114:0/71014906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56640ffe50 0x7f566419a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:25.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.418+0000 7f5668cfb640 1 -- 172.21.15.114:0/71014906 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56640fec60 msgr2=0x7f5664068110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:25.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.418+0000 7f5668cfb640 1 --2- 172.21.15.114:0/71014906 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56640fec60 0x7f5664068110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:25.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.418+0000 7f5668cfb640 1 -- 172.21.15.114:0/71014906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56640ffe50 msgr2=0x7f566419a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:25.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.418+0000 7f5668cfb640 1 --2- 172.21.15.114:0/71014906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56640ffe50 0x7f566419a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:25.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.418+0000 7f5668cfb640 1 -- 172.21.15.114:0/71014906 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5658009560 con 0x7f5664100810 2024-09-13T18:19:25.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.418+0000 7f5668cfb640 1 --2- 172.21.15.114:0/71014906 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5664100810 0x7f566419d010 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f565c009fd0 tx=0x7f565c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:25.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.419+0000 7f56617fa640 1 -- 172.21.15.114:0/71014906 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565c00bea0 con 0x7f5664100810 2024-09-13T18:19:25.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.419+0000 7f566a75e640 1 -- 172.21.15.114:0/71014906 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f566419dda0 con 0x7f5664100810 2024-09-13T18:19:25.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.419+0000 7f56617fa640 1 -- 172.21.15.114:0/71014906 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f565c009940 con 0x7f5664100810 2024-09-13T18:19:25.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.419+0000 7f56617fa640 1 -- 172.21.15.114:0/71014906 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565c010480 con 0x7f5664100810 2024-09-13T18:19:25.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.419+0000 7f566a75e640 1 -- 172.21.15.114:0/71014906 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f566419e220 con 0x7f5664100810 2024-09-13T18:19:25.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.420+0000 7f566a75e640 1 -- 172.21.15.114:0/71014906 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56641061b0 con 0x7f5664100810 2024-09-13T18:19:25.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.421+0000 7f56617fa640 1 -- 172.21.15.114:0/71014906 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f565c004a40 con 0x7f5664100810 2024-09-13T18:19:25.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.422+0000 7f56617fa640 1 --2- 172.21.15.114:0/71014906 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f564c0a1090 0x7f564c0a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:25.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.422+0000 7f5663fff640 1 --2- 172.21.15.114:0/71014906 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f564c0a1090 0x7f564c0a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:25.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.422+0000 7f56617fa640 1 -- 172.21.15.114:0/71014906 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f565c0cc100 con 0x7f5664100810 2024-09-13T18:19:25.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.422+0000 7f5663fff640 1 --2- 172.21.15.114:0/71014906 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f564c0a1090 0x7f564c0a3550 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f56640ffcb0 tx=0x7f565400a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:25.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.425+0000 7f56617fa640 1 -- 172.21.15.114:0/71014906 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f565c095d40 con 0x7f5664100810 2024-09-13T18:19:25.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.588+0000 7f566a75e640 1 -- 172.21.15.114:0/71014906 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f566410a3b0 con 0x7f5664100810 2024-09-13T18:19:25.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.588+0000 7f56617fa640 1 -- 172.21.15.114:0/71014906 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f565c099360 con 0x7f5664100810 2024-09-13T18:19:25.591 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 1 2024-09-13T18:19:25.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.590+0000 7f566a75e640 1 -- 172.21.15.114:0/71014906 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f564c0a1090 msgr2=0x7f564c0a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:25.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.590+0000 7f566a75e640 1 --2- 172.21.15.114:0/71014906 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f564c0a1090 0x7f564c0a3550 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f56640ffcb0 tx=0x7f565400a2d0 comp rx=0 tx=0).stop 2024-09-13T18:19:25.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.590+0000 7f566a75e640 1 -- 172.21.15.114:0/71014906 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5664100810 msgr2=0x7f566419d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:25.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.590+0000 7f566a75e640 1 --2- 172.21.15.114:0/71014906 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5664100810 0x7f566419d010 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f565c009fd0 tx=0x7f565c00b260 comp rx=0 tx=0).stop 2024-09-13T18:19:25.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.590+0000 7f566a75e640 1 -- 172.21.15.114:0/71014906 shutdown_connections 2024-09-13T18:19:25.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.590+0000 7f566a75e640 1 --2- 172.21.15.114:0/71014906 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f564c0a1090 0x7f564c0a3550 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:25.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.590+0000 7f566a75e640 1 --2- 172.21.15.114:0/71014906 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5664100810 0x7f566419d010 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:25.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.590+0000 7f566a75e640 1 --2- 172.21.15.114:0/71014906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56640ffe50 0x7f566419a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:25.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.590+0000 7f566a75e640 1 --2- 172.21.15.114:0/71014906 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56640fec60 0x7f5664068110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:25.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.590+0000 7f566a75e640 1 -- 172.21.15.114:0/71014906 >> 172.21.15.114:0/71014906 conn(0x7f56640fa3e0 msgr2=0x7f5664103380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:25.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.590+0000 7f566a75e640 1 -- 172.21.15.114:0/71014906 shutdown_connections 2024-09-13T18:19:25.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.590+0000 7f566a75e640 1 -- 172.21.15.114:0/71014906 wait complete. 2024-09-13T18:19:25.604 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:19:25.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.699+0000 7f6e65118640 1 -- 172.21.15.114:0/4288159612 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e60105b10 msgr2=0x7f6e60107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:25.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.699+0000 7f6e65118640 1 --2- 172.21.15.114:0/4288159612 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e60105b10 0x7f6e60107f20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6e50009ca0 tx=0x7f6e5002a8c0 comp rx=0 tx=0).stop 2024-09-13T18:19:25.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.700+0000 7f6e65118640 1 -- 172.21.15.114:0/4288159612 shutdown_connections 2024-09-13T18:19:25.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.700+0000 7f6e65118640 1 --2- 172.21.15.114:0/4288159612 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e60105b10 0x7f6e60107f20 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:25.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.700+0000 7f6e65118640 1 --2- 172.21.15.114:0/4288159612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e601031e0 0x7f6e601055d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:25.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.700+0000 7f6e65118640 1 --2- 172.21.15.114:0/4288159612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e601008b0 0x7f6e60102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:25.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.700+0000 7f6e65118640 1 -- 172.21.15.114:0/4288159612 >> 172.21.15.114:0/4288159612 conn(0x7f6e600fa430 msgr2=0x7f6e600fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:25.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.700+0000 7f6e65118640 1 -- 172.21.15.114:0/4288159612 shutdown_connections 2024-09-13T18:19:25.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.700+0000 7f6e65118640 1 -- 172.21.15.114:0/4288159612 wait complete. 2024-09-13T18:19:25.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.700+0000 7f6e65118640 1 Processor -- start 2024-09-13T18:19:25.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.700+0000 7f6e65118640 1 -- start start 2024-09-13T18:19:25.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.701+0000 7f6e65118640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e601008b0 0x7f6e60100220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:25.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.701+0000 7f6e65118640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e601031e0 0x7f6e600fe860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:25.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.701+0000 7f6e65118640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e60105b10 0x7f6e600feda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:25.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.701+0000 7f6e65118640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e600ff500 con 0x7f6e601008b0 2024-09-13T18:19:25.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.701+0000 7f6e65118640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e600ff670 con 0x7f6e601031e0 2024-09-13T18:19:25.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.701+0000 7f6e65118640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6019fd20 con 0x7f6e60105b10 2024-09-13T18:19:25.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.701+0000 7f6e5ed9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e601008b0 0x7f6e60100220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:25.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.701+0000 7f6e5ed9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e601008b0 0x7f6e60100220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42458/0 (socket says 172.21.15.114:42458) 2024-09-13T18:19:25.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.701+0000 7f6e5ed9d640 1 -- 172.21.15.114:0/390251062 learned_addr learned my addr 172.21.15.114:0/390251062 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:19:25.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.701+0000 7f6e5f59e640 1 --2- 172.21.15.114:0/390251062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e60105b10 0x7f6e600feda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:25.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.701+0000 7f6e5e59c640 1 --2- 172.21.15.114:0/390251062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e601031e0 0x7f6e600fe860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:25.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.701+0000 7f6e5ed9d640 1 -- 172.21.15.114:0/390251062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e60105b10 msgr2=0x7f6e600feda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:25.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.701+0000 7f6e5ed9d640 1 --2- 172.21.15.114:0/390251062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e60105b10 0x7f6e600feda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:25.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.701+0000 7f6e5ed9d640 1 -- 172.21.15.114:0/390251062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e601031e0 msgr2=0x7f6e600fe860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:25.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.701+0000 7f6e5ed9d640 1 --2- 172.21.15.114:0/390251062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e601031e0 0x7f6e600fe860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:25.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.701+0000 7f6e5ed9d640 1 -- 172.21.15.114:0/390251062 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e50009900 con 0x7f6e601008b0 2024-09-13T18:19:25.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.701+0000 7f6e5e59c640 1 --2- 172.21.15.114:0/390251062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e601031e0 0x7f6e600fe860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:19:25.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.701+0000 7f6e5ed9d640 1 --2- 172.21.15.114:0/390251062 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e601008b0 0x7f6e60100220 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6e5800c9a0 tx=0x7f6e5800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:25.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.701+0000 7f6e57fff640 1 -- 172.21.15.114:0/390251062 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e58007bb0 con 0x7f6e601008b0 2024-09-13T18:19:25.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.701+0000 7f6e57fff640 1 -- 172.21.15.114:0/390251062 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6e58007d10 con 0x7f6e601008b0 2024-09-13T18:19:25.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.701+0000 7f6e65118640 1 -- 172.21.15.114:0/390251062 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e6019ffd0 con 0x7f6e601008b0 2024-09-13T18:19:25.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.701+0000 7f6e57fff640 1 -- 172.21.15.114:0/390251062 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e58004de0 con 0x7f6e601008b0 2024-09-13T18:19:25.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.701+0000 7f6e65118640 1 -- 172.21.15.114:0/390251062 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e601a03f0 con 0x7f6e601008b0 2024-09-13T18:19:25.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.702+0000 7f6e65118640 1 -- 172.21.15.114:0/390251062 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e600fbf20 con 0x7f6e601008b0 2024-09-13T18:19:25.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.703+0000 7f6e57fff640 1 -- 172.21.15.114:0/390251062 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f6e58004fa0 con 0x7f6e601008b0 2024-09-13T18:19:25.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.704+0000 7f6e57fff640 1 --2- 172.21.15.114:0/390251062 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f6e4c0a1090 0x7f6e4c0a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:25.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.704+0000 7f6e5e59c640 1 --2- 172.21.15.114:0/390251062 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f6e4c0a1090 0x7f6e4c0a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:25.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.704+0000 7f6e57fff640 1 -- 172.21.15.114:0/390251062 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6e580cc1e0 con 0x7f6e601008b0 2024-09-13T18:19:25.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.705+0000 7f6e5e59c640 1 --2- 172.21.15.114:0/390251062 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f6e4c0a1090 0x7f6e4c0a3550 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6e600fff90 tx=0x7f6e4800a5c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:25.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.708+0000 7f6e57fff640 1 -- 172.21.15.114:0/390251062 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f6e58095e10 con 0x7f6e601008b0 2024-09-13T18:19:25.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.905+0000 7f6e65118640 1 -- 172.21.15.114:0/390251062 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6e6010b580 con 0x7f6e601008b0 2024-09-13T18:19:25.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.906+0000 7f6e57fff640 1 -- 172.21.15.114:0/390251062 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+501 (secure 0 0 0) 0x7f6e5801d210 con 0x7f6e601008b0 2024-09-13T18:19:25.908 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:19:25.908 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-13T18:19:25.909 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 1 2024-09-13T18:19:25.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.908+0000 7f6e65118640 1 -- 172.21.15.114:0/390251062 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f6e4c0a1090 msgr2=0x7f6e4c0a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:25.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.908+0000 7f6e65118640 1 --2- 172.21.15.114:0/390251062 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f6e4c0a1090 0x7f6e4c0a3550 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6e600fff90 tx=0x7f6e4800a5c0 comp rx=0 tx=0).stop 2024-09-13T18:19:25.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.908+0000 7f6e65118640 1 -- 172.21.15.114:0/390251062 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e601008b0 msgr2=0x7f6e60100220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:25.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.908+0000 7f6e65118640 1 --2- 172.21.15.114:0/390251062 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e601008b0 0x7f6e60100220 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6e5800c9a0 tx=0x7f6e5800ce60 comp rx=0 tx=0).stop 2024-09-13T18:19:25.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.908+0000 7f6e65118640 1 -- 172.21.15.114:0/390251062 shutdown_connections 2024-09-13T18:19:25.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.908+0000 7f6e65118640 1 --2- 172.21.15.114:0/390251062 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f6e4c0a1090 0x7f6e4c0a3550 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:25.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.908+0000 7f6e65118640 1 --2- 172.21.15.114:0/390251062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e60105b10 0x7f6e600feda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:25.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.908+0000 7f6e65118640 1 --2- 172.21.15.114:0/390251062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e601031e0 0x7f6e600fe860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:25.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.908+0000 7f6e65118640 1 --2- 172.21.15.114:0/390251062 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e601008b0 0x7f6e60100220 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:25.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.908+0000 7f6e65118640 1 -- 172.21.15.114:0/390251062 >> 172.21.15.114:0/390251062 conn(0x7f6e600fa430 msgr2=0x7f6e601012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:25.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.908+0000 7f6e65118640 1 -- 172.21.15.114:0/390251062 shutdown_connections 2024-09-13T18:19:25.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:25.908+0000 7f6e65118640 1 -- 172.21.15.114:0/390251062 wait complete. 2024-09-13T18:19:26.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:25 smithi204 ceph-mon[34777]: Deploying daemon mds.l on smithi204 2024-09-13T18:19:26.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:25 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/4268874734' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:19:26.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:25 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/71014906' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T18:19:26.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:25 smithi114 ceph-mon[30897]: Deploying daemon mds.l on smithi204 2024-09-13T18:19:26.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:25 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/4268874734' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:19:26.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:25 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/71014906' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T18:19:26.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:25 smithi134 ceph-mon[34283]: Deploying daemon mds.l on smithi204 2024-09-13T18:19:26.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:25 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/4268874734' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:19:26.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:25 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/71014906' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T18:19:26.608 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 13 18:19:26 smithi204 systemd[1]: Starting Ceph mds.l for 97e7edf2-71fb-11ef-bceb-c7b262605968... 2024-09-13T18:19:26.882 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 13 18:19:26 smithi204 podman[64121]: 2024-09-13 18:19:26.509439125 +0000 UTC m=+0.016008406 image pull 8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256 2024-09-13T18:19:26.882 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 13 18:19:26 smithi204 podman[64121]: 2024-09-13 18:19:26.644272187 +0000 UTC m=+0.150841455 container create 11004bae139144d39017446d0ecaab4ee5f777648e31522391fd08903a5d7703 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-l, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T18:19:27.198 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:26 smithi204 ceph-mon[34777]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:27.198 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:26 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/390251062' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:19:27.198 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 13 18:19:26 smithi204 podman[64121]: 2024-09-13 18:19:26.896653741 +0000 UTC m=+0.403223014 container init 11004bae139144d39017446d0ecaab4ee5f777648e31522391fd08903a5d7703 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-l, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-13T18:19:27.198 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 13 18:19:26 smithi204 podman[64121]: 2024-09-13 18:19:26.899581929 +0000 UTC m=+0.406151204 container start 11004bae139144d39017446d0ecaab4ee5f777648e31522391fd08903a5d7703 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-09-13T18:19:27.198 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 13 18:19:26 smithi204 ceph-mds[64140]: -- 172.21.15.204:0/431984895 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55fabad7ad00 con 0x55fabad4e800 2024-09-13T18:19:27.198 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 13 18:19:26 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-l[64136]: starting mds.l at 2024-09-13T18:19:27.199 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 13 18:19:27 smithi204 bash[64121]: 11004bae139144d39017446d0ecaab4ee5f777648e31522391fd08903a5d7703 2024-09-13T18:19:27.199 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 13 18:19:27 smithi204 systemd[1]: Started Ceph mds.l for 97e7edf2-71fb-11ef-bceb-c7b262605968. 2024-09-13T18:19:27.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:26 smithi114 ceph-mon[30897]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:27.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:26 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/390251062' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:19:27.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:26 smithi134 ceph-mon[34283]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:27.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:26 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/390251062' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:19:27.922 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:19:28.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:28.031+0000 7f41e8080640 1 -- 172.21.15.114:0/1975986139 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41e01031d0 msgr2=0x7f41e01055c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:28.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:28.031+0000 7f41e8080640 1 --2- 172.21.15.114:0/1975986139 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41e01031d0 0x7f41e01055c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f41d0009900 tx=0x7f41d002a410 comp rx=0 tx=0).stop 2024-09-13T18:19:28.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:28.031+0000 7f41e8080640 1 -- 172.21.15.114:0/1975986139 shutdown_connections 2024-09-13T18:19:28.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:28.031+0000 7f41e8080640 1 --2- 172.21.15.114:0/1975986139 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41e0105b00 0x7f41e0107ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:28.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:28.031+0000 7f41e8080640 1 --2- 172.21.15.114:0/1975986139 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41e01031d0 0x7f41e01055c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:28.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:28.031+0000 7f41e8080640 1 --2- 172.21.15.114:0/1975986139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41e01008a0 0x7f41e0102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:28.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:28.031+0000 7f41e8080640 1 -- 172.21.15.114:0/1975986139 >> 172.21.15.114:0/1975986139 conn(0x7f41e00fa440 msgr2=0x7f41e00fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:28.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:28.031+0000 7f41e8080640 1 -- 172.21.15.114:0/1975986139 shutdown_connections 2024-09-13T18:19:28.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:28.031+0000 7f41e8080640 1 -- 172.21.15.114:0/1975986139 wait complete. 2024-09-13T18:19:28.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:28.031+0000 7f41e8080640 1 Processor -- start 2024-09-13T18:19:28.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:28.032+0000 7f41e8080640 1 -- start start 2024-09-13T18:19:28.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:28.032+0000 7f41e8080640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41e01008a0 0x7f41e0195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:28.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:28.032+0000 7f41e8080640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41e01031d0 0x7f41e01962f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:28.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:28.032+0000 7f41e8080640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41e0105b00 0x7f41e019d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:28.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:28.032+0000 7f41e8080640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e019d890 con 0x7f41e01008a0 2024-09-13T18:19:28.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:28.032+0000 7f41e8080640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e019da00 con 0x7f41e0105b00 2024-09-13T18:19:28.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:28.032+0000 7f41e8080640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e019dbc0 con 0x7f41e01031d0 2024-09-13T18:19:28.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:28.032+0000 7f41e5e1c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41e01008a0 0x7f41e0195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:28.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:28.032+0000 7f41e5e1c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41e01008a0 0x7f41e0195db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42472/0 (socket says 172.21.15.114:42472) 2024-09-13T18:19:28.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:28.032+0000 7f41e5e1c640 1 -- 172.21.15.114:0/4032834453 learned_addr learned my addr 172.21.15.114:0/4032834453 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:19:28.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:28.032+0000 7f41e661d640 1 --2- 172.21.15.114:0/4032834453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41e0105b00 0x7f41e019d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:28.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:28.032+0000 7f41e561b640 1 --2- 172.21.15.114:0/4032834453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41e01031d0 0x7f41e01962f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:28.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:28.032+0000 7f41e5e1c640 1 -- 172.21.15.114:0/4032834453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41e01031d0 msgr2=0x7f41e01962f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:28.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:28.032+0000 7f41e5e1c640 1 --2- 172.21.15.114:0/4032834453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41e01031d0 0x7f41e01962f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:28.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:28.032+0000 7f41e5e1c640 1 -- 172.21.15.114:0/4032834453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41e0105b00 msgr2=0x7f41e019d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:28.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:28.032+0000 7f41e5e1c640 1 --2- 172.21.15.114:0/4032834453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41e0105b00 0x7f41e019d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:28.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:28.032+0000 7f41e5e1c640 1 -- 172.21.15.114:0/4032834453 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d0009560 con 0x7f41e01008a0 2024-09-13T18:19:28.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:28.032+0000 7f41e5e1c640 1 --2- 172.21.15.114:0/4032834453 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41e01008a0 0x7f41e0195db0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f41dc00e9f0 tx=0x7f41dc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:28.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:28.032+0000 7f41d6ffd640 1 -- 172.21.15.114:0/4032834453 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41dc009700 con 0x7f41e01008a0 2024-09-13T18:19:28.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:28.032+0000 7f41e8080640 1 -- 172.21.15.114:0/4032834453 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41e019dea0 con 0x7f41e01008a0 2024-09-13T18:19:28.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:28.032+0000 7f41d6ffd640 1 -- 172.21.15.114:0/4032834453 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f41dc00cd80 con 0x7f41e01008a0 2024-09-13T18:19:28.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:28.033+0000 7f41d6ffd640 1 -- 172.21.15.114:0/4032834453 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41dc0187e0 con 0x7f41e01008a0 2024-09-13T18:19:28.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:28.033+0000 7f41e8080640 1 -- 172.21.15.114:0/4032834453 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41e019e2c0 con 0x7f41e01008a0 2024-09-13T18:19:28.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:28.033+0000 7f41e8080640 1 -- 172.21.15.114:0/4032834453 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41e00fbf30 con 0x7f41e01008a0 2024-09-13T18:19:28.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:28.035+0000 7f41d6ffd640 1 -- 172.21.15.114:0/4032834453 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f41dc010430 con 0x7f41e01008a0 2024-09-13T18:19:28.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:28.035+0000 7f41d6ffd640 1 --2- 172.21.15.114:0/4032834453 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f41cc0a1090 0x7f41cc0a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:28.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:28.035+0000 7f41e561b640 1 --2- 172.21.15.114:0/4032834453 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f41cc0a1090 0x7f41cc0a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:28.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:28.035+0000 7f41d6ffd640 1 -- 172.21.15.114:0/4032834453 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f41dc014070 con 0x7f41e01008a0 2024-09-13T18:19:28.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:28.036+0000 7f41e561b640 1 --2- 172.21.15.114:0/4032834453 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f41cc0a1090 0x7f41cc0a3550 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f41d0006010 tx=0x7f41d0000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:28.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:28.039+0000 7f41d6ffd640 1 -- 172.21.15.114:0/4032834453 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f41dc052e20 con 0x7f41e01008a0 2024-09-13T18:19:28.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:27 smithi134 ceph-mon[34283]: mds.? [v2:172.21.15.204:6832/507301496,v1:172.21.15.204:6833/507301496] up:boot 2024-09-13T18:19:28.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:27 smithi134 ceph-mon[34283]: fsmap 1 up:standby 2024-09-13T18:19:28.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:27 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-13T18:19:28.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:27 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:28.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:27 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:28.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:27 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:28.120 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:27 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T18:19:28.120 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:27 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T18:19:28.120 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:27 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:19:28.120 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:27 smithi134 ceph-mon[34283]: Deploying daemon mds.k on smithi134 2024-09-13T18:19:28.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:27 smithi204 ceph-mon[34777]: mds.? [v2:172.21.15.204:6832/507301496,v1:172.21.15.204:6833/507301496] up:boot 2024-09-13T18:19:28.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:27 smithi204 ceph-mon[34777]: fsmap 1 up:standby 2024-09-13T18:19:28.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:27 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-13T18:19:28.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:27 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:28.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:27 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:28.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:27 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:28.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:27 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T18:19:28.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:27 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T18:19:28.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:27 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:19:28.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:27 smithi204 ceph-mon[34777]: Deploying daemon mds.k on smithi134 2024-09-13T18:19:28.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:27 smithi114 ceph-mon[30897]: mds.? [v2:172.21.15.204:6832/507301496,v1:172.21.15.204:6833/507301496] up:boot 2024-09-13T18:19:28.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:27 smithi114 ceph-mon[30897]: fsmap 1 up:standby 2024-09-13T18:19:28.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:27 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-13T18:19:28.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:27 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:28.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:27 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:28.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:27 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:28.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:27 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T18:19:28.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:27 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T18:19:28.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:27 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:19:28.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:27 smithi114 ceph-mon[30897]: Deploying daemon mds.k on smithi134 2024-09-13T18:19:28.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:28.237+0000 7f41e8080640 1 -- 172.21.15.114:0/4032834453 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f41e010b5a0 con 0x7f41e01008a0 2024-09-13T18:19:28.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:28.237+0000 7f41d6ffd640 1 -- 172.21.15.114:0/4032834453 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1257 (secure 0 0 0) 0x7f41dc096430 con 0x7f41e01008a0 2024-09-13T18:19:28.240 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:19:28.240 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/507301496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":507301496},{"type":"v1","addr":"172.21.15.204:6833","nonce":507301496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-09-13T18:19:28.240 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 2 2024-09-13T18:19:28.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:28.239+0000 7f41e8080640 1 -- 172.21.15.114:0/4032834453 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f41cc0a1090 msgr2=0x7f41cc0a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:28.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:28.239+0000 7f41e8080640 1 --2- 172.21.15.114:0/4032834453 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f41cc0a1090 0x7f41cc0a3550 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f41d0006010 tx=0x7f41d0000bc0 comp rx=0 tx=0).stop 2024-09-13T18:19:28.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:28.239+0000 7f41e8080640 1 -- 172.21.15.114:0/4032834453 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41e01008a0 msgr2=0x7f41e0195db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:28.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:28.239+0000 7f41e8080640 1 --2- 172.21.15.114:0/4032834453 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41e01008a0 0x7f41e0195db0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f41dc00e9f0 tx=0x7f41dc00eeb0 comp rx=0 tx=0).stop 2024-09-13T18:19:28.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:28.239+0000 7f41e8080640 1 -- 172.21.15.114:0/4032834453 shutdown_connections 2024-09-13T18:19:28.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:28.239+0000 7f41e8080640 1 --2- 172.21.15.114:0/4032834453 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f41cc0a1090 0x7f41cc0a3550 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:28.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:28.239+0000 7f41e8080640 1 --2- 172.21.15.114:0/4032834453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41e0105b00 0x7f41e019d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:28.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:28.239+0000 7f41e8080640 1 --2- 172.21.15.114:0/4032834453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41e01031d0 0x7f41e01962f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:28.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:28.239+0000 7f41e8080640 1 --2- 172.21.15.114:0/4032834453 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41e01008a0 0x7f41e0195db0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:28.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:28.239+0000 7f41e8080640 1 -- 172.21.15.114:0/4032834453 >> 172.21.15.114:0/4032834453 conn(0x7f41e00fa440 msgr2=0x7f41e01011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:28.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:28.239+0000 7f41e8080640 1 -- 172.21.15.114:0/4032834453 shutdown_connections 2024-09-13T18:19:28.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:28.239+0000 7f41e8080640 1 -- 172.21.15.114:0/4032834453 wait complete. 2024-09-13T18:19:29.236 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:29 smithi134 ceph-mon[34283]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:29.237 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:29 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/4032834453' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:19:29.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:29 smithi204 ceph-mon[34777]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:29.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:29 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/4032834453' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:19:29.469 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:29 smithi114 ceph-mon[30897]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:29.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:29 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/4032834453' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:19:29.869 INFO:journalctl@ceph.mds.k.smithi134.stdout:Sep 13 18:19:29 smithi134 systemd[1]: Started Ceph mds.k for 97e7edf2-71fb-11ef-bceb-c7b262605968. 2024-09-13T18:19:30.253 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:19:30.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.353+0000 7f2e88e6b640 1 -- 172.21.15.114:0/2533888607 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e840feca0 msgr2=0x7f2e840ff0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:30.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.353+0000 7f2e88e6b640 1 --2- 172.21.15.114:0/2533888607 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e840feca0 0x7f2e840ff0a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2e74009900 tx=0x7f2e7402a410 comp rx=0 tx=0).stop 2024-09-13T18:19:30.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.354+0000 7f2e88e6b640 1 -- 172.21.15.114:0/2533888607 shutdown_connections 2024-09-13T18:19:30.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.354+0000 7f2e88e6b640 1 --2- 172.21.15.114:0/2533888607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e84100850 0x7f2e841070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:30.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.354+0000 7f2e88e6b640 1 --2- 172.21.15.114:0/2533888607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e840ffe90 0x7f2e84100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:30.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.354+0000 7f2e88e6b640 1 --2- 172.21.15.114:0/2533888607 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e840feca0 0x7f2e840ff0a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:30.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.354+0000 7f2e88e6b640 1 -- 172.21.15.114:0/2533888607 >> 172.21.15.114:0/2533888607 conn(0x7f2e840fa440 msgr2=0x7f2e840fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:30.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.354+0000 7f2e88e6b640 1 -- 172.21.15.114:0/2533888607 shutdown_connections 2024-09-13T18:19:30.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.354+0000 7f2e88e6b640 1 -- 172.21.15.114:0/2533888607 wait complete. 2024-09-13T18:19:30.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.354+0000 7f2e88e6b640 1 Processor -- start 2024-09-13T18:19:30.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.354+0000 7f2e88e6b640 1 -- start start 2024-09-13T18:19:30.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.354+0000 7f2e88e6b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e840feca0 0x7f2e84194bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:30.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.355+0000 7f2e88e6b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e840ffe90 0x7f2e84195100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:30.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.355+0000 7f2e88e6b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e84100850 0x7f2e8419c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:30.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.355+0000 7f2e88e6b640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8419c6b0 con 0x7f2e840ffe90 2024-09-13T18:19:30.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.355+0000 7f2e88e6b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8419c820 con 0x7f2e840feca0 2024-09-13T18:19:30.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.355+0000 7f2e88e6b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8419c9e0 con 0x7f2e84100850 2024-09-13T18:19:30.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.355+0000 7f2e81d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e840ffe90 0x7f2e84195100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:30.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.355+0000 7f2e81d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e840ffe90 0x7f2e84195100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42484/0 (socket says 172.21.15.114:42484) 2024-09-13T18:19:30.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.355+0000 7f2e81d9b640 1 -- 172.21.15.114:0/3790151859 learned_addr learned my addr 172.21.15.114:0/3790151859 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:19:30.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.355+0000 7f2e8259c640 1 --2- 172.21.15.114:0/3790151859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e840feca0 0x7f2e84194bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:30.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.355+0000 7f2e82d9d640 1 --2- 172.21.15.114:0/3790151859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e84100850 0x7f2e8419c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:30.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.355+0000 7f2e81d9b640 1 -- 172.21.15.114:0/3790151859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e84100850 msgr2=0x7f2e8419c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:30.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.355+0000 7f2e81d9b640 1 --2- 172.21.15.114:0/3790151859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e84100850 0x7f2e8419c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:30.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.355+0000 7f2e81d9b640 1 -- 172.21.15.114:0/3790151859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e840feca0 msgr2=0x7f2e84194bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:30.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.355+0000 7f2e81d9b640 1 --2- 172.21.15.114:0/3790151859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e840feca0 0x7f2e84194bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:30.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.355+0000 7f2e81d9b640 1 -- 172.21.15.114:0/3790151859 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e74009560 con 0x7f2e840ffe90 2024-09-13T18:19:30.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.355+0000 7f2e8259c640 1 --2- 172.21.15.114:0/3790151859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e840feca0 0x7f2e84194bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:19:30.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.355+0000 7f2e81d9b640 1 --2- 172.21.15.114:0/3790151859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e840ffe90 0x7f2e84195100 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2e7800e9f0 tx=0x7f2e7800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:30.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.355+0000 7f2e737fe640 1 -- 172.21.15.114:0/3790151859 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e78009700 con 0x7f2e840ffe90 2024-09-13T18:19:30.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.355+0000 7f2e737fe640 1 -- 172.21.15.114:0/3790151859 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2e7800cd80 con 0x7f2e840ffe90 2024-09-13T18:19:30.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.355+0000 7f2e88e6b640 1 -- 172.21.15.114:0/3790151859 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e8419ccc0 con 0x7f2e840ffe90 2024-09-13T18:19:30.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.355+0000 7f2e737fe640 1 -- 172.21.15.114:0/3790151859 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e780187e0 con 0x7f2e840ffe90 2024-09-13T18:19:30.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.355+0000 7f2e88e6b640 1 -- 172.21.15.114:0/3790151859 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e8419d1d0 con 0x7f2e840ffe90 2024-09-13T18:19:30.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.356+0000 7f2e88e6b640 1 -- 172.21.15.114:0/3790151859 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e841061d0 con 0x7f2e840ffe90 2024-09-13T18:19:30.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.358+0000 7f2e737fe640 1 -- 172.21.15.114:0/3790151859 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f2e78010430 con 0x7f2e840ffe90 2024-09-13T18:19:30.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.359+0000 7f2e737fe640 1 --2- 172.21.15.114:0/3790151859 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f2e6c0a1110 0x7f2e6c0a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:30.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.359+0000 7f2e737fe640 1 -- 172.21.15.114:0/3790151859 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2e78014070 con 0x7f2e840ffe90 2024-09-13T18:19:30.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.359+0000 7f2e8259c640 1 --2- 172.21.15.114:0/3790151859 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f2e6c0a1110 0x7f2e6c0a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:30.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.359+0000 7f2e8259c640 1 --2- 172.21.15.114:0/3790151859 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f2e6c0a1110 0x7f2e6c0a35d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2e74005ab0 tx=0x7f2e74005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:30.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.364+0000 7f2e737fe640 1 -- 172.21.15.114:0/3790151859 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f2e78096930 con 0x7f2e840ffe90 2024-09-13T18:19:30.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.573+0000 7f2e88e6b640 1 -- 172.21.15.114:0/3790151859 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e8410a3f0 con 0x7f2e840ffe90 2024-09-13T18:19:30.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.574+0000 7f2e737fe640 1 -- 172.21.15.114:0/3790151859 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2017 (secure 0 0 0) 0x7f2e78099f50 con 0x7f2e840ffe90 2024-09-13T18:19:30.576 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:19:30.576 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/507301496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":507301496},{"type":"v1","addr":"172.21.15.204:6833","nonce":507301496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/4028657219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4028657219},{"type":"v1","addr":"172.21.15.134:6833","nonce":4028657219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-09-13T18:19:30.577 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 3 2024-09-13T18:19:30.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.576+0000 7f2e88e6b640 1 -- 172.21.15.114:0/3790151859 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f2e6c0a1110 msgr2=0x7f2e6c0a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:30.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.576+0000 7f2e88e6b640 1 --2- 172.21.15.114:0/3790151859 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f2e6c0a1110 0x7f2e6c0a35d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2e74005ab0 tx=0x7f2e74005a20 comp rx=0 tx=0).stop 2024-09-13T18:19:30.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.576+0000 7f2e88e6b640 1 -- 172.21.15.114:0/3790151859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e840ffe90 msgr2=0x7f2e84195100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:30.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.576+0000 7f2e88e6b640 1 --2- 172.21.15.114:0/3790151859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e840ffe90 0x7f2e84195100 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2e7800e9f0 tx=0x7f2e7800eeb0 comp rx=0 tx=0).stop 2024-09-13T18:19:30.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.576+0000 7f2e88e6b640 1 -- 172.21.15.114:0/3790151859 shutdown_connections 2024-09-13T18:19:30.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.576+0000 7f2e88e6b640 1 --2- 172.21.15.114:0/3790151859 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f2e6c0a1110 0x7f2e6c0a35d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:30.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.576+0000 7f2e88e6b640 1 --2- 172.21.15.114:0/3790151859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e84100850 0x7f2e8419c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:30.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.576+0000 7f2e88e6b640 1 --2- 172.21.15.114:0/3790151859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e840ffe90 0x7f2e84195100 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:30.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.576+0000 7f2e88e6b640 1 --2- 172.21.15.114:0/3790151859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e840feca0 0x7f2e84194bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:30.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.576+0000 7f2e88e6b640 1 -- 172.21.15.114:0/3790151859 >> 172.21.15.114:0/3790151859 conn(0x7f2e840fa440 msgr2=0x7f2e841033a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:30.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.576+0000 7f2e88e6b640 1 -- 172.21.15.114:0/3790151859 shutdown_connections 2024-09-13T18:19:30.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:30.576+0000 7f2e88e6b640 1 -- 172.21.15.114:0/3790151859 wait complete. 2024-09-13T18:19:31.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:30 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:31.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:30 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:31.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:30 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:31.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:30 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T18:19:31.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:30 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T18:19:31.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:30 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:19:31.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:30 smithi134 ceph-mon[34283]: Deploying daemon mds.j on smithi114 2024-09-13T18:19:31.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:30 smithi134 ceph-mon[34283]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:31.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:30 smithi134 ceph-mon[34283]: mds.? [v2:172.21.15.134:6832/4028657219,v1:172.21.15.134:6833/4028657219] up:boot 2024-09-13T18:19:31.120 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:30 smithi134 ceph-mon[34283]: fsmap 2 up:standby 2024-09-13T18:19:31.120 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:30 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-13T18:19:31.120 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:30 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/3790151859' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:19:31.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:30 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:31.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:30 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:31.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:30 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:31.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:30 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T18:19:31.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:30 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T18:19:31.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:30 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:19:31.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:30 smithi204 ceph-mon[34777]: Deploying daemon mds.j on smithi114 2024-09-13T18:19:31.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:30 smithi204 ceph-mon[34777]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:31.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:30 smithi204 ceph-mon[34777]: mds.? [v2:172.21.15.134:6832/4028657219,v1:172.21.15.134:6833/4028657219] up:boot 2024-09-13T18:19:31.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:30 smithi204 ceph-mon[34777]: fsmap 2 up:standby 2024-09-13T18:19:31.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:30 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-13T18:19:31.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:30 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/3790151859' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:19:31.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:30 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:31.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:30 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:31.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:30 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:31.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:30 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T18:19:31.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:30 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T18:19:31.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:30 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:19:31.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:30 smithi114 ceph-mon[30897]: Deploying daemon mds.j on smithi114 2024-09-13T18:19:31.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:30 smithi114 ceph-mon[30897]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:31.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:30 smithi114 ceph-mon[30897]: mds.? [v2:172.21.15.134:6832/4028657219,v1:172.21.15.134:6833/4028657219] up:boot 2024-09-13T18:19:31.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:30 smithi114 ceph-mon[30897]: fsmap 2 up:standby 2024-09-13T18:19:31.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:30 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-13T18:19:31.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:30 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/3790151859' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:19:31.970 INFO:journalctl@ceph.mds.j.smithi114.stdout:Sep 13 18:19:31 smithi114 systemd[1]: Starting Ceph mds.j for 97e7edf2-71fb-11ef-bceb-c7b262605968... 2024-09-13T18:19:32.338 INFO:journalctl@ceph.mds.j.smithi114.stdout:Sep 13 18:19:32 smithi114 podman[69907]: 2024-09-13 18:19:31.938483014 +0000 UTC m=+0.016109828 image pull 8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256 2024-09-13T18:19:32.338 INFO:journalctl@ceph.mds.j.smithi114.stdout:Sep 13 18:19:32 smithi114 podman[69907]: 2024-09-13 18:19:32.074769851 +0000 UTC m=+0.152396710 container create 4687664a4f360d8730d0b856f8dcb0ce2fdf89aaa5c1972491ef918dd1fed490 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-j, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, io.buildah.version=1.37.2) 2024-09-13T18:19:32.338 INFO:journalctl@ceph.mds.j.smithi114.stdout:Sep 13 18:19:32 smithi114 podman[69907]: 2024-09-13 18:19:32.336220175 +0000 UTC m=+0.413846987 container init 4687664a4f360d8730d0b856f8dcb0ce2fdf89aaa5c1972491ef918dd1fed490 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-j, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-13T18:19:32.589 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:19:32.613 INFO:journalctl@ceph.mds.j.smithi114.stdout:Sep 13 18:19:32 smithi114 podman[69907]: 2024-09-13 18:19:32.33920622 +0000 UTC m=+0.416833027 container start 4687664a4f360d8730d0b856f8dcb0ce2fdf89aaa5c1972491ef918dd1fed490 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-j, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-13T18:19:32.613 INFO:journalctl@ceph.mds.j.smithi114.stdout:Sep 13 18:19:32 smithi114 ceph-mds[69926]: -- 172.21.15.114:0/2456842261 <== mon.0 v2:172.21.15.114:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5577edef0d00 con 0x5577edec5400 2024-09-13T18:19:32.613 INFO:journalctl@ceph.mds.j.smithi114.stdout:Sep 13 18:19:32 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-j[69922]: starting mds.j at 2024-09-13T18:19:32.613 INFO:journalctl@ceph.mds.j.smithi114.stdout:Sep 13 18:19:32 smithi114 bash[69907]: 4687664a4f360d8730d0b856f8dcb0ce2fdf89aaa5c1972491ef918dd1fed490 2024-09-13T18:19:32.613 INFO:journalctl@ceph.mds.j.smithi114.stdout:Sep 13 18:19:32 smithi114 systemd[1]: Started Ceph mds.j for 97e7edf2-71fb-11ef-bceb-c7b262605968. 2024-09-13T18:19:32.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:32.698+0000 7fe39dae5640 1 -- 172.21.15.114:0/1633456488 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe398100820 msgr2=0x7fe398102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:32.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:32.698+0000 7fe39dae5640 1 --2- 172.21.15.114:0/1633456488 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe398100820 0x7fe398102c10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe38c009900 tx=0x7fe38c02a410 comp rx=0 tx=0).stop 2024-09-13T18:19:32.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:32.698+0000 7fe39dae5640 1 -- 172.21.15.114:0/1633456488 shutdown_connections 2024-09-13T18:19:32.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:32.698+0000 7fe39dae5640 1 --2- 172.21.15.114:0/1633456488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe398105a80 0x7fe398107ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:32.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:32.698+0000 7fe39dae5640 1 --2- 172.21.15.114:0/1633456488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe398103150 0x7fe398105540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:32.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:32.698+0000 7fe39dae5640 1 --2- 172.21.15.114:0/1633456488 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe398100820 0x7fe398102c10 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:32.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:32.698+0000 7fe39dae5640 1 -- 172.21.15.114:0/1633456488 >> 172.21.15.114:0/1633456488 conn(0x7fe3980fa440 msgr2=0x7fe3980fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:32.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:32.698+0000 7fe39dae5640 1 -- 172.21.15.114:0/1633456488 shutdown_connections 2024-09-13T18:19:32.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:32.698+0000 7fe39dae5640 1 -- 172.21.15.114:0/1633456488 wait complete. 2024-09-13T18:19:32.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:32.698+0000 7fe39dae5640 1 Processor -- start 2024-09-13T18:19:32.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:32.699+0000 7fe39dae5640 1 -- start start 2024-09-13T18:19:32.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:32.699+0000 7fe39dae5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe398100820 0x7fe398195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:32.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:32.699+0000 7fe39dae5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe398103150 0x7fe398196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:32.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:32.699+0000 7fe39dae5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe398105a80 0x7fe39819d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:32.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:32.699+0000 7fe39dae5640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe39819d830 con 0x7fe398100820 2024-09-13T18:19:32.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:32.699+0000 7fe39dae5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe39819d9a0 con 0x7fe398105a80 2024-09-13T18:19:32.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:32.699+0000 7fe39dae5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe39819db60 con 0x7fe398103150 2024-09-13T18:19:32.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:32.699+0000 7fe3977fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe398100820 0x7fe398195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:32.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:32.699+0000 7fe3977fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe398100820 0x7fe398195d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42530/0 (socket says 172.21.15.114:42530) 2024-09-13T18:19:32.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:32.699+0000 7fe3977fe640 1 -- 172.21.15.114:0/3521854849 learned_addr learned my addr 172.21.15.114:0/3521854849 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:19:32.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:32.699+0000 7fe397fff640 1 --2- 172.21.15.114:0/3521854849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe398105a80 0x7fe39819d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:32.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:32.699+0000 7fe396ffd640 1 --2- 172.21.15.114:0/3521854849 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe398103150 0x7fe398196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:32.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:32.699+0000 7fe3977fe640 1 -- 172.21.15.114:0/3521854849 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe398103150 msgr2=0x7fe398196280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:32.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:32.699+0000 7fe3977fe640 1 --2- 172.21.15.114:0/3521854849 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe398103150 0x7fe398196280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:32.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:32.699+0000 7fe3977fe640 1 -- 172.21.15.114:0/3521854849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe398105a80 msgr2=0x7fe39819d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:32.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:32.699+0000 7fe3977fe640 1 --2- 172.21.15.114:0/3521854849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe398105a80 0x7fe39819d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:32.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:32.699+0000 7fe3977fe640 1 -- 172.21.15.114:0/3521854849 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe38c009560 con 0x7fe398100820 2024-09-13T18:19:32.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:32.699+0000 7fe3977fe640 1 --2- 172.21.15.114:0/3521854849 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe398100820 0x7fe398195d40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe38c006010 tx=0x7fe38c0045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:32.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:32.700+0000 7fe394ff9640 1 -- 172.21.15.114:0/3521854849 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe38c004c50 con 0x7fe398100820 2024-09-13T18:19:32.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:32.700+0000 7fe394ff9640 1 -- 172.21.15.114:0/3521854849 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe38c004db0 con 0x7fe398100820 2024-09-13T18:19:32.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:32.700+0000 7fe39dae5640 1 -- 172.21.15.114:0/3521854849 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe39819dde0 con 0x7fe398100820 2024-09-13T18:19:32.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:32.700+0000 7fe394ff9640 1 -- 172.21.15.114:0/3521854849 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe38c03c900 con 0x7fe398100820 2024-09-13T18:19:32.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:32.700+0000 7fe39dae5640 1 -- 172.21.15.114:0/3521854849 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe39819e220 con 0x7fe398100820 2024-09-13T18:19:32.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:32.701+0000 7fe39dae5640 1 -- 172.21.15.114:0/3521854849 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3980fbf30 con 0x7fe398100820 2024-09-13T18:19:32.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:32.702+0000 7fe394ff9640 1 -- 172.21.15.114:0/3521854849 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fe38c003c80 con 0x7fe398100820 2024-09-13T18:19:32.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:32.703+0000 7fe394ff9640 1 --2- 172.21.15.114:0/3521854849 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fe3800a1110 0x7fe3800a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:32.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:32.703+0000 7fe396ffd640 1 --2- 172.21.15.114:0/3521854849 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fe3800a1110 0x7fe3800a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:32.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:32.703+0000 7fe394ff9640 1 -- 172.21.15.114:0/3521854849 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe38c0ed230 con 0x7fe398100820 2024-09-13T18:19:32.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:32.703+0000 7fe396ffd640 1 --2- 172.21.15.114:0/3521854849 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fe3800a1110 0x7fe3800a35d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe398197250 tx=0x7fe388005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:32.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:32.708+0000 7fe394ff9640 1 -- 172.21.15.114:0/3521854849 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fe38c0b6f10 con 0x7fe398100820 2024-09-13T18:19:32.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:32.913+0000 7fe39dae5640 1 -- 172.21.15.114:0/3521854849 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe39810b5d0 con 0x7fe398100820 2024-09-13T18:19:32.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:32.914+0000 7fe394ff9640 1 -- 172.21.15.114:0/3521854849 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2017 (secure 0 0 0) 0x7fe39810b5d0 con 0x7fe398100820 2024-09-13T18:19:32.916 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:19:32.916 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/507301496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":507301496},{"type":"v1","addr":"172.21.15.204:6833","nonce":507301496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/4028657219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4028657219},{"type":"v1","addr":"172.21.15.134:6833","nonce":4028657219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-09-13T18:19:32.916 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 3 2024-09-13T18:19:32.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:32.916+0000 7fe39dae5640 1 -- 172.21.15.114:0/3521854849 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fe3800a1110 msgr2=0x7fe3800a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:32.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:32.916+0000 7fe39dae5640 1 --2- 172.21.15.114:0/3521854849 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fe3800a1110 0x7fe3800a35d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe398197250 tx=0x7fe388005f90 comp rx=0 tx=0).stop 2024-09-13T18:19:32.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:32.916+0000 7fe39dae5640 1 -- 172.21.15.114:0/3521854849 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe398100820 msgr2=0x7fe398195d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:32.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:32.916+0000 7fe39dae5640 1 --2- 172.21.15.114:0/3521854849 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe398100820 0x7fe398195d40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe38c006010 tx=0x7fe38c0045b0 comp rx=0 tx=0).stop 2024-09-13T18:19:32.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:32.916+0000 7fe39dae5640 1 -- 172.21.15.114:0/3521854849 shutdown_connections 2024-09-13T18:19:32.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:32.916+0000 7fe39dae5640 1 --2- 172.21.15.114:0/3521854849 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fe3800a1110 0x7fe3800a35d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:32.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:32.916+0000 7fe39dae5640 1 --2- 172.21.15.114:0/3521854849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe398105a80 0x7fe39819d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:32.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:32.916+0000 7fe39dae5640 1 --2- 172.21.15.114:0/3521854849 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe398103150 0x7fe398196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:32.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:32.916+0000 7fe39dae5640 1 --2- 172.21.15.114:0/3521854849 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe398100820 0x7fe398195d40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:32.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:32.916+0000 7fe39dae5640 1 -- 172.21.15.114:0/3521854849 >> 172.21.15.114:0/3521854849 conn(0x7fe3980fa440 msgr2=0x7fe3981010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:32.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:32.916+0000 7fe39dae5640 1 -- 172.21.15.114:0/3521854849 shutdown_connections 2024-09-13T18:19:32.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:32.916+0000 7fe39dae5640 1 -- 172.21.15.114:0/3521854849 wait complete. 2024-09-13T18:19:33.175 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:32 smithi114 ceph-mon[30897]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:33.175 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:32 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:33.176 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:32 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:33.176 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:32 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:33.176 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:32 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T18:19:33.176 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:32 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T18:19:33.176 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:32 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:19:33.176 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:32 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/3521854849' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:19:33.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:32 smithi134 ceph-mon[34283]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:33.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:32 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:33.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:32 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:33.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:32 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:33.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:32 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T18:19:33.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:32 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T18:19:33.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:32 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:19:33.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:32 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/3521854849' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:19:33.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:32 smithi204 ceph-mon[34777]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:33.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:32 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:33.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:32 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:33.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:32 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:33.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:32 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T18:19:33.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:32 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T18:19:33.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:32 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:19:33.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:32 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/3521854849' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:19:34.191 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:33 smithi114 ceph-mon[30897]: Deploying daemon mds.d on smithi114 2024-09-13T18:19:34.191 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:33 smithi114 ceph-mon[30897]: mds.? [v2:172.21.15.114:6834/4162817497,v1:172.21.15.114:6835/4162817497] up:boot 2024-09-13T18:19:34.191 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:33 smithi114 ceph-mon[30897]: fsmap 3 up:standby 2024-09-13T18:19:34.191 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:33 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-13T18:19:34.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:33 smithi134 ceph-mon[34283]: Deploying daemon mds.d on smithi114 2024-09-13T18:19:34.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:33 smithi134 ceph-mon[34283]: mds.? [v2:172.21.15.114:6834/4162817497,v1:172.21.15.114:6835/4162817497] up:boot 2024-09-13T18:19:34.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:33 smithi134 ceph-mon[34283]: fsmap 3 up:standby 2024-09-13T18:19:34.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:33 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-13T18:19:34.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:33 smithi204 ceph-mon[34777]: Deploying daemon mds.d on smithi114 2024-09-13T18:19:34.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:33 smithi204 ceph-mon[34777]: mds.? [v2:172.21.15.114:6834/4162817497,v1:172.21.15.114:6835/4162817497] up:boot 2024-09-13T18:19:34.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:33 smithi204 ceph-mon[34777]: fsmap 3 up:standby 2024-09-13T18:19:34.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:33 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-13T18:19:34.470 INFO:journalctl@ceph.mds.d.smithi114.stdout:Sep 13 18:19:34 smithi114 systemd[1]: Starting Ceph mds.d for 97e7edf2-71fb-11ef-bceb-c7b262605968... 2024-09-13T18:19:34.894 INFO:journalctl@ceph.mds.d.smithi114.stdout:Sep 13 18:19:34 smithi114 podman[70251]: 2024-09-13 18:19:34.481621286 +0000 UTC m=+0.016530004 image pull 8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256 2024-09-13T18:19:34.894 INFO:journalctl@ceph.mds.d.smithi114.stdout:Sep 13 18:19:34 smithi114 podman[70251]: 2024-09-13 18:19:34.597913034 +0000 UTC m=+0.132821734 container create d50a04dd9db45efb7b47f511f3d0e4d5cc8dbf5579fefb3f954b6a5c4a03b01d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-d, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-13T18:19:34.894 INFO:journalctl@ceph.mds.d.smithi114.stdout:Sep 13 18:19:34 smithi114 podman[70251]: 2024-09-13 18:19:34.892371042 +0000 UTC m=+0.427279746 container init d50a04dd9db45efb7b47f511f3d0e4d5cc8dbf5579fefb3f954b6a5c4a03b01d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-d, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-13T18:19:34.929 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:19:35.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:35.037+0000 7f88fa86c640 1 -- 172.21.15.114:0/3380265582 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88f4100f80 msgr2=0x7f88f4101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:35.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:35.037+0000 7f88fa86c640 1 --2- 172.21.15.114:0/3380265582 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88f4100f80 0x7f88f4101400 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f88dc009900 tx=0x7f88dc02a410 comp rx=0 tx=0).stop 2024-09-13T18:19:35.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:35.037+0000 7f88fa86c640 1 -- 172.21.15.114:0/3380265582 shutdown_connections 2024-09-13T18:19:35.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:35.037+0000 7f88fa86c640 1 --2- 172.21.15.114:0/3380265582 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88f4101940 0x7f88f41081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:35.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:35.037+0000 7f88fa86c640 1 --2- 172.21.15.114:0/3380265582 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88f4100f80 0x7f88f4101400 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:35.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:35.037+0000 7f88fa86c640 1 --2- 172.21.15.114:0/3380265582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88f40ffd90 0x7f88f4100190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:35.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:35.037+0000 7f88fa86c640 1 -- 172.21.15.114:0/3380265582 >> 172.21.15.114:0/3380265582 conn(0x7f88f40fb510 msgr2=0x7f88f40fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:35.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:35.037+0000 7f88fa86c640 1 -- 172.21.15.114:0/3380265582 shutdown_connections 2024-09-13T18:19:35.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:35.037+0000 7f88fa86c640 1 -- 172.21.15.114:0/3380265582 wait complete. 2024-09-13T18:19:35.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:35.037+0000 7f88fa86c640 1 Processor -- start 2024-09-13T18:19:35.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:35.037+0000 7f88fa86c640 1 -- start start 2024-09-13T18:19:35.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:35.037+0000 7f88fa86c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88f40ffd90 0x7f88f4074260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:35.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:35.037+0000 7f88fa86c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88f4100f80 0x7f88f40728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:35.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:35.037+0000 7f88fa86c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88f4101940 0x7f88f4072de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:35.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:35.037+0000 7f88fa86c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f4073540 con 0x7f88f4101940 2024-09-13T18:19:35.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:35.037+0000 7f88fa86c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f40736b0 con 0x7f88f40ffd90 2024-09-13T18:19:35.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:35.037+0000 7f88fa86c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f419fcd0 con 0x7f88f4100f80 2024-09-13T18:19:35.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:35.038+0000 7f88f8e09640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88f4101940 0x7f88f4072de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:35.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:35.038+0000 7f88f8e09640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88f4101940 0x7f88f4072de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57484/0 (socket says 172.21.15.114:57484) 2024-09-13T18:19:35.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:35.038+0000 7f88f8e09640 1 -- 172.21.15.114:0/3464786635 learned_addr learned my addr 172.21.15.114:0/3464786635 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:19:35.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:35.038+0000 7f88ebfff640 1 --2- 172.21.15.114:0/3464786635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88f4100f80 0x7f88f40728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:35.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:35.038+0000 7f88f3fff640 1 --2- 172.21.15.114:0/3464786635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88f40ffd90 0x7f88f4074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:35.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:35.038+0000 7f88f8e09640 1 -- 172.21.15.114:0/3464786635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88f4100f80 msgr2=0x7f88f40728a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:35.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:35.038+0000 7f88f8e09640 1 --2- 172.21.15.114:0/3464786635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88f4100f80 0x7f88f40728a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:35.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:35.038+0000 7f88f8e09640 1 -- 172.21.15.114:0/3464786635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88f40ffd90 msgr2=0x7f88f4074260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:35.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:35.038+0000 7f88f8e09640 1 --2- 172.21.15.114:0/3464786635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88f40ffd90 0x7f88f4074260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:35.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:35.038+0000 7f88f8e09640 1 -- 172.21.15.114:0/3464786635 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88dc009560 con 0x7f88f4101940 2024-09-13T18:19:35.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:35.038+0000 7f88f8e09640 1 --2- 172.21.15.114:0/3464786635 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88f4101940 0x7f88f4072de0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f88ec00eb40 tx=0x7f88ec00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:35.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:35.039+0000 7f88f1ffb640 1 -- 172.21.15.114:0/3464786635 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ec004b20 con 0x7f88f4101940 2024-09-13T18:19:35.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:35.040+0000 7f88f1ffb640 1 -- 172.21.15.114:0/3464786635 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f88ec004c80 con 0x7f88f4101940 2024-09-13T18:19:35.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:35.040+0000 7f88fa86c640 1 -- 172.21.15.114:0/3464786635 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88f419ff80 con 0x7f88f4101940 2024-09-13T18:19:35.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:35.040+0000 7f88f1ffb640 1 -- 172.21.15.114:0/3464786635 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ec010690 con 0x7f88f4101940 2024-09-13T18:19:35.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:35.040+0000 7f88fa86c640 1 -- 172.21.15.114:0/3464786635 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88f41a0400 con 0x7f88f4101940 2024-09-13T18:19:35.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:35.041+0000 7f88fa86c640 1 -- 172.21.15.114:0/3464786635 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88f4107260 con 0x7f88f4101940 2024-09-13T18:19:35.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:35.042+0000 7f88f1ffb640 1 -- 172.21.15.114:0/3464786635 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f88ec004fd0 con 0x7f88f4101940 2024-09-13T18:19:35.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:35.042+0000 7f88f1ffb640 1 --2- 172.21.15.114:0/3464786635 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f88e00a1090 0x7f88e00a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:35.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:35.043+0000 7f88f3fff640 1 --2- 172.21.15.114:0/3464786635 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f88e00a1090 0x7f88e00a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:35.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:35.043+0000 7f88f1ffb640 1 -- 172.21.15.114:0/3464786635 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f88ec014070 con 0x7f88f4101940 2024-09-13T18:19:35.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:35.043+0000 7f88f3fff640 1 --2- 172.21.15.114:0/3464786635 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f88e00a1090 0x7f88e00a3550 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f88f4100de0 tx=0x7f88e4005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:35.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:35.046+0000 7f88f1ffb640 1 -- 172.21.15.114:0/3464786635 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f88ec096aa0 con 0x7f88f4101940 2024-09-13T18:19:35.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:34 smithi114 ceph-mon[30897]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:35.220 INFO:journalctl@ceph.mds.d.smithi114.stdout:Sep 13 18:19:34 smithi114 podman[70251]: 2024-09-13 18:19:34.895232529 +0000 UTC m=+0.430141242 container start d50a04dd9db45efb7b47f511f3d0e4d5cc8dbf5579fefb3f954b6a5c4a03b01d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-d, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-13T18:19:35.220 INFO:journalctl@ceph.mds.d.smithi114.stdout:Sep 13 18:19:34 smithi114 ceph-mds[70270]: -- 172.21.15.114:0/1409442773 <== mon.0 v2:172.21.15.114:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560f5370ed00 con 0x560f536e3000 2024-09-13T18:19:35.220 INFO:journalctl@ceph.mds.d.smithi114.stdout:Sep 13 18:19:34 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-d[70266]: starting mds.d at 2024-09-13T18:19:35.220 INFO:journalctl@ceph.mds.d.smithi114.stdout:Sep 13 18:19:34 smithi114 bash[70251]: d50a04dd9db45efb7b47f511f3d0e4d5cc8dbf5579fefb3f954b6a5c4a03b01d 2024-09-13T18:19:35.220 INFO:journalctl@ceph.mds.d.smithi114.stdout:Sep 13 18:19:34 smithi114 systemd[1]: Started Ceph mds.d for 97e7edf2-71fb-11ef-bceb-c7b262605968. 2024-09-13T18:19:35.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:35.243+0000 7f88fa86c640 1 -- 172.21.15.114:0/3464786635 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f88f410b500 con 0x7f88f4101940 2024-09-13T18:19:35.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:35.244+0000 7f88f1ffb640 1 -- 172.21.15.114:0/3464786635 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3537 (secure 0 0 0) 0x7f88ec09a0c0 con 0x7f88f4101940 2024-09-13T18:19:35.247 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:19:35.247 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4162817497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4162817497},{"type":"v1","addr":"172.21.15.114:6835","nonce":4162817497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/2435428478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":2435428478},{"type":"v1","addr":"172.21.15.114:6837","nonce":2435428478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/507301496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":507301496},{"type":"v1","addr":"172.21.15.204:6833","nonce":507301496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/4028657219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4028657219},{"type":"v1","addr":"172.21.15.134:6833","nonce":4028657219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-09-13T18:19:35.247 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 5 2024-09-13T18:19:35.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:35.246+0000 7f88fa86c640 1 -- 172.21.15.114:0/3464786635 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f88e00a1090 msgr2=0x7f88e00a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:35.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:35.246+0000 7f88fa86c640 1 --2- 172.21.15.114:0/3464786635 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f88e00a1090 0x7f88e00a3550 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f88f4100de0 tx=0x7f88e4005f90 comp rx=0 tx=0).stop 2024-09-13T18:19:35.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:35.246+0000 7f88fa86c640 1 -- 172.21.15.114:0/3464786635 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88f4101940 msgr2=0x7f88f4072de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:35.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:35.246+0000 7f88fa86c640 1 --2- 172.21.15.114:0/3464786635 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88f4101940 0x7f88f4072de0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f88ec00eb40 tx=0x7f88ec00c2d0 comp rx=0 tx=0).stop 2024-09-13T18:19:35.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:35.246+0000 7f88fa86c640 1 -- 172.21.15.114:0/3464786635 shutdown_connections 2024-09-13T18:19:35.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:35.246+0000 7f88fa86c640 1 --2- 172.21.15.114:0/3464786635 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f88e00a1090 0x7f88e00a3550 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:35.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:35.246+0000 7f88fa86c640 1 --2- 172.21.15.114:0/3464786635 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88f4101940 0x7f88f4072de0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:35.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:35.246+0000 7f88fa86c640 1 --2- 172.21.15.114:0/3464786635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88f4100f80 0x7f88f40728a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:35.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:35.246+0000 7f88fa86c640 1 --2- 172.21.15.114:0/3464786635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88f40ffd90 0x7f88f4074260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:35.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:35.246+0000 7f88fa86c640 1 -- 172.21.15.114:0/3464786635 >> 172.21.15.114:0/3464786635 conn(0x7f88f40fb510 msgr2=0x7f88f4104430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:35.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:35.246+0000 7f88fa86c640 1 -- 172.21.15.114:0/3464786635 shutdown_connections 2024-09-13T18:19:35.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:35.246+0000 7f88fa86c640 1 -- 172.21.15.114:0/3464786635 wait complete. 2024-09-13T18:19:35.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:34 smithi134 ceph-mon[34283]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:35.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:34 smithi204 ceph-mon[34777]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:36.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:35 smithi134 ceph-mon[34283]: mds.? [v2:172.21.15.114:6836/2435428478,v1:172.21.15.114:6837/2435428478] up:boot 2024-09-13T18:19:36.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:35 smithi134 ceph-mon[34283]: fsmap 4 up:standby 2024-09-13T18:19:36.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:35 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-13T18:19:36.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:35 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:36.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:35 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:36.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:35 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:36.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:35 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T18:19:36.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:35 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T18:19:36.120 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:35 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:19:36.120 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:35 smithi134 ceph-mon[34283]: Deploying daemon mds.e on smithi134 2024-09-13T18:19:36.120 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:35 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/3464786635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:19:36.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:35 smithi114 ceph-mon[30897]: mds.? [v2:172.21.15.114:6836/2435428478,v1:172.21.15.114:6837/2435428478] up:boot 2024-09-13T18:19:36.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:35 smithi114 ceph-mon[30897]: fsmap 4 up:standby 2024-09-13T18:19:36.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:35 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-13T18:19:36.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:35 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:36.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:35 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:36.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:35 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:36.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:35 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T18:19:36.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:35 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T18:19:36.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:35 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:19:36.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:35 smithi114 ceph-mon[30897]: Deploying daemon mds.e on smithi134 2024-09-13T18:19:36.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:35 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/3464786635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:19:36.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:35 smithi204 ceph-mon[34777]: mds.? [v2:172.21.15.114:6836/2435428478,v1:172.21.15.114:6837/2435428478] up:boot 2024-09-13T18:19:36.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:35 smithi204 ceph-mon[34777]: fsmap 4 up:standby 2024-09-13T18:19:36.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:35 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-13T18:19:36.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:35 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:36.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:35 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:36.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:35 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:36.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:35 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T18:19:36.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:35 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T18:19:36.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:35 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:19:36.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:35 smithi204 ceph-mon[34777]: Deploying daemon mds.e on smithi134 2024-09-13T18:19:36.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:35 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/3464786635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:19:37.261 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:19:37.283 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:37 smithi134 ceph-mon[34283]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:37.283 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 13 18:19:37 smithi134 systemd[1]: Starting Ceph mds.e for 97e7edf2-71fb-11ef-bceb-c7b262605968... 2024-09-13T18:19:37.346 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:37 smithi204 ceph-mon[34777]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:37.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.369+0000 7f624318a640 1 -- 172.21.15.114:0/1035355465 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f623c0ff8b0 msgr2=0x7f623c107150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:37.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.369+0000 7f624318a640 1 --2- 172.21.15.114:0/1035355465 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f623c0ff8b0 0x7f623c107150 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6234009ca0 tx=0x7f623402a8b0 comp rx=0 tx=0).stop 2024-09-13T18:19:37.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.369+0000 7f624318a640 1 -- 172.21.15.114:0/1035355465 shutdown_connections 2024-09-13T18:19:37.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.369+0000 7f624318a640 1 --2- 172.21.15.114:0/1035355465 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f623c0ff8b0 0x7f623c107150 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:37.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.369+0000 7f624318a640 1 --2- 172.21.15.114:0/1035355465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f623c0feef0 0x7f623c0ff370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:37.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.369+0000 7f624318a640 1 --2- 172.21.15.114:0/1035355465 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f623c100330 0x7f623c100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:37.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.369+0000 7f624318a640 1 -- 172.21.15.114:0/1035355465 >> 172.21.15.114:0/1035355465 conn(0x7f623c0f9bb0 msgr2=0x7f623c0fbfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:37.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.369+0000 7f624318a640 1 -- 172.21.15.114:0/1035355465 shutdown_connections 2024-09-13T18:19:37.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.369+0000 7f624318a640 1 -- 172.21.15.114:0/1035355465 wait complete. 2024-09-13T18:19:37.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.370+0000 7f624318a640 1 Processor -- start 2024-09-13T18:19:37.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.370+0000 7f624318a640 1 -- start start 2024-09-13T18:19:37.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.370+0000 7f624318a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f623c0feef0 0x7f623c0fe950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:37.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.370+0000 7f624318a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f623c0ff8b0 0x7f623c19a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:37.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.370+0000 7f624318a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f623c100330 0x7f623c19d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:37.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.370+0000 7f624318a640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623c19d7f0 con 0x7f623c100330 2024-09-13T18:19:37.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.370+0000 7f624318a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623c19d960 con 0x7f623c0feef0 2024-09-13T18:19:37.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.370+0000 7f624318a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623c19db20 con 0x7f623c0ff8b0 2024-09-13T18:19:37.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.370+0000 7f6241727640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f623c100330 0x7f623c19d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:37.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.370+0000 7f6240f26640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f623c0feef0 0x7f623c0fe950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:37.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.370+0000 7f6241727640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f623c100330 0x7f623c19d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57490/0 (socket says 172.21.15.114:57490) 2024-09-13T18:19:37.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.370+0000 7f6241727640 1 -- 172.21.15.114:0/133178098 learned_addr learned my addr 172.21.15.114:0/133178098 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:19:37.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.370+0000 7f623bfff640 1 --2- 172.21.15.114:0/133178098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f623c0ff8b0 0x7f623c19a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:37.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.370+0000 7f6240f26640 1 -- 172.21.15.114:0/133178098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f623c0ff8b0 msgr2=0x7f623c19a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:37.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.370+0000 7f6240f26640 1 --2- 172.21.15.114:0/133178098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f623c0ff8b0 0x7f623c19a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:37.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.370+0000 7f6240f26640 1 -- 172.21.15.114:0/133178098 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f623c100330 msgr2=0x7f623c19d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:37.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.370+0000 7f6240f26640 1 --2- 172.21.15.114:0/133178098 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f623c100330 0x7f623c19d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:37.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.370+0000 7f6240f26640 1 -- 172.21.15.114:0/133178098 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6228009490 con 0x7f623c0feef0 2024-09-13T18:19:37.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.370+0000 7f6241727640 1 --2- 172.21.15.114:0/133178098 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f623c100330 0x7f623c19d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:19:37.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.371+0000 7f6240f26640 1 --2- 172.21.15.114:0/133178098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f623c0feef0 0x7f623c0fe950 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f623000e9f0 tx=0x7f623000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:37.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.371+0000 7f6239ffb640 1 -- 172.21.15.114:0/133178098 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623000ccc0 con 0x7f623c0feef0 2024-09-13T18:19:37.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.371+0000 7f6239ffb640 1 -- 172.21.15.114:0/133178098 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f623000ce20 con 0x7f623c0feef0 2024-09-13T18:19:37.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.371+0000 7f624318a640 1 -- 172.21.15.114:0/133178098 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6234009900 con 0x7f623c0feef0 2024-09-13T18:19:37.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.371+0000 7f6239ffb640 1 -- 172.21.15.114:0/133178098 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6230018750 con 0x7f623c0feef0 2024-09-13T18:19:37.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.371+0000 7f624318a640 1 -- 172.21.15.114:0/133178098 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f623c19e120 con 0x7f623c0feef0 2024-09-13T18:19:37.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.372+0000 7f624318a640 1 -- 172.21.15.114:0/133178098 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f623c106240 con 0x7f623c0feef0 2024-09-13T18:19:37.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.373+0000 7f6239ffb640 1 -- 172.21.15.114:0/133178098 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f6230010430 con 0x7f623c0feef0 2024-09-13T18:19:37.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.374+0000 7f6239ffb640 1 --2- 172.21.15.114:0/133178098 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f622c0a1090 0x7f622c0a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:37.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.374+0000 7f6239ffb640 1 -- 172.21.15.114:0/133178098 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6230014070 con 0x7f623c0feef0 2024-09-13T18:19:37.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.374+0000 7f623bfff640 1 --2- 172.21.15.114:0/133178098 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f622c0a1090 0x7f622c0a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:37.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.374+0000 7f623bfff640 1 --2- 172.21.15.114:0/133178098 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f622c0a1090 0x7f622c0a3550 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f623c0fe6c0 tx=0x7f6228005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:37.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.377+0000 7f6239ffb640 1 -- 172.21.15.114:0/133178098 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f6230096410 con 0x7f623c0feef0 2024-09-13T18:19:37.469 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:37 smithi114 ceph-mon[30897]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:37.556 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 13 18:19:37 smithi134 podman[63274]: 2024-09-13 18:19:37.183109412 +0000 UTC m=+0.016012947 image pull 8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256 2024-09-13T18:19:37.557 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 13 18:19:37 smithi134 podman[63274]: 2024-09-13 18:19:37.30211981 +0000 UTC m=+0.135023339 container create b4bc60a2e73a3a2152ec93200f3558c89a41607330006d53a2a267405432ff36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-e, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-13T18:19:37.557 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 13 18:19:37 smithi134 podman[63274]: 2024-09-13 18:19:37.554829978 +0000 UTC m=+0.387733511 container init b4bc60a2e73a3a2152ec93200f3558c89a41607330006d53a2a267405432ff36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083) 2024-09-13T18:19:37.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.577+0000 7f624318a640 1 -- 172.21.15.114:0/133178098 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f623c10a430 con 0x7f623c0feef0 2024-09-13T18:19:37.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.577+0000 7f6239ffb640 1 -- 172.21.15.114:0/133178098 <== mon.2 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3537 (secure 0 0 0) 0x7f6230099a30 con 0x7f623c0feef0 2024-09-13T18:19:37.581 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:19:37.581 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4162817497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4162817497},{"type":"v1","addr":"172.21.15.114:6835","nonce":4162817497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/2435428478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":2435428478},{"type":"v1","addr":"172.21.15.114:6837","nonce":2435428478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/507301496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":507301496},{"type":"v1","addr":"172.21.15.204:6833","nonce":507301496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/4028657219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4028657219},{"type":"v1","addr":"172.21.15.134:6833","nonce":4028657219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-09-13T18:19:37.581 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 5 2024-09-13T18:19:37.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.579+0000 7f624318a640 1 -- 172.21.15.114:0/133178098 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f622c0a1090 msgr2=0x7f622c0a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:37.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.579+0000 7f624318a640 1 --2- 172.21.15.114:0/133178098 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f622c0a1090 0x7f622c0a3550 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f623c0fe6c0 tx=0x7f6228005f90 comp rx=0 tx=0).stop 2024-09-13T18:19:37.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.579+0000 7f624318a640 1 -- 172.21.15.114:0/133178098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f623c0feef0 msgr2=0x7f623c0fe950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:37.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.579+0000 7f624318a640 1 --2- 172.21.15.114:0/133178098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f623c0feef0 0x7f623c0fe950 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f623000e9f0 tx=0x7f623000eeb0 comp rx=0 tx=0).stop 2024-09-13T18:19:37.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.580+0000 7f624318a640 1 -- 172.21.15.114:0/133178098 shutdown_connections 2024-09-13T18:19:37.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.580+0000 7f624318a640 1 --2- 172.21.15.114:0/133178098 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f622c0a1090 0x7f622c0a3550 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:37.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.580+0000 7f624318a640 1 --2- 172.21.15.114:0/133178098 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f623c100330 0x7f623c19d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:37.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.580+0000 7f624318a640 1 --2- 172.21.15.114:0/133178098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f623c0ff8b0 0x7f623c19a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:37.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.580+0000 7f624318a640 1 --2- 172.21.15.114:0/133178098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f623c0feef0 0x7f623c0fe950 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:37.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.580+0000 7f624318a640 1 -- 172.21.15.114:0/133178098 >> 172.21.15.114:0/133178098 conn(0x7f623c0f9bb0 msgr2=0x7f623c103410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:37.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.580+0000 7f624318a640 1 -- 172.21.15.114:0/133178098 shutdown_connections 2024-09-13T18:19:37.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:37.580+0000 7f624318a640 1 -- 172.21.15.114:0/133178098 wait complete. 2024-09-13T18:19:37.869 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 13 18:19:37 smithi134 podman[63274]: 2024-09-13 18:19:37.558036351 +0000 UTC m=+0.390939887 container start b4bc60a2e73a3a2152ec93200f3558c89a41607330006d53a2a267405432ff36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-e, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-13T18:19:37.869 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 13 18:19:37 smithi134 ceph-mds[63293]: -- 172.21.15.134:0/882139409 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5617c452ad00 con 0x5617c44fe800 2024-09-13T18:19:37.869 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 13 18:19:37 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-e[63289]: starting mds.e at 2024-09-13T18:19:37.869 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 13 18:19:37 smithi134 bash[63274]: b4bc60a2e73a3a2152ec93200f3558c89a41607330006d53a2a267405432ff36 2024-09-13T18:19:37.869 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 13 18:19:37 smithi134 systemd[1]: Started Ceph mds.e for 97e7edf2-71fb-11ef-bceb-c7b262605968. 2024-09-13T18:19:38.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:38 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/133178098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:19:38.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:38 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:38.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:38 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:38.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:38 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:38.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:38 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T18:19:38.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:38 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T18:19:38.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:38 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:19:38.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:38 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/133178098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:19:38.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:38 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:38.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:38 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:38.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:38 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:38.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:38 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T18:19:38.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:38 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T18:19:38.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:38 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:19:38.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:38 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/133178098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:19:38.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:38 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:38.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:38 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:38.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:38 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:38.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:38 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T18:19:38.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:38 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T18:19:38.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:38 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:19:39.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:39 smithi134 ceph-mon[34283]: Deploying daemon mds.f on smithi204 2024-09-13T18:19:39.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:39 smithi134 ceph-mon[34283]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:39.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:39 smithi134 ceph-mon[34283]: mds.? [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] up:boot 2024-09-13T18:19:39.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:39 smithi134 ceph-mon[34283]: fsmap 5 up:standby 2024-09-13T18:19:39.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:39 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-13T18:19:39.399 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:39 smithi204 ceph-mon[34777]: Deploying daemon mds.f on smithi204 2024-09-13T18:19:39.399 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:39 smithi204 ceph-mon[34777]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:39.399 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:39 smithi204 ceph-mon[34777]: mds.? [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] up:boot 2024-09-13T18:19:39.400 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:39 smithi204 ceph-mon[34777]: fsmap 5 up:standby 2024-09-13T18:19:39.400 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:39 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-13T18:19:39.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:39 smithi114 ceph-mon[30897]: Deploying daemon mds.f on smithi204 2024-09-13T18:19:39.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:39 smithi114 ceph-mon[30897]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:39.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:39 smithi114 ceph-mon[30897]: mds.? [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] up:boot 2024-09-13T18:19:39.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:39 smithi114 ceph-mon[30897]: fsmap 5 up:standby 2024-09-13T18:19:39.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:39 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-13T18:19:39.595 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:19:39.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.703+0000 7f4ca959c640 1 -- 172.21.15.114:0/4234513580 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ca4100850 msgr2=0x7f4ca41070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:39.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.703+0000 7f4ca959c640 1 --2- 172.21.15.114:0/4234513580 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ca4100850 0x7f4ca41070e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4c94009ca0 tx=0x7f4c9402a8c0 comp rx=0 tx=0).stop 2024-09-13T18:19:39.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.703+0000 7f4ca959c640 1 -- 172.21.15.114:0/4234513580 shutdown_connections 2024-09-13T18:19:39.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.703+0000 7f4ca959c640 1 --2- 172.21.15.114:0/4234513580 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ca4100850 0x7f4ca41070e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:39.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.703+0000 7f4ca959c640 1 --2- 172.21.15.114:0/4234513580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ca40ffe90 0x7f4ca4100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:39.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.703+0000 7f4ca959c640 1 --2- 172.21.15.114:0/4234513580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ca40feca0 0x7f4ca40ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:39.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.703+0000 7f4ca959c640 1 -- 172.21.15.114:0/4234513580 >> 172.21.15.114:0/4234513580 conn(0x7f4ca40fa400 msgr2=0x7f4ca40fc840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:39.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.703+0000 7f4ca959c640 1 -- 172.21.15.114:0/4234513580 shutdown_connections 2024-09-13T18:19:39.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.703+0000 7f4ca959c640 1 -- 172.21.15.114:0/4234513580 wait complete. 2024-09-13T18:19:39.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.704+0000 7f4ca959c640 1 Processor -- start 2024-09-13T18:19:39.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.704+0000 7f4ca959c640 1 -- start start 2024-09-13T18:19:39.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.704+0000 7f4ca959c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ca40feca0 0x7f4ca4102dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:39.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.704+0000 7f4ca959c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ca40ffe90 0x7f4ca4101400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:39.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.704+0000 7f4ca959c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ca4100850 0x7f4ca4101940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:39.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.704+0000 7f4ca959c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca41020a0 con 0x7f4ca4100850 2024-09-13T18:19:39.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.704+0000 7f4ca959c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca4102210 con 0x7f4ca40ffe90 2024-09-13T18:19:39.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.704+0000 7f4ca959c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca419fce0 con 0x7f4ca40feca0 2024-09-13T18:19:39.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.704+0000 7f4ca37fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ca4100850 0x7f4ca4101940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:39.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.704+0000 7f4ca2ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ca40feca0 0x7f4ca4102dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:39.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.704+0000 7f4ca37fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ca4100850 0x7f4ca4101940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57506/0 (socket says 172.21.15.114:57506) 2024-09-13T18:19:39.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.704+0000 7f4ca37fe640 1 -- 172.21.15.114:0/1478172367 learned_addr learned my addr 172.21.15.114:0/1478172367 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:19:39.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.705+0000 7f4ca27fc640 1 --2- 172.21.15.114:0/1478172367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ca40ffe90 0x7f4ca4101400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:39.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.705+0000 7f4ca37fe640 1 -- 172.21.15.114:0/1478172367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ca40feca0 msgr2=0x7f4ca4102dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:39.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.705+0000 7f4ca37fe640 1 --2- 172.21.15.114:0/1478172367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ca40feca0 0x7f4ca4102dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:39.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.705+0000 7f4ca37fe640 1 -- 172.21.15.114:0/1478172367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ca40ffe90 msgr2=0x7f4ca4101400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:39.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.705+0000 7f4ca37fe640 1 --2- 172.21.15.114:0/1478172367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ca40ffe90 0x7f4ca4101400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:39.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.705+0000 7f4ca37fe640 1 -- 172.21.15.114:0/1478172367 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c94009900 con 0x7f4ca4100850 2024-09-13T18:19:39.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.705+0000 7f4ca2ffd640 1 --2- 172.21.15.114:0/1478172367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ca40feca0 0x7f4ca4102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:19:39.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.705+0000 7f4ca37fe640 1 --2- 172.21.15.114:0/1478172367 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ca4100850 0x7f4ca4101940 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4c94009430 tx=0x7f4c94003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:39.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.705+0000 7f4c9bfff640 1 -- 172.21.15.114:0/1478172367 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c940307c0 con 0x7f4ca4100850 2024-09-13T18:19:39.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.705+0000 7f4c9bfff640 1 -- 172.21.15.114:0/1478172367 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4c94040070 con 0x7f4ca4100850 2024-09-13T18:19:39.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.705+0000 7f4ca959c640 1 -- 172.21.15.114:0/1478172367 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ca419ff30 con 0x7f4ca4100850 2024-09-13T18:19:39.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.705+0000 7f4c9bfff640 1 -- 172.21.15.114:0/1478172367 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c940043f0 con 0x7f4ca4100850 2024-09-13T18:19:39.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.705+0000 7f4ca959c640 1 -- 172.21.15.114:0/1478172367 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ca41a03b0 con 0x7f4ca4100850 2024-09-13T18:19:39.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.706+0000 7f4ca959c640 1 -- 172.21.15.114:0/1478172367 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ca410a410 con 0x7f4ca4100850 2024-09-13T18:19:39.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.707+0000 7f4c9bfff640 1 -- 172.21.15.114:0/1478172367 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f4c94042030 con 0x7f4ca4100850 2024-09-13T18:19:39.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.708+0000 7f4c9bfff640 1 --2- 172.21.15.114:0/1478172367 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f4c900a1090 0x7f4c900a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:39.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.708+0000 7f4ca2ffd640 1 --2- 172.21.15.114:0/1478172367 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f4c900a1090 0x7f4c900a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:39.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.708+0000 7f4c9bfff640 1 -- 172.21.15.114:0/1478172367 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4c940edb80 con 0x7f4ca4100850 2024-09-13T18:19:39.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.708+0000 7f4ca2ffd640 1 --2- 172.21.15.114:0/1478172367 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f4c900a1090 0x7f4c900a3550 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4ca40ffcf0 tx=0x7f4c9c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:39.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.711+0000 7f4c9bfff640 1 -- 172.21.15.114:0/1478172367 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f4c940b77b0 con 0x7f4ca4100850 2024-09-13T18:19:39.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.906+0000 7f4ca959c640 1 -- 172.21.15.114:0/1478172367 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4ca410a5a0 con 0x7f4ca4100850 2024-09-13T18:19:39.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.907+0000 7f4c9bfff640 1 -- 172.21.15.114:0/1478172367 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4297 (secure 0 0 0) 0x7f4c940badd0 con 0x7f4ca4100850 2024-09-13T18:19:39.910 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:19:39.910 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4162817497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4162817497},{"type":"v1","addr":"172.21.15.114:6835","nonce":4162817497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/2435428478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":2435428478},{"type":"v1","addr":"172.21.15.114:6837","nonce":2435428478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/507301496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":507301496},{"type":"v1","addr":"172.21.15.204:6833","nonce":507301496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/4028657219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4028657219},{"type":"v1","addr":"172.21.15.134:6833","nonce":4028657219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1150003275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1150003275},{"type":"v1","addr":"172.21.15.134:6835","nonce":1150003275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2024-09-13T18:19:39.910 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 6 2024-09-13T18:19:39.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.908+0000 7f4ca959c640 1 -- 172.21.15.114:0/1478172367 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f4c900a1090 msgr2=0x7f4c900a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:39.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.908+0000 7f4ca959c640 1 --2- 172.21.15.114:0/1478172367 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f4c900a1090 0x7f4c900a3550 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4ca40ffcf0 tx=0x7f4c9c005f90 comp rx=0 tx=0).stop 2024-09-13T18:19:39.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.908+0000 7f4ca959c640 1 -- 172.21.15.114:0/1478172367 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ca4100850 msgr2=0x7f4ca4101940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:39.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.908+0000 7f4ca959c640 1 --2- 172.21.15.114:0/1478172367 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ca4100850 0x7f4ca4101940 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4c94009430 tx=0x7f4c94003c20 comp rx=0 tx=0).stop 2024-09-13T18:19:39.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.908+0000 7f4ca959c640 1 -- 172.21.15.114:0/1478172367 shutdown_connections 2024-09-13T18:19:39.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.908+0000 7f4ca959c640 1 --2- 172.21.15.114:0/1478172367 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f4c900a1090 0x7f4c900a3550 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:39.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.908+0000 7f4ca959c640 1 --2- 172.21.15.114:0/1478172367 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ca4100850 0x7f4ca4101940 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:39.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.908+0000 7f4ca959c640 1 --2- 172.21.15.114:0/1478172367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ca40ffe90 0x7f4ca4101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:39.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.908+0000 7f4ca959c640 1 --2- 172.21.15.114:0/1478172367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ca40feca0 0x7f4ca4102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:39.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.909+0000 7f4ca959c640 1 -- 172.21.15.114:0/1478172367 >> 172.21.15.114:0/1478172367 conn(0x7f4ca40fa400 msgr2=0x7f4ca4068610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:39.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.909+0000 7f4ca959c640 1 -- 172.21.15.114:0/1478172367 shutdown_connections 2024-09-13T18:19:39.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:39.909+0000 7f4ca959c640 1 -- 172.21.15.114:0/1478172367 wait complete. 2024-09-13T18:19:40.072 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 13 18:19:40 smithi204 podman[64442]: 2024-09-13 18:19:40.017407063 +0000 UTC m=+0.134832414 container create 047c0dcb239a088987f2da965ca414ada76ec41e4ef1e2dee16c5946b9095fea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-f, ceph=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-13T18:19:40.219 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:40 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/1478172367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:19:40.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:40 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/1478172367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:19:40.397 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:40 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/1478172367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:19:40.397 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 13 18:19:40 smithi204 podman[64442]: 2024-09-13 18:19:40.282213991 +0000 UTC m=+0.399639343 container init 047c0dcb239a088987f2da965ca414ada76ec41e4ef1e2dee16c5946b9095fea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-f, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083) 2024-09-13T18:19:40.397 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 13 18:19:40 smithi204 podman[64442]: 2024-09-13 18:19:40.284985521 +0000 UTC m=+0.402410882 container start 047c0dcb239a088987f2da965ca414ada76ec41e4ef1e2dee16c5946b9095fea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-f, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-13T18:19:40.397 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 13 18:19:40 smithi204 ceph-mds[64461]: -- 172.21.15.204:0/2975255865 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555fd51f4d00 con 0x555fd51c9400 2024-09-13T18:19:40.398 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 13 18:19:40 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-f[64457]: starting mds.f at 2024-09-13T18:19:40.694 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 13 18:19:40 smithi204 bash[64442]: 047c0dcb239a088987f2da965ca414ada76ec41e4ef1e2dee16c5946b9095fea 2024-09-13T18:19:40.694 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 13 18:19:40 smithi204 systemd[1]: Started Ceph mds.f for 97e7edf2-71fb-11ef-bceb-c7b262605968. 2024-09-13T18:19:41.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:41 smithi134 ceph-mon[34283]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:41.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:41 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:41.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:41 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:41.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:41 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:41.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:41 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T18:19:41.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:41 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T18:19:41.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:41 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:19:41.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:41 smithi204 ceph-mon[34777]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:41.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:41 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:41.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:41 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:41.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:41 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:41.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:41 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T18:19:41.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:41 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T18:19:41.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:41 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:19:41.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:41 smithi114 ceph-mon[30897]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:41.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:41 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:41.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:41 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:41.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:41 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:41.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:41 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T18:19:41.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:41 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T18:19:41.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:41 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:19:41.923 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:19:42.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.027+0000 7f415eb9d640 1 -- 172.21.15.114:0/3966947059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4158100640 msgr2=0x7f4158106ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:42.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.027+0000 7f415eb9d640 1 --2- 172.21.15.114:0/3966947059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4158100640 0x7f4158106ed0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4150009ca0 tx=0x7f415002a8c0 comp rx=0 tx=0).stop 2024-09-13T18:19:42.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.027+0000 7f415eb9d640 1 -- 172.21.15.114:0/3966947059 shutdown_connections 2024-09-13T18:19:42.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.027+0000 7f415eb9d640 1 --2- 172.21.15.114:0/3966947059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4158100640 0x7f4158106ed0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:42.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.027+0000 7f415eb9d640 1 --2- 172.21.15.114:0/3966947059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41580ffc80 0x7f4158100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:42.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.027+0000 7f415eb9d640 1 --2- 172.21.15.114:0/3966947059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41580fea90 0x7f41580fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:42.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.027+0000 7f415eb9d640 1 -- 172.21.15.114:0/3966947059 >> 172.21.15.114:0/3966947059 conn(0x7f41580fa1f0 msgr2=0x7f41580fc630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:42.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.027+0000 7f415eb9d640 1 -- 172.21.15.114:0/3966947059 shutdown_connections 2024-09-13T18:19:42.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.027+0000 7f415eb9d640 1 -- 172.21.15.114:0/3966947059 wait complete. 2024-09-13T18:19:42.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.027+0000 7f415eb9d640 1 Processor -- start 2024-09-13T18:19:42.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.027+0000 7f415eb9d640 1 -- start start 2024-09-13T18:19:42.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.028+0000 7f415eb9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41580fea90 0x7f4158194940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:42.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.028+0000 7f415eb9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41580ffc80 0x7f4158194e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:42.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.028+0000 7f415eb9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4158100640 0x7f415819bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:42.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.028+0000 7f415eb9d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f415819c430 con 0x7f41580fea90 2024-09-13T18:19:42.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.028+0000 7f415eb9d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f415819c5a0 con 0x7f41580ffc80 2024-09-13T18:19:42.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.028+0000 7f415eb9d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f415819c760 con 0x7f4158100640 2024-09-13T18:19:42.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.028+0000 7f415db9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41580fea90 0x7f4158194940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:42.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.028+0000 7f415db9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41580fea90 0x7f4158194940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57538/0 (socket says 172.21.15.114:57538) 2024-09-13T18:19:42.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.028+0000 7f415db9b640 1 -- 172.21.15.114:0/1298212156 learned_addr learned my addr 172.21.15.114:0/1298212156 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:19:42.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.028+0000 7f415e39c640 1 --2- 172.21.15.114:0/1298212156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4158100640 0x7f415819bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:42.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.028+0000 7f415d39a640 1 --2- 172.21.15.114:0/1298212156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41580ffc80 0x7f4158194e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:42.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.028+0000 7f415db9b640 1 -- 172.21.15.114:0/1298212156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4158100640 msgr2=0x7f415819bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:42.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.028+0000 7f415db9b640 1 --2- 172.21.15.114:0/1298212156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4158100640 0x7f415819bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:42.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.028+0000 7f415db9b640 1 -- 172.21.15.114:0/1298212156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41580ffc80 msgr2=0x7f4158194e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:42.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.028+0000 7f415db9b640 1 --2- 172.21.15.114:0/1298212156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41580ffc80 0x7f4158194e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:42.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.028+0000 7f415db9b640 1 -- 172.21.15.114:0/1298212156 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4150009900 con 0x7f41580fea90 2024-09-13T18:19:42.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.028+0000 7f415d39a640 1 --2- 172.21.15.114:0/1298212156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41580ffc80 0x7f4158194e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:19:42.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.028+0000 7f415db9b640 1 --2- 172.21.15.114:0/1298212156 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41580fea90 0x7f4158194940 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f415400e9c0 tx=0x7f415400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:42.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.028+0000 7f414affd640 1 -- 172.21.15.114:0/1298212156 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41540096e0 con 0x7f41580fea90 2024-09-13T18:19:42.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.028+0000 7f415eb9d640 1 -- 172.21.15.114:0/1298212156 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f415819ca40 con 0x7f41580fea90 2024-09-13T18:19:42.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.028+0000 7f415eb9d640 1 -- 172.21.15.114:0/1298212156 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f415819cf50 con 0x7f41580fea90 2024-09-13T18:19:42.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.028+0000 7f414affd640 1 -- 172.21.15.114:0/1298212156 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f41540044c0 con 0x7f41580fea90 2024-09-13T18:19:42.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.028+0000 7f414affd640 1 -- 172.21.15.114:0/1298212156 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4154003e00 con 0x7f41580fea90 2024-09-13T18:19:42.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.029+0000 7f415eb9d640 1 -- 172.21.15.114:0/1298212156 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41580fee90 con 0x7f41580fea90 2024-09-13T18:19:42.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.030+0000 7f414affd640 1 -- 172.21.15.114:0/1298212156 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f41540035f0 con 0x7f41580fea90 2024-09-13T18:19:42.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.031+0000 7f414affd640 1 --2- 172.21.15.114:0/1298212156 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f41440a1090 0x7f41440a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:42.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.031+0000 7f415d39a640 1 --2- 172.21.15.114:0/1298212156 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f41440a1090 0x7f41440a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:42.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.031+0000 7f414affd640 1 -- 172.21.15.114:0/1298212156 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4154014070 con 0x7f41580fea90 2024-09-13T18:19:42.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.031+0000 7f415d39a640 1 --2- 172.21.15.114:0/1298212156 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f41440a1090 0x7f41440a3550 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4158195e50 tx=0x7f414c005e50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:42.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.035+0000 7f414affd640 1 -- 172.21.15.114:0/1298212156 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f4154096480 con 0x7f41580fea90 2024-09-13T18:19:42.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:42 smithi204 ceph-mon[34777]: Deploying daemon mds.c on smithi204 2024-09-13T18:19:42.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:42 smithi204 ceph-mon[34777]: mds.? [v2:172.21.15.204:6834/57029119,v1:172.21.15.204:6835/57029119] up:boot 2024-09-13T18:19:42.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:42 smithi204 ceph-mon[34777]: fsmap 6 up:standby 2024-09-13T18:19:42.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:42 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-13T18:19:42.219 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:42 smithi114 ceph-mon[30897]: Deploying daemon mds.c on smithi204 2024-09-13T18:19:42.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:42 smithi114 ceph-mon[30897]: mds.? [v2:172.21.15.204:6834/57029119,v1:172.21.15.204:6835/57029119] up:boot 2024-09-13T18:19:42.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:42 smithi114 ceph-mon[30897]: fsmap 6 up:standby 2024-09-13T18:19:42.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:42 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-13T18:19:42.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.236+0000 7f415eb9d640 1 -- 172.21.15.114:0/1298212156 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4158105f70 con 0x7f41580fea90 2024-09-13T18:19:42.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.237+0000 7f414affd640 1 -- 172.21.15.114:0/1298212156 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5051 (secure 0 0 0) 0x7f4154099aa0 con 0x7f41580fea90 2024-09-13T18:19:42.240 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:19:42.240 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4162817497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4162817497},{"type":"v1","addr":"172.21.15.114:6835","nonce":4162817497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/2435428478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":2435428478},{"type":"v1","addr":"172.21.15.114:6837","nonce":2435428478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/507301496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":507301496},{"type":"v1","addr":"172.21.15.204:6833","nonce":507301496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/4028657219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4028657219},{"type":"v1","addr":"172.21.15.134:6833","nonce":4028657219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/57029119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":57029119},{"type":"v1","addr":"172.21.15.204:6835","nonce":57029119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1150003275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1150003275},{"type":"v1","addr":"172.21.15.134:6835","nonce":1150003275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2024-09-13T18:19:42.240 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 7 2024-09-13T18:19:42.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.238+0000 7f415eb9d640 1 -- 172.21.15.114:0/1298212156 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f41440a1090 msgr2=0x7f41440a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:42.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.239+0000 7f415eb9d640 1 --2- 172.21.15.114:0/1298212156 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f41440a1090 0x7f41440a3550 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4158195e50 tx=0x7f414c005e50 comp rx=0 tx=0).stop 2024-09-13T18:19:42.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.239+0000 7f415eb9d640 1 -- 172.21.15.114:0/1298212156 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41580fea90 msgr2=0x7f4158194940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:42.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.239+0000 7f415eb9d640 1 --2- 172.21.15.114:0/1298212156 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41580fea90 0x7f4158194940 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f415400e9c0 tx=0x7f415400ee80 comp rx=0 tx=0).stop 2024-09-13T18:19:42.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.239+0000 7f415eb9d640 1 -- 172.21.15.114:0/1298212156 shutdown_connections 2024-09-13T18:19:42.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.239+0000 7f415eb9d640 1 --2- 172.21.15.114:0/1298212156 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f41440a1090 0x7f41440a3550 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:42.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.239+0000 7f415eb9d640 1 --2- 172.21.15.114:0/1298212156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4158100640 0x7f415819bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:42.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.239+0000 7f415eb9d640 1 --2- 172.21.15.114:0/1298212156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41580ffc80 0x7f4158194e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:42.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.239+0000 7f415eb9d640 1 --2- 172.21.15.114:0/1298212156 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41580fea90 0x7f4158194940 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:42.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.239+0000 7f415eb9d640 1 -- 172.21.15.114:0/1298212156 >> 172.21.15.114:0/1298212156 conn(0x7f41580fa1f0 msgr2=0x7f4158103140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:42.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.239+0000 7f415eb9d640 1 -- 172.21.15.114:0/1298212156 shutdown_connections 2024-09-13T18:19:42.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:42.239+0000 7f415eb9d640 1 -- 172.21.15.114:0/1298212156 wait complete. 2024-09-13T18:19:42.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:42 smithi134 ceph-mon[34283]: Deploying daemon mds.c on smithi204 2024-09-13T18:19:42.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:42 smithi134 ceph-mon[34283]: mds.? [v2:172.21.15.204:6834/57029119,v1:172.21.15.204:6835/57029119] up:boot 2024-09-13T18:19:42.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:42 smithi134 ceph-mon[34283]: fsmap 6 up:standby 2024-09-13T18:19:42.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:42 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-13T18:19:42.444 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 13 18:19:42 smithi204 systemd[1]: Starting Ceph mds.c for 97e7edf2-71fb-11ef-bceb-c7b262605968... 2024-09-13T18:19:42.902 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 13 18:19:42 smithi204 podman[64745]: 2024-09-13 18:19:42.460030982 +0000 UTC m=+0.015919106 image pull 8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256 2024-09-13T18:19:42.902 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 13 18:19:42 smithi204 podman[64745]: 2024-09-13 18:19:42.590275666 +0000 UTC m=+0.146163785 container create 9a125d62aba44f255ccf980327a9f39d806369fd92a803eea120457f7e46ca1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-c, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T18:19:43.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:43 smithi204 ceph-mon[34777]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:43.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:43 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/1298212156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:19:43.194 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 13 18:19:42 smithi204 podman[64745]: 2024-09-13 18:19:42.900631824 +0000 UTC m=+0.456519944 container init 9a125d62aba44f255ccf980327a9f39d806369fd92a803eea120457f7e46ca1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-c, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-13T18:19:43.194 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 13 18:19:42 smithi204 podman[64745]: 2024-09-13 18:19:42.90373898 +0000 UTC m=+0.459627103 container start 9a125d62aba44f255ccf980327a9f39d806369fd92a803eea120457f7e46ca1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-c, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-13T18:19:43.195 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 13 18:19:42 smithi204 ceph-mds[64764]: -- 172.21.15.204:0/2045230557 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b0d7752d00 con 0x55b0d7727000 2024-09-13T18:19:43.195 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 13 18:19:42 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-c[64760]: starting mds.c at 2024-09-13T18:19:43.195 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 13 18:19:43 smithi204 bash[64745]: 9a125d62aba44f255ccf980327a9f39d806369fd92a803eea120457f7e46ca1a 2024-09-13T18:19:43.195 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 13 18:19:43 smithi204 systemd[1]: Started Ceph mds.c for 97e7edf2-71fb-11ef-bceb-c7b262605968. 2024-09-13T18:19:43.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:43 smithi134 ceph-mon[34283]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:43.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:43 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/1298212156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:19:43.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:43 smithi114 ceph-mon[30897]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:43.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:43 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/1298212156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:19:44.253 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:19:44.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:44.361+0000 7fb79385a640 1 -- 172.21.15.114:0/2481351881 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb78c100f90 msgr2=0x7fb78c101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:44.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:44.361+0000 7fb79385a640 1 --2- 172.21.15.114:0/2481351881 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb78c100f90 0x7fb78c101410 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb77c009900 tx=0x7fb77c02a410 comp rx=0 tx=0).stop 2024-09-13T18:19:44.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:44.362+0000 7fb79385a640 1 -- 172.21.15.114:0/2481351881 shutdown_connections 2024-09-13T18:19:44.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:44.362+0000 7fb79385a640 1 --2- 172.21.15.114:0/2481351881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb78c101950 0x7fb78c1081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:44.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:44.362+0000 7fb79385a640 1 --2- 172.21.15.114:0/2481351881 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb78c100f90 0x7fb78c101410 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:44.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:44.362+0000 7fb79385a640 1 --2- 172.21.15.114:0/2481351881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb78c0ffda0 0x7fb78c1001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:44.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:44.362+0000 7fb79385a640 1 -- 172.21.15.114:0/2481351881 >> 172.21.15.114:0/2481351881 conn(0x7fb78c0fb520 msgr2=0x7fb78c0fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:44.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:44.362+0000 7fb79385a640 1 -- 172.21.15.114:0/2481351881 shutdown_connections 2024-09-13T18:19:44.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:44.362+0000 7fb79385a640 1 -- 172.21.15.114:0/2481351881 wait complete. 2024-09-13T18:19:44.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:44.362+0000 7fb79385a640 1 Processor -- start 2024-09-13T18:19:44.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:44.362+0000 7fb79385a640 1 -- start start 2024-09-13T18:19:44.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:44.362+0000 7fb79385a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb78c0ffda0 0x7fb78c195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:44.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:44.362+0000 7fb79385a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb78c100f90 0x7fb78c196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:44.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:44.362+0000 7fb79385a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb78c101950 0x7fb78c19d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:44.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:44.362+0000 7fb79385a640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78c19d840 con 0x7fb78c0ffda0 2024-09-13T18:19:44.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:44.362+0000 7fb79385a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78c19d9b0 con 0x7fb78c100f90 2024-09-13T18:19:44.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:44.362+0000 7fb79385a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78c19db70 con 0x7fb78c101950 2024-09-13T18:19:44.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:44.363+0000 7fb7915f6640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb78c0ffda0 0x7fb78c195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:44.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:44.363+0000 7fb7915f6640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb78c0ffda0 0x7fb78c195d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55042/0 (socket says 172.21.15.114:55042) 2024-09-13T18:19:44.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:44.363+0000 7fb7915f6640 1 -- 172.21.15.114:0/452256199 learned_addr learned my addr 172.21.15.114:0/452256199 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:19:44.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:44.363+0000 7fb790df5640 1 --2- 172.21.15.114:0/452256199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb78c100f90 0x7fb78c196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:44.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:44.363+0000 7fb791df7640 1 --2- 172.21.15.114:0/452256199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb78c101950 0x7fb78c19d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:44.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:44.363+0000 7fb7915f6640 1 -- 172.21.15.114:0/452256199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb78c101950 msgr2=0x7fb78c19d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:44.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:44.363+0000 7fb7915f6640 1 --2- 172.21.15.114:0/452256199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb78c101950 0x7fb78c19d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:44.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:44.363+0000 7fb7915f6640 1 -- 172.21.15.114:0/452256199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb78c100f90 msgr2=0x7fb78c196290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:44.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:44.363+0000 7fb7915f6640 1 --2- 172.21.15.114:0/452256199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb78c100f90 0x7fb78c196290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:44.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:44.363+0000 7fb7915f6640 1 -- 172.21.15.114:0/452256199 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb77c009560 con 0x7fb78c0ffda0 2024-09-13T18:19:44.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:44.363+0000 7fb7915f6640 1 --2- 172.21.15.114:0/452256199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb78c0ffda0 0x7fb78c195d50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb78800b450 tx=0x7fb78800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:44.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:44.363+0000 7fb7827fc640 1 -- 172.21.15.114:0/452256199 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb788011890 con 0x7fb78c0ffda0 2024-09-13T18:19:44.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:44.363+0000 7fb7827fc640 1 -- 172.21.15.114:0/452256199 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb788011e80 con 0x7fb78c0ffda0 2024-09-13T18:19:44.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:44.363+0000 7fb79385a640 1 -- 172.21.15.114:0/452256199 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb78c19de50 con 0x7fb78c0ffda0 2024-09-13T18:19:44.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:44.363+0000 7fb7827fc640 1 -- 172.21.15.114:0/452256199 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78800f340 con 0x7fb78c0ffda0 2024-09-13T18:19:44.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:44.363+0000 7fb79385a640 1 -- 172.21.15.114:0/452256199 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb78c19e270 con 0x7fb78c0ffda0 2024-09-13T18:19:44.373 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:44 smithi134 ceph-mon[34283]: mds.? [v2:172.21.15.204:6836/3068880939,v1:172.21.15.204:6837/3068880939] up:boot 2024-09-13T18:19:44.373 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:44 smithi134 ceph-mon[34283]: fsmap 7 up:standby 2024-09-13T18:19:44.373 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:44 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-13T18:19:44.373 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:44 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:44.373 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:44 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:44.373 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:44 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:44.373 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:44 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T18:19:44.373 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:44 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T18:19:44.373 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:44 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:19:44.373 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:44 smithi134 ceph-mon[34283]: Deploying daemon mds.h on smithi134 2024-09-13T18:19:44.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:44.364+0000 7fb79385a640 1 -- 172.21.15.114:0/452256199 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb78c1072c0 con 0x7fb78c0ffda0 2024-09-13T18:19:44.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:44.365+0000 7fb7827fc640 1 -- 172.21.15.114:0/452256199 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fb7880119f0 con 0x7fb78c0ffda0 2024-09-13T18:19:44.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:44.366+0000 7fb7827fc640 1 --2- 172.21.15.114:0/452256199 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb7780a1090 0x7fb7780a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:44.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:44.366+0000 7fb790df5640 1 --2- 172.21.15.114:0/452256199 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb7780a1090 0x7fb7780a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:44.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:44.366+0000 7fb7827fc640 1 -- 172.21.15.114:0/452256199 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb7880cc1e0 con 0x7fb78c0ffda0 2024-09-13T18:19:44.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:44.366+0000 7fb790df5640 1 --2- 172.21.15.114:0/452256199 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb7780a1090 0x7fb7780a3550 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb77c02aa00 tx=0x7fb77c005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:44.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:44.370+0000 7fb7827fc640 1 -- 172.21.15.114:0/452256199 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fb788095e10 con 0x7fb78c0ffda0 2024-09-13T18:19:44.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:44 smithi204 ceph-mon[34777]: mds.? [v2:172.21.15.204:6836/3068880939,v1:172.21.15.204:6837/3068880939] up:boot 2024-09-13T18:19:44.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:44 smithi204 ceph-mon[34777]: fsmap 7 up:standby 2024-09-13T18:19:44.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:44 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-13T18:19:44.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:44 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:44.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:44 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:44.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:44 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:44.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:44 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T18:19:44.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:44 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T18:19:44.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:44 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:19:44.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:44 smithi204 ceph-mon[34777]: Deploying daemon mds.h on smithi134 2024-09-13T18:19:44.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:44 smithi114 ceph-mon[30897]: mds.? [v2:172.21.15.204:6836/3068880939,v1:172.21.15.204:6837/3068880939] up:boot 2024-09-13T18:19:44.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:44 smithi114 ceph-mon[30897]: fsmap 7 up:standby 2024-09-13T18:19:44.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:44 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-13T18:19:44.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:44 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:44.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:44 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:44.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:44 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:44.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:44 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T18:19:44.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:44 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T18:19:44.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:44 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:19:44.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:44 smithi114 ceph-mon[30897]: Deploying daemon mds.h on smithi134 2024-09-13T18:19:44.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:44.568+0000 7fb79385a640 1 -- 172.21.15.114:0/452256199 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb78c106d30 con 0x7fb78c0ffda0 2024-09-13T18:19:44.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:44.568+0000 7fb7827fc640 1 -- 172.21.15.114:0/452256199 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+5811 (secure 0 0 0) 0x7fb788099430 con 0x7fb78c0ffda0 2024-09-13T18:19:44.572 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:19:44.572 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4162817497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4162817497},{"type":"v1","addr":"172.21.15.114:6835","nonce":4162817497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/2435428478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":2435428478},{"type":"v1","addr":"172.21.15.114:6837","nonce":2435428478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/507301496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":507301496},{"type":"v1","addr":"172.21.15.204:6833","nonce":507301496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/4028657219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4028657219},{"type":"v1","addr":"172.21.15.134:6833","nonce":4028657219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/57029119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":57029119},{"type":"v1","addr":"172.21.15.204:6835","nonce":57029119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3068880939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3068880939},{"type":"v1","addr":"172.21.15.204:6837","nonce":3068880939}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1150003275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1150003275},{"type":"v1","addr":"172.21.15.134:6835","nonce":1150003275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2024-09-13T18:19:44.572 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 8 2024-09-13T18:19:44.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:44.570+0000 7fb79385a640 1 -- 172.21.15.114:0/452256199 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb7780a1090 msgr2=0x7fb7780a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:44.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:44.570+0000 7fb79385a640 1 --2- 172.21.15.114:0/452256199 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb7780a1090 0x7fb7780a3550 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb77c02aa00 tx=0x7fb77c005a20 comp rx=0 tx=0).stop 2024-09-13T18:19:44.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:44.570+0000 7fb79385a640 1 -- 172.21.15.114:0/452256199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb78c0ffda0 msgr2=0x7fb78c195d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:44.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:44.570+0000 7fb79385a640 1 --2- 172.21.15.114:0/452256199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb78c0ffda0 0x7fb78c195d50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb78800b450 tx=0x7fb78800b910 comp rx=0 tx=0).stop 2024-09-13T18:19:44.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:44.570+0000 7fb79385a640 1 -- 172.21.15.114:0/452256199 shutdown_connections 2024-09-13T18:19:44.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:44.570+0000 7fb79385a640 1 --2- 172.21.15.114:0/452256199 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb7780a1090 0x7fb7780a3550 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:44.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:44.570+0000 7fb79385a640 1 --2- 172.21.15.114:0/452256199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb78c101950 0x7fb78c19d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:44.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:44.570+0000 7fb79385a640 1 --2- 172.21.15.114:0/452256199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb78c100f90 0x7fb78c196290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:44.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:44.570+0000 7fb79385a640 1 --2- 172.21.15.114:0/452256199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb78c0ffda0 0x7fb78c195d50 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:44.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:44.570+0000 7fb79385a640 1 -- 172.21.15.114:0/452256199 >> 172.21.15.114:0/452256199 conn(0x7fb78c0fb520 msgr2=0x7fb78c104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:44.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:44.570+0000 7fb79385a640 1 -- 172.21.15.114:0/452256199 shutdown_connections 2024-09-13T18:19:44.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:44.570+0000 7fb79385a640 1 -- 172.21.15.114:0/452256199 wait complete. 2024-09-13T18:19:45.119 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 13 18:19:45 smithi134 systemd[1]: Starting Ceph mds.h for 97e7edf2-71fb-11ef-bceb-c7b262605968... 2024-09-13T18:19:45.426 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:45 smithi134 ceph-mon[34283]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:45.427 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:45 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/452256199' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:19:45.427 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 13 18:19:45 smithi134 podman[63578]: 2024-09-13 18:19:45.146056843 +0000 UTC m=+0.017001440 image pull 8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256 2024-09-13T18:19:45.427 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 13 18:19:45 smithi134 podman[63578]: 2024-09-13 18:19:45.283784629 +0000 UTC m=+0.154729260 container create a14a3375eea7e014f462bc28e778dd55377d4e3ab5b6473213c6962dbd7cdc1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-h, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-13T18:19:45.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:45 smithi204 ceph-mon[34777]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:45.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:45 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/452256199' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:19:45.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:45 smithi114 ceph-mon[30897]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:45.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:45 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/452256199' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:19:45.869 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 13 18:19:45 smithi134 podman[63578]: 2024-09-13 18:19:45.559327702 +0000 UTC m=+0.430272293 container init a14a3375eea7e014f462bc28e778dd55377d4e3ab5b6473213c6962dbd7cdc1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-h, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-13T18:19:45.869 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 13 18:19:45 smithi134 podman[63578]: 2024-09-13 18:19:45.562378922 +0000 UTC m=+0.433323516 container start a14a3375eea7e014f462bc28e778dd55377d4e3ab5b6473213c6962dbd7cdc1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-h, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-13T18:19:45.869 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 13 18:19:45 smithi134 ceph-mds[63598]: -- 172.21.15.134:0/4273477986 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5653fefead00 con 0x5653fefbe800 2024-09-13T18:19:45.869 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 13 18:19:45 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-h[63594]: starting mds.h at 2024-09-13T18:19:45.869 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 13 18:19:45 smithi134 bash[63578]: a14a3375eea7e014f462bc28e778dd55377d4e3ab5b6473213c6962dbd7cdc1f 2024-09-13T18:19:45.869 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 13 18:19:45 smithi134 systemd[1]: Started Ceph mds.h for 97e7edf2-71fb-11ef-bceb-c7b262605968. 2024-09-13T18:19:46.585 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:19:46.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.694+0000 7f8f34253640 1 -- 172.21.15.114:0/1065185961 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8f2c0ffe90 msgr2=0x7f8f2c100310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:46.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.694+0000 7f8f34253640 1 --2- 172.21.15.114:0/1065185961 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8f2c0ffe90 0x7f8f2c100310 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c009900 tx=0x7f8f1c02a410 comp rx=0 tx=0).stop 2024-09-13T18:19:46.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.694+0000 7f8f34253640 1 -- 172.21.15.114:0/1065185961 shutdown_connections 2024-09-13T18:19:46.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.694+0000 7f8f34253640 1 --2- 172.21.15.114:0/1065185961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f2c100850 0x7f8f2c1070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:46.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.694+0000 7f8f34253640 1 --2- 172.21.15.114:0/1065185961 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8f2c0ffe90 0x7f8f2c100310 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:46.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.694+0000 7f8f34253640 1 --2- 172.21.15.114:0/1065185961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f2c0feca0 0x7f8f2c0ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:46.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.694+0000 7f8f34253640 1 -- 172.21.15.114:0/1065185961 >> 172.21.15.114:0/1065185961 conn(0x7f8f2c0fa440 msgr2=0x7f8f2c0fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:46.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.694+0000 7f8f34253640 1 -- 172.21.15.114:0/1065185961 shutdown_connections 2024-09-13T18:19:46.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.694+0000 7f8f34253640 1 -- 172.21.15.114:0/1065185961 wait complete. 2024-09-13T18:19:46.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.695+0000 7f8f34253640 1 Processor -- start 2024-09-13T18:19:46.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.695+0000 7f8f34253640 1 -- start start 2024-09-13T18:19:46.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.695+0000 7f8f34253640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f2c0feca0 0x7f8f2c0680c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:46.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.695+0000 7f8f34253640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8f2c0ffe90 0x7f8f2c19aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:46.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.695+0000 7f8f34253640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f2c100850 0x7f8f2c19d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:46.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.695+0000 7f8f34253640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2c19d7e0 con 0x7f8f2c0ffe90 2024-09-13T18:19:46.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.695+0000 7f8f34253640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2c19d950 con 0x7f8f2c100850 2024-09-13T18:19:46.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.695+0000 7f8f34253640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2c19db10 con 0x7f8f2c0feca0 2024-09-13T18:19:46.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.695+0000 7f8f317ee640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8f2c0ffe90 0x7f8f2c19aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:46.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.695+0000 7f8f317ee640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8f2c0ffe90 0x7f8f2c19aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55064/0 (socket says 172.21.15.114:55064) 2024-09-13T18:19:46.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.695+0000 7f8f317ee640 1 -- 172.21.15.114:0/1461484156 learned_addr learned my addr 172.21.15.114:0/1461484156 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:19:46.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.695+0000 7f8f31fef640 1 --2- 172.21.15.114:0/1461484156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f2c0feca0 0x7f8f2c0680c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:46.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.695+0000 7f8f327f0640 1 --2- 172.21.15.114:0/1461484156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f2c100850 0x7f8f2c19d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:46.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.695+0000 7f8f317ee640 1 -- 172.21.15.114:0/1461484156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f2c0feca0 msgr2=0x7f8f2c0680c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:46.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.695+0000 7f8f317ee640 1 --2- 172.21.15.114:0/1461484156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f2c0feca0 0x7f8f2c0680c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:46.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.695+0000 7f8f317ee640 1 -- 172.21.15.114:0/1461484156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f2c100850 msgr2=0x7f8f2c19d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:46.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.695+0000 7f8f317ee640 1 --2- 172.21.15.114:0/1461484156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f2c100850 0x7f8f2c19d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:46.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.695+0000 7f8f317ee640 1 -- 172.21.15.114:0/1461484156 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f1c009560 con 0x7f8f2c0ffe90 2024-09-13T18:19:46.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.695+0000 7f8f327f0640 1 --2- 172.21.15.114:0/1461484156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f2c100850 0x7f8f2c19d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:19:46.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.695+0000 7f8f317ee640 1 --2- 172.21.15.114:0/1461484156 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8f2c0ffe90 0x7f8f2c19aa80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c0057c0 tx=0x7f8f1c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:46.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.696+0000 7f8f22ffd640 1 -- 172.21.15.114:0/1461484156 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c031d70 con 0x7f8f2c0ffe90 2024-09-13T18:19:46.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.696+0000 7f8f22ffd640 1 -- 172.21.15.114:0/1461484156 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8f1c0044e0 con 0x7f8f2c0ffe90 2024-09-13T18:19:46.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.696+0000 7f8f34253640 1 -- 172.21.15.114:0/1461484156 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f2c19dd90 con 0x7f8f2c0ffe90 2024-09-13T18:19:46.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.696+0000 7f8f22ffd640 1 -- 172.21.15.114:0/1461484156 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c003b10 con 0x7f8f2c0ffe90 2024-09-13T18:19:46.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.696+0000 7f8f34253640 1 -- 172.21.15.114:0/1461484156 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f2c19e1e0 con 0x7f8f2c0ffe90 2024-09-13T18:19:46.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.697+0000 7f8f34253640 1 -- 172.21.15.114:0/1461484156 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f2c1061a0 con 0x7f8f2c0ffe90 2024-09-13T18:19:46.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.698+0000 7f8f22ffd640 1 -- 172.21.15.114:0/1461484156 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f8f1c003c70 con 0x7f8f2c0ffe90 2024-09-13T18:19:46.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.698+0000 7f8f22ffd640 1 --2- 172.21.15.114:0/1461484156 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f8f180a1090 0x7f8f180a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:46.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.699+0000 7f8f31fef640 1 --2- 172.21.15.114:0/1461484156 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f8f180a1090 0x7f8f180a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:46.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.699+0000 7f8f22ffd640 1 -- 172.21.15.114:0/1461484156 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8f1c0eda40 con 0x7f8f2c0ffe90 2024-09-13T18:19:46.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.699+0000 7f8f31fef640 1 --2- 172.21.15.114:0/1461484156 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f8f180a1090 0x7f8f180a3550 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c0ffcf0 tx=0x7f8f2800b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:46.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.701+0000 7f8f22ffd640 1 -- 172.21.15.114:0/1461484156 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f8f1c0b7670 con 0x7f8f2c0ffe90 2024-09-13T18:19:46.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.900+0000 7f8f34253640 1 -- 172.21.15.114:0/1461484156 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8f2c10a410 con 0x7f8f2c0ffe90 2024-09-13T18:19:46.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.901+0000 7f8f22ffd640 1 -- 172.21.15.114:0/1461484156 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6568 (secure 0 0 0) 0x7f8f1c0bac90 con 0x7f8f2c0ffe90 2024-09-13T18:19:46.904 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:19:46.904 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4162817497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4162817497},{"type":"v1","addr":"172.21.15.114:6835","nonce":4162817497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/2435428478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":2435428478},{"type":"v1","addr":"172.21.15.114:6837","nonce":2435428478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/507301496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":507301496},{"type":"v1","addr":"172.21.15.204:6833","nonce":507301496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/4028657219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4028657219},{"type":"v1","addr":"172.21.15.134:6833","nonce":4028657219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/57029119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":57029119},{"type":"v1","addr":"172.21.15.204:6835","nonce":57029119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3068880939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3068880939},{"type":"v1","addr":"172.21.15.204:6837","nonce":3068880939}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1150003275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1150003275},{"type":"v1","addr":"172.21.15.134:6835","nonce":1150003275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/891692210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":891692210},{"type":"v1","addr":"172.21.15.134:6837","nonce":891692210}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2024-09-13T18:19:46.904 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 9 2024-09-13T18:19:46.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.903+0000 7f8f34253640 1 -- 172.21.15.114:0/1461484156 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f8f180a1090 msgr2=0x7f8f180a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:46.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.903+0000 7f8f34253640 1 --2- 172.21.15.114:0/1461484156 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f8f180a1090 0x7f8f180a3550 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c0ffcf0 tx=0x7f8f2800b390 comp rx=0 tx=0).stop 2024-09-13T18:19:46.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.903+0000 7f8f34253640 1 -- 172.21.15.114:0/1461484156 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8f2c0ffe90 msgr2=0x7f8f2c19aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:46.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.903+0000 7f8f34253640 1 --2- 172.21.15.114:0/1461484156 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8f2c0ffe90 0x7f8f2c19aa80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c0057c0 tx=0x7f8f1c0042f0 comp rx=0 tx=0).stop 2024-09-13T18:19:46.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.903+0000 7f8f34253640 1 -- 172.21.15.114:0/1461484156 shutdown_connections 2024-09-13T18:19:46.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.903+0000 7f8f34253640 1 --2- 172.21.15.114:0/1461484156 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f8f180a1090 0x7f8f180a3550 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:46.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.903+0000 7f8f34253640 1 --2- 172.21.15.114:0/1461484156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f2c100850 0x7f8f2c19d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:46.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.903+0000 7f8f34253640 1 --2- 172.21.15.114:0/1461484156 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8f2c0ffe90 0x7f8f2c19aa80 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:46.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.903+0000 7f8f34253640 1 --2- 172.21.15.114:0/1461484156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f2c0feca0 0x7f8f2c0680c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:46.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.903+0000 7f8f34253640 1 -- 172.21.15.114:0/1461484156 >> 172.21.15.114:0/1461484156 conn(0x7f8f2c0fa440 msgr2=0x7f8f2c103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:46.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.903+0000 7f8f34253640 1 -- 172.21.15.114:0/1461484156 shutdown_connections 2024-09-13T18:19:46.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:46.903+0000 7f8f34253640 1 -- 172.21.15.114:0/1461484156 wait complete. 2024-09-13T18:19:46.916 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:46 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:46.916 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:46 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:46.916 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:46 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:46.916 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:46 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T18:19:46.916 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:46 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T18:19:46.916 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:46 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:19:46.916 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:46 smithi114 ceph-mon[30897]: Deploying daemon mds.i on smithi204 2024-09-13T18:19:46.916 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:46 smithi114 ceph-mon[30897]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:46.916 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:46 smithi114 ceph-mon[30897]: mds.? [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] up:boot 2024-09-13T18:19:46.917 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:46 smithi114 ceph-mon[30897]: fsmap 8 up:standby 2024-09-13T18:19:46.917 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:46 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-13T18:19:47.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:46 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:47.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:46 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:47.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:46 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:47.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:46 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T18:19:47.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:46 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T18:19:47.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:46 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:19:47.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:46 smithi134 ceph-mon[34283]: Deploying daemon mds.i on smithi204 2024-09-13T18:19:47.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:46 smithi134 ceph-mon[34283]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:47.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:46 smithi134 ceph-mon[34283]: mds.? [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] up:boot 2024-09-13T18:19:47.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:46 smithi134 ceph-mon[34283]: fsmap 8 up:standby 2024-09-13T18:19:47.120 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:46 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-13T18:19:47.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:46 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:47.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:46 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:47.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:46 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:47.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:46 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T18:19:47.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:46 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T18:19:47.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:46 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:19:47.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:46 smithi204 ceph-mon[34777]: Deploying daemon mds.i on smithi204 2024-09-13T18:19:47.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:46 smithi204 ceph-mon[34777]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:47.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:46 smithi204 ceph-mon[34777]: mds.? [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] up:boot 2024-09-13T18:19:47.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:46 smithi204 ceph-mon[34777]: fsmap 8 up:standby 2024-09-13T18:19:47.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:46 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-13T18:19:47.694 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 13 18:19:47 smithi204 systemd[1]: Starting Ceph mds.i for 97e7edf2-71fb-11ef-bceb-c7b262605968... 2024-09-13T18:19:47.980 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:47 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/1461484156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:19:48.001 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:47 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/1461484156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:19:48.001 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 13 18:19:47 smithi204 podman[65047]: 2024-09-13 18:19:47.731419498 +0000 UTC m=+0.015902468 image pull 8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256 2024-09-13T18:19:48.001 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 13 18:19:47 smithi204 podman[65047]: 2024-09-13 18:19:47.848912698 +0000 UTC m=+0.133395668 container create aecc8a6696be291f6a46b1184f3c2186c4e73a9bce18e359fda5f33b9459d07f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-i, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-13T18:19:48.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:47 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/1461484156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:19:48.445 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 13 18:19:48 smithi204 podman[65047]: 2024-09-13 18:19:48.168588389 +0000 UTC m=+0.453071358 container init aecc8a6696be291f6a46b1184f3c2186c4e73a9bce18e359fda5f33b9459d07f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-i, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-13T18:19:48.445 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 13 18:19:48 smithi204 podman[65047]: 2024-09-13 18:19:48.171582539 +0000 UTC m=+0.456065517 container start aecc8a6696be291f6a46b1184f3c2186c4e73a9bce18e359fda5f33b9459d07f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-i, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-13T18:19:48.445 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 13 18:19:48 smithi204 ceph-mds[65068]: -- 172.21.15.204:0/750555552 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5611c0ae0d00 con 0x5611c0ab5000 2024-09-13T18:19:48.445 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 13 18:19:48 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-i[65062]: starting mds.i at 2024-09-13T18:19:48.445 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 13 18:19:48 smithi204 bash[65047]: aecc8a6696be291f6a46b1184f3c2186c4e73a9bce18e359fda5f33b9459d07f 2024-09-13T18:19:48.445 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 13 18:19:48 smithi204 systemd[1]: Started Ceph mds.i for 97e7edf2-71fb-11ef-bceb-c7b262605968. 2024-09-13T18:19:48.919 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:19:48.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:48 smithi114 ceph-mon[30897]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:48.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:48 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:48.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:48 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:48.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:48 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:48.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:48 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T18:19:48.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:48 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T18:19:48.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:48 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:19:49.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:49.021+0000 7f736b376640 1 -- 172.21.15.114:0/1057562166 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73641031d0 msgr2=0x7f73641055c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:49.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:49.021+0000 7f736b376640 1 --2- 172.21.15.114:0/1057562166 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73641031d0 0x7f73641055c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7360009900 tx=0x7f736002a410 comp rx=0 tx=0).stop 2024-09-13T18:19:49.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:49.021+0000 7f736b376640 1 -- 172.21.15.114:0/1057562166 shutdown_connections 2024-09-13T18:19:49.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:49.021+0000 7f736b376640 1 --2- 172.21.15.114:0/1057562166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7364105b00 0x7f7364107ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:49.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:49.021+0000 7f736b376640 1 --2- 172.21.15.114:0/1057562166 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73641031d0 0x7f73641055c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:49.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:49.021+0000 7f736b376640 1 --2- 172.21.15.114:0/1057562166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73641008a0 0x7f7364102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:49.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:49.021+0000 7f736b376640 1 -- 172.21.15.114:0/1057562166 >> 172.21.15.114:0/1057562166 conn(0x7f73640fa440 msgr2=0x7f73640fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:49.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:49.021+0000 7f736b376640 1 -- 172.21.15.114:0/1057562166 shutdown_connections 2024-09-13T18:19:49.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:49.021+0000 7f736b376640 1 -- 172.21.15.114:0/1057562166 wait complete. 2024-09-13T18:19:49.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:49.022+0000 7f736b376640 1 Processor -- start 2024-09-13T18:19:49.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:49.022+0000 7f736b376640 1 -- start start 2024-09-13T18:19:49.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:49.022+0000 7f736b376640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73641008a0 0x7f7364195c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:49.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:49.022+0000 7f736b376640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73641031d0 0x7f73641961d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:49.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:49.022+0000 7f736b376640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7364105b00 0x7f736419d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:49.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:49.022+0000 7f736b376640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f736419d7a0 con 0x7f73641008a0 2024-09-13T18:19:49.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:49.022+0000 7f736b376640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f736419d910 con 0x7f7364105b00 2024-09-13T18:19:49.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:49.022+0000 7f736b376640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f736419dad0 con 0x7f73641031d0 2024-09-13T18:19:49.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:49.022+0000 7f7369112640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73641008a0 0x7f7364195c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:49.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:49.022+0000 7f7369112640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73641008a0 0x7f7364195c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55080/0 (socket says 172.21.15.114:55080) 2024-09-13T18:19:49.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:49.022+0000 7f7369112640 1 -- 172.21.15.114:0/2423094792 learned_addr learned my addr 172.21.15.114:0/2423094792 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:19:49.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:49.023+0000 7f7368911640 1 --2- 172.21.15.114:0/2423094792 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73641031d0 0x7f73641961d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:49.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:49.023+0000 7f7369913640 1 --2- 172.21.15.114:0/2423094792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7364105b00 0x7f736419d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:49.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:49.023+0000 7f7369112640 1 -- 172.21.15.114:0/2423094792 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73641031d0 msgr2=0x7f73641961d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:49.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:49.023+0000 7f7369112640 1 --2- 172.21.15.114:0/2423094792 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73641031d0 0x7f73641961d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:49.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:49.023+0000 7f7369112640 1 -- 172.21.15.114:0/2423094792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7364105b00 msgr2=0x7f736419d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:49.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:49.023+0000 7f7369112640 1 --2- 172.21.15.114:0/2423094792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7364105b00 0x7f736419d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:49.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:49.023+0000 7f7369112640 1 -- 172.21.15.114:0/2423094792 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7360009560 con 0x7f73641008a0 2024-09-13T18:19:49.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:49.023+0000 7f7369112640 1 --2- 172.21.15.114:0/2423094792 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73641008a0 0x7f7364195c90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f735800c920 tx=0x7f735800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:49.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:49.023+0000 7f73567fc640 1 -- 172.21.15.114:0/2423094792 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7358007c70 con 0x7f73641008a0 2024-09-13T18:19:49.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:49.023+0000 7f736b376640 1 -- 172.21.15.114:0/2423094792 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73640fe6f0 con 0x7f73641008a0 2024-09-13T18:19:49.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:49.023+0000 7f73567fc640 1 -- 172.21.15.114:0/2423094792 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7358007dd0 con 0x7f73641008a0 2024-09-13T18:19:49.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:49.023+0000 7f73567fc640 1 -- 172.21.15.114:0/2423094792 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7358018730 con 0x7f73641008a0 2024-09-13T18:19:49.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:49.023+0000 7f736b376640 1 -- 172.21.15.114:0/2423094792 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73640fec20 con 0x7f73641008a0 2024-09-13T18:19:49.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:49.025+0000 7f736b376640 1 -- 172.21.15.114:0/2423094792 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73640fbf30 con 0x7f73641008a0 2024-09-13T18:19:49.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:49.025+0000 7f73567fc640 1 -- 172.21.15.114:0/2423094792 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f735800f450 con 0x7f73641008a0 2024-09-13T18:19:49.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:49.026+0000 7f73567fc640 1 --2- 172.21.15.114:0/2423094792 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f73500a1110 0x7f73500a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:49.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:49.026+0000 7f7368911640 1 --2- 172.21.15.114:0/2423094792 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f73500a1110 0x7f73500a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:49.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:49.026+0000 7f73567fc640 1 -- 172.21.15.114:0/2423094792 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f73580cc4a0 con 0x7f73641008a0 2024-09-13T18:19:49.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:49.026+0000 7f7368911640 1 --2- 172.21.15.114:0/2423094792 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f73500a1110 0x7f73500a35d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f73600057c0 tx=0x7f7360005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:49.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:49.030+0000 7f73567fc640 1 -- 172.21.15.114:0/2423094792 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f73580960d0 con 0x7f73641008a0 2024-09-13T18:19:49.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:48 smithi134 ceph-mon[34283]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:49.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:48 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:49.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:48 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:49.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:48 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:49.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:48 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T18:19:49.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:48 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T18:19:49.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:48 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:19:49.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:48 smithi204 ceph-mon[34777]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:49.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:48 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:49.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:48 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:49.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:48 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:49.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:48 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T18:19:49.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:48 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T18:19:49.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:48 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:19:49.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:49.229+0000 7f736b376640 1 -- 172.21.15.114:0/2423094792 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f73640fb9a0 con 0x7f73641008a0 2024-09-13T18:19:49.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:49.230+0000 7f73567fc640 1 -- 172.21.15.114:0/2423094792 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7327 (secure 0 0 0) 0x7f73580996f0 con 0x7f73641008a0 2024-09-13T18:19:49.233 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:19:49.233 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4162817497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4162817497},{"type":"v1","addr":"172.21.15.114:6835","nonce":4162817497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/2435428478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":2435428478},{"type":"v1","addr":"172.21.15.114:6837","nonce":2435428478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/507301496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":507301496},{"type":"v1","addr":"172.21.15.204:6833","nonce":507301496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/4028657219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4028657219},{"type":"v1","addr":"172.21.15.134:6833","nonce":4028657219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/57029119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":57029119},{"type":"v1","addr":"172.21.15.204:6835","nonce":57029119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3068880939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3068880939},{"type":"v1","addr":"172.21.15.204:6837","nonce":3068880939}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1150003275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1150003275},{"type":"v1","addr":"172.21.15.134:6835","nonce":1150003275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24475,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/344883912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":344883912},{"type":"v1","addr":"172.21.15.204:6839","nonce":344883912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/891692210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":891692210},{"type":"v1","addr":"172.21.15.134:6837","nonce":891692210}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2024-09-13T18:19:49.233 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 10 2024-09-13T18:19:49.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:49.231+0000 7f736b376640 1 -- 172.21.15.114:0/2423094792 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f73500a1110 msgr2=0x7f73500a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:49.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:49.231+0000 7f736b376640 1 --2- 172.21.15.114:0/2423094792 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f73500a1110 0x7f73500a35d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f73600057c0 tx=0x7f7360005a20 comp rx=0 tx=0).stop 2024-09-13T18:19:49.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:49.231+0000 7f736b376640 1 -- 172.21.15.114:0/2423094792 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73641008a0 msgr2=0x7f7364195c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:49.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:49.231+0000 7f736b376640 1 --2- 172.21.15.114:0/2423094792 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73641008a0 0x7f7364195c90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f735800c920 tx=0x7f735800cde0 comp rx=0 tx=0).stop 2024-09-13T18:19:49.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:49.232+0000 7f736b376640 1 -- 172.21.15.114:0/2423094792 shutdown_connections 2024-09-13T18:19:49.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:49.232+0000 7f736b376640 1 --2- 172.21.15.114:0/2423094792 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f73500a1110 0x7f73500a35d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:49.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:49.232+0000 7f736b376640 1 --2- 172.21.15.114:0/2423094792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7364105b00 0x7f736419d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:49.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:49.232+0000 7f736b376640 1 --2- 172.21.15.114:0/2423094792 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73641031d0 0x7f73641961d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:49.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:49.232+0000 7f736b376640 1 --2- 172.21.15.114:0/2423094792 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73641008a0 0x7f7364195c90 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:49.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:49.232+0000 7f736b376640 1 -- 172.21.15.114:0/2423094792 >> 172.21.15.114:0/2423094792 conn(0x7f73640fa440 msgr2=0x7f7364101120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:49.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:49.232+0000 7f736b376640 1 -- 172.21.15.114:0/2423094792 shutdown_connections 2024-09-13T18:19:49.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:49.232+0000 7f736b376640 1 -- 172.21.15.114:0/2423094792 wait complete. 2024-09-13T18:19:50.118 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:49 smithi134 ceph-mon[34283]: Deploying daemon mds.g on smithi114 2024-09-13T18:19:50.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:49 smithi134 ceph-mon[34283]: mds.? [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] up:boot 2024-09-13T18:19:50.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:49 smithi134 ceph-mon[34283]: fsmap 9 up:standby 2024-09-13T18:19:50.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:49 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-13T18:19:50.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:49 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/2423094792' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:19:50.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:49 smithi204 ceph-mon[34777]: Deploying daemon mds.g on smithi114 2024-09-13T18:19:50.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:49 smithi204 ceph-mon[34777]: mds.? [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] up:boot 2024-09-13T18:19:50.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:49 smithi204 ceph-mon[34777]: fsmap 9 up:standby 2024-09-13T18:19:50.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:49 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-13T18:19:50.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:49 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/2423094792' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:19:50.202 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:49 smithi114 ceph-mon[30897]: Deploying daemon mds.g on smithi114 2024-09-13T18:19:50.202 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:49 smithi114 ceph-mon[30897]: mds.? [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] up:boot 2024-09-13T18:19:50.202 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:49 smithi114 ceph-mon[30897]: fsmap 9 up:standby 2024-09-13T18:19:50.202 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:49 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-13T18:19:50.202 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:49 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2423094792' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:19:50.470 INFO:journalctl@ceph.mds.g.smithi114.stdout:Sep 13 18:19:50 smithi114 systemd[1]: Starting Ceph mds.g for 97e7edf2-71fb-11ef-bceb-c7b262605968... 2024-09-13T18:19:50.893 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:50 smithi114 ceph-mon[30897]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:50.893 INFO:journalctl@ceph.mds.g.smithi114.stdout:Sep 13 18:19:50 smithi114 podman[70840]: 2024-09-13 18:19:50.514151654 +0000 UTC m=+0.016235757 image pull 8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256 2024-09-13T18:19:50.893 INFO:journalctl@ceph.mds.g.smithi114.stdout:Sep 13 18:19:50 smithi114 podman[70840]: 2024-09-13 18:19:50.646185399 +0000 UTC m=+0.148269547 container create 15d44722d0352257ac825a8476f4550b182366d907fd53a6c26099ea2a4dd971 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-g, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-13T18:19:50.893 INFO:journalctl@ceph.mds.g.smithi114.stdout:Sep 13 18:19:50 smithi114 podman[70840]: 2024-09-13 18:19:50.890745924 +0000 UTC m=+0.392830036 container init 15d44722d0352257ac825a8476f4550b182366d907fd53a6c26099ea2a4dd971 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-g, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-13T18:19:51.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:50 smithi134 ceph-mon[34283]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:51.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:50 smithi204 ceph-mon[34777]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:51.220 INFO:journalctl@ceph.mds.g.smithi114.stdout:Sep 13 18:19:50 smithi114 podman[70840]: 2024-09-13 18:19:50.89378506 +0000 UTC m=+0.395869173 container start 15d44722d0352257ac825a8476f4550b182366d907fd53a6c26099ea2a4dd971 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-g, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-13T18:19:51.220 INFO:journalctl@ceph.mds.g.smithi114.stdout:Sep 13 18:19:50 smithi114 ceph-mds[70862]: -- 172.21.15.114:0/2873233950 <== mon.0 v2:172.21.15.114:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55701d50ed00 con 0x55701d4e3000 2024-09-13T18:19:51.220 INFO:journalctl@ceph.mds.g.smithi114.stdout:Sep 13 18:19:50 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-g[70858]: starting mds.g at 2024-09-13T18:19:51.220 INFO:journalctl@ceph.mds.g.smithi114.stdout:Sep 13 18:19:50 smithi114 bash[70840]: 15d44722d0352257ac825a8476f4550b182366d907fd53a6c26099ea2a4dd971 2024-09-13T18:19:51.220 INFO:journalctl@ceph.mds.g.smithi114.stdout:Sep 13 18:19:50 smithi114 systemd[1]: Started Ceph mds.g for 97e7edf2-71fb-11ef-bceb-c7b262605968. 2024-09-13T18:19:51.246 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:19:51.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.351+0000 7f89e83e2640 1 -- 172.21.15.114:0/600264453 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89e0100850 msgr2=0x7f89e01070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:51.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.351+0000 7f89e83e2640 1 --2- 172.21.15.114:0/600264453 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89e0100850 0x7f89e01070e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f89d8009ca0 tx=0x7f89d802a8c0 comp rx=0 tx=0).stop 2024-09-13T18:19:51.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.351+0000 7f89e83e2640 1 -- 172.21.15.114:0/600264453 shutdown_connections 2024-09-13T18:19:51.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.351+0000 7f89e83e2640 1 --2- 172.21.15.114:0/600264453 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89e0100850 0x7f89e01070e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:51.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.351+0000 7f89e83e2640 1 --2- 172.21.15.114:0/600264453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89e00ffe90 0x7f89e0100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:51.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.351+0000 7f89e83e2640 1 --2- 172.21.15.114:0/600264453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89e00feca0 0x7f89e00ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:51.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.351+0000 7f89e83e2640 1 -- 172.21.15.114:0/600264453 >> 172.21.15.114:0/600264453 conn(0x7f89e00fa420 msgr2=0x7f89e00fc840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:51.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.351+0000 7f89e83e2640 1 -- 172.21.15.114:0/600264453 shutdown_connections 2024-09-13T18:19:51.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.351+0000 7f89e83e2640 1 -- 172.21.15.114:0/600264453 wait complete. 2024-09-13T18:19:51.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.352+0000 7f89e83e2640 1 Processor -- start 2024-09-13T18:19:51.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.352+0000 7f89e83e2640 1 -- start start 2024-09-13T18:19:51.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.352+0000 7f89e83e2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89e00feca0 0x7f89e0195e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:51.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.352+0000 7f89e83e2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89e00ffe90 0x7f89e0196360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:51.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.352+0000 7f89e83e2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89e0100850 0x7f89e019d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:51.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.352+0000 7f89e83e2640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e019d910 con 0x7f89e0100850 2024-09-13T18:19:51.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.352+0000 7f89e83e2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e019da80 con 0x7f89e00feca0 2024-09-13T18:19:51.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.352+0000 7f89e83e2640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e019dc40 con 0x7f89e00ffe90 2024-09-13T18:19:51.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.352+0000 7f89e697f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89e0100850 0x7f89e019d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:51.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.352+0000 7f89e697f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89e0100850 0x7f89e019d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55130/0 (socket says 172.21.15.114:55130) 2024-09-13T18:19:51.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.352+0000 7f89e697f640 1 -- 172.21.15.114:0/3602834841 learned_addr learned my addr 172.21.15.114:0/3602834841 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:19:51.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.352+0000 7f89e597d640 1 --2- 172.21.15.114:0/3602834841 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89e00ffe90 0x7f89e0196360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:51.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.352+0000 7f89e617e640 1 --2- 172.21.15.114:0/3602834841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89e00feca0 0x7f89e0195e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:51.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.352+0000 7f89e697f640 1 -- 172.21.15.114:0/3602834841 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89e00ffe90 msgr2=0x7f89e0196360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:51.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.352+0000 7f89e697f640 1 --2- 172.21.15.114:0/3602834841 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89e00ffe90 0x7f89e0196360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:51.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.353+0000 7f89e697f640 1 -- 172.21.15.114:0/3602834841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89e00feca0 msgr2=0x7f89e0195e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:51.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.353+0000 7f89e697f640 1 --2- 172.21.15.114:0/3602834841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89e00feca0 0x7f89e0195e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:51.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.353+0000 7f89e697f640 1 -- 172.21.15.114:0/3602834841 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89d8009900 con 0x7f89e0100850 2024-09-13T18:19:51.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.353+0000 7f89e597d640 1 --2- 172.21.15.114:0/3602834841 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89e00ffe90 0x7f89e0196360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:19:51.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.353+0000 7f89e617e640 1 --2- 172.21.15.114:0/3602834841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89e00feca0 0x7f89e0195e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:19:51.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.353+0000 7f89e697f640 1 --2- 172.21.15.114:0/3602834841 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89e0100850 0x7f89e019d3d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f89d80316f0 tx=0x7f89d8031720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:51.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.353+0000 7f89d77fe640 1 -- 172.21.15.114:0/3602834841 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d8003b70 con 0x7f89e0100850 2024-09-13T18:19:51.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.353+0000 7f89e83e2640 1 -- 172.21.15.114:0/3602834841 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89e019dec0 con 0x7f89e0100850 2024-09-13T18:19:51.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.353+0000 7f89d77fe640 1 -- 172.21.15.114:0/3602834841 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f89d8040070 con 0x7f89e0100850 2024-09-13T18:19:51.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.353+0000 7f89d77fe640 1 -- 172.21.15.114:0/3602834841 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d80043a0 con 0x7f89e0100850 2024-09-13T18:19:51.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.353+0000 7f89e83e2640 1 -- 172.21.15.114:0/3602834841 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89e019e340 con 0x7f89e0100850 2024-09-13T18:19:51.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.354+0000 7f89e83e2640 1 -- 172.21.15.114:0/3602834841 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89e010a4c0 con 0x7f89e0100850 2024-09-13T18:19:51.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.355+0000 7f89d77fe640 1 -- 172.21.15.114:0/3602834841 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f89d8042030 con 0x7f89e0100850 2024-09-13T18:19:51.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.355+0000 7f89d77fe640 1 --2- 172.21.15.114:0/3602834841 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f89cc0a1090 0x7f89cc0a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:51.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.356+0000 7f89e617e640 1 --2- 172.21.15.114:0/3602834841 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f89cc0a1090 0x7f89cc0a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:51.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.356+0000 7f89d77fe640 1 -- 172.21.15.114:0/3602834841 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f89d80edb00 con 0x7f89e0100850 2024-09-13T18:19:51.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.356+0000 7f89e617e640 1 --2- 172.21.15.114:0/3602834841 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f89cc0a1090 0x7f89cc0a3550 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f89dc0058c0 tx=0x7f89dc005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:51.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.359+0000 7f89d77fe640 1 -- 172.21.15.114:0/3602834841 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f89d80b7730 con 0x7f89e0100850 2024-09-13T18:19:51.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.564+0000 7f89e83e2640 1 -- 172.21.15.114:0/3602834841 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f89e010a6a0 con 0x7f89e0100850 2024-09-13T18:19:51.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.565+0000 7f89d77fe640 1 -- 172.21.15.114:0/3602834841 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7327 (secure 0 0 0) 0x7f89d8003e60 con 0x7f89e0100850 2024-09-13T18:19:51.569 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:19:51.569 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4162817497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4162817497},{"type":"v1","addr":"172.21.15.114:6835","nonce":4162817497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/2435428478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":2435428478},{"type":"v1","addr":"172.21.15.114:6837","nonce":2435428478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/507301496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":507301496},{"type":"v1","addr":"172.21.15.204:6833","nonce":507301496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/4028657219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4028657219},{"type":"v1","addr":"172.21.15.134:6833","nonce":4028657219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/57029119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":57029119},{"type":"v1","addr":"172.21.15.204:6835","nonce":57029119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3068880939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3068880939},{"type":"v1","addr":"172.21.15.204:6837","nonce":3068880939}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1150003275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1150003275},{"type":"v1","addr":"172.21.15.134:6835","nonce":1150003275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24475,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/344883912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":344883912},{"type":"v1","addr":"172.21.15.204:6839","nonce":344883912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/891692210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":891692210},{"type":"v1","addr":"172.21.15.134:6837","nonce":891692210}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2024-09-13T18:19:51.569 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 10 2024-09-13T18:19:51.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.567+0000 7f89e83e2640 1 -- 172.21.15.114:0/3602834841 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f89cc0a1090 msgr2=0x7f89cc0a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:51.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.567+0000 7f89e83e2640 1 --2- 172.21.15.114:0/3602834841 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f89cc0a1090 0x7f89cc0a3550 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f89dc0058c0 tx=0x7f89dc005850 comp rx=0 tx=0).stop 2024-09-13T18:19:51.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.567+0000 7f89e83e2640 1 -- 172.21.15.114:0/3602834841 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89e0100850 msgr2=0x7f89e019d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:51.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.567+0000 7f89e83e2640 1 --2- 172.21.15.114:0/3602834841 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89e0100850 0x7f89e019d3d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f89d80316f0 tx=0x7f89d8031720 comp rx=0 tx=0).stop 2024-09-13T18:19:51.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.567+0000 7f89e83e2640 1 -- 172.21.15.114:0/3602834841 shutdown_connections 2024-09-13T18:19:51.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.567+0000 7f89e83e2640 1 --2- 172.21.15.114:0/3602834841 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f89cc0a1090 0x7f89cc0a3550 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:51.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.567+0000 7f89e83e2640 1 --2- 172.21.15.114:0/3602834841 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89e0100850 0x7f89e019d3d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:51.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.567+0000 7f89e83e2640 1 --2- 172.21.15.114:0/3602834841 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89e00ffe90 0x7f89e0196360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:51.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.567+0000 7f89e83e2640 1 --2- 172.21.15.114:0/3602834841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89e00feca0 0x7f89e0195e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:51.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.567+0000 7f89e83e2640 1 -- 172.21.15.114:0/3602834841 >> 172.21.15.114:0/3602834841 conn(0x7f89e00fa420 msgr2=0x7f89e0068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:51.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.567+0000 7f89e83e2640 1 -- 172.21.15.114:0/3602834841 shutdown_connections 2024-09-13T18:19:51.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:51.567+0000 7f89e83e2640 1 -- 172.21.15.114:0/3602834841 wait complete. 2024-09-13T18:19:52.372 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:52 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:52.372 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:52 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:52.372 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:52 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:52.373 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:52 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T18:19:52.373 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:52 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T18:19:52.373 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:52 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:19:52.373 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:52 smithi134 ceph-mon[34283]: Deploying daemon mds.a on smithi114 2024-09-13T18:19:52.373 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:52 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/3602834841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:19:52.373 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:52 smithi134 ceph-mon[34283]: mds.? [v2:172.21.15.114:6838/2046516451,v1:172.21.15.114:6839/2046516451] up:boot 2024-09-13T18:19:52.373 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:52 smithi134 ceph-mon[34283]: fsmap 10 up:standby 2024-09-13T18:19:52.373 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:52 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-13T18:19:52.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:52 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:52.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:52 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:52.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:52 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:52.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:52 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T18:19:52.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:52 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T18:19:52.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:52 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:19:52.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:52 smithi204 ceph-mon[34777]: Deploying daemon mds.a on smithi114 2024-09-13T18:19:52.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:52 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/3602834841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:19:52.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:52 smithi204 ceph-mon[34777]: mds.? [v2:172.21.15.114:6838/2046516451,v1:172.21.15.114:6839/2046516451] up:boot 2024-09-13T18:19:52.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:52 smithi204 ceph-mon[34777]: fsmap 10 up:standby 2024-09-13T18:19:52.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:52 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-13T18:19:52.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:52 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:52.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:52 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:52.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:52 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:52.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:52 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T18:19:52.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:52 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T18:19:52.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:52 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:19:52.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:52 smithi114 ceph-mon[30897]: Deploying daemon mds.a on smithi114 2024-09-13T18:19:52.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:52 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/3602834841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:19:52.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:52 smithi114 ceph-mon[30897]: mds.? [v2:172.21.15.114:6838/2046516451,v1:172.21.15.114:6839/2046516451] up:boot 2024-09-13T18:19:52.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:52 smithi114 ceph-mon[30897]: fsmap 10 up:standby 2024-09-13T18:19:52.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:52 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-13T18:19:53.195 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:53 smithi114 ceph-mon[30897]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:53.196 INFO:journalctl@ceph.mds.a.smithi114.stdout:Sep 13 18:19:53 smithi114 systemd[1]: Starting Ceph mds.a for 97e7edf2-71fb-11ef-bceb-c7b262605968... 2024-09-13T18:19:53.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:53 smithi134 ceph-mon[34283]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:53.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:53 smithi204 ceph-mon[34777]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:53.455 INFO:journalctl@ceph.mds.a.smithi114.stdout:Sep 13 18:19:53 smithi114 podman[71198]: 2024-09-13 18:19:53.224484499 +0000 UTC m=+0.023526811 image pull 8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256 2024-09-13T18:19:53.455 INFO:journalctl@ceph.mds.a.smithi114.stdout:Sep 13 18:19:53 smithi114 podman[71198]: 2024-09-13 18:19:53.337503745 +0000 UTC m=+0.136546045 container create c8f4947b120ffba2412be0786d1097022fb9a1a6e996ca9be9c3fc1cec61307c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-a, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD) 2024-09-13T18:19:53.582 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:19:53.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:53.686+0000 7fcecbded640 1 -- 172.21.15.114:0/1790674607 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcec4100850 msgr2=0x7fcec41070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:53.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:53.686+0000 7fcecbded640 1 --2- 172.21.15.114:0/1790674607 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcec4100850 0x7fcec41070e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcebc009ca0 tx=0x7fcebc02a8c0 comp rx=0 tx=0).stop 2024-09-13T18:19:53.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:53.686+0000 7fcecbded640 1 -- 172.21.15.114:0/1790674607 shutdown_connections 2024-09-13T18:19:53.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:53.686+0000 7fcecbded640 1 --2- 172.21.15.114:0/1790674607 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcec4100850 0x7fcec41070e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:53.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:53.686+0000 7fcecbded640 1 --2- 172.21.15.114:0/1790674607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcec40ffe90 0x7fcec4100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:53.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:53.686+0000 7fcecbded640 1 --2- 172.21.15.114:0/1790674607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcec40feca0 0x7fcec40ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:53.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:53.686+0000 7fcecbded640 1 -- 172.21.15.114:0/1790674607 >> 172.21.15.114:0/1790674607 conn(0x7fcec40fa440 msgr2=0x7fcec40fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:53.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:53.686+0000 7fcecbded640 1 -- 172.21.15.114:0/1790674607 shutdown_connections 2024-09-13T18:19:53.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:53.686+0000 7fcecbded640 1 -- 172.21.15.114:0/1790674607 wait complete. 2024-09-13T18:19:53.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:53.686+0000 7fcecbded640 1 Processor -- start 2024-09-13T18:19:53.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:53.686+0000 7fcecbded640 1 -- start start 2024-09-13T18:19:53.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:53.687+0000 7fcecbded640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcec40feca0 0x7fcec4068110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:53.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:53.687+0000 7fcecbded640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcec40ffe90 0x7fcec419a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:53.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:53.687+0000 7fcecbded640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcec4100850 0x7fcec419cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:53.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:53.687+0000 7fcecbded640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec419d770 con 0x7fcec40ffe90 2024-09-13T18:19:53.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:53.687+0000 7fcecbded640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec419d8e0 con 0x7fcec4100850 2024-09-13T18:19:53.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:53.687+0000 7fcecbded640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec419daa0 con 0x7fcec40feca0 2024-09-13T18:19:53.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:53.687+0000 7fcec9388640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcec40ffe90 0x7fcec419a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:53.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:53.687+0000 7fcec9388640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcec40ffe90 0x7fcec419a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36146/0 (socket says 172.21.15.114:36146) 2024-09-13T18:19:53.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:53.687+0000 7fcec9388640 1 -- 172.21.15.114:0/1745144597 learned_addr learned my addr 172.21.15.114:0/1745144597 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:19:53.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:53.687+0000 7fcec9b89640 1 --2- 172.21.15.114:0/1745144597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcec40feca0 0x7fcec4068110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:53.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:53.687+0000 7fcec9388640 1 -- 172.21.15.114:0/1745144597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcec40feca0 msgr2=0x7fcec4068110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:53.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:53.687+0000 7fcec9388640 1 --2- 172.21.15.114:0/1745144597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcec40feca0 0x7fcec4068110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:53.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:53.687+0000 7fcec9388640 1 -- 172.21.15.114:0/1745144597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcec4100850 msgr2=0x7fcec419cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:53.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:53.687+0000 7fcec9388640 1 --2- 172.21.15.114:0/1745144597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcec4100850 0x7fcec419cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:53.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:53.687+0000 7fcec9388640 1 -- 172.21.15.114:0/1745144597 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcebc009900 con 0x7fcec40ffe90 2024-09-13T18:19:53.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:53.687+0000 7fcec9388640 1 --2- 172.21.15.114:0/1745144597 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcec40ffe90 0x7fcec419a970 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcec000c890 tx=0x7fcec000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:53.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:53.687+0000 7fcebaffd640 1 -- 172.21.15.114:0/1745144597 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec0004220 con 0x7fcec40ffe90 2024-09-13T18:19:53.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:53.687+0000 7fcecbded640 1 -- 172.21.15.114:0/1745144597 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcec419dd20 con 0x7fcec40ffe90 2024-09-13T18:19:53.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:53.687+0000 7fcebaffd640 1 -- 172.21.15.114:0/1745144597 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcec0007e40 con 0x7fcec40ffe90 2024-09-13T18:19:53.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:53.687+0000 7fcebaffd640 1 -- 172.21.15.114:0/1745144597 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec0004a00 con 0x7fcec40ffe90 2024-09-13T18:19:53.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:53.687+0000 7fcecbded640 1 -- 172.21.15.114:0/1745144597 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcec419e1d0 con 0x7fcec40ffe90 2024-09-13T18:19:53.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:53.688+0000 7fcecbded640 1 -- 172.21.15.114:0/1745144597 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcec41060d0 con 0x7fcec40ffe90 2024-09-13T18:19:53.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:53.690+0000 7fcebaffd640 1 -- 172.21.15.114:0/1745144597 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fcec00079b0 con 0x7fcec40ffe90 2024-09-13T18:19:53.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:53.690+0000 7fcebaffd640 1 --2- 172.21.15.114:0/1745144597 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fceb00a1090 0x7fceb00a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:53.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:53.690+0000 7fcebaffd640 1 -- 172.21.15.114:0/1745144597 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fcec00cc450 con 0x7fcec40ffe90 2024-09-13T18:19:53.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:53.690+0000 7fcec9b89640 1 --2- 172.21.15.114:0/1745144597 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fceb00a1090 0x7fceb00a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:53.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:53.691+0000 7fcec9b89640 1 --2- 172.21.15.114:0/1745144597 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fceb00a1090 0x7fceb00a3550 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fceb4005fd0 tx=0x7fceb400b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:53.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:53.698+0000 7fcebaffd640 1 -- 172.21.15.114:0/1745144597 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fcec0096080 con 0x7fcec40ffe90 2024-09-13T18:19:53.720 INFO:journalctl@ceph.mds.a.smithi114.stdout:Sep 13 18:19:53 smithi114 podman[71198]: 2024-09-13 18:19:53.59071929 +0000 UTC m=+0.389761600 container init c8f4947b120ffba2412be0786d1097022fb9a1a6e996ca9be9c3fc1cec61307c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-a, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-13T18:19:53.720 INFO:journalctl@ceph.mds.a.smithi114.stdout:Sep 13 18:19:53 smithi114 podman[71198]: 2024-09-13 18:19:53.593725115 +0000 UTC m=+0.392767420 container start c8f4947b120ffba2412be0786d1097022fb9a1a6e996ca9be9c3fc1cec61307c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083) 2024-09-13T18:19:53.720 INFO:journalctl@ceph.mds.a.smithi114.stdout:Sep 13 18:19:53 smithi114 ceph-mds[71233]: -- 172.21.15.114:0/698619036 <== mon.0 v2:172.21.15.114:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559c5613ad00 con 0x559c5610e800 2024-09-13T18:19:53.720 INFO:journalctl@ceph.mds.a.smithi114.stdout:Sep 13 18:19:53 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-a[71215]: starting mds.a at 2024-09-13T18:19:53.720 INFO:journalctl@ceph.mds.a.smithi114.stdout:Sep 13 18:19:53 smithi114 bash[71198]: c8f4947b120ffba2412be0786d1097022fb9a1a6e996ca9be9c3fc1cec61307c 2024-09-13T18:19:53.720 INFO:journalctl@ceph.mds.a.smithi114.stdout:Sep 13 18:19:53 smithi114 systemd[1]: Started Ceph mds.a for 97e7edf2-71fb-11ef-bceb-c7b262605968. 2024-09-13T18:19:53.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:53.905+0000 7fcecbded640 1 -- 172.21.15.114:0/1745144597 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcec410a3f0 con 0x7fcec40ffe90 2024-09-13T18:19:53.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:53.906+0000 7fcebaffd640 1 -- 172.21.15.114:0/1745144597 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8088 (secure 0 0 0) 0x7fcec00996a0 con 0x7fcec40ffe90 2024-09-13T18:19:53.909 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:19:53.909 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4162817497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4162817497},{"type":"v1","addr":"172.21.15.114:6835","nonce":4162817497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/2435428478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":2435428478},{"type":"v1","addr":"172.21.15.114:6837","nonce":2435428478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6839/2046516451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":2046516451},{"type":"v1","addr":"172.21.15.114:6839","nonce":2046516451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/507301496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":507301496},{"type":"v1","addr":"172.21.15.204:6833","nonce":507301496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/4028657219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4028657219},{"type":"v1","addr":"172.21.15.134:6833","nonce":4028657219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/57029119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":57029119},{"type":"v1","addr":"172.21.15.204:6835","nonce":57029119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3068880939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3068880939},{"type":"v1","addr":"172.21.15.204:6837","nonce":3068880939}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1150003275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1150003275},{"type":"v1","addr":"172.21.15.134:6835","nonce":1150003275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24475,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/344883912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":344883912},{"type":"v1","addr":"172.21.15.204:6839","nonce":344883912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/891692210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":891692210},{"type":"v1","addr":"172.21.15.134:6837","nonce":891692210}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2024-09-13T18:19:53.910 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 11 2024-09-13T18:19:53.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:53.908+0000 7fcecbded640 1 -- 172.21.15.114:0/1745144597 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fceb00a1090 msgr2=0x7fceb00a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:53.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:53.908+0000 7fcecbded640 1 --2- 172.21.15.114:0/1745144597 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fceb00a1090 0x7fceb00a3550 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fceb4005fd0 tx=0x7fceb400b410 comp rx=0 tx=0).stop 2024-09-13T18:19:53.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:53.908+0000 7fcecbded640 1 -- 172.21.15.114:0/1745144597 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcec40ffe90 msgr2=0x7fcec419a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:53.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:53.908+0000 7fcecbded640 1 --2- 172.21.15.114:0/1745144597 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcec40ffe90 0x7fcec419a970 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcec000c890 tx=0x7fcec000cd50 comp rx=0 tx=0).stop 2024-09-13T18:19:53.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:53.908+0000 7fcecbded640 1 -- 172.21.15.114:0/1745144597 shutdown_connections 2024-09-13T18:19:53.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:53.908+0000 7fcecbded640 1 --2- 172.21.15.114:0/1745144597 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fceb00a1090 0x7fceb00a3550 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:53.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:53.908+0000 7fcecbded640 1 --2- 172.21.15.114:0/1745144597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcec4100850 0x7fcec419cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:53.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:53.908+0000 7fcecbded640 1 --2- 172.21.15.114:0/1745144597 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcec40ffe90 0x7fcec419a970 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:53.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:53.908+0000 7fcecbded640 1 --2- 172.21.15.114:0/1745144597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcec40feca0 0x7fcec4068110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:53.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:53.908+0000 7fcecbded640 1 -- 172.21.15.114:0/1745144597 >> 172.21.15.114:0/1745144597 conn(0x7fcec40fa440 msgr2=0x7fcec41032a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:53.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:53.908+0000 7fcecbded640 1 -- 172.21.15.114:0/1745144597 shutdown_connections 2024-09-13T18:19:53.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:53.908+0000 7fcecbded640 1 -- 172.21.15.114:0/1745144597 wait complete. 2024-09-13T18:19:55.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:54 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:55.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:54 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:55.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:54 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:55.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:54 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T18:19:55.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:54 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T18:19:55.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:54 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:19:55.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:54 smithi134 ceph-mon[34283]: Deploying daemon mds.b on smithi134 2024-09-13T18:19:55.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:54 smithi134 ceph-mon[34283]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:55.120 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:54 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/1745144597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:19:55.120 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:54 smithi134 ceph-mon[34283]: mds.? [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] up:boot 2024-09-13T18:19:55.120 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:54 smithi134 ceph-mon[34283]: fsmap 11 up:standby 2024-09-13T18:19:55.120 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:54 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-13T18:19:55.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:54 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:55.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:54 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:55.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:54 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:55.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:54 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T18:19:55.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:54 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T18:19:55.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:54 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:19:55.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:54 smithi204 ceph-mon[34777]: Deploying daemon mds.b on smithi134 2024-09-13T18:19:55.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:54 smithi204 ceph-mon[34777]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:55.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:54 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/1745144597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:19:55.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:54 smithi204 ceph-mon[34777]: mds.? [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] up:boot 2024-09-13T18:19:55.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:54 smithi204 ceph-mon[34777]: fsmap 11 up:standby 2024-09-13T18:19:55.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:54 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-13T18:19:55.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:54 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:55.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:54 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:55.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:54 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:55.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:54 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-13T18:19:55.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:54 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-13T18:19:55.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:54 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:19:55.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:54 smithi114 ceph-mon[30897]: Deploying daemon mds.b on smithi134 2024-09-13T18:19:55.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:54 smithi114 ceph-mon[30897]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:55.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:54 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/1745144597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:19:55.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:54 smithi114 ceph-mon[30897]: mds.? [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] up:boot 2024-09-13T18:19:55.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:54 smithi114 ceph-mon[30897]: fsmap 11 up:standby 2024-09-13T18:19:55.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:54 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-13T18:19:55.869 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 13 18:19:55 smithi134 systemd[1]: Starting Ceph mds.b for 97e7edf2-71fb-11ef-bceb-c7b262605968... 2024-09-13T18:19:55.924 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:19:56.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.033+0000 7f4567b5e640 1 -- 172.21.15.114:0/3773724035 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45600ffe90 msgr2=0x7f4560100310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:56.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.033+0000 7f4567b5e640 1 --2- 172.21.15.114:0/3773724035 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45600ffe90 0x7f4560100310 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4550009900 tx=0x7f455002a410 comp rx=0 tx=0).stop 2024-09-13T18:19:56.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.033+0000 7f4567b5e640 1 -- 172.21.15.114:0/3773724035 shutdown_connections 2024-09-13T18:19:56.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.033+0000 7f4567b5e640 1 --2- 172.21.15.114:0/3773724035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4560100850 0x7f45601070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:56.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.033+0000 7f4567b5e640 1 --2- 172.21.15.114:0/3773724035 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45600ffe90 0x7f4560100310 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:56.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.033+0000 7f4567b5e640 1 --2- 172.21.15.114:0/3773724035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45600feca0 0x7f45600ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:56.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.033+0000 7f4567b5e640 1 -- 172.21.15.114:0/3773724035 >> 172.21.15.114:0/3773724035 conn(0x7f45600fa440 msgr2=0x7f45600fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:56.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.033+0000 7f4567b5e640 1 -- 172.21.15.114:0/3773724035 shutdown_connections 2024-09-13T18:19:56.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.033+0000 7f4567b5e640 1 -- 172.21.15.114:0/3773724035 wait complete. 2024-09-13T18:19:56.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.033+0000 7f4567b5e640 1 Processor -- start 2024-09-13T18:19:56.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.033+0000 7f4567b5e640 1 -- start start 2024-09-13T18:19:56.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.034+0000 7f4567b5e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45600feca0 0x7f4560068110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:56.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.034+0000 7f4567b5e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45600ffe90 0x7f456019aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:56.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.034+0000 7f4567b5e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4560100850 0x7f456019d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:56.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.034+0000 7f4567b5e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456019d830 con 0x7f45600ffe90 2024-09-13T18:19:56.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.034+0000 7f4567b5e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456019d9a0 con 0x7f4560100850 2024-09-13T18:19:56.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.034+0000 7f4567b5e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456019db60 con 0x7f45600feca0 2024-09-13T18:19:56.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.034+0000 7f45650f9640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45600ffe90 0x7f456019aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:56.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.034+0000 7f45650f9640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45600ffe90 0x7f456019aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36156/0 (socket says 172.21.15.114:36156) 2024-09-13T18:19:56.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.034+0000 7f45650f9640 1 -- 172.21.15.114:0/684820860 learned_addr learned my addr 172.21.15.114:0/684820860 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:19:56.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.034+0000 7f45658fa640 1 --2- 172.21.15.114:0/684820860 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45600feca0 0x7f4560068110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:56.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.034+0000 7f45660fb640 1 --2- 172.21.15.114:0/684820860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4560100850 0x7f456019d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:56.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.034+0000 7f45650f9640 1 -- 172.21.15.114:0/684820860 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45600feca0 msgr2=0x7f4560068110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:56.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.034+0000 7f45650f9640 1 --2- 172.21.15.114:0/684820860 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45600feca0 0x7f4560068110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:56.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.034+0000 7f45650f9640 1 -- 172.21.15.114:0/684820860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4560100850 msgr2=0x7f456019d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:56.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.034+0000 7f45650f9640 1 --2- 172.21.15.114:0/684820860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4560100850 0x7f456019d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:56.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.034+0000 7f45650f9640 1 -- 172.21.15.114:0/684820860 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4550009560 con 0x7f45600ffe90 2024-09-13T18:19:56.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.034+0000 7f45658fa640 1 --2- 172.21.15.114:0/684820860 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45600feca0 0x7f4560068110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:19:56.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.034+0000 7f45650f9640 1 --2- 172.21.15.114:0/684820860 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45600ffe90 0x7f456019aa30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f45500057c0 tx=0x7f45500042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:56.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.034+0000 7f4556ffd640 1 -- 172.21.15.114:0/684820860 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4550031d70 con 0x7f45600ffe90 2024-09-13T18:19:56.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.034+0000 7f4556ffd640 1 -- 172.21.15.114:0/684820860 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f45500044e0 con 0x7f45600ffe90 2024-09-13T18:19:56.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.034+0000 7f4556ffd640 1 -- 172.21.15.114:0/684820860 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4550003b80 con 0x7f45600ffe90 2024-09-13T18:19:56.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.034+0000 7f4567b5e640 1 -- 172.21.15.114:0/684820860 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f456019dd80 con 0x7f45600ffe90 2024-09-13T18:19:56.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.034+0000 7f4567b5e640 1 -- 172.21.15.114:0/684820860 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f456019e180 con 0x7f45600ffe90 2024-09-13T18:19:56.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.035+0000 7f4567b5e640 1 -- 172.21.15.114:0/684820860 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4560106220 con 0x7f45600ffe90 2024-09-13T18:19:56.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.037+0000 7f4556ffd640 1 -- 172.21.15.114:0/684820860 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f4550003ce0 con 0x7f45600ffe90 2024-09-13T18:19:56.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.037+0000 7f4556ffd640 1 --2- 172.21.15.114:0/684820860 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f454c0a1110 0x7f454c0a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:56.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.037+0000 7f45658fa640 1 --2- 172.21.15.114:0/684820860 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f454c0a1110 0x7f454c0a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:56.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.037+0000 7f4556ffd640 1 -- 172.21.15.114:0/684820860 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f45500eda40 con 0x7f45600ffe90 2024-09-13T18:19:56.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.038+0000 7f45658fa640 1 --2- 172.21.15.114:0/684820860 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f454c0a1110 0x7f454c0a35d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f45600ffcf0 tx=0x7f455c00a560 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:56.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.040+0000 7f4556ffd640 1 -- 172.21.15.114:0/684820860 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f45500b7670 con 0x7f45600ffe90 2024-09-13T18:19:56.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.238+0000 7f4567b5e640 1 -- 172.21.15.114:0/684820860 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f456010a3f0 con 0x7f45600ffe90 2024-09-13T18:19:56.241 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 13 18:19:55 smithi134 podman[63898]: 2024-09-13 18:19:55.845844611 +0000 UTC m=+0.015661810 image pull 8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256 2024-09-13T18:19:56.241 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 13 18:19:55 smithi134 podman[63898]: 2024-09-13 18:19:55.977948146 +0000 UTC m=+0.147765332 container create 90dfb0348c834e5dac0df30a466f40ecabb783e9263d332ba0808114dc4990ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-b, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2) 2024-09-13T18:19:56.241 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 13 18:19:56 smithi134 podman[63898]: 2024-09-13 18:19:56.238816336 +0000 UTC m=+0.408633528 container init 90dfb0348c834e5dac0df30a466f40ecabb783e9263d332ba0808114dc4990ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T18:19:56.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.238+0000 7f4556ffd640 1 -- 172.21.15.114:0/684820860 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+8849 (secure 0 0 0) 0x7f45500bac90 con 0x7f45600ffe90 2024-09-13T18:19:56.241 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:19:56.242 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4162817497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4162817497},{"type":"v1","addr":"172.21.15.114:6835","nonce":4162817497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/2435428478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":2435428478},{"type":"v1","addr":"172.21.15.114:6837","nonce":2435428478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6839/2046516451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":2046516451},{"type":"v1","addr":"172.21.15.114:6839","nonce":2046516451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6841/1956923446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":1956923446},{"type":"v1","addr":"172.21.15.114:6841","nonce":1956923446}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/507301496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":507301496},{"type":"v1","addr":"172.21.15.204:6833","nonce":507301496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/4028657219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4028657219},{"type":"v1","addr":"172.21.15.134:6833","nonce":4028657219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/57029119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":57029119},{"type":"v1","addr":"172.21.15.204:6835","nonce":57029119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3068880939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3068880939},{"type":"v1","addr":"172.21.15.204:6837","nonce":3068880939}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1150003275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1150003275},{"type":"v1","addr":"172.21.15.134:6835","nonce":1150003275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24475,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/344883912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":344883912},{"type":"v1","addr":"172.21.15.204:6839","nonce":344883912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/891692210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":891692210},{"type":"v1","addr":"172.21.15.134:6837","nonce":891692210}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2024-09-13T18:19:56.242 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 12 2024-09-13T18:19:56.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.241+0000 7f4567b5e640 1 -- 172.21.15.114:0/684820860 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f454c0a1110 msgr2=0x7f454c0a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:56.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.241+0000 7f4567b5e640 1 --2- 172.21.15.114:0/684820860 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f454c0a1110 0x7f454c0a35d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f45600ffcf0 tx=0x7f455c00a560 comp rx=0 tx=0).stop 2024-09-13T18:19:56.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.241+0000 7f4567b5e640 1 -- 172.21.15.114:0/684820860 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45600ffe90 msgr2=0x7f456019aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:56.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.241+0000 7f4567b5e640 1 --2- 172.21.15.114:0/684820860 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45600ffe90 0x7f456019aa30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f45500057c0 tx=0x7f45500042f0 comp rx=0 tx=0).stop 2024-09-13T18:19:56.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.241+0000 7f4567b5e640 1 -- 172.21.15.114:0/684820860 shutdown_connections 2024-09-13T18:19:56.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.241+0000 7f4567b5e640 1 --2- 172.21.15.114:0/684820860 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f454c0a1110 0x7f454c0a35d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:56.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.241+0000 7f4567b5e640 1 --2- 172.21.15.114:0/684820860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4560100850 0x7f456019d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:56.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.241+0000 7f4567b5e640 1 --2- 172.21.15.114:0/684820860 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45600ffe90 0x7f456019aa30 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:56.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.241+0000 7f4567b5e640 1 --2- 172.21.15.114:0/684820860 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45600feca0 0x7f4560068110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:56.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.241+0000 7f4567b5e640 1 -- 172.21.15.114:0/684820860 >> 172.21.15.114:0/684820860 conn(0x7f45600fa440 msgr2=0x7f45601033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:56.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.241+0000 7f4567b5e640 1 -- 172.21.15.114:0/684820860 shutdown_connections 2024-09-13T18:19:56.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:56.241+0000 7f4567b5e640 1 -- 172.21.15.114:0/684820860 wait complete. 2024-09-13T18:19:56.619 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 13 18:19:56 smithi134 podman[63898]: 2024-09-13 18:19:56.241955933 +0000 UTC m=+0.411773118 container start 90dfb0348c834e5dac0df30a466f40ecabb783e9263d332ba0808114dc4990ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.build-date=20240731) 2024-09-13T18:19:56.619 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 13 18:19:56 smithi134 ceph-mds[63920]: -- 172.21.15.134:0/502053883 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d96a4a8d00 con 0x55d96a47c800 2024-09-13T18:19:56.619 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 13 18:19:56 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-b[63915]: starting mds.b at 2024-09-13T18:19:56.619 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 13 18:19:56 smithi134 bash[63898]: 90dfb0348c834e5dac0df30a466f40ecabb783e9263d332ba0808114dc4990ca 2024-09-13T18:19:56.620 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 13 18:19:56 smithi134 systemd[1]: Started Ceph mds.b for 97e7edf2-71fb-11ef-bceb-c7b262605968. 2024-09-13T18:19:57.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:56 smithi114 ceph-mon[30897]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:57.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:56 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/684820860' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:19:57.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:56 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:57.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:56 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:57.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:56 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:57.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:56 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:57.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:56 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:57.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:56 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:19:57.222 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:56 smithi204 ceph-mon[34777]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:57.222 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:56 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/684820860' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:19:57.222 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:56 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:57.223 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:56 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:57.223 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:56 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:57.223 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:56 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:57.223 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:56 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:57.223 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:56 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:19:57.293 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:56 smithi134 ceph-mon[34283]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:57.294 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:56 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/684820860' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:19:57.294 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:56 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:57.294 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:56 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:57.294 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:56 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:57.294 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:56 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:57.294 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:56 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:57.294 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:56 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:19:58.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:57 smithi114 ceph-mon[30897]: mds.? [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] up:boot 2024-09-13T18:19:58.243 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:57 smithi114 ceph-mon[30897]: fsmap 12 up:standby 2024-09-13T18:19:58.243 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:57 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-13T18:19:58.243 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:57 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:58.244 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:57 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:58.244 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:57 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:58.244 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:57 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:58.244 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:57 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:58.244 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:57 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:58.245 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:57 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:19:58.245 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:57 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:19:58.245 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:57 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:58.255 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:19:58.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.360+0000 7f0bb7819640 1 -- 172.21.15.114:0/3959407717 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0bb00feca0 msgr2=0x7f0bb00ff0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:58.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.360+0000 7f0bb7819640 1 --2- 172.21.15.114:0/3959407717 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0bb00feca0 0x7f0bb00ff0a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0bac009900 tx=0x7f0bac02a410 comp rx=0 tx=0).stop 2024-09-13T18:19:58.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.360+0000 7f0bb7819640 1 -- 172.21.15.114:0/3959407717 shutdown_connections 2024-09-13T18:19:58.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.360+0000 7f0bb7819640 1 --2- 172.21.15.114:0/3959407717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bb0100850 0x7f0bb01070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:58.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.360+0000 7f0bb7819640 1 --2- 172.21.15.114:0/3959407717 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0bb00ffe90 0x7f0bb0100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:58.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.360+0000 7f0bb7819640 1 --2- 172.21.15.114:0/3959407717 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0bb00feca0 0x7f0bb00ff0a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:58.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.360+0000 7f0bb7819640 1 -- 172.21.15.114:0/3959407717 >> 172.21.15.114:0/3959407717 conn(0x7f0bb00fa440 msgr2=0x7f0bb00fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:58.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.360+0000 7f0bb7819640 1 -- 172.21.15.114:0/3959407717 shutdown_connections 2024-09-13T18:19:58.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.360+0000 7f0bb7819640 1 -- 172.21.15.114:0/3959407717 wait complete. 2024-09-13T18:19:58.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.361+0000 7f0bb7819640 1 Processor -- start 2024-09-13T18:19:58.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.361+0000 7f0bb7819640 1 -- start start 2024-09-13T18:19:58.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.361+0000 7f0bb7819640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0bb00feca0 0x7f0bb0194c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:58.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.361+0000 7f0bb7819640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0bb00ffe90 0x7f0bb01951c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:58.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.361+0000 7f0bb7819640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bb0100850 0x7f0bb019c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:58.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.361+0000 7f0bb7819640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb019c760 con 0x7f0bb00ffe90 2024-09-13T18:19:58.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.361+0000 7f0bb7819640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb019c8d0 con 0x7f0bb00feca0 2024-09-13T18:19:58.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.361+0000 7f0bb7819640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb019ca90 con 0x7f0bb0100850 2024-09-13T18:19:58.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.361+0000 7f0bb4db4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0bb00ffe90 0x7f0bb01951c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:58.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.361+0000 7f0bb55b5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0bb00feca0 0x7f0bb0194c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:58.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.361+0000 7f0bb4db4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0bb00ffe90 0x7f0bb01951c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36170/0 (socket says 172.21.15.114:36170) 2024-09-13T18:19:58.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.361+0000 7f0bb4db4640 1 -- 172.21.15.114:0/797159049 learned_addr learned my addr 172.21.15.114:0/797159049 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:19:58.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.361+0000 7f0bb5db6640 1 --2- 172.21.15.114:0/797159049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bb0100850 0x7f0bb019c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:58.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.361+0000 7f0bb4db4640 1 -- 172.21.15.114:0/797159049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bb0100850 msgr2=0x7f0bb019c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:58.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.361+0000 7f0bb4db4640 1 --2- 172.21.15.114:0/797159049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bb0100850 0x7f0bb019c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:58.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.361+0000 7f0bb4db4640 1 -- 172.21.15.114:0/797159049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0bb00feca0 msgr2=0x7f0bb0194c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:58.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.361+0000 7f0bb4db4640 1 --2- 172.21.15.114:0/797159049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0bb00feca0 0x7f0bb0194c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:58.373 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:57 smithi134 ceph-mon[34283]: mds.? [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] up:boot 2024-09-13T18:19:58.373 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:57 smithi134 ceph-mon[34283]: fsmap 12 up:standby 2024-09-13T18:19:58.374 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:57 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-13T18:19:58.374 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:57 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:58.374 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:57 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:58.374 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:57 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:58.374 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:57 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:58.374 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:57 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:58.374 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:57 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:58.374 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:57 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:19:58.374 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:57 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:19:58.374 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:57 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:58.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.361+0000 7f0bb4db4640 1 -- 172.21.15.114:0/797159049 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bac009560 con 0x7f0bb00ffe90 2024-09-13T18:19:58.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.361+0000 7f0bb4db4640 1 --2- 172.21.15.114:0/797159049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0bb00ffe90 0x7f0bb01951c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0ba000b4a0 tx=0x7f0ba000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:58.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.362+0000 7f0ba67fc640 1 -- 172.21.15.114:0/797159049 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba0010610 con 0x7f0bb00ffe90 2024-09-13T18:19:58.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.362+0000 7f0ba67fc640 1 -- 172.21.15.114:0/797159049 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0ba0010c00 con 0x7f0bb00ffe90 2024-09-13T18:19:58.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.362+0000 7f0bb7819640 1 -- 172.21.15.114:0/797159049 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bb019cd70 con 0x7f0bb00ffe90 2024-09-13T18:19:58.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.362+0000 7f0ba67fc640 1 -- 172.21.15.114:0/797159049 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba0017ca0 con 0x7f0bb00ffe90 2024-09-13T18:19:58.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.362+0000 7f0bb7819640 1 -- 172.21.15.114:0/797159049 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bb019d1a0 con 0x7f0bb00ffe90 2024-09-13T18:19:58.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.363+0000 7f0bb7819640 1 -- 172.21.15.114:0/797159049 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bb01061f0 con 0x7f0bb00ffe90 2024-09-13T18:19:58.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.364+0000 7f0ba67fc640 1 -- 172.21.15.114:0/797159049 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f0ba0010770 con 0x7f0bb00ffe90 2024-09-13T18:19:58.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.365+0000 7f0ba67fc640 1 --2- 172.21.15.114:0/797159049 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f0b9c0a1090 0x7f0b9c0a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:58.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.365+0000 7f0ba67fc640 1 -- 172.21.15.114:0/797159049 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0ba00cbea0 con 0x7f0bb00ffe90 2024-09-13T18:19:58.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.365+0000 7f0bb55b5640 1 --2- 172.21.15.114:0/797159049 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f0b9c0a1090 0x7f0b9c0a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:58.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.365+0000 7f0bb55b5640 1 --2- 172.21.15.114:0/797159049 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f0b9c0a1090 0x7f0b9c0a3550 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0bac0057c0 tx=0x7f0bac005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:58.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.369+0000 7f0ba67fc640 1 -- 172.21.15.114:0/797159049 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f0ba0095a50 con 0x7f0bb00ffe90 2024-09-13T18:19:58.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:57 smithi204 ceph-mon[34777]: mds.? [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] up:boot 2024-09-13T18:19:58.447 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:57 smithi204 ceph-mon[34777]: fsmap 12 up:standby 2024-09-13T18:19:58.447 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:57 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-13T18:19:58.447 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:57 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:58.447 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:57 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:58.447 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:57 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:58.447 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:57 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:58.447 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:57 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:58.447 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:57 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:58.447 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:57 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:19:58.447 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:57 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:19:58.447 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:57 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:19:58.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.572+0000 7f0bb7819640 1 -- 172.21.15.114:0/797159049 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0bb010a470 con 0x7f0bb00ffe90 2024-09-13T18:19:58.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.573+0000 7f0ba67fc640 1 -- 172.21.15.114:0/797159049 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+9610 (secure 0 0 0) 0x7f0ba0097980 con 0x7f0bb00ffe90 2024-09-13T18:19:58.576 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:19:58.576 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4162817497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4162817497},{"type":"v1","addr":"172.21.15.114:6835","nonce":4162817497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/2435428478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":2435428478},{"type":"v1","addr":"172.21.15.114:6837","nonce":2435428478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6839/2046516451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":2046516451},{"type":"v1","addr":"172.21.15.114:6839","nonce":2046516451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6841/1956923446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":1956923446},{"type":"v1","addr":"172.21.15.114:6841","nonce":1956923446}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/507301496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":507301496},{"type":"v1","addr":"172.21.15.204:6833","nonce":507301496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/4028657219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4028657219},{"type":"v1","addr":"172.21.15.134:6833","nonce":4028657219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/57029119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":57029119},{"type":"v1","addr":"172.21.15.204:6835","nonce":57029119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3068880939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3068880939},{"type":"v1","addr":"172.21.15.204:6837","nonce":3068880939}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1150003275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1150003275},{"type":"v1","addr":"172.21.15.134:6835","nonce":1150003275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24475,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/344883912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":344883912},{"type":"v1","addr":"172.21.15.204:6839","nonce":344883912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/891692210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":891692210},{"type":"v1","addr":"172.21.15.134:6837","nonce":891692210}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24521,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6839/3466309724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3466309724},{"type":"v1","addr":"172.21.15.134:6839","nonce":3466309724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[]} 2024-09-13T18:19:58.577 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 13 2024-09-13T18:19:58.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.575+0000 7f0bb7819640 1 -- 172.21.15.114:0/797159049 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f0b9c0a1090 msgr2=0x7f0b9c0a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:58.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.575+0000 7f0bb7819640 1 --2- 172.21.15.114:0/797159049 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f0b9c0a1090 0x7f0b9c0a3550 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0bac0057c0 tx=0x7f0bac005a20 comp rx=0 tx=0).stop 2024-09-13T18:19:58.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.575+0000 7f0bb7819640 1 -- 172.21.15.114:0/797159049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0bb00ffe90 msgr2=0x7f0bb01951c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:58.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.575+0000 7f0bb7819640 1 --2- 172.21.15.114:0/797159049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0bb00ffe90 0x7f0bb01951c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0ba000b4a0 tx=0x7f0ba000b960 comp rx=0 tx=0).stop 2024-09-13T18:19:58.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.575+0000 7f0bb7819640 1 -- 172.21.15.114:0/797159049 shutdown_connections 2024-09-13T18:19:58.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.575+0000 7f0bb7819640 1 --2- 172.21.15.114:0/797159049 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f0b9c0a1090 0x7f0b9c0a3550 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:58.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.575+0000 7f0bb7819640 1 --2- 172.21.15.114:0/797159049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bb0100850 0x7f0bb019c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:58.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.575+0000 7f0bb7819640 1 --2- 172.21.15.114:0/797159049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0bb00ffe90 0x7f0bb01951c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:58.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.575+0000 7f0bb7819640 1 --2- 172.21.15.114:0/797159049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0bb00feca0 0x7f0bb0194c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:58.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.575+0000 7f0bb7819640 1 -- 172.21.15.114:0/797159049 >> 172.21.15.114:0/797159049 conn(0x7f0bb00fa440 msgr2=0x7f0bb01033c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:58.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.575+0000 7f0bb7819640 1 -- 172.21.15.114:0/797159049 shutdown_connections 2024-09-13T18:19:58.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.575+0000 7f0bb7819640 1 -- 172.21.15.114:0/797159049 wait complete. 2024-09-13T18:19:58.589 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:19:58.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.689+0000 7f5a7c184640 1 -- 172.21.15.114:0/1318983749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a740fe610 msgr2=0x7f5a74100a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:58.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.689+0000 7f5a7c184640 1 --2- 172.21.15.114:0/1318983749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a740fe610 0x7f5a74100a20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5a70009900 tx=0x7f5a7002a410 comp rx=0 tx=0).stop 2024-09-13T18:19:58.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.689+0000 7f5a7c184640 1 -- 172.21.15.114:0/1318983749 shutdown_connections 2024-09-13T18:19:58.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.689+0000 7f5a7c184640 1 --2- 172.21.15.114:0/1318983749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a74103910 0x7f5a74103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:58.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.689+0000 7f5a7c184640 1 --2- 172.21.15.114:0/1318983749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a74100f60 0x7f5a741033d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:58.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.689+0000 7f5a7c184640 1 --2- 172.21.15.114:0/1318983749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a740fe610 0x7f5a74100a20 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:58.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.689+0000 7f5a7c184640 1 -- 172.21.15.114:0/1318983749 >> 172.21.15.114:0/1318983749 conn(0x7f5a740fa440 msgr2=0x7f5a740fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:58.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.689+0000 7f5a7c184640 1 -- 172.21.15.114:0/1318983749 shutdown_connections 2024-09-13T18:19:58.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.689+0000 7f5a7c184640 1 -- 172.21.15.114:0/1318983749 wait complete. 2024-09-13T18:19:58.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.690+0000 7f5a7c184640 1 Processor -- start 2024-09-13T18:19:58.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.690+0000 7f5a7c184640 1 -- start start 2024-09-13T18:19:58.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.690+0000 7f5a7c184640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a740fe610 0x7f5a7419c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:58.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.690+0000 7f5a7c184640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a74100f60 0x7f5a7419c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:58.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.690+0000 7f5a7c184640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a74103910 0x7f5a741a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:58.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.690+0000 7f5a7c184640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a741a3d30 con 0x7f5a740fe610 2024-09-13T18:19:58.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.690+0000 7f5a7c184640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a741a3ea0 con 0x7f5a74103910 2024-09-13T18:19:58.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.690+0000 7f5a7c184640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a741a4060 con 0x7f5a74100f60 2024-09-13T18:19:58.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.690+0000 7f5a79f20640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a740fe610 0x7f5a7419c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:58.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.690+0000 7f5a79f20640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a740fe610 0x7f5a7419c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36192/0 (socket says 172.21.15.114:36192) 2024-09-13T18:19:58.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.690+0000 7f5a79f20640 1 -- 172.21.15.114:0/2313064554 learned_addr learned my addr 172.21.15.114:0/2313064554 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:19:58.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.690+0000 7f5a7a721640 1 --2- 172.21.15.114:0/2313064554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a74103910 0x7f5a741a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:58.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.690+0000 7f5a7971f640 1 --2- 172.21.15.114:0/2313064554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a74100f60 0x7f5a7419c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:58.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.691+0000 7f5a79f20640 1 -- 172.21.15.114:0/2313064554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a74100f60 msgr2=0x7f5a7419c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:58.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.691+0000 7f5a79f20640 1 --2- 172.21.15.114:0/2313064554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a74100f60 0x7f5a7419c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:58.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.691+0000 7f5a79f20640 1 -- 172.21.15.114:0/2313064554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a74103910 msgr2=0x7f5a741a37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:58.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.691+0000 7f5a79f20640 1 --2- 172.21.15.114:0/2313064554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a74103910 0x7f5a741a37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:58.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.691+0000 7f5a79f20640 1 -- 172.21.15.114:0/2313064554 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a70009560 con 0x7f5a740fe610 2024-09-13T18:19:58.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.691+0000 7f5a79f20640 1 --2- 172.21.15.114:0/2313064554 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a740fe610 0x7f5a7419c240 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5a700057c0 tx=0x7f5a700042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:58.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.691+0000 7f5a6affd640 1 -- 172.21.15.114:0/2313064554 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a70031d70 con 0x7f5a740fe610 2024-09-13T18:19:58.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.691+0000 7f5a7c184640 1 -- 172.21.15.114:0/2313064554 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a741a42e0 con 0x7f5a740fe610 2024-09-13T18:19:58.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.691+0000 7f5a6affd640 1 -- 172.21.15.114:0/2313064554 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5a700044e0 con 0x7f5a740fe610 2024-09-13T18:19:58.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.691+0000 7f5a6affd640 1 -- 172.21.15.114:0/2313064554 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a700039c0 con 0x7f5a740fe610 2024-09-13T18:19:58.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.691+0000 7f5a7c184640 1 -- 172.21.15.114:0/2313064554 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a741a4790 con 0x7f5a740fe610 2024-09-13T18:19:58.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.692+0000 7f5a7c184640 1 -- 172.21.15.114:0/2313064554 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a740fbf30 con 0x7f5a740fe610 2024-09-13T18:19:58.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.694+0000 7f5a6affd640 1 -- 172.21.15.114:0/2313064554 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f5a70003b40 con 0x7f5a740fe610 2024-09-13T18:19:58.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.695+0000 7f5a6affd640 1 --2- 172.21.15.114:0/2313064554 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f5a600a1090 0x7f5a600a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:58.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.695+0000 7f5a7971f640 1 --2- 172.21.15.114:0/2313064554 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f5a600a1090 0x7f5a600a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:58.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.695+0000 7f5a6affd640 1 -- 172.21.15.114:0/2313064554 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5a700ed970 con 0x7f5a740fe610 2024-09-13T18:19:58.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.695+0000 7f5a7971f640 1 --2- 172.21.15.114:0/2313064554 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f5a600a1090 0x7f5a600a3550 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5a7419d750 tx=0x7f5a6400a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:58.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.697+0000 7f5a6affd640 1 -- 172.21.15.114:0/2313064554 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f5a700b75a0 con 0x7f5a740fe610 2024-09-13T18:19:58.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.863+0000 7f5a7c184640 1 -- 172.21.15.114:0/2313064554 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5a74111b70 con 0x7f5a740fe610 2024-09-13T18:19:58.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.864+0000 7f5a6affd640 1 -- 172.21.15.114:0/2313064554 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18491 (secure 0 0 0) 0x7f5a700babc0 con 0x7f5a740fe610 2024-09-13T18:19:58.867 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:19:58.867 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":81,"fsid":"97e7edf2-71fb-11ef-bceb-c7b262605968","created":"2024-09-13T18:12:36.360625+0000","modified":"2024-09-13T18:19:13.938715+0000","last_up_change":"2024-09-13T18:19:12.350370+0000","last_in_change":"2024-09-13T18:18:52.321596+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:15:40.129659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9914f01a-9f6f-4878-b955-83570199c354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6803","nonce":3499525826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6805","nonce":3499525826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6809","nonce":3499525826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6807","nonce":3499525826}]},"public_addr":"172.21.15.114:6803/3499525826","cluster_addr":"172.21.15.114:6805/3499525826","heartbeat_back_addr":"172.21.15.114:6809/3499525826","heartbeat_front_addr":"172.21.15.114:6807/3499525826","state":["exists","up"]},{"osd":1,"uuid":"ef65e012-7469-4c20-b354-17022bca63c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6801","nonce":4264264866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6803","nonce":4264264866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6807","nonce":4264264866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6805","nonce":4264264866}]},"public_addr":"172.21.15.134:6801/4264264866","cluster_addr":"172.21.15.134:6803/4264264866","heartbeat_back_addr":"172.21.15.134:6807/4264264866","heartbeat_front_addr":"172.21.15.134:6805/4264264866","state":["exists","up"]},{"osd":2,"uuid":"578213fd-715f-4be7-91fa-254f3d0e3022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6801","nonce":2288807707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6803","nonce":2288807707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6807","nonce":2288807707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6805","nonce":2288807707}]},"public_addr":"172.21.15.204:6801/2288807707","cluster_addr":"172.21.15.204:6803/2288807707","heartbeat_back_addr":"172.21.15.204:6807/2288807707","heartbeat_front_addr":"172.21.15.204:6805/2288807707","state":["exists","up"]},{"osd":3,"uuid":"b64aab3f-1e52-4828-8397-74af5e05f2cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6811","nonce":2904204254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6813","nonce":2904204254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6817","nonce":2904204254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6815","nonce":2904204254}]},"public_addr":"172.21.15.114:6811/2904204254","cluster_addr":"172.21.15.114:6813/2904204254","heartbeat_back_addr":"172.21.15.114:6817/2904204254","heartbeat_front_addr":"172.21.15.114:6815/2904204254","state":["exists","up"]},{"osd":4,"uuid":"f38943d6-39c0-42e1-b427-cb6bb90ac38a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6809","nonce":1773945333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6811","nonce":1773945333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6815","nonce":1773945333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6813","nonce":1773945333}]},"public_addr":"172.21.15.134:6809/1773945333","cluster_addr":"172.21.15.134:6811/1773945333","heartbeat_back_addr":"172.21.15.134:6815/1773945333","heartbeat_front_addr":"172.21.15.134:6813/1773945333","state":["exists","up"]},{"osd":5,"uuid":"bb38f361-8e00-4178-9752-5ddbf3af20e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6809","nonce":2371778055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6811","nonce":2371778055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6815","nonce":2371778055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6813","nonce":2371778055}]},"public_addr":"172.21.15.204:6809/2371778055","cluster_addr":"172.21.15.204:6811/2371778055","heartbeat_back_addr":"172.21.15.204:6815/2371778055","heartbeat_front_addr":"172.21.15.204:6813/2371778055","state":["exists","up"]},{"osd":6,"uuid":"93d40a03-e7ff-4a3f-9efa-5c5403c9983d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6819","nonce":1484325044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6821","nonce":1484325044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6825","nonce":1484325044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6823","nonce":1484325044}]},"public_addr":"172.21.15.114:6819/1484325044","cluster_addr":"172.21.15.114:6821/1484325044","heartbeat_back_addr":"172.21.15.114:6825/1484325044","heartbeat_front_addr":"172.21.15.114:6823/1484325044","state":["exists","up"]},{"osd":7,"uuid":"982f629a-1b95-4be1-8e75-5ddce1ad77b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6817","nonce":799246614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6819","nonce":799246614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6823","nonce":799246614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6821","nonce":799246614}]},"public_addr":"172.21.15.134:6817/799246614","cluster_addr":"172.21.15.134:6819/799246614","heartbeat_back_addr":"172.21.15.134:6823/799246614","heartbeat_front_addr":"172.21.15.134:6821/799246614","state":["exists","up"]},{"osd":8,"uuid":"409beb9b-8227-4dad-a4a0-f0e5ce70dc5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6817","nonce":858331805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6819","nonce":858331805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6823","nonce":858331805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6821","nonce":858331805}]},"public_addr":"172.21.15.204:6817/858331805","cluster_addr":"172.21.15.204:6819/858331805","heartbeat_back_addr":"172.21.15.204:6823/858331805","heartbeat_front_addr":"172.21.15.204:6821/858331805","state":["exists","up"]},{"osd":9,"uuid":"a345a15c-b1fc-4877-b445-418eebf0d0ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6827","nonce":2172392854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6829","nonce":2172392854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6833","nonce":2172392854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6831","nonce":2172392854}]},"public_addr":"172.21.15.114:6827/2172392854","cluster_addr":"172.21.15.114:6829/2172392854","heartbeat_back_addr":"172.21.15.114:6833/2172392854","heartbeat_front_addr":"172.21.15.114:6831/2172392854","state":["exists","up"]},{"osd":10,"uuid":"464d995f-d90b-4caa-addb-1f7562bd74b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6825","nonce":3073513468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6827","nonce":3073513468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6831","nonce":3073513468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6829","nonce":3073513468}]},"public_addr":"172.21.15.134:6825/3073513468","cluster_addr":"172.21.15.134:6827/3073513468","heartbeat_back_addr":"172.21.15.134:6831/3073513468","heartbeat_front_addr":"172.21.15.134:6829/3073513468","state":["exists","up"]},{"osd":11,"uuid":"71dd794d-326c-477f-8d02-ee32bff1350d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6825","nonce":3239521765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6827","nonce":3239521765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6831","nonce":3239521765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6829","nonce":3239521765}]},"public_addr":"172.21.15.204:6825/3239521765","cluster_addr":"172.21.15.204:6827/3239521765","heartbeat_back_addr":"172.21.15.204:6831/3239521765","heartbeat_front_addr":"172.21.15.204:6829/3239521765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:14:54.542687+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:15:15.746055+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:15:38.371131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:01.383932+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:24.796237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:48.422214+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:11.284323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:34.570365+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:58.232855+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:18:21.609596+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:18:45.565859+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:19:10.017969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:6801/4281866858":"2024-09-14T18:13:25.823027+0000","172.21.15.114:6800/4281866858":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2949422849":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2034742251":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/722571999":"2024-09-14T18:13:03.521116+0000","172.21.15.114:6801/830738794":"2024-09-14T18:13:03.521116+0000","172.21.15.114:6800/830738794":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/2828364892":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/4257828906":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/3544951322":"2024-09-14T18:13:03.521116+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:19:58.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.866+0000 7f5a7c184640 1 -- 172.21.15.114:0/2313064554 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f5a600a1090 msgr2=0x7f5a600a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:58.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.866+0000 7f5a7c184640 1 --2- 172.21.15.114:0/2313064554 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f5a600a1090 0x7f5a600a3550 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5a7419d750 tx=0x7f5a6400a2d0 comp rx=0 tx=0).stop 2024-09-13T18:19:58.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.866+0000 7f5a7c184640 1 -- 172.21.15.114:0/2313064554 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a740fe610 msgr2=0x7f5a7419c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:58.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.866+0000 7f5a7c184640 1 --2- 172.21.15.114:0/2313064554 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a740fe610 0x7f5a7419c240 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5a700057c0 tx=0x7f5a700042f0 comp rx=0 tx=0).stop 2024-09-13T18:19:58.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.866+0000 7f5a7c184640 1 -- 172.21.15.114:0/2313064554 shutdown_connections 2024-09-13T18:19:58.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.866+0000 7f5a7c184640 1 --2- 172.21.15.114:0/2313064554 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f5a600a1090 0x7f5a600a3550 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:58.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.866+0000 7f5a7c184640 1 --2- 172.21.15.114:0/2313064554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a74103910 0x7f5a741a37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:58.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.866+0000 7f5a7c184640 1 --2- 172.21.15.114:0/2313064554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a74100f60 0x7f5a7419c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:58.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.866+0000 7f5a7c184640 1 --2- 172.21.15.114:0/2313064554 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a740fe610 0x7f5a7419c240 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:58.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.866+0000 7f5a7c184640 1 -- 172.21.15.114:0/2313064554 >> 172.21.15.114:0/2313064554 conn(0x7f5a740fa440 msgr2=0x7f5a741019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:58.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.866+0000 7f5a7c184640 1 -- 172.21.15.114:0/2313064554 shutdown_connections 2024-09-13T18:19:58.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.866+0000 7f5a7c184640 1 -- 172.21.15.114:0/2313064554 wait complete. 2024-09-13T18:19:58.879 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T18:15:40.129659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T18:19:58.879 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T18:19:58.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.981+0000 7fb6e79f4640 1 -- 172.21.15.114:0/851811080 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb6e0100830 msgr2=0x7fb6e01070c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:58.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.981+0000 7fb6e79f4640 1 --2- 172.21.15.114:0/851811080 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb6e0100830 0x7fb6e01070c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8009ca0 tx=0x7fb6d802a8c0 comp rx=0 tx=0).stop 2024-09-13T18:19:58.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.981+0000 7fb6e79f4640 1 -- 172.21.15.114:0/851811080 shutdown_connections 2024-09-13T18:19:58.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.981+0000 7fb6e79f4640 1 --2- 172.21.15.114:0/851811080 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb6e0100830 0x7fb6e01070c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:58.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.981+0000 7fb6e79f4640 1 --2- 172.21.15.114:0/851811080 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6e00ffe70 0x7fb6e01002f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:58.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.981+0000 7fb6e79f4640 1 --2- 172.21.15.114:0/851811080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6e00fec80 0x7fb6e00ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:58.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.981+0000 7fb6e79f4640 1 -- 172.21.15.114:0/851811080 >> 172.21.15.114:0/851811080 conn(0x7fb6e00fa3e0 msgr2=0x7fb6e00fc820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:58.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.981+0000 7fb6e79f4640 1 -- 172.21.15.114:0/851811080 shutdown_connections 2024-09-13T18:19:58.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.981+0000 7fb6e79f4640 1 -- 172.21.15.114:0/851811080 wait complete. 2024-09-13T18:19:58.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.981+0000 7fb6e79f4640 1 Processor -- start 2024-09-13T18:19:58.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.981+0000 7fb6e79f4640 1 -- start start 2024-09-13T18:19:58.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.981+0000 7fb6e79f4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6e00fec80 0x7fb6e0069120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:58.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.982+0000 7fb6e79f4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb6e00ffe70 0x7fb6e019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:58.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.982+0000 7fb6e79f4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6e0100830 0x7fb6e019d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:58.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.982+0000 7fb6e79f4640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e019d7e0 con 0x7fb6e00ffe70 2024-09-13T18:19:58.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.982+0000 7fb6e79f4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e019d950 con 0x7fb6e0100830 2024-09-13T18:19:58.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.982+0000 7fb6e79f4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e019db10 con 0x7fb6e00fec80 2024-09-13T18:19:58.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.982+0000 7fb6e4f8f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb6e00ffe70 0x7fb6e019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:58.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.982+0000 7fb6e4f8f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb6e00ffe70 0x7fb6e019a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36210/0 (socket says 172.21.15.114:36210) 2024-09-13T18:19:58.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.982+0000 7fb6e4f8f640 1 -- 172.21.15.114:0/2236579703 learned_addr learned my addr 172.21.15.114:0/2236579703 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:19:58.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.982+0000 7fb6e5f91640 1 --2- 172.21.15.114:0/2236579703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6e0100830 0x7fb6e019d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:58.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.982+0000 7fb6e5790640 1 --2- 172.21.15.114:0/2236579703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6e00fec80 0x7fb6e0069120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:58.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.982+0000 7fb6e4f8f640 1 -- 172.21.15.114:0/2236579703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6e00fec80 msgr2=0x7fb6e0069120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:58.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.982+0000 7fb6e4f8f640 1 --2- 172.21.15.114:0/2236579703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6e00fec80 0x7fb6e0069120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:58.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.982+0000 7fb6e4f8f640 1 -- 172.21.15.114:0/2236579703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6e0100830 msgr2=0x7fb6e019d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:58.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.982+0000 7fb6e4f8f640 1 --2- 172.21.15.114:0/2236579703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6e0100830 0x7fb6e019d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:58.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.982+0000 7fb6e4f8f640 1 -- 172.21.15.114:0/2236579703 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d8009900 con 0x7fb6e00ffe70 2024-09-13T18:19:58.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.982+0000 7fb6e4f8f640 1 --2- 172.21.15.114:0/2236579703 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb6e00ffe70 0x7fb6e019a9e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc00b4a0 tx=0x7fb6dc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:58.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.982+0000 7fb6d27fc640 1 -- 172.21.15.114:0/2236579703 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc010610 con 0x7fb6e00ffe70 2024-09-13T18:19:58.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.982+0000 7fb6d27fc640 1 -- 172.21.15.114:0/2236579703 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb6dc010c00 con 0x7fb6e00ffe70 2024-09-13T18:19:58.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.982+0000 7fb6e79f4640 1 -- 172.21.15.114:0/2236579703 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6e019dd90 con 0x7fb6e00ffe70 2024-09-13T18:19:58.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.982+0000 7fb6d27fc640 1 -- 172.21.15.114:0/2236579703 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc017ca0 con 0x7fb6e00ffe70 2024-09-13T18:19:58.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.982+0000 7fb6e79f4640 1 -- 172.21.15.114:0/2236579703 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6e019e1f0 con 0x7fb6e00ffe70 2024-09-13T18:19:58.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.983+0000 7fb6e79f4640 1 -- 172.21.15.114:0/2236579703 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6e01061b0 con 0x7fb6e00ffe70 2024-09-13T18:19:58.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.984+0000 7fb6d27fc640 1 -- 172.21.15.114:0/2236579703 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fb6dc010770 con 0x7fb6e00ffe70 2024-09-13T18:19:58.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.985+0000 7fb6d27fc640 1 --2- 172.21.15.114:0/2236579703 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb6cc0a1090 0x7fb6cc0a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:58.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.985+0000 7fb6e5790640 1 --2- 172.21.15.114:0/2236579703 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb6cc0a1090 0x7fb6cc0a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:58.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.985+0000 7fb6d27fc640 1 -- 172.21.15.114:0/2236579703 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb6dc0cbb00 con 0x7fb6e00ffe70 2024-09-13T18:19:58.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.985+0000 7fb6e5790640 1 --2- 172.21.15.114:0/2236579703 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb6cc0a1090 0x7fb6cc0a3550 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4005fd0 tx=0x7fb6d4005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:58.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:58.989+0000 7fb6d27fc640 1 -- 172.21.15.114:0/2236579703 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fb6dc095730 con 0x7fb6e00ffe70 2024-09-13T18:19:58.999 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:58 smithi134 ceph-mon[34283]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:58.999 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:58 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:19:58.999 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:58 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/797159049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:19:58.999 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:19:58 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/2313064554' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:19:59.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.180+0000 7fb6e79f4640 1 -- 172.21.15.114:0/2236579703 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb6e010a620 con 0x7fb6e00ffe70 2024-09-13T18:19:59.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.181+0000 7fb6d27fc640 1 -- 172.21.15.114:0/2236579703 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb6dc098d50 con 0x7fb6e00ffe70 2024-09-13T18:19:59.183 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 1 2024-09-13T18:19:59.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.183+0000 7fb6e79f4640 1 -- 172.21.15.114:0/2236579703 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb6cc0a1090 msgr2=0x7fb6cc0a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:59.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.183+0000 7fb6e79f4640 1 --2- 172.21.15.114:0/2236579703 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb6cc0a1090 0x7fb6cc0a3550 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4005fd0 tx=0x7fb6d4005850 comp rx=0 tx=0).stop 2024-09-13T18:19:59.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.183+0000 7fb6e79f4640 1 -- 172.21.15.114:0/2236579703 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb6e00ffe70 msgr2=0x7fb6e019a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:59.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.183+0000 7fb6e79f4640 1 --2- 172.21.15.114:0/2236579703 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb6e00ffe70 0x7fb6e019a9e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc00b4a0 tx=0x7fb6dc00b960 comp rx=0 tx=0).stop 2024-09-13T18:19:59.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.183+0000 7fb6e79f4640 1 -- 172.21.15.114:0/2236579703 shutdown_connections 2024-09-13T18:19:59.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.183+0000 7fb6e79f4640 1 --2- 172.21.15.114:0/2236579703 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb6cc0a1090 0x7fb6cc0a3550 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:59.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.183+0000 7fb6e79f4640 1 --2- 172.21.15.114:0/2236579703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6e0100830 0x7fb6e019d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:59.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.183+0000 7fb6e79f4640 1 --2- 172.21.15.114:0/2236579703 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb6e00ffe70 0x7fb6e019a9e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:59.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.183+0000 7fb6e79f4640 1 --2- 172.21.15.114:0/2236579703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6e00fec80 0x7fb6e0069120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:59.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.183+0000 7fb6e79f4640 1 -- 172.21.15.114:0/2236579703 >> 172.21.15.114:0/2236579703 conn(0x7fb6e00fa3e0 msgr2=0x7fb6e0103380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:59.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.183+0000 7fb6e79f4640 1 -- 172.21.15.114:0/2236579703 shutdown_connections 2024-09-13T18:19:59.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.183+0000 7fb6e79f4640 1 -- 172.21.15.114:0/2236579703 wait complete. 2024-09-13T18:19:59.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:58 smithi204 ceph-mon[34777]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:59.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:58 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:19:59.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:58 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/797159049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:19:59.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:19:58 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/2313064554' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:19:59.196 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:58 smithi114 ceph-mon[30897]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:19:59.196 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:58 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:19:59.196 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:58 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/797159049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:19:59.196 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:19:58 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2313064554' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:19:59.198 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-13T18:19:59.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.301+0000 7f7ca4945640 1 -- 172.21.15.114:0/4096211522 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ca0100f80 msgr2=0x7f7ca0101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:59.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.301+0000 7f7ca4945640 1 --2- 172.21.15.114:0/4096211522 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ca0100f80 0x7f7ca0101400 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7c94009900 tx=0x7f7c9402a410 comp rx=0 tx=0).stop 2024-09-13T18:19:59.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.301+0000 7f7ca4945640 1 -- 172.21.15.114:0/4096211522 shutdown_connections 2024-09-13T18:19:59.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.301+0000 7f7ca4945640 1 --2- 172.21.15.114:0/4096211522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ca0101940 0x7f7ca01081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:59.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.301+0000 7f7ca4945640 1 --2- 172.21.15.114:0/4096211522 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ca0100f80 0x7f7ca0101400 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:59.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.301+0000 7f7ca4945640 1 --2- 172.21.15.114:0/4096211522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ca00ffd90 0x7f7ca0100190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:59.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.301+0000 7f7ca4945640 1 -- 172.21.15.114:0/4096211522 >> 172.21.15.114:0/4096211522 conn(0x7f7ca00fb510 msgr2=0x7f7ca00fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:59.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.301+0000 7f7ca4945640 1 -- 172.21.15.114:0/4096211522 shutdown_connections 2024-09-13T18:19:59.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.301+0000 7f7ca4945640 1 -- 172.21.15.114:0/4096211522 wait complete. 2024-09-13T18:19:59.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.302+0000 7f7ca4945640 1 Processor -- start 2024-09-13T18:19:59.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.302+0000 7f7ca4945640 1 -- start start 2024-09-13T18:19:59.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.302+0000 7f7ca4945640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ca00ffd90 0x7f7ca0074260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:59.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.302+0000 7f7ca4945640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ca0100f80 0x7f7ca00728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:59.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.302+0000 7f7ca4945640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ca0101940 0x7f7ca0072de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:59.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.302+0000 7f7ca4945640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca0073540 con 0x7f7ca00ffd90 2024-09-13T18:19:59.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.302+0000 7f7ca4945640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca00736b0 con 0x7f7ca0100f80 2024-09-13T18:19:59.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.302+0000 7f7ca4945640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca019fcd0 con 0x7f7ca0101940 2024-09-13T18:19:59.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.303+0000 7f7c9e59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ca00ffd90 0x7f7ca0074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:59.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.303+0000 7f7c9e59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ca00ffd90 0x7f7ca0074260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36234/0 (socket says 172.21.15.114:36234) 2024-09-13T18:19:59.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.303+0000 7f7c9e59c640 1 -- 172.21.15.114:0/3324645908 learned_addr learned my addr 172.21.15.114:0/3324645908 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:19:59.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.303+0000 7f7c9dd9b640 1 --2- 172.21.15.114:0/3324645908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ca0100f80 0x7f7ca00728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:59.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.303+0000 7f7c9ed9d640 1 --2- 172.21.15.114:0/3324645908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ca0101940 0x7f7ca0072de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:59.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.303+0000 7f7c9ed9d640 1 -- 172.21.15.114:0/3324645908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ca0100f80 msgr2=0x7f7ca00728a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:59.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.303+0000 7f7c9ed9d640 1 --2- 172.21.15.114:0/3324645908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ca0100f80 0x7f7ca00728a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:59.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.303+0000 7f7c9ed9d640 1 -- 172.21.15.114:0/3324645908 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ca00ffd90 msgr2=0x7f7ca0074260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:59.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.303+0000 7f7c9ed9d640 1 --2- 172.21.15.114:0/3324645908 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ca00ffd90 0x7f7ca0074260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:59.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.303+0000 7f7c9ed9d640 1 -- 172.21.15.114:0/3324645908 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c94009560 con 0x7f7ca0101940 2024-09-13T18:19:59.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.303+0000 7f7c9e59c640 1 --2- 172.21.15.114:0/3324645908 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ca00ffd90 0x7f7ca0074260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:19:59.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.303+0000 7f7c9ed9d640 1 --2- 172.21.15.114:0/3324645908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ca0101940 0x7f7ca0072de0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7c98009fd0 tx=0x7f7c9800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:59.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.303+0000 7f7c8f7fe640 1 -- 172.21.15.114:0/3324645908 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9800ce20 con 0x7f7ca0101940 2024-09-13T18:19:59.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.303+0000 7f7c8f7fe640 1 -- 172.21.15.114:0/3324645908 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7c98005020 con 0x7f7ca0101940 2024-09-13T18:19:59.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.303+0000 7f7ca4945640 1 -- 172.21.15.114:0/3324645908 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ca019ff80 con 0x7f7ca0101940 2024-09-13T18:19:59.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.303+0000 7f7c8f7fe640 1 -- 172.21.15.114:0/3324645908 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c98010690 con 0x7f7ca0101940 2024-09-13T18:19:59.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.303+0000 7f7ca4945640 1 -- 172.21.15.114:0/3324645908 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ca01a0400 con 0x7f7ca0101940 2024-09-13T18:19:59.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.304+0000 7f7ca4945640 1 -- 172.21.15.114:0/3324645908 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ca0107260 con 0x7f7ca0101940 2024-09-13T18:19:59.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.306+0000 7f7c8f7fe640 1 -- 172.21.15.114:0/3324645908 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f7c98005190 con 0x7f7ca0101940 2024-09-13T18:19:59.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.307+0000 7f7c8f7fe640 1 --2- 172.21.15.114:0/3324645908 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f7c880a1090 0x7f7c880a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:59.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.307+0000 7f7c9e59c640 1 --2- 172.21.15.114:0/3324645908 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f7c880a1090 0x7f7c880a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:59.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.307+0000 7f7c8f7fe640 1 -- 172.21.15.114:0/3324645908 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7c98014070 con 0x7f7ca0101940 2024-09-13T18:19:59.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.308+0000 7f7c9e59c640 1 --2- 172.21.15.114:0/3324645908 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f7c880a1090 0x7f7c880a3550 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7c900058c0 tx=0x7f7c90005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:59.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.309+0000 7f7c8f7fe640 1 -- 172.21.15.114:0/3324645908 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f7c980967b0 con 0x7f7ca0101940 2024-09-13T18:19:59.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.495+0000 7f7ca4945640 1 -- 172.21.15.114:0/3324645908 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f7ca010b500 con 0x7f7ca0101940 2024-09-13T18:19:59.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.495+0000 7f7c8f7fe640 1 -- 172.21.15.114:0/3324645908 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) v1 ==== 84+0+65 (secure 0 0 0) 0x7f7c98099dd0 con 0x7f7ca0101940 2024-09-13T18:19:59.498 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:19:59.498 INFO:teuthology.orchestra.run.smithi114.stdout:[ 2024-09-13T18:19:59.498 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:19:59.498 INFO:teuthology.orchestra.run.smithi114.stdout: "poolnum": 1, 2024-09-13T18:19:59.498 INFO:teuthology.orchestra.run.smithi114.stdout: "poolname": ".mgr" 2024-09-13T18:19:59.498 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:19:59.498 INFO:teuthology.orchestra.run.smithi114.stdout:] 2024-09-13T18:19:59.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.497+0000 7f7ca4945640 1 -- 172.21.15.114:0/3324645908 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f7c880a1090 msgr2=0x7f7c880a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:59.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.497+0000 7f7ca4945640 1 --2- 172.21.15.114:0/3324645908 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f7c880a1090 0x7f7c880a3550 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7c900058c0 tx=0x7f7c90005f90 comp rx=0 tx=0).stop 2024-09-13T18:19:59.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.497+0000 7f7ca4945640 1 -- 172.21.15.114:0/3324645908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ca0101940 msgr2=0x7f7ca0072de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:59.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.497+0000 7f7ca4945640 1 --2- 172.21.15.114:0/3324645908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ca0101940 0x7f7ca0072de0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7c98009fd0 tx=0x7f7c9800ef10 comp rx=0 tx=0).stop 2024-09-13T18:19:59.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.497+0000 7f7ca4945640 1 -- 172.21.15.114:0/3324645908 shutdown_connections 2024-09-13T18:19:59.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.497+0000 7f7ca4945640 1 --2- 172.21.15.114:0/3324645908 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f7c880a1090 0x7f7c880a3550 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:59.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.497+0000 7f7ca4945640 1 --2- 172.21.15.114:0/3324645908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ca0101940 0x7f7ca0072de0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:59.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.497+0000 7f7ca4945640 1 --2- 172.21.15.114:0/3324645908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ca0100f80 0x7f7ca00728a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:59.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.497+0000 7f7ca4945640 1 --2- 172.21.15.114:0/3324645908 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ca00ffd90 0x7f7ca0074260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:59.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.497+0000 7f7ca4945640 1 -- 172.21.15.114:0/3324645908 >> 172.21.15.114:0/3324645908 conn(0x7f7ca00fb510 msgr2=0x7f7ca0104430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:59.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.498+0000 7f7ca4945640 1 -- 172.21.15.114:0/3324645908 shutdown_connections 2024-09-13T18:19:59.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.498+0000 7f7ca4945640 1 -- 172.21.15.114:0/3324645908 wait complete. 2024-09-13T18:19:59.510 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-13T18:19:59.510 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-13T18:19:59.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.616+0000 7f5d235d5640 1 -- 172.21.15.114:0/2652018851 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d1c100840 msgr2=0x7f5d1c1070d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:59.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.616+0000 7f5d235d5640 1 --2- 172.21.15.114:0/2652018851 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d1c100840 0x7f5d1c1070d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c009900 tx=0x7f5d0c02a410 comp rx=0 tx=0).stop 2024-09-13T18:19:59.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.616+0000 7f5d235d5640 1 -- 172.21.15.114:0/2652018851 shutdown_connections 2024-09-13T18:19:59.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.616+0000 7f5d235d5640 1 --2- 172.21.15.114:0/2652018851 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d1c100840 0x7f5d1c1070d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:59.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.616+0000 7f5d235d5640 1 --2- 172.21.15.114:0/2652018851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d1c0ffe80 0x7f5d1c100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:59.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.616+0000 7f5d235d5640 1 --2- 172.21.15.114:0/2652018851 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d1c0fec90 0x7f5d1c0ff090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:59.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.616+0000 7f5d235d5640 1 -- 172.21.15.114:0/2652018851 >> 172.21.15.114:0/2652018851 conn(0x7f5d1c0fa3f0 msgr2=0x7f5d1c0fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:59.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.616+0000 7f5d235d5640 1 -- 172.21.15.114:0/2652018851 shutdown_connections 2024-09-13T18:19:59.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.616+0000 7f5d235d5640 1 -- 172.21.15.114:0/2652018851 wait complete. 2024-09-13T18:19:59.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.616+0000 7f5d235d5640 1 Processor -- start 2024-09-13T18:19:59.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.617+0000 7f5d235d5640 1 -- start start 2024-09-13T18:19:59.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.617+0000 7f5d235d5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d1c0fec90 0x7f5d1c102c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:59.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.617+0000 7f5d235d5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d1c0ffe80 0x7f5d1c101260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:59.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.617+0000 7f5d235d5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d1c100840 0x7f5d1c1017a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:59.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.617+0000 7f5d235d5640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d1c101f00 con 0x7f5d1c0fec90 2024-09-13T18:19:59.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.617+0000 7f5d235d5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d1c102070 con 0x7f5d1c100840 2024-09-13T18:19:59.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.617+0000 7f5d235d5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d1c19fcd0 con 0x7f5d1c0ffe80 2024-09-13T18:19:59.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.617+0000 7f5d21371640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d1c0fec90 0x7f5d1c102c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:59.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.617+0000 7f5d20b70640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d1c0ffe80 0x7f5d1c101260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:59.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.617+0000 7f5d21371640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d1c0fec90 0x7f5d1c102c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36240/0 (socket says 172.21.15.114:36240) 2024-09-13T18:19:59.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.617+0000 7f5d21371640 1 -- 172.21.15.114:0/2288140132 learned_addr learned my addr 172.21.15.114:0/2288140132 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:19:59.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.617+0000 7f5d21b72640 1 --2- 172.21.15.114:0/2288140132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d1c100840 0x7f5d1c1017a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:59.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.617+0000 7f5d21371640 1 -- 172.21.15.114:0/2288140132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d1c0ffe80 msgr2=0x7f5d1c101260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:59.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.617+0000 7f5d21371640 1 --2- 172.21.15.114:0/2288140132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d1c0ffe80 0x7f5d1c101260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:59.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.617+0000 7f5d21371640 1 -- 172.21.15.114:0/2288140132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d1c100840 msgr2=0x7f5d1c1017a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:59.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.617+0000 7f5d21371640 1 --2- 172.21.15.114:0/2288140132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d1c100840 0x7f5d1c1017a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:59.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.617+0000 7f5d21371640 1 -- 172.21.15.114:0/2288140132 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d0c009560 con 0x7f5d1c0fec90 2024-09-13T18:19:59.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.617+0000 7f5d21371640 1 --2- 172.21.15.114:0/2288140132 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d1c0fec90 0x7f5d1c102c20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5d1400cce0 tx=0x7f5d1400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:59.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.617+0000 7f5d1a7fc640 1 -- 172.21.15.114:0/2288140132 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d14004b20 con 0x7f5d1c0fec90 2024-09-13T18:19:59.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.617+0000 7f5d1a7fc640 1 -- 172.21.15.114:0/2288140132 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5d14003e20 con 0x7f5d1c0fec90 2024-09-13T18:19:59.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.617+0000 7f5d235d5640 1 -- 172.21.15.114:0/2288140132 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d1c19ff80 con 0x7f5d1c0fec90 2024-09-13T18:19:59.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.617+0000 7f5d1a7fc640 1 -- 172.21.15.114:0/2288140132 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d14004690 con 0x7f5d1c0fec90 2024-09-13T18:19:59.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.618+0000 7f5d235d5640 1 -- 172.21.15.114:0/2288140132 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d1c1a03a0 con 0x7f5d1c0fec90 2024-09-13T18:19:59.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.619+0000 7f5d235d5640 1 -- 172.21.15.114:0/2288140132 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d1c0ff090 con 0x7f5d1c0fec90 2024-09-13T18:19:59.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.619+0000 7f5d1a7fc640 1 -- 172.21.15.114:0/2288140132 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f5d14003990 con 0x7f5d1c0fec90 2024-09-13T18:19:59.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.620+0000 7f5d1a7fc640 1 --2- 172.21.15.114:0/2288140132 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f5d040a1110 0x7f5d040a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:59.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.620+0000 7f5d1a7fc640 1 -- 172.21.15.114:0/2288140132 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5d140cc460 con 0x7f5d1c0fec90 2024-09-13T18:19:59.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.620+0000 7f5d20b70640 1 --2- 172.21.15.114:0/2288140132 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f5d040a1110 0x7f5d040a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:59.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.621+0000 7f5d20b70640 1 --2- 172.21.15.114:0/2288140132 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f5d040a1110 0x7f5d040a35d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c102990 tx=0x7f5d10005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:59.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.624+0000 7f5d1a7fc640 1 -- 172.21.15.114:0/2288140132 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f5d14096140 con 0x7f5d1c0fec90 2024-09-13T18:19:59.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:19:59.788+0000 7f5d235d5640 1 -- 172.21.15.114:0/2288140132 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f5d1c10a400 con 0x7f5d1c0fec90 2024-09-13T18:20:00.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:00 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/2236579703' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T18:20:00.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:00 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:20:00.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:00 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:20:00.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:00 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:20:00.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:00 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:20:00.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:00 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/3324645908' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-13T18:20:00.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:00 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:20:00.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:00 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:20:00.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:00 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:20:00.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:00 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:20:00.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:00 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:20:00.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:00 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/2288140132' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-13T18:20:00.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:00 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/2236579703' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T18:20:00.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:00 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:20:00.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:00 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:20:00.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:00 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:20:00.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:00 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:20:00.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:00 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/3324645908' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-13T18:20:00.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:00 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:20:00.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:00 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:20:00.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:00 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:20:00.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:00 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:20:00.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:00 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:20:00.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:00 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/2288140132' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-13T18:20:00.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:00 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2236579703' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T18:20:00.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:00 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:20:00.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:00 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:20:00.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:00 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:20:00.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:00 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:20:00.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:00 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/3324645908' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-13T18:20:00.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:00 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:20:00.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:00 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:20:00.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:00 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:20:00.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:00 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:20:00.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:00 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:20:00.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:00 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2288140132' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-13T18:20:00.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:00.555+0000 7f5d1a7fc640 1 -- 172.21.15.114:0/2288140132 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) v1 ==== 138+0+0 (secure 0 0 0) 0x7f5d14099760 con 0x7f5d1c0fec90 2024-09-13T18:20:00.557 INFO:teuthology.orchestra.run.smithi114.stderr:pool 'cephfs_metadata' created 2024-09-13T18:20:00.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:00.559+0000 7f5d235d5640 1 -- 172.21.15.114:0/2288140132 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f5d040a1110 msgr2=0x7f5d040a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:00.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:00.559+0000 7f5d235d5640 1 --2- 172.21.15.114:0/2288140132 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f5d040a1110 0x7f5d040a35d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c102990 tx=0x7f5d10005f90 comp rx=0 tx=0).stop 2024-09-13T18:20:00.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:00.559+0000 7f5d235d5640 1 -- 172.21.15.114:0/2288140132 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d1c0fec90 msgr2=0x7f5d1c102c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:00.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:00.559+0000 7f5d235d5640 1 --2- 172.21.15.114:0/2288140132 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d1c0fec90 0x7f5d1c102c20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5d1400cce0 tx=0x7f5d1400a620 comp rx=0 tx=0).stop 2024-09-13T18:20:00.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:00.559+0000 7f5d235d5640 1 -- 172.21.15.114:0/2288140132 shutdown_connections 2024-09-13T18:20:00.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:00.559+0000 7f5d235d5640 1 --2- 172.21.15.114:0/2288140132 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f5d040a1110 0x7f5d040a35d0 secure :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c102990 tx=0x7f5d10005f90 comp rx=0 tx=0).stop 2024-09-13T18:20:00.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:00.559+0000 7f5d235d5640 1 --2- 172.21.15.114:0/2288140132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d1c100840 0x7f5d1c1017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:00.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:00.559+0000 7f5d235d5640 1 --2- 172.21.15.114:0/2288140132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d1c0ffe80 0x7f5d1c101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:00.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:00.559+0000 7f5d235d5640 1 --2- 172.21.15.114:0/2288140132 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d1c0fec90 0x7f5d1c102c20 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:00.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:00.559+0000 7f5d235d5640 1 -- 172.21.15.114:0/2288140132 >> 172.21.15.114:0/2288140132 conn(0x7f5d1c0fa3f0 msgr2=0x7f5d1c068600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:00.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:00.560+0000 7f5d235d5640 1 -- 172.21.15.114:0/2288140132 shutdown_connections 2024-09-13T18:20:00.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:00.560+0000 7f5d235d5640 1 -- 172.21.15.114:0/2288140132 wait complete. 2024-09-13T18:20:00.574 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-13T18:20:00.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:00.679+0000 7f018bc61640 1 -- 172.21.15.114:0/688032796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0184103900 msgr2=0x7f0184103db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:00.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:00.679+0000 7f018bc61640 1 --2- 172.21.15.114:0/688032796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0184103900 0x7f0184103db0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f017c009ca0 tx=0x7f017c02a8c0 comp rx=0 tx=0).stop 2024-09-13T18:20:00.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:00.679+0000 7f018bc61640 1 -- 172.21.15.114:0/688032796 shutdown_connections 2024-09-13T18:20:00.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:00.679+0000 7f018bc61640 1 --2- 172.21.15.114:0/688032796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0184103900 0x7f0184103db0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:00.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:00.679+0000 7f018bc61640 1 --2- 172.21.15.114:0/688032796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0184100f50 0x7f01841033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:00.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:00.679+0000 7f018bc61640 1 --2- 172.21.15.114:0/688032796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01840fe600 0x7f0184100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:00.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:00.679+0000 7f018bc61640 1 -- 172.21.15.114:0/688032796 >> 172.21.15.114:0/688032796 conn(0x7f01840fa3d0 msgr2=0x7f01840fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:00.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:00.679+0000 7f018bc61640 1 -- 172.21.15.114:0/688032796 shutdown_connections 2024-09-13T18:20:00.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:00.679+0000 7f018bc61640 1 -- 172.21.15.114:0/688032796 wait complete. 2024-09-13T18:20:00.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:00.680+0000 7f018bc61640 1 Processor -- start 2024-09-13T18:20:00.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:00.680+0000 7f018bc61640 1 -- start start 2024-09-13T18:20:00.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:00.680+0000 7f018bc61640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01840fe600 0x7f018419c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:00.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:00.680+0000 7f018bc61640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0184100f50 0x7f018419c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:00.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:00.680+0000 7f018bc61640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0184103900 0x7f01841a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:00.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:00.680+0000 7f018bc61640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01841a3d80 con 0x7f01840fe600 2024-09-13T18:20:00.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:00.680+0000 7f018bc61640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01841a3ef0 con 0x7f0184100f50 2024-09-13T18:20:00.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:00.680+0000 7f018bc61640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01841a40b0 con 0x7f0184103900 2024-09-13T18:20:00.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:00.680+0000 7f01899fd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01840fe600 0x7f018419c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:00.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:00.680+0000 7f01899fd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01840fe600 0x7f018419c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36246/0 (socket says 172.21.15.114:36246) 2024-09-13T18:20:00.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:00.680+0000 7f01899fd640 1 -- 172.21.15.114:0/1459130065 learned_addr learned my addr 172.21.15.114:0/1459130065 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:20:00.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:00.680+0000 7f01891fc640 1 --2- 172.21.15.114:0/1459130065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0184100f50 0x7f018419c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:00.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:00.680+0000 7f01899fd640 1 -- 172.21.15.114:0/1459130065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0184103900 msgr2=0x7f01841a3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:00.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:00.680+0000 7f01899fd640 1 --2- 172.21.15.114:0/1459130065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0184103900 0x7f01841a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:00.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:00.680+0000 7f01899fd640 1 -- 172.21.15.114:0/1459130065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0184100f50 msgr2=0x7f018419c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:00.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:00.680+0000 7f01899fd640 1 --2- 172.21.15.114:0/1459130065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0184100f50 0x7f018419c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:00.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:00.680+0000 7f01899fd640 1 -- 172.21.15.114:0/1459130065 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f017c009900 con 0x7f01840fe600 2024-09-13T18:20:00.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:00.680+0000 7f01891fc640 1 --2- 172.21.15.114:0/1459130065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0184100f50 0x7f018419c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:20:00.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:00.681+0000 7f01899fd640 1 --2- 172.21.15.114:0/1459130065 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01840fe600 0x7f018419c290 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f018000d950 tx=0x7f018000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:00.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:00.681+0000 7f017affd640 1 -- 172.21.15.114:0/1459130065 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0180009c30 con 0x7f01840fe600 2024-09-13T18:20:00.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:00.681+0000 7f017affd640 1 -- 172.21.15.114:0/1459130065 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0180009d90 con 0x7f01840fe600 2024-09-13T18:20:00.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:00.681+0000 7f018bc61640 1 -- 172.21.15.114:0/1459130065 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01841a4390 con 0x7f01840fe600 2024-09-13T18:20:00.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:00.681+0000 7f017affd640 1 -- 172.21.15.114:0/1459130065 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018000f620 con 0x7f01840fe600 2024-09-13T18:20:00.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:00.681+0000 7f018bc61640 1 -- 172.21.15.114:0/1459130065 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01841a48a0 con 0x7f01840fe600 2024-09-13T18:20:00.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:00.681+0000 7f018bc61640 1 -- 172.21.15.114:0/1459130065 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01840fbec0 con 0x7f01840fe600 2024-09-13T18:20:00.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:00.683+0000 7f017affd640 1 -- 172.21.15.114:0/1459130065 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f01800109f0 con 0x7f01840fe600 2024-09-13T18:20:00.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:00.683+0000 7f017affd640 1 --2- 172.21.15.114:0/1459130065 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f01700a1090 0x7f01700a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:00.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:00.684+0000 7f017affd640 1 -- 172.21.15.114:0/1459130065 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f01800cc260 con 0x7f01840fe600 2024-09-13T18:20:00.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:00.684+0000 7f01891fc640 1 --2- 172.21.15.114:0/1459130065 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f01700a1090 0x7f01700a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:00.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:00.684+0000 7f01891fc640 1 --2- 172.21.15.114:0/1459130065 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f01700a1090 0x7f01700a3550 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f018419d7a0 tx=0x7f0174008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:00.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:00.687+0000 7f017affd640 1 -- 172.21.15.114:0/1459130065 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f0180095d20 con 0x7f01840fe600 2024-09-13T18:20:00.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:00.851+0000 7f018bc61640 1 -- 172.21.15.114:0/1459130065 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f0184111c50 con 0x7f01840fe600 2024-09-13T18:20:01.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:01 smithi134 ceph-mon[34283]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:20:01.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:01 smithi134 ceph-mon[34283]: overall HEALTH_OK 2024-09-13T18:20:01.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:01 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/2288140132' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-13T18:20:01.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:01 smithi134 ceph-mon[34283]: osdmap e82: 12 total, 12 up, 12 in 2024-09-13T18:20:01.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:01 smithi204 ceph-mon[34777]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:20:01.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:01 smithi204 ceph-mon[34777]: overall HEALTH_OK 2024-09-13T18:20:01.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:01 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/2288140132' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-13T18:20:01.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:01 smithi204 ceph-mon[34777]: osdmap e82: 12 total, 12 up, 12 in 2024-09-13T18:20:01.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:01 smithi114 ceph-mon[30897]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:20:01.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:01 smithi114 ceph-mon[30897]: overall HEALTH_OK 2024-09-13T18:20:01.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:01 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2288140132' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-13T18:20:01.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:01 smithi114 ceph-mon[30897]: osdmap e82: 12 total, 12 up, 12 in 2024-09-13T18:20:01.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.563+0000 7f017affd640 1 -- 172.21.15.114:0/1459130065 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v83) v1 ==== 171+0+0 (secure 0 0 0) 0x7f0180099340 con 0x7f01840fe600 2024-09-13T18:20:01.566 INFO:teuthology.orchestra.run.smithi114.stderr:pool 'cephfs_data' created 2024-09-13T18:20:01.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.567+0000 7f018bc61640 1 -- 172.21.15.114:0/1459130065 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f01700a1090 msgr2=0x7f01700a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:01.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.567+0000 7f018bc61640 1 --2- 172.21.15.114:0/1459130065 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f01700a1090 0x7f01700a3550 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f018419d7a0 tx=0x7f0174008040 comp rx=0 tx=0).stop 2024-09-13T18:20:01.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.567+0000 7f018bc61640 1 -- 172.21.15.114:0/1459130065 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01840fe600 msgr2=0x7f018419c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:01.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.567+0000 7f018bc61640 1 --2- 172.21.15.114:0/1459130065 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01840fe600 0x7f018419c290 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f018000d950 tx=0x7f018000de10 comp rx=0 tx=0).stop 2024-09-13T18:20:01.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.569+0000 7f018bc61640 1 -- 172.21.15.114:0/1459130065 shutdown_connections 2024-09-13T18:20:01.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.569+0000 7f018bc61640 1 --2- 172.21.15.114:0/1459130065 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f01700a1090 0x7f01700a3550 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:01.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.569+0000 7f018bc61640 1 --2- 172.21.15.114:0/1459130065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0184103900 0x7f01841a3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:01.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.569+0000 7f018bc61640 1 --2- 172.21.15.114:0/1459130065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0184100f50 0x7f018419c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:01.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.569+0000 7f018bc61640 1 --2- 172.21.15.114:0/1459130065 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01840fe600 0x7f018419c290 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:01.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.569+0000 7f018bc61640 1 -- 172.21.15.114:0/1459130065 >> 172.21.15.114:0/1459130065 conn(0x7f01840fa3d0 msgr2=0x7f01841018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:01.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.569+0000 7f018bc61640 1 -- 172.21.15.114:0/1459130065 shutdown_connections 2024-09-13T18:20:01.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.569+0000 7f018bc61640 1 -- 172.21.15.114:0/1459130065 wait complete. 2024-09-13T18:20:01.589 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-13T18:20:01.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.690+0000 7f4c64ad7640 1 -- 172.21.15.114:0/241479440 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4c60101950 msgr2=0x7f4c601081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:01.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.690+0000 7f4c64ad7640 1 --2- 172.21.15.114:0/241479440 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4c60101950 0x7f4c601081e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4c58009ca0 tx=0x7f4c5802a8c0 comp rx=0 tx=0).stop 2024-09-13T18:20:01.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.690+0000 7f4c64ad7640 1 -- 172.21.15.114:0/241479440 shutdown_connections 2024-09-13T18:20:01.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.690+0000 7f4c64ad7640 1 --2- 172.21.15.114:0/241479440 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4c60101950 0x7f4c601081e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:01.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.690+0000 7f4c64ad7640 1 --2- 172.21.15.114:0/241479440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c60100f90 0x7f4c60101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:01.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.690+0000 7f4c64ad7640 1 --2- 172.21.15.114:0/241479440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c600ffda0 0x7f4c601001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:01.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.690+0000 7f4c64ad7640 1 -- 172.21.15.114:0/241479440 >> 172.21.15.114:0/241479440 conn(0x7f4c600fb520 msgr2=0x7f4c600fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:01.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.690+0000 7f4c64ad7640 1 -- 172.21.15.114:0/241479440 shutdown_connections 2024-09-13T18:20:01.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.690+0000 7f4c64ad7640 1 -- 172.21.15.114:0/241479440 wait complete. 2024-09-13T18:20:01.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.691+0000 7f4c64ad7640 1 Processor -- start 2024-09-13T18:20:01.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.691+0000 7f4c64ad7640 1 -- start start 2024-09-13T18:20:01.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.691+0000 7f4c64ad7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c600ffda0 0x7f4c60195c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:01.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.691+0000 7f4c64ad7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c60100f90 0x7f4c60196180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:01.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.691+0000 7f4c64ad7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4c60101950 0x7f4c6019d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:01.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.691+0000 7f4c64ad7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6019d720 con 0x7f4c60101950 2024-09-13T18:20:01.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.691+0000 7f4c64ad7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6019d890 con 0x7f4c600ffda0 2024-09-13T18:20:01.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.691+0000 7f4c64ad7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6019da50 con 0x7f4c60100f90 2024-09-13T18:20:01.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.691+0000 7f4c5ed9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4c60101950 0x7f4c6019d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:01.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.691+0000 7f4c5ed9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4c60101950 0x7f4c6019d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36254/0 (socket says 172.21.15.114:36254) 2024-09-13T18:20:01.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.691+0000 7f4c5ed9d640 1 -- 172.21.15.114:0/3893313335 learned_addr learned my addr 172.21.15.114:0/3893313335 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:20:01.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.691+0000 7f4c5e59c640 1 --2- 172.21.15.114:0/3893313335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c600ffda0 0x7f4c60195c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:01.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.691+0000 7f4c5dd9b640 1 --2- 172.21.15.114:0/3893313335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c60100f90 0x7f4c60196180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:01.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.691+0000 7f4c5ed9d640 1 -- 172.21.15.114:0/3893313335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c60100f90 msgr2=0x7f4c60196180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:01.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.691+0000 7f4c5ed9d640 1 --2- 172.21.15.114:0/3893313335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c60100f90 0x7f4c60196180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:01.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.691+0000 7f4c5ed9d640 1 -- 172.21.15.114:0/3893313335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c600ffda0 msgr2=0x7f4c60195c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:01.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.691+0000 7f4c5ed9d640 1 --2- 172.21.15.114:0/3893313335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c600ffda0 0x7f4c60195c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:01.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.691+0000 7f4c5ed9d640 1 -- 172.21.15.114:0/3893313335 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c58009900 con 0x7f4c60101950 2024-09-13T18:20:01.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.691+0000 7f4c5ed9d640 1 --2- 172.21.15.114:0/3893313335 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4c60101950 0x7f4c6019d1b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4c58003ba0 tx=0x7f4c58003bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:01.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.692+0000 7f4c577fe640 1 -- 172.21.15.114:0/3893313335 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c580317c0 con 0x7f4c60101950 2024-09-13T18:20:01.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.692+0000 7f4c577fe640 1 -- 172.21.15.114:0/3893313335 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4c58040070 con 0x7f4c60101950 2024-09-13T18:20:01.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.692+0000 7f4c577fe640 1 -- 172.21.15.114:0/3893313335 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c58004610 con 0x7f4c60101950 2024-09-13T18:20:01.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.692+0000 7f4c64ad7640 1 -- 172.21.15.114:0/3893313335 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c6019dcd0 con 0x7f4c60101950 2024-09-13T18:20:01.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.692+0000 7f4c64ad7640 1 -- 172.21.15.114:0/3893313335 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c6019e190 con 0x7f4c60101950 2024-09-13T18:20:01.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.693+0000 7f4c64ad7640 1 -- 172.21.15.114:0/3893313335 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c60107190 con 0x7f4c60101950 2024-09-13T18:20:01.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.694+0000 7f4c577fe640 1 -- 172.21.15.114:0/3893313335 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f4c58042030 con 0x7f4c60101950 2024-09-13T18:20:01.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.694+0000 7f4c577fe640 1 --2- 172.21.15.114:0/3893313335 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f4c4c0a1090 0x7f4c4c0a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:01.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.695+0000 7f4c5e59c640 1 --2- 172.21.15.114:0/3893313335 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f4c4c0a1090 0x7f4c4c0a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:01.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.695+0000 7f4c577fe640 1 -- 172.21.15.114:0/3893313335 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f4c580ee1f0 con 0x7f4c60101950 2024-09-13T18:20:01.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.695+0000 7f4c5e59c640 1 --2- 172.21.15.114:0/3893313335 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f4c4c0a1090 0x7f4c4c0a3550 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4c60197110 tx=0x7f4c48009110 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:01.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.698+0000 7f4c577fe640 1 -- 172.21.15.114:0/3893313335 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f4c580b7aa0 con 0x7f4c60101950 2024-09-13T18:20:01.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.893+0000 7f4c64ad7640 1 -- 172.21.15.114:0/3893313335 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f4c6002c480 con 0x7f4c60101950 2024-09-13T18:20:01.904 INFO:teuthology.orchestra.run.smithi114.stderr:new fs with metadata pool 2 and data pool 3 2024-09-13T18:20:01.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.902+0000 7f4c577fe640 1 -- 172.21.15.114:0/3893313335 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f4c580b99d0 con 0x7f4c60101950 2024-09-13T18:20:01.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.907+0000 7f4c64ad7640 1 -- 172.21.15.114:0/3893313335 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f4c4c0a1090 msgr2=0x7f4c4c0a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:01.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.907+0000 7f4c64ad7640 1 --2- 172.21.15.114:0/3893313335 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f4c4c0a1090 0x7f4c4c0a3550 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4c60197110 tx=0x7f4c48009110 comp rx=0 tx=0).stop 2024-09-13T18:20:01.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.907+0000 7f4c64ad7640 1 -- 172.21.15.114:0/3893313335 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4c60101950 msgr2=0x7f4c6019d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:01.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.907+0000 7f4c64ad7640 1 --2- 172.21.15.114:0/3893313335 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4c60101950 0x7f4c6019d1b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4c58003ba0 tx=0x7f4c58003bd0 comp rx=0 tx=0).stop 2024-09-13T18:20:01.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.908+0000 7f4c64ad7640 1 -- 172.21.15.114:0/3893313335 shutdown_connections 2024-09-13T18:20:01.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.908+0000 7f4c64ad7640 1 --2- 172.21.15.114:0/3893313335 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f4c4c0a1090 0x7f4c4c0a3550 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:01.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.908+0000 7f4c64ad7640 1 --2- 172.21.15.114:0/3893313335 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4c60101950 0x7f4c6019d1b0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:01.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.908+0000 7f4c64ad7640 1 --2- 172.21.15.114:0/3893313335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c60100f90 0x7f4c60196180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:01.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.908+0000 7f4c64ad7640 1 --2- 172.21.15.114:0/3893313335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c600ffda0 0x7f4c60195c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:01.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.908+0000 7f4c64ad7640 1 -- 172.21.15.114:0/3893313335 >> 172.21.15.114:0/3893313335 conn(0x7f4c600fb520 msgr2=0x7f4c60104360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:01.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.908+0000 7f4c64ad7640 1 -- 172.21.15.114:0/3893313335 shutdown_connections 2024-09-13T18:20:01.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:01.909+0000 7f4c64ad7640 1 -- 172.21.15.114:0/3893313335 wait complete. 2024-09-13T18:20:01.930 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-13T18:20:01.930 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-13T18:20:02.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:02.032+0000 7f76bbd9c640 1 -- 172.21.15.114:0/3065472763 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76b410c340 msgr2=0x7f76b410e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:02.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:02.032+0000 7f76bbd9c640 1 --2- 172.21.15.114:0/3065472763 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76b410c340 0x7f76b410e730 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f76ac009ca0 tx=0x7f76ac02a8c0 comp rx=0 tx=0).stop 2024-09-13T18:20:02.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:02.032+0000 7f76bbd9c640 1 -- 172.21.15.114:0/3065472763 shutdown_connections 2024-09-13T18:20:02.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:02.032+0000 7f76bbd9c640 1 --2- 172.21.15.114:0/3065472763 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76b410c340 0x7f76b410e730 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:02.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:02.032+0000 7f76bbd9c640 1 --2- 172.21.15.114:0/3065472763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76b4072f10 0x7f76b4073370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:02.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:02.032+0000 7f76bbd9c640 1 --2- 172.21.15.114:0/3065472763 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76b40744f0 0x7f76b4072940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:02.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:02.032+0000 7f76bbd9c640 1 -- 172.21.15.114:0/3065472763 >> 172.21.15.114:0/3065472763 conn(0x7f76b40fb4d0 msgr2=0x7f76b40fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:02.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:02.032+0000 7f76bbd9c640 1 -- 172.21.15.114:0/3065472763 shutdown_connections 2024-09-13T18:20:02.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:02.032+0000 7f76bbd9c640 1 -- 172.21.15.114:0/3065472763 wait complete. 2024-09-13T18:20:02.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:02.033+0000 7f76bbd9c640 1 Processor -- start 2024-09-13T18:20:02.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:02.033+0000 7f76bbd9c640 1 -- start start 2024-09-13T18:20:02.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:02.033+0000 7f76bbd9c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76b4072f10 0x7f76b419c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:02.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:02.033+0000 7f76bbd9c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76b40744f0 0x7f76b419c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:02.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:02.033+0000 7f76bbd9c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76b410c340 0x7f76b41a3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:02.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:02.033+0000 7f76bbd9c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b41a3d60 con 0x7f76b4072f10 2024-09-13T18:20:02.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:02.033+0000 7f76bbd9c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b41a3ed0 con 0x7f76b410c340 2024-09-13T18:20:02.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:02.033+0000 7f76bbd9c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b41a4090 con 0x7f76b40744f0 2024-09-13T18:20:02.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:02.033+0000 7f76b9b38640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76b4072f10 0x7f76b419c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:02.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:02.033+0000 7f76b9b38640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76b4072f10 0x7f76b419c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36264/0 (socket says 172.21.15.114:36264) 2024-09-13T18:20:02.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:02.033+0000 7f76b9b38640 1 -- 172.21.15.114:0/2241160003 learned_addr learned my addr 172.21.15.114:0/2241160003 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:20:02.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:02.033+0000 7f76b9b38640 1 -- 172.21.15.114:0/2241160003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76b40744f0 msgr2=0x7f76b419c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:02.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:02.033+0000 7f76b9b38640 1 --2- 172.21.15.114:0/2241160003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76b40744f0 0x7f76b419c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:02.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:02.033+0000 7f76b9b38640 1 -- 172.21.15.114:0/2241160003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76b410c340 msgr2=0x7f76b41a3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:02.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:02.033+0000 7f76b9b38640 1 --2- 172.21.15.114:0/2241160003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76b410c340 0x7f76b41a3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:02.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:02.033+0000 7f76b9b38640 1 -- 172.21.15.114:0/2241160003 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76ac009900 con 0x7f76b4072f10 2024-09-13T18:20:02.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:02.034+0000 7f76b9b38640 1 --2- 172.21.15.114:0/2241160003 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76b4072f10 0x7f76b419c270 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f76b000b4a0 tx=0x7f76b000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:02.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:02.034+0000 7f76aaffd640 1 -- 172.21.15.114:0/2241160003 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b0010610 con 0x7f76b4072f10 2024-09-13T18:20:02.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:02.034+0000 7f76aaffd640 1 -- 172.21.15.114:0/2241160003 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f76b0010c00 con 0x7f76b4072f10 2024-09-13T18:20:02.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:02.034+0000 7f76bbd9c640 1 -- 172.21.15.114:0/2241160003 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76b41a4370 con 0x7f76b4072f10 2024-09-13T18:20:02.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:02.034+0000 7f76aaffd640 1 -- 172.21.15.114:0/2241160003 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b0017ca0 con 0x7f76b4072f10 2024-09-13T18:20:02.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:02.034+0000 7f76bbd9c640 1 -- 172.21.15.114:0/2241160003 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76b41a48d0 con 0x7f76b4072f10 2024-09-13T18:20:02.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:02.035+0000 7f76bbd9c640 1 -- 172.21.15.114:0/2241160003 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76b4102510 con 0x7f76b4072f10 2024-09-13T18:20:02.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:02.036+0000 7f76aaffd640 1 -- 172.21.15.114:0/2241160003 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f76b0010770 con 0x7f76b4072f10 2024-09-13T18:20:02.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:02.036+0000 7f76aaffd640 1 --2- 172.21.15.114:0/2241160003 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f76a00a1110 0x7f76a00a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:02.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:02.037+0000 7f76b9337640 1 --2- 172.21.15.114:0/2241160003 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f76a00a1110 0x7f76a00a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:02.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:02.037+0000 7f76aaffd640 1 -- 172.21.15.114:0/2241160003 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f76b00cc1b0 con 0x7f76b4072f10 2024-09-13T18:20:02.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:02.037+0000 7f76b9337640 1 --2- 172.21.15.114:0/2241160003 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f76a00a1110 0x7f76a00a35d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f76b419d780 tx=0x7f76a4005bf0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:02.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:02.041+0000 7f76aaffd640 1 -- 172.21.15.114:0/2241160003 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f76b0095a90 con 0x7f76b4072f10 2024-09-13T18:20:02.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:02.212+0000 7f76bbd9c640 1 -- 172.21.15.114:0/2241160003 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f76b4111b30 con 0x7f76b4072f10 2024-09-13T18:20:02.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:02 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/1459130065' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-13T18:20:02.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:02 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:20:02.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:02 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/1459130065' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-13T18:20:02.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:02 smithi114 ceph-mon[30897]: osdmap e83: 12 total, 12 up, 12 in 2024-09-13T18:20:02.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:02 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/3893313335' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-13T18:20:02.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:02 smithi114 ceph-mon[30897]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-13T18:20:02.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:02 smithi114 ceph-mon[30897]: osdmap e84: 12 total, 12 up, 12 in 2024-09-13T18:20:02.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:02 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/3893313335' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-13T18:20:02.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:02 smithi114 ceph-mon[30897]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-13T18:20:02.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:02 smithi114 ceph-mon[30897]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-13T18:20:02.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:02 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/1459130065' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-13T18:20:02.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:02 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:20:02.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:02 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/1459130065' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-13T18:20:02.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:02 smithi134 ceph-mon[34283]: osdmap e83: 12 total, 12 up, 12 in 2024-09-13T18:20:02.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:02 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/3893313335' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-13T18:20:02.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:02 smithi134 ceph-mon[34283]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-13T18:20:02.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:02 smithi134 ceph-mon[34283]: osdmap e84: 12 total, 12 up, 12 in 2024-09-13T18:20:02.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:02 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/3893313335' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-13T18:20:02.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:02 smithi134 ceph-mon[34283]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-13T18:20:02.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:02 smithi134 ceph-mon[34283]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-13T18:20:02.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:02 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/1459130065' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-13T18:20:02.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:02 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:20:02.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:02 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/1459130065' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-13T18:20:02.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:02 smithi204 ceph-mon[34777]: osdmap e83: 12 total, 12 up, 12 in 2024-09-13T18:20:02.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:02 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/3893313335' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-13T18:20:02.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:02 smithi204 ceph-mon[34777]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-13T18:20:02.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:02 smithi204 ceph-mon[34777]: osdmap e84: 12 total, 12 up, 12 in 2024-09-13T18:20:02.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:02 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/3893313335' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-13T18:20:02.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:02 smithi204 ceph-mon[34777]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-13T18:20:02.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:02 smithi204 ceph-mon[34777]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-13T18:20:02.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:02.905+0000 7f76aaffd640 1 -- 172.21.15.114:0/2241160003 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v85) v1 ==== 157+0+0 (secure 0 0 0) 0x7f76b00990b0 con 0x7f76b4072f10 2024-09-13T18:20:02.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:02.907+0000 7f76bbd9c640 1 -- 172.21.15.114:0/2241160003 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f76a00a1110 msgr2=0x7f76a00a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:02.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:02.907+0000 7f76bbd9c640 1 --2- 172.21.15.114:0/2241160003 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f76a00a1110 0x7f76a00a35d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f76b419d780 tx=0x7f76a4005bf0 comp rx=0 tx=0).stop 2024-09-13T18:20:02.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:02.907+0000 7f76bbd9c640 1 -- 172.21.15.114:0/2241160003 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76b4072f10 msgr2=0x7f76b419c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:02.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:02.907+0000 7f76bbd9c640 1 --2- 172.21.15.114:0/2241160003 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76b4072f10 0x7f76b419c270 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f76b000b4a0 tx=0x7f76b000b960 comp rx=0 tx=0).stop 2024-09-13T18:20:02.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:02.907+0000 7f76bbd9c640 1 -- 172.21.15.114:0/2241160003 shutdown_connections 2024-09-13T18:20:02.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:02.907+0000 7f76bbd9c640 1 --2- 172.21.15.114:0/2241160003 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f76a00a1110 0x7f76a00a35d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:02.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:02.907+0000 7f76bbd9c640 1 --2- 172.21.15.114:0/2241160003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76b410c340 0x7f76b41a3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:02.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:02.907+0000 7f76bbd9c640 1 --2- 172.21.15.114:0/2241160003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76b40744f0 0x7f76b419c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:02.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:02.907+0000 7f76bbd9c640 1 --2- 172.21.15.114:0/2241160003 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76b4072f10 0x7f76b419c270 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:02.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:02.907+0000 7f76bbd9c640 1 -- 172.21.15.114:0/2241160003 >> 172.21.15.114:0/2241160003 conn(0x7f76b40fb4d0 msgr2=0x7f76b40ff6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:02.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:02.907+0000 7f76bbd9c640 1 -- 172.21.15.114:0/2241160003 shutdown_connections 2024-09-13T18:20:02.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:02.907+0000 7f76bbd9c640 1 -- 172.21.15.114:0/2241160003 wait complete. 2024-09-13T18:20:02.922 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-13T18:20:03.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:03.033+0000 7f43c5672640 1 -- 172.21.15.114:0/3791423770 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43c0100830 msgr2=0x7f43c01070c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:03.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:03.033+0000 7f43c5672640 1 --2- 172.21.15.114:0/3791423770 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43c0100830 0x7f43c01070c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f43b8009ca0 tx=0x7f43b802a8c0 comp rx=0 tx=0).stop 2024-09-13T18:20:03.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:03.033+0000 7f43c5672640 1 -- 172.21.15.114:0/3791423770 shutdown_connections 2024-09-13T18:20:03.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:03.033+0000 7f43c5672640 1 --2- 172.21.15.114:0/3791423770 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43c0100830 0x7f43c01070c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:03.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:03.033+0000 7f43c5672640 1 --2- 172.21.15.114:0/3791423770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43c00ffe70 0x7f43c01002f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:03.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:03.033+0000 7f43c5672640 1 --2- 172.21.15.114:0/3791423770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43c00fec80 0x7f43c00ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:03.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:03.033+0000 7f43c5672640 1 -- 172.21.15.114:0/3791423770 >> 172.21.15.114:0/3791423770 conn(0x7f43c00fa3e0 msgr2=0x7f43c00fc820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:03.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:03.033+0000 7f43c5672640 1 -- 172.21.15.114:0/3791423770 shutdown_connections 2024-09-13T18:20:03.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:03.033+0000 7f43c5672640 1 -- 172.21.15.114:0/3791423770 wait complete. 2024-09-13T18:20:03.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:03.033+0000 7f43c5672640 1 Processor -- start 2024-09-13T18:20:03.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:03.034+0000 7f43c5672640 1 -- start start 2024-09-13T18:20:03.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:03.034+0000 7f43c5672640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43c00fec80 0x7f43c0109540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:03.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:03.034+0000 7f43c5672640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43c00ffe70 0x7f43c0107b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:03.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:03.034+0000 7f43c5672640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43c0100830 0x7f43c01080c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:03.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:03.034+0000 7f43c5672640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c0108820 con 0x7f43c00fec80 2024-09-13T18:20:03.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:03.034+0000 7f43c5672640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c0108990 con 0x7f43c00ffe70 2024-09-13T18:20:03.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:03.034+0000 7f43c5672640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c019fc80 con 0x7f43c0100830 2024-09-13T18:20:03.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:03.034+0000 7f43beffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43c00fec80 0x7f43c0109540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:03.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:03.034+0000 7f43beffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43c00fec80 0x7f43c0109540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36284/0 (socket says 172.21.15.114:36284) 2024-09-13T18:20:03.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:03.034+0000 7f43beffd640 1 -- 172.21.15.114:0/90068886 learned_addr learned my addr 172.21.15.114:0/90068886 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:20:03.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:03.034+0000 7f43bf7fe640 1 --2- 172.21.15.114:0/90068886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43c0100830 0x7f43c01080c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:03.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:03.034+0000 7f43be7fc640 1 --2- 172.21.15.114:0/90068886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43c00ffe70 0x7f43c0107b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:03.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:03.034+0000 7f43beffd640 1 -- 172.21.15.114:0/90068886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43c0100830 msgr2=0x7f43c01080c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:03.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:03.034+0000 7f43beffd640 1 --2- 172.21.15.114:0/90068886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43c0100830 0x7f43c01080c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:03.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:03.034+0000 7f43beffd640 1 -- 172.21.15.114:0/90068886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43c00ffe70 msgr2=0x7f43c0107b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:03.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:03.034+0000 7f43beffd640 1 --2- 172.21.15.114:0/90068886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43c00ffe70 0x7f43c0107b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:03.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:03.034+0000 7f43beffd640 1 -- 172.21.15.114:0/90068886 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b8009900 con 0x7f43c00fec80 2024-09-13T18:20:03.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:03.034+0000 7f43be7fc640 1 --2- 172.21.15.114:0/90068886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43c00ffe70 0x7f43c0107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:20:03.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:03.034+0000 7f43beffd640 1 --2- 172.21.15.114:0/90068886 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43c00fec80 0x7f43c0109540 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f43b000c9a0 tx=0x7f43b000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:03.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:03.034+0000 7f43a7fff640 1 -- 172.21.15.114:0/90068886 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b0007c10 con 0x7f43c00fec80 2024-09-13T18:20:03.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:03.034+0000 7f43a7fff640 1 -- 172.21.15.114:0/90068886 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f43b0007d70 con 0x7f43c00fec80 2024-09-13T18:20:03.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:03.034+0000 7f43c5672640 1 -- 172.21.15.114:0/90068886 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43c019ff30 con 0x7f43c00fec80 2024-09-13T18:20:03.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:03.034+0000 7f43a7fff640 1 -- 172.21.15.114:0/90068886 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b000f7d0 con 0x7f43c00fec80 2024-09-13T18:20:03.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:03.034+0000 7f43c5672640 1 -- 172.21.15.114:0/90068886 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43c01a0350 con 0x7f43c00fec80 2024-09-13T18:20:03.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:03.035+0000 7f43c5672640 1 -- 172.21.15.114:0/90068886 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43c00ff0f0 con 0x7f43c00fec80 2024-09-13T18:20:03.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:03.036+0000 7f43a7fff640 1 -- 172.21.15.114:0/90068886 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f43b0018480 con 0x7f43c00fec80 2024-09-13T18:20:03.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:03.037+0000 7f43a7fff640 1 --2- 172.21.15.114:0/90068886 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f43a80a1090 0x7f43a80a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:03.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:03.037+0000 7f43be7fc640 1 --2- 172.21.15.114:0/90068886 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f43a80a1090 0x7f43a80a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:03.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:03.037+0000 7f43a7fff640 1 -- 172.21.15.114:0/90068886 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8205+0+0 (secure 0 0 0) 0x7f43b0013070 con 0x7f43c00fec80 2024-09-13T18:20:03.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:03.037+0000 7f43be7fc640 1 --2- 172.21.15.114:0/90068886 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f43a80a1090 0x7f43a80a3550 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f43b4006fd0 tx=0x7f43b4008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:03.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:03.040+0000 7f43a7fff640 1 -- 172.21.15.114:0/90068886 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f43b00966a0 con 0x7f43c00fec80 2024-09-13T18:20:03.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:03.204+0000 7f43c5672640 1 -- 172.21.15.114:0/90068886 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f43c010b650 con 0x7f43c00fec80 2024-09-13T18:20:03.219 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:03 smithi114 ceph-mon[30897]: pgmap v272: 97 pgs: 19 creating+peering, 4 active+clean, 74 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:20:03.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:03 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2241160003' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-13T18:20:03.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:03 smithi114 ceph-mon[30897]: mds.? [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] up:active 2024-09-13T18:20:03.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:03 smithi114 ceph-mon[30897]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-13T18:20:03.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:03 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2241160003' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-13T18:20:03.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:03 smithi114 ceph-mon[30897]: osdmap e85: 12 total, 12 up, 12 in 2024-09-13T18:20:03.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:03 smithi114 ceph-mon[30897]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-13T18:20:03.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:03 smithi134 ceph-mon[34283]: pgmap v272: 97 pgs: 19 creating+peering, 4 active+clean, 74 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:20:03.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:03 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/2241160003' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-13T18:20:03.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:03 smithi134 ceph-mon[34283]: mds.? [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] up:active 2024-09-13T18:20:03.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:03 smithi134 ceph-mon[34283]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-13T18:20:03.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:03 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/2241160003' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-13T18:20:03.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:03 smithi134 ceph-mon[34283]: osdmap e85: 12 total, 12 up, 12 in 2024-09-13T18:20:03.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:03 smithi134 ceph-mon[34283]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-13T18:20:03.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:03 smithi204 ceph-mon[34777]: pgmap v272: 97 pgs: 19 creating+peering, 4 active+clean, 74 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:20:03.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:03 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/2241160003' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-13T18:20:03.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:03 smithi204 ceph-mon[34777]: mds.? [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] up:active 2024-09-13T18:20:03.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:03 smithi204 ceph-mon[34777]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-13T18:20:03.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:03 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/2241160003' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-13T18:20:03.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:03 smithi204 ceph-mon[34777]: osdmap e85: 12 total, 12 up, 12 in 2024-09-13T18:20:03.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:03 smithi204 ceph-mon[34777]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-13T18:20:04.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:04 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/90068886' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-13T18:20:04.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:04 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/90068886' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-13T18:20:04.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:04 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/90068886' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-13T18:20:05.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.109+0000 7f43a7fff640 1 -- 172.21.15.114:0/90068886 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v87) v1 ==== 228+0+0 (secure 0 0 0) 0x7f43b0018730 con 0x7f43c00fec80 2024-09-13T18:20:05.111 INFO:teuthology.orchestra.run.smithi114.stderr:pool 'cephfs_data_ec' created 2024-09-13T18:20:05.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.113+0000 7f43c5672640 1 -- 172.21.15.114:0/90068886 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f43a80a1090 msgr2=0x7f43a80a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:05.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.113+0000 7f43c5672640 1 --2- 172.21.15.114:0/90068886 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f43a80a1090 0x7f43a80a3550 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f43b4006fd0 tx=0x7f43b4008040 comp rx=0 tx=0).stop 2024-09-13T18:20:05.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.113+0000 7f43c5672640 1 -- 172.21.15.114:0/90068886 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43c00fec80 msgr2=0x7f43c0109540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:05.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.113+0000 7f43c5672640 1 --2- 172.21.15.114:0/90068886 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43c00fec80 0x7f43c0109540 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f43b000c9a0 tx=0x7f43b000ce60 comp rx=0 tx=0).stop 2024-09-13T18:20:05.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.113+0000 7f43c5672640 1 -- 172.21.15.114:0/90068886 shutdown_connections 2024-09-13T18:20:05.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.113+0000 7f43c5672640 1 --2- 172.21.15.114:0/90068886 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f43a80a1090 0x7f43a80a3550 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:05.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.113+0000 7f43c5672640 1 --2- 172.21.15.114:0/90068886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43c0100830 0x7f43c01080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:05.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.113+0000 7f43c5672640 1 --2- 172.21.15.114:0/90068886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43c00ffe70 0x7f43c0107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:05.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.113+0000 7f43c5672640 1 --2- 172.21.15.114:0/90068886 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43c00fec80 0x7f43c0109540 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:05.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.113+0000 7f43c5672640 1 -- 172.21.15.114:0/90068886 >> 172.21.15.114:0/90068886 conn(0x7f43c00fa3e0 msgr2=0x7f43c0103230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:05.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.114+0000 7f43c5672640 1 -- 172.21.15.114:0/90068886 shutdown_connections 2024-09-13T18:20:05.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.114+0000 7f43c5672640 1 -- 172.21.15.114:0/90068886 wait complete. 2024-09-13T18:20:05.131 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-13T18:20:05.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.236+0000 7fb3ea419640 1 -- 172.21.15.114:0/4093939601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3e40ffe80 msgr2=0x7fb3e4100300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:05.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.236+0000 7fb3ea419640 1 --2- 172.21.15.114:0/4093939601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3e40ffe80 0x7fb3e4100300 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc009900 tx=0x7fb3cc02a410 comp rx=0 tx=0).stop 2024-09-13T18:20:05.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.236+0000 7fb3ea419640 1 -- 172.21.15.114:0/4093939601 shutdown_connections 2024-09-13T18:20:05.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.236+0000 7fb3ea419640 1 --2- 172.21.15.114:0/4093939601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3e4100840 0x7fb3e41070d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:05.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.236+0000 7fb3ea419640 1 --2- 172.21.15.114:0/4093939601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3e40ffe80 0x7fb3e4100300 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:05.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.236+0000 7fb3ea419640 1 --2- 172.21.15.114:0/4093939601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3e40fec90 0x7fb3e40ff090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:05.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.236+0000 7fb3ea419640 1 -- 172.21.15.114:0/4093939601 >> 172.21.15.114:0/4093939601 conn(0x7fb3e40fa3f0 msgr2=0x7fb3e40fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:05.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.236+0000 7fb3ea419640 1 -- 172.21.15.114:0/4093939601 shutdown_connections 2024-09-13T18:20:05.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.236+0000 7fb3ea419640 1 -- 172.21.15.114:0/4093939601 wait complete. 2024-09-13T18:20:05.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.236+0000 7fb3ea419640 1 Processor -- start 2024-09-13T18:20:05.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.236+0000 7fb3ea419640 1 -- start start 2024-09-13T18:20:05.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.237+0000 7fb3ea419640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3e40fec90 0x7fb3e40690e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:05.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.237+0000 7fb3ea419640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3e40ffe80 0x7fb3e419aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:05.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.237+0000 7fb3ea419640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3e4100840 0x7fb3e419d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:05.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.237+0000 7fb3ea419640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e419d810 con 0x7fb3e40fec90 2024-09-13T18:20:05.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.237+0000 7fb3ea419640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e419d980 con 0x7fb3e40ffe80 2024-09-13T18:20:05.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.237+0000 7fb3ea419640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e419db40 con 0x7fb3e4100840 2024-09-13T18:20:05.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.237+0000 7fb3e3fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3e40fec90 0x7fb3e40690e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:05.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.237+0000 7fb3e3fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3e40fec90 0x7fb3e40690e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44780/0 (socket says 172.21.15.114:44780) 2024-09-13T18:20:05.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.237+0000 7fb3e3fff640 1 -- 172.21.15.114:0/2037859411 learned_addr learned my addr 172.21.15.114:0/2037859411 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:20:05.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.237+0000 7fb3e89b6640 1 --2- 172.21.15.114:0/2037859411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3e4100840 0x7fb3e419d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:05.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.237+0000 7fb3e37fe640 1 --2- 172.21.15.114:0/2037859411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3e40ffe80 0x7fb3e419aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:05.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.237+0000 7fb3e3fff640 1 -- 172.21.15.114:0/2037859411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3e4100840 msgr2=0x7fb3e419d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:05.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.237+0000 7fb3e3fff640 1 --2- 172.21.15.114:0/2037859411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3e4100840 0x7fb3e419d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:05.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.237+0000 7fb3e3fff640 1 -- 172.21.15.114:0/2037859411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3e40ffe80 msgr2=0x7fb3e419aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:05.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.237+0000 7fb3e3fff640 1 --2- 172.21.15.114:0/2037859411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3e40ffe80 0x7fb3e419aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:05.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.237+0000 7fb3e3fff640 1 -- 172.21.15.114:0/2037859411 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3cc009560 con 0x7fb3e40fec90 2024-09-13T18:20:05.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.237+0000 7fb3e37fe640 1 --2- 172.21.15.114:0/2037859411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3e40ffe80 0x7fb3e419aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:20:05.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.237+0000 7fb3e89b6640 1 --2- 172.21.15.114:0/2037859411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3e4100840 0x7fb3e419d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:20:05.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.237+0000 7fb3e3fff640 1 --2- 172.21.15.114:0/2037859411 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3e40fec90 0x7fb3e40690e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb3d400d770 tx=0x7fb3d400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:05.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.237+0000 7fb3e17fa640 1 -- 172.21.15.114:0/2037859411 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d4009c30 con 0x7fb3e40fec90 2024-09-13T18:20:05.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.237+0000 7fb3e17fa640 1 -- 172.21.15.114:0/2037859411 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb3d400de80 con 0x7fb3e40fec90 2024-09-13T18:20:05.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.237+0000 7fb3ea419640 1 -- 172.21.15.114:0/2037859411 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3e419ddc0 con 0x7fb3e40fec90 2024-09-13T18:20:05.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.237+0000 7fb3e17fa640 1 -- 172.21.15.114:0/2037859411 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d4003d40 con 0x7fb3e40fec90 2024-09-13T18:20:05.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.237+0000 7fb3ea419640 1 -- 172.21.15.114:0/2037859411 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3e419e1e0 con 0x7fb3e40fec90 2024-09-13T18:20:05.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.239+0000 7fb3ea419640 1 -- 172.21.15.114:0/2037859411 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3e4106170 con 0x7fb3e40fec90 2024-09-13T18:20:05.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.240+0000 7fb3e17fa640 1 -- 172.21.15.114:0/2037859411 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fb3d4003ea0 con 0x7fb3e40fec90 2024-09-13T18:20:05.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.241+0000 7fb3e17fa640 1 --2- 172.21.15.114:0/2037859411 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb3d00a1090 0x7fb3d00a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:05.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.241+0000 7fb3e37fe640 1 --2- 172.21.15.114:0/2037859411 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb3d00a1090 0x7fb3d00a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:05.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.241+0000 7fb3e17fa640 1 -- 172.21.15.114:0/2037859411 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8698+0+0 (secure 0 0 0) 0x7fb3d40cc940 con 0x7fb3e40fec90 2024-09-13T18:20:05.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.241+0000 7fb3e37fe640 1 --2- 172.21.15.114:0/2037859411 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb3d00a1090 0x7fb3d00a3550 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc000c00 tx=0x7fb3cc0058b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:05.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.244+0000 7fb3e17fa640 1 -- 172.21.15.114:0/2037859411 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fb3d4095f00 con 0x7fb3e40fec90 2024-09-13T18:20:05.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:05 smithi134 ceph-mon[34283]: pgmap v274: 97 pgs: 19 creating+peering, 52 active+clean, 26 unknown; 578 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 2 op/s 2024-09-13T18:20:05.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:05 smithi134 ceph-mon[34283]: osdmap e86: 12 total, 12 up, 12 in 2024-09-13T18:20:05.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:05.408+0000 7fb3ea419640 1 -- 172.21.15.114:0/2037859411 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7fb3e410a460 con 0x7fb3e40fec90 2024-09-13T18:20:05.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:05 smithi204 ceph-mon[34777]: pgmap v274: 97 pgs: 19 creating+peering, 52 active+clean, 26 unknown; 578 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 2 op/s 2024-09-13T18:20:05.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:05 smithi204 ceph-mon[34777]: osdmap e86: 12 total, 12 up, 12 in 2024-09-13T18:20:05.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:05 smithi114 ceph-mon[30897]: pgmap v274: 97 pgs: 19 creating+peering, 52 active+clean, 26 unknown; 578 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 2 op/s 2024-09-13T18:20:05.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:05 smithi114 ceph-mon[30897]: osdmap e86: 12 total, 12 up, 12 in 2024-09-13T18:20:06.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.118+0000 7fb3e17fa640 1 -- 172.21.15.114:0/2037859411 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v88) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb3d40170c0 con 0x7fb3e40fec90 2024-09-13T18:20:06.121 INFO:teuthology.orchestra.run.smithi114.stderr:set pool 4 allow_ec_overwrites to true 2024-09-13T18:20:06.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.122+0000 7fb3ea419640 1 -- 172.21.15.114:0/2037859411 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb3d00a1090 msgr2=0x7fb3d00a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:06.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.122+0000 7fb3ea419640 1 --2- 172.21.15.114:0/2037859411 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb3d00a1090 0x7fb3d00a3550 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc000c00 tx=0x7fb3cc0058b0 comp rx=0 tx=0).stop 2024-09-13T18:20:06.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.122+0000 7fb3ea419640 1 -- 172.21.15.114:0/2037859411 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3e40fec90 msgr2=0x7fb3e40690e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:06.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.122+0000 7fb3ea419640 1 --2- 172.21.15.114:0/2037859411 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3e40fec90 0x7fb3e40690e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb3d400d770 tx=0x7fb3d400dc30 comp rx=0 tx=0).stop 2024-09-13T18:20:06.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.124+0000 7fb3ea419640 1 -- 172.21.15.114:0/2037859411 shutdown_connections 2024-09-13T18:20:06.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.124+0000 7fb3ea419640 1 --2- 172.21.15.114:0/2037859411 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb3d00a1090 0x7fb3d00a3550 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:06.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.124+0000 7fb3ea419640 1 --2- 172.21.15.114:0/2037859411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3e4100840 0x7fb3e419d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:06.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.124+0000 7fb3ea419640 1 --2- 172.21.15.114:0/2037859411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3e40ffe80 0x7fb3e419aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:06.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.124+0000 7fb3ea419640 1 --2- 172.21.15.114:0/2037859411 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3e40fec90 0x7fb3e40690e0 secure :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb3d400d770 tx=0x7fb3d400dc30 comp rx=0 tx=0).stop 2024-09-13T18:20:06.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.124+0000 7fb3ea419640 1 -- 172.21.15.114:0/2037859411 >> 172.21.15.114:0/2037859411 conn(0x7fb3e40fa3f0 msgr2=0x7fb3e4103340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:06.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.125+0000 7fb3ea419640 1 -- 172.21.15.114:0/2037859411 shutdown_connections 2024-09-13T18:20:06.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.125+0000 7fb3ea419640 1 -- 172.21.15.114:0/2037859411 wait complete. 2024-09-13T18:20:06.146 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-13T18:20:06.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.246+0000 7fc469244640 1 -- 172.21.15.114:0/392170148 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4640fec90 msgr2=0x7fc4640ff090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:06.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.246+0000 7fc469244640 1 --2- 172.21.15.114:0/392170148 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4640fec90 0x7fc4640ff090 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc454009ca0 tx=0x7fc45402a8c0 comp rx=0 tx=0).stop 2024-09-13T18:20:06.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.246+0000 7fc469244640 1 -- 172.21.15.114:0/392170148 shutdown_connections 2024-09-13T18:20:06.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.246+0000 7fc469244640 1 --2- 172.21.15.114:0/392170148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc464100840 0x7fc4641070d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:06.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.246+0000 7fc469244640 1 --2- 172.21.15.114:0/392170148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4640ffe80 0x7fc464100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:06.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.246+0000 7fc469244640 1 --2- 172.21.15.114:0/392170148 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4640fec90 0x7fc4640ff090 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:06.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.246+0000 7fc469244640 1 -- 172.21.15.114:0/392170148 >> 172.21.15.114:0/392170148 conn(0x7fc4640fa3f0 msgr2=0x7fc4640fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:06.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.246+0000 7fc469244640 1 -- 172.21.15.114:0/392170148 shutdown_connections 2024-09-13T18:20:06.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.246+0000 7fc469244640 1 -- 172.21.15.114:0/392170148 wait complete. 2024-09-13T18:20:06.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.246+0000 7fc469244640 1 Processor -- start 2024-09-13T18:20:06.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.247+0000 7fc469244640 1 -- start start 2024-09-13T18:20:06.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.247+0000 7fc469244640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4640fec90 0x7fc4640690e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:06.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.247+0000 7fc469244640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4640ffe80 0x7fc46419cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:06.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.247+0000 7fc469244640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc464100840 0x7fc46419f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:06.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.247+0000 7fc469244640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc46419f980 con 0x7fc4640ffe80 2024-09-13T18:20:06.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.247+0000 7fc469244640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc46419faf0 con 0x7fc464100840 2024-09-13T18:20:06.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.247+0000 7fc469244640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc46419fcb0 con 0x7fc4640fec90 2024-09-13T18:20:06.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.247+0000 7fc46259c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4640ffe80 0x7fc46419cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:06.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.247+0000 7fc46259c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4640ffe80 0x7fc46419cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44806/0 (socket says 172.21.15.114:44806) 2024-09-13T18:20:06.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.247+0000 7fc46259c640 1 -- 172.21.15.114:0/2185728546 learned_addr learned my addr 172.21.15.114:0/2185728546 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:20:06.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.247+0000 7fc46359e640 1 --2- 172.21.15.114:0/2185728546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc464100840 0x7fc46419f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:06.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.247+0000 7fc462d9d640 1 --2- 172.21.15.114:0/2185728546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4640fec90 0x7fc4640690e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:06.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.247+0000 7fc46259c640 1 -- 172.21.15.114:0/2185728546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4640fec90 msgr2=0x7fc4640690e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:06.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.247+0000 7fc46259c640 1 --2- 172.21.15.114:0/2185728546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4640fec90 0x7fc4640690e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:06.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.247+0000 7fc46259c640 1 -- 172.21.15.114:0/2185728546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc464100840 msgr2=0x7fc46419f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:06.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.247+0000 7fc46259c640 1 --2- 172.21.15.114:0/2185728546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc464100840 0x7fc46419f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:06.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.247+0000 7fc46259c640 1 -- 172.21.15.114:0/2185728546 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc454009900 con 0x7fc4640ffe80 2024-09-13T18:20:06.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.247+0000 7fc46359e640 1 --2- 172.21.15.114:0/2185728546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc464100840 0x7fc46419f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:20:06.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.247+0000 7fc462d9d640 1 --2- 172.21.15.114:0/2185728546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4640fec90 0x7fc4640690e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:20:06.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.247+0000 7fc46259c640 1 --2- 172.21.15.114:0/2185728546 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4640ffe80 0x7fc46419cd70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc44c00c920 tx=0x7fc44c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:06.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.247+0000 7fc45bfff640 1 -- 172.21.15.114:0/2185728546 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44c007c70 con 0x7fc4640ffe80 2024-09-13T18:20:06.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.248+0000 7fc45bfff640 1 -- 172.21.15.114:0/2185728546 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc44c007dd0 con 0x7fc4640ffe80 2024-09-13T18:20:06.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.248+0000 7fc469244640 1 -- 172.21.15.114:0/2185728546 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc46419ff30 con 0x7fc4640ffe80 2024-09-13T18:20:06.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.248+0000 7fc45bfff640 1 -- 172.21.15.114:0/2185728546 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44c018730 con 0x7fc4640ffe80 2024-09-13T18:20:06.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.248+0000 7fc469244640 1 -- 172.21.15.114:0/2185728546 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4641a03b0 con 0x7fc4640ffe80 2024-09-13T18:20:06.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.249+0000 7fc469244640 1 -- 172.21.15.114:0/2185728546 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4641061c0 con 0x7fc4640ffe80 2024-09-13T18:20:06.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.250+0000 7fc45bfff640 1 -- 172.21.15.114:0/2185728546 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fc44c00f450 con 0x7fc4640ffe80 2024-09-13T18:20:06.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.250+0000 7fc45bfff640 1 --2- 172.21.15.114:0/2185728546 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fc4500a10c0 0x7fc4500a3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:06.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.250+0000 7fc462d9d640 1 --2- 172.21.15.114:0/2185728546 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fc4500a10c0 0x7fc4500a3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:06.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.250+0000 7fc45bfff640 1 -- 172.21.15.114:0/2185728546 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8698+0+0 (secure 0 0 0) 0x7fc44c0ccc40 con 0x7fc4640ffe80 2024-09-13T18:20:06.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.251+0000 7fc462d9d640 1 --2- 172.21.15.114:0/2185728546 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fc4500a10c0 0x7fc4500a3580 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc454031040 tx=0x7fc45402f310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:06.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.254+0000 7fc45bfff640 1 -- 172.21.15.114:0/2185728546 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fc44c096200 con 0x7fc4640ffe80 2024-09-13T18:20:06.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:06 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/90068886' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-13T18:20:06.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:06 smithi134 ceph-mon[34283]: osdmap e87: 12 total, 12 up, 12 in 2024-09-13T18:20:06.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:06 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/2037859411' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-13T18:20:06.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:06 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/90068886' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-13T18:20:06.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:06 smithi204 ceph-mon[34777]: osdmap e87: 12 total, 12 up, 12 in 2024-09-13T18:20:06.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:06 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/2037859411' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-13T18:20:06.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.454+0000 7fc469244640 1 -- 172.21.15.114:0/2185728546 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7fc464105c30 con 0x7fc4640ffe80 2024-09-13T18:20:06.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.459+0000 7fc45bfff640 1 -- 172.21.15.114:0/2185728546 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) v1 ==== 137+0+0 (secure 0 0 0) 0x7fc44c099820 con 0x7fc4640ffe80 2024-09-13T18:20:06.462 INFO:teuthology.orchestra.run.smithi114.stderr:added data pool 4 to fsmap 2024-09-13T18:20:06.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.461+0000 7fc469244640 1 -- 172.21.15.114:0/2185728546 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fc4500a10c0 msgr2=0x7fc4500a3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:06.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.461+0000 7fc469244640 1 --2- 172.21.15.114:0/2185728546 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fc4500a10c0 0x7fc4500a3580 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc454031040 tx=0x7fc45402f310 comp rx=0 tx=0).stop 2024-09-13T18:20:06.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.461+0000 7fc469244640 1 -- 172.21.15.114:0/2185728546 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4640ffe80 msgr2=0x7fc46419cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:06.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.461+0000 7fc469244640 1 --2- 172.21.15.114:0/2185728546 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4640ffe80 0x7fc46419cd70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc44c00c920 tx=0x7fc44c00cde0 comp rx=0 tx=0).stop 2024-09-13T18:20:06.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.462+0000 7fc469244640 1 -- 172.21.15.114:0/2185728546 shutdown_connections 2024-09-13T18:20:06.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.462+0000 7fc469244640 1 --2- 172.21.15.114:0/2185728546 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fc4500a10c0 0x7fc4500a3580 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:06.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.462+0000 7fc469244640 1 --2- 172.21.15.114:0/2185728546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc464100840 0x7fc46419f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:06.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.462+0000 7fc469244640 1 --2- 172.21.15.114:0/2185728546 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4640ffe80 0x7fc46419cd70 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:06.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.462+0000 7fc469244640 1 --2- 172.21.15.114:0/2185728546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4640fec90 0x7fc4640690e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:06.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.462+0000 7fc469244640 1 -- 172.21.15.114:0/2185728546 >> 172.21.15.114:0/2185728546 conn(0x7fc4640fa3f0 msgr2=0x7fc464103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:06.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.462+0000 7fc469244640 1 -- 172.21.15.114:0/2185728546 shutdown_connections 2024-09-13T18:20:06.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.462+0000 7fc469244640 1 -- 172.21.15.114:0/2185728546 wait complete. 2024-09-13T18:20:06.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:06 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/90068886' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-13T18:20:06.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:06 smithi114 ceph-mon[30897]: osdmap e87: 12 total, 12 up, 12 in 2024-09-13T18:20:06.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:06 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2037859411' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-13T18:20:06.477 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:20:06.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.573+0000 7f4d7a345640 1 -- 172.21.15.114:0/2028180140 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d74105ae0 msgr2=0x7f4d74107ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:06.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.573+0000 7f4d7a345640 1 --2- 172.21.15.114:0/2028180140 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d74105ae0 0x7f4d74107ef0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c009ca0 tx=0x7f4d6c02a8c0 comp rx=0 tx=0).stop 2024-09-13T18:20:06.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.573+0000 7f4d7a345640 1 -- 172.21.15.114:0/2028180140 shutdown_connections 2024-09-13T18:20:06.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.573+0000 7f4d7a345640 1 --2- 172.21.15.114:0/2028180140 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d74105ae0 0x7f4d74107ef0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:06.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.573+0000 7f4d7a345640 1 --2- 172.21.15.114:0/2028180140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d741031b0 0x7f4d741055a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:06.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.573+0000 7f4d7a345640 1 --2- 172.21.15.114:0/2028180140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d74100880 0x7f4d74102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:06.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.573+0000 7f4d7a345640 1 -- 172.21.15.114:0/2028180140 >> 172.21.15.114:0/2028180140 conn(0x7f4d740fa440 msgr2=0x7f4d740fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:06.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.573+0000 7f4d7a345640 1 -- 172.21.15.114:0/2028180140 shutdown_connections 2024-09-13T18:20:06.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.573+0000 7f4d7a345640 1 -- 172.21.15.114:0/2028180140 wait complete. 2024-09-13T18:20:06.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.573+0000 7f4d7a345640 1 Processor -- start 2024-09-13T18:20:06.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.573+0000 7f4d7a345640 1 -- start start 2024-09-13T18:20:06.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.573+0000 7f4d7a345640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d74100880 0x7f4d741001a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:06.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.573+0000 7f4d7a345640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d741031b0 0x7f4d740fe830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:06.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.573+0000 7f4d7a345640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d74105ae0 0x7f4d740fed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:06.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.573+0000 7f4d7a345640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d740ff500 con 0x7f4d74105ae0 2024-09-13T18:20:06.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.573+0000 7f4d7a345640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d740ff670 con 0x7f4d741031b0 2024-09-13T18:20:06.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.573+0000 7f4d7a345640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7419fda0 con 0x7f4d74100880 2024-09-13T18:20:06.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.574+0000 7f4d788e2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d74105ae0 0x7f4d740fed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:06.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.574+0000 7f4d788e2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d74105ae0 0x7f4d740fed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44826/0 (socket says 172.21.15.114:44826) 2024-09-13T18:20:06.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.574+0000 7f4d788e2640 1 -- 172.21.15.114:0/3480288595 learned_addr learned my addr 172.21.15.114:0/3480288595 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:20:06.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.574+0000 7f4d73fff640 1 --2- 172.21.15.114:0/3480288595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d74100880 0x7f4d741001a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:06.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.574+0000 7f4d788e2640 1 -- 172.21.15.114:0/3480288595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d74100880 msgr2=0x7f4d741001a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:06.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.574+0000 7f4d788e2640 1 --2- 172.21.15.114:0/3480288595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d74100880 0x7f4d741001a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:06.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.574+0000 7f4d788e2640 1 -- 172.21.15.114:0/3480288595 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d741031b0 msgr2=0x7f4d740fe830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:06.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.574+0000 7f4d788e2640 1 --2- 172.21.15.114:0/3480288595 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d741031b0 0x7f4d740fe830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:06.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.574+0000 7f4d788e2640 1 -- 172.21.15.114:0/3480288595 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d6c009900 con 0x7f4d74105ae0 2024-09-13T18:20:06.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.574+0000 7f4d73fff640 1 --2- 172.21.15.114:0/3480288595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d74100880 0x7f4d741001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:20:06.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.574+0000 7f4d788e2640 1 --2- 172.21.15.114:0/3480288595 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d74105ae0 0x7f4d740fed70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c02ade0 tx=0x7f4d6c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:06.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.574+0000 7f4d71ffb640 1 -- 172.21.15.114:0/3480288595 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c0307c0 con 0x7f4d74105ae0 2024-09-13T18:20:06.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.574+0000 7f4d71ffb640 1 -- 172.21.15.114:0/3480288595 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4d6c040070 con 0x7f4d74105ae0 2024-09-13T18:20:06.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.574+0000 7f4d71ffb640 1 -- 172.21.15.114:0/3480288595 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c0044e0 con 0x7f4d74105ae0 2024-09-13T18:20:06.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.574+0000 7f4d7a345640 1 -- 172.21.15.114:0/3480288595 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d7419ffa0 con 0x7f4d74105ae0 2024-09-13T18:20:06.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.574+0000 7f4d7a345640 1 -- 172.21.15.114:0/3480288595 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d741a0480 con 0x7f4d74105ae0 2024-09-13T18:20:06.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.575+0000 7f4d7a345640 1 -- 172.21.15.114:0/3480288595 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d740fbf30 con 0x7f4d74105ae0 2024-09-13T18:20:06.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.576+0000 7f4d71ffb640 1 -- 172.21.15.114:0/3480288595 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f4d6c042030 con 0x7f4d74105ae0 2024-09-13T18:20:06.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.577+0000 7f4d71ffb640 1 --2- 172.21.15.114:0/3480288595 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f4d600a1110 0x7f4d600a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:06.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.577+0000 7f4d73fff640 1 --2- 172.21.15.114:0/3480288595 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f4d600a1110 0x7f4d600a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:06.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.577+0000 7f4d71ffb640 1 -- 172.21.15.114:0/3480288595 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f4d6c0ee1c0 con 0x7f4d74105ae0 2024-09-13T18:20:06.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.577+0000 7f4d73fff640 1 --2- 172.21.15.114:0/3480288595 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f4d600a1110 0x7f4d600a35d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4d64005d00 tx=0x7f4d64005c70 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:06.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.580+0000 7f4d71ffb640 1 -- 172.21.15.114:0/3480288595 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f4d6c0b7800 con 0x7f4d74105ae0 2024-09-13T18:20:06.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.775+0000 7f4d7a345640 1 -- 172.21.15.114:0/3480288595 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d7410b550 con 0x7f4d74105ae0 2024-09-13T18:20:06.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.776+0000 7f4d71ffb640 1 -- 172.21.15.114:0/3480288595 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+10819 (secure 0 0 0) 0x7f4d6c0bae20 con 0x7f4d74105ae0 2024-09-13T18:20:06.778 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:20:06.779 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4162817497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4162817497},{"type":"v1","addr":"172.21.15.114:6835","nonce":4162817497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/2435428478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":2435428478},{"type":"v1","addr":"172.21.15.114:6837","nonce":2435428478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6839/2046516451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":2046516451},{"type":"v1","addr":"172.21.15.114:6839","nonce":2046516451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6841/1956923446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":1956923446},{"type":"v1","addr":"172.21.15.114:6841","nonce":1956923446}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/507301496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":507301496},{"type":"v1","addr":"172.21.15.204:6833","nonce":507301496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/4028657219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4028657219},{"type":"v1","addr":"172.21.15.134:6833","nonce":4028657219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/57029119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":57029119},{"type":"v1","addr":"172.21.15.204:6835","nonce":57029119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3068880939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3068880939},{"type":"v1","addr":"172.21.15.204:6837","nonce":3068880939}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1150003275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1150003275},{"type":"v1","addr":"172.21.15.134:6835","nonce":1150003275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24475,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/344883912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":344883912},{"type":"v1","addr":"172.21.15.204:6839","nonce":344883912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/891692210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":891692210},{"type":"v1","addr":"172.21.15.134:6837","nonce":891692210}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:20:01.895117+0000","modified":"2024-09-13T18:20:06.455916+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6839/3466309724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3466309724},{"type":"v1","addr":"172.21.15.134:6839","nonce":3466309724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-09-13T18:20:06.779 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 17 2024-09-13T18:20:06.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.778+0000 7f4d7a345640 1 -- 172.21.15.114:0/3480288595 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f4d600a1110 msgr2=0x7f4d600a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:06.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.778+0000 7f4d7a345640 1 --2- 172.21.15.114:0/3480288595 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f4d600a1110 0x7f4d600a35d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4d64005d00 tx=0x7f4d64005c70 comp rx=0 tx=0).stop 2024-09-13T18:20:06.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.778+0000 7f4d7a345640 1 -- 172.21.15.114:0/3480288595 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d74105ae0 msgr2=0x7f4d740fed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:06.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.778+0000 7f4d7a345640 1 --2- 172.21.15.114:0/3480288595 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d74105ae0 0x7f4d740fed70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c02ade0 tx=0x7f4d6c003c20 comp rx=0 tx=0).stop 2024-09-13T18:20:06.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.778+0000 7f4d7a345640 1 -- 172.21.15.114:0/3480288595 shutdown_connections 2024-09-13T18:20:06.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.778+0000 7f4d7a345640 1 --2- 172.21.15.114:0/3480288595 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f4d600a1110 0x7f4d600a35d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:06.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.778+0000 7f4d7a345640 1 --2- 172.21.15.114:0/3480288595 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d74105ae0 0x7f4d740fed70 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:06.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.778+0000 7f4d7a345640 1 --2- 172.21.15.114:0/3480288595 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d741031b0 0x7f4d740fe830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:06.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.778+0000 7f4d7a345640 1 --2- 172.21.15.114:0/3480288595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d74100880 0x7f4d741001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:06.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.778+0000 7f4d7a345640 1 -- 172.21.15.114:0/3480288595 >> 172.21.15.114:0/3480288595 conn(0x7f4d740fa440 msgr2=0x7f4d74101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:06.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.778+0000 7f4d7a345640 1 -- 172.21.15.114:0/3480288595 shutdown_connections 2024-09-13T18:20:06.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.778+0000 7f4d7a345640 1 -- 172.21.15.114:0/3480288595 wait complete. 2024-09-13T18:20:06.792 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:20:06.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.888+0000 7f7ff4f5e640 1 -- 172.21.15.114:0/2272746174 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ff00fde10 msgr2=0x7f7ff00fe290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:06.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.888+0000 7f7ff4f5e640 1 --2- 172.21.15.114:0/2272746174 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ff00fde10 0x7f7ff00fe290 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0009900 tx=0x7f7fe002a410 comp rx=0 tx=0).stop 2024-09-13T18:20:06.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.888+0000 7f7ff4f5e640 1 -- 172.21.15.114:0/2272746174 shutdown_connections 2024-09-13T18:20:06.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.888+0000 7f7ff4f5e640 1 --2- 172.21.15.114:0/2272746174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ff00fe7d0 0x7f7ff0108380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:06.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.888+0000 7f7ff4f5e640 1 --2- 172.21.15.114:0/2272746174 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ff00fde10 0x7f7ff00fe290 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:06.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.888+0000 7f7ff4f5e640 1 --2- 172.21.15.114:0/2272746174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ff01013a0 0x7f7ff01017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:06.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.888+0000 7f7ff4f5e640 1 -- 172.21.15.114:0/2272746174 >> 172.21.15.114:0/2272746174 conn(0x7f7ff00f9bc0 msgr2=0x7f7ff00fbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:06.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.888+0000 7f7ff4f5e640 1 -- 172.21.15.114:0/2272746174 shutdown_connections 2024-09-13T18:20:06.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.888+0000 7f7ff4f5e640 1 -- 172.21.15.114:0/2272746174 wait complete. 2024-09-13T18:20:06.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.889+0000 7f7ff4f5e640 1 Processor -- start 2024-09-13T18:20:06.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.889+0000 7f7ff4f5e640 1 -- start start 2024-09-13T18:20:06.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.889+0000 7f7ff4f5e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ff00fde10 0x7f7ff0195ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:06.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.889+0000 7f7ff4f5e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ff00fe7d0 0x7f7ff01961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:06.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.889+0000 7f7ff4f5e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ff01013a0 0x7f7ff019d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:06.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.889+0000 7f7ff4f5e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff019d7b0 con 0x7f7ff00fde10 2024-09-13T18:20:06.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.889+0000 7f7ff4f5e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff019d920 con 0x7f7ff01013a0 2024-09-13T18:20:06.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.889+0000 7f7ff4f5e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff019dae0 con 0x7f7ff00fe7d0 2024-09-13T18:20:06.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.889+0000 7f7fee59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ff00fde10 0x7f7ff0195ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:06.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.889+0000 7f7fee59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ff00fde10 0x7f7ff0195ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44848/0 (socket says 172.21.15.114:44848) 2024-09-13T18:20:06.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.889+0000 7f7fee59c640 1 -- 172.21.15.114:0/1109703220 learned_addr learned my addr 172.21.15.114:0/1109703220 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:20:06.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.889+0000 7f7feed9d640 1 --2- 172.21.15.114:0/1109703220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ff01013a0 0x7f7ff019d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:06.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.889+0000 7f7fee59c640 1 -- 172.21.15.114:0/1109703220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ff00fe7d0 msgr2=0x7f7ff01961e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:06.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.889+0000 7f7fee59c640 1 --2- 172.21.15.114:0/1109703220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ff00fe7d0 0x7f7ff01961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:06.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.889+0000 7f7fee59c640 1 -- 172.21.15.114:0/1109703220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ff01013a0 msgr2=0x7f7ff019d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:06.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.889+0000 7f7fee59c640 1 --2- 172.21.15.114:0/1109703220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ff01013a0 0x7f7ff019d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:06.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.889+0000 7f7fee59c640 1 -- 172.21.15.114:0/1109703220 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe0009560 con 0x7f7ff00fde10 2024-09-13T18:20:06.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.889+0000 7f7fee59c640 1 --2- 172.21.15.114:0/1109703220 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ff00fde10 0x7f7ff0195ca0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7fd800c970 tx=0x7f7fd800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:06.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.890+0000 7f7fe77fe640 1 -- 172.21.15.114:0/1109703220 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd8007c10 con 0x7f7ff00fde10 2024-09-13T18:20:06.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.890+0000 7f7ff4f5e640 1 -- 172.21.15.114:0/1109703220 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ff019ddc0 con 0x7f7ff00fde10 2024-09-13T18:20:06.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.890+0000 7f7fe77fe640 1 -- 172.21.15.114:0/1109703220 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7fd8007d70 con 0x7f7ff00fde10 2024-09-13T18:20:06.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.890+0000 7f7fe77fe640 1 -- 172.21.15.114:0/1109703220 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd800f690 con 0x7f7ff00fde10 2024-09-13T18:20:06.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.890+0000 7f7ff4f5e640 1 -- 172.21.15.114:0/1109703220 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ff019e270 con 0x7f7ff00fde10 2024-09-13T18:20:06.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.891+0000 7f7ff4f5e640 1 -- 172.21.15.114:0/1109703220 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ff0107470 con 0x7f7ff00fde10 2024-09-13T18:20:06.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.892+0000 7f7fe77fe640 1 -- 172.21.15.114:0/1109703220 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f7fd8018480 con 0x7f7ff00fde10 2024-09-13T18:20:06.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.892+0000 7f7fe77fe640 1 --2- 172.21.15.114:0/1109703220 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f7fdc0a1090 0x7f7fdc0a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:06.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.892+0000 7f7fedd9b640 1 --2- 172.21.15.114:0/1109703220 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f7fdc0a1090 0x7f7fdc0a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:06.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.893+0000 7f7fe77fe640 1 -- 172.21.15.114:0/1109703220 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f7fd8013070 con 0x7f7ff00fde10 2024-09-13T18:20:06.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.893+0000 7f7fedd9b640 1 --2- 172.21.15.114:0/1109703220 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f7fdc0a1090 0x7f7fdc0a3550 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7fe002a930 tx=0x7f7fe0000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:06.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:06.896+0000 7f7fe77fe640 1 -- 172.21.15.114:0/1109703220 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f7fd8096450 con 0x7f7ff00fde10 2024-09-13T18:20:07.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.064+0000 7f7ff4f5e640 1 -- 172.21.15.114:0/1109703220 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ff002c4b0 con 0x7f7ff00fde10 2024-09-13T18:20:07.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.066+0000 7f7fe77fe640 1 -- 172.21.15.114:0/1109703220 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24029 (secure 0 0 0) 0x7f7fd809ee60 con 0x7f7ff00fde10 2024-09-13T18:20:07.068 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:20:07.068 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":89,"fsid":"97e7edf2-71fb-11ef-bceb-c7b262605968","created":"2024-09-13T18:12:36.360625+0000","modified":"2024-09-13T18:20:06.455681+0000","last_up_change":"2024-09-13T18:19:12.350370+0000","last_in_change":"2024-09-13T18:18:52.321596+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:15:40.129659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:20:00.024407+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:20:01.085549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T18:20:04.334216+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9914f01a-9f6f-4878-b955-83570199c354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6803","nonce":3499525826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6805","nonce":3499525826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6809","nonce":3499525826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6807","nonce":3499525826}]},"public_addr":"172.21.15.114:6803/3499525826","cluster_addr":"172.21.15.114:6805/3499525826","heartbeat_back_addr":"172.21.15.114:6809/3499525826","heartbeat_front_addr":"172.21.15.114:6807/3499525826","state":["exists","up"]},{"osd":1,"uuid":"ef65e012-7469-4c20-b354-17022bca63c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6801","nonce":4264264866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6803","nonce":4264264866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6807","nonce":4264264866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6805","nonce":4264264866}]},"public_addr":"172.21.15.134:6801/4264264866","cluster_addr":"172.21.15.134:6803/4264264866","heartbeat_back_addr":"172.21.15.134:6807/4264264866","heartbeat_front_addr":"172.21.15.134:6805/4264264866","state":["exists","up"]},{"osd":2,"uuid":"578213fd-715f-4be7-91fa-254f3d0e3022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6801","nonce":2288807707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6803","nonce":2288807707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6807","nonce":2288807707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6805","nonce":2288807707}]},"public_addr":"172.21.15.204:6801/2288807707","cluster_addr":"172.21.15.204:6803/2288807707","heartbeat_back_addr":"172.21.15.204:6807/2288807707","heartbeat_front_addr":"172.21.15.204:6805/2288807707","state":["exists","up"]},{"osd":3,"uuid":"b64aab3f-1e52-4828-8397-74af5e05f2cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6811","nonce":2904204254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6813","nonce":2904204254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6817","nonce":2904204254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6815","nonce":2904204254}]},"public_addr":"172.21.15.114:6811/2904204254","cluster_addr":"172.21.15.114:6813/2904204254","heartbeat_back_addr":"172.21.15.114:6817/2904204254","heartbeat_front_addr":"172.21.15.114:6815/2904204254","state":["exists","up"]},{"osd":4,"uuid":"f38943d6-39c0-42e1-b427-cb6bb90ac38a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6809","nonce":1773945333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6811","nonce":1773945333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6815","nonce":1773945333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6813","nonce":1773945333}]},"public_addr":"172.21.15.134:6809/1773945333","cluster_addr":"172.21.15.134:6811/1773945333","heartbeat_back_addr":"172.21.15.134:6815/1773945333","heartbeat_front_addr":"172.21.15.134:6813/1773945333","state":["exists","up"]},{"osd":5,"uuid":"bb38f361-8e00-4178-9752-5ddbf3af20e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6809","nonce":2371778055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6811","nonce":2371778055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6815","nonce":2371778055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6813","nonce":2371778055}]},"public_addr":"172.21.15.204:6809/2371778055","cluster_addr":"172.21.15.204:6811/2371778055","heartbeat_back_addr":"172.21.15.204:6815/2371778055","heartbeat_front_addr":"172.21.15.204:6813/2371778055","state":["exists","up"]},{"osd":6,"uuid":"93d40a03-e7ff-4a3f-9efa-5c5403c9983d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6819","nonce":1484325044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6821","nonce":1484325044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6825","nonce":1484325044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6823","nonce":1484325044}]},"public_addr":"172.21.15.114:6819/1484325044","cluster_addr":"172.21.15.114:6821/1484325044","heartbeat_back_addr":"172.21.15.114:6825/1484325044","heartbeat_front_addr":"172.21.15.114:6823/1484325044","state":["exists","up"]},{"osd":7,"uuid":"982f629a-1b95-4be1-8e75-5ddce1ad77b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6817","nonce":799246614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6819","nonce":799246614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6823","nonce":799246614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6821","nonce":799246614}]},"public_addr":"172.21.15.134:6817/799246614","cluster_addr":"172.21.15.134:6819/799246614","heartbeat_back_addr":"172.21.15.134:6823/799246614","heartbeat_front_addr":"172.21.15.134:6821/799246614","state":["exists","up"]},{"osd":8,"uuid":"409beb9b-8227-4dad-a4a0-f0e5ce70dc5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6817","nonce":858331805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6819","nonce":858331805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6823","nonce":858331805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6821","nonce":858331805}]},"public_addr":"172.21.15.204:6817/858331805","cluster_addr":"172.21.15.204:6819/858331805","heartbeat_back_addr":"172.21.15.204:6823/858331805","heartbeat_front_addr":"172.21.15.204:6821/858331805","state":["exists","up"]},{"osd":9,"uuid":"a345a15c-b1fc-4877-b445-418eebf0d0ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6827","nonce":2172392854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6829","nonce":2172392854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6833","nonce":2172392854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6831","nonce":2172392854}]},"public_addr":"172.21.15.114:6827/2172392854","cluster_addr":"172.21.15.114:6829/2172392854","heartbeat_back_addr":"172.21.15.114:6833/2172392854","heartbeat_front_addr":"172.21.15.114:6831/2172392854","state":["exists","up"]},{"osd":10,"uuid":"464d995f-d90b-4caa-addb-1f7562bd74b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6825","nonce":3073513468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6827","nonce":3073513468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6831","nonce":3073513468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6829","nonce":3073513468}]},"public_addr":"172.21.15.134:6825/3073513468","cluster_addr":"172.21.15.134:6827/3073513468","heartbeat_back_addr":"172.21.15.134:6831/3073513468","heartbeat_front_addr":"172.21.15.134:6829/3073513468","state":["exists","up"]},{"osd":11,"uuid":"71dd794d-326c-477f-8d02-ee32bff1350d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6825","nonce":3239521765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6827","nonce":3239521765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6831","nonce":3239521765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6829","nonce":3239521765}]},"public_addr":"172.21.15.204:6825/3239521765","cluster_addr":"172.21.15.204:6827/3239521765","heartbeat_back_addr":"172.21.15.204:6831/3239521765","heartbeat_front_addr":"172.21.15.204:6829/3239521765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:14:54.542687+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:15:15.746055+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:15:38.371131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:01.383932+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:24.796237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:48.422214+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:11.284323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:34.570365+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:58.232855+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:18:21.609596+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:18:45.565859+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:19:10.017969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:6801/4281866858":"2024-09-14T18:13:25.823027+0000","172.21.15.114:6800/4281866858":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2949422849":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2034742251":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/722571999":"2024-09-14T18:13:03.521116+0000","172.21.15.114:6801/830738794":"2024-09-14T18:13:03.521116+0000","172.21.15.114:6800/830738794":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/2828364892":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/4257828906":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/3544951322":"2024-09-14T18:13:03.521116+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:20:07.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.067+0000 7f7ff4f5e640 1 -- 172.21.15.114:0/1109703220 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f7fdc0a1090 msgr2=0x7f7fdc0a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:07.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.067+0000 7f7ff4f5e640 1 --2- 172.21.15.114:0/1109703220 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f7fdc0a1090 0x7f7fdc0a3550 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7fe002a930 tx=0x7f7fe0000bc0 comp rx=0 tx=0).stop 2024-09-13T18:20:07.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.068+0000 7f7ff4f5e640 1 -- 172.21.15.114:0/1109703220 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ff00fde10 msgr2=0x7f7ff0195ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:07.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.068+0000 7f7ff4f5e640 1 --2- 172.21.15.114:0/1109703220 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ff00fde10 0x7f7ff0195ca0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7fd800c970 tx=0x7f7fd800ce30 comp rx=0 tx=0).stop 2024-09-13T18:20:07.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.068+0000 7f7ff4f5e640 1 -- 172.21.15.114:0/1109703220 shutdown_connections 2024-09-13T18:20:07.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.068+0000 7f7ff4f5e640 1 --2- 172.21.15.114:0/1109703220 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f7fdc0a1090 0x7f7fdc0a3550 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:07.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.068+0000 7f7ff4f5e640 1 --2- 172.21.15.114:0/1109703220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ff01013a0 0x7f7ff019d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:07.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.068+0000 7f7ff4f5e640 1 --2- 172.21.15.114:0/1109703220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ff00fe7d0 0x7f7ff01961e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:07.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.068+0000 7f7ff4f5e640 1 --2- 172.21.15.114:0/1109703220 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ff00fde10 0x7f7ff0195ca0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:07.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.068+0000 7f7ff4f5e640 1 -- 172.21.15.114:0/1109703220 >> 172.21.15.114:0/1109703220 conn(0x7f7ff00f9bc0 msgr2=0x7f7ff0104640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:07.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.068+0000 7f7ff4f5e640 1 -- 172.21.15.114:0/1109703220 shutdown_connections 2024-09-13T18:20:07.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.068+0000 7f7ff4f5e640 1 -- 172.21.15.114:0/1109703220 wait complete. 2024-09-13T18:20:07.081 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:20:07.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.178+0000 7f95531ae640 1 -- 172.21.15.114:0/393608871 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f954c100850 msgr2=0x7f954c1070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:07.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.178+0000 7f95531ae640 1 --2- 172.21.15.114:0/393608871 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f954c100850 0x7f954c1070e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9544009ca0 tx=0x7f954402a8c0 comp rx=0 tx=0).stop 2024-09-13T18:20:07.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.178+0000 7f95531ae640 1 -- 172.21.15.114:0/393608871 shutdown_connections 2024-09-13T18:20:07.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.178+0000 7f95531ae640 1 --2- 172.21.15.114:0/393608871 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f954c100850 0x7f954c1070e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:07.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.178+0000 7f95531ae640 1 --2- 172.21.15.114:0/393608871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f954c0ffe90 0x7f954c100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:07.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.178+0000 7f95531ae640 1 --2- 172.21.15.114:0/393608871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f954c0feca0 0x7f954c0ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:07.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.178+0000 7f95531ae640 1 -- 172.21.15.114:0/393608871 >> 172.21.15.114:0/393608871 conn(0x7f954c0fa420 msgr2=0x7f954c0fc840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:07.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.178+0000 7f95531ae640 1 -- 172.21.15.114:0/393608871 shutdown_connections 2024-09-13T18:20:07.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.178+0000 7f95531ae640 1 -- 172.21.15.114:0/393608871 wait complete. 2024-09-13T18:20:07.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.178+0000 7f95531ae640 1 Processor -- start 2024-09-13T18:20:07.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.178+0000 7f95531ae640 1 -- start start 2024-09-13T18:20:07.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.178+0000 7f95531ae640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f954c0feca0 0x7f954c069160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:07.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.178+0000 7f95531ae640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f954c0ffe90 0x7f954c19ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:07.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.178+0000 7f95531ae640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f954c100850 0x7f954c19d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:07.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.178+0000 7f95531ae640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954c19d900 con 0x7f954c100850 2024-09-13T18:20:07.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.178+0000 7f95531ae640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954c19da70 con 0x7f954c0ffe90 2024-09-13T18:20:07.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.178+0000 7f95531ae640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954c19dc30 con 0x7f954c0feca0 2024-09-13T18:20:07.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.179+0000 7f955174b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f954c100850 0x7f954c19d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:07.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.179+0000 7f955174b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f954c100850 0x7f954c19d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44860/0 (socket says 172.21.15.114:44860) 2024-09-13T18:20:07.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.179+0000 7f955174b640 1 -- 172.21.15.114:0/504068213 learned_addr learned my addr 172.21.15.114:0/504068213 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:20:07.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.179+0000 7f9550f4a640 1 --2- 172.21.15.114:0/504068213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f954c0feca0 0x7f954c069160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:07.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.179+0000 7f954bfff640 1 --2- 172.21.15.114:0/504068213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f954c0ffe90 0x7f954c19ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:07.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.179+0000 7f955174b640 1 -- 172.21.15.114:0/504068213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f954c0feca0 msgr2=0x7f954c069160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:07.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.179+0000 7f955174b640 1 --2- 172.21.15.114:0/504068213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f954c0feca0 0x7f954c069160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:07.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.179+0000 7f955174b640 1 -- 172.21.15.114:0/504068213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f954c0ffe90 msgr2=0x7f954c19ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:07.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.179+0000 7f955174b640 1 --2- 172.21.15.114:0/504068213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f954c0ffe90 0x7f954c19ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:07.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.179+0000 7f955174b640 1 -- 172.21.15.114:0/504068213 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9544009900 con 0x7f954c100850 2024-09-13T18:20:07.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.179+0000 7f9550f4a640 1 --2- 172.21.15.114:0/504068213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f954c0feca0 0x7f954c069160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:20:07.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.179+0000 7f955174b640 1 --2- 172.21.15.114:0/504068213 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f954c100850 0x7f954c19d120 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9544009dc0 tx=0x7f954402f450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:07.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.179+0000 7f9549ffb640 1 -- 172.21.15.114:0/504068213 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9544003d30 con 0x7f954c100850 2024-09-13T18:20:07.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.179+0000 7f9549ffb640 1 -- 172.21.15.114:0/504068213 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9544040070 con 0x7f954c100850 2024-09-13T18:20:07.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.179+0000 7f9549ffb640 1 -- 172.21.15.114:0/504068213 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95440045b0 con 0x7f954c100850 2024-09-13T18:20:07.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.179+0000 7f95531ae640 1 -- 172.21.15.114:0/504068213 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f954c19de50 con 0x7f954c100850 2024-09-13T18:20:07.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.179+0000 7f95531ae640 1 -- 172.21.15.114:0/504068213 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f954c19e350 con 0x7f954c100850 2024-09-13T18:20:07.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.180+0000 7f95531ae640 1 -- 172.21.15.114:0/504068213 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f954c1061a0 con 0x7f954c100850 2024-09-13T18:20:07.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.181+0000 7f9549ffb640 1 -- 172.21.15.114:0/504068213 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f9544042030 con 0x7f954c100850 2024-09-13T18:20:07.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.182+0000 7f9549ffb640 1 --2- 172.21.15.114:0/504068213 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f953c0a1110 0x7f953c0a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:07.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.182+0000 7f9550f4a640 1 --2- 172.21.15.114:0/504068213 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f953c0a1110 0x7f953c0a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:07.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.182+0000 7f9549ffb640 1 -- 172.21.15.114:0/504068213 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f95440ee420 con 0x7f954c100850 2024-09-13T18:20:07.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.183+0000 7f9550f4a640 1 --2- 172.21.15.114:0/504068213 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f953c0a1110 0x7f953c0a35d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f954c0ffcf0 tx=0x7f9540008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:07.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.186+0000 7f9549ffb640 1 -- 172.21.15.114:0/504068213 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f95440b79c0 con 0x7f954c100850 2024-09-13T18:20:07.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.349+0000 7f95531ae640 1 -- 172.21.15.114:0/504068213 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f954c02c480 con 0x7f954c100850 2024-09-13T18:20:07.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.350+0000 7f9549ffb640 1 -- 172.21.15.114:0/504068213 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24029 (secure 0 0 0) 0x7f95440bafe0 con 0x7f954c100850 2024-09-13T18:20:07.353 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:20:07.353 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":89,"fsid":"97e7edf2-71fb-11ef-bceb-c7b262605968","created":"2024-09-13T18:12:36.360625+0000","modified":"2024-09-13T18:20:06.455681+0000","last_up_change":"2024-09-13T18:19:12.350370+0000","last_in_change":"2024-09-13T18:18:52.321596+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:15:40.129659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:20:00.024407+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:20:01.085549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T18:20:04.334216+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9914f01a-9f6f-4878-b955-83570199c354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6803","nonce":3499525826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6805","nonce":3499525826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6809","nonce":3499525826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6807","nonce":3499525826}]},"public_addr":"172.21.15.114:6803/3499525826","cluster_addr":"172.21.15.114:6805/3499525826","heartbeat_back_addr":"172.21.15.114:6809/3499525826","heartbeat_front_addr":"172.21.15.114:6807/3499525826","state":["exists","up"]},{"osd":1,"uuid":"ef65e012-7469-4c20-b354-17022bca63c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6801","nonce":4264264866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6803","nonce":4264264866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6807","nonce":4264264866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6805","nonce":4264264866}]},"public_addr":"172.21.15.134:6801/4264264866","cluster_addr":"172.21.15.134:6803/4264264866","heartbeat_back_addr":"172.21.15.134:6807/4264264866","heartbeat_front_addr":"172.21.15.134:6805/4264264866","state":["exists","up"]},{"osd":2,"uuid":"578213fd-715f-4be7-91fa-254f3d0e3022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6801","nonce":2288807707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6803","nonce":2288807707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6807","nonce":2288807707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6805","nonce":2288807707}]},"public_addr":"172.21.15.204:6801/2288807707","cluster_addr":"172.21.15.204:6803/2288807707","heartbeat_back_addr":"172.21.15.204:6807/2288807707","heartbeat_front_addr":"172.21.15.204:6805/2288807707","state":["exists","up"]},{"osd":3,"uuid":"b64aab3f-1e52-4828-8397-74af5e05f2cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6811","nonce":2904204254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6813","nonce":2904204254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6817","nonce":2904204254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6815","nonce":2904204254}]},"public_addr":"172.21.15.114:6811/2904204254","cluster_addr":"172.21.15.114:6813/2904204254","heartbeat_back_addr":"172.21.15.114:6817/2904204254","heartbeat_front_addr":"172.21.15.114:6815/2904204254","state":["exists","up"]},{"osd":4,"uuid":"f38943d6-39c0-42e1-b427-cb6bb90ac38a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6809","nonce":1773945333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6811","nonce":1773945333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6815","nonce":1773945333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6813","nonce":1773945333}]},"public_addr":"172.21.15.134:6809/1773945333","cluster_addr":"172.21.15.134:6811/1773945333","heartbeat_back_addr":"172.21.15.134:6815/1773945333","heartbeat_front_addr":"172.21.15.134:6813/1773945333","state":["exists","up"]},{"osd":5,"uuid":"bb38f361-8e00-4178-9752-5ddbf3af20e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6809","nonce":2371778055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6811","nonce":2371778055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6815","nonce":2371778055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6813","nonce":2371778055}]},"public_addr":"172.21.15.204:6809/2371778055","cluster_addr":"172.21.15.204:6811/2371778055","heartbeat_back_addr":"172.21.15.204:6815/2371778055","heartbeat_front_addr":"172.21.15.204:6813/2371778055","state":["exists","up"]},{"osd":6,"uuid":"93d40a03-e7ff-4a3f-9efa-5c5403c9983d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6819","nonce":1484325044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6821","nonce":1484325044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6825","nonce":1484325044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6823","nonce":1484325044}]},"public_addr":"172.21.15.114:6819/1484325044","cluster_addr":"172.21.15.114:6821/1484325044","heartbeat_back_addr":"172.21.15.114:6825/1484325044","heartbeat_front_addr":"172.21.15.114:6823/1484325044","state":["exists","up"]},{"osd":7,"uuid":"982f629a-1b95-4be1-8e75-5ddce1ad77b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6817","nonce":799246614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6819","nonce":799246614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6823","nonce":799246614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6821","nonce":799246614}]},"public_addr":"172.21.15.134:6817/799246614","cluster_addr":"172.21.15.134:6819/799246614","heartbeat_back_addr":"172.21.15.134:6823/799246614","heartbeat_front_addr":"172.21.15.134:6821/799246614","state":["exists","up"]},{"osd":8,"uuid":"409beb9b-8227-4dad-a4a0-f0e5ce70dc5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6817","nonce":858331805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6819","nonce":858331805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6823","nonce":858331805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6821","nonce":858331805}]},"public_addr":"172.21.15.204:6817/858331805","cluster_addr":"172.21.15.204:6819/858331805","heartbeat_back_addr":"172.21.15.204:6823/858331805","heartbeat_front_addr":"172.21.15.204:6821/858331805","state":["exists","up"]},{"osd":9,"uuid":"a345a15c-b1fc-4877-b445-418eebf0d0ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6827","nonce":2172392854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6829","nonce":2172392854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6833","nonce":2172392854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6831","nonce":2172392854}]},"public_addr":"172.21.15.114:6827/2172392854","cluster_addr":"172.21.15.114:6829/2172392854","heartbeat_back_addr":"172.21.15.114:6833/2172392854","heartbeat_front_addr":"172.21.15.114:6831/2172392854","state":["exists","up"]},{"osd":10,"uuid":"464d995f-d90b-4caa-addb-1f7562bd74b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6825","nonce":3073513468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6827","nonce":3073513468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6831","nonce":3073513468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6829","nonce":3073513468}]},"public_addr":"172.21.15.134:6825/3073513468","cluster_addr":"172.21.15.134:6827/3073513468","heartbeat_back_addr":"172.21.15.134:6831/3073513468","heartbeat_front_addr":"172.21.15.134:6829/3073513468","state":["exists","up"]},{"osd":11,"uuid":"71dd794d-326c-477f-8d02-ee32bff1350d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6825","nonce":3239521765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6827","nonce":3239521765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6831","nonce":3239521765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6829","nonce":3239521765}]},"public_addr":"172.21.15.204:6825/3239521765","cluster_addr":"172.21.15.204:6827/3239521765","heartbeat_back_addr":"172.21.15.204:6831/3239521765","heartbeat_front_addr":"172.21.15.204:6829/3239521765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:14:54.542687+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:15:15.746055+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:15:38.371131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:01.383932+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:24.796237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:48.422214+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:11.284323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:34.570365+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:58.232855+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:18:21.609596+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:18:45.565859+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:19:10.017969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:6801/4281866858":"2024-09-14T18:13:25.823027+0000","172.21.15.114:6800/4281866858":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2949422849":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2034742251":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/722571999":"2024-09-14T18:13:03.521116+0000","172.21.15.114:6801/830738794":"2024-09-14T18:13:03.521116+0000","172.21.15.114:6800/830738794":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/2828364892":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/4257828906":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/3544951322":"2024-09-14T18:13:03.521116+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:20:07.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.352+0000 7f95531ae640 1 -- 172.21.15.114:0/504068213 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f953c0a1110 msgr2=0x7f953c0a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:07.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.352+0000 7f95531ae640 1 --2- 172.21.15.114:0/504068213 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f953c0a1110 0x7f953c0a35d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f954c0ffcf0 tx=0x7f9540008040 comp rx=0 tx=0).stop 2024-09-13T18:20:07.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.353+0000 7f95531ae640 1 -- 172.21.15.114:0/504068213 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f954c100850 msgr2=0x7f954c19d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:07.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.353+0000 7f95531ae640 1 --2- 172.21.15.114:0/504068213 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f954c100850 0x7f954c19d120 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9544009dc0 tx=0x7f954402f450 comp rx=0 tx=0).stop 2024-09-13T18:20:07.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.353+0000 7f95531ae640 1 -- 172.21.15.114:0/504068213 shutdown_connections 2024-09-13T18:20:07.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.353+0000 7f95531ae640 1 --2- 172.21.15.114:0/504068213 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f953c0a1110 0x7f953c0a35d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:07.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.353+0000 7f95531ae640 1 --2- 172.21.15.114:0/504068213 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f954c100850 0x7f954c19d120 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:07.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.353+0000 7f95531ae640 1 --2- 172.21.15.114:0/504068213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f954c0ffe90 0x7f954c19ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:07.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.353+0000 7f95531ae640 1 --2- 172.21.15.114:0/504068213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f954c0feca0 0x7f954c069160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:07.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.353+0000 7f95531ae640 1 -- 172.21.15.114:0/504068213 >> 172.21.15.114:0/504068213 conn(0x7f954c0fa420 msgr2=0x7f954c103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:07.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.353+0000 7f95531ae640 1 -- 172.21.15.114:0/504068213 shutdown_connections 2024-09-13T18:20:07.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.353+0000 7f95531ae640 1 -- 172.21.15.114:0/504068213 wait complete. 2024-09-13T18:20:07.364 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:07 smithi114 ceph-mon[30897]: pgmap v277: 129 pgs: 20 creating+peering, 64 active+clean, 45 unknown; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 3 op/s 2024-09-13T18:20:07.364 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:07 smithi114 ceph-mon[30897]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T18:20:07.364 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:07 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2037859411' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-13T18:20:07.364 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:07 smithi114 ceph-mon[30897]: osdmap e88: 12 total, 12 up, 12 in 2024-09-13T18:20:07.364 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:07 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2185728546' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-13T18:20:07.364 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:07 smithi114 ceph-mon[30897]: osdmap e89: 12 total, 12 up, 12 in 2024-09-13T18:20:07.364 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:07 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2185728546' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-13T18:20:07.365 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:07 smithi114 ceph-mon[30897]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-13T18:20:07.365 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:07 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/3480288595' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:20:07.365 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:07 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/1109703220' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:20:07.384 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:20:07.385 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 13 18:20:06 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-b[63915]: 2024-09-13T18:20:06.959+0000 7f4e94321640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T18:20:07.385 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:07 smithi204 ceph-mon[34777]: pgmap v277: 129 pgs: 20 creating+peering, 64 active+clean, 45 unknown; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 3 op/s 2024-09-13T18:20:07.385 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:07 smithi204 ceph-mon[34777]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T18:20:07.385 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:07 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/2037859411' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-13T18:20:07.385 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:07 smithi204 ceph-mon[34777]: osdmap e88: 12 total, 12 up, 12 in 2024-09-13T18:20:07.385 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:07 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/2185728546' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-13T18:20:07.385 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:07 smithi204 ceph-mon[34777]: osdmap e89: 12 total, 12 up, 12 in 2024-09-13T18:20:07.385 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:07 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/2185728546' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-13T18:20:07.385 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:07 smithi204 ceph-mon[34777]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-13T18:20:07.385 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:07 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/3480288595' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:20:07.385 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:07 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/1109703220' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:20:07.386 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:07 smithi134 ceph-mon[34283]: pgmap v277: 129 pgs: 20 creating+peering, 64 active+clean, 45 unknown; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 3 op/s 2024-09-13T18:20:07.386 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:07 smithi134 ceph-mon[34283]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-13T18:20:07.386 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:07 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/2037859411' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-13T18:20:07.386 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:07 smithi134 ceph-mon[34283]: osdmap e88: 12 total, 12 up, 12 in 2024-09-13T18:20:07.386 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:07 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/2185728546' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-13T18:20:07.386 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:07 smithi134 ceph-mon[34283]: osdmap e89: 12 total, 12 up, 12 in 2024-09-13T18:20:07.386 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:07 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/2185728546' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-13T18:20:07.386 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:07 smithi134 ceph-mon[34283]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-13T18:20:07.386 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:07 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/3480288595' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:20:07.386 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:07 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/1109703220' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:20:07.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.482+0000 7f624b8b8640 1 -- 172.21.15.114:0/723527122 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6244100850 msgr2=0x7f62441070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:07.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.482+0000 7f624b8b8640 1 --2- 172.21.15.114:0/723527122 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6244100850 0x7f62441070e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f623c009ca0 tx=0x7f623c02a8c0 comp rx=0 tx=0).stop 2024-09-13T18:20:07.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.482+0000 7f624b8b8640 1 -- 172.21.15.114:0/723527122 shutdown_connections 2024-09-13T18:20:07.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.482+0000 7f624b8b8640 1 --2- 172.21.15.114:0/723527122 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6244100850 0x7f62441070e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:07.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.482+0000 7f624b8b8640 1 --2- 172.21.15.114:0/723527122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62440ffe90 0x7f6244100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:07.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.482+0000 7f624b8b8640 1 --2- 172.21.15.114:0/723527122 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62440feca0 0x7f62440ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:07.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.482+0000 7f624b8b8640 1 -- 172.21.15.114:0/723527122 >> 172.21.15.114:0/723527122 conn(0x7f62440fa440 msgr2=0x7f62440fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:07.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.482+0000 7f624b8b8640 1 -- 172.21.15.114:0/723527122 shutdown_connections 2024-09-13T18:20:07.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.482+0000 7f624b8b8640 1 -- 172.21.15.114:0/723527122 wait complete. 2024-09-13T18:20:07.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.483+0000 7f624b8b8640 1 Processor -- start 2024-09-13T18:20:07.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.483+0000 7f624b8b8640 1 -- start start 2024-09-13T18:20:07.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.483+0000 7f624b8b8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f62440feca0 0x7f6244195bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:07.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.483+0000 7f624b8b8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62440ffe90 0x7f6244196100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:07.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.483+0000 7f624b8b8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6244100850 0x7f624419d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:07.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.483+0000 7f624b8b8640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f624419d6b0 con 0x7f62440feca0 2024-09-13T18:20:07.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.483+0000 7f624b8b8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f624419d820 con 0x7f6244100850 2024-09-13T18:20:07.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.483+0000 7f624b8b8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f624419d9e0 con 0x7f62440ffe90 2024-09-13T18:20:07.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.483+0000 7f6249654640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f62440feca0 0x7f6244195bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:07.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.483+0000 7f6249654640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f62440feca0 0x7f6244195bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44878/0 (socket says 172.21.15.114:44878) 2024-09-13T18:20:07.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.483+0000 7f6249654640 1 -- 172.21.15.114:0/816150721 learned_addr learned my addr 172.21.15.114:0/816150721 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:20:07.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.483+0000 7f6248e53640 1 --2- 172.21.15.114:0/816150721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62440ffe90 0x7f6244196100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:07.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.484+0000 7f6249654640 1 -- 172.21.15.114:0/816150721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62440ffe90 msgr2=0x7f6244196100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:07.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.484+0000 7f6249654640 1 --2- 172.21.15.114:0/816150721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62440ffe90 0x7f6244196100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:07.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.484+0000 7f6249654640 1 -- 172.21.15.114:0/816150721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6244100850 msgr2=0x7f624419d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:07.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.484+0000 7f6249654640 1 --2- 172.21.15.114:0/816150721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6244100850 0x7f624419d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:07.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.484+0000 7f6249654640 1 -- 172.21.15.114:0/816150721 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f623c009900 con 0x7f62440feca0 2024-09-13T18:20:07.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.484+0000 7f6249654640 1 --2- 172.21.15.114:0/816150721 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f62440feca0 0x7f6244195bc0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f624000c970 tx=0x7f624000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:07.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.484+0000 7f623a7fc640 1 -- 172.21.15.114:0/816150721 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6240007c10 con 0x7f62440feca0 2024-09-13T18:20:07.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.484+0000 7f623a7fc640 1 -- 172.21.15.114:0/816150721 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6240007d70 con 0x7f62440feca0 2024-09-13T18:20:07.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.484+0000 7f623a7fc640 1 -- 172.21.15.114:0/816150721 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624000f720 con 0x7f62440feca0 2024-09-13T18:20:07.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.484+0000 7f624b8b8640 1 -- 172.21.15.114:0/816150721 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f624419dcc0 con 0x7f62440feca0 2024-09-13T18:20:07.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.484+0000 7f624b8b8640 1 -- 172.21.15.114:0/816150721 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f624419e1d0 con 0x7f62440feca0 2024-09-13T18:20:07.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.485+0000 7f624b8b8640 1 -- 172.21.15.114:0/816150721 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62441060d0 con 0x7f62440feca0 2024-09-13T18:20:07.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.486+0000 7f623a7fc640 1 -- 172.21.15.114:0/816150721 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f6240018480 con 0x7f62440feca0 2024-09-13T18:20:07.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.486+0000 7f623a7fc640 1 --2- 172.21.15.114:0/816150721 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f62300a1110 0x7f62300a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:07.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.487+0000 7f6248e53640 1 --2- 172.21.15.114:0/816150721 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f62300a1110 0x7f62300a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:07.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.487+0000 7f623a7fc640 1 -- 172.21.15.114:0/816150721 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f6240013070 con 0x7f62440feca0 2024-09-13T18:20:07.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.487+0000 7f6248e53640 1 --2- 172.21.15.114:0/816150721 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f62300a1110 0x7f62300a35d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f62441970d0 tx=0x7f6234005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:07.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.490+0000 7f623a7fc640 1 -- 172.21.15.114:0/816150721 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f62400963a0 con 0x7f62440feca0 2024-09-13T18:20:07.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.688+0000 7f624b8b8640 1 -- 172.21.15.114:0/816150721 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f624410b590 con 0x7f62440feca0 2024-09-13T18:20:07.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.689+0000 7f623a7fc640 1 -- 172.21.15.114:0/816150721 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+10819 (secure 0 0 0) 0x7f62400999c0 con 0x7f62440feca0 2024-09-13T18:20:07.691 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:20:07.691 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4162817497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4162817497},{"type":"v1","addr":"172.21.15.114:6835","nonce":4162817497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/2435428478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":2435428478},{"type":"v1","addr":"172.21.15.114:6837","nonce":2435428478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6839/2046516451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":2046516451},{"type":"v1","addr":"172.21.15.114:6839","nonce":2046516451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6841/1956923446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":1956923446},{"type":"v1","addr":"172.21.15.114:6841","nonce":1956923446}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/507301496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":507301496},{"type":"v1","addr":"172.21.15.204:6833","nonce":507301496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/4028657219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4028657219},{"type":"v1","addr":"172.21.15.134:6833","nonce":4028657219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/57029119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":57029119},{"type":"v1","addr":"172.21.15.204:6835","nonce":57029119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/3068880939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3068880939},{"type":"v1","addr":"172.21.15.204:6837","nonce":3068880939}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1150003275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1150003275},{"type":"v1","addr":"172.21.15.134:6835","nonce":1150003275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24475,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/344883912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":344883912},{"type":"v1","addr":"172.21.15.204:6839","nonce":344883912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/891692210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":891692210},{"type":"v1","addr":"172.21.15.134:6837","nonce":891692210}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:20:01.895117+0000","modified":"2024-09-13T18:20:06.455916+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6839/3466309724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3466309724},{"type":"v1","addr":"172.21.15.134:6839","nonce":3466309724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-09-13T18:20:07.692 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 17 2024-09-13T18:20:07.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.691+0000 7f624b8b8640 1 -- 172.21.15.114:0/816150721 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f62300a1110 msgr2=0x7f62300a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:07.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.691+0000 7f624b8b8640 1 --2- 172.21.15.114:0/816150721 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f62300a1110 0x7f62300a35d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f62441970d0 tx=0x7f6234005f90 comp rx=0 tx=0).stop 2024-09-13T18:20:07.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.691+0000 7f624b8b8640 1 -- 172.21.15.114:0/816150721 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f62440feca0 msgr2=0x7f6244195bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:07.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.691+0000 7f624b8b8640 1 --2- 172.21.15.114:0/816150721 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f62440feca0 0x7f6244195bc0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f624000c970 tx=0x7f624000ce30 comp rx=0 tx=0).stop 2024-09-13T18:20:07.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.691+0000 7f624b8b8640 1 -- 172.21.15.114:0/816150721 shutdown_connections 2024-09-13T18:20:07.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.691+0000 7f624b8b8640 1 --2- 172.21.15.114:0/816150721 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f62300a1110 0x7f62300a35d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:07.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.691+0000 7f624b8b8640 1 --2- 172.21.15.114:0/816150721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6244100850 0x7f624419d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:07.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.691+0000 7f624b8b8640 1 --2- 172.21.15.114:0/816150721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62440ffe90 0x7f6244196100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:07.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.691+0000 7f624b8b8640 1 --2- 172.21.15.114:0/816150721 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f62440feca0 0x7f6244195bc0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:07.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.691+0000 7f624b8b8640 1 -- 172.21.15.114:0/816150721 >> 172.21.15.114:0/816150721 conn(0x7f62440fa440 msgr2=0x7f62441032a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:07.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.691+0000 7f624b8b8640 1 -- 172.21.15.114:0/816150721 shutdown_connections 2024-09-13T18:20:07.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.691+0000 7f624b8b8640 1 -- 172.21.15.114:0/816150721 wait complete. 2024-09-13T18:20:07.704 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:20:07.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.799+0000 7f87daede640 1 -- 172.21.15.114:0/3270262341 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87d40ff630 msgr2=0x7f87d40ffab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:07.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.799+0000 7f87daede640 1 --2- 172.21.15.114:0/3270262341 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87d40ff630 0x7f87d40ffab0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f87c0009900 tx=0x7f87c002a410 comp rx=0 tx=0).stop 2024-09-13T18:20:07.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.799+0000 7f87daede640 1 -- 172.21.15.114:0/3270262341 shutdown_connections 2024-09-13T18:20:07.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.799+0000 7f87daede640 1 --2- 172.21.15.114:0/3270262341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87d40ffff0 0x7f87d4106880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:07.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.799+0000 7f87daede640 1 --2- 172.21.15.114:0/3270262341 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87d40ff630 0x7f87d40ffab0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:07.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.799+0000 7f87daede640 1 --2- 172.21.15.114:0/3270262341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87d40fe440 0x7f87d40fe840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:07.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.799+0000 7f87daede640 1 -- 172.21.15.114:0/3270262341 >> 172.21.15.114:0/3270262341 conn(0x7f87d40f9bc0 msgr2=0x7f87d40fbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:07.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.799+0000 7f87daede640 1 -- 172.21.15.114:0/3270262341 shutdown_connections 2024-09-13T18:20:07.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.799+0000 7f87daede640 1 -- 172.21.15.114:0/3270262341 wait complete. 2024-09-13T18:20:07.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.799+0000 7f87daede640 1 Processor -- start 2024-09-13T18:20:07.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.800+0000 7f87daede640 1 -- start start 2024-09-13T18:20:07.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.800+0000 7f87daede640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87d40fe440 0x7f87d4101670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:07.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.800+0000 7f87daede640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87d40ff630 0x7f87d419aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:07.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.800+0000 7f87daede640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87d40ffff0 0x7f87d419d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:07.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.800+0000 7f87daede640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d419d840 con 0x7f87d40fe440 2024-09-13T18:20:07.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.800+0000 7f87daede640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d419d9b0 con 0x7f87d40ffff0 2024-09-13T18:20:07.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.800+0000 7f87daede640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d419db70 con 0x7f87d40ff630 2024-09-13T18:20:07.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.800+0000 7f87d8c7a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87d40fe440 0x7f87d4101670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:07.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.800+0000 7f87d8c7a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87d40fe440 0x7f87d4101670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44902/0 (socket says 172.21.15.114:44902) 2024-09-13T18:20:07.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.800+0000 7f87d8c7a640 1 -- 172.21.15.114:0/3220170609 learned_addr learned my addr 172.21.15.114:0/3220170609 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:20:07.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.800+0000 7f87d3fff640 1 --2- 172.21.15.114:0/3220170609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87d40ff630 0x7f87d419aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:07.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.800+0000 7f87d947b640 1 --2- 172.21.15.114:0/3220170609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87d40ffff0 0x7f87d419d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:07.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.800+0000 7f87d8c7a640 1 -- 172.21.15.114:0/3220170609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87d40ff630 msgr2=0x7f87d419aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:07.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.800+0000 7f87d8c7a640 1 --2- 172.21.15.114:0/3220170609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87d40ff630 0x7f87d419aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:07.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.800+0000 7f87d8c7a640 1 -- 172.21.15.114:0/3220170609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87d40ffff0 msgr2=0x7f87d419d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:07.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.800+0000 7f87d8c7a640 1 --2- 172.21.15.114:0/3220170609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87d40ffff0 0x7f87d419d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:07.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.800+0000 7f87d8c7a640 1 -- 172.21.15.114:0/3220170609 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c0009560 con 0x7f87d40fe440 2024-09-13T18:20:07.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.800+0000 7f87d947b640 1 --2- 172.21.15.114:0/3220170609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87d40ffff0 0x7f87d419d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:20:07.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.800+0000 7f87d8c7a640 1 --2- 172.21.15.114:0/3220170609 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87d40fe440 0x7f87d4101670 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f87c800c970 tx=0x7f87c800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:07.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.800+0000 7f87d1ffb640 1 -- 172.21.15.114:0/3220170609 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c8004220 con 0x7f87d40fe440 2024-09-13T18:20:07.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.800+0000 7f87d1ffb640 1 -- 172.21.15.114:0/3220170609 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f87c8007e40 con 0x7f87d40fe440 2024-09-13T18:20:07.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.800+0000 7f87daede640 1 -- 172.21.15.114:0/3220170609 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87d419de50 con 0x7f87d40fe440 2024-09-13T18:20:07.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.800+0000 7f87d1ffb640 1 -- 172.21.15.114:0/3220170609 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c8003e30 con 0x7f87d40fe440 2024-09-13T18:20:07.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.801+0000 7f87daede640 1 -- 172.21.15.114:0/3220170609 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87d419e2d0 con 0x7f87d40fe440 2024-09-13T18:20:07.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.802+0000 7f87daede640 1 -- 172.21.15.114:0/3220170609 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87d4106390 con 0x7f87d40fe440 2024-09-13T18:20:07.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.802+0000 7f87d1ffb640 1 -- 172.21.15.114:0/3220170609 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f87c80079b0 con 0x7f87d40fe440 2024-09-13T18:20:07.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.803+0000 7f87d1ffb640 1 --2- 172.21.15.114:0/3220170609 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f87c40a1110 0x7f87c40a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:07.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.803+0000 7f87d3fff640 1 --2- 172.21.15.114:0/3220170609 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f87c40a1110 0x7f87c40a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:07.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.803+0000 7f87d1ffb640 1 -- 172.21.15.114:0/3220170609 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f87c80cca90 con 0x7f87d40fe440 2024-09-13T18:20:07.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.804+0000 7f87d3fff640 1 --2- 172.21.15.114:0/3220170609 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f87c40a1110 0x7f87c40a35d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f87d4101c30 tx=0x7f87c0005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:07.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.807+0000 7f87d1ffb640 1 -- 172.21.15.114:0/3220170609 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f87c8096030 con 0x7f87d40fe440 2024-09-13T18:20:07.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.970+0000 7f87daede640 1 -- 172.21.15.114:0/3220170609 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f87d402c410 con 0x7f87d40fe440 2024-09-13T18:20:07.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.972+0000 7f87d1ffb640 1 -- 172.21.15.114:0/3220170609 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24029 (secure 0 0 0) 0x7f87c801d090 con 0x7f87d40fe440 2024-09-13T18:20:07.974 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:20:07.974 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":89,"fsid":"97e7edf2-71fb-11ef-bceb-c7b262605968","created":"2024-09-13T18:12:36.360625+0000","modified":"2024-09-13T18:20:06.455681+0000","last_up_change":"2024-09-13T18:19:12.350370+0000","last_in_change":"2024-09-13T18:18:52.321596+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:15:40.129659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:20:00.024407+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:20:01.085549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T18:20:04.334216+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9914f01a-9f6f-4878-b955-83570199c354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6803","nonce":3499525826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6805","nonce":3499525826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6809","nonce":3499525826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6807","nonce":3499525826}]},"public_addr":"172.21.15.114:6803/3499525826","cluster_addr":"172.21.15.114:6805/3499525826","heartbeat_back_addr":"172.21.15.114:6809/3499525826","heartbeat_front_addr":"172.21.15.114:6807/3499525826","state":["exists","up"]},{"osd":1,"uuid":"ef65e012-7469-4c20-b354-17022bca63c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6801","nonce":4264264866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6803","nonce":4264264866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6807","nonce":4264264866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6805","nonce":4264264866}]},"public_addr":"172.21.15.134:6801/4264264866","cluster_addr":"172.21.15.134:6803/4264264866","heartbeat_back_addr":"172.21.15.134:6807/4264264866","heartbeat_front_addr":"172.21.15.134:6805/4264264866","state":["exists","up"]},{"osd":2,"uuid":"578213fd-715f-4be7-91fa-254f3d0e3022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6801","nonce":2288807707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6803","nonce":2288807707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6807","nonce":2288807707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6805","nonce":2288807707}]},"public_addr":"172.21.15.204:6801/2288807707","cluster_addr":"172.21.15.204:6803/2288807707","heartbeat_back_addr":"172.21.15.204:6807/2288807707","heartbeat_front_addr":"172.21.15.204:6805/2288807707","state":["exists","up"]},{"osd":3,"uuid":"b64aab3f-1e52-4828-8397-74af5e05f2cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6811","nonce":2904204254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6813","nonce":2904204254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6817","nonce":2904204254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6815","nonce":2904204254}]},"public_addr":"172.21.15.114:6811/2904204254","cluster_addr":"172.21.15.114:6813/2904204254","heartbeat_back_addr":"172.21.15.114:6817/2904204254","heartbeat_front_addr":"172.21.15.114:6815/2904204254","state":["exists","up"]},{"osd":4,"uuid":"f38943d6-39c0-42e1-b427-cb6bb90ac38a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6809","nonce":1773945333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6811","nonce":1773945333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6815","nonce":1773945333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6813","nonce":1773945333}]},"public_addr":"172.21.15.134:6809/1773945333","cluster_addr":"172.21.15.134:6811/1773945333","heartbeat_back_addr":"172.21.15.134:6815/1773945333","heartbeat_front_addr":"172.21.15.134:6813/1773945333","state":["exists","up"]},{"osd":5,"uuid":"bb38f361-8e00-4178-9752-5ddbf3af20e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6809","nonce":2371778055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6811","nonce":2371778055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6815","nonce":2371778055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6813","nonce":2371778055}]},"public_addr":"172.21.15.204:6809/2371778055","cluster_addr":"172.21.15.204:6811/2371778055","heartbeat_back_addr":"172.21.15.204:6815/2371778055","heartbeat_front_addr":"172.21.15.204:6813/2371778055","state":["exists","up"]},{"osd":6,"uuid":"93d40a03-e7ff-4a3f-9efa-5c5403c9983d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6819","nonce":1484325044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6821","nonce":1484325044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6825","nonce":1484325044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6823","nonce":1484325044}]},"public_addr":"172.21.15.114:6819/1484325044","cluster_addr":"172.21.15.114:6821/1484325044","heartbeat_back_addr":"172.21.15.114:6825/1484325044","heartbeat_front_addr":"172.21.15.114:6823/1484325044","state":["exists","up"]},{"osd":7,"uuid":"982f629a-1b95-4be1-8e75-5ddce1ad77b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6817","nonce":799246614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6819","nonce":799246614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6823","nonce":799246614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6821","nonce":799246614}]},"public_addr":"172.21.15.134:6817/799246614","cluster_addr":"172.21.15.134:6819/799246614","heartbeat_back_addr":"172.21.15.134:6823/799246614","heartbeat_front_addr":"172.21.15.134:6821/799246614","state":["exists","up"]},{"osd":8,"uuid":"409beb9b-8227-4dad-a4a0-f0e5ce70dc5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6817","nonce":858331805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6819","nonce":858331805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6823","nonce":858331805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6821","nonce":858331805}]},"public_addr":"172.21.15.204:6817/858331805","cluster_addr":"172.21.15.204:6819/858331805","heartbeat_back_addr":"172.21.15.204:6823/858331805","heartbeat_front_addr":"172.21.15.204:6821/858331805","state":["exists","up"]},{"osd":9,"uuid":"a345a15c-b1fc-4877-b445-418eebf0d0ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6827","nonce":2172392854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6829","nonce":2172392854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6833","nonce":2172392854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6831","nonce":2172392854}]},"public_addr":"172.21.15.114:6827/2172392854","cluster_addr":"172.21.15.114:6829/2172392854","heartbeat_back_addr":"172.21.15.114:6833/2172392854","heartbeat_front_addr":"172.21.15.114:6831/2172392854","state":["exists","up"]},{"osd":10,"uuid":"464d995f-d90b-4caa-addb-1f7562bd74b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6825","nonce":3073513468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6827","nonce":3073513468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6831","nonce":3073513468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6829","nonce":3073513468}]},"public_addr":"172.21.15.134:6825/3073513468","cluster_addr":"172.21.15.134:6827/3073513468","heartbeat_back_addr":"172.21.15.134:6831/3073513468","heartbeat_front_addr":"172.21.15.134:6829/3073513468","state":["exists","up"]},{"osd":11,"uuid":"71dd794d-326c-477f-8d02-ee32bff1350d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6825","nonce":3239521765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6827","nonce":3239521765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6831","nonce":3239521765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6829","nonce":3239521765}]},"public_addr":"172.21.15.204:6825/3239521765","cluster_addr":"172.21.15.204:6827/3239521765","heartbeat_back_addr":"172.21.15.204:6831/3239521765","heartbeat_front_addr":"172.21.15.204:6829/3239521765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:14:54.542687+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:15:15.746055+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:15:38.371131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:01.383932+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:24.796237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:48.422214+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:11.284323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:34.570365+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:58.232855+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:18:21.609596+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:18:45.565859+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:19:10.017969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:6801/4281866858":"2024-09-14T18:13:25.823027+0000","172.21.15.114:6800/4281866858":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2949422849":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2034742251":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/722571999":"2024-09-14T18:13:03.521116+0000","172.21.15.114:6801/830738794":"2024-09-14T18:13:03.521116+0000","172.21.15.114:6800/830738794":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/2828364892":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/4257828906":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/3544951322":"2024-09-14T18:13:03.521116+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:20:07.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.974+0000 7f87daede640 1 -- 172.21.15.114:0/3220170609 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f87c40a1110 msgr2=0x7f87c40a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:07.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.974+0000 7f87daede640 1 --2- 172.21.15.114:0/3220170609 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f87c40a1110 0x7f87c40a35d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f87d4101c30 tx=0x7f87c0005a20 comp rx=0 tx=0).stop 2024-09-13T18:20:07.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.974+0000 7f87daede640 1 -- 172.21.15.114:0/3220170609 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87d40fe440 msgr2=0x7f87d4101670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:07.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.974+0000 7f87daede640 1 --2- 172.21.15.114:0/3220170609 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87d40fe440 0x7f87d4101670 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f87c800c970 tx=0x7f87c800ce30 comp rx=0 tx=0).stop 2024-09-13T18:20:07.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.974+0000 7f87daede640 1 -- 172.21.15.114:0/3220170609 shutdown_connections 2024-09-13T18:20:07.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.974+0000 7f87daede640 1 --2- 172.21.15.114:0/3220170609 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f87c40a1110 0x7f87c40a35d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:07.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.974+0000 7f87daede640 1 --2- 172.21.15.114:0/3220170609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87d40ffff0 0x7f87d419d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:07.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.974+0000 7f87daede640 1 --2- 172.21.15.114:0/3220170609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87d40ff630 0x7f87d419aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:07.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.974+0000 7f87daede640 1 --2- 172.21.15.114:0/3220170609 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87d40fe440 0x7f87d4101670 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:07.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.974+0000 7f87daede640 1 -- 172.21.15.114:0/3220170609 >> 172.21.15.114:0/3220170609 conn(0x7f87d40f9bc0 msgr2=0x7f87d40fb710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:07.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.974+0000 7f87daede640 1 -- 172.21.15.114:0/3220170609 shutdown_connections 2024-09-13T18:20:07.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:07.974+0000 7f87daede640 1 -- 172.21.15.114:0/3220170609 wait complete. 2024-09-13T18:20:07.988 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T18:20:01.895117+0000', 'modified': '2024-09-13T18:20:06.455916+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24521}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24521': {'gid': 24521, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.134:6839/3466309724', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6838', 'nonce': 3466309724}, {'type': 'v1', 'addr': '172.21.15.134:6839', 'nonce': 3466309724}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2024-09-13T18:20:07.988 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-13T18:20:07.988 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-13T18:20:08.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.087+0000 7f5c51dae640 1 -- 172.21.15.114:0/4156732111 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5c4c100830 msgr2=0x7f5c4c1070c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:08.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.087+0000 7f5c51dae640 1 --2- 172.21.15.114:0/4156732111 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5c4c100830 0x7f5c4c1070c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5c44009ca0 tx=0x7f5c4402a8c0 comp rx=0 tx=0).stop 2024-09-13T18:20:08.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.087+0000 7f5c51dae640 1 -- 172.21.15.114:0/4156732111 shutdown_connections 2024-09-13T18:20:08.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.087+0000 7f5c51dae640 1 --2- 172.21.15.114:0/4156732111 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5c4c100830 0x7f5c4c1070c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:08.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.087+0000 7f5c51dae640 1 --2- 172.21.15.114:0/4156732111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c4c0ffe70 0x7f5c4c1002f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:08.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.087+0000 7f5c51dae640 1 --2- 172.21.15.114:0/4156732111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c4c0fec80 0x7f5c4c0ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:08.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.087+0000 7f5c51dae640 1 -- 172.21.15.114:0/4156732111 >> 172.21.15.114:0/4156732111 conn(0x7f5c4c0fa3e0 msgr2=0x7f5c4c0fc820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:08.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.087+0000 7f5c51dae640 1 -- 172.21.15.114:0/4156732111 shutdown_connections 2024-09-13T18:20:08.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.087+0000 7f5c51dae640 1 -- 172.21.15.114:0/4156732111 wait complete. 2024-09-13T18:20:08.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.087+0000 7f5c51dae640 1 Processor -- start 2024-09-13T18:20:08.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.088+0000 7f5c51dae640 1 -- start start 2024-09-13T18:20:08.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.088+0000 7f5c51dae640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c4c0fec80 0x7f5c4c195c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:08.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.088+0000 7f5c51dae640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c4c0ffe70 0x7f5c4c196190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:08.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.088+0000 7f5c51dae640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5c4c100830 0x7f5c4c19d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:08.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.088+0000 7f5c51dae640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4c19d730 con 0x7f5c4c100830 2024-09-13T18:20:08.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.088+0000 7f5c51dae640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4c19d8a0 con 0x7f5c4c0ffe70 2024-09-13T18:20:08.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.088+0000 7f5c51dae640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4c19da60 con 0x7f5c4c0fec80 2024-09-13T18:20:08.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.088+0000 7f5c4bfff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5c4c100830 0x7f5c4c19d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:08.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.088+0000 7f5c4bfff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5c4c100830 0x7f5c4c19d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44922/0 (socket says 172.21.15.114:44922) 2024-09-13T18:20:08.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.088+0000 7f5c4bfff640 1 -- 172.21.15.114:0/1597254646 learned_addr learned my addr 172.21.15.114:0/1597254646 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:20:08.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.088+0000 7f5c4b7fe640 1 --2- 172.21.15.114:0/1597254646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c4c0fec80 0x7f5c4c195c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:08.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.088+0000 7f5c4affd640 1 --2- 172.21.15.114:0/1597254646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c4c0ffe70 0x7f5c4c196190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:08.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.088+0000 7f5c4bfff640 1 -- 172.21.15.114:0/1597254646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c4c0fec80 msgr2=0x7f5c4c195c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:08.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.088+0000 7f5c4bfff640 1 --2- 172.21.15.114:0/1597254646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c4c0fec80 0x7f5c4c195c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:08.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.088+0000 7f5c4bfff640 1 -- 172.21.15.114:0/1597254646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c4c0ffe70 msgr2=0x7f5c4c196190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:08.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.088+0000 7f5c4bfff640 1 --2- 172.21.15.114:0/1597254646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c4c0ffe70 0x7f5c4c196190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:08.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.088+0000 7f5c4bfff640 1 -- 172.21.15.114:0/1597254646 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c44009900 con 0x7f5c4c100830 2024-09-13T18:20:08.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.088+0000 7f5c4affd640 1 --2- 172.21.15.114:0/1597254646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c4c0ffe70 0x7f5c4c196190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:20:08.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.088+0000 7f5c4bfff640 1 --2- 172.21.15.114:0/1597254646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5c4c100830 0x7f5c4c19d1c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5c4402aeb0 tx=0x7f5c44003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:08.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.089+0000 7f5c48ff9640 1 -- 172.21.15.114:0/1597254646 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c440307c0 con 0x7f5c4c100830 2024-09-13T18:20:08.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.089+0000 7f5c48ff9640 1 -- 172.21.15.114:0/1597254646 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5c44040070 con 0x7f5c4c100830 2024-09-13T18:20:08.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.089+0000 7f5c51dae640 1 -- 172.21.15.114:0/1597254646 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c4c19dce0 con 0x7f5c4c100830 2024-09-13T18:20:08.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.089+0000 7f5c48ff9640 1 -- 172.21.15.114:0/1597254646 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c440043c0 con 0x7f5c4c100830 2024-09-13T18:20:08.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.089+0000 7f5c51dae640 1 -- 172.21.15.114:0/1597254646 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c4c19e1a0 con 0x7f5c4c100830 2024-09-13T18:20:08.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.091+0000 7f5c48ff9640 1 -- 172.21.15.114:0/1597254646 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f5c44042030 con 0x7f5c4c100830 2024-09-13T18:20:08.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.092+0000 7f5c54ad0740 1 Processor -- start 2024-09-13T18:20:08.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.092+0000 7f5c54ad0740 1 -- start start 2024-09-13T18:20:08.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.092+0000 7f5c54ad0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56114a1ae500 0x56114a1ae900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:08.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.092+0000 7f5c54ad0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56114a1aee40 0x56114a1b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:08.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.092+0000 7f5c54ad0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56114a1b3ed0 0x56114a1b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:08.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.092+0000 7f5c54ad0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114a1097c0 con 0x56114a1b3ed0 2024-09-13T18:20:08.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.092+0000 7f5c54ad0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114a118b70 con 0x56114a1ae500 2024-09-13T18:20:08.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.092+0000 7f5c54ad0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114a116300 con 0x56114a1aee40 2024-09-13T18:20:08.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.092+0000 7f5c4affd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56114a1b3ed0 0x56114a1b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:08.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.092+0000 7f5c4affd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56114a1b3ed0 0x56114a1b6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44926/0 (socket says 172.21.15.114:44926) 2024-09-13T18:20:08.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.092+0000 7f5c4affd640 1 -- 172.21.15.114:0/288113655 learned_addr learned my addr 172.21.15.114:0/288113655 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:20:08.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.092+0000 7f5c4b7fe640 1 --2- 172.21.15.114:0/288113655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56114a1aee40 0x56114a1b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:08.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.092+0000 7f5c4affd640 1 --2- 172.21.15.114:0/288113655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56114a1ae500 0x56114a1ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:08.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.092+0000 7f5c48ff9640 1 --2- 172.21.15.114:0/1597254646 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f5c380a1090 0x7f5c380a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:08.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.092+0000 7f5c4affd640 1 -- 172.21.15.114:0/288113655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56114a1aee40 msgr2=0x56114a1b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:20:08.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.092+0000 7f5c4affd640 1 --2- 172.21.15.114:0/288113655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56114a1aee40 0x56114a1b3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:08.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.092+0000 7f5c4affd640 1 -- 172.21.15.114:0/288113655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56114a1ae500 msgr2=0x56114a1ae900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:20:08.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.092+0000 7f5c4affd640 1 --2- 172.21.15.114:0/288113655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56114a1ae500 0x56114a1ae900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:08.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.092+0000 7f5c4bfff640 1 --2- 172.21.15.114:0/1597254646 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f5c380a1090 0x7f5c380a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:08.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.092+0000 7f5c4affd640 1 -- 172.21.15.114:0/288113655 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56114a09e4b0 con 0x56114a1b3ed0 2024-09-13T18:20:08.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.092+0000 7f5c4b7fe640 1 --2- 172.21.15.114:0/288113655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56114a1aee40 0x56114a1b3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:20:08.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.092+0000 7f5c48ff9640 1 -- 172.21.15.114:0/1597254646 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f5c440ee1b0 con 0x7f5c4c100830 2024-09-13T18:20:08.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.092+0000 7f5c4affd640 1 --2- 172.21.15.114:0/288113655 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56114a1b3ed0 0x56114a1b6390 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5c34014a80 tx=0x7f5c3400c5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3da2fc7e38527a97 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:08.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.093+0000 7f5c41ffb640 1 -- 172.21.15.114:0/288113655 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c340142d0 con 0x56114a1b3ed0 2024-09-13T18:20:08.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.093+0000 7f5c41ffb640 1 -- 172.21.15.114:0/288113655 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5c3401a020 con 0x56114a1b3ed0 2024-09-13T18:20:08.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.093+0000 7f5c41ffb640 1 -- 172.21.15.114:0/288113655 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3400feb0 con 0x56114a1b3ed0 2024-09-13T18:20:08.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.093+0000 7f5c4bfff640 1 --2- 172.21.15.114:0/1597254646 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f5c380a1090 0x7f5c380a3550 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5c44006010 tx=0x7f5c440db040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:08.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.093+0000 7f5c54ad0740 1 -- 172.21.15.114:0/288113655 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56114a1b3ed0 msgr2=0x56114a1b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:08.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.093+0000 7f5c54ad0740 1 --2- 172.21.15.114:0/288113655 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56114a1b3ed0 0x56114a1b6390 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5c34014a80 tx=0x7f5c3400c5a0 comp rx=0 tx=0).stop 2024-09-13T18:20:08.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.093+0000 7f5c54ad0740 1 -- 172.21.15.114:0/288113655 shutdown_connections 2024-09-13T18:20:08.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.093+0000 7f5c54ad0740 1 --2- 172.21.15.114:0/288113655 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56114a1b3ed0 0x56114a1b6390 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:08.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.093+0000 7f5c54ad0740 1 --2- 172.21.15.114:0/288113655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56114a1aee40 0x56114a1b3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:08.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.093+0000 7f5c54ad0740 1 --2- 172.21.15.114:0/288113655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56114a1ae500 0x56114a1ae900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:08.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.093+0000 7f5c54ad0740 1 -- 172.21.15.114:0/288113655 >> 172.21.15.114:0/288113655 conn(0x56114a1a9b70 msgr2=0x56114a1a9f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:08.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.093+0000 7f5c54ad0740 1 -- 172.21.15.114:0/288113655 shutdown_connections 2024-09-13T18:20:08.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.093+0000 7f5c54ad0740 1 -- 172.21.15.114:0/288113655 wait complete. 2024-09-13T18:20:08.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.094+0000 7f5c54ad0740 1 -- start start 2024-09-13T18:20:08.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.094+0000 7f5c54ad0740 1 Processor -- start 2024-09-13T18:20:08.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.094+0000 7f5c54ad0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56114a1ae500 0x56114a1b8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:08.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.094+0000 7f5c54ad0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56114a1aee40 0x56114a1b86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:08.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.094+0000 7f5c54ad0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56114a1b3ed0 0x56114a1bf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:08.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.094+0000 7f5c54ad0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114a111bf0 con 0x56114a1aee40 2024-09-13T18:20:08.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.094+0000 7f5c54ad0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114a0d0250 con 0x56114a1ae500 2024-09-13T18:20:08.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.094+0000 7f5c54ad0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114a0cfef0 con 0x56114a1b3ed0 2024-09-13T18:20:08.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.094+0000 7f5c54ad0740 10 client.0 _ll_register_callbacks cb 0x56114a102740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:20:08.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.094+0000 7f5c4b7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56114a1aee40 0x56114a1b86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:08.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.094+0000 7f5c4b7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56114a1aee40 0x56114a1b86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44940/0 (socket says 172.21.15.114:44940) 2024-09-13T18:20:08.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.094+0000 7f5c4b7fe640 1 -- 172.21.15.114:0/2871265170 learned_addr learned my addr 172.21.15.114:0/2871265170 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:20:08.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.094+0000 7f5c4affd640 1 --2- 172.21.15.114:0/2871265170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56114a1b3ed0 0x56114a1bf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:08.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.094+0000 7f5c4affd640 1 --2- 172.21.15.114:0/2871265170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56114a1ae500 0x56114a1b8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:08.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.094+0000 7f5c4b7fe640 1 -- 172.21.15.114:0/2871265170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56114a1b3ed0 msgr2=0x56114a1bf6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:20:08.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.094+0000 7f5c4b7fe640 1 --2- 172.21.15.114:0/2871265170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56114a1b3ed0 0x56114a1bf6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:08.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.094+0000 7f5c4b7fe640 1 -- 172.21.15.114:0/2871265170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56114a1ae500 msgr2=0x56114a1b8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:20:08.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.094+0000 7f5c4b7fe640 1 --2- 172.21.15.114:0/2871265170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56114a1ae500 0x56114a1b8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:08.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.094+0000 7f5c4b7fe640 1 -- 172.21.15.114:0/2871265170 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c34014730 con 0x56114a1aee40 2024-09-13T18:20:08.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.094+0000 7f5c4affd640 1 --2- 172.21.15.114:0/2871265170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56114a1ae500 0x56114a1b8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:20:08.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.094+0000 7f5c4b7fe640 1 --2- 172.21.15.114:0/2871265170 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56114a1aee40 0x56114a1b86b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c0126e0 tx=0x7f5c3c012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bed00bb49e23d23d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:08.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.094+0000 7f5c2affd640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:20:08.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.095+0000 7f5c2affd640 1 -- 172.21.15.114:0/2871265170 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c017070 con 0x56114a1aee40 2024-09-13T18:20:08.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.095+0000 7f5c2affd640 1 -- 172.21.15.114:0/2871265170 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f5c3c004030 con 0x56114a1aee40 2024-09-13T18:20:08.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.095+0000 7f5c54ad0740 1 -- 172.21.15.114:0/2871265170 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561149e053c0 con 0x56114a1aee40 2024-09-13T18:20:08.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.095+0000 7f5c2affd640 1 -- 172.21.15.114:0/2871265170 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c3c0041f0 con 0x56114a1aee40 2024-09-13T18:20:08.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.095+0000 7f5c54ad0740 10 client.14751 fetch_fsmap learned FSMap version 17 2024-09-13T18:20:08.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.095+0000 7f5c54ad0740 1 -- 172.21.15.114:0/2871265170 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x56114a110270 con 0x56114a1aee40 2024-09-13T18:20:08.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.095+0000 7f5c2affd640 1 -- 172.21.15.114:0/2871265170 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5c3c027080 con 0x56114a1aee40 2024-09-13T18:20:08.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.095+0000 7f5c2affd640 1 -- 172.21.15.114:0/2871265170 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 17) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f5c3c0037b0 con 0x56114a1aee40 2024-09-13T18:20:08.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.095+0000 7f5c54ad0740 10 client.14751 fetch_fsmap finished waiting for FSMap version 17 2024-09-13T18:20:08.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.095+0000 7f5c54ad0740 10 client.14751 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-13T18:20:08.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.095+0000 7f5c54ad0740 20 client.14751 populate_metadata read hostname 'smithi114' 2024-09-13T18:20:08.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.095+0000 7f5c54ad0740 1 --2- 172.21.15.114:0/2871265170 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56114a1c4720 0x56114a1c88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:08.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.095+0000 7f5c54ad0740 4 client.14751 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:20:08.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.095+0000 7f5c54ad0740 1 -- 172.21.15.114:0/2871265170 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561149e166c0 con 0x56114a1c4720 2024-09-13T18:20:08.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.095+0000 7f5c4affd640 1 --2- 172.21.15.114:0/2871265170 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56114a1c4720 0x56114a1c88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:08.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.096+0000 7f5c2affd640 10 client.14751 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:20:08.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.096+0000 7f5c4affd640 1 --2- 172.21.15.114:0/2871265170 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56114a1c4720 0x56114a1c88a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e84e1b211e85887 server_cookie=70b4b8b40ec33be2 in_seq=0 out_seq=0 2024-09-13T18:20:08.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.097+0000 7f5c2affd640 1 -- 172.21.15.114:0/2871265170 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x561149e166c0 con 0x56114a1c4720 2024-09-13T18:20:08.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.097+0000 7f5c2affd640 10 client.14751 handle_command_reply: tid=0 2024-09-13T18:20:08.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.097+0000 7f5c54ad0740 1 client.14751 shutdown 2024-09-13T18:20:08.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.097+0000 7f5c54ad0740 1 -- 172.21.15.114:0/2871265170 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56114a1aee40 msgr2=0x56114a1b86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:08.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.097+0000 7f5c54ad0740 1 --2- 172.21.15.114:0/2871265170 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56114a1aee40 0x56114a1b86b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c0126e0 tx=0x7f5c3c012ba0 comp rx=0 tx=0).stop 2024-09-13T18:20:08.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.097+0000 7f5c54ad0740 1 -- 172.21.15.114:0/2871265170 shutdown_connections 2024-09-13T18:20:08.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.097+0000 7f5c54ad0740 1 --2- 172.21.15.114:0/2871265170 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56114a1c4720 0x56114a1c88a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:08.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.097+0000 7f5c54ad0740 1 --2- 172.21.15.114:0/2871265170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56114a1b3ed0 0x56114a1bf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:08.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.097+0000 7f5c54ad0740 1 --2- 172.21.15.114:0/2871265170 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56114a1aee40 0x56114a1b86b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:08.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.097+0000 7f5c2affd640 0 client.14751 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:20:08.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.097+0000 7f5c54ad0740 1 --2- 172.21.15.114:0/2871265170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56114a1ae500 0x56114a1b8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:08.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.097+0000 7f5c54ad0740 1 -- 172.21.15.114:0/2871265170 >> 172.21.15.114:0/2871265170 conn(0x56114a1a9b70 msgr2=0x56114a1ab380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:08.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.097+0000 7f5c54ad0740 1 -- 172.21.15.114:0/2871265170 shutdown_connections 2024-09-13T18:20:08.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.097+0000 7f5c54ad0740 1 -- 172.21.15.114:0/2871265170 wait complete. 2024-09-13T18:20:08.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.097+0000 7f5c54ad0740 20 client.14751 trim_cache size 0 max 16384 2024-09-13T18:20:08.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.111+0000 7f5c54ad0740 1 Processor -- start 2024-09-13T18:20:08.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.111+0000 7f5c54ad0740 1 -- start start 2024-09-13T18:20:08.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.111+0000 7f5c54ad0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56114a1ae500 0x56114a1b8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:08.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.111+0000 7f5c54ad0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56114a1aee40 0x56114a1bf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:08.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.111+0000 7f5c54ad0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56114a1b3ed0 0x56114a1b86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:08.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.111+0000 7f5c54ad0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114a095460 con 0x56114a1b3ed0 2024-09-13T18:20:08.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.111+0000 7f5c54ad0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561149e166c0 con 0x56114a1aee40 2024-09-13T18:20:08.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.111+0000 7f5c54ad0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114a10e850 con 0x56114a1ae500 2024-09-13T18:20:08.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.112+0000 7f5c4affd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56114a1b3ed0 0x56114a1b86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:08.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.112+0000 7f5c4affd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56114a1b3ed0 0x56114a1b86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44948/0 (socket says 172.21.15.114:44948) 2024-09-13T18:20:08.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.112+0000 7f5c4affd640 1 -- 172.21.15.114:0/2950677343 learned_addr learned my addr 172.21.15.114:0/2950677343 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:20:08.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.112+0000 7f5c4affd640 1 --2- 172.21.15.114:0/2950677343 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56114a1ae500 0x56114a1b8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:08.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.112+0000 7f5c4b7fe640 1 --2- 172.21.15.114:0/2950677343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56114a1aee40 0x56114a1bf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:08.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.112+0000 7f5c4affd640 1 -- 172.21.15.114:0/2950677343 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56114a1ae500 msgr2=0x56114a1b8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:20:08.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.112+0000 7f5c4affd640 1 --2- 172.21.15.114:0/2950677343 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56114a1ae500 0x56114a1b8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:08.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.112+0000 7f5c4affd640 1 -- 172.21.15.114:0/2950677343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56114a1aee40 msgr2=0x56114a1bf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:20:08.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.112+0000 7f5c4affd640 1 --2- 172.21.15.114:0/2950677343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56114a1aee40 0x56114a1bf6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:08.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.112+0000 7f5c4affd640 1 -- 172.21.15.114:0/2950677343 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56114a0b25f0 con 0x56114a1b3ed0 2024-09-13T18:20:08.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.112+0000 7f5c4b7fe640 1 --2- 172.21.15.114:0/2950677343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56114a1aee40 0x56114a1bf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:20:08.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.112+0000 7f5c4affd640 1 --2- 172.21.15.114:0/2950677343 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56114a1b3ed0 0x56114a1b86b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5c3400d010 tx=0x7f5c34016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dcf7e54cba270fd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:08.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.112+0000 7f5c427fc640 1 -- 172.21.15.114:0/2950677343 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c34028b70 con 0x56114a1b3ed0 2024-09-13T18:20:08.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.112+0000 7f5c427fc640 1 -- 172.21.15.114:0/2950677343 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5c34027070 con 0x56114a1b3ed0 2024-09-13T18:20:08.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.112+0000 7f5c427fc640 1 -- 172.21.15.114:0/2950677343 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c340281c0 con 0x56114a1b3ed0 2024-09-13T18:20:08.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.113+0000 7f5c54ad0740 1 -- 172.21.15.114:0/2950677343 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56114a1b3ed0 msgr2=0x56114a1b86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:08.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.113+0000 7f5c54ad0740 1 --2- 172.21.15.114:0/2950677343 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56114a1b3ed0 0x56114a1b86b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5c3400d010 tx=0x7f5c34016040 comp rx=0 tx=0).stop 2024-09-13T18:20:08.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.113+0000 7f5c54ad0740 1 -- 172.21.15.114:0/2950677343 shutdown_connections 2024-09-13T18:20:08.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.113+0000 7f5c54ad0740 1 --2- 172.21.15.114:0/2950677343 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56114a1b3ed0 0x56114a1b86b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:08.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.113+0000 7f5c54ad0740 1 --2- 172.21.15.114:0/2950677343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56114a1aee40 0x56114a1bf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:08.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.113+0000 7f5c54ad0740 1 --2- 172.21.15.114:0/2950677343 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56114a1ae500 0x56114a1b8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:08.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.113+0000 7f5c54ad0740 1 -- 172.21.15.114:0/2950677343 >> 172.21.15.114:0/2950677343 conn(0x56114a1a9b70 msgr2=0x56114a1afd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:08.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.113+0000 7f5c54ad0740 1 -- 172.21.15.114:0/2950677343 shutdown_connections 2024-09-13T18:20:08.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.113+0000 7f5c54ad0740 1 -- 172.21.15.114:0/2950677343 wait complete. 2024-09-13T18:20:08.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.113+0000 7f5c54ad0740 1 -- start start 2024-09-13T18:20:08.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.113+0000 7f5c54ad0740 1 Processor -- start 2024-09-13T18:20:08.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.114+0000 7f5c54ad0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56114a1ae500 0x56114a1ba160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:08.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.114+0000 7f5c54ad0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56114a1aee40 0x56114a1bf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:08.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.114+0000 7f5c54ad0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56114a1b3ed0 0x56114a1ba6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:08.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.114+0000 7f5c54ad0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114a1bada0 con 0x56114a1ae500 2024-09-13T18:20:08.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.114+0000 7f5c54ad0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114a21a560 con 0x56114a1b3ed0 2024-09-13T18:20:08.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.114+0000 7f5c54ad0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114a21a720 con 0x56114a1aee40 2024-09-13T18:20:08.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.114+0000 7f5c54ad0740 10 client.0 _ll_register_callbacks cb 0x7f5c3400cb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:20:08.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.114+0000 7f5c4affd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56114a1ae500 0x56114a1ba160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:08.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.114+0000 7f5c4b7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56114a1aee40 0x56114a1bf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:08.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.114+0000 7f5c4affd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56114a1ae500 0x56114a1ba160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44952/0 (socket says 172.21.15.114:44952) 2024-09-13T18:20:08.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.114+0000 7f5c4affd640 1 -- 172.21.15.114:0/1970246758 learned_addr learned my addr 172.21.15.114:0/1970246758 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:20:08.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.114+0000 7f5c4affd640 1 --2- 172.21.15.114:0/1970246758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56114a1b3ed0 0x56114a1ba6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:08.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.114+0000 7f5c4affd640 1 -- 172.21.15.114:0/1970246758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56114a1aee40 msgr2=0x56114a1bf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:20:08.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.114+0000 7f5c4affd640 1 --2- 172.21.15.114:0/1970246758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56114a1aee40 0x56114a1bf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:08.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.114+0000 7f5c4affd640 1 -- 172.21.15.114:0/1970246758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56114a1b3ed0 msgr2=0x56114a1ba6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:20:08.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.114+0000 7f5c4affd640 1 --2- 172.21.15.114:0/1970246758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56114a1b3ed0 0x56114a1ba6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:08.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.114+0000 7f5c4affd640 1 -- 172.21.15.114:0/1970246758 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56114a09e4b0 con 0x56114a1ae500 2024-09-13T18:20:08.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.114+0000 7f5c4affd640 1 --2- 172.21.15.114:0/1970246758 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56114a1ae500 0x56114a1ba160 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5c3402e040 tx=0x7f5c3400cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a9cc2bac8f70c8a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:08.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.114+0000 7f5c2affd640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:20:08.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.115+0000 7f5c2affd640 1 -- 172.21.15.114:0/1970246758 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c340142e0 con 0x56114a1ae500 2024-09-13T18:20:08.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.115+0000 7f5c2affd640 1 -- 172.21.15.114:0/1970246758 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f5c34000e40 con 0x56114a1ae500 2024-09-13T18:20:08.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.115+0000 7f5c54ad0740 1 -- 172.21.15.114:0/1970246758 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5c3401a020 con 0x56114a1ae500 2024-09-13T18:20:08.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.115+0000 7f5c2affd640 1 -- 172.21.15.114:0/1970246758 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c3400f9c0 con 0x56114a1ae500 2024-09-13T18:20:08.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.115+0000 7f5c54ad0740 10 client.14757 fetch_fsmap learned FSMap version 17 2024-09-13T18:20:08.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.115+0000 7f5c54ad0740 1 -- 172.21.15.114:0/1970246758 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x56114a0d7d00 con 0x56114a1ae500 2024-09-13T18:20:08.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.115+0000 7f5c2affd640 1 -- 172.21.15.114:0/1970246758 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5c3401a020 con 0x56114a1ae500 2024-09-13T18:20:08.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.116+0000 7f5c2affd640 1 -- 172.21.15.114:0/1970246758 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 17) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f5c34004c80 con 0x56114a1ae500 2024-09-13T18:20:08.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.116+0000 7f5c54ad0740 10 client.14757 fetch_fsmap finished waiting for FSMap version 17 2024-09-13T18:20:08.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.116+0000 7f5c54ad0740 10 client.14757 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-13T18:20:08.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.116+0000 7f5c54ad0740 20 client.14757 populate_metadata read hostname 'smithi114' 2024-09-13T18:20:08.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.116+0000 7f5c54ad0740 1 --2- 172.21.15.114:0/1970246758 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56114a21f120 0x56114a21bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:08.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.116+0000 7f5c54ad0740 4 client.14757 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T18:20:08.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.116+0000 7f5c54ad0740 1 -- 172.21.15.114:0/1970246758 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561149e69d10 con 0x56114a21f120 2024-09-13T18:20:08.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.116+0000 7f5c4b7fe640 1 --2- 172.21.15.114:0/1970246758 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56114a21f120 0x56114a21bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:08.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.116+0000 7f5c4b7fe640 1 --2- 172.21.15.114:0/1970246758 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56114a21f120 0x56114a21bab0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8a0567daa9ee91e server_cookie=c4e9f5c7a4697ca4 in_seq=0 out_seq=0 2024-09-13T18:20:08.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.116+0000 7f5c2affd640 10 client.14757 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:20:08.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.116+0000 7f5c2affd640 1 -- 172.21.15.114:0/1970246758 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x561149e69d10 con 0x56114a21f120 2024-09-13T18:20:08.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.116+0000 7f5c2affd640 10 client.14757 handle_command_reply: tid=0 2024-09-13T18:20:08.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.117+0000 7f5c54ad0740 1 client.14757 shutdown 2024-09-13T18:20:08.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.117+0000 7f5c54ad0740 1 -- 172.21.15.114:0/1970246758 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56114a1ae500 msgr2=0x56114a1ba160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:08.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.117+0000 7f5c54ad0740 1 --2- 172.21.15.114:0/1970246758 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56114a1ae500 0x56114a1ba160 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5c3402e040 tx=0x7f5c3400cd90 comp rx=0 tx=0).stop 2024-09-13T18:20:08.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.117+0000 7f5c54ad0740 1 -- 172.21.15.114:0/1970246758 shutdown_connections 2024-09-13T18:20:08.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.117+0000 7f5c54ad0740 1 --2- 172.21.15.114:0/1970246758 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56114a21f120 0x56114a21bab0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:08.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.117+0000 7f5c54ad0740 1 --2- 172.21.15.114:0/1970246758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56114a1b3ed0 0x56114a1ba6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:08.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.117+0000 7f5c54ad0740 1 --2- 172.21.15.114:0/1970246758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56114a1aee40 0x56114a1bf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:08.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.117+0000 7f5c2affd640 0 client.14757 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:20:08.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.117+0000 7f5c54ad0740 1 --2- 172.21.15.114:0/1970246758 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56114a1ae500 0x56114a1ba160 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:08.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.117+0000 7f5c54ad0740 1 -- 172.21.15.114:0/1970246758 >> 172.21.15.114:0/1970246758 conn(0x56114a1a9b70 msgr2=0x56114a1cc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:08.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.117+0000 7f5c54ad0740 1 -- 172.21.15.114:0/1970246758 shutdown_connections 2024-09-13T18:20:08.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.117+0000 7f5c54ad0740 1 -- 172.21.15.114:0/1970246758 wait complete. 2024-09-13T18:20:08.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.117+0000 7f5c54ad0740 20 client.14757 trim_cache size 0 max 16384 2024-09-13T18:20:08.125 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:20:08.125 INFO:teuthology.orchestra.run.smithi114.stdout: "cluster_fsid": "97e7edf2-71fb-11ef-bceb-c7b262605968", 2024-09-13T18:20:08.125 INFO:teuthology.orchestra.run.smithi114.stdout: "whoami": 0, 2024-09-13T18:20:08.125 INFO:teuthology.orchestra.run.smithi114.stdout: "id": 24521, 2024-09-13T18:20:08.125 INFO:teuthology.orchestra.run.smithi114.stdout: "want_state": "up:active", 2024-09-13T18:20:08.125 INFO:teuthology.orchestra.run.smithi114.stdout: "state": "up:active", 2024-09-13T18:20:08.125 INFO:teuthology.orchestra.run.smithi114.stdout: "fs_name": "cephfs", 2024-09-13T18:20:08.126 INFO:teuthology.orchestra.run.smithi114.stdout: "rank_uptime": 6.2189105619999996, 2024-09-13T18:20:08.126 INFO:teuthology.orchestra.run.smithi114.stdout: "mdsmap_epoch": 17, 2024-09-13T18:20:08.126 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch": 89, 2024-09-13T18:20:08.126 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch_barrier": 88, 2024-09-13T18:20:08.126 INFO:teuthology.orchestra.run.smithi114.stdout: "uptime": 11.849617957 2024-09-13T18:20:08.126 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:20:08.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.117+0000 7f5c51dae640 1 -- 172.21.15.114:0/1597254646 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f5c380a1090 msgr2=0x7f5c380a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:08.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.117+0000 7f5c51dae640 1 --2- 172.21.15.114:0/1597254646 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f5c380a1090 0x7f5c380a3550 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5c44006010 tx=0x7f5c440db040 comp rx=0 tx=0).stop 2024-09-13T18:20:08.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.117+0000 7f5c51dae640 1 -- 172.21.15.114:0/1597254646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5c4c100830 msgr2=0x7f5c4c19d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:08.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.117+0000 7f5c51dae640 1 --2- 172.21.15.114:0/1597254646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5c4c100830 0x7f5c4c19d1c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5c4402aeb0 tx=0x7f5c44003c20 comp rx=0 tx=0).stop 2024-09-13T18:20:08.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.117+0000 7f5c51dae640 1 -- 172.21.15.114:0/1597254646 shutdown_connections 2024-09-13T18:20:08.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.117+0000 7f5c51dae640 1 --2- 172.21.15.114:0/1597254646 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f5c380a1090 0x7f5c380a3550 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:08.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.117+0000 7f5c51dae640 1 --2- 172.21.15.114:0/1597254646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5c4c100830 0x7f5c4c19d1c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:08.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.117+0000 7f5c51dae640 1 --2- 172.21.15.114:0/1597254646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c4c0ffe70 0x7f5c4c196190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:08.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.117+0000 7f5c51dae640 1 --2- 172.21.15.114:0/1597254646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c4c0fec80 0x7f5c4c195c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:08.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.117+0000 7f5c51dae640 1 -- 172.21.15.114:0/1597254646 >> 172.21.15.114:0/1597254646 conn(0x7f5c4c0fa3e0 msgr2=0x7f5c4c0686a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:08.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.118+0000 7f5c51dae640 1 -- 172.21.15.114:0/1597254646 shutdown_connections 2024-09-13T18:20:08.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:08.118+0000 7f5c51dae640 1 -- 172.21.15.114:0/1597254646 wait complete. 2024-09-13T18:20:08.131 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-13T18:20:08.131 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-13T18:20:08.132 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph fs ls 2024-09-13T18:20:08.325 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:20:08.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:08 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/504068213' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:20:08.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:08 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/816150721' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:20:08.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:08 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/3220170609' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:20:08.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:08 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/504068213' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:20:08.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:08 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/816150721' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:20:08.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:08 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/3220170609' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:20:08.445 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:08 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/504068213' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:20:08.445 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:08 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/816150721' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:20:08.445 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:08 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/3220170609' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:20:09.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:09 smithi204 ceph-mon[34777]: pgmap v280: 129 pgs: 9 creating+peering, 107 active+clean, 13 unknown; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 8 op/s 2024-09-13T18:20:09.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:09 smithi204 ceph-mon[34777]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-13T18:20:09.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:09 smithi204 ceph-mon[34777]: Cluster is now healthy 2024-09-13T18:20:09.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:09 smithi114 ceph-mon[30897]: pgmap v280: 129 pgs: 9 creating+peering, 107 active+clean, 13 unknown; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 8 op/s 2024-09-13T18:20:09.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:09 smithi114 ceph-mon[30897]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-13T18:20:09.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:09 smithi114 ceph-mon[30897]: Cluster is now healthy 2024-09-13T18:20:09.621 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:09 smithi134 ceph-mon[34283]: pgmap v280: 129 pgs: 9 creating+peering, 107 active+clean, 13 unknown; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 8 op/s 2024-09-13T18:20:09.622 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:09 smithi134 ceph-mon[34283]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-13T18:20:09.622 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:09 smithi134 ceph-mon[34283]: Cluster is now healthy 2024-09-13T18:20:10.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:10.016+0000 7fa2fd025640 1 -- 172.21.15.114:0/323339452 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2f80fffc0 msgr2=0x7fa2f81003a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:10.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:10.016+0000 7fa2fd025640 1 --2- 172.21.15.114:0/323339452 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2f80fffc0 0x7fa2f81003a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec009930 tx=0x7fa2ec02a420 comp rx=0 tx=0).stop 2024-09-13T18:20:10.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:10.016+0000 7fa2fd025640 1 -- 172.21.15.114:0/323339452 shutdown_connections 2024-09-13T18:20:10.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:10.016+0000 7fa2fd025640 1 --2- 172.21.15.114:0/323339452 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2f81050f0 0x7fa2f810cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:10.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:10.016+0000 7fa2fd025640 1 --2- 172.21.15.114:0/323339452 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2f8100970 0x7fa2f81049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:10.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:10.016+0000 7fa2fd025640 1 --2- 172.21.15.114:0/323339452 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2f80fffc0 0x7fa2f81003a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:10.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:10.016+0000 7fa2fd025640 1 -- 172.21.15.114:0/323339452 >> 172.21.15.114:0/323339452 conn(0x7fa2f80fbaa0 msgr2=0x7fa2f80fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:10.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:10.016+0000 7fa2fd025640 1 -- 172.21.15.114:0/323339452 shutdown_connections 2024-09-13T18:20:10.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:10.016+0000 7fa2fd025640 1 -- 172.21.15.114:0/323339452 wait complete. 2024-09-13T18:20:10.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:10.016+0000 7fa2fd025640 1 Processor -- start 2024-09-13T18:20:10.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:10.017+0000 7fa2fd025640 1 -- start start 2024-09-13T18:20:10.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:10.017+0000 7fa2fd025640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2f80fffc0 0x7fa2f80727b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:10.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:10.017+0000 7fa2fd025640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2f8100970 0x7fa2f8072cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:10.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:10.017+0000 7fa2fd025640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2f81050f0 0x7fa2f8073230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:10.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:10.017+0000 7fa2fd025640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f81a4690 con 0x7fa2f81050f0 2024-09-13T18:20:10.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:10.017+0000 7fa2fd025640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f81a4800 con 0x7fa2f8100970 2024-09-13T18:20:10.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:10.017+0000 7fa2fd025640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f81a49c0 con 0x7fa2f80fffc0 2024-09-13T18:20:10.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:10.017+0000 7fa2f759e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2f81050f0 0x7fa2f8073230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:10.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:10.017+0000 7fa2f759e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2f81050f0 0x7fa2f8073230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44978/0 (socket says 172.21.15.114:44978) 2024-09-13T18:20:10.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:10.017+0000 7fa2f759e640 1 -- 172.21.15.114:0/2257478110 learned_addr learned my addr 172.21.15.114:0/2257478110 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:20:10.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:10.017+0000 7fa2f6d9d640 1 --2- 172.21.15.114:0/2257478110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2f80fffc0 0x7fa2f80727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:10.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:10.017+0000 7fa2f659c640 1 --2- 172.21.15.114:0/2257478110 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2f8100970 0x7fa2f8072cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:10.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:10.017+0000 7fa2f759e640 1 -- 172.21.15.114:0/2257478110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2f80fffc0 msgr2=0x7fa2f80727b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:10.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:10.017+0000 7fa2f759e640 1 --2- 172.21.15.114:0/2257478110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2f80fffc0 0x7fa2f80727b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:10.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:10.017+0000 7fa2f759e640 1 -- 172.21.15.114:0/2257478110 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2f8100970 msgr2=0x7fa2f8072cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:10.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:10.017+0000 7fa2f759e640 1 --2- 172.21.15.114:0/2257478110 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2f8100970 0x7fa2f8072cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:10.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:10.017+0000 7fa2f759e640 1 -- 172.21.15.114:0/2257478110 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2ec009560 con 0x7fa2f81050f0 2024-09-13T18:20:10.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:10.018+0000 7fa2f759e640 1 --2- 172.21.15.114:0/2257478110 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2f81050f0 0x7fa2f8073230 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0009fd0 tx=0x7fa2f000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:10.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:10.018+0000 7fa2dffff640 1 -- 172.21.15.114:0/2257478110 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0011830 con 0x7fa2f81050f0 2024-09-13T18:20:10.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:10.018+0000 7fa2fd025640 1 -- 172.21.15.114:0/2257478110 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2f81a4ca0 con 0x7fa2f81050f0 2024-09-13T18:20:10.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:10.018+0000 7fa2dffff640 1 -- 172.21.15.114:0/2257478110 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa2f0011e20 con 0x7fa2f81050f0 2024-09-13T18:20:10.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:10.018+0000 7fa2dffff640 1 -- 172.21.15.114:0/2257478110 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0010e50 con 0x7fa2f81050f0 2024-09-13T18:20:10.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:10.018+0000 7fa2fd025640 1 -- 172.21.15.114:0/2257478110 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2f81a5150 con 0x7fa2f81050f0 2024-09-13T18:20:10.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:10.019+0000 7fa2fd025640 1 -- 172.21.15.114:0/2257478110 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2f81014a0 con 0x7fa2f81050f0 2024-09-13T18:20:10.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:10.020+0000 7fa2dffff640 1 -- 172.21.15.114:0/2257478110 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fa2f0011990 con 0x7fa2f81050f0 2024-09-13T18:20:10.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:10.020+0000 7fa2dffff640 1 --2- 172.21.15.114:0/2257478110 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fa2e00a1080 0x7fa2e00a3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:10.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:10.020+0000 7fa2f6d9d640 1 --2- 172.21.15.114:0/2257478110 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fa2e00a1080 0x7fa2e00a3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:10.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:10.021+0000 7fa2dffff640 1 -- 172.21.15.114:0/2257478110 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fa2f00cc190 con 0x7fa2f81050f0 2024-09-13T18:20:10.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:10.021+0000 7fa2f6d9d640 1 --2- 172.21.15.114:0/2257478110 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fa2e00a1080 0x7fa2e00a3540 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec009900 tx=0x7fa2ec005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:10.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:10.025+0000 7fa2dffff640 1 -- 172.21.15.114:0/2257478110 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fa2f00957d0 con 0x7fa2f81050f0 2024-09-13T18:20:10.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:10.225+0000 7fa2fd025640 1 -- 172.21.15.114:0/2257478110 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa2f8073d30 con 0x7fa2f81050f0 2024-09-13T18:20:10.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:10.225+0000 7fa2dffff640 1 -- 172.21.15.114:0/2257478110 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+88 (secure 0 0 0) 0x7fa2f0098df0 con 0x7fa2f81050f0 2024-09-13T18:20:10.227 INFO:teuthology.orchestra.run.smithi114.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-13T18:20:10.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:10.227+0000 7fa2fd025640 1 -- 172.21.15.114:0/2257478110 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fa2e00a1080 msgr2=0x7fa2e00a3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:10.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:10.227+0000 7fa2fd025640 1 --2- 172.21.15.114:0/2257478110 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fa2e00a1080 0x7fa2e00a3540 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec009900 tx=0x7fa2ec005a20 comp rx=0 tx=0).stop 2024-09-13T18:20:10.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:10.227+0000 7fa2fd025640 1 -- 172.21.15.114:0/2257478110 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2f81050f0 msgr2=0x7fa2f8073230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:10.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:10.227+0000 7fa2fd025640 1 --2- 172.21.15.114:0/2257478110 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2f81050f0 0x7fa2f8073230 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0009fd0 tx=0x7fa2f000b8e0 comp rx=0 tx=0).stop 2024-09-13T18:20:10.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:10.227+0000 7fa2fd025640 1 -- 172.21.15.114:0/2257478110 shutdown_connections 2024-09-13T18:20:10.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:10.227+0000 7fa2fd025640 1 --2- 172.21.15.114:0/2257478110 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fa2e00a1080 0x7fa2e00a3540 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:10.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:10.227+0000 7fa2fd025640 1 --2- 172.21.15.114:0/2257478110 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2f81050f0 0x7fa2f8073230 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:10.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:10.227+0000 7fa2fd025640 1 --2- 172.21.15.114:0/2257478110 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2f8100970 0x7fa2f8072cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:10.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:10.227+0000 7fa2fd025640 1 --2- 172.21.15.114:0/2257478110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2f80fffc0 0x7fa2f80727b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:10.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:10.227+0000 7fa2fd025640 1 -- 172.21.15.114:0/2257478110 >> 172.21.15.114:0/2257478110 conn(0x7fa2f80fbaa0 msgr2=0x7fa2f80fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:10.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:10.227+0000 7fa2fd025640 1 -- 172.21.15.114:0/2257478110 shutdown_connections 2024-09-13T18:20:10.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:10.227+0000 7fa2fd025640 1 -- 172.21.15.114:0/2257478110 wait complete. 2024-09-13T18:20:10.965 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-13T18:20:10.965 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T18:20:10.966 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi114.front.sepia.ceph.com 2024-09-13T18:20:10.966 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-13T18:20:10.966 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-13T18:20:10.966 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-13T18:20:10.966 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-13T18:20:10.966 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T18:20:10.966 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-13T18:20:10.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:20:10.966 DEBUG:teuthology.orchestra.run.smithi114:> ip addr 2024-09-13T18:20:10.983 INFO:teuthology.orchestra.run.smithi114.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T18:20:10.983 INFO:teuthology.orchestra.run.smithi114.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T18:20:10.983 INFO:teuthology.orchestra.run.smithi114.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T18:20:10.983 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:20:10.983 INFO:teuthology.orchestra.run.smithi114.stdout: inet6 ::1/128 scope host 2024-09-13T18:20:10.983 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:20:10.984 INFO:teuthology.orchestra.run.smithi114.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-13T18:20:10.984 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether 0c:c4:7a:d8:a8:92 brd ff:ff:ff:ff:ff:ff 2024-09-13T18:20:10.984 INFO:teuthology.orchestra.run.smithi114.stdout: altname enp7s0f0 2024-09-13T18:20:10.984 INFO:teuthology.orchestra.run.smithi114.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-13T18:20:10.984 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether 0c:c4:7a:d8:a8:93 brd ff:ff:ff:ff:ff:ff 2024-09-13T18:20:10.984 INFO:teuthology.orchestra.run.smithi114.stdout: altname enp7s0f1 2024-09-13T18:20:10.984 INFO:teuthology.orchestra.run.smithi114.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-13T18:20:10.984 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether 0c:c4:7a:8f:cb:62 brd ff:ff:ff:ff:ff:ff 2024-09-13T18:20:10.984 INFO:teuthology.orchestra.run.smithi114.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T18:20:10.984 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether 0c:c4:7a:8f:cb:63 brd ff:ff:ff:ff:ff:ff 2024-09-13T18:20:10.984 INFO:teuthology.orchestra.run.smithi114.stdout: inet 172.21.15.114/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-13T18:20:10.984 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft 42299sec preferred_lft 42299sec 2024-09-13T18:20:10.984 INFO:teuthology.orchestra.run.smithi114.stdout: inet6 fe80::ec4:7aff:fe8f:cb63/64 scope link noprefixroute 2024-09-13T18:20:10.985 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:20:10.985 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-13T18:20:10.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:20:10.985 DEBUG:teuthology.orchestra.run.smithi114:> (cd / && exec bash -c ' 2024-09-13T18:20:10.986 DEBUG:teuthology.orchestra.run.smithi114:> set -e 2024-09-13T18:20:10.986 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link add name ceph-brx type bridge 2024-09-13T18:20:10.986 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip addr flush dev ceph-brx 2024-09-13T18:20:10.986 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link set ceph-brx up 2024-09-13T18:20:10.986 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-13T18:20:10.986 DEBUG:teuthology.orchestra.run.smithi114:> ') 2024-09-13T18:20:11.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:20:11.149 DEBUG:teuthology.orchestra.run.smithi114:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T18:20:11.174 INFO:teuthology.orchestra.run.smithi114.stdout:1 2024-09-13T18:20:11.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:20:11.176 DEBUG:teuthology.orchestra.run.smithi114:> route 2024-09-13T18:20:11.219 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:11 smithi114 ceph-mon[30897]: pgmap v281: 129 pgs: 9 creating+peering, 120 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-09-13T18:20:11.219 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:11 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2257478110' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-13T18:20:11.249 INFO:teuthology.orchestra.run.smithi114.stdout:Kernel IP routing table 2024-09-13T18:20:11.249 INFO:teuthology.orchestra.run.smithi114.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T18:20:11.250 INFO:teuthology.orchestra.run.smithi114.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-13T18:20:11.250 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-13T18:20:11.250 INFO:teuthology.orchestra.run.smithi114.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T18:20:11.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:20:11.250 DEBUG:teuthology.orchestra.run.smithi114:> (cd / && exec bash -c ' 2024-09-13T18:20:11.250 DEBUG:teuthology.orchestra.run.smithi114:> set -e 2024-09-13T18:20:11.250 DEBUG:teuthology.orchestra.run.smithi114:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-13T18:20:11.250 DEBUG:teuthology.orchestra.run.smithi114:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-13T18:20:11.251 DEBUG:teuthology.orchestra.run.smithi114:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-13T18:20:11.251 DEBUG:teuthology.orchestra.run.smithi114:> ') 2024-09-13T18:20:11.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:11 smithi204 ceph-mon[34777]: pgmap v281: 129 pgs: 9 creating+peering, 120 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-09-13T18:20:11.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:11 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/2257478110' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-13T18:20:11.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:20:11.505 DEBUG:teuthology.orchestra.run.smithi114:> ip netns list 2024-09-13T18:20:11.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:20:11.521 DEBUG:teuthology.orchestra.run.smithi114:> ip netns list-id 2024-09-13T18:20:11.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:20:11.577 DEBUG:teuthology.orchestra.run.smithi114:> (cd / && exec bash -c ' 2024-09-13T18:20:11.577 DEBUG:teuthology.orchestra.run.smithi114:> set -e 2024-09-13T18:20:11.577 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-13T18:20:11.577 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-13T18:20:11.577 DEBUG:teuthology.orchestra.run.smithi114:> ') 2024-09-13T18:20:11.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:11 smithi134 ceph-mon[34283]: pgmap v281: 129 pgs: 9 creating+peering, 120 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-09-13T18:20:11.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:11 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/2257478110' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-13T18:20:11.655 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-13T18:20:11.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:20:11.656 DEBUG:teuthology.orchestra.run.smithi114:> (cd / && exec bash -c ' 2024-09-13T18:20:11.656 DEBUG:teuthology.orchestra.run.smithi114:> set -e 2024-09-13T18:20:11.656 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-13T18:20:11.656 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-13T18:20:11.656 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-13T18:20:11.656 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-13T18:20:11.656 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-13T18:20:11.656 DEBUG:teuthology.orchestra.run.smithi114:> ') 2024-09-13T18:20:11.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:20:11.781 DEBUG:teuthology.orchestra.run.smithi114:> (cd / && exec bash -c ' 2024-09-13T18:20:11.781 DEBUG:teuthology.orchestra.run.smithi114:> set -e 2024-09-13T18:20:11.782 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link set brx.0 up 2024-09-13T18:20:11.782 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link set dev brx.0 master ceph-brx 2024-09-13T18:20:11.782 DEBUG:teuthology.orchestra.run.smithi114:> ') 2024-09-13T18:20:11.818 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-13T18:20:11.818 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T18:20:11.818 DEBUG:teuthology.orchestra.run.smithi114:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-13T18:20:11.875 INFO:teuthology.orchestra.run.smithi114.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-13T18:20:11.875 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T18:20:11.875 DEBUG:teuthology.orchestra.run.smithi114:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-13T18:20:11.930 DEBUG:teuthology.orchestra.run.smithi114:> sudo modprobe fuse 2024-09-13T18:20:11.995 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T18:20:11.995 DEBUG:teuthology.orchestra.run.smithi114:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T18:20:12.060 INFO:teuthology.orchestra.run.smithi114.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T18:20:12.062 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T18:20:12.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:20:12.062 DEBUG:teuthology.orchestra.run.smithi114:> ls /sys/fs/fuse/connections 2024-09-13T18:20:12.117 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-13T18:20:12.118 DEBUG:teuthology.orchestra.run.smithi114:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-13T18:20:12.160 DEBUG:teuthology.orchestra.run.smithi114:> sudo modprobe fuse 2024-09-13T18:20:12.184 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T18:20:12.184 DEBUG:teuthology.orchestra.run.smithi114:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T18:20:12.248 INFO:teuthology.orchestra.run.smithi114.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T18:20:12.249 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T18:20:12.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:20:12.250 DEBUG:teuthology.orchestra.run.smithi114:> ls /sys/fs/fuse/connections 2024-09-13T18:20:12.288 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi114.stderr:2024-09-13T18:20:12.286+0000 7fe04667c480 -1 init, newargv = 0x55caec4aa2a0 newargc=15 2024-09-13T18:20:12.289 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi114.stderr:ceph-fuse[73373]: starting ceph client 2024-09-13T18:20:12.294 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi114.stderr:ceph-fuse[73373]: starting fuse 2024-09-13T18:20:12.304 INFO:teuthology.orchestra.run.smithi114.stdout:120 2024-09-13T18:20:12.304 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [120] 2024-09-13T18:20:12.305 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T18:20:12.305 DEBUG:teuthology.orchestra.run.smithi114:> import glob 2024-09-13T18:20:12.305 DEBUG:teuthology.orchestra.run.smithi114:> import re 2024-09-13T18:20:12.305 DEBUG:teuthology.orchestra.run.smithi114:> import os 2024-09-13T18:20:12.305 DEBUG:teuthology.orchestra.run.smithi114:> import subprocess 2024-09-13T18:20:12.305 DEBUG:teuthology.orchestra.run.smithi114:> 2024-09-13T18:20:12.305 DEBUG:teuthology.orchestra.run.smithi114:> def _find_admin_socket(client_name): 2024-09-13T18:20:12.305 DEBUG:teuthology.orchestra.run.smithi114:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-13T18:20:12.306 DEBUG:teuthology.orchestra.run.smithi114:> files = glob.glob(asok_path) 2024-09-13T18:20:12.306 DEBUG:teuthology.orchestra.run.smithi114:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-13T18:20:12.306 DEBUG:teuthology.orchestra.run.smithi114:> 2024-09-13T18:20:12.306 DEBUG:teuthology.orchestra.run.smithi114:> # Given a non-glob path, it better be there 2024-09-13T18:20:12.306 DEBUG:teuthology.orchestra.run.smithi114:> if "*" not in asok_path: 2024-09-13T18:20:12.306 DEBUG:teuthology.orchestra.run.smithi114:> assert(len(files) == 1) 2024-09-13T18:20:12.306 DEBUG:teuthology.orchestra.run.smithi114:> return files[0] 2024-09-13T18:20:12.306 DEBUG:teuthology.orchestra.run.smithi114:> 2024-09-13T18:20:12.306 DEBUG:teuthology.orchestra.run.smithi114:> for f in files: 2024-09-13T18:20:12.306 DEBUG:teuthology.orchestra.run.smithi114:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-13T18:20:12.306 DEBUG:teuthology.orchestra.run.smithi114:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-13T18:20:12.306 DEBUG:teuthology.orchestra.run.smithi114:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-13T18:20:12.306 DEBUG:teuthology.orchestra.run.smithi114:> contents = proc_f.read() 2024-09-13T18:20:12.306 DEBUG:teuthology.orchestra.run.smithi114:> if mountpoint in contents: 2024-09-13T18:20:12.306 DEBUG:teuthology.orchestra.run.smithi114:> return f 2024-09-13T18:20:12.307 DEBUG:teuthology.orchestra.run.smithi114:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-13T18:20:12.307 DEBUG:teuthology.orchestra.run.smithi114:> 2024-09-13T18:20:12.307 DEBUG:teuthology.orchestra.run.smithi114:> print(_find_admin_socket("client.admin")) 2024-09-13T18:20:12.307 DEBUG:teuthology.orchestra.run.smithi114:> ' 2024-09-13T18:20:12.419 INFO:teuthology.orchestra.run.smithi114.stdout:/var/run/ceph/ceph-client.admin.73373.asok 2024-09-13T18:20:12.605 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.73373.asok 2024-09-13T18:20:12.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:20:12.605 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.73373.asok status 2024-09-13T18:20:12.738 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:20:12.738 INFO:teuthology.orchestra.run.smithi114.stdout: "metadata": { 2024-09-13T18:20:12.738 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_sha1": "6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3", 2024-09-13T18:20:12.739 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_version": "ceph version 18.2.4-860-g6c6b083f (6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3) reef (stable)", 2024-09-13T18:20:12.739 INFO:teuthology.orchestra.run.smithi114.stdout: "entity_id": "admin", 2024-09-13T18:20:12.739 INFO:teuthology.orchestra.run.smithi114.stdout: "hostname": "smithi114", 2024-09-13T18:20:12.739 INFO:teuthology.orchestra.run.smithi114.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-13T18:20:12.739 INFO:teuthology.orchestra.run.smithi114.stdout: "pid": "73373", 2024-09-13T18:20:12.739 INFO:teuthology.orchestra.run.smithi114.stdout: "root": "/" 2024-09-13T18:20:12.740 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:20:12.740 INFO:teuthology.orchestra.run.smithi114.stdout: "dentry_count": 0, 2024-09-13T18:20:12.740 INFO:teuthology.orchestra.run.smithi114.stdout: "dentry_pinned_count": 0, 2024-09-13T18:20:12.740 INFO:teuthology.orchestra.run.smithi114.stdout: "id": 14769, 2024-09-13T18:20:12.740 INFO:teuthology.orchestra.run.smithi114.stdout: "inst": { 2024-09-13T18:20:12.740 INFO:teuthology.orchestra.run.smithi114.stdout: "name": { 2024-09-13T18:20:12.741 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "client", 2024-09-13T18:20:12.741 INFO:teuthology.orchestra.run.smithi114.stdout: "num": 14769 2024-09-13T18:20:12.741 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:20:12.741 INFO:teuthology.orchestra.run.smithi114.stdout: "addr": { 2024-09-13T18:20:12.741 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "v1", 2024-09-13T18:20:12.741 INFO:teuthology.orchestra.run.smithi114.stdout: "addr": "192.168.0.1:0", 2024-09-13T18:20:12.742 INFO:teuthology.orchestra.run.smithi114.stdout: "nonce": 35296839 2024-09-13T18:20:12.742 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:20:12.742 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:20:12.742 INFO:teuthology.orchestra.run.smithi114.stdout: "addr": { 2024-09-13T18:20:12.742 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "v1", 2024-09-13T18:20:12.742 INFO:teuthology.orchestra.run.smithi114.stdout: "addr": "192.168.0.1:0", 2024-09-13T18:20:12.743 INFO:teuthology.orchestra.run.smithi114.stdout: "nonce": 35296839 2024-09-13T18:20:12.743 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:20:12.743 INFO:teuthology.orchestra.run.smithi114.stdout: "inst_str": "client.14769 192.168.0.1:0/35296839", 2024-09-13T18:20:12.743 INFO:teuthology.orchestra.run.smithi114.stdout: "addr_str": "192.168.0.1:0/35296839", 2024-09-13T18:20:12.743 INFO:teuthology.orchestra.run.smithi114.stdout: "inode_count": 1, 2024-09-13T18:20:12.743 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_epoch": 17, 2024-09-13T18:20:12.744 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_epoch": 89, 2024-09-13T18:20:12.744 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_epoch_barrier": 0, 2024-09-13T18:20:12.744 INFO:teuthology.orchestra.run.smithi114.stdout: "blocklisted": false, 2024-09-13T18:20:12.744 INFO:teuthology.orchestra.run.smithi114.stdout: "fs_name": "cephfs" 2024-09-13T18:20:12.744 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:20:12.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:20:12.747 DEBUG:teuthology.orchestra.run.smithi114:> stat --file-system '--printf=%T 2024-09-13T18:20:12.747 DEBUG:teuthology.orchestra.run.smithi114:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-13T18:20:12.764 INFO:teuthology.orchestra.run.smithi114.stdout:fuseblk 2024-09-13T18:20:12.764 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-13T18:20:12.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:20:12.764 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-13T18:20:12.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:20:12.830 DEBUG:teuthology.orchestra.run.smithi114:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-13T18:20:12.905 INFO:teuthology.orchestra.run.smithi114.stdout:# file: . 2024-09-13T18:20:12.905 INFO:teuthology.orchestra.run.smithi114.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-13T18:20:12.905 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:20:12.905 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-13T18:20:12.905 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:20:12.962 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:20:13.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.097+0000 7f16db472640 1 -- 172.21.15.114:0/1402520824 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16d4105ab0 msgr2=0x7f16d4107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:13.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.097+0000 7f16db472640 1 --2- 172.21.15.114:0/1402520824 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16d4105ab0 0x7f16d4107f20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f16cc009ca0 tx=0x7f16cc02a8c0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.097+0000 7f16db472640 1 -- 172.21.15.114:0/1402520824 shutdown_connections 2024-09-13T18:20:13.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.097+0000 7f16db472640 1 --2- 172.21.15.114:0/1402520824 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16d4105ab0 0x7f16d4107f20 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.097+0000 7f16db472640 1 --2- 172.21.15.114:0/1402520824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16d4103180 0x7f16d4105570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.097+0000 7f16db472640 1 --2- 172.21.15.114:0/1402520824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16d4100850 0x7f16d4102c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.097+0000 7f16db472640 1 -- 172.21.15.114:0/1402520824 >> 172.21.15.114:0/1402520824 conn(0x7f16d40fa430 msgr2=0x7f16d40fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:13.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.097+0000 7f16db472640 1 -- 172.21.15.114:0/1402520824 shutdown_connections 2024-09-13T18:20:13.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.097+0000 7f16db472640 1 -- 172.21.15.114:0/1402520824 wait complete. 2024-09-13T18:20:13.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.098+0000 7f16db472640 1 Processor -- start 2024-09-13T18:20:13.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.098+0000 7f16db472640 1 -- start start 2024-09-13T18:20:13.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.098+0000 7f16db472640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16d4100850 0x7f16d4195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:13.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.098+0000 7f16db472640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16d4103180 0x7f16d4196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:13.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.098+0000 7f16db472640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16d4105ab0 0x7f16d419d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:13.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.098+0000 7f16db472640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d419d830 con 0x7f16d4100850 2024-09-13T18:20:13.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.098+0000 7f16db472640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d419d9a0 con 0x7f16d4105ab0 2024-09-13T18:20:13.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.098+0000 7f16db472640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d419db60 con 0x7f16d4103180 2024-09-13T18:20:13.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.098+0000 7f16d920e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16d4100850 0x7f16d4195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:13.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.098+0000 7f16d920e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16d4100850 0x7f16d4195d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44992/0 (socket says 172.21.15.114:44992) 2024-09-13T18:20:13.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.098+0000 7f16d920e640 1 -- 172.21.15.114:0/1701032434 learned_addr learned my addr 172.21.15.114:0/1701032434 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:20:13.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.098+0000 7f16d9a0f640 1 --2- 172.21.15.114:0/1701032434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16d4105ab0 0x7f16d419d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:13.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.098+0000 7f16d920e640 1 -- 172.21.15.114:0/1701032434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16d4103180 msgr2=0x7f16d4196290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:13.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.098+0000 7f16d920e640 1 --2- 172.21.15.114:0/1701032434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16d4103180 0x7f16d4196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.098+0000 7f16d920e640 1 -- 172.21.15.114:0/1701032434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16d4105ab0 msgr2=0x7f16d419d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:13.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.098+0000 7f16d920e640 1 --2- 172.21.15.114:0/1701032434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16d4105ab0 0x7f16d419d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.098+0000 7f16d920e640 1 -- 172.21.15.114:0/1701032434 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16cc009900 con 0x7f16d4100850 2024-09-13T18:20:13.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.098+0000 7f16d9a0f640 1 --2- 172.21.15.114:0/1701032434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16d4105ab0 0x7f16d419d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:20:13.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.098+0000 7f16d920e640 1 --2- 172.21.15.114:0/1701032434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16d4100850 0x7f16d4195d50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f16d000d770 tx=0x7f16d000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:13.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.099+0000 7f16ca7fc640 1 -- 172.21.15.114:0/1701032434 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d0009c30 con 0x7f16d4100850 2024-09-13T18:20:13.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.099+0000 7f16ca7fc640 1 -- 172.21.15.114:0/1701032434 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f16d000de80 con 0x7f16d4100850 2024-09-13T18:20:13.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.099+0000 7f16db472640 1 -- 172.21.15.114:0/1701032434 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16d419de40 con 0x7f16d4100850 2024-09-13T18:20:13.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.099+0000 7f16ca7fc640 1 -- 172.21.15.114:0/1701032434 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d0004d40 con 0x7f16d4100850 2024-09-13T18:20:13.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.099+0000 7f16db472640 1 -- 172.21.15.114:0/1701032434 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16d419e210 con 0x7f16d4100850 2024-09-13T18:20:13.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.100+0000 7f16db472640 1 -- 172.21.15.114:0/1701032434 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16d4103560 con 0x7f16d4100850 2024-09-13T18:20:13.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.101+0000 7f16ca7fc640 1 -- 172.21.15.114:0/1701032434 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f16d0004ea0 con 0x7f16d4100850 2024-09-13T18:20:13.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.101+0000 7f16ca7fc640 1 --2- 172.21.15.114:0/1701032434 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f16c00a1090 0x7f16c00a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:13.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.102+0000 7f16d8a0d640 1 --2- 172.21.15.114:0/1701032434 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f16c00a1090 0x7f16c00a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:13.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.102+0000 7f16ca7fc640 1 -- 172.21.15.114:0/1701032434 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f16d00cc670 con 0x7f16d4100850 2024-09-13T18:20:13.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.102+0000 7f16d8a0d640 1 --2- 172.21.15.114:0/1701032434 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f16c00a1090 0x7f16c00a3550 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f16c40058c0 tx=0x7f16c4005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:13.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.105+0000 7f16ca7fc640 1 -- 172.21.15.114:0/1701032434 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f16d0095b80 con 0x7f16d4100850 2024-09-13T18:20:13.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.269+0000 7f16db472640 1 -- 172.21.15.114:0/1701032434 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16d40fbf20 con 0x7f16d4100850 2024-09-13T18:20:13.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.271+0000 7f16ca7fc640 1 -- 172.21.15.114:0/1701032434 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24029 (secure 0 0 0) 0x7f16d0097ab0 con 0x7f16d4100850 2024-09-13T18:20:13.273 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:20:13.273 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":89,"fsid":"97e7edf2-71fb-11ef-bceb-c7b262605968","created":"2024-09-13T18:12:36.360625+0000","modified":"2024-09-13T18:20:06.455681+0000","last_up_change":"2024-09-13T18:19:12.350370+0000","last_in_change":"2024-09-13T18:18:52.321596+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:15:40.129659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:20:00.024407+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:20:01.085549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T18:20:04.334216+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9914f01a-9f6f-4878-b955-83570199c354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6803","nonce":3499525826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6805","nonce":3499525826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6809","nonce":3499525826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6807","nonce":3499525826}]},"public_addr":"172.21.15.114:6803/3499525826","cluster_addr":"172.21.15.114:6805/3499525826","heartbeat_back_addr":"172.21.15.114:6809/3499525826","heartbeat_front_addr":"172.21.15.114:6807/3499525826","state":["exists","up"]},{"osd":1,"uuid":"ef65e012-7469-4c20-b354-17022bca63c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6801","nonce":4264264866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6803","nonce":4264264866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6807","nonce":4264264866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6805","nonce":4264264866}]},"public_addr":"172.21.15.134:6801/4264264866","cluster_addr":"172.21.15.134:6803/4264264866","heartbeat_back_addr":"172.21.15.134:6807/4264264866","heartbeat_front_addr":"172.21.15.134:6805/4264264866","state":["exists","up"]},{"osd":2,"uuid":"578213fd-715f-4be7-91fa-254f3d0e3022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6801","nonce":2288807707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6803","nonce":2288807707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6807","nonce":2288807707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6805","nonce":2288807707}]},"public_addr":"172.21.15.204:6801/2288807707","cluster_addr":"172.21.15.204:6803/2288807707","heartbeat_back_addr":"172.21.15.204:6807/2288807707","heartbeat_front_addr":"172.21.15.204:6805/2288807707","state":["exists","up"]},{"osd":3,"uuid":"b64aab3f-1e52-4828-8397-74af5e05f2cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6811","nonce":2904204254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6813","nonce":2904204254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6817","nonce":2904204254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6815","nonce":2904204254}]},"public_addr":"172.21.15.114:6811/2904204254","cluster_addr":"172.21.15.114:6813/2904204254","heartbeat_back_addr":"172.21.15.114:6817/2904204254","heartbeat_front_addr":"172.21.15.114:6815/2904204254","state":["exists","up"]},{"osd":4,"uuid":"f38943d6-39c0-42e1-b427-cb6bb90ac38a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6809","nonce":1773945333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6811","nonce":1773945333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6815","nonce":1773945333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6813","nonce":1773945333}]},"public_addr":"172.21.15.134:6809/1773945333","cluster_addr":"172.21.15.134:6811/1773945333","heartbeat_back_addr":"172.21.15.134:6815/1773945333","heartbeat_front_addr":"172.21.15.134:6813/1773945333","state":["exists","up"]},{"osd":5,"uuid":"bb38f361-8e00-4178-9752-5ddbf3af20e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6809","nonce":2371778055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6811","nonce":2371778055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6815","nonce":2371778055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6813","nonce":2371778055}]},"public_addr":"172.21.15.204:6809/2371778055","cluster_addr":"172.21.15.204:6811/2371778055","heartbeat_back_addr":"172.21.15.204:6815/2371778055","heartbeat_front_addr":"172.21.15.204:6813/2371778055","state":["exists","up"]},{"osd":6,"uuid":"93d40a03-e7ff-4a3f-9efa-5c5403c9983d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6819","nonce":1484325044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6821","nonce":1484325044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6825","nonce":1484325044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6823","nonce":1484325044}]},"public_addr":"172.21.15.114:6819/1484325044","cluster_addr":"172.21.15.114:6821/1484325044","heartbeat_back_addr":"172.21.15.114:6825/1484325044","heartbeat_front_addr":"172.21.15.114:6823/1484325044","state":["exists","up"]},{"osd":7,"uuid":"982f629a-1b95-4be1-8e75-5ddce1ad77b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6817","nonce":799246614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6819","nonce":799246614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6823","nonce":799246614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6821","nonce":799246614}]},"public_addr":"172.21.15.134:6817/799246614","cluster_addr":"172.21.15.134:6819/799246614","heartbeat_back_addr":"172.21.15.134:6823/799246614","heartbeat_front_addr":"172.21.15.134:6821/799246614","state":["exists","up"]},{"osd":8,"uuid":"409beb9b-8227-4dad-a4a0-f0e5ce70dc5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6817","nonce":858331805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6819","nonce":858331805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6823","nonce":858331805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6821","nonce":858331805}]},"public_addr":"172.21.15.204:6817/858331805","cluster_addr":"172.21.15.204:6819/858331805","heartbeat_back_addr":"172.21.15.204:6823/858331805","heartbeat_front_addr":"172.21.15.204:6821/858331805","state":["exists","up"]},{"osd":9,"uuid":"a345a15c-b1fc-4877-b445-418eebf0d0ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6827","nonce":2172392854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6829","nonce":2172392854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6833","nonce":2172392854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6831","nonce":2172392854}]},"public_addr":"172.21.15.114:6827/2172392854","cluster_addr":"172.21.15.114:6829/2172392854","heartbeat_back_addr":"172.21.15.114:6833/2172392854","heartbeat_front_addr":"172.21.15.114:6831/2172392854","state":["exists","up"]},{"osd":10,"uuid":"464d995f-d90b-4caa-addb-1f7562bd74b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6825","nonce":3073513468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6827","nonce":3073513468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6831","nonce":3073513468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6829","nonce":3073513468}]},"public_addr":"172.21.15.134:6825/3073513468","cluster_addr":"172.21.15.134:6827/3073513468","heartbeat_back_addr":"172.21.15.134:6831/3073513468","heartbeat_front_addr":"172.21.15.134:6829/3073513468","state":["exists","up"]},{"osd":11,"uuid":"71dd794d-326c-477f-8d02-ee32bff1350d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6825","nonce":3239521765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6827","nonce":3239521765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6831","nonce":3239521765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6829","nonce":3239521765}]},"public_addr":"172.21.15.204:6825/3239521765","cluster_addr":"172.21.15.204:6827/3239521765","heartbeat_back_addr":"172.21.15.204:6831/3239521765","heartbeat_front_addr":"172.21.15.204:6829/3239521765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:14:54.542687+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:15:15.746055+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:15:38.371131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:01.383932+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:24.796237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:48.422214+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:11.284323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:34.570365+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:58.232855+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:18:21.609596+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:18:45.565859+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:19:10.017969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:6801/4281866858":"2024-09-14T18:13:25.823027+0000","172.21.15.114:6800/4281866858":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2949422849":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2034742251":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/722571999":"2024-09-14T18:13:03.521116+0000","172.21.15.114:6801/830738794":"2024-09-14T18:13:03.521116+0000","172.21.15.114:6800/830738794":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/2828364892":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/4257828906":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/3544951322":"2024-09-14T18:13:03.521116+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:20:13.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.273+0000 7f16db472640 1 -- 172.21.15.114:0/1701032434 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f16c00a1090 msgr2=0x7f16c00a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:13.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.273+0000 7f16db472640 1 --2- 172.21.15.114:0/1701032434 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f16c00a1090 0x7f16c00a3550 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f16c40058c0 tx=0x7f16c4005850 comp rx=0 tx=0).stop 2024-09-13T18:20:13.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.273+0000 7f16db472640 1 -- 172.21.15.114:0/1701032434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16d4100850 msgr2=0x7f16d4195d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:13.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.273+0000 7f16db472640 1 --2- 172.21.15.114:0/1701032434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16d4100850 0x7f16d4195d50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f16d000d770 tx=0x7f16d000dc30 comp rx=0 tx=0).stop 2024-09-13T18:20:13.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.273+0000 7f16db472640 1 -- 172.21.15.114:0/1701032434 shutdown_connections 2024-09-13T18:20:13.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.273+0000 7f16db472640 1 --2- 172.21.15.114:0/1701032434 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f16c00a1090 0x7f16c00a3550 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.273+0000 7f16db472640 1 --2- 172.21.15.114:0/1701032434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16d4105ab0 0x7f16d419d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.273+0000 7f16db472640 1 --2- 172.21.15.114:0/1701032434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16d4103180 0x7f16d4196290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.273+0000 7f16db472640 1 --2- 172.21.15.114:0/1701032434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16d4100850 0x7f16d4195d50 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.273+0000 7f16db472640 1 -- 172.21.15.114:0/1701032434 >> 172.21.15.114:0/1701032434 conn(0x7f16d40fa430 msgr2=0x7f16d41012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:13.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.273+0000 7f16db472640 1 -- 172.21.15.114:0/1701032434 shutdown_connections 2024-09-13T18:20:13.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.273+0000 7f16db472640 1 -- 172.21.15.114:0/1701032434 wait complete. 2024-09-13T18:20:13.284 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:13 smithi114 ceph-mon[30897]: pgmap v282: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-09-13T18:20:13.286 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T18:15:40.129659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T18:20:00.024407+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T18:20:01.085549+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T18:20:04.334216+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T18:20:13.286 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T18:20:13.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.385+0000 7f8fc8dd5640 1 -- 172.21.15.114:0/1107799237 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8fc4105a50 msgr2=0x7f8fc4107ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:13.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.385+0000 7f8fc8dd5640 1 --2- 172.21.15.114:0/1107799237 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8fc4105a50 0x7f8fc4107ec0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4009900 tx=0x7f8fb402a410 comp rx=0 tx=0).stop 2024-09-13T18:20:13.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.385+0000 7f8fc8dd5640 1 -- 172.21.15.114:0/1107799237 shutdown_connections 2024-09-13T18:20:13.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.385+0000 7f8fc8dd5640 1 --2- 172.21.15.114:0/1107799237 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8fc4105a50 0x7f8fc4107ec0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.385+0000 7f8fc8dd5640 1 --2- 172.21.15.114:0/1107799237 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fc4103120 0x7f8fc4105510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.385+0000 7f8fc8dd5640 1 --2- 172.21.15.114:0/1107799237 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8fc41007f0 0x7f8fc4102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.385+0000 7f8fc8dd5640 1 -- 172.21.15.114:0/1107799237 >> 172.21.15.114:0/1107799237 conn(0x7f8fc40fa3d0 msgr2=0x7f8fc40fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:13.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.385+0000 7f8fc8dd5640 1 -- 172.21.15.114:0/1107799237 shutdown_connections 2024-09-13T18:20:13.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.385+0000 7f8fc8dd5640 1 -- 172.21.15.114:0/1107799237 wait complete. 2024-09-13T18:20:13.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.386+0000 7f8fc8dd5640 1 Processor -- start 2024-09-13T18:20:13.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.386+0000 7f8fc8dd5640 1 -- start start 2024-09-13T18:20:13.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.386+0000 7f8fc8dd5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8fc41007f0 0x7f8fc4195c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:13.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.386+0000 7f8fc8dd5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fc4103120 0x7f8fc41961c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:13.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.386+0000 7f8fc8dd5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8fc4105a50 0x7f8fc419d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:13.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.386+0000 7f8fc8dd5640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc419d760 con 0x7f8fc41007f0 2024-09-13T18:20:13.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.386+0000 7f8fc8dd5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc419d8d0 con 0x7f8fc4103120 2024-09-13T18:20:13.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.386+0000 7f8fc8dd5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc419da90 con 0x7f8fc4105a50 2024-09-13T18:20:13.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.386+0000 7f8fc259c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8fc41007f0 0x7f8fc4195c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:13.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.386+0000 7f8fc259c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8fc41007f0 0x7f8fc4195c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39678/0 (socket says 172.21.15.114:39678) 2024-09-13T18:20:13.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.386+0000 7f8fc259c640 1 -- 172.21.15.114:0/3961557044 learned_addr learned my addr 172.21.15.114:0/3961557044 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:20:13.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.387+0000 7f8fc1d9b640 1 --2- 172.21.15.114:0/3961557044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fc4103120 0x7f8fc41961c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:13.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.387+0000 7f8fc2d9d640 1 --2- 172.21.15.114:0/3961557044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8fc4105a50 0x7f8fc419d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:13.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.387+0000 7f8fc259c640 1 -- 172.21.15.114:0/3961557044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8fc4105a50 msgr2=0x7f8fc419d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:13.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.387+0000 7f8fc259c640 1 --2- 172.21.15.114:0/3961557044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8fc4105a50 0x7f8fc419d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.387+0000 7f8fc259c640 1 -- 172.21.15.114:0/3961557044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fc4103120 msgr2=0x7f8fc41961c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:13.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.387+0000 7f8fc259c640 1 --2- 172.21.15.114:0/3961557044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fc4103120 0x7f8fc41961c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.387+0000 7f8fc259c640 1 -- 172.21.15.114:0/3961557044 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb4009560 con 0x7f8fc41007f0 2024-09-13T18:20:13.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.387+0000 7f8fc259c640 1 --2- 172.21.15.114:0/3961557044 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8fc41007f0 0x7f8fc4195c80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc00cce0 tx=0x7f8fbc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:13.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.387+0000 7f8fb37fe640 1 -- 172.21.15.114:0/3961557044 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc004b20 con 0x7f8fc41007f0 2024-09-13T18:20:13.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.387+0000 7f8fb37fe640 1 -- 172.21.15.114:0/3961557044 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8fbc004c80 con 0x7f8fc41007f0 2024-09-13T18:20:13.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.387+0000 7f8fc8dd5640 1 -- 172.21.15.114:0/3961557044 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fc419dd70 con 0x7f8fc41007f0 2024-09-13T18:20:13.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.387+0000 7f8fb37fe640 1 -- 172.21.15.114:0/3961557044 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc00f6b0 con 0x7f8fc41007f0 2024-09-13T18:20:13.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.387+0000 7f8fc8dd5640 1 -- 172.21.15.114:0/3961557044 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fc419e1d0 con 0x7f8fc41007f0 2024-09-13T18:20:13.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.388+0000 7f8fc8dd5640 1 -- 172.21.15.114:0/3961557044 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fc40fbec0 con 0x7f8fc41007f0 2024-09-13T18:20:13.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.389+0000 7f8fb37fe640 1 -- 172.21.15.114:0/3961557044 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f8fbc00f810 con 0x7f8fc41007f0 2024-09-13T18:20:13.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.390+0000 7f8fb37fe640 1 --2- 172.21.15.114:0/3961557044 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f8fac0a1090 0x7f8fac0a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:13.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.390+0000 7f8fb37fe640 1 -- 172.21.15.114:0/3961557044 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f8fbc0ccf70 con 0x7f8fc41007f0 2024-09-13T18:20:13.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.390+0000 7f8fc1d9b640 1 --2- 172.21.15.114:0/3961557044 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f8fac0a1090 0x7f8fac0a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:13.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.390+0000 7f8fc1d9b640 1 --2- 172.21.15.114:0/3961557044 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f8fac0a1090 0x7f8fac0a3550 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8fc40690d0 tx=0x7f8fb800a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:13.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.394+0000 7f8fb37fe640 1 -- 172.21.15.114:0/3961557044 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f8fbc096480 con 0x7f8fc41007f0 2024-09-13T18:20:13.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:13 smithi204 ceph-mon[34777]: pgmap v282: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-09-13T18:20:13.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.558+0000 7f8fc8dd5640 1 -- 172.21.15.114:0/3961557044 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8fc402c480 con 0x7f8fc41007f0 2024-09-13T18:20:13.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.558+0000 7f8fb37fe640 1 -- 172.21.15.114:0/3961557044 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8fbc01d020 con 0x7f8fc41007f0 2024-09-13T18:20:13.561 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 1 2024-09-13T18:20:13.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.560+0000 7f8fc8dd5640 1 -- 172.21.15.114:0/3961557044 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f8fac0a1090 msgr2=0x7f8fac0a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:13.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.560+0000 7f8fc8dd5640 1 --2- 172.21.15.114:0/3961557044 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f8fac0a1090 0x7f8fac0a3550 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8fc40690d0 tx=0x7f8fb800a2d0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.560+0000 7f8fc8dd5640 1 -- 172.21.15.114:0/3961557044 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8fc41007f0 msgr2=0x7f8fc4195c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:13.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.560+0000 7f8fc8dd5640 1 --2- 172.21.15.114:0/3961557044 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8fc41007f0 0x7f8fc4195c80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc00cce0 tx=0x7f8fbc00a620 comp rx=0 tx=0).stop 2024-09-13T18:20:13.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.560+0000 7f8fc8dd5640 1 -- 172.21.15.114:0/3961557044 shutdown_connections 2024-09-13T18:20:13.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.560+0000 7f8fc8dd5640 1 --2- 172.21.15.114:0/3961557044 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f8fac0a1090 0x7f8fac0a3550 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.560+0000 7f8fc8dd5640 1 --2- 172.21.15.114:0/3961557044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8fc4105a50 0x7f8fc419d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.560+0000 7f8fc8dd5640 1 --2- 172.21.15.114:0/3961557044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fc4103120 0x7f8fc41961c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.560+0000 7f8fc8dd5640 1 --2- 172.21.15.114:0/3961557044 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8fc41007f0 0x7f8fc4195c80 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.560+0000 7f8fc8dd5640 1 -- 172.21.15.114:0/3961557044 >> 172.21.15.114:0/3961557044 conn(0x7f8fc40fa3d0 msgr2=0x7f8fc4101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:13.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.561+0000 7f8fc8dd5640 1 -- 172.21.15.114:0/3961557044 shutdown_connections 2024-09-13T18:20:13.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.561+0000 7f8fc8dd5640 1 -- 172.21.15.114:0/3961557044 wait complete. 2024-09-13T18:20:13.573 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T18:20:13.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:13 smithi134 ceph-mon[34283]: pgmap v282: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-09-13T18:20:13.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.674+0000 7fe16243a640 1 -- 172.21.15.114:0/3425977296 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe15c103140 msgr2=0x7fe15c105530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:13.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.674+0000 7fe16243a640 1 --2- 172.21.15.114:0/3425977296 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe15c103140 0x7fe15c105530 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe144009900 tx=0x7fe14402a410 comp rx=0 tx=0).stop 2024-09-13T18:20:13.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.674+0000 7fe16243a640 1 -- 172.21.15.114:0/3425977296 shutdown_connections 2024-09-13T18:20:13.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.674+0000 7fe16243a640 1 --2- 172.21.15.114:0/3425977296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe15c105a70 0x7fe15c107ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.674+0000 7fe16243a640 1 --2- 172.21.15.114:0/3425977296 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe15c103140 0x7fe15c105530 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.674+0000 7fe16243a640 1 --2- 172.21.15.114:0/3425977296 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe15c100810 0x7fe15c102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.674+0000 7fe16243a640 1 -- 172.21.15.114:0/3425977296 >> 172.21.15.114:0/3425977296 conn(0x7fe15c0fa3f0 msgr2=0x7fe15c0fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:13.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.674+0000 7fe16243a640 1 -- 172.21.15.114:0/3425977296 shutdown_connections 2024-09-13T18:20:13.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.674+0000 7fe16243a640 1 -- 172.21.15.114:0/3425977296 wait complete. 2024-09-13T18:20:13.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.674+0000 7fe16243a640 1 Processor -- start 2024-09-13T18:20:13.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.674+0000 7fe16243a640 1 -- start start 2024-09-13T18:20:13.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.674+0000 7fe16243a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe15c100810 0x7fe15c195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:13.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.674+0000 7fe16243a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe15c103140 0x7fe15c196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:13.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.674+0000 7fe16243a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe15c105a70 0x7fe15c19d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:13.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.674+0000 7fe16243a640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15c19d7e0 con 0x7fe15c105a70 2024-09-13T18:20:13.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.674+0000 7fe16243a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15c19d950 con 0x7fe15c100810 2024-09-13T18:20:13.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.674+0000 7fe16243a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15c19db10 con 0x7fe15c103140 2024-09-13T18:20:13.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.675+0000 7fe1609d7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe15c105a70 0x7fe15c19d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:13.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.675+0000 7fe1609d7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe15c105a70 0x7fe15c19d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39696/0 (socket says 172.21.15.114:39696) 2024-09-13T18:20:13.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.675+0000 7fe1609d7640 1 -- 172.21.15.114:0/2669040625 learned_addr learned my addr 172.21.15.114:0/2669040625 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:20:13.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.675+0000 7fe153fff640 1 --2- 172.21.15.114:0/2669040625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe15c103140 0x7fe15c196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:13.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.675+0000 7fe15bfff640 1 --2- 172.21.15.114:0/2669040625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe15c100810 0x7fe15c195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:13.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.675+0000 7fe1609d7640 1 -- 172.21.15.114:0/2669040625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe15c103140 msgr2=0x7fe15c196290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:13.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.675+0000 7fe1609d7640 1 --2- 172.21.15.114:0/2669040625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe15c103140 0x7fe15c196290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.675+0000 7fe1609d7640 1 -- 172.21.15.114:0/2669040625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe15c100810 msgr2=0x7fe15c195d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:13.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.675+0000 7fe1609d7640 1 --2- 172.21.15.114:0/2669040625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe15c100810 0x7fe15c195d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.675+0000 7fe1609d7640 1 -- 172.21.15.114:0/2669040625 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe144009560 con 0x7fe15c105a70 2024-09-13T18:20:13.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.675+0000 7fe15bfff640 1 --2- 172.21.15.114:0/2669040625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe15c100810 0x7fe15c195d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:20:13.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.675+0000 7fe1609d7640 1 --2- 172.21.15.114:0/2669040625 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe15c105a70 0x7fe15c19d270 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe15400e9d0 tx=0x7fe15400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:13.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.675+0000 7fe159ffb640 1 -- 172.21.15.114:0/2669040625 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15400cd80 con 0x7fe15c105a70 2024-09-13T18:20:13.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.675+0000 7fe16243a640 1 -- 172.21.15.114:0/2669040625 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe15c19ddf0 con 0x7fe15c105a70 2024-09-13T18:20:13.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.675+0000 7fe159ffb640 1 -- 172.21.15.114:0/2669040625 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe154005020 con 0x7fe15c105a70 2024-09-13T18:20:13.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.675+0000 7fe159ffb640 1 -- 172.21.15.114:0/2669040625 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1540116d0 con 0x7fe15c105a70 2024-09-13T18:20:13.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.675+0000 7fe16243a640 1 -- 172.21.15.114:0/2669040625 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe15c19e220 con 0x7fe15c105a70 2024-09-13T18:20:13.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.676+0000 7fe16243a640 1 -- 172.21.15.114:0/2669040625 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe15c0fbee0 con 0x7fe15c105a70 2024-09-13T18:20:13.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.677+0000 7fe159ffb640 1 -- 172.21.15.114:0/2669040625 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fe154005190 con 0x7fe15c105a70 2024-09-13T18:20:13.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.678+0000 7fe159ffb640 1 --2- 172.21.15.114:0/2669040625 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fe1480a1090 0x7fe1480a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:13.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.678+0000 7fe15bfff640 1 --2- 172.21.15.114:0/2669040625 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fe1480a1090 0x7fe1480a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:13.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.678+0000 7fe159ffb640 1 -- 172.21.15.114:0/2669040625 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fe1540ccd30 con 0x7fe15c105a70 2024-09-13T18:20:13.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.678+0000 7fe15bfff640 1 --2- 172.21.15.114:0/2669040625 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fe1480a1090 0x7fe1480a3550 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe15c1971d0 tx=0x7fe14c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:13.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.682+0000 7fe159ffb640 1 -- 172.21.15.114:0/2669040625 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fe154096240 con 0x7fe15c105a70 2024-09-13T18:20:13.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.843+0000 7fe16243a640 1 -- 172.21.15.114:0/2669040625 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe15c0fb950 con 0x7fe15c105a70 2024-09-13T18:20:13.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.843+0000 7fe159ffb640 1 -- 172.21.15.114:0/2669040625 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe15401f0a0 con 0x7fe15c105a70 2024-09-13T18:20:13.845 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2024-09-13T18:20:13.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.845+0000 7fe16243a640 1 -- 172.21.15.114:0/2669040625 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fe1480a1090 msgr2=0x7fe1480a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:13.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.845+0000 7fe16243a640 1 --2- 172.21.15.114:0/2669040625 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fe1480a1090 0x7fe1480a3550 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe15c1971d0 tx=0x7fe14c005f90 comp rx=0 tx=0).stop 2024-09-13T18:20:13.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.845+0000 7fe16243a640 1 -- 172.21.15.114:0/2669040625 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe15c105a70 msgr2=0x7fe15c19d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:13.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.845+0000 7fe16243a640 1 --2- 172.21.15.114:0/2669040625 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe15c105a70 0x7fe15c19d270 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe15400e9d0 tx=0x7fe15400ee90 comp rx=0 tx=0).stop 2024-09-13T18:20:13.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.845+0000 7fe16243a640 1 -- 172.21.15.114:0/2669040625 shutdown_connections 2024-09-13T18:20:13.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.845+0000 7fe16243a640 1 --2- 172.21.15.114:0/2669040625 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fe1480a1090 0x7fe1480a3550 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.845+0000 7fe16243a640 1 --2- 172.21.15.114:0/2669040625 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe15c105a70 0x7fe15c19d270 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.845+0000 7fe16243a640 1 --2- 172.21.15.114:0/2669040625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe15c103140 0x7fe15c196290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.845+0000 7fe16243a640 1 --2- 172.21.15.114:0/2669040625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe15c100810 0x7fe15c195d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.845+0000 7fe16243a640 1 -- 172.21.15.114:0/2669040625 >> 172.21.15.114:0/2669040625 conn(0x7fe15c0fa3f0 msgr2=0x7fe15c101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:13.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.845+0000 7fe16243a640 1 -- 172.21.15.114:0/2669040625 shutdown_connections 2024-09-13T18:20:13.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.845+0000 7fe16243a640 1 -- 172.21.15.114:0/2669040625 wait complete. 2024-09-13T18:20:13.858 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T18:20:13.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.957+0000 7f53ae65a640 1 -- 172.21.15.114:0/3394202415 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53a80ffd70 msgr2=0x7f53a8100170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:13.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.957+0000 7f53ae65a640 1 --2- 172.21.15.114:0/3394202415 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53a80ffd70 0x7f53a8100170 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5398009900 tx=0x7f539802a410 comp rx=0 tx=0).stop 2024-09-13T18:20:13.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.957+0000 7f53ae65a640 1 -- 172.21.15.114:0/3394202415 shutdown_connections 2024-09-13T18:20:13.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.957+0000 7f53ae65a640 1 --2- 172.21.15.114:0/3394202415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53a8101920 0x7f53a81081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.957+0000 7f53ae65a640 1 --2- 172.21.15.114:0/3394202415 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53a8100f60 0x7f53a81013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.957+0000 7f53ae65a640 1 --2- 172.21.15.114:0/3394202415 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53a80ffd70 0x7f53a8100170 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.957+0000 7f53ae65a640 1 -- 172.21.15.114:0/3394202415 >> 172.21.15.114:0/3394202415 conn(0x7f53a80fb4d0 msgr2=0x7f53a80fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:13.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.957+0000 7f53ae65a640 1 -- 172.21.15.114:0/3394202415 shutdown_connections 2024-09-13T18:20:13.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.957+0000 7f53ae65a640 1 -- 172.21.15.114:0/3394202415 wait complete. 2024-09-13T18:20:13.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.958+0000 7f53ae65a640 1 Processor -- start 2024-09-13T18:20:13.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.958+0000 7f53ae65a640 1 -- start start 2024-09-13T18:20:13.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.958+0000 7f53ae65a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53a80ffd70 0x7f53a8195c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:13.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.958+0000 7f53ae65a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53a8100f60 0x7f53a8196160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:13.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.958+0000 7f53ae65a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53a8101920 0x7f53a819d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:13.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.958+0000 7f53ae65a640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a819d700 con 0x7f53a8101920 2024-09-13T18:20:13.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.958+0000 7f53ae65a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a819d870 con 0x7f53a80ffd70 2024-09-13T18:20:13.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.958+0000 7f53ae65a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a819da30 con 0x7f53a8100f60 2024-09-13T18:20:13.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.958+0000 7f53acbf7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53a8101920 0x7f53a819d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:13.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.958+0000 7f53acbf7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53a8101920 0x7f53a819d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39708/0 (socket says 172.21.15.114:39708) 2024-09-13T18:20:13.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.958+0000 7f53acbf7640 1 -- 172.21.15.114:0/2296897706 learned_addr learned my addr 172.21.15.114:0/2296897706 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:20:13.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.958+0000 7f53a7fff640 1 --2- 172.21.15.114:0/2296897706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53a80ffd70 0x7f53a8195c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:13.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.958+0000 7f539ffff640 1 --2- 172.21.15.114:0/2296897706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53a8100f60 0x7f53a8196160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:13.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.958+0000 7f53acbf7640 1 -- 172.21.15.114:0/2296897706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53a8100f60 msgr2=0x7f53a8196160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:13.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.958+0000 7f53acbf7640 1 --2- 172.21.15.114:0/2296897706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53a8100f60 0x7f53a8196160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.958+0000 7f53acbf7640 1 -- 172.21.15.114:0/2296897706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53a80ffd70 msgr2=0x7f53a8195c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:13.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.958+0000 7f53acbf7640 1 --2- 172.21.15.114:0/2296897706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53a80ffd70 0x7f53a8195c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.958+0000 7f53acbf7640 1 -- 172.21.15.114:0/2296897706 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5398009560 con 0x7f53a8101920 2024-09-13T18:20:13.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.959+0000 7f53acbf7640 1 --2- 172.21.15.114:0/2296897706 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53a8101920 0x7f53a819d190 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f53a000eaa0 tx=0x7f53a000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:13.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.959+0000 7f53a5ffb640 1 -- 172.21.15.114:0/2296897706 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a0004d20 con 0x7f53a8101920 2024-09-13T18:20:13.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.959+0000 7f53a5ffb640 1 -- 172.21.15.114:0/2296897706 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f53a0004e80 con 0x7f53a8101920 2024-09-13T18:20:13.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.959+0000 7f53ae65a640 1 -- 172.21.15.114:0/2296897706 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53a819dd10 con 0x7f53a8101920 2024-09-13T18:20:13.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.959+0000 7f53a5ffb640 1 -- 172.21.15.114:0/2296897706 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a0011470 con 0x7f53a8101920 2024-09-13T18:20:13.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.959+0000 7f53ae65a640 1 -- 172.21.15.114:0/2296897706 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53a819e1a0 con 0x7f53a8101920 2024-09-13T18:20:13.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.960+0000 7f53ae65a640 1 -- 172.21.15.114:0/2296897706 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53a8107200 con 0x7f53a8101920 2024-09-13T18:20:13.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.961+0000 7f53a5ffb640 1 -- 172.21.15.114:0/2296897706 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f53a000caf0 con 0x7f53a8101920 2024-09-13T18:20:13.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.962+0000 7f53a5ffb640 1 --2- 172.21.15.114:0/2296897706 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f53940a1090 0x7f53940a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:13.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.962+0000 7f53a7fff640 1 --2- 172.21.15.114:0/2296897706 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f53940a1090 0x7f53940a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:13.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.962+0000 7f53a5ffb640 1 -- 172.21.15.114:0/2296897706 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f53a00ccd60 con 0x7f53a8101920 2024-09-13T18:20:13.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.962+0000 7f53a7fff640 1 --2- 172.21.15.114:0/2296897706 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f53940a1090 0x7f53940a3550 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f53980057c0 tx=0x7f5398005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:13.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:13.966+0000 7f53a5ffb640 1 -- 172.21.15.114:0/2296897706 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f53a0096270 con 0x7f53a8101920 2024-09-13T18:20:14.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.131+0000 7f53ae65a640 1 -- 172.21.15.114:0/2296897706 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f53a802c410 con 0x7f53a8101920 2024-09-13T18:20:14.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.133+0000 7f53a5ffb640 1 -- 172.21.15.114:0/2296897706 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7f53a001f020 con 0x7f53a8101920 2024-09-13T18:20:14.135 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 64 2024-09-13T18:20:14.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.135+0000 7f53ae65a640 1 -- 172.21.15.114:0/2296897706 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f53940a1090 msgr2=0x7f53940a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:14.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.135+0000 7f53ae65a640 1 --2- 172.21.15.114:0/2296897706 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f53940a1090 0x7f53940a3550 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f53980057c0 tx=0x7f5398005a20 comp rx=0 tx=0).stop 2024-09-13T18:20:14.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.135+0000 7f53ae65a640 1 -- 172.21.15.114:0/2296897706 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53a8101920 msgr2=0x7f53a819d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:14.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.135+0000 7f53ae65a640 1 --2- 172.21.15.114:0/2296897706 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53a8101920 0x7f53a819d190 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f53a000eaa0 tx=0x7f53a000eea0 comp rx=0 tx=0).stop 2024-09-13T18:20:14.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.135+0000 7f53ae65a640 1 -- 172.21.15.114:0/2296897706 shutdown_connections 2024-09-13T18:20:14.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.135+0000 7f53ae65a640 1 --2- 172.21.15.114:0/2296897706 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f53940a1090 0x7f53940a3550 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:14.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.135+0000 7f53ae65a640 1 --2- 172.21.15.114:0/2296897706 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53a8101920 0x7f53a819d190 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:14.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.135+0000 7f53ae65a640 1 --2- 172.21.15.114:0/2296897706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53a8100f60 0x7f53a8196160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:14.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.135+0000 7f53ae65a640 1 --2- 172.21.15.114:0/2296897706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53a80ffd70 0x7f53a8195c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:14.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.135+0000 7f53ae65a640 1 -- 172.21.15.114:0/2296897706 >> 172.21.15.114:0/2296897706 conn(0x7f53a80fb4d0 msgr2=0x7f53a81043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:14.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.135+0000 7f53ae65a640 1 -- 172.21.15.114:0/2296897706 shutdown_connections 2024-09-13T18:20:14.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.135+0000 7f53ae65a640 1 -- 172.21.15.114:0/2296897706 wait complete. 2024-09-13T18:20:14.148 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T18:20:14.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.248+0000 7fa638420640 1 -- 172.21.15.114:0/2816085301 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa630105a40 msgr2=0x7fa630107eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:14.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.248+0000 7fa638420640 1 --2- 172.21.15.114:0/2816085301 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa630105a40 0x7fa630107eb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa628009ca0 tx=0x7fa62802a8c0 comp rx=0 tx=0).stop 2024-09-13T18:20:14.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.248+0000 7fa638420640 1 -- 172.21.15.114:0/2816085301 shutdown_connections 2024-09-13T18:20:14.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.248+0000 7fa638420640 1 --2- 172.21.15.114:0/2816085301 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa630105a40 0x7fa630107eb0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:14.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.248+0000 7fa638420640 1 --2- 172.21.15.114:0/2816085301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa630103110 0x7fa630105500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:14.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.248+0000 7fa638420640 1 --2- 172.21.15.114:0/2816085301 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6301007e0 0x7fa630102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:14.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.248+0000 7fa638420640 1 -- 172.21.15.114:0/2816085301 >> 172.21.15.114:0/2816085301 conn(0x7fa6300fa3e0 msgr2=0x7fa6300fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:14.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.249+0000 7fa638420640 1 -- 172.21.15.114:0/2816085301 shutdown_connections 2024-09-13T18:20:14.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.249+0000 7fa638420640 1 -- 172.21.15.114:0/2816085301 wait complete. 2024-09-13T18:20:14.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.249+0000 7fa638420640 1 Processor -- start 2024-09-13T18:20:14.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.249+0000 7fa638420640 1 -- start start 2024-09-13T18:20:14.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.249+0000 7fa638420640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6301007e0 0x7fa630195c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:14.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.249+0000 7fa638420640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa630103110 0x7fa630196180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:14.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.249+0000 7fa638420640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa630105a40 0x7fa63019d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:14.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.249+0000 7fa638420640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63019d720 con 0x7fa630103110 2024-09-13T18:20:14.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.249+0000 7fa638420640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63019d890 con 0x7fa6301007e0 2024-09-13T18:20:14.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.249+0000 7fa638420640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63019da50 con 0x7fa630105a40 2024-09-13T18:20:14.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.249+0000 7fa6359bb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa630103110 0x7fa630196180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:14.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.249+0000 7fa6359bb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa630103110 0x7fa630196180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39712/0 (socket says 172.21.15.114:39712) 2024-09-13T18:20:14.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.249+0000 7fa6359bb640 1 -- 172.21.15.114:0/1936095824 learned_addr learned my addr 172.21.15.114:0/1936095824 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:20:14.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.250+0000 7fa6369bd640 1 --2- 172.21.15.114:0/1936095824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa630105a40 0x7fa63019d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:14.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.250+0000 7fa6359bb640 1 -- 172.21.15.114:0/1936095824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa630105a40 msgr2=0x7fa63019d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:14.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.250+0000 7fa6359bb640 1 --2- 172.21.15.114:0/1936095824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa630105a40 0x7fa63019d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:14.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.250+0000 7fa6359bb640 1 -- 172.21.15.114:0/1936095824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6301007e0 msgr2=0x7fa630195c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:14.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.250+0000 7fa6361bc640 1 --2- 172.21.15.114:0/1936095824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6301007e0 0x7fa630195c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:14.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.250+0000 7fa6359bb640 1 --2- 172.21.15.114:0/1936095824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6301007e0 0x7fa630195c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:14.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.250+0000 7fa6359bb640 1 -- 172.21.15.114:0/1936095824 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa628009900 con 0x7fa630103110 2024-09-13T18:20:14.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.250+0000 7fa6369bd640 1 --2- 172.21.15.114:0/1936095824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa630105a40 0x7fa63019d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:20:14.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.250+0000 7fa6359bb640 1 --2- 172.21.15.114:0/1936095824 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa630103110 0x7fa630196180 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa62000c970 tx=0x7fa62000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:14.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.250+0000 7fa6277fe640 1 -- 172.21.15.114:0/1936095824 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa620004440 con 0x7fa630103110 2024-09-13T18:20:14.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.250+0000 7fa6277fe640 1 -- 172.21.15.114:0/1936095824 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa620007cf0 con 0x7fa630103110 2024-09-13T18:20:14.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.250+0000 7fa638420640 1 -- 172.21.15.114:0/1936095824 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa63019dd30 con 0x7fa630103110 2024-09-13T18:20:14.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.250+0000 7fa6277fe640 1 -- 172.21.15.114:0/1936095824 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa62000f450 con 0x7fa630103110 2024-09-13T18:20:14.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.250+0000 7fa638420640 1 -- 172.21.15.114:0/1936095824 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa63019e190 con 0x7fa630103110 2024-09-13T18:20:14.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.251+0000 7fa638420640 1 -- 172.21.15.114:0/1936095824 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6300fbed0 con 0x7fa630103110 2024-09-13T18:20:14.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.252+0000 7fa6277fe640 1 -- 172.21.15.114:0/1936095824 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fa620007860 con 0x7fa630103110 2024-09-13T18:20:14.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.252+0000 7fa6277fe640 1 --2- 172.21.15.114:0/1936095824 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fa61c0a1090 0x7fa61c0a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:14.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.253+0000 7fa6361bc640 1 --2- 172.21.15.114:0/1936095824 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fa61c0a1090 0x7fa61c0a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:14.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.253+0000 7fa6277fe640 1 -- 172.21.15.114:0/1936095824 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fa6200ccd00 con 0x7fa630103110 2024-09-13T18:20:14.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.253+0000 7fa6361bc640 1 --2- 172.21.15.114:0/1936095824 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fa61c0a1090 0x7fa61c0a3550 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa62c005fd0 tx=0x7fa62c005c70 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:14.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.256+0000 7fa6277fe640 1 -- 172.21.15.114:0/1936095824 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fa620096210 con 0x7fa630103110 2024-09-13T18:20:14.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.418+0000 7fa638420640 1 -- 172.21.15.114:0/1936095824 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa63002c480 con 0x7fa630103110 2024-09-13T18:20:14.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.418+0000 7fa6277fe640 1 -- 172.21.15.114:0/1936095824 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa6200140a0 con 0x7fa630103110 2024-09-13T18:20:14.420 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2024-09-13T18:20:14.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.420+0000 7fa638420640 1 -- 172.21.15.114:0/1936095824 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fa61c0a1090 msgr2=0x7fa61c0a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:14.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.420+0000 7fa638420640 1 --2- 172.21.15.114:0/1936095824 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fa61c0a1090 0x7fa61c0a3550 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa62c005fd0 tx=0x7fa62c005c70 comp rx=0 tx=0).stop 2024-09-13T18:20:14.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.420+0000 7fa638420640 1 -- 172.21.15.114:0/1936095824 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa630103110 msgr2=0x7fa630196180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:14.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.420+0000 7fa638420640 1 --2- 172.21.15.114:0/1936095824 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa630103110 0x7fa630196180 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa62000c970 tx=0x7fa62000ce30 comp rx=0 tx=0).stop 2024-09-13T18:20:14.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.420+0000 7fa638420640 1 -- 172.21.15.114:0/1936095824 shutdown_connections 2024-09-13T18:20:14.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.420+0000 7fa638420640 1 --2- 172.21.15.114:0/1936095824 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fa61c0a1090 0x7fa61c0a3550 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:14.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.420+0000 7fa638420640 1 --2- 172.21.15.114:0/1936095824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa630105a40 0x7fa63019d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:14.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.420+0000 7fa638420640 1 --2- 172.21.15.114:0/1936095824 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa630103110 0x7fa630196180 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:14.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.420+0000 7fa638420640 1 --2- 172.21.15.114:0/1936095824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6301007e0 0x7fa630195c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:14.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.420+0000 7fa638420640 1 -- 172.21.15.114:0/1936095824 >> 172.21.15.114:0/1936095824 conn(0x7fa6300fa3e0 msgr2=0x7fa630101160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:14.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.420+0000 7fa638420640 1 -- 172.21.15.114:0/1936095824 shutdown_connections 2024-09-13T18:20:14.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.420+0000 7fa638420640 1 -- 172.21.15.114:0/1936095824 wait complete. 2024-09-13T18:20:14.432 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:14 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/1701032434' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:20:14.432 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:14 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/3961557044' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T18:20:14.432 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:14 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2669040625' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T18:20:14.433 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T18:20:14.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:14 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/1701032434' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:20:14.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:14 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/3961557044' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T18:20:14.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:14 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/2669040625' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T18:20:14.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.530+0000 7fbea3c9c640 1 -- 172.21.15.114:0/4217505555 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe9c0ffea0 msgr2=0x7fbe9c100320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:14.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.530+0000 7fbea3c9c640 1 --2- 172.21.15.114:0/4217505555 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe9c0ffea0 0x7fbe9c100320 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fbe90009900 tx=0x7fbe9002a410 comp rx=0 tx=0).stop 2024-09-13T18:20:14.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.530+0000 7fbea3c9c640 1 -- 172.21.15.114:0/4217505555 shutdown_connections 2024-09-13T18:20:14.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.530+0000 7fbea3c9c640 1 --2- 172.21.15.114:0/4217505555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe9c100860 0x7fbe9c1070f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:14.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.530+0000 7fbea3c9c640 1 --2- 172.21.15.114:0/4217505555 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe9c0ffea0 0x7fbe9c100320 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:14.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.530+0000 7fbea3c9c640 1 --2- 172.21.15.114:0/4217505555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe9c0fecb0 0x7fbe9c0ff0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:14.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.530+0000 7fbea3c9c640 1 -- 172.21.15.114:0/4217505555 >> 172.21.15.114:0/4217505555 conn(0x7fbe9c0fa430 msgr2=0x7fbe9c0fc850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:14.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.530+0000 7fbea3c9c640 1 -- 172.21.15.114:0/4217505555 shutdown_connections 2024-09-13T18:20:14.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.530+0000 7fbea3c9c640 1 -- 172.21.15.114:0/4217505555 wait complete. 2024-09-13T18:20:14.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.530+0000 7fbea3c9c640 1 Processor -- start 2024-09-13T18:20:14.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.531+0000 7fbea3c9c640 1 -- start start 2024-09-13T18:20:14.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.531+0000 7fbea3c9c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe9c0fecb0 0x7fbe9c102f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:14.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.531+0000 7fbea3c9c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe9c0ffea0 0x7fbe9c1015b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:14.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.531+0000 7fbea3c9c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe9c100860 0x7fbe9c101af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:14.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.531+0000 7fbea3c9c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9c102250 con 0x7fbe9c100860 2024-09-13T18:20:14.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.531+0000 7fbea3c9c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9c1023c0 con 0x7fbe9c0fecb0 2024-09-13T18:20:14.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.531+0000 7fbea3c9c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9c19fdb0 con 0x7fbe9c0ffea0 2024-09-13T18:20:14.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.531+0000 7fbea2239640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe9c100860 0x7fbe9c101af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:14.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.531+0000 7fbea2239640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe9c100860 0x7fbe9c101af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39732/0 (socket says 172.21.15.114:39732) 2024-09-13T18:20:14.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.531+0000 7fbea2239640 1 -- 172.21.15.114:0/3174628228 learned_addr learned my addr 172.21.15.114:0/3174628228 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:20:14.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.531+0000 7fbea1237640 1 --2- 172.21.15.114:0/3174628228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe9c0ffea0 0x7fbe9c1015b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:14.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.531+0000 7fbea1a38640 1 --2- 172.21.15.114:0/3174628228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe9c0fecb0 0x7fbe9c102f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:14.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.531+0000 7fbea2239640 1 -- 172.21.15.114:0/3174628228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe9c0ffea0 msgr2=0x7fbe9c1015b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:14.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.531+0000 7fbea2239640 1 --2- 172.21.15.114:0/3174628228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe9c0ffea0 0x7fbe9c1015b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:14.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.531+0000 7fbea2239640 1 -- 172.21.15.114:0/3174628228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe9c0fecb0 msgr2=0x7fbe9c102f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:14.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.531+0000 7fbea2239640 1 --2- 172.21.15.114:0/3174628228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe9c0fecb0 0x7fbe9c102f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:14.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.531+0000 7fbea2239640 1 -- 172.21.15.114:0/3174628228 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe90009560 con 0x7fbe9c100860 2024-09-13T18:20:14.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.531+0000 7fbea2239640 1 --2- 172.21.15.114:0/3174628228 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe9c100860 0x7fbe9c101af0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c00cb00 tx=0x7fbe8c00a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:14.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.531+0000 7fbe9affd640 1 -- 172.21.15.114:0/3174628228 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c00ae90 con 0x7fbe9c100860 2024-09-13T18:20:14.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.531+0000 7fbe9affd640 1 -- 172.21.15.114:0/3174628228 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbe8c005020 con 0x7fbe9c100860 2024-09-13T18:20:14.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.531+0000 7fbea3c9c640 1 -- 172.21.15.114:0/3174628228 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe9c1a0060 con 0x7fbe9c100860 2024-09-13T18:20:14.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.531+0000 7fbe9affd640 1 -- 172.21.15.114:0/3174628228 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c00f6b0 con 0x7fbe9c100860 2024-09-13T18:20:14.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.532+0000 7fbea3c9c640 1 -- 172.21.15.114:0/3174628228 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe9c1a0540 con 0x7fbe9c100860 2024-09-13T18:20:14.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.533+0000 7fbea3c9c640 1 -- 172.21.15.114:0/3174628228 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe9c10a400 con 0x7fbe9c100860 2024-09-13T18:20:14.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.534+0000 7fbe9affd640 1 -- 172.21.15.114:0/3174628228 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fbe8c005190 con 0x7fbe9c100860 2024-09-13T18:20:14.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.534+0000 7fbe9affd640 1 --2- 172.21.15.114:0/3174628228 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fbe840a1110 0x7fbe840a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:14.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.535+0000 7fbea1a38640 1 --2- 172.21.15.114:0/3174628228 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fbe840a1110 0x7fbe840a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:14.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.535+0000 7fbe9affd640 1 -- 172.21.15.114:0/3174628228 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fbe8c0ccf50 con 0x7fbe9c100860 2024-09-13T18:20:14.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.535+0000 7fbea1a38640 1 --2- 172.21.15.114:0/3174628228 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fbe840a1110 0x7fbe840a35d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbe94005db0 tx=0x7fbe94005d40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:14.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.538+0000 7fbe9affd640 1 -- 172.21.15.114:0/3174628228 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fbe8c096570 con 0x7fbe9c100860 2024-09-13T18:20:14.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:14 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/1701032434' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:20:14.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:14 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/3961557044' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T18:20:14.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:14 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/2669040625' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T18:20:14.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.712+0000 7fbea3c9c640 1 -- 172.21.15.114:0/3174628228 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fbe9c10a5c0 con 0x7fbe9c100860 2024-09-13T18:20:14.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.713+0000 7fbe9affd640 1 -- 172.21.15.114:0/3174628228 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) v1 ==== 81+0+599 (secure 0 0 0) 0x7fbe8c01d070 con 0x7fbe9c100860 2024-09-13T18:20:14.715 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6801/4281866858 2024-09-14T18:13:25.823027+0000 2024-09-13T18:20:14.715 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6800/4281866858 2024-09-14T18:13:25.823027+0000 2024-09-13T18:20:14.715 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/2949422849 2024-09-14T18:13:25.823027+0000 2024-09-13T18:20:14.716 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/2034742251 2024-09-14T18:13:03.521116+0000 2024-09-13T18:20:14.716 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/722571999 2024-09-14T18:13:03.521116+0000 2024-09-13T18:20:14.716 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6801/830738794 2024-09-14T18:13:03.521116+0000 2024-09-13T18:20:14.716 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6800/830738794 2024-09-14T18:13:03.521116+0000 2024-09-13T18:20:14.716 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/2828364892 2024-09-14T18:13:25.823027+0000 2024-09-13T18:20:14.716 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/4257828906 2024-09-14T18:13:25.823027+0000 2024-09-13T18:20:14.716 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/3544951322 2024-09-14T18:13:03.521116+0000 2024-09-13T18:20:14.717 INFO:teuthology.orchestra.run.smithi114.stderr:listed 10 entries 2024-09-13T18:20:14.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.715+0000 7fbea3c9c640 1 -- 172.21.15.114:0/3174628228 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fbe840a1110 msgr2=0x7fbe840a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:14.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.715+0000 7fbea3c9c640 1 --2- 172.21.15.114:0/3174628228 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fbe840a1110 0x7fbe840a35d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbe94005db0 tx=0x7fbe94005d40 comp rx=0 tx=0).stop 2024-09-13T18:20:14.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.715+0000 7fbea3c9c640 1 -- 172.21.15.114:0/3174628228 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe9c100860 msgr2=0x7fbe9c101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:14.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.715+0000 7fbea3c9c640 1 --2- 172.21.15.114:0/3174628228 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe9c100860 0x7fbe9c101af0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c00cb00 tx=0x7fbe8c00a250 comp rx=0 tx=0).stop 2024-09-13T18:20:14.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.715+0000 7fbea3c9c640 1 -- 172.21.15.114:0/3174628228 shutdown_connections 2024-09-13T18:20:14.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.715+0000 7fbea3c9c640 1 --2- 172.21.15.114:0/3174628228 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fbe840a1110 0x7fbe840a35d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:14.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.715+0000 7fbea3c9c640 1 --2- 172.21.15.114:0/3174628228 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe9c100860 0x7fbe9c101af0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:14.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.715+0000 7fbea3c9c640 1 --2- 172.21.15.114:0/3174628228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe9c0ffea0 0x7fbe9c1015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:14.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.715+0000 7fbea3c9c640 1 --2- 172.21.15.114:0/3174628228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe9c0fecb0 0x7fbe9c102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:14.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.715+0000 7fbea3c9c640 1 -- 172.21.15.114:0/3174628228 >> 172.21.15.114:0/3174628228 conn(0x7fbe9c0fa430 msgr2=0x7fbe9c0685d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:14.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.715+0000 7fbea3c9c640 1 -- 172.21.15.114:0/3174628228 shutdown_connections 2024-09-13T18:20:14.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.715+0000 7fbea3c9c640 1 -- 172.21.15.114:0/3174628228 wait complete. 2024-09-13T18:20:14.727 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-13T18:20:14.727 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:20:14.744 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:20:14.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.880+0000 7f70e75aa640 1 -- 172.21.15.114:0/1338983990 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70e00feca0 msgr2=0x7f70e00ff0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:14.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.880+0000 7f70e75aa640 1 --2- 172.21.15.114:0/1338983990 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70e00feca0 0x7f70e00ff0a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f70d0009900 tx=0x7f70d002a410 comp rx=0 tx=0).stop 2024-09-13T18:20:14.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.880+0000 7f70e75aa640 1 -- 172.21.15.114:0/1338983990 shutdown_connections 2024-09-13T18:20:14.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.880+0000 7f70e75aa640 1 --2- 172.21.15.114:0/1338983990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70e0100850 0x7f70e01070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:14.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.880+0000 7f70e75aa640 1 --2- 172.21.15.114:0/1338983990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70e00ffe90 0x7f70e0100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:14.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.880+0000 7f70e75aa640 1 --2- 172.21.15.114:0/1338983990 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70e00feca0 0x7f70e00ff0a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:14.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.880+0000 7f70e75aa640 1 -- 172.21.15.114:0/1338983990 >> 172.21.15.114:0/1338983990 conn(0x7f70e00fa440 msgr2=0x7f70e00fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:14.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.880+0000 7f70e75aa640 1 -- 172.21.15.114:0/1338983990 shutdown_connections 2024-09-13T18:20:14.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.880+0000 7f70e75aa640 1 -- 172.21.15.114:0/1338983990 wait complete. 2024-09-13T18:20:14.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.880+0000 7f70e75aa640 1 Processor -- start 2024-09-13T18:20:14.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.880+0000 7f70e75aa640 1 -- start start 2024-09-13T18:20:14.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.880+0000 7f70e75aa640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70e00feca0 0x7f70e0068110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:14.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.880+0000 7f70e75aa640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70e00ffe90 0x7f70e019aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:14.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.880+0000 7f70e75aa640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70e0100850 0x7f70e019d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:14.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.880+0000 7f70e75aa640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e019d890 con 0x7f70e00feca0 2024-09-13T18:20:14.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.880+0000 7f70e75aa640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e019da00 con 0x7f70e00ffe90 2024-09-13T18:20:14.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.880+0000 7f70e75aa640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e019dbc0 con 0x7f70e0100850 2024-09-13T18:20:14.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.881+0000 7f70e5346640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70e00feca0 0x7f70e0068110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:14.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.881+0000 7f70e5346640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70e00feca0 0x7f70e0068110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39760/0 (socket says 172.21.15.114:39760) 2024-09-13T18:20:14.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.881+0000 7f70e5346640 1 -- 172.21.15.114:0/1956626436 learned_addr learned my addr 172.21.15.114:0/1956626436 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:20:14.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.881+0000 7f70e5b47640 1 --2- 172.21.15.114:0/1956626436 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70e0100850 0x7f70e019d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:14.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.881+0000 7f70e5346640 1 -- 172.21.15.114:0/1956626436 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70e0100850 msgr2=0x7f70e019d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:14.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.881+0000 7f70e5346640 1 --2- 172.21.15.114:0/1956626436 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70e0100850 0x7f70e019d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:14.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.881+0000 7f70e5346640 1 -- 172.21.15.114:0/1956626436 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70e00ffe90 msgr2=0x7f70e019aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:14.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.881+0000 7f70e5346640 1 --2- 172.21.15.114:0/1956626436 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70e00ffe90 0x7f70e019aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:14.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.881+0000 7f70e5346640 1 -- 172.21.15.114:0/1956626436 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d0009560 con 0x7f70e00feca0 2024-09-13T18:20:14.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.881+0000 7f70e5b47640 1 --2- 172.21.15.114:0/1956626436 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70e0100850 0x7f70e019d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:20:14.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.881+0000 7f70e5346640 1 --2- 172.21.15.114:0/1956626436 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70e00feca0 0x7f70e0068110 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f70d0005ab0 tx=0x7f70d00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:14.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.881+0000 7f70de7fc640 1 -- 172.21.15.114:0/1956626436 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d0031d70 con 0x7f70e00feca0 2024-09-13T18:20:14.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.881+0000 7f70de7fc640 1 -- 172.21.15.114:0/1956626436 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f70d00044e0 con 0x7f70e00feca0 2024-09-13T18:20:14.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.881+0000 7f70e75aa640 1 -- 172.21.15.114:0/1956626436 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70e019dde0 con 0x7f70e00feca0 2024-09-13T18:20:14.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.881+0000 7f70de7fc640 1 -- 172.21.15.114:0/1956626436 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d0003b00 con 0x7f70e00feca0 2024-09-13T18:20:14.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.881+0000 7f70e75aa640 1 -- 172.21.15.114:0/1956626436 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70e019e230 con 0x7f70e00feca0 2024-09-13T18:20:14.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.882+0000 7f70e75aa640 1 -- 172.21.15.114:0/1956626436 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70e01061d0 con 0x7f70e00feca0 2024-09-13T18:20:14.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.883+0000 7f70de7fc640 1 -- 172.21.15.114:0/1956626436 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f70d0004c50 con 0x7f70e00feca0 2024-09-13T18:20:14.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.884+0000 7f70de7fc640 1 --2- 172.21.15.114:0/1956626436 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f70c80a1090 0x7f70c80a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:14.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.884+0000 7f70e4b45640 1 --2- 172.21.15.114:0/1956626436 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f70c80a1090 0x7f70c80a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:14.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.884+0000 7f70de7fc640 1 -- 172.21.15.114:0/1956626436 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f70d00edcc0 con 0x7f70e00feca0 2024-09-13T18:20:14.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.884+0000 7f70e4b45640 1 --2- 172.21.15.114:0/1956626436 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f70c80a1090 0x7f70c80a3550 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f70e0067e80 tx=0x7f70d4009250 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:14.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:14.888+0000 7f70de7fc640 1 -- 172.21.15.114:0/1956626436 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f70d00b7260 con 0x7f70e00feca0 2024-09-13T18:20:15.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.051+0000 7f70e75aa640 1 -- 172.21.15.114:0/1956626436 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f70e002c480 con 0x7f70e00feca0 2024-09-13T18:20:15.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.052+0000 7f70de7fc640 1 -- 172.21.15.114:0/1956626436 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24029 (secure 0 0 0) 0x7f70d0040090 con 0x7f70e00feca0 2024-09-13T18:20:15.055 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:20:15.055 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":89,"fsid":"97e7edf2-71fb-11ef-bceb-c7b262605968","created":"2024-09-13T18:12:36.360625+0000","modified":"2024-09-13T18:20:06.455681+0000","last_up_change":"2024-09-13T18:19:12.350370+0000","last_in_change":"2024-09-13T18:18:52.321596+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:15:40.129659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:20:00.024407+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:20:01.085549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T18:20:04.334216+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9914f01a-9f6f-4878-b955-83570199c354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6803","nonce":3499525826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6805","nonce":3499525826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6809","nonce":3499525826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6807","nonce":3499525826}]},"public_addr":"172.21.15.114:6803/3499525826","cluster_addr":"172.21.15.114:6805/3499525826","heartbeat_back_addr":"172.21.15.114:6809/3499525826","heartbeat_front_addr":"172.21.15.114:6807/3499525826","state":["exists","up"]},{"osd":1,"uuid":"ef65e012-7469-4c20-b354-17022bca63c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6801","nonce":4264264866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6803","nonce":4264264866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6807","nonce":4264264866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6805","nonce":4264264866}]},"public_addr":"172.21.15.134:6801/4264264866","cluster_addr":"172.21.15.134:6803/4264264866","heartbeat_back_addr":"172.21.15.134:6807/4264264866","heartbeat_front_addr":"172.21.15.134:6805/4264264866","state":["exists","up"]},{"osd":2,"uuid":"578213fd-715f-4be7-91fa-254f3d0e3022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6801","nonce":2288807707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6803","nonce":2288807707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6807","nonce":2288807707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6805","nonce":2288807707}]},"public_addr":"172.21.15.204:6801/2288807707","cluster_addr":"172.21.15.204:6803/2288807707","heartbeat_back_addr":"172.21.15.204:6807/2288807707","heartbeat_front_addr":"172.21.15.204:6805/2288807707","state":["exists","up"]},{"osd":3,"uuid":"b64aab3f-1e52-4828-8397-74af5e05f2cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6811","nonce":2904204254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6813","nonce":2904204254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6817","nonce":2904204254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6815","nonce":2904204254}]},"public_addr":"172.21.15.114:6811/2904204254","cluster_addr":"172.21.15.114:6813/2904204254","heartbeat_back_addr":"172.21.15.114:6817/2904204254","heartbeat_front_addr":"172.21.15.114:6815/2904204254","state":["exists","up"]},{"osd":4,"uuid":"f38943d6-39c0-42e1-b427-cb6bb90ac38a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6809","nonce":1773945333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6811","nonce":1773945333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6815","nonce":1773945333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6813","nonce":1773945333}]},"public_addr":"172.21.15.134:6809/1773945333","cluster_addr":"172.21.15.134:6811/1773945333","heartbeat_back_addr":"172.21.15.134:6815/1773945333","heartbeat_front_addr":"172.21.15.134:6813/1773945333","state":["exists","up"]},{"osd":5,"uuid":"bb38f361-8e00-4178-9752-5ddbf3af20e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6809","nonce":2371778055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6811","nonce":2371778055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6815","nonce":2371778055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6813","nonce":2371778055}]},"public_addr":"172.21.15.204:6809/2371778055","cluster_addr":"172.21.15.204:6811/2371778055","heartbeat_back_addr":"172.21.15.204:6815/2371778055","heartbeat_front_addr":"172.21.15.204:6813/2371778055","state":["exists","up"]},{"osd":6,"uuid":"93d40a03-e7ff-4a3f-9efa-5c5403c9983d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6819","nonce":1484325044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6821","nonce":1484325044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6825","nonce":1484325044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6823","nonce":1484325044}]},"public_addr":"172.21.15.114:6819/1484325044","cluster_addr":"172.21.15.114:6821/1484325044","heartbeat_back_addr":"172.21.15.114:6825/1484325044","heartbeat_front_addr":"172.21.15.114:6823/1484325044","state":["exists","up"]},{"osd":7,"uuid":"982f629a-1b95-4be1-8e75-5ddce1ad77b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6817","nonce":799246614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6819","nonce":799246614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6823","nonce":799246614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6821","nonce":799246614}]},"public_addr":"172.21.15.134:6817/799246614","cluster_addr":"172.21.15.134:6819/799246614","heartbeat_back_addr":"172.21.15.134:6823/799246614","heartbeat_front_addr":"172.21.15.134:6821/799246614","state":["exists","up"]},{"osd":8,"uuid":"409beb9b-8227-4dad-a4a0-f0e5ce70dc5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6817","nonce":858331805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6819","nonce":858331805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6823","nonce":858331805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6821","nonce":858331805}]},"public_addr":"172.21.15.204:6817/858331805","cluster_addr":"172.21.15.204:6819/858331805","heartbeat_back_addr":"172.21.15.204:6823/858331805","heartbeat_front_addr":"172.21.15.204:6821/858331805","state":["exists","up"]},{"osd":9,"uuid":"a345a15c-b1fc-4877-b445-418eebf0d0ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6827","nonce":2172392854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6829","nonce":2172392854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6833","nonce":2172392854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6831","nonce":2172392854}]},"public_addr":"172.21.15.114:6827/2172392854","cluster_addr":"172.21.15.114:6829/2172392854","heartbeat_back_addr":"172.21.15.114:6833/2172392854","heartbeat_front_addr":"172.21.15.114:6831/2172392854","state":["exists","up"]},{"osd":10,"uuid":"464d995f-d90b-4caa-addb-1f7562bd74b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6825","nonce":3073513468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6827","nonce":3073513468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6831","nonce":3073513468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6829","nonce":3073513468}]},"public_addr":"172.21.15.134:6825/3073513468","cluster_addr":"172.21.15.134:6827/3073513468","heartbeat_back_addr":"172.21.15.134:6831/3073513468","heartbeat_front_addr":"172.21.15.134:6829/3073513468","state":["exists","up"]},{"osd":11,"uuid":"71dd794d-326c-477f-8d02-ee32bff1350d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6825","nonce":3239521765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6827","nonce":3239521765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6831","nonce":3239521765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6829","nonce":3239521765}]},"public_addr":"172.21.15.204:6825/3239521765","cluster_addr":"172.21.15.204:6827/3239521765","heartbeat_back_addr":"172.21.15.204:6831/3239521765","heartbeat_front_addr":"172.21.15.204:6829/3239521765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:14:54.542687+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:15:15.746055+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:15:38.371131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:01.383932+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:24.796237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:48.422214+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:11.284323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:34.570365+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:58.232855+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:18:21.609596+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:18:45.565859+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:19:10.017969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:6801/4281866858":"2024-09-14T18:13:25.823027+0000","172.21.15.114:6800/4281866858":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2949422849":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2034742251":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/722571999":"2024-09-14T18:13:03.521116+0000","172.21.15.114:6801/830738794":"2024-09-14T18:13:03.521116+0000","172.21.15.114:6800/830738794":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/2828364892":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/4257828906":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/3544951322":"2024-09-14T18:13:03.521116+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:20:15.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.054+0000 7f70e75aa640 1 -- 172.21.15.114:0/1956626436 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f70c80a1090 msgr2=0x7f70c80a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:15.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.054+0000 7f70e75aa640 1 --2- 172.21.15.114:0/1956626436 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f70c80a1090 0x7f70c80a3550 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f70e0067e80 tx=0x7f70d4009250 comp rx=0 tx=0).stop 2024-09-13T18:20:15.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.054+0000 7f70e75aa640 1 -- 172.21.15.114:0/1956626436 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70e00feca0 msgr2=0x7f70e0068110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:15.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.054+0000 7f70e75aa640 1 --2- 172.21.15.114:0/1956626436 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70e00feca0 0x7f70e0068110 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f70d0005ab0 tx=0x7f70d00042f0 comp rx=0 tx=0).stop 2024-09-13T18:20:15.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.054+0000 7f70e75aa640 1 -- 172.21.15.114:0/1956626436 shutdown_connections 2024-09-13T18:20:15.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.054+0000 7f70e75aa640 1 --2- 172.21.15.114:0/1956626436 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f70c80a1090 0x7f70c80a3550 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:15.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.054+0000 7f70e75aa640 1 --2- 172.21.15.114:0/1956626436 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70e0100850 0x7f70e019d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:15.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.054+0000 7f70e75aa640 1 --2- 172.21.15.114:0/1956626436 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70e00ffe90 0x7f70e019aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:15.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.054+0000 7f70e75aa640 1 --2- 172.21.15.114:0/1956626436 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70e00feca0 0x7f70e0068110 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:15.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.054+0000 7f70e75aa640 1 -- 172.21.15.114:0/1956626436 >> 172.21.15.114:0/1956626436 conn(0x7f70e00fa440 msgr2=0x7f70e01033a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:15.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.054+0000 7f70e75aa640 1 -- 172.21.15.114:0/1956626436 shutdown_connections 2024-09-13T18:20:15.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.054+0000 7f70e75aa640 1 -- 172.21.15.114:0/1956626436 wait complete. 2024-09-13T18:20:15.068 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T18:15:40.129659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T18:20:00.024407+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T18:20:01.085549+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T18:20:04.334216+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T18:20:15.068 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T18:20:15.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.169+0000 7f154b753640 1 -- 172.21.15.114:0/1422322027 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f15440fddf0 msgr2=0x7f15440fe270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:15.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.169+0000 7f154b753640 1 --2- 172.21.15.114:0/1422322027 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f15440fddf0 0x7f15440fe270 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1538009900 tx=0x7f153802a410 comp rx=0 tx=0).stop 2024-09-13T18:20:15.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.169+0000 7f154b753640 1 -- 172.21.15.114:0/1422322027 shutdown_connections 2024-09-13T18:20:15.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.169+0000 7f154b753640 1 --2- 172.21.15.114:0/1422322027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15440fe7b0 0x7f1544108360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:15.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.169+0000 7f154b753640 1 --2- 172.21.15.114:0/1422322027 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f15440fddf0 0x7f15440fe270 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:15.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.169+0000 7f154b753640 1 --2- 172.21.15.114:0/1422322027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1544101380 0x7f1544101780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:15.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.169+0000 7f154b753640 1 -- 172.21.15.114:0/1422322027 >> 172.21.15.114:0/1422322027 conn(0x7f15440f9b80 msgr2=0x7f15440fbfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:15.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.169+0000 7f154b753640 1 -- 172.21.15.114:0/1422322027 shutdown_connections 2024-09-13T18:20:15.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.169+0000 7f154b753640 1 -- 172.21.15.114:0/1422322027 wait complete. 2024-09-13T18:20:15.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.170+0000 7f154b753640 1 Processor -- start 2024-09-13T18:20:15.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.170+0000 7f154b753640 1 -- start start 2024-09-13T18:20:15.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.170+0000 7f154b753640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15440fddf0 0x7f1544195c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:15.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.170+0000 7f154b753640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15440fe7b0 0x7f15441961a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:15.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.170+0000 7f154b753640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1544101380 0x7f154419d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:15.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.170+0000 7f154b753640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f154419d740 con 0x7f1544101380 2024-09-13T18:20:15.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.170+0000 7f154b753640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f154419d8b0 con 0x7f15440fe7b0 2024-09-13T18:20:15.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.170+0000 7f154b753640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f154419da70 con 0x7f15440fddf0 2024-09-13T18:20:15.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.170+0000 7f1549cf0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1544101380 0x7f154419d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:15.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.170+0000 7f1549cf0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1544101380 0x7f154419d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39776/0 (socket says 172.21.15.114:39776) 2024-09-13T18:20:15.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.170+0000 7f1549cf0640 1 -- 172.21.15.114:0/2501322835 learned_addr learned my addr 172.21.15.114:0/2501322835 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:20:15.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.170+0000 7f15494ef640 1 --2- 172.21.15.114:0/2501322835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15440fddf0 0x7f1544195c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:15.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.170+0000 7f1548cee640 1 --2- 172.21.15.114:0/2501322835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15440fe7b0 0x7f15441961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:15.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.170+0000 7f1549cf0640 1 -- 172.21.15.114:0/2501322835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15440fddf0 msgr2=0x7f1544195c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:15.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.170+0000 7f1549cf0640 1 --2- 172.21.15.114:0/2501322835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15440fddf0 0x7f1544195c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:15.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.170+0000 7f1549cf0640 1 -- 172.21.15.114:0/2501322835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15440fe7b0 msgr2=0x7f15441961a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:15.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.170+0000 7f1549cf0640 1 --2- 172.21.15.114:0/2501322835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15440fe7b0 0x7f15441961a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:15.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.170+0000 7f1549cf0640 1 -- 172.21.15.114:0/2501322835 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1538009560 con 0x7f1544101380 2024-09-13T18:20:15.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.170+0000 7f1548cee640 1 --2- 172.21.15.114:0/2501322835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15440fe7b0 0x7f15441961a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:20:15.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.170+0000 7f1549cf0640 1 --2- 172.21.15.114:0/2501322835 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1544101380 0x7f154419d1d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f153c00cce0 tx=0x7f153c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:15.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.170+0000 7f15367fc640 1 -- 172.21.15.114:0/2501322835 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153c00f450 con 0x7f1544101380 2024-09-13T18:20:15.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.171+0000 7f15367fc640 1 -- 172.21.15.114:0/2501322835 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f153c00fa40 con 0x7f1544101380 2024-09-13T18:20:15.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.171+0000 7f154b753640 1 -- 172.21.15.114:0/2501322835 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f154419dd50 con 0x7f1544101380 2024-09-13T18:20:15.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.171+0000 7f15367fc640 1 -- 172.21.15.114:0/2501322835 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153c0189a0 con 0x7f1544101380 2024-09-13T18:20:15.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.171+0000 7f154b753640 1 -- 172.21.15.114:0/2501322835 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f154419e1b0 con 0x7f1544101380 2024-09-13T18:20:15.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.171+0000 7f154b753640 1 -- 172.21.15.114:0/2501322835 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1544107450 con 0x7f1544101380 2024-09-13T18:20:15.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.173+0000 7f15367fc640 1 -- 172.21.15.114:0/2501322835 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f153c00fbb0 con 0x7f1544101380 2024-09-13T18:20:15.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.173+0000 7f15367fc640 1 --2- 172.21.15.114:0/2501322835 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f15300a1090 0x7f15300a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:15.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.173+0000 7f15494ef640 1 --2- 172.21.15.114:0/2501322835 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f15300a1090 0x7f15300a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:15.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.173+0000 7f15367fc640 1 -- 172.21.15.114:0/2501322835 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f153c0ccf70 con 0x7f1544101380 2024-09-13T18:20:15.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.174+0000 7f15494ef640 1 --2- 172.21.15.114:0/2501322835 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f15300a1090 0x7f15300a3550 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1544197130 tx=0x7f154000b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:15.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.176+0000 7f15367fc640 1 -- 172.21.15.114:0/2501322835 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f153c096510 con 0x7f1544101380 2024-09-13T18:20:15.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.338+0000 7f154b753640 1 -- 172.21.15.114:0/2501322835 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f154402c480 con 0x7f1544101380 2024-09-13T18:20:15.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.339+0000 7f15367fc640 1 -- 172.21.15.114:0/2501322835 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f153c099b30 con 0x7f1544101380 2024-09-13T18:20:15.341 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 1 2024-09-13T18:20:15.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.341+0000 7f154b753640 1 -- 172.21.15.114:0/2501322835 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f15300a1090 msgr2=0x7f15300a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:15.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.341+0000 7f154b753640 1 --2- 172.21.15.114:0/2501322835 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f15300a1090 0x7f15300a3550 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1544197130 tx=0x7f154000b390 comp rx=0 tx=0).stop 2024-09-13T18:20:15.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.341+0000 7f154b753640 1 -- 172.21.15.114:0/2501322835 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1544101380 msgr2=0x7f154419d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:15.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.341+0000 7f154b753640 1 --2- 172.21.15.114:0/2501322835 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1544101380 0x7f154419d1d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f153c00cce0 tx=0x7f153c00a620 comp rx=0 tx=0).stop 2024-09-13T18:20:15.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.341+0000 7f154b753640 1 -- 172.21.15.114:0/2501322835 shutdown_connections 2024-09-13T18:20:15.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.341+0000 7f154b753640 1 --2- 172.21.15.114:0/2501322835 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f15300a1090 0x7f15300a3550 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:15.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.341+0000 7f154b753640 1 --2- 172.21.15.114:0/2501322835 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1544101380 0x7f154419d1d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:15.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.341+0000 7f154b753640 1 --2- 172.21.15.114:0/2501322835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15440fe7b0 0x7f15441961a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:15.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.341+0000 7f154b753640 1 --2- 172.21.15.114:0/2501322835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15440fddf0 0x7f1544195c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:15.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.341+0000 7f154b753640 1 -- 172.21.15.114:0/2501322835 >> 172.21.15.114:0/2501322835 conn(0x7f15440f9b80 msgr2=0x7f1544104620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:15.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.341+0000 7f154b753640 1 -- 172.21.15.114:0/2501322835 shutdown_connections 2024-09-13T18:20:15.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.341+0000 7f154b753640 1 -- 172.21.15.114:0/2501322835 wait complete. 2024-09-13T18:20:15.352 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:15 smithi114 ceph-mon[30897]: pgmap v283: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 3 op/s 2024-09-13T18:20:15.353 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:15 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2296897706' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T18:20:15.353 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:15 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/1936095824' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T18:20:15.353 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:15 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/3174628228' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-13T18:20:15.353 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:15 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/1956626436' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:20:15.354 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T18:20:15.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:15 smithi204 ceph-mon[34777]: pgmap v283: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 3 op/s 2024-09-13T18:20:15.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:15 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/2296897706' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T18:20:15.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:15 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/1936095824' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T18:20:15.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:15 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/3174628228' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-13T18:20:15.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:15 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/1956626436' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:20:15.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.453+0000 7f47bd8d0640 1 -- 172.21.15.114:0/3347707745 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f47b80fec90 msgr2=0x7f47b80ff090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:15.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.453+0000 7f47bd8d0640 1 --2- 172.21.15.114:0/3347707745 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f47b80fec90 0x7f47b80ff090 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f47ac009900 tx=0x7f47ac02a410 comp rx=0 tx=0).stop 2024-09-13T18:20:15.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.453+0000 7f47bd8d0640 1 -- 172.21.15.114:0/3347707745 shutdown_connections 2024-09-13T18:20:15.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.453+0000 7f47bd8d0640 1 --2- 172.21.15.114:0/3347707745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47b8100840 0x7f47b81070d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:15.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.453+0000 7f47bd8d0640 1 --2- 172.21.15.114:0/3347707745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47b80ffe80 0x7f47b8100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:15.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.453+0000 7f47bd8d0640 1 --2- 172.21.15.114:0/3347707745 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f47b80fec90 0x7f47b80ff090 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:15.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.453+0000 7f47bd8d0640 1 -- 172.21.15.114:0/3347707745 >> 172.21.15.114:0/3347707745 conn(0x7f47b80fa3f0 msgr2=0x7f47b80fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:15.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.453+0000 7f47bd8d0640 1 -- 172.21.15.114:0/3347707745 shutdown_connections 2024-09-13T18:20:15.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.453+0000 7f47bd8d0640 1 -- 172.21.15.114:0/3347707745 wait complete. 2024-09-13T18:20:15.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.453+0000 7f47bd8d0640 1 Processor -- start 2024-09-13T18:20:15.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.453+0000 7f47bd8d0640 1 -- start start 2024-09-13T18:20:15.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.454+0000 7f47bd8d0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47b80fec90 0x7f47b8195d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:15.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.454+0000 7f47bd8d0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f47b80ffe80 0x7f47b8196240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:15.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.454+0000 7f47bd8d0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47b8100840 0x7f47b819d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:15.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.454+0000 7f47bd8d0640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b819d7e0 con 0x7f47b80ffe80 2024-09-13T18:20:15.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.454+0000 7f47bd8d0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b819d950 con 0x7f47b80fec90 2024-09-13T18:20:15.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.454+0000 7f47bd8d0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b819db10 con 0x7f47b8100840 2024-09-13T18:20:15.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.454+0000 7f47b67fc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f47b80ffe80 0x7f47b8196240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:15.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.454+0000 7f47b67fc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f47b80ffe80 0x7f47b8196240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39798/0 (socket says 172.21.15.114:39798) 2024-09-13T18:20:15.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.454+0000 7f47b67fc640 1 -- 172.21.15.114:0/3801838984 learned_addr learned my addr 172.21.15.114:0/3801838984 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:20:15.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.454+0000 7f47b77fe640 1 --2- 172.21.15.114:0/3801838984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47b8100840 0x7f47b819d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:15.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.454+0000 7f47b6ffd640 1 --2- 172.21.15.114:0/3801838984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47b80fec90 0x7f47b8195d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:15.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.454+0000 7f47b67fc640 1 -- 172.21.15.114:0/3801838984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47b8100840 msgr2=0x7f47b819d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:15.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.454+0000 7f47b67fc640 1 --2- 172.21.15.114:0/3801838984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47b8100840 0x7f47b819d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:15.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.454+0000 7f47b67fc640 1 -- 172.21.15.114:0/3801838984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47b80fec90 msgr2=0x7f47b8195d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:15.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.454+0000 7f47b67fc640 1 --2- 172.21.15.114:0/3801838984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47b80fec90 0x7f47b8195d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:15.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.454+0000 7f47b67fc640 1 -- 172.21.15.114:0/3801838984 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47ac009560 con 0x7f47b80ffe80 2024-09-13T18:20:15.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.454+0000 7f47b6ffd640 1 --2- 172.21.15.114:0/3801838984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47b80fec90 0x7f47b8195d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:20:15.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.454+0000 7f47b67fc640 1 --2- 172.21.15.114:0/3801838984 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f47b80ffe80 0x7f47b8196240 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f47a800d950 tx=0x7f47a800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:15.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.454+0000 7f47bc8ce640 1 -- 172.21.15.114:0/3801838984 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a8009c30 con 0x7f47b80ffe80 2024-09-13T18:20:15.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.454+0000 7f47bd8d0640 1 -- 172.21.15.114:0/3801838984 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47b819ddf0 con 0x7f47b80ffe80 2024-09-13T18:20:15.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.454+0000 7f47bc8ce640 1 -- 172.21.15.114:0/3801838984 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f47a8009d90 con 0x7f47b80ffe80 2024-09-13T18:20:15.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.454+0000 7f47bc8ce640 1 -- 172.21.15.114:0/3801838984 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a800f620 con 0x7f47b80ffe80 2024-09-13T18:20:15.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.454+0000 7f47bd8d0640 1 -- 172.21.15.114:0/3801838984 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47b819e270 con 0x7f47b80ffe80 2024-09-13T18:20:15.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.455+0000 7f47bd8d0640 1 -- 172.21.15.114:0/3801838984 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47b810a400 con 0x7f47b80ffe80 2024-09-13T18:20:15.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.456+0000 7f47bc8ce640 1 -- 172.21.15.114:0/3801838984 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f47a80109f0 con 0x7f47b80ffe80 2024-09-13T18:20:15.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.457+0000 7f47bc8ce640 1 --2- 172.21.15.114:0/3801838984 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f47a00a1090 0x7f47a00a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:15.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.457+0000 7f47b6ffd640 1 --2- 172.21.15.114:0/3801838984 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f47a00a1090 0x7f47a00a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:15.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.457+0000 7f47bc8ce640 1 -- 172.21.15.114:0/3801838984 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f47a80cc870 con 0x7f47b80ffe80 2024-09-13T18:20:15.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.457+0000 7f47b6ffd640 1 --2- 172.21.15.114:0/3801838984 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f47a00a1090 0x7f47a00a3550 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f47ac000c00 tx=0x7f47ac005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:15.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.461+0000 7f47bc8ce640 1 -- 172.21.15.114:0/3801838984 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f47a8095d80 con 0x7f47b80ffe80 2024-09-13T18:20:15.520 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:15 smithi134 ceph-mon[34283]: pgmap v283: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 3 op/s 2024-09-13T18:20:15.520 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:15 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/2296897706' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T18:20:15.520 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:15 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/1936095824' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T18:20:15.521 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:15 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/3174628228' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-13T18:20:15.521 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:15 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/1956626436' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:20:15.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.625+0000 7f47bd8d0640 1 -- 172.21.15.114:0/3801838984 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f47b819e690 con 0x7f47b80ffe80 2024-09-13T18:20:15.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.626+0000 7f47bc8ce640 1 -- 172.21.15.114:0/3801838984 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7f47a8017020 con 0x7f47b80ffe80 2024-09-13T18:20:15.628 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2024-09-13T18:20:15.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.628+0000 7f47bd8d0640 1 -- 172.21.15.114:0/3801838984 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f47a00a1090 msgr2=0x7f47a00a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:15.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.628+0000 7f47bd8d0640 1 --2- 172.21.15.114:0/3801838984 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f47a00a1090 0x7f47a00a3550 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f47ac000c00 tx=0x7f47ac005a20 comp rx=0 tx=0).stop 2024-09-13T18:20:15.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.628+0000 7f47bd8d0640 1 -- 172.21.15.114:0/3801838984 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f47b80ffe80 msgr2=0x7f47b8196240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:15.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.628+0000 7f47bd8d0640 1 --2- 172.21.15.114:0/3801838984 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f47b80ffe80 0x7f47b8196240 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f47a800d950 tx=0x7f47a800de10 comp rx=0 tx=0).stop 2024-09-13T18:20:15.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.628+0000 7f47bd8d0640 1 -- 172.21.15.114:0/3801838984 shutdown_connections 2024-09-13T18:20:15.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.628+0000 7f47bd8d0640 1 --2- 172.21.15.114:0/3801838984 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f47a00a1090 0x7f47a00a3550 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:15.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.628+0000 7f47bd8d0640 1 --2- 172.21.15.114:0/3801838984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47b8100840 0x7f47b819d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:15.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.628+0000 7f47bd8d0640 1 --2- 172.21.15.114:0/3801838984 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f47b80ffe80 0x7f47b8196240 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:15.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.628+0000 7f47bd8d0640 1 --2- 172.21.15.114:0/3801838984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47b80fec90 0x7f47b8195d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:15.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.628+0000 7f47bd8d0640 1 -- 172.21.15.114:0/3801838984 >> 172.21.15.114:0/3801838984 conn(0x7f47b80fa3f0 msgr2=0x7f47b80685e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:15.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.628+0000 7f47bd8d0640 1 -- 172.21.15.114:0/3801838984 shutdown_connections 2024-09-13T18:20:15.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.628+0000 7f47bd8d0640 1 -- 172.21.15.114:0/3801838984 wait complete. 2024-09-13T18:20:15.641 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T18:20:15.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.741+0000 7fbad4e9c640 1 -- 172.21.15.114:0/3146267598 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbad0100f60 msgr2=0x7fbad01013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:15.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.741+0000 7fbad4e9c640 1 --2- 172.21.15.114:0/3146267598 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbad0100f60 0x7fbad01013e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbab8009900 tx=0x7fbab802a410 comp rx=0 tx=0).stop 2024-09-13T18:20:15.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.741+0000 7fbad4e9c640 1 -- 172.21.15.114:0/3146267598 shutdown_connections 2024-09-13T18:20:15.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.741+0000 7fbad4e9c640 1 --2- 172.21.15.114:0/3146267598 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbad0101920 0x7fbad01081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:15.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.741+0000 7fbad4e9c640 1 --2- 172.21.15.114:0/3146267598 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbad0100f60 0x7fbad01013e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:15.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.741+0000 7fbad4e9c640 1 --2- 172.21.15.114:0/3146267598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbad00ffd70 0x7fbad0100170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:15.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.741+0000 7fbad4e9c640 1 -- 172.21.15.114:0/3146267598 >> 172.21.15.114:0/3146267598 conn(0x7fbad00fb4d0 msgr2=0x7fbad00fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:15.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.741+0000 7fbad4e9c640 1 -- 172.21.15.114:0/3146267598 shutdown_connections 2024-09-13T18:20:15.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.741+0000 7fbad4e9c640 1 -- 172.21.15.114:0/3146267598 wait complete. 2024-09-13T18:20:15.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.741+0000 7fbad4e9c640 1 Processor -- start 2024-09-13T18:20:15.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.742+0000 7fbad4e9c640 1 -- start start 2024-09-13T18:20:15.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.742+0000 7fbad4e9c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbad00ffd70 0x7fbad0074260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:15.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.742+0000 7fbad4e9c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbad0100f60 0x7fbad00728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:15.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.742+0000 7fbad4e9c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbad0101920 0x7fbad0072de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:15.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.742+0000 7fbad4e9c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad0073540 con 0x7fbad0101920 2024-09-13T18:20:15.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.742+0000 7fbad4e9c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad00736b0 con 0x7fbad00ffd70 2024-09-13T18:20:15.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.742+0000 7fbad4e9c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad019fd00 con 0x7fbad0100f60 2024-09-13T18:20:15.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.742+0000 7fbaced9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbad0101920 0x7fbad0072de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:15.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.742+0000 7fbaced9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbad0101920 0x7fbad0072de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39818/0 (socket says 172.21.15.114:39818) 2024-09-13T18:20:15.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.742+0000 7fbaced9d640 1 -- 172.21.15.114:0/1786206712 learned_addr learned my addr 172.21.15.114:0/1786206712 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:20:15.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.742+0000 7fbace59c640 1 --2- 172.21.15.114:0/1786206712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbad00ffd70 0x7fbad0074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:15.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.742+0000 7fbacdd9b640 1 --2- 172.21.15.114:0/1786206712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbad0100f60 0x7fbad00728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:15.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.742+0000 7fbaced9d640 1 -- 172.21.15.114:0/1786206712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbad0100f60 msgr2=0x7fbad00728a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:15.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.742+0000 7fbaced9d640 1 --2- 172.21.15.114:0/1786206712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbad0100f60 0x7fbad00728a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:15.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.742+0000 7fbaced9d640 1 -- 172.21.15.114:0/1786206712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbad00ffd70 msgr2=0x7fbad0074260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:15.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.742+0000 7fbaced9d640 1 --2- 172.21.15.114:0/1786206712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbad00ffd70 0x7fbad0074260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:15.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.742+0000 7fbaced9d640 1 -- 172.21.15.114:0/1786206712 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab8009560 con 0x7fbad0101920 2024-09-13T18:20:15.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.742+0000 7fbacdd9b640 1 --2- 172.21.15.114:0/1786206712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbad0100f60 0x7fbad00728a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:20:15.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.742+0000 7fbace59c640 1 --2- 172.21.15.114:0/1786206712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbad00ffd70 0x7fbad0074260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:20:15.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.742+0000 7fbaced9d640 1 --2- 172.21.15.114:0/1786206712 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbad0101920 0x7fbad0072de0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbac800cb00 tx=0x7fbac800a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:15.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.742+0000 7fbac77fe640 1 -- 172.21.15.114:0/1786206712 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac800ae90 con 0x7fbad0101920 2024-09-13T18:20:15.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.742+0000 7fbac77fe640 1 -- 172.21.15.114:0/1786206712 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbac8005020 con 0x7fbad0101920 2024-09-13T18:20:15.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.743+0000 7fbad4e9c640 1 -- 172.21.15.114:0/1786206712 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbad019ffb0 con 0x7fbad0101920 2024-09-13T18:20:15.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.743+0000 7fbac77fe640 1 -- 172.21.15.114:0/1786206712 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac800f6b0 con 0x7fbad0101920 2024-09-13T18:20:15.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.743+0000 7fbad4e9c640 1 -- 172.21.15.114:0/1786206712 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbad01a03e0 con 0x7fbad0101920 2024-09-13T18:20:15.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.743+0000 7fbad4e9c640 1 -- 172.21.15.114:0/1786206712 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbad0107290 con 0x7fbad0101920 2024-09-13T18:20:15.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.745+0000 7fbac77fe640 1 -- 172.21.15.114:0/1786206712 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fbac8005190 con 0x7fbad0101920 2024-09-13T18:20:15.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.745+0000 7fbac77fe640 1 --2- 172.21.15.114:0/1786206712 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fbabc0a1090 0x7fbabc0a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:15.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.745+0000 7fbace59c640 1 --2- 172.21.15.114:0/1786206712 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fbabc0a1090 0x7fbabc0a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:15.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.745+0000 7fbac77fe640 1 -- 172.21.15.114:0/1786206712 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fbac80ccbb0 con 0x7fbad0101920 2024-09-13T18:20:15.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.746+0000 7fbace59c640 1 --2- 172.21.15.114:0/1786206712 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fbabc0a1090 0x7fbabc0a3550 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbac00058c0 tx=0x7fbac0005ec0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:15.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.749+0000 7fbac77fe640 1 -- 172.21.15.114:0/1786206712 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fbac8096150 con 0x7fbad0101920 2024-09-13T18:20:15.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.909+0000 7fbad4e9c640 1 -- 172.21.15.114:0/1786206712 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbad002c410 con 0x7fbad0101920 2024-09-13T18:20:15.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.909+0000 7fbac77fe640 1 -- 172.21.15.114:0/1786206712 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbac801d020 con 0x7fbad0101920 2024-09-13T18:20:15.912 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 64 2024-09-13T18:20:15.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.911+0000 7fbad4e9c640 1 -- 172.21.15.114:0/1786206712 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fbabc0a1090 msgr2=0x7fbabc0a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:15.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.911+0000 7fbad4e9c640 1 --2- 172.21.15.114:0/1786206712 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fbabc0a1090 0x7fbabc0a3550 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbac00058c0 tx=0x7fbac0005ec0 comp rx=0 tx=0).stop 2024-09-13T18:20:15.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.911+0000 7fbad4e9c640 1 -- 172.21.15.114:0/1786206712 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbad0101920 msgr2=0x7fbad0072de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:15.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.911+0000 7fbad4e9c640 1 --2- 172.21.15.114:0/1786206712 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbad0101920 0x7fbad0072de0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbac800cb00 tx=0x7fbac800a250 comp rx=0 tx=0).stop 2024-09-13T18:20:15.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.911+0000 7fbad4e9c640 1 -- 172.21.15.114:0/1786206712 shutdown_connections 2024-09-13T18:20:15.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.911+0000 7fbad4e9c640 1 --2- 172.21.15.114:0/1786206712 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fbabc0a1090 0x7fbabc0a3550 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:15.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.911+0000 7fbad4e9c640 1 --2- 172.21.15.114:0/1786206712 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbad0101920 0x7fbad0072de0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:15.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.911+0000 7fbad4e9c640 1 --2- 172.21.15.114:0/1786206712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbad0100f60 0x7fbad00728a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:15.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.911+0000 7fbad4e9c640 1 --2- 172.21.15.114:0/1786206712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbad00ffd70 0x7fbad0074260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:15.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.911+0000 7fbad4e9c640 1 -- 172.21.15.114:0/1786206712 >> 172.21.15.114:0/1786206712 conn(0x7fbad00fb4d0 msgr2=0x7fbad0104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:15.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.911+0000 7fbad4e9c640 1 -- 172.21.15.114:0/1786206712 shutdown_connections 2024-09-13T18:20:15.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:15.911+0000 7fbad4e9c640 1 -- 172.21.15.114:0/1786206712 wait complete. 2024-09-13T18:20:15.924 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T18:20:16.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.023+0000 7fa86731f640 1 -- 172.21.15.114:0/3624774324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa860100850 msgr2=0x7fa8601070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:16.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.023+0000 7fa86731f640 1 --2- 172.21.15.114:0/3624774324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa860100850 0x7fa8601070e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa858009ca0 tx=0x7fa85802a8c0 comp rx=0 tx=0).stop 2024-09-13T18:20:16.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.023+0000 7fa86731f640 1 -- 172.21.15.114:0/3624774324 shutdown_connections 2024-09-13T18:20:16.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.023+0000 7fa86731f640 1 --2- 172.21.15.114:0/3624774324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa860100850 0x7fa8601070e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:16.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.023+0000 7fa86731f640 1 --2- 172.21.15.114:0/3624774324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8600ffe90 0x7fa860100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:16.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.023+0000 7fa86731f640 1 --2- 172.21.15.114:0/3624774324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8600feca0 0x7fa8600ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:16.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.023+0000 7fa86731f640 1 -- 172.21.15.114:0/3624774324 >> 172.21.15.114:0/3624774324 conn(0x7fa8600fa420 msgr2=0x7fa8600fc840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:16.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.023+0000 7fa86731f640 1 -- 172.21.15.114:0/3624774324 shutdown_connections 2024-09-13T18:20:16.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.023+0000 7fa86731f640 1 -- 172.21.15.114:0/3624774324 wait complete. 2024-09-13T18:20:16.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.024+0000 7fa86731f640 1 Processor -- start 2024-09-13T18:20:16.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.024+0000 7fa86731f640 1 -- start start 2024-09-13T18:20:16.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.024+0000 7fa86731f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8600feca0 0x7fa860194ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:16.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.024+0000 7fa86731f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa8600ffe90 0x7fa8601950e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:16.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.024+0000 7fa86731f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa860100850 0x7fa86019c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:16.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.024+0000 7fa86731f640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86019c690 con 0x7fa8600ffe90 2024-09-13T18:20:16.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.024+0000 7fa86731f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86019c800 con 0x7fa8600feca0 2024-09-13T18:20:16.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.024+0000 7fa86731f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86019c9c0 con 0x7fa860100850 2024-09-13T18:20:16.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.024+0000 7fa8648ba640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa8600ffe90 0x7fa8601950e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:16.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.024+0000 7fa8648ba640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa8600ffe90 0x7fa8601950e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39832/0 (socket says 172.21.15.114:39832) 2024-09-13T18:20:16.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.024+0000 7fa8648ba640 1 -- 172.21.15.114:0/4086747068 learned_addr learned my addr 172.21.15.114:0/4086747068 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:20:16.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.024+0000 7fa8650bb640 1 --2- 172.21.15.114:0/4086747068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8600feca0 0x7fa860194ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:16.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.024+0000 7fa8658bc640 1 --2- 172.21.15.114:0/4086747068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa860100850 0x7fa86019c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:16.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.024+0000 7fa8648ba640 1 -- 172.21.15.114:0/4086747068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa860100850 msgr2=0x7fa86019c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:16.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.024+0000 7fa8648ba640 1 --2- 172.21.15.114:0/4086747068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa860100850 0x7fa86019c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:16.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.024+0000 7fa8648ba640 1 -- 172.21.15.114:0/4086747068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8600feca0 msgr2=0x7fa860194ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:16.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.024+0000 7fa8648ba640 1 --2- 172.21.15.114:0/4086747068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8600feca0 0x7fa860194ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:16.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.024+0000 7fa8648ba640 1 -- 172.21.15.114:0/4086747068 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa858009900 con 0x7fa8600ffe90 2024-09-13T18:20:16.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.025+0000 7fa8648ba640 1 --2- 172.21.15.114:0/4086747068 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa8600ffe90 0x7fa8601950e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa85000d950 tx=0x7fa85000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:16.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.025+0000 7fa8567fc640 1 -- 172.21.15.114:0/4086747068 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa850009a30 con 0x7fa8600ffe90 2024-09-13T18:20:16.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.025+0000 7fa8567fc640 1 -- 172.21.15.114:0/4086747068 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa8500044b0 con 0x7fa8600ffe90 2024-09-13T18:20:16.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.025+0000 7fa86731f640 1 -- 172.21.15.114:0/4086747068 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa86019cca0 con 0x7fa8600ffe90 2024-09-13T18:20:16.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.025+0000 7fa8567fc640 1 -- 172.21.15.114:0/4086747068 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8500047a0 con 0x7fa8600ffe90 2024-09-13T18:20:16.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.025+0000 7fa86731f640 1 -- 172.21.15.114:0/4086747068 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa86019d1b0 con 0x7fa8600ffe90 2024-09-13T18:20:16.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.026+0000 7fa86731f640 1 -- 172.21.15.114:0/4086747068 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8601061d0 con 0x7fa8600ffe90 2024-09-13T18:20:16.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.027+0000 7fa8567fc640 1 -- 172.21.15.114:0/4086747068 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fa850004020 con 0x7fa8600ffe90 2024-09-13T18:20:16.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.028+0000 7fa8567fc640 1 --2- 172.21.15.114:0/4086747068 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fa84c0a1090 0x7fa84c0a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:16.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.028+0000 7fa8650bb640 1 --2- 172.21.15.114:0/4086747068 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fa84c0a1090 0x7fa84c0a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:16.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.028+0000 7fa8567fc640 1 -- 172.21.15.114:0/4086747068 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fa8500cc420 con 0x7fa8600ffe90 2024-09-13T18:20:16.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.028+0000 7fa8650bb640 1 --2- 172.21.15.114:0/4086747068 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fa84c0a1090 0x7fa84c0a3550 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa8600ffcf0 tx=0x7fa85c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:16.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.032+0000 7fa8567fc640 1 -- 172.21.15.114:0/4086747068 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fa8500959c0 con 0x7fa8600ffe90 2024-09-13T18:20:16.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.197+0000 7fa86731f640 1 -- 172.21.15.114:0/4086747068 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa86002c410 con 0x7fa8600ffe90 2024-09-13T18:20:16.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.198+0000 7fa8567fc640 1 -- 172.21.15.114:0/4086747068 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa850098fe0 con 0x7fa8600ffe90 2024-09-13T18:20:16.200 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2024-09-13T18:20:16.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.200+0000 7fa86731f640 1 -- 172.21.15.114:0/4086747068 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fa84c0a1090 msgr2=0x7fa84c0a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:16.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.200+0000 7fa86731f640 1 --2- 172.21.15.114:0/4086747068 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fa84c0a1090 0x7fa84c0a3550 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa8600ffcf0 tx=0x7fa85c005f90 comp rx=0 tx=0).stop 2024-09-13T18:20:16.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.200+0000 7fa86731f640 1 -- 172.21.15.114:0/4086747068 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa8600ffe90 msgr2=0x7fa8601950e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:16.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.200+0000 7fa86731f640 1 --2- 172.21.15.114:0/4086747068 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa8600ffe90 0x7fa8601950e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa85000d950 tx=0x7fa85000de10 comp rx=0 tx=0).stop 2024-09-13T18:20:16.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.200+0000 7fa86731f640 1 -- 172.21.15.114:0/4086747068 shutdown_connections 2024-09-13T18:20:16.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.200+0000 7fa86731f640 1 --2- 172.21.15.114:0/4086747068 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fa84c0a1090 0x7fa84c0a3550 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:16.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.200+0000 7fa86731f640 1 --2- 172.21.15.114:0/4086747068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa860100850 0x7fa86019c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:16.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.200+0000 7fa86731f640 1 --2- 172.21.15.114:0/4086747068 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa8600ffe90 0x7fa8601950e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:16.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.200+0000 7fa86731f640 1 --2- 172.21.15.114:0/4086747068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8600feca0 0x7fa860194ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:16.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.200+0000 7fa86731f640 1 -- 172.21.15.114:0/4086747068 >> 172.21.15.114:0/4086747068 conn(0x7fa8600fa420 msgr2=0x7fa8601033a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:16.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.200+0000 7fa86731f640 1 -- 172.21.15.114:0/4086747068 shutdown_connections 2024-09-13T18:20:16.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.200+0000 7fa86731f640 1 -- 172.21.15.114:0/4086747068 wait complete. 2024-09-13T18:20:16.212 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:16 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2501322835' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T18:20:16.212 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:16 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/3801838984' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T18:20:16.212 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:16 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/1786206712' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T18:20:16.214 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T18:20:16.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.312+0000 7fa3c6718640 1 -- 172.21.15.114:0/715900580 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3c0100240 msgr2=0x7fa3c0100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:16.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.312+0000 7fa3c6718640 1 --2- 172.21.15.114:0/715900580 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3c0100240 0x7fa3c0100640 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0009900 tx=0x7fa3b002a410 comp rx=0 tx=0).stop 2024-09-13T18:20:16.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.312+0000 7fa3c6718640 1 -- 172.21.15.114:0/715900580 shutdown_connections 2024-09-13T18:20:16.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.312+0000 7fa3c6718640 1 --2- 172.21.15.114:0/715900580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3c00ff930 0x7fa3c01071d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:16.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.312+0000 7fa3c6718640 1 --2- 172.21.15.114:0/715900580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3c00fef70 0x7fa3c00ff3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:16.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.312+0000 7fa3c6718640 1 --2- 172.21.15.114:0/715900580 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3c0100240 0x7fa3c0100640 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:16.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.312+0000 7fa3c6718640 1 -- 172.21.15.114:0/715900580 >> 172.21.15.114:0/715900580 conn(0x7fa3c00f9bb0 msgr2=0x7fa3c00fbfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:16.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.312+0000 7fa3c6718640 1 -- 172.21.15.114:0/715900580 shutdown_connections 2024-09-13T18:20:16.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.312+0000 7fa3c6718640 1 -- 172.21.15.114:0/715900580 wait complete. 2024-09-13T18:20:16.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.313+0000 7fa3c6718640 1 Processor -- start 2024-09-13T18:20:16.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.313+0000 7fa3c6718640 1 -- start start 2024-09-13T18:20:16.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.313+0000 7fa3c6718640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3c00fef70 0x7fa3c00fe9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:16.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.313+0000 7fa3c6718640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3c00ff930 0x7fa3c019aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:16.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.313+0000 7fa3c6718640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3c0100240 0x7fa3c019d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:16.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.313+0000 7fa3c6718640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c019d870 con 0x7fa3c00fef70 2024-09-13T18:20:16.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.313+0000 7fa3c6718640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c019d9e0 con 0x7fa3c0100240 2024-09-13T18:20:16.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.313+0000 7fa3c6718640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c019dba0 con 0x7fa3c00ff930 2024-09-13T18:20:16.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.313+0000 7fa3bffff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3c00fef70 0x7fa3c00fe9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:16.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.313+0000 7fa3bffff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3c00fef70 0x7fa3c00fe9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39852/0 (socket says 172.21.15.114:39852) 2024-09-13T18:20:16.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.313+0000 7fa3bffff640 1 -- 172.21.15.114:0/4030567459 learned_addr learned my addr 172.21.15.114:0/4030567459 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:20:16.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.313+0000 7fa3c4cb5640 1 --2- 172.21.15.114:0/4030567459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3c0100240 0x7fa3c019d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:16.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.313+0000 7fa3bf7fe640 1 --2- 172.21.15.114:0/4030567459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3c00ff930 0x7fa3c019aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:16.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.313+0000 7fa3bffff640 1 -- 172.21.15.114:0/4030567459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3c00ff930 msgr2=0x7fa3c019aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:16.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.313+0000 7fa3bffff640 1 --2- 172.21.15.114:0/4030567459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3c00ff930 0x7fa3c019aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:16.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.313+0000 7fa3bffff640 1 -- 172.21.15.114:0/4030567459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3c0100240 msgr2=0x7fa3c019d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:16.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.313+0000 7fa3bffff640 1 --2- 172.21.15.114:0/4030567459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3c0100240 0x7fa3c019d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:16.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.314+0000 7fa3bffff640 1 -- 172.21.15.114:0/4030567459 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b0009560 con 0x7fa3c00fef70 2024-09-13T18:20:16.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.314+0000 7fa3c4cb5640 1 --2- 172.21.15.114:0/4030567459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3c0100240 0x7fa3c019d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:20:16.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.314+0000 7fa3bffff640 1 --2- 172.21.15.114:0/4030567459 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3c00fef70 0x7fa3c00fe9d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0005ab0 tx=0x7fa3b00052b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:16.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.314+0000 7fa3bd7fa640 1 -- 172.21.15.114:0/4030567459 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b0031d30 con 0x7fa3c00fef70 2024-09-13T18:20:16.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.314+0000 7fa3bd7fa640 1 -- 172.21.15.114:0/4030567459 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa3b0031e90 con 0x7fa3c00fef70 2024-09-13T18:20:16.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.314+0000 7fa3bd7fa640 1 -- 172.21.15.114:0/4030567459 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b0003a90 con 0x7fa3c00fef70 2024-09-13T18:20:16.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.314+0000 7fa3c6718640 1 -- 172.21.15.114:0/4030567459 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3c019ddc0 con 0x7fa3c00fef70 2024-09-13T18:20:16.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.314+0000 7fa3c6718640 1 -- 172.21.15.114:0/4030567459 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3c019e1c0 con 0x7fa3c00fef70 2024-09-13T18:20:16.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.315+0000 7fa3c6718640 1 -- 172.21.15.114:0/4030567459 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3c0106310 con 0x7fa3c00fef70 2024-09-13T18:20:16.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.317+0000 7fa3bd7fa640 1 -- 172.21.15.114:0/4030567459 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fa3b0003bf0 con 0x7fa3c00fef70 2024-09-13T18:20:16.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.317+0000 7fa3bd7fa640 1 --2- 172.21.15.114:0/4030567459 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fa3a80a1090 0x7fa3a80a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:16.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.317+0000 7fa3bf7fe640 1 --2- 172.21.15.114:0/4030567459 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fa3a80a1090 0x7fa3a80a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:16.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.317+0000 7fa3bd7fa640 1 -- 172.21.15.114:0/4030567459 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fa3b00ee100 con 0x7fa3c00fef70 2024-09-13T18:20:16.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.318+0000 7fa3bf7fe640 1 --2- 172.21.15.114:0/4030567459 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fa3a80a1090 0x7fa3a80a3550 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4005fd0 tx=0x7fa3b4005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:16.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.320+0000 7fa3bd7fa640 1 -- 172.21.15.114:0/4030567459 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fa3b007fe20 con 0x7fa3c00fef70 2024-09-13T18:20:16.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:16 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/2501322835' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T18:20:16.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:16 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/3801838984' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T18:20:16.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:16 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/1786206712' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T18:20:16.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.495+0000 7fa3c6718640 1 -- 172.21.15.114:0/4030567459 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa3c010a430 con 0x7fa3c00fef70 2024-09-13T18:20:16.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.496+0000 7fa3bd7fa640 1 -- 172.21.15.114:0/4030567459 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) v1 ==== 81+0+599 (secure 0 0 0) 0x7fa3b00b76a0 con 0x7fa3c00fef70 2024-09-13T18:20:16.498 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6801/4281866858 2024-09-14T18:13:25.823027+0000 2024-09-13T18:20:16.498 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6800/4281866858 2024-09-14T18:13:25.823027+0000 2024-09-13T18:20:16.499 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/2949422849 2024-09-14T18:13:25.823027+0000 2024-09-13T18:20:16.499 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/2034742251 2024-09-14T18:13:03.521116+0000 2024-09-13T18:20:16.499 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/722571999 2024-09-14T18:13:03.521116+0000 2024-09-13T18:20:16.499 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6801/830738794 2024-09-14T18:13:03.521116+0000 2024-09-13T18:20:16.499 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6800/830738794 2024-09-14T18:13:03.521116+0000 2024-09-13T18:20:16.499 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/2828364892 2024-09-14T18:13:25.823027+0000 2024-09-13T18:20:16.499 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/4257828906 2024-09-14T18:13:25.823027+0000 2024-09-13T18:20:16.499 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/3544951322 2024-09-14T18:13:03.521116+0000 2024-09-13T18:20:16.500 INFO:teuthology.orchestra.run.smithi114.stderr:listed 10 entries 2024-09-13T18:20:16.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.498+0000 7fa3c6718640 1 -- 172.21.15.114:0/4030567459 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fa3a80a1090 msgr2=0x7fa3a80a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:16.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.498+0000 7fa3c6718640 1 --2- 172.21.15.114:0/4030567459 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fa3a80a1090 0x7fa3a80a3550 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4005fd0 tx=0x7fa3b4005850 comp rx=0 tx=0).stop 2024-09-13T18:20:16.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.498+0000 7fa3c6718640 1 -- 172.21.15.114:0/4030567459 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3c00fef70 msgr2=0x7fa3c00fe9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:16.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.498+0000 7fa3c6718640 1 --2- 172.21.15.114:0/4030567459 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3c00fef70 0x7fa3c00fe9d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0005ab0 tx=0x7fa3b00052b0 comp rx=0 tx=0).stop 2024-09-13T18:20:16.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.498+0000 7fa3c6718640 1 -- 172.21.15.114:0/4030567459 shutdown_connections 2024-09-13T18:20:16.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.498+0000 7fa3c6718640 1 --2- 172.21.15.114:0/4030567459 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fa3a80a1090 0x7fa3a80a3550 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:16.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.498+0000 7fa3c6718640 1 --2- 172.21.15.114:0/4030567459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3c0100240 0x7fa3c019d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:16.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.498+0000 7fa3c6718640 1 --2- 172.21.15.114:0/4030567459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3c00ff930 0x7fa3c019aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:16.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.498+0000 7fa3c6718640 1 --2- 172.21.15.114:0/4030567459 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3c00fef70 0x7fa3c00fe9d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:16.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.498+0000 7fa3c6718640 1 -- 172.21.15.114:0/4030567459 >> 172.21.15.114:0/4030567459 conn(0x7fa3c00f9bb0 msgr2=0x7fa3c01034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:16.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.498+0000 7fa3c6718640 1 -- 172.21.15.114:0/4030567459 shutdown_connections 2024-09-13T18:20:16.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:16.498+0000 7fa3c6718640 1 -- 172.21.15.114:0/4030567459 wait complete. 2024-09-13T18:20:16.511 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi114.front.sepia.ceph.com... 2024-09-13T18:20:16.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:20:16.511 DEBUG:teuthology.orchestra.run.smithi114:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-13T18:20:16.534 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi114.stderr:ceph-fuse[73373]: fuse finished with error 0 and tester_r 0 2024-09-13T18:20:16.536 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:20:16.536 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:16 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/2501322835' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T18:20:16.536 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:16 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/3801838984' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T18:20:16.536 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:16 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/1786206712' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T18:20:17.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:17 smithi204 ceph-mon[34777]: pgmap v284: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 313 B/s wr, 0 op/s 2024-09-13T18:20:17.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:17 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/4086747068' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T18:20:17.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:17 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/4030567459' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-13T18:20:17.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:17 smithi114 ceph-mon[30897]: pgmap v284: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 313 B/s wr, 0 op/s 2024-09-13T18:20:17.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:17 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/4086747068' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T18:20:17.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:17 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/4030567459' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-13T18:20:17.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:17 smithi134 ceph-mon[34283]: pgmap v284: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 313 B/s wr, 0 op/s 2024-09-13T18:20:17.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:17 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/4086747068' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T18:20:17.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:17 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/4030567459' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-13T18:20:19.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:19 smithi204 ceph-mon[34777]: pgmap v285: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 268 B/s wr, 0 op/s 2024-09-13T18:20:19.469 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:19 smithi114 ceph-mon[30897]: pgmap v285: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 268 B/s wr, 0 op/s 2024-09-13T18:20:19.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:19 smithi134 ceph-mon[34283]: pgmap v285: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 268 B/s wr, 0 op/s 2024-09-13T18:20:21.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:21 smithi204 ceph-mon[34777]: pgmap v286: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-13T18:20:21.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:21 smithi114 ceph-mon[30897]: pgmap v286: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-13T18:20:21.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:21 smithi134 ceph-mon[34283]: pgmap v286: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-13T18:20:22.637 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi114.front.sepia.ceph.com 2024-09-13T18:20:22.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:20:22.637 DEBUG:teuthology.orchestra.run.smithi114:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-13T18:20:22.667 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:20:22.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.815+0000 7f106790f640 1 -- 172.21.15.114:0/1380201131 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10601008a0 msgr2=0x7f1060102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:22.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.815+0000 7f106790f640 1 --2- 172.21.15.114:0/1380201131 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10601008a0 0x7f1060102c90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f105c009900 tx=0x7f105c02a410 comp rx=0 tx=0).stop 2024-09-13T18:20:22.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.815+0000 7f106790f640 1 -- 172.21.15.114:0/1380201131 shutdown_connections 2024-09-13T18:20:22.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.815+0000 7f106790f640 1 --2- 172.21.15.114:0/1380201131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1060105b00 0x7f1060107ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:22.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.815+0000 7f106790f640 1 --2- 172.21.15.114:0/1380201131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10601031d0 0x7f10601055c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:22.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.815+0000 7f106790f640 1 --2- 172.21.15.114:0/1380201131 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10601008a0 0x7f1060102c90 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:22.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.815+0000 7f106790f640 1 -- 172.21.15.114:0/1380201131 >> 172.21.15.114:0/1380201131 conn(0x7f10600fa440 msgr2=0x7f10600fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:22.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.815+0000 7f106790f640 1 -- 172.21.15.114:0/1380201131 shutdown_connections 2024-09-13T18:20:22.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.815+0000 7f106790f640 1 -- 172.21.15.114:0/1380201131 wait complete. 2024-09-13T18:20:22.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.815+0000 7f106790f640 1 Processor -- start 2024-09-13T18:20:22.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.815+0000 7f106790f640 1 -- start start 2024-09-13T18:20:22.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.816+0000 7f106790f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10601008a0 0x7f1060195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:22.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.816+0000 7f106790f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10601031d0 0x7f10601962b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:22.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.816+0000 7f106790f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1060105b00 0x7f106019d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:22.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.816+0000 7f106790f640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106019d860 con 0x7f1060105b00 2024-09-13T18:20:22.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.816+0000 7f106790f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106019d9d0 con 0x7f10601008a0 2024-09-13T18:20:22.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.816+0000 7f106790f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106019db90 con 0x7f10601031d0 2024-09-13T18:20:22.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.816+0000 7f1065eac640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1060105b00 0x7f106019d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:22.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.816+0000 7f1065eac640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1060105b00 0x7f106019d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39868/0 (socket says 172.21.15.114:39868) 2024-09-13T18:20:22.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.816+0000 7f1065eac640 1 -- 172.21.15.114:0/380925753 learned_addr learned my addr 172.21.15.114:0/380925753 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:20:22.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.816+0000 7f1064eaa640 1 --2- 172.21.15.114:0/380925753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10601031d0 0x7f10601962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:22.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.816+0000 7f10656ab640 1 --2- 172.21.15.114:0/380925753 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10601008a0 0x7f1060195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:22.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.816+0000 7f1065eac640 1 -- 172.21.15.114:0/380925753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10601031d0 msgr2=0x7f10601962b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:22.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.816+0000 7f1065eac640 1 --2- 172.21.15.114:0/380925753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10601031d0 0x7f10601962b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:22.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.816+0000 7f1065eac640 1 -- 172.21.15.114:0/380925753 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10601008a0 msgr2=0x7f1060195d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:22.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.816+0000 7f1065eac640 1 --2- 172.21.15.114:0/380925753 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10601008a0 0x7f1060195d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:22.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.816+0000 7f1065eac640 1 -- 172.21.15.114:0/380925753 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f105c009560 con 0x7f1060105b00 2024-09-13T18:20:22.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.816+0000 7f10656ab640 1 --2- 172.21.15.114:0/380925753 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10601008a0 0x7f1060195d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:20:22.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.816+0000 7f1065eac640 1 --2- 172.21.15.114:0/380925753 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1060105b00 0x7f106019d320 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f105800cab0 tx=0x7f105800cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:22.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.816+0000 7f10567fc640 1 -- 172.21.15.114:0/380925753 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105800ae20 con 0x7f1060105b00 2024-09-13T18:20:22.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.816+0000 7f10567fc640 1 -- 172.21.15.114:0/380925753 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f105800f8e0 con 0x7f1060105b00 2024-09-13T18:20:22.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.816+0000 7f106790f640 1 -- 172.21.15.114:0/380925753 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f106019de70 con 0x7f1060105b00 2024-09-13T18:20:22.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.816+0000 7f10567fc640 1 -- 172.21.15.114:0/380925753 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1058018920 con 0x7f1060105b00 2024-09-13T18:20:22.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.816+0000 7f106790f640 1 -- 172.21.15.114:0/380925753 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f106019e2a0 con 0x7f1060105b00 2024-09-13T18:20:22.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.817+0000 7f106790f640 1 -- 172.21.15.114:0/380925753 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10600fbf30 con 0x7f1060105b00 2024-09-13T18:20:22.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.819+0000 7f10567fc640 1 -- 172.21.15.114:0/380925753 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f105800fa50 con 0x7f1060105b00 2024-09-13T18:20:22.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.819+0000 7f10567fc640 1 --2- 172.21.15.114:0/380925753 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f104c0a1090 0x7f104c0a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:22.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.819+0000 7f10656ab640 1 --2- 172.21.15.114:0/380925753 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f104c0a1090 0x7f104c0a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:22.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.819+0000 7f10567fc640 1 -- 172.21.15.114:0/380925753 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f10580ccfe0 con 0x7f1060105b00 2024-09-13T18:20:22.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.820+0000 7f10656ab640 1 --2- 172.21.15.114:0/380925753 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f104c0a1090 0x7f104c0a3550 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f105c0057c0 tx=0x7f105c005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:22.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.822+0000 7f10567fc640 1 -- 172.21.15.114:0/380925753 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f1058096610 con 0x7f1060105b00 2024-09-13T18:20:22.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.986+0000 7f106790f640 1 -- 172.21.15.114:0/380925753 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f106002c3e0 con 0x7f1060105b00 2024-09-13T18:20:22.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.987+0000 7f10567fc640 1 -- 172.21.15.114:0/380925753 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24029 (secure 0 0 0) 0x7f1058099c30 con 0x7f1060105b00 2024-09-13T18:20:22.991 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:20:22.991 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":89,"fsid":"97e7edf2-71fb-11ef-bceb-c7b262605968","created":"2024-09-13T18:12:36.360625+0000","modified":"2024-09-13T18:20:06.455681+0000","last_up_change":"2024-09-13T18:19:12.350370+0000","last_in_change":"2024-09-13T18:18:52.321596+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:15:40.129659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:20:00.024407+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:20:01.085549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T18:20:04.334216+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9914f01a-9f6f-4878-b955-83570199c354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6803","nonce":3499525826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6805","nonce":3499525826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6809","nonce":3499525826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6807","nonce":3499525826}]},"public_addr":"172.21.15.114:6803/3499525826","cluster_addr":"172.21.15.114:6805/3499525826","heartbeat_back_addr":"172.21.15.114:6809/3499525826","heartbeat_front_addr":"172.21.15.114:6807/3499525826","state":["exists","up"]},{"osd":1,"uuid":"ef65e012-7469-4c20-b354-17022bca63c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6801","nonce":4264264866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6803","nonce":4264264866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6807","nonce":4264264866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6805","nonce":4264264866}]},"public_addr":"172.21.15.134:6801/4264264866","cluster_addr":"172.21.15.134:6803/4264264866","heartbeat_back_addr":"172.21.15.134:6807/4264264866","heartbeat_front_addr":"172.21.15.134:6805/4264264866","state":["exists","up"]},{"osd":2,"uuid":"578213fd-715f-4be7-91fa-254f3d0e3022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6801","nonce":2288807707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6803","nonce":2288807707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6807","nonce":2288807707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6805","nonce":2288807707}]},"public_addr":"172.21.15.204:6801/2288807707","cluster_addr":"172.21.15.204:6803/2288807707","heartbeat_back_addr":"172.21.15.204:6807/2288807707","heartbeat_front_addr":"172.21.15.204:6805/2288807707","state":["exists","up"]},{"osd":3,"uuid":"b64aab3f-1e52-4828-8397-74af5e05f2cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6811","nonce":2904204254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6813","nonce":2904204254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6817","nonce":2904204254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6815","nonce":2904204254}]},"public_addr":"172.21.15.114:6811/2904204254","cluster_addr":"172.21.15.114:6813/2904204254","heartbeat_back_addr":"172.21.15.114:6817/2904204254","heartbeat_front_addr":"172.21.15.114:6815/2904204254","state":["exists","up"]},{"osd":4,"uuid":"f38943d6-39c0-42e1-b427-cb6bb90ac38a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6809","nonce":1773945333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6811","nonce":1773945333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6815","nonce":1773945333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6813","nonce":1773945333}]},"public_addr":"172.21.15.134:6809/1773945333","cluster_addr":"172.21.15.134:6811/1773945333","heartbeat_back_addr":"172.21.15.134:6815/1773945333","heartbeat_front_addr":"172.21.15.134:6813/1773945333","state":["exists","up"]},{"osd":5,"uuid":"bb38f361-8e00-4178-9752-5ddbf3af20e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6809","nonce":2371778055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6811","nonce":2371778055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6815","nonce":2371778055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6813","nonce":2371778055}]},"public_addr":"172.21.15.204:6809/2371778055","cluster_addr":"172.21.15.204:6811/2371778055","heartbeat_back_addr":"172.21.15.204:6815/2371778055","heartbeat_front_addr":"172.21.15.204:6813/2371778055","state":["exists","up"]},{"osd":6,"uuid":"93d40a03-e7ff-4a3f-9efa-5c5403c9983d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6819","nonce":1484325044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6821","nonce":1484325044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6825","nonce":1484325044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6823","nonce":1484325044}]},"public_addr":"172.21.15.114:6819/1484325044","cluster_addr":"172.21.15.114:6821/1484325044","heartbeat_back_addr":"172.21.15.114:6825/1484325044","heartbeat_front_addr":"172.21.15.114:6823/1484325044","state":["exists","up"]},{"osd":7,"uuid":"982f629a-1b95-4be1-8e75-5ddce1ad77b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6817","nonce":799246614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6819","nonce":799246614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6823","nonce":799246614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6821","nonce":799246614}]},"public_addr":"172.21.15.134:6817/799246614","cluster_addr":"172.21.15.134:6819/799246614","heartbeat_back_addr":"172.21.15.134:6823/799246614","heartbeat_front_addr":"172.21.15.134:6821/799246614","state":["exists","up"]},{"osd":8,"uuid":"409beb9b-8227-4dad-a4a0-f0e5ce70dc5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6817","nonce":858331805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6819","nonce":858331805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6823","nonce":858331805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6821","nonce":858331805}]},"public_addr":"172.21.15.204:6817/858331805","cluster_addr":"172.21.15.204:6819/858331805","heartbeat_back_addr":"172.21.15.204:6823/858331805","heartbeat_front_addr":"172.21.15.204:6821/858331805","state":["exists","up"]},{"osd":9,"uuid":"a345a15c-b1fc-4877-b445-418eebf0d0ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6827","nonce":2172392854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6829","nonce":2172392854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6833","nonce":2172392854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6831","nonce":2172392854}]},"public_addr":"172.21.15.114:6827/2172392854","cluster_addr":"172.21.15.114:6829/2172392854","heartbeat_back_addr":"172.21.15.114:6833/2172392854","heartbeat_front_addr":"172.21.15.114:6831/2172392854","state":["exists","up"]},{"osd":10,"uuid":"464d995f-d90b-4caa-addb-1f7562bd74b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6825","nonce":3073513468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6827","nonce":3073513468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6831","nonce":3073513468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6829","nonce":3073513468}]},"public_addr":"172.21.15.134:6825/3073513468","cluster_addr":"172.21.15.134:6827/3073513468","heartbeat_back_addr":"172.21.15.134:6831/3073513468","heartbeat_front_addr":"172.21.15.134:6829/3073513468","state":["exists","up"]},{"osd":11,"uuid":"71dd794d-326c-477f-8d02-ee32bff1350d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6825","nonce":3239521765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6827","nonce":3239521765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6831","nonce":3239521765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6829","nonce":3239521765}]},"public_addr":"172.21.15.204:6825/3239521765","cluster_addr":"172.21.15.204:6827/3239521765","heartbeat_back_addr":"172.21.15.204:6831/3239521765","heartbeat_front_addr":"172.21.15.204:6829/3239521765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:14:54.542687+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:15:15.746055+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:15:38.371131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:01.383932+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:24.796237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:48.422214+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:11.284323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:34.570365+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:58.232855+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:18:21.609596+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:18:45.565859+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:19:10.017969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:6801/4281866858":"2024-09-14T18:13:25.823027+0000","172.21.15.114:6800/4281866858":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2949422849":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2034742251":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/722571999":"2024-09-14T18:13:03.521116+0000","172.21.15.114:6801/830738794":"2024-09-14T18:13:03.521116+0000","172.21.15.114:6800/830738794":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/2828364892":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/4257828906":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/3544951322":"2024-09-14T18:13:03.521116+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:20:22.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.989+0000 7f106790f640 1 -- 172.21.15.114:0/380925753 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f104c0a1090 msgr2=0x7f104c0a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:22.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.989+0000 7f106790f640 1 --2- 172.21.15.114:0/380925753 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f104c0a1090 0x7f104c0a3550 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f105c0057c0 tx=0x7f105c005a20 comp rx=0 tx=0).stop 2024-09-13T18:20:22.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.989+0000 7f106790f640 1 -- 172.21.15.114:0/380925753 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1060105b00 msgr2=0x7f106019d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:22.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.989+0000 7f106790f640 1 --2- 172.21.15.114:0/380925753 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1060105b00 0x7f106019d320 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f105800cab0 tx=0x7f105800cf70 comp rx=0 tx=0).stop 2024-09-13T18:20:22.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.989+0000 7f106790f640 1 -- 172.21.15.114:0/380925753 shutdown_connections 2024-09-13T18:20:22.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.989+0000 7f106790f640 1 --2- 172.21.15.114:0/380925753 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f104c0a1090 0x7f104c0a3550 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:22.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.989+0000 7f106790f640 1 --2- 172.21.15.114:0/380925753 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1060105b00 0x7f106019d320 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:22.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.989+0000 7f106790f640 1 --2- 172.21.15.114:0/380925753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10601031d0 0x7f10601962b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:22.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.989+0000 7f106790f640 1 --2- 172.21.15.114:0/380925753 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10601008a0 0x7f1060195d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:22.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.989+0000 7f106790f640 1 -- 172.21.15.114:0/380925753 >> 172.21.15.114:0/380925753 conn(0x7f10600fa440 msgr2=0x7f1060101360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:22.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.989+0000 7f106790f640 1 -- 172.21.15.114:0/380925753 shutdown_connections 2024-09-13T18:20:22.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:22.989+0000 7f106790f640 1 -- 172.21.15.114:0/380925753 wait complete. 2024-09-13T18:20:23.004 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:20:23.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.100+0000 7faef9d18640 1 -- 172.21.15.114:0/632364246 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faef4100850 msgr2=0x7faef41070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:23.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.100+0000 7faef9d18640 1 --2- 172.21.15.114:0/632364246 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faef4100850 0x7faef41070e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7faeec009ca0 tx=0x7faeec02a8c0 comp rx=0 tx=0).stop 2024-09-13T18:20:23.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.100+0000 7faef9d18640 1 -- 172.21.15.114:0/632364246 shutdown_connections 2024-09-13T18:20:23.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.100+0000 7faef9d18640 1 --2- 172.21.15.114:0/632364246 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faef4100850 0x7faef41070e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:23.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.100+0000 7faef9d18640 1 --2- 172.21.15.114:0/632364246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faef40ffe90 0x7faef4100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:23.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.100+0000 7faef9d18640 1 --2- 172.21.15.114:0/632364246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faef40feca0 0x7faef40ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:23.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.100+0000 7faef9d18640 1 -- 172.21.15.114:0/632364246 >> 172.21.15.114:0/632364246 conn(0x7faef40fa420 msgr2=0x7faef40fc840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:23.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.100+0000 7faef9d18640 1 -- 172.21.15.114:0/632364246 shutdown_connections 2024-09-13T18:20:23.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.100+0000 7faef9d18640 1 -- 172.21.15.114:0/632364246 wait complete. 2024-09-13T18:20:23.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.101+0000 7faef9d18640 1 Processor -- start 2024-09-13T18:20:23.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.101+0000 7faef9d18640 1 -- start start 2024-09-13T18:20:23.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.101+0000 7faef9d18640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faef40feca0 0x7faef4195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:23.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.101+0000 7faef9d18640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faef40ffe90 0x7faef4196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:23.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.101+0000 7faef9d18640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faef4100850 0x7faef419d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:23.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.101+0000 7faef9d18640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef419d820 con 0x7faef4100850 2024-09-13T18:20:23.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.101+0000 7faef9d18640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef419d990 con 0x7faef40feca0 2024-09-13T18:20:23.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.101+0000 7faef9d18640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef419db50 con 0x7faef40ffe90 2024-09-13T18:20:23.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.101+0000 7faef3fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faef4100850 0x7faef419d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:23.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.101+0000 7faef3fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faef4100850 0x7faef419d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39878/0 (socket says 172.21.15.114:39878) 2024-09-13T18:20:23.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.101+0000 7faef3fff640 1 -- 172.21.15.114:0/393088203 learned_addr learned my addr 172.21.15.114:0/393088203 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:20:23.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.101+0000 7faeebfff640 1 --2- 172.21.15.114:0/393088203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faef40ffe90 0x7faef4196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:23.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.101+0000 7faef37fe640 1 --2- 172.21.15.114:0/393088203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faef40feca0 0x7faef4195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:23.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.101+0000 7faef3fff640 1 -- 172.21.15.114:0/393088203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faef40ffe90 msgr2=0x7faef4196280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:23.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.101+0000 7faef3fff640 1 --2- 172.21.15.114:0/393088203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faef40ffe90 0x7faef4196280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:23.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.101+0000 7faef3fff640 1 -- 172.21.15.114:0/393088203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faef40feca0 msgr2=0x7faef4195d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:23.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.101+0000 7faef3fff640 1 --2- 172.21.15.114:0/393088203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faef40feca0 0x7faef4195d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:23.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.101+0000 7faef3fff640 1 -- 172.21.15.114:0/393088203 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeec009900 con 0x7faef4100850 2024-09-13T18:20:23.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.101+0000 7faef37fe640 1 --2- 172.21.15.114:0/393088203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faef40feca0 0x7faef4195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:20:23.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.101+0000 7faeebfff640 1 --2- 172.21.15.114:0/393088203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faef40ffe90 0x7faef4196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:20:23.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.101+0000 7faef3fff640 1 --2- 172.21.15.114:0/393088203 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faef4100850 0x7faef419d2b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7faeec009dc0 tx=0x7faeec02f310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:23.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.102+0000 7faef17fa640 1 -- 172.21.15.114:0/393088203 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeec003bf0 con 0x7faef4100850 2024-09-13T18:20:23.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.102+0000 7faef17fa640 1 -- 172.21.15.114:0/393088203 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faeec040070 con 0x7faef4100850 2024-09-13T18:20:23.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.102+0000 7faef17fa640 1 -- 172.21.15.114:0/393088203 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeec004460 con 0x7faef4100850 2024-09-13T18:20:23.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.102+0000 7faef9d18640 1 -- 172.21.15.114:0/393088203 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faef419ddd0 con 0x7faef4100850 2024-09-13T18:20:23.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.102+0000 7faef9d18640 1 -- 172.21.15.114:0/393088203 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faef419e1d0 con 0x7faef4100850 2024-09-13T18:20:23.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.103+0000 7faef9d18640 1 -- 172.21.15.114:0/393088203 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faef4106220 con 0x7faef4100850 2024-09-13T18:20:23.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.104+0000 7faef17fa640 1 -- 172.21.15.114:0/393088203 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7faeec042030 con 0x7faef4100850 2024-09-13T18:20:23.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.104+0000 7faef17fa640 1 --2- 172.21.15.114:0/393088203 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7faee00a1090 0x7faee00a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:23.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.104+0000 7faef37fe640 1 --2- 172.21.15.114:0/393088203 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7faee00a1090 0x7faee00a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:23.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.104+0000 7faef17fa640 1 -- 172.21.15.114:0/393088203 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7faeec0ee300 con 0x7faef4100850 2024-09-13T18:20:23.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.105+0000 7faef37fe640 1 --2- 172.21.15.114:0/393088203 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7faee00a1090 0x7faee00a3550 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7faef4197210 tx=0x7faee4008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:23.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.108+0000 7faef17fa640 1 -- 172.21.15.114:0/393088203 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7faeec0b7810 con 0x7faef4100850 2024-09-13T18:20:23.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.273+0000 7faef9d18640 1 -- 172.21.15.114:0/393088203 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faef410b760 con 0x7faef4100850 2024-09-13T18:20:23.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.274+0000 7faef17fa640 1 -- 172.21.15.114:0/393088203 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24029 (secure 0 0 0) 0x7faeec0b9740 con 0x7faef4100850 2024-09-13T18:20:23.278 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:20:23.278 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":89,"fsid":"97e7edf2-71fb-11ef-bceb-c7b262605968","created":"2024-09-13T18:12:36.360625+0000","modified":"2024-09-13T18:20:06.455681+0000","last_up_change":"2024-09-13T18:19:12.350370+0000","last_in_change":"2024-09-13T18:18:52.321596+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:15:40.129659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:20:00.024407+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:20:01.085549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T18:20:04.334216+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9914f01a-9f6f-4878-b955-83570199c354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6803","nonce":3499525826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6805","nonce":3499525826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6809","nonce":3499525826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6807","nonce":3499525826}]},"public_addr":"172.21.15.114:6803/3499525826","cluster_addr":"172.21.15.114:6805/3499525826","heartbeat_back_addr":"172.21.15.114:6809/3499525826","heartbeat_front_addr":"172.21.15.114:6807/3499525826","state":["exists","up"]},{"osd":1,"uuid":"ef65e012-7469-4c20-b354-17022bca63c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6801","nonce":4264264866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6803","nonce":4264264866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6807","nonce":4264264866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6805","nonce":4264264866}]},"public_addr":"172.21.15.134:6801/4264264866","cluster_addr":"172.21.15.134:6803/4264264866","heartbeat_back_addr":"172.21.15.134:6807/4264264866","heartbeat_front_addr":"172.21.15.134:6805/4264264866","state":["exists","up"]},{"osd":2,"uuid":"578213fd-715f-4be7-91fa-254f3d0e3022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6801","nonce":2288807707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6803","nonce":2288807707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6807","nonce":2288807707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6805","nonce":2288807707}]},"public_addr":"172.21.15.204:6801/2288807707","cluster_addr":"172.21.15.204:6803/2288807707","heartbeat_back_addr":"172.21.15.204:6807/2288807707","heartbeat_front_addr":"172.21.15.204:6805/2288807707","state":["exists","up"]},{"osd":3,"uuid":"b64aab3f-1e52-4828-8397-74af5e05f2cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6811","nonce":2904204254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6813","nonce":2904204254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6817","nonce":2904204254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6815","nonce":2904204254}]},"public_addr":"172.21.15.114:6811/2904204254","cluster_addr":"172.21.15.114:6813/2904204254","heartbeat_back_addr":"172.21.15.114:6817/2904204254","heartbeat_front_addr":"172.21.15.114:6815/2904204254","state":["exists","up"]},{"osd":4,"uuid":"f38943d6-39c0-42e1-b427-cb6bb90ac38a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6809","nonce":1773945333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6811","nonce":1773945333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6815","nonce":1773945333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6813","nonce":1773945333}]},"public_addr":"172.21.15.134:6809/1773945333","cluster_addr":"172.21.15.134:6811/1773945333","heartbeat_back_addr":"172.21.15.134:6815/1773945333","heartbeat_front_addr":"172.21.15.134:6813/1773945333","state":["exists","up"]},{"osd":5,"uuid":"bb38f361-8e00-4178-9752-5ddbf3af20e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6809","nonce":2371778055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6811","nonce":2371778055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6815","nonce":2371778055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6813","nonce":2371778055}]},"public_addr":"172.21.15.204:6809/2371778055","cluster_addr":"172.21.15.204:6811/2371778055","heartbeat_back_addr":"172.21.15.204:6815/2371778055","heartbeat_front_addr":"172.21.15.204:6813/2371778055","state":["exists","up"]},{"osd":6,"uuid":"93d40a03-e7ff-4a3f-9efa-5c5403c9983d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6819","nonce":1484325044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6821","nonce":1484325044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6825","nonce":1484325044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6823","nonce":1484325044}]},"public_addr":"172.21.15.114:6819/1484325044","cluster_addr":"172.21.15.114:6821/1484325044","heartbeat_back_addr":"172.21.15.114:6825/1484325044","heartbeat_front_addr":"172.21.15.114:6823/1484325044","state":["exists","up"]},{"osd":7,"uuid":"982f629a-1b95-4be1-8e75-5ddce1ad77b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6817","nonce":799246614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6819","nonce":799246614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6823","nonce":799246614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6821","nonce":799246614}]},"public_addr":"172.21.15.134:6817/799246614","cluster_addr":"172.21.15.134:6819/799246614","heartbeat_back_addr":"172.21.15.134:6823/799246614","heartbeat_front_addr":"172.21.15.134:6821/799246614","state":["exists","up"]},{"osd":8,"uuid":"409beb9b-8227-4dad-a4a0-f0e5ce70dc5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6817","nonce":858331805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6819","nonce":858331805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6823","nonce":858331805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6821","nonce":858331805}]},"public_addr":"172.21.15.204:6817/858331805","cluster_addr":"172.21.15.204:6819/858331805","heartbeat_back_addr":"172.21.15.204:6823/858331805","heartbeat_front_addr":"172.21.15.204:6821/858331805","state":["exists","up"]},{"osd":9,"uuid":"a345a15c-b1fc-4877-b445-418eebf0d0ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6827","nonce":2172392854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6829","nonce":2172392854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6833","nonce":2172392854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6831","nonce":2172392854}]},"public_addr":"172.21.15.114:6827/2172392854","cluster_addr":"172.21.15.114:6829/2172392854","heartbeat_back_addr":"172.21.15.114:6833/2172392854","heartbeat_front_addr":"172.21.15.114:6831/2172392854","state":["exists","up"]},{"osd":10,"uuid":"464d995f-d90b-4caa-addb-1f7562bd74b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6825","nonce":3073513468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6827","nonce":3073513468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6831","nonce":3073513468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6829","nonce":3073513468}]},"public_addr":"172.21.15.134:6825/3073513468","cluster_addr":"172.21.15.134:6827/3073513468","heartbeat_back_addr":"172.21.15.134:6831/3073513468","heartbeat_front_addr":"172.21.15.134:6829/3073513468","state":["exists","up"]},{"osd":11,"uuid":"71dd794d-326c-477f-8d02-ee32bff1350d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6825","nonce":3239521765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6827","nonce":3239521765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6831","nonce":3239521765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6829","nonce":3239521765}]},"public_addr":"172.21.15.204:6825/3239521765","cluster_addr":"172.21.15.204:6827/3239521765","heartbeat_back_addr":"172.21.15.204:6831/3239521765","heartbeat_front_addr":"172.21.15.204:6829/3239521765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:14:54.542687+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:15:15.746055+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:15:38.371131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:01.383932+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:24.796237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:48.422214+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:11.284323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:34.570365+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:58.232855+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:18:21.609596+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:18:45.565859+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:19:10.017969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:6801/4281866858":"2024-09-14T18:13:25.823027+0000","172.21.15.114:6800/4281866858":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2949422849":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2034742251":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/722571999":"2024-09-14T18:13:03.521116+0000","172.21.15.114:6801/830738794":"2024-09-14T18:13:03.521116+0000","172.21.15.114:6800/830738794":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/2828364892":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/4257828906":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/3544951322":"2024-09-14T18:13:03.521116+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:20:23.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.276+0000 7faef9d18640 1 -- 172.21.15.114:0/393088203 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7faee00a1090 msgr2=0x7faee00a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:23.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.276+0000 7faef9d18640 1 --2- 172.21.15.114:0/393088203 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7faee00a1090 0x7faee00a3550 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7faef4197210 tx=0x7faee4008040 comp rx=0 tx=0).stop 2024-09-13T18:20:23.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.276+0000 7faef9d18640 1 -- 172.21.15.114:0/393088203 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faef4100850 msgr2=0x7faef419d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:23.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.276+0000 7faef9d18640 1 --2- 172.21.15.114:0/393088203 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faef4100850 0x7faef419d2b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7faeec009dc0 tx=0x7faeec02f310 comp rx=0 tx=0).stop 2024-09-13T18:20:23.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.276+0000 7faef9d18640 1 -- 172.21.15.114:0/393088203 shutdown_connections 2024-09-13T18:20:23.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.276+0000 7faef9d18640 1 --2- 172.21.15.114:0/393088203 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7faee00a1090 0x7faee00a3550 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:23.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.276+0000 7faef9d18640 1 --2- 172.21.15.114:0/393088203 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faef4100850 0x7faef419d2b0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:23.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.276+0000 7faef9d18640 1 --2- 172.21.15.114:0/393088203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faef40ffe90 0x7faef4196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:23.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.276+0000 7faef9d18640 1 --2- 172.21.15.114:0/393088203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faef40feca0 0x7faef4195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:23.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.276+0000 7faef9d18640 1 -- 172.21.15.114:0/393088203 >> 172.21.15.114:0/393088203 conn(0x7faef40fa420 msgr2=0x7faef41033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:23.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.277+0000 7faef9d18640 1 -- 172.21.15.114:0/393088203 shutdown_connections 2024-09-13T18:20:23.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.277+0000 7faef9d18640 1 -- 172.21.15.114:0/393088203 wait complete. 2024-09-13T18:20:23.289 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:23 smithi114 ceph-mon[30897]: pgmap v287: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-13T18:20:23.289 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:23 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/380925753' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:20:23.291 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-13T18:20:23.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.388+0000 7f460a480640 1 -- 172.21.15.114:0/467610554 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46040ffe60 msgr2=0x7f46041002e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:23.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.388+0000 7f460a480640 1 --2- 172.21.15.114:0/467610554 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46040ffe60 0x7f46041002e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f45f8009900 tx=0x7f45f802a410 comp rx=0 tx=0).stop 2024-09-13T18:20:23.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.388+0000 7f460a480640 1 -- 172.21.15.114:0/467610554 shutdown_connections 2024-09-13T18:20:23.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.388+0000 7f460a480640 1 --2- 172.21.15.114:0/467610554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4604100820 0x7f46041070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:23.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.388+0000 7f460a480640 1 --2- 172.21.15.114:0/467610554 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46040ffe60 0x7f46041002e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:23.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.388+0000 7f460a480640 1 --2- 172.21.15.114:0/467610554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46040fec70 0x7f46040ff070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:23.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.388+0000 7f460a480640 1 -- 172.21.15.114:0/467610554 >> 172.21.15.114:0/467610554 conn(0x7f46040fa3d0 msgr2=0x7f46040fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:23.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.388+0000 7f460a480640 1 -- 172.21.15.114:0/467610554 shutdown_connections 2024-09-13T18:20:23.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.388+0000 7f460a480640 1 -- 172.21.15.114:0/467610554 wait complete. 2024-09-13T18:20:23.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.389+0000 7f460a480640 1 Processor -- start 2024-09-13T18:20:23.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.389+0000 7f460a480640 1 -- start start 2024-09-13T18:20:23.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.389+0000 7f460a480640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46040fec70 0x7f4604195c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:23.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.389+0000 7f460a480640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46040ffe60 0x7f46041961d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:23.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.389+0000 7f460a480640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4604100820 0x7f460419d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:23.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.389+0000 7f460a480640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460419d7a0 con 0x7f4604100820 2024-09-13T18:20:23.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.389+0000 7f460a480640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460419d910 con 0x7f46040fec70 2024-09-13T18:20:23.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.389+0000 7f460a480640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460419dad0 con 0x7f46040ffe60 2024-09-13T18:20:23.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.390+0000 7f4608a1d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4604100820 0x7f460419d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:23.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.390+0000 7f4608a1d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4604100820 0x7f460419d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60610/0 (socket says 172.21.15.114:60610) 2024-09-13T18:20:23.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.390+0000 7f4608a1d640 1 -- 172.21.15.114:0/3494754112 learned_addr learned my addr 172.21.15.114:0/3494754112 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:20:23.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.390+0000 7f4603fff640 1 --2- 172.21.15.114:0/3494754112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46040fec70 0x7f4604195c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:23.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.390+0000 7f46037fe640 1 --2- 172.21.15.114:0/3494754112 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46040ffe60 0x7f46041961d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:23.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.390+0000 7f4608a1d640 1 -- 172.21.15.114:0/3494754112 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46040ffe60 msgr2=0x7f46041961d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:23.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.390+0000 7f4608a1d640 1 --2- 172.21.15.114:0/3494754112 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46040ffe60 0x7f46041961d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:23.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.390+0000 7f4608a1d640 1 -- 172.21.15.114:0/3494754112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46040fec70 msgr2=0x7f4604195c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:23.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.390+0000 7f4608a1d640 1 --2- 172.21.15.114:0/3494754112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46040fec70 0x7f4604195c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:23.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.390+0000 7f4608a1d640 1 -- 172.21.15.114:0/3494754112 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f8009560 con 0x7f4604100820 2024-09-13T18:20:23.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.390+0000 7f46037fe640 1 --2- 172.21.15.114:0/3494754112 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46040ffe60 0x7f46041961d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:20:23.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.390+0000 7f4608a1d640 1 --2- 172.21.15.114:0/3494754112 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4604100820 0x7f460419d230 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f45fc00eaa0 tx=0x7f45fc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:23.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.390+0000 7f46017fa640 1 -- 172.21.15.114:0/3494754112 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45fc00cdb0 con 0x7f4604100820 2024-09-13T18:20:23.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.390+0000 7f46017fa640 1 -- 172.21.15.114:0/3494754112 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f45fc005020 con 0x7f4604100820 2024-09-13T18:20:23.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.390+0000 7f460a480640 1 -- 172.21.15.114:0/3494754112 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f460419ddb0 con 0x7f4604100820 2024-09-13T18:20:23.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.390+0000 7f46017fa640 1 -- 172.21.15.114:0/3494754112 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45fc0116d0 con 0x7f4604100820 2024-09-13T18:20:23.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.390+0000 7f460a480640 1 -- 172.21.15.114:0/3494754112 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f460419e230 con 0x7f4604100820 2024-09-13T18:20:23.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.391+0000 7f460a480640 1 -- 172.21.15.114:0/3494754112 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4604106150 con 0x7f4604100820 2024-09-13T18:20:23.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.392+0000 7f46017fa640 1 -- 172.21.15.114:0/3494754112 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f45fc005190 con 0x7f4604100820 2024-09-13T18:20:23.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.393+0000 7f46017fa640 1 --2- 172.21.15.114:0/3494754112 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f45ec0a1090 0x7f45ec0a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:23.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.393+0000 7f46017fa640 1 -- 172.21.15.114:0/3494754112 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f45fc0ccfb0 con 0x7f4604100820 2024-09-13T18:20:23.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.393+0000 7f4603fff640 1 --2- 172.21.15.114:0/3494754112 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f45ec0a1090 0x7f45ec0a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:23.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.393+0000 7f4603fff640 1 --2- 172.21.15.114:0/3494754112 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f45ec0a1090 0x7f45ec0a3550 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4604196b60 tx=0x7f45f400b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:23.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.396+0000 7f46017fa640 1 -- 172.21.15.114:0/3494754112 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f45fc096550 con 0x7f4604100820 2024-09-13T18:20:23.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:23 smithi204 ceph-mon[34777]: pgmap v287: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-13T18:20:23.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:23 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/380925753' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:20:23.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:23.595+0000 7f460a480640 1 -- 172.21.15.114:0/3494754112 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f460410b560 con 0x7f4604100820 2024-09-13T18:20:23.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:23 smithi134 ceph-mon[34283]: pgmap v287: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-13T18:20:23.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:23 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/380925753' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:20:24.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:24.157+0000 7f46017fa640 1 -- 172.21.15.114:0/3494754112 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7f45fc01f0b0 con 0x7f4604100820 2024-09-13T18:20:24.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:24.160+0000 7f460a480640 1 -- 172.21.15.114:0/3494754112 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f45ec0a1090 msgr2=0x7f45ec0a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:24.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:24.160+0000 7f460a480640 1 --2- 172.21.15.114:0/3494754112 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f45ec0a1090 0x7f45ec0a3550 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4604196b60 tx=0x7f45f400b390 comp rx=0 tx=0).stop 2024-09-13T18:20:24.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:24.160+0000 7f460a480640 1 -- 172.21.15.114:0/3494754112 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4604100820 msgr2=0x7f460419d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:24.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:24.160+0000 7f460a480640 1 --2- 172.21.15.114:0/3494754112 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4604100820 0x7f460419d230 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f45fc00eaa0 tx=0x7f45fc00ef60 comp rx=0 tx=0).stop 2024-09-13T18:20:24.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:24.160+0000 7f460a480640 1 -- 172.21.15.114:0/3494754112 shutdown_connections 2024-09-13T18:20:24.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:24.160+0000 7f460a480640 1 --2- 172.21.15.114:0/3494754112 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f45ec0a1090 0x7f45ec0a3550 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:24.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:24.160+0000 7f460a480640 1 --2- 172.21.15.114:0/3494754112 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4604100820 0x7f460419d230 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:24.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:24.160+0000 7f460a480640 1 --2- 172.21.15.114:0/3494754112 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46040ffe60 0x7f46041961d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:24.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:24.160+0000 7f460a480640 1 --2- 172.21.15.114:0/3494754112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46040fec70 0x7f4604195c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:24.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:24.160+0000 7f460a480640 1 -- 172.21.15.114:0/3494754112 >> 172.21.15.114:0/3494754112 conn(0x7f46040fa3d0 msgr2=0x7f4604103320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:24.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:24.160+0000 7f460a480640 1 -- 172.21.15.114:0/3494754112 shutdown_connections 2024-09-13T18:20:24.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:24.160+0000 7f460a480640 1 -- 172.21.15.114:0/3494754112 wait complete. 2024-09-13T18:20:24.174 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-09-13T18:20:24.174 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-09-13T18:20:24.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:24.272+0000 7f911400d640 1 -- 172.21.15.114:0/3056145123 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f910c105a40 msgr2=0x7f910c107eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:24.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:24.272+0000 7f911400d640 1 --2- 172.21.15.114:0/3056145123 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f910c105a40 0x7f910c107eb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9104009900 tx=0x7f910402a410 comp rx=0 tx=0).stop 2024-09-13T18:20:24.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:24.273+0000 7f911400d640 1 -- 172.21.15.114:0/3056145123 shutdown_connections 2024-09-13T18:20:24.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:24.273+0000 7f911400d640 1 --2- 172.21.15.114:0/3056145123 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f910c105a40 0x7f910c107eb0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:24.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:24.273+0000 7f911400d640 1 --2- 172.21.15.114:0/3056145123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f910c103110 0x7f910c105500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:24.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:24.273+0000 7f911400d640 1 --2- 172.21.15.114:0/3056145123 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f910c1007e0 0x7f910c102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:24.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:24.273+0000 7f911400d640 1 -- 172.21.15.114:0/3056145123 >> 172.21.15.114:0/3056145123 conn(0x7f910c0fa3e0 msgr2=0x7f910c0fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:24.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:24.273+0000 7f911400d640 1 -- 172.21.15.114:0/3056145123 shutdown_connections 2024-09-13T18:20:24.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:24.273+0000 7f911400d640 1 -- 172.21.15.114:0/3056145123 wait complete. 2024-09-13T18:20:24.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:24.273+0000 7f911400d640 1 Processor -- start 2024-09-13T18:20:24.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:24.273+0000 7f911400d640 1 -- start start 2024-09-13T18:20:24.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:24.273+0000 7f911400d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f910c1007e0 0x7f910c100150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:24.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:24.273+0000 7f911400d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f910c103110 0x7f910c0fe790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:24.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:24.273+0000 7f911400d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f910c105a40 0x7f910c0fecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:24.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:24.273+0000 7f911400d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f910c0ff430 con 0x7f910c103110 2024-09-13T18:20:24.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:24.273+0000 7f911400d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f910c0ff5a0 con 0x7f910c1007e0 2024-09-13T18:20:24.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:24.273+0000 7f911400d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f910c19fd00 con 0x7f910c105a40 2024-09-13T18:20:24.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:24.273+0000 7f91115a8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f910c103110 0x7f910c0fe790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:24.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:24.274+0000 7f91115a8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f910c103110 0x7f910c0fe790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60634/0 (socket says 172.21.15.114:60634) 2024-09-13T18:20:24.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:24.274+0000 7f91115a8640 1 -- 172.21.15.114:0/3925565345 learned_addr learned my addr 172.21.15.114:0/3925565345 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:20:24.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:24.274+0000 7f9111da9640 1 --2- 172.21.15.114:0/3925565345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f910c1007e0 0x7f910c100150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:24.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:24.274+0000 7f91125aa640 1 --2- 172.21.15.114:0/3925565345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f910c105a40 0x7f910c0fecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:24.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:24.274+0000 7f91115a8640 1 -- 172.21.15.114:0/3925565345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f910c105a40 msgr2=0x7f910c0fecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:24.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:24.274+0000 7f91115a8640 1 --2- 172.21.15.114:0/3925565345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f910c105a40 0x7f910c0fecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:24.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:24.274+0000 7f91115a8640 1 -- 172.21.15.114:0/3925565345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f910c1007e0 msgr2=0x7f910c100150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:24.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:24.274+0000 7f91115a8640 1 --2- 172.21.15.114:0/3925565345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f910c1007e0 0x7f910c100150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:24.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:24.274+0000 7f91115a8640 1 -- 172.21.15.114:0/3925565345 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9104009560 con 0x7f910c103110 2024-09-13T18:20:24.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:24.274+0000 7f91115a8640 1 --2- 172.21.15.114:0/3925565345 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f910c103110 0x7f910c0fe790 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f910000c9a0 tx=0x7f910000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:24.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:24.274+0000 7f910affd640 1 -- 172.21.15.114:0/3925565345 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9100007bb0 con 0x7f910c103110 2024-09-13T18:20:24.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:24.274+0000 7f910affd640 1 -- 172.21.15.114:0/3925565345 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9100007d10 con 0x7f910c103110 2024-09-13T18:20:24.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:24.274+0000 7f911400d640 1 -- 172.21.15.114:0/3925565345 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f910c19ffb0 con 0x7f910c103110 2024-09-13T18:20:24.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:24.274+0000 7f910affd640 1 -- 172.21.15.114:0/3925565345 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9100004e10 con 0x7f910c103110 2024-09-13T18:20:24.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:24.274+0000 7f911400d640 1 -- 172.21.15.114:0/3925565345 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f910c1a0430 con 0x7f910c103110 2024-09-13T18:20:24.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:24.275+0000 7f911400d640 1 -- 172.21.15.114:0/3925565345 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f910c0fbed0 con 0x7f910c103110 2024-09-13T18:20:24.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:24.276+0000 7f910affd640 1 -- 172.21.15.114:0/3925565345 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f9100004f70 con 0x7f910c103110 2024-09-13T18:20:24.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:24.277+0000 7f910affd640 1 --2- 172.21.15.114:0/3925565345 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f90f40a1090 0x7f90f40a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:24.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:24.277+0000 7f9111da9640 1 --2- 172.21.15.114:0/3925565345 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f90f40a1090 0x7f90f40a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:24.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:24.277+0000 7f910affd640 1 -- 172.21.15.114:0/3925565345 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f91000ccb30 con 0x7f910c103110 2024-09-13T18:20:24.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:24.277+0000 7f9111da9640 1 --2- 172.21.15.114:0/3925565345 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f90f40a1090 0x7f90f40a3550 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f90fc005c40 tx=0x7f90fc00a570 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:24.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:24.280+0000 7f910affd640 1 -- 172.21.15.114:0/3925565345 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f91000960d0 con 0x7f910c103110 2024-09-13T18:20:24.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:24 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/393088203' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:20:24.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:24 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/3494754112' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-13T18:20:24.469 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:24 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/393088203' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:20:24.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:24 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/3494754112' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-13T18:20:24.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:24.477+0000 7f911400d640 1 -- 172.21.15.114:0/3925565345 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7f910c02c410 con 0x7f910c103110 2024-09-13T18:20:24.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:24 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/393088203' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:20:24.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:24 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/3494754112' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-13T18:20:25.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:25.165+0000 7f910affd640 1 -- 172.21.15.114:0/3925565345 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) v1 ==== 105+0+0 (secure 0 0 0) 0x7f910001d0b0 con 0x7f910c103110 2024-09-13T18:20:25.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:25.167+0000 7f911400d640 1 -- 172.21.15.114:0/3925565345 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f90f40a1090 msgr2=0x7f90f40a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:25.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:25.167+0000 7f911400d640 1 --2- 172.21.15.114:0/3925565345 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f90f40a1090 0x7f90f40a3550 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f90fc005c40 tx=0x7f90fc00a570 comp rx=0 tx=0).stop 2024-09-13T18:20:25.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:25.167+0000 7f911400d640 1 -- 172.21.15.114:0/3925565345 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f910c103110 msgr2=0x7f910c0fe790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:25.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:25.167+0000 7f911400d640 1 --2- 172.21.15.114:0/3925565345 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f910c103110 0x7f910c0fe790 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f910000c9a0 tx=0x7f910000ce60 comp rx=0 tx=0).stop 2024-09-13T18:20:25.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:25.167+0000 7f911400d640 1 -- 172.21.15.114:0/3925565345 shutdown_connections 2024-09-13T18:20:25.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:25.167+0000 7f911400d640 1 --2- 172.21.15.114:0/3925565345 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f90f40a1090 0x7f90f40a3550 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:25.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:25.167+0000 7f911400d640 1 --2- 172.21.15.114:0/3925565345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f910c105a40 0x7f910c0fecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:25.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:25.167+0000 7f911400d640 1 --2- 172.21.15.114:0/3925565345 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f910c103110 0x7f910c0fe790 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:25.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:25.167+0000 7f911400d640 1 --2- 172.21.15.114:0/3925565345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f910c1007e0 0x7f910c100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:25.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:25.168+0000 7f911400d640 1 -- 172.21.15.114:0/3925565345 >> 172.21.15.114:0/3925565345 conn(0x7f910c0fa3e0 msgr2=0x7f910c101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:25.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:25.168+0000 7f911400d640 1 -- 172.21.15.114:0/3925565345 shutdown_connections 2024-09-13T18:20:25.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:25.168+0000 7f911400d640 1 -- 172.21.15.114:0/3925565345 wait complete. 2024-09-13T18:20:25.186 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-09-13T18:20:25.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:25.287+0000 7f7b1863a640 1 -- 172.21.15.114:0/2734286240 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b100fec70 msgr2=0x7f7b100ff070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:25.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:25.287+0000 7f7b1863a640 1 --2- 172.21.15.114:0/2734286240 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b100fec70 0x7f7b100ff070 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c009900 tx=0x7f7b0c02a410 comp rx=0 tx=0).stop 2024-09-13T18:20:25.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:25.287+0000 7f7b1863a640 1 -- 172.21.15.114:0/2734286240 shutdown_connections 2024-09-13T18:20:25.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:25.287+0000 7f7b1863a640 1 --2- 172.21.15.114:0/2734286240 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b10100820 0x7f7b101070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:25.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:25.287+0000 7f7b1863a640 1 --2- 172.21.15.114:0/2734286240 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b100ffe60 0x7f7b101002e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:25.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:25.287+0000 7f7b1863a640 1 --2- 172.21.15.114:0/2734286240 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b100fec70 0x7f7b100ff070 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:25.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:25.287+0000 7f7b1863a640 1 -- 172.21.15.114:0/2734286240 >> 172.21.15.114:0/2734286240 conn(0x7f7b100fa3d0 msgr2=0x7f7b100fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:25.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:25.287+0000 7f7b1863a640 1 -- 172.21.15.114:0/2734286240 shutdown_connections 2024-09-13T18:20:25.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:25.287+0000 7f7b1863a640 1 -- 172.21.15.114:0/2734286240 wait complete. 2024-09-13T18:20:25.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:25.288+0000 7f7b1863a640 1 Processor -- start 2024-09-13T18:20:25.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:25.288+0000 7f7b1863a640 1 -- start start 2024-09-13T18:20:25.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:25.288+0000 7f7b1863a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b100fec70 0x7f7b10102c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:25.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:25.288+0000 7f7b1863a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b100ffe60 0x7f7b10101240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:25.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:25.288+0000 7f7b1863a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b10100820 0x7f7b10101780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:25.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:25.288+0000 7f7b1863a640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b10101ee0 con 0x7f7b100fec70 2024-09-13T18:20:25.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:25.288+0000 7f7b1863a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b10102050 con 0x7f7b10100820 2024-09-13T18:20:25.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:25.288+0000 7f7b1863a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1019fe30 con 0x7f7b100ffe60 2024-09-13T18:20:25.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:25.288+0000 7f7b163d6640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b100fec70 0x7f7b10102c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:25.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:25.288+0000 7f7b163d6640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b100fec70 0x7f7b10102c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60658/0 (socket says 172.21.15.114:60658) 2024-09-13T18:20:25.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:25.288+0000 7f7b163d6640 1 -- 172.21.15.114:0/2323212211 learned_addr learned my addr 172.21.15.114:0/2323212211 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:20:25.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:25.288+0000 7f7b16bd7640 1 --2- 172.21.15.114:0/2323212211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b10100820 0x7f7b10101780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:25.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:25.288+0000 7f7b163d6640 1 -- 172.21.15.114:0/2323212211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b100ffe60 msgr2=0x7f7b10101240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:25.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:25.288+0000 7f7b163d6640 1 --2- 172.21.15.114:0/2323212211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b100ffe60 0x7f7b10101240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:25.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:25.288+0000 7f7b163d6640 1 -- 172.21.15.114:0/2323212211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b10100820 msgr2=0x7f7b10101780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:25.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:25.288+0000 7f7b163d6640 1 --2- 172.21.15.114:0/2323212211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b10100820 0x7f7b10101780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:25.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:25.288+0000 7f7b163d6640 1 -- 172.21.15.114:0/2323212211 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b0c009560 con 0x7f7b100fec70 2024-09-13T18:20:25.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:25.288+0000 7f7b16bd7640 1 --2- 172.21.15.114:0/2323212211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b10100820 0x7f7b10101780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:20:25.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:25.289+0000 7f7b163d6640 1 --2- 172.21.15.114:0/2323212211 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b100fec70 0x7f7b10102c00 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c009230 tx=0x7f7b0c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:25.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:25.289+0000 7f7b077fe640 1 -- 172.21.15.114:0/2323212211 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c031d70 con 0x7f7b100fec70 2024-09-13T18:20:25.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:25.289+0000 7f7b077fe640 1 -- 172.21.15.114:0/2323212211 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7b0c0044e0 con 0x7f7b100fec70 2024-09-13T18:20:25.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:25.289+0000 7f7b1863a640 1 -- 172.21.15.114:0/2323212211 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b101a0080 con 0x7f7b100fec70 2024-09-13T18:20:25.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:25.289+0000 7f7b077fe640 1 -- 172.21.15.114:0/2323212211 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c003b60 con 0x7f7b100fec70 2024-09-13T18:20:25.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:25.289+0000 7f7b1863a640 1 -- 172.21.15.114:0/2323212211 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b101a0530 con 0x7f7b100fec70 2024-09-13T18:20:25.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:25.290+0000 7f7b1863a640 1 -- 172.21.15.114:0/2323212211 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b1010a440 con 0x7f7b100fec70 2024-09-13T18:20:25.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:25.291+0000 7f7b077fe640 1 -- 172.21.15.114:0/2323212211 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f7b0c003cc0 con 0x7f7b100fec70 2024-09-13T18:20:25.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:25.291+0000 7f7b077fe640 1 --2- 172.21.15.114:0/2323212211 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f7afc0a1090 0x7f7afc0a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:25.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:25.291+0000 7f7b15bd5640 1 --2- 172.21.15.114:0/2323212211 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f7afc0a1090 0x7f7afc0a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:25.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:25.292+0000 7f7b077fe640 1 -- 172.21.15.114:0/2323212211 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f7b0c0ee000 con 0x7f7b100fec70 2024-09-13T18:20:25.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:25.292+0000 7f7b15bd5640 1 --2- 172.21.15.114:0/2323212211 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f7afc0a1090 0x7f7afc0a3550 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7b10102970 tx=0x7f7b00006c90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:25.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:25.294+0000 7f7b077fe640 1 -- 172.21.15.114:0/2323212211 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f7b0c0b75a0 con 0x7f7b100fec70 2024-09-13T18:20:25.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:25 smithi204 ceph-mon[34777]: pgmap v288: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-13T18:20:25.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:25 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/3494754112' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-13T18:20:25.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:25 smithi204 ceph-mon[34777]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-13T18:20:25.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:25 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/3925565345' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-13T18:20:25.469 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:25 smithi114 ceph-mon[30897]: pgmap v288: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-13T18:20:25.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:25 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/3494754112' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-13T18:20:25.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:25 smithi114 ceph-mon[30897]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-13T18:20:25.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:25 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/3925565345' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-13T18:20:25.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:25.491+0000 7f7b1863a640 1 -- 172.21.15.114:0/2323212211 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f7b1010a5d0 con 0x7f7b100fec70 2024-09-13T18:20:25.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:25 smithi134 ceph-mon[34283]: pgmap v288: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-13T18:20:25.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:25 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/3494754112' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-13T18:20:25.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:25 smithi134 ceph-mon[34283]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-13T18:20:25.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:25 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/3925565345' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-13T18:20:26.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:26.177+0000 7f7b077fe640 1 -- 172.21.15.114:0/2323212211 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) v1 ==== 121+0+0 (secure 0 0 0) 0x7f7b0c0babc0 con 0x7f7b100fec70 2024-09-13T18:20:26.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:26.180+0000 7f7b1863a640 1 -- 172.21.15.114:0/2323212211 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f7afc0a1090 msgr2=0x7f7afc0a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:26.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:26.180+0000 7f7b1863a640 1 --2- 172.21.15.114:0/2323212211 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f7afc0a1090 0x7f7afc0a3550 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7b10102970 tx=0x7f7b00006c90 comp rx=0 tx=0).stop 2024-09-13T18:20:26.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:26.180+0000 7f7b1863a640 1 -- 172.21.15.114:0/2323212211 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b100fec70 msgr2=0x7f7b10102c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:26.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:26.180+0000 7f7b1863a640 1 --2- 172.21.15.114:0/2323212211 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b100fec70 0x7f7b10102c00 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c009230 tx=0x7f7b0c0042f0 comp rx=0 tx=0).stop 2024-09-13T18:20:26.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:26.180+0000 7f7b1863a640 1 -- 172.21.15.114:0/2323212211 shutdown_connections 2024-09-13T18:20:26.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:26.180+0000 7f7b1863a640 1 --2- 172.21.15.114:0/2323212211 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f7afc0a1090 0x7f7afc0a3550 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:26.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:26.180+0000 7f7b1863a640 1 --2- 172.21.15.114:0/2323212211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b10100820 0x7f7b10101780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:26.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:26.180+0000 7f7b1863a640 1 --2- 172.21.15.114:0/2323212211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b100ffe60 0x7f7b10101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:26.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:26.180+0000 7f7b1863a640 1 --2- 172.21.15.114:0/2323212211 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b100fec70 0x7f7b10102c00 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:26.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:26.180+0000 7f7b1863a640 1 -- 172.21.15.114:0/2323212211 >> 172.21.15.114:0/2323212211 conn(0x7f7b100fa3d0 msgr2=0x7f7b10068600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:26.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:26.180+0000 7f7b1863a640 1 -- 172.21.15.114:0/2323212211 shutdown_connections 2024-09-13T18:20:26.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:26.180+0000 7f7b1863a640 1 -- 172.21.15.114:0/2323212211 wait complete. 2024-09-13T18:20:26.199 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-09-13T18:20:26.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:26.297+0000 7f3d9cd0f640 1 -- 172.21.15.114:0/3587912779 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d981007f0 msgr2=0x7f3d98102be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:26.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:26.297+0000 7f3d9cd0f640 1 --2- 172.21.15.114:0/3587912779 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d981007f0 0x7f3d98102be0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3d88009900 tx=0x7f3d8802a410 comp rx=0 tx=0).stop 2024-09-13T18:20:26.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:26.297+0000 7f3d9cd0f640 1 -- 172.21.15.114:0/3587912779 shutdown_connections 2024-09-13T18:20:26.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:26.297+0000 7f3d9cd0f640 1 --2- 172.21.15.114:0/3587912779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d98105a50 0x7f3d98107ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:26.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:26.297+0000 7f3d9cd0f640 1 --2- 172.21.15.114:0/3587912779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d98103120 0x7f3d98105510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:26.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:26.297+0000 7f3d9cd0f640 1 --2- 172.21.15.114:0/3587912779 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d981007f0 0x7f3d98102be0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:26.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:26.298+0000 7f3d9cd0f640 1 -- 172.21.15.114:0/3587912779 >> 172.21.15.114:0/3587912779 conn(0x7f3d980fa3d0 msgr2=0x7f3d980fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:26.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:26.298+0000 7f3d9cd0f640 1 -- 172.21.15.114:0/3587912779 shutdown_connections 2024-09-13T18:20:26.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:26.298+0000 7f3d9cd0f640 1 -- 172.21.15.114:0/3587912779 wait complete. 2024-09-13T18:20:26.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:26.298+0000 7f3d9cd0f640 1 Processor -- start 2024-09-13T18:20:26.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:26.298+0000 7f3d9cd0f640 1 -- start start 2024-09-13T18:20:26.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:26.298+0000 7f3d9cd0f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d981007f0 0x7f3d98195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:26.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:26.298+0000 7f3d9cd0f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d98103120 0x7f3d981963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:26.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:26.298+0000 7f3d9cd0f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d98105a50 0x7f3d9819d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:26.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:26.298+0000 7f3d9cd0f640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9819d950 con 0x7f3d981007f0 2024-09-13T18:20:26.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:26.298+0000 7f3d9cd0f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9819dac0 con 0x7f3d98103120 2024-09-13T18:20:26.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:26.298+0000 7f3d9cd0f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9819dc80 con 0x7f3d98105a50 2024-09-13T18:20:26.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:26.299+0000 7f3d9659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d981007f0 0x7f3d98195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:26.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:26.299+0000 7f3d9659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d981007f0 0x7f3d98195e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60678/0 (socket says 172.21.15.114:60678) 2024-09-13T18:20:26.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:26.299+0000 7f3d9659c640 1 -- 172.21.15.114:0/1900324420 learned_addr learned my addr 172.21.15.114:0/1900324420 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:20:26.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:26.299+0000 7f3d9659c640 1 -- 172.21.15.114:0/1900324420 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d98105a50 msgr2=0x7f3d9819d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:26.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:26.299+0000 7f3d9659c640 1 --2- 172.21.15.114:0/1900324420 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d98105a50 0x7f3d9819d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:26.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:26.299+0000 7f3d9659c640 1 -- 172.21.15.114:0/1900324420 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d98103120 msgr2=0x7f3d981963a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:26.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:26.299+0000 7f3d9659c640 1 --2- 172.21.15.114:0/1900324420 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d98103120 0x7f3d981963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:26.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:26.299+0000 7f3d9659c640 1 -- 172.21.15.114:0/1900324420 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d88009560 con 0x7f3d981007f0 2024-09-13T18:20:26.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:26.299+0000 7f3d9659c640 1 --2- 172.21.15.114:0/1900324420 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d981007f0 0x7f3d98195e60 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3d88009230 tx=0x7f3d88004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:26.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:26.299+0000 7f3d877fe640 1 -- 172.21.15.114:0/1900324420 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d88031d70 con 0x7f3d981007f0 2024-09-13T18:20:26.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:26.299+0000 7f3d877fe640 1 -- 172.21.15.114:0/1900324420 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3d88004b50 con 0x7f3d981007f0 2024-09-13T18:20:26.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:26.299+0000 7f3d9cd0f640 1 -- 172.21.15.114:0/1900324420 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d9819df00 con 0x7f3d981007f0 2024-09-13T18:20:26.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:26.299+0000 7f3d877fe640 1 -- 172.21.15.114:0/1900324420 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d88003bd0 con 0x7f3d981007f0 2024-09-13T18:20:26.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:26.299+0000 7f3d9cd0f640 1 -- 172.21.15.114:0/1900324420 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d9819e320 con 0x7f3d981007f0 2024-09-13T18:20:26.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:26.300+0000 7f3d9cd0f640 1 -- 172.21.15.114:0/1900324420 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d980fbec0 con 0x7f3d981007f0 2024-09-13T18:20:26.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:26.301+0000 7f3d877fe640 1 -- 172.21.15.114:0/1900324420 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f3d88003d30 con 0x7f3d981007f0 2024-09-13T18:20:26.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:26.302+0000 7f3d877fe640 1 --2- 172.21.15.114:0/1900324420 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f3d800a1090 0x7f3d800a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:26.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:26.302+0000 7f3d877fe640 1 -- 172.21.15.114:0/1900324420 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f3d880ee240 con 0x7f3d981007f0 2024-09-13T18:20:26.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:26.302+0000 7f3d95d9b640 1 --2- 172.21.15.114:0/1900324420 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f3d800a1090 0x7f3d800a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:26.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:26.303+0000 7f3d95d9b640 1 --2- 172.21.15.114:0/1900324420 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f3d800a1090 0x7f3d800a3550 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3d98197370 tx=0x7f3d8c006c90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:26.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:26.305+0000 7f3d877fe640 1 -- 172.21.15.114:0/1900324420 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f3d880b76c0 con 0x7f3d981007f0 2024-09-13T18:20:26.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:26 smithi204 ceph-mon[34777]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-13T18:20:26.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:26 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/3925565345' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-13T18:20:26.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:26 smithi204 ceph-mon[34777]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-13T18:20:26.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:26 smithi204 ceph-mon[34777]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-13T18:20:26.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:26 smithi204 ceph-mon[34777]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-13T18:20:26.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:26 smithi204 ceph-mon[34777]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-13T18:20:26.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:26 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/2323212211' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-13T18:20:26.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:26 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-13T18:20:26.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:26 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-09-13T18:20:26.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:26 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]: dispatch 2024-09-13T18:20:26.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:26 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-09-13T18:20:26.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:26 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]: dispatch 2024-09-13T18:20:26.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:26 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-09-13T18:20:26.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:26 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]: dispatch 2024-09-13T18:20:26.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:26 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-09-13T18:20:26.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:26 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-09-13T18:20:26.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:26 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2024-09-13T18:20:26.469 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:26 smithi114 ceph-mon[30897]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-13T18:20:26.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:26 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/3925565345' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-13T18:20:26.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:26 smithi114 ceph-mon[30897]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-13T18:20:26.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:26 smithi114 ceph-mon[30897]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-13T18:20:26.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:26 smithi114 ceph-mon[30897]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-13T18:20:26.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:26 smithi114 ceph-mon[30897]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-13T18:20:26.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:26 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2323212211' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-13T18:20:26.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:26 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-13T18:20:26.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:26 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-09-13T18:20:26.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:26 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]: dispatch 2024-09-13T18:20:26.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:26 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-09-13T18:20:26.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:26 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]: dispatch 2024-09-13T18:20:26.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:26 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-09-13T18:20:26.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:26 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]: dispatch 2024-09-13T18:20:26.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:26 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-09-13T18:20:26.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:26 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-09-13T18:20:26.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:26 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2024-09-13T18:20:26.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:26.503+0000 7f3d9cd0f640 1 -- 172.21.15.114:0/1900324420 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f3d9810b850 con 0x7f3d981007f0 2024-09-13T18:20:26.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:26 smithi134 ceph-mon[34283]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-13T18:20:26.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:26 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/3925565345' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-13T18:20:26.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:26 smithi134 ceph-mon[34283]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-13T18:20:26.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:26 smithi134 ceph-mon[34283]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-13T18:20:26.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:26 smithi134 ceph-mon[34283]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-13T18:20:26.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:26 smithi134 ceph-mon[34283]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-13T18:20:26.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:26 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/2323212211' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-13T18:20:26.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:26 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-13T18:20:26.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:26 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-09-13T18:20:26.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:26 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]: dispatch 2024-09-13T18:20:26.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:26 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-09-13T18:20:26.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:26 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]: dispatch 2024-09-13T18:20:26.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:26 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-09-13T18:20:26.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:26 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]: dispatch 2024-09-13T18:20:26.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:26 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-09-13T18:20:26.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:26 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-09-13T18:20:26.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:26 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2024-09-13T18:20:27.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:27.188+0000 7f3d877fe640 1 -- 172.21.15.114:0/1900324420 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) v1 ==== 115+0+0 (secure 0 0 0) 0x7f3d880bace0 con 0x7f3d981007f0 2024-09-13T18:20:27.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:27.191+0000 7f3d9cd0f640 1 -- 172.21.15.114:0/1900324420 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f3d800a1090 msgr2=0x7f3d800a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:27.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:27.191+0000 7f3d9cd0f640 1 --2- 172.21.15.114:0/1900324420 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f3d800a1090 0x7f3d800a3550 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3d98197370 tx=0x7f3d8c006c90 comp rx=0 tx=0).stop 2024-09-13T18:20:27.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:27.191+0000 7f3d9cd0f640 1 -- 172.21.15.114:0/1900324420 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d981007f0 msgr2=0x7f3d98195e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:27.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:27.191+0000 7f3d9cd0f640 1 --2- 172.21.15.114:0/1900324420 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d981007f0 0x7f3d98195e60 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3d88009230 tx=0x7f3d88004360 comp rx=0 tx=0).stop 2024-09-13T18:20:27.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:27.191+0000 7f3d9cd0f640 1 -- 172.21.15.114:0/1900324420 shutdown_connections 2024-09-13T18:20:27.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:27.191+0000 7f3d9cd0f640 1 --2- 172.21.15.114:0/1900324420 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f3d800a1090 0x7f3d800a3550 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:27.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:27.191+0000 7f3d9cd0f640 1 --2- 172.21.15.114:0/1900324420 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d98105a50 0x7f3d9819d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:27.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:27.191+0000 7f3d9cd0f640 1 --2- 172.21.15.114:0/1900324420 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d98103120 0x7f3d981963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:27.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:27.191+0000 7f3d9cd0f640 1 --2- 172.21.15.114:0/1900324420 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d981007f0 0x7f3d98195e60 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:27.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:27.191+0000 7f3d9cd0f640 1 -- 172.21.15.114:0/1900324420 >> 172.21.15.114:0/1900324420 conn(0x7f3d980fa3d0 msgr2=0x7f3d981010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:27.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:27.191+0000 7f3d9cd0f640 1 -- 172.21.15.114:0/1900324420 shutdown_connections 2024-09-13T18:20:27.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:27.191+0000 7f3d9cd0f640 1 -- 172.21.15.114:0/1900324420 wait complete. 2024-09-13T18:20:27.206 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-09-13T18:20:27.207 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-09-13T18:20:27.258 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 13 18:20:26 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-b[63915]: 2024-09-13T18:20:26.960+0000 7f4e94321640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-13T18:20:27.258 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:27 smithi134 ceph-mon[34283]: pgmap v289: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-13T18:20:27.258 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:27 smithi134 ceph-mon[34283]: mds.? [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] up:active 2024-09-13T18:20:27.258 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:27 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/2323212211' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-13T18:20:27.258 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:27 smithi134 ceph-mon[34283]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-13T18:20:27.258 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:27 smithi134 ceph-mon[34283]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-13T18:20:27.258 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:27 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-13T18:20:27.258 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:27 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-13T18:20:27.259 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:27 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]': finished 2024-09-13T18:20:27.259 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:27 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-09-13T18:20:27.259 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:27 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]': finished 2024-09-13T18:20:27.259 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:27 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-09-13T18:20:27.259 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:27 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]': finished 2024-09-13T18:20:27.259 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:27 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-13T18:20:27.259 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:27 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-09-13T18:20:27.259 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:27 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-09-13T18:20:27.259 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:27 smithi134 ceph-mon[34283]: osdmap e90: 12 total, 12 up, 12 in 2024-09-13T18:20:27.259 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:27 smithi134 ceph-mon[34283]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-13T18:20:27.259 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:27 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/1900324420' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-13T18:20:27.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:27.308+0000 7f9727326640 1 -- 172.21.15.114:0/1314872943 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f97200fec60 msgr2=0x7f97200ff060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:27.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:27.308+0000 7f9727326640 1 --2- 172.21.15.114:0/1314872943 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f97200fec60 0x7f97200ff060 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f971c009ca0 tx=0x7f971c02a8c0 comp rx=0 tx=0).stop 2024-09-13T18:20:27.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:27.309+0000 7f9727326640 1 -- 172.21.15.114:0/1314872943 shutdown_connections 2024-09-13T18:20:27.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:27.309+0000 7f9727326640 1 --2- 172.21.15.114:0/1314872943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9720100810 0x7f97201070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:27.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:27.309+0000 7f9727326640 1 --2- 172.21.15.114:0/1314872943 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97200ffe50 0x7f97201002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:27.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:27.309+0000 7f9727326640 1 --2- 172.21.15.114:0/1314872943 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f97200fec60 0x7f97200ff060 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:27.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:27.309+0000 7f9727326640 1 -- 172.21.15.114:0/1314872943 >> 172.21.15.114:0/1314872943 conn(0x7f97200fa3e0 msgr2=0x7f97200fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:27.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:27.309+0000 7f9727326640 1 -- 172.21.15.114:0/1314872943 shutdown_connections 2024-09-13T18:20:27.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:27.309+0000 7f9727326640 1 -- 172.21.15.114:0/1314872943 wait complete. 2024-09-13T18:20:27.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:27.309+0000 7f9727326640 1 Processor -- start 2024-09-13T18:20:27.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:27.309+0000 7f9727326640 1 -- start start 2024-09-13T18:20:27.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:27.309+0000 7f9727326640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97200fec60 0x7f97200680c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:27.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:27.309+0000 7f9727326640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f97200ffe50 0x7f972019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:27.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:27.309+0000 7f9727326640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9720100810 0x7f972019cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:27.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:27.309+0000 7f9727326640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972019d780 con 0x7f97200ffe50 2024-09-13T18:20:27.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:27.309+0000 7f9727326640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972019d8f0 con 0x7f97200fec60 2024-09-13T18:20:27.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:27.309+0000 7f9727326640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972019dab0 con 0x7f9720100810 2024-09-13T18:20:27.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:27.310+0000 7f97248c1640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f97200ffe50 0x7f972019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:27.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:27.310+0000 7f97248c1640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f97200ffe50 0x7f972019aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60696/0 (socket says 172.21.15.114:60696) 2024-09-13T18:20:27.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:27.310+0000 7f97248c1640 1 -- 172.21.15.114:0/2544753825 learned_addr learned my addr 172.21.15.114:0/2544753825 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:20:27.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:27.310+0000 7f97258c3640 1 --2- 172.21.15.114:0/2544753825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9720100810 0x7f972019cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:27.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:27.310+0000 7f97250c2640 1 --2- 172.21.15.114:0/2544753825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97200fec60 0x7f97200680c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:27.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:27.310+0000 7f97248c1640 1 -- 172.21.15.114:0/2544753825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9720100810 msgr2=0x7f972019cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:27.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:27.310+0000 7f97248c1640 1 --2- 172.21.15.114:0/2544753825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9720100810 0x7f972019cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:27.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:27.310+0000 7f97248c1640 1 -- 172.21.15.114:0/2544753825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97200fec60 msgr2=0x7f97200680c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:27.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:27.310+0000 7f97248c1640 1 --2- 172.21.15.114:0/2544753825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97200fec60 0x7f97200680c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:27.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:27.310+0000 7f97248c1640 1 -- 172.21.15.114:0/2544753825 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f971c009900 con 0x7f97200ffe50 2024-09-13T18:20:27.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:27.310+0000 7f97248c1640 1 --2- 172.21.15.114:0/2544753825 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f97200ffe50 0x7f972019aa20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f971000c890 tx=0x7f971000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:27.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:27.310+0000 7f97167fc640 1 -- 172.21.15.114:0/2544753825 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9710004220 con 0x7f97200ffe50 2024-09-13T18:20:27.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:27.310+0000 7f9727326640 1 -- 172.21.15.114:0/2544753825 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f972019dd90 con 0x7f97200ffe50 2024-09-13T18:20:27.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:27.310+0000 7f97167fc640 1 -- 172.21.15.114:0/2544753825 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9710007e40 con 0x7f97200ffe50 2024-09-13T18:20:27.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:27.310+0000 7f97167fc640 1 -- 172.21.15.114:0/2544753825 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9710004a00 con 0x7f97200ffe50 2024-09-13T18:20:27.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:27.310+0000 7f9727326640 1 -- 172.21.15.114:0/2544753825 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f972019e240 con 0x7f97200ffe50 2024-09-13T18:20:27.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:27.311+0000 7f9727326640 1 -- 172.21.15.114:0/2544753825 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9720106230 con 0x7f97200ffe50 2024-09-13T18:20:27.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:27.312+0000 7f97167fc640 1 -- 172.21.15.114:0/2544753825 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f97100079b0 con 0x7f97200ffe50 2024-09-13T18:20:27.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:27.313+0000 7f97167fc640 1 --2- 172.21.15.114:0/2544753825 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f970c0a1090 0x7f970c0a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:27.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:27.313+0000 7f97250c2640 1 --2- 172.21.15.114:0/2544753825 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f970c0a1090 0x7f970c0a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:27.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:27.313+0000 7f97167fc640 1 -- 172.21.15.114:0/2544753825 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f97100ccb70 con 0x7f97200ffe50 2024-09-13T18:20:27.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:27.313+0000 7f97250c2640 1 --2- 172.21.15.114:0/2544753825 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f970c0a1090 0x7f970c0a3550 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f971c005e30 tx=0x7f971c009800 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:27.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:27.317+0000 7f97167fc640 1 -- 172.21.15.114:0/2544753825 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f97100960a0 con 0x7f97200ffe50 2024-09-13T18:20:27.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:27 smithi204 ceph-mon[34777]: pgmap v289: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-13T18:20:27.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:27 smithi204 ceph-mon[34777]: mds.? [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] up:active 2024-09-13T18:20:27.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:27 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/2323212211' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-13T18:20:27.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:27 smithi204 ceph-mon[34777]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-13T18:20:27.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:27 smithi204 ceph-mon[34777]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-13T18:20:27.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:27 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-13T18:20:27.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:27 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-13T18:20:27.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:27 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]': finished 2024-09-13T18:20:27.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:27 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-09-13T18:20:27.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:27 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]': finished 2024-09-13T18:20:27.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:27 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-09-13T18:20:27.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:27 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]': finished 2024-09-13T18:20:27.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:27 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-13T18:20:27.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:27 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-09-13T18:20:27.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:27 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-09-13T18:20:27.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:27 smithi204 ceph-mon[34777]: osdmap e90: 12 total, 12 up, 12 in 2024-09-13T18:20:27.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:27 smithi204 ceph-mon[34777]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-13T18:20:27.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:27 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/1900324420' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-13T18:20:27.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:27 smithi114 ceph-mon[30897]: pgmap v289: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-13T18:20:27.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:27 smithi114 ceph-mon[30897]: mds.? [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] up:active 2024-09-13T18:20:27.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:27 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2323212211' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-13T18:20:27.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:27 smithi114 ceph-mon[30897]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-13T18:20:27.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:27 smithi114 ceph-mon[30897]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-13T18:20:27.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:27 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-13T18:20:27.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:27 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-13T18:20:27.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:27 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]': finished 2024-09-13T18:20:27.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:27 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-09-13T18:20:27.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:27 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]': finished 2024-09-13T18:20:27.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:27 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-09-13T18:20:27.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:27 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]': finished 2024-09-13T18:20:27.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:27 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-13T18:20:27.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:27 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-09-13T18:20:27.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:27 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-09-13T18:20:27.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:27 smithi114 ceph-mon[30897]: osdmap e90: 12 total, 12 up, 12 in 2024-09-13T18:20:27.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:27 smithi114 ceph-mon[30897]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-13T18:20:27.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:27 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/1900324420' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-13T18:20:27.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:27.504+0000 7f9727326640 1 -- 172.21.15.114:0/2544753825 --> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f972002c4f0 con 0x7f970c0a1090 2024-09-13T18:20:27.970 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:20:27 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:20:27.509+0000 7fb0b5bbf640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T18:20:27.970 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:20:27 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:20:27.509+0000 7fb0b5bbf640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T18:20:27.970 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:20:27 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:20:27.509+0000 7fb0b5bbf640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T18:20:27.970 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:20:27 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:20:27.509+0000 7fb0b5bbf640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T18:20:27.970 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:20:27 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:20:27.509+0000 7fb0b5bbf640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T18:20:28.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.209+0000 7f97167fc640 1 -- 172.21.15.114:0/2544753825 <== mgr.14148 v2:172.21.15.114:6800/3396185541 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f972002c4f0 con 0x7f970c0a1090 2024-09-13T18:20:28.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.211+0000 7f9727326640 1 -- 172.21.15.114:0/2544753825 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f970c0a1090 msgr2=0x7f970c0a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:28.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.211+0000 7f9727326640 1 --2- 172.21.15.114:0/2544753825 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f970c0a1090 0x7f970c0a3550 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f971c005e30 tx=0x7f971c009800 comp rx=0 tx=0).stop 2024-09-13T18:20:28.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.211+0000 7f9727326640 1 -- 172.21.15.114:0/2544753825 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f97200ffe50 msgr2=0x7f972019aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:28.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.211+0000 7f9727326640 1 --2- 172.21.15.114:0/2544753825 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f97200ffe50 0x7f972019aa20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f971000c890 tx=0x7f971000cd50 comp rx=0 tx=0).stop 2024-09-13T18:20:28.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.211+0000 7f9727326640 1 -- 172.21.15.114:0/2544753825 shutdown_connections 2024-09-13T18:20:28.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.211+0000 7f9727326640 1 --2- 172.21.15.114:0/2544753825 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f970c0a1090 0x7f970c0a3550 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:28.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.211+0000 7f9727326640 1 --2- 172.21.15.114:0/2544753825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9720100810 0x7f972019cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:28.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.211+0000 7f9727326640 1 --2- 172.21.15.114:0/2544753825 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f97200ffe50 0x7f972019aa20 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:28.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.211+0000 7f9727326640 1 --2- 172.21.15.114:0/2544753825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97200fec60 0x7f97200680c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:28.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.211+0000 7f9727326640 1 -- 172.21.15.114:0/2544753825 >> 172.21.15.114:0/2544753825 conn(0x7f97200fa3e0 msgr2=0x7f9720103400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:28.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.211+0000 7f9727326640 1 -- 172.21.15.114:0/2544753825 shutdown_connections 2024-09-13T18:20:28.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.211+0000 7f9727326640 1 -- 172.21.15.114:0/2544753825 wait complete. 2024-09-13T18:20:28.227 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-13T18:20:28.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.334+0000 7ff57e108640 1 -- 172.21.15.114:0/2911318358 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff578105a40 msgr2=0x7ff578107eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:28.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.334+0000 7ff57e108640 1 --2- 172.21.15.114:0/2911318358 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff578105a40 0x7ff578107eb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff570009ca0 tx=0x7ff57002a8c0 comp rx=0 tx=0).stop 2024-09-13T18:20:28.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.335+0000 7ff57e108640 1 -- 172.21.15.114:0/2911318358 shutdown_connections 2024-09-13T18:20:28.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.335+0000 7ff57e108640 1 --2- 172.21.15.114:0/2911318358 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff578105a40 0x7ff578107eb0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:28.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.335+0000 7ff57e108640 1 --2- 172.21.15.114:0/2911318358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff578103110 0x7ff578105500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:28.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.335+0000 7ff57e108640 1 --2- 172.21.15.114:0/2911318358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5781007e0 0x7ff578102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:28.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.335+0000 7ff57e108640 1 -- 172.21.15.114:0/2911318358 >> 172.21.15.114:0/2911318358 conn(0x7ff5780fa3e0 msgr2=0x7ff5780fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:28.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.335+0000 7ff57e108640 1 -- 172.21.15.114:0/2911318358 shutdown_connections 2024-09-13T18:20:28.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.335+0000 7ff57e108640 1 -- 172.21.15.114:0/2911318358 wait complete. 2024-09-13T18:20:28.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.335+0000 7ff57e108640 1 Processor -- start 2024-09-13T18:20:28.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.335+0000 7ff57e108640 1 -- start start 2024-09-13T18:20:28.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.335+0000 7ff57e108640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5781007e0 0x7ff578195ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:28.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.335+0000 7ff57e108640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff578103110 0x7ff5781961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:28.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.335+0000 7ff57e108640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff578105a40 0x7ff57819d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:28.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.335+0000 7ff57e108640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff57819d7b0 con 0x7ff5781007e0 2024-09-13T18:20:28.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.335+0000 7ff57e108640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff57819d920 con 0x7ff578105a40 2024-09-13T18:20:28.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.335+0000 7ff57e108640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff57819dae0 con 0x7ff578103110 2024-09-13T18:20:28.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.336+0000 7ff5777fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5781007e0 0x7ff578195ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:28.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.336+0000 7ff5777fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5781007e0 0x7ff578195ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60720/0 (socket says 172.21.15.114:60720) 2024-09-13T18:20:28.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.336+0000 7ff5777fe640 1 -- 172.21.15.114:0/2921117521 learned_addr learned my addr 172.21.15.114:0/2921117521 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:20:28.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.336+0000 7ff576ffd640 1 --2- 172.21.15.114:0/2921117521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff578103110 0x7ff5781961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:28.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.336+0000 7ff5777fe640 1 -- 172.21.15.114:0/2921117521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff578103110 msgr2=0x7ff5781961e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:28.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.336+0000 7ff5777fe640 1 --2- 172.21.15.114:0/2921117521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff578103110 0x7ff5781961e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:28.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.336+0000 7ff5777fe640 1 -- 172.21.15.114:0/2921117521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff578105a40 msgr2=0x7ff57819d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:28.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.336+0000 7ff5777fe640 1 --2- 172.21.15.114:0/2921117521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff578105a40 0x7ff57819d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:28.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.336+0000 7ff5777fe640 1 -- 172.21.15.114:0/2921117521 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff570009900 con 0x7ff5781007e0 2024-09-13T18:20:28.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.336+0000 7ff5777fe640 1 --2- 172.21.15.114:0/2921117521 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5781007e0 0x7ff578195ca0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff56800d950 tx=0x7ff56800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:28.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.336+0000 7ff574ff9640 1 -- 172.21.15.114:0/2921117521 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff568009c30 con 0x7ff5781007e0 2024-09-13T18:20:28.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.336+0000 7ff57e108640 1 -- 172.21.15.114:0/2921117521 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff57819ddc0 con 0x7ff5781007e0 2024-09-13T18:20:28.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.336+0000 7ff574ff9640 1 -- 172.21.15.114:0/2921117521 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff568009d90 con 0x7ff5781007e0 2024-09-13T18:20:28.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.336+0000 7ff574ff9640 1 -- 172.21.15.114:0/2921117521 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56800f620 con 0x7ff5781007e0 2024-09-13T18:20:28.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.336+0000 7ff57e108640 1 -- 172.21.15.114:0/2921117521 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff57819e210 con 0x7ff5781007e0 2024-09-13T18:20:28.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.337+0000 7ff57e108640 1 -- 172.21.15.114:0/2921117521 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5780fbed0 con 0x7ff5781007e0 2024-09-13T18:20:28.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.338+0000 7ff574ff9640 1 -- 172.21.15.114:0/2921117521 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7ff5680109f0 con 0x7ff5781007e0 2024-09-13T18:20:28.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.339+0000 7ff574ff9640 1 --2- 172.21.15.114:0/2921117521 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7ff5600a1090 0x7ff5600a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:28.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.339+0000 7ff576ffd640 1 --2- 172.21.15.114:0/2921117521 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7ff5600a1090 0x7ff5600a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:28.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.339+0000 7ff574ff9640 1 -- 172.21.15.114:0/2921117521 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff5680cc8d0 con 0x7ff5781007e0 2024-09-13T18:20:28.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.340+0000 7ff576ffd640 1 --2- 172.21.15.114:0/2921117521 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7ff5600a1090 0x7ff5600a3550 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff56c005fd0 tx=0x7ff56c005e20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:28.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.342+0000 7ff574ff9640 1 -- 172.21.15.114:0/2921117521 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7ff568095cf0 con 0x7ff5781007e0 2024-09-13T18:20:28.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:28 smithi204 ceph-mon[34777]: osdmap e91: 12 total, 12 up, 12 in 2024-09-13T18:20:28.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:28 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/1900324420' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-13T18:20:28.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:28 smithi204 ceph-mon[34777]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-13T18:20:28.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:28 smithi204 ceph-mon[34777]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-13T18:20:28.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:28 smithi204 ceph-mon[34777]: from='client.14883 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:20:28.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:28 smithi204 ceph-mon[34777]: pgmap v292: 129 pgs: 1 activating, 128 active+clean; 583 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 3 op/s 2024-09-13T18:20:28.469 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:28 smithi114 ceph-mon[30897]: osdmap e91: 12 total, 12 up, 12 in 2024-09-13T18:20:28.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:28 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/1900324420' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-13T18:20:28.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:28 smithi114 ceph-mon[30897]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-13T18:20:28.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:28 smithi114 ceph-mon[30897]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-13T18:20:28.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:28 smithi114 ceph-mon[30897]: from='client.14883 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:20:28.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:28 smithi114 ceph-mon[30897]: pgmap v292: 129 pgs: 1 activating, 128 active+clean; 583 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 3 op/s 2024-09-13T18:20:28.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.532+0000 7ff57e108640 1 -- 172.21.15.114:0/2921117521 --> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7ff57802c4f0 con 0x7ff5600a1090 2024-09-13T18:20:28.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.555+0000 7ff574ff9640 1 -- 172.21.15.114:0/2921117521 <== mgr.14148 v2:172.21.15.114:6800/3396185541 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ff57802c4f0 con 0x7ff5600a1090 2024-09-13T18:20:28.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.557+0000 7ff57e108640 1 -- 172.21.15.114:0/2921117521 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7ff5600a1090 msgr2=0x7ff5600a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:28.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.557+0000 7ff57e108640 1 --2- 172.21.15.114:0/2921117521 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7ff5600a1090 0x7ff5600a3550 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff56c005fd0 tx=0x7ff56c005e20 comp rx=0 tx=0).stop 2024-09-13T18:20:28.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.557+0000 7ff57e108640 1 -- 172.21.15.114:0/2921117521 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5781007e0 msgr2=0x7ff578195ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:28.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.557+0000 7ff57e108640 1 --2- 172.21.15.114:0/2921117521 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5781007e0 0x7ff578195ca0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff56800d950 tx=0x7ff56800de10 comp rx=0 tx=0).stop 2024-09-13T18:20:28.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.557+0000 7ff57e108640 1 -- 172.21.15.114:0/2921117521 shutdown_connections 2024-09-13T18:20:28.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.557+0000 7ff57e108640 1 --2- 172.21.15.114:0/2921117521 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7ff5600a1090 0x7ff5600a3550 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:28.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.557+0000 7ff57e108640 1 --2- 172.21.15.114:0/2921117521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff578105a40 0x7ff57819d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:28.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.557+0000 7ff57e108640 1 --2- 172.21.15.114:0/2921117521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff578103110 0x7ff5781961e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:28.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.557+0000 7ff57e108640 1 --2- 172.21.15.114:0/2921117521 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5781007e0 0x7ff578195ca0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:28.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.557+0000 7ff57e108640 1 -- 172.21.15.114:0/2921117521 >> 172.21.15.114:0/2921117521 conn(0x7ff5780fa3e0 msgr2=0x7ff578101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:28.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.557+0000 7ff57e108640 1 -- 172.21.15.114:0/2921117521 shutdown_connections 2024-09-13T18:20:28.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.557+0000 7ff57e108640 1 -- 172.21.15.114:0/2921117521 wait complete. 2024-09-13T18:20:28.571 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-13T18:20:28.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:28 smithi134 ceph-mon[34283]: osdmap e91: 12 total, 12 up, 12 in 2024-09-13T18:20:28.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:28 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/1900324420' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-13T18:20:28.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:28 smithi134 ceph-mon[34283]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-13T18:20:28.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:28 smithi134 ceph-mon[34283]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-13T18:20:28.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:28 smithi134 ceph-mon[34283]: from='client.14883 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:20:28.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:28 smithi134 ceph-mon[34283]: pgmap v292: 129 pgs: 1 activating, 128 active+clean; 583 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 3 op/s 2024-09-13T18:20:28.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.672+0000 7fd7b849c640 1 -- 172.21.15.114:0/486619642 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd7b0100810 msgr2=0x7fd7b01070a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:28.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.672+0000 7fd7b849c640 1 --2- 172.21.15.114:0/486619642 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd7b0100810 0x7fd7b01070a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8009ca0 tx=0x7fd7a802a8c0 comp rx=0 tx=0).stop 2024-09-13T18:20:28.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.672+0000 7fd7b849c640 1 -- 172.21.15.114:0/486619642 shutdown_connections 2024-09-13T18:20:28.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.672+0000 7fd7b849c640 1 --2- 172.21.15.114:0/486619642 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd7b0100810 0x7fd7b01070a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:28.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.672+0000 7fd7b849c640 1 --2- 172.21.15.114:0/486619642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7b00ffe50 0x7fd7b01002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:28.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.672+0000 7fd7b849c640 1 --2- 172.21.15.114:0/486619642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7b00fec60 0x7fd7b00ff060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:28.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.672+0000 7fd7b849c640 1 -- 172.21.15.114:0/486619642 >> 172.21.15.114:0/486619642 conn(0x7fd7b00fa3e0 msgr2=0x7fd7b00fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:28.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.672+0000 7fd7b849c640 1 -- 172.21.15.114:0/486619642 shutdown_connections 2024-09-13T18:20:28.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.672+0000 7fd7b849c640 1 -- 172.21.15.114:0/486619642 wait complete. 2024-09-13T18:20:28.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.672+0000 7fd7b849c640 1 Processor -- start 2024-09-13T18:20:28.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.672+0000 7fd7b849c640 1 -- start start 2024-09-13T18:20:28.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.672+0000 7fd7b849c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7b00fec60 0x7fd7b0068110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:28.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.672+0000 7fd7b849c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd7b00ffe50 0x7fd7b019a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:28.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.672+0000 7fd7b849c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7b0100810 0x7fd7b019cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:28.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.672+0000 7fd7b849c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b019d760 con 0x7fd7b00ffe50 2024-09-13T18:20:28.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.672+0000 7fd7b849c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b019d8d0 con 0x7fd7b00fec60 2024-09-13T18:20:28.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.672+0000 7fd7b849c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b019da90 con 0x7fd7b0100810 2024-09-13T18:20:28.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.672+0000 7fd7b5a37640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd7b00ffe50 0x7fd7b019a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:28.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.673+0000 7fd7b5a37640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd7b00ffe50 0x7fd7b019a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60734/0 (socket says 172.21.15.114:60734) 2024-09-13T18:20:28.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.673+0000 7fd7b5a37640 1 -- 172.21.15.114:0/3916555993 learned_addr learned my addr 172.21.15.114:0/3916555993 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:20:28.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.673+0000 7fd7b6a39640 1 --2- 172.21.15.114:0/3916555993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7b0100810 0x7fd7b019cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:28.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.673+0000 7fd7b5a37640 1 -- 172.21.15.114:0/3916555993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7b0100810 msgr2=0x7fd7b019cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:28.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.673+0000 7fd7b6238640 1 --2- 172.21.15.114:0/3916555993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7b00fec60 0x7fd7b0068110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:28.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.673+0000 7fd7b5a37640 1 --2- 172.21.15.114:0/3916555993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7b0100810 0x7fd7b019cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:28.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.673+0000 7fd7b5a37640 1 -- 172.21.15.114:0/3916555993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7b00fec60 msgr2=0x7fd7b0068110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:28.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.673+0000 7fd7b5a37640 1 --2- 172.21.15.114:0/3916555993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7b00fec60 0x7fd7b0068110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:28.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.673+0000 7fd7b5a37640 1 -- 172.21.15.114:0/3916555993 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a8009900 con 0x7fd7b00ffe50 2024-09-13T18:20:28.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.673+0000 7fd7b5a37640 1 --2- 172.21.15.114:0/3916555993 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd7b00ffe50 0x7fd7b019a960 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd7a000c890 tx=0x7fd7a000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:28.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.673+0000 7fd7a77fe640 1 -- 172.21.15.114:0/3916555993 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a0004220 con 0x7fd7b00ffe50 2024-09-13T18:20:28.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.673+0000 7fd7a77fe640 1 -- 172.21.15.114:0/3916555993 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd7a0007e40 con 0x7fd7b00ffe50 2024-09-13T18:20:28.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.673+0000 7fd7b849c640 1 -- 172.21.15.114:0/3916555993 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7b019dd10 con 0x7fd7b00ffe50 2024-09-13T18:20:28.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.673+0000 7fd7a77fe640 1 -- 172.21.15.114:0/3916555993 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a0004a00 con 0x7fd7b00ffe50 2024-09-13T18:20:28.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.673+0000 7fd7b849c640 1 -- 172.21.15.114:0/3916555993 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7b019e1d0 con 0x7fd7b00ffe50 2024-09-13T18:20:28.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.674+0000 7fd7b849c640 1 -- 172.21.15.114:0/3916555993 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7b0106140 con 0x7fd7b00ffe50 2024-09-13T18:20:28.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.676+0000 7fd7a77fe640 1 -- 172.21.15.114:0/3916555993 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fd7a00079b0 con 0x7fd7b00ffe50 2024-09-13T18:20:28.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.676+0000 7fd7a77fe640 1 --2- 172.21.15.114:0/3916555993 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fd79c0a1090 0x7fd79c0a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:28.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.677+0000 7fd7b6238640 1 --2- 172.21.15.114:0/3916555993 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fd79c0a1090 0x7fd79c0a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:28.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.677+0000 7fd7a77fe640 1 -- 172.21.15.114:0/3916555993 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd7a00ccce0 con 0x7fd7b00ffe50 2024-09-13T18:20:28.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.677+0000 7fd7b6238640 1 --2- 172.21.15.114:0/3916555993 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fd79c0a1090 0x7fd79c0a3550 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac005fd0 tx=0x7fd7ac005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:28.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.680+0000 7fd7a77fe640 1 -- 172.21.15.114:0/3916555993 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fd7a0096160 con 0x7fd7b00ffe50 2024-09-13T18:20:28.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.867+0000 7fd7b849c640 1 -- 172.21.15.114:0/3916555993 --> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7fd7b002c450 con 0x7fd79c0a1090 2024-09-13T18:20:28.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.872+0000 7fd7a77fe640 1 -- 172.21.15.114:0/3916555993 <== mgr.14148 v2:172.21.15.114:6800/3396185541 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7fd7b002c450 con 0x7fd79c0a1090 2024-09-13T18:20:28.875 INFO:teuthology.orchestra.run.smithi114.stdout:/volumes/qa/sv_0/fc039f5d-b06e-441a-b01f-3622a83ec36a 2024-09-13T18:20:28.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.874+0000 7fd7b849c640 1 -- 172.21.15.114:0/3916555993 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fd79c0a1090 msgr2=0x7fd79c0a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:28.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.874+0000 7fd7b849c640 1 --2- 172.21.15.114:0/3916555993 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fd79c0a1090 0x7fd79c0a3550 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac005fd0 tx=0x7fd7ac005850 comp rx=0 tx=0).stop 2024-09-13T18:20:28.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.874+0000 7fd7b849c640 1 -- 172.21.15.114:0/3916555993 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd7b00ffe50 msgr2=0x7fd7b019a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:28.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.874+0000 7fd7b849c640 1 --2- 172.21.15.114:0/3916555993 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd7b00ffe50 0x7fd7b019a960 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd7a000c890 tx=0x7fd7a000cd50 comp rx=0 tx=0).stop 2024-09-13T18:20:28.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.874+0000 7fd7b849c640 1 -- 172.21.15.114:0/3916555993 shutdown_connections 2024-09-13T18:20:28.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.874+0000 7fd7b849c640 1 --2- 172.21.15.114:0/3916555993 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fd79c0a1090 0x7fd79c0a3550 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:28.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.874+0000 7fd7b849c640 1 --2- 172.21.15.114:0/3916555993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7b0100810 0x7fd7b019cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:28.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.874+0000 7fd7b849c640 1 --2- 172.21.15.114:0/3916555993 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd7b00ffe50 0x7fd7b019a960 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:28.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.874+0000 7fd7b849c640 1 --2- 172.21.15.114:0/3916555993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7b00fec60 0x7fd7b0068110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:28.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.874+0000 7fd7b849c640 1 -- 172.21.15.114:0/3916555993 >> 172.21.15.114:0/3916555993 conn(0x7fd7b00fa3e0 msgr2=0x7fd7b0103310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:28.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.874+0000 7fd7b849c640 1 -- 172.21.15.114:0/3916555993 shutdown_connections 2024-09-13T18:20:28.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.874+0000 7fd7b849c640 1 -- 172.21.15.114:0/3916555993 wait complete. 2024-09-13T18:20:28.888 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-13T18:20:28.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.988+0000 7f038a50f640 1 -- 172.21.15.114:0/1677606560 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0384100810 msgr2=0x7f0384102c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:28.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.988+0000 7f038a50f640 1 --2- 172.21.15.114:0/1677606560 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0384100810 0x7f0384102c00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0374009900 tx=0x7f037402a410 comp rx=0 tx=0).stop 2024-09-13T18:20:28.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.988+0000 7f038a50f640 1 -- 172.21.15.114:0/1677606560 shutdown_connections 2024-09-13T18:20:28.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.988+0000 7f038a50f640 1 --2- 172.21.15.114:0/1677606560 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0384105a70 0x7f0384107ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:28.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.988+0000 7f038a50f640 1 --2- 172.21.15.114:0/1677606560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0384103140 0x7f0384105530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:28.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.988+0000 7f038a50f640 1 --2- 172.21.15.114:0/1677606560 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0384100810 0x7f0384102c00 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:28.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.988+0000 7f038a50f640 1 -- 172.21.15.114:0/1677606560 >> 172.21.15.114:0/1677606560 conn(0x7f03840fa3f0 msgr2=0x7f03840fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:28.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.988+0000 7f038a50f640 1 -- 172.21.15.114:0/1677606560 shutdown_connections 2024-09-13T18:20:28.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.988+0000 7f038a50f640 1 -- 172.21.15.114:0/1677606560 wait complete. 2024-09-13T18:20:28.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.989+0000 7f038a50f640 1 Processor -- start 2024-09-13T18:20:28.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.989+0000 7f038a50f640 1 -- start start 2024-09-13T18:20:28.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.989+0000 7f038a50f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0384100810 0x7f0384195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:28.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.989+0000 7f038a50f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0384103140 0x7f0384196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:28.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.989+0000 7f038a50f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0384105a70 0x7f038419d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:28.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.989+0000 7f038a50f640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f038419d810 con 0x7f0384103140 2024-09-13T18:20:28.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.989+0000 7f038a50f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f038419d980 con 0x7f0384105a70 2024-09-13T18:20:28.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.989+0000 7f038a50f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f038419db40 con 0x7f0384100810 2024-09-13T18:20:28.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.990+0000 7f03837fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0384103140 0x7f0384196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:28.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.990+0000 7f03837fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0384103140 0x7f0384196270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60746/0 (socket says 172.21.15.114:60746) 2024-09-13T18:20:28.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.990+0000 7f03837fe640 1 -- 172.21.15.114:0/3140549139 learned_addr learned my addr 172.21.15.114:0/3140549139 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:20:28.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.990+0000 7f0388aac640 1 --2- 172.21.15.114:0/3140549139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0384105a70 0x7f038419d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:28.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.990+0000 7f0383fff640 1 --2- 172.21.15.114:0/3140549139 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0384100810 0x7f0384195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:28.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.990+0000 7f03837fe640 1 -- 172.21.15.114:0/3140549139 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0384100810 msgr2=0x7f0384195d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:28.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.990+0000 7f03837fe640 1 --2- 172.21.15.114:0/3140549139 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0384100810 0x7f0384195d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:28.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.990+0000 7f03837fe640 1 -- 172.21.15.114:0/3140549139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0384105a70 msgr2=0x7f038419d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:28.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.990+0000 7f03837fe640 1 --2- 172.21.15.114:0/3140549139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0384105a70 0x7f038419d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:28.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.990+0000 7f03837fe640 1 -- 172.21.15.114:0/3140549139 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0374009560 con 0x7f0384103140 2024-09-13T18:20:28.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.990+0000 7f03837fe640 1 --2- 172.21.15.114:0/3140549139 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0384103140 0x7f0384196270 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f037800d740 tx=0x7f037800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:28.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.990+0000 7f03817fa640 1 -- 172.21.15.114:0/3140549139 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0378009c30 con 0x7f0384103140 2024-09-13T18:20:28.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.990+0000 7f03817fa640 1 -- 172.21.15.114:0/3140549139 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f037800de50 con 0x7f0384103140 2024-09-13T18:20:28.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.990+0000 7f038a50f640 1 -- 172.21.15.114:0/3140549139 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f038419de20 con 0x7f0384103140 2024-09-13T18:20:28.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.990+0000 7f03817fa640 1 -- 172.21.15.114:0/3140549139 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0378004d30 con 0x7f0384103140 2024-09-13T18:20:28.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.990+0000 7f038a50f640 1 -- 172.21.15.114:0/3140549139 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f038419e1f0 con 0x7f0384103140 2024-09-13T18:20:28.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.991+0000 7f038a50f640 1 -- 172.21.15.114:0/3140549139 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0384103520 con 0x7f0384103140 2024-09-13T18:20:28.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.992+0000 7f03817fa640 1 -- 172.21.15.114:0/3140549139 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f0378004e90 con 0x7f0384103140 2024-09-13T18:20:28.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.993+0000 7f03817fa640 1 --2- 172.21.15.114:0/3140549139 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f036c0a1090 0x7f036c0a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:28.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.993+0000 7f0383fff640 1 --2- 172.21.15.114:0/3140549139 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f036c0a1090 0x7f036c0a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:28.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.993+0000 7f03817fa640 1 -- 172.21.15.114:0/3140549139 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f03780cc5b0 con 0x7f0384103140 2024-09-13T18:20:28.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.993+0000 7f0383fff640 1 --2- 172.21.15.114:0/3140549139 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f036c0a1090 0x7f036c0a3550 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0384197200 tx=0x7f0374005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:29.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:28.997+0000 7f03817fa640 1 -- 172.21.15.114:0/3140549139 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f0378095a50 con 0x7f0384103140 2024-09-13T18:20:29.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.183+0000 7f038a50f640 1 -- 172.21.15.114:0/3140549139 --> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f038402c580 con 0x7f036c0a1090 2024-09-13T18:20:29.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.208+0000 7f03817fa640 1 -- 172.21.15.114:0/3140549139 <== mgr.14148 v2:172.21.15.114:6800/3396185541 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f038402c580 con 0x7f036c0a1090 2024-09-13T18:20:29.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.210+0000 7f038a50f640 1 -- 172.21.15.114:0/3140549139 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f036c0a1090 msgr2=0x7f036c0a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:29.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.210+0000 7f038a50f640 1 --2- 172.21.15.114:0/3140549139 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f036c0a1090 0x7f036c0a3550 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0384197200 tx=0x7f0374005a20 comp rx=0 tx=0).stop 2024-09-13T18:20:29.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.210+0000 7f038a50f640 1 -- 172.21.15.114:0/3140549139 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0384103140 msgr2=0x7f0384196270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:29.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.210+0000 7f038a50f640 1 --2- 172.21.15.114:0/3140549139 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0384103140 0x7f0384196270 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f037800d740 tx=0x7f037800dc00 comp rx=0 tx=0).stop 2024-09-13T18:20:29.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.210+0000 7f038a50f640 1 -- 172.21.15.114:0/3140549139 shutdown_connections 2024-09-13T18:20:29.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.210+0000 7f038a50f640 1 --2- 172.21.15.114:0/3140549139 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f036c0a1090 0x7f036c0a3550 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:29.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.210+0000 7f038a50f640 1 --2- 172.21.15.114:0/3140549139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0384105a70 0x7f038419d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:29.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.210+0000 7f038a50f640 1 --2- 172.21.15.114:0/3140549139 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0384103140 0x7f0384196270 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:29.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.210+0000 7f038a50f640 1 --2- 172.21.15.114:0/3140549139 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0384100810 0x7f0384195d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:29.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.210+0000 7f038a50f640 1 -- 172.21.15.114:0/3140549139 >> 172.21.15.114:0/3140549139 conn(0x7f03840fa3f0 msgr2=0x7f0384101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:29.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.210+0000 7f038a50f640 1 -- 172.21.15.114:0/3140549139 shutdown_connections 2024-09-13T18:20:29.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.210+0000 7f038a50f640 1 -- 172.21.15.114:0/3140549139 wait complete. 2024-09-13T18:20:29.382 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-13T18:20:29.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:29 smithi204 ceph-mon[34777]: mds.? [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] up:active 2024-09-13T18:20:29.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:29 smithi204 ceph-mon[34777]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-13T18:20:29.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:29 smithi204 ceph-mon[34777]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-09-13T18:20:29.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:29 smithi204 ceph-mon[34777]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=e=up:creating} 8 up:standby 2024-09-13T18:20:29.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:29 smithi204 ceph-mon[34777]: daemon mds.e is now active in filesystem cephfs as rank 3 2024-09-13T18:20:29.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:29 smithi204 ceph-mon[34777]: mds.? [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] up:active 2024-09-13T18:20:29.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:29 smithi204 ceph-mon[34777]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-13T18:20:29.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:29 smithi204 ceph-mon[34777]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-13T18:20:29.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:29 smithi204 ceph-mon[34777]: Cluster is now healthy 2024-09-13T18:20:29.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:29 smithi204 ceph-mon[34777]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=e=up:active} 4 up:standby-replay 4 up:standby 2024-09-13T18:20:29.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:29 smithi204 ceph-mon[34777]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-09-13T18:20:29.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:29 smithi204 ceph-mon[34777]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-09-13T18:20:29.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:29 smithi204 ceph-mon[34777]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:20:29.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:29 smithi204 ceph-mon[34777]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:20:29.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:29 smithi114 ceph-mon[30897]: mds.? [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] up:active 2024-09-13T18:20:29.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:29 smithi114 ceph-mon[30897]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-13T18:20:29.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:29 smithi114 ceph-mon[30897]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-09-13T18:20:29.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:29 smithi114 ceph-mon[30897]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=e=up:creating} 8 up:standby 2024-09-13T18:20:29.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:29 smithi114 ceph-mon[30897]: daemon mds.e is now active in filesystem cephfs as rank 3 2024-09-13T18:20:29.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:29 smithi114 ceph-mon[30897]: mds.? [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] up:active 2024-09-13T18:20:29.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:29 smithi114 ceph-mon[30897]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-13T18:20:29.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:29 smithi114 ceph-mon[30897]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-13T18:20:29.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:29 smithi114 ceph-mon[30897]: Cluster is now healthy 2024-09-13T18:20:29.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:29 smithi114 ceph-mon[30897]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=e=up:active} 4 up:standby-replay 4 up:standby 2024-09-13T18:20:29.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:29 smithi114 ceph-mon[30897]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-09-13T18:20:29.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:29 smithi114 ceph-mon[30897]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-09-13T18:20:29.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:29 smithi114 ceph-mon[30897]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:20:29.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:29 smithi114 ceph-mon[30897]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:20:29.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.493+0000 7fefd0f1f640 1 -- 172.21.15.114:0/4120442710 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fefcc103110 msgr2=0x7fefcc105500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:29.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.493+0000 7fefd0f1f640 1 --2- 172.21.15.114:0/4120442710 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fefcc103110 0x7fefcc105500 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fefbc009900 tx=0x7fefbc02a410 comp rx=0 tx=0).stop 2024-09-13T18:20:29.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.494+0000 7fefd0f1f640 1 -- 172.21.15.114:0/4120442710 shutdown_connections 2024-09-13T18:20:29.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.494+0000 7fefd0f1f640 1 --2- 172.21.15.114:0/4120442710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fefcc105a40 0x7fefcc107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:29.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.494+0000 7fefd0f1f640 1 --2- 172.21.15.114:0/4120442710 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fefcc103110 0x7fefcc105500 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:29.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.494+0000 7fefd0f1f640 1 --2- 172.21.15.114:0/4120442710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefcc1007e0 0x7fefcc102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:29.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.494+0000 7fefd0f1f640 1 -- 172.21.15.114:0/4120442710 >> 172.21.15.114:0/4120442710 conn(0x7fefcc0fa3e0 msgr2=0x7fefcc0fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:29.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.494+0000 7fefd0f1f640 1 -- 172.21.15.114:0/4120442710 shutdown_connections 2024-09-13T18:20:29.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.494+0000 7fefd0f1f640 1 -- 172.21.15.114:0/4120442710 wait complete. 2024-09-13T18:20:29.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.494+0000 7fefd0f1f640 1 Processor -- start 2024-09-13T18:20:29.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.494+0000 7fefd0f1f640 1 -- start start 2024-09-13T18:20:29.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.494+0000 7fefd0f1f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefcc1007e0 0x7fefcc195ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:29.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.495+0000 7fefd0f1f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fefcc103110 0x7fefcc1961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:29.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.495+0000 7fefd0f1f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fefcc105a40 0x7fefcc19d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:29.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.495+0000 7fefd0f1f640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefcc19d7b0 con 0x7fefcc105a40 2024-09-13T18:20:29.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.495+0000 7fefd0f1f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefcc19d920 con 0x7fefcc103110 2024-09-13T18:20:29.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.495+0000 7fefd0f1f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefcc19dae0 con 0x7fefcc1007e0 2024-09-13T18:20:29.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.495+0000 7fefcad9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fefcc105a40 0x7fefcc19d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:29.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.495+0000 7fefcad9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fefcc105a40 0x7fefcc19d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60766/0 (socket says 172.21.15.114:60766) 2024-09-13T18:20:29.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.495+0000 7fefcad9d640 1 -- 172.21.15.114:0/2705129389 learned_addr learned my addr 172.21.15.114:0/2705129389 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:20:29.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.495+0000 7fefc9d9b640 1 --2- 172.21.15.114:0/2705129389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fefcc103110 0x7fefcc1961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:29.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.495+0000 7fefca59c640 1 --2- 172.21.15.114:0/2705129389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefcc1007e0 0x7fefcc195ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:29.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.495+0000 7fefcad9d640 1 -- 172.21.15.114:0/2705129389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefcc1007e0 msgr2=0x7fefcc195ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:29.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.495+0000 7fefcad9d640 1 --2- 172.21.15.114:0/2705129389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefcc1007e0 0x7fefcc195ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:29.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.495+0000 7fefcad9d640 1 -- 172.21.15.114:0/2705129389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fefcc103110 msgr2=0x7fefcc1961e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:29.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.495+0000 7fefcad9d640 1 --2- 172.21.15.114:0/2705129389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fefcc103110 0x7fefcc1961e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:29.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.495+0000 7fefcad9d640 1 -- 172.21.15.114:0/2705129389 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefbc009560 con 0x7fefcc105a40 2024-09-13T18:20:29.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.495+0000 7fefc9d9b640 1 --2- 172.21.15.114:0/2705129389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fefcc103110 0x7fefcc1961e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:20:29.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.495+0000 7fefcad9d640 1 --2- 172.21.15.114:0/2705129389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fefcc105a40 0x7fefcc19d240 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fefc400e9d0 tx=0x7fefc400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:29.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.495+0000 7fefbb7fe640 1 -- 172.21.15.114:0/2705129389 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc400cd80 con 0x7fefcc105a40 2024-09-13T18:20:29.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.495+0000 7fefd0f1f640 1 -- 172.21.15.114:0/2705129389 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefcc19ddc0 con 0x7fefcc105a40 2024-09-13T18:20:29.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.495+0000 7fefbb7fe640 1 -- 172.21.15.114:0/2705129389 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fefc4005020 con 0x7fefcc105a40 2024-09-13T18:20:29.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.495+0000 7fefbb7fe640 1 -- 172.21.15.114:0/2705129389 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc40116d0 con 0x7fefcc105a40 2024-09-13T18:20:29.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.495+0000 7fefd0f1f640 1 -- 172.21.15.114:0/2705129389 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefcc19e1f0 con 0x7fefcc105a40 2024-09-13T18:20:29.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.496+0000 7fefd0f1f640 1 -- 172.21.15.114:0/2705129389 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefcc0fbed0 con 0x7fefcc105a40 2024-09-13T18:20:29.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.497+0000 7fefbb7fe640 1 -- 172.21.15.114:0/2705129389 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fefc4005190 con 0x7fefcc105a40 2024-09-13T18:20:29.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.498+0000 7fefbb7fe640 1 --2- 172.21.15.114:0/2705129389 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fefb40a1090 0x7fefb40a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:29.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.498+0000 7fefca59c640 1 --2- 172.21.15.114:0/2705129389 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fefb40a1090 0x7fefb40a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:29.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.498+0000 7fefbb7fe640 1 -- 172.21.15.114:0/2705129389 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fefc40cce70 con 0x7fefcc105a40 2024-09-13T18:20:29.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.498+0000 7fefca59c640 1 --2- 172.21.15.114:0/2705129389 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fefb40a1090 0x7fefb40a3550 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fefc0005fd0 tx=0x7fefc0005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:29.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.506+0000 7fefbb7fe640 1 -- 172.21.15.114:0/2705129389 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fefc40962f0 con 0x7fefcc105a40 2024-09-13T18:20:29.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:29 smithi134 ceph-mon[34283]: mds.? [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] up:active 2024-09-13T18:20:29.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:29 smithi134 ceph-mon[34283]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-13T18:20:29.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:29 smithi134 ceph-mon[34283]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-09-13T18:20:29.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:29 smithi134 ceph-mon[34283]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=e=up:creating} 8 up:standby 2024-09-13T18:20:29.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:29 smithi134 ceph-mon[34283]: daemon mds.e is now active in filesystem cephfs as rank 3 2024-09-13T18:20:29.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:29 smithi134 ceph-mon[34283]: mds.? [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] up:active 2024-09-13T18:20:29.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:29 smithi134 ceph-mon[34283]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-13T18:20:29.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:29 smithi134 ceph-mon[34283]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-13T18:20:29.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:29 smithi134 ceph-mon[34283]: Cluster is now healthy 2024-09-13T18:20:29.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:29 smithi134 ceph-mon[34283]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=e=up:active} 4 up:standby-replay 4 up:standby 2024-09-13T18:20:29.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:29 smithi134 ceph-mon[34283]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-09-13T18:20:29.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:29 smithi134 ceph-mon[34283]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-09-13T18:20:29.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:29 smithi134 ceph-mon[34283]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:20:29.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:29 smithi134 ceph-mon[34283]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:20:29.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.734+0000 7fefd0f1f640 1 -- 172.21.15.114:0/2705129389 --> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fefcc02c450 con 0x7fefb40a1090 2024-09-13T18:20:29.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.738+0000 7fefbb7fe640 1 -- 172.21.15.114:0/2705129389 <== mgr.14148 v2:172.21.15.114:6800/3396185541 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7fefcc02c450 con 0x7fefb40a1090 2024-09-13T18:20:29.741 INFO:teuthology.orchestra.run.smithi114.stdout:/volumes/qa/sv_1/602a0858-8235-41c5-ac37-2915dd15bd9e 2024-09-13T18:20:29.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.740+0000 7fefd0f1f640 1 -- 172.21.15.114:0/2705129389 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fefb40a1090 msgr2=0x7fefb40a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:29.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.740+0000 7fefd0f1f640 1 --2- 172.21.15.114:0/2705129389 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fefb40a1090 0x7fefb40a3550 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fefc0005fd0 tx=0x7fefc0005850 comp rx=0 tx=0).stop 2024-09-13T18:20:29.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.740+0000 7fefd0f1f640 1 -- 172.21.15.114:0/2705129389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fefcc105a40 msgr2=0x7fefcc19d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:29.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.740+0000 7fefd0f1f640 1 --2- 172.21.15.114:0/2705129389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fefcc105a40 0x7fefcc19d240 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fefc400e9d0 tx=0x7fefc400ee90 comp rx=0 tx=0).stop 2024-09-13T18:20:29.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.740+0000 7fefd0f1f640 1 -- 172.21.15.114:0/2705129389 shutdown_connections 2024-09-13T18:20:29.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.740+0000 7fefd0f1f640 1 --2- 172.21.15.114:0/2705129389 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fefb40a1090 0x7fefb40a3550 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:29.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.740+0000 7fefd0f1f640 1 --2- 172.21.15.114:0/2705129389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fefcc105a40 0x7fefcc19d240 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:29.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.740+0000 7fefd0f1f640 1 --2- 172.21.15.114:0/2705129389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fefcc103110 0x7fefcc1961e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:29.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.740+0000 7fefd0f1f640 1 --2- 172.21.15.114:0/2705129389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefcc1007e0 0x7fefcc195ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:29.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.740+0000 7fefd0f1f640 1 -- 172.21.15.114:0/2705129389 >> 172.21.15.114:0/2705129389 conn(0x7fefcc0fa3e0 msgr2=0x7fefcc101140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:29.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.740+0000 7fefd0f1f640 1 -- 172.21.15.114:0/2705129389 shutdown_connections 2024-09-13T18:20:29.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.740+0000 7fefd0f1f640 1 -- 172.21.15.114:0/2705129389 wait complete. 2024-09-13T18:20:29.755 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:20:29.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.853+0000 7f7f1f0d0640 1 -- 172.21.15.114:0/1302738945 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f180ffe90 msgr2=0x7f7f18100310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:29.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.853+0000 7f7f1f0d0640 1 --2- 172.21.15.114:0/1302738945 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f180ffe90 0x7f7f18100310 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7f04009900 tx=0x7f7f0402a410 comp rx=0 tx=0).stop 2024-09-13T18:20:29.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.853+0000 7f7f1f0d0640 1 -- 172.21.15.114:0/1302738945 shutdown_connections 2024-09-13T18:20:29.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.853+0000 7f7f1f0d0640 1 --2- 172.21.15.114:0/1302738945 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f18100850 0x7f7f181070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:29.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.853+0000 7f7f1f0d0640 1 --2- 172.21.15.114:0/1302738945 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f180ffe90 0x7f7f18100310 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:29.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.853+0000 7f7f1f0d0640 1 --2- 172.21.15.114:0/1302738945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f180feca0 0x7f7f180ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:29.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.853+0000 7f7f1f0d0640 1 -- 172.21.15.114:0/1302738945 >> 172.21.15.114:0/1302738945 conn(0x7f7f180fa440 msgr2=0x7f7f180fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:29.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.853+0000 7f7f1f0d0640 1 -- 172.21.15.114:0/1302738945 shutdown_connections 2024-09-13T18:20:29.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.853+0000 7f7f1f0d0640 1 -- 172.21.15.114:0/1302738945 wait complete. 2024-09-13T18:20:29.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.854+0000 7f7f1f0d0640 1 Processor -- start 2024-09-13T18:20:29.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.854+0000 7f7f1f0d0640 1 -- start start 2024-09-13T18:20:29.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.854+0000 7f7f1f0d0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f180feca0 0x7f7f18195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:29.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.854+0000 7f7f1f0d0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f180ffe90 0x7f7f181962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:29.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.854+0000 7f7f1f0d0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f18100850 0x7f7f1819d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:29.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.854+0000 7f7f1f0d0640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1819d850 con 0x7f7f180ffe90 2024-09-13T18:20:29.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.854+0000 7f7f1f0d0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1819d9c0 con 0x7f7f18100850 2024-09-13T18:20:29.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.854+0000 7f7f1f0d0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1819db80 con 0x7f7f180feca0 2024-09-13T18:20:29.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.854+0000 7f7f17fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f180ffe90 0x7f7f181962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:29.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.854+0000 7f7f17fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f180ffe90 0x7f7f181962a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60780/0 (socket says 172.21.15.114:60780) 2024-09-13T18:20:29.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.854+0000 7f7f17fff640 1 -- 172.21.15.114:0/1587228232 learned_addr learned my addr 172.21.15.114:0/1587228232 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:20:29.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.854+0000 7f7f1ce6c640 1 --2- 172.21.15.114:0/1587228232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f180feca0 0x7f7f18195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:29.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.854+0000 7f7f1d66d640 1 --2- 172.21.15.114:0/1587228232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f18100850 0x7f7f1819d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:29.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.854+0000 7f7f17fff640 1 -- 172.21.15.114:0/1587228232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f180feca0 msgr2=0x7f7f18195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:29.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.854+0000 7f7f17fff640 1 --2- 172.21.15.114:0/1587228232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f180feca0 0x7f7f18195d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:29.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.854+0000 7f7f17fff640 1 -- 172.21.15.114:0/1587228232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f18100850 msgr2=0x7f7f1819d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:29.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.854+0000 7f7f17fff640 1 --2- 172.21.15.114:0/1587228232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f18100850 0x7f7f1819d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:29.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.854+0000 7f7f17fff640 1 -- 172.21.15.114:0/1587228232 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f04009560 con 0x7f7f180ffe90 2024-09-13T18:20:29.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.854+0000 7f7f1d66d640 1 --2- 172.21.15.114:0/1587228232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f18100850 0x7f7f1819d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:20:29.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.854+0000 7f7f17fff640 1 --2- 172.21.15.114:0/1587228232 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f180ffe90 0x7f7f181962a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7f04006010 tx=0x7f7f04004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:29.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.855+0000 7f7f15ffb640 1 -- 172.21.15.114:0/1587228232 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f040037f0 con 0x7f7f180ffe90 2024-09-13T18:20:29.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.855+0000 7f7f1f0d0640 1 -- 172.21.15.114:0/1587228232 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f1819de00 con 0x7f7f180ffe90 2024-09-13T18:20:29.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.855+0000 7f7f15ffb640 1 -- 172.21.15.114:0/1587228232 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7f040044f0 con 0x7f7f180ffe90 2024-09-13T18:20:29.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.855+0000 7f7f15ffb640 1 -- 172.21.15.114:0/1587228232 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0403ca80 con 0x7f7f180ffe90 2024-09-13T18:20:29.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.855+0000 7f7f1f0d0640 1 -- 172.21.15.114:0/1587228232 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f1819e220 con 0x7f7f180ffe90 2024-09-13T18:20:29.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.856+0000 7f7f1f0d0640 1 -- 172.21.15.114:0/1587228232 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f1810a440 con 0x7f7f180ffe90 2024-09-13T18:20:29.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.857+0000 7f7f15ffb640 1 -- 172.21.15.114:0/1587228232 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f7f0403cbe0 con 0x7f7f180ffe90 2024-09-13T18:20:29.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.858+0000 7f7f15ffb640 1 --2- 172.21.15.114:0/1587228232 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f7f080a1090 0x7f7f080a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:29.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.858+0000 7f7f1ce6c640 1 --2- 172.21.15.114:0/1587228232 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f7f080a1090 0x7f7f080a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:29.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.858+0000 7f7f15ffb640 1 -- 172.21.15.114:0/1587228232 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7f040ee090 con 0x7f7f180ffe90 2024-09-13T18:20:29.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.858+0000 7f7f1ce6c640 1 --2- 172.21.15.114:0/1587228232 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f7f080a1090 0x7f7f080a3550 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c005fd0 tx=0x7f7f0c005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:29.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:29.864+0000 7f7f15ffb640 1 -- 172.21.15.114:0/1587228232 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f7f040b7510 con 0x7f7f180ffe90 2024-09-13T18:20:30.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.069+0000 7f7f1f0d0640 1 -- 172.21.15.114:0/1587228232 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f1810a710 con 0x7f7f180ffe90 2024-09-13T18:20:30.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.070+0000 7f7f15ffb640 1 -- 172.21.15.114:0/1587228232 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10930 (secure 0 0 0) 0x7f7f04040090 con 0x7f7f180ffe90 2024-09-13T18:20:30.077 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:20:30.077 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4162817497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4162817497},{"type":"v1","addr":"172.21.15.114:6835","nonce":4162817497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/2435428478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":2435428478},{"type":"v1","addr":"172.21.15.114:6837","nonce":2435428478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T18:20:01.895117+0000","modified":"2024-09-13T18:20:29.250121+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24482,"mds_2":24475,"mds_3":24458,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/2046516451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":2046516451},{"type":"v1","addr":"172.21.15.114:6839","nonce":2046516451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":10,"addr":"172.21.15.114:6841/1956923446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":1956923446},{"type":"v1","addr":"172.21.15.114:6841","nonce":1956923446}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24391":{"gid":24391,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/507301496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":507301496},{"type":"v1","addr":"172.21.15.204:6833","nonce":507301496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4028657219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4028657219},{"type":"v1","addr":"172.21.15.134:6833","nonce":4028657219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/57029119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":57029119},{"type":"v1","addr":"172.21.15.204:6835","nonce":57029119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3068880939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3068880939},{"type":"v1","addr":"172.21.15.204:6837","nonce":3068880939}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"e","rank":3,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.134:6835/1150003275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1150003275},{"type":"v1","addr":"172.21.15.134:6835","nonce":1150003275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/344883912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":344883912},{"type":"v1","addr":"172.21.15.204:6839","nonce":344883912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6837/891692210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":891692210},{"type":"v1","addr":"172.21.15.134:6837","nonce":891692210}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6839/3466309724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3466309724},{"type":"v1","addr":"172.21.15.134:6839","nonce":3466309724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:20:30.078 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 29 2024-09-13T18:20:30.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.072+0000 7f7f1f0d0640 1 -- 172.21.15.114:0/1587228232 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f7f080a1090 msgr2=0x7f7f080a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:30.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.072+0000 7f7f1f0d0640 1 --2- 172.21.15.114:0/1587228232 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f7f080a1090 0x7f7f080a3550 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c005fd0 tx=0x7f7f0c005850 comp rx=0 tx=0).stop 2024-09-13T18:20:30.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.072+0000 7f7f1f0d0640 1 -- 172.21.15.114:0/1587228232 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f180ffe90 msgr2=0x7f7f181962a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:30.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.072+0000 7f7f1f0d0640 1 --2- 172.21.15.114:0/1587228232 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f180ffe90 0x7f7f181962a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7f04006010 tx=0x7f7f04004970 comp rx=0 tx=0).stop 2024-09-13T18:20:30.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.072+0000 7f7f1f0d0640 1 -- 172.21.15.114:0/1587228232 shutdown_connections 2024-09-13T18:20:30.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.072+0000 7f7f1f0d0640 1 --2- 172.21.15.114:0/1587228232 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f7f080a1090 0x7f7f080a3550 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:30.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.072+0000 7f7f1f0d0640 1 --2- 172.21.15.114:0/1587228232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f18100850 0x7f7f1819d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:30.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.072+0000 7f7f1f0d0640 1 --2- 172.21.15.114:0/1587228232 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7f180ffe90 0x7f7f181962a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:30.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.072+0000 7f7f1f0d0640 1 --2- 172.21.15.114:0/1587228232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f180feca0 0x7f7f18195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:30.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.072+0000 7f7f1f0d0640 1 -- 172.21.15.114:0/1587228232 >> 172.21.15.114:0/1587228232 conn(0x7f7f180fa440 msgr2=0x7f7f18067550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:30.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.072+0000 7f7f1f0d0640 1 -- 172.21.15.114:0/1587228232 shutdown_connections 2024-09-13T18:20:30.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.072+0000 7f7f1f0d0640 1 -- 172.21.15.114:0/1587228232 wait complete. 2024-09-13T18:20:30.087 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:20:30.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.186+0000 7f3027409640 1 -- 172.21.15.114:0/3884316555 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30201008c0 msgr2=0x7f3020102cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:30.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.186+0000 7f3027409640 1 --2- 172.21.15.114:0/3884316555 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30201008c0 0x7f3020102cb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f301c009900 tx=0x7f301c02a410 comp rx=0 tx=0).stop 2024-09-13T18:20:30.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.187+0000 7f3027409640 1 -- 172.21.15.114:0/3884316555 shutdown_connections 2024-09-13T18:20:30.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.187+0000 7f3027409640 1 --2- 172.21.15.114:0/3884316555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3020105b20 0x7f3020107f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:30.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.187+0000 7f3027409640 1 --2- 172.21.15.114:0/3884316555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30201031f0 0x7f30201055e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:30.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.187+0000 7f3027409640 1 --2- 172.21.15.114:0/3884316555 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30201008c0 0x7f3020102cb0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:30.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.187+0000 7f3027409640 1 -- 172.21.15.114:0/3884316555 >> 172.21.15.114:0/3884316555 conn(0x7f30200fa440 msgr2=0x7f30200fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:30.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.187+0000 7f3027409640 1 -- 172.21.15.114:0/3884316555 shutdown_connections 2024-09-13T18:20:30.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.187+0000 7f3027409640 1 -- 172.21.15.114:0/3884316555 wait complete. 2024-09-13T18:20:30.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.187+0000 7f3027409640 1 Processor -- start 2024-09-13T18:20:30.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.187+0000 7f3027409640 1 -- start start 2024-09-13T18:20:30.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.187+0000 7f3027409640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30201008c0 0x7f3020195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:30.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.187+0000 7f3027409640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30201031f0 0x7f3020196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:30.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.187+0000 7f3027409640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3020105b20 0x7f302019d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:30.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.187+0000 7f3027409640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302019d8a0 con 0x7f30201008c0 2024-09-13T18:20:30.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.187+0000 7f3027409640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302019da10 con 0x7f3020105b20 2024-09-13T18:20:30.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.187+0000 7f3027409640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302019dbd0 con 0x7f30201031f0 2024-09-13T18:20:30.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.187+0000 7f30251a5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30201008c0 0x7f3020195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:30.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.187+0000 7f30251a5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30201008c0 0x7f3020195dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60804/0 (socket says 172.21.15.114:60804) 2024-09-13T18:20:30.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.187+0000 7f30251a5640 1 -- 172.21.15.114:0/2106708598 learned_addr learned my addr 172.21.15.114:0/2106708598 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:20:30.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.188+0000 7f30259a6640 1 --2- 172.21.15.114:0/2106708598 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3020105b20 0x7f302019d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:30.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.188+0000 7f30249a4640 1 --2- 172.21.15.114:0/2106708598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30201031f0 0x7f3020196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:30.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.188+0000 7f30251a5640 1 -- 172.21.15.114:0/2106708598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30201031f0 msgr2=0x7f3020196300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:30.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.188+0000 7f30251a5640 1 --2- 172.21.15.114:0/2106708598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30201031f0 0x7f3020196300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:30.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.188+0000 7f30251a5640 1 -- 172.21.15.114:0/2106708598 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3020105b20 msgr2=0x7f302019d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:30.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.188+0000 7f30251a5640 1 --2- 172.21.15.114:0/2106708598 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3020105b20 0x7f302019d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:30.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.188+0000 7f30251a5640 1 -- 172.21.15.114:0/2106708598 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f301c009560 con 0x7f30201008c0 2024-09-13T18:20:30.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.188+0000 7f30249a4640 1 --2- 172.21.15.114:0/2106708598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30201031f0 0x7f3020196300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:20:30.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.188+0000 7f30251a5640 1 --2- 172.21.15.114:0/2106708598 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30201008c0 0x7f3020195dc0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f301c005ab0 tx=0x7f301c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:30.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.188+0000 7f30127fc640 1 -- 172.21.15.114:0/2106708598 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f301c031d70 con 0x7f30201008c0 2024-09-13T18:20:30.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.188+0000 7f30127fc640 1 -- 172.21.15.114:0/2106708598 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f301c0044e0 con 0x7f30201008c0 2024-09-13T18:20:30.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.188+0000 7f3027409640 1 -- 172.21.15.114:0/2106708598 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f302019de50 con 0x7f30201008c0 2024-09-13T18:20:30.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.188+0000 7f30127fc640 1 -- 172.21.15.114:0/2106708598 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f301c003b50 con 0x7f30201008c0 2024-09-13T18:20:30.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.188+0000 7f3027409640 1 -- 172.21.15.114:0/2106708598 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f302019e210 con 0x7f30201008c0 2024-09-13T18:20:30.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.189+0000 7f3027409640 1 -- 172.21.15.114:0/2106708598 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30201035d0 con 0x7f30201008c0 2024-09-13T18:20:30.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.190+0000 7f30127fc640 1 -- 172.21.15.114:0/2106708598 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f301c003cb0 con 0x7f30201008c0 2024-09-13T18:20:30.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.191+0000 7f30127fc640 1 --2- 172.21.15.114:0/2106708598 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f300c0a1090 0x7f300c0a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:30.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.191+0000 7f30127fc640 1 -- 172.21.15.114:0/2106708598 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f301c0ee2d0 con 0x7f30201008c0 2024-09-13T18:20:30.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.191+0000 7f30249a4640 1 --2- 172.21.15.114:0/2106708598 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f300c0a1090 0x7f300c0a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:30.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.192+0000 7f30249a4640 1 --2- 172.21.15.114:0/2106708598 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f300c0a1090 0x7f300c0a3550 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f30140058c0 tx=0x7f3014005d30 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:30.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.194+0000 7f30127fc640 1 -- 172.21.15.114:0/2106708598 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f301c0b7760 con 0x7f30201008c0 2024-09-13T18:20:30.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.271+0000 7f30127fc640 1 -- 172.21.15.114:0/2106708598 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f301c0b7b30 con 0x7f30201008c0 2024-09-13T18:20:30.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.368+0000 7f3027409640 1 -- 172.21.15.114:0/2106708598 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f30200fbf30 con 0x7f30201008c0 2024-09-13T18:20:30.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.370+0000 7f30127fc640 1 -- 172.21.15.114:0/2106708598 <== mon.0 v2:172.21.15.114:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24556 (secure 0 0 0) 0x7f301c0bb280 con 0x7f30201008c0 2024-09-13T18:20:30.373 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:20:30.373 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":91,"fsid":"97e7edf2-71fb-11ef-bceb-c7b262605968","created":"2024-09-13T18:12:36.360625+0000","modified":"2024-09-13T18:20:27.176653+0000","last_up_change":"2024-09-13T18:19:12.350370+0000","last_in_change":"2024-09-13T18:18:52.321596+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:15:40.129659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:20:00.024407+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:20:01.085549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T18:20:04.334216+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9914f01a-9f6f-4878-b955-83570199c354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6803","nonce":3499525826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6805","nonce":3499525826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6809","nonce":3499525826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6807","nonce":3499525826}]},"public_addr":"172.21.15.114:6803/3499525826","cluster_addr":"172.21.15.114:6805/3499525826","heartbeat_back_addr":"172.21.15.114:6809/3499525826","heartbeat_front_addr":"172.21.15.114:6807/3499525826","state":["exists","up"]},{"osd":1,"uuid":"ef65e012-7469-4c20-b354-17022bca63c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6801","nonce":4264264866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6803","nonce":4264264866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6807","nonce":4264264866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6805","nonce":4264264866}]},"public_addr":"172.21.15.134:6801/4264264866","cluster_addr":"172.21.15.134:6803/4264264866","heartbeat_back_addr":"172.21.15.134:6807/4264264866","heartbeat_front_addr":"172.21.15.134:6805/4264264866","state":["exists","up"]},{"osd":2,"uuid":"578213fd-715f-4be7-91fa-254f3d0e3022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6801","nonce":2288807707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6803","nonce":2288807707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6807","nonce":2288807707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6805","nonce":2288807707}]},"public_addr":"172.21.15.204:6801/2288807707","cluster_addr":"172.21.15.204:6803/2288807707","heartbeat_back_addr":"172.21.15.204:6807/2288807707","heartbeat_front_addr":"172.21.15.204:6805/2288807707","state":["exists","up"]},{"osd":3,"uuid":"b64aab3f-1e52-4828-8397-74af5e05f2cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6811","nonce":2904204254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6813","nonce":2904204254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6817","nonce":2904204254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6815","nonce":2904204254}]},"public_addr":"172.21.15.114:6811/2904204254","cluster_addr":"172.21.15.114:6813/2904204254","heartbeat_back_addr":"172.21.15.114:6817/2904204254","heartbeat_front_addr":"172.21.15.114:6815/2904204254","state":["exists","up"]},{"osd":4,"uuid":"f38943d6-39c0-42e1-b427-cb6bb90ac38a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6809","nonce":1773945333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6811","nonce":1773945333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6815","nonce":1773945333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6813","nonce":1773945333}]},"public_addr":"172.21.15.134:6809/1773945333","cluster_addr":"172.21.15.134:6811/1773945333","heartbeat_back_addr":"172.21.15.134:6815/1773945333","heartbeat_front_addr":"172.21.15.134:6813/1773945333","state":["exists","up"]},{"osd":5,"uuid":"bb38f361-8e00-4178-9752-5ddbf3af20e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6809","nonce":2371778055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6811","nonce":2371778055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6815","nonce":2371778055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6813","nonce":2371778055}]},"public_addr":"172.21.15.204:6809/2371778055","cluster_addr":"172.21.15.204:6811/2371778055","heartbeat_back_addr":"172.21.15.204:6815/2371778055","heartbeat_front_addr":"172.21.15.204:6813/2371778055","state":["exists","up"]},{"osd":6,"uuid":"93d40a03-e7ff-4a3f-9efa-5c5403c9983d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6819","nonce":1484325044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6821","nonce":1484325044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6825","nonce":1484325044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6823","nonce":1484325044}]},"public_addr":"172.21.15.114:6819/1484325044","cluster_addr":"172.21.15.114:6821/1484325044","heartbeat_back_addr":"172.21.15.114:6825/1484325044","heartbeat_front_addr":"172.21.15.114:6823/1484325044","state":["exists","up"]},{"osd":7,"uuid":"982f629a-1b95-4be1-8e75-5ddce1ad77b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6817","nonce":799246614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6819","nonce":799246614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6823","nonce":799246614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6821","nonce":799246614}]},"public_addr":"172.21.15.134:6817/799246614","cluster_addr":"172.21.15.134:6819/799246614","heartbeat_back_addr":"172.21.15.134:6823/799246614","heartbeat_front_addr":"172.21.15.134:6821/799246614","state":["exists","up"]},{"osd":8,"uuid":"409beb9b-8227-4dad-a4a0-f0e5ce70dc5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6817","nonce":858331805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6819","nonce":858331805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6823","nonce":858331805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6821","nonce":858331805}]},"public_addr":"172.21.15.204:6817/858331805","cluster_addr":"172.21.15.204:6819/858331805","heartbeat_back_addr":"172.21.15.204:6823/858331805","heartbeat_front_addr":"172.21.15.204:6821/858331805","state":["exists","up"]},{"osd":9,"uuid":"a345a15c-b1fc-4877-b445-418eebf0d0ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6827","nonce":2172392854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6829","nonce":2172392854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6833","nonce":2172392854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6831","nonce":2172392854}]},"public_addr":"172.21.15.114:6827/2172392854","cluster_addr":"172.21.15.114:6829/2172392854","heartbeat_back_addr":"172.21.15.114:6833/2172392854","heartbeat_front_addr":"172.21.15.114:6831/2172392854","state":["exists","up"]},{"osd":10,"uuid":"464d995f-d90b-4caa-addb-1f7562bd74b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6825","nonce":3073513468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6827","nonce":3073513468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6831","nonce":3073513468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6829","nonce":3073513468}]},"public_addr":"172.21.15.134:6825/3073513468","cluster_addr":"172.21.15.134:6827/3073513468","heartbeat_back_addr":"172.21.15.134:6831/3073513468","heartbeat_front_addr":"172.21.15.134:6829/3073513468","state":["exists","up"]},{"osd":11,"uuid":"71dd794d-326c-477f-8d02-ee32bff1350d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6825","nonce":3239521765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6827","nonce":3239521765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6831","nonce":3239521765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6829","nonce":3239521765}]},"public_addr":"172.21.15.204:6825/3239521765","cluster_addr":"172.21.15.204:6827/3239521765","heartbeat_back_addr":"172.21.15.204:6831/3239521765","heartbeat_front_addr":"172.21.15.204:6829/3239521765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:14:54.542687+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:15:15.746055+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:15:38.371131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:01.383932+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:24.796237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:48.422214+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:11.284323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:34.570365+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:58.232855+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:18:21.609596+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:18:45.565859+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:19:10.017969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:6801/4281866858":"2024-09-14T18:13:25.823027+0000","172.21.15.114:6800/4281866858":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2949422849":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2034742251":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/722571999":"2024-09-14T18:13:03.521116+0000","172.21.15.114:6801/830738794":"2024-09-14T18:13:03.521116+0000","172.21.15.114:6800/830738794":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/2828364892":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/4257828906":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/3544951322":"2024-09-14T18:13:03.521116+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:20:30.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.372+0000 7f3027409640 1 -- 172.21.15.114:0/2106708598 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f300c0a1090 msgr2=0x7f300c0a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:30.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.372+0000 7f3027409640 1 --2- 172.21.15.114:0/2106708598 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f300c0a1090 0x7f300c0a3550 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f30140058c0 tx=0x7f3014005d30 comp rx=0 tx=0).stop 2024-09-13T18:20:30.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.372+0000 7f3027409640 1 -- 172.21.15.114:0/2106708598 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30201008c0 msgr2=0x7f3020195dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:30.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.372+0000 7f3027409640 1 --2- 172.21.15.114:0/2106708598 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30201008c0 0x7f3020195dc0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f301c005ab0 tx=0x7f301c0042f0 comp rx=0 tx=0).stop 2024-09-13T18:20:30.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.372+0000 7f3027409640 1 -- 172.21.15.114:0/2106708598 shutdown_connections 2024-09-13T18:20:30.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.372+0000 7f3027409640 1 --2- 172.21.15.114:0/2106708598 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f300c0a1090 0x7f300c0a3550 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:30.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.372+0000 7f3027409640 1 --2- 172.21.15.114:0/2106708598 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3020105b20 0x7f302019d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:30.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.372+0000 7f3027409640 1 --2- 172.21.15.114:0/2106708598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30201031f0 0x7f3020196300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:30.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.372+0000 7f3027409640 1 --2- 172.21.15.114:0/2106708598 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30201008c0 0x7f3020195dc0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:30.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.372+0000 7f3027409640 1 -- 172.21.15.114:0/2106708598 >> 172.21.15.114:0/2106708598 conn(0x7f30200fa440 msgr2=0x7f3020101240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:30.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.372+0000 7f3027409640 1 -- 172.21.15.114:0/2106708598 shutdown_connections 2024-09-13T18:20:30.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.372+0000 7f3027409640 1 -- 172.21.15.114:0/2106708598 wait complete. 2024-09-13T18:20:30.384 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:30 smithi114 ceph-mon[30897]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:20:30.384 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:30 smithi114 ceph-mon[30897]: mds.? [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] up:active 2024-09-13T18:20:30.384 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:30 smithi114 ceph-mon[30897]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-13T18:20:30.385 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:30 smithi114 ceph-mon[30897]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-13T18:20:30.385 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:30 smithi114 ceph-mon[30897]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:20:30.385 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:30 smithi114 ceph-mon[30897]: pgmap v293: 129 pgs: 1 activating, 128 active+clean; 590 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 4.1 KiB/s wr, 12 op/s; 2 B/s, 0 objects/s recovering 2024-09-13T18:20:30.385 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:30 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/1587228232' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:20:30.386 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-09-13T18:20:30.386 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:20:30.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.483+0000 7f6819a19640 1 -- 172.21.15.114:0/2346135736 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6814100860 msgr2=0x7f6814102c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:30.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.483+0000 7f6819a19640 1 --2- 172.21.15.114:0/2346135736 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6814100860 0x7f6814102c50 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6808009900 tx=0x7f680802a410 comp rx=0 tx=0).stop 2024-09-13T18:20:30.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.483+0000 7f6819a19640 1 -- 172.21.15.114:0/2346135736 shutdown_connections 2024-09-13T18:20:30.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.483+0000 7f6819a19640 1 --2- 172.21.15.114:0/2346135736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6814105ac0 0x7f6814107ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:30.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.483+0000 7f6819a19640 1 --2- 172.21.15.114:0/2346135736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6814103190 0x7f6814105580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:30.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.483+0000 7f6819a19640 1 --2- 172.21.15.114:0/2346135736 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6814100860 0x7f6814102c50 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:30.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.483+0000 7f6819a19640 1 -- 172.21.15.114:0/2346135736 >> 172.21.15.114:0/2346135736 conn(0x7f68140fa440 msgr2=0x7f68140fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:30.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.483+0000 7f6819a19640 1 -- 172.21.15.114:0/2346135736 shutdown_connections 2024-09-13T18:20:30.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.483+0000 7f6819a19640 1 -- 172.21.15.114:0/2346135736 wait complete. 2024-09-13T18:20:30.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.484+0000 7f6819a19640 1 Processor -- start 2024-09-13T18:20:30.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.484+0000 7f6819a19640 1 -- start start 2024-09-13T18:20:30.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.484+0000 7f6819a19640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6814100860 0x7f68141001d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:30.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.484+0000 7f6819a19640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6814103190 0x7f68140fe810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:30.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.484+0000 7f6819a19640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6814105ac0 0x7f68140fed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:30.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.484+0000 7f6819a19640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68140ff4b0 con 0x7f6814100860 2024-09-13T18:20:30.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.484+0000 7f6819a19640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68140ff620 con 0x7f6814105ac0 2024-09-13T18:20:30.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.484+0000 7f6819a19640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f681419fe20 con 0x7f6814103190 2024-09-13T18:20:30.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.484+0000 7f6812ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6814100860 0x7f68141001d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:30.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.484+0000 7f6812ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6814100860 0x7f68141001d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60828/0 (socket says 172.21.15.114:60828) 2024-09-13T18:20:30.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.484+0000 7f6812ffd640 1 -- 172.21.15.114:0/2437580573 learned_addr learned my addr 172.21.15.114:0/2437580573 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:20:30.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.485+0000 7f68137fe640 1 --2- 172.21.15.114:0/2437580573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6814105ac0 0x7f68140fed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:30.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.485+0000 7f68127fc640 1 --2- 172.21.15.114:0/2437580573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6814103190 0x7f68140fe810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:30.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.485+0000 7f6812ffd640 1 -- 172.21.15.114:0/2437580573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6814103190 msgr2=0x7f68140fe810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:30.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.485+0000 7f6812ffd640 1 --2- 172.21.15.114:0/2437580573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6814103190 0x7f68140fe810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:30.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.485+0000 7f6812ffd640 1 -- 172.21.15.114:0/2437580573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6814105ac0 msgr2=0x7f68140fed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:30.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.485+0000 7f6812ffd640 1 --2- 172.21.15.114:0/2437580573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6814105ac0 0x7f68140fed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:30.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.485+0000 7f6812ffd640 1 -- 172.21.15.114:0/2437580573 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6808009560 con 0x7f6814100860 2024-09-13T18:20:30.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.485+0000 7f6812ffd640 1 --2- 172.21.15.114:0/2437580573 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6814100860 0x7f68141001d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6808009a20 tx=0x7f6808004380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:30.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.485+0000 7f6818a17640 1 -- 172.21.15.114:0/2437580573 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6808031d70 con 0x7f6814100860 2024-09-13T18:20:30.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.485+0000 7f6818a17640 1 -- 172.21.15.114:0/2437580573 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6808004b50 con 0x7f6814100860 2024-09-13T18:20:30.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.485+0000 7f6819a19640 1 -- 172.21.15.114:0/2437580573 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68141a0070 con 0x7f6814100860 2024-09-13T18:20:30.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.485+0000 7f6818a17640 1 -- 172.21.15.114:0/2437580573 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6808003bf0 con 0x7f6814100860 2024-09-13T18:20:30.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.485+0000 7f6819a19640 1 -- 172.21.15.114:0/2437580573 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68141a0520 con 0x7f6814100860 2024-09-13T18:20:30.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.486+0000 7f6819a19640 1 -- 172.21.15.114:0/2437580573 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68140fbf30 con 0x7f6814100860 2024-09-13T18:20:30.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.487+0000 7f6818a17640 1 -- 172.21.15.114:0/2437580573 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f6808003d50 con 0x7f6814100860 2024-09-13T18:20:30.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.488+0000 7f6818a17640 1 --2- 172.21.15.114:0/2437580573 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f67fc0a1120 0x7f67fc0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:30.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.488+0000 7f6818a17640 1 -- 172.21.15.114:0/2437580573 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f68080ee620 con 0x7f6814100860 2024-09-13T18:20:30.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.488+0000 7f68127fc640 1 --2- 172.21.15.114:0/2437580573 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f67fc0a1120 0x7f67fc0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:30.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.488+0000 7f68127fc640 1 --2- 172.21.15.114:0/2437580573 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f67fc0a1120 0x7f67fc0a35e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f68140fff40 tx=0x7f6804008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:30.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.491+0000 7f6818a17640 1 -- 172.21.15.114:0/2437580573 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f68080b7940 con 0x7f6814100860 2024-09-13T18:20:30.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:30 smithi134 ceph-mon[34283]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:20:30.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:30 smithi134 ceph-mon[34283]: mds.? [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] up:active 2024-09-13T18:20:30.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:30 smithi134 ceph-mon[34283]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-13T18:20:30.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:30 smithi134 ceph-mon[34283]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-13T18:20:30.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:30 smithi134 ceph-mon[34283]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:20:30.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:30 smithi134 ceph-mon[34283]: pgmap v293: 129 pgs: 1 activating, 128 active+clean; 590 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 4.1 KiB/s wr, 12 op/s; 2 B/s, 0 objects/s recovering 2024-09-13T18:20:30.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:30 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/1587228232' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:20:30.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.651+0000 7f6819a19640 1 -- 172.21.15.114:0/2437580573 --> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f68140608f0 con 0x7f67fc0a1120 2024-09-13T18:20:30.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.663+0000 7f6818a17640 1 -- 172.21.15.114:0/2437580573 <== mgr.14148 v2:172.21.15.114:6800/3396185541 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+424028 (secure 0 0 0) 0x7f68140608f0 con 0x7f67fc0a1120 2024-09-13T18:20:30.671 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:20:30.673 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-13T18:20:30.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.666+0000 7f6819a19640 1 -- 172.21.15.114:0/2437580573 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f67fc0a1120 msgr2=0x7f67fc0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:30.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.666+0000 7f6819a19640 1 --2- 172.21.15.114:0/2437580573 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f67fc0a1120 0x7f67fc0a35e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f68140fff40 tx=0x7f6804008040 comp rx=0 tx=0).stop 2024-09-13T18:20:30.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.666+0000 7f6819a19640 1 -- 172.21.15.114:0/2437580573 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6814100860 msgr2=0x7f68141001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:30.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.666+0000 7f6819a19640 1 --2- 172.21.15.114:0/2437580573 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6814100860 0x7f68141001d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6808009a20 tx=0x7f6808004380 comp rx=0 tx=0).stop 2024-09-13T18:20:30.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.666+0000 7f6819a19640 1 -- 172.21.15.114:0/2437580573 shutdown_connections 2024-09-13T18:20:30.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.666+0000 7f6819a19640 1 --2- 172.21.15.114:0/2437580573 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f67fc0a1120 0x7f67fc0a35e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:30.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.666+0000 7f6819a19640 1 --2- 172.21.15.114:0/2437580573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6814105ac0 0x7f68140fed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:30.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.666+0000 7f6819a19640 1 --2- 172.21.15.114:0/2437580573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6814103190 0x7f68140fe810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:30.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.666+0000 7f6819a19640 1 --2- 172.21.15.114:0/2437580573 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6814100860 0x7f68141001d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:30.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.666+0000 7f6819a19640 1 -- 172.21.15.114:0/2437580573 >> 172.21.15.114:0/2437580573 conn(0x7f68140fa440 msgr2=0x7f6814101320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:30.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.667+0000 7f6819a19640 1 -- 172.21.15.114:0/2437580573 shutdown_connections 2024-09-13T18:20:30.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.667+0000 7f6819a19640 1 -- 172.21.15.114:0/2437580573 wait complete. 2024-09-13T18:20:30.681 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-09-13T18:20:29.908599+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":603786,"num_objects":63,"num_object_clones":0,"num_object_copies":189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":145,"num_read_kb":136,"num_write":320,"num_write_kb":2989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":257,"ondisk_log_size":257,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":421,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360636,"kb_used_data":11828,"kb_used_omap":18,"kb_used_meta":348653,"kb_avail":1124581188,"statfs":{"total":1151940427776,"available":1151571136512,"internally_reserved":0,"allocated":12111872,"data_stored":7645949,"data_compressed":288644,"data_compressed_allocated":389120,"data_compressed_original":2277376,"omap_allocated":19129,"internal_metadata":357020999},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8825,"num_objects":39,"num_object_clones":0,"num_object_copies":117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":9,"num_read_kb":17,"num_write":91,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002330"},"pg_stats":[{"pgid":"2.1f","version":"91'13","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.256848+0000","last_change":"2024-09-13T18:20:01.577029+0000","last_active":"2024-09-13T18:20:28.256848+0000","last_peered":"2024-09-13T18:20:28.256848+0000","last_clean":"2024-09-13T18:20:28.256848+0000","last_became_active":"2024-09-13T18:20:01.576710+0000","last_became_peered":"2024-09-13T18:20:01.576710+0000","last_unstale":"2024-09-13T18:20:28.256848+0000","last_undegraded":"2024-09-13T18:20:28.256848+0000","last_fullsized":"2024-09-13T18:20:28.256848+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:25:11.977436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.190449+0000","last_change":"2024-09-13T18:20:01.922955+0000","last_active":"2024-09-13T18:20:26.190449+0000","last_peered":"2024-09-13T18:20:26.190449+0000","last_clean":"2024-09-13T18:20:26.190449+0000","last_became_active":"2024-09-13T18:20:01.922704+0000","last_became_peered":"2024-09-13T18:20:01.922704+0000","last_unstale":"2024-09-13T18:20:26.190449+0000","last_undegraded":"2024-09-13T18:20:26.190449+0000","last_fullsized":"2024-09-13T18:20:26.190449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:44:20.672784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.132781+0000","last_change":"2024-09-13T18:20:06.130388+0000","last_active":"2024-09-13T18:20:06.132781+0000","last_peered":"2024-09-13T18:20:06.132781+0000","last_clean":"2024-09-13T18:20:06.132781+0000","last_became_active":"2024-09-13T18:20:06.129895+0000","last_became_peered":"2024-09-13T18:20:06.129895+0000","last_unstale":"2024-09-13T18:20:06.132781+0000","last_undegraded":"2024-09-13T18:20:06.132781+0000","last_fullsized":"2024-09-13T18:20:06.132781+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:44:33.537849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.197866+0000","last_change":"2024-09-13T18:20:01.578921+0000","last_active":"2024-09-13T18:20:26.197866+0000","last_peered":"2024-09-13T18:20:26.197866+0000","last_clean":"2024-09-13T18:20:26.197866+0000","last_became_active":"2024-09-13T18:20:01.578627+0000","last_became_peered":"2024-09-13T18:20:01.578627+0000","last_unstale":"2024-09-13T18:20:26.197866+0000","last_undegraded":"2024-09-13T18:20:26.197866+0000","last_fullsized":"2024-09-13T18:20:26.197866+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:43:32.919156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.190717+0000","last_change":"2024-09-13T18:20:01.922901+0000","last_active":"2024-09-13T18:20:26.190717+0000","last_peered":"2024-09-13T18:20:26.190717+0000","last_clean":"2024-09-13T18:20:26.190717+0000","last_became_active":"2024-09-13T18:20:01.922604+0000","last_became_peered":"2024-09-13T18:20:01.922604+0000","last_unstale":"2024-09-13T18:20:26.190717+0000","last_undegraded":"2024-09-13T18:20:26.190717+0000","last_fullsized":"2024-09-13T18:20:26.190717+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:27:30.650010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.744777+0000","last_change":"2024-09-13T18:20:06.135406+0000","last_active":"2024-09-13T18:20:06.744777+0000","last_peered":"2024-09-13T18:20:06.744777+0000","last_clean":"2024-09-13T18:20:06.744777+0000","last_became_active":"2024-09-13T18:20:06.134558+0000","last_became_peered":"2024-09-13T18:20:06.134558+0000","last_unstale":"2024-09-13T18:20:06.744777+0000","last_undegraded":"2024-09-13T18:20:06.744777+0000","last_fullsized":"2024-09-13T18:20:06.744777+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:10.901966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"84'2","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.122416+0000","last_change":"2024-09-13T18:20:01.580040+0000","last_active":"2024-09-13T18:20:06.122416+0000","last_peered":"2024-09-13T18:20:06.122416+0000","last_clean":"2024-09-13T18:20:06.122416+0000","last_became_active":"2024-09-13T18:20:01.579715+0000","last_became_peered":"2024-09-13T18:20:01.579715+0000","last_unstale":"2024-09-13T18:20:06.122416+0000","last_undegraded":"2024-09-13T18:20:06.122416+0000","last_fullsized":"2024-09-13T18:20:06.122416+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:57:03.588169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.184089+0000","last_change":"2024-09-13T18:20:01.928314+0000","last_active":"2024-09-13T18:20:26.184089+0000","last_peered":"2024-09-13T18:20:26.184089+0000","last_clean":"2024-09-13T18:20:26.184089+0000","last_became_active":"2024-09-13T18:20:01.925356+0000","last_became_peered":"2024-09-13T18:20:01.925356+0000","last_unstale":"2024-09-13T18:20:26.184089+0000","last_undegraded":"2024-09-13T18:20:26.184089+0000","last_fullsized":"2024-09-13T18:20:26.184089+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:43.906842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.744757+0000","last_change":"2024-09-13T18:20:06.135886+0000","last_active":"2024-09-13T18:20:06.744757+0000","last_peered":"2024-09-13T18:20:06.744757+0000","last_clean":"2024-09-13T18:20:06.744757+0000","last_became_active":"2024-09-13T18:20:06.134587+0000","last_became_peered":"2024-09-13T18:20:06.134587+0000","last_unstale":"2024-09-13T18:20:06.744757+0000","last_undegraded":"2024-09-13T18:20:06.744757+0000","last_fullsized":"2024-09-13T18:20:06.744757+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:44:42.434729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.122797+0000","last_change":"2024-09-13T18:20:01.576908+0000","last_active":"2024-09-13T18:20:06.122797+0000","last_peered":"2024-09-13T18:20:06.122797+0000","last_clean":"2024-09-13T18:20:06.122797+0000","last_became_active":"2024-09-13T18:20:01.576478+0000","last_became_peered":"2024-09-13T18:20:01.576478+0000","last_unstale":"2024-09-13T18:20:06.122797+0000","last_undegraded":"2024-09-13T18:20:06.122797+0000","last_fullsized":"2024-09-13T18:20:06.122797+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:34:52.126901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.126411+0000","last_change":"2024-09-13T18:20:01.921960+0000","last_active":"2024-09-13T18:20:06.126411+0000","last_peered":"2024-09-13T18:20:06.126411+0000","last_clean":"2024-09-13T18:20:06.126411+0000","last_became_active":"2024-09-13T18:20:01.921652+0000","last_became_peered":"2024-09-13T18:20:01.921652+0000","last_unstale":"2024-09-13T18:20:06.126411+0000","last_undegraded":"2024-09-13T18:20:06.126411+0000","last_fullsized":"2024-09-13T18:20:06.126411+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:00:31.277613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.744407+0000","last_change":"2024-09-13T18:20:06.138983+0000","last_active":"2024-09-13T18:20:06.744407+0000","last_peered":"2024-09-13T18:20:06.744407+0000","last_clean":"2024-09-13T18:20:06.744407+0000","last_became_active":"2024-09-13T18:20:06.138708+0000","last_became_peered":"2024-09-13T18:20:06.138708+0000","last_unstale":"2024-09-13T18:20:06.744407+0000","last_undegraded":"2024-09-13T18:20:06.744407+0000","last_fullsized":"2024-09-13T18:20:06.744407+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:45:21.978217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.177004+0000","last_change":"2024-09-13T18:20:01.929817+0000","last_active":"2024-09-13T18:20:06.177004+0000","last_peered":"2024-09-13T18:20:06.177004+0000","last_clean":"2024-09-13T18:20:06.177004+0000","last_became_active":"2024-09-13T18:20:01.928907+0000","last_became_peered":"2024-09-13T18:20:01.928907+0000","last_unstale":"2024-09-13T18:20:06.177004+0000","last_undegraded":"2024-09-13T18:20:06.177004+0000","last_fullsized":"2024-09-13T18:20:06.177004+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:56:25.847230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:25.180585+0000","last_change":"2024-09-13T18:20:01.578968+0000","last_active":"2024-09-13T18:20:25.180585+0000","last_peered":"2024-09-13T18:20:25.180585+0000","last_clean":"2024-09-13T18:20:25.180585+0000","last_became_active":"2024-09-13T18:20:01.578623+0000","last_became_peered":"2024-09-13T18:20:01.578623+0000","last_unstale":"2024-09-13T18:20:25.180585+0000","last_undegraded":"2024-09-13T18:20:25.180585+0000","last_fullsized":"2024-09-13T18:20:25.180585+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:03:30.333107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.744596+0000","last_change":"2024-09-13T18:20:06.135977+0000","last_active":"2024-09-13T18:20:06.744596+0000","last_peered":"2024-09-13T18:20:06.744596+0000","last_clean":"2024-09-13T18:20:06.744596+0000","last_became_active":"2024-09-13T18:20:06.134844+0000","last_became_peered":"2024-09-13T18:20:06.134844+0000","last_unstale":"2024-09-13T18:20:06.744596+0000","last_undegraded":"2024-09-13T18:20:06.744596+0000","last_fullsized":"2024-09-13T18:20:06.744596+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:58:36.139240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.190165+0000","last_change":"2024-09-13T18:20:01.916675+0000","last_active":"2024-09-13T18:20:26.190165+0000","last_peered":"2024-09-13T18:20:26.190165+0000","last_clean":"2024-09-13T18:20:26.190165+0000","last_became_active":"2024-09-13T18:20:01.912902+0000","last_became_peered":"2024-09-13T18:20:01.912902+0000","last_unstale":"2024-09-13T18:20:26.190165+0000","last_undegraded":"2024-09-13T18:20:26.190165+0000","last_fullsized":"2024-09-13T18:20:26.190165+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:24:54.271569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:27.301309+0000","last_change":"2024-09-13T18:20:27.300845+0000","last_active":"2024-09-13T18:20:27.301309+0000","last_peered":"2024-09-13T18:20:27.301309+0000","last_clean":"2024-09-13T18:20:27.301309+0000","last_became_active":"2024-09-13T18:20:27.300417+0000","last_became_peered":"2024-09-13T18:20:27.300417+0000","last_unstale":"2024-09-13T18:20:27.301309+0000","last_undegraded":"2024-09-13T18:20:27.301309+0000","last_fullsized":"2024-09-13T18:20:27.301309+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:44:27.507140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.745268+0000","last_change":"2024-09-13T18:20:06.131539+0000","last_active":"2024-09-13T18:20:06.745268+0000","last_peered":"2024-09-13T18:20:06.745268+0000","last_clean":"2024-09-13T18:20:06.745268+0000","last_became_active":"2024-09-13T18:20:06.131148+0000","last_became_peered":"2024-09-13T18:20:06.131148+0000","last_unstale":"2024-09-13T18:20:06.745268+0000","last_undegraded":"2024-09-13T18:20:06.745268+0000","last_fullsized":"2024-09-13T18:20:06.745268+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:32:49.759831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183043+0000","last_change":"2024-09-13T18:20:01.913634+0000","last_active":"2024-09-13T18:20:26.183043+0000","last_peered":"2024-09-13T18:20:26.183043+0000","last_clean":"2024-09-13T18:20:26.183043+0000","last_became_active":"2024-09-13T18:20:01.913128+0000","last_became_peered":"2024-09-13T18:20:01.913128+0000","last_unstale":"2024-09-13T18:20:26.183043+0000","last_undegraded":"2024-09-13T18:20:26.183043+0000","last_fullsized":"2024-09-13T18:20:26.183043+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:37:15.251756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.192501+0000","last_change":"2024-09-13T18:20:01.579385+0000","last_active":"2024-09-13T18:20:26.192501+0000","last_peered":"2024-09-13T18:20:26.192501+0000","last_clean":"2024-09-13T18:20:26.192501+0000","last_became_active":"2024-09-13T18:20:01.579102+0000","last_became_peered":"2024-09-13T18:20:01.579102+0000","last_unstale":"2024-09-13T18:20:26.192501+0000","last_undegraded":"2024-09-13T18:20:26.192501+0000","last_fullsized":"2024-09-13T18:20:26.192501+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:22:00.432052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183422+0000","last_change":"2024-09-13T18:20:06.138110+0000","last_active":"2024-09-13T18:20:26.183422+0000","last_peered":"2024-09-13T18:20:26.183422+0000","last_clean":"2024-09-13T18:20:26.183422+0000","last_became_active":"2024-09-13T18:20:06.137763+0000","last_became_peered":"2024-09-13T18:20:06.137763+0000","last_unstale":"2024-09-13T18:20:26.183422+0000","last_undegraded":"2024-09-13T18:20:26.183422+0000","last_fullsized":"2024-09-13T18:20:26.183422+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:16:15.956216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.122199+0000","last_change":"2024-09-13T18:20:01.918318+0000","last_active":"2024-09-13T18:20:06.122199+0000","last_peered":"2024-09-13T18:20:06.122199+0000","last_clean":"2024-09-13T18:20:06.122199+0000","last_became_active":"2024-09-13T18:20:01.917977+0000","last_became_peered":"2024-09-13T18:20:01.917977+0000","last_unstale":"2024-09-13T18:20:06.122199+0000","last_undegraded":"2024-09-13T18:20:06.122199+0000","last_fullsized":"2024-09-13T18:20:06.122199+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:23.368866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.184253+0000","last_change":"2024-09-13T18:20:01.572009+0000","last_active":"2024-09-13T18:20:26.184253+0000","last_peered":"2024-09-13T18:20:26.184253+0000","last_clean":"2024-09-13T18:20:26.184253+0000","last_became_active":"2024-09-13T18:20:01.571313+0000","last_became_peered":"2024-09-13T18:20:01.571313+0000","last_unstale":"2024-09-13T18:20:26.184253+0000","last_undegraded":"2024-09-13T18:20:26.184253+0000","last_fullsized":"2024-09-13T18:20:26.184253+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:32:59.340476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.744412+0000","last_change":"2024-09-13T18:20:06.132212+0000","last_active":"2024-09-13T18:20:06.744412+0000","last_peered":"2024-09-13T18:20:06.744412+0000","last_clean":"2024-09-13T18:20:06.744412+0000","last_became_active":"2024-09-13T18:20:06.131809+0000","last_became_peered":"2024-09-13T18:20:06.131809+0000","last_unstale":"2024-09-13T18:20:06.744412+0000","last_undegraded":"2024-09-13T18:20:06.744412+0000","last_fullsized":"2024-09-13T18:20:06.744412+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:18:11.667551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"activating","last_fresh":"2024-09-13T18:20:27.181672+0000","last_change":"2024-09-13T18:20:27.181672+0000","last_active":"2024-09-13T18:20:26.189486+0000","last_peered":"2024-09-13T18:20:06.959341+0000","last_clean":"2024-09-13T18:20:06.959341+0000","last_became_active":"2024-09-13T18:20:01.922815+0000","last_became_peered":"2024-09-13T18:20:01.922815+0000","last_unstale":"2024-09-13T18:20:27.181672+0000","last_undegraded":"2024-09-13T18:20:27.181672+0000","last_fullsized":"2024-09-13T18:20:27.181672+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:59:57.606105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":["0","3","4","7"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:25.181259+0000","last_change":"2024-09-13T18:20:01.572025+0000","last_active":"2024-09-13T18:20:25.181259+0000","last_peered":"2024-09-13T18:20:25.181259+0000","last_clean":"2024-09-13T18:20:25.181259+0000","last_became_active":"2024-09-13T18:20:01.571321+0000","last_became_peered":"2024-09-13T18:20:01.571321+0000","last_unstale":"2024-09-13T18:20:25.181259+0000","last_undegraded":"2024-09-13T18:20:25.181259+0000","last_fullsized":"2024-09-13T18:20:25.181259+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:09.374364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.132797+0000","last_change":"2024-09-13T18:20:06.130687+0000","last_active":"2024-09-13T18:20:06.132797+0000","last_peered":"2024-09-13T18:20:06.132797+0000","last_clean":"2024-09-13T18:20:06.132797+0000","last_became_active":"2024-09-13T18:20:06.129839+0000","last_became_peered":"2024-09-13T18:20:06.129839+0000","last_unstale":"2024-09-13T18:20:06.132797+0000","last_undegraded":"2024-09-13T18:20:06.132797+0000","last_fullsized":"2024-09-13T18:20:06.132797+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:12:52.334945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.190458+0000","last_change":"2024-09-13T18:20:01.917244+0000","last_active":"2024-09-13T18:20:26.190458+0000","last_peered":"2024-09-13T18:20:26.190458+0000","last_clean":"2024-09-13T18:20:26.190458+0000","last_became_active":"2024-09-13T18:20:01.916948+0000","last_became_peered":"2024-09-13T18:20:01.916948+0000","last_unstale":"2024-09-13T18:20:26.190458+0000","last_undegraded":"2024-09-13T18:20:26.190458+0000","last_fullsized":"2024-09-13T18:20:26.190458+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:17:11.967096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'7","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.206617+0000","last_change":"2024-09-13T18:20:01.569123+0000","last_active":"2024-09-13T18:20:28.206617+0000","last_peered":"2024-09-13T18:20:28.206617+0000","last_clean":"2024-09-13T18:20:28.206617+0000","last_became_active":"2024-09-13T18:20:01.568720+0000","last_became_peered":"2024-09-13T18:20:01.568720+0000","last_unstale":"2024-09-13T18:20:28.206617+0000","last_undegraded":"2024-09-13T18:20:28.206617+0000","last_fullsized":"2024-09-13T18:20:28.206617+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:04:29.561013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.190854+0000","last_change":"2024-09-13T18:20:06.141771+0000","last_active":"2024-09-13T18:20:26.190854+0000","last_peered":"2024-09-13T18:20:26.190854+0000","last_clean":"2024-09-13T18:20:26.190854+0000","last_became_active":"2024-09-13T18:20:06.141362+0000","last_became_peered":"2024-09-13T18:20:06.141362+0000","last_unstale":"2024-09-13T18:20:26.190854+0000","last_undegraded":"2024-09-13T18:20:26.190854+0000","last_fullsized":"2024-09-13T18:20:26.190854+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:09:58.597605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.179047+0000","last_change":"2024-09-13T18:20:01.911048+0000","last_active":"2024-09-13T18:20:26.179047+0000","last_peered":"2024-09-13T18:20:26.179047+0000","last_clean":"2024-09-13T18:20:26.179047+0000","last_became_active":"2024-09-13T18:20:01.910735+0000","last_became_peered":"2024-09-13T18:20:01.910735+0000","last_unstale":"2024-09-13T18:20:26.179047+0000","last_undegraded":"2024-09-13T18:20:26.179047+0000","last_fullsized":"2024-09-13T18:20:26.179047+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:07:35.447676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.745685+0000","last_change":"2024-09-13T18:20:01.571627+0000","last_active":"2024-09-13T18:20:06.745685+0000","last_peered":"2024-09-13T18:20:06.745685+0000","last_clean":"2024-09-13T18:20:06.745685+0000","last_became_active":"2024-09-13T18:20:01.571250+0000","last_became_peered":"2024-09-13T18:20:01.571250+0000","last_unstale":"2024-09-13T18:20:06.745685+0000","last_undegraded":"2024-09-13T18:20:06.745685+0000","last_fullsized":"2024-09-13T18:20:06.745685+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:34:03.479324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.182932+0000","last_change":"2024-09-13T18:20:06.132974+0000","last_active":"2024-09-13T18:20:26.182932+0000","last_peered":"2024-09-13T18:20:26.182932+0000","last_clean":"2024-09-13T18:20:26.182932+0000","last_became_active":"2024-09-13T18:20:06.131668+0000","last_became_peered":"2024-09-13T18:20:06.131668+0000","last_unstale":"2024-09-13T18:20:26.182932+0000","last_undegraded":"2024-09-13T18:20:26.182932+0000","last_fullsized":"2024-09-13T18:20:26.182932+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:36:30.329761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.197759+0000","last_change":"2024-09-13T18:20:01.904670+0000","last_active":"2024-09-13T18:20:26.197759+0000","last_peered":"2024-09-13T18:20:26.197759+0000","last_clean":"2024-09-13T18:20:26.197759+0000","last_became_active":"2024-09-13T18:20:01.904316+0000","last_became_peered":"2024-09-13T18:20:01.904316+0000","last_unstale":"2024-09-13T18:20:26.197759+0000","last_undegraded":"2024-09-13T18:20:26.197759+0000","last_fullsized":"2024-09-13T18:20:26.197759+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:08:17.687179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"91'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:27.179594+0000","last_change":"2024-09-13T18:20:01.564750+0000","last_active":"2024-09-13T18:20:27.179594+0000","last_peered":"2024-09-13T18:20:27.179594+0000","last_clean":"2024-09-13T18:20:27.179594+0000","last_became_active":"2024-09-13T18:20:01.564096+0000","last_became_peered":"2024-09-13T18:20:01.564096+0000","last_unstale":"2024-09-13T18:20:27.179594+0000","last_undegraded":"2024-09-13T18:20:27.179594+0000","last_fullsized":"2024-09-13T18:20:27.179594+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:42:19.588824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.134398+0000","last_change":"2024-09-13T18:20:06.133386+0000","last_active":"2024-09-13T18:20:06.134398+0000","last_peered":"2024-09-13T18:20:06.134398+0000","last_clean":"2024-09-13T18:20:06.134398+0000","last_became_active":"2024-09-13T18:20:06.133004+0000","last_became_peered":"2024-09-13T18:20:06.133004+0000","last_unstale":"2024-09-13T18:20:06.134398+0000","last_undegraded":"2024-09-13T18:20:06.134398+0000","last_fullsized":"2024-09-13T18:20:06.134398+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:54:46.440410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.197632+0000","last_change":"2024-09-13T18:20:01.912823+0000","last_active":"2024-09-13T18:20:26.197632+0000","last_peered":"2024-09-13T18:20:26.197632+0000","last_clean":"2024-09-13T18:20:26.197632+0000","last_became_active":"2024-09-13T18:20:01.912123+0000","last_became_peered":"2024-09-13T18:20:01.912123+0000","last_unstale":"2024-09-13T18:20:26.197632+0000","last_undegraded":"2024-09-13T18:20:26.197632+0000","last_fullsized":"2024-09-13T18:20:26.197632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:32:33.544816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.122475+0000","last_change":"2024-09-13T18:20:01.577095+0000","last_active":"2024-09-13T18:20:06.122475+0000","last_peered":"2024-09-13T18:20:06.122475+0000","last_clean":"2024-09-13T18:20:06.122475+0000","last_became_active":"2024-09-13T18:20:01.576765+0000","last_became_peered":"2024-09-13T18:20:01.576765+0000","last_unstale":"2024-09-13T18:20:06.122475+0000","last_undegraded":"2024-09-13T18:20:06.122475+0000","last_fullsized":"2024-09-13T18:20:06.122475+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:44:08.750441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.178862+0000","last_change":"2024-09-13T18:20:06.131650+0000","last_active":"2024-09-13T18:20:26.178862+0000","last_peered":"2024-09-13T18:20:26.178862+0000","last_clean":"2024-09-13T18:20:26.178862+0000","last_became_active":"2024-09-13T18:20:06.130902+0000","last_became_peered":"2024-09-13T18:20:06.130902+0000","last_unstale":"2024-09-13T18:20:26.178862+0000","last_undegraded":"2024-09-13T18:20:26.178862+0000","last_fullsized":"2024-09-13T18:20:26.178862+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.366694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.122314+0000","last_change":"2024-09-13T18:20:01.918221+0000","last_active":"2024-09-13T18:20:06.122314+0000","last_peered":"2024-09-13T18:20:06.122314+0000","last_clean":"2024-09-13T18:20:06.122314+0000","last_became_active":"2024-09-13T18:20:01.917134+0000","last_became_peered":"2024-09-13T18:20:01.917134+0000","last_unstale":"2024-09-13T18:20:06.122314+0000","last_undegraded":"2024-09-13T18:20:06.122314+0000","last_fullsized":"2024-09-13T18:20:06.122314+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:19:06.269454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.249346+0000","last_change":"2024-09-13T18:20:01.572258+0000","last_active":"2024-09-13T18:20:28.249346+0000","last_peered":"2024-09-13T18:20:28.249346+0000","last_clean":"2024-09-13T18:20:28.249346+0000","last_became_active":"2024-09-13T18:20:01.571968+0000","last_became_peered":"2024-09-13T18:20:01.571968+0000","last_unstale":"2024-09-13T18:20:28.249346+0000","last_undegraded":"2024-09-13T18:20:28.249346+0000","last_fullsized":"2024-09-13T18:20:28.249346+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:40:59.143123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.179018+0000","last_change":"2024-09-13T18:20:06.131714+0000","last_active":"2024-09-13T18:20:26.179018+0000","last_peered":"2024-09-13T18:20:26.179018+0000","last_clean":"2024-09-13T18:20:26.179018+0000","last_became_active":"2024-09-13T18:20:06.131244+0000","last_became_peered":"2024-09-13T18:20:06.131244+0000","last_unstale":"2024-09-13T18:20:26.179018+0000","last_undegraded":"2024-09-13T18:20:26.179018+0000","last_fullsized":"2024-09-13T18:20:26.179018+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:37.946002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.178584+0000","last_change":"2024-09-13T18:20:01.917351+0000","last_active":"2024-09-13T18:20:26.178584+0000","last_peered":"2024-09-13T18:20:26.178584+0000","last_clean":"2024-09-13T18:20:26.178584+0000","last_became_active":"2024-09-13T18:20:01.917067+0000","last_became_peered":"2024-09-13T18:20:01.917067+0000","last_unstale":"2024-09-13T18:20:26.178584+0000","last_undegraded":"2024-09-13T18:20:26.178584+0000","last_fullsized":"2024-09-13T18:20:26.178584+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:35:37.723132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'1","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.462263+0000","last_change":"2024-09-13T18:20:01.573515+0000","last_active":"2024-09-13T18:20:06.462263+0000","last_peered":"2024-09-13T18:20:06.462263+0000","last_clean":"2024-09-13T18:20:06.462263+0000","last_became_active":"2024-09-13T18:20:01.573211+0000","last_became_peered":"2024-09-13T18:20:01.573211+0000","last_unstale":"2024-09-13T18:20:06.462263+0000","last_undegraded":"2024-09-13T18:20:06.462263+0000","last_fullsized":"2024-09-13T18:20:06.462263+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:05:30.912467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.134638+0000","last_change":"2024-09-13T18:20:06.133713+0000","last_active":"2024-09-13T18:20:06.134638+0000","last_peered":"2024-09-13T18:20:06.134638+0000","last_clean":"2024-09-13T18:20:06.134638+0000","last_became_active":"2024-09-13T18:20:06.133330+0000","last_became_peered":"2024-09-13T18:20:06.133330+0000","last_unstale":"2024-09-13T18:20:06.134638+0000","last_undegraded":"2024-09-13T18:20:06.134638+0000","last_fullsized":"2024-09-13T18:20:06.134638+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:48:07.133155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.192066+0000","last_change":"2024-09-13T18:20:01.921597+0000","last_active":"2024-09-13T18:20:26.192066+0000","last_peered":"2024-09-13T18:20:26.192066+0000","last_clean":"2024-09-13T18:20:26.192066+0000","last_became_active":"2024-09-13T18:20:01.920200+0000","last_became_peered":"2024-09-13T18:20:01.920200+0000","last_unstale":"2024-09-13T18:20:26.192066+0000","last_undegraded":"2024-09-13T18:20:26.192066+0000","last_fullsized":"2024-09-13T18:20:26.192066+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:22:40.517941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.186261+0000","last_change":"2024-09-13T18:20:01.571313+0000","last_active":"2024-09-13T18:20:26.186261+0000","last_peered":"2024-09-13T18:20:26.186261+0000","last_clean":"2024-09-13T18:20:26.186261+0000","last_became_active":"2024-09-13T18:20:01.570977+0000","last_became_peered":"2024-09-13T18:20:01.570977+0000","last_unstale":"2024-09-13T18:20:26.186261+0000","last_undegraded":"2024-09-13T18:20:26.186261+0000","last_fullsized":"2024-09-13T18:20:26.186261+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:06.870320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":341,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.122460+0000","last_change":"2024-09-13T18:18:51.237409+0000","last_active":"2024-09-13T18:20:06.122460+0000","last_peered":"2024-09-13T18:20:06.122460+0000","last_clean":"2024-09-13T18:20:06.122460+0000","last_became_active":"2024-09-13T18:18:51.236986+0000","last_became_peered":"2024-09-13T18:18:51.236986+0000","last_unstale":"2024-09-13T18:20:06.122460+0000","last_undegraded":"2024-09-13T18:20:06.122460+0000","last_fullsized":"2024-09-13T18:20:06.122460+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:15:40.757299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:15:40.757299+0000","last_clean_scrub_stamp":"2024-09-13T18:15:40.757299+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:58:29.679404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.177041+0000","last_change":"2024-09-13T18:20:06.130803+0000","last_active":"2024-09-13T18:20:06.177041+0000","last_peered":"2024-09-13T18:20:06.177041+0000","last_clean":"2024-09-13T18:20:06.177041+0000","last_became_active":"2024-09-13T18:20:06.130433+0000","last_became_peered":"2024-09-13T18:20:06.130433+0000","last_unstale":"2024-09-13T18:20:06.177041+0000","last_undegraded":"2024-09-13T18:20:06.177041+0000","last_fullsized":"2024-09-13T18:20:06.177041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:23:11.787084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.177067+0000","last_change":"2024-09-13T18:20:01.922702+0000","last_active":"2024-09-13T18:20:06.177067+0000","last_peered":"2024-09-13T18:20:06.177067+0000","last_clean":"2024-09-13T18:20:06.177067+0000","last_became_active":"2024-09-13T18:20:01.921109+0000","last_became_peered":"2024-09-13T18:20:01.921109+0000","last_unstale":"2024-09-13T18:20:06.177067+0000","last_undegraded":"2024-09-13T18:20:06.177067+0000","last_fullsized":"2024-09-13T18:20:06.177067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:03:34.618054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.123936+0000","last_change":"2024-09-13T18:20:01.571024+0000","last_active":"2024-09-13T18:20:06.123936+0000","last_peered":"2024-09-13T18:20:06.123936+0000","last_clean":"2024-09-13T18:20:06.123936+0000","last_became_active":"2024-09-13T18:20:01.570687+0000","last_became_peered":"2024-09-13T18:20:01.570687+0000","last_unstale":"2024-09-13T18:20:06.123936+0000","last_undegraded":"2024-09-13T18:20:06.123936+0000","last_fullsized":"2024-09-13T18:20:06.123936+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:10:25.126179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183401+0000","last_change":"2024-09-13T18:20:06.129025+0000","last_active":"2024-09-13T18:20:26.183401+0000","last_peered":"2024-09-13T18:20:26.183401+0000","last_clean":"2024-09-13T18:20:26.183401+0000","last_became_active":"2024-09-13T18:20:06.128618+0000","last_became_peered":"2024-09-13T18:20:06.128618+0000","last_unstale":"2024-09-13T18:20:26.183401+0000","last_undegraded":"2024-09-13T18:20:26.183401+0000","last_fullsized":"2024-09-13T18:20:26.183401+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:20:08.891259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.177033+0000","last_change":"2024-09-13T18:20:01.921697+0000","last_active":"2024-09-13T18:20:06.177033+0000","last_peered":"2024-09-13T18:20:06.177033+0000","last_clean":"2024-09-13T18:20:06.177033+0000","last_became_active":"2024-09-13T18:20:01.920174+0000","last_became_peered":"2024-09-13T18:20:01.920174+0000","last_unstale":"2024-09-13T18:20:06.177033+0000","last_undegraded":"2024-09-13T18:20:06.177033+0000","last_fullsized":"2024-09-13T18:20:06.177033+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:57:03.922725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.126610+0000","last_change":"2024-09-13T18:20:01.582102+0000","last_active":"2024-09-13T18:20:06.126610+0000","last_peered":"2024-09-13T18:20:06.126610+0000","last_clean":"2024-09-13T18:20:06.126610+0000","last_became_active":"2024-09-13T18:20:01.581841+0000","last_became_peered":"2024-09-13T18:20:01.581841+0000","last_unstale":"2024-09-13T18:20:06.126610+0000","last_undegraded":"2024-09-13T18:20:06.126610+0000","last_fullsized":"2024-09-13T18:20:06.126610+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:59:57.929703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.189924+0000","last_change":"2024-09-13T18:20:06.133126+0000","last_active":"2024-09-13T18:20:26.189924+0000","last_peered":"2024-09-13T18:20:26.189924+0000","last_clean":"2024-09-13T18:20:26.189924+0000","last_became_active":"2024-09-13T18:20:06.132747+0000","last_became_peered":"2024-09-13T18:20:06.132747+0000","last_unstale":"2024-09-13T18:20:26.189924+0000","last_undegraded":"2024-09-13T18:20:26.189924+0000","last_fullsized":"2024-09-13T18:20:26.189924+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:54:21.843090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.177082+0000","last_change":"2024-09-13T18:20:01.916306+0000","last_active":"2024-09-13T18:20:06.177082+0000","last_peered":"2024-09-13T18:20:06.177082+0000","last_clean":"2024-09-13T18:20:06.177082+0000","last_became_active":"2024-09-13T18:20:01.915739+0000","last_became_peered":"2024-09-13T18:20:01.915739+0000","last_unstale":"2024-09-13T18:20:06.177082+0000","last_undegraded":"2024-09-13T18:20:06.177082+0000","last_fullsized":"2024-09-13T18:20:06.177082+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:19:09.689909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"91'5","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.252883+0000","last_change":"2024-09-13T18:20:01.565821+0000","last_active":"2024-09-13T18:20:28.252883+0000","last_peered":"2024-09-13T18:20:28.252883+0000","last_clean":"2024-09-13T18:20:28.252883+0000","last_became_active":"2024-09-13T18:20:01.565467+0000","last_became_peered":"2024-09-13T18:20:01.565467+0000","last_unstale":"2024-09-13T18:20:28.252883+0000","last_undegraded":"2024-09-13T18:20:28.252883+0000","last_fullsized":"2024-09-13T18:20:28.252883+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:11.609400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.132093+0000","last_change":"2024-09-13T18:20:06.131348+0000","last_active":"2024-09-13T18:20:06.132093+0000","last_peered":"2024-09-13T18:20:06.132093+0000","last_clean":"2024-09-13T18:20:06.132093+0000","last_became_active":"2024-09-13T18:20:06.131019+0000","last_became_peered":"2024-09-13T18:20:06.131019+0000","last_unstale":"2024-09-13T18:20:06.132093+0000","last_undegraded":"2024-09-13T18:20:06.132093+0000","last_fullsized":"2024-09-13T18:20:06.132093+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:13:49.717503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183415+0000","last_change":"2024-09-13T18:20:01.922306+0000","last_active":"2024-09-13T18:20:26.183415+0000","last_peered":"2024-09-13T18:20:26.183415+0000","last_clean":"2024-09-13T18:20:26.183415+0000","last_became_active":"2024-09-13T18:20:01.921956+0000","last_became_peered":"2024-09-13T18:20:01.921956+0000","last_unstale":"2024-09-13T18:20:26.183415+0000","last_undegraded":"2024-09-13T18:20:26.183415+0000","last_fullsized":"2024-09-13T18:20:26.183415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:17:48.755318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:27.194592+0000","last_change":"2024-09-13T18:20:27.194085+0000","last_active":"2024-09-13T18:20:27.194592+0000","last_peered":"2024-09-13T18:20:27.194592+0000","last_clean":"2024-09-13T18:20:27.194592+0000","last_became_active":"2024-09-13T18:20:27.190136+0000","last_became_peered":"2024-09-13T18:20:27.190136+0000","last_unstale":"2024-09-13T18:20:27.194592+0000","last_undegraded":"2024-09-13T18:20:27.194592+0000","last_fullsized":"2024-09-13T18:20:27.194592+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:57:07.202901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.135975+0000","last_change":"2024-09-13T18:20:06.135498+0000","last_active":"2024-09-13T18:20:06.135975+0000","last_peered":"2024-09-13T18:20:06.135975+0000","last_clean":"2024-09-13T18:20:06.135975+0000","last_became_active":"2024-09-13T18:20:06.135095+0000","last_became_peered":"2024-09-13T18:20:06.135095+0000","last_unstale":"2024-09-13T18:20:06.135975+0000","last_undegraded":"2024-09-13T18:20:06.135975+0000","last_fullsized":"2024-09-13T18:20:06.135975+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:06:40.609718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:27.187296+0000","last_change":"2024-09-13T18:20:27.186825+0000","last_active":"2024-09-13T18:20:27.187296+0000","last_peered":"2024-09-13T18:20:27.187296+0000","last_clean":"2024-09-13T18:20:27.187296+0000","last_became_active":"2024-09-13T18:20:27.186523+0000","last_became_peered":"2024-09-13T18:20:27.186523+0000","last_unstale":"2024-09-13T18:20:27.187296+0000","last_undegraded":"2024-09-13T18:20:27.187296+0000","last_fullsized":"2024-09-13T18:20:27.187296+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:28:30.453945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.10","version":"89'10","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:25.180438+0000","last_change":"2024-09-13T18:20:01.569755+0000","last_active":"2024-09-13T18:20:25.180438+0000","last_peered":"2024-09-13T18:20:25.180438+0000","last_clean":"2024-09-13T18:20:25.180438+0000","last_became_active":"2024-09-13T18:20:01.569445+0000","last_became_peered":"2024-09-13T18:20:01.569445+0000","last_unstale":"2024-09-13T18:20:25.180438+0000","last_undegraded":"2024-09-13T18:20:25.180438+0000","last_fullsized":"2024-09-13T18:20:25.180438+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:45:20.733014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.179043+0000","last_change":"2024-09-13T18:20:06.136742+0000","last_active":"2024-09-13T18:20:26.179043+0000","last_peered":"2024-09-13T18:20:26.179043+0000","last_clean":"2024-09-13T18:20:26.179043+0000","last_became_active":"2024-09-13T18:20:06.136373+0000","last_became_peered":"2024-09-13T18:20:06.136373+0000","last_unstale":"2024-09-13T18:20:26.179043+0000","last_undegraded":"2024-09-13T18:20:26.179043+0000","last_fullsized":"2024-09-13T18:20:26.179043+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:49:11.782489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.176954+0000","last_change":"2024-09-13T18:20:01.922910+0000","last_active":"2024-09-13T18:20:06.176954+0000","last_peered":"2024-09-13T18:20:06.176954+0000","last_clean":"2024-09-13T18:20:06.176954+0000","last_became_active":"2024-09-13T18:20:01.921162+0000","last_became_peered":"2024-09-13T18:20:01.921162+0000","last_unstale":"2024-09-13T18:20:06.176954+0000","last_undegraded":"2024-09-13T18:20:06.176954+0000","last_fullsized":"2024-09-13T18:20:06.176954+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:14:47.243418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'3","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.197268+0000","last_change":"2024-09-13T18:20:01.575830+0000","last_active":"2024-09-13T18:20:26.197268+0000","last_peered":"2024-09-13T18:20:26.197268+0000","last_clean":"2024-09-13T18:20:26.197268+0000","last_became_active":"2024-09-13T18:20:01.575334+0000","last_became_peered":"2024-09-13T18:20:01.575334+0000","last_unstale":"2024-09-13T18:20:26.197268+0000","last_undegraded":"2024-09-13T18:20:26.197268+0000","last_fullsized":"2024-09-13T18:20:26.197268+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:13:00.935600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183501+0000","last_change":"2024-09-13T18:20:06.138609+0000","last_active":"2024-09-13T18:20:26.183501+0000","last_peered":"2024-09-13T18:20:26.183501+0000","last_clean":"2024-09-13T18:20:26.183501+0000","last_became_active":"2024-09-13T18:20:06.138232+0000","last_became_peered":"2024-09-13T18:20:06.138232+0000","last_unstale":"2024-09-13T18:20:26.183501+0000","last_undegraded":"2024-09-13T18:20:26.183501+0000","last_fullsized":"2024-09-13T18:20:26.183501+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:46:06.227407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.177024+0000","last_change":"2024-09-13T18:20:01.928757+0000","last_active":"2024-09-13T18:20:06.177024+0000","last_peered":"2024-09-13T18:20:06.177024+0000","last_clean":"2024-09-13T18:20:06.177024+0000","last_became_active":"2024-09-13T18:20:01.928350+0000","last_became_peered":"2024-09-13T18:20:01.928350+0000","last_unstale":"2024-09-13T18:20:06.177024+0000","last_undegraded":"2024-09-13T18:20:06.177024+0000","last_fullsized":"2024-09-13T18:20:06.177024+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:17:43.208106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:25.180579+0000","last_change":"2024-09-13T18:20:01.579128+0000","last_active":"2024-09-13T18:20:25.180579+0000","last_peered":"2024-09-13T18:20:25.180579+0000","last_clean":"2024-09-13T18:20:25.180579+0000","last_became_active":"2024-09-13T18:20:01.578796+0000","last_became_peered":"2024-09-13T18:20:01.578796+0000","last_unstale":"2024-09-13T18:20:25.180579+0000","last_undegraded":"2024-09-13T18:20:25.180579+0000","last_fullsized":"2024-09-13T18:20:25.180579+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:30:04.836103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.182950+0000","last_change":"2024-09-13T18:20:06.131404+0000","last_active":"2024-09-13T18:20:26.182950+0000","last_peered":"2024-09-13T18:20:26.182950+0000","last_clean":"2024-09-13T18:20:26.182950+0000","last_became_active":"2024-09-13T18:20:06.130931+0000","last_became_peered":"2024-09-13T18:20:06.130931+0000","last_unstale":"2024-09-13T18:20:26.182950+0000","last_undegraded":"2024-09-13T18:20:26.182950+0000","last_fullsized":"2024-09-13T18:20:26.182950+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:55:54.568816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.178724+0000","last_change":"2024-09-13T18:20:01.916089+0000","last_active":"2024-09-13T18:20:26.178724+0000","last_peered":"2024-09-13T18:20:26.178724+0000","last_clean":"2024-09-13T18:20:26.178724+0000","last_became_active":"2024-09-13T18:20:01.915783+0000","last_became_peered":"2024-09-13T18:20:01.915783+0000","last_unstale":"2024-09-13T18:20:26.178724+0000","last_undegraded":"2024-09-13T18:20:26.178724+0000","last_fullsized":"2024-09-13T18:20:26.178724+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:43:25.886029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"89'4","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:25.181228+0000","last_change":"2024-09-13T18:20:01.577381+0000","last_active":"2024-09-13T18:20:25.181228+0000","last_peered":"2024-09-13T18:20:25.181228+0000","last_clean":"2024-09-13T18:20:25.181228+0000","last_became_active":"2024-09-13T18:20:01.577044+0000","last_became_peered":"2024-09-13T18:20:01.577044+0000","last_unstale":"2024-09-13T18:20:25.181228+0000","last_undegraded":"2024-09-13T18:20:25.181228+0000","last_fullsized":"2024-09-13T18:20:25.181228+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:00.907948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.140300+0000","last_change":"2024-09-13T18:20:06.139971+0000","last_active":"2024-09-13T18:20:06.140300+0000","last_peered":"2024-09-13T18:20:06.140300+0000","last_clean":"2024-09-13T18:20:06.140300+0000","last_became_active":"2024-09-13T18:20:06.139507+0000","last_became_peered":"2024-09-13T18:20:06.139507+0000","last_unstale":"2024-09-13T18:20:06.140300+0000","last_undegraded":"2024-09-13T18:20:06.140300+0000","last_fullsized":"2024-09-13T18:20:06.140300+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:40:21.037768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.122321+0000","last_change":"2024-09-13T18:20:01.915348+0000","last_active":"2024-09-13T18:20:06.122321+0000","last_peered":"2024-09-13T18:20:06.122321+0000","last_clean":"2024-09-13T18:20:06.122321+0000","last_became_active":"2024-09-13T18:20:01.915042+0000","last_became_peered":"2024-09-13T18:20:01.915042+0000","last_unstale":"2024-09-13T18:20:06.122321+0000","last_undegraded":"2024-09-13T18:20:06.122321+0000","last_fullsized":"2024-09-13T18:20:06.122321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:31:33.712314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"91'17","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.256191+0000","last_change":"2024-09-13T18:20:01.578456+0000","last_active":"2024-09-13T18:20:28.256191+0000","last_peered":"2024-09-13T18:20:28.256191+0000","last_clean":"2024-09-13T18:20:28.256191+0000","last_became_active":"2024-09-13T18:20:01.578051+0000","last_became_peered":"2024-09-13T18:20:01.578051+0000","last_unstale":"2024-09-13T18:20:28.256191+0000","last_undegraded":"2024-09-13T18:20:28.256191+0000","last_fullsized":"2024-09-13T18:20:28.256191+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:34:17.651620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10396,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":12,"num_write":19,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183092+0000","last_change":"2024-09-13T18:20:06.131410+0000","last_active":"2024-09-13T18:20:26.183092+0000","last_peered":"2024-09-13T18:20:26.183092+0000","last_clean":"2024-09-13T18:20:26.183092+0000","last_became_active":"2024-09-13T18:20:06.131060+0000","last_became_peered":"2024-09-13T18:20:06.131060+0000","last_unstale":"2024-09-13T18:20:26.183092+0000","last_undegraded":"2024-09-13T18:20:26.183092+0000","last_fullsized":"2024-09-13T18:20:26.183092+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:08:25.266971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183935+0000","last_change":"2024-09-13T18:20:01.928839+0000","last_active":"2024-09-13T18:20:26.183935+0000","last_peered":"2024-09-13T18:20:26.183935+0000","last_clean":"2024-09-13T18:20:26.183935+0000","last_became_active":"2024-09-13T18:20:01.925774+0000","last_became_peered":"2024-09-13T18:20:01.925774+0000","last_unstale":"2024-09-13T18:20:26.183935+0000","last_undegraded":"2024-09-13T18:20:26.183935+0000","last_fullsized":"2024-09-13T18:20:26.183935+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:15:23.049671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.746467+0000","last_change":"2024-09-13T18:20:01.576332+0000","last_active":"2024-09-13T18:20:06.746467+0000","last_peered":"2024-09-13T18:20:06.746467+0000","last_clean":"2024-09-13T18:20:06.746467+0000","last_became_active":"2024-09-13T18:20:01.575862+0000","last_became_peered":"2024-09-13T18:20:01.575862+0000","last_unstale":"2024-09-13T18:20:06.746467+0000","last_undegraded":"2024-09-13T18:20:06.746467+0000","last_fullsized":"2024-09-13T18:20:06.746467+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:17:00.452480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.745244+0000","last_change":"2024-09-13T18:20:06.133142+0000","last_active":"2024-09-13T18:20:06.745244+0000","last_peered":"2024-09-13T18:20:06.745244+0000","last_clean":"2024-09-13T18:20:06.745244+0000","last_became_active":"2024-09-13T18:20:06.132740+0000","last_became_peered":"2024-09-13T18:20:06.132740+0000","last_unstale":"2024-09-13T18:20:06.745244+0000","last_undegraded":"2024-09-13T18:20:06.745244+0000","last_fullsized":"2024-09-13T18:20:06.745244+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:12:46.442481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.190799+0000","last_change":"2024-09-13T18:20:01.919673+0000","last_active":"2024-09-13T18:20:26.190799+0000","last_peered":"2024-09-13T18:20:26.190799+0000","last_clean":"2024-09-13T18:20:26.190799+0000","last_became_active":"2024-09-13T18:20:01.919358+0000","last_became_peered":"2024-09-13T18:20:01.919358+0000","last_unstale":"2024-09-13T18:20:26.190799+0000","last_undegraded":"2024-09-13T18:20:26.190799+0000","last_fullsized":"2024-09-13T18:20:26.190799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:40:11.407643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:25.181604+0000","last_change":"2024-09-13T18:20:01.575341+0000","last_active":"2024-09-13T18:20:25.181604+0000","last_peered":"2024-09-13T18:20:25.181604+0000","last_clean":"2024-09-13T18:20:25.181604+0000","last_became_active":"2024-09-13T18:20:01.575035+0000","last_became_peered":"2024-09-13T18:20:01.575035+0000","last_unstale":"2024-09-13T18:20:25.181604+0000","last_undegraded":"2024-09-13T18:20:25.181604+0000","last_fullsized":"2024-09-13T18:20:25.181604+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:24:29.891308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.182915+0000","last_change":"2024-09-13T18:20:06.133475+0000","last_active":"2024-09-13T18:20:26.182915+0000","last_peered":"2024-09-13T18:20:26.182915+0000","last_clean":"2024-09-13T18:20:26.182915+0000","last_became_active":"2024-09-13T18:20:06.133109+0000","last_became_peered":"2024-09-13T18:20:06.133109+0000","last_unstale":"2024-09-13T18:20:26.182915+0000","last_undegraded":"2024-09-13T18:20:26.182915+0000","last_fullsized":"2024-09-13T18:20:26.182915+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:47:19.567115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.184156+0000","last_change":"2024-09-13T18:20:01.922660+0000","last_active":"2024-09-13T18:20:26.184156+0000","last_peered":"2024-09-13T18:20:26.184156+0000","last_clean":"2024-09-13T18:20:26.184156+0000","last_became_active":"2024-09-13T18:20:01.922370+0000","last_became_peered":"2024-09-13T18:20:01.922370+0000","last_unstale":"2024-09-13T18:20:26.184156+0000","last_undegraded":"2024-09-13T18:20:26.184156+0000","last_fullsized":"2024-09-13T18:20:26.184156+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:54:32.029704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"91'4","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.249427+0000","last_change":"2024-09-13T18:20:01.580975+0000","last_active":"2024-09-13T18:20:28.249427+0000","last_peered":"2024-09-13T18:20:28.249427+0000","last_clean":"2024-09-13T18:20:28.249427+0000","last_became_active":"2024-09-13T18:20:01.580647+0000","last_became_peered":"2024-09-13T18:20:01.580647+0000","last_unstale":"2024-09-13T18:20:28.249427+0000","last_undegraded":"2024-09-13T18:20:28.249427+0000","last_fullsized":"2024-09-13T18:20:28.249427+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:26:20.942915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.192336+0000","last_change":"2024-09-13T18:20:06.140003+0000","last_active":"2024-09-13T18:20:26.192336+0000","last_peered":"2024-09-13T18:20:26.192336+0000","last_clean":"2024-09-13T18:20:26.192336+0000","last_became_active":"2024-09-13T18:20:06.139663+0000","last_became_peered":"2024-09-13T18:20:06.139663+0000","last_unstale":"2024-09-13T18:20:26.192336+0000","last_undegraded":"2024-09-13T18:20:26.192336+0000","last_fullsized":"2024-09-13T18:20:26.192336+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:54:49.662401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.189993+0000","last_change":"2024-09-13T18:20:06.134448+0000","last_active":"2024-09-13T18:20:26.189993+0000","last_peered":"2024-09-13T18:20:26.189993+0000","last_clean":"2024-09-13T18:20:26.189993+0000","last_became_active":"2024-09-13T18:20:06.134084+0000","last_became_peered":"2024-09-13T18:20:06.134084+0000","last_unstale":"2024-09-13T18:20:26.189993+0000","last_undegraded":"2024-09-13T18:20:26.189993+0000","last_fullsized":"2024-09-13T18:20:26.189993+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:55:50.475665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183131+0000","last_change":"2024-09-13T18:20:01.918955+0000","last_active":"2024-09-13T18:20:26.183131+0000","last_peered":"2024-09-13T18:20:26.183131+0000","last_clean":"2024-09-13T18:20:26.183131+0000","last_became_active":"2024-09-13T18:20:01.918666+0000","last_became_peered":"2024-09-13T18:20:01.918666+0000","last_unstale":"2024-09-13T18:20:26.183131+0000","last_undegraded":"2024-09-13T18:20:26.183131+0000","last_fullsized":"2024-09-13T18:20:26.183131+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:08:50.697710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.745892+0000","last_change":"2024-09-13T18:20:01.578789+0000","last_active":"2024-09-13T18:20:06.745892+0000","last_peered":"2024-09-13T18:20:06.745892+0000","last_clean":"2024-09-13T18:20:06.745892+0000","last_became_active":"2024-09-13T18:20:01.578476+0000","last_became_peered":"2024-09-13T18:20:01.578476+0000","last_unstale":"2024-09-13T18:20:06.745892+0000","last_undegraded":"2024-09-13T18:20:06.745892+0000","last_fullsized":"2024-09-13T18:20:06.745892+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:40:58.355492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183716+0000","last_change":"2024-09-13T18:20:06.129765+0000","last_active":"2024-09-13T18:20:26.183716+0000","last_peered":"2024-09-13T18:20:26.183716+0000","last_clean":"2024-09-13T18:20:26.183716+0000","last_became_active":"2024-09-13T18:20:06.129310+0000","last_became_peered":"2024-09-13T18:20:06.129310+0000","last_unstale":"2024-09-13T18:20:26.183716+0000","last_undegraded":"2024-09-13T18:20:26.183716+0000","last_fullsized":"2024-09-13T18:20:26.183716+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:34:07.394871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.191040+0000","last_change":"2024-09-13T18:20:01.914804+0000","last_active":"2024-09-13T18:20:26.191040+0000","last_peered":"2024-09-13T18:20:26.191040+0000","last_clean":"2024-09-13T18:20:26.191040+0000","last_became_active":"2024-09-13T18:20:01.914517+0000","last_became_peered":"2024-09-13T18:20:01.914517+0000","last_unstale":"2024-09-13T18:20:26.191040+0000","last_undegraded":"2024-09-13T18:20:26.191040+0000","last_fullsized":"2024-09-13T18:20:26.191040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:57:10.734935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"91'3","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:27.300317+0000","last_change":"2024-09-13T18:20:01.569201+0000","last_active":"2024-09-13T18:20:27.300317+0000","last_peered":"2024-09-13T18:20:27.300317+0000","last_clean":"2024-09-13T18:20:27.300317+0000","last_became_active":"2024-09-13T18:20:01.568908+0000","last_became_peered":"2024-09-13T18:20:01.568908+0000","last_unstale":"2024-09-13T18:20:27.300317+0000","last_undegraded":"2024-09-13T18:20:27.300317+0000","last_fullsized":"2024-09-13T18:20:27.300317+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:16:34.069130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"91'7","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.205826+0000","last_change":"2024-09-13T18:20:01.579225+0000","last_active":"2024-09-13T18:20:28.205826+0000","last_peered":"2024-09-13T18:20:28.205826+0000","last_clean":"2024-09-13T18:20:28.205826+0000","last_became_active":"2024-09-13T18:20:01.578826+0000","last_became_peered":"2024-09-13T18:20:01.578826+0000","last_unstale":"2024-09-13T18:20:28.205826+0000","last_undegraded":"2024-09-13T18:20:28.205826+0000","last_fullsized":"2024-09-13T18:20:28.205826+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:48:07.632239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.124287+0000","last_change":"2024-09-13T18:20:01.914885+0000","last_active":"2024-09-13T18:20:06.124287+0000","last_peered":"2024-09-13T18:20:06.124287+0000","last_clean":"2024-09-13T18:20:06.124287+0000","last_became_active":"2024-09-13T18:20:01.914599+0000","last_became_peered":"2024-09-13T18:20:01.914599+0000","last_unstale":"2024-09-13T18:20:06.124287+0000","last_undegraded":"2024-09-13T18:20:06.124287+0000","last_fullsized":"2024-09-13T18:20:06.124287+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:41:30.888675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.132751+0000","last_change":"2024-09-13T18:20:06.129376+0000","last_active":"2024-09-13T18:20:06.132751+0000","last_peered":"2024-09-13T18:20:06.132751+0000","last_clean":"2024-09-13T18:20:06.132751+0000","last_became_active":"2024-09-13T18:20:06.129032+0000","last_became_peered":"2024-09-13T18:20:06.129032+0000","last_unstale":"2024-09-13T18:20:06.132751+0000","last_undegraded":"2024-09-13T18:20:06.132751+0000","last_fullsized":"2024-09-13T18:20:06.132751+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:21:53.511813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:25.181451+0000","last_change":"2024-09-13T18:20:01.576069+0000","last_active":"2024-09-13T18:20:25.181451+0000","last_peered":"2024-09-13T18:20:25.181451+0000","last_clean":"2024-09-13T18:20:25.181451+0000","last_became_active":"2024-09-13T18:20:01.575760+0000","last_became_peered":"2024-09-13T18:20:01.575760+0000","last_unstale":"2024-09-13T18:20:25.181451+0000","last_undegraded":"2024-09-13T18:20:25.181451+0000","last_fullsized":"2024-09-13T18:20:25.181451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:05:47.677848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.122985+0000","last_change":"2024-09-13T18:20:01.917073+0000","last_active":"2024-09-13T18:20:06.122985+0000","last_peered":"2024-09-13T18:20:06.122985+0000","last_clean":"2024-09-13T18:20:06.122985+0000","last_became_active":"2024-09-13T18:20:01.916765+0000","last_became_peered":"2024-09-13T18:20:01.916765+0000","last_unstale":"2024-09-13T18:20:06.122985+0000","last_undegraded":"2024-09-13T18:20:06.122985+0000","last_fullsized":"2024-09-13T18:20:06.122985+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:22:07.788313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.133135+0000","last_change":"2024-09-13T18:20:06.131429+0000","last_active":"2024-09-13T18:20:06.133135+0000","last_peered":"2024-09-13T18:20:06.133135+0000","last_clean":"2024-09-13T18:20:06.133135+0000","last_became_active":"2024-09-13T18:20:06.131019+0000","last_became_peered":"2024-09-13T18:20:06.131019+0000","last_unstale":"2024-09-13T18:20:06.133135+0000","last_undegraded":"2024-09-13T18:20:06.133135+0000","last_fullsized":"2024-09-13T18:20:06.133135+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:44:37.892574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.182865+0000","last_change":"2024-09-13T18:20:01.914050+0000","last_active":"2024-09-13T18:20:26.182865+0000","last_peered":"2024-09-13T18:20:26.182865+0000","last_clean":"2024-09-13T18:20:26.182865+0000","last_became_active":"2024-09-13T18:20:01.913733+0000","last_became_peered":"2024-09-13T18:20:01.913733+0000","last_unstale":"2024-09-13T18:20:26.182865+0000","last_undegraded":"2024-09-13T18:20:26.182865+0000","last_fullsized":"2024-09-13T18:20:26.182865+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:52:43.672096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.123732+0000","last_change":"2024-09-13T18:20:01.924290+0000","last_active":"2024-09-13T18:20:06.123732+0000","last_peered":"2024-09-13T18:20:06.123732+0000","last_clean":"2024-09-13T18:20:06.123732+0000","last_became_active":"2024-09-13T18:20:01.923883+0000","last_became_peered":"2024-09-13T18:20:01.923883+0000","last_unstale":"2024-09-13T18:20:06.123732+0000","last_undegraded":"2024-09-13T18:20:06.123732+0000","last_fullsized":"2024-09-13T18:20:06.123732+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:11:18.737501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.125469+0000","last_change":"2024-09-13T18:20:01.928609+0000","last_active":"2024-09-13T18:20:06.125469+0000","last_peered":"2024-09-13T18:20:06.125469+0000","last_clean":"2024-09-13T18:20:06.125469+0000","last_became_active":"2024-09-13T18:20:01.927466+0000","last_became_peered":"2024-09-13T18:20:01.927466+0000","last_unstale":"2024-09-13T18:20:06.125469+0000","last_undegraded":"2024-09-13T18:20:06.125469+0000","last_fullsized":"2024-09-13T18:20:06.125469+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:18:58.610470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.189852+0000","last_change":"2024-09-13T18:20:01.915799+0000","last_active":"2024-09-13T18:20:26.189852+0000","last_peered":"2024-09-13T18:20:26.189852+0000","last_clean":"2024-09-13T18:20:26.189852+0000","last_became_active":"2024-09-13T18:20:01.915519+0000","last_became_peered":"2024-09-13T18:20:01.915519+0000","last_unstale":"2024-09-13T18:20:26.189852+0000","last_undegraded":"2024-09-13T18:20:26.189852+0000","last_fullsized":"2024-09-13T18:20:26.189852+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:08:26.065052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.178968+0000","last_change":"2024-09-13T18:20:01.918547+0000","last_active":"2024-09-13T18:20:26.178968+0000","last_peered":"2024-09-13T18:20:26.178968+0000","last_clean":"2024-09-13T18:20:26.178968+0000","last_became_active":"2024-09-13T18:20:01.918053+0000","last_became_peered":"2024-09-13T18:20:01.918053+0000","last_unstale":"2024-09-13T18:20:26.178968+0000","last_undegraded":"2024-09-13T18:20:26.178968+0000","last_fullsized":"2024-09-13T18:20:26.178968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:25:58.235101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183342+0000","last_change":"2024-09-13T18:20:01.916205+0000","last_active":"2024-09-13T18:20:26.183342+0000","last_peered":"2024-09-13T18:20:26.183342+0000","last_clean":"2024-09-13T18:20:26.183342+0000","last_became_active":"2024-09-13T18:20:01.915909+0000","last_became_peered":"2024-09-13T18:20:01.915909+0000","last_unstale":"2024-09-13T18:20:26.183342+0000","last_undegraded":"2024-09-13T18:20:26.183342+0000","last_fullsized":"2024-09-13T18:20:26.183342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:23:03.976513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.123579+0000","last_change":"2024-09-13T18:20:01.916621+0000","last_active":"2024-09-13T18:20:06.123579+0000","last_peered":"2024-09-13T18:20:06.123579+0000","last_clean":"2024-09-13T18:20:06.123579+0000","last_became_active":"2024-09-13T18:20:01.916335+0000","last_became_peered":"2024-09-13T18:20:01.916335+0000","last_unstale":"2024-09-13T18:20:06.123579+0000","last_undegraded":"2024-09-13T18:20:06.123579+0000","last_fullsized":"2024-09-13T18:20:06.123579+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:08:16.104571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.190215+0000","last_change":"2024-09-13T18:20:01.921195+0000","last_active":"2024-09-13T18:20:26.190215+0000","last_peered":"2024-09-13T18:20:26.190215+0000","last_clean":"2024-09-13T18:20:26.190215+0000","last_became_active":"2024-09-13T18:20:01.920895+0000","last_became_peered":"2024-09-13T18:20:01.920895+0000","last_unstale":"2024-09-13T18:20:26.190215+0000","last_undegraded":"2024-09-13T18:20:26.190215+0000","last_fullsized":"2024-09-13T18:20:26.190215+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:59:53.920047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.122232+0000","last_change":"2024-09-13T18:20:01.910889+0000","last_active":"2024-09-13T18:20:06.122232+0000","last_peered":"2024-09-13T18:20:06.122232+0000","last_clean":"2024-09-13T18:20:06.122232+0000","last_became_active":"2024-09-13T18:20:01.910451+0000","last_became_peered":"2024-09-13T18:20:01.910451+0000","last_unstale":"2024-09-13T18:20:06.122232+0000","last_undegraded":"2024-09-13T18:20:06.122232+0000","last_fullsized":"2024-09-13T18:20:06.122232+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:06:52.552785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.192420+0000","last_change":"2024-09-13T18:20:01.914041+0000","last_active":"2024-09-13T18:20:26.192420+0000","last_peered":"2024-09-13T18:20:26.192420+0000","last_clean":"2024-09-13T18:20:26.192420+0000","last_became_active":"2024-09-13T18:20:01.913240+0000","last_became_peered":"2024-09-13T18:20:01.913240+0000","last_unstale":"2024-09-13T18:20:26.192420+0000","last_undegraded":"2024-09-13T18:20:26.192420+0000","last_fullsized":"2024-09-13T18:20:26.192420+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:39:37.062544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.184154+0000","last_change":"2024-09-13T18:20:01.927322+0000","last_active":"2024-09-13T18:20:26.184154+0000","last_peered":"2024-09-13T18:20:26.184154+0000","last_clean":"2024-09-13T18:20:26.184154+0000","last_became_active":"2024-09-13T18:20:01.926767+0000","last_became_peered":"2024-09-13T18:20:01.926767+0000","last_unstale":"2024-09-13T18:20:26.184154+0000","last_undegraded":"2024-09-13T18:20:26.184154+0000","last_fullsized":"2024-09-13T18:20:26.184154+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:20.986089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.178873+0000","last_change":"2024-09-13T18:20:01.916314+0000","last_active":"2024-09-13T18:20:26.178873+0000","last_peered":"2024-09-13T18:20:26.178873+0000","last_clean":"2024-09-13T18:20:26.178873+0000","last_became_active":"2024-09-13T18:20:01.915790+0000","last_became_peered":"2024-09-13T18:20:01.915790+0000","last_unstale":"2024-09-13T18:20:26.178873+0000","last_undegraded":"2024-09-13T18:20:26.178873+0000","last_fullsized":"2024-09-13T18:20:26.178873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:20:30.063814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:27.187172+0000","last_change":"2024-09-13T18:20:27.187172+0000","last_active":"2024-09-13T18:20:27.187172+0000","last_peered":"2024-09-13T18:20:27.187172+0000","last_clean":"2024-09-13T18:20:27.187172+0000","last_became_active":"2024-09-13T18:20:27.186765+0000","last_became_peered":"2024-09-13T18:20:27.186765+0000","last_unstale":"2024-09-13T18:20:27.187172+0000","last_undegraded":"2024-09-13T18:20:27.187172+0000","last_fullsized":"2024-09-13T18:20:27.187172+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:37:01.469560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.124324+0000","last_change":"2024-09-13T18:20:01.924074+0000","last_active":"2024-09-13T18:20:06.124324+0000","last_peered":"2024-09-13T18:20:06.124324+0000","last_clean":"2024-09-13T18:20:06.124324+0000","last_became_active":"2024-09-13T18:20:01.922364+0000","last_became_peered":"2024-09-13T18:20:01.922364+0000","last_unstale":"2024-09-13T18:20:06.124324+0000","last_undegraded":"2024-09-13T18:20:06.124324+0000","last_fullsized":"2024-09-13T18:20:06.124324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:51:33.824362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183604+0000","last_change":"2024-09-13T18:20:01.914606+0000","last_active":"2024-09-13T18:20:26.183604+0000","last_peered":"2024-09-13T18:20:26.183604+0000","last_clean":"2024-09-13T18:20:26.183604+0000","last_became_active":"2024-09-13T18:20:01.914281+0000","last_became_peered":"2024-09-13T18:20:01.914281+0000","last_unstale":"2024-09-13T18:20:26.183604+0000","last_undegraded":"2024-09-13T18:20:26.183604+0000","last_fullsized":"2024-09-13T18:20:26.183604+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:55:42.211959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.184054+0000","last_change":"2024-09-13T18:20:01.928876+0000","last_active":"2024-09-13T18:20:26.184054+0000","last_peered":"2024-09-13T18:20:26.184054+0000","last_clean":"2024-09-13T18:20:26.184054+0000","last_became_active":"2024-09-13T18:20:01.928534+0000","last_became_peered":"2024-09-13T18:20:01.928534+0000","last_unstale":"2024-09-13T18:20:26.184054+0000","last_undegraded":"2024-09-13T18:20:26.184054+0000","last_fullsized":"2024-09-13T18:20:26.184054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:14:28.196746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183517+0000","last_change":"2024-09-13T18:20:01.920642+0000","last_active":"2024-09-13T18:20:26.183517+0000","last_peered":"2024-09-13T18:20:26.183517+0000","last_clean":"2024-09-13T18:20:26.183517+0000","last_became_active":"2024-09-13T18:20:01.920335+0000","last_became_peered":"2024-09-13T18:20:01.920335+0000","last_unstale":"2024-09-13T18:20:26.183517+0000","last_undegraded":"2024-09-13T18:20:26.183517+0000","last_fullsized":"2024-09-13T18:20:26.183517+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:59:20.894563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:27.187529+0000","last_change":"2024-09-13T18:20:27.187529+0000","last_active":"2024-09-13T18:20:27.187529+0000","last_peered":"2024-09-13T18:20:27.187529+0000","last_clean":"2024-09-13T18:20:27.187529+0000","last_became_active":"2024-09-13T18:20:27.187123+0000","last_became_peered":"2024-09-13T18:20:27.187123+0000","last_unstale":"2024-09-13T18:20:27.187529+0000","last_undegraded":"2024-09-13T18:20:27.187529+0000","last_fullsized":"2024-09-13T18:20:27.187529+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:43:19.601235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.189462+0000","last_change":"2024-09-13T18:20:01.917111+0000","last_active":"2024-09-13T18:20:26.189462+0000","last_peered":"2024-09-13T18:20:26.189462+0000","last_clean":"2024-09-13T18:20:26.189462+0000","last_became_active":"2024-09-13T18:20:01.916826+0000","last_became_peered":"2024-09-13T18:20:01.916826+0000","last_unstale":"2024-09-13T18:20:26.189462+0000","last_undegraded":"2024-09-13T18:20:26.189462+0000","last_fullsized":"2024-09-13T18:20:26.189462+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:36:08.150028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.178921+0000","last_change":"2024-09-13T18:20:01.904496+0000","last_active":"2024-09-13T18:20:26.178921+0000","last_peered":"2024-09-13T18:20:26.178921+0000","last_clean":"2024-09-13T18:20:26.178921+0000","last_became_active":"2024-09-13T18:20:01.904199+0000","last_became_peered":"2024-09-13T18:20:01.904199+0000","last_unstale":"2024-09-13T18:20:26.178921+0000","last_undegraded":"2024-09-13T18:20:26.178921+0000","last_fullsized":"2024-09-13T18:20:26.178921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:44.044162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.745431+0000","last_change":"2024-09-13T18:20:01.919023+0000","last_active":"2024-09-13T18:20:06.745431+0000","last_peered":"2024-09-13T18:20:06.745431+0000","last_clean":"2024-09-13T18:20:06.745431+0000","last_became_active":"2024-09-13T18:20:01.918661+0000","last_became_peered":"2024-09-13T18:20:01.918661+0000","last_unstale":"2024-09-13T18:20:06.745431+0000","last_undegraded":"2024-09-13T18:20:06.745431+0000","last_fullsized":"2024-09-13T18:20:06.745431+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:25:26.779029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.177051+0000","last_change":"2024-09-13T18:20:01.922833+0000","last_active":"2024-09-13T18:20:06.177051+0000","last_peered":"2024-09-13T18:20:06.177051+0000","last_clean":"2024-09-13T18:20:06.177051+0000","last_became_active":"2024-09-13T18:20:01.922533+0000","last_became_peered":"2024-09-13T18:20:01.922533+0000","last_unstale":"2024-09-13T18:20:06.177051+0000","last_undegraded":"2024-09-13T18:20:06.177051+0000","last_fullsized":"2024-09-13T18:20:06.177051+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:27:11.557419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.197907+0000","last_change":"2024-09-13T18:20:01.925916+0000","last_active":"2024-09-13T18:20:26.197907+0000","last_peered":"2024-09-13T18:20:26.197907+0000","last_clean":"2024-09-13T18:20:26.197907+0000","last_became_active":"2024-09-13T18:20:01.925568+0000","last_became_peered":"2024-09-13T18:20:01.925568+0000","last_unstale":"2024-09-13T18:20:26.197907+0000","last_undegraded":"2024-09-13T18:20:26.197907+0000","last_fullsized":"2024-09-13T18:20:26.197907+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:58:24.955273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.184513+0000","last_change":"2024-09-13T18:20:01.924328+0000","last_active":"2024-09-13T18:20:26.184513+0000","last_peered":"2024-09-13T18:20:26.184513+0000","last_clean":"2024-09-13T18:20:26.184513+0000","last_became_active":"2024-09-13T18:20:01.924004+0000","last_became_peered":"2024-09-13T18:20:01.924004+0000","last_unstale":"2024-09-13T18:20:26.184513+0000","last_undegraded":"2024-09-13T18:20:26.184513+0000","last_fullsized":"2024-09-13T18:20:26.184513+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:23:29.204157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.190973+0000","last_change":"2024-09-13T18:20:01.922234+0000","last_active":"2024-09-13T18:20:26.190973+0000","last_peered":"2024-09-13T18:20:26.190973+0000","last_clean":"2024-09-13T18:20:26.190973+0000","last_became_active":"2024-09-13T18:20:01.921196+0000","last_became_peered":"2024-09-13T18:20:01.921196+0000","last_unstale":"2024-09-13T18:20:26.190973+0000","last_undegraded":"2024-09-13T18:20:26.190973+0000","last_fullsized":"2024-09-13T18:20:26.190973+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:24:10.144346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.176993+0000","last_change":"2024-09-13T18:20:01.921740+0000","last_active":"2024-09-13T18:20:06.176993+0000","last_peered":"2024-09-13T18:20:06.176993+0000","last_clean":"2024-09-13T18:20:06.176993+0000","last_became_active":"2024-09-13T18:20:01.919634+0000","last_became_peered":"2024-09-13T18:20:01.919634+0000","last_unstale":"2024-09-13T18:20:06.176993+0000","last_undegraded":"2024-09-13T18:20:06.176993+0000","last_fullsized":"2024-09-13T18:20:06.176993+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:43:27.600637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.197426+0000","last_change":"2024-09-13T18:20:01.924806+0000","last_active":"2024-09-13T18:20:26.197426+0000","last_peered":"2024-09-13T18:20:26.197426+0000","last_clean":"2024-09-13T18:20:26.197426+0000","last_became_active":"2024-09-13T18:20:01.924514+0000","last_became_peered":"2024-09-13T18:20:01.924514+0000","last_unstale":"2024-09-13T18:20:26.197426+0000","last_undegraded":"2024-09-13T18:20:26.197426+0000","last_fullsized":"2024-09-13T18:20:26.197426+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:47.323362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.122321+0000","last_change":"2024-09-13T18:20:01.912402+0000","last_active":"2024-09-13T18:20:06.122321+0000","last_peered":"2024-09-13T18:20:06.122321+0000","last_clean":"2024-09-13T18:20:06.122321+0000","last_became_active":"2024-09-13T18:20:01.911534+0000","last_became_peered":"2024-09-13T18:20:01.911534+0000","last_unstale":"2024-09-13T18:20:06.122321+0000","last_undegraded":"2024-09-13T18:20:06.122321+0000","last_fullsized":"2024-09-13T18:20:06.122321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:22:52.021688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.745369+0000","last_change":"2024-09-13T18:20:01.930248+0000","last_active":"2024-09-13T18:20:06.745369+0000","last_peered":"2024-09-13T18:20:06.745369+0000","last_clean":"2024-09-13T18:20:06.745369+0000","last_became_active":"2024-09-13T18:20:01.929931+0000","last_became_peered":"2024-09-13T18:20:01.929931+0000","last_unstale":"2024-09-13T18:20:06.745369+0000","last_undegraded":"2024-09-13T18:20:06.745369+0000","last_fullsized":"2024-09-13T18:20:06.745369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:37:33.086722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.190113+0000","last_change":"2024-09-13T18:20:01.921913+0000","last_active":"2024-09-13T18:20:26.190113+0000","last_peered":"2024-09-13T18:20:26.190113+0000","last_clean":"2024-09-13T18:20:26.190113+0000","last_became_active":"2024-09-13T18:20:01.921395+0000","last_became_peered":"2024-09-13T18:20:01.921395+0000","last_unstale":"2024-09-13T18:20:26.190113+0000","last_undegraded":"2024-09-13T18:20:26.190113+0000","last_fullsized":"2024-09-13T18:20:26.190113+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:54:50.816870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.190140+0000","last_change":"2024-09-13T18:20:01.912654+0000","last_active":"2024-09-13T18:20:26.190140+0000","last_peered":"2024-09-13T18:20:26.190140+0000","last_clean":"2024-09-13T18:20:26.190140+0000","last_became_active":"2024-09-13T18:20:01.912356+0000","last_became_peered":"2024-09-13T18:20:01.912356+0000","last_unstale":"2024-09-13T18:20:26.190140+0000","last_undegraded":"2024-09-13T18:20:26.190140+0000","last_fullsized":"2024-09-13T18:20:26.190140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:25:18.099396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183955+0000","last_change":"2024-09-13T18:20:01.923373+0000","last_active":"2024-09-13T18:20:26.183955+0000","last_peered":"2024-09-13T18:20:26.183955+0000","last_clean":"2024-09-13T18:20:26.183955+0000","last_became_active":"2024-09-13T18:20:01.923077+0000","last_became_peered":"2024-09-13T18:20:01.923077+0000","last_unstale":"2024-09-13T18:20:26.183955+0000","last_undegraded":"2024-09-13T18:20:26.183955+0000","last_fullsized":"2024-09-13T18:20:26.183955+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:07:52.955705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":13418,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":9,"num_read_kb":17,"num_write":139,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":303104,"data_stored":36667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281684,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383698,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":995328,"data_stored":495945,"data_compressed":17567,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.47599999999999998}]},{"osd":1,"last update":"Fri Sep 13 18:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.58499999999999996}]},{"osd":2,"last update":"Fri Sep 13 18:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.622}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":4,"last update":"Fri Sep 13 18:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.64100000000000001}]},{"osd":5,"last update":"Fri Sep 13 18:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.69699999999999995}]},{"osd":6,"last update":"Fri Sep 13 18:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.56699999999999995}]},{"osd":7,"last update":"Fri Sep 13 18:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.68100000000000005}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":9,"last update":"Fri Sep 13 18:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.84999999999999998}]},{"osd":10,"last update":"Fri Sep 13 18:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.751}]}]},{"osd":10,"up_from":72,"seq":309237645335,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1077248,"data_stored":1090703,"data_compressed":53391,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":1,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.66700000000000004}]},{"osd":2,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.625}]},{"osd":3,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.60999999999999999}]},{"osd":4,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.64500000000000002}]},{"osd":5,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.56599999999999995}]},{"osd":6,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.76100000000000001}]},{"osd":7,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.81599999999999995}]},{"osd":8,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.80400000000000005}]},{"osd":9,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.83399999999999996}]},{"osd":11,"last update":"Fri Sep 13 18:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.872}]}]},{"osd":9,"up_from":67,"seq":287762808860,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":500559,"data_compressed":17567,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.311}]},{"osd":1,"last update":"Fri Sep 13 18:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.5}]},{"osd":2,"last update":"Fri Sep 13 18:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.53900000000000003}]},{"osd":3,"last update":"Fri Sep 13 18:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.34300000000000003}]},{"osd":4,"last update":"Fri Sep 13 18:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.57599999999999996}]},{"osd":5,"last update":"Fri Sep 13 18:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.55400000000000005}]},{"osd":6,"last update":"Fri Sep 13 18:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.39800000000000002}]},{"osd":7,"last update":"Fri Sep 13 18:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.47999999999999998}]},{"osd":8,"last update":"Fri Sep 13 18:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.624}]},{"osd":10,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.63900000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]}]},{"osd":8,"up_from":59,"seq":253403070496,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":974848,"data_stored":477189,"data_compressed":11642,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.53600000000000003,"15min":0.54800000000000004},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.60199999999999998,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.54900000000000004,"15min":0.56200000000000006},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.68600000000000005,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.53500000000000003}]},{"osd":1,"last update":"Fri Sep 13 18:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56000000000000005,"15min":0.55900000000000005},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56999999999999995,"15min":0.57099999999999995},"min":{"1min":0.40600000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72199999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.73699999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.42599999999999999,"15min":0.42999999999999999},"min":{"1min":0.26600000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.55200000000000005,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.247,"5min":0.224,"15min":0.224},"max":{"1min":0.59799999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.48399999999999999}]},{"osd":3,"last update":"Fri Sep 13 18:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.54700000000000004,"15min":0.55800000000000005},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.71999999999999997,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53800000000000003,"15min":0.53900000000000003},"min":{"1min":0.38600000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.44600000000000001}]},{"osd":4,"last update":"Fri Sep 13 18:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.66400000000000003,"15min":0.63600000000000001},"min":{"1min":0.45100000000000001,"5min":0.37,"15min":0.37},"max":{"1min":5.3540000000000001,"5min":5.3540000000000001,"15min":5.3540000000000001},"last":0.68000000000000005}]},{"osd":5,"last update":"Fri Sep 13 18:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.53100000000000003,"15min":0.53700000000000003},"min":{"1min":0.31900000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.70099999999999996,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.51900000000000002,"15min":0.52300000000000002},"min":{"1min":0.32100000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.63500000000000001,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.40100000000000002}]},{"osd":6,"last update":"Fri Sep 13 18:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.60599999999999998,"15min":0.61199999999999999},"min":{"1min":0.35199999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.75600000000000001,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.64400000000000002,"15min":0.65000000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.77200000000000002,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.65800000000000003}]},{"osd":7,"last update":"Fri Sep 13 18:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.66600000000000004,"15min":0.67500000000000004},"min":{"1min":0.44800000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.749,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.65600000000000003,"15min":0.66300000000000003},"min":{"1min":0.39700000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.73899999999999999,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.62}]},{"osd":9,"last update":"Fri Sep 13 18:20:24 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63800000000000001,"15min":0.63300000000000001},"min":{"1min":0.46600000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.65900000000000003,"15min":0.65600000000000003},"min":{"1min":0.52800000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.95399999999999996,"5min":1.012,"15min":1.012},"last":0.57899999999999996}]},{"osd":10,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":5.3360000000000003,"5min":5.3360000000000003,"15min":5.3360000000000003},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.72499999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]}]},{"osd":7,"up_from":51,"seq":219043332132,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":1024000,"data_stored":1068087,"data_compressed":47466,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.94399999999999995,"15min":1.012},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.84299999999999997,"5min":12.068,"15min":12.068},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.46999999999999997,"15min":0.45900000000000002},"min":{"1min":0.35199999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.61599999999999999}]},{"osd":1,"last update":"Fri Sep 13 18:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.36499999999999999,"15min":0.35699999999999998},"min":{"1min":0.26100000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.38100000000000001,"15min":0.375},"min":{"1min":0.27800000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.62,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.44900000000000001}]},{"osd":2,"last update":"Fri Sep 13 18:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.59099999999999997,"15min":0.53700000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":7.6820000000000004,"5min":7.6820000000000004,"15min":7.6820000000000004},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.51600000000000001,"15min":0.50700000000000001},"min":{"1min":0.378,"5min":0.371,"15min":0.371},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.58799999999999997}]},{"osd":3,"last update":"Fri Sep 13 18:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.49299999999999999,"15min":0.48099999999999998},"min":{"1min":0.36299999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.754},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.49299999999999999,"15min":0.48299999999999998},"min":{"1min":0.35399999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.63100000000000001}]},{"osd":4,"last update":"Fri Sep 13 18:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.44,"15min":0.433},"min":{"1min":0.29099999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.621,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.45400000000000001,"15min":0.45000000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.439}]},{"osd":5,"last update":"Fri Sep 13 18:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.55700000000000005,"15min":0.55000000000000004},"min":{"1min":0.39900000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.53700000000000003,"15min":0.53100000000000003},"min":{"1min":0.39900000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.68200000000000005}]},{"osd":6,"last update":"Fri Sep 13 18:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.55000000000000004,"15min":0.54000000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.59299999999999997,"15min":0.58199999999999996},"min":{"1min":0.48099999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.77700000000000002}]},{"osd":8,"last update":"Fri Sep 13 18:20:01 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.58999999999999997,"15min":0.58699999999999997},"min":{"1min":0.42499999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.72999999999999998,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.76600000000000001},{"interface":"front","average":{"1min":1.0669999999999999,"5min":0.71899999999999997,"15min":0.66100000000000003},"min":{"1min":0.443,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":7.7149999999999999,"5min":7.7149999999999999,"15min":7.7149999999999999},"last":0.71999999999999997}]},{"osd":9,"last update":"Fri Sep 13 18:19:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.84599999999999997}]},{"osd":10,"last update":"Fri Sep 13 18:19:50 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.64600000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]}]},{"osd":6,"up_from":46,"seq":197568495658,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":1003520,"data_stored":496015,"data_compressed":17567,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.248,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.43099999999999999,"15min":0.437},"min":{"1min":0.27600000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.218}]},{"osd":1,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.52700000000000002,"15min":0.52200000000000002},"min":{"1min":0.35499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.69199999999999995,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55000000000000004,"15min":0.53600000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.66500000000000004}]},{"osd":2,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.54200000000000004,"15min":0.54600000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53600000000000003,"15min":0.54700000000000004},"min":{"1min":0.36599999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.75,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.52000000000000002}]},{"osd":3,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42299999999999999,"15min":0.42599999999999999},"min":{"1min":0.28999999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.69499999999999995,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.309},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.46700000000000003,"15min":0.47099999999999997},"min":{"1min":0.30099999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.35599999999999998}]},{"osd":4,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.52300000000000002,"15min":0.51500000000000001},"min":{"1min":0.33300000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55500000000000005,"15min":0.55300000000000005},"min":{"1min":0.39600000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.74199999999999999,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.68300000000000005}]},{"osd":5,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59899999999999998,"15min":0.60799999999999998},"min":{"1min":0.436,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.57399999999999995,"15min":0.59299999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.77900000000000003,"5min":0.874,"15min":0.874},"last":0.496}]},{"osd":7,"last update":"Fri Sep 13 18:19:43 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57399999999999995,"15min":0.57299999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.71299999999999997,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56599999999999995,"15min":0.56200000000000006},"min":{"1min":0.435,"5min":0.434,"15min":0.434},"max":{"1min":0.77100000000000002,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.56899999999999995}]},{"osd":8,"last update":"Fri Sep 13 18:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.61799999999999999,"15min":0.621},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.79000000000000004,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.59399999999999997,"15min":0.59899999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.82999999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.65200000000000002}]},{"osd":9,"last update":"Fri Sep 13 18:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.39500000000000002}]},{"osd":10,"last update":"Fri Sep 13 18:19:49 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.58199999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]}]},{"osd":5,"up_from":38,"seq":163208757294,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":1068359,"data_compressed":47466,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:19:58 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.49299999999999999,"15min":0.49399999999999999},"min":{"1min":0.35299999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.61099999999999999,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.51400000000000001,"15min":0.51200000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.66600000000000004,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.52800000000000002}]},{"osd":1,"last update":"Fri Sep 13 18:19:58 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.51600000000000001,"15min":0.502},"min":{"1min":0.41899999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.77200000000000002,"5min":0.878,"15min":0.878},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.52000000000000002,"15min":0.51000000000000001},"min":{"1min":0.34399999999999997,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.78600000000000003,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.59199999999999997}]},{"osd":2,"last update":"Fri Sep 13 18:19:58 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.376,"15min":0.36199999999999999},"min":{"1min":0.27700000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.39000000000000001,"15min":0.38400000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.58499999999999996,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.41199999999999998}]},{"osd":3,"last update":"Fri Sep 13 18:19:58 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.51600000000000001,"15min":0.501},"min":{"1min":0.35799999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.51500000000000001,"15min":0.51800000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.70499999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.505}]},{"osd":4,"last update":"Fri Sep 13 18:19:58 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.54700000000000004,"15min":0.53800000000000003},"min":{"1min":0.40400000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.75900000000000001,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.76400000000000001,"15min":0.60799999999999998},"min":{"1min":0.376,"5min":0.374,"15min":0.374},"max":{"1min":0.80200000000000005,"5min":18.571000000000002,"15min":18.571000000000002},"last":0.48199999999999998}]},{"osd":6,"last update":"Fri Sep 13 18:20:22 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.42399999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55400000000000005,"15min":0.55000000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.629}]},{"osd":7,"last update":"Fri Sep 13 18:19:43 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.61499999999999999,"15min":0.60799999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.61699999999999999,"15min":0.61099999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.85299999999999998,"5min":0.88,"15min":0.88},"last":0.67600000000000005}]},{"osd":8,"last update":"Fri Sep 13 18:20:02 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.51700000000000002,"15min":0.51300000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.73699999999999999,"5min":0.753,"15min":0.753},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.54300000000000004,"15min":0.53700000000000003},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.69099999999999995}]},{"osd":9,"last update":"Fri Sep 13 18:19:27 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.70199999999999996}]},{"osd":10,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.72499999999999998}]}]},{"osd":4,"up_from":33,"seq":141733920818,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":486669,"data_compressed":14601,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:19:33 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.48299999999999998,"15min":0.47699999999999998},"min":{"1min":0.35399999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.69399999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.433},{"interface":"front","average":{"1min":0.501,"5min":0.51200000000000001,"15min":0.47499999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.84599999999999997,"5min":4.2539999999999996,"15min":4.2539999999999996},"last":0.51900000000000002}]},{"osd":1,"last update":"Fri Sep 13 18:19:33 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.34899999999999998,"15min":0.34399999999999997},"min":{"1min":0.27600000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.372,"5min":0.34699999999999998,"15min":0.34300000000000003},"min":{"1min":0.25600000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.52700000000000002,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.42099999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:19:33 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.53100000000000003,"15min":0.52300000000000002},"min":{"1min":0.40999999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.53700000000000003,"15min":0.53300000000000003},"min":{"1min":0.38500000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.74399999999999999}]},{"osd":3,"last update":"Fri Sep 13 18:19:33 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.49299999999999999},"min":{"1min":0.35199999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.72599999999999998,"5min":4.2679999999999998,"15min":4.2679999999999998},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.48899999999999999,"15min":0.48499999999999999},"min":{"1min":0.40200000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.69299999999999995}]},{"osd":5,"last update":"Fri Sep 13 18:19:55 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.54800000000000004,"15min":0.53500000000000003},"min":{"1min":0.38500000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78100000000000003,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.55000000000000004,"15min":0.53700000000000003},"min":{"1min":0.39900000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.71099999999999997,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.59599999999999997}]},{"osd":6,"last update":"Fri Sep 13 18:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.52300000000000002,"15min":0.51100000000000001},"min":{"1min":0.41999999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.52300000000000002,"15min":0.50900000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.76600000000000001}]},{"osd":7,"last update":"Fri Sep 13 18:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.44400000000000001,"15min":0.438},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.48499999999999999,"15min":0.47699999999999998},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.48899999999999999}]},{"osd":8,"last update":"Fri Sep 13 18:20:07 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.63,"15min":0.626},"min":{"1min":0.505,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.85899999999999999,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.65000000000000002,"15min":0.64500000000000002},"min":{"1min":0.52300000000000002,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.70599999999999996}]},{"osd":9,"last update":"Fri Sep 13 18:19:25 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.77900000000000003}]},{"osd":10,"last update":"Fri Sep 13 18:19:51 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.625},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.63900000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]}]},{"osd":3,"up_from":25,"seq":107374182455,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":495344,"data_compressed":17567,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:20:10 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.32900000000000001,"15min":0.307},"min":{"1min":0.248,"5min":0.183,"15min":0.183},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.33900000000000002,"15min":0.32700000000000001},"min":{"1min":0.222,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.47599999999999998,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.314}]},{"osd":1,"last update":"Fri Sep 13 18:20:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.50900000000000001,"15min":0.48799999999999999},"min":{"1min":0.40000000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.65600000000000003,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.51400000000000001,"15min":0.48799999999999999},"min":{"1min":0.39400000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.71299999999999997,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.61599999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:20:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52400000000000002,"15min":0.52200000000000002},"min":{"1min":0.35999999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.68700000000000006,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52300000000000002,"15min":0.501},"min":{"1min":0.36599999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.73399999999999999,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.45800000000000002}]},{"osd":4,"last update":"Fri Sep 13 18:19:32 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.5,"15min":0.48599999999999999},"min":{"1min":0.374,"5min":0.312,"15min":0.312},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.501},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.52400000000000002,"15min":0.51400000000000001},"min":{"1min":0.372,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.749,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.433}]},{"osd":5,"last update":"Fri Sep 13 18:19:54 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54000000000000004,"15min":0.53500000000000003},"min":{"1min":0.38300000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.749,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.39600000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72899999999999998,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.51500000000000001}]},{"osd":6,"last update":"Fri Sep 13 18:20:23 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.443,"15min":0.44700000000000001},"min":{"1min":0.30299999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.59199999999999997,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.48099999999999998,"15min":0.46700000000000003},"min":{"1min":0.313,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.70699999999999996,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.443}]},{"osd":7,"last update":"Fri Sep 13 18:19:39 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.80400000000000005,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.57999999999999996,"15min":0.57399999999999995},"min":{"1min":0.44800000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.77500000000000002,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.60899999999999999}]},{"osd":8,"last update":"Fri Sep 13 18:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.69599999999999995}]},{"osd":9,"last update":"Fri Sep 13 18:19:27 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.55900000000000005}]},{"osd":10,"last update":"Fri Sep 13 18:19:49 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.626}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]}]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":480392,"data_compressed":11642,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:19:45 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47399999999999998,"15min":0.47899999999999998},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.57599999999999996,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.495},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.48099999999999998,"15min":0.47599999999999998},"min":{"1min":0.318,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.61899999999999999}]},{"osd":1,"last update":"Fri Sep 13 18:19:45 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.49199999999999999,"15min":0.47199999999999998},"min":{"1min":0.34999999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.78400000000000003,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.49099999999999999,"15min":0.46500000000000002},"min":{"1min":0.32300000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.78100000000000003,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.53600000000000003}]},{"osd":3,"last update":"Fri Sep 13 18:20:13 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.52700000000000002,"15min":0.51600000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.66100000000000003,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.55200000000000005,"15min":0.55100000000000005},"min":{"1min":0.377,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.67100000000000004,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.75700000000000001}]},{"osd":4,"last update":"Fri Sep 13 18:19:35 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.504,"15min":0.49099999999999999},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.72899999999999998,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.50900000000000001,"15min":0.503},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.68899999999999995,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.54700000000000004}]},{"osd":5,"last update":"Fri Sep 13 18:19:52 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.38400000000000001,"15min":0.375},"min":{"1min":0.30399999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.371},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.40000000000000002,"15min":0.39000000000000001},"min":{"1min":0.26900000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.58899999999999997,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.31900000000000001}]},{"osd":6,"last update":"Fri Sep 13 18:20:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.56299999999999994,"15min":0.54000000000000004},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.57799999999999996,"15min":0.56299999999999994},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.80100000000000005,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.70199999999999996}]},{"osd":7,"last update":"Fri Sep 13 18:19:37 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78200000000000003,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.61599999999999999,"15min":0.61899999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.76600000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.52300000000000002}]},{"osd":8,"last update":"Fri Sep 13 18:20:07 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.54000000000000004,"15min":0.54200000000000004},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.73899999999999999,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.36599999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.44700000000000001}]},{"osd":9,"last update":"Fri Sep 13 18:19:25 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.56399999999999995}]},{"osd":10,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.57399999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]}]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":1007616,"data_stored":499248,"data_compressed":17567,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:20:24 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.502,"15min":0.495},"min":{"1min":0.35499999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.626},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.52300000000000002,"15min":0.51500000000000001},"min":{"1min":0.33200000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.77800000000000002,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.439}]},{"osd":2,"last update":"Fri Sep 13 18:19:46 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.501,"15min":0.48999999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.46999999999999997,"15min":0.46899999999999997},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.64200000000000002,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.496}]},{"osd":3,"last update":"Fri Sep 13 18:20:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.53900000000000003,"15min":0.52100000000000002},"min":{"1min":0.38600000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.74399999999999999,"5min":0.749,"15min":0.749},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56499999999999995,"15min":0.54400000000000004},"min":{"1min":0.39700000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76900000000000002,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.52700000000000002}]},{"osd":4,"last update":"Fri Sep 13 18:19:32 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.38,"15min":0.36099999999999999},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.434,"5min":0.376,"15min":0.35599999999999998},"min":{"1min":0.28399999999999997,"5min":0.217,"15min":0.217},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.40500000000000003}]},{"osd":5,"last update":"Fri Sep 13 18:19:53 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56699999999999995,"15min":0.56200000000000006},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55100000000000005,"15min":0.53700000000000003},"min":{"1min":0.40500000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.71299999999999997}]},{"osd":6,"last update":"Fri Sep 13 18:20:23 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57699999999999996,"15min":0.56699999999999995},"min":{"1min":0.41299999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.621,"15min":0.61099999999999999},"min":{"1min":0.45100000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.88300000000000001,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.72899999999999998}]},{"osd":7,"last update":"Fri Sep 13 18:19:38 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.49299999999999999,"15min":0.48699999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.502,"15min":0.5},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.745}]},{"osd":8,"last update":"Fri Sep 13 18:20:07 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.627,"15min":0.629},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.66000000000000003}]},{"osd":9,"last update":"Fri Sep 13 18:19:27 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.70099999999999996}]},{"osd":10,"last update":"Fri Sep 13 18:19:49 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.76600000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":995328,"data_stored":487439,"data_compressed":14601,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 18:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.501,"15min":0.496},"min":{"1min":0.35599999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.37},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.50700000000000001,"15min":0.505},"min":{"1min":0.35699999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.69199999999999995,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.504}]},{"osd":2,"last update":"Fri Sep 13 18:19:47 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.52900000000000003,"15min":0.52700000000000002},"min":{"1min":0.34399999999999997,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.70099999999999996,"5min":0.754,"15min":0.754},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.53000000000000003,"15min":0.51100000000000001},"min":{"1min":0.38500000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.81799999999999995,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.47699999999999998}]},{"osd":3,"last update":"Fri Sep 13 18:19:05 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.38400000000000001,"15min":0.378},"min":{"1min":0.26300000000000001,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.437,"5min":0.39400000000000002,"15min":0.376},"min":{"1min":0.28100000000000003,"5min":0.251,"15min":0.251},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.28799999999999998}]},{"osd":4,"last update":"Fri Sep 13 18:19:33 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.503,"15min":0.49099999999999999},"min":{"1min":0.38600000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.85699999999999998,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.51900000000000002,"15min":0.50700000000000001},"min":{"1min":0.35499999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.45100000000000001}]},{"osd":5,"last update":"Fri Sep 13 18:19:53 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.61499999999999999,"15min":0.56699999999999995},"min":{"1min":0.371,"5min":0.313,"15min":0.313},"max":{"1min":0.89600000000000002,"5min":7.194,"15min":7.194},"last":0.495},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.55100000000000005,"15min":0.52300000000000002},"min":{"1min":0.378,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.87,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.53900000000000003}]},{"osd":6,"last update":"Fri Sep 13 18:20:21 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.46400000000000002,"15min":0.45400000000000001},"min":{"1min":0.28899999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.79000000000000004,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.45800000000000002,"15min":0.46100000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.56699999999999995,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.32400000000000001}]},{"osd":7,"last update":"Fri Sep 13 18:19:38 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.58599999999999997,"15min":0.57599999999999996},"min":{"1min":0.435,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.55500000000000005,"15min":0.54600000000000004},"min":{"1min":0.47299999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.437}]},{"osd":8,"last update":"Fri Sep 13 18:20:01 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.60799999999999998,"15min":0.60999999999999999},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61899999999999999,"15min":0.61699999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.52900000000000003}]},{"osd":9,"last update":"Fri Sep 13 18:19:28 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.41899999999999998}]},{"osd":10,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.55400000000000005}]},{"osd":11,"last update":"Fri Sep 13 18:20:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.621},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.66300000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34188,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34188,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34188,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35824,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35824,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35824,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35824,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35824,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":12212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":12084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:20:30.694 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:20:30.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:30 smithi204 ceph-mon[34777]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:20:30.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:30 smithi204 ceph-mon[34777]: mds.? [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] up:active 2024-09-13T18:20:30.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:30 smithi204 ceph-mon[34777]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-13T18:20:30.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:30 smithi204 ceph-mon[34777]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-13T18:20:30.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:30 smithi204 ceph-mon[34777]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:20:30.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:30 smithi204 ceph-mon[34777]: pgmap v293: 129 pgs: 1 activating, 128 active+clean; 590 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 4.1 KiB/s wr, 12 op/s; 2 B/s, 0 objects/s recovering 2024-09-13T18:20:30.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:30 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/1587228232' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:20:30.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.819+0000 7fee75602640 1 -- 172.21.15.114:0/877856755 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fee700ffe20 msgr2=0x7fee701066b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:30.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.819+0000 7fee75602640 1 --2- 172.21.15.114:0/877856755 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fee700ffe20 0x7fee701066b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fee68009ca0 tx=0x7fee6802a8c0 comp rx=0 tx=0).stop 2024-09-13T18:20:30.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.819+0000 7fee75602640 1 -- 172.21.15.114:0/877856755 shutdown_connections 2024-09-13T18:20:30.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.819+0000 7fee75602640 1 --2- 172.21.15.114:0/877856755 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fee700ffe20 0x7fee701066b0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:30.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.819+0000 7fee75602640 1 --2- 172.21.15.114:0/877856755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee700ff460 0x7fee700ff8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:30.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.819+0000 7fee75602640 1 --2- 172.21.15.114:0/877856755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee700fe270 0x7fee700fe670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:30.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.819+0000 7fee75602640 1 -- 172.21.15.114:0/877856755 >> 172.21.15.114:0/877856755 conn(0x7fee700f99d0 msgr2=0x7fee700fbe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:30.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.819+0000 7fee75602640 1 -- 172.21.15.114:0/877856755 shutdown_connections 2024-09-13T18:20:30.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.819+0000 7fee75602640 1 -- 172.21.15.114:0/877856755 wait complete. 2024-09-13T18:20:30.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.820+0000 7fee75602640 1 Processor -- start 2024-09-13T18:20:30.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.820+0000 7fee75602640 1 -- start start 2024-09-13T18:20:30.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.820+0000 7fee75602640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fee700fe270 0x7fee70195030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:30.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.820+0000 7fee75602640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee700ff460 0x7fee70193670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:30.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.820+0000 7fee75602640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee700ffe20 0x7fee70193bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:30.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.820+0000 7fee75602640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee70194310 con 0x7fee700fe270 2024-09-13T18:20:30.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.820+0000 7fee75602640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee70194480 con 0x7fee700ffe20 2024-09-13T18:20:30.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.820+0000 7fee75602640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee7019fb00 con 0x7fee700ff460 2024-09-13T18:20:30.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.820+0000 7fee6effd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fee700fe270 0x7fee70195030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:30.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.820+0000 7fee6effd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fee700fe270 0x7fee70195030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60836/0 (socket says 172.21.15.114:60836) 2024-09-13T18:20:30.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.820+0000 7fee6effd640 1 -- 172.21.15.114:0/2010142148 learned_addr learned my addr 172.21.15.114:0/2010142148 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:20:30.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.821+0000 7fee6f7fe640 1 --2- 172.21.15.114:0/2010142148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee700ffe20 0x7fee70193bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:30.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.821+0000 7fee6effd640 1 -- 172.21.15.114:0/2010142148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee700ff460 msgr2=0x7fee70193670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:30.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.821+0000 7fee6e7fc640 1 --2- 172.21.15.114:0/2010142148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee700ff460 0x7fee70193670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:30.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.821+0000 7fee6effd640 1 --2- 172.21.15.114:0/2010142148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee700ff460 0x7fee70193670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:30.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.821+0000 7fee6effd640 1 -- 172.21.15.114:0/2010142148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee700ffe20 msgr2=0x7fee70193bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:30.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.821+0000 7fee6effd640 1 --2- 172.21.15.114:0/2010142148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee700ffe20 0x7fee70193bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:30.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.821+0000 7fee6effd640 1 -- 172.21.15.114:0/2010142148 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee68009900 con 0x7fee700fe270 2024-09-13T18:20:30.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.821+0000 7fee6effd640 1 --2- 172.21.15.114:0/2010142148 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fee700fe270 0x7fee70195030 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fee6000dbf0 tx=0x7fee6000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:30.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.821+0000 7fee67fff640 1 -- 172.21.15.114:0/2010142148 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee60004220 con 0x7fee700fe270 2024-09-13T18:20:30.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.821+0000 7fee67fff640 1 -- 172.21.15.114:0/2010142148 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fee6000ae50 con 0x7fee700fe270 2024-09-13T18:20:30.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.821+0000 7fee75602640 1 -- 172.21.15.114:0/2010142148 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee7019fdb0 con 0x7fee700fe270 2024-09-13T18:20:30.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.821+0000 7fee67fff640 1 -- 172.21.15.114:0/2010142148 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6000e6b0 con 0x7fee700fe270 2024-09-13T18:20:30.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.821+0000 7fee75602640 1 -- 172.21.15.114:0/2010142148 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee701a0230 con 0x7fee700fe270 2024-09-13T18:20:30.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.822+0000 7fee75602640 1 -- 172.21.15.114:0/2010142148 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee70105750 con 0x7fee700fe270 2024-09-13T18:20:30.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.823+0000 7fee67fff640 1 -- 172.21.15.114:0/2010142148 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fee600035f0 con 0x7fee700fe270 2024-09-13T18:20:30.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.824+0000 7fee67fff640 1 --2- 172.21.15.114:0/2010142148 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fee5c0a1220 0x7fee5c0a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:30.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.824+0000 7fee6e7fc640 1 --2- 172.21.15.114:0/2010142148 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fee5c0a1220 0x7fee5c0a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:30.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.824+0000 7fee67fff640 1 -- 172.21.15.114:0/2010142148 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fee600ccc10 con 0x7fee700fe270 2024-09-13T18:20:30.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.824+0000 7fee6e7fc640 1 --2- 172.21.15.114:0/2010142148 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fee5c0a1220 0x7fee5c0a36e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fee58005fd0 tx=0x7fee58005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:30.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.828+0000 7fee67fff640 1 -- 172.21.15.114:0/2010142148 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fee60095fe0 con 0x7fee700fe270 2024-09-13T18:20:30.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.989+0000 7fee75602640 1 -- 172.21.15.114:0/2010142148 --> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fee7002c4f0 con 0x7fee5c0a1220 2024-09-13T18:20:31.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:30.999+0000 7fee67fff640 1 -- 172.21.15.114:0/2010142148 <== mgr.14148 v2:172.21.15.114:6800/3396185541 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+424028 (secure 0 0 0) 0x7fee7002c4f0 con 0x7fee5c0a1220 2024-09-13T18:20:31.007 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:20:31.009 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-13T18:20:31.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.003+0000 7fee75602640 1 -- 172.21.15.114:0/2010142148 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fee5c0a1220 msgr2=0x7fee5c0a36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:31.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.003+0000 7fee75602640 1 --2- 172.21.15.114:0/2010142148 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fee5c0a1220 0x7fee5c0a36e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fee58005fd0 tx=0x7fee58005850 comp rx=0 tx=0).stop 2024-09-13T18:20:31.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.003+0000 7fee75602640 1 -- 172.21.15.114:0/2010142148 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fee700fe270 msgr2=0x7fee70195030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:31.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.003+0000 7fee75602640 1 --2- 172.21.15.114:0/2010142148 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fee700fe270 0x7fee70195030 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fee6000dbf0 tx=0x7fee6000a620 comp rx=0 tx=0).stop 2024-09-13T18:20:31.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.003+0000 7fee75602640 1 -- 172.21.15.114:0/2010142148 shutdown_connections 2024-09-13T18:20:31.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.003+0000 7fee75602640 1 --2- 172.21.15.114:0/2010142148 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fee5c0a1220 0x7fee5c0a36e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:31.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.003+0000 7fee75602640 1 --2- 172.21.15.114:0/2010142148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee700ffe20 0x7fee70193bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:31.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.003+0000 7fee75602640 1 --2- 172.21.15.114:0/2010142148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee700ff460 0x7fee70193670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:31.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.003+0000 7fee75602640 1 --2- 172.21.15.114:0/2010142148 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fee700fe270 0x7fee70195030 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:31.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.003+0000 7fee75602640 1 -- 172.21.15.114:0/2010142148 >> 172.21.15.114:0/2010142148 conn(0x7fee700f99d0 msgr2=0x7fee70102920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:31.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.003+0000 7fee75602640 1 -- 172.21.15.114:0/2010142148 shutdown_connections 2024-09-13T18:20:31.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.003+0000 7fee75602640 1 -- 172.21.15.114:0/2010142148 wait complete. 2024-09-13T18:20:31.017 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-09-13T18:20:29.908599+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":603786,"num_objects":63,"num_object_clones":0,"num_object_copies":189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":145,"num_read_kb":136,"num_write":320,"num_write_kb":2989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":257,"ondisk_log_size":257,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":421,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360636,"kb_used_data":11828,"kb_used_omap":18,"kb_used_meta":348653,"kb_avail":1124581188,"statfs":{"total":1151940427776,"available":1151571136512,"internally_reserved":0,"allocated":12111872,"data_stored":7645949,"data_compressed":288644,"data_compressed_allocated":389120,"data_compressed_original":2277376,"omap_allocated":19129,"internal_metadata":357020999},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8825,"num_objects":39,"num_object_clones":0,"num_object_copies":117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":9,"num_read_kb":17,"num_write":91,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002330"},"pg_stats":[{"pgid":"2.1f","version":"91'13","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.256848+0000","last_change":"2024-09-13T18:20:01.577029+0000","last_active":"2024-09-13T18:20:28.256848+0000","last_peered":"2024-09-13T18:20:28.256848+0000","last_clean":"2024-09-13T18:20:28.256848+0000","last_became_active":"2024-09-13T18:20:01.576710+0000","last_became_peered":"2024-09-13T18:20:01.576710+0000","last_unstale":"2024-09-13T18:20:28.256848+0000","last_undegraded":"2024-09-13T18:20:28.256848+0000","last_fullsized":"2024-09-13T18:20:28.256848+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:25:11.977436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.190449+0000","last_change":"2024-09-13T18:20:01.922955+0000","last_active":"2024-09-13T18:20:26.190449+0000","last_peered":"2024-09-13T18:20:26.190449+0000","last_clean":"2024-09-13T18:20:26.190449+0000","last_became_active":"2024-09-13T18:20:01.922704+0000","last_became_peered":"2024-09-13T18:20:01.922704+0000","last_unstale":"2024-09-13T18:20:26.190449+0000","last_undegraded":"2024-09-13T18:20:26.190449+0000","last_fullsized":"2024-09-13T18:20:26.190449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:44:20.672784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.132781+0000","last_change":"2024-09-13T18:20:06.130388+0000","last_active":"2024-09-13T18:20:06.132781+0000","last_peered":"2024-09-13T18:20:06.132781+0000","last_clean":"2024-09-13T18:20:06.132781+0000","last_became_active":"2024-09-13T18:20:06.129895+0000","last_became_peered":"2024-09-13T18:20:06.129895+0000","last_unstale":"2024-09-13T18:20:06.132781+0000","last_undegraded":"2024-09-13T18:20:06.132781+0000","last_fullsized":"2024-09-13T18:20:06.132781+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:44:33.537849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.197866+0000","last_change":"2024-09-13T18:20:01.578921+0000","last_active":"2024-09-13T18:20:26.197866+0000","last_peered":"2024-09-13T18:20:26.197866+0000","last_clean":"2024-09-13T18:20:26.197866+0000","last_became_active":"2024-09-13T18:20:01.578627+0000","last_became_peered":"2024-09-13T18:20:01.578627+0000","last_unstale":"2024-09-13T18:20:26.197866+0000","last_undegraded":"2024-09-13T18:20:26.197866+0000","last_fullsized":"2024-09-13T18:20:26.197866+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:43:32.919156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.190717+0000","last_change":"2024-09-13T18:20:01.922901+0000","last_active":"2024-09-13T18:20:26.190717+0000","last_peered":"2024-09-13T18:20:26.190717+0000","last_clean":"2024-09-13T18:20:26.190717+0000","last_became_active":"2024-09-13T18:20:01.922604+0000","last_became_peered":"2024-09-13T18:20:01.922604+0000","last_unstale":"2024-09-13T18:20:26.190717+0000","last_undegraded":"2024-09-13T18:20:26.190717+0000","last_fullsized":"2024-09-13T18:20:26.190717+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:27:30.650010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.744777+0000","last_change":"2024-09-13T18:20:06.135406+0000","last_active":"2024-09-13T18:20:06.744777+0000","last_peered":"2024-09-13T18:20:06.744777+0000","last_clean":"2024-09-13T18:20:06.744777+0000","last_became_active":"2024-09-13T18:20:06.134558+0000","last_became_peered":"2024-09-13T18:20:06.134558+0000","last_unstale":"2024-09-13T18:20:06.744777+0000","last_undegraded":"2024-09-13T18:20:06.744777+0000","last_fullsized":"2024-09-13T18:20:06.744777+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:10.901966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"84'2","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.122416+0000","last_change":"2024-09-13T18:20:01.580040+0000","last_active":"2024-09-13T18:20:06.122416+0000","last_peered":"2024-09-13T18:20:06.122416+0000","last_clean":"2024-09-13T18:20:06.122416+0000","last_became_active":"2024-09-13T18:20:01.579715+0000","last_became_peered":"2024-09-13T18:20:01.579715+0000","last_unstale":"2024-09-13T18:20:06.122416+0000","last_undegraded":"2024-09-13T18:20:06.122416+0000","last_fullsized":"2024-09-13T18:20:06.122416+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:57:03.588169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.184089+0000","last_change":"2024-09-13T18:20:01.928314+0000","last_active":"2024-09-13T18:20:26.184089+0000","last_peered":"2024-09-13T18:20:26.184089+0000","last_clean":"2024-09-13T18:20:26.184089+0000","last_became_active":"2024-09-13T18:20:01.925356+0000","last_became_peered":"2024-09-13T18:20:01.925356+0000","last_unstale":"2024-09-13T18:20:26.184089+0000","last_undegraded":"2024-09-13T18:20:26.184089+0000","last_fullsized":"2024-09-13T18:20:26.184089+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:43.906842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.744757+0000","last_change":"2024-09-13T18:20:06.135886+0000","last_active":"2024-09-13T18:20:06.744757+0000","last_peered":"2024-09-13T18:20:06.744757+0000","last_clean":"2024-09-13T18:20:06.744757+0000","last_became_active":"2024-09-13T18:20:06.134587+0000","last_became_peered":"2024-09-13T18:20:06.134587+0000","last_unstale":"2024-09-13T18:20:06.744757+0000","last_undegraded":"2024-09-13T18:20:06.744757+0000","last_fullsized":"2024-09-13T18:20:06.744757+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:44:42.434729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.122797+0000","last_change":"2024-09-13T18:20:01.576908+0000","last_active":"2024-09-13T18:20:06.122797+0000","last_peered":"2024-09-13T18:20:06.122797+0000","last_clean":"2024-09-13T18:20:06.122797+0000","last_became_active":"2024-09-13T18:20:01.576478+0000","last_became_peered":"2024-09-13T18:20:01.576478+0000","last_unstale":"2024-09-13T18:20:06.122797+0000","last_undegraded":"2024-09-13T18:20:06.122797+0000","last_fullsized":"2024-09-13T18:20:06.122797+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:34:52.126901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.126411+0000","last_change":"2024-09-13T18:20:01.921960+0000","last_active":"2024-09-13T18:20:06.126411+0000","last_peered":"2024-09-13T18:20:06.126411+0000","last_clean":"2024-09-13T18:20:06.126411+0000","last_became_active":"2024-09-13T18:20:01.921652+0000","last_became_peered":"2024-09-13T18:20:01.921652+0000","last_unstale":"2024-09-13T18:20:06.126411+0000","last_undegraded":"2024-09-13T18:20:06.126411+0000","last_fullsized":"2024-09-13T18:20:06.126411+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:00:31.277613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.744407+0000","last_change":"2024-09-13T18:20:06.138983+0000","last_active":"2024-09-13T18:20:06.744407+0000","last_peered":"2024-09-13T18:20:06.744407+0000","last_clean":"2024-09-13T18:20:06.744407+0000","last_became_active":"2024-09-13T18:20:06.138708+0000","last_became_peered":"2024-09-13T18:20:06.138708+0000","last_unstale":"2024-09-13T18:20:06.744407+0000","last_undegraded":"2024-09-13T18:20:06.744407+0000","last_fullsized":"2024-09-13T18:20:06.744407+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:45:21.978217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.177004+0000","last_change":"2024-09-13T18:20:01.929817+0000","last_active":"2024-09-13T18:20:06.177004+0000","last_peered":"2024-09-13T18:20:06.177004+0000","last_clean":"2024-09-13T18:20:06.177004+0000","last_became_active":"2024-09-13T18:20:01.928907+0000","last_became_peered":"2024-09-13T18:20:01.928907+0000","last_unstale":"2024-09-13T18:20:06.177004+0000","last_undegraded":"2024-09-13T18:20:06.177004+0000","last_fullsized":"2024-09-13T18:20:06.177004+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:56:25.847230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:25.180585+0000","last_change":"2024-09-13T18:20:01.578968+0000","last_active":"2024-09-13T18:20:25.180585+0000","last_peered":"2024-09-13T18:20:25.180585+0000","last_clean":"2024-09-13T18:20:25.180585+0000","last_became_active":"2024-09-13T18:20:01.578623+0000","last_became_peered":"2024-09-13T18:20:01.578623+0000","last_unstale":"2024-09-13T18:20:25.180585+0000","last_undegraded":"2024-09-13T18:20:25.180585+0000","last_fullsized":"2024-09-13T18:20:25.180585+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:03:30.333107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.744596+0000","last_change":"2024-09-13T18:20:06.135977+0000","last_active":"2024-09-13T18:20:06.744596+0000","last_peered":"2024-09-13T18:20:06.744596+0000","last_clean":"2024-09-13T18:20:06.744596+0000","last_became_active":"2024-09-13T18:20:06.134844+0000","last_became_peered":"2024-09-13T18:20:06.134844+0000","last_unstale":"2024-09-13T18:20:06.744596+0000","last_undegraded":"2024-09-13T18:20:06.744596+0000","last_fullsized":"2024-09-13T18:20:06.744596+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:58:36.139240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.190165+0000","last_change":"2024-09-13T18:20:01.916675+0000","last_active":"2024-09-13T18:20:26.190165+0000","last_peered":"2024-09-13T18:20:26.190165+0000","last_clean":"2024-09-13T18:20:26.190165+0000","last_became_active":"2024-09-13T18:20:01.912902+0000","last_became_peered":"2024-09-13T18:20:01.912902+0000","last_unstale":"2024-09-13T18:20:26.190165+0000","last_undegraded":"2024-09-13T18:20:26.190165+0000","last_fullsized":"2024-09-13T18:20:26.190165+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:24:54.271569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:27.301309+0000","last_change":"2024-09-13T18:20:27.300845+0000","last_active":"2024-09-13T18:20:27.301309+0000","last_peered":"2024-09-13T18:20:27.301309+0000","last_clean":"2024-09-13T18:20:27.301309+0000","last_became_active":"2024-09-13T18:20:27.300417+0000","last_became_peered":"2024-09-13T18:20:27.300417+0000","last_unstale":"2024-09-13T18:20:27.301309+0000","last_undegraded":"2024-09-13T18:20:27.301309+0000","last_fullsized":"2024-09-13T18:20:27.301309+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:44:27.507140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.745268+0000","last_change":"2024-09-13T18:20:06.131539+0000","last_active":"2024-09-13T18:20:06.745268+0000","last_peered":"2024-09-13T18:20:06.745268+0000","last_clean":"2024-09-13T18:20:06.745268+0000","last_became_active":"2024-09-13T18:20:06.131148+0000","last_became_peered":"2024-09-13T18:20:06.131148+0000","last_unstale":"2024-09-13T18:20:06.745268+0000","last_undegraded":"2024-09-13T18:20:06.745268+0000","last_fullsized":"2024-09-13T18:20:06.745268+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:32:49.759831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183043+0000","last_change":"2024-09-13T18:20:01.913634+0000","last_active":"2024-09-13T18:20:26.183043+0000","last_peered":"2024-09-13T18:20:26.183043+0000","last_clean":"2024-09-13T18:20:26.183043+0000","last_became_active":"2024-09-13T18:20:01.913128+0000","last_became_peered":"2024-09-13T18:20:01.913128+0000","last_unstale":"2024-09-13T18:20:26.183043+0000","last_undegraded":"2024-09-13T18:20:26.183043+0000","last_fullsized":"2024-09-13T18:20:26.183043+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:37:15.251756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.192501+0000","last_change":"2024-09-13T18:20:01.579385+0000","last_active":"2024-09-13T18:20:26.192501+0000","last_peered":"2024-09-13T18:20:26.192501+0000","last_clean":"2024-09-13T18:20:26.192501+0000","last_became_active":"2024-09-13T18:20:01.579102+0000","last_became_peered":"2024-09-13T18:20:01.579102+0000","last_unstale":"2024-09-13T18:20:26.192501+0000","last_undegraded":"2024-09-13T18:20:26.192501+0000","last_fullsized":"2024-09-13T18:20:26.192501+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:22:00.432052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183422+0000","last_change":"2024-09-13T18:20:06.138110+0000","last_active":"2024-09-13T18:20:26.183422+0000","last_peered":"2024-09-13T18:20:26.183422+0000","last_clean":"2024-09-13T18:20:26.183422+0000","last_became_active":"2024-09-13T18:20:06.137763+0000","last_became_peered":"2024-09-13T18:20:06.137763+0000","last_unstale":"2024-09-13T18:20:26.183422+0000","last_undegraded":"2024-09-13T18:20:26.183422+0000","last_fullsized":"2024-09-13T18:20:26.183422+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:16:15.956216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.122199+0000","last_change":"2024-09-13T18:20:01.918318+0000","last_active":"2024-09-13T18:20:06.122199+0000","last_peered":"2024-09-13T18:20:06.122199+0000","last_clean":"2024-09-13T18:20:06.122199+0000","last_became_active":"2024-09-13T18:20:01.917977+0000","last_became_peered":"2024-09-13T18:20:01.917977+0000","last_unstale":"2024-09-13T18:20:06.122199+0000","last_undegraded":"2024-09-13T18:20:06.122199+0000","last_fullsized":"2024-09-13T18:20:06.122199+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:23.368866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.184253+0000","last_change":"2024-09-13T18:20:01.572009+0000","last_active":"2024-09-13T18:20:26.184253+0000","last_peered":"2024-09-13T18:20:26.184253+0000","last_clean":"2024-09-13T18:20:26.184253+0000","last_became_active":"2024-09-13T18:20:01.571313+0000","last_became_peered":"2024-09-13T18:20:01.571313+0000","last_unstale":"2024-09-13T18:20:26.184253+0000","last_undegraded":"2024-09-13T18:20:26.184253+0000","last_fullsized":"2024-09-13T18:20:26.184253+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:32:59.340476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.744412+0000","last_change":"2024-09-13T18:20:06.132212+0000","last_active":"2024-09-13T18:20:06.744412+0000","last_peered":"2024-09-13T18:20:06.744412+0000","last_clean":"2024-09-13T18:20:06.744412+0000","last_became_active":"2024-09-13T18:20:06.131809+0000","last_became_peered":"2024-09-13T18:20:06.131809+0000","last_unstale":"2024-09-13T18:20:06.744412+0000","last_undegraded":"2024-09-13T18:20:06.744412+0000","last_fullsized":"2024-09-13T18:20:06.744412+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:18:11.667551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"activating","last_fresh":"2024-09-13T18:20:27.181672+0000","last_change":"2024-09-13T18:20:27.181672+0000","last_active":"2024-09-13T18:20:26.189486+0000","last_peered":"2024-09-13T18:20:06.959341+0000","last_clean":"2024-09-13T18:20:06.959341+0000","last_became_active":"2024-09-13T18:20:01.922815+0000","last_became_peered":"2024-09-13T18:20:01.922815+0000","last_unstale":"2024-09-13T18:20:27.181672+0000","last_undegraded":"2024-09-13T18:20:27.181672+0000","last_fullsized":"2024-09-13T18:20:27.181672+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:59:57.606105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":["0","3","4","7"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:25.181259+0000","last_change":"2024-09-13T18:20:01.572025+0000","last_active":"2024-09-13T18:20:25.181259+0000","last_peered":"2024-09-13T18:20:25.181259+0000","last_clean":"2024-09-13T18:20:25.181259+0000","last_became_active":"2024-09-13T18:20:01.571321+0000","last_became_peered":"2024-09-13T18:20:01.571321+0000","last_unstale":"2024-09-13T18:20:25.181259+0000","last_undegraded":"2024-09-13T18:20:25.181259+0000","last_fullsized":"2024-09-13T18:20:25.181259+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:09.374364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.132797+0000","last_change":"2024-09-13T18:20:06.130687+0000","last_active":"2024-09-13T18:20:06.132797+0000","last_peered":"2024-09-13T18:20:06.132797+0000","last_clean":"2024-09-13T18:20:06.132797+0000","last_became_active":"2024-09-13T18:20:06.129839+0000","last_became_peered":"2024-09-13T18:20:06.129839+0000","last_unstale":"2024-09-13T18:20:06.132797+0000","last_undegraded":"2024-09-13T18:20:06.132797+0000","last_fullsized":"2024-09-13T18:20:06.132797+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:12:52.334945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.190458+0000","last_change":"2024-09-13T18:20:01.917244+0000","last_active":"2024-09-13T18:20:26.190458+0000","last_peered":"2024-09-13T18:20:26.190458+0000","last_clean":"2024-09-13T18:20:26.190458+0000","last_became_active":"2024-09-13T18:20:01.916948+0000","last_became_peered":"2024-09-13T18:20:01.916948+0000","last_unstale":"2024-09-13T18:20:26.190458+0000","last_undegraded":"2024-09-13T18:20:26.190458+0000","last_fullsized":"2024-09-13T18:20:26.190458+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:17:11.967096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'7","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.206617+0000","last_change":"2024-09-13T18:20:01.569123+0000","last_active":"2024-09-13T18:20:28.206617+0000","last_peered":"2024-09-13T18:20:28.206617+0000","last_clean":"2024-09-13T18:20:28.206617+0000","last_became_active":"2024-09-13T18:20:01.568720+0000","last_became_peered":"2024-09-13T18:20:01.568720+0000","last_unstale":"2024-09-13T18:20:28.206617+0000","last_undegraded":"2024-09-13T18:20:28.206617+0000","last_fullsized":"2024-09-13T18:20:28.206617+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:04:29.561013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.190854+0000","last_change":"2024-09-13T18:20:06.141771+0000","last_active":"2024-09-13T18:20:26.190854+0000","last_peered":"2024-09-13T18:20:26.190854+0000","last_clean":"2024-09-13T18:20:26.190854+0000","last_became_active":"2024-09-13T18:20:06.141362+0000","last_became_peered":"2024-09-13T18:20:06.141362+0000","last_unstale":"2024-09-13T18:20:26.190854+0000","last_undegraded":"2024-09-13T18:20:26.190854+0000","last_fullsized":"2024-09-13T18:20:26.190854+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:09:58.597605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.179047+0000","last_change":"2024-09-13T18:20:01.911048+0000","last_active":"2024-09-13T18:20:26.179047+0000","last_peered":"2024-09-13T18:20:26.179047+0000","last_clean":"2024-09-13T18:20:26.179047+0000","last_became_active":"2024-09-13T18:20:01.910735+0000","last_became_peered":"2024-09-13T18:20:01.910735+0000","last_unstale":"2024-09-13T18:20:26.179047+0000","last_undegraded":"2024-09-13T18:20:26.179047+0000","last_fullsized":"2024-09-13T18:20:26.179047+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:07:35.447676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.745685+0000","last_change":"2024-09-13T18:20:01.571627+0000","last_active":"2024-09-13T18:20:06.745685+0000","last_peered":"2024-09-13T18:20:06.745685+0000","last_clean":"2024-09-13T18:20:06.745685+0000","last_became_active":"2024-09-13T18:20:01.571250+0000","last_became_peered":"2024-09-13T18:20:01.571250+0000","last_unstale":"2024-09-13T18:20:06.745685+0000","last_undegraded":"2024-09-13T18:20:06.745685+0000","last_fullsized":"2024-09-13T18:20:06.745685+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:34:03.479324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.182932+0000","last_change":"2024-09-13T18:20:06.132974+0000","last_active":"2024-09-13T18:20:26.182932+0000","last_peered":"2024-09-13T18:20:26.182932+0000","last_clean":"2024-09-13T18:20:26.182932+0000","last_became_active":"2024-09-13T18:20:06.131668+0000","last_became_peered":"2024-09-13T18:20:06.131668+0000","last_unstale":"2024-09-13T18:20:26.182932+0000","last_undegraded":"2024-09-13T18:20:26.182932+0000","last_fullsized":"2024-09-13T18:20:26.182932+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:36:30.329761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.197759+0000","last_change":"2024-09-13T18:20:01.904670+0000","last_active":"2024-09-13T18:20:26.197759+0000","last_peered":"2024-09-13T18:20:26.197759+0000","last_clean":"2024-09-13T18:20:26.197759+0000","last_became_active":"2024-09-13T18:20:01.904316+0000","last_became_peered":"2024-09-13T18:20:01.904316+0000","last_unstale":"2024-09-13T18:20:26.197759+0000","last_undegraded":"2024-09-13T18:20:26.197759+0000","last_fullsized":"2024-09-13T18:20:26.197759+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:08:17.687179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"91'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:27.179594+0000","last_change":"2024-09-13T18:20:01.564750+0000","last_active":"2024-09-13T18:20:27.179594+0000","last_peered":"2024-09-13T18:20:27.179594+0000","last_clean":"2024-09-13T18:20:27.179594+0000","last_became_active":"2024-09-13T18:20:01.564096+0000","last_became_peered":"2024-09-13T18:20:01.564096+0000","last_unstale":"2024-09-13T18:20:27.179594+0000","last_undegraded":"2024-09-13T18:20:27.179594+0000","last_fullsized":"2024-09-13T18:20:27.179594+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:42:19.588824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.134398+0000","last_change":"2024-09-13T18:20:06.133386+0000","last_active":"2024-09-13T18:20:06.134398+0000","last_peered":"2024-09-13T18:20:06.134398+0000","last_clean":"2024-09-13T18:20:06.134398+0000","last_became_active":"2024-09-13T18:20:06.133004+0000","last_became_peered":"2024-09-13T18:20:06.133004+0000","last_unstale":"2024-09-13T18:20:06.134398+0000","last_undegraded":"2024-09-13T18:20:06.134398+0000","last_fullsized":"2024-09-13T18:20:06.134398+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:54:46.440410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.197632+0000","last_change":"2024-09-13T18:20:01.912823+0000","last_active":"2024-09-13T18:20:26.197632+0000","last_peered":"2024-09-13T18:20:26.197632+0000","last_clean":"2024-09-13T18:20:26.197632+0000","last_became_active":"2024-09-13T18:20:01.912123+0000","last_became_peered":"2024-09-13T18:20:01.912123+0000","last_unstale":"2024-09-13T18:20:26.197632+0000","last_undegraded":"2024-09-13T18:20:26.197632+0000","last_fullsized":"2024-09-13T18:20:26.197632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:32:33.544816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.122475+0000","last_change":"2024-09-13T18:20:01.577095+0000","last_active":"2024-09-13T18:20:06.122475+0000","last_peered":"2024-09-13T18:20:06.122475+0000","last_clean":"2024-09-13T18:20:06.122475+0000","last_became_active":"2024-09-13T18:20:01.576765+0000","last_became_peered":"2024-09-13T18:20:01.576765+0000","last_unstale":"2024-09-13T18:20:06.122475+0000","last_undegraded":"2024-09-13T18:20:06.122475+0000","last_fullsized":"2024-09-13T18:20:06.122475+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:44:08.750441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.178862+0000","last_change":"2024-09-13T18:20:06.131650+0000","last_active":"2024-09-13T18:20:26.178862+0000","last_peered":"2024-09-13T18:20:26.178862+0000","last_clean":"2024-09-13T18:20:26.178862+0000","last_became_active":"2024-09-13T18:20:06.130902+0000","last_became_peered":"2024-09-13T18:20:06.130902+0000","last_unstale":"2024-09-13T18:20:26.178862+0000","last_undegraded":"2024-09-13T18:20:26.178862+0000","last_fullsized":"2024-09-13T18:20:26.178862+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.366694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.122314+0000","last_change":"2024-09-13T18:20:01.918221+0000","last_active":"2024-09-13T18:20:06.122314+0000","last_peered":"2024-09-13T18:20:06.122314+0000","last_clean":"2024-09-13T18:20:06.122314+0000","last_became_active":"2024-09-13T18:20:01.917134+0000","last_became_peered":"2024-09-13T18:20:01.917134+0000","last_unstale":"2024-09-13T18:20:06.122314+0000","last_undegraded":"2024-09-13T18:20:06.122314+0000","last_fullsized":"2024-09-13T18:20:06.122314+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:19:06.269454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.249346+0000","last_change":"2024-09-13T18:20:01.572258+0000","last_active":"2024-09-13T18:20:28.249346+0000","last_peered":"2024-09-13T18:20:28.249346+0000","last_clean":"2024-09-13T18:20:28.249346+0000","last_became_active":"2024-09-13T18:20:01.571968+0000","last_became_peered":"2024-09-13T18:20:01.571968+0000","last_unstale":"2024-09-13T18:20:28.249346+0000","last_undegraded":"2024-09-13T18:20:28.249346+0000","last_fullsized":"2024-09-13T18:20:28.249346+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:40:59.143123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.179018+0000","last_change":"2024-09-13T18:20:06.131714+0000","last_active":"2024-09-13T18:20:26.179018+0000","last_peered":"2024-09-13T18:20:26.179018+0000","last_clean":"2024-09-13T18:20:26.179018+0000","last_became_active":"2024-09-13T18:20:06.131244+0000","last_became_peered":"2024-09-13T18:20:06.131244+0000","last_unstale":"2024-09-13T18:20:26.179018+0000","last_undegraded":"2024-09-13T18:20:26.179018+0000","last_fullsized":"2024-09-13T18:20:26.179018+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:37.946002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.178584+0000","last_change":"2024-09-13T18:20:01.917351+0000","last_active":"2024-09-13T18:20:26.178584+0000","last_peered":"2024-09-13T18:20:26.178584+0000","last_clean":"2024-09-13T18:20:26.178584+0000","last_became_active":"2024-09-13T18:20:01.917067+0000","last_became_peered":"2024-09-13T18:20:01.917067+0000","last_unstale":"2024-09-13T18:20:26.178584+0000","last_undegraded":"2024-09-13T18:20:26.178584+0000","last_fullsized":"2024-09-13T18:20:26.178584+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:35:37.723132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'1","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.462263+0000","last_change":"2024-09-13T18:20:01.573515+0000","last_active":"2024-09-13T18:20:06.462263+0000","last_peered":"2024-09-13T18:20:06.462263+0000","last_clean":"2024-09-13T18:20:06.462263+0000","last_became_active":"2024-09-13T18:20:01.573211+0000","last_became_peered":"2024-09-13T18:20:01.573211+0000","last_unstale":"2024-09-13T18:20:06.462263+0000","last_undegraded":"2024-09-13T18:20:06.462263+0000","last_fullsized":"2024-09-13T18:20:06.462263+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:05:30.912467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.134638+0000","last_change":"2024-09-13T18:20:06.133713+0000","last_active":"2024-09-13T18:20:06.134638+0000","last_peered":"2024-09-13T18:20:06.134638+0000","last_clean":"2024-09-13T18:20:06.134638+0000","last_became_active":"2024-09-13T18:20:06.133330+0000","last_became_peered":"2024-09-13T18:20:06.133330+0000","last_unstale":"2024-09-13T18:20:06.134638+0000","last_undegraded":"2024-09-13T18:20:06.134638+0000","last_fullsized":"2024-09-13T18:20:06.134638+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:48:07.133155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.192066+0000","last_change":"2024-09-13T18:20:01.921597+0000","last_active":"2024-09-13T18:20:26.192066+0000","last_peered":"2024-09-13T18:20:26.192066+0000","last_clean":"2024-09-13T18:20:26.192066+0000","last_became_active":"2024-09-13T18:20:01.920200+0000","last_became_peered":"2024-09-13T18:20:01.920200+0000","last_unstale":"2024-09-13T18:20:26.192066+0000","last_undegraded":"2024-09-13T18:20:26.192066+0000","last_fullsized":"2024-09-13T18:20:26.192066+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:22:40.517941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.186261+0000","last_change":"2024-09-13T18:20:01.571313+0000","last_active":"2024-09-13T18:20:26.186261+0000","last_peered":"2024-09-13T18:20:26.186261+0000","last_clean":"2024-09-13T18:20:26.186261+0000","last_became_active":"2024-09-13T18:20:01.570977+0000","last_became_peered":"2024-09-13T18:20:01.570977+0000","last_unstale":"2024-09-13T18:20:26.186261+0000","last_undegraded":"2024-09-13T18:20:26.186261+0000","last_fullsized":"2024-09-13T18:20:26.186261+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:06.870320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":341,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.122460+0000","last_change":"2024-09-13T18:18:51.237409+0000","last_active":"2024-09-13T18:20:06.122460+0000","last_peered":"2024-09-13T18:20:06.122460+0000","last_clean":"2024-09-13T18:20:06.122460+0000","last_became_active":"2024-09-13T18:18:51.236986+0000","last_became_peered":"2024-09-13T18:18:51.236986+0000","last_unstale":"2024-09-13T18:20:06.122460+0000","last_undegraded":"2024-09-13T18:20:06.122460+0000","last_fullsized":"2024-09-13T18:20:06.122460+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:15:40.757299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:15:40.757299+0000","last_clean_scrub_stamp":"2024-09-13T18:15:40.757299+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:58:29.679404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.177041+0000","last_change":"2024-09-13T18:20:06.130803+0000","last_active":"2024-09-13T18:20:06.177041+0000","last_peered":"2024-09-13T18:20:06.177041+0000","last_clean":"2024-09-13T18:20:06.177041+0000","last_became_active":"2024-09-13T18:20:06.130433+0000","last_became_peered":"2024-09-13T18:20:06.130433+0000","last_unstale":"2024-09-13T18:20:06.177041+0000","last_undegraded":"2024-09-13T18:20:06.177041+0000","last_fullsized":"2024-09-13T18:20:06.177041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:23:11.787084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.177067+0000","last_change":"2024-09-13T18:20:01.922702+0000","last_active":"2024-09-13T18:20:06.177067+0000","last_peered":"2024-09-13T18:20:06.177067+0000","last_clean":"2024-09-13T18:20:06.177067+0000","last_became_active":"2024-09-13T18:20:01.921109+0000","last_became_peered":"2024-09-13T18:20:01.921109+0000","last_unstale":"2024-09-13T18:20:06.177067+0000","last_undegraded":"2024-09-13T18:20:06.177067+0000","last_fullsized":"2024-09-13T18:20:06.177067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:03:34.618054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.123936+0000","last_change":"2024-09-13T18:20:01.571024+0000","last_active":"2024-09-13T18:20:06.123936+0000","last_peered":"2024-09-13T18:20:06.123936+0000","last_clean":"2024-09-13T18:20:06.123936+0000","last_became_active":"2024-09-13T18:20:01.570687+0000","last_became_peered":"2024-09-13T18:20:01.570687+0000","last_unstale":"2024-09-13T18:20:06.123936+0000","last_undegraded":"2024-09-13T18:20:06.123936+0000","last_fullsized":"2024-09-13T18:20:06.123936+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:10:25.126179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183401+0000","last_change":"2024-09-13T18:20:06.129025+0000","last_active":"2024-09-13T18:20:26.183401+0000","last_peered":"2024-09-13T18:20:26.183401+0000","last_clean":"2024-09-13T18:20:26.183401+0000","last_became_active":"2024-09-13T18:20:06.128618+0000","last_became_peered":"2024-09-13T18:20:06.128618+0000","last_unstale":"2024-09-13T18:20:26.183401+0000","last_undegraded":"2024-09-13T18:20:26.183401+0000","last_fullsized":"2024-09-13T18:20:26.183401+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:20:08.891259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.177033+0000","last_change":"2024-09-13T18:20:01.921697+0000","last_active":"2024-09-13T18:20:06.177033+0000","last_peered":"2024-09-13T18:20:06.177033+0000","last_clean":"2024-09-13T18:20:06.177033+0000","last_became_active":"2024-09-13T18:20:01.920174+0000","last_became_peered":"2024-09-13T18:20:01.920174+0000","last_unstale":"2024-09-13T18:20:06.177033+0000","last_undegraded":"2024-09-13T18:20:06.177033+0000","last_fullsized":"2024-09-13T18:20:06.177033+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:57:03.922725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.126610+0000","last_change":"2024-09-13T18:20:01.582102+0000","last_active":"2024-09-13T18:20:06.126610+0000","last_peered":"2024-09-13T18:20:06.126610+0000","last_clean":"2024-09-13T18:20:06.126610+0000","last_became_active":"2024-09-13T18:20:01.581841+0000","last_became_peered":"2024-09-13T18:20:01.581841+0000","last_unstale":"2024-09-13T18:20:06.126610+0000","last_undegraded":"2024-09-13T18:20:06.126610+0000","last_fullsized":"2024-09-13T18:20:06.126610+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:59:57.929703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.189924+0000","last_change":"2024-09-13T18:20:06.133126+0000","last_active":"2024-09-13T18:20:26.189924+0000","last_peered":"2024-09-13T18:20:26.189924+0000","last_clean":"2024-09-13T18:20:26.189924+0000","last_became_active":"2024-09-13T18:20:06.132747+0000","last_became_peered":"2024-09-13T18:20:06.132747+0000","last_unstale":"2024-09-13T18:20:26.189924+0000","last_undegraded":"2024-09-13T18:20:26.189924+0000","last_fullsized":"2024-09-13T18:20:26.189924+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:54:21.843090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.177082+0000","last_change":"2024-09-13T18:20:01.916306+0000","last_active":"2024-09-13T18:20:06.177082+0000","last_peered":"2024-09-13T18:20:06.177082+0000","last_clean":"2024-09-13T18:20:06.177082+0000","last_became_active":"2024-09-13T18:20:01.915739+0000","last_became_peered":"2024-09-13T18:20:01.915739+0000","last_unstale":"2024-09-13T18:20:06.177082+0000","last_undegraded":"2024-09-13T18:20:06.177082+0000","last_fullsized":"2024-09-13T18:20:06.177082+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:19:09.689909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"91'5","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.252883+0000","last_change":"2024-09-13T18:20:01.565821+0000","last_active":"2024-09-13T18:20:28.252883+0000","last_peered":"2024-09-13T18:20:28.252883+0000","last_clean":"2024-09-13T18:20:28.252883+0000","last_became_active":"2024-09-13T18:20:01.565467+0000","last_became_peered":"2024-09-13T18:20:01.565467+0000","last_unstale":"2024-09-13T18:20:28.252883+0000","last_undegraded":"2024-09-13T18:20:28.252883+0000","last_fullsized":"2024-09-13T18:20:28.252883+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:11.609400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.132093+0000","last_change":"2024-09-13T18:20:06.131348+0000","last_active":"2024-09-13T18:20:06.132093+0000","last_peered":"2024-09-13T18:20:06.132093+0000","last_clean":"2024-09-13T18:20:06.132093+0000","last_became_active":"2024-09-13T18:20:06.131019+0000","last_became_peered":"2024-09-13T18:20:06.131019+0000","last_unstale":"2024-09-13T18:20:06.132093+0000","last_undegraded":"2024-09-13T18:20:06.132093+0000","last_fullsized":"2024-09-13T18:20:06.132093+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:13:49.717503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183415+0000","last_change":"2024-09-13T18:20:01.922306+0000","last_active":"2024-09-13T18:20:26.183415+0000","last_peered":"2024-09-13T18:20:26.183415+0000","last_clean":"2024-09-13T18:20:26.183415+0000","last_became_active":"2024-09-13T18:20:01.921956+0000","last_became_peered":"2024-09-13T18:20:01.921956+0000","last_unstale":"2024-09-13T18:20:26.183415+0000","last_undegraded":"2024-09-13T18:20:26.183415+0000","last_fullsized":"2024-09-13T18:20:26.183415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:17:48.755318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:27.194592+0000","last_change":"2024-09-13T18:20:27.194085+0000","last_active":"2024-09-13T18:20:27.194592+0000","last_peered":"2024-09-13T18:20:27.194592+0000","last_clean":"2024-09-13T18:20:27.194592+0000","last_became_active":"2024-09-13T18:20:27.190136+0000","last_became_peered":"2024-09-13T18:20:27.190136+0000","last_unstale":"2024-09-13T18:20:27.194592+0000","last_undegraded":"2024-09-13T18:20:27.194592+0000","last_fullsized":"2024-09-13T18:20:27.194592+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:57:07.202901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.135975+0000","last_change":"2024-09-13T18:20:06.135498+0000","last_active":"2024-09-13T18:20:06.135975+0000","last_peered":"2024-09-13T18:20:06.135975+0000","last_clean":"2024-09-13T18:20:06.135975+0000","last_became_active":"2024-09-13T18:20:06.135095+0000","last_became_peered":"2024-09-13T18:20:06.135095+0000","last_unstale":"2024-09-13T18:20:06.135975+0000","last_undegraded":"2024-09-13T18:20:06.135975+0000","last_fullsized":"2024-09-13T18:20:06.135975+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:06:40.609718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:27.187296+0000","last_change":"2024-09-13T18:20:27.186825+0000","last_active":"2024-09-13T18:20:27.187296+0000","last_peered":"2024-09-13T18:20:27.187296+0000","last_clean":"2024-09-13T18:20:27.187296+0000","last_became_active":"2024-09-13T18:20:27.186523+0000","last_became_peered":"2024-09-13T18:20:27.186523+0000","last_unstale":"2024-09-13T18:20:27.187296+0000","last_undegraded":"2024-09-13T18:20:27.187296+0000","last_fullsized":"2024-09-13T18:20:27.187296+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:28:30.453945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.10","version":"89'10","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:25.180438+0000","last_change":"2024-09-13T18:20:01.569755+0000","last_active":"2024-09-13T18:20:25.180438+0000","last_peered":"2024-09-13T18:20:25.180438+0000","last_clean":"2024-09-13T18:20:25.180438+0000","last_became_active":"2024-09-13T18:20:01.569445+0000","last_became_peered":"2024-09-13T18:20:01.569445+0000","last_unstale":"2024-09-13T18:20:25.180438+0000","last_undegraded":"2024-09-13T18:20:25.180438+0000","last_fullsized":"2024-09-13T18:20:25.180438+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:45:20.733014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.179043+0000","last_change":"2024-09-13T18:20:06.136742+0000","last_active":"2024-09-13T18:20:26.179043+0000","last_peered":"2024-09-13T18:20:26.179043+0000","last_clean":"2024-09-13T18:20:26.179043+0000","last_became_active":"2024-09-13T18:20:06.136373+0000","last_became_peered":"2024-09-13T18:20:06.136373+0000","last_unstale":"2024-09-13T18:20:26.179043+0000","last_undegraded":"2024-09-13T18:20:26.179043+0000","last_fullsized":"2024-09-13T18:20:26.179043+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:49:11.782489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.176954+0000","last_change":"2024-09-13T18:20:01.922910+0000","last_active":"2024-09-13T18:20:06.176954+0000","last_peered":"2024-09-13T18:20:06.176954+0000","last_clean":"2024-09-13T18:20:06.176954+0000","last_became_active":"2024-09-13T18:20:01.921162+0000","last_became_peered":"2024-09-13T18:20:01.921162+0000","last_unstale":"2024-09-13T18:20:06.176954+0000","last_undegraded":"2024-09-13T18:20:06.176954+0000","last_fullsized":"2024-09-13T18:20:06.176954+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:14:47.243418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'3","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.197268+0000","last_change":"2024-09-13T18:20:01.575830+0000","last_active":"2024-09-13T18:20:26.197268+0000","last_peered":"2024-09-13T18:20:26.197268+0000","last_clean":"2024-09-13T18:20:26.197268+0000","last_became_active":"2024-09-13T18:20:01.575334+0000","last_became_peered":"2024-09-13T18:20:01.575334+0000","last_unstale":"2024-09-13T18:20:26.197268+0000","last_undegraded":"2024-09-13T18:20:26.197268+0000","last_fullsized":"2024-09-13T18:20:26.197268+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:13:00.935600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183501+0000","last_change":"2024-09-13T18:20:06.138609+0000","last_active":"2024-09-13T18:20:26.183501+0000","last_peered":"2024-09-13T18:20:26.183501+0000","last_clean":"2024-09-13T18:20:26.183501+0000","last_became_active":"2024-09-13T18:20:06.138232+0000","last_became_peered":"2024-09-13T18:20:06.138232+0000","last_unstale":"2024-09-13T18:20:26.183501+0000","last_undegraded":"2024-09-13T18:20:26.183501+0000","last_fullsized":"2024-09-13T18:20:26.183501+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:46:06.227407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.177024+0000","last_change":"2024-09-13T18:20:01.928757+0000","last_active":"2024-09-13T18:20:06.177024+0000","last_peered":"2024-09-13T18:20:06.177024+0000","last_clean":"2024-09-13T18:20:06.177024+0000","last_became_active":"2024-09-13T18:20:01.928350+0000","last_became_peered":"2024-09-13T18:20:01.928350+0000","last_unstale":"2024-09-13T18:20:06.177024+0000","last_undegraded":"2024-09-13T18:20:06.177024+0000","last_fullsized":"2024-09-13T18:20:06.177024+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:17:43.208106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:25.180579+0000","last_change":"2024-09-13T18:20:01.579128+0000","last_active":"2024-09-13T18:20:25.180579+0000","last_peered":"2024-09-13T18:20:25.180579+0000","last_clean":"2024-09-13T18:20:25.180579+0000","last_became_active":"2024-09-13T18:20:01.578796+0000","last_became_peered":"2024-09-13T18:20:01.578796+0000","last_unstale":"2024-09-13T18:20:25.180579+0000","last_undegraded":"2024-09-13T18:20:25.180579+0000","last_fullsized":"2024-09-13T18:20:25.180579+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:30:04.836103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.182950+0000","last_change":"2024-09-13T18:20:06.131404+0000","last_active":"2024-09-13T18:20:26.182950+0000","last_peered":"2024-09-13T18:20:26.182950+0000","last_clean":"2024-09-13T18:20:26.182950+0000","last_became_active":"2024-09-13T18:20:06.130931+0000","last_became_peered":"2024-09-13T18:20:06.130931+0000","last_unstale":"2024-09-13T18:20:26.182950+0000","last_undegraded":"2024-09-13T18:20:26.182950+0000","last_fullsized":"2024-09-13T18:20:26.182950+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:55:54.568816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.178724+0000","last_change":"2024-09-13T18:20:01.916089+0000","last_active":"2024-09-13T18:20:26.178724+0000","last_peered":"2024-09-13T18:20:26.178724+0000","last_clean":"2024-09-13T18:20:26.178724+0000","last_became_active":"2024-09-13T18:20:01.915783+0000","last_became_peered":"2024-09-13T18:20:01.915783+0000","last_unstale":"2024-09-13T18:20:26.178724+0000","last_undegraded":"2024-09-13T18:20:26.178724+0000","last_fullsized":"2024-09-13T18:20:26.178724+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:43:25.886029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"89'4","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:25.181228+0000","last_change":"2024-09-13T18:20:01.577381+0000","last_active":"2024-09-13T18:20:25.181228+0000","last_peered":"2024-09-13T18:20:25.181228+0000","last_clean":"2024-09-13T18:20:25.181228+0000","last_became_active":"2024-09-13T18:20:01.577044+0000","last_became_peered":"2024-09-13T18:20:01.577044+0000","last_unstale":"2024-09-13T18:20:25.181228+0000","last_undegraded":"2024-09-13T18:20:25.181228+0000","last_fullsized":"2024-09-13T18:20:25.181228+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:00.907948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.140300+0000","last_change":"2024-09-13T18:20:06.139971+0000","last_active":"2024-09-13T18:20:06.140300+0000","last_peered":"2024-09-13T18:20:06.140300+0000","last_clean":"2024-09-13T18:20:06.140300+0000","last_became_active":"2024-09-13T18:20:06.139507+0000","last_became_peered":"2024-09-13T18:20:06.139507+0000","last_unstale":"2024-09-13T18:20:06.140300+0000","last_undegraded":"2024-09-13T18:20:06.140300+0000","last_fullsized":"2024-09-13T18:20:06.140300+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:40:21.037768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.122321+0000","last_change":"2024-09-13T18:20:01.915348+0000","last_active":"2024-09-13T18:20:06.122321+0000","last_peered":"2024-09-13T18:20:06.122321+0000","last_clean":"2024-09-13T18:20:06.122321+0000","last_became_active":"2024-09-13T18:20:01.915042+0000","last_became_peered":"2024-09-13T18:20:01.915042+0000","last_unstale":"2024-09-13T18:20:06.122321+0000","last_undegraded":"2024-09-13T18:20:06.122321+0000","last_fullsized":"2024-09-13T18:20:06.122321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:31:33.712314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"91'17","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.256191+0000","last_change":"2024-09-13T18:20:01.578456+0000","last_active":"2024-09-13T18:20:28.256191+0000","last_peered":"2024-09-13T18:20:28.256191+0000","last_clean":"2024-09-13T18:20:28.256191+0000","last_became_active":"2024-09-13T18:20:01.578051+0000","last_became_peered":"2024-09-13T18:20:01.578051+0000","last_unstale":"2024-09-13T18:20:28.256191+0000","last_undegraded":"2024-09-13T18:20:28.256191+0000","last_fullsized":"2024-09-13T18:20:28.256191+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:34:17.651620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10396,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":12,"num_write":19,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183092+0000","last_change":"2024-09-13T18:20:06.131410+0000","last_active":"2024-09-13T18:20:26.183092+0000","last_peered":"2024-09-13T18:20:26.183092+0000","last_clean":"2024-09-13T18:20:26.183092+0000","last_became_active":"2024-09-13T18:20:06.131060+0000","last_became_peered":"2024-09-13T18:20:06.131060+0000","last_unstale":"2024-09-13T18:20:26.183092+0000","last_undegraded":"2024-09-13T18:20:26.183092+0000","last_fullsized":"2024-09-13T18:20:26.183092+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:08:25.266971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183935+0000","last_change":"2024-09-13T18:20:01.928839+0000","last_active":"2024-09-13T18:20:26.183935+0000","last_peered":"2024-09-13T18:20:26.183935+0000","last_clean":"2024-09-13T18:20:26.183935+0000","last_became_active":"2024-09-13T18:20:01.925774+0000","last_became_peered":"2024-09-13T18:20:01.925774+0000","last_unstale":"2024-09-13T18:20:26.183935+0000","last_undegraded":"2024-09-13T18:20:26.183935+0000","last_fullsized":"2024-09-13T18:20:26.183935+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:15:23.049671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.746467+0000","last_change":"2024-09-13T18:20:01.576332+0000","last_active":"2024-09-13T18:20:06.746467+0000","last_peered":"2024-09-13T18:20:06.746467+0000","last_clean":"2024-09-13T18:20:06.746467+0000","last_became_active":"2024-09-13T18:20:01.575862+0000","last_became_peered":"2024-09-13T18:20:01.575862+0000","last_unstale":"2024-09-13T18:20:06.746467+0000","last_undegraded":"2024-09-13T18:20:06.746467+0000","last_fullsized":"2024-09-13T18:20:06.746467+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:17:00.452480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.745244+0000","last_change":"2024-09-13T18:20:06.133142+0000","last_active":"2024-09-13T18:20:06.745244+0000","last_peered":"2024-09-13T18:20:06.745244+0000","last_clean":"2024-09-13T18:20:06.745244+0000","last_became_active":"2024-09-13T18:20:06.132740+0000","last_became_peered":"2024-09-13T18:20:06.132740+0000","last_unstale":"2024-09-13T18:20:06.745244+0000","last_undegraded":"2024-09-13T18:20:06.745244+0000","last_fullsized":"2024-09-13T18:20:06.745244+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:12:46.442481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.190799+0000","last_change":"2024-09-13T18:20:01.919673+0000","last_active":"2024-09-13T18:20:26.190799+0000","last_peered":"2024-09-13T18:20:26.190799+0000","last_clean":"2024-09-13T18:20:26.190799+0000","last_became_active":"2024-09-13T18:20:01.919358+0000","last_became_peered":"2024-09-13T18:20:01.919358+0000","last_unstale":"2024-09-13T18:20:26.190799+0000","last_undegraded":"2024-09-13T18:20:26.190799+0000","last_fullsized":"2024-09-13T18:20:26.190799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:40:11.407643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:25.181604+0000","last_change":"2024-09-13T18:20:01.575341+0000","last_active":"2024-09-13T18:20:25.181604+0000","last_peered":"2024-09-13T18:20:25.181604+0000","last_clean":"2024-09-13T18:20:25.181604+0000","last_became_active":"2024-09-13T18:20:01.575035+0000","last_became_peered":"2024-09-13T18:20:01.575035+0000","last_unstale":"2024-09-13T18:20:25.181604+0000","last_undegraded":"2024-09-13T18:20:25.181604+0000","last_fullsized":"2024-09-13T18:20:25.181604+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:24:29.891308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.182915+0000","last_change":"2024-09-13T18:20:06.133475+0000","last_active":"2024-09-13T18:20:26.182915+0000","last_peered":"2024-09-13T18:20:26.182915+0000","last_clean":"2024-09-13T18:20:26.182915+0000","last_became_active":"2024-09-13T18:20:06.133109+0000","last_became_peered":"2024-09-13T18:20:06.133109+0000","last_unstale":"2024-09-13T18:20:26.182915+0000","last_undegraded":"2024-09-13T18:20:26.182915+0000","last_fullsized":"2024-09-13T18:20:26.182915+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:47:19.567115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.184156+0000","last_change":"2024-09-13T18:20:01.922660+0000","last_active":"2024-09-13T18:20:26.184156+0000","last_peered":"2024-09-13T18:20:26.184156+0000","last_clean":"2024-09-13T18:20:26.184156+0000","last_became_active":"2024-09-13T18:20:01.922370+0000","last_became_peered":"2024-09-13T18:20:01.922370+0000","last_unstale":"2024-09-13T18:20:26.184156+0000","last_undegraded":"2024-09-13T18:20:26.184156+0000","last_fullsized":"2024-09-13T18:20:26.184156+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:54:32.029704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"91'4","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.249427+0000","last_change":"2024-09-13T18:20:01.580975+0000","last_active":"2024-09-13T18:20:28.249427+0000","last_peered":"2024-09-13T18:20:28.249427+0000","last_clean":"2024-09-13T18:20:28.249427+0000","last_became_active":"2024-09-13T18:20:01.580647+0000","last_became_peered":"2024-09-13T18:20:01.580647+0000","last_unstale":"2024-09-13T18:20:28.249427+0000","last_undegraded":"2024-09-13T18:20:28.249427+0000","last_fullsized":"2024-09-13T18:20:28.249427+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:26:20.942915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.192336+0000","last_change":"2024-09-13T18:20:06.140003+0000","last_active":"2024-09-13T18:20:26.192336+0000","last_peered":"2024-09-13T18:20:26.192336+0000","last_clean":"2024-09-13T18:20:26.192336+0000","last_became_active":"2024-09-13T18:20:06.139663+0000","last_became_peered":"2024-09-13T18:20:06.139663+0000","last_unstale":"2024-09-13T18:20:26.192336+0000","last_undegraded":"2024-09-13T18:20:26.192336+0000","last_fullsized":"2024-09-13T18:20:26.192336+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:54:49.662401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.189993+0000","last_change":"2024-09-13T18:20:06.134448+0000","last_active":"2024-09-13T18:20:26.189993+0000","last_peered":"2024-09-13T18:20:26.189993+0000","last_clean":"2024-09-13T18:20:26.189993+0000","last_became_active":"2024-09-13T18:20:06.134084+0000","last_became_peered":"2024-09-13T18:20:06.134084+0000","last_unstale":"2024-09-13T18:20:26.189993+0000","last_undegraded":"2024-09-13T18:20:26.189993+0000","last_fullsized":"2024-09-13T18:20:26.189993+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:55:50.475665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183131+0000","last_change":"2024-09-13T18:20:01.918955+0000","last_active":"2024-09-13T18:20:26.183131+0000","last_peered":"2024-09-13T18:20:26.183131+0000","last_clean":"2024-09-13T18:20:26.183131+0000","last_became_active":"2024-09-13T18:20:01.918666+0000","last_became_peered":"2024-09-13T18:20:01.918666+0000","last_unstale":"2024-09-13T18:20:26.183131+0000","last_undegraded":"2024-09-13T18:20:26.183131+0000","last_fullsized":"2024-09-13T18:20:26.183131+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:08:50.697710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.745892+0000","last_change":"2024-09-13T18:20:01.578789+0000","last_active":"2024-09-13T18:20:06.745892+0000","last_peered":"2024-09-13T18:20:06.745892+0000","last_clean":"2024-09-13T18:20:06.745892+0000","last_became_active":"2024-09-13T18:20:01.578476+0000","last_became_peered":"2024-09-13T18:20:01.578476+0000","last_unstale":"2024-09-13T18:20:06.745892+0000","last_undegraded":"2024-09-13T18:20:06.745892+0000","last_fullsized":"2024-09-13T18:20:06.745892+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:40:58.355492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183716+0000","last_change":"2024-09-13T18:20:06.129765+0000","last_active":"2024-09-13T18:20:26.183716+0000","last_peered":"2024-09-13T18:20:26.183716+0000","last_clean":"2024-09-13T18:20:26.183716+0000","last_became_active":"2024-09-13T18:20:06.129310+0000","last_became_peered":"2024-09-13T18:20:06.129310+0000","last_unstale":"2024-09-13T18:20:26.183716+0000","last_undegraded":"2024-09-13T18:20:26.183716+0000","last_fullsized":"2024-09-13T18:20:26.183716+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:34:07.394871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.191040+0000","last_change":"2024-09-13T18:20:01.914804+0000","last_active":"2024-09-13T18:20:26.191040+0000","last_peered":"2024-09-13T18:20:26.191040+0000","last_clean":"2024-09-13T18:20:26.191040+0000","last_became_active":"2024-09-13T18:20:01.914517+0000","last_became_peered":"2024-09-13T18:20:01.914517+0000","last_unstale":"2024-09-13T18:20:26.191040+0000","last_undegraded":"2024-09-13T18:20:26.191040+0000","last_fullsized":"2024-09-13T18:20:26.191040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:57:10.734935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"91'3","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:27.300317+0000","last_change":"2024-09-13T18:20:01.569201+0000","last_active":"2024-09-13T18:20:27.300317+0000","last_peered":"2024-09-13T18:20:27.300317+0000","last_clean":"2024-09-13T18:20:27.300317+0000","last_became_active":"2024-09-13T18:20:01.568908+0000","last_became_peered":"2024-09-13T18:20:01.568908+0000","last_unstale":"2024-09-13T18:20:27.300317+0000","last_undegraded":"2024-09-13T18:20:27.300317+0000","last_fullsized":"2024-09-13T18:20:27.300317+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:16:34.069130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"91'7","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.205826+0000","last_change":"2024-09-13T18:20:01.579225+0000","last_active":"2024-09-13T18:20:28.205826+0000","last_peered":"2024-09-13T18:20:28.205826+0000","last_clean":"2024-09-13T18:20:28.205826+0000","last_became_active":"2024-09-13T18:20:01.578826+0000","last_became_peered":"2024-09-13T18:20:01.578826+0000","last_unstale":"2024-09-13T18:20:28.205826+0000","last_undegraded":"2024-09-13T18:20:28.205826+0000","last_fullsized":"2024-09-13T18:20:28.205826+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:48:07.632239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.124287+0000","last_change":"2024-09-13T18:20:01.914885+0000","last_active":"2024-09-13T18:20:06.124287+0000","last_peered":"2024-09-13T18:20:06.124287+0000","last_clean":"2024-09-13T18:20:06.124287+0000","last_became_active":"2024-09-13T18:20:01.914599+0000","last_became_peered":"2024-09-13T18:20:01.914599+0000","last_unstale":"2024-09-13T18:20:06.124287+0000","last_undegraded":"2024-09-13T18:20:06.124287+0000","last_fullsized":"2024-09-13T18:20:06.124287+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:41:30.888675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.132751+0000","last_change":"2024-09-13T18:20:06.129376+0000","last_active":"2024-09-13T18:20:06.132751+0000","last_peered":"2024-09-13T18:20:06.132751+0000","last_clean":"2024-09-13T18:20:06.132751+0000","last_became_active":"2024-09-13T18:20:06.129032+0000","last_became_peered":"2024-09-13T18:20:06.129032+0000","last_unstale":"2024-09-13T18:20:06.132751+0000","last_undegraded":"2024-09-13T18:20:06.132751+0000","last_fullsized":"2024-09-13T18:20:06.132751+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:21:53.511813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:25.181451+0000","last_change":"2024-09-13T18:20:01.576069+0000","last_active":"2024-09-13T18:20:25.181451+0000","last_peered":"2024-09-13T18:20:25.181451+0000","last_clean":"2024-09-13T18:20:25.181451+0000","last_became_active":"2024-09-13T18:20:01.575760+0000","last_became_peered":"2024-09-13T18:20:01.575760+0000","last_unstale":"2024-09-13T18:20:25.181451+0000","last_undegraded":"2024-09-13T18:20:25.181451+0000","last_fullsized":"2024-09-13T18:20:25.181451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:05:47.677848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.122985+0000","last_change":"2024-09-13T18:20:01.917073+0000","last_active":"2024-09-13T18:20:06.122985+0000","last_peered":"2024-09-13T18:20:06.122985+0000","last_clean":"2024-09-13T18:20:06.122985+0000","last_became_active":"2024-09-13T18:20:01.916765+0000","last_became_peered":"2024-09-13T18:20:01.916765+0000","last_unstale":"2024-09-13T18:20:06.122985+0000","last_undegraded":"2024-09-13T18:20:06.122985+0000","last_fullsized":"2024-09-13T18:20:06.122985+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:22:07.788313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.133135+0000","last_change":"2024-09-13T18:20:06.131429+0000","last_active":"2024-09-13T18:20:06.133135+0000","last_peered":"2024-09-13T18:20:06.133135+0000","last_clean":"2024-09-13T18:20:06.133135+0000","last_became_active":"2024-09-13T18:20:06.131019+0000","last_became_peered":"2024-09-13T18:20:06.131019+0000","last_unstale":"2024-09-13T18:20:06.133135+0000","last_undegraded":"2024-09-13T18:20:06.133135+0000","last_fullsized":"2024-09-13T18:20:06.133135+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:44:37.892574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.182865+0000","last_change":"2024-09-13T18:20:01.914050+0000","last_active":"2024-09-13T18:20:26.182865+0000","last_peered":"2024-09-13T18:20:26.182865+0000","last_clean":"2024-09-13T18:20:26.182865+0000","last_became_active":"2024-09-13T18:20:01.913733+0000","last_became_peered":"2024-09-13T18:20:01.913733+0000","last_unstale":"2024-09-13T18:20:26.182865+0000","last_undegraded":"2024-09-13T18:20:26.182865+0000","last_fullsized":"2024-09-13T18:20:26.182865+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:52:43.672096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.123732+0000","last_change":"2024-09-13T18:20:01.924290+0000","last_active":"2024-09-13T18:20:06.123732+0000","last_peered":"2024-09-13T18:20:06.123732+0000","last_clean":"2024-09-13T18:20:06.123732+0000","last_became_active":"2024-09-13T18:20:01.923883+0000","last_became_peered":"2024-09-13T18:20:01.923883+0000","last_unstale":"2024-09-13T18:20:06.123732+0000","last_undegraded":"2024-09-13T18:20:06.123732+0000","last_fullsized":"2024-09-13T18:20:06.123732+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:11:18.737501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.125469+0000","last_change":"2024-09-13T18:20:01.928609+0000","last_active":"2024-09-13T18:20:06.125469+0000","last_peered":"2024-09-13T18:20:06.125469+0000","last_clean":"2024-09-13T18:20:06.125469+0000","last_became_active":"2024-09-13T18:20:01.927466+0000","last_became_peered":"2024-09-13T18:20:01.927466+0000","last_unstale":"2024-09-13T18:20:06.125469+0000","last_undegraded":"2024-09-13T18:20:06.125469+0000","last_fullsized":"2024-09-13T18:20:06.125469+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:18:58.610470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.189852+0000","last_change":"2024-09-13T18:20:01.915799+0000","last_active":"2024-09-13T18:20:26.189852+0000","last_peered":"2024-09-13T18:20:26.189852+0000","last_clean":"2024-09-13T18:20:26.189852+0000","last_became_active":"2024-09-13T18:20:01.915519+0000","last_became_peered":"2024-09-13T18:20:01.915519+0000","last_unstale":"2024-09-13T18:20:26.189852+0000","last_undegraded":"2024-09-13T18:20:26.189852+0000","last_fullsized":"2024-09-13T18:20:26.189852+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:08:26.065052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.178968+0000","last_change":"2024-09-13T18:20:01.918547+0000","last_active":"2024-09-13T18:20:26.178968+0000","last_peered":"2024-09-13T18:20:26.178968+0000","last_clean":"2024-09-13T18:20:26.178968+0000","last_became_active":"2024-09-13T18:20:01.918053+0000","last_became_peered":"2024-09-13T18:20:01.918053+0000","last_unstale":"2024-09-13T18:20:26.178968+0000","last_undegraded":"2024-09-13T18:20:26.178968+0000","last_fullsized":"2024-09-13T18:20:26.178968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:25:58.235101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183342+0000","last_change":"2024-09-13T18:20:01.916205+0000","last_active":"2024-09-13T18:20:26.183342+0000","last_peered":"2024-09-13T18:20:26.183342+0000","last_clean":"2024-09-13T18:20:26.183342+0000","last_became_active":"2024-09-13T18:20:01.915909+0000","last_became_peered":"2024-09-13T18:20:01.915909+0000","last_unstale":"2024-09-13T18:20:26.183342+0000","last_undegraded":"2024-09-13T18:20:26.183342+0000","last_fullsized":"2024-09-13T18:20:26.183342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:23:03.976513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.123579+0000","last_change":"2024-09-13T18:20:01.916621+0000","last_active":"2024-09-13T18:20:06.123579+0000","last_peered":"2024-09-13T18:20:06.123579+0000","last_clean":"2024-09-13T18:20:06.123579+0000","last_became_active":"2024-09-13T18:20:01.916335+0000","last_became_peered":"2024-09-13T18:20:01.916335+0000","last_unstale":"2024-09-13T18:20:06.123579+0000","last_undegraded":"2024-09-13T18:20:06.123579+0000","last_fullsized":"2024-09-13T18:20:06.123579+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:08:16.104571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.190215+0000","last_change":"2024-09-13T18:20:01.921195+0000","last_active":"2024-09-13T18:20:26.190215+0000","last_peered":"2024-09-13T18:20:26.190215+0000","last_clean":"2024-09-13T18:20:26.190215+0000","last_became_active":"2024-09-13T18:20:01.920895+0000","last_became_peered":"2024-09-13T18:20:01.920895+0000","last_unstale":"2024-09-13T18:20:26.190215+0000","last_undegraded":"2024-09-13T18:20:26.190215+0000","last_fullsized":"2024-09-13T18:20:26.190215+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:59:53.920047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.122232+0000","last_change":"2024-09-13T18:20:01.910889+0000","last_active":"2024-09-13T18:20:06.122232+0000","last_peered":"2024-09-13T18:20:06.122232+0000","last_clean":"2024-09-13T18:20:06.122232+0000","last_became_active":"2024-09-13T18:20:01.910451+0000","last_became_peered":"2024-09-13T18:20:01.910451+0000","last_unstale":"2024-09-13T18:20:06.122232+0000","last_undegraded":"2024-09-13T18:20:06.122232+0000","last_fullsized":"2024-09-13T18:20:06.122232+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:06:52.552785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.192420+0000","last_change":"2024-09-13T18:20:01.914041+0000","last_active":"2024-09-13T18:20:26.192420+0000","last_peered":"2024-09-13T18:20:26.192420+0000","last_clean":"2024-09-13T18:20:26.192420+0000","last_became_active":"2024-09-13T18:20:01.913240+0000","last_became_peered":"2024-09-13T18:20:01.913240+0000","last_unstale":"2024-09-13T18:20:26.192420+0000","last_undegraded":"2024-09-13T18:20:26.192420+0000","last_fullsized":"2024-09-13T18:20:26.192420+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:39:37.062544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.184154+0000","last_change":"2024-09-13T18:20:01.927322+0000","last_active":"2024-09-13T18:20:26.184154+0000","last_peered":"2024-09-13T18:20:26.184154+0000","last_clean":"2024-09-13T18:20:26.184154+0000","last_became_active":"2024-09-13T18:20:01.926767+0000","last_became_peered":"2024-09-13T18:20:01.926767+0000","last_unstale":"2024-09-13T18:20:26.184154+0000","last_undegraded":"2024-09-13T18:20:26.184154+0000","last_fullsized":"2024-09-13T18:20:26.184154+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:20.986089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.178873+0000","last_change":"2024-09-13T18:20:01.916314+0000","last_active":"2024-09-13T18:20:26.178873+0000","last_peered":"2024-09-13T18:20:26.178873+0000","last_clean":"2024-09-13T18:20:26.178873+0000","last_became_active":"2024-09-13T18:20:01.915790+0000","last_became_peered":"2024-09-13T18:20:01.915790+0000","last_unstale":"2024-09-13T18:20:26.178873+0000","last_undegraded":"2024-09-13T18:20:26.178873+0000","last_fullsized":"2024-09-13T18:20:26.178873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:20:30.063814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:27.187172+0000","last_change":"2024-09-13T18:20:27.187172+0000","last_active":"2024-09-13T18:20:27.187172+0000","last_peered":"2024-09-13T18:20:27.187172+0000","last_clean":"2024-09-13T18:20:27.187172+0000","last_became_active":"2024-09-13T18:20:27.186765+0000","last_became_peered":"2024-09-13T18:20:27.186765+0000","last_unstale":"2024-09-13T18:20:27.187172+0000","last_undegraded":"2024-09-13T18:20:27.187172+0000","last_fullsized":"2024-09-13T18:20:27.187172+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:37:01.469560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.124324+0000","last_change":"2024-09-13T18:20:01.924074+0000","last_active":"2024-09-13T18:20:06.124324+0000","last_peered":"2024-09-13T18:20:06.124324+0000","last_clean":"2024-09-13T18:20:06.124324+0000","last_became_active":"2024-09-13T18:20:01.922364+0000","last_became_peered":"2024-09-13T18:20:01.922364+0000","last_unstale":"2024-09-13T18:20:06.124324+0000","last_undegraded":"2024-09-13T18:20:06.124324+0000","last_fullsized":"2024-09-13T18:20:06.124324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:51:33.824362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183604+0000","last_change":"2024-09-13T18:20:01.914606+0000","last_active":"2024-09-13T18:20:26.183604+0000","last_peered":"2024-09-13T18:20:26.183604+0000","last_clean":"2024-09-13T18:20:26.183604+0000","last_became_active":"2024-09-13T18:20:01.914281+0000","last_became_peered":"2024-09-13T18:20:01.914281+0000","last_unstale":"2024-09-13T18:20:26.183604+0000","last_undegraded":"2024-09-13T18:20:26.183604+0000","last_fullsized":"2024-09-13T18:20:26.183604+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:55:42.211959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.184054+0000","last_change":"2024-09-13T18:20:01.928876+0000","last_active":"2024-09-13T18:20:26.184054+0000","last_peered":"2024-09-13T18:20:26.184054+0000","last_clean":"2024-09-13T18:20:26.184054+0000","last_became_active":"2024-09-13T18:20:01.928534+0000","last_became_peered":"2024-09-13T18:20:01.928534+0000","last_unstale":"2024-09-13T18:20:26.184054+0000","last_undegraded":"2024-09-13T18:20:26.184054+0000","last_fullsized":"2024-09-13T18:20:26.184054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:14:28.196746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183517+0000","last_change":"2024-09-13T18:20:01.920642+0000","last_active":"2024-09-13T18:20:26.183517+0000","last_peered":"2024-09-13T18:20:26.183517+0000","last_clean":"2024-09-13T18:20:26.183517+0000","last_became_active":"2024-09-13T18:20:01.920335+0000","last_became_peered":"2024-09-13T18:20:01.920335+0000","last_unstale":"2024-09-13T18:20:26.183517+0000","last_undegraded":"2024-09-13T18:20:26.183517+0000","last_fullsized":"2024-09-13T18:20:26.183517+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:59:20.894563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:27.187529+0000","last_change":"2024-09-13T18:20:27.187529+0000","last_active":"2024-09-13T18:20:27.187529+0000","last_peered":"2024-09-13T18:20:27.187529+0000","last_clean":"2024-09-13T18:20:27.187529+0000","last_became_active":"2024-09-13T18:20:27.187123+0000","last_became_peered":"2024-09-13T18:20:27.187123+0000","last_unstale":"2024-09-13T18:20:27.187529+0000","last_undegraded":"2024-09-13T18:20:27.187529+0000","last_fullsized":"2024-09-13T18:20:27.187529+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:43:19.601235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.189462+0000","last_change":"2024-09-13T18:20:01.917111+0000","last_active":"2024-09-13T18:20:26.189462+0000","last_peered":"2024-09-13T18:20:26.189462+0000","last_clean":"2024-09-13T18:20:26.189462+0000","last_became_active":"2024-09-13T18:20:01.916826+0000","last_became_peered":"2024-09-13T18:20:01.916826+0000","last_unstale":"2024-09-13T18:20:26.189462+0000","last_undegraded":"2024-09-13T18:20:26.189462+0000","last_fullsized":"2024-09-13T18:20:26.189462+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:36:08.150028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.178921+0000","last_change":"2024-09-13T18:20:01.904496+0000","last_active":"2024-09-13T18:20:26.178921+0000","last_peered":"2024-09-13T18:20:26.178921+0000","last_clean":"2024-09-13T18:20:26.178921+0000","last_became_active":"2024-09-13T18:20:01.904199+0000","last_became_peered":"2024-09-13T18:20:01.904199+0000","last_unstale":"2024-09-13T18:20:26.178921+0000","last_undegraded":"2024-09-13T18:20:26.178921+0000","last_fullsized":"2024-09-13T18:20:26.178921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:44.044162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.745431+0000","last_change":"2024-09-13T18:20:01.919023+0000","last_active":"2024-09-13T18:20:06.745431+0000","last_peered":"2024-09-13T18:20:06.745431+0000","last_clean":"2024-09-13T18:20:06.745431+0000","last_became_active":"2024-09-13T18:20:01.918661+0000","last_became_peered":"2024-09-13T18:20:01.918661+0000","last_unstale":"2024-09-13T18:20:06.745431+0000","last_undegraded":"2024-09-13T18:20:06.745431+0000","last_fullsized":"2024-09-13T18:20:06.745431+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:25:26.779029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.177051+0000","last_change":"2024-09-13T18:20:01.922833+0000","last_active":"2024-09-13T18:20:06.177051+0000","last_peered":"2024-09-13T18:20:06.177051+0000","last_clean":"2024-09-13T18:20:06.177051+0000","last_became_active":"2024-09-13T18:20:01.922533+0000","last_became_peered":"2024-09-13T18:20:01.922533+0000","last_unstale":"2024-09-13T18:20:06.177051+0000","last_undegraded":"2024-09-13T18:20:06.177051+0000","last_fullsized":"2024-09-13T18:20:06.177051+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:27:11.557419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.197907+0000","last_change":"2024-09-13T18:20:01.925916+0000","last_active":"2024-09-13T18:20:26.197907+0000","last_peered":"2024-09-13T18:20:26.197907+0000","last_clean":"2024-09-13T18:20:26.197907+0000","last_became_active":"2024-09-13T18:20:01.925568+0000","last_became_peered":"2024-09-13T18:20:01.925568+0000","last_unstale":"2024-09-13T18:20:26.197907+0000","last_undegraded":"2024-09-13T18:20:26.197907+0000","last_fullsized":"2024-09-13T18:20:26.197907+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:58:24.955273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.184513+0000","last_change":"2024-09-13T18:20:01.924328+0000","last_active":"2024-09-13T18:20:26.184513+0000","last_peered":"2024-09-13T18:20:26.184513+0000","last_clean":"2024-09-13T18:20:26.184513+0000","last_became_active":"2024-09-13T18:20:01.924004+0000","last_became_peered":"2024-09-13T18:20:01.924004+0000","last_unstale":"2024-09-13T18:20:26.184513+0000","last_undegraded":"2024-09-13T18:20:26.184513+0000","last_fullsized":"2024-09-13T18:20:26.184513+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:23:29.204157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.190973+0000","last_change":"2024-09-13T18:20:01.922234+0000","last_active":"2024-09-13T18:20:26.190973+0000","last_peered":"2024-09-13T18:20:26.190973+0000","last_clean":"2024-09-13T18:20:26.190973+0000","last_became_active":"2024-09-13T18:20:01.921196+0000","last_became_peered":"2024-09-13T18:20:01.921196+0000","last_unstale":"2024-09-13T18:20:26.190973+0000","last_undegraded":"2024-09-13T18:20:26.190973+0000","last_fullsized":"2024-09-13T18:20:26.190973+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:24:10.144346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.176993+0000","last_change":"2024-09-13T18:20:01.921740+0000","last_active":"2024-09-13T18:20:06.176993+0000","last_peered":"2024-09-13T18:20:06.176993+0000","last_clean":"2024-09-13T18:20:06.176993+0000","last_became_active":"2024-09-13T18:20:01.919634+0000","last_became_peered":"2024-09-13T18:20:01.919634+0000","last_unstale":"2024-09-13T18:20:06.176993+0000","last_undegraded":"2024-09-13T18:20:06.176993+0000","last_fullsized":"2024-09-13T18:20:06.176993+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:43:27.600637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.197426+0000","last_change":"2024-09-13T18:20:01.924806+0000","last_active":"2024-09-13T18:20:26.197426+0000","last_peered":"2024-09-13T18:20:26.197426+0000","last_clean":"2024-09-13T18:20:26.197426+0000","last_became_active":"2024-09-13T18:20:01.924514+0000","last_became_peered":"2024-09-13T18:20:01.924514+0000","last_unstale":"2024-09-13T18:20:26.197426+0000","last_undegraded":"2024-09-13T18:20:26.197426+0000","last_fullsized":"2024-09-13T18:20:26.197426+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:47.323362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.122321+0000","last_change":"2024-09-13T18:20:01.912402+0000","last_active":"2024-09-13T18:20:06.122321+0000","last_peered":"2024-09-13T18:20:06.122321+0000","last_clean":"2024-09-13T18:20:06.122321+0000","last_became_active":"2024-09-13T18:20:01.911534+0000","last_became_peered":"2024-09-13T18:20:01.911534+0000","last_unstale":"2024-09-13T18:20:06.122321+0000","last_undegraded":"2024-09-13T18:20:06.122321+0000","last_fullsized":"2024-09-13T18:20:06.122321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:22:52.021688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.745369+0000","last_change":"2024-09-13T18:20:01.930248+0000","last_active":"2024-09-13T18:20:06.745369+0000","last_peered":"2024-09-13T18:20:06.745369+0000","last_clean":"2024-09-13T18:20:06.745369+0000","last_became_active":"2024-09-13T18:20:01.929931+0000","last_became_peered":"2024-09-13T18:20:01.929931+0000","last_unstale":"2024-09-13T18:20:06.745369+0000","last_undegraded":"2024-09-13T18:20:06.745369+0000","last_fullsized":"2024-09-13T18:20:06.745369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:37:33.086722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.190113+0000","last_change":"2024-09-13T18:20:01.921913+0000","last_active":"2024-09-13T18:20:26.190113+0000","last_peered":"2024-09-13T18:20:26.190113+0000","last_clean":"2024-09-13T18:20:26.190113+0000","last_became_active":"2024-09-13T18:20:01.921395+0000","last_became_peered":"2024-09-13T18:20:01.921395+0000","last_unstale":"2024-09-13T18:20:26.190113+0000","last_undegraded":"2024-09-13T18:20:26.190113+0000","last_fullsized":"2024-09-13T18:20:26.190113+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:54:50.816870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.190140+0000","last_change":"2024-09-13T18:20:01.912654+0000","last_active":"2024-09-13T18:20:26.190140+0000","last_peered":"2024-09-13T18:20:26.190140+0000","last_clean":"2024-09-13T18:20:26.190140+0000","last_became_active":"2024-09-13T18:20:01.912356+0000","last_became_peered":"2024-09-13T18:20:01.912356+0000","last_unstale":"2024-09-13T18:20:26.190140+0000","last_undegraded":"2024-09-13T18:20:26.190140+0000","last_fullsized":"2024-09-13T18:20:26.190140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:25:18.099396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183955+0000","last_change":"2024-09-13T18:20:01.923373+0000","last_active":"2024-09-13T18:20:26.183955+0000","last_peered":"2024-09-13T18:20:26.183955+0000","last_clean":"2024-09-13T18:20:26.183955+0000","last_became_active":"2024-09-13T18:20:01.923077+0000","last_became_peered":"2024-09-13T18:20:01.923077+0000","last_unstale":"2024-09-13T18:20:26.183955+0000","last_undegraded":"2024-09-13T18:20:26.183955+0000","last_fullsized":"2024-09-13T18:20:26.183955+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:07:52.955705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":13418,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":9,"num_read_kb":17,"num_write":139,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":303104,"data_stored":36667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281684,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383698,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":995328,"data_stored":495945,"data_compressed":17567,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.47599999999999998}]},{"osd":1,"last update":"Fri Sep 13 18:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.58499999999999996}]},{"osd":2,"last update":"Fri Sep 13 18:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.622}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":4,"last update":"Fri Sep 13 18:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.64100000000000001}]},{"osd":5,"last update":"Fri Sep 13 18:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.69699999999999995}]},{"osd":6,"last update":"Fri Sep 13 18:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.56699999999999995}]},{"osd":7,"last update":"Fri Sep 13 18:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.68100000000000005}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":9,"last update":"Fri Sep 13 18:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.84999999999999998}]},{"osd":10,"last update":"Fri Sep 13 18:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.751}]}]},{"osd":10,"up_from":72,"seq":309237645335,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1077248,"data_stored":1090703,"data_compressed":53391,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":1,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.66700000000000004}]},{"osd":2,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.625}]},{"osd":3,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.60999999999999999}]},{"osd":4,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.64500000000000002}]},{"osd":5,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.56599999999999995}]},{"osd":6,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.76100000000000001}]},{"osd":7,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.81599999999999995}]},{"osd":8,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.80400000000000005}]},{"osd":9,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.83399999999999996}]},{"osd":11,"last update":"Fri Sep 13 18:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.872}]}]},{"osd":9,"up_from":67,"seq":287762808860,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":500559,"data_compressed":17567,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.311}]},{"osd":1,"last update":"Fri Sep 13 18:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.5}]},{"osd":2,"last update":"Fri Sep 13 18:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.53900000000000003}]},{"osd":3,"last update":"Fri Sep 13 18:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.34300000000000003}]},{"osd":4,"last update":"Fri Sep 13 18:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.57599999999999996}]},{"osd":5,"last update":"Fri Sep 13 18:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.55400000000000005}]},{"osd":6,"last update":"Fri Sep 13 18:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.39800000000000002}]},{"osd":7,"last update":"Fri Sep 13 18:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.47999999999999998}]},{"osd":8,"last update":"Fri Sep 13 18:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.624}]},{"osd":10,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.63900000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]}]},{"osd":8,"up_from":59,"seq":253403070496,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":974848,"data_stored":477189,"data_compressed":11642,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.53600000000000003,"15min":0.54800000000000004},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.60199999999999998,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.54900000000000004,"15min":0.56200000000000006},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.68600000000000005,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.53500000000000003}]},{"osd":1,"last update":"Fri Sep 13 18:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56000000000000005,"15min":0.55900000000000005},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56999999999999995,"15min":0.57099999999999995},"min":{"1min":0.40600000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72199999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.73699999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.42599999999999999,"15min":0.42999999999999999},"min":{"1min":0.26600000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.55200000000000005,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.247,"5min":0.224,"15min":0.224},"max":{"1min":0.59799999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.48399999999999999}]},{"osd":3,"last update":"Fri Sep 13 18:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.54700000000000004,"15min":0.55800000000000005},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.71999999999999997,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53800000000000003,"15min":0.53900000000000003},"min":{"1min":0.38600000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.44600000000000001}]},{"osd":4,"last update":"Fri Sep 13 18:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.66400000000000003,"15min":0.63600000000000001},"min":{"1min":0.45100000000000001,"5min":0.37,"15min":0.37},"max":{"1min":5.3540000000000001,"5min":5.3540000000000001,"15min":5.3540000000000001},"last":0.68000000000000005}]},{"osd":5,"last update":"Fri Sep 13 18:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.53100000000000003,"15min":0.53700000000000003},"min":{"1min":0.31900000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.70099999999999996,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.51900000000000002,"15min":0.52300000000000002},"min":{"1min":0.32100000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.63500000000000001,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.40100000000000002}]},{"osd":6,"last update":"Fri Sep 13 18:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.60599999999999998,"15min":0.61199999999999999},"min":{"1min":0.35199999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.75600000000000001,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.64400000000000002,"15min":0.65000000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.77200000000000002,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.65800000000000003}]},{"osd":7,"last update":"Fri Sep 13 18:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.66600000000000004,"15min":0.67500000000000004},"min":{"1min":0.44800000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.749,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.65600000000000003,"15min":0.66300000000000003},"min":{"1min":0.39700000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.73899999999999999,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.62}]},{"osd":9,"last update":"Fri Sep 13 18:20:24 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63800000000000001,"15min":0.63300000000000001},"min":{"1min":0.46600000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.65900000000000003,"15min":0.65600000000000003},"min":{"1min":0.52800000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.95399999999999996,"5min":1.012,"15min":1.012},"last":0.57899999999999996}]},{"osd":10,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":5.3360000000000003,"5min":5.3360000000000003,"15min":5.3360000000000003},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.72499999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]}]},{"osd":7,"up_from":51,"seq":219043332132,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":1024000,"data_stored":1068087,"data_compressed":47466,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.94399999999999995,"15min":1.012},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.84299999999999997,"5min":12.068,"15min":12.068},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.46999999999999997,"15min":0.45900000000000002},"min":{"1min":0.35199999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.61599999999999999}]},{"osd":1,"last update":"Fri Sep 13 18:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.36499999999999999,"15min":0.35699999999999998},"min":{"1min":0.26100000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.38100000000000001,"15min":0.375},"min":{"1min":0.27800000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.62,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.44900000000000001}]},{"osd":2,"last update":"Fri Sep 13 18:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.59099999999999997,"15min":0.53700000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":7.6820000000000004,"5min":7.6820000000000004,"15min":7.6820000000000004},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.51600000000000001,"15min":0.50700000000000001},"min":{"1min":0.378,"5min":0.371,"15min":0.371},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.58799999999999997}]},{"osd":3,"last update":"Fri Sep 13 18:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.49299999999999999,"15min":0.48099999999999998},"min":{"1min":0.36299999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.754},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.49299999999999999,"15min":0.48299999999999998},"min":{"1min":0.35399999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.63100000000000001}]},{"osd":4,"last update":"Fri Sep 13 18:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.44,"15min":0.433},"min":{"1min":0.29099999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.621,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.45400000000000001,"15min":0.45000000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.439}]},{"osd":5,"last update":"Fri Sep 13 18:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.55700000000000005,"15min":0.55000000000000004},"min":{"1min":0.39900000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.53700000000000003,"15min":0.53100000000000003},"min":{"1min":0.39900000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.68200000000000005}]},{"osd":6,"last update":"Fri Sep 13 18:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.55000000000000004,"15min":0.54000000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.59299999999999997,"15min":0.58199999999999996},"min":{"1min":0.48099999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.77700000000000002}]},{"osd":8,"last update":"Fri Sep 13 18:20:01 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.58999999999999997,"15min":0.58699999999999997},"min":{"1min":0.42499999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.72999999999999998,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.76600000000000001},{"interface":"front","average":{"1min":1.0669999999999999,"5min":0.71899999999999997,"15min":0.66100000000000003},"min":{"1min":0.443,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":7.7149999999999999,"5min":7.7149999999999999,"15min":7.7149999999999999},"last":0.71999999999999997}]},{"osd":9,"last update":"Fri Sep 13 18:19:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.84599999999999997}]},{"osd":10,"last update":"Fri Sep 13 18:19:50 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.64600000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]}]},{"osd":6,"up_from":46,"seq":197568495658,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":1003520,"data_stored":496015,"data_compressed":17567,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.248,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.43099999999999999,"15min":0.437},"min":{"1min":0.27600000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.218}]},{"osd":1,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.52700000000000002,"15min":0.52200000000000002},"min":{"1min":0.35499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.69199999999999995,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55000000000000004,"15min":0.53600000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.66500000000000004}]},{"osd":2,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.54200000000000004,"15min":0.54600000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53600000000000003,"15min":0.54700000000000004},"min":{"1min":0.36599999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.75,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.52000000000000002}]},{"osd":3,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42299999999999999,"15min":0.42599999999999999},"min":{"1min":0.28999999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.69499999999999995,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.309},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.46700000000000003,"15min":0.47099999999999997},"min":{"1min":0.30099999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.35599999999999998}]},{"osd":4,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.52300000000000002,"15min":0.51500000000000001},"min":{"1min":0.33300000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55500000000000005,"15min":0.55300000000000005},"min":{"1min":0.39600000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.74199999999999999,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.68300000000000005}]},{"osd":5,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59899999999999998,"15min":0.60799999999999998},"min":{"1min":0.436,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.57399999999999995,"15min":0.59299999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.77900000000000003,"5min":0.874,"15min":0.874},"last":0.496}]},{"osd":7,"last update":"Fri Sep 13 18:19:43 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57399999999999995,"15min":0.57299999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.71299999999999997,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56599999999999995,"15min":0.56200000000000006},"min":{"1min":0.435,"5min":0.434,"15min":0.434},"max":{"1min":0.77100000000000002,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.56899999999999995}]},{"osd":8,"last update":"Fri Sep 13 18:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.61799999999999999,"15min":0.621},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.79000000000000004,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.59399999999999997,"15min":0.59899999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.82999999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.65200000000000002}]},{"osd":9,"last update":"Fri Sep 13 18:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.39500000000000002}]},{"osd":10,"last update":"Fri Sep 13 18:19:49 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.58199999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]}]},{"osd":5,"up_from":38,"seq":163208757294,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":1068359,"data_compressed":47466,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:19:58 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.49299999999999999,"15min":0.49399999999999999},"min":{"1min":0.35299999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.61099999999999999,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.51400000000000001,"15min":0.51200000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.66600000000000004,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.52800000000000002}]},{"osd":1,"last update":"Fri Sep 13 18:19:58 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.51600000000000001,"15min":0.502},"min":{"1min":0.41899999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.77200000000000002,"5min":0.878,"15min":0.878},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.52000000000000002,"15min":0.51000000000000001},"min":{"1min":0.34399999999999997,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.78600000000000003,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.59199999999999997}]},{"osd":2,"last update":"Fri Sep 13 18:19:58 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.376,"15min":0.36199999999999999},"min":{"1min":0.27700000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.39000000000000001,"15min":0.38400000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.58499999999999996,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.41199999999999998}]},{"osd":3,"last update":"Fri Sep 13 18:19:58 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.51600000000000001,"15min":0.501},"min":{"1min":0.35799999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.51500000000000001,"15min":0.51800000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.70499999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.505}]},{"osd":4,"last update":"Fri Sep 13 18:19:58 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.54700000000000004,"15min":0.53800000000000003},"min":{"1min":0.40400000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.75900000000000001,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.76400000000000001,"15min":0.60799999999999998},"min":{"1min":0.376,"5min":0.374,"15min":0.374},"max":{"1min":0.80200000000000005,"5min":18.571000000000002,"15min":18.571000000000002},"last":0.48199999999999998}]},{"osd":6,"last update":"Fri Sep 13 18:20:22 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.42399999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55400000000000005,"15min":0.55000000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.629}]},{"osd":7,"last update":"Fri Sep 13 18:19:43 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.61499999999999999,"15min":0.60799999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.61699999999999999,"15min":0.61099999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.85299999999999998,"5min":0.88,"15min":0.88},"last":0.67600000000000005}]},{"osd":8,"last update":"Fri Sep 13 18:20:02 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.51700000000000002,"15min":0.51300000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.73699999999999999,"5min":0.753,"15min":0.753},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.54300000000000004,"15min":0.53700000000000003},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.69099999999999995}]},{"osd":9,"last update":"Fri Sep 13 18:19:27 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.70199999999999996}]},{"osd":10,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.72499999999999998}]}]},{"osd":4,"up_from":33,"seq":141733920818,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":486669,"data_compressed":14601,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:19:33 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.48299999999999998,"15min":0.47699999999999998},"min":{"1min":0.35399999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.69399999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.433},{"interface":"front","average":{"1min":0.501,"5min":0.51200000000000001,"15min":0.47499999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.84599999999999997,"5min":4.2539999999999996,"15min":4.2539999999999996},"last":0.51900000000000002}]},{"osd":1,"last update":"Fri Sep 13 18:19:33 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.34899999999999998,"15min":0.34399999999999997},"min":{"1min":0.27600000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.372,"5min":0.34699999999999998,"15min":0.34300000000000003},"min":{"1min":0.25600000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.52700000000000002,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.42099999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:19:33 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.53100000000000003,"15min":0.52300000000000002},"min":{"1min":0.40999999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.53700000000000003,"15min":0.53300000000000003},"min":{"1min":0.38500000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.74399999999999999}]},{"osd":3,"last update":"Fri Sep 13 18:19:33 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.49299999999999999},"min":{"1min":0.35199999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.72599999999999998,"5min":4.2679999999999998,"15min":4.2679999999999998},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.48899999999999999,"15min":0.48499999999999999},"min":{"1min":0.40200000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.69299999999999995}]},{"osd":5,"last update":"Fri Sep 13 18:19:55 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.54800000000000004,"15min":0.53500000000000003},"min":{"1min":0.38500000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78100000000000003,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.55000000000000004,"15min":0.53700000000000003},"min":{"1min":0.39900000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.71099999999999997,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.59599999999999997}]},{"osd":6,"last update":"Fri Sep 13 18:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.52300000000000002,"15min":0.51100000000000001},"min":{"1min":0.41999999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.52300000000000002,"15min":0.50900000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.76600000000000001}]},{"osd":7,"last update":"Fri Sep 13 18:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.44400000000000001,"15min":0.438},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.48499999999999999,"15min":0.47699999999999998},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.48899999999999999}]},{"osd":8,"last update":"Fri Sep 13 18:20:07 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.63,"15min":0.626},"min":{"1min":0.505,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.85899999999999999,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.65000000000000002,"15min":0.64500000000000002},"min":{"1min":0.52300000000000002,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.70599999999999996}]},{"osd":9,"last update":"Fri Sep 13 18:19:25 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.77900000000000003}]},{"osd":10,"last update":"Fri Sep 13 18:19:51 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.625},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.63900000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]}]},{"osd":3,"up_from":25,"seq":107374182455,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":495344,"data_compressed":17567,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:20:10 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.32900000000000001,"15min":0.307},"min":{"1min":0.248,"5min":0.183,"15min":0.183},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.33900000000000002,"15min":0.32700000000000001},"min":{"1min":0.222,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.47599999999999998,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.314}]},{"osd":1,"last update":"Fri Sep 13 18:20:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.50900000000000001,"15min":0.48799999999999999},"min":{"1min":0.40000000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.65600000000000003,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.51400000000000001,"15min":0.48799999999999999},"min":{"1min":0.39400000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.71299999999999997,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.61599999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:20:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52400000000000002,"15min":0.52200000000000002},"min":{"1min":0.35999999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.68700000000000006,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52300000000000002,"15min":0.501},"min":{"1min":0.36599999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.73399999999999999,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.45800000000000002}]},{"osd":4,"last update":"Fri Sep 13 18:19:32 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.5,"15min":0.48599999999999999},"min":{"1min":0.374,"5min":0.312,"15min":0.312},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.501},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.52400000000000002,"15min":0.51400000000000001},"min":{"1min":0.372,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.749,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.433}]},{"osd":5,"last update":"Fri Sep 13 18:19:54 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54000000000000004,"15min":0.53500000000000003},"min":{"1min":0.38300000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.749,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.39600000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72899999999999998,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.51500000000000001}]},{"osd":6,"last update":"Fri Sep 13 18:20:23 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.443,"15min":0.44700000000000001},"min":{"1min":0.30299999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.59199999999999997,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.48099999999999998,"15min":0.46700000000000003},"min":{"1min":0.313,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.70699999999999996,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.443}]},{"osd":7,"last update":"Fri Sep 13 18:19:39 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.80400000000000005,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.57999999999999996,"15min":0.57399999999999995},"min":{"1min":0.44800000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.77500000000000002,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.60899999999999999}]},{"osd":8,"last update":"Fri Sep 13 18:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.69599999999999995}]},{"osd":9,"last update":"Fri Sep 13 18:19:27 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.55900000000000005}]},{"osd":10,"last update":"Fri Sep 13 18:19:49 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.626}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]}]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":480392,"data_compressed":11642,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:19:45 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47399999999999998,"15min":0.47899999999999998},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.57599999999999996,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.495},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.48099999999999998,"15min":0.47599999999999998},"min":{"1min":0.318,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.61899999999999999}]},{"osd":1,"last update":"Fri Sep 13 18:19:45 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.49199999999999999,"15min":0.47199999999999998},"min":{"1min":0.34999999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.78400000000000003,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.49099999999999999,"15min":0.46500000000000002},"min":{"1min":0.32300000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.78100000000000003,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.53600000000000003}]},{"osd":3,"last update":"Fri Sep 13 18:20:13 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.52700000000000002,"15min":0.51600000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.66100000000000003,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.55200000000000005,"15min":0.55100000000000005},"min":{"1min":0.377,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.67100000000000004,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.75700000000000001}]},{"osd":4,"last update":"Fri Sep 13 18:19:35 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.504,"15min":0.49099999999999999},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.72899999999999998,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.50900000000000001,"15min":0.503},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.68899999999999995,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.54700000000000004}]},{"osd":5,"last update":"Fri Sep 13 18:19:52 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.38400000000000001,"15min":0.375},"min":{"1min":0.30399999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.371},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.40000000000000002,"15min":0.39000000000000001},"min":{"1min":0.26900000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.58899999999999997,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.31900000000000001}]},{"osd":6,"last update":"Fri Sep 13 18:20:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.56299999999999994,"15min":0.54000000000000004},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.57799999999999996,"15min":0.56299999999999994},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.80100000000000005,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.70199999999999996}]},{"osd":7,"last update":"Fri Sep 13 18:19:37 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78200000000000003,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.61599999999999999,"15min":0.61899999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.76600000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.52300000000000002}]},{"osd":8,"last update":"Fri Sep 13 18:20:07 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.54000000000000004,"15min":0.54200000000000004},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.73899999999999999,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.36599999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.44700000000000001}]},{"osd":9,"last update":"Fri Sep 13 18:19:25 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.56399999999999995}]},{"osd":10,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.57399999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]}]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":1007616,"data_stored":499248,"data_compressed":17567,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:20:24 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.502,"15min":0.495},"min":{"1min":0.35499999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.626},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.52300000000000002,"15min":0.51500000000000001},"min":{"1min":0.33200000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.77800000000000002,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.439}]},{"osd":2,"last update":"Fri Sep 13 18:19:46 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.501,"15min":0.48999999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.46999999999999997,"15min":0.46899999999999997},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.64200000000000002,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.496}]},{"osd":3,"last update":"Fri Sep 13 18:20:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.53900000000000003,"15min":0.52100000000000002},"min":{"1min":0.38600000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.74399999999999999,"5min":0.749,"15min":0.749},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56499999999999995,"15min":0.54400000000000004},"min":{"1min":0.39700000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76900000000000002,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.52700000000000002}]},{"osd":4,"last update":"Fri Sep 13 18:19:32 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.38,"15min":0.36099999999999999},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.434,"5min":0.376,"15min":0.35599999999999998},"min":{"1min":0.28399999999999997,"5min":0.217,"15min":0.217},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.40500000000000003}]},{"osd":5,"last update":"Fri Sep 13 18:19:53 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56699999999999995,"15min":0.56200000000000006},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55100000000000005,"15min":0.53700000000000003},"min":{"1min":0.40500000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.71299999999999997}]},{"osd":6,"last update":"Fri Sep 13 18:20:23 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57699999999999996,"15min":0.56699999999999995},"min":{"1min":0.41299999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.621,"15min":0.61099999999999999},"min":{"1min":0.45100000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.88300000000000001,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.72899999999999998}]},{"osd":7,"last update":"Fri Sep 13 18:19:38 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.49299999999999999,"15min":0.48699999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.502,"15min":0.5},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.745}]},{"osd":8,"last update":"Fri Sep 13 18:20:07 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.627,"15min":0.629},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.66000000000000003}]},{"osd":9,"last update":"Fri Sep 13 18:19:27 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.70099999999999996}]},{"osd":10,"last update":"Fri Sep 13 18:19:49 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.76600000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":995328,"data_stored":487439,"data_compressed":14601,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 18:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.501,"15min":0.496},"min":{"1min":0.35599999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.37},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.50700000000000001,"15min":0.505},"min":{"1min":0.35699999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.69199999999999995,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.504}]},{"osd":2,"last update":"Fri Sep 13 18:19:47 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.52900000000000003,"15min":0.52700000000000002},"min":{"1min":0.34399999999999997,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.70099999999999996,"5min":0.754,"15min":0.754},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.53000000000000003,"15min":0.51100000000000001},"min":{"1min":0.38500000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.81799999999999995,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.47699999999999998}]},{"osd":3,"last update":"Fri Sep 13 18:19:05 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.38400000000000001,"15min":0.378},"min":{"1min":0.26300000000000001,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.437,"5min":0.39400000000000002,"15min":0.376},"min":{"1min":0.28100000000000003,"5min":0.251,"15min":0.251},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.28799999999999998}]},{"osd":4,"last update":"Fri Sep 13 18:19:33 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.503,"15min":0.49099999999999999},"min":{"1min":0.38600000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.85699999999999998,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.51900000000000002,"15min":0.50700000000000001},"min":{"1min":0.35499999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.45100000000000001}]},{"osd":5,"last update":"Fri Sep 13 18:19:53 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.61499999999999999,"15min":0.56699999999999995},"min":{"1min":0.371,"5min":0.313,"15min":0.313},"max":{"1min":0.89600000000000002,"5min":7.194,"15min":7.194},"last":0.495},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.55100000000000005,"15min":0.52300000000000002},"min":{"1min":0.378,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.87,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.53900000000000003}]},{"osd":6,"last update":"Fri Sep 13 18:20:21 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.46400000000000002,"15min":0.45400000000000001},"min":{"1min":0.28899999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.79000000000000004,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.45800000000000002,"15min":0.46100000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.56699999999999995,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.32400000000000001}]},{"osd":7,"last update":"Fri Sep 13 18:19:38 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.58599999999999997,"15min":0.57599999999999996},"min":{"1min":0.435,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.55500000000000005,"15min":0.54600000000000004},"min":{"1min":0.47299999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.437}]},{"osd":8,"last update":"Fri Sep 13 18:20:01 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.60799999999999998,"15min":0.60999999999999999},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61899999999999999,"15min":0.61699999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.52900000000000003}]},{"osd":9,"last update":"Fri Sep 13 18:19:28 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.41899999999999998}]},{"osd":10,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.55400000000000005}]},{"osd":11,"last update":"Fri Sep 13 18:20:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.621},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.66300000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34188,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34188,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34188,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35824,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35824,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35824,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35824,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35824,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":12212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":12084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:20:31.025 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:20:31.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.154+0000 7f436a3b5640 1 -- 172.21.15.114:0/1041234688 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43640fe440 msgr2=0x7f43640fe840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:31.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.154+0000 7f436a3b5640 1 --2- 172.21.15.114:0/1041234688 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43640fe440 0x7f43640fe840 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4354009900 tx=0x7f435402a410 comp rx=0 tx=0).stop 2024-09-13T18:20:31.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.154+0000 7f436a3b5640 1 -- 172.21.15.114:0/1041234688 shutdown_connections 2024-09-13T18:20:31.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.154+0000 7f436a3b5640 1 --2- 172.21.15.114:0/1041234688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43640ffff0 0x7f4364106880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:31.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.154+0000 7f436a3b5640 1 --2- 172.21.15.114:0/1041234688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43640ff630 0x7f43640ffab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:31.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.154+0000 7f436a3b5640 1 --2- 172.21.15.114:0/1041234688 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43640fe440 0x7f43640fe840 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:31.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.154+0000 7f436a3b5640 1 -- 172.21.15.114:0/1041234688 >> 172.21.15.114:0/1041234688 conn(0x7f43640f9bc0 msgr2=0x7f43640fbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:31.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.154+0000 7f436a3b5640 1 -- 172.21.15.114:0/1041234688 shutdown_connections 2024-09-13T18:20:31.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.154+0000 7f436a3b5640 1 -- 172.21.15.114:0/1041234688 wait complete. 2024-09-13T18:20:31.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.154+0000 7f436a3b5640 1 Processor -- start 2024-09-13T18:20:31.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.155+0000 7f436a3b5640 1 -- start start 2024-09-13T18:20:31.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.155+0000 7f436a3b5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43640fe440 0x7f4364109b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:31.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.155+0000 7f436a3b5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43640ff630 0x7f43641081d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:31.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.155+0000 7f436a3b5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43640ffff0 0x7f4364108710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:31.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.155+0000 7f436a3b5640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4364108e70 con 0x7f43640ff630 2024-09-13T18:20:31.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.155+0000 7f436a3b5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4364108fe0 con 0x7f43640fe440 2024-09-13T18:20:31.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.155+0000 7f436a3b5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436419fe00 con 0x7f43640ffff0 2024-09-13T18:20:31.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.155+0000 7f43637fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43640ff630 0x7f43641081d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:31.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.155+0000 7f43637fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43640ff630 0x7f43641081d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60844/0 (socket says 172.21.15.114:60844) 2024-09-13T18:20:31.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.155+0000 7f43637fe640 1 -- 172.21.15.114:0/1230089520 learned_addr learned my addr 172.21.15.114:0/1230089520 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:20:31.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.155+0000 7f4363fff640 1 --2- 172.21.15.114:0/1230089520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43640fe440 0x7f4364109b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:31.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.155+0000 7f4368952640 1 --2- 172.21.15.114:0/1230089520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43640ffff0 0x7f4364108710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:31.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.155+0000 7f43637fe640 1 -- 172.21.15.114:0/1230089520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43640ffff0 msgr2=0x7f4364108710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:31.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.155+0000 7f43637fe640 1 --2- 172.21.15.114:0/1230089520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43640ffff0 0x7f4364108710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:31.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.155+0000 7f43637fe640 1 -- 172.21.15.114:0/1230089520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43640fe440 msgr2=0x7f4364109b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:31.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.155+0000 7f43637fe640 1 --2- 172.21.15.114:0/1230089520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43640fe440 0x7f4364109b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:31.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.155+0000 7f43637fe640 1 -- 172.21.15.114:0/1230089520 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4354009560 con 0x7f43640ff630 2024-09-13T18:20:31.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.155+0000 7f4368952640 1 --2- 172.21.15.114:0/1230089520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43640ffff0 0x7f4364108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:20:31.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.155+0000 7f43637fe640 1 --2- 172.21.15.114:0/1230089520 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43640ff630 0x7f43641081d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f435800c8c0 tx=0x7f435800cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:31.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.156+0000 7f43617fa640 1 -- 172.21.15.114:0/1230089520 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4358004220 con 0x7f43640ff630 2024-09-13T18:20:31.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.156+0000 7f436a3b5640 1 -- 172.21.15.114:0/1230089520 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43641a00b0 con 0x7f43640ff630 2024-09-13T18:20:31.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.156+0000 7f43617fa640 1 -- 172.21.15.114:0/1230089520 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4358007e40 con 0x7f43640ff630 2024-09-13T18:20:31.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.156+0000 7f43617fa640 1 -- 172.21.15.114:0/1230089520 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43580047a0 con 0x7f43640ff630 2024-09-13T18:20:31.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.156+0000 7f436a3b5640 1 -- 172.21.15.114:0/1230089520 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43641a05c0 con 0x7f43640ff630 2024-09-13T18:20:31.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.157+0000 7f436a3b5640 1 -- 172.21.15.114:0/1230089520 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4364105a00 con 0x7f43640ff630 2024-09-13T18:20:31.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.158+0000 7f43617fa640 1 -- 172.21.15.114:0/1230089520 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f43580079b0 con 0x7f43640ff630 2024-09-13T18:20:31.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.158+0000 7f43617fa640 1 --2- 172.21.15.114:0/1230089520 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f434c0a1120 0x7f434c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:31.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.158+0000 7f4363fff640 1 --2- 172.21.15.114:0/1230089520 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f434c0a1120 0x7f434c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:31.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.158+0000 7f43617fa640 1 -- 172.21.15.114:0/1230089520 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f43580cca60 con 0x7f43640ff630 2024-09-13T18:20:31.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.159+0000 7f4363fff640 1 --2- 172.21.15.114:0/1230089520 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f434c0a1120 0x7f434c0a35e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f435402aa00 tx=0x7f4354005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:31.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.162+0000 7f43617fa640 1 -- 172.21.15.114:0/1230089520 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f4358095d80 con 0x7f43640ff630 2024-09-13T18:20:31.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.325+0000 7f436a3b5640 1 -- 172.21.15.114:0/1230089520 --> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f436402c450 con 0x7f434c0a1120 2024-09-13T18:20:31.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.336+0000 7f43617fa640 1 -- 172.21.15.114:0/1230089520 <== mgr.14148 v2:172.21.15.114:6800/3396185541 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+424028 (secure 0 0 0) 0x7f436402c450 con 0x7f434c0a1120 2024-09-13T18:20:31.344 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:20:31.347 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-13T18:20:31.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.340+0000 7f436a3b5640 1 -- 172.21.15.114:0/1230089520 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f434c0a1120 msgr2=0x7f434c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:31.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.340+0000 7f436a3b5640 1 --2- 172.21.15.114:0/1230089520 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f434c0a1120 0x7f434c0a35e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f435402aa00 tx=0x7f4354005a20 comp rx=0 tx=0).stop 2024-09-13T18:20:31.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.340+0000 7f436a3b5640 1 -- 172.21.15.114:0/1230089520 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43640ff630 msgr2=0x7f43641081d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:31.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.340+0000 7f436a3b5640 1 --2- 172.21.15.114:0/1230089520 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43640ff630 0x7f43641081d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f435800c8c0 tx=0x7f435800cd80 comp rx=0 tx=0).stop 2024-09-13T18:20:31.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.340+0000 7f436a3b5640 1 -- 172.21.15.114:0/1230089520 shutdown_connections 2024-09-13T18:20:31.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.340+0000 7f436a3b5640 1 --2- 172.21.15.114:0/1230089520 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f434c0a1120 0x7f434c0a35e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:31.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.340+0000 7f436a3b5640 1 --2- 172.21.15.114:0/1230089520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43640ffff0 0x7f4364108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:31.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.340+0000 7f436a3b5640 1 --2- 172.21.15.114:0/1230089520 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43640ff630 0x7f43641081d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:31.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.340+0000 7f436a3b5640 1 --2- 172.21.15.114:0/1230089520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43640fe440 0x7f4364109b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:31.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.340+0000 7f436a3b5640 1 -- 172.21.15.114:0/1230089520 >> 172.21.15.114:0/1230089520 conn(0x7f43640f9bc0 msgr2=0x7f4364102bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:31.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.340+0000 7f436a3b5640 1 -- 172.21.15.114:0/1230089520 shutdown_connections 2024-09-13T18:20:31.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.340+0000 7f436a3b5640 1 -- 172.21.15.114:0/1230089520 wait complete. 2024-09-13T18:20:31.353 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:31 smithi114 ceph-mon[30897]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-13T18:20:31.353 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:31 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2106708598' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:20:31.353 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:31 smithi114 ceph-mon[30897]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T18:20:31.353 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:31 smithi114 ceph-mon[30897]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T18:20:31.354 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-09-13T18:20:29.908599+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":603786,"num_objects":63,"num_object_clones":0,"num_object_copies":189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":145,"num_read_kb":136,"num_write":320,"num_write_kb":2989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":257,"ondisk_log_size":257,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":421,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360636,"kb_used_data":11828,"kb_used_omap":18,"kb_used_meta":348653,"kb_avail":1124581188,"statfs":{"total":1151940427776,"available":1151571136512,"internally_reserved":0,"allocated":12111872,"data_stored":7645949,"data_compressed":288644,"data_compressed_allocated":389120,"data_compressed_original":2277376,"omap_allocated":19129,"internal_metadata":357020999},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8825,"num_objects":39,"num_object_clones":0,"num_object_copies":117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":9,"num_read_kb":17,"num_write":91,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002330"},"pg_stats":[{"pgid":"2.1f","version":"91'13","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.256848+0000","last_change":"2024-09-13T18:20:01.577029+0000","last_active":"2024-09-13T18:20:28.256848+0000","last_peered":"2024-09-13T18:20:28.256848+0000","last_clean":"2024-09-13T18:20:28.256848+0000","last_became_active":"2024-09-13T18:20:01.576710+0000","last_became_peered":"2024-09-13T18:20:01.576710+0000","last_unstale":"2024-09-13T18:20:28.256848+0000","last_undegraded":"2024-09-13T18:20:28.256848+0000","last_fullsized":"2024-09-13T18:20:28.256848+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:25:11.977436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.190449+0000","last_change":"2024-09-13T18:20:01.922955+0000","last_active":"2024-09-13T18:20:26.190449+0000","last_peered":"2024-09-13T18:20:26.190449+0000","last_clean":"2024-09-13T18:20:26.190449+0000","last_became_active":"2024-09-13T18:20:01.922704+0000","last_became_peered":"2024-09-13T18:20:01.922704+0000","last_unstale":"2024-09-13T18:20:26.190449+0000","last_undegraded":"2024-09-13T18:20:26.190449+0000","last_fullsized":"2024-09-13T18:20:26.190449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:44:20.672784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.132781+0000","last_change":"2024-09-13T18:20:06.130388+0000","last_active":"2024-09-13T18:20:06.132781+0000","last_peered":"2024-09-13T18:20:06.132781+0000","last_clean":"2024-09-13T18:20:06.132781+0000","last_became_active":"2024-09-13T18:20:06.129895+0000","last_became_peered":"2024-09-13T18:20:06.129895+0000","last_unstale":"2024-09-13T18:20:06.132781+0000","last_undegraded":"2024-09-13T18:20:06.132781+0000","last_fullsized":"2024-09-13T18:20:06.132781+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:44:33.537849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.197866+0000","last_change":"2024-09-13T18:20:01.578921+0000","last_active":"2024-09-13T18:20:26.197866+0000","last_peered":"2024-09-13T18:20:26.197866+0000","last_clean":"2024-09-13T18:20:26.197866+0000","last_became_active":"2024-09-13T18:20:01.578627+0000","last_became_peered":"2024-09-13T18:20:01.578627+0000","last_unstale":"2024-09-13T18:20:26.197866+0000","last_undegraded":"2024-09-13T18:20:26.197866+0000","last_fullsized":"2024-09-13T18:20:26.197866+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:43:32.919156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.190717+0000","last_change":"2024-09-13T18:20:01.922901+0000","last_active":"2024-09-13T18:20:26.190717+0000","last_peered":"2024-09-13T18:20:26.190717+0000","last_clean":"2024-09-13T18:20:26.190717+0000","last_became_active":"2024-09-13T18:20:01.922604+0000","last_became_peered":"2024-09-13T18:20:01.922604+0000","last_unstale":"2024-09-13T18:20:26.190717+0000","last_undegraded":"2024-09-13T18:20:26.190717+0000","last_fullsized":"2024-09-13T18:20:26.190717+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:27:30.650010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.744777+0000","last_change":"2024-09-13T18:20:06.135406+0000","last_active":"2024-09-13T18:20:06.744777+0000","last_peered":"2024-09-13T18:20:06.744777+0000","last_clean":"2024-09-13T18:20:06.744777+0000","last_became_active":"2024-09-13T18:20:06.134558+0000","last_became_peered":"2024-09-13T18:20:06.134558+0000","last_unstale":"2024-09-13T18:20:06.744777+0000","last_undegraded":"2024-09-13T18:20:06.744777+0000","last_fullsized":"2024-09-13T18:20:06.744777+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:10.901966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"84'2","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.122416+0000","last_change":"2024-09-13T18:20:01.580040+0000","last_active":"2024-09-13T18:20:06.122416+0000","last_peered":"2024-09-13T18:20:06.122416+0000","last_clean":"2024-09-13T18:20:06.122416+0000","last_became_active":"2024-09-13T18:20:01.579715+0000","last_became_peered":"2024-09-13T18:20:01.579715+0000","last_unstale":"2024-09-13T18:20:06.122416+0000","last_undegraded":"2024-09-13T18:20:06.122416+0000","last_fullsized":"2024-09-13T18:20:06.122416+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:57:03.588169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.184089+0000","last_change":"2024-09-13T18:20:01.928314+0000","last_active":"2024-09-13T18:20:26.184089+0000","last_peered":"2024-09-13T18:20:26.184089+0000","last_clean":"2024-09-13T18:20:26.184089+0000","last_became_active":"2024-09-13T18:20:01.925356+0000","last_became_peered":"2024-09-13T18:20:01.925356+0000","last_unstale":"2024-09-13T18:20:26.184089+0000","last_undegraded":"2024-09-13T18:20:26.184089+0000","last_fullsized":"2024-09-13T18:20:26.184089+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:43.906842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.744757+0000","last_change":"2024-09-13T18:20:06.135886+0000","last_active":"2024-09-13T18:20:06.744757+0000","last_peered":"2024-09-13T18:20:06.744757+0000","last_clean":"2024-09-13T18:20:06.744757+0000","last_became_active":"2024-09-13T18:20:06.134587+0000","last_became_peered":"2024-09-13T18:20:06.134587+0000","last_unstale":"2024-09-13T18:20:06.744757+0000","last_undegraded":"2024-09-13T18:20:06.744757+0000","last_fullsized":"2024-09-13T18:20:06.744757+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:44:42.434729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.122797+0000","last_change":"2024-09-13T18:20:01.576908+0000","last_active":"2024-09-13T18:20:06.122797+0000","last_peered":"2024-09-13T18:20:06.122797+0000","last_clean":"2024-09-13T18:20:06.122797+0000","last_became_active":"2024-09-13T18:20:01.576478+0000","last_became_peered":"2024-09-13T18:20:01.576478+0000","last_unstale":"2024-09-13T18:20:06.122797+0000","last_undegraded":"2024-09-13T18:20:06.122797+0000","last_fullsized":"2024-09-13T18:20:06.122797+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:34:52.126901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.126411+0000","last_change":"2024-09-13T18:20:01.921960+0000","last_active":"2024-09-13T18:20:06.126411+0000","last_peered":"2024-09-13T18:20:06.126411+0000","last_clean":"2024-09-13T18:20:06.126411+0000","last_became_active":"2024-09-13T18:20:01.921652+0000","last_became_peered":"2024-09-13T18:20:01.921652+0000","last_unstale":"2024-09-13T18:20:06.126411+0000","last_undegraded":"2024-09-13T18:20:06.126411+0000","last_fullsized":"2024-09-13T18:20:06.126411+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:00:31.277613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.744407+0000","last_change":"2024-09-13T18:20:06.138983+0000","last_active":"2024-09-13T18:20:06.744407+0000","last_peered":"2024-09-13T18:20:06.744407+0000","last_clean":"2024-09-13T18:20:06.744407+0000","last_became_active":"2024-09-13T18:20:06.138708+0000","last_became_peered":"2024-09-13T18:20:06.138708+0000","last_unstale":"2024-09-13T18:20:06.744407+0000","last_undegraded":"2024-09-13T18:20:06.744407+0000","last_fullsized":"2024-09-13T18:20:06.744407+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:45:21.978217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.177004+0000","last_change":"2024-09-13T18:20:01.929817+0000","last_active":"2024-09-13T18:20:06.177004+0000","last_peered":"2024-09-13T18:20:06.177004+0000","last_clean":"2024-09-13T18:20:06.177004+0000","last_became_active":"2024-09-13T18:20:01.928907+0000","last_became_peered":"2024-09-13T18:20:01.928907+0000","last_unstale":"2024-09-13T18:20:06.177004+0000","last_undegraded":"2024-09-13T18:20:06.177004+0000","last_fullsized":"2024-09-13T18:20:06.177004+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:56:25.847230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:25.180585+0000","last_change":"2024-09-13T18:20:01.578968+0000","last_active":"2024-09-13T18:20:25.180585+0000","last_peered":"2024-09-13T18:20:25.180585+0000","last_clean":"2024-09-13T18:20:25.180585+0000","last_became_active":"2024-09-13T18:20:01.578623+0000","last_became_peered":"2024-09-13T18:20:01.578623+0000","last_unstale":"2024-09-13T18:20:25.180585+0000","last_undegraded":"2024-09-13T18:20:25.180585+0000","last_fullsized":"2024-09-13T18:20:25.180585+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:03:30.333107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.744596+0000","last_change":"2024-09-13T18:20:06.135977+0000","last_active":"2024-09-13T18:20:06.744596+0000","last_peered":"2024-09-13T18:20:06.744596+0000","last_clean":"2024-09-13T18:20:06.744596+0000","last_became_active":"2024-09-13T18:20:06.134844+0000","last_became_peered":"2024-09-13T18:20:06.134844+0000","last_unstale":"2024-09-13T18:20:06.744596+0000","last_undegraded":"2024-09-13T18:20:06.744596+0000","last_fullsized":"2024-09-13T18:20:06.744596+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:58:36.139240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.190165+0000","last_change":"2024-09-13T18:20:01.916675+0000","last_active":"2024-09-13T18:20:26.190165+0000","last_peered":"2024-09-13T18:20:26.190165+0000","last_clean":"2024-09-13T18:20:26.190165+0000","last_became_active":"2024-09-13T18:20:01.912902+0000","last_became_peered":"2024-09-13T18:20:01.912902+0000","last_unstale":"2024-09-13T18:20:26.190165+0000","last_undegraded":"2024-09-13T18:20:26.190165+0000","last_fullsized":"2024-09-13T18:20:26.190165+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:24:54.271569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:27.301309+0000","last_change":"2024-09-13T18:20:27.300845+0000","last_active":"2024-09-13T18:20:27.301309+0000","last_peered":"2024-09-13T18:20:27.301309+0000","last_clean":"2024-09-13T18:20:27.301309+0000","last_became_active":"2024-09-13T18:20:27.300417+0000","last_became_peered":"2024-09-13T18:20:27.300417+0000","last_unstale":"2024-09-13T18:20:27.301309+0000","last_undegraded":"2024-09-13T18:20:27.301309+0000","last_fullsized":"2024-09-13T18:20:27.301309+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:44:27.507140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.745268+0000","last_change":"2024-09-13T18:20:06.131539+0000","last_active":"2024-09-13T18:20:06.745268+0000","last_peered":"2024-09-13T18:20:06.745268+0000","last_clean":"2024-09-13T18:20:06.745268+0000","last_became_active":"2024-09-13T18:20:06.131148+0000","last_became_peered":"2024-09-13T18:20:06.131148+0000","last_unstale":"2024-09-13T18:20:06.745268+0000","last_undegraded":"2024-09-13T18:20:06.745268+0000","last_fullsized":"2024-09-13T18:20:06.745268+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:32:49.759831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183043+0000","last_change":"2024-09-13T18:20:01.913634+0000","last_active":"2024-09-13T18:20:26.183043+0000","last_peered":"2024-09-13T18:20:26.183043+0000","last_clean":"2024-09-13T18:20:26.183043+0000","last_became_active":"2024-09-13T18:20:01.913128+0000","last_became_peered":"2024-09-13T18:20:01.913128+0000","last_unstale":"2024-09-13T18:20:26.183043+0000","last_undegraded":"2024-09-13T18:20:26.183043+0000","last_fullsized":"2024-09-13T18:20:26.183043+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:37:15.251756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.192501+0000","last_change":"2024-09-13T18:20:01.579385+0000","last_active":"2024-09-13T18:20:26.192501+0000","last_peered":"2024-09-13T18:20:26.192501+0000","last_clean":"2024-09-13T18:20:26.192501+0000","last_became_active":"2024-09-13T18:20:01.579102+0000","last_became_peered":"2024-09-13T18:20:01.579102+0000","last_unstale":"2024-09-13T18:20:26.192501+0000","last_undegraded":"2024-09-13T18:20:26.192501+0000","last_fullsized":"2024-09-13T18:20:26.192501+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:22:00.432052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183422+0000","last_change":"2024-09-13T18:20:06.138110+0000","last_active":"2024-09-13T18:20:26.183422+0000","last_peered":"2024-09-13T18:20:26.183422+0000","last_clean":"2024-09-13T18:20:26.183422+0000","last_became_active":"2024-09-13T18:20:06.137763+0000","last_became_peered":"2024-09-13T18:20:06.137763+0000","last_unstale":"2024-09-13T18:20:26.183422+0000","last_undegraded":"2024-09-13T18:20:26.183422+0000","last_fullsized":"2024-09-13T18:20:26.183422+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:16:15.956216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.122199+0000","last_change":"2024-09-13T18:20:01.918318+0000","last_active":"2024-09-13T18:20:06.122199+0000","last_peered":"2024-09-13T18:20:06.122199+0000","last_clean":"2024-09-13T18:20:06.122199+0000","last_became_active":"2024-09-13T18:20:01.917977+0000","last_became_peered":"2024-09-13T18:20:01.917977+0000","last_unstale":"2024-09-13T18:20:06.122199+0000","last_undegraded":"2024-09-13T18:20:06.122199+0000","last_fullsized":"2024-09-13T18:20:06.122199+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:23.368866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.184253+0000","last_change":"2024-09-13T18:20:01.572009+0000","last_active":"2024-09-13T18:20:26.184253+0000","last_peered":"2024-09-13T18:20:26.184253+0000","last_clean":"2024-09-13T18:20:26.184253+0000","last_became_active":"2024-09-13T18:20:01.571313+0000","last_became_peered":"2024-09-13T18:20:01.571313+0000","last_unstale":"2024-09-13T18:20:26.184253+0000","last_undegraded":"2024-09-13T18:20:26.184253+0000","last_fullsized":"2024-09-13T18:20:26.184253+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:32:59.340476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.744412+0000","last_change":"2024-09-13T18:20:06.132212+0000","last_active":"2024-09-13T18:20:06.744412+0000","last_peered":"2024-09-13T18:20:06.744412+0000","last_clean":"2024-09-13T18:20:06.744412+0000","last_became_active":"2024-09-13T18:20:06.131809+0000","last_became_peered":"2024-09-13T18:20:06.131809+0000","last_unstale":"2024-09-13T18:20:06.744412+0000","last_undegraded":"2024-09-13T18:20:06.744412+0000","last_fullsized":"2024-09-13T18:20:06.744412+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:18:11.667551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"activating","last_fresh":"2024-09-13T18:20:27.181672+0000","last_change":"2024-09-13T18:20:27.181672+0000","last_active":"2024-09-13T18:20:26.189486+0000","last_peered":"2024-09-13T18:20:06.959341+0000","last_clean":"2024-09-13T18:20:06.959341+0000","last_became_active":"2024-09-13T18:20:01.922815+0000","last_became_peered":"2024-09-13T18:20:01.922815+0000","last_unstale":"2024-09-13T18:20:27.181672+0000","last_undegraded":"2024-09-13T18:20:27.181672+0000","last_fullsized":"2024-09-13T18:20:27.181672+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:59:57.606105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":["0","3","4","7"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:25.181259+0000","last_change":"2024-09-13T18:20:01.572025+0000","last_active":"2024-09-13T18:20:25.181259+0000","last_peered":"2024-09-13T18:20:25.181259+0000","last_clean":"2024-09-13T18:20:25.181259+0000","last_became_active":"2024-09-13T18:20:01.571321+0000","last_became_peered":"2024-09-13T18:20:01.571321+0000","last_unstale":"2024-09-13T18:20:25.181259+0000","last_undegraded":"2024-09-13T18:20:25.181259+0000","last_fullsized":"2024-09-13T18:20:25.181259+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:09.374364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.132797+0000","last_change":"2024-09-13T18:20:06.130687+0000","last_active":"2024-09-13T18:20:06.132797+0000","last_peered":"2024-09-13T18:20:06.132797+0000","last_clean":"2024-09-13T18:20:06.132797+0000","last_became_active":"2024-09-13T18:20:06.129839+0000","last_became_peered":"2024-09-13T18:20:06.129839+0000","last_unstale":"2024-09-13T18:20:06.132797+0000","last_undegraded":"2024-09-13T18:20:06.132797+0000","last_fullsized":"2024-09-13T18:20:06.132797+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:12:52.334945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.190458+0000","last_change":"2024-09-13T18:20:01.917244+0000","last_active":"2024-09-13T18:20:26.190458+0000","last_peered":"2024-09-13T18:20:26.190458+0000","last_clean":"2024-09-13T18:20:26.190458+0000","last_became_active":"2024-09-13T18:20:01.916948+0000","last_became_peered":"2024-09-13T18:20:01.916948+0000","last_unstale":"2024-09-13T18:20:26.190458+0000","last_undegraded":"2024-09-13T18:20:26.190458+0000","last_fullsized":"2024-09-13T18:20:26.190458+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:17:11.967096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'7","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.206617+0000","last_change":"2024-09-13T18:20:01.569123+0000","last_active":"2024-09-13T18:20:28.206617+0000","last_peered":"2024-09-13T18:20:28.206617+0000","last_clean":"2024-09-13T18:20:28.206617+0000","last_became_active":"2024-09-13T18:20:01.568720+0000","last_became_peered":"2024-09-13T18:20:01.568720+0000","last_unstale":"2024-09-13T18:20:28.206617+0000","last_undegraded":"2024-09-13T18:20:28.206617+0000","last_fullsized":"2024-09-13T18:20:28.206617+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:04:29.561013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.190854+0000","last_change":"2024-09-13T18:20:06.141771+0000","last_active":"2024-09-13T18:20:26.190854+0000","last_peered":"2024-09-13T18:20:26.190854+0000","last_clean":"2024-09-13T18:20:26.190854+0000","last_became_active":"2024-09-13T18:20:06.141362+0000","last_became_peered":"2024-09-13T18:20:06.141362+0000","last_unstale":"2024-09-13T18:20:26.190854+0000","last_undegraded":"2024-09-13T18:20:26.190854+0000","last_fullsized":"2024-09-13T18:20:26.190854+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:09:58.597605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.179047+0000","last_change":"2024-09-13T18:20:01.911048+0000","last_active":"2024-09-13T18:20:26.179047+0000","last_peered":"2024-09-13T18:20:26.179047+0000","last_clean":"2024-09-13T18:20:26.179047+0000","last_became_active":"2024-09-13T18:20:01.910735+0000","last_became_peered":"2024-09-13T18:20:01.910735+0000","last_unstale":"2024-09-13T18:20:26.179047+0000","last_undegraded":"2024-09-13T18:20:26.179047+0000","last_fullsized":"2024-09-13T18:20:26.179047+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:07:35.447676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.745685+0000","last_change":"2024-09-13T18:20:01.571627+0000","last_active":"2024-09-13T18:20:06.745685+0000","last_peered":"2024-09-13T18:20:06.745685+0000","last_clean":"2024-09-13T18:20:06.745685+0000","last_became_active":"2024-09-13T18:20:01.571250+0000","last_became_peered":"2024-09-13T18:20:01.571250+0000","last_unstale":"2024-09-13T18:20:06.745685+0000","last_undegraded":"2024-09-13T18:20:06.745685+0000","last_fullsized":"2024-09-13T18:20:06.745685+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:34:03.479324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.182932+0000","last_change":"2024-09-13T18:20:06.132974+0000","last_active":"2024-09-13T18:20:26.182932+0000","last_peered":"2024-09-13T18:20:26.182932+0000","last_clean":"2024-09-13T18:20:26.182932+0000","last_became_active":"2024-09-13T18:20:06.131668+0000","last_became_peered":"2024-09-13T18:20:06.131668+0000","last_unstale":"2024-09-13T18:20:26.182932+0000","last_undegraded":"2024-09-13T18:20:26.182932+0000","last_fullsized":"2024-09-13T18:20:26.182932+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:36:30.329761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.197759+0000","last_change":"2024-09-13T18:20:01.904670+0000","last_active":"2024-09-13T18:20:26.197759+0000","last_peered":"2024-09-13T18:20:26.197759+0000","last_clean":"2024-09-13T18:20:26.197759+0000","last_became_active":"2024-09-13T18:20:01.904316+0000","last_became_peered":"2024-09-13T18:20:01.904316+0000","last_unstale":"2024-09-13T18:20:26.197759+0000","last_undegraded":"2024-09-13T18:20:26.197759+0000","last_fullsized":"2024-09-13T18:20:26.197759+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:08:17.687179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"91'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:27.179594+0000","last_change":"2024-09-13T18:20:01.564750+0000","last_active":"2024-09-13T18:20:27.179594+0000","last_peered":"2024-09-13T18:20:27.179594+0000","last_clean":"2024-09-13T18:20:27.179594+0000","last_became_active":"2024-09-13T18:20:01.564096+0000","last_became_peered":"2024-09-13T18:20:01.564096+0000","last_unstale":"2024-09-13T18:20:27.179594+0000","last_undegraded":"2024-09-13T18:20:27.179594+0000","last_fullsized":"2024-09-13T18:20:27.179594+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:42:19.588824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.134398+0000","last_change":"2024-09-13T18:20:06.133386+0000","last_active":"2024-09-13T18:20:06.134398+0000","last_peered":"2024-09-13T18:20:06.134398+0000","last_clean":"2024-09-13T18:20:06.134398+0000","last_became_active":"2024-09-13T18:20:06.133004+0000","last_became_peered":"2024-09-13T18:20:06.133004+0000","last_unstale":"2024-09-13T18:20:06.134398+0000","last_undegraded":"2024-09-13T18:20:06.134398+0000","last_fullsized":"2024-09-13T18:20:06.134398+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:54:46.440410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.197632+0000","last_change":"2024-09-13T18:20:01.912823+0000","last_active":"2024-09-13T18:20:26.197632+0000","last_peered":"2024-09-13T18:20:26.197632+0000","last_clean":"2024-09-13T18:20:26.197632+0000","last_became_active":"2024-09-13T18:20:01.912123+0000","last_became_peered":"2024-09-13T18:20:01.912123+0000","last_unstale":"2024-09-13T18:20:26.197632+0000","last_undegraded":"2024-09-13T18:20:26.197632+0000","last_fullsized":"2024-09-13T18:20:26.197632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:32:33.544816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.122475+0000","last_change":"2024-09-13T18:20:01.577095+0000","last_active":"2024-09-13T18:20:06.122475+0000","last_peered":"2024-09-13T18:20:06.122475+0000","last_clean":"2024-09-13T18:20:06.122475+0000","last_became_active":"2024-09-13T18:20:01.576765+0000","last_became_peered":"2024-09-13T18:20:01.576765+0000","last_unstale":"2024-09-13T18:20:06.122475+0000","last_undegraded":"2024-09-13T18:20:06.122475+0000","last_fullsized":"2024-09-13T18:20:06.122475+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:44:08.750441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.178862+0000","last_change":"2024-09-13T18:20:06.131650+0000","last_active":"2024-09-13T18:20:26.178862+0000","last_peered":"2024-09-13T18:20:26.178862+0000","last_clean":"2024-09-13T18:20:26.178862+0000","last_became_active":"2024-09-13T18:20:06.130902+0000","last_became_peered":"2024-09-13T18:20:06.130902+0000","last_unstale":"2024-09-13T18:20:26.178862+0000","last_undegraded":"2024-09-13T18:20:26.178862+0000","last_fullsized":"2024-09-13T18:20:26.178862+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.366694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.122314+0000","last_change":"2024-09-13T18:20:01.918221+0000","last_active":"2024-09-13T18:20:06.122314+0000","last_peered":"2024-09-13T18:20:06.122314+0000","last_clean":"2024-09-13T18:20:06.122314+0000","last_became_active":"2024-09-13T18:20:01.917134+0000","last_became_peered":"2024-09-13T18:20:01.917134+0000","last_unstale":"2024-09-13T18:20:06.122314+0000","last_undegraded":"2024-09-13T18:20:06.122314+0000","last_fullsized":"2024-09-13T18:20:06.122314+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:19:06.269454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.249346+0000","last_change":"2024-09-13T18:20:01.572258+0000","last_active":"2024-09-13T18:20:28.249346+0000","last_peered":"2024-09-13T18:20:28.249346+0000","last_clean":"2024-09-13T18:20:28.249346+0000","last_became_active":"2024-09-13T18:20:01.571968+0000","last_became_peered":"2024-09-13T18:20:01.571968+0000","last_unstale":"2024-09-13T18:20:28.249346+0000","last_undegraded":"2024-09-13T18:20:28.249346+0000","last_fullsized":"2024-09-13T18:20:28.249346+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:40:59.143123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.179018+0000","last_change":"2024-09-13T18:20:06.131714+0000","last_active":"2024-09-13T18:20:26.179018+0000","last_peered":"2024-09-13T18:20:26.179018+0000","last_clean":"2024-09-13T18:20:26.179018+0000","last_became_active":"2024-09-13T18:20:06.131244+0000","last_became_peered":"2024-09-13T18:20:06.131244+0000","last_unstale":"2024-09-13T18:20:26.179018+0000","last_undegraded":"2024-09-13T18:20:26.179018+0000","last_fullsized":"2024-09-13T18:20:26.179018+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:37.946002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.178584+0000","last_change":"2024-09-13T18:20:01.917351+0000","last_active":"2024-09-13T18:20:26.178584+0000","last_peered":"2024-09-13T18:20:26.178584+0000","last_clean":"2024-09-13T18:20:26.178584+0000","last_became_active":"2024-09-13T18:20:01.917067+0000","last_became_peered":"2024-09-13T18:20:01.917067+0000","last_unstale":"2024-09-13T18:20:26.178584+0000","last_undegraded":"2024-09-13T18:20:26.178584+0000","last_fullsized":"2024-09-13T18:20:26.178584+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:35:37.723132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'1","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.462263+0000","last_change":"2024-09-13T18:20:01.573515+0000","last_active":"2024-09-13T18:20:06.462263+0000","last_peered":"2024-09-13T18:20:06.462263+0000","last_clean":"2024-09-13T18:20:06.462263+0000","last_became_active":"2024-09-13T18:20:01.573211+0000","last_became_peered":"2024-09-13T18:20:01.573211+0000","last_unstale":"2024-09-13T18:20:06.462263+0000","last_undegraded":"2024-09-13T18:20:06.462263+0000","last_fullsized":"2024-09-13T18:20:06.462263+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:05:30.912467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.134638+0000","last_change":"2024-09-13T18:20:06.133713+0000","last_active":"2024-09-13T18:20:06.134638+0000","last_peered":"2024-09-13T18:20:06.134638+0000","last_clean":"2024-09-13T18:20:06.134638+0000","last_became_active":"2024-09-13T18:20:06.133330+0000","last_became_peered":"2024-09-13T18:20:06.133330+0000","last_unstale":"2024-09-13T18:20:06.134638+0000","last_undegraded":"2024-09-13T18:20:06.134638+0000","last_fullsized":"2024-09-13T18:20:06.134638+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:48:07.133155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.192066+0000","last_change":"2024-09-13T18:20:01.921597+0000","last_active":"2024-09-13T18:20:26.192066+0000","last_peered":"2024-09-13T18:20:26.192066+0000","last_clean":"2024-09-13T18:20:26.192066+0000","last_became_active":"2024-09-13T18:20:01.920200+0000","last_became_peered":"2024-09-13T18:20:01.920200+0000","last_unstale":"2024-09-13T18:20:26.192066+0000","last_undegraded":"2024-09-13T18:20:26.192066+0000","last_fullsized":"2024-09-13T18:20:26.192066+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:22:40.517941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.186261+0000","last_change":"2024-09-13T18:20:01.571313+0000","last_active":"2024-09-13T18:20:26.186261+0000","last_peered":"2024-09-13T18:20:26.186261+0000","last_clean":"2024-09-13T18:20:26.186261+0000","last_became_active":"2024-09-13T18:20:01.570977+0000","last_became_peered":"2024-09-13T18:20:01.570977+0000","last_unstale":"2024-09-13T18:20:26.186261+0000","last_undegraded":"2024-09-13T18:20:26.186261+0000","last_fullsized":"2024-09-13T18:20:26.186261+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:06.870320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":341,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.122460+0000","last_change":"2024-09-13T18:18:51.237409+0000","last_active":"2024-09-13T18:20:06.122460+0000","last_peered":"2024-09-13T18:20:06.122460+0000","last_clean":"2024-09-13T18:20:06.122460+0000","last_became_active":"2024-09-13T18:18:51.236986+0000","last_became_peered":"2024-09-13T18:18:51.236986+0000","last_unstale":"2024-09-13T18:20:06.122460+0000","last_undegraded":"2024-09-13T18:20:06.122460+0000","last_fullsized":"2024-09-13T18:20:06.122460+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:15:40.757299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:15:40.757299+0000","last_clean_scrub_stamp":"2024-09-13T18:15:40.757299+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:58:29.679404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.177041+0000","last_change":"2024-09-13T18:20:06.130803+0000","last_active":"2024-09-13T18:20:06.177041+0000","last_peered":"2024-09-13T18:20:06.177041+0000","last_clean":"2024-09-13T18:20:06.177041+0000","last_became_active":"2024-09-13T18:20:06.130433+0000","last_became_peered":"2024-09-13T18:20:06.130433+0000","last_unstale":"2024-09-13T18:20:06.177041+0000","last_undegraded":"2024-09-13T18:20:06.177041+0000","last_fullsized":"2024-09-13T18:20:06.177041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:23:11.787084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.177067+0000","last_change":"2024-09-13T18:20:01.922702+0000","last_active":"2024-09-13T18:20:06.177067+0000","last_peered":"2024-09-13T18:20:06.177067+0000","last_clean":"2024-09-13T18:20:06.177067+0000","last_became_active":"2024-09-13T18:20:01.921109+0000","last_became_peered":"2024-09-13T18:20:01.921109+0000","last_unstale":"2024-09-13T18:20:06.177067+0000","last_undegraded":"2024-09-13T18:20:06.177067+0000","last_fullsized":"2024-09-13T18:20:06.177067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:03:34.618054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.123936+0000","last_change":"2024-09-13T18:20:01.571024+0000","last_active":"2024-09-13T18:20:06.123936+0000","last_peered":"2024-09-13T18:20:06.123936+0000","last_clean":"2024-09-13T18:20:06.123936+0000","last_became_active":"2024-09-13T18:20:01.570687+0000","last_became_peered":"2024-09-13T18:20:01.570687+0000","last_unstale":"2024-09-13T18:20:06.123936+0000","last_undegraded":"2024-09-13T18:20:06.123936+0000","last_fullsized":"2024-09-13T18:20:06.123936+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:10:25.126179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183401+0000","last_change":"2024-09-13T18:20:06.129025+0000","last_active":"2024-09-13T18:20:26.183401+0000","last_peered":"2024-09-13T18:20:26.183401+0000","last_clean":"2024-09-13T18:20:26.183401+0000","last_became_active":"2024-09-13T18:20:06.128618+0000","last_became_peered":"2024-09-13T18:20:06.128618+0000","last_unstale":"2024-09-13T18:20:26.183401+0000","last_undegraded":"2024-09-13T18:20:26.183401+0000","last_fullsized":"2024-09-13T18:20:26.183401+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:20:08.891259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.177033+0000","last_change":"2024-09-13T18:20:01.921697+0000","last_active":"2024-09-13T18:20:06.177033+0000","last_peered":"2024-09-13T18:20:06.177033+0000","last_clean":"2024-09-13T18:20:06.177033+0000","last_became_active":"2024-09-13T18:20:01.920174+0000","last_became_peered":"2024-09-13T18:20:01.920174+0000","last_unstale":"2024-09-13T18:20:06.177033+0000","last_undegraded":"2024-09-13T18:20:06.177033+0000","last_fullsized":"2024-09-13T18:20:06.177033+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:57:03.922725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.126610+0000","last_change":"2024-09-13T18:20:01.582102+0000","last_active":"2024-09-13T18:20:06.126610+0000","last_peered":"2024-09-13T18:20:06.126610+0000","last_clean":"2024-09-13T18:20:06.126610+0000","last_became_active":"2024-09-13T18:20:01.581841+0000","last_became_peered":"2024-09-13T18:20:01.581841+0000","last_unstale":"2024-09-13T18:20:06.126610+0000","last_undegraded":"2024-09-13T18:20:06.126610+0000","last_fullsized":"2024-09-13T18:20:06.126610+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:59:57.929703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.189924+0000","last_change":"2024-09-13T18:20:06.133126+0000","last_active":"2024-09-13T18:20:26.189924+0000","last_peered":"2024-09-13T18:20:26.189924+0000","last_clean":"2024-09-13T18:20:26.189924+0000","last_became_active":"2024-09-13T18:20:06.132747+0000","last_became_peered":"2024-09-13T18:20:06.132747+0000","last_unstale":"2024-09-13T18:20:26.189924+0000","last_undegraded":"2024-09-13T18:20:26.189924+0000","last_fullsized":"2024-09-13T18:20:26.189924+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:54:21.843090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.177082+0000","last_change":"2024-09-13T18:20:01.916306+0000","last_active":"2024-09-13T18:20:06.177082+0000","last_peered":"2024-09-13T18:20:06.177082+0000","last_clean":"2024-09-13T18:20:06.177082+0000","last_became_active":"2024-09-13T18:20:01.915739+0000","last_became_peered":"2024-09-13T18:20:01.915739+0000","last_unstale":"2024-09-13T18:20:06.177082+0000","last_undegraded":"2024-09-13T18:20:06.177082+0000","last_fullsized":"2024-09-13T18:20:06.177082+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:19:09.689909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"91'5","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.252883+0000","last_change":"2024-09-13T18:20:01.565821+0000","last_active":"2024-09-13T18:20:28.252883+0000","last_peered":"2024-09-13T18:20:28.252883+0000","last_clean":"2024-09-13T18:20:28.252883+0000","last_became_active":"2024-09-13T18:20:01.565467+0000","last_became_peered":"2024-09-13T18:20:01.565467+0000","last_unstale":"2024-09-13T18:20:28.252883+0000","last_undegraded":"2024-09-13T18:20:28.252883+0000","last_fullsized":"2024-09-13T18:20:28.252883+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:11.609400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.132093+0000","last_change":"2024-09-13T18:20:06.131348+0000","last_active":"2024-09-13T18:20:06.132093+0000","last_peered":"2024-09-13T18:20:06.132093+0000","last_clean":"2024-09-13T18:20:06.132093+0000","last_became_active":"2024-09-13T18:20:06.131019+0000","last_became_peered":"2024-09-13T18:20:06.131019+0000","last_unstale":"2024-09-13T18:20:06.132093+0000","last_undegraded":"2024-09-13T18:20:06.132093+0000","last_fullsized":"2024-09-13T18:20:06.132093+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:13:49.717503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183415+0000","last_change":"2024-09-13T18:20:01.922306+0000","last_active":"2024-09-13T18:20:26.183415+0000","last_peered":"2024-09-13T18:20:26.183415+0000","last_clean":"2024-09-13T18:20:26.183415+0000","last_became_active":"2024-09-13T18:20:01.921956+0000","last_became_peered":"2024-09-13T18:20:01.921956+0000","last_unstale":"2024-09-13T18:20:26.183415+0000","last_undegraded":"2024-09-13T18:20:26.183415+0000","last_fullsized":"2024-09-13T18:20:26.183415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:17:48.755318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:27.194592+0000","last_change":"2024-09-13T18:20:27.194085+0000","last_active":"2024-09-13T18:20:27.194592+0000","last_peered":"2024-09-13T18:20:27.194592+0000","last_clean":"2024-09-13T18:20:27.194592+0000","last_became_active":"2024-09-13T18:20:27.190136+0000","last_became_peered":"2024-09-13T18:20:27.190136+0000","last_unstale":"2024-09-13T18:20:27.194592+0000","last_undegraded":"2024-09-13T18:20:27.194592+0000","last_fullsized":"2024-09-13T18:20:27.194592+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:57:07.202901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.135975+0000","last_change":"2024-09-13T18:20:06.135498+0000","last_active":"2024-09-13T18:20:06.135975+0000","last_peered":"2024-09-13T18:20:06.135975+0000","last_clean":"2024-09-13T18:20:06.135975+0000","last_became_active":"2024-09-13T18:20:06.135095+0000","last_became_peered":"2024-09-13T18:20:06.135095+0000","last_unstale":"2024-09-13T18:20:06.135975+0000","last_undegraded":"2024-09-13T18:20:06.135975+0000","last_fullsized":"2024-09-13T18:20:06.135975+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:06:40.609718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:27.187296+0000","last_change":"2024-09-13T18:20:27.186825+0000","last_active":"2024-09-13T18:20:27.187296+0000","last_peered":"2024-09-13T18:20:27.187296+0000","last_clean":"2024-09-13T18:20:27.187296+0000","last_became_active":"2024-09-13T18:20:27.186523+0000","last_became_peered":"2024-09-13T18:20:27.186523+0000","last_unstale":"2024-09-13T18:20:27.187296+0000","last_undegraded":"2024-09-13T18:20:27.187296+0000","last_fullsized":"2024-09-13T18:20:27.187296+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:28:30.453945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.10","version":"89'10","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:25.180438+0000","last_change":"2024-09-13T18:20:01.569755+0000","last_active":"2024-09-13T18:20:25.180438+0000","last_peered":"2024-09-13T18:20:25.180438+0000","last_clean":"2024-09-13T18:20:25.180438+0000","last_became_active":"2024-09-13T18:20:01.569445+0000","last_became_peered":"2024-09-13T18:20:01.569445+0000","last_unstale":"2024-09-13T18:20:25.180438+0000","last_undegraded":"2024-09-13T18:20:25.180438+0000","last_fullsized":"2024-09-13T18:20:25.180438+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:45:20.733014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.179043+0000","last_change":"2024-09-13T18:20:06.136742+0000","last_active":"2024-09-13T18:20:26.179043+0000","last_peered":"2024-09-13T18:20:26.179043+0000","last_clean":"2024-09-13T18:20:26.179043+0000","last_became_active":"2024-09-13T18:20:06.136373+0000","last_became_peered":"2024-09-13T18:20:06.136373+0000","last_unstale":"2024-09-13T18:20:26.179043+0000","last_undegraded":"2024-09-13T18:20:26.179043+0000","last_fullsized":"2024-09-13T18:20:26.179043+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:49:11.782489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.176954+0000","last_change":"2024-09-13T18:20:01.922910+0000","last_active":"2024-09-13T18:20:06.176954+0000","last_peered":"2024-09-13T18:20:06.176954+0000","last_clean":"2024-09-13T18:20:06.176954+0000","last_became_active":"2024-09-13T18:20:01.921162+0000","last_became_peered":"2024-09-13T18:20:01.921162+0000","last_unstale":"2024-09-13T18:20:06.176954+0000","last_undegraded":"2024-09-13T18:20:06.176954+0000","last_fullsized":"2024-09-13T18:20:06.176954+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:14:47.243418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'3","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.197268+0000","last_change":"2024-09-13T18:20:01.575830+0000","last_active":"2024-09-13T18:20:26.197268+0000","last_peered":"2024-09-13T18:20:26.197268+0000","last_clean":"2024-09-13T18:20:26.197268+0000","last_became_active":"2024-09-13T18:20:01.575334+0000","last_became_peered":"2024-09-13T18:20:01.575334+0000","last_unstale":"2024-09-13T18:20:26.197268+0000","last_undegraded":"2024-09-13T18:20:26.197268+0000","last_fullsized":"2024-09-13T18:20:26.197268+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:13:00.935600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183501+0000","last_change":"2024-09-13T18:20:06.138609+0000","last_active":"2024-09-13T18:20:26.183501+0000","last_peered":"2024-09-13T18:20:26.183501+0000","last_clean":"2024-09-13T18:20:26.183501+0000","last_became_active":"2024-09-13T18:20:06.138232+0000","last_became_peered":"2024-09-13T18:20:06.138232+0000","last_unstale":"2024-09-13T18:20:26.183501+0000","last_undegraded":"2024-09-13T18:20:26.183501+0000","last_fullsized":"2024-09-13T18:20:26.183501+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:46:06.227407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.177024+0000","last_change":"2024-09-13T18:20:01.928757+0000","last_active":"2024-09-13T18:20:06.177024+0000","last_peered":"2024-09-13T18:20:06.177024+0000","last_clean":"2024-09-13T18:20:06.177024+0000","last_became_active":"2024-09-13T18:20:01.928350+0000","last_became_peered":"2024-09-13T18:20:01.928350+0000","last_unstale":"2024-09-13T18:20:06.177024+0000","last_undegraded":"2024-09-13T18:20:06.177024+0000","last_fullsized":"2024-09-13T18:20:06.177024+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:17:43.208106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:25.180579+0000","last_change":"2024-09-13T18:20:01.579128+0000","last_active":"2024-09-13T18:20:25.180579+0000","last_peered":"2024-09-13T18:20:25.180579+0000","last_clean":"2024-09-13T18:20:25.180579+0000","last_became_active":"2024-09-13T18:20:01.578796+0000","last_became_peered":"2024-09-13T18:20:01.578796+0000","last_unstale":"2024-09-13T18:20:25.180579+0000","last_undegraded":"2024-09-13T18:20:25.180579+0000","last_fullsized":"2024-09-13T18:20:25.180579+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:30:04.836103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.182950+0000","last_change":"2024-09-13T18:20:06.131404+0000","last_active":"2024-09-13T18:20:26.182950+0000","last_peered":"2024-09-13T18:20:26.182950+0000","last_clean":"2024-09-13T18:20:26.182950+0000","last_became_active":"2024-09-13T18:20:06.130931+0000","last_became_peered":"2024-09-13T18:20:06.130931+0000","last_unstale":"2024-09-13T18:20:26.182950+0000","last_undegraded":"2024-09-13T18:20:26.182950+0000","last_fullsized":"2024-09-13T18:20:26.182950+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:55:54.568816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.178724+0000","last_change":"2024-09-13T18:20:01.916089+0000","last_active":"2024-09-13T18:20:26.178724+0000","last_peered":"2024-09-13T18:20:26.178724+0000","last_clean":"2024-09-13T18:20:26.178724+0000","last_became_active":"2024-09-13T18:20:01.915783+0000","last_became_peered":"2024-09-13T18:20:01.915783+0000","last_unstale":"2024-09-13T18:20:26.178724+0000","last_undegraded":"2024-09-13T18:20:26.178724+0000","last_fullsized":"2024-09-13T18:20:26.178724+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:43:25.886029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"89'4","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:25.181228+0000","last_change":"2024-09-13T18:20:01.577381+0000","last_active":"2024-09-13T18:20:25.181228+0000","last_peered":"2024-09-13T18:20:25.181228+0000","last_clean":"2024-09-13T18:20:25.181228+0000","last_became_active":"2024-09-13T18:20:01.577044+0000","last_became_peered":"2024-09-13T18:20:01.577044+0000","last_unstale":"2024-09-13T18:20:25.181228+0000","last_undegraded":"2024-09-13T18:20:25.181228+0000","last_fullsized":"2024-09-13T18:20:25.181228+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:00.907948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.140300+0000","last_change":"2024-09-13T18:20:06.139971+0000","last_active":"2024-09-13T18:20:06.140300+0000","last_peered":"2024-09-13T18:20:06.140300+0000","last_clean":"2024-09-13T18:20:06.140300+0000","last_became_active":"2024-09-13T18:20:06.139507+0000","last_became_peered":"2024-09-13T18:20:06.139507+0000","last_unstale":"2024-09-13T18:20:06.140300+0000","last_undegraded":"2024-09-13T18:20:06.140300+0000","last_fullsized":"2024-09-13T18:20:06.140300+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:40:21.037768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.122321+0000","last_change":"2024-09-13T18:20:01.915348+0000","last_active":"2024-09-13T18:20:06.122321+0000","last_peered":"2024-09-13T18:20:06.122321+0000","last_clean":"2024-09-13T18:20:06.122321+0000","last_became_active":"2024-09-13T18:20:01.915042+0000","last_became_peered":"2024-09-13T18:20:01.915042+0000","last_unstale":"2024-09-13T18:20:06.122321+0000","last_undegraded":"2024-09-13T18:20:06.122321+0000","last_fullsized":"2024-09-13T18:20:06.122321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:31:33.712314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"91'17","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.256191+0000","last_change":"2024-09-13T18:20:01.578456+0000","last_active":"2024-09-13T18:20:28.256191+0000","last_peered":"2024-09-13T18:20:28.256191+0000","last_clean":"2024-09-13T18:20:28.256191+0000","last_became_active":"2024-09-13T18:20:01.578051+0000","last_became_peered":"2024-09-13T18:20:01.578051+0000","last_unstale":"2024-09-13T18:20:28.256191+0000","last_undegraded":"2024-09-13T18:20:28.256191+0000","last_fullsized":"2024-09-13T18:20:28.256191+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:34:17.651620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10396,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":12,"num_write":19,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183092+0000","last_change":"2024-09-13T18:20:06.131410+0000","last_active":"2024-09-13T18:20:26.183092+0000","last_peered":"2024-09-13T18:20:26.183092+0000","last_clean":"2024-09-13T18:20:26.183092+0000","last_became_active":"2024-09-13T18:20:06.131060+0000","last_became_peered":"2024-09-13T18:20:06.131060+0000","last_unstale":"2024-09-13T18:20:26.183092+0000","last_undegraded":"2024-09-13T18:20:26.183092+0000","last_fullsized":"2024-09-13T18:20:26.183092+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:08:25.266971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183935+0000","last_change":"2024-09-13T18:20:01.928839+0000","last_active":"2024-09-13T18:20:26.183935+0000","last_peered":"2024-09-13T18:20:26.183935+0000","last_clean":"2024-09-13T18:20:26.183935+0000","last_became_active":"2024-09-13T18:20:01.925774+0000","last_became_peered":"2024-09-13T18:20:01.925774+0000","last_unstale":"2024-09-13T18:20:26.183935+0000","last_undegraded":"2024-09-13T18:20:26.183935+0000","last_fullsized":"2024-09-13T18:20:26.183935+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:15:23.049671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.746467+0000","last_change":"2024-09-13T18:20:01.576332+0000","last_active":"2024-09-13T18:20:06.746467+0000","last_peered":"2024-09-13T18:20:06.746467+0000","last_clean":"2024-09-13T18:20:06.746467+0000","last_became_active":"2024-09-13T18:20:01.575862+0000","last_became_peered":"2024-09-13T18:20:01.575862+0000","last_unstale":"2024-09-13T18:20:06.746467+0000","last_undegraded":"2024-09-13T18:20:06.746467+0000","last_fullsized":"2024-09-13T18:20:06.746467+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:17:00.452480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.745244+0000","last_change":"2024-09-13T18:20:06.133142+0000","last_active":"2024-09-13T18:20:06.745244+0000","last_peered":"2024-09-13T18:20:06.745244+0000","last_clean":"2024-09-13T18:20:06.745244+0000","last_became_active":"2024-09-13T18:20:06.132740+0000","last_became_peered":"2024-09-13T18:20:06.132740+0000","last_unstale":"2024-09-13T18:20:06.745244+0000","last_undegraded":"2024-09-13T18:20:06.745244+0000","last_fullsized":"2024-09-13T18:20:06.745244+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:12:46.442481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.190799+0000","last_change":"2024-09-13T18:20:01.919673+0000","last_active":"2024-09-13T18:20:26.190799+0000","last_peered":"2024-09-13T18:20:26.190799+0000","last_clean":"2024-09-13T18:20:26.190799+0000","last_became_active":"2024-09-13T18:20:01.919358+0000","last_became_peered":"2024-09-13T18:20:01.919358+0000","last_unstale":"2024-09-13T18:20:26.190799+0000","last_undegraded":"2024-09-13T18:20:26.190799+0000","last_fullsized":"2024-09-13T18:20:26.190799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:40:11.407643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:25.181604+0000","last_change":"2024-09-13T18:20:01.575341+0000","last_active":"2024-09-13T18:20:25.181604+0000","last_peered":"2024-09-13T18:20:25.181604+0000","last_clean":"2024-09-13T18:20:25.181604+0000","last_became_active":"2024-09-13T18:20:01.575035+0000","last_became_peered":"2024-09-13T18:20:01.575035+0000","last_unstale":"2024-09-13T18:20:25.181604+0000","last_undegraded":"2024-09-13T18:20:25.181604+0000","last_fullsized":"2024-09-13T18:20:25.181604+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:24:29.891308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.182915+0000","last_change":"2024-09-13T18:20:06.133475+0000","last_active":"2024-09-13T18:20:26.182915+0000","last_peered":"2024-09-13T18:20:26.182915+0000","last_clean":"2024-09-13T18:20:26.182915+0000","last_became_active":"2024-09-13T18:20:06.133109+0000","last_became_peered":"2024-09-13T18:20:06.133109+0000","last_unstale":"2024-09-13T18:20:26.182915+0000","last_undegraded":"2024-09-13T18:20:26.182915+0000","last_fullsized":"2024-09-13T18:20:26.182915+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:47:19.567115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.184156+0000","last_change":"2024-09-13T18:20:01.922660+0000","last_active":"2024-09-13T18:20:26.184156+0000","last_peered":"2024-09-13T18:20:26.184156+0000","last_clean":"2024-09-13T18:20:26.184156+0000","last_became_active":"2024-09-13T18:20:01.922370+0000","last_became_peered":"2024-09-13T18:20:01.922370+0000","last_unstale":"2024-09-13T18:20:26.184156+0000","last_undegraded":"2024-09-13T18:20:26.184156+0000","last_fullsized":"2024-09-13T18:20:26.184156+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:54:32.029704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"91'4","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.249427+0000","last_change":"2024-09-13T18:20:01.580975+0000","last_active":"2024-09-13T18:20:28.249427+0000","last_peered":"2024-09-13T18:20:28.249427+0000","last_clean":"2024-09-13T18:20:28.249427+0000","last_became_active":"2024-09-13T18:20:01.580647+0000","last_became_peered":"2024-09-13T18:20:01.580647+0000","last_unstale":"2024-09-13T18:20:28.249427+0000","last_undegraded":"2024-09-13T18:20:28.249427+0000","last_fullsized":"2024-09-13T18:20:28.249427+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:26:20.942915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.192336+0000","last_change":"2024-09-13T18:20:06.140003+0000","last_active":"2024-09-13T18:20:26.192336+0000","last_peered":"2024-09-13T18:20:26.192336+0000","last_clean":"2024-09-13T18:20:26.192336+0000","last_became_active":"2024-09-13T18:20:06.139663+0000","last_became_peered":"2024-09-13T18:20:06.139663+0000","last_unstale":"2024-09-13T18:20:26.192336+0000","last_undegraded":"2024-09-13T18:20:26.192336+0000","last_fullsized":"2024-09-13T18:20:26.192336+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:54:49.662401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.189993+0000","last_change":"2024-09-13T18:20:06.134448+0000","last_active":"2024-09-13T18:20:26.189993+0000","last_peered":"2024-09-13T18:20:26.189993+0000","last_clean":"2024-09-13T18:20:26.189993+0000","last_became_active":"2024-09-13T18:20:06.134084+0000","last_became_peered":"2024-09-13T18:20:06.134084+0000","last_unstale":"2024-09-13T18:20:26.189993+0000","last_undegraded":"2024-09-13T18:20:26.189993+0000","last_fullsized":"2024-09-13T18:20:26.189993+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:55:50.475665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183131+0000","last_change":"2024-09-13T18:20:01.918955+0000","last_active":"2024-09-13T18:20:26.183131+0000","last_peered":"2024-09-13T18:20:26.183131+0000","last_clean":"2024-09-13T18:20:26.183131+0000","last_became_active":"2024-09-13T18:20:01.918666+0000","last_became_peered":"2024-09-13T18:20:01.918666+0000","last_unstale":"2024-09-13T18:20:26.183131+0000","last_undegraded":"2024-09-13T18:20:26.183131+0000","last_fullsized":"2024-09-13T18:20:26.183131+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:08:50.697710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.745892+0000","last_change":"2024-09-13T18:20:01.578789+0000","last_active":"2024-09-13T18:20:06.745892+0000","last_peered":"2024-09-13T18:20:06.745892+0000","last_clean":"2024-09-13T18:20:06.745892+0000","last_became_active":"2024-09-13T18:20:01.578476+0000","last_became_peered":"2024-09-13T18:20:01.578476+0000","last_unstale":"2024-09-13T18:20:06.745892+0000","last_undegraded":"2024-09-13T18:20:06.745892+0000","last_fullsized":"2024-09-13T18:20:06.745892+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:40:58.355492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183716+0000","last_change":"2024-09-13T18:20:06.129765+0000","last_active":"2024-09-13T18:20:26.183716+0000","last_peered":"2024-09-13T18:20:26.183716+0000","last_clean":"2024-09-13T18:20:26.183716+0000","last_became_active":"2024-09-13T18:20:06.129310+0000","last_became_peered":"2024-09-13T18:20:06.129310+0000","last_unstale":"2024-09-13T18:20:26.183716+0000","last_undegraded":"2024-09-13T18:20:26.183716+0000","last_fullsized":"2024-09-13T18:20:26.183716+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:34:07.394871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.191040+0000","last_change":"2024-09-13T18:20:01.914804+0000","last_active":"2024-09-13T18:20:26.191040+0000","last_peered":"2024-09-13T18:20:26.191040+0000","last_clean":"2024-09-13T18:20:26.191040+0000","last_became_active":"2024-09-13T18:20:01.914517+0000","last_became_peered":"2024-09-13T18:20:01.914517+0000","last_unstale":"2024-09-13T18:20:26.191040+0000","last_undegraded":"2024-09-13T18:20:26.191040+0000","last_fullsized":"2024-09-13T18:20:26.191040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:57:10.734935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"91'3","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:27.300317+0000","last_change":"2024-09-13T18:20:01.569201+0000","last_active":"2024-09-13T18:20:27.300317+0000","last_peered":"2024-09-13T18:20:27.300317+0000","last_clean":"2024-09-13T18:20:27.300317+0000","last_became_active":"2024-09-13T18:20:01.568908+0000","last_became_peered":"2024-09-13T18:20:01.568908+0000","last_unstale":"2024-09-13T18:20:27.300317+0000","last_undegraded":"2024-09-13T18:20:27.300317+0000","last_fullsized":"2024-09-13T18:20:27.300317+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:16:34.069130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"91'7","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.205826+0000","last_change":"2024-09-13T18:20:01.579225+0000","last_active":"2024-09-13T18:20:28.205826+0000","last_peered":"2024-09-13T18:20:28.205826+0000","last_clean":"2024-09-13T18:20:28.205826+0000","last_became_active":"2024-09-13T18:20:01.578826+0000","last_became_peered":"2024-09-13T18:20:01.578826+0000","last_unstale":"2024-09-13T18:20:28.205826+0000","last_undegraded":"2024-09-13T18:20:28.205826+0000","last_fullsized":"2024-09-13T18:20:28.205826+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:48:07.632239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.124287+0000","last_change":"2024-09-13T18:20:01.914885+0000","last_active":"2024-09-13T18:20:06.124287+0000","last_peered":"2024-09-13T18:20:06.124287+0000","last_clean":"2024-09-13T18:20:06.124287+0000","last_became_active":"2024-09-13T18:20:01.914599+0000","last_became_peered":"2024-09-13T18:20:01.914599+0000","last_unstale":"2024-09-13T18:20:06.124287+0000","last_undegraded":"2024-09-13T18:20:06.124287+0000","last_fullsized":"2024-09-13T18:20:06.124287+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:41:30.888675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.132751+0000","last_change":"2024-09-13T18:20:06.129376+0000","last_active":"2024-09-13T18:20:06.132751+0000","last_peered":"2024-09-13T18:20:06.132751+0000","last_clean":"2024-09-13T18:20:06.132751+0000","last_became_active":"2024-09-13T18:20:06.129032+0000","last_became_peered":"2024-09-13T18:20:06.129032+0000","last_unstale":"2024-09-13T18:20:06.132751+0000","last_undegraded":"2024-09-13T18:20:06.132751+0000","last_fullsized":"2024-09-13T18:20:06.132751+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:21:53.511813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:25.181451+0000","last_change":"2024-09-13T18:20:01.576069+0000","last_active":"2024-09-13T18:20:25.181451+0000","last_peered":"2024-09-13T18:20:25.181451+0000","last_clean":"2024-09-13T18:20:25.181451+0000","last_became_active":"2024-09-13T18:20:01.575760+0000","last_became_peered":"2024-09-13T18:20:01.575760+0000","last_unstale":"2024-09-13T18:20:25.181451+0000","last_undegraded":"2024-09-13T18:20:25.181451+0000","last_fullsized":"2024-09-13T18:20:25.181451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:05:47.677848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.122985+0000","last_change":"2024-09-13T18:20:01.917073+0000","last_active":"2024-09-13T18:20:06.122985+0000","last_peered":"2024-09-13T18:20:06.122985+0000","last_clean":"2024-09-13T18:20:06.122985+0000","last_became_active":"2024-09-13T18:20:01.916765+0000","last_became_peered":"2024-09-13T18:20:01.916765+0000","last_unstale":"2024-09-13T18:20:06.122985+0000","last_undegraded":"2024-09-13T18:20:06.122985+0000","last_fullsized":"2024-09-13T18:20:06.122985+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:22:07.788313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.133135+0000","last_change":"2024-09-13T18:20:06.131429+0000","last_active":"2024-09-13T18:20:06.133135+0000","last_peered":"2024-09-13T18:20:06.133135+0000","last_clean":"2024-09-13T18:20:06.133135+0000","last_became_active":"2024-09-13T18:20:06.131019+0000","last_became_peered":"2024-09-13T18:20:06.131019+0000","last_unstale":"2024-09-13T18:20:06.133135+0000","last_undegraded":"2024-09-13T18:20:06.133135+0000","last_fullsized":"2024-09-13T18:20:06.133135+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:44:37.892574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.182865+0000","last_change":"2024-09-13T18:20:01.914050+0000","last_active":"2024-09-13T18:20:26.182865+0000","last_peered":"2024-09-13T18:20:26.182865+0000","last_clean":"2024-09-13T18:20:26.182865+0000","last_became_active":"2024-09-13T18:20:01.913733+0000","last_became_peered":"2024-09-13T18:20:01.913733+0000","last_unstale":"2024-09-13T18:20:26.182865+0000","last_undegraded":"2024-09-13T18:20:26.182865+0000","last_fullsized":"2024-09-13T18:20:26.182865+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:52:43.672096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.123732+0000","last_change":"2024-09-13T18:20:01.924290+0000","last_active":"2024-09-13T18:20:06.123732+0000","last_peered":"2024-09-13T18:20:06.123732+0000","last_clean":"2024-09-13T18:20:06.123732+0000","last_became_active":"2024-09-13T18:20:01.923883+0000","last_became_peered":"2024-09-13T18:20:01.923883+0000","last_unstale":"2024-09-13T18:20:06.123732+0000","last_undegraded":"2024-09-13T18:20:06.123732+0000","last_fullsized":"2024-09-13T18:20:06.123732+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:11:18.737501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.125469+0000","last_change":"2024-09-13T18:20:01.928609+0000","last_active":"2024-09-13T18:20:06.125469+0000","last_peered":"2024-09-13T18:20:06.125469+0000","last_clean":"2024-09-13T18:20:06.125469+0000","last_became_active":"2024-09-13T18:20:01.927466+0000","last_became_peered":"2024-09-13T18:20:01.927466+0000","last_unstale":"2024-09-13T18:20:06.125469+0000","last_undegraded":"2024-09-13T18:20:06.125469+0000","last_fullsized":"2024-09-13T18:20:06.125469+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:18:58.610470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.189852+0000","last_change":"2024-09-13T18:20:01.915799+0000","last_active":"2024-09-13T18:20:26.189852+0000","last_peered":"2024-09-13T18:20:26.189852+0000","last_clean":"2024-09-13T18:20:26.189852+0000","last_became_active":"2024-09-13T18:20:01.915519+0000","last_became_peered":"2024-09-13T18:20:01.915519+0000","last_unstale":"2024-09-13T18:20:26.189852+0000","last_undegraded":"2024-09-13T18:20:26.189852+0000","last_fullsized":"2024-09-13T18:20:26.189852+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:08:26.065052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.178968+0000","last_change":"2024-09-13T18:20:01.918547+0000","last_active":"2024-09-13T18:20:26.178968+0000","last_peered":"2024-09-13T18:20:26.178968+0000","last_clean":"2024-09-13T18:20:26.178968+0000","last_became_active":"2024-09-13T18:20:01.918053+0000","last_became_peered":"2024-09-13T18:20:01.918053+0000","last_unstale":"2024-09-13T18:20:26.178968+0000","last_undegraded":"2024-09-13T18:20:26.178968+0000","last_fullsized":"2024-09-13T18:20:26.178968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:25:58.235101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183342+0000","last_change":"2024-09-13T18:20:01.916205+0000","last_active":"2024-09-13T18:20:26.183342+0000","last_peered":"2024-09-13T18:20:26.183342+0000","last_clean":"2024-09-13T18:20:26.183342+0000","last_became_active":"2024-09-13T18:20:01.915909+0000","last_became_peered":"2024-09-13T18:20:01.915909+0000","last_unstale":"2024-09-13T18:20:26.183342+0000","last_undegraded":"2024-09-13T18:20:26.183342+0000","last_fullsized":"2024-09-13T18:20:26.183342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:23:03.976513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.123579+0000","last_change":"2024-09-13T18:20:01.916621+0000","last_active":"2024-09-13T18:20:06.123579+0000","last_peered":"2024-09-13T18:20:06.123579+0000","last_clean":"2024-09-13T18:20:06.123579+0000","last_became_active":"2024-09-13T18:20:01.916335+0000","last_became_peered":"2024-09-13T18:20:01.916335+0000","last_unstale":"2024-09-13T18:20:06.123579+0000","last_undegraded":"2024-09-13T18:20:06.123579+0000","last_fullsized":"2024-09-13T18:20:06.123579+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:08:16.104571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.190215+0000","last_change":"2024-09-13T18:20:01.921195+0000","last_active":"2024-09-13T18:20:26.190215+0000","last_peered":"2024-09-13T18:20:26.190215+0000","last_clean":"2024-09-13T18:20:26.190215+0000","last_became_active":"2024-09-13T18:20:01.920895+0000","last_became_peered":"2024-09-13T18:20:01.920895+0000","last_unstale":"2024-09-13T18:20:26.190215+0000","last_undegraded":"2024-09-13T18:20:26.190215+0000","last_fullsized":"2024-09-13T18:20:26.190215+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:59:53.920047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.122232+0000","last_change":"2024-09-13T18:20:01.910889+0000","last_active":"2024-09-13T18:20:06.122232+0000","last_peered":"2024-09-13T18:20:06.122232+0000","last_clean":"2024-09-13T18:20:06.122232+0000","last_became_active":"2024-09-13T18:20:01.910451+0000","last_became_peered":"2024-09-13T18:20:01.910451+0000","last_unstale":"2024-09-13T18:20:06.122232+0000","last_undegraded":"2024-09-13T18:20:06.122232+0000","last_fullsized":"2024-09-13T18:20:06.122232+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:06:52.552785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.192420+0000","last_change":"2024-09-13T18:20:01.914041+0000","last_active":"2024-09-13T18:20:26.192420+0000","last_peered":"2024-09-13T18:20:26.192420+0000","last_clean":"2024-09-13T18:20:26.192420+0000","last_became_active":"2024-09-13T18:20:01.913240+0000","last_became_peered":"2024-09-13T18:20:01.913240+0000","last_unstale":"2024-09-13T18:20:26.192420+0000","last_undegraded":"2024-09-13T18:20:26.192420+0000","last_fullsized":"2024-09-13T18:20:26.192420+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:39:37.062544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.184154+0000","last_change":"2024-09-13T18:20:01.927322+0000","last_active":"2024-09-13T18:20:26.184154+0000","last_peered":"2024-09-13T18:20:26.184154+0000","last_clean":"2024-09-13T18:20:26.184154+0000","last_became_active":"2024-09-13T18:20:01.926767+0000","last_became_peered":"2024-09-13T18:20:01.926767+0000","last_unstale":"2024-09-13T18:20:26.184154+0000","last_undegraded":"2024-09-13T18:20:26.184154+0000","last_fullsized":"2024-09-13T18:20:26.184154+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:20.986089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.178873+0000","last_change":"2024-09-13T18:20:01.916314+0000","last_active":"2024-09-13T18:20:26.178873+0000","last_peered":"2024-09-13T18:20:26.178873+0000","last_clean":"2024-09-13T18:20:26.178873+0000","last_became_active":"2024-09-13T18:20:01.915790+0000","last_became_peered":"2024-09-13T18:20:01.915790+0000","last_unstale":"2024-09-13T18:20:26.178873+0000","last_undegraded":"2024-09-13T18:20:26.178873+0000","last_fullsized":"2024-09-13T18:20:26.178873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:20:30.063814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:27.187172+0000","last_change":"2024-09-13T18:20:27.187172+0000","last_active":"2024-09-13T18:20:27.187172+0000","last_peered":"2024-09-13T18:20:27.187172+0000","last_clean":"2024-09-13T18:20:27.187172+0000","last_became_active":"2024-09-13T18:20:27.186765+0000","last_became_peered":"2024-09-13T18:20:27.186765+0000","last_unstale":"2024-09-13T18:20:27.187172+0000","last_undegraded":"2024-09-13T18:20:27.187172+0000","last_fullsized":"2024-09-13T18:20:27.187172+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:37:01.469560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.124324+0000","last_change":"2024-09-13T18:20:01.924074+0000","last_active":"2024-09-13T18:20:06.124324+0000","last_peered":"2024-09-13T18:20:06.124324+0000","last_clean":"2024-09-13T18:20:06.124324+0000","last_became_active":"2024-09-13T18:20:01.922364+0000","last_became_peered":"2024-09-13T18:20:01.922364+0000","last_unstale":"2024-09-13T18:20:06.124324+0000","last_undegraded":"2024-09-13T18:20:06.124324+0000","last_fullsized":"2024-09-13T18:20:06.124324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:51:33.824362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183604+0000","last_change":"2024-09-13T18:20:01.914606+0000","last_active":"2024-09-13T18:20:26.183604+0000","last_peered":"2024-09-13T18:20:26.183604+0000","last_clean":"2024-09-13T18:20:26.183604+0000","last_became_active":"2024-09-13T18:20:01.914281+0000","last_became_peered":"2024-09-13T18:20:01.914281+0000","last_unstale":"2024-09-13T18:20:26.183604+0000","last_undegraded":"2024-09-13T18:20:26.183604+0000","last_fullsized":"2024-09-13T18:20:26.183604+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:55:42.211959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.184054+0000","last_change":"2024-09-13T18:20:01.928876+0000","last_active":"2024-09-13T18:20:26.184054+0000","last_peered":"2024-09-13T18:20:26.184054+0000","last_clean":"2024-09-13T18:20:26.184054+0000","last_became_active":"2024-09-13T18:20:01.928534+0000","last_became_peered":"2024-09-13T18:20:01.928534+0000","last_unstale":"2024-09-13T18:20:26.184054+0000","last_undegraded":"2024-09-13T18:20:26.184054+0000","last_fullsized":"2024-09-13T18:20:26.184054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:14:28.196746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183517+0000","last_change":"2024-09-13T18:20:01.920642+0000","last_active":"2024-09-13T18:20:26.183517+0000","last_peered":"2024-09-13T18:20:26.183517+0000","last_clean":"2024-09-13T18:20:26.183517+0000","last_became_active":"2024-09-13T18:20:01.920335+0000","last_became_peered":"2024-09-13T18:20:01.920335+0000","last_unstale":"2024-09-13T18:20:26.183517+0000","last_undegraded":"2024-09-13T18:20:26.183517+0000","last_fullsized":"2024-09-13T18:20:26.183517+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:59:20.894563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:27.187529+0000","last_change":"2024-09-13T18:20:27.187529+0000","last_active":"2024-09-13T18:20:27.187529+0000","last_peered":"2024-09-13T18:20:27.187529+0000","last_clean":"2024-09-13T18:20:27.187529+0000","last_became_active":"2024-09-13T18:20:27.187123+0000","last_became_peered":"2024-09-13T18:20:27.187123+0000","last_unstale":"2024-09-13T18:20:27.187529+0000","last_undegraded":"2024-09-13T18:20:27.187529+0000","last_fullsized":"2024-09-13T18:20:27.187529+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:43:19.601235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.189462+0000","last_change":"2024-09-13T18:20:01.917111+0000","last_active":"2024-09-13T18:20:26.189462+0000","last_peered":"2024-09-13T18:20:26.189462+0000","last_clean":"2024-09-13T18:20:26.189462+0000","last_became_active":"2024-09-13T18:20:01.916826+0000","last_became_peered":"2024-09-13T18:20:01.916826+0000","last_unstale":"2024-09-13T18:20:26.189462+0000","last_undegraded":"2024-09-13T18:20:26.189462+0000","last_fullsized":"2024-09-13T18:20:26.189462+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:36:08.150028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.178921+0000","last_change":"2024-09-13T18:20:01.904496+0000","last_active":"2024-09-13T18:20:26.178921+0000","last_peered":"2024-09-13T18:20:26.178921+0000","last_clean":"2024-09-13T18:20:26.178921+0000","last_became_active":"2024-09-13T18:20:01.904199+0000","last_became_peered":"2024-09-13T18:20:01.904199+0000","last_unstale":"2024-09-13T18:20:26.178921+0000","last_undegraded":"2024-09-13T18:20:26.178921+0000","last_fullsized":"2024-09-13T18:20:26.178921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:44.044162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.745431+0000","last_change":"2024-09-13T18:20:01.919023+0000","last_active":"2024-09-13T18:20:06.745431+0000","last_peered":"2024-09-13T18:20:06.745431+0000","last_clean":"2024-09-13T18:20:06.745431+0000","last_became_active":"2024-09-13T18:20:01.918661+0000","last_became_peered":"2024-09-13T18:20:01.918661+0000","last_unstale":"2024-09-13T18:20:06.745431+0000","last_undegraded":"2024-09-13T18:20:06.745431+0000","last_fullsized":"2024-09-13T18:20:06.745431+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:25:26.779029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.177051+0000","last_change":"2024-09-13T18:20:01.922833+0000","last_active":"2024-09-13T18:20:06.177051+0000","last_peered":"2024-09-13T18:20:06.177051+0000","last_clean":"2024-09-13T18:20:06.177051+0000","last_became_active":"2024-09-13T18:20:01.922533+0000","last_became_peered":"2024-09-13T18:20:01.922533+0000","last_unstale":"2024-09-13T18:20:06.177051+0000","last_undegraded":"2024-09-13T18:20:06.177051+0000","last_fullsized":"2024-09-13T18:20:06.177051+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:27:11.557419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.197907+0000","last_change":"2024-09-13T18:20:01.925916+0000","last_active":"2024-09-13T18:20:26.197907+0000","last_peered":"2024-09-13T18:20:26.197907+0000","last_clean":"2024-09-13T18:20:26.197907+0000","last_became_active":"2024-09-13T18:20:01.925568+0000","last_became_peered":"2024-09-13T18:20:01.925568+0000","last_unstale":"2024-09-13T18:20:26.197907+0000","last_undegraded":"2024-09-13T18:20:26.197907+0000","last_fullsized":"2024-09-13T18:20:26.197907+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:58:24.955273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.184513+0000","last_change":"2024-09-13T18:20:01.924328+0000","last_active":"2024-09-13T18:20:26.184513+0000","last_peered":"2024-09-13T18:20:26.184513+0000","last_clean":"2024-09-13T18:20:26.184513+0000","last_became_active":"2024-09-13T18:20:01.924004+0000","last_became_peered":"2024-09-13T18:20:01.924004+0000","last_unstale":"2024-09-13T18:20:26.184513+0000","last_undegraded":"2024-09-13T18:20:26.184513+0000","last_fullsized":"2024-09-13T18:20:26.184513+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:23:29.204157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.190973+0000","last_change":"2024-09-13T18:20:01.922234+0000","last_active":"2024-09-13T18:20:26.190973+0000","last_peered":"2024-09-13T18:20:26.190973+0000","last_clean":"2024-09-13T18:20:26.190973+0000","last_became_active":"2024-09-13T18:20:01.921196+0000","last_became_peered":"2024-09-13T18:20:01.921196+0000","last_unstale":"2024-09-13T18:20:26.190973+0000","last_undegraded":"2024-09-13T18:20:26.190973+0000","last_fullsized":"2024-09-13T18:20:26.190973+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:24:10.144346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.176993+0000","last_change":"2024-09-13T18:20:01.921740+0000","last_active":"2024-09-13T18:20:06.176993+0000","last_peered":"2024-09-13T18:20:06.176993+0000","last_clean":"2024-09-13T18:20:06.176993+0000","last_became_active":"2024-09-13T18:20:01.919634+0000","last_became_peered":"2024-09-13T18:20:01.919634+0000","last_unstale":"2024-09-13T18:20:06.176993+0000","last_undegraded":"2024-09-13T18:20:06.176993+0000","last_fullsized":"2024-09-13T18:20:06.176993+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:43:27.600637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.197426+0000","last_change":"2024-09-13T18:20:01.924806+0000","last_active":"2024-09-13T18:20:26.197426+0000","last_peered":"2024-09-13T18:20:26.197426+0000","last_clean":"2024-09-13T18:20:26.197426+0000","last_became_active":"2024-09-13T18:20:01.924514+0000","last_became_peered":"2024-09-13T18:20:01.924514+0000","last_unstale":"2024-09-13T18:20:26.197426+0000","last_undegraded":"2024-09-13T18:20:26.197426+0000","last_fullsized":"2024-09-13T18:20:26.197426+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:47.323362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.122321+0000","last_change":"2024-09-13T18:20:01.912402+0000","last_active":"2024-09-13T18:20:06.122321+0000","last_peered":"2024-09-13T18:20:06.122321+0000","last_clean":"2024-09-13T18:20:06.122321+0000","last_became_active":"2024-09-13T18:20:01.911534+0000","last_became_peered":"2024-09-13T18:20:01.911534+0000","last_unstale":"2024-09-13T18:20:06.122321+0000","last_undegraded":"2024-09-13T18:20:06.122321+0000","last_fullsized":"2024-09-13T18:20:06.122321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:22:52.021688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T18:20:06.745369+0000","last_change":"2024-09-13T18:20:01.930248+0000","last_active":"2024-09-13T18:20:06.745369+0000","last_peered":"2024-09-13T18:20:06.745369+0000","last_clean":"2024-09-13T18:20:06.745369+0000","last_became_active":"2024-09-13T18:20:01.929931+0000","last_became_peered":"2024-09-13T18:20:01.929931+0000","last_unstale":"2024-09-13T18:20:06.745369+0000","last_undegraded":"2024-09-13T18:20:06.745369+0000","last_fullsized":"2024-09-13T18:20:06.745369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:37:33.086722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.190113+0000","last_change":"2024-09-13T18:20:01.921913+0000","last_active":"2024-09-13T18:20:26.190113+0000","last_peered":"2024-09-13T18:20:26.190113+0000","last_clean":"2024-09-13T18:20:26.190113+0000","last_became_active":"2024-09-13T18:20:01.921395+0000","last_became_peered":"2024-09-13T18:20:01.921395+0000","last_unstale":"2024-09-13T18:20:26.190113+0000","last_undegraded":"2024-09-13T18:20:26.190113+0000","last_fullsized":"2024-09-13T18:20:26.190113+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:54:50.816870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.190140+0000","last_change":"2024-09-13T18:20:01.912654+0000","last_active":"2024-09-13T18:20:26.190140+0000","last_peered":"2024-09-13T18:20:26.190140+0000","last_clean":"2024-09-13T18:20:26.190140+0000","last_became_active":"2024-09-13T18:20:01.912356+0000","last_became_peered":"2024-09-13T18:20:01.912356+0000","last_unstale":"2024-09-13T18:20:26.190140+0000","last_undegraded":"2024-09-13T18:20:26.190140+0000","last_fullsized":"2024-09-13T18:20:26.190140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:25:18.099396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183955+0000","last_change":"2024-09-13T18:20:01.923373+0000","last_active":"2024-09-13T18:20:26.183955+0000","last_peered":"2024-09-13T18:20:26.183955+0000","last_clean":"2024-09-13T18:20:26.183955+0000","last_became_active":"2024-09-13T18:20:01.923077+0000","last_became_peered":"2024-09-13T18:20:01.923077+0000","last_unstale":"2024-09-13T18:20:26.183955+0000","last_undegraded":"2024-09-13T18:20:26.183955+0000","last_fullsized":"2024-09-13T18:20:26.183955+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:07:52.955705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":13418,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":9,"num_read_kb":17,"num_write":139,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":303104,"data_stored":36667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281684,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383698,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":995328,"data_stored":495945,"data_compressed":17567,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.47599999999999998}]},{"osd":1,"last update":"Fri Sep 13 18:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.58499999999999996}]},{"osd":2,"last update":"Fri Sep 13 18:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.622}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":4,"last update":"Fri Sep 13 18:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.64100000000000001}]},{"osd":5,"last update":"Fri Sep 13 18:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.69699999999999995}]},{"osd":6,"last update":"Fri Sep 13 18:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.56699999999999995}]},{"osd":7,"last update":"Fri Sep 13 18:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.68100000000000005}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":9,"last update":"Fri Sep 13 18:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.84999999999999998}]},{"osd":10,"last update":"Fri Sep 13 18:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.751}]}]},{"osd":10,"up_from":72,"seq":309237645335,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1077248,"data_stored":1090703,"data_compressed":53391,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":1,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.66700000000000004}]},{"osd":2,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.625}]},{"osd":3,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.60999999999999999}]},{"osd":4,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.64500000000000002}]},{"osd":5,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.56599999999999995}]},{"osd":6,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.76100000000000001}]},{"osd":7,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.81599999999999995}]},{"osd":8,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.80400000000000005}]},{"osd":9,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.83399999999999996}]},{"osd":11,"last update":"Fri Sep 13 18:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.872}]}]},{"osd":9,"up_from":67,"seq":287762808860,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":500559,"data_compressed":17567,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.311}]},{"osd":1,"last update":"Fri Sep 13 18:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.5}]},{"osd":2,"last update":"Fri Sep 13 18:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.53900000000000003}]},{"osd":3,"last update":"Fri Sep 13 18:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.34300000000000003}]},{"osd":4,"last update":"Fri Sep 13 18:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.57599999999999996}]},{"osd":5,"last update":"Fri Sep 13 18:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.55400000000000005}]},{"osd":6,"last update":"Fri Sep 13 18:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.39800000000000002}]},{"osd":7,"last update":"Fri Sep 13 18:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.47999999999999998}]},{"osd":8,"last update":"Fri Sep 13 18:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.624}]},{"osd":10,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.63900000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]}]},{"osd":8,"up_from":59,"seq":253403070496,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":974848,"data_stored":477189,"data_compressed":11642,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.53600000000000003,"15min":0.54800000000000004},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.60199999999999998,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.54900000000000004,"15min":0.56200000000000006},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.68600000000000005,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.53500000000000003}]},{"osd":1,"last update":"Fri Sep 13 18:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56000000000000005,"15min":0.55900000000000005},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56999999999999995,"15min":0.57099999999999995},"min":{"1min":0.40600000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72199999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.73699999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.42599999999999999,"15min":0.42999999999999999},"min":{"1min":0.26600000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.55200000000000005,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.247,"5min":0.224,"15min":0.224},"max":{"1min":0.59799999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.48399999999999999}]},{"osd":3,"last update":"Fri Sep 13 18:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.54700000000000004,"15min":0.55800000000000005},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.71999999999999997,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53800000000000003,"15min":0.53900000000000003},"min":{"1min":0.38600000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.44600000000000001}]},{"osd":4,"last update":"Fri Sep 13 18:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.66400000000000003,"15min":0.63600000000000001},"min":{"1min":0.45100000000000001,"5min":0.37,"15min":0.37},"max":{"1min":5.3540000000000001,"5min":5.3540000000000001,"15min":5.3540000000000001},"last":0.68000000000000005}]},{"osd":5,"last update":"Fri Sep 13 18:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.53100000000000003,"15min":0.53700000000000003},"min":{"1min":0.31900000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.70099999999999996,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.51900000000000002,"15min":0.52300000000000002},"min":{"1min":0.32100000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.63500000000000001,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.40100000000000002}]},{"osd":6,"last update":"Fri Sep 13 18:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.60599999999999998,"15min":0.61199999999999999},"min":{"1min":0.35199999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.75600000000000001,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.64400000000000002,"15min":0.65000000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.77200000000000002,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.65800000000000003}]},{"osd":7,"last update":"Fri Sep 13 18:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.66600000000000004,"15min":0.67500000000000004},"min":{"1min":0.44800000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.749,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.65600000000000003,"15min":0.66300000000000003},"min":{"1min":0.39700000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.73899999999999999,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.62}]},{"osd":9,"last update":"Fri Sep 13 18:20:24 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63800000000000001,"15min":0.63300000000000001},"min":{"1min":0.46600000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.65900000000000003,"15min":0.65600000000000003},"min":{"1min":0.52800000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.95399999999999996,"5min":1.012,"15min":1.012},"last":0.57899999999999996}]},{"osd":10,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":5.3360000000000003,"5min":5.3360000000000003,"15min":5.3360000000000003},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.72499999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]}]},{"osd":7,"up_from":51,"seq":219043332132,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":1024000,"data_stored":1068087,"data_compressed":47466,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.94399999999999995,"15min":1.012},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.84299999999999997,"5min":12.068,"15min":12.068},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.46999999999999997,"15min":0.45900000000000002},"min":{"1min":0.35199999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.61599999999999999}]},{"osd":1,"last update":"Fri Sep 13 18:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.36499999999999999,"15min":0.35699999999999998},"min":{"1min":0.26100000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.38100000000000001,"15min":0.375},"min":{"1min":0.27800000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.62,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.44900000000000001}]},{"osd":2,"last update":"Fri Sep 13 18:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.59099999999999997,"15min":0.53700000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":7.6820000000000004,"5min":7.6820000000000004,"15min":7.6820000000000004},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.51600000000000001,"15min":0.50700000000000001},"min":{"1min":0.378,"5min":0.371,"15min":0.371},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.58799999999999997}]},{"osd":3,"last update":"Fri Sep 13 18:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.49299999999999999,"15min":0.48099999999999998},"min":{"1min":0.36299999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.754},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.49299999999999999,"15min":0.48299999999999998},"min":{"1min":0.35399999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.63100000000000001}]},{"osd":4,"last update":"Fri Sep 13 18:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.44,"15min":0.433},"min":{"1min":0.29099999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.621,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.45400000000000001,"15min":0.45000000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.439}]},{"osd":5,"last update":"Fri Sep 13 18:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.55700000000000005,"15min":0.55000000000000004},"min":{"1min":0.39900000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.53700000000000003,"15min":0.53100000000000003},"min":{"1min":0.39900000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.68200000000000005}]},{"osd":6,"last update":"Fri Sep 13 18:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.55000000000000004,"15min":0.54000000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.59299999999999997,"15min":0.58199999999999996},"min":{"1min":0.48099999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.77700000000000002}]},{"osd":8,"last update":"Fri Sep 13 18:20:01 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.58999999999999997,"15min":0.58699999999999997},"min":{"1min":0.42499999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.72999999999999998,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.76600000000000001},{"interface":"front","average":{"1min":1.0669999999999999,"5min":0.71899999999999997,"15min":0.66100000000000003},"min":{"1min":0.443,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":7.7149999999999999,"5min":7.7149999999999999,"15min":7.7149999999999999},"last":0.71999999999999997}]},{"osd":9,"last update":"Fri Sep 13 18:19:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.84599999999999997}]},{"osd":10,"last update":"Fri Sep 13 18:19:50 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.64600000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]}]},{"osd":6,"up_from":46,"seq":197568495658,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":1003520,"data_stored":496015,"data_compressed":17567,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.248,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.43099999999999999,"15min":0.437},"min":{"1min":0.27600000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.218}]},{"osd":1,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.52700000000000002,"15min":0.52200000000000002},"min":{"1min":0.35499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.69199999999999995,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55000000000000004,"15min":0.53600000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.66500000000000004}]},{"osd":2,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.54200000000000004,"15min":0.54600000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53600000000000003,"15min":0.54700000000000004},"min":{"1min":0.36599999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.75,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.52000000000000002}]},{"osd":3,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42299999999999999,"15min":0.42599999999999999},"min":{"1min":0.28999999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.69499999999999995,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.309},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.46700000000000003,"15min":0.47099999999999997},"min":{"1min":0.30099999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.35599999999999998}]},{"osd":4,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.52300000000000002,"15min":0.51500000000000001},"min":{"1min":0.33300000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55500000000000005,"15min":0.55300000000000005},"min":{"1min":0.39600000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.74199999999999999,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.68300000000000005}]},{"osd":5,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59899999999999998,"15min":0.60799999999999998},"min":{"1min":0.436,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.57399999999999995,"15min":0.59299999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.77900000000000003,"5min":0.874,"15min":0.874},"last":0.496}]},{"osd":7,"last update":"Fri Sep 13 18:19:43 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57399999999999995,"15min":0.57299999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.71299999999999997,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56599999999999995,"15min":0.56200000000000006},"min":{"1min":0.435,"5min":0.434,"15min":0.434},"max":{"1min":0.77100000000000002,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.56899999999999995}]},{"osd":8,"last update":"Fri Sep 13 18:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.61799999999999999,"15min":0.621},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.79000000000000004,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.59399999999999997,"15min":0.59899999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.82999999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.65200000000000002}]},{"osd":9,"last update":"Fri Sep 13 18:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.39500000000000002}]},{"osd":10,"last update":"Fri Sep 13 18:19:49 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.58199999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]}]},{"osd":5,"up_from":38,"seq":163208757294,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":1068359,"data_compressed":47466,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:19:58 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.49299999999999999,"15min":0.49399999999999999},"min":{"1min":0.35299999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.61099999999999999,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.51400000000000001,"15min":0.51200000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.66600000000000004,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.52800000000000002}]},{"osd":1,"last update":"Fri Sep 13 18:19:58 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.51600000000000001,"15min":0.502},"min":{"1min":0.41899999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.77200000000000002,"5min":0.878,"15min":0.878},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.52000000000000002,"15min":0.51000000000000001},"min":{"1min":0.34399999999999997,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.78600000000000003,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.59199999999999997}]},{"osd":2,"last update":"Fri Sep 13 18:19:58 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.376,"15min":0.36199999999999999},"min":{"1min":0.27700000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.39000000000000001,"15min":0.38400000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.58499999999999996,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.41199999999999998}]},{"osd":3,"last update":"Fri Sep 13 18:19:58 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.51600000000000001,"15min":0.501},"min":{"1min":0.35799999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.51500000000000001,"15min":0.51800000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.70499999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.505}]},{"osd":4,"last update":"Fri Sep 13 18:19:58 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.54700000000000004,"15min":0.53800000000000003},"min":{"1min":0.40400000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.75900000000000001,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.76400000000000001,"15min":0.60799999999999998},"min":{"1min":0.376,"5min":0.374,"15min":0.374},"max":{"1min":0.80200000000000005,"5min":18.571000000000002,"15min":18.571000000000002},"last":0.48199999999999998}]},{"osd":6,"last update":"Fri Sep 13 18:20:22 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.42399999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55400000000000005,"15min":0.55000000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.629}]},{"osd":7,"last update":"Fri Sep 13 18:19:43 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.61499999999999999,"15min":0.60799999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.61699999999999999,"15min":0.61099999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.85299999999999998,"5min":0.88,"15min":0.88},"last":0.67600000000000005}]},{"osd":8,"last update":"Fri Sep 13 18:20:02 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.51700000000000002,"15min":0.51300000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.73699999999999999,"5min":0.753,"15min":0.753},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.54300000000000004,"15min":0.53700000000000003},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.69099999999999995}]},{"osd":9,"last update":"Fri Sep 13 18:19:27 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.70199999999999996}]},{"osd":10,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.72499999999999998}]}]},{"osd":4,"up_from":33,"seq":141733920818,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":486669,"data_compressed":14601,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:19:33 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.48299999999999998,"15min":0.47699999999999998},"min":{"1min":0.35399999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.69399999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.433},{"interface":"front","average":{"1min":0.501,"5min":0.51200000000000001,"15min":0.47499999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.84599999999999997,"5min":4.2539999999999996,"15min":4.2539999999999996},"last":0.51900000000000002}]},{"osd":1,"last update":"Fri Sep 13 18:19:33 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.34899999999999998,"15min":0.34399999999999997},"min":{"1min":0.27600000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.372,"5min":0.34699999999999998,"15min":0.34300000000000003},"min":{"1min":0.25600000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.52700000000000002,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.42099999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:19:33 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.53100000000000003,"15min":0.52300000000000002},"min":{"1min":0.40999999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.53700000000000003,"15min":0.53300000000000003},"min":{"1min":0.38500000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.74399999999999999}]},{"osd":3,"last update":"Fri Sep 13 18:19:33 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.49299999999999999},"min":{"1min":0.35199999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.72599999999999998,"5min":4.2679999999999998,"15min":4.2679999999999998},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.48899999999999999,"15min":0.48499999999999999},"min":{"1min":0.40200000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.69299999999999995}]},{"osd":5,"last update":"Fri Sep 13 18:19:55 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.54800000000000004,"15min":0.53500000000000003},"min":{"1min":0.38500000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78100000000000003,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.55000000000000004,"15min":0.53700000000000003},"min":{"1min":0.39900000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.71099999999999997,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.59599999999999997}]},{"osd":6,"last update":"Fri Sep 13 18:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.52300000000000002,"15min":0.51100000000000001},"min":{"1min":0.41999999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.52300000000000002,"15min":0.50900000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.76600000000000001}]},{"osd":7,"last update":"Fri Sep 13 18:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.44400000000000001,"15min":0.438},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.48499999999999999,"15min":0.47699999999999998},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.48899999999999999}]},{"osd":8,"last update":"Fri Sep 13 18:20:07 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.63,"15min":0.626},"min":{"1min":0.505,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.85899999999999999,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.65000000000000002,"15min":0.64500000000000002},"min":{"1min":0.52300000000000002,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.70599999999999996}]},{"osd":9,"last update":"Fri Sep 13 18:19:25 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.77900000000000003}]},{"osd":10,"last update":"Fri Sep 13 18:19:51 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.625},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.63900000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]}]},{"osd":3,"up_from":25,"seq":107374182455,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":495344,"data_compressed":17567,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:20:10 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.32900000000000001,"15min":0.307},"min":{"1min":0.248,"5min":0.183,"15min":0.183},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.33900000000000002,"15min":0.32700000000000001},"min":{"1min":0.222,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.47599999999999998,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.314}]},{"osd":1,"last update":"Fri Sep 13 18:20:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.50900000000000001,"15min":0.48799999999999999},"min":{"1min":0.40000000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.65600000000000003,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.51400000000000001,"15min":0.48799999999999999},"min":{"1min":0.39400000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.71299999999999997,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.61599999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:20:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52400000000000002,"15min":0.52200000000000002},"min":{"1min":0.35999999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.68700000000000006,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52300000000000002,"15min":0.501},"min":{"1min":0.36599999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.73399999999999999,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.45800000000000002}]},{"osd":4,"last update":"Fri Sep 13 18:19:32 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.5,"15min":0.48599999999999999},"min":{"1min":0.374,"5min":0.312,"15min":0.312},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.501},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.52400000000000002,"15min":0.51400000000000001},"min":{"1min":0.372,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.749,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.433}]},{"osd":5,"last update":"Fri Sep 13 18:19:54 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54000000000000004,"15min":0.53500000000000003},"min":{"1min":0.38300000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.749,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.39600000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72899999999999998,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.51500000000000001}]},{"osd":6,"last update":"Fri Sep 13 18:20:23 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.443,"15min":0.44700000000000001},"min":{"1min":0.30299999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.59199999999999997,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.48099999999999998,"15min":0.46700000000000003},"min":{"1min":0.313,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.70699999999999996,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.443}]},{"osd":7,"last update":"Fri Sep 13 18:19:39 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.80400000000000005,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.57999999999999996,"15min":0.57399999999999995},"min":{"1min":0.44800000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.77500000000000002,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.60899999999999999}]},{"osd":8,"last update":"Fri Sep 13 18:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.69599999999999995}]},{"osd":9,"last update":"Fri Sep 13 18:19:27 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.55900000000000005}]},{"osd":10,"last update":"Fri Sep 13 18:19:49 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.626}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]}]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":480392,"data_compressed":11642,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:19:45 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47399999999999998,"15min":0.47899999999999998},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.57599999999999996,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.495},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.48099999999999998,"15min":0.47599999999999998},"min":{"1min":0.318,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.61899999999999999}]},{"osd":1,"last update":"Fri Sep 13 18:19:45 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.49199999999999999,"15min":0.47199999999999998},"min":{"1min":0.34999999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.78400000000000003,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.49099999999999999,"15min":0.46500000000000002},"min":{"1min":0.32300000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.78100000000000003,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.53600000000000003}]},{"osd":3,"last update":"Fri Sep 13 18:20:13 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.52700000000000002,"15min":0.51600000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.66100000000000003,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.55200000000000005,"15min":0.55100000000000005},"min":{"1min":0.377,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.67100000000000004,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.75700000000000001}]},{"osd":4,"last update":"Fri Sep 13 18:19:35 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.504,"15min":0.49099999999999999},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.72899999999999998,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.50900000000000001,"15min":0.503},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.68899999999999995,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.54700000000000004}]},{"osd":5,"last update":"Fri Sep 13 18:19:52 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.38400000000000001,"15min":0.375},"min":{"1min":0.30399999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.371},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.40000000000000002,"15min":0.39000000000000001},"min":{"1min":0.26900000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.58899999999999997,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.31900000000000001}]},{"osd":6,"last update":"Fri Sep 13 18:20:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.56299999999999994,"15min":0.54000000000000004},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.57799999999999996,"15min":0.56299999999999994},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.80100000000000005,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.70199999999999996}]},{"osd":7,"last update":"Fri Sep 13 18:19:37 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78200000000000003,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.61599999999999999,"15min":0.61899999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.76600000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.52300000000000002}]},{"osd":8,"last update":"Fri Sep 13 18:20:07 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.54000000000000004,"15min":0.54200000000000004},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.73899999999999999,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.36599999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.44700000000000001}]},{"osd":9,"last update":"Fri Sep 13 18:19:25 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.56399999999999995}]},{"osd":10,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.57399999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]}]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":1007616,"data_stored":499248,"data_compressed":17567,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:20:24 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.502,"15min":0.495},"min":{"1min":0.35499999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.626},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.52300000000000002,"15min":0.51500000000000001},"min":{"1min":0.33200000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.77800000000000002,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.439}]},{"osd":2,"last update":"Fri Sep 13 18:19:46 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.501,"15min":0.48999999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.46999999999999997,"15min":0.46899999999999997},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.64200000000000002,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.496}]},{"osd":3,"last update":"Fri Sep 13 18:20:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.53900000000000003,"15min":0.52100000000000002},"min":{"1min":0.38600000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.74399999999999999,"5min":0.749,"15min":0.749},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56499999999999995,"15min":0.54400000000000004},"min":{"1min":0.39700000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76900000000000002,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.52700000000000002}]},{"osd":4,"last update":"Fri Sep 13 18:19:32 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.38,"15min":0.36099999999999999},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.434,"5min":0.376,"15min":0.35599999999999998},"min":{"1min":0.28399999999999997,"5min":0.217,"15min":0.217},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.40500000000000003}]},{"osd":5,"last update":"Fri Sep 13 18:19:53 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56699999999999995,"15min":0.56200000000000006},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55100000000000005,"15min":0.53700000000000003},"min":{"1min":0.40500000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.71299999999999997}]},{"osd":6,"last update":"Fri Sep 13 18:20:23 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57699999999999996,"15min":0.56699999999999995},"min":{"1min":0.41299999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.621,"15min":0.61099999999999999},"min":{"1min":0.45100000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.88300000000000001,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.72899999999999998}]},{"osd":7,"last update":"Fri Sep 13 18:19:38 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.49299999999999999,"15min":0.48699999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.502,"15min":0.5},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.745}]},{"osd":8,"last update":"Fri Sep 13 18:20:07 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.627,"15min":0.629},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.66000000000000003}]},{"osd":9,"last update":"Fri Sep 13 18:19:27 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.70099999999999996}]},{"osd":10,"last update":"Fri Sep 13 18:19:49 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.76600000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":995328,"data_stored":487439,"data_compressed":14601,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 18:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.501,"15min":0.496},"min":{"1min":0.35599999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.37},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.50700000000000001,"15min":0.505},"min":{"1min":0.35699999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.69199999999999995,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.504}]},{"osd":2,"last update":"Fri Sep 13 18:19:47 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.52900000000000003,"15min":0.52700000000000002},"min":{"1min":0.34399999999999997,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.70099999999999996,"5min":0.754,"15min":0.754},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.53000000000000003,"15min":0.51100000000000001},"min":{"1min":0.38500000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.81799999999999995,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.47699999999999998}]},{"osd":3,"last update":"Fri Sep 13 18:19:05 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.38400000000000001,"15min":0.378},"min":{"1min":0.26300000000000001,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.437,"5min":0.39400000000000002,"15min":0.376},"min":{"1min":0.28100000000000003,"5min":0.251,"15min":0.251},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.28799999999999998}]},{"osd":4,"last update":"Fri Sep 13 18:19:33 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.503,"15min":0.49099999999999999},"min":{"1min":0.38600000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.85699999999999998,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.51900000000000002,"15min":0.50700000000000001},"min":{"1min":0.35499999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.45100000000000001}]},{"osd":5,"last update":"Fri Sep 13 18:19:53 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.61499999999999999,"15min":0.56699999999999995},"min":{"1min":0.371,"5min":0.313,"15min":0.313},"max":{"1min":0.89600000000000002,"5min":7.194,"15min":7.194},"last":0.495},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.55100000000000005,"15min":0.52300000000000002},"min":{"1min":0.378,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.87,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.53900000000000003}]},{"osd":6,"last update":"Fri Sep 13 18:20:21 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.46400000000000002,"15min":0.45400000000000001},"min":{"1min":0.28899999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.79000000000000004,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.45800000000000002,"15min":0.46100000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.56699999999999995,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.32400000000000001}]},{"osd":7,"last update":"Fri Sep 13 18:19:38 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.58599999999999997,"15min":0.57599999999999996},"min":{"1min":0.435,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.55500000000000005,"15min":0.54600000000000004},"min":{"1min":0.47299999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.437}]},{"osd":8,"last update":"Fri Sep 13 18:20:01 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.60799999999999998,"15min":0.60999999999999999},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61899999999999999,"15min":0.61699999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.52900000000000003}]},{"osd":9,"last update":"Fri Sep 13 18:19:28 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.41899999999999998}]},{"osd":10,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.55400000000000005}]},{"osd":11,"last update":"Fri Sep 13 18:20:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.621},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.66300000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34188,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34188,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34188,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35824,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35824,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35824,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35824,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35824,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":12212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":12084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:20:31.362 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.6 is not active+clean 2024-09-13T18:20:31.362 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.6', 'version': "0'0", 'reported_seq': 20, 'reported_epoch': 91, 'state': 'activating', 'last_fresh': '2024-09-13T18:20:27.181672+0000', 'last_change': '2024-09-13T18:20:27.181672+0000', 'last_active': '2024-09-13T18:20:26.189486+0000', 'last_peered': '2024-09-13T18:20:06.959341+0000', 'last_clean': '2024-09-13T18:20:06.959341+0000', 'last_became_active': '2024-09-13T18:20:01.922815+0000', 'last_became_peered': '2024-09-13T18:20:01.922815+0000', 'last_unstale': '2024-09-13T18:20:27.181672+0000', 'last_undegraded': '2024-09-13T18:20:27.181672+0000', 'last_fullsized': '2024-09-13T18:20:27.181672+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 83, 'last_epoch_clean': 84, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T18:20:01.553959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T18:20:01.553959+0000', 'last_clean_scrub_stamp': '2024-09-13T18:20:01.553959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-15T04:59:57.606105+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3, 7], 'acting': [0, 3, 7], 'avail_no_missing': ['0', '3', '4', '7'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-13T18:20:31.363 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-13T18:20:31.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.489+0000 7f4e99d18640 1 -- 172.21.15.114:0/847818087 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e94100830 msgr2=0x7f4e941070c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:31.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.489+0000 7f4e99d18640 1 --2- 172.21.15.114:0/847818087 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e94100830 0x7f4e941070c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c009ca0 tx=0x7f4e8c02a8c0 comp rx=0 tx=0).stop 2024-09-13T18:20:31.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.489+0000 7f4e99d18640 1 -- 172.21.15.114:0/847818087 shutdown_connections 2024-09-13T18:20:31.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.489+0000 7f4e99d18640 1 --2- 172.21.15.114:0/847818087 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e94100830 0x7f4e941070c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:31.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.489+0000 7f4e99d18640 1 --2- 172.21.15.114:0/847818087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e940ffe70 0x7f4e941002f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:31.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.489+0000 7f4e99d18640 1 --2- 172.21.15.114:0/847818087 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e940fec80 0x7f4e940ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:31.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.489+0000 7f4e99d18640 1 -- 172.21.15.114:0/847818087 >> 172.21.15.114:0/847818087 conn(0x7f4e940fa400 msgr2=0x7f4e940fc820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:31.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.489+0000 7f4e99d18640 1 -- 172.21.15.114:0/847818087 shutdown_connections 2024-09-13T18:20:31.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.489+0000 7f4e99d18640 1 -- 172.21.15.114:0/847818087 wait complete. 2024-09-13T18:20:31.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.489+0000 7f4e99d18640 1 Processor -- start 2024-09-13T18:20:31.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.489+0000 7f4e99d18640 1 -- start start 2024-09-13T18:20:31.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.489+0000 7f4e99d18640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e940fec80 0x7f4e94195bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:31.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.489+0000 7f4e99d18640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e940ffe70 0x7f4e941960f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:31.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.490+0000 7f4e99d18640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e94100830 0x7f4e9419d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:31.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.490+0000 7f4e99d18640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9419d6a0 con 0x7f4e94100830 2024-09-13T18:20:31.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.490+0000 7f4e99d18640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9419d810 con 0x7f4e940ffe70 2024-09-13T18:20:31.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.490+0000 7f4e99d18640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9419d9d0 con 0x7f4e940fec80 2024-09-13T18:20:31.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.490+0000 7f4e93fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e94100830 0x7f4e9419d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:31.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.490+0000 7f4e93fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e94100830 0x7f4e9419d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60852/0 (socket says 172.21.15.114:60852) 2024-09-13T18:20:31.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.490+0000 7f4e93fff640 1 -- 172.21.15.114:0/2650764674 learned_addr learned my addr 172.21.15.114:0/2650764674 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:20:31.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.490+0000 7f4e937fe640 1 --2- 172.21.15.114:0/2650764674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e940fec80 0x7f4e94195bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:31.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.490+0000 7f4e92ffd640 1 --2- 172.21.15.114:0/2650764674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e940ffe70 0x7f4e941960f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:31.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.490+0000 7f4e93fff640 1 -- 172.21.15.114:0/2650764674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e940fec80 msgr2=0x7f4e94195bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:31.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.490+0000 7f4e93fff640 1 --2- 172.21.15.114:0/2650764674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e940fec80 0x7f4e94195bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:31.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.490+0000 7f4e93fff640 1 -- 172.21.15.114:0/2650764674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e940ffe70 msgr2=0x7f4e941960f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:31.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.490+0000 7f4e93fff640 1 --2- 172.21.15.114:0/2650764674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e940ffe70 0x7f4e941960f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:31.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.490+0000 7f4e93fff640 1 -- 172.21.15.114:0/2650764674 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e8c009900 con 0x7f4e94100830 2024-09-13T18:20:31.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.490+0000 7f4e93fff640 1 --2- 172.21.15.114:0/2650764674 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e94100830 0x7f4e9419d160 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c003ba0 tx=0x7f4e8c003bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:31.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.490+0000 7f4e90ff9640 1 -- 172.21.15.114:0/2650764674 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c0317c0 con 0x7f4e94100830 2024-09-13T18:20:31.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.490+0000 7f4e90ff9640 1 -- 172.21.15.114:0/2650764674 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4e8c040070 con 0x7f4e94100830 2024-09-13T18:20:31.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.490+0000 7f4e99d18640 1 -- 172.21.15.114:0/2650764674 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e9419dc50 con 0x7f4e94100830 2024-09-13T18:20:31.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.490+0000 7f4e90ff9640 1 -- 172.21.15.114:0/2650764674 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c004720 con 0x7f4e94100830 2024-09-13T18:20:31.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.490+0000 7f4e99d18640 1 -- 172.21.15.114:0/2650764674 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e9419e1b0 con 0x7f4e94100830 2024-09-13T18:20:31.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.491+0000 7f4e99d18640 1 -- 172.21.15.114:0/2650764674 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e9410a3d0 con 0x7f4e94100830 2024-09-13T18:20:31.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.492+0000 7f4e90ff9640 1 -- 172.21.15.114:0/2650764674 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f4e8c042030 con 0x7f4e94100830 2024-09-13T18:20:31.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.493+0000 7f4e90ff9640 1 --2- 172.21.15.114:0/2650764674 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f4e7c0a11a0 0x7f4e7c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:31.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.493+0000 7f4e937fe640 1 --2- 172.21.15.114:0/2650764674 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f4e7c0a11a0 0x7f4e7c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:31.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.493+0000 7f4e90ff9640 1 -- 172.21.15.114:0/2650764674 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4e8c0ee500 con 0x7f4e94100830 2024-09-13T18:20:31.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.493+0000 7f4e937fe640 1 --2- 172.21.15.114:0/2650764674 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f4e7c0a11a0 0x7f4e7c0a3660 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4e940ffcd0 tx=0x7f4e84005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:31.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.497+0000 7f4e90ff9640 1 -- 172.21.15.114:0/2650764674 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f4e8c0b7930 con 0x7f4e94100830 2024-09-13T18:20:31.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:31 smithi134 ceph-mon[34283]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-13T18:20:31.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:31 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/2106708598' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:20:31.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:31 smithi134 ceph-mon[34283]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T18:20:31.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:31 smithi134 ceph-mon[34283]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T18:20:31.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:31 smithi204 ceph-mon[34777]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-13T18:20:31.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:31 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/2106708598' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:20:31.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:31 smithi204 ceph-mon[34777]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T18:20:31.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:31 smithi204 ceph-mon[34777]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T18:20:31.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.721+0000 7f4e99d18640 1 -- 172.21.15.114:0/2650764674 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f4e9410a5b0 con 0x7f4e94100830 2024-09-13T18:20:31.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.722+0000 7f4e90ff9640 1 -- 172.21.15.114:0/2650764674 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+3499 (secure 0 0 0) 0x7f4e8c0baf50 con 0x7f4e94100830 2024-09-13T18:20:31.725 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:20:31.725 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"97e7edf2-71fb-11ef-bceb-c7b262605968","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","c","b"],"quorum_age":368,"monmap":{"epoch":3,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":91,"num_osds":12,"num_up_osds":12,"osd_up_since":1726251552,"num_in_osds":12,"osd_in_since":1726251532,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":128},{"state_name":"activating","count":1}],"num_pgs":129,"num_pools":4,"num_objects":63,"data_bytes":603786,"bytes_used":369291264,"bytes_avail":1151571136512,"bytes_total":1151940427776,"inactive_pgs_ratio":0.0077519379556179047,"recovering_objects_per_sec":0,"recovering_bytes_per_sec":2,"recovering_keys_per_sec":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"read_bytes_sec":2175,"write_bytes_sec":4222,"read_op_per_sec":1,"write_op_per_sec":11},"fsmap":{"epoch":29,"id":1,"up":5,"in":5,"max":5,"by_rank":[{"filesystem_id":1,"rank":4,"name":"g","status":"up:standby-replay","gid":14616},{"filesystem_id":1,"rank":4,"name":"a","status":"up:active","gid":14628},{"filesystem_id":1,"rank":3,"name":"l","status":"up:standby-replay","gid":24391},{"filesystem_id":1,"rank":2,"name":"k","status":"up:standby-replay","gid":24422},{"filesystem_id":1,"rank":1,"name":"f","status":"up:standby-replay","gid":24445},{"filesystem_id":1,"rank":0,"name":"c","status":"up:standby-replay","gid":24457},{"filesystem_id":1,"rank":3,"name":"e","status":"up:active","gid":24458},{"filesystem_id":1,"rank":2,"name":"i","status":"up:active","gid":24475},{"filesystem_id":1,"rank":1,"name":"h","status":"up:active","gid":24482},{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":24521}],"up:standby":2},"mgrmap":{"available":true,"num_standbys":2,"modules":["cephadm","dashboard","iostat","nfs","restful"],"services":{"dashboard":"https://172.21.15.114:8443/"}},"servicemap":{"epoch":19,"modified":"2024-09-13T18:20:27.908252+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"e":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"f":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"g":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"h":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"i":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"j":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"k":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-13T18:20:31.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.724+0000 7f4e99d18640 1 -- 172.21.15.114:0/2650764674 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f4e7c0a11a0 msgr2=0x7f4e7c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:31.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.724+0000 7f4e99d18640 1 --2- 172.21.15.114:0/2650764674 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f4e7c0a11a0 0x7f4e7c0a3660 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4e940ffcd0 tx=0x7f4e84005f90 comp rx=0 tx=0).stop 2024-09-13T18:20:31.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.724+0000 7f4e99d18640 1 -- 172.21.15.114:0/2650764674 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e94100830 msgr2=0x7f4e9419d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:31.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.724+0000 7f4e99d18640 1 --2- 172.21.15.114:0/2650764674 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e94100830 0x7f4e9419d160 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c003ba0 tx=0x7f4e8c003bd0 comp rx=0 tx=0).stop 2024-09-13T18:20:31.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.724+0000 7f4e99d18640 1 -- 172.21.15.114:0/2650764674 shutdown_connections 2024-09-13T18:20:31.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.724+0000 7f4e99d18640 1 --2- 172.21.15.114:0/2650764674 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f4e7c0a11a0 0x7f4e7c0a3660 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:31.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.724+0000 7f4e99d18640 1 --2- 172.21.15.114:0/2650764674 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e94100830 0x7f4e9419d160 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:31.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.724+0000 7f4e99d18640 1 --2- 172.21.15.114:0/2650764674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e940ffe70 0x7f4e941960f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:31.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.724+0000 7f4e99d18640 1 --2- 172.21.15.114:0/2650764674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e940fec80 0x7f4e94195bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:31.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.724+0000 7f4e99d18640 1 -- 172.21.15.114:0/2650764674 >> 172.21.15.114:0/2650764674 conn(0x7f4e940fa400 msgr2=0x7f4e94067670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:31.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.724+0000 7f4e99d18640 1 -- 172.21.15.114:0/2650764674 shutdown_connections 2024-09-13T18:20:31.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.724+0000 7f4e99d18640 1 -- 172.21.15.114:0/2650764674 wait complete. 2024-09-13T18:20:31.738 INFO:tasks.cephfs.filesystem.ceph_manager:making progress, resetting timeout 2024-09-13T18:20:31.738 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:20:31.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.833+0000 7fdceb2e5640 1 -- 172.21.15.114:0/777912125 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdce4100820 msgr2=0x7fdce4102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:31.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.833+0000 7fdceb2e5640 1 --2- 172.21.15.114:0/777912125 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdce4100820 0x7fdce4102c10 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4009ca0 tx=0x7fdcd402a8c0 comp rx=0 tx=0).stop 2024-09-13T18:20:31.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.834+0000 7fdceb2e5640 1 -- 172.21.15.114:0/777912125 shutdown_connections 2024-09-13T18:20:31.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.834+0000 7fdceb2e5640 1 --2- 172.21.15.114:0/777912125 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdce4105a80 0x7fdce4107ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:31.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.834+0000 7fdceb2e5640 1 --2- 172.21.15.114:0/777912125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdce4103150 0x7fdce4105540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:31.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.834+0000 7fdceb2e5640 1 --2- 172.21.15.114:0/777912125 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdce4100820 0x7fdce4102c10 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:31.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.834+0000 7fdceb2e5640 1 -- 172.21.15.114:0/777912125 >> 172.21.15.114:0/777912125 conn(0x7fdce40fa440 msgr2=0x7fdce40fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:31.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.834+0000 7fdceb2e5640 1 -- 172.21.15.114:0/777912125 shutdown_connections 2024-09-13T18:20:31.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.834+0000 7fdceb2e5640 1 -- 172.21.15.114:0/777912125 wait complete. 2024-09-13T18:20:31.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.834+0000 7fdceb2e5640 1 Processor -- start 2024-09-13T18:20:31.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.834+0000 7fdceb2e5640 1 -- start start 2024-09-13T18:20:31.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.834+0000 7fdceb2e5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdce4100820 0x7fdce4195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:31.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.834+0000 7fdceb2e5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdce4103150 0x7fdce4196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:31.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.834+0000 7fdceb2e5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdce4105a80 0x7fdce419d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:31.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.834+0000 7fdceb2e5640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce419d830 con 0x7fdce4103150 2024-09-13T18:20:31.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.834+0000 7fdceb2e5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce419d9a0 con 0x7fdce4105a80 2024-09-13T18:20:31.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.834+0000 7fdceb2e5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce419db60 con 0x7fdce4100820 2024-09-13T18:20:31.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.834+0000 7fdce8880640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdce4103150 0x7fdce4196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:31.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.834+0000 7fdce8880640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdce4103150 0x7fdce4196280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60870/0 (socket says 172.21.15.114:60870) 2024-09-13T18:20:31.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.834+0000 7fdce8880640 1 -- 172.21.15.114:0/983564016 learned_addr learned my addr 172.21.15.114:0/983564016 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:20:31.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.835+0000 7fdce9081640 1 --2- 172.21.15.114:0/983564016 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdce4100820 0x7fdce4195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:31.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.835+0000 7fdce8880640 1 -- 172.21.15.114:0/983564016 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdce4100820 msgr2=0x7fdce4195d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:31.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.835+0000 7fdce8880640 1 --2- 172.21.15.114:0/983564016 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdce4100820 0x7fdce4195d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:31.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.835+0000 7fdce8880640 1 -- 172.21.15.114:0/983564016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdce4105a80 msgr2=0x7fdce419d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:31.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.835+0000 7fdce9882640 1 --2- 172.21.15.114:0/983564016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdce4105a80 0x7fdce419d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:31.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.835+0000 7fdce8880640 1 --2- 172.21.15.114:0/983564016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdce4105a80 0x7fdce419d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:31.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.835+0000 7fdce8880640 1 -- 172.21.15.114:0/983564016 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd4009900 con 0x7fdce4103150 2024-09-13T18:20:31.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.835+0000 7fdce9081640 1 --2- 172.21.15.114:0/983564016 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdce4100820 0x7fdce4195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:20:31.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.835+0000 7fdce8880640 1 --2- 172.21.15.114:0/983564016 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdce4103150 0x7fdce4196280 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8005fd0 tx=0x7fdcd800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:31.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.835+0000 7fdce27fc640 1 -- 172.21.15.114:0/983564016 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd8009c30 con 0x7fdce4103150 2024-09-13T18:20:31.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.835+0000 7fdce27fc640 1 -- 172.21.15.114:0/983564016 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdcd80044b0 con 0x7fdce4103150 2024-09-13T18:20:31.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.835+0000 7fdceb2e5640 1 -- 172.21.15.114:0/983564016 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdce419de40 con 0x7fdce4103150 2024-09-13T18:20:31.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.835+0000 7fdce27fc640 1 -- 172.21.15.114:0/983564016 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd8003da0 con 0x7fdce4103150 2024-09-13T18:20:31.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.835+0000 7fdceb2e5640 1 -- 172.21.15.114:0/983564016 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdce419e2c0 con 0x7fdce4103150 2024-09-13T18:20:31.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.836+0000 7fdceb2e5640 1 -- 172.21.15.114:0/983564016 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdce40fbf30 con 0x7fdce4103150 2024-09-13T18:20:31.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.837+0000 7fdce27fc640 1 -- 172.21.15.114:0/983564016 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fdcd8004000 con 0x7fdce4103150 2024-09-13T18:20:31.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.838+0000 7fdce27fc640 1 --2- 172.21.15.114:0/983564016 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fdccc0a11a0 0x7fdccc0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:31.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.838+0000 7fdce9081640 1 --2- 172.21.15.114:0/983564016 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fdccc0a11a0 0x7fdccc0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:31.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.838+0000 7fdce27fc640 1 -- 172.21.15.114:0/983564016 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fdcd80ccba0 con 0x7fdce4103150 2024-09-13T18:20:31.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.838+0000 7fdce9081640 1 --2- 172.21.15.114:0/983564016 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fdccc0a11a0 0x7fdccc0a3660 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4031040 tx=0x7fdcd402f310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:31.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:31.842+0000 7fdce27fc640 1 -- 172.21.15.114:0/983564016 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fdcd8095e60 con 0x7fdce4103150 2024-09-13T18:20:32.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:32.003+0000 7fdceb2e5640 1 -- 172.21.15.114:0/983564016 --> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdce402c480 con 0x7fdccc0a11a0 2024-09-13T18:20:32.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:32.014+0000 7fdce27fc640 1 -- 172.21.15.114:0/983564016 <== mgr.14148 v2:172.21.15.114:6800/3396185541 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+424103 (secure 0 0 0) 0x7fdce402c480 con 0x7fdccc0a11a0 2024-09-13T18:20:32.022 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:20:32.024 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-13T18:20:32.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:32.018+0000 7fdceb2e5640 1 -- 172.21.15.114:0/983564016 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fdccc0a11a0 msgr2=0x7fdccc0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:32.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:32.018+0000 7fdceb2e5640 1 --2- 172.21.15.114:0/983564016 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fdccc0a11a0 0x7fdccc0a3660 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4031040 tx=0x7fdcd402f310 comp rx=0 tx=0).stop 2024-09-13T18:20:32.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:32.018+0000 7fdceb2e5640 1 -- 172.21.15.114:0/983564016 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdce4103150 msgr2=0x7fdce4196280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:32.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:32.018+0000 7fdceb2e5640 1 --2- 172.21.15.114:0/983564016 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdce4103150 0x7fdce4196280 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8005fd0 tx=0x7fdcd800dde0 comp rx=0 tx=0).stop 2024-09-13T18:20:32.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:32.018+0000 7fdceb2e5640 1 -- 172.21.15.114:0/983564016 shutdown_connections 2024-09-13T18:20:32.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:32.018+0000 7fdceb2e5640 1 --2- 172.21.15.114:0/983564016 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fdccc0a11a0 0x7fdccc0a3660 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:32.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:32.018+0000 7fdceb2e5640 1 --2- 172.21.15.114:0/983564016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdce4105a80 0x7fdce419d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:32.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:32.018+0000 7fdceb2e5640 1 --2- 172.21.15.114:0/983564016 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdce4103150 0x7fdce4196280 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:32.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:32.018+0000 7fdceb2e5640 1 --2- 172.21.15.114:0/983564016 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdce4100820 0x7fdce4195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:32.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:32.018+0000 7fdceb2e5640 1 -- 172.21.15.114:0/983564016 >> 172.21.15.114:0/983564016 conn(0x7fdce40fa440 msgr2=0x7fdce4101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:32.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:32.018+0000 7fdceb2e5640 1 -- 172.21.15.114:0/983564016 shutdown_connections 2024-09-13T18:20:32.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:32.018+0000 7fdceb2e5640 1 -- 172.21.15.114:0/983564016 wait complete. 2024-09-13T18:20:32.032 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-09-13T18:20:31.909274+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":607228,"num_objects":96,"num_object_clones":0,"num_object_copies":290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":0,"num_read":184,"num_read_kb":163,"num_write":385,"num_write_kb":3015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":60,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":316,"ondisk_log_size":316,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":417,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360888,"kb_used_data":12072,"kb_used_omap":18,"kb_used_meta":348653,"kb_avail":1124580936,"statfs":{"total":1151940427776,"available":1151570878464,"internally_reserved":0,"allocated":12361728,"data_stored":7793263,"data_compressed":315310,"data_compressed_allocated":425984,"data_compressed_original":2351104,"omap_allocated":19129,"internal_metadata":357020999},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":12267,"num_objects":72,"num_object_clones":0,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":48,"num_read_kb":44,"num_write":156,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":46,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002196"},"pg_stats":[{"pgid":"2.1f","version":"91'13","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.256848+0000","last_change":"2024-09-13T18:20:01.577029+0000","last_active":"2024-09-13T18:20:28.256848+0000","last_peered":"2024-09-13T18:20:28.256848+0000","last_clean":"2024-09-13T18:20:28.256848+0000","last_became_active":"2024-09-13T18:20:01.576710+0000","last_became_peered":"2024-09-13T18:20:01.576710+0000","last_unstale":"2024-09-13T18:20:28.256848+0000","last_undegraded":"2024-09-13T18:20:28.256848+0000","last_fullsized":"2024-09-13T18:20:28.256848+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:25:11.977436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.190449+0000","last_change":"2024-09-13T18:20:01.922955+0000","last_active":"2024-09-13T18:20:26.190449+0000","last_peered":"2024-09-13T18:20:26.190449+0000","last_clean":"2024-09-13T18:20:26.190449+0000","last_became_active":"2024-09-13T18:20:01.922704+0000","last_became_peered":"2024-09-13T18:20:01.922704+0000","last_unstale":"2024-09-13T18:20:26.190449+0000","last_undegraded":"2024-09-13T18:20:26.190449+0000","last_fullsized":"2024-09-13T18:20:26.190449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:44:20.672784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:27.300985+0000","last_change":"2024-09-13T18:20:27.300985+0000","last_active":"2024-09-13T18:20:27.300985+0000","last_peered":"2024-09-13T18:20:27.300985+0000","last_clean":"2024-09-13T18:20:27.300985+0000","last_became_active":"2024-09-13T18:20:27.300546+0000","last_became_peered":"2024-09-13T18:20:27.300546+0000","last_unstale":"2024-09-13T18:20:27.300985+0000","last_undegraded":"2024-09-13T18:20:27.300985+0000","last_fullsized":"2024-09-13T18:20:27.300985+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:00:11.750929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.197866+0000","last_change":"2024-09-13T18:20:01.578921+0000","last_active":"2024-09-13T18:20:26.197866+0000","last_peered":"2024-09-13T18:20:26.197866+0000","last_clean":"2024-09-13T18:20:26.197866+0000","last_became_active":"2024-09-13T18:20:01.578627+0000","last_became_peered":"2024-09-13T18:20:01.578627+0000","last_unstale":"2024-09-13T18:20:26.197866+0000","last_undegraded":"2024-09-13T18:20:26.197866+0000","last_fullsized":"2024-09-13T18:20:26.197866+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:43:32.919156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.190717+0000","last_change":"2024-09-13T18:20:01.922901+0000","last_active":"2024-09-13T18:20:26.190717+0000","last_peered":"2024-09-13T18:20:26.190717+0000","last_clean":"2024-09-13T18:20:26.190717+0000","last_became_active":"2024-09-13T18:20:01.922604+0000","last_became_peered":"2024-09-13T18:20:01.922604+0000","last_unstale":"2024-09-13T18:20:26.190717+0000","last_undegraded":"2024-09-13T18:20:26.190717+0000","last_fullsized":"2024-09-13T18:20:26.190717+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:27:30.650010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.185155+0000","last_change":"2024-09-13T18:20:06.135406+0000","last_active":"2024-09-13T18:20:26.185155+0000","last_peered":"2024-09-13T18:20:26.185155+0000","last_clean":"2024-09-13T18:20:26.185155+0000","last_became_active":"2024-09-13T18:20:06.134558+0000","last_became_peered":"2024-09-13T18:20:06.134558+0000","last_unstale":"2024-09-13T18:20:26.185155+0000","last_undegraded":"2024-09-13T18:20:26.185155+0000","last_fullsized":"2024-09-13T18:20:26.185155+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:10.901966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'11","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.257190+0000","last_change":"2024-09-13T18:20:01.580040+0000","last_active":"2024-09-13T18:20:28.257190+0000","last_peered":"2024-09-13T18:20:28.257190+0000","last_clean":"2024-09-13T18:20:28.257190+0000","last_became_active":"2024-09-13T18:20:01.579715+0000","last_became_peered":"2024-09-13T18:20:01.579715+0000","last_unstale":"2024-09-13T18:20:28.257190+0000","last_undegraded":"2024-09-13T18:20:28.257190+0000","last_fullsized":"2024-09-13T18:20:28.257190+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:57:03.588169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.184089+0000","last_change":"2024-09-13T18:20:01.928314+0000","last_active":"2024-09-13T18:20:26.184089+0000","last_peered":"2024-09-13T18:20:26.184089+0000","last_clean":"2024-09-13T18:20:26.184089+0000","last_became_active":"2024-09-13T18:20:01.925356+0000","last_became_peered":"2024-09-13T18:20:01.925356+0000","last_unstale":"2024-09-13T18:20:26.184089+0000","last_undegraded":"2024-09-13T18:20:26.184089+0000","last_fullsized":"2024-09-13T18:20:26.184089+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:43.906842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.185560+0000","last_change":"2024-09-13T18:20:06.135886+0000","last_active":"2024-09-13T18:20:26.185560+0000","last_peered":"2024-09-13T18:20:26.185560+0000","last_clean":"2024-09-13T18:20:26.185560+0000","last_became_active":"2024-09-13T18:20:06.134587+0000","last_became_peered":"2024-09-13T18:20:06.134587+0000","last_unstale":"2024-09-13T18:20:26.185560+0000","last_undegraded":"2024-09-13T18:20:26.185560+0000","last_fullsized":"2024-09-13T18:20:26.185560+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:44:42.434729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"91'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.256632+0000","last_change":"2024-09-13T18:20:01.576908+0000","last_active":"2024-09-13T18:20:28.256632+0000","last_peered":"2024-09-13T18:20:28.256632+0000","last_clean":"2024-09-13T18:20:28.256632+0000","last_became_active":"2024-09-13T18:20:01.576478+0000","last_became_peered":"2024-09-13T18:20:01.576478+0000","last_unstale":"2024-09-13T18:20:28.256632+0000","last_undegraded":"2024-09-13T18:20:28.256632+0000","last_fullsized":"2024-09-13T18:20:28.256632+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:34:52.126901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.180704+0000","last_change":"2024-09-13T18:20:01.921960+0000","last_active":"2024-09-13T18:20:26.180704+0000","last_peered":"2024-09-13T18:20:26.180704+0000","last_clean":"2024-09-13T18:20:26.180704+0000","last_became_active":"2024-09-13T18:20:01.921652+0000","last_became_peered":"2024-09-13T18:20:01.921652+0000","last_unstale":"2024-09-13T18:20:26.180704+0000","last_undegraded":"2024-09-13T18:20:26.180704+0000","last_fullsized":"2024-09-13T18:20:26.180704+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:00:31.277613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.184886+0000","last_change":"2024-09-13T18:20:06.138983+0000","last_active":"2024-09-13T18:20:26.184886+0000","last_peered":"2024-09-13T18:20:26.184886+0000","last_clean":"2024-09-13T18:20:26.184886+0000","last_became_active":"2024-09-13T18:20:06.138708+0000","last_became_peered":"2024-09-13T18:20:06.138708+0000","last_unstale":"2024-09-13T18:20:26.184886+0000","last_undegraded":"2024-09-13T18:20:26.184886+0000","last_fullsized":"2024-09-13T18:20:26.184886+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:45:21.978217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.187833+0000","last_change":"2024-09-13T18:20:01.929817+0000","last_active":"2024-09-13T18:20:26.187833+0000","last_peered":"2024-09-13T18:20:26.187833+0000","last_clean":"2024-09-13T18:20:26.187833+0000","last_became_active":"2024-09-13T18:20:01.928907+0000","last_became_peered":"2024-09-13T18:20:01.928907+0000","last_unstale":"2024-09-13T18:20:26.187833+0000","last_undegraded":"2024-09-13T18:20:26.187833+0000","last_fullsized":"2024-09-13T18:20:26.187833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:56:25.847230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:27.185633+0000","last_change":"2024-09-13T18:20:01.578968+0000","last_active":"2024-09-13T18:20:27.185633+0000","last_peered":"2024-09-13T18:20:27.185633+0000","last_clean":"2024-09-13T18:20:27.185633+0000","last_became_active":"2024-09-13T18:20:01.578623+0000","last_became_peered":"2024-09-13T18:20:01.578623+0000","last_unstale":"2024-09-13T18:20:27.185633+0000","last_undegraded":"2024-09-13T18:20:27.185633+0000","last_fullsized":"2024-09-13T18:20:27.185633+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:03:30.333107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.186277+0000","last_change":"2024-09-13T18:20:06.135977+0000","last_active":"2024-09-13T18:20:26.186277+0000","last_peered":"2024-09-13T18:20:26.186277+0000","last_clean":"2024-09-13T18:20:26.186277+0000","last_became_active":"2024-09-13T18:20:06.134844+0000","last_became_peered":"2024-09-13T18:20:06.134844+0000","last_unstale":"2024-09-13T18:20:26.186277+0000","last_undegraded":"2024-09-13T18:20:26.186277+0000","last_fullsized":"2024-09-13T18:20:26.186277+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:58:36.139240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.190165+0000","last_change":"2024-09-13T18:20:01.916675+0000","last_active":"2024-09-13T18:20:26.190165+0000","last_peered":"2024-09-13T18:20:26.190165+0000","last_clean":"2024-09-13T18:20:26.190165+0000","last_became_active":"2024-09-13T18:20:01.912902+0000","last_became_peered":"2024-09-13T18:20:01.912902+0000","last_unstale":"2024-09-13T18:20:26.190165+0000","last_undegraded":"2024-09-13T18:20:26.190165+0000","last_fullsized":"2024-09-13T18:20:26.190165+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:24:54.271569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:27.301309+0000","last_change":"2024-09-13T18:20:27.300845+0000","last_active":"2024-09-13T18:20:27.301309+0000","last_peered":"2024-09-13T18:20:27.301309+0000","last_clean":"2024-09-13T18:20:27.301309+0000","last_became_active":"2024-09-13T18:20:27.300417+0000","last_became_peered":"2024-09-13T18:20:27.300417+0000","last_unstale":"2024-09-13T18:20:27.301309+0000","last_undegraded":"2024-09-13T18:20:27.301309+0000","last_fullsized":"2024-09-13T18:20:27.301309+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:44:27.507140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:27.180386+0000","last_change":"2024-09-13T18:20:06.131539+0000","last_active":"2024-09-13T18:20:27.180386+0000","last_peered":"2024-09-13T18:20:27.180386+0000","last_clean":"2024-09-13T18:20:27.180386+0000","last_became_active":"2024-09-13T18:20:06.131148+0000","last_became_peered":"2024-09-13T18:20:06.131148+0000","last_unstale":"2024-09-13T18:20:27.180386+0000","last_undegraded":"2024-09-13T18:20:27.180386+0000","last_fullsized":"2024-09-13T18:20:27.180386+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:32:49.759831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183043+0000","last_change":"2024-09-13T18:20:01.913634+0000","last_active":"2024-09-13T18:20:26.183043+0000","last_peered":"2024-09-13T18:20:26.183043+0000","last_clean":"2024-09-13T18:20:26.183043+0000","last_became_active":"2024-09-13T18:20:01.913128+0000","last_became_peered":"2024-09-13T18:20:01.913128+0000","last_unstale":"2024-09-13T18:20:26.183043+0000","last_undegraded":"2024-09-13T18:20:26.183043+0000","last_fullsized":"2024-09-13T18:20:26.183043+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:37:15.251756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.192501+0000","last_change":"2024-09-13T18:20:01.579385+0000","last_active":"2024-09-13T18:20:26.192501+0000","last_peered":"2024-09-13T18:20:26.192501+0000","last_clean":"2024-09-13T18:20:26.192501+0000","last_became_active":"2024-09-13T18:20:01.579102+0000","last_became_peered":"2024-09-13T18:20:01.579102+0000","last_unstale":"2024-09-13T18:20:26.192501+0000","last_undegraded":"2024-09-13T18:20:26.192501+0000","last_fullsized":"2024-09-13T18:20:26.192501+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:22:00.432052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183422+0000","last_change":"2024-09-13T18:20:06.138110+0000","last_active":"2024-09-13T18:20:26.183422+0000","last_peered":"2024-09-13T18:20:26.183422+0000","last_clean":"2024-09-13T18:20:26.183422+0000","last_became_active":"2024-09-13T18:20:06.137763+0000","last_became_peered":"2024-09-13T18:20:06.137763+0000","last_unstale":"2024-09-13T18:20:26.183422+0000","last_undegraded":"2024-09-13T18:20:26.183422+0000","last_fullsized":"2024-09-13T18:20:26.183422+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:16:15.956216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.185225+0000","last_change":"2024-09-13T18:20:01.918318+0000","last_active":"2024-09-13T18:20:26.185225+0000","last_peered":"2024-09-13T18:20:26.185225+0000","last_clean":"2024-09-13T18:20:26.185225+0000","last_became_active":"2024-09-13T18:20:01.917977+0000","last_became_peered":"2024-09-13T18:20:01.917977+0000","last_unstale":"2024-09-13T18:20:26.185225+0000","last_undegraded":"2024-09-13T18:20:26.185225+0000","last_fullsized":"2024-09-13T18:20:26.185225+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:23.368866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.184253+0000","last_change":"2024-09-13T18:20:01.572009+0000","last_active":"2024-09-13T18:20:26.184253+0000","last_peered":"2024-09-13T18:20:26.184253+0000","last_clean":"2024-09-13T18:20:26.184253+0000","last_became_active":"2024-09-13T18:20:01.571313+0000","last_became_peered":"2024-09-13T18:20:01.571313+0000","last_unstale":"2024-09-13T18:20:26.184253+0000","last_undegraded":"2024-09-13T18:20:26.184253+0000","last_fullsized":"2024-09-13T18:20:26.184253+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:32:59.340476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.186594+0000","last_change":"2024-09-13T18:20:06.132212+0000","last_active":"2024-09-13T18:20:26.186594+0000","last_peered":"2024-09-13T18:20:26.186594+0000","last_clean":"2024-09-13T18:20:26.186594+0000","last_became_active":"2024-09-13T18:20:06.131809+0000","last_became_peered":"2024-09-13T18:20:06.131809+0000","last_unstale":"2024-09-13T18:20:26.186594+0000","last_undegraded":"2024-09-13T18:20:26.186594+0000","last_fullsized":"2024-09-13T18:20:26.186594+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:18:11.667551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"activating","last_fresh":"2024-09-13T18:20:27.181672+0000","last_change":"2024-09-13T18:20:27.181672+0000","last_active":"2024-09-13T18:20:26.189486+0000","last_peered":"2024-09-13T18:20:06.959341+0000","last_clean":"2024-09-13T18:20:06.959341+0000","last_became_active":"2024-09-13T18:20:01.922815+0000","last_became_peered":"2024-09-13T18:20:01.922815+0000","last_unstale":"2024-09-13T18:20:27.181672+0000","last_undegraded":"2024-09-13T18:20:27.181672+0000","last_fullsized":"2024-09-13T18:20:27.181672+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:59:57.606105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":["0","3","4","7"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"91'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.255245+0000","last_change":"2024-09-13T18:20:01.572025+0000","last_active":"2024-09-13T18:20:28.255245+0000","last_peered":"2024-09-13T18:20:28.255245+0000","last_clean":"2024-09-13T18:20:28.255245+0000","last_became_active":"2024-09-13T18:20:01.571321+0000","last_became_peered":"2024-09-13T18:20:01.571321+0000","last_unstale":"2024-09-13T18:20:28.255245+0000","last_undegraded":"2024-09-13T18:20:28.255245+0000","last_fullsized":"2024-09-13T18:20:28.255245+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:09.374364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.188182+0000","last_change":"2024-09-13T18:20:06.130687+0000","last_active":"2024-09-13T18:20:26.188182+0000","last_peered":"2024-09-13T18:20:26.188182+0000","last_clean":"2024-09-13T18:20:26.188182+0000","last_became_active":"2024-09-13T18:20:06.129839+0000","last_became_peered":"2024-09-13T18:20:06.129839+0000","last_unstale":"2024-09-13T18:20:26.188182+0000","last_undegraded":"2024-09-13T18:20:26.188182+0000","last_fullsized":"2024-09-13T18:20:26.188182+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:12:52.334945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.190458+0000","last_change":"2024-09-13T18:20:01.917244+0000","last_active":"2024-09-13T18:20:26.190458+0000","last_peered":"2024-09-13T18:20:26.190458+0000","last_clean":"2024-09-13T18:20:26.190458+0000","last_became_active":"2024-09-13T18:20:01.916948+0000","last_became_peered":"2024-09-13T18:20:01.916948+0000","last_unstale":"2024-09-13T18:20:26.190458+0000","last_undegraded":"2024-09-13T18:20:26.190458+0000","last_fullsized":"2024-09-13T18:20:26.190458+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:17:11.967096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'7","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.206617+0000","last_change":"2024-09-13T18:20:01.569123+0000","last_active":"2024-09-13T18:20:28.206617+0000","last_peered":"2024-09-13T18:20:28.206617+0000","last_clean":"2024-09-13T18:20:28.206617+0000","last_became_active":"2024-09-13T18:20:01.568720+0000","last_became_peered":"2024-09-13T18:20:01.568720+0000","last_unstale":"2024-09-13T18:20:28.206617+0000","last_undegraded":"2024-09-13T18:20:28.206617+0000","last_fullsized":"2024-09-13T18:20:28.206617+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:04:29.561013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.190854+0000","last_change":"2024-09-13T18:20:06.141771+0000","last_active":"2024-09-13T18:20:26.190854+0000","last_peered":"2024-09-13T18:20:26.190854+0000","last_clean":"2024-09-13T18:20:26.190854+0000","last_became_active":"2024-09-13T18:20:06.141362+0000","last_became_peered":"2024-09-13T18:20:06.141362+0000","last_unstale":"2024-09-13T18:20:26.190854+0000","last_undegraded":"2024-09-13T18:20:26.190854+0000","last_fullsized":"2024-09-13T18:20:26.190854+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:09:58.597605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.179047+0000","last_change":"2024-09-13T18:20:01.911048+0000","last_active":"2024-09-13T18:20:26.179047+0000","last_peered":"2024-09-13T18:20:26.179047+0000","last_clean":"2024-09-13T18:20:26.179047+0000","last_became_active":"2024-09-13T18:20:01.910735+0000","last_became_peered":"2024-09-13T18:20:01.910735+0000","last_unstale":"2024-09-13T18:20:26.179047+0000","last_undegraded":"2024-09-13T18:20:26.179047+0000","last_fullsized":"2024-09-13T18:20:26.179047+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:07:35.447676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"91'4","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.248823+0000","last_change":"2024-09-13T18:20:01.571627+0000","last_active":"2024-09-13T18:20:28.248823+0000","last_peered":"2024-09-13T18:20:28.248823+0000","last_clean":"2024-09-13T18:20:28.248823+0000","last_became_active":"2024-09-13T18:20:01.571250+0000","last_became_peered":"2024-09-13T18:20:01.571250+0000","last_unstale":"2024-09-13T18:20:28.248823+0000","last_undegraded":"2024-09-13T18:20:28.248823+0000","last_fullsized":"2024-09-13T18:20:28.248823+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:34:03.479324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.182932+0000","last_change":"2024-09-13T18:20:06.132974+0000","last_active":"2024-09-13T18:20:26.182932+0000","last_peered":"2024-09-13T18:20:26.182932+0000","last_clean":"2024-09-13T18:20:26.182932+0000","last_became_active":"2024-09-13T18:20:06.131668+0000","last_became_peered":"2024-09-13T18:20:06.131668+0000","last_unstale":"2024-09-13T18:20:26.182932+0000","last_undegraded":"2024-09-13T18:20:26.182932+0000","last_fullsized":"2024-09-13T18:20:26.182932+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:36:30.329761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.197759+0000","last_change":"2024-09-13T18:20:01.904670+0000","last_active":"2024-09-13T18:20:26.197759+0000","last_peered":"2024-09-13T18:20:26.197759+0000","last_clean":"2024-09-13T18:20:26.197759+0000","last_became_active":"2024-09-13T18:20:01.904316+0000","last_became_peered":"2024-09-13T18:20:01.904316+0000","last_unstale":"2024-09-13T18:20:26.197759+0000","last_undegraded":"2024-09-13T18:20:26.197759+0000","last_fullsized":"2024-09-13T18:20:26.197759+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:08:17.687179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"91'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:27.179594+0000","last_change":"2024-09-13T18:20:01.564750+0000","last_active":"2024-09-13T18:20:27.179594+0000","last_peered":"2024-09-13T18:20:27.179594+0000","last_clean":"2024-09-13T18:20:27.179594+0000","last_became_active":"2024-09-13T18:20:01.564096+0000","last_became_peered":"2024-09-13T18:20:01.564096+0000","last_unstale":"2024-09-13T18:20:27.179594+0000","last_undegraded":"2024-09-13T18:20:27.179594+0000","last_fullsized":"2024-09-13T18:20:27.179594+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:42:19.588824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.185367+0000","last_change":"2024-09-13T18:20:06.133386+0000","last_active":"2024-09-13T18:20:26.185367+0000","last_peered":"2024-09-13T18:20:26.185367+0000","last_clean":"2024-09-13T18:20:26.185367+0000","last_became_active":"2024-09-13T18:20:06.133004+0000","last_became_peered":"2024-09-13T18:20:06.133004+0000","last_unstale":"2024-09-13T18:20:26.185367+0000","last_undegraded":"2024-09-13T18:20:26.185367+0000","last_fullsized":"2024-09-13T18:20:26.185367+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:54:46.440410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.197632+0000","last_change":"2024-09-13T18:20:01.912823+0000","last_active":"2024-09-13T18:20:26.197632+0000","last_peered":"2024-09-13T18:20:26.197632+0000","last_clean":"2024-09-13T18:20:26.197632+0000","last_became_active":"2024-09-13T18:20:01.912123+0000","last_became_peered":"2024-09-13T18:20:01.912123+0000","last_unstale":"2024-09-13T18:20:26.197632+0000","last_undegraded":"2024-09-13T18:20:26.197632+0000","last_fullsized":"2024-09-13T18:20:26.197632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:32:33.544816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"91'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:29.259117+0000","last_change":"2024-09-13T18:20:01.577095+0000","last_active":"2024-09-13T18:20:29.259117+0000","last_peered":"2024-09-13T18:20:29.259117+0000","last_clean":"2024-09-13T18:20:29.259117+0000","last_became_active":"2024-09-13T18:20:01.576765+0000","last_became_peered":"2024-09-13T18:20:01.576765+0000","last_unstale":"2024-09-13T18:20:29.259117+0000","last_undegraded":"2024-09-13T18:20:29.259117+0000","last_fullsized":"2024-09-13T18:20:29.259117+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:44:08.750441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.178862+0000","last_change":"2024-09-13T18:20:06.131650+0000","last_active":"2024-09-13T18:20:26.178862+0000","last_peered":"2024-09-13T18:20:26.178862+0000","last_clean":"2024-09-13T18:20:26.178862+0000","last_became_active":"2024-09-13T18:20:06.130902+0000","last_became_peered":"2024-09-13T18:20:06.130902+0000","last_unstale":"2024-09-13T18:20:26.178862+0000","last_undegraded":"2024-09-13T18:20:26.178862+0000","last_fullsized":"2024-09-13T18:20:26.178862+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.366694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.180095+0000","last_change":"2024-09-13T18:20:01.918221+0000","last_active":"2024-09-13T18:20:26.180095+0000","last_peered":"2024-09-13T18:20:26.180095+0000","last_clean":"2024-09-13T18:20:26.180095+0000","last_became_active":"2024-09-13T18:20:01.917134+0000","last_became_peered":"2024-09-13T18:20:01.917134+0000","last_unstale":"2024-09-13T18:20:26.180095+0000","last_undegraded":"2024-09-13T18:20:26.180095+0000","last_fullsized":"2024-09-13T18:20:26.180095+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:19:06.269454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.249346+0000","last_change":"2024-09-13T18:20:01.572258+0000","last_active":"2024-09-13T18:20:28.249346+0000","last_peered":"2024-09-13T18:20:28.249346+0000","last_clean":"2024-09-13T18:20:28.249346+0000","last_became_active":"2024-09-13T18:20:01.571968+0000","last_became_peered":"2024-09-13T18:20:01.571968+0000","last_unstale":"2024-09-13T18:20:28.249346+0000","last_undegraded":"2024-09-13T18:20:28.249346+0000","last_fullsized":"2024-09-13T18:20:28.249346+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:40:59.143123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.179018+0000","last_change":"2024-09-13T18:20:06.131714+0000","last_active":"2024-09-13T18:20:26.179018+0000","last_peered":"2024-09-13T18:20:26.179018+0000","last_clean":"2024-09-13T18:20:26.179018+0000","last_became_active":"2024-09-13T18:20:06.131244+0000","last_became_peered":"2024-09-13T18:20:06.131244+0000","last_unstale":"2024-09-13T18:20:26.179018+0000","last_undegraded":"2024-09-13T18:20:26.179018+0000","last_fullsized":"2024-09-13T18:20:26.179018+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:37.946002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.178584+0000","last_change":"2024-09-13T18:20:01.917351+0000","last_active":"2024-09-13T18:20:26.178584+0000","last_peered":"2024-09-13T18:20:26.178584+0000","last_clean":"2024-09-13T18:20:26.178584+0000","last_became_active":"2024-09-13T18:20:01.917067+0000","last_became_peered":"2024-09-13T18:20:01.917067+0000","last_unstale":"2024-09-13T18:20:26.178584+0000","last_undegraded":"2024-09-13T18:20:26.178584+0000","last_fullsized":"2024-09-13T18:20:26.178584+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:35:37.723132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'4","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:27.182812+0000","last_change":"2024-09-13T18:20:01.573515+0000","last_active":"2024-09-13T18:20:27.182812+0000","last_peered":"2024-09-13T18:20:27.182812+0000","last_clean":"2024-09-13T18:20:27.182812+0000","last_became_active":"2024-09-13T18:20:01.573211+0000","last_became_peered":"2024-09-13T18:20:01.573211+0000","last_unstale":"2024-09-13T18:20:27.182812+0000","last_undegraded":"2024-09-13T18:20:27.182812+0000","last_fullsized":"2024-09-13T18:20:27.182812+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:05:30.912467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.180826+0000","last_change":"2024-09-13T18:20:06.133713+0000","last_active":"2024-09-13T18:20:26.180826+0000","last_peered":"2024-09-13T18:20:26.180826+0000","last_clean":"2024-09-13T18:20:26.180826+0000","last_became_active":"2024-09-13T18:20:06.133330+0000","last_became_peered":"2024-09-13T18:20:06.133330+0000","last_unstale":"2024-09-13T18:20:26.180826+0000","last_undegraded":"2024-09-13T18:20:26.180826+0000","last_fullsized":"2024-09-13T18:20:26.180826+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:48:07.133155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.192066+0000","last_change":"2024-09-13T18:20:01.921597+0000","last_active":"2024-09-13T18:20:26.192066+0000","last_peered":"2024-09-13T18:20:26.192066+0000","last_clean":"2024-09-13T18:20:26.192066+0000","last_became_active":"2024-09-13T18:20:01.920200+0000","last_became_peered":"2024-09-13T18:20:01.920200+0000","last_unstale":"2024-09-13T18:20:26.192066+0000","last_undegraded":"2024-09-13T18:20:26.192066+0000","last_fullsized":"2024-09-13T18:20:26.192066+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:22:40.517941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.255802+0000","last_change":"2024-09-13T18:20:01.571313+0000","last_active":"2024-09-13T18:20:28.255802+0000","last_peered":"2024-09-13T18:20:28.255802+0000","last_clean":"2024-09-13T18:20:28.255802+0000","last_became_active":"2024-09-13T18:20:01.570977+0000","last_became_peered":"2024-09-13T18:20:01.570977+0000","last_unstale":"2024-09-13T18:20:28.255802+0000","last_undegraded":"2024-09-13T18:20:28.255802+0000","last_fullsized":"2024-09-13T18:20:28.255802+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:06.870320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":343,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.187790+0000","last_change":"2024-09-13T18:18:51.237409+0000","last_active":"2024-09-13T18:20:26.187790+0000","last_peered":"2024-09-13T18:20:26.187790+0000","last_clean":"2024-09-13T18:20:26.187790+0000","last_became_active":"2024-09-13T18:18:51.236986+0000","last_became_peered":"2024-09-13T18:18:51.236986+0000","last_unstale":"2024-09-13T18:20:26.187790+0000","last_undegraded":"2024-09-13T18:20:26.187790+0000","last_fullsized":"2024-09-13T18:20:26.187790+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:15:40.757299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:15:40.757299+0000","last_clean_scrub_stamp":"2024-09-13T18:15:40.757299+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:58:29.679404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:27.182752+0000","last_change":"2024-09-13T18:20:06.130803+0000","last_active":"2024-09-13T18:20:27.182752+0000","last_peered":"2024-09-13T18:20:27.182752+0000","last_clean":"2024-09-13T18:20:27.182752+0000","last_became_active":"2024-09-13T18:20:06.130433+0000","last_became_peered":"2024-09-13T18:20:06.130433+0000","last_unstale":"2024-09-13T18:20:27.182752+0000","last_undegraded":"2024-09-13T18:20:27.182752+0000","last_fullsized":"2024-09-13T18:20:27.182752+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:23:11.787084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.188184+0000","last_change":"2024-09-13T18:20:01.922702+0000","last_active":"2024-09-13T18:20:26.188184+0000","last_peered":"2024-09-13T18:20:26.188184+0000","last_clean":"2024-09-13T18:20:26.188184+0000","last_became_active":"2024-09-13T18:20:01.921109+0000","last_became_peered":"2024-09-13T18:20:01.921109+0000","last_unstale":"2024-09-13T18:20:26.188184+0000","last_undegraded":"2024-09-13T18:20:26.188184+0000","last_fullsized":"2024-09-13T18:20:26.188184+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:03:34.618054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"91'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:29.257448+0000","last_change":"2024-09-13T18:20:01.571024+0000","last_active":"2024-09-13T18:20:29.257448+0000","last_peered":"2024-09-13T18:20:29.257448+0000","last_clean":"2024-09-13T18:20:29.257448+0000","last_became_active":"2024-09-13T18:20:01.570687+0000","last_became_peered":"2024-09-13T18:20:01.570687+0000","last_unstale":"2024-09-13T18:20:29.257448+0000","last_undegraded":"2024-09-13T18:20:29.257448+0000","last_fullsized":"2024-09-13T18:20:29.257448+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:10:25.126179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183401+0000","last_change":"2024-09-13T18:20:06.129025+0000","last_active":"2024-09-13T18:20:26.183401+0000","last_peered":"2024-09-13T18:20:26.183401+0000","last_clean":"2024-09-13T18:20:26.183401+0000","last_became_active":"2024-09-13T18:20:06.128618+0000","last_became_peered":"2024-09-13T18:20:06.128618+0000","last_unstale":"2024-09-13T18:20:26.183401+0000","last_undegraded":"2024-09-13T18:20:26.183401+0000","last_fullsized":"2024-09-13T18:20:26.183401+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:20:08.891259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.187706+0000","last_change":"2024-09-13T18:20:01.921697+0000","last_active":"2024-09-13T18:20:26.187706+0000","last_peered":"2024-09-13T18:20:26.187706+0000","last_clean":"2024-09-13T18:20:26.187706+0000","last_became_active":"2024-09-13T18:20:01.920174+0000","last_became_peered":"2024-09-13T18:20:01.920174+0000","last_unstale":"2024-09-13T18:20:26.187706+0000","last_undegraded":"2024-09-13T18:20:26.187706+0000","last_fullsized":"2024-09-13T18:20:26.187706+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:57:03.922725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"91'3","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.247610+0000","last_change":"2024-09-13T18:20:01.582102+0000","last_active":"2024-09-13T18:20:28.247610+0000","last_peered":"2024-09-13T18:20:28.247610+0000","last_clean":"2024-09-13T18:20:28.247610+0000","last_became_active":"2024-09-13T18:20:01.581841+0000","last_became_peered":"2024-09-13T18:20:01.581841+0000","last_unstale":"2024-09-13T18:20:28.247610+0000","last_undegraded":"2024-09-13T18:20:28.247610+0000","last_fullsized":"2024-09-13T18:20:28.247610+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:59:57.929703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.189924+0000","last_change":"2024-09-13T18:20:06.133126+0000","last_active":"2024-09-13T18:20:26.189924+0000","last_peered":"2024-09-13T18:20:26.189924+0000","last_clean":"2024-09-13T18:20:26.189924+0000","last_became_active":"2024-09-13T18:20:06.132747+0000","last_became_peered":"2024-09-13T18:20:06.132747+0000","last_unstale":"2024-09-13T18:20:26.189924+0000","last_undegraded":"2024-09-13T18:20:26.189924+0000","last_fullsized":"2024-09-13T18:20:26.189924+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:54:21.843090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.187974+0000","last_change":"2024-09-13T18:20:01.916306+0000","last_active":"2024-09-13T18:20:26.187974+0000","last_peered":"2024-09-13T18:20:26.187974+0000","last_clean":"2024-09-13T18:20:26.187974+0000","last_became_active":"2024-09-13T18:20:01.915739+0000","last_became_peered":"2024-09-13T18:20:01.915739+0000","last_unstale":"2024-09-13T18:20:26.187974+0000","last_undegraded":"2024-09-13T18:20:26.187974+0000","last_fullsized":"2024-09-13T18:20:26.187974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:19:09.689909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"91'5","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.252883+0000","last_change":"2024-09-13T18:20:01.565821+0000","last_active":"2024-09-13T18:20:28.252883+0000","last_peered":"2024-09-13T18:20:28.252883+0000","last_clean":"2024-09-13T18:20:28.252883+0000","last_became_active":"2024-09-13T18:20:01.565467+0000","last_became_peered":"2024-09-13T18:20:01.565467+0000","last_unstale":"2024-09-13T18:20:28.252883+0000","last_undegraded":"2024-09-13T18:20:28.252883+0000","last_fullsized":"2024-09-13T18:20:28.252883+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:11.609400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.179986+0000","last_change":"2024-09-13T18:20:06.131348+0000","last_active":"2024-09-13T18:20:26.179986+0000","last_peered":"2024-09-13T18:20:26.179986+0000","last_clean":"2024-09-13T18:20:26.179986+0000","last_became_active":"2024-09-13T18:20:06.131019+0000","last_became_peered":"2024-09-13T18:20:06.131019+0000","last_unstale":"2024-09-13T18:20:26.179986+0000","last_undegraded":"2024-09-13T18:20:26.179986+0000","last_fullsized":"2024-09-13T18:20:26.179986+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:13:49.717503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183415+0000","last_change":"2024-09-13T18:20:01.922306+0000","last_active":"2024-09-13T18:20:26.183415+0000","last_peered":"2024-09-13T18:20:26.183415+0000","last_clean":"2024-09-13T18:20:26.183415+0000","last_became_active":"2024-09-13T18:20:01.921956+0000","last_became_peered":"2024-09-13T18:20:01.921956+0000","last_unstale":"2024-09-13T18:20:26.183415+0000","last_undegraded":"2024-09-13T18:20:26.183415+0000","last_fullsized":"2024-09-13T18:20:26.183415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:17:48.755318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:27.194592+0000","last_change":"2024-09-13T18:20:27.194085+0000","last_active":"2024-09-13T18:20:27.194592+0000","last_peered":"2024-09-13T18:20:27.194592+0000","last_clean":"2024-09-13T18:20:27.194592+0000","last_became_active":"2024-09-13T18:20:27.190136+0000","last_became_peered":"2024-09-13T18:20:27.190136+0000","last_unstale":"2024-09-13T18:20:27.194592+0000","last_undegraded":"2024-09-13T18:20:27.194592+0000","last_fullsized":"2024-09-13T18:20:27.194592+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:57:07.202901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.180041+0000","last_change":"2024-09-13T18:20:06.135498+0000","last_active":"2024-09-13T18:20:26.180041+0000","last_peered":"2024-09-13T18:20:26.180041+0000","last_clean":"2024-09-13T18:20:26.180041+0000","last_became_active":"2024-09-13T18:20:06.135095+0000","last_became_peered":"2024-09-13T18:20:06.135095+0000","last_unstale":"2024-09-13T18:20:26.180041+0000","last_undegraded":"2024-09-13T18:20:26.180041+0000","last_fullsized":"2024-09-13T18:20:26.180041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:06:40.609718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:27.187296+0000","last_change":"2024-09-13T18:20:27.186825+0000","last_active":"2024-09-13T18:20:27.187296+0000","last_peered":"2024-09-13T18:20:27.187296+0000","last_clean":"2024-09-13T18:20:27.187296+0000","last_became_active":"2024-09-13T18:20:27.186523+0000","last_became_peered":"2024-09-13T18:20:27.186523+0000","last_unstale":"2024-09-13T18:20:27.187296+0000","last_undegraded":"2024-09-13T18:20:27.187296+0000","last_fullsized":"2024-09-13T18:20:27.187296+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:28:30.453945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.10","version":"91'17","reported_seq":57,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.248094+0000","last_change":"2024-09-13T18:20:27.200142+0000","last_active":"2024-09-13T18:20:28.248094+0000","last_peered":"2024-09-13T18:20:28.248094+0000","last_clean":"2024-09-13T18:20:28.248094+0000","last_became_active":"2024-09-13T18:20:27.189670+0000","last_became_peered":"2024-09-13T18:20:27.189670+0000","last_unstale":"2024-09-13T18:20:28.248094+0000","last_undegraded":"2024-09-13T18:20:28.248094+0000","last_fullsized":"2024-09-13T18:20:28.248094+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:16:02.934269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.179043+0000","last_change":"2024-09-13T18:20:06.136742+0000","last_active":"2024-09-13T18:20:26.179043+0000","last_peered":"2024-09-13T18:20:26.179043+0000","last_clean":"2024-09-13T18:20:26.179043+0000","last_became_active":"2024-09-13T18:20:06.136373+0000","last_became_peered":"2024-09-13T18:20:06.136373+0000","last_unstale":"2024-09-13T18:20:26.179043+0000","last_undegraded":"2024-09-13T18:20:26.179043+0000","last_fullsized":"2024-09-13T18:20:26.179043+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:49:11.782489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.187637+0000","last_change":"2024-09-13T18:20:01.922910+0000","last_active":"2024-09-13T18:20:26.187637+0000","last_peered":"2024-09-13T18:20:26.187637+0000","last_clean":"2024-09-13T18:20:26.187637+0000","last_became_active":"2024-09-13T18:20:01.921162+0000","last_became_peered":"2024-09-13T18:20:01.921162+0000","last_unstale":"2024-09-13T18:20:26.187637+0000","last_undegraded":"2024-09-13T18:20:26.187637+0000","last_fullsized":"2024-09-13T18:20:26.187637+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:14:47.243418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"91'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:30.255078+0000","last_change":"2024-09-13T18:20:01.575830+0000","last_active":"2024-09-13T18:20:30.255078+0000","last_peered":"2024-09-13T18:20:30.255078+0000","last_clean":"2024-09-13T18:20:30.255078+0000","last_became_active":"2024-09-13T18:20:01.575334+0000","last_became_peered":"2024-09-13T18:20:01.575334+0000","last_unstale":"2024-09-13T18:20:30.255078+0000","last_undegraded":"2024-09-13T18:20:30.255078+0000","last_fullsized":"2024-09-13T18:20:30.255078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:13:00.935600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":7,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183501+0000","last_change":"2024-09-13T18:20:06.138609+0000","last_active":"2024-09-13T18:20:26.183501+0000","last_peered":"2024-09-13T18:20:26.183501+0000","last_clean":"2024-09-13T18:20:26.183501+0000","last_became_active":"2024-09-13T18:20:06.138232+0000","last_became_peered":"2024-09-13T18:20:06.138232+0000","last_unstale":"2024-09-13T18:20:26.183501+0000","last_undegraded":"2024-09-13T18:20:26.183501+0000","last_fullsized":"2024-09-13T18:20:26.183501+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:46:06.227407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.187782+0000","last_change":"2024-09-13T18:20:01.928757+0000","last_active":"2024-09-13T18:20:26.187782+0000","last_peered":"2024-09-13T18:20:26.187782+0000","last_clean":"2024-09-13T18:20:26.187782+0000","last_became_active":"2024-09-13T18:20:01.928350+0000","last_became_peered":"2024-09-13T18:20:01.928350+0000","last_unstale":"2024-09-13T18:20:26.187782+0000","last_undegraded":"2024-09-13T18:20:26.187782+0000","last_fullsized":"2024-09-13T18:20:26.187782+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:17:43.208106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"91'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.257161+0000","last_change":"2024-09-13T18:20:01.579128+0000","last_active":"2024-09-13T18:20:28.257161+0000","last_peered":"2024-09-13T18:20:28.257161+0000","last_clean":"2024-09-13T18:20:28.257161+0000","last_became_active":"2024-09-13T18:20:01.578796+0000","last_became_peered":"2024-09-13T18:20:01.578796+0000","last_unstale":"2024-09-13T18:20:28.257161+0000","last_undegraded":"2024-09-13T18:20:28.257161+0000","last_fullsized":"2024-09-13T18:20:28.257161+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:30:04.836103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.182950+0000","last_change":"2024-09-13T18:20:06.131404+0000","last_active":"2024-09-13T18:20:26.182950+0000","last_peered":"2024-09-13T18:20:26.182950+0000","last_clean":"2024-09-13T18:20:26.182950+0000","last_became_active":"2024-09-13T18:20:06.130931+0000","last_became_peered":"2024-09-13T18:20:06.130931+0000","last_unstale":"2024-09-13T18:20:26.182950+0000","last_undegraded":"2024-09-13T18:20:26.182950+0000","last_fullsized":"2024-09-13T18:20:26.182950+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:55:54.568816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.178724+0000","last_change":"2024-09-13T18:20:01.916089+0000","last_active":"2024-09-13T18:20:26.178724+0000","last_peered":"2024-09-13T18:20:26.178724+0000","last_clean":"2024-09-13T18:20:26.178724+0000","last_became_active":"2024-09-13T18:20:01.915783+0000","last_became_peered":"2024-09-13T18:20:01.915783+0000","last_unstale":"2024-09-13T18:20:26.178724+0000","last_undegraded":"2024-09-13T18:20:26.178724+0000","last_fullsized":"2024-09-13T18:20:26.178724+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:43:25.886029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"91'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.255522+0000","last_change":"2024-09-13T18:20:01.577381+0000","last_active":"2024-09-13T18:20:28.255522+0000","last_peered":"2024-09-13T18:20:28.255522+0000","last_clean":"2024-09-13T18:20:28.255522+0000","last_became_active":"2024-09-13T18:20:01.577044+0000","last_became_peered":"2024-09-13T18:20:01.577044+0000","last_unstale":"2024-09-13T18:20:28.255522+0000","last_undegraded":"2024-09-13T18:20:28.255522+0000","last_fullsized":"2024-09-13T18:20:28.255522+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:00.907948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.180609+0000","last_change":"2024-09-13T18:20:06.139971+0000","last_active":"2024-09-13T18:20:26.180609+0000","last_peered":"2024-09-13T18:20:26.180609+0000","last_clean":"2024-09-13T18:20:26.180609+0000","last_became_active":"2024-09-13T18:20:06.139507+0000","last_became_peered":"2024-09-13T18:20:06.139507+0000","last_unstale":"2024-09-13T18:20:26.180609+0000","last_undegraded":"2024-09-13T18:20:26.180609+0000","last_fullsized":"2024-09-13T18:20:26.180609+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:40:21.037768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.186323+0000","last_change":"2024-09-13T18:20:01.915348+0000","last_active":"2024-09-13T18:20:26.186323+0000","last_peered":"2024-09-13T18:20:26.186323+0000","last_clean":"2024-09-13T18:20:26.186323+0000","last_became_active":"2024-09-13T18:20:01.915042+0000","last_became_peered":"2024-09-13T18:20:01.915042+0000","last_unstale":"2024-09-13T18:20:26.186323+0000","last_undegraded":"2024-09-13T18:20:26.186323+0000","last_fullsized":"2024-09-13T18:20:26.186323+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:31:33.712314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"91'17","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.256191+0000","last_change":"2024-09-13T18:20:01.578456+0000","last_active":"2024-09-13T18:20:28.256191+0000","last_peered":"2024-09-13T18:20:28.256191+0000","last_clean":"2024-09-13T18:20:28.256191+0000","last_became_active":"2024-09-13T18:20:01.578051+0000","last_became_peered":"2024-09-13T18:20:01.578051+0000","last_unstale":"2024-09-13T18:20:28.256191+0000","last_undegraded":"2024-09-13T18:20:28.256191+0000","last_fullsized":"2024-09-13T18:20:28.256191+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:34:17.651620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10396,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":12,"num_write":19,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183092+0000","last_change":"2024-09-13T18:20:06.131410+0000","last_active":"2024-09-13T18:20:26.183092+0000","last_peered":"2024-09-13T18:20:26.183092+0000","last_clean":"2024-09-13T18:20:26.183092+0000","last_became_active":"2024-09-13T18:20:06.131060+0000","last_became_peered":"2024-09-13T18:20:06.131060+0000","last_unstale":"2024-09-13T18:20:26.183092+0000","last_undegraded":"2024-09-13T18:20:26.183092+0000","last_fullsized":"2024-09-13T18:20:26.183092+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:08:25.266971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183935+0000","last_change":"2024-09-13T18:20:01.928839+0000","last_active":"2024-09-13T18:20:26.183935+0000","last_peered":"2024-09-13T18:20:26.183935+0000","last_clean":"2024-09-13T18:20:26.183935+0000","last_became_active":"2024-09-13T18:20:01.925774+0000","last_became_peered":"2024-09-13T18:20:01.925774+0000","last_unstale":"2024-09-13T18:20:26.183935+0000","last_undegraded":"2024-09-13T18:20:26.183935+0000","last_fullsized":"2024-09-13T18:20:26.183935+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:15:23.049671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"91'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.206178+0000","last_change":"2024-09-13T18:20:01.576332+0000","last_active":"2024-09-13T18:20:28.206178+0000","last_peered":"2024-09-13T18:20:28.206178+0000","last_clean":"2024-09-13T18:20:28.206178+0000","last_became_active":"2024-09-13T18:20:01.575862+0000","last_became_peered":"2024-09-13T18:20:01.575862+0000","last_unstale":"2024-09-13T18:20:28.206178+0000","last_undegraded":"2024-09-13T18:20:28.206178+0000","last_fullsized":"2024-09-13T18:20:28.206178+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:17:00.452480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.186104+0000","last_change":"2024-09-13T18:20:06.133142+0000","last_active":"2024-09-13T18:20:26.186104+0000","last_peered":"2024-09-13T18:20:26.186104+0000","last_clean":"2024-09-13T18:20:26.186104+0000","last_became_active":"2024-09-13T18:20:06.132740+0000","last_became_peered":"2024-09-13T18:20:06.132740+0000","last_unstale":"2024-09-13T18:20:26.186104+0000","last_undegraded":"2024-09-13T18:20:26.186104+0000","last_fullsized":"2024-09-13T18:20:26.186104+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:12:46.442481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.190799+0000","last_change":"2024-09-13T18:20:01.919673+0000","last_active":"2024-09-13T18:20:26.190799+0000","last_peered":"2024-09-13T18:20:26.190799+0000","last_clean":"2024-09-13T18:20:26.190799+0000","last_became_active":"2024-09-13T18:20:01.919358+0000","last_became_peered":"2024-09-13T18:20:01.919358+0000","last_unstale":"2024-09-13T18:20:26.190799+0000","last_undegraded":"2024-09-13T18:20:26.190799+0000","last_fullsized":"2024-09-13T18:20:26.190799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:40:11.407643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"91'4","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.256430+0000","last_change":"2024-09-13T18:20:27.190451+0000","last_active":"2024-09-13T18:20:28.256430+0000","last_peered":"2024-09-13T18:20:28.256430+0000","last_clean":"2024-09-13T18:20:28.256430+0000","last_became_active":"2024-09-13T18:20:27.187091+0000","last_became_peered":"2024-09-13T18:20:27.187091+0000","last_unstale":"2024-09-13T18:20:28.256430+0000","last_undegraded":"2024-09-13T18:20:28.256430+0000","last_fullsized":"2024-09-13T18:20:28.256430+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:57:07.976008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.182915+0000","last_change":"2024-09-13T18:20:06.133475+0000","last_active":"2024-09-13T18:20:26.182915+0000","last_peered":"2024-09-13T18:20:26.182915+0000","last_clean":"2024-09-13T18:20:26.182915+0000","last_became_active":"2024-09-13T18:20:06.133109+0000","last_became_peered":"2024-09-13T18:20:06.133109+0000","last_unstale":"2024-09-13T18:20:26.182915+0000","last_undegraded":"2024-09-13T18:20:26.182915+0000","last_fullsized":"2024-09-13T18:20:26.182915+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:47:19.567115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.184156+0000","last_change":"2024-09-13T18:20:01.922660+0000","last_active":"2024-09-13T18:20:26.184156+0000","last_peered":"2024-09-13T18:20:26.184156+0000","last_clean":"2024-09-13T18:20:26.184156+0000","last_became_active":"2024-09-13T18:20:01.922370+0000","last_became_peered":"2024-09-13T18:20:01.922370+0000","last_unstale":"2024-09-13T18:20:26.184156+0000","last_undegraded":"2024-09-13T18:20:26.184156+0000","last_fullsized":"2024-09-13T18:20:26.184156+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:54:32.029704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"91'4","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.249427+0000","last_change":"2024-09-13T18:20:01.580975+0000","last_active":"2024-09-13T18:20:28.249427+0000","last_peered":"2024-09-13T18:20:28.249427+0000","last_clean":"2024-09-13T18:20:28.249427+0000","last_became_active":"2024-09-13T18:20:01.580647+0000","last_became_peered":"2024-09-13T18:20:01.580647+0000","last_unstale":"2024-09-13T18:20:28.249427+0000","last_undegraded":"2024-09-13T18:20:28.249427+0000","last_fullsized":"2024-09-13T18:20:28.249427+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:26:20.942915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.192336+0000","last_change":"2024-09-13T18:20:06.140003+0000","last_active":"2024-09-13T18:20:26.192336+0000","last_peered":"2024-09-13T18:20:26.192336+0000","last_clean":"2024-09-13T18:20:26.192336+0000","last_became_active":"2024-09-13T18:20:06.139663+0000","last_became_peered":"2024-09-13T18:20:06.139663+0000","last_unstale":"2024-09-13T18:20:26.192336+0000","last_undegraded":"2024-09-13T18:20:26.192336+0000","last_fullsized":"2024-09-13T18:20:26.192336+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:54:49.662401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.189993+0000","last_change":"2024-09-13T18:20:06.134448+0000","last_active":"2024-09-13T18:20:26.189993+0000","last_peered":"2024-09-13T18:20:26.189993+0000","last_clean":"2024-09-13T18:20:26.189993+0000","last_became_active":"2024-09-13T18:20:06.134084+0000","last_became_peered":"2024-09-13T18:20:06.134084+0000","last_unstale":"2024-09-13T18:20:26.189993+0000","last_undegraded":"2024-09-13T18:20:26.189993+0000","last_fullsized":"2024-09-13T18:20:26.189993+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:55:50.475665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183131+0000","last_change":"2024-09-13T18:20:01.918955+0000","last_active":"2024-09-13T18:20:26.183131+0000","last_peered":"2024-09-13T18:20:26.183131+0000","last_clean":"2024-09-13T18:20:26.183131+0000","last_became_active":"2024-09-13T18:20:01.918666+0000","last_became_peered":"2024-09-13T18:20:01.918666+0000","last_unstale":"2024-09-13T18:20:26.183131+0000","last_undegraded":"2024-09-13T18:20:26.183131+0000","last_fullsized":"2024-09-13T18:20:26.183131+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:08:50.697710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"91'3","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.251385+0000","last_change":"2024-09-13T18:20:01.578789+0000","last_active":"2024-09-13T18:20:28.251385+0000","last_peered":"2024-09-13T18:20:28.251385+0000","last_clean":"2024-09-13T18:20:28.251385+0000","last_became_active":"2024-09-13T18:20:01.578476+0000","last_became_peered":"2024-09-13T18:20:01.578476+0000","last_unstale":"2024-09-13T18:20:28.251385+0000","last_undegraded":"2024-09-13T18:20:28.251385+0000","last_fullsized":"2024-09-13T18:20:28.251385+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:40:58.355492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183716+0000","last_change":"2024-09-13T18:20:06.129765+0000","last_active":"2024-09-13T18:20:26.183716+0000","last_peered":"2024-09-13T18:20:26.183716+0000","last_clean":"2024-09-13T18:20:26.183716+0000","last_became_active":"2024-09-13T18:20:06.129310+0000","last_became_peered":"2024-09-13T18:20:06.129310+0000","last_unstale":"2024-09-13T18:20:26.183716+0000","last_undegraded":"2024-09-13T18:20:26.183716+0000","last_fullsized":"2024-09-13T18:20:26.183716+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:34:07.394871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.191040+0000","last_change":"2024-09-13T18:20:01.914804+0000","last_active":"2024-09-13T18:20:26.191040+0000","last_peered":"2024-09-13T18:20:26.191040+0000","last_clean":"2024-09-13T18:20:26.191040+0000","last_became_active":"2024-09-13T18:20:01.914517+0000","last_became_peered":"2024-09-13T18:20:01.914517+0000","last_unstale":"2024-09-13T18:20:26.191040+0000","last_undegraded":"2024-09-13T18:20:26.191040+0000","last_fullsized":"2024-09-13T18:20:26.191040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:57:10.734935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"91'3","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:27.300317+0000","last_change":"2024-09-13T18:20:01.569201+0000","last_active":"2024-09-13T18:20:27.300317+0000","last_peered":"2024-09-13T18:20:27.300317+0000","last_clean":"2024-09-13T18:20:27.300317+0000","last_became_active":"2024-09-13T18:20:01.568908+0000","last_became_peered":"2024-09-13T18:20:01.568908+0000","last_unstale":"2024-09-13T18:20:27.300317+0000","last_undegraded":"2024-09-13T18:20:27.300317+0000","last_fullsized":"2024-09-13T18:20:27.300317+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:16:34.069130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"91'7","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.205826+0000","last_change":"2024-09-13T18:20:01.579225+0000","last_active":"2024-09-13T18:20:28.205826+0000","last_peered":"2024-09-13T18:20:28.205826+0000","last_clean":"2024-09-13T18:20:28.205826+0000","last_became_active":"2024-09-13T18:20:01.578826+0000","last_became_peered":"2024-09-13T18:20:01.578826+0000","last_unstale":"2024-09-13T18:20:28.205826+0000","last_undegraded":"2024-09-13T18:20:28.205826+0000","last_fullsized":"2024-09-13T18:20:28.205826+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:48:07.632239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.180541+0000","last_change":"2024-09-13T18:20:01.914885+0000","last_active":"2024-09-13T18:20:26.180541+0000","last_peered":"2024-09-13T18:20:26.180541+0000","last_clean":"2024-09-13T18:20:26.180541+0000","last_became_active":"2024-09-13T18:20:01.914599+0000","last_became_peered":"2024-09-13T18:20:01.914599+0000","last_unstale":"2024-09-13T18:20:26.180541+0000","last_undegraded":"2024-09-13T18:20:26.180541+0000","last_fullsized":"2024-09-13T18:20:26.180541+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:41:30.888675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.186528+0000","last_change":"2024-09-13T18:20:06.129376+0000","last_active":"2024-09-13T18:20:26.186528+0000","last_peered":"2024-09-13T18:20:26.186528+0000","last_clean":"2024-09-13T18:20:26.186528+0000","last_became_active":"2024-09-13T18:20:06.129032+0000","last_became_peered":"2024-09-13T18:20:06.129032+0000","last_unstale":"2024-09-13T18:20:26.186528+0000","last_undegraded":"2024-09-13T18:20:26.186528+0000","last_fullsized":"2024-09-13T18:20:26.186528+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:21:53.511813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:29.256218+0000","last_change":"2024-09-13T18:20:01.576069+0000","last_active":"2024-09-13T18:20:29.256218+0000","last_peered":"2024-09-13T18:20:29.256218+0000","last_clean":"2024-09-13T18:20:29.256218+0000","last_became_active":"2024-09-13T18:20:01.575760+0000","last_became_peered":"2024-09-13T18:20:01.575760+0000","last_unstale":"2024-09-13T18:20:29.256218+0000","last_undegraded":"2024-09-13T18:20:29.256218+0000","last_fullsized":"2024-09-13T18:20:29.256218+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:05:47.677848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.186432+0000","last_change":"2024-09-13T18:20:01.917073+0000","last_active":"2024-09-13T18:20:26.186432+0000","last_peered":"2024-09-13T18:20:26.186432+0000","last_clean":"2024-09-13T18:20:26.186432+0000","last_became_active":"2024-09-13T18:20:01.916765+0000","last_became_peered":"2024-09-13T18:20:01.916765+0000","last_unstale":"2024-09-13T18:20:26.186432+0000","last_undegraded":"2024-09-13T18:20:26.186432+0000","last_fullsized":"2024-09-13T18:20:26.186432+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:22:07.788313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.189329+0000","last_change":"2024-09-13T18:20:06.131429+0000","last_active":"2024-09-13T18:20:26.189329+0000","last_peered":"2024-09-13T18:20:26.189329+0000","last_clean":"2024-09-13T18:20:26.189329+0000","last_became_active":"2024-09-13T18:20:06.131019+0000","last_became_peered":"2024-09-13T18:20:06.131019+0000","last_unstale":"2024-09-13T18:20:26.189329+0000","last_undegraded":"2024-09-13T18:20:26.189329+0000","last_fullsized":"2024-09-13T18:20:26.189329+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:44:37.892574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.182865+0000","last_change":"2024-09-13T18:20:01.914050+0000","last_active":"2024-09-13T18:20:26.182865+0000","last_peered":"2024-09-13T18:20:26.182865+0000","last_clean":"2024-09-13T18:20:26.182865+0000","last_became_active":"2024-09-13T18:20:01.913733+0000","last_became_peered":"2024-09-13T18:20:01.913733+0000","last_unstale":"2024-09-13T18:20:26.182865+0000","last_undegraded":"2024-09-13T18:20:26.182865+0000","last_fullsized":"2024-09-13T18:20:26.182865+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:52:43.672096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.188338+0000","last_change":"2024-09-13T18:20:01.924290+0000","last_active":"2024-09-13T18:20:26.188338+0000","last_peered":"2024-09-13T18:20:26.188338+0000","last_clean":"2024-09-13T18:20:26.188338+0000","last_became_active":"2024-09-13T18:20:01.923883+0000","last_became_peered":"2024-09-13T18:20:01.923883+0000","last_unstale":"2024-09-13T18:20:26.188338+0000","last_undegraded":"2024-09-13T18:20:26.188338+0000","last_fullsized":"2024-09-13T18:20:26.188338+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:11:18.737501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:27.184256+0000","last_change":"2024-09-13T18:20:27.184256+0000","last_active":"2024-09-13T18:20:27.184256+0000","last_peered":"2024-09-13T18:20:27.184256+0000","last_clean":"2024-09-13T18:20:27.184256+0000","last_became_active":"2024-09-13T18:20:27.183790+0000","last_became_peered":"2024-09-13T18:20:27.183790+0000","last_unstale":"2024-09-13T18:20:27.184256+0000","last_undegraded":"2024-09-13T18:20:27.184256+0000","last_fullsized":"2024-09-13T18:20:27.184256+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:06:48.885753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.189852+0000","last_change":"2024-09-13T18:20:01.915799+0000","last_active":"2024-09-13T18:20:26.189852+0000","last_peered":"2024-09-13T18:20:26.189852+0000","last_clean":"2024-09-13T18:20:26.189852+0000","last_became_active":"2024-09-13T18:20:01.915519+0000","last_became_peered":"2024-09-13T18:20:01.915519+0000","last_unstale":"2024-09-13T18:20:26.189852+0000","last_undegraded":"2024-09-13T18:20:26.189852+0000","last_fullsized":"2024-09-13T18:20:26.189852+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:08:26.065052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.178968+0000","last_change":"2024-09-13T18:20:01.918547+0000","last_active":"2024-09-13T18:20:26.178968+0000","last_peered":"2024-09-13T18:20:26.178968+0000","last_clean":"2024-09-13T18:20:26.178968+0000","last_became_active":"2024-09-13T18:20:01.918053+0000","last_became_peered":"2024-09-13T18:20:01.918053+0000","last_unstale":"2024-09-13T18:20:26.178968+0000","last_undegraded":"2024-09-13T18:20:26.178968+0000","last_fullsized":"2024-09-13T18:20:26.178968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:25:58.235101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183342+0000","last_change":"2024-09-13T18:20:01.916205+0000","last_active":"2024-09-13T18:20:26.183342+0000","last_peered":"2024-09-13T18:20:26.183342+0000","last_clean":"2024-09-13T18:20:26.183342+0000","last_became_active":"2024-09-13T18:20:01.915909+0000","last_became_peered":"2024-09-13T18:20:01.915909+0000","last_unstale":"2024-09-13T18:20:26.183342+0000","last_undegraded":"2024-09-13T18:20:26.183342+0000","last_fullsized":"2024-09-13T18:20:26.183342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:23:03.976513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.189236+0000","last_change":"2024-09-13T18:20:01.916621+0000","last_active":"2024-09-13T18:20:26.189236+0000","last_peered":"2024-09-13T18:20:26.189236+0000","last_clean":"2024-09-13T18:20:26.189236+0000","last_became_active":"2024-09-13T18:20:01.916335+0000","last_became_peered":"2024-09-13T18:20:01.916335+0000","last_unstale":"2024-09-13T18:20:26.189236+0000","last_undegraded":"2024-09-13T18:20:26.189236+0000","last_fullsized":"2024-09-13T18:20:26.189236+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:08:16.104571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.190215+0000","last_change":"2024-09-13T18:20:01.921195+0000","last_active":"2024-09-13T18:20:26.190215+0000","last_peered":"2024-09-13T18:20:26.190215+0000","last_clean":"2024-09-13T18:20:26.190215+0000","last_became_active":"2024-09-13T18:20:01.920895+0000","last_became_peered":"2024-09-13T18:20:01.920895+0000","last_unstale":"2024-09-13T18:20:26.190215+0000","last_undegraded":"2024-09-13T18:20:26.190215+0000","last_fullsized":"2024-09-13T18:20:26.190215+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:59:53.920047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.180047+0000","last_change":"2024-09-13T18:20:01.910889+0000","last_active":"2024-09-13T18:20:26.180047+0000","last_peered":"2024-09-13T18:20:26.180047+0000","last_clean":"2024-09-13T18:20:26.180047+0000","last_became_active":"2024-09-13T18:20:01.910451+0000","last_became_peered":"2024-09-13T18:20:01.910451+0000","last_unstale":"2024-09-13T18:20:26.180047+0000","last_undegraded":"2024-09-13T18:20:26.180047+0000","last_fullsized":"2024-09-13T18:20:26.180047+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:06:52.552785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.192420+0000","last_change":"2024-09-13T18:20:01.914041+0000","last_active":"2024-09-13T18:20:26.192420+0000","last_peered":"2024-09-13T18:20:26.192420+0000","last_clean":"2024-09-13T18:20:26.192420+0000","last_became_active":"2024-09-13T18:20:01.913240+0000","last_became_peered":"2024-09-13T18:20:01.913240+0000","last_unstale":"2024-09-13T18:20:26.192420+0000","last_undegraded":"2024-09-13T18:20:26.192420+0000","last_fullsized":"2024-09-13T18:20:26.192420+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:39:37.062544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.184154+0000","last_change":"2024-09-13T18:20:01.927322+0000","last_active":"2024-09-13T18:20:26.184154+0000","last_peered":"2024-09-13T18:20:26.184154+0000","last_clean":"2024-09-13T18:20:26.184154+0000","last_became_active":"2024-09-13T18:20:01.926767+0000","last_became_peered":"2024-09-13T18:20:01.926767+0000","last_unstale":"2024-09-13T18:20:26.184154+0000","last_undegraded":"2024-09-13T18:20:26.184154+0000","last_fullsized":"2024-09-13T18:20:26.184154+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:20.986089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.178873+0000","last_change":"2024-09-13T18:20:01.916314+0000","last_active":"2024-09-13T18:20:26.178873+0000","last_peered":"2024-09-13T18:20:26.178873+0000","last_clean":"2024-09-13T18:20:26.178873+0000","last_became_active":"2024-09-13T18:20:01.915790+0000","last_became_peered":"2024-09-13T18:20:01.915790+0000","last_unstale":"2024-09-13T18:20:26.178873+0000","last_undegraded":"2024-09-13T18:20:26.178873+0000","last_fullsized":"2024-09-13T18:20:26.178873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:20:30.063814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:27.187172+0000","last_change":"2024-09-13T18:20:27.187172+0000","last_active":"2024-09-13T18:20:27.187172+0000","last_peered":"2024-09-13T18:20:27.187172+0000","last_clean":"2024-09-13T18:20:27.187172+0000","last_became_active":"2024-09-13T18:20:27.186765+0000","last_became_peered":"2024-09-13T18:20:27.186765+0000","last_unstale":"2024-09-13T18:20:27.187172+0000","last_undegraded":"2024-09-13T18:20:27.187172+0000","last_fullsized":"2024-09-13T18:20:27.187172+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:37:01.469560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.189533+0000","last_change":"2024-09-13T18:20:01.924074+0000","last_active":"2024-09-13T18:20:26.189533+0000","last_peered":"2024-09-13T18:20:26.189533+0000","last_clean":"2024-09-13T18:20:26.189533+0000","last_became_active":"2024-09-13T18:20:01.922364+0000","last_became_peered":"2024-09-13T18:20:01.922364+0000","last_unstale":"2024-09-13T18:20:26.189533+0000","last_undegraded":"2024-09-13T18:20:26.189533+0000","last_fullsized":"2024-09-13T18:20:26.189533+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:51:33.824362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183604+0000","last_change":"2024-09-13T18:20:01.914606+0000","last_active":"2024-09-13T18:20:26.183604+0000","last_peered":"2024-09-13T18:20:26.183604+0000","last_clean":"2024-09-13T18:20:26.183604+0000","last_became_active":"2024-09-13T18:20:01.914281+0000","last_became_peered":"2024-09-13T18:20:01.914281+0000","last_unstale":"2024-09-13T18:20:26.183604+0000","last_undegraded":"2024-09-13T18:20:26.183604+0000","last_fullsized":"2024-09-13T18:20:26.183604+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:55:42.211959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.184054+0000","last_change":"2024-09-13T18:20:01.928876+0000","last_active":"2024-09-13T18:20:26.184054+0000","last_peered":"2024-09-13T18:20:26.184054+0000","last_clean":"2024-09-13T18:20:26.184054+0000","last_became_active":"2024-09-13T18:20:01.928534+0000","last_became_peered":"2024-09-13T18:20:01.928534+0000","last_unstale":"2024-09-13T18:20:26.184054+0000","last_undegraded":"2024-09-13T18:20:26.184054+0000","last_fullsized":"2024-09-13T18:20:26.184054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:14:28.196746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183517+0000","last_change":"2024-09-13T18:20:01.920642+0000","last_active":"2024-09-13T18:20:26.183517+0000","last_peered":"2024-09-13T18:20:26.183517+0000","last_clean":"2024-09-13T18:20:26.183517+0000","last_became_active":"2024-09-13T18:20:01.920335+0000","last_became_peered":"2024-09-13T18:20:01.920335+0000","last_unstale":"2024-09-13T18:20:26.183517+0000","last_undegraded":"2024-09-13T18:20:26.183517+0000","last_fullsized":"2024-09-13T18:20:26.183517+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:59:20.894563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:27.187529+0000","last_change":"2024-09-13T18:20:27.187529+0000","last_active":"2024-09-13T18:20:27.187529+0000","last_peered":"2024-09-13T18:20:27.187529+0000","last_clean":"2024-09-13T18:20:27.187529+0000","last_became_active":"2024-09-13T18:20:27.187123+0000","last_became_peered":"2024-09-13T18:20:27.187123+0000","last_unstale":"2024-09-13T18:20:27.187529+0000","last_undegraded":"2024-09-13T18:20:27.187529+0000","last_fullsized":"2024-09-13T18:20:27.187529+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:43:19.601235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.189462+0000","last_change":"2024-09-13T18:20:01.917111+0000","last_active":"2024-09-13T18:20:26.189462+0000","last_peered":"2024-09-13T18:20:26.189462+0000","last_clean":"2024-09-13T18:20:26.189462+0000","last_became_active":"2024-09-13T18:20:01.916826+0000","last_became_peered":"2024-09-13T18:20:01.916826+0000","last_unstale":"2024-09-13T18:20:26.189462+0000","last_undegraded":"2024-09-13T18:20:26.189462+0000","last_fullsized":"2024-09-13T18:20:26.189462+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:36:08.150028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.178921+0000","last_change":"2024-09-13T18:20:01.904496+0000","last_active":"2024-09-13T18:20:26.178921+0000","last_peered":"2024-09-13T18:20:26.178921+0000","last_clean":"2024-09-13T18:20:26.178921+0000","last_became_active":"2024-09-13T18:20:01.904199+0000","last_became_peered":"2024-09-13T18:20:01.904199+0000","last_unstale":"2024-09-13T18:20:26.178921+0000","last_undegraded":"2024-09-13T18:20:26.178921+0000","last_fullsized":"2024-09-13T18:20:26.178921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:44.044162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.186384+0000","last_change":"2024-09-13T18:20:01.919023+0000","last_active":"2024-09-13T18:20:26.186384+0000","last_peered":"2024-09-13T18:20:26.186384+0000","last_clean":"2024-09-13T18:20:26.186384+0000","last_became_active":"2024-09-13T18:20:01.918661+0000","last_became_peered":"2024-09-13T18:20:01.918661+0000","last_unstale":"2024-09-13T18:20:26.186384+0000","last_undegraded":"2024-09-13T18:20:26.186384+0000","last_fullsized":"2024-09-13T18:20:26.186384+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:25:26.779029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.188169+0000","last_change":"2024-09-13T18:20:01.922833+0000","last_active":"2024-09-13T18:20:26.188169+0000","last_peered":"2024-09-13T18:20:26.188169+0000","last_clean":"2024-09-13T18:20:26.188169+0000","last_became_active":"2024-09-13T18:20:01.922533+0000","last_became_peered":"2024-09-13T18:20:01.922533+0000","last_unstale":"2024-09-13T18:20:26.188169+0000","last_undegraded":"2024-09-13T18:20:26.188169+0000","last_fullsized":"2024-09-13T18:20:26.188169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:27:11.557419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.197907+0000","last_change":"2024-09-13T18:20:01.925916+0000","last_active":"2024-09-13T18:20:26.197907+0000","last_peered":"2024-09-13T18:20:26.197907+0000","last_clean":"2024-09-13T18:20:26.197907+0000","last_became_active":"2024-09-13T18:20:01.925568+0000","last_became_peered":"2024-09-13T18:20:01.925568+0000","last_unstale":"2024-09-13T18:20:26.197907+0000","last_undegraded":"2024-09-13T18:20:26.197907+0000","last_fullsized":"2024-09-13T18:20:26.197907+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:58:24.955273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.184513+0000","last_change":"2024-09-13T18:20:01.924328+0000","last_active":"2024-09-13T18:20:26.184513+0000","last_peered":"2024-09-13T18:20:26.184513+0000","last_clean":"2024-09-13T18:20:26.184513+0000","last_became_active":"2024-09-13T18:20:01.924004+0000","last_became_peered":"2024-09-13T18:20:01.924004+0000","last_unstale":"2024-09-13T18:20:26.184513+0000","last_undegraded":"2024-09-13T18:20:26.184513+0000","last_fullsized":"2024-09-13T18:20:26.184513+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:23:29.204157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.190973+0000","last_change":"2024-09-13T18:20:01.922234+0000","last_active":"2024-09-13T18:20:26.190973+0000","last_peered":"2024-09-13T18:20:26.190973+0000","last_clean":"2024-09-13T18:20:26.190973+0000","last_became_active":"2024-09-13T18:20:01.921196+0000","last_became_peered":"2024-09-13T18:20:01.921196+0000","last_unstale":"2024-09-13T18:20:26.190973+0000","last_undegraded":"2024-09-13T18:20:26.190973+0000","last_fullsized":"2024-09-13T18:20:26.190973+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:24:10.144346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.187949+0000","last_change":"2024-09-13T18:20:01.921740+0000","last_active":"2024-09-13T18:20:26.187949+0000","last_peered":"2024-09-13T18:20:26.187949+0000","last_clean":"2024-09-13T18:20:26.187949+0000","last_became_active":"2024-09-13T18:20:01.919634+0000","last_became_peered":"2024-09-13T18:20:01.919634+0000","last_unstale":"2024-09-13T18:20:26.187949+0000","last_undegraded":"2024-09-13T18:20:26.187949+0000","last_fullsized":"2024-09-13T18:20:26.187949+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:43:27.600637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.197426+0000","last_change":"2024-09-13T18:20:01.924806+0000","last_active":"2024-09-13T18:20:26.197426+0000","last_peered":"2024-09-13T18:20:26.197426+0000","last_clean":"2024-09-13T18:20:26.197426+0000","last_became_active":"2024-09-13T18:20:01.924514+0000","last_became_peered":"2024-09-13T18:20:01.924514+0000","last_unstale":"2024-09-13T18:20:26.197426+0000","last_undegraded":"2024-09-13T18:20:26.197426+0000","last_fullsized":"2024-09-13T18:20:26.197426+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:47.323362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.185515+0000","last_change":"2024-09-13T18:20:01.912402+0000","last_active":"2024-09-13T18:20:26.185515+0000","last_peered":"2024-09-13T18:20:26.185515+0000","last_clean":"2024-09-13T18:20:26.185515+0000","last_became_active":"2024-09-13T18:20:01.911534+0000","last_became_peered":"2024-09-13T18:20:01.911534+0000","last_unstale":"2024-09-13T18:20:26.185515+0000","last_undegraded":"2024-09-13T18:20:26.185515+0000","last_fullsized":"2024-09-13T18:20:26.185515+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:22:52.021688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.186416+0000","last_change":"2024-09-13T18:20:01.930248+0000","last_active":"2024-09-13T18:20:26.186416+0000","last_peered":"2024-09-13T18:20:26.186416+0000","last_clean":"2024-09-13T18:20:26.186416+0000","last_became_active":"2024-09-13T18:20:01.929931+0000","last_became_peered":"2024-09-13T18:20:01.929931+0000","last_unstale":"2024-09-13T18:20:26.186416+0000","last_undegraded":"2024-09-13T18:20:26.186416+0000","last_fullsized":"2024-09-13T18:20:26.186416+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:37:33.086722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.190113+0000","last_change":"2024-09-13T18:20:01.921913+0000","last_active":"2024-09-13T18:20:26.190113+0000","last_peered":"2024-09-13T18:20:26.190113+0000","last_clean":"2024-09-13T18:20:26.190113+0000","last_became_active":"2024-09-13T18:20:01.921395+0000","last_became_peered":"2024-09-13T18:20:01.921395+0000","last_unstale":"2024-09-13T18:20:26.190113+0000","last_undegraded":"2024-09-13T18:20:26.190113+0000","last_fullsized":"2024-09-13T18:20:26.190113+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:54:50.816870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.190140+0000","last_change":"2024-09-13T18:20:01.912654+0000","last_active":"2024-09-13T18:20:26.190140+0000","last_peered":"2024-09-13T18:20:26.190140+0000","last_clean":"2024-09-13T18:20:26.190140+0000","last_became_active":"2024-09-13T18:20:01.912356+0000","last_became_peered":"2024-09-13T18:20:01.912356+0000","last_unstale":"2024-09-13T18:20:26.190140+0000","last_undegraded":"2024-09-13T18:20:26.190140+0000","last_fullsized":"2024-09-13T18:20:26.190140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:25:18.099396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183955+0000","last_change":"2024-09-13T18:20:01.923373+0000","last_active":"2024-09-13T18:20:26.183955+0000","last_peered":"2024-09-13T18:20:26.183955+0000","last_clean":"2024-09-13T18:20:26.183955+0000","last_became_active":"2024-09-13T18:20:01.923077+0000","last_became_peered":"2024-09-13T18:20:01.923077+0000","last_unstale":"2024-09-13T18:20:26.183955+0000","last_undegraded":"2024-09-13T18:20:26.183955+0000","last_fullsized":"2024-09-13T18:20:26.183955+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:07:52.955705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":28672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":16626,"num_objects":92,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":0,"num_read":48,"num_read_kb":44,"num_write":202,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":60,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":78547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":177,"ondisk_log_size":177,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281684,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383698,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":995328,"data_stored":495945,"data_compressed":17567,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.47599999999999998}]},{"osd":1,"last update":"Fri Sep 13 18:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.58499999999999996}]},{"osd":2,"last update":"Fri Sep 13 18:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.622}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":4,"last update":"Fri Sep 13 18:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.64100000000000001}]},{"osd":5,"last update":"Fri Sep 13 18:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.69699999999999995}]},{"osd":6,"last update":"Fri Sep 13 18:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.56699999999999995}]},{"osd":7,"last update":"Fri Sep 13 18:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.68100000000000005}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":9,"last update":"Fri Sep 13 18:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.84999999999999998}]},{"osd":10,"last update":"Fri Sep 13 18:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.751}]}]},{"osd":10,"up_from":72,"seq":309237645335,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1077248,"data_stored":1090703,"data_compressed":53391,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":1,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.66700000000000004}]},{"osd":2,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.625}]},{"osd":3,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.60999999999999999}]},{"osd":4,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.64500000000000002}]},{"osd":5,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.56599999999999995}]},{"osd":6,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.76100000000000001}]},{"osd":7,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.81599999999999995}]},{"osd":8,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.80400000000000005}]},{"osd":9,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.83399999999999996}]},{"osd":11,"last update":"Fri Sep 13 18:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.872}]}]},{"osd":9,"up_from":67,"seq":287762808860,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":500559,"data_compressed":17567,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.311}]},{"osd":1,"last update":"Fri Sep 13 18:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.5}]},{"osd":2,"last update":"Fri Sep 13 18:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.53900000000000003}]},{"osd":3,"last update":"Fri Sep 13 18:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.34300000000000003}]},{"osd":4,"last update":"Fri Sep 13 18:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.57599999999999996}]},{"osd":5,"last update":"Fri Sep 13 18:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.55400000000000005}]},{"osd":6,"last update":"Fri Sep 13 18:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.39800000000000002}]},{"osd":7,"last update":"Fri Sep 13 18:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.47999999999999998}]},{"osd":8,"last update":"Fri Sep 13 18:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.624}]},{"osd":10,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.63900000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]}]},{"osd":8,"up_from":59,"seq":253403070497,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":1019904,"data_stored":500915,"data_compressed":17567,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.53600000000000003,"15min":0.54800000000000004},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.60199999999999998,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.54900000000000004,"15min":0.56200000000000006},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.68600000000000005,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.44700000000000001}]},{"osd":1,"last update":"Fri Sep 13 18:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56000000000000005,"15min":0.55900000000000005},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56999999999999995,"15min":0.57099999999999995},"min":{"1min":0.40600000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72199999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.52300000000000002}]},{"osd":2,"last update":"Fri Sep 13 18:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.42599999999999999,"15min":0.42999999999999999},"min":{"1min":0.26600000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.55200000000000005,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.247,"5min":0.224,"15min":0.224},"max":{"1min":0.59799999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.39300000000000002}]},{"osd":3,"last update":"Fri Sep 13 18:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.54700000000000004,"15min":0.55800000000000005},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.71999999999999997,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53800000000000003,"15min":0.53900000000000003},"min":{"1min":0.38600000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.60799999999999998}]},{"osd":4,"last update":"Fri Sep 13 18:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.66400000000000003,"15min":0.63600000000000001},"min":{"1min":0.45100000000000001,"5min":0.37,"15min":0.37},"max":{"1min":5.3540000000000001,"5min":5.3540000000000001,"15min":5.3540000000000001},"last":0.68000000000000005}]},{"osd":5,"last update":"Fri Sep 13 18:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.53100000000000003,"15min":0.53700000000000003},"min":{"1min":0.31900000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.70099999999999996,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.51900000000000002,"15min":0.52300000000000002},"min":{"1min":0.32100000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.63500000000000001,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.58799999999999997}]},{"osd":6,"last update":"Fri Sep 13 18:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.60599999999999998,"15min":0.61199999999999999},"min":{"1min":0.35199999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.75600000000000001,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.64400000000000002,"15min":0.65000000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.77200000000000002,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.628}]},{"osd":7,"last update":"Fri Sep 13 18:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.66600000000000004,"15min":0.67500000000000004},"min":{"1min":0.44800000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.749,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.65600000000000003,"15min":0.66300000000000003},"min":{"1min":0.39700000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.73899999999999999,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.67200000000000004}]},{"osd":9,"last update":"Fri Sep 13 18:20:24 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63800000000000001,"15min":0.63300000000000001},"min":{"1min":0.46600000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.65900000000000003,"15min":0.65600000000000003},"min":{"1min":0.52800000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.95399999999999996,"5min":1.012,"15min":1.012},"last":0.68999999999999995}]},{"osd":10,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":5.3360000000000003,"5min":5.3360000000000003,"15min":5.3360000000000003},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.72299999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]}]},{"osd":7,"up_from":51,"seq":219043332133,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1077248,"data_stored":1096899,"data_compressed":53391,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.94399999999999995,"15min":1.012},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.84299999999999997,"5min":12.068,"15min":12.068},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.46999999999999997,"15min":0.45900000000000002},"min":{"1min":0.35199999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.61599999999999999}]},{"osd":1,"last update":"Fri Sep 13 18:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.36499999999999999,"15min":0.35699999999999998},"min":{"1min":0.26100000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.38100000000000001,"15min":0.375},"min":{"1min":0.27800000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.62,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.54200000000000004}]},{"osd":2,"last update":"Fri Sep 13 18:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.59099999999999997,"15min":0.53700000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":7.6820000000000004,"5min":7.6820000000000004,"15min":7.6820000000000004},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.51600000000000001,"15min":0.50700000000000001},"min":{"1min":0.378,"5min":0.371,"15min":0.371},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.59999999999999998}]},{"osd":3,"last update":"Fri Sep 13 18:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.49299999999999999,"15min":0.48099999999999998},"min":{"1min":0.36299999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.49299999999999999,"15min":0.48299999999999998},"min":{"1min":0.35399999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.61099999999999999}]},{"osd":4,"last update":"Fri Sep 13 18:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.44,"15min":0.433},"min":{"1min":0.29099999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.621,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.45400000000000001,"15min":0.45000000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.58999999999999997}]},{"osd":5,"last update":"Fri Sep 13 18:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.55700000000000005,"15min":0.55000000000000004},"min":{"1min":0.39900000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.53700000000000003,"15min":0.53100000000000003},"min":{"1min":0.39900000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.68200000000000005}]},{"osd":6,"last update":"Fri Sep 13 18:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.55000000000000004,"15min":0.54000000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.59299999999999997,"15min":0.58199999999999996},"min":{"1min":0.48099999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.74299999999999999}]},{"osd":8,"last update":"Fri Sep 13 18:20:01 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.58999999999999997,"15min":0.58699999999999997},"min":{"1min":0.42499999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.72999999999999998,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.89400000000000002},{"interface":"front","average":{"1min":1.0669999999999999,"5min":0.71899999999999997,"15min":0.66100000000000003},"min":{"1min":0.443,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":7.7149999999999999,"5min":7.7149999999999999,"15min":7.7149999999999999},"last":0.78100000000000003}]},{"osd":9,"last update":"Fri Sep 13 18:20:26 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.80400000000000005,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63600000000000001,"15min":0.63700000000000001},"min":{"1min":0.48599999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.84599999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.73199999999999998}]},{"osd":10,"last update":"Fri Sep 13 18:19:50 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.76800000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]}]},{"osd":6,"up_from":46,"seq":197568495658,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":1003520,"data_stored":496015,"data_compressed":17567,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.248,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.43099999999999999,"15min":0.437},"min":{"1min":0.27600000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.218}]},{"osd":1,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.52700000000000002,"15min":0.52200000000000002},"min":{"1min":0.35499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.69199999999999995,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55000000000000004,"15min":0.53600000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.66500000000000004}]},{"osd":2,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.54200000000000004,"15min":0.54600000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53600000000000003,"15min":0.54700000000000004},"min":{"1min":0.36599999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.75,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.52000000000000002}]},{"osd":3,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42299999999999999,"15min":0.42599999999999999},"min":{"1min":0.28999999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.69499999999999995,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.309},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.46700000000000003,"15min":0.47099999999999997},"min":{"1min":0.30099999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.35599999999999998}]},{"osd":4,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.52300000000000002,"15min":0.51500000000000001},"min":{"1min":0.33300000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55500000000000005,"15min":0.55300000000000005},"min":{"1min":0.39600000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.74199999999999999,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.68300000000000005}]},{"osd":5,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59899999999999998,"15min":0.60799999999999998},"min":{"1min":0.436,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.57399999999999995,"15min":0.59299999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.77900000000000003,"5min":0.874,"15min":0.874},"last":0.496}]},{"osd":7,"last update":"Fri Sep 13 18:19:43 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57399999999999995,"15min":0.57299999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.71299999999999997,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56599999999999995,"15min":0.56200000000000006},"min":{"1min":0.435,"5min":0.434,"15min":0.434},"max":{"1min":0.77100000000000002,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.56899999999999995}]},{"osd":8,"last update":"Fri Sep 13 18:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.61799999999999999,"15min":0.621},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.79000000000000004,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.59399999999999997,"15min":0.59899999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.82999999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.65200000000000002}]},{"osd":9,"last update":"Fri Sep 13 18:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.39500000000000002}]},{"osd":10,"last update":"Fri Sep 13 18:19:49 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.58199999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]}]},{"osd":5,"up_from":38,"seq":163208757295,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1085440,"data_stored":1092577,"data_compressed":53391,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:19:58 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.49299999999999999,"15min":0.49399999999999999},"min":{"1min":0.35299999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.61099999999999999,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.51400000000000001,"15min":0.51200000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.66600000000000004,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.52800000000000002}]},{"osd":1,"last update":"Fri Sep 13 18:19:58 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.51600000000000001,"15min":0.502},"min":{"1min":0.41899999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.77200000000000002,"5min":0.878,"15min":0.878},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.52000000000000002,"15min":0.51000000000000001},"min":{"1min":0.34399999999999997,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.78600000000000003,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.51100000000000001}]},{"osd":2,"last update":"Fri Sep 13 18:19:58 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.376,"15min":0.36199999999999999},"min":{"1min":0.27700000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.39000000000000001,"15min":0.38400000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.58499999999999996,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.33000000000000002}]},{"osd":3,"last update":"Fri Sep 13 18:19:58 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.51600000000000001,"15min":0.501},"min":{"1min":0.35799999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.51500000000000001,"15min":0.51800000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.70499999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.54000000000000004}]},{"osd":4,"last update":"Fri Sep 13 18:19:58 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.54700000000000004,"15min":0.53800000000000003},"min":{"1min":0.40400000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.75900000000000001,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.76400000000000001,"15min":0.60799999999999998},"min":{"1min":0.376,"5min":0.374,"15min":0.374},"max":{"1min":0.80200000000000005,"5min":18.571000000000002,"15min":18.571000000000002},"last":0.60699999999999998}]},{"osd":6,"last update":"Fri Sep 13 18:20:22 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.42399999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55400000000000005,"15min":0.55000000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.749}]},{"osd":7,"last update":"Fri Sep 13 18:19:43 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.61499999999999999,"15min":0.60799999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.61699999999999999,"15min":0.61099999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.85299999999999998,"5min":0.88,"15min":0.88},"last":0.81899999999999995}]},{"osd":8,"last update":"Fri Sep 13 18:20:02 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.51700000000000002,"15min":0.51300000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.73699999999999999,"5min":0.753,"15min":0.753},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.54300000000000004,"15min":0.53700000000000003},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.48299999999999998}]},{"osd":9,"last update":"Fri Sep 13 18:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.63200000000000001,"15min":0.622},"min":{"1min":0.55000000000000004,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.63},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.63300000000000001,"15min":0.626},"min":{"1min":0.53500000000000003,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.78800000000000003}]},{"osd":10,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.86199999999999999}]}]},{"osd":4,"up_from":33,"seq":141733920819,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":500089,"data_compressed":17567,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:19:33 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.48299999999999998,"15min":0.47699999999999998},"min":{"1min":0.35399999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.69399999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.433},{"interface":"front","average":{"1min":0.501,"5min":0.51200000000000001,"15min":0.47499999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.84599999999999997,"5min":4.2539999999999996,"15min":4.2539999999999996},"last":0.51900000000000002}]},{"osd":1,"last update":"Fri Sep 13 18:19:33 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.34899999999999998,"15min":0.34399999999999997},"min":{"1min":0.27600000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.372,"5min":0.34699999999999998,"15min":0.34300000000000003},"min":{"1min":0.25600000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.52700000000000002,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.42199999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:19:33 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.53100000000000003,"15min":0.52300000000000002},"min":{"1min":0.40999999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.53700000000000003,"15min":0.53300000000000003},"min":{"1min":0.38500000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.60599999999999998}]},{"osd":3,"last update":"Fri Sep 13 18:19:33 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.49299999999999999},"min":{"1min":0.35199999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.72599999999999998,"5min":4.2679999999999998,"15min":4.2679999999999998},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.48899999999999999,"15min":0.48499999999999999},"min":{"1min":0.40200000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.50600000000000001}]},{"osd":5,"last update":"Fri Sep 13 18:19:55 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.54800000000000004,"15min":0.53500000000000003},"min":{"1min":0.38500000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78100000000000003,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.55000000000000004,"15min":0.53700000000000003},"min":{"1min":0.39900000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.71099999999999997,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.70999999999999996}]},{"osd":6,"last update":"Fri Sep 13 18:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.52300000000000002,"15min":0.51100000000000001},"min":{"1min":0.41999999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.52300000000000002,"15min":0.50900000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.627}]},{"osd":7,"last update":"Fri Sep 13 18:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.44400000000000001,"15min":0.438},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.442},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.48499999999999999,"15min":0.47699999999999998},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.752}]},{"osd":8,"last update":"Fri Sep 13 18:20:07 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.63,"15min":0.626},"min":{"1min":0.505,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.85899999999999999,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.65000000000000002,"15min":0.64500000000000002},"min":{"1min":0.52300000000000002,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.78200000000000003}]},{"osd":9,"last update":"Fri Sep 13 18:20:26 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.61599999999999999,"15min":0.60999999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.629,"15min":0.627},"min":{"1min":0.46400000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.64000000000000001}]},{"osd":10,"last update":"Fri Sep 13 18:19:51 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.73699999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]}]},{"osd":3,"up_from":25,"seq":107374182455,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":495344,"data_compressed":17567,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:20:10 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.32900000000000001,"15min":0.307},"min":{"1min":0.248,"5min":0.183,"15min":0.183},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.33900000000000002,"15min":0.32700000000000001},"min":{"1min":0.222,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.47599999999999998,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.314}]},{"osd":1,"last update":"Fri Sep 13 18:20:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.50900000000000001,"15min":0.48799999999999999},"min":{"1min":0.40000000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.65600000000000003,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.51400000000000001,"15min":0.48799999999999999},"min":{"1min":0.39400000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.71299999999999997,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.61599999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:20:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52400000000000002,"15min":0.52200000000000002},"min":{"1min":0.35999999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.68700000000000006,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52300000000000002,"15min":0.501},"min":{"1min":0.36599999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.73399999999999999,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.45800000000000002}]},{"osd":4,"last update":"Fri Sep 13 18:19:32 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.5,"15min":0.48599999999999999},"min":{"1min":0.374,"5min":0.312,"15min":0.312},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.501},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.52400000000000002,"15min":0.51400000000000001},"min":{"1min":0.372,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.749,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.433}]},{"osd":5,"last update":"Fri Sep 13 18:19:54 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54000000000000004,"15min":0.53500000000000003},"min":{"1min":0.38300000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.749,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.39600000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72899999999999998,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.51500000000000001}]},{"osd":6,"last update":"Fri Sep 13 18:20:23 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.443,"15min":0.44700000000000001},"min":{"1min":0.30299999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.59199999999999997,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.48099999999999998,"15min":0.46700000000000003},"min":{"1min":0.313,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.70699999999999996,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.443}]},{"osd":7,"last update":"Fri Sep 13 18:19:39 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.80400000000000005,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.57999999999999996,"15min":0.57399999999999995},"min":{"1min":0.44800000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.77500000000000002,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.60899999999999999}]},{"osd":8,"last update":"Fri Sep 13 18:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.69599999999999995}]},{"osd":9,"last update":"Fri Sep 13 18:19:27 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.55900000000000005}]},{"osd":10,"last update":"Fri Sep 13 18:19:49 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.626}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]}]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":537530,"data_compressed":17567,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:19:45 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47399999999999998,"15min":0.47899999999999998},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.57599999999999996,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.48099999999999998,"15min":0.47599999999999998},"min":{"1min":0.318,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.41499999999999998}]},{"osd":1,"last update":"Fri Sep 13 18:19:45 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.49199999999999999,"15min":0.47199999999999998},"min":{"1min":0.34999999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.78400000000000003,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.49099999999999999,"15min":0.46500000000000002},"min":{"1min":0.32300000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.78100000000000003,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.83499999999999996}]},{"osd":3,"last update":"Fri Sep 13 18:20:13 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.52700000000000002,"15min":0.51600000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.66100000000000003,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.55200000000000005,"15min":0.55100000000000005},"min":{"1min":0.377,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.67100000000000004,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.59199999999999997}]},{"osd":4,"last update":"Fri Sep 13 18:19:35 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.504,"15min":0.49099999999999999},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.72899999999999998,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.50900000000000001,"15min":0.503},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.68899999999999995,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.84799999999999998}]},{"osd":5,"last update":"Fri Sep 13 18:19:52 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.38400000000000001,"15min":0.375},"min":{"1min":0.30399999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.434},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.40000000000000002,"15min":0.39000000000000001},"min":{"1min":0.26900000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.58899999999999997,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.53400000000000003}]},{"osd":6,"last update":"Fri Sep 13 18:20:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.56299999999999994,"15min":0.54000000000000004},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.57799999999999996,"15min":0.56299999999999994},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.80100000000000005,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.73599999999999999}]},{"osd":7,"last update":"Fri Sep 13 18:19:37 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78200000000000003,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.61599999999999999,"15min":0.61899999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.76600000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.76700000000000002}]},{"osd":8,"last update":"Fri Sep 13 18:20:07 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.54000000000000004,"15min":0.54200000000000004},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.73899999999999999,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.629},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.36599999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.61399999999999999}]},{"osd":9,"last update":"Fri Sep 13 18:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.624,"15min":0.621},"min":{"1min":0.439,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.78100000000000003,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.65100000000000002,"15min":0.64900000000000002},"min":{"1min":0.45500000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.94399999999999995,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.68500000000000005}]},{"osd":10,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.78700000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":1007616,"data_stored":499248,"data_compressed":17567,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:20:24 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.502,"15min":0.495},"min":{"1min":0.35499999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.626},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.52300000000000002,"15min":0.51500000000000001},"min":{"1min":0.33200000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.77800000000000002,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.439}]},{"osd":2,"last update":"Fri Sep 13 18:19:46 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.501,"15min":0.48999999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.46999999999999997,"15min":0.46899999999999997},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.64200000000000002,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.496}]},{"osd":3,"last update":"Fri Sep 13 18:20:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.53900000000000003,"15min":0.52100000000000002},"min":{"1min":0.38600000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.74399999999999999,"5min":0.749,"15min":0.749},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56499999999999995,"15min":0.54400000000000004},"min":{"1min":0.39700000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76900000000000002,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.52700000000000002}]},{"osd":4,"last update":"Fri Sep 13 18:19:32 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.38,"15min":0.36099999999999999},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.434,"5min":0.376,"15min":0.35599999999999998},"min":{"1min":0.28399999999999997,"5min":0.217,"15min":0.217},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.40500000000000003}]},{"osd":5,"last update":"Fri Sep 13 18:19:53 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56699999999999995,"15min":0.56200000000000006},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55100000000000005,"15min":0.53700000000000003},"min":{"1min":0.40500000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.71299999999999997}]},{"osd":6,"last update":"Fri Sep 13 18:20:23 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57699999999999996,"15min":0.56699999999999995},"min":{"1min":0.41299999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.621,"15min":0.61099999999999999},"min":{"1min":0.45100000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.88300000000000001,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.72899999999999998}]},{"osd":7,"last update":"Fri Sep 13 18:19:38 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.49299999999999999,"15min":0.48699999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.502,"15min":0.5},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.745}]},{"osd":8,"last update":"Fri Sep 13 18:20:07 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.627,"15min":0.629},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.66000000000000003}]},{"osd":9,"last update":"Fri Sep 13 18:19:27 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.70099999999999996}]},{"osd":10,"last update":"Fri Sep 13 18:19:49 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.76600000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":995328,"data_stored":487439,"data_compressed":14601,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 18:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.501,"15min":0.496},"min":{"1min":0.35599999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.37},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.50700000000000001,"15min":0.505},"min":{"1min":0.35699999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.69199999999999995,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.504}]},{"osd":2,"last update":"Fri Sep 13 18:19:47 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.52900000000000003,"15min":0.52700000000000002},"min":{"1min":0.34399999999999997,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.70099999999999996,"5min":0.754,"15min":0.754},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.53000000000000003,"15min":0.51100000000000001},"min":{"1min":0.38500000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.81799999999999995,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.47699999999999998}]},{"osd":3,"last update":"Fri Sep 13 18:19:05 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.38400000000000001,"15min":0.378},"min":{"1min":0.26300000000000001,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.437,"5min":0.39400000000000002,"15min":0.376},"min":{"1min":0.28100000000000003,"5min":0.251,"15min":0.251},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.28799999999999998}]},{"osd":4,"last update":"Fri Sep 13 18:19:33 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.503,"15min":0.49099999999999999},"min":{"1min":0.38600000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.85699999999999998,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.51900000000000002,"15min":0.50700000000000001},"min":{"1min":0.35499999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.45100000000000001}]},{"osd":5,"last update":"Fri Sep 13 18:19:53 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.61499999999999999,"15min":0.56699999999999995},"min":{"1min":0.371,"5min":0.313,"15min":0.313},"max":{"1min":0.89600000000000002,"5min":7.194,"15min":7.194},"last":0.495},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.55100000000000005,"15min":0.52300000000000002},"min":{"1min":0.378,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.87,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.53900000000000003}]},{"osd":6,"last update":"Fri Sep 13 18:20:21 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.46400000000000002,"15min":0.45400000000000001},"min":{"1min":0.28899999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.79000000000000004,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.45800000000000002,"15min":0.46100000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.56699999999999995,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.32400000000000001}]},{"osd":7,"last update":"Fri Sep 13 18:19:38 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.58599999999999997,"15min":0.57599999999999996},"min":{"1min":0.435,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.55500000000000005,"15min":0.54600000000000004},"min":{"1min":0.47299999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.437}]},{"osd":8,"last update":"Fri Sep 13 18:20:01 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.60799999999999998,"15min":0.60999999999999999},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61899999999999999,"15min":0.61699999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.52900000000000003}]},{"osd":9,"last update":"Fri Sep 13 18:19:28 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.41899999999999998}]},{"osd":10,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.55400000000000005}]},{"osd":11,"last update":"Fri Sep 13 18:20:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.621},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.66300000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34188,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34188,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34188,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35824,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35824,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35824,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35824,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35824,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":12212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":42189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":12084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:20:32.268 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 13 18:20:31 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-b[63915]: 2024-09-13T18:20:31.960+0000 7f4e94321640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-13T18:20:32.269 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 13 18:20:31 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-b[63915]: 2024-09-13T18:20:31.960+0000 7f4e94321640 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-09-13T18:20:32.269 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 13 18:20:31 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-b[63915]: 2024-09-13T18:20:31.960+0000 7f4e94321640 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-09-13T18:20:32.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:32 smithi114 ceph-mon[30897]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T18:20:32.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:32 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2650764674' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-09-13T18:20:32.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:32 smithi114 ceph-mon[30897]: pgmap v294: 129 pgs: 1 activating, 128 active+clean; 593 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 7.4 KiB/s wr, 25 op/s; 2 B/s, 0 objects/s recovering 2024-09-13T18:20:32.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:32 smithi114 ceph-mon[30897]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T18:20:32.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:32 smithi134 ceph-mon[34283]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T18:20:32.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:32 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/2650764674' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-09-13T18:20:32.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:32 smithi134 ceph-mon[34283]: pgmap v294: 129 pgs: 1 activating, 128 active+clean; 593 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 7.4 KiB/s wr, 25 op/s; 2 B/s, 0 objects/s recovering 2024-09-13T18:20:32.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:32 smithi134 ceph-mon[34283]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T18:20:32.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:32 smithi204 ceph-mon[34777]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T18:20:32.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:32 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/2650764674' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-09-13T18:20:32.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:32 smithi204 ceph-mon[34777]: pgmap v294: 129 pgs: 1 activating, 128 active+clean; 593 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 7.4 KiB/s wr, 25 op/s; 2 B/s, 0 objects/s recovering 2024-09-13T18:20:32.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:32 smithi204 ceph-mon[34777]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T18:20:35.040 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:20:35.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.149+0000 7f9e50ef7640 1 -- 172.21.15.114:0/3646349527 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e4c0ffe90 msgr2=0x7f9e4c100310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:35.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.149+0000 7f9e50ef7640 1 --2- 172.21.15.114:0/3646349527 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e4c0ffe90 0x7f9e4c100310 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c009900 tx=0x7f9e3c02a410 comp rx=0 tx=0).stop 2024-09-13T18:20:35.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.149+0000 7f9e50ef7640 1 -- 172.21.15.114:0/3646349527 shutdown_connections 2024-09-13T18:20:35.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.149+0000 7f9e50ef7640 1 --2- 172.21.15.114:0/3646349527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e4c100850 0x7f9e4c1070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:35.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.149+0000 7f9e50ef7640 1 --2- 172.21.15.114:0/3646349527 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e4c0ffe90 0x7f9e4c100310 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:35.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.149+0000 7f9e50ef7640 1 --2- 172.21.15.114:0/3646349527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e4c0feca0 0x7f9e4c0ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:35.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.149+0000 7f9e50ef7640 1 -- 172.21.15.114:0/3646349527 >> 172.21.15.114:0/3646349527 conn(0x7f9e4c0fa440 msgr2=0x7f9e4c0fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:35.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.149+0000 7f9e50ef7640 1 -- 172.21.15.114:0/3646349527 shutdown_connections 2024-09-13T18:20:35.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.149+0000 7f9e50ef7640 1 -- 172.21.15.114:0/3646349527 wait complete. 2024-09-13T18:20:35.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.149+0000 7f9e50ef7640 1 Processor -- start 2024-09-13T18:20:35.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.150+0000 7f9e50ef7640 1 -- start start 2024-09-13T18:20:35.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.150+0000 7f9e50ef7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e4c0feca0 0x7f9e4c0680c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:35.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.150+0000 7f9e50ef7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e4c0ffe90 0x7f9e4c19aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:35.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.150+0000 7f9e50ef7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e4c100850 0x7f9e4c19d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:35.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.150+0000 7f9e50ef7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4c19d830 con 0x7f9e4c0ffe90 2024-09-13T18:20:35.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.150+0000 7f9e50ef7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4c19d9a0 con 0x7f9e4c100850 2024-09-13T18:20:35.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.150+0000 7f9e50ef7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4c19db60 con 0x7f9e4c0feca0 2024-09-13T18:20:35.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.150+0000 7f9e49d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e4c0ffe90 0x7f9e4c19aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:35.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.150+0000 7f9e49d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e4c0ffe90 0x7f9e4c19aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:54716/0 (socket says 172.21.15.114:54716) 2024-09-13T18:20:35.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.150+0000 7f9e49d9b640 1 -- 172.21.15.114:0/1826353623 learned_addr learned my addr 172.21.15.114:0/1826353623 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:20:35.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.150+0000 7f9e4ad9d640 1 --2- 172.21.15.114:0/1826353623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e4c100850 0x7f9e4c19d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:35.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.150+0000 7f9e4a59c640 1 --2- 172.21.15.114:0/1826353623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e4c0feca0 0x7f9e4c0680c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:35.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.150+0000 7f9e49d9b640 1 -- 172.21.15.114:0/1826353623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e4c0feca0 msgr2=0x7f9e4c0680c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:35.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.150+0000 7f9e49d9b640 1 --2- 172.21.15.114:0/1826353623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e4c0feca0 0x7f9e4c0680c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:35.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.150+0000 7f9e49d9b640 1 -- 172.21.15.114:0/1826353623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e4c100850 msgr2=0x7f9e4c19d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:35.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.150+0000 7f9e49d9b640 1 --2- 172.21.15.114:0/1826353623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e4c100850 0x7f9e4c19d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:35.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.150+0000 7f9e49d9b640 1 -- 172.21.15.114:0/1826353623 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e3c009560 con 0x7f9e4c0ffe90 2024-09-13T18:20:35.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.150+0000 7f9e4ad9d640 1 --2- 172.21.15.114:0/1826353623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e4c100850 0x7f9e4c19d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:20:35.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.150+0000 7f9e49d9b640 1 --2- 172.21.15.114:0/1826353623 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e4c0ffe90 0x7f9e4c19aad0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c006010 tx=0x7f9e3c004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:35.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.150+0000 7f9e437fe640 1 -- 172.21.15.114:0/1826353623 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c0037f0 con 0x7f9e4c0ffe90 2024-09-13T18:20:35.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.150+0000 7f9e437fe640 1 -- 172.21.15.114:0/1826353623 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9e3c0044f0 con 0x7f9e4c0ffe90 2024-09-13T18:20:35.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.150+0000 7f9e50ef7640 1 -- 172.21.15.114:0/1826353623 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e4c19dde0 con 0x7f9e4c0ffe90 2024-09-13T18:20:35.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.151+0000 7f9e437fe640 1 -- 172.21.15.114:0/1826353623 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c03ca80 con 0x7f9e4c0ffe90 2024-09-13T18:20:35.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.151+0000 7f9e50ef7640 1 -- 172.21.15.114:0/1826353623 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e4c19e230 con 0x7f9e4c0ffe90 2024-09-13T18:20:35.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.152+0000 7f9e50ef7640 1 -- 172.21.15.114:0/1826353623 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e4c1061a0 con 0x7f9e4c0ffe90 2024-09-13T18:20:35.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.153+0000 7f9e437fe640 1 -- 172.21.15.114:0/1826353623 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f9e3c003950 con 0x7f9e4c0ffe90 2024-09-13T18:20:35.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.153+0000 7f9e437fe640 1 --2- 172.21.15.114:0/1826353623 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f9e380a1120 0x7f9e380a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:35.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.153+0000 7f9e4a59c640 1 --2- 172.21.15.114:0/1826353623 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f9e380a1120 0x7f9e380a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:35.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.153+0000 7f9e437fe640 1 -- 172.21.15.114:0/1826353623 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9e3c0eded0 con 0x7f9e4c0ffe90 2024-09-13T18:20:35.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.154+0000 7f9e4a59c640 1 --2- 172.21.15.114:0/1826353623 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f9e380a1120 0x7f9e380a35e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9e34005fd0 tx=0x7f9e34005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:35.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.158+0000 7f9e437fe640 1 -- 172.21.15.114:0/1826353623 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f9e3c0b71f0 con 0x7f9e4c0ffe90 2024-09-13T18:20:35.219 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:34 smithi114 ceph-mon[30897]: pgmap v295: 129 pgs: 129 active+clean; 627 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 30 op/s; 2 B/s, 0 objects/s recovering 2024-09-13T18:20:35.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.321+0000 7f9e50ef7640 1 -- 172.21.15.114:0/1826353623 --> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9e4c02c4f0 con 0x7f9e380a1120 2024-09-13T18:20:35.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.331+0000 7f9e437fe640 1 -- 172.21.15.114:0/1826353623 <== mgr.14148 v2:172.21.15.114:6800/3396185541 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+424223 (secure 0 0 0) 0x7f9e4c02c4f0 con 0x7f9e380a1120 2024-09-13T18:20:35.340 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:20:35.343 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-13T18:20:35.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.335+0000 7f9e50ef7640 1 -- 172.21.15.114:0/1826353623 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f9e380a1120 msgr2=0x7f9e380a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:35.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.335+0000 7f9e50ef7640 1 --2- 172.21.15.114:0/1826353623 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f9e380a1120 0x7f9e380a35e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9e34005fd0 tx=0x7f9e34005850 comp rx=0 tx=0).stop 2024-09-13T18:20:35.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.335+0000 7f9e50ef7640 1 -- 172.21.15.114:0/1826353623 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e4c0ffe90 msgr2=0x7f9e4c19aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:35.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.335+0000 7f9e50ef7640 1 --2- 172.21.15.114:0/1826353623 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e4c0ffe90 0x7f9e4c19aad0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c006010 tx=0x7f9e3c004970 comp rx=0 tx=0).stop 2024-09-13T18:20:35.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.335+0000 7f9e50ef7640 1 -- 172.21.15.114:0/1826353623 shutdown_connections 2024-09-13T18:20:35.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.335+0000 7f9e50ef7640 1 --2- 172.21.15.114:0/1826353623 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f9e380a1120 0x7f9e380a35e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:35.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.335+0000 7f9e50ef7640 1 --2- 172.21.15.114:0/1826353623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e4c100850 0x7f9e4c19d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:35.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.335+0000 7f9e50ef7640 1 --2- 172.21.15.114:0/1826353623 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e4c0ffe90 0x7f9e4c19aad0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:35.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.335+0000 7f9e50ef7640 1 --2- 172.21.15.114:0/1826353623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e4c0feca0 0x7f9e4c0680c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:35.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.335+0000 7f9e50ef7640 1 -- 172.21.15.114:0/1826353623 >> 172.21.15.114:0/1826353623 conn(0x7f9e4c0fa440 msgr2=0x7f9e4c103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:35.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.335+0000 7f9e50ef7640 1 -- 172.21.15.114:0/1826353623 shutdown_connections 2024-09-13T18:20:35.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:35.335+0000 7f9e50ef7640 1 -- 172.21.15.114:0/1826353623 wait complete. 2024-09-13T18:20:35.349 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":295,"stamp":"2024-09-13T18:20:33.909773+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":642457,"num_objects":101,"num_object_clones":0,"num_object_copies":305,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":213,"num_read_kb":221,"num_write":403,"num_write_kb":3053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":65,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":333,"ondisk_log_size":333,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":417,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361024,"kb_used_data":12232,"kb_used_omap":18,"kb_used_meta":348653,"kb_avail":1124580800,"statfs":{"total":1151940427776,"available":1151570739200,"internally_reserved":0,"allocated":12525568,"data_stored":7893621,"data_compressed":318276,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":19129,"internal_metadata":357020999},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":47496,"num_objects":76,"num_object_clones":0,"num_object_copies":230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":77,"num_read_kb":102,"num_write":171,"num_write_kb":97,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":50,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002118"},"pg_stats":[{"pgid":"2.1f","version":"91'13","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.256848+0000","last_change":"2024-09-13T18:20:01.577029+0000","last_active":"2024-09-13T18:20:28.256848+0000","last_peered":"2024-09-13T18:20:28.256848+0000","last_clean":"2024-09-13T18:20:28.256848+0000","last_became_active":"2024-09-13T18:20:01.576710+0000","last_became_peered":"2024-09-13T18:20:01.576710+0000","last_unstale":"2024-09-13T18:20:28.256848+0000","last_undegraded":"2024-09-13T18:20:28.256848+0000","last_fullsized":"2024-09-13T18:20:28.256848+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:25:11.977436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.190449+0000","last_change":"2024-09-13T18:20:01.922955+0000","last_active":"2024-09-13T18:20:26.190449+0000","last_peered":"2024-09-13T18:20:26.190449+0000","last_clean":"2024-09-13T18:20:26.190449+0000","last_became_active":"2024-09-13T18:20:01.922704+0000","last_became_peered":"2024-09-13T18:20:01.922704+0000","last_unstale":"2024-09-13T18:20:26.190449+0000","last_undegraded":"2024-09-13T18:20:26.190449+0000","last_fullsized":"2024-09-13T18:20:26.190449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:44:20.672784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:27.300985+0000","last_change":"2024-09-13T18:20:27.300985+0000","last_active":"2024-09-13T18:20:27.300985+0000","last_peered":"2024-09-13T18:20:27.300985+0000","last_clean":"2024-09-13T18:20:27.300985+0000","last_became_active":"2024-09-13T18:20:27.300546+0000","last_became_peered":"2024-09-13T18:20:27.300546+0000","last_unstale":"2024-09-13T18:20:27.300985+0000","last_undegraded":"2024-09-13T18:20:27.300985+0000","last_fullsized":"2024-09-13T18:20:27.300985+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:00:11.750929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.197866+0000","last_change":"2024-09-13T18:20:01.578921+0000","last_active":"2024-09-13T18:20:26.197866+0000","last_peered":"2024-09-13T18:20:26.197866+0000","last_clean":"2024-09-13T18:20:26.197866+0000","last_became_active":"2024-09-13T18:20:01.578627+0000","last_became_peered":"2024-09-13T18:20:01.578627+0000","last_unstale":"2024-09-13T18:20:26.197866+0000","last_undegraded":"2024-09-13T18:20:26.197866+0000","last_fullsized":"2024-09-13T18:20:26.197866+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:43:32.919156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.190717+0000","last_change":"2024-09-13T18:20:01.922901+0000","last_active":"2024-09-13T18:20:26.190717+0000","last_peered":"2024-09-13T18:20:26.190717+0000","last_clean":"2024-09-13T18:20:26.190717+0000","last_became_active":"2024-09-13T18:20:01.922604+0000","last_became_peered":"2024-09-13T18:20:01.922604+0000","last_unstale":"2024-09-13T18:20:26.190717+0000","last_undegraded":"2024-09-13T18:20:26.190717+0000","last_fullsized":"2024-09-13T18:20:26.190717+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:27:30.650010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.185155+0000","last_change":"2024-09-13T18:20:06.135406+0000","last_active":"2024-09-13T18:20:26.185155+0000","last_peered":"2024-09-13T18:20:26.185155+0000","last_clean":"2024-09-13T18:20:26.185155+0000","last_became_active":"2024-09-13T18:20:06.134558+0000","last_became_peered":"2024-09-13T18:20:06.134558+0000","last_unstale":"2024-09-13T18:20:26.185155+0000","last_undegraded":"2024-09-13T18:20:26.185155+0000","last_fullsized":"2024-09-13T18:20:26.185155+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:10.901966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'11","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.257190+0000","last_change":"2024-09-13T18:20:01.580040+0000","last_active":"2024-09-13T18:20:28.257190+0000","last_peered":"2024-09-13T18:20:28.257190+0000","last_clean":"2024-09-13T18:20:28.257190+0000","last_became_active":"2024-09-13T18:20:01.579715+0000","last_became_peered":"2024-09-13T18:20:01.579715+0000","last_unstale":"2024-09-13T18:20:28.257190+0000","last_undegraded":"2024-09-13T18:20:28.257190+0000","last_fullsized":"2024-09-13T18:20:28.257190+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:57:03.588169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.184089+0000","last_change":"2024-09-13T18:20:01.928314+0000","last_active":"2024-09-13T18:20:26.184089+0000","last_peered":"2024-09-13T18:20:26.184089+0000","last_clean":"2024-09-13T18:20:26.184089+0000","last_became_active":"2024-09-13T18:20:01.925356+0000","last_became_peered":"2024-09-13T18:20:01.925356+0000","last_unstale":"2024-09-13T18:20:26.184089+0000","last_undegraded":"2024-09-13T18:20:26.184089+0000","last_fullsized":"2024-09-13T18:20:26.184089+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:43.906842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.185560+0000","last_change":"2024-09-13T18:20:06.135886+0000","last_active":"2024-09-13T18:20:26.185560+0000","last_peered":"2024-09-13T18:20:26.185560+0000","last_clean":"2024-09-13T18:20:26.185560+0000","last_became_active":"2024-09-13T18:20:06.134587+0000","last_became_peered":"2024-09-13T18:20:06.134587+0000","last_unstale":"2024-09-13T18:20:26.185560+0000","last_undegraded":"2024-09-13T18:20:26.185560+0000","last_fullsized":"2024-09-13T18:20:26.185560+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:44:42.434729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"91'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.256632+0000","last_change":"2024-09-13T18:20:01.576908+0000","last_active":"2024-09-13T18:20:28.256632+0000","last_peered":"2024-09-13T18:20:28.256632+0000","last_clean":"2024-09-13T18:20:28.256632+0000","last_became_active":"2024-09-13T18:20:01.576478+0000","last_became_peered":"2024-09-13T18:20:01.576478+0000","last_unstale":"2024-09-13T18:20:28.256632+0000","last_undegraded":"2024-09-13T18:20:28.256632+0000","last_fullsized":"2024-09-13T18:20:28.256632+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:34:52.126901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.180704+0000","last_change":"2024-09-13T18:20:01.921960+0000","last_active":"2024-09-13T18:20:26.180704+0000","last_peered":"2024-09-13T18:20:26.180704+0000","last_clean":"2024-09-13T18:20:26.180704+0000","last_became_active":"2024-09-13T18:20:01.921652+0000","last_became_peered":"2024-09-13T18:20:01.921652+0000","last_unstale":"2024-09-13T18:20:26.180704+0000","last_undegraded":"2024-09-13T18:20:26.180704+0000","last_fullsized":"2024-09-13T18:20:26.180704+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:00:31.277613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.184886+0000","last_change":"2024-09-13T18:20:06.138983+0000","last_active":"2024-09-13T18:20:26.184886+0000","last_peered":"2024-09-13T18:20:26.184886+0000","last_clean":"2024-09-13T18:20:26.184886+0000","last_became_active":"2024-09-13T18:20:06.138708+0000","last_became_peered":"2024-09-13T18:20:06.138708+0000","last_unstale":"2024-09-13T18:20:26.184886+0000","last_undegraded":"2024-09-13T18:20:26.184886+0000","last_fullsized":"2024-09-13T18:20:26.184886+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:45:21.978217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.187833+0000","last_change":"2024-09-13T18:20:01.929817+0000","last_active":"2024-09-13T18:20:26.187833+0000","last_peered":"2024-09-13T18:20:26.187833+0000","last_clean":"2024-09-13T18:20:26.187833+0000","last_became_active":"2024-09-13T18:20:01.928907+0000","last_became_peered":"2024-09-13T18:20:01.928907+0000","last_unstale":"2024-09-13T18:20:26.187833+0000","last_undegraded":"2024-09-13T18:20:26.187833+0000","last_fullsized":"2024-09-13T18:20:26.187833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:56:25.847230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:27.185633+0000","last_change":"2024-09-13T18:20:01.578968+0000","last_active":"2024-09-13T18:20:27.185633+0000","last_peered":"2024-09-13T18:20:27.185633+0000","last_clean":"2024-09-13T18:20:27.185633+0000","last_became_active":"2024-09-13T18:20:01.578623+0000","last_became_peered":"2024-09-13T18:20:01.578623+0000","last_unstale":"2024-09-13T18:20:27.185633+0000","last_undegraded":"2024-09-13T18:20:27.185633+0000","last_fullsized":"2024-09-13T18:20:27.185633+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:03:30.333107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.186277+0000","last_change":"2024-09-13T18:20:06.135977+0000","last_active":"2024-09-13T18:20:26.186277+0000","last_peered":"2024-09-13T18:20:26.186277+0000","last_clean":"2024-09-13T18:20:26.186277+0000","last_became_active":"2024-09-13T18:20:06.134844+0000","last_became_peered":"2024-09-13T18:20:06.134844+0000","last_unstale":"2024-09-13T18:20:26.186277+0000","last_undegraded":"2024-09-13T18:20:26.186277+0000","last_fullsized":"2024-09-13T18:20:26.186277+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:58:36.139240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.190165+0000","last_change":"2024-09-13T18:20:01.916675+0000","last_active":"2024-09-13T18:20:26.190165+0000","last_peered":"2024-09-13T18:20:26.190165+0000","last_clean":"2024-09-13T18:20:26.190165+0000","last_became_active":"2024-09-13T18:20:01.912902+0000","last_became_peered":"2024-09-13T18:20:01.912902+0000","last_unstale":"2024-09-13T18:20:26.190165+0000","last_undegraded":"2024-09-13T18:20:26.190165+0000","last_fullsized":"2024-09-13T18:20:26.190165+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:24:54.271569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"91'3","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:30.593926+0000","last_change":"2024-09-13T18:20:27.300845+0000","last_active":"2024-09-13T18:20:30.593926+0000","last_peered":"2024-09-13T18:20:30.593926+0000","last_clean":"2024-09-13T18:20:30.593926+0000","last_became_active":"2024-09-13T18:20:27.300417+0000","last_became_peered":"2024-09-13T18:20:27.300417+0000","last_unstale":"2024-09-13T18:20:30.593926+0000","last_undegraded":"2024-09-13T18:20:30.593926+0000","last_fullsized":"2024-09-13T18:20:30.593926+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:44:27.507140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:27.180386+0000","last_change":"2024-09-13T18:20:06.131539+0000","last_active":"2024-09-13T18:20:27.180386+0000","last_peered":"2024-09-13T18:20:27.180386+0000","last_clean":"2024-09-13T18:20:27.180386+0000","last_became_active":"2024-09-13T18:20:06.131148+0000","last_became_peered":"2024-09-13T18:20:06.131148+0000","last_unstale":"2024-09-13T18:20:27.180386+0000","last_undegraded":"2024-09-13T18:20:27.180386+0000","last_fullsized":"2024-09-13T18:20:27.180386+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:32:49.759831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183043+0000","last_change":"2024-09-13T18:20:01.913634+0000","last_active":"2024-09-13T18:20:26.183043+0000","last_peered":"2024-09-13T18:20:26.183043+0000","last_clean":"2024-09-13T18:20:26.183043+0000","last_became_active":"2024-09-13T18:20:01.913128+0000","last_became_peered":"2024-09-13T18:20:01.913128+0000","last_unstale":"2024-09-13T18:20:26.183043+0000","last_undegraded":"2024-09-13T18:20:26.183043+0000","last_fullsized":"2024-09-13T18:20:26.183043+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:37:15.251756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"91'7","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.255486+0000","last_change":"2024-09-13T18:20:01.579385+0000","last_active":"2024-09-13T18:20:28.255486+0000","last_peered":"2024-09-13T18:20:28.255486+0000","last_clean":"2024-09-13T18:20:28.255486+0000","last_became_active":"2024-09-13T18:20:01.579102+0000","last_became_peered":"2024-09-13T18:20:01.579102+0000","last_unstale":"2024-09-13T18:20:28.255486+0000","last_undegraded":"2024-09-13T18:20:28.255486+0000","last_fullsized":"2024-09-13T18:20:28.255486+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:22:00.432052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183422+0000","last_change":"2024-09-13T18:20:06.138110+0000","last_active":"2024-09-13T18:20:26.183422+0000","last_peered":"2024-09-13T18:20:26.183422+0000","last_clean":"2024-09-13T18:20:26.183422+0000","last_became_active":"2024-09-13T18:20:06.137763+0000","last_became_peered":"2024-09-13T18:20:06.137763+0000","last_unstale":"2024-09-13T18:20:26.183422+0000","last_undegraded":"2024-09-13T18:20:26.183422+0000","last_fullsized":"2024-09-13T18:20:26.183422+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:16:15.956216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.185225+0000","last_change":"2024-09-13T18:20:01.918318+0000","last_active":"2024-09-13T18:20:26.185225+0000","last_peered":"2024-09-13T18:20:26.185225+0000","last_clean":"2024-09-13T18:20:26.185225+0000","last_became_active":"2024-09-13T18:20:01.917977+0000","last_became_peered":"2024-09-13T18:20:01.917977+0000","last_unstale":"2024-09-13T18:20:26.185225+0000","last_undegraded":"2024-09-13T18:20:26.185225+0000","last_fullsized":"2024-09-13T18:20:26.185225+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:23.368866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.184253+0000","last_change":"2024-09-13T18:20:01.572009+0000","last_active":"2024-09-13T18:20:26.184253+0000","last_peered":"2024-09-13T18:20:26.184253+0000","last_clean":"2024-09-13T18:20:26.184253+0000","last_became_active":"2024-09-13T18:20:01.571313+0000","last_became_peered":"2024-09-13T18:20:01.571313+0000","last_unstale":"2024-09-13T18:20:26.184253+0000","last_undegraded":"2024-09-13T18:20:26.184253+0000","last_fullsized":"2024-09-13T18:20:26.184253+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:32:59.340476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.186594+0000","last_change":"2024-09-13T18:20:06.132212+0000","last_active":"2024-09-13T18:20:26.186594+0000","last_peered":"2024-09-13T18:20:26.186594+0000","last_clean":"2024-09-13T18:20:26.186594+0000","last_became_active":"2024-09-13T18:20:06.131809+0000","last_became_peered":"2024-09-13T18:20:06.131809+0000","last_unstale":"2024-09-13T18:20:26.186594+0000","last_undegraded":"2024-09-13T18:20:26.186594+0000","last_fullsized":"2024-09-13T18:20:26.186594+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:18:11.667551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:27.301112+0000","last_change":"2024-09-13T18:20:27.301112+0000","last_active":"2024-09-13T18:20:27.301112+0000","last_peered":"2024-09-13T18:20:27.301112+0000","last_clean":"2024-09-13T18:20:27.301112+0000","last_became_active":"2024-09-13T18:20:27.300772+0000","last_became_peered":"2024-09-13T18:20:27.300772+0000","last_unstale":"2024-09-13T18:20:27.301112+0000","last_undegraded":"2024-09-13T18:20:27.301112+0000","last_fullsized":"2024-09-13T18:20:27.301112+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:59:57.606105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"91'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.255245+0000","last_change":"2024-09-13T18:20:01.572025+0000","last_active":"2024-09-13T18:20:28.255245+0000","last_peered":"2024-09-13T18:20:28.255245+0000","last_clean":"2024-09-13T18:20:28.255245+0000","last_became_active":"2024-09-13T18:20:01.571321+0000","last_became_peered":"2024-09-13T18:20:01.571321+0000","last_unstale":"2024-09-13T18:20:28.255245+0000","last_undegraded":"2024-09-13T18:20:28.255245+0000","last_fullsized":"2024-09-13T18:20:28.255245+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:09.374364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.188182+0000","last_change":"2024-09-13T18:20:06.130687+0000","last_active":"2024-09-13T18:20:26.188182+0000","last_peered":"2024-09-13T18:20:26.188182+0000","last_clean":"2024-09-13T18:20:26.188182+0000","last_became_active":"2024-09-13T18:20:06.129839+0000","last_became_peered":"2024-09-13T18:20:06.129839+0000","last_unstale":"2024-09-13T18:20:26.188182+0000","last_undegraded":"2024-09-13T18:20:26.188182+0000","last_fullsized":"2024-09-13T18:20:26.188182+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:12:52.334945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.190458+0000","last_change":"2024-09-13T18:20:01.917244+0000","last_active":"2024-09-13T18:20:26.190458+0000","last_peered":"2024-09-13T18:20:26.190458+0000","last_clean":"2024-09-13T18:20:26.190458+0000","last_became_active":"2024-09-13T18:20:01.916948+0000","last_became_peered":"2024-09-13T18:20:01.916948+0000","last_unstale":"2024-09-13T18:20:26.190458+0000","last_undegraded":"2024-09-13T18:20:26.190458+0000","last_fullsized":"2024-09-13T18:20:26.190458+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:17:11.967096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'7","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.206617+0000","last_change":"2024-09-13T18:20:01.569123+0000","last_active":"2024-09-13T18:20:28.206617+0000","last_peered":"2024-09-13T18:20:28.206617+0000","last_clean":"2024-09-13T18:20:28.206617+0000","last_became_active":"2024-09-13T18:20:01.568720+0000","last_became_peered":"2024-09-13T18:20:01.568720+0000","last_unstale":"2024-09-13T18:20:28.206617+0000","last_undegraded":"2024-09-13T18:20:28.206617+0000","last_fullsized":"2024-09-13T18:20:28.206617+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:04:29.561013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.190854+0000","last_change":"2024-09-13T18:20:06.141771+0000","last_active":"2024-09-13T18:20:26.190854+0000","last_peered":"2024-09-13T18:20:26.190854+0000","last_clean":"2024-09-13T18:20:26.190854+0000","last_became_active":"2024-09-13T18:20:06.141362+0000","last_became_peered":"2024-09-13T18:20:06.141362+0000","last_unstale":"2024-09-13T18:20:26.190854+0000","last_undegraded":"2024-09-13T18:20:26.190854+0000","last_fullsized":"2024-09-13T18:20:26.190854+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:09:58.597605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.179047+0000","last_change":"2024-09-13T18:20:01.911048+0000","last_active":"2024-09-13T18:20:26.179047+0000","last_peered":"2024-09-13T18:20:26.179047+0000","last_clean":"2024-09-13T18:20:26.179047+0000","last_became_active":"2024-09-13T18:20:01.910735+0000","last_became_peered":"2024-09-13T18:20:01.910735+0000","last_unstale":"2024-09-13T18:20:26.179047+0000","last_undegraded":"2024-09-13T18:20:26.179047+0000","last_fullsized":"2024-09-13T18:20:26.179047+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:07:35.447676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"91'4","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.248823+0000","last_change":"2024-09-13T18:20:01.571627+0000","last_active":"2024-09-13T18:20:28.248823+0000","last_peered":"2024-09-13T18:20:28.248823+0000","last_clean":"2024-09-13T18:20:28.248823+0000","last_became_active":"2024-09-13T18:20:01.571250+0000","last_became_peered":"2024-09-13T18:20:01.571250+0000","last_unstale":"2024-09-13T18:20:28.248823+0000","last_undegraded":"2024-09-13T18:20:28.248823+0000","last_fullsized":"2024-09-13T18:20:28.248823+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:34:03.479324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.182932+0000","last_change":"2024-09-13T18:20:06.132974+0000","last_active":"2024-09-13T18:20:26.182932+0000","last_peered":"2024-09-13T18:20:26.182932+0000","last_clean":"2024-09-13T18:20:26.182932+0000","last_became_active":"2024-09-13T18:20:06.131668+0000","last_became_peered":"2024-09-13T18:20:06.131668+0000","last_unstale":"2024-09-13T18:20:26.182932+0000","last_undegraded":"2024-09-13T18:20:26.182932+0000","last_fullsized":"2024-09-13T18:20:26.182932+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:36:30.329761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.197759+0000","last_change":"2024-09-13T18:20:01.904670+0000","last_active":"2024-09-13T18:20:26.197759+0000","last_peered":"2024-09-13T18:20:26.197759+0000","last_clean":"2024-09-13T18:20:26.197759+0000","last_became_active":"2024-09-13T18:20:01.904316+0000","last_became_peered":"2024-09-13T18:20:01.904316+0000","last_unstale":"2024-09-13T18:20:26.197759+0000","last_undegraded":"2024-09-13T18:20:26.197759+0000","last_fullsized":"2024-09-13T18:20:26.197759+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:08:17.687179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"91'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:27.179594+0000","last_change":"2024-09-13T18:20:01.564750+0000","last_active":"2024-09-13T18:20:27.179594+0000","last_peered":"2024-09-13T18:20:27.179594+0000","last_clean":"2024-09-13T18:20:27.179594+0000","last_became_active":"2024-09-13T18:20:01.564096+0000","last_became_peered":"2024-09-13T18:20:01.564096+0000","last_unstale":"2024-09-13T18:20:27.179594+0000","last_undegraded":"2024-09-13T18:20:27.179594+0000","last_fullsized":"2024-09-13T18:20:27.179594+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:42:19.588824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.185367+0000","last_change":"2024-09-13T18:20:06.133386+0000","last_active":"2024-09-13T18:20:26.185367+0000","last_peered":"2024-09-13T18:20:26.185367+0000","last_clean":"2024-09-13T18:20:26.185367+0000","last_became_active":"2024-09-13T18:20:06.133004+0000","last_became_peered":"2024-09-13T18:20:06.133004+0000","last_unstale":"2024-09-13T18:20:26.185367+0000","last_undegraded":"2024-09-13T18:20:26.185367+0000","last_fullsized":"2024-09-13T18:20:26.185367+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:54:46.440410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.197632+0000","last_change":"2024-09-13T18:20:01.912823+0000","last_active":"2024-09-13T18:20:26.197632+0000","last_peered":"2024-09-13T18:20:26.197632+0000","last_clean":"2024-09-13T18:20:26.197632+0000","last_became_active":"2024-09-13T18:20:01.912123+0000","last_became_peered":"2024-09-13T18:20:01.912123+0000","last_unstale":"2024-09-13T18:20:26.197632+0000","last_undegraded":"2024-09-13T18:20:26.197632+0000","last_fullsized":"2024-09-13T18:20:26.197632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:32:33.544816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"91'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:29.259117+0000","last_change":"2024-09-13T18:20:01.577095+0000","last_active":"2024-09-13T18:20:29.259117+0000","last_peered":"2024-09-13T18:20:29.259117+0000","last_clean":"2024-09-13T18:20:29.259117+0000","last_became_active":"2024-09-13T18:20:01.576765+0000","last_became_peered":"2024-09-13T18:20:01.576765+0000","last_unstale":"2024-09-13T18:20:29.259117+0000","last_undegraded":"2024-09-13T18:20:29.259117+0000","last_fullsized":"2024-09-13T18:20:29.259117+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:44:08.750441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.178862+0000","last_change":"2024-09-13T18:20:06.131650+0000","last_active":"2024-09-13T18:20:26.178862+0000","last_peered":"2024-09-13T18:20:26.178862+0000","last_clean":"2024-09-13T18:20:26.178862+0000","last_became_active":"2024-09-13T18:20:06.130902+0000","last_became_peered":"2024-09-13T18:20:06.130902+0000","last_unstale":"2024-09-13T18:20:26.178862+0000","last_undegraded":"2024-09-13T18:20:26.178862+0000","last_fullsized":"2024-09-13T18:20:26.178862+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.366694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.180095+0000","last_change":"2024-09-13T18:20:01.918221+0000","last_active":"2024-09-13T18:20:26.180095+0000","last_peered":"2024-09-13T18:20:26.180095+0000","last_clean":"2024-09-13T18:20:26.180095+0000","last_became_active":"2024-09-13T18:20:01.917134+0000","last_became_peered":"2024-09-13T18:20:01.917134+0000","last_unstale":"2024-09-13T18:20:26.180095+0000","last_undegraded":"2024-09-13T18:20:26.180095+0000","last_fullsized":"2024-09-13T18:20:26.180095+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:19:06.269454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:32.257017+0000","last_change":"2024-09-13T18:20:01.572258+0000","last_active":"2024-09-13T18:20:32.257017+0000","last_peered":"2024-09-13T18:20:32.257017+0000","last_clean":"2024-09-13T18:20:32.257017+0000","last_became_active":"2024-09-13T18:20:01.571968+0000","last_became_peered":"2024-09-13T18:20:01.571968+0000","last_unstale":"2024-09-13T18:20:32.257017+0000","last_undegraded":"2024-09-13T18:20:32.257017+0000","last_fullsized":"2024-09-13T18:20:32.257017+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:40:59.143123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.179018+0000","last_change":"2024-09-13T18:20:06.131714+0000","last_active":"2024-09-13T18:20:26.179018+0000","last_peered":"2024-09-13T18:20:26.179018+0000","last_clean":"2024-09-13T18:20:26.179018+0000","last_became_active":"2024-09-13T18:20:06.131244+0000","last_became_peered":"2024-09-13T18:20:06.131244+0000","last_unstale":"2024-09-13T18:20:26.179018+0000","last_undegraded":"2024-09-13T18:20:26.179018+0000","last_fullsized":"2024-09-13T18:20:26.179018+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:37.946002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.178584+0000","last_change":"2024-09-13T18:20:01.917351+0000","last_active":"2024-09-13T18:20:26.178584+0000","last_peered":"2024-09-13T18:20:26.178584+0000","last_clean":"2024-09-13T18:20:26.178584+0000","last_became_active":"2024-09-13T18:20:01.917067+0000","last_became_peered":"2024-09-13T18:20:01.917067+0000","last_unstale":"2024-09-13T18:20:26.178584+0000","last_undegraded":"2024-09-13T18:20:26.178584+0000","last_fullsized":"2024-09-13T18:20:26.178584+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:35:37.723132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'4","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:27.182812+0000","last_change":"2024-09-13T18:20:01.573515+0000","last_active":"2024-09-13T18:20:27.182812+0000","last_peered":"2024-09-13T18:20:27.182812+0000","last_clean":"2024-09-13T18:20:27.182812+0000","last_became_active":"2024-09-13T18:20:01.573211+0000","last_became_peered":"2024-09-13T18:20:01.573211+0000","last_unstale":"2024-09-13T18:20:27.182812+0000","last_undegraded":"2024-09-13T18:20:27.182812+0000","last_fullsized":"2024-09-13T18:20:27.182812+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:05:30.912467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.180826+0000","last_change":"2024-09-13T18:20:06.133713+0000","last_active":"2024-09-13T18:20:26.180826+0000","last_peered":"2024-09-13T18:20:26.180826+0000","last_clean":"2024-09-13T18:20:26.180826+0000","last_became_active":"2024-09-13T18:20:06.133330+0000","last_became_peered":"2024-09-13T18:20:06.133330+0000","last_unstale":"2024-09-13T18:20:26.180826+0000","last_undegraded":"2024-09-13T18:20:26.180826+0000","last_fullsized":"2024-09-13T18:20:26.180826+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:48:07.133155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.192066+0000","last_change":"2024-09-13T18:20:01.921597+0000","last_active":"2024-09-13T18:20:26.192066+0000","last_peered":"2024-09-13T18:20:26.192066+0000","last_clean":"2024-09-13T18:20:26.192066+0000","last_became_active":"2024-09-13T18:20:01.920200+0000","last_became_peered":"2024-09-13T18:20:01.920200+0000","last_unstale":"2024-09-13T18:20:26.192066+0000","last_undegraded":"2024-09-13T18:20:26.192066+0000","last_fullsized":"2024-09-13T18:20:26.192066+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:22:40.517941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.255802+0000","last_change":"2024-09-13T18:20:01.571313+0000","last_active":"2024-09-13T18:20:28.255802+0000","last_peered":"2024-09-13T18:20:28.255802+0000","last_clean":"2024-09-13T18:20:28.255802+0000","last_became_active":"2024-09-13T18:20:01.570977+0000","last_became_peered":"2024-09-13T18:20:01.570977+0000","last_unstale":"2024-09-13T18:20:28.255802+0000","last_undegraded":"2024-09-13T18:20:28.255802+0000","last_fullsized":"2024-09-13T18:20:28.255802+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:06.870320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":343,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.187790+0000","last_change":"2024-09-13T18:18:51.237409+0000","last_active":"2024-09-13T18:20:26.187790+0000","last_peered":"2024-09-13T18:20:26.187790+0000","last_clean":"2024-09-13T18:20:26.187790+0000","last_became_active":"2024-09-13T18:18:51.236986+0000","last_became_peered":"2024-09-13T18:18:51.236986+0000","last_unstale":"2024-09-13T18:20:26.187790+0000","last_undegraded":"2024-09-13T18:20:26.187790+0000","last_fullsized":"2024-09-13T18:20:26.187790+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:15:40.757299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:15:40.757299+0000","last_clean_scrub_stamp":"2024-09-13T18:15:40.757299+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:58:29.679404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:27.182752+0000","last_change":"2024-09-13T18:20:06.130803+0000","last_active":"2024-09-13T18:20:27.182752+0000","last_peered":"2024-09-13T18:20:27.182752+0000","last_clean":"2024-09-13T18:20:27.182752+0000","last_became_active":"2024-09-13T18:20:06.130433+0000","last_became_peered":"2024-09-13T18:20:06.130433+0000","last_unstale":"2024-09-13T18:20:27.182752+0000","last_undegraded":"2024-09-13T18:20:27.182752+0000","last_fullsized":"2024-09-13T18:20:27.182752+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:23:11.787084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.188184+0000","last_change":"2024-09-13T18:20:01.922702+0000","last_active":"2024-09-13T18:20:26.188184+0000","last_peered":"2024-09-13T18:20:26.188184+0000","last_clean":"2024-09-13T18:20:26.188184+0000","last_became_active":"2024-09-13T18:20:01.921109+0000","last_became_peered":"2024-09-13T18:20:01.921109+0000","last_unstale":"2024-09-13T18:20:26.188184+0000","last_undegraded":"2024-09-13T18:20:26.188184+0000","last_fullsized":"2024-09-13T18:20:26.188184+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:03:34.618054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"91'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:29.257448+0000","last_change":"2024-09-13T18:20:01.571024+0000","last_active":"2024-09-13T18:20:29.257448+0000","last_peered":"2024-09-13T18:20:29.257448+0000","last_clean":"2024-09-13T18:20:29.257448+0000","last_became_active":"2024-09-13T18:20:01.570687+0000","last_became_peered":"2024-09-13T18:20:01.570687+0000","last_unstale":"2024-09-13T18:20:29.257448+0000","last_undegraded":"2024-09-13T18:20:29.257448+0000","last_fullsized":"2024-09-13T18:20:29.257448+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:10:25.126179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183401+0000","last_change":"2024-09-13T18:20:06.129025+0000","last_active":"2024-09-13T18:20:26.183401+0000","last_peered":"2024-09-13T18:20:26.183401+0000","last_clean":"2024-09-13T18:20:26.183401+0000","last_became_active":"2024-09-13T18:20:06.128618+0000","last_became_peered":"2024-09-13T18:20:06.128618+0000","last_unstale":"2024-09-13T18:20:26.183401+0000","last_undegraded":"2024-09-13T18:20:26.183401+0000","last_fullsized":"2024-09-13T18:20:26.183401+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:20:08.891259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.187706+0000","last_change":"2024-09-13T18:20:01.921697+0000","last_active":"2024-09-13T18:20:26.187706+0000","last_peered":"2024-09-13T18:20:26.187706+0000","last_clean":"2024-09-13T18:20:26.187706+0000","last_became_active":"2024-09-13T18:20:01.920174+0000","last_became_peered":"2024-09-13T18:20:01.920174+0000","last_unstale":"2024-09-13T18:20:26.187706+0000","last_undegraded":"2024-09-13T18:20:26.187706+0000","last_fullsized":"2024-09-13T18:20:26.187706+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:57:03.922725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"91'3","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.247610+0000","last_change":"2024-09-13T18:20:01.582102+0000","last_active":"2024-09-13T18:20:28.247610+0000","last_peered":"2024-09-13T18:20:28.247610+0000","last_clean":"2024-09-13T18:20:28.247610+0000","last_became_active":"2024-09-13T18:20:01.581841+0000","last_became_peered":"2024-09-13T18:20:01.581841+0000","last_unstale":"2024-09-13T18:20:28.247610+0000","last_undegraded":"2024-09-13T18:20:28.247610+0000","last_fullsized":"2024-09-13T18:20:28.247610+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:59:57.929703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.189924+0000","last_change":"2024-09-13T18:20:06.133126+0000","last_active":"2024-09-13T18:20:26.189924+0000","last_peered":"2024-09-13T18:20:26.189924+0000","last_clean":"2024-09-13T18:20:26.189924+0000","last_became_active":"2024-09-13T18:20:06.132747+0000","last_became_peered":"2024-09-13T18:20:06.132747+0000","last_unstale":"2024-09-13T18:20:26.189924+0000","last_undegraded":"2024-09-13T18:20:26.189924+0000","last_fullsized":"2024-09-13T18:20:26.189924+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:54:21.843090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.187974+0000","last_change":"2024-09-13T18:20:01.916306+0000","last_active":"2024-09-13T18:20:26.187974+0000","last_peered":"2024-09-13T18:20:26.187974+0000","last_clean":"2024-09-13T18:20:26.187974+0000","last_became_active":"2024-09-13T18:20:01.915739+0000","last_became_peered":"2024-09-13T18:20:01.915739+0000","last_unstale":"2024-09-13T18:20:26.187974+0000","last_undegraded":"2024-09-13T18:20:26.187974+0000","last_fullsized":"2024-09-13T18:20:26.187974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:19:09.689909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"91'5","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.252883+0000","last_change":"2024-09-13T18:20:01.565821+0000","last_active":"2024-09-13T18:20:28.252883+0000","last_peered":"2024-09-13T18:20:28.252883+0000","last_clean":"2024-09-13T18:20:28.252883+0000","last_became_active":"2024-09-13T18:20:01.565467+0000","last_became_peered":"2024-09-13T18:20:01.565467+0000","last_unstale":"2024-09-13T18:20:28.252883+0000","last_undegraded":"2024-09-13T18:20:28.252883+0000","last_fullsized":"2024-09-13T18:20:28.252883+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:11.609400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.179986+0000","last_change":"2024-09-13T18:20:06.131348+0000","last_active":"2024-09-13T18:20:26.179986+0000","last_peered":"2024-09-13T18:20:26.179986+0000","last_clean":"2024-09-13T18:20:26.179986+0000","last_became_active":"2024-09-13T18:20:06.131019+0000","last_became_peered":"2024-09-13T18:20:06.131019+0000","last_unstale":"2024-09-13T18:20:26.179986+0000","last_undegraded":"2024-09-13T18:20:26.179986+0000","last_fullsized":"2024-09-13T18:20:26.179986+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:13:49.717503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183415+0000","last_change":"2024-09-13T18:20:01.922306+0000","last_active":"2024-09-13T18:20:26.183415+0000","last_peered":"2024-09-13T18:20:26.183415+0000","last_clean":"2024-09-13T18:20:26.183415+0000","last_became_active":"2024-09-13T18:20:01.921956+0000","last_became_peered":"2024-09-13T18:20:01.921956+0000","last_unstale":"2024-09-13T18:20:26.183415+0000","last_undegraded":"2024-09-13T18:20:26.183415+0000","last_fullsized":"2024-09-13T18:20:26.183415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:17:48.755318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.201613+0000","last_change":"2024-09-13T18:20:27.194085+0000","last_active":"2024-09-13T18:20:28.201613+0000","last_peered":"2024-09-13T18:20:28.201613+0000","last_clean":"2024-09-13T18:20:28.201613+0000","last_became_active":"2024-09-13T18:20:27.190136+0000","last_became_peered":"2024-09-13T18:20:27.190136+0000","last_unstale":"2024-09-13T18:20:28.201613+0000","last_undegraded":"2024-09-13T18:20:28.201613+0000","last_fullsized":"2024-09-13T18:20:28.201613+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:57:07.202901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.180041+0000","last_change":"2024-09-13T18:20:06.135498+0000","last_active":"2024-09-13T18:20:26.180041+0000","last_peered":"2024-09-13T18:20:26.180041+0000","last_clean":"2024-09-13T18:20:26.180041+0000","last_became_active":"2024-09-13T18:20:06.135095+0000","last_became_peered":"2024-09-13T18:20:06.135095+0000","last_unstale":"2024-09-13T18:20:26.180041+0000","last_undegraded":"2024-09-13T18:20:26.180041+0000","last_fullsized":"2024-09-13T18:20:26.180041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:06:40.609718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:27.187296+0000","last_change":"2024-09-13T18:20:27.186825+0000","last_active":"2024-09-13T18:20:27.187296+0000","last_peered":"2024-09-13T18:20:27.187296+0000","last_clean":"2024-09-13T18:20:27.187296+0000","last_became_active":"2024-09-13T18:20:27.186523+0000","last_became_peered":"2024-09-13T18:20:27.186523+0000","last_unstale":"2024-09-13T18:20:27.187296+0000","last_undegraded":"2024-09-13T18:20:27.187296+0000","last_fullsized":"2024-09-13T18:20:27.187296+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:28:30.453945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.10","version":"91'17","reported_seq":57,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.248094+0000","last_change":"2024-09-13T18:20:27.200142+0000","last_active":"2024-09-13T18:20:28.248094+0000","last_peered":"2024-09-13T18:20:28.248094+0000","last_clean":"2024-09-13T18:20:28.248094+0000","last_became_active":"2024-09-13T18:20:27.189670+0000","last_became_peered":"2024-09-13T18:20:27.189670+0000","last_unstale":"2024-09-13T18:20:28.248094+0000","last_undegraded":"2024-09-13T18:20:28.248094+0000","last_fullsized":"2024-09-13T18:20:28.248094+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:16:02.934269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.179043+0000","last_change":"2024-09-13T18:20:06.136742+0000","last_active":"2024-09-13T18:20:26.179043+0000","last_peered":"2024-09-13T18:20:26.179043+0000","last_clean":"2024-09-13T18:20:26.179043+0000","last_became_active":"2024-09-13T18:20:06.136373+0000","last_became_peered":"2024-09-13T18:20:06.136373+0000","last_unstale":"2024-09-13T18:20:26.179043+0000","last_undegraded":"2024-09-13T18:20:26.179043+0000","last_fullsized":"2024-09-13T18:20:26.179043+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:49:11.782489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.187637+0000","last_change":"2024-09-13T18:20:01.922910+0000","last_active":"2024-09-13T18:20:26.187637+0000","last_peered":"2024-09-13T18:20:26.187637+0000","last_clean":"2024-09-13T18:20:26.187637+0000","last_became_active":"2024-09-13T18:20:01.921162+0000","last_became_peered":"2024-09-13T18:20:01.921162+0000","last_unstale":"2024-09-13T18:20:26.187637+0000","last_undegraded":"2024-09-13T18:20:26.187637+0000","last_fullsized":"2024-09-13T18:20:26.187637+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:14:47.243418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"91'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:30.255078+0000","last_change":"2024-09-13T18:20:01.575830+0000","last_active":"2024-09-13T18:20:30.255078+0000","last_peered":"2024-09-13T18:20:30.255078+0000","last_clean":"2024-09-13T18:20:30.255078+0000","last_became_active":"2024-09-13T18:20:01.575334+0000","last_became_peered":"2024-09-13T18:20:01.575334+0000","last_unstale":"2024-09-13T18:20:30.255078+0000","last_undegraded":"2024-09-13T18:20:30.255078+0000","last_fullsized":"2024-09-13T18:20:30.255078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:13:00.935600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":7,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183501+0000","last_change":"2024-09-13T18:20:06.138609+0000","last_active":"2024-09-13T18:20:26.183501+0000","last_peered":"2024-09-13T18:20:26.183501+0000","last_clean":"2024-09-13T18:20:26.183501+0000","last_became_active":"2024-09-13T18:20:06.138232+0000","last_became_peered":"2024-09-13T18:20:06.138232+0000","last_unstale":"2024-09-13T18:20:26.183501+0000","last_undegraded":"2024-09-13T18:20:26.183501+0000","last_fullsized":"2024-09-13T18:20:26.183501+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:46:06.227407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.187782+0000","last_change":"2024-09-13T18:20:01.928757+0000","last_active":"2024-09-13T18:20:26.187782+0000","last_peered":"2024-09-13T18:20:26.187782+0000","last_clean":"2024-09-13T18:20:26.187782+0000","last_became_active":"2024-09-13T18:20:01.928350+0000","last_became_peered":"2024-09-13T18:20:01.928350+0000","last_unstale":"2024-09-13T18:20:26.187782+0000","last_undegraded":"2024-09-13T18:20:26.187782+0000","last_fullsized":"2024-09-13T18:20:26.187782+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:17:43.208106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"91'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.257161+0000","last_change":"2024-09-13T18:20:01.579128+0000","last_active":"2024-09-13T18:20:28.257161+0000","last_peered":"2024-09-13T18:20:28.257161+0000","last_clean":"2024-09-13T18:20:28.257161+0000","last_became_active":"2024-09-13T18:20:01.578796+0000","last_became_peered":"2024-09-13T18:20:01.578796+0000","last_unstale":"2024-09-13T18:20:28.257161+0000","last_undegraded":"2024-09-13T18:20:28.257161+0000","last_fullsized":"2024-09-13T18:20:28.257161+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:30:04.836103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.182950+0000","last_change":"2024-09-13T18:20:06.131404+0000","last_active":"2024-09-13T18:20:26.182950+0000","last_peered":"2024-09-13T18:20:26.182950+0000","last_clean":"2024-09-13T18:20:26.182950+0000","last_became_active":"2024-09-13T18:20:06.130931+0000","last_became_peered":"2024-09-13T18:20:06.130931+0000","last_unstale":"2024-09-13T18:20:26.182950+0000","last_undegraded":"2024-09-13T18:20:26.182950+0000","last_fullsized":"2024-09-13T18:20:26.182950+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:55:54.568816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.178724+0000","last_change":"2024-09-13T18:20:01.916089+0000","last_active":"2024-09-13T18:20:26.178724+0000","last_peered":"2024-09-13T18:20:26.178724+0000","last_clean":"2024-09-13T18:20:26.178724+0000","last_became_active":"2024-09-13T18:20:01.915783+0000","last_became_peered":"2024-09-13T18:20:01.915783+0000","last_unstale":"2024-09-13T18:20:26.178724+0000","last_undegraded":"2024-09-13T18:20:26.178724+0000","last_fullsized":"2024-09-13T18:20:26.178724+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:43:25.886029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"91'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.255522+0000","last_change":"2024-09-13T18:20:01.577381+0000","last_active":"2024-09-13T18:20:28.255522+0000","last_peered":"2024-09-13T18:20:28.255522+0000","last_clean":"2024-09-13T18:20:28.255522+0000","last_became_active":"2024-09-13T18:20:01.577044+0000","last_became_peered":"2024-09-13T18:20:01.577044+0000","last_unstale":"2024-09-13T18:20:28.255522+0000","last_undegraded":"2024-09-13T18:20:28.255522+0000","last_fullsized":"2024-09-13T18:20:28.255522+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:00.907948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.180609+0000","last_change":"2024-09-13T18:20:06.139971+0000","last_active":"2024-09-13T18:20:26.180609+0000","last_peered":"2024-09-13T18:20:26.180609+0000","last_clean":"2024-09-13T18:20:26.180609+0000","last_became_active":"2024-09-13T18:20:06.139507+0000","last_became_peered":"2024-09-13T18:20:06.139507+0000","last_unstale":"2024-09-13T18:20:26.180609+0000","last_undegraded":"2024-09-13T18:20:26.180609+0000","last_fullsized":"2024-09-13T18:20:26.180609+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:40:21.037768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.186323+0000","last_change":"2024-09-13T18:20:01.915348+0000","last_active":"2024-09-13T18:20:26.186323+0000","last_peered":"2024-09-13T18:20:26.186323+0000","last_clean":"2024-09-13T18:20:26.186323+0000","last_became_active":"2024-09-13T18:20:01.915042+0000","last_became_peered":"2024-09-13T18:20:01.915042+0000","last_unstale":"2024-09-13T18:20:26.186323+0000","last_undegraded":"2024-09-13T18:20:26.186323+0000","last_fullsized":"2024-09-13T18:20:26.186323+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:31:33.712314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"91'26","reported_seq":60,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:33.263314+0000","last_change":"2024-09-13T18:20:01.578456+0000","last_active":"2024-09-13T18:20:33.263314+0000","last_peered":"2024-09-13T18:20:33.263314+0000","last_clean":"2024-09-13T18:20:33.263314+0000","last_became_active":"2024-09-13T18:20:01.578051+0000","last_became_peered":"2024-09-13T18:20:01.578051+0000","last_unstale":"2024-09-13T18:20:33.263314+0000","last_undegraded":"2024-09-13T18:20:33.263314+0000","last_fullsized":"2024-09-13T18:20:33.263314+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:34:17.651620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45625,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":16,"num_read_kb":53,"num_write":28,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183092+0000","last_change":"2024-09-13T18:20:06.131410+0000","last_active":"2024-09-13T18:20:26.183092+0000","last_peered":"2024-09-13T18:20:26.183092+0000","last_clean":"2024-09-13T18:20:26.183092+0000","last_became_active":"2024-09-13T18:20:06.131060+0000","last_became_peered":"2024-09-13T18:20:06.131060+0000","last_unstale":"2024-09-13T18:20:26.183092+0000","last_undegraded":"2024-09-13T18:20:26.183092+0000","last_fullsized":"2024-09-13T18:20:26.183092+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:08:25.266971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183935+0000","last_change":"2024-09-13T18:20:01.928839+0000","last_active":"2024-09-13T18:20:26.183935+0000","last_peered":"2024-09-13T18:20:26.183935+0000","last_clean":"2024-09-13T18:20:26.183935+0000","last_became_active":"2024-09-13T18:20:01.925774+0000","last_became_peered":"2024-09-13T18:20:01.925774+0000","last_unstale":"2024-09-13T18:20:26.183935+0000","last_undegraded":"2024-09-13T18:20:26.183935+0000","last_fullsized":"2024-09-13T18:20:26.183935+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:15:23.049671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"91'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.206178+0000","last_change":"2024-09-13T18:20:01.576332+0000","last_active":"2024-09-13T18:20:28.206178+0000","last_peered":"2024-09-13T18:20:28.206178+0000","last_clean":"2024-09-13T18:20:28.206178+0000","last_became_active":"2024-09-13T18:20:01.575862+0000","last_became_peered":"2024-09-13T18:20:01.575862+0000","last_unstale":"2024-09-13T18:20:28.206178+0000","last_undegraded":"2024-09-13T18:20:28.206178+0000","last_fullsized":"2024-09-13T18:20:28.206178+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:17:00.452480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.186104+0000","last_change":"2024-09-13T18:20:06.133142+0000","last_active":"2024-09-13T18:20:26.186104+0000","last_peered":"2024-09-13T18:20:26.186104+0000","last_clean":"2024-09-13T18:20:26.186104+0000","last_became_active":"2024-09-13T18:20:06.132740+0000","last_became_peered":"2024-09-13T18:20:06.132740+0000","last_unstale":"2024-09-13T18:20:26.186104+0000","last_undegraded":"2024-09-13T18:20:26.186104+0000","last_fullsized":"2024-09-13T18:20:26.186104+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:12:46.442481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.190799+0000","last_change":"2024-09-13T18:20:01.919673+0000","last_active":"2024-09-13T18:20:26.190799+0000","last_peered":"2024-09-13T18:20:26.190799+0000","last_clean":"2024-09-13T18:20:26.190799+0000","last_became_active":"2024-09-13T18:20:01.919358+0000","last_became_peered":"2024-09-13T18:20:01.919358+0000","last_unstale":"2024-09-13T18:20:26.190799+0000","last_undegraded":"2024-09-13T18:20:26.190799+0000","last_fullsized":"2024-09-13T18:20:26.190799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:40:11.407643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"91'4","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.256430+0000","last_change":"2024-09-13T18:20:27.190451+0000","last_active":"2024-09-13T18:20:28.256430+0000","last_peered":"2024-09-13T18:20:28.256430+0000","last_clean":"2024-09-13T18:20:28.256430+0000","last_became_active":"2024-09-13T18:20:27.187091+0000","last_became_peered":"2024-09-13T18:20:27.187091+0000","last_unstale":"2024-09-13T18:20:28.256430+0000","last_undegraded":"2024-09-13T18:20:28.256430+0000","last_fullsized":"2024-09-13T18:20:28.256430+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:57:07.976008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.182915+0000","last_change":"2024-09-13T18:20:06.133475+0000","last_active":"2024-09-13T18:20:26.182915+0000","last_peered":"2024-09-13T18:20:26.182915+0000","last_clean":"2024-09-13T18:20:26.182915+0000","last_became_active":"2024-09-13T18:20:06.133109+0000","last_became_peered":"2024-09-13T18:20:06.133109+0000","last_unstale":"2024-09-13T18:20:26.182915+0000","last_undegraded":"2024-09-13T18:20:26.182915+0000","last_fullsized":"2024-09-13T18:20:26.182915+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:47:19.567115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.184156+0000","last_change":"2024-09-13T18:20:01.922660+0000","last_active":"2024-09-13T18:20:26.184156+0000","last_peered":"2024-09-13T18:20:26.184156+0000","last_clean":"2024-09-13T18:20:26.184156+0000","last_became_active":"2024-09-13T18:20:01.922370+0000","last_became_peered":"2024-09-13T18:20:01.922370+0000","last_unstale":"2024-09-13T18:20:26.184156+0000","last_undegraded":"2024-09-13T18:20:26.184156+0000","last_fullsized":"2024-09-13T18:20:26.184156+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:54:32.029704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"91'4","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:33.261100+0000","last_change":"2024-09-13T18:20:01.580975+0000","last_active":"2024-09-13T18:20:33.261100+0000","last_peered":"2024-09-13T18:20:33.261100+0000","last_clean":"2024-09-13T18:20:33.261100+0000","last_became_active":"2024-09-13T18:20:01.580647+0000","last_became_peered":"2024-09-13T18:20:01.580647+0000","last_unstale":"2024-09-13T18:20:33.261100+0000","last_undegraded":"2024-09-13T18:20:33.261100+0000","last_fullsized":"2024-09-13T18:20:33.261100+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:26:20.942915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.192336+0000","last_change":"2024-09-13T18:20:06.140003+0000","last_active":"2024-09-13T18:20:26.192336+0000","last_peered":"2024-09-13T18:20:26.192336+0000","last_clean":"2024-09-13T18:20:26.192336+0000","last_became_active":"2024-09-13T18:20:06.139663+0000","last_became_peered":"2024-09-13T18:20:06.139663+0000","last_unstale":"2024-09-13T18:20:26.192336+0000","last_undegraded":"2024-09-13T18:20:26.192336+0000","last_fullsized":"2024-09-13T18:20:26.192336+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:54:49.662401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.189993+0000","last_change":"2024-09-13T18:20:06.134448+0000","last_active":"2024-09-13T18:20:26.189993+0000","last_peered":"2024-09-13T18:20:26.189993+0000","last_clean":"2024-09-13T18:20:26.189993+0000","last_became_active":"2024-09-13T18:20:06.134084+0000","last_became_peered":"2024-09-13T18:20:06.134084+0000","last_unstale":"2024-09-13T18:20:26.189993+0000","last_undegraded":"2024-09-13T18:20:26.189993+0000","last_fullsized":"2024-09-13T18:20:26.189993+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:55:50.475665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183131+0000","last_change":"2024-09-13T18:20:01.918955+0000","last_active":"2024-09-13T18:20:26.183131+0000","last_peered":"2024-09-13T18:20:26.183131+0000","last_clean":"2024-09-13T18:20:26.183131+0000","last_became_active":"2024-09-13T18:20:01.918666+0000","last_became_peered":"2024-09-13T18:20:01.918666+0000","last_unstale":"2024-09-13T18:20:26.183131+0000","last_undegraded":"2024-09-13T18:20:26.183131+0000","last_fullsized":"2024-09-13T18:20:26.183131+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:08:50.697710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"91'3","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.251385+0000","last_change":"2024-09-13T18:20:01.578789+0000","last_active":"2024-09-13T18:20:28.251385+0000","last_peered":"2024-09-13T18:20:28.251385+0000","last_clean":"2024-09-13T18:20:28.251385+0000","last_became_active":"2024-09-13T18:20:01.578476+0000","last_became_peered":"2024-09-13T18:20:01.578476+0000","last_unstale":"2024-09-13T18:20:28.251385+0000","last_undegraded":"2024-09-13T18:20:28.251385+0000","last_fullsized":"2024-09-13T18:20:28.251385+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:40:58.355492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183716+0000","last_change":"2024-09-13T18:20:06.129765+0000","last_active":"2024-09-13T18:20:26.183716+0000","last_peered":"2024-09-13T18:20:26.183716+0000","last_clean":"2024-09-13T18:20:26.183716+0000","last_became_active":"2024-09-13T18:20:06.129310+0000","last_became_peered":"2024-09-13T18:20:06.129310+0000","last_unstale":"2024-09-13T18:20:26.183716+0000","last_undegraded":"2024-09-13T18:20:26.183716+0000","last_fullsized":"2024-09-13T18:20:26.183716+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:34:07.394871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.191040+0000","last_change":"2024-09-13T18:20:01.914804+0000","last_active":"2024-09-13T18:20:26.191040+0000","last_peered":"2024-09-13T18:20:26.191040+0000","last_clean":"2024-09-13T18:20:26.191040+0000","last_became_active":"2024-09-13T18:20:01.914517+0000","last_became_peered":"2024-09-13T18:20:01.914517+0000","last_unstale":"2024-09-13T18:20:26.191040+0000","last_undegraded":"2024-09-13T18:20:26.191040+0000","last_fullsized":"2024-09-13T18:20:26.191040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:57:10.734935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"91'3","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:27.300317+0000","last_change":"2024-09-13T18:20:01.569201+0000","last_active":"2024-09-13T18:20:27.300317+0000","last_peered":"2024-09-13T18:20:27.300317+0000","last_clean":"2024-09-13T18:20:27.300317+0000","last_became_active":"2024-09-13T18:20:01.568908+0000","last_became_peered":"2024-09-13T18:20:01.568908+0000","last_unstale":"2024-09-13T18:20:27.300317+0000","last_undegraded":"2024-09-13T18:20:27.300317+0000","last_fullsized":"2024-09-13T18:20:27.300317+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:16:34.069130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"91'7","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:28.205826+0000","last_change":"2024-09-13T18:20:01.579225+0000","last_active":"2024-09-13T18:20:28.205826+0000","last_peered":"2024-09-13T18:20:28.205826+0000","last_clean":"2024-09-13T18:20:28.205826+0000","last_became_active":"2024-09-13T18:20:01.578826+0000","last_became_peered":"2024-09-13T18:20:01.578826+0000","last_unstale":"2024-09-13T18:20:28.205826+0000","last_undegraded":"2024-09-13T18:20:28.205826+0000","last_fullsized":"2024-09-13T18:20:28.205826+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:48:07.632239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.180541+0000","last_change":"2024-09-13T18:20:01.914885+0000","last_active":"2024-09-13T18:20:26.180541+0000","last_peered":"2024-09-13T18:20:26.180541+0000","last_clean":"2024-09-13T18:20:26.180541+0000","last_became_active":"2024-09-13T18:20:01.914599+0000","last_became_peered":"2024-09-13T18:20:01.914599+0000","last_unstale":"2024-09-13T18:20:26.180541+0000","last_undegraded":"2024-09-13T18:20:26.180541+0000","last_fullsized":"2024-09-13T18:20:26.180541+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:41:30.888675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.186528+0000","last_change":"2024-09-13T18:20:06.129376+0000","last_active":"2024-09-13T18:20:26.186528+0000","last_peered":"2024-09-13T18:20:26.186528+0000","last_clean":"2024-09-13T18:20:26.186528+0000","last_became_active":"2024-09-13T18:20:06.129032+0000","last_became_peered":"2024-09-13T18:20:06.129032+0000","last_unstale":"2024-09-13T18:20:26.186528+0000","last_undegraded":"2024-09-13T18:20:26.186528+0000","last_fullsized":"2024-09-13T18:20:26.186528+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:21:53.511813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:29.256218+0000","last_change":"2024-09-13T18:20:01.576069+0000","last_active":"2024-09-13T18:20:29.256218+0000","last_peered":"2024-09-13T18:20:29.256218+0000","last_clean":"2024-09-13T18:20:29.256218+0000","last_became_active":"2024-09-13T18:20:01.575760+0000","last_became_peered":"2024-09-13T18:20:01.575760+0000","last_unstale":"2024-09-13T18:20:29.256218+0000","last_undegraded":"2024-09-13T18:20:29.256218+0000","last_fullsized":"2024-09-13T18:20:29.256218+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:05:47.677848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.186432+0000","last_change":"2024-09-13T18:20:01.917073+0000","last_active":"2024-09-13T18:20:26.186432+0000","last_peered":"2024-09-13T18:20:26.186432+0000","last_clean":"2024-09-13T18:20:26.186432+0000","last_became_active":"2024-09-13T18:20:01.916765+0000","last_became_peered":"2024-09-13T18:20:01.916765+0000","last_unstale":"2024-09-13T18:20:26.186432+0000","last_undegraded":"2024-09-13T18:20:26.186432+0000","last_fullsized":"2024-09-13T18:20:26.186432+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:22:07.788313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.189329+0000","last_change":"2024-09-13T18:20:06.131429+0000","last_active":"2024-09-13T18:20:26.189329+0000","last_peered":"2024-09-13T18:20:26.189329+0000","last_clean":"2024-09-13T18:20:26.189329+0000","last_became_active":"2024-09-13T18:20:06.131019+0000","last_became_peered":"2024-09-13T18:20:06.131019+0000","last_unstale":"2024-09-13T18:20:26.189329+0000","last_undegraded":"2024-09-13T18:20:26.189329+0000","last_fullsized":"2024-09-13T18:20:26.189329+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:44:37.892574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.182865+0000","last_change":"2024-09-13T18:20:01.914050+0000","last_active":"2024-09-13T18:20:26.182865+0000","last_peered":"2024-09-13T18:20:26.182865+0000","last_clean":"2024-09-13T18:20:26.182865+0000","last_became_active":"2024-09-13T18:20:01.913733+0000","last_became_peered":"2024-09-13T18:20:01.913733+0000","last_unstale":"2024-09-13T18:20:26.182865+0000","last_undegraded":"2024-09-13T18:20:26.182865+0000","last_fullsized":"2024-09-13T18:20:26.182865+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:52:43.672096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.188338+0000","last_change":"2024-09-13T18:20:01.924290+0000","last_active":"2024-09-13T18:20:26.188338+0000","last_peered":"2024-09-13T18:20:26.188338+0000","last_clean":"2024-09-13T18:20:26.188338+0000","last_became_active":"2024-09-13T18:20:01.923883+0000","last_became_peered":"2024-09-13T18:20:01.923883+0000","last_unstale":"2024-09-13T18:20:26.188338+0000","last_undegraded":"2024-09-13T18:20:26.188338+0000","last_fullsized":"2024-09-13T18:20:26.188338+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:11:18.737501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:27.184256+0000","last_change":"2024-09-13T18:20:27.184256+0000","last_active":"2024-09-13T18:20:27.184256+0000","last_peered":"2024-09-13T18:20:27.184256+0000","last_clean":"2024-09-13T18:20:27.184256+0000","last_became_active":"2024-09-13T18:20:27.183790+0000","last_became_peered":"2024-09-13T18:20:27.183790+0000","last_unstale":"2024-09-13T18:20:27.184256+0000","last_undegraded":"2024-09-13T18:20:27.184256+0000","last_fullsized":"2024-09-13T18:20:27.184256+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:06:48.885753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.189852+0000","last_change":"2024-09-13T18:20:01.915799+0000","last_active":"2024-09-13T18:20:26.189852+0000","last_peered":"2024-09-13T18:20:26.189852+0000","last_clean":"2024-09-13T18:20:26.189852+0000","last_became_active":"2024-09-13T18:20:01.915519+0000","last_became_peered":"2024-09-13T18:20:01.915519+0000","last_unstale":"2024-09-13T18:20:26.189852+0000","last_undegraded":"2024-09-13T18:20:26.189852+0000","last_fullsized":"2024-09-13T18:20:26.189852+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:08:26.065052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.178968+0000","last_change":"2024-09-13T18:20:01.918547+0000","last_active":"2024-09-13T18:20:26.178968+0000","last_peered":"2024-09-13T18:20:26.178968+0000","last_clean":"2024-09-13T18:20:26.178968+0000","last_became_active":"2024-09-13T18:20:01.918053+0000","last_became_peered":"2024-09-13T18:20:01.918053+0000","last_unstale":"2024-09-13T18:20:26.178968+0000","last_undegraded":"2024-09-13T18:20:26.178968+0000","last_fullsized":"2024-09-13T18:20:26.178968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:25:58.235101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183342+0000","last_change":"2024-09-13T18:20:01.916205+0000","last_active":"2024-09-13T18:20:26.183342+0000","last_peered":"2024-09-13T18:20:26.183342+0000","last_clean":"2024-09-13T18:20:26.183342+0000","last_became_active":"2024-09-13T18:20:01.915909+0000","last_became_peered":"2024-09-13T18:20:01.915909+0000","last_unstale":"2024-09-13T18:20:26.183342+0000","last_undegraded":"2024-09-13T18:20:26.183342+0000","last_fullsized":"2024-09-13T18:20:26.183342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:23:03.976513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.189236+0000","last_change":"2024-09-13T18:20:01.916621+0000","last_active":"2024-09-13T18:20:26.189236+0000","last_peered":"2024-09-13T18:20:26.189236+0000","last_clean":"2024-09-13T18:20:26.189236+0000","last_became_active":"2024-09-13T18:20:01.916335+0000","last_became_peered":"2024-09-13T18:20:01.916335+0000","last_unstale":"2024-09-13T18:20:26.189236+0000","last_undegraded":"2024-09-13T18:20:26.189236+0000","last_fullsized":"2024-09-13T18:20:26.189236+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:08:16.104571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.190215+0000","last_change":"2024-09-13T18:20:01.921195+0000","last_active":"2024-09-13T18:20:26.190215+0000","last_peered":"2024-09-13T18:20:26.190215+0000","last_clean":"2024-09-13T18:20:26.190215+0000","last_became_active":"2024-09-13T18:20:01.920895+0000","last_became_peered":"2024-09-13T18:20:01.920895+0000","last_unstale":"2024-09-13T18:20:26.190215+0000","last_undegraded":"2024-09-13T18:20:26.190215+0000","last_fullsized":"2024-09-13T18:20:26.190215+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:59:53.920047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.180047+0000","last_change":"2024-09-13T18:20:01.910889+0000","last_active":"2024-09-13T18:20:26.180047+0000","last_peered":"2024-09-13T18:20:26.180047+0000","last_clean":"2024-09-13T18:20:26.180047+0000","last_became_active":"2024-09-13T18:20:01.910451+0000","last_became_peered":"2024-09-13T18:20:01.910451+0000","last_unstale":"2024-09-13T18:20:26.180047+0000","last_undegraded":"2024-09-13T18:20:26.180047+0000","last_fullsized":"2024-09-13T18:20:26.180047+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:06:52.552785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.192420+0000","last_change":"2024-09-13T18:20:01.914041+0000","last_active":"2024-09-13T18:20:26.192420+0000","last_peered":"2024-09-13T18:20:26.192420+0000","last_clean":"2024-09-13T18:20:26.192420+0000","last_became_active":"2024-09-13T18:20:01.913240+0000","last_became_peered":"2024-09-13T18:20:01.913240+0000","last_unstale":"2024-09-13T18:20:26.192420+0000","last_undegraded":"2024-09-13T18:20:26.192420+0000","last_fullsized":"2024-09-13T18:20:26.192420+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:39:37.062544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.184154+0000","last_change":"2024-09-13T18:20:01.927322+0000","last_active":"2024-09-13T18:20:26.184154+0000","last_peered":"2024-09-13T18:20:26.184154+0000","last_clean":"2024-09-13T18:20:26.184154+0000","last_became_active":"2024-09-13T18:20:01.926767+0000","last_became_peered":"2024-09-13T18:20:01.926767+0000","last_unstale":"2024-09-13T18:20:26.184154+0000","last_undegraded":"2024-09-13T18:20:26.184154+0000","last_fullsized":"2024-09-13T18:20:26.184154+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:20.986089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.178873+0000","last_change":"2024-09-13T18:20:01.916314+0000","last_active":"2024-09-13T18:20:26.178873+0000","last_peered":"2024-09-13T18:20:26.178873+0000","last_clean":"2024-09-13T18:20:26.178873+0000","last_became_active":"2024-09-13T18:20:01.915790+0000","last_became_peered":"2024-09-13T18:20:01.915790+0000","last_unstale":"2024-09-13T18:20:26.178873+0000","last_undegraded":"2024-09-13T18:20:26.178873+0000","last_fullsized":"2024-09-13T18:20:26.178873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:20:30.063814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:27.187172+0000","last_change":"2024-09-13T18:20:27.187172+0000","last_active":"2024-09-13T18:20:27.187172+0000","last_peered":"2024-09-13T18:20:27.187172+0000","last_clean":"2024-09-13T18:20:27.187172+0000","last_became_active":"2024-09-13T18:20:27.186765+0000","last_became_peered":"2024-09-13T18:20:27.186765+0000","last_unstale":"2024-09-13T18:20:27.187172+0000","last_undegraded":"2024-09-13T18:20:27.187172+0000","last_fullsized":"2024-09-13T18:20:27.187172+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:37:01.469560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.189533+0000","last_change":"2024-09-13T18:20:01.924074+0000","last_active":"2024-09-13T18:20:26.189533+0000","last_peered":"2024-09-13T18:20:26.189533+0000","last_clean":"2024-09-13T18:20:26.189533+0000","last_became_active":"2024-09-13T18:20:01.922364+0000","last_became_peered":"2024-09-13T18:20:01.922364+0000","last_unstale":"2024-09-13T18:20:26.189533+0000","last_undegraded":"2024-09-13T18:20:26.189533+0000","last_fullsized":"2024-09-13T18:20:26.189533+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:51:33.824362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183604+0000","last_change":"2024-09-13T18:20:01.914606+0000","last_active":"2024-09-13T18:20:26.183604+0000","last_peered":"2024-09-13T18:20:26.183604+0000","last_clean":"2024-09-13T18:20:26.183604+0000","last_became_active":"2024-09-13T18:20:01.914281+0000","last_became_peered":"2024-09-13T18:20:01.914281+0000","last_unstale":"2024-09-13T18:20:26.183604+0000","last_undegraded":"2024-09-13T18:20:26.183604+0000","last_fullsized":"2024-09-13T18:20:26.183604+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:55:42.211959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.184054+0000","last_change":"2024-09-13T18:20:01.928876+0000","last_active":"2024-09-13T18:20:26.184054+0000","last_peered":"2024-09-13T18:20:26.184054+0000","last_clean":"2024-09-13T18:20:26.184054+0000","last_became_active":"2024-09-13T18:20:01.928534+0000","last_became_peered":"2024-09-13T18:20:01.928534+0000","last_unstale":"2024-09-13T18:20:26.184054+0000","last_undegraded":"2024-09-13T18:20:26.184054+0000","last_fullsized":"2024-09-13T18:20:26.184054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:14:28.196746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183517+0000","last_change":"2024-09-13T18:20:01.920642+0000","last_active":"2024-09-13T18:20:26.183517+0000","last_peered":"2024-09-13T18:20:26.183517+0000","last_clean":"2024-09-13T18:20:26.183517+0000","last_became_active":"2024-09-13T18:20:01.920335+0000","last_became_peered":"2024-09-13T18:20:01.920335+0000","last_unstale":"2024-09-13T18:20:26.183517+0000","last_undegraded":"2024-09-13T18:20:26.183517+0000","last_fullsized":"2024-09-13T18:20:26.183517+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:59:20.894563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T18:20:27.187529+0000","last_change":"2024-09-13T18:20:27.187529+0000","last_active":"2024-09-13T18:20:27.187529+0000","last_peered":"2024-09-13T18:20:27.187529+0000","last_clean":"2024-09-13T18:20:27.187529+0000","last_became_active":"2024-09-13T18:20:27.187123+0000","last_became_peered":"2024-09-13T18:20:27.187123+0000","last_unstale":"2024-09-13T18:20:27.187529+0000","last_undegraded":"2024-09-13T18:20:27.187529+0000","last_fullsized":"2024-09-13T18:20:27.187529+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:43:19.601235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.189462+0000","last_change":"2024-09-13T18:20:01.917111+0000","last_active":"2024-09-13T18:20:26.189462+0000","last_peered":"2024-09-13T18:20:26.189462+0000","last_clean":"2024-09-13T18:20:26.189462+0000","last_became_active":"2024-09-13T18:20:01.916826+0000","last_became_peered":"2024-09-13T18:20:01.916826+0000","last_unstale":"2024-09-13T18:20:26.189462+0000","last_undegraded":"2024-09-13T18:20:26.189462+0000","last_fullsized":"2024-09-13T18:20:26.189462+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:36:08.150028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.178921+0000","last_change":"2024-09-13T18:20:01.904496+0000","last_active":"2024-09-13T18:20:26.178921+0000","last_peered":"2024-09-13T18:20:26.178921+0000","last_clean":"2024-09-13T18:20:26.178921+0000","last_became_active":"2024-09-13T18:20:01.904199+0000","last_became_peered":"2024-09-13T18:20:01.904199+0000","last_unstale":"2024-09-13T18:20:26.178921+0000","last_undegraded":"2024-09-13T18:20:26.178921+0000","last_fullsized":"2024-09-13T18:20:26.178921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:44.044162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.186384+0000","last_change":"2024-09-13T18:20:01.919023+0000","last_active":"2024-09-13T18:20:26.186384+0000","last_peered":"2024-09-13T18:20:26.186384+0000","last_clean":"2024-09-13T18:20:26.186384+0000","last_became_active":"2024-09-13T18:20:01.918661+0000","last_became_peered":"2024-09-13T18:20:01.918661+0000","last_unstale":"2024-09-13T18:20:26.186384+0000","last_undegraded":"2024-09-13T18:20:26.186384+0000","last_fullsized":"2024-09-13T18:20:26.186384+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:25:26.779029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.188169+0000","last_change":"2024-09-13T18:20:01.922833+0000","last_active":"2024-09-13T18:20:26.188169+0000","last_peered":"2024-09-13T18:20:26.188169+0000","last_clean":"2024-09-13T18:20:26.188169+0000","last_became_active":"2024-09-13T18:20:01.922533+0000","last_became_peered":"2024-09-13T18:20:01.922533+0000","last_unstale":"2024-09-13T18:20:26.188169+0000","last_undegraded":"2024-09-13T18:20:26.188169+0000","last_fullsized":"2024-09-13T18:20:26.188169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:27:11.557419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.197907+0000","last_change":"2024-09-13T18:20:01.925916+0000","last_active":"2024-09-13T18:20:26.197907+0000","last_peered":"2024-09-13T18:20:26.197907+0000","last_clean":"2024-09-13T18:20:26.197907+0000","last_became_active":"2024-09-13T18:20:01.925568+0000","last_became_peered":"2024-09-13T18:20:01.925568+0000","last_unstale":"2024-09-13T18:20:26.197907+0000","last_undegraded":"2024-09-13T18:20:26.197907+0000","last_fullsized":"2024-09-13T18:20:26.197907+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:58:24.955273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.184513+0000","last_change":"2024-09-13T18:20:01.924328+0000","last_active":"2024-09-13T18:20:26.184513+0000","last_peered":"2024-09-13T18:20:26.184513+0000","last_clean":"2024-09-13T18:20:26.184513+0000","last_became_active":"2024-09-13T18:20:01.924004+0000","last_became_peered":"2024-09-13T18:20:01.924004+0000","last_unstale":"2024-09-13T18:20:26.184513+0000","last_undegraded":"2024-09-13T18:20:26.184513+0000","last_fullsized":"2024-09-13T18:20:26.184513+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:23:29.204157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.190973+0000","last_change":"2024-09-13T18:20:01.922234+0000","last_active":"2024-09-13T18:20:26.190973+0000","last_peered":"2024-09-13T18:20:26.190973+0000","last_clean":"2024-09-13T18:20:26.190973+0000","last_became_active":"2024-09-13T18:20:01.921196+0000","last_became_peered":"2024-09-13T18:20:01.921196+0000","last_unstale":"2024-09-13T18:20:26.190973+0000","last_undegraded":"2024-09-13T18:20:26.190973+0000","last_fullsized":"2024-09-13T18:20:26.190973+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:24:10.144346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.187949+0000","last_change":"2024-09-13T18:20:01.921740+0000","last_active":"2024-09-13T18:20:26.187949+0000","last_peered":"2024-09-13T18:20:26.187949+0000","last_clean":"2024-09-13T18:20:26.187949+0000","last_became_active":"2024-09-13T18:20:01.919634+0000","last_became_peered":"2024-09-13T18:20:01.919634+0000","last_unstale":"2024-09-13T18:20:26.187949+0000","last_undegraded":"2024-09-13T18:20:26.187949+0000","last_fullsized":"2024-09-13T18:20:26.187949+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:43:27.600637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.197426+0000","last_change":"2024-09-13T18:20:01.924806+0000","last_active":"2024-09-13T18:20:26.197426+0000","last_peered":"2024-09-13T18:20:26.197426+0000","last_clean":"2024-09-13T18:20:26.197426+0000","last_became_active":"2024-09-13T18:20:01.924514+0000","last_became_peered":"2024-09-13T18:20:01.924514+0000","last_unstale":"2024-09-13T18:20:26.197426+0000","last_undegraded":"2024-09-13T18:20:26.197426+0000","last_fullsized":"2024-09-13T18:20:26.197426+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:47.323362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.185515+0000","last_change":"2024-09-13T18:20:01.912402+0000","last_active":"2024-09-13T18:20:26.185515+0000","last_peered":"2024-09-13T18:20:26.185515+0000","last_clean":"2024-09-13T18:20:26.185515+0000","last_became_active":"2024-09-13T18:20:01.911534+0000","last_became_peered":"2024-09-13T18:20:01.911534+0000","last_unstale":"2024-09-13T18:20:26.185515+0000","last_undegraded":"2024-09-13T18:20:26.185515+0000","last_fullsized":"2024-09-13T18:20:26.185515+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:22:52.021688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.186416+0000","last_change":"2024-09-13T18:20:01.930248+0000","last_active":"2024-09-13T18:20:26.186416+0000","last_peered":"2024-09-13T18:20:26.186416+0000","last_clean":"2024-09-13T18:20:26.186416+0000","last_became_active":"2024-09-13T18:20:01.929931+0000","last_became_peered":"2024-09-13T18:20:01.929931+0000","last_unstale":"2024-09-13T18:20:26.186416+0000","last_undegraded":"2024-09-13T18:20:26.186416+0000","last_fullsized":"2024-09-13T18:20:26.186416+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:37:33.086722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.190113+0000","last_change":"2024-09-13T18:20:01.921913+0000","last_active":"2024-09-13T18:20:26.190113+0000","last_peered":"2024-09-13T18:20:26.190113+0000","last_clean":"2024-09-13T18:20:26.190113+0000","last_became_active":"2024-09-13T18:20:01.921395+0000","last_became_peered":"2024-09-13T18:20:01.921395+0000","last_unstale":"2024-09-13T18:20:26.190113+0000","last_undegraded":"2024-09-13T18:20:26.190113+0000","last_fullsized":"2024-09-13T18:20:26.190113+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:54:50.816870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.190140+0000","last_change":"2024-09-13T18:20:01.912654+0000","last_active":"2024-09-13T18:20:26.190140+0000","last_peered":"2024-09-13T18:20:26.190140+0000","last_clean":"2024-09-13T18:20:26.190140+0000","last_became_active":"2024-09-13T18:20:01.912356+0000","last_became_peered":"2024-09-13T18:20:01.912356+0000","last_unstale":"2024-09-13T18:20:26.190140+0000","last_undegraded":"2024-09-13T18:20:26.190140+0000","last_fullsized":"2024-09-13T18:20:26.190140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:25:18.099396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T18:20:26.183955+0000","last_change":"2024-09-13T18:20:01.923373+0000","last_active":"2024-09-13T18:20:26.183955+0000","last_peered":"2024-09-13T18:20:26.183955+0000","last_clean":"2024-09-13T18:20:26.183955+0000","last_became_active":"2024-09-13T18:20:01.923077+0000","last_became_peered":"2024-09-13T18:20:01.923077+0000","last_unstale":"2024-09-13T18:20:26.183955+0000","last_undegraded":"2024-09-13T18:20:26.183955+0000","last_fullsized":"2024-09-13T18:20:26.183955+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:07:52.955705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":51855,"num_objects":97,"num_object_clones":0,"num_object_copies":291,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":77,"num_read_kb":102,"num_write":220,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":65,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":512000,"data_stored":150365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":194,"ondisk_log_size":194,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281684,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383699,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":1007616,"data_stored":500603,"data_compressed":17567,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.46899999999999997}]},{"osd":1,"last update":"Fri Sep 13 18:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.623},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.60999999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.496}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":4,"last update":"Fri Sep 13 18:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.54700000000000004}]},{"osd":5,"last update":"Fri Sep 13 18:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.73399999999999999}]},{"osd":6,"last update":"Fri Sep 13 18:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.63200000000000001}]},{"osd":7,"last update":"Fri Sep 13 18:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.72299999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":9,"last update":"Fri Sep 13 18:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.80300000000000005}]},{"osd":10,"last update":"Fri Sep 13 18:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.752},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.71699999999999997}]}]},{"osd":10,"up_from":72,"seq":309237645336,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1130496,"data_stored":1132691,"data_compressed":53391,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":1,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.44500000000000001}]},{"osd":2,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.77200000000000002}]},{"osd":3,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.60599999999999998}]},{"osd":4,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.55600000000000005}]},{"osd":5,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.81599999999999995}]},{"osd":6,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.68500000000000005}]},{"osd":7,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.81599999999999995}]},{"osd":8,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.752},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.63}]},{"osd":9,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.69999999999999996}]},{"osd":11,"last update":"Fri Sep 13 18:20:12 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.80600000000000005}]}]},{"osd":9,"up_from":67,"seq":287762808860,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":500559,"data_compressed":17567,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.311}]},{"osd":1,"last update":"Fri Sep 13 18:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.5}]},{"osd":2,"last update":"Fri Sep 13 18:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.53900000000000003}]},{"osd":3,"last update":"Fri Sep 13 18:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.34300000000000003}]},{"osd":4,"last update":"Fri Sep 13 18:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.57599999999999996}]},{"osd":5,"last update":"Fri Sep 13 18:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.55400000000000005}]},{"osd":6,"last update":"Fri Sep 13 18:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.39800000000000002}]},{"osd":7,"last update":"Fri Sep 13 18:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.47999999999999998}]},{"osd":8,"last update":"Fri Sep 13 18:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.624}]},{"osd":10,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.63900000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]}]},{"osd":8,"up_from":59,"seq":253403070497,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":1019904,"data_stored":500915,"data_compressed":17567,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.53600000000000003,"15min":0.54800000000000004},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.60199999999999998,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.54900000000000004,"15min":0.56200000000000006},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.68600000000000005,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.44700000000000001}]},{"osd":1,"last update":"Fri Sep 13 18:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56000000000000005,"15min":0.55900000000000005},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56999999999999995,"15min":0.57099999999999995},"min":{"1min":0.40600000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72199999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.52300000000000002}]},{"osd":2,"last update":"Fri Sep 13 18:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.42599999999999999,"15min":0.42999999999999999},"min":{"1min":0.26600000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.55200000000000005,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.247,"5min":0.224,"15min":0.224},"max":{"1min":0.59799999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.39300000000000002}]},{"osd":3,"last update":"Fri Sep 13 18:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.54700000000000004,"15min":0.55800000000000005},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.71999999999999997,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53800000000000003,"15min":0.53900000000000003},"min":{"1min":0.38600000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.60799999999999998}]},{"osd":4,"last update":"Fri Sep 13 18:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.66400000000000003,"15min":0.63600000000000001},"min":{"1min":0.45100000000000001,"5min":0.37,"15min":0.37},"max":{"1min":5.3540000000000001,"5min":5.3540000000000001,"15min":5.3540000000000001},"last":0.68000000000000005}]},{"osd":5,"last update":"Fri Sep 13 18:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.53100000000000003,"15min":0.53700000000000003},"min":{"1min":0.31900000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.70099999999999996,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.51900000000000002,"15min":0.52300000000000002},"min":{"1min":0.32100000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.63500000000000001,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.58799999999999997}]},{"osd":6,"last update":"Fri Sep 13 18:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.60599999999999998,"15min":0.61199999999999999},"min":{"1min":0.35199999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.75600000000000001,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.64400000000000002,"15min":0.65000000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.77200000000000002,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.628}]},{"osd":7,"last update":"Fri Sep 13 18:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.66600000000000004,"15min":0.67500000000000004},"min":{"1min":0.44800000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.749,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.65600000000000003,"15min":0.66300000000000003},"min":{"1min":0.39700000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.73899999999999999,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.67200000000000004}]},{"osd":9,"last update":"Fri Sep 13 18:20:24 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63800000000000001,"15min":0.63300000000000001},"min":{"1min":0.46600000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.65900000000000003,"15min":0.65600000000000003},"min":{"1min":0.52800000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.95399999999999996,"5min":1.012,"15min":1.012},"last":0.68999999999999995}]},{"osd":10,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":5.3360000000000003,"5min":5.3360000000000003,"15min":5.3360000000000003},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.72299999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]}]},{"osd":7,"up_from":51,"seq":219043332133,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1077248,"data_stored":1096899,"data_compressed":53391,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.94399999999999995,"15min":1.012},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.84299999999999997,"5min":12.068,"15min":12.068},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.46999999999999997,"15min":0.45900000000000002},"min":{"1min":0.35199999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.61599999999999999}]},{"osd":1,"last update":"Fri Sep 13 18:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.36499999999999999,"15min":0.35699999999999998},"min":{"1min":0.26100000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.38100000000000001,"15min":0.375},"min":{"1min":0.27800000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.62,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.54200000000000004}]},{"osd":2,"last update":"Fri Sep 13 18:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.59099999999999997,"15min":0.53700000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":7.6820000000000004,"5min":7.6820000000000004,"15min":7.6820000000000004},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.51600000000000001,"15min":0.50700000000000001},"min":{"1min":0.378,"5min":0.371,"15min":0.371},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.59999999999999998}]},{"osd":3,"last update":"Fri Sep 13 18:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.49299999999999999,"15min":0.48099999999999998},"min":{"1min":0.36299999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.49299999999999999,"15min":0.48299999999999998},"min":{"1min":0.35399999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.61099999999999999}]},{"osd":4,"last update":"Fri Sep 13 18:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.44,"15min":0.433},"min":{"1min":0.29099999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.621,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.45400000000000001,"15min":0.45000000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.58999999999999997}]},{"osd":5,"last update":"Fri Sep 13 18:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.55700000000000005,"15min":0.55000000000000004},"min":{"1min":0.39900000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.53700000000000003,"15min":0.53100000000000003},"min":{"1min":0.39900000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.68200000000000005}]},{"osd":6,"last update":"Fri Sep 13 18:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.55000000000000004,"15min":0.54000000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.59299999999999997,"15min":0.58199999999999996},"min":{"1min":0.48099999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.74299999999999999}]},{"osd":8,"last update":"Fri Sep 13 18:20:01 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.58999999999999997,"15min":0.58699999999999997},"min":{"1min":0.42499999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.72999999999999998,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.89400000000000002},{"interface":"front","average":{"1min":1.0669999999999999,"5min":0.71899999999999997,"15min":0.66100000000000003},"min":{"1min":0.443,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":7.7149999999999999,"5min":7.7149999999999999,"15min":7.7149999999999999},"last":0.78100000000000003}]},{"osd":9,"last update":"Fri Sep 13 18:20:26 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.80400000000000005,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63600000000000001,"15min":0.63700000000000001},"min":{"1min":0.48599999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.84599999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.73199999999999998}]},{"osd":10,"last update":"Fri Sep 13 18:19:50 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.76800000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]}]},{"osd":6,"up_from":46,"seq":197568495659,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":1003520,"data_stored":496015,"data_compressed":17567,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.248,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.315},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.43099999999999999,"15min":0.437},"min":{"1min":0.27600000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.36199999999999999}]},{"osd":1,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.52700000000000002,"15min":0.52200000000000002},"min":{"1min":0.35499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.69199999999999995,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55000000000000004,"15min":0.53600000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.54700000000000004}]},{"osd":2,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.54200000000000004,"15min":0.54600000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53600000000000003,"15min":0.54700000000000004},"min":{"1min":0.36599999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.75,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.57999999999999996}]},{"osd":3,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42299999999999999,"15min":0.42599999999999999},"min":{"1min":0.28999999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.69499999999999995,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.46700000000000003,"15min":0.47099999999999997},"min":{"1min":0.30099999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.38400000000000001}]},{"osd":4,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.52300000000000002,"15min":0.51500000000000001},"min":{"1min":0.33300000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55500000000000005,"15min":0.55300000000000005},"min":{"1min":0.39600000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.74199999999999999,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.53100000000000003}]},{"osd":5,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59899999999999998,"15min":0.60799999999999998},"min":{"1min":0.436,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.57399999999999995,"15min":0.59299999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.77900000000000003,"5min":0.874,"15min":0.874},"last":0.67300000000000004}]},{"osd":7,"last update":"Fri Sep 13 18:19:43 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57399999999999995,"15min":0.57299999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.71299999999999997,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.628},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56599999999999995,"15min":0.56200000000000006},"min":{"1min":0.435,"5min":0.434,"15min":0.434},"max":{"1min":0.77100000000000002,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.55600000000000005}]},{"osd":8,"last update":"Fri Sep 13 18:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.61799999999999999,"15min":0.621},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.79000000000000004,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.59399999999999997,"15min":0.59899999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.82999999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.51000000000000001}]},{"osd":9,"last update":"Fri Sep 13 18:20:30 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.53100000000000003,"15min":0.53600000000000003},"min":{"1min":0.33600000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.85899999999999999,"5min":0.871,"15min":0.871},"last":0.375},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.60299999999999998,"15min":0.60899999999999999},"min":{"1min":0.39500000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.48099999999999998}]},{"osd":10,"last update":"Fri Sep 13 18:19:49 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.64000000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]}]},{"osd":5,"up_from":38,"seq":163208757295,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1085440,"data_stored":1092577,"data_compressed":53391,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:19:58 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.49299999999999999,"15min":0.49399999999999999},"min":{"1min":0.35299999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.61099999999999999,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.51400000000000001,"15min":0.51200000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.66600000000000004,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.52800000000000002}]},{"osd":1,"last update":"Fri Sep 13 18:19:58 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.51600000000000001,"15min":0.502},"min":{"1min":0.41899999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.77200000000000002,"5min":0.878,"15min":0.878},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.52000000000000002,"15min":0.51000000000000001},"min":{"1min":0.34399999999999997,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.78600000000000003,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.51100000000000001}]},{"osd":2,"last update":"Fri Sep 13 18:19:58 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.376,"15min":0.36199999999999999},"min":{"1min":0.27700000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.39000000000000001,"15min":0.38400000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.58499999999999996,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.33000000000000002}]},{"osd":3,"last update":"Fri Sep 13 18:19:58 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.51600000000000001,"15min":0.501},"min":{"1min":0.35799999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.51500000000000001,"15min":0.51800000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.70499999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.54000000000000004}]},{"osd":4,"last update":"Fri Sep 13 18:19:58 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.54700000000000004,"15min":0.53800000000000003},"min":{"1min":0.40400000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.75900000000000001,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.76400000000000001,"15min":0.60799999999999998},"min":{"1min":0.376,"5min":0.374,"15min":0.374},"max":{"1min":0.80200000000000005,"5min":18.571000000000002,"15min":18.571000000000002},"last":0.60699999999999998}]},{"osd":6,"last update":"Fri Sep 13 18:20:22 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.42399999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55400000000000005,"15min":0.55000000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.749}]},{"osd":7,"last update":"Fri Sep 13 18:19:43 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.61499999999999999,"15min":0.60799999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.61699999999999999,"15min":0.61099999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.85299999999999998,"5min":0.88,"15min":0.88},"last":0.81899999999999995}]},{"osd":8,"last update":"Fri Sep 13 18:20:02 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.51700000000000002,"15min":0.51300000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.73699999999999999,"5min":0.753,"15min":0.753},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.54300000000000004,"15min":0.53700000000000003},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.48299999999999998}]},{"osd":9,"last update":"Fri Sep 13 18:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.63200000000000001,"15min":0.622},"min":{"1min":0.55000000000000004,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.63},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.63300000000000001,"15min":0.626},"min":{"1min":0.53500000000000003,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.78800000000000003}]},{"osd":10,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.86199999999999999}]}]},{"osd":4,"up_from":33,"seq":141733920819,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":500089,"data_compressed":17567,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:19:33 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.48299999999999998,"15min":0.47699999999999998},"min":{"1min":0.35399999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.69399999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.433},{"interface":"front","average":{"1min":0.501,"5min":0.51200000000000001,"15min":0.47499999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.84599999999999997,"5min":4.2539999999999996,"15min":4.2539999999999996},"last":0.51900000000000002}]},{"osd":1,"last update":"Fri Sep 13 18:19:33 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.34899999999999998,"15min":0.34399999999999997},"min":{"1min":0.27600000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.372,"5min":0.34699999999999998,"15min":0.34300000000000003},"min":{"1min":0.25600000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.52700000000000002,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.42199999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:19:33 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.53100000000000003,"15min":0.52300000000000002},"min":{"1min":0.40999999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.53700000000000003,"15min":0.53300000000000003},"min":{"1min":0.38500000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.60599999999999998}]},{"osd":3,"last update":"Fri Sep 13 18:19:33 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.49299999999999999},"min":{"1min":0.35199999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.72599999999999998,"5min":4.2679999999999998,"15min":4.2679999999999998},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.48899999999999999,"15min":0.48499999999999999},"min":{"1min":0.40200000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.50600000000000001}]},{"osd":5,"last update":"Fri Sep 13 18:19:55 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.54800000000000004,"15min":0.53500000000000003},"min":{"1min":0.38500000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78100000000000003,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.55000000000000004,"15min":0.53700000000000003},"min":{"1min":0.39900000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.71099999999999997,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.70999999999999996}]},{"osd":6,"last update":"Fri Sep 13 18:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.52300000000000002,"15min":0.51100000000000001},"min":{"1min":0.41999999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.52300000000000002,"15min":0.50900000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.627}]},{"osd":7,"last update":"Fri Sep 13 18:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.44400000000000001,"15min":0.438},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.442},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.48499999999999999,"15min":0.47699999999999998},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.752}]},{"osd":8,"last update":"Fri Sep 13 18:20:07 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.63,"15min":0.626},"min":{"1min":0.505,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.85899999999999999,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.65000000000000002,"15min":0.64500000000000002},"min":{"1min":0.52300000000000002,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.78200000000000003}]},{"osd":9,"last update":"Fri Sep 13 18:20:26 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.61599999999999999,"15min":0.60999999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.629,"15min":0.627},"min":{"1min":0.46400000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.64000000000000001}]},{"osd":10,"last update":"Fri Sep 13 18:19:51 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.73699999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]}]},{"osd":3,"up_from":25,"seq":107374182456,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":495490,"data_compressed":17567,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:20:10 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.32900000000000001,"15min":0.307},"min":{"1min":0.248,"5min":0.183,"15min":0.183},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.33900000000000002,"15min":0.32700000000000001},"min":{"1min":0.222,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.47599999999999998,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.28199999999999997}]},{"osd":1,"last update":"Fri Sep 13 18:20:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.50900000000000001,"15min":0.48799999999999999},"min":{"1min":0.40000000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.65600000000000003,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.51400000000000001,"15min":0.48799999999999999},"min":{"1min":0.39400000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.71299999999999997,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.58899999999999997}]},{"osd":2,"last update":"Fri Sep 13 18:20:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52400000000000002,"15min":0.52200000000000002},"min":{"1min":0.35999999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.68700000000000006,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52300000000000002,"15min":0.501},"min":{"1min":0.36599999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.73399999999999999,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.56799999999999995}]},{"osd":4,"last update":"Fri Sep 13 18:19:32 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.5,"15min":0.48599999999999999},"min":{"1min":0.374,"5min":0.312,"15min":0.312},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.52400000000000002,"15min":0.51400000000000001},"min":{"1min":0.372,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.749,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.66400000000000003}]},{"osd":5,"last update":"Fri Sep 13 18:19:54 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54000000000000004,"15min":0.53500000000000003},"min":{"1min":0.38300000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.749,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.39600000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72899999999999998,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.61299999999999999}]},{"osd":6,"last update":"Fri Sep 13 18:20:23 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.443,"15min":0.44700000000000001},"min":{"1min":0.30299999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.59199999999999997,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.48099999999999998,"15min":0.46700000000000003},"min":{"1min":0.313,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.70699999999999996,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.439}]},{"osd":7,"last update":"Fri Sep 13 18:19:39 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.80400000000000005,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.623},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.57999999999999996,"15min":0.57399999999999995},"min":{"1min":0.44800000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.77500000000000002,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.65400000000000003}]},{"osd":8,"last update":"Fri Sep 13 18:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.69599999999999995}]},{"osd":9,"last update":"Fri Sep 13 18:20:30 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.54800000000000004,"15min":0.55600000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.67200000000000004,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.58399999999999996,"15min":0.58899999999999997},"min":{"1min":0.36299999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.74399999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.49399999999999999}]},{"osd":10,"last update":"Fri Sep 13 18:19:49 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.67400000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":537530,"data_compressed":17567,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:19:45 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47399999999999998,"15min":0.47899999999999998},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.57599999999999996,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.48099999999999998,"15min":0.47599999999999998},"min":{"1min":0.318,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.41499999999999998}]},{"osd":1,"last update":"Fri Sep 13 18:19:45 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.49199999999999999,"15min":0.47199999999999998},"min":{"1min":0.34999999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.78400000000000003,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.49099999999999999,"15min":0.46500000000000002},"min":{"1min":0.32300000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.78100000000000003,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.83499999999999996}]},{"osd":3,"last update":"Fri Sep 13 18:20:13 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.52700000000000002,"15min":0.51600000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.66100000000000003,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.55200000000000005,"15min":0.55100000000000005},"min":{"1min":0.377,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.67100000000000004,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.59199999999999997}]},{"osd":4,"last update":"Fri Sep 13 18:19:35 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.504,"15min":0.49099999999999999},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.72899999999999998,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.50900000000000001,"15min":0.503},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.68899999999999995,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.84799999999999998}]},{"osd":5,"last update":"Fri Sep 13 18:19:52 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.38400000000000001,"15min":0.375},"min":{"1min":0.30399999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.434},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.40000000000000002,"15min":0.39000000000000001},"min":{"1min":0.26900000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.58899999999999997,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.53400000000000003}]},{"osd":6,"last update":"Fri Sep 13 18:20:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.56299999999999994,"15min":0.54000000000000004},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.57799999999999996,"15min":0.56299999999999994},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.80100000000000005,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.73599999999999999}]},{"osd":7,"last update":"Fri Sep 13 18:19:37 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78200000000000003,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.61599999999999999,"15min":0.61899999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.76600000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.76700000000000002}]},{"osd":8,"last update":"Fri Sep 13 18:20:07 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.54000000000000004,"15min":0.54200000000000004},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.73899999999999999,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.629},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.36599999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.61399999999999999}]},{"osd":9,"last update":"Fri Sep 13 18:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.624,"15min":0.621},"min":{"1min":0.439,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.78100000000000003,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.65100000000000002,"15min":0.64900000000000002},"min":{"1min":0.45500000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.94399999999999995,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.68500000000000005}]},{"osd":10,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.78700000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574914,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":1060864,"data_stored":542782,"data_compressed":17567,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:20:24 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.502,"15min":0.495},"min":{"1min":0.35499999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.442},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.52300000000000002,"15min":0.51500000000000001},"min":{"1min":0.33200000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.77800000000000002,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.47299999999999998}]},{"osd":2,"last update":"Fri Sep 13 18:19:46 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.501,"15min":0.48999999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.46999999999999997,"15min":0.46899999999999997},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.64200000000000002,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.51300000000000001}]},{"osd":3,"last update":"Fri Sep 13 18:20:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.53900000000000003,"15min":0.52100000000000002},"min":{"1min":0.38600000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.74399999999999999,"5min":0.749,"15min":0.749},"last":0.31},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56499999999999995,"15min":0.54400000000000004},"min":{"1min":0.39700000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76900000000000002,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.52600000000000002}]},{"osd":4,"last update":"Fri Sep 13 18:19:32 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.38,"15min":0.36099999999999999},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.434,"5min":0.376,"15min":0.35599999999999998},"min":{"1min":0.28399999999999997,"5min":0.217,"15min":0.217},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.36699999999999999}]},{"osd":5,"last update":"Fri Sep 13 18:19:53 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56699999999999995,"15min":0.56200000000000006},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55100000000000005,"15min":0.53700000000000003},"min":{"1min":0.40500000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.63500000000000001}]},{"osd":6,"last update":"Fri Sep 13 18:20:23 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57699999999999996,"15min":0.56699999999999995},"min":{"1min":0.41299999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.621,"15min":0.61099999999999999},"min":{"1min":0.45100000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.88300000000000001,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.60099999999999998}]},{"osd":7,"last update":"Fri Sep 13 18:19:38 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.49299999999999999,"15min":0.48699999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.502,"15min":0.5},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.56200000000000006}]},{"osd":8,"last update":"Fri Sep 13 18:20:07 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.627,"15min":0.629},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.68899999999999995}]},{"osd":9,"last update":"Fri Sep 13 18:20:28 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.63700000000000001,"15min":0.64300000000000002},"min":{"1min":0.432,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83199999999999996,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.63800000000000001,"15min":0.64200000000000002},"min":{"1min":0.46500000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.61599999999999999}]},{"osd":10,"last update":"Fri Sep 13 18:19:49 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.66000000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738438,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":1028096,"data_stored":497471,"data_compressed":17567,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 18:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.501,"15min":0.496},"min":{"1min":0.35599999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.50700000000000001,"15min":0.505},"min":{"1min":0.35699999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.69199999999999995,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.54400000000000004}]},{"osd":2,"last update":"Fri Sep 13 18:19:47 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.52900000000000003,"15min":0.52700000000000002},"min":{"1min":0.34399999999999997,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.70099999999999996,"5min":0.754,"15min":0.754},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.53000000000000003,"15min":0.51100000000000001},"min":{"1min":0.38500000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.81799999999999995,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.67100000000000004}]},{"osd":3,"last update":"Fri Sep 13 18:19:05 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.38400000000000001,"15min":0.378},"min":{"1min":0.26300000000000001,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.437,"5min":0.39400000000000002,"15min":0.376},"min":{"1min":0.28100000000000003,"5min":0.251,"15min":0.251},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.39300000000000002}]},{"osd":4,"last update":"Fri Sep 13 18:19:33 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.503,"15min":0.49099999999999999},"min":{"1min":0.38600000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.85699999999999998,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.51900000000000002,"15min":0.50700000000000001},"min":{"1min":0.35499999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.502}]},{"osd":5,"last update":"Fri Sep 13 18:19:53 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.61499999999999999,"15min":0.56699999999999995},"min":{"1min":0.371,"5min":0.313,"15min":0.313},"max":{"1min":0.89600000000000002,"5min":7.194,"15min":7.194},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.55100000000000005,"15min":0.52300000000000002},"min":{"1min":0.378,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.87,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.69899999999999995}]},{"osd":6,"last update":"Fri Sep 13 18:20:21 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.46400000000000002,"15min":0.45400000000000001},"min":{"1min":0.28899999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.79000000000000004,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.45800000000000002,"15min":0.46100000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.56699999999999995,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.37}]},{"osd":7,"last update":"Fri Sep 13 18:19:38 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.58599999999999997,"15min":0.57599999999999996},"min":{"1min":0.435,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.55500000000000005,"15min":0.54600000000000004},"min":{"1min":0.47299999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.53700000000000003}]},{"osd":8,"last update":"Fri Sep 13 18:20:01 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.60799999999999998,"15min":0.60999999999999999},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61899999999999999,"15min":0.61699999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.64000000000000001}]},{"osd":9,"last update":"Fri Sep 13 18:20:29 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.59899999999999998,"15min":0.61099999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.91100000000000003,"5min":1.089,"15min":1.089},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.56599999999999995,"15min":0.58099999999999996},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.70899999999999996,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.48699999999999999}]},{"osd":10,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.625}]},{"osd":11,"last update":"Fri Sep 13 18:20:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.68200000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34188,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34188,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34188,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35824,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35824,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35824,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35824,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35824,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":47441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":42189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":47313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:20:35.421 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-09-13T18:20:35.422 INFO:tasks.cephadm:Setting up client nodes... 2024-09-13T18:20:35.422 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-13T18:20:35.423 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:34 smithi134 ceph-mon[34283]: pgmap v295: 129 pgs: 129 active+clean; 627 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 30 op/s; 2 B/s, 0 objects/s recovering 2024-09-13T18:20:35.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:34 smithi204 ceph-mon[34777]: pgmap v295: 129 pgs: 129 active+clean; 627 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 30 op/s; 2 B/s, 0 objects/s recovering 2024-09-13T18:20:35.620 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:20:37.093 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:36 smithi114 ceph-mon[30897]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T18:20:37.093 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:36 smithi114 ceph-mon[30897]: pgmap v296: 129 pgs: 129 active+clean; 627 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 25 op/s; 2 B/s, 0 objects/s recovering 2024-09-13T18:20:37.249 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:36 smithi204 ceph-mon[34777]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T18:20:37.250 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:36 smithi204 ceph-mon[34777]: pgmap v296: 129 pgs: 129 active+clean; 627 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 25 op/s; 2 B/s, 0 objects/s recovering 2024-09-13T18:20:37.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:36 smithi134 ceph-mon[34283]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T18:20:37.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:36 smithi134 ceph-mon[34283]: pgmap v296: 129 pgs: 129 active+clean; 627 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 25 op/s; 2 B/s, 0 objects/s recovering 2024-09-13T18:20:37.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.358+0000 7f0be1fa4640 1 -- 172.21.15.114:0/3176908759 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0bdc100f60 msgr2=0x7f0bdc1013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:37.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.358+0000 7f0be1fa4640 1 --2- 172.21.15.114:0/3176908759 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0bdc100f60 0x7f0bdc1013e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0009900 tx=0x7f0bd002a3f0 comp rx=0 tx=0).stop 2024-09-13T18:20:37.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.358+0000 7f0be1fa4640 1 -- 172.21.15.114:0/3176908759 shutdown_connections 2024-09-13T18:20:37.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.358+0000 7f0be1fa4640 1 --2- 172.21.15.114:0/3176908759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bdc101920 0x7f0bdc108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:37.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.358+0000 7f0be1fa4640 1 --2- 172.21.15.114:0/3176908759 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0bdc100f60 0x7f0bdc1013e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:37.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.358+0000 7f0be1fa4640 1 --2- 172.21.15.114:0/3176908759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0bdc0ffe00 0x7f0bdc100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:37.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.358+0000 7f0be1fa4640 1 -- 172.21.15.114:0/3176908759 >> 172.21.15.114:0/3176908759 conn(0x7f0bdc0fb590 msgr2=0x7f0bdc0fd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:37.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.358+0000 7f0be1fa4640 1 -- 172.21.15.114:0/3176908759 shutdown_connections 2024-09-13T18:20:37.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.358+0000 7f0be1fa4640 1 -- 172.21.15.114:0/3176908759 wait complete. 2024-09-13T18:20:37.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.359+0000 7f0be1fa4640 1 Processor -- start 2024-09-13T18:20:37.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.359+0000 7f0be1fa4640 1 -- start start 2024-09-13T18:20:37.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.359+0000 7f0be1fa4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bdc0ffe00 0x7f0bdc074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:37.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.359+0000 7f0be1fa4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0bdc100f60 0x7f0bdc0728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:37.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.359+0000 7f0be1fa4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0bdc101920 0x7f0bdc072de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:37.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.359+0000 7f0be1fa4640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bdc073570 con 0x7f0bdc100f60 2024-09-13T18:20:37.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.359+0000 7f0be1fa4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bdc0736e0 con 0x7f0bdc101920 2024-09-13T18:20:37.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.359+0000 7f0be1fa4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bdc19fe50 con 0x7f0bdc0ffe00 2024-09-13T18:20:37.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.360+0000 7f0bdaffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0bdc100f60 0x7f0bdc0728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:37.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.360+0000 7f0bdaffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0bdc100f60 0x7f0bdc0728a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:54734/0 (socket says 172.21.15.114:54734) 2024-09-13T18:20:37.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.360+0000 7f0bdaffd640 1 -- 172.21.15.114:0/347038081 learned_addr learned my addr 172.21.15.114:0/347038081 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:20:37.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.360+0000 7f0bdb7fe640 1 --2- 172.21.15.114:0/347038081 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bdc0ffe00 0x7f0bdc074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:37.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.360+0000 7f0bdbfff640 1 --2- 172.21.15.114:0/347038081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0bdc101920 0x7f0bdc072de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:37.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.360+0000 7f0bdaffd640 1 -- 172.21.15.114:0/347038081 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bdc0ffe00 msgr2=0x7f0bdc074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:37.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.360+0000 7f0bdaffd640 1 --2- 172.21.15.114:0/347038081 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bdc0ffe00 0x7f0bdc074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:37.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.360+0000 7f0bdaffd640 1 -- 172.21.15.114:0/347038081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0bdc101920 msgr2=0x7f0bdc072de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:37.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.360+0000 7f0bdaffd640 1 --2- 172.21.15.114:0/347038081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0bdc101920 0x7f0bdc072de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:37.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.360+0000 7f0bdaffd640 1 -- 172.21.15.114:0/347038081 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bd0009560 con 0x7f0bdc100f60 2024-09-13T18:20:37.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.360+0000 7f0bdbfff640 1 --2- 172.21.15.114:0/347038081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0bdc101920 0x7f0bdc072de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:20:37.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.360+0000 7f0bdaffd640 1 --2- 172.21.15.114:0/347038081 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0bdc100f60 0x7f0bdc0728a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0bd00048f0 tx=0x7f0bd0004920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:37.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.360+0000 7f0bd8ff9640 1 -- 172.21.15.114:0/347038081 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd00035f0 con 0x7f0bdc100f60 2024-09-13T18:20:37.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.360+0000 7f0bd8ff9640 1 -- 172.21.15.114:0/347038081 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0bd0003be0 con 0x7f0bdc100f60 2024-09-13T18:20:37.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.360+0000 7f0be1fa4640 1 -- 172.21.15.114:0/347038081 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bdc1a00a0 con 0x7f0bdc100f60 2024-09-13T18:20:37.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.360+0000 7f0bd8ff9640 1 -- 172.21.15.114:0/347038081 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd003cac0 con 0x7f0bdc100f60 2024-09-13T18:20:37.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.360+0000 7f0be1fa4640 1 -- 172.21.15.114:0/347038081 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bdc1a05c0 con 0x7f0bdc100f60 2024-09-13T18:20:37.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.361+0000 7f0be1fa4640 1 -- 172.21.15.114:0/347038081 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bdc107330 con 0x7f0bdc100f60 2024-09-13T18:20:37.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.362+0000 7f0bd8ff9640 1 -- 172.21.15.114:0/347038081 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f0bd0003750 con 0x7f0bdc100f60 2024-09-13T18:20:37.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.363+0000 7f0bd8ff9640 1 --2- 172.21.15.114:0/347038081 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f0bc40a1110 0x7f0bc40a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:37.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.363+0000 7f0bdb7fe640 1 --2- 172.21.15.114:0/347038081 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f0bc40a1110 0x7f0bc40a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:37.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.363+0000 7f0bd8ff9640 1 -- 172.21.15.114:0/347038081 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0bd00ee060 con 0x7f0bdc100f60 2024-09-13T18:20:37.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.364+0000 7f0bdb7fe640 1 --2- 172.21.15.114:0/347038081 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f0bc40a1110 0x7f0bc40a35d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc009990 tx=0x7f0bcc009110 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:37.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.367+0000 7f0bd8ff9640 1 -- 172.21.15.114:0/347038081 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f0bd00b7320 con 0x7f0bdc100f60 2024-09-13T18:20:37.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.590+0000 7f0be1fa4640 1 -- 172.21.15.114:0/347038081 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f0bdc10b6d0 con 0x7f0bdc100f60 2024-09-13T18:20:37.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.595+0000 7f0bd8ff9640 1 -- 172.21.15.114:0/347038081 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) v1 ==== 170+0+59 (secure 0 0 0) 0x7f0bd00ba940 con 0x7f0bdc100f60 2024-09-13T18:20:37.598 INFO:teuthology.orchestra.run.smithi114.stdout:[client.0] 2024-09-13T18:20:37.598 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQB1guRmYXxVIxAA4ltKGHzUWYfmgbQtnj+Q2Q== 2024-09-13T18:20:37.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.597+0000 7f0be1fa4640 1 -- 172.21.15.114:0/347038081 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f0bc40a1110 msgr2=0x7f0bc40a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:37.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.597+0000 7f0be1fa4640 1 --2- 172.21.15.114:0/347038081 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f0bc40a1110 0x7f0bc40a35d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc009990 tx=0x7f0bcc009110 comp rx=0 tx=0).stop 2024-09-13T18:20:37.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.597+0000 7f0be1fa4640 1 -- 172.21.15.114:0/347038081 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0bdc100f60 msgr2=0x7f0bdc0728a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:37.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.597+0000 7f0be1fa4640 1 --2- 172.21.15.114:0/347038081 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0bdc100f60 0x7f0bdc0728a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0bd00048f0 tx=0x7f0bd0004920 comp rx=0 tx=0).stop 2024-09-13T18:20:37.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.597+0000 7f0be1fa4640 1 -- 172.21.15.114:0/347038081 shutdown_connections 2024-09-13T18:20:37.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.597+0000 7f0be1fa4640 1 --2- 172.21.15.114:0/347038081 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f0bc40a1110 0x7f0bc40a35d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:37.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.597+0000 7f0be1fa4640 1 --2- 172.21.15.114:0/347038081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0bdc101920 0x7f0bdc072de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:37.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.597+0000 7f0be1fa4640 1 --2- 172.21.15.114:0/347038081 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0bdc100f60 0x7f0bdc0728a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:37.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.597+0000 7f0be1fa4640 1 --2- 172.21.15.114:0/347038081 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bdc0ffe00 0x7f0bdc074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:37.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.597+0000 7f0be1fa4640 1 -- 172.21.15.114:0/347038081 >> 172.21.15.114:0/347038081 conn(0x7f0bdc0fb590 msgr2=0x7f0bdc104500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:37.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.597+0000 7f0be1fa4640 1 -- 172.21.15.114:0/347038081 shutdown_connections 2024-09-13T18:20:37.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:37.597+0000 7f0be1fa4640 1 -- 172.21.15.114:0/347038081 wait complete. 2024-09-13T18:20:38.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:37 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/347038081' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-13T18:20:38.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:37 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/347038081' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-13T18:20:38.415 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:37 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/347038081' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-13T18:20:38.415 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:37 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/347038081' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-13T18:20:38.436 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-13T18:20:38.436 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-13T18:20:38.436 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-13T18:20:38.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:37 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/347038081' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-13T18:20:38.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:37 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/347038081' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-13T18:20:38.476 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-13T18:20:38.476 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-13T18:20:38.476 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph mgr dump --format=json 2024-09-13T18:20:38.714 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:20:39.169 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:38 smithi114 ceph-mon[30897]: pgmap v297: 129 pgs: 129 active+clean; 627 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.0 KiB/s wr, 23 op/s; 2 B/s, 0 objects/s recovering 2024-09-13T18:20:39.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:38 smithi134 ceph-mon[34283]: pgmap v297: 129 pgs: 129 active+clean; 627 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.0 KiB/s wr, 23 op/s; 2 B/s, 0 objects/s recovering 2024-09-13T18:20:39.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:38 smithi204 ceph-mon[34777]: pgmap v297: 129 pgs: 129 active+clean; 627 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.0 KiB/s wr, 23 op/s; 2 B/s, 0 objects/s recovering 2024-09-13T18:20:40.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.470+0000 7fb04bed0640 1 -- 172.21.15.114:0/3904900246 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb044105110 msgr2=0x7fb04410cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:40.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.470+0000 7fb04bed0640 1 --2- 172.21.15.114:0/3904900246 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb044105110 0x7fb04410cc90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb03c009cd0 tx=0x7fb03c02a8e0 comp rx=0 tx=0).stop 2024-09-13T18:20:40.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.470+0000 7fb04bed0640 1 -- 172.21.15.114:0/3904900246 shutdown_connections 2024-09-13T18:20:40.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.470+0000 7fb04bed0640 1 --2- 172.21.15.114:0/3904900246 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb044105110 0x7fb04410cc90 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:40.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.470+0000 7fb04bed0640 1 --2- 172.21.15.114:0/3904900246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb044100990 0x7fb0441049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:40.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.470+0000 7fb04bed0640 1 --2- 172.21.15.114:0/3904900246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0440fffe0 0x7fb0441003c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:40.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.470+0000 7fb04bed0640 1 -- 172.21.15.114:0/3904900246 >> 172.21.15.114:0/3904900246 conn(0x7fb0440fbae0 msgr2=0x7fb0440fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:40.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.470+0000 7fb04bed0640 1 -- 172.21.15.114:0/3904900246 shutdown_connections 2024-09-13T18:20:40.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.470+0000 7fb04bed0640 1 -- 172.21.15.114:0/3904900246 wait complete. 2024-09-13T18:20:40.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.471+0000 7fb04bed0640 1 Processor -- start 2024-09-13T18:20:40.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.471+0000 7fb04bed0640 1 -- start start 2024-09-13T18:20:40.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.471+0000 7fb04bed0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0440fffe0 0x7fb0441962e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:40.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.471+0000 7fb04bed0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb044100990 0x7fb044196820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:40.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.471+0000 7fb04bed0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb044105110 0x7fb04419ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:40.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.471+0000 7fb04bed0640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04419b220 con 0x7fb0440fffe0 2024-09-13T18:20:40.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.471+0000 7fb04bed0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04419b390 con 0x7fb044105110 2024-09-13T18:20:40.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.471+0000 7fb04bed0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04419b500 con 0x7fb044100990 2024-09-13T18:20:40.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.471+0000 7fb049c6c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0440fffe0 0x7fb0441962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:40.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.471+0000 7fb049c6c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0440fffe0 0x7fb0441962e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:54756/0 (socket says 172.21.15.114:54756) 2024-09-13T18:20:40.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.471+0000 7fb049c6c640 1 -- 172.21.15.114:0/3212363292 learned_addr learned my addr 172.21.15.114:0/3212363292 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:20:40.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.471+0000 7fb04946b640 1 --2- 172.21.15.114:0/3212363292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb044100990 0x7fb044196820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:40.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.471+0000 7fb04a46d640 1 --2- 172.21.15.114:0/3212363292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb044105110 0x7fb04419ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:40.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.472+0000 7fb049c6c640 1 -- 172.21.15.114:0/3212363292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb044100990 msgr2=0x7fb044196820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:40.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.472+0000 7fb049c6c640 1 --2- 172.21.15.114:0/3212363292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb044100990 0x7fb044196820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:40.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.472+0000 7fb049c6c640 1 -- 172.21.15.114:0/3212363292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb044105110 msgr2=0x7fb04419ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:40.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.472+0000 7fb049c6c640 1 --2- 172.21.15.114:0/3212363292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb044105110 0x7fb04419ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:40.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.472+0000 7fb049c6c640 1 -- 172.21.15.114:0/3212363292 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb03c009900 con 0x7fb0440fffe0 2024-09-13T18:20:40.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.472+0000 7fb04946b640 1 --2- 172.21.15.114:0/3212363292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb044100990 0x7fb044196820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:20:40.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.472+0000 7fb04a46d640 1 --2- 172.21.15.114:0/3212363292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb044105110 0x7fb04419ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:20:40.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.472+0000 7fb049c6c640 1 --2- 172.21.15.114:0/3212363292 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0440fffe0 0x7fb0441962e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb04000d770 tx=0x7fb04000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:40.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.472+0000 7fb03affd640 1 -- 172.21.15.114:0/3212363292 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb040009c30 con 0x7fb0440fffe0 2024-09-13T18:20:40.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.472+0000 7fb04bed0640 1 -- 172.21.15.114:0/3212363292 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb04419b7e0 con 0x7fb0440fffe0 2024-09-13T18:20:40.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.472+0000 7fb03affd640 1 -- 172.21.15.114:0/3212363292 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb04000de80 con 0x7fb0440fffe0 2024-09-13T18:20:40.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.472+0000 7fb03affd640 1 -- 172.21.15.114:0/3212363292 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb040004d40 con 0x7fb0440fffe0 2024-09-13T18:20:40.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.472+0000 7fb04bed0640 1 -- 172.21.15.114:0/3212363292 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0441a30a0 con 0x7fb0440fffe0 2024-09-13T18:20:40.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.473+0000 7fb04bed0640 1 -- 172.21.15.114:0/3212363292 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0441014c0 con 0x7fb0440fffe0 2024-09-13T18:20:40.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.474+0000 7fb03affd640 1 -- 172.21.15.114:0/3212363292 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fb040004ea0 con 0x7fb0440fffe0 2024-09-13T18:20:40.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.474+0000 7fb03affd640 1 --2- 172.21.15.114:0/3212363292 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb0300a1110 0x7fb0300a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:40.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.475+0000 7fb03affd640 1 -- 172.21.15.114:0/3212363292 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb0400cc890 con 0x7fb0440fffe0 2024-09-13T18:20:40.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.475+0000 7fb04946b640 1 --2- 172.21.15.114:0/3212363292 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb0300a1110 0x7fb0300a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:40.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.475+0000 7fb04946b640 1 --2- 172.21.15.114:0/3212363292 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb0300a1110 0x7fb0300a35d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb034005fd0 tx=0x7fb034005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:40.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.479+0000 7fb03affd640 1 -- 172.21.15.114:0/3212363292 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fb040095bb0 con 0x7fb0440fffe0 2024-09-13T18:20:40.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.688+0000 7fb04bed0640 1 -- 172.21.15.114:0/3212363292 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fb04419ba30 con 0x7fb0440fffe0 2024-09-13T18:20:40.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.694+0000 7fb03affd640 1 -- 172.21.15.114:0/3212363292 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260294 (secure 0 0 0) 0x7fb040020090 con 0x7fb0440fffe0 2024-09-13T18:20:40.700 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:20:40.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.697+0000 7fb04bed0640 1 -- 172.21.15.114:0/3212363292 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb0300a1110 msgr2=0x7fb0300a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:40.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.697+0000 7fb04bed0640 1 --2- 172.21.15.114:0/3212363292 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb0300a1110 0x7fb0300a35d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb034005fd0 tx=0x7fb034005850 comp rx=0 tx=0).stop 2024-09-13T18:20:40.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.697+0000 7fb04bed0640 1 -- 172.21.15.114:0/3212363292 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0440fffe0 msgr2=0x7fb0441962e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:40.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.697+0000 7fb04bed0640 1 --2- 172.21.15.114:0/3212363292 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0440fffe0 0x7fb0441962e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb04000d770 tx=0x7fb04000dc30 comp rx=0 tx=0).stop 2024-09-13T18:20:40.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.697+0000 7fb04bed0640 1 -- 172.21.15.114:0/3212363292 shutdown_connections 2024-09-13T18:20:40.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.697+0000 7fb04bed0640 1 --2- 172.21.15.114:0/3212363292 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb0300a1110 0x7fb0300a35d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:40.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.697+0000 7fb04bed0640 1 --2- 172.21.15.114:0/3212363292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb044105110 0x7fb04419ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:40.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.697+0000 7fb04bed0640 1 --2- 172.21.15.114:0/3212363292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb044100990 0x7fb044196820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:40.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.697+0000 7fb04bed0640 1 --2- 172.21.15.114:0/3212363292 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0440fffe0 0x7fb0441962e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:40.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.697+0000 7fb04bed0640 1 -- 172.21.15.114:0/3212363292 >> 172.21.15.114:0/3212363292 conn(0x7fb0440fbae0 msgr2=0x7fb0440fcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:40.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.697+0000 7fb04bed0640 1 -- 172.21.15.114:0/3212363292 shutdown_connections 2024-09-13T18:20:40.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:40.697+0000 7fb04bed0640 1 -- 172.21.15.114:0/3212363292 wait complete. 2024-09-13T18:20:41.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:40 smithi134 ceph-mon[34283]: pgmap v298: 129 pgs: 129 active+clean; 640 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.4 KiB/s wr, 23 op/s; 1 B/s, 0 objects/s recovering 2024-09-13T18:20:41.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:40 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/3212363292' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-13T18:20:41.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:40 smithi204 ceph-mon[34777]: pgmap v298: 129 pgs: 129 active+clean; 640 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.4 KiB/s wr, 23 op/s; 1 B/s, 0 objects/s recovering 2024-09-13T18:20:41.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:40 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/3212363292' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-13T18:20:41.469 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:40 smithi114 ceph-mon[30897]: pgmap v298: 129 pgs: 129 active+clean; 640 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.4 KiB/s wr, 23 op/s; 1 B/s, 0 objects/s recovering 2024-09-13T18:20:41.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:40 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/3212363292' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-13T18:20:41.506 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":17,"flags":0,"active_gid":14148,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":3396185541},{"type":"v1","addr":"172.21.15.114:6801","nonce":3396185541}]},"active_addr":"172.21.15.114:6801/3396185541","active_change":"2024-09-13T18:13:25.823127+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24115,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24119,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.114:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.114:0","nonce":2714091073}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.114:0","nonce":842237781}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.114:0","nonce":1684954293}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.114:0","nonce":3407523320}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.114:0","nonce":2942437413}]}]} 2024-09-13T18:20:41.511 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-13T18:20:41.512 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-13T18:20:41.512 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-13T18:20:41.716 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:20:43.216 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:42 smithi114 ceph-mon[30897]: pgmap v299: 129 pgs: 129 active+clean; 640 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.7 KiB/s wr, 19 op/s; 0 B/s, 0 objects/s recovering 2024-09-13T18:20:43.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:42 smithi134 ceph-mon[34283]: pgmap v299: 129 pgs: 129 active+clean; 640 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.7 KiB/s wr, 19 op/s; 0 B/s, 0 objects/s recovering 2024-09-13T18:20:43.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:42 smithi204 ceph-mon[34777]: pgmap v299: 129 pgs: 129 active+clean; 640 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.7 KiB/s wr, 19 op/s; 0 B/s, 0 objects/s recovering 2024-09-13T18:20:43.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.430+0000 7f3cc7420640 1 -- 172.21.15.114:0/3007829213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3cc00fef00 msgr2=0x7f3cc00ff2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:43.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.430+0000 7f3cc7420640 1 --2- 172.21.15.114:0/3007829213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3cc00fef00 0x7f3cc00ff2e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc009860 tx=0x7f3cbc02a340 comp rx=0 tx=0).stop 2024-09-13T18:20:43.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.430+0000 7f3cc7420640 1 -- 172.21.15.114:0/3007829213 shutdown_connections 2024-09-13T18:20:43.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.430+0000 7f3cc7420640 1 --2- 172.21.15.114:0/3007829213 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3cc0104030 0x7f3cc010bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:43.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.430+0000 7f3cc7420640 1 --2- 172.21.15.114:0/3007829213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cc00ff8b0 0x7f3cc0103900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:43.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.430+0000 7f3cc7420640 1 --2- 172.21.15.114:0/3007829213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3cc00fef00 0x7f3cc00ff2e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:43.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.430+0000 7f3cc7420640 1 -- 172.21.15.114:0/3007829213 >> 172.21.15.114:0/3007829213 conn(0x7f3cc00fa9a0 msgr2=0x7f3cc00fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:43.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.431+0000 7f3cc7420640 1 -- 172.21.15.114:0/3007829213 shutdown_connections 2024-09-13T18:20:43.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.431+0000 7f3cc7420640 1 -- 172.21.15.114:0/3007829213 wait complete. 2024-09-13T18:20:43.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.431+0000 7f3cc7420640 1 Processor -- start 2024-09-13T18:20:43.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.431+0000 7f3cc7420640 1 -- start start 2024-09-13T18:20:43.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.432+0000 7f3cc7420640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cc00fef00 0x7f3cc0196230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:43.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.432+0000 7f3cc7420640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3cc00ff8b0 0x7f3cc0196770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:43.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.432+0000 7f3cc7420640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3cc0104030 0x7f3cc019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:43.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.432+0000 7f3cc7420640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc019b120 con 0x7f3cc00ff8b0 2024-09-13T18:20:43.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.432+0000 7f3cc7420640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc019b290 con 0x7f3cc00fef00 2024-09-13T18:20:43.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.432+0000 7f3cc7420640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc019b400 con 0x7f3cc0104030 2024-09-13T18:20:43.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.432+0000 7f3cc49bb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3cc00ff8b0 0x7f3cc0196770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:43.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.432+0000 7f3cc49bb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3cc00ff8b0 0x7f3cc0196770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51024/0 (socket says 172.21.15.114:51024) 2024-09-13T18:20:43.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.432+0000 7f3cc49bb640 1 -- 172.21.15.114:0/707987271 learned_addr learned my addr 172.21.15.114:0/707987271 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:20:43.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.432+0000 7f3cc59bd640 1 --2- 172.21.15.114:0/707987271 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3cc0104030 0x7f3cc019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:43.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.432+0000 7f3cc51bc640 1 --2- 172.21.15.114:0/707987271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cc00fef00 0x7f3cc0196230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:43.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.432+0000 7f3cc49bb640 1 -- 172.21.15.114:0/707987271 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3cc0104030 msgr2=0x7f3cc019ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:43.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.432+0000 7f3cc49bb640 1 --2- 172.21.15.114:0/707987271 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3cc0104030 0x7f3cc019ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:43.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.432+0000 7f3cc49bb640 1 -- 172.21.15.114:0/707987271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cc00fef00 msgr2=0x7f3cc0196230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:43.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.432+0000 7f3cc49bb640 1 --2- 172.21.15.114:0/707987271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cc00fef00 0x7f3cc0196230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:43.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.432+0000 7f3cc49bb640 1 -- 172.21.15.114:0/707987271 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cbc009490 con 0x7f3cc00ff8b0 2024-09-13T18:20:43.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.432+0000 7f3cc59bd640 1 --2- 172.21.15.114:0/707987271 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3cc0104030 0x7f3cc019ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:20:43.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.432+0000 7f3cc51bc640 1 --2- 172.21.15.114:0/707987271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cc00fef00 0x7f3cc0196230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:20:43.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.432+0000 7f3cc49bb640 1 --2- 172.21.15.114:0/707987271 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3cc00ff8b0 0x7f3cc0196770 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3cb000e9f0 tx=0x7f3cb000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:43.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.432+0000 7f3cb67fc640 1 -- 172.21.15.114:0/707987271 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb0009720 con 0x7f3cc00ff8b0 2024-09-13T18:20:43.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.432+0000 7f3cc7420640 1 -- 172.21.15.114:0/707987271 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cc019b6e0 con 0x7f3cc00ff8b0 2024-09-13T18:20:43.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.432+0000 7f3cb67fc640 1 -- 172.21.15.114:0/707987271 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3cb000cd80 con 0x7f3cc00ff8b0 2024-09-13T18:20:43.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.432+0000 7f3cb67fc640 1 -- 172.21.15.114:0/707987271 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb0010430 con 0x7f3cc00ff8b0 2024-09-13T18:20:43.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.432+0000 7f3cc7420640 1 -- 172.21.15.114:0/707987271 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cc01a2fa0 con 0x7f3cc00ff8b0 2024-09-13T18:20:43.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.433+0000 7f3cc7420640 1 -- 172.21.15.114:0/707987271 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cc01003e0 con 0x7f3cc00ff8b0 2024-09-13T18:20:43.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.434+0000 7f3cb67fc640 1 -- 172.21.15.114:0/707987271 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f3cb0004fb0 con 0x7f3cc00ff8b0 2024-09-13T18:20:43.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.435+0000 7f3cb67fc640 1 --2- 172.21.15.114:0/707987271 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f3cac0a1110 0x7f3cac0a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:43.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.435+0000 7f3cc51bc640 1 --2- 172.21.15.114:0/707987271 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f3cac0a1110 0x7f3cac0a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:43.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.435+0000 7f3cb67fc640 1 -- 172.21.15.114:0/707987271 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f3cb0014070 con 0x7f3cc00ff8b0 2024-09-13T18:20:43.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.436+0000 7f3cc51bc640 1 --2- 172.21.15.114:0/707987271 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f3cac0a1110 0x7f3cac0a35d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc005940 tx=0x7f3cbc005a90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:43.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.439+0000 7f3cb67fc640 1 -- 172.21.15.114:0/707987271 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f3cb0096590 con 0x7f3cc00ff8b0 2024-09-13T18:20:43.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.606+0000 7f3cc7420640 1 -- 172.21.15.114:0/707987271 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3cc010ebb0 con 0x7f3cc00ff8b0 2024-09-13T18:20:43.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.607+0000 7f3cb67fc640 1 -- 172.21.15.114:0/707987271 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24556 (secure 0 0 0) 0x7f3cb0094e60 con 0x7f3cc00ff8b0 2024-09-13T18:20:43.610 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:20:43.610 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":91,"fsid":"97e7edf2-71fb-11ef-bceb-c7b262605968","created":"2024-09-13T18:12:36.360625+0000","modified":"2024-09-13T18:20:27.176653+0000","last_up_change":"2024-09-13T18:19:12.350370+0000","last_in_change":"2024-09-13T18:18:52.321596+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:15:40.129659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:20:00.024407+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:20:01.085549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T18:20:04.334216+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9914f01a-9f6f-4878-b955-83570199c354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6803","nonce":3499525826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6805","nonce":3499525826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6809","nonce":3499525826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6807","nonce":3499525826}]},"public_addr":"172.21.15.114:6803/3499525826","cluster_addr":"172.21.15.114:6805/3499525826","heartbeat_back_addr":"172.21.15.114:6809/3499525826","heartbeat_front_addr":"172.21.15.114:6807/3499525826","state":["exists","up"]},{"osd":1,"uuid":"ef65e012-7469-4c20-b354-17022bca63c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6801","nonce":4264264866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6803","nonce":4264264866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6807","nonce":4264264866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6805","nonce":4264264866}]},"public_addr":"172.21.15.134:6801/4264264866","cluster_addr":"172.21.15.134:6803/4264264866","heartbeat_back_addr":"172.21.15.134:6807/4264264866","heartbeat_front_addr":"172.21.15.134:6805/4264264866","state":["exists","up"]},{"osd":2,"uuid":"578213fd-715f-4be7-91fa-254f3d0e3022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6801","nonce":2288807707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6803","nonce":2288807707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6807","nonce":2288807707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6805","nonce":2288807707}]},"public_addr":"172.21.15.204:6801/2288807707","cluster_addr":"172.21.15.204:6803/2288807707","heartbeat_back_addr":"172.21.15.204:6807/2288807707","heartbeat_front_addr":"172.21.15.204:6805/2288807707","state":["exists","up"]},{"osd":3,"uuid":"b64aab3f-1e52-4828-8397-74af5e05f2cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6811","nonce":2904204254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6813","nonce":2904204254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6817","nonce":2904204254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6815","nonce":2904204254}]},"public_addr":"172.21.15.114:6811/2904204254","cluster_addr":"172.21.15.114:6813/2904204254","heartbeat_back_addr":"172.21.15.114:6817/2904204254","heartbeat_front_addr":"172.21.15.114:6815/2904204254","state":["exists","up"]},{"osd":4,"uuid":"f38943d6-39c0-42e1-b427-cb6bb90ac38a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6809","nonce":1773945333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6811","nonce":1773945333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6815","nonce":1773945333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6813","nonce":1773945333}]},"public_addr":"172.21.15.134:6809/1773945333","cluster_addr":"172.21.15.134:6811/1773945333","heartbeat_back_addr":"172.21.15.134:6815/1773945333","heartbeat_front_addr":"172.21.15.134:6813/1773945333","state":["exists","up"]},{"osd":5,"uuid":"bb38f361-8e00-4178-9752-5ddbf3af20e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6809","nonce":2371778055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6811","nonce":2371778055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6815","nonce":2371778055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6813","nonce":2371778055}]},"public_addr":"172.21.15.204:6809/2371778055","cluster_addr":"172.21.15.204:6811/2371778055","heartbeat_back_addr":"172.21.15.204:6815/2371778055","heartbeat_front_addr":"172.21.15.204:6813/2371778055","state":["exists","up"]},{"osd":6,"uuid":"93d40a03-e7ff-4a3f-9efa-5c5403c9983d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6819","nonce":1484325044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6821","nonce":1484325044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6825","nonce":1484325044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6823","nonce":1484325044}]},"public_addr":"172.21.15.114:6819/1484325044","cluster_addr":"172.21.15.114:6821/1484325044","heartbeat_back_addr":"172.21.15.114:6825/1484325044","heartbeat_front_addr":"172.21.15.114:6823/1484325044","state":["exists","up"]},{"osd":7,"uuid":"982f629a-1b95-4be1-8e75-5ddce1ad77b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6817","nonce":799246614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6819","nonce":799246614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6823","nonce":799246614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6821","nonce":799246614}]},"public_addr":"172.21.15.134:6817/799246614","cluster_addr":"172.21.15.134:6819/799246614","heartbeat_back_addr":"172.21.15.134:6823/799246614","heartbeat_front_addr":"172.21.15.134:6821/799246614","state":["exists","up"]},{"osd":8,"uuid":"409beb9b-8227-4dad-a4a0-f0e5ce70dc5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6817","nonce":858331805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6819","nonce":858331805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6823","nonce":858331805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6821","nonce":858331805}]},"public_addr":"172.21.15.204:6817/858331805","cluster_addr":"172.21.15.204:6819/858331805","heartbeat_back_addr":"172.21.15.204:6823/858331805","heartbeat_front_addr":"172.21.15.204:6821/858331805","state":["exists","up"]},{"osd":9,"uuid":"a345a15c-b1fc-4877-b445-418eebf0d0ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6827","nonce":2172392854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6829","nonce":2172392854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6833","nonce":2172392854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6831","nonce":2172392854}]},"public_addr":"172.21.15.114:6827/2172392854","cluster_addr":"172.21.15.114:6829/2172392854","heartbeat_back_addr":"172.21.15.114:6833/2172392854","heartbeat_front_addr":"172.21.15.114:6831/2172392854","state":["exists","up"]},{"osd":10,"uuid":"464d995f-d90b-4caa-addb-1f7562bd74b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6825","nonce":3073513468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6827","nonce":3073513468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6831","nonce":3073513468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6829","nonce":3073513468}]},"public_addr":"172.21.15.134:6825/3073513468","cluster_addr":"172.21.15.134:6827/3073513468","heartbeat_back_addr":"172.21.15.134:6831/3073513468","heartbeat_front_addr":"172.21.15.134:6829/3073513468","state":["exists","up"]},{"osd":11,"uuid":"71dd794d-326c-477f-8d02-ee32bff1350d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6825","nonce":3239521765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6827","nonce":3239521765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6831","nonce":3239521765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6829","nonce":3239521765}]},"public_addr":"172.21.15.204:6825/3239521765","cluster_addr":"172.21.15.204:6827/3239521765","heartbeat_back_addr":"172.21.15.204:6831/3239521765","heartbeat_front_addr":"172.21.15.204:6829/3239521765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:14:54.542687+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:15:15.746055+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:15:38.371131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:01.383932+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:24.796237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:48.422214+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:11.284323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:34.570365+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:58.232855+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:18:21.609596+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:18:45.565859+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:19:10.017969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:6801/4281866858":"2024-09-14T18:13:25.823027+0000","172.21.15.114:6800/4281866858":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2949422849":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2034742251":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/722571999":"2024-09-14T18:13:03.521116+0000","172.21.15.114:6801/830738794":"2024-09-14T18:13:03.521116+0000","172.21.15.114:6800/830738794":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/2828364892":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/4257828906":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/3544951322":"2024-09-14T18:13:03.521116+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:20:43.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.609+0000 7f3cc7420640 1 -- 172.21.15.114:0/707987271 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f3cac0a1110 msgr2=0x7f3cac0a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:43.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.609+0000 7f3cc7420640 1 --2- 172.21.15.114:0/707987271 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f3cac0a1110 0x7f3cac0a35d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc005940 tx=0x7f3cbc005a90 comp rx=0 tx=0).stop 2024-09-13T18:20:43.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.609+0000 7f3cc7420640 1 -- 172.21.15.114:0/707987271 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3cc00ff8b0 msgr2=0x7f3cc0196770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:43.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.609+0000 7f3cc7420640 1 --2- 172.21.15.114:0/707987271 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3cc00ff8b0 0x7f3cc0196770 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3cb000e9f0 tx=0x7f3cb000edf0 comp rx=0 tx=0).stop 2024-09-13T18:20:43.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.609+0000 7f3cc7420640 1 -- 172.21.15.114:0/707987271 shutdown_connections 2024-09-13T18:20:43.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.609+0000 7f3cc7420640 1 --2- 172.21.15.114:0/707987271 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f3cac0a1110 0x7f3cac0a35d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:43.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.609+0000 7f3cc7420640 1 --2- 172.21.15.114:0/707987271 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3cc0104030 0x7f3cc019ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:43.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.609+0000 7f3cc7420640 1 --2- 172.21.15.114:0/707987271 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3cc00ff8b0 0x7f3cc0196770 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:43.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.609+0000 7f3cc7420640 1 --2- 172.21.15.114:0/707987271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cc00fef00 0x7f3cc0196230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:43.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.610+0000 7f3cc7420640 1 -- 172.21.15.114:0/707987271 >> 172.21.15.114:0/707987271 conn(0x7f3cc00fa9a0 msgr2=0x7f3cc00fbe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:43.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.610+0000 7f3cc7420640 1 -- 172.21.15.114:0/707987271 shutdown_connections 2024-09-13T18:20:43.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:43.610+0000 7f3cc7420640 1 -- 172.21.15.114:0/707987271 wait complete. 2024-09-13T18:20:44.300 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:43 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/707987271' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:20:44.322 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-13T18:20:44.322 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-13T18:20:44.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:43 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/707987271' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:20:44.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:43 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/707987271' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:20:44.520 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:20:45.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:44 smithi114 ceph-mon[30897]: pgmap v300: 129 pgs: 129 active+clean; 640 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.5 KiB/s wr, 13 op/s 2024-09-13T18:20:45.270 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:44 smithi134 ceph-mon[34283]: pgmap v300: 129 pgs: 129 active+clean; 640 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.5 KiB/s wr, 13 op/s 2024-09-13T18:20:45.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:44 smithi204 ceph-mon[34777]: pgmap v300: 129 pgs: 129 active+clean; 640 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.5 KiB/s wr, 13 op/s 2024-09-13T18:20:46.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:46.231+0000 7f9e079df640 1 -- 172.21.15.114:0/1684553180 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e00100990 msgr2=0x7f9e001049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:46.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:46.231+0000 7f9e079df640 1 --2- 172.21.15.114:0/1684553180 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e00100990 0x7f9e001049e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9df4009930 tx=0x7f9df402a420 comp rx=0 tx=0).stop 2024-09-13T18:20:46.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:46.231+0000 7f9e079df640 1 -- 172.21.15.114:0/1684553180 shutdown_connections 2024-09-13T18:20:46.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:46.231+0000 7f9e079df640 1 --2- 172.21.15.114:0/1684553180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e00105110 0x7f9e0010cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:46.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:46.231+0000 7f9e079df640 1 --2- 172.21.15.114:0/1684553180 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e00100990 0x7f9e001049e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:46.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:46.231+0000 7f9e079df640 1 --2- 172.21.15.114:0/1684553180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e000fffe0 0x7f9e001003c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:46.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:46.231+0000 7f9e079df640 1 -- 172.21.15.114:0/1684553180 >> 172.21.15.114:0/1684553180 conn(0x7f9e000fbae0 msgr2=0x7f9e000fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:46.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:46.231+0000 7f9e079df640 1 -- 172.21.15.114:0/1684553180 shutdown_connections 2024-09-13T18:20:46.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:46.231+0000 7f9e079df640 1 -- 172.21.15.114:0/1684553180 wait complete. 2024-09-13T18:20:46.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:46.231+0000 7f9e079df640 1 Processor -- start 2024-09-13T18:20:46.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:46.232+0000 7f9e079df640 1 -- start start 2024-09-13T18:20:46.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:46.232+0000 7f9e079df640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e000fffe0 0x7f9e00196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:46.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:46.232+0000 7f9e079df640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e00100990 0x7f9e001967d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:46.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:46.232+0000 7f9e079df640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e00105110 0x7f9e0019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:46.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:46.232+0000 7f9e079df640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0019b130 con 0x7f9e00100990 2024-09-13T18:20:46.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:46.232+0000 7f9e079df640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0019b2a0 con 0x7f9e000fffe0 2024-09-13T18:20:46.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:46.232+0000 7f9e079df640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0019b410 con 0x7f9e00105110 2024-09-13T18:20:46.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:46.232+0000 7f9e04f7a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e00100990 0x7f9e001967d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:46.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:46.232+0000 7f9e04f7a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e00100990 0x7f9e001967d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51036/0 (socket says 172.21.15.114:51036) 2024-09-13T18:20:46.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:46.232+0000 7f9e04f7a640 1 -- 172.21.15.114:0/4283130304 learned_addr learned my addr 172.21.15.114:0/4283130304 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:20:46.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:46.232+0000 7f9e0577b640 1 --2- 172.21.15.114:0/4283130304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e000fffe0 0x7f9e00196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:46.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:46.232+0000 7f9e05f7c640 1 --2- 172.21.15.114:0/4283130304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e00105110 0x7f9e0019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:46.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:46.232+0000 7f9e04f7a640 1 -- 172.21.15.114:0/4283130304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e00105110 msgr2=0x7f9e0019ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:46.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:46.232+0000 7f9e04f7a640 1 --2- 172.21.15.114:0/4283130304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e00105110 0x7f9e0019ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:46.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:46.232+0000 7f9e04f7a640 1 -- 172.21.15.114:0/4283130304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e000fffe0 msgr2=0x7f9e00196290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:46.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:46.232+0000 7f9e04f7a640 1 --2- 172.21.15.114:0/4283130304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e000fffe0 0x7f9e00196290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:46.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:46.232+0000 7f9e04f7a640 1 -- 172.21.15.114:0/4283130304 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df4009560 con 0x7f9e00100990 2024-09-13T18:20:46.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:46.233+0000 7f9e04f7a640 1 --2- 172.21.15.114:0/4283130304 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e00100990 0x7f9e001967d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9df402a940 tx=0x7f9df40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:46.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:46.233+0000 7f9dfe7fc640 1 -- 172.21.15.114:0/4283130304 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df4004cf0 con 0x7f9e00100990 2024-09-13T18:20:46.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:46.233+0000 7f9e079df640 1 -- 172.21.15.114:0/4283130304 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e0019b690 con 0x7f9e00100990 2024-09-13T18:20:46.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:46.233+0000 7f9dfe7fc640 1 -- 172.21.15.114:0/4283130304 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9df4003a80 con 0x7f9e00100990 2024-09-13T18:20:46.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:46.233+0000 7f9dfe7fc640 1 -- 172.21.15.114:0/4283130304 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df403ca70 con 0x7f9e00100990 2024-09-13T18:20:46.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:46.233+0000 7f9e079df640 1 -- 172.21.15.114:0/4283130304 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e001a2de0 con 0x7f9e00100990 2024-09-13T18:20:46.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:46.234+0000 7f9e079df640 1 -- 172.21.15.114:0/4283130304 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e001014c0 con 0x7f9e00100990 2024-09-13T18:20:46.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:46.235+0000 7f9dfe7fc640 1 -- 172.21.15.114:0/4283130304 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f9df4003bf0 con 0x7f9e00100990 2024-09-13T18:20:46.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:46.235+0000 7f9dfe7fc640 1 --2- 172.21.15.114:0/4283130304 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f9de80a1110 0x7f9de80a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:46.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:46.235+0000 7f9e0577b640 1 --2- 172.21.15.114:0/4283130304 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f9de80a1110 0x7f9de80a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:46.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:46.236+0000 7f9dfe7fc640 1 -- 172.21.15.114:0/4283130304 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9df40ee0a0 con 0x7f9e00100990 2024-09-13T18:20:46.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:46.236+0000 7f9e0577b640 1 --2- 172.21.15.114:0/4283130304 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f9de80a1110 0x7f9de80a35d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9df0005fd0 tx=0x7f9df000b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:46.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:46.239+0000 7f9dfe7fc640 1 -- 172.21.15.114:0/4283130304 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f9df40b73c0 con 0x7f9e00100990 2024-09-13T18:20:46.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:46.403+0000 7f9e079df640 1 -- 172.21.15.114:0/4283130304 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e001083f0 con 0x7f9e00100990 2024-09-13T18:20:46.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:46.405+0000 7f9dfe7fc640 1 -- 172.21.15.114:0/4283130304 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24556 (secure 0 0 0) 0x7f9df40ba9e0 con 0x7f9e00100990 2024-09-13T18:20:46.407 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:20:46.407 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":91,"fsid":"97e7edf2-71fb-11ef-bceb-c7b262605968","created":"2024-09-13T18:12:36.360625+0000","modified":"2024-09-13T18:20:27.176653+0000","last_up_change":"2024-09-13T18:19:12.350370+0000","last_in_change":"2024-09-13T18:18:52.321596+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:15:40.129659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:20:00.024407+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:20:01.085549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T18:20:04.334216+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9914f01a-9f6f-4878-b955-83570199c354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6803","nonce":3499525826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6805","nonce":3499525826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6809","nonce":3499525826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6807","nonce":3499525826}]},"public_addr":"172.21.15.114:6803/3499525826","cluster_addr":"172.21.15.114:6805/3499525826","heartbeat_back_addr":"172.21.15.114:6809/3499525826","heartbeat_front_addr":"172.21.15.114:6807/3499525826","state":["exists","up"]},{"osd":1,"uuid":"ef65e012-7469-4c20-b354-17022bca63c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6801","nonce":4264264866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6803","nonce":4264264866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6807","nonce":4264264866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6805","nonce":4264264866}]},"public_addr":"172.21.15.134:6801/4264264866","cluster_addr":"172.21.15.134:6803/4264264866","heartbeat_back_addr":"172.21.15.134:6807/4264264866","heartbeat_front_addr":"172.21.15.134:6805/4264264866","state":["exists","up"]},{"osd":2,"uuid":"578213fd-715f-4be7-91fa-254f3d0e3022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6801","nonce":2288807707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6803","nonce":2288807707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6807","nonce":2288807707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6805","nonce":2288807707}]},"public_addr":"172.21.15.204:6801/2288807707","cluster_addr":"172.21.15.204:6803/2288807707","heartbeat_back_addr":"172.21.15.204:6807/2288807707","heartbeat_front_addr":"172.21.15.204:6805/2288807707","state":["exists","up"]},{"osd":3,"uuid":"b64aab3f-1e52-4828-8397-74af5e05f2cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6811","nonce":2904204254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6813","nonce":2904204254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6817","nonce":2904204254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6815","nonce":2904204254}]},"public_addr":"172.21.15.114:6811/2904204254","cluster_addr":"172.21.15.114:6813/2904204254","heartbeat_back_addr":"172.21.15.114:6817/2904204254","heartbeat_front_addr":"172.21.15.114:6815/2904204254","state":["exists","up"]},{"osd":4,"uuid":"f38943d6-39c0-42e1-b427-cb6bb90ac38a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6809","nonce":1773945333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6811","nonce":1773945333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6815","nonce":1773945333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6813","nonce":1773945333}]},"public_addr":"172.21.15.134:6809/1773945333","cluster_addr":"172.21.15.134:6811/1773945333","heartbeat_back_addr":"172.21.15.134:6815/1773945333","heartbeat_front_addr":"172.21.15.134:6813/1773945333","state":["exists","up"]},{"osd":5,"uuid":"bb38f361-8e00-4178-9752-5ddbf3af20e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6809","nonce":2371778055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6811","nonce":2371778055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6815","nonce":2371778055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6813","nonce":2371778055}]},"public_addr":"172.21.15.204:6809/2371778055","cluster_addr":"172.21.15.204:6811/2371778055","heartbeat_back_addr":"172.21.15.204:6815/2371778055","heartbeat_front_addr":"172.21.15.204:6813/2371778055","state":["exists","up"]},{"osd":6,"uuid":"93d40a03-e7ff-4a3f-9efa-5c5403c9983d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6819","nonce":1484325044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6821","nonce":1484325044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6825","nonce":1484325044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6823","nonce":1484325044}]},"public_addr":"172.21.15.114:6819/1484325044","cluster_addr":"172.21.15.114:6821/1484325044","heartbeat_back_addr":"172.21.15.114:6825/1484325044","heartbeat_front_addr":"172.21.15.114:6823/1484325044","state":["exists","up"]},{"osd":7,"uuid":"982f629a-1b95-4be1-8e75-5ddce1ad77b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6817","nonce":799246614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6819","nonce":799246614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6823","nonce":799246614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6821","nonce":799246614}]},"public_addr":"172.21.15.134:6817/799246614","cluster_addr":"172.21.15.134:6819/799246614","heartbeat_back_addr":"172.21.15.134:6823/799246614","heartbeat_front_addr":"172.21.15.134:6821/799246614","state":["exists","up"]},{"osd":8,"uuid":"409beb9b-8227-4dad-a4a0-f0e5ce70dc5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6817","nonce":858331805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6819","nonce":858331805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6823","nonce":858331805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6821","nonce":858331805}]},"public_addr":"172.21.15.204:6817/858331805","cluster_addr":"172.21.15.204:6819/858331805","heartbeat_back_addr":"172.21.15.204:6823/858331805","heartbeat_front_addr":"172.21.15.204:6821/858331805","state":["exists","up"]},{"osd":9,"uuid":"a345a15c-b1fc-4877-b445-418eebf0d0ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6827","nonce":2172392854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6829","nonce":2172392854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6833","nonce":2172392854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6831","nonce":2172392854}]},"public_addr":"172.21.15.114:6827/2172392854","cluster_addr":"172.21.15.114:6829/2172392854","heartbeat_back_addr":"172.21.15.114:6833/2172392854","heartbeat_front_addr":"172.21.15.114:6831/2172392854","state":["exists","up"]},{"osd":10,"uuid":"464d995f-d90b-4caa-addb-1f7562bd74b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6825","nonce":3073513468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6827","nonce":3073513468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6831","nonce":3073513468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6829","nonce":3073513468}]},"public_addr":"172.21.15.134:6825/3073513468","cluster_addr":"172.21.15.134:6827/3073513468","heartbeat_back_addr":"172.21.15.134:6831/3073513468","heartbeat_front_addr":"172.21.15.134:6829/3073513468","state":["exists","up"]},{"osd":11,"uuid":"71dd794d-326c-477f-8d02-ee32bff1350d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6825","nonce":3239521765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6827","nonce":3239521765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6831","nonce":3239521765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6829","nonce":3239521765}]},"public_addr":"172.21.15.204:6825/3239521765","cluster_addr":"172.21.15.204:6827/3239521765","heartbeat_back_addr":"172.21.15.204:6831/3239521765","heartbeat_front_addr":"172.21.15.204:6829/3239521765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:14:54.542687+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:15:15.746055+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:15:38.371131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:01.383932+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:24.796237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:48.422214+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:11.284323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:34.570365+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:58.232855+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:18:21.609596+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:18:45.565859+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:19:10.017969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:6801/4281866858":"2024-09-14T18:13:25.823027+0000","172.21.15.114:6800/4281866858":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2949422849":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2034742251":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/722571999":"2024-09-14T18:13:03.521116+0000","172.21.15.114:6801/830738794":"2024-09-14T18:13:03.521116+0000","172.21.15.114:6800/830738794":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/2828364892":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/4257828906":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/3544951322":"2024-09-14T18:13:03.521116+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:20:46.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:46.407+0000 7f9e079df640 1 -- 172.21.15.114:0/4283130304 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f9de80a1110 msgr2=0x7f9de80a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:46.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:46.407+0000 7f9e079df640 1 --2- 172.21.15.114:0/4283130304 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f9de80a1110 0x7f9de80a35d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9df0005fd0 tx=0x7f9df000b390 comp rx=0 tx=0).stop 2024-09-13T18:20:46.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:46.407+0000 7f9e079df640 1 -- 172.21.15.114:0/4283130304 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e00100990 msgr2=0x7f9e001967d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:46.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:46.407+0000 7f9e079df640 1 --2- 172.21.15.114:0/4283130304 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e00100990 0x7f9e001967d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9df402a940 tx=0x7f9df40046c0 comp rx=0 tx=0).stop 2024-09-13T18:20:46.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:46.407+0000 7f9e079df640 1 -- 172.21.15.114:0/4283130304 shutdown_connections 2024-09-13T18:20:46.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:46.407+0000 7f9e079df640 1 --2- 172.21.15.114:0/4283130304 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f9de80a1110 0x7f9de80a35d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:46.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:46.407+0000 7f9e079df640 1 --2- 172.21.15.114:0/4283130304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e00105110 0x7f9e0019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:46.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:46.407+0000 7f9e079df640 1 --2- 172.21.15.114:0/4283130304 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e00100990 0x7f9e001967d0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:46.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:46.407+0000 7f9e079df640 1 --2- 172.21.15.114:0/4283130304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e000fffe0 0x7f9e00196290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:46.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:46.407+0000 7f9e079df640 1 -- 172.21.15.114:0/4283130304 >> 172.21.15.114:0/4283130304 conn(0x7f9e000fbae0 msgr2=0x7f9e000fcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:46.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:46.407+0000 7f9e079df640 1 -- 172.21.15.114:0/4283130304 shutdown_connections 2024-09-13T18:20:46.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:46.407+0000 7f9e079df640 1 -- 172.21.15.114:0/4283130304 wait complete. 2024-09-13T18:20:47.055 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-13T18:20:47.055 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-13T18:20:47.056 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-13T18:20:47.056 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-13T18:20:47.057 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-13T18:20:47.057 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-13T18:20:47.057 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-13T18:20:47.058 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-13T18:20:47.058 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-13T18:20:47.059 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-13T18:20:47.059 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-13T18:20:47.059 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-13T18:20:47.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:46 smithi134 ceph-mon[34283]: pgmap v301: 129 pgs: 129 active+clean; 640 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-09-13T18:20:47.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:46 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/4283130304' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:20:47.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:46 smithi204 ceph-mon[34777]: pgmap v301: 129 pgs: 129 active+clean; 640 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-09-13T18:20:47.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:46 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/4283130304' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:20:47.472 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:46 smithi114 ceph-mon[30897]: pgmap v301: 129 pgs: 129 active+clean; 640 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-09-13T18:20:47.472 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:46 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/4283130304' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:20:47.593 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:20:47.769 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:20:47.781 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:20:47.781 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:20:47.781 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:20:47.782 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:20:47.785 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:20:47.785 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:20:47.787 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:20:47.792 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:20:47.822 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:20:47.827 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:20:49.049 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:48 smithi114 ceph-mon[30897]: pgmap v302: 129 pgs: 129 active+clean; 640 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-13T18:20:49.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:48 smithi134 ceph-mon[34283]: pgmap v302: 129 pgs: 129 active+clean; 640 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-13T18:20:49.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:48 smithi204 ceph-mon[34777]: pgmap v302: 129 pgs: 129 active+clean; 640 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-13T18:20:51.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:50 smithi114 ceph-mon[30897]: pgmap v303: 129 pgs: 129 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-13T18:20:51.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:50 smithi134 ceph-mon[34283]: pgmap v303: 129 pgs: 129 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-13T18:20:51.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:50 smithi204 ceph-mon[34777]: pgmap v303: 129 pgs: 129 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-13T18:20:53.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:52 smithi134 ceph-mon[34283]: pgmap v304: 129 pgs: 129 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-13T18:20:53.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:52 smithi204 ceph-mon[34777]: pgmap v304: 129 pgs: 129 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-13T18:20:53.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:52 smithi114 ceph-mon[30897]: pgmap v304: 129 pgs: 129 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-13T18:20:55.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:55 smithi114 ceph-mon[30897]: pgmap v305: 129 pgs: 129 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-13T18:20:55.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:55 smithi134 ceph-mon[34283]: pgmap v305: 129 pgs: 129 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-13T18:20:55.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:55 smithi204 ceph-mon[34777]: pgmap v305: 129 pgs: 129 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-13T18:20:57.101 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:57 smithi114 ceph-mon[30897]: pgmap v306: 129 pgs: 129 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-13T18:20:57.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:57 smithi134 ceph-mon[34283]: pgmap v306: 129 pgs: 129 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-13T18:20:57.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:57 smithi204 ceph-mon[34777]: pgmap v306: 129 pgs: 129 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-13T18:20:59.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:20:59 smithi114 ceph-mon[30897]: pgmap v307: 129 pgs: 129 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-13T18:20:59.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.332+0000 7f5f3710f640 1 -- 172.21.15.114:0/1304392822 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f30105110 msgr2=0x7f5f3010cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:59.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.332+0000 7f5f3710f640 1 --2- 172.21.15.114:0/1304392822 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f30105110 0x7f5f3010cc90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5f28009cd0 tx=0x7f5f2802a8e0 comp rx=0 tx=0).stop 2024-09-13T18:20:59.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.332+0000 7f5f3710f640 1 -- 172.21.15.114:0/1304392822 shutdown_connections 2024-09-13T18:20:59.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.332+0000 7f5f3710f640 1 --2- 172.21.15.114:0/1304392822 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f30105110 0x7f5f3010cc90 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:59.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.332+0000 7f5f3710f640 1 --2- 172.21.15.114:0/1304392822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f30100990 0x7f5f301049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:59.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.332+0000 7f5f3710f640 1 --2- 172.21.15.114:0/1304392822 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f300fffe0 0x7f5f301003c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:59.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.332+0000 7f5f3710f640 1 -- 172.21.15.114:0/1304392822 >> 172.21.15.114:0/1304392822 conn(0x7f5f300fbae0 msgr2=0x7f5f300fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:59.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.332+0000 7f5f3710f640 1 -- 172.21.15.114:0/1304392822 shutdown_connections 2024-09-13T18:20:59.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.332+0000 7f5f3710f640 1 -- 172.21.15.114:0/1304392822 wait complete. 2024-09-13T18:20:59.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.333+0000 7f5f3710f640 1 Processor -- start 2024-09-13T18:20:59.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.333+0000 7f5f3710f640 1 -- start start 2024-09-13T18:20:59.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.333+0000 7f5f3710f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f300fffe0 0x7f5f301962c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:59.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.333+0000 7f5f3710f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f30100990 0x7f5f30196800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:59.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.333+0000 7f5f3710f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f30105110 0x7f5f3019ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:59.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.333+0000 7f5f3710f640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3019b200 con 0x7f5f300fffe0 2024-09-13T18:20:59.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.333+0000 7f5f3710f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3019b370 con 0x7f5f30105110 2024-09-13T18:20:59.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.333+0000 7f5f3710f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3019b4e0 con 0x7f5f30100990 2024-09-13T18:20:59.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.333+0000 7f5f34eab640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f300fffe0 0x7f5f301962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:59.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.333+0000 7f5f34eab640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f300fffe0 0x7f5f301962c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44854/0 (socket says 172.21.15.114:44854) 2024-09-13T18:20:59.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.333+0000 7f5f34eab640 1 -- 172.21.15.114:0/3605413902 learned_addr learned my addr 172.21.15.114:0/3605413902 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:20:59.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.334+0000 7f5f34eab640 1 -- 172.21.15.114:0/3605413902 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f30100990 msgr2=0x7f5f30196800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:59.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.334+0000 7f5f34eab640 1 --2- 172.21.15.114:0/3605413902 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f30100990 0x7f5f30196800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:59.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.334+0000 7f5f356ac640 1 --2- 172.21.15.114:0/3605413902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f30105110 0x7f5f3019ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:59.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.334+0000 7f5f34eab640 1 -- 172.21.15.114:0/3605413902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f30105110 msgr2=0x7f5f3019ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:59.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.334+0000 7f5f34eab640 1 --2- 172.21.15.114:0/3605413902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f30105110 0x7f5f3019ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:59.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.334+0000 7f5f34eab640 1 -- 172.21.15.114:0/3605413902 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f28009900 con 0x7f5f300fffe0 2024-09-13T18:20:59.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.334+0000 7f5f356ac640 1 --2- 172.21.15.114:0/3605413902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f30105110 0x7f5f3019ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:20:59.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.334+0000 7f5f34eab640 1 --2- 172.21.15.114:0/3605413902 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f300fffe0 0x7f5f301962c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5f2400b4a0 tx=0x7f5f2400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:59.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.334+0000 7f5f2dffb640 1 -- 172.21.15.114:0/3605413902 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f24010610 con 0x7f5f300fffe0 2024-09-13T18:20:59.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.334+0000 7f5f2dffb640 1 -- 172.21.15.114:0/3605413902 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5f24010c00 con 0x7f5f300fffe0 2024-09-13T18:20:59.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.334+0000 7f5f3710f640 1 -- 172.21.15.114:0/3605413902 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f3019b7c0 con 0x7f5f300fffe0 2024-09-13T18:20:59.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.334+0000 7f5f2dffb640 1 -- 172.21.15.114:0/3605413902 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f24017ca0 con 0x7f5f300fffe0 2024-09-13T18:20:59.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.334+0000 7f5f3710f640 1 -- 172.21.15.114:0/3605413902 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f301a3080 con 0x7f5f300fffe0 2024-09-13T18:20:59.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.335+0000 7f5f3710f640 1 -- 172.21.15.114:0/3605413902 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5f30110010 con 0x7f5f300fffe0 2024-09-13T18:20:59.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.336+0000 7f5f2dffb640 1 -- 172.21.15.114:0/3605413902 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f5f24010770 con 0x7f5f300fffe0 2024-09-13T18:20:59.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.337+0000 7f5f2dffb640 1 --2- 172.21.15.114:0/3605413902 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f5f200a1110 0x7f5f200a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:59.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.337+0000 7f5f2ffff640 1 --2- 172.21.15.114:0/3605413902 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f5f200a1110 0x7f5f200a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:59.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.337+0000 7f5f2dffb640 1 -- 172.21.15.114:0/3605413902 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5f240cc700 con 0x7f5f300fffe0 2024-09-13T18:20:59.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.337+0000 7f5f2dffb640 1 --2- 172.21.15.114:0/3605413902 >> [v2:172.21.15.114:6818/1484325044,v1:172.21.15.114:6819/1484325044] conn(0x7f5f200ac1a0 0x7f5f200ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:59.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.337+0000 7f5f2dffb640 1 -- 172.21.15.114:0/3605413902 --> [v2:172.21.15.114:6818/1484325044,v1:172.21.15.114:6819/1484325044] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5f200aed10 con 0x7f5f200ac1a0 2024-09-13T18:20:59.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.337+0000 7f5f2dffb640 1 -- 172.21.15.114:0/3605413902 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f240ced80 con 0x7f5f300fffe0 2024-09-13T18:20:59.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.337+0000 7f5f356ac640 1 --2- 172.21.15.114:0/3605413902 >> [v2:172.21.15.114:6818/1484325044,v1:172.21.15.114:6819/1484325044] conn(0x7f5f200ac1a0 0x7f5f200ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:59.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.337+0000 7f5f2ffff640 1 --2- 172.21.15.114:0/3605413902 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f5f200a1110 0x7f5f200a35d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5f30197870 tx=0x7f5f1c006c90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:59.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.338+0000 7f5f356ac640 1 --2- 172.21.15.114:0/3605413902 >> [v2:172.21.15.114:6818/1484325044,v1:172.21.15.114:6819/1484325044] conn(0x7f5f200ac1a0 0x7f5f200ae640 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:59.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.338+0000 7f5f2dffb640 1 -- 172.21.15.114:0/3605413902 <== osd.6 v2:172.21.15.114:6818/1484325044 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f5f200aed10 con 0x7f5f200ac1a0 2024-09-13T18:20:59.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.351+0000 7f5f3710f640 1 -- 172.21.15.114:0/3605413902 --> [v2:172.21.15.114:6818/1484325044,v1:172.21.15.114:6819/1484325044] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5f30060bb0 con 0x7f5f200ac1a0 2024-09-13T18:20:59.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.352+0000 7f5f2dffb640 1 -- 172.21.15.114:0/3605413902 <== osd.6 v2:172.21.15.114:6818/1484325044 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f5f30060bb0 con 0x7f5f200ac1a0 2024-09-13T18:20:59.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.352+0000 7f5f3710f640 1 -- 172.21.15.114:0/3605413902 >> [v2:172.21.15.114:6818/1484325044,v1:172.21.15.114:6819/1484325044] conn(0x7f5f200ac1a0 msgr2=0x7f5f200ae640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:59.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.352+0000 7f5f3710f640 1 --2- 172.21.15.114:0/3605413902 >> [v2:172.21.15.114:6818/1484325044,v1:172.21.15.114:6819/1484325044] conn(0x7f5f200ac1a0 0x7f5f200ae640 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:59.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.352+0000 7f5f3710f640 1 -- 172.21.15.114:0/3605413902 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f5f200a1110 msgr2=0x7f5f200a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:59.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.352+0000 7f5f3710f640 1 --2- 172.21.15.114:0/3605413902 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f5f200a1110 0x7f5f200a35d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5f30197870 tx=0x7f5f1c006c90 comp rx=0 tx=0).stop 2024-09-13T18:20:59.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.352+0000 7f5f3710f640 1 -- 172.21.15.114:0/3605413902 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f300fffe0 msgr2=0x7f5f301962c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:59.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.352+0000 7f5f3710f640 1 --2- 172.21.15.114:0/3605413902 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f300fffe0 0x7f5f301962c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5f2400b4a0 tx=0x7f5f2400b960 comp rx=0 tx=0).stop 2024-09-13T18:20:59.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.352+0000 7f5f356ac640 1 -- 172.21.15.114:0/3605413902 reap_dead start 2024-09-13T18:20:59.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.352+0000 7f5f3710f640 1 -- 172.21.15.114:0/3605413902 shutdown_connections 2024-09-13T18:20:59.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.352+0000 7f5f3710f640 1 -- 172.21.15.114:0/3605413902 >> 172.21.15.114:0/3605413902 conn(0x7f5f300fbae0 msgr2=0x7f5f300fcab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:59.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.352+0000 7f5f3710f640 1 -- 172.21.15.114:0/3605413902 shutdown_connections 2024-09-13T18:20:59.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:20:59.352+0000 7f5f3710f640 1 -- 172.21.15.114:0/3605413902 wait complete. 2024-09-13T18:20:59.368 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:20:59 smithi134 ceph-mon[34283]: pgmap v307: 129 pgs: 129 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-13T18:20:59.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:20:59 smithi204 ceph-mon[34777]: pgmap v307: 129 pgs: 129 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-13T18:21:00.271 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:00 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:21:00.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:00 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:21:00.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:00 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:21:00.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.819+0000 7f1e12b55640 1 -- 172.21.15.114:0/993611567 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e0c104000 msgr2=0x7f1e0c10bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:00.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.819+0000 7f1e12b55640 1 --2- 172.21.15.114:0/993611567 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e0c104000 0x7f1e0c10bb80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1e04009cd0 tx=0x7f1e0402a8e0 comp rx=0 tx=0).stop 2024-09-13T18:21:00.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.819+0000 7f1e12b55640 1 -- 172.21.15.114:0/993611567 shutdown_connections 2024-09-13T18:21:00.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.819+0000 7f1e12b55640 1 --2- 172.21.15.114:0/993611567 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e0c104000 0x7f1e0c10bb80 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:00.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.819+0000 7f1e12b55640 1 --2- 172.21.15.114:0/993611567 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e0c0ff880 0x7f1e0c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:00.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.819+0000 7f1e12b55640 1 --2- 172.21.15.114:0/993611567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e0c0feed0 0x7f1e0c0ff2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:00.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.819+0000 7f1e12b55640 1 -- 172.21.15.114:0/993611567 >> 172.21.15.114:0/993611567 conn(0x7f1e0c0fa9b0 msgr2=0x7f1e0c0fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:00.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.819+0000 7f1e12b55640 1 -- 172.21.15.114:0/993611567 shutdown_connections 2024-09-13T18:21:00.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.819+0000 7f1e12b55640 1 -- 172.21.15.114:0/993611567 wait complete. 2024-09-13T18:21:00.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.820+0000 7f1e12b55640 1 Processor -- start 2024-09-13T18:21:00.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.820+0000 7f1e12b55640 1 -- start start 2024-09-13T18:21:00.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.820+0000 7f1e12b55640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e0c0feed0 0x7f1e0c1961b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:00.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.820+0000 7f1e12b55640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e0c0ff880 0x7f1e0c1966f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:00.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.820+0000 7f1e12b55640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e0c104000 0x7f1e0c19aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:00.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.820+0000 7f1e12b55640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c19b0f0 con 0x7f1e0c0ff880 2024-09-13T18:21:00.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.820+0000 7f1e12b55640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c19b260 con 0x7f1e0c0feed0 2024-09-13T18:21:00.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.820+0000 7f1e12b55640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c19b3d0 con 0x7f1e0c104000 2024-09-13T18:21:00.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.821+0000 7f1e0bfff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e0c0ff880 0x7f1e0c1966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:00.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.821+0000 7f1e0bfff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e0c0ff880 0x7f1e0c1966f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44872/0 (socket says 172.21.15.114:44872) 2024-09-13T18:21:00.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.821+0000 7f1e0bfff640 1 -- 172.21.15.114:0/3620904207 learned_addr learned my addr 172.21.15.114:0/3620904207 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:00.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.821+0000 7f1e108f1640 1 --2- 172.21.15.114:0/3620904207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e0c0feed0 0x7f1e0c1961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:00.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.821+0000 7f1e110f2640 1 --2- 172.21.15.114:0/3620904207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e0c104000 0x7f1e0c19aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:00.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.821+0000 7f1e0bfff640 1 -- 172.21.15.114:0/3620904207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e0c104000 msgr2=0x7f1e0c19aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:00.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.821+0000 7f1e0bfff640 1 --2- 172.21.15.114:0/3620904207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e0c104000 0x7f1e0c19aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:00.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.821+0000 7f1e0bfff640 1 -- 172.21.15.114:0/3620904207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e0c0feed0 msgr2=0x7f1e0c1961b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:00.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.821+0000 7f1e0bfff640 1 --2- 172.21.15.114:0/3620904207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e0c0feed0 0x7f1e0c1961b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:00.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.821+0000 7f1e0bfff640 1 -- 172.21.15.114:0/3620904207 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e04009900 con 0x7f1e0c0ff880 2024-09-13T18:21:00.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.821+0000 7f1e0bfff640 1 --2- 172.21.15.114:0/3620904207 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e0c0ff880 0x7f1e0c1966f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1df800e9f0 tx=0x7f1df800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:00.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.821+0000 7f1e09ffb640 1 -- 172.21.15.114:0/3620904207 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df8009720 con 0x7f1e0c0ff880 2024-09-13T18:21:00.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.821+0000 7f1e09ffb640 1 -- 172.21.15.114:0/3620904207 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1df800cd80 con 0x7f1e0c0ff880 2024-09-13T18:21:00.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.821+0000 7f1e12b55640 1 -- 172.21.15.114:0/3620904207 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e0c19b6b0 con 0x7f1e0c0ff880 2024-09-13T18:21:00.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.821+0000 7f1e09ffb640 1 -- 172.21.15.114:0/3620904207 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df8010430 con 0x7f1e0c0ff880 2024-09-13T18:21:00.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.821+0000 7f1e12b55640 1 -- 172.21.15.114:0/3620904207 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e0c1a2f70 con 0x7f1e0c0ff880 2024-09-13T18:21:00.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.822+0000 7f1e12b55640 1 -- 172.21.15.114:0/3620904207 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1e0c10ee80 con 0x7f1e0c0ff880 2024-09-13T18:21:00.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.823+0000 7f1e09ffb640 1 -- 172.21.15.114:0/3620904207 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f1df8004f90 con 0x7f1e0c0ff880 2024-09-13T18:21:00.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.824+0000 7f1e09ffb640 1 --2- 172.21.15.114:0/3620904207 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f1dfc0a1190 0x7f1dfc0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:00.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.824+0000 7f1e108f1640 1 --2- 172.21.15.114:0/3620904207 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f1dfc0a1190 0x7f1dfc0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:00.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.824+0000 7f1e09ffb640 1 -- 172.21.15.114:0/3620904207 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1df8014070 con 0x7f1e0c0ff880 2024-09-13T18:21:00.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.824+0000 7f1e09ffb640 1 --2- 172.21.15.114:0/3620904207 >> [v2:172.21.15.204:6824/3239521765,v1:172.21.15.204:6825/3239521765] conn(0x7f1dfc0ac220 0x7f1dfc0ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:00.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.824+0000 7f1e09ffb640 1 -- 172.21.15.114:0/3620904207 --> [v2:172.21.15.204:6824/3239521765,v1:172.21.15.204:6825/3239521765] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1dfc0aed90 con 0x7f1dfc0ac220 2024-09-13T18:21:00.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.824+0000 7f1e09ffb640 1 -- 172.21.15.114:0/3620904207 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1df80cd750 con 0x7f1e0c0ff880 2024-09-13T18:21:00.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.824+0000 7f1e110f2640 1 --2- 172.21.15.114:0/3620904207 >> [v2:172.21.15.204:6824/3239521765,v1:172.21.15.204:6825/3239521765] conn(0x7f1dfc0ac220 0x7f1dfc0ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:00.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.824+0000 7f1e108f1640 1 --2- 172.21.15.114:0/3620904207 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f1dfc0a1190 0x7f1dfc0a3650 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1e00009bd0 tx=0x7f1e00005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:00.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.825+0000 7f1e110f2640 1 --2- 172.21.15.114:0/3620904207 >> [v2:172.21.15.204:6824/3239521765,v1:172.21.15.204:6825/3239521765] conn(0x7f1dfc0ac220 0x7f1dfc0ae6c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:00.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.826+0000 7f1e09ffb640 1 -- 172.21.15.114:0/3620904207 <== osd.11 v2:172.21.15.204:6824/3239521765 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f1dfc0aed90 con 0x7f1dfc0ac220 2024-09-13T18:21:00.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.839+0000 7f1e12b55640 1 -- 172.21.15.114:0/3620904207 --> [v2:172.21.15.204:6824/3239521765,v1:172.21.15.204:6825/3239521765] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1e0c0fc830 con 0x7f1dfc0ac220 2024-09-13T18:21:00.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.839+0000 7f1e09ffb640 1 -- 172.21.15.114:0/3620904207 <== osd.11 v2:172.21.15.204:6824/3239521765 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f1e0c0fc830 con 0x7f1dfc0ac220 2024-09-13T18:21:00.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.840+0000 7f1e12b55640 1 -- 172.21.15.114:0/3620904207 >> [v2:172.21.15.204:6824/3239521765,v1:172.21.15.204:6825/3239521765] conn(0x7f1dfc0ac220 msgr2=0x7f1dfc0ae6c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:00.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.840+0000 7f1e12b55640 1 --2- 172.21.15.114:0/3620904207 >> [v2:172.21.15.204:6824/3239521765,v1:172.21.15.204:6825/3239521765] conn(0x7f1dfc0ac220 0x7f1dfc0ae6c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:00.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.840+0000 7f1e12b55640 1 -- 172.21.15.114:0/3620904207 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f1dfc0a1190 msgr2=0x7f1dfc0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:00.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.840+0000 7f1e12b55640 1 --2- 172.21.15.114:0/3620904207 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f1dfc0a1190 0x7f1dfc0a3650 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1e00009bd0 tx=0x7f1e00005f90 comp rx=0 tx=0).stop 2024-09-13T18:21:00.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.840+0000 7f1e12b55640 1 -- 172.21.15.114:0/3620904207 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e0c0ff880 msgr2=0x7f1e0c1966f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:00.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.840+0000 7f1e12b55640 1 --2- 172.21.15.114:0/3620904207 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e0c0ff880 0x7f1e0c1966f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1df800e9f0 tx=0x7f1df800edf0 comp rx=0 tx=0).stop 2024-09-13T18:21:00.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.840+0000 7f1e110f2640 1 -- 172.21.15.114:0/3620904207 reap_dead start 2024-09-13T18:21:00.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.840+0000 7f1e12b55640 1 -- 172.21.15.114:0/3620904207 shutdown_connections 2024-09-13T18:21:00.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.840+0000 7f1e12b55640 1 -- 172.21.15.114:0/3620904207 >> 172.21.15.114:0/3620904207 conn(0x7f1e0c0fa9b0 msgr2=0x7f1e0c0681e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:00.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.840+0000 7f1e12b55640 1 -- 172.21.15.114:0/3620904207 shutdown_connections 2024-09-13T18:21:00.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:00.840+0000 7f1e12b55640 1 -- 172.21.15.114:0/3620904207 wait complete. 2024-09-13T18:21:01.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:01 smithi114 ceph-mon[30897]: pgmap v308: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-13T18:21:01.368 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:01 smithi134 ceph-mon[34283]: pgmap v308: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-13T18:21:01.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:01 smithi204 ceph-mon[34777]: pgmap v308: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-13T18:21:01.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.844+0000 7f852f760640 1 -- 172.21.15.114:0/2377255993 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85280ff880 msgr2=0x7f85281038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:01.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.844+0000 7f852f760640 1 --2- 172.21.15.114:0/2377255993 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85280ff880 0x7f85281038d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8518009930 tx=0x7f851802a420 comp rx=0 tx=0).stop 2024-09-13T18:21:01.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.844+0000 7f852f760640 1 -- 172.21.15.114:0/2377255993 shutdown_connections 2024-09-13T18:21:01.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.844+0000 7f852f760640 1 --2- 172.21.15.114:0/2377255993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8528104000 0x7f852810bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:01.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.844+0000 7f852f760640 1 --2- 172.21.15.114:0/2377255993 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85280ff880 0x7f85281038d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:01.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.844+0000 7f852f760640 1 --2- 172.21.15.114:0/2377255993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85280feed0 0x7f85280ff2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:01.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.844+0000 7f852f760640 1 -- 172.21.15.114:0/2377255993 >> 172.21.15.114:0/2377255993 conn(0x7f85280fa9b0 msgr2=0x7f85280fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:01.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.844+0000 7f852f760640 1 -- 172.21.15.114:0/2377255993 shutdown_connections 2024-09-13T18:21:01.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.844+0000 7f852f760640 1 -- 172.21.15.114:0/2377255993 wait complete. 2024-09-13T18:21:01.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.844+0000 7f852f760640 1 Processor -- start 2024-09-13T18:21:01.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.844+0000 7f852f760640 1 -- start start 2024-09-13T18:21:01.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.845+0000 7f852f760640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85280feed0 0x7f8528194dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:01.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.845+0000 7f852f760640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85280ff880 0x7f8528195310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:01.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.845+0000 7f852f760640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8528104000 0x7f8528195850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:01.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.845+0000 7f852f760640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85281a4580 con 0x7f85280ff880 2024-09-13T18:21:01.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.845+0000 7f852f760640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85281a46c0 con 0x7f8528104000 2024-09-13T18:21:01.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.845+0000 7f852f760640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85281a4880 con 0x7f85280feed0 2024-09-13T18:21:01.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.845+0000 7f852ccfb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85280ff880 0x7f8528195310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:01.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.845+0000 7f852d4fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85280feed0 0x7f8528194dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:01.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.845+0000 7f852ccfb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85280ff880 0x7f8528195310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44894/0 (socket says 172.21.15.114:44894) 2024-09-13T18:21:01.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.845+0000 7f852ccfb640 1 -- 172.21.15.114:0/4192670075 learned_addr learned my addr 172.21.15.114:0/4192670075 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:01.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.845+0000 7f852dcfd640 1 --2- 172.21.15.114:0/4192670075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8528104000 0x7f8528195850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:01.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.845+0000 7f852ccfb640 1 -- 172.21.15.114:0/4192670075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85280feed0 msgr2=0x7f8528194dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:01.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.845+0000 7f852ccfb640 1 --2- 172.21.15.114:0/4192670075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85280feed0 0x7f8528194dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:01.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.845+0000 7f852ccfb640 1 -- 172.21.15.114:0/4192670075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8528104000 msgr2=0x7f8528195850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:01.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.845+0000 7f852ccfb640 1 --2- 172.21.15.114:0/4192670075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8528104000 0x7f8528195850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:01.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.845+0000 7f852ccfb640 1 -- 172.21.15.114:0/4192670075 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8518009560 con 0x7f85280ff880 2024-09-13T18:21:01.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.845+0000 7f852ccfb640 1 --2- 172.21.15.114:0/4192670075 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85280ff880 0x7f8528195310 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8518005ab0 tx=0x7f8518032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:01.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.845+0000 7f851e7fc640 1 -- 172.21.15.114:0/4192670075 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8518032e10 con 0x7f85280ff880 2024-09-13T18:21:01.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.845+0000 7f851e7fc640 1 -- 172.21.15.114:0/4192670075 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f85180046c0 con 0x7f85280ff880 2024-09-13T18:21:01.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.845+0000 7f851e7fc640 1 -- 172.21.15.114:0/4192670075 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8518003830 con 0x7f85280ff880 2024-09-13T18:21:01.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.845+0000 7f852f760640 1 -- 172.21.15.114:0/4192670075 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85281a4b00 con 0x7f85280ff880 2024-09-13T18:21:01.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.846+0000 7f852f760640 1 -- 172.21.15.114:0/4192670075 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85281a4fa0 con 0x7f85280ff880 2024-09-13T18:21:01.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.846+0000 7f852f760640 1 -- 172.21.15.114:0/4192670075 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f85281003b0 con 0x7f85280ff880 2024-09-13T18:21:01.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.848+0000 7f851e7fc640 1 -- 172.21.15.114:0/4192670075 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f8518004830 con 0x7f85280ff880 2024-09-13T18:21:01.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.848+0000 7f851e7fc640 1 --2- 172.21.15.114:0/4192670075 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f85140a1190 0x7f85140a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:01.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.848+0000 7f852d4fc640 1 --2- 172.21.15.114:0/4192670075 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f85140a1190 0x7f85140a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:01.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.848+0000 7f851e7fc640 1 -- 172.21.15.114:0/4192670075 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f85180edc30 con 0x7f85280ff880 2024-09-13T18:21:01.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.849+0000 7f851e7fc640 1 --2- 172.21.15.114:0/4192670075 >> [v2:172.21.15.114:6810/2904204254,v1:172.21.15.114:6811/2904204254] conn(0x7f85140ac220 0x7f85140ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:01.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.849+0000 7f851e7fc640 1 -- 172.21.15.114:0/4192670075 --> [v2:172.21.15.114:6810/2904204254,v1:172.21.15.114:6811/2904204254] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f85140aed90 con 0x7f85140ac220 2024-09-13T18:21:01.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.849+0000 7f851e7fc640 1 -- 172.21.15.114:0/4192670075 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85180ee040 con 0x7f85280ff880 2024-09-13T18:21:01.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.849+0000 7f852dcfd640 1 --2- 172.21.15.114:0/4192670075 >> [v2:172.21.15.114:6810/2904204254,v1:172.21.15.114:6811/2904204254] conn(0x7f85140ac220 0x7f85140ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:01.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.849+0000 7f852d4fc640 1 --2- 172.21.15.114:0/4192670075 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f85140a1190 0x7f85140a3650 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8524005d00 tx=0x7f852400b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:01.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.849+0000 7f852dcfd640 1 --2- 172.21.15.114:0/4192670075 >> [v2:172.21.15.114:6810/2904204254,v1:172.21.15.114:6811/2904204254] conn(0x7f85140ac220 0x7f85140ae6c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:01.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.850+0000 7f851e7fc640 1 -- 172.21.15.114:0/4192670075 <== osd.3 v2:172.21.15.114:6810/2904204254 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f85140aed90 con 0x7f85140ac220 2024-09-13T18:21:01.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.863+0000 7f852f760640 1 -- 172.21.15.114:0/4192670075 --> [v2:172.21.15.114:6810/2904204254,v1:172.21.15.114:6811/2904204254] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8528060b30 con 0x7f85140ac220 2024-09-13T18:21:01.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.863+0000 7f851e7fc640 1 -- 172.21.15.114:0/4192670075 <== osd.3 v2:172.21.15.114:6810/2904204254 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f8528060b30 con 0x7f85140ac220 2024-09-13T18:21:01.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.863+0000 7f852f760640 1 -- 172.21.15.114:0/4192670075 >> [v2:172.21.15.114:6810/2904204254,v1:172.21.15.114:6811/2904204254] conn(0x7f85140ac220 msgr2=0x7f85140ae6c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:01.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.863+0000 7f852f760640 1 --2- 172.21.15.114:0/4192670075 >> [v2:172.21.15.114:6810/2904204254,v1:172.21.15.114:6811/2904204254] conn(0x7f85140ac220 0x7f85140ae6c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:01.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.863+0000 7f852f760640 1 -- 172.21.15.114:0/4192670075 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f85140a1190 msgr2=0x7f85140a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:01.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.863+0000 7f852f760640 1 --2- 172.21.15.114:0/4192670075 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f85140a1190 0x7f85140a3650 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8524005d00 tx=0x7f852400b410 comp rx=0 tx=0).stop 2024-09-13T18:21:01.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.863+0000 7f852f760640 1 -- 172.21.15.114:0/4192670075 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85280ff880 msgr2=0x7f8528195310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:01.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.863+0000 7f852f760640 1 --2- 172.21.15.114:0/4192670075 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85280ff880 0x7f8528195310 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8518005ab0 tx=0x7f8518032c20 comp rx=0 tx=0).stop 2024-09-13T18:21:01.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.863+0000 7f852dcfd640 1 -- 172.21.15.114:0/4192670075 reap_dead start 2024-09-13T18:21:01.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.863+0000 7f852f760640 1 -- 172.21.15.114:0/4192670075 shutdown_connections 2024-09-13T18:21:01.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.863+0000 7f852f760640 1 -- 172.21.15.114:0/4192670075 >> 172.21.15.114:0/4192670075 conn(0x7f85280fa9b0 msgr2=0x7f85280fbf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:01.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.863+0000 7f852f760640 1 -- 172.21.15.114:0/4192670075 shutdown_connections 2024-09-13T18:21:01.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:01.863+0000 7f852f760640 1 -- 172.21.15.114:0/4192670075 wait complete. 2024-09-13T18:21:02.341 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:02 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:21:02.341 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:02 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:21:02.341 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:02 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:21:02.341 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:02 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:21:02.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:02 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:21:02.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:02 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:21:02.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:02 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:21:02.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:02 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:21:02.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.338+0000 7ffa277e9640 1 -- 172.21.15.114:0/4104634475 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa20101910 msgr2=0x7ffa20109e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:02.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.338+0000 7ffa277e9640 1 --2- 172.21.15.114:0/4104634475 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa20101910 0x7ffa20109e00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ffa10009900 tx=0x7ffa1002a480 comp rx=0 tx=0).stop 2024-09-13T18:21:02.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.338+0000 7ffa277e9640 1 -- 172.21.15.114:0/4104634475 shutdown_connections 2024-09-13T18:21:02.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.338+0000 7ffa277e9640 1 --2- 172.21.15.114:0/4104634475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa2010a390 0x7ffa2010c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:02.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.338+0000 7ffa277e9640 1 --2- 172.21.15.114:0/4104634475 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa20101910 0x7ffa20109e00 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:02.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.338+0000 7ffa277e9640 1 --2- 172.21.15.114:0/4104634475 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa20100ff0 0x7ffa201013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:02.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.338+0000 7ffa277e9640 1 -- 172.21.15.114:0/4104634475 >> 172.21.15.114:0/4104634475 conn(0x7ffa200fa9b0 msgr2=0x7ffa200fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:02.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.338+0000 7ffa277e9640 1 -- 172.21.15.114:0/4104634475 shutdown_connections 2024-09-13T18:21:02.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.338+0000 7ffa277e9640 1 -- 172.21.15.114:0/4104634475 wait complete. 2024-09-13T18:21:02.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.339+0000 7ffa277e9640 1 Processor -- start 2024-09-13T18:21:02.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.339+0000 7ffa277e9640 1 -- start start 2024-09-13T18:21:02.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.340+0000 7ffa277e9640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa20100ff0 0x7ffa20196260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:02.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.340+0000 7ffa277e9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa20101910 0x7ffa201967a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:02.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.340+0000 7ffa277e9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa2010a390 0x7ffa2019ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:02.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.340+0000 7ffa277e9640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2019b150 con 0x7ffa20100ff0 2024-09-13T18:21:02.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.340+0000 7ffa277e9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2019b2c0 con 0x7ffa20101910 2024-09-13T18:21:02.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.340+0000 7ffa277e9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2019b430 con 0x7ffa2010a390 2024-09-13T18:21:02.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.340+0000 7ffa25585640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa20100ff0 0x7ffa20196260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:02.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.340+0000 7ffa25585640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa20100ff0 0x7ffa20196260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44910/0 (socket says 172.21.15.114:44910) 2024-09-13T18:21:02.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.340+0000 7ffa25585640 1 -- 172.21.15.114:0/1521277622 learned_addr learned my addr 172.21.15.114:0/1521277622 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:02.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.340+0000 7ffa24d84640 1 --2- 172.21.15.114:0/1521277622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa20101910 0x7ffa201967a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:02.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.340+0000 7ffa25d86640 1 --2- 172.21.15.114:0/1521277622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa2010a390 0x7ffa2019ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:02.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.340+0000 7ffa25585640 1 -- 172.21.15.114:0/1521277622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa2010a390 msgr2=0x7ffa2019ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:02.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.340+0000 7ffa25585640 1 --2- 172.21.15.114:0/1521277622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa2010a390 0x7ffa2019ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:02.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.340+0000 7ffa25585640 1 -- 172.21.15.114:0/1521277622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa20101910 msgr2=0x7ffa201967a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:02.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.340+0000 7ffa25585640 1 --2- 172.21.15.114:0/1521277622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa20101910 0x7ffa201967a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:02.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.340+0000 7ffa25585640 1 -- 172.21.15.114:0/1521277622 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa10009560 con 0x7ffa20100ff0 2024-09-13T18:21:02.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.340+0000 7ffa25585640 1 --2- 172.21.15.114:0/1521277622 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa20100ff0 0x7ffa20196260 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c00b4a0 tx=0x7ffa1c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:02.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.340+0000 7ffa167fc640 1 -- 172.21.15.114:0/1521277622 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c010610 con 0x7ffa20100ff0 2024-09-13T18:21:02.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.340+0000 7ffa277e9640 1 -- 172.21.15.114:0/1521277622 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa2019b710 con 0x7ffa20100ff0 2024-09-13T18:21:02.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.340+0000 7ffa167fc640 1 -- 172.21.15.114:0/1521277622 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffa1c010c00 con 0x7ffa20100ff0 2024-09-13T18:21:02.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.340+0000 7ffa167fc640 1 -- 172.21.15.114:0/1521277622 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c017ca0 con 0x7ffa20100ff0 2024-09-13T18:21:02.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.341+0000 7ffa277e9640 1 -- 172.21.15.114:0/1521277622 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa201a2fd0 con 0x7ffa20100ff0 2024-09-13T18:21:02.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.341+0000 7ffa277e9640 1 -- 172.21.15.114:0/1521277622 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ffa2010ff90 con 0x7ffa20100ff0 2024-09-13T18:21:02.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.343+0000 7ffa167fc640 1 -- 172.21.15.114:0/1521277622 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7ffa1c010770 con 0x7ffa20100ff0 2024-09-13T18:21:02.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.343+0000 7ffa167fc640 1 --2- 172.21.15.114:0/1521277622 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7ffa0c0a1110 0x7ffa0c0a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:02.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.343+0000 7ffa24d84640 1 --2- 172.21.15.114:0/1521277622 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7ffa0c0a1110 0x7ffa0c0a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:02.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.343+0000 7ffa167fc640 1 -- 172.21.15.114:0/1521277622 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ffa1c0cc670 con 0x7ffa20100ff0 2024-09-13T18:21:02.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.344+0000 7ffa167fc640 1 --2- 172.21.15.114:0/1521277622 >> [v2:172.21.15.204:6816/858331805,v1:172.21.15.204:6817/858331805] conn(0x7ffa0c0ac120 0x7ffa0c0ae5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:02.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.344+0000 7ffa167fc640 1 -- 172.21.15.114:0/1521277622 --> [v2:172.21.15.204:6816/858331805,v1:172.21.15.204:6817/858331805] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffa0c0aec90 con 0x7ffa0c0ac120 2024-09-13T18:21:02.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.344+0000 7ffa167fc640 1 -- 172.21.15.114:0/1521277622 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa1c0cca80 con 0x7ffa20100ff0 2024-09-13T18:21:02.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.344+0000 7ffa25d86640 1 --2- 172.21.15.114:0/1521277622 >> [v2:172.21.15.204:6816/858331805,v1:172.21.15.204:6817/858331805] conn(0x7ffa0c0ac120 0x7ffa0c0ae5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:02.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.344+0000 7ffa24d84640 1 --2- 172.21.15.114:0/1521277622 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7ffa0c0a1110 0x7ffa0c0a35d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ffa10006010 tx=0x7ffa10005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:02.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.344+0000 7ffa25d86640 1 --2- 172.21.15.114:0/1521277622 >> [v2:172.21.15.204:6816/858331805,v1:172.21.15.204:6817/858331805] conn(0x7ffa0c0ac120 0x7ffa0c0ae5c0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:02.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.345+0000 7ffa167fc640 1 -- 172.21.15.114:0/1521277622 <== osd.8 v2:172.21.15.204:6816/858331805 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7ffa0c0aec90 con 0x7ffa0c0ac120 2024-09-13T18:21:02.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.358+0000 7ffa277e9640 1 -- 172.21.15.114:0/1521277622 --> [v2:172.21.15.204:6816/858331805,v1:172.21.15.204:6817/858331805] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ffa20068720 con 0x7ffa0c0ac120 2024-09-13T18:21:02.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.358+0000 7ffa167fc640 1 -- 172.21.15.114:0/1521277622 <== osd.8 v2:172.21.15.204:6816/858331805 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ffa20068720 con 0x7ffa0c0ac120 2024-09-13T18:21:02.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.359+0000 7ffa277e9640 1 -- 172.21.15.114:0/1521277622 >> [v2:172.21.15.204:6816/858331805,v1:172.21.15.204:6817/858331805] conn(0x7ffa0c0ac120 msgr2=0x7ffa0c0ae5c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:02.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.359+0000 7ffa277e9640 1 --2- 172.21.15.114:0/1521277622 >> [v2:172.21.15.204:6816/858331805,v1:172.21.15.204:6817/858331805] conn(0x7ffa0c0ac120 0x7ffa0c0ae5c0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:02.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.359+0000 7ffa277e9640 1 -- 172.21.15.114:0/1521277622 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7ffa0c0a1110 msgr2=0x7ffa0c0a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:02.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.359+0000 7ffa277e9640 1 --2- 172.21.15.114:0/1521277622 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7ffa0c0a1110 0x7ffa0c0a35d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ffa10006010 tx=0x7ffa10005a20 comp rx=0 tx=0).stop 2024-09-13T18:21:02.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.359+0000 7ffa277e9640 1 -- 172.21.15.114:0/1521277622 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa20100ff0 msgr2=0x7ffa20196260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:02.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.359+0000 7ffa277e9640 1 --2- 172.21.15.114:0/1521277622 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa20100ff0 0x7ffa20196260 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c00b4a0 tx=0x7ffa1c00b960 comp rx=0 tx=0).stop 2024-09-13T18:21:02.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.359+0000 7ffa25d86640 1 -- 172.21.15.114:0/1521277622 reap_dead start 2024-09-13T18:21:02.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.359+0000 7ffa277e9640 1 -- 172.21.15.114:0/1521277622 shutdown_connections 2024-09-13T18:21:02.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.359+0000 7ffa277e9640 1 -- 172.21.15.114:0/1521277622 >> 172.21.15.114:0/1521277622 conn(0x7ffa200fa9b0 msgr2=0x7ffa200fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:02.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.359+0000 7ffa277e9640 1 -- 172.21.15.114:0/1521277622 shutdown_connections 2024-09-13T18:21:02.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:02.359+0000 7ffa277e9640 1 -- 172.21.15.114:0/1521277622 wait complete. 2024-09-13T18:21:02.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:02 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:21:02.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:02 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:21:02.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:02 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:21:02.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:02 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:21:03.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:03 smithi134 ceph-mon[34283]: pgmap v309: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-13T18:21:03.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:03 smithi204 ceph-mon[34777]: pgmap v309: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-13T18:21:03.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:03 smithi114 ceph-mon[30897]: pgmap v309: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-13T18:21:04.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.675+0000 7f2765dba640 1 -- 172.21.15.114:0/3169769246 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f276010a360 msgr2=0x7f276010c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:04.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.675+0000 7f2765dba640 1 --2- 172.21.15.114:0/3169769246 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f276010a360 0x7f276010c800 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2758009ca0 tx=0x7f275802a950 comp rx=0 tx=0).stop 2024-09-13T18:21:04.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.675+0000 7f2765dba640 1 -- 172.21.15.114:0/3169769246 shutdown_connections 2024-09-13T18:21:04.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.675+0000 7f2765dba640 1 --2- 172.21.15.114:0/3169769246 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f276010a360 0x7f276010c800 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:04.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.675+0000 7f2765dba640 1 --2- 172.21.15.114:0/3169769246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2760101850 0x7f2760109d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:04.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.675+0000 7f2765dba640 1 --2- 172.21.15.114:0/3169769246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2760100f30 0x7f2760101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:04.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.675+0000 7f2765dba640 1 -- 172.21.15.114:0/3169769246 >> 172.21.15.114:0/3169769246 conn(0x7f27600fa9b0 msgr2=0x7f27600fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:04.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.675+0000 7f2765dba640 1 -- 172.21.15.114:0/3169769246 shutdown_connections 2024-09-13T18:21:04.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.675+0000 7f2765dba640 1 -- 172.21.15.114:0/3169769246 wait complete. 2024-09-13T18:21:04.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.675+0000 7f2765dba640 1 Processor -- start 2024-09-13T18:21:04.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.675+0000 7f2765dba640 1 -- start start 2024-09-13T18:21:04.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.676+0000 7f2765dba640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2760100f30 0x7f27600feb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:04.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.676+0000 7f2765dba640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2760101850 0x7f27600ff0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:04.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.676+0000 7f2765dba640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f276010a360 0x7f27600ff610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:04.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.676+0000 7f2765dba640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27601a4670 con 0x7f2760101850 2024-09-13T18:21:04.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.676+0000 7f2765dba640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27601a47e0 con 0x7f276010a360 2024-09-13T18:21:04.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.676+0000 7f2765dba640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27601a49a0 con 0x7f2760100f30 2024-09-13T18:21:04.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.676+0000 7f2756ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2760101850 0x7f27600ff0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:04.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.676+0000 7f2756ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2760101850 0x7f27600ff0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44888/0 (socket says 172.21.15.114:44888) 2024-09-13T18:21:04.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.676+0000 7f2756ffd640 1 -- 172.21.15.114:0/2401961067 learned_addr learned my addr 172.21.15.114:0/2401961067 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:04.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.676+0000 7f275f7fe640 1 --2- 172.21.15.114:0/2401961067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2760100f30 0x7f27600feb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:04.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.676+0000 7f275ffff640 1 --2- 172.21.15.114:0/2401961067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f276010a360 0x7f27600ff610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:04.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.676+0000 7f2756ffd640 1 -- 172.21.15.114:0/2401961067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2760100f30 msgr2=0x7f27600feb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:04.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.676+0000 7f2756ffd640 1 --2- 172.21.15.114:0/2401961067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2760100f30 0x7f27600feb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:04.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.676+0000 7f2756ffd640 1 -- 172.21.15.114:0/2401961067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f276010a360 msgr2=0x7f27600ff610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:04.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.676+0000 7f2756ffd640 1 --2- 172.21.15.114:0/2401961067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f276010a360 0x7f27600ff610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:04.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.676+0000 7f2756ffd640 1 -- 172.21.15.114:0/2401961067 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2758009900 con 0x7f2760101850 2024-09-13T18:21:04.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.676+0000 7f2756ffd640 1 --2- 172.21.15.114:0/2401961067 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2760101850 0x7f27600ff0d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f274800b3f0 tx=0x7f274800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:04.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.677+0000 7f275d7fa640 1 -- 172.21.15.114:0/2401961067 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2748011830 con 0x7f2760101850 2024-09-13T18:21:04.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.677+0000 7f2765dba640 1 -- 172.21.15.114:0/2401961067 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27601a4c80 con 0x7f2760101850 2024-09-13T18:21:04.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.677+0000 7f275d7fa640 1 -- 172.21.15.114:0/2401961067 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2748011e20 con 0x7f2760101850 2024-09-13T18:21:04.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.677+0000 7f275d7fa640 1 -- 172.21.15.114:0/2401961067 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2748010df0 con 0x7f2760101850 2024-09-13T18:21:04.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.677+0000 7f2765dba640 1 -- 172.21.15.114:0/2401961067 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27601a5180 con 0x7f2760101850 2024-09-13T18:21:04.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.678+0000 7f2765dba640 1 -- 172.21.15.114:0/2401961067 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f276010ff50 con 0x7f2760101850 2024-09-13T18:21:04.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.679+0000 7f275d7fa640 1 -- 172.21.15.114:0/2401961067 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f2748011990 con 0x7f2760101850 2024-09-13T18:21:04.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.679+0000 7f275d7fa640 1 --2- 172.21.15.114:0/2401961067 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f274c0a1110 0x7f274c0a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:04.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.679+0000 7f275f7fe640 1 --2- 172.21.15.114:0/2401961067 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f274c0a1110 0x7f274c0a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:04.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.680+0000 7f275d7fa640 1 -- 172.21.15.114:0/2401961067 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f27480ccda0 con 0x7f2760101850 2024-09-13T18:21:04.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.680+0000 7f275d7fa640 1 --2- 172.21.15.114:0/2401961067 >> [v2:172.21.15.134:6824/3073513468,v1:172.21.15.134:6825/3073513468] conn(0x7f274c0ac1a0 0x7f274c0ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:04.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.680+0000 7f275d7fa640 1 -- 172.21.15.114:0/2401961067 --> [v2:172.21.15.134:6824/3073513468,v1:172.21.15.134:6825/3073513468] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f274c0aed10 con 0x7f274c0ac1a0 2024-09-13T18:21:04.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.680+0000 7f275d7fa640 1 -- 172.21.15.114:0/2401961067 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27480cd1b0 con 0x7f2760101850 2024-09-13T18:21:04.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.680+0000 7f275f7fe640 1 --2- 172.21.15.114:0/2401961067 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f274c0a1110 0x7f274c0a35d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2750009890 tx=0x7f2750008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:04.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.680+0000 7f275ffff640 1 --2- 172.21.15.114:0/2401961067 >> [v2:172.21.15.134:6824/3073513468,v1:172.21.15.134:6825/3073513468] conn(0x7f274c0ac1a0 0x7f274c0ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:04.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.680+0000 7f275ffff640 1 --2- 172.21.15.114:0/2401961067 >> [v2:172.21.15.134:6824/3073513468,v1:172.21.15.134:6825/3073513468] conn(0x7f274c0ac1a0 0x7f274c0ae640 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:04.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.681+0000 7f275d7fa640 1 -- 172.21.15.114:0/2401961067 <== osd.10 v2:172.21.15.134:6824/3073513468 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f274c0aed10 con 0x7f274c0ac1a0 2024-09-13T18:21:04.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.694+0000 7f2765dba640 1 -- 172.21.15.114:0/2401961067 --> [v2:172.21.15.134:6824/3073513468,v1:172.21.15.134:6825/3073513468] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2760060b30 con 0x7f274c0ac1a0 2024-09-13T18:21:04.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.695+0000 7f275d7fa640 1 -- 172.21.15.114:0/2401961067 <== osd.10 v2:172.21.15.134:6824/3073513468 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f2760060b30 con 0x7f274c0ac1a0 2024-09-13T18:21:04.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.695+0000 7f2765dba640 1 -- 172.21.15.114:0/2401961067 >> [v2:172.21.15.134:6824/3073513468,v1:172.21.15.134:6825/3073513468] conn(0x7f274c0ac1a0 msgr2=0x7f274c0ae640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:04.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.695+0000 7f2765dba640 1 --2- 172.21.15.114:0/2401961067 >> [v2:172.21.15.134:6824/3073513468,v1:172.21.15.134:6825/3073513468] conn(0x7f274c0ac1a0 0x7f274c0ae640 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:04.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.695+0000 7f2765dba640 1 -- 172.21.15.114:0/2401961067 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f274c0a1110 msgr2=0x7f274c0a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:04.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.695+0000 7f2765dba640 1 --2- 172.21.15.114:0/2401961067 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f274c0a1110 0x7f274c0a35d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2750009890 tx=0x7f2750008040 comp rx=0 tx=0).stop 2024-09-13T18:21:04.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.695+0000 7f2765dba640 1 -- 172.21.15.114:0/2401961067 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2760101850 msgr2=0x7f27600ff0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:04.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.695+0000 7f2765dba640 1 --2- 172.21.15.114:0/2401961067 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2760101850 0x7f27600ff0d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f274800b3f0 tx=0x7f274800b8b0 comp rx=0 tx=0).stop 2024-09-13T18:21:04.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.695+0000 7f275ffff640 1 -- 172.21.15.114:0/2401961067 reap_dead start 2024-09-13T18:21:04.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.695+0000 7f2765dba640 1 -- 172.21.15.114:0/2401961067 shutdown_connections 2024-09-13T18:21:04.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.695+0000 7f2765dba640 1 -- 172.21.15.114:0/2401961067 >> 172.21.15.114:0/2401961067 conn(0x7f27600fa9b0 msgr2=0x7f27600fc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:04.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.695+0000 7f2765dba640 1 -- 172.21.15.114:0/2401961067 shutdown_connections 2024-09-13T18:21:04.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:04.695+0000 7f2765dba640 1 -- 172.21.15.114:0/2401961067 wait complete. 2024-09-13T18:21:05.114 INFO:teuthology.orchestra.run.smithi114.stdout:197568495665 2024-09-13T18:21:05.114 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-13T18:21:05.192 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:05 smithi114 ceph-mon[30897]: pgmap v310: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-13T18:21:05.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:05 smithi134 ceph-mon[34283]: pgmap v310: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-13T18:21:05.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:05 smithi204 ceph-mon[34777]: pgmap v310: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-13T18:21:05.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.802+0000 7fa3502e8640 1 -- 172.21.15.114:0/1628000318 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa34810a360 msgr2=0x7fa34810c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:05.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.802+0000 7fa3502e8640 1 --2- 172.21.15.114:0/1628000318 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa34810a360 0x7fa34810c800 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa340009ca0 tx=0x7fa34002a950 comp rx=0 tx=0).stop 2024-09-13T18:21:05.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.802+0000 7fa3502e8640 1 -- 172.21.15.114:0/1628000318 shutdown_connections 2024-09-13T18:21:05.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.802+0000 7fa3502e8640 1 --2- 172.21.15.114:0/1628000318 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa34810a360 0x7fa34810c800 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:05.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.802+0000 7fa3502e8640 1 --2- 172.21.15.114:0/1628000318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa348101850 0x7fa348109d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:05.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.802+0000 7fa3502e8640 1 --2- 172.21.15.114:0/1628000318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa348100f30 0x7fa348101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:05.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.802+0000 7fa3502e8640 1 -- 172.21.15.114:0/1628000318 >> 172.21.15.114:0/1628000318 conn(0x7fa3480fa9b0 msgr2=0x7fa3480fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:05.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.802+0000 7fa3502e8640 1 -- 172.21.15.114:0/1628000318 shutdown_connections 2024-09-13T18:21:05.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.802+0000 7fa3502e8640 1 -- 172.21.15.114:0/1628000318 wait complete. 2024-09-13T18:21:05.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.802+0000 7fa3502e8640 1 Processor -- start 2024-09-13T18:21:05.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.802+0000 7fa3502e8640 1 -- start start 2024-09-13T18:21:05.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.803+0000 7fa3502e8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa348100f30 0x7fa3480feb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:05.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.803+0000 7fa3502e8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa348101850 0x7fa3480ff0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:05.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.803+0000 7fa3502e8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa34810a360 0x7fa3480ff610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:05.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.803+0000 7fa3502e8640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3481a4490 con 0x7fa348101850 2024-09-13T18:21:05.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.803+0000 7fa3502e8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3481a4600 con 0x7fa34810a360 2024-09-13T18:21:05.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.803+0000 7fa3502e8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3481a47c0 con 0x7fa348100f30 2024-09-13T18:21:05.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.803+0000 7fa34d883640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa348101850 0x7fa3480ff0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:05.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.803+0000 7fa34d883640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa348101850 0x7fa3480ff0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44914/0 (socket says 172.21.15.114:44914) 2024-09-13T18:21:05.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.803+0000 7fa34d883640 1 -- 172.21.15.114:0/2657579828 learned_addr learned my addr 172.21.15.114:0/2657579828 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:05.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.803+0000 7fa34e084640 1 --2- 172.21.15.114:0/2657579828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa348100f30 0x7fa3480feb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:05.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.803+0000 7fa34e885640 1 --2- 172.21.15.114:0/2657579828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa34810a360 0x7fa3480ff610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:05.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.803+0000 7fa34d883640 1 -- 172.21.15.114:0/2657579828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa348100f30 msgr2=0x7fa3480feb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:05.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.803+0000 7fa34d883640 1 --2- 172.21.15.114:0/2657579828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa348100f30 0x7fa3480feb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:05.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.803+0000 7fa34d883640 1 -- 172.21.15.114:0/2657579828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa34810a360 msgr2=0x7fa3480ff610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:05.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.803+0000 7fa34d883640 1 --2- 172.21.15.114:0/2657579828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa34810a360 0x7fa3480ff610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:05.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.803+0000 7fa34d883640 1 -- 172.21.15.114:0/2657579828 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa340009900 con 0x7fa348101850 2024-09-13T18:21:05.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.803+0000 7fa34e885640 1 --2- 172.21.15.114:0/2657579828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa34810a360 0x7fa3480ff610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:21:05.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.803+0000 7fa34d883640 1 --2- 172.21.15.114:0/2657579828 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa348101850 0x7fa3480ff0d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa33800d750 tx=0x7fa33800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:05.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.803+0000 7fa33f7fe640 1 -- 172.21.15.114:0/2657579828 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa338009a30 con 0x7fa348101850 2024-09-13T18:21:05.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.803+0000 7fa3502e8640 1 -- 172.21.15.114:0/2657579828 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3481a4aa0 con 0x7fa348101850 2024-09-13T18:21:05.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.803+0000 7fa33f7fe640 1 -- 172.21.15.114:0/2657579828 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa3380054b0 con 0x7fa348101850 2024-09-13T18:21:05.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.803+0000 7fa33f7fe640 1 -- 172.21.15.114:0/2657579828 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa338004cf0 con 0x7fa348101850 2024-09-13T18:21:05.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.803+0000 7fa3502e8640 1 -- 172.21.15.114:0/2657579828 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3481a4f50 con 0x7fa348101850 2024-09-13T18:21:05.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.804+0000 7fa3502e8640 1 -- 172.21.15.114:0/2657579828 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa3480687e0 con 0x7fa348101850 2024-09-13T18:21:05.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.806+0000 7fa33f7fe640 1 -- 172.21.15.114:0/2657579828 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fa338009b90 con 0x7fa348101850 2024-09-13T18:21:05.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.806+0000 7fa33f7fe640 1 --2- 172.21.15.114:0/2657579828 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fa3340a1110 0x7fa3340a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:05.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.806+0000 7fa33f7fe640 1 -- 172.21.15.114:0/2657579828 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa3380ccaf0 con 0x7fa348101850 2024-09-13T18:21:05.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.806+0000 7fa34e084640 1 --2- 172.21.15.114:0/2657579828 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fa3340a1110 0x7fa3340a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:05.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.807+0000 7fa33f7fe640 1 --2- 172.21.15.114:0/2657579828 >> [v2:172.21.15.134:6808/1773945333,v1:172.21.15.134:6809/1773945333] conn(0x7fa3340ac1a0 0x7fa3340ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:05.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.807+0000 7fa33f7fe640 1 -- 172.21.15.114:0/2657579828 --> [v2:172.21.15.134:6808/1773945333,v1:172.21.15.134:6809/1773945333] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa3340aed10 con 0x7fa3340ac1a0 2024-09-13T18:21:05.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.807+0000 7fa33f7fe640 1 -- 172.21.15.114:0/2657579828 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3380cce60 con 0x7fa348101850 2024-09-13T18:21:05.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.807+0000 7fa34e084640 1 --2- 172.21.15.114:0/2657579828 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fa3340a1110 0x7fa3340a35d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa3440058c0 tx=0x7fa34400a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:05.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.807+0000 7fa34e885640 1 --2- 172.21.15.114:0/2657579828 >> [v2:172.21.15.134:6808/1773945333,v1:172.21.15.134:6809/1773945333] conn(0x7fa3340ac1a0 0x7fa3340ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:05.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.807+0000 7fa34e885640 1 --2- 172.21.15.114:0/2657579828 >> [v2:172.21.15.134:6808/1773945333,v1:172.21.15.134:6809/1773945333] conn(0x7fa3340ac1a0 0x7fa3340ae640 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:05.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.808+0000 7fa33f7fe640 1 -- 172.21.15.114:0/2657579828 <== osd.4 v2:172.21.15.134:6808/1773945333 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fa3340aed10 con 0x7fa3340ac1a0 2024-09-13T18:21:05.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.821+0000 7fa3502e8640 1 -- 172.21.15.114:0/2657579828 --> [v2:172.21.15.134:6808/1773945333,v1:172.21.15.134:6809/1773945333] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa348060b30 con 0x7fa3340ac1a0 2024-09-13T18:21:05.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.822+0000 7fa33f7fe640 1 -- 172.21.15.114:0/2657579828 <== osd.4 v2:172.21.15.134:6808/1773945333 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fa348060b30 con 0x7fa3340ac1a0 2024-09-13T18:21:05.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.822+0000 7fa3502e8640 1 -- 172.21.15.114:0/2657579828 >> [v2:172.21.15.134:6808/1773945333,v1:172.21.15.134:6809/1773945333] conn(0x7fa3340ac1a0 msgr2=0x7fa3340ae640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:05.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.822+0000 7fa3502e8640 1 --2- 172.21.15.114:0/2657579828 >> [v2:172.21.15.134:6808/1773945333,v1:172.21.15.134:6809/1773945333] conn(0x7fa3340ac1a0 0x7fa3340ae640 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:05.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.822+0000 7fa3502e8640 1 -- 172.21.15.114:0/2657579828 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fa3340a1110 msgr2=0x7fa3340a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:05.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.822+0000 7fa3502e8640 1 --2- 172.21.15.114:0/2657579828 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fa3340a1110 0x7fa3340a35d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa3440058c0 tx=0x7fa34400a2d0 comp rx=0 tx=0).stop 2024-09-13T18:21:05.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.822+0000 7fa3502e8640 1 -- 172.21.15.114:0/2657579828 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa348101850 msgr2=0x7fa3480ff0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:05.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.822+0000 7fa3502e8640 1 --2- 172.21.15.114:0/2657579828 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa348101850 0x7fa3480ff0d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa33800d750 tx=0x7fa33800dc10 comp rx=0 tx=0).stop 2024-09-13T18:21:05.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.822+0000 7fa34e885640 1 -- 172.21.15.114:0/2657579828 reap_dead start 2024-09-13T18:21:05.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.823+0000 7fa3502e8640 1 -- 172.21.15.114:0/2657579828 shutdown_connections 2024-09-13T18:21:05.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.823+0000 7fa3502e8640 1 -- 172.21.15.114:0/2657579828 >> 172.21.15.114:0/2657579828 conn(0x7fa3480fa9b0 msgr2=0x7fa3480fc250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:05.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.823+0000 7fa3502e8640 1 -- 172.21.15.114:0/2657579828 shutdown_connections 2024-09-13T18:21:05.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:05.823+0000 7fa3502e8640 1 -- 172.21.15.114:0/2657579828 wait complete. 2024-09-13T18:21:06.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.364+0000 7fb79bc46640 1 -- 172.21.15.114:0/3829555569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb794104000 msgr2=0x7fb79410bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:06.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.364+0000 7fb79bc46640 1 --2- 172.21.15.114:0/3829555569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb794104000 0x7fb79410bb80 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb78c009cd0 tx=0x7fb78c02a8e0 comp rx=0 tx=0).stop 2024-09-13T18:21:06.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.364+0000 7fb79bc46640 1 -- 172.21.15.114:0/3829555569 shutdown_connections 2024-09-13T18:21:06.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.364+0000 7fb79bc46640 1 --2- 172.21.15.114:0/3829555569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb794104000 0x7fb79410bb80 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:06.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.364+0000 7fb79bc46640 1 --2- 172.21.15.114:0/3829555569 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7940ff880 0x7fb7941038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:06.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.364+0000 7fb79bc46640 1 --2- 172.21.15.114:0/3829555569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7940feed0 0x7fb7940ff2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:06.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.364+0000 7fb79bc46640 1 -- 172.21.15.114:0/3829555569 >> 172.21.15.114:0/3829555569 conn(0x7fb7940fa9b0 msgr2=0x7fb7940fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:06.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.364+0000 7fb79bc46640 1 -- 172.21.15.114:0/3829555569 shutdown_connections 2024-09-13T18:21:06.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.364+0000 7fb79bc46640 1 -- 172.21.15.114:0/3829555569 wait complete. 2024-09-13T18:21:06.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.365+0000 7fb79bc46640 1 Processor -- start 2024-09-13T18:21:06.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.365+0000 7fb79bc46640 1 -- start start 2024-09-13T18:21:06.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.365+0000 7fb79bc46640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7940feed0 0x7fb7941960e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:06.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.365+0000 7fb79bc46640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7940ff880 0x7fb794196620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:06.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.365+0000 7fb79bc46640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb794104000 0x7fb79419aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:06.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.365+0000 7fb79bc46640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb79419b020 con 0x7fb7940ff880 2024-09-13T18:21:06.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.365+0000 7fb79bc46640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb79419b190 con 0x7fb7940feed0 2024-09-13T18:21:06.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.365+0000 7fb79bc46640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb79419b300 con 0x7fb794104000 2024-09-13T18:21:06.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.365+0000 7fb7991e1640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7940ff880 0x7fb794196620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:06.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.365+0000 7fb7991e1640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7940ff880 0x7fb794196620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44930/0 (socket says 172.21.15.114:44930) 2024-09-13T18:21:06.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.365+0000 7fb7991e1640 1 -- 172.21.15.114:0/523876417 learned_addr learned my addr 172.21.15.114:0/523876417 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:06.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.365+0000 7fb7999e2640 1 --2- 172.21.15.114:0/523876417 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7940feed0 0x7fb7941960e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:06.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.366+0000 7fb79a1e3640 1 --2- 172.21.15.114:0/523876417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb794104000 0x7fb79419aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:06.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.366+0000 7fb7991e1640 1 -- 172.21.15.114:0/523876417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb794104000 msgr2=0x7fb79419aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:06.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.366+0000 7fb7991e1640 1 --2- 172.21.15.114:0/523876417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb794104000 0x7fb79419aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:06.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.366+0000 7fb7991e1640 1 -- 172.21.15.114:0/523876417 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7940feed0 msgr2=0x7fb7941960e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:06.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.366+0000 7fb7991e1640 1 --2- 172.21.15.114:0/523876417 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7940feed0 0x7fb7941960e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:06.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.366+0000 7fb7991e1640 1 -- 172.21.15.114:0/523876417 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb78c009900 con 0x7fb7940ff880 2024-09-13T18:21:06.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.366+0000 7fb7991e1640 1 --2- 172.21.15.114:0/523876417 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7940ff880 0x7fb794196620 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb78400c920 tx=0x7fb78400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:06.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.366+0000 7fb78affd640 1 -- 172.21.15.114:0/523876417 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb784007c70 con 0x7fb7940ff880 2024-09-13T18:21:06.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.366+0000 7fb79bc46640 1 -- 172.21.15.114:0/523876417 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb79419b5e0 con 0x7fb7940ff880 2024-09-13T18:21:06.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.366+0000 7fb78affd640 1 -- 172.21.15.114:0/523876417 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb784007dd0 con 0x7fb7940ff880 2024-09-13T18:21:06.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.366+0000 7fb78affd640 1 -- 172.21.15.114:0/523876417 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb784018730 con 0x7fb7940ff880 2024-09-13T18:21:06.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.366+0000 7fb79bc46640 1 -- 172.21.15.114:0/523876417 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7941a2ea0 con 0x7fb7940ff880 2024-09-13T18:21:06.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.367+0000 7fb79bc46640 1 -- 172.21.15.114:0/523876417 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb79410ee80 con 0x7fb7940ff880 2024-09-13T18:21:06.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.368+0000 7fb78affd640 1 -- 172.21.15.114:0/523876417 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fb78400f450 con 0x7fb7940ff880 2024-09-13T18:21:06.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.369+0000 7fb78affd640 1 --2- 172.21.15.114:0/523876417 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb7800a1110 0x7fb7800a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:06.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.369+0000 7fb7999e2640 1 --2- 172.21.15.114:0/523876417 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb7800a1110 0x7fb7800a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:06.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.369+0000 7fb78affd640 1 -- 172.21.15.114:0/523876417 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb7840cd1f0 con 0x7fb7940ff880 2024-09-13T18:21:06.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.369+0000 7fb78affd640 1 --2- 172.21.15.114:0/523876417 >> [v2:172.21.15.134:6816/799246614,v1:172.21.15.134:6817/799246614] conn(0x7fb7800ac1a0 0x7fb7800ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:06.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.369+0000 7fb78affd640 1 -- 172.21.15.114:0/523876417 --> [v2:172.21.15.134:6816/799246614,v1:172.21.15.134:6817/799246614] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb7800aed10 con 0x7fb7800ac1a0 2024-09-13T18:21:06.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.369+0000 7fb78affd640 1 -- 172.21.15.114:0/523876417 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7840cd600 con 0x7fb7940ff880 2024-09-13T18:21:06.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.369+0000 7fb7999e2640 1 --2- 172.21.15.114:0/523876417 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb7800a1110 0x7fb7800a35d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb790005fd0 tx=0x7fb79000b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:06.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.369+0000 7fb79a1e3640 1 --2- 172.21.15.114:0/523876417 >> [v2:172.21.15.134:6816/799246614,v1:172.21.15.134:6817/799246614] conn(0x7fb7800ac1a0 0x7fb7800ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:06.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.370+0000 7fb79a1e3640 1 --2- 172.21.15.114:0/523876417 >> [v2:172.21.15.134:6816/799246614,v1:172.21.15.134:6817/799246614] conn(0x7fb7800ac1a0 0x7fb7800ae640 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:06.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.371+0000 7fb78affd640 1 -- 172.21.15.114:0/523876417 <== osd.7 v2:172.21.15.134:6816/799246614 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fb7800aed10 con 0x7fb7800ac1a0 2024-09-13T18:21:06.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.384+0000 7fb79bc46640 1 -- 172.21.15.114:0/523876417 --> [v2:172.21.15.134:6816/799246614,v1:172.21.15.134:6817/799246614] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb794102cf0 con 0x7fb7800ac1a0 2024-09-13T18:21:06.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.384+0000 7fb78affd640 1 -- 172.21.15.114:0/523876417 <== osd.7 v2:172.21.15.134:6816/799246614 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb794102cf0 con 0x7fb7800ac1a0 2024-09-13T18:21:06.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.385+0000 7fb79bc46640 1 -- 172.21.15.114:0/523876417 >> [v2:172.21.15.134:6816/799246614,v1:172.21.15.134:6817/799246614] conn(0x7fb7800ac1a0 msgr2=0x7fb7800ae640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:06.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.385+0000 7fb79bc46640 1 --2- 172.21.15.114:0/523876417 >> [v2:172.21.15.134:6816/799246614,v1:172.21.15.134:6817/799246614] conn(0x7fb7800ac1a0 0x7fb7800ae640 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:06.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.385+0000 7fb79bc46640 1 -- 172.21.15.114:0/523876417 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb7800a1110 msgr2=0x7fb7800a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:06.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.385+0000 7fb79bc46640 1 --2- 172.21.15.114:0/523876417 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb7800a1110 0x7fb7800a35d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb790005fd0 tx=0x7fb79000b390 comp rx=0 tx=0).stop 2024-09-13T18:21:06.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.385+0000 7fb79bc46640 1 -- 172.21.15.114:0/523876417 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7940ff880 msgr2=0x7fb794196620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:06.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.385+0000 7fb79bc46640 1 --2- 172.21.15.114:0/523876417 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7940ff880 0x7fb794196620 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb78400c920 tx=0x7fb78400cde0 comp rx=0 tx=0).stop 2024-09-13T18:21:06.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.385+0000 7fb79a1e3640 1 -- 172.21.15.114:0/523876417 reap_dead start 2024-09-13T18:21:06.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.385+0000 7fb79bc46640 1 -- 172.21.15.114:0/523876417 shutdown_connections 2024-09-13T18:21:06.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.385+0000 7fb79bc46640 1 -- 172.21.15.114:0/523876417 >> 172.21.15.114:0/523876417 conn(0x7fb7940fa9b0 msgr2=0x7fb7940fbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:06.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.385+0000 7fb79bc46640 1 -- 172.21.15.114:0/523876417 shutdown_connections 2024-09-13T18:21:06.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.385+0000 7fb79bc46640 1 -- 172.21.15.114:0/523876417 wait complete. 2024-09-13T18:21:07.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.969+0000 7f44fb370640 1 -- 172.21.15.114:0/3982916921 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f44f4104000 msgr2=0x7f44f410bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:07.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.969+0000 7f44fb370640 1 --2- 172.21.15.114:0/3982916921 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f44f4104000 0x7f44f410bb80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f44ec009930 tx=0x7f44ec02a420 comp rx=0 tx=0).stop 2024-09-13T18:21:07.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.969+0000 7f44fb370640 1 -- 172.21.15.114:0/3982916921 shutdown_connections 2024-09-13T18:21:07.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.969+0000 7f44fb370640 1 --2- 172.21.15.114:0/3982916921 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f44f4104000 0x7f44f410bb80 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:07.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.969+0000 7f44fb370640 1 --2- 172.21.15.114:0/3982916921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44f40ff880 0x7f44f41038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:07.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.969+0000 7f44fb370640 1 --2- 172.21.15.114:0/3982916921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44f40feed0 0x7f44f40ff2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:07.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.969+0000 7f44fb370640 1 -- 172.21.15.114:0/3982916921 >> 172.21.15.114:0/3982916921 conn(0x7f44f40fa9b0 msgr2=0x7f44f40fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:07.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.969+0000 7f44fb370640 1 -- 172.21.15.114:0/3982916921 shutdown_connections 2024-09-13T18:21:07.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.969+0000 7f44fb370640 1 -- 172.21.15.114:0/3982916921 wait complete. 2024-09-13T18:21:07.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.969+0000 7f44fb370640 1 Processor -- start 2024-09-13T18:21:07.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.970+0000 7f44fb370640 1 -- start start 2024-09-13T18:21:07.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.970+0000 7f44fb370640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44f40feed0 0x7f44f4196f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:07.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.970+0000 7f44fb370640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44f40ff880 0x7f44f41974a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:07.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.970+0000 7f44fb370640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f44f4104000 0x7f44f41a1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:07.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.970+0000 7f44fb370640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f4197d40 con 0x7f44f4104000 2024-09-13T18:21:07.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.970+0000 7f44fb370640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f41a2520 con 0x7f44f40ff880 2024-09-13T18:21:07.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.970+0000 7f44fb370640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f41a2660 con 0x7f44f40feed0 2024-09-13T18:21:07.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.970+0000 7f44f990d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f44f4104000 0x7f44f41a1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:07.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.970+0000 7f44f990d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f44f4104000 0x7f44f41a1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44936/0 (socket says 172.21.15.114:44936) 2024-09-13T18:21:07.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.970+0000 7f44f990d640 1 -- 172.21.15.114:0/217838037 learned_addr learned my addr 172.21.15.114:0/217838037 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:07.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.970+0000 7f44f910c640 1 --2- 172.21.15.114:0/217838037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44f40feed0 0x7f44f4196f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:07.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.970+0000 7f44f890b640 1 --2- 172.21.15.114:0/217838037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44f40ff880 0x7f44f41974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:07.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.970+0000 7f44f990d640 1 -- 172.21.15.114:0/217838037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44f40feed0 msgr2=0x7f44f4196f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:07.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.970+0000 7f44f990d640 1 --2- 172.21.15.114:0/217838037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44f40feed0 0x7f44f4196f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:07.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.970+0000 7f44f990d640 1 -- 172.21.15.114:0/217838037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44f40ff880 msgr2=0x7f44f41974a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:07.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.970+0000 7f44f990d640 1 --2- 172.21.15.114:0/217838037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44f40ff880 0x7f44f41974a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:07.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.970+0000 7f44f990d640 1 -- 172.21.15.114:0/217838037 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44ec009560 con 0x7f44f4104000 2024-09-13T18:21:07.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.970+0000 7f44f890b640 1 --2- 172.21.15.114:0/217838037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44f40ff880 0x7f44f41974a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:21:07.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.970+0000 7f44f990d640 1 --2- 172.21.15.114:0/217838037 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f44f4104000 0x7f44f41a1fe0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f44ec02a940 tx=0x7f44ec032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:07.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.971+0000 7f44f27fc640 1 -- 172.21.15.114:0/217838037 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ec0044f0 con 0x7f44f4104000 2024-09-13T18:21:07.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.971+0000 7f44f27fc640 1 -- 172.21.15.114:0/217838037 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f44ec032e40 con 0x7f44f4104000 2024-09-13T18:21:07.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.971+0000 7f44fb370640 1 -- 172.21.15.114:0/217838037 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44f41a28b0 con 0x7f44f4104000 2024-09-13T18:21:07.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.971+0000 7f44f27fc640 1 -- 172.21.15.114:0/217838037 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ec003850 con 0x7f44f4104000 2024-09-13T18:21:07.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.971+0000 7f44fb370640 1 -- 172.21.15.114:0/217838037 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44f41a2d50 con 0x7f44f4104000 2024-09-13T18:21:07.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.971+0000 7f44fb370640 1 -- 172.21.15.114:0/217838037 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f44f410ee40 con 0x7f44f4104000 2024-09-13T18:21:07.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.973+0000 7f44f27fc640 1 -- 172.21.15.114:0/217838037 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f44ec004930 con 0x7f44f4104000 2024-09-13T18:21:07.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.973+0000 7f44f27fc640 1 --2- 172.21.15.114:0/217838037 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f44dc0a1110 0x7f44dc0a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:07.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.973+0000 7f44f910c640 1 --2- 172.21.15.114:0/217838037 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f44dc0a1110 0x7f44dc0a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:07.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.973+0000 7f44f27fc640 1 -- 172.21.15.114:0/217838037 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f44ec0eddb0 con 0x7f44f4104000 2024-09-13T18:21:07.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.973+0000 7f44f27fc640 1 --2- 172.21.15.114:0/217838037 >> [v2:172.21.15.114:6826/2172392854,v1:172.21.15.114:6827/2172392854] conn(0x7f44dc0ac1a0 0x7f44dc0ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:07.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.973+0000 7f44f27fc640 1 -- 172.21.15.114:0/217838037 --> [v2:172.21.15.114:6826/2172392854,v1:172.21.15.114:6827/2172392854] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f44dc0aed10 con 0x7f44dc0ac1a0 2024-09-13T18:21:07.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.973+0000 7f44f27fc640 1 -- 172.21.15.114:0/217838037 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44ec0ee1c0 con 0x7f44f4104000 2024-09-13T18:21:07.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.974+0000 7f44f890b640 1 --2- 172.21.15.114:0/217838037 >> [v2:172.21.15.114:6826/2172392854,v1:172.21.15.114:6827/2172392854] conn(0x7f44dc0ac1a0 0x7f44dc0ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:07.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.974+0000 7f44f910c640 1 --2- 172.21.15.114:0/217838037 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f44dc0a1110 0x7f44dc0a35d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f44e4007ec0 tx=0x7f44e40075b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:07.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.974+0000 7f44f890b640 1 --2- 172.21.15.114:0/217838037 >> [v2:172.21.15.114:6826/2172392854,v1:172.21.15.114:6827/2172392854] conn(0x7f44dc0ac1a0 0x7f44dc0ae640 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:07.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.975+0000 7f44f27fc640 1 -- 172.21.15.114:0/217838037 <== osd.9 v2:172.21.15.114:6826/2172392854 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f44dc0aed10 con 0x7f44dc0ac1a0 2024-09-13T18:21:07.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.987+0000 7f44fb370640 1 -- 172.21.15.114:0/217838037 --> [v2:172.21.15.114:6826/2172392854,v1:172.21.15.114:6827/2172392854] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f44f41003b0 con 0x7f44dc0ac1a0 2024-09-13T18:21:07.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.988+0000 7f44f27fc640 1 -- 172.21.15.114:0/217838037 <== osd.9 v2:172.21.15.114:6826/2172392854 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f44f41003b0 con 0x7f44dc0ac1a0 2024-09-13T18:21:07.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.988+0000 7f44fb370640 1 -- 172.21.15.114:0/217838037 >> [v2:172.21.15.114:6826/2172392854,v1:172.21.15.114:6827/2172392854] conn(0x7f44dc0ac1a0 msgr2=0x7f44dc0ae640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:07.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.988+0000 7f44fb370640 1 --2- 172.21.15.114:0/217838037 >> [v2:172.21.15.114:6826/2172392854,v1:172.21.15.114:6827/2172392854] conn(0x7f44dc0ac1a0 0x7f44dc0ae640 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:07.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.988+0000 7f44fb370640 1 -- 172.21.15.114:0/217838037 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f44dc0a1110 msgr2=0x7f44dc0a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:07.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.988+0000 7f44fb370640 1 --2- 172.21.15.114:0/217838037 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f44dc0a1110 0x7f44dc0a35d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f44e4007ec0 tx=0x7f44e40075b0 comp rx=0 tx=0).stop 2024-09-13T18:21:07.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.988+0000 7f44fb370640 1 -- 172.21.15.114:0/217838037 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f44f4104000 msgr2=0x7f44f41a1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:07.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.988+0000 7f44fb370640 1 --2- 172.21.15.114:0/217838037 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f44f4104000 0x7f44f41a1fe0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f44ec02a940 tx=0x7f44ec032c20 comp rx=0 tx=0).stop 2024-09-13T18:21:07.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.988+0000 7f44f990d640 1 -- 172.21.15.114:0/217838037 reap_dead start 2024-09-13T18:21:07.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.988+0000 7f44fb370640 1 -- 172.21.15.114:0/217838037 shutdown_connections 2024-09-13T18:21:07.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.988+0000 7f44fb370640 1 -- 172.21.15.114:0/217838037 >> 172.21.15.114:0/217838037 conn(0x7f44f40fa9b0 msgr2=0x7f44f40fbd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:07.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.988+0000 7f44fb370640 1 -- 172.21.15.114:0/217838037 shutdown_connections 2024-09-13T18:21:07.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:06.988+0000 7f44fb370640 1 -- 172.21.15.114:0/217838037 wait complete. 2024-09-13T18:21:07.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:07 smithi114 ceph-mon[30897]: pgmap v311: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-13T18:21:07.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:07 smithi134 ceph-mon[34283]: pgmap v311: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-13T18:21:07.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.395+0000 7f2033d31640 1 -- 172.21.15.114:0/2346101380 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f202c0ff880 msgr2=0x7f202c1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:07.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.395+0000 7f2033d31640 1 --2- 172.21.15.114:0/2346101380 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f202c0ff880 0x7f202c1038d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2020009930 tx=0x7f202002a420 comp rx=0 tx=0).stop 2024-09-13T18:21:07.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.395+0000 7f2033d31640 1 -- 172.21.15.114:0/2346101380 shutdown_connections 2024-09-13T18:21:07.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.395+0000 7f2033d31640 1 --2- 172.21.15.114:0/2346101380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f202c104000 0x7f202c10bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:07.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.395+0000 7f2033d31640 1 --2- 172.21.15.114:0/2346101380 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f202c0ff880 0x7f202c1038d0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:07.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.395+0000 7f2033d31640 1 --2- 172.21.15.114:0/2346101380 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f202c0feed0 0x7f202c0ff2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:07.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.395+0000 7f2033d31640 1 -- 172.21.15.114:0/2346101380 >> 172.21.15.114:0/2346101380 conn(0x7f202c0fa9b0 msgr2=0x7f202c0fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:07.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.395+0000 7f2033d31640 1 -- 172.21.15.114:0/2346101380 shutdown_connections 2024-09-13T18:21:07.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.395+0000 7f2033d31640 1 -- 172.21.15.114:0/2346101380 wait complete. 2024-09-13T18:21:07.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.395+0000 7f2033d31640 1 Processor -- start 2024-09-13T18:21:07.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.395+0000 7f2033d31640 1 -- start start 2024-09-13T18:21:07.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.396+0000 7f2033d31640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f202c0feed0 0x7f202c196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:07.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.396+0000 7f2033d31640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f202c0ff880 0x7f202c1974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:07.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.396+0000 7f2033d31640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f202c104000 0x7f202c1a2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:07.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.396+0000 7f2033d31640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202c197cf0 con 0x7f202c0ff880 2024-09-13T18:21:07.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.396+0000 7f2033d31640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202c1a2540 con 0x7f202c104000 2024-09-13T18:21:07.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.396+0000 7f2033d31640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202c1a2680 con 0x7f202c0feed0 2024-09-13T18:21:07.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.396+0000 7f20312cc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f202c0ff880 0x7f202c1974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:07.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.396+0000 7f2031acd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f202c0feed0 0x7f202c196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:07.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.396+0000 7f20312cc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f202c0ff880 0x7f202c1974c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44954/0 (socket says 172.21.15.114:44954) 2024-09-13T18:21:07.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.396+0000 7f20312cc640 1 -- 172.21.15.114:0/109603757 learned_addr learned my addr 172.21.15.114:0/109603757 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:07.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.396+0000 7f20322ce640 1 --2- 172.21.15.114:0/109603757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f202c104000 0x7f202c1a2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:07.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.396+0000 7f20312cc640 1 -- 172.21.15.114:0/109603757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f202c0feed0 msgr2=0x7f202c196f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:07.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.396+0000 7f20312cc640 1 --2- 172.21.15.114:0/109603757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f202c0feed0 0x7f202c196f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:07.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.396+0000 7f20312cc640 1 -- 172.21.15.114:0/109603757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f202c104000 msgr2=0x7f202c1a2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:07.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.396+0000 7f20312cc640 1 --2- 172.21.15.114:0/109603757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f202c104000 0x7f202c1a2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:07.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.396+0000 7f20312cc640 1 -- 172.21.15.114:0/109603757 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2020009560 con 0x7f202c0ff880 2024-09-13T18:21:07.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.396+0000 7f20322ce640 1 --2- 172.21.15.114:0/109603757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f202c104000 0x7f202c1a2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:21:07.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.396+0000 7f20312cc640 1 --2- 172.21.15.114:0/109603757 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f202c0ff880 0x7f202c1974c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2020005ab0 tx=0x7f2020032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:07.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.397+0000 7f202affd640 1 -- 172.21.15.114:0/109603757 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2020032e10 con 0x7f202c0ff880 2024-09-13T18:21:07.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.397+0000 7f2033d31640 1 -- 172.21.15.114:0/109603757 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f202c1a2880 con 0x7f202c0ff880 2024-09-13T18:21:07.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.397+0000 7f202affd640 1 -- 172.21.15.114:0/109603757 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f20200046c0 con 0x7f202c0ff880 2024-09-13T18:21:07.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.397+0000 7f202affd640 1 -- 172.21.15.114:0/109603757 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2020003850 con 0x7f202c0ff880 2024-09-13T18:21:07.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.397+0000 7f2033d31640 1 -- 172.21.15.114:0/109603757 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f202c1a2d40 con 0x7f202c0ff880 2024-09-13T18:21:07.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.397+0000 7f2033d31640 1 -- 172.21.15.114:0/109603757 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f202c1003b0 con 0x7f202c0ff880 2024-09-13T18:21:07.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.399+0000 7f202affd640 1 -- 172.21.15.114:0/109603757 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f2020004830 con 0x7f202c0ff880 2024-09-13T18:21:07.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.399+0000 7f202affd640 1 --2- 172.21.15.114:0/109603757 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f20140a1110 0x7f20140a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:07.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.399+0000 7f2031acd640 1 --2- 172.21.15.114:0/109603757 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f20140a1110 0x7f20140a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:07.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.399+0000 7f202affd640 1 -- 172.21.15.114:0/109603757 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f20200eddf0 con 0x7f202c0ff880 2024-09-13T18:21:07.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.399+0000 7f202affd640 1 --2- 172.21.15.114:0/109603757 >> [v2:172.21.15.204:6800/2288807707,v1:172.21.15.204:6801/2288807707] conn(0x7f20140ac1a0 0x7f20140ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:07.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.399+0000 7f202affd640 1 -- 172.21.15.114:0/109603757 --> [v2:172.21.15.204:6800/2288807707,v1:172.21.15.204:6801/2288807707] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f20140aed10 con 0x7f20140ac1a0 2024-09-13T18:21:07.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.399+0000 7f202affd640 1 -- 172.21.15.114:0/109603757 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20200ee200 con 0x7f202c0ff880 2024-09-13T18:21:07.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.400+0000 7f2031acd640 1 --2- 172.21.15.114:0/109603757 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f20140a1110 0x7f20140a35d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f202400bbe0 tx=0x7f202400b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:07.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.400+0000 7f20322ce640 1 --2- 172.21.15.114:0/109603757 >> [v2:172.21.15.204:6800/2288807707,v1:172.21.15.204:6801/2288807707] conn(0x7f20140ac1a0 0x7f20140ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:07.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.400+0000 7f20322ce640 1 --2- 172.21.15.114:0/109603757 >> [v2:172.21.15.204:6800/2288807707,v1:172.21.15.204:6801/2288807707] conn(0x7f20140ac1a0 0x7f20140ae640 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:07.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.401+0000 7f202affd640 1 -- 172.21.15.114:0/109603757 <== osd.2 v2:172.21.15.204:6800/2288807707 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f20140aed10 con 0x7f20140ac1a0 2024-09-13T18:21:07.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.414+0000 7f2033d31640 1 -- 172.21.15.114:0/109603757 --> [v2:172.21.15.204:6800/2288807707,v1:172.21.15.204:6801/2288807707] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f202c060b30 con 0x7f20140ac1a0 2024-09-13T18:21:07.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.415+0000 7f202affd640 1 -- 172.21.15.114:0/109603757 <== osd.2 v2:172.21.15.204:6800/2288807707 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f202c060b30 con 0x7f20140ac1a0 2024-09-13T18:21:07.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.415+0000 7f2033d31640 1 -- 172.21.15.114:0/109603757 >> [v2:172.21.15.204:6800/2288807707,v1:172.21.15.204:6801/2288807707] conn(0x7f20140ac1a0 msgr2=0x7f20140ae640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:07.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.415+0000 7f2033d31640 1 --2- 172.21.15.114:0/109603757 >> [v2:172.21.15.204:6800/2288807707,v1:172.21.15.204:6801/2288807707] conn(0x7f20140ac1a0 0x7f20140ae640 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:07.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.415+0000 7f2033d31640 1 -- 172.21.15.114:0/109603757 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f20140a1110 msgr2=0x7f20140a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:07.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.416+0000 7f2033d31640 1 --2- 172.21.15.114:0/109603757 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f20140a1110 0x7f20140a35d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f202400bbe0 tx=0x7f202400b390 comp rx=0 tx=0).stop 2024-09-13T18:21:07.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.416+0000 7f2033d31640 1 -- 172.21.15.114:0/109603757 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f202c0ff880 msgr2=0x7f202c1974c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:07.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.416+0000 7f2033d31640 1 --2- 172.21.15.114:0/109603757 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f202c0ff880 0x7f202c1974c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2020005ab0 tx=0x7f2020032c20 comp rx=0 tx=0).stop 2024-09-13T18:21:07.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.416+0000 7f20322ce640 1 -- 172.21.15.114:0/109603757 reap_dead start 2024-09-13T18:21:07.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.416+0000 7f2033d31640 1 -- 172.21.15.114:0/109603757 shutdown_connections 2024-09-13T18:21:07.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.416+0000 7f2033d31640 1 -- 172.21.15.114:0/109603757 >> 172.21.15.114:0/109603757 conn(0x7f202c0fa9b0 msgr2=0x7f202c0fbd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:07.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.416+0000 7f2033d31640 1 -- 172.21.15.114:0/109603757 shutdown_connections 2024-09-13T18:21:07.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:07.416+0000 7f2033d31640 1 -- 172.21.15.114:0/109603757 wait complete. 2024-09-13T18:21:07.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:07 smithi204 ceph-mon[34777]: pgmap v311: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-13T18:21:08.092 INFO:teuthology.orchestra.run.smithi114.stdout:77309411396 2024-09-13T18:21:08.092 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-13T18:21:08.961 INFO:teuthology.orchestra.run.smithi114.stdout:141733920826 2024-09-13T18:21:08.961 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-13T18:21:09.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.143+0000 7f5f8f17f640 1 -- 172.21.15.114:0/677327724 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f8810c9f0 msgr2=0x7f5f8810ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:09.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.143+0000 7f5f8f17f640 1 --2- 172.21.15.114:0/677327724 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f8810c9f0 0x7f5f8810ede0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5f80009ca0 tx=0x7f5f8002a8c0 comp rx=0 tx=0).stop 2024-09-13T18:21:09.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.143+0000 7f5f8f17f640 1 -- 172.21.15.114:0/677327724 shutdown_connections 2024-09-13T18:21:09.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.143+0000 7f5f8f17f640 1 --2- 172.21.15.114:0/677327724 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f8810c9f0 0x7f5f8810ede0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:09.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.143+0000 7f5f8f17f640 1 --2- 172.21.15.114:0/677327724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f881010f0 0x7f5f880fe7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:09.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.143+0000 7f5f8f17f640 1 --2- 172.21.15.114:0/677327724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f88100c80 0x7f5f880fe270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:09.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.143+0000 7f5f8f17f640 1 -- 172.21.15.114:0/677327724 >> 172.21.15.114:0/677327724 conn(0x7f5f880753c0 msgr2=0x7f5f880fc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:09.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.143+0000 7f5f8f17f640 1 -- 172.21.15.114:0/677327724 shutdown_connections 2024-09-13T18:21:09.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.143+0000 7f5f8f17f640 1 -- 172.21.15.114:0/677327724 wait complete. 2024-09-13T18:21:09.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.144+0000 7f5f8f17f640 1 Processor -- start 2024-09-13T18:21:09.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.144+0000 7f5f8f17f640 1 -- start start 2024-09-13T18:21:09.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.144+0000 7f5f8f17f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f88100c80 0x7f5f881983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:09.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.144+0000 7f5f8f17f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f881010f0 0x7f5f88198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:09.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.144+0000 7f5f8f17f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f8810c9f0 0x7f5f8819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:09.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.144+0000 7f5f8f17f640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8819d320 con 0x7f5f881010f0 2024-09-13T18:21:09.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.144+0000 7f5f8f17f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8819d490 con 0x7f5f88100c80 2024-09-13T18:21:09.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.144+0000 7f5f8f17f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8819d600 con 0x7f5f8810c9f0 2024-09-13T18:21:09.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.144+0000 7f5f87fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f881010f0 0x7f5f88198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:09.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.144+0000 7f5f87fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f881010f0 0x7f5f88198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44960/0 (socket says 172.21.15.114:44960) 2024-09-13T18:21:09.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.144+0000 7f5f87fff640 1 -- 172.21.15.114:0/1172899807 learned_addr learned my addr 172.21.15.114:0/1172899807 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:09.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.144+0000 7f5f8d71c640 1 --2- 172.21.15.114:0/1172899807 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f8810c9f0 0x7f5f8819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:09.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.144+0000 7f5f8cf1b640 1 --2- 172.21.15.114:0/1172899807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f88100c80 0x7f5f881983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:09.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.144+0000 7f5f87fff640 1 -- 172.21.15.114:0/1172899807 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f8810c9f0 msgr2=0x7f5f8819cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:09.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.144+0000 7f5f87fff640 1 --2- 172.21.15.114:0/1172899807 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f8810c9f0 0x7f5f8819cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:09.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.144+0000 7f5f87fff640 1 -- 172.21.15.114:0/1172899807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f88100c80 msgr2=0x7f5f881983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:09.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.144+0000 7f5f87fff640 1 --2- 172.21.15.114:0/1172899807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f88100c80 0x7f5f881983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:09.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.144+0000 7f5f87fff640 1 -- 172.21.15.114:0/1172899807 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f80009900 con 0x7f5f881010f0 2024-09-13T18:21:09.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.144+0000 7f5f87fff640 1 --2- 172.21.15.114:0/1172899807 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f881010f0 0x7f5f88198920 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5f7400b4a0 tx=0x7f5f7400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:09.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.145+0000 7f5f85ffb640 1 -- 172.21.15.114:0/1172899807 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f74010610 con 0x7f5f881010f0 2024-09-13T18:21:09.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.145+0000 7f5f85ffb640 1 -- 172.21.15.114:0/1172899807 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5f74010c00 con 0x7f5f881010f0 2024-09-13T18:21:09.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.145+0000 7f5f8f17f640 1 -- 172.21.15.114:0/1172899807 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f8819d8e0 con 0x7f5f881010f0 2024-09-13T18:21:09.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.145+0000 7f5f85ffb640 1 -- 172.21.15.114:0/1172899807 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f74017ca0 con 0x7f5f881010f0 2024-09-13T18:21:09.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.145+0000 7f5f8f17f640 1 -- 172.21.15.114:0/1172899807 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f881a51a0 con 0x7f5f881010f0 2024-09-13T18:21:09.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.146+0000 7f5f8f17f640 1 -- 172.21.15.114:0/1172899807 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5f880ff490 con 0x7f5f881010f0 2024-09-13T18:21:09.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.147+0000 7f5f85ffb640 1 -- 172.21.15.114:0/1172899807 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f5f74010770 con 0x7f5f881010f0 2024-09-13T18:21:09.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.147+0000 7f5f85ffb640 1 --2- 172.21.15.114:0/1172899807 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f5f780a1190 0x7f5f780a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:09.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.148+0000 7f5f85ffb640 1 -- 172.21.15.114:0/1172899807 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5f740cc5e0 con 0x7f5f881010f0 2024-09-13T18:21:09.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.148+0000 7f5f8cf1b640 1 --2- 172.21.15.114:0/1172899807 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f5f780a1190 0x7f5f780a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:09.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.148+0000 7f5f85ffb640 1 --2- 172.21.15.114:0/1172899807 >> [v2:172.21.15.134:6800/4264264866,v1:172.21.15.134:6801/4264264866] conn(0x7f5f780ac220 0x7f5f780ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:09.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.148+0000 7f5f85ffb640 1 -- 172.21.15.114:0/1172899807 --> [v2:172.21.15.134:6800/4264264866,v1:172.21.15.134:6801/4264264866] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5f780aed90 con 0x7f5f780ac220 2024-09-13T18:21:09.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.148+0000 7f5f85ffb640 1 -- 172.21.15.114:0/1172899807 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f740cec60 con 0x7f5f881010f0 2024-09-13T18:21:09.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.148+0000 7f5f8cf1b640 1 --2- 172.21.15.114:0/1172899807 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f5f780a1190 0x7f5f780a3650 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c009b60 tx=0x7f5f7c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:09.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.148+0000 7f5f8d71c640 1 --2- 172.21.15.114:0/1172899807 >> [v2:172.21.15.134:6800/4264264866,v1:172.21.15.134:6801/4264264866] conn(0x7f5f780ac220 0x7f5f780ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:09.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.148+0000 7f5f8d71c640 1 --2- 172.21.15.114:0/1172899807 >> [v2:172.21.15.134:6800/4264264866,v1:172.21.15.134:6801/4264264866] conn(0x7f5f780ac220 0x7f5f780ae6c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:09.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.149+0000 7f5f85ffb640 1 -- 172.21.15.114:0/1172899807 <== osd.1 v2:172.21.15.134:6800/4264264866 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f5f780aed90 con 0x7f5f780ac220 2024-09-13T18:21:09.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.162+0000 7f5f8f17f640 1 -- 172.21.15.114:0/1172899807 --> [v2:172.21.15.134:6800/4264264866,v1:172.21.15.134:6801/4264264866] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5f881073d0 con 0x7f5f780ac220 2024-09-13T18:21:09.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.163+0000 7f5f85ffb640 1 -- 172.21.15.114:0/1172899807 <== osd.1 v2:172.21.15.134:6800/4264264866 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5f881073d0 con 0x7f5f780ac220 2024-09-13T18:21:09.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.163+0000 7f5f8f17f640 1 -- 172.21.15.114:0/1172899807 >> [v2:172.21.15.134:6800/4264264866,v1:172.21.15.134:6801/4264264866] conn(0x7f5f780ac220 msgr2=0x7f5f780ae6c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:09.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.163+0000 7f5f8f17f640 1 --2- 172.21.15.114:0/1172899807 >> [v2:172.21.15.134:6800/4264264866,v1:172.21.15.134:6801/4264264866] conn(0x7f5f780ac220 0x7f5f780ae6c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:09.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.163+0000 7f5f8f17f640 1 -- 172.21.15.114:0/1172899807 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f5f780a1190 msgr2=0x7f5f780a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:09.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.163+0000 7f5f8f17f640 1 --2- 172.21.15.114:0/1172899807 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f5f780a1190 0x7f5f780a3650 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c009b60 tx=0x7f5f7c005f90 comp rx=0 tx=0).stop 2024-09-13T18:21:09.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.163+0000 7f5f8f17f640 1 -- 172.21.15.114:0/1172899807 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f881010f0 msgr2=0x7f5f88198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:09.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.163+0000 7f5f8f17f640 1 --2- 172.21.15.114:0/1172899807 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f881010f0 0x7f5f88198920 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5f7400b4a0 tx=0x7f5f7400b960 comp rx=0 tx=0).stop 2024-09-13T18:21:09.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.163+0000 7f5f8d71c640 1 -- 172.21.15.114:0/1172899807 reap_dead start 2024-09-13T18:21:09.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.163+0000 7f5f8f17f640 1 -- 172.21.15.114:0/1172899807 shutdown_connections 2024-09-13T18:21:09.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.163+0000 7f5f8f17f640 1 -- 172.21.15.114:0/1172899807 >> 172.21.15.114:0/1172899807 conn(0x7f5f880753c0 msgr2=0x7f5f88106a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:09.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.164+0000 7f5f8f17f640 1 -- 172.21.15.114:0/1172899807 shutdown_connections 2024-09-13T18:21:09.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:09.164+0000 7f5f8f17f640 1 -- 172.21.15.114:0/1172899807 wait complete. 2024-09-13T18:21:09.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:09 smithi114 ceph-mon[30897]: pgmap v312: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-13T18:21:09.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:09 smithi134 ceph-mon[34283]: pgmap v312: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-13T18:21:09.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:09 smithi204 ceph-mon[34777]: pgmap v312: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-13T18:21:09.812 INFO:teuthology.orchestra.run.smithi114.stdout:107374182462 2024-09-13T18:21:09.813 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-13T18:21:10.737 INFO:teuthology.orchestra.run.smithi114.stdout:219043332141 2024-09-13T18:21:10.737 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-13T18:21:11.191 INFO:teuthology.orchestra.run.smithi114.stdout:287762808868 2024-09-13T18:21:11.191 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-13T18:21:11.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:11 smithi134 ceph-mon[34283]: pgmap v313: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-13T18:21:11.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:11 smithi204 ceph-mon[34777]: pgmap v313: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-13T18:21:11.469 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:11 smithi114 ceph-mon[30897]: pgmap v313: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-13T18:21:11.612 INFO:teuthology.orchestra.run.smithi114.stdout:253403070504 2024-09-13T18:21:11.613 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-13T18:21:11.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.780+0000 7fd10bc35640 1 -- 172.21.15.114:0/3092491011 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd104100f30 msgr2=0x7fd104101310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:11.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.781+0000 7fd10bc35640 1 --2- 172.21.15.114:0/3092491011 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd104100f30 0x7fd104101310 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4009900 tx=0x7fd0f402a480 comp rx=0 tx=0).stop 2024-09-13T18:21:11.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.781+0000 7fd10bc35640 1 -- 172.21.15.114:0/3092491011 shutdown_connections 2024-09-13T18:21:11.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.781+0000 7fd10bc35640 1 --2- 172.21.15.114:0/3092491011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd10410a360 0x7fd10410c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:11.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.781+0000 7fd10bc35640 1 --2- 172.21.15.114:0/3092491011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd104101850 0x7fd104109d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:11.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.781+0000 7fd10bc35640 1 --2- 172.21.15.114:0/3092491011 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd104100f30 0x7fd104101310 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:11.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.781+0000 7fd10bc35640 1 -- 172.21.15.114:0/3092491011 >> 172.21.15.114:0/3092491011 conn(0x7fd1040fa9b0 msgr2=0x7fd1040fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:11.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.781+0000 7fd10bc35640 1 -- 172.21.15.114:0/3092491011 shutdown_connections 2024-09-13T18:21:11.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.781+0000 7fd10bc35640 1 -- 172.21.15.114:0/3092491011 wait complete. 2024-09-13T18:21:11.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.782+0000 7fd10bc35640 1 Processor -- start 2024-09-13T18:21:11.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.782+0000 7fd10bc35640 1 -- start start 2024-09-13T18:21:11.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.782+0000 7fd10bc35640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd104100f30 0x7fd104196140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:11.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.782+0000 7fd10bc35640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd104101850 0x7fd104196680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:11.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.782+0000 7fd10bc35640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd10410a360 0x7fd10419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:11.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.782+0000 7fd10bc35640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10419b080 con 0x7fd104100f30 2024-09-13T18:21:11.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.782+0000 7fd10bc35640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10419b1f0 con 0x7fd104101850 2024-09-13T18:21:11.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.782+0000 7fd10bc35640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10419b360 con 0x7fd10410a360 2024-09-13T18:21:11.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.782+0000 7fd1099d1640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd104100f30 0x7fd104196140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:11.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.782+0000 7fd1099d1640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd104100f30 0x7fd104196140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44972/0 (socket says 172.21.15.114:44972) 2024-09-13T18:21:11.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.782+0000 7fd1099d1640 1 -- 172.21.15.114:0/897259933 learned_addr learned my addr 172.21.15.114:0/897259933 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:11.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.782+0000 7fd1091d0640 1 --2- 172.21.15.114:0/897259933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd104101850 0x7fd104196680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:11.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.782+0000 7fd10a1d2640 1 --2- 172.21.15.114:0/897259933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd10410a360 0x7fd10419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:11.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.782+0000 7fd1099d1640 1 -- 172.21.15.114:0/897259933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd10410a360 msgr2=0x7fd10419aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:11.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.782+0000 7fd1099d1640 1 --2- 172.21.15.114:0/897259933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd10410a360 0x7fd10419aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:11.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.782+0000 7fd1099d1640 1 -- 172.21.15.114:0/897259933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd104101850 msgr2=0x7fd104196680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:11.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.782+0000 7fd1099d1640 1 --2- 172.21.15.114:0/897259933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd104101850 0x7fd104196680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:11.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.782+0000 7fd1099d1640 1 -- 172.21.15.114:0/897259933 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f4009560 con 0x7fd104100f30 2024-09-13T18:21:11.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.782+0000 7fd10a1d2640 1 --2- 172.21.15.114:0/897259933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd10410a360 0x7fd10419aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:21:11.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.783+0000 7fd1099d1640 1 --2- 172.21.15.114:0/897259933 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd104100f30 0x7fd104196140 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd0f40057c0 tx=0x7fd0f4032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:11.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.783+0000 7fd0faffd640 1 -- 172.21.15.114:0/897259933 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4032e10 con 0x7fd104100f30 2024-09-13T18:21:11.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.783+0000 7fd0faffd640 1 -- 172.21.15.114:0/897259933 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd0f40046c0 con 0x7fd104100f30 2024-09-13T18:21:11.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.783+0000 7fd10bc35640 1 -- 172.21.15.114:0/897259933 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd10419b5e0 con 0x7fd104100f30 2024-09-13T18:21:11.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.783+0000 7fd0faffd640 1 -- 172.21.15.114:0/897259933 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4003850 con 0x7fd104100f30 2024-09-13T18:21:11.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.783+0000 7fd10bc35640 1 -- 172.21.15.114:0/897259933 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1041a2d30 con 0x7fd104100f30 2024-09-13T18:21:11.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.784+0000 7fd10bc35640 1 -- 172.21.15.114:0/897259933 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd1040687e0 con 0x7fd104100f30 2024-09-13T18:21:11.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.785+0000 7fd0faffd640 1 -- 172.21.15.114:0/897259933 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fd0f4005210 con 0x7fd104100f30 2024-09-13T18:21:11.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.785+0000 7fd0faffd640 1 --2- 172.21.15.114:0/897259933 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fd0f00a1110 0x7fd0f00a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:11.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.786+0000 7fd1091d0640 1 --2- 172.21.15.114:0/897259933 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fd0f00a1110 0x7fd0f00a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:11.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.786+0000 7fd0faffd640 1 -- 172.21.15.114:0/897259933 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd0f40edfd0 con 0x7fd104100f30 2024-09-13T18:21:11.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.786+0000 7fd0faffd640 1 --2- 172.21.15.114:0/897259933 >> [v2:172.21.15.114:6802/3499525826,v1:172.21.15.114:6803/3499525826] conn(0x7fd0f00ac1a0 0x7fd0f00ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:11.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.786+0000 7fd0faffd640 1 -- 172.21.15.114:0/897259933 --> [v2:172.21.15.114:6802/3499525826,v1:172.21.15.114:6803/3499525826] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd0f00aed10 con 0x7fd0f00ac1a0 2024-09-13T18:21:11.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.786+0000 7fd0faffd640 1 -- 172.21.15.114:0/897259933 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd0f40ee3e0 con 0x7fd104100f30 2024-09-13T18:21:11.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.786+0000 7fd10a1d2640 1 --2- 172.21.15.114:0/897259933 >> [v2:172.21.15.114:6802/3499525826,v1:172.21.15.114:6803/3499525826] conn(0x7fd0f00ac1a0 0x7fd0f00ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:11.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.786+0000 7fd1091d0640 1 --2- 172.21.15.114:0/897259933 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fd0f00a1110 0x7fd0f00a35d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd1041976f0 tx=0x7fd10000b2e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:11.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.786+0000 7fd10a1d2640 1 --2- 172.21.15.114:0/897259933 >> [v2:172.21.15.114:6802/3499525826,v1:172.21.15.114:6803/3499525826] conn(0x7fd0f00ac1a0 0x7fd0f00ae640 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:11.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.787+0000 7fd0faffd640 1 -- 172.21.15.114:0/897259933 <== osd.0 v2:172.21.15.114:6802/3499525826 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fd0f00aed10 con 0x7fd0f00ac1a0 2024-09-13T18:21:11.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.800+0000 7fd10bc35640 1 -- 172.21.15.114:0/897259933 --> [v2:172.21.15.114:6802/3499525826,v1:172.21.15.114:6803/3499525826] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd10410ff70 con 0x7fd0f00ac1a0 2024-09-13T18:21:11.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.800+0000 7fd0faffd640 1 -- 172.21.15.114:0/897259933 <== osd.0 v2:172.21.15.114:6802/3499525826 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd0fc0099f0 con 0x7fd0f00ac1a0 2024-09-13T18:21:11.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.801+0000 7fd10bc35640 1 -- 172.21.15.114:0/897259933 >> [v2:172.21.15.114:6802/3499525826,v1:172.21.15.114:6803/3499525826] conn(0x7fd0f00ac1a0 msgr2=0x7fd0f00ae640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:11.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.801+0000 7fd10bc35640 1 --2- 172.21.15.114:0/897259933 >> [v2:172.21.15.114:6802/3499525826,v1:172.21.15.114:6803/3499525826] conn(0x7fd0f00ac1a0 0x7fd0f00ae640 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:11.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.801+0000 7fd10bc35640 1 -- 172.21.15.114:0/897259933 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fd0f00a1110 msgr2=0x7fd0f00a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:11.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.801+0000 7fd10bc35640 1 --2- 172.21.15.114:0/897259933 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fd0f00a1110 0x7fd0f00a35d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd1041976f0 tx=0x7fd10000b2e0 comp rx=0 tx=0).stop 2024-09-13T18:21:11.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.801+0000 7fd10bc35640 1 -- 172.21.15.114:0/897259933 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd104100f30 msgr2=0x7fd104196140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:11.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.801+0000 7fd10bc35640 1 --2- 172.21.15.114:0/897259933 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd104100f30 0x7fd104196140 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd0f40057c0 tx=0x7fd0f4032c20 comp rx=0 tx=0).stop 2024-09-13T18:21:11.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.801+0000 7fd10a1d2640 1 -- 172.21.15.114:0/897259933 reap_dead start 2024-09-13T18:21:11.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.801+0000 7fd10bc35640 1 -- 172.21.15.114:0/897259933 shutdown_connections 2024-09-13T18:21:11.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.801+0000 7fd10bc35640 1 -- 172.21.15.114:0/897259933 >> 172.21.15.114:0/897259933 conn(0x7fd1040fa9b0 msgr2=0x7fd1040fc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:11.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.801+0000 7fd10bc35640 1 -- 172.21.15.114:0/897259933 shutdown_connections 2024-09-13T18:21:11.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:11.801+0000 7fd10bc35640 1 -- 172.21.15.114:0/897259933 wait complete. 2024-09-13T18:21:12.636 INFO:teuthology.orchestra.run.smithi114.stdout:309237645343 2024-09-13T18:21:12.636 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-13T18:21:13.117 INFO:teuthology.orchestra.run.smithi114.stdout:343597383705 2024-09-13T18:21:13.118 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-13T18:21:13.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:13 smithi134 ceph-mon[34283]: pgmap v314: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-13T18:21:13.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:13 smithi204 ceph-mon[34777]: pgmap v314: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-13T18:21:13.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:13 smithi114 ceph-mon[30897]: pgmap v314: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-13T18:21:13.550 INFO:teuthology.orchestra.run.smithi114.stdout:55834574922 2024-09-13T18:21:13.550 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-13T18:21:13.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.772+0000 7f0e7a7d0640 1 -- 172.21.15.114:0/426001541 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e741009b0 msgr2=0x7f0e74104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:13.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.772+0000 7f0e7a7d0640 1 --2- 172.21.15.114:0/426001541 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e741009b0 0x7f0e74104a00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c009930 tx=0x7f0e5c02a420 comp rx=0 tx=0).stop 2024-09-13T18:21:13.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.772+0000 7f0e7a7d0640 1 -- 172.21.15.114:0/426001541 shutdown_connections 2024-09-13T18:21:13.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.772+0000 7f0e7a7d0640 1 --2- 172.21.15.114:0/426001541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e74105130 0x7f0e7410ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:13.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.772+0000 7f0e7a7d0640 1 --2- 172.21.15.114:0/426001541 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e741009b0 0x7f0e74104a00 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:13.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.772+0000 7f0e7a7d0640 1 --2- 172.21.15.114:0/426001541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e74100000 0x7f0e741003e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:13.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.772+0000 7f0e7a7d0640 1 -- 172.21.15.114:0/426001541 >> 172.21.15.114:0/426001541 conn(0x7f0e740fba80 msgr2=0x7f0e740fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:13.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.772+0000 7f0e7a7d0640 1 -- 172.21.15.114:0/426001541 shutdown_connections 2024-09-13T18:21:13.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.772+0000 7f0e7a7d0640 1 -- 172.21.15.114:0/426001541 wait complete. 2024-09-13T18:21:13.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.773+0000 7f0e7a7d0640 1 Processor -- start 2024-09-13T18:21:13.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.773+0000 7f0e7a7d0640 1 -- start start 2024-09-13T18:21:13.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.773+0000 7f0e7a7d0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e74100000 0x7f0e741962c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:13.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.773+0000 7f0e7a7d0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e741009b0 0x7f0e74196800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:13.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.773+0000 7f0e7a7d0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e74105130 0x7f0e7419ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:13.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.773+0000 7f0e7a7d0640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7419b200 con 0x7f0e741009b0 2024-09-13T18:21:13.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.773+0000 7f0e7a7d0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7419b370 con 0x7f0e74100000 2024-09-13T18:21:13.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.773+0000 7f0e7a7d0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7419b4e0 con 0x7f0e74105130 2024-09-13T18:21:13.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.773+0000 7f0e6bfff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e741009b0 0x7f0e74196800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:13.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.773+0000 7f0e6bfff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e741009b0 0x7f0e74196800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44536/0 (socket says 172.21.15.114:44536) 2024-09-13T18:21:13.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.773+0000 7f0e6bfff640 1 -- 172.21.15.114:0/3152636467 learned_addr learned my addr 172.21.15.114:0/3152636467 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:13.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.773+0000 7f0e73fff640 1 --2- 172.21.15.114:0/3152636467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e74100000 0x7f0e741962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:13.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.773+0000 7f0e78d6d640 1 --2- 172.21.15.114:0/3152636467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e74105130 0x7f0e7419ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:13.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.773+0000 7f0e6bfff640 1 -- 172.21.15.114:0/3152636467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e74105130 msgr2=0x7f0e7419ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:13.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.773+0000 7f0e6bfff640 1 --2- 172.21.15.114:0/3152636467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e74105130 0x7f0e7419ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:13.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.773+0000 7f0e6bfff640 1 -- 172.21.15.114:0/3152636467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e74100000 msgr2=0x7f0e741962c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:13.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.773+0000 7f0e6bfff640 1 --2- 172.21.15.114:0/3152636467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e74100000 0x7f0e741962c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:13.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.773+0000 7f0e6bfff640 1 -- 172.21.15.114:0/3152636467 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e5c009560 con 0x7f0e741009b0 2024-09-13T18:21:13.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.773+0000 7f0e6bfff640 1 --2- 172.21.15.114:0/3152636467 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e741009b0 0x7f0e74196800 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c02aa10 tx=0x7f0e5c032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:13.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.774+0000 7f0e71ffb640 1 -- 172.21.15.114:0/3152636467 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c005210 con 0x7f0e741009b0 2024-09-13T18:21:13.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.774+0000 7f0e7a7d0640 1 -- 172.21.15.114:0/3152636467 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e7419b760 con 0x7f0e741009b0 2024-09-13T18:21:13.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.774+0000 7f0e71ffb640 1 -- 172.21.15.114:0/3152636467 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0e5c005370 con 0x7f0e741009b0 2024-09-13T18:21:13.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.774+0000 7f0e71ffb640 1 -- 172.21.15.114:0/3152636467 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c0038e0 con 0x7f0e741009b0 2024-09-13T18:21:13.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.774+0000 7f0e7a7d0640 1 -- 172.21.15.114:0/3152636467 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e7419ba80 con 0x7f0e741009b0 2024-09-13T18:21:13.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.775+0000 7f0e7a7d0640 1 -- 172.21.15.114:0/3152636467 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0e7410ff70 con 0x7f0e741009b0 2024-09-13T18:21:13.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.776+0000 7f0e71ffb640 1 -- 172.21.15.114:0/3152636467 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f0e5c003a40 con 0x7f0e741009b0 2024-09-13T18:21:13.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.776+0000 7f0e71ffb640 1 --2- 172.21.15.114:0/3152636467 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f0e600a1110 0x7f0e600a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:13.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.776+0000 7f0e73fff640 1 --2- 172.21.15.114:0/3152636467 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f0e600a1110 0x7f0e600a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:13.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.776+0000 7f0e71ffb640 1 -- 172.21.15.114:0/3152636467 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0e5c0edab0 con 0x7f0e741009b0 2024-09-13T18:21:13.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.777+0000 7f0e71ffb640 1 --2- 172.21.15.114:0/3152636467 >> [v2:172.21.15.204:6808/2371778055,v1:172.21.15.204:6809/2371778055] conn(0x7f0e600ac1a0 0x7f0e600ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:13.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.777+0000 7f0e71ffb640 1 -- 172.21.15.114:0/3152636467 --> [v2:172.21.15.204:6808/2371778055,v1:172.21.15.204:6809/2371778055] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0e600aed10 con 0x7f0e600ac1a0 2024-09-13T18:21:13.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.777+0000 7f0e71ffb640 1 -- 172.21.15.114:0/3152636467 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e5c0f22a0 con 0x7f0e741009b0 2024-09-13T18:21:13.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.777+0000 7f0e73fff640 1 --2- 172.21.15.114:0/3152636467 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f0e600a1110 0x7f0e600a35d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0e640058c0 tx=0x7f0e6400a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:13.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.777+0000 7f0e78d6d640 1 --2- 172.21.15.114:0/3152636467 >> [v2:172.21.15.204:6808/2371778055,v1:172.21.15.204:6809/2371778055] conn(0x7f0e600ac1a0 0x7f0e600ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:13.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.777+0000 7f0e78d6d640 1 --2- 172.21.15.114:0/3152636467 >> [v2:172.21.15.204:6808/2371778055,v1:172.21.15.204:6809/2371778055] conn(0x7f0e600ac1a0 0x7f0e600ae640 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:13.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.778+0000 7f0e71ffb640 1 -- 172.21.15.114:0/3152636467 <== osd.5 v2:172.21.15.204:6808/2371778055 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f0e600aed10 con 0x7f0e600ac1a0 2024-09-13T18:21:13.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.791+0000 7f0e7a7d0640 1 -- 172.21.15.114:0/3152636467 --> [v2:172.21.15.204:6808/2371778055,v1:172.21.15.204:6809/2371778055] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0e74060b30 con 0x7f0e600ac1a0 2024-09-13T18:21:13.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.792+0000 7f0e71ffb640 1 -- 172.21.15.114:0/3152636467 <== osd.5 v2:172.21.15.204:6808/2371778055 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f0e74060b30 con 0x7f0e600ac1a0 2024-09-13T18:21:13.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.792+0000 7f0e7a7d0640 1 -- 172.21.15.114:0/3152636467 >> [v2:172.21.15.204:6808/2371778055,v1:172.21.15.204:6809/2371778055] conn(0x7f0e600ac1a0 msgr2=0x7f0e600ae640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:13.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.792+0000 7f0e7a7d0640 1 --2- 172.21.15.114:0/3152636467 >> [v2:172.21.15.204:6808/2371778055,v1:172.21.15.204:6809/2371778055] conn(0x7f0e600ac1a0 0x7f0e600ae640 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:13.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.792+0000 7f0e7a7d0640 1 -- 172.21.15.114:0/3152636467 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f0e600a1110 msgr2=0x7f0e600a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:13.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.792+0000 7f0e7a7d0640 1 --2- 172.21.15.114:0/3152636467 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f0e600a1110 0x7f0e600a35d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0e640058c0 tx=0x7f0e6400a2d0 comp rx=0 tx=0).stop 2024-09-13T18:21:13.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.792+0000 7f0e7a7d0640 1 -- 172.21.15.114:0/3152636467 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e741009b0 msgr2=0x7f0e74196800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:13.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.792+0000 7f0e7a7d0640 1 --2- 172.21.15.114:0/3152636467 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e741009b0 0x7f0e74196800 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c02aa10 tx=0x7f0e5c032c20 comp rx=0 tx=0).stop 2024-09-13T18:21:13.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.792+0000 7f0e78d6d640 1 -- 172.21.15.114:0/3152636467 reap_dead start 2024-09-13T18:21:13.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.792+0000 7f0e7a7d0640 1 -- 172.21.15.114:0/3152636467 shutdown_connections 2024-09-13T18:21:13.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.792+0000 7f0e7a7d0640 1 -- 172.21.15.114:0/3152636467 >> 172.21.15.114:0/3152636467 conn(0x7f0e740fba80 msgr2=0x7f0e740fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:13.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.792+0000 7f0e7a7d0640 1 -- 172.21.15.114:0/3152636467 shutdown_connections 2024-09-13T18:21:13.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:13.792+0000 7f0e7a7d0640 1 -- 172.21.15.114:0/3152636467 wait complete. 2024-09-13T18:21:14.334 INFO:teuthology.orchestra.run.smithi114.stdout:34359738446 2024-09-13T18:21:14.334 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-13T18:21:14.914 INFO:teuthology.orchestra.run.smithi114.stdout:163208757304 2024-09-13T18:21:14.915 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-13T18:21:14.936 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:21:14.972 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:21:14.979 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:21:14.979 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:21:15.212 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:15 smithi114 ceph-mon[30897]: pgmap v315: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-13T18:21:15.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:15 smithi134 ceph-mon[34283]: pgmap v315: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-13T18:21:15.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:15 smithi204 ceph-mon[34777]: pgmap v315: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-13T18:21:17.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:17.332+0000 7f894b02f640 1 -- 172.21.15.114:0/3419696728 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8944104000 msgr2=0x7f894410bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:17.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:17.332+0000 7f894b02f640 1 --2- 172.21.15.114:0/3419696728 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8944104000 0x7f894410bb80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f893c009cd0 tx=0x7f893c02a8e0 comp rx=0 tx=0).stop 2024-09-13T18:21:17.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:17.332+0000 7f894b02f640 1 -- 172.21.15.114:0/3419696728 shutdown_connections 2024-09-13T18:21:17.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:17.332+0000 7f894b02f640 1 --2- 172.21.15.114:0/3419696728 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8944104000 0x7f894410bb80 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:17.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:17.332+0000 7f894b02f640 1 --2- 172.21.15.114:0/3419696728 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89440ff880 0x7f89441038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:17.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:17.332+0000 7f894b02f640 1 --2- 172.21.15.114:0/3419696728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89440feed0 0x7f89440ff2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:17.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:17.332+0000 7f894b02f640 1 -- 172.21.15.114:0/3419696728 >> 172.21.15.114:0/3419696728 conn(0x7f89440fa9b0 msgr2=0x7f89440fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:17.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:17.332+0000 7f894b02f640 1 -- 172.21.15.114:0/3419696728 shutdown_connections 2024-09-13T18:21:17.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:17.332+0000 7f894b02f640 1 -- 172.21.15.114:0/3419696728 wait complete. 2024-09-13T18:21:17.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:17.333+0000 7f894b02f640 1 Processor -- start 2024-09-13T18:21:17.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:17.333+0000 7f894b02f640 1 -- start start 2024-09-13T18:21:17.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:17.333+0000 7f894b02f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89440feed0 0x7f8944195150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:17.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:17.333+0000 7f894b02f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89440ff880 0x7f8944195690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:17.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:17.333+0000 7f894b02f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8944104000 0x7f8944199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:17.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:17.333+0000 7f894b02f640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f894419a090 con 0x7f89440ff880 2024-09-13T18:21:17.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:17.333+0000 7f894b02f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f894419a200 con 0x7f8944104000 2024-09-13T18:21:17.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:17.333+0000 7f894b02f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f894419a370 con 0x7f89440feed0 2024-09-13T18:21:17.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:17.333+0000 7f8943fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89440ff880 0x7f8944195690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:17.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:17.333+0000 7f8943fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89440ff880 0x7f8944195690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44546/0 (socket says 172.21.15.114:44546) 2024-09-13T18:21:17.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:17.334+0000 7f8943fff640 1 -- 172.21.15.114:0/984686246 learned_addr learned my addr 172.21.15.114:0/984686246 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:17.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:17.334+0000 7f8948dcb640 1 --2- 172.21.15.114:0/984686246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89440feed0 0x7f8944195150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:17.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:17.334+0000 7f89495cc640 1 --2- 172.21.15.114:0/984686246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8944104000 0x7f8944199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:17.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:17.334+0000 7f8943fff640 1 -- 172.21.15.114:0/984686246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89440feed0 msgr2=0x7f8944195150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:17.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:17.334+0000 7f8943fff640 1 --2- 172.21.15.114:0/984686246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89440feed0 0x7f8944195150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:17.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:17.334+0000 7f8943fff640 1 -- 172.21.15.114:0/984686246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8944104000 msgr2=0x7f8944199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:17.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:17.334+0000 7f8943fff640 1 --2- 172.21.15.114:0/984686246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8944104000 0x7f8944199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:17.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:17.334+0000 7f8943fff640 1 -- 172.21.15.114:0/984686246 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f893c009900 con 0x7f89440ff880 2024-09-13T18:21:17.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:17.334+0000 7f8943fff640 1 --2- 172.21.15.114:0/984686246 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89440ff880 0x7f8944195690 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f893800c890 tx=0x7f893800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:17.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:17.334+0000 7f8941ffb640 1 -- 172.21.15.114:0/984686246 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8938004220 con 0x7f89440ff880 2024-09-13T18:21:17.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:17.334+0000 7f8941ffb640 1 -- 172.21.15.114:0/984686246 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8938007e40 con 0x7f89440ff880 2024-09-13T18:21:17.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:17.334+0000 7f894b02f640 1 -- 172.21.15.114:0/984686246 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f894419a650 con 0x7f89440ff880 2024-09-13T18:21:17.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:17.334+0000 7f8941ffb640 1 -- 172.21.15.114:0/984686246 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89380047a0 con 0x7f89440ff880 2024-09-13T18:21:17.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:17.334+0000 7f894b02f640 1 -- 172.21.15.114:0/984686246 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89441a1f10 con 0x7f89440ff880 2024-09-13T18:21:17.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:17.335+0000 7f894b02f640 1 -- 172.21.15.114:0/984686246 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89441003b0 con 0x7f89440ff880 2024-09-13T18:21:17.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:17.336+0000 7f8941ffb640 1 -- 172.21.15.114:0/984686246 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f89380079b0 con 0x7f89440ff880 2024-09-13T18:21:17.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:17.337+0000 7f8941ffb640 1 --2- 172.21.15.114:0/984686246 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f892c0a1110 0x7f892c0a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:17.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:17.337+0000 7f8948dcb640 1 --2- 172.21.15.114:0/984686246 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f892c0a1110 0x7f892c0a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:17.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:17.337+0000 7f8941ffb640 1 -- 172.21.15.114:0/984686246 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f89380ccde0 con 0x7f89440ff880 2024-09-13T18:21:17.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:17.338+0000 7f8948dcb640 1 --2- 172.21.15.114:0/984686246 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f892c0a1110 0x7f892c0a35d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8934009990 tx=0x7f8934009110 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:17.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:17.341+0000 7f8941ffb640 1 -- 172.21.15.114:0/984686246 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f8938096100 con 0x7f89440ff880 2024-09-13T18:21:17.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:17 smithi134 ceph-mon[34283]: pgmap v316: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-13T18:21:17.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:17 smithi204 ceph-mon[34777]: pgmap v316: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-13T18:21:17.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:17 smithi114 ceph-mon[30897]: pgmap v316: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-13T18:21:17.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:17.509+0000 7f894b02f640 1 -- 172.21.15.114:0/984686246 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f8944196510 con 0x7f89440ff880 2024-09-13T18:21:17.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:17.510+0000 7f8941ffb640 1 -- 172.21.15.114:0/984686246 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8938099720 con 0x7f89440ff880 2024-09-13T18:21:17.513 INFO:teuthology.orchestra.run.smithi114.stdout:197568495668 2024-09-13T18:21:17.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:17.511+0000 7f894b02f640 1 -- 172.21.15.114:0/984686246 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f892c0a1110 msgr2=0x7f892c0a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:17.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:17.511+0000 7f894b02f640 1 --2- 172.21.15.114:0/984686246 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f892c0a1110 0x7f892c0a35d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8934009990 tx=0x7f8934009110 comp rx=0 tx=0).stop 2024-09-13T18:21:17.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:17.511+0000 7f894b02f640 1 -- 172.21.15.114:0/984686246 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89440ff880 msgr2=0x7f8944195690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:17.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:17.511+0000 7f894b02f640 1 --2- 172.21.15.114:0/984686246 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89440ff880 0x7f8944195690 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f893800c890 tx=0x7f893800cc90 comp rx=0 tx=0).stop 2024-09-13T18:21:17.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:17.512+0000 7f894b02f640 1 -- 172.21.15.114:0/984686246 shutdown_connections 2024-09-13T18:21:17.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:17.512+0000 7f894b02f640 1 --2- 172.21.15.114:0/984686246 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f892c0a1110 0x7f892c0a35d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:17.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:17.512+0000 7f894b02f640 1 --2- 172.21.15.114:0/984686246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8944104000 0x7f8944199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:17.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:17.512+0000 7f894b02f640 1 --2- 172.21.15.114:0/984686246 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89440ff880 0x7f8944195690 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:17.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:17.512+0000 7f894b02f640 1 --2- 172.21.15.114:0/984686246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89440feed0 0x7f8944195150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:17.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:17.512+0000 7f894b02f640 1 -- 172.21.15.114:0/984686246 >> 172.21.15.114:0/984686246 conn(0x7f89440fa9b0 msgr2=0x7f89440fbd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:17.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:17.512+0000 7f894b02f640 1 -- 172.21.15.114:0/984686246 shutdown_connections 2024-09-13T18:21:17.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:17.512+0000 7f894b02f640 1 -- 172.21.15.114:0/984686246 wait complete. 2024-09-13T18:21:18.052 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:21:18.052 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:21:18.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:18 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/984686246' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-13T18:21:18.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:18 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/984686246' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-13T18:21:18.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:18 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/984686246' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-13T18:21:19.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:19 smithi114 ceph-mon[30897]: pgmap v317: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-09-13T18:21:19.295 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:21:19.297 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:21:19.297 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:21:19.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:19 smithi204 ceph-mon[34777]: pgmap v317: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-09-13T18:21:19.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:19 smithi134 ceph-mon[34283]: pgmap v317: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-09-13T18:21:19.722 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:21:19.723 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:21:20.312 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495665 got 197568495668 for osd.6 2024-09-13T18:21:20.312 DEBUG:teuthology.parallel:result is None 2024-09-13T18:21:21.418 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:21 smithi114 ceph-mon[30897]: pgmap v318: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-13T18:21:21.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:21 smithi204 ceph-mon[34777]: pgmap v318: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-13T18:21:21.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:21 smithi134 ceph-mon[34283]: pgmap v318: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-13T18:21:23.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.340+0000 7f15bb6c0640 1 -- 172.21.15.114:0/1635134543 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f15b40feef0 msgr2=0x7f15b40ff2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:23.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.340+0000 7f15bb6c0640 1 --2- 172.21.15.114:0/1635134543 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f15b40feef0 0x7f15b40ff2d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f15b0009930 tx=0x7f15b002a420 comp rx=0 tx=0).stop 2024-09-13T18:21:23.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.340+0000 7f15bb6c0640 1 -- 172.21.15.114:0/1635134543 shutdown_connections 2024-09-13T18:21:23.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.340+0000 7f15bb6c0640 1 --2- 172.21.15.114:0/1635134543 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15b4104020 0x7f15b410bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:23.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.340+0000 7f15bb6c0640 1 --2- 172.21.15.114:0/1635134543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15b40ff8a0 0x7f15b41038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:23.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.340+0000 7f15bb6c0640 1 --2- 172.21.15.114:0/1635134543 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f15b40feef0 0x7f15b40ff2d0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:23.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.340+0000 7f15bb6c0640 1 -- 172.21.15.114:0/1635134543 >> 172.21.15.114:0/1635134543 conn(0x7f15b40fa990 msgr2=0x7f15b40fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:23.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.340+0000 7f15bb6c0640 1 -- 172.21.15.114:0/1635134543 shutdown_connections 2024-09-13T18:21:23.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.340+0000 7f15bb6c0640 1 -- 172.21.15.114:0/1635134543 wait complete. 2024-09-13T18:21:23.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.340+0000 7f15bb6c0640 1 Processor -- start 2024-09-13T18:21:23.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.340+0000 7f15bb6c0640 1 -- start start 2024-09-13T18:21:23.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.341+0000 7f15bb6c0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15b40feef0 0x7f15b4195130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:23.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.341+0000 7f15bb6c0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f15b40ff8a0 0x7f15b4195670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:23.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.341+0000 7f15bb6c0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15b4104020 0x7f15b4199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:23.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.341+0000 7f15bb6c0640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b419a070 con 0x7f15b40ff8a0 2024-09-13T18:21:23.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.341+0000 7f15bb6c0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b419a1e0 con 0x7f15b40feef0 2024-09-13T18:21:23.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.341+0000 7f15bb6c0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b419a350 con 0x7f15b4104020 2024-09-13T18:21:23.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.341+0000 7f15b8c5b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f15b40ff8a0 0x7f15b4195670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:23.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.341+0000 7f15b8c5b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f15b40ff8a0 0x7f15b4195670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49074/0 (socket says 172.21.15.114:49074) 2024-09-13T18:21:23.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.341+0000 7f15b8c5b640 1 -- 172.21.15.114:0/1304561180 learned_addr learned my addr 172.21.15.114:0/1304561180 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:23.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.341+0000 7f15b9c5d640 1 --2- 172.21.15.114:0/1304561180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15b4104020 0x7f15b4199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:23.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.341+0000 7f15b945c640 1 --2- 172.21.15.114:0/1304561180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15b40feef0 0x7f15b4195130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:23.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.341+0000 7f15b8c5b640 1 -- 172.21.15.114:0/1304561180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15b4104020 msgr2=0x7f15b4199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:23.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.341+0000 7f15b8c5b640 1 --2- 172.21.15.114:0/1304561180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15b4104020 0x7f15b4199a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:23.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.341+0000 7f15b8c5b640 1 -- 172.21.15.114:0/1304561180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15b40feef0 msgr2=0x7f15b4195130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:23.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.341+0000 7f15b8c5b640 1 --2- 172.21.15.114:0/1304561180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15b40feef0 0x7f15b4195130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:23.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.341+0000 7f15b8c5b640 1 -- 172.21.15.114:0/1304561180 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b0009560 con 0x7f15b40ff8a0 2024-09-13T18:21:23.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.341+0000 7f15b945c640 1 --2- 172.21.15.114:0/1304561180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15b40feef0 0x7f15b4195130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:21:23.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.341+0000 7f15b9c5d640 1 --2- 172.21.15.114:0/1304561180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15b4104020 0x7f15b4199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:21:23.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.341+0000 7f15b8c5b640 1 --2- 172.21.15.114:0/1304561180 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f15b40ff8a0 0x7f15b4195670 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f15a400c890 tx=0x7f15a400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:23.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.341+0000 7f15aa7fc640 1 -- 172.21.15.114:0/1304561180 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a4004220 con 0x7f15b40ff8a0 2024-09-13T18:21:23.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.341+0000 7f15aa7fc640 1 -- 172.21.15.114:0/1304561180 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f15a4007e40 con 0x7f15b40ff8a0 2024-09-13T18:21:23.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.341+0000 7f15aa7fc640 1 -- 172.21.15.114:0/1304561180 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a4004a00 con 0x7f15b40ff8a0 2024-09-13T18:21:23.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.341+0000 7f15bb6c0640 1 -- 172.21.15.114:0/1304561180 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15b419a630 con 0x7f15b40ff8a0 2024-09-13T18:21:23.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.341+0000 7f15bb6c0640 1 -- 172.21.15.114:0/1304561180 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15b41a1ef0 con 0x7f15b40ff8a0 2024-09-13T18:21:23.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.342+0000 7f15bb6c0640 1 -- 172.21.15.114:0/1304561180 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15b41003d0 con 0x7f15b40ff8a0 2024-09-13T18:21:23.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.343+0000 7f15aa7fc640 1 -- 172.21.15.114:0/1304561180 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f15a40079b0 con 0x7f15b40ff8a0 2024-09-13T18:21:23.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.344+0000 7f15aa7fc640 1 --2- 172.21.15.114:0/1304561180 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f15a00a1110 0x7f15a00a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:23.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.344+0000 7f15b945c640 1 --2- 172.21.15.114:0/1304561180 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f15a00a1110 0x7f15a00a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:23.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.344+0000 7f15aa7fc640 1 -- 172.21.15.114:0/1304561180 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f15a40ccef0 con 0x7f15b40ff8a0 2024-09-13T18:21:23.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.345+0000 7f15b945c640 1 --2- 172.21.15.114:0/1304561180 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f15a00a1110 0x7f15a00a35d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f15b00057c0 tx=0x7f15b0000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:23.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.349+0000 7f15aa7fc640 1 -- 172.21.15.114:0/1304561180 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f15a4096210 con 0x7f15b40ff8a0 2024-09-13T18:21:23.419 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:23 smithi114 ceph-mon[30897]: pgmap v319: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-13T18:21:23.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:23 smithi204 ceph-mon[34777]: pgmap v319: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-13T18:21:23.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.516+0000 7f15bb6c0640 1 -- 172.21.15.114:0/1304561180 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f15b40ff2d0 con 0x7f15b40ff8a0 2024-09-13T18:21:23.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.516+0000 7f15aa7fc640 1 -- 172.21.15.114:0/1304561180 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f15a401d020 con 0x7f15b40ff8a0 2024-09-13T18:21:23.519 INFO:teuthology.orchestra.run.smithi114.stdout:107374182466 2024-09-13T18:21:23.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.518+0000 7f15bb6c0640 1 -- 172.21.15.114:0/1304561180 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f15a00a1110 msgr2=0x7f15a00a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:23.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.518+0000 7f15bb6c0640 1 --2- 172.21.15.114:0/1304561180 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f15a00a1110 0x7f15a00a35d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f15b00057c0 tx=0x7f15b0000bc0 comp rx=0 tx=0).stop 2024-09-13T18:21:23.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.518+0000 7f15bb6c0640 1 -- 172.21.15.114:0/1304561180 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f15b40ff8a0 msgr2=0x7f15b4195670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:23.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.518+0000 7f15bb6c0640 1 --2- 172.21.15.114:0/1304561180 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f15b40ff8a0 0x7f15b4195670 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f15a400c890 tx=0x7f15a400cd50 comp rx=0 tx=0).stop 2024-09-13T18:21:23.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.518+0000 7f15bb6c0640 1 -- 172.21.15.114:0/1304561180 shutdown_connections 2024-09-13T18:21:23.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.518+0000 7f15bb6c0640 1 --2- 172.21.15.114:0/1304561180 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f15a00a1110 0x7f15a00a35d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:23.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.518+0000 7f15bb6c0640 1 --2- 172.21.15.114:0/1304561180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15b4104020 0x7f15b4199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:23.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.518+0000 7f15bb6c0640 1 --2- 172.21.15.114:0/1304561180 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f15b40ff8a0 0x7f15b4195670 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:23.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.518+0000 7f15bb6c0640 1 --2- 172.21.15.114:0/1304561180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15b40feef0 0x7f15b4195130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:23.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.518+0000 7f15bb6c0640 1 -- 172.21.15.114:0/1304561180 >> 172.21.15.114:0/1304561180 conn(0x7f15b40fa990 msgr2=0x7f15b40fbeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:23.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.518+0000 7f15bb6c0640 1 -- 172.21.15.114:0/1304561180 shutdown_connections 2024-09-13T18:21:23.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:23.518+0000 7f15bb6c0640 1 -- 172.21.15.114:0/1304561180 wait complete. 2024-09-13T18:21:23.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:23 smithi134 ceph-mon[34283]: pgmap v319: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-13T18:21:24.429 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:24 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/1304561180' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-13T18:21:24.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:24 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/1304561180' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-13T18:21:24.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:24 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/1304561180' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-13T18:21:25.293 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:25 smithi114 ceph-mon[30897]: pgmap v320: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-13T18:21:25.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:25 smithi204 ceph-mon[34777]: pgmap v320: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-13T18:21:25.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:25 smithi134 ceph-mon[34283]: pgmap v320: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-13T18:21:27.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:26.902+0000 7f0c17250640 1 -- 172.21.15.114:0/3886057887 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c10104000 msgr2=0x7f0c1010bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:27.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:26.902+0000 7f0c17250640 1 --2- 172.21.15.114:0/3886057887 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c10104000 0x7f0c1010bb80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0c08009930 tx=0x7f0c0802a420 comp rx=0 tx=0).stop 2024-09-13T18:21:27.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:26.902+0000 7f0c17250640 1 -- 172.21.15.114:0/3886057887 shutdown_connections 2024-09-13T18:21:27.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:26.902+0000 7f0c17250640 1 --2- 172.21.15.114:0/3886057887 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c10104000 0x7f0c1010bb80 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:27.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:26.902+0000 7f0c17250640 1 --2- 172.21.15.114:0/3886057887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c100ff880 0x7f0c101038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:27.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:26.902+0000 7f0c17250640 1 --2- 172.21.15.114:0/3886057887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c100feed0 0x7f0c100ff2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:27.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:26.902+0000 7f0c17250640 1 -- 172.21.15.114:0/3886057887 >> 172.21.15.114:0/3886057887 conn(0x7f0c100fa9b0 msgr2=0x7f0c100fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:27.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:26.902+0000 7f0c17250640 1 -- 172.21.15.114:0/3886057887 shutdown_connections 2024-09-13T18:21:27.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:26.902+0000 7f0c17250640 1 -- 172.21.15.114:0/3886057887 wait complete. 2024-09-13T18:21:27.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:26.902+0000 7f0c17250640 1 Processor -- start 2024-09-13T18:21:27.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:26.902+0000 7f0c17250640 1 -- start start 2024-09-13T18:21:27.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:26.903+0000 7f0c17250640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c100feed0 0x7f0c10196ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:27.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:26.903+0000 7f0c17250640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c100ff880 0x7f0c10197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:27.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:26.903+0000 7f0c17250640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c10104000 0x7f0c101a1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:27.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:26.903+0000 7f0c17250640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c10197c50 con 0x7f0c100ff880 2024-09-13T18:21:27.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:26.903+0000 7f0c17250640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c101a24a0 con 0x7f0c100feed0 2024-09-13T18:21:27.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:26.903+0000 7f0c17250640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c101a25e0 con 0x7f0c10104000 2024-09-13T18:21:27.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:26.903+0000 7f0c0ffff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c100ff880 0x7f0c10197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:27.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:26.903+0000 7f0c0ffff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c100ff880 0x7f0c10197420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49092/0 (socket says 172.21.15.114:49092) 2024-09-13T18:21:27.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:26.903+0000 7f0c0ffff640 1 -- 172.21.15.114:0/3619583409 learned_addr learned my addr 172.21.15.114:0/3619583409 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:27.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:26.903+0000 7f0c14fec640 1 --2- 172.21.15.114:0/3619583409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c100feed0 0x7f0c10196ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:27.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:26.903+0000 7f0c157ed640 1 --2- 172.21.15.114:0/3619583409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c10104000 0x7f0c101a1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:27.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:26.903+0000 7f0c0ffff640 1 -- 172.21.15.114:0/3619583409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c10104000 msgr2=0x7f0c101a1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:27.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:26.903+0000 7f0c0ffff640 1 --2- 172.21.15.114:0/3619583409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c10104000 0x7f0c101a1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:27.262 INFO:teuthology.orchestra.run.smithi114.stdout:309237645347 2024-09-13T18:21:27.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:26.903+0000 7f0c0ffff640 1 -- 172.21.15.114:0/3619583409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c100feed0 msgr2=0x7f0c10196ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:27.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:26.903+0000 7f0c0ffff640 1 --2- 172.21.15.114:0/3619583409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c100feed0 0x7f0c10196ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:27.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:26.903+0000 7f0c0ffff640 1 -- 172.21.15.114:0/3619583409 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c08009560 con 0x7f0c100ff880 2024-09-13T18:21:27.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:26.903+0000 7f0c0ffff640 1 --2- 172.21.15.114:0/3619583409 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c100ff880 0x7f0c10197420 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0c0400c920 tx=0x7f0c0400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:27.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:26.903+0000 7f0c0dffb640 1 -- 172.21.15.114:0/3619583409 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c04007c70 con 0x7f0c100ff880 2024-09-13T18:21:27.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:26.903+0000 7f0c17250640 1 -- 172.21.15.114:0/3619583409 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c101a2840 con 0x7f0c100ff880 2024-09-13T18:21:27.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:26.903+0000 7f0c0dffb640 1 -- 172.21.15.114:0/3619583409 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0c04007dd0 con 0x7f0c100ff880 2024-09-13T18:21:27.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:26.903+0000 7f0c0dffb640 1 -- 172.21.15.114:0/3619583409 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c04018730 con 0x7f0c100ff880 2024-09-13T18:21:27.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:26.904+0000 7f0c17250640 1 -- 172.21.15.114:0/3619583409 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c101a2d60 con 0x7f0c100ff880 2024-09-13T18:21:27.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:26.904+0000 7f0c17250640 1 -- 172.21.15.114:0/3619583409 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c101003b0 con 0x7f0c100ff880 2024-09-13T18:21:27.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:26.906+0000 7f0c0dffb640 1 -- 172.21.15.114:0/3619583409 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f0c0400f450 con 0x7f0c100ff880 2024-09-13T18:21:27.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:26.906+0000 7f0c0dffb640 1 --2- 172.21.15.114:0/3619583409 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f0bf80a1190 0x7f0bf80a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:27.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:26.906+0000 7f0c14fec640 1 --2- 172.21.15.114:0/3619583409 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f0bf80a1190 0x7f0bf80a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:27.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:26.906+0000 7f0c0dffb640 1 -- 172.21.15.114:0/3619583409 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0c040ccdf0 con 0x7f0c100ff880 2024-09-13T18:21:27.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:26.907+0000 7f0c14fec640 1 --2- 172.21.15.114:0/3619583409 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f0bf80a1190 0x7f0bf80a3650 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0c00007ec0 tx=0x7f0c000075b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:27.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:26.910+0000 7f0c0dffb640 1 -- 172.21.15.114:0/3619583409 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f0c04096110 con 0x7f0c100ff880 2024-09-13T18:21:27.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:27.081+0000 7f0c17250640 1 -- 172.21.15.114:0/3619583409 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f0c100ff2b0 con 0x7f0c100ff880 2024-09-13T18:21:27.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:27.081+0000 7f0c0dffb640 1 -- 172.21.15.114:0/3619583409 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f0c04099730 con 0x7f0c100ff880 2024-09-13T18:21:27.264 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:27.083+0000 7f0c17250640 1 -- 172.21.15.114:0/3619583409 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f0bf80a1190 msgr2=0x7f0bf80a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:27.264 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:27.083+0000 7f0c17250640 1 --2- 172.21.15.114:0/3619583409 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f0bf80a1190 0x7f0bf80a3650 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0c00007ec0 tx=0x7f0c000075b0 comp rx=0 tx=0).stop 2024-09-13T18:21:27.264 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:27.083+0000 7f0c17250640 1 -- 172.21.15.114:0/3619583409 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c100ff880 msgr2=0x7f0c10197420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:27.264 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:27.083+0000 7f0c17250640 1 --2- 172.21.15.114:0/3619583409 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c100ff880 0x7f0c10197420 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0c0400c920 tx=0x7f0c0400cde0 comp rx=0 tx=0).stop 2024-09-13T18:21:27.264 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:27.083+0000 7f0c17250640 1 -- 172.21.15.114:0/3619583409 shutdown_connections 2024-09-13T18:21:27.264 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:27.083+0000 7f0c17250640 1 --2- 172.21.15.114:0/3619583409 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f0bf80a1190 0x7f0bf80a3650 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:27.264 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:27.083+0000 7f0c17250640 1 --2- 172.21.15.114:0/3619583409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c10104000 0x7f0c101a1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:27.264 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:27.083+0000 7f0c17250640 1 --2- 172.21.15.114:0/3619583409 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c100ff880 0x7f0c10197420 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:27.264 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:27.083+0000 7f0c17250640 1 --2- 172.21.15.114:0/3619583409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c100feed0 0x7f0c10196ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:27.264 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:27.083+0000 7f0c17250640 1 -- 172.21.15.114:0/3619583409 >> 172.21.15.114:0/3619583409 conn(0x7f0c100fa9b0 msgr2=0x7f0c100fbcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:27.264 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:27.083+0000 7f0c17250640 1 -- 172.21.15.114:0/3619583409 shutdown_connections 2024-09-13T18:21:27.264 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:27.083+0000 7f0c17250640 1 -- 172.21.15.114:0/3619583409 wait complete. 2024-09-13T18:21:27.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:27 smithi204 ceph-mon[34777]: pgmap v321: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-13T18:21:27.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:27 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]: dispatch 2024-09-13T18:21:27.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:27 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-09-13T18:21:27.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:27 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-09-13T18:21:27.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:27 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/3619583409' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-13T18:21:27.469 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:27 smithi114 ceph-mon[30897]: pgmap v321: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-13T18:21:27.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:27 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]: dispatch 2024-09-13T18:21:27.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:27 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-09-13T18:21:27.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:27 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-09-13T18:21:27.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:27 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/3619583409' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-13T18:21:27.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:27 smithi134 ceph-mon[34283]: pgmap v321: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-13T18:21:27.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:27 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]: dispatch 2024-09-13T18:21:27.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:27 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-09-13T18:21:27.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:27 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-09-13T18:21:27.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:27 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/3619583409' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-13T18:21:28.445 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:28 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]': finished 2024-09-13T18:21:28.445 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:28 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-09-13T18:21:28.445 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:28 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-09-13T18:21:28.445 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:28 smithi114 ceph-mon[30897]: osdmap e92: 12 total, 12 up, 12 in 2024-09-13T18:21:28.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:28 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]': finished 2024-09-13T18:21:28.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:28 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-09-13T18:21:28.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:28 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-09-13T18:21:28.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:28 smithi204 ceph-mon[34777]: osdmap e92: 12 total, 12 up, 12 in 2024-09-13T18:21:28.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:28 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]': finished 2024-09-13T18:21:28.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:28 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-09-13T18:21:28.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:28 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-09-13T18:21:28.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:28 smithi134 ceph-mon[34283]: osdmap e92: 12 total, 12 up, 12 in 2024-09-13T18:21:29.260 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182462 got 107374182466 for osd.3 2024-09-13T18:21:29.260 DEBUG:teuthology.parallel:result is None 2024-09-13T18:21:29.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:29 smithi204 ceph-mon[34777]: pgmap v323: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-13T18:21:29.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:29 smithi204 ceph-mon[34777]: osdmap e93: 12 total, 12 up, 12 in 2024-09-13T18:21:29.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:29 smithi114 ceph-mon[30897]: pgmap v323: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-13T18:21:29.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:29 smithi114 ceph-mon[30897]: osdmap e93: 12 total, 12 up, 12 in 2024-09-13T18:21:29.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:29 smithi134 ceph-mon[34283]: pgmap v323: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-13T18:21:29.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:29 smithi134 ceph-mon[34283]: osdmap e93: 12 total, 12 up, 12 in 2024-09-13T18:21:29.751 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645343 got 309237645347 for osd.10 2024-09-13T18:21:29.751 DEBUG:teuthology.parallel:result is None 2024-09-13T18:21:30.311 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:30 smithi114 ceph-mon[30897]: pgmap v325: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 9 op/s 2024-09-13T18:21:30.311 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:30 smithi114 ceph-mon[30897]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-13T18:21:30.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:30 smithi134 ceph-mon[34283]: pgmap v325: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 9 op/s 2024-09-13T18:21:30.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:30 smithi134 ceph-mon[34283]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-13T18:21:30.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:30 smithi204 ceph-mon[34777]: pgmap v325: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 9 op/s 2024-09-13T18:21:30.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:30 smithi204 ceph-mon[34777]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-13T18:21:31.515 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:21:33.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.160+0000 7f1e0f70f640 1 -- 172.21.15.114:0/1060999118 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e080fef00 msgr2=0x7f1e080ff2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:33.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.160+0000 7f1e0f70f640 1 --2- 172.21.15.114:0/1060999118 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e080fef00 0x7f1e080ff2e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1df8009930 tx=0x7f1df802a420 comp rx=0 tx=0).stop 2024-09-13T18:21:33.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.161+0000 7f1e0f70f640 1 -- 172.21.15.114:0/1060999118 shutdown_connections 2024-09-13T18:21:33.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.161+0000 7f1e0f70f640 1 --2- 172.21.15.114:0/1060999118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e08104030 0x7f1e0810bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:33.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.161+0000 7f1e0f70f640 1 --2- 172.21.15.114:0/1060999118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e080ff8b0 0x7f1e08103900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:33.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.161+0000 7f1e0f70f640 1 --2- 172.21.15.114:0/1060999118 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e080fef00 0x7f1e080ff2e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:33.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.161+0000 7f1e0f70f640 1 -- 172.21.15.114:0/1060999118 >> 172.21.15.114:0/1060999118 conn(0x7f1e080fa9a0 msgr2=0x7f1e080fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:33.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.161+0000 7f1e0f70f640 1 -- 172.21.15.114:0/1060999118 shutdown_connections 2024-09-13T18:21:33.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.161+0000 7f1e0f70f640 1 -- 172.21.15.114:0/1060999118 wait complete. 2024-09-13T18:21:33.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.161+0000 7f1e0f70f640 1 Processor -- start 2024-09-13T18:21:33.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.161+0000 7f1e0f70f640 1 -- start start 2024-09-13T18:21:33.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.161+0000 7f1e0f70f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e080fef00 0x7f1e081961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:33.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.161+0000 7f1e0f70f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e080ff8b0 0x7f1e08196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:33.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.161+0000 7f1e0f70f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e08104030 0x7f1e0819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:33.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.161+0000 7f1e0f70f640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0819b120 con 0x7f1e080ff8b0 2024-09-13T18:21:33.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.161+0000 7f1e0f70f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0819b290 con 0x7f1e08104030 2024-09-13T18:21:33.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.161+0000 7f1e0f70f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0819b400 con 0x7f1e080fef00 2024-09-13T18:21:33.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.161+0000 7f1e0ccaa640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e080ff8b0 0x7f1e08196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:33.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.161+0000 7f1e0ccaa640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e080ff8b0 0x7f1e08196720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49126/0 (socket says 172.21.15.114:49126) 2024-09-13T18:21:33.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.162+0000 7f1e0ccaa640 1 -- 172.21.15.114:0/1823176802 learned_addr learned my addr 172.21.15.114:0/1823176802 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:33.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.162+0000 7f1e0d4ab640 1 --2- 172.21.15.114:0/1823176802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e080fef00 0x7f1e081961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:33.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.162+0000 7f1e0ccaa640 1 -- 172.21.15.114:0/1823176802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e080fef00 msgr2=0x7f1e081961e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:33.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.162+0000 7f1e0ccaa640 1 --2- 172.21.15.114:0/1823176802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e080fef00 0x7f1e081961e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:33.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.162+0000 7f1e0dcac640 1 --2- 172.21.15.114:0/1823176802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e08104030 0x7f1e0819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:33.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.162+0000 7f1e0ccaa640 1 -- 172.21.15.114:0/1823176802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e08104030 msgr2=0x7f1e0819ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:33.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.162+0000 7f1e0ccaa640 1 --2- 172.21.15.114:0/1823176802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e08104030 0x7f1e0819ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:33.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.162+0000 7f1e0ccaa640 1 -- 172.21.15.114:0/1823176802 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df8009560 con 0x7f1e080ff8b0 2024-09-13T18:21:33.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.162+0000 7f1e0dcac640 1 --2- 172.21.15.114:0/1823176802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e08104030 0x7f1e0819ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:21:33.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.162+0000 7f1e0ccaa640 1 --2- 172.21.15.114:0/1823176802 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e080ff8b0 0x7f1e08196720 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1e0400d950 tx=0x7f1e0400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:33.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.162+0000 7f1dfe7fc640 1 -- 172.21.15.114:0/1823176802 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e04009c30 con 0x7f1e080ff8b0 2024-09-13T18:21:33.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.162+0000 7f1dfe7fc640 1 -- 172.21.15.114:0/1823176802 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1e04009d90 con 0x7f1e080ff8b0 2024-09-13T18:21:33.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.162+0000 7f1e0f70f640 1 -- 172.21.15.114:0/1823176802 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e0819b6e0 con 0x7f1e080ff8b0 2024-09-13T18:21:33.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.162+0000 7f1dfe7fc640 1 -- 172.21.15.114:0/1823176802 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0400f620 con 0x7f1e080ff8b0 2024-09-13T18:21:33.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.162+0000 7f1e0f70f640 1 -- 172.21.15.114:0/1823176802 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e081a2fa0 con 0x7f1e080ff8b0 2024-09-13T18:21:33.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.163+0000 7f1e0f70f640 1 -- 172.21.15.114:0/1823176802 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e081003e0 con 0x7f1e080ff8b0 2024-09-13T18:21:33.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.164+0000 7f1dfe7fc640 1 -- 172.21.15.114:0/1823176802 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f1e040109f0 con 0x7f1e080ff8b0 2024-09-13T18:21:33.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.165+0000 7f1dfe7fc640 1 --2- 172.21.15.114:0/1823176802 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f1df40a1110 0x7f1df40a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:33.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.165+0000 7f1dfe7fc640 1 -- 172.21.15.114:0/1823176802 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f1e040cca90 con 0x7f1e080ff8b0 2024-09-13T18:21:33.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.165+0000 7f1e0d4ab640 1 --2- 172.21.15.114:0/1823176802 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f1df40a1110 0x7f1df40a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:33.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.165+0000 7f1e0d4ab640 1 --2- 172.21.15.114:0/1823176802 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f1df40a1110 0x7f1df40a35d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1df8006010 tx=0x7f1df8000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:33.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.168+0000 7f1dfe7fc640 1 -- 172.21.15.114:0/1823176802 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f1e04095d50 con 0x7f1e080ff8b0 2024-09-13T18:21:33.298 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:32 smithi114 ceph-mon[30897]: pgmap v326: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 9 op/s; 14 B/s, 0 objects/s recovering 2024-09-13T18:21:33.298 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:32 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/1284716419' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-13T18:21:33.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.346+0000 7f1e0f70f640 1 -- 172.21.15.114:0/1823176802 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f1e080ff2e0 con 0x7f1e080ff8b0 2024-09-13T18:21:33.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.346+0000 7f1dfe7fc640 1 -- 172.21.15.114:0/1823176802 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1e04020090 con 0x7f1e080ff8b0 2024-09-13T18:21:33.349 INFO:teuthology.orchestra.run.smithi114.stdout:219043332146 2024-09-13T18:21:33.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.348+0000 7f1e0f70f640 1 -- 172.21.15.114:0/1823176802 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f1df40a1110 msgr2=0x7f1df40a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:33.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.348+0000 7f1e0f70f640 1 --2- 172.21.15.114:0/1823176802 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f1df40a1110 0x7f1df40a35d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1df8006010 tx=0x7f1df8000bc0 comp rx=0 tx=0).stop 2024-09-13T18:21:33.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.348+0000 7f1e0f70f640 1 -- 172.21.15.114:0/1823176802 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e080ff8b0 msgr2=0x7f1e08196720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:33.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.348+0000 7f1e0f70f640 1 --2- 172.21.15.114:0/1823176802 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e080ff8b0 0x7f1e08196720 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1e0400d950 tx=0x7f1e0400de10 comp rx=0 tx=0).stop 2024-09-13T18:21:33.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.348+0000 7f1e0f70f640 1 -- 172.21.15.114:0/1823176802 shutdown_connections 2024-09-13T18:21:33.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.348+0000 7f1e0f70f640 1 --2- 172.21.15.114:0/1823176802 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f1df40a1110 0x7f1df40a35d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:33.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.348+0000 7f1e0f70f640 1 --2- 172.21.15.114:0/1823176802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e08104030 0x7f1e0819ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:33.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.348+0000 7f1e0f70f640 1 --2- 172.21.15.114:0/1823176802 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e080ff8b0 0x7f1e08196720 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:33.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.348+0000 7f1e0f70f640 1 --2- 172.21.15.114:0/1823176802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e080fef00 0x7f1e081961e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:33.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.348+0000 7f1e0f70f640 1 -- 172.21.15.114:0/1823176802 >> 172.21.15.114:0/1823176802 conn(0x7f1e080fa9a0 msgr2=0x7f1e080fbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:33.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.348+0000 7f1e0f70f640 1 -- 172.21.15.114:0/1823176802 shutdown_connections 2024-09-13T18:21:33.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.349+0000 7f1e0f70f640 1 -- 172.21.15.114:0/1823176802 wait complete. 2024-09-13T18:21:33.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:32 smithi134 ceph-mon[34283]: pgmap v326: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 9 op/s; 14 B/s, 0 objects/s recovering 2024-09-13T18:21:33.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:32 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/1284716419' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-13T18:21:33.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:32 smithi204 ceph-mon[34777]: pgmap v326: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 9 op/s; 14 B/s, 0 objects/s recovering 2024-09-13T18:21:33.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:32 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/1284716419' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-13T18:21:33.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.809+0000 7ff129a67640 1 -- 172.21.15.114:0/1447359014 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff124100920 msgr2=0x7ff124104970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:33.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.809+0000 7ff129a67640 1 --2- 172.21.15.114:0/1447359014 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff124100920 0x7ff124104970 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff10c009930 tx=0x7ff10c02a420 comp rx=0 tx=0).stop 2024-09-13T18:21:33.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.809+0000 7ff129a67640 1 -- 172.21.15.114:0/1447359014 shutdown_connections 2024-09-13T18:21:33.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.809+0000 7ff129a67640 1 --2- 172.21.15.114:0/1447359014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1241050a0 0x7ff12410cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:33.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.809+0000 7ff129a67640 1 --2- 172.21.15.114:0/1447359014 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff124100920 0x7ff124104970 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:33.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.809+0000 7ff129a67640 1 --2- 172.21.15.114:0/1447359014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1240fff70 0x7ff124100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:33.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.809+0000 7ff129a67640 1 -- 172.21.15.114:0/1447359014 >> 172.21.15.114:0/1447359014 conn(0x7ff1240fba30 msgr2=0x7ff1240fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:33.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.809+0000 7ff129a67640 1 -- 172.21.15.114:0/1447359014 shutdown_connections 2024-09-13T18:21:33.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.809+0000 7ff129a67640 1 -- 172.21.15.114:0/1447359014 wait complete. 2024-09-13T18:21:33.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.809+0000 7ff129a67640 1 Processor -- start 2024-09-13T18:21:33.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.810+0000 7ff129a67640 1 -- start start 2024-09-13T18:21:33.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.810+0000 7ff129a67640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1240fff70 0x7ff1240727b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:33.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.810+0000 7ff129a67640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff124100920 0x7ff124072cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:33.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.810+0000 7ff129a67640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1241050a0 0x7ff124073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:33.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.810+0000 7ff129a67640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1241a45e0 con 0x7ff1241050a0 2024-09-13T18:21:33.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.810+0000 7ff129a67640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1241a4750 con 0x7ff1240fff70 2024-09-13T18:21:33.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.810+0000 7ff129a67640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1241a4910 con 0x7ff124100920 2024-09-13T18:21:33.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.810+0000 7ff123fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1241050a0 0x7ff124073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:33.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.810+0000 7ff123fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1241050a0 0x7ff124073250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50304/0 (socket says 172.21.15.114:50304) 2024-09-13T18:21:33.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.810+0000 7ff123fff640 1 -- 172.21.15.114:0/2802267973 learned_addr learned my addr 172.21.15.114:0/2802267973 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:33.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.810+0000 7ff1237fe640 1 --2- 172.21.15.114:0/2802267973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1240fff70 0x7ff1240727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:33.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.810+0000 7ff11bfff640 1 --2- 172.21.15.114:0/2802267973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff124100920 0x7ff124072cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:33.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.810+0000 7ff123fff640 1 -- 172.21.15.114:0/2802267973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff124100920 msgr2=0x7ff124072cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:33.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.810+0000 7ff123fff640 1 --2- 172.21.15.114:0/2802267973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff124100920 0x7ff124072cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:33.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.810+0000 7ff123fff640 1 -- 172.21.15.114:0/2802267973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1240fff70 msgr2=0x7ff1240727b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:33.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.810+0000 7ff123fff640 1 --2- 172.21.15.114:0/2802267973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1240fff70 0x7ff1240727b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:33.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.810+0000 7ff123fff640 1 -- 172.21.15.114:0/2802267973 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff10c009560 con 0x7ff1241050a0 2024-09-13T18:21:33.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.810+0000 7ff123fff640 1 --2- 172.21.15.114:0/2802267973 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1241050a0 0x7ff124073250 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff11c00eb10 tx=0x7ff11c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:33.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.811+0000 7ff1217fa640 1 -- 172.21.15.114:0/2802267973 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff11c004b20 con 0x7ff1241050a0 2024-09-13T18:21:33.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.811+0000 7ff1217fa640 1 -- 172.21.15.114:0/2802267973 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff11c004c80 con 0x7ff1241050a0 2024-09-13T18:21:33.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.811+0000 7ff129a67640 1 -- 172.21.15.114:0/2802267973 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1241a4bf0 con 0x7ff1241050a0 2024-09-13T18:21:33.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.811+0000 7ff1217fa640 1 -- 172.21.15.114:0/2802267973 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff11c005a10 con 0x7ff1241050a0 2024-09-13T18:21:33.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.811+0000 7ff129a67640 1 -- 172.21.15.114:0/2802267973 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1241a5070 con 0x7ff1241050a0 2024-09-13T18:21:33.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.812+0000 7ff129a67640 1 -- 172.21.15.114:0/2802267973 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff124101450 con 0x7ff1241050a0 2024-09-13T18:21:33.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.813+0000 7ff1217fa640 1 -- 172.21.15.114:0/2802267973 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7ff11c005020 con 0x7ff1241050a0 2024-09-13T18:21:33.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.813+0000 7ff1217fa640 1 --2- 172.21.15.114:0/2802267973 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7ff1100a1110 0x7ff1100a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:33.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.814+0000 7ff1237fe640 1 --2- 172.21.15.114:0/2802267973 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7ff1100a1110 0x7ff1100a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:33.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.814+0000 7ff1217fa640 1 -- 172.21.15.114:0/2802267973 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7ff11c014070 con 0x7ff1241050a0 2024-09-13T18:21:33.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.814+0000 7ff1237fe640 1 --2- 172.21.15.114:0/2802267973 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7ff1100a1110 0x7ff1100a35d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff114009bd0 tx=0x7ff114005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:33.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.818+0000 7ff1217fa640 1 -- 172.21.15.114:0/2802267973 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7ff11c0966b0 con 0x7ff1241050a0 2024-09-13T18:21:33.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.991+0000 7ff129a67640 1 -- 172.21.15.114:0/2802267973 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff124100350 con 0x7ff1241050a0 2024-09-13T18:21:33.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.991+0000 7ff1217fa640 1 -- 172.21.15.114:0/2802267973 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff11c099cd0 con 0x7ff1241050a0 2024-09-13T18:21:33.994 INFO:teuthology.orchestra.run.smithi114.stdout:77309411401 2024-09-13T18:21:33.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.994+0000 7ff129a67640 1 -- 172.21.15.114:0/2802267973 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7ff1100a1110 msgr2=0x7ff1100a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:33.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.994+0000 7ff129a67640 1 --2- 172.21.15.114:0/2802267973 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7ff1100a1110 0x7ff1100a35d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff114009bd0 tx=0x7ff114005f90 comp rx=0 tx=0).stop 2024-09-13T18:21:33.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.994+0000 7ff129a67640 1 -- 172.21.15.114:0/2802267973 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1241050a0 msgr2=0x7ff124073250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:33.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.994+0000 7ff129a67640 1 --2- 172.21.15.114:0/2802267973 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1241050a0 0x7ff124073250 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff11c00eb10 tx=0x7ff11c00c2d0 comp rx=0 tx=0).stop 2024-09-13T18:21:33.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.994+0000 7ff129a67640 1 -- 172.21.15.114:0/2802267973 shutdown_connections 2024-09-13T18:21:33.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.994+0000 7ff129a67640 1 --2- 172.21.15.114:0/2802267973 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7ff1100a1110 0x7ff1100a35d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:33.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.994+0000 7ff129a67640 1 --2- 172.21.15.114:0/2802267973 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1241050a0 0x7ff124073250 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:33.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.994+0000 7ff129a67640 1 --2- 172.21.15.114:0/2802267973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff124100920 0x7ff124072cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:33.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.994+0000 7ff129a67640 1 --2- 172.21.15.114:0/2802267973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1240fff70 0x7ff1240727b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:33.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.994+0000 7ff129a67640 1 -- 172.21.15.114:0/2802267973 >> 172.21.15.114:0/2802267973 conn(0x7ff1240fba30 msgr2=0x7ff1240fce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:33.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.994+0000 7ff129a67640 1 -- 172.21.15.114:0/2802267973 shutdown_connections 2024-09-13T18:21:33.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.994+0000 7ff129a67640 1 -- 172.21.15.114:0/2802267973 wait complete. 2024-09-13T18:21:34.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.987+0000 7f1066745640 1 -- 172.21.15.114:0/1760522058 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1060104000 msgr2=0x7f106010bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:34.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.987+0000 7f1066745640 1 --2- 172.21.15.114:0/1760522058 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1060104000 0x7f106010bb80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1058009cd0 tx=0x7f105802a8e0 comp rx=0 tx=0).stop 2024-09-13T18:21:34.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.987+0000 7f1066745640 1 -- 172.21.15.114:0/1760522058 shutdown_connections 2024-09-13T18:21:34.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.987+0000 7f1066745640 1 --2- 172.21.15.114:0/1760522058 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1060104000 0x7f106010bb80 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:34.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.987+0000 7f1066745640 1 --2- 172.21.15.114:0/1760522058 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10600ff880 0x7f10601038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:34.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.987+0000 7f1066745640 1 --2- 172.21.15.114:0/1760522058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10600feed0 0x7f10600ff2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:34.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.987+0000 7f1066745640 1 -- 172.21.15.114:0/1760522058 >> 172.21.15.114:0/1760522058 conn(0x7f10600fa9b0 msgr2=0x7f10600fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:34.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.987+0000 7f1066745640 1 -- 172.21.15.114:0/1760522058 shutdown_connections 2024-09-13T18:21:34.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.987+0000 7f1066745640 1 -- 172.21.15.114:0/1760522058 wait complete. 2024-09-13T18:21:34.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.988+0000 7f1066745640 1 Processor -- start 2024-09-13T18:21:34.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.988+0000 7f1066745640 1 -- start start 2024-09-13T18:21:34.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.988+0000 7f1066745640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10600feed0 0x7f1060197000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:34.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.988+0000 7f1066745640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10600ff880 0x7f1060197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:34.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.988+0000 7f1066745640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1060104000 0x7f1060197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:34.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.988+0000 7f1066745640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10601a22d0 con 0x7f1060104000 2024-09-13T18:21:34.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.988+0000 7f1066745640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10601a2440 con 0x7f10600feed0 2024-09-13T18:21:34.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.988+0000 7f1066745640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10601a2600 con 0x7f10600ff880 2024-09-13T18:21:34.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.989+0000 7f105ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10600feed0 0x7f1060197000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:34.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.989+0000 7f1064ce2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1060104000 0x7f1060197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:34.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.989+0000 7f105f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10600ff880 0x7f1060197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:34.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.989+0000 7f105ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10600feed0 0x7f1060197000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:49662/0 (socket says 172.21.15.114:49662) 2024-09-13T18:21:34.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.989+0000 7f1064ce2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1060104000 0x7f1060197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50314/0 (socket says 172.21.15.114:50314) 2024-09-13T18:21:34.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.989+0000 7f105ffff640 1 -- 172.21.15.114:0/714942497 learned_addr learned my addr 172.21.15.114:0/714942497 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:34.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.989+0000 7f105f7fe640 1 -- 172.21.15.114:0/714942497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10600feed0 msgr2=0x7f1060197000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:34.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.989+0000 7f105f7fe640 1 --2- 172.21.15.114:0/714942497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10600feed0 0x7f1060197000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:34.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.989+0000 7f105f7fe640 1 -- 172.21.15.114:0/714942497 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1060104000 msgr2=0x7f1060197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:34.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.989+0000 7f105f7fe640 1 --2- 172.21.15.114:0/714942497 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1060104000 0x7f1060197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:34.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.989+0000 7f105f7fe640 1 -- 172.21.15.114:0/714942497 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1058009900 con 0x7f10600ff880 2024-09-13T18:21:34.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.989+0000 7f105ffff640 1 --2- 172.21.15.114:0/714942497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10600feed0 0x7f1060197000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:21:34.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.989+0000 7f1064ce2640 1 --2- 172.21.15.114:0/714942497 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1060104000 0x7f1060197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:21:34.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.989+0000 7f105f7fe640 1 --2- 172.21.15.114:0/714942497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10600ff880 0x7f1060197540 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f105400e9f0 tx=0x7f105400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:34.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.991+0000 7f105d7fa640 1 -- 172.21.15.114:0/714942497 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105400ccc0 con 0x7f10600ff880 2024-09-13T18:21:34.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.991+0000 7f105d7fa640 1 -- 172.21.15.114:0/714942497 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f10540044b0 con 0x7f10600ff880 2024-09-13T18:21:34.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.991+0000 7f1066745640 1 -- 172.21.15.114:0/714942497 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1060194e50 con 0x7f10600ff880 2024-09-13T18:21:34.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.991+0000 7f105d7fa640 1 -- 172.21.15.114:0/714942497 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1054004a00 con 0x7f10600ff880 2024-09-13T18:21:34.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.991+0000 7f1066745640 1 -- 172.21.15.114:0/714942497 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1060195350 con 0x7f10600ff880 2024-09-13T18:21:34.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.992+0000 7f1066745640 1 -- 172.21.15.114:0/714942497 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10601003b0 con 0x7f10600ff880 2024-09-13T18:21:34.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.993+0000 7f105d7fa640 1 -- 172.21.15.114:0/714942497 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f1054005020 con 0x7f10600ff880 2024-09-13T18:21:34.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.994+0000 7f105d7fa640 1 --2- 172.21.15.114:0/714942497 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f10480a1190 0x7f10480a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:34.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.994+0000 7f105d7fa640 1 -- 172.21.15.114:0/714942497 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f1054014070 con 0x7f10600ff880 2024-09-13T18:21:34.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.994+0000 7f105ffff640 1 --2- 172.21.15.114:0/714942497 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f10480a1190 0x7f10480a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:34.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.995+0000 7f105ffff640 1 --2- 172.21.15.114:0/714942497 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f10480a1190 0x7f10480a3650 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1050011550 tx=0x7f1050005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:34.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:33.997+0000 7f105d7fa640 1 -- 172.21.15.114:0/714942497 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f1054096660 con 0x7f10600ff880 2024-09-13T18:21:34.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.168+0000 7f1066745640 1 -- 172.21.15.114:0/714942497 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f10600ff2b0 con 0x7f10600ff880 2024-09-13T18:21:34.171 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:33 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/1823176802' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-13T18:21:34.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.169+0000 7f105d7fa640 1 -- 172.21.15.114:0/714942497 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f105401c020 con 0x7f10600ff880 2024-09-13T18:21:34.172 INFO:teuthology.orchestra.run.smithi114.stdout:55834574927 2024-09-13T18:21:34.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.171+0000 7f1066745640 1 -- 172.21.15.114:0/714942497 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f10480a1190 msgr2=0x7f10480a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:34.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.171+0000 7f1066745640 1 --2- 172.21.15.114:0/714942497 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f10480a1190 0x7f10480a3650 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1050011550 tx=0x7f1050005f90 comp rx=0 tx=0).stop 2024-09-13T18:21:34.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.171+0000 7f1066745640 1 -- 172.21.15.114:0/714942497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10600ff880 msgr2=0x7f1060197540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:34.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.171+0000 7f1066745640 1 --2- 172.21.15.114:0/714942497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10600ff880 0x7f1060197540 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f105400e9f0 tx=0x7f105400eeb0 comp rx=0 tx=0).stop 2024-09-13T18:21:34.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.171+0000 7f1066745640 1 -- 172.21.15.114:0/714942497 shutdown_connections 2024-09-13T18:21:34.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.171+0000 7f1066745640 1 --2- 172.21.15.114:0/714942497 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f10480a1190 0x7f10480a3650 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:34.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.171+0000 7f1066745640 1 --2- 172.21.15.114:0/714942497 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1060104000 0x7f1060197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:34.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.171+0000 7f1066745640 1 --2- 172.21.15.114:0/714942497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10600ff880 0x7f1060197540 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:34.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.171+0000 7f1066745640 1 --2- 172.21.15.114:0/714942497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10600feed0 0x7f1060197000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:34.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.171+0000 7f1066745640 1 -- 172.21.15.114:0/714942497 >> 172.21.15.114:0/714942497 conn(0x7f10600fa9b0 msgr2=0x7f10600fbde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:34.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.171+0000 7f1066745640 1 -- 172.21.15.114:0/714942497 shutdown_connections 2024-09-13T18:21:34.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.171+0000 7f1066745640 1 -- 172.21.15.114:0/714942497 wait complete. 2024-09-13T18:21:34.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:33 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/1823176802' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-13T18:21:34.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:33 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/1823176802' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-13T18:21:35.148 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:34 smithi114 ceph-mon[30897]: pgmap v327: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s; 14 B/s, 0 objects/s recovering 2024-09-13T18:21:35.148 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:34 smithi114 ceph-mon[30897]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-13T18:21:35.148 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:34 smithi114 ceph-mon[30897]: Cluster is now healthy 2024-09-13T18:21:35.148 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:34 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2802267973' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-13T18:21:35.148 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:34 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/714942497' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-13T18:21:35.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:34 smithi134 ceph-mon[34283]: pgmap v327: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s; 14 B/s, 0 objects/s recovering 2024-09-13T18:21:35.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:34 smithi134 ceph-mon[34283]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-13T18:21:35.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:34 smithi134 ceph-mon[34283]: Cluster is now healthy 2024-09-13T18:21:35.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:34 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/2802267973' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-13T18:21:35.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:34 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/714942497' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-13T18:21:35.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:34 smithi204 ceph-mon[34777]: pgmap v327: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s; 14 B/s, 0 objects/s recovering 2024-09-13T18:21:35.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:34 smithi204 ceph-mon[34777]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-13T18:21:35.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:34 smithi204 ceph-mon[34777]: Cluster is now healthy 2024-09-13T18:21:35.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:34 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/2802267973' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-13T18:21:35.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:34 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/714942497' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-13T18:21:35.535 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411396 got 77309411401 for osd.2 2024-09-13T18:21:35.536 DEBUG:teuthology.parallel:result is None 2024-09-13T18:21:35.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.954+0000 7f42b9d5e640 1 -- 172.21.15.114:0/2357446136 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42b40fe270 msgr2=0x7f42b40fe6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:35.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.954+0000 7f42b9d5e640 1 --2- 172.21.15.114:0/2357446136 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42b40fe270 0x7f42b40fe6d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f42a8009900 tx=0x7f42a802a410 comp rx=0 tx=0).stop 2024-09-13T18:21:35.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.954+0000 7f42b9d5e640 1 -- 172.21.15.114:0/2357446136 shutdown_connections 2024-09-13T18:21:35.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.954+0000 7f42b9d5e640 1 --2- 172.21.15.114:0/2357446136 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42b40fec10 0x7f42b410cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:35.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.954+0000 7f42b9d5e640 1 --2- 172.21.15.114:0/2357446136 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42b40fe270 0x7f42b40fe6d0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:35.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.954+0000 7f42b9d5e640 1 --2- 172.21.15.114:0/2357446136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42b4104dc0 0x7f42b41051a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:35.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.954+0000 7f42b9d5e640 1 -- 172.21.15.114:0/2357446136 >> 172.21.15.114:0/2357446136 conn(0x7f42b40753c0 msgr2=0x7f42b40fc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:35.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.954+0000 7f42b9d5e640 1 -- 172.21.15.114:0/2357446136 shutdown_connections 2024-09-13T18:21:35.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.954+0000 7f42b9d5e640 1 -- 172.21.15.114:0/2357446136 wait complete. 2024-09-13T18:21:35.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.955+0000 7f42b9d5e640 1 Processor -- start 2024-09-13T18:21:35.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.955+0000 7f42b9d5e640 1 -- start start 2024-09-13T18:21:35.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.955+0000 7f42b9d5e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42b40fe270 0x7f42b41961d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:35.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.955+0000 7f42b9d5e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42b40fec10 0x7f42b4196710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:35.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.955+0000 7f42b9d5e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42b4104dc0 0x7f42b419ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:35.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.955+0000 7f42b9d5e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b419b110 con 0x7f42b4104dc0 2024-09-13T18:21:35.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.955+0000 7f42b9d5e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b419b280 con 0x7f42b40fe270 2024-09-13T18:21:35.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.955+0000 7f42b9d5e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b419b3f0 con 0x7f42b40fec10 2024-09-13T18:21:35.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.955+0000 7f42b3fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42b4104dc0 0x7f42b419ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:35.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.955+0000 7f42b3fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42b4104dc0 0x7f42b419ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50332/0 (socket says 172.21.15.114:50332) 2024-09-13T18:21:35.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.955+0000 7f42b3fff640 1 -- 172.21.15.114:0/3470514379 learned_addr learned my addr 172.21.15.114:0/3470514379 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:35.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.955+0000 7f42b37fe640 1 --2- 172.21.15.114:0/3470514379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42b40fe270 0x7f42b41961d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:35.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.955+0000 7f42b3fff640 1 -- 172.21.15.114:0/3470514379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42b40fec10 msgr2=0x7f42b4196710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:35.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.955+0000 7f42b2ffd640 1 --2- 172.21.15.114:0/3470514379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42b40fec10 0x7f42b4196710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:35.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.956+0000 7f42b3fff640 1 --2- 172.21.15.114:0/3470514379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42b40fec10 0x7f42b4196710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:35.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.956+0000 7f42b3fff640 1 -- 172.21.15.114:0/3470514379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42b40fe270 msgr2=0x7f42b41961d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:35.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.956+0000 7f42b3fff640 1 --2- 172.21.15.114:0/3470514379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42b40fe270 0x7f42b41961d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:35.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.956+0000 7f42b3fff640 1 -- 172.21.15.114:0/3470514379 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a8009560 con 0x7f42b4104dc0 2024-09-13T18:21:35.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.956+0000 7f42b37fe640 1 --2- 172.21.15.114:0/3470514379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42b40fe270 0x7f42b41961d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:21:35.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.956+0000 7f42b3fff640 1 --2- 172.21.15.114:0/3470514379 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42b4104dc0 0x7f42b419ab10 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f42ac00ccb0 tx=0x7f42ac00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:35.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.956+0000 7f42b0ff9640 1 -- 172.21.15.114:0/3470514379 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ac003970 con 0x7f42b4104dc0 2024-09-13T18:21:35.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.956+0000 7f42b0ff9640 1 -- 172.21.15.114:0/3470514379 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f42ac004030 con 0x7f42b4104dc0 2024-09-13T18:21:35.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.956+0000 7f42b0ff9640 1 -- 172.21.15.114:0/3470514379 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ac00f6b0 con 0x7f42b4104dc0 2024-09-13T18:21:35.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.956+0000 7f42b9d5e640 1 -- 172.21.15.114:0/3470514379 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42b419b6d0 con 0x7f42b4104dc0 2024-09-13T18:21:35.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.956+0000 7f42b9d5e640 1 -- 172.21.15.114:0/3470514379 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42b41a2f90 con 0x7f42b4104dc0 2024-09-13T18:21:35.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.957+0000 7f42b9d5e640 1 -- 172.21.15.114:0/3470514379 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42b4101590 con 0x7f42b4104dc0 2024-09-13T18:21:35.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.958+0000 7f42b0ff9640 1 -- 172.21.15.114:0/3470514379 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f42ac0041a0 con 0x7f42b4104dc0 2024-09-13T18:21:35.904 INFO:teuthology.orchestra.run.smithi114.stdout:253403070510 2024-09-13T18:21:35.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.959+0000 7f42b0ff9640 1 --2- 172.21.15.114:0/3470514379 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f429c0a1110 0x7f429c0a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:35.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.959+0000 7f42b37fe640 1 --2- 172.21.15.114:0/3470514379 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f429c0a1110 0x7f429c0a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:35.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.959+0000 7f42b0ff9640 1 -- 172.21.15.114:0/3470514379 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f42ac0cd2a0 con 0x7f42b4104dc0 2024-09-13T18:21:35.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.959+0000 7f42b37fe640 1 --2- 172.21.15.114:0/3470514379 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f429c0a1110 0x7f429c0a35d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f42a4005d00 tx=0x7f42a4009550 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:35.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:34.962+0000 7f42b0ff9640 1 -- 172.21.15.114:0/3470514379 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f42ac096560 con 0x7f42b4104dc0 2024-09-13T18:21:35.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:35.145+0000 7f42b9d5e640 1 -- 172.21.15.114:0/3470514379 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f42b40fe6d0 con 0x7f42b4104dc0 2024-09-13T18:21:35.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:35.146+0000 7f42b0ff9640 1 -- 172.21.15.114:0/3470514379 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f42ac01d090 con 0x7f42b4104dc0 2024-09-13T18:21:35.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:35.148+0000 7f42b9d5e640 1 -- 172.21.15.114:0/3470514379 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f429c0a1110 msgr2=0x7f429c0a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:35.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:35.148+0000 7f42b9d5e640 1 --2- 172.21.15.114:0/3470514379 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f429c0a1110 0x7f429c0a35d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f42a4005d00 tx=0x7f42a4009550 comp rx=0 tx=0).stop 2024-09-13T18:21:35.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:35.148+0000 7f42b9d5e640 1 -- 172.21.15.114:0/3470514379 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42b4104dc0 msgr2=0x7f42b419ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:35.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:35.148+0000 7f42b9d5e640 1 --2- 172.21.15.114:0/3470514379 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42b4104dc0 0x7f42b419ab10 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f42ac00ccb0 tx=0x7f42ac00a620 comp rx=0 tx=0).stop 2024-09-13T18:21:35.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:35.148+0000 7f42b9d5e640 1 -- 172.21.15.114:0/3470514379 shutdown_connections 2024-09-13T18:21:35.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:35.148+0000 7f42b9d5e640 1 --2- 172.21.15.114:0/3470514379 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f429c0a1110 0x7f429c0a35d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:35.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:35.148+0000 7f42b9d5e640 1 --2- 172.21.15.114:0/3470514379 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42b4104dc0 0x7f42b419ab10 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:35.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:35.148+0000 7f42b9d5e640 1 --2- 172.21.15.114:0/3470514379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42b40fec10 0x7f42b4196710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:35.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:35.148+0000 7f42b9d5e640 1 --2- 172.21.15.114:0/3470514379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42b40fe270 0x7f42b41961d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:35.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:35.148+0000 7f42b9d5e640 1 -- 172.21.15.114:0/3470514379 >> 172.21.15.114:0/3470514379 conn(0x7f42b40753c0 msgr2=0x7f42b4102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:35.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:35.148+0000 7f42b9d5e640 1 -- 172.21.15.114:0/3470514379 shutdown_connections 2024-09-13T18:21:35.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:35.148+0000 7f42b9d5e640 1 -- 172.21.15.114:0/3470514379 wait complete. 2024-09-13T18:21:36.211 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:35 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/3470514379' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-13T18:21:36.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.291+0000 7fee1bc21640 1 -- 172.21.15.114:0/2333478491 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fee14104000 msgr2=0x7fee1410bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:36.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.291+0000 7fee1bc21640 1 --2- 172.21.15.114:0/2333478491 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fee14104000 0x7fee1410bb80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fee0c009cd0 tx=0x7fee0c02a8e0 comp rx=0 tx=0).stop 2024-09-13T18:21:36.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.291+0000 7fee1bc21640 1 -- 172.21.15.114:0/2333478491 shutdown_connections 2024-09-13T18:21:36.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.291+0000 7fee1bc21640 1 --2- 172.21.15.114:0/2333478491 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fee14104000 0x7fee1410bb80 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:36.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.291+0000 7fee1bc21640 1 --2- 172.21.15.114:0/2333478491 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee140ff880 0x7fee141038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:36.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.291+0000 7fee1bc21640 1 --2- 172.21.15.114:0/2333478491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee140feed0 0x7fee140ff2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:36.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.291+0000 7fee1bc21640 1 -- 172.21.15.114:0/2333478491 >> 172.21.15.114:0/2333478491 conn(0x7fee140fa9b0 msgr2=0x7fee140fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:36.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.291+0000 7fee1bc21640 1 -- 172.21.15.114:0/2333478491 shutdown_connections 2024-09-13T18:21:36.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.291+0000 7fee1bc21640 1 -- 172.21.15.114:0/2333478491 wait complete. 2024-09-13T18:21:36.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.292+0000 7fee1bc21640 1 Processor -- start 2024-09-13T18:21:36.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.292+0000 7fee1bc21640 1 -- start start 2024-09-13T18:21:36.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.292+0000 7fee1bc21640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fee140feed0 0x7fee14197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:36.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.292+0000 7fee1bc21640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee140ff880 0x7fee14197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:36.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.292+0000 7fee1bc21640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee14104000 0x7fee141a20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:36.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.292+0000 7fee1bc21640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee14197db0 con 0x7fee140feed0 2024-09-13T18:21:36.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.292+0000 7fee1bc21640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee141a2600 con 0x7fee14104000 2024-09-13T18:21:36.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.292+0000 7fee1bc21640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee141a2740 con 0x7fee140ff880 2024-09-13T18:21:36.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.292+0000 7fee199bd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fee140feed0 0x7fee14197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:36.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.292+0000 7fee199bd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fee140feed0 0x7fee14197040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50350/0 (socket says 172.21.15.114:50350) 2024-09-13T18:21:36.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.292+0000 7fee199bd640 1 -- 172.21.15.114:0/4257155202 learned_addr learned my addr 172.21.15.114:0/4257155202 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:36.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.292+0000 7fee1a1be640 1 --2- 172.21.15.114:0/4257155202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee14104000 0x7fee141a20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:36.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.292+0000 7fee191bc640 1 --2- 172.21.15.114:0/4257155202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee140ff880 0x7fee14197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:36.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.292+0000 7fee199bd640 1 -- 172.21.15.114:0/4257155202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee140ff880 msgr2=0x7fee14197580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:36.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.292+0000 7fee199bd640 1 --2- 172.21.15.114:0/4257155202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee140ff880 0x7fee14197580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:36.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.292+0000 7fee199bd640 1 -- 172.21.15.114:0/4257155202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee14104000 msgr2=0x7fee141a20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:36.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.292+0000 7fee199bd640 1 --2- 172.21.15.114:0/4257155202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee14104000 0x7fee141a20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:36.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.292+0000 7fee199bd640 1 -- 172.21.15.114:0/4257155202 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee0c009900 con 0x7fee140feed0 2024-09-13T18:21:36.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.292+0000 7fee1a1be640 1 --2- 172.21.15.114:0/4257155202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee14104000 0x7fee141a20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:21:36.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.292+0000 7fee191bc640 1 --2- 172.21.15.114:0/4257155202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee140ff880 0x7fee14197580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:21:36.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.292+0000 7fee199bd640 1 --2- 172.21.15.114:0/4257155202 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fee140feed0 0x7fee14197040 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fee1000d770 tx=0x7fee1000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:36.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.292+0000 7fee0affd640 1 -- 172.21.15.114:0/4257155202 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee10009c30 con 0x7fee140feed0 2024-09-13T18:21:36.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.293+0000 7fee1bc21640 1 -- 172.21.15.114:0/4257155202 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee141a29a0 con 0x7fee140feed0 2024-09-13T18:21:36.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.293+0000 7fee0affd640 1 -- 172.21.15.114:0/4257155202 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fee1000de80 con 0x7fee140feed0 2024-09-13T18:21:36.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.293+0000 7fee0affd640 1 -- 172.21.15.114:0/4257155202 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee10004d40 con 0x7fee140feed0 2024-09-13T18:21:36.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.293+0000 7fee1bc21640 1 -- 172.21.15.114:0/4257155202 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee141a2e70 con 0x7fee140feed0 2024-09-13T18:21:36.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.294+0000 7fee1bc21640 1 -- 172.21.15.114:0/4257155202 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee141003b0 con 0x7fee140feed0 2024-09-13T18:21:36.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.295+0000 7fee0affd640 1 -- 172.21.15.114:0/4257155202 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fee10004ea0 con 0x7fee140feed0 2024-09-13T18:21:36.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.295+0000 7fee0affd640 1 --2- 172.21.15.114:0/4257155202 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fee000a1110 0x7fee000a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:36.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.295+0000 7fee191bc640 1 --2- 172.21.15.114:0/4257155202 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fee000a1110 0x7fee000a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:36.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.295+0000 7fee0affd640 1 -- 172.21.15.114:0/4257155202 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fee100cc8f0 con 0x7fee140feed0 2024-09-13T18:21:36.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.296+0000 7fee191bc640 1 --2- 172.21.15.114:0/4257155202 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fee000a1110 0x7fee000a35d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fee04005e00 tx=0x7fee04005c70 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:36.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.299+0000 7fee0affd640 1 -- 172.21.15.114:0/4257155202 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fee10095bb0 con 0x7fee140feed0 2024-09-13T18:21:36.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:35 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/3470514379' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-13T18:21:36.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:35 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/3470514379' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-13T18:21:36.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.470+0000 7fee1bc21640 1 -- 172.21.15.114:0/4257155202 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fee140ff2b0 con 0x7fee140feed0 2024-09-13T18:21:36.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.470+0000 7fee0affd640 1 -- 172.21.15.114:0/4257155202 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fee10020090 con 0x7fee140feed0 2024-09-13T18:21:36.473 INFO:teuthology.orchestra.run.smithi114.stdout:141733920832 2024-09-13T18:21:36.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.472+0000 7fee1bc21640 1 -- 172.21.15.114:0/4257155202 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fee000a1110 msgr2=0x7fee000a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:36.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.472+0000 7fee1bc21640 1 --2- 172.21.15.114:0/4257155202 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fee000a1110 0x7fee000a35d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fee04005e00 tx=0x7fee04005c70 comp rx=0 tx=0).stop 2024-09-13T18:21:36.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.472+0000 7fee1bc21640 1 -- 172.21.15.114:0/4257155202 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fee140feed0 msgr2=0x7fee14197040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:36.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.472+0000 7fee1bc21640 1 --2- 172.21.15.114:0/4257155202 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fee140feed0 0x7fee14197040 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fee1000d770 tx=0x7fee1000dc30 comp rx=0 tx=0).stop 2024-09-13T18:21:36.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.472+0000 7fee1bc21640 1 -- 172.21.15.114:0/4257155202 shutdown_connections 2024-09-13T18:21:36.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.472+0000 7fee1bc21640 1 --2- 172.21.15.114:0/4257155202 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fee000a1110 0x7fee000a35d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:36.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.472+0000 7fee1bc21640 1 --2- 172.21.15.114:0/4257155202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee14104000 0x7fee141a20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:36.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.472+0000 7fee1bc21640 1 --2- 172.21.15.114:0/4257155202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee140ff880 0x7fee14197580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:36.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.472+0000 7fee1bc21640 1 --2- 172.21.15.114:0/4257155202 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fee140feed0 0x7fee14197040 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:36.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.472+0000 7fee1bc21640 1 -- 172.21.15.114:0/4257155202 >> 172.21.15.114:0/4257155202 conn(0x7fee140fa9b0 msgr2=0x7fee140fbd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:36.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.473+0000 7fee1bc21640 1 -- 172.21.15.114:0/4257155202 shutdown_connections 2024-09-13T18:21:36.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.473+0000 7fee1bc21640 1 -- 172.21.15.114:0/4257155202 wait complete. 2024-09-13T18:21:36.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.535+0000 7f3eb9a1c640 1 -- 172.21.15.114:0/2575534659 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3eb40fffe0 msgr2=0x7f3eb41003c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:36.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.535+0000 7f3eb9a1c640 1 --2- 172.21.15.114:0/2575534659 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3eb40fffe0 0x7f3eb41003c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4009930 tx=0x7f3ea402a420 comp rx=0 tx=0).stop 2024-09-13T18:21:36.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.535+0000 7f3eb9a1c640 1 -- 172.21.15.114:0/2575534659 shutdown_connections 2024-09-13T18:21:36.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.535+0000 7f3eb9a1c640 1 --2- 172.21.15.114:0/2575534659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3eb4105110 0x7f3eb410cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:36.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.535+0000 7f3eb9a1c640 1 --2- 172.21.15.114:0/2575534659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3eb4100990 0x7f3eb41049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:36.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.535+0000 7f3eb9a1c640 1 --2- 172.21.15.114:0/2575534659 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3eb40fffe0 0x7f3eb41003c0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:36.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.535+0000 7f3eb9a1c640 1 -- 172.21.15.114:0/2575534659 >> 172.21.15.114:0/2575534659 conn(0x7f3eb40fba80 msgr2=0x7f3eb40fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:36.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.535+0000 7f3eb9a1c640 1 -- 172.21.15.114:0/2575534659 shutdown_connections 2024-09-13T18:21:36.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.535+0000 7f3eb9a1c640 1 -- 172.21.15.114:0/2575534659 wait complete. 2024-09-13T18:21:36.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.536+0000 7f3eb9a1c640 1 Processor -- start 2024-09-13T18:21:36.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.536+0000 7f3eb9a1c640 1 -- start start 2024-09-13T18:21:36.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.536+0000 7f3eb9a1c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3eb40fffe0 0x7f3eb4196260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:36.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.536+0000 7f3eb9a1c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3eb4100990 0x7f3eb41967a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:36.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.536+0000 7f3eb9a1c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3eb4105110 0x7f3eb419aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:36.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.536+0000 7f3eb9a1c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb419b1a0 con 0x7f3eb4100990 2024-09-13T18:21:36.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.536+0000 7f3eb9a1c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb419b310 con 0x7f3eb4105110 2024-09-13T18:21:36.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.536+0000 7f3eb9a1c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb419b480 con 0x7f3eb40fffe0 2024-09-13T18:21:36.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.536+0000 7f3eaa7fc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3eb4100990 0x7f3eb41967a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:36.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.536+0000 7f3eaa7fc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3eb4100990 0x7f3eb41967a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50366/0 (socket says 172.21.15.114:50366) 2024-09-13T18:21:36.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.536+0000 7f3eaa7fc640 1 -- 172.21.15.114:0/3326307819 learned_addr learned my addr 172.21.15.114:0/3326307819 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:36.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.537+0000 7f3eb37fe640 1 --2- 172.21.15.114:0/3326307819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3eb4105110 0x7f3eb419aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:36.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.537+0000 7f3eb2ffd640 1 --2- 172.21.15.114:0/3326307819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3eb40fffe0 0x7f3eb4196260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:36.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.537+0000 7f3eaa7fc640 1 -- 172.21.15.114:0/3326307819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3eb40fffe0 msgr2=0x7f3eb4196260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:36.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.537+0000 7f3eaa7fc640 1 --2- 172.21.15.114:0/3326307819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3eb40fffe0 0x7f3eb4196260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:36.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.537+0000 7f3eaa7fc640 1 -- 172.21.15.114:0/3326307819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3eb4105110 msgr2=0x7f3eb419aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:36.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.537+0000 7f3eaa7fc640 1 --2- 172.21.15.114:0/3326307819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3eb4105110 0x7f3eb419aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:36.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.537+0000 7f3eaa7fc640 1 -- 172.21.15.114:0/3326307819 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea4009560 con 0x7f3eb4100990 2024-09-13T18:21:36.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.537+0000 7f3eaa7fc640 1 --2- 172.21.15.114:0/3326307819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3eb4100990 0x7f3eb41967a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c00d950 tx=0x7f3e9c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:36.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.537+0000 7f3eb8a1a640 1 -- 172.21.15.114:0/3326307819 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c009c30 con 0x7f3eb4100990 2024-09-13T18:21:36.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.537+0000 7f3eb8a1a640 1 -- 172.21.15.114:0/3326307819 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3e9c009d90 con 0x7f3eb4100990 2024-09-13T18:21:36.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.537+0000 7f3eb9a1c640 1 -- 172.21.15.114:0/3326307819 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eb419b760 con 0x7f3eb4100990 2024-09-13T18:21:36.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.537+0000 7f3eb8a1a640 1 -- 172.21.15.114:0/3326307819 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c00f620 con 0x7f3eb4100990 2024-09-13T18:21:36.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.537+0000 7f3eb9a1c640 1 -- 172.21.15.114:0/3326307819 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eb41a3020 con 0x7f3eb4100990 2024-09-13T18:21:36.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.538+0000 7f3eb9a1c640 1 -- 172.21.15.114:0/3326307819 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eb41014c0 con 0x7f3eb4100990 2024-09-13T18:21:36.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.539+0000 7f3eb8a1a640 1 -- 172.21.15.114:0/3326307819 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f3e9c0109f0 con 0x7f3eb4100990 2024-09-13T18:21:36.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.540+0000 7f3eb8a1a640 1 --2- 172.21.15.114:0/3326307819 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f3ea00a1190 0x7f3ea00a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:36.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.540+0000 7f3eb2ffd640 1 --2- 172.21.15.114:0/3326307819 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f3ea00a1190 0x7f3ea00a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:36.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.540+0000 7f3eb8a1a640 1 -- 172.21.15.114:0/3326307819 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f3e9c0cc930 con 0x7f3eb4100990 2024-09-13T18:21:36.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.541+0000 7f3eb2ffd640 1 --2- 172.21.15.114:0/3326307819 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f3ea00a1190 0x7f3ea00a3650 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4006010 tx=0x7f3ea4005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:36.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.543+0000 7f3eb8a1a640 1 -- 172.21.15.114:0/3326307819 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f3e9c095c70 con 0x7f3eb4100990 2024-09-13T18:21:36.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.709+0000 7f3eb9a1c640 1 -- 172.21.15.114:0/3326307819 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f3eb41003c0 con 0x7f3eb4100990 2024-09-13T18:21:36.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.709+0000 7f3eb8a1a640 1 -- 172.21.15.114:0/3326307819 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3e9c017020 con 0x7f3eb4100990 2024-09-13T18:21:36.712 INFO:teuthology.orchestra.run.smithi114.stdout:287762808874 2024-09-13T18:21:36.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.711+0000 7f3eb9a1c640 1 -- 172.21.15.114:0/3326307819 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f3ea00a1190 msgr2=0x7f3ea00a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:36.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.711+0000 7f3eb9a1c640 1 --2- 172.21.15.114:0/3326307819 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f3ea00a1190 0x7f3ea00a3650 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4006010 tx=0x7f3ea4005a20 comp rx=0 tx=0).stop 2024-09-13T18:21:36.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.711+0000 7f3eb9a1c640 1 -- 172.21.15.114:0/3326307819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3eb4100990 msgr2=0x7f3eb41967a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:36.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.711+0000 7f3eb9a1c640 1 --2- 172.21.15.114:0/3326307819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3eb4100990 0x7f3eb41967a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c00d950 tx=0x7f3e9c00de10 comp rx=0 tx=0).stop 2024-09-13T18:21:36.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.712+0000 7f3eb9a1c640 1 -- 172.21.15.114:0/3326307819 shutdown_connections 2024-09-13T18:21:36.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.712+0000 7f3eb9a1c640 1 --2- 172.21.15.114:0/3326307819 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f3ea00a1190 0x7f3ea00a3650 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:36.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.712+0000 7f3eb9a1c640 1 --2- 172.21.15.114:0/3326307819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3eb4105110 0x7f3eb419aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:36.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.712+0000 7f3eb9a1c640 1 --2- 172.21.15.114:0/3326307819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3eb4100990 0x7f3eb41967a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:36.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.712+0000 7f3eb9a1c640 1 --2- 172.21.15.114:0/3326307819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3eb40fffe0 0x7f3eb4196260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:36.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.712+0000 7f3eb9a1c640 1 -- 172.21.15.114:0/3326307819 >> 172.21.15.114:0/3326307819 conn(0x7f3eb40fba80 msgr2=0x7f3eb40fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:36.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.712+0000 7f3eb9a1c640 1 -- 172.21.15.114:0/3326307819 shutdown_connections 2024-09-13T18:21:36.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:36.712+0000 7f3eb9a1c640 1 -- 172.21.15.114:0/3326307819 wait complete. 2024-09-13T18:21:37.269 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:36 smithi114 ceph-mon[30897]: pgmap v328: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9 op/s; 13 B/s, 0 objects/s recovering 2024-09-13T18:21:37.269 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:36 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/4257155202' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-13T18:21:37.269 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:36 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/3326307819' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-13T18:21:37.294 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920826 got 141733920832 for osd.4 2024-09-13T18:21:37.294 DEBUG:teuthology.parallel:result is None 2024-09-13T18:21:37.345 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:36 smithi204 ceph-mon[34777]: pgmap v328: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9 op/s; 13 B/s, 0 objects/s recovering 2024-09-13T18:21:37.346 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:36 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/4257155202' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-13T18:21:37.346 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:36 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/3326307819' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-13T18:21:37.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:36 smithi134 ceph-mon[34283]: pgmap v328: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9 op/s; 13 B/s, 0 objects/s recovering 2024-09-13T18:21:37.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:36 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/4257155202' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-13T18:21:37.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:36 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/3326307819' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-13T18:21:38.106 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332141 got 219043332146 for osd.7 2024-09-13T18:21:38.106 DEBUG:teuthology.parallel:result is None 2024-09-13T18:21:38.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:38.438+0000 7f7a27261640 1 -- 172.21.15.114:0/475683547 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a200ff8b0 msgr2=0x7f7a20103900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:38.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:38.438+0000 7f7a27261640 1 --2- 172.21.15.114:0/475683547 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a200ff8b0 0x7f7a20103900 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c009930 tx=0x7f7a0c02a420 comp rx=0 tx=0).stop 2024-09-13T18:21:38.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:38.438+0000 7f7a27261640 1 -- 172.21.15.114:0/475683547 shutdown_connections 2024-09-13T18:21:38.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:38.438+0000 7f7a27261640 1 --2- 172.21.15.114:0/475683547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a20104030 0x7f7a2010bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:38.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:38.438+0000 7f7a27261640 1 --2- 172.21.15.114:0/475683547 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a200ff8b0 0x7f7a20103900 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:38.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:38.438+0000 7f7a27261640 1 --2- 172.21.15.114:0/475683547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a200fef00 0x7f7a200ff2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:38.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:38.438+0000 7f7a27261640 1 -- 172.21.15.114:0/475683547 >> 172.21.15.114:0/475683547 conn(0x7f7a200fa9a0 msgr2=0x7f7a200fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:38.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:38.438+0000 7f7a27261640 1 -- 172.21.15.114:0/475683547 shutdown_connections 2024-09-13T18:21:38.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:38.438+0000 7f7a27261640 1 -- 172.21.15.114:0/475683547 wait complete. 2024-09-13T18:21:38.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:38.439+0000 7f7a27261640 1 Processor -- start 2024-09-13T18:21:38.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:38.439+0000 7f7a27261640 1 -- start start 2024-09-13T18:21:38.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:38.439+0000 7f7a27261640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a200fef00 0x7f7a20196110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:38.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:38.439+0000 7f7a27261640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a200ff8b0 0x7f7a20196650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:38.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:38.439+0000 7f7a27261640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a20104030 0x7f7a2019aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:38.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:38.439+0000 7f7a27261640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2019b050 con 0x7f7a20104030 2024-09-13T18:21:38.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:38.439+0000 7f7a27261640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2019b1c0 con 0x7f7a200fef00 2024-09-13T18:21:38.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:38.439+0000 7f7a27261640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2019b330 con 0x7f7a200ff8b0 2024-09-13T18:21:38.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:38.439+0000 7f7a257fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a20104030 0x7f7a2019aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:38.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:38.439+0000 7f7a257fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a20104030 0x7f7a2019aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50390/0 (socket says 172.21.15.114:50390) 2024-09-13T18:21:38.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:38.439+0000 7f7a257fe640 1 -- 172.21.15.114:0/2609915747 learned_addr learned my addr 172.21.15.114:0/2609915747 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:38.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:38.439+0000 7f7a24ffd640 1 --2- 172.21.15.114:0/2609915747 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a200fef00 0x7f7a20196110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:38.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:38.439+0000 7f7a257fe640 1 -- 172.21.15.114:0/2609915747 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a200ff8b0 msgr2=0x7f7a20196650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:38.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:38.439+0000 7f7a257fe640 1 --2- 172.21.15.114:0/2609915747 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a200ff8b0 0x7f7a20196650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:38.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:38.439+0000 7f7a257fe640 1 -- 172.21.15.114:0/2609915747 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a200fef00 msgr2=0x7f7a20196110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:38.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:38.440+0000 7f7a257fe640 1 --2- 172.21.15.114:0/2609915747 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a200fef00 0x7f7a20196110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:38.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:38.440+0000 7f7a257fe640 1 -- 172.21.15.114:0/2609915747 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a0c009560 con 0x7f7a20104030 2024-09-13T18:21:38.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:38.440+0000 7f7a257fe640 1 --2- 172.21.15.114:0/2609915747 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a20104030 0x7f7a2019aa50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7a1800cce0 tx=0x7f7a1800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:38.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:38.440+0000 7f7a1dffb640 1 -- 172.21.15.114:0/2609915747 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a1800f450 con 0x7f7a20104030 2024-09-13T18:21:38.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:38.440+0000 7f7a1dffb640 1 -- 172.21.15.114:0/2609915747 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7a1800fa40 con 0x7f7a20104030 2024-09-13T18:21:38.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:38.440+0000 7f7a1dffb640 1 -- 172.21.15.114:0/2609915747 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a180189a0 con 0x7f7a20104030 2024-09-13T18:21:38.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:38.440+0000 7f7a27261640 1 -- 172.21.15.114:0/2609915747 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a2019b610 con 0x7f7a20104030 2024-09-13T18:21:38.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:38.440+0000 7f7a27261640 1 -- 172.21.15.114:0/2609915747 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a201a2ed0 con 0x7f7a20104030 2024-09-13T18:21:38.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:38.441+0000 7f7a27261640 1 -- 172.21.15.114:0/2609915747 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a201003e0 con 0x7f7a20104030 2024-09-13T18:21:38.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:38.442+0000 7f7a1dffb640 1 -- 172.21.15.114:0/2609915747 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f7a1800fbb0 con 0x7f7a20104030 2024-09-13T18:21:38.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:38.443+0000 7f7a1dffb640 1 --2- 172.21.15.114:0/2609915747 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f7a100a1190 0x7f7a100a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:38.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:38.443+0000 7f7a24ffd640 1 --2- 172.21.15.114:0/2609915747 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f7a100a1190 0x7f7a100a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:38.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:38.443+0000 7f7a1dffb640 1 -- 172.21.15.114:0/2609915747 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f7a180cd290 con 0x7f7a20104030 2024-09-13T18:21:38.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:38.443+0000 7f7a24ffd640 1 --2- 172.21.15.114:0/2609915747 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f7a100a1190 0x7f7a100a3650 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7a1400ba40 tx=0x7f7a14005cc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:38.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:38.447+0000 7f7a1dffb640 1 -- 172.21.15.114:0/2609915747 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f7a180964f0 con 0x7f7a20104030 2024-09-13T18:21:38.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:38.612+0000 7f7a27261640 1 -- 172.21.15.114:0/2609915747 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f7a200ff2e0 con 0x7f7a20104030 2024-09-13T18:21:38.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:38.612+0000 7f7a1dffb640 1 -- 172.21.15.114:0/2609915747 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7a18099b10 con 0x7f7a20104030 2024-09-13T18:21:38.615 INFO:teuthology.orchestra.run.smithi114.stdout:34359738452 2024-09-13T18:21:38.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:38.614+0000 7f7a27261640 1 -- 172.21.15.114:0/2609915747 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f7a100a1190 msgr2=0x7f7a100a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:38.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:38.614+0000 7f7a27261640 1 --2- 172.21.15.114:0/2609915747 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f7a100a1190 0x7f7a100a3650 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7a1400ba40 tx=0x7f7a14005cc0 comp rx=0 tx=0).stop 2024-09-13T18:21:38.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:38.614+0000 7f7a27261640 1 -- 172.21.15.114:0/2609915747 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a20104030 msgr2=0x7f7a2019aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:38.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:38.614+0000 7f7a27261640 1 --2- 172.21.15.114:0/2609915747 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a20104030 0x7f7a2019aa50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7a1800cce0 tx=0x7f7a1800a620 comp rx=0 tx=0).stop 2024-09-13T18:21:38.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:38.614+0000 7f7a27261640 1 -- 172.21.15.114:0/2609915747 shutdown_connections 2024-09-13T18:21:38.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:38.614+0000 7f7a27261640 1 --2- 172.21.15.114:0/2609915747 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f7a100a1190 0x7f7a100a3650 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:38.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:38.614+0000 7f7a27261640 1 --2- 172.21.15.114:0/2609915747 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a20104030 0x7f7a2019aa50 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:38.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:38.614+0000 7f7a27261640 1 --2- 172.21.15.114:0/2609915747 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a200ff8b0 0x7f7a20196650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:38.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:38.614+0000 7f7a27261640 1 --2- 172.21.15.114:0/2609915747 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a200fef00 0x7f7a20196110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:38.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:38.614+0000 7f7a27261640 1 -- 172.21.15.114:0/2609915747 >> 172.21.15.114:0/2609915747 conn(0x7f7a200fa9a0 msgr2=0x7f7a20068af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:38.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:38.614+0000 7f7a27261640 1 -- 172.21.15.114:0/2609915747 shutdown_connections 2024-09-13T18:21:38.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:38.614+0000 7f7a27261640 1 -- 172.21.15.114:0/2609915747 wait complete. 2024-09-13T18:21:38.918 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070504 got 253403070510 for osd.8 2024-09-13T18:21:38.919 DEBUG:teuthology.parallel:result is None 2024-09-13T18:21:39.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:38 smithi114 ceph-mon[30897]: pgmap v329: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 11 B/s, 0 objects/s recovering 2024-09-13T18:21:39.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:38 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2609915747' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-13T18:21:39.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:32.295+0000 7fa35a4da640 1 -- 172.21.15.114:0/3732486498 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa354101020 msgr2=0x7fa354101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:39.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:32.295+0000 7fa35a4da640 1 --2- 172.21.15.114:0/3732486498 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa354101020 0x7fa354101400 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa344009900 tx=0x7fa34402a480 comp rx=0 tx=0).stop 2024-09-13T18:21:39.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:32.295+0000 7fa35a4da640 1 -- 172.21.15.114:0/3732486498 shutdown_connections 2024-09-13T18:21:39.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:32.295+0000 7fa35a4da640 1 --2- 172.21.15.114:0/3732486498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa35410a3c0 0x7fa35410c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:39.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:32.295+0000 7fa35a4da640 1 --2- 172.21.15.114:0/3732486498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa354101940 0x7fa354109e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:39.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:32.295+0000 7fa35a4da640 1 --2- 172.21.15.114:0/3732486498 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa354101020 0x7fa354101400 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:39.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:32.295+0000 7fa35a4da640 1 -- 172.21.15.114:0/3732486498 >> 172.21.15.114:0/3732486498 conn(0x7fa3540fa9a0 msgr2=0x7fa3540fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:39.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:32.295+0000 7fa35a4da640 1 -- 172.21.15.114:0/3732486498 shutdown_connections 2024-09-13T18:21:39.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:32.295+0000 7fa35a4da640 1 -- 172.21.15.114:0/3732486498 wait complete. 2024-09-13T18:21:39.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:32.296+0000 7fa35a4da640 1 Processor -- start 2024-09-13T18:21:39.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:32.296+0000 7fa35a4da640 1 -- start start 2024-09-13T18:21:39.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:32.296+0000 7fa35a4da640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa354101020 0x7fa3541961f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:39.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:32.296+0000 7fa35a4da640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa354101940 0x7fa354196730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:39.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:32.296+0000 7fa35a4da640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa35410a3c0 0x7fa35419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:39.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:32.296+0000 7fa35a4da640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35419b130 con 0x7fa35410a3c0 2024-09-13T18:21:39.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:32.296+0000 7fa35a4da640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35419b2a0 con 0x7fa354101940 2024-09-13T18:21:39.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:32.296+0000 7fa35a4da640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35419b410 con 0x7fa354101020 2024-09-13T18:21:39.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:32.297+0000 7fa353fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa354101020 0x7fa3541961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:39.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:32.297+0000 7fa353fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa354101020 0x7fa3541961f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:48064/0 (socket says 172.21.15.114:48064) 2024-09-13T18:21:39.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:32.297+0000 7fa353fff640 1 -- 172.21.15.114:0/1284716419 learned_addr learned my addr 172.21.15.114:0/1284716419 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:39.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:32.297+0000 7fa3537fe640 1 --2- 172.21.15.114:0/1284716419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa354101940 0x7fa354196730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:39.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:32.297+0000 7fa358a77640 1 --2- 172.21.15.114:0/1284716419 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa35410a3c0 0x7fa35419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:39.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:32.297+0000 7fa353fff640 1 -- 172.21.15.114:0/1284716419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa354101940 msgr2=0x7fa354196730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:39.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:32.297+0000 7fa353fff640 1 --2- 172.21.15.114:0/1284716419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa354101940 0x7fa354196730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:39.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:32.297+0000 7fa353fff640 1 -- 172.21.15.114:0/1284716419 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa35410a3c0 msgr2=0x7fa35419ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:39.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:32.297+0000 7fa353fff640 1 --2- 172.21.15.114:0/1284716419 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa35410a3c0 0x7fa35419ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:39.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:32.297+0000 7fa353fff640 1 -- 172.21.15.114:0/1284716419 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa344009560 con 0x7fa354101020 2024-09-13T18:21:39.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:32.297+0000 7fa353fff640 1 --2- 172.21.15.114:0/1284716419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa354101020 0x7fa3541961f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa3440057c0 tx=0x7fa344032c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:39.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:32.297+0000 7fa3517fa640 1 -- 172.21.15.114:0/1284716419 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa344032e10 con 0x7fa354101020 2024-09-13T18:21:39.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:32.297+0000 7fa35a4da640 1 -- 172.21.15.114:0/1284716419 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa35419b690 con 0x7fa354101020 2024-09-13T18:21:39.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:32.297+0000 7fa3517fa640 1 -- 172.21.15.114:0/1284716419 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa3440046c0 con 0x7fa354101020 2024-09-13T18:21:39.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:32.297+0000 7fa3517fa640 1 -- 172.21.15.114:0/1284716419 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa344003850 con 0x7fa354101020 2024-09-13T18:21:39.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:32.298+0000 7fa35a4da640 1 -- 172.21.15.114:0/1284716419 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa35419b9b0 con 0x7fa354101020 2024-09-13T18:21:39.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:32.298+0000 7fa35a4da640 1 -- 172.21.15.114:0/1284716419 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa354068f40 con 0x7fa354101020 2024-09-13T18:21:39.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:32.300+0000 7fa3517fa640 1 -- 172.21.15.114:0/1284716419 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fa344005210 con 0x7fa354101020 2024-09-13T18:21:39.271 INFO:teuthology.orchestra.run.smithi114.stdout:343597383711 2024-09-13T18:21:39.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:32.300+0000 7fa3517fa640 1 --2- 172.21.15.114:0/1284716419 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fa33c0a1110 0x7fa33c0a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:39.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:32.300+0000 7fa3537fe640 1 --2- 172.21.15.114:0/1284716419 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fa33c0a1110 0x7fa33c0a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:39.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:32.300+0000 7fa3517fa640 1 -- 172.21.15.114:0/1284716419 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fa3440edf40 con 0x7fa354101020 2024-09-13T18:21:39.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:32.301+0000 7fa3537fe640 1 --2- 172.21.15.114:0/1284716419 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fa33c0a1110 0x7fa33c0a35d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa3541977a0 tx=0x7fa34800b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:39.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:32.304+0000 7fa3517fa640 1 -- 172.21.15.114:0/1284716419 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fa3440b72b0 con 0x7fa354101020 2024-09-13T18:21:39.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:32.469+0000 7fa35a4da640 1 -- 172.21.15.114:0/1284716419 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7fa354101400 con 0x7fa354101020 2024-09-13T18:21:39.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:32.470+0000 7fa3517fa640 1 -- 172.21.15.114:0/1284716419 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fa3440ba8d0 con 0x7fa354101020 2024-09-13T18:21:39.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:32.472+0000 7fa35a4da640 1 -- 172.21.15.114:0/1284716419 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fa33c0a1110 msgr2=0x7fa33c0a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:39.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:32.472+0000 7fa35a4da640 1 --2- 172.21.15.114:0/1284716419 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fa33c0a1110 0x7fa33c0a35d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa3541977a0 tx=0x7fa34800b390 comp rx=0 tx=0).stop 2024-09-13T18:21:39.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:32.472+0000 7fa35a4da640 1 -- 172.21.15.114:0/1284716419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa354101020 msgr2=0x7fa3541961f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:39.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:32.472+0000 7fa35a4da640 1 --2- 172.21.15.114:0/1284716419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa354101020 0x7fa3541961f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa3440057c0 tx=0x7fa344032c20 comp rx=0 tx=0).stop 2024-09-13T18:21:39.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:32.472+0000 7fa35a4da640 1 -- 172.21.15.114:0/1284716419 shutdown_connections 2024-09-13T18:21:39.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:32.472+0000 7fa35a4da640 1 --2- 172.21.15.114:0/1284716419 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fa33c0a1110 0x7fa33c0a35d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:39.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:32.472+0000 7fa35a4da640 1 --2- 172.21.15.114:0/1284716419 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa35410a3c0 0x7fa35419ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:39.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:32.472+0000 7fa35a4da640 1 --2- 172.21.15.114:0/1284716419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa354101940 0x7fa354196730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:39.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:32.472+0000 7fa35a4da640 1 --2- 172.21.15.114:0/1284716419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa354101020 0x7fa3541961f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:39.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:32.472+0000 7fa35a4da640 1 -- 172.21.15.114:0/1284716419 >> 172.21.15.114:0/1284716419 conn(0x7fa3540fa9a0 msgr2=0x7fa3540fc300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:39.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:32.472+0000 7fa35a4da640 1 -- 172.21.15.114:0/1284716419 shutdown_connections 2024-09-13T18:21:39.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:32.472+0000 7fa35a4da640 1 -- 172.21.15.114:0/1284716419 wait complete. 2024-09-13T18:21:39.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:38 smithi134 ceph-mon[34283]: pgmap v329: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 11 B/s, 0 objects/s recovering 2024-09-13T18:21:39.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:38 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/2609915747' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-13T18:21:39.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:38 smithi204 ceph-mon[34777]: pgmap v329: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 11 B/s, 0 objects/s recovering 2024-09-13T18:21:39.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:38 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/2609915747' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-13T18:21:39.912 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383705 got 343597383711 for osd.11 2024-09-13T18:21:39.912 DEBUG:teuthology.parallel:result is None 2024-09-13T18:21:40.355 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738446 got 34359738452 for osd.0 2024-09-13T18:21:40.355 DEBUG:teuthology.parallel:result is None 2024-09-13T18:21:41.117 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:40 smithi114 ceph-mon[30897]: pgmap v330: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 11 op/s; 9 B/s, 0 objects/s recovering 2024-09-13T18:21:41.117 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:40 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:21:41.117 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:40 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:21:41.118 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:40 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:21:41.118 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:40 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:21:41.118 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:40 smithi114 ceph-mon[30897]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:21:41.135 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808868 got 287762808874 for osd.9 2024-09-13T18:21:41.135 DEBUG:teuthology.parallel:result is None 2024-09-13T18:21:41.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:40 smithi204 ceph-mon[34777]: pgmap v330: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 11 op/s; 9 B/s, 0 objects/s recovering 2024-09-13T18:21:41.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:40 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:21:41.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:40 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:21:41.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:40 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:21:41.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:40 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:21:41.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:40 smithi204 ceph-mon[34777]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:21:41.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:40 smithi134 ceph-mon[34283]: pgmap v330: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 11 op/s; 9 B/s, 0 objects/s recovering 2024-09-13T18:21:41.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:40 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:21:41.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:40 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:21:41.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:40 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:21:41.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:40 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:21:41.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:40 smithi134 ceph-mon[34283]: from='mgr.14148 172.21.15.114:0/3055869758' entity='mgr.x' 2024-09-13T18:21:41.977 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574922 got 55834574927 for osd.1 2024-09-13T18:21:41.977 DEBUG:teuthology.parallel:result is None 2024-09-13T18:21:43.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:43.039+0000 7f2bcc018640 1 -- 172.21.15.114:0/3805640176 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2bc40fffe0 msgr2=0x7f2bc41003c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:43.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:43.039+0000 7f2bcc018640 1 --2- 172.21.15.114:0/3805640176 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2bc40fffe0 0x7f2bc41003c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc009930 tx=0x7f2bbc02a420 comp rx=0 tx=0).stop 2024-09-13T18:21:43.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:43.039+0000 7f2bcc018640 1 -- 172.21.15.114:0/3805640176 shutdown_connections 2024-09-13T18:21:43.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:43.039+0000 7f2bcc018640 1 --2- 172.21.15.114:0/3805640176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bc4105110 0x7f2bc410cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:43.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:43.039+0000 7f2bcc018640 1 --2- 172.21.15.114:0/3805640176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2bc4100990 0x7f2bc41049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:43.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:43.039+0000 7f2bcc018640 1 --2- 172.21.15.114:0/3805640176 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2bc40fffe0 0x7f2bc41003c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:43.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:43.039+0000 7f2bcc018640 1 -- 172.21.15.114:0/3805640176 >> 172.21.15.114:0/3805640176 conn(0x7f2bc40fbae0 msgr2=0x7f2bc40fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:43.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:43.039+0000 7f2bcc018640 1 -- 172.21.15.114:0/3805640176 shutdown_connections 2024-09-13T18:21:43.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:43.039+0000 7f2bcc018640 1 -- 172.21.15.114:0/3805640176 wait complete. 2024-09-13T18:21:43.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:43.040+0000 7f2bcc018640 1 Processor -- start 2024-09-13T18:21:43.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:43.040+0000 7f2bcc018640 1 -- start start 2024-09-13T18:21:43.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:43.040+0000 7f2bcc018640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2bc40fffe0 0x7f2bc40727b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:43.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:43.040+0000 7f2bcc018640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bc4100990 0x7f2bc4072cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:43.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:43.040+0000 7f2bcc018640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2bc4105110 0x7f2bc4073230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:43.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:43.040+0000 7f2bcc018640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc41a45b0 con 0x7f2bc40fffe0 2024-09-13T18:21:43.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:43.040+0000 7f2bcc018640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc41a4720 con 0x7f2bc4100990 2024-09-13T18:21:43.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:43.040+0000 7f2bcc018640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc41a48e0 con 0x7f2bc4105110 2024-09-13T18:21:43.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:43.040+0000 7f2bc9db4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2bc40fffe0 0x7f2bc40727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:43.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:43.040+0000 7f2bc9db4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2bc40fffe0 0x7f2bc40727b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50410/0 (socket says 172.21.15.114:50410) 2024-09-13T18:21:43.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:43.040+0000 7f2bc9db4640 1 -- 172.21.15.114:0/835834108 learned_addr learned my addr 172.21.15.114:0/835834108 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:43.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:43.040+0000 7f2bc95b3640 1 --2- 172.21.15.114:0/835834108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bc4100990 0x7f2bc4072cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:43.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:43.040+0000 7f2bc9db4640 1 -- 172.21.15.114:0/835834108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2bc4105110 msgr2=0x7f2bc4073230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:43.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:43.040+0000 7f2bc9db4640 1 --2- 172.21.15.114:0/835834108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2bc4105110 0x7f2bc4073230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:43.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:43.040+0000 7f2bc9db4640 1 -- 172.21.15.114:0/835834108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bc4100990 msgr2=0x7f2bc4072cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:43.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:43.040+0000 7f2bc9db4640 1 --2- 172.21.15.114:0/835834108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bc4100990 0x7f2bc4072cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:43.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:43.040+0000 7f2bc9db4640 1 -- 172.21.15.114:0/835834108 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bbc009560 con 0x7f2bc40fffe0 2024-09-13T18:21:43.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:43.040+0000 7f2bc9db4640 1 --2- 172.21.15.114:0/835834108 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2bc40fffe0 0x7f2bc40727b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc005ab0 tx=0x7f2bbc032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:43.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:43.040+0000 7f2bbaffd640 1 -- 172.21.15.114:0/835834108 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc032e10 con 0x7f2bc40fffe0 2024-09-13T18:21:43.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:43.041+0000 7f2bcc018640 1 -- 172.21.15.114:0/835834108 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bc41a4b60 con 0x7f2bc40fffe0 2024-09-13T18:21:43.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:43.041+0000 7f2bbaffd640 1 -- 172.21.15.114:0/835834108 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2bbc0046c0 con 0x7f2bc40fffe0 2024-09-13T18:21:43.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:43.041+0000 7f2bbaffd640 1 -- 172.21.15.114:0/835834108 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc003850 con 0x7f2bc40fffe0 2024-09-13T18:21:43.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:43.041+0000 7f2bcc018640 1 -- 172.21.15.114:0/835834108 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bc41a5050 con 0x7f2bc40fffe0 2024-09-13T18:21:43.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:43.042+0000 7f2bcc018640 1 -- 172.21.15.114:0/835834108 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bc41014c0 con 0x7f2bc40fffe0 2024-09-13T18:21:43.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:43.043+0000 7f2bbaffd640 1 -- 172.21.15.114:0/835834108 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f2bbc004830 con 0x7f2bc40fffe0 2024-09-13T18:21:43.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:43.043+0000 7f2bbaffd640 1 --2- 172.21.15.114:0/835834108 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f2bb00a1110 0x7f2bb00a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:43.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:43.043+0000 7f2bbaffd640 1 -- 172.21.15.114:0/835834108 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f2bbc0eda90 con 0x7f2bc40fffe0 2024-09-13T18:21:43.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:43.043+0000 7f2bc95b3640 1 --2- 172.21.15.114:0/835834108 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f2bb00a1110 0x7f2bb00a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:43.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:43.044+0000 7f2bc95b3640 1 --2- 172.21.15.114:0/835834108 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f2bb00a1110 0x7f2bb00a35d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4073f30 tx=0x7f2bb400b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:43.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:43.047+0000 7f2bbaffd640 1 -- 172.21.15.114:0/835834108 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f2bbc0b6cf0 con 0x7f2bc40fffe0 2024-09-13T18:21:43.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:42 smithi114 ceph-mon[30897]: pgmap v331: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 9 B/s, 0 objects/s recovering 2024-09-13T18:21:43.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:42 smithi204 ceph-mon[34777]: pgmap v331: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 9 B/s, 0 objects/s recovering 2024-09-13T18:21:43.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:43.211+0000 7f2bcc018640 1 -- 172.21.15.114:0/835834108 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f2bc41082f0 con 0x7f2bc40fffe0 2024-09-13T18:21:43.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:43.211+0000 7f2bbaffd640 1 -- 172.21.15.114:0/835834108 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2bbc0ba310 con 0x7f2bc40fffe0 2024-09-13T18:21:43.214 INFO:teuthology.orchestra.run.smithi114.stdout:163208757310 2024-09-13T18:21:43.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:43.214+0000 7f2bcc018640 1 -- 172.21.15.114:0/835834108 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f2bb00a1110 msgr2=0x7f2bb00a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:43.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:43.214+0000 7f2bcc018640 1 --2- 172.21.15.114:0/835834108 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f2bb00a1110 0x7f2bb00a35d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4073f30 tx=0x7f2bb400b390 comp rx=0 tx=0).stop 2024-09-13T18:21:43.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:43.214+0000 7f2bcc018640 1 -- 172.21.15.114:0/835834108 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2bc40fffe0 msgr2=0x7f2bc40727b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:43.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:43.214+0000 7f2bcc018640 1 --2- 172.21.15.114:0/835834108 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2bc40fffe0 0x7f2bc40727b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc005ab0 tx=0x7f2bbc032c20 comp rx=0 tx=0).stop 2024-09-13T18:21:43.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:43.214+0000 7f2bcc018640 1 -- 172.21.15.114:0/835834108 shutdown_connections 2024-09-13T18:21:43.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:43.214+0000 7f2bcc018640 1 --2- 172.21.15.114:0/835834108 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f2bb00a1110 0x7f2bb00a35d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:43.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:43.214+0000 7f2bcc018640 1 --2- 172.21.15.114:0/835834108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2bc4105110 0x7f2bc4073230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:43.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:43.214+0000 7f2bcc018640 1 --2- 172.21.15.114:0/835834108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bc4100990 0x7f2bc4072cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:43.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:43.214+0000 7f2bcc018640 1 --2- 172.21.15.114:0/835834108 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2bc40fffe0 0x7f2bc40727b0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:43.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:43.214+0000 7f2bcc018640 1 -- 172.21.15.114:0/835834108 >> 172.21.15.114:0/835834108 conn(0x7f2bc40fbae0 msgr2=0x7f2bc40fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:43.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:43.214+0000 7f2bcc018640 1 -- 172.21.15.114:0/835834108 shutdown_connections 2024-09-13T18:21:43.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:43.214+0000 7f2bcc018640 1 -- 172.21.15.114:0/835834108 wait complete. 2024-09-13T18:21:43.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:42 smithi134 ceph-mon[34283]: pgmap v331: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 9 B/s, 0 objects/s recovering 2024-09-13T18:21:43.914 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757304 got 163208757310 for osd.5 2024-09-13T18:21:43.914 DEBUG:teuthology.parallel:result is None 2024-09-13T18:21:43.915 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-13T18:21:43.915 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-13T18:21:44.121 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:21:44.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:43 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/835834108' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-13T18:21:44.219 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:43 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/835834108' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-13T18:21:44.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:43 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/835834108' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-13T18:21:45.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:44 smithi114 ceph-mon[30897]: pgmap v332: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-13T18:21:45.270 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:44 smithi134 ceph-mon[34283]: pgmap v332: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-13T18:21:45.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:44 smithi204 ceph-mon[34777]: pgmap v332: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-13T18:21:45.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:45.877+0000 7f001e83f640 1 -- 172.21.15.114:0/1870018451 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0018104160 msgr2=0x7f00181045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:45.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:45.877+0000 7f001e83f640 1 --2- 172.21.15.114:0/1870018451 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0018104160 0x7f00181045f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0010009ca0 tx=0x7f001002a940 comp rx=0 tx=0).stop 2024-09-13T18:21:45.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:45.877+0000 7f001e83f640 1 -- 172.21.15.114:0/1870018451 shutdown_connections 2024-09-13T18:21:45.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:45.877+0000 7f001e83f640 1 --2- 172.21.15.114:0/1870018451 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0018104160 0x7f00181045f0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:45.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:45.877+0000 7f001e83f640 1 --2- 172.21.15.114:0/1870018451 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00180ff670 0x7f0018103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:45.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:45.877+0000 7f001e83f640 1 --2- 172.21.15.114:0/1870018451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00180fed50 0x7f00180ff130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:45.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:45.877+0000 7f001e83f640 1 -- 172.21.15.114:0/1870018451 >> 172.21.15.114:0/1870018451 conn(0x7f00180fa9a0 msgr2=0x7f00180fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:45.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:45.877+0000 7f001e83f640 1 -- 172.21.15.114:0/1870018451 shutdown_connections 2024-09-13T18:21:45.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:45.877+0000 7f001e83f640 1 -- 172.21.15.114:0/1870018451 wait complete. 2024-09-13T18:21:45.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:45.878+0000 7f001e83f640 1 Processor -- start 2024-09-13T18:21:45.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:45.878+0000 7f001e83f640 1 -- start start 2024-09-13T18:21:45.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:45.879+0000 7f001e83f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f00180fed50 0x7f00181a2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:45.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:45.879+0000 7f001e83f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00180ff670 0x7f00181a2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:45.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:45.879+0000 7f001e83f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0018104160 0x7f001819c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:45.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:45.879+0000 7f001e83f640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001819cb70 con 0x7f00180fed50 2024-09-13T18:21:45.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:45.879+0000 7f001e83f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001819cce0 con 0x7f0018104160 2024-09-13T18:21:45.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:45.879+0000 7f001e83f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001819cea0 con 0x7f00180ff670 2024-09-13T18:21:45.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:45.879+0000 7f0017fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f00180fed50 0x7f00181a2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:45.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:45.879+0000 7f0017fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f00180fed50 0x7f00181a2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47274/0 (socket says 172.21.15.114:47274) 2024-09-13T18:21:45.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:45.879+0000 7f0017fff640 1 -- 172.21.15.114:0/2552529836 learned_addr learned my addr 172.21.15.114:0/2552529836 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:45.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:45.879+0000 7f00177fe640 1 --2- 172.21.15.114:0/2552529836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00180ff670 0x7f00181a2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:45.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:45.879+0000 7f001cddc640 1 --2- 172.21.15.114:0/2552529836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0018104160 0x7f001819c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:45.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:45.879+0000 7f0017fff640 1 -- 172.21.15.114:0/2552529836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00180ff670 msgr2=0x7f00181a2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:45.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:45.879+0000 7f0017fff640 1 --2- 172.21.15.114:0/2552529836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00180ff670 0x7f00181a2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:45.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:45.879+0000 7f0017fff640 1 -- 172.21.15.114:0/2552529836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0018104160 msgr2=0x7f001819c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:45.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:45.879+0000 7f0017fff640 1 --2- 172.21.15.114:0/2552529836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0018104160 0x7f001819c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:45.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:45.879+0000 7f0017fff640 1 -- 172.21.15.114:0/2552529836 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0010009900 con 0x7f00180fed50 2024-09-13T18:21:45.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:45.879+0000 7f0017fff640 1 --2- 172.21.15.114:0/2552529836 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f00180fed50 0x7f00181a2420 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f000800c920 tx=0x7f000800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:45.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:45.880+0000 7f00157fa640 1 -- 172.21.15.114:0/2552529836 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0008007c70 con 0x7f00180fed50 2024-09-13T18:21:45.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:45.880+0000 7f001e83f640 1 -- 172.21.15.114:0/2552529836 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f001819d180 con 0x7f00180fed50 2024-09-13T18:21:45.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:45.880+0000 7f001e83f640 1 -- 172.21.15.114:0/2552529836 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00181a94c0 con 0x7f00180fed50 2024-09-13T18:21:45.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:45.880+0000 7f00157fa640 1 -- 172.21.15.114:0/2552529836 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0008007dd0 con 0x7f00180fed50 2024-09-13T18:21:45.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:45.880+0000 7f00157fa640 1 -- 172.21.15.114:0/2552529836 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0008018730 con 0x7f00180fed50 2024-09-13T18:21:45.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:45.881+0000 7f001e83f640 1 -- 172.21.15.114:0/2552529836 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0004005250 con 0x7f00180fed50 2024-09-13T18:21:45.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:45.883+0000 7f00157fa640 1 -- 172.21.15.114:0/2552529836 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f0008018890 con 0x7f00180fed50 2024-09-13T18:21:45.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:45.883+0000 7f00157fa640 1 --2- 172.21.15.114:0/2552529836 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f00000a1190 0x7f00000a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:45.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:45.883+0000 7f00177fe640 1 --2- 172.21.15.114:0/2552529836 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f00000a1190 0x7f00000a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:45.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:45.883+0000 7f00157fa640 1 -- 172.21.15.114:0/2552529836 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f00080cd120 con 0x7f00180fed50 2024-09-13T18:21:45.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:45.884+0000 7f00177fe640 1 --2- 172.21.15.114:0/2552529836 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f00000a1190 0x7f00000a3650 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f001819dcf0 tx=0x7f000c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:45.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:45.886+0000 7f00157fa640 1 -- 172.21.15.114:0/2552529836 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f00080964f0 con 0x7f00180fed50 2024-09-13T18:21:46.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:46.055+0000 7f001e83f640 1 -- 172.21.15.114:0/2552529836 --> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0004000bc0 con 0x7f00000a1190 2024-09-13T18:21:46.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:46.066+0000 7f00157fa640 1 -- 172.21.15.114:0/2552529836 <== mgr.14148 v2:172.21.15.114:6800/3396185541 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+428862 (secure 0 0 0) 0x7f0004000bc0 con 0x7f00000a1190 2024-09-13T18:21:46.074 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:21:46.077 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-13T18:21:46.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:46.071+0000 7f001e83f640 1 -- 172.21.15.114:0/2552529836 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f00000a1190 msgr2=0x7f00000a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:46.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:46.071+0000 7f001e83f640 1 --2- 172.21.15.114:0/2552529836 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f00000a1190 0x7f00000a3650 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f001819dcf0 tx=0x7f000c005f90 comp rx=0 tx=0).stop 2024-09-13T18:21:46.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:46.071+0000 7f001e83f640 1 -- 172.21.15.114:0/2552529836 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f00180fed50 msgr2=0x7f00181a2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:46.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:46.071+0000 7f001e83f640 1 --2- 172.21.15.114:0/2552529836 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f00180fed50 0x7f00181a2420 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f000800c920 tx=0x7f000800cde0 comp rx=0 tx=0).stop 2024-09-13T18:21:46.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:46.071+0000 7f001e83f640 1 -- 172.21.15.114:0/2552529836 shutdown_connections 2024-09-13T18:21:46.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:46.071+0000 7f001e83f640 1 --2- 172.21.15.114:0/2552529836 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f00000a1190 0x7f00000a3650 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:46.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:46.071+0000 7f001e83f640 1 --2- 172.21.15.114:0/2552529836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0018104160 0x7f001819c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:46.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:46.071+0000 7f001e83f640 1 --2- 172.21.15.114:0/2552529836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00180ff670 0x7f00181a2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:46.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:46.071+0000 7f001e83f640 1 --2- 172.21.15.114:0/2552529836 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f00180fed50 0x7f00181a2420 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:46.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:46.071+0000 7f001e83f640 1 -- 172.21.15.114:0/2552529836 >> 172.21.15.114:0/2552529836 conn(0x7f00180fa9a0 msgr2=0x7f001810d920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:46.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:46.071+0000 7f001e83f640 1 -- 172.21.15.114:0/2552529836 shutdown_connections 2024-09-13T18:21:46.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:46.071+0000 7f001e83f640 1 -- 172.21.15.114:0/2552529836 wait complete. 2024-09-13T18:21:46.761 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":333,"stamp":"2024-09-13T18:21:45.930120+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660142,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":926,"num_read_kb":595,"num_write":416,"num_write_kb":3076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180875,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":344,"ondisk_log_size":344,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361416,"kb_used_data":12576,"kb_used_omap":18,"kb_used_meta":348653,"kb_avail":1124580408,"statfs":{"total":1151940427776,"available":1151570337792,"internally_reserved":0,"allocated":12877824,"data_stored":8180672,"data_compressed":394440,"data_compressed_allocated":540672,"data_compressed_original":2580480,"omap_allocated":19129,"internal_metadata":357020999},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003321"},"pg_stats":[{"pgid":"2.1f","version":"91'13","reported_seq":110,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:43.354444+0000","last_change":"2024-09-13T18:20:01.577029+0000","last_active":"2024-09-13T18:21:43.354444+0000","last_peered":"2024-09-13T18:21:43.354444+0000","last_clean":"2024-09-13T18:21:43.354444+0000","last_became_active":"2024-09-13T18:20:01.576710+0000","last_became_peered":"2024-09-13T18:20:01.576710+0000","last_unstale":"2024-09-13T18:21:43.354444+0000","last_undegraded":"2024-09-13T18:21:43.354444+0000","last_fullsized":"2024-09-13T18:21:43.354444+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:25:11.977436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":77,"num_read_kb":77,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.730811+0000","last_change":"2024-09-13T18:20:01.922955+0000","last_active":"2024-09-13T18:21:27.730811+0000","last_peered":"2024-09-13T18:21:27.730811+0000","last_clean":"2024-09-13T18:21:27.730811+0000","last_became_active":"2024-09-13T18:20:01.922704+0000","last_became_peered":"2024-09-13T18:20:01.922704+0000","last_unstale":"2024-09-13T18:21:27.730811+0000","last_undegraded":"2024-09-13T18:21:27.730811+0000","last_fullsized":"2024-09-13T18:21:27.730811+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:44:20.672784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.338127+0000","last_change":"2024-09-13T18:20:27.300985+0000","last_active":"2024-09-13T18:21:27.338127+0000","last_peered":"2024-09-13T18:21:27.338127+0000","last_clean":"2024-09-13T18:21:27.338127+0000","last_became_active":"2024-09-13T18:20:27.300546+0000","last_became_peered":"2024-09-13T18:20:27.300546+0000","last_unstale":"2024-09-13T18:21:27.338127+0000","last_undegraded":"2024-09-13T18:21:27.338127+0000","last_fullsized":"2024-09-13T18:21:27.338127+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:00:11.750929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.730958+0000","last_change":"2024-09-13T18:20:01.578921+0000","last_active":"2024-09-13T18:21:27.730958+0000","last_peered":"2024-09-13T18:21:27.730958+0000","last_clean":"2024-09-13T18:21:27.730958+0000","last_became_active":"2024-09-13T18:20:01.578627+0000","last_became_peered":"2024-09-13T18:20:01.578627+0000","last_unstale":"2024-09-13T18:21:27.730958+0000","last_undegraded":"2024-09-13T18:21:27.730958+0000","last_fullsized":"2024-09-13T18:21:27.730958+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:43:32.919156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.147483+0000","last_change":"2024-09-13T18:20:01.922901+0000","last_active":"2024-09-13T18:21:27.147483+0000","last_peered":"2024-09-13T18:21:27.147483+0000","last_clean":"2024-09-13T18:21:27.147483+0000","last_became_active":"2024-09-13T18:20:01.922604+0000","last_became_peered":"2024-09-13T18:20:01.922604+0000","last_unstale":"2024-09-13T18:21:27.147483+0000","last_undegraded":"2024-09-13T18:21:27.147483+0000","last_fullsized":"2024-09-13T18:21:27.147483+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:27:30.650010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.142773+0000","last_change":"2024-09-13T18:20:06.135406+0000","last_active":"2024-09-13T18:21:27.142773+0000","last_peered":"2024-09-13T18:21:27.142773+0000","last_clean":"2024-09-13T18:21:27.142773+0000","last_became_active":"2024-09-13T18:20:06.134558+0000","last_became_peered":"2024-09-13T18:20:06.134558+0000","last_unstale":"2024-09-13T18:21:27.142773+0000","last_undegraded":"2024-09-13T18:21:27.142773+0000","last_fullsized":"2024-09-13T18:21:27.142773+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:10.901966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'12","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.141998+0000","last_change":"2024-09-13T18:20:01.580040+0000","last_active":"2024-09-13T18:21:27.141998+0000","last_peered":"2024-09-13T18:21:27.141998+0000","last_clean":"2024-09-13T18:21:27.141998+0000","last_became_active":"2024-09-13T18:20:01.579715+0000","last_became_peered":"2024-09-13T18:20:01.579715+0000","last_unstale":"2024-09-13T18:21:27.141998+0000","last_undegraded":"2024-09-13T18:21:27.141998+0000","last_fullsized":"2024-09-13T18:21:27.141998+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:57:03.588169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.322207+0000","last_change":"2024-09-13T18:20:01.928314+0000","last_active":"2024-09-13T18:21:27.322207+0000","last_peered":"2024-09-13T18:21:27.322207+0000","last_clean":"2024-09-13T18:21:27.322207+0000","last_became_active":"2024-09-13T18:20:01.925356+0000","last_became_peered":"2024-09-13T18:20:01.925356+0000","last_unstale":"2024-09-13T18:21:27.322207+0000","last_undegraded":"2024-09-13T18:21:27.322207+0000","last_fullsized":"2024-09-13T18:21:27.322207+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:43.906842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.143221+0000","last_change":"2024-09-13T18:20:06.135886+0000","last_active":"2024-09-13T18:21:27.143221+0000","last_peered":"2024-09-13T18:21:27.143221+0000","last_clean":"2024-09-13T18:21:27.143221+0000","last_became_active":"2024-09-13T18:20:06.134587+0000","last_became_peered":"2024-09-13T18:20:06.134587+0000","last_unstale":"2024-09-13T18:21:27.143221+0000","last_undegraded":"2024-09-13T18:21:27.143221+0000","last_fullsized":"2024-09-13T18:21:27.143221+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:44:42.434729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"91'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.141890+0000","last_change":"2024-09-13T18:20:01.576908+0000","last_active":"2024-09-13T18:21:27.141890+0000","last_peered":"2024-09-13T18:21:27.141890+0000","last_clean":"2024-09-13T18:21:27.141890+0000","last_became_active":"2024-09-13T18:20:01.576478+0000","last_became_peered":"2024-09-13T18:20:01.576478+0000","last_unstale":"2024-09-13T18:21:27.141890+0000","last_undegraded":"2024-09-13T18:21:27.141890+0000","last_fullsized":"2024-09-13T18:21:27.141890+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:34:52.126901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.145190+0000","last_change":"2024-09-13T18:20:01.921960+0000","last_active":"2024-09-13T18:21:27.145190+0000","last_peered":"2024-09-13T18:21:27.145190+0000","last_clean":"2024-09-13T18:21:27.145190+0000","last_became_active":"2024-09-13T18:20:01.921652+0000","last_became_peered":"2024-09-13T18:20:01.921652+0000","last_unstale":"2024-09-13T18:21:27.145190+0000","last_undegraded":"2024-09-13T18:21:27.145190+0000","last_fullsized":"2024-09-13T18:21:27.145190+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:00:31.277613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.143043+0000","last_change":"2024-09-13T18:20:06.138983+0000","last_active":"2024-09-13T18:21:27.143043+0000","last_peered":"2024-09-13T18:21:27.143043+0000","last_clean":"2024-09-13T18:21:27.143043+0000","last_became_active":"2024-09-13T18:20:06.138708+0000","last_became_peered":"2024-09-13T18:20:06.138708+0000","last_unstale":"2024-09-13T18:21:27.143043+0000","last_undegraded":"2024-09-13T18:21:27.143043+0000","last_fullsized":"2024-09-13T18:21:27.143043+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:45:21.978217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.145723+0000","last_change":"2024-09-13T18:20:01.929817+0000","last_active":"2024-09-13T18:21:27.145723+0000","last_peered":"2024-09-13T18:21:27.145723+0000","last_clean":"2024-09-13T18:21:27.145723+0000","last_became_active":"2024-09-13T18:20:01.928907+0000","last_became_peered":"2024-09-13T18:20:01.928907+0000","last_unstale":"2024-09-13T18:21:27.145723+0000","last_undegraded":"2024-09-13T18:21:27.145723+0000","last_fullsized":"2024-09-13T18:21:27.145723+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:56:25.847230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.142094+0000","last_change":"2024-09-13T18:20:01.578968+0000","last_active":"2024-09-13T18:21:27.142094+0000","last_peered":"2024-09-13T18:21:27.142094+0000","last_clean":"2024-09-13T18:21:27.142094+0000","last_became_active":"2024-09-13T18:20:01.578623+0000","last_became_peered":"2024-09-13T18:20:01.578623+0000","last_unstale":"2024-09-13T18:21:27.142094+0000","last_undegraded":"2024-09-13T18:21:27.142094+0000","last_fullsized":"2024-09-13T18:21:27.142094+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:03:30.333107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.142804+0000","last_change":"2024-09-13T18:20:06.135977+0000","last_active":"2024-09-13T18:21:27.142804+0000","last_peered":"2024-09-13T18:21:27.142804+0000","last_clean":"2024-09-13T18:21:27.142804+0000","last_became_active":"2024-09-13T18:20:06.134844+0000","last_became_peered":"2024-09-13T18:20:06.134844+0000","last_unstale":"2024-09-13T18:21:27.142804+0000","last_undegraded":"2024-09-13T18:21:27.142804+0000","last_fullsized":"2024-09-13T18:21:27.142804+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:58:36.139240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.730582+0000","last_change":"2024-09-13T18:20:01.916675+0000","last_active":"2024-09-13T18:21:27.730582+0000","last_peered":"2024-09-13T18:21:27.730582+0000","last_clean":"2024-09-13T18:21:27.730582+0000","last_became_active":"2024-09-13T18:20:01.912902+0000","last_became_peered":"2024-09-13T18:20:01.912902+0000","last_unstale":"2024-09-13T18:21:27.730582+0000","last_undegraded":"2024-09-13T18:21:27.730582+0000","last_fullsized":"2024-09-13T18:21:27.730582+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:24:54.271569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"91'3","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.143363+0000","last_change":"2024-09-13T18:20:27.300845+0000","last_active":"2024-09-13T18:21:27.143363+0000","last_peered":"2024-09-13T18:21:27.143363+0000","last_clean":"2024-09-13T18:21:27.143363+0000","last_became_active":"2024-09-13T18:20:27.300417+0000","last_became_peered":"2024-09-13T18:20:27.300417+0000","last_unstale":"2024-09-13T18:21:27.143363+0000","last_undegraded":"2024-09-13T18:21:27.143363+0000","last_fullsized":"2024-09-13T18:21:27.143363+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:44:27.507140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:28.235707+0000","last_change":"2024-09-13T18:21:28.235707+0000","last_active":"2024-09-13T18:21:28.235707+0000","last_peered":"2024-09-13T18:21:28.235707+0000","last_clean":"2024-09-13T18:21:28.235707+0000","last_became_active":"2024-09-13T18:21:28.231794+0000","last_became_peered":"2024-09-13T18:21:28.231794+0000","last_unstale":"2024-09-13T18:21:28.235707+0000","last_undegraded":"2024-09-13T18:21:28.235707+0000","last_fullsized":"2024-09-13T18:21:28.235707+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:01:00.113821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.143150+0000","last_change":"2024-09-13T18:20:01.913634+0000","last_active":"2024-09-13T18:21:27.143150+0000","last_peered":"2024-09-13T18:21:27.143150+0000","last_clean":"2024-09-13T18:21:27.143150+0000","last_became_active":"2024-09-13T18:20:01.913128+0000","last_became_peered":"2024-09-13T18:20:01.913128+0000","last_unstale":"2024-09-13T18:21:27.143150+0000","last_undegraded":"2024-09-13T18:21:27.143150+0000","last_fullsized":"2024-09-13T18:21:27.143150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:37:15.251756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"91'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.730712+0000","last_change":"2024-09-13T18:20:01.579385+0000","last_active":"2024-09-13T18:21:27.730712+0000","last_peered":"2024-09-13T18:21:27.730712+0000","last_clean":"2024-09-13T18:21:27.730712+0000","last_became_active":"2024-09-13T18:20:01.579102+0000","last_became_peered":"2024-09-13T18:20:01.579102+0000","last_unstale":"2024-09-13T18:21:27.730712+0000","last_undegraded":"2024-09-13T18:21:27.730712+0000","last_fullsized":"2024-09-13T18:21:27.730712+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:22:00.432052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.140056+0000","last_change":"2024-09-13T18:20:06.138110+0000","last_active":"2024-09-13T18:21:27.140056+0000","last_peered":"2024-09-13T18:21:27.140056+0000","last_clean":"2024-09-13T18:21:27.140056+0000","last_became_active":"2024-09-13T18:20:06.137763+0000","last_became_peered":"2024-09-13T18:20:06.137763+0000","last_unstale":"2024-09-13T18:21:27.140056+0000","last_undegraded":"2024-09-13T18:21:27.140056+0000","last_fullsized":"2024-09-13T18:21:27.140056+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:16:15.956216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.141263+0000","last_change":"2024-09-13T18:20:01.918318+0000","last_active":"2024-09-13T18:21:27.141263+0000","last_peered":"2024-09-13T18:21:27.141263+0000","last_clean":"2024-09-13T18:21:27.141263+0000","last_became_active":"2024-09-13T18:20:01.917977+0000","last_became_peered":"2024-09-13T18:20:01.917977+0000","last_unstale":"2024-09-13T18:21:27.141263+0000","last_undegraded":"2024-09-13T18:21:27.141263+0000","last_fullsized":"2024-09-13T18:21:27.141263+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:23.368866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.321998+0000","last_change":"2024-09-13T18:20:01.572009+0000","last_active":"2024-09-13T18:21:27.321998+0000","last_peered":"2024-09-13T18:21:27.321998+0000","last_clean":"2024-09-13T18:21:27.321998+0000","last_became_active":"2024-09-13T18:20:01.571313+0000","last_became_peered":"2024-09-13T18:20:01.571313+0000","last_unstale":"2024-09-13T18:21:27.321998+0000","last_undegraded":"2024-09-13T18:21:27.321998+0000","last_fullsized":"2024-09-13T18:21:27.321998+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:32:59.340476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.143064+0000","last_change":"2024-09-13T18:20:06.132212+0000","last_active":"2024-09-13T18:21:27.143064+0000","last_peered":"2024-09-13T18:21:27.143064+0000","last_clean":"2024-09-13T18:21:27.143064+0000","last_became_active":"2024-09-13T18:20:06.131809+0000","last_became_peered":"2024-09-13T18:20:06.131809+0000","last_unstale":"2024-09-13T18:21:27.143064+0000","last_undegraded":"2024-09-13T18:21:27.143064+0000","last_fullsized":"2024-09-13T18:21:27.143064+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:18:11.667551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.147352+0000","last_change":"2024-09-13T18:20:27.301112+0000","last_active":"2024-09-13T18:21:27.147352+0000","last_peered":"2024-09-13T18:21:27.147352+0000","last_clean":"2024-09-13T18:21:27.147352+0000","last_became_active":"2024-09-13T18:20:27.300772+0000","last_became_peered":"2024-09-13T18:20:27.300772+0000","last_unstale":"2024-09-13T18:21:27.147352+0000","last_undegraded":"2024-09-13T18:21:27.147352+0000","last_fullsized":"2024-09-13T18:21:27.147352+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:59:57.606105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"91'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.143891+0000","last_change":"2024-09-13T18:20:01.572025+0000","last_active":"2024-09-13T18:21:27.143891+0000","last_peered":"2024-09-13T18:21:27.143891+0000","last_clean":"2024-09-13T18:21:27.143891+0000","last_became_active":"2024-09-13T18:20:01.571321+0000","last_became_peered":"2024-09-13T18:20:01.571321+0000","last_unstale":"2024-09-13T18:21:27.143891+0000","last_undegraded":"2024-09-13T18:21:27.143891+0000","last_fullsized":"2024-09-13T18:21:27.143891+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:09.374364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.338246+0000","last_change":"2024-09-13T18:20:06.130687+0000","last_active":"2024-09-13T18:21:27.338246+0000","last_peered":"2024-09-13T18:21:27.338246+0000","last_clean":"2024-09-13T18:21:27.338246+0000","last_became_active":"2024-09-13T18:20:06.129839+0000","last_became_peered":"2024-09-13T18:20:06.129839+0000","last_unstale":"2024-09-13T18:21:27.338246+0000","last_undegraded":"2024-09-13T18:21:27.338246+0000","last_fullsized":"2024-09-13T18:21:27.338246+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:12:52.334945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.730952+0000","last_change":"2024-09-13T18:20:01.917244+0000","last_active":"2024-09-13T18:21:27.730952+0000","last_peered":"2024-09-13T18:21:27.730952+0000","last_clean":"2024-09-13T18:21:27.730952+0000","last_became_active":"2024-09-13T18:20:01.916948+0000","last_became_peered":"2024-09-13T18:20:01.916948+0000","last_unstale":"2024-09-13T18:21:27.730952+0000","last_undegraded":"2024-09-13T18:21:27.730952+0000","last_fullsized":"2024-09-13T18:21:27.730952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:17:11.967096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'9","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.140219+0000","last_change":"2024-09-13T18:20:01.569123+0000","last_active":"2024-09-13T18:21:27.140219+0000","last_peered":"2024-09-13T18:21:27.140219+0000","last_clean":"2024-09-13T18:21:27.140219+0000","last_became_active":"2024-09-13T18:20:01.568720+0000","last_became_peered":"2024-09-13T18:20:01.568720+0000","last_unstale":"2024-09-13T18:21:27.140219+0000","last_undegraded":"2024-09-13T18:21:27.140219+0000","last_fullsized":"2024-09-13T18:21:27.140219+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:04:29.561013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.730418+0000","last_change":"2024-09-13T18:20:06.141771+0000","last_active":"2024-09-13T18:21:27.730418+0000","last_peered":"2024-09-13T18:21:27.730418+0000","last_clean":"2024-09-13T18:21:27.730418+0000","last_became_active":"2024-09-13T18:20:06.141362+0000","last_became_peered":"2024-09-13T18:20:06.141362+0000","last_unstale":"2024-09-13T18:21:27.730418+0000","last_undegraded":"2024-09-13T18:21:27.730418+0000","last_fullsized":"2024-09-13T18:21:27.730418+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:09:58.597605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.144199+0000","last_change":"2024-09-13T18:20:01.911048+0000","last_active":"2024-09-13T18:21:27.144199+0000","last_peered":"2024-09-13T18:21:27.144199+0000","last_clean":"2024-09-13T18:21:27.144199+0000","last_became_active":"2024-09-13T18:20:01.910735+0000","last_became_peered":"2024-09-13T18:20:01.910735+0000","last_unstale":"2024-09-13T18:21:27.144199+0000","last_undegraded":"2024-09-13T18:21:27.144199+0000","last_fullsized":"2024-09-13T18:21:27.144199+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:07:35.447676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"91'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.142192+0000","last_change":"2024-09-13T18:20:01.571627+0000","last_active":"2024-09-13T18:21:27.142192+0000","last_peered":"2024-09-13T18:21:27.142192+0000","last_clean":"2024-09-13T18:21:27.142192+0000","last_became_active":"2024-09-13T18:20:01.571250+0000","last_became_peered":"2024-09-13T18:20:01.571250+0000","last_unstale":"2024-09-13T18:21:27.142192+0000","last_undegraded":"2024-09-13T18:21:27.142192+0000","last_fullsized":"2024-09-13T18:21:27.142192+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:34:03.479324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.143306+0000","last_change":"2024-09-13T18:20:06.132974+0000","last_active":"2024-09-13T18:21:27.143306+0000","last_peered":"2024-09-13T18:21:27.143306+0000","last_clean":"2024-09-13T18:21:27.143306+0000","last_became_active":"2024-09-13T18:20:06.131668+0000","last_became_peered":"2024-09-13T18:20:06.131668+0000","last_unstale":"2024-09-13T18:21:27.143306+0000","last_undegraded":"2024-09-13T18:21:27.143306+0000","last_fullsized":"2024-09-13T18:21:27.143306+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:36:30.329761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.141677+0000","last_change":"2024-09-13T18:20:01.904670+0000","last_active":"2024-09-13T18:21:27.141677+0000","last_peered":"2024-09-13T18:21:27.141677+0000","last_clean":"2024-09-13T18:21:27.141677+0000","last_became_active":"2024-09-13T18:20:01.904316+0000","last_became_peered":"2024-09-13T18:20:01.904316+0000","last_unstale":"2024-09-13T18:21:27.141677+0000","last_undegraded":"2024-09-13T18:21:27.141677+0000","last_fullsized":"2024-09-13T18:21:27.141677+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:08:17.687179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"91'1","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.140077+0000","last_change":"2024-09-13T18:20:01.564750+0000","last_active":"2024-09-13T18:21:27.140077+0000","last_peered":"2024-09-13T18:21:27.140077+0000","last_clean":"2024-09-13T18:21:27.140077+0000","last_became_active":"2024-09-13T18:20:01.564096+0000","last_became_peered":"2024-09-13T18:20:01.564096+0000","last_unstale":"2024-09-13T18:21:27.140077+0000","last_undegraded":"2024-09-13T18:21:27.140077+0000","last_fullsized":"2024-09-13T18:21:27.140077+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:42:19.588824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.141170+0000","last_change":"2024-09-13T18:20:06.133386+0000","last_active":"2024-09-13T18:21:27.141170+0000","last_peered":"2024-09-13T18:21:27.141170+0000","last_clean":"2024-09-13T18:21:27.141170+0000","last_became_active":"2024-09-13T18:20:06.133004+0000","last_became_peered":"2024-09-13T18:20:06.133004+0000","last_unstale":"2024-09-13T18:21:27.141170+0000","last_undegraded":"2024-09-13T18:21:27.141170+0000","last_fullsized":"2024-09-13T18:21:27.141170+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:54:46.440410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.141335+0000","last_change":"2024-09-13T18:20:01.912823+0000","last_active":"2024-09-13T18:21:27.141335+0000","last_peered":"2024-09-13T18:21:27.141335+0000","last_clean":"2024-09-13T18:21:27.141335+0000","last_became_active":"2024-09-13T18:20:01.912123+0000","last_became_peered":"2024-09-13T18:20:01.912123+0000","last_unstale":"2024-09-13T18:21:27.141335+0000","last_undegraded":"2024-09-13T18:21:27.141335+0000","last_fullsized":"2024-09-13T18:21:27.141335+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:32:33.544816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"91'5","reported_seq":98,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:39.350147+0000","last_change":"2024-09-13T18:20:01.577095+0000","last_active":"2024-09-13T18:21:39.350147+0000","last_peered":"2024-09-13T18:21:39.350147+0000","last_clean":"2024-09-13T18:21:39.350147+0000","last_became_active":"2024-09-13T18:20:01.576765+0000","last_became_peered":"2024-09-13T18:20:01.576765+0000","last_unstale":"2024-09-13T18:21:39.350147+0000","last_undegraded":"2024-09-13T18:21:39.350147+0000","last_fullsized":"2024-09-13T18:21:39.350147+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:44:08.750441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.143896+0000","last_change":"2024-09-13T18:20:06.131650+0000","last_active":"2024-09-13T18:21:27.143896+0000","last_peered":"2024-09-13T18:21:27.143896+0000","last_clean":"2024-09-13T18:21:27.143896+0000","last_became_active":"2024-09-13T18:20:06.130902+0000","last_became_peered":"2024-09-13T18:20:06.130902+0000","last_unstale":"2024-09-13T18:21:27.143896+0000","last_undegraded":"2024-09-13T18:21:27.143896+0000","last_fullsized":"2024-09-13T18:21:27.143896+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.366694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.144134+0000","last_change":"2024-09-13T18:20:01.918221+0000","last_active":"2024-09-13T18:21:27.144134+0000","last_peered":"2024-09-13T18:21:27.144134+0000","last_clean":"2024-09-13T18:21:27.144134+0000","last_became_active":"2024-09-13T18:20:01.917134+0000","last_became_peered":"2024-09-13T18:20:01.917134+0000","last_unstale":"2024-09-13T18:21:27.144134+0000","last_undegraded":"2024-09-13T18:21:27.144134+0000","last_fullsized":"2024-09-13T18:21:27.144134+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:19:06.269454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'5","reported_seq":102,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:42.353281+0000","last_change":"2024-09-13T18:20:01.572258+0000","last_active":"2024-09-13T18:21:42.353281+0000","last_peered":"2024-09-13T18:21:42.353281+0000","last_clean":"2024-09-13T18:21:42.353281+0000","last_became_active":"2024-09-13T18:20:01.571968+0000","last_became_peered":"2024-09-13T18:20:01.571968+0000","last_unstale":"2024-09-13T18:21:42.353281+0000","last_undegraded":"2024-09-13T18:21:42.353281+0000","last_fullsized":"2024-09-13T18:21:42.353281+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:40:59.143123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":77,"num_read_kb":77,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.144035+0000","last_change":"2024-09-13T18:20:06.131714+0000","last_active":"2024-09-13T18:21:27.144035+0000","last_peered":"2024-09-13T18:21:27.144035+0000","last_clean":"2024-09-13T18:21:27.144035+0000","last_became_active":"2024-09-13T18:20:06.131244+0000","last_became_peered":"2024-09-13T18:20:06.131244+0000","last_unstale":"2024-09-13T18:21:27.144035+0000","last_undegraded":"2024-09-13T18:21:27.144035+0000","last_fullsized":"2024-09-13T18:21:27.144035+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:37.946002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.143667+0000","last_change":"2024-09-13T18:20:01.917351+0000","last_active":"2024-09-13T18:21:27.143667+0000","last_peered":"2024-09-13T18:21:27.143667+0000","last_clean":"2024-09-13T18:21:27.143667+0000","last_became_active":"2024-09-13T18:20:01.917067+0000","last_became_peered":"2024-09-13T18:20:01.917067+0000","last_unstale":"2024-09-13T18:21:27.143667+0000","last_undegraded":"2024-09-13T18:21:27.143667+0000","last_fullsized":"2024-09-13T18:21:27.143667+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:35:37.723132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'4","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.145517+0000","last_change":"2024-09-13T18:20:01.573515+0000","last_active":"2024-09-13T18:21:27.145517+0000","last_peered":"2024-09-13T18:21:27.145517+0000","last_clean":"2024-09-13T18:21:27.145517+0000","last_became_active":"2024-09-13T18:20:01.573211+0000","last_became_peered":"2024-09-13T18:20:01.573211+0000","last_unstale":"2024-09-13T18:21:27.145517+0000","last_undegraded":"2024-09-13T18:21:27.145517+0000","last_fullsized":"2024-09-13T18:21:27.145517+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:05:30.912467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:28.232099+0000","last_change":"2024-09-13T18:21:28.231613+0000","last_active":"2024-09-13T18:21:28.232099+0000","last_peered":"2024-09-13T18:21:28.232099+0000","last_clean":"2024-09-13T18:21:28.232099+0000","last_became_active":"2024-09-13T18:21:28.231243+0000","last_became_peered":"2024-09-13T18:21:28.231243+0000","last_unstale":"2024-09-13T18:21:28.232099+0000","last_undegraded":"2024-09-13T18:21:28.232099+0000","last_fullsized":"2024-09-13T18:21:28.232099+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:58:46.066195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.730679+0000","last_change":"2024-09-13T18:20:01.921597+0000","last_active":"2024-09-13T18:21:27.730679+0000","last_peered":"2024-09-13T18:21:27.730679+0000","last_clean":"2024-09-13T18:21:27.730679+0000","last_became_active":"2024-09-13T18:20:01.920200+0000","last_became_peered":"2024-09-13T18:20:01.920200+0000","last_unstale":"2024-09-13T18:21:27.730679+0000","last_undegraded":"2024-09-13T18:21:27.730679+0000","last_fullsized":"2024-09-13T18:21:27.730679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:22:40.517941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'6","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.143488+0000","last_change":"2024-09-13T18:20:01.571313+0000","last_active":"2024-09-13T18:21:27.143488+0000","last_peered":"2024-09-13T18:21:27.143488+0000","last_clean":"2024-09-13T18:21:27.143488+0000","last_became_active":"2024-09-13T18:20:01.570977+0000","last_became_peered":"2024-09-13T18:20:01.570977+0000","last_unstale":"2024-09-13T18:21:27.143488+0000","last_undegraded":"2024-09-13T18:21:27.143488+0000","last_fullsized":"2024-09-13T18:21:27.143488+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:06.870320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":346,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:28.154829+0000","last_change":"2024-09-13T18:18:51.237409+0000","last_active":"2024-09-13T18:21:28.154829+0000","last_peered":"2024-09-13T18:21:28.154829+0000","last_clean":"2024-09-13T18:21:28.154829+0000","last_became_active":"2024-09-13T18:18:51.236986+0000","last_became_peered":"2024-09-13T18:18:51.236986+0000","last_unstale":"2024-09-13T18:21:28.154829+0000","last_undegraded":"2024-09-13T18:21:28.154829+0000","last_fullsized":"2024-09-13T18:21:28.154829+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:15:40.757299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:15:40.757299+0000","last_clean_scrub_stamp":"2024-09-13T18:15:40.757299+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:58:29.679404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.145643+0000","last_change":"2024-09-13T18:20:06.130803+0000","last_active":"2024-09-13T18:21:27.145643+0000","last_peered":"2024-09-13T18:21:27.145643+0000","last_clean":"2024-09-13T18:21:27.145643+0000","last_became_active":"2024-09-13T18:20:06.130433+0000","last_became_peered":"2024-09-13T18:20:06.130433+0000","last_unstale":"2024-09-13T18:21:27.145643+0000","last_undegraded":"2024-09-13T18:21:27.145643+0000","last_fullsized":"2024-09-13T18:21:27.145643+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:23:11.787084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.145903+0000","last_change":"2024-09-13T18:20:01.922702+0000","last_active":"2024-09-13T18:21:27.145903+0000","last_peered":"2024-09-13T18:21:27.145903+0000","last_clean":"2024-09-13T18:21:27.145903+0000","last_became_active":"2024-09-13T18:20:01.921109+0000","last_became_peered":"2024-09-13T18:20:01.921109+0000","last_unstale":"2024-09-13T18:21:27.145903+0000","last_undegraded":"2024-09-13T18:21:27.145903+0000","last_fullsized":"2024-09-13T18:21:27.145903+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:03:34.618054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"91'5","reported_seq":105,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:44.359394+0000","last_change":"2024-09-13T18:20:01.571024+0000","last_active":"2024-09-13T18:21:44.359394+0000","last_peered":"2024-09-13T18:21:44.359394+0000","last_clean":"2024-09-13T18:21:44.359394+0000","last_became_active":"2024-09-13T18:20:01.570687+0000","last_became_peered":"2024-09-13T18:20:01.570687+0000","last_unstale":"2024-09-13T18:21:44.359394+0000","last_undegraded":"2024-09-13T18:21:44.359394+0000","last_fullsized":"2024-09-13T18:21:44.359394+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:10:25.126179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":80,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.321866+0000","last_change":"2024-09-13T18:20:06.129025+0000","last_active":"2024-09-13T18:21:27.321866+0000","last_peered":"2024-09-13T18:21:27.321866+0000","last_clean":"2024-09-13T18:21:27.321866+0000","last_became_active":"2024-09-13T18:20:06.128618+0000","last_became_peered":"2024-09-13T18:20:06.128618+0000","last_unstale":"2024-09-13T18:21:27.321866+0000","last_undegraded":"2024-09-13T18:21:27.321866+0000","last_fullsized":"2024-09-13T18:21:27.321866+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:20:08.891259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.146304+0000","last_change":"2024-09-13T18:20:01.921697+0000","last_active":"2024-09-13T18:21:27.146304+0000","last_peered":"2024-09-13T18:21:27.146304+0000","last_clean":"2024-09-13T18:21:27.146304+0000","last_became_active":"2024-09-13T18:20:01.920174+0000","last_became_peered":"2024-09-13T18:20:01.920174+0000","last_unstale":"2024-09-13T18:21:27.146304+0000","last_undegraded":"2024-09-13T18:21:27.146304+0000","last_fullsized":"2024-09-13T18:21:27.146304+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:57:03.922725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"91'3","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.145321+0000","last_change":"2024-09-13T18:20:01.582102+0000","last_active":"2024-09-13T18:21:27.145321+0000","last_peered":"2024-09-13T18:21:27.145321+0000","last_clean":"2024-09-13T18:21:27.145321+0000","last_became_active":"2024-09-13T18:20:01.581841+0000","last_became_peered":"2024-09-13T18:20:01.581841+0000","last_unstale":"2024-09-13T18:21:27.145321+0000","last_undegraded":"2024-09-13T18:21:27.145321+0000","last_fullsized":"2024-09-13T18:21:27.145321+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:59:57.929703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.147124+0000","last_change":"2024-09-13T18:20:06.133126+0000","last_active":"2024-09-13T18:21:27.147124+0000","last_peered":"2024-09-13T18:21:27.147124+0000","last_clean":"2024-09-13T18:21:27.147124+0000","last_became_active":"2024-09-13T18:20:06.132747+0000","last_became_peered":"2024-09-13T18:20:06.132747+0000","last_unstale":"2024-09-13T18:21:27.147124+0000","last_undegraded":"2024-09-13T18:21:27.147124+0000","last_fullsized":"2024-09-13T18:21:27.147124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:54:21.843090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.145788+0000","last_change":"2024-09-13T18:20:01.916306+0000","last_active":"2024-09-13T18:21:27.145788+0000","last_peered":"2024-09-13T18:21:27.145788+0000","last_clean":"2024-09-13T18:21:27.145788+0000","last_became_active":"2024-09-13T18:20:01.915739+0000","last_became_peered":"2024-09-13T18:20:01.915739+0000","last_unstale":"2024-09-13T18:21:27.145788+0000","last_undegraded":"2024-09-13T18:21:27.145788+0000","last_fullsized":"2024-09-13T18:21:27.145788+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:19:09.689909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"91'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.144275+0000","last_change":"2024-09-13T18:20:01.565821+0000","last_active":"2024-09-13T18:21:27.144275+0000","last_peered":"2024-09-13T18:21:27.144275+0000","last_clean":"2024-09-13T18:21:27.144275+0000","last_became_active":"2024-09-13T18:20:01.565467+0000","last_became_peered":"2024-09-13T18:20:01.565467+0000","last_unstale":"2024-09-13T18:21:27.144275+0000","last_undegraded":"2024-09-13T18:21:27.144275+0000","last_fullsized":"2024-09-13T18:21:27.144275+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:11.609400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.143920+0000","last_change":"2024-09-13T18:20:06.131348+0000","last_active":"2024-09-13T18:21:27.143920+0000","last_peered":"2024-09-13T18:21:27.143920+0000","last_clean":"2024-09-13T18:21:27.143920+0000","last_became_active":"2024-09-13T18:20:06.131019+0000","last_became_peered":"2024-09-13T18:20:06.131019+0000","last_unstale":"2024-09-13T18:21:27.143920+0000","last_undegraded":"2024-09-13T18:21:27.143920+0000","last_fullsized":"2024-09-13T18:21:27.143920+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:13:49.717503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.143749+0000","last_change":"2024-09-13T18:20:01.922306+0000","last_active":"2024-09-13T18:21:27.143749+0000","last_peered":"2024-09-13T18:21:27.143749+0000","last_clean":"2024-09-13T18:21:27.143749+0000","last_became_active":"2024-09-13T18:20:01.921956+0000","last_became_peered":"2024-09-13T18:20:01.921956+0000","last_unstale":"2024-09-13T18:21:27.143749+0000","last_undegraded":"2024-09-13T18:21:27.143749+0000","last_fullsized":"2024-09-13T18:21:27.143749+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:17:48.755318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.144213+0000","last_change":"2024-09-13T18:20:27.194085+0000","last_active":"2024-09-13T18:21:27.144213+0000","last_peered":"2024-09-13T18:21:27.144213+0000","last_clean":"2024-09-13T18:21:27.144213+0000","last_became_active":"2024-09-13T18:20:27.190136+0000","last_became_peered":"2024-09-13T18:20:27.190136+0000","last_unstale":"2024-09-13T18:21:27.144213+0000","last_undegraded":"2024-09-13T18:21:27.144213+0000","last_fullsized":"2024-09-13T18:21:27.144213+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:57:07.202901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.144120+0000","last_change":"2024-09-13T18:20:06.135498+0000","last_active":"2024-09-13T18:21:27.144120+0000","last_peered":"2024-09-13T18:21:27.144120+0000","last_clean":"2024-09-13T18:21:27.144120+0000","last_became_active":"2024-09-13T18:20:06.135095+0000","last_became_peered":"2024-09-13T18:20:06.135095+0000","last_unstale":"2024-09-13T18:21:27.144120+0000","last_undegraded":"2024-09-13T18:21:27.144120+0000","last_fullsized":"2024-09-13T18:21:27.144120+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:06:40.609718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.139672+0000","last_change":"2024-09-13T18:20:27.186825+0000","last_active":"2024-09-13T18:21:27.139672+0000","last_peered":"2024-09-13T18:21:27.139672+0000","last_clean":"2024-09-13T18:21:27.139672+0000","last_became_active":"2024-09-13T18:20:27.186523+0000","last_became_peered":"2024-09-13T18:20:27.186523+0000","last_unstale":"2024-09-13T18:21:27.139672+0000","last_undegraded":"2024-09-13T18:21:27.139672+0000","last_fullsized":"2024-09-13T18:21:27.139672+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:28:30.453945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.10","version":"91'17","reported_seq":59,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.140911+0000","last_change":"2024-09-13T18:20:27.200142+0000","last_active":"2024-09-13T18:21:27.140911+0000","last_peered":"2024-09-13T18:21:27.140911+0000","last_clean":"2024-09-13T18:21:27.140911+0000","last_became_active":"2024-09-13T18:20:27.189670+0000","last_became_peered":"2024-09-13T18:20:27.189670+0000","last_unstale":"2024-09-13T18:21:27.140911+0000","last_undegraded":"2024-09-13T18:21:27.140911+0000","last_fullsized":"2024-09-13T18:21:27.140911+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:16:02.934269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.144245+0000","last_change":"2024-09-13T18:20:06.136742+0000","last_active":"2024-09-13T18:21:27.144245+0000","last_peered":"2024-09-13T18:21:27.144245+0000","last_clean":"2024-09-13T18:21:27.144245+0000","last_became_active":"2024-09-13T18:20:06.136373+0000","last_became_peered":"2024-09-13T18:20:06.136373+0000","last_unstale":"2024-09-13T18:21:27.144245+0000","last_undegraded":"2024-09-13T18:21:27.144245+0000","last_fullsized":"2024-09-13T18:21:27.144245+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:49:11.782489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.145415+0000","last_change":"2024-09-13T18:20:01.922910+0000","last_active":"2024-09-13T18:21:27.145415+0000","last_peered":"2024-09-13T18:21:27.145415+0000","last_clean":"2024-09-13T18:21:27.145415+0000","last_became_active":"2024-09-13T18:20:01.921162+0000","last_became_peered":"2024-09-13T18:20:01.921162+0000","last_unstale":"2024-09-13T18:21:27.145415+0000","last_undegraded":"2024-09-13T18:21:27.145415+0000","last_fullsized":"2024-09-13T18:21:27.145415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:14:47.243418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"91'5","reported_seq":101,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:40.351501+0000","last_change":"2024-09-13T18:20:01.575830+0000","last_active":"2024-09-13T18:21:40.351501+0000","last_peered":"2024-09-13T18:21:40.351501+0000","last_clean":"2024-09-13T18:21:40.351501+0000","last_became_active":"2024-09-13T18:20:01.575334+0000","last_became_peered":"2024-09-13T18:20:01.575334+0000","last_unstale":"2024-09-13T18:21:40.351501+0000","last_undegraded":"2024-09-13T18:21:40.351501+0000","last_fullsized":"2024-09-13T18:21:40.351501+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:13:00.935600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":77,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.140407+0000","last_change":"2024-09-13T18:20:06.138609+0000","last_active":"2024-09-13T18:21:27.140407+0000","last_peered":"2024-09-13T18:21:27.140407+0000","last_clean":"2024-09-13T18:21:27.140407+0000","last_became_active":"2024-09-13T18:20:06.138232+0000","last_became_peered":"2024-09-13T18:20:06.138232+0000","last_unstale":"2024-09-13T18:21:27.140407+0000","last_undegraded":"2024-09-13T18:21:27.140407+0000","last_fullsized":"2024-09-13T18:21:27.140407+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:46:06.227407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.145886+0000","last_change":"2024-09-13T18:20:01.928757+0000","last_active":"2024-09-13T18:21:27.145886+0000","last_peered":"2024-09-13T18:21:27.145886+0000","last_clean":"2024-09-13T18:21:27.145886+0000","last_became_active":"2024-09-13T18:20:01.928350+0000","last_became_peered":"2024-09-13T18:20:01.928350+0000","last_unstale":"2024-09-13T18:21:27.145886+0000","last_undegraded":"2024-09-13T18:21:27.145886+0000","last_fullsized":"2024-09-13T18:21:27.145886+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:17:43.208106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"91'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.145386+0000","last_change":"2024-09-13T18:20:01.579128+0000","last_active":"2024-09-13T18:21:27.145386+0000","last_peered":"2024-09-13T18:21:27.145386+0000","last_clean":"2024-09-13T18:21:27.145386+0000","last_became_active":"2024-09-13T18:20:01.578796+0000","last_became_peered":"2024-09-13T18:20:01.578796+0000","last_unstale":"2024-09-13T18:21:27.145386+0000","last_undegraded":"2024-09-13T18:21:27.145386+0000","last_fullsized":"2024-09-13T18:21:27.145386+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:30:04.836103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.143306+0000","last_change":"2024-09-13T18:20:06.131404+0000","last_active":"2024-09-13T18:21:27.143306+0000","last_peered":"2024-09-13T18:21:27.143306+0000","last_clean":"2024-09-13T18:21:27.143306+0000","last_became_active":"2024-09-13T18:20:06.130931+0000","last_became_peered":"2024-09-13T18:20:06.130931+0000","last_unstale":"2024-09-13T18:21:27.143306+0000","last_undegraded":"2024-09-13T18:21:27.143306+0000","last_fullsized":"2024-09-13T18:21:27.143306+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:55:54.568816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.144146+0000","last_change":"2024-09-13T18:20:01.916089+0000","last_active":"2024-09-13T18:21:27.144146+0000","last_peered":"2024-09-13T18:21:27.144146+0000","last_clean":"2024-09-13T18:21:27.144146+0000","last_became_active":"2024-09-13T18:20:01.915783+0000","last_became_peered":"2024-09-13T18:20:01.915783+0000","last_unstale":"2024-09-13T18:21:27.144146+0000","last_undegraded":"2024-09-13T18:21:27.144146+0000","last_fullsized":"2024-09-13T18:21:27.144146+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:43:25.886029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"91'9","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.338482+0000","last_change":"2024-09-13T18:20:01.577381+0000","last_active":"2024-09-13T18:21:27.338482+0000","last_peered":"2024-09-13T18:21:27.338482+0000","last_clean":"2024-09-13T18:21:27.338482+0000","last_became_active":"2024-09-13T18:20:01.577044+0000","last_became_peered":"2024-09-13T18:20:01.577044+0000","last_unstale":"2024-09-13T18:21:27.338482+0000","last_undegraded":"2024-09-13T18:21:27.338482+0000","last_fullsized":"2024-09-13T18:21:27.338482+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:00.907948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.145091+0000","last_change":"2024-09-13T18:20:06.139971+0000","last_active":"2024-09-13T18:21:27.145091+0000","last_peered":"2024-09-13T18:21:27.145091+0000","last_clean":"2024-09-13T18:21:27.145091+0000","last_became_active":"2024-09-13T18:20:06.139507+0000","last_became_peered":"2024-09-13T18:20:06.139507+0000","last_unstale":"2024-09-13T18:21:27.145091+0000","last_undegraded":"2024-09-13T18:21:27.145091+0000","last_fullsized":"2024-09-13T18:21:27.145091+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:40:21.037768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.141865+0000","last_change":"2024-09-13T18:20:01.915348+0000","last_active":"2024-09-13T18:21:27.141865+0000","last_peered":"2024-09-13T18:21:27.141865+0000","last_clean":"2024-09-13T18:21:27.141865+0000","last_became_active":"2024-09-13T18:20:01.915042+0000","last_became_peered":"2024-09-13T18:20:01.915042+0000","last_unstale":"2024-09-13T18:21:27.141865+0000","last_undegraded":"2024-09-13T18:21:27.141865+0000","last_fullsized":"2024-09-13T18:21:27.141865+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:31:33.712314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"91'34","reported_seq":213,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:43.357276+0000","last_change":"2024-09-13T18:20:01.578456+0000","last_active":"2024-09-13T18:21:43.357276+0000","last_peered":"2024-09-13T18:21:43.357276+0000","last_clean":"2024-09-13T18:21:43.357276+0000","last_became_active":"2024-09-13T18:20:01.578051+0000","last_became_peered":"2024-09-13T18:20:01.578051+0000","last_unstale":"2024-09-13T18:21:43.357276+0000","last_undegraded":"2024-09-13T18:21:43.357276+0000","last_fullsized":"2024-09-13T18:21:43.357276+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:34:17.651620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63310,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":159,"num_read_kb":142,"num_write":36,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.139797+0000","last_change":"2024-09-13T18:20:06.131410+0000","last_active":"2024-09-13T18:21:27.139797+0000","last_peered":"2024-09-13T18:21:27.139797+0000","last_clean":"2024-09-13T18:21:27.139797+0000","last_became_active":"2024-09-13T18:20:06.131060+0000","last_became_peered":"2024-09-13T18:20:06.131060+0000","last_unstale":"2024-09-13T18:21:27.139797+0000","last_undegraded":"2024-09-13T18:21:27.139797+0000","last_fullsized":"2024-09-13T18:21:27.139797+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:08:25.266971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.322247+0000","last_change":"2024-09-13T18:20:01.928839+0000","last_active":"2024-09-13T18:21:27.322247+0000","last_peered":"2024-09-13T18:21:27.322247+0000","last_clean":"2024-09-13T18:21:27.322247+0000","last_became_active":"2024-09-13T18:20:01.925774+0000","last_became_peered":"2024-09-13T18:20:01.925774+0000","last_unstale":"2024-09-13T18:21:27.322247+0000","last_undegraded":"2024-09-13T18:21:27.322247+0000","last_fullsized":"2024-09-13T18:21:27.322247+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:15:23.049671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"91'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.145283+0000","last_change":"2024-09-13T18:20:01.576332+0000","last_active":"2024-09-13T18:21:27.145283+0000","last_peered":"2024-09-13T18:21:27.145283+0000","last_clean":"2024-09-13T18:21:27.145283+0000","last_became_active":"2024-09-13T18:20:01.575862+0000","last_became_peered":"2024-09-13T18:20:01.575862+0000","last_unstale":"2024-09-13T18:21:27.145283+0000","last_undegraded":"2024-09-13T18:21:27.145283+0000","last_fullsized":"2024-09-13T18:21:27.145283+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:17:00.452480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.143332+0000","last_change":"2024-09-13T18:20:06.133142+0000","last_active":"2024-09-13T18:21:27.143332+0000","last_peered":"2024-09-13T18:21:27.143332+0000","last_clean":"2024-09-13T18:21:27.143332+0000","last_became_active":"2024-09-13T18:20:06.132740+0000","last_became_peered":"2024-09-13T18:20:06.132740+0000","last_unstale":"2024-09-13T18:21:27.143332+0000","last_undegraded":"2024-09-13T18:21:27.143332+0000","last_fullsized":"2024-09-13T18:21:27.143332+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:12:46.442481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.147495+0000","last_change":"2024-09-13T18:20:01.919673+0000","last_active":"2024-09-13T18:21:27.147495+0000","last_peered":"2024-09-13T18:21:27.147495+0000","last_clean":"2024-09-13T18:21:27.147495+0000","last_became_active":"2024-09-13T18:20:01.919358+0000","last_became_peered":"2024-09-13T18:20:01.919358+0000","last_unstale":"2024-09-13T18:21:27.147495+0000","last_undegraded":"2024-09-13T18:21:27.147495+0000","last_fullsized":"2024-09-13T18:21:27.147495+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:40:11.407643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"91'4","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.146085+0000","last_change":"2024-09-13T18:20:27.190451+0000","last_active":"2024-09-13T18:21:27.146085+0000","last_peered":"2024-09-13T18:21:27.146085+0000","last_clean":"2024-09-13T18:21:27.146085+0000","last_became_active":"2024-09-13T18:20:27.187091+0000","last_became_peered":"2024-09-13T18:20:27.187091+0000","last_unstale":"2024-09-13T18:21:27.146085+0000","last_undegraded":"2024-09-13T18:21:27.146085+0000","last_fullsized":"2024-09-13T18:21:27.146085+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:57:07.976008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.139632+0000","last_change":"2024-09-13T18:20:06.133475+0000","last_active":"2024-09-13T18:21:27.139632+0000","last_peered":"2024-09-13T18:21:27.139632+0000","last_clean":"2024-09-13T18:21:27.139632+0000","last_became_active":"2024-09-13T18:20:06.133109+0000","last_became_peered":"2024-09-13T18:20:06.133109+0000","last_unstale":"2024-09-13T18:21:27.139632+0000","last_undegraded":"2024-09-13T18:21:27.139632+0000","last_fullsized":"2024-09-13T18:21:27.139632+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:47:19.567115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.322369+0000","last_change":"2024-09-13T18:20:01.922660+0000","last_active":"2024-09-13T18:21:27.322369+0000","last_peered":"2024-09-13T18:21:27.322369+0000","last_clean":"2024-09-13T18:21:27.322369+0000","last_became_active":"2024-09-13T18:20:01.922370+0000","last_became_peered":"2024-09-13T18:20:01.922370+0000","last_unstale":"2024-09-13T18:21:27.322369+0000","last_undegraded":"2024-09-13T18:21:27.322369+0000","last_fullsized":"2024-09-13T18:21:27.322369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:54:32.029704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"91'4","reported_seq":176,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:43.357288+0000","last_change":"2024-09-13T18:20:01.580975+0000","last_active":"2024-09-13T18:21:43.357288+0000","last_peered":"2024-09-13T18:21:43.357288+0000","last_clean":"2024-09-13T18:21:43.357288+0000","last_became_active":"2024-09-13T18:20:01.580647+0000","last_became_peered":"2024-09-13T18:20:01.580647+0000","last_unstale":"2024-09-13T18:21:43.357288+0000","last_undegraded":"2024-09-13T18:21:43.357288+0000","last_fullsized":"2024-09-13T18:21:43.357288+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:26:20.942915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":152,"num_read_kb":152,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.730531+0000","last_change":"2024-09-13T18:20:06.140003+0000","last_active":"2024-09-13T18:21:27.730531+0000","last_peered":"2024-09-13T18:21:27.730531+0000","last_clean":"2024-09-13T18:21:27.730531+0000","last_became_active":"2024-09-13T18:20:06.139663+0000","last_became_peered":"2024-09-13T18:20:06.139663+0000","last_unstale":"2024-09-13T18:21:27.730531+0000","last_undegraded":"2024-09-13T18:21:27.730531+0000","last_fullsized":"2024-09-13T18:21:27.730531+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:54:49.662401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.147273+0000","last_change":"2024-09-13T18:20:06.134448+0000","last_active":"2024-09-13T18:21:27.147273+0000","last_peered":"2024-09-13T18:21:27.147273+0000","last_clean":"2024-09-13T18:21:27.147273+0000","last_became_active":"2024-09-13T18:20:06.134084+0000","last_became_peered":"2024-09-13T18:20:06.134084+0000","last_unstale":"2024-09-13T18:21:27.147273+0000","last_undegraded":"2024-09-13T18:21:27.147273+0000","last_fullsized":"2024-09-13T18:21:27.147273+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:55:50.475665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.139809+0000","last_change":"2024-09-13T18:20:01.918955+0000","last_active":"2024-09-13T18:21:27.139809+0000","last_peered":"2024-09-13T18:21:27.139809+0000","last_clean":"2024-09-13T18:21:27.139809+0000","last_became_active":"2024-09-13T18:20:01.918666+0000","last_became_peered":"2024-09-13T18:20:01.918666+0000","last_unstale":"2024-09-13T18:21:27.139809+0000","last_undegraded":"2024-09-13T18:21:27.139809+0000","last_fullsized":"2024-09-13T18:21:27.139809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:08:50.697710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"91'3","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.144225+0000","last_change":"2024-09-13T18:20:01.578789+0000","last_active":"2024-09-13T18:21:27.144225+0000","last_peered":"2024-09-13T18:21:27.144225+0000","last_clean":"2024-09-13T18:21:27.144225+0000","last_became_active":"2024-09-13T18:20:01.578476+0000","last_became_peered":"2024-09-13T18:20:01.578476+0000","last_unstale":"2024-09-13T18:21:27.144225+0000","last_undegraded":"2024-09-13T18:21:27.144225+0000","last_fullsized":"2024-09-13T18:21:27.144225+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:40:58.355492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:28.423633+0000","last_change":"2024-09-13T18:21:28.423633+0000","last_active":"2024-09-13T18:21:28.423633+0000","last_peered":"2024-09-13T18:21:28.423633+0000","last_clean":"2024-09-13T18:21:28.423633+0000","last_became_active":"2024-09-13T18:21:28.423128+0000","last_became_peered":"2024-09-13T18:21:28.423128+0000","last_unstale":"2024-09-13T18:21:28.423633+0000","last_undegraded":"2024-09-13T18:21:28.423633+0000","last_fullsized":"2024-09-13T18:21:28.423633+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:06:32.687874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.730540+0000","last_change":"2024-09-13T18:20:01.914804+0000","last_active":"2024-09-13T18:21:27.730540+0000","last_peered":"2024-09-13T18:21:27.730540+0000","last_clean":"2024-09-13T18:21:27.730540+0000","last_became_active":"2024-09-13T18:20:01.914517+0000","last_became_peered":"2024-09-13T18:20:01.914517+0000","last_unstale":"2024-09-13T18:21:27.730540+0000","last_undegraded":"2024-09-13T18:21:27.730540+0000","last_fullsized":"2024-09-13T18:21:27.730540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:57:10.734935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"91'3","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.143676+0000","last_change":"2024-09-13T18:20:01.569201+0000","last_active":"2024-09-13T18:21:27.143676+0000","last_peered":"2024-09-13T18:21:27.143676+0000","last_clean":"2024-09-13T18:21:27.143676+0000","last_became_active":"2024-09-13T18:20:01.568908+0000","last_became_peered":"2024-09-13T18:20:01.568908+0000","last_unstale":"2024-09-13T18:21:27.143676+0000","last_undegraded":"2024-09-13T18:21:27.143676+0000","last_fullsized":"2024-09-13T18:21:27.143676+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:16:34.069130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"91'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.141443+0000","last_change":"2024-09-13T18:20:01.579225+0000","last_active":"2024-09-13T18:21:27.141443+0000","last_peered":"2024-09-13T18:21:27.141443+0000","last_clean":"2024-09-13T18:21:27.141443+0000","last_became_active":"2024-09-13T18:20:01.578826+0000","last_became_peered":"2024-09-13T18:20:01.578826+0000","last_unstale":"2024-09-13T18:21:27.141443+0000","last_undegraded":"2024-09-13T18:21:27.141443+0000","last_fullsized":"2024-09-13T18:21:27.141443+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:48:07.632239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.145492+0000","last_change":"2024-09-13T18:20:01.914885+0000","last_active":"2024-09-13T18:21:27.145492+0000","last_peered":"2024-09-13T18:21:27.145492+0000","last_clean":"2024-09-13T18:21:27.145492+0000","last_became_active":"2024-09-13T18:20:01.914599+0000","last_became_peered":"2024-09-13T18:20:01.914599+0000","last_unstale":"2024-09-13T18:21:27.145492+0000","last_undegraded":"2024-09-13T18:21:27.145492+0000","last_fullsized":"2024-09-13T18:21:27.145492+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:41:30.888675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.338340+0000","last_change":"2024-09-13T18:20:06.129376+0000","last_active":"2024-09-13T18:21:27.338340+0000","last_peered":"2024-09-13T18:21:27.338340+0000","last_clean":"2024-09-13T18:21:27.338340+0000","last_became_active":"2024-09-13T18:20:06.129032+0000","last_became_peered":"2024-09-13T18:20:06.129032+0000","last_unstale":"2024-09-13T18:21:27.338340+0000","last_undegraded":"2024-09-13T18:21:27.338340+0000","last_fullsized":"2024-09-13T18:21:27.338340+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:21:53.511813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":100,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:39.352857+0000","last_change":"2024-09-13T18:20:01.576069+0000","last_active":"2024-09-13T18:21:39.352857+0000","last_peered":"2024-09-13T18:21:39.352857+0000","last_clean":"2024-09-13T18:21:39.352857+0000","last_became_active":"2024-09-13T18:20:01.575760+0000","last_became_peered":"2024-09-13T18:20:01.575760+0000","last_unstale":"2024-09-13T18:21:39.352857+0000","last_undegraded":"2024-09-13T18:21:39.352857+0000","last_fullsized":"2024-09-13T18:21:39.352857+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:05:47.677848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":75,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.141389+0000","last_change":"2024-09-13T18:20:01.917073+0000","last_active":"2024-09-13T18:21:27.141389+0000","last_peered":"2024-09-13T18:21:27.141389+0000","last_clean":"2024-09-13T18:21:27.141389+0000","last_became_active":"2024-09-13T18:20:01.916765+0000","last_became_peered":"2024-09-13T18:20:01.916765+0000","last_unstale":"2024-09-13T18:21:27.141389+0000","last_undegraded":"2024-09-13T18:21:27.141389+0000","last_fullsized":"2024-09-13T18:21:27.141389+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:22:07.788313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.338478+0000","last_change":"2024-09-13T18:20:06.131429+0000","last_active":"2024-09-13T18:21:27.338478+0000","last_peered":"2024-09-13T18:21:27.338478+0000","last_clean":"2024-09-13T18:21:27.338478+0000","last_became_active":"2024-09-13T18:20:06.131019+0000","last_became_peered":"2024-09-13T18:20:06.131019+0000","last_unstale":"2024-09-13T18:21:27.338478+0000","last_undegraded":"2024-09-13T18:21:27.338478+0000","last_fullsized":"2024-09-13T18:21:27.338478+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:44:37.892574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.143073+0000","last_change":"2024-09-13T18:20:01.914050+0000","last_active":"2024-09-13T18:21:27.143073+0000","last_peered":"2024-09-13T18:21:27.143073+0000","last_clean":"2024-09-13T18:21:27.143073+0000","last_became_active":"2024-09-13T18:20:01.913733+0000","last_became_peered":"2024-09-13T18:20:01.913733+0000","last_unstale":"2024-09-13T18:21:27.143073+0000","last_undegraded":"2024-09-13T18:21:27.143073+0000","last_fullsized":"2024-09-13T18:21:27.143073+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:52:43.672096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.338412+0000","last_change":"2024-09-13T18:20:01.924290+0000","last_active":"2024-09-13T18:21:27.338412+0000","last_peered":"2024-09-13T18:21:27.338412+0000","last_clean":"2024-09-13T18:21:27.338412+0000","last_became_active":"2024-09-13T18:20:01.923883+0000","last_became_peered":"2024-09-13T18:20:01.923883+0000","last_unstale":"2024-09-13T18:21:27.338412+0000","last_undegraded":"2024-09-13T18:21:27.338412+0000","last_fullsized":"2024-09-13T18:21:27.338412+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:11:18.737501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.338245+0000","last_change":"2024-09-13T18:20:27.184256+0000","last_active":"2024-09-13T18:21:27.338245+0000","last_peered":"2024-09-13T18:21:27.338245+0000","last_clean":"2024-09-13T18:21:27.338245+0000","last_became_active":"2024-09-13T18:20:27.183790+0000","last_became_peered":"2024-09-13T18:20:27.183790+0000","last_unstale":"2024-09-13T18:21:27.338245+0000","last_undegraded":"2024-09-13T18:21:27.338245+0000","last_fullsized":"2024-09-13T18:21:27.338245+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:06:48.885753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.147139+0000","last_change":"2024-09-13T18:20:01.915799+0000","last_active":"2024-09-13T18:21:27.147139+0000","last_peered":"2024-09-13T18:21:27.147139+0000","last_clean":"2024-09-13T18:21:27.147139+0000","last_became_active":"2024-09-13T18:20:01.915519+0000","last_became_peered":"2024-09-13T18:20:01.915519+0000","last_unstale":"2024-09-13T18:21:27.147139+0000","last_undegraded":"2024-09-13T18:21:27.147139+0000","last_fullsized":"2024-09-13T18:21:27.147139+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:08:26.065052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.144088+0000","last_change":"2024-09-13T18:20:01.918547+0000","last_active":"2024-09-13T18:21:27.144088+0000","last_peered":"2024-09-13T18:21:27.144088+0000","last_clean":"2024-09-13T18:21:27.144088+0000","last_became_active":"2024-09-13T18:20:01.918053+0000","last_became_peered":"2024-09-13T18:20:01.918053+0000","last_unstale":"2024-09-13T18:21:27.144088+0000","last_undegraded":"2024-09-13T18:21:27.144088+0000","last_fullsized":"2024-09-13T18:21:27.144088+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:25:58.235101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.143607+0000","last_change":"2024-09-13T18:20:01.916205+0000","last_active":"2024-09-13T18:21:27.143607+0000","last_peered":"2024-09-13T18:21:27.143607+0000","last_clean":"2024-09-13T18:21:27.143607+0000","last_became_active":"2024-09-13T18:20:01.915909+0000","last_became_peered":"2024-09-13T18:20:01.915909+0000","last_unstale":"2024-09-13T18:21:27.143607+0000","last_undegraded":"2024-09-13T18:21:27.143607+0000","last_fullsized":"2024-09-13T18:21:27.143607+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:23:03.976513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.338788+0000","last_change":"2024-09-13T18:20:01.916621+0000","last_active":"2024-09-13T18:21:27.338788+0000","last_peered":"2024-09-13T18:21:27.338788+0000","last_clean":"2024-09-13T18:21:27.338788+0000","last_became_active":"2024-09-13T18:20:01.916335+0000","last_became_peered":"2024-09-13T18:20:01.916335+0000","last_unstale":"2024-09-13T18:21:27.338788+0000","last_undegraded":"2024-09-13T18:21:27.338788+0000","last_fullsized":"2024-09-13T18:21:27.338788+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:08:16.104571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.730943+0000","last_change":"2024-09-13T18:20:01.921195+0000","last_active":"2024-09-13T18:21:27.730943+0000","last_peered":"2024-09-13T18:21:27.730943+0000","last_clean":"2024-09-13T18:21:27.730943+0000","last_became_active":"2024-09-13T18:20:01.920895+0000","last_became_peered":"2024-09-13T18:20:01.920895+0000","last_unstale":"2024-09-13T18:21:27.730943+0000","last_undegraded":"2024-09-13T18:21:27.730943+0000","last_fullsized":"2024-09-13T18:21:27.730943+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:59:53.920047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.144052+0000","last_change":"2024-09-13T18:20:01.910889+0000","last_active":"2024-09-13T18:21:27.144052+0000","last_peered":"2024-09-13T18:21:27.144052+0000","last_clean":"2024-09-13T18:21:27.144052+0000","last_became_active":"2024-09-13T18:20:01.910451+0000","last_became_peered":"2024-09-13T18:20:01.910451+0000","last_unstale":"2024-09-13T18:21:27.144052+0000","last_undegraded":"2024-09-13T18:21:27.144052+0000","last_fullsized":"2024-09-13T18:21:27.144052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:06:52.552785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.730619+0000","last_change":"2024-09-13T18:20:01.914041+0000","last_active":"2024-09-13T18:21:27.730619+0000","last_peered":"2024-09-13T18:21:27.730619+0000","last_clean":"2024-09-13T18:21:27.730619+0000","last_became_active":"2024-09-13T18:20:01.913240+0000","last_became_peered":"2024-09-13T18:20:01.913240+0000","last_unstale":"2024-09-13T18:21:27.730619+0000","last_undegraded":"2024-09-13T18:21:27.730619+0000","last_fullsized":"2024-09-13T18:21:27.730619+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:39:37.062544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.322260+0000","last_change":"2024-09-13T18:20:01.927322+0000","last_active":"2024-09-13T18:21:27.322260+0000","last_peered":"2024-09-13T18:21:27.322260+0000","last_clean":"2024-09-13T18:21:27.322260+0000","last_became_active":"2024-09-13T18:20:01.926767+0000","last_became_peered":"2024-09-13T18:20:01.926767+0000","last_unstale":"2024-09-13T18:21:27.322260+0000","last_undegraded":"2024-09-13T18:21:27.322260+0000","last_fullsized":"2024-09-13T18:21:27.322260+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:20.986089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.144026+0000","last_change":"2024-09-13T18:20:01.916314+0000","last_active":"2024-09-13T18:21:27.144026+0000","last_peered":"2024-09-13T18:21:27.144026+0000","last_clean":"2024-09-13T18:21:27.144026+0000","last_became_active":"2024-09-13T18:20:01.915790+0000","last_became_peered":"2024-09-13T18:20:01.915790+0000","last_unstale":"2024-09-13T18:21:27.144026+0000","last_undegraded":"2024-09-13T18:21:27.144026+0000","last_fullsized":"2024-09-13T18:21:27.144026+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:20:30.063814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.143481+0000","last_change":"2024-09-13T18:20:27.187172+0000","last_active":"2024-09-13T18:21:27.143481+0000","last_peered":"2024-09-13T18:21:27.143481+0000","last_clean":"2024-09-13T18:21:27.143481+0000","last_became_active":"2024-09-13T18:20:27.186765+0000","last_became_peered":"2024-09-13T18:20:27.186765+0000","last_unstale":"2024-09-13T18:21:27.143481+0000","last_undegraded":"2024-09-13T18:21:27.143481+0000","last_fullsized":"2024-09-13T18:21:27.143481+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:37:01.469560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.338630+0000","last_change":"2024-09-13T18:20:01.924074+0000","last_active":"2024-09-13T18:21:27.338630+0000","last_peered":"2024-09-13T18:21:27.338630+0000","last_clean":"2024-09-13T18:21:27.338630+0000","last_became_active":"2024-09-13T18:20:01.922364+0000","last_became_peered":"2024-09-13T18:20:01.922364+0000","last_unstale":"2024-09-13T18:21:27.338630+0000","last_undegraded":"2024-09-13T18:21:27.338630+0000","last_fullsized":"2024-09-13T18:21:27.338630+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:51:33.824362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.140169+0000","last_change":"2024-09-13T18:20:01.914606+0000","last_active":"2024-09-13T18:21:27.140169+0000","last_peered":"2024-09-13T18:21:27.140169+0000","last_clean":"2024-09-13T18:21:27.140169+0000","last_became_active":"2024-09-13T18:20:01.914281+0000","last_became_peered":"2024-09-13T18:20:01.914281+0000","last_unstale":"2024-09-13T18:21:27.140169+0000","last_undegraded":"2024-09-13T18:21:27.140169+0000","last_fullsized":"2024-09-13T18:21:27.140169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:55:42.211959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.144515+0000","last_change":"2024-09-13T18:20:01.928876+0000","last_active":"2024-09-13T18:21:27.144515+0000","last_peered":"2024-09-13T18:21:27.144515+0000","last_clean":"2024-09-13T18:21:27.144515+0000","last_became_active":"2024-09-13T18:20:01.928534+0000","last_became_peered":"2024-09-13T18:20:01.928534+0000","last_unstale":"2024-09-13T18:21:27.144515+0000","last_undegraded":"2024-09-13T18:21:27.144515+0000","last_fullsized":"2024-09-13T18:21:27.144515+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:14:28.196746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.321912+0000","last_change":"2024-09-13T18:20:01.920642+0000","last_active":"2024-09-13T18:21:27.321912+0000","last_peered":"2024-09-13T18:21:27.321912+0000","last_clean":"2024-09-13T18:21:27.321912+0000","last_became_active":"2024-09-13T18:20:01.920335+0000","last_became_peered":"2024-09-13T18:20:01.920335+0000","last_unstale":"2024-09-13T18:21:27.321912+0000","last_undegraded":"2024-09-13T18:21:27.321912+0000","last_fullsized":"2024-09-13T18:21:27.321912+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:59:20.894563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.146999+0000","last_change":"2024-09-13T18:20:27.187529+0000","last_active":"2024-09-13T18:21:27.146999+0000","last_peered":"2024-09-13T18:21:27.146999+0000","last_clean":"2024-09-13T18:21:27.146999+0000","last_became_active":"2024-09-13T18:20:27.187123+0000","last_became_peered":"2024-09-13T18:20:27.187123+0000","last_unstale":"2024-09-13T18:21:27.146999+0000","last_undegraded":"2024-09-13T18:21:27.146999+0000","last_fullsized":"2024-09-13T18:21:27.146999+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:43:19.601235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.147000+0000","last_change":"2024-09-13T18:20:01.917111+0000","last_active":"2024-09-13T18:21:27.147000+0000","last_peered":"2024-09-13T18:21:27.147000+0000","last_clean":"2024-09-13T18:21:27.147000+0000","last_became_active":"2024-09-13T18:20:01.916826+0000","last_became_peered":"2024-09-13T18:20:01.916826+0000","last_unstale":"2024-09-13T18:21:27.147000+0000","last_undegraded":"2024-09-13T18:21:27.147000+0000","last_fullsized":"2024-09-13T18:21:27.147000+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:36:08.150028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.143951+0000","last_change":"2024-09-13T18:20:01.904496+0000","last_active":"2024-09-13T18:21:27.143951+0000","last_peered":"2024-09-13T18:21:27.143951+0000","last_clean":"2024-09-13T18:21:27.143951+0000","last_became_active":"2024-09-13T18:20:01.904199+0000","last_became_peered":"2024-09-13T18:20:01.904199+0000","last_unstale":"2024-09-13T18:21:27.143951+0000","last_undegraded":"2024-09-13T18:21:27.143951+0000","last_fullsized":"2024-09-13T18:21:27.143951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:44.044162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.144107+0000","last_change":"2024-09-13T18:20:01.919023+0000","last_active":"2024-09-13T18:21:27.144107+0000","last_peered":"2024-09-13T18:21:27.144107+0000","last_clean":"2024-09-13T18:21:27.144107+0000","last_became_active":"2024-09-13T18:20:01.918661+0000","last_became_peered":"2024-09-13T18:20:01.918661+0000","last_unstale":"2024-09-13T18:21:27.144107+0000","last_undegraded":"2024-09-13T18:21:27.144107+0000","last_fullsized":"2024-09-13T18:21:27.144107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:25:26.779029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.145834+0000","last_change":"2024-09-13T18:20:01.922833+0000","last_active":"2024-09-13T18:21:27.145834+0000","last_peered":"2024-09-13T18:21:27.145834+0000","last_clean":"2024-09-13T18:21:27.145834+0000","last_became_active":"2024-09-13T18:20:01.922533+0000","last_became_peered":"2024-09-13T18:20:01.922533+0000","last_unstale":"2024-09-13T18:21:27.145834+0000","last_undegraded":"2024-09-13T18:21:27.145834+0000","last_fullsized":"2024-09-13T18:21:27.145834+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:27:11.557419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.142558+0000","last_change":"2024-09-13T18:20:01.925916+0000","last_active":"2024-09-13T18:21:27.142558+0000","last_peered":"2024-09-13T18:21:27.142558+0000","last_clean":"2024-09-13T18:21:27.142558+0000","last_became_active":"2024-09-13T18:20:01.925568+0000","last_became_peered":"2024-09-13T18:20:01.925568+0000","last_unstale":"2024-09-13T18:21:27.142558+0000","last_undegraded":"2024-09-13T18:21:27.142558+0000","last_fullsized":"2024-09-13T18:21:27.142558+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:58:24.955273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.322384+0000","last_change":"2024-09-13T18:20:01.924328+0000","last_active":"2024-09-13T18:21:27.322384+0000","last_peered":"2024-09-13T18:21:27.322384+0000","last_clean":"2024-09-13T18:21:27.322384+0000","last_became_active":"2024-09-13T18:20:01.924004+0000","last_became_peered":"2024-09-13T18:20:01.924004+0000","last_unstale":"2024-09-13T18:21:27.322384+0000","last_undegraded":"2024-09-13T18:21:27.322384+0000","last_fullsized":"2024-09-13T18:21:27.322384+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:23:29.204157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.730492+0000","last_change":"2024-09-13T18:20:01.922234+0000","last_active":"2024-09-13T18:21:27.730492+0000","last_peered":"2024-09-13T18:21:27.730492+0000","last_clean":"2024-09-13T18:21:27.730492+0000","last_became_active":"2024-09-13T18:20:01.921196+0000","last_became_peered":"2024-09-13T18:20:01.921196+0000","last_unstale":"2024-09-13T18:21:27.730492+0000","last_undegraded":"2024-09-13T18:21:27.730492+0000","last_fullsized":"2024-09-13T18:21:27.730492+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:24:10.144346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.145467+0000","last_change":"2024-09-13T18:20:01.921740+0000","last_active":"2024-09-13T18:21:27.145467+0000","last_peered":"2024-09-13T18:21:27.145467+0000","last_clean":"2024-09-13T18:21:27.145467+0000","last_became_active":"2024-09-13T18:20:01.919634+0000","last_became_peered":"2024-09-13T18:20:01.919634+0000","last_unstale":"2024-09-13T18:21:27.145467+0000","last_undegraded":"2024-09-13T18:21:27.145467+0000","last_fullsized":"2024-09-13T18:21:27.145467+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:43:27.600637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.141401+0000","last_change":"2024-09-13T18:20:01.924806+0000","last_active":"2024-09-13T18:21:27.141401+0000","last_peered":"2024-09-13T18:21:27.141401+0000","last_clean":"2024-09-13T18:21:27.141401+0000","last_became_active":"2024-09-13T18:20:01.924514+0000","last_became_peered":"2024-09-13T18:20:01.924514+0000","last_unstale":"2024-09-13T18:21:27.141401+0000","last_undegraded":"2024-09-13T18:21:27.141401+0000","last_fullsized":"2024-09-13T18:21:27.141401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:47.323362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.141446+0000","last_change":"2024-09-13T18:20:01.912402+0000","last_active":"2024-09-13T18:21:27.141446+0000","last_peered":"2024-09-13T18:21:27.141446+0000","last_clean":"2024-09-13T18:21:27.141446+0000","last_became_active":"2024-09-13T18:20:01.911534+0000","last_became_peered":"2024-09-13T18:20:01.911534+0000","last_unstale":"2024-09-13T18:21:27.141446+0000","last_undegraded":"2024-09-13T18:21:27.141446+0000","last_fullsized":"2024-09-13T18:21:27.141446+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:22:52.021688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.143978+0000","last_change":"2024-09-13T18:20:01.930248+0000","last_active":"2024-09-13T18:21:27.143978+0000","last_peered":"2024-09-13T18:21:27.143978+0000","last_clean":"2024-09-13T18:21:27.143978+0000","last_became_active":"2024-09-13T18:20:01.929931+0000","last_became_peered":"2024-09-13T18:20:01.929931+0000","last_unstale":"2024-09-13T18:21:27.143978+0000","last_undegraded":"2024-09-13T18:21:27.143978+0000","last_fullsized":"2024-09-13T18:21:27.143978+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:37:33.086722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.147445+0000","last_change":"2024-09-13T18:20:01.921913+0000","last_active":"2024-09-13T18:21:27.147445+0000","last_peered":"2024-09-13T18:21:27.147445+0000","last_clean":"2024-09-13T18:21:27.147445+0000","last_became_active":"2024-09-13T18:20:01.921395+0000","last_became_peered":"2024-09-13T18:20:01.921395+0000","last_unstale":"2024-09-13T18:21:27.147445+0000","last_undegraded":"2024-09-13T18:21:27.147445+0000","last_fullsized":"2024-09-13T18:21:27.147445+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:54:50.816870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.147312+0000","last_change":"2024-09-13T18:20:01.912654+0000","last_active":"2024-09-13T18:21:27.147312+0000","last_peered":"2024-09-13T18:21:27.147312+0000","last_clean":"2024-09-13T18:21:27.147312+0000","last_became_active":"2024-09-13T18:20:01.912356+0000","last_became_peered":"2024-09-13T18:20:01.912356+0000","last_unstale":"2024-09-13T18:21:27.147312+0000","last_undegraded":"2024-09-13T18:21:27.147312+0000","last_fullsized":"2024-09-13T18:21:27.147312+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:25:18.099396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.144431+0000","last_change":"2024-09-13T18:20:01.923373+0000","last_active":"2024-09-13T18:21:27.144431+0000","last_peered":"2024-09-13T18:21:27.144431+0000","last_clean":"2024-09-13T18:21:27.144431+0000","last_became_active":"2024-09-13T18:20:01.923077+0000","last_became_peered":"2024-09-13T18:20:01.923077+0000","last_unstale":"2024-09-13T18:21:27.144431+0000","last_undegraded":"2024-09-13T18:21:27.144431+0000","last_fullsized":"2024-09-13T18:21:27.144431+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:07:52.955705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69540,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":790,"num_read_kb":476,"num_write":233,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":208620,"data_compressed":4824,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281684,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383714,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":1032192,"data_stored":519328,"data_compressed":23512,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:21:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.495,"15min":0.48199999999999998},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.47599999999999998,"15min":0.46500000000000002},"min":{"1min":0.318,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.70099999999999996}]},{"osd":1,"last update":"Fri Sep 13 18:21:14 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.57699999999999996,"15min":0.56899999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.86499999999999999,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56699999999999995,"15min":0.56499999999999995},"min":{"1min":0.35599999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.75,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.74399999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:21:14 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.34499999999999997,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.63700000000000001,"5min":1.238,"15min":1.238},"last":0.628},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.44,"15min":0.432},"min":{"1min":0.33300000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.71499999999999997,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.43099999999999999}]},{"osd":3,"last update":"Fri Sep 13 18:21:27 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.81599999999999995}]},{"osd":4,"last update":"Fri Sep 13 18:21:14 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.61299999999999999,"15min":0.60399999999999998},"min":{"1min":0.43099999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.79200000000000004,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56799999999999995,"15min":0.56200000000000006},"min":{"1min":0.44500000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.85299999999999998,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.59599999999999997}]},{"osd":5,"last update":"Fri Sep 13 18:21:14 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.504,"15min":0.501},"min":{"1min":0.36299999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.75700000000000001,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.52400000000000002,"15min":0.51400000000000001},"min":{"1min":0.42299999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.55500000000000005}]},{"osd":6,"last update":"Fri Sep 13 18:21:14 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.57499999999999996,"15min":0.56799999999999995},"min":{"1min":0.41899999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78900000000000003,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.59899999999999998,"15min":0.59299999999999997},"min":{"1min":0.499,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.81499999999999995,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.72799999999999998}]},{"osd":7,"last update":"Fri Sep 13 18:21:14 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.65300000000000002,"15min":0.63800000000000001},"min":{"1min":0.54400000000000004,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.92800000000000005,"5min":1.123,"15min":1.123},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.65000000000000002,"15min":0.64500000000000002},"min":{"1min":0.46800000000000003,"5min":0.439,"15min":0.439},"max":{"1min":0.90200000000000002,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.79200000000000004}]},{"osd":8,"last update":"Fri Sep 13 18:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.88700000000000001}]},{"osd":9,"last update":"Fri Sep 13 18:21:14 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.69099999999999995,"15min":0.68500000000000005},"min":{"1min":0.57799999999999996,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.88,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.69399999999999995,"15min":0.68000000000000005},"min":{"1min":0.54700000000000004,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.93400000000000005,"5min":1.208,"15min":1.208},"last":0.90600000000000003}]},{"osd":10,"last update":"Fri Sep 13 18:21:14 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.68100000000000005,"15min":0.67300000000000004},"min":{"1min":0.55400000000000005,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.94399999999999995,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.70799999999999996,"15min":0.69999999999999996},"min":{"1min":0.622,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.95699999999999996,"5min":1.583,"15min":1.583},"last":0.84399999999999997}]}]},{"osd":10,"up_from":72,"seq":309237645351,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28172,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716980,"statfs":{"total":95995035648,"available":95966187520,"internally_reserved":0,"allocated":1175552,"data_stored":1173197,"data_compressed":60944,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:21:01 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.40300000000000002},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":14.589,"5min":14.589,"15min":14.589},"last":0.35799999999999998}]},{"osd":1,"last update":"Fri Sep 13 18:20:53 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.42399999999999999,"15min":0.42299999999999999},"min":{"1min":0.28699999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.318},{"interface":"front","average":{"1min":0.432,"5min":0.41899999999999998,"15min":0.41699999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.372}]},{"osd":2,"last update":"Fri Sep 13 18:20:53 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.54600000000000004,"15min":0.54300000000000004},"min":{"1min":0.39800000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.78400000000000003,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.55900000000000005,"15min":0.55300000000000005},"min":{"1min":0.436,"5min":0.374,"15min":0.374},"max":{"1min":0.77200000000000002,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.50600000000000001}]},{"osd":3,"last update":"Fri Sep 13 18:20:53 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57199999999999995,"15min":0.57599999999999996},"min":{"1min":0.378,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72799999999999998,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.44400000000000001},{"interface":"front","average":{"1min":1.131,"5min":0.68700000000000006,"15min":0.61299999999999999},"min":{"1min":0.40999999999999998,"5min":0.376,"15min":0.376},"max":{"1min":14.505000000000001,"5min":14.505000000000001,"15min":14.505000000000001},"last":0.55000000000000004}]},{"osd":4,"last update":"Fri Sep 13 18:20:53 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.50800000000000001,"15min":0.51000000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.73499999999999999,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.5,"15min":0.5},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.68300000000000005,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.47199999999999998}]},{"osd":5,"last update":"Fri Sep 13 18:20:53 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.60299999999999998,"15min":0.59999999999999998},"min":{"1min":0.432,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.79700000000000004,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.627,"15min":0.626},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.88200000000000001,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.58399999999999996}]},{"osd":6,"last update":"Fri Sep 13 18:20:53 2024","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":0.75700000000000001,"15min":0.67900000000000005},"min":{"1min":0.41599999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":14.539999999999999,"5min":14.539999999999999,"15min":14.539999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.64600000000000002,"15min":0.64800000000000002},"min":{"1min":0.46200000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.60099999999999998}]},{"osd":7,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.81599999999999995}]},{"osd":8,"last update":"Fri Sep 13 18:20:53 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.67100000000000004,"15min":0.67300000000000004},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.80200000000000005,"5min":1.01,"15min":1.01},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.69299999999999995,"15min":0.69499999999999995},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.86499999999999999,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.61799999999999999}]},{"osd":9,"last update":"Fri Sep 13 18:20:53 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.68799999999999994,"15min":0.69199999999999995},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.83199999999999996,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.71499999999999997,"15min":0.71799999999999997},"min":{"1min":0.53200000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89300000000000002,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.64900000000000002}]},{"osd":11,"last update":"Fri Sep 13 18:21:13 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.72699999999999998,"15min":0.73099999999999998},"min":{"1min":0.58899999999999997,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.89200000000000002,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.73699999999999999,"15min":0.73999999999999999},"min":{"1min":0.58999999999999997,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.872,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.67700000000000005}]}]},{"osd":9,"up_from":67,"seq":287762808876,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":519284,"data_compressed":23512,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:21:33 2024","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.36099999999999999,"15min":0.38100000000000001},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.45800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.41399999999999998,"15min":0.442},"min":{"1min":0.249,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.495,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.376}]},{"osd":1,"last update":"Fri Sep 13 18:21:33 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.56299999999999994,"15min":0.58399999999999996},"min":{"1min":0.35699999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.67300000000000004,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.496,"5min":0.56399999999999995,"15min":0.58499999999999996},"min":{"1min":0.36399999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.67700000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.44800000000000001}]},{"osd":2,"last update":"Fri Sep 13 18:21:33 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.55000000000000004,"15min":0.57399999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.65000000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.52600000000000002,"15min":0.52800000000000002},"min":{"1min":0.34899999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.54100000000000004}]},{"osd":3,"last update":"Fri Sep 13 18:21:33 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.44700000000000001,"15min":0.48099999999999998},"min":{"1min":0.28499999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.50900000000000001,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.37,"5min":0.437,"15min":0.46300000000000002},"min":{"1min":0.28000000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.496,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.46100000000000002}]},{"osd":4,"last update":"Fri Sep 13 18:21:33 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.55600000000000005,"15min":0.57299999999999995},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.67700000000000005,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.503},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.61299999999999999,"15min":0.64700000000000002},"min":{"1min":0.39000000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.65300000000000002,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.55600000000000005}]},{"osd":5,"last update":"Fri Sep 13 18:21:33 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.57199999999999995,"15min":0.59099999999999997},"min":{"1min":0.36599999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.59299999999999997,"15min":0.61099999999999999},"min":{"1min":0.40600000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.752,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.60499999999999998}]},{"osd":6,"last update":"Fri Sep 13 18:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.39800000000000002}]},{"osd":7,"last update":"Fri Sep 13 18:21:33 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.63600000000000001,"15min":0.67400000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.70899999999999996,"5min":0.871,"15min":0.871},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.64700000000000002,"15min":0.68100000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.63900000000000001,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.57199999999999995}]},{"osd":8,"last update":"Fri Sep 13 18:21:33 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.63500000000000001,"15min":0.66500000000000004},"min":{"1min":0.40999999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.75,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.66500000000000004,"15min":0.69999999999999996},"min":{"1min":0.436,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.81899999999999995,"5min":1.02,"15min":1.02},"last":0.67800000000000005}]},{"osd":10,"last update":"Fri Sep 13 18:20:51 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.65300000000000002,"15min":0.66600000000000004},"min":{"1min":0.47899999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.70299999999999996,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.67700000000000005,"15min":0.68899999999999995},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.75800000000000001,"5min":0.879,"15min":0.879},"last":0.64400000000000002}]},{"osd":11,"last update":"Fri Sep 13 18:21:08 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.63300000000000001}]}]},{"osd":8,"up_from":59,"seq":253403070512,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":1044480,"data_stored":519640,"data_compressed":23512,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:21:07 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.52800000000000002,"15min":0.54500000000000004},"min":{"1min":0.38500000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.76000000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.56200000000000006,"15min":0.56599999999999995},"min":{"1min":0.33400000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":3.2989999999999999,"5min":3.2989999999999999,"15min":3.2989999999999999},"last":0.58199999999999996}]},{"osd":1,"last update":"Fri Sep 13 18:21:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56599999999999995,"15min":0.56100000000000005},"min":{"1min":0.41899999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.747},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57499999999999996,"15min":0.57299999999999995},"min":{"1min":0.44400000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.63}]},{"osd":2,"last update":"Fri Sep 13 18:21:07 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.42899999999999999,"15min":0.43099999999999999},"min":{"1min":0.29599999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.438,"15min":0.437},"min":{"1min":0.27600000000000002,"5min":0.224,"15min":0.224},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.56299999999999994}]},{"osd":3,"last update":"Fri Sep 13 18:21:07 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54400000000000004,"15min":0.55700000000000005},"min":{"1min":0.377,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53800000000000003,"15min":0.53900000000000003},"min":{"1min":0.437,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.67800000000000005}]},{"osd":4,"last update":"Fri Sep 13 18:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.66400000000000003,"15min":0.63600000000000001},"min":{"1min":0.45100000000000001,"5min":0.37,"15min":0.37},"max":{"1min":5.3540000000000001,"5min":5.3540000000000001,"15min":5.3540000000000001},"last":0.68000000000000005}]},{"osd":5,"last update":"Fri Sep 13 18:21:07 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.52600000000000002,"15min":0.53600000000000003},"min":{"1min":0.32600000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.442},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.52400000000000002,"15min":0.52500000000000002},"min":{"1min":0.40100000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.498}]},{"osd":6,"last update":"Fri Sep 13 18:21:07 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.629,"15min":0.62},"min":{"1min":0.436,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":3.2719999999999998,"5min":3.2719999999999998,"15min":3.2719999999999998},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.63400000000000001,"15min":0.64600000000000002},"min":{"1min":0.42899999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.85199999999999998,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.66600000000000004}]},{"osd":7,"last update":"Fri Sep 13 18:21:07 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.66000000000000003,"15min":0.67300000000000004},"min":{"1min":0.48099999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89300000000000002,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.65800000000000003,"15min":0.66400000000000003},"min":{"1min":0.504,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.76200000000000001}]},{"osd":9,"last update":"Fri Sep 13 18:21:25 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63800000000000001,"15min":0.63300000000000001},"min":{"1min":0.48299999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.79500000000000004,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.64600000000000002,"15min":0.65200000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.78300000000000003,"5min":1.012,"15min":1.012},"last":0.82199999999999995}]},{"osd":10,"last update":"Fri Sep 13 18:20:51 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.91600000000000004,"15min":0.95399999999999996},"min":{"1min":0.53800000000000003,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.92600000000000005,"5min":5.3360000000000003,"15min":5.3360000000000003},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.68000000000000005,"15min":0.67500000000000004},"min":{"1min":0.55300000000000005,"5min":0.498,"15min":0.498},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.79900000000000004}]},{"osd":11,"last update":"Fri Sep 13 18:21:02 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.70899999999999996}]}]},{"osd":7,"up_from":51,"seq":219043332148,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1101824,"data_stored":1115624,"data_compressed":59336,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.94399999999999995,"15min":1.012},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.84299999999999997,"5min":12.068,"15min":12.068},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.46999999999999997,"15min":0.45900000000000002},"min":{"1min":0.35199999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.61599999999999999}]},{"osd":1,"last update":"Fri Sep 13 18:20:45 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.376,"15min":0.35999999999999999},"min":{"1min":0.29099999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.51900000000000002,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.435,"5min":0.39400000000000002,"15min":0.379},"min":{"1min":0.27500000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.56899999999999995,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.375}]},{"osd":2,"last update":"Fri Sep 13 18:20:45 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60899999999999999,"15min":0.54300000000000004},"min":{"1min":0.46700000000000003,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.88700000000000001,"5min":7.6820000000000004,"15min":7.6820000000000004},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.52800000000000002,"15min":0.51100000000000001},"min":{"1min":0.47999999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.65800000000000003,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.48599999999999999}]},{"osd":3,"last update":"Fri Sep 13 18:20:45 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.51700000000000002,"15min":0.48899999999999999},"min":{"1min":0.42199999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80500000000000005,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.51200000000000001,"15min":0.48999999999999999},"min":{"1min":0.376,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.66400000000000003,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.46899999999999997}]},{"osd":4,"last update":"Fri Sep 13 18:20:45 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.45800000000000002,"15min":0.439},"min":{"1min":0.35399999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.505,"5min":0.46600000000000003,"15min":0.45400000000000001},"min":{"1min":0.33300000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.50900000000000001}]},{"osd":5,"last update":"Fri Sep 13 18:20:45 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.57699999999999996,"15min":0.55700000000000005},"min":{"1min":0.51700000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.55400000000000005,"15min":0.53600000000000003},"min":{"1min":0.46899999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73499999999999999,"5min":0.754,"15min":0.754},"last":0.61399999999999999}]},{"osd":6,"last update":"Fri Sep 13 18:20:45 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.56799999999999995,"15min":0.54600000000000004},"min":{"1min":0.434,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.60899999999999999,"15min":0.58699999999999997},"min":{"1min":0.432,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.81999999999999995,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.60399999999999998}]},{"osd":8,"last update":"Fri Sep 13 18:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.60299999999999998,"15min":0.59099999999999997},"min":{"1min":0.498,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.73999999999999999,"15min":0.66800000000000004},"min":{"1min":0.61699999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.93300000000000005,"5min":7.7149999999999999,"15min":7.7149999999999999},"last":0.65500000000000003}]},{"osd":9,"last update":"Fri Sep 13 18:21:29 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.66300000000000003,"15min":0.66100000000000003},"min":{"1min":0.40500000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.88,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.625},{"interface":"front","average":{"1min":1.609,"5min":0.82999999999999996,"15min":0.70199999999999996},"min":{"1min":0.42699999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":20.056999999999999,"5min":20.056999999999999,"15min":20.056999999999999},"last":0.57399999999999995}]},{"osd":10,"last update":"Fri Sep 13 18:20:51 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.66300000000000003,"15min":0.65700000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.85699999999999998,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.65000000000000002,"15min":0.64600000000000002},"min":{"1min":0.53100000000000003,"5min":0.434,"15min":0.434},"max":{"1min":0.81699999999999995,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.64300000000000002}]},{"osd":11,"last update":"Fri Sep 13 18:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.69199999999999995}]}]},{"osd":6,"up_from":46,"seq":197568495674,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":1028096,"data_stored":514740,"data_compressed":23512,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:21:21 2024","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.35999999999999999,"15min":0.36399999999999999},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.48599999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.374,"5min":0.41699999999999998,"15min":0.433},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.60299999999999998,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.43099999999999999}]},{"osd":1,"last update":"Fri Sep 13 18:21:21 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.53900000000000003,"15min":0.52600000000000002},"min":{"1min":0.40999999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.72199999999999998,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.55200000000000005,"15min":0.53700000000000003},"min":{"1min":0.40600000000000003,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.68600000000000005,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.53200000000000003}]},{"osd":2,"last update":"Fri Sep 13 18:21:21 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.37,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.69899999999999995,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.54000000000000004,"15min":0.54800000000000004},"min":{"1min":0.40600000000000003,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.71299999999999997,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.56000000000000005}]},{"osd":3,"last update":"Fri Sep 13 18:21:21 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.41099999999999998,"15min":0.42199999999999999},"min":{"1min":0.29099999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.51200000000000001,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.45700000000000002,"15min":0.46800000000000003},"min":{"1min":0.314,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.65600000000000003,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.48199999999999998}]},{"osd":4,"last update":"Fri Sep 13 18:21:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.52900000000000003,"15min":0.51700000000000002},"min":{"1min":0.42699999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.68000000000000005,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55800000000000005,"15min":0.55400000000000005},"min":{"1min":0.48299999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.68300000000000005,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.67200000000000004}]},{"osd":5,"last update":"Fri Sep 13 18:21:21 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59499999999999997,"15min":0.60699999999999998},"min":{"1min":0.47099999999999997,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.752,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56799999999999995,"15min":0.59099999999999997},"min":{"1min":0.41999999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.73999999999999999,"5min":0.874,"15min":0.874},"last":0.64100000000000001}]},{"osd":7,"last update":"Fri Sep 13 18:20:44 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.39100000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.76400000000000001,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.503},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56599999999999995,"15min":0.56200000000000006},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.58499999999999996}]},{"osd":8,"last update":"Fri Sep 13 18:21:08 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.621,"15min":0.622},"min":{"1min":0.51300000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.59599999999999997,"15min":0.59999999999999998},"min":{"1min":0.47799999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.62}]},{"osd":9,"last update":"Fri Sep 13 18:21:34 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.51500000000000001,"15min":0.53100000000000003},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.57199999999999995,"5min":0.871,"15min":0.871},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.59099999999999997,"15min":0.60499999999999998},"min":{"1min":0.41399999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.75700000000000001,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.60699999999999998}]},{"osd":10,"last update":"Fri Sep 13 18:20:54 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.61499999999999999,"15min":0.61799999999999999},"min":{"1min":0.51100000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.56200000000000006,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.68700000000000006}]},{"osd":11,"last update":"Fri Sep 13 18:21:08 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.71799999999999997}]}]},{"osd":5,"up_from":38,"seq":163208757310,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1110016,"data_stored":1111302,"data_compressed":59336,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:20:59 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.499,"15min":0.496},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.69999999999999996,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.51800000000000002,"15min":0.51300000000000001},"min":{"1min":0.34100000000000003,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.77400000000000002,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.48999999999999999}]},{"osd":1,"last update":"Fri Sep 13 18:20:59 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.53100000000000003,"15min":0.50700000000000001},"min":{"1min":0.378,"5min":0.308,"15min":0.308},"max":{"1min":0.81200000000000006,"5min":0.878,"15min":0.878},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.52500000000000002,"15min":0.51100000000000001},"min":{"1min":0.379,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.71599999999999997,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.59999999999999998}]},{"osd":2,"last update":"Fri Sep 13 18:20:59 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.38500000000000001,"15min":0.36499999999999999},"min":{"1min":0.311,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.39100000000000001,"15min":0.38500000000000001},"min":{"1min":0.30599999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.52200000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.40200000000000002}]},{"osd":3,"last update":"Fri Sep 13 18:20:59 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.52500000000000002,"15min":0.504},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.73999999999999999,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.40500000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.70299999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.51200000000000001}]},{"osd":4,"last update":"Fri Sep 13 18:20:59 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55300000000000005,"15min":0.54000000000000004},"min":{"1min":0.432,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.76600000000000001,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.77800000000000002,"15min":0.61299999999999999},"min":{"1min":0.44,"5min":0.374,"15min":0.374},"max":{"1min":1.002,"5min":18.571000000000002,"15min":18.571000000000002},"last":0.65900000000000003}]},{"osd":6,"last update":"Fri Sep 13 18:21:24 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.40600000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55600000000000005,"15min":0.55100000000000005},"min":{"1min":0.378,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.78000000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.60799999999999998}]},{"osd":7,"last update":"Fri Sep 13 18:20:43 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.61299999999999999,"15min":0.60699999999999998},"min":{"1min":0.38900000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.626,"15min":0.61399999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.872,"5min":0.88,"15min":0.88},"last":0.67200000000000004}]},{"osd":8,"last update":"Fri Sep 13 18:21:02 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.51300000000000001,"15min":0.51100000000000001},"min":{"1min":0.372,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.69999999999999996,"5min":0.753,"15min":0.753},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.54800000000000004,"15min":0.53800000000000003},"min":{"1min":0.39100000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.69099999999999995,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.629}]},{"osd":9,"last update":"Fri Sep 13 18:21:30 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.64500000000000002,"15min":0.627},"min":{"1min":0.48199999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.63900000000000001,"15min":0.628},"min":{"1min":0.48799999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.81299999999999994,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.71499999999999997}]},{"osd":10,"last update":"Fri Sep 13 18:20:50 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.58999999999999997,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.84099999999999997,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.753},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.67500000000000004,"15min":0.66900000000000004},"min":{"1min":0.55700000000000005,"5min":0.37,"15min":0.37},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.74199999999999999}]}]},{"osd":4,"up_from":33,"seq":141733920834,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":518814,"data_compressed":23512,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:19:33 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.48299999999999998,"15min":0.47699999999999998},"min":{"1min":0.35399999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.69399999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.433},{"interface":"front","average":{"1min":0.501,"5min":0.51200000000000001,"15min":0.47499999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.84599999999999997,"5min":4.2539999999999996,"15min":4.2539999999999996},"last":0.51900000000000002}]},{"osd":1,"last update":"Fri Sep 13 18:21:38 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.373,"15min":0.35199999999999998},"min":{"1min":0.25800000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.59199999999999997,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.379,"15min":0.35399999999999998},"min":{"1min":0.29199999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.68400000000000005,"5min":0.748,"15min":0.748},"last":0.311}]},{"osd":2,"last update":"Fri Sep 13 18:21:38 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.60699999999999998,"15min":0.54900000000000004},"min":{"1min":0.40999999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":5.2930000000000001,"5min":5.2930000000000001,"15min":5.2930000000000001},"last":0.629},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55300000000000005,"15min":0.53800000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72499999999999998,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.58099999999999996}]},{"osd":3,"last update":"Fri Sep 13 18:21:38 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56000000000000005,"15min":0.505},"min":{"1min":0.434,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.82199999999999995,"5min":4.2679999999999998,"15min":4.2679999999999998},"last":0.434},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.52100000000000002,"15min":0.496},"min":{"1min":0.46100000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.59099999999999997}]},{"osd":5,"last update":"Fri Sep 13 18:20:55 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55700000000000005,"15min":0.53800000000000003},"min":{"1min":0.39800000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.73799999999999999,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.56699999999999995,"15min":0.54300000000000004},"min":{"1min":0.41999999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.80700000000000005,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.61599999999999999}]},{"osd":6,"last update":"Fri Sep 13 18:21:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.55000000000000004,"15min":0.52000000000000002},"min":{"1min":0.441,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.54900000000000004,"15min":0.51700000000000002},"min":{"1min":0.42999999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.79200000000000004,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.47799999999999998}]},{"osd":7,"last update":"Fri Sep 13 18:20:45 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.45300000000000001,"15min":0.441},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.68999999999999995,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.497,"15min":0.48099999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.752,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.48699999999999999}]},{"osd":8,"last update":"Fri Sep 13 18:21:07 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.63500000000000001,"15min":0.627},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.82199999999999995,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.66000000000000003,"15min":0.64800000000000002},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.83099999999999996,"5min":0.874,"15min":0.874},"last":0.64600000000000002}]},{"osd":9,"last update":"Fri Sep 13 18:21:28 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.63300000000000001,"15min":0.61499999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.63700000000000001,"15min":0.629},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.83299999999999996,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.65600000000000003}]},{"osd":10,"last update":"Fri Sep 13 18:20:52 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.629,"15min":0.63500000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.82599999999999996,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.66200000000000003,"15min":0.65900000000000003},"min":{"1min":0.51600000000000001,"5min":0.44,"15min":0.44},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.56299999999999994}]},{"osd":11,"last update":"Fri Sep 13 18:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.67600000000000005}]}]},{"osd":3,"up_from":25,"seq":107374182471,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":514215,"data_compressed":23512,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:21:14 2024","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33600000000000002,"15min":0.309},"min":{"1min":0.24199999999999999,"5min":0.183,"15min":0.183},"max":{"1min":0.42699999999999999,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.224},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.34300000000000003,"15min":0.32900000000000001},"min":{"1min":0.251,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"last":0.24199999999999999}]},{"osd":1,"last update":"Fri Sep 13 18:21:14 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.51900000000000002,"15min":0.49099999999999999},"min":{"1min":0.36699999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.71399999999999997,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.53200000000000003,"15min":0.49399999999999999},"min":{"1min":0.40500000000000003,"5min":0.313,"15min":0.313},"max":{"1min":0.70299999999999996,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.61699999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:21:14 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.51900000000000002,"15min":0.52100000000000002},"min":{"1min":0.33000000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52700000000000002,"15min":0.502},"min":{"1min":0.34300000000000003,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.69299999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.59699999999999998}]},{"osd":4,"last update":"Fri Sep 13 18:21:38 2024","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":0.622,"15min":0.52600000000000002},"min":{"1min":0.40500000000000003,"5min":0.312,"15min":0.312},"max":{"1min":9.3149999999999995,"5min":9.3149999999999995,"15min":9.3149999999999995},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.53900000000000003,"15min":0.51900000000000002},"min":{"1min":0.39900000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.72399999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.71299999999999997}]},{"osd":5,"last update":"Fri Sep 13 18:20:55 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54300000000000004,"15min":0.53600000000000003},"min":{"1min":0.42599999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.751,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54900000000000004,"15min":0.55100000000000005},"min":{"1min":0.39600000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77100000000000002,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.54700000000000004}]},{"osd":6,"last update":"Fri Sep 13 18:21:24 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.436,"15min":0.44400000000000001},"min":{"1min":0.29099999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.61599999999999999,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.502,"5min":0.48999999999999999,"15min":0.46999999999999997},"min":{"1min":0.372,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.74199999999999999,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.35299999999999998}]},{"osd":7,"last update":"Fri Sep 13 18:20:44 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.60599999999999998,"15min":0.59799999999999998},"min":{"1min":0.505,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.80900000000000005,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57999999999999996,"15min":0.57399999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.79600000000000004,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.57299999999999995}]},{"osd":8,"last update":"Fri Sep 13 18:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.69599999999999995}]},{"osd":9,"last update":"Fri Sep 13 18:21:31 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.52500000000000002,"15min":0.54900000000000004},"min":{"1min":0.33800000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.59999999999999998,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.57099999999999995,"15min":0.58499999999999996},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.73899999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.379}]},{"osd":10,"last update":"Fri Sep 13 18:20:49 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.68100000000000005,"15min":0.68300000000000005},"min":{"1min":0.53600000000000003,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.78800000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.66100000000000003,"15min":0.66900000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.77100000000000002,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.49399999999999999}]},{"osd":11,"last update":"Fri Sep 13 18:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.65100000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411404,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1105920,"data_stored":579140,"data_compressed":25120,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:20:49 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.47599999999999998,"15min":0.47999999999999998},"min":{"1min":0.34399999999999997,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.75900000000000001,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.496,"5min":0.48499999999999999,"15min":0.47699999999999998},"min":{"1min":0.40699999999999997,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.69499999999999995,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.627}]},{"osd":1,"last update":"Fri Sep 13 18:20:49 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.51600000000000001,"15min":0.47999999999999998},"min":{"1min":0.38100000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.53100000000000003,"15min":0.47899999999999998},"min":{"1min":0.45200000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.79000000000000004}]},{"osd":3,"last update":"Fri Sep 13 18:21:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.53900000000000003,"15min":0.52000000000000002},"min":{"1min":0.42299999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.74399999999999999,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.57099999999999995,"15min":0.55700000000000005},"min":{"1min":0.442,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.82299999999999995,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.70899999999999996}]},{"osd":4,"last update":"Fri Sep 13 18:21:02 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.753},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.72199999999999998}]},{"osd":5,"last update":"Fri Sep 13 18:20:52 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.40799999999999997,"15min":0.38300000000000001},"min":{"1min":0.34200000000000003,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.42499999999999999,"15min":0.39800000000000002},"min":{"1min":0.31900000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.49399999999999999}]},{"osd":6,"last update":"Fri Sep 13 18:21:22 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.59199999999999997,"15min":0.55000000000000004},"min":{"1min":0.48099999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78700000000000003,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.60599999999999998,"15min":0.57199999999999995},"min":{"1min":0.57599999999999996,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.88900000000000001,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.74099999999999999}]},{"osd":7,"last update":"Fri Sep 13 18:21:07 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.86099999999999999}]},{"osd":8,"last update":"Fri Sep 13 18:21:07 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.55100000000000005,"15min":0.54600000000000004},"min":{"1min":0.39500000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.59599999999999997,"15min":0.58799999999999997},"min":{"1min":0.38200000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.877,"5min":0.998,"15min":0.998},"last":0.83999999999999997}]},{"osd":9,"last update":"Fri Sep 13 18:21:28 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.63700000000000001,"15min":0.625},"min":{"1min":0.50900000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.86399999999999999,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.66900000000000004,"15min":0.65500000000000003},"min":{"1min":0.50900000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.96499999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.82599999999999996}]},{"osd":10,"last update":"Fri Sep 13 18:20:49 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.64300000000000002,"15min":0.626},"min":{"1min":0.58299999999999996,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.65600000000000003,"15min":0.64200000000000002},"min":{"1min":0.57399999999999995,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.88}]},{"osd":11,"last update":"Fri Sep 13 18:21:00 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.92000000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1101824,"data_stored":579192,"data_compressed":25120,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:21:25 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.496,"15min":0.49299999999999999},"min":{"1min":0.30299999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.56399999999999995,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.437},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.52900000000000003,"15min":0.51700000000000002},"min":{"1min":0.33900000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.56799999999999995}]},{"osd":2,"last update":"Fri Sep 13 18:20:51 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.53000000000000003,"15min":0.5},"min":{"1min":0.35799999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.48899999999999999,"15min":0.47499999999999998},"min":{"1min":0.40500000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.71099999999999997}]},{"osd":3,"last update":"Fri Sep 13 18:21:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.54400000000000004,"15min":0.52200000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.69799999999999995,"5min":0.749,"15min":0.749},"last":0.496},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.57999999999999996,"15min":0.54900000000000004},"min":{"1min":0.503,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.83399999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.68500000000000005}]},{"osd":4,"last update":"Fri Sep 13 18:21:34 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.41099999999999998,"15min":0.371},"min":{"1min":0.29499999999999998,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.755,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.40500000000000003,"15min":0.36499999999999999},"min":{"1min":0.29599999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.65600000000000003,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.51500000000000001}]},{"osd":5,"last update":"Fri Sep 13 18:20:56 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.59799999999999998,"15min":0.57199999999999995},"min":{"1min":0.45900000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.58399999999999996,"15min":0.54800000000000004},"min":{"1min":0.40699999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.73499999999999999}]},{"osd":6,"last update":"Fri Sep 13 18:21:25 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58199999999999996,"15min":0.56899999999999995},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78900000000000003,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.63100000000000001,"15min":0.61399999999999999},"min":{"1min":0.41599999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.81200000000000006}]},{"osd":7,"last update":"Fri Sep 13 18:21:40 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.51700000000000002,"15min":0.495},"min":{"1min":0.312,"5min":0.246,"15min":0.246},"max":{"1min":0.74299999999999999,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.52200000000000002,"15min":0.50700000000000001},"min":{"1min":0.35499999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.82499999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.61799999999999999}]},{"osd":8,"last update":"Fri Sep 13 18:21:08 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.63,"15min":0.61799999999999999},"min":{"1min":0.46600000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.94199999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.747},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.64300000000000002,"15min":0.63400000000000001},"min":{"1min":0.56000000000000005,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.78100000000000003}]},{"osd":9,"last update":"Fri Sep 13 18:21:34 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.63600000000000001,"15min":0.64200000000000002},"min":{"1min":0.49399999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.84399999999999997,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.64000000000000001,"15min":0.64200000000000002},"min":{"1min":0.433,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.55200000000000005}]},{"osd":10,"last update":"Fri Sep 13 18:20:51 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.60999999999999999,"15min":0.59599999999999997},"min":{"1min":0.432,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.90800000000000003,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.625,"15min":0.61299999999999999},"min":{"1min":0.498,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.80300000000000005}]},{"osd":11,"last update":"Fri Sep 13 18:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.83999999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":1052672,"data_stored":516196,"data_compressed":23512,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 18:21:28 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.50700000000000001,"15min":0.498},"min":{"1min":0.36799999999999999,"5min":0.318,"15min":0.30199999999999999},"max":{"1min":0.76900000000000002,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.52100000000000002,"15min":0.50900000000000001},"min":{"1min":0.434,"5min":0.307,"15min":0.307},"max":{"1min":0.73999999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.58999999999999997}]},{"osd":2,"last update":"Fri Sep 13 18:20:47 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53000000000000003,"15min":0.52800000000000002},"min":{"1min":0.35699999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.54200000000000004,"15min":0.51500000000000001},"min":{"1min":0.36899999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.77300000000000002,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.56699999999999995}]},{"osd":3,"last update":"Fri Sep 13 18:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.57899999999999996}]},{"osd":4,"last update":"Fri Sep 13 18:21:39 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52100000000000002,"15min":0.497},"min":{"1min":0.41299999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.54100000000000004,"15min":0.51500000000000001},"min":{"1min":0.36399999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76000000000000001,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.64400000000000002}]},{"osd":5,"last update":"Fri Sep 13 18:20:56 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.61699999999999999,"15min":0.56699999999999995},"min":{"1min":0.38700000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.78800000000000003,"5min":7.194,"15min":7.194},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56699999999999995,"15min":0.52900000000000003},"min":{"1min":0.42399999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.80700000000000005,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.61299999999999999}]},{"osd":6,"last update":"Fri Sep 13 18:21:23 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.46500000000000002,"15min":0.45500000000000002},"min":{"1min":0.32600000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.56000000000000005,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.44500000000000001,"15min":0.45700000000000002},"min":{"1min":0.27800000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.64700000000000002,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.36499999999999999}]},{"osd":7,"last update":"Fri Sep 13 18:21:07 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.44900000000000001}]},{"osd":8,"last update":"Fri Sep 13 18:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.59699999999999998,"15min":0.60599999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79100000000000004,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.627,"5min":0.621,"15min":0.61699999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.77900000000000003,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.499}]},{"osd":9,"last update":"Fri Sep 13 18:21:33 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.58499999999999996,"15min":0.60699999999999998},"min":{"1min":0.33700000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78200000000000003,"5min":1.089,"15min":1.089},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.54100000000000004,"15min":0.57299999999999995},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.61799999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.43099999999999999}]},{"osd":10,"last update":"Fri Sep 13 18:20:49 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.67000000000000004,"15min":0.68000000000000005},"min":{"1min":0.51000000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.82899999999999996,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.69199999999999995,"15min":0.70099999999999996},"min":{"1min":0.48299999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.84899999999999998,"5min":1.048,"15min":1.048},"last":0.66400000000000003}]},{"osd":11,"last update":"Fri Sep 13 18:21:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.629,"15min":0.63300000000000001},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.76200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.68000000000000005,"15min":0.68300000000000005},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.88700000000000001,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.623}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34188,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34188,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34188,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35824,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35824,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35824,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35824,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35824,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":65126,"data_compressed":1608,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":65074,"data_compressed":1608,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":64998,"data_compressed":1608,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:21:46.771 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-13T18:21:46.971 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:21:47.091 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:46 smithi114 ceph-mon[30897]: pgmap v333: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-13T18:21:47.092 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:46 smithi114 ceph-mon[30897]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T18:21:47.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:46 smithi134 ceph-mon[34283]: pgmap v333: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-13T18:21:47.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:46 smithi134 ceph-mon[34283]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T18:21:47.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:46 smithi204 ceph-mon[34777]: pgmap v333: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-13T18:21:47.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:46 smithi204 ceph-mon[34777]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T18:21:48.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.652+0000 7fa40b397640 1 -- 172.21.15.114:0/717579886 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa40410a440 msgr2=0x7fa40410c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:48.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.652+0000 7fa40b397640 1 --2- 172.21.15.114:0/717579886 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa40410a440 0x7fa40410c830 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc009ca0 tx=0x7fa3fc02a8c0 comp rx=0 tx=0).stop 2024-09-13T18:21:48.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.652+0000 7fa40b397640 1 -- 172.21.15.114:0/717579886 shutdown_connections 2024-09-13T18:21:48.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.652+0000 7fa40b397640 1 --2- 172.21.15.114:0/717579886 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa40410a440 0x7fa40410c830 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:48.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.652+0000 7fa40b397640 1 --2- 172.21.15.114:0/717579886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4041018e0 0x7fa404109dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:48.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.652+0000 7fa40b397640 1 --2- 172.21.15.114:0/717579886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa404100fc0 0x7fa4041013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:48.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.652+0000 7fa40b397640 1 -- 172.21.15.114:0/717579886 >> 172.21.15.114:0/717579886 conn(0x7fa4040fa9a0 msgr2=0x7fa4040fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:48.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.652+0000 7fa40b397640 1 -- 172.21.15.114:0/717579886 shutdown_connections 2024-09-13T18:21:48.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.652+0000 7fa40b397640 1 -- 172.21.15.114:0/717579886 wait complete. 2024-09-13T18:21:48.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.653+0000 7fa40b397640 1 Processor -- start 2024-09-13T18:21:48.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.653+0000 7fa40b397640 1 -- start start 2024-09-13T18:21:48.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.653+0000 7fa40b397640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa404100fc0 0x7fa404196170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:48.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.653+0000 7fa40b397640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4041018e0 0x7fa4041966b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:48.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.653+0000 7fa40b397640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa40410a440 0x7fa40419aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:48.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.653+0000 7fa40b397640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa40419b0b0 con 0x7fa404100fc0 2024-09-13T18:21:48.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.653+0000 7fa40b397640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa40419b220 con 0x7fa40410a440 2024-09-13T18:21:48.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.653+0000 7fa40b397640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa40419b390 con 0x7fa4041018e0 2024-09-13T18:21:48.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.653+0000 7fa409133640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa404100fc0 0x7fa404196170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:48.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.653+0000 7fa409133640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa404100fc0 0x7fa404196170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47288/0 (socket says 172.21.15.114:47288) 2024-09-13T18:21:48.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.653+0000 7fa409133640 1 -- 172.21.15.114:0/2493781104 learned_addr learned my addr 172.21.15.114:0/2493781104 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:48.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.653+0000 7fa408932640 1 --2- 172.21.15.114:0/2493781104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4041018e0 0x7fa4041966b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:48.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.653+0000 7fa409934640 1 --2- 172.21.15.114:0/2493781104 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa40410a440 0x7fa40419aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:48.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.654+0000 7fa409133640 1 -- 172.21.15.114:0/2493781104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4041018e0 msgr2=0x7fa4041966b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:48.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.654+0000 7fa409133640 1 --2- 172.21.15.114:0/2493781104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4041018e0 0x7fa4041966b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:48.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.654+0000 7fa409133640 1 -- 172.21.15.114:0/2493781104 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa40410a440 msgr2=0x7fa40419aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:48.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.654+0000 7fa409133640 1 --2- 172.21.15.114:0/2493781104 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa40410a440 0x7fa40419aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:48.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.654+0000 7fa409133640 1 -- 172.21.15.114:0/2493781104 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3fc009900 con 0x7fa404100fc0 2024-09-13T18:21:48.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.654+0000 7fa408932640 1 --2- 172.21.15.114:0/2493781104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4041018e0 0x7fa4041966b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:21:48.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.654+0000 7fa409133640 1 --2- 172.21.15.114:0/2493781104 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa404100fc0 0x7fa404196170 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa3f800d770 tx=0x7fa3f800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:48.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.654+0000 7fa3f67fc640 1 -- 172.21.15.114:0/2493781104 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f8009c30 con 0x7fa404100fc0 2024-09-13T18:21:48.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.654+0000 7fa3f67fc640 1 -- 172.21.15.114:0/2493781104 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa3f800de80 con 0x7fa404100fc0 2024-09-13T18:21:48.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.654+0000 7fa40b397640 1 -- 172.21.15.114:0/2493781104 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa40419b670 con 0x7fa404100fc0 2024-09-13T18:21:48.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.654+0000 7fa3f67fc640 1 -- 172.21.15.114:0/2493781104 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f8004d40 con 0x7fa404100fc0 2024-09-13T18:21:48.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.654+0000 7fa40b397640 1 -- 172.21.15.114:0/2493781104 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4041a2f30 con 0x7fa404100fc0 2024-09-13T18:21:48.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.655+0000 7fa40b397640 1 -- 172.21.15.114:0/2493781104 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa404068fa0 con 0x7fa404100fc0 2024-09-13T18:21:48.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.656+0000 7fa3f67fc640 1 -- 172.21.15.114:0/2493781104 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fa3f8004ea0 con 0x7fa404100fc0 2024-09-13T18:21:48.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.657+0000 7fa3f67fc640 1 --2- 172.21.15.114:0/2493781104 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fa3f00a1110 0x7fa3f00a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:48.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.657+0000 7fa408932640 1 --2- 172.21.15.114:0/2493781104 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fa3f00a1110 0x7fa3f00a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:48.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.657+0000 7fa3f67fc640 1 -- 172.21.15.114:0/2493781104 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fa3f80ccd40 con 0x7fa404100fc0 2024-09-13T18:21:48.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.657+0000 7fa408932640 1 --2- 172.21.15.114:0/2493781104 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fa3f00a1110 0x7fa3f00a35d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa404197720 tx=0x7fa400005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:48.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.660+0000 7fa3f67fc640 1 -- 172.21.15.114:0/2493781104 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fa3f8096000 con 0x7fa404100fc0 2024-09-13T18:21:48.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.818+0000 7fa40b397640 1 -- 172.21.15.114:0/2493781104 --> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa4041974d0 con 0x7fa3f00a1110 2024-09-13T18:21:48.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.829+0000 7fa3f67fc640 1 -- 172.21.15.114:0/2493781104 <== mgr.14148 v2:172.21.15.114:6800/3396185541 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+428876 (secure 0 0 0) 0x7fa4041974d0 con 0x7fa3f00a1110 2024-09-13T18:21:48.837 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:21:48.839 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-13T18:21:48.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.833+0000 7fa40b397640 1 -- 172.21.15.114:0/2493781104 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fa3f00a1110 msgr2=0x7fa3f00a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:48.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.833+0000 7fa40b397640 1 --2- 172.21.15.114:0/2493781104 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fa3f00a1110 0x7fa3f00a35d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa404197720 tx=0x7fa400005f90 comp rx=0 tx=0).stop 2024-09-13T18:21:48.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.833+0000 7fa40b397640 1 -- 172.21.15.114:0/2493781104 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa404100fc0 msgr2=0x7fa404196170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:48.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.833+0000 7fa40b397640 1 --2- 172.21.15.114:0/2493781104 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa404100fc0 0x7fa404196170 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa3f800d770 tx=0x7fa3f800dc30 comp rx=0 tx=0).stop 2024-09-13T18:21:48.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.833+0000 7fa40b397640 1 -- 172.21.15.114:0/2493781104 shutdown_connections 2024-09-13T18:21:48.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.833+0000 7fa40b397640 1 --2- 172.21.15.114:0/2493781104 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fa3f00a1110 0x7fa3f00a35d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:48.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.833+0000 7fa40b397640 1 --2- 172.21.15.114:0/2493781104 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa40410a440 0x7fa40419aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:48.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.833+0000 7fa40b397640 1 --2- 172.21.15.114:0/2493781104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4041018e0 0x7fa4041966b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:48.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.833+0000 7fa40b397640 1 --2- 172.21.15.114:0/2493781104 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa404100fc0 0x7fa404196170 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:48.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.833+0000 7fa40b397640 1 -- 172.21.15.114:0/2493781104 >> 172.21.15.114:0/2493781104 conn(0x7fa4040fa9a0 msgr2=0x7fa4040fc300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:48.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.833+0000 7fa40b397640 1 -- 172.21.15.114:0/2493781104 shutdown_connections 2024-09-13T18:21:48.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:48.833+0000 7fa40b397640 1 -- 172.21.15.114:0/2493781104 wait complete. 2024-09-13T18:21:49.155 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:48 smithi114 ceph-mon[30897]: pgmap v334: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-13T18:21:49.368 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:48 smithi134 ceph-mon[34283]: pgmap v334: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-13T18:21:49.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:48 smithi204 ceph-mon[34777]: pgmap v334: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-13T18:21:49.617 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":334,"stamp":"2024-09-13T18:21:47.930598+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660142,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":941,"num_read_kb":595,"num_write":416,"num_write_kb":3076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180875,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":344,"ondisk_log_size":344,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361416,"kb_used_data":12576,"kb_used_omap":18,"kb_used_meta":348653,"kb_avail":1124580408,"statfs":{"total":1151940427776,"available":1151570337792,"internally_reserved":0,"allocated":12877824,"data_stored":8180672,"data_compressed":394440,"data_compressed_allocated":540672,"data_compressed_original":2580480,"omap_allocated":19129,"internal_metadata":357020999},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003245"},"pg_stats":[{"pgid":"2.1f","version":"91'13","reported_seq":110,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:43.354444+0000","last_change":"2024-09-13T18:20:01.577029+0000","last_active":"2024-09-13T18:21:43.354444+0000","last_peered":"2024-09-13T18:21:43.354444+0000","last_clean":"2024-09-13T18:21:43.354444+0000","last_became_active":"2024-09-13T18:20:01.576710+0000","last_became_peered":"2024-09-13T18:20:01.576710+0000","last_unstale":"2024-09-13T18:21:43.354444+0000","last_undegraded":"2024-09-13T18:21:43.354444+0000","last_fullsized":"2024-09-13T18:21:43.354444+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:25:11.977436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":77,"num_read_kb":77,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.730811+0000","last_change":"2024-09-13T18:20:01.922955+0000","last_active":"2024-09-13T18:21:27.730811+0000","last_peered":"2024-09-13T18:21:27.730811+0000","last_clean":"2024-09-13T18:21:27.730811+0000","last_became_active":"2024-09-13T18:20:01.922704+0000","last_became_peered":"2024-09-13T18:20:01.922704+0000","last_unstale":"2024-09-13T18:21:27.730811+0000","last_undegraded":"2024-09-13T18:21:27.730811+0000","last_fullsized":"2024-09-13T18:21:27.730811+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:44:20.672784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.338127+0000","last_change":"2024-09-13T18:20:27.300985+0000","last_active":"2024-09-13T18:21:27.338127+0000","last_peered":"2024-09-13T18:21:27.338127+0000","last_clean":"2024-09-13T18:21:27.338127+0000","last_became_active":"2024-09-13T18:20:27.300546+0000","last_became_peered":"2024-09-13T18:20:27.300546+0000","last_unstale":"2024-09-13T18:21:27.338127+0000","last_undegraded":"2024-09-13T18:21:27.338127+0000","last_fullsized":"2024-09-13T18:21:27.338127+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:00:11.750929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.730958+0000","last_change":"2024-09-13T18:20:01.578921+0000","last_active":"2024-09-13T18:21:27.730958+0000","last_peered":"2024-09-13T18:21:27.730958+0000","last_clean":"2024-09-13T18:21:27.730958+0000","last_became_active":"2024-09-13T18:20:01.578627+0000","last_became_peered":"2024-09-13T18:20:01.578627+0000","last_unstale":"2024-09-13T18:21:27.730958+0000","last_undegraded":"2024-09-13T18:21:27.730958+0000","last_fullsized":"2024-09-13T18:21:27.730958+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:43:32.919156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.147483+0000","last_change":"2024-09-13T18:20:01.922901+0000","last_active":"2024-09-13T18:21:27.147483+0000","last_peered":"2024-09-13T18:21:27.147483+0000","last_clean":"2024-09-13T18:21:27.147483+0000","last_became_active":"2024-09-13T18:20:01.922604+0000","last_became_peered":"2024-09-13T18:20:01.922604+0000","last_unstale":"2024-09-13T18:21:27.147483+0000","last_undegraded":"2024-09-13T18:21:27.147483+0000","last_fullsized":"2024-09-13T18:21:27.147483+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:27:30.650010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.142773+0000","last_change":"2024-09-13T18:20:06.135406+0000","last_active":"2024-09-13T18:21:27.142773+0000","last_peered":"2024-09-13T18:21:27.142773+0000","last_clean":"2024-09-13T18:21:27.142773+0000","last_became_active":"2024-09-13T18:20:06.134558+0000","last_became_peered":"2024-09-13T18:20:06.134558+0000","last_unstale":"2024-09-13T18:21:27.142773+0000","last_undegraded":"2024-09-13T18:21:27.142773+0000","last_fullsized":"2024-09-13T18:21:27.142773+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:10.901966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'12","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.141998+0000","last_change":"2024-09-13T18:20:01.580040+0000","last_active":"2024-09-13T18:21:27.141998+0000","last_peered":"2024-09-13T18:21:27.141998+0000","last_clean":"2024-09-13T18:21:27.141998+0000","last_became_active":"2024-09-13T18:20:01.579715+0000","last_became_peered":"2024-09-13T18:20:01.579715+0000","last_unstale":"2024-09-13T18:21:27.141998+0000","last_undegraded":"2024-09-13T18:21:27.141998+0000","last_fullsized":"2024-09-13T18:21:27.141998+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:57:03.588169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.322207+0000","last_change":"2024-09-13T18:20:01.928314+0000","last_active":"2024-09-13T18:21:27.322207+0000","last_peered":"2024-09-13T18:21:27.322207+0000","last_clean":"2024-09-13T18:21:27.322207+0000","last_became_active":"2024-09-13T18:20:01.925356+0000","last_became_peered":"2024-09-13T18:20:01.925356+0000","last_unstale":"2024-09-13T18:21:27.322207+0000","last_undegraded":"2024-09-13T18:21:27.322207+0000","last_fullsized":"2024-09-13T18:21:27.322207+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:43.906842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.143221+0000","last_change":"2024-09-13T18:20:06.135886+0000","last_active":"2024-09-13T18:21:27.143221+0000","last_peered":"2024-09-13T18:21:27.143221+0000","last_clean":"2024-09-13T18:21:27.143221+0000","last_became_active":"2024-09-13T18:20:06.134587+0000","last_became_peered":"2024-09-13T18:20:06.134587+0000","last_unstale":"2024-09-13T18:21:27.143221+0000","last_undegraded":"2024-09-13T18:21:27.143221+0000","last_fullsized":"2024-09-13T18:21:27.143221+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:44:42.434729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"91'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.141890+0000","last_change":"2024-09-13T18:20:01.576908+0000","last_active":"2024-09-13T18:21:27.141890+0000","last_peered":"2024-09-13T18:21:27.141890+0000","last_clean":"2024-09-13T18:21:27.141890+0000","last_became_active":"2024-09-13T18:20:01.576478+0000","last_became_peered":"2024-09-13T18:20:01.576478+0000","last_unstale":"2024-09-13T18:21:27.141890+0000","last_undegraded":"2024-09-13T18:21:27.141890+0000","last_fullsized":"2024-09-13T18:21:27.141890+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:34:52.126901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.145190+0000","last_change":"2024-09-13T18:20:01.921960+0000","last_active":"2024-09-13T18:21:27.145190+0000","last_peered":"2024-09-13T18:21:27.145190+0000","last_clean":"2024-09-13T18:21:27.145190+0000","last_became_active":"2024-09-13T18:20:01.921652+0000","last_became_peered":"2024-09-13T18:20:01.921652+0000","last_unstale":"2024-09-13T18:21:27.145190+0000","last_undegraded":"2024-09-13T18:21:27.145190+0000","last_fullsized":"2024-09-13T18:21:27.145190+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:00:31.277613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.143043+0000","last_change":"2024-09-13T18:20:06.138983+0000","last_active":"2024-09-13T18:21:27.143043+0000","last_peered":"2024-09-13T18:21:27.143043+0000","last_clean":"2024-09-13T18:21:27.143043+0000","last_became_active":"2024-09-13T18:20:06.138708+0000","last_became_peered":"2024-09-13T18:20:06.138708+0000","last_unstale":"2024-09-13T18:21:27.143043+0000","last_undegraded":"2024-09-13T18:21:27.143043+0000","last_fullsized":"2024-09-13T18:21:27.143043+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:45:21.978217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.145723+0000","last_change":"2024-09-13T18:20:01.929817+0000","last_active":"2024-09-13T18:21:27.145723+0000","last_peered":"2024-09-13T18:21:27.145723+0000","last_clean":"2024-09-13T18:21:27.145723+0000","last_became_active":"2024-09-13T18:20:01.928907+0000","last_became_peered":"2024-09-13T18:20:01.928907+0000","last_unstale":"2024-09-13T18:21:27.145723+0000","last_undegraded":"2024-09-13T18:21:27.145723+0000","last_fullsized":"2024-09-13T18:21:27.145723+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:56:25.847230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.142094+0000","last_change":"2024-09-13T18:20:01.578968+0000","last_active":"2024-09-13T18:21:27.142094+0000","last_peered":"2024-09-13T18:21:27.142094+0000","last_clean":"2024-09-13T18:21:27.142094+0000","last_became_active":"2024-09-13T18:20:01.578623+0000","last_became_peered":"2024-09-13T18:20:01.578623+0000","last_unstale":"2024-09-13T18:21:27.142094+0000","last_undegraded":"2024-09-13T18:21:27.142094+0000","last_fullsized":"2024-09-13T18:21:27.142094+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:03:30.333107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.142804+0000","last_change":"2024-09-13T18:20:06.135977+0000","last_active":"2024-09-13T18:21:27.142804+0000","last_peered":"2024-09-13T18:21:27.142804+0000","last_clean":"2024-09-13T18:21:27.142804+0000","last_became_active":"2024-09-13T18:20:06.134844+0000","last_became_peered":"2024-09-13T18:20:06.134844+0000","last_unstale":"2024-09-13T18:21:27.142804+0000","last_undegraded":"2024-09-13T18:21:27.142804+0000","last_fullsized":"2024-09-13T18:21:27.142804+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:58:36.139240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.730582+0000","last_change":"2024-09-13T18:20:01.916675+0000","last_active":"2024-09-13T18:21:27.730582+0000","last_peered":"2024-09-13T18:21:27.730582+0000","last_clean":"2024-09-13T18:21:27.730582+0000","last_became_active":"2024-09-13T18:20:01.912902+0000","last_became_peered":"2024-09-13T18:20:01.912902+0000","last_unstale":"2024-09-13T18:21:27.730582+0000","last_undegraded":"2024-09-13T18:21:27.730582+0000","last_fullsized":"2024-09-13T18:21:27.730582+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:24:54.271569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"91'3","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.143363+0000","last_change":"2024-09-13T18:20:27.300845+0000","last_active":"2024-09-13T18:21:27.143363+0000","last_peered":"2024-09-13T18:21:27.143363+0000","last_clean":"2024-09-13T18:21:27.143363+0000","last_became_active":"2024-09-13T18:20:27.300417+0000","last_became_peered":"2024-09-13T18:20:27.300417+0000","last_unstale":"2024-09-13T18:21:27.143363+0000","last_undegraded":"2024-09-13T18:21:27.143363+0000","last_fullsized":"2024-09-13T18:21:27.143363+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:44:27.507140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:28.235707+0000","last_change":"2024-09-13T18:21:28.235707+0000","last_active":"2024-09-13T18:21:28.235707+0000","last_peered":"2024-09-13T18:21:28.235707+0000","last_clean":"2024-09-13T18:21:28.235707+0000","last_became_active":"2024-09-13T18:21:28.231794+0000","last_became_peered":"2024-09-13T18:21:28.231794+0000","last_unstale":"2024-09-13T18:21:28.235707+0000","last_undegraded":"2024-09-13T18:21:28.235707+0000","last_fullsized":"2024-09-13T18:21:28.235707+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:01:00.113821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.143150+0000","last_change":"2024-09-13T18:20:01.913634+0000","last_active":"2024-09-13T18:21:27.143150+0000","last_peered":"2024-09-13T18:21:27.143150+0000","last_clean":"2024-09-13T18:21:27.143150+0000","last_became_active":"2024-09-13T18:20:01.913128+0000","last_became_peered":"2024-09-13T18:20:01.913128+0000","last_unstale":"2024-09-13T18:21:27.143150+0000","last_undegraded":"2024-09-13T18:21:27.143150+0000","last_fullsized":"2024-09-13T18:21:27.143150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:37:15.251756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"91'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.730712+0000","last_change":"2024-09-13T18:20:01.579385+0000","last_active":"2024-09-13T18:21:27.730712+0000","last_peered":"2024-09-13T18:21:27.730712+0000","last_clean":"2024-09-13T18:21:27.730712+0000","last_became_active":"2024-09-13T18:20:01.579102+0000","last_became_peered":"2024-09-13T18:20:01.579102+0000","last_unstale":"2024-09-13T18:21:27.730712+0000","last_undegraded":"2024-09-13T18:21:27.730712+0000","last_fullsized":"2024-09-13T18:21:27.730712+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:22:00.432052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.140056+0000","last_change":"2024-09-13T18:20:06.138110+0000","last_active":"2024-09-13T18:21:27.140056+0000","last_peered":"2024-09-13T18:21:27.140056+0000","last_clean":"2024-09-13T18:21:27.140056+0000","last_became_active":"2024-09-13T18:20:06.137763+0000","last_became_peered":"2024-09-13T18:20:06.137763+0000","last_unstale":"2024-09-13T18:21:27.140056+0000","last_undegraded":"2024-09-13T18:21:27.140056+0000","last_fullsized":"2024-09-13T18:21:27.140056+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:16:15.956216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.141263+0000","last_change":"2024-09-13T18:20:01.918318+0000","last_active":"2024-09-13T18:21:27.141263+0000","last_peered":"2024-09-13T18:21:27.141263+0000","last_clean":"2024-09-13T18:21:27.141263+0000","last_became_active":"2024-09-13T18:20:01.917977+0000","last_became_peered":"2024-09-13T18:20:01.917977+0000","last_unstale":"2024-09-13T18:21:27.141263+0000","last_undegraded":"2024-09-13T18:21:27.141263+0000","last_fullsized":"2024-09-13T18:21:27.141263+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:23.368866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.321998+0000","last_change":"2024-09-13T18:20:01.572009+0000","last_active":"2024-09-13T18:21:27.321998+0000","last_peered":"2024-09-13T18:21:27.321998+0000","last_clean":"2024-09-13T18:21:27.321998+0000","last_became_active":"2024-09-13T18:20:01.571313+0000","last_became_peered":"2024-09-13T18:20:01.571313+0000","last_unstale":"2024-09-13T18:21:27.321998+0000","last_undegraded":"2024-09-13T18:21:27.321998+0000","last_fullsized":"2024-09-13T18:21:27.321998+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:32:59.340476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.143064+0000","last_change":"2024-09-13T18:20:06.132212+0000","last_active":"2024-09-13T18:21:27.143064+0000","last_peered":"2024-09-13T18:21:27.143064+0000","last_clean":"2024-09-13T18:21:27.143064+0000","last_became_active":"2024-09-13T18:20:06.131809+0000","last_became_peered":"2024-09-13T18:20:06.131809+0000","last_unstale":"2024-09-13T18:21:27.143064+0000","last_undegraded":"2024-09-13T18:21:27.143064+0000","last_fullsized":"2024-09-13T18:21:27.143064+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:18:11.667551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.147352+0000","last_change":"2024-09-13T18:20:27.301112+0000","last_active":"2024-09-13T18:21:27.147352+0000","last_peered":"2024-09-13T18:21:27.147352+0000","last_clean":"2024-09-13T18:21:27.147352+0000","last_became_active":"2024-09-13T18:20:27.300772+0000","last_became_peered":"2024-09-13T18:20:27.300772+0000","last_unstale":"2024-09-13T18:21:27.147352+0000","last_undegraded":"2024-09-13T18:21:27.147352+0000","last_fullsized":"2024-09-13T18:21:27.147352+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:59:57.606105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"91'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.143891+0000","last_change":"2024-09-13T18:20:01.572025+0000","last_active":"2024-09-13T18:21:27.143891+0000","last_peered":"2024-09-13T18:21:27.143891+0000","last_clean":"2024-09-13T18:21:27.143891+0000","last_became_active":"2024-09-13T18:20:01.571321+0000","last_became_peered":"2024-09-13T18:20:01.571321+0000","last_unstale":"2024-09-13T18:21:27.143891+0000","last_undegraded":"2024-09-13T18:21:27.143891+0000","last_fullsized":"2024-09-13T18:21:27.143891+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:09.374364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.338246+0000","last_change":"2024-09-13T18:20:06.130687+0000","last_active":"2024-09-13T18:21:27.338246+0000","last_peered":"2024-09-13T18:21:27.338246+0000","last_clean":"2024-09-13T18:21:27.338246+0000","last_became_active":"2024-09-13T18:20:06.129839+0000","last_became_peered":"2024-09-13T18:20:06.129839+0000","last_unstale":"2024-09-13T18:21:27.338246+0000","last_undegraded":"2024-09-13T18:21:27.338246+0000","last_fullsized":"2024-09-13T18:21:27.338246+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:12:52.334945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.730952+0000","last_change":"2024-09-13T18:20:01.917244+0000","last_active":"2024-09-13T18:21:27.730952+0000","last_peered":"2024-09-13T18:21:27.730952+0000","last_clean":"2024-09-13T18:21:27.730952+0000","last_became_active":"2024-09-13T18:20:01.916948+0000","last_became_peered":"2024-09-13T18:20:01.916948+0000","last_unstale":"2024-09-13T18:21:27.730952+0000","last_undegraded":"2024-09-13T18:21:27.730952+0000","last_fullsized":"2024-09-13T18:21:27.730952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:17:11.967096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'9","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.140219+0000","last_change":"2024-09-13T18:20:01.569123+0000","last_active":"2024-09-13T18:21:27.140219+0000","last_peered":"2024-09-13T18:21:27.140219+0000","last_clean":"2024-09-13T18:21:27.140219+0000","last_became_active":"2024-09-13T18:20:01.568720+0000","last_became_peered":"2024-09-13T18:20:01.568720+0000","last_unstale":"2024-09-13T18:21:27.140219+0000","last_undegraded":"2024-09-13T18:21:27.140219+0000","last_fullsized":"2024-09-13T18:21:27.140219+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:04:29.561013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.730418+0000","last_change":"2024-09-13T18:20:06.141771+0000","last_active":"2024-09-13T18:21:27.730418+0000","last_peered":"2024-09-13T18:21:27.730418+0000","last_clean":"2024-09-13T18:21:27.730418+0000","last_became_active":"2024-09-13T18:20:06.141362+0000","last_became_peered":"2024-09-13T18:20:06.141362+0000","last_unstale":"2024-09-13T18:21:27.730418+0000","last_undegraded":"2024-09-13T18:21:27.730418+0000","last_fullsized":"2024-09-13T18:21:27.730418+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:09:58.597605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.144199+0000","last_change":"2024-09-13T18:20:01.911048+0000","last_active":"2024-09-13T18:21:27.144199+0000","last_peered":"2024-09-13T18:21:27.144199+0000","last_clean":"2024-09-13T18:21:27.144199+0000","last_became_active":"2024-09-13T18:20:01.910735+0000","last_became_peered":"2024-09-13T18:20:01.910735+0000","last_unstale":"2024-09-13T18:21:27.144199+0000","last_undegraded":"2024-09-13T18:21:27.144199+0000","last_fullsized":"2024-09-13T18:21:27.144199+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:07:35.447676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"91'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.142192+0000","last_change":"2024-09-13T18:20:01.571627+0000","last_active":"2024-09-13T18:21:27.142192+0000","last_peered":"2024-09-13T18:21:27.142192+0000","last_clean":"2024-09-13T18:21:27.142192+0000","last_became_active":"2024-09-13T18:20:01.571250+0000","last_became_peered":"2024-09-13T18:20:01.571250+0000","last_unstale":"2024-09-13T18:21:27.142192+0000","last_undegraded":"2024-09-13T18:21:27.142192+0000","last_fullsized":"2024-09-13T18:21:27.142192+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:34:03.479324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.143306+0000","last_change":"2024-09-13T18:20:06.132974+0000","last_active":"2024-09-13T18:21:27.143306+0000","last_peered":"2024-09-13T18:21:27.143306+0000","last_clean":"2024-09-13T18:21:27.143306+0000","last_became_active":"2024-09-13T18:20:06.131668+0000","last_became_peered":"2024-09-13T18:20:06.131668+0000","last_unstale":"2024-09-13T18:21:27.143306+0000","last_undegraded":"2024-09-13T18:21:27.143306+0000","last_fullsized":"2024-09-13T18:21:27.143306+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:36:30.329761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.141677+0000","last_change":"2024-09-13T18:20:01.904670+0000","last_active":"2024-09-13T18:21:27.141677+0000","last_peered":"2024-09-13T18:21:27.141677+0000","last_clean":"2024-09-13T18:21:27.141677+0000","last_became_active":"2024-09-13T18:20:01.904316+0000","last_became_peered":"2024-09-13T18:20:01.904316+0000","last_unstale":"2024-09-13T18:21:27.141677+0000","last_undegraded":"2024-09-13T18:21:27.141677+0000","last_fullsized":"2024-09-13T18:21:27.141677+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:08:17.687179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"91'1","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.140077+0000","last_change":"2024-09-13T18:20:01.564750+0000","last_active":"2024-09-13T18:21:27.140077+0000","last_peered":"2024-09-13T18:21:27.140077+0000","last_clean":"2024-09-13T18:21:27.140077+0000","last_became_active":"2024-09-13T18:20:01.564096+0000","last_became_peered":"2024-09-13T18:20:01.564096+0000","last_unstale":"2024-09-13T18:21:27.140077+0000","last_undegraded":"2024-09-13T18:21:27.140077+0000","last_fullsized":"2024-09-13T18:21:27.140077+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:42:19.588824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.141170+0000","last_change":"2024-09-13T18:20:06.133386+0000","last_active":"2024-09-13T18:21:27.141170+0000","last_peered":"2024-09-13T18:21:27.141170+0000","last_clean":"2024-09-13T18:21:27.141170+0000","last_became_active":"2024-09-13T18:20:06.133004+0000","last_became_peered":"2024-09-13T18:20:06.133004+0000","last_unstale":"2024-09-13T18:21:27.141170+0000","last_undegraded":"2024-09-13T18:21:27.141170+0000","last_fullsized":"2024-09-13T18:21:27.141170+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:54:46.440410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.141335+0000","last_change":"2024-09-13T18:20:01.912823+0000","last_active":"2024-09-13T18:21:27.141335+0000","last_peered":"2024-09-13T18:21:27.141335+0000","last_clean":"2024-09-13T18:21:27.141335+0000","last_became_active":"2024-09-13T18:20:01.912123+0000","last_became_peered":"2024-09-13T18:20:01.912123+0000","last_unstale":"2024-09-13T18:21:27.141335+0000","last_undegraded":"2024-09-13T18:21:27.141335+0000","last_fullsized":"2024-09-13T18:21:27.141335+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:32:33.544816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"91'5","reported_seq":103,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:44.356142+0000","last_change":"2024-09-13T18:20:01.577095+0000","last_active":"2024-09-13T18:21:44.356142+0000","last_peered":"2024-09-13T18:21:44.356142+0000","last_clean":"2024-09-13T18:21:44.356142+0000","last_became_active":"2024-09-13T18:20:01.576765+0000","last_became_peered":"2024-09-13T18:20:01.576765+0000","last_unstale":"2024-09-13T18:21:44.356142+0000","last_undegraded":"2024-09-13T18:21:44.356142+0000","last_fullsized":"2024-09-13T18:21:44.356142+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:44:08.750441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":78,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.143896+0000","last_change":"2024-09-13T18:20:06.131650+0000","last_active":"2024-09-13T18:21:27.143896+0000","last_peered":"2024-09-13T18:21:27.143896+0000","last_clean":"2024-09-13T18:21:27.143896+0000","last_became_active":"2024-09-13T18:20:06.130902+0000","last_became_peered":"2024-09-13T18:20:06.130902+0000","last_unstale":"2024-09-13T18:21:27.143896+0000","last_undegraded":"2024-09-13T18:21:27.143896+0000","last_fullsized":"2024-09-13T18:21:27.143896+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.366694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.144134+0000","last_change":"2024-09-13T18:20:01.918221+0000","last_active":"2024-09-13T18:21:27.144134+0000","last_peered":"2024-09-13T18:21:27.144134+0000","last_clean":"2024-09-13T18:21:27.144134+0000","last_became_active":"2024-09-13T18:20:01.917134+0000","last_became_peered":"2024-09-13T18:20:01.917134+0000","last_unstale":"2024-09-13T18:21:27.144134+0000","last_undegraded":"2024-09-13T18:21:27.144134+0000","last_fullsized":"2024-09-13T18:21:27.144134+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:19:06.269454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'5","reported_seq":102,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:42.353281+0000","last_change":"2024-09-13T18:20:01.572258+0000","last_active":"2024-09-13T18:21:42.353281+0000","last_peered":"2024-09-13T18:21:42.353281+0000","last_clean":"2024-09-13T18:21:42.353281+0000","last_became_active":"2024-09-13T18:20:01.571968+0000","last_became_peered":"2024-09-13T18:20:01.571968+0000","last_unstale":"2024-09-13T18:21:42.353281+0000","last_undegraded":"2024-09-13T18:21:42.353281+0000","last_fullsized":"2024-09-13T18:21:42.353281+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:40:59.143123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":77,"num_read_kb":77,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.144035+0000","last_change":"2024-09-13T18:20:06.131714+0000","last_active":"2024-09-13T18:21:27.144035+0000","last_peered":"2024-09-13T18:21:27.144035+0000","last_clean":"2024-09-13T18:21:27.144035+0000","last_became_active":"2024-09-13T18:20:06.131244+0000","last_became_peered":"2024-09-13T18:20:06.131244+0000","last_unstale":"2024-09-13T18:21:27.144035+0000","last_undegraded":"2024-09-13T18:21:27.144035+0000","last_fullsized":"2024-09-13T18:21:27.144035+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:37.946002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.143667+0000","last_change":"2024-09-13T18:20:01.917351+0000","last_active":"2024-09-13T18:21:27.143667+0000","last_peered":"2024-09-13T18:21:27.143667+0000","last_clean":"2024-09-13T18:21:27.143667+0000","last_became_active":"2024-09-13T18:20:01.917067+0000","last_became_peered":"2024-09-13T18:20:01.917067+0000","last_unstale":"2024-09-13T18:21:27.143667+0000","last_undegraded":"2024-09-13T18:21:27.143667+0000","last_fullsized":"2024-09-13T18:21:27.143667+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:35:37.723132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'4","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.145517+0000","last_change":"2024-09-13T18:20:01.573515+0000","last_active":"2024-09-13T18:21:27.145517+0000","last_peered":"2024-09-13T18:21:27.145517+0000","last_clean":"2024-09-13T18:21:27.145517+0000","last_became_active":"2024-09-13T18:20:01.573211+0000","last_became_peered":"2024-09-13T18:20:01.573211+0000","last_unstale":"2024-09-13T18:21:27.145517+0000","last_undegraded":"2024-09-13T18:21:27.145517+0000","last_fullsized":"2024-09-13T18:21:27.145517+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:05:30.912467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:28.232099+0000","last_change":"2024-09-13T18:21:28.231613+0000","last_active":"2024-09-13T18:21:28.232099+0000","last_peered":"2024-09-13T18:21:28.232099+0000","last_clean":"2024-09-13T18:21:28.232099+0000","last_became_active":"2024-09-13T18:21:28.231243+0000","last_became_peered":"2024-09-13T18:21:28.231243+0000","last_unstale":"2024-09-13T18:21:28.232099+0000","last_undegraded":"2024-09-13T18:21:28.232099+0000","last_fullsized":"2024-09-13T18:21:28.232099+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:58:46.066195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.730679+0000","last_change":"2024-09-13T18:20:01.921597+0000","last_active":"2024-09-13T18:21:27.730679+0000","last_peered":"2024-09-13T18:21:27.730679+0000","last_clean":"2024-09-13T18:21:27.730679+0000","last_became_active":"2024-09-13T18:20:01.920200+0000","last_became_peered":"2024-09-13T18:20:01.920200+0000","last_unstale":"2024-09-13T18:21:27.730679+0000","last_undegraded":"2024-09-13T18:21:27.730679+0000","last_fullsized":"2024-09-13T18:21:27.730679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:22:40.517941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'6","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.143488+0000","last_change":"2024-09-13T18:20:01.571313+0000","last_active":"2024-09-13T18:21:27.143488+0000","last_peered":"2024-09-13T18:21:27.143488+0000","last_clean":"2024-09-13T18:21:27.143488+0000","last_became_active":"2024-09-13T18:20:01.570977+0000","last_became_peered":"2024-09-13T18:20:01.570977+0000","last_unstale":"2024-09-13T18:21:27.143488+0000","last_undegraded":"2024-09-13T18:21:27.143488+0000","last_fullsized":"2024-09-13T18:21:27.143488+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:06.870320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":346,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:28.154829+0000","last_change":"2024-09-13T18:18:51.237409+0000","last_active":"2024-09-13T18:21:28.154829+0000","last_peered":"2024-09-13T18:21:28.154829+0000","last_clean":"2024-09-13T18:21:28.154829+0000","last_became_active":"2024-09-13T18:18:51.236986+0000","last_became_peered":"2024-09-13T18:18:51.236986+0000","last_unstale":"2024-09-13T18:21:28.154829+0000","last_undegraded":"2024-09-13T18:21:28.154829+0000","last_fullsized":"2024-09-13T18:21:28.154829+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:15:40.757299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:15:40.757299+0000","last_clean_scrub_stamp":"2024-09-13T18:15:40.757299+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:58:29.679404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.145643+0000","last_change":"2024-09-13T18:20:06.130803+0000","last_active":"2024-09-13T18:21:27.145643+0000","last_peered":"2024-09-13T18:21:27.145643+0000","last_clean":"2024-09-13T18:21:27.145643+0000","last_became_active":"2024-09-13T18:20:06.130433+0000","last_became_peered":"2024-09-13T18:20:06.130433+0000","last_unstale":"2024-09-13T18:21:27.145643+0000","last_undegraded":"2024-09-13T18:21:27.145643+0000","last_fullsized":"2024-09-13T18:21:27.145643+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:23:11.787084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.145903+0000","last_change":"2024-09-13T18:20:01.922702+0000","last_active":"2024-09-13T18:21:27.145903+0000","last_peered":"2024-09-13T18:21:27.145903+0000","last_clean":"2024-09-13T18:21:27.145903+0000","last_became_active":"2024-09-13T18:20:01.921109+0000","last_became_peered":"2024-09-13T18:20:01.921109+0000","last_unstale":"2024-09-13T18:21:27.145903+0000","last_undegraded":"2024-09-13T18:21:27.145903+0000","last_fullsized":"2024-09-13T18:21:27.145903+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:03:34.618054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"91'5","reported_seq":105,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:44.359394+0000","last_change":"2024-09-13T18:20:01.571024+0000","last_active":"2024-09-13T18:21:44.359394+0000","last_peered":"2024-09-13T18:21:44.359394+0000","last_clean":"2024-09-13T18:21:44.359394+0000","last_became_active":"2024-09-13T18:20:01.570687+0000","last_became_peered":"2024-09-13T18:20:01.570687+0000","last_unstale":"2024-09-13T18:21:44.359394+0000","last_undegraded":"2024-09-13T18:21:44.359394+0000","last_fullsized":"2024-09-13T18:21:44.359394+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:10:25.126179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":80,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.321866+0000","last_change":"2024-09-13T18:20:06.129025+0000","last_active":"2024-09-13T18:21:27.321866+0000","last_peered":"2024-09-13T18:21:27.321866+0000","last_clean":"2024-09-13T18:21:27.321866+0000","last_became_active":"2024-09-13T18:20:06.128618+0000","last_became_peered":"2024-09-13T18:20:06.128618+0000","last_unstale":"2024-09-13T18:21:27.321866+0000","last_undegraded":"2024-09-13T18:21:27.321866+0000","last_fullsized":"2024-09-13T18:21:27.321866+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:20:08.891259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.146304+0000","last_change":"2024-09-13T18:20:01.921697+0000","last_active":"2024-09-13T18:21:27.146304+0000","last_peered":"2024-09-13T18:21:27.146304+0000","last_clean":"2024-09-13T18:21:27.146304+0000","last_became_active":"2024-09-13T18:20:01.920174+0000","last_became_peered":"2024-09-13T18:20:01.920174+0000","last_unstale":"2024-09-13T18:21:27.146304+0000","last_undegraded":"2024-09-13T18:21:27.146304+0000","last_fullsized":"2024-09-13T18:21:27.146304+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:57:03.922725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"91'3","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.145321+0000","last_change":"2024-09-13T18:20:01.582102+0000","last_active":"2024-09-13T18:21:27.145321+0000","last_peered":"2024-09-13T18:21:27.145321+0000","last_clean":"2024-09-13T18:21:27.145321+0000","last_became_active":"2024-09-13T18:20:01.581841+0000","last_became_peered":"2024-09-13T18:20:01.581841+0000","last_unstale":"2024-09-13T18:21:27.145321+0000","last_undegraded":"2024-09-13T18:21:27.145321+0000","last_fullsized":"2024-09-13T18:21:27.145321+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:59:57.929703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.147124+0000","last_change":"2024-09-13T18:20:06.133126+0000","last_active":"2024-09-13T18:21:27.147124+0000","last_peered":"2024-09-13T18:21:27.147124+0000","last_clean":"2024-09-13T18:21:27.147124+0000","last_became_active":"2024-09-13T18:20:06.132747+0000","last_became_peered":"2024-09-13T18:20:06.132747+0000","last_unstale":"2024-09-13T18:21:27.147124+0000","last_undegraded":"2024-09-13T18:21:27.147124+0000","last_fullsized":"2024-09-13T18:21:27.147124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:54:21.843090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.145788+0000","last_change":"2024-09-13T18:20:01.916306+0000","last_active":"2024-09-13T18:21:27.145788+0000","last_peered":"2024-09-13T18:21:27.145788+0000","last_clean":"2024-09-13T18:21:27.145788+0000","last_became_active":"2024-09-13T18:20:01.915739+0000","last_became_peered":"2024-09-13T18:20:01.915739+0000","last_unstale":"2024-09-13T18:21:27.145788+0000","last_undegraded":"2024-09-13T18:21:27.145788+0000","last_fullsized":"2024-09-13T18:21:27.145788+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:19:09.689909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"91'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.144275+0000","last_change":"2024-09-13T18:20:01.565821+0000","last_active":"2024-09-13T18:21:27.144275+0000","last_peered":"2024-09-13T18:21:27.144275+0000","last_clean":"2024-09-13T18:21:27.144275+0000","last_became_active":"2024-09-13T18:20:01.565467+0000","last_became_peered":"2024-09-13T18:20:01.565467+0000","last_unstale":"2024-09-13T18:21:27.144275+0000","last_undegraded":"2024-09-13T18:21:27.144275+0000","last_fullsized":"2024-09-13T18:21:27.144275+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:11.609400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.143920+0000","last_change":"2024-09-13T18:20:06.131348+0000","last_active":"2024-09-13T18:21:27.143920+0000","last_peered":"2024-09-13T18:21:27.143920+0000","last_clean":"2024-09-13T18:21:27.143920+0000","last_became_active":"2024-09-13T18:20:06.131019+0000","last_became_peered":"2024-09-13T18:20:06.131019+0000","last_unstale":"2024-09-13T18:21:27.143920+0000","last_undegraded":"2024-09-13T18:21:27.143920+0000","last_fullsized":"2024-09-13T18:21:27.143920+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:13:49.717503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.143749+0000","last_change":"2024-09-13T18:20:01.922306+0000","last_active":"2024-09-13T18:21:27.143749+0000","last_peered":"2024-09-13T18:21:27.143749+0000","last_clean":"2024-09-13T18:21:27.143749+0000","last_became_active":"2024-09-13T18:20:01.921956+0000","last_became_peered":"2024-09-13T18:20:01.921956+0000","last_unstale":"2024-09-13T18:21:27.143749+0000","last_undegraded":"2024-09-13T18:21:27.143749+0000","last_fullsized":"2024-09-13T18:21:27.143749+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:17:48.755318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.144213+0000","last_change":"2024-09-13T18:20:27.194085+0000","last_active":"2024-09-13T18:21:27.144213+0000","last_peered":"2024-09-13T18:21:27.144213+0000","last_clean":"2024-09-13T18:21:27.144213+0000","last_became_active":"2024-09-13T18:20:27.190136+0000","last_became_peered":"2024-09-13T18:20:27.190136+0000","last_unstale":"2024-09-13T18:21:27.144213+0000","last_undegraded":"2024-09-13T18:21:27.144213+0000","last_fullsized":"2024-09-13T18:21:27.144213+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:57:07.202901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.144120+0000","last_change":"2024-09-13T18:20:06.135498+0000","last_active":"2024-09-13T18:21:27.144120+0000","last_peered":"2024-09-13T18:21:27.144120+0000","last_clean":"2024-09-13T18:21:27.144120+0000","last_became_active":"2024-09-13T18:20:06.135095+0000","last_became_peered":"2024-09-13T18:20:06.135095+0000","last_unstale":"2024-09-13T18:21:27.144120+0000","last_undegraded":"2024-09-13T18:21:27.144120+0000","last_fullsized":"2024-09-13T18:21:27.144120+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:06:40.609718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.139672+0000","last_change":"2024-09-13T18:20:27.186825+0000","last_active":"2024-09-13T18:21:27.139672+0000","last_peered":"2024-09-13T18:21:27.139672+0000","last_clean":"2024-09-13T18:21:27.139672+0000","last_became_active":"2024-09-13T18:20:27.186523+0000","last_became_peered":"2024-09-13T18:20:27.186523+0000","last_unstale":"2024-09-13T18:21:27.139672+0000","last_undegraded":"2024-09-13T18:21:27.139672+0000","last_fullsized":"2024-09-13T18:21:27.139672+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:28:30.453945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.10","version":"91'17","reported_seq":59,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.140911+0000","last_change":"2024-09-13T18:20:27.200142+0000","last_active":"2024-09-13T18:21:27.140911+0000","last_peered":"2024-09-13T18:21:27.140911+0000","last_clean":"2024-09-13T18:21:27.140911+0000","last_became_active":"2024-09-13T18:20:27.189670+0000","last_became_peered":"2024-09-13T18:20:27.189670+0000","last_unstale":"2024-09-13T18:21:27.140911+0000","last_undegraded":"2024-09-13T18:21:27.140911+0000","last_fullsized":"2024-09-13T18:21:27.140911+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:16:02.934269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.144245+0000","last_change":"2024-09-13T18:20:06.136742+0000","last_active":"2024-09-13T18:21:27.144245+0000","last_peered":"2024-09-13T18:21:27.144245+0000","last_clean":"2024-09-13T18:21:27.144245+0000","last_became_active":"2024-09-13T18:20:06.136373+0000","last_became_peered":"2024-09-13T18:20:06.136373+0000","last_unstale":"2024-09-13T18:21:27.144245+0000","last_undegraded":"2024-09-13T18:21:27.144245+0000","last_fullsized":"2024-09-13T18:21:27.144245+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:49:11.782489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.145415+0000","last_change":"2024-09-13T18:20:01.922910+0000","last_active":"2024-09-13T18:21:27.145415+0000","last_peered":"2024-09-13T18:21:27.145415+0000","last_clean":"2024-09-13T18:21:27.145415+0000","last_became_active":"2024-09-13T18:20:01.921162+0000","last_became_peered":"2024-09-13T18:20:01.921162+0000","last_unstale":"2024-09-13T18:21:27.145415+0000","last_undegraded":"2024-09-13T18:21:27.145415+0000","last_fullsized":"2024-09-13T18:21:27.145415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:14:47.243418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"91'5","reported_seq":106,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:45.357582+0000","last_change":"2024-09-13T18:20:01.575830+0000","last_active":"2024-09-13T18:21:45.357582+0000","last_peered":"2024-09-13T18:21:45.357582+0000","last_clean":"2024-09-13T18:21:45.357582+0000","last_became_active":"2024-09-13T18:20:01.575334+0000","last_became_peered":"2024-09-13T18:20:01.575334+0000","last_unstale":"2024-09-13T18:21:45.357582+0000","last_undegraded":"2024-09-13T18:21:45.357582+0000","last_fullsized":"2024-09-13T18:21:45.357582+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:13:00.935600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.140407+0000","last_change":"2024-09-13T18:20:06.138609+0000","last_active":"2024-09-13T18:21:27.140407+0000","last_peered":"2024-09-13T18:21:27.140407+0000","last_clean":"2024-09-13T18:21:27.140407+0000","last_became_active":"2024-09-13T18:20:06.138232+0000","last_became_peered":"2024-09-13T18:20:06.138232+0000","last_unstale":"2024-09-13T18:21:27.140407+0000","last_undegraded":"2024-09-13T18:21:27.140407+0000","last_fullsized":"2024-09-13T18:21:27.140407+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:46:06.227407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.145886+0000","last_change":"2024-09-13T18:20:01.928757+0000","last_active":"2024-09-13T18:21:27.145886+0000","last_peered":"2024-09-13T18:21:27.145886+0000","last_clean":"2024-09-13T18:21:27.145886+0000","last_became_active":"2024-09-13T18:20:01.928350+0000","last_became_peered":"2024-09-13T18:20:01.928350+0000","last_unstale":"2024-09-13T18:21:27.145886+0000","last_undegraded":"2024-09-13T18:21:27.145886+0000","last_fullsized":"2024-09-13T18:21:27.145886+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:17:43.208106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"91'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.145386+0000","last_change":"2024-09-13T18:20:01.579128+0000","last_active":"2024-09-13T18:21:27.145386+0000","last_peered":"2024-09-13T18:21:27.145386+0000","last_clean":"2024-09-13T18:21:27.145386+0000","last_became_active":"2024-09-13T18:20:01.578796+0000","last_became_peered":"2024-09-13T18:20:01.578796+0000","last_unstale":"2024-09-13T18:21:27.145386+0000","last_undegraded":"2024-09-13T18:21:27.145386+0000","last_fullsized":"2024-09-13T18:21:27.145386+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:30:04.836103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.143306+0000","last_change":"2024-09-13T18:20:06.131404+0000","last_active":"2024-09-13T18:21:27.143306+0000","last_peered":"2024-09-13T18:21:27.143306+0000","last_clean":"2024-09-13T18:21:27.143306+0000","last_became_active":"2024-09-13T18:20:06.130931+0000","last_became_peered":"2024-09-13T18:20:06.130931+0000","last_unstale":"2024-09-13T18:21:27.143306+0000","last_undegraded":"2024-09-13T18:21:27.143306+0000","last_fullsized":"2024-09-13T18:21:27.143306+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:55:54.568816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.144146+0000","last_change":"2024-09-13T18:20:01.916089+0000","last_active":"2024-09-13T18:21:27.144146+0000","last_peered":"2024-09-13T18:21:27.144146+0000","last_clean":"2024-09-13T18:21:27.144146+0000","last_became_active":"2024-09-13T18:20:01.915783+0000","last_became_peered":"2024-09-13T18:20:01.915783+0000","last_unstale":"2024-09-13T18:21:27.144146+0000","last_undegraded":"2024-09-13T18:21:27.144146+0000","last_fullsized":"2024-09-13T18:21:27.144146+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:43:25.886029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"91'9","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.338482+0000","last_change":"2024-09-13T18:20:01.577381+0000","last_active":"2024-09-13T18:21:27.338482+0000","last_peered":"2024-09-13T18:21:27.338482+0000","last_clean":"2024-09-13T18:21:27.338482+0000","last_became_active":"2024-09-13T18:20:01.577044+0000","last_became_peered":"2024-09-13T18:20:01.577044+0000","last_unstale":"2024-09-13T18:21:27.338482+0000","last_undegraded":"2024-09-13T18:21:27.338482+0000","last_fullsized":"2024-09-13T18:21:27.338482+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:00.907948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.145091+0000","last_change":"2024-09-13T18:20:06.139971+0000","last_active":"2024-09-13T18:21:27.145091+0000","last_peered":"2024-09-13T18:21:27.145091+0000","last_clean":"2024-09-13T18:21:27.145091+0000","last_became_active":"2024-09-13T18:20:06.139507+0000","last_became_peered":"2024-09-13T18:20:06.139507+0000","last_unstale":"2024-09-13T18:21:27.145091+0000","last_undegraded":"2024-09-13T18:21:27.145091+0000","last_fullsized":"2024-09-13T18:21:27.145091+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:40:21.037768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.141865+0000","last_change":"2024-09-13T18:20:01.915348+0000","last_active":"2024-09-13T18:21:27.141865+0000","last_peered":"2024-09-13T18:21:27.141865+0000","last_clean":"2024-09-13T18:21:27.141865+0000","last_became_active":"2024-09-13T18:20:01.915042+0000","last_became_peered":"2024-09-13T18:20:01.915042+0000","last_unstale":"2024-09-13T18:21:27.141865+0000","last_undegraded":"2024-09-13T18:21:27.141865+0000","last_fullsized":"2024-09-13T18:21:27.141865+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:31:33.712314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"91'34","reported_seq":213,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:43.357276+0000","last_change":"2024-09-13T18:20:01.578456+0000","last_active":"2024-09-13T18:21:43.357276+0000","last_peered":"2024-09-13T18:21:43.357276+0000","last_clean":"2024-09-13T18:21:43.357276+0000","last_became_active":"2024-09-13T18:20:01.578051+0000","last_became_peered":"2024-09-13T18:20:01.578051+0000","last_unstale":"2024-09-13T18:21:43.357276+0000","last_undegraded":"2024-09-13T18:21:43.357276+0000","last_fullsized":"2024-09-13T18:21:43.357276+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:34:17.651620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63310,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":159,"num_read_kb":142,"num_write":36,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.139797+0000","last_change":"2024-09-13T18:20:06.131410+0000","last_active":"2024-09-13T18:21:27.139797+0000","last_peered":"2024-09-13T18:21:27.139797+0000","last_clean":"2024-09-13T18:21:27.139797+0000","last_became_active":"2024-09-13T18:20:06.131060+0000","last_became_peered":"2024-09-13T18:20:06.131060+0000","last_unstale":"2024-09-13T18:21:27.139797+0000","last_undegraded":"2024-09-13T18:21:27.139797+0000","last_fullsized":"2024-09-13T18:21:27.139797+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:08:25.266971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.322247+0000","last_change":"2024-09-13T18:20:01.928839+0000","last_active":"2024-09-13T18:21:27.322247+0000","last_peered":"2024-09-13T18:21:27.322247+0000","last_clean":"2024-09-13T18:21:27.322247+0000","last_became_active":"2024-09-13T18:20:01.925774+0000","last_became_peered":"2024-09-13T18:20:01.925774+0000","last_unstale":"2024-09-13T18:21:27.322247+0000","last_undegraded":"2024-09-13T18:21:27.322247+0000","last_fullsized":"2024-09-13T18:21:27.322247+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:15:23.049671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"91'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.145283+0000","last_change":"2024-09-13T18:20:01.576332+0000","last_active":"2024-09-13T18:21:27.145283+0000","last_peered":"2024-09-13T18:21:27.145283+0000","last_clean":"2024-09-13T18:21:27.145283+0000","last_became_active":"2024-09-13T18:20:01.575862+0000","last_became_peered":"2024-09-13T18:20:01.575862+0000","last_unstale":"2024-09-13T18:21:27.145283+0000","last_undegraded":"2024-09-13T18:21:27.145283+0000","last_fullsized":"2024-09-13T18:21:27.145283+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:17:00.452480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.143332+0000","last_change":"2024-09-13T18:20:06.133142+0000","last_active":"2024-09-13T18:21:27.143332+0000","last_peered":"2024-09-13T18:21:27.143332+0000","last_clean":"2024-09-13T18:21:27.143332+0000","last_became_active":"2024-09-13T18:20:06.132740+0000","last_became_peered":"2024-09-13T18:20:06.132740+0000","last_unstale":"2024-09-13T18:21:27.143332+0000","last_undegraded":"2024-09-13T18:21:27.143332+0000","last_fullsized":"2024-09-13T18:21:27.143332+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:12:46.442481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.147495+0000","last_change":"2024-09-13T18:20:01.919673+0000","last_active":"2024-09-13T18:21:27.147495+0000","last_peered":"2024-09-13T18:21:27.147495+0000","last_clean":"2024-09-13T18:21:27.147495+0000","last_became_active":"2024-09-13T18:20:01.919358+0000","last_became_peered":"2024-09-13T18:20:01.919358+0000","last_unstale":"2024-09-13T18:21:27.147495+0000","last_undegraded":"2024-09-13T18:21:27.147495+0000","last_fullsized":"2024-09-13T18:21:27.147495+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:40:11.407643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"91'4","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.146085+0000","last_change":"2024-09-13T18:20:27.190451+0000","last_active":"2024-09-13T18:21:27.146085+0000","last_peered":"2024-09-13T18:21:27.146085+0000","last_clean":"2024-09-13T18:21:27.146085+0000","last_became_active":"2024-09-13T18:20:27.187091+0000","last_became_peered":"2024-09-13T18:20:27.187091+0000","last_unstale":"2024-09-13T18:21:27.146085+0000","last_undegraded":"2024-09-13T18:21:27.146085+0000","last_fullsized":"2024-09-13T18:21:27.146085+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:57:07.976008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.139632+0000","last_change":"2024-09-13T18:20:06.133475+0000","last_active":"2024-09-13T18:21:27.139632+0000","last_peered":"2024-09-13T18:21:27.139632+0000","last_clean":"2024-09-13T18:21:27.139632+0000","last_became_active":"2024-09-13T18:20:06.133109+0000","last_became_peered":"2024-09-13T18:20:06.133109+0000","last_unstale":"2024-09-13T18:21:27.139632+0000","last_undegraded":"2024-09-13T18:21:27.139632+0000","last_fullsized":"2024-09-13T18:21:27.139632+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:47:19.567115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.322369+0000","last_change":"2024-09-13T18:20:01.922660+0000","last_active":"2024-09-13T18:21:27.322369+0000","last_peered":"2024-09-13T18:21:27.322369+0000","last_clean":"2024-09-13T18:21:27.322369+0000","last_became_active":"2024-09-13T18:20:01.922370+0000","last_became_peered":"2024-09-13T18:20:01.922370+0000","last_unstale":"2024-09-13T18:21:27.322369+0000","last_undegraded":"2024-09-13T18:21:27.322369+0000","last_fullsized":"2024-09-13T18:21:27.322369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:54:32.029704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"91'4","reported_seq":176,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:43.357288+0000","last_change":"2024-09-13T18:20:01.580975+0000","last_active":"2024-09-13T18:21:43.357288+0000","last_peered":"2024-09-13T18:21:43.357288+0000","last_clean":"2024-09-13T18:21:43.357288+0000","last_became_active":"2024-09-13T18:20:01.580647+0000","last_became_peered":"2024-09-13T18:20:01.580647+0000","last_unstale":"2024-09-13T18:21:43.357288+0000","last_undegraded":"2024-09-13T18:21:43.357288+0000","last_fullsized":"2024-09-13T18:21:43.357288+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:26:20.942915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":152,"num_read_kb":152,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.730531+0000","last_change":"2024-09-13T18:20:06.140003+0000","last_active":"2024-09-13T18:21:27.730531+0000","last_peered":"2024-09-13T18:21:27.730531+0000","last_clean":"2024-09-13T18:21:27.730531+0000","last_became_active":"2024-09-13T18:20:06.139663+0000","last_became_peered":"2024-09-13T18:20:06.139663+0000","last_unstale":"2024-09-13T18:21:27.730531+0000","last_undegraded":"2024-09-13T18:21:27.730531+0000","last_fullsized":"2024-09-13T18:21:27.730531+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:54:49.662401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.147273+0000","last_change":"2024-09-13T18:20:06.134448+0000","last_active":"2024-09-13T18:21:27.147273+0000","last_peered":"2024-09-13T18:21:27.147273+0000","last_clean":"2024-09-13T18:21:27.147273+0000","last_became_active":"2024-09-13T18:20:06.134084+0000","last_became_peered":"2024-09-13T18:20:06.134084+0000","last_unstale":"2024-09-13T18:21:27.147273+0000","last_undegraded":"2024-09-13T18:21:27.147273+0000","last_fullsized":"2024-09-13T18:21:27.147273+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:55:50.475665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.139809+0000","last_change":"2024-09-13T18:20:01.918955+0000","last_active":"2024-09-13T18:21:27.139809+0000","last_peered":"2024-09-13T18:21:27.139809+0000","last_clean":"2024-09-13T18:21:27.139809+0000","last_became_active":"2024-09-13T18:20:01.918666+0000","last_became_peered":"2024-09-13T18:20:01.918666+0000","last_unstale":"2024-09-13T18:21:27.139809+0000","last_undegraded":"2024-09-13T18:21:27.139809+0000","last_fullsized":"2024-09-13T18:21:27.139809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:08:50.697710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"91'3","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.144225+0000","last_change":"2024-09-13T18:20:01.578789+0000","last_active":"2024-09-13T18:21:27.144225+0000","last_peered":"2024-09-13T18:21:27.144225+0000","last_clean":"2024-09-13T18:21:27.144225+0000","last_became_active":"2024-09-13T18:20:01.578476+0000","last_became_peered":"2024-09-13T18:20:01.578476+0000","last_unstale":"2024-09-13T18:21:27.144225+0000","last_undegraded":"2024-09-13T18:21:27.144225+0000","last_fullsized":"2024-09-13T18:21:27.144225+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:40:58.355492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:28.423633+0000","last_change":"2024-09-13T18:21:28.423633+0000","last_active":"2024-09-13T18:21:28.423633+0000","last_peered":"2024-09-13T18:21:28.423633+0000","last_clean":"2024-09-13T18:21:28.423633+0000","last_became_active":"2024-09-13T18:21:28.423128+0000","last_became_peered":"2024-09-13T18:21:28.423128+0000","last_unstale":"2024-09-13T18:21:28.423633+0000","last_undegraded":"2024-09-13T18:21:28.423633+0000","last_fullsized":"2024-09-13T18:21:28.423633+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:06:32.687874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.730540+0000","last_change":"2024-09-13T18:20:01.914804+0000","last_active":"2024-09-13T18:21:27.730540+0000","last_peered":"2024-09-13T18:21:27.730540+0000","last_clean":"2024-09-13T18:21:27.730540+0000","last_became_active":"2024-09-13T18:20:01.914517+0000","last_became_peered":"2024-09-13T18:20:01.914517+0000","last_unstale":"2024-09-13T18:21:27.730540+0000","last_undegraded":"2024-09-13T18:21:27.730540+0000","last_fullsized":"2024-09-13T18:21:27.730540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:57:10.734935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"91'3","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.143676+0000","last_change":"2024-09-13T18:20:01.569201+0000","last_active":"2024-09-13T18:21:27.143676+0000","last_peered":"2024-09-13T18:21:27.143676+0000","last_clean":"2024-09-13T18:21:27.143676+0000","last_became_active":"2024-09-13T18:20:01.568908+0000","last_became_peered":"2024-09-13T18:20:01.568908+0000","last_unstale":"2024-09-13T18:21:27.143676+0000","last_undegraded":"2024-09-13T18:21:27.143676+0000","last_fullsized":"2024-09-13T18:21:27.143676+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:16:34.069130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"91'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.141443+0000","last_change":"2024-09-13T18:20:01.579225+0000","last_active":"2024-09-13T18:21:27.141443+0000","last_peered":"2024-09-13T18:21:27.141443+0000","last_clean":"2024-09-13T18:21:27.141443+0000","last_became_active":"2024-09-13T18:20:01.578826+0000","last_became_peered":"2024-09-13T18:20:01.578826+0000","last_unstale":"2024-09-13T18:21:27.141443+0000","last_undegraded":"2024-09-13T18:21:27.141443+0000","last_fullsized":"2024-09-13T18:21:27.141443+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:48:07.632239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.145492+0000","last_change":"2024-09-13T18:20:01.914885+0000","last_active":"2024-09-13T18:21:27.145492+0000","last_peered":"2024-09-13T18:21:27.145492+0000","last_clean":"2024-09-13T18:21:27.145492+0000","last_became_active":"2024-09-13T18:20:01.914599+0000","last_became_peered":"2024-09-13T18:20:01.914599+0000","last_unstale":"2024-09-13T18:21:27.145492+0000","last_undegraded":"2024-09-13T18:21:27.145492+0000","last_fullsized":"2024-09-13T18:21:27.145492+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:41:30.888675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.338340+0000","last_change":"2024-09-13T18:20:06.129376+0000","last_active":"2024-09-13T18:21:27.338340+0000","last_peered":"2024-09-13T18:21:27.338340+0000","last_clean":"2024-09-13T18:21:27.338340+0000","last_became_active":"2024-09-13T18:20:06.129032+0000","last_became_peered":"2024-09-13T18:20:06.129032+0000","last_unstale":"2024-09-13T18:21:27.338340+0000","last_undegraded":"2024-09-13T18:21:27.338340+0000","last_fullsized":"2024-09-13T18:21:27.338340+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:21:53.511813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":105,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:44.359343+0000","last_change":"2024-09-13T18:20:01.576069+0000","last_active":"2024-09-13T18:21:44.359343+0000","last_peered":"2024-09-13T18:21:44.359343+0000","last_clean":"2024-09-13T18:21:44.359343+0000","last_became_active":"2024-09-13T18:20:01.575760+0000","last_became_peered":"2024-09-13T18:20:01.575760+0000","last_unstale":"2024-09-13T18:21:44.359343+0000","last_undegraded":"2024-09-13T18:21:44.359343+0000","last_fullsized":"2024-09-13T18:21:44.359343+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:05:47.677848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":80,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.141389+0000","last_change":"2024-09-13T18:20:01.917073+0000","last_active":"2024-09-13T18:21:27.141389+0000","last_peered":"2024-09-13T18:21:27.141389+0000","last_clean":"2024-09-13T18:21:27.141389+0000","last_became_active":"2024-09-13T18:20:01.916765+0000","last_became_peered":"2024-09-13T18:20:01.916765+0000","last_unstale":"2024-09-13T18:21:27.141389+0000","last_undegraded":"2024-09-13T18:21:27.141389+0000","last_fullsized":"2024-09-13T18:21:27.141389+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:22:07.788313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.338478+0000","last_change":"2024-09-13T18:20:06.131429+0000","last_active":"2024-09-13T18:21:27.338478+0000","last_peered":"2024-09-13T18:21:27.338478+0000","last_clean":"2024-09-13T18:21:27.338478+0000","last_became_active":"2024-09-13T18:20:06.131019+0000","last_became_peered":"2024-09-13T18:20:06.131019+0000","last_unstale":"2024-09-13T18:21:27.338478+0000","last_undegraded":"2024-09-13T18:21:27.338478+0000","last_fullsized":"2024-09-13T18:21:27.338478+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:44:37.892574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.143073+0000","last_change":"2024-09-13T18:20:01.914050+0000","last_active":"2024-09-13T18:21:27.143073+0000","last_peered":"2024-09-13T18:21:27.143073+0000","last_clean":"2024-09-13T18:21:27.143073+0000","last_became_active":"2024-09-13T18:20:01.913733+0000","last_became_peered":"2024-09-13T18:20:01.913733+0000","last_unstale":"2024-09-13T18:21:27.143073+0000","last_undegraded":"2024-09-13T18:21:27.143073+0000","last_fullsized":"2024-09-13T18:21:27.143073+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:52:43.672096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.338412+0000","last_change":"2024-09-13T18:20:01.924290+0000","last_active":"2024-09-13T18:21:27.338412+0000","last_peered":"2024-09-13T18:21:27.338412+0000","last_clean":"2024-09-13T18:21:27.338412+0000","last_became_active":"2024-09-13T18:20:01.923883+0000","last_became_peered":"2024-09-13T18:20:01.923883+0000","last_unstale":"2024-09-13T18:21:27.338412+0000","last_undegraded":"2024-09-13T18:21:27.338412+0000","last_fullsized":"2024-09-13T18:21:27.338412+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:11:18.737501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.338245+0000","last_change":"2024-09-13T18:20:27.184256+0000","last_active":"2024-09-13T18:21:27.338245+0000","last_peered":"2024-09-13T18:21:27.338245+0000","last_clean":"2024-09-13T18:21:27.338245+0000","last_became_active":"2024-09-13T18:20:27.183790+0000","last_became_peered":"2024-09-13T18:20:27.183790+0000","last_unstale":"2024-09-13T18:21:27.338245+0000","last_undegraded":"2024-09-13T18:21:27.338245+0000","last_fullsized":"2024-09-13T18:21:27.338245+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:06:48.885753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.147139+0000","last_change":"2024-09-13T18:20:01.915799+0000","last_active":"2024-09-13T18:21:27.147139+0000","last_peered":"2024-09-13T18:21:27.147139+0000","last_clean":"2024-09-13T18:21:27.147139+0000","last_became_active":"2024-09-13T18:20:01.915519+0000","last_became_peered":"2024-09-13T18:20:01.915519+0000","last_unstale":"2024-09-13T18:21:27.147139+0000","last_undegraded":"2024-09-13T18:21:27.147139+0000","last_fullsized":"2024-09-13T18:21:27.147139+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:08:26.065052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.144088+0000","last_change":"2024-09-13T18:20:01.918547+0000","last_active":"2024-09-13T18:21:27.144088+0000","last_peered":"2024-09-13T18:21:27.144088+0000","last_clean":"2024-09-13T18:21:27.144088+0000","last_became_active":"2024-09-13T18:20:01.918053+0000","last_became_peered":"2024-09-13T18:20:01.918053+0000","last_unstale":"2024-09-13T18:21:27.144088+0000","last_undegraded":"2024-09-13T18:21:27.144088+0000","last_fullsized":"2024-09-13T18:21:27.144088+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:25:58.235101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.143607+0000","last_change":"2024-09-13T18:20:01.916205+0000","last_active":"2024-09-13T18:21:27.143607+0000","last_peered":"2024-09-13T18:21:27.143607+0000","last_clean":"2024-09-13T18:21:27.143607+0000","last_became_active":"2024-09-13T18:20:01.915909+0000","last_became_peered":"2024-09-13T18:20:01.915909+0000","last_unstale":"2024-09-13T18:21:27.143607+0000","last_undegraded":"2024-09-13T18:21:27.143607+0000","last_fullsized":"2024-09-13T18:21:27.143607+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:23:03.976513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.338788+0000","last_change":"2024-09-13T18:20:01.916621+0000","last_active":"2024-09-13T18:21:27.338788+0000","last_peered":"2024-09-13T18:21:27.338788+0000","last_clean":"2024-09-13T18:21:27.338788+0000","last_became_active":"2024-09-13T18:20:01.916335+0000","last_became_peered":"2024-09-13T18:20:01.916335+0000","last_unstale":"2024-09-13T18:21:27.338788+0000","last_undegraded":"2024-09-13T18:21:27.338788+0000","last_fullsized":"2024-09-13T18:21:27.338788+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:08:16.104571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.730943+0000","last_change":"2024-09-13T18:20:01.921195+0000","last_active":"2024-09-13T18:21:27.730943+0000","last_peered":"2024-09-13T18:21:27.730943+0000","last_clean":"2024-09-13T18:21:27.730943+0000","last_became_active":"2024-09-13T18:20:01.920895+0000","last_became_peered":"2024-09-13T18:20:01.920895+0000","last_unstale":"2024-09-13T18:21:27.730943+0000","last_undegraded":"2024-09-13T18:21:27.730943+0000","last_fullsized":"2024-09-13T18:21:27.730943+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:59:53.920047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.144052+0000","last_change":"2024-09-13T18:20:01.910889+0000","last_active":"2024-09-13T18:21:27.144052+0000","last_peered":"2024-09-13T18:21:27.144052+0000","last_clean":"2024-09-13T18:21:27.144052+0000","last_became_active":"2024-09-13T18:20:01.910451+0000","last_became_peered":"2024-09-13T18:20:01.910451+0000","last_unstale":"2024-09-13T18:21:27.144052+0000","last_undegraded":"2024-09-13T18:21:27.144052+0000","last_fullsized":"2024-09-13T18:21:27.144052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:06:52.552785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.730619+0000","last_change":"2024-09-13T18:20:01.914041+0000","last_active":"2024-09-13T18:21:27.730619+0000","last_peered":"2024-09-13T18:21:27.730619+0000","last_clean":"2024-09-13T18:21:27.730619+0000","last_became_active":"2024-09-13T18:20:01.913240+0000","last_became_peered":"2024-09-13T18:20:01.913240+0000","last_unstale":"2024-09-13T18:21:27.730619+0000","last_undegraded":"2024-09-13T18:21:27.730619+0000","last_fullsized":"2024-09-13T18:21:27.730619+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:39:37.062544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.322260+0000","last_change":"2024-09-13T18:20:01.927322+0000","last_active":"2024-09-13T18:21:27.322260+0000","last_peered":"2024-09-13T18:21:27.322260+0000","last_clean":"2024-09-13T18:21:27.322260+0000","last_became_active":"2024-09-13T18:20:01.926767+0000","last_became_peered":"2024-09-13T18:20:01.926767+0000","last_unstale":"2024-09-13T18:21:27.322260+0000","last_undegraded":"2024-09-13T18:21:27.322260+0000","last_fullsized":"2024-09-13T18:21:27.322260+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:20.986089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.144026+0000","last_change":"2024-09-13T18:20:01.916314+0000","last_active":"2024-09-13T18:21:27.144026+0000","last_peered":"2024-09-13T18:21:27.144026+0000","last_clean":"2024-09-13T18:21:27.144026+0000","last_became_active":"2024-09-13T18:20:01.915790+0000","last_became_peered":"2024-09-13T18:20:01.915790+0000","last_unstale":"2024-09-13T18:21:27.144026+0000","last_undegraded":"2024-09-13T18:21:27.144026+0000","last_fullsized":"2024-09-13T18:21:27.144026+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:20:30.063814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.143481+0000","last_change":"2024-09-13T18:20:27.187172+0000","last_active":"2024-09-13T18:21:27.143481+0000","last_peered":"2024-09-13T18:21:27.143481+0000","last_clean":"2024-09-13T18:21:27.143481+0000","last_became_active":"2024-09-13T18:20:27.186765+0000","last_became_peered":"2024-09-13T18:20:27.186765+0000","last_unstale":"2024-09-13T18:21:27.143481+0000","last_undegraded":"2024-09-13T18:21:27.143481+0000","last_fullsized":"2024-09-13T18:21:27.143481+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:37:01.469560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.338630+0000","last_change":"2024-09-13T18:20:01.924074+0000","last_active":"2024-09-13T18:21:27.338630+0000","last_peered":"2024-09-13T18:21:27.338630+0000","last_clean":"2024-09-13T18:21:27.338630+0000","last_became_active":"2024-09-13T18:20:01.922364+0000","last_became_peered":"2024-09-13T18:20:01.922364+0000","last_unstale":"2024-09-13T18:21:27.338630+0000","last_undegraded":"2024-09-13T18:21:27.338630+0000","last_fullsized":"2024-09-13T18:21:27.338630+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:51:33.824362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.140169+0000","last_change":"2024-09-13T18:20:01.914606+0000","last_active":"2024-09-13T18:21:27.140169+0000","last_peered":"2024-09-13T18:21:27.140169+0000","last_clean":"2024-09-13T18:21:27.140169+0000","last_became_active":"2024-09-13T18:20:01.914281+0000","last_became_peered":"2024-09-13T18:20:01.914281+0000","last_unstale":"2024-09-13T18:21:27.140169+0000","last_undegraded":"2024-09-13T18:21:27.140169+0000","last_fullsized":"2024-09-13T18:21:27.140169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:55:42.211959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.144515+0000","last_change":"2024-09-13T18:20:01.928876+0000","last_active":"2024-09-13T18:21:27.144515+0000","last_peered":"2024-09-13T18:21:27.144515+0000","last_clean":"2024-09-13T18:21:27.144515+0000","last_became_active":"2024-09-13T18:20:01.928534+0000","last_became_peered":"2024-09-13T18:20:01.928534+0000","last_unstale":"2024-09-13T18:21:27.144515+0000","last_undegraded":"2024-09-13T18:21:27.144515+0000","last_fullsized":"2024-09-13T18:21:27.144515+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:14:28.196746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.321912+0000","last_change":"2024-09-13T18:20:01.920642+0000","last_active":"2024-09-13T18:21:27.321912+0000","last_peered":"2024-09-13T18:21:27.321912+0000","last_clean":"2024-09-13T18:21:27.321912+0000","last_became_active":"2024-09-13T18:20:01.920335+0000","last_became_peered":"2024-09-13T18:20:01.920335+0000","last_unstale":"2024-09-13T18:21:27.321912+0000","last_undegraded":"2024-09-13T18:21:27.321912+0000","last_fullsized":"2024-09-13T18:21:27.321912+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:59:20.894563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.146999+0000","last_change":"2024-09-13T18:20:27.187529+0000","last_active":"2024-09-13T18:21:27.146999+0000","last_peered":"2024-09-13T18:21:27.146999+0000","last_clean":"2024-09-13T18:21:27.146999+0000","last_became_active":"2024-09-13T18:20:27.187123+0000","last_became_peered":"2024-09-13T18:20:27.187123+0000","last_unstale":"2024-09-13T18:21:27.146999+0000","last_undegraded":"2024-09-13T18:21:27.146999+0000","last_fullsized":"2024-09-13T18:21:27.146999+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:43:19.601235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.147000+0000","last_change":"2024-09-13T18:20:01.917111+0000","last_active":"2024-09-13T18:21:27.147000+0000","last_peered":"2024-09-13T18:21:27.147000+0000","last_clean":"2024-09-13T18:21:27.147000+0000","last_became_active":"2024-09-13T18:20:01.916826+0000","last_became_peered":"2024-09-13T18:20:01.916826+0000","last_unstale":"2024-09-13T18:21:27.147000+0000","last_undegraded":"2024-09-13T18:21:27.147000+0000","last_fullsized":"2024-09-13T18:21:27.147000+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:36:08.150028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.143951+0000","last_change":"2024-09-13T18:20:01.904496+0000","last_active":"2024-09-13T18:21:27.143951+0000","last_peered":"2024-09-13T18:21:27.143951+0000","last_clean":"2024-09-13T18:21:27.143951+0000","last_became_active":"2024-09-13T18:20:01.904199+0000","last_became_peered":"2024-09-13T18:20:01.904199+0000","last_unstale":"2024-09-13T18:21:27.143951+0000","last_undegraded":"2024-09-13T18:21:27.143951+0000","last_fullsized":"2024-09-13T18:21:27.143951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:44.044162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.144107+0000","last_change":"2024-09-13T18:20:01.919023+0000","last_active":"2024-09-13T18:21:27.144107+0000","last_peered":"2024-09-13T18:21:27.144107+0000","last_clean":"2024-09-13T18:21:27.144107+0000","last_became_active":"2024-09-13T18:20:01.918661+0000","last_became_peered":"2024-09-13T18:20:01.918661+0000","last_unstale":"2024-09-13T18:21:27.144107+0000","last_undegraded":"2024-09-13T18:21:27.144107+0000","last_fullsized":"2024-09-13T18:21:27.144107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:25:26.779029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.145834+0000","last_change":"2024-09-13T18:20:01.922833+0000","last_active":"2024-09-13T18:21:27.145834+0000","last_peered":"2024-09-13T18:21:27.145834+0000","last_clean":"2024-09-13T18:21:27.145834+0000","last_became_active":"2024-09-13T18:20:01.922533+0000","last_became_peered":"2024-09-13T18:20:01.922533+0000","last_unstale":"2024-09-13T18:21:27.145834+0000","last_undegraded":"2024-09-13T18:21:27.145834+0000","last_fullsized":"2024-09-13T18:21:27.145834+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:27:11.557419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.142558+0000","last_change":"2024-09-13T18:20:01.925916+0000","last_active":"2024-09-13T18:21:27.142558+0000","last_peered":"2024-09-13T18:21:27.142558+0000","last_clean":"2024-09-13T18:21:27.142558+0000","last_became_active":"2024-09-13T18:20:01.925568+0000","last_became_peered":"2024-09-13T18:20:01.925568+0000","last_unstale":"2024-09-13T18:21:27.142558+0000","last_undegraded":"2024-09-13T18:21:27.142558+0000","last_fullsized":"2024-09-13T18:21:27.142558+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:58:24.955273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.322384+0000","last_change":"2024-09-13T18:20:01.924328+0000","last_active":"2024-09-13T18:21:27.322384+0000","last_peered":"2024-09-13T18:21:27.322384+0000","last_clean":"2024-09-13T18:21:27.322384+0000","last_became_active":"2024-09-13T18:20:01.924004+0000","last_became_peered":"2024-09-13T18:20:01.924004+0000","last_unstale":"2024-09-13T18:21:27.322384+0000","last_undegraded":"2024-09-13T18:21:27.322384+0000","last_fullsized":"2024-09-13T18:21:27.322384+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:23:29.204157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.730492+0000","last_change":"2024-09-13T18:20:01.922234+0000","last_active":"2024-09-13T18:21:27.730492+0000","last_peered":"2024-09-13T18:21:27.730492+0000","last_clean":"2024-09-13T18:21:27.730492+0000","last_became_active":"2024-09-13T18:20:01.921196+0000","last_became_peered":"2024-09-13T18:20:01.921196+0000","last_unstale":"2024-09-13T18:21:27.730492+0000","last_undegraded":"2024-09-13T18:21:27.730492+0000","last_fullsized":"2024-09-13T18:21:27.730492+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:24:10.144346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.145467+0000","last_change":"2024-09-13T18:20:01.921740+0000","last_active":"2024-09-13T18:21:27.145467+0000","last_peered":"2024-09-13T18:21:27.145467+0000","last_clean":"2024-09-13T18:21:27.145467+0000","last_became_active":"2024-09-13T18:20:01.919634+0000","last_became_peered":"2024-09-13T18:20:01.919634+0000","last_unstale":"2024-09-13T18:21:27.145467+0000","last_undegraded":"2024-09-13T18:21:27.145467+0000","last_fullsized":"2024-09-13T18:21:27.145467+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:43:27.600637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.141401+0000","last_change":"2024-09-13T18:20:01.924806+0000","last_active":"2024-09-13T18:21:27.141401+0000","last_peered":"2024-09-13T18:21:27.141401+0000","last_clean":"2024-09-13T18:21:27.141401+0000","last_became_active":"2024-09-13T18:20:01.924514+0000","last_became_peered":"2024-09-13T18:20:01.924514+0000","last_unstale":"2024-09-13T18:21:27.141401+0000","last_undegraded":"2024-09-13T18:21:27.141401+0000","last_fullsized":"2024-09-13T18:21:27.141401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:47.323362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.141446+0000","last_change":"2024-09-13T18:20:01.912402+0000","last_active":"2024-09-13T18:21:27.141446+0000","last_peered":"2024-09-13T18:21:27.141446+0000","last_clean":"2024-09-13T18:21:27.141446+0000","last_became_active":"2024-09-13T18:20:01.911534+0000","last_became_peered":"2024-09-13T18:20:01.911534+0000","last_unstale":"2024-09-13T18:21:27.141446+0000","last_undegraded":"2024-09-13T18:21:27.141446+0000","last_fullsized":"2024-09-13T18:21:27.141446+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:22:52.021688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.143978+0000","last_change":"2024-09-13T18:20:01.930248+0000","last_active":"2024-09-13T18:21:27.143978+0000","last_peered":"2024-09-13T18:21:27.143978+0000","last_clean":"2024-09-13T18:21:27.143978+0000","last_became_active":"2024-09-13T18:20:01.929931+0000","last_became_peered":"2024-09-13T18:20:01.929931+0000","last_unstale":"2024-09-13T18:21:27.143978+0000","last_undegraded":"2024-09-13T18:21:27.143978+0000","last_fullsized":"2024-09-13T18:21:27.143978+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:37:33.086722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.147445+0000","last_change":"2024-09-13T18:20:01.921913+0000","last_active":"2024-09-13T18:21:27.147445+0000","last_peered":"2024-09-13T18:21:27.147445+0000","last_clean":"2024-09-13T18:21:27.147445+0000","last_became_active":"2024-09-13T18:20:01.921395+0000","last_became_peered":"2024-09-13T18:20:01.921395+0000","last_unstale":"2024-09-13T18:21:27.147445+0000","last_undegraded":"2024-09-13T18:21:27.147445+0000","last_fullsized":"2024-09-13T18:21:27.147445+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:54:50.816870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.147312+0000","last_change":"2024-09-13T18:20:01.912654+0000","last_active":"2024-09-13T18:21:27.147312+0000","last_peered":"2024-09-13T18:21:27.147312+0000","last_clean":"2024-09-13T18:21:27.147312+0000","last_became_active":"2024-09-13T18:20:01.912356+0000","last_became_peered":"2024-09-13T18:20:01.912356+0000","last_unstale":"2024-09-13T18:21:27.147312+0000","last_undegraded":"2024-09-13T18:21:27.147312+0000","last_fullsized":"2024-09-13T18:21:27.147312+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:25:18.099396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.144431+0000","last_change":"2024-09-13T18:20:01.923373+0000","last_active":"2024-09-13T18:21:27.144431+0000","last_peered":"2024-09-13T18:21:27.144431+0000","last_clean":"2024-09-13T18:21:27.144431+0000","last_became_active":"2024-09-13T18:20:01.923077+0000","last_became_peered":"2024-09-13T18:20:01.923077+0000","last_unstale":"2024-09-13T18:21:27.144431+0000","last_undegraded":"2024-09-13T18:21:27.144431+0000","last_fullsized":"2024-09-13T18:21:27.144431+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:07:52.955705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69540,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":805,"num_read_kb":476,"num_write":233,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":208620,"data_compressed":4824,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281684,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383714,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":1032192,"data_stored":519328,"data_compressed":23512,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:21:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.495,"15min":0.48199999999999998},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.47599999999999998,"15min":0.46500000000000002},"min":{"1min":0.318,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.70099999999999996}]},{"osd":1,"last update":"Fri Sep 13 18:21:14 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.57699999999999996,"15min":0.56899999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.86499999999999999,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56699999999999995,"15min":0.56499999999999995},"min":{"1min":0.35599999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.75,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.74399999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:21:14 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.34499999999999997,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.63700000000000001,"5min":1.238,"15min":1.238},"last":0.628},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.44,"15min":0.432},"min":{"1min":0.33300000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.71499999999999997,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.43099999999999999}]},{"osd":3,"last update":"Fri Sep 13 18:21:27 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.81599999999999995}]},{"osd":4,"last update":"Fri Sep 13 18:21:14 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.61299999999999999,"15min":0.60399999999999998},"min":{"1min":0.43099999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.79200000000000004,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56799999999999995,"15min":0.56200000000000006},"min":{"1min":0.44500000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.85299999999999998,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.59599999999999997}]},{"osd":5,"last update":"Fri Sep 13 18:21:14 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.504,"15min":0.501},"min":{"1min":0.36299999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.75700000000000001,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.52400000000000002,"15min":0.51400000000000001},"min":{"1min":0.42299999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.55500000000000005}]},{"osd":6,"last update":"Fri Sep 13 18:21:14 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.57499999999999996,"15min":0.56799999999999995},"min":{"1min":0.41899999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78900000000000003,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.59899999999999998,"15min":0.59299999999999997},"min":{"1min":0.499,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.81499999999999995,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.72799999999999998}]},{"osd":7,"last update":"Fri Sep 13 18:21:14 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.65300000000000002,"15min":0.63800000000000001},"min":{"1min":0.54400000000000004,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.92800000000000005,"5min":1.123,"15min":1.123},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.65000000000000002,"15min":0.64500000000000002},"min":{"1min":0.46800000000000003,"5min":0.439,"15min":0.439},"max":{"1min":0.90200000000000002,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.79200000000000004}]},{"osd":8,"last update":"Fri Sep 13 18:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.88700000000000001}]},{"osd":9,"last update":"Fri Sep 13 18:21:14 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.69099999999999995,"15min":0.68500000000000005},"min":{"1min":0.57799999999999996,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.88,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.69399999999999995,"15min":0.68000000000000005},"min":{"1min":0.54700000000000004,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.93400000000000005,"5min":1.208,"15min":1.208},"last":0.90600000000000003}]},{"osd":10,"last update":"Fri Sep 13 18:21:14 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.68100000000000005,"15min":0.67300000000000004},"min":{"1min":0.55400000000000005,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.94399999999999995,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.70799999999999996,"15min":0.69999999999999996},"min":{"1min":0.622,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.95699999999999996,"5min":1.583,"15min":1.583},"last":0.84399999999999997}]}]},{"osd":10,"up_from":72,"seq":309237645351,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28172,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716980,"statfs":{"total":95995035648,"available":95966187520,"internally_reserved":0,"allocated":1175552,"data_stored":1173197,"data_compressed":60944,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:21:01 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.40300000000000002},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":14.589,"5min":14.589,"15min":14.589},"last":0.35799999999999998}]},{"osd":1,"last update":"Fri Sep 13 18:20:53 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.42399999999999999,"15min":0.42299999999999999},"min":{"1min":0.28699999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.318},{"interface":"front","average":{"1min":0.432,"5min":0.41899999999999998,"15min":0.41699999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.372}]},{"osd":2,"last update":"Fri Sep 13 18:20:53 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.54600000000000004,"15min":0.54300000000000004},"min":{"1min":0.39800000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.78400000000000003,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.55900000000000005,"15min":0.55300000000000005},"min":{"1min":0.436,"5min":0.374,"15min":0.374},"max":{"1min":0.77200000000000002,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.50600000000000001}]},{"osd":3,"last update":"Fri Sep 13 18:20:53 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57199999999999995,"15min":0.57599999999999996},"min":{"1min":0.378,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72799999999999998,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.44400000000000001},{"interface":"front","average":{"1min":1.131,"5min":0.68700000000000006,"15min":0.61299999999999999},"min":{"1min":0.40999999999999998,"5min":0.376,"15min":0.376},"max":{"1min":14.505000000000001,"5min":14.505000000000001,"15min":14.505000000000001},"last":0.55000000000000004}]},{"osd":4,"last update":"Fri Sep 13 18:20:53 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.50800000000000001,"15min":0.51000000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.73499999999999999,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.5,"15min":0.5},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.68300000000000005,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.47199999999999998}]},{"osd":5,"last update":"Fri Sep 13 18:20:53 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.60299999999999998,"15min":0.59999999999999998},"min":{"1min":0.432,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.79700000000000004,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.627,"15min":0.626},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.88200000000000001,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.58399999999999996}]},{"osd":6,"last update":"Fri Sep 13 18:20:53 2024","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":0.75700000000000001,"15min":0.67900000000000005},"min":{"1min":0.41599999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":14.539999999999999,"5min":14.539999999999999,"15min":14.539999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.64600000000000002,"15min":0.64800000000000002},"min":{"1min":0.46200000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.60099999999999998}]},{"osd":7,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.81599999999999995}]},{"osd":8,"last update":"Fri Sep 13 18:20:53 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.67100000000000004,"15min":0.67300000000000004},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.80200000000000005,"5min":1.01,"15min":1.01},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.69299999999999995,"15min":0.69499999999999995},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.86499999999999999,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.61799999999999999}]},{"osd":9,"last update":"Fri Sep 13 18:20:53 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.68799999999999994,"15min":0.69199999999999995},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.83199999999999996,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.71499999999999997,"15min":0.71799999999999997},"min":{"1min":0.53200000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89300000000000002,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.64900000000000002}]},{"osd":11,"last update":"Fri Sep 13 18:21:13 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.72699999999999998,"15min":0.73099999999999998},"min":{"1min":0.58899999999999997,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.89200000000000002,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.73699999999999999,"15min":0.73999999999999999},"min":{"1min":0.58999999999999997,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.872,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.67700000000000005}]}]},{"osd":9,"up_from":67,"seq":287762808876,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":519284,"data_compressed":23512,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:21:33 2024","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.36099999999999999,"15min":0.38100000000000001},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.45800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.41399999999999998,"15min":0.442},"min":{"1min":0.249,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.495,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.376}]},{"osd":1,"last update":"Fri Sep 13 18:21:33 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.56299999999999994,"15min":0.58399999999999996},"min":{"1min":0.35699999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.67300000000000004,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.496,"5min":0.56399999999999995,"15min":0.58499999999999996},"min":{"1min":0.36399999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.67700000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.44800000000000001}]},{"osd":2,"last update":"Fri Sep 13 18:21:33 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.55000000000000004,"15min":0.57399999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.65000000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.52600000000000002,"15min":0.52800000000000002},"min":{"1min":0.34899999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.54100000000000004}]},{"osd":3,"last update":"Fri Sep 13 18:21:33 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.44700000000000001,"15min":0.48099999999999998},"min":{"1min":0.28499999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.50900000000000001,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.37,"5min":0.437,"15min":0.46300000000000002},"min":{"1min":0.28000000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.496,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.46100000000000002}]},{"osd":4,"last update":"Fri Sep 13 18:21:33 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.55600000000000005,"15min":0.57299999999999995},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.67700000000000005,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.503},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.61299999999999999,"15min":0.64700000000000002},"min":{"1min":0.39000000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.65300000000000002,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.55600000000000005}]},{"osd":5,"last update":"Fri Sep 13 18:21:33 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.57199999999999995,"15min":0.59099999999999997},"min":{"1min":0.36599999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.59299999999999997,"15min":0.61099999999999999},"min":{"1min":0.40600000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.752,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.60499999999999998}]},{"osd":6,"last update":"Fri Sep 13 18:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.39800000000000002}]},{"osd":7,"last update":"Fri Sep 13 18:21:33 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.63600000000000001,"15min":0.67400000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.70899999999999996,"5min":0.871,"15min":0.871},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.64700000000000002,"15min":0.68100000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.63900000000000001,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.57199999999999995}]},{"osd":8,"last update":"Fri Sep 13 18:21:33 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.63500000000000001,"15min":0.66500000000000004},"min":{"1min":0.40999999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.75,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.66500000000000004,"15min":0.69999999999999996},"min":{"1min":0.436,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.81899999999999995,"5min":1.02,"15min":1.02},"last":0.67800000000000005}]},{"osd":10,"last update":"Fri Sep 13 18:20:51 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.65300000000000002,"15min":0.66600000000000004},"min":{"1min":0.47899999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.70299999999999996,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.67700000000000005,"15min":0.68899999999999995},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.75800000000000001,"5min":0.879,"15min":0.879},"last":0.64400000000000002}]},{"osd":11,"last update":"Fri Sep 13 18:21:08 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.63300000000000001}]}]},{"osd":8,"up_from":59,"seq":253403070513,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":1044480,"data_stored":519640,"data_compressed":23512,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:21:07 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.52800000000000002,"15min":0.54500000000000004},"min":{"1min":0.38500000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.76000000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.432},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.56200000000000006,"15min":0.56599999999999995},"min":{"1min":0.33400000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":3.2989999999999999,"5min":3.2989999999999999,"15min":3.2989999999999999},"last":0.58399999999999996}]},{"osd":1,"last update":"Fri Sep 13 18:21:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56599999999999995,"15min":0.56100000000000005},"min":{"1min":0.41899999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57499999999999996,"15min":0.57299999999999995},"min":{"1min":0.44400000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.72699999999999998}]},{"osd":2,"last update":"Fri Sep 13 18:21:07 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.42899999999999999,"15min":0.43099999999999999},"min":{"1min":0.29599999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.438,"15min":0.437},"min":{"1min":0.27600000000000002,"5min":0.224,"15min":0.224},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.50800000000000001}]},{"osd":3,"last update":"Fri Sep 13 18:21:07 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54400000000000004,"15min":0.55700000000000005},"min":{"1min":0.377,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53800000000000003,"15min":0.53900000000000003},"min":{"1min":0.437,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.41299999999999998}]},{"osd":4,"last update":"Fri Sep 13 18:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.66400000000000003,"15min":0.63600000000000001},"min":{"1min":0.45100000000000001,"5min":0.37,"15min":0.37},"max":{"1min":5.3540000000000001,"5min":5.3540000000000001,"15min":5.3540000000000001},"last":0.68000000000000005}]},{"osd":5,"last update":"Fri Sep 13 18:21:07 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.52600000000000002,"15min":0.53600000000000003},"min":{"1min":0.32600000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.376},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.52400000000000002,"15min":0.52500000000000002},"min":{"1min":0.40100000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.46800000000000003}]},{"osd":6,"last update":"Fri Sep 13 18:21:07 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.629,"15min":0.62},"min":{"1min":0.436,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":3.2719999999999998,"5min":3.2719999999999998,"15min":3.2719999999999998},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.63400000000000001,"15min":0.64600000000000002},"min":{"1min":0.42899999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.85199999999999998,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.56699999999999995}]},{"osd":7,"last update":"Fri Sep 13 18:21:07 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.66000000000000003,"15min":0.67300000000000004},"min":{"1min":0.48099999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89300000000000002,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.65800000000000003,"15min":0.66400000000000003},"min":{"1min":0.504,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.71799999999999997}]},{"osd":9,"last update":"Fri Sep 13 18:21:25 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63800000000000001,"15min":0.63300000000000001},"min":{"1min":0.48299999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.79500000000000004,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.64600000000000002,"15min":0.65200000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.78300000000000003,"5min":1.012,"15min":1.012},"last":0.52800000000000002}]},{"osd":10,"last update":"Fri Sep 13 18:20:51 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.91600000000000004,"15min":0.95399999999999996},"min":{"1min":0.53800000000000003,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.92600000000000005,"5min":5.3360000000000003,"15min":5.3360000000000003},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.68000000000000005,"15min":0.67500000000000004},"min":{"1min":0.55300000000000005,"5min":0.498,"15min":0.498},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.66200000000000003}]},{"osd":11,"last update":"Fri Sep 13 18:21:02 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.75700000000000001}]}]},{"osd":7,"up_from":51,"seq":219043332149,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1101824,"data_stored":1115624,"data_compressed":59336,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.94399999999999995,"15min":1.012},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.84299999999999997,"5min":12.068,"15min":12.068},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.46999999999999997,"15min":0.45900000000000002},"min":{"1min":0.35199999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.61599999999999999}]},{"osd":1,"last update":"Fri Sep 13 18:20:45 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.376,"15min":0.35999999999999999},"min":{"1min":0.29099999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.51900000000000002,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.435,"5min":0.39400000000000002,"15min":0.379},"min":{"1min":0.27500000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.56899999999999995,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.375}]},{"osd":2,"last update":"Fri Sep 13 18:20:45 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60899999999999999,"15min":0.54300000000000004},"min":{"1min":0.46700000000000003,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.88700000000000001,"5min":7.6820000000000004,"15min":7.6820000000000004},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.52800000000000002,"15min":0.51100000000000001},"min":{"1min":0.47999999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.65800000000000003,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.48599999999999999}]},{"osd":3,"last update":"Fri Sep 13 18:20:45 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.51700000000000002,"15min":0.48899999999999999},"min":{"1min":0.42199999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80500000000000005,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.51200000000000001,"15min":0.48999999999999999},"min":{"1min":0.376,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.66400000000000003,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.46899999999999997}]},{"osd":4,"last update":"Fri Sep 13 18:20:45 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.45800000000000002,"15min":0.439},"min":{"1min":0.35399999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.505,"5min":0.46600000000000003,"15min":0.45400000000000001},"min":{"1min":0.33300000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.50900000000000001}]},{"osd":5,"last update":"Fri Sep 13 18:20:45 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.57699999999999996,"15min":0.55700000000000005},"min":{"1min":0.51700000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.55400000000000005,"15min":0.53600000000000003},"min":{"1min":0.46899999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73499999999999999,"5min":0.754,"15min":0.754},"last":0.61399999999999999}]},{"osd":6,"last update":"Fri Sep 13 18:20:45 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.56799999999999995,"15min":0.54600000000000004},"min":{"1min":0.434,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.60899999999999999,"15min":0.58699999999999997},"min":{"1min":0.432,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.81999999999999995,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.60399999999999998}]},{"osd":8,"last update":"Fri Sep 13 18:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.60299999999999998,"15min":0.59099999999999997},"min":{"1min":0.498,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.73999999999999999,"15min":0.66800000000000004},"min":{"1min":0.61699999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.93300000000000005,"5min":7.7149999999999999,"15min":7.7149999999999999},"last":0.65500000000000003}]},{"osd":9,"last update":"Fri Sep 13 18:21:29 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.66300000000000003,"15min":0.66100000000000003},"min":{"1min":0.40500000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.88,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.625},{"interface":"front","average":{"1min":1.609,"5min":0.82999999999999996,"15min":0.70199999999999996},"min":{"1min":0.42699999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":20.056999999999999,"5min":20.056999999999999,"15min":20.056999999999999},"last":0.57399999999999995}]},{"osd":10,"last update":"Fri Sep 13 18:20:51 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.66300000000000003,"15min":0.65700000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.85699999999999998,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.65000000000000002,"15min":0.64600000000000002},"min":{"1min":0.53100000000000003,"5min":0.434,"15min":0.434},"max":{"1min":0.81699999999999995,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.64300000000000002}]},{"osd":11,"last update":"Fri Sep 13 18:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.69199999999999995}]}]},{"osd":6,"up_from":46,"seq":197568495674,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":1028096,"data_stored":514740,"data_compressed":23512,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:21:21 2024","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.35999999999999999,"15min":0.36399999999999999},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.48599999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.374,"5min":0.41699999999999998,"15min":0.433},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.60299999999999998,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.43099999999999999}]},{"osd":1,"last update":"Fri Sep 13 18:21:21 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.53900000000000003,"15min":0.52600000000000002},"min":{"1min":0.40999999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.72199999999999998,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.55200000000000005,"15min":0.53700000000000003},"min":{"1min":0.40600000000000003,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.68600000000000005,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.53200000000000003}]},{"osd":2,"last update":"Fri Sep 13 18:21:21 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.37,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.69899999999999995,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.54000000000000004,"15min":0.54800000000000004},"min":{"1min":0.40600000000000003,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.71299999999999997,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.56000000000000005}]},{"osd":3,"last update":"Fri Sep 13 18:21:21 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.41099999999999998,"15min":0.42199999999999999},"min":{"1min":0.29099999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.51200000000000001,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.45700000000000002,"15min":0.46800000000000003},"min":{"1min":0.314,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.65600000000000003,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.48199999999999998}]},{"osd":4,"last update":"Fri Sep 13 18:21:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.52900000000000003,"15min":0.51700000000000002},"min":{"1min":0.42699999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.68000000000000005,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55800000000000005,"15min":0.55400000000000005},"min":{"1min":0.48299999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.68300000000000005,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.67200000000000004}]},{"osd":5,"last update":"Fri Sep 13 18:21:21 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59499999999999997,"15min":0.60699999999999998},"min":{"1min":0.47099999999999997,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.752,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56799999999999995,"15min":0.59099999999999997},"min":{"1min":0.41999999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.73999999999999999,"5min":0.874,"15min":0.874},"last":0.64100000000000001}]},{"osd":7,"last update":"Fri Sep 13 18:20:44 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.39100000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.76400000000000001,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.503},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56599999999999995,"15min":0.56200000000000006},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.58499999999999996}]},{"osd":8,"last update":"Fri Sep 13 18:21:08 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.621,"15min":0.622},"min":{"1min":0.51300000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.59599999999999997,"15min":0.59999999999999998},"min":{"1min":0.47799999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.62}]},{"osd":9,"last update":"Fri Sep 13 18:21:34 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.51500000000000001,"15min":0.53100000000000003},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.57199999999999995,"5min":0.871,"15min":0.871},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.59099999999999997,"15min":0.60499999999999998},"min":{"1min":0.41399999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.75700000000000001,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.60699999999999998}]},{"osd":10,"last update":"Fri Sep 13 18:20:54 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.61499999999999999,"15min":0.61799999999999999},"min":{"1min":0.51100000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.56200000000000006,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.68700000000000006}]},{"osd":11,"last update":"Fri Sep 13 18:21:08 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.71799999999999997}]}]},{"osd":5,"up_from":38,"seq":163208757311,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1110016,"data_stored":1111302,"data_compressed":59336,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:20:59 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.499,"15min":0.496},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.69999999999999996,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.51800000000000002,"15min":0.51300000000000001},"min":{"1min":0.34100000000000003,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.77400000000000002,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.53000000000000003}]},{"osd":1,"last update":"Fri Sep 13 18:20:59 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.53100000000000003,"15min":0.50700000000000001},"min":{"1min":0.378,"5min":0.308,"15min":0.308},"max":{"1min":0.81200000000000006,"5min":0.878,"15min":0.878},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.52500000000000002,"15min":0.51100000000000001},"min":{"1min":0.379,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.71599999999999997,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.51800000000000002}]},{"osd":2,"last update":"Fri Sep 13 18:20:59 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.38500000000000001,"15min":0.36499999999999999},"min":{"1min":0.311,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.377},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.39100000000000001,"15min":0.38500000000000001},"min":{"1min":0.30599999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.52200000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.316}]},{"osd":3,"last update":"Fri Sep 13 18:20:59 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.52500000000000002,"15min":0.504},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.73999999999999999,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.40500000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.70299999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.48599999999999999}]},{"osd":4,"last update":"Fri Sep 13 18:20:59 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55300000000000005,"15min":0.54000000000000004},"min":{"1min":0.432,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.76600000000000001,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.436},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.77800000000000002,"15min":0.61299999999999999},"min":{"1min":0.44,"5min":0.374,"15min":0.374},"max":{"1min":1.002,"5min":18.571000000000002,"15min":18.571000000000002},"last":0.41799999999999998}]},{"osd":6,"last update":"Fri Sep 13 18:21:24 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.40600000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.5},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55600000000000005,"15min":0.55100000000000005},"min":{"1min":0.378,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.78000000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.54600000000000004}]},{"osd":7,"last update":"Fri Sep 13 18:21:45 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61599999999999999,"15min":0.60799999999999998},"min":{"1min":0.46300000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76400000000000001,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63100000000000001,"15min":0.61599999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.84999999999999998,"5min":0.88,"15min":0.88},"last":0.59399999999999997}]},{"osd":8,"last update":"Fri Sep 13 18:21:02 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.51300000000000001,"15min":0.51100000000000001},"min":{"1min":0.372,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.69999999999999996,"5min":0.753,"15min":0.753},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.54800000000000004,"15min":0.53800000000000003},"min":{"1min":0.39100000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.69099999999999995,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.57899999999999996}]},{"osd":9,"last update":"Fri Sep 13 18:21:30 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.64500000000000002,"15min":0.627},"min":{"1min":0.48199999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.63900000000000001,"15min":0.628},"min":{"1min":0.48799999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.81299999999999994,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.622}]},{"osd":10,"last update":"Fri Sep 13 18:20:50 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.58999999999999997,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.84099999999999997,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.67500000000000004,"15min":0.66900000000000004},"min":{"1min":0.55700000000000005,"5min":0.37,"15min":0.37},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.64200000000000002}]}]},{"osd":4,"up_from":33,"seq":141733920835,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":518814,"data_compressed":23512,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:19:33 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.48299999999999998,"15min":0.47699999999999998},"min":{"1min":0.35399999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.69399999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.433},{"interface":"front","average":{"1min":0.501,"5min":0.51200000000000001,"15min":0.47499999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.84599999999999997,"5min":4.2539999999999996,"15min":4.2539999999999996},"last":0.51900000000000002}]},{"osd":1,"last update":"Fri Sep 13 18:21:38 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.373,"15min":0.35199999999999998},"min":{"1min":0.25800000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.59199999999999997,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.379,"15min":0.35399999999999998},"min":{"1min":0.29199999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.68400000000000005,"5min":0.748,"15min":0.748},"last":0.40000000000000002}]},{"osd":2,"last update":"Fri Sep 13 18:21:38 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.60699999999999998,"15min":0.54900000000000004},"min":{"1min":0.40999999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":5.2930000000000001,"5min":5.2930000000000001,"15min":5.2930000000000001},"last":0.628},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55300000000000005,"15min":0.53800000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72499999999999998,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.61299999999999999}]},{"osd":3,"last update":"Fri Sep 13 18:21:38 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56000000000000005,"15min":0.505},"min":{"1min":0.434,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.82199999999999995,"5min":4.2679999999999998,"15min":4.2679999999999998},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.52100000000000002,"15min":0.496},"min":{"1min":0.46100000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.57399999999999995}]},{"osd":5,"last update":"Fri Sep 13 18:20:55 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55700000000000005,"15min":0.53800000000000003},"min":{"1min":0.39800000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.73799999999999999,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.56699999999999995,"15min":0.54300000000000004},"min":{"1min":0.41999999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.80700000000000005,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.65000000000000002}]},{"osd":6,"last update":"Fri Sep 13 18:21:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.55000000000000004,"15min":0.52000000000000002},"min":{"1min":0.441,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.54900000000000004,"15min":0.51700000000000002},"min":{"1min":0.42999999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.79200000000000004,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.59599999999999997}]},{"osd":7,"last update":"Fri Sep 13 18:20:45 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.45300000000000001,"15min":0.441},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.68999999999999995,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.497,"15min":0.48099999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.752,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.71699999999999997}]},{"osd":8,"last update":"Fri Sep 13 18:21:07 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.63500000000000001,"15min":0.627},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.82199999999999995,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.66000000000000003,"15min":0.64800000000000002},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.83099999999999996,"5min":0.874,"15min":0.874},"last":0.68799999999999994}]},{"osd":9,"last update":"Fri Sep 13 18:21:28 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.63300000000000001,"15min":0.61499999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.63700000000000001,"15min":0.629},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.83299999999999996,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.66400000000000003}]},{"osd":10,"last update":"Fri Sep 13 18:20:52 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.629,"15min":0.63500000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.82599999999999996,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.66200000000000003,"15min":0.65900000000000003},"min":{"1min":0.51600000000000001,"5min":0.44,"15min":0.44},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.70899999999999996}]},{"osd":11,"last update":"Fri Sep 13 18:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.753}]}]},{"osd":3,"up_from":25,"seq":107374182471,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":514215,"data_compressed":23512,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:21:14 2024","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33600000000000002,"15min":0.309},"min":{"1min":0.24199999999999999,"5min":0.183,"15min":0.183},"max":{"1min":0.42699999999999999,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.224},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.34300000000000003,"15min":0.32900000000000001},"min":{"1min":0.251,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"last":0.24199999999999999}]},{"osd":1,"last update":"Fri Sep 13 18:21:14 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.51900000000000002,"15min":0.49099999999999999},"min":{"1min":0.36699999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.71399999999999997,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.53200000000000003,"15min":0.49399999999999999},"min":{"1min":0.40500000000000003,"5min":0.313,"15min":0.313},"max":{"1min":0.70299999999999996,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.61699999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:21:14 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.51900000000000002,"15min":0.52100000000000002},"min":{"1min":0.33000000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52700000000000002,"15min":0.502},"min":{"1min":0.34300000000000003,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.69299999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.59699999999999998}]},{"osd":4,"last update":"Fri Sep 13 18:21:38 2024","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":0.622,"15min":0.52600000000000002},"min":{"1min":0.40500000000000003,"5min":0.312,"15min":0.312},"max":{"1min":9.3149999999999995,"5min":9.3149999999999995,"15min":9.3149999999999995},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.53900000000000003,"15min":0.51900000000000002},"min":{"1min":0.39900000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.72399999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.71299999999999997}]},{"osd":5,"last update":"Fri Sep 13 18:20:55 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54300000000000004,"15min":0.53600000000000003},"min":{"1min":0.42599999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.751,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54900000000000004,"15min":0.55100000000000005},"min":{"1min":0.39600000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77100000000000002,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.54700000000000004}]},{"osd":6,"last update":"Fri Sep 13 18:21:24 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.436,"15min":0.44400000000000001},"min":{"1min":0.29099999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.61599999999999999,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.502,"5min":0.48999999999999999,"15min":0.46999999999999997},"min":{"1min":0.372,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.74199999999999999,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.35299999999999998}]},{"osd":7,"last update":"Fri Sep 13 18:20:44 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.60599999999999998,"15min":0.59799999999999998},"min":{"1min":0.505,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.80900000000000005,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57999999999999996,"15min":0.57399999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.79600000000000004,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.57299999999999995}]},{"osd":8,"last update":"Fri Sep 13 18:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.69599999999999995}]},{"osd":9,"last update":"Fri Sep 13 18:21:31 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.52500000000000002,"15min":0.54900000000000004},"min":{"1min":0.33800000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.59999999999999998,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.57099999999999995,"15min":0.58499999999999996},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.73899999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.379}]},{"osd":10,"last update":"Fri Sep 13 18:20:49 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.68100000000000005,"15min":0.68300000000000005},"min":{"1min":0.53600000000000003,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.78800000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.66100000000000003,"15min":0.66900000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.77100000000000002,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.49399999999999999}]},{"osd":11,"last update":"Fri Sep 13 18:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.65100000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411404,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1105920,"data_stored":579140,"data_compressed":25120,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:20:49 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.47599999999999998,"15min":0.47999999999999998},"min":{"1min":0.34399999999999997,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.75900000000000001,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.496,"5min":0.48499999999999999,"15min":0.47699999999999998},"min":{"1min":0.40699999999999997,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.69499999999999995,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.627}]},{"osd":1,"last update":"Fri Sep 13 18:20:49 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.51600000000000001,"15min":0.47999999999999998},"min":{"1min":0.38100000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.53100000000000003,"15min":0.47899999999999998},"min":{"1min":0.45200000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.79000000000000004}]},{"osd":3,"last update":"Fri Sep 13 18:21:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.53900000000000003,"15min":0.52000000000000002},"min":{"1min":0.42299999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.74399999999999999,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.57099999999999995,"15min":0.55700000000000005},"min":{"1min":0.442,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.82299999999999995,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.70899999999999996}]},{"osd":4,"last update":"Fri Sep 13 18:21:02 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.753},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.72199999999999998}]},{"osd":5,"last update":"Fri Sep 13 18:20:52 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.40799999999999997,"15min":0.38300000000000001},"min":{"1min":0.34200000000000003,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.42499999999999999,"15min":0.39800000000000002},"min":{"1min":0.31900000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.49399999999999999}]},{"osd":6,"last update":"Fri Sep 13 18:21:22 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.59199999999999997,"15min":0.55000000000000004},"min":{"1min":0.48099999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78700000000000003,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.60599999999999998,"15min":0.57199999999999995},"min":{"1min":0.57599999999999996,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.88900000000000001,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.74099999999999999}]},{"osd":7,"last update":"Fri Sep 13 18:21:07 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.86099999999999999}]},{"osd":8,"last update":"Fri Sep 13 18:21:07 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.55100000000000005,"15min":0.54600000000000004},"min":{"1min":0.39500000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.59599999999999997,"15min":0.58799999999999997},"min":{"1min":0.38200000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.877,"5min":0.998,"15min":0.998},"last":0.83999999999999997}]},{"osd":9,"last update":"Fri Sep 13 18:21:28 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.63700000000000001,"15min":0.625},"min":{"1min":0.50900000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.86399999999999999,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.66900000000000004,"15min":0.65500000000000003},"min":{"1min":0.50900000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.96499999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.82599999999999996}]},{"osd":10,"last update":"Fri Sep 13 18:20:49 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.64300000000000002,"15min":0.626},"min":{"1min":0.58299999999999996,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.65600000000000003,"15min":0.64200000000000002},"min":{"1min":0.57399999999999995,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.88}]},{"osd":11,"last update":"Fri Sep 13 18:21:00 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.92000000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1101824,"data_stored":579192,"data_compressed":25120,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:21:25 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.496,"15min":0.49299999999999999},"min":{"1min":0.30299999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.56399999999999995,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.437},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.52900000000000003,"15min":0.51700000000000002},"min":{"1min":0.33900000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.56799999999999995}]},{"osd":2,"last update":"Fri Sep 13 18:20:51 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.53000000000000003,"15min":0.5},"min":{"1min":0.35799999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.48899999999999999,"15min":0.47499999999999998},"min":{"1min":0.40500000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.71099999999999997}]},{"osd":3,"last update":"Fri Sep 13 18:21:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.54400000000000004,"15min":0.52200000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.69799999999999995,"5min":0.749,"15min":0.749},"last":0.496},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.57999999999999996,"15min":0.54900000000000004},"min":{"1min":0.503,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.83399999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.68500000000000005}]},{"osd":4,"last update":"Fri Sep 13 18:21:34 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.41099999999999998,"15min":0.371},"min":{"1min":0.29499999999999998,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.755,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.40500000000000003,"15min":0.36499999999999999},"min":{"1min":0.29599999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.65600000000000003,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.51500000000000001}]},{"osd":5,"last update":"Fri Sep 13 18:20:56 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.59799999999999998,"15min":0.57199999999999995},"min":{"1min":0.45900000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.58399999999999996,"15min":0.54800000000000004},"min":{"1min":0.40699999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.73499999999999999}]},{"osd":6,"last update":"Fri Sep 13 18:21:25 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58199999999999996,"15min":0.56899999999999995},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78900000000000003,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.63100000000000001,"15min":0.61399999999999999},"min":{"1min":0.41599999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.81200000000000006}]},{"osd":7,"last update":"Fri Sep 13 18:21:40 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.51700000000000002,"15min":0.495},"min":{"1min":0.312,"5min":0.246,"15min":0.246},"max":{"1min":0.74299999999999999,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.52200000000000002,"15min":0.50700000000000001},"min":{"1min":0.35499999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.82499999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.61799999999999999}]},{"osd":8,"last update":"Fri Sep 13 18:21:08 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.63,"15min":0.61799999999999999},"min":{"1min":0.46600000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.94199999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.747},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.64300000000000002,"15min":0.63400000000000001},"min":{"1min":0.56000000000000005,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.78100000000000003}]},{"osd":9,"last update":"Fri Sep 13 18:21:34 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.63600000000000001,"15min":0.64200000000000002},"min":{"1min":0.49399999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.84399999999999997,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.64000000000000001,"15min":0.64200000000000002},"min":{"1min":0.433,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.55200000000000005}]},{"osd":10,"last update":"Fri Sep 13 18:20:51 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.60999999999999999,"15min":0.59599999999999997},"min":{"1min":0.432,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.90800000000000003,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.625,"15min":0.61299999999999999},"min":{"1min":0.498,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.80300000000000005}]},{"osd":11,"last update":"Fri Sep 13 18:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.83999999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738454,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":1052672,"data_stored":516196,"data_compressed":23512,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 18:21:28 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.50700000000000001,"15min":0.498},"min":{"1min":0.36799999999999999,"5min":0.318,"15min":0.30199999999999999},"max":{"1min":0.76900000000000002,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.52100000000000002,"15min":0.50900000000000001},"min":{"1min":0.434,"5min":0.307,"15min":0.307},"max":{"1min":0.73999999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.55900000000000005}]},{"osd":2,"last update":"Fri Sep 13 18:20:47 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53000000000000003,"15min":0.52800000000000002},"min":{"1min":0.35699999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.54200000000000004,"15min":0.51500000000000001},"min":{"1min":0.36899999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.77300000000000002,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.47099999999999997}]},{"osd":3,"last update":"Fri Sep 13 18:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.29799999999999999}]},{"osd":4,"last update":"Fri Sep 13 18:21:39 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52100000000000002,"15min":0.497},"min":{"1min":0.41299999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.54100000000000004,"15min":0.51500000000000001},"min":{"1min":0.36399999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76000000000000001,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.52200000000000002}]},{"osd":5,"last update":"Fri Sep 13 18:20:56 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.61699999999999999,"15min":0.56699999999999995},"min":{"1min":0.38700000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.78800000000000003,"5min":7.194,"15min":7.194},"last":0.372},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56699999999999995,"15min":0.52900000000000003},"min":{"1min":0.42399999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.80700000000000005,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.497}]},{"osd":6,"last update":"Fri Sep 13 18:21:23 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.46500000000000002,"15min":0.45500000000000002},"min":{"1min":0.32600000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.56000000000000005,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.44500000000000001,"15min":0.45700000000000002},"min":{"1min":0.27800000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.64700000000000002,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.41499999999999998}]},{"osd":7,"last update":"Fri Sep 13 18:21:07 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.51200000000000001}]},{"osd":8,"last update":"Fri Sep 13 18:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.59699999999999998,"15min":0.60599999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79100000000000004,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.621,"15min":0.61699999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.77900000000000003,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.441}]},{"osd":9,"last update":"Fri Sep 13 18:21:33 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.58499999999999996,"15min":0.60699999999999998},"min":{"1min":0.33700000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78200000000000003,"5min":1.089,"15min":1.089},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.54100000000000004,"15min":0.57299999999999995},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.61799999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.64100000000000001}]},{"osd":10,"last update":"Fri Sep 13 18:20:49 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.67000000000000004,"15min":0.68000000000000005},"min":{"1min":0.51000000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.82899999999999996,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.69199999999999995,"15min":0.70099999999999996},"min":{"1min":0.48299999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.84899999999999998,"5min":1.048,"15min":1.048},"last":0.60799999999999998}]},{"osd":11,"last update":"Fri Sep 13 18:21:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.629,"15min":0.63300000000000001},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.76200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.68000000000000005,"15min":0.68300000000000005},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.88700000000000001,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.48499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34188,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34188,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34188,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35824,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35824,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35824,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35824,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35824,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":65126,"data_compressed":1608,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":65074,"data_compressed":1608,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":64998,"data_compressed":1608,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:21:49.625 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-13T18:21:49.625 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-13T18:21:49.625 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-13T18:21:49.625 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph health --format=json 2024-09-13T18:21:49.830 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:21:50.219 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:49 smithi114 ceph-mon[30897]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T18:21:50.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:49 smithi134 ceph-mon[34283]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T18:21:50.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:49 smithi204 ceph-mon[34777]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T18:21:50.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:50 smithi114 ceph-mon[30897]: pgmap v335: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-13T18:21:51.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:50 smithi134 ceph-mon[34283]: pgmap v335: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-13T18:21:51.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:50 smithi204 ceph-mon[34777]: pgmap v335: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-13T18:21:51.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.546+0000 7f7cb6167640 1 -- 172.21.15.114:0/3941914936 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cb0101880 msgr2=0x7f7cb0109d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:51.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.546+0000 7f7cb6167640 1 --2- 172.21.15.114:0/3941914936 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cb0101880 0x7f7cb0109d70 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7c98009900 tx=0x7f7c9802a410 comp rx=0 tx=0).stop 2024-09-13T18:21:51.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.546+0000 7f7cb6167640 1 -- 172.21.15.114:0/3941914936 shutdown_connections 2024-09-13T18:21:51.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.546+0000 7f7cb6167640 1 --2- 172.21.15.114:0/3941914936 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7cb010a3e0 0x7f7cb010c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:51.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.546+0000 7f7cb6167640 1 --2- 172.21.15.114:0/3941914936 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cb0101880 0x7f7cb0109d70 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:51.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.546+0000 7f7cb6167640 1 --2- 172.21.15.114:0/3941914936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7cb0100f60 0x7f7cb0101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:51.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.546+0000 7f7cb6167640 1 -- 172.21.15.114:0/3941914936 >> 172.21.15.114:0/3941914936 conn(0x7f7cb00fa9a0 msgr2=0x7f7cb00fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:51.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.546+0000 7f7cb6167640 1 -- 172.21.15.114:0/3941914936 shutdown_connections 2024-09-13T18:21:51.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.546+0000 7f7cb6167640 1 -- 172.21.15.114:0/3941914936 wait complete. 2024-09-13T18:21:51.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.546+0000 7f7cb6167640 1 Processor -- start 2024-09-13T18:21:51.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.546+0000 7f7cb6167640 1 -- start start 2024-09-13T18:21:51.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.547+0000 7f7cb6167640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7cb0100f60 0x7f7cb0196210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:51.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.547+0000 7f7cb6167640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cb0101880 0x7f7cb0196750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:51.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.547+0000 7f7cb6167640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7cb010a3e0 0x7f7cb019ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:51.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.547+0000 7f7cb6167640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb019b150 con 0x7f7cb0101880 2024-09-13T18:21:51.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.547+0000 7f7cb6167640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb019b2c0 con 0x7f7cb0100f60 2024-09-13T18:21:51.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.547+0000 7f7cb6167640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb019b430 con 0x7f7cb010a3e0 2024-09-13T18:21:51.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.547+0000 7f7ca6ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cb0101880 0x7f7cb0196750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:51.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.547+0000 7f7ca6ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cb0101880 0x7f7cb0196750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47304/0 (socket says 172.21.15.114:47304) 2024-09-13T18:21:51.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.547+0000 7f7ca6ffd640 1 -- 172.21.15.114:0/460484749 learned_addr learned my addr 172.21.15.114:0/460484749 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:51.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.547+0000 7f7caf7fe640 1 --2- 172.21.15.114:0/460484749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7cb0100f60 0x7f7cb0196210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:51.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.547+0000 7f7caffff640 1 --2- 172.21.15.114:0/460484749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7cb010a3e0 0x7f7cb019ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:51.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.547+0000 7f7ca6ffd640 1 -- 172.21.15.114:0/460484749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7cb010a3e0 msgr2=0x7f7cb019ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:51.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.547+0000 7f7ca6ffd640 1 --2- 172.21.15.114:0/460484749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7cb010a3e0 0x7f7cb019ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:51.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.547+0000 7f7ca6ffd640 1 -- 172.21.15.114:0/460484749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7cb0100f60 msgr2=0x7f7cb0196210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:51.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.547+0000 7f7ca6ffd640 1 --2- 172.21.15.114:0/460484749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7cb0100f60 0x7f7cb0196210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:51.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.547+0000 7f7ca6ffd640 1 -- 172.21.15.114:0/460484749 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c98009560 con 0x7f7cb0101880 2024-09-13T18:21:51.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.547+0000 7f7caffff640 1 --2- 172.21.15.114:0/460484749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7cb010a3e0 0x7f7cb019ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:21:51.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.547+0000 7f7ca6ffd640 1 --2- 172.21.15.114:0/460484749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cb0101880 0x7f7cb0196750 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7c98005ab0 tx=0x7f7c98032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:51.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.548+0000 7f7cad7fa640 1 -- 172.21.15.114:0/460484749 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c98032e10 con 0x7f7cb0101880 2024-09-13T18:21:51.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.548+0000 7f7cad7fa640 1 -- 172.21.15.114:0/460484749 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7c980046c0 con 0x7f7cb0101880 2024-09-13T18:21:51.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.548+0000 7f7cb6167640 1 -- 172.21.15.114:0/460484749 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cb019b6b0 con 0x7f7cb0101880 2024-09-13T18:21:51.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.548+0000 7f7cad7fa640 1 -- 172.21.15.114:0/460484749 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c98003850 con 0x7f7cb0101880 2024-09-13T18:21:51.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.548+0000 7f7cb6167640 1 -- 172.21.15.114:0/460484749 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cb01a2e00 con 0x7f7cb0101880 2024-09-13T18:21:51.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.549+0000 7f7cb6167640 1 -- 172.21.15.114:0/460484749 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cb0069000 con 0x7f7cb0101880 2024-09-13T18:21:51.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.550+0000 7f7cad7fa640 1 -- 172.21.15.114:0/460484749 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f7c98004830 con 0x7f7cb0101880 2024-09-13T18:21:51.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.550+0000 7f7cad7fa640 1 --2- 172.21.15.114:0/460484749 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f7c9c0a1110 0x7f7c9c0a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:51.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.550+0000 7f7caf7fe640 1 --2- 172.21.15.114:0/460484749 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f7c9c0a1110 0x7f7c9c0a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:51.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.550+0000 7f7cad7fa640 1 -- 172.21.15.114:0/460484749 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f7c980eda00 con 0x7f7cb0101880 2024-09-13T18:21:51.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.551+0000 7f7caf7fe640 1 --2- 172.21.15.114:0/460484749 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f7c9c0a1110 0x7f7c9c0a35d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7ca000bbe0 tx=0x7f7ca000b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:51.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.554+0000 7f7cad7fa640 1 -- 172.21.15.114:0/460484749 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f7c980b6cc0 con 0x7f7cb0101880 2024-09-13T18:21:51.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.753+0000 7f7cb6167640 1 -- 172.21.15.114:0/460484749 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f7cb0101340 con 0x7f7cb0101880 2024-09-13T18:21:51.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.753+0000 7f7cad7fa640 1 -- 172.21.15.114:0/460484749 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f7c980ba2e0 con 0x7f7cb0101880 2024-09-13T18:21:51.761 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:21:51.762 INFO:teuthology.orchestra.run.smithi114.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-13T18:21:51.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.755+0000 7f7cb6167640 1 -- 172.21.15.114:0/460484749 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f7c9c0a1110 msgr2=0x7f7c9c0a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:51.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.755+0000 7f7cb6167640 1 --2- 172.21.15.114:0/460484749 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f7c9c0a1110 0x7f7c9c0a35d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7ca000bbe0 tx=0x7f7ca000b390 comp rx=0 tx=0).stop 2024-09-13T18:21:51.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.755+0000 7f7cb6167640 1 -- 172.21.15.114:0/460484749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cb0101880 msgr2=0x7f7cb0196750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:51.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.755+0000 7f7cb6167640 1 --2- 172.21.15.114:0/460484749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cb0101880 0x7f7cb0196750 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7c98005ab0 tx=0x7f7c98032c20 comp rx=0 tx=0).stop 2024-09-13T18:21:51.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.755+0000 7f7cb6167640 1 -- 172.21.15.114:0/460484749 shutdown_connections 2024-09-13T18:21:51.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.755+0000 7f7cb6167640 1 --2- 172.21.15.114:0/460484749 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f7c9c0a1110 0x7f7c9c0a35d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:51.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.755+0000 7f7cb6167640 1 --2- 172.21.15.114:0/460484749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7cb010a3e0 0x7f7cb019ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:51.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.755+0000 7f7cb6167640 1 --2- 172.21.15.114:0/460484749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cb0101880 0x7f7cb0196750 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:51.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.755+0000 7f7cb6167640 1 --2- 172.21.15.114:0/460484749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7cb0100f60 0x7f7cb0196210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:51.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.755+0000 7f7cb6167640 1 -- 172.21.15.114:0/460484749 >> 172.21.15.114:0/460484749 conn(0x7f7cb00fa9a0 msgr2=0x7f7cb00fc300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:51.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.755+0000 7f7cb6167640 1 -- 172.21.15.114:0/460484749 shutdown_connections 2024-09-13T18:21:51.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:51.755+0000 7f7cb6167640 1 -- 172.21.15.114:0/460484749 wait complete. 2024-09-13T18:21:52.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:51 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/460484749' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-13T18:21:52.368 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:51 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/460484749' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-13T18:21:52.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:51 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/460484749' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-13T18:21:52.542 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-13T18:21:52.542 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:21:52.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.643+0000 7f222bdbc640 1 -- 172.21.15.114:0/1047451828 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22240fe7d0 msgr2=0x7f2224108380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:52.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.643+0000 7f222bdbc640 1 --2- 172.21.15.114:0/1047451828 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22240fe7d0 0x7f2224108380 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2214009900 tx=0x7f221402a410 comp rx=0 tx=0).stop 2024-09-13T18:21:52.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.643+0000 7f222bdbc640 1 -- 172.21.15.114:0/1047451828 shutdown_connections 2024-09-13T18:21:52.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.643+0000 7f222bdbc640 1 --2- 172.21.15.114:0/1047451828 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22240fe7d0 0x7f2224108380 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:52.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.643+0000 7f222bdbc640 1 --2- 172.21.15.114:0/1047451828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22240fde10 0x7f22240fe290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:52.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.643+0000 7f222bdbc640 1 --2- 172.21.15.114:0/1047451828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22241013a0 0x7f22241017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:52.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.643+0000 7f222bdbc640 1 -- 172.21.15.114:0/1047451828 >> 172.21.15.114:0/1047451828 conn(0x7f22240f9b80 msgr2=0x7f22240fbfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:52.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.643+0000 7f222bdbc640 1 -- 172.21.15.114:0/1047451828 shutdown_connections 2024-09-13T18:21:52.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.643+0000 7f222bdbc640 1 -- 172.21.15.114:0/1047451828 wait complete. 2024-09-13T18:21:52.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.644+0000 7f222bdbc640 1 Processor -- start 2024-09-13T18:21:52.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.644+0000 7f222bdbc640 1 -- start start 2024-09-13T18:21:52.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.644+0000 7f222bdbc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22240fde10 0x7f2224195c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:52.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.644+0000 7f222bdbc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22240fe7d0 0x7f2224196190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:52.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.644+0000 7f222bdbc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22241013a0 0x7f222419d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:52.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.644+0000 7f222bdbc640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222419d730 con 0x7f22240fde10 2024-09-13T18:21:52.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.644+0000 7f222bdbc640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222419d8a0 con 0x7f22241013a0 2024-09-13T18:21:52.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.644+0000 7f222bdbc640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222419da60 con 0x7f22240fe7d0 2024-09-13T18:21:52.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.644+0000 7f2229b58640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22240fde10 0x7f2224195c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:52.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.644+0000 7f2229b58640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22240fde10 0x7f2224195c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47326/0 (socket says 172.21.15.114:47326) 2024-09-13T18:21:52.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.644+0000 7f2229b58640 1 -- 172.21.15.114:0/957959375 learned_addr learned my addr 172.21.15.114:0/957959375 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:52.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.644+0000 7f2229357640 1 --2- 172.21.15.114:0/957959375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22240fe7d0 0x7f2224196190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:52.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.644+0000 7f2229b58640 1 -- 172.21.15.114:0/957959375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22240fe7d0 msgr2=0x7f2224196190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:52.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.644+0000 7f2229b58640 1 --2- 172.21.15.114:0/957959375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22240fe7d0 0x7f2224196190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:52.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.644+0000 7f222a359640 1 --2- 172.21.15.114:0/957959375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22241013a0 0x7f222419d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:52.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.644+0000 7f2229b58640 1 -- 172.21.15.114:0/957959375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22241013a0 msgr2=0x7f222419d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:52.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.644+0000 7f2229b58640 1 --2- 172.21.15.114:0/957959375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22241013a0 0x7f222419d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:52.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.644+0000 7f2229b58640 1 -- 172.21.15.114:0/957959375 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2214009560 con 0x7f22240fde10 2024-09-13T18:21:52.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.644+0000 7f222a359640 1 --2- 172.21.15.114:0/957959375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22241013a0 0x7f222419d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:21:52.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.644+0000 7f2229b58640 1 --2- 172.21.15.114:0/957959375 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22240fde10 0x7f2224195c50 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f221c00d950 tx=0x7f221c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:52.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.644+0000 7f2222ffd640 1 -- 172.21.15.114:0/957959375 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221c009c30 con 0x7f22240fde10 2024-09-13T18:21:52.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.645+0000 7f2222ffd640 1 -- 172.21.15.114:0/957959375 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f221c009d90 con 0x7f22240fde10 2024-09-13T18:21:52.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.645+0000 7f222bdbc640 1 -- 172.21.15.114:0/957959375 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f222419dd40 con 0x7f22240fde10 2024-09-13T18:21:52.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.645+0000 7f2222ffd640 1 -- 172.21.15.114:0/957959375 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221c00f620 con 0x7f22240fde10 2024-09-13T18:21:52.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.645+0000 7f222bdbc640 1 -- 172.21.15.114:0/957959375 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f222419e1a0 con 0x7f22240fde10 2024-09-13T18:21:52.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.646+0000 7f222bdbc640 1 -- 172.21.15.114:0/957959375 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2224107380 con 0x7f22240fde10 2024-09-13T18:21:52.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.647+0000 7f2222ffd640 1 -- 172.21.15.114:0/957959375 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f221c010980 con 0x7f22240fde10 2024-09-13T18:21:52.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.647+0000 7f2222ffd640 1 --2- 172.21.15.114:0/957959375 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f220c0a11a0 0x7f220c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:52.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.647+0000 7f2229357640 1 --2- 172.21.15.114:0/957959375 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f220c0a11a0 0x7f220c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:52.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.647+0000 7f2222ffd640 1 -- 172.21.15.114:0/957959375 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f221c0ccc70 con 0x7f22240fde10 2024-09-13T18:21:52.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.648+0000 7f2229357640 1 --2- 172.21.15.114:0/957959375 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f220c0a11a0 0x7f220c0a3660 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2218005c40 tx=0x7f221800a390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:52.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.651+0000 7f2222ffd640 1 -- 172.21.15.114:0/957959375 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f221c095ea0 con 0x7f22240fde10 2024-09-13T18:21:52.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.816+0000 7f222bdbc640 1 -- 172.21.15.114:0/957959375 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f222410b550 con 0x7f22240fde10 2024-09-13T18:21:52.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.818+0000 7f2222ffd640 1 -- 172.21.15.114:0/957959375 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24696 (secure 0 0 0) 0x7f221c017090 con 0x7f22240fde10 2024-09-13T18:21:52.821 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:21:52.822 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":93,"fsid":"97e7edf2-71fb-11ef-bceb-c7b262605968","created":"2024-09-13T18:12:36.360625+0000","modified":"2024-09-13T18:21:28.143797+0000","last_up_change":"2024-09-13T18:19:12.350370+0000","last_in_change":"2024-09-13T18:18:52.321596+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:15:40.129659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:20:00.024407+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:20:01.085549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T18:20:04.334216+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9914f01a-9f6f-4878-b955-83570199c354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6803","nonce":3499525826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6805","nonce":3499525826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6809","nonce":3499525826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6807","nonce":3499525826}]},"public_addr":"172.21.15.114:6803/3499525826","cluster_addr":"172.21.15.114:6805/3499525826","heartbeat_back_addr":"172.21.15.114:6809/3499525826","heartbeat_front_addr":"172.21.15.114:6807/3499525826","state":["exists","up"]},{"osd":1,"uuid":"ef65e012-7469-4c20-b354-17022bca63c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6801","nonce":4264264866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6803","nonce":4264264866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6807","nonce":4264264866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6805","nonce":4264264866}]},"public_addr":"172.21.15.134:6801/4264264866","cluster_addr":"172.21.15.134:6803/4264264866","heartbeat_back_addr":"172.21.15.134:6807/4264264866","heartbeat_front_addr":"172.21.15.134:6805/4264264866","state":["exists","up"]},{"osd":2,"uuid":"578213fd-715f-4be7-91fa-254f3d0e3022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6801","nonce":2288807707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6803","nonce":2288807707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6807","nonce":2288807707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6805","nonce":2288807707}]},"public_addr":"172.21.15.204:6801/2288807707","cluster_addr":"172.21.15.204:6803/2288807707","heartbeat_back_addr":"172.21.15.204:6807/2288807707","heartbeat_front_addr":"172.21.15.204:6805/2288807707","state":["exists","up"]},{"osd":3,"uuid":"b64aab3f-1e52-4828-8397-74af5e05f2cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6811","nonce":2904204254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6813","nonce":2904204254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6817","nonce":2904204254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6815","nonce":2904204254}]},"public_addr":"172.21.15.114:6811/2904204254","cluster_addr":"172.21.15.114:6813/2904204254","heartbeat_back_addr":"172.21.15.114:6817/2904204254","heartbeat_front_addr":"172.21.15.114:6815/2904204254","state":["exists","up"]},{"osd":4,"uuid":"f38943d6-39c0-42e1-b427-cb6bb90ac38a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6809","nonce":1773945333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6811","nonce":1773945333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6815","nonce":1773945333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6813","nonce":1773945333}]},"public_addr":"172.21.15.134:6809/1773945333","cluster_addr":"172.21.15.134:6811/1773945333","heartbeat_back_addr":"172.21.15.134:6815/1773945333","heartbeat_front_addr":"172.21.15.134:6813/1773945333","state":["exists","up"]},{"osd":5,"uuid":"bb38f361-8e00-4178-9752-5ddbf3af20e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6809","nonce":2371778055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6811","nonce":2371778055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6815","nonce":2371778055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6813","nonce":2371778055}]},"public_addr":"172.21.15.204:6809/2371778055","cluster_addr":"172.21.15.204:6811/2371778055","heartbeat_back_addr":"172.21.15.204:6815/2371778055","heartbeat_front_addr":"172.21.15.204:6813/2371778055","state":["exists","up"]},{"osd":6,"uuid":"93d40a03-e7ff-4a3f-9efa-5c5403c9983d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6819","nonce":1484325044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6821","nonce":1484325044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6825","nonce":1484325044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6823","nonce":1484325044}]},"public_addr":"172.21.15.114:6819/1484325044","cluster_addr":"172.21.15.114:6821/1484325044","heartbeat_back_addr":"172.21.15.114:6825/1484325044","heartbeat_front_addr":"172.21.15.114:6823/1484325044","state":["exists","up"]},{"osd":7,"uuid":"982f629a-1b95-4be1-8e75-5ddce1ad77b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6817","nonce":799246614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6819","nonce":799246614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6823","nonce":799246614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6821","nonce":799246614}]},"public_addr":"172.21.15.134:6817/799246614","cluster_addr":"172.21.15.134:6819/799246614","heartbeat_back_addr":"172.21.15.134:6823/799246614","heartbeat_front_addr":"172.21.15.134:6821/799246614","state":["exists","up"]},{"osd":8,"uuid":"409beb9b-8227-4dad-a4a0-f0e5ce70dc5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6817","nonce":858331805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6819","nonce":858331805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6823","nonce":858331805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6821","nonce":858331805}]},"public_addr":"172.21.15.204:6817/858331805","cluster_addr":"172.21.15.204:6819/858331805","heartbeat_back_addr":"172.21.15.204:6823/858331805","heartbeat_front_addr":"172.21.15.204:6821/858331805","state":["exists","up"]},{"osd":9,"uuid":"a345a15c-b1fc-4877-b445-418eebf0d0ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6827","nonce":2172392854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6829","nonce":2172392854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6833","nonce":2172392854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6831","nonce":2172392854}]},"public_addr":"172.21.15.114:6827/2172392854","cluster_addr":"172.21.15.114:6829/2172392854","heartbeat_back_addr":"172.21.15.114:6833/2172392854","heartbeat_front_addr":"172.21.15.114:6831/2172392854","state":["exists","up"]},{"osd":10,"uuid":"464d995f-d90b-4caa-addb-1f7562bd74b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6825","nonce":3073513468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6827","nonce":3073513468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6831","nonce":3073513468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6829","nonce":3073513468}]},"public_addr":"172.21.15.134:6825/3073513468","cluster_addr":"172.21.15.134:6827/3073513468","heartbeat_back_addr":"172.21.15.134:6831/3073513468","heartbeat_front_addr":"172.21.15.134:6829/3073513468","state":["exists","up"]},{"osd":11,"uuid":"71dd794d-326c-477f-8d02-ee32bff1350d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6825","nonce":3239521765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6827","nonce":3239521765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6831","nonce":3239521765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6829","nonce":3239521765}]},"public_addr":"172.21.15.204:6825/3239521765","cluster_addr":"172.21.15.204:6827/3239521765","heartbeat_back_addr":"172.21.15.204:6831/3239521765","heartbeat_front_addr":"172.21.15.204:6829/3239521765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:14:54.542687+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:15:15.746055+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:15:38.371131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:01.383932+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:24.796237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:48.422214+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:11.284323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:34.570365+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:58.232855+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:18:21.609596+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:18:45.565859+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:19:10.017969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:6801/4281866858":"2024-09-14T18:13:25.823027+0000","172.21.15.114:6800/4281866858":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2949422849":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2034742251":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/722571999":"2024-09-14T18:13:03.521116+0000","172.21.15.114:6801/830738794":"2024-09-14T18:13:03.521116+0000","172.21.15.114:6800/830738794":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/2828364892":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/4257828906":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/3544951322":"2024-09-14T18:13:03.521116+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:21:52.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.820+0000 7f222bdbc640 1 -- 172.21.15.114:0/957959375 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f220c0a11a0 msgr2=0x7f220c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:52.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.820+0000 7f222bdbc640 1 --2- 172.21.15.114:0/957959375 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f220c0a11a0 0x7f220c0a3660 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2218005c40 tx=0x7f221800a390 comp rx=0 tx=0).stop 2024-09-13T18:21:52.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.820+0000 7f222bdbc640 1 -- 172.21.15.114:0/957959375 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22240fde10 msgr2=0x7f2224195c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:52.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.820+0000 7f222bdbc640 1 --2- 172.21.15.114:0/957959375 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22240fde10 0x7f2224195c50 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f221c00d950 tx=0x7f221c00de10 comp rx=0 tx=0).stop 2024-09-13T18:21:52.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.820+0000 7f222bdbc640 1 -- 172.21.15.114:0/957959375 shutdown_connections 2024-09-13T18:21:52.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.820+0000 7f222bdbc640 1 --2- 172.21.15.114:0/957959375 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f220c0a11a0 0x7f220c0a3660 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:52.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.820+0000 7f222bdbc640 1 --2- 172.21.15.114:0/957959375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22241013a0 0x7f222419d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:52.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.820+0000 7f222bdbc640 1 --2- 172.21.15.114:0/957959375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22240fe7d0 0x7f2224196190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:52.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.820+0000 7f222bdbc640 1 --2- 172.21.15.114:0/957959375 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22240fde10 0x7f2224195c50 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:52.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.820+0000 7f222bdbc640 1 -- 172.21.15.114:0/957959375 >> 172.21.15.114:0/957959375 conn(0x7f22240f9b80 msgr2=0x7f2224104550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:52.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.820+0000 7f222bdbc640 1 -- 172.21.15.114:0/957959375 shutdown_connections 2024-09-13T18:21:52.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.820+0000 7f222bdbc640 1 -- 172.21.15.114:0/957959375 wait complete. 2024-09-13T18:21:52.834 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T18:15:40.129659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T18:20:00.024407+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T18:20:01.085549+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T18:20:04.334216+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T18:21:52.835 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T18:21:52.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.933+0000 7fb850d11640 1 -- 172.21.15.114:0/2635036977 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb84c100810 msgr2=0x7fb84c102c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:52.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.933+0000 7fb850d11640 1 --2- 172.21.15.114:0/2635036977 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb84c100810 0x7fb84c102c00 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb83c009900 tx=0x7fb83c02a410 comp rx=0 tx=0).stop 2024-09-13T18:21:52.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.933+0000 7fb850d11640 1 -- 172.21.15.114:0/2635036977 shutdown_connections 2024-09-13T18:21:52.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.933+0000 7fb850d11640 1 --2- 172.21.15.114:0/2635036977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb84c105a70 0x7fb84c107ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:52.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.933+0000 7fb850d11640 1 --2- 172.21.15.114:0/2635036977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb84c103140 0x7fb84c105530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:52.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.933+0000 7fb850d11640 1 --2- 172.21.15.114:0/2635036977 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb84c100810 0x7fb84c102c00 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:52.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.933+0000 7fb850d11640 1 -- 172.21.15.114:0/2635036977 >> 172.21.15.114:0/2635036977 conn(0x7fb84c0fa3f0 msgr2=0x7fb84c0fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:52.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.933+0000 7fb850d11640 1 -- 172.21.15.114:0/2635036977 shutdown_connections 2024-09-13T18:21:52.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.933+0000 7fb850d11640 1 -- 172.21.15.114:0/2635036977 wait complete. 2024-09-13T18:21:52.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.934+0000 7fb850d11640 1 Processor -- start 2024-09-13T18:21:52.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.934+0000 7fb850d11640 1 -- start start 2024-09-13T18:21:52.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.934+0000 7fb850d11640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb84c100810 0x7fb84c100180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:52.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.934+0000 7fb850d11640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb84c103140 0x7fb84c0fe7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:52.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.934+0000 7fb850d11640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb84c105a70 0x7fb84c0fed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:52.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.934+0000 7fb850d11640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb84c0ff460 con 0x7fb84c103140 2024-09-13T18:21:52.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.934+0000 7fb850d11640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb84c0ff5d0 con 0x7fb84c100810 2024-09-13T18:21:52.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.934+0000 7fb850d11640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb84c19fe30 con 0x7fb84c105a70 2024-09-13T18:21:52.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.934+0000 7fb849d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb84c103140 0x7fb84c0fe7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:52.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.934+0000 7fb849d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb84c103140 0x7fb84c0fe7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47334/0 (socket says 172.21.15.114:47334) 2024-09-13T18:21:52.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.934+0000 7fb849d9b640 1 -- 172.21.15.114:0/957473066 learned_addr learned my addr 172.21.15.114:0/957473066 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:52.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.935+0000 7fb84a59c640 1 --2- 172.21.15.114:0/957473066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb84c100810 0x7fb84c100180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:52.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.935+0000 7fb849d9b640 1 -- 172.21.15.114:0/957473066 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb84c105a70 msgr2=0x7fb84c0fed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:52.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.935+0000 7fb849d9b640 1 --2- 172.21.15.114:0/957473066 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb84c105a70 0x7fb84c0fed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:52.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.935+0000 7fb849d9b640 1 -- 172.21.15.114:0/957473066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb84c100810 msgr2=0x7fb84c100180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:52.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.935+0000 7fb849d9b640 1 --2- 172.21.15.114:0/957473066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb84c100810 0x7fb84c100180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:52.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.935+0000 7fb849d9b640 1 -- 172.21.15.114:0/957473066 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb83c009560 con 0x7fb84c103140 2024-09-13T18:21:52.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.935+0000 7fb849d9b640 1 --2- 172.21.15.114:0/957473066 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb84c103140 0x7fb84c0fe7c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb84000d7a0 tx=0x7fb84000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:52.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.935+0000 7fb83b7fe640 1 -- 172.21.15.114:0/957473066 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb840009c30 con 0x7fb84c103140 2024-09-13T18:21:52.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.935+0000 7fb83b7fe640 1 -- 172.21.15.114:0/957473066 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb84000deb0 con 0x7fb84c103140 2024-09-13T18:21:52.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.935+0000 7fb850d11640 1 -- 172.21.15.114:0/957473066 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb84c1a00e0 con 0x7fb84c103140 2024-09-13T18:21:52.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.935+0000 7fb83b7fe640 1 -- 172.21.15.114:0/957473066 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84000f620 con 0x7fb84c103140 2024-09-13T18:21:52.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.935+0000 7fb850d11640 1 -- 172.21.15.114:0/957473066 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb84c1a05f0 con 0x7fb84c103140 2024-09-13T18:21:52.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.936+0000 7fb850d11640 1 -- 172.21.15.114:0/957473066 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb84c0fbee0 con 0x7fb84c103140 2024-09-13T18:21:52.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.938+0000 7fb83b7fe640 1 -- 172.21.15.114:0/957473066 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fb840010460 con 0x7fb84c103140 2024-09-13T18:21:52.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.939+0000 7fb83b7fe640 1 --2- 172.21.15.114:0/957473066 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb8340a1120 0x7fb8340a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:52.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.939+0000 7fb84a59c640 1 --2- 172.21.15.114:0/957473066 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb8340a1120 0x7fb8340a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:52.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.939+0000 7fb83b7fe640 1 -- 172.21.15.114:0/957473066 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fb8400ccc40 con 0x7fb84c103140 2024-09-13T18:21:52.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.939+0000 7fb84a59c640 1 --2- 172.21.15.114:0/957473066 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb8340a1120 0x7fb8340a35e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb83c000c00 tx=0x7fb83c005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:52.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:52.942+0000 7fb83b7fe640 1 -- 172.21.15.114:0/957473066 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fb840095f00 con 0x7fb84c103140 2024-09-13T18:21:53.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.106+0000 7fb850d11640 1 -- 172.21.15.114:0/957473066 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb84c02c3e0 con 0x7fb84c103140 2024-09-13T18:21:53.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.107+0000 7fb83b7fe640 1 -- 172.21.15.114:0/957473066 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb840021020 con 0x7fb84c103140 2024-09-13T18:21:53.110 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 1 2024-09-13T18:21:53.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.109+0000 7fb850d11640 1 -- 172.21.15.114:0/957473066 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb8340a1120 msgr2=0x7fb8340a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:53.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.109+0000 7fb850d11640 1 --2- 172.21.15.114:0/957473066 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb8340a1120 0x7fb8340a35e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb83c000c00 tx=0x7fb83c005a20 comp rx=0 tx=0).stop 2024-09-13T18:21:53.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.109+0000 7fb850d11640 1 -- 172.21.15.114:0/957473066 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb84c103140 msgr2=0x7fb84c0fe7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:53.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.109+0000 7fb850d11640 1 --2- 172.21.15.114:0/957473066 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb84c103140 0x7fb84c0fe7c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb84000d7a0 tx=0x7fb84000dc60 comp rx=0 tx=0).stop 2024-09-13T18:21:53.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.109+0000 7fb850d11640 1 -- 172.21.15.114:0/957473066 shutdown_connections 2024-09-13T18:21:53.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.109+0000 7fb850d11640 1 --2- 172.21.15.114:0/957473066 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb8340a1120 0x7fb8340a35e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:53.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.109+0000 7fb850d11640 1 --2- 172.21.15.114:0/957473066 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb84c105a70 0x7fb84c0fed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:53.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.109+0000 7fb850d11640 1 --2- 172.21.15.114:0/957473066 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb84c103140 0x7fb84c0fe7c0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:53.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.109+0000 7fb850d11640 1 --2- 172.21.15.114:0/957473066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb84c100810 0x7fb84c100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:53.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.109+0000 7fb850d11640 1 -- 172.21.15.114:0/957473066 >> 172.21.15.114:0/957473066 conn(0x7fb84c0fa3f0 msgr2=0x7fb84c101050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:53.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.109+0000 7fb850d11640 1 -- 172.21.15.114:0/957473066 shutdown_connections 2024-09-13T18:21:53.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.109+0000 7fb850d11640 1 -- 172.21.15.114:0/957473066 wait complete. 2024-09-13T18:21:53.122 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:52 smithi114 ceph-mon[30897]: pgmap v336: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-13T18:21:53.122 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:52 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/957959375' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:21:53.123 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T18:21:53.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.221+0000 7f0ac87ec640 1 -- 172.21.15.114:0/594585240 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ac01008b0 msgr2=0x7f0ac0102ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:53.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.221+0000 7f0ac87ec640 1 --2- 172.21.15.114:0/594585240 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ac01008b0 0x7f0ac0102ca0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0abc009900 tx=0x7f0abc02a410 comp rx=0 tx=0).stop 2024-09-13T18:21:53.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.221+0000 7f0ac87ec640 1 -- 172.21.15.114:0/594585240 shutdown_connections 2024-09-13T18:21:53.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.221+0000 7f0ac87ec640 1 --2- 172.21.15.114:0/594585240 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ac0105b10 0x7f0ac0107f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:53.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.221+0000 7f0ac87ec640 1 --2- 172.21.15.114:0/594585240 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ac01031e0 0x7f0ac01055d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:53.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.221+0000 7f0ac87ec640 1 --2- 172.21.15.114:0/594585240 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ac01008b0 0x7f0ac0102ca0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:53.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.221+0000 7f0ac87ec640 1 -- 172.21.15.114:0/594585240 >> 172.21.15.114:0/594585240 conn(0x7f0ac00fa3d0 msgr2=0x7f0ac00fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:53.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.221+0000 7f0ac87ec640 1 -- 172.21.15.114:0/594585240 shutdown_connections 2024-09-13T18:21:53.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.222+0000 7f0ac87ec640 1 -- 172.21.15.114:0/594585240 wait complete. 2024-09-13T18:21:53.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.222+0000 7f0ac87ec640 1 Processor -- start 2024-09-13T18:21:53.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.222+0000 7f0ac87ec640 1 -- start start 2024-09-13T18:21:53.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.222+0000 7f0ac87ec640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ac01008b0 0x7f0ac0197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:53.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.222+0000 7f0ac87ec640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ac01031e0 0x7f0ac0198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:53.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.222+0000 7f0ac87ec640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ac0105b10 0x7f0ac019f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:53.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.222+0000 7f0ac87ec640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac019f940 con 0x7f0ac01008b0 2024-09-13T18:21:53.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.222+0000 7f0ac87ec640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac019fab0 con 0x7f0ac0105b10 2024-09-13T18:21:53.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.222+0000 7f0ac87ec640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac019fc70 con 0x7f0ac01031e0 2024-09-13T18:21:53.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.222+0000 7f0ac6588640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ac01008b0 0x7f0ac0197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:53.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.222+0000 7f0ac6588640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ac01008b0 0x7f0ac0197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47348/0 (socket says 172.21.15.114:47348) 2024-09-13T18:21:53.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.222+0000 7f0ac6588640 1 -- 172.21.15.114:0/2286177912 learned_addr learned my addr 172.21.15.114:0/2286177912 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:53.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.222+0000 7f0ac6d89640 1 --2- 172.21.15.114:0/2286177912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ac0105b10 0x7f0ac019f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:53.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.223+0000 7f0ac5d87640 1 --2- 172.21.15.114:0/2286177912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ac01031e0 0x7f0ac0198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:53.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.223+0000 7f0ac6588640 1 -- 172.21.15.114:0/2286177912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ac01031e0 msgr2=0x7f0ac0198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:53.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.223+0000 7f0ac6588640 1 --2- 172.21.15.114:0/2286177912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ac01031e0 0x7f0ac0198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:53.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.223+0000 7f0ac6588640 1 -- 172.21.15.114:0/2286177912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ac0105b10 msgr2=0x7f0ac019f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:53.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.223+0000 7f0ac6588640 1 --2- 172.21.15.114:0/2286177912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ac0105b10 0x7f0ac019f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:53.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.223+0000 7f0ac6588640 1 -- 172.21.15.114:0/2286177912 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0abc009560 con 0x7f0ac01008b0 2024-09-13T18:21:53.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.223+0000 7f0ac5d87640 1 --2- 172.21.15.114:0/2286177912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ac01031e0 0x7f0ac0198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:21:53.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.223+0000 7f0ac6588640 1 --2- 172.21.15.114:0/2286177912 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ac01008b0 0x7f0ac0197e50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0abc009230 tx=0x7f0abc004380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:53.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.223+0000 7f0ab77fe640 1 -- 172.21.15.114:0/2286177912 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0abc031d70 con 0x7f0ac01008b0 2024-09-13T18:21:53.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.223+0000 7f0ab77fe640 1 -- 172.21.15.114:0/2286177912 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0abc004b50 con 0x7f0ac01008b0 2024-09-13T18:21:53.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.223+0000 7f0ac87ec640 1 -- 172.21.15.114:0/2286177912 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ac019fef0 con 0x7f0ac01008b0 2024-09-13T18:21:53.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.223+0000 7f0ab77fe640 1 -- 172.21.15.114:0/2286177912 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0abc003bf0 con 0x7f0ac01008b0 2024-09-13T18:21:53.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.223+0000 7f0ac87ec640 1 -- 172.21.15.114:0/2286177912 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ac01a02c0 con 0x7f0ac01008b0 2024-09-13T18:21:53.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.224+0000 7f0ac87ec640 1 -- 172.21.15.114:0/2286177912 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ac0107600 con 0x7f0ac01008b0 2024-09-13T18:21:53.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.225+0000 7f0ab77fe640 1 -- 172.21.15.114:0/2286177912 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f0abc003d50 con 0x7f0ac01008b0 2024-09-13T18:21:53.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.226+0000 7f0ab77fe640 1 --2- 172.21.15.114:0/2286177912 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f0aac0a1120 0x7f0aac0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:53.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.226+0000 7f0ac5d87640 1 --2- 172.21.15.114:0/2286177912 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f0aac0a1120 0x7f0aac0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:53.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.226+0000 7f0ab77fe640 1 -- 172.21.15.114:0/2286177912 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f0abc0ee2b0 con 0x7f0ac01008b0 2024-09-13T18:21:53.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.226+0000 7f0ac5d87640 1 --2- 172.21.15.114:0/2286177912 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f0aac0a1120 0x7f0aac0a35e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0199360 tx=0x7f0ab0005d90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:53.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.229+0000 7f0ab77fe640 1 -- 172.21.15.114:0/2286177912 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f0abc0b7570 con 0x7f0ac01008b0 2024-09-13T18:21:53.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:52 smithi134 ceph-mon[34283]: pgmap v336: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-13T18:21:53.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:52 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/957959375' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:21:53.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.393+0000 7f0ac87ec640 1 -- 172.21.15.114:0/2286177912 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0ac002c480 con 0x7f0ac01008b0 2024-09-13T18:21:53.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.393+0000 7f0ab77fe640 1 -- 172.21.15.114:0/2286177912 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0abc0bab90 con 0x7f0ac01008b0 2024-09-13T18:21:53.400 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2024-09-13T18:21:53.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.395+0000 7f0ac87ec640 1 -- 172.21.15.114:0/2286177912 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f0aac0a1120 msgr2=0x7f0aac0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:53.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.395+0000 7f0ac87ec640 1 --2- 172.21.15.114:0/2286177912 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f0aac0a1120 0x7f0aac0a35e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0199360 tx=0x7f0ab0005d90 comp rx=0 tx=0).stop 2024-09-13T18:21:53.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.395+0000 7f0ac87ec640 1 -- 172.21.15.114:0/2286177912 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ac01008b0 msgr2=0x7f0ac0197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:53.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.395+0000 7f0ac87ec640 1 --2- 172.21.15.114:0/2286177912 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ac01008b0 0x7f0ac0197e50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0abc009230 tx=0x7f0abc004380 comp rx=0 tx=0).stop 2024-09-13T18:21:53.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.395+0000 7f0ac87ec640 1 -- 172.21.15.114:0/2286177912 shutdown_connections 2024-09-13T18:21:53.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.395+0000 7f0ac87ec640 1 --2- 172.21.15.114:0/2286177912 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f0aac0a1120 0x7f0aac0a35e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:53.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.395+0000 7f0ac87ec640 1 --2- 172.21.15.114:0/2286177912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ac0105b10 0x7f0ac019f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:53.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.395+0000 7f0ac87ec640 1 --2- 172.21.15.114:0/2286177912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ac01031e0 0x7f0ac0198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:53.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.395+0000 7f0ac87ec640 1 --2- 172.21.15.114:0/2286177912 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ac01008b0 0x7f0ac0197e50 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:53.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.395+0000 7f0ac87ec640 1 -- 172.21.15.114:0/2286177912 >> 172.21.15.114:0/2286177912 conn(0x7f0ac00fa3d0 msgr2=0x7f0ac0100010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:53.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.395+0000 7f0ac87ec640 1 -- 172.21.15.114:0/2286177912 shutdown_connections 2024-09-13T18:21:53.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.395+0000 7f0ac87ec640 1 -- 172.21.15.114:0/2286177912 wait complete. 2024-09-13T18:21:53.409 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T18:21:53.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:52 smithi204 ceph-mon[34777]: pgmap v336: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-13T18:21:53.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:52 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/957959375' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:21:53.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.508+0000 7f64c0583640 1 -- 172.21.15.114:0/737986675 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f64b80feef0 msgr2=0x7f64b80ff370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:53.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.508+0000 7f64c0583640 1 --2- 172.21.15.114:0/737986675 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f64b80feef0 0x7f64b80ff370 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f64a8009900 tx=0x7f64a802a410 comp rx=0 tx=0).stop 2024-09-13T18:21:53.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.508+0000 7f64c0583640 1 -- 172.21.15.114:0/737986675 shutdown_connections 2024-09-13T18:21:53.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.508+0000 7f64c0583640 1 --2- 172.21.15.114:0/737986675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64b80ff8b0 0x7f64b8107150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:53.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.508+0000 7f64c0583640 1 --2- 172.21.15.114:0/737986675 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f64b80feef0 0x7f64b80ff370 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:53.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.508+0000 7f64c0583640 1 --2- 172.21.15.114:0/737986675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64b81002a0 0x7f64b81006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:53.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.508+0000 7f64c0583640 1 -- 172.21.15.114:0/737986675 >> 172.21.15.114:0/737986675 conn(0x7f64b80f9b70 msgr2=0x7f64b80fbfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:53.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.508+0000 7f64c0583640 1 -- 172.21.15.114:0/737986675 shutdown_connections 2024-09-13T18:21:53.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.508+0000 7f64c0583640 1 -- 172.21.15.114:0/737986675 wait complete. 2024-09-13T18:21:53.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.508+0000 7f64c0583640 1 Processor -- start 2024-09-13T18:21:53.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.508+0000 7f64c0583640 1 -- start start 2024-09-13T18:21:53.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.509+0000 7f64c0583640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64b80feef0 0x7f64b8195c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:53.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.509+0000 7f64c0583640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f64b80ff8b0 0x7f64b81961a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:53.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.509+0000 7f64c0583640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64b81002a0 0x7f64b819d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:53.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.509+0000 7f64c0583640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b819d740 con 0x7f64b80ff8b0 2024-09-13T18:21:53.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.509+0000 7f64c0583640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b819d8b0 con 0x7f64b81002a0 2024-09-13T18:21:53.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.509+0000 7f64c0583640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b819da70 con 0x7f64b80feef0 2024-09-13T18:21:53.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.509+0000 7f64bdb1e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f64b80ff8b0 0x7f64b81961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:53.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.509+0000 7f64bdb1e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f64b80ff8b0 0x7f64b81961a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33668/0 (socket says 172.21.15.114:33668) 2024-09-13T18:21:53.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.509+0000 7f64bdb1e640 1 -- 172.21.15.114:0/2351996700 learned_addr learned my addr 172.21.15.114:0/2351996700 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:53.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.509+0000 7f64be31f640 1 --2- 172.21.15.114:0/2351996700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64b80feef0 0x7f64b8195c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:53.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.509+0000 7f64beb20640 1 --2- 172.21.15.114:0/2351996700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64b81002a0 0x7f64b819d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:53.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.509+0000 7f64bdb1e640 1 -- 172.21.15.114:0/2351996700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64b80feef0 msgr2=0x7f64b8195c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:53.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.509+0000 7f64bdb1e640 1 --2- 172.21.15.114:0/2351996700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64b80feef0 0x7f64b8195c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:53.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.509+0000 7f64bdb1e640 1 -- 172.21.15.114:0/2351996700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64b81002a0 msgr2=0x7f64b819d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:53.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.509+0000 7f64bdb1e640 1 --2- 172.21.15.114:0/2351996700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64b81002a0 0x7f64b819d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:53.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.509+0000 7f64bdb1e640 1 -- 172.21.15.114:0/2351996700 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64a8009560 con 0x7f64b80ff8b0 2024-09-13T18:21:53.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.509+0000 7f64bdb1e640 1 --2- 172.21.15.114:0/2351996700 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f64b80ff8b0 0x7f64b81961a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f64a8005ab0 tx=0x7f64a80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:53.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.509+0000 7f64af7fe640 1 -- 172.21.15.114:0/2351996700 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a8031d70 con 0x7f64b80ff8b0 2024-09-13T18:21:53.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.509+0000 7f64af7fe640 1 -- 172.21.15.114:0/2351996700 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f64a80044e0 con 0x7f64b80ff8b0 2024-09-13T18:21:53.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.509+0000 7f64c0583640 1 -- 172.21.15.114:0/2351996700 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64b819dcf0 con 0x7f64b80ff8b0 2024-09-13T18:21:53.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.509+0000 7f64af7fe640 1 -- 172.21.15.114:0/2351996700 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a8003b00 con 0x7f64b80ff8b0 2024-09-13T18:21:53.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.509+0000 7f64c0583640 1 -- 172.21.15.114:0/2351996700 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64b819e1a0 con 0x7f64b80ff8b0 2024-09-13T18:21:53.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.510+0000 7f64c0583640 1 -- 172.21.15.114:0/2351996700 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64b81061c0 con 0x7f64b80ff8b0 2024-09-13T18:21:53.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.511+0000 7f64af7fe640 1 -- 172.21.15.114:0/2351996700 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f64a8004c50 con 0x7f64b80ff8b0 2024-09-13T18:21:53.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.512+0000 7f64af7fe640 1 --2- 172.21.15.114:0/2351996700 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f64a40a1120 0x7f64a40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:53.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.512+0000 7f64be31f640 1 --2- 172.21.15.114:0/2351996700 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f64a40a1120 0x7f64a40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:53.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.512+0000 7f64af7fe640 1 -- 172.21.15.114:0/2351996700 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f64a80edeb0 con 0x7f64b80ff8b0 2024-09-13T18:21:53.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.512+0000 7f64be31f640 1 --2- 172.21.15.114:0/2351996700 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f64a40a1120 0x7f64a40a35e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f64b8197130 tx=0x7f64b4005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:53.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.516+0000 7f64af7fe640 1 -- 172.21.15.114:0/2351996700 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f64a80b7190 con 0x7f64b80ff8b0 2024-09-13T18:21:53.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.680+0000 7f64c0583640 1 -- 172.21.15.114:0/2351996700 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f64b802c3e0 con 0x7f64b80ff8b0 2024-09-13T18:21:53.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.680+0000 7f64af7fe640 1 -- 172.21.15.114:0/2351996700 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7f64a80400a0 con 0x7f64b80ff8b0 2024-09-13T18:21:53.684 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 64 2024-09-13T18:21:53.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.682+0000 7f64c0583640 1 -- 172.21.15.114:0/2351996700 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f64a40a1120 msgr2=0x7f64a40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:53.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.682+0000 7f64c0583640 1 --2- 172.21.15.114:0/2351996700 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f64a40a1120 0x7f64a40a35e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f64b8197130 tx=0x7f64b4005f90 comp rx=0 tx=0).stop 2024-09-13T18:21:53.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.682+0000 7f64c0583640 1 -- 172.21.15.114:0/2351996700 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f64b80ff8b0 msgr2=0x7f64b81961a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:53.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.682+0000 7f64c0583640 1 --2- 172.21.15.114:0/2351996700 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f64b80ff8b0 0x7f64b81961a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f64a8005ab0 tx=0x7f64a80042f0 comp rx=0 tx=0).stop 2024-09-13T18:21:53.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.682+0000 7f64c0583640 1 -- 172.21.15.114:0/2351996700 shutdown_connections 2024-09-13T18:21:53.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.682+0000 7f64c0583640 1 --2- 172.21.15.114:0/2351996700 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f64a40a1120 0x7f64a40a35e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:53.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.682+0000 7f64c0583640 1 --2- 172.21.15.114:0/2351996700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64b81002a0 0x7f64b819d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:53.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.682+0000 7f64c0583640 1 --2- 172.21.15.114:0/2351996700 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f64b80ff8b0 0x7f64b81961a0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:53.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.682+0000 7f64c0583640 1 --2- 172.21.15.114:0/2351996700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64b80feef0 0x7f64b8195c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:53.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.682+0000 7f64c0583640 1 -- 172.21.15.114:0/2351996700 >> 172.21.15.114:0/2351996700 conn(0x7f64b80f9b70 msgr2=0x7f64b8103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:53.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.682+0000 7f64c0583640 1 -- 172.21.15.114:0/2351996700 shutdown_connections 2024-09-13T18:21:53.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.682+0000 7f64c0583640 1 -- 172.21.15.114:0/2351996700 wait complete. 2024-09-13T18:21:53.696 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T18:21:53.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.794+0000 7f66251a0640 1 -- 172.21.15.114:0/3211308507 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66200ff840 msgr2=0x7f66200ffcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:53.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.794+0000 7f66251a0640 1 --2- 172.21.15.114:0/3211308507 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66200ff840 0x7f66200ffcf0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6618009ca0 tx=0x7f661802a8c0 comp rx=0 tx=0).stop 2024-09-13T18:21:53.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.794+0000 7f66251a0640 1 -- 172.21.15.114:0/3211308507 shutdown_connections 2024-09-13T18:21:53.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.794+0000 7f66251a0640 1 --2- 172.21.15.114:0/3211308507 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66200ff840 0x7f66200ffcf0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:53.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.794+0000 7f66251a0640 1 --2- 172.21.15.114:0/3211308507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66200feea0 0x7f66200ff300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:53.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.794+0000 7f66251a0640 1 --2- 172.21.15.114:0/3211308507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66200fe560 0x7f66200fe960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:53.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.794+0000 7f66251a0640 1 -- 172.21.15.114:0/3211308507 >> 172.21.15.114:0/3211308507 conn(0x7f66200fa330 msgr2=0x7f66200fc790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:53.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.794+0000 7f66251a0640 1 -- 172.21.15.114:0/3211308507 shutdown_connections 2024-09-13T18:21:53.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.794+0000 7f66251a0640 1 -- 172.21.15.114:0/3211308507 wait complete. 2024-09-13T18:21:53.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.795+0000 7f66251a0640 1 Processor -- start 2024-09-13T18:21:53.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.795+0000 7f66251a0640 1 -- start start 2024-09-13T18:21:53.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.795+0000 7f66251a0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66200fe560 0x7f6620197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:53.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.795+0000 7f66251a0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66200feea0 0x7f6620198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:53.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.795+0000 7f66251a0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66200ff840 0x7f662019f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:53.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.795+0000 7f66251a0640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662019f8f0 con 0x7f66200fe560 2024-09-13T18:21:53.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.795+0000 7f66251a0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662019fa60 con 0x7f66200feea0 2024-09-13T18:21:53.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.795+0000 7f66251a0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662019fc20 con 0x7f66200ff840 2024-09-13T18:21:53.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.795+0000 7f661ffff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66200fe560 0x7f6620197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:53.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.795+0000 7f661ffff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66200fe560 0x7f6620197e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33678/0 (socket says 172.21.15.114:33678) 2024-09-13T18:21:53.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.795+0000 7f661ffff640 1 -- 172.21.15.114:0/3839115729 learned_addr learned my addr 172.21.15.114:0/3839115729 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:53.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.796+0000 7f661f7fe640 1 --2- 172.21.15.114:0/3839115729 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66200feea0 0x7f6620198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:53.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.796+0000 7f661ffff640 1 -- 172.21.15.114:0/3839115729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66200ff840 msgr2=0x7f662019f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:53.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.796+0000 7f661ffff640 1 --2- 172.21.15.114:0/3839115729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66200ff840 0x7f662019f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:53.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.796+0000 7f661ffff640 1 -- 172.21.15.114:0/3839115729 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66200feea0 msgr2=0x7f6620198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:53.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.796+0000 7f661ffff640 1 --2- 172.21.15.114:0/3839115729 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66200feea0 0x7f6620198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:53.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.796+0000 7f661ffff640 1 -- 172.21.15.114:0/3839115729 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6618009900 con 0x7f66200fe560 2024-09-13T18:21:53.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.796+0000 7f661f7fe640 1 --2- 172.21.15.114:0/3839115729 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66200feea0 0x7f6620198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:21:53.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.796+0000 7f661ffff640 1 --2- 172.21.15.114:0/3839115729 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66200fe560 0x7f6620197e00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f661000e9f0 tx=0x7f661000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:53.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.796+0000 7f661d7fa640 1 -- 172.21.15.114:0/3839115729 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6610009720 con 0x7f66200fe560 2024-09-13T18:21:53.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.796+0000 7f661d7fa640 1 -- 172.21.15.114:0/3839115729 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f661000cd80 con 0x7f66200fe560 2024-09-13T18:21:53.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.796+0000 7f66251a0640 1 -- 172.21.15.114:0/3839115729 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f662019ff00 con 0x7f66200fe560 2024-09-13T18:21:53.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.796+0000 7f661d7fa640 1 -- 172.21.15.114:0/3839115729 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6610010430 con 0x7f66200fe560 2024-09-13T18:21:53.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.796+0000 7f66251a0640 1 -- 172.21.15.114:0/3839115729 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66201a0320 con 0x7f66200fe560 2024-09-13T18:21:53.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.797+0000 7f66251a0640 1 -- 172.21.15.114:0/3839115729 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6620109580 con 0x7f66200fe560 2024-09-13T18:21:53.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.798+0000 7f661d7fa640 1 -- 172.21.15.114:0/3839115729 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f6610004f90 con 0x7f66200fe560 2024-09-13T18:21:53.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.799+0000 7f661d7fa640 1 --2- 172.21.15.114:0/3839115729 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f66080a1120 0x7f66080a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:53.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.799+0000 7f661f7fe640 1 --2- 172.21.15.114:0/3839115729 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f66080a1120 0x7f66080a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:53.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.799+0000 7f661d7fa640 1 -- 172.21.15.114:0/3839115729 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f6610014070 con 0x7f66200fe560 2024-09-13T18:21:53.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.799+0000 7f661f7fe640 1 --2- 172.21.15.114:0/3839115729 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f66080a1120 0x7f66080a35e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f66140058c0 tx=0x7f6614005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:53.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.802+0000 7f661d7fa640 1 -- 172.21.15.114:0/3839115729 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f6610096770 con 0x7f66200fe560 2024-09-13T18:21:53.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.968+0000 7f66251a0640 1 -- 172.21.15.114:0/3839115729 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f662010d660 con 0x7f66200fe560 2024-09-13T18:21:53.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.969+0000 7f661d7fa640 1 -- 172.21.15.114:0/3839115729 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7f6610099d90 con 0x7f66200fe560 2024-09-13T18:21:53.972 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2024-09-13T18:21:53.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.971+0000 7f66251a0640 1 -- 172.21.15.114:0/3839115729 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f66080a1120 msgr2=0x7f66080a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:53.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.971+0000 7f66251a0640 1 --2- 172.21.15.114:0/3839115729 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f66080a1120 0x7f66080a35e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f66140058c0 tx=0x7f6614005850 comp rx=0 tx=0).stop 2024-09-13T18:21:53.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.971+0000 7f66251a0640 1 -- 172.21.15.114:0/3839115729 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66200fe560 msgr2=0x7f6620197e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:53.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.971+0000 7f66251a0640 1 --2- 172.21.15.114:0/3839115729 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66200fe560 0x7f6620197e00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f661000e9f0 tx=0x7f661000edf0 comp rx=0 tx=0).stop 2024-09-13T18:21:53.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.971+0000 7f66251a0640 1 -- 172.21.15.114:0/3839115729 shutdown_connections 2024-09-13T18:21:53.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.971+0000 7f66251a0640 1 --2- 172.21.15.114:0/3839115729 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f66080a1120 0x7f66080a35e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:53.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.971+0000 7f66251a0640 1 --2- 172.21.15.114:0/3839115729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66200ff840 0x7f662019f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:53.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.971+0000 7f66251a0640 1 --2- 172.21.15.114:0/3839115729 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66200feea0 0x7f6620198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:53.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.971+0000 7f66251a0640 1 --2- 172.21.15.114:0/3839115729 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66200fe560 0x7f6620197e00 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:53.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.971+0000 7f66251a0640 1 -- 172.21.15.114:0/3839115729 >> 172.21.15.114:0/3839115729 conn(0x7f66200fa330 msgr2=0x7f66200fbc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:53.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.971+0000 7f66251a0640 1 -- 172.21.15.114:0/3839115729 shutdown_connections 2024-09-13T18:21:53.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:53.971+0000 7f66251a0640 1 -- 172.21.15.114:0/3839115729 wait complete. 2024-09-13T18:21:53.985 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:21:54.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.080+0000 7ff103b65640 1 -- 172.21.15.114:0/2840271591 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0fc100850 msgr2=0x7ff0fc1070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:54.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.080+0000 7ff103b65640 1 --2- 172.21.15.114:0/2840271591 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0fc100850 0x7ff0fc1070e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4009ca0 tx=0x7ff0f402a8c0 comp rx=0 tx=0).stop 2024-09-13T18:21:54.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.081+0000 7ff103b65640 1 -- 172.21.15.114:0/2840271591 shutdown_connections 2024-09-13T18:21:54.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.081+0000 7ff103b65640 1 --2- 172.21.15.114:0/2840271591 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0fc100850 0x7ff0fc1070e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:54.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.081+0000 7ff103b65640 1 --2- 172.21.15.114:0/2840271591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0fc0ffe90 0x7ff0fc100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:54.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.081+0000 7ff103b65640 1 --2- 172.21.15.114:0/2840271591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0fc0feca0 0x7ff0fc0ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:54.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.081+0000 7ff103b65640 1 -- 172.21.15.114:0/2840271591 >> 172.21.15.114:0/2840271591 conn(0x7ff0fc0fa440 msgr2=0x7ff0fc0fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:54.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.081+0000 7ff103b65640 1 -- 172.21.15.114:0/2840271591 shutdown_connections 2024-09-13T18:21:54.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.081+0000 7ff103b65640 1 -- 172.21.15.114:0/2840271591 wait complete. 2024-09-13T18:21:54.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.081+0000 7ff103b65640 1 Processor -- start 2024-09-13T18:21:54.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.081+0000 7ff103b65640 1 -- start start 2024-09-13T18:21:54.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.081+0000 7ff103b65640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0fc0feca0 0x7ff0fc068110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:54.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.081+0000 7ff103b65640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0fc0ffe90 0x7ff0fc19a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:54.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.081+0000 7ff103b65640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0fc100850 0x7ff0fc19cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:54.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.081+0000 7ff103b65640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0fc19d770 con 0x7ff0fc0ffe90 2024-09-13T18:21:54.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.081+0000 7ff103b65640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0fc19d8e0 con 0x7ff0fc100850 2024-09-13T18:21:54.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.081+0000 7ff103b65640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0fc19daa0 con 0x7ff0fc0feca0 2024-09-13T18:21:54.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.081+0000 7ff101100640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0fc0ffe90 0x7ff0fc19a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:54.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.081+0000 7ff101100640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0fc0ffe90 0x7ff0fc19a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33696/0 (socket says 172.21.15.114:33696) 2024-09-13T18:21:54.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.081+0000 7ff101100640 1 -- 172.21.15.114:0/2946645366 learned_addr learned my addr 172.21.15.114:0/2946645366 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:54.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.082+0000 7ff101901640 1 --2- 172.21.15.114:0/2946645366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0fc0feca0 0x7ff0fc068110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:54.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.082+0000 7ff102102640 1 --2- 172.21.15.114:0/2946645366 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0fc100850 0x7ff0fc19cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:54.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.082+0000 7ff101100640 1 -- 172.21.15.114:0/2946645366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0fc0feca0 msgr2=0x7ff0fc068110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:54.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.082+0000 7ff101100640 1 --2- 172.21.15.114:0/2946645366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0fc0feca0 0x7ff0fc068110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:54.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.082+0000 7ff101100640 1 -- 172.21.15.114:0/2946645366 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0fc100850 msgr2=0x7ff0fc19cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:54.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.082+0000 7ff101100640 1 --2- 172.21.15.114:0/2946645366 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0fc100850 0x7ff0fc19cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:54.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.082+0000 7ff101100640 1 -- 172.21.15.114:0/2946645366 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f4009900 con 0x7ff0fc0ffe90 2024-09-13T18:21:54.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.082+0000 7ff101100640 1 --2- 172.21.15.114:0/2946645366 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0fc0ffe90 0x7ff0fc19a970 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff0f800d950 tx=0x7ff0f800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:54.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.082+0000 7ff0eeffd640 1 -- 172.21.15.114:0/2946645366 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8009c30 con 0x7ff0fc0ffe90 2024-09-13T18:21:54.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.082+0000 7ff0eeffd640 1 -- 172.21.15.114:0/2946645366 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff0f8009d90 con 0x7ff0fc0ffe90 2024-09-13T18:21:54.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.082+0000 7ff103b65640 1 -- 172.21.15.114:0/2946645366 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0fc19dd20 con 0x7ff0fc0ffe90 2024-09-13T18:21:54.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.082+0000 7ff0eeffd640 1 -- 172.21.15.114:0/2946645366 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f800f620 con 0x7ff0fc0ffe90 2024-09-13T18:21:54.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.082+0000 7ff103b65640 1 -- 172.21.15.114:0/2946645366 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0fc19e1d0 con 0x7ff0fc0ffe90 2024-09-13T18:21:54.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.083+0000 7ff103b65640 1 -- 172.21.15.114:0/2946645366 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0fc1060d0 con 0x7ff0fc0ffe90 2024-09-13T18:21:54.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.084+0000 7ff0eeffd640 1 -- 172.21.15.114:0/2946645366 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7ff0f80109f0 con 0x7ff0fc0ffe90 2024-09-13T18:21:54.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.085+0000 7ff0eeffd640 1 --2- 172.21.15.114:0/2946645366 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7ff0e80a1120 0x7ff0e80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:54.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.085+0000 7ff0eeffd640 1 -- 172.21.15.114:0/2946645366 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7ff0f80cc980 con 0x7ff0fc0ffe90 2024-09-13T18:21:54.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.085+0000 7ff101901640 1 --2- 172.21.15.114:0/2946645366 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7ff0e80a1120 0x7ff0e80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:54.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.085+0000 7ff101901640 1 --2- 172.21.15.114:0/2946645366 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7ff0e80a1120 0x7ff0e80a35e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc0ffcf0 tx=0x7ff0f000a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:54.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.089+0000 7ff0eeffd640 1 -- 172.21.15.114:0/2946645366 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7ff0f8095c90 con 0x7ff0fc0ffe90 2024-09-13T18:21:54.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:53 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/957473066' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T18:21:54.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:53 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2286177912' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T18:21:54.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:53 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2351996700' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T18:21:54.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.289+0000 7ff103b65640 1 -- 172.21.15.114:0/2946645366 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff0fc10a3f0 con 0x7ff0fc0ffe90 2024-09-13T18:21:54.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.290+0000 7ff0eeffd640 1 -- 172.21.15.114:0/2946645366 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10930 (secure 0 0 0) 0x7ff0fc10a3f0 con 0x7ff0fc0ffe90 2024-09-13T18:21:54.293 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:21:54.293 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4162817497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4162817497},{"type":"v1","addr":"172.21.15.114:6835","nonce":4162817497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/2435428478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":2435428478},{"type":"v1","addr":"172.21.15.114:6837","nonce":2435428478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T18:20:01.895117+0000","modified":"2024-09-13T18:20:29.250121+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24482,"mds_2":24475,"mds_3":24458,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/2046516451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":2046516451},{"type":"v1","addr":"172.21.15.114:6839","nonce":2046516451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":10,"addr":"172.21.15.114:6841/1956923446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":1956923446},{"type":"v1","addr":"172.21.15.114:6841","nonce":1956923446}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24391":{"gid":24391,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/507301496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":507301496},{"type":"v1","addr":"172.21.15.204:6833","nonce":507301496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4028657219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4028657219},{"type":"v1","addr":"172.21.15.134:6833","nonce":4028657219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/57029119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":57029119},{"type":"v1","addr":"172.21.15.204:6835","nonce":57029119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3068880939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3068880939},{"type":"v1","addr":"172.21.15.204:6837","nonce":3068880939}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"e","rank":3,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.134:6835/1150003275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1150003275},{"type":"v1","addr":"172.21.15.134:6835","nonce":1150003275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/344883912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":344883912},{"type":"v1","addr":"172.21.15.204:6839","nonce":344883912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6837/891692210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":891692210},{"type":"v1","addr":"172.21.15.134:6837","nonce":891692210}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6839/3466309724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3466309724},{"type":"v1","addr":"172.21.15.134:6839","nonce":3466309724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:21:54.293 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 29 2024-09-13T18:21:54.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.292+0000 7ff103b65640 1 -- 172.21.15.114:0/2946645366 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7ff0e80a1120 msgr2=0x7ff0e80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:54.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.292+0000 7ff103b65640 1 --2- 172.21.15.114:0/2946645366 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7ff0e80a1120 0x7ff0e80a35e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc0ffcf0 tx=0x7ff0f000a2d0 comp rx=0 tx=0).stop 2024-09-13T18:21:54.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.292+0000 7ff103b65640 1 -- 172.21.15.114:0/2946645366 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0fc0ffe90 msgr2=0x7ff0fc19a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:54.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.292+0000 7ff103b65640 1 --2- 172.21.15.114:0/2946645366 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0fc0ffe90 0x7ff0fc19a970 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff0f800d950 tx=0x7ff0f800de10 comp rx=0 tx=0).stop 2024-09-13T18:21:54.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.292+0000 7ff103b65640 1 -- 172.21.15.114:0/2946645366 shutdown_connections 2024-09-13T18:21:54.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.292+0000 7ff103b65640 1 --2- 172.21.15.114:0/2946645366 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7ff0e80a1120 0x7ff0e80a35e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:54.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.292+0000 7ff103b65640 1 --2- 172.21.15.114:0/2946645366 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0fc100850 0x7ff0fc19cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:54.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.292+0000 7ff103b65640 1 --2- 172.21.15.114:0/2946645366 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0fc0ffe90 0x7ff0fc19a970 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:54.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.292+0000 7ff103b65640 1 --2- 172.21.15.114:0/2946645366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0fc0feca0 0x7ff0fc068110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:54.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.292+0000 7ff103b65640 1 -- 172.21.15.114:0/2946645366 >> 172.21.15.114:0/2946645366 conn(0x7ff0fc0fa440 msgr2=0x7ff0fc1032a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:54.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.292+0000 7ff103b65640 1 -- 172.21.15.114:0/2946645366 shutdown_connections 2024-09-13T18:21:54.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.292+0000 7ff103b65640 1 -- 172.21.15.114:0/2946645366 wait complete. 2024-09-13T18:21:54.307 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:21:54.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:53 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/957473066' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T18:21:54.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:53 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/2286177912' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T18:21:54.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:53 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/2351996700' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T18:21:54.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.402+0000 7f0fd6ad7640 1 -- 172.21.15.114:0/1476307884 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fd0100850 msgr2=0x7f0fd01070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:54.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.402+0000 7f0fd6ad7640 1 --2- 172.21.15.114:0/1476307884 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fd0100850 0x7f0fd01070e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0fc8009ca0 tx=0x7f0fc802a8c0 comp rx=0 tx=0).stop 2024-09-13T18:21:54.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.403+0000 7f0fd6ad7640 1 -- 172.21.15.114:0/1476307884 shutdown_connections 2024-09-13T18:21:54.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.403+0000 7f0fd6ad7640 1 --2- 172.21.15.114:0/1476307884 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fd0100850 0x7f0fd01070e0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:54.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.403+0000 7f0fd6ad7640 1 --2- 172.21.15.114:0/1476307884 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fd00ffe90 0x7f0fd0100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:54.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.403+0000 7f0fd6ad7640 1 --2- 172.21.15.114:0/1476307884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0fd00feca0 0x7f0fd00ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:54.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.403+0000 7f0fd6ad7640 1 -- 172.21.15.114:0/1476307884 >> 172.21.15.114:0/1476307884 conn(0x7f0fd00fa440 msgr2=0x7f0fd00fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:54.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.403+0000 7f0fd6ad7640 1 -- 172.21.15.114:0/1476307884 shutdown_connections 2024-09-13T18:21:54.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.403+0000 7f0fd6ad7640 1 -- 172.21.15.114:0/1476307884 wait complete. 2024-09-13T18:21:54.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.403+0000 7f0fd6ad7640 1 Processor -- start 2024-09-13T18:21:54.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.403+0000 7f0fd6ad7640 1 -- start start 2024-09-13T18:21:54.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.403+0000 7f0fd6ad7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0fd00feca0 0x7f0fd0194b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:54.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.403+0000 7f0fd6ad7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fd00ffe90 0x7f0fd0195060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:54.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.403+0000 7f0fd6ad7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fd0100850 0x7f0fd019c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:54.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.403+0000 7f0fd6ad7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd019c610 con 0x7f0fd00ffe90 2024-09-13T18:21:54.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.403+0000 7f0fd6ad7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd019c780 con 0x7f0fd00feca0 2024-09-13T18:21:54.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.403+0000 7f0fd6ad7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd019c940 con 0x7f0fd0100850 2024-09-13T18:21:54.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.403+0000 7f0fcffff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fd00ffe90 0x7f0fd0195060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:54.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.403+0000 7f0fcffff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fd00ffe90 0x7f0fd0195060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33714/0 (socket says 172.21.15.114:33714) 2024-09-13T18:21:54.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.403+0000 7f0fcffff640 1 -- 172.21.15.114:0/3325249249 learned_addr learned my addr 172.21.15.114:0/3325249249 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:54.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.404+0000 7f0fd4873640 1 --2- 172.21.15.114:0/3325249249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0fd00feca0 0x7f0fd0194b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:54.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.404+0000 7f0fcffff640 1 -- 172.21.15.114:0/3325249249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fd0100850 msgr2=0x7f0fd019c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:54.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.404+0000 7f0fcffff640 1 --2- 172.21.15.114:0/3325249249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fd0100850 0x7f0fd019c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:54.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.404+0000 7f0fcffff640 1 -- 172.21.15.114:0/3325249249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0fd00feca0 msgr2=0x7f0fd0194b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:54.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.404+0000 7f0fcffff640 1 --2- 172.21.15.114:0/3325249249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0fd00feca0 0x7f0fd0194b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:54.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.404+0000 7f0fcffff640 1 -- 172.21.15.114:0/3325249249 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc8009900 con 0x7f0fd00ffe90 2024-09-13T18:21:54.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.404+0000 7f0fd4873640 1 --2- 172.21.15.114:0/3325249249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0fd00feca0 0x7f0fd0194b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:21:54.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.404+0000 7f0fcffff640 1 --2- 172.21.15.114:0/3325249249 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fd00ffe90 0x7f0fd0195060 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc00d950 tx=0x7f0fbc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:54.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.404+0000 7f0fcdffb640 1 -- 172.21.15.114:0/3325249249 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc009a30 con 0x7f0fd00ffe90 2024-09-13T18:21:54.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.404+0000 7f0fcdffb640 1 -- 172.21.15.114:0/3325249249 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0fbc0044b0 con 0x7f0fd00ffe90 2024-09-13T18:21:54.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.404+0000 7f0fd6ad7640 1 -- 172.21.15.114:0/3325249249 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fd019cc20 con 0x7f0fd00ffe90 2024-09-13T18:21:54.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.404+0000 7f0fcdffb640 1 -- 172.21.15.114:0/3325249249 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc0047a0 con 0x7f0fd00ffe90 2024-09-13T18:21:54.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.404+0000 7f0fd6ad7640 1 -- 172.21.15.114:0/3325249249 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fd019d130 con 0x7f0fd00ffe90 2024-09-13T18:21:54.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.405+0000 7f0fd6ad7640 1 -- 172.21.15.114:0/3325249249 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fd01060d0 con 0x7f0fd00ffe90 2024-09-13T18:21:54.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.406+0000 7f0fcdffb640 1 -- 172.21.15.114:0/3325249249 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f0fbc004020 con 0x7f0fd00ffe90 2024-09-13T18:21:54.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.407+0000 7f0fcdffb640 1 --2- 172.21.15.114:0/3325249249 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f0fc00a1120 0x7f0fc00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:54.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.407+0000 7f0fcdffb640 1 -- 172.21.15.114:0/3325249249 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f0fbc0cc890 con 0x7f0fd00ffe90 2024-09-13T18:21:54.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.407+0000 7f0fd4873640 1 --2- 172.21.15.114:0/3325249249 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f0fc00a1120 0x7f0fc00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:54.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.407+0000 7f0fd4873640 1 --2- 172.21.15.114:0/3325249249 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f0fc00a1120 0x7f0fc00a35e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0fc40058c0 tx=0x7f0fc4005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:54.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.411+0000 7f0fcdffb640 1 -- 172.21.15.114:0/3325249249 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f0fbc095b50 con 0x7f0fd00ffe90 2024-09-13T18:21:54.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:53 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/957473066' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T18:21:54.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:53 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/2286177912' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T18:21:54.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:53 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/2351996700' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T18:21:54.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.578+0000 7f0fd6ad7640 1 -- 172.21.15.114:0/3325249249 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0fd002c480 con 0x7f0fd00ffe90 2024-09-13T18:21:54.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.580+0000 7f0fcdffb640 1 -- 172.21.15.114:0/3325249249 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24696 (secure 0 0 0) 0x7f0fbc099170 con 0x7f0fd00ffe90 2024-09-13T18:21:54.583 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:21:54.583 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":93,"fsid":"97e7edf2-71fb-11ef-bceb-c7b262605968","created":"2024-09-13T18:12:36.360625+0000","modified":"2024-09-13T18:21:28.143797+0000","last_up_change":"2024-09-13T18:19:12.350370+0000","last_in_change":"2024-09-13T18:18:52.321596+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:15:40.129659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:20:00.024407+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:20:01.085549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T18:20:04.334216+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9914f01a-9f6f-4878-b955-83570199c354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6803","nonce":3499525826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6805","nonce":3499525826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6809","nonce":3499525826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6807","nonce":3499525826}]},"public_addr":"172.21.15.114:6803/3499525826","cluster_addr":"172.21.15.114:6805/3499525826","heartbeat_back_addr":"172.21.15.114:6809/3499525826","heartbeat_front_addr":"172.21.15.114:6807/3499525826","state":["exists","up"]},{"osd":1,"uuid":"ef65e012-7469-4c20-b354-17022bca63c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6801","nonce":4264264866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6803","nonce":4264264866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6807","nonce":4264264866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6805","nonce":4264264866}]},"public_addr":"172.21.15.134:6801/4264264866","cluster_addr":"172.21.15.134:6803/4264264866","heartbeat_back_addr":"172.21.15.134:6807/4264264866","heartbeat_front_addr":"172.21.15.134:6805/4264264866","state":["exists","up"]},{"osd":2,"uuid":"578213fd-715f-4be7-91fa-254f3d0e3022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6801","nonce":2288807707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6803","nonce":2288807707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6807","nonce":2288807707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6805","nonce":2288807707}]},"public_addr":"172.21.15.204:6801/2288807707","cluster_addr":"172.21.15.204:6803/2288807707","heartbeat_back_addr":"172.21.15.204:6807/2288807707","heartbeat_front_addr":"172.21.15.204:6805/2288807707","state":["exists","up"]},{"osd":3,"uuid":"b64aab3f-1e52-4828-8397-74af5e05f2cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6811","nonce":2904204254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6813","nonce":2904204254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6817","nonce":2904204254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6815","nonce":2904204254}]},"public_addr":"172.21.15.114:6811/2904204254","cluster_addr":"172.21.15.114:6813/2904204254","heartbeat_back_addr":"172.21.15.114:6817/2904204254","heartbeat_front_addr":"172.21.15.114:6815/2904204254","state":["exists","up"]},{"osd":4,"uuid":"f38943d6-39c0-42e1-b427-cb6bb90ac38a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6809","nonce":1773945333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6811","nonce":1773945333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6815","nonce":1773945333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6813","nonce":1773945333}]},"public_addr":"172.21.15.134:6809/1773945333","cluster_addr":"172.21.15.134:6811/1773945333","heartbeat_back_addr":"172.21.15.134:6815/1773945333","heartbeat_front_addr":"172.21.15.134:6813/1773945333","state":["exists","up"]},{"osd":5,"uuid":"bb38f361-8e00-4178-9752-5ddbf3af20e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6809","nonce":2371778055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6811","nonce":2371778055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6815","nonce":2371778055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6813","nonce":2371778055}]},"public_addr":"172.21.15.204:6809/2371778055","cluster_addr":"172.21.15.204:6811/2371778055","heartbeat_back_addr":"172.21.15.204:6815/2371778055","heartbeat_front_addr":"172.21.15.204:6813/2371778055","state":["exists","up"]},{"osd":6,"uuid":"93d40a03-e7ff-4a3f-9efa-5c5403c9983d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6819","nonce":1484325044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6821","nonce":1484325044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6825","nonce":1484325044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6823","nonce":1484325044}]},"public_addr":"172.21.15.114:6819/1484325044","cluster_addr":"172.21.15.114:6821/1484325044","heartbeat_back_addr":"172.21.15.114:6825/1484325044","heartbeat_front_addr":"172.21.15.114:6823/1484325044","state":["exists","up"]},{"osd":7,"uuid":"982f629a-1b95-4be1-8e75-5ddce1ad77b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6817","nonce":799246614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6819","nonce":799246614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6823","nonce":799246614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6821","nonce":799246614}]},"public_addr":"172.21.15.134:6817/799246614","cluster_addr":"172.21.15.134:6819/799246614","heartbeat_back_addr":"172.21.15.134:6823/799246614","heartbeat_front_addr":"172.21.15.134:6821/799246614","state":["exists","up"]},{"osd":8,"uuid":"409beb9b-8227-4dad-a4a0-f0e5ce70dc5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6817","nonce":858331805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6819","nonce":858331805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6823","nonce":858331805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6821","nonce":858331805}]},"public_addr":"172.21.15.204:6817/858331805","cluster_addr":"172.21.15.204:6819/858331805","heartbeat_back_addr":"172.21.15.204:6823/858331805","heartbeat_front_addr":"172.21.15.204:6821/858331805","state":["exists","up"]},{"osd":9,"uuid":"a345a15c-b1fc-4877-b445-418eebf0d0ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6827","nonce":2172392854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6829","nonce":2172392854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6833","nonce":2172392854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6831","nonce":2172392854}]},"public_addr":"172.21.15.114:6827/2172392854","cluster_addr":"172.21.15.114:6829/2172392854","heartbeat_back_addr":"172.21.15.114:6833/2172392854","heartbeat_front_addr":"172.21.15.114:6831/2172392854","state":["exists","up"]},{"osd":10,"uuid":"464d995f-d90b-4caa-addb-1f7562bd74b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6825","nonce":3073513468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6827","nonce":3073513468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6831","nonce":3073513468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6829","nonce":3073513468}]},"public_addr":"172.21.15.134:6825/3073513468","cluster_addr":"172.21.15.134:6827/3073513468","heartbeat_back_addr":"172.21.15.134:6831/3073513468","heartbeat_front_addr":"172.21.15.134:6829/3073513468","state":["exists","up"]},{"osd":11,"uuid":"71dd794d-326c-477f-8d02-ee32bff1350d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6825","nonce":3239521765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6827","nonce":3239521765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6831","nonce":3239521765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6829","nonce":3239521765}]},"public_addr":"172.21.15.204:6825/3239521765","cluster_addr":"172.21.15.204:6827/3239521765","heartbeat_back_addr":"172.21.15.204:6831/3239521765","heartbeat_front_addr":"172.21.15.204:6829/3239521765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:14:54.542687+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:15:15.746055+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:15:38.371131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:01.383932+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:24.796237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:48.422214+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:11.284323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:34.570365+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:58.232855+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:18:21.609596+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:18:45.565859+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:19:10.017969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:6801/4281866858":"2024-09-14T18:13:25.823027+0000","172.21.15.114:6800/4281866858":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2949422849":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2034742251":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/722571999":"2024-09-14T18:13:03.521116+0000","172.21.15.114:6801/830738794":"2024-09-14T18:13:03.521116+0000","172.21.15.114:6800/830738794":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/2828364892":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/4257828906":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/3544951322":"2024-09-14T18:13:03.521116+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:21:54.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.582+0000 7f0fd6ad7640 1 -- 172.21.15.114:0/3325249249 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f0fc00a1120 msgr2=0x7f0fc00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:54.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.582+0000 7f0fd6ad7640 1 --2- 172.21.15.114:0/3325249249 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f0fc00a1120 0x7f0fc00a35e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0fc40058c0 tx=0x7f0fc4005850 comp rx=0 tx=0).stop 2024-09-13T18:21:54.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.582+0000 7f0fd6ad7640 1 -- 172.21.15.114:0/3325249249 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fd00ffe90 msgr2=0x7f0fd0195060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:54.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.582+0000 7f0fd6ad7640 1 --2- 172.21.15.114:0/3325249249 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fd00ffe90 0x7f0fd0195060 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc00d950 tx=0x7f0fbc00de10 comp rx=0 tx=0).stop 2024-09-13T18:21:54.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.582+0000 7f0fd6ad7640 1 -- 172.21.15.114:0/3325249249 shutdown_connections 2024-09-13T18:21:54.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.582+0000 7f0fd6ad7640 1 --2- 172.21.15.114:0/3325249249 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f0fc00a1120 0x7f0fc00a35e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:54.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.582+0000 7f0fd6ad7640 1 --2- 172.21.15.114:0/3325249249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fd0100850 0x7f0fd019c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:54.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.582+0000 7f0fd6ad7640 1 --2- 172.21.15.114:0/3325249249 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fd00ffe90 0x7f0fd0195060 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:54.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.582+0000 7f0fd6ad7640 1 --2- 172.21.15.114:0/3325249249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0fd00feca0 0x7f0fd0194b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:54.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.582+0000 7f0fd6ad7640 1 -- 172.21.15.114:0/3325249249 >> 172.21.15.114:0/3325249249 conn(0x7f0fd00fa440 msgr2=0x7f0fd01032a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:54.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.582+0000 7f0fd6ad7640 1 -- 172.21.15.114:0/3325249249 shutdown_connections 2024-09-13T18:21:54.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.582+0000 7f0fd6ad7640 1 -- 172.21.15.114:0/3325249249 wait complete. 2024-09-13T18:21:54.597 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T18:15:40.129659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T18:20:00.024407+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T18:20:01.085549+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T18:20:04.334216+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T18:21:54.597 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T18:21:54.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.695+0000 7fac7629d640 1 -- 172.21.15.114:0/3304381731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac700fef50 msgr2=0x7fac700ff3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:54.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.695+0000 7fac7629d640 1 --2- 172.21.15.114:0/3304381731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac700fef50 0x7fac700ff3d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fac58009900 tx=0x7fac5802a410 comp rx=0 tx=0).stop 2024-09-13T18:21:54.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.695+0000 7fac7629d640 1 -- 172.21.15.114:0/3304381731 shutdown_connections 2024-09-13T18:21:54.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.695+0000 7fac7629d640 1 --2- 172.21.15.114:0/3304381731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac700ff910 0x7fac701071b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:54.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.695+0000 7fac7629d640 1 --2- 172.21.15.114:0/3304381731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac700fef50 0x7fac700ff3d0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:54.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.695+0000 7fac7629d640 1 --2- 172.21.15.114:0/3304381731 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fac70100220 0x7fac70100620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:54.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.695+0000 7fac7629d640 1 -- 172.21.15.114:0/3304381731 >> 172.21.15.114:0/3304381731 conn(0x7fac700f9b70 msgr2=0x7fac700fbfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:54.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.695+0000 7fac7629d640 1 -- 172.21.15.114:0/3304381731 shutdown_connections 2024-09-13T18:21:54.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.695+0000 7fac7629d640 1 -- 172.21.15.114:0/3304381731 wait complete. 2024-09-13T18:21:54.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.695+0000 7fac7629d640 1 Processor -- start 2024-09-13T18:21:54.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.695+0000 7fac7629d640 1 -- start start 2024-09-13T18:21:54.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.695+0000 7fac7629d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fac700fef50 0x7fac70194c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:54.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.695+0000 7fac7629d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac700ff910 0x7fac70195170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:54.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.695+0000 7fac7629d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac70100220 0x7fac7019c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:54.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.695+0000 7fac7629d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac7019c710 con 0x7fac700ff910 2024-09-13T18:21:54.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.695+0000 7fac7629d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac7019c880 con 0x7fac70100220 2024-09-13T18:21:54.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.695+0000 7fac7629d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac7019ca40 con 0x7fac700fef50 2024-09-13T18:21:54.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.696+0000 7fac67fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac700ff910 0x7fac70195170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:54.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.696+0000 7fac67fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac700ff910 0x7fac70195170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33732/0 (socket says 172.21.15.114:33732) 2024-09-13T18:21:54.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.696+0000 7fac67fff640 1 -- 172.21.15.114:0/2616492707 learned_addr learned my addr 172.21.15.114:0/2616492707 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:54.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.696+0000 7fac6ffff640 1 --2- 172.21.15.114:0/2616492707 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fac700fef50 0x7fac70194c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:54.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.696+0000 7fac7483a640 1 --2- 172.21.15.114:0/2616492707 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac70100220 0x7fac7019c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:54.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.696+0000 7fac67fff640 1 -- 172.21.15.114:0/2616492707 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fac700fef50 msgr2=0x7fac70194c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:54.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.696+0000 7fac67fff640 1 --2- 172.21.15.114:0/2616492707 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fac700fef50 0x7fac70194c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:54.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.696+0000 7fac67fff640 1 -- 172.21.15.114:0/2616492707 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac70100220 msgr2=0x7fac7019c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:54.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.696+0000 7fac67fff640 1 --2- 172.21.15.114:0/2616492707 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac70100220 0x7fac7019c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:54.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.696+0000 7fac67fff640 1 -- 172.21.15.114:0/2616492707 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac58009560 con 0x7fac700ff910 2024-09-13T18:21:54.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.696+0000 7fac67fff640 1 --2- 172.21.15.114:0/2616492707 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac700ff910 0x7fac70195170 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fac580057c0 tx=0x7fac58004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:54.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.696+0000 7fac6dffb640 1 -- 172.21.15.114:0/2616492707 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac58031d30 con 0x7fac700ff910 2024-09-13T18:21:54.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.696+0000 7fac6dffb640 1 -- 172.21.15.114:0/2616492707 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fac58031e90 con 0x7fac700ff910 2024-09-13T18:21:54.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.696+0000 7fac7629d640 1 -- 172.21.15.114:0/2616492707 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac7019ccc0 con 0x7fac700ff910 2024-09-13T18:21:54.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.696+0000 7fac6dffb640 1 -- 172.21.15.114:0/2616492707 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac58003b80 con 0x7fac700ff910 2024-09-13T18:21:54.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.696+0000 7fac7629d640 1 -- 172.21.15.114:0/2616492707 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac7019d170 con 0x7fac700ff910 2024-09-13T18:21:54.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.697+0000 7fac7629d640 1 -- 172.21.15.114:0/2616492707 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac70106270 con 0x7fac700ff910 2024-09-13T18:21:54.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.698+0000 7fac6dffb640 1 -- 172.21.15.114:0/2616492707 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fac58003ce0 con 0x7fac700ff910 2024-09-13T18:21:54.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.699+0000 7fac6dffb640 1 --2- 172.21.15.114:0/2616492707 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fac5c0a1120 0x7fac5c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:54.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.699+0000 7fac6ffff640 1 --2- 172.21.15.114:0/2616492707 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fac5c0a1120 0x7fac5c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:54.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.699+0000 7fac6dffb640 1 -- 172.21.15.114:0/2616492707 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fac580ee230 con 0x7fac700ff910 2024-09-13T18:21:54.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.699+0000 7fac6ffff640 1 --2- 172.21.15.114:0/2616492707 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fac5c0a1120 0x7fac5c0a35e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fac70196100 tx=0x7fac6000b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:54.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.703+0000 7fac6dffb640 1 -- 172.21.15.114:0/2616492707 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fac580b7510 con 0x7fac700ff910 2024-09-13T18:21:54.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.865+0000 7fac7629d640 1 -- 172.21.15.114:0/2616492707 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fac7002c3e0 con 0x7fac700ff910 2024-09-13T18:21:54.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.866+0000 7fac6dffb640 1 -- 172.21.15.114:0/2616492707 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7fac5804a020 con 0x7fac700ff910 2024-09-13T18:21:54.869 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 1 2024-09-13T18:21:54.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.867+0000 7fac7629d640 1 -- 172.21.15.114:0/2616492707 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fac5c0a1120 msgr2=0x7fac5c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:54.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.867+0000 7fac7629d640 1 --2- 172.21.15.114:0/2616492707 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fac5c0a1120 0x7fac5c0a35e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fac70196100 tx=0x7fac6000b390 comp rx=0 tx=0).stop 2024-09-13T18:21:54.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.867+0000 7fac7629d640 1 -- 172.21.15.114:0/2616492707 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac700ff910 msgr2=0x7fac70195170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:54.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.867+0000 7fac7629d640 1 --2- 172.21.15.114:0/2616492707 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac700ff910 0x7fac70195170 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fac580057c0 tx=0x7fac58004360 comp rx=0 tx=0).stop 2024-09-13T18:21:54.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.868+0000 7fac7629d640 1 -- 172.21.15.114:0/2616492707 shutdown_connections 2024-09-13T18:21:54.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.868+0000 7fac7629d640 1 --2- 172.21.15.114:0/2616492707 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fac5c0a1120 0x7fac5c0a35e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:54.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.868+0000 7fac7629d640 1 --2- 172.21.15.114:0/2616492707 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac70100220 0x7fac7019c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:54.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.868+0000 7fac7629d640 1 --2- 172.21.15.114:0/2616492707 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac700ff910 0x7fac70195170 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:54.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.868+0000 7fac7629d640 1 --2- 172.21.15.114:0/2616492707 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fac700fef50 0x7fac70194c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:54.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.868+0000 7fac7629d640 1 -- 172.21.15.114:0/2616492707 >> 172.21.15.114:0/2616492707 conn(0x7fac700f9b70 msgr2=0x7fac70103440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:54.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.868+0000 7fac7629d640 1 -- 172.21.15.114:0/2616492707 shutdown_connections 2024-09-13T18:21:54.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.868+0000 7fac7629d640 1 -- 172.21.15.114:0/2616492707 wait complete. 2024-09-13T18:21:54.882 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T18:21:54.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.980+0000 7fb54999d640 1 -- 172.21.15.114:0/2722307672 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb544103850 msgr2=0x7fb544103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:54.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.980+0000 7fb54999d640 1 --2- 172.21.15.114:0/2722307672 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb544103850 0x7fb544103d00 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb53c009ca0 tx=0x7fb53c02a8c0 comp rx=0 tx=0).stop 2024-09-13T18:21:54.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.980+0000 7fb54999d640 1 -- 172.21.15.114:0/2722307672 shutdown_connections 2024-09-13T18:21:54.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.980+0000 7fb54999d640 1 --2- 172.21.15.114:0/2722307672 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb544103850 0x7fb544103d00 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:54.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.980+0000 7fb54999d640 1 --2- 172.21.15.114:0/2722307672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb544100ea0 0x7fb544103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:54.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.980+0000 7fb54999d640 1 --2- 172.21.15.114:0/2722307672 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5440fe550 0x7fb544100960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:54.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.980+0000 7fb54999d640 1 -- 172.21.15.114:0/2722307672 >> 172.21.15.114:0/2722307672 conn(0x7fb5440fa360 msgr2=0x7fb5440fc780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:54.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.980+0000 7fb54999d640 1 -- 172.21.15.114:0/2722307672 shutdown_connections 2024-09-13T18:21:54.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.980+0000 7fb54999d640 1 -- 172.21.15.114:0/2722307672 wait complete. 2024-09-13T18:21:54.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.981+0000 7fb54999d640 1 Processor -- start 2024-09-13T18:21:54.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.981+0000 7fb54999d640 1 -- start start 2024-09-13T18:21:54.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.981+0000 7fb54999d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5440fe550 0x7fb54419c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:54.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.981+0000 7fb54999d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb544100ea0 0x7fb54419c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:54.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.981+0000 7fb54999d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb544103850 0x7fb5441a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:54.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.981+0000 7fb54999d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5441a3cb0 con 0x7fb544103850 2024-09-13T18:21:54.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.981+0000 7fb54999d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5441a3e20 con 0x7fb5440fe550 2024-09-13T18:21:54.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.981+0000 7fb54999d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5441a3fe0 con 0x7fb544100ea0 2024-09-13T18:21:54.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.981+0000 7fb54919c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb544103850 0x7fb5441a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:54.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.981+0000 7fb54919c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb544103850 0x7fb5441a3770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33746/0 (socket says 172.21.15.114:33746) 2024-09-13T18:21:54.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.981+0000 7fb54919c640 1 -- 172.21.15.114:0/2323446108 learned_addr learned my addr 172.21.15.114:0/2323446108 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:54.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.981+0000 7fb543fff640 1 --2- 172.21.15.114:0/2323446108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb544100ea0 0x7fb54419c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:54.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.981+0000 7fb54899b640 1 --2- 172.21.15.114:0/2323446108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5440fe550 0x7fb54419c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:54.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.981+0000 7fb54919c640 1 -- 172.21.15.114:0/2323446108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb544100ea0 msgr2=0x7fb54419c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:54.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.981+0000 7fb54919c640 1 --2- 172.21.15.114:0/2323446108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb544100ea0 0x7fb54419c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:54.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.981+0000 7fb54919c640 1 -- 172.21.15.114:0/2323446108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5440fe550 msgr2=0x7fb54419c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:54.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.981+0000 7fb54919c640 1 --2- 172.21.15.114:0/2323446108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5440fe550 0x7fb54419c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:54.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.982+0000 7fb54919c640 1 -- 172.21.15.114:0/2323446108 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb53c009900 con 0x7fb544103850 2024-09-13T18:21:54.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.982+0000 7fb543fff640 1 --2- 172.21.15.114:0/2323446108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb544100ea0 0x7fb54419c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:21:54.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.982+0000 7fb54899b640 1 --2- 172.21.15.114:0/2323446108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5440fe550 0x7fb54419c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:21:54.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.982+0000 7fb54919c640 1 --2- 172.21.15.114:0/2323446108 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb544103850 0x7fb5441a3770 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb53c02aeb0 tx=0x7fb53c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:54.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.982+0000 7fb541ffb640 1 -- 172.21.15.114:0/2323446108 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53c0307c0 con 0x7fb544103850 2024-09-13T18:21:54.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.982+0000 7fb541ffb640 1 -- 172.21.15.114:0/2323446108 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb53c040070 con 0x7fb544103850 2024-09-13T18:21:54.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.982+0000 7fb54999d640 1 -- 172.21.15.114:0/2323446108 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5441a4260 con 0x7fb544103850 2024-09-13T18:21:54.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.982+0000 7fb541ffb640 1 -- 172.21.15.114:0/2323446108 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53c0044e0 con 0x7fb544103850 2024-09-13T18:21:54.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.982+0000 7fb54999d640 1 -- 172.21.15.114:0/2323446108 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5441a47c0 con 0x7fb544103850 2024-09-13T18:21:54.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.983+0000 7fb54999d640 1 -- 172.21.15.114:0/2323446108 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5440fbe50 con 0x7fb544103850 2024-09-13T18:21:54.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.985+0000 7fb541ffb640 1 -- 172.21.15.114:0/2323446108 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fb53c042030 con 0x7fb544103850 2024-09-13T18:21:54.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.986+0000 7fb541ffb640 1 --2- 172.21.15.114:0/2323446108 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb5340a11a0 0x7fb5340a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:54.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.986+0000 7fb54899b640 1 --2- 172.21.15.114:0/2323446108 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb5340a11a0 0x7fb5340a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:54.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.986+0000 7fb541ffb640 1 -- 172.21.15.114:0/2323446108 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fb53c0ee530 con 0x7fb544103850 2024-09-13T18:21:54.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.986+0000 7fb54899b640 1 --2- 172.21.15.114:0/2323446108 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb5340a11a0 0x7fb5340a3660 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb5380058c0 tx=0x7fb538005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:54.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:54.989+0000 7fb541ffb640 1 -- 172.21.15.114:0/2323446108 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fb53c0b7900 con 0x7fb544103850 2024-09-13T18:21:55.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.153+0000 7fb54999d640 1 -- 172.21.15.114:0/2323446108 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb5440fb8c0 con 0x7fb544103850 2024-09-13T18:21:55.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.153+0000 7fb541ffb640 1 -- 172.21.15.114:0/2323446108 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb53c0baf20 con 0x7fb544103850 2024-09-13T18:21:55.157 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2024-09-13T18:21:55.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.155+0000 7fb54999d640 1 -- 172.21.15.114:0/2323446108 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb5340a11a0 msgr2=0x7fb5340a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:55.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.155+0000 7fb54999d640 1 --2- 172.21.15.114:0/2323446108 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb5340a11a0 0x7fb5340a3660 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb5380058c0 tx=0x7fb538005850 comp rx=0 tx=0).stop 2024-09-13T18:21:55.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.155+0000 7fb54999d640 1 -- 172.21.15.114:0/2323446108 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb544103850 msgr2=0x7fb5441a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:55.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.155+0000 7fb54999d640 1 --2- 172.21.15.114:0/2323446108 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb544103850 0x7fb5441a3770 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb53c02aeb0 tx=0x7fb53c003c20 comp rx=0 tx=0).stop 2024-09-13T18:21:55.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.155+0000 7fb54999d640 1 -- 172.21.15.114:0/2323446108 shutdown_connections 2024-09-13T18:21:55.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.155+0000 7fb54999d640 1 --2- 172.21.15.114:0/2323446108 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb5340a11a0 0x7fb5340a3660 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:55.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.155+0000 7fb54999d640 1 --2- 172.21.15.114:0/2323446108 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb544103850 0x7fb5441a3770 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:55.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.155+0000 7fb54999d640 1 --2- 172.21.15.114:0/2323446108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb544100ea0 0x7fb54419c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:55.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.155+0000 7fb54999d640 1 --2- 172.21.15.114:0/2323446108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5440fe550 0x7fb54419c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:55.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.155+0000 7fb54999d640 1 -- 172.21.15.114:0/2323446108 >> 172.21.15.114:0/2323446108 conn(0x7fb5440fa360 msgr2=0x7fb544101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:55.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.156+0000 7fb54999d640 1 -- 172.21.15.114:0/2323446108 shutdown_connections 2024-09-13T18:21:55.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.156+0000 7fb54999d640 1 -- 172.21.15.114:0/2323446108 wait complete. 2024-09-13T18:21:55.169 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:54 smithi114 ceph-mon[30897]: pgmap v337: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-13T18:21:55.169 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:54 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/3839115729' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T18:21:55.169 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:54 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2946645366' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:21:55.169 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:54 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/3325249249' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:21:55.169 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:54 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2616492707' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T18:21:55.170 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T18:21:55.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.270+0000 7fb7649bc640 1 -- 172.21.15.114:0/1938616899 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb760100840 msgr2=0x7fb7601070d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:55.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.270+0000 7fb7649bc640 1 --2- 172.21.15.114:0/1938616899 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb760100840 0x7fb7601070d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb758009ca0 tx=0x7fb75802a8c0 comp rx=0 tx=0).stop 2024-09-13T18:21:55.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.270+0000 7fb7649bc640 1 -- 172.21.15.114:0/1938616899 shutdown_connections 2024-09-13T18:21:55.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.270+0000 7fb7649bc640 1 --2- 172.21.15.114:0/1938616899 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb760100840 0x7fb7601070d0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:55.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.270+0000 7fb7649bc640 1 --2- 172.21.15.114:0/1938616899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7600ffe80 0x7fb760100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:55.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.270+0000 7fb7649bc640 1 --2- 172.21.15.114:0/1938616899 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7600fec90 0x7fb7600ff090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:55.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.270+0000 7fb7649bc640 1 -- 172.21.15.114:0/1938616899 >> 172.21.15.114:0/1938616899 conn(0x7fb7600fa3f0 msgr2=0x7fb7600fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:55.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.270+0000 7fb7649bc640 1 -- 172.21.15.114:0/1938616899 shutdown_connections 2024-09-13T18:21:55.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.270+0000 7fb7649bc640 1 -- 172.21.15.114:0/1938616899 wait complete. 2024-09-13T18:21:55.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.271+0000 7fb7649bc640 1 Processor -- start 2024-09-13T18:21:55.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.271+0000 7fb7649bc640 1 -- start start 2024-09-13T18:21:55.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.271+0000 7fb7649bc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7600fec90 0x7fb760195c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:55.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.271+0000 7fb7649bc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7600ffe80 0x7fb760196190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:55.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.271+0000 7fb7649bc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb760100840 0x7fb76019d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:55.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.271+0000 7fb7649bc640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb76019d730 con 0x7fb7600ffe80 2024-09-13T18:21:55.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.271+0000 7fb7649bc640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb76019d8a0 con 0x7fb760100840 2024-09-13T18:21:55.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.271+0000 7fb7649bc640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb76019da60 con 0x7fb7600fec90 2024-09-13T18:21:55.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.271+0000 7fb75dd9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7600ffe80 0x7fb760196190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:55.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.271+0000 7fb75dd9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7600ffe80 0x7fb760196190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33768/0 (socket says 172.21.15.114:33768) 2024-09-13T18:21:55.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.271+0000 7fb75dd9b640 1 -- 172.21.15.114:0/579761851 learned_addr learned my addr 172.21.15.114:0/579761851 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:55.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.271+0000 7fb75ed9d640 1 --2- 172.21.15.114:0/579761851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb760100840 0x7fb76019d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:55.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.271+0000 7fb75e59c640 1 --2- 172.21.15.114:0/579761851 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7600fec90 0x7fb760195c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:55.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.271+0000 7fb75dd9b640 1 -- 172.21.15.114:0/579761851 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7600fec90 msgr2=0x7fb760195c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:55.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.271+0000 7fb75dd9b640 1 --2- 172.21.15.114:0/579761851 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7600fec90 0x7fb760195c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:55.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.271+0000 7fb75dd9b640 1 -- 172.21.15.114:0/579761851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb760100840 msgr2=0x7fb76019d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:55.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.271+0000 7fb75dd9b640 1 --2- 172.21.15.114:0/579761851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb760100840 0x7fb76019d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:55.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.271+0000 7fb75dd9b640 1 -- 172.21.15.114:0/579761851 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb758009900 con 0x7fb7600ffe80 2024-09-13T18:21:55.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.272+0000 7fb75dd9b640 1 --2- 172.21.15.114:0/579761851 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7600ffe80 0x7fb760196190 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb75400d770 tx=0x7fb75400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:55.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.272+0000 7fb74f7fe640 1 -- 172.21.15.114:0/579761851 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb754009c30 con 0x7fb7600ffe80 2024-09-13T18:21:55.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.272+0000 7fb74f7fe640 1 -- 172.21.15.114:0/579761851 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb75400de80 con 0x7fb7600ffe80 2024-09-13T18:21:55.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.272+0000 7fb7649bc640 1 -- 172.21.15.114:0/579761851 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb76019dd40 con 0x7fb7600ffe80 2024-09-13T18:21:55.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.272+0000 7fb74f7fe640 1 -- 172.21.15.114:0/579761851 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb754004d40 con 0x7fb7600ffe80 2024-09-13T18:21:55.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.272+0000 7fb7649bc640 1 -- 172.21.15.114:0/579761851 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb76019e1a0 con 0x7fb7600ffe80 2024-09-13T18:21:55.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.273+0000 7fb7649bc640 1 -- 172.21.15.114:0/579761851 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb760106140 con 0x7fb7600ffe80 2024-09-13T18:21:55.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.274+0000 7fb74f7fe640 1 -- 172.21.15.114:0/579761851 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fb754004ea0 con 0x7fb7600ffe80 2024-09-13T18:21:55.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.275+0000 7fb74f7fe640 1 --2- 172.21.15.114:0/579761851 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb7480a1120 0x7fb7480a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:55.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.275+0000 7fb75e59c640 1 --2- 172.21.15.114:0/579761851 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb7480a1120 0x7fb7480a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:55.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.275+0000 7fb74f7fe640 1 -- 172.21.15.114:0/579761851 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fb7540cc930 con 0x7fb7600ffe80 2024-09-13T18:21:55.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.275+0000 7fb75e59c640 1 --2- 172.21.15.114:0/579761851 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb7480a1120 0x7fb7480a35e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb750005fd0 tx=0x7fb750005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:55.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.279+0000 7fb74f7fe640 1 -- 172.21.15.114:0/579761851 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fb754095b00 con 0x7fb7600ffe80 2024-09-13T18:21:55.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:54 smithi134 ceph-mon[34283]: pgmap v337: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-13T18:21:55.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:54 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/3839115729' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T18:21:55.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:54 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/2946645366' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:21:55.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:54 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/3325249249' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:21:55.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:54 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/2616492707' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T18:21:55.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:54 smithi204 ceph-mon[34777]: pgmap v337: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-13T18:21:55.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:54 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/3839115729' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T18:21:55.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:54 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/2946645366' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:21:55.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:54 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/3325249249' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:21:55.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:54 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/2616492707' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T18:21:55.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.442+0000 7fb7649bc640 1 -- 172.21.15.114:0/579761851 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb76002c480 con 0x7fb7600ffe80 2024-09-13T18:21:55.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.442+0000 7fb74f7fe640 1 -- 172.21.15.114:0/579761851 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb754017020 con 0x7fb7600ffe80 2024-09-13T18:21:55.445 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 64 2024-09-13T18:21:55.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.444+0000 7fb7649bc640 1 -- 172.21.15.114:0/579761851 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb7480a1120 msgr2=0x7fb7480a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:55.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.444+0000 7fb7649bc640 1 --2- 172.21.15.114:0/579761851 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb7480a1120 0x7fb7480a35e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb750005fd0 tx=0x7fb750005850 comp rx=0 tx=0).stop 2024-09-13T18:21:55.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.444+0000 7fb7649bc640 1 -- 172.21.15.114:0/579761851 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7600ffe80 msgr2=0x7fb760196190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:55.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.444+0000 7fb7649bc640 1 --2- 172.21.15.114:0/579761851 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7600ffe80 0x7fb760196190 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb75400d770 tx=0x7fb75400dc30 comp rx=0 tx=0).stop 2024-09-13T18:21:55.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.444+0000 7fb7649bc640 1 -- 172.21.15.114:0/579761851 shutdown_connections 2024-09-13T18:21:55.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.444+0000 7fb7649bc640 1 --2- 172.21.15.114:0/579761851 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb7480a1120 0x7fb7480a35e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:55.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.444+0000 7fb7649bc640 1 --2- 172.21.15.114:0/579761851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb760100840 0x7fb76019d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:55.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.444+0000 7fb7649bc640 1 --2- 172.21.15.114:0/579761851 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7600ffe80 0x7fb760196190 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:55.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.444+0000 7fb7649bc640 1 --2- 172.21.15.114:0/579761851 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7600fec90 0x7fb760195c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:55.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.444+0000 7fb7649bc640 1 -- 172.21.15.114:0/579761851 >> 172.21.15.114:0/579761851 conn(0x7fb7600fa3f0 msgr2=0x7fb760103310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:55.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.444+0000 7fb7649bc640 1 -- 172.21.15.114:0/579761851 shutdown_connections 2024-09-13T18:21:55.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.444+0000 7fb7649bc640 1 -- 172.21.15.114:0/579761851 wait complete. 2024-09-13T18:21:55.458 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T18:21:55.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.557+0000 7f42ac43a640 1 -- 172.21.15.114:0/770775659 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42a4103140 msgr2=0x7f42a4105530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:55.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.557+0000 7f42ac43a640 1 --2- 172.21.15.114:0/770775659 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42a4103140 0x7f42a4105530 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f42a0009900 tx=0x7f42a002a3f0 comp rx=0 tx=0).stop 2024-09-13T18:21:55.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.557+0000 7f42ac43a640 1 -- 172.21.15.114:0/770775659 shutdown_connections 2024-09-13T18:21:55.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.557+0000 7f42ac43a640 1 --2- 172.21.15.114:0/770775659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42a4105a70 0x7f42a4107ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:55.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.557+0000 7f42ac43a640 1 --2- 172.21.15.114:0/770775659 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42a4103140 0x7f42a4105530 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:55.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.557+0000 7f42ac43a640 1 --2- 172.21.15.114:0/770775659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42a4100810 0x7f42a4102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:55.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.557+0000 7f42ac43a640 1 -- 172.21.15.114:0/770775659 >> 172.21.15.114:0/770775659 conn(0x7f42a40fa3f0 msgr2=0x7f42a40fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:55.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.557+0000 7f42ac43a640 1 -- 172.21.15.114:0/770775659 shutdown_connections 2024-09-13T18:21:55.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.557+0000 7f42ac43a640 1 -- 172.21.15.114:0/770775659 wait complete. 2024-09-13T18:21:55.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.558+0000 7f42ac43a640 1 Processor -- start 2024-09-13T18:21:55.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.558+0000 7f42ac43a640 1 -- start start 2024-09-13T18:21:55.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.558+0000 7f42ac43a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42a4100810 0x7f42a4195be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:55.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.558+0000 7f42ac43a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42a4103140 0x7f42a4196120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:55.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.558+0000 7f42ac43a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42a4105a70 0x7f42a419d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:55.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.558+0000 7f42ac43a640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a419d6d0 con 0x7f42a4105a70 2024-09-13T18:21:55.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.558+0000 7f42ac43a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a419d840 con 0x7f42a4103140 2024-09-13T18:21:55.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.558+0000 7f42ac43a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a419da00 con 0x7f42a4100810 2024-09-13T18:21:55.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.558+0000 7f42aa9d7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42a4105a70 0x7f42a419d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:55.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.558+0000 7f42aa9d7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42a4105a70 0x7f42a419d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33786/0 (socket says 172.21.15.114:33786) 2024-09-13T18:21:55.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.558+0000 7f42aa9d7640 1 -- 172.21.15.114:0/3013303405 learned_addr learned my addr 172.21.15.114:0/3013303405 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:55.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.558+0000 7f42a99d5640 1 --2- 172.21.15.114:0/3013303405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42a4103140 0x7f42a4196120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:55.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.558+0000 7f42aa1d6640 1 --2- 172.21.15.114:0/3013303405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42a4100810 0x7f42a4195be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:55.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.558+0000 7f42aa9d7640 1 -- 172.21.15.114:0/3013303405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42a4100810 msgr2=0x7f42a4195be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:55.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.558+0000 7f42aa9d7640 1 --2- 172.21.15.114:0/3013303405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42a4100810 0x7f42a4195be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:55.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.558+0000 7f42aa9d7640 1 -- 172.21.15.114:0/3013303405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42a4103140 msgr2=0x7f42a4196120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:55.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.558+0000 7f42aa9d7640 1 --2- 172.21.15.114:0/3013303405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42a4103140 0x7f42a4196120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:55.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.558+0000 7f42aa9d7640 1 -- 172.21.15.114:0/3013303405 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a0009560 con 0x7f42a4105a70 2024-09-13T18:21:55.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.558+0000 7f42aa1d6640 1 --2- 172.21.15.114:0/3013303405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42a4100810 0x7f42a4195be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:21:55.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.558+0000 7f42aa9d7640 1 --2- 172.21.15.114:0/3013303405 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42a4105a70 0x7f42a419d190 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f429c00ccb0 tx=0x7f429c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:55.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.558+0000 7f42977fe640 1 -- 172.21.15.114:0/3013303405 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429c00f450 con 0x7f42a4105a70 2024-09-13T18:21:55.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.558+0000 7f42ac43a640 1 -- 172.21.15.114:0/3013303405 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42a419dce0 con 0x7f42a4105a70 2024-09-13T18:21:55.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.558+0000 7f42977fe640 1 -- 172.21.15.114:0/3013303405 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f429c00fa40 con 0x7f42a4105a70 2024-09-13T18:21:55.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.558+0000 7f42977fe640 1 -- 172.21.15.114:0/3013303405 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429c018970 con 0x7f42a4105a70 2024-09-13T18:21:55.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.559+0000 7f42ac43a640 1 -- 172.21.15.114:0/3013303405 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42a419e1c0 con 0x7f42a4105a70 2024-09-13T18:21:55.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.559+0000 7f42ac43a640 1 -- 172.21.15.114:0/3013303405 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42a40fbee0 con 0x7f42a4105a70 2024-09-13T18:21:55.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.561+0000 7f42977fe640 1 -- 172.21.15.114:0/3013303405 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f429c00fbb0 con 0x7f42a4105a70 2024-09-13T18:21:55.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.561+0000 7f42977fe640 1 --2- 172.21.15.114:0/3013303405 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f42900a1120 0x7f42900a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:55.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.562+0000 7f42aa1d6640 1 --2- 172.21.15.114:0/3013303405 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f42900a1120 0x7f42900a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:55.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.562+0000 7f42977fe640 1 -- 172.21.15.114:0/3013303405 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f429c0cd2c0 con 0x7f42a4105a70 2024-09-13T18:21:55.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.562+0000 7f42aa1d6640 1 --2- 172.21.15.114:0/3013303405 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f42900a1120 0x7f42900a35e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f429800baa0 tx=0x7f4298005d90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:55.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.566+0000 7f42977fe640 1 -- 172.21.15.114:0/3013303405 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f429c096580 con 0x7f42a4105a70 2024-09-13T18:21:55.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.728+0000 7f42ac43a640 1 -- 172.21.15.114:0/3013303405 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f42a402c3e0 con 0x7f42a4105a70 2024-09-13T18:21:55.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.729+0000 7f42977fe640 1 -- 172.21.15.114:0/3013303405 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7f429c099ba0 con 0x7f42a4105a70 2024-09-13T18:21:55.732 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2024-09-13T18:21:55.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.731+0000 7f42ac43a640 1 -- 172.21.15.114:0/3013303405 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f42900a1120 msgr2=0x7f42900a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:55.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.731+0000 7f42ac43a640 1 --2- 172.21.15.114:0/3013303405 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f42900a1120 0x7f42900a35e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f429800baa0 tx=0x7f4298005d90 comp rx=0 tx=0).stop 2024-09-13T18:21:55.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.731+0000 7f42ac43a640 1 -- 172.21.15.114:0/3013303405 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42a4105a70 msgr2=0x7f42a419d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:55.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.731+0000 7f42ac43a640 1 --2- 172.21.15.114:0/3013303405 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42a4105a70 0x7f42a419d190 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f429c00ccb0 tx=0x7f429c00a620 comp rx=0 tx=0).stop 2024-09-13T18:21:55.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.731+0000 7f42ac43a640 1 -- 172.21.15.114:0/3013303405 shutdown_connections 2024-09-13T18:21:55.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.731+0000 7f42ac43a640 1 --2- 172.21.15.114:0/3013303405 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f42900a1120 0x7f42900a35e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:55.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.731+0000 7f42ac43a640 1 --2- 172.21.15.114:0/3013303405 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42a4105a70 0x7f42a419d190 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:55.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.731+0000 7f42ac43a640 1 --2- 172.21.15.114:0/3013303405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42a4103140 0x7f42a4196120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:55.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.731+0000 7f42ac43a640 1 --2- 172.21.15.114:0/3013303405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42a4100810 0x7f42a4195be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:55.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.731+0000 7f42ac43a640 1 -- 172.21.15.114:0/3013303405 >> 172.21.15.114:0/3013303405 conn(0x7f42a40fa3f0 msgr2=0x7f42a41011f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:55.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.731+0000 7f42ac43a640 1 -- 172.21.15.114:0/3013303405 shutdown_connections 2024-09-13T18:21:55.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.731+0000 7f42ac43a640 1 -- 172.21.15.114:0/3013303405 wait complete. 2024-09-13T18:21:55.745 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:21:55.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.840+0000 7fd8c8165640 1 -- 172.21.15.114:0/2091432327 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8c0100820 msgr2=0x7fd8c0102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:55.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.840+0000 7fd8c8165640 1 --2- 172.21.15.114:0/2091432327 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8c0100820 0x7fd8c0102c10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc009900 tx=0x7fd8bc02a410 comp rx=0 tx=0).stop 2024-09-13T18:21:55.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.840+0000 7fd8c8165640 1 -- 172.21.15.114:0/2091432327 shutdown_connections 2024-09-13T18:21:55.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.840+0000 7fd8c8165640 1 --2- 172.21.15.114:0/2091432327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8c0105a80 0x7fd8c0107ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:55.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.840+0000 7fd8c8165640 1 --2- 172.21.15.114:0/2091432327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8c0103150 0x7fd8c0105540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:55.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.840+0000 7fd8c8165640 1 --2- 172.21.15.114:0/2091432327 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8c0100820 0x7fd8c0102c10 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:55.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.840+0000 7fd8c8165640 1 -- 172.21.15.114:0/2091432327 >> 172.21.15.114:0/2091432327 conn(0x7fd8c00fa420 msgr2=0x7fd8c00fc840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:55.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.840+0000 7fd8c8165640 1 -- 172.21.15.114:0/2091432327 shutdown_connections 2024-09-13T18:21:55.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.840+0000 7fd8c8165640 1 -- 172.21.15.114:0/2091432327 wait complete. 2024-09-13T18:21:55.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.841+0000 7fd8c8165640 1 Processor -- start 2024-09-13T18:21:55.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.841+0000 7fd8c8165640 1 -- start start 2024-09-13T18:21:55.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.841+0000 7fd8c8165640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8c0100820 0x7fd8c0195ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:55.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.841+0000 7fd8c8165640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8c0103150 0x7fd8c01963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:55.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.841+0000 7fd8c8165640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8c0105a80 0x7fd8c019d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:55.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.841+0000 7fd8c8165640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c019d990 con 0x7fd8c0100820 2024-09-13T18:21:55.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.841+0000 7fd8c8165640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c019db00 con 0x7fd8c0103150 2024-09-13T18:21:55.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.841+0000 7fd8c8165640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c019dcc0 con 0x7fd8c0105a80 2024-09-13T18:21:55.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.841+0000 7fd8c5f01640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8c0100820 0x7fd8c0195ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:55.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.841+0000 7fd8c5f01640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8c0100820 0x7fd8c0195ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33796/0 (socket says 172.21.15.114:33796) 2024-09-13T18:21:55.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.841+0000 7fd8c5f01640 1 -- 172.21.15.114:0/1676610167 learned_addr learned my addr 172.21.15.114:0/1676610167 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:55.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.841+0000 7fd8c5700640 1 --2- 172.21.15.114:0/1676610167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8c0103150 0x7fd8c01963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:55.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.841+0000 7fd8c6702640 1 --2- 172.21.15.114:0/1676610167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8c0105a80 0x7fd8c019d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:55.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.841+0000 7fd8c5f01640 1 -- 172.21.15.114:0/1676610167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8c0105a80 msgr2=0x7fd8c019d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:55.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.841+0000 7fd8c5f01640 1 --2- 172.21.15.114:0/1676610167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8c0105a80 0x7fd8c019d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:55.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.841+0000 7fd8c5f01640 1 -- 172.21.15.114:0/1676610167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8c0103150 msgr2=0x7fd8c01963e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:55.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.841+0000 7fd8c5f01640 1 --2- 172.21.15.114:0/1676610167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8c0103150 0x7fd8c01963e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:55.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.841+0000 7fd8c5f01640 1 -- 172.21.15.114:0/1676610167 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8bc009560 con 0x7fd8c0100820 2024-09-13T18:21:55.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.841+0000 7fd8c5700640 1 --2- 172.21.15.114:0/1676610167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8c0103150 0x7fd8c01963e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:21:55.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.841+0000 7fd8c6702640 1 --2- 172.21.15.114:0/1676610167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8c0105a80 0x7fd8c019d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:21:55.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.842+0000 7fd8c5f01640 1 --2- 172.21.15.114:0/1676610167 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8c0100820 0x7fd8c0195ea0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc005ab0 tx=0x7fd8bc005320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:55.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.842+0000 7fd8b6ffd640 1 -- 172.21.15.114:0/1676610167 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc031d70 con 0x7fd8c0100820 2024-09-13T18:21:55.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.842+0000 7fd8b6ffd640 1 -- 172.21.15.114:0/1676610167 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd8bc005510 con 0x7fd8c0100820 2024-09-13T18:21:55.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.842+0000 7fd8c8165640 1 -- 172.21.15.114:0/1676610167 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8c019df40 con 0x7fd8c0100820 2024-09-13T18:21:55.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.842+0000 7fd8b6ffd640 1 -- 172.21.15.114:0/1676610167 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc003b30 con 0x7fd8c0100820 2024-09-13T18:21:55.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.842+0000 7fd8c8165640 1 -- 172.21.15.114:0/1676610167 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8c019e310 con 0x7fd8c0100820 2024-09-13T18:21:55.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.843+0000 7fd8c8165640 1 -- 172.21.15.114:0/1676610167 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8c00fbf10 con 0x7fd8c0100820 2024-09-13T18:21:55.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.844+0000 7fd8b6ffd640 1 -- 172.21.15.114:0/1676610167 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fd8bc003cf0 con 0x7fd8c0100820 2024-09-13T18:21:55.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.844+0000 7fd8b6ffd640 1 --2- 172.21.15.114:0/1676610167 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fd8ac0a1120 0x7fd8ac0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:55.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.845+0000 7fd8c5700640 1 --2- 172.21.15.114:0/1676610167 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fd8ac0a1120 0x7fd8ac0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:55.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.845+0000 7fd8b6ffd640 1 -- 172.21.15.114:0/1676610167 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fd8bc0edfe0 con 0x7fd8c0100820 2024-09-13T18:21:55.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.845+0000 7fd8c5700640 1 --2- 172.21.15.114:0/1676610167 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fd8ac0a1120 0x7fd8ac0a35e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd8c01973b0 tx=0x7fd8b0008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:55.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:55.848+0000 7fd8b6ffd640 1 -- 172.21.15.114:0/1676610167 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fd8bc0b7280 con 0x7fd8c0100820 2024-09-13T18:21:56.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.073+0000 7fd8c8165640 1 -- 172.21.15.114:0/1676610167 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd8c010b5c0 con 0x7fd8c0100820 2024-09-13T18:21:56.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.074+0000 7fd8b6ffd640 1 -- 172.21.15.114:0/1676610167 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10930 (secure 0 0 0) 0x7fd8c010b5c0 con 0x7fd8c0100820 2024-09-13T18:21:56.077 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:21:56.077 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4162817497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4162817497},{"type":"v1","addr":"172.21.15.114:6835","nonce":4162817497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/2435428478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":2435428478},{"type":"v1","addr":"172.21.15.114:6837","nonce":2435428478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T18:20:01.895117+0000","modified":"2024-09-13T18:20:29.250121+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24482,"mds_2":24475,"mds_3":24458,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/2046516451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":2046516451},{"type":"v1","addr":"172.21.15.114:6839","nonce":2046516451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":10,"addr":"172.21.15.114:6841/1956923446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":1956923446},{"type":"v1","addr":"172.21.15.114:6841","nonce":1956923446}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24391":{"gid":24391,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/507301496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":507301496},{"type":"v1","addr":"172.21.15.204:6833","nonce":507301496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4028657219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4028657219},{"type":"v1","addr":"172.21.15.134:6833","nonce":4028657219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/57029119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":57029119},{"type":"v1","addr":"172.21.15.204:6835","nonce":57029119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3068880939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3068880939},{"type":"v1","addr":"172.21.15.204:6837","nonce":3068880939}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"e","rank":3,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.134:6835/1150003275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1150003275},{"type":"v1","addr":"172.21.15.134:6835","nonce":1150003275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/344883912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":344883912},{"type":"v1","addr":"172.21.15.204:6839","nonce":344883912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6837/891692210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":891692210},{"type":"v1","addr":"172.21.15.134:6837","nonce":891692210}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6839/3466309724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3466309724},{"type":"v1","addr":"172.21.15.134:6839","nonce":3466309724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:21:56.078 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 29 2024-09-13T18:21:56.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.076+0000 7fd8c8165640 1 -- 172.21.15.114:0/1676610167 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fd8ac0a1120 msgr2=0x7fd8ac0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:56.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.076+0000 7fd8c8165640 1 --2- 172.21.15.114:0/1676610167 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fd8ac0a1120 0x7fd8ac0a35e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd8c01973b0 tx=0x7fd8b0008040 comp rx=0 tx=0).stop 2024-09-13T18:21:56.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.076+0000 7fd8c8165640 1 -- 172.21.15.114:0/1676610167 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8c0100820 msgr2=0x7fd8c0195ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:56.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.076+0000 7fd8c8165640 1 --2- 172.21.15.114:0/1676610167 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8c0100820 0x7fd8c0195ea0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc005ab0 tx=0x7fd8bc005320 comp rx=0 tx=0).stop 2024-09-13T18:21:56.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.076+0000 7fd8c8165640 1 -- 172.21.15.114:0/1676610167 shutdown_connections 2024-09-13T18:21:56.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.076+0000 7fd8c8165640 1 --2- 172.21.15.114:0/1676610167 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fd8ac0a1120 0x7fd8ac0a35e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.076+0000 7fd8c8165640 1 --2- 172.21.15.114:0/1676610167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8c0105a80 0x7fd8c019d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.076+0000 7fd8c8165640 1 --2- 172.21.15.114:0/1676610167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8c0103150 0x7fd8c01963e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.076+0000 7fd8c8165640 1 --2- 172.21.15.114:0/1676610167 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8c0100820 0x7fd8c0195ea0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.076+0000 7fd8c8165640 1 -- 172.21.15.114:0/1676610167 >> 172.21.15.114:0/1676610167 conn(0x7fd8c00fa420 msgr2=0x7fd8c01026a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:56.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.076+0000 7fd8c8165640 1 -- 172.21.15.114:0/1676610167 shutdown_connections 2024-09-13T18:21:56.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.076+0000 7fd8c8165640 1 -- 172.21.15.114:0/1676610167 wait complete. 2024-09-13T18:21:56.089 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:55 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2323446108' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T18:21:56.089 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:55 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/579761851' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T18:21:56.089 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:55 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/3013303405' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T18:21:56.090 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:21:56.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.185+0000 7f4e831c7640 1 -- 172.21.15.114:0/3879097775 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e7c105ae0 msgr2=0x7f4e7c107ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:56.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.185+0000 7f4e831c7640 1 --2- 172.21.15.114:0/3879097775 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e7c105ae0 0x7f4e7c107ef0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c009900 tx=0x7f4e6c02a410 comp rx=0 tx=0).stop 2024-09-13T18:21:56.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.185+0000 7f4e831c7640 1 -- 172.21.15.114:0/3879097775 shutdown_connections 2024-09-13T18:21:56.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.185+0000 7f4e831c7640 1 --2- 172.21.15.114:0/3879097775 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e7c105ae0 0x7f4e7c107ef0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.185+0000 7f4e831c7640 1 --2- 172.21.15.114:0/3879097775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e7c1031b0 0x7f4e7c1055a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.185+0000 7f4e831c7640 1 --2- 172.21.15.114:0/3879097775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e7c100880 0x7f4e7c102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.185+0000 7f4e831c7640 1 -- 172.21.15.114:0/3879097775 >> 172.21.15.114:0/3879097775 conn(0x7f4e7c0fa440 msgr2=0x7f4e7c0fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:56.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.185+0000 7f4e831c7640 1 -- 172.21.15.114:0/3879097775 shutdown_connections 2024-09-13T18:21:56.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.185+0000 7f4e831c7640 1 -- 172.21.15.114:0/3879097775 wait complete. 2024-09-13T18:21:56.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.186+0000 7f4e831c7640 1 Processor -- start 2024-09-13T18:21:56.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.186+0000 7f4e831c7640 1 -- start start 2024-09-13T18:21:56.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.186+0000 7f4e831c7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e7c100880 0x7f4e7c195e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:56.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.186+0000 7f4e831c7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e7c1031b0 0x7f4e7c196350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:56.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.186+0000 7f4e831c7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e7c105ae0 0x7f4e7c19d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:56.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.186+0000 7f4e831c7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7c19d8b0 con 0x7f4e7c1031b0 2024-09-13T18:21:56.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.186+0000 7f4e831c7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7c19da20 con 0x7f4e7c100880 2024-09-13T18:21:56.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.186+0000 7f4e831c7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7c19dbe0 con 0x7f4e7c105ae0 2024-09-13T18:21:56.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.186+0000 7f4e7bfff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e7c1031b0 0x7f4e7c196350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:56.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.186+0000 7f4e7bfff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e7c1031b0 0x7f4e7c196350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33810/0 (socket says 172.21.15.114:33810) 2024-09-13T18:21:56.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.186+0000 7f4e7bfff640 1 -- 172.21.15.114:0/32698352 learned_addr learned my addr 172.21.15.114:0/32698352 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:56.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.186+0000 7f4e80f63640 1 --2- 172.21.15.114:0/32698352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e7c100880 0x7f4e7c195e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:56.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.187+0000 7f4e7bfff640 1 -- 172.21.15.114:0/32698352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e7c105ae0 msgr2=0x7f4e7c19d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:56.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.187+0000 7f4e7bfff640 1 --2- 172.21.15.114:0/32698352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e7c105ae0 0x7f4e7c19d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.187+0000 7f4e7bfff640 1 -- 172.21.15.114:0/32698352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e7c100880 msgr2=0x7f4e7c195e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:56.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.187+0000 7f4e7bfff640 1 --2- 172.21.15.114:0/32698352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e7c100880 0x7f4e7c195e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.187+0000 7f4e7bfff640 1 -- 172.21.15.114:0/32698352 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e6c009560 con 0x7f4e7c1031b0 2024-09-13T18:21:56.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.187+0000 7f4e80f63640 1 --2- 172.21.15.114:0/32698352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e7c100880 0x7f4e7c195e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:21:56.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.187+0000 7f4e7bfff640 1 --2- 172.21.15.114:0/32698352 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e7c1031b0 0x7f4e7c196350 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4e6400b450 tx=0x7f4e6400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:56.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.187+0000 7f4e79ffb640 1 -- 172.21.15.114:0/32698352 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e64011890 con 0x7f4e7c1031b0 2024-09-13T18:21:56.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.187+0000 7f4e79ffb640 1 -- 172.21.15.114:0/32698352 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4e64011e80 con 0x7f4e7c1031b0 2024-09-13T18:21:56.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.187+0000 7f4e831c7640 1 -- 172.21.15.114:0/32698352 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e7c19dec0 con 0x7f4e7c1031b0 2024-09-13T18:21:56.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.187+0000 7f4e79ffb640 1 -- 172.21.15.114:0/32698352 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6400f340 con 0x7f4e7c1031b0 2024-09-13T18:21:56.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.187+0000 7f4e831c7640 1 -- 172.21.15.114:0/32698352 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e7c19e340 con 0x7f4e7c1031b0 2024-09-13T18:21:56.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.188+0000 7f4e831c7640 1 -- 172.21.15.114:0/32698352 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e7c0fbf30 con 0x7f4e7c1031b0 2024-09-13T18:21:56.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.189+0000 7f4e79ffb640 1 -- 172.21.15.114:0/32698352 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f4e640119f0 con 0x7f4e7c1031b0 2024-09-13T18:21:56.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.190+0000 7f4e79ffb640 1 --2- 172.21.15.114:0/32698352 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f4e680a1120 0x7f4e680a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:56.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.190+0000 7f4e80f63640 1 --2- 172.21.15.114:0/32698352 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f4e680a1120 0x7f4e680a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:56.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.190+0000 7f4e79ffb640 1 -- 172.21.15.114:0/32698352 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f4e640cd060 con 0x7f4e7c1031b0 2024-09-13T18:21:56.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.190+0000 7f4e80f63640 1 --2- 172.21.15.114:0/32698352 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f4e680a1120 0x7f4e680a35e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4e74009fa0 tx=0x7f4e74009570 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:56.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.193+0000 7f4e79ffb640 1 -- 172.21.15.114:0/32698352 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f4e64096430 con 0x7f4e7c1031b0 2024-09-13T18:21:56.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.359+0000 7f4e831c7640 1 -- 172.21.15.114:0/32698352 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4e7c10b820 con 0x7f4e7c1031b0 2024-09-13T18:21:56.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.360+0000 7f4e79ffb640 1 -- 172.21.15.114:0/32698352 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24696 (secure 0 0 0) 0x7f4e64099a50 con 0x7f4e7c1031b0 2024-09-13T18:21:56.364 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:21:56.364 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":93,"fsid":"97e7edf2-71fb-11ef-bceb-c7b262605968","created":"2024-09-13T18:12:36.360625+0000","modified":"2024-09-13T18:21:28.143797+0000","last_up_change":"2024-09-13T18:19:12.350370+0000","last_in_change":"2024-09-13T18:18:52.321596+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:15:40.129659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:20:00.024407+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:20:01.085549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T18:20:04.334216+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9914f01a-9f6f-4878-b955-83570199c354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6803","nonce":3499525826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6805","nonce":3499525826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6809","nonce":3499525826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6807","nonce":3499525826}]},"public_addr":"172.21.15.114:6803/3499525826","cluster_addr":"172.21.15.114:6805/3499525826","heartbeat_back_addr":"172.21.15.114:6809/3499525826","heartbeat_front_addr":"172.21.15.114:6807/3499525826","state":["exists","up"]},{"osd":1,"uuid":"ef65e012-7469-4c20-b354-17022bca63c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6801","nonce":4264264866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6803","nonce":4264264866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6807","nonce":4264264866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6805","nonce":4264264866}]},"public_addr":"172.21.15.134:6801/4264264866","cluster_addr":"172.21.15.134:6803/4264264866","heartbeat_back_addr":"172.21.15.134:6807/4264264866","heartbeat_front_addr":"172.21.15.134:6805/4264264866","state":["exists","up"]},{"osd":2,"uuid":"578213fd-715f-4be7-91fa-254f3d0e3022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6801","nonce":2288807707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6803","nonce":2288807707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6807","nonce":2288807707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6805","nonce":2288807707}]},"public_addr":"172.21.15.204:6801/2288807707","cluster_addr":"172.21.15.204:6803/2288807707","heartbeat_back_addr":"172.21.15.204:6807/2288807707","heartbeat_front_addr":"172.21.15.204:6805/2288807707","state":["exists","up"]},{"osd":3,"uuid":"b64aab3f-1e52-4828-8397-74af5e05f2cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6811","nonce":2904204254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6813","nonce":2904204254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6817","nonce":2904204254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6815","nonce":2904204254}]},"public_addr":"172.21.15.114:6811/2904204254","cluster_addr":"172.21.15.114:6813/2904204254","heartbeat_back_addr":"172.21.15.114:6817/2904204254","heartbeat_front_addr":"172.21.15.114:6815/2904204254","state":["exists","up"]},{"osd":4,"uuid":"f38943d6-39c0-42e1-b427-cb6bb90ac38a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6809","nonce":1773945333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6811","nonce":1773945333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6815","nonce":1773945333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6813","nonce":1773945333}]},"public_addr":"172.21.15.134:6809/1773945333","cluster_addr":"172.21.15.134:6811/1773945333","heartbeat_back_addr":"172.21.15.134:6815/1773945333","heartbeat_front_addr":"172.21.15.134:6813/1773945333","state":["exists","up"]},{"osd":5,"uuid":"bb38f361-8e00-4178-9752-5ddbf3af20e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6809","nonce":2371778055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6811","nonce":2371778055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6815","nonce":2371778055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6813","nonce":2371778055}]},"public_addr":"172.21.15.204:6809/2371778055","cluster_addr":"172.21.15.204:6811/2371778055","heartbeat_back_addr":"172.21.15.204:6815/2371778055","heartbeat_front_addr":"172.21.15.204:6813/2371778055","state":["exists","up"]},{"osd":6,"uuid":"93d40a03-e7ff-4a3f-9efa-5c5403c9983d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6819","nonce":1484325044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6821","nonce":1484325044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6825","nonce":1484325044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6823","nonce":1484325044}]},"public_addr":"172.21.15.114:6819/1484325044","cluster_addr":"172.21.15.114:6821/1484325044","heartbeat_back_addr":"172.21.15.114:6825/1484325044","heartbeat_front_addr":"172.21.15.114:6823/1484325044","state":["exists","up"]},{"osd":7,"uuid":"982f629a-1b95-4be1-8e75-5ddce1ad77b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6817","nonce":799246614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6819","nonce":799246614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6823","nonce":799246614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6821","nonce":799246614}]},"public_addr":"172.21.15.134:6817/799246614","cluster_addr":"172.21.15.134:6819/799246614","heartbeat_back_addr":"172.21.15.134:6823/799246614","heartbeat_front_addr":"172.21.15.134:6821/799246614","state":["exists","up"]},{"osd":8,"uuid":"409beb9b-8227-4dad-a4a0-f0e5ce70dc5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6817","nonce":858331805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6819","nonce":858331805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6823","nonce":858331805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6821","nonce":858331805}]},"public_addr":"172.21.15.204:6817/858331805","cluster_addr":"172.21.15.204:6819/858331805","heartbeat_back_addr":"172.21.15.204:6823/858331805","heartbeat_front_addr":"172.21.15.204:6821/858331805","state":["exists","up"]},{"osd":9,"uuid":"a345a15c-b1fc-4877-b445-418eebf0d0ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6827","nonce":2172392854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6829","nonce":2172392854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6833","nonce":2172392854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6831","nonce":2172392854}]},"public_addr":"172.21.15.114:6827/2172392854","cluster_addr":"172.21.15.114:6829/2172392854","heartbeat_back_addr":"172.21.15.114:6833/2172392854","heartbeat_front_addr":"172.21.15.114:6831/2172392854","state":["exists","up"]},{"osd":10,"uuid":"464d995f-d90b-4caa-addb-1f7562bd74b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6825","nonce":3073513468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6827","nonce":3073513468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6831","nonce":3073513468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6829","nonce":3073513468}]},"public_addr":"172.21.15.134:6825/3073513468","cluster_addr":"172.21.15.134:6827/3073513468","heartbeat_back_addr":"172.21.15.134:6831/3073513468","heartbeat_front_addr":"172.21.15.134:6829/3073513468","state":["exists","up"]},{"osd":11,"uuid":"71dd794d-326c-477f-8d02-ee32bff1350d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6825","nonce":3239521765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6827","nonce":3239521765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6831","nonce":3239521765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6829","nonce":3239521765}]},"public_addr":"172.21.15.204:6825/3239521765","cluster_addr":"172.21.15.204:6827/3239521765","heartbeat_back_addr":"172.21.15.204:6831/3239521765","heartbeat_front_addr":"172.21.15.204:6829/3239521765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:14:54.542687+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:15:15.746055+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:15:38.371131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:01.383932+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:24.796237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:48.422214+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:11.284323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:34.570365+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:58.232855+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:18:21.609596+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:18:45.565859+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:19:10.017969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:6801/4281866858":"2024-09-14T18:13:25.823027+0000","172.21.15.114:6800/4281866858":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2949422849":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2034742251":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/722571999":"2024-09-14T18:13:03.521116+0000","172.21.15.114:6801/830738794":"2024-09-14T18:13:03.521116+0000","172.21.15.114:6800/830738794":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/2828364892":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/4257828906":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/3544951322":"2024-09-14T18:13:03.521116+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:21:56.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.362+0000 7f4e831c7640 1 -- 172.21.15.114:0/32698352 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f4e680a1120 msgr2=0x7f4e680a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:56.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.362+0000 7f4e831c7640 1 --2- 172.21.15.114:0/32698352 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f4e680a1120 0x7f4e680a35e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4e74009fa0 tx=0x7f4e74009570 comp rx=0 tx=0).stop 2024-09-13T18:21:56.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.362+0000 7f4e831c7640 1 -- 172.21.15.114:0/32698352 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e7c1031b0 msgr2=0x7f4e7c196350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:56.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.362+0000 7f4e831c7640 1 --2- 172.21.15.114:0/32698352 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e7c1031b0 0x7f4e7c196350 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4e6400b450 tx=0x7f4e6400b910 comp rx=0 tx=0).stop 2024-09-13T18:21:56.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.362+0000 7f4e831c7640 1 -- 172.21.15.114:0/32698352 shutdown_connections 2024-09-13T18:21:56.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.362+0000 7f4e831c7640 1 --2- 172.21.15.114:0/32698352 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f4e680a1120 0x7f4e680a35e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.362+0000 7f4e831c7640 1 --2- 172.21.15.114:0/32698352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e7c105ae0 0x7f4e7c19d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.362+0000 7f4e831c7640 1 --2- 172.21.15.114:0/32698352 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e7c1031b0 0x7f4e7c196350 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.362+0000 7f4e831c7640 1 --2- 172.21.15.114:0/32698352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e7c100880 0x7f4e7c195e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.362+0000 7f4e831c7640 1 -- 172.21.15.114:0/32698352 >> 172.21.15.114:0/32698352 conn(0x7f4e7c0fa440 msgr2=0x7f4e7c101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:56.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.362+0000 7f4e831c7640 1 -- 172.21.15.114:0/32698352 shutdown_connections 2024-09-13T18:21:56.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.362+0000 7f4e831c7640 1 -- 172.21.15.114:0/32698352 wait complete. 2024-09-13T18:21:56.368 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:55 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/2323446108' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T18:21:56.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:55 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/579761851' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T18:21:56.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:55 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/3013303405' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T18:21:56.377 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:21:56.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:55 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/2323446108' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T18:21:56.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:55 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/579761851' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T18:21:56.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:55 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/3013303405' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T18:21:56.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.471+0000 7fd484ff4640 1 -- 172.21.15.114:0/1636993336 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4801008b0 msgr2=0x7fd480102ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:56.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.471+0000 7fd484ff4640 1 --2- 172.21.15.114:0/1636993336 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4801008b0 0x7fd480102ca0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd470009900 tx=0x7fd47002a410 comp rx=0 tx=0).stop 2024-09-13T18:21:56.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.471+0000 7fd484ff4640 1 -- 172.21.15.114:0/1636993336 shutdown_connections 2024-09-13T18:21:56.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.471+0000 7fd484ff4640 1 --2- 172.21.15.114:0/1636993336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd480105b10 0x7fd480107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.471+0000 7fd484ff4640 1 --2- 172.21.15.114:0/1636993336 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4801031e0 0x7fd4801055d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.471+0000 7fd484ff4640 1 --2- 172.21.15.114:0/1636993336 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4801008b0 0x7fd480102ca0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.471+0000 7fd484ff4640 1 -- 172.21.15.114:0/1636993336 >> 172.21.15.114:0/1636993336 conn(0x7fd4800fa430 msgr2=0x7fd4800fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:56.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.471+0000 7fd484ff4640 1 -- 172.21.15.114:0/1636993336 shutdown_connections 2024-09-13T18:21:56.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.471+0000 7fd484ff4640 1 -- 172.21.15.114:0/1636993336 wait complete. 2024-09-13T18:21:56.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.472+0000 7fd484ff4640 1 Processor -- start 2024-09-13T18:21:56.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.472+0000 7fd484ff4640 1 -- start start 2024-09-13T18:21:56.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.472+0000 7fd484ff4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4801008b0 0x7fd480195dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:56.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.472+0000 7fd484ff4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4801031e0 0x7fd480196310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:56.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.472+0000 7fd484ff4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd480105b10 0x7fd48019d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:56.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.472+0000 7fd484ff4640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48019d8c0 con 0x7fd480105b10 2024-09-13T18:21:56.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.472+0000 7fd484ff4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48019da30 con 0x7fd4801008b0 2024-09-13T18:21:56.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.472+0000 7fd484ff4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48019dbf0 con 0x7fd4801031e0 2024-09-13T18:21:56.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.472+0000 7fd47ed9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd480105b10 0x7fd48019d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:56.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.472+0000 7fd47ed9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd480105b10 0x7fd48019d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33830/0 (socket says 172.21.15.114:33830) 2024-09-13T18:21:56.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.472+0000 7fd47ed9d640 1 -- 172.21.15.114:0/2051971392 learned_addr learned my addr 172.21.15.114:0/2051971392 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:56.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.472+0000 7fd47dd9b640 1 --2- 172.21.15.114:0/2051971392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4801031e0 0x7fd480196310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:56.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.472+0000 7fd47ed9d640 1 -- 172.21.15.114:0/2051971392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4801031e0 msgr2=0x7fd480196310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:56.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.472+0000 7fd47ed9d640 1 --2- 172.21.15.114:0/2051971392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4801031e0 0x7fd480196310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.472+0000 7fd47e59c640 1 --2- 172.21.15.114:0/2051971392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4801008b0 0x7fd480195dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:56.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.472+0000 7fd47ed9d640 1 -- 172.21.15.114:0/2051971392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4801008b0 msgr2=0x7fd480195dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:56.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.472+0000 7fd47ed9d640 1 --2- 172.21.15.114:0/2051971392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4801008b0 0x7fd480195dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.473+0000 7fd47ed9d640 1 -- 172.21.15.114:0/2051971392 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd470009560 con 0x7fd480105b10 2024-09-13T18:21:56.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.473+0000 7fd47e59c640 1 --2- 172.21.15.114:0/2051971392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4801008b0 0x7fd480195dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:21:56.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.473+0000 7fd47ed9d640 1 --2- 172.21.15.114:0/2051971392 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd480105b10 0x7fd48019d380 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd47800cab0 tx=0x7fd47800cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:56.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.473+0000 7fd46f7fe640 1 -- 172.21.15.114:0/2051971392 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd47800ae20 con 0x7fd480105b10 2024-09-13T18:21:56.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.473+0000 7fd46f7fe640 1 -- 172.21.15.114:0/2051971392 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd47800f8e0 con 0x7fd480105b10 2024-09-13T18:21:56.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.473+0000 7fd484ff4640 1 -- 172.21.15.114:0/2051971392 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd48019ded0 con 0x7fd480105b10 2024-09-13T18:21:56.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.473+0000 7fd46f7fe640 1 -- 172.21.15.114:0/2051971392 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd478018920 con 0x7fd480105b10 2024-09-13T18:21:56.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.473+0000 7fd484ff4640 1 -- 172.21.15.114:0/2051971392 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd48019e300 con 0x7fd480105b10 2024-09-13T18:21:56.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.474+0000 7fd484ff4640 1 -- 172.21.15.114:0/2051971392 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4800fbf20 con 0x7fd480105b10 2024-09-13T18:21:56.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.475+0000 7fd46f7fe640 1 -- 172.21.15.114:0/2051971392 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fd47800fa50 con 0x7fd480105b10 2024-09-13T18:21:56.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.475+0000 7fd46f7fe640 1 --2- 172.21.15.114:0/2051971392 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fd4680a1120 0x7fd4680a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:56.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.476+0000 7fd47e59c640 1 --2- 172.21.15.114:0/2051971392 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fd4680a1120 0x7fd4680a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:56.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.476+0000 7fd46f7fe640 1 -- 172.21.15.114:0/2051971392 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fd4780cd2f0 con 0x7fd480105b10 2024-09-13T18:21:56.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.476+0000 7fd47e59c640 1 --2- 172.21.15.114:0/2051971392 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fd4680a1120 0x7fd4680a35e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd4700057c0 tx=0x7fd470005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:56.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.479+0000 7fd46f7fe640 1 -- 172.21.15.114:0/2051971392 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fd478096640 con 0x7fd480105b10 2024-09-13T18:21:56.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.674+0000 7fd484ff4640 1 -- 172.21.15.114:0/2051971392 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd48010b5b0 con 0x7fd480105b10 2024-09-13T18:21:56.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.675+0000 7fd46f7fe640 1 -- 172.21.15.114:0/2051971392 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10930 (secure 0 0 0) 0x7fd478099c60 con 0x7fd480105b10 2024-09-13T18:21:56.678 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:21:56.678 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4162817497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4162817497},{"type":"v1","addr":"172.21.15.114:6835","nonce":4162817497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/2435428478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":2435428478},{"type":"v1","addr":"172.21.15.114:6837","nonce":2435428478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T18:20:01.895117+0000","modified":"2024-09-13T18:20:29.250121+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24482,"mds_2":24475,"mds_3":24458,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/2046516451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":2046516451},{"type":"v1","addr":"172.21.15.114:6839","nonce":2046516451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":10,"addr":"172.21.15.114:6841/1956923446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":1956923446},{"type":"v1","addr":"172.21.15.114:6841","nonce":1956923446}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24391":{"gid":24391,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/507301496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":507301496},{"type":"v1","addr":"172.21.15.204:6833","nonce":507301496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4028657219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4028657219},{"type":"v1","addr":"172.21.15.134:6833","nonce":4028657219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/57029119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":57029119},{"type":"v1","addr":"172.21.15.204:6835","nonce":57029119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3068880939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3068880939},{"type":"v1","addr":"172.21.15.204:6837","nonce":3068880939}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"e","rank":3,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.134:6835/1150003275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1150003275},{"type":"v1","addr":"172.21.15.134:6835","nonce":1150003275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/344883912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":344883912},{"type":"v1","addr":"172.21.15.204:6839","nonce":344883912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6837/891692210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":891692210},{"type":"v1","addr":"172.21.15.134:6837","nonce":891692210}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6839/3466309724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3466309724},{"type":"v1","addr":"172.21.15.134:6839","nonce":3466309724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:21:56.679 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 29 2024-09-13T18:21:56.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.677+0000 7fd484ff4640 1 -- 172.21.15.114:0/2051971392 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fd4680a1120 msgr2=0x7fd4680a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:56.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.677+0000 7fd484ff4640 1 --2- 172.21.15.114:0/2051971392 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fd4680a1120 0x7fd4680a35e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd4700057c0 tx=0x7fd470005a20 comp rx=0 tx=0).stop 2024-09-13T18:21:56.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.677+0000 7fd484ff4640 1 -- 172.21.15.114:0/2051971392 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd480105b10 msgr2=0x7fd48019d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:56.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.677+0000 7fd484ff4640 1 --2- 172.21.15.114:0/2051971392 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd480105b10 0x7fd48019d380 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd47800cab0 tx=0x7fd47800cf70 comp rx=0 tx=0).stop 2024-09-13T18:21:56.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.677+0000 7fd484ff4640 1 -- 172.21.15.114:0/2051971392 shutdown_connections 2024-09-13T18:21:56.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.677+0000 7fd484ff4640 1 --2- 172.21.15.114:0/2051971392 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fd4680a1120 0x7fd4680a35e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.677+0000 7fd484ff4640 1 --2- 172.21.15.114:0/2051971392 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd480105b10 0x7fd48019d380 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.677+0000 7fd484ff4640 1 --2- 172.21.15.114:0/2051971392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4801031e0 0x7fd480196310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.677+0000 7fd484ff4640 1 --2- 172.21.15.114:0/2051971392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4801008b0 0x7fd480195dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.677+0000 7fd484ff4640 1 -- 172.21.15.114:0/2051971392 >> 172.21.15.114:0/2051971392 conn(0x7fd4800fa430 msgr2=0x7fd480101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:56.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.677+0000 7fd484ff4640 1 -- 172.21.15.114:0/2051971392 shutdown_connections 2024-09-13T18:21:56.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.677+0000 7fd484ff4640 1 -- 172.21.15.114:0/2051971392 wait complete. 2024-09-13T18:21:56.692 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-13T18:20:01.895117+0000', 'modified': '2024-09-13T18:20:29.250121+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24521, 'mds_1': 24482, 'mds_2': 24475, 'mds_3': 24458, 'mds_4': 14628}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14616': {'gid': 14616, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.114:6839/2046516451', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6838', 'nonce': 2046516451}, {'type': 'v1', 'addr': '172.21.15.114:6839', 'nonce': 2046516451}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_14628': {'gid': 14628, 'name': 'a', 'rank': 4, 'incarnation': 27, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.114:6841/1956923446', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6840', 'nonce': 1956923446}, {'type': 'v1', 'addr': '172.21.15.114:6841', 'nonce': 1956923446}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24391': {'gid': 24391, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6833/507301496', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6832', 'nonce': 507301496}, {'type': 'v1', 'addr': '172.21.15.204:6833', 'nonce': 507301496}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24422': {'gid': 24422, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6833/4028657219', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6832', 'nonce': 4028657219}, {'type': 'v1', 'addr': '172.21.15.134:6833', 'nonce': 4028657219}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24445': {'gid': 24445, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6835/57029119', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6834', 'nonce': 57029119}, {'type': 'v1', 'addr': '172.21.15.204:6835', 'nonce': 57029119}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24457': {'gid': 24457, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6837/3068880939', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6836', 'nonce': 3068880939}, {'type': 'v1', 'addr': '172.21.15.204:6837', 'nonce': 3068880939}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24458': {'gid': 24458, 'name': 'e', 'rank': 3, 'incarnation': 25, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.134:6835/1150003275', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6834', 'nonce': 1150003275}, {'type': 'v1', 'addr': '172.21.15.134:6835', 'nonce': 1150003275}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24475': {'gid': 24475, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.204:6839/344883912', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6838', 'nonce': 344883912}, {'type': 'v1', 'addr': '172.21.15.204:6839', 'nonce': 344883912}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24482': {'gid': 24482, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.134:6837/891692210', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6836', 'nonce': 891692210}, {'type': 'v1', 'addr': '172.21.15.134:6837', 'nonce': 891692210}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24521': {'gid': 24521, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.134:6839/3466309724', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6838', 'nonce': 3466309724}, {'type': 'v1', 'addr': '172.21.15.134:6839', 'nonce': 3466309724}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-09-13T18:21:56.692 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-13T18:21:56.692 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-13T18:21:56.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.789+0000 7f9c28e53640 1 -- 172.21.15.114:0/2766621941 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c240fe7b0 msgr2=0x7f9c24108360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:56.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.789+0000 7f9c28e53640 1 --2- 172.21.15.114:0/2766621941 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c240fe7b0 0x7f9c24108360 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9c14009900 tx=0x7f9c1402a410 comp rx=0 tx=0).stop 2024-09-13T18:21:56.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.789+0000 7f9c28e53640 1 -- 172.21.15.114:0/2766621941 shutdown_connections 2024-09-13T18:21:56.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.789+0000 7f9c28e53640 1 --2- 172.21.15.114:0/2766621941 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c240fe7b0 0x7f9c24108360 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.789+0000 7f9c28e53640 1 --2- 172.21.15.114:0/2766621941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c240fddf0 0x7f9c240fe270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.789+0000 7f9c28e53640 1 --2- 172.21.15.114:0/2766621941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c24101380 0x7f9c24101780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.789+0000 7f9c28e53640 1 -- 172.21.15.114:0/2766621941 >> 172.21.15.114:0/2766621941 conn(0x7f9c240f9b60 msgr2=0x7f9c240fbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:56.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.789+0000 7f9c28e53640 1 -- 172.21.15.114:0/2766621941 shutdown_connections 2024-09-13T18:21:56.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.789+0000 7f9c28e53640 1 -- 172.21.15.114:0/2766621941 wait complete. 2024-09-13T18:21:56.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.790+0000 7f9c28e53640 1 Processor -- start 2024-09-13T18:21:56.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.790+0000 7f9c28e53640 1 -- start start 2024-09-13T18:21:56.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.790+0000 7f9c28e53640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c240fddf0 0x7f9c24195c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:56.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.790+0000 7f9c28e53640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c240fe7b0 0x7f9c241961a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:56.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.790+0000 7f9c28e53640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c24101380 0x7f9c2419d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:56.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.790+0000 7f9c28e53640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2419d740 con 0x7f9c240fe7b0 2024-09-13T18:21:56.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.790+0000 7f9c28e53640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2419d8b0 con 0x7f9c24101380 2024-09-13T18:21:56.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.790+0000 7f9c28e53640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2419da70 con 0x7f9c240fddf0 2024-09-13T18:21:56.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.790+0000 7f9c21d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c240fe7b0 0x7f9c241961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:56.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.790+0000 7f9c21d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c240fe7b0 0x7f9c241961a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33848/0 (socket says 172.21.15.114:33848) 2024-09-13T18:21:56.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.790+0000 7f9c21d9b640 1 -- 172.21.15.114:0/212025182 learned_addr learned my addr 172.21.15.114:0/212025182 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:56.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.790+0000 7f9c2259c640 1 --2- 172.21.15.114:0/212025182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c240fddf0 0x7f9c24195c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:56.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.790+0000 7f9c22d9d640 1 --2- 172.21.15.114:0/212025182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c24101380 0x7f9c2419d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:56.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.790+0000 7f9c21d9b640 1 -- 172.21.15.114:0/212025182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c240fddf0 msgr2=0x7f9c24195c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:56.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.790+0000 7f9c21d9b640 1 --2- 172.21.15.114:0/212025182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c240fddf0 0x7f9c24195c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.790+0000 7f9c21d9b640 1 -- 172.21.15.114:0/212025182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c24101380 msgr2=0x7f9c2419d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:56.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.790+0000 7f9c21d9b640 1 --2- 172.21.15.114:0/212025182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c24101380 0x7f9c2419d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.790+0000 7f9c21d9b640 1 -- 172.21.15.114:0/212025182 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c14009560 con 0x7f9c240fe7b0 2024-09-13T18:21:56.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.790+0000 7f9c22d9d640 1 --2- 172.21.15.114:0/212025182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c24101380 0x7f9c2419d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:21:56.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.790+0000 7f9c21d9b640 1 --2- 172.21.15.114:0/212025182 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c240fe7b0 0x7f9c241961a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c00c890 tx=0x7f9c0c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:56.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.791+0000 7f9c1b7fe640 1 -- 172.21.15.114:0/212025182 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c004220 con 0x7f9c240fe7b0 2024-09-13T18:21:56.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.791+0000 7f9c1b7fe640 1 -- 172.21.15.114:0/212025182 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9c0c007e40 con 0x7f9c240fe7b0 2024-09-13T18:21:56.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.791+0000 7f9c28e53640 1 -- 172.21.15.114:0/212025182 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c2419dd50 con 0x7f9c240fe7b0 2024-09-13T18:21:56.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.791+0000 7f9c1b7fe640 1 -- 172.21.15.114:0/212025182 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c004a00 con 0x7f9c240fe7b0 2024-09-13T18:21:56.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.791+0000 7f9c28e53640 1 -- 172.21.15.114:0/212025182 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c2419e1b0 con 0x7f9c240fe7b0 2024-09-13T18:21:56.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.793+0000 7f9c1b7fe640 1 -- 172.21.15.114:0/212025182 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f9c0c0079b0 con 0x7f9c240fe7b0 2024-09-13T18:21:56.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.793+0000 7f9c2bc50740 1 Processor -- start 2024-09-13T18:21:56.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.793+0000 7f9c2bc50740 1 -- start start 2024-09-13T18:21:56.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.793+0000 7f9c2bc50740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562541ae8500 0x562541ae8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:56.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.793+0000 7f9c2bc50740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562541ae8e40 0x562541aed900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:56.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.793+0000 7f9c2bc50740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562541aeded0 0x562541af0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:56.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.793+0000 7f9c2bc50740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x562541a437c0 con 0x562541ae8e40 2024-09-13T18:21:56.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.793+0000 7f9c2bc50740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562541a52b70 con 0x562541aeded0 2024-09-13T18:21:56.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.793+0000 7f9c2bc50740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562541a50300 con 0x562541ae8500 2024-09-13T18:21:56.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.793+0000 7f9c2259c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562541ae8e40 0x562541aed900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:56.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.794+0000 7f9c2259c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562541ae8e40 0x562541aed900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33860/0 (socket says 172.21.15.114:33860) 2024-09-13T18:21:56.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.794+0000 7f9c22d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562541ae8500 0x562541ae8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:56.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.794+0000 7f9c2259c640 1 -- 172.21.15.114:0/701121321 learned_addr learned my addr 172.21.15.114:0/701121321 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:56.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.794+0000 7f9c21d9b640 1 --2- 172.21.15.114:0/701121321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562541aeded0 0x562541af0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:56.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.794+0000 7f9c1b7fe640 1 --2- 172.21.15.114:0/212025182 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f9c100a1120 0x7f9c100a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:56.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.794+0000 7f9c2259c640 1 -- 172.21.15.114:0/701121321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562541ae8500 msgr2=0x562541ae8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:56.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.794+0000 7f9c2259c640 1 --2- 172.21.15.114:0/701121321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562541ae8500 0x562541ae8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.794+0000 7f9c2259c640 1 -- 172.21.15.114:0/701121321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562541aeded0 msgr2=0x562541af0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:56.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.794+0000 7f9c2259c640 1 --2- 172.21.15.114:0/701121321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562541aeded0 0x562541af0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.794+0000 7f9c2259c640 1 -- 172.21.15.114:0/701121321 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625419d84b0 con 0x562541ae8e40 2024-09-13T18:21:56.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.794+0000 7f9c21d9b640 1 --2- 172.21.15.114:0/701121321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562541aeded0 0x562541af0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:21:56.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.794+0000 7f9c22d9d640 1 --2- 172.21.15.114:0/212025182 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f9c100a1120 0x7f9c100a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:56.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.794+0000 7f9c1b7fe640 1 -- 172.21.15.114:0/212025182 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f9c0c0ccfe0 con 0x7f9c240fe7b0 2024-09-13T18:21:56.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.794+0000 7f9c2259c640 1 --2- 172.21.15.114:0/701121321 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562541ae8e40 0x562541aed900 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c009f20 tx=0x7f9c1c010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32470a2b62bde9f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:56.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.794+0000 7f9c18ff9640 1 -- 172.21.15.114:0/701121321 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c01b660 con 0x562541ae8e40 2024-09-13T18:21:56.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.794+0000 7f9c18ff9640 1 -- 172.21.15.114:0/701121321 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9c1c01bc50 con 0x562541ae8e40 2024-09-13T18:21:56.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.794+0000 7f9c18ff9640 1 -- 172.21.15.114:0/701121321 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c012cc0 con 0x562541ae8e40 2024-09-13T18:21:56.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.795+0000 7f9c2bc50740 1 -- 172.21.15.114:0/701121321 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562541ae8e40 msgr2=0x562541aed900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:56.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.795+0000 7f9c22d9d640 1 --2- 172.21.15.114:0/212025182 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f9c100a1120 0x7f9c100a35e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9c14005c40 tx=0x7f9c14009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:56.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.795+0000 7f9c2bc50740 1 --2- 172.21.15.114:0/701121321 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562541ae8e40 0x562541aed900 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c009f20 tx=0x7f9c1c010800 comp rx=0 tx=0).stop 2024-09-13T18:21:56.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.795+0000 7f9c2bc50740 1 -- 172.21.15.114:0/701121321 shutdown_connections 2024-09-13T18:21:56.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.795+0000 7f9c2bc50740 1 --2- 172.21.15.114:0/701121321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562541aeded0 0x562541af0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.795+0000 7f9c2bc50740 1 --2- 172.21.15.114:0/701121321 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562541ae8e40 0x562541aed900 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.795+0000 7f9c2bc50740 1 --2- 172.21.15.114:0/701121321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562541ae8500 0x562541ae8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.795+0000 7f9c2bc50740 1 -- 172.21.15.114:0/701121321 >> 172.21.15.114:0/701121321 conn(0x562541ae3b70 msgr2=0x562541ae3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:56.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.795+0000 7f9c2bc50740 1 -- 172.21.15.114:0/701121321 shutdown_connections 2024-09-13T18:21:56.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.795+0000 7f9c2bc50740 1 -- 172.21.15.114:0/701121321 wait complete. 2024-09-13T18:21:56.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.795+0000 7f9c2bc50740 1 -- start start 2024-09-13T18:21:56.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.795+0000 7f9c2bc50740 1 Processor -- start 2024-09-13T18:21:56.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.796+0000 7f9c2bc50740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562541ae8500 0x562541af2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:56.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.796+0000 7f9c2bc50740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562541ae8e40 0x562541af2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:56.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.796+0000 7f9c2bc50740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562541aeded0 0x562541af96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:56.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.796+0000 7f9c2bc50740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x562541a4bbf0 con 0x562541ae8500 2024-09-13T18:21:56.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.796+0000 7f9c2bc50740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562541a0a250 con 0x562541aeded0 2024-09-13T18:21:56.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.796+0000 7f9c2bc50740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562541a09ef0 con 0x562541ae8e40 2024-09-13T18:21:56.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.796+0000 7f9c2bc50740 10 client.0 _ll_register_callbacks cb 0x562541a3c740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:21:56.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.796+0000 7f9c2259c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562541ae8500 0x562541af2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:56.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.796+0000 7f9c2259c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562541ae8500 0x562541af2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33872/0 (socket says 172.21.15.114:33872) 2024-09-13T18:21:56.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.796+0000 7f9c2259c640 1 -- 172.21.15.114:0/719276476 learned_addr learned my addr 172.21.15.114:0/719276476 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:56.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.796+0000 7f9c21d9b640 1 --2- 172.21.15.114:0/719276476 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562541ae8e40 0x562541af2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:56.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.796+0000 7f9c2259c640 1 -- 172.21.15.114:0/719276476 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562541ae8e40 msgr2=0x562541af2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:56.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.796+0000 7f9c2259c640 1 --2- 172.21.15.114:0/719276476 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562541ae8e40 0x562541af2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.796+0000 7f9c2259c640 1 -- 172.21.15.114:0/719276476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562541aeded0 msgr2=0x562541af96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:56.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.796+0000 7f9c2259c640 1 --2- 172.21.15.114:0/719276476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562541aeded0 0x562541af96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.796+0000 7f9c2259c640 1 -- 172.21.15.114:0/719276476 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c1c009b80 con 0x562541ae8500 2024-09-13T18:21:56.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.796+0000 7f9c21d9b640 1 --2- 172.21.15.114:0/719276476 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562541ae8e40 0x562541af2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:21:56.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.796+0000 7f9c2259c640 1 --2- 172.21.15.114:0/719276476 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562541ae8500 0x562541af2130 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c00a010 tx=0x7f9c1c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50c7da7f9e95c0d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:56.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.796+0000 7f9c09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:21:56.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.796+0000 7f9c09ffb640 1 -- 172.21.15.114:0/719276476 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c012e10 con 0x562541ae8500 2024-09-13T18:21:56.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.796+0000 7f9c09ffb640 1 -- 172.21.15.114:0/719276476 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f9c1c003970 con 0x562541ae8500 2024-09-13T18:21:56.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.796+0000 7f9c2bc50740 1 -- 172.21.15.114:0/719276476 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56254173f3c0 con 0x562541ae8500 2024-09-13T18:21:56.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.796+0000 7f9c09ffb640 1 -- 172.21.15.114:0/719276476 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c1c003bb0 con 0x562541ae8500 2024-09-13T18:21:56.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.796+0000 7f9c2bc50740 10 client.15243 fetch_fsmap learned FSMap version 29 2024-09-13T18:21:56.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.796+0000 7f9c2bc50740 1 -- 172.21.15.114:0/719276476 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x562541a4a270 con 0x562541ae8500 2024-09-13T18:21:56.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.797+0000 7f9c09ffb640 1 -- 172.21.15.114:0/719276476 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56254173f3c0 con 0x562541ae8500 2024-09-13T18:21:56.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.797+0000 7f9c09ffb640 1 -- 172.21.15.114:0/719276476 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9c1c0041d0 con 0x562541ae8500 2024-09-13T18:21:56.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.797+0000 7f9c2bc50740 10 client.15243 fetch_fsmap finished waiting for FSMap version 29 2024-09-13T18:21:56.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.797+0000 7f9c2bc50740 10 client.15243 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-13T18:21:56.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.797+0000 7f9c2bc50740 20 client.15243 populate_metadata read hostname 'smithi114' 2024-09-13T18:21:56.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.797+0000 7f9c2bc50740 1 --2- 172.21.15.114:0/719276476 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x562541afdf00 0x562541b026b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:56.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.797+0000 7f9c2bc50740 4 client.15243 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:21:56.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.797+0000 7f9c2bc50740 1 -- 172.21.15.114:0/719276476 --> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5625417506c0 con 0x562541afdf00 2024-09-13T18:21:56.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.797+0000 7f9c21d9b640 1 --2- 172.21.15.114:0/719276476 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x562541afdf00 0x562541b026b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:56.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.797+0000 7f9c09ffb640 10 client.15243 ms_handle_connect on v2:172.21.15.114:6840/1956923446 2024-09-13T18:21:56.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.797+0000 7f9c21d9b640 1 --2- 172.21.15.114:0/719276476 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x562541afdf00 0x562541b026b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=72361fafb5ddb4a0 server_cookie=9d73e2cf50922b88 in_seq=0 out_seq=0 2024-09-13T18:21:56.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.798+0000 7f9c09ffb640 1 -- 172.21.15.114:0/719276476 <== mds.4 v2:172.21.15.114:6840/1956923446 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5625417506c0 con 0x562541afdf00 2024-09-13T18:21:56.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.798+0000 7f9c09ffb640 10 client.15243 handle_command_reply: tid=0 2024-09-13T18:21:56.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.798+0000 7f9c2bc50740 1 client.15243 shutdown 2024-09-13T18:21:56.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.798+0000 7f9c2bc50740 1 -- 172.21.15.114:0/719276476 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562541ae8500 msgr2=0x562541af2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:56.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.798+0000 7f9c2bc50740 1 --2- 172.21.15.114:0/719276476 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562541ae8500 0x562541af2130 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c00a010 tx=0x7f9c1c01d000 comp rx=0 tx=0).stop 2024-09-13T18:21:56.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.798+0000 7f9c2bc50740 1 -- 172.21.15.114:0/719276476 shutdown_connections 2024-09-13T18:21:56.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.798+0000 7f9c2bc50740 1 --2- 172.21.15.114:0/719276476 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x562541afdf00 0x562541b026b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.798+0000 7f9c2bc50740 1 --2- 172.21.15.114:0/719276476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562541aeded0 0x562541af96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.798+0000 7f9c2bc50740 1 --2- 172.21.15.114:0/719276476 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562541ae8e40 0x562541af2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.798+0000 7f9c09ffb640 0 client.15243 ms_handle_reset on v2:172.21.15.114:6840/1956923446 2024-09-13T18:21:56.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.798+0000 7f9c2bc50740 1 --2- 172.21.15.114:0/719276476 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562541ae8500 0x562541af2130 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.798+0000 7f9c2bc50740 1 -- 172.21.15.114:0/719276476 >> 172.21.15.114:0/719276476 conn(0x562541ae3b70 msgr2=0x562541ae5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:56.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.798+0000 7f9c2bc50740 1 -- 172.21.15.114:0/719276476 shutdown_connections 2024-09-13T18:21:56.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.798+0000 7f9c2bc50740 1 -- 172.21.15.114:0/719276476 wait complete. 2024-09-13T18:21:56.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.798+0000 7f9c2bc50740 20 client.15243 trim_cache size 0 max 16384 2024-09-13T18:21:56.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.812+0000 7f9c2bc50740 1 Processor -- start 2024-09-13T18:21:56.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.812+0000 7f9c2bc50740 1 -- start start 2024-09-13T18:21:56.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.812+0000 7f9c2bc50740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562541ae3b70 0x562541ae3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:56.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.812+0000 7f9c2bc50740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562541ae8500 0x562541af95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:56.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.813+0000 7f9c2bc50740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562541ae8e40 0x562541af1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:56.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.813+0000 7f9c2bc50740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625419cf460 con 0x562541ae8e40 2024-09-13T18:21:56.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.813+0000 7f9c2bc50740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562541affde0 con 0x562541ae3b70 2024-09-13T18:21:56.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.813+0000 7f9c2bc50740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625417506c0 con 0x562541ae8500 2024-09-13T18:21:56.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.813+0000 7f9c22d9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562541ae8e40 0x562541af1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:56.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.813+0000 7f9c22d9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562541ae8e40 0x562541af1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33874/0 (socket says 172.21.15.114:33874) 2024-09-13T18:21:56.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.813+0000 7f9c22d9d640 1 -- 172.21.15.114:0/1754093803 learned_addr learned my addr 172.21.15.114:0/1754093803 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:56.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.813+0000 7f9c21d9b640 1 --2- 172.21.15.114:0/1754093803 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562541ae8500 0x562541af95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:56.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.813+0000 7f9c2259c640 1 --2- 172.21.15.114:0/1754093803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562541ae3b70 0x562541ae3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:56.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.813+0000 7f9c22d9d640 1 -- 172.21.15.114:0/1754093803 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562541ae8500 msgr2=0x562541af95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:56.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.813+0000 7f9c22d9d640 1 --2- 172.21.15.114:0/1754093803 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562541ae8500 0x562541af95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.813+0000 7f9c22d9d640 1 -- 172.21.15.114:0/1754093803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562541ae3b70 msgr2=0x562541ae3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:56.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.813+0000 7f9c22d9d640 1 --2- 172.21.15.114:0/1754093803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562541ae3b70 0x562541ae3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.813+0000 7f9c22d9d640 1 -- 172.21.15.114:0/1754093803 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625419ec5f0 con 0x562541ae8e40 2024-09-13T18:21:56.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.813+0000 7f9c2259c640 1 --2- 172.21.15.114:0/1754093803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562541ae3b70 0x562541ae3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:21:56.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.813+0000 7f9c22d9d640 1 --2- 172.21.15.114:0/1754093803 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562541ae8e40 0x562541af1ee0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9c14005ab0 tx=0x7f9c1402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb2bdaa08b9127ab server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:56.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.813+0000 7f9c197fa640 1 -- 172.21.15.114:0/1754093803 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c14031b60 con 0x562541ae8e40 2024-09-13T18:21:56.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.813+0000 7f9c197fa640 1 -- 172.21.15.114:0/1754093803 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9c140044c0 con 0x562541ae8e40 2024-09-13T18:21:56.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.813+0000 7f9c197fa640 1 -- 172.21.15.114:0/1754093803 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1404a690 con 0x562541ae8e40 2024-09-13T18:21:56.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.814+0000 7f9c2bc50740 1 -- 172.21.15.114:0/1754093803 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562541ae8e40 msgr2=0x562541af1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:56.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.814+0000 7f9c2bc50740 1 --2- 172.21.15.114:0/1754093803 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562541ae8e40 0x562541af1ee0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9c14005ab0 tx=0x7f9c1402a410 comp rx=0 tx=0).stop 2024-09-13T18:21:56.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.814+0000 7f9c2bc50740 1 -- 172.21.15.114:0/1754093803 shutdown_connections 2024-09-13T18:21:56.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.814+0000 7f9c2bc50740 1 --2- 172.21.15.114:0/1754093803 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562541ae8e40 0x562541af1ee0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.814+0000 7f9c2bc50740 1 --2- 172.21.15.114:0/1754093803 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562541ae8500 0x562541af95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.814+0000 7f9c2bc50740 1 --2- 172.21.15.114:0/1754093803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562541ae3b70 0x562541ae3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.814+0000 7f9c2bc50740 1 -- 172.21.15.114:0/1754093803 >> 172.21.15.114:0/1754093803 conn(0x562541af0630 msgr2=0x562541ae9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:56.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.814+0000 7f9c2bc50740 1 -- 172.21.15.114:0/1754093803 shutdown_connections 2024-09-13T18:21:56.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.814+0000 7f9c2bc50740 1 -- 172.21.15.114:0/1754093803 wait complete. 2024-09-13T18:21:56.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.814+0000 7f9c2bc50740 1 -- start start 2024-09-13T18:21:56.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.814+0000 7f9c2bc50740 1 Processor -- start 2024-09-13T18:21:56.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.814+0000 7f9c2bc50740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562541ae3b70 0x562541af6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:56.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.814+0000 7f9c2bc50740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562541ae8500 0x562541af6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:56.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.814+0000 7f9c2bc50740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562541ae8e40 0x562541b54370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:56.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.814+0000 7f9c2bc50740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x562541af9820 con 0x562541ae3b70 2024-09-13T18:21:56.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.814+0000 7f9c2bc50740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562541af9990 con 0x562541ae8e40 2024-09-13T18:21:56.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.814+0000 7f9c2bc50740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562541b548b0 con 0x562541ae8500 2024-09-13T18:21:56.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.814+0000 7f9c2bc50740 10 client.0 _ll_register_callbacks cb 0x7f9c1c0096e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:21:56.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.814+0000 7f9c2259c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562541ae3b70 0x562541af6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:56.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.815+0000 7f9c2259c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562541ae3b70 0x562541af6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33884/0 (socket says 172.21.15.114:33884) 2024-09-13T18:21:56.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.815+0000 7f9c2259c640 1 -- 172.21.15.114:0/4003913681 learned_addr learned my addr 172.21.15.114:0/4003913681 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:56.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.815+0000 7f9c21d9b640 1 --2- 172.21.15.114:0/4003913681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562541ae8500 0x562541af6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:56.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.815+0000 7f9c22d9d640 1 --2- 172.21.15.114:0/4003913681 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562541ae8e40 0x562541b54370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:56.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.815+0000 7f9c2259c640 1 -- 172.21.15.114:0/4003913681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562541ae8500 msgr2=0x562541af6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:56.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.815+0000 7f9c2259c640 1 --2- 172.21.15.114:0/4003913681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562541ae8500 0x562541af6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.815+0000 7f9c2259c640 1 -- 172.21.15.114:0/4003913681 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562541ae8e40 msgr2=0x562541b54370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:56.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.815+0000 7f9c2259c640 1 --2- 172.21.15.114:0/4003913681 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562541ae8e40 0x562541b54370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.815+0000 7f9c2259c640 1 -- 172.21.15.114:0/4003913681 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c24108ae0 con 0x562541ae3b70 2024-09-13T18:21:56.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.815+0000 7f9c21d9b640 1 --2- 172.21.15.114:0/4003913681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562541ae8500 0x562541af6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:21:56.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.815+0000 7f9c2259c640 1 --2- 172.21.15.114:0/4003913681 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562541ae3b70 0x562541af6310 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x562541af15f0 tx=0x7f9c1c009430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26ee07680e99074b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:56.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.815+0000 7f9c09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:21:56.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.815+0000 7f9c09ffb640 1 -- 172.21.15.114:0/4003913681 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c0047a0 con 0x562541ae3b70 2024-09-13T18:21:56.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.815+0000 7f9c09ffb640 1 -- 172.21.15.114:0/4003913681 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f9c1c004bf0 con 0x562541ae3b70 2024-09-13T18:21:56.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.815+0000 7f9c2bc50740 1 -- 172.21.15.114:0/4003913681 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9c1c01bc50 con 0x562541ae3b70 2024-09-13T18:21:56.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.815+0000 7f9c09ffb640 1 -- 172.21.15.114:0/4003913681 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c1c004df0 con 0x562541ae3b70 2024-09-13T18:21:56.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.815+0000 7f9c2bc50740 10 client.15249 fetch_fsmap learned FSMap version 29 2024-09-13T18:21:56.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.815+0000 7f9c2bc50740 1 -- 172.21.15.114:0/4003913681 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x562541a11d00 con 0x562541ae3b70 2024-09-13T18:21:56.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.815+0000 7f9c09ffb640 1 -- 172.21.15.114:0/4003913681 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9c1c01bc50 con 0x562541ae3b70 2024-09-13T18:21:56.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.816+0000 7f9c09ffb640 1 -- 172.21.15.114:0/4003913681 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9c1c02e040 con 0x562541ae3b70 2024-09-13T18:21:56.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.816+0000 7f9c2bc50740 10 client.15249 fetch_fsmap finished waiting for FSMap version 29 2024-09-13T18:21:56.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.816+0000 7f9c2bc50740 10 client.15249 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-13T18:21:56.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.816+0000 7f9c2bc50740 20 client.15249 populate_metadata read hostname 'smithi114' 2024-09-13T18:21:56.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.816+0000 7f9c2bc50740 1 --2- 172.21.15.114:0/4003913681 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x562541aeded0 0x562541b55c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:56.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.816+0000 7f9c2bc50740 4 client.15249 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T18:21:56.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.816+0000 7f9c2bc50740 1 -- 172.21.15.114:0/4003913681 --> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5625417a3d10 con 0x562541aeded0 2024-09-13T18:21:56.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.816+0000 7f9c21d9b640 1 --2- 172.21.15.114:0/4003913681 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x562541aeded0 0x562541b55c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:56.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.816+0000 7f9c21d9b640 1 --2- 172.21.15.114:0/4003913681 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x562541aeded0 0x562541b55c00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b318f9390cc2fa60 server_cookie=8f6ee342b45b1764 in_seq=0 out_seq=0 2024-09-13T18:21:56.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.816+0000 7f9c09ffb640 10 client.15249 ms_handle_connect on v2:172.21.15.114:6840/1956923446 2024-09-13T18:21:56.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.816+0000 7f9c09ffb640 1 -- 172.21.15.114:0/4003913681 <== mds.4 v2:172.21.15.114:6840/1956923446 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5625417a3d10 con 0x562541aeded0 2024-09-13T18:21:56.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.816+0000 7f9c09ffb640 10 client.15249 handle_command_reply: tid=0 2024-09-13T18:21:56.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.816+0000 7f9c2bc50740 1 client.15249 shutdown 2024-09-13T18:21:56.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.816+0000 7f9c2bc50740 1 -- 172.21.15.114:0/4003913681 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562541ae3b70 msgr2=0x562541af6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:56.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.816+0000 7f9c2bc50740 1 --2- 172.21.15.114:0/4003913681 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562541ae3b70 0x562541af6310 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x562541af15f0 tx=0x7f9c1c009430 comp rx=0 tx=0).stop 2024-09-13T18:21:56.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.816+0000 7f9c2bc50740 1 -- 172.21.15.114:0/4003913681 shutdown_connections 2024-09-13T18:21:56.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.816+0000 7f9c2bc50740 1 --2- 172.21.15.114:0/4003913681 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x562541aeded0 0x562541b55c00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.816+0000 7f9c2bc50740 1 --2- 172.21.15.114:0/4003913681 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562541ae8e40 0x562541b54370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.816+0000 7f9c09ffb640 0 client.15249 ms_handle_reset on v2:172.21.15.114:6840/1956923446 2024-09-13T18:21:56.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.816+0000 7f9c2bc50740 1 --2- 172.21.15.114:0/4003913681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562541ae8500 0x562541af6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.816+0000 7f9c2bc50740 1 --2- 172.21.15.114:0/4003913681 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562541ae3b70 0x562541af6310 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.816+0000 7f9c2bc50740 1 -- 172.21.15.114:0/4003913681 >> 172.21.15.114:0/4003913681 conn(0x562541af0630 msgr2=0x562541af1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:56.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.816+0000 7f9c2bc50740 1 -- 172.21.15.114:0/4003913681 shutdown_connections 2024-09-13T18:21:56.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.816+0000 7f9c2bc50740 1 -- 172.21.15.114:0/4003913681 wait complete. 2024-09-13T18:21:56.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.817+0000 7f9c2bc50740 20 client.15249 trim_cache size 0 max 16384 2024-09-13T18:21:56.827 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:21:56.828 INFO:teuthology.orchestra.run.smithi114.stdout: "cluster_fsid": "97e7edf2-71fb-11ef-bceb-c7b262605968", 2024-09-13T18:21:56.828 INFO:teuthology.orchestra.run.smithi114.stdout: "whoami": 4, 2024-09-13T18:21:56.828 INFO:teuthology.orchestra.run.smithi114.stdout: "id": 14628, 2024-09-13T18:21:56.828 INFO:teuthology.orchestra.run.smithi114.stdout: "want_state": "up:active", 2024-09-13T18:21:56.828 INFO:teuthology.orchestra.run.smithi114.stdout: "state": "up:active", 2024-09-13T18:21:56.828 INFO:teuthology.orchestra.run.smithi114.stdout: "fs_name": "cephfs", 2024-09-13T18:21:56.828 INFO:teuthology.orchestra.run.smithi114.stdout: "rank_uptime": 88.571585984999999, 2024-09-13T18:21:56.828 INFO:teuthology.orchestra.run.smithi114.stdout: "mdsmap_epoch": 29, 2024-09-13T18:21:56.828 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch": 93, 2024-09-13T18:21:56.828 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch_barrier": 91, 2024-09-13T18:21:56.828 INFO:teuthology.orchestra.run.smithi114.stdout: "uptime": 123.198040316 2024-09-13T18:21:56.828 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:21:56.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.817+0000 7f9c28e53640 1 -- 172.21.15.114:0/212025182 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f9c100a1120 msgr2=0x7f9c100a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:56.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.817+0000 7f9c28e53640 1 --2- 172.21.15.114:0/212025182 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f9c100a1120 0x7f9c100a35e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9c14005c40 tx=0x7f9c14009260 comp rx=0 tx=0).stop 2024-09-13T18:21:56.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.817+0000 7f9c28e53640 1 -- 172.21.15.114:0/212025182 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c240fe7b0 msgr2=0x7f9c241961a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:56.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.817+0000 7f9c28e53640 1 --2- 172.21.15.114:0/212025182 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c240fe7b0 0x7f9c241961a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c00c890 tx=0x7f9c0c00cd50 comp rx=0 tx=0).stop 2024-09-13T18:21:56.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.817+0000 7f9c28e53640 1 -- 172.21.15.114:0/212025182 shutdown_connections 2024-09-13T18:21:56.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.817+0000 7f9c28e53640 1 --2- 172.21.15.114:0/212025182 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f9c100a1120 0x7f9c100a35e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.817+0000 7f9c28e53640 1 --2- 172.21.15.114:0/212025182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c24101380 0x7f9c2419d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.817+0000 7f9c28e53640 1 --2- 172.21.15.114:0/212025182 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c240fe7b0 0x7f9c241961a0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.817+0000 7f9c28e53640 1 --2- 172.21.15.114:0/212025182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c240fddf0 0x7f9c24195c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.817+0000 7f9c28e53640 1 -- 172.21.15.114:0/212025182 >> 172.21.15.114:0/212025182 conn(0x7f9c240f9b60 msgr2=0x7f9c24104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:56.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.817+0000 7f9c28e53640 1 -- 172.21.15.114:0/212025182 shutdown_connections 2024-09-13T18:21:56.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.817+0000 7f9c28e53640 1 -- 172.21.15.114:0/212025182 wait complete. 2024-09-13T18:21:56.831 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-09-13T18:21:56.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.927+0000 7ff97eae9640 1 -- 172.21.15.114:0/3960864615 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9780ffe70 msgr2=0x7ff9781002f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:56.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.927+0000 7ff97eae9640 1 --2- 172.21.15.114:0/3960864615 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9780ffe70 0x7ff9781002f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff964009900 tx=0x7ff96402a410 comp rx=0 tx=0).stop 2024-09-13T18:21:56.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.927+0000 7ff97eae9640 1 -- 172.21.15.114:0/3960864615 shutdown_connections 2024-09-13T18:21:56.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.927+0000 7ff97eae9640 1 --2- 172.21.15.114:0/3960864615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff978100830 0x7ff9781070c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.927+0000 7ff97eae9640 1 --2- 172.21.15.114:0/3960864615 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9780ffe70 0x7ff9781002f0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.927+0000 7ff97eae9640 1 --2- 172.21.15.114:0/3960864615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9780fec80 0x7ff9780ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.927+0000 7ff97eae9640 1 -- 172.21.15.114:0/3960864615 >> 172.21.15.114:0/3960864615 conn(0x7ff9780fa3e0 msgr2=0x7ff9780fc820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:56.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.927+0000 7ff97eae9640 1 -- 172.21.15.114:0/3960864615 shutdown_connections 2024-09-13T18:21:56.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.927+0000 7ff97eae9640 1 -- 172.21.15.114:0/3960864615 wait complete. 2024-09-13T18:21:56.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.928+0000 7ff97eae9640 1 Processor -- start 2024-09-13T18:21:56.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.928+0000 7ff97eae9640 1 -- start start 2024-09-13T18:21:56.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.928+0000 7ff97eae9640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9780fec80 0x7ff978069160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:56.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.928+0000 7ff97eae9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9780ffe70 0x7ff97819a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:56.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.928+0000 7ff97eae9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff978100830 0x7ff97819cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:56.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.928+0000 7ff97eae9640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff97819d7b0 con 0x7ff9780fec80 2024-09-13T18:21:56.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.928+0000 7ff97eae9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff97819d920 con 0x7ff978100830 2024-09-13T18:21:56.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.928+0000 7ff97eae9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff97819dae0 con 0x7ff9780ffe70 2024-09-13T18:21:56.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.928+0000 7ff97c885640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9780fec80 0x7ff978069160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:56.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.928+0000 7ff97c885640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9780fec80 0x7ff978069160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33898/0 (socket says 172.21.15.114:33898) 2024-09-13T18:21:56.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.928+0000 7ff97c885640 1 -- 172.21.15.114:0/1166381407 learned_addr learned my addr 172.21.15.114:0/1166381407 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:56.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.928+0000 7ff977fff640 1 --2- 172.21.15.114:0/1166381407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9780ffe70 0x7ff97819a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:56.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.928+0000 7ff97d086640 1 --2- 172.21.15.114:0/1166381407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff978100830 0x7ff97819cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:56.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.928+0000 7ff97c885640 1 -- 172.21.15.114:0/1166381407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9780ffe70 msgr2=0x7ff97819a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:56.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.928+0000 7ff97c885640 1 --2- 172.21.15.114:0/1166381407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9780ffe70 0x7ff97819a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.928+0000 7ff97c885640 1 -- 172.21.15.114:0/1166381407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff978100830 msgr2=0x7ff97819cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:56.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.928+0000 7ff97c885640 1 --2- 172.21.15.114:0/1166381407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff978100830 0x7ff97819cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.928+0000 7ff97c885640 1 -- 172.21.15.114:0/1166381407 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff964009560 con 0x7ff9780fec80 2024-09-13T18:21:56.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.928+0000 7ff97d086640 1 --2- 172.21.15.114:0/1166381407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff978100830 0x7ff97819cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:21:56.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.929+0000 7ff97c885640 1 --2- 172.21.15.114:0/1166381407 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9780fec80 0x7ff978069160 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff96c00e9c0 tx=0x7ff96c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:56.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.929+0000 7ff975ffb640 1 -- 172.21.15.114:0/1166381407 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96c0096e0 con 0x7ff9780fec80 2024-09-13T18:21:56.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.929+0000 7ff975ffb640 1 -- 172.21.15.114:0/1166381407 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff96c0044c0 con 0x7ff9780fec80 2024-09-13T18:21:56.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.929+0000 7ff97eae9640 1 -- 172.21.15.114:0/1166381407 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff97819dd60 con 0x7ff9780fec80 2024-09-13T18:21:56.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.929+0000 7ff975ffb640 1 -- 172.21.15.114:0/1166381407 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96c003e30 con 0x7ff9780fec80 2024-09-13T18:21:56.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.929+0000 7ff97eae9640 1 -- 172.21.15.114:0/1166381407 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff97819e190 con 0x7ff9780fec80 2024-09-13T18:21:56.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.931+0000 7ff975ffb640 1 -- 172.21.15.114:0/1166381407 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7ff96c0035f0 con 0x7ff9780fec80 2024-09-13T18:21:56.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.932+0000 7ff98180d740 1 Processor -- start 2024-09-13T18:21:56.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.932+0000 7ff98180d740 1 -- start start 2024-09-13T18:21:56.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.932+0000 7ff975ffb640 1 --2- 172.21.15.114:0/1166381407 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7ff9680a1120 0x7ff9680a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:56.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.932+0000 7ff98180d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df1f645500 0x55df1f645900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:56.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.932+0000 7ff98180d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df1f645e40 0x55df1f64a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:56.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.932+0000 7ff98180d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df1f64aed0 0x55df1f64d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:56.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.932+0000 7ff98180d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df1f5a07c0 con 0x55df1f64aed0 2024-09-13T18:21:56.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.932+0000 7ff98180d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df1f5afb70 con 0x55df1f645500 2024-09-13T18:21:56.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.932+0000 7ff98180d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df1f5ad300 con 0x55df1f645e40 2024-09-13T18:21:56.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.932+0000 7ff97d086640 1 --2- 172.21.15.114:0/1166381407 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7ff9680a1120 0x7ff9680a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:56.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.932+0000 7ff975ffb640 1 -- 172.21.15.114:0/1166381407 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7ff96c014070 con 0x7ff9780fec80 2024-09-13T18:21:56.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.932+0000 7ff97d086640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df1f64aed0 0x55df1f64d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:56.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.932+0000 7ff977fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df1f645e40 0x55df1f64a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:56.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.932+0000 7ff97d086640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df1f64aed0 0x55df1f64d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33914/0 (socket says 172.21.15.114:33914) 2024-09-13T18:21:56.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.932+0000 7ff97d086640 1 -- 172.21.15.114:0/1000729012 learned_addr learned my addr 172.21.15.114:0/1000729012 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:56.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.932+0000 7ff97c885640 1 --2- 172.21.15.114:0/1000729012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df1f645500 0x55df1f645900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:56.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.932+0000 7ff97d086640 1 -- 172.21.15.114:0/1000729012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df1f645e40 msgr2=0x55df1f64a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:56.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.932+0000 7ff97d086640 1 --2- 172.21.15.114:0/1000729012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df1f645e40 0x55df1f64a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.932+0000 7ff97d086640 1 -- 172.21.15.114:0/1000729012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df1f645500 msgr2=0x55df1f645900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:56.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.932+0000 7ff97d086640 1 --2- 172.21.15.114:0/1000729012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df1f645500 0x55df1f645900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.932+0000 7ff97d086640 1 -- 172.21.15.114:0/1000729012 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df1f5354b0 con 0x55df1f64aed0 2024-09-13T18:21:56.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.932+0000 7ff97c885640 1 --2- 172.21.15.114:0/1000729012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df1f645500 0x55df1f645900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:21:56.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.932+0000 7ff977fff640 1 --2- 172.21.15.114:0/1000729012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df1f645e40 0x55df1f64a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:21:56.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.932+0000 7ff97d086640 1 --2- 172.21.15.114:0/1166381407 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7ff9680a1120 0x7ff9680a35e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff970014ca0 tx=0x7ff970009ea0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:56.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.932+0000 7ff97d086640 1 --2- 172.21.15.114:0/1000729012 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df1f64aed0 0x55df1f64d390 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff970005c40 tx=0x7ff97001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=566a8a01ff6c49d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:56.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.933+0000 7ff962ffd640 1 -- 172.21.15.114:0/1000729012 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff970014310 con 0x55df1f64aed0 2024-09-13T18:21:56.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.933+0000 7ff962ffd640 1 -- 172.21.15.114:0/1000729012 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff97001a040 con 0x55df1f64aed0 2024-09-13T18:21:56.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.933+0000 7ff962ffd640 1 -- 172.21.15.114:0/1000729012 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff970004270 con 0x55df1f64aed0 2024-09-13T18:21:56.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.933+0000 7ff98180d740 1 -- 172.21.15.114:0/1000729012 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df1f64aed0 msgr2=0x55df1f64d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:56.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.933+0000 7ff98180d740 1 --2- 172.21.15.114:0/1000729012 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df1f64aed0 0x55df1f64d390 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff970005c40 tx=0x7ff97001b000 comp rx=0 tx=0).stop 2024-09-13T18:21:56.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.933+0000 7ff98180d740 1 -- 172.21.15.114:0/1000729012 shutdown_connections 2024-09-13T18:21:56.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.933+0000 7ff98180d740 1 --2- 172.21.15.114:0/1000729012 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df1f64aed0 0x55df1f64d390 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.933+0000 7ff98180d740 1 --2- 172.21.15.114:0/1000729012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df1f645e40 0x55df1f64a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.933+0000 7ff98180d740 1 --2- 172.21.15.114:0/1000729012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df1f645500 0x55df1f645900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.933+0000 7ff98180d740 1 -- 172.21.15.114:0/1000729012 >> 172.21.15.114:0/1000729012 conn(0x55df1f640b70 msgr2=0x55df1f640f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:56.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.933+0000 7ff98180d740 1 -- 172.21.15.114:0/1000729012 shutdown_connections 2024-09-13T18:21:56.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.933+0000 7ff98180d740 1 -- 172.21.15.114:0/1000729012 wait complete. 2024-09-13T18:21:56.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.934+0000 7ff98180d740 1 -- start start 2024-09-13T18:21:56.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.934+0000 7ff98180d740 1 Processor -- start 2024-09-13T18:21:56.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.934+0000 7ff98180d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df1f645500 0x55df1f64f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:56.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.934+0000 7ff98180d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df1f645e40 0x55df1f64f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:56.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.934+0000 7ff98180d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df1f64aed0 0x55df1f6566c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:56.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.934+0000 7ff98180d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df1f5a8bf0 con 0x55df1f64aed0 2024-09-13T18:21:56.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.934+0000 7ff98180d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df1f567250 con 0x55df1f645500 2024-09-13T18:21:56.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.934+0000 7ff98180d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df1f566ef0 con 0x55df1f645e40 2024-09-13T18:21:56.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.934+0000 7ff98180d740 10 client.0 _ll_register_callbacks cb 0x55df1f599740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:21:56.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.934+0000 7ff97d086640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df1f64aed0 0x55df1f6566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:56.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.934+0000 7ff97d086640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df1f64aed0 0x55df1f6566c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33920/0 (socket says 172.21.15.114:33920) 2024-09-13T18:21:56.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.934+0000 7ff97d086640 1 -- 172.21.15.114:0/4039073873 learned_addr learned my addr 172.21.15.114:0/4039073873 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:56.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.935+0000 7ff977fff640 1 --2- 172.21.15.114:0/4039073873 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df1f645e40 0x55df1f64f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:56.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.935+0000 7ff97d086640 1 -- 172.21.15.114:0/4039073873 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df1f645e40 msgr2=0x55df1f64f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:56.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.935+0000 7ff97c885640 1 --2- 172.21.15.114:0/4039073873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df1f645500 0x55df1f64f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:56.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.935+0000 7ff97d086640 1 --2- 172.21.15.114:0/4039073873 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df1f645e40 0x55df1f64f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.935+0000 7ff97d086640 1 -- 172.21.15.114:0/4039073873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df1f645500 msgr2=0x55df1f64f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:56.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.935+0000 7ff97d086640 1 --2- 172.21.15.114:0/4039073873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df1f645500 0x55df1f64f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.935+0000 7ff97d086640 1 -- 172.21.15.114:0/4039073873 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff970014cd0 con 0x55df1f64aed0 2024-09-13T18:21:56.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.935+0000 7ff977fff640 1 --2- 172.21.15.114:0/4039073873 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df1f645e40 0x55df1f64f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:21:56.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.935+0000 7ff97d086640 1 --2- 172.21.15.114:0/4039073873 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df1f64aed0 0x55df1f6566c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff970009540 tx=0x7ff97000fc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44cd4a0a703c13b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:56.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.935+0000 7ff957fff640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:21:56.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.935+0000 7ff957fff640 1 -- 172.21.15.114:0/4039073873 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff970004a00 con 0x55df1f64aed0 2024-09-13T18:21:56.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.935+0000 7ff957fff640 1 -- 172.21.15.114:0/4039073873 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7ff970004b60 con 0x55df1f64aed0 2024-09-13T18:21:56.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.935+0000 7ff98180d740 1 -- 172.21.15.114:0/4039073873 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55df1f29c3c0 con 0x55df1f64aed0 2024-09-13T18:21:56.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.935+0000 7ff957fff640 1 -- 172.21.15.114:0/4039073873 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff970004d20 con 0x55df1f64aed0 2024-09-13T18:21:56.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.935+0000 7ff98180d740 10 client.15261 fetch_fsmap learned FSMap version 29 2024-09-13T18:21:56.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.935+0000 7ff98180d740 1 -- 172.21.15.114:0/4039073873 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55df1f5a7270 con 0x55df1f64aed0 2024-09-13T18:21:56.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.935+0000 7ff957fff640 1 -- 172.21.15.114:0/4039073873 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55df1f29c3c0 con 0x55df1f64aed0 2024-09-13T18:21:56.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.935+0000 7ff957fff640 1 -- 172.21.15.114:0/4039073873 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff970030020 con 0x55df1f64aed0 2024-09-13T18:21:56.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.935+0000 7ff98180d740 10 client.15261 fetch_fsmap finished waiting for FSMap version 29 2024-09-13T18:21:56.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.935+0000 7ff98180d740 10 client.15261 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-13T18:21:56.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.935+0000 7ff98180d740 20 client.15261 populate_metadata read hostname 'smithi114' 2024-09-13T18:21:56.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.935+0000 7ff98180d740 1 --2- 172.21.15.114:0/4039073873 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55df1f65b570 0x55df1f65f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:56.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.935+0000 7ff98180d740 4 client.15261 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:21:56.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.935+0000 7ff98180d740 1 -- 172.21.15.114:0/4039073873 --> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55df1f2ad6c0 con 0x55df1f65b570 2024-09-13T18:21:56.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.936+0000 7ff97c885640 1 --2- 172.21.15.114:0/4039073873 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55df1f65b570 0x55df1f65f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:56.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.936+0000 7ff957fff640 10 client.15261 ms_handle_connect on v2:172.21.15.134:6834/1150003275 2024-09-13T18:21:56.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.936+0000 7ff97c885640 1 --2- 172.21.15.114:0/4039073873 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55df1f65b570 0x55df1f65f710 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cd04e851464ffd33 server_cookie=a4d4eac219afd858 in_seq=0 out_seq=0 2024-09-13T18:21:56.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.937+0000 7ff957fff640 1 -- 172.21.15.114:0/4039073873 <== mds.3 v2:172.21.15.134:6834/1150003275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55df1f2ad6c0 con 0x55df1f65b570 2024-09-13T18:21:56.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.937+0000 7ff957fff640 10 client.15261 handle_command_reply: tid=0 2024-09-13T18:21:56.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.937+0000 7ff98180d740 1 client.15261 shutdown 2024-09-13T18:21:56.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.937+0000 7ff98180d740 1 -- 172.21.15.114:0/4039073873 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df1f64aed0 msgr2=0x55df1f6566c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:56.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.937+0000 7ff98180d740 1 --2- 172.21.15.114:0/4039073873 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df1f64aed0 0x55df1f6566c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff970009540 tx=0x7ff97000fc30 comp rx=0 tx=0).stop 2024-09-13T18:21:56.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.937+0000 7ff98180d740 1 -- 172.21.15.114:0/4039073873 shutdown_connections 2024-09-13T18:21:56.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.937+0000 7ff98180d740 1 --2- 172.21.15.114:0/4039073873 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55df1f65b570 0x55df1f65f710 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.937+0000 7ff98180d740 1 --2- 172.21.15.114:0/4039073873 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df1f64aed0 0x55df1f6566c0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.937+0000 7ff98180d740 1 --2- 172.21.15.114:0/4039073873 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df1f645e40 0x55df1f64f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.937+0000 7ff98180d740 1 --2- 172.21.15.114:0/4039073873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df1f645500 0x55df1f64f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.937+0000 7ff957fff640 0 client.15261 ms_handle_reset on v2:172.21.15.134:6834/1150003275 2024-09-13T18:21:56.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.937+0000 7ff98180d740 1 -- 172.21.15.114:0/4039073873 >> 172.21.15.114:0/4039073873 conn(0x55df1f640b70 msgr2=0x55df1f642380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:56.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.937+0000 7ff98180d740 1 -- 172.21.15.114:0/4039073873 shutdown_connections 2024-09-13T18:21:56.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.937+0000 7ff98180d740 1 -- 172.21.15.114:0/4039073873 wait complete. 2024-09-13T18:21:56.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.937+0000 7ff98180d740 20 client.15261 trim_cache size 0 max 16384 2024-09-13T18:21:56.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.952+0000 7ff98180d740 1 Processor -- start 2024-09-13T18:21:56.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.952+0000 7ff98180d740 1 -- start start 2024-09-13T18:21:56.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.952+0000 7ff98180d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df1f645500 0x55df1f64ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:56.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.952+0000 7ff98180d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df1f645e40 0x55df1f6565e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:56.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.952+0000 7ff98180d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df1f64aed0 0x55df1f64f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:56.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.952+0000 7ff98180d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df1f52c460 con 0x55df1f645e40 2024-09-13T18:21:56.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.952+0000 7ff98180d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df1f5a1990 con 0x55df1f645500 2024-09-13T18:21:56.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.952+0000 7ff98180d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df1f54a410 con 0x55df1f64aed0 2024-09-13T18:21:56.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.952+0000 7ff977fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df1f645e40 0x55df1f6565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:56.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.952+0000 7ff977fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df1f645e40 0x55df1f6565e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33924/0 (socket says 172.21.15.114:33924) 2024-09-13T18:21:56.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.952+0000 7ff977fff640 1 -- 172.21.15.114:0/4007305803 learned_addr learned my addr 172.21.15.114:0/4007305803 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:56.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.952+0000 7ff97d086640 1 --2- 172.21.15.114:0/4007305803 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df1f64aed0 0x55df1f64f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:56.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.952+0000 7ff97c885640 1 --2- 172.21.15.114:0/4007305803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df1f645500 0x55df1f64ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:56.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.952+0000 7ff977fff640 1 -- 172.21.15.114:0/4007305803 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df1f64aed0 msgr2=0x55df1f64f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:56.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.952+0000 7ff977fff640 1 --2- 172.21.15.114:0/4007305803 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df1f64aed0 0x55df1f64f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.952+0000 7ff977fff640 1 -- 172.21.15.114:0/4007305803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df1f645500 msgr2=0x55df1f64ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:56.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.952+0000 7ff977fff640 1 --2- 172.21.15.114:0/4007305803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df1f645500 0x55df1f64ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.952+0000 7ff977fff640 1 -- 172.21.15.114:0/4007305803 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df1f5495f0 con 0x55df1f645e40 2024-09-13T18:21:56.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.952+0000 7ff97d086640 1 --2- 172.21.15.114:0/4007305803 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df1f64aed0 0x55df1f64f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:21:56.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.952+0000 7ff977fff640 1 --2- 172.21.15.114:0/4007305803 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df1f645e40 0x55df1f6565e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff96402a930 tx=0x7ff9640317c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a91a0ccb3893d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:56.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.952+0000 7ff9637fe640 1 -- 172.21.15.114:0/4007305803 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff964004380 con 0x55df1f645e40 2024-09-13T18:21:56.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.952+0000 7ff9637fe640 1 -- 172.21.15.114:0/4007305803 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff9640044e0 con 0x55df1f645e40 2024-09-13T18:21:56.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.952+0000 7ff9637fe640 1 -- 172.21.15.114:0/4007305803 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff964003e50 con 0x55df1f645e40 2024-09-13T18:21:56.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.953+0000 7ff98180d740 1 -- 172.21.15.114:0/4007305803 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df1f645e40 msgr2=0x55df1f6565e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:56.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.953+0000 7ff98180d740 1 --2- 172.21.15.114:0/4007305803 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df1f645e40 0x55df1f6565e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff96402a930 tx=0x7ff9640317c0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.953+0000 7ff98180d740 1 -- 172.21.15.114:0/4007305803 shutdown_connections 2024-09-13T18:21:56.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.953+0000 7ff98180d740 1 --2- 172.21.15.114:0/4007305803 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df1f64aed0 0x55df1f64f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.953+0000 7ff98180d740 1 --2- 172.21.15.114:0/4007305803 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df1f645e40 0x55df1f6565e0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.953+0000 7ff98180d740 1 --2- 172.21.15.114:0/4007305803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df1f645500 0x55df1f64ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.953+0000 7ff98180d740 1 -- 172.21.15.114:0/4007305803 >> 172.21.15.114:0/4007305803 conn(0x55df1f640b70 msgr2=0x55df1f646d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:56.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.953+0000 7ff98180d740 1 -- 172.21.15.114:0/4007305803 shutdown_connections 2024-09-13T18:21:56.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.953+0000 7ff98180d740 1 -- 172.21.15.114:0/4007305803 wait complete. 2024-09-13T18:21:56.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.953+0000 7ff98180d740 1 -- start start 2024-09-13T18:21:56.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.953+0000 7ff98180d740 1 Processor -- start 2024-09-13T18:21:56.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.954+0000 7ff98180d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df1f645500 0x55df1f651040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:56.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.954+0000 7ff98180d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df1f645e40 0x55df1f656560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:56.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.954+0000 7ff98180d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df1f64aed0 0x55df1f651670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:56.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.954+0000 7ff98180d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df1f6b13d0 con 0x55df1f64aed0 2024-09-13T18:21:56.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.954+0000 7ff98180d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df1f6b1510 con 0x55df1f645500 2024-09-13T18:21:56.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.954+0000 7ff98180d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df1f6b1650 con 0x55df1f645e40 2024-09-13T18:21:56.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.954+0000 7ff98180d740 10 client.0 _ll_register_callbacks cb 0x7ff9700147f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:21:56.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.954+0000 7ff97d086640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df1f64aed0 0x55df1f651670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:56.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.954+0000 7ff97d086640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df1f64aed0 0x55df1f651670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33926/0 (socket says 172.21.15.114:33926) 2024-09-13T18:21:56.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.954+0000 7ff97d086640 1 -- 172.21.15.114:0/2883163196 learned_addr learned my addr 172.21.15.114:0/2883163196 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:56.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.954+0000 7ff977fff640 1 --2- 172.21.15.114:0/2883163196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df1f645e40 0x55df1f656560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:56.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.954+0000 7ff97c885640 1 --2- 172.21.15.114:0/2883163196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df1f645500 0x55df1f651040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:56.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.954+0000 7ff97d086640 1 -- 172.21.15.114:0/2883163196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df1f645e40 msgr2=0x55df1f656560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:56.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.954+0000 7ff97d086640 1 --2- 172.21.15.114:0/2883163196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df1f645e40 0x55df1f656560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.954+0000 7ff97d086640 1 -- 172.21.15.114:0/2883163196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df1f645500 msgr2=0x55df1f651040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:56.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.954+0000 7ff97d086640 1 --2- 172.21.15.114:0/2883163196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df1f645500 0x55df1f651040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.954+0000 7ff97d086640 1 -- 172.21.15.114:0/2883163196 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9640314f0 con 0x55df1f64aed0 2024-09-13T18:21:56.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.954+0000 7ff97d086640 1 --2- 172.21.15.114:0/2883163196 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df1f64aed0 0x55df1f651670 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff970004990 tx=0x7ff970003e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b04f259d77233a11 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:56.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.954+0000 7ff957fff640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:21:56.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.954+0000 7ff957fff640 1 -- 172.21.15.114:0/2883163196 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97000fea0 con 0x55df1f64aed0 2024-09-13T18:21:56.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.954+0000 7ff957fff640 1 -- 172.21.15.114:0/2883163196 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7ff9700051c0 con 0x55df1f64aed0 2024-09-13T18:21:56.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.954+0000 7ff98180d740 1 -- 172.21.15.114:0/2883163196 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff97001a040 con 0x55df1f64aed0 2024-09-13T18:21:56.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.955+0000 7ff957fff640 1 -- 172.21.15.114:0/2883163196 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff970004a00 con 0x55df1f64aed0 2024-09-13T18:21:56.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.955+0000 7ff98180d740 10 client.15267 fetch_fsmap learned FSMap version 29 2024-09-13T18:21:56.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.955+0000 7ff98180d740 1 -- 172.21.15.114:0/2883163196 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55df1f56ed00 con 0x55df1f64aed0 2024-09-13T18:21:56.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.955+0000 7ff957fff640 1 -- 172.21.15.114:0/2883163196 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff97001a040 con 0x55df1f64aed0 2024-09-13T18:21:56.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.955+0000 7ff957fff640 1 -- 172.21.15.114:0/2883163196 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff970030040 con 0x55df1f64aed0 2024-09-13T18:21:56.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.955+0000 7ff98180d740 10 client.15267 fetch_fsmap finished waiting for FSMap version 29 2024-09-13T18:21:56.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.955+0000 7ff98180d740 10 client.15267 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-13T18:21:56.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.955+0000 7ff98180d740 20 client.15267 populate_metadata read hostname 'smithi114' 2024-09-13T18:21:56.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.955+0000 7ff98180d740 1 --2- 172.21.15.114:0/2883163196 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55df1f6b60d0 0x55df1f6b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:56.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.955+0000 7ff98180d740 4 client.15267 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T18:21:56.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.955+0000 7ff98180d740 1 -- 172.21.15.114:0/2883163196 --> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55df1f300d10 con 0x55df1f6b60d0 2024-09-13T18:21:56.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.955+0000 7ff97c885640 1 --2- 172.21.15.114:0/2883163196 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55df1f6b60d0 0x55df1f6b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:56.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.956+0000 7ff97c885640 1 --2- 172.21.15.114:0/2883163196 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55df1f6b60d0 0x55df1f6b29e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a096cca596528a41 server_cookie=964e40e3fdd5d3a0 in_seq=0 out_seq=0 2024-09-13T18:21:56.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.956+0000 7ff957fff640 10 client.15267 ms_handle_connect on v2:172.21.15.134:6834/1150003275 2024-09-13T18:21:56.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.956+0000 7ff957fff640 1 -- 172.21.15.114:0/2883163196 <== mds.3 v2:172.21.15.134:6834/1150003275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55df1f300d10 con 0x55df1f6b60d0 2024-09-13T18:21:56.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.956+0000 7ff957fff640 10 client.15267 handle_command_reply: tid=0 2024-09-13T18:21:56.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.956+0000 7ff98180d740 1 client.15267 shutdown 2024-09-13T18:21:56.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.956+0000 7ff98180d740 1 -- 172.21.15.114:0/2883163196 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df1f64aed0 msgr2=0x55df1f651670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:56.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.956+0000 7ff98180d740 1 --2- 172.21.15.114:0/2883163196 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df1f64aed0 0x55df1f651670 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff970004990 tx=0x7ff970003e30 comp rx=0 tx=0).stop 2024-09-13T18:21:56.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.956+0000 7ff98180d740 1 -- 172.21.15.114:0/2883163196 shutdown_connections 2024-09-13T18:21:56.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.956+0000 7ff98180d740 1 --2- 172.21.15.114:0/2883163196 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55df1f6b60d0 0x55df1f6b29e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.956+0000 7ff98180d740 1 --2- 172.21.15.114:0/2883163196 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df1f64aed0 0x55df1f651670 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.956+0000 7ff98180d740 1 --2- 172.21.15.114:0/2883163196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df1f645e40 0x55df1f656560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.956+0000 7ff957fff640 0 client.15267 ms_handle_reset on v2:172.21.15.134:6834/1150003275 2024-09-13T18:21:56.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.956+0000 7ff98180d740 1 --2- 172.21.15.114:0/2883163196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df1f645500 0x55df1f651040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.956+0000 7ff98180d740 1 -- 172.21.15.114:0/2883163196 >> 172.21.15.114:0/2883163196 conn(0x55df1f640b70 msgr2=0x55df1f64ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:56.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.956+0000 7ff98180d740 1 -- 172.21.15.114:0/2883163196 shutdown_connections 2024-09-13T18:21:56.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.956+0000 7ff98180d740 1 -- 172.21.15.114:0/2883163196 wait complete. 2024-09-13T18:21:56.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.956+0000 7ff98180d740 20 client.15267 trim_cache size 0 max 16384 2024-09-13T18:21:56.967 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:21:56.967 INFO:teuthology.orchestra.run.smithi114.stdout: "cluster_fsid": "97e7edf2-71fb-11ef-bceb-c7b262605968", 2024-09-13T18:21:56.967 INFO:teuthology.orchestra.run.smithi114.stdout: "whoami": 3, 2024-09-13T18:21:56.967 INFO:teuthology.orchestra.run.smithi114.stdout: "id": 24458, 2024-09-13T18:21:56.967 INFO:teuthology.orchestra.run.smithi114.stdout: "want_state": "up:active", 2024-09-13T18:21:56.967 INFO:teuthology.orchestra.run.smithi114.stdout: "state": "up:active", 2024-09-13T18:21:56.967 INFO:teuthology.orchestra.run.smithi114.stdout: "fs_name": "cephfs", 2024-09-13T18:21:56.968 INFO:teuthology.orchestra.run.smithi114.stdout: "rank_uptime": 88.761804100000006, 2024-09-13T18:21:56.968 INFO:teuthology.orchestra.run.smithi114.stdout: "mdsmap_epoch": 29, 2024-09-13T18:21:56.968 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch": 93, 2024-09-13T18:21:56.968 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch_barrier": 91, 2024-09-13T18:21:56.968 INFO:teuthology.orchestra.run.smithi114.stdout: "uptime": 139.37446531099999 2024-09-13T18:21:56.968 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:21:56.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.957+0000 7ff97eae9640 1 -- 172.21.15.114:0/1166381407 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7ff9680a1120 msgr2=0x7ff9680a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:56.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.957+0000 7ff97eae9640 1 --2- 172.21.15.114:0/1166381407 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7ff9680a1120 0x7ff9680a35e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff970014ca0 tx=0x7ff970009ea0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.957+0000 7ff97eae9640 1 -- 172.21.15.114:0/1166381407 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9780fec80 msgr2=0x7ff978069160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:56.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.957+0000 7ff97eae9640 1 --2- 172.21.15.114:0/1166381407 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9780fec80 0x7ff978069160 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff96c00e9c0 tx=0x7ff96c00ee80 comp rx=0 tx=0).stop 2024-09-13T18:21:56.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.957+0000 7ff97eae9640 1 -- 172.21.15.114:0/1166381407 shutdown_connections 2024-09-13T18:21:56.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.957+0000 7ff97eae9640 1 --2- 172.21.15.114:0/1166381407 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7ff9680a1120 0x7ff9680a35e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.957+0000 7ff97eae9640 1 --2- 172.21.15.114:0/1166381407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff978100830 0x7ff97819cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.957+0000 7ff97eae9640 1 --2- 172.21.15.114:0/1166381407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9780ffe70 0x7ff97819a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.957+0000 7ff97eae9640 1 --2- 172.21.15.114:0/1166381407 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9780fec80 0x7ff978069160 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:56.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.957+0000 7ff97eae9640 1 -- 172.21.15.114:0/1166381407 >> 172.21.15.114:0/1166381407 conn(0x7ff9780fa3e0 msgr2=0x7ff978103320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:56.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.957+0000 7ff97eae9640 1 -- 172.21.15.114:0/1166381407 shutdown_connections 2024-09-13T18:21:56.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:56.957+0000 7ff97eae9640 1 -- 172.21.15.114:0/1166381407 wait complete. 2024-09-13T18:21:56.971 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-09-13T18:21:57.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.071+0000 7fd6416f0640 1 -- 172.21.15.114:0/2839161588 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd63c0fec70 msgr2=0x7fd63c0ff070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:57.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.071+0000 7fd6416f0640 1 --2- 172.21.15.114:0/2839161588 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd63c0fec70 0x7fd63c0ff070 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd62c009900 tx=0x7fd62c02a410 comp rx=0 tx=0).stop 2024-09-13T18:21:57.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.071+0000 7fd6416f0640 1 -- 172.21.15.114:0/2839161588 shutdown_connections 2024-09-13T18:21:57.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.071+0000 7fd6416f0640 1 --2- 172.21.15.114:0/2839161588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd63c100820 0x7fd63c1070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.071+0000 7fd6416f0640 1 --2- 172.21.15.114:0/2839161588 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd63c0ffe60 0x7fd63c1002e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.071+0000 7fd6416f0640 1 --2- 172.21.15.114:0/2839161588 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd63c0fec70 0x7fd63c0ff070 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.071+0000 7fd6416f0640 1 -- 172.21.15.114:0/2839161588 >> 172.21.15.114:0/2839161588 conn(0x7fd63c0fa410 msgr2=0x7fd63c0fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:57.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.071+0000 7fd6416f0640 1 -- 172.21.15.114:0/2839161588 shutdown_connections 2024-09-13T18:21:57.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.071+0000 7fd6416f0640 1 -- 172.21.15.114:0/2839161588 wait complete. 2024-09-13T18:21:57.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.072+0000 7fd6416f0640 1 Processor -- start 2024-09-13T18:21:57.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.072+0000 7fd6416f0640 1 -- start start 2024-09-13T18:21:57.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.072+0000 7fd6416f0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd63c0fec70 0x7fd63c068110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:57.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.072+0000 7fd6416f0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd63c0ffe60 0x7fd63c19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:57.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.072+0000 7fd6416f0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd63c100820 0x7fd63c19d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:57.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.072+0000 7fd6416f0640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c19d840 con 0x7fd63c0ffe60 2024-09-13T18:21:57.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.072+0000 7fd6416f0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c19d9b0 con 0x7fd63c0fec70 2024-09-13T18:21:57.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.072+0000 7fd6416f0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c19db70 con 0x7fd63c100820 2024-09-13T18:21:57.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.072+0000 7fd633fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd63c0ffe60 0x7fd63c19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:57.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.072+0000 7fd633fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd63c0ffe60 0x7fd63c19aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33944/0 (socket says 172.21.15.114:33944) 2024-09-13T18:21:57.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.072+0000 7fd633fff640 1 -- 172.21.15.114:0/4098362943 learned_addr learned my addr 172.21.15.114:0/4098362943 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:57.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.072+0000 7fd63b7fe640 1 --2- 172.21.15.114:0/4098362943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd63c100820 0x7fd63c19d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:57.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.073+0000 7fd633fff640 1 -- 172.21.15.114:0/4098362943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd63c100820 msgr2=0x7fd63c19d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:57.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.073+0000 7fd633fff640 1 --2- 172.21.15.114:0/4098362943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd63c100820 0x7fd63c19d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.073+0000 7fd633fff640 1 -- 172.21.15.114:0/4098362943 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd63c0fec70 msgr2=0x7fd63c068110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:57.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.073+0000 7fd63affd640 1 --2- 172.21.15.114:0/4098362943 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd63c0fec70 0x7fd63c068110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:57.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.073+0000 7fd633fff640 1 --2- 172.21.15.114:0/4098362943 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd63c0fec70 0x7fd63c068110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.073+0000 7fd633fff640 1 -- 172.21.15.114:0/4098362943 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd62c009560 con 0x7fd63c0ffe60 2024-09-13T18:21:57.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.073+0000 7fd633fff640 1 --2- 172.21.15.114:0/4098362943 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd63c0ffe60 0x7fd63c19aa40 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd62400d770 tx=0x7fd62400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:57.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.073+0000 7fd638ff9640 1 -- 172.21.15.114:0/4098362943 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd624009c30 con 0x7fd63c0ffe60 2024-09-13T18:21:57.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.073+0000 7fd6416f0640 1 -- 172.21.15.114:0/4098362943 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd63c19ddf0 con 0x7fd63c0ffe60 2024-09-13T18:21:57.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.073+0000 7fd638ff9640 1 -- 172.21.15.114:0/4098362943 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd62400de80 con 0x7fd63c0ffe60 2024-09-13T18:21:57.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.073+0000 7fd638ff9640 1 -- 172.21.15.114:0/4098362943 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd624004d40 con 0x7fd63c0ffe60 2024-09-13T18:21:57.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.073+0000 7fd6416f0640 1 -- 172.21.15.114:0/4098362943 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd63c19e220 con 0x7fd63c0ffe60 2024-09-13T18:21:57.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.075+0000 7fd638ff9640 1 -- 172.21.15.114:0/4098362943 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fd624004ea0 con 0x7fd63c0ffe60 2024-09-13T18:21:57.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.075+0000 7fd644414740 1 Processor -- start 2024-09-13T18:21:57.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.075+0000 7fd644414740 1 -- start start 2024-09-13T18:21:57.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.076+0000 7fd644414740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb986b4500 0x55fb986b4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:57.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.076+0000 7fd644414740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb986b4e40 0x55fb986b9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:57.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.076+0000 7fd644414740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb986b9ed0 0x55fb986bc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:57.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.076+0000 7fd644414740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb9860f7c0 con 0x55fb986b4e40 2024-09-13T18:21:57.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.076+0000 7fd644414740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb9861eb70 con 0x55fb986b9ed0 2024-09-13T18:21:57.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.076+0000 7fd644414740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb9861c300 con 0x55fb986b4500 2024-09-13T18:21:57.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.076+0000 7fd63affd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb986b4e40 0x55fb986b9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:57.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.076+0000 7fd63affd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb986b4e40 0x55fb986b9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33958/0 (socket says 172.21.15.114:33958) 2024-09-13T18:21:57.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.076+0000 7fd63affd640 1 -- 172.21.15.114:0/1116530283 learned_addr learned my addr 172.21.15.114:0/1116530283 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:57.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.076+0000 7fd63b7fe640 1 --2- 172.21.15.114:0/1116530283 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb986b4500 0x55fb986b4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:57.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.076+0000 7fd633fff640 1 --2- 172.21.15.114:0/1116530283 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb986b9ed0 0x55fb986bc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:57.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.076+0000 7fd638ff9640 1 --2- 172.21.15.114:0/4098362943 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fd6280a1090 0x7fd6280a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:57.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.076+0000 7fd63affd640 1 -- 172.21.15.114:0/1116530283 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb986b4500 msgr2=0x55fb986b4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:57.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.076+0000 7fd63affd640 1 --2- 172.21.15.114:0/1116530283 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb986b4500 0x55fb986b4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.076+0000 7fd63affd640 1 -- 172.21.15.114:0/1116530283 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb986b9ed0 msgr2=0x55fb986bc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:57.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.076+0000 7fd63affd640 1 --2- 172.21.15.114:0/1116530283 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb986b9ed0 0x55fb986bc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.076+0000 7fd63affd640 1 -- 172.21.15.114:0/1116530283 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb985a44b0 con 0x55fb986b4e40 2024-09-13T18:21:57.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.076+0000 7fd63b7fe640 1 --2- 172.21.15.114:0/4098362943 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fd6280a1090 0x7fd6280a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:57.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.076+0000 7fd638ff9640 1 -- 172.21.15.114:0/4098362943 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fd6240cc7c0 con 0x7fd63c0ffe60 2024-09-13T18:21:57.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.076+0000 7fd63affd640 1 --2- 172.21.15.114:0/1116530283 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb986b4e40 0x55fb986b9900 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd62c005e90 tx=0x7fd62c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0c8f0ef6d63fcda server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:57.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.077+0000 7fd6317fa640 1 -- 172.21.15.114:0/1116530283 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62c0044f0 con 0x55fb986b4e40 2024-09-13T18:21:57.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.077+0000 7fd6317fa640 1 -- 172.21.15.114:0/1116530283 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd62c003a80 con 0x55fb986b4e40 2024-09-13T18:21:57.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.077+0000 7fd6317fa640 1 -- 172.21.15.114:0/1116530283 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62c03dac0 con 0x55fb986b4e40 2024-09-13T18:21:57.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.077+0000 7fd644414740 1 -- 172.21.15.114:0/1116530283 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb986b4e40 msgr2=0x55fb986b9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:57.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.077+0000 7fd63b7fe640 1 --2- 172.21.15.114:0/4098362943 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fd6280a1090 0x7fd6280a3550 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd634005fd0 tx=0x7fd634004a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:57.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.077+0000 7fd644414740 1 --2- 172.21.15.114:0/1116530283 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb986b4e40 0x55fb986b9900 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd62c005e90 tx=0x7fd62c004040 comp rx=0 tx=0).stop 2024-09-13T18:21:57.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.077+0000 7fd644414740 1 -- 172.21.15.114:0/1116530283 shutdown_connections 2024-09-13T18:21:57.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.077+0000 7fd644414740 1 --2- 172.21.15.114:0/1116530283 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb986b9ed0 0x55fb986bc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.077+0000 7fd644414740 1 --2- 172.21.15.114:0/1116530283 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb986b4e40 0x55fb986b9900 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.077+0000 7fd644414740 1 --2- 172.21.15.114:0/1116530283 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb986b4500 0x55fb986b4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.077+0000 7fd644414740 1 -- 172.21.15.114:0/1116530283 >> 172.21.15.114:0/1116530283 conn(0x55fb986afb70 msgr2=0x55fb986aff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:57.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.077+0000 7fd644414740 1 -- 172.21.15.114:0/1116530283 shutdown_connections 2024-09-13T18:21:57.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.078+0000 7fd644414740 1 -- 172.21.15.114:0/1116530283 wait complete. 2024-09-13T18:21:57.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.078+0000 7fd644414740 1 -- start start 2024-09-13T18:21:57.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.078+0000 7fd644414740 1 Processor -- start 2024-09-13T18:21:57.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.078+0000 7fd644414740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb986b4500 0x55fb986be0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:57.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.078+0000 7fd644414740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb986b4e40 0x55fb986be620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:57.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.078+0000 7fd644414740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb986b9ed0 0x55fb986c5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:57.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.078+0000 7fd644414740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb98617bf0 con 0x55fb986b4e40 2024-09-13T18:21:57.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.078+0000 7fd644414740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb985d6250 con 0x55fb986b4500 2024-09-13T18:21:57.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.078+0000 7fd644414740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb985d5ef0 con 0x55fb986b9ed0 2024-09-13T18:21:57.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.078+0000 7fd644414740 10 client.0 _ll_register_callbacks cb 0x55fb98608740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:21:57.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.078+0000 7fd633fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb986b4e40 0x55fb986be620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:57.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.078+0000 7fd633fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb986b4e40 0x55fb986be620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33960/0 (socket says 172.21.15.114:33960) 2024-09-13T18:21:57.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.078+0000 7fd633fff640 1 -- 172.21.15.114:0/1440260183 learned_addr learned my addr 172.21.15.114:0/1440260183 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:57.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.078+0000 7fd63b7fe640 1 --2- 172.21.15.114:0/1440260183 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb986b9ed0 0x55fb986c5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:57.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.078+0000 7fd633fff640 1 -- 172.21.15.114:0/1440260183 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb986b9ed0 msgr2=0x55fb986c5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:57.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.078+0000 7fd633fff640 1 --2- 172.21.15.114:0/1440260183 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb986b9ed0 0x55fb986c5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.078+0000 7fd633fff640 1 -- 172.21.15.114:0/1440260183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb986b4500 msgr2=0x55fb986be0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:57.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.078+0000 7fd633fff640 1 --2- 172.21.15.114:0/1440260183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb986b4500 0x55fb986be0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.078+0000 7fd633fff640 1 -- 172.21.15.114:0/1440260183 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd63c1079b0 con 0x55fb986b4e40 2024-09-13T18:21:57.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.079+0000 7fd633fff640 1 --2- 172.21.15.114:0/1440260183 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb986b4e40 0x55fb986be620 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd6240c9ef0 tx=0x7fd624004800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f09d14796ee5e091 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:57.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.079+0000 7fd6227fc640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:21:57.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.079+0000 7fd6227fc640 1 -- 172.21.15.114:0/1440260183 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd624095f50 con 0x55fb986b4e40 2024-09-13T18:21:57.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.079+0000 7fd6227fc640 1 -- 172.21.15.114:0/1440260183 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fd6240960b0 con 0x55fb986b4e40 2024-09-13T18:21:57.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.079+0000 7fd644414740 1 -- 172.21.15.114:0/1440260183 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fb9830b3c0 con 0x55fb986b4e40 2024-09-13T18:21:57.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.079+0000 7fd6227fc640 1 -- 172.21.15.114:0/1440260183 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd62409d1e0 con 0x55fb986b4e40 2024-09-13T18:21:57.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.079+0000 7fd644414740 10 client.15279 fetch_fsmap learned FSMap version 29 2024-09-13T18:21:57.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.079+0000 7fd644414740 1 -- 172.21.15.114:0/1440260183 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55fb98616270 con 0x55fb986b4e40 2024-09-13T18:21:57.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.079+0000 7fd6227fc640 1 -- 172.21.15.114:0/1440260183 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd6240b1080 con 0x55fb986b4e40 2024-09-13T18:21:57.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.079+0000 7fd6227fc640 1 -- 172.21.15.114:0/1440260183 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd624098d30 con 0x55fb986b4e40 2024-09-13T18:21:57.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.079+0000 7fd644414740 10 client.15279 fetch_fsmap finished waiting for FSMap version 29 2024-09-13T18:21:57.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.079+0000 7fd644414740 10 client.15279 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-13T18:21:57.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.079+0000 7fd644414740 20 client.15279 populate_metadata read hostname 'smithi114' 2024-09-13T18:21:57.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.079+0000 7fd644414740 1 --2- 172.21.15.114:0/1440260183 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55fb986ca6b0 0x55fb986ce890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:57.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.079+0000 7fd644414740 4 client.15279 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:21:57.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.079+0000 7fd644414740 1 -- 172.21.15.114:0/1440260183 --> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fb9831c6c0 con 0x55fb986ca6b0 2024-09-13T18:21:57.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.080+0000 7fd63affd640 1 --2- 172.21.15.114:0/1440260183 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55fb986ca6b0 0x55fb986ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:57.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.080+0000 7fd6227fc640 10 client.15279 ms_handle_connect on v2:172.21.15.204:6838/344883912 2024-09-13T18:21:57.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.080+0000 7fd63affd640 1 --2- 172.21.15.114:0/1440260183 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55fb986ca6b0 0x55fb986ce890 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=714c14d1aea6f584 server_cookie=c27fc9529cf8c158 in_seq=0 out_seq=0 2024-09-13T18:21:57.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.081+0000 7fd6227fc640 1 -- 172.21.15.114:0/1440260183 <== mds.2 v2:172.21.15.204:6838/344883912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55fb9831c6c0 con 0x55fb986ca6b0 2024-09-13T18:21:57.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.081+0000 7fd6227fc640 10 client.15279 handle_command_reply: tid=0 2024-09-13T18:21:57.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.081+0000 7fd644414740 1 client.15279 shutdown 2024-09-13T18:21:57.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.081+0000 7fd644414740 1 -- 172.21.15.114:0/1440260183 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb986b4e40 msgr2=0x55fb986be620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:57.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.081+0000 7fd644414740 1 --2- 172.21.15.114:0/1440260183 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb986b4e40 0x55fb986be620 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd6240c9ef0 tx=0x7fd624004800 comp rx=0 tx=0).stop 2024-09-13T18:21:57.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.081+0000 7fd644414740 1 -- 172.21.15.114:0/1440260183 shutdown_connections 2024-09-13T18:21:57.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.081+0000 7fd644414740 1 --2- 172.21.15.114:0/1440260183 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55fb986ca6b0 0x55fb986ce890 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.081+0000 7fd644414740 1 --2- 172.21.15.114:0/1440260183 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb986b9ed0 0x55fb986c5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.081+0000 7fd644414740 1 --2- 172.21.15.114:0/1440260183 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb986b4e40 0x55fb986be620 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.081+0000 7fd6227fc640 0 client.15279 ms_handle_reset on v2:172.21.15.204:6838/344883912 2024-09-13T18:21:57.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.081+0000 7fd644414740 1 --2- 172.21.15.114:0/1440260183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb986b4500 0x55fb986be0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.081+0000 7fd644414740 1 -- 172.21.15.114:0/1440260183 >> 172.21.15.114:0/1440260183 conn(0x55fb986afb70 msgr2=0x55fb986b1380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:57.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.081+0000 7fd644414740 1 -- 172.21.15.114:0/1440260183 shutdown_connections 2024-09-13T18:21:57.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.081+0000 7fd644414740 1 -- 172.21.15.114:0/1440260183 wait complete. 2024-09-13T18:21:57.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.081+0000 7fd644414740 20 client.15279 trim_cache size 0 max 16384 2024-09-13T18:21:57.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.095+0000 7fd644414740 1 Processor -- start 2024-09-13T18:21:57.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.095+0000 7fd644414740 1 -- start start 2024-09-13T18:21:57.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.096+0000 7fd644414740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb986afb70 0x55fb986be5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:57.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.096+0000 7fd644414740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb986b4500 0x55fb986d28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:57.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.096+0000 7fd644414740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb986b4e40 0x55fb986d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:57.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.096+0000 7fd644414740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb9859b460 con 0x55fb986b4e40 2024-09-13T18:21:57.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.096+0000 7fd644414740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb986c8720 con 0x55fb986afb70 2024-09-13T18:21:57.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.096+0000 7fd644414740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb985dad80 con 0x55fb986b4500 2024-09-13T18:21:57.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.096+0000 7fd63b7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb986b4e40 0x55fb986d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:57.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.096+0000 7fd63b7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb986b4e40 0x55fb986d2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33968/0 (socket says 172.21.15.114:33968) 2024-09-13T18:21:57.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.096+0000 7fd63b7fe640 1 -- 172.21.15.114:0/2333800590 learned_addr learned my addr 172.21.15.114:0/2333800590 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:57.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.096+0000 7fd63affd640 1 --2- 172.21.15.114:0/2333800590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb986afb70 0x55fb986be5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:57.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.096+0000 7fd633fff640 1 --2- 172.21.15.114:0/2333800590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb986b4500 0x55fb986d28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:57.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.096+0000 7fd63b7fe640 1 -- 172.21.15.114:0/2333800590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb986b4500 msgr2=0x55fb986d28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:57.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.096+0000 7fd63b7fe640 1 --2- 172.21.15.114:0/2333800590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb986b4500 0x55fb986d28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.096+0000 7fd63b7fe640 1 -- 172.21.15.114:0/2333800590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb986afb70 msgr2=0x55fb986be5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:57.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.096+0000 7fd63b7fe640 1 --2- 172.21.15.114:0/2333800590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb986afb70 0x55fb986be5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.096+0000 7fd63b7fe640 1 -- 172.21.15.114:0/2333800590 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb985b85f0 con 0x55fb986b4e40 2024-09-13T18:21:57.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.096+0000 7fd63b7fe640 1 --2- 172.21.15.114:0/2333800590 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb986b4e40 0x55fb986d2df0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd634009be0 tx=0x7fd634015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a62a5a7b348dd030 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:57.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.096+0000 7fd631ffb640 1 -- 172.21.15.114:0/2333800590 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6340116a0 con 0x55fb986b4e40 2024-09-13T18:21:57.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.096+0000 7fd631ffb640 1 -- 172.21.15.114:0/2333800590 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd634011c90 con 0x55fb986b4e40 2024-09-13T18:21:57.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.096+0000 7fd631ffb640 1 -- 172.21.15.114:0/2333800590 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63401ebe0 con 0x55fb986b4e40 2024-09-13T18:21:57.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.097+0000 7fd644414740 1 -- 172.21.15.114:0/2333800590 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb986b4e40 msgr2=0x55fb986d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:57.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.097+0000 7fd644414740 1 --2- 172.21.15.114:0/2333800590 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb986b4e40 0x55fb986d2df0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd634009be0 tx=0x7fd634015000 comp rx=0 tx=0).stop 2024-09-13T18:21:57.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.097+0000 7fd644414740 1 -- 172.21.15.114:0/2333800590 shutdown_connections 2024-09-13T18:21:57.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.097+0000 7fd644414740 1 --2- 172.21.15.114:0/2333800590 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb986b4e40 0x55fb986d2df0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.097+0000 7fd644414740 1 --2- 172.21.15.114:0/2333800590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb986b4500 0x55fb986d28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.097+0000 7fd644414740 1 --2- 172.21.15.114:0/2333800590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb986afb70 0x55fb986be5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.097+0000 7fd644414740 1 -- 172.21.15.114:0/2333800590 >> 172.21.15.114:0/2333800590 conn(0x55fb986bc630 msgr2=0x55fb986bde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:57.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.097+0000 7fd644414740 1 -- 172.21.15.114:0/2333800590 shutdown_connections 2024-09-13T18:21:57.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.097+0000 7fd644414740 1 -- 172.21.15.114:0/2333800590 wait complete. 2024-09-13T18:21:57.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.097+0000 7fd644414740 1 -- start start 2024-09-13T18:21:57.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.097+0000 7fd644414740 1 Processor -- start 2024-09-13T18:21:57.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.097+0000 7fd644414740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb986afb70 0x55fb9871d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:57.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.097+0000 7fd644414740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb986b4500 0x55fb9871def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:57.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.097+0000 7fd644414740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb986b4e40 0x55fb98724e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:57.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.097+0000 7fd644414740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb987253d0 con 0x55fb986b4e40 2024-09-13T18:21:57.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.097+0000 7fd644414740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb98725510 con 0x55fb986afb70 2024-09-13T18:21:57.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.098+0000 7fd644414740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb987256a0 con 0x55fb986b4500 2024-09-13T18:21:57.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.098+0000 7fd644414740 10 client.0 _ll_register_callbacks cb 0x7fd6340093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:21:57.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.098+0000 7fd63b7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb986b4e40 0x55fb98724e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:57.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.098+0000 7fd63b7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb986b4e40 0x55fb98724e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33976/0 (socket says 172.21.15.114:33976) 2024-09-13T18:21:57.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.098+0000 7fd63affd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb986afb70 0x55fb9871d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:57.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.098+0000 7fd63b7fe640 1 -- 172.21.15.114:0/3128177985 learned_addr learned my addr 172.21.15.114:0/3128177985 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:57.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.098+0000 7fd633fff640 1 --2- 172.21.15.114:0/3128177985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb986b4500 0x55fb9871def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:57.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.098+0000 7fd63b7fe640 1 -- 172.21.15.114:0/3128177985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb986b4500 msgr2=0x55fb9871def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:57.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.098+0000 7fd63b7fe640 1 --2- 172.21.15.114:0/3128177985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb986b4500 0x55fb9871def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.098+0000 7fd63b7fe640 1 -- 172.21.15.114:0/3128177985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb986afb70 msgr2=0x55fb9871d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:57.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.098+0000 7fd63b7fe640 1 --2- 172.21.15.114:0/3128177985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb986afb70 0x55fb9871d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.098+0000 7fd63b7fe640 1 -- 172.21.15.114:0/3128177985 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd634003ad0 con 0x55fb986b4e40 2024-09-13T18:21:57.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.098+0000 7fd63b7fe640 1 --2- 172.21.15.114:0/3128177985 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb986b4e40 0x55fb98724e90 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd63401e4c0 tx=0x7fd63401e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49830301ca69f4ef server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:57.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.098+0000 7fd6227fc640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:21:57.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.098+0000 7fd6227fc640 1 -- 172.21.15.114:0/3128177985 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd634003970 con 0x55fb986b4e40 2024-09-13T18:21:57.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.098+0000 7fd6227fc640 1 -- 172.21.15.114:0/3128177985 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fd6340118b0 con 0x55fb986b4e40 2024-09-13T18:21:57.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.098+0000 7fd644414740 1 -- 172.21.15.114:0/3128177985 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd62c003a80 con 0x55fb986b4e40 2024-09-13T18:21:57.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.098+0000 7fd6227fc640 1 -- 172.21.15.114:0/3128177985 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd634011a70 con 0x55fb986b4e40 2024-09-13T18:21:57.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.098+0000 7fd644414740 10 client.15285 fetch_fsmap learned FSMap version 29 2024-09-13T18:21:57.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.098+0000 7fd644414740 1 -- 172.21.15.114:0/3128177985 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55fb985ddd00 con 0x55fb986b4e40 2024-09-13T18:21:57.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.099+0000 7fd6227fc640 1 -- 172.21.15.114:0/3128177985 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd62c003a80 con 0x55fb986b4e40 2024-09-13T18:21:57.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.099+0000 7fd6227fc640 1 -- 172.21.15.114:0/3128177985 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd63401a040 con 0x55fb986b4e40 2024-09-13T18:21:57.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.099+0000 7fd644414740 10 client.15285 fetch_fsmap finished waiting for FSMap version 29 2024-09-13T18:21:57.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.099+0000 7fd644414740 10 client.15285 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-13T18:21:57.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.099+0000 7fd644414740 20 client.15285 populate_metadata read hostname 'smithi114' 2024-09-13T18:21:57.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.099+0000 7fd644414740 1 --2- 172.21.15.114:0/3128177985 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55fb986b9ed0 0x55fb98726d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:57.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.099+0000 7fd644414740 4 client.15285 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T18:21:57.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.099+0000 7fd644414740 1 -- 172.21.15.114:0/3128177985 --> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fb9836fd10 con 0x55fb986b9ed0 2024-09-13T18:21:57.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.099+0000 7fd63affd640 1 --2- 172.21.15.114:0/3128177985 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55fb986b9ed0 0x55fb98726d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:57.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.099+0000 7fd63affd640 1 --2- 172.21.15.114:0/3128177985 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55fb986b9ed0 0x55fb98726d00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fcc5f744dec59e5d server_cookie=ca1be57f3283b383 in_seq=0 out_seq=0 2024-09-13T18:21:57.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.099+0000 7fd6227fc640 10 client.15285 ms_handle_connect on v2:172.21.15.204:6838/344883912 2024-09-13T18:21:57.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.099+0000 7fd6227fc640 1 -- 172.21.15.114:0/3128177985 <== mds.2 v2:172.21.15.204:6838/344883912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55fb9836fd10 con 0x55fb986b9ed0 2024-09-13T18:21:57.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.099+0000 7fd6227fc640 10 client.15285 handle_command_reply: tid=0 2024-09-13T18:21:57.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.100+0000 7fd644414740 1 client.15285 shutdown 2024-09-13T18:21:57.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.100+0000 7fd644414740 1 -- 172.21.15.114:0/3128177985 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb986b4e40 msgr2=0x55fb98724e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:57.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.100+0000 7fd644414740 1 --2- 172.21.15.114:0/3128177985 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb986b4e40 0x55fb98724e90 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd63401e4c0 tx=0x7fd63401e530 comp rx=0 tx=0).stop 2024-09-13T18:21:57.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.100+0000 7fd644414740 1 -- 172.21.15.114:0/3128177985 shutdown_connections 2024-09-13T18:21:57.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.100+0000 7fd644414740 1 --2- 172.21.15.114:0/3128177985 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55fb986b9ed0 0x55fb98726d00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.100+0000 7fd644414740 1 --2- 172.21.15.114:0/3128177985 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb986b4e40 0x55fb98724e90 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.100+0000 7fd6227fc640 0 client.15285 ms_handle_reset on v2:172.21.15.204:6838/344883912 2024-09-13T18:21:57.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.100+0000 7fd644414740 1 --2- 172.21.15.114:0/3128177985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb986b4500 0x55fb9871def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.100+0000 7fd644414740 1 --2- 172.21.15.114:0/3128177985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb986afb70 0x55fb9871d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.100+0000 7fd644414740 1 -- 172.21.15.114:0/3128177985 >> 172.21.15.114:0/3128177985 conn(0x55fb986bc630 msgr2=0x55fb986b96d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:57.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.100+0000 7fd644414740 1 -- 172.21.15.114:0/3128177985 shutdown_connections 2024-09-13T18:21:57.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.100+0000 7fd644414740 1 -- 172.21.15.114:0/3128177985 wait complete. 2024-09-13T18:21:57.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.100+0000 7fd644414740 20 client.15285 trim_cache size 0 max 16384 2024-09-13T18:21:57.111 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:21:57.111 INFO:teuthology.orchestra.run.smithi114.stdout: "cluster_fsid": "97e7edf2-71fb-11ef-bceb-c7b262605968", 2024-09-13T18:21:57.111 INFO:teuthology.orchestra.run.smithi114.stdout: "whoami": 2, 2024-09-13T18:21:57.111 INFO:teuthology.orchestra.run.smithi114.stdout: "id": 24475, 2024-09-13T18:21:57.112 INFO:teuthology.orchestra.run.smithi114.stdout: "want_state": "up:active", 2024-09-13T18:21:57.112 INFO:teuthology.orchestra.run.smithi114.stdout: "state": "up:active", 2024-09-13T18:21:57.112 INFO:teuthology.orchestra.run.smithi114.stdout: "fs_name": "cephfs", 2024-09-13T18:21:57.112 INFO:teuthology.orchestra.run.smithi114.stdout: "rank_uptime": 90.913923605999997, 2024-09-13T18:21:57.112 INFO:teuthology.orchestra.run.smithi114.stdout: "mdsmap_epoch": 29, 2024-09-13T18:21:57.112 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch": 93, 2024-09-13T18:21:57.112 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch_barrier": 91, 2024-09-13T18:21:57.112 INFO:teuthology.orchestra.run.smithi114.stdout: "uptime": 128.90430859599999 2024-09-13T18:21:57.112 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:21:57.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.100+0000 7fd6416f0640 1 -- 172.21.15.114:0/4098362943 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fd6280a1090 msgr2=0x7fd6280a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:57.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.100+0000 7fd6416f0640 1 --2- 172.21.15.114:0/4098362943 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fd6280a1090 0x7fd6280a3550 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd634005fd0 tx=0x7fd634004a40 comp rx=0 tx=0).stop 2024-09-13T18:21:57.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.100+0000 7fd6416f0640 1 -- 172.21.15.114:0/4098362943 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd63c0ffe60 msgr2=0x7fd63c19aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:57.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.100+0000 7fd6416f0640 1 --2- 172.21.15.114:0/4098362943 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd63c0ffe60 0x7fd63c19aa40 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd62400d770 tx=0x7fd62400dc30 comp rx=0 tx=0).stop 2024-09-13T18:21:57.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.100+0000 7fd6416f0640 1 -- 172.21.15.114:0/4098362943 shutdown_connections 2024-09-13T18:21:57.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.100+0000 7fd6416f0640 1 --2- 172.21.15.114:0/4098362943 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fd6280a1090 0x7fd6280a3550 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.100+0000 7fd6416f0640 1 --2- 172.21.15.114:0/4098362943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd63c100820 0x7fd63c19d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.100+0000 7fd6416f0640 1 --2- 172.21.15.114:0/4098362943 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd63c0ffe60 0x7fd63c19aa40 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.100+0000 7fd6416f0640 1 --2- 172.21.15.114:0/4098362943 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd63c0fec70 0x7fd63c068110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.100+0000 7fd6416f0640 1 -- 172.21.15.114:0/4098362943 >> 172.21.15.114:0/4098362943 conn(0x7fd63c0fa410 msgr2=0x7fd63c103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:57.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.100+0000 7fd6416f0640 1 -- 172.21.15.114:0/4098362943 shutdown_connections 2024-09-13T18:21:57.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.100+0000 7fd6416f0640 1 -- 172.21.15.114:0/4098362943 wait complete. 2024-09-13T18:21:57.115 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-09-13T18:21:57.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.212+0000 7f403cbc3640 1 -- 172.21.15.114:0/2601418941 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4038101920 msgr2=0x7f40381081b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:57.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.212+0000 7f403cbc3640 1 --2- 172.21.15.114:0/2601418941 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4038101920 0x7f40381081b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f4030009ca0 tx=0x7f403002a8c0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.212+0000 7f403cbc3640 1 -- 172.21.15.114:0/2601418941 shutdown_connections 2024-09-13T18:21:57.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.212+0000 7f403cbc3640 1 --2- 172.21.15.114:0/2601418941 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4038101920 0x7f40381081b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.212+0000 7f403cbc3640 1 --2- 172.21.15.114:0/2601418941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4038100f60 0x7f40381013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.212+0000 7f403cbc3640 1 --2- 172.21.15.114:0/2601418941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40380ffd70 0x7f4038100170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.212+0000 7f403cbc3640 1 -- 172.21.15.114:0/2601418941 >> 172.21.15.114:0/2601418941 conn(0x7f40380fb4d0 msgr2=0x7f40380fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:57.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.212+0000 7f403cbc3640 1 -- 172.21.15.114:0/2601418941 shutdown_connections 2024-09-13T18:21:57.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.212+0000 7f403cbc3640 1 -- 172.21.15.114:0/2601418941 wait complete. 2024-09-13T18:21:57.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.213+0000 7f403cbc3640 1 Processor -- start 2024-09-13T18:21:57.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.213+0000 7f403cbc3640 1 -- start start 2024-09-13T18:21:57.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.213+0000 7f403cbc3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40380ffd70 0x7f4038195bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:57.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.213+0000 7f403cbc3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4038100f60 0x7f4038196130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:57.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.213+0000 7f403cbc3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4038101920 0x7f403819d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:57.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.213+0000 7f403cbc3640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f403819d6e0 con 0x7f4038100f60 2024-09-13T18:21:57.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.213+0000 7f403cbc3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f403819d850 con 0x7f4038101920 2024-09-13T18:21:57.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.213+0000 7f403cbc3640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f403819da10 con 0x7f40380ffd70 2024-09-13T18:21:57.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.213+0000 7f4035d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4038100f60 0x7f4038196130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:57.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.213+0000 7f4035d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4038100f60 0x7f4038196130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33996/0 (socket says 172.21.15.114:33996) 2024-09-13T18:21:57.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.213+0000 7f4035d9b640 1 -- 172.21.15.114:0/3794407309 learned_addr learned my addr 172.21.15.114:0/3794407309 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:57.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.213+0000 7f4036d9d640 1 --2- 172.21.15.114:0/3794407309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4038101920 0x7f403819d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:57.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.213+0000 7f403659c640 1 --2- 172.21.15.114:0/3794407309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40380ffd70 0x7f4038195bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:57.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.213+0000 7f4035d9b640 1 -- 172.21.15.114:0/3794407309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40380ffd70 msgr2=0x7f4038195bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:57.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.213+0000 7f4035d9b640 1 --2- 172.21.15.114:0/3794407309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40380ffd70 0x7f4038195bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.213+0000 7f4035d9b640 1 -- 172.21.15.114:0/3794407309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4038101920 msgr2=0x7f403819d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:57.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.213+0000 7f4035d9b640 1 --2- 172.21.15.114:0/3794407309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4038101920 0x7f403819d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.213+0000 7f4035d9b640 1 -- 172.21.15.114:0/3794407309 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4030009900 con 0x7f4038100f60 2024-09-13T18:21:57.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.213+0000 7f403659c640 1 --2- 172.21.15.114:0/3794407309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40380ffd70 0x7f4038195bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:21:57.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.213+0000 7f4035d9b640 1 --2- 172.21.15.114:0/3794407309 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4038100f60 0x7f4038196130 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f402c00e9f0 tx=0x7f402c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:57.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.214+0000 7f40277fe640 1 -- 172.21.15.114:0/3794407309 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402c009720 con 0x7f4038100f60 2024-09-13T18:21:57.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.214+0000 7f40277fe640 1 -- 172.21.15.114:0/3794407309 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f402c00cd80 con 0x7f4038100f60 2024-09-13T18:21:57.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.214+0000 7f40277fe640 1 -- 172.21.15.114:0/3794407309 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402c010430 con 0x7f4038100f60 2024-09-13T18:21:57.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.214+0000 7f403cbc3640 1 -- 172.21.15.114:0/3794407309 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f403819dcf0 con 0x7f4038100f60 2024-09-13T18:21:57.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.214+0000 7f403cbc3640 1 -- 172.21.15.114:0/3794407309 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f403819e1b0 con 0x7f4038100f60 2024-09-13T18:21:57.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.216+0000 7f40277fe640 1 -- 172.21.15.114:0/3794407309 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f402c004fb0 con 0x7f4038100f60 2024-09-13T18:21:57.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.216+0000 7f403f9bf740 1 Processor -- start 2024-09-13T18:21:57.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.216+0000 7f403f9bf740 1 -- start start 2024-09-13T18:21:57.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.216+0000 7f403f9bf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cfe8ba7500 0x55cfe8ba7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:57.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.216+0000 7f403f9bf740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cfe8ba7e40 0x55cfe8bac900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:57.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.216+0000 7f40277fe640 1 --2- 172.21.15.114:0/3794407309 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f40200a1110 0x7f40200a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:57.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.216+0000 7f403f9bf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cfe8baced0 0x55cfe8baf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:57.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.216+0000 7f403f9bf740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfe8b027c0 con 0x55cfe8ba7e40 2024-09-13T18:21:57.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.216+0000 7f403f9bf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfe8b11b70 con 0x55cfe8ba7500 2024-09-13T18:21:57.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.216+0000 7f403f9bf740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfe8b0f300 con 0x55cfe8baced0 2024-09-13T18:21:57.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.217+0000 7f403659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cfe8ba7e40 0x55cfe8bac900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:57.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.217+0000 7f4035d9b640 1 --2- 172.21.15.114:0/3794407309 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f40200a1110 0x7f40200a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:57.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.217+0000 7f403659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cfe8ba7e40 0x55cfe8bac900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34002/0 (socket says 172.21.15.114:34002) 2024-09-13T18:21:57.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.217+0000 7f403659c640 1 -- 172.21.15.114:0/1216613864 learned_addr learned my addr 172.21.15.114:0/1216613864 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:57.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.217+0000 7f4036d9d640 1 --2- 172.21.15.114:0/1216613864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cfe8ba7500 0x55cfe8ba7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:57.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.217+0000 7f40277fe640 1 -- 172.21.15.114:0/3794407309 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f402c014070 con 0x7f4038100f60 2024-09-13T18:21:57.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.217+0000 7f4036d9d640 1 --2- 172.21.15.114:0/1216613864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cfe8baced0 0x55cfe8baf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:57.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.217+0000 7f403659c640 1 -- 172.21.15.114:0/1216613864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cfe8baced0 msgr2=0x55cfe8baf390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:57.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.217+0000 7f403659c640 1 --2- 172.21.15.114:0/1216613864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cfe8baced0 0x55cfe8baf390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.217+0000 7f403659c640 1 -- 172.21.15.114:0/1216613864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cfe8ba7500 msgr2=0x55cfe8ba7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:57.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.217+0000 7f403659c640 1 --2- 172.21.15.114:0/1216613864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cfe8ba7500 0x55cfe8ba7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.217+0000 7f403659c640 1 -- 172.21.15.114:0/1216613864 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cfe8a974b0 con 0x55cfe8ba7e40 2024-09-13T18:21:57.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.217+0000 7f4035d9b640 1 --2- 172.21.15.114:0/3794407309 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f40200a1110 0x7f40200a35d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f402c0965e0 tx=0x7f402c08f880 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:57.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.217+0000 7f403659c640 1 --2- 172.21.15.114:0/1216613864 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cfe8ba7e40 0x55cfe8bac900 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4028009c30 tx=0x7f402800e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74ea5cd839436041 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:57.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.217+0000 7f4024ff9640 1 -- 172.21.15.114:0/1216613864 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402800eed0 con 0x55cfe8ba7e40 2024-09-13T18:21:57.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.217+0000 7f4024ff9640 1 -- 172.21.15.114:0/1216613864 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4028009e40 con 0x55cfe8ba7e40 2024-09-13T18:21:57.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.217+0000 7f4024ff9640 1 -- 172.21.15.114:0/1216613864 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4028011470 con 0x55cfe8ba7e40 2024-09-13T18:21:57.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.218+0000 7f403f9bf740 1 -- 172.21.15.114:0/1216613864 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cfe8ba7e40 msgr2=0x55cfe8bac900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:57.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.218+0000 7f403f9bf740 1 --2- 172.21.15.114:0/1216613864 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cfe8ba7e40 0x55cfe8bac900 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4028009c30 tx=0x7f402800e6c0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.218+0000 7f403f9bf740 1 -- 172.21.15.114:0/1216613864 shutdown_connections 2024-09-13T18:21:57.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.218+0000 7f403f9bf740 1 --2- 172.21.15.114:0/1216613864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cfe8baced0 0x55cfe8baf390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.218+0000 7f403f9bf740 1 --2- 172.21.15.114:0/1216613864 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cfe8ba7e40 0x55cfe8bac900 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.218+0000 7f403f9bf740 1 --2- 172.21.15.114:0/1216613864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cfe8ba7500 0x55cfe8ba7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.218+0000 7f403f9bf740 1 -- 172.21.15.114:0/1216613864 >> 172.21.15.114:0/1216613864 conn(0x55cfe8ba2b70 msgr2=0x55cfe8ba2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:57.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.218+0000 7f403f9bf740 1 -- 172.21.15.114:0/1216613864 shutdown_connections 2024-09-13T18:21:57.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.218+0000 7f403f9bf740 1 -- 172.21.15.114:0/1216613864 wait complete. 2024-09-13T18:21:57.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.218+0000 7f403f9bf740 1 -- start start 2024-09-13T18:21:57.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.219+0000 7f403f9bf740 1 Processor -- start 2024-09-13T18:21:57.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.219+0000 7f403f9bf740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cfe8ba7500 0x55cfe8bb1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:57.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.219+0000 7f403f9bf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cfe8ba7e40 0x55cfe8bb1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:57.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.219+0000 7f403f9bf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cfe8baced0 0x55cfe8bb8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:57.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.219+0000 7f403f9bf740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfe8b0abf0 con 0x55cfe8ba7500 2024-09-13T18:21:57.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.219+0000 7f403f9bf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfe8ac9250 con 0x55cfe8ba7e40 2024-09-13T18:21:57.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.219+0000 7f403f9bf740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfe8ac8ef0 con 0x55cfe8baced0 2024-09-13T18:21:57.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.219+0000 7f403f9bf740 10 client.0 _ll_register_callbacks cb 0x55cfe8afb740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:21:57.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.219+0000 7f4036d9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cfe8ba7500 0x55cfe8bb1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:57.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.219+0000 7f4036d9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cfe8ba7500 0x55cfe8bb1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34004/0 (socket says 172.21.15.114:34004) 2024-09-13T18:21:57.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.219+0000 7f4036d9d640 1 -- 172.21.15.114:0/2969982337 learned_addr learned my addr 172.21.15.114:0/2969982337 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:57.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.219+0000 7f403659c640 1 --2- 172.21.15.114:0/2969982337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cfe8ba7e40 0x55cfe8bb1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:57.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.219+0000 7f4036d9d640 1 -- 172.21.15.114:0/2969982337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cfe8baced0 msgr2=0x55cfe8bb8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:57.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.219+0000 7f4036d9d640 1 --2- 172.21.15.114:0/2969982337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cfe8baced0 0x55cfe8bb8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.219+0000 7f4036d9d640 1 -- 172.21.15.114:0/2969982337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cfe8ba7e40 msgr2=0x55cfe8bb1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:57.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.219+0000 7f4036d9d640 1 --2- 172.21.15.114:0/2969982337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cfe8ba7e40 0x55cfe8bb1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.219+0000 7f4036d9d640 1 -- 172.21.15.114:0/2969982337 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4028009860 con 0x55cfe8ba7500 2024-09-13T18:21:57.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.219+0000 7f4036d9d640 1 --2- 172.21.15.114:0/2969982337 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cfe8ba7500 0x55cfe8bb1100 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4030004ab0 tx=0x7f4030031900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=937b0ae098246937 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:57.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.219+0000 7f401dffb640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:21:57.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.219+0000 7f401dffb640 1 -- 172.21.15.114:0/2969982337 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4030043070 con 0x55cfe8ba7500 2024-09-13T18:21:57.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.219+0000 7f401dffb640 1 -- 172.21.15.114:0/2969982337 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f4030003b80 con 0x55cfe8ba7500 2024-09-13T18:21:57.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.219+0000 7f403f9bf740 1 -- 172.21.15.114:0/2969982337 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cfe87fe3c0 con 0x55cfe8ba7500 2024-09-13T18:21:57.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.220+0000 7f401dffb640 1 -- 172.21.15.114:0/2969982337 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4030003d40 con 0x55cfe8ba7500 2024-09-13T18:21:57.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.220+0000 7f403f9bf740 10 client.15297 fetch_fsmap learned FSMap version 29 2024-09-13T18:21:57.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.220+0000 7f403f9bf740 1 -- 172.21.15.114:0/2969982337 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55cfe8b09270 con 0x55cfe8ba7500 2024-09-13T18:21:57.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.220+0000 7f401dffb640 1 -- 172.21.15.114:0/2969982337 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55cfe87fe3c0 con 0x55cfe8ba7500 2024-09-13T18:21:57.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.220+0000 7f401dffb640 1 -- 172.21.15.114:0/2969982337 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4030053070 con 0x55cfe8ba7500 2024-09-13T18:21:57.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.220+0000 7f403f9bf740 10 client.15297 fetch_fsmap finished waiting for FSMap version 29 2024-09-13T18:21:57.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.220+0000 7f403f9bf740 10 client.15297 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-13T18:21:57.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.220+0000 7f403f9bf740 20 client.15297 populate_metadata read hostname 'smithi114' 2024-09-13T18:21:57.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.220+0000 7f403f9bf740 1 --2- 172.21.15.114:0/2969982337 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55cfe8bbd520 0x55cfe8bc16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:57.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.220+0000 7f403f9bf740 4 client.15297 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:21:57.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.220+0000 7f403f9bf740 1 -- 172.21.15.114:0/2969982337 --> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cfe880f6c0 con 0x55cfe8bbd520 2024-09-13T18:21:57.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.220+0000 7f403659c640 1 --2- 172.21.15.114:0/2969982337 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55cfe8bbd520 0x55cfe8bc16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:57.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.221+0000 7f401dffb640 10 client.15297 ms_handle_connect on v2:172.21.15.134:6836/891692210 2024-09-13T18:21:57.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.221+0000 7f403659c640 1 --2- 172.21.15.114:0/2969982337 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55cfe8bbd520 0x55cfe8bc16c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=29f06438a2c69026 server_cookie=1657738db569699e in_seq=0 out_seq=0 2024-09-13T18:21:57.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.221+0000 7f401dffb640 1 -- 172.21.15.114:0/2969982337 <== mds.1 v2:172.21.15.134:6836/891692210 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55cfe880f6c0 con 0x55cfe8bbd520 2024-09-13T18:21:57.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.221+0000 7f401dffb640 10 client.15297 handle_command_reply: tid=0 2024-09-13T18:21:57.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.221+0000 7f403f9bf740 1 client.15297 shutdown 2024-09-13T18:21:57.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.221+0000 7f403f9bf740 1 -- 172.21.15.114:0/2969982337 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cfe8ba7500 msgr2=0x55cfe8bb1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:57.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.221+0000 7f403f9bf740 1 --2- 172.21.15.114:0/2969982337 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cfe8ba7500 0x55cfe8bb1100 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4030004ab0 tx=0x7f4030031900 comp rx=0 tx=0).stop 2024-09-13T18:21:57.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.221+0000 7f403f9bf740 1 -- 172.21.15.114:0/2969982337 shutdown_connections 2024-09-13T18:21:57.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.221+0000 7f403f9bf740 1 --2- 172.21.15.114:0/2969982337 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55cfe8bbd520 0x55cfe8bc16c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.221+0000 7f403f9bf740 1 --2- 172.21.15.114:0/2969982337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cfe8baced0 0x55cfe8bb8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.221+0000 7f403f9bf740 1 --2- 172.21.15.114:0/2969982337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cfe8ba7e40 0x55cfe8bb1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.221+0000 7f401dffb640 0 client.15297 ms_handle_reset on v2:172.21.15.134:6836/891692210 2024-09-13T18:21:57.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.221+0000 7f403f9bf740 1 --2- 172.21.15.114:0/2969982337 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cfe8ba7500 0x55cfe8bb1100 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.221+0000 7f403f9bf740 1 -- 172.21.15.114:0/2969982337 >> 172.21.15.114:0/2969982337 conn(0x55cfe8ba2b70 msgr2=0x55cfe8ba4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:57.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.221+0000 7f403f9bf740 1 -- 172.21.15.114:0/2969982337 shutdown_connections 2024-09-13T18:21:57.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.221+0000 7f403f9bf740 1 -- 172.21.15.114:0/2969982337 wait complete. 2024-09-13T18:21:57.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.222+0000 7f403f9bf740 20 client.15297 trim_cache size 0 max 16384 2024-09-13T18:21:57.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.236+0000 7f403f9bf740 1 Processor -- start 2024-09-13T18:21:57.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.236+0000 7f403f9bf740 1 -- start start 2024-09-13T18:21:57.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.236+0000 7f403f9bf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cfe8ba7500 0x55cfe8bac2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:57.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.236+0000 7f403f9bf740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cfe8ba7e40 0x55cfe8bac7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:57.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.236+0000 7f403f9bf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cfe8baced0 0x55cfe8bb0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:57.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.236+0000 7f403f9bf740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfe8a8e460 con 0x55cfe8ba7e40 2024-09-13T18:21:57.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.236+0000 7f403f9bf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfe8bb9320 con 0x55cfe8ba7500 2024-09-13T18:21:57.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.236+0000 7f403f9bf740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfe880f6c0 con 0x55cfe8baced0 2024-09-13T18:21:57.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.236+0000 7f403659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cfe8ba7e40 0x55cfe8bac7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:57.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.236+0000 7f4036d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cfe8ba7500 0x55cfe8bac2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:57.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.236+0000 7f4036d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cfe8ba7500 0x55cfe8bac2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:35354/0 (socket says 172.21.15.114:35354) 2024-09-13T18:21:57.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.236+0000 7f403659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cfe8ba7e40 0x55cfe8bac7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34008/0 (socket says 172.21.15.114:34008) 2024-09-13T18:21:57.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.236+0000 7f4036d9d640 1 -- 172.21.15.114:0/3226161153 learned_addr learned my addr 172.21.15.114:0/3226161153 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:57.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.236+0000 7f4036d9d640 1 --2- 172.21.15.114:0/3226161153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cfe8baced0 0x55cfe8bb0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:57.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.236+0000 7f4036d9d640 1 -- 172.21.15.114:0/3226161153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cfe8baced0 msgr2=0x55cfe8bb0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:57.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.236+0000 7f4036d9d640 1 --2- 172.21.15.114:0/3226161153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cfe8baced0 0x55cfe8bb0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.236+0000 7f4036d9d640 1 -- 172.21.15.114:0/3226161153 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cfe8ba7e40 msgr2=0x55cfe8bac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:57.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.236+0000 7f4036d9d640 1 --2- 172.21.15.114:0/3226161153 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cfe8ba7e40 0x55cfe8bac7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.236+0000 7f4036d9d640 1 -- 172.21.15.114:0/3226161153 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cfe8aab5f0 con 0x55cfe8ba7500 2024-09-13T18:21:57.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.236+0000 7f403659c640 1 --2- 172.21.15.114:0/3226161153 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cfe8ba7e40 0x55cfe8bac7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:21:57.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.237+0000 7f4036d9d640 1 --2- 172.21.15.114:0/3226161153 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cfe8ba7500 0x55cfe8bac2a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f403002ade0 tx=0x7f4030031ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82ce930484f41e85 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:57.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.237+0000 7f40257fa640 1 -- 172.21.15.114:0/3226161153 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4030043030 con 0x55cfe8ba7500 2024-09-13T18:21:57.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.237+0000 7f40257fa640 1 -- 172.21.15.114:0/3226161153 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4030041040 con 0x55cfe8ba7500 2024-09-13T18:21:57.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.237+0000 7f40257fa640 1 -- 172.21.15.114:0/3226161153 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403003a890 con 0x55cfe8ba7500 2024-09-13T18:21:57.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.237+0000 7f403f9bf740 1 -- 172.21.15.114:0/3226161153 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cfe8ba7500 msgr2=0x55cfe8bac2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:57.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.237+0000 7f403f9bf740 1 --2- 172.21.15.114:0/3226161153 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cfe8ba7500 0x55cfe8bac2a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f403002ade0 tx=0x7f4030031ab0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.237+0000 7f403f9bf740 1 -- 172.21.15.114:0/3226161153 shutdown_connections 2024-09-13T18:21:57.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.237+0000 7f403f9bf740 1 --2- 172.21.15.114:0/3226161153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cfe8baced0 0x55cfe8bb0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.237+0000 7f403f9bf740 1 --2- 172.21.15.114:0/3226161153 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cfe8ba7e40 0x55cfe8bac7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.237+0000 7f403f9bf740 1 --2- 172.21.15.114:0/3226161153 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cfe8ba7500 0x55cfe8bac2a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.237+0000 7f403f9bf740 1 -- 172.21.15.114:0/3226161153 >> 172.21.15.114:0/3226161153 conn(0x55cfe8ba2b70 msgr2=0x55cfe8baf250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:57.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.237+0000 7f403f9bf740 1 -- 172.21.15.114:0/3226161153 shutdown_connections 2024-09-13T18:21:57.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.237+0000 7f403f9bf740 1 -- 172.21.15.114:0/3226161153 wait complete. 2024-09-13T18:21:57.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.238+0000 7f403f9bf740 1 -- start start 2024-09-13T18:21:57.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.238+0000 7f403f9bf740 1 Processor -- start 2024-09-13T18:21:57.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.238+0000 7f403f9bf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cfe8ba7500 0x55cfe8baf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:57.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.238+0000 7f403f9bf740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cfe8ba7e40 0x55cfe8bada70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:57.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.238+0000 7f403f9bf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cfe8baced0 0x55cfe8bae010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:57.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.238+0000 7f403f9bf740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfe8bae710 con 0x55cfe8ba7e40 2024-09-13T18:21:57.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.238+0000 7f403f9bf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfe8bae880 con 0x55cfe8baced0 2024-09-13T18:21:57.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.238+0000 7f403f9bf740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfe8c173a0 con 0x55cfe8ba7500 2024-09-13T18:21:57.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.238+0000 7f403f9bf740 10 client.0 _ll_register_callbacks cb 0x7f40280093e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:21:57.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.238+0000 7f403659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cfe8ba7e40 0x55cfe8bada70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:57.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.238+0000 7f403659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cfe8ba7e40 0x55cfe8bada70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34018/0 (socket says 172.21.15.114:34018) 2024-09-13T18:21:57.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.238+0000 7f403659c640 1 -- 172.21.15.114:0/1716268171 learned_addr learned my addr 172.21.15.114:0/1716268171 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:57.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.238+0000 7f4036d9d640 1 --2- 172.21.15.114:0/1716268171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cfe8baced0 0x55cfe8bae010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:57.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.238+0000 7f4036d9d640 1 --2- 172.21.15.114:0/1716268171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cfe8ba7500 0x55cfe8baf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:57.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.238+0000 7f403659c640 1 -- 172.21.15.114:0/1716268171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cfe8ba7500 msgr2=0x55cfe8baf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:57.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.238+0000 7f403659c640 1 --2- 172.21.15.114:0/1716268171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cfe8ba7500 0x55cfe8baf410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.238+0000 7f403659c640 1 -- 172.21.15.114:0/1716268171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cfe8baced0 msgr2=0x55cfe8bae010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:57.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.238+0000 7f403659c640 1 --2- 172.21.15.114:0/1716268171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cfe8baced0 0x55cfe8bae010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.238+0000 7f403659c640 1 -- 172.21.15.114:0/1716268171 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cfe8a974b0 con 0x55cfe8ba7e40 2024-09-13T18:21:57.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.238+0000 7f4036d9d640 1 --2- 172.21.15.114:0/1716268171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cfe8ba7500 0x55cfe8baf410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:21:57.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.239+0000 7f403659c640 1 --2- 172.21.15.114:0/1716268171 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cfe8ba7e40 0x55cfe8bada70 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x55cfe8ba8870 tx=0x7f402800ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac627ab8b946a0d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:57.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.239+0000 7f401dffb640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:21:57.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.239+0000 7f401dffb640 1 -- 172.21.15.114:0/1716268171 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4028028630 con 0x55cfe8ba7e40 2024-09-13T18:21:57.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.239+0000 7f401dffb640 1 -- 172.21.15.114:0/1716268171 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f4028004870 con 0x55cfe8ba7e40 2024-09-13T18:21:57.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.239+0000 7f403f9bf740 1 -- 172.21.15.114:0/1716268171 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4028009e40 con 0x55cfe8ba7e40 2024-09-13T18:21:57.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.239+0000 7f401dffb640 1 -- 172.21.15.114:0/1716268171 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4028028790 con 0x55cfe8ba7e40 2024-09-13T18:21:57.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.239+0000 7f403f9bf740 10 client.15303 fetch_fsmap learned FSMap version 29 2024-09-13T18:21:57.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.239+0000 7f403f9bf740 1 -- 172.21.15.114:0/1716268171 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55cfe8b09270 con 0x55cfe8ba7e40 2024-09-13T18:21:57.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.239+0000 7f401dffb640 1 -- 172.21.15.114:0/1716268171 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4028009e40 con 0x55cfe8ba7e40 2024-09-13T18:21:57.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.239+0000 7f401dffb640 1 -- 172.21.15.114:0/1716268171 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f402800c070 con 0x55cfe8ba7e40 2024-09-13T18:21:57.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.239+0000 7f403f9bf740 10 client.15303 fetch_fsmap finished waiting for FSMap version 29 2024-09-13T18:21:57.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.239+0000 7f403f9bf740 10 client.15303 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-13T18:21:57.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.239+0000 7f403f9bf740 20 client.15303 populate_metadata read hostname 'smithi114' 2024-09-13T18:21:57.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.239+0000 7f403f9bf740 1 --2- 172.21.15.114:0/1716268171 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55cfe8c1c1d0 0x55cfe8c18b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:57.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.239+0000 7f403f9bf740 4 client.15303 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T18:21:57.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.239+0000 7f403f9bf740 1 -- 172.21.15.114:0/1716268171 --> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55cfe8862d10 con 0x55cfe8c1c1d0 2024-09-13T18:21:57.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.239+0000 7f4036d9d640 1 --2- 172.21.15.114:0/1716268171 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55cfe8c1c1d0 0x55cfe8c18b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:57.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.240+0000 7f401dffb640 10 client.15303 ms_handle_connect on v2:172.21.15.134:6836/891692210 2024-09-13T18:21:57.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.240+0000 7f4036d9d640 1 --2- 172.21.15.114:0/1716268171 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55cfe8c1c1d0 0x55cfe8c18b40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=896bdfa86ca438b6 server_cookie=8d83d7a35f775c77 in_seq=0 out_seq=0 2024-09-13T18:21:57.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.240+0000 7f401dffb640 1 -- 172.21.15.114:0/1716268171 <== mds.1 v2:172.21.15.134:6836/891692210 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55cfe8862d10 con 0x55cfe8c1c1d0 2024-09-13T18:21:57.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.240+0000 7f401dffb640 10 client.15303 handle_command_reply: tid=0 2024-09-13T18:21:57.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.240+0000 7f403f9bf740 1 client.15303 shutdown 2024-09-13T18:21:57.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.240+0000 7f403f9bf740 1 -- 172.21.15.114:0/1716268171 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cfe8ba7e40 msgr2=0x55cfe8bada70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:57.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.240+0000 7f403f9bf740 1 --2- 172.21.15.114:0/1716268171 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cfe8ba7e40 0x55cfe8bada70 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x55cfe8ba8870 tx=0x7f402800ede0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.240+0000 7f403f9bf740 1 -- 172.21.15.114:0/1716268171 shutdown_connections 2024-09-13T18:21:57.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.240+0000 7f403f9bf740 1 --2- 172.21.15.114:0/1716268171 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55cfe8c1c1d0 0x55cfe8c18b40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.240+0000 7f403f9bf740 1 --2- 172.21.15.114:0/1716268171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cfe8baced0 0x55cfe8bae010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.240+0000 7f401dffb640 0 client.15303 ms_handle_reset on v2:172.21.15.134:6836/891692210 2024-09-13T18:21:57.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.240+0000 7f403f9bf740 1 --2- 172.21.15.114:0/1716268171 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cfe8ba7e40 0x55cfe8bada70 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.240+0000 7f403f9bf740 1 --2- 172.21.15.114:0/1716268171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cfe8ba7500 0x55cfe8baf410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.240+0000 7f403f9bf740 1 -- 172.21.15.114:0/1716268171 >> 172.21.15.114:0/1716268171 conn(0x55cfe8ba2b70 msgr2=0x55cfe8babe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:57.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.240+0000 7f403f9bf740 1 -- 172.21.15.114:0/1716268171 shutdown_connections 2024-09-13T18:21:57.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.240+0000 7f403f9bf740 1 -- 172.21.15.114:0/1716268171 wait complete. 2024-09-13T18:21:57.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.240+0000 7f403f9bf740 20 client.15303 trim_cache size 0 max 16384 2024-09-13T18:21:57.255 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:21:57.255 INFO:teuthology.orchestra.run.smithi114.stdout: "cluster_fsid": "97e7edf2-71fb-11ef-bceb-c7b262605968", 2024-09-13T18:21:57.255 INFO:teuthology.orchestra.run.smithi114.stdout: "whoami": 1, 2024-09-13T18:21:57.255 INFO:teuthology.orchestra.run.smithi114.stdout: "id": 24482, 2024-09-13T18:21:57.255 INFO:teuthology.orchestra.run.smithi114.stdout: "want_state": "up:active", 2024-09-13T18:21:57.255 INFO:teuthology.orchestra.run.smithi114.stdout: "state": "up:active", 2024-09-13T18:21:57.255 INFO:teuthology.orchestra.run.smithi114.stdout: "fs_name": "cephfs", 2024-09-13T18:21:57.256 INFO:teuthology.orchestra.run.smithi114.stdout: "rank_uptime": 92.070098516000002, 2024-09-13T18:21:57.256 INFO:teuthology.orchestra.run.smithi114.stdout: "mdsmap_epoch": 29, 2024-09-13T18:21:57.256 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch": 93, 2024-09-13T18:21:57.256 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch_barrier": 91, 2024-09-13T18:21:57.256 INFO:teuthology.orchestra.run.smithi114.stdout: "uptime": 131.653333205 2024-09-13T18:21:57.256 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:21:57.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.241+0000 7f403cbc3640 1 -- 172.21.15.114:0/3794407309 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f40200a1110 msgr2=0x7f40200a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:57.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.241+0000 7f403cbc3640 1 --2- 172.21.15.114:0/3794407309 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f40200a1110 0x7f40200a35d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f402c0965e0 tx=0x7f402c08f880 comp rx=0 tx=0).stop 2024-09-13T18:21:57.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.241+0000 7f403cbc3640 1 -- 172.21.15.114:0/3794407309 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4038100f60 msgr2=0x7f4038196130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:57.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.241+0000 7f403cbc3640 1 --2- 172.21.15.114:0/3794407309 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4038100f60 0x7f4038196130 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f402c00e9f0 tx=0x7f402c00edf0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.241+0000 7f403cbc3640 1 -- 172.21.15.114:0/3794407309 shutdown_connections 2024-09-13T18:21:57.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.241+0000 7f403cbc3640 1 --2- 172.21.15.114:0/3794407309 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f40200a1110 0x7f40200a35d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.241+0000 7f403cbc3640 1 --2- 172.21.15.114:0/3794407309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4038101920 0x7f403819d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.241+0000 7f403cbc3640 1 --2- 172.21.15.114:0/3794407309 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4038100f60 0x7f4038196130 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.241+0000 7f403cbc3640 1 --2- 172.21.15.114:0/3794407309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40380ffd70 0x7f4038195bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.241+0000 7f403cbc3640 1 -- 172.21.15.114:0/3794407309 >> 172.21.15.114:0/3794407309 conn(0x7f40380fb4d0 msgr2=0x7f40381043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:57.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.241+0000 7f403cbc3640 1 -- 172.21.15.114:0/3794407309 shutdown_connections 2024-09-13T18:21:57.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.241+0000 7f403cbc3640 1 -- 172.21.15.114:0/3794407309 wait complete. 2024-09-13T18:21:57.259 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:56 smithi114 ceph-mon[30897]: pgmap v338: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-13T18:21:57.259 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:56 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/1676610167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:21:57.259 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:56 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/32698352' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:21:57.259 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:56 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2051971392' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:21:57.261 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-13T18:21:57.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.359+0000 7fb82873a640 1 -- 172.21.15.114:0/4019900322 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb820105ab0 msgr2=0x7fb820107ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:57.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.359+0000 7fb82873a640 1 --2- 172.21.15.114:0/4019900322 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb820105ab0 0x7fb820107ec0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb818009ca0 tx=0x7fb81802a8c0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.359+0000 7fb82873a640 1 -- 172.21.15.114:0/4019900322 shutdown_connections 2024-09-13T18:21:57.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.359+0000 7fb82873a640 1 --2- 172.21.15.114:0/4019900322 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb820105ab0 0x7fb820107ec0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.359+0000 7fb82873a640 1 --2- 172.21.15.114:0/4019900322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb820103180 0x7fb820105570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.359+0000 7fb82873a640 1 --2- 172.21.15.114:0/4019900322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb820100850 0x7fb820102c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.359+0000 7fb82873a640 1 -- 172.21.15.114:0/4019900322 >> 172.21.15.114:0/4019900322 conn(0x7fb8200fa410 msgr2=0x7fb8200fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:57.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.359+0000 7fb82873a640 1 -- 172.21.15.114:0/4019900322 shutdown_connections 2024-09-13T18:21:57.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.359+0000 7fb82873a640 1 -- 172.21.15.114:0/4019900322 wait complete. 2024-09-13T18:21:57.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.360+0000 7fb82873a640 1 Processor -- start 2024-09-13T18:21:57.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.360+0000 7fb82873a640 1 -- start start 2024-09-13T18:21:57.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.360+0000 7fb82873a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb820100850 0x7fb820195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:57.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.360+0000 7fb82873a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb820103180 0x7fb8201962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:57.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.360+0000 7fb82873a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb820105ab0 0x7fb82019d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:57.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.360+0000 7fb82873a640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82019d7f0 con 0x7fb820105ab0 2024-09-13T18:21:57.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.360+0000 7fb82873a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82019d960 con 0x7fb820100850 2024-09-13T18:21:57.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.360+0000 7fb82873a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82019db20 con 0x7fb820103180 2024-09-13T18:21:57.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.360+0000 7fb826cd7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb820105ab0 0x7fb82019d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:57.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.360+0000 7fb826cd7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb820105ab0 0x7fb82019d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34036/0 (socket says 172.21.15.114:34036) 2024-09-13T18:21:57.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.360+0000 7fb826cd7640 1 -- 172.21.15.114:0/1519324412 learned_addr learned my addr 172.21.15.114:0/1519324412 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:57.371 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:56 smithi134 ceph-mon[34283]: pgmap v338: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-13T18:21:57.371 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:56 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/1676610167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:21:57.371 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:56 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/32698352' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:21:57.371 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:56 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/2051971392' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:21:57.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.360+0000 7fb825cd5640 1 --2- 172.21.15.114:0/1519324412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb820103180 0x7fb8201962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:57.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.360+0000 7fb8264d6640 1 --2- 172.21.15.114:0/1519324412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb820100850 0x7fb820195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:57.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.360+0000 7fb826cd7640 1 -- 172.21.15.114:0/1519324412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb820103180 msgr2=0x7fb8201962a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:57.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.360+0000 7fb826cd7640 1 --2- 172.21.15.114:0/1519324412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb820103180 0x7fb8201962a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.360+0000 7fb826cd7640 1 -- 172.21.15.114:0/1519324412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb820100850 msgr2=0x7fb820195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:57.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.360+0000 7fb826cd7640 1 --2- 172.21.15.114:0/1519324412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb820100850 0x7fb820195d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.360+0000 7fb826cd7640 1 -- 172.21.15.114:0/1519324412 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb818009900 con 0x7fb820105ab0 2024-09-13T18:21:57.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.360+0000 7fb825cd5640 1 --2- 172.21.15.114:0/1519324412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb820103180 0x7fb8201962a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:21:57.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.361+0000 7fb826cd7640 1 --2- 172.21.15.114:0/1519324412 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb820105ab0 0x7fb82019d280 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb81802ade0 tx=0x7fb8180316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:57.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.361+0000 7fb8177fe640 1 -- 172.21.15.114:0/1519324412 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb818003b70 con 0x7fb820105ab0 2024-09-13T18:21:57.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.361+0000 7fb8177fe640 1 -- 172.21.15.114:0/1519324412 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb818040070 con 0x7fb820105ab0 2024-09-13T18:21:57.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.361+0000 7fb82873a640 1 -- 172.21.15.114:0/1519324412 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb82019dda0 con 0x7fb820105ab0 2024-09-13T18:21:57.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.361+0000 7fb8177fe640 1 -- 172.21.15.114:0/1519324412 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8180043a0 con 0x7fb820105ab0 2024-09-13T18:21:57.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.361+0000 7fb82873a640 1 -- 172.21.15.114:0/1519324412 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb82019e1f0 con 0x7fb820105ab0 2024-09-13T18:21:57.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.363+0000 7fb8177fe640 1 -- 172.21.15.114:0/1519324412 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fb818042030 con 0x7fb820105ab0 2024-09-13T18:21:57.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.363+0000 7fb82b45d740 1 Processor -- start 2024-09-13T18:21:57.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.363+0000 7fb82b45d740 1 -- start start 2024-09-13T18:21:57.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.363+0000 7fb82b45d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9fcb8e500 0x55a9fcb8e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:57.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.363+0000 7fb82b45d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9fcb8ee40 0x55a9fcb93900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:57.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.363+0000 7fb82b45d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9fcb93ed0 0x55a9fcb96390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:57.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.363+0000 7fb82b45d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9fcae97c0 con 0x55a9fcb8ee40 2024-09-13T18:21:57.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.363+0000 7fb82b45d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9fcaf8b70 con 0x55a9fcb8e500 2024-09-13T18:21:57.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.363+0000 7fb82b45d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9fcaf6300 con 0x55a9fcb93ed0 2024-09-13T18:21:57.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.364+0000 7fb8264d6640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9fcb8ee40 0x55a9fcb93900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:57.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.364+0000 7fb8264d6640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9fcb8ee40 0x55a9fcb93900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34044/0 (socket says 172.21.15.114:34044) 2024-09-13T18:21:57.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.364+0000 7fb8264d6640 1 -- 172.21.15.114:0/2797014309 learned_addr learned my addr 172.21.15.114:0/2797014309 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:57.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.364+0000 7fb8177fe640 1 --2- 172.21.15.114:0/1519324412 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb80c0a1090 0x7fb80c0a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:57.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.364+0000 7fb825cd5640 1 --2- 172.21.15.114:0/2797014309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9fcb93ed0 0x55a9fcb96390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:57.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.364+0000 7fb826cd7640 1 --2- 172.21.15.114:0/1519324412 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb80c0a1090 0x7fb80c0a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:57.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.364+0000 7fb825cd5640 1 --2- 172.21.15.114:0/2797014309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9fcb8e500 0x55a9fcb8e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:57.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.364+0000 7fb8264d6640 1 -- 172.21.15.114:0/2797014309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9fcb93ed0 msgr2=0x55a9fcb96390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:57.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.364+0000 7fb8264d6640 1 --2- 172.21.15.114:0/2797014309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9fcb93ed0 0x55a9fcb96390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.364+0000 7fb8264d6640 1 -- 172.21.15.114:0/2797014309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9fcb8e500 msgr2=0x55a9fcb8e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:57.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.364+0000 7fb8264d6640 1 --2- 172.21.15.114:0/2797014309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9fcb8e500 0x55a9fcb8e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.364+0000 7fb8264d6640 1 -- 172.21.15.114:0/2797014309 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9fca7e4b0 con 0x55a9fcb8ee40 2024-09-13T18:21:57.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.364+0000 7fb825cd5640 1 --2- 172.21.15.114:0/2797014309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9fcb8e500 0x55a9fcb8e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:21:57.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.364+0000 7fb8177fe640 1 -- 172.21.15.114:0/1519324412 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fb8180ee310 con 0x7fb820105ab0 2024-09-13T18:21:57.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.364+0000 7fb8264d6640 1 --2- 172.21.15.114:0/2797014309 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9fcb8ee40 0x55a9fcb93900 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb81c009be0 tx=0x7fb81c00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c4defab902db100 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:57.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.364+0000 7fb826cd7640 1 --2- 172.21.15.114:0/1519324412 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb80c0a1090 0x7fb80c0a3550 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb8180bb420 tx=0x7fb81808b3d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:57.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.364+0000 7fb814ff9640 1 -- 172.21.15.114:0/2797014309 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81c011470 con 0x55a9fcb8ee40 2024-09-13T18:21:57.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.364+0000 7fb814ff9640 1 -- 172.21.15.114:0/2797014309 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb81c011a60 con 0x55a9fcb8ee40 2024-09-13T18:21:57.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.364+0000 7fb814ff9640 1 -- 172.21.15.114:0/2797014309 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81c010810 con 0x55a9fcb8ee40 2024-09-13T18:21:57.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.365+0000 7fb82b45d740 1 -- 172.21.15.114:0/2797014309 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9fcb8ee40 msgr2=0x55a9fcb93900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:57.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.365+0000 7fb82b45d740 1 --2- 172.21.15.114:0/2797014309 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9fcb8ee40 0x55a9fcb93900 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb81c009be0 tx=0x7fb81c00e6c0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.365+0000 7fb82b45d740 1 -- 172.21.15.114:0/2797014309 shutdown_connections 2024-09-13T18:21:57.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.365+0000 7fb82b45d740 1 --2- 172.21.15.114:0/2797014309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9fcb93ed0 0x55a9fcb96390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.365+0000 7fb82b45d740 1 --2- 172.21.15.114:0/2797014309 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9fcb8ee40 0x55a9fcb93900 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.365+0000 7fb82b45d740 1 --2- 172.21.15.114:0/2797014309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9fcb8e500 0x55a9fcb8e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.365+0000 7fb82b45d740 1 -- 172.21.15.114:0/2797014309 >> 172.21.15.114:0/2797014309 conn(0x55a9fcb89b70 msgr2=0x55a9fcb89f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:57.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.365+0000 7fb82b45d740 1 -- 172.21.15.114:0/2797014309 shutdown_connections 2024-09-13T18:21:57.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.365+0000 7fb82b45d740 1 -- 172.21.15.114:0/2797014309 wait complete. 2024-09-13T18:21:57.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.365+0000 7fb82b45d740 1 -- start start 2024-09-13T18:21:57.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.365+0000 7fb82b45d740 1 Processor -- start 2024-09-13T18:21:57.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.366+0000 7fb82b45d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9fcb8e500 0x55a9fcb98160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:57.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.366+0000 7fb82b45d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9fcb8ee40 0x55a9fcb986a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:57.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.366+0000 7fb82b45d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9fcb93ed0 0x55a9fcb9f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:57.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.366+0000 7fb82b45d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9fcaf1bf0 con 0x55a9fcb8e500 2024-09-13T18:21:57.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.366+0000 7fb82b45d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9fcab0250 con 0x55a9fcb8ee40 2024-09-13T18:21:57.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.366+0000 7fb82b45d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9fcaafef0 con 0x55a9fcb93ed0 2024-09-13T18:21:57.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.366+0000 7fb82b45d740 10 client.0 _ll_register_callbacks cb 0x55a9fcae2740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:21:57.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.366+0000 7fb825cd5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9fcb8e500 0x55a9fcb98160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:57.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.366+0000 7fb825cd5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9fcb8e500 0x55a9fcb98160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34056/0 (socket says 172.21.15.114:34056) 2024-09-13T18:21:57.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.366+0000 7fb825cd5640 1 -- 172.21.15.114:0/1293711361 learned_addr learned my addr 172.21.15.114:0/1293711361 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:57.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.366+0000 7fb8264d6640 1 --2- 172.21.15.114:0/1293711361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9fcb8ee40 0x55a9fcb986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:57.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.366+0000 7fb825cd5640 1 -- 172.21.15.114:0/1293711361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9fcb93ed0 msgr2=0x55a9fcb9f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:57.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.366+0000 7fb825cd5640 1 --2- 172.21.15.114:0/1293711361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9fcb93ed0 0x55a9fcb9f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.366+0000 7fb825cd5640 1 -- 172.21.15.114:0/1293711361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9fcb8ee40 msgr2=0x55a9fcb986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:57.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.366+0000 7fb825cd5640 1 --2- 172.21.15.114:0/1293711361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9fcb8ee40 0x55a9fcb986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.366+0000 7fb825cd5640 1 -- 172.21.15.114:0/1293711361 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb81c009840 con 0x55a9fcb8e500 2024-09-13T18:21:57.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.366+0000 7fb825cd5640 1 --2- 172.21.15.114:0/1293711361 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9fcb8e500 0x55a9fcb98160 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb810013710 tx=0x7fb810013bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff881ba703828d2b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:57.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.366+0000 7fb809ffb640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:21:57.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.366+0000 7fb809ffb640 1 -- 172.21.15.114:0/1293711361 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb810010dc0 con 0x55a9fcb8e500 2024-09-13T18:21:57.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.366+0000 7fb809ffb640 1 -- 172.21.15.114:0/1293711361 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fb810013e20 con 0x55a9fcb8e500 2024-09-13T18:21:57.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.366+0000 7fb82b45d740 1 -- 172.21.15.114:0/1293711361 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a9fc7e53c0 con 0x55a9fcb8e500 2024-09-13T18:21:57.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.367+0000 7fb809ffb640 1 -- 172.21.15.114:0/1293711361 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb810004020 con 0x55a9fcb8e500 2024-09-13T18:21:57.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.367+0000 7fb82b45d740 10 client.15315 fetch_fsmap learned FSMap version 29 2024-09-13T18:21:57.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.367+0000 7fb82b45d740 1 -- 172.21.15.114:0/1293711361 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55a9fcaf0270 con 0x55a9fcb8e500 2024-09-13T18:21:57.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.367+0000 7fb809ffb640 1 -- 172.21.15.114:0/1293711361 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a9fc7e53c0 con 0x55a9fcb8e500 2024-09-13T18:21:57.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.367+0000 7fb809ffb640 1 -- 172.21.15.114:0/1293711361 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb8100047a0 con 0x55a9fcb8e500 2024-09-13T18:21:57.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.367+0000 7fb82b45d740 10 client.15315 fetch_fsmap finished waiting for FSMap version 29 2024-09-13T18:21:57.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.367+0000 7fb82b45d740 10 client.15315 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-13T18:21:57.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.367+0000 7fb82b45d740 20 client.15315 populate_metadata read hostname 'smithi114' 2024-09-13T18:21:57.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.367+0000 7fb82b45d740 1 --2- 172.21.15.114:0/1293711361 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55a9fcba4680 0x55a9fcba8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:57.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.367+0000 7fb82b45d740 4 client.15315 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:21:57.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.367+0000 7fb82b45d740 1 -- 172.21.15.114:0/1293711361 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a9fc7f66c0 con 0x55a9fcba4680 2024-09-13T18:21:57.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.367+0000 7fb8264d6640 1 --2- 172.21.15.114:0/1293711361 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55a9fcba4680 0x55a9fcba8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:57.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.368+0000 7fb809ffb640 10 client.15315 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:21:57.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.368+0000 7fb8264d6640 1 --2- 172.21.15.114:0/1293711361 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55a9fcba4680 0x55a9fcba8800 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b54769c4f1c72915 server_cookie=c563b56594a8bacd in_seq=0 out_seq=0 2024-09-13T18:21:57.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.368+0000 7fb809ffb640 1 -- 172.21.15.114:0/1293711361 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55a9fc7f66c0 con 0x55a9fcba4680 2024-09-13T18:21:57.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.368+0000 7fb809ffb640 10 client.15315 handle_command_reply: tid=0 2024-09-13T18:21:57.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.368+0000 7fb82b45d740 1 client.15315 shutdown 2024-09-13T18:21:57.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.368+0000 7fb82b45d740 1 -- 172.21.15.114:0/1293711361 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9fcb8e500 msgr2=0x55a9fcb98160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:57.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.368+0000 7fb82b45d740 1 --2- 172.21.15.114:0/1293711361 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9fcb8e500 0x55a9fcb98160 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb810013710 tx=0x7fb810013bd0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.368+0000 7fb82b45d740 1 -- 172.21.15.114:0/1293711361 shutdown_connections 2024-09-13T18:21:57.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.368+0000 7fb82b45d740 1 --2- 172.21.15.114:0/1293711361 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55a9fcba4680 0x55a9fcba8800 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.368+0000 7fb82b45d740 1 --2- 172.21.15.114:0/1293711361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9fcb93ed0 0x55a9fcb9f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.368+0000 7fb809ffb640 0 client.15315 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:21:57.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.368+0000 7fb82b45d740 1 --2- 172.21.15.114:0/1293711361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9fcb8ee40 0x55a9fcb986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.368+0000 7fb82b45d740 1 --2- 172.21.15.114:0/1293711361 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9fcb8e500 0x55a9fcb98160 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.368+0000 7fb82b45d740 1 -- 172.21.15.114:0/1293711361 >> 172.21.15.114:0/1293711361 conn(0x55a9fcb89b70 msgr2=0x55a9fcb8b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:57.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.368+0000 7fb82b45d740 1 -- 172.21.15.114:0/1293711361 shutdown_connections 2024-09-13T18:21:57.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.369+0000 7fb82b45d740 1 -- 172.21.15.114:0/1293711361 wait complete. 2024-09-13T18:21:57.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.369+0000 7fb82b45d740 20 client.15315 trim_cache size 0 max 16384 2024-09-13T18:21:57.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.383+0000 7fb82b45d740 1 Processor -- start 2024-09-13T18:21:57.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.383+0000 7fb82b45d740 1 -- start start 2024-09-13T18:21:57.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.383+0000 7fb82b45d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9fcb8e500 0x55a9fcb97f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:57.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.383+0000 7fb82b45d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9fcb8ee40 0x55a9fcb9f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:57.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.383+0000 7fb82b45d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9fcb93ed0 0x55a9fcb98450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:57.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.383+0000 7fb82b45d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9fca75460 con 0x55a9fcb8ee40 2024-09-13T18:21:57.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.383+0000 7fb82b45d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9fcb8f6e0 con 0x55a9fcb8e500 2024-09-13T18:21:57.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.383+0000 7fb82b45d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9fcaea990 con 0x55a9fcb93ed0 2024-09-13T18:21:57.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.383+0000 7fb8264d6640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9fcb8ee40 0x55a9fcb9f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:57.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.383+0000 7fb8264d6640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9fcb8ee40 0x55a9fcb9f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34066/0 (socket says 172.21.15.114:34066) 2024-09-13T18:21:57.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.383+0000 7fb8264d6640 1 -- 172.21.15.114:0/2003410601 learned_addr learned my addr 172.21.15.114:0/2003410601 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:57.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.383+0000 7fb825cd5640 1 --2- 172.21.15.114:0/2003410601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9fcb8e500 0x55a9fcb97f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:57.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.383+0000 7fb8264d6640 1 -- 172.21.15.114:0/2003410601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9fcb93ed0 msgr2=0x55a9fcb98450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:57.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.383+0000 7fb825cd5640 1 --2- 172.21.15.114:0/2003410601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9fcb93ed0 0x55a9fcb98450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:57.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.383+0000 7fb8264d6640 1 --2- 172.21.15.114:0/2003410601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9fcb93ed0 0x55a9fcb98450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.383+0000 7fb8264d6640 1 -- 172.21.15.114:0/2003410601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9fcb8e500 msgr2=0x55a9fcb97f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:57.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.383+0000 7fb8264d6640 1 --2- 172.21.15.114:0/2003410601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9fcb8e500 0x55a9fcb97f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.384+0000 7fb8264d6640 1 -- 172.21.15.114:0/2003410601 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9fca925f0 con 0x55a9fcb8ee40 2024-09-13T18:21:57.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.384+0000 7fb8264d6640 1 --2- 172.21.15.114:0/2003410601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9fcb8ee40 0x55a9fcb9f600 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb81c011bf0 tx=0x7fb81c009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e47235987a961db server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:57.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.384+0000 7fb8157fa640 1 -- 172.21.15.114:0/2003410601 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81c020040 con 0x55a9fcb8ee40 2024-09-13T18:21:57.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.384+0000 7fb8157fa640 1 -- 172.21.15.114:0/2003410601 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb81c02e070 con 0x55a9fcb8ee40 2024-09-13T18:21:57.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.384+0000 7fb8157fa640 1 -- 172.21.15.114:0/2003410601 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81c02a330 con 0x55a9fcb8ee40 2024-09-13T18:21:57.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.384+0000 7fb82b45d740 1 -- 172.21.15.114:0/2003410601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9fcb8ee40 msgr2=0x55a9fcb9f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:57.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.384+0000 7fb82b45d740 1 --2- 172.21.15.114:0/2003410601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9fcb8ee40 0x55a9fcb9f600 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb81c011bf0 tx=0x7fb81c009ea0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.384+0000 7fb82b45d740 1 -- 172.21.15.114:0/2003410601 shutdown_connections 2024-09-13T18:21:57.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.384+0000 7fb82b45d740 1 --2- 172.21.15.114:0/2003410601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9fcb93ed0 0x55a9fcb98450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.384+0000 7fb82b45d740 1 --2- 172.21.15.114:0/2003410601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9fcb8ee40 0x55a9fcb9f600 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.384+0000 7fb82b45d740 1 --2- 172.21.15.114:0/2003410601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9fcb8e500 0x55a9fcb97f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.384+0000 7fb82b45d740 1 -- 172.21.15.114:0/2003410601 >> 172.21.15.114:0/2003410601 conn(0x55a9fcb89b70 msgr2=0x55a9fcb8fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:57.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.384+0000 7fb82b45d740 1 -- 172.21.15.114:0/2003410601 shutdown_connections 2024-09-13T18:21:57.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.384+0000 7fb82b45d740 1 -- 172.21.15.114:0/2003410601 wait complete. 2024-09-13T18:21:57.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.385+0000 7fb82b45d740 1 -- start start 2024-09-13T18:21:57.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.385+0000 7fb82b45d740 1 Processor -- start 2024-09-13T18:21:57.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.385+0000 7fb82b45d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9fcb8e500 0x55a9fcb9f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:57.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.385+0000 7fb82b45d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9fcb8ee40 0x55a9fcb9c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:57.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.385+0000 7fb82b45d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9fcb93ed0 0x55a9fcb9c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:57.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.385+0000 7fb82b45d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9fcb9cbe0 con 0x55a9fcb93ed0 2024-09-13T18:21:57.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.385+0000 7fb82b45d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9fcbfa4c0 con 0x55a9fcb8e500 2024-09-13T18:21:57.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.385+0000 7fb82b45d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9fcbfa600 con 0x55a9fcb8ee40 2024-09-13T18:21:57.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.385+0000 7fb82b45d740 10 client.0 _ll_register_callbacks cb 0x7fb81c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:21:57.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.385+0000 7fb825cd5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9fcb93ed0 0x55a9fcb9c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:57.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.385+0000 7fb825cd5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9fcb93ed0 0x55a9fcb9c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34068/0 (socket says 172.21.15.114:34068) 2024-09-13T18:21:57.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.385+0000 7fb825cd5640 1 -- 172.21.15.114:0/2081665961 learned_addr learned my addr 172.21.15.114:0/2081665961 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:57.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.385+0000 7fb825cd5640 1 --2- 172.21.15.114:0/2081665961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9fcb8e500 0x55a9fcb9f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:57.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.385+0000 7fb8264d6640 1 --2- 172.21.15.114:0/2081665961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9fcb8ee40 0x55a9fcb9c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:57.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.385+0000 7fb825cd5640 1 -- 172.21.15.114:0/2081665961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9fcb8ee40 msgr2=0x55a9fcb9c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:57.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.385+0000 7fb825cd5640 1 --2- 172.21.15.114:0/2081665961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9fcb8ee40 0x55a9fcb9c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.385+0000 7fb825cd5640 1 -- 172.21.15.114:0/2081665961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9fcb8e500 msgr2=0x55a9fcb9f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:57.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.385+0000 7fb825cd5640 1 --2- 172.21.15.114:0/2081665961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9fcb8e500 0x55a9fcb9f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.385+0000 7fb825cd5640 1 -- 172.21.15.114:0/2081665961 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a9fca7e4b0 con 0x55a9fcb93ed0 2024-09-13T18:21:57.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.385+0000 7fb8264d6640 1 --2- 172.21.15.114:0/2081665961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9fcb8ee40 0x55a9fcb9c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:21:57.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.385+0000 7fb825cd5640 1 --2- 172.21.15.114:0/2081665961 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9fcb93ed0 0x55a9fcb9c5d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb810005890 tx=0x7fb810004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1490d1730d0abafd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:57.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.386+0000 7fb809ffb640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:21:57.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.386+0000 7fb809ffb640 1 -- 172.21.15.114:0/2081665961 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb810010e90 con 0x55a9fcb93ed0 2024-09-13T18:21:57.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.386+0000 7fb809ffb640 1 -- 172.21.15.114:0/2081665961 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fb810000e40 con 0x55a9fcb93ed0 2024-09-13T18:21:57.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.386+0000 7fb82b45d740 1 -- 172.21.15.114:0/2081665961 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb81c011a60 con 0x55a9fcb93ed0 2024-09-13T18:21:57.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.386+0000 7fb809ffb640 1 -- 172.21.15.114:0/2081665961 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb810004f30 con 0x55a9fcb93ed0 2024-09-13T18:21:57.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.386+0000 7fb82b45d740 10 client.15321 fetch_fsmap learned FSMap version 29 2024-09-13T18:21:57.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.386+0000 7fb82b45d740 1 -- 172.21.15.114:0/2081665961 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55a9fcab7d00 con 0x55a9fcb93ed0 2024-09-13T18:21:57.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.386+0000 7fb809ffb640 1 -- 172.21.15.114:0/2081665961 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb81c011a60 con 0x55a9fcb93ed0 2024-09-13T18:21:57.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.386+0000 7fb809ffb640 1 -- 172.21.15.114:0/2081665961 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb81002b760 con 0x55a9fcb93ed0 2024-09-13T18:21:57.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.386+0000 7fb82b45d740 10 client.15321 fetch_fsmap finished waiting for FSMap version 29 2024-09-13T18:21:57.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.386+0000 7fb82b45d740 10 client.15321 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-13T18:21:57.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.386+0000 7fb82b45d740 20 client.15321 populate_metadata read hostname 'smithi114' 2024-09-13T18:21:57.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.386+0000 7fb82b45d740 1 --2- 172.21.15.114:0/2081665961 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55a9fcbff3c0 0x55a9fcbfbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:57.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.386+0000 7fb82b45d740 4 client.15321 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T18:21:57.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.386+0000 7fb82b45d740 1 -- 172.21.15.114:0/2081665961 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a9fc849d10 con 0x55a9fcbff3c0 2024-09-13T18:21:57.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.386+0000 7fb8264d6640 1 --2- 172.21.15.114:0/2081665961 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55a9fcbff3c0 0x55a9fcbfbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:57.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.387+0000 7fb809ffb640 10 client.15321 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:21:57.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.387+0000 7fb8264d6640 1 --2- 172.21.15.114:0/2081665961 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55a9fcbff3c0 0x55a9fcbfbcf0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=390bfa2f1e7d5e0c server_cookie=43afdc6f0ea3d11d in_seq=0 out_seq=0 2024-09-13T18:21:57.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.387+0000 7fb809ffb640 1 -- 172.21.15.114:0/2081665961 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55a9fc849d10 con 0x55a9fcbff3c0 2024-09-13T18:21:57.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.387+0000 7fb809ffb640 10 client.15321 handle_command_reply: tid=0 2024-09-13T18:21:57.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.387+0000 7fb82b45d740 1 client.15321 shutdown 2024-09-13T18:21:57.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.387+0000 7fb82b45d740 1 -- 172.21.15.114:0/2081665961 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9fcb93ed0 msgr2=0x55a9fcb9c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:57.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.387+0000 7fb82b45d740 1 --2- 172.21.15.114:0/2081665961 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9fcb93ed0 0x55a9fcb9c5d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb810005890 tx=0x7fb810004730 comp rx=0 tx=0).stop 2024-09-13T18:21:57.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.387+0000 7fb82b45d740 1 -- 172.21.15.114:0/2081665961 shutdown_connections 2024-09-13T18:21:57.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.387+0000 7fb82b45d740 1 --2- 172.21.15.114:0/2081665961 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55a9fcbff3c0 0x55a9fcbfbcf0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.387+0000 7fb82b45d740 1 --2- 172.21.15.114:0/2081665961 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9fcb93ed0 0x55a9fcb9c5d0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.387+0000 7fb82b45d740 1 --2- 172.21.15.114:0/2081665961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9fcb8ee40 0x55a9fcb9c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.387+0000 7fb82b45d740 1 --2- 172.21.15.114:0/2081665961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9fcb8e500 0x55a9fcb9f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.387+0000 7fb809ffb640 0 client.15321 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:21:57.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.387+0000 7fb82b45d740 1 -- 172.21.15.114:0/2081665961 >> 172.21.15.114:0/2081665961 conn(0x55a9fcb89b70 msgr2=0x55a9fcb97f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:57.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.387+0000 7fb82b45d740 1 -- 172.21.15.114:0/2081665961 shutdown_connections 2024-09-13T18:21:57.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.387+0000 7fb82b45d740 1 -- 172.21.15.114:0/2081665961 wait complete. 2024-09-13T18:21:57.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.387+0000 7fb82b45d740 20 client.15321 trim_cache size 0 max 16384 2024-09-13T18:21:57.399 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:21:57.399 INFO:teuthology.orchestra.run.smithi114.stdout: "cluster_fsid": "97e7edf2-71fb-11ef-bceb-c7b262605968", 2024-09-13T18:21:57.399 INFO:teuthology.orchestra.run.smithi114.stdout: "whoami": 0, 2024-09-13T18:21:57.399 INFO:teuthology.orchestra.run.smithi114.stdout: "id": 24521, 2024-09-13T18:21:57.399 INFO:teuthology.orchestra.run.smithi114.stdout: "want_state": "up:active", 2024-09-13T18:21:57.399 INFO:teuthology.orchestra.run.smithi114.stdout: "state": "up:active", 2024-09-13T18:21:57.399 INFO:teuthology.orchestra.run.smithi114.stdout: "fs_name": "cephfs", 2024-09-13T18:21:57.400 INFO:teuthology.orchestra.run.smithi114.stdout: "rank_uptime": 115.490341501, 2024-09-13T18:21:57.400 INFO:teuthology.orchestra.run.smithi114.stdout: "mdsmap_epoch": 29, 2024-09-13T18:21:57.400 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch": 93, 2024-09-13T18:21:57.400 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch_barrier": 91, 2024-09-13T18:21:57.400 INFO:teuthology.orchestra.run.smithi114.stdout: "uptime": 121.12104864699999 2024-09-13T18:21:57.400 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:21:57.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.388+0000 7fb82873a640 1 -- 172.21.15.114:0/1519324412 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb80c0a1090 msgr2=0x7fb80c0a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:57.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.388+0000 7fb82873a640 1 --2- 172.21.15.114:0/1519324412 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb80c0a1090 0x7fb80c0a3550 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb8180bb420 tx=0x7fb81808b3d0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.388+0000 7fb82873a640 1 -- 172.21.15.114:0/1519324412 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb820105ab0 msgr2=0x7fb82019d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:57.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.388+0000 7fb82873a640 1 --2- 172.21.15.114:0/1519324412 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb820105ab0 0x7fb82019d280 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb81802ade0 tx=0x7fb8180316f0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.388+0000 7fb82873a640 1 -- 172.21.15.114:0/1519324412 shutdown_connections 2024-09-13T18:21:57.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.388+0000 7fb82873a640 1 --2- 172.21.15.114:0/1519324412 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb80c0a1090 0x7fb80c0a3550 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.388+0000 7fb82873a640 1 --2- 172.21.15.114:0/1519324412 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb820105ab0 0x7fb82019d280 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.388+0000 7fb82873a640 1 --2- 172.21.15.114:0/1519324412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb820103180 0x7fb8201962a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.388+0000 7fb82873a640 1 --2- 172.21.15.114:0/1519324412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb820100850 0x7fb820195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:57.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.388+0000 7fb82873a640 1 -- 172.21.15.114:0/1519324412 >> 172.21.15.114:0/1519324412 conn(0x7fb8200fa410 msgr2=0x7fb820101330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:57.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.388+0000 7fb82873a640 1 -- 172.21.15.114:0/1519324412 shutdown_connections 2024-09-13T18:21:57.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:57.388+0000 7fb82873a640 1 -- 172.21.15.114:0/1519324412 wait complete. 2024-09-13T18:21:57.404 INFO:tasks.cephadm:Setup complete, yielding 2024-09-13T18:21:57.404 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-13T18:21:57.438 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi114.front.sepia.ceph.com 2024-09-13T18:21:57.439 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- bash -c 'ceph orch status' 2024-09-13T18:21:57.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:56 smithi204 ceph-mon[34777]: pgmap v338: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-13T18:21:57.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:56 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/1676610167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:21:57.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:56 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/32698352' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:21:57.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:56 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/2051971392' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:21:57.634 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:21:58.013 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:57 smithi114 ceph-mon[30897]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-13T18:21:58.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:57 smithi134 ceph-mon[34283]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-13T18:21:58.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:57 smithi204 ceph-mon[34777]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-13T18:21:59.037 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:21:58 smithi114 ceph-mon[30897]: pgmap v339: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-13T18:21:59.368 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:21:58 smithi134 ceph-mon[34283]: pgmap v339: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-13T18:21:59.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.368+0000 7f12c2985640 1 -- 172.21.15.114:0/3637583787 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12bc100fe0 msgr2=0x7f12bc103450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:59.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.368+0000 7f12c2985640 1 --2- 172.21.15.114:0/3637583787 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12bc100fe0 0x7f12bc103450 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f12a4009900 tx=0x7f12a402a410 comp rx=0 tx=0).stop 2024-09-13T18:21:59.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.368+0000 7f12c2985640 1 -- 172.21.15.114:0/3637583787 shutdown_connections 2024-09-13T18:21:59.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.368+0000 7f12c2985640 1 --2- 172.21.15.114:0/3637583787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12bc103990 0x7f12bc103e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:59.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.368+0000 7f12c2985640 1 --2- 172.21.15.114:0/3637583787 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12bc100fe0 0x7f12bc103450 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:59.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.368+0000 7f12c2985640 1 --2- 172.21.15.114:0/3637583787 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12bc0fe690 0x7f12bc100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:59.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.368+0000 7f12c2985640 1 -- 172.21.15.114:0/3637583787 >> 172.21.15.114:0/3637583787 conn(0x7f12bc0fa4c0 msgr2=0x7f12bc0fc8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:59.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.368+0000 7f12c2985640 1 -- 172.21.15.114:0/3637583787 shutdown_connections 2024-09-13T18:21:59.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.368+0000 7f12c2985640 1 -- 172.21.15.114:0/3637583787 wait complete. 2024-09-13T18:21:59.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.368+0000 7f12c2985640 1 Processor -- start 2024-09-13T18:21:59.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.368+0000 7f12c2985640 1 -- start start 2024-09-13T18:21:59.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.369+0000 7f12c2985640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12bc0fe690 0x7f12bc19c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:59.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.369+0000 7f12c2985640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12bc100fe0 0x7f12bc19c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:59.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.369+0000 7f12c2985640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12bc103990 0x7f12bc1a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:59.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.369+0000 7f12c2985640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12bc1a3dd0 con 0x7f12bc103990 2024-09-13T18:21:59.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.369+0000 7f12c2985640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12bc1a3f40 con 0x7f12bc100fe0 2024-09-13T18:21:59.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.369+0000 7f12c2985640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12bc1a4100 con 0x7f12bc0fe690 2024-09-13T18:21:59.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.369+0000 7f12c0f22640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12bc103990 0x7f12bc1a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:59.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.369+0000 7f12c0f22640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12bc103990 0x7f12bc1a3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34092/0 (socket says 172.21.15.114:34092) 2024-09-13T18:21:59.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.369+0000 7f12c0f22640 1 -- 172.21.15.114:0/2883472556 learned_addr learned my addr 172.21.15.114:0/2883472556 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:21:59.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.369+0000 7f12bbfff640 1 --2- 172.21.15.114:0/2883472556 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12bc0fe690 0x7f12bc19c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:59.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.369+0000 7f12b3fff640 1 --2- 172.21.15.114:0/2883472556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12bc100fe0 0x7f12bc19c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:59.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.369+0000 7f12c0f22640 1 -- 172.21.15.114:0/2883472556 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12bc0fe690 msgr2=0x7f12bc19c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:59.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.369+0000 7f12c0f22640 1 --2- 172.21.15.114:0/2883472556 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12bc0fe690 0x7f12bc19c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:59.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.369+0000 7f12c0f22640 1 -- 172.21.15.114:0/2883472556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12bc100fe0 msgr2=0x7f12bc19c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:59.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.369+0000 7f12c0f22640 1 --2- 172.21.15.114:0/2883472556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12bc100fe0 0x7f12bc19c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:59.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.369+0000 7f12c0f22640 1 -- 172.21.15.114:0/2883472556 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a4009560 con 0x7f12bc103990 2024-09-13T18:21:59.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.369+0000 7f12b3fff640 1 --2- 172.21.15.114:0/2883472556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12bc100fe0 0x7f12bc19c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:21:59.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.369+0000 7f12c0f22640 1 --2- 172.21.15.114:0/2883472556 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12bc103990 0x7f12bc1a3890 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f12b400eb10 tx=0x7f12b400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:59.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.369+0000 7f12b9ffb640 1 -- 172.21.15.114:0/2883472556 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b4004b20 con 0x7f12bc103990 2024-09-13T18:21:59.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.369+0000 7f12b9ffb640 1 -- 172.21.15.114:0/2883472556 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f12b4004c80 con 0x7f12bc103990 2024-09-13T18:21:59.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.369+0000 7f12b9ffb640 1 -- 172.21.15.114:0/2883472556 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b4005a10 con 0x7f12bc103990 2024-09-13T18:21:59.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.369+0000 7f12c2985640 1 -- 172.21.15.114:0/2883472556 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12bc1a43e0 con 0x7f12bc103990 2024-09-13T18:21:59.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.369+0000 7f12c2985640 1 -- 172.21.15.114:0/2883472556 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12bc1a4890 con 0x7f12bc103990 2024-09-13T18:21:59.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.370+0000 7f12c2985640 1 -- 172.21.15.114:0/2883472556 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12bc0fbfb0 con 0x7f12bc103990 2024-09-13T18:21:59.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.371+0000 7f12b9ffb640 1 -- 172.21.15.114:0/2883472556 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f12b4005020 con 0x7f12bc103990 2024-09-13T18:21:59.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.372+0000 7f12b9ffb640 1 --2- 172.21.15.114:0/2883472556 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f12a80a10b0 0x7f12a80a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:59.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.372+0000 7f12bbfff640 1 --2- 172.21.15.114:0/2883472556 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f12a80a10b0 0x7f12a80a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:59.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.372+0000 7f12b9ffb640 1 -- 172.21.15.114:0/2883472556 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f12b4014070 con 0x7f12bc103990 2024-09-13T18:21:59.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.373+0000 7f12bbfff640 1 --2- 172.21.15.114:0/2883472556 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f12a80a10b0 0x7f12a80a3570 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f12ac005fd0 tx=0x7f12ac005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:59.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.376+0000 7f12b9ffb640 1 -- 172.21.15.114:0/2883472556 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f12b4096760 con 0x7f12bc103990 2024-09-13T18:21:59.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:21:58 smithi204 ceph-mon[34777]: pgmap v339: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-13T18:21:59.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.557+0000 7f12c2985640 1 -- 172.21.15.114:0/2883472556 --> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f12bc02c480 con 0x7f12a80a10b0 2024-09-13T18:21:59.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.558+0000 7f12b9ffb640 1 -- 172.21.15.114:0/2883472556 <== mgr.14148 v2:172.21.15.114:6800/3396185541 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f12bc02c480 con 0x7f12a80a10b0 2024-09-13T18:21:59.561 INFO:teuthology.orchestra.run.smithi114.stdout:Backend: cephadm 2024-09-13T18:21:59.561 INFO:teuthology.orchestra.run.smithi114.stdout:Available: Yes 2024-09-13T18:21:59.562 INFO:teuthology.orchestra.run.smithi114.stdout:Paused: No 2024-09-13T18:21:59.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.561+0000 7f12c2985640 1 -- 172.21.15.114:0/2883472556 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f12a80a10b0 msgr2=0x7f12a80a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:59.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.561+0000 7f12c2985640 1 --2- 172.21.15.114:0/2883472556 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f12a80a10b0 0x7f12a80a3570 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f12ac005fd0 tx=0x7f12ac005850 comp rx=0 tx=0).stop 2024-09-13T18:21:59.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.561+0000 7f12c2985640 1 -- 172.21.15.114:0/2883472556 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12bc103990 msgr2=0x7f12bc1a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:59.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.561+0000 7f12c2985640 1 --2- 172.21.15.114:0/2883472556 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12bc103990 0x7f12bc1a3890 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f12b400eb10 tx=0x7f12b400c2d0 comp rx=0 tx=0).stop 2024-09-13T18:21:59.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.561+0000 7f12c2985640 1 -- 172.21.15.114:0/2883472556 shutdown_connections 2024-09-13T18:21:59.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.561+0000 7f12c2985640 1 --2- 172.21.15.114:0/2883472556 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f12a80a10b0 0x7f12a80a3570 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:59.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.561+0000 7f12c2985640 1 --2- 172.21.15.114:0/2883472556 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12bc103990 0x7f12bc1a3890 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:59.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.561+0000 7f12c2985640 1 --2- 172.21.15.114:0/2883472556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12bc100fe0 0x7f12bc19c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:59.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.561+0000 7f12c2985640 1 --2- 172.21.15.114:0/2883472556 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12bc0fe690 0x7f12bc19c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:59.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.561+0000 7f12c2985640 1 -- 172.21.15.114:0/2883472556 >> 172.21.15.114:0/2883472556 conn(0x7f12bc0fa4c0 msgr2=0x7f12bc0ff170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:59.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.561+0000 7f12c2985640 1 -- 172.21.15.114:0/2883472556 shutdown_connections 2024-09-13T18:21:59.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:21:59.561+0000 7f12c2985640 1 -- 172.21.15.114:0/2883472556 wait complete. 2024-09-13T18:22:00.286 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-13T18:22:00.495 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:22:01.219 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:00 smithi114 ceph-mon[30897]: from='client.15327 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:22:01.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:00 smithi114 ceph-mon[30897]: pgmap v340: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-13T18:22:01.368 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:00 smithi134 ceph-mon[34283]: from='client.15327 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:22:01.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:00 smithi134 ceph-mon[34283]: pgmap v340: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-13T18:22:01.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:00 smithi204 ceph-mon[34777]: from='client.15327 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:22:01.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:00 smithi204 ceph-mon[34777]: pgmap v340: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-13T18:22:02.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.325+0000 7f95ac80b640 1 -- 172.21.15.114:0/1564327258 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95a81031a0 msgr2=0x7f95a8105590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:02.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.325+0000 7f95ac80b640 1 --2- 172.21.15.114:0/1564327258 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95a81031a0 0x7f95a8105590 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9590009900 tx=0x7f959002a410 comp rx=0 tx=0).stop 2024-09-13T18:22:02.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.325+0000 7f95ac80b640 1 -- 172.21.15.114:0/1564327258 shutdown_connections 2024-09-13T18:22:02.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.325+0000 7f95ac80b640 1 --2- 172.21.15.114:0/1564327258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95a8105ad0 0x7f95a8107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:02.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.325+0000 7f95ac80b640 1 --2- 172.21.15.114:0/1564327258 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95a81031a0 0x7f95a8105590 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:02.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.325+0000 7f95ac80b640 1 --2- 172.21.15.114:0/1564327258 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95a8100870 0x7f95a8102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:02.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.325+0000 7f95ac80b640 1 -- 172.21.15.114:0/1564327258 >> 172.21.15.114:0/1564327258 conn(0x7f95a80fa490 msgr2=0x7f95a80fc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:02.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.325+0000 7f95ac80b640 1 -- 172.21.15.114:0/1564327258 shutdown_connections 2024-09-13T18:22:02.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.325+0000 7f95ac80b640 1 -- 172.21.15.114:0/1564327258 wait complete. 2024-09-13T18:22:02.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.326+0000 7f95ac80b640 1 Processor -- start 2024-09-13T18:22:02.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.326+0000 7f95ac80b640 1 -- start start 2024-09-13T18:22:02.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.326+0000 7f95ac80b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95a8100870 0x7f95a8195c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:02.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.326+0000 7f95ac80b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95a81031a0 0x7f95a8196190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:02.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.326+0000 7f95ac80b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95a8105ad0 0x7f95a819d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:02.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.326+0000 7f95ac80b640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a819d740 con 0x7f95a8100870 2024-09-13T18:22:02.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.326+0000 7f95ac80b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a819d8b0 con 0x7f95a81031a0 2024-09-13T18:22:02.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.326+0000 7f95ac80b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a819da70 con 0x7f95a8105ad0 2024-09-13T18:22:02.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.327+0000 7f95a659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95a8100870 0x7f95a8195c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:02.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.327+0000 7f95a659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95a8100870 0x7f95a8195c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34110/0 (socket says 172.21.15.114:34110) 2024-09-13T18:22:02.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.327+0000 7f95a659c640 1 -- 172.21.15.114:0/2670416645 learned_addr learned my addr 172.21.15.114:0/2670416645 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:02.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.327+0000 7f959ffff640 1 --2- 172.21.15.114:0/2670416645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95a81031a0 0x7f95a8196190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:02.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.327+0000 7f95a6d9d640 1 --2- 172.21.15.114:0/2670416645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95a8105ad0 0x7f95a819d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:02.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.327+0000 7f95a659c640 1 -- 172.21.15.114:0/2670416645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95a8105ad0 msgr2=0x7f95a819d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:02.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.327+0000 7f95a659c640 1 --2- 172.21.15.114:0/2670416645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95a8105ad0 0x7f95a819d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:02.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.327+0000 7f95a659c640 1 -- 172.21.15.114:0/2670416645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95a81031a0 msgr2=0x7f95a8196190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:02.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.327+0000 7f95a659c640 1 --2- 172.21.15.114:0/2670416645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95a81031a0 0x7f95a8196190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:02.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.327+0000 7f95a659c640 1 -- 172.21.15.114:0/2670416645 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9590009560 con 0x7f95a8100870 2024-09-13T18:22:02.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.327+0000 7f95a6d9d640 1 --2- 172.21.15.114:0/2670416645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95a8105ad0 0x7f95a819d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:22:02.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.327+0000 7f95a659c640 1 --2- 172.21.15.114:0/2670416645 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95a8100870 0x7f95a8195c50 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f959800c920 tx=0x7f959800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:02.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.327+0000 7f959f7fe640 1 -- 172.21.15.114:0/2670416645 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9598007c70 con 0x7f95a8100870 2024-09-13T18:22:02.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.327+0000 7f959f7fe640 1 -- 172.21.15.114:0/2670416645 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9598007dd0 con 0x7f95a8100870 2024-09-13T18:22:02.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.327+0000 7f95ac80b640 1 -- 172.21.15.114:0/2670416645 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95a819dd50 con 0x7f95a8100870 2024-09-13T18:22:02.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.327+0000 7f959f7fe640 1 -- 172.21.15.114:0/2670416645 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9598018730 con 0x7f95a8100870 2024-09-13T18:22:02.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.327+0000 7f95ac80b640 1 -- 172.21.15.114:0/2670416645 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95a819e1a0 con 0x7f95a8100870 2024-09-13T18:22:02.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.328+0000 7f95ac80b640 1 -- 172.21.15.114:0/2670416645 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95a80fbf80 con 0x7f95a8100870 2024-09-13T18:22:02.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.329+0000 7f959f7fe640 1 -- 172.21.15.114:0/2670416645 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f959800f450 con 0x7f95a8100870 2024-09-13T18:22:02.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.330+0000 7f959f7fe640 1 --2- 172.21.15.114:0/2670416645 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f95940a1080 0x7f95940a3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:02.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.330+0000 7f959ffff640 1 --2- 172.21.15.114:0/2670416645 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f95940a1080 0x7f95940a3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:02.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.330+0000 7f959f7fe640 1 -- 172.21.15.114:0/2670416645 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f95980cd030 con 0x7f95a8100870 2024-09-13T18:22:02.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.331+0000 7f959ffff640 1 --2- 172.21.15.114:0/2670416645 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f95940a1080 0x7f95940a3540 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f959002aa00 tx=0x7f9590005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:02.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.334+0000 7f959f7fe640 1 -- 172.21.15.114:0/2670416645 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f95980964e0 con 0x7f95a8100870 2024-09-13T18:22:02.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.504+0000 7f95ac80b640 1 -- 172.21.15.114:0/2670416645 --> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f95a802c4f0 con 0x7f95940a1080 2024-09-13T18:22:02.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.516+0000 7f959f7fe640 1 -- 172.21.15.114:0/2670416645 <== mgr.14148 v2:172.21.15.114:6800/3396185541 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4061 (secure 0 0 0) 0x7f95a802c4f0 con 0x7f95940a1080 2024-09-13T18:22:02.519 INFO:teuthology.orchestra.run.smithi114.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-13T18:22:02.520 INFO:teuthology.orchestra.run.smithi114.stdout:mds.a smithi114 running (2m) 22s ago 2m 17.1M - 18.2.4-860-g6c6b083f 8a91f241d7d4 c8f4947b120f 2024-09-13T18:22:02.520 INFO:teuthology.orchestra.run.smithi114.stdout:mds.b smithi134 running (2m) 61s ago 2m 25.9M - 18.2.4-860-g6c6b083f 8a91f241d7d4 90dfb0348c83 2024-09-13T18:22:02.520 INFO:teuthology.orchestra.run.smithi114.stdout:mds.c smithi204 running (2m) 61s ago 2m 17.4M - 18.2.4-860-g6c6b083f 8a91f241d7d4 9a125d62aba4 2024-09-13T18:22:02.520 INFO:teuthology.orchestra.run.smithi114.stdout:mds.d smithi114 running (2m) 22s ago 2m 13.4M - 18.2.4-860-g6c6b083f 8a91f241d7d4 d50a04dd9db4 2024-09-13T18:22:02.520 INFO:teuthology.orchestra.run.smithi114.stdout:mds.e smithi134 running (2m) 61s ago 2m 16.2M - 18.2.4-860-g6c6b083f 8a91f241d7d4 b4bc60a2e73a 2024-09-13T18:22:02.520 INFO:teuthology.orchestra.run.smithi114.stdout:mds.f smithi204 running (2m) 61s ago 2m 18.5M - 18.2.4-860-g6c6b083f 8a91f241d7d4 047c0dcb239a 2024-09-13T18:22:02.520 INFO:teuthology.orchestra.run.smithi114.stdout:mds.g smithi114 running (2m) 22s ago 2m 15.9M - 18.2.4-860-g6c6b083f 8a91f241d7d4 15d44722d035 2024-09-13T18:22:02.520 INFO:teuthology.orchestra.run.smithi114.stdout:mds.h smithi134 running (2m) 61s ago 2m 19.7M - 18.2.4-860-g6c6b083f 8a91f241d7d4 a14a3375eea7 2024-09-13T18:22:02.520 INFO:teuthology.orchestra.run.smithi114.stdout:mds.i smithi204 running (2m) 61s ago 2m 18.5M - 18.2.4-860-g6c6b083f 8a91f241d7d4 aecc8a6696be 2024-09-13T18:22:02.520 INFO:teuthology.orchestra.run.smithi114.stdout:mds.j smithi114 running (2m) 22s ago 2m 13.5M - 18.2.4-860-g6c6b083f 8a91f241d7d4 4687664a4f36 2024-09-13T18:22:02.520 INFO:teuthology.orchestra.run.smithi114.stdout:mds.k smithi134 running (2m) 61s ago 2m 17.6M - 18.2.4-860-g6c6b083f 8a91f241d7d4 b3714723861e 2024-09-13T18:22:02.520 INFO:teuthology.orchestra.run.smithi114.stdout:mds.l smithi204 running (2m) 61s ago 2m 16.4M - 18.2.4-860-g6c6b083f 8a91f241d7d4 11004bae1391 2024-09-13T18:22:02.520 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.x smithi114 *:9283,8765 running (9m) 22s ago 9m 500M - 18.2.4-860-g6c6b083f 8a91f241d7d4 254a2146bf84 2024-09-13T18:22:02.521 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.y smithi134 *:8443,8765 running (7m) 61s ago 7m 440M - 18.2.4-860-g6c6b083f 8a91f241d7d4 2a4536a41e93 2024-09-13T18:22:02.521 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.z smithi204 *:8443,8765 running (7m) 61s ago 7m 442M - 18.2.4-860-g6c6b083f 8a91f241d7d4 130ab02f9a47 2024-09-13T18:22:02.521 INFO:teuthology.orchestra.run.smithi114.stdout:mon.a smithi114 running (9m) 22s ago 9m 49.9M 2048M 18.2.4-860-g6c6b083f 8a91f241d7d4 cc359b54bfa5 2024-09-13T18:22:02.521 INFO:teuthology.orchestra.run.smithi114.stdout:mon.b smithi134 running (7m) 61s ago 7m 38.9M 2048M 18.2.4-860-g6c6b083f 8a91f241d7d4 ba2043c577c2 2024-09-13T18:22:02.521 INFO:teuthology.orchestra.run.smithi114.stdout:mon.c smithi204 running (7m) 61s ago 7m 39.2M 2048M 18.2.4-860-g6c6b083f 8a91f241d7d4 ce3d7d936a94 2024-09-13T18:22:02.521 INFO:teuthology.orchestra.run.smithi114.stdout:osd.0 smithi114 running (7m) 22s ago 7m 56.2M 4272M 18.2.4-860-g6c6b083f 8a91f241d7d4 9364668bed48 2024-09-13T18:22:02.521 INFO:teuthology.orchestra.run.smithi114.stdout:osd.1 smithi134 running (6m) 61s ago 6m 57.3M 4272M 18.2.4-860-g6c6b083f 8a91f241d7d4 c0b5b27a010a 2024-09-13T18:22:02.521 INFO:teuthology.orchestra.run.smithi114.stdout:osd.2 smithi204 running (6m) 61s ago 6m 46.8M 4272M 18.2.4-860-g6c6b083f 8a91f241d7d4 f862f9bc03c9 2024-09-13T18:22:02.521 INFO:teuthology.orchestra.run.smithi114.stdout:osd.3 smithi114 running (6m) 22s ago 6m 44.0M 4272M 18.2.4-860-g6c6b083f 8a91f241d7d4 a5a5d5b5f348 2024-09-13T18:22:02.521 INFO:teuthology.orchestra.run.smithi114.stdout:osd.4 smithi134 running (5m) 61s ago 5m 43.5M 4272M 18.2.4-860-g6c6b083f 8a91f241d7d4 497774642f45 2024-09-13T18:22:02.521 INFO:teuthology.orchestra.run.smithi114.stdout:osd.5 smithi204 running (5m) 61s ago 5m 45.1M 4272M 18.2.4-860-g6c6b083f 8a91f241d7d4 d279a8bb0f21 2024-09-13T18:22:02.521 INFO:teuthology.orchestra.run.smithi114.stdout:osd.6 smithi114 running (4m) 22s ago 4m 52.2M 4272M 18.2.4-860-g6c6b083f 8a91f241d7d4 ac5aa81b2a31 2024-09-13T18:22:02.521 INFO:teuthology.orchestra.run.smithi114.stdout:osd.7 smithi134 running (4m) 61s ago 4m 55.2M 4272M 18.2.4-860-g6c6b083f 8a91f241d7d4 a7c8dc269fa3 2024-09-13T18:22:02.521 INFO:teuthology.orchestra.run.smithi114.stdout:osd.8 smithi204 running (4m) 61s ago 4m 53.5M 4272M 18.2.4-860-g6c6b083f 8a91f241d7d4 12a7be3c1e86 2024-09-13T18:22:02.522 INFO:teuthology.orchestra.run.smithi114.stdout:osd.9 smithi114 running (3m) 22s ago 3m 42.9M 4272M 18.2.4-860-g6c6b083f 8a91f241d7d4 3907e244d658 2024-09-13T18:22:02.522 INFO:teuthology.orchestra.run.smithi114.stdout:osd.10 smithi134 running (3m) 61s ago 3m 43.4M 4272M 18.2.4-860-g6c6b083f 8a91f241d7d4 9ac9b7f8e79c 2024-09-13T18:22:02.522 INFO:teuthology.orchestra.run.smithi114.stdout:osd.11 smithi204 running (2m) 61s ago 2m 51.0M 4272M 18.2.4-860-g6c6b083f 8a91f241d7d4 8236824a6098 2024-09-13T18:22:02.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.518+0000 7f95ac80b640 1 -- 172.21.15.114:0/2670416645 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f95940a1080 msgr2=0x7f95940a3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:02.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.518+0000 7f95ac80b640 1 --2- 172.21.15.114:0/2670416645 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f95940a1080 0x7f95940a3540 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f959002aa00 tx=0x7f9590005a20 comp rx=0 tx=0).stop 2024-09-13T18:22:02.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.518+0000 7f95ac80b640 1 -- 172.21.15.114:0/2670416645 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95a8100870 msgr2=0x7f95a8195c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:02.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.518+0000 7f95ac80b640 1 --2- 172.21.15.114:0/2670416645 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95a8100870 0x7f95a8195c50 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f959800c920 tx=0x7f959800cde0 comp rx=0 tx=0).stop 2024-09-13T18:22:02.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.518+0000 7f95ac80b640 1 -- 172.21.15.114:0/2670416645 shutdown_connections 2024-09-13T18:22:02.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.518+0000 7f95ac80b640 1 --2- 172.21.15.114:0/2670416645 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f95940a1080 0x7f95940a3540 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:02.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.518+0000 7f95ac80b640 1 --2- 172.21.15.114:0/2670416645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95a8105ad0 0x7f95a819d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:02.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.518+0000 7f95ac80b640 1 --2- 172.21.15.114:0/2670416645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95a81031a0 0x7f95a8196190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:02.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.518+0000 7f95ac80b640 1 --2- 172.21.15.114:0/2670416645 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95a8100870 0x7f95a8195c50 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:02.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.518+0000 7f95ac80b640 1 -- 172.21.15.114:0/2670416645 >> 172.21.15.114:0/2670416645 conn(0x7f95a80fa490 msgr2=0x7f95a8101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:02.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.518+0000 7f95ac80b640 1 -- 172.21.15.114:0/2670416645 shutdown_connections 2024-09-13T18:22:02.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:02.518+0000 7f95ac80b640 1 -- 172.21.15.114:0/2670416645 wait complete. 2024-09-13T18:22:03.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:03 smithi114 ceph-mon[30897]: pgmap v341: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-13T18:22:03.328 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-13T18:22:03.368 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:03 smithi134 ceph-mon[34283]: pgmap v341: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-13T18:22:03.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:03 smithi204 ceph-mon[34777]: pgmap v341: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-13T18:22:03.527 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:22:04.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:04 smithi114 ceph-mon[30897]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:22:04.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:04 smithi134 ceph-mon[34283]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:22:04.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:04 smithi204 ceph-mon[34777]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:22:05.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.235+0000 7fcea1875640 1 -- 172.21.15.114:0/3299372222 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce9c0fed10 msgr2=0x7fce9c0ff110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:05.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.235+0000 7fcea1875640 1 --2- 172.21.15.114:0/3299372222 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce9c0fed10 0x7fce9c0ff110 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fce94009ca0 tx=0x7fce9402a8c0 comp rx=0 tx=0).stop 2024-09-13T18:22:05.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.235+0000 7fcea1875640 1 -- 172.21.15.114:0/3299372222 shutdown_connections 2024-09-13T18:22:05.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.235+0000 7fcea1875640 1 --2- 172.21.15.114:0/3299372222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce9c1008d0 0x7fce9c107160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:05.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.235+0000 7fcea1875640 1 --2- 172.21.15.114:0/3299372222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce9c0fff10 0x7fce9c100390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:05.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.235+0000 7fcea1875640 1 --2- 172.21.15.114:0/3299372222 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce9c0fed10 0x7fce9c0ff110 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:05.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.235+0000 7fcea1875640 1 -- 172.21.15.114:0/3299372222 >> 172.21.15.114:0/3299372222 conn(0x7fce9c0fa4c0 msgr2=0x7fce9c0fc8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:05.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.235+0000 7fcea1875640 1 -- 172.21.15.114:0/3299372222 shutdown_connections 2024-09-13T18:22:05.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.235+0000 7fcea1875640 1 -- 172.21.15.114:0/3299372222 wait complete. 2024-09-13T18:22:05.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.236+0000 7fcea1875640 1 Processor -- start 2024-09-13T18:22:05.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.236+0000 7fcea1875640 1 -- start start 2024-09-13T18:22:05.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.236+0000 7fcea1875640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce9c0fed10 0x7fce9c194c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:05.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.236+0000 7fcea1875640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce9c0fff10 0x7fce9c1951a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:05.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.236+0000 7fcea1875640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce9c1008d0 0x7fce9c19c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:05.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.236+0000 7fcea1875640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce9c19c750 con 0x7fce9c1008d0 2024-09-13T18:22:05.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.236+0000 7fcea1875640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce9c19c8c0 con 0x7fce9c0fed10 2024-09-13T18:22:05.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.236+0000 7fcea1875640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce9c19ca80 con 0x7fce9c0fff10 2024-09-13T18:22:05.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.236+0000 7fce9b7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce9c1008d0 0x7fce9c19c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:05.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.236+0000 7fce9b7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce9c1008d0 0x7fce9c19c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44002/0 (socket says 172.21.15.114:44002) 2024-09-13T18:22:05.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.236+0000 7fce9b7fe640 1 -- 172.21.15.114:0/1194893065 learned_addr learned my addr 172.21.15.114:0/1194893065 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:05.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.237+0000 7fce9a7fc640 1 --2- 172.21.15.114:0/1194893065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce9c0fff10 0x7fce9c1951a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:05.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.237+0000 7fce9b7fe640 1 -- 172.21.15.114:0/1194893065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce9c0fff10 msgr2=0x7fce9c1951a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:05.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.237+0000 7fce9b7fe640 1 --2- 172.21.15.114:0/1194893065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce9c0fff10 0x7fce9c1951a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:05.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.237+0000 7fce9b7fe640 1 -- 172.21.15.114:0/1194893065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce9c0fed10 msgr2=0x7fce9c194c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:05.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.237+0000 7fce9affd640 1 --2- 172.21.15.114:0/1194893065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce9c0fed10 0x7fce9c194c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:05.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.237+0000 7fce9b7fe640 1 --2- 172.21.15.114:0/1194893065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce9c0fed10 0x7fce9c194c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:05.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.237+0000 7fce9b7fe640 1 -- 172.21.15.114:0/1194893065 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce94009900 con 0x7fce9c1008d0 2024-09-13T18:22:05.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.237+0000 7fce9a7fc640 1 --2- 172.21.15.114:0/1194893065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce9c0fff10 0x7fce9c1951a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:22:05.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.237+0000 7fce9b7fe640 1 --2- 172.21.15.114:0/1194893065 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce9c1008d0 0x7fce9c19c1e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fce8c00d960 tx=0x7fce8c00de20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:05.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.237+0000 7fcea0873640 1 -- 172.21.15.114:0/1194893065 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8c00ed60 con 0x7fce9c1008d0 2024-09-13T18:22:05.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.237+0000 7fcea1875640 1 -- 172.21.15.114:0/1194893065 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce9c068780 con 0x7fce9c1008d0 2024-09-13T18:22:05.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.237+0000 7fcea0873640 1 -- 172.21.15.114:0/1194893065 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fce8c0168d0 con 0x7fce9c1008d0 2024-09-13T18:22:05.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.237+0000 7fcea0873640 1 -- 172.21.15.114:0/1194893065 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8c014740 con 0x7fce9c1008d0 2024-09-13T18:22:05.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.237+0000 7fcea1875640 1 -- 172.21.15.114:0/1194893065 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce9c068d50 con 0x7fce9c1008d0 2024-09-13T18:22:05.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.238+0000 7fcea1875640 1 -- 172.21.15.114:0/1194893065 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce9c10a4e0 con 0x7fce9c1008d0 2024-09-13T18:22:05.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.239+0000 7fcea0873640 1 -- 172.21.15.114:0/1194893065 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fce8c016a40 con 0x7fce9c1008d0 2024-09-13T18:22:05.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.240+0000 7fcea0873640 1 --2- 172.21.15.114:0/1194893065 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fce840a1080 0x7fce840a3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:05.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.240+0000 7fce9affd640 1 --2- 172.21.15.114:0/1194893065 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fce840a1080 0x7fce840a3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:05.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.240+0000 7fcea0873640 1 -- 172.21.15.114:0/1194893065 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fce8c0ce670 con 0x7fce9c1008d0 2024-09-13T18:22:05.264 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.240+0000 7fce9affd640 1 --2- 172.21.15.114:0/1194893065 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fce840a1080 0x7fce840a3540 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fce94000f80 tx=0x7fce94009890 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:05.264 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.244+0000 7fcea0873640 1 -- 172.21.15.114:0/1194893065 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fce8c097a90 con 0x7fce9c1008d0 2024-09-13T18:22:05.264 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:05 smithi114 ceph-mon[30897]: pgmap v342: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-13T18:22:05.368 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:05 smithi134 ceph-mon[34283]: pgmap v342: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-13T18:22:05.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.415+0000 7fcea1875640 1 -- 172.21.15.114:0/1194893065 --> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fce9c068910 con 0x7fce840a1080 2024-09-13T18:22:05.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.420+0000 7fcea0873640 1 -- 172.21.15.114:0/1194893065 <== mgr.14148 v2:172.21.15.114:6800/3396185541 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7fce9c068910 con 0x7fce840a1080 2024-09-13T18:22:05.423 INFO:teuthology.orchestra.run.smithi114.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-13T18:22:05.423 INFO:teuthology.orchestra.run.smithi114.stdout:mds.all 12/12 64s ago 2m smithi114=a;smithi114=d;smithi114=g;smithi114=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2024-09-13T18:22:05.424 INFO:teuthology.orchestra.run.smithi114.stdout:mgr 3/3 64s ago 7m smithi114=x;smithi134=y;smithi204=z;count:3 2024-09-13T18:22:05.424 INFO:teuthology.orchestra.run.smithi114.stdout:mon 3/3 64s ago 8m smithi114:172.21.15.114=a;smithi134:172.21.15.134=b;smithi204:172.21.15.204=c;count:3 2024-09-13T18:22:05.424 INFO:teuthology.orchestra.run.smithi114.stdout:osd 12 64s ago - 2024-09-13T18:22:05.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.422+0000 7fcea1875640 1 -- 172.21.15.114:0/1194893065 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fce840a1080 msgr2=0x7fce840a3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:05.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.422+0000 7fcea1875640 1 --2- 172.21.15.114:0/1194893065 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fce840a1080 0x7fce840a3540 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fce94000f80 tx=0x7fce94009890 comp rx=0 tx=0).stop 2024-09-13T18:22:05.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.422+0000 7fcea1875640 1 -- 172.21.15.114:0/1194893065 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce9c1008d0 msgr2=0x7fce9c19c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:05.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.422+0000 7fcea1875640 1 --2- 172.21.15.114:0/1194893065 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce9c1008d0 0x7fce9c19c1e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fce8c00d960 tx=0x7fce8c00de20 comp rx=0 tx=0).stop 2024-09-13T18:22:05.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.422+0000 7fcea1875640 1 -- 172.21.15.114:0/1194893065 shutdown_connections 2024-09-13T18:22:05.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.422+0000 7fcea1875640 1 --2- 172.21.15.114:0/1194893065 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fce840a1080 0x7fce840a3540 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:05.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.422+0000 7fcea1875640 1 --2- 172.21.15.114:0/1194893065 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce9c1008d0 0x7fce9c19c1e0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:05.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.422+0000 7fcea1875640 1 --2- 172.21.15.114:0/1194893065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce9c0fff10 0x7fce9c1951a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:05.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.422+0000 7fcea1875640 1 --2- 172.21.15.114:0/1194893065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce9c0fed10 0x7fce9c194c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:05.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.422+0000 7fcea1875640 1 -- 172.21.15.114:0/1194893065 >> 172.21.15.114:0/1194893065 conn(0x7fce9c0fa4c0 msgr2=0x7fce9c103470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:05.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.422+0000 7fcea1875640 1 -- 172.21.15.114:0/1194893065 shutdown_connections 2024-09-13T18:22:05.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:05.422+0000 7fcea1875640 1 -- 172.21.15.114:0/1194893065 wait complete. 2024-09-13T18:22:05.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:05 smithi204 ceph-mon[34777]: pgmap v342: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-13T18:22:06.228 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-13T18:22:06.428 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:22:07.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:07 smithi114 ceph-mon[30897]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:22:07.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:07 smithi114 ceph-mon[30897]: pgmap v343: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-13T18:22:07.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:07 smithi134 ceph-mon[34283]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:22:07.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:07 smithi134 ceph-mon[34283]: pgmap v343: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-13T18:22:07.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:07 smithi204 ceph-mon[34777]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:22:07.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:07 smithi204 ceph-mon[34777]: pgmap v343: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-13T18:22:08.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.214+0000 7f2ee4496640 1 -- 172.21.15.114:0/52400653 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2edc0fed00 msgr2=0x7f2edc0ff100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:08.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.214+0000 7f2ee4496640 1 --2- 172.21.15.114:0/52400653 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2edc0fed00 0x7f2edc0ff100 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8009900 tx=0x7f2ed802a410 comp rx=0 tx=0).stop 2024-09-13T18:22:08.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.214+0000 7f2ee4496640 1 -- 172.21.15.114:0/52400653 shutdown_connections 2024-09-13T18:22:08.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.214+0000 7f2ee4496640 1 --2- 172.21.15.114:0/52400653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2edc1008c0 0x7f2edc107150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:08.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.214+0000 7f2ee4496640 1 --2- 172.21.15.114:0/52400653 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2edc0fff00 0x7f2edc100380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:08.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.214+0000 7f2ee4496640 1 --2- 172.21.15.114:0/52400653 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2edc0fed00 0x7f2edc0ff100 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:08.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.214+0000 7f2ee4496640 1 -- 172.21.15.114:0/52400653 >> 172.21.15.114:0/52400653 conn(0x7f2edc0fa490 msgr2=0x7f2edc0fc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:08.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.214+0000 7f2ee4496640 1 -- 172.21.15.114:0/52400653 shutdown_connections 2024-09-13T18:22:08.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.214+0000 7f2ee4496640 1 -- 172.21.15.114:0/52400653 wait complete. 2024-09-13T18:22:08.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.214+0000 7f2ee4496640 1 Processor -- start 2024-09-13T18:22:08.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.214+0000 7f2ee4496640 1 -- start start 2024-09-13T18:22:08.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.215+0000 7f2ee4496640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2edc0fed00 0x7f2edc195d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:08.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.215+0000 7f2ee4496640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2edc0fff00 0x7f2edc196260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:08.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.215+0000 7f2ee4496640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2edc1008c0 0x7f2edc19d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:08.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.215+0000 7f2ee4496640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2edc19d810 con 0x7f2edc1008c0 2024-09-13T18:22:08.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.215+0000 7f2ee4496640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2edc19d980 con 0x7f2edc0fff00 2024-09-13T18:22:08.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.215+0000 7f2ee4496640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2edc19db40 con 0x7f2edc0fed00 2024-09-13T18:22:08.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.215+0000 7f2ee2a33640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2edc1008c0 0x7f2edc19d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:08.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.215+0000 7f2ee2a33640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2edc1008c0 0x7f2edc19d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44018/0 (socket says 172.21.15.114:44018) 2024-09-13T18:22:08.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.215+0000 7f2ee2a33640 1 -- 172.21.15.114:0/1106530735 learned_addr learned my addr 172.21.15.114:0/1106530735 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:08.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.215+0000 7f2ee2232640 1 --2- 172.21.15.114:0/1106530735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2edc0fed00 0x7f2edc195d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:08.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.215+0000 7f2ee1a31640 1 --2- 172.21.15.114:0/1106530735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2edc0fff00 0x7f2edc196260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:08.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.215+0000 7f2ee2a33640 1 -- 172.21.15.114:0/1106530735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2edc0fed00 msgr2=0x7f2edc195d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:08.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.215+0000 7f2ee2a33640 1 --2- 172.21.15.114:0/1106530735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2edc0fed00 0x7f2edc195d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:08.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.215+0000 7f2ee2a33640 1 -- 172.21.15.114:0/1106530735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2edc0fff00 msgr2=0x7f2edc196260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:08.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.215+0000 7f2ee2a33640 1 --2- 172.21.15.114:0/1106530735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2edc0fff00 0x7f2edc196260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:08.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.215+0000 7f2ee2a33640 1 -- 172.21.15.114:0/1106530735 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed8009560 con 0x7f2edc1008c0 2024-09-13T18:22:08.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.215+0000 7f2ee1a31640 1 --2- 172.21.15.114:0/1106530735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2edc0fff00 0x7f2edc196260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:22:08.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.215+0000 7f2ee2a33640 1 --2- 172.21.15.114:0/1106530735 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2edc1008c0 0x7f2edc19d2a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2ed400eaa0 tx=0x7f2ed400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:08.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.215+0000 7f2ed37fe640 1 -- 172.21.15.114:0/1106530735 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed400cdb0 con 0x7f2edc1008c0 2024-09-13T18:22:08.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.215+0000 7f2ed37fe640 1 -- 172.21.15.114:0/1106530735 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2ed4005020 con 0x7f2edc1008c0 2024-09-13T18:22:08.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.215+0000 7f2ed37fe640 1 -- 172.21.15.114:0/1106530735 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed40116d0 con 0x7f2edc1008c0 2024-09-13T18:22:08.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.215+0000 7f2ee4496640 1 -- 172.21.15.114:0/1106530735 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2edc19de20 con 0x7f2edc1008c0 2024-09-13T18:22:08.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.216+0000 7f2ee4496640 1 -- 172.21.15.114:0/1106530735 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2edc19e1e0 con 0x7f2edc1008c0 2024-09-13T18:22:08.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.216+0000 7f2ee4496640 1 -- 172.21.15.114:0/1106530735 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2edc0ff170 con 0x7f2edc1008c0 2024-09-13T18:22:08.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.217+0000 7f2ed37fe640 1 -- 172.21.15.114:0/1106530735 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f2ed4005190 con 0x7f2edc1008c0 2024-09-13T18:22:08.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.218+0000 7f2ed37fe640 1 --2- 172.21.15.114:0/1106530735 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f2ec80a1080 0x7f2ec80a3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:08.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.218+0000 7f2ee2232640 1 --2- 172.21.15.114:0/1106530735 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f2ec80a1080 0x7f2ec80a3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:08.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.218+0000 7f2ed37fe640 1 -- 172.21.15.114:0/1106530735 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f2ed40ccf20 con 0x7f2edc1008c0 2024-09-13T18:22:08.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.219+0000 7f2ee2232640 1 --2- 172.21.15.114:0/1106530735 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f2ec80a1080 0x7f2ec80a3540 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2ed802aa00 tx=0x7f2ed8005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:08.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.222+0000 7f2ed37fe640 1 -- 172.21.15.114:0/1106530735 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f2edc0ff170 con 0x7f2edc1008c0 2024-09-13T18:22:08.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.391+0000 7f2ee4496640 1 -- 172.21.15.114:0/1106530735 --> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f2edc0ff100 con 0x7f2ec80a1080 2024-09-13T18:22:08.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.392+0000 7f2ed37fe640 1 -- 172.21.15.114:0/1106530735 <== mgr.14148 v2:172.21.15.114:6800/3396185541 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f2edc0ff100 con 0x7f2ec80a1080 2024-09-13T18:22:08.394 INFO:teuthology.orchestra.run.smithi114.stdout:HOST ADDR LABELS STATUS 2024-09-13T18:22:08.394 INFO:teuthology.orchestra.run.smithi114.stdout:smithi114 172.21.15.114 2024-09-13T18:22:08.395 INFO:teuthology.orchestra.run.smithi114.stdout:smithi134 172.21.15.134 2024-09-13T18:22:08.395 INFO:teuthology.orchestra.run.smithi114.stdout:smithi204 172.21.15.204 2024-09-13T18:22:08.395 INFO:teuthology.orchestra.run.smithi114.stdout:3 hosts in cluster 2024-09-13T18:22:08.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.394+0000 7f2ee4496640 1 -- 172.21.15.114:0/1106530735 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f2ec80a1080 msgr2=0x7f2ec80a3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:08.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.394+0000 7f2ee4496640 1 --2- 172.21.15.114:0/1106530735 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f2ec80a1080 0x7f2ec80a3540 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2ed802aa00 tx=0x7f2ed8005a20 comp rx=0 tx=0).stop 2024-09-13T18:22:08.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.394+0000 7f2ee4496640 1 -- 172.21.15.114:0/1106530735 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2edc1008c0 msgr2=0x7f2edc19d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:08.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.394+0000 7f2ee4496640 1 --2- 172.21.15.114:0/1106530735 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2edc1008c0 0x7f2edc19d2a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2ed400eaa0 tx=0x7f2ed400ef60 comp rx=0 tx=0).stop 2024-09-13T18:22:08.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.394+0000 7f2ee4496640 1 -- 172.21.15.114:0/1106530735 shutdown_connections 2024-09-13T18:22:08.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.394+0000 7f2ee4496640 1 --2- 172.21.15.114:0/1106530735 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f2ec80a1080 0x7f2ec80a3540 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:08.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.394+0000 7f2ee4496640 1 --2- 172.21.15.114:0/1106530735 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2edc1008c0 0x7f2edc19d2a0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:08.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.394+0000 7f2ee4496640 1 --2- 172.21.15.114:0/1106530735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2edc0fff00 0x7f2edc196260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:08.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.394+0000 7f2ee4496640 1 --2- 172.21.15.114:0/1106530735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2edc0fed00 0x7f2edc195d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:08.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.394+0000 7f2ee4496640 1 -- 172.21.15.114:0/1106530735 >> 172.21.15.114:0/1106530735 conn(0x7f2edc0fa490 msgr2=0x7f2edc103360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:08.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.394+0000 7f2ee4496640 1 -- 172.21.15.114:0/1106530735 shutdown_connections 2024-09-13T18:22:08.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:08.394+0000 7f2ee4496640 1 -- 172.21.15.114:0/1106530735 wait complete. 2024-09-13T18:22:09.175 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:09 smithi114 ceph-mon[30897]: pgmap v344: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-13T18:22:09.197 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-13T18:22:09.368 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:09 smithi134 ceph-mon[34283]: pgmap v344: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-13T18:22:09.396 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:22:09.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:09 smithi204 ceph-mon[34777]: pgmap v344: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-13T18:22:10.215 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:10 smithi114 ceph-mon[30897]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:22:10.368 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:10 smithi134 ceph-mon[34283]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:22:10.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:10 smithi204 ceph-mon[34777]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:22:11.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.186+0000 7f89461f3640 1 -- 172.21.15.114:0/1770420376 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8940100f60 msgr2=0x7f89401013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:11.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.186+0000 7f89461f3640 1 --2- 172.21.15.114:0/1770420376 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8940100f60 0x7f89401013e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8934009900 tx=0x7f893402a410 comp rx=0 tx=0).stop 2024-09-13T18:22:11.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.186+0000 7f89461f3640 1 -- 172.21.15.114:0/1770420376 shutdown_connections 2024-09-13T18:22:11.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.186+0000 7f89461f3640 1 --2- 172.21.15.114:0/1770420376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8940101920 0x7f8940108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:11.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.186+0000 7f89461f3640 1 --2- 172.21.15.114:0/1770420376 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8940100f60 0x7f89401013e0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:11.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.186+0000 7f89461f3640 1 --2- 172.21.15.114:0/1770420376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89400ffe00 0x7f8940100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:11.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.186+0000 7f89461f3640 1 -- 172.21.15.114:0/1770420376 >> 172.21.15.114:0/1770420376 conn(0x7f89400fb5d0 msgr2=0x7f89400fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:11.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.186+0000 7f89461f3640 1 -- 172.21.15.114:0/1770420376 shutdown_connections 2024-09-13T18:22:11.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.186+0000 7f89461f3640 1 -- 172.21.15.114:0/1770420376 wait complete. 2024-09-13T18:22:11.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.187+0000 7f89461f3640 1 Processor -- start 2024-09-13T18:22:11.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.187+0000 7f89461f3640 1 -- start start 2024-09-13T18:22:11.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.187+0000 7f89461f3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89400ffe00 0x7f8940074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:11.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.187+0000 7f89461f3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8940100f60 0x7f89400728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:11.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.187+0000 7f89461f3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8940101920 0x7f8940072de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:11.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.187+0000 7f89461f3640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8940073570 con 0x7f89400ffe00 2024-09-13T18:22:11.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.187+0000 7f89461f3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89400736e0 con 0x7f8940100f60 2024-09-13T18:22:11.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.187+0000 7f89461f3640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f894019fcb0 con 0x7f8940101920 2024-09-13T18:22:11.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.187+0000 7f893f7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89400ffe00 0x7f8940074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:11.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.187+0000 7f893f7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89400ffe00 0x7f8940074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44034/0 (socket says 172.21.15.114:44034) 2024-09-13T18:22:11.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.187+0000 7f893f7fe640 1 -- 172.21.15.114:0/2218644805 learned_addr learned my addr 172.21.15.114:0/2218644805 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:11.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.187+0000 7f893ffff640 1 --2- 172.21.15.114:0/2218644805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8940101920 0x7f8940072de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:11.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.187+0000 7f893effd640 1 --2- 172.21.15.114:0/2218644805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8940100f60 0x7f89400728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:11.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.188+0000 7f893f7fe640 1 -- 172.21.15.114:0/2218644805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8940101920 msgr2=0x7f8940072de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:11.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.188+0000 7f893f7fe640 1 --2- 172.21.15.114:0/2218644805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8940101920 0x7f8940072de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:11.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.188+0000 7f893f7fe640 1 -- 172.21.15.114:0/2218644805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8940100f60 msgr2=0x7f89400728a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:11.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.188+0000 7f893f7fe640 1 --2- 172.21.15.114:0/2218644805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8940100f60 0x7f89400728a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:11.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.188+0000 7f893f7fe640 1 -- 172.21.15.114:0/2218644805 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8934009560 con 0x7f89400ffe00 2024-09-13T18:22:11.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.188+0000 7f893effd640 1 --2- 172.21.15.114:0/2218644805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8940100f60 0x7f89400728a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:22:11.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.188+0000 7f893f7fe640 1 --2- 172.21.15.114:0/2218644805 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89400ffe00 0x7f8940074250 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f893000c8f0 tx=0x7f893000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:11.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.188+0000 7f893cff9640 1 -- 172.21.15.114:0/2218644805 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8930004220 con 0x7f89400ffe00 2024-09-13T18:22:11.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.188+0000 7f893cff9640 1 -- 172.21.15.114:0/2218644805 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8930007e40 con 0x7f89400ffe00 2024-09-13T18:22:11.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.188+0000 7f89461f3640 1 -- 172.21.15.114:0/2218644805 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f894019ff60 con 0x7f89400ffe00 2024-09-13T18:22:11.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.188+0000 7f893cff9640 1 -- 172.21.15.114:0/2218644805 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89300047a0 con 0x7f89400ffe00 2024-09-13T18:22:11.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.188+0000 7f89461f3640 1 -- 172.21.15.114:0/2218644805 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89401a0370 con 0x7f89400ffe00 2024-09-13T18:22:11.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.189+0000 7f89461f3640 1 -- 172.21.15.114:0/2218644805 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8940107260 con 0x7f89400ffe00 2024-09-13T18:22:11.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.190+0000 7f893cff9640 1 -- 172.21.15.114:0/2218644805 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f89300079b0 con 0x7f89400ffe00 2024-09-13T18:22:11.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.191+0000 7f893cff9640 1 --2- 172.21.15.114:0/2218644805 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f89280a1080 0x7f89280a3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:11.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.191+0000 7f893cff9640 1 -- 172.21.15.114:0/2218644805 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f89300cc970 con 0x7f89400ffe00 2024-09-13T18:22:11.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.191+0000 7f893effd640 1 --2- 172.21.15.114:0/2218644805 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f89280a1080 0x7f89280a3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:11.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.191+0000 7f893effd640 1 --2- 172.21.15.114:0/2218644805 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f89280a1080 0x7f89280a3540 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8934006010 tx=0x7f8934031470 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:11.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.195+0000 7f893cff9640 1 -- 172.21.15.114:0/2218644805 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f8930095d90 con 0x7f89400ffe00 2024-09-13T18:22:11.219 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:11 smithi114 ceph-mon[30897]: pgmap v345: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-13T18:22:11.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:11 smithi134 ceph-mon[34283]: pgmap v345: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-13T18:22:11.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.368+0000 7f89461f3640 1 -- 172.21.15.114:0/2218644805 --> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f894002c520 con 0x7f89280a1080 2024-09-13T18:22:11.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.370+0000 7f893cff9640 1 -- 172.21.15.114:0/2218644805 <== mgr.14148 v2:172.21.15.114:6800/3396185541 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7f894002c520 con 0x7f89280a1080 2024-09-13T18:22:11.372 INFO:teuthology.orchestra.run.smithi114.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-13T18:22:11.372 INFO:teuthology.orchestra.run.smithi114.stdout:smithi114 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400WB400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-13T18:22:11.372 INFO:teuthology.orchestra.run.smithi114.stdout:smithi134 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT62420002400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-13T18:22:11.373 INFO:teuthology.orchestra.run.smithi114.stdout:smithi204 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GD400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-13T18:22:11.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.372+0000 7f89461f3640 1 -- 172.21.15.114:0/2218644805 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f89280a1080 msgr2=0x7f89280a3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:11.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.372+0000 7f89461f3640 1 --2- 172.21.15.114:0/2218644805 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f89280a1080 0x7f89280a3540 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8934006010 tx=0x7f8934031470 comp rx=0 tx=0).stop 2024-09-13T18:22:11.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.372+0000 7f89461f3640 1 -- 172.21.15.114:0/2218644805 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89400ffe00 msgr2=0x7f8940074250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:11.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.372+0000 7f89461f3640 1 --2- 172.21.15.114:0/2218644805 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89400ffe00 0x7f8940074250 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f893000c8f0 tx=0x7f893000cdb0 comp rx=0 tx=0).stop 2024-09-13T18:22:11.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.372+0000 7f89461f3640 1 -- 172.21.15.114:0/2218644805 shutdown_connections 2024-09-13T18:22:11.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.372+0000 7f89461f3640 1 --2- 172.21.15.114:0/2218644805 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f89280a1080 0x7f89280a3540 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:11.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.372+0000 7f89461f3640 1 --2- 172.21.15.114:0/2218644805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8940101920 0x7f8940072de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:11.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.372+0000 7f89461f3640 1 --2- 172.21.15.114:0/2218644805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8940100f60 0x7f89400728a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:11.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.372+0000 7f89461f3640 1 --2- 172.21.15.114:0/2218644805 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89400ffe00 0x7f8940074250 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:11.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.372+0000 7f89461f3640 1 -- 172.21.15.114:0/2218644805 >> 172.21.15.114:0/2218644805 conn(0x7f89400fb5d0 msgr2=0x7f8940104430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:11.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.372+0000 7f89461f3640 1 -- 172.21.15.114:0/2218644805 shutdown_connections 2024-09-13T18:22:11.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:11.372+0000 7f89461f3640 1 -- 172.21.15.114:0/2218644805 wait complete. 2024-09-13T18:22:11.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:11 smithi204 ceph-mon[34777]: pgmap v345: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-13T18:22:12.149 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-13T18:22:12.158 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi114.front.sepia.ceph.com 2024-09-13T18:22:12.158 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-13T18:22:12.357 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:22:13.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:13 smithi134 ceph-mon[34283]: from='client.15351 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:22:13.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:13 smithi134 ceph-mon[34283]: pgmap v346: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-13T18:22:13.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:13 smithi204 ceph-mon[34777]: from='client.15351 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:22:13.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:13 smithi204 ceph-mon[34777]: pgmap v346: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-13T18:22:13.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:13 smithi114 ceph-mon[30897]: from='client.15351 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:22:13.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:13 smithi114 ceph-mon[30897]: pgmap v346: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-13T18:22:14.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.206+0000 7f73b08e2640 1 -- 172.21.15.114:0/598248748 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73ac0ffee0 msgr2=0x7f73ac100360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:14.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.206+0000 7f73b08e2640 1 --2- 172.21.15.114:0/598248748 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73ac0ffee0 0x7f73ac100360 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7394009900 tx=0x7f739402a410 comp rx=0 tx=0).stop 2024-09-13T18:22:14.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.206+0000 7f73b08e2640 1 -- 172.21.15.114:0/598248748 shutdown_connections 2024-09-13T18:22:14.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.206+0000 7f73b08e2640 1 --2- 172.21.15.114:0/598248748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73ac1008a0 0x7f73ac107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:14.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.206+0000 7f73b08e2640 1 --2- 172.21.15.114:0/598248748 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73ac0ffee0 0x7f73ac100360 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:14.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.206+0000 7f73b08e2640 1 --2- 172.21.15.114:0/598248748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73ac0fece0 0x7f73ac0ff0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:14.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.206+0000 7f73b08e2640 1 -- 172.21.15.114:0/598248748 >> 172.21.15.114:0/598248748 conn(0x7f73ac0fa490 msgr2=0x7f73ac0fc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:14.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.206+0000 7f73b08e2640 1 -- 172.21.15.114:0/598248748 shutdown_connections 2024-09-13T18:22:14.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.206+0000 7f73b08e2640 1 -- 172.21.15.114:0/598248748 wait complete. 2024-09-13T18:22:14.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.207+0000 7f73b08e2640 1 Processor -- start 2024-09-13T18:22:14.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.207+0000 7f73b08e2640 1 -- start start 2024-09-13T18:22:14.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.207+0000 7f73b08e2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73ac0fece0 0x7f73ac195ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:14.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.207+0000 7f73b08e2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73ac0ffee0 0x7f73ac196220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:14.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.207+0000 7f73b08e2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73ac1008a0 0x7f73ac19d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:14.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.207+0000 7f73b08e2640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac19d780 con 0x7f73ac1008a0 2024-09-13T18:22:14.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.207+0000 7f73b08e2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac19d8f0 con 0x7f73ac0ffee0 2024-09-13T18:22:14.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.207+0000 7f73b08e2640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac19dab0 con 0x7f73ac0fece0 2024-09-13T18:22:14.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.207+0000 7f73aad9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73ac1008a0 0x7f73ac19d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:14.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.207+0000 7f73aad9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73ac1008a0 0x7f73ac19d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43096/0 (socket says 172.21.15.114:43096) 2024-09-13T18:22:14.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.207+0000 7f73aad9d640 1 -- 172.21.15.114:0/2592632594 learned_addr learned my addr 172.21.15.114:0/2592632594 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:14.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.207+0000 7f73a9d9b640 1 --2- 172.21.15.114:0/2592632594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73ac0ffee0 0x7f73ac196220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:14.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.207+0000 7f73aa59c640 1 --2- 172.21.15.114:0/2592632594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73ac0fece0 0x7f73ac195ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:14.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.207+0000 7f73aad9d640 1 -- 172.21.15.114:0/2592632594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73ac0fece0 msgr2=0x7f73ac195ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:14.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.207+0000 7f73aad9d640 1 --2- 172.21.15.114:0/2592632594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73ac0fece0 0x7f73ac195ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:14.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.207+0000 7f73aad9d640 1 -- 172.21.15.114:0/2592632594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73ac0ffee0 msgr2=0x7f73ac196220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:14.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.207+0000 7f73aad9d640 1 --2- 172.21.15.114:0/2592632594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73ac0ffee0 0x7f73ac196220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:14.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.207+0000 7f73aad9d640 1 -- 172.21.15.114:0/2592632594 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7394009560 con 0x7f73ac1008a0 2024-09-13T18:22:14.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.207+0000 7f73a9d9b640 1 --2- 172.21.15.114:0/2592632594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73ac0ffee0 0x7f73ac196220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:22:14.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.208+0000 7f73aad9d640 1 --2- 172.21.15.114:0/2592632594 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73ac1008a0 0x7f73ac19d240 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f73a400eaa0 tx=0x7f73a400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:14.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.208+0000 7f73a37fe640 1 -- 172.21.15.114:0/2592632594 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a400cdb0 con 0x7f73ac1008a0 2024-09-13T18:22:14.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.208+0000 7f73b08e2640 1 -- 172.21.15.114:0/2592632594 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73ac19dd90 con 0x7f73ac1008a0 2024-09-13T18:22:14.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.208+0000 7f73a37fe640 1 -- 172.21.15.114:0/2592632594 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f73a4005020 con 0x7f73ac1008a0 2024-09-13T18:22:14.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.208+0000 7f73a37fe640 1 -- 172.21.15.114:0/2592632594 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a40116d0 con 0x7f73ac1008a0 2024-09-13T18:22:14.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.208+0000 7f73b08e2640 1 -- 172.21.15.114:0/2592632594 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73ac19e200 con 0x7f73ac1008a0 2024-09-13T18:22:14.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.209+0000 7f73b08e2640 1 -- 172.21.15.114:0/2592632594 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73ac1061a0 con 0x7f73ac1008a0 2024-09-13T18:22:14.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.211+0000 7f73a37fe640 1 -- 172.21.15.114:0/2592632594 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f73a4005190 con 0x7f73ac1008a0 2024-09-13T18:22:14.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.212+0000 7f73a37fe640 1 --2- 172.21.15.114:0/2592632594 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f73980a1080 0x7f73980a3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:14.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.212+0000 7f73aa59c640 1 --2- 172.21.15.114:0/2592632594 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f73980a1080 0x7f73980a3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:14.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.212+0000 7f73a37fe640 1 -- 172.21.15.114:0/2592632594 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f73a40ccd50 con 0x7f73ac1008a0 2024-09-13T18:22:14.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.212+0000 7f73aa59c640 1 --2- 172.21.15.114:0/2592632594 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f73980a1080 0x7f73980a3540 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f73ac0fdbc0 tx=0x7f739c00b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:14.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.220+0000 7f73a37fe640 1 -- 172.21.15.114:0/2592632594 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f73a40d0030 con 0x7f73ac1008a0 2024-09-13T18:22:14.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.422+0000 7f73b08e2640 1 -- 172.21.15.114:0/2592632594 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f73ac10b590 con 0x7f73ac1008a0 2024-09-13T18:22:14.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.423+0000 7f73a37fe640 1 -- 172.21.15.114:0/2592632594 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+2937 (secure 0 0 0) 0x7f73a401f020 con 0x7f73ac1008a0 2024-09-13T18:22:14.425 INFO:teuthology.orchestra.run.smithi114.stdout:e29 2024-09-13T18:22:14.426 INFO:teuthology.orchestra.run.smithi114.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-13T18:22:14.426 INFO:teuthology.orchestra.run.smithi114.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-13T18:22:14.426 INFO:teuthology.orchestra.run.smithi114.stdout:legacy client fscid: 1 2024-09-13T18:22:14.426 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:22:14.426 INFO:teuthology.orchestra.run.smithi114.stdout:Filesystem 'cephfs' (1) 2024-09-13T18:22:14.426 INFO:teuthology.orchestra.run.smithi114.stdout:fs_name cephfs 2024-09-13T18:22:14.426 INFO:teuthology.orchestra.run.smithi114.stdout:epoch 29 2024-09-13T18:22:14.426 INFO:teuthology.orchestra.run.smithi114.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-13T18:22:14.426 INFO:teuthology.orchestra.run.smithi114.stdout:created 2024-09-13T18:20:01.895117+0000 2024-09-13T18:22:14.426 INFO:teuthology.orchestra.run.smithi114.stdout:modified 2024-09-13T18:20:29.250121+0000 2024-09-13T18:22:14.426 INFO:teuthology.orchestra.run.smithi114.stdout:tableserver 0 2024-09-13T18:22:14.426 INFO:teuthology.orchestra.run.smithi114.stdout:root 0 2024-09-13T18:22:14.426 INFO:teuthology.orchestra.run.smithi114.stdout:session_timeout 300 2024-09-13T18:22:14.427 INFO:teuthology.orchestra.run.smithi114.stdout:session_autoclose 300 2024-09-13T18:22:14.427 INFO:teuthology.orchestra.run.smithi114.stdout:max_file_size 1099511627776 2024-09-13T18:22:14.427 INFO:teuthology.orchestra.run.smithi114.stdout:max_xattr_size 65536 2024-09-13T18:22:14.427 INFO:teuthology.orchestra.run.smithi114.stdout:required_client_features {} 2024-09-13T18:22:14.427 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure 0 2024-09-13T18:22:14.427 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure_osd_epoch 0 2024-09-13T18:22:14.427 INFO:teuthology.orchestra.run.smithi114.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-13T18:22:14.427 INFO:teuthology.orchestra.run.smithi114.stdout:max_mds 5 2024-09-13T18:22:14.427 INFO:teuthology.orchestra.run.smithi114.stdout:in 0,1,2,3,4 2024-09-13T18:22:14.427 INFO:teuthology.orchestra.run.smithi114.stdout:up {0=24521,1=24482,2=24475,3=24458,4=14628} 2024-09-13T18:22:14.427 INFO:teuthology.orchestra.run.smithi114.stdout:failed 2024-09-13T18:22:14.427 INFO:teuthology.orchestra.run.smithi114.stdout:damaged 2024-09-13T18:22:14.427 INFO:teuthology.orchestra.run.smithi114.stdout:stopped 2024-09-13T18:22:14.427 INFO:teuthology.orchestra.run.smithi114.stdout:data_pools [3,4] 2024-09-13T18:22:14.428 INFO:teuthology.orchestra.run.smithi114.stdout:metadata_pool 2 2024-09-13T18:22:14.428 INFO:teuthology.orchestra.run.smithi114.stdout:inline_data disabled 2024-09-13T18:22:14.428 INFO:teuthology.orchestra.run.smithi114.stdout:balancer 2024-09-13T18:22:14.428 INFO:teuthology.orchestra.run.smithi114.stdout:bal_rank_mask -1 2024-09-13T18:22:14.428 INFO:teuthology.orchestra.run.smithi114.stdout:standby_count_wanted 0 2024-09-13T18:22:14.428 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.b{0:24521} state up:active seq 3 addr [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] compat {c=[1],r=[1],i=[7ff]}] 2024-09-13T18:22:14.428 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.c{0:24457} state up:standby-replay seq 1 addr [v2:172.21.15.204:6836/3068880939,v1:172.21.15.204:6837/3068880939] compat {c=[1],r=[1],i=[7ff]}] 2024-09-13T18:22:14.428 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.h{1:24482} state up:active seq 11 addr [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] compat {c=[1],r=[1],i=[7ff]}] 2024-09-13T18:22:14.428 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.f{1:24445} state up:standby-replay seq 1 addr [v2:172.21.15.204:6834/57029119,v1:172.21.15.204:6835/57029119] compat {c=[1],r=[1],i=[7ff]}] 2024-09-13T18:22:14.428 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.i{2:24475} state up:active seq 11 addr [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] compat {c=[1],r=[1],i=[7ff]}] 2024-09-13T18:22:14.428 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.k{2:24422} state up:standby-replay seq 1 addr [v2:172.21.15.134:6832/4028657219,v1:172.21.15.134:6833/4028657219] compat {c=[1],r=[1],i=[7ff]}] 2024-09-13T18:22:14.428 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.e{3:24458} state up:active seq 14 addr [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] compat {c=[1],r=[1],i=[7ff]}] 2024-09-13T18:22:14.428 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.l{3:24391} state up:standby-replay seq 1 addr [v2:172.21.15.204:6832/507301496,v1:172.21.15.204:6833/507301496] compat {c=[1],r=[1],i=[7ff]}] 2024-09-13T18:22:14.428 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.a{4:14628} state up:active seq 10 addr [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] compat {c=[1],r=[1],i=[7ff]}] 2024-09-13T18:22:14.429 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.g{4:14616} state up:standby-replay seq 1 addr [v2:172.21.15.114:6838/2046516451,v1:172.21.15.114:6839/2046516451] compat {c=[1],r=[1],i=[7ff]}] 2024-09-13T18:22:14.429 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:22:14.429 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:22:14.429 INFO:teuthology.orchestra.run.smithi114.stdout:Standby daemons: 2024-09-13T18:22:14.429 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:22:14.429 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.j{-1:14538} state up:standby seq 1 addr [v2:172.21.15.114:6834/4162817497,v1:172.21.15.114:6835/4162817497] compat {c=[1],r=[1],i=[7ff]}] 2024-09-13T18:22:14.429 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.d{-1:14550} state up:standby seq 1 addr [v2:172.21.15.114:6836/2435428478,v1:172.21.15.114:6837/2435428478] compat {c=[1],r=[1],i=[7ff]}] 2024-09-13T18:22:14.430 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 29 2024-09-13T18:22:14.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.425+0000 7f73b08e2640 1 -- 172.21.15.114:0/2592632594 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f73980a1080 msgr2=0x7f73980a3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:14.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.425+0000 7f73b08e2640 1 --2- 172.21.15.114:0/2592632594 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f73980a1080 0x7f73980a3540 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f73ac0fdbc0 tx=0x7f739c00b390 comp rx=0 tx=0).stop 2024-09-13T18:22:14.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.425+0000 7f73b08e2640 1 -- 172.21.15.114:0/2592632594 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73ac1008a0 msgr2=0x7f73ac19d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:14.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.425+0000 7f73b08e2640 1 --2- 172.21.15.114:0/2592632594 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73ac1008a0 0x7f73ac19d240 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f73a400eaa0 tx=0x7f73a400ef60 comp rx=0 tx=0).stop 2024-09-13T18:22:14.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.425+0000 7f73b08e2640 1 -- 172.21.15.114:0/2592632594 shutdown_connections 2024-09-13T18:22:14.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.425+0000 7f73b08e2640 1 --2- 172.21.15.114:0/2592632594 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f73980a1080 0x7f73980a3540 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:14.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.425+0000 7f73b08e2640 1 --2- 172.21.15.114:0/2592632594 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73ac1008a0 0x7f73ac19d240 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:14.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.425+0000 7f73b08e2640 1 --2- 172.21.15.114:0/2592632594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73ac0ffee0 0x7f73ac196220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:14.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.425+0000 7f73b08e2640 1 --2- 172.21.15.114:0/2592632594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73ac0fece0 0x7f73ac195ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:14.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.425+0000 7f73b08e2640 1 -- 172.21.15.114:0/2592632594 >> 172.21.15.114:0/2592632594 conn(0x7f73ac0fa490 msgr2=0x7f73ac103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:14.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.425+0000 7f73b08e2640 1 -- 172.21.15.114:0/2592632594 shutdown_connections 2024-09-13T18:22:14.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:14.425+0000 7f73b08e2640 1 -- 172.21.15.114:0/2592632594 wait complete. 2024-09-13T18:22:15.150 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-13T18:22:15.356 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:22:15.368 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:15 smithi134 ceph-mon[34283]: pgmap v347: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-13T18:22:15.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:15 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/2592632594' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-13T18:22:15.401 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:15 smithi114 ceph-mon[30897]: pgmap v347: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-13T18:22:15.401 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:15 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2592632594' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-13T18:22:15.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:15 smithi204 ceph-mon[34777]: pgmap v347: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-13T18:22:15.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:15 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/2592632594' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-13T18:22:17.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:17.069+0000 7fc8d76d5640 1 -- 172.21.15.114:0/2262889958 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8d00ffee0 msgr2=0x7fc8d0100360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:17.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:17.069+0000 7fc8d76d5640 1 --2- 172.21.15.114:0/2262889958 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8d00ffee0 0x7fc8d0100360 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc8c4009900 tx=0x7fc8c402a410 comp rx=0 tx=0).stop 2024-09-13T18:22:17.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:17.069+0000 7fc8d76d5640 1 -- 172.21.15.114:0/2262889958 shutdown_connections 2024-09-13T18:22:17.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:17.069+0000 7fc8d76d5640 1 --2- 172.21.15.114:0/2262889958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8d01008a0 0x7fc8d0107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:17.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:17.069+0000 7fc8d76d5640 1 --2- 172.21.15.114:0/2262889958 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8d00ffee0 0x7fc8d0100360 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:17.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:17.069+0000 7fc8d76d5640 1 --2- 172.21.15.114:0/2262889958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8d00fece0 0x7fc8d00ff0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:17.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:17.069+0000 7fc8d76d5640 1 -- 172.21.15.114:0/2262889958 >> 172.21.15.114:0/2262889958 conn(0x7fc8d00fa490 msgr2=0x7fc8d00fc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:17.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:17.069+0000 7fc8d76d5640 1 -- 172.21.15.114:0/2262889958 shutdown_connections 2024-09-13T18:22:17.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:17.069+0000 7fc8d76d5640 1 -- 172.21.15.114:0/2262889958 wait complete. 2024-09-13T18:22:17.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:17.069+0000 7fc8d76d5640 1 Processor -- start 2024-09-13T18:22:17.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:17.069+0000 7fc8d76d5640 1 -- start start 2024-09-13T18:22:17.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:17.070+0000 7fc8d76d5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8d00fece0 0x7fc8d0068a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:17.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:17.070+0000 7fc8d76d5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8d00ffee0 0x7fc8d019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:17.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:17.070+0000 7fc8d76d5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8d01008a0 0x7fc8d019cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:17.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:17.070+0000 7fc8d76d5640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d019d740 con 0x7fc8d01008a0 2024-09-13T18:22:17.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:17.070+0000 7fc8d76d5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d019d8b0 con 0x7fc8d00ffee0 2024-09-13T18:22:17.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:17.070+0000 7fc8d76d5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d019da70 con 0x7fc8d00fece0 2024-09-13T18:22:17.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:17.070+0000 7fc8d5c72640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8d01008a0 0x7fc8d019cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:17.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:17.070+0000 7fc8d5c72640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8d01008a0 0x7fc8d019cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43118/0 (socket says 172.21.15.114:43118) 2024-09-13T18:22:17.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:17.070+0000 7fc8d5471640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8d00fece0 0x7fc8d0068a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:17.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:17.070+0000 7fc8d5c72640 1 -- 172.21.15.114:0/585969433 learned_addr learned my addr 172.21.15.114:0/585969433 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:17.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:17.070+0000 7fc8d4c70640 1 --2- 172.21.15.114:0/585969433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8d00ffee0 0x7fc8d019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:17.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:17.070+0000 7fc8d5c72640 1 -- 172.21.15.114:0/585969433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8d00fece0 msgr2=0x7fc8d0068a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:17.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:17.070+0000 7fc8d5c72640 1 --2- 172.21.15.114:0/585969433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8d00fece0 0x7fc8d0068a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:17.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:17.070+0000 7fc8d5c72640 1 -- 172.21.15.114:0/585969433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8d00ffee0 msgr2=0x7fc8d019aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:17.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:17.070+0000 7fc8d5c72640 1 --2- 172.21.15.114:0/585969433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8d00ffee0 0x7fc8d019aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:17.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:17.070+0000 7fc8d5c72640 1 -- 172.21.15.114:0/585969433 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8c4009560 con 0x7fc8d01008a0 2024-09-13T18:22:17.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:17.070+0000 7fc8d5c72640 1 --2- 172.21.15.114:0/585969433 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8d01008a0 0x7fc8d019cfb0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc8c800c970 tx=0x7fc8c800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:17.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:17.070+0000 7fc8ce7fc640 1 -- 172.21.15.114:0/585969433 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c8007bb0 con 0x7fc8d01008a0 2024-09-13T18:22:17.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:17.071+0000 7fc8ce7fc640 1 -- 172.21.15.114:0/585969433 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc8c8007d10 con 0x7fc8d01008a0 2024-09-13T18:22:17.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:17.071+0000 7fc8d76d5640 1 -- 172.21.15.114:0/585969433 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8d019dd50 con 0x7fc8d01008a0 2024-09-13T18:22:17.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:17.071+0000 7fc8ce7fc640 1 -- 172.21.15.114:0/585969433 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c8018660 con 0x7fc8d01008a0 2024-09-13T18:22:17.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:17.071+0000 7fc8d76d5640 1 -- 172.21.15.114:0/585969433 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8d019e170 con 0x7fc8d01008a0 2024-09-13T18:22:17.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:17.072+0000 7fc8d76d5640 1 -- 172.21.15.114:0/585969433 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8d01061a0 con 0x7fc8d01008a0 2024-09-13T18:22:17.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:17.073+0000 7fc8ce7fc640 1 -- 172.21.15.114:0/585969433 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fc8c800f450 con 0x7fc8d01008a0 2024-09-13T18:22:17.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:17.073+0000 7fc8ce7fc640 1 --2- 172.21.15.114:0/585969433 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fc8b80a1080 0x7fc8b80a3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:17.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:17.073+0000 7fc8d5471640 1 --2- 172.21.15.114:0/585969433 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fc8b80a1080 0x7fc8b80a3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:17.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:17.073+0000 7fc8ce7fc640 1 -- 172.21.15.114:0/585969433 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fc8c80ccc80 con 0x7fc8d01008a0 2024-09-13T18:22:17.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:17.074+0000 7fc8d5471640 1 --2- 172.21.15.114:0/585969433 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fc8b80a1080 0x7fc8b80a3540 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc8d00ffd40 tx=0x7fc8c0009670 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:17.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:17.077+0000 7fc8ce7fc640 1 -- 172.21.15.114:0/585969433 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fc8c8096040 con 0x7fc8d01008a0 2024-09-13T18:22:17.219 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:17 smithi114 ceph-mon[30897]: pgmap v348: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-13T18:22:17.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:17.244+0000 7fc8d76d5640 1 -- 172.21.15.114:0/585969433 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7fc8d002c480 con 0x7fc8d01008a0 2024-09-13T18:22:17.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:17.245+0000 7fc8ce7fc640 1 -- 172.21.15.114:0/585969433 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v93) v1 ==== 56+0+5707 (secure 0 0 0) 0x7fc8c8099660 con 0x7fc8d01008a0 2024-09-13T18:22:17.247 INFO:teuthology.orchestra.run.smithi114.stdout:epoch 93 2024-09-13T18:22:17.247 INFO:teuthology.orchestra.run.smithi114.stdout:fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 2024-09-13T18:22:17.247 INFO:teuthology.orchestra.run.smithi114.stdout:created 2024-09-13T18:12:36.360625+0000 2024-09-13T18:22:17.247 INFO:teuthology.orchestra.run.smithi114.stdout:modified 2024-09-13T18:21:28.143797+0000 2024-09-13T18:22:17.248 INFO:teuthology.orchestra.run.smithi114.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-13T18:22:17.248 INFO:teuthology.orchestra.run.smithi114.stdout:crush_version 27 2024-09-13T18:22:17.248 INFO:teuthology.orchestra.run.smithi114.stdout:full_ratio 0.95 2024-09-13T18:22:17.248 INFO:teuthology.orchestra.run.smithi114.stdout:backfillfull_ratio 0.9 2024-09-13T18:22:17.248 INFO:teuthology.orchestra.run.smithi114.stdout:nearfull_ratio 0.85 2024-09-13T18:22:17.248 INFO:teuthology.orchestra.run.smithi114.stdout:require_min_compat_client luminous 2024-09-13T18:22:17.248 INFO:teuthology.orchestra.run.smithi114.stdout:min_compat_client luminous 2024-09-13T18:22:17.248 INFO:teuthology.orchestra.run.smithi114.stdout:require_osd_release reef 2024-09-13T18:22:17.248 INFO:teuthology.orchestra.run.smithi114.stdout:stretch_mode_enabled false 2024-09-13T18:22:17.248 INFO:teuthology.orchestra.run.smithi114.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-13T18:22:17.248 INFO:teuthology.orchestra.run.smithi114.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-13T18:22:17.249 INFO:teuthology.orchestra.run.smithi114.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-09-13T18:22:17.249 INFO:teuthology.orchestra.run.smithi114.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 89 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-09-13T18:22:17.249 INFO:teuthology.orchestra.run.smithi114.stdout:max_osd 12 2024-09-13T18:22:17.249 INFO:teuthology.orchestra.run.smithi114.stdout:osd.0 up in weight 1 up_from 8 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.114:6802/3499525826,v1:172.21.15.114:6803/3499525826] [v2:172.21.15.114:6804/3499525826,v1:172.21.15.114:6805/3499525826] exists,up 9914f01a-9f6f-4878-b955-83570199c354 2024-09-13T18:22:17.249 INFO:teuthology.orchestra.run.smithi114.stdout:osd.1 up in weight 1 up_from 13 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6800/4264264866,v1:172.21.15.134:6801/4264264866] [v2:172.21.15.134:6802/4264264866,v1:172.21.15.134:6803/4264264866] exists,up ef65e012-7469-4c20-b354-17022bca63c5 2024-09-13T18:22:17.249 INFO:teuthology.orchestra.run.smithi114.stdout:osd.2 up in weight 1 up_from 18 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6800/2288807707,v1:172.21.15.204:6801/2288807707] [v2:172.21.15.204:6802/2288807707,v1:172.21.15.204:6803/2288807707] exists,up 578213fd-715f-4be7-91fa-254f3d0e3022 2024-09-13T18:22:17.249 INFO:teuthology.orchestra.run.smithi114.stdout:osd.3 up in weight 1 up_from 25 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.114:6810/2904204254,v1:172.21.15.114:6811/2904204254] [v2:172.21.15.114:6812/2904204254,v1:172.21.15.114:6813/2904204254] exists,up b64aab3f-1e52-4828-8397-74af5e05f2cf 2024-09-13T18:22:17.249 INFO:teuthology.orchestra.run.smithi114.stdout:osd.4 up in weight 1 up_from 33 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6808/1773945333,v1:172.21.15.134:6809/1773945333] [v2:172.21.15.134:6810/1773945333,v1:172.21.15.134:6811/1773945333] exists,up f38943d6-39c0-42e1-b427-cb6bb90ac38a 2024-09-13T18:22:17.249 INFO:teuthology.orchestra.run.smithi114.stdout:osd.5 up in weight 1 up_from 38 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6808/2371778055,v1:172.21.15.204:6809/2371778055] [v2:172.21.15.204:6810/2371778055,v1:172.21.15.204:6811/2371778055] exists,up bb38f361-8e00-4178-9752-5ddbf3af20e1 2024-09-13T18:22:17.249 INFO:teuthology.orchestra.run.smithi114.stdout:osd.6 up in weight 1 up_from 46 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.114:6818/1484325044,v1:172.21.15.114:6819/1484325044] [v2:172.21.15.114:6820/1484325044,v1:172.21.15.114:6821/1484325044] exists,up 93d40a03-e7ff-4a3f-9efa-5c5403c9983d 2024-09-13T18:22:17.250 INFO:teuthology.orchestra.run.smithi114.stdout:osd.7 up in weight 1 up_from 51 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6816/799246614,v1:172.21.15.134:6817/799246614] [v2:172.21.15.134:6818/799246614,v1:172.21.15.134:6819/799246614] exists,up 982f629a-1b95-4be1-8e75-5ddce1ad77b4 2024-09-13T18:22:17.250 INFO:teuthology.orchestra.run.smithi114.stdout:osd.8 up in weight 1 up_from 59 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6816/858331805,v1:172.21.15.204:6817/858331805] [v2:172.21.15.204:6818/858331805,v1:172.21.15.204:6819/858331805] exists,up 409beb9b-8227-4dad-a4a0-f0e5ce70dc5b 2024-09-13T18:22:17.250 INFO:teuthology.orchestra.run.smithi114.stdout:osd.9 up in weight 1 up_from 67 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.114:6826/2172392854,v1:172.21.15.114:6827/2172392854] [v2:172.21.15.114:6828/2172392854,v1:172.21.15.114:6829/2172392854] exists,up a345a15c-b1fc-4877-b445-418eebf0d0ce 2024-09-13T18:22:17.250 INFO:teuthology.orchestra.run.smithi114.stdout:osd.10 up in weight 1 up_from 72 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6824/3073513468,v1:172.21.15.134:6825/3073513468] [v2:172.21.15.134:6826/3073513468,v1:172.21.15.134:6827/3073513468] exists,up 464d995f-d90b-4caa-addb-1f7562bd74b1 2024-09-13T18:22:17.250 INFO:teuthology.orchestra.run.smithi114.stdout:osd.11 up in weight 1 up_from 80 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6824/3239521765,v1:172.21.15.204:6825/3239521765] [v2:172.21.15.204:6826/3239521765,v1:172.21.15.204:6827/3239521765] exists,up 71dd794d-326c-477f-8d02-ee32bff1350d 2024-09-13T18:22:17.250 INFO:teuthology.orchestra.run.smithi114.stdout:pg_upmap_items 2.a [2,11] 2024-09-13T18:22:17.250 INFO:teuthology.orchestra.run.smithi114.stdout:pg_upmap_items 2.f [8,11] 2024-09-13T18:22:17.250 INFO:teuthology.orchestra.run.smithi114.stdout:pg_upmap_items 2.10 [2,1] 2024-09-13T18:22:17.250 INFO:teuthology.orchestra.run.smithi114.stdout:pg_upmap_items 2.16 [2,4] 2024-09-13T18:22:17.250 INFO:teuthology.orchestra.run.smithi114.stdout:pg_upmap_items 3.6 [4,7] 2024-09-13T18:22:17.250 INFO:teuthology.orchestra.run.smithi114.stdout:pg_upmap_items 3.11 [5,9] 2024-09-13T18:22:17.251 INFO:teuthology.orchestra.run.smithi114.stdout:pg_upmap_items 3.22 [3,9] 2024-09-13T18:22:17.251 INFO:teuthology.orchestra.run.smithi114.stdout:pg_upmap_items 3.2c [4,9] 2024-09-13T18:22:17.251 INFO:teuthology.orchestra.run.smithi114.stdout:pg_upmap_items 3.31 [4,8] 2024-09-13T18:22:17.251 INFO:teuthology.orchestra.run.smithi114.stdout:pg_upmap_items 4.6 [5,8] 2024-09-13T18:22:17.251 INFO:teuthology.orchestra.run.smithi114.stdout:pg_upmap_items 4.c [9,10] 2024-09-13T18:22:17.251 INFO:teuthology.orchestra.run.smithi114.stdout:pg_upmap_items 4.19 [9,8] 2024-09-13T18:22:17.251 INFO:teuthology.orchestra.run.smithi114.stdout:pg_upmap_items 4.1f [9,8] 2024-09-13T18:22:17.251 INFO:teuthology.orchestra.run.smithi114.stdout:blocklist 172.21.15.114:6801/4281866858 expires 2024-09-14T18:13:25.823027+0000 2024-09-13T18:22:17.251 INFO:teuthology.orchestra.run.smithi114.stdout:blocklist 172.21.15.114:6800/4281866858 expires 2024-09-14T18:13:25.823027+0000 2024-09-13T18:22:17.251 INFO:teuthology.orchestra.run.smithi114.stdout:blocklist 172.21.15.114:0/2949422849 expires 2024-09-14T18:13:25.823027+0000 2024-09-13T18:22:17.251 INFO:teuthology.orchestra.run.smithi114.stdout:blocklist 172.21.15.114:0/2034742251 expires 2024-09-14T18:13:03.521116+0000 2024-09-13T18:22:17.251 INFO:teuthology.orchestra.run.smithi114.stdout:blocklist 172.21.15.114:0/722571999 expires 2024-09-14T18:13:03.521116+0000 2024-09-13T18:22:17.251 INFO:teuthology.orchestra.run.smithi114.stdout:blocklist 172.21.15.114:6801/830738794 expires 2024-09-14T18:13:03.521116+0000 2024-09-13T18:22:17.251 INFO:teuthology.orchestra.run.smithi114.stdout:blocklist 172.21.15.114:6800/830738794 expires 2024-09-14T18:13:03.521116+0000 2024-09-13T18:22:17.252 INFO:teuthology.orchestra.run.smithi114.stdout:blocklist 172.21.15.114:0/2828364892 expires 2024-09-14T18:13:25.823027+0000 2024-09-13T18:22:17.252 INFO:teuthology.orchestra.run.smithi114.stdout:blocklist 172.21.15.114:0/4257828906 expires 2024-09-14T18:13:25.823027+0000 2024-09-13T18:22:17.252 INFO:teuthology.orchestra.run.smithi114.stdout:blocklist 172.21.15.114:0/3544951322 expires 2024-09-14T18:13:03.521116+0000 2024-09-13T18:22:17.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:17.248+0000 7fc8d76d5640 1 -- 172.21.15.114:0/585969433 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fc8b80a1080 msgr2=0x7fc8b80a3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:17.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:17.248+0000 7fc8d76d5640 1 --2- 172.21.15.114:0/585969433 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fc8b80a1080 0x7fc8b80a3540 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc8d00ffd40 tx=0x7fc8c0009670 comp rx=0 tx=0).stop 2024-09-13T18:22:17.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:17.248+0000 7fc8d76d5640 1 -- 172.21.15.114:0/585969433 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8d01008a0 msgr2=0x7fc8d019cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:17.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:17.248+0000 7fc8d76d5640 1 --2- 172.21.15.114:0/585969433 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8d01008a0 0x7fc8d019cfb0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc8c800c970 tx=0x7fc8c800ce30 comp rx=0 tx=0).stop 2024-09-13T18:22:17.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:17.248+0000 7fc8d76d5640 1 -- 172.21.15.114:0/585969433 shutdown_connections 2024-09-13T18:22:17.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:17.248+0000 7fc8d76d5640 1 --2- 172.21.15.114:0/585969433 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fc8b80a1080 0x7fc8b80a3540 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:17.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:17.248+0000 7fc8d76d5640 1 --2- 172.21.15.114:0/585969433 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8d01008a0 0x7fc8d019cfb0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:17.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:17.248+0000 7fc8d76d5640 1 --2- 172.21.15.114:0/585969433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8d00ffee0 0x7fc8d019aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:17.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:17.248+0000 7fc8d76d5640 1 --2- 172.21.15.114:0/585969433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8d00fece0 0x7fc8d0068a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:17.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:17.248+0000 7fc8d76d5640 1 -- 172.21.15.114:0/585969433 >> 172.21.15.114:0/585969433 conn(0x7fc8d00fa490 msgr2=0x7fc8d0103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:17.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:17.248+0000 7fc8d76d5640 1 -- 172.21.15.114:0/585969433 shutdown_connections 2024-09-13T18:22:17.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:17.248+0000 7fc8d76d5640 1 -- 172.21.15.114:0/585969433 wait complete. 2024-09-13T18:22:17.368 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:17 smithi134 ceph-mon[34283]: pgmap v348: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-13T18:22:17.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:17 smithi204 ceph-mon[34777]: pgmap v348: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-13T18:22:17.992 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-13T18:22:18.004 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:22:18.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.106+0000 7faa9cd45640 1 -- 172.21.15.114:0/2942226533 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faa98100890 msgr2=0x7faa98107120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:18.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.106+0000 7faa9cd45640 1 --2- 172.21.15.114:0/2942226533 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faa98100890 0x7faa98107120 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7faa90009ca0 tx=0x7faa9002a950 comp rx=0 tx=0).stop 2024-09-13T18:22:18.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.106+0000 7faa9cd45640 1 -- 172.21.15.114:0/2942226533 shutdown_connections 2024-09-13T18:22:18.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.106+0000 7faa9cd45640 1 --2- 172.21.15.114:0/2942226533 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faa98100890 0x7faa98107120 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:18.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.106+0000 7faa9cd45640 1 --2- 172.21.15.114:0/2942226533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa980ffed0 0x7faa98100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:18.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.106+0000 7faa9cd45640 1 --2- 172.21.15.114:0/2942226533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa980ff1c0 0x7faa980ff5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:18.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.106+0000 7faa9cd45640 1 -- 172.21.15.114:0/2942226533 >> 172.21.15.114:0/2942226533 conn(0x7faa980f9bc0 msgr2=0x7faa980fbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:18.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.106+0000 7faa9cd45640 1 -- 172.21.15.114:0/2942226533 shutdown_connections 2024-09-13T18:22:18.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.106+0000 7faa9cd45640 1 -- 172.21.15.114:0/2942226533 wait complete. 2024-09-13T18:22:18.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.107+0000 7faa9cd45640 1 Processor -- start 2024-09-13T18:22:18.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.107+0000 7faa9cd45640 1 -- start start 2024-09-13T18:22:18.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.107+0000 7faa9cd45640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa980ff1c0 0x7faa98194b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:18.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.107+0000 7faa9cd45640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faa980ffed0 0x7faa98195070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:18.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.107+0000 7faa9cd45640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa98100890 0x7faa9819c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:18.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.107+0000 7faa9cd45640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9819c620 con 0x7faa980ffed0 2024-09-13T18:22:18.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.107+0000 7faa9cd45640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9819c790 con 0x7faa98100890 2024-09-13T18:22:18.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.107+0000 7faa9cd45640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9819c950 con 0x7faa980ff1c0 2024-09-13T18:22:18.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.107+0000 7faa95d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faa980ffed0 0x7faa98195070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:18.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.107+0000 7faa95d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faa980ffed0 0x7faa98195070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43138/0 (socket says 172.21.15.114:43138) 2024-09-13T18:22:18.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.107+0000 7faa95d9b640 1 -- 172.21.15.114:0/844815740 learned_addr learned my addr 172.21.15.114:0/844815740 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:18.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.108+0000 7faa96d9d640 1 --2- 172.21.15.114:0/844815740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa98100890 0x7faa9819c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:18.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.108+0000 7faa9659c640 1 --2- 172.21.15.114:0/844815740 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa980ff1c0 0x7faa98194b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:18.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.108+0000 7faa95d9b640 1 -- 172.21.15.114:0/844815740 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa980ff1c0 msgr2=0x7faa98194b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:18.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.108+0000 7faa95d9b640 1 --2- 172.21.15.114:0/844815740 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa980ff1c0 0x7faa98194b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:18.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.108+0000 7faa95d9b640 1 -- 172.21.15.114:0/844815740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa98100890 msgr2=0x7faa9819c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:18.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.108+0000 7faa95d9b640 1 --2- 172.21.15.114:0/844815740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa98100890 0x7faa9819c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:18.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.108+0000 7faa95d9b640 1 -- 172.21.15.114:0/844815740 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa90009900 con 0x7faa980ffed0 2024-09-13T18:22:18.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.108+0000 7faa95d9b640 1 --2- 172.21.15.114:0/844815740 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faa980ffed0 0x7faa98195070 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7faa8c0078f0 tx=0x7faa8c007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:18.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.108+0000 7faa877fe640 1 -- 172.21.15.114:0/844815740 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8c011070 con 0x7faa980ffed0 2024-09-13T18:22:18.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.108+0000 7faa877fe640 1 -- 172.21.15.114:0/844815740 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faa8c00c2e0 con 0x7faa980ffed0 2024-09-13T18:22:18.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.108+0000 7faa9cd45640 1 -- 172.21.15.114:0/844815740 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa980fdd70 con 0x7faa980ffed0 2024-09-13T18:22:18.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.108+0000 7faa877fe640 1 -- 172.21.15.114:0/844815740 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8c00e630 con 0x7faa980ffed0 2024-09-13T18:22:18.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.108+0000 7faa9cd45640 1 -- 172.21.15.114:0/844815740 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa980fe230 con 0x7faa980ffed0 2024-09-13T18:22:18.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.109+0000 7faa9cd45640 1 -- 172.21.15.114:0/844815740 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa980fec10 con 0x7faa980ffed0 2024-09-13T18:22:18.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.110+0000 7faa877fe640 1 -- 172.21.15.114:0/844815740 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7faa8c004fb0 con 0x7faa980ffed0 2024-09-13T18:22:18.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.111+0000 7faa877fe640 1 --2- 172.21.15.114:0/844815740 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7faa800a1090 0x7faa800a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:18.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.111+0000 7faa877fe640 1 -- 172.21.15.114:0/844815740 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7faa8c0ce130 con 0x7faa980ffed0 2024-09-13T18:22:18.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.111+0000 7faa9659c640 1 --2- 172.21.15.114:0/844815740 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7faa800a1090 0x7faa800a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:18.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.111+0000 7faa9659c640 1 --2- 172.21.15.114:0/844815740 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7faa800a1090 0x7faa800a3550 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7faa88005fd0 tx=0x7faa88005c70 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:18.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.115+0000 7faa877fe640 1 -- 172.21.15.114:0/844815740 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7faa8c0975e0 con 0x7faa980ffed0 2024-09-13T18:22:18.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.278+0000 7faa9cd45640 1 -- 172.21.15.114:0/844815740 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faa98105c10 con 0x7faa980ffed0 2024-09-13T18:22:18.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.279+0000 7faa877fe640 1 -- 172.21.15.114:0/844815740 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24696 (secure 0 0 0) 0x7faa8c09ac00 con 0x7faa980ffed0 2024-09-13T18:22:18.282 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:22:18.282 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":93,"fsid":"97e7edf2-71fb-11ef-bceb-c7b262605968","created":"2024-09-13T18:12:36.360625+0000","modified":"2024-09-13T18:21:28.143797+0000","last_up_change":"2024-09-13T18:19:12.350370+0000","last_in_change":"2024-09-13T18:18:52.321596+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:15:40.129659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:20:00.024407+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:20:01.085549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T18:20:04.334216+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9914f01a-9f6f-4878-b955-83570199c354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6803","nonce":3499525826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6805","nonce":3499525826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6809","nonce":3499525826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6807","nonce":3499525826}]},"public_addr":"172.21.15.114:6803/3499525826","cluster_addr":"172.21.15.114:6805/3499525826","heartbeat_back_addr":"172.21.15.114:6809/3499525826","heartbeat_front_addr":"172.21.15.114:6807/3499525826","state":["exists","up"]},{"osd":1,"uuid":"ef65e012-7469-4c20-b354-17022bca63c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6801","nonce":4264264866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6803","nonce":4264264866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6807","nonce":4264264866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6805","nonce":4264264866}]},"public_addr":"172.21.15.134:6801/4264264866","cluster_addr":"172.21.15.134:6803/4264264866","heartbeat_back_addr":"172.21.15.134:6807/4264264866","heartbeat_front_addr":"172.21.15.134:6805/4264264866","state":["exists","up"]},{"osd":2,"uuid":"578213fd-715f-4be7-91fa-254f3d0e3022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6801","nonce":2288807707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6803","nonce":2288807707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6807","nonce":2288807707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6805","nonce":2288807707}]},"public_addr":"172.21.15.204:6801/2288807707","cluster_addr":"172.21.15.204:6803/2288807707","heartbeat_back_addr":"172.21.15.204:6807/2288807707","heartbeat_front_addr":"172.21.15.204:6805/2288807707","state":["exists","up"]},{"osd":3,"uuid":"b64aab3f-1e52-4828-8397-74af5e05f2cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6811","nonce":2904204254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6813","nonce":2904204254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6817","nonce":2904204254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6815","nonce":2904204254}]},"public_addr":"172.21.15.114:6811/2904204254","cluster_addr":"172.21.15.114:6813/2904204254","heartbeat_back_addr":"172.21.15.114:6817/2904204254","heartbeat_front_addr":"172.21.15.114:6815/2904204254","state":["exists","up"]},{"osd":4,"uuid":"f38943d6-39c0-42e1-b427-cb6bb90ac38a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6809","nonce":1773945333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6811","nonce":1773945333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6815","nonce":1773945333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6813","nonce":1773945333}]},"public_addr":"172.21.15.134:6809/1773945333","cluster_addr":"172.21.15.134:6811/1773945333","heartbeat_back_addr":"172.21.15.134:6815/1773945333","heartbeat_front_addr":"172.21.15.134:6813/1773945333","state":["exists","up"]},{"osd":5,"uuid":"bb38f361-8e00-4178-9752-5ddbf3af20e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6809","nonce":2371778055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6811","nonce":2371778055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6815","nonce":2371778055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6813","nonce":2371778055}]},"public_addr":"172.21.15.204:6809/2371778055","cluster_addr":"172.21.15.204:6811/2371778055","heartbeat_back_addr":"172.21.15.204:6815/2371778055","heartbeat_front_addr":"172.21.15.204:6813/2371778055","state":["exists","up"]},{"osd":6,"uuid":"93d40a03-e7ff-4a3f-9efa-5c5403c9983d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6819","nonce":1484325044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6821","nonce":1484325044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6825","nonce":1484325044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6823","nonce":1484325044}]},"public_addr":"172.21.15.114:6819/1484325044","cluster_addr":"172.21.15.114:6821/1484325044","heartbeat_back_addr":"172.21.15.114:6825/1484325044","heartbeat_front_addr":"172.21.15.114:6823/1484325044","state":["exists","up"]},{"osd":7,"uuid":"982f629a-1b95-4be1-8e75-5ddce1ad77b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6817","nonce":799246614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6819","nonce":799246614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6823","nonce":799246614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6821","nonce":799246614}]},"public_addr":"172.21.15.134:6817/799246614","cluster_addr":"172.21.15.134:6819/799246614","heartbeat_back_addr":"172.21.15.134:6823/799246614","heartbeat_front_addr":"172.21.15.134:6821/799246614","state":["exists","up"]},{"osd":8,"uuid":"409beb9b-8227-4dad-a4a0-f0e5ce70dc5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6817","nonce":858331805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6819","nonce":858331805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6823","nonce":858331805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6821","nonce":858331805}]},"public_addr":"172.21.15.204:6817/858331805","cluster_addr":"172.21.15.204:6819/858331805","heartbeat_back_addr":"172.21.15.204:6823/858331805","heartbeat_front_addr":"172.21.15.204:6821/858331805","state":["exists","up"]},{"osd":9,"uuid":"a345a15c-b1fc-4877-b445-418eebf0d0ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6827","nonce":2172392854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6829","nonce":2172392854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6833","nonce":2172392854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6831","nonce":2172392854}]},"public_addr":"172.21.15.114:6827/2172392854","cluster_addr":"172.21.15.114:6829/2172392854","heartbeat_back_addr":"172.21.15.114:6833/2172392854","heartbeat_front_addr":"172.21.15.114:6831/2172392854","state":["exists","up"]},{"osd":10,"uuid":"464d995f-d90b-4caa-addb-1f7562bd74b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6825","nonce":3073513468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6827","nonce":3073513468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6831","nonce":3073513468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6829","nonce":3073513468}]},"public_addr":"172.21.15.134:6825/3073513468","cluster_addr":"172.21.15.134:6827/3073513468","heartbeat_back_addr":"172.21.15.134:6831/3073513468","heartbeat_front_addr":"172.21.15.134:6829/3073513468","state":["exists","up"]},{"osd":11,"uuid":"71dd794d-326c-477f-8d02-ee32bff1350d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6825","nonce":3239521765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6827","nonce":3239521765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6831","nonce":3239521765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6829","nonce":3239521765}]},"public_addr":"172.21.15.204:6825/3239521765","cluster_addr":"172.21.15.204:6827/3239521765","heartbeat_back_addr":"172.21.15.204:6831/3239521765","heartbeat_front_addr":"172.21.15.204:6829/3239521765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:14:54.542687+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:15:15.746055+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:15:38.371131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:01.383932+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:24.796237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:48.422214+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:11.284323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:34.570365+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:58.232855+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:18:21.609596+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:18:45.565859+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:19:10.017969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:6801/4281866858":"2024-09-14T18:13:25.823027+0000","172.21.15.114:6800/4281866858":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2949422849":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2034742251":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/722571999":"2024-09-14T18:13:03.521116+0000","172.21.15.114:6801/830738794":"2024-09-14T18:13:03.521116+0000","172.21.15.114:6800/830738794":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/2828364892":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/4257828906":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/3544951322":"2024-09-14T18:13:03.521116+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:22:18.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.281+0000 7faa9cd45640 1 -- 172.21.15.114:0/844815740 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7faa800a1090 msgr2=0x7faa800a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:18.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.281+0000 7faa9cd45640 1 --2- 172.21.15.114:0/844815740 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7faa800a1090 0x7faa800a3550 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7faa88005fd0 tx=0x7faa88005c70 comp rx=0 tx=0).stop 2024-09-13T18:22:18.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.281+0000 7faa9cd45640 1 -- 172.21.15.114:0/844815740 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faa980ffed0 msgr2=0x7faa98195070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:18.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.281+0000 7faa9cd45640 1 --2- 172.21.15.114:0/844815740 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faa980ffed0 0x7faa98195070 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7faa8c0078f0 tx=0x7faa8c007db0 comp rx=0 tx=0).stop 2024-09-13T18:22:18.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.281+0000 7faa9cd45640 1 -- 172.21.15.114:0/844815740 shutdown_connections 2024-09-13T18:22:18.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.281+0000 7faa9cd45640 1 --2- 172.21.15.114:0/844815740 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7faa800a1090 0x7faa800a3550 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:18.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.281+0000 7faa9cd45640 1 --2- 172.21.15.114:0/844815740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa98100890 0x7faa9819c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:18.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.281+0000 7faa9cd45640 1 --2- 172.21.15.114:0/844815740 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faa980ffed0 0x7faa98195070 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:18.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.281+0000 7faa9cd45640 1 --2- 172.21.15.114:0/844815740 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa980ff1c0 0x7faa98194b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:18.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.281+0000 7faa9cd45640 1 -- 172.21.15.114:0/844815740 >> 172.21.15.114:0/844815740 conn(0x7faa980f9bc0 msgr2=0x7faa98103340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:18.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.281+0000 7faa9cd45640 1 -- 172.21.15.114:0/844815740 shutdown_connections 2024-09-13T18:22:18.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.282+0000 7faa9cd45640 1 -- 172.21.15.114:0/844815740 wait complete. 2024-09-13T18:22:18.293 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:18 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/585969433' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-13T18:22:18.295 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T18:15:40.129659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T18:20:00.024407+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T18:20:01.085549+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T18:20:04.334216+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T18:22:18.295 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T18:22:18.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:18 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/585969433' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-13T18:22:18.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.395+0000 7f1ccf59e640 1 -- 172.21.15.114:0/3785200616 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1cd0103140 msgr2=0x7f1cd0105530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:18.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.395+0000 7f1ccf59e640 1 --2- 172.21.15.114:0/3785200616 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1cd0103140 0x7f1cd0105530 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8009900 tx=0x7f1cb802a410 comp rx=0 tx=0).stop 2024-09-13T18:22:18.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.395+0000 7f1ccf59e640 1 -- 172.21.15.114:0/3785200616 shutdown_connections 2024-09-13T18:22:18.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.395+0000 7f1ccf59e640 1 --2- 172.21.15.114:0/3785200616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cd0105a70 0x7f1cd0107e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:18.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.395+0000 7f1ccf59e640 1 --2- 172.21.15.114:0/3785200616 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1cd0103140 0x7f1cd0105530 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:18.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.395+0000 7f1ccf59e640 1 --2- 172.21.15.114:0/3785200616 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1cd0100810 0x7f1cd0102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:18.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.395+0000 7f1ccf59e640 1 -- 172.21.15.114:0/3785200616 >> 172.21.15.114:0/3785200616 conn(0x7f1cd00fa330 msgr2=0x7f1cd00fc790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:18.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.395+0000 7f1ccf59e640 1 -- 172.21.15.114:0/3785200616 shutdown_connections 2024-09-13T18:22:18.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.395+0000 7f1ccf59e640 1 -- 172.21.15.114:0/3785200616 wait complete. 2024-09-13T18:22:18.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.396+0000 7f1ccf59e640 1 Processor -- start 2024-09-13T18:22:18.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.396+0000 7f1ccf59e640 1 -- start start 2024-09-13T18:22:18.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.396+0000 7f1ccf59e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1cd0100810 0x7f1cd0195cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:18.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.396+0000 7f1ccf59e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1cd0103140 0x7f1cd01961f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:18.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.396+0000 7f1ccf59e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cd0105a70 0x7f1cd019d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:18.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.396+0000 7f1ccf59e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd019d7c0 con 0x7f1cd0100810 2024-09-13T18:22:18.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.396+0000 7f1ccf59e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd019d930 con 0x7f1cd0105a70 2024-09-13T18:22:18.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.396+0000 7f1ccf59e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd019daf0 con 0x7f1cd0103140 2024-09-13T18:22:18.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.396+0000 7f1cce59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1cd0100810 0x7f1cd0195cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:18.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.396+0000 7f1cce59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1cd0100810 0x7f1cd0195cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43166/0 (socket says 172.21.15.114:43166) 2024-09-13T18:22:18.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.396+0000 7f1cce59c640 1 -- 172.21.15.114:0/1484852693 learned_addr learned my addr 172.21.15.114:0/1484852693 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:18.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.396+0000 7f1cced9d640 1 --2- 172.21.15.114:0/1484852693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cd0105a70 0x7f1cd019d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:18.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.396+0000 7f1ccdd9b640 1 --2- 172.21.15.114:0/1484852693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1cd0103140 0x7f1cd01961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:18.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.396+0000 7f1cce59c640 1 -- 172.21.15.114:0/1484852693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1cd0103140 msgr2=0x7f1cd01961f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:18.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.396+0000 7f1cce59c640 1 --2- 172.21.15.114:0/1484852693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1cd0103140 0x7f1cd01961f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:18.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.396+0000 7f1cce59c640 1 -- 172.21.15.114:0/1484852693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cd0105a70 msgr2=0x7f1cd019d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:18.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.396+0000 7f1cce59c640 1 --2- 172.21.15.114:0/1484852693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cd0105a70 0x7f1cd019d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:18.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.396+0000 7f1cce59c640 1 -- 172.21.15.114:0/1484852693 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb8009560 con 0x7f1cd0100810 2024-09-13T18:22:18.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.396+0000 7f1ccdd9b640 1 --2- 172.21.15.114:0/1484852693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1cd0103140 0x7f1cd01961f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:22:18.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.396+0000 7f1cce59c640 1 --2- 172.21.15.114:0/1484852693 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1cd0100810 0x7f1cd0195cb0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1cc000e9f0 tx=0x7f1cc000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:18.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.396+0000 7f1cc77fe640 1 -- 172.21.15.114:0/1484852693 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc0009700 con 0x7f1cd0100810 2024-09-13T18:22:18.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.396+0000 7f1cc77fe640 1 -- 172.21.15.114:0/1484852693 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1cc000cd80 con 0x7f1cd0100810 2024-09-13T18:22:18.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.396+0000 7f1ccf59e640 1 -- 172.21.15.114:0/1484852693 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cd019ddd0 con 0x7f1cd0100810 2024-09-13T18:22:18.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.397+0000 7f1cc77fe640 1 -- 172.21.15.114:0/1484852693 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc00187e0 con 0x7f1cd0100810 2024-09-13T18:22:18.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.397+0000 7f1ccf59e640 1 -- 172.21.15.114:0/1484852693 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cd019e1f0 con 0x7f1cd0100810 2024-09-13T18:22:18.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.398+0000 7f1ccf59e640 1 -- 172.21.15.114:0/1484852693 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cd00fbe20 con 0x7f1cd0100810 2024-09-13T18:22:18.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.399+0000 7f1cc77fe640 1 -- 172.21.15.114:0/1484852693 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f1cc0010430 con 0x7f1cd0100810 2024-09-13T18:22:18.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.399+0000 7f1cc77fe640 1 --2- 172.21.15.114:0/1484852693 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f1cbc0a1090 0x7f1cbc0a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:18.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.399+0000 7f1ccdd9b640 1 --2- 172.21.15.114:0/1484852693 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f1cbc0a1090 0x7f1cbc0a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:18.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.399+0000 7f1cc77fe640 1 -- 172.21.15.114:0/1484852693 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f1cc0014070 con 0x7f1cd0100810 2024-09-13T18:22:18.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.400+0000 7f1ccdd9b640 1 --2- 172.21.15.114:0/1484852693 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f1cbc0a1090 0x7f1cbc0a3550 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8006010 tx=0x7f1cb8005e00 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:18.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.403+0000 7f1cc77fe640 1 -- 172.21.15.114:0/1484852693 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f1cc0052e20 con 0x7f1cd0100810 2024-09-13T18:22:18.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:18 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/585969433' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-13T18:22:18.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.566+0000 7f1ccf59e640 1 -- 172.21.15.114:0/1484852693 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1cd00fb890 con 0x7f1cd0100810 2024-09-13T18:22:18.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.567+0000 7f1cc77fe640 1 -- 172.21.15.114:0/1484852693 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1cc0096430 con 0x7f1cd0100810 2024-09-13T18:22:18.569 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 1 2024-09-13T18:22:18.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.568+0000 7f1ccf59e640 1 -- 172.21.15.114:0/1484852693 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f1cbc0a1090 msgr2=0x7f1cbc0a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:18.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.568+0000 7f1ccf59e640 1 --2- 172.21.15.114:0/1484852693 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f1cbc0a1090 0x7f1cbc0a3550 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8006010 tx=0x7f1cb8005e00 comp rx=0 tx=0).stop 2024-09-13T18:22:18.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.568+0000 7f1ccf59e640 1 -- 172.21.15.114:0/1484852693 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1cd0100810 msgr2=0x7f1cd0195cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:18.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.568+0000 7f1ccf59e640 1 --2- 172.21.15.114:0/1484852693 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1cd0100810 0x7f1cd0195cb0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1cc000e9f0 tx=0x7f1cc000eeb0 comp rx=0 tx=0).stop 2024-09-13T18:22:18.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.568+0000 7f1ccf59e640 1 -- 172.21.15.114:0/1484852693 shutdown_connections 2024-09-13T18:22:18.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.568+0000 7f1ccf59e640 1 --2- 172.21.15.114:0/1484852693 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f1cbc0a1090 0x7f1cbc0a3550 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:18.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.568+0000 7f1ccf59e640 1 --2- 172.21.15.114:0/1484852693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cd0105a70 0x7f1cd019d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:18.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.568+0000 7f1ccf59e640 1 --2- 172.21.15.114:0/1484852693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1cd0103140 0x7f1cd01961f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:18.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.568+0000 7f1ccf59e640 1 --2- 172.21.15.114:0/1484852693 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1cd0100810 0x7f1cd0195cb0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:18.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.569+0000 7f1ccf59e640 1 -- 172.21.15.114:0/1484852693 >> 172.21.15.114:0/1484852693 conn(0x7f1cd00fa330 msgr2=0x7f1cd0101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:18.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.569+0000 7f1ccf59e640 1 -- 172.21.15.114:0/1484852693 shutdown_connections 2024-09-13T18:22:18.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.569+0000 7f1ccf59e640 1 -- 172.21.15.114:0/1484852693 wait complete. 2024-09-13T18:22:18.582 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T18:22:18.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.680+0000 7efd6a1c7640 1 -- 172.21.15.114:0/3813222545 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efd640fffe0 msgr2=0x7efd64106870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:18.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.681+0000 7efd6a1c7640 1 --2- 172.21.15.114:0/3813222545 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efd640fffe0 0x7efd64106870 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7efd5c00aa10 tx=0x7efd5c02b720 comp rx=0 tx=0).stop 2024-09-13T18:22:18.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.681+0000 7efd6a1c7640 1 -- 172.21.15.114:0/3813222545 shutdown_connections 2024-09-13T18:22:18.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.681+0000 7efd6a1c7640 1 --2- 172.21.15.114:0/3813222545 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efd640fffe0 0x7efd64106870 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:18.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.681+0000 7efd6a1c7640 1 --2- 172.21.15.114:0/3813222545 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd640ff620 0x7efd640ffaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:18.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.681+0000 7efd6a1c7640 1 --2- 172.21.15.114:0/3813222545 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd640fe430 0x7efd640fe830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:18.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.681+0000 7efd6a1c7640 1 -- 172.21.15.114:0/3813222545 >> 172.21.15.114:0/3813222545 conn(0x7efd640f9b70 msgr2=0x7efd640fbfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:18.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.681+0000 7efd6a1c7640 1 -- 172.21.15.114:0/3813222545 shutdown_connections 2024-09-13T18:22:18.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.681+0000 7efd6a1c7640 1 -- 172.21.15.114:0/3813222545 wait complete. 2024-09-13T18:22:18.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.681+0000 7efd6a1c7640 1 Processor -- start 2024-09-13T18:22:18.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.681+0000 7efd6a1c7640 1 -- start start 2024-09-13T18:22:18.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.682+0000 7efd6a1c7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efd640fe430 0x7efd64195cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:18.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.682+0000 7efd6a1c7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd640ff620 0x7efd641961f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:18.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.682+0000 7efd6a1c7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd640fffe0 0x7efd6419d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:18.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.682+0000 7efd6a1c7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6419d7c0 con 0x7efd640fe430 2024-09-13T18:22:18.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.682+0000 7efd6a1c7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6419d930 con 0x7efd640fffe0 2024-09-13T18:22:18.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.682+0000 7efd6a1c7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6419daf0 con 0x7efd640ff620 2024-09-13T18:22:18.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.682+0000 7efd637fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efd640fe430 0x7efd64195cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:18.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.682+0000 7efd637fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efd640fe430 0x7efd64195cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43176/0 (socket says 172.21.15.114:43176) 2024-09-13T18:22:18.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.682+0000 7efd637fe640 1 -- 172.21.15.114:0/3112291411 learned_addr learned my addr 172.21.15.114:0/3112291411 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:18.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.682+0000 7efd63fff640 1 --2- 172.21.15.114:0/3112291411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd640fffe0 0x7efd6419d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:18.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.682+0000 7efd62ffd640 1 --2- 172.21.15.114:0/3112291411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd640ff620 0x7efd641961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:18.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.682+0000 7efd637fe640 1 -- 172.21.15.114:0/3112291411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd640ff620 msgr2=0x7efd641961f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:18.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.682+0000 7efd637fe640 1 --2- 172.21.15.114:0/3112291411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd640ff620 0x7efd641961f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:18.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.682+0000 7efd637fe640 1 -- 172.21.15.114:0/3112291411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd640fffe0 msgr2=0x7efd6419d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:18.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.682+0000 7efd637fe640 1 --2- 172.21.15.114:0/3112291411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd640fffe0 0x7efd6419d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:18.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.682+0000 7efd637fe640 1 -- 172.21.15.114:0/3112291411 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd5c00a670 con 0x7efd640fe430 2024-09-13T18:22:18.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.682+0000 7efd637fe640 1 --2- 172.21.15.114:0/3112291411 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efd640fe430 0x7efd64195cb0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7efd5400ea80 tx=0x7efd5400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:18.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.682+0000 7efd60ff9640 1 -- 172.21.15.114:0/3112291411 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5400ce40 con 0x7efd640fe430 2024-09-13T18:22:18.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.682+0000 7efd60ff9640 1 -- 172.21.15.114:0/3112291411 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efd54011900 con 0x7efd640fe430 2024-09-13T18:22:18.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.682+0000 7efd6a1c7640 1 -- 172.21.15.114:0/3112291411 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd6419ddd0 con 0x7efd640fe430 2024-09-13T18:22:18.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.682+0000 7efd60ff9640 1 -- 172.21.15.114:0/3112291411 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5401a970 con 0x7efd640fe430 2024-09-13T18:22:18.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.682+0000 7efd6a1c7640 1 -- 172.21.15.114:0/3112291411 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd6419e220 con 0x7efd640fe430 2024-09-13T18:22:18.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.683+0000 7efd6a1c7640 1 -- 172.21.15.114:0/3112291411 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd64109370 con 0x7efd640fe430 2024-09-13T18:22:18.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.684+0000 7efd60ff9640 1 -- 172.21.15.114:0/3112291411 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7efd54011470 con 0x7efd640fe430 2024-09-13T18:22:18.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.685+0000 7efd60ff9640 1 --2- 172.21.15.114:0/3112291411 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7efd4c0a1090 0x7efd4c0a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:18.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.685+0000 7efd62ffd640 1 --2- 172.21.15.114:0/3112291411 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7efd4c0a1090 0x7efd4c0a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:18.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.685+0000 7efd60ff9640 1 -- 172.21.15.114:0/3112291411 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7efd540cd410 con 0x7efd640fe430 2024-09-13T18:22:18.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.686+0000 7efd62ffd640 1 --2- 172.21.15.114:0/3112291411 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7efd4c0a1090 0x7efd4c0a3550 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7efd641971b0 tx=0x7efd58005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:18.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.688+0000 7efd60ff9640 1 -- 172.21.15.114:0/3112291411 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7efd54096830 con 0x7efd640fe430 2024-09-13T18:22:18.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.851+0000 7efd6a1c7640 1 -- 172.21.15.114:0/3112291411 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efd6419e4a0 con 0x7efd640fe430 2024-09-13T18:22:18.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.852+0000 7efd60ff9640 1 -- 172.21.15.114:0/3112291411 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7efd54099e50 con 0x7efd640fe430 2024-09-13T18:22:18.854 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2024-09-13T18:22:18.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.853+0000 7efd6a1c7640 1 -- 172.21.15.114:0/3112291411 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7efd4c0a1090 msgr2=0x7efd4c0a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:18.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.853+0000 7efd6a1c7640 1 --2- 172.21.15.114:0/3112291411 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7efd4c0a1090 0x7efd4c0a3550 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7efd641971b0 tx=0x7efd58005f90 comp rx=0 tx=0).stop 2024-09-13T18:22:18.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.853+0000 7efd6a1c7640 1 -- 172.21.15.114:0/3112291411 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efd640fe430 msgr2=0x7efd64195cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:18.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.854+0000 7efd6a1c7640 1 --2- 172.21.15.114:0/3112291411 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efd640fe430 0x7efd64195cb0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7efd5400ea80 tx=0x7efd5400ef40 comp rx=0 tx=0).stop 2024-09-13T18:22:18.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.854+0000 7efd6a1c7640 1 -- 172.21.15.114:0/3112291411 shutdown_connections 2024-09-13T18:22:18.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.854+0000 7efd6a1c7640 1 --2- 172.21.15.114:0/3112291411 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7efd4c0a1090 0x7efd4c0a3550 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:18.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.854+0000 7efd6a1c7640 1 --2- 172.21.15.114:0/3112291411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd640fffe0 0x7efd6419d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:18.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.854+0000 7efd6a1c7640 1 --2- 172.21.15.114:0/3112291411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd640ff620 0x7efd641961f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:18.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.854+0000 7efd6a1c7640 1 --2- 172.21.15.114:0/3112291411 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efd640fe430 0x7efd64195cb0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:18.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.854+0000 7efd6a1c7640 1 -- 172.21.15.114:0/3112291411 >> 172.21.15.114:0/3112291411 conn(0x7efd640f9b70 msgr2=0x7efd64104d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:18.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.854+0000 7efd6a1c7640 1 -- 172.21.15.114:0/3112291411 shutdown_connections 2024-09-13T18:22:18.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.854+0000 7efd6a1c7640 1 -- 172.21.15.114:0/3112291411 wait complete. 2024-09-13T18:22:18.867 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T18:22:18.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.966+0000 7fc22bb9c640 1 -- 172.21.15.114:0/2005164208 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2240ffe70 msgr2=0x7fc2241002f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:18.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.966+0000 7fc22bb9c640 1 --2- 172.21.15.114:0/2005164208 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2240ffe70 0x7fc2241002f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fc214009900 tx=0x7fc21402a410 comp rx=0 tx=0).stop 2024-09-13T18:22:18.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.966+0000 7fc22bb9c640 1 -- 172.21.15.114:0/2005164208 shutdown_connections 2024-09-13T18:22:18.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.966+0000 7fc22bb9c640 1 --2- 172.21.15.114:0/2005164208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc224100830 0x7fc2241070c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:18.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.966+0000 7fc22bb9c640 1 --2- 172.21.15.114:0/2005164208 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2240ffe70 0x7fc2241002f0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:18.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.966+0000 7fc22bb9c640 1 --2- 172.21.15.114:0/2005164208 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2240fec80 0x7fc2240ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:18.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.966+0000 7fc22bb9c640 1 -- 172.21.15.114:0/2005164208 >> 172.21.15.114:0/2005164208 conn(0x7fc2240fa3e0 msgr2=0x7fc2240fc820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:18.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.966+0000 7fc22bb9c640 1 -- 172.21.15.114:0/2005164208 shutdown_connections 2024-09-13T18:22:18.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.966+0000 7fc22bb9c640 1 -- 172.21.15.114:0/2005164208 wait complete. 2024-09-13T18:22:18.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.967+0000 7fc22bb9c640 1 Processor -- start 2024-09-13T18:22:18.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.967+0000 7fc22bb9c640 1 -- start start 2024-09-13T18:22:18.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.967+0000 7fc22bb9c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2240fec80 0x7fc224069100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:18.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.967+0000 7fc22bb9c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2240ffe70 0x7fc22419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:18.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.967+0000 7fc22bb9c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc224100830 0x7fc22419d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:18.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.967+0000 7fc22bb9c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22419d830 con 0x7fc2240ffe70 2024-09-13T18:22:18.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.967+0000 7fc22bb9c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22419d9a0 con 0x7fc2240fec80 2024-09-13T18:22:18.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.967+0000 7fc22bb9c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22419db60 con 0x7fc224100830 2024-09-13T18:22:18.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.967+0000 7fc229137640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2240ffe70 0x7fc22419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:18.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.967+0000 7fc229137640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2240ffe70 0x7fc22419aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43194/0 (socket says 172.21.15.114:43194) 2024-09-13T18:22:18.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.967+0000 7fc229137640 1 -- 172.21.15.114:0/1422518311 learned_addr learned my addr 172.21.15.114:0/1422518311 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:18.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.967+0000 7fc229938640 1 --2- 172.21.15.114:0/1422518311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2240fec80 0x7fc224069100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:18.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.967+0000 7fc22a139640 1 --2- 172.21.15.114:0/1422518311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc224100830 0x7fc22419d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:18.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.967+0000 7fc229137640 1 -- 172.21.15.114:0/1422518311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc224100830 msgr2=0x7fc22419d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:18.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.967+0000 7fc229137640 1 --2- 172.21.15.114:0/1422518311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc224100830 0x7fc22419d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:18.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.967+0000 7fc229137640 1 -- 172.21.15.114:0/1422518311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2240fec80 msgr2=0x7fc224069100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:18.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.967+0000 7fc229137640 1 --2- 172.21.15.114:0/1422518311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2240fec80 0x7fc224069100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:18.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.967+0000 7fc229137640 1 -- 172.21.15.114:0/1422518311 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc214009560 con 0x7fc2240ffe70 2024-09-13T18:22:18.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.967+0000 7fc229137640 1 --2- 172.21.15.114:0/1422518311 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2240ffe70 0x7fc22419aa30 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fc2140057c0 tx=0x7fc2140052b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:18.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.968+0000 7fc21affd640 1 -- 172.21.15.114:0/1422518311 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc214031d70 con 0x7fc2240ffe70 2024-09-13T18:22:18.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.968+0000 7fc21affd640 1 -- 172.21.15.114:0/1422518311 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc2140054a0 con 0x7fc2240ffe70 2024-09-13T18:22:18.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.968+0000 7fc22bb9c640 1 -- 172.21.15.114:0/1422518311 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc22419dd80 con 0x7fc2240ffe70 2024-09-13T18:22:18.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.968+0000 7fc21affd640 1 -- 172.21.15.114:0/1422518311 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc214003b10 con 0x7fc2240ffe70 2024-09-13T18:22:18.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.968+0000 7fc22bb9c640 1 -- 172.21.15.114:0/1422518311 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc22419e1d0 con 0x7fc2240ffe70 2024-09-13T18:22:18.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.969+0000 7fc22bb9c640 1 -- 172.21.15.114:0/1422518311 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc224106250 con 0x7fc2240ffe70 2024-09-13T18:22:18.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.970+0000 7fc21affd640 1 -- 172.21.15.114:0/1422518311 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fc214003c70 con 0x7fc2240ffe70 2024-09-13T18:22:18.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.970+0000 7fc21affd640 1 --2- 172.21.15.114:0/1422518311 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fc2100a1090 0x7fc2100a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:18.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.971+0000 7fc229938640 1 --2- 172.21.15.114:0/1422518311 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fc2100a1090 0x7fc2100a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:18.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.971+0000 7fc21affd640 1 -- 172.21.15.114:0/1422518311 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fc2140edf50 con 0x7fc2240ffe70 2024-09-13T18:22:18.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.971+0000 7fc229938640 1 --2- 172.21.15.114:0/1422518311 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fc2100a1090 0x7fc2100a3550 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc2240ffcd0 tx=0x7fc22000a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:18.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:18.975+0000 7fc21affd640 1 -- 172.21.15.114:0/1422518311 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fc2140b7420 con 0x7fc2240ffe70 2024-09-13T18:22:19.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.137+0000 7fc22bb9c640 1 -- 172.21.15.114:0/1422518311 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc22402c480 con 0x7fc2240ffe70 2024-09-13T18:22:19.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.138+0000 7fc21affd640 1 -- 172.21.15.114:0/1422518311 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc21404a0a0 con 0x7fc2240ffe70 2024-09-13T18:22:19.140 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 64 2024-09-13T18:22:19.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.140+0000 7fc22bb9c640 1 -- 172.21.15.114:0/1422518311 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fc2100a1090 msgr2=0x7fc2100a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:19.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.140+0000 7fc22bb9c640 1 --2- 172.21.15.114:0/1422518311 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fc2100a1090 0x7fc2100a3550 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc2240ffcd0 tx=0x7fc22000a2d0 comp rx=0 tx=0).stop 2024-09-13T18:22:19.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.140+0000 7fc22bb9c640 1 -- 172.21.15.114:0/1422518311 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2240ffe70 msgr2=0x7fc22419aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:19.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.140+0000 7fc22bb9c640 1 --2- 172.21.15.114:0/1422518311 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2240ffe70 0x7fc22419aa30 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fc2140057c0 tx=0x7fc2140052b0 comp rx=0 tx=0).stop 2024-09-13T18:22:19.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.140+0000 7fc22bb9c640 1 -- 172.21.15.114:0/1422518311 shutdown_connections 2024-09-13T18:22:19.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.140+0000 7fc22bb9c640 1 --2- 172.21.15.114:0/1422518311 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fc2100a1090 0x7fc2100a3550 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:19.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.140+0000 7fc22bb9c640 1 --2- 172.21.15.114:0/1422518311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc224100830 0x7fc22419d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:19.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.140+0000 7fc22bb9c640 1 --2- 172.21.15.114:0/1422518311 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2240ffe70 0x7fc22419aa30 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:19.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.140+0000 7fc22bb9c640 1 --2- 172.21.15.114:0/1422518311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2240fec80 0x7fc224069100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:19.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.140+0000 7fc22bb9c640 1 -- 172.21.15.114:0/1422518311 >> 172.21.15.114:0/1422518311 conn(0x7fc2240fa3e0 msgr2=0x7fc224103420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:19.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.140+0000 7fc22bb9c640 1 -- 172.21.15.114:0/1422518311 shutdown_connections 2024-09-13T18:22:19.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.140+0000 7fc22bb9c640 1 -- 172.21.15.114:0/1422518311 wait complete. 2024-09-13T18:22:19.151 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:19 smithi114 ceph-mon[30897]: pgmap v349: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-13T18:22:19.152 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:19 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/844815740' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:22:19.152 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:19 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/1484852693' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T18:22:19.152 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:19 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/3112291411' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T18:22:19.153 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T18:22:19.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.253+0000 7f4d4ce20640 1 -- 172.21.15.114:0/3614050402 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d480ff8b0 msgr2=0x7f4d48107150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:19.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.253+0000 7f4d4ce20640 1 --2- 172.21.15.114:0/3614050402 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d480ff8b0 0x7f4d48107150 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f4d38009ca0 tx=0x7f4d3802a8b0 comp rx=0 tx=0).stop 2024-09-13T18:22:19.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.253+0000 7f4d4ce20640 1 -- 172.21.15.114:0/3614050402 shutdown_connections 2024-09-13T18:22:19.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.253+0000 7f4d4ce20640 1 --2- 172.21.15.114:0/3614050402 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d480ff8b0 0x7f4d48107150 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:19.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.253+0000 7f4d4ce20640 1 --2- 172.21.15.114:0/3614050402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d480feef0 0x7f4d480ff370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:19.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.253+0000 7f4d4ce20640 1 --2- 172.21.15.114:0/3614050402 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d48100330 0x7f4d48100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:19.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.253+0000 7f4d4ce20640 1 -- 172.21.15.114:0/3614050402 >> 172.21.15.114:0/3614050402 conn(0x7f4d480f9b70 msgr2=0x7f4d480fbfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:19.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.253+0000 7f4d4ce20640 1 -- 172.21.15.114:0/3614050402 shutdown_connections 2024-09-13T18:22:19.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.253+0000 7f4d4ce20640 1 -- 172.21.15.114:0/3614050402 wait complete. 2024-09-13T18:22:19.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.253+0000 7f4d4ce20640 1 Processor -- start 2024-09-13T18:22:19.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.253+0000 7f4d4ce20640 1 -- start start 2024-09-13T18:22:19.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.254+0000 7f4d4ce20640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d480feef0 0x7f4d480fe950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:19.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.254+0000 7f4d4ce20640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d480ff8b0 0x7f4d4819a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:19.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.254+0000 7f4d4ce20640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d48100330 0x7f4d4819d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:19.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.254+0000 7f4d4ce20640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4819d7f0 con 0x7f4d48100330 2024-09-13T18:22:19.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.254+0000 7f4d4ce20640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4819d960 con 0x7f4d480ff8b0 2024-09-13T18:22:19.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.254+0000 7f4d4ce20640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4819db20 con 0x7f4d480feef0 2024-09-13T18:22:19.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.254+0000 7f4d46d9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d48100330 0x7f4d4819d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:19.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.254+0000 7f4d46d9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d48100330 0x7f4d4819d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43212/0 (socket says 172.21.15.114:43212) 2024-09-13T18:22:19.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.254+0000 7f4d46d9d640 1 -- 172.21.15.114:0/319272332 learned_addr learned my addr 172.21.15.114:0/319272332 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:19.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.254+0000 7f4d4659c640 1 --2- 172.21.15.114:0/319272332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d480feef0 0x7f4d480fe950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:19.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.254+0000 7f4d45d9b640 1 --2- 172.21.15.114:0/319272332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d480ff8b0 0x7f4d4819a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:19.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.254+0000 7f4d46d9d640 1 -- 172.21.15.114:0/319272332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d480feef0 msgr2=0x7f4d480fe950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:19.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.254+0000 7f4d46d9d640 1 --2- 172.21.15.114:0/319272332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d480feef0 0x7f4d480fe950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:19.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.254+0000 7f4d46d9d640 1 -- 172.21.15.114:0/319272332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d480ff8b0 msgr2=0x7f4d4819a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:19.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.254+0000 7f4d46d9d640 1 --2- 172.21.15.114:0/319272332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d480ff8b0 0x7f4d4819a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:19.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.254+0000 7f4d46d9d640 1 -- 172.21.15.114:0/319272332 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d38009900 con 0x7f4d48100330 2024-09-13T18:22:19.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.254+0000 7f4d46d9d640 1 --2- 172.21.15.114:0/319272332 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d48100330 0x7f4d4819d010 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4d380097b0 tx=0x7f4d38003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:19.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.254+0000 7f4d3f7fe640 1 -- 172.21.15.114:0/319272332 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d38004450 con 0x7f4d48100330 2024-09-13T18:22:19.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.254+0000 7f4d3f7fe640 1 -- 172.21.15.114:0/319272332 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4d38003e10 con 0x7f4d48100330 2024-09-13T18:22:19.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.254+0000 7f4d4ce20640 1 -- 172.21.15.114:0/319272332 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d4819dd40 con 0x7f4d48100330 2024-09-13T18:22:19.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.254+0000 7f4d3f7fe640 1 -- 172.21.15.114:0/319272332 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d38045dc0 con 0x7f4d48100330 2024-09-13T18:22:19.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.254+0000 7f4d4ce20640 1 -- 172.21.15.114:0/319272332 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d4819e1f0 con 0x7f4d48100330 2024-09-13T18:22:19.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.255+0000 7f4d4ce20640 1 -- 172.21.15.114:0/319272332 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d481061f0 con 0x7f4d48100330 2024-09-13T18:22:19.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.257+0000 7f4d3f7fe640 1 -- 172.21.15.114:0/319272332 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f4d38042070 con 0x7f4d48100330 2024-09-13T18:22:19.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.257+0000 7f4d3f7fe640 1 --2- 172.21.15.114:0/319272332 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f4d340a1090 0x7f4d340a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:19.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.257+0000 7f4d3f7fe640 1 -- 172.21.15.114:0/319272332 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f4d380ee350 con 0x7f4d48100330 2024-09-13T18:22:19.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.257+0000 7f4d4659c640 1 --2- 172.21.15.114:0/319272332 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f4d340a1090 0x7f4d340a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:19.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.258+0000 7f4d4659c640 1 --2- 172.21.15.114:0/319272332 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f4d340a1090 0x7f4d340a3550 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4d4000bb60 tx=0x7f4d4000b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:19.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.260+0000 7f4d3f7fe640 1 -- 172.21.15.114:0/319272332 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f4d380b7890 con 0x7f4d48100330 2024-09-13T18:22:19.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:19 smithi134 ceph-mon[34283]: pgmap v349: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-13T18:22:19.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:19 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/844815740' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:22:19.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:19 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/1484852693' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T18:22:19.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:19 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/3112291411' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T18:22:19.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.422+0000 7f4d4ce20640 1 -- 172.21.15.114:0/319272332 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f4d4802c480 con 0x7f4d48100330 2024-09-13T18:22:19.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.422+0000 7f4d3f7fe640 1 -- 172.21.15.114:0/319272332 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7f4d380baeb0 con 0x7f4d48100330 2024-09-13T18:22:19.425 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2024-09-13T18:22:19.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.424+0000 7f4d4ce20640 1 -- 172.21.15.114:0/319272332 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f4d340a1090 msgr2=0x7f4d340a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:19.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.424+0000 7f4d4ce20640 1 --2- 172.21.15.114:0/319272332 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f4d340a1090 0x7f4d340a3550 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4d4000bb60 tx=0x7f4d4000b390 comp rx=0 tx=0).stop 2024-09-13T18:22:19.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.424+0000 7f4d4ce20640 1 -- 172.21.15.114:0/319272332 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d48100330 msgr2=0x7f4d4819d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:19.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.424+0000 7f4d4ce20640 1 --2- 172.21.15.114:0/319272332 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d48100330 0x7f4d4819d010 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4d380097b0 tx=0x7f4d38003c20 comp rx=0 tx=0).stop 2024-09-13T18:22:19.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.424+0000 7f4d4ce20640 1 -- 172.21.15.114:0/319272332 shutdown_connections 2024-09-13T18:22:19.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.424+0000 7f4d4ce20640 1 --2- 172.21.15.114:0/319272332 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f4d340a1090 0x7f4d340a3550 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:19.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.424+0000 7f4d4ce20640 1 --2- 172.21.15.114:0/319272332 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d48100330 0x7f4d4819d010 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:19.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.424+0000 7f4d4ce20640 1 --2- 172.21.15.114:0/319272332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d480ff8b0 0x7f4d4819a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:19.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.424+0000 7f4d4ce20640 1 --2- 172.21.15.114:0/319272332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d480feef0 0x7f4d480fe950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:19.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.424+0000 7f4d4ce20640 1 -- 172.21.15.114:0/319272332 >> 172.21.15.114:0/319272332 conn(0x7f4d480f9b70 msgr2=0x7f4d481033c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:19.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.424+0000 7f4d4ce20640 1 -- 172.21.15.114:0/319272332 shutdown_connections 2024-09-13T18:22:19.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.424+0000 7f4d4ce20640 1 -- 172.21.15.114:0/319272332 wait complete. 2024-09-13T18:22:19.438 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:22:19.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:19 smithi204 ceph-mon[34777]: pgmap v349: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-13T18:22:19.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:19 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/844815740' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:22:19.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:19 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/1484852693' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T18:22:19.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:19 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/3112291411' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T18:22:19.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.534+0000 7f767ad53640 1 -- 172.21.15.114:0/664051563 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76740ff620 msgr2=0x7f76740ffaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:19.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.534+0000 7f767ad53640 1 --2- 172.21.15.114:0/664051563 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76740ff620 0x7f76740ffaa0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7660009900 tx=0x7f766002a410 comp rx=0 tx=0).stop 2024-09-13T18:22:19.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.534+0000 7f767ad53640 1 -- 172.21.15.114:0/664051563 shutdown_connections 2024-09-13T18:22:19.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.534+0000 7f767ad53640 1 --2- 172.21.15.114:0/664051563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76740fffe0 0x7f7674106870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:19.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.534+0000 7f767ad53640 1 --2- 172.21.15.114:0/664051563 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76740ff620 0x7f76740ffaa0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:19.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.534+0000 7f767ad53640 1 --2- 172.21.15.114:0/664051563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76740fe430 0x7f76740fe830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:19.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.534+0000 7f767ad53640 1 -- 172.21.15.114:0/664051563 >> 172.21.15.114:0/664051563 conn(0x7f76740f9bb0 msgr2=0x7f76740fbfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:19.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.534+0000 7f767ad53640 1 -- 172.21.15.114:0/664051563 shutdown_connections 2024-09-13T18:22:19.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.534+0000 7f767ad53640 1 -- 172.21.15.114:0/664051563 wait complete. 2024-09-13T18:22:19.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.534+0000 7f767ad53640 1 Processor -- start 2024-09-13T18:22:19.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.534+0000 7f767ad53640 1 -- start start 2024-09-13T18:22:19.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.534+0000 7f767ad53640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76740fe430 0x7f7674194c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:19.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.534+0000 7f767ad53640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76740ff620 0x7f7674195190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:19.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.534+0000 7f767ad53640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76740fffe0 0x7f767419c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:19.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.534+0000 7f767ad53640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f767419c730 con 0x7f76740fe430 2024-09-13T18:22:19.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.534+0000 7f767ad53640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f767419c8a0 con 0x7f76740fffe0 2024-09-13T18:22:19.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.534+0000 7f767ad53640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f767419ca60 con 0x7f76740ff620 2024-09-13T18:22:19.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.534+0000 7f7678aef640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76740fe430 0x7f7674194c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:19.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.534+0000 7f7678aef640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76740fe430 0x7f7674194c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43242/0 (socket says 172.21.15.114:43242) 2024-09-13T18:22:19.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.535+0000 7f7678aef640 1 -- 172.21.15.114:0/889664911 learned_addr learned my addr 172.21.15.114:0/889664911 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:19.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.535+0000 7f7673fff640 1 --2- 172.21.15.114:0/889664911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76740ff620 0x7f7674195190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:19.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.535+0000 7f7678aef640 1 -- 172.21.15.114:0/889664911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76740ff620 msgr2=0x7f7674195190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:19.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.535+0000 7f7678aef640 1 --2- 172.21.15.114:0/889664911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76740ff620 0x7f7674195190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:19.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.535+0000 7f7678aef640 1 -- 172.21.15.114:0/889664911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76740fffe0 msgr2=0x7f767419c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:19.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.535+0000 7f7678aef640 1 --2- 172.21.15.114:0/889664911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76740fffe0 0x7f767419c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:19.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.535+0000 7f7678aef640 1 -- 172.21.15.114:0/889664911 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7660009560 con 0x7f76740fe430 2024-09-13T18:22:19.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.535+0000 7f7678aef640 1 --2- 172.21.15.114:0/889664911 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76740fe430 0x7f7674194c50 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f766800d770 tx=0x7f766800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:19.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.535+0000 7f7671ffb640 1 -- 172.21.15.114:0/889664911 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7668009c30 con 0x7f76740fe430 2024-09-13T18:22:19.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.535+0000 7f7671ffb640 1 -- 172.21.15.114:0/889664911 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f766800de80 con 0x7f76740fe430 2024-09-13T18:22:19.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.535+0000 7f767ad53640 1 -- 172.21.15.114:0/889664911 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f767419cd40 con 0x7f76740fe430 2024-09-13T18:22:19.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.535+0000 7f7671ffb640 1 -- 172.21.15.114:0/889664911 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7668003d40 con 0x7f76740fe430 2024-09-13T18:22:19.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.535+0000 7f767ad53640 1 -- 172.21.15.114:0/889664911 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f767419d1e0 con 0x7f76740fe430 2024-09-13T18:22:19.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.536+0000 7f767ad53640 1 -- 172.21.15.114:0/889664911 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76741059b0 con 0x7f76740fe430 2024-09-13T18:22:19.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.537+0000 7f7671ffb640 1 -- 172.21.15.114:0/889664911 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f7668003ea0 con 0x7f76740fe430 2024-09-13T18:22:19.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.538+0000 7f7671ffb640 1 --2- 172.21.15.114:0/889664911 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f76640a1110 0x7f76640a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:19.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.538+0000 7f7673fff640 1 --2- 172.21.15.114:0/889664911 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f76640a1110 0x7f76640a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:19.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.538+0000 7f7671ffb640 1 -- 172.21.15.114:0/889664911 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f76680cc700 con 0x7f76740fe430 2024-09-13T18:22:19.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.539+0000 7f7673fff640 1 --2- 172.21.15.114:0/889664911 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f76640a1110 0x7f76640a35d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7660000c00 tx=0x7f7660005a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:19.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.542+0000 7f7671ffb640 1 -- 172.21.15.114:0/889664911 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f7668095a30 con 0x7f76740fe430 2024-09-13T18:22:19.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.738+0000 7f767ad53640 1 -- 172.21.15.114:0/889664911 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f767402c4b0 con 0x7f76740fe430 2024-09-13T18:22:19.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.739+0000 7f7671ffb640 1 -- 172.21.15.114:0/889664911 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10930 (secure 0 0 0) 0x7f7668097960 con 0x7f76740fe430 2024-09-13T18:22:19.741 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:22:19.741 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4162817497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4162817497},{"type":"v1","addr":"172.21.15.114:6835","nonce":4162817497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/2435428478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":2435428478},{"type":"v1","addr":"172.21.15.114:6837","nonce":2435428478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T18:20:01.895117+0000","modified":"2024-09-13T18:20:29.250121+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24482,"mds_2":24475,"mds_3":24458,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/2046516451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":2046516451},{"type":"v1","addr":"172.21.15.114:6839","nonce":2046516451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":10,"addr":"172.21.15.114:6841/1956923446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":1956923446},{"type":"v1","addr":"172.21.15.114:6841","nonce":1956923446}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24391":{"gid":24391,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/507301496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":507301496},{"type":"v1","addr":"172.21.15.204:6833","nonce":507301496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4028657219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4028657219},{"type":"v1","addr":"172.21.15.134:6833","nonce":4028657219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/57029119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":57029119},{"type":"v1","addr":"172.21.15.204:6835","nonce":57029119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3068880939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3068880939},{"type":"v1","addr":"172.21.15.204:6837","nonce":3068880939}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"e","rank":3,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.134:6835/1150003275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1150003275},{"type":"v1","addr":"172.21.15.134:6835","nonce":1150003275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/344883912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":344883912},{"type":"v1","addr":"172.21.15.204:6839","nonce":344883912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6837/891692210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":891692210},{"type":"v1","addr":"172.21.15.134:6837","nonce":891692210}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6839/3466309724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3466309724},{"type":"v1","addr":"172.21.15.134:6839","nonce":3466309724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:22:19.742 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 29 2024-09-13T18:22:19.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.741+0000 7f767ad53640 1 -- 172.21.15.114:0/889664911 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f76640a1110 msgr2=0x7f76640a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:19.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.741+0000 7f767ad53640 1 --2- 172.21.15.114:0/889664911 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f76640a1110 0x7f76640a35d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7660000c00 tx=0x7f7660005a40 comp rx=0 tx=0).stop 2024-09-13T18:22:19.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.741+0000 7f767ad53640 1 -- 172.21.15.114:0/889664911 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76740fe430 msgr2=0x7f7674194c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:19.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.741+0000 7f767ad53640 1 --2- 172.21.15.114:0/889664911 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76740fe430 0x7f7674194c50 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f766800d770 tx=0x7f766800dc30 comp rx=0 tx=0).stop 2024-09-13T18:22:19.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.741+0000 7f767ad53640 1 -- 172.21.15.114:0/889664911 shutdown_connections 2024-09-13T18:22:19.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.741+0000 7f767ad53640 1 --2- 172.21.15.114:0/889664911 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f76640a1110 0x7f76640a35d0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:19.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.741+0000 7f767ad53640 1 --2- 172.21.15.114:0/889664911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76740fffe0 0x7f767419c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:19.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.741+0000 7f767ad53640 1 --2- 172.21.15.114:0/889664911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76740ff620 0x7f7674195190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:19.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.741+0000 7f767ad53640 1 --2- 172.21.15.114:0/889664911 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76740fe430 0x7f7674194c50 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:19.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.741+0000 7f767ad53640 1 -- 172.21.15.114:0/889664911 >> 172.21.15.114:0/889664911 conn(0x7f76740f9bb0 msgr2=0x7f7674102b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:19.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.741+0000 7f767ad53640 1 -- 172.21.15.114:0/889664911 shutdown_connections 2024-09-13T18:22:19.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.741+0000 7f767ad53640 1 -- 172.21.15.114:0/889664911 wait complete. 2024-09-13T18:22:19.754 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:22:19.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.851+0000 7f37e0661640 1 -- 172.21.15.114:0/2815454263 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37d8101050 msgr2=0x7f37d81014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:19.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.851+0000 7f37e0661640 1 --2- 172.21.15.114:0/2815454263 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37d8101050 0x7f37d81014d0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f37cc009900 tx=0x7f37cc02a410 comp rx=0 tx=0).stop 2024-09-13T18:22:19.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.851+0000 7f37e0661640 1 -- 172.21.15.114:0/2815454263 shutdown_connections 2024-09-13T18:22:19.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.851+0000 7f37e0661640 1 --2- 172.21.15.114:0/2815454263 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37d8101a10 0x7f37d81082a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:19.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.851+0000 7f37e0661640 1 --2- 172.21.15.114:0/2815454263 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37d8101050 0x7f37d81014d0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:19.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.851+0000 7f37e0661640 1 --2- 172.21.15.114:0/2815454263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37d80fe080 0x7f37d80fe480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:19.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.851+0000 7f37e0661640 1 -- 172.21.15.114:0/2815454263 >> 172.21.15.114:0/2815454263 conn(0x7f37d80f9bc0 msgr2=0x7f37d80fbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:19.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.851+0000 7f37e0661640 1 -- 172.21.15.114:0/2815454263 shutdown_connections 2024-09-13T18:22:19.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.851+0000 7f37e0661640 1 -- 172.21.15.114:0/2815454263 wait complete. 2024-09-13T18:22:19.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.852+0000 7f37e0661640 1 Processor -- start 2024-09-13T18:22:19.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.852+0000 7f37e0661640 1 -- start start 2024-09-13T18:22:19.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.852+0000 7f37e0661640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37d80fe080 0x7f37d8195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:19.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.852+0000 7f37e0661640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37d8101050 0x7f37d8196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:19.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.852+0000 7f37e0661640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37d8101a10 0x7f37d819d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:19.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.852+0000 7f37e0661640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d819d8b0 con 0x7f37d80fe080 2024-09-13T18:22:19.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.852+0000 7f37e0661640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d819da20 con 0x7f37d8101a10 2024-09-13T18:22:19.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.852+0000 7f37e0661640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d819dbe0 con 0x7f37d8101050 2024-09-13T18:22:19.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.852+0000 7f37de3fd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37d80fe080 0x7f37d8195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:19.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.852+0000 7f37de3fd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37d80fe080 0x7f37d8195dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43264/0 (socket says 172.21.15.114:43264) 2024-09-13T18:22:19.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.852+0000 7f37de3fd640 1 -- 172.21.15.114:0/4216458725 learned_addr learned my addr 172.21.15.114:0/4216458725 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:19.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.852+0000 7f37ddbfc640 1 --2- 172.21.15.114:0/4216458725 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37d8101050 0x7f37d8196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:19.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.852+0000 7f37de3fd640 1 -- 172.21.15.114:0/4216458725 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37d8101050 msgr2=0x7f37d8196300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:19.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.852+0000 7f37de3fd640 1 --2- 172.21.15.114:0/4216458725 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37d8101050 0x7f37d8196300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:19.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.852+0000 7f37de3fd640 1 -- 172.21.15.114:0/4216458725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37d8101a10 msgr2=0x7f37d819d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:19.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.852+0000 7f37debfe640 1 --2- 172.21.15.114:0/4216458725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37d8101a10 0x7f37d819d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:19.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.852+0000 7f37de3fd640 1 --2- 172.21.15.114:0/4216458725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37d8101a10 0x7f37d819d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:19.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.852+0000 7f37de3fd640 1 -- 172.21.15.114:0/4216458725 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37cc009560 con 0x7f37d80fe080 2024-09-13T18:22:19.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.852+0000 7f37ddbfc640 1 --2- 172.21.15.114:0/4216458725 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37d8101050 0x7f37d8196300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:22:19.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.852+0000 7f37de3fd640 1 --2- 172.21.15.114:0/4216458725 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37d80fe080 0x7f37d8195dc0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f37d000ea80 tx=0x7f37d000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:19.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.852+0000 7f37d77fe640 1 -- 172.21.15.114:0/4216458725 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d000ce40 con 0x7f37d80fe080 2024-09-13T18:22:19.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.853+0000 7f37d77fe640 1 -- 172.21.15.114:0/4216458725 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f37d0011900 con 0x7f37d80fe080 2024-09-13T18:22:19.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.853+0000 7f37e0661640 1 -- 172.21.15.114:0/4216458725 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37d819dec0 con 0x7f37d80fe080 2024-09-13T18:22:19.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.853+0000 7f37d77fe640 1 -- 172.21.15.114:0/4216458725 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d001a970 con 0x7f37d80fe080 2024-09-13T18:22:19.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.853+0000 7f37e0661640 1 -- 172.21.15.114:0/4216458725 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37d819e2e0 con 0x7f37d80fe080 2024-09-13T18:22:19.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.854+0000 7f37e0661640 1 -- 172.21.15.114:0/4216458725 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37d80fe480 con 0x7f37d80fe080 2024-09-13T18:22:19.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.855+0000 7f37d77fe640 1 -- 172.21.15.114:0/4216458725 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f37d0011470 con 0x7f37d80fe080 2024-09-13T18:22:19.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.855+0000 7f37d77fe640 1 --2- 172.21.15.114:0/4216458725 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f37c00a1090 0x7f37c00a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:19.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.855+0000 7f37d77fe640 1 -- 172.21.15.114:0/4216458725 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f37d00cd2e0 con 0x7f37d80fe080 2024-09-13T18:22:19.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.855+0000 7f37ddbfc640 1 --2- 172.21.15.114:0/4216458725 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f37c00a1090 0x7f37c00a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:19.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.856+0000 7f37ddbfc640 1 --2- 172.21.15.114:0/4216458725 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f37c00a1090 0x7f37c00a3550 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f37cc005ab0 tx=0x7f37cc005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:19.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:19.859+0000 7f37d77fe640 1 -- 172.21.15.114:0/4216458725 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f37d0096700 con 0x7f37d80fe080 2024-09-13T18:22:20.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.023+0000 7f37e0661640 1 -- 172.21.15.114:0/4216458725 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f37d81074d0 con 0x7f37d80fe080 2024-09-13T18:22:20.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.025+0000 7f37d77fe640 1 -- 172.21.15.114:0/4216458725 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24696 (secure 0 0 0) 0x7f37d0099d20 con 0x7f37d80fe080 2024-09-13T18:22:20.027 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:22:20.027 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":93,"fsid":"97e7edf2-71fb-11ef-bceb-c7b262605968","created":"2024-09-13T18:12:36.360625+0000","modified":"2024-09-13T18:21:28.143797+0000","last_up_change":"2024-09-13T18:19:12.350370+0000","last_in_change":"2024-09-13T18:18:52.321596+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:15:40.129659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:20:00.024407+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:20:01.085549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T18:20:04.334216+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9914f01a-9f6f-4878-b955-83570199c354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6803","nonce":3499525826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6805","nonce":3499525826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6809","nonce":3499525826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6807","nonce":3499525826}]},"public_addr":"172.21.15.114:6803/3499525826","cluster_addr":"172.21.15.114:6805/3499525826","heartbeat_back_addr":"172.21.15.114:6809/3499525826","heartbeat_front_addr":"172.21.15.114:6807/3499525826","state":["exists","up"]},{"osd":1,"uuid":"ef65e012-7469-4c20-b354-17022bca63c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6801","nonce":4264264866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6803","nonce":4264264866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6807","nonce":4264264866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6805","nonce":4264264866}]},"public_addr":"172.21.15.134:6801/4264264866","cluster_addr":"172.21.15.134:6803/4264264866","heartbeat_back_addr":"172.21.15.134:6807/4264264866","heartbeat_front_addr":"172.21.15.134:6805/4264264866","state":["exists","up"]},{"osd":2,"uuid":"578213fd-715f-4be7-91fa-254f3d0e3022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6801","nonce":2288807707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6803","nonce":2288807707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6807","nonce":2288807707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6805","nonce":2288807707}]},"public_addr":"172.21.15.204:6801/2288807707","cluster_addr":"172.21.15.204:6803/2288807707","heartbeat_back_addr":"172.21.15.204:6807/2288807707","heartbeat_front_addr":"172.21.15.204:6805/2288807707","state":["exists","up"]},{"osd":3,"uuid":"b64aab3f-1e52-4828-8397-74af5e05f2cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6811","nonce":2904204254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6813","nonce":2904204254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6817","nonce":2904204254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6815","nonce":2904204254}]},"public_addr":"172.21.15.114:6811/2904204254","cluster_addr":"172.21.15.114:6813/2904204254","heartbeat_back_addr":"172.21.15.114:6817/2904204254","heartbeat_front_addr":"172.21.15.114:6815/2904204254","state":["exists","up"]},{"osd":4,"uuid":"f38943d6-39c0-42e1-b427-cb6bb90ac38a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6809","nonce":1773945333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6811","nonce":1773945333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6815","nonce":1773945333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6813","nonce":1773945333}]},"public_addr":"172.21.15.134:6809/1773945333","cluster_addr":"172.21.15.134:6811/1773945333","heartbeat_back_addr":"172.21.15.134:6815/1773945333","heartbeat_front_addr":"172.21.15.134:6813/1773945333","state":["exists","up"]},{"osd":5,"uuid":"bb38f361-8e00-4178-9752-5ddbf3af20e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6809","nonce":2371778055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6811","nonce":2371778055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6815","nonce":2371778055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6813","nonce":2371778055}]},"public_addr":"172.21.15.204:6809/2371778055","cluster_addr":"172.21.15.204:6811/2371778055","heartbeat_back_addr":"172.21.15.204:6815/2371778055","heartbeat_front_addr":"172.21.15.204:6813/2371778055","state":["exists","up"]},{"osd":6,"uuid":"93d40a03-e7ff-4a3f-9efa-5c5403c9983d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6819","nonce":1484325044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6821","nonce":1484325044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6825","nonce":1484325044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6823","nonce":1484325044}]},"public_addr":"172.21.15.114:6819/1484325044","cluster_addr":"172.21.15.114:6821/1484325044","heartbeat_back_addr":"172.21.15.114:6825/1484325044","heartbeat_front_addr":"172.21.15.114:6823/1484325044","state":["exists","up"]},{"osd":7,"uuid":"982f629a-1b95-4be1-8e75-5ddce1ad77b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6817","nonce":799246614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6819","nonce":799246614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6823","nonce":799246614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6821","nonce":799246614}]},"public_addr":"172.21.15.134:6817/799246614","cluster_addr":"172.21.15.134:6819/799246614","heartbeat_back_addr":"172.21.15.134:6823/799246614","heartbeat_front_addr":"172.21.15.134:6821/799246614","state":["exists","up"]},{"osd":8,"uuid":"409beb9b-8227-4dad-a4a0-f0e5ce70dc5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6817","nonce":858331805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6819","nonce":858331805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6823","nonce":858331805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6821","nonce":858331805}]},"public_addr":"172.21.15.204:6817/858331805","cluster_addr":"172.21.15.204:6819/858331805","heartbeat_back_addr":"172.21.15.204:6823/858331805","heartbeat_front_addr":"172.21.15.204:6821/858331805","state":["exists","up"]},{"osd":9,"uuid":"a345a15c-b1fc-4877-b445-418eebf0d0ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6827","nonce":2172392854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6829","nonce":2172392854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6833","nonce":2172392854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6831","nonce":2172392854}]},"public_addr":"172.21.15.114:6827/2172392854","cluster_addr":"172.21.15.114:6829/2172392854","heartbeat_back_addr":"172.21.15.114:6833/2172392854","heartbeat_front_addr":"172.21.15.114:6831/2172392854","state":["exists","up"]},{"osd":10,"uuid":"464d995f-d90b-4caa-addb-1f7562bd74b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6825","nonce":3073513468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6827","nonce":3073513468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6831","nonce":3073513468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6829","nonce":3073513468}]},"public_addr":"172.21.15.134:6825/3073513468","cluster_addr":"172.21.15.134:6827/3073513468","heartbeat_back_addr":"172.21.15.134:6831/3073513468","heartbeat_front_addr":"172.21.15.134:6829/3073513468","state":["exists","up"]},{"osd":11,"uuid":"71dd794d-326c-477f-8d02-ee32bff1350d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6825","nonce":3239521765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6827","nonce":3239521765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6831","nonce":3239521765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6829","nonce":3239521765}]},"public_addr":"172.21.15.204:6825/3239521765","cluster_addr":"172.21.15.204:6827/3239521765","heartbeat_back_addr":"172.21.15.204:6831/3239521765","heartbeat_front_addr":"172.21.15.204:6829/3239521765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:14:54.542687+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:15:15.746055+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:15:38.371131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:01.383932+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:24.796237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:48.422214+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:11.284323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:34.570365+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:58.232855+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:18:21.609596+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:18:45.565859+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:19:10.017969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:6801/4281866858":"2024-09-14T18:13:25.823027+0000","172.21.15.114:6800/4281866858":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2949422849":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2034742251":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/722571999":"2024-09-14T18:13:03.521116+0000","172.21.15.114:6801/830738794":"2024-09-14T18:13:03.521116+0000","172.21.15.114:6800/830738794":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/2828364892":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/4257828906":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/3544951322":"2024-09-14T18:13:03.521116+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:22:20.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.027+0000 7f37e0661640 1 -- 172.21.15.114:0/4216458725 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f37c00a1090 msgr2=0x7f37c00a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:20.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.027+0000 7f37e0661640 1 --2- 172.21.15.114:0/4216458725 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f37c00a1090 0x7f37c00a3550 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f37cc005ab0 tx=0x7f37cc005a20 comp rx=0 tx=0).stop 2024-09-13T18:22:20.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.027+0000 7f37e0661640 1 -- 172.21.15.114:0/4216458725 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37d80fe080 msgr2=0x7f37d8195dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:20.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.027+0000 7f37e0661640 1 --2- 172.21.15.114:0/4216458725 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37d80fe080 0x7f37d8195dc0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f37d000ea80 tx=0x7f37d000ef40 comp rx=0 tx=0).stop 2024-09-13T18:22:20.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.027+0000 7f37e0661640 1 -- 172.21.15.114:0/4216458725 shutdown_connections 2024-09-13T18:22:20.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.027+0000 7f37e0661640 1 --2- 172.21.15.114:0/4216458725 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f37c00a1090 0x7f37c00a3550 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:20.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.027+0000 7f37e0661640 1 --2- 172.21.15.114:0/4216458725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37d8101a10 0x7f37d819d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:20.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.027+0000 7f37e0661640 1 --2- 172.21.15.114:0/4216458725 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37d8101050 0x7f37d8196300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:20.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.027+0000 7f37e0661640 1 --2- 172.21.15.114:0/4216458725 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37d80fe080 0x7f37d8195dc0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:20.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.027+0000 7f37e0661640 1 -- 172.21.15.114:0/4216458725 >> 172.21.15.114:0/4216458725 conn(0x7f37d80f9bc0 msgr2=0x7f37d81046a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:20.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.027+0000 7f37e0661640 1 -- 172.21.15.114:0/4216458725 shutdown_connections 2024-09-13T18:22:20.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.027+0000 7f37e0661640 1 -- 172.21.15.114:0/4216458725 wait complete. 2024-09-13T18:22:20.040 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T18:15:40.129659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T18:20:00.024407+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T18:20:01.085549+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T18:20:04.334216+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T18:22:20.041 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T18:22:20.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.140+0000 7f7a4bde9640 1 -- 172.21.15.114:0/3457943850 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a44100810 msgr2=0x7f7a44102c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:20.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.140+0000 7f7a4bde9640 1 --2- 172.21.15.114:0/3457943850 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a44100810 0x7f7a44102c00 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7a40009900 tx=0x7f7a4002a410 comp rx=0 tx=0).stop 2024-09-13T18:22:20.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.140+0000 7f7a4bde9640 1 -- 172.21.15.114:0/3457943850 shutdown_connections 2024-09-13T18:22:20.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.140+0000 7f7a4bde9640 1 --2- 172.21.15.114:0/3457943850 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a44105a70 0x7f7a44107ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:20.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.140+0000 7f7a4bde9640 1 --2- 172.21.15.114:0/3457943850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a44103140 0x7f7a44105530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:20.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.140+0000 7f7a4bde9640 1 --2- 172.21.15.114:0/3457943850 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a44100810 0x7f7a44102c00 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:20.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.140+0000 7f7a4bde9640 1 -- 172.21.15.114:0/3457943850 >> 172.21.15.114:0/3457943850 conn(0x7f7a440fa3f0 msgr2=0x7f7a440fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:20.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.140+0000 7f7a4bde9640 1 -- 172.21.15.114:0/3457943850 shutdown_connections 2024-09-13T18:22:20.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.141+0000 7f7a4bde9640 1 -- 172.21.15.114:0/3457943850 wait complete. 2024-09-13T18:22:20.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.141+0000 7f7a4bde9640 1 Processor -- start 2024-09-13T18:22:20.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.141+0000 7f7a4bde9640 1 -- start start 2024-09-13T18:22:20.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.141+0000 7f7a4bde9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a44100810 0x7f7a44195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:20.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.141+0000 7f7a4bde9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a44103140 0x7f7a44196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:20.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.141+0000 7f7a4bde9640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a44105a70 0x7f7a4419d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:20.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.141+0000 7f7a4bde9640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4419d8a0 con 0x7f7a44105a70 2024-09-13T18:22:20.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.141+0000 7f7a4bde9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4419da10 con 0x7f7a44100810 2024-09-13T18:22:20.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.141+0000 7f7a4bde9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4419dbd0 con 0x7f7a44103140 2024-09-13T18:22:20.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.141+0000 7f7a4a386640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a44105a70 0x7f7a4419d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:20.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.141+0000 7f7a4a386640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a44105a70 0x7f7a4419d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43288/0 (socket says 172.21.15.114:43288) 2024-09-13T18:22:20.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.141+0000 7f7a4a386640 1 -- 172.21.15.114:0/1288329365 learned_addr learned my addr 172.21.15.114:0/1288329365 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:20.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.142+0000 7f7a49b85640 1 --2- 172.21.15.114:0/1288329365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a44100810 0x7f7a44195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:20.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.142+0000 7f7a49384640 1 --2- 172.21.15.114:0/1288329365 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a44103140 0x7f7a44196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:20.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.142+0000 7f7a4a386640 1 -- 172.21.15.114:0/1288329365 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a44103140 msgr2=0x7f7a44196300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:20.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.142+0000 7f7a4a386640 1 --2- 172.21.15.114:0/1288329365 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a44103140 0x7f7a44196300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:20.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.142+0000 7f7a4a386640 1 -- 172.21.15.114:0/1288329365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a44100810 msgr2=0x7f7a44195dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:20.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.142+0000 7f7a4a386640 1 --2- 172.21.15.114:0/1288329365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a44100810 0x7f7a44195dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:20.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.142+0000 7f7a4a386640 1 -- 172.21.15.114:0/1288329365 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a40009560 con 0x7f7a44105a70 2024-09-13T18:22:20.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.142+0000 7f7a49b85640 1 --2- 172.21.15.114:0/1288329365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a44100810 0x7f7a44195dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:22:20.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.142+0000 7f7a4a386640 1 --2- 172.21.15.114:0/1288329365 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a44105a70 0x7f7a4419d330 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c00b9e0 tx=0x7f7a3c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:20.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.142+0000 7f7a3affd640 1 -- 172.21.15.114:0/1288329365 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c00cb20 con 0x7f7a44105a70 2024-09-13T18:22:20.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.142+0000 7f7a3affd640 1 -- 172.21.15.114:0/1288329365 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7a3c00cc80 con 0x7f7a44105a70 2024-09-13T18:22:20.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.142+0000 7f7a4bde9640 1 -- 172.21.15.114:0/1288329365 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a4419deb0 con 0x7f7a44105a70 2024-09-13T18:22:20.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.142+0000 7f7a3affd640 1 -- 172.21.15.114:0/1288329365 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c0125a0 con 0x7f7a44105a70 2024-09-13T18:22:20.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.142+0000 7f7a4bde9640 1 -- 172.21.15.114:0/1288329365 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a4419e2d0 con 0x7f7a44105a70 2024-09-13T18:22:20.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.143+0000 7f7a4bde9640 1 -- 172.21.15.114:0/1288329365 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a440fbee0 con 0x7f7a44105a70 2024-09-13T18:22:20.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.144+0000 7f7a3affd640 1 -- 172.21.15.114:0/1288329365 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f7a3c014440 con 0x7f7a44105a70 2024-09-13T18:22:20.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.145+0000 7f7a3affd640 1 --2- 172.21.15.114:0/1288329365 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f7a300a1110 0x7f7a300a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:20.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.145+0000 7f7a49b85640 1 --2- 172.21.15.114:0/1288329365 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f7a300a1110 0x7f7a300a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:20.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.145+0000 7f7a3affd640 1 -- 172.21.15.114:0/1288329365 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f7a3c0ccb10 con 0x7f7a44105a70 2024-09-13T18:22:20.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.145+0000 7f7a49b85640 1 --2- 172.21.15.114:0/1288329365 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f7a300a1110 0x7f7a300a35d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7a40005ab0 tx=0x7f7a40005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:20.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.148+0000 7f7a3affd640 1 -- 172.21.15.114:0/1288329365 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f7a3c095ea0 con 0x7f7a44105a70 2024-09-13T18:22:20.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.308+0000 7f7a4bde9640 1 -- 172.21.15.114:0/1288329365 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7a4402c480 con 0x7f7a44105a70 2024-09-13T18:22:20.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.308+0000 7f7a3affd640 1 -- 172.21.15.114:0/1288329365 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7a3c019020 con 0x7f7a44105a70 2024-09-13T18:22:20.311 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 1 2024-09-13T18:22:20.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.310+0000 7f7a4bde9640 1 -- 172.21.15.114:0/1288329365 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f7a300a1110 msgr2=0x7f7a300a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:20.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.310+0000 7f7a4bde9640 1 --2- 172.21.15.114:0/1288329365 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f7a300a1110 0x7f7a300a35d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7a40005ab0 tx=0x7f7a40005a20 comp rx=0 tx=0).stop 2024-09-13T18:22:20.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.310+0000 7f7a4bde9640 1 -- 172.21.15.114:0/1288329365 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a44105a70 msgr2=0x7f7a4419d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:20.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.310+0000 7f7a4bde9640 1 --2- 172.21.15.114:0/1288329365 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a44105a70 0x7f7a4419d330 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c00b9e0 tx=0x7f7a3c00bea0 comp rx=0 tx=0).stop 2024-09-13T18:22:20.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.310+0000 7f7a4bde9640 1 -- 172.21.15.114:0/1288329365 shutdown_connections 2024-09-13T18:22:20.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.310+0000 7f7a4bde9640 1 --2- 172.21.15.114:0/1288329365 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f7a300a1110 0x7f7a300a35d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:20.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.310+0000 7f7a4bde9640 1 --2- 172.21.15.114:0/1288329365 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a44105a70 0x7f7a4419d330 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:20.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.310+0000 7f7a4bde9640 1 --2- 172.21.15.114:0/1288329365 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a44103140 0x7f7a44196300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:20.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.310+0000 7f7a4bde9640 1 --2- 172.21.15.114:0/1288329365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a44100810 0x7f7a44195dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:20.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.310+0000 7f7a4bde9640 1 -- 172.21.15.114:0/1288329365 >> 172.21.15.114:0/1288329365 conn(0x7f7a440fa3f0 msgr2=0x7f7a44101050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:20.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.310+0000 7f7a4bde9640 1 -- 172.21.15.114:0/1288329365 shutdown_connections 2024-09-13T18:22:20.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.310+0000 7f7a4bde9640 1 -- 172.21.15.114:0/1288329365 wait complete. 2024-09-13T18:22:20.322 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:20 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/1422518311' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T18:22:20.322 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:20 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/319272332' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T18:22:20.322 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:20 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/889664911' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:22:20.322 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:20 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/4216458725' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:22:20.323 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T18:22:20.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:20 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/1422518311' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T18:22:20.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:20 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/319272332' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T18:22:20.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:20 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/889664911' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:22:20.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:20 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/4216458725' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:22:20.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.423+0000 7fbb1b508640 1 -- 172.21.15.114:0/3008835253 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb140ffe80 msgr2=0x7fbb14100300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:20.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.423+0000 7fbb1b508640 1 --2- 172.21.15.114:0/3008835253 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb140ffe80 0x7fbb14100300 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fbb04009900 tx=0x7fbb0402a410 comp rx=0 tx=0).stop 2024-09-13T18:22:20.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.423+0000 7fbb1b508640 1 -- 172.21.15.114:0/3008835253 shutdown_connections 2024-09-13T18:22:20.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.423+0000 7fbb1b508640 1 --2- 172.21.15.114:0/3008835253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb14100840 0x7fbb141070d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:20.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.423+0000 7fbb1b508640 1 --2- 172.21.15.114:0/3008835253 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb140ffe80 0x7fbb14100300 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:20.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.423+0000 7fbb1b508640 1 --2- 172.21.15.114:0/3008835253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb140fec90 0x7fbb140ff090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:20.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.423+0000 7fbb1b508640 1 -- 172.21.15.114:0/3008835253 >> 172.21.15.114:0/3008835253 conn(0x7fbb140fa3f0 msgr2=0x7fbb140fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:20.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.423+0000 7fbb1b508640 1 -- 172.21.15.114:0/3008835253 shutdown_connections 2024-09-13T18:22:20.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.423+0000 7fbb1b508640 1 -- 172.21.15.114:0/3008835253 wait complete. 2024-09-13T18:22:20.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.424+0000 7fbb1b508640 1 Processor -- start 2024-09-13T18:22:20.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.424+0000 7fbb1b508640 1 -- start start 2024-09-13T18:22:20.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.424+0000 7fbb1b508640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb140fec90 0x7fbb14195cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:20.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.424+0000 7fbb1b508640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb140ffe80 0x7fbb141961f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:20.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.424+0000 7fbb1b508640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb14100840 0x7fbb1419d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:20.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.424+0000 7fbb1b508640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1419d7c0 con 0x7fbb14100840 2024-09-13T18:22:20.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.424+0000 7fbb1b508640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1419d930 con 0x7fbb140ffe80 2024-09-13T18:22:20.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.424+0000 7fbb1b508640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1419daf0 con 0x7fbb140fec90 2024-09-13T18:22:20.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.424+0000 7fbb19aa5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb14100840 0x7fbb1419d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:20.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.424+0000 7fbb19aa5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb14100840 0x7fbb1419d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43314/0 (socket says 172.21.15.114:43314) 2024-09-13T18:22:20.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.424+0000 7fbb19aa5640 1 -- 172.21.15.114:0/2713621921 learned_addr learned my addr 172.21.15.114:0/2713621921 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:20.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.424+0000 7fbb18aa3640 1 --2- 172.21.15.114:0/2713621921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb140ffe80 0x7fbb141961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:20.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.424+0000 7fbb192a4640 1 --2- 172.21.15.114:0/2713621921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb140fec90 0x7fbb14195cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:20.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.424+0000 7fbb19aa5640 1 -- 172.21.15.114:0/2713621921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb140fec90 msgr2=0x7fbb14195cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:20.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.424+0000 7fbb19aa5640 1 --2- 172.21.15.114:0/2713621921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb140fec90 0x7fbb14195cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:20.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.424+0000 7fbb19aa5640 1 -- 172.21.15.114:0/2713621921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb140ffe80 msgr2=0x7fbb141961f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:20.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.424+0000 7fbb19aa5640 1 --2- 172.21.15.114:0/2713621921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb140ffe80 0x7fbb141961f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:20.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.424+0000 7fbb19aa5640 1 -- 172.21.15.114:0/2713621921 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb04009560 con 0x7fbb14100840 2024-09-13T18:22:20.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.424+0000 7fbb192a4640 1 --2- 172.21.15.114:0/2713621921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb140fec90 0x7fbb14195cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:22:20.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.424+0000 7fbb19aa5640 1 --2- 172.21.15.114:0/2713621921 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb14100840 0x7fbb1419d250 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c00eb10 tx=0x7fbb0c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:20.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.425+0000 7fbb0a7fc640 1 -- 172.21.15.114:0/2713621921 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c004b20 con 0x7fbb14100840 2024-09-13T18:22:20.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.425+0000 7fbb1b508640 1 -- 172.21.15.114:0/2713621921 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb1419ddd0 con 0x7fbb14100840 2024-09-13T18:22:20.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.425+0000 7fbb0a7fc640 1 -- 172.21.15.114:0/2713621921 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbb0c004c80 con 0x7fbb14100840 2024-09-13T18:22:20.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.425+0000 7fbb0a7fc640 1 -- 172.21.15.114:0/2713621921 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c005a10 con 0x7fbb14100840 2024-09-13T18:22:20.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.425+0000 7fbb1b508640 1 -- 172.21.15.114:0/2713621921 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb1419e200 con 0x7fbb14100840 2024-09-13T18:22:20.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.426+0000 7fbb1b508640 1 -- 172.21.15.114:0/2713621921 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb141061c0 con 0x7fbb14100840 2024-09-13T18:22:20.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.427+0000 7fbb0a7fc640 1 -- 172.21.15.114:0/2713621921 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fbb0c005020 con 0x7fbb14100840 2024-09-13T18:22:20.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.427+0000 7fbb0a7fc640 1 --2- 172.21.15.114:0/2713621921 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fbb000a1090 0x7fbb000a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:20.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.427+0000 7fbb192a4640 1 --2- 172.21.15.114:0/2713621921 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fbb000a1090 0x7fbb000a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:20.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.427+0000 7fbb0a7fc640 1 -- 172.21.15.114:0/2713621921 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fbb0c014070 con 0x7fbb14100840 2024-09-13T18:22:20.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.428+0000 7fbb192a4640 1 --2- 172.21.15.114:0/2713621921 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fbb000a1090 0x7fbb000a3550 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbb10005fd0 tx=0x7fbb10005ec0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:20.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.431+0000 7fbb0a7fc640 1 -- 172.21.15.114:0/2713621921 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fbb0c0966b0 con 0x7fbb14100840 2024-09-13T18:22:20.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:20 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/1422518311' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T18:22:20.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:20 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/319272332' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T18:22:20.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:20 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/889664911' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:22:20.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:20 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/4216458725' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:22:20.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.593+0000 7fbb1b508640 1 -- 172.21.15.114:0/2713621921 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbb1402c3e0 con 0x7fbb14100840 2024-09-13T18:22:20.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.593+0000 7fbb0a7fc640 1 -- 172.21.15.114:0/2713621921 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbb0c099cd0 con 0x7fbb14100840 2024-09-13T18:22:20.595 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2024-09-13T18:22:20.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.595+0000 7fbb1b508640 1 -- 172.21.15.114:0/2713621921 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fbb000a1090 msgr2=0x7fbb000a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:20.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.595+0000 7fbb1b508640 1 --2- 172.21.15.114:0/2713621921 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fbb000a1090 0x7fbb000a3550 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbb10005fd0 tx=0x7fbb10005ec0 comp rx=0 tx=0).stop 2024-09-13T18:22:20.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.595+0000 7fbb1b508640 1 -- 172.21.15.114:0/2713621921 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb14100840 msgr2=0x7fbb1419d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:20.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.595+0000 7fbb1b508640 1 --2- 172.21.15.114:0/2713621921 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb14100840 0x7fbb1419d250 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c00eb10 tx=0x7fbb0c00c2d0 comp rx=0 tx=0).stop 2024-09-13T18:22:20.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.595+0000 7fbb1b508640 1 -- 172.21.15.114:0/2713621921 shutdown_connections 2024-09-13T18:22:20.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.595+0000 7fbb1b508640 1 --2- 172.21.15.114:0/2713621921 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fbb000a1090 0x7fbb000a3550 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:20.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.595+0000 7fbb1b508640 1 --2- 172.21.15.114:0/2713621921 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb14100840 0x7fbb1419d250 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:20.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.595+0000 7fbb1b508640 1 --2- 172.21.15.114:0/2713621921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb140ffe80 0x7fbb141961f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:20.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.595+0000 7fbb1b508640 1 --2- 172.21.15.114:0/2713621921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb140fec90 0x7fbb14195cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:20.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.595+0000 7fbb1b508640 1 -- 172.21.15.114:0/2713621921 >> 172.21.15.114:0/2713621921 conn(0x7fbb140fa3f0 msgr2=0x7fbb14103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:20.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.595+0000 7fbb1b508640 1 -- 172.21.15.114:0/2713621921 shutdown_connections 2024-09-13T18:22:20.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.595+0000 7fbb1b508640 1 -- 172.21.15.114:0/2713621921 wait complete. 2024-09-13T18:22:20.608 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T18:22:20.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.707+0000 7ffbddaf7640 1 -- 172.21.15.114:0/1787332666 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffbd80fec90 msgr2=0x7ffbd80ff090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:20.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.707+0000 7ffbddaf7640 1 --2- 172.21.15.114:0/1787332666 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffbd80fec90 0x7ffbd80ff090 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8009900 tx=0x7ffbc802a410 comp rx=0 tx=0).stop 2024-09-13T18:22:20.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.707+0000 7ffbddaf7640 1 -- 172.21.15.114:0/1787332666 shutdown_connections 2024-09-13T18:22:20.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.707+0000 7ffbddaf7640 1 --2- 172.21.15.114:0/1787332666 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbd8100840 0x7ffbd81070d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:20.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.707+0000 7ffbddaf7640 1 --2- 172.21.15.114:0/1787332666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffbd80ffe80 0x7ffbd8100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:20.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.707+0000 7ffbddaf7640 1 --2- 172.21.15.114:0/1787332666 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffbd80fec90 0x7ffbd80ff090 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:20.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.707+0000 7ffbddaf7640 1 -- 172.21.15.114:0/1787332666 >> 172.21.15.114:0/1787332666 conn(0x7ffbd80fa3f0 msgr2=0x7ffbd80fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:20.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.707+0000 7ffbddaf7640 1 -- 172.21.15.114:0/1787332666 shutdown_connections 2024-09-13T18:22:20.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.707+0000 7ffbddaf7640 1 -- 172.21.15.114:0/1787332666 wait complete. 2024-09-13T18:22:20.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.708+0000 7ffbddaf7640 1 Processor -- start 2024-09-13T18:22:20.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.708+0000 7ffbddaf7640 1 -- start start 2024-09-13T18:22:20.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.708+0000 7ffbddaf7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffbd80fec90 0x7ffbd8102c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:20.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.708+0000 7ffbddaf7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbd80ffe80 0x7ffbd8101260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:20.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.708+0000 7ffbddaf7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffbd8100840 0x7ffbd81017a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:20.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.708+0000 7ffbddaf7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd8101f00 con 0x7ffbd80fec90 2024-09-13T18:22:20.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.708+0000 7ffbddaf7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd8102070 con 0x7ffbd80ffe80 2024-09-13T18:22:20.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.708+0000 7ffbddaf7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd819fd50 con 0x7ffbd8100840 2024-09-13T18:22:20.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.708+0000 7ffbd77fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffbd80fec90 0x7ffbd8102c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:20.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.708+0000 7ffbd77fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffbd80fec90 0x7ffbd8102c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43334/0 (socket says 172.21.15.114:43334) 2024-09-13T18:22:20.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.708+0000 7ffbd77fe640 1 -- 172.21.15.114:0/1446116470 learned_addr learned my addr 172.21.15.114:0/1446116470 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:20.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.708+0000 7ffbcffff640 1 --2- 172.21.15.114:0/1446116470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbd80ffe80 0x7ffbd8101260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:20.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.708+0000 7ffbd7fff640 1 --2- 172.21.15.114:0/1446116470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffbd8100840 0x7ffbd81017a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:20.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.708+0000 7ffbd77fe640 1 -- 172.21.15.114:0/1446116470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffbd8100840 msgr2=0x7ffbd81017a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:20.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.708+0000 7ffbd77fe640 1 --2- 172.21.15.114:0/1446116470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffbd8100840 0x7ffbd81017a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:20.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.708+0000 7ffbd77fe640 1 -- 172.21.15.114:0/1446116470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbd80ffe80 msgr2=0x7ffbd8101260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:20.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.708+0000 7ffbd77fe640 1 --2- 172.21.15.114:0/1446116470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbd80ffe80 0x7ffbd8101260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:20.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.708+0000 7ffbd77fe640 1 -- 172.21.15.114:0/1446116470 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc8009560 con 0x7ffbd80fec90 2024-09-13T18:22:20.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.708+0000 7ffbd7fff640 1 --2- 172.21.15.114:0/1446116470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffbd8100840 0x7ffbd81017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:22:20.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.708+0000 7ffbd77fe640 1 --2- 172.21.15.114:0/1446116470 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffbd80fec90 0x7ffbd8102c20 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8009230 tx=0x7ffbc80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:20.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.709+0000 7ffbd57fa640 1 -- 172.21.15.114:0/1446116470 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc8031d70 con 0x7ffbd80fec90 2024-09-13T18:22:20.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.709+0000 7ffbd57fa640 1 -- 172.21.15.114:0/1446116470 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffbc80044e0 con 0x7ffbd80fec90 2024-09-13T18:22:20.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.709+0000 7ffbddaf7640 1 -- 172.21.15.114:0/1446116470 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbd819ffa0 con 0x7ffbd80fec90 2024-09-13T18:22:20.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.709+0000 7ffbd57fa640 1 -- 172.21.15.114:0/1446116470 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc8003b00 con 0x7ffbd80fec90 2024-09-13T18:22:20.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.709+0000 7ffbddaf7640 1 -- 172.21.15.114:0/1446116470 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbd81a0370 con 0x7ffbd80fec90 2024-09-13T18:22:20.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.709+0000 7ffbddaf7640 1 -- 172.21.15.114:0/1446116470 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbd810a3e0 con 0x7ffbd80fec90 2024-09-13T18:22:20.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.711+0000 7ffbd57fa640 1 -- 172.21.15.114:0/1446116470 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7ffbc8004c50 con 0x7ffbd80fec90 2024-09-13T18:22:20.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.711+0000 7ffbd57fa640 1 --2- 172.21.15.114:0/1446116470 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7ffbc40a1090 0x7ffbc40a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:20.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.711+0000 7ffbcffff640 1 --2- 172.21.15.114:0/1446116470 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7ffbc40a1090 0x7ffbc40a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:20.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.711+0000 7ffbd57fa640 1 -- 172.21.15.114:0/1446116470 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7ffbc80ee090 con 0x7ffbd80fec90 2024-09-13T18:22:20.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.712+0000 7ffbcffff640 1 --2- 172.21.15.114:0/1446116470 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7ffbc40a1090 0x7ffbc40a3550 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8102990 tx=0x7ffbc000b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:20.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.714+0000 7ffbd57fa640 1 -- 172.21.15.114:0/1446116470 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7ffbc80b74b0 con 0x7ffbd80fec90 2024-09-13T18:22:20.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.877+0000 7ffbddaf7640 1 -- 172.21.15.114:0/1446116470 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ffbd81a0560 con 0x7ffbd80fec90 2024-09-13T18:22:20.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.878+0000 7ffbd57fa640 1 -- 172.21.15.114:0/1446116470 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7ffbc80400a0 con 0x7ffbd80fec90 2024-09-13T18:22:20.880 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 64 2024-09-13T18:22:20.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.880+0000 7ffbddaf7640 1 -- 172.21.15.114:0/1446116470 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7ffbc40a1090 msgr2=0x7ffbc40a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:20.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.880+0000 7ffbddaf7640 1 --2- 172.21.15.114:0/1446116470 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7ffbc40a1090 0x7ffbc40a3550 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8102990 tx=0x7ffbc000b390 comp rx=0 tx=0).stop 2024-09-13T18:22:20.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.880+0000 7ffbddaf7640 1 -- 172.21.15.114:0/1446116470 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffbd80fec90 msgr2=0x7ffbd8102c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:20.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.880+0000 7ffbddaf7640 1 --2- 172.21.15.114:0/1446116470 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffbd80fec90 0x7ffbd8102c20 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8009230 tx=0x7ffbc80042f0 comp rx=0 tx=0).stop 2024-09-13T18:22:20.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.880+0000 7ffbddaf7640 1 -- 172.21.15.114:0/1446116470 shutdown_connections 2024-09-13T18:22:20.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.880+0000 7ffbddaf7640 1 --2- 172.21.15.114:0/1446116470 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7ffbc40a1090 0x7ffbc40a3550 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:20.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.880+0000 7ffbddaf7640 1 --2- 172.21.15.114:0/1446116470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffbd8100840 0x7ffbd81017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:20.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.880+0000 7ffbddaf7640 1 --2- 172.21.15.114:0/1446116470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbd80ffe80 0x7ffbd8101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:20.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.880+0000 7ffbddaf7640 1 --2- 172.21.15.114:0/1446116470 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffbd80fec90 0x7ffbd8102c20 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:20.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.880+0000 7ffbddaf7640 1 -- 172.21.15.114:0/1446116470 >> 172.21.15.114:0/1446116470 conn(0x7ffbd80fa3f0 msgr2=0x7ffbd8068660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:20.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.880+0000 7ffbddaf7640 1 -- 172.21.15.114:0/1446116470 shutdown_connections 2024-09-13T18:22:20.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.880+0000 7ffbddaf7640 1 -- 172.21.15.114:0/1446116470 wait complete. 2024-09-13T18:22:20.893 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T18:22:20.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.992+0000 7fe364318640 1 -- 172.21.15.114:0/247516016 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe35c100810 msgr2=0x7fe35c1070a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:20.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.992+0000 7fe364318640 1 --2- 172.21.15.114:0/247516016 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe35c100810 0x7fe35c1070a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe354009ca0 tx=0x7fe35402a8c0 comp rx=0 tx=0).stop 2024-09-13T18:22:20.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.993+0000 7fe364318640 1 -- 172.21.15.114:0/247516016 shutdown_connections 2024-09-13T18:22:20.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.993+0000 7fe364318640 1 --2- 172.21.15.114:0/247516016 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe35c100810 0x7fe35c1070a0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:20.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.993+0000 7fe364318640 1 --2- 172.21.15.114:0/247516016 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe35c0ffe50 0x7fe35c1002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:20.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.993+0000 7fe364318640 1 --2- 172.21.15.114:0/247516016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe35c0fec60 0x7fe35c0ff060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:20.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.993+0000 7fe364318640 1 -- 172.21.15.114:0/247516016 >> 172.21.15.114:0/247516016 conn(0x7fe35c0fa3e0 msgr2=0x7fe35c0fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:20.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.993+0000 7fe364318640 1 -- 172.21.15.114:0/247516016 shutdown_connections 2024-09-13T18:22:20.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.993+0000 7fe364318640 1 -- 172.21.15.114:0/247516016 wait complete. 2024-09-13T18:22:20.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.993+0000 7fe364318640 1 Processor -- start 2024-09-13T18:22:20.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.993+0000 7fe364318640 1 -- start start 2024-09-13T18:22:20.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.993+0000 7fe364318640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe35c0fec60 0x7fe35c195c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:20.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.993+0000 7fe364318640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe35c0ffe50 0x7fe35c196170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:20.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.993+0000 7fe364318640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe35c100810 0x7fe35c19d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:20.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.993+0000 7fe364318640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c19d710 con 0x7fe35c0ffe50 2024-09-13T18:22:20.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.993+0000 7fe364318640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c19d880 con 0x7fe35c0fec60 2024-09-13T18:22:20.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.993+0000 7fe364318640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c19da40 con 0x7fe35c100810 2024-09-13T18:22:20.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.993+0000 7fe3618b3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe35c0ffe50 0x7fe35c196170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:20.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.993+0000 7fe3618b3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe35c0ffe50 0x7fe35c196170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43356/0 (socket says 172.21.15.114:43356) 2024-09-13T18:22:20.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.993+0000 7fe3618b3640 1 -- 172.21.15.114:0/2614911608 learned_addr learned my addr 172.21.15.114:0/2614911608 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:20.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.994+0000 7fe3628b5640 1 --2- 172.21.15.114:0/2614911608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe35c100810 0x7fe35c19d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:20.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.994+0000 7fe3620b4640 1 --2- 172.21.15.114:0/2614911608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe35c0fec60 0x7fe35c195c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:20.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.994+0000 7fe3618b3640 1 -- 172.21.15.114:0/2614911608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe35c100810 msgr2=0x7fe35c19d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:20.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.994+0000 7fe3618b3640 1 --2- 172.21.15.114:0/2614911608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe35c100810 0x7fe35c19d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.994+0000 7fe3618b3640 1 -- 172.21.15.114:0/2614911608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe35c0fec60 msgr2=0x7fe35c195c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:21.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.994+0000 7fe3618b3640 1 --2- 172.21.15.114:0/2614911608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe35c0fec60 0x7fe35c195c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.994+0000 7fe3618b3640 1 -- 172.21.15.114:0/2614911608 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe354009900 con 0x7fe35c0ffe50 2024-09-13T18:22:21.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.994+0000 7fe3620b4640 1 --2- 172.21.15.114:0/2614911608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe35c0fec60 0x7fe35c195c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:22:21.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.994+0000 7fe3618b3640 1 --2- 172.21.15.114:0/2614911608 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe35c0ffe50 0x7fe35c196170 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe34c00d950 tx=0x7fe34c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:21.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.994+0000 7fe3537fe640 1 -- 172.21.15.114:0/2614911608 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c009a30 con 0x7fe35c0ffe50 2024-09-13T18:22:21.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.994+0000 7fe3537fe640 1 -- 172.21.15.114:0/2614911608 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe34c0044b0 con 0x7fe35c0ffe50 2024-09-13T18:22:21.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.994+0000 7fe364318640 1 -- 172.21.15.114:0/2614911608 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe35c19dd20 con 0x7fe35c0ffe50 2024-09-13T18:22:21.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.994+0000 7fe3537fe640 1 -- 172.21.15.114:0/2614911608 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c0047a0 con 0x7fe35c0ffe50 2024-09-13T18:22:21.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.994+0000 7fe364318640 1 -- 172.21.15.114:0/2614911608 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe35c19e180 con 0x7fe35c0ffe50 2024-09-13T18:22:21.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.995+0000 7fe364318640 1 -- 172.21.15.114:0/2614911608 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe35c10a3b0 con 0x7fe35c0ffe50 2024-09-13T18:22:21.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.996+0000 7fe3537fe640 1 -- 172.21.15.114:0/2614911608 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fe34c004020 con 0x7fe35c0ffe50 2024-09-13T18:22:21.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.997+0000 7fe3537fe640 1 --2- 172.21.15.114:0/2614911608 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fe3480a1090 0x7fe3480a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:21.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.997+0000 7fe3620b4640 1 --2- 172.21.15.114:0/2614911608 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fe3480a1090 0x7fe3480a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:21.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.997+0000 7fe3537fe640 1 -- 172.21.15.114:0/2614911608 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fe34c0cc7f0 con 0x7fe35c0ffe50 2024-09-13T18:22:21.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:20.997+0000 7fe3620b4640 1 --2- 172.21.15.114:0/2614911608 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fe3480a1090 0x7fe3480a3550 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe35c197100 tx=0x7fe358005d50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:21.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.001+0000 7fe3537fe640 1 -- 172.21.15.114:0/2614911608 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fe34c095b80 con 0x7fe35c0ffe50 2024-09-13T18:22:21.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.165+0000 7fe364318640 1 -- 172.21.15.114:0/2614911608 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe35c19e670 con 0x7fe35c0ffe50 2024-09-13T18:22:21.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.165+0000 7fe3537fe640 1 -- 172.21.15.114:0/2614911608 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe34c097ab0 con 0x7fe35c0ffe50 2024-09-13T18:22:21.168 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2024-09-13T18:22:21.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.167+0000 7fe364318640 1 -- 172.21.15.114:0/2614911608 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fe3480a1090 msgr2=0x7fe3480a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:21.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.167+0000 7fe364318640 1 --2- 172.21.15.114:0/2614911608 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fe3480a1090 0x7fe3480a3550 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe35c197100 tx=0x7fe358005d50 comp rx=0 tx=0).stop 2024-09-13T18:22:21.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.167+0000 7fe364318640 1 -- 172.21.15.114:0/2614911608 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe35c0ffe50 msgr2=0x7fe35c196170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:21.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.167+0000 7fe364318640 1 --2- 172.21.15.114:0/2614911608 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe35c0ffe50 0x7fe35c196170 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe34c00d950 tx=0x7fe34c00de10 comp rx=0 tx=0).stop 2024-09-13T18:22:21.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.167+0000 7fe364318640 1 -- 172.21.15.114:0/2614911608 shutdown_connections 2024-09-13T18:22:21.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.167+0000 7fe364318640 1 --2- 172.21.15.114:0/2614911608 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fe3480a1090 0x7fe3480a3550 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.167+0000 7fe364318640 1 --2- 172.21.15.114:0/2614911608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe35c100810 0x7fe35c19d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.167+0000 7fe364318640 1 --2- 172.21.15.114:0/2614911608 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe35c0ffe50 0x7fe35c196170 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.167+0000 7fe364318640 1 --2- 172.21.15.114:0/2614911608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe35c0fec60 0x7fe35c195c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.167+0000 7fe364318640 1 -- 172.21.15.114:0/2614911608 >> 172.21.15.114:0/2614911608 conn(0x7fe35c0fa3e0 msgr2=0x7fe35c0675a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:21.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.167+0000 7fe364318640 1 -- 172.21.15.114:0/2614911608 shutdown_connections 2024-09-13T18:22:21.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.167+0000 7fe364318640 1 -- 172.21.15.114:0/2614911608 wait complete. 2024-09-13T18:22:21.179 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:21 smithi114 ceph-mon[30897]: pgmap v350: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-13T18:22:21.179 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:21 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/1288329365' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T18:22:21.179 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:21 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2713621921' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T18:22:21.179 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:21 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/1446116470' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T18:22:21.180 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:22:21.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.278+0000 7fc308ff4640 1 -- 172.21.15.114:0/3237720940 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc3040ffe90 msgr2=0x7fc304100310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:21.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.278+0000 7fc308ff4640 1 --2- 172.21.15.114:0/3237720940 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc3040ffe90 0x7fc304100310 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec009900 tx=0x7fc2ec02a410 comp rx=0 tx=0).stop 2024-09-13T18:22:21.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.278+0000 7fc308ff4640 1 -- 172.21.15.114:0/3237720940 shutdown_connections 2024-09-13T18:22:21.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.278+0000 7fc308ff4640 1 --2- 172.21.15.114:0/3237720940 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc304100850 0x7fc3041070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.278+0000 7fc308ff4640 1 --2- 172.21.15.114:0/3237720940 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc3040ffe90 0x7fc304100310 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.278+0000 7fc308ff4640 1 --2- 172.21.15.114:0/3237720940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3040feca0 0x7fc3040ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.278+0000 7fc308ff4640 1 -- 172.21.15.114:0/3237720940 >> 172.21.15.114:0/3237720940 conn(0x7fc3040fa440 msgr2=0x7fc3040fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:21.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.278+0000 7fc308ff4640 1 -- 172.21.15.114:0/3237720940 shutdown_connections 2024-09-13T18:22:21.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.278+0000 7fc308ff4640 1 -- 172.21.15.114:0/3237720940 wait complete. 2024-09-13T18:22:21.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.279+0000 7fc308ff4640 1 Processor -- start 2024-09-13T18:22:21.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.279+0000 7fc308ff4640 1 -- start start 2024-09-13T18:22:21.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.279+0000 7fc308ff4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3040feca0 0x7fc304109510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:21.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.279+0000 7fc308ff4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc3040ffe90 0x7fc304107ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:21.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.279+0000 7fc308ff4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc304100850 0x7fc3041080e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:21.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.279+0000 7fc308ff4640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc304108870 con 0x7fc3040ffe90 2024-09-13T18:22:21.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.279+0000 7fc308ff4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3041089e0 con 0x7fc304100850 2024-09-13T18:22:21.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.279+0000 7fc308ff4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30419fcc0 con 0x7fc3040feca0 2024-09-13T18:22:21.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.280+0000 7fc2f9d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc3040ffe90 0x7fc304107ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:21.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.280+0000 7fc2f9d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc3040ffe90 0x7fc304107ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43372/0 (socket says 172.21.15.114:43372) 2024-09-13T18:22:21.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.280+0000 7fc2f9d9b640 1 -- 172.21.15.114:0/691757847 learned_addr learned my addr 172.21.15.114:0/691757847 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:21.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.280+0000 7fc30259c640 1 --2- 172.21.15.114:0/691757847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3040feca0 0x7fc304109510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:21.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.280+0000 7fc302d9d640 1 --2- 172.21.15.114:0/691757847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc304100850 0x7fc3041080e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:21.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.280+0000 7fc2f9d9b640 1 -- 172.21.15.114:0/691757847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3040feca0 msgr2=0x7fc304109510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:21.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.280+0000 7fc2f9d9b640 1 --2- 172.21.15.114:0/691757847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3040feca0 0x7fc304109510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.280+0000 7fc2f9d9b640 1 -- 172.21.15.114:0/691757847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc304100850 msgr2=0x7fc3041080e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:21.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.280+0000 7fc2f9d9b640 1 --2- 172.21.15.114:0/691757847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc304100850 0x7fc3041080e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.280+0000 7fc2f9d9b640 1 -- 172.21.15.114:0/691757847 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2ec009560 con 0x7fc3040ffe90 2024-09-13T18:22:21.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.280+0000 7fc2f9d9b640 1 --2- 172.21.15.114:0/691757847 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc3040ffe90 0x7fc304107ba0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec005ab0 tx=0x7fc2ec0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:21.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.280+0000 7fc2fbfff640 1 -- 172.21.15.114:0/691757847 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec031d70 con 0x7fc3040ffe90 2024-09-13T18:22:21.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.280+0000 7fc2fbfff640 1 -- 172.21.15.114:0/691757847 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc2ec0044e0 con 0x7fc3040ffe90 2024-09-13T18:22:21.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.280+0000 7fc308ff4640 1 -- 172.21.15.114:0/691757847 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc30419fec0 con 0x7fc3040ffe90 2024-09-13T18:22:21.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.280+0000 7fc2fbfff640 1 -- 172.21.15.114:0/691757847 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec003b30 con 0x7fc3040ffe90 2024-09-13T18:22:21.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.280+0000 7fc308ff4640 1 -- 172.21.15.114:0/691757847 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3041a02e0 con 0x7fc3040ffe90 2024-09-13T18:22:21.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.281+0000 7fc308ff4640 1 -- 172.21.15.114:0/691757847 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3041061a0 con 0x7fc3040ffe90 2024-09-13T18:22:21.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.282+0000 7fc2fbfff640 1 -- 172.21.15.114:0/691757847 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fc2ec003cb0 con 0x7fc3040ffe90 2024-09-13T18:22:21.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.283+0000 7fc2fbfff640 1 --2- 172.21.15.114:0/691757847 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fc2f00a1090 0x7fc2f00a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:21.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.283+0000 7fc30259c640 1 --2- 172.21.15.114:0/691757847 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fc2f00a1090 0x7fc2f00a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:21.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.283+0000 7fc2fbfff640 1 -- 172.21.15.114:0/691757847 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fc2ec0ee050 con 0x7fc3040ffe90 2024-09-13T18:22:21.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.284+0000 7fc30259c640 1 --2- 172.21.15.114:0/691757847 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fc2f00a1090 0x7fc2f00a3550 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc3040ffcf0 tx=0x7fc2f4005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:21.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.287+0000 7fc2fbfff640 1 -- 172.21.15.114:0/691757847 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fc2ec0b7470 con 0x7fc3040ffe90 2024-09-13T18:22:21.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:21 smithi134 ceph-mon[34283]: pgmap v350: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-13T18:22:21.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:21 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/1288329365' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T18:22:21.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:21 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/2713621921' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T18:22:21.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:21 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/1446116470' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T18:22:21.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:21 smithi204 ceph-mon[34777]: pgmap v350: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-13T18:22:21.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:21 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/1288329365' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T18:22:21.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:21 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/2713621921' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T18:22:21.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:21 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/1446116470' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T18:22:21.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.485+0000 7fc308ff4640 1 -- 172.21.15.114:0/691757847 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc30410b590 con 0x7fc3040ffe90 2024-09-13T18:22:21.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.485+0000 7fc2fbfff640 1 -- 172.21.15.114:0/691757847 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10930 (secure 0 0 0) 0x7fc2ec0baa90 con 0x7fc3040ffe90 2024-09-13T18:22:21.488 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:22:21.488 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4162817497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4162817497},{"type":"v1","addr":"172.21.15.114:6835","nonce":4162817497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/2435428478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":2435428478},{"type":"v1","addr":"172.21.15.114:6837","nonce":2435428478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T18:20:01.895117+0000","modified":"2024-09-13T18:20:29.250121+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24482,"mds_2":24475,"mds_3":24458,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/2046516451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":2046516451},{"type":"v1","addr":"172.21.15.114:6839","nonce":2046516451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":10,"addr":"172.21.15.114:6841/1956923446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":1956923446},{"type":"v1","addr":"172.21.15.114:6841","nonce":1956923446}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24391":{"gid":24391,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/507301496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":507301496},{"type":"v1","addr":"172.21.15.204:6833","nonce":507301496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4028657219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4028657219},{"type":"v1","addr":"172.21.15.134:6833","nonce":4028657219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/57029119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":57029119},{"type":"v1","addr":"172.21.15.204:6835","nonce":57029119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3068880939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3068880939},{"type":"v1","addr":"172.21.15.204:6837","nonce":3068880939}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"e","rank":3,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.134:6835/1150003275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1150003275},{"type":"v1","addr":"172.21.15.134:6835","nonce":1150003275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/344883912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":344883912},{"type":"v1","addr":"172.21.15.204:6839","nonce":344883912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6837/891692210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":891692210},{"type":"v1","addr":"172.21.15.134:6837","nonce":891692210}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6839/3466309724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3466309724},{"type":"v1","addr":"172.21.15.134:6839","nonce":3466309724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:22:21.488 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 29 2024-09-13T18:22:21.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.487+0000 7fc308ff4640 1 -- 172.21.15.114:0/691757847 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fc2f00a1090 msgr2=0x7fc2f00a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:21.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.487+0000 7fc308ff4640 1 --2- 172.21.15.114:0/691757847 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fc2f00a1090 0x7fc2f00a3550 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc3040ffcf0 tx=0x7fc2f4005f90 comp rx=0 tx=0).stop 2024-09-13T18:22:21.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.487+0000 7fc308ff4640 1 -- 172.21.15.114:0/691757847 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc3040ffe90 msgr2=0x7fc304107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:21.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.487+0000 7fc308ff4640 1 --2- 172.21.15.114:0/691757847 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc3040ffe90 0x7fc304107ba0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec005ab0 tx=0x7fc2ec0042f0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.487+0000 7fc308ff4640 1 -- 172.21.15.114:0/691757847 shutdown_connections 2024-09-13T18:22:21.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.487+0000 7fc308ff4640 1 --2- 172.21.15.114:0/691757847 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fc2f00a1090 0x7fc2f00a3550 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.487+0000 7fc308ff4640 1 --2- 172.21.15.114:0/691757847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc304100850 0x7fc3041080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.487+0000 7fc308ff4640 1 --2- 172.21.15.114:0/691757847 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc3040ffe90 0x7fc304107ba0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.487+0000 7fc308ff4640 1 --2- 172.21.15.114:0/691757847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3040feca0 0x7fc304109510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.487+0000 7fc308ff4640 1 -- 172.21.15.114:0/691757847 >> 172.21.15.114:0/691757847 conn(0x7fc3040fa440 msgr2=0x7fc304103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:21.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.487+0000 7fc308ff4640 1 -- 172.21.15.114:0/691757847 shutdown_connections 2024-09-13T18:22:21.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.487+0000 7fc308ff4640 1 -- 172.21.15.114:0/691757847 wait complete. 2024-09-13T18:22:21.501 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:22:21.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.597+0000 7fabe4518640 1 -- 172.21.15.114:0/698941092 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fabdc100850 msgr2=0x7fabdc1070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:21.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.598+0000 7fabe4518640 1 --2- 172.21.15.114:0/698941092 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fabdc100850 0x7fabdc1070e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fabd4009ca0 tx=0x7fabd402a8c0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.598+0000 7fabe4518640 1 -- 172.21.15.114:0/698941092 shutdown_connections 2024-09-13T18:22:21.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.598+0000 7fabe4518640 1 --2- 172.21.15.114:0/698941092 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fabdc100850 0x7fabdc1070e0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.598+0000 7fabe4518640 1 --2- 172.21.15.114:0/698941092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabdc0ffe90 0x7fabdc100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.598+0000 7fabe4518640 1 --2- 172.21.15.114:0/698941092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabdc0feca0 0x7fabdc0ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.598+0000 7fabe4518640 1 -- 172.21.15.114:0/698941092 >> 172.21.15.114:0/698941092 conn(0x7fabdc0fa440 msgr2=0x7fabdc0fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:21.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.598+0000 7fabe4518640 1 -- 172.21.15.114:0/698941092 shutdown_connections 2024-09-13T18:22:21.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.598+0000 7fabe4518640 1 -- 172.21.15.114:0/698941092 wait complete. 2024-09-13T18:22:21.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.598+0000 7fabe4518640 1 Processor -- start 2024-09-13T18:22:21.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.598+0000 7fabe4518640 1 -- start start 2024-09-13T18:22:21.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.598+0000 7fabe4518640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabdc0feca0 0x7fabdc195c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:21.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.598+0000 7fabe4518640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fabdc0ffe90 0x7fabdc1961d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:21.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.598+0000 7fabe4518640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabdc100850 0x7fabdc19d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:21.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.598+0000 7fabe4518640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc19d7a0 con 0x7fabdc0ffe90 2024-09-13T18:22:21.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.598+0000 7fabe4518640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc19d910 con 0x7fabdc100850 2024-09-13T18:22:21.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.598+0000 7fabe4518640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc19dad0 con 0x7fabdc0feca0 2024-09-13T18:22:21.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.599+0000 7fabe1ab3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fabdc0ffe90 0x7fabdc1961d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:21.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.599+0000 7fabe1ab3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fabdc0ffe90 0x7fabdc1961d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43390/0 (socket says 172.21.15.114:43390) 2024-09-13T18:22:21.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.599+0000 7fabe1ab3640 1 -- 172.21.15.114:0/1078211570 learned_addr learned my addr 172.21.15.114:0/1078211570 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:21.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.599+0000 7fabe22b4640 1 --2- 172.21.15.114:0/1078211570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabdc0feca0 0x7fabdc195c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:21.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.599+0000 7fabe1ab3640 1 -- 172.21.15.114:0/1078211570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabdc0feca0 msgr2=0x7fabdc195c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:21.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.599+0000 7fabe1ab3640 1 --2- 172.21.15.114:0/1078211570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabdc0feca0 0x7fabdc195c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.599+0000 7fabe1ab3640 1 -- 172.21.15.114:0/1078211570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabdc100850 msgr2=0x7fabdc19d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:21.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.599+0000 7fabe1ab3640 1 --2- 172.21.15.114:0/1078211570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabdc100850 0x7fabdc19d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.599+0000 7fabe1ab3640 1 -- 172.21.15.114:0/1078211570 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd4009900 con 0x7fabdc0ffe90 2024-09-13T18:22:21.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.599+0000 7fabe22b4640 1 --2- 172.21.15.114:0/1078211570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabdc0feca0 0x7fabdc195c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:22:21.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.599+0000 7fabe1ab3640 1 --2- 172.21.15.114:0/1078211570 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fabdc0ffe90 0x7fabdc1961d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fabd800b4a0 tx=0x7fabd800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:21.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.599+0000 7fabcf7fe640 1 -- 172.21.15.114:0/1078211570 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd8010610 con 0x7fabdc0ffe90 2024-09-13T18:22:21.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.599+0000 7fabcf7fe640 1 -- 172.21.15.114:0/1078211570 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fabd8010c00 con 0x7fabdc0ffe90 2024-09-13T18:22:21.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.599+0000 7fabe4518640 1 -- 172.21.15.114:0/1078211570 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabdc19ddb0 con 0x7fabdc0ffe90 2024-09-13T18:22:21.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.599+0000 7fabcf7fe640 1 -- 172.21.15.114:0/1078211570 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd8017ca0 con 0x7fabdc0ffe90 2024-09-13T18:22:21.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.599+0000 7fabe4518640 1 -- 172.21.15.114:0/1078211570 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabdc19e260 con 0x7fabdc0ffe90 2024-09-13T18:22:21.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.600+0000 7fabe4518640 1 -- 172.21.15.114:0/1078211570 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabdc1060e0 con 0x7fabdc0ffe90 2024-09-13T18:22:21.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.601+0000 7fabcf7fe640 1 -- 172.21.15.114:0/1078211570 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fabd8010770 con 0x7fabdc0ffe90 2024-09-13T18:22:21.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.602+0000 7fabcf7fe640 1 --2- 172.21.15.114:0/1078211570 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fabc80a1110 0x7fabc80a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:21.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.602+0000 7fabe22b4640 1 --2- 172.21.15.114:0/1078211570 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fabc80a1110 0x7fabc80a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:21.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.602+0000 7fabcf7fe640 1 -- 172.21.15.114:0/1078211570 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fabd80cc320 con 0x7fabdc0ffe90 2024-09-13T18:22:21.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.603+0000 7fabe22b4640 1 --2- 172.21.15.114:0/1078211570 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fabc80a1110 0x7fabc80a35d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fabd00058c0 tx=0x7fabd0005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:21.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.606+0000 7fabcf7fe640 1 -- 172.21.15.114:0/1078211570 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fabd8095740 con 0x7fabdc0ffe90 2024-09-13T18:22:21.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.768+0000 7fabe4518640 1 -- 172.21.15.114:0/1078211570 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fabdc02c480 con 0x7fabdc0ffe90 2024-09-13T18:22:21.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.769+0000 7fabcf7fe640 1 -- 172.21.15.114:0/1078211570 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24696 (secure 0 0 0) 0x7fabd8098d60 con 0x7fabdc0ffe90 2024-09-13T18:22:21.772 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:22:21.772 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":93,"fsid":"97e7edf2-71fb-11ef-bceb-c7b262605968","created":"2024-09-13T18:12:36.360625+0000","modified":"2024-09-13T18:21:28.143797+0000","last_up_change":"2024-09-13T18:19:12.350370+0000","last_in_change":"2024-09-13T18:18:52.321596+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:15:40.129659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:20:00.024407+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:20:01.085549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T18:20:04.334216+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9914f01a-9f6f-4878-b955-83570199c354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6803","nonce":3499525826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6805","nonce":3499525826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6809","nonce":3499525826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6807","nonce":3499525826}]},"public_addr":"172.21.15.114:6803/3499525826","cluster_addr":"172.21.15.114:6805/3499525826","heartbeat_back_addr":"172.21.15.114:6809/3499525826","heartbeat_front_addr":"172.21.15.114:6807/3499525826","state":["exists","up"]},{"osd":1,"uuid":"ef65e012-7469-4c20-b354-17022bca63c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6801","nonce":4264264866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6803","nonce":4264264866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6807","nonce":4264264866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6805","nonce":4264264866}]},"public_addr":"172.21.15.134:6801/4264264866","cluster_addr":"172.21.15.134:6803/4264264866","heartbeat_back_addr":"172.21.15.134:6807/4264264866","heartbeat_front_addr":"172.21.15.134:6805/4264264866","state":["exists","up"]},{"osd":2,"uuid":"578213fd-715f-4be7-91fa-254f3d0e3022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6801","nonce":2288807707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6803","nonce":2288807707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6807","nonce":2288807707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6805","nonce":2288807707}]},"public_addr":"172.21.15.204:6801/2288807707","cluster_addr":"172.21.15.204:6803/2288807707","heartbeat_back_addr":"172.21.15.204:6807/2288807707","heartbeat_front_addr":"172.21.15.204:6805/2288807707","state":["exists","up"]},{"osd":3,"uuid":"b64aab3f-1e52-4828-8397-74af5e05f2cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6811","nonce":2904204254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6813","nonce":2904204254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6817","nonce":2904204254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6815","nonce":2904204254}]},"public_addr":"172.21.15.114:6811/2904204254","cluster_addr":"172.21.15.114:6813/2904204254","heartbeat_back_addr":"172.21.15.114:6817/2904204254","heartbeat_front_addr":"172.21.15.114:6815/2904204254","state":["exists","up"]},{"osd":4,"uuid":"f38943d6-39c0-42e1-b427-cb6bb90ac38a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6809","nonce":1773945333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6811","nonce":1773945333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6815","nonce":1773945333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6813","nonce":1773945333}]},"public_addr":"172.21.15.134:6809/1773945333","cluster_addr":"172.21.15.134:6811/1773945333","heartbeat_back_addr":"172.21.15.134:6815/1773945333","heartbeat_front_addr":"172.21.15.134:6813/1773945333","state":["exists","up"]},{"osd":5,"uuid":"bb38f361-8e00-4178-9752-5ddbf3af20e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6809","nonce":2371778055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6811","nonce":2371778055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6815","nonce":2371778055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6813","nonce":2371778055}]},"public_addr":"172.21.15.204:6809/2371778055","cluster_addr":"172.21.15.204:6811/2371778055","heartbeat_back_addr":"172.21.15.204:6815/2371778055","heartbeat_front_addr":"172.21.15.204:6813/2371778055","state":["exists","up"]},{"osd":6,"uuid":"93d40a03-e7ff-4a3f-9efa-5c5403c9983d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6819","nonce":1484325044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6821","nonce":1484325044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6825","nonce":1484325044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6823","nonce":1484325044}]},"public_addr":"172.21.15.114:6819/1484325044","cluster_addr":"172.21.15.114:6821/1484325044","heartbeat_back_addr":"172.21.15.114:6825/1484325044","heartbeat_front_addr":"172.21.15.114:6823/1484325044","state":["exists","up"]},{"osd":7,"uuid":"982f629a-1b95-4be1-8e75-5ddce1ad77b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6817","nonce":799246614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6819","nonce":799246614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6823","nonce":799246614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6821","nonce":799246614}]},"public_addr":"172.21.15.134:6817/799246614","cluster_addr":"172.21.15.134:6819/799246614","heartbeat_back_addr":"172.21.15.134:6823/799246614","heartbeat_front_addr":"172.21.15.134:6821/799246614","state":["exists","up"]},{"osd":8,"uuid":"409beb9b-8227-4dad-a4a0-f0e5ce70dc5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6817","nonce":858331805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6819","nonce":858331805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6823","nonce":858331805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6821","nonce":858331805}]},"public_addr":"172.21.15.204:6817/858331805","cluster_addr":"172.21.15.204:6819/858331805","heartbeat_back_addr":"172.21.15.204:6823/858331805","heartbeat_front_addr":"172.21.15.204:6821/858331805","state":["exists","up"]},{"osd":9,"uuid":"a345a15c-b1fc-4877-b445-418eebf0d0ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6827","nonce":2172392854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6829","nonce":2172392854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6833","nonce":2172392854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6831","nonce":2172392854}]},"public_addr":"172.21.15.114:6827/2172392854","cluster_addr":"172.21.15.114:6829/2172392854","heartbeat_back_addr":"172.21.15.114:6833/2172392854","heartbeat_front_addr":"172.21.15.114:6831/2172392854","state":["exists","up"]},{"osd":10,"uuid":"464d995f-d90b-4caa-addb-1f7562bd74b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6825","nonce":3073513468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6827","nonce":3073513468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6831","nonce":3073513468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6829","nonce":3073513468}]},"public_addr":"172.21.15.134:6825/3073513468","cluster_addr":"172.21.15.134:6827/3073513468","heartbeat_back_addr":"172.21.15.134:6831/3073513468","heartbeat_front_addr":"172.21.15.134:6829/3073513468","state":["exists","up"]},{"osd":11,"uuid":"71dd794d-326c-477f-8d02-ee32bff1350d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6825","nonce":3239521765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6827","nonce":3239521765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6831","nonce":3239521765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6829","nonce":3239521765}]},"public_addr":"172.21.15.204:6825/3239521765","cluster_addr":"172.21.15.204:6827/3239521765","heartbeat_back_addr":"172.21.15.204:6831/3239521765","heartbeat_front_addr":"172.21.15.204:6829/3239521765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:14:54.542687+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:15:15.746055+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:15:38.371131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:01.383932+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:24.796237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:48.422214+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:11.284323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:34.570365+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:58.232855+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:18:21.609596+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:18:45.565859+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:19:10.017969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:6801/4281866858":"2024-09-14T18:13:25.823027+0000","172.21.15.114:6800/4281866858":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2949422849":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2034742251":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/722571999":"2024-09-14T18:13:03.521116+0000","172.21.15.114:6801/830738794":"2024-09-14T18:13:03.521116+0000","172.21.15.114:6800/830738794":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/2828364892":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/4257828906":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/3544951322":"2024-09-14T18:13:03.521116+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:22:21.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.771+0000 7fabe4518640 1 -- 172.21.15.114:0/1078211570 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fabc80a1110 msgr2=0x7fabc80a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:21.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.771+0000 7fabe4518640 1 --2- 172.21.15.114:0/1078211570 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fabc80a1110 0x7fabc80a35d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fabd00058c0 tx=0x7fabd0005850 comp rx=0 tx=0).stop 2024-09-13T18:22:21.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.771+0000 7fabe4518640 1 -- 172.21.15.114:0/1078211570 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fabdc0ffe90 msgr2=0x7fabdc1961d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:21.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.771+0000 7fabe4518640 1 --2- 172.21.15.114:0/1078211570 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fabdc0ffe90 0x7fabdc1961d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fabd800b4a0 tx=0x7fabd800b960 comp rx=0 tx=0).stop 2024-09-13T18:22:21.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.771+0000 7fabe4518640 1 -- 172.21.15.114:0/1078211570 shutdown_connections 2024-09-13T18:22:21.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.771+0000 7fabe4518640 1 --2- 172.21.15.114:0/1078211570 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fabc80a1110 0x7fabc80a35d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.771+0000 7fabe4518640 1 --2- 172.21.15.114:0/1078211570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabdc100850 0x7fabdc19d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.771+0000 7fabe4518640 1 --2- 172.21.15.114:0/1078211570 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fabdc0ffe90 0x7fabdc1961d0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.771+0000 7fabe4518640 1 --2- 172.21.15.114:0/1078211570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabdc0feca0 0x7fabdc195c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.771+0000 7fabe4518640 1 -- 172.21.15.114:0/1078211570 >> 172.21.15.114:0/1078211570 conn(0x7fabdc0fa440 msgr2=0x7fabdc1032b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:21.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.771+0000 7fabe4518640 1 -- 172.21.15.114:0/1078211570 shutdown_connections 2024-09-13T18:22:21.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.772+0000 7fabe4518640 1 -- 172.21.15.114:0/1078211570 wait complete. 2024-09-13T18:22:21.785 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-13T18:20:01.895117+0000', 'modified': '2024-09-13T18:20:29.250121+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24521, 'mds_1': 24482, 'mds_2': 24475, 'mds_3': 24458, 'mds_4': 14628}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14616': {'gid': 14616, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.114:6839/2046516451', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6838', 'nonce': 2046516451}, {'type': 'v1', 'addr': '172.21.15.114:6839', 'nonce': 2046516451}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_14628': {'gid': 14628, 'name': 'a', 'rank': 4, 'incarnation': 27, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.114:6841/1956923446', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6840', 'nonce': 1956923446}, {'type': 'v1', 'addr': '172.21.15.114:6841', 'nonce': 1956923446}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24391': {'gid': 24391, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6833/507301496', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6832', 'nonce': 507301496}, {'type': 'v1', 'addr': '172.21.15.204:6833', 'nonce': 507301496}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24422': {'gid': 24422, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6833/4028657219', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6832', 'nonce': 4028657219}, {'type': 'v1', 'addr': '172.21.15.134:6833', 'nonce': 4028657219}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24445': {'gid': 24445, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6835/57029119', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6834', 'nonce': 57029119}, {'type': 'v1', 'addr': '172.21.15.204:6835', 'nonce': 57029119}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24457': {'gid': 24457, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6837/3068880939', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6836', 'nonce': 3068880939}, {'type': 'v1', 'addr': '172.21.15.204:6837', 'nonce': 3068880939}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24458': {'gid': 24458, 'name': 'e', 'rank': 3, 'incarnation': 25, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.134:6835/1150003275', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6834', 'nonce': 1150003275}, {'type': 'v1', 'addr': '172.21.15.134:6835', 'nonce': 1150003275}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24475': {'gid': 24475, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.204:6839/344883912', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6838', 'nonce': 344883912}, {'type': 'v1', 'addr': '172.21.15.204:6839', 'nonce': 344883912}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24482': {'gid': 24482, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.134:6837/891692210', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6836', 'nonce': 891692210}, {'type': 'v1', 'addr': '172.21.15.134:6837', 'nonce': 891692210}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24521': {'gid': 24521, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.134:6839/3466309724', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6838', 'nonce': 3466309724}, {'type': 'v1', 'addr': '172.21.15.134:6839', 'nonce': 3466309724}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-09-13T18:22:21.785 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-13T18:22:21.786 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-13T18:22:21.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.883+0000 7fe9706d7640 1 -- 172.21.15.114:0/322462120 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe968101360 msgr2=0x7fe968101760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:21.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.883+0000 7fe9706d7640 1 --2- 172.21.15.114:0/322462120 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe968101360 0x7fe968101760 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe960009900 tx=0x7fe96002a3f0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.883+0000 7fe9706d7640 1 -- 172.21.15.114:0/322462120 shutdown_connections 2024-09-13T18:22:21.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.883+0000 7fe9706d7640 1 --2- 172.21.15.114:0/322462120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9680fe790 0x7fe968108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.883+0000 7fe9706d7640 1 --2- 172.21.15.114:0/322462120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9680fddd0 0x7fe9680fe250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.883+0000 7fe9706d7640 1 --2- 172.21.15.114:0/322462120 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe968101360 0x7fe968101760 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.883+0000 7fe9706d7640 1 -- 172.21.15.114:0/322462120 >> 172.21.15.114:0/322462120 conn(0x7fe9680f9b60 msgr2=0x7fe9680fbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:21.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.883+0000 7fe9706d7640 1 -- 172.21.15.114:0/322462120 shutdown_connections 2024-09-13T18:22:21.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.883+0000 7fe9706d7640 1 -- 172.21.15.114:0/322462120 wait complete. 2024-09-13T18:22:21.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.883+0000 7fe9706d7640 1 Processor -- start 2024-09-13T18:22:21.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.883+0000 7fe9706d7640 1 -- start start 2024-09-13T18:22:21.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.884+0000 7fe9706d7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9680fddd0 0x7fe968195bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:21.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.884+0000 7fe9706d7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9680fe790 0x7fe968196110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:21.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.884+0000 7fe9706d7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe968101360 0x7fe96819d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:21.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.884+0000 7fe9706d7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96819d6c0 con 0x7fe968101360 2024-09-13T18:22:21.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.884+0000 7fe9706d7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96819d830 con 0x7fe9680fe790 2024-09-13T18:22:21.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.884+0000 7fe9706d7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96819d9f0 con 0x7fe9680fddd0 2024-09-13T18:22:21.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.884+0000 7fe96ec74640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe968101360 0x7fe96819d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:21.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.884+0000 7fe96ec74640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe968101360 0x7fe96819d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43410/0 (socket says 172.21.15.114:43410) 2024-09-13T18:22:21.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.884+0000 7fe96ec74640 1 -- 172.21.15.114:0/596668817 learned_addr learned my addr 172.21.15.114:0/596668817 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:21.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.884+0000 7fe96e473640 1 --2- 172.21.15.114:0/596668817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9680fddd0 0x7fe968195bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:21.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.884+0000 7fe96dc72640 1 --2- 172.21.15.114:0/596668817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9680fe790 0x7fe968196110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:21.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.884+0000 7fe96ec74640 1 -- 172.21.15.114:0/596668817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9680fddd0 msgr2=0x7fe968195bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:21.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.884+0000 7fe96ec74640 1 --2- 172.21.15.114:0/596668817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9680fddd0 0x7fe968195bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.884+0000 7fe96ec74640 1 -- 172.21.15.114:0/596668817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9680fe790 msgr2=0x7fe968196110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:21.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.884+0000 7fe96ec74640 1 --2- 172.21.15.114:0/596668817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9680fe790 0x7fe968196110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.884+0000 7fe96ec74640 1 -- 172.21.15.114:0/596668817 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe960009560 con 0x7fe968101360 2024-09-13T18:22:21.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.884+0000 7fe96dc72640 1 --2- 172.21.15.114:0/596668817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9680fe790 0x7fe968196110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:22:21.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.884+0000 7fe96ec74640 1 --2- 172.21.15.114:0/596668817 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe968101360 0x7fe96819d180 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe95800c970 tx=0x7fe95800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:21.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.884+0000 7fe9677fe640 1 -- 172.21.15.114:0/596668817 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe958007bb0 con 0x7fe968101360 2024-09-13T18:22:21.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.884+0000 7fe9677fe640 1 -- 172.21.15.114:0/596668817 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe958007d10 con 0x7fe968101360 2024-09-13T18:22:21.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.884+0000 7fe9706d7640 1 -- 172.21.15.114:0/596668817 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe96819dcd0 con 0x7fe968101360 2024-09-13T18:22:21.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.884+0000 7fe9677fe640 1 -- 172.21.15.114:0/596668817 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe958018660 con 0x7fe968101360 2024-09-13T18:22:21.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.884+0000 7fe9706d7640 1 -- 172.21.15.114:0/596668817 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe96819e190 con 0x7fe968101360 2024-09-13T18:22:21.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.887+0000 7fe9677fe640 1 -- 172.21.15.114:0/596668817 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fe95800f450 con 0x7fe968101360 2024-09-13T18:22:21.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.887+0000 7fe9733f8740 1 Processor -- start 2024-09-13T18:22:21.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.887+0000 7fe9733f8740 1 -- start start 2024-09-13T18:22:21.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.887+0000 7fe9733f8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571bf1ff500 0x5571bf1ff900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:21.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.887+0000 7fe9733f8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571bf1ffe40 0x5571bf204900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:21.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.887+0000 7fe9733f8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5571bf204ed0 0x5571bf207390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:21.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.887+0000 7fe9733f8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571bf15a7c0 con 0x5571bf204ed0 2024-09-13T18:22:21.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.887+0000 7fe9733f8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571bf169b70 con 0x5571bf1ff500 2024-09-13T18:22:21.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.887+0000 7fe9733f8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571bf167300 con 0x5571bf1ffe40 2024-09-13T18:22:21.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.887+0000 7fe96dc72640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5571bf204ed0 0x5571bf207390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:21.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.887+0000 7fe96dc72640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5571bf204ed0 0x5571bf207390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43424/0 (socket says 172.21.15.114:43424) 2024-09-13T18:22:21.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.887+0000 7fe96dc72640 1 -- 172.21.15.114:0/1984325760 learned_addr learned my addr 172.21.15.114:0/1984325760 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:21.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.888+0000 7fe96e473640 1 --2- 172.21.15.114:0/1984325760 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571bf1ffe40 0x5571bf204900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:21.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.888+0000 7fe9677fe640 1 --2- 172.21.15.114:0/596668817 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fe9500a1110 0x7fe9500a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:21.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.888+0000 7fe96dc72640 1 --2- 172.21.15.114:0/1984325760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571bf1ff500 0x5571bf1ff900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:21.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.888+0000 7fe96dc72640 1 -- 172.21.15.114:0/1984325760 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571bf1ffe40 msgr2=0x5571bf204900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:21.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.888+0000 7fe96dc72640 1 --2- 172.21.15.114:0/1984325760 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571bf1ffe40 0x5571bf204900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.888+0000 7fe96dc72640 1 -- 172.21.15.114:0/1984325760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571bf1ff500 msgr2=0x5571bf1ff900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:21.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.888+0000 7fe96dc72640 1 --2- 172.21.15.114:0/1984325760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571bf1ff500 0x5571bf1ff900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.888+0000 7fe96ec74640 1 --2- 172.21.15.114:0/596668817 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fe9500a1110 0x7fe9500a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:21.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.888+0000 7fe96dc72640 1 -- 172.21.15.114:0/1984325760 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571bf0ef4b0 con 0x5571bf204ed0 2024-09-13T18:22:21.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.888+0000 7fe9677fe640 1 -- 172.21.15.114:0/596668817 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fe9580ccc20 con 0x7fe968101360 2024-09-13T18:22:21.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.888+0000 7fe96dc72640 1 --2- 172.21.15.114:0/1984325760 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5571bf204ed0 0x5571bf207390 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe95c007350 tx=0x7fe95c007940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae039820a3e55a94 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:21.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.888+0000 7fe964ff9640 1 -- 172.21.15.114:0/1984325760 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95c012e60 con 0x5571bf204ed0 2024-09-13T18:22:21.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.888+0000 7fe964ff9640 1 -- 172.21.15.114:0/1984325760 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe95c010a80 con 0x5571bf204ed0 2024-09-13T18:22:21.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.888+0000 7fe964ff9640 1 -- 172.21.15.114:0/1984325760 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95c005580 con 0x5571bf204ed0 2024-09-13T18:22:21.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.889+0000 7fe9733f8740 1 -- 172.21.15.114:0/1984325760 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5571bf204ed0 msgr2=0x5571bf207390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:21.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.889+0000 7fe96ec74640 1 --2- 172.21.15.114:0/596668817 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fe9500a1110 0x7fe9500a35d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe95800afd0 tx=0x7fe958065680 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:21.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.889+0000 7fe9733f8740 1 --2- 172.21.15.114:0/1984325760 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5571bf204ed0 0x5571bf207390 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe95c007350 tx=0x7fe95c007940 comp rx=0 tx=0).stop 2024-09-13T18:22:21.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.889+0000 7fe9733f8740 1 -- 172.21.15.114:0/1984325760 shutdown_connections 2024-09-13T18:22:21.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.889+0000 7fe9733f8740 1 --2- 172.21.15.114:0/1984325760 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5571bf204ed0 0x5571bf207390 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.889+0000 7fe9733f8740 1 --2- 172.21.15.114:0/1984325760 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571bf1ffe40 0x5571bf204900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.889+0000 7fe9733f8740 1 --2- 172.21.15.114:0/1984325760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571bf1ff500 0x5571bf1ff900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.889+0000 7fe9733f8740 1 -- 172.21.15.114:0/1984325760 >> 172.21.15.114:0/1984325760 conn(0x5571bf1fab70 msgr2=0x5571bf1faf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:21.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.889+0000 7fe9733f8740 1 -- 172.21.15.114:0/1984325760 shutdown_connections 2024-09-13T18:22:21.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.889+0000 7fe9733f8740 1 -- 172.21.15.114:0/1984325760 wait complete. 2024-09-13T18:22:21.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.889+0000 7fe9733f8740 1 -- start start 2024-09-13T18:22:21.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.889+0000 7fe9733f8740 1 Processor -- start 2024-09-13T18:22:21.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.890+0000 7fe9733f8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571bf1ff500 0x5571bf209160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:21.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.890+0000 7fe9733f8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5571bf1ffe40 0x5571bf2096a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:21.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.890+0000 7fe9733f8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571bf204ed0 0x5571bf2106e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:21.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.890+0000 7fe9733f8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571bf162bf0 con 0x5571bf1ffe40 2024-09-13T18:22:21.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.890+0000 7fe9733f8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571bf121250 con 0x5571bf204ed0 2024-09-13T18:22:21.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.890+0000 7fe9733f8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571bf120ef0 con 0x5571bf1ff500 2024-09-13T18:22:21.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.890+0000 7fe9733f8740 10 client.0 _ll_register_callbacks cb 0x5571bf153740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:22:21.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.890+0000 7fe96e473640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5571bf1ffe40 0x5571bf2096a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:21.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.890+0000 7fe96e473640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5571bf1ffe40 0x5571bf2096a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43440/0 (socket says 172.21.15.114:43440) 2024-09-13T18:22:21.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.890+0000 7fe96e473640 1 -- 172.21.15.114:0/1532215703 learned_addr learned my addr 172.21.15.114:0/1532215703 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:21.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.890+0000 7fe96dc72640 1 --2- 172.21.15.114:0/1532215703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571bf1ff500 0x5571bf209160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:21.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.890+0000 7fe96dc72640 1 --2- 172.21.15.114:0/1532215703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571bf204ed0 0x5571bf2106e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:21.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.890+0000 7fe96e473640 1 -- 172.21.15.114:0/1532215703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571bf1ff500 msgr2=0x5571bf209160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:21.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.890+0000 7fe96e473640 1 --2- 172.21.15.114:0/1532215703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571bf1ff500 0x5571bf209160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.890+0000 7fe96e473640 1 -- 172.21.15.114:0/1532215703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571bf204ed0 msgr2=0x5571bf2106e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:21.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.890+0000 7fe96e473640 1 --2- 172.21.15.114:0/1532215703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571bf204ed0 0x5571bf2106e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.890+0000 7fe96e473640 1 -- 172.21.15.114:0/1532215703 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe95c012490 con 0x5571bf1ffe40 2024-09-13T18:22:21.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.890+0000 7fe96dc72640 1 --2- 172.21.15.114:0/1532215703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571bf204ed0 0x5571bf2106e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:22:21.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.890+0000 7fe96e473640 1 --2- 172.21.15.114:0/1532215703 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5571bf1ffe40 0x5571bf2096a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe968068390 tx=0x7fe960004580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0e28b718edb2c88 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:21.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.890+0000 7fe94dffb640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:22:21.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.890+0000 7fe94dffb640 1 -- 172.21.15.114:0/1532215703 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe960004b50 con 0x5571bf1ffe40 2024-09-13T18:22:21.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.890+0000 7fe94dffb640 1 -- 172.21.15.114:0/1532215703 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fe960004cb0 con 0x5571bf1ffe40 2024-09-13T18:22:21.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.890+0000 7fe9733f8740 1 -- 172.21.15.114:0/1532215703 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5571bee563c0 con 0x5571bf1ffe40 2024-09-13T18:22:21.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.891+0000 7fe94dffb640 1 -- 172.21.15.114:0/1532215703 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe960004e70 con 0x5571bf1ffe40 2024-09-13T18:22:21.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.891+0000 7fe9733f8740 10 client.15453 fetch_fsmap learned FSMap version 29 2024-09-13T18:22:21.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.891+0000 7fe9733f8740 1 -- 172.21.15.114:0/1532215703 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5571bf161270 con 0x5571bf1ffe40 2024-09-13T18:22:21.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.891+0000 7fe94dffb640 1 -- 172.21.15.114:0/1532215703 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5571bee563c0 con 0x5571bf1ffe40 2024-09-13T18:22:21.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.891+0000 7fe94dffb640 1 -- 172.21.15.114:0/1532215703 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe960031760 con 0x5571bf1ffe40 2024-09-13T18:22:21.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.891+0000 7fe9733f8740 10 client.15453 fetch_fsmap finished waiting for FSMap version 29 2024-09-13T18:22:21.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.891+0000 7fe9733f8740 10 client.15453 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-13T18:22:21.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.891+0000 7fe9733f8740 20 client.15453 populate_metadata read hostname 'smithi114' 2024-09-13T18:22:21.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.891+0000 7fe9733f8740 1 --2- 172.21.15.114:0/1532215703 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x5571bf2156e0 0x5571bf219840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:21.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.891+0000 7fe9733f8740 4 client.15453 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:22:21.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.891+0000 7fe9733f8740 1 -- 172.21.15.114:0/1532215703 --> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5571bee676c0 con 0x5571bf2156e0 2024-09-13T18:22:21.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.891+0000 7fe96dc72640 1 --2- 172.21.15.114:0/1532215703 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x5571bf2156e0 0x5571bf219840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:21.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.891+0000 7fe96dc72640 1 --2- 172.21.15.114:0/1532215703 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x5571bf2156e0 0x5571bf219840 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b7530c5ffd597cfc server_cookie=791cab00adccb5c4 in_seq=0 out_seq=0 2024-09-13T18:22:21.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.891+0000 7fe94dffb640 10 client.15453 ms_handle_connect on v2:172.21.15.114:6840/1956923446 2024-09-13T18:22:21.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.892+0000 7fe94dffb640 1 -- 172.21.15.114:0/1532215703 <== mds.4 v2:172.21.15.114:6840/1956923446 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5571bee676c0 con 0x5571bf2156e0 2024-09-13T18:22:21.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.892+0000 7fe94dffb640 10 client.15453 handle_command_reply: tid=0 2024-09-13T18:22:21.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.892+0000 7fe9733f8740 1 client.15453 shutdown 2024-09-13T18:22:21.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.892+0000 7fe9733f8740 1 -- 172.21.15.114:0/1532215703 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5571bf1ffe40 msgr2=0x5571bf2096a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:21.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.892+0000 7fe9733f8740 1 --2- 172.21.15.114:0/1532215703 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5571bf1ffe40 0x5571bf2096a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe968068390 tx=0x7fe960004580 comp rx=0 tx=0).stop 2024-09-13T18:22:21.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.892+0000 7fe9733f8740 1 -- 172.21.15.114:0/1532215703 shutdown_connections 2024-09-13T18:22:21.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.892+0000 7fe9733f8740 1 --2- 172.21.15.114:0/1532215703 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x5571bf2156e0 0x5571bf219840 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.892+0000 7fe9733f8740 1 --2- 172.21.15.114:0/1532215703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571bf204ed0 0x5571bf2106e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.892+0000 7fe9733f8740 1 --2- 172.21.15.114:0/1532215703 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5571bf1ffe40 0x5571bf2096a0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.892+0000 7fe94dffb640 0 client.15453 ms_handle_reset on v2:172.21.15.114:6840/1956923446 2024-09-13T18:22:21.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.892+0000 7fe9733f8740 1 --2- 172.21.15.114:0/1532215703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571bf1ff500 0x5571bf209160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.892+0000 7fe9733f8740 1 -- 172.21.15.114:0/1532215703 >> 172.21.15.114:0/1532215703 conn(0x5571bf1fab70 msgr2=0x5571bf1fc380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:21.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.892+0000 7fe9733f8740 1 -- 172.21.15.114:0/1532215703 shutdown_connections 2024-09-13T18:22:21.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.892+0000 7fe9733f8740 1 -- 172.21.15.114:0/1532215703 wait complete. 2024-09-13T18:22:21.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.892+0000 7fe9733f8740 20 client.15453 trim_cache size 0 max 16384 2024-09-13T18:22:21.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.907+0000 7fe9733f8740 1 Processor -- start 2024-09-13T18:22:21.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.907+0000 7fe9733f8740 1 -- start start 2024-09-13T18:22:21.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.907+0000 7fe9733f8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5571bf1ff500 0x5571bf21d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:21.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.907+0000 7fe9733f8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571bf1ffe40 0x5571bf2106a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:21.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.907+0000 7fe9733f8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571bf204ed0 0x5571bf21dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:21.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.907+0000 7fe9733f8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571bf208d30 con 0x5571bf1ff500 2024-09-13T18:22:21.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.907+0000 7fe9733f8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571bf0e6460 con 0x5571bf1ffe40 2024-09-13T18:22:21.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.907+0000 7fe9733f8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571bf1f6600 con 0x5571bf204ed0 2024-09-13T18:22:21.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.907+0000 7fe96dc72640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5571bf1ff500 0x5571bf21d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:21.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.907+0000 7fe96dc72640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5571bf1ff500 0x5571bf21d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43444/0 (socket says 172.21.15.114:43444) 2024-09-13T18:22:21.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.907+0000 7fe96dc72640 1 -- 172.21.15.114:0/1888029809 learned_addr learned my addr 172.21.15.114:0/1888029809 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:21.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.907+0000 7fe96e473640 1 --2- 172.21.15.114:0/1888029809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571bf1ffe40 0x5571bf2106a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:21.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.907+0000 7fe96dc72640 1 -- 172.21.15.114:0/1888029809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571bf204ed0 msgr2=0x5571bf21dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:21.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.907+0000 7fe96dc72640 1 --2- 172.21.15.114:0/1888029809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571bf204ed0 0x5571bf21dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.907+0000 7fe96dc72640 1 -- 172.21.15.114:0/1888029809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571bf1ffe40 msgr2=0x5571bf2106a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:21.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.907+0000 7fe96dc72640 1 --2- 172.21.15.114:0/1888029809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571bf1ffe40 0x5571bf2106a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.907+0000 7fe96dc72640 1 -- 172.21.15.114:0/1888029809 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571bf1035f0 con 0x5571bf1ff500 2024-09-13T18:22:21.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.907+0000 7fe96e473640 1 --2- 172.21.15.114:0/1888029809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571bf1ffe40 0x5571bf2106a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:22:21.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.907+0000 7fe96dc72640 1 --2- 172.21.15.114:0/1888029809 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5571bf1ff500 0x5571bf21d860 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe95c010740 tx=0x7fe95c007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13ead211eb604549 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:21.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.907+0000 7fe9657fa640 1 -- 172.21.15.114:0/1888029809 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95c02e030 con 0x5571bf1ff500 2024-09-13T18:22:21.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.907+0000 7fe9657fa640 1 -- 172.21.15.114:0/1888029809 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe95c012e60 con 0x5571bf1ff500 2024-09-13T18:22:21.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.907+0000 7fe9657fa640 1 -- 172.21.15.114:0/1888029809 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95c02a720 con 0x5571bf1ff500 2024-09-13T18:22:21.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.908+0000 7fe9733f8740 1 -- 172.21.15.114:0/1888029809 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5571bf1ff500 msgr2=0x5571bf21d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:21.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.908+0000 7fe9733f8740 1 --2- 172.21.15.114:0/1888029809 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5571bf1ff500 0x5571bf21d860 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe95c010740 tx=0x7fe95c007f30 comp rx=0 tx=0).stop 2024-09-13T18:22:21.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.908+0000 7fe9733f8740 1 -- 172.21.15.114:0/1888029809 shutdown_connections 2024-09-13T18:22:21.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.908+0000 7fe9733f8740 1 --2- 172.21.15.114:0/1888029809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571bf204ed0 0x5571bf21dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.908+0000 7fe9733f8740 1 --2- 172.21.15.114:0/1888029809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571bf1ffe40 0x5571bf2106a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.908+0000 7fe9733f8740 1 --2- 172.21.15.114:0/1888029809 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5571bf1ff500 0x5571bf21d860 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.908+0000 7fe9733f8740 1 -- 172.21.15.114:0/1888029809 >> 172.21.15.114:0/1888029809 conn(0x5571bf1fab70 msgr2=0x5571bf209160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:21.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.908+0000 7fe9733f8740 1 -- 172.21.15.114:0/1888029809 shutdown_connections 2024-09-13T18:22:21.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.908+0000 7fe9733f8740 1 -- 172.21.15.114:0/1888029809 wait complete. 2024-09-13T18:22:21.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.908+0000 7fe9733f8740 1 -- start start 2024-09-13T18:22:21.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.908+0000 7fe9733f8740 1 Processor -- start 2024-09-13T18:22:21.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.908+0000 7fe9733f8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5571bf1ff500 0x5571bf20af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:21.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.908+0000 7fe9733f8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571bf1ffe40 0x5571bf2104a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:21.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.908+0000 7fe9733f8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571bf204ed0 0x5571bf20b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:21.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.908+0000 7fe9733f8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571bf20bab0 con 0x5571bf1ff500 2024-09-13T18:22:21.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.908+0000 7fe9733f8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571bf26b500 con 0x5571bf1ffe40 2024-09-13T18:22:21.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.908+0000 7fe9733f8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571bf26b640 con 0x5571bf204ed0 2024-09-13T18:22:21.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.909+0000 7fe9733f8740 10 client.0 _ll_register_callbacks cb 0x7fe95c0077a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:22:21.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.909+0000 7fe96dc72640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5571bf1ff500 0x5571bf20af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:21.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.909+0000 7fe96dc72640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5571bf1ff500 0x5571bf20af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43456/0 (socket says 172.21.15.114:43456) 2024-09-13T18:22:21.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.909+0000 7fe96dc72640 1 -- 172.21.15.114:0/635981761 learned_addr learned my addr 172.21.15.114:0/635981761 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:21.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.909+0000 7fe96e473640 1 --2- 172.21.15.114:0/635981761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571bf1ffe40 0x5571bf2104a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:21.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.909+0000 7fe96dc72640 1 --2- 172.21.15.114:0/635981761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571bf204ed0 0x5571bf20b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:21.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.909+0000 7fe96dc72640 1 -- 172.21.15.114:0/635981761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571bf204ed0 msgr2=0x5571bf20b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:21.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.909+0000 7fe96dc72640 1 --2- 172.21.15.114:0/635981761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571bf204ed0 0x5571bf20b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.909+0000 7fe96dc72640 1 -- 172.21.15.114:0/635981761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571bf1ffe40 msgr2=0x5571bf2104a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:21.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.909+0000 7fe96dc72640 1 --2- 172.21.15.114:0/635981761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571bf1ffe40 0x5571bf2104a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.909+0000 7fe96dc72640 1 -- 172.21.15.114:0/635981761 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe95c025040 con 0x5571bf1ff500 2024-09-13T18:22:21.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.909+0000 7fe96dc72640 1 --2- 172.21.15.114:0/635981761 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5571bf1ff500 0x5571bf20af30 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe95c005f10 tx=0x7fe95c012640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7370fda6307702a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:21.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.909+0000 7fe94dffb640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:22:21.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.909+0000 7fe94dffb640 1 -- 172.21.15.114:0/635981761 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95c02e070 con 0x5571bf1ff500 2024-09-13T18:22:21.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.909+0000 7fe94dffb640 1 -- 172.21.15.114:0/635981761 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fe95c02f070 con 0x5571bf1ff500 2024-09-13T18:22:21.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.909+0000 7fe9733f8740 1 -- 172.21.15.114:0/635981761 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe95c010a80 con 0x5571bf1ff500 2024-09-13T18:22:21.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.909+0000 7fe94dffb640 1 -- 172.21.15.114:0/635981761 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe95c004c10 con 0x5571bf1ff500 2024-09-13T18:22:21.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.909+0000 7fe9733f8740 10 client.15459 fetch_fsmap learned FSMap version 29 2024-09-13T18:22:21.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.909+0000 7fe9733f8740 1 -- 172.21.15.114:0/635981761 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5571bf128d00 con 0x5571bf1ff500 2024-09-13T18:22:21.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.909+0000 7fe94dffb640 1 -- 172.21.15.114:0/635981761 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe95c010a80 con 0x5571bf1ff500 2024-09-13T18:22:21.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.910+0000 7fe94dffb640 1 -- 172.21.15.114:0/635981761 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe95c028070 con 0x5571bf1ff500 2024-09-13T18:22:21.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.910+0000 7fe9733f8740 10 client.15459 fetch_fsmap finished waiting for FSMap version 29 2024-09-13T18:22:21.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.910+0000 7fe9733f8740 10 client.15459 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-13T18:22:21.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.910+0000 7fe9733f8740 20 client.15459 populate_metadata read hostname 'smithi114' 2024-09-13T18:22:21.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.910+0000 7fe9733f8740 1 --2- 172.21.15.114:0/635981761 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x5571bf26ff80 0x5571bf26c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:21.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.910+0000 7fe9733f8740 4 client.15459 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T18:22:21.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.910+0000 7fe9733f8740 1 -- 172.21.15.114:0/635981761 --> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5571beebad10 con 0x5571bf26ff80 2024-09-13T18:22:21.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.910+0000 7fe96e473640 1 --2- 172.21.15.114:0/635981761 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x5571bf26ff80 0x5571bf26c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:21.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.910+0000 7fe94dffb640 10 client.15459 ms_handle_connect on v2:172.21.15.114:6840/1956923446 2024-09-13T18:22:21.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.910+0000 7fe96e473640 1 --2- 172.21.15.114:0/635981761 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x5571bf26ff80 0x5571bf26c920 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=88b516cbaf051118 server_cookie=a9001db769403816 in_seq=0 out_seq=0 2024-09-13T18:22:21.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.910+0000 7fe94dffb640 1 -- 172.21.15.114:0/635981761 <== mds.4 v2:172.21.15.114:6840/1956923446 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5571beebad10 con 0x5571bf26ff80 2024-09-13T18:22:21.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.910+0000 7fe94dffb640 10 client.15459 handle_command_reply: tid=0 2024-09-13T18:22:21.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.910+0000 7fe9733f8740 1 client.15459 shutdown 2024-09-13T18:22:21.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.910+0000 7fe9733f8740 1 -- 172.21.15.114:0/635981761 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5571bf1ff500 msgr2=0x5571bf20af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:21.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.910+0000 7fe9733f8740 1 --2- 172.21.15.114:0/635981761 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5571bf1ff500 0x5571bf20af30 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe95c005f10 tx=0x7fe95c012640 comp rx=0 tx=0).stop 2024-09-13T18:22:21.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.910+0000 7fe9733f8740 1 -- 172.21.15.114:0/635981761 shutdown_connections 2024-09-13T18:22:21.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.910+0000 7fe9733f8740 1 --2- 172.21.15.114:0/635981761 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x5571bf26ff80 0x5571bf26c920 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.910+0000 7fe9733f8740 1 --2- 172.21.15.114:0/635981761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571bf204ed0 0x5571bf20b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.910+0000 7fe94dffb640 0 client.15459 ms_handle_reset on v2:172.21.15.114:6840/1956923446 2024-09-13T18:22:21.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.910+0000 7fe9733f8740 1 --2- 172.21.15.114:0/635981761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571bf1ffe40 0x5571bf2104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.910+0000 7fe9733f8740 1 --2- 172.21.15.114:0/635981761 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5571bf1ff500 0x5571bf20af30 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.910+0000 7fe9733f8740 1 -- 172.21.15.114:0/635981761 >> 172.21.15.114:0/635981761 conn(0x5571bf1fab70 msgr2=0x5571bf21d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:21.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.910+0000 7fe9733f8740 1 -- 172.21.15.114:0/635981761 shutdown_connections 2024-09-13T18:22:21.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.910+0000 7fe9733f8740 1 -- 172.21.15.114:0/635981761 wait complete. 2024-09-13T18:22:21.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.911+0000 7fe9733f8740 20 client.15459 trim_cache size 0 max 16384 2024-09-13T18:22:21.920 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:22:21.920 INFO:teuthology.orchestra.run.smithi114.stdout: "cluster_fsid": "97e7edf2-71fb-11ef-bceb-c7b262605968", 2024-09-13T18:22:21.920 INFO:teuthology.orchestra.run.smithi114.stdout: "whoami": 4, 2024-09-13T18:22:21.920 INFO:teuthology.orchestra.run.smithi114.stdout: "id": 14628, 2024-09-13T18:22:21.920 INFO:teuthology.orchestra.run.smithi114.stdout: "want_state": "up:active", 2024-09-13T18:22:21.920 INFO:teuthology.orchestra.run.smithi114.stdout: "state": "up:active", 2024-09-13T18:22:21.920 INFO:teuthology.orchestra.run.smithi114.stdout: "fs_name": "cephfs", 2024-09-13T18:22:21.921 INFO:teuthology.orchestra.run.smithi114.stdout: "rank_uptime": 113.664638504, 2024-09-13T18:22:21.921 INFO:teuthology.orchestra.run.smithi114.stdout: "mdsmap_epoch": 29, 2024-09-13T18:22:21.921 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch": 93, 2024-09-13T18:22:21.921 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch_barrier": 91, 2024-09-13T18:22:21.921 INFO:teuthology.orchestra.run.smithi114.stdout: "uptime": 148.29109277200001 2024-09-13T18:22:21.921 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:22:21.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.911+0000 7fe9706d7640 1 -- 172.21.15.114:0/596668817 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fe9500a1110 msgr2=0x7fe9500a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:21.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.911+0000 7fe9706d7640 1 --2- 172.21.15.114:0/596668817 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fe9500a1110 0x7fe9500a35d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe95800afd0 tx=0x7fe958065680 comp rx=0 tx=0).stop 2024-09-13T18:22:21.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.911+0000 7fe9706d7640 1 -- 172.21.15.114:0/596668817 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe968101360 msgr2=0x7fe96819d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:21.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.911+0000 7fe9706d7640 1 --2- 172.21.15.114:0/596668817 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe968101360 0x7fe96819d180 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe95800c970 tx=0x7fe95800ce30 comp rx=0 tx=0).stop 2024-09-13T18:22:21.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.911+0000 7fe9706d7640 1 -- 172.21.15.114:0/596668817 shutdown_connections 2024-09-13T18:22:21.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.911+0000 7fe9706d7640 1 --2- 172.21.15.114:0/596668817 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fe9500a1110 0x7fe9500a35d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.911+0000 7fe9706d7640 1 --2- 172.21.15.114:0/596668817 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe968101360 0x7fe96819d180 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.911+0000 7fe9706d7640 1 --2- 172.21.15.114:0/596668817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9680fe790 0x7fe968196110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.911+0000 7fe9706d7640 1 --2- 172.21.15.114:0/596668817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9680fddd0 0x7fe968195bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:21.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.911+0000 7fe9706d7640 1 -- 172.21.15.114:0/596668817 >> 172.21.15.114:0/596668817 conn(0x7fe9680f9b60 msgr2=0x7fe968104500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:21.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.911+0000 7fe9706d7640 1 -- 172.21.15.114:0/596668817 shutdown_connections 2024-09-13T18:22:21.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:21.911+0000 7fe9706d7640 1 -- 172.21.15.114:0/596668817 wait complete. 2024-09-13T18:22:21.924 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-09-13T18:22:22.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.022+0000 7fb6911c3640 1 -- 172.21.15.114:0/3510313466 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb68c0fec70 msgr2=0x7fb68c0ff070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:22.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.022+0000 7fb6911c3640 1 --2- 172.21.15.114:0/3510313466 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb68c0fec70 0x7fb68c0ff070 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fb67c009900 tx=0x7fb67c02a410 comp rx=0 tx=0).stop 2024-09-13T18:22:22.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.022+0000 7fb6911c3640 1 -- 172.21.15.114:0/3510313466 shutdown_connections 2024-09-13T18:22:22.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.022+0000 7fb6911c3640 1 --2- 172.21.15.114:0/3510313466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb68c100820 0x7fb68c1070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.022+0000 7fb6911c3640 1 --2- 172.21.15.114:0/3510313466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb68c0ffe60 0x7fb68c1002e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.022+0000 7fb6911c3640 1 --2- 172.21.15.114:0/3510313466 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb68c0fec70 0x7fb68c0ff070 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.022+0000 7fb6911c3640 1 -- 172.21.15.114:0/3510313466 >> 172.21.15.114:0/3510313466 conn(0x7fb68c0fa410 msgr2=0x7fb68c0fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:22.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.022+0000 7fb6911c3640 1 -- 172.21.15.114:0/3510313466 shutdown_connections 2024-09-13T18:22:22.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.022+0000 7fb6911c3640 1 -- 172.21.15.114:0/3510313466 wait complete. 2024-09-13T18:22:22.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.023+0000 7fb6911c3640 1 Processor -- start 2024-09-13T18:22:22.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.023+0000 7fb6911c3640 1 -- start start 2024-09-13T18:22:22.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.023+0000 7fb6911c3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb68c0fec70 0x7fb68c068110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.023+0000 7fb6911c3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb68c0ffe60 0x7fb68c19a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.023+0000 7fb6911c3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb68c100820 0x7fb68c19cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.023+0000 7fb6911c3640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68c19d7c0 con 0x7fb68c0ffe60 2024-09-13T18:22:22.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.023+0000 7fb6911c3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68c19d930 con 0x7fb68c100820 2024-09-13T18:22:22.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.023+0000 7fb6911c3640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68c19daf0 con 0x7fb68c0fec70 2024-09-13T18:22:22.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.024+0000 7fb68259c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb68c0ffe60 0x7fb68c19a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.024+0000 7fb68259c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb68c0ffe60 0x7fb68c19a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43472/0 (socket says 172.21.15.114:43472) 2024-09-13T18:22:22.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.024+0000 7fb68259c640 1 -- 172.21.15.114:0/463164664 learned_addr learned my addr 172.21.15.114:0/463164664 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:22.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.024+0000 7fb68ad9d640 1 --2- 172.21.15.114:0/463164664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb68c0fec70 0x7fb68c068110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.024+0000 7fb68b59e640 1 --2- 172.21.15.114:0/463164664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb68c100820 0x7fb68c19cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.024+0000 7fb68259c640 1 -- 172.21.15.114:0/463164664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb68c0fec70 msgr2=0x7fb68c068110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:22.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.024+0000 7fb68259c640 1 --2- 172.21.15.114:0/463164664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb68c0fec70 0x7fb68c068110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.024+0000 7fb68259c640 1 -- 172.21.15.114:0/463164664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb68c100820 msgr2=0x7fb68c19cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:22.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.024+0000 7fb68259c640 1 --2- 172.21.15.114:0/463164664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb68c100820 0x7fb68c19cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.024+0000 7fb68259c640 1 -- 172.21.15.114:0/463164664 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb67c009560 con 0x7fb68c0ffe60 2024-09-13T18:22:22.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.024+0000 7fb68b59e640 1 --2- 172.21.15.114:0/463164664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb68c100820 0x7fb68c19cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:22:22.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.024+0000 7fb68259c640 1 --2- 172.21.15.114:0/463164664 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb68c0ffe60 0x7fb68c19a9c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb67400d950 tx=0x7fb67400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:22.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.024+0000 7fb688d99640 1 -- 172.21.15.114:0/463164664 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb674009a30 con 0x7fb68c0ffe60 2024-09-13T18:22:22.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.024+0000 7fb688d99640 1 -- 172.21.15.114:0/463164664 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb6740044b0 con 0x7fb68c0ffe60 2024-09-13T18:22:22.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.024+0000 7fb6911c3640 1 -- 172.21.15.114:0/463164664 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb68c19dd70 con 0x7fb68c0ffe60 2024-09-13T18:22:22.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.024+0000 7fb688d99640 1 -- 172.21.15.114:0/463164664 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6740047a0 con 0x7fb68c0ffe60 2024-09-13T18:22:22.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.024+0000 7fb6911c3640 1 -- 172.21.15.114:0/463164664 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb68c19e220 con 0x7fb68c0ffe60 2024-09-13T18:22:22.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.026+0000 7fb688d99640 1 -- 172.21.15.114:0/463164664 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fb674004020 con 0x7fb68c0ffe60 2024-09-13T18:22:22.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.027+0000 7fb693fbf740 1 Processor -- start 2024-09-13T18:22:22.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.027+0000 7fb693fbf740 1 -- start start 2024-09-13T18:22:22.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.027+0000 7fb693fbf740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595b71c0500 0x5595b71c0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.027+0000 7fb693fbf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595b71c0e40 0x5595b71c5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.027+0000 7fb693fbf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595b71c5ed0 0x5595b71c8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.027+0000 7fb693fbf740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b711b7c0 con 0x5595b71c0500 2024-09-13T18:22:22.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.027+0000 7fb693fbf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b712ab70 con 0x5595b71c5ed0 2024-09-13T18:22:22.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.027+0000 7fb693fbf740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b7128300 con 0x5595b71c0e40 2024-09-13T18:22:22.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.027+0000 7fb688d99640 1 --2- 172.21.15.114:0/463164664 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb6780a1090 0x7fb6780a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.027+0000 7fb68b59e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595b71c0500 0x5595b71c0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.027+0000 7fb68b59e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595b71c0500 0x5595b71c0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43480/0 (socket says 172.21.15.114:43480) 2024-09-13T18:22:22.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.027+0000 7fb68b59e640 1 -- 172.21.15.114:0/2592934995 learned_addr learned my addr 172.21.15.114:0/2592934995 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:22.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.027+0000 7fb68b59e640 1 --2- 172.21.15.114:0/463164664 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb6780a1090 0x7fb6780a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.027+0000 7fb68ad9d640 1 --2- 172.21.15.114:0/2592934995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595b71c0e40 0x5595b71c5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.027+0000 7fb688d99640 1 -- 172.21.15.114:0/463164664 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fb6740cc790 con 0x7fb68c0ffe60 2024-09-13T18:22:22.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.027+0000 7fb68259c640 1 --2- 172.21.15.114:0/2592934995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595b71c5ed0 0x5595b71c8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.028+0000 7fb68b59e640 1 -- 172.21.15.114:0/2592934995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595b71c0e40 msgr2=0x5595b71c5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:22.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.028+0000 7fb68b59e640 1 --2- 172.21.15.114:0/2592934995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595b71c0e40 0x5595b71c5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.028+0000 7fb68b59e640 1 -- 172.21.15.114:0/2592934995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595b71c5ed0 msgr2=0x5595b71c8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:22.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.028+0000 7fb68b59e640 1 --2- 172.21.15.114:0/2592934995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595b71c5ed0 0x5595b71c8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.028+0000 7fb68b59e640 1 -- 172.21.15.114:0/2592934995 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595b70b04b0 con 0x5595b71c0500 2024-09-13T18:22:22.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.028+0000 7fb68259c640 1 --2- 172.21.15.114:0/2592934995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595b71c5ed0 0x5595b71c8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:22:22.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.028+0000 7fb68b59e640 1 --2- 172.21.15.114:0/2592934995 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595b71c0500 0x5595b71c0900 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb684014eb0 tx=0x7fb684011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4e8d7f891d5f1cd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:22.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.028+0000 7fb68b59e640 1 --2- 172.21.15.114:0/463164664 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb6780a1090 0x7fb6780a3550 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb684011c50 tx=0x7fb68401c000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:22.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.028+0000 7fb68159a640 1 -- 172.21.15.114:0/2592934995 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68400bb50 con 0x5595b71c0500 2024-09-13T18:22:22.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.028+0000 7fb68159a640 1 -- 172.21.15.114:0/2592934995 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb68401a040 con 0x5595b71c0500 2024-09-13T18:22:22.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.028+0000 7fb68159a640 1 -- 172.21.15.114:0/2592934995 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb684020cb0 con 0x5595b71c0500 2024-09-13T18:22:22.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.029+0000 7fb693fbf740 1 -- 172.21.15.114:0/2592934995 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595b71c0500 msgr2=0x5595b71c0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:22.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.029+0000 7fb693fbf740 1 --2- 172.21.15.114:0/2592934995 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595b71c0500 0x5595b71c0900 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb684014eb0 tx=0x7fb684011750 comp rx=0 tx=0).stop 2024-09-13T18:22:22.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.029+0000 7fb693fbf740 1 -- 172.21.15.114:0/2592934995 shutdown_connections 2024-09-13T18:22:22.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.029+0000 7fb693fbf740 1 --2- 172.21.15.114:0/2592934995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595b71c5ed0 0x5595b71c8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.029+0000 7fb693fbf740 1 --2- 172.21.15.114:0/2592934995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595b71c0e40 0x5595b71c5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.029+0000 7fb693fbf740 1 --2- 172.21.15.114:0/2592934995 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595b71c0500 0x5595b71c0900 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.029+0000 7fb693fbf740 1 -- 172.21.15.114:0/2592934995 >> 172.21.15.114:0/2592934995 conn(0x5595b71bbb70 msgr2=0x5595b71bbf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:22.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.029+0000 7fb693fbf740 1 -- 172.21.15.114:0/2592934995 shutdown_connections 2024-09-13T18:22:22.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.029+0000 7fb693fbf740 1 -- 172.21.15.114:0/2592934995 wait complete. 2024-09-13T18:22:22.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.029+0000 7fb693fbf740 1 -- start start 2024-09-13T18:22:22.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.029+0000 7fb693fbf740 1 Processor -- start 2024-09-13T18:22:22.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.029+0000 7fb693fbf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595b71c0500 0x5595b71ca1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.029+0000 7fb693fbf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595b71c0e40 0x5595b71ca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.029+0000 7fb693fbf740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595b71c5ed0 0x5595b71d16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.029+0000 7fb693fbf740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b7123bf0 con 0x5595b71c5ed0 2024-09-13T18:22:22.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.029+0000 7fb693fbf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b70e2250 con 0x5595b71c0500 2024-09-13T18:22:22.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.029+0000 7fb693fbf740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b70e1ef0 con 0x5595b71c0e40 2024-09-13T18:22:22.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.029+0000 7fb693fbf740 10 client.0 _ll_register_callbacks cb 0x5595b7114740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:22:22.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.029+0000 7fb68b59e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595b71c5ed0 0x5595b71d16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.029+0000 7fb68b59e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595b71c5ed0 0x5595b71d16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43486/0 (socket says 172.21.15.114:43486) 2024-09-13T18:22:22.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.029+0000 7fb68b59e640 1 -- 172.21.15.114:0/82709519 learned_addr learned my addr 172.21.15.114:0/82709519 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:22.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.030+0000 7fb68259c640 1 --2- 172.21.15.114:0/82709519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595b71c0e40 0x5595b71ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.030+0000 7fb68ad9d640 1 --2- 172.21.15.114:0/82709519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595b71c0500 0x5595b71ca1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.030+0000 7fb68b59e640 1 -- 172.21.15.114:0/82709519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595b71c0e40 msgr2=0x5595b71ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:22.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.030+0000 7fb68b59e640 1 --2- 172.21.15.114:0/82709519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595b71c0e40 0x5595b71ca700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.030+0000 7fb68b59e640 1 -- 172.21.15.114:0/82709519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595b71c0500 msgr2=0x5595b71ca1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:22.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.030+0000 7fb68b59e640 1 --2- 172.21.15.114:0/82709519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595b71c0500 0x5595b71ca1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.030+0000 7fb68b59e640 1 -- 172.21.15.114:0/82709519 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb684014a90 con 0x5595b71c5ed0 2024-09-13T18:22:22.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.030+0000 7fb68259c640 1 --2- 172.21.15.114:0/82709519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595b71c0e40 0x5595b71ca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:22:22.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.030+0000 7fb68ad9d640 1 --2- 172.21.15.114:0/82709519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595b71c0500 0x5595b71ca1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:22:22.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.030+0000 7fb68b59e640 1 --2- 172.21.15.114:0/82709519 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595b71c5ed0 0x5595b71d16a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fb684005c10 tx=0x7fb684005a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12e517c5b163449f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:22.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.030+0000 7fb6727fc640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:22:22.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.030+0000 7fb6727fc640 1 -- 172.21.15.114:0/82709519 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb684009d70 con 0x5595b71c5ed0 2024-09-13T18:22:22.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.030+0000 7fb6727fc640 1 -- 172.21.15.114:0/82709519 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fb684020440 con 0x5595b71c5ed0 2024-09-13T18:22:22.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.030+0000 7fb693fbf740 1 -- 172.21.15.114:0/82709519 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5595b6e173c0 con 0x5595b71c5ed0 2024-09-13T18:22:22.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.030+0000 7fb6727fc640 1 -- 172.21.15.114:0/82709519 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb684020600 con 0x5595b71c5ed0 2024-09-13T18:22:22.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.030+0000 7fb693fbf740 10 client.15471 fetch_fsmap learned FSMap version 29 2024-09-13T18:22:22.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.030+0000 7fb693fbf740 1 -- 172.21.15.114:0/82709519 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5595b7122270 con 0x5595b71c5ed0 2024-09-13T18:22:22.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.030+0000 7fb6727fc640 1 -- 172.21.15.114:0/82709519 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5595b6e173c0 con 0x5595b71c5ed0 2024-09-13T18:22:22.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.031+0000 7fb6727fc640 1 -- 172.21.15.114:0/82709519 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb684033410 con 0x5595b71c5ed0 2024-09-13T18:22:22.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.031+0000 7fb693fbf740 10 client.15471 fetch_fsmap finished waiting for FSMap version 29 2024-09-13T18:22:22.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.031+0000 7fb693fbf740 10 client.15471 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-13T18:22:22.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.031+0000 7fb693fbf740 20 client.15471 populate_metadata read hostname 'smithi114' 2024-09-13T18:22:22.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.031+0000 7fb693fbf740 1 --2- 172.21.15.114:0/82709519 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x5595b71d6580 0x5595b71da720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.031+0000 7fb693fbf740 4 client.15471 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:22:22.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.031+0000 7fb693fbf740 1 -- 172.21.15.114:0/82709519 --> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5595b6e286c0 con 0x5595b71d6580 2024-09-13T18:22:22.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.031+0000 7fb68ad9d640 1 --2- 172.21.15.114:0/82709519 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x5595b71d6580 0x5595b71da720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.031+0000 7fb6727fc640 10 client.15471 ms_handle_connect on v2:172.21.15.134:6834/1150003275 2024-09-13T18:22:22.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.031+0000 7fb68ad9d640 1 --2- 172.21.15.114:0/82709519 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x5595b71d6580 0x5595b71da720 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=43dc547bdcecea67 server_cookie=22ff6dc1d30065a1 in_seq=0 out_seq=0 2024-09-13T18:22:22.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.032+0000 7fb6727fc640 1 -- 172.21.15.114:0/82709519 <== mds.3 v2:172.21.15.134:6834/1150003275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5595b6e286c0 con 0x5595b71d6580 2024-09-13T18:22:22.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.032+0000 7fb6727fc640 10 client.15471 handle_command_reply: tid=0 2024-09-13T18:22:22.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.032+0000 7fb693fbf740 1 client.15471 shutdown 2024-09-13T18:22:22.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.032+0000 7fb693fbf740 1 -- 172.21.15.114:0/82709519 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595b71c5ed0 msgr2=0x5595b71d16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:22.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.032+0000 7fb693fbf740 1 --2- 172.21.15.114:0/82709519 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595b71c5ed0 0x5595b71d16a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fb684005c10 tx=0x7fb684005a10 comp rx=0 tx=0).stop 2024-09-13T18:22:22.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.032+0000 7fb693fbf740 1 -- 172.21.15.114:0/82709519 shutdown_connections 2024-09-13T18:22:22.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.032+0000 7fb693fbf740 1 --2- 172.21.15.114:0/82709519 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x5595b71d6580 0x5595b71da720 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.032+0000 7fb693fbf740 1 --2- 172.21.15.114:0/82709519 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595b71c5ed0 0x5595b71d16a0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.032+0000 7fb693fbf740 1 --2- 172.21.15.114:0/82709519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595b71c0e40 0x5595b71ca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.032+0000 7fb6727fc640 0 client.15471 ms_handle_reset on v2:172.21.15.134:6834/1150003275 2024-09-13T18:22:22.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.032+0000 7fb693fbf740 1 --2- 172.21.15.114:0/82709519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595b71c0500 0x5595b71ca1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.032+0000 7fb693fbf740 1 -- 172.21.15.114:0/82709519 >> 172.21.15.114:0/82709519 conn(0x5595b71bbb70 msgr2=0x5595b71bd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:22.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.032+0000 7fb693fbf740 1 -- 172.21.15.114:0/82709519 shutdown_connections 2024-09-13T18:22:22.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.032+0000 7fb693fbf740 1 -- 172.21.15.114:0/82709519 wait complete. 2024-09-13T18:22:22.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.033+0000 7fb693fbf740 20 client.15471 trim_cache size 0 max 16384 2024-09-13T18:22:22.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.047+0000 7fb693fbf740 1 Processor -- start 2024-09-13T18:22:22.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.047+0000 7fb693fbf740 1 -- start start 2024-09-13T18:22:22.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.047+0000 7fb693fbf740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595b71c0500 0x5595b71ca680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.047+0000 7fb693fbf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595b71c0e40 0x5595b71d15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.047+0000 7fb693fbf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595b71c5ed0 0x5595b71c9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.047+0000 7fb693fbf740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b70a7460 con 0x5595b71c0500 2024-09-13T18:22:22.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.047+0000 7fb693fbf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b6e286c0 con 0x5595b71c0e40 2024-09-13T18:22:22.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.047+0000 7fb693fbf740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b6e173c0 con 0x5595b71c5ed0 2024-09-13T18:22:22.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.047+0000 7fb68ad9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595b71c0500 0x5595b71ca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.047+0000 7fb68ad9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595b71c0500 0x5595b71ca680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43490/0 (socket says 172.21.15.114:43490) 2024-09-13T18:22:22.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.047+0000 7fb68ad9d640 1 -- 172.21.15.114:0/373912793 learned_addr learned my addr 172.21.15.114:0/373912793 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:22.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.048+0000 7fb68b59e640 1 --2- 172.21.15.114:0/373912793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595b71c5ed0 0x5595b71c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.048+0000 7fb68259c640 1 --2- 172.21.15.114:0/373912793 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595b71c0e40 0x5595b71d15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.048+0000 7fb68ad9d640 1 -- 172.21.15.114:0/373912793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595b71c5ed0 msgr2=0x5595b71c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:22.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.048+0000 7fb68ad9d640 1 --2- 172.21.15.114:0/373912793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595b71c5ed0 0x5595b71c9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.048+0000 7fb68ad9d640 1 -- 172.21.15.114:0/373912793 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595b71c0e40 msgr2=0x5595b71d15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:22.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.048+0000 7fb68ad9d640 1 --2- 172.21.15.114:0/373912793 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595b71c0e40 0x5595b71d15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.048+0000 7fb68ad9d640 1 -- 172.21.15.114:0/373912793 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595b70c45f0 con 0x5595b71c0500 2024-09-13T18:22:22.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.048+0000 7fb68ad9d640 1 --2- 172.21.15.114:0/373912793 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595b71c0500 0x5595b71ca680 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb67c006010 tx=0x7fb67c0058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7e1b5d3874c8308 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:22.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.048+0000 7fb681d9b640 1 -- 172.21.15.114:0/373912793 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67c048070 con 0x5595b71c0500 2024-09-13T18:22:22.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.048+0000 7fb681d9b640 1 -- 172.21.15.114:0/373912793 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb67c0040c0 con 0x5595b71c0500 2024-09-13T18:22:22.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.048+0000 7fb681d9b640 1 -- 172.21.15.114:0/373912793 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67c031870 con 0x5595b71c0500 2024-09-13T18:22:22.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.049+0000 7fb693fbf740 1 -- 172.21.15.114:0/373912793 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595b71c0500 msgr2=0x5595b71ca680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:22.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.049+0000 7fb693fbf740 1 --2- 172.21.15.114:0/373912793 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595b71c0500 0x5595b71ca680 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb67c006010 tx=0x7fb67c0058d0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.049+0000 7fb693fbf740 1 -- 172.21.15.114:0/373912793 shutdown_connections 2024-09-13T18:22:22.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.049+0000 7fb693fbf740 1 --2- 172.21.15.114:0/373912793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595b71c5ed0 0x5595b71c9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.049+0000 7fb693fbf740 1 --2- 172.21.15.114:0/373912793 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595b71c0e40 0x5595b71d15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.049+0000 7fb693fbf740 1 --2- 172.21.15.114:0/373912793 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595b71c0500 0x5595b71ca680 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.049+0000 7fb693fbf740 1 -- 172.21.15.114:0/373912793 >> 172.21.15.114:0/373912793 conn(0x5595b71bbb70 msgr2=0x5595b71c9f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:22.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.049+0000 7fb693fbf740 1 -- 172.21.15.114:0/373912793 shutdown_connections 2024-09-13T18:22:22.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.049+0000 7fb693fbf740 1 -- 172.21.15.114:0/373912793 wait complete. 2024-09-13T18:22:22.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.049+0000 7fb693fbf740 1 -- start start 2024-09-13T18:22:22.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.049+0000 7fb693fbf740 1 Processor -- start 2024-09-13T18:22:22.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.050+0000 7fb693fbf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595b71c0500 0x5595b71cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.050+0000 7fb693fbf740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595b71c0e40 0x5595b71d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.050+0000 7fb693fbf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595b71c5ed0 0x5595b71cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.050+0000 7fb693fbf740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b71cc8d0 con 0x5595b71c0e40 2024-09-13T18:22:22.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.050+0000 7fb693fbf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b71cca40 con 0x5595b71c5ed0 2024-09-13T18:22:22.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.050+0000 7fb693fbf740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b722c430 con 0x5595b71c0500 2024-09-13T18:22:22.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.050+0000 7fb693fbf740 10 client.0 _ll_register_callbacks cb 0x7fb684009f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:22:22.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.050+0000 7fb68259c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595b71c0e40 0x5595b71d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.050+0000 7fb68ad9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595b71c0500 0x5595b71cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.050+0000 7fb68b59e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595b71c5ed0 0x5595b71cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.050+0000 7fb68259c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595b71c0e40 0x5595b71d1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43506/0 (socket says 172.21.15.114:43506) 2024-09-13T18:22:22.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.050+0000 7fb68259c640 1 -- 172.21.15.114:0/4115435783 learned_addr learned my addr 172.21.15.114:0/4115435783 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:22.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.050+0000 7fb68259c640 1 -- 172.21.15.114:0/4115435783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595b71c0500 msgr2=0x5595b71cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:22.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.050+0000 7fb68259c640 1 --2- 172.21.15.114:0/4115435783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595b71c0500 0x5595b71cbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.050+0000 7fb68259c640 1 -- 172.21.15.114:0/4115435783 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595b71c5ed0 msgr2=0x5595b71cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:22.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.050+0000 7fb68259c640 1 --2- 172.21.15.114:0/4115435783 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595b71c5ed0 0x5595b71cc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.050+0000 7fb68259c640 1 -- 172.21.15.114:0/4115435783 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb68c1079b0 con 0x5595b71c0e40 2024-09-13T18:22:22.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.050+0000 7fb68259c640 1 --2- 172.21.15.114:0/4115435783 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595b71c0e40 0x5595b71d1340 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb674095d70 tx=0x7fb67400e350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3109825c0aa31da server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:22.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.050+0000 7fb6727fc640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:22:22.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.051+0000 7fb6727fc640 1 -- 172.21.15.114:0/4115435783 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb674099870 con 0x5595b71c0e40 2024-09-13T18:22:22.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.051+0000 7fb6727fc640 1 -- 172.21.15.114:0/4115435783 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fb67409d6a0 con 0x5595b71c0e40 2024-09-13T18:22:22.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.051+0000 7fb693fbf740 1 -- 172.21.15.114:0/4115435783 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb68401a040 con 0x5595b71c0e40 2024-09-13T18:22:22.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.051+0000 7fb6727fc640 1 -- 172.21.15.114:0/4115435783 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6740a1cf0 con 0x5595b71c0e40 2024-09-13T18:22:22.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.051+0000 7fb693fbf740 10 client.15477 fetch_fsmap learned FSMap version 29 2024-09-13T18:22:22.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.051+0000 7fb693fbf740 1 -- 172.21.15.114:0/4115435783 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5595b70e9d00 con 0x5595b71c0e40 2024-09-13T18:22:22.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.051+0000 7fb6727fc640 1 -- 172.21.15.114:0/4115435783 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb68401a040 con 0x5595b71c0e40 2024-09-13T18:22:22.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.051+0000 7fb6727fc640 1 -- 172.21.15.114:0/4115435783 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb674098e60 con 0x5595b71c0e40 2024-09-13T18:22:22.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.052+0000 7fb693fbf740 10 client.15477 fetch_fsmap finished waiting for FSMap version 29 2024-09-13T18:22:22.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.052+0000 7fb693fbf740 10 client.15477 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-13T18:22:22.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.052+0000 7fb693fbf740 20 client.15477 populate_metadata read hostname 'smithi114' 2024-09-13T18:22:22.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.052+0000 7fb693fbf740 1 --2- 172.21.15.114:0/4115435783 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x5595b72312a0 0x5595b722dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.052+0000 7fb693fbf740 4 client.15477 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T18:22:22.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.052+0000 7fb693fbf740 1 -- 172.21.15.114:0/4115435783 --> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5595b6e7bd10 con 0x5595b72312a0 2024-09-13T18:22:22.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.052+0000 7fb68ad9d640 1 --2- 172.21.15.114:0/4115435783 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x5595b72312a0 0x5595b722dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.052+0000 7fb68ad9d640 1 --2- 172.21.15.114:0/4115435783 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x5595b72312a0 0x5595b722dbd0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4d3405b36ff5d62d server_cookie=dc0b9082864dff6c in_seq=0 out_seq=0 2024-09-13T18:22:22.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.052+0000 7fb6727fc640 10 client.15477 ms_handle_connect on v2:172.21.15.134:6834/1150003275 2024-09-13T18:22:22.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.052+0000 7fb6727fc640 1 -- 172.21.15.114:0/4115435783 <== mds.3 v2:172.21.15.134:6834/1150003275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x5595b6e7bd10 con 0x5595b72312a0 2024-09-13T18:22:22.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.052+0000 7fb6727fc640 10 client.15477 handle_command_reply: tid=0 2024-09-13T18:22:22.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.052+0000 7fb693fbf740 1 client.15477 shutdown 2024-09-13T18:22:22.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.053+0000 7fb693fbf740 1 -- 172.21.15.114:0/4115435783 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595b71c0e40 msgr2=0x5595b71d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:22.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.053+0000 7fb693fbf740 1 --2- 172.21.15.114:0/4115435783 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595b71c0e40 0x5595b71d1340 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb674095d70 tx=0x7fb67400e350 comp rx=0 tx=0).stop 2024-09-13T18:22:22.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.053+0000 7fb693fbf740 1 -- 172.21.15.114:0/4115435783 shutdown_connections 2024-09-13T18:22:22.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.053+0000 7fb693fbf740 1 --2- 172.21.15.114:0/4115435783 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x5595b72312a0 0x5595b722dbd0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.053+0000 7fb693fbf740 1 --2- 172.21.15.114:0/4115435783 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595b71c5ed0 0x5595b71cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.053+0000 7fb6727fc640 0 client.15477 ms_handle_reset on v2:172.21.15.134:6834/1150003275 2024-09-13T18:22:22.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.053+0000 7fb693fbf740 1 --2- 172.21.15.114:0/4115435783 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595b71c0e40 0x5595b71d1340 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.053+0000 7fb693fbf740 1 --2- 172.21.15.114:0/4115435783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595b71c0500 0x5595b71cbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.053+0000 7fb693fbf740 1 -- 172.21.15.114:0/4115435783 >> 172.21.15.114:0/4115435783 conn(0x5595b71bbb70 msgr2=0x5595b71c9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:22.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.053+0000 7fb693fbf740 1 -- 172.21.15.114:0/4115435783 shutdown_connections 2024-09-13T18:22:22.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.053+0000 7fb693fbf740 1 -- 172.21.15.114:0/4115435783 wait complete. 2024-09-13T18:22:22.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.053+0000 7fb693fbf740 20 client.15477 trim_cache size 0 max 16384 2024-09-13T18:22:22.060 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:22:22.061 INFO:teuthology.orchestra.run.smithi114.stdout: "cluster_fsid": "97e7edf2-71fb-11ef-bceb-c7b262605968", 2024-09-13T18:22:22.061 INFO:teuthology.orchestra.run.smithi114.stdout: "whoami": 3, 2024-09-13T18:22:22.061 INFO:teuthology.orchestra.run.smithi114.stdout: "id": 24458, 2024-09-13T18:22:22.061 INFO:teuthology.orchestra.run.smithi114.stdout: "want_state": "up:active", 2024-09-13T18:22:22.061 INFO:teuthology.orchestra.run.smithi114.stdout: "state": "up:active", 2024-09-13T18:22:22.061 INFO:teuthology.orchestra.run.smithi114.stdout: "fs_name": "cephfs", 2024-09-13T18:22:22.061 INFO:teuthology.orchestra.run.smithi114.stdout: "rank_uptime": 113.857466198, 2024-09-13T18:22:22.061 INFO:teuthology.orchestra.run.smithi114.stdout: "mdsmap_epoch": 29, 2024-09-13T18:22:22.061 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch": 93, 2024-09-13T18:22:22.061 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch_barrier": 91, 2024-09-13T18:22:22.061 INFO:teuthology.orchestra.run.smithi114.stdout: "uptime": 164.470129445 2024-09-13T18:22:22.061 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:22:22.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.053+0000 7fb6911c3640 1 -- 172.21.15.114:0/463164664 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb6780a1090 msgr2=0x7fb6780a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:22.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.053+0000 7fb6911c3640 1 --2- 172.21.15.114:0/463164664 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb6780a1090 0x7fb6780a3550 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb684011c50 tx=0x7fb68401c000 comp rx=0 tx=0).stop 2024-09-13T18:22:22.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.053+0000 7fb6911c3640 1 -- 172.21.15.114:0/463164664 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb68c0ffe60 msgr2=0x7fb68c19a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:22.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.053+0000 7fb6911c3640 1 --2- 172.21.15.114:0/463164664 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb68c0ffe60 0x7fb68c19a9c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb67400d950 tx=0x7fb67400de10 comp rx=0 tx=0).stop 2024-09-13T18:22:22.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.053+0000 7fb6911c3640 1 -- 172.21.15.114:0/463164664 shutdown_connections 2024-09-13T18:22:22.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.053+0000 7fb6911c3640 1 --2- 172.21.15.114:0/463164664 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fb6780a1090 0x7fb6780a3550 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.053+0000 7fb6911c3640 1 --2- 172.21.15.114:0/463164664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb68c100820 0x7fb68c19cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.053+0000 7fb6911c3640 1 --2- 172.21.15.114:0/463164664 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb68c0ffe60 0x7fb68c19a9c0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.053+0000 7fb6911c3640 1 --2- 172.21.15.114:0/463164664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb68c0fec70 0x7fb68c068110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.053+0000 7fb6911c3640 1 -- 172.21.15.114:0/463164664 >> 172.21.15.114:0/463164664 conn(0x7fb68c0fa410 msgr2=0x7fb68c103340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:22.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.053+0000 7fb6911c3640 1 -- 172.21.15.114:0/463164664 shutdown_connections 2024-09-13T18:22:22.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.053+0000 7fb6911c3640 1 -- 172.21.15.114:0/463164664 wait complete. 2024-09-13T18:22:22.066 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-09-13T18:22:22.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.166+0000 7f43a3c32640 1 -- 172.21.15.114:0/2512761654 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f439c100f60 msgr2=0x7f439c1013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:22.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.166+0000 7f43a3c32640 1 --2- 172.21.15.114:0/2512761654 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f439c100f60 0x7f439c1013e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f4390009900 tx=0x7f439002a410 comp rx=0 tx=0).stop 2024-09-13T18:22:22.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.166+0000 7f43a3c32640 1 -- 172.21.15.114:0/2512761654 shutdown_connections 2024-09-13T18:22:22.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.166+0000 7f43a3c32640 1 --2- 172.21.15.114:0/2512761654 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f439c101920 0x7f439c1081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.166+0000 7f43a3c32640 1 --2- 172.21.15.114:0/2512761654 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f439c100f60 0x7f439c1013e0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.166+0000 7f43a3c32640 1 --2- 172.21.15.114:0/2512761654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f439c0ffd70 0x7f439c100170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.166+0000 7f43a3c32640 1 -- 172.21.15.114:0/2512761654 >> 172.21.15.114:0/2512761654 conn(0x7f439c0fb4d0 msgr2=0x7f439c0fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:22.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.166+0000 7f43a3c32640 1 -- 172.21.15.114:0/2512761654 shutdown_connections 2024-09-13T18:22:22.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.166+0000 7f43a3c32640 1 -- 172.21.15.114:0/2512761654 wait complete. 2024-09-13T18:22:22.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.166+0000 7f43a3c32640 1 Processor -- start 2024-09-13T18:22:22.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.166+0000 7f43a3c32640 1 -- start start 2024-09-13T18:22:22.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.167+0000 7f43a3c32640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f439c0ffd70 0x7f439c074260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.167+0000 7f43a3c32640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f439c100f60 0x7f439c0728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.167+0000 7f43a3c32640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f439c101920 0x7f439c072de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.167+0000 7f43a3c32640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f439c073540 con 0x7f439c100f60 2024-09-13T18:22:22.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.167+0000 7f43a3c32640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f439c0736b0 con 0x7f439c0ffd70 2024-09-13T18:22:22.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.167+0000 7f43a3c32640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f439c19fcd0 con 0x7f439c101920 2024-09-13T18:22:22.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.167+0000 7f43a11cd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f439c100f60 0x7f439c0728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.167+0000 7f43a11cd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f439c100f60 0x7f439c0728a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43514/0 (socket says 172.21.15.114:43514) 2024-09-13T18:22:22.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.167+0000 7f43a11cd640 1 -- 172.21.15.114:0/1414727786 learned_addr learned my addr 172.21.15.114:0/1414727786 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:22.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.167+0000 7f43a21cf640 1 --2- 172.21.15.114:0/1414727786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f439c101920 0x7f439c072de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.167+0000 7f43a19ce640 1 --2- 172.21.15.114:0/1414727786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f439c0ffd70 0x7f439c074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.167+0000 7f43a11cd640 1 -- 172.21.15.114:0/1414727786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f439c101920 msgr2=0x7f439c072de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:22.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.167+0000 7f43a11cd640 1 --2- 172.21.15.114:0/1414727786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f439c101920 0x7f439c072de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.167+0000 7f43a11cd640 1 -- 172.21.15.114:0/1414727786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f439c0ffd70 msgr2=0x7f439c074260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:22.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.167+0000 7f43a11cd640 1 --2- 172.21.15.114:0/1414727786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f439c0ffd70 0x7f439c074260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.167+0000 7f43a11cd640 1 -- 172.21.15.114:0/1414727786 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4390009560 con 0x7f439c100f60 2024-09-13T18:22:22.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.167+0000 7f43a11cd640 1 --2- 172.21.15.114:0/1414727786 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f439c100f60 0x7f439c0728a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4390005ab0 tx=0x7f43900042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:22.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.167+0000 7f438effd640 1 -- 172.21.15.114:0/1414727786 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4390031d70 con 0x7f439c100f60 2024-09-13T18:22:22.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.167+0000 7f438effd640 1 -- 172.21.15.114:0/1414727786 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f43900044e0 con 0x7f439c100f60 2024-09-13T18:22:22.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.167+0000 7f43a3c32640 1 -- 172.21.15.114:0/1414727786 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f439c19ff20 con 0x7f439c100f60 2024-09-13T18:22:22.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.167+0000 7f438effd640 1 -- 172.21.15.114:0/1414727786 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4390003ac0 con 0x7f439c100f60 2024-09-13T18:22:22.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.167+0000 7f43a3c32640 1 -- 172.21.15.114:0/1414727786 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f439c1a0340 con 0x7f439c100f60 2024-09-13T18:22:22.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.170+0000 7f438effd640 1 -- 172.21.15.114:0/1414727786 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f4390003c20 con 0x7f439c100f60 2024-09-13T18:22:22.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.170+0000 7f43a6a30740 1 Processor -- start 2024-09-13T18:22:22.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.170+0000 7f43a6a30740 1 -- start start 2024-09-13T18:22:22.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.170+0000 7f43a6a30740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c69292e500 0x55c69292e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.170+0000 7f43a6a30740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c69292ee40 0x55c692933900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.170+0000 7f43a6a30740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c692933ed0 0x55c692936390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.170+0000 7f43a6a30740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6928897c0 con 0x55c692933ed0 2024-09-13T18:22:22.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.170+0000 7f43a6a30740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c692898b70 con 0x55c69292e500 2024-09-13T18:22:22.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.170+0000 7f43a6a30740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c692896300 con 0x55c69292ee40 2024-09-13T18:22:22.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.170+0000 7f43a11cd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c692933ed0 0x55c692936390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.170+0000 7f43a11cd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c692933ed0 0x55c692936390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43528/0 (socket says 172.21.15.114:43528) 2024-09-13T18:22:22.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.170+0000 7f43a11cd640 1 -- 172.21.15.114:0/1850157165 learned_addr learned my addr 172.21.15.114:0/1850157165 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:22.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.170+0000 7f43a21cf640 1 --2- 172.21.15.114:0/1850157165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c69292e500 0x55c69292e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.171+0000 7f43a19ce640 1 --2- 172.21.15.114:0/1850157165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c69292ee40 0x55c692933900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.171+0000 7f438effd640 1 --2- 172.21.15.114:0/1414727786 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f43880a1090 0x7f43880a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.171+0000 7f43a11cd640 1 -- 172.21.15.114:0/1850157165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c69292ee40 msgr2=0x55c692933900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:22.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.171+0000 7f43a11cd640 1 --2- 172.21.15.114:0/1850157165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c69292ee40 0x55c692933900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.171+0000 7f43a11cd640 1 -- 172.21.15.114:0/1850157165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c69292e500 msgr2=0x55c69292e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:22.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.171+0000 7f43a11cd640 1 --2- 172.21.15.114:0/1850157165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c69292e500 0x55c69292e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.171+0000 7f43a11cd640 1 -- 172.21.15.114:0/1850157165 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c69281e4b0 con 0x55c692933ed0 2024-09-13T18:22:22.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.171+0000 7f43a19ce640 1 --2- 172.21.15.114:0/1850157165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c69292ee40 0x55c692933900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:22:22.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.171+0000 7f43a21cf640 1 --2- 172.21.15.114:0/1414727786 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f43880a1090 0x7f43880a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.171+0000 7f438effd640 1 -- 172.21.15.114:0/1414727786 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f43900edb90 con 0x7f439c100f60 2024-09-13T18:22:22.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.171+0000 7f43a11cd640 1 --2- 172.21.15.114:0/1850157165 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c692933ed0 0x55c692936390 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f43900d5150 tx=0x7f43900b6b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25c2cff89abcf15d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:22.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.171+0000 7f4387fff640 1 -- 172.21.15.114:0/1850157165 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4390003ed0 con 0x55c692933ed0 2024-09-13T18:22:22.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.171+0000 7f4387fff640 1 -- 172.21.15.114:0/1850157165 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f43900f2040 con 0x55c692933ed0 2024-09-13T18:22:22.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.171+0000 7f4387fff640 1 -- 172.21.15.114:0/1850157165 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43900c2a50 con 0x55c692933ed0 2024-09-13T18:22:22.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.172+0000 7f43a6a30740 1 -- 172.21.15.114:0/1850157165 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c692933ed0 msgr2=0x55c692936390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:22.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.172+0000 7f43a21cf640 1 --2- 172.21.15.114:0/1414727786 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f43880a1090 0x7f43880a3550 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4394005fd0 tx=0x7f4394004a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:22.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.172+0000 7f43a6a30740 1 --2- 172.21.15.114:0/1850157165 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c692933ed0 0x55c692936390 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f43900d5150 tx=0x7f43900b6b10 comp rx=0 tx=0).stop 2024-09-13T18:22:22.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.172+0000 7f43a6a30740 1 -- 172.21.15.114:0/1850157165 shutdown_connections 2024-09-13T18:22:22.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.172+0000 7f43a6a30740 1 --2- 172.21.15.114:0/1850157165 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c692933ed0 0x55c692936390 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.172+0000 7f43a6a30740 1 --2- 172.21.15.114:0/1850157165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c69292ee40 0x55c692933900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.172+0000 7f43a6a30740 1 --2- 172.21.15.114:0/1850157165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c69292e500 0x55c69292e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.172+0000 7f43a6a30740 1 -- 172.21.15.114:0/1850157165 >> 172.21.15.114:0/1850157165 conn(0x55c692929b70 msgr2=0x55c692929f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:22.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.172+0000 7f43a6a30740 1 -- 172.21.15.114:0/1850157165 shutdown_connections 2024-09-13T18:22:22.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.172+0000 7f43a6a30740 1 -- 172.21.15.114:0/1850157165 wait complete. 2024-09-13T18:22:22.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.172+0000 7f43a6a30740 1 -- start start 2024-09-13T18:22:22.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.172+0000 7f43a6a30740 1 Processor -- start 2024-09-13T18:22:22.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.173+0000 7f43a6a30740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c69292e500 0x55c692937cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.173+0000 7f43a6a30740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c69292ee40 0x55c692938210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.173+0000 7f43a6a30740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c692933ed0 0x55c69293f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.173+0000 7f43a6a30740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c692891bf0 con 0x55c692933ed0 2024-09-13T18:22:22.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.173+0000 7f43a6a30740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c692850250 con 0x55c69292ee40 2024-09-13T18:22:22.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.173+0000 7f43a6a30740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c69284fef0 con 0x55c69292e500 2024-09-13T18:22:22.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.173+0000 7f43a6a30740 10 client.0 _ll_register_callbacks cb 0x55c692882740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:22:22.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.173+0000 7f43a21cf640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c692933ed0 0x55c69293f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.173+0000 7f43a21cf640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c692933ed0 0x55c69293f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43538/0 (socket says 172.21.15.114:43538) 2024-09-13T18:22:22.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.173+0000 7f43a21cf640 1 -- 172.21.15.114:0/462183645 learned_addr learned my addr 172.21.15.114:0/462183645 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:22.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.173+0000 7f43a19ce640 1 --2- 172.21.15.114:0/462183645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c69292e500 0x55c692937cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.173+0000 7f43a11cd640 1 --2- 172.21.15.114:0/462183645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c69292ee40 0x55c692938210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.173+0000 7f43a21cf640 1 -- 172.21.15.114:0/462183645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c69292e500 msgr2=0x55c692937cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:22.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.173+0000 7f43a21cf640 1 --2- 172.21.15.114:0/462183645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c69292e500 0x55c692937cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.173+0000 7f43a21cf640 1 -- 172.21.15.114:0/462183645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c69292ee40 msgr2=0x55c692938210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:22.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.173+0000 7f43a21cf640 1 --2- 172.21.15.114:0/462183645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c69292ee40 0x55c692938210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.173+0000 7f43a21cf640 1 -- 172.21.15.114:0/462183645 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f439c1a0340 con 0x55c692933ed0 2024-09-13T18:22:22.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.173+0000 7f43a21cf640 1 --2- 172.21.15.114:0/462183645 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c692933ed0 0x55c69293f250 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f43940126a0 tx=0x7f4394004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e5429b4d5ae1b45 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:22.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.173+0000 7f43857fa640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:22:22.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.174+0000 7f43857fa640 1 -- 172.21.15.114:0/462183645 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4394010890 con 0x55c692933ed0 2024-09-13T18:22:22.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.174+0000 7f43a6a30740 1 -- 172.21.15.114:0/462183645 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c6925853c0 con 0x55c692933ed0 2024-09-13T18:22:22.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.174+0000 7f43857fa640 1 -- 172.21.15.114:0/462183645 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f43940109f0 con 0x55c692933ed0 2024-09-13T18:22:22.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.174+0000 7f43857fa640 1 -- 172.21.15.114:0/462183645 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4394010bb0 con 0x55c692933ed0 2024-09-13T18:22:22.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.174+0000 7f43a6a30740 10 client.15489 fetch_fsmap learned FSMap version 29 2024-09-13T18:22:22.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.174+0000 7f43a6a30740 1 -- 172.21.15.114:0/462183645 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c692890270 con 0x55c692933ed0 2024-09-13T18:22:22.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.174+0000 7f43857fa640 1 -- 172.21.15.114:0/462183645 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c6925853c0 con 0x55c692933ed0 2024-09-13T18:22:22.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.174+0000 7f43857fa640 1 -- 172.21.15.114:0/462183645 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f43940113e0 con 0x55c692933ed0 2024-09-13T18:22:22.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.174+0000 7f43a6a30740 10 client.15489 fetch_fsmap finished waiting for FSMap version 29 2024-09-13T18:22:22.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.174+0000 7f43a6a30740 10 client.15489 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-13T18:22:22.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.174+0000 7f43a6a30740 20 client.15489 populate_metadata read hostname 'smithi114' 2024-09-13T18:22:22.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.174+0000 7f43a6a30740 1 --2- 172.21.15.114:0/462183645 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55c692943780 0x55c692948440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.174+0000 7f43a6a30740 4 client.15489 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:22:22.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.174+0000 7f43a6a30740 1 -- 172.21.15.114:0/462183645 --> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c6925966c0 con 0x55c692943780 2024-09-13T18:22:22.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.174+0000 7f43a19ce640 1 --2- 172.21.15.114:0/462183645 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55c692943780 0x55c692948440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.175+0000 7f43857fa640 10 client.15489 ms_handle_connect on v2:172.21.15.204:6838/344883912 2024-09-13T18:22:22.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.175+0000 7f43a19ce640 1 --2- 172.21.15.114:0/462183645 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55c692943780 0x55c692948440 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=49523378c21adc23 server_cookie=79ed2dd29d03fd4e in_seq=0 out_seq=0 2024-09-13T18:22:22.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.175+0000 7f43857fa640 1 -- 172.21.15.114:0/462183645 <== mds.2 v2:172.21.15.204:6838/344883912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55c6925966c0 con 0x55c692943780 2024-09-13T18:22:22.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.175+0000 7f43857fa640 10 client.15489 handle_command_reply: tid=0 2024-09-13T18:22:22.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.175+0000 7f43a6a30740 1 client.15489 shutdown 2024-09-13T18:22:22.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.175+0000 7f43a6a30740 1 -- 172.21.15.114:0/462183645 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c692933ed0 msgr2=0x55c69293f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:22.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.175+0000 7f43a6a30740 1 --2- 172.21.15.114:0/462183645 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c692933ed0 0x55c69293f250 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f43940126a0 tx=0x7f4394004ab0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.176+0000 7f43a6a30740 1 -- 172.21.15.114:0/462183645 shutdown_connections 2024-09-13T18:22:22.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.176+0000 7f43a6a30740 1 --2- 172.21.15.114:0/462183645 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55c692943780 0x55c692948440 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.176+0000 7f43a6a30740 1 --2- 172.21.15.114:0/462183645 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c692933ed0 0x55c69293f250 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.176+0000 7f43a6a30740 1 --2- 172.21.15.114:0/462183645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c69292ee40 0x55c692938210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.176+0000 7f43a6a30740 1 --2- 172.21.15.114:0/462183645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c69292e500 0x55c692937cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.176+0000 7f43857fa640 0 client.15489 ms_handle_reset on v2:172.21.15.204:6838/344883912 2024-09-13T18:22:22.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.176+0000 7f43a6a30740 1 -- 172.21.15.114:0/462183645 >> 172.21.15.114:0/462183645 conn(0x55c692929b70 msgr2=0x55c69292b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:22.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.176+0000 7f43a6a30740 1 -- 172.21.15.114:0/462183645 shutdown_connections 2024-09-13T18:22:22.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.176+0000 7f43a6a30740 1 -- 172.21.15.114:0/462183645 wait complete. 2024-09-13T18:22:22.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.176+0000 7f43a6a30740 20 client.15489 trim_cache size 0 max 16384 2024-09-13T18:22:22.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.190+0000 7f43a6a30740 1 Processor -- start 2024-09-13T18:22:22.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.190+0000 7f43a6a30740 1 -- start start 2024-09-13T18:22:22.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.190+0000 7f43a6a30740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c69292e500 0x55c692937a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.190+0000 7f43a6a30740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c69292ee40 0x55c69293f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.190+0000 7f43a6a30740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c692933ed0 0x55c692937fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.190+0000 7f43a6a30740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c692815460 con 0x55c69292ee40 2024-09-13T18:22:22.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.190+0000 7f43a6a30740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c69293fef0 con 0x55c69292e500 2024-09-13T18:22:22.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.190+0000 7f43a6a30740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c692924c80 con 0x55c692933ed0 2024-09-13T18:22:22.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.190+0000 7f43a11cd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c69292ee40 0x55c69293f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.190+0000 7f43a11cd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c69292ee40 0x55c69293f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43552/0 (socket says 172.21.15.114:43552) 2024-09-13T18:22:22.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.190+0000 7f43a11cd640 1 -- 172.21.15.114:0/510874627 learned_addr learned my addr 172.21.15.114:0/510874627 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:22.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.191+0000 7f43a11cd640 1 -- 172.21.15.114:0/510874627 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c692933ed0 msgr2=0x55c692937fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:22.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.191+0000 7f43a11cd640 1 --2- 172.21.15.114:0/510874627 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c692933ed0 0x55c692937fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.191+0000 7f43a11cd640 1 -- 172.21.15.114:0/510874627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c69292e500 msgr2=0x55c692937a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:22.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.191+0000 7f43a11cd640 1 --2- 172.21.15.114:0/510874627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c69292e500 0x55c692937a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.191+0000 7f43a11cd640 1 -- 172.21.15.114:0/510874627 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6928325f0 con 0x55c69292ee40 2024-09-13T18:22:22.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.191+0000 7f43a11cd640 1 --2- 172.21.15.114:0/510874627 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c69292ee40 0x55c69293f170 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f43900baa20 tx=0x7f43900f1140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aa65e4e965a6f4c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:22.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.191+0000 7f438ce05640 1 -- 172.21.15.114:0/510874627 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4390003ed0 con 0x55c69292ee40 2024-09-13T18:22:22.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.191+0000 7f438ce05640 1 -- 172.21.15.114:0/510874627 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f43900bad20 con 0x55c69292ee40 2024-09-13T18:22:22.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.191+0000 7f438ce05640 1 -- 172.21.15.114:0/510874627 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43900c30f0 con 0x55c69292ee40 2024-09-13T18:22:22.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.191+0000 7f43a6a30740 1 -- 172.21.15.114:0/510874627 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c69292ee40 msgr2=0x55c69293f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:22.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.191+0000 7f43a6a30740 1 --2- 172.21.15.114:0/510874627 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c69292ee40 0x55c69293f170 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f43900baa20 tx=0x7f43900f1140 comp rx=0 tx=0).stop 2024-09-13T18:22:22.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.191+0000 7f43a6a30740 1 -- 172.21.15.114:0/510874627 shutdown_connections 2024-09-13T18:22:22.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.191+0000 7f43a6a30740 1 --2- 172.21.15.114:0/510874627 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c692933ed0 0x55c692937fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.191+0000 7f43a6a30740 1 --2- 172.21.15.114:0/510874627 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c69292ee40 0x55c69293f170 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.191+0000 7f43a6a30740 1 --2- 172.21.15.114:0/510874627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c69292e500 0x55c692937a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.191+0000 7f43a6a30740 1 -- 172.21.15.114:0/510874627 >> 172.21.15.114:0/510874627 conn(0x55c692929b70 msgr2=0x55c692936dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:22.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.192+0000 7f43a6a30740 1 -- 172.21.15.114:0/510874627 shutdown_connections 2024-09-13T18:22:22.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.192+0000 7f43a6a30740 1 -- 172.21.15.114:0/510874627 wait complete. 2024-09-13T18:22:22.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.192+0000 7f43a6a30740 1 -- start start 2024-09-13T18:22:22.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.192+0000 7f43a6a30740 1 Processor -- start 2024-09-13T18:22:22.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.192+0000 7f43a6a30740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c69292e500 0x55c69293ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.192+0000 7f43a6a30740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c69292ee40 0x55c69293f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.192+0000 7f43a6a30740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c692933ed0 0x55c69293b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.192+0000 7f43a6a30740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c69293bfb0 con 0x55c69292e500 2024-09-13T18:22:22.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.192+0000 7f43a6a30740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c69293c120 con 0x55c69292ee40 2024-09-13T18:22:22.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.192+0000 7f43a6a30740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c69293c2e0 con 0x55c692933ed0 2024-09-13T18:22:22.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.192+0000 7f43a6a30740 10 client.0 _ll_register_callbacks cb 0x7f43900f1500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:22:22.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.192+0000 7f43a19ce640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c69292e500 0x55c69293ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.192+0000 7f43a19ce640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c69292e500 0x55c69293ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43554/0 (socket says 172.21.15.114:43554) 2024-09-13T18:22:22.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.192+0000 7f43a19ce640 1 -- 172.21.15.114:0/3490150394 learned_addr learned my addr 172.21.15.114:0/3490150394 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:22.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.192+0000 7f43a21cf640 1 --2- 172.21.15.114:0/3490150394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c692933ed0 0x55c69293b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.192+0000 7f43a19ce640 1 -- 172.21.15.114:0/3490150394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c692933ed0 msgr2=0x55c69293b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:22.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.192+0000 7f43a11cd640 1 --2- 172.21.15.114:0/3490150394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c69292ee40 0x55c69293f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.192+0000 7f43a19ce640 1 --2- 172.21.15.114:0/3490150394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c692933ed0 0x55c69293b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.193+0000 7f43a19ce640 1 -- 172.21.15.114:0/3490150394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c69292ee40 msgr2=0x55c69293f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:22.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.193+0000 7f43a19ce640 1 --2- 172.21.15.114:0/3490150394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c69292ee40 0x55c69293f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.193+0000 7f43a19ce640 1 -- 172.21.15.114:0/3490150394 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c69281e4b0 con 0x55c69292e500 2024-09-13T18:22:22.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.193+0000 7f43a21cf640 1 --2- 172.21.15.114:0/3490150394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c692933ed0 0x55c69293b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:22:22.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.193+0000 7f43a11cd640 1 --2- 172.21.15.114:0/3490150394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c69292ee40 0x55c69293f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:22:22.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.193+0000 7f43a19ce640 1 --2- 172.21.15.114:0/3490150394 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c69292e500 0x55c69293ecb0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x55c69292a9f0 tx=0x7f4398010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32fb712267941311 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:22.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.193+0000 7f43857fa640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:22:22.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.193+0000 7f43857fa640 1 -- 172.21.15.114:0/3490150394 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4398017030 con 0x55c69292e500 2024-09-13T18:22:22.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.193+0000 7f43a6a30740 1 -- 172.21.15.114:0/3490150394 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c692930620 con 0x55c69292e500 2024-09-13T18:22:22.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.193+0000 7f43857fa640 1 -- 172.21.15.114:0/3490150394 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f4398004460 con 0x55c69292e500 2024-09-13T18:22:22.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.193+0000 7f43857fa640 1 -- 172.21.15.114:0/3490150394 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43980035f0 con 0x55c69292e500 2024-09-13T18:22:22.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.193+0000 7f43a6a30740 10 client.15495 fetch_fsmap learned FSMap version 29 2024-09-13T18:22:22.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.193+0000 7f43a6a30740 1 -- 172.21.15.114:0/3490150394 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c692857d00 con 0x55c69292e500 2024-09-13T18:22:22.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.193+0000 7f43857fa640 1 -- 172.21.15.114:0/3490150394 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c692930620 con 0x55c69292e500 2024-09-13T18:22:22.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.193+0000 7f43857fa640 1 -- 172.21.15.114:0/3490150394 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f439801f4f0 con 0x55c69292e500 2024-09-13T18:22:22.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.193+0000 7f43a6a30740 10 client.15495 fetch_fsmap finished waiting for FSMap version 29 2024-09-13T18:22:22.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.193+0000 7f43a6a30740 10 client.15495 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-13T18:22:22.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.193+0000 7f43a6a30740 20 client.15495 populate_metadata read hostname 'smithi114' 2024-09-13T18:22:22.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.193+0000 7f43a6a30740 1 --2- 172.21.15.114:0/3490150394 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55c69299ed60 0x55c69299b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.193+0000 7f43a6a30740 4 client.15495 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T18:22:22.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.193+0000 7f43a6a30740 1 -- 172.21.15.114:0/3490150394 --> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f43900f2040 con 0x55c69299ed60 2024-09-13T18:22:22.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.194+0000 7f43a11cd640 1 --2- 172.21.15.114:0/3490150394 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55c69299ed60 0x55c69299b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.194+0000 7f43a11cd640 1 --2- 172.21.15.114:0/3490150394 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55c69299ed60 0x55c69299b700 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=15e4a19952c89afb server_cookie=b307c0d7b7ef0b45 in_seq=0 out_seq=0 2024-09-13T18:22:22.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.194+0000 7f43857fa640 10 client.15495 ms_handle_connect on v2:172.21.15.204:6838/344883912 2024-09-13T18:22:22.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.194+0000 7f43857fa640 1 -- 172.21.15.114:0/3490150394 <== mds.2 v2:172.21.15.204:6838/344883912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x7f43900f2040 con 0x55c69299ed60 2024-09-13T18:22:22.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.194+0000 7f43857fa640 10 client.15495 handle_command_reply: tid=0 2024-09-13T18:22:22.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.194+0000 7f43a6a30740 1 client.15495 shutdown 2024-09-13T18:22:22.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.194+0000 7f43a6a30740 1 -- 172.21.15.114:0/3490150394 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c69292e500 msgr2=0x55c69293ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:22.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.194+0000 7f43a6a30740 1 --2- 172.21.15.114:0/3490150394 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c69292e500 0x55c69293ecb0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x55c69292a9f0 tx=0x7f4398010800 comp rx=0 tx=0).stop 2024-09-13T18:22:22.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.195+0000 7f43a6a30740 1 -- 172.21.15.114:0/3490150394 shutdown_connections 2024-09-13T18:22:22.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.195+0000 7f43a6a30740 1 --2- 172.21.15.114:0/3490150394 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55c69299ed60 0x55c69299b700 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.195+0000 7f43a6a30740 1 --2- 172.21.15.114:0/3490150394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c692933ed0 0x55c69293b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.195+0000 7f43857fa640 0 client.15495 ms_handle_reset on v2:172.21.15.204:6838/344883912 2024-09-13T18:22:22.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.195+0000 7f43a6a30740 1 --2- 172.21.15.114:0/3490150394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c69292ee40 0x55c69293f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.195+0000 7f43a6a30740 1 --2- 172.21.15.114:0/3490150394 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c69292e500 0x55c69293ecb0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.195+0000 7f43a6a30740 1 -- 172.21.15.114:0/3490150394 >> 172.21.15.114:0/3490150394 conn(0x55c692929b70 msgr2=0x55c692937a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:22.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.195+0000 7f43a6a30740 1 -- 172.21.15.114:0/3490150394 shutdown_connections 2024-09-13T18:22:22.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.195+0000 7f43a6a30740 1 -- 172.21.15.114:0/3490150394 wait complete. 2024-09-13T18:22:22.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.195+0000 7f43a6a30740 20 client.15495 trim_cache size 0 max 16384 2024-09-13T18:22:22.205 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:22:22.205 INFO:teuthology.orchestra.run.smithi114.stdout: "cluster_fsid": "97e7edf2-71fb-11ef-bceb-c7b262605968", 2024-09-13T18:22:22.205 INFO:teuthology.orchestra.run.smithi114.stdout: "whoami": 2, 2024-09-13T18:22:22.205 INFO:teuthology.orchestra.run.smithi114.stdout: "id": 24475, 2024-09-13T18:22:22.205 INFO:teuthology.orchestra.run.smithi114.stdout: "want_state": "up:active", 2024-09-13T18:22:22.205 INFO:teuthology.orchestra.run.smithi114.stdout: "state": "up:active", 2024-09-13T18:22:22.205 INFO:teuthology.orchestra.run.smithi114.stdout: "fs_name": "cephfs", 2024-09-13T18:22:22.205 INFO:teuthology.orchestra.run.smithi114.stdout: "rank_uptime": 116.007742315, 2024-09-13T18:22:22.205 INFO:teuthology.orchestra.run.smithi114.stdout: "mdsmap_epoch": 29, 2024-09-13T18:22:22.206 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch": 93, 2024-09-13T18:22:22.206 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch_barrier": 91, 2024-09-13T18:22:22.206 INFO:teuthology.orchestra.run.smithi114.stdout: "uptime": 153.99812837499999 2024-09-13T18:22:22.206 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:22:22.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.195+0000 7f43a3c32640 1 -- 172.21.15.114:0/1414727786 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f43880a1090 msgr2=0x7f43880a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:22.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.195+0000 7f43a3c32640 1 --2- 172.21.15.114:0/1414727786 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f43880a1090 0x7f43880a3550 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4394005fd0 tx=0x7f4394004a40 comp rx=0 tx=0).stop 2024-09-13T18:22:22.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.195+0000 7f43a3c32640 1 -- 172.21.15.114:0/1414727786 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f439c100f60 msgr2=0x7f439c0728a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:22.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.195+0000 7f43a3c32640 1 --2- 172.21.15.114:0/1414727786 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f439c100f60 0x7f439c0728a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4390005ab0 tx=0x7f43900042f0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.195+0000 7f43a3c32640 1 -- 172.21.15.114:0/1414727786 shutdown_connections 2024-09-13T18:22:22.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.195+0000 7f43a3c32640 1 --2- 172.21.15.114:0/1414727786 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f43880a1090 0x7f43880a3550 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.195+0000 7f43a3c32640 1 --2- 172.21.15.114:0/1414727786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f439c101920 0x7f439c072de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.195+0000 7f43a3c32640 1 --2- 172.21.15.114:0/1414727786 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f439c100f60 0x7f439c0728a0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.195+0000 7f43a3c32640 1 --2- 172.21.15.114:0/1414727786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f439c0ffd70 0x7f439c074260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.195+0000 7f43a3c32640 1 -- 172.21.15.114:0/1414727786 >> 172.21.15.114:0/1414727786 conn(0x7f439c0fb4d0 msgr2=0x7f439c104470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:22.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.195+0000 7f43a3c32640 1 -- 172.21.15.114:0/1414727786 shutdown_connections 2024-09-13T18:22:22.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.195+0000 7f43a3c32640 1 -- 172.21.15.114:0/1414727786 wait complete. 2024-09-13T18:22:22.208 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-09-13T18:22:22.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.305+0000 7fec9de12640 1 -- 172.21.15.114:0/3092635548 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec98105af0 msgr2=0x7fec98107ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:22.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.305+0000 7fec9de12640 1 --2- 172.21.15.114:0/3092635548 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec98105af0 0x7fec98107ee0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fec90009ca0 tx=0x7fec9002a8c0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.305+0000 7fec9de12640 1 -- 172.21.15.114:0/3092635548 shutdown_connections 2024-09-13T18:22:22.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.305+0000 7fec9de12640 1 --2- 172.21.15.114:0/3092635548 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec98105af0 0x7fec98107ee0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.305+0000 7fec9de12640 1 --2- 172.21.15.114:0/3092635548 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec981031c0 0x7fec981055b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.305+0000 7fec9de12640 1 --2- 172.21.15.114:0/3092635548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec98100890 0x7fec98102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.305+0000 7fec9de12640 1 -- 172.21.15.114:0/3092635548 >> 172.21.15.114:0/3092635548 conn(0x7fec980fa410 msgr2=0x7fec980fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:22.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.306+0000 7fec9de12640 1 -- 172.21.15.114:0/3092635548 shutdown_connections 2024-09-13T18:22:22.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.306+0000 7fec9de12640 1 -- 172.21.15.114:0/3092635548 wait complete. 2024-09-13T18:22:22.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.306+0000 7fec9de12640 1 Processor -- start 2024-09-13T18:22:22.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.306+0000 7fec9de12640 1 -- start start 2024-09-13T18:22:22.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.306+0000 7fec9de12640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec98100890 0x7fec98100200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.306+0000 7fec9de12640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec981031c0 0x7fec980fe840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.306+0000 7fec9de12640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec98105af0 0x7fec980fed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.306+0000 7fec9de12640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec980ff4e0 con 0x7fec98105af0 2024-09-13T18:22:22.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.306+0000 7fec9de12640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec980ff650 con 0x7fec98100890 2024-09-13T18:22:22.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.306+0000 7fec9de12640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9819fc80 con 0x7fec981031c0 2024-09-13T18:22:22.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.307+0000 7fec97fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec98105af0 0x7fec980fed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.307+0000 7fec97fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec98105af0 0x7fec980fed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43576/0 (socket says 172.21.15.114:43576) 2024-09-13T18:22:22.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.307+0000 7fec97fff640 1 -- 172.21.15.114:0/787470678 learned_addr learned my addr 172.21.15.114:0/787470678 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:22.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.307+0000 7fec977fe640 1 --2- 172.21.15.114:0/787470678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec98100890 0x7fec98100200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.307+0000 7fec97fff640 1 -- 172.21.15.114:0/787470678 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec981031c0 msgr2=0x7fec980fe840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:22.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.307+0000 7fec96ffd640 1 --2- 172.21.15.114:0/787470678 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec981031c0 0x7fec980fe840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.307+0000 7fec97fff640 1 --2- 172.21.15.114:0/787470678 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec981031c0 0x7fec980fe840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.307+0000 7fec97fff640 1 -- 172.21.15.114:0/787470678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec98100890 msgr2=0x7fec98100200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:22.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.307+0000 7fec97fff640 1 --2- 172.21.15.114:0/787470678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec98100890 0x7fec98100200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.307+0000 7fec97fff640 1 -- 172.21.15.114:0/787470678 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec90009900 con 0x7fec98105af0 2024-09-13T18:22:22.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.307+0000 7fec977fe640 1 --2- 172.21.15.114:0/787470678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec98100890 0x7fec98100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:22:22.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.307+0000 7fec96ffd640 1 --2- 172.21.15.114:0/787470678 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec981031c0 0x7fec980fe840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:22:22.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.307+0000 7fec97fff640 1 --2- 172.21.15.114:0/787470678 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec98105af0 0x7fec980fed80 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fec9002ade0 tx=0x7fec90003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:22.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.307+0000 7fec94ff9640 1 -- 172.21.15.114:0/787470678 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec900307c0 con 0x7fec98105af0 2024-09-13T18:22:22.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.307+0000 7fec94ff9640 1 -- 172.21.15.114:0/787470678 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec90040070 con 0x7fec98105af0 2024-09-13T18:22:22.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.307+0000 7fec9de12640 1 -- 172.21.15.114:0/787470678 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec9819fed0 con 0x7fec98105af0 2024-09-13T18:22:22.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.307+0000 7fec94ff9640 1 -- 172.21.15.114:0/787470678 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec900044e0 con 0x7fec98105af0 2024-09-13T18:22:22.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.307+0000 7fec9de12640 1 -- 172.21.15.114:0/787470678 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec981a03a0 con 0x7fec98105af0 2024-09-13T18:22:22.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.309+0000 7fec94ff9640 1 -- 172.21.15.114:0/787470678 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fec90042030 con 0x7fec98105af0 2024-09-13T18:22:22.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.310+0000 7feca0c13740 1 Processor -- start 2024-09-13T18:22:22.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.310+0000 7feca0c13740 1 -- start start 2024-09-13T18:22:22.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.310+0000 7feca0c13740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55741d9e4500 0x55741d9e4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.310+0000 7fec94ff9640 1 --2- 172.21.15.114:0/787470678 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fec800a1110 0x7fec800a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.310+0000 7feca0c13740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55741d9e4e40 0x55741d9e9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.310+0000 7feca0c13740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55741d9e9ed0 0x55741d9ec390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.310+0000 7feca0c13740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55741d93f7c0 con 0x55741d9e4500 2024-09-13T18:22:22.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.310+0000 7feca0c13740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55741d94eb70 con 0x55741d9e4e40 2024-09-13T18:22:22.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.310+0000 7feca0c13740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55741d94c300 con 0x55741d9e9ed0 2024-09-13T18:22:22.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.310+0000 7fec96ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55741d9e4500 0x55741d9e4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.310+0000 7fec977fe640 1 --2- 172.21.15.114:0/787470678 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fec800a1110 0x7fec800a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.310+0000 7fec96ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55741d9e4500 0x55741d9e4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43588/0 (socket says 172.21.15.114:43588) 2024-09-13T18:22:22.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.310+0000 7fec96ffd640 1 -- 172.21.15.114:0/3206063179 learned_addr learned my addr 172.21.15.114:0/3206063179 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:22.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.310+0000 7fec94ff9640 1 -- 172.21.15.114:0/787470678 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fec900ee420 con 0x7fec98105af0 2024-09-13T18:22:22.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.310+0000 7fec96ffd640 1 --2- 172.21.15.114:0/3206063179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55741d9e4e40 0x55741d9e9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.310+0000 7fec97fff640 1 --2- 172.21.15.114:0/3206063179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55741d9e9ed0 0x55741d9ec390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.310+0000 7fec96ffd640 1 -- 172.21.15.114:0/3206063179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55741d9e9ed0 msgr2=0x55741d9ec390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:22.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.310+0000 7fec96ffd640 1 --2- 172.21.15.114:0/3206063179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55741d9e9ed0 0x55741d9ec390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.310+0000 7fec96ffd640 1 -- 172.21.15.114:0/3206063179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55741d9e4e40 msgr2=0x55741d9e9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:22.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.311+0000 7fec96ffd640 1 --2- 172.21.15.114:0/3206063179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55741d9e4e40 0x55741d9e9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.311+0000 7fec96ffd640 1 -- 172.21.15.114:0/3206063179 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55741d8d44b0 con 0x55741d9e4500 2024-09-13T18:22:22.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.311+0000 7fec96ffd640 1 --2- 172.21.15.114:0/3206063179 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55741d9e4500 0x55741d9e4900 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fec8c009fd0 tx=0x7fec8c0095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ee3f3e933b56fbf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:22.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.311+0000 7fec977fe640 1 --2- 172.21.15.114:0/787470678 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fec800a1110 0x7fec800a35d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fec8800abe0 tx=0x7fec8800a5c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:22.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.311+0000 7fec7dffb640 1 -- 172.21.15.114:0/3206063179 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8c0134c0 con 0x55741d9e4500 2024-09-13T18:22:22.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.311+0000 7fec7dffb640 1 -- 172.21.15.114:0/3206063179 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec8c013ab0 con 0x55741d9e4500 2024-09-13T18:22:22.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.311+0000 7fec7dffb640 1 -- 172.21.15.114:0/3206063179 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8c012850 con 0x55741d9e4500 2024-09-13T18:22:22.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.311+0000 7feca0c13740 1 -- 172.21.15.114:0/3206063179 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55741d9e4500 msgr2=0x55741d9e4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:22.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.311+0000 7feca0c13740 1 --2- 172.21.15.114:0/3206063179 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55741d9e4500 0x55741d9e4900 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fec8c009fd0 tx=0x7fec8c0095f0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.311+0000 7feca0c13740 1 -- 172.21.15.114:0/3206063179 shutdown_connections 2024-09-13T18:22:22.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.311+0000 7feca0c13740 1 --2- 172.21.15.114:0/3206063179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55741d9e9ed0 0x55741d9ec390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.311+0000 7feca0c13740 1 --2- 172.21.15.114:0/3206063179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55741d9e4e40 0x55741d9e9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.311+0000 7feca0c13740 1 --2- 172.21.15.114:0/3206063179 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55741d9e4500 0x55741d9e4900 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.311+0000 7feca0c13740 1 -- 172.21.15.114:0/3206063179 >> 172.21.15.114:0/3206063179 conn(0x55741d9dfb70 msgr2=0x55741d9dff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:22.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.312+0000 7feca0c13740 1 -- 172.21.15.114:0/3206063179 shutdown_connections 2024-09-13T18:22:22.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.312+0000 7feca0c13740 1 -- 172.21.15.114:0/3206063179 wait complete. 2024-09-13T18:22:22.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.312+0000 7feca0c13740 1 -- start start 2024-09-13T18:22:22.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.312+0000 7feca0c13740 1 Processor -- start 2024-09-13T18:22:22.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.312+0000 7feca0c13740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55741d9e4500 0x55741d9ee0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.312+0000 7feca0c13740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55741d9e4e40 0x55741d9ee620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.312+0000 7feca0c13740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55741d9e9ed0 0x55741d9f5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.312+0000 7feca0c13740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55741d947bf0 con 0x55741d9e4e40 2024-09-13T18:22:22.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.312+0000 7feca0c13740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55741d906250 con 0x55741d9e4500 2024-09-13T18:22:22.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.312+0000 7feca0c13740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55741d905ef0 con 0x55741d9e9ed0 2024-09-13T18:22:22.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.312+0000 7feca0c13740 10 client.0 _ll_register_callbacks cb 0x55741d938740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:22:22.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.312+0000 7fec96ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55741d9e4e40 0x55741d9ee620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.312+0000 7fec96ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55741d9e4e40 0x55741d9ee620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43604/0 (socket says 172.21.15.114:43604) 2024-09-13T18:22:22.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.312+0000 7fec96ffd640 1 -- 172.21.15.114:0/2794058568 learned_addr learned my addr 172.21.15.114:0/2794058568 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:22.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.313+0000 7fec97fff640 1 --2- 172.21.15.114:0/2794058568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55741d9e9ed0 0x55741d9f5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.313+0000 7fec96ffd640 1 -- 172.21.15.114:0/2794058568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55741d9e9ed0 msgr2=0x55741d9f5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:22.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.313+0000 7fec977fe640 1 --2- 172.21.15.114:0/2794058568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55741d9e4500 0x55741d9ee0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.313+0000 7fec96ffd640 1 --2- 172.21.15.114:0/2794058568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55741d9e9ed0 0x55741d9f5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.313+0000 7fec96ffd640 1 -- 172.21.15.114:0/2794058568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55741d9e4500 msgr2=0x55741d9ee0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:22.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.313+0000 7fec96ffd640 1 --2- 172.21.15.114:0/2794058568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55741d9e4500 0x55741d9ee0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.313+0000 7fec96ffd640 1 -- 172.21.15.114:0/2794058568 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec8c009d20 con 0x55741d9e4e40 2024-09-13T18:22:22.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.313+0000 7fec977fe640 1 --2- 172.21.15.114:0/2794058568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55741d9e4500 0x55741d9ee0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:22:22.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.313+0000 7fec96ffd640 1 --2- 172.21.15.114:0/2794058568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55741d9e4e40 0x55741d9ee620 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fec8c012500 tx=0x7fec8c0134c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35e64b876d38e641 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:22.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.313+0000 7fec76ffd640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:22:22.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.313+0000 7fec76ffd640 1 -- 172.21.15.114:0/2794058568 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8c012810 con 0x55741d9e4e40 2024-09-13T18:22:22.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.313+0000 7fec76ffd640 1 -- 172.21.15.114:0/2794058568 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fec8c023070 con 0x55741d9e4e40 2024-09-13T18:22:22.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.313+0000 7feca0c13740 1 -- 172.21.15.114:0/2794058568 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55741d63b3c0 con 0x55741d9e4e40 2024-09-13T18:22:22.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.313+0000 7fec76ffd640 1 -- 172.21.15.114:0/2794058568 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec8c0138c0 con 0x55741d9e4e40 2024-09-13T18:22:22.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.313+0000 7feca0c13740 10 client.15507 fetch_fsmap learned FSMap version 29 2024-09-13T18:22:22.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.313+0000 7feca0c13740 1 -- 172.21.15.114:0/2794058568 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55741d946270 con 0x55741d9e4e40 2024-09-13T18:22:22.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.313+0000 7fec76ffd640 1 -- 172.21.15.114:0/2794058568 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55741d63b3c0 con 0x55741d9e4e40 2024-09-13T18:22:22.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.313+0000 7fec76ffd640 1 -- 172.21.15.114:0/2794058568 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fec8c027ad0 con 0x55741d9e4e40 2024-09-13T18:22:22.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.313+0000 7feca0c13740 10 client.15507 fetch_fsmap finished waiting for FSMap version 29 2024-09-13T18:22:22.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.314+0000 7feca0c13740 10 client.15507 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-13T18:22:22.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.314+0000 7feca0c13740 20 client.15507 populate_metadata read hostname 'smithi114' 2024-09-13T18:22:22.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.314+0000 7feca0c13740 1 --2- 172.21.15.114:0/2794058568 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55741d9fa510 0x55741d9fe690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.314+0000 7feca0c13740 4 client.15507 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:22:22.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.314+0000 7feca0c13740 1 -- 172.21.15.114:0/2794058568 --> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55741d64c6c0 con 0x55741d9fa510 2024-09-13T18:22:22.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.314+0000 7fec977fe640 1 --2- 172.21.15.114:0/2794058568 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55741d9fa510 0x55741d9fe690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.314+0000 7fec76ffd640 10 client.15507 ms_handle_connect on v2:172.21.15.134:6836/891692210 2024-09-13T18:22:22.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.314+0000 7fec977fe640 1 --2- 172.21.15.114:0/2794058568 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55741d9fa510 0x55741d9fe690 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=70794bac14ca9f55 server_cookie=bcf4f3def79c6a79 in_seq=0 out_seq=0 2024-09-13T18:22:22.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.315+0000 7fec76ffd640 1 -- 172.21.15.114:0/2794058568 <== mds.1 v2:172.21.15.134:6836/891692210 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55741d64c6c0 con 0x55741d9fa510 2024-09-13T18:22:22.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.315+0000 7fec76ffd640 10 client.15507 handle_command_reply: tid=0 2024-09-13T18:22:22.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.315+0000 7feca0c13740 1 client.15507 shutdown 2024-09-13T18:22:22.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.315+0000 7feca0c13740 1 -- 172.21.15.114:0/2794058568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55741d9e4e40 msgr2=0x55741d9ee620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:22.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.315+0000 7feca0c13740 1 --2- 172.21.15.114:0/2794058568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55741d9e4e40 0x55741d9ee620 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fec8c012500 tx=0x7fec8c0134c0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.315+0000 7feca0c13740 1 -- 172.21.15.114:0/2794058568 shutdown_connections 2024-09-13T18:22:22.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.315+0000 7feca0c13740 1 --2- 172.21.15.114:0/2794058568 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55741d9fa510 0x55741d9fe690 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.315+0000 7feca0c13740 1 --2- 172.21.15.114:0/2794058568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55741d9e9ed0 0x55741d9f5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.315+0000 7feca0c13740 1 --2- 172.21.15.114:0/2794058568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55741d9e4e40 0x55741d9ee620 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.315+0000 7feca0c13740 1 --2- 172.21.15.114:0/2794058568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55741d9e4500 0x55741d9ee0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.315+0000 7fec76ffd640 0 client.15507 ms_handle_reset on v2:172.21.15.134:6836/891692210 2024-09-13T18:22:22.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.315+0000 7feca0c13740 1 -- 172.21.15.114:0/2794058568 >> 172.21.15.114:0/2794058568 conn(0x55741d9dfb70 msgr2=0x55741d9e1380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:22.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.315+0000 7feca0c13740 1 -- 172.21.15.114:0/2794058568 shutdown_connections 2024-09-13T18:22:22.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.315+0000 7feca0c13740 1 -- 172.21.15.114:0/2794058568 wait complete. 2024-09-13T18:22:22.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.315+0000 7feca0c13740 20 client.15507 trim_cache size 0 max 16384 2024-09-13T18:22:22.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.330+0000 7feca0c13740 1 Processor -- start 2024-09-13T18:22:22.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.330+0000 7feca0c13740 1 -- start start 2024-09-13T18:22:22.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.330+0000 7feca0c13740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55741d9dfb70 0x55741d9ede90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.330+0000 7feca0c13740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55741d9e4500 0x55741d9f5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.330+0000 7feca0c13740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55741d9e4e40 0x55741d9ee3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.330+0000 7feca0c13740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55741d8cb460 con 0x55741d9e4500 2024-09-13T18:22:22.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.330+0000 7feca0c13740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55741d64c6c0 con 0x55741d9dfb70 2024-09-13T18:22:22.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.330+0000 7feca0c13740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55741d63b3c0 con 0x55741d9e4e40 2024-09-13T18:22:22.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.330+0000 7fec96ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55741d9e4500 0x55741d9f5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.330+0000 7fec96ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55741d9e4500 0x55741d9f5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43608/0 (socket says 172.21.15.114:43608) 2024-09-13T18:22:22.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.330+0000 7fec96ffd640 1 -- 172.21.15.114:0/512426513 learned_addr learned my addr 172.21.15.114:0/512426513 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:22.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.330+0000 7fec96ffd640 1 -- 172.21.15.114:0/512426513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55741d9e4e40 msgr2=0x55741d9ee3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:22.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.330+0000 7fec977fe640 1 --2- 172.21.15.114:0/512426513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55741d9dfb70 0x55741d9ede90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.330+0000 7fec96ffd640 1 --2- 172.21.15.114:0/512426513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55741d9e4e40 0x55741d9ee3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.330+0000 7fec96ffd640 1 -- 172.21.15.114:0/512426513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55741d9dfb70 msgr2=0x55741d9ede90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:22.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.330+0000 7fec96ffd640 1 --2- 172.21.15.114:0/512426513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55741d9dfb70 0x55741d9ede90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.330+0000 7fec96ffd640 1 -- 172.21.15.114:0/512426513 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55741d8e85f0 con 0x55741d9e4500 2024-09-13T18:22:22.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.330+0000 7fec96ffd640 1 --2- 172.21.15.114:0/512426513 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55741d9e4500 0x55741d9f5620 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fec8c027eb0 tx=0x7fec8c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=773d831271dc1d67 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:22.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.330+0000 7fec7e7fc640 1 -- 172.21.15.114:0/512426513 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8c013530 con 0x55741d9e4500 2024-09-13T18:22:22.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.330+0000 7fec7e7fc640 1 -- 172.21.15.114:0/512426513 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec8c0054c0 con 0x55741d9e4500 2024-09-13T18:22:22.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.330+0000 7fec7e7fc640 1 -- 172.21.15.114:0/512426513 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8c01fb30 con 0x55741d9e4500 2024-09-13T18:22:22.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.331+0000 7feca0c13740 1 -- 172.21.15.114:0/512426513 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55741d9e4500 msgr2=0x55741d9f5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:22.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.331+0000 7feca0c13740 1 --2- 172.21.15.114:0/512426513 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55741d9e4500 0x55741d9f5620 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fec8c027eb0 tx=0x7fec8c000bc0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.331+0000 7feca0c13740 1 -- 172.21.15.114:0/512426513 shutdown_connections 2024-09-13T18:22:22.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.331+0000 7feca0c13740 1 --2- 172.21.15.114:0/512426513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55741d9e4e40 0x55741d9ee3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.331+0000 7feca0c13740 1 --2- 172.21.15.114:0/512426513 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55741d9e4500 0x55741d9f5620 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.331+0000 7feca0c13740 1 --2- 172.21.15.114:0/512426513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55741d9dfb70 0x55741d9ede90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.331+0000 7feca0c13740 1 -- 172.21.15.114:0/512426513 >> 172.21.15.114:0/512426513 conn(0x55741d9ec630 msgr2=0x55741d9e5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:22.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.331+0000 7feca0c13740 1 -- 172.21.15.114:0/512426513 shutdown_connections 2024-09-13T18:22:22.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.331+0000 7feca0c13740 1 -- 172.21.15.114:0/512426513 wait complete. 2024-09-13T18:22:22.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.331+0000 7feca0c13740 1 -- start start 2024-09-13T18:22:22.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.332+0000 7feca0c13740 1 Processor -- start 2024-09-13T18:22:22.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.332+0000 7feca0c13740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55741d9dfb70 0x55741d9f2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.332+0000 7feca0c13740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55741d9e4500 0x55741d9f27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.332+0000 7feca0c13740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55741d9e4e40 0x55741da50350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.332+0000 7feca0c13740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55741d9f56b0 con 0x55741d9e4e40 2024-09-13T18:22:22.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.332+0000 7feca0c13740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55741d9f5820 con 0x55741d9dfb70 2024-09-13T18:22:22.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.332+0000 7feca0c13740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55741d9f59e0 con 0x55741d9e4500 2024-09-13T18:22:22.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.332+0000 7feca0c13740 10 client.0 _ll_register_callbacks cb 0x7fec8c009470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:22:22.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.332+0000 7fec97fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55741d9e4e40 0x55741da50350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.332+0000 7fec97fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55741d9e4e40 0x55741da50350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43612/0 (socket says 172.21.15.114:43612) 2024-09-13T18:22:22.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.332+0000 7fec97fff640 1 -- 172.21.15.114:0/2030299211 learned_addr learned my addr 172.21.15.114:0/2030299211 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:22.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.332+0000 7fec977fe640 1 --2- 172.21.15.114:0/2030299211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55741d9dfb70 0x55741d9f2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.332+0000 7fec96ffd640 1 --2- 172.21.15.114:0/2030299211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55741d9e4500 0x55741d9f27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.332+0000 7fec97fff640 1 -- 172.21.15.114:0/2030299211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55741d9e4500 msgr2=0x55741d9f27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:22.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.332+0000 7fec97fff640 1 --2- 172.21.15.114:0/2030299211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55741d9e4500 0x55741d9f27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.332+0000 7fec97fff640 1 -- 172.21.15.114:0/2030299211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55741d9dfb70 msgr2=0x55741d9f2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:22.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.332+0000 7fec97fff640 1 --2- 172.21.15.114:0/2030299211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55741d9dfb70 0x55741d9f2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.332+0000 7fec97fff640 1 -- 172.21.15.114:0/2030299211 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55741d946270 con 0x55741d9e4e40 2024-09-13T18:22:22.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.332+0000 7fec97fff640 1 --2- 172.21.15.114:0/2030299211 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55741d9e4e40 0x55741da50350 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fec900ebf20 tx=0x7fec900baf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b5fec299f9536e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:22.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.332+0000 7fec76ffd640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:22:22.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.332+0000 7fec76ffd640 1 -- 172.21.15.114:0/2030299211 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec900bb500 con 0x55741d9e4e40 2024-09-13T18:22:22.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.332+0000 7fec76ffd640 1 -- 172.21.15.114:0/2030299211 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fec900bf330 con 0x55741d9e4e40 2024-09-13T18:22:22.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.332+0000 7feca0c13740 1 -- 172.21.15.114:0/2030299211 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fec8c013ab0 con 0x55741d9e4e40 2024-09-13T18:22:22.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.333+0000 7fec76ffd640 1 -- 172.21.15.114:0/2030299211 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec900c3980 con 0x55741d9e4e40 2024-09-13T18:22:22.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.333+0000 7feca0c13740 10 client.15513 fetch_fsmap learned FSMap version 29 2024-09-13T18:22:22.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.333+0000 7feca0c13740 1 -- 172.21.15.114:0/2030299211 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55741d90dd00 con 0x55741d9e4e40 2024-09-13T18:22:22.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.333+0000 7fec76ffd640 1 -- 172.21.15.114:0/2030299211 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec8c013ab0 con 0x55741d9e4e40 2024-09-13T18:22:22.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.333+0000 7fec76ffd640 1 -- 172.21.15.114:0/2030299211 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fec9004c060 con 0x55741d9e4e40 2024-09-13T18:22:22.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.333+0000 7feca0c13740 10 client.15513 fetch_fsmap finished waiting for FSMap version 29 2024-09-13T18:22:22.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.333+0000 7feca0c13740 10 client.15513 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-13T18:22:22.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.333+0000 7feca0c13740 20 client.15513 populate_metadata read hostname 'smithi114' 2024-09-13T18:22:22.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.333+0000 7feca0c13740 1 --2- 172.21.15.114:0/2030299211 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55741d9e9ed0 0x55741da51b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.333+0000 7feca0c13740 4 client.15513 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T18:22:22.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.333+0000 7feca0c13740 1 -- 172.21.15.114:0/2030299211 --> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55741d69fd10 con 0x55741d9e9ed0 2024-09-13T18:22:22.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.333+0000 7fec977fe640 1 --2- 172.21.15.114:0/2030299211 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55741d9e9ed0 0x55741da51b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.334+0000 7fec76ffd640 10 client.15513 ms_handle_connect on v2:172.21.15.134:6836/891692210 2024-09-13T18:22:22.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.334+0000 7fec977fe640 1 --2- 172.21.15.114:0/2030299211 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55741d9e9ed0 0x55741da51b20 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9ded47cb125650dd server_cookie=58b1bb44963655e in_seq=0 out_seq=0 2024-09-13T18:22:22.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.334+0000 7fec76ffd640 1 -- 172.21.15.114:0/2030299211 <== mds.1 v2:172.21.15.134:6836/891692210 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55741d69fd10 con 0x55741d9e9ed0 2024-09-13T18:22:22.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.334+0000 7fec76ffd640 10 client.15513 handle_command_reply: tid=0 2024-09-13T18:22:22.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.334+0000 7feca0c13740 1 client.15513 shutdown 2024-09-13T18:22:22.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.334+0000 7feca0c13740 1 -- 172.21.15.114:0/2030299211 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55741d9e4e40 msgr2=0x55741da50350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:22.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.334+0000 7feca0c13740 1 --2- 172.21.15.114:0/2030299211 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55741d9e4e40 0x55741da50350 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fec900ebf20 tx=0x7fec900baf20 comp rx=0 tx=0).stop 2024-09-13T18:22:22.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.334+0000 7feca0c13740 1 -- 172.21.15.114:0/2030299211 shutdown_connections 2024-09-13T18:22:22.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.334+0000 7feca0c13740 1 --2- 172.21.15.114:0/2030299211 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55741d9e9ed0 0x55741da51b20 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.334+0000 7feca0c13740 1 --2- 172.21.15.114:0/2030299211 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55741d9e4e40 0x55741da50350 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.334+0000 7feca0c13740 1 --2- 172.21.15.114:0/2030299211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55741d9e4500 0x55741d9f27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.334+0000 7fec76ffd640 0 client.15513 ms_handle_reset on v2:172.21.15.134:6836/891692210 2024-09-13T18:22:22.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.334+0000 7feca0c13740 1 --2- 172.21.15.114:0/2030299211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55741d9dfb70 0x55741d9f2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.334+0000 7feca0c13740 1 -- 172.21.15.114:0/2030299211 >> 172.21.15.114:0/2030299211 conn(0x55741d9ec630 msgr2=0x55741d9ede90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:22.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.334+0000 7feca0c13740 1 -- 172.21.15.114:0/2030299211 shutdown_connections 2024-09-13T18:22:22.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.334+0000 7feca0c13740 1 -- 172.21.15.114:0/2030299211 wait complete. 2024-09-13T18:22:22.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.334+0000 7feca0c13740 20 client.15513 trim_cache size 0 max 16384 2024-09-13T18:22:22.343 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:22:22.343 INFO:teuthology.orchestra.run.smithi114.stdout: "cluster_fsid": "97e7edf2-71fb-11ef-bceb-c7b262605968", 2024-09-13T18:22:22.343 INFO:teuthology.orchestra.run.smithi114.stdout: "whoami": 1, 2024-09-13T18:22:22.343 INFO:teuthology.orchestra.run.smithi114.stdout: "id": 24482, 2024-09-13T18:22:22.343 INFO:teuthology.orchestra.run.smithi114.stdout: "want_state": "up:active", 2024-09-13T18:22:22.343 INFO:teuthology.orchestra.run.smithi114.stdout: "state": "up:active", 2024-09-13T18:22:22.344 INFO:teuthology.orchestra.run.smithi114.stdout: "fs_name": "cephfs", 2024-09-13T18:22:22.344 INFO:teuthology.orchestra.run.smithi114.stdout: "rank_uptime": 117.16295027300001, 2024-09-13T18:22:22.344 INFO:teuthology.orchestra.run.smithi114.stdout: "mdsmap_epoch": 29, 2024-09-13T18:22:22.344 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch": 93, 2024-09-13T18:22:22.344 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch_barrier": 91, 2024-09-13T18:22:22.344 INFO:teuthology.orchestra.run.smithi114.stdout: "uptime": 156.746187448 2024-09-13T18:22:22.344 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:22:22.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.334+0000 7fec9de12640 1 -- 172.21.15.114:0/787470678 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fec800a1110 msgr2=0x7fec800a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:22.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.334+0000 7fec9de12640 1 --2- 172.21.15.114:0/787470678 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fec800a1110 0x7fec800a35d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fec8800abe0 tx=0x7fec8800a5c0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.335+0000 7fec9de12640 1 -- 172.21.15.114:0/787470678 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec98105af0 msgr2=0x7fec980fed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:22.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.335+0000 7fec9de12640 1 --2- 172.21.15.114:0/787470678 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec98105af0 0x7fec980fed80 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fec9002ade0 tx=0x7fec90003c20 comp rx=0 tx=0).stop 2024-09-13T18:22:22.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.335+0000 7fec9de12640 1 -- 172.21.15.114:0/787470678 shutdown_connections 2024-09-13T18:22:22.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.335+0000 7fec9de12640 1 --2- 172.21.15.114:0/787470678 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fec800a1110 0x7fec800a35d0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.335+0000 7fec9de12640 1 --2- 172.21.15.114:0/787470678 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec98105af0 0x7fec980fed80 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.335+0000 7fec9de12640 1 --2- 172.21.15.114:0/787470678 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec981031c0 0x7fec980fe840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.335+0000 7fec9de12640 1 --2- 172.21.15.114:0/787470678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec98100890 0x7fec98100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.335+0000 7fec9de12640 1 -- 172.21.15.114:0/787470678 >> 172.21.15.114:0/787470678 conn(0x7fec980fa410 msgr2=0x7fec98101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:22.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.335+0000 7fec9de12640 1 -- 172.21.15.114:0/787470678 shutdown_connections 2024-09-13T18:22:22.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.335+0000 7fec9de12640 1 -- 172.21.15.114:0/787470678 wait complete. 2024-09-13T18:22:22.346 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:22 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2614911608' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T18:22:22.346 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:22 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/691757847' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:22:22.346 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:22 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/1078211570' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:22:22.347 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-13T18:22:22.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:22 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/2614911608' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T18:22:22.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:22 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/691757847' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:22:22.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:22 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/1078211570' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:22:22.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:22 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/2614911608' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T18:22:22.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:22 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/691757847' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:22:22.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:22 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/1078211570' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:22:22.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.445+0000 7f8d12ab8640 1 -- 172.21.15.114:0/2215200280 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d0c0ffd70 msgr2=0x7f8d0c100170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:22.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.445+0000 7f8d12ab8640 1 --2- 172.21.15.114:0/2215200280 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d0c0ffd70 0x7f8d0c100170 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc009900 tx=0x7f8cfc02a410 comp rx=0 tx=0).stop 2024-09-13T18:22:22.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.445+0000 7f8d12ab8640 1 -- 172.21.15.114:0/2215200280 shutdown_connections 2024-09-13T18:22:22.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.445+0000 7f8d12ab8640 1 --2- 172.21.15.114:0/2215200280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d0c101920 0x7f8d0c1081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.445+0000 7f8d12ab8640 1 --2- 172.21.15.114:0/2215200280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d0c100f60 0x7f8d0c1013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.445+0000 7f8d12ab8640 1 --2- 172.21.15.114:0/2215200280 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d0c0ffd70 0x7f8d0c100170 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.445+0000 7f8d12ab8640 1 -- 172.21.15.114:0/2215200280 >> 172.21.15.114:0/2215200280 conn(0x7f8d0c0fb4d0 msgr2=0x7f8d0c0fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:22.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.445+0000 7f8d12ab8640 1 -- 172.21.15.114:0/2215200280 shutdown_connections 2024-09-13T18:22:22.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.445+0000 7f8d12ab8640 1 -- 172.21.15.114:0/2215200280 wait complete. 2024-09-13T18:22:22.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.446+0000 7f8d12ab8640 1 Processor -- start 2024-09-13T18:22:22.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.446+0000 7f8d12ab8640 1 -- start start 2024-09-13T18:22:22.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.446+0000 7f8d12ab8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d0c0ffd70 0x7f8d0c195cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.446+0000 7f8d12ab8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d0c100f60 0x7f8d0c196230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.446+0000 7f8d12ab8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d0c101920 0x7f8d0c19d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.446+0000 7f8d12ab8640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0c19d7d0 con 0x7f8d0c100f60 2024-09-13T18:22:22.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.446+0000 7f8d12ab8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0c19d940 con 0x7f8d0c0ffd70 2024-09-13T18:22:22.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.446+0000 7f8d12ab8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0c19db00 con 0x7f8d0c101920 2024-09-13T18:22:22.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.446+0000 7f8d0bfff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d0c100f60 0x7f8d0c196230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.446+0000 7f8d0bfff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d0c100f60 0x7f8d0c196230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43632/0 (socket says 172.21.15.114:43632) 2024-09-13T18:22:22.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.446+0000 7f8d0bfff640 1 -- 172.21.15.114:0/2918848581 learned_addr learned my addr 172.21.15.114:0/2918848581 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:22.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.446+0000 7f8d11055640 1 --2- 172.21.15.114:0/2918848581 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d0c101920 0x7f8d0c19d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.446+0000 7f8d10854640 1 --2- 172.21.15.114:0/2918848581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d0c0ffd70 0x7f8d0c195cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.446+0000 7f8d0bfff640 1 -- 172.21.15.114:0/2918848581 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d0c101920 msgr2=0x7f8d0c19d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:22.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.446+0000 7f8d0bfff640 1 --2- 172.21.15.114:0/2918848581 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d0c101920 0x7f8d0c19d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.447+0000 7f8d0bfff640 1 -- 172.21.15.114:0/2918848581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d0c0ffd70 msgr2=0x7f8d0c195cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:22.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.447+0000 7f8d0bfff640 1 --2- 172.21.15.114:0/2918848581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d0c0ffd70 0x7f8d0c195cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.447+0000 7f8d0bfff640 1 -- 172.21.15.114:0/2918848581 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cfc009560 con 0x7f8d0c100f60 2024-09-13T18:22:22.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.447+0000 7f8d11055640 1 --2- 172.21.15.114:0/2918848581 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d0c101920 0x7f8d0c19d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:22:22.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.447+0000 7f8d0bfff640 1 --2- 172.21.15.114:0/2918848581 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d0c100f60 0x7f8d0c196230 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f8d0000d770 tx=0x7f8d0000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:22.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.447+0000 7f8d09ffb640 1 -- 172.21.15.114:0/2918848581 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d00009c30 con 0x7f8d0c100f60 2024-09-13T18:22:22.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.447+0000 7f8d09ffb640 1 -- 172.21.15.114:0/2918848581 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8d0000de80 con 0x7f8d0c100f60 2024-09-13T18:22:22.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.447+0000 7f8d12ab8640 1 -- 172.21.15.114:0/2918848581 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d0c19dde0 con 0x7f8d0c100f60 2024-09-13T18:22:22.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.447+0000 7f8d09ffb640 1 -- 172.21.15.114:0/2918848581 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d00004d40 con 0x7f8d0c100f60 2024-09-13T18:22:22.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.447+0000 7f8d12ab8640 1 -- 172.21.15.114:0/2918848581 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d0c19e260 con 0x7f8d0c100f60 2024-09-13T18:22:22.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.449+0000 7f8d09ffb640 1 -- 172.21.15.114:0/2918848581 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f8d00004ea0 con 0x7f8d0c100f60 2024-09-13T18:22:22.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.450+0000 7f8d157d9740 1 Processor -- start 2024-09-13T18:22:22.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.450+0000 7f8d157d9740 1 -- start start 2024-09-13T18:22:22.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.450+0000 7f8d157d9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c1ba50500 0x556c1ba50900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.450+0000 7f8d157d9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556c1ba50e40 0x556c1ba55900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.450+0000 7f8d09ffb640 1 --2- 172.21.15.114:0/2918848581 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f8cf40a1110 0x7f8cf40a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.450+0000 7f8d157d9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c1ba55ed0 0x556c1ba58390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.450+0000 7f8d157d9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c1b9ab7c0 con 0x556c1ba50e40 2024-09-13T18:22:22.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.450+0000 7f8d157d9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c1b9bab70 con 0x556c1ba50500 2024-09-13T18:22:22.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.450+0000 7f8d157d9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c1b9b8300 con 0x556c1ba55ed0 2024-09-13T18:22:22.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.450+0000 7f8d10854640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556c1ba50e40 0x556c1ba55900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.450+0000 7f8d10854640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556c1ba50e40 0x556c1ba55900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43638/0 (socket says 172.21.15.114:43638) 2024-09-13T18:22:22.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.450+0000 7f8d10854640 1 -- 172.21.15.114:0/1931401923 learned_addr learned my addr 172.21.15.114:0/1931401923 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:22.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.450+0000 7f8d11055640 1 --2- 172.21.15.114:0/1931401923 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c1ba50500 0x556c1ba50900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.450+0000 7f8d09ffb640 1 -- 172.21.15.114:0/2918848581 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f8d000cc7f0 con 0x7f8d0c100f60 2024-09-13T18:22:22.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.450+0000 7f8d0bfff640 1 --2- 172.21.15.114:0/2918848581 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f8cf40a1110 0x7f8cf40a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.450+0000 7f8d11055640 1 --2- 172.21.15.114:0/1931401923 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c1ba55ed0 0x556c1ba58390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.450+0000 7f8d10854640 1 -- 172.21.15.114:0/1931401923 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c1ba55ed0 msgr2=0x556c1ba58390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:22.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.450+0000 7f8d10854640 1 --2- 172.21.15.114:0/1931401923 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c1ba55ed0 0x556c1ba58390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.450+0000 7f8d10854640 1 -- 172.21.15.114:0/1931401923 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c1ba50500 msgr2=0x556c1ba50900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:22.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.450+0000 7f8d10854640 1 --2- 172.21.15.114:0/1931401923 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c1ba50500 0x556c1ba50900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.450+0000 7f8d10854640 1 -- 172.21.15.114:0/1931401923 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c1b9404b0 con 0x556c1ba50e40 2024-09-13T18:22:22.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.450+0000 7f8d11055640 1 --2- 172.21.15.114:0/1931401923 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c1ba50500 0x556c1ba50900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:22:22.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.451+0000 7f8d0bfff640 1 --2- 172.21.15.114:0/2918848581 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f8cf40a1110 0x7f8cf40a35d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8d0000d740 tx=0x7f8d000b9040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:22.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.451+0000 7f8d10854640 1 --2- 172.21.15.114:0/1931401923 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556c1ba50e40 0x556c1ba55900 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc006010 tx=0x7f8cfc0315c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5058c90ea8c3fe9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:22.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.451+0000 7f8cf2ffd640 1 -- 172.21.15.114:0/1931401923 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc0317b0 con 0x556c1ba50e40 2024-09-13T18:22:22.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.451+0000 7f8cf2ffd640 1 -- 172.21.15.114:0/1931401923 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8cfc031df0 con 0x556c1ba50e40 2024-09-13T18:22:22.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.451+0000 7f8cf2ffd640 1 -- 172.21.15.114:0/1931401923 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc003b90 con 0x556c1ba50e40 2024-09-13T18:22:22.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.451+0000 7f8d157d9740 1 -- 172.21.15.114:0/1931401923 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556c1ba50e40 msgr2=0x556c1ba55900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:22.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.451+0000 7f8d157d9740 1 --2- 172.21.15.114:0/1931401923 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556c1ba50e40 0x556c1ba55900 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc006010 tx=0x7f8cfc0315c0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.451+0000 7f8d157d9740 1 -- 172.21.15.114:0/1931401923 shutdown_connections 2024-09-13T18:22:22.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.451+0000 7f8d157d9740 1 --2- 172.21.15.114:0/1931401923 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c1ba55ed0 0x556c1ba58390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.451+0000 7f8d157d9740 1 --2- 172.21.15.114:0/1931401923 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556c1ba50e40 0x556c1ba55900 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.451+0000 7f8d157d9740 1 --2- 172.21.15.114:0/1931401923 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c1ba50500 0x556c1ba50900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.451+0000 7f8d157d9740 1 -- 172.21.15.114:0/1931401923 >> 172.21.15.114:0/1931401923 conn(0x556c1ba4bb70 msgr2=0x556c1ba4bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:22.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.451+0000 7f8d157d9740 1 -- 172.21.15.114:0/1931401923 shutdown_connections 2024-09-13T18:22:22.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.451+0000 7f8d157d9740 1 -- 172.21.15.114:0/1931401923 wait complete. 2024-09-13T18:22:22.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.452+0000 7f8d157d9740 1 -- start start 2024-09-13T18:22:22.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.452+0000 7f8d157d9740 1 Processor -- start 2024-09-13T18:22:22.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.452+0000 7f8d157d9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556c1ba50500 0x556c1ba5a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.452+0000 7f8d157d9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c1ba50e40 0x556c1ba5a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.452+0000 7f8d157d9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c1ba55ed0 0x556c1ba61680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.452+0000 7f8d157d9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c1b9b3bf0 con 0x556c1ba50500 2024-09-13T18:22:22.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.452+0000 7f8d157d9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c1b972250 con 0x556c1ba55ed0 2024-09-13T18:22:22.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.452+0000 7f8d157d9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c1b971ef0 con 0x556c1ba50e40 2024-09-13T18:22:22.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.452+0000 7f8d157d9740 10 client.0 _ll_register_callbacks cb 0x556c1b9a4740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:22:22.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.452+0000 7f8d11055640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556c1ba50500 0x556c1ba5a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.452+0000 7f8d11055640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556c1ba50500 0x556c1ba5a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43648/0 (socket says 172.21.15.114:43648) 2024-09-13T18:22:22.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.452+0000 7f8d11055640 1 -- 172.21.15.114:0/2157927318 learned_addr learned my addr 172.21.15.114:0/2157927318 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:22.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.452+0000 7f8d10854640 1 --2- 172.21.15.114:0/2157927318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c1ba50e40 0x556c1ba5a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.453+0000 7f8d11055640 1 -- 172.21.15.114:0/2157927318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c1ba50e40 msgr2=0x556c1ba5a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:22.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.453+0000 7f8d11055640 1 --2- 172.21.15.114:0/2157927318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c1ba50e40 0x556c1ba5a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.453+0000 7f8d11055640 1 -- 172.21.15.114:0/2157927318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c1ba55ed0 msgr2=0x556c1ba61680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:22.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.453+0000 7f8d11055640 1 --2- 172.21.15.114:0/2157927318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c1ba55ed0 0x556c1ba61680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.453+0000 7f8d11055640 1 -- 172.21.15.114:0/2157927318 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d0c108ab0 con 0x556c1ba50500 2024-09-13T18:22:22.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.453+0000 7f8d11055640 1 --2- 172.21.15.114:0/2157927318 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556c1ba50500 0x556c1ba5a100 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8d0400a570 tx=0x7f8d0400e410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19840e2bf36cf88b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:22.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.453+0000 7f8cebfff640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:22:22.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.453+0000 7f8cebfff640 1 -- 172.21.15.114:0/2157927318 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0401c400 con 0x556c1ba50500 2024-09-13T18:22:22.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.453+0000 7f8cebfff640 1 -- 172.21.15.114:0/2157927318 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f8d0401c560 con 0x556c1ba50500 2024-09-13T18:22:22.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.453+0000 7f8d157d9740 1 -- 172.21.15.114:0/2157927318 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556c1b6a73c0 con 0x556c1ba50500 2024-09-13T18:22:22.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.453+0000 7f8cebfff640 1 -- 172.21.15.114:0/2157927318 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d0401c720 con 0x556c1ba50500 2024-09-13T18:22:22.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.453+0000 7f8d157d9740 10 client.15525 fetch_fsmap learned FSMap version 29 2024-09-13T18:22:22.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.453+0000 7f8d157d9740 1 -- 172.21.15.114:0/2157927318 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x556c1b9b2270 con 0x556c1ba50500 2024-09-13T18:22:22.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.453+0000 7f8cebfff640 1 -- 172.21.15.114:0/2157927318 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x556c1b6a73c0 con 0x556c1ba50500 2024-09-13T18:22:22.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.453+0000 7f8cebfff640 1 -- 172.21.15.114:0/2157927318 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8d0401a030 con 0x556c1ba50500 2024-09-13T18:22:22.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.453+0000 7f8d157d9740 10 client.15525 fetch_fsmap finished waiting for FSMap version 29 2024-09-13T18:22:22.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.453+0000 7f8d157d9740 10 client.15525 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-13T18:22:22.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.453+0000 7f8d157d9740 20 client.15525 populate_metadata read hostname 'smithi114' 2024-09-13T18:22:22.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.454+0000 7f8d157d9740 1 --2- 172.21.15.114:0/2157927318 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x556c1ba666d0 0x556c1ba6a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.454+0000 7f8d157d9740 4 client.15525 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:22:22.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.454+0000 7f8d157d9740 1 -- 172.21.15.114:0/2157927318 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556c1b6b86c0 con 0x556c1ba666d0 2024-09-13T18:22:22.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.454+0000 7f8d10854640 1 --2- 172.21.15.114:0/2157927318 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x556c1ba666d0 0x556c1ba6a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.454+0000 7f8d10854640 1 --2- 172.21.15.114:0/2157927318 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x556c1ba666d0 0x556c1ba6a850 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4836304b9a61e8fe server_cookie=f019cdeb1bd88f59 in_seq=0 out_seq=0 2024-09-13T18:22:22.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.454+0000 7f8cebfff640 10 client.15525 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:22:22.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.455+0000 7f8cebfff640 1 -- 172.21.15.114:0/2157927318 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x556c1b6b86c0 con 0x556c1ba666d0 2024-09-13T18:22:22.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.455+0000 7f8cebfff640 10 client.15525 handle_command_reply: tid=0 2024-09-13T18:22:22.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.455+0000 7f8d157d9740 1 client.15525 shutdown 2024-09-13T18:22:22.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.455+0000 7f8d157d9740 1 -- 172.21.15.114:0/2157927318 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556c1ba50500 msgr2=0x556c1ba5a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:22.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.455+0000 7f8d157d9740 1 --2- 172.21.15.114:0/2157927318 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556c1ba50500 0x556c1ba5a100 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8d0400a570 tx=0x7f8d0400e410 comp rx=0 tx=0).stop 2024-09-13T18:22:22.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.455+0000 7f8d157d9740 1 -- 172.21.15.114:0/2157927318 shutdown_connections 2024-09-13T18:22:22.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.455+0000 7f8d157d9740 1 --2- 172.21.15.114:0/2157927318 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x556c1ba666d0 0x556c1ba6a850 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.455+0000 7f8d157d9740 1 --2- 172.21.15.114:0/2157927318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c1ba55ed0 0x556c1ba61680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.455+0000 7f8cebfff640 0 client.15525 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:22:22.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.455+0000 7f8d157d9740 1 --2- 172.21.15.114:0/2157927318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c1ba50e40 0x556c1ba5a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.455+0000 7f8d157d9740 1 --2- 172.21.15.114:0/2157927318 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556c1ba50500 0x556c1ba5a100 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.455+0000 7f8d157d9740 1 -- 172.21.15.114:0/2157927318 >> 172.21.15.114:0/2157927318 conn(0x556c1ba4bb70 msgr2=0x556c1ba4d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:22.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.455+0000 7f8d157d9740 1 -- 172.21.15.114:0/2157927318 shutdown_connections 2024-09-13T18:22:22.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.455+0000 7f8d157d9740 1 -- 172.21.15.114:0/2157927318 wait complete. 2024-09-13T18:22:22.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.456+0000 7f8d157d9740 20 client.15525 trim_cache size 0 max 16384 2024-09-13T18:22:22.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.470+0000 7f8d157d9740 1 Processor -- start 2024-09-13T18:22:22.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.470+0000 7f8d157d9740 1 -- start start 2024-09-13T18:22:22.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.470+0000 7f8d157d9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c1ba50500 0x556c1ba59eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.470+0000 7f8d157d9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c1ba50e40 0x556c1ba5a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.470+0000 7f8d157d9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556c1ba55ed0 0x556c1ba6e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.470+0000 7f8d157d9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c1b937460 con 0x556c1ba55ed0 2024-09-13T18:22:22.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.470+0000 7f8d157d9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c1ba516e0 con 0x556c1ba50e40 2024-09-13T18:22:22.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.470+0000 7f8d157d9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c1ba5ae60 con 0x556c1ba50500 2024-09-13T18:22:22.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.470+0000 7f8d11055640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556c1ba55ed0 0x556c1ba6e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.470+0000 7f8d11055640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556c1ba55ed0 0x556c1ba6e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43662/0 (socket says 172.21.15.114:43662) 2024-09-13T18:22:22.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.470+0000 7f8d11055640 1 -- 172.21.15.114:0/266369501 learned_addr learned my addr 172.21.15.114:0/266369501 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:22.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.470+0000 7f8d11055640 1 -- 172.21.15.114:0/266369501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c1ba50500 msgr2=0x556c1ba59eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:22.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.470+0000 7f8d11055640 1 --2- 172.21.15.114:0/266369501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c1ba50500 0x556c1ba59eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.470+0000 7f8d11055640 1 -- 172.21.15.114:0/266369501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c1ba50e40 msgr2=0x556c1ba5a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:22.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.470+0000 7f8d11055640 1 --2- 172.21.15.114:0/266369501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c1ba50e40 0x556c1ba5a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.471+0000 7f8d11055640 1 -- 172.21.15.114:0/266369501 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c1ba59430 con 0x556c1ba55ed0 2024-09-13T18:22:22.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.471+0000 7f8d11055640 1 --2- 172.21.15.114:0/266369501 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556c1ba55ed0 0x556c1ba6e870 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8d04000f80 tx=0x7f8d0400a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbde38a4453bff12 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:22.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.471+0000 7f8cf37fe640 1 -- 172.21.15.114:0/266369501 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0400a9f0 con 0x556c1ba55ed0 2024-09-13T18:22:22.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.471+0000 7f8cf37fe640 1 -- 172.21.15.114:0/266369501 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8d04010b80 con 0x556c1ba55ed0 2024-09-13T18:22:22.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.471+0000 7f8cf37fe640 1 -- 172.21.15.114:0/266369501 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0400f560 con 0x556c1ba55ed0 2024-09-13T18:22:22.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.471+0000 7f8d157d9740 1 -- 172.21.15.114:0/266369501 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556c1ba55ed0 msgr2=0x556c1ba6e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:22.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.471+0000 7f8d157d9740 1 --2- 172.21.15.114:0/266369501 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556c1ba55ed0 0x556c1ba6e870 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8d04000f80 tx=0x7f8d0400a5b0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.471+0000 7f8d157d9740 1 -- 172.21.15.114:0/266369501 shutdown_connections 2024-09-13T18:22:22.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.471+0000 7f8d157d9740 1 --2- 172.21.15.114:0/266369501 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556c1ba55ed0 0x556c1ba6e870 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.471+0000 7f8d157d9740 1 --2- 172.21.15.114:0/266369501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c1ba50e40 0x556c1ba5a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.471+0000 7f8d157d9740 1 --2- 172.21.15.114:0/266369501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c1ba50500 0x556c1ba59eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.471+0000 7f8d157d9740 1 -- 172.21.15.114:0/266369501 >> 172.21.15.114:0/266369501 conn(0x556c1ba4bb70 msgr2=0x556c1ba51d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:22.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.471+0000 7f8d157d9740 1 -- 172.21.15.114:0/266369501 shutdown_connections 2024-09-13T18:22:22.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.471+0000 7f8d157d9740 1 -- 172.21.15.114:0/266369501 wait complete. 2024-09-13T18:22:22.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.472+0000 7f8d157d9740 1 -- start start 2024-09-13T18:22:22.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.472+0000 7f8d157d9740 1 Processor -- start 2024-09-13T18:22:22.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.472+0000 7f8d157d9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c1ba50500 0x556c1bab8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.472+0000 7f8d157d9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556c1ba50e40 0x556c1bab9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.472+0000 7f8d157d9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c1ba55ed0 0x556c1bac00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.472+0000 7f8d157d9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c1bac0600 con 0x556c1ba50e40 2024-09-13T18:22:22.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.472+0000 7f8d157d9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c1bac0740 con 0x556c1ba55ed0 2024-09-13T18:22:22.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.472+0000 7f8d157d9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c1bac08d0 con 0x556c1ba50500 2024-09-13T18:22:22.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.472+0000 7f8d157d9740 10 client.0 _ll_register_callbacks cb 0x7f8cfc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:22:22.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.472+0000 7f8d10854640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556c1ba50e40 0x556c1bab9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.472+0000 7f8d10854640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556c1ba50e40 0x556c1bab9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43670/0 (socket says 172.21.15.114:43670) 2024-09-13T18:22:22.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.472+0000 7f8d10854640 1 -- 172.21.15.114:0/102135863 learned_addr learned my addr 172.21.15.114:0/102135863 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:22.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.472+0000 7f8d11055640 1 --2- 172.21.15.114:0/102135863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c1ba50500 0x556c1bab8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.472+0000 7f8d10854640 1 -- 172.21.15.114:0/102135863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c1ba50500 msgr2=0x556c1bab8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:22.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.472+0000 7f8d10854640 1 --2- 172.21.15.114:0/102135863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c1ba50500 0x556c1bab8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.472+0000 7f8d10854640 1 -- 172.21.15.114:0/102135863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c1ba55ed0 msgr2=0x556c1bac00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:22.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.472+0000 7f8d10854640 1 --2- 172.21.15.114:0/102135863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c1ba55ed0 0x556c1bac00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.472+0000 7f8d10854640 1 -- 172.21.15.114:0/102135863 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556c1b9b2270 con 0x556c1ba50e40 2024-09-13T18:22:22.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.472+0000 7f8d11055640 1 --2- 172.21.15.114:0/102135863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c1ba50500 0x556c1bab8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:22:22.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.473+0000 7f8d10854640 1 --2- 172.21.15.114:0/102135863 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556c1ba50e40 0x556c1bab9170 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc02a930 tx=0x7f8cfc031b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb18740e98780671 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:22.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.473+0000 7f8cebfff640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:22:22.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.473+0000 7f8cebfff640 1 -- 172.21.15.114:0/102135863 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc04c040 con 0x556c1ba50e40 2024-09-13T18:22:22.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.473+0000 7f8cebfff640 1 -- 172.21.15.114:0/102135863 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f8cfc053250 con 0x556c1ba50e40 2024-09-13T18:22:22.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.473+0000 7f8d157d9740 1 -- 172.21.15.114:0/102135863 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8cfc031df0 con 0x556c1ba50e40 2024-09-13T18:22:22.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.473+0000 7f8cebfff640 1 -- 172.21.15.114:0/102135863 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8cfc031c90 con 0x556c1ba50e40 2024-09-13T18:22:22.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.473+0000 7f8d157d9740 10 client.15531 fetch_fsmap learned FSMap version 29 2024-09-13T18:22:22.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.473+0000 7f8d157d9740 1 -- 172.21.15.114:0/102135863 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x556c1b9545f0 con 0x556c1ba50e40 2024-09-13T18:22:22.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.473+0000 7f8cebfff640 1 -- 172.21.15.114:0/102135863 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8cfc031df0 con 0x556c1ba50e40 2024-09-13T18:22:22.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.473+0000 7f8cebfff640 1 -- 172.21.15.114:0/102135863 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8cfc043020 con 0x556c1ba50e40 2024-09-13T18:22:22.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.473+0000 7f8d157d9740 10 client.15531 fetch_fsmap finished waiting for FSMap version 29 2024-09-13T18:22:22.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.473+0000 7f8d157d9740 10 client.15531 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-13T18:22:22.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.473+0000 7f8d157d9740 20 client.15531 populate_metadata read hostname 'smithi114' 2024-09-13T18:22:22.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.473+0000 7f8d157d9740 1 --2- 172.21.15.114:0/102135863 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x556c1bac5320 0x556c1bac1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.473+0000 7f8d157d9740 4 client.15531 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T18:22:22.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.473+0000 7f8d157d9740 1 -- 172.21.15.114:0/102135863 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556c1b70bd10 con 0x556c1bac5320 2024-09-13T18:22:22.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.473+0000 7f8d11055640 1 --2- 172.21.15.114:0/102135863 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x556c1bac5320 0x556c1bac1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.474+0000 7f8cebfff640 10 client.15531 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:22:22.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.474+0000 7f8d11055640 1 --2- 172.21.15.114:0/102135863 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x556c1bac5320 0x556c1bac1c90 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4bc94c31ea68621 server_cookie=f8948a68ddc78e78 in_seq=0 out_seq=0 2024-09-13T18:22:22.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.474+0000 7f8cebfff640 1 -- 172.21.15.114:0/102135863 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x556c1b70bd10 con 0x556c1bac5320 2024-09-13T18:22:22.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.474+0000 7f8cebfff640 10 client.15531 handle_command_reply: tid=0 2024-09-13T18:22:22.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.474+0000 7f8d157d9740 1 client.15531 shutdown 2024-09-13T18:22:22.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.474+0000 7f8d157d9740 1 -- 172.21.15.114:0/102135863 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556c1ba50e40 msgr2=0x556c1bab9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:22.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.474+0000 7f8d157d9740 1 --2- 172.21.15.114:0/102135863 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556c1ba50e40 0x556c1bab9170 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc02a930 tx=0x7f8cfc031b20 comp rx=0 tx=0).stop 2024-09-13T18:22:22.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.474+0000 7f8d157d9740 1 -- 172.21.15.114:0/102135863 shutdown_connections 2024-09-13T18:22:22.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.474+0000 7f8d157d9740 1 --2- 172.21.15.114:0/102135863 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x556c1bac5320 0x556c1bac1c90 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.474+0000 7f8d157d9740 1 --2- 172.21.15.114:0/102135863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c1ba55ed0 0x556c1bac00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.474+0000 7f8d157d9740 1 --2- 172.21.15.114:0/102135863 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556c1ba50e40 0x556c1bab9170 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.474+0000 7f8d157d9740 1 --2- 172.21.15.114:0/102135863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c1ba50500 0x556c1bab8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.474+0000 7f8cebfff640 0 client.15531 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:22:22.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.474+0000 7f8d157d9740 1 -- 172.21.15.114:0/102135863 >> 172.21.15.114:0/102135863 conn(0x556c1ba4bb70 msgr2=0x556c1ba59eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:22.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.474+0000 7f8d157d9740 1 -- 172.21.15.114:0/102135863 shutdown_connections 2024-09-13T18:22:22.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.474+0000 7f8d157d9740 1 -- 172.21.15.114:0/102135863 wait complete. 2024-09-13T18:22:22.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.474+0000 7f8d157d9740 20 client.15531 trim_cache size 0 max 16384 2024-09-13T18:22:22.488 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:22:22.488 INFO:teuthology.orchestra.run.smithi114.stdout: "cluster_fsid": "97e7edf2-71fb-11ef-bceb-c7b262605968", 2024-09-13T18:22:22.488 INFO:teuthology.orchestra.run.smithi114.stdout: "whoami": 0, 2024-09-13T18:22:22.488 INFO:teuthology.orchestra.run.smithi114.stdout: "id": 24521, 2024-09-13T18:22:22.488 INFO:teuthology.orchestra.run.smithi114.stdout: "want_state": "up:active", 2024-09-13T18:22:22.488 INFO:teuthology.orchestra.run.smithi114.stdout: "state": "up:active", 2024-09-13T18:22:22.488 INFO:teuthology.orchestra.run.smithi114.stdout: "fs_name": "cephfs", 2024-09-13T18:22:22.488 INFO:teuthology.orchestra.run.smithi114.stdout: "rank_uptime": 140.576483591, 2024-09-13T18:22:22.488 INFO:teuthology.orchestra.run.smithi114.stdout: "mdsmap_epoch": 29, 2024-09-13T18:22:22.489 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch": 93, 2024-09-13T18:22:22.489 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch_barrier": 91, 2024-09-13T18:22:22.489 INFO:teuthology.orchestra.run.smithi114.stdout: "uptime": 146.207193037 2024-09-13T18:22:22.489 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:22:22.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.475+0000 7f8d12ab8640 1 -- 172.21.15.114:0/2918848581 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f8cf40a1110 msgr2=0x7f8cf40a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:22.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.475+0000 7f8d12ab8640 1 --2- 172.21.15.114:0/2918848581 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f8cf40a1110 0x7f8cf40a35d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8d0000d740 tx=0x7f8d000b9040 comp rx=0 tx=0).stop 2024-09-13T18:22:22.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.475+0000 7f8d12ab8640 1 -- 172.21.15.114:0/2918848581 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d0c100f60 msgr2=0x7f8d0c196230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:22.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.475+0000 7f8d12ab8640 1 --2- 172.21.15.114:0/2918848581 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d0c100f60 0x7f8d0c196230 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f8d0000d770 tx=0x7f8d0000dc30 comp rx=0 tx=0).stop 2024-09-13T18:22:22.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.475+0000 7f8d12ab8640 1 -- 172.21.15.114:0/2918848581 shutdown_connections 2024-09-13T18:22:22.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.475+0000 7f8d12ab8640 1 --2- 172.21.15.114:0/2918848581 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f8cf40a1110 0x7f8cf40a35d0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.475+0000 7f8d12ab8640 1 --2- 172.21.15.114:0/2918848581 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d0c101920 0x7f8d0c19d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.475+0000 7f8d12ab8640 1 --2- 172.21.15.114:0/2918848581 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8d0c100f60 0x7f8d0c196230 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.475+0000 7f8d12ab8640 1 --2- 172.21.15.114:0/2918848581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d0c0ffd70 0x7f8d0c195cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.475+0000 7f8d12ab8640 1 -- 172.21.15.114:0/2918848581 >> 172.21.15.114:0/2918848581 conn(0x7f8d0c0fb4d0 msgr2=0x7f8d0c1044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:22.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.475+0000 7f8d12ab8640 1 -- 172.21.15.114:0/2918848581 shutdown_connections 2024-09-13T18:22:22.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.475+0000 7f8d12ab8640 1 -- 172.21.15.114:0/2918848581 wait complete. 2024-09-13T18:22:22.491 INFO:teuthology.run_tasks:Running task sequential... 2024-09-13T18:22:22.500 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-13T18:22:22.500 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-13T18:22:22.501 INFO:teuthology.task.print:Enabling mgr modules 2024-09-13T18:22:22.501 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-09-13T18:22:22.502 INFO:teuthology.task.exec:Executing custom commands... 2024-09-13T18:22:22.502 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi114.front.sepia.ceph.com 2024-09-13T18:22:22.502 DEBUG:teuthology.orchestra.run.smithi114:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-09-13T18:22:22.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.599+0000 7f3154733640 1 -- 172.21.15.114:0/3840596149 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f314c0fec30 msgr2=0x7f314c0ff030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:22.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.599+0000 7f3154733640 1 --2- 172.21.15.114:0/3840596149 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f314c0fec30 0x7f314c0ff030 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3140009900 tx=0x7f314002a410 comp rx=0 tx=0).stop 2024-09-13T18:22:22.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.599+0000 7f3154733640 1 -- 172.21.15.114:0/3840596149 shutdown_connections 2024-09-13T18:22:22.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.599+0000 7f3154733640 1 --2- 172.21.15.114:0/3840596149 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f314c1007e0 0x7f314c107070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.599+0000 7f3154733640 1 --2- 172.21.15.114:0/3840596149 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f314c0ffe20 0x7f314c1002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.599+0000 7f3154733640 1 --2- 172.21.15.114:0/3840596149 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f314c0fec30 0x7f314c0ff030 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.599+0000 7f3154733640 1 -- 172.21.15.114:0/3840596149 >> 172.21.15.114:0/3840596149 conn(0x7f314c0fa3b0 msgr2=0x7f314c0fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:22.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.599+0000 7f3154733640 1 -- 172.21.15.114:0/3840596149 shutdown_connections 2024-09-13T18:22:22.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.599+0000 7f3154733640 1 -- 172.21.15.114:0/3840596149 wait complete. 2024-09-13T18:22:22.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.599+0000 7f3154733640 1 Processor -- start 2024-09-13T18:22:22.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.599+0000 7f3154733640 1 -- start start 2024-09-13T18:22:22.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.599+0000 7f3154733640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f314c0fec30 0x7f314c195c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.599+0000 7f3154733640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f314c0ffe20 0x7f314c1961c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.599+0000 7f3154733640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f314c1007e0 0x7f314c19d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.599+0000 7f3154733640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f314c19d790 con 0x7f314c0ffe20 2024-09-13T18:22:22.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.599+0000 7f3154733640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f314c19d900 con 0x7f314c0fec30 2024-09-13T18:22:22.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.599+0000 7f3154733640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f314c19dac0 con 0x7f314c1007e0 2024-09-13T18:22:22.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.600+0000 7f3151cce640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f314c0ffe20 0x7f314c1961c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.600+0000 7f3151cce640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f314c0ffe20 0x7f314c1961c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43688/0 (socket says 172.21.15.114:43688) 2024-09-13T18:22:22.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.600+0000 7f3151cce640 1 -- 172.21.15.114:0/1987150353 learned_addr learned my addr 172.21.15.114:0/1987150353 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:22.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.600+0000 7f31524cf640 1 --2- 172.21.15.114:0/1987150353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f314c0fec30 0x7f314c195c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.600+0000 7f3151cce640 1 -- 172.21.15.114:0/1987150353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f314c1007e0 msgr2=0x7f314c19d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:22.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.600+0000 7f3151cce640 1 --2- 172.21.15.114:0/1987150353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f314c1007e0 0x7f314c19d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.600+0000 7f3151cce640 1 -- 172.21.15.114:0/1987150353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f314c0fec30 msgr2=0x7f314c195c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:22.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.600+0000 7f3151cce640 1 --2- 172.21.15.114:0/1987150353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f314c0fec30 0x7f314c195c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:22.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.600+0000 7f3151cce640 1 -- 172.21.15.114:0/1987150353 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3140009560 con 0x7f314c0ffe20 2024-09-13T18:22:22.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.600+0000 7f3151cce640 1 --2- 172.21.15.114:0/1987150353 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f314c0ffe20 0x7f314c1961c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f314800d770 tx=0x7f314800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:22.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.600+0000 7f313f7fe640 1 -- 172.21.15.114:0/1987150353 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3148009c30 con 0x7f314c0ffe20 2024-09-13T18:22:22.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.600+0000 7f313f7fe640 1 -- 172.21.15.114:0/1987150353 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f314800de80 con 0x7f314c0ffe20 2024-09-13T18:22:22.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.600+0000 7f3154733640 1 -- 172.21.15.114:0/1987150353 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f314c19dda0 con 0x7f314c0ffe20 2024-09-13T18:22:22.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.600+0000 7f313f7fe640 1 -- 172.21.15.114:0/1987150353 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3148004d40 con 0x7f314c0ffe20 2024-09-13T18:22:22.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.600+0000 7f3154733640 1 -- 172.21.15.114:0/1987150353 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f314c19e1d0 con 0x7f314c0ffe20 2024-09-13T18:22:22.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.601+0000 7f3154733640 1 -- 172.21.15.114:0/1987150353 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f314c1061b0 con 0x7f314c0ffe20 2024-09-13T18:22:22.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.602+0000 7f313f7fe640 1 -- 172.21.15.114:0/1987150353 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f3148004ea0 con 0x7f314c0ffe20 2024-09-13T18:22:22.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.603+0000 7f313f7fe640 1 --2- 172.21.15.114:0/1987150353 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f31380a1090 0x7f31380a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:22.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.603+0000 7f31524cf640 1 --2- 172.21.15.114:0/1987150353 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f31380a1090 0x7f31380a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:22.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.603+0000 7f313f7fe640 1 -- 172.21.15.114:0/1987150353 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f31480cc6b0 con 0x7f314c0ffe20 2024-09-13T18:22:22.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.604+0000 7f31524cf640 1 --2- 172.21.15.114:0/1987150353 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f31380a1090 0x7f31380a3550 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f31400057c0 tx=0x7f3140005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:22.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.607+0000 7f313f7fe640 1 -- 172.21.15.114:0/1987150353 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f3148095ad0 con 0x7f314c0ffe20 2024-09-13T18:22:22.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:22.814+0000 7f3154733640 1 -- 172.21.15.114:0/1987150353 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f314c02c4b0 con 0x7f314c0ffe20 2024-09-13T18:22:23.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:23.060+0000 7f313f7fe640 1 -- 172.21.15.114:0/1987150353 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mgrmap(e 19) v1 ==== 135891+0+0 (secure 0 0 0) 0x7f31480b90c0 con 0x7f314c0ffe20 2024-09-13T18:22:23.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:23.062+0000 7f313f7fe640 1 -- 172.21.15.114:0/1987150353 <== mon.0 v2:172.21.15.114:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) v1 ==== 92+0+0 (secure 0 0 0) 0x7f3148095ad0 con 0x7f314c0ffe20 2024-09-13T18:22:23.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:23.064+0000 7f3154733640 1 -- 172.21.15.114:0/1987150353 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f31380a1090 msgr2=0x7f31380a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:23.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:23.064+0000 7f3154733640 1 --2- 172.21.15.114:0/1987150353 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f31380a1090 0x7f31380a3550 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f31400057c0 tx=0x7f3140005a20 comp rx=0 tx=0).stop 2024-09-13T18:22:23.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:23.064+0000 7f3154733640 1 -- 172.21.15.114:0/1987150353 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f314c0ffe20 msgr2=0x7f314c1961c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:23.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:23.064+0000 7f3154733640 1 --2- 172.21.15.114:0/1987150353 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f314c0ffe20 0x7f314c1961c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f314800d770 tx=0x7f314800dc30 comp rx=0 tx=0).stop 2024-09-13T18:22:23.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:23.064+0000 7f3154733640 1 -- 172.21.15.114:0/1987150353 shutdown_connections 2024-09-13T18:22:23.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:23.064+0000 7f3154733640 1 --2- 172.21.15.114:0/1987150353 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f31380a1090 0x7f31380a3550 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:23.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:23.064+0000 7f3154733640 1 --2- 172.21.15.114:0/1987150353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f314c1007e0 0x7f314c19d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:23.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:23.064+0000 7f3154733640 1 --2- 172.21.15.114:0/1987150353 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f314c0ffe20 0x7f314c1961c0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:23.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:23.064+0000 7f3154733640 1 --2- 172.21.15.114:0/1987150353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f314c0fec30 0x7f314c195c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:23.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:23.064+0000 7f3154733640 1 -- 172.21.15.114:0/1987150353 >> 172.21.15.114:0/1987150353 conn(0x7f314c0fa3b0 msgr2=0x7f314c103380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:23.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:23.064+0000 7f3154733640 1 -- 172.21.15.114:0/1987150353 shutdown_connections 2024-09-13T18:22:23.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:23.064+0000 7f3154733640 1 -- 172.21.15.114:0/1987150353 wait complete. 2024-09-13T18:22:23.078 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-09-13T18:22:23.089 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-09-13T18:22:23.089 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-09-13T18:22:23.089 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-09-13T18:22:23.089 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-13T18:22:23.089 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-13T18:22:23.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:22:23.090 DEBUG:teuthology.orchestra.run.smithi114:> ip netns list 2024-09-13T18:22:23.106 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-13T18:22:23.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:22:23.107 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-13T18:22:23.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:22:23.174 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link delete ceph-brx 2024-09-13T18:22:23.285 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-09-13T18:22:23.285 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-13T18:22:23.285 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph fs ls 2024-09-13T18:22:23.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:23 smithi134 ceph-mon[34283]: pgmap v351: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-13T18:22:23.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:23 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/1987150353' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-13T18:22:23.369 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:22:23 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y[35685]: ignoring --setuser ceph since I am not root 2024-09-13T18:22:23.369 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:22:23 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y[35685]: ignoring --setgroup ceph since I am not root 2024-09-13T18:22:23.369 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:22:23 smithi134 ceph-mgr[35689]: -- 172.21.15.134:0/2418498311 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562692274d00 con 0x562692249400 2024-09-13T18:22:23.369 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:22:23 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y[35685]: 2024-09-13T18:22:23.204+0000 7f5a8ee45040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T18:22:23.369 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:22:23 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y[35685]: 2024-09-13T18:22:23.259+0000 7f5a8ee45040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T18:22:23.409 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:23 smithi204 ceph-mon[34777]: pgmap v351: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-13T18:22:23.409 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:23 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/1987150353' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-13T18:22:23.409 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:22:23 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z[36485]: ignoring --setuser ceph since I am not root 2024-09-13T18:22:23.409 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:22:23 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z[36485]: ignoring --setgroup ceph since I am not root 2024-09-13T18:22:23.409 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:22:23 smithi204 ceph-mgr[36523]: -- 172.21.15.204:0/804586617 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55dfd5568d00 con 0x55dfd553d000 2024-09-13T18:22:23.409 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:22:23 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z[36485]: 2024-09-13T18:22:23.200+0000 7f693a8bc040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T18:22:23.410 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:22:23 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z[36485]: 2024-09-13T18:22:23.256+0000 7f693a8bc040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T18:22:23.410 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:22:23 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z[36485]: 2024-09-13T18:22:23.405+0000 7f693a8bc040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T18:22:23.434 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:23 smithi114 ceph-mon[30897]: pgmap v351: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-13T18:22:23.434 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:23 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/1987150353' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-13T18:22:23.435 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:23 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: ignoring --setuser ceph since I am not root 2024-09-13T18:22:23.435 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:23 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: ignoring --setgroup ceph since I am not root 2024-09-13T18:22:23.435 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:23 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:22:23.207+0000 7fd9c57a6040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T18:22:23.435 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:23 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:22:23.267+0000 7fd9c57a6040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T18:22:23.435 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:23 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:22:23.432+0000 7fd9c57a6040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T18:22:23.504 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:22:23.694 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:22:23 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z[36485]: 2024-09-13T18:22:23.471+0000 7f693a8bc040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T18:22:23.694 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:22:23 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z[36485]: 2024-09-13T18:22:23.614+0000 7f693a8bc040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T18:22:23.720 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:23 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:22:23.511+0000 7fd9c57a6040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T18:22:23.720 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:23 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:22:23.660+0000 7fd9c57a6040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T18:22:23.756 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:22:23 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y[35685]: 2024-09-13T18:22:23.407+0000 7f5a8ee45040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T18:22:23.756 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:22:23 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y[35685]: 2024-09-13T18:22:23.472+0000 7f5a8ee45040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T18:22:23.757 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:22:23 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y[35685]: 2024-09-13T18:22:23.614+0000 7f5a8ee45040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T18:22:23.757 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:22:23 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y[35685]: 2024-09-13T18:22:23.753+0000 7f5a8ee45040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T18:22:24.060 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:22:23 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z[36485]: 2024-09-13T18:22:23.753+0000 7f693a8bc040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T18:22:24.067 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:23 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:22:23.805+0000 7fd9c57a6040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T18:22:24.362 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:24 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/1987150353' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-13T18:22:24.362 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:24 smithi114 ceph-mon[30897]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-13T18:22:24.362 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:24 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:22:24.116+0000 7fd9c57a6040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T18:22:24.362 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:24 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:22:24.176+0000 7fd9c57a6040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T18:22:24.362 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:24 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:22:24.359+0000 7fd9c57a6040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T18:22:24.368 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:24 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/1987150353' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-13T18:22:24.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:24 smithi134 ceph-mon[34283]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-13T18:22:24.369 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:22:24 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y[35685]: 2024-09-13T18:22:24.057+0000 7f5a8ee45040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T18:22:24.369 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:22:24 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y[35685]: 2024-09-13T18:22:24.118+0000 7f5a8ee45040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T18:22:24.369 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:22:24 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y[35685]: 2024-09-13T18:22:24.300+0000 7f5a8ee45040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T18:22:24.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:24 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/1987150353' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-13T18:22:24.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:24 smithi204 ceph-mon[34777]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-13T18:22:24.444 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:22:24 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z[36485]: 2024-09-13T18:22:24.056+0000 7f693a8bc040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T18:22:24.444 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:22:24 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z[36485]: 2024-09-13T18:22:24.118+0000 7f693a8bc040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T18:22:24.445 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:22:24 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z[36485]: 2024-09-13T18:22:24.299+0000 7f693a8bc040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T18:22:25.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.152+0000 7f092c6cf640 1 -- 172.21.15.114:0/1964084220 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09240feed0 msgr2=0x7f09240ff2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:25.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.152+0000 7f092c6cf640 1 --2- 172.21.15.114:0/1964084220 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09240feed0 0x7f09240ff2b0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0914009930 tx=0x7f091402a420 comp rx=0 tx=0).stop 2024-09-13T18:22:25.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.152+0000 7f092c6cf640 1 -- 172.21.15.114:0/1964084220 shutdown_connections 2024-09-13T18:22:25.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.152+0000 7f092c6cf640 1 --2- 172.21.15.114:0/1964084220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0924104000 0x7f092410bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:25.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.152+0000 7f092c6cf640 1 --2- 172.21.15.114:0/1964084220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09240ff880 0x7f09241038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:25.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.152+0000 7f092c6cf640 1 --2- 172.21.15.114:0/1964084220 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09240feed0 0x7f09240ff2b0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:25.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.152+0000 7f092c6cf640 1 -- 172.21.15.114:0/1964084220 >> 172.21.15.114:0/1964084220 conn(0x7f09240fa9b0 msgr2=0x7f09240fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:25.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.152+0000 7f092c6cf640 1 -- 172.21.15.114:0/1964084220 shutdown_connections 2024-09-13T18:22:25.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.152+0000 7f092c6cf640 1 -- 172.21.15.114:0/1964084220 wait complete. 2024-09-13T18:22:25.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.153+0000 7f092c6cf640 1 Processor -- start 2024-09-13T18:22:25.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.153+0000 7f092c6cf640 1 -- start start 2024-09-13T18:22:25.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.153+0000 7f092c6cf640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09240feed0 0x7f0924197060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:25.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.153+0000 7f092c6cf640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09240ff880 0x7f09241975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:25.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.153+0000 7f092c6cf640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0924104000 0x7f09241a20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:25.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.153+0000 7f092c6cf640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0924197dd0 con 0x7f0924104000 2024-09-13T18:22:25.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.153+0000 7f092c6cf640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09241a2620 con 0x7f09240ff880 2024-09-13T18:22:25.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.153+0000 7f092c6cf640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09241a2760 con 0x7f09240feed0 2024-09-13T18:22:25.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.153+0000 7f092ac6c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0924104000 0x7f09241a20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:25.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.153+0000 7f0929c6a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09240ff880 0x7f09241975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:25.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.153+0000 7f092a46b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09240feed0 0x7f0924197060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:25.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.153+0000 7f092ac6c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0924104000 0x7f09241a20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33044/0 (socket says 172.21.15.114:33044) 2024-09-13T18:22:25.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.153+0000 7f0929c6a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09240ff880 0x7f09241975a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:53500/0 (socket says 172.21.15.114:53500) 2024-09-13T18:22:25.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.153+0000 7f092a46b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09240feed0 0x7f0924197060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:58188/0 (socket says 172.21.15.114:58188) 2024-09-13T18:22:25.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.153+0000 7f092ac6c640 1 -- 172.21.15.114:0/628796131 learned_addr learned my addr 172.21.15.114:0/628796131 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:25.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.154+0000 7f092ac6c640 1 -- 172.21.15.114:0/628796131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09240feed0 msgr2=0x7f0924197060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:25.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.154+0000 7f092ac6c640 1 --2- 172.21.15.114:0/628796131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09240feed0 0x7f0924197060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:25.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.154+0000 7f092ac6c640 1 -- 172.21.15.114:0/628796131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09240ff880 msgr2=0x7f09241975a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:25.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.154+0000 7f092ac6c640 1 --2- 172.21.15.114:0/628796131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09240ff880 0x7f09241975a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:25.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.154+0000 7f092ac6c640 1 -- 172.21.15.114:0/628796131 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f091c009840 con 0x7f0924104000 2024-09-13T18:22:25.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.154+0000 7f0929c6a640 1 --2- 172.21.15.114:0/628796131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09240ff880 0x7f09241975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:22:25.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.154+0000 7f092ac6c640 1 --2- 172.21.15.114:0/628796131 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0924104000 0x7f09241a20e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f091c00efd0 tx=0x7f091c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:25.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.154+0000 7f091b7fe640 1 -- 172.21.15.114:0/628796131 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091c009b40 con 0x7f0924104000 2024-09-13T18:22:25.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.154+0000 7f091b7fe640 1 -- 172.21.15.114:0/628796131 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f091c004020 con 0x7f0924104000 2024-09-13T18:22:25.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.154+0000 7f092c6cf640 1 -- 172.21.15.114:0/628796131 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0914009560 con 0x7f0924104000 2024-09-13T18:22:25.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.154+0000 7f091b7fe640 1 -- 172.21.15.114:0/628796131 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091c010740 con 0x7f0924104000 2024-09-13T18:22:25.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.154+0000 7f092c6cf640 1 -- 172.21.15.114:0/628796131 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09241a2d00 con 0x7f0924104000 2024-09-13T18:22:25.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.155+0000 7f092c6cf640 1 -- 172.21.15.114:0/628796131 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09241003b0 con 0x7f0924104000 2024-09-13T18:22:25.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.156+0000 7f091b7fe640 1 -- 172.21.15.114:0/628796131 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 19) v1 ==== 135891+0+0 (secure 0 0 0) 0x7f091c003b70 con 0x7f0924104000 2024-09-13T18:22:25.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.157+0000 7f091b7fe640 1 --2- 172.21.15.114:0/628796131 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f09100a1150 0x7f09100a3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:25.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.157+0000 7f092a46b640 1 -- 172.21.15.114:0/628796131 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f09100a1150 msgr2=0x7f09100a3610 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.114:6800/3396185541 2024-09-13T18:22:25.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.157+0000 7f091b7fe640 1 -- 172.21.15.114:0/628796131 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f091c014070 con 0x7f0924104000 2024-09-13T18:22:25.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.157+0000 7f092a46b640 1 --2- 172.21.15.114:0/628796131 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f09100a1150 0x7f09100a3610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:22:25.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.161+0000 7f091b7fe640 1 -- 172.21.15.114:0/628796131 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f091c0965f0 con 0x7f0924104000 2024-09-13T18:22:25.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.358+0000 7f092a46b640 1 -- 172.21.15.114:0/628796131 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f09100a1150 msgr2=0x7f09100a3610 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.114:6800/3396185541 2024-09-13T18:22:25.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.358+0000 7f092a46b640 1 --2- 172.21.15.114:0/628796131 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f09100a1150 0x7f09100a3610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:22:25.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.366+0000 7f092c6cf640 1 -- 172.21.15.114:0/628796131 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f09240684f0 con 0x7f0924104000 2024-09-13T18:22:25.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.367+0000 7f091b7fe640 1 -- 172.21.15.114:0/628796131 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v29) v1 ==== 53+0+88 (secure 0 0 0) 0x7f091c099c10 con 0x7f0924104000 2024-09-13T18:22:25.369 INFO:teuthology.orchestra.run.smithi114.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-13T18:22:25.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.369+0000 7f092c6cf640 1 -- 172.21.15.114:0/628796131 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f09100a1150 msgr2=0x7f09100a3610 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:22:25.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.369+0000 7f092c6cf640 1 --2- 172.21.15.114:0/628796131 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f09100a1150 0x7f09100a3610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:25.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.369+0000 7f092c6cf640 1 -- 172.21.15.114:0/628796131 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0924104000 msgr2=0x7f09241a20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:25.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.369+0000 7f092c6cf640 1 --2- 172.21.15.114:0/628796131 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0924104000 0x7f09241a20e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f091c00efd0 tx=0x7f091c00c6a0 comp rx=0 tx=0).stop 2024-09-13T18:22:25.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.369+0000 7f092c6cf640 1 -- 172.21.15.114:0/628796131 shutdown_connections 2024-09-13T18:22:25.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.369+0000 7f092c6cf640 1 --2- 172.21.15.114:0/628796131 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7f09100a1150 0x7f09100a3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:25.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.369+0000 7f092c6cf640 1 --2- 172.21.15.114:0/628796131 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0924104000 0x7f09241a20e0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:25.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.369+0000 7f092c6cf640 1 --2- 172.21.15.114:0/628796131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09240ff880 0x7f09241975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:25.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.369+0000 7f092c6cf640 1 --2- 172.21.15.114:0/628796131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09240feed0 0x7f0924197060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:25.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.369+0000 7f092c6cf640 1 -- 172.21.15.114:0/628796131 >> 172.21.15.114:0/628796131 conn(0x7f09240fa9b0 msgr2=0x7f09240fbe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:25.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.369+0000 7f092c6cf640 1 -- 172.21.15.114:0/628796131 shutdown_connections 2024-09-13T18:22:25.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:25.369+0000 7f092c6cf640 1 -- 172.21.15.114:0/628796131 wait complete. 2024-09-13T18:22:25.420 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:22:25 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y[35685]: 2024-09-13T18:22:25.167+0000 7f5a8ee45040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T18:22:25.421 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:22:25 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y[35685]: 2024-09-13T18:22:25.233+0000 7f5a8ee45040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T18:22:25.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:25 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/628796131' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-13T18:22:25.445 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:22:25 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z[36485]: 2024-09-13T18:22:25.167+0000 7f693a8bc040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T18:22:25.445 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:22:25 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z[36485]: 2024-09-13T18:22:25.233+0000 7f693a8bc040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T18:22:25.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:25 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/628796131' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-13T18:22:25.720 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:25 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:22:25.262+0000 7fd9c57a6040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T18:22:25.720 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:25 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:22:25.329+0000 7fd9c57a6040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T18:22:25.868 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:25 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/628796131' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-13T18:22:26.136 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-13T18:22:26.136 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T18:22:26.136 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi114.front.sepia.ceph.com 2024-09-13T18:22:26.137 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-13T18:22:26.137 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-13T18:22:26.137 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-13T18:22:26.137 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-13T18:22:26.137 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T18:22:26.137 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-13T18:22:26.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:22:26.138 DEBUG:teuthology.orchestra.run.smithi114:> ip addr 2024-09-13T18:22:26.155 INFO:teuthology.orchestra.run.smithi114.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T18:22:26.155 INFO:teuthology.orchestra.run.smithi114.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T18:22:26.155 INFO:teuthology.orchestra.run.smithi114.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T18:22:26.156 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:22:26.156 INFO:teuthology.orchestra.run.smithi114.stdout: inet6 ::1/128 scope host 2024-09-13T18:22:26.156 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:22:26.156 INFO:teuthology.orchestra.run.smithi114.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-13T18:22:26.156 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether 0c:c4:7a:d8:a8:92 brd ff:ff:ff:ff:ff:ff 2024-09-13T18:22:26.156 INFO:teuthology.orchestra.run.smithi114.stdout: altname enp7s0f0 2024-09-13T18:22:26.156 INFO:teuthology.orchestra.run.smithi114.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-13T18:22:26.156 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether 0c:c4:7a:d8:a8:93 brd ff:ff:ff:ff:ff:ff 2024-09-13T18:22:26.156 INFO:teuthology.orchestra.run.smithi114.stdout: altname enp7s0f1 2024-09-13T18:22:26.156 INFO:teuthology.orchestra.run.smithi114.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-13T18:22:26.156 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether 0c:c4:7a:8f:cb:62 brd ff:ff:ff:ff:ff:ff 2024-09-13T18:22:26.156 INFO:teuthology.orchestra.run.smithi114.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T18:22:26.157 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether 0c:c4:7a:8f:cb:63 brd ff:ff:ff:ff:ff:ff 2024-09-13T18:22:26.157 INFO:teuthology.orchestra.run.smithi114.stdout: inet 172.21.15.114/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-13T18:22:26.157 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft 42164sec preferred_lft 42164sec 2024-09-13T18:22:26.157 INFO:teuthology.orchestra.run.smithi114.stdout: inet6 fe80::ec4:7aff:fe8f:cb63/64 scope link noprefixroute 2024-09-13T18:22:26.157 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:22:26.157 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-13T18:22:26.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:22:26.158 DEBUG:teuthology.orchestra.run.smithi114:> (cd / && exec bash -c ' 2024-09-13T18:22:26.158 DEBUG:teuthology.orchestra.run.smithi114:> set -e 2024-09-13T18:22:26.158 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link add name ceph-brx type bridge 2024-09-13T18:22:26.158 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip addr flush dev ceph-brx 2024-09-13T18:22:26.158 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link set ceph-brx up 2024-09-13T18:22:26.158 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-13T18:22:26.159 DEBUG:teuthology.orchestra.run.smithi114:> ') 2024-09-13T18:22:26.206 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:22:25 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y[35685]: 2024-09-13T18:22:25.947+0000 7f5a8ee45040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-13T18:22:26.206 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:22:26 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y[35685]: 2024-09-13T18:22:26.061+0000 7f5a8ee45040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T18:22:26.213 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:22:25 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z[36485]: 2024-09-13T18:22:25.954+0000 7f693a8bc040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-13T18:22:26.214 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:22:26 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z[36485]: 2024-09-13T18:22:26.069+0000 7f693a8bc040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T18:22:26.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:22:26.260 DEBUG:teuthology.orchestra.run.smithi114:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T18:22:26.285 INFO:teuthology.orchestra.run.smithi114.stdout:1 2024-09-13T18:22:26.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:22:26.286 DEBUG:teuthology.orchestra.run.smithi114:> route 2024-09-13T18:22:26.343 INFO:teuthology.orchestra.run.smithi114.stdout:Kernel IP routing table 2024-09-13T18:22:26.343 INFO:teuthology.orchestra.run.smithi114.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T18:22:26.344 INFO:teuthology.orchestra.run.smithi114.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-13T18:22:26.344 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-13T18:22:26.344 INFO:teuthology.orchestra.run.smithi114.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T18:22:26.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:22:26.344 DEBUG:teuthology.orchestra.run.smithi114:> (cd / && exec bash -c ' 2024-09-13T18:22:26.344 DEBUG:teuthology.orchestra.run.smithi114:> set -e 2024-09-13T18:22:26.344 DEBUG:teuthology.orchestra.run.smithi114:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-13T18:22:26.344 DEBUG:teuthology.orchestra.run.smithi114:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-13T18:22:26.344 DEBUG:teuthology.orchestra.run.smithi114:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-13T18:22:26.345 DEBUG:teuthology.orchestra.run.smithi114:> ') 2024-09-13T18:22:26.364 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:26 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:22:26.085+0000 7fd9c57a6040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-13T18:22:26.365 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:26 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:22:26.208+0000 7fd9c57a6040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T18:22:26.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:22:26.413 DEBUG:teuthology.orchestra.run.smithi114:> ip netns list 2024-09-13T18:22:26.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:22:26.430 DEBUG:teuthology.orchestra.run.smithi114:> ip netns list-id 2024-09-13T18:22:26.481 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:22:26 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y[35685]: 2024-09-13T18:22:26.203+0000 7f5a8ee45040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T18:22:26.481 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:22:26 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y[35685]: 2024-09-13T18:22:26.265+0000 7f5a8ee45040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T18:22:26.481 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:22:26 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y[35685]: 2024-09-13T18:22:26.412+0000 7f5a8ee45040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-13T18:22:26.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:22:26.486 DEBUG:teuthology.orchestra.run.smithi114:> (cd / && exec bash -c ' 2024-09-13T18:22:26.486 DEBUG:teuthology.orchestra.run.smithi114:> set -e 2024-09-13T18:22:26.486 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T18:22:26.486 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-13T18:22:26.486 DEBUG:teuthology.orchestra.run.smithi114:> ') 2024-09-13T18:22:26.492 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:22:26 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z[36485]: 2024-09-13T18:22:26.210+0000 7f693a8bc040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T18:22:26.492 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:22:26 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z[36485]: 2024-09-13T18:22:26.273+0000 7f693a8bc040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T18:22:26.492 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:22:26 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z[36485]: 2024-09-13T18:22:26.420+0000 7f693a8bc040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-13T18:22:26.565 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-13T18:22:26.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:22:26.566 DEBUG:teuthology.orchestra.run.smithi114:> (cd / && exec bash -c ' 2024-09-13T18:22:26.566 DEBUG:teuthology.orchestra.run.smithi114:> set -e 2024-09-13T18:22:26.566 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-13T18:22:26.566 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-13T18:22:26.566 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-13T18:22:26.566 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-13T18:22:26.566 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-13T18:22:26.566 DEBUG:teuthology.orchestra.run.smithi114:> ') 2024-09-13T18:22:26.629 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:26 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:22:26.362+0000 7fd9c57a6040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T18:22:26.629 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:26 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:22:26.429+0000 7fd9c57a6040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T18:22:26.629 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:26 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:22:26.583+0000 7fd9c57a6040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-13T18:22:26.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:22:26.683 DEBUG:teuthology.orchestra.run.smithi114:> (cd / && exec bash -c ' 2024-09-13T18:22:26.683 DEBUG:teuthology.orchestra.run.smithi114:> set -e 2024-09-13T18:22:26.683 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link set brx.0 up 2024-09-13T18:22:26.683 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link set dev brx.0 master ceph-brx 2024-09-13T18:22:26.683 DEBUG:teuthology.orchestra.run.smithi114:> ') 2024-09-13T18:22:26.760 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:22:26 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y[35685]: 2024-09-13T18:22:26.479+0000 7f5a8ee45040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T18:22:26.760 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:22:26 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y[35685]: 2024-09-13T18:22:26.538+0000 7f5a8ee45040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T18:22:26.760 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:22:26 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y[35685]: 2024-09-13T18:22:26.693+0000 7f5a8ee45040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T18:22:26.762 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-09-13T18:22:26.762 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T18:22:26.762 DEBUG:teuthology.orchestra.run.smithi114:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-13T18:22:26.771 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:22:26 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z[36485]: 2024-09-13T18:22:26.488+0000 7f693a8bc040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T18:22:26.771 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:22:26 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z[36485]: 2024-09-13T18:22:26.548+0000 7f693a8bc040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T18:22:26.771 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:22:26 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z[36485]: 2024-09-13T18:22:26.703+0000 7f693a8bc040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T18:22:26.818 INFO:teuthology.orchestra.run.smithi114.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-13T18:22:26.818 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T18:22:26.818 DEBUG:teuthology.orchestra.run.smithi114:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-13T18:22:26.873 DEBUG:teuthology.orchestra.run.smithi114:> sudo modprobe fuse 2024-09-13T18:22:26.886 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:26 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:22:26.658+0000 7fd9c57a6040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T18:22:26.887 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:26 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:22:26.720+0000 7fd9c57a6040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T18:22:26.887 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:26 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:22:26.884+0000 7fd9c57a6040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T18:22:26.912 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T18:22:26.912 DEBUG:teuthology.orchestra.run.smithi114:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T18:22:26.977 INFO:teuthology.orchestra.run.smithi114.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T18:22:26.978 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T18:22:26.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:22:26.979 DEBUG:teuthology.orchestra.run.smithi114:> ls /sys/fs/fuse/connections 2024-09-13T18:22:27.024 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:22:26 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y[35685]: 2024-09-13T18:22:26.757+0000 7f5a8ee45040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T18:22:27.025 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:22:26 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y[35685]: 2024-09-13T18:22:26.823+0000 7f5a8ee45040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T18:22:27.025 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:22:26 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y[35685]: 2024-09-13T18:22:26.906+0000 7f5a8ee45040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T18:22:27.025 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:22:26 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y[35685]: 2024-09-13T18:22:26.964+0000 7f5a8ee45040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T18:22:27.035 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-13T18:22:27.035 DEBUG:teuthology.orchestra.run.smithi114:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/fc039f5d-b06e-441a-b01f-3622a83ec36a --client_fs=cephfs) 2024-09-13T18:22:27.038 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:22:26 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z[36485]: 2024-09-13T18:22:26.768+0000 7f693a8bc040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T18:22:27.038 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:22:26 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z[36485]: 2024-09-13T18:22:26.833+0000 7f693a8bc040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T18:22:27.038 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:22:26 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z[36485]: 2024-09-13T18:22:26.916+0000 7f693a8bc040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T18:22:27.038 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:22:26 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z[36485]: 2024-09-13T18:22:26.976+0000 7f693a8bc040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T18:22:27.077 DEBUG:teuthology.orchestra.run.smithi114:> sudo modprobe fuse 2024-09-13T18:22:27.102 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T18:22:27.102 DEBUG:teuthology.orchestra.run.smithi114:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T18:22:27.163 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi114.stderr:2024-09-13T18:22:27.161+0000 7f950c265480 -1 init, newargv = 0x55a1ceedd3d0 newargc=15 2024-09-13T18:22:27.164 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi114.stderr:ceph-fuse[83676]: starting ceph client 2024-09-13T18:22:27.166 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:26 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:22:26.952+0000 7fd9c57a6040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T18:22:27.166 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:27 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:22:27.021+0000 7fd9c57a6040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T18:22:27.167 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:27 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:22:27.110+0000 7fd9c57a6040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T18:22:27.169 INFO:teuthology.orchestra.run.smithi114.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T18:22:27.171 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T18:22:27.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:22:27.171 DEBUG:teuthology.orchestra.run.smithi114:> ls /sys/fs/fuse/connections 2024-09-13T18:22:27.182 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi114.stderr:ceph-fuse[83676]: starting fuse 2024-09-13T18:22:27.197 INFO:teuthology.orchestra.run.smithi114.stdout:120 2024-09-13T18:22:27.197 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [120] 2024-09-13T18:22:27.198 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T18:22:27.198 DEBUG:teuthology.orchestra.run.smithi114:> import glob 2024-09-13T18:22:27.198 DEBUG:teuthology.orchestra.run.smithi114:> import re 2024-09-13T18:22:27.198 DEBUG:teuthology.orchestra.run.smithi114:> import os 2024-09-13T18:22:27.198 DEBUG:teuthology.orchestra.run.smithi114:> import subprocess 2024-09-13T18:22:27.198 DEBUG:teuthology.orchestra.run.smithi114:> 2024-09-13T18:22:27.198 DEBUG:teuthology.orchestra.run.smithi114:> def _find_admin_socket(client_name): 2024-09-13T18:22:27.198 DEBUG:teuthology.orchestra.run.smithi114:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-13T18:22:27.198 DEBUG:teuthology.orchestra.run.smithi114:> files = glob.glob(asok_path) 2024-09-13T18:22:27.198 DEBUG:teuthology.orchestra.run.smithi114:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-13T18:22:27.199 DEBUG:teuthology.orchestra.run.smithi114:> 2024-09-13T18:22:27.199 DEBUG:teuthology.orchestra.run.smithi114:> # Given a non-glob path, it better be there 2024-09-13T18:22:27.199 DEBUG:teuthology.orchestra.run.smithi114:> if "*" not in asok_path: 2024-09-13T18:22:27.199 DEBUG:teuthology.orchestra.run.smithi114:> assert(len(files) == 1) 2024-09-13T18:22:27.199 DEBUG:teuthology.orchestra.run.smithi114:> return files[0] 2024-09-13T18:22:27.199 DEBUG:teuthology.orchestra.run.smithi114:> 2024-09-13T18:22:27.199 DEBUG:teuthology.orchestra.run.smithi114:> for f in files: 2024-09-13T18:22:27.199 DEBUG:teuthology.orchestra.run.smithi114:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-13T18:22:27.199 DEBUG:teuthology.orchestra.run.smithi114:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-13T18:22:27.199 DEBUG:teuthology.orchestra.run.smithi114:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-13T18:22:27.199 DEBUG:teuthology.orchestra.run.smithi114:> contents = proc_f.read() 2024-09-13T18:22:27.199 DEBUG:teuthology.orchestra.run.smithi114:> if mountpoint in contents: 2024-09-13T18:22:27.199 DEBUG:teuthology.orchestra.run.smithi114:> return f 2024-09-13T18:22:27.199 DEBUG:teuthology.orchestra.run.smithi114:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-13T18:22:27.199 DEBUG:teuthology.orchestra.run.smithi114:> 2024-09-13T18:22:27.200 DEBUG:teuthology.orchestra.run.smithi114:> print(_find_admin_socket("client.0")) 2024-09-13T18:22:27.200 DEBUG:teuthology.orchestra.run.smithi114:> ' 2024-09-13T18:22:27.314 INFO:teuthology.orchestra.run.smithi114.stdout:/var/run/ceph/ceph-client.0.83676.asok 2024-09-13T18:22:27.368 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:22:27 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y[35685]: 2024-09-13T18:22:27.022+0000 7f5a8ee45040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T18:22:27.444 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:22:27 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z[36485]: 2024-09-13T18:22:27.035+0000 7f693a8bc040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T18:22:27.469 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:27 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:22:27.175+0000 7fd9c57a6040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T18:22:27.470 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:27 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:22:27.239+0000 7fd9c57a6040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T18:22:27.500 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.83676.asok 2024-09-13T18:22:27.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:22:27.500 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.83676.asok status 2024-09-13T18:22:27.635 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:22:27.635 INFO:teuthology.orchestra.run.smithi114.stdout: "metadata": { 2024-09-13T18:22:27.636 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_sha1": "6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3", 2024-09-13T18:22:27.636 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_version": "ceph version 18.2.4-860-g6c6b083f (6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3) reef (stable)", 2024-09-13T18:22:27.636 INFO:teuthology.orchestra.run.smithi114.stdout: "entity_id": "0", 2024-09-13T18:22:27.636 INFO:teuthology.orchestra.run.smithi114.stdout: "hostname": "smithi114", 2024-09-13T18:22:27.636 INFO:teuthology.orchestra.run.smithi114.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-13T18:22:27.636 INFO:teuthology.orchestra.run.smithi114.stdout: "pid": "83676", 2024-09-13T18:22:27.636 INFO:teuthology.orchestra.run.smithi114.stdout: "root": "/volumes/qa/sv_0/fc039f5d-b06e-441a-b01f-3622a83ec36a" 2024-09-13T18:22:27.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:22:27.636 INFO:teuthology.orchestra.run.smithi114.stdout: "dentry_count": 0, 2024-09-13T18:22:27.636 INFO:teuthology.orchestra.run.smithi114.stdout: "dentry_pinned_count": 0, 2024-09-13T18:22:27.636 INFO:teuthology.orchestra.run.smithi114.stdout: "id": 15567, 2024-09-13T18:22:27.636 INFO:teuthology.orchestra.run.smithi114.stdout: "inst": { 2024-09-13T18:22:27.637 INFO:teuthology.orchestra.run.smithi114.stdout: "name": { 2024-09-13T18:22:27.637 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "client", 2024-09-13T18:22:27.637 INFO:teuthology.orchestra.run.smithi114.stdout: "num": 15567 2024-09-13T18:22:27.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:22:27.637 INFO:teuthology.orchestra.run.smithi114.stdout: "addr": { 2024-09-13T18:22:27.637 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "v1", 2024-09-13T18:22:27.637 INFO:teuthology.orchestra.run.smithi114.stdout: "addr": "192.168.0.1:0", 2024-09-13T18:22:27.637 INFO:teuthology.orchestra.run.smithi114.stdout: "nonce": 3288587182 2024-09-13T18:22:27.637 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:22:27.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:22:27.637 INFO:teuthology.orchestra.run.smithi114.stdout: "addr": { 2024-09-13T18:22:27.637 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "v1", 2024-09-13T18:22:27.637 INFO:teuthology.orchestra.run.smithi114.stdout: "addr": "192.168.0.1:0", 2024-09-13T18:22:27.638 INFO:teuthology.orchestra.run.smithi114.stdout: "nonce": 3288587182 2024-09-13T18:22:27.638 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:22:27.638 INFO:teuthology.orchestra.run.smithi114.stdout: "inst_str": "client.15567 192.168.0.1:0/3288587182", 2024-09-13T18:22:27.638 INFO:teuthology.orchestra.run.smithi114.stdout: "addr_str": "192.168.0.1:0/3288587182", 2024-09-13T18:22:27.638 INFO:teuthology.orchestra.run.smithi114.stdout: "inode_count": 5, 2024-09-13T18:22:27.638 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_epoch": 29, 2024-09-13T18:22:27.638 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_epoch": 93, 2024-09-13T18:22:27.638 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_epoch_barrier": 0, 2024-09-13T18:22:27.638 INFO:teuthology.orchestra.run.smithi114.stdout: "blocklisted": false, 2024-09-13T18:22:27.638 INFO:teuthology.orchestra.run.smithi114.stdout: "fs_name": "cephfs" 2024-09-13T18:22:27.638 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:22:27.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:22:27.644 DEBUG:teuthology.orchestra.run.smithi114:> stat --file-system '--printf=%T 2024-09-13T18:22:27.644 DEBUG:teuthology.orchestra.run.smithi114:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-13T18:22:27.661 INFO:teuthology.orchestra.run.smithi114.stdout:fuseblk 2024-09-13T18:22:27.661 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-13T18:22:27.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:22:27.662 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-13T18:22:27.729 INFO:teuthology.run_tasks:Running task exec... 2024-09-13T18:22:27.738 INFO:teuthology.task.exec:Executing custom commands... 2024-09-13T18:22:27.739 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi114.front.sepia.ceph.com 2024-09-13T18:22:27.739 DEBUG:teuthology.orchestra.run.smithi114:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-09-13T18:22:27.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:27.875+0000 7fc4e9f1f640 1 -- 172.21.15.114:0/440497337 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4e4103120 msgr2=0x7fc4e4105510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:27.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:27.875+0000 7fc4e9f1f640 1 --2- 172.21.15.114:0/440497337 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4e4103120 0x7fc4e4105510 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8009900 tx=0x7fc4d802a410 comp rx=0 tx=0).stop 2024-09-13T18:22:27.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:27.875+0000 7fc4e9f1f640 1 -- 172.21.15.114:0/440497337 shutdown_connections 2024-09-13T18:22:27.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:27.875+0000 7fc4e9f1f640 1 --2- 172.21.15.114:0/440497337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4e4105a50 0x7fc4e4107ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:27.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:27.875+0000 7fc4e9f1f640 1 --2- 172.21.15.114:0/440497337 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4e4103120 0x7fc4e4105510 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:27.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:27.875+0000 7fc4e9f1f640 1 --2- 172.21.15.114:0/440497337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4e41007f0 0x7fc4e4102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:27.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:27.875+0000 7fc4e9f1f640 1 -- 172.21.15.114:0/440497337 >> 172.21.15.114:0/440497337 conn(0x7fc4e40fa3d0 msgr2=0x7fc4e40fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:27.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:27.875+0000 7fc4e9f1f640 1 -- 172.21.15.114:0/440497337 shutdown_connections 2024-09-13T18:22:27.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:27.875+0000 7fc4e9f1f640 1 -- 172.21.15.114:0/440497337 wait complete. 2024-09-13T18:22:27.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:27.876+0000 7fc4e9f1f640 1 Processor -- start 2024-09-13T18:22:27.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:27.876+0000 7fc4e9f1f640 1 -- start start 2024-09-13T18:22:27.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:27.876+0000 7fc4e9f1f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4e41007f0 0x7fc4e4195d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:27.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:27.876+0000 7fc4e9f1f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4e4103120 0x7fc4e4196250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:27.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:27.876+0000 7fc4e9f1f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4e4105a50 0x7fc4e419d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:27.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:27.876+0000 7fc4e9f1f640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e419d7f0 con 0x7fc4e4103120 2024-09-13T18:22:27.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:27.876+0000 7fc4e9f1f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e419d960 con 0x7fc4e41007f0 2024-09-13T18:22:27.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:27.876+0000 7fc4e9f1f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e419db20 con 0x7fc4e4105a50 2024-09-13T18:22:27.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:27.876+0000 7fc4e2ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4e4103120 0x7fc4e4196250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:27.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:27.876+0000 7fc4e2ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4e4103120 0x7fc4e4196250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33062/0 (socket says 172.21.15.114:33062) 2024-09-13T18:22:27.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:27.876+0000 7fc4e2ffd640 1 -- 172.21.15.114:0/2695892655 learned_addr learned my addr 172.21.15.114:0/2695892655 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:27.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:27.876+0000 7fc4e37fe640 1 --2- 172.21.15.114:0/2695892655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4e41007f0 0x7fc4e4195d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:27.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:27.876+0000 7fc4e3fff640 1 --2- 172.21.15.114:0/2695892655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4e4105a50 0x7fc4e419d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:27.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:27.876+0000 7fc4e2ffd640 1 -- 172.21.15.114:0/2695892655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4e4105a50 msgr2=0x7fc4e419d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:27.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:27.877+0000 7fc4e2ffd640 1 --2- 172.21.15.114:0/2695892655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4e4105a50 0x7fc4e419d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:27.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:27.877+0000 7fc4e2ffd640 1 -- 172.21.15.114:0/2695892655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4e41007f0 msgr2=0x7fc4e4195d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:27.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:27.877+0000 7fc4e2ffd640 1 --2- 172.21.15.114:0/2695892655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4e41007f0 0x7fc4e4195d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:27.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:27.877+0000 7fc4e2ffd640 1 -- 172.21.15.114:0/2695892655 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4dc009840 con 0x7fc4e4103120 2024-09-13T18:22:27.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:27.877+0000 7fc4e2ffd640 1 --2- 172.21.15.114:0/2695892655 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4e4103120 0x7fc4e4196250 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8005ab0 tx=0x7fc4d80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:27.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:27.877+0000 7fc4e0ff9640 1 -- 172.21.15.114:0/2695892655 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d8031d70 con 0x7fc4e4103120 2024-09-13T18:22:27.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:27.877+0000 7fc4e0ff9640 1 -- 172.21.15.114:0/2695892655 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc4d8005490 con 0x7fc4e4103120 2024-09-13T18:22:27.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:27.877+0000 7fc4e9f1f640 1 -- 172.21.15.114:0/2695892655 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4d8009560 con 0x7fc4e4103120 2024-09-13T18:22:27.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:27.877+0000 7fc4e0ff9640 1 -- 172.21.15.114:0/2695892655 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d80038e0 con 0x7fc4e4103120 2024-09-13T18:22:27.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:27.877+0000 7fc4e9f1f640 1 -- 172.21.15.114:0/2695892655 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4e419e0c0 con 0x7fc4e4103120 2024-09-13T18:22:27.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:27.878+0000 7fc4e9f1f640 1 -- 172.21.15.114:0/2695892655 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4e40fbec0 con 0x7fc4e4103120 2024-09-13T18:22:27.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:27.879+0000 7fc4e0ff9640 1 -- 172.21.15.114:0/2695892655 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 20) v1 ==== 135891+0+0 (secure 0 0 0) 0x7fc4d80046c0 con 0x7fc4e4103120 2024-09-13T18:22:27.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:27.879+0000 7fc4e0ff9640 1 --2- 172.21.15.114:0/2695892655 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fc4cc0a10e0 0x7fc4cc0a35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:27.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:27.880+0000 7fc4e37fe640 1 -- 172.21.15.114:0/2695892655 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fc4cc0a10e0 msgr2=0x7fc4cc0a35a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.114:6800/3396185541 2024-09-13T18:22:27.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:27.880+0000 7fc4e0ff9640 1 -- 172.21.15.114:0/2695892655 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fc4d80edbd0 con 0x7fc4e4103120 2024-09-13T18:22:27.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:27.880+0000 7fc4e37fe640 1 --2- 172.21.15.114:0/2695892655 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fc4cc0a10e0 0x7fc4cc0a35a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:22:27.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:27.883+0000 7fc4e0ff9640 1 -- 172.21.15.114:0/2695892655 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fc4d80b6f60 con 0x7fc4e4103120 2024-09-13T18:22:28.058 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:27 smithi114 ceph-mon[30897]: Standby manager daemon y restarted 2024-09-13T18:22:28.058 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:27 smithi114 ceph-mon[30897]: Standby manager daemon y started 2024-09-13T18:22:28.058 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:27 smithi114 ceph-mon[30897]: from='mgr.? 172.21.15.134:0/2099523346' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-13T18:22:28.058 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:27 smithi114 ceph-mon[30897]: from='mgr.? 172.21.15.134:0/2099523346' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-13T18:22:28.058 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:27 smithi114 ceph-mon[30897]: from='mgr.? 172.21.15.134:0/2099523346' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-13T18:22:28.058 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:27 smithi114 ceph-mon[30897]: from='mgr.? 172.21.15.134:0/2099523346' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-13T18:22:28.058 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:27 smithi114 ceph-mon[30897]: Standby manager daemon z restarted 2024-09-13T18:22:28.058 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:27 smithi114 ceph-mon[30897]: Standby manager daemon z started 2024-09-13T18:22:28.058 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:27 smithi114 ceph-mon[30897]: from='mgr.? 172.21.15.204:0/893446787' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-13T18:22:28.058 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:27 smithi114 ceph-mon[30897]: from='mgr.? 172.21.15.204:0/893446787' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-13T18:22:28.059 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:27 smithi114 ceph-mon[30897]: from='mgr.? 172.21.15.204:0/893446787' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-13T18:22:28.059 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:27 smithi114 ceph-mon[30897]: from='mgr.? 172.21.15.204:0/893446787' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-13T18:22:28.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:28.072+0000 7fc4e0ff9640 1 -- 172.21.15.114:0/2695892655 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mgrmap(e 21) v1 ==== 135558+0+0 (secure 0 0 0) 0x7fc4d80b7330 con 0x7fc4e4103120 2024-09-13T18:22:28.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:28.073+0000 7fc4e0ff9640 1 -- 172.21.15.114:0/2695892655 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fc4cc0a10e0 msgr2=0x7fc4cc0a35a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:22:28.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:28.073+0000 7fc4e0ff9640 1 --2- 172.21.15.114:0/2695892655 >> [v2:172.21.15.114:6800/3396185541,v1:172.21.15.114:6801/3396185541] conn(0x7fc4cc0a10e0 0x7fc4cc0a35a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:28.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:28.096+0000 7fc4e9f1f640 1 -- 172.21.15.114:0/2695892655 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) v1 -- 0x7fc4e410b4f0 con 0x7fc4e4103120 2024-09-13T18:22:28.119 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:22:27 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y[35685]: 2024-09-13T18:22:27.806+0000 7f5a8ee45040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T18:22:28.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:27 smithi134 ceph-mon[34283]: Standby manager daemon y restarted 2024-09-13T18:22:28.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:27 smithi134 ceph-mon[34283]: Standby manager daemon y started 2024-09-13T18:22:28.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:27 smithi134 ceph-mon[34283]: from='mgr.? 172.21.15.134:0/2099523346' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-13T18:22:28.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:27 smithi134 ceph-mon[34283]: from='mgr.? 172.21.15.134:0/2099523346' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-13T18:22:28.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:27 smithi134 ceph-mon[34283]: from='mgr.? 172.21.15.134:0/2099523346' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-13T18:22:28.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:27 smithi134 ceph-mon[34283]: from='mgr.? 172.21.15.134:0/2099523346' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-13T18:22:28.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:27 smithi134 ceph-mon[34283]: Standby manager daemon z restarted 2024-09-13T18:22:28.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:27 smithi134 ceph-mon[34283]: Standby manager daemon z started 2024-09-13T18:22:28.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:27 smithi134 ceph-mon[34283]: from='mgr.? 172.21.15.204:0/893446787' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-13T18:22:28.120 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:27 smithi134 ceph-mon[34283]: from='mgr.? 172.21.15.204:0/893446787' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-13T18:22:28.120 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:27 smithi134 ceph-mon[34283]: from='mgr.? 172.21.15.204:0/893446787' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-13T18:22:28.120 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:27 smithi134 ceph-mon[34283]: from='mgr.? 172.21.15.204:0/893446787' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-13T18:22:28.194 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:22:27 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z[36485]: 2024-09-13T18:22:27.820+0000 7f693a8bc040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T18:22:28.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:27 smithi204 ceph-mon[34777]: Standby manager daemon y restarted 2024-09-13T18:22:28.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:27 smithi204 ceph-mon[34777]: Standby manager daemon y started 2024-09-13T18:22:28.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:27 smithi204 ceph-mon[34777]: from='mgr.? 172.21.15.134:0/2099523346' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-13T18:22:28.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:27 smithi204 ceph-mon[34777]: from='mgr.? 172.21.15.134:0/2099523346' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-13T18:22:28.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:27 smithi204 ceph-mon[34777]: from='mgr.? 172.21.15.134:0/2099523346' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-13T18:22:28.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:27 smithi204 ceph-mon[34777]: from='mgr.? 172.21.15.134:0/2099523346' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-13T18:22:28.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:27 smithi204 ceph-mon[34777]: Standby manager daemon z restarted 2024-09-13T18:22:28.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:27 smithi204 ceph-mon[34777]: Standby manager daemon z started 2024-09-13T18:22:28.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:27 smithi204 ceph-mon[34777]: from='mgr.? 172.21.15.204:0/893446787' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-13T18:22:28.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:27 smithi204 ceph-mon[34777]: from='mgr.? 172.21.15.204:0/893446787' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-13T18:22:28.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:27 smithi204 ceph-mon[34777]: from='mgr.? 172.21.15.204:0/893446787' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-13T18:22:28.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:27 smithi204 ceph-mon[34777]: from='mgr.? 172.21.15.204:0/893446787' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-13T18:22:28.397 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:22:28.055+0000 7fd9c57a6040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T18:22:28.398 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:22:28.279+0000 7fd92e275640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T18:22:28.398 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:22:28.279+0000 7fd92e275640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T18:22:28.398 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:22:28.279+0000 7fd92e275640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T18:22:28.398 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:22:28.279+0000 7fd92e275640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T18:22:28.398 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:22:28.279+0000 7fd92e275640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T18:22:28.398 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:22:28.279+0000 7fd93127b640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T18:22:28.398 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:22:28.279+0000 7fd93127b640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T18:22:28.398 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:22:28.279+0000 7fd93127b640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T18:22:28.398 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:22:28.279+0000 7fd93127b640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T18:22:28.398 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:22:28.281+0000 7fd93127b640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T18:22:29.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.075+0000 7fc4e0ff9640 1 -- 172.21.15.114:0/2695892655 <== mon.0 v2:172.21.15.114:3300/0 8 ==== mgrmap(e 22) v1 ==== 135685+0+0 (secure 0 0 0) 0x7fc4d80eb3b0 con 0x7fc4e4103120 2024-09-13T18:22:29.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.076+0000 7fc4e0ff9640 1 --2- 172.21.15.114:0/2695892655 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc4cc0509a0 0x7fc4cc050d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:29.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.078+0000 7fc4e37fe640 1 --2- 172.21.15.114:0/2695892655 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc4cc0509a0 0x7fc4cc050d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:29.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.081+0000 7fc4e37fe640 1 --2- 172.21.15.114:0/2695892655 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc4cc0509a0 0x7fc4cc050d80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc4e41971e0 tx=0x7fc4d4005f90 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:29.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.083+0000 7fc4e0ff9640 1 -- 172.21.15.114:0/2695892655 <== mon.0 v2:172.21.15.114:3300/0 9 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v30) v1 ==== 117+0+0 (secure 0 0 0) 0x7fc4d80b76f0 con 0x7fc4e4103120 2024-09-13T18:22:29.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.086+0000 7fc4e9f1f640 1 -- 172.21.15.114:0/2695892655 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc4cc0509a0 msgr2=0x7fc4cc050d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:29.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.086+0000 7fc4e9f1f640 1 --2- 172.21.15.114:0/2695892655 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc4cc0509a0 0x7fc4cc050d80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc4e41971e0 tx=0x7fc4d4005f90 comp rx=0 tx=0).stop 2024-09-13T18:22:29.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.086+0000 7fc4e9f1f640 1 -- 172.21.15.114:0/2695892655 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4e4103120 msgr2=0x7fc4e4196250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:29.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.086+0000 7fc4e9f1f640 1 --2- 172.21.15.114:0/2695892655 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4e4103120 0x7fc4e4196250 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8005ab0 tx=0x7fc4d80042f0 comp rx=0 tx=0).stop 2024-09-13T18:22:29.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.086+0000 7fc4e3fff640 1 -- 172.21.15.114:0/2695892655 reap_dead start 2024-09-13T18:22:29.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.086+0000 7fc4e9f1f640 1 -- 172.21.15.114:0/2695892655 shutdown_connections 2024-09-13T18:22:29.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.086+0000 7fc4e9f1f640 1 -- 172.21.15.114:0/2695892655 >> 172.21.15.114:0/2695892655 conn(0x7fc4e40fa3d0 msgr2=0x7fc4e4101050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:29.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.086+0000 7fc4e9f1f640 1 -- 172.21.15.114:0/2695892655 shutdown_connections 2024-09-13T18:22:29.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.087+0000 7fc4e9f1f640 1 -- 172.21.15.114:0/2695892655 wait complete. 2024-09-13T18:22:29.101 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-13T18:22:29.112 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-13T18:22:29.133 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:22:29.139 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:28 smithi134 ceph-mon[34283]: mgrmap e20: x(active, since 9m), standbys: y, z 2024-09-13T18:22:29.139 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:28 smithi134 ceph-mon[34283]: Active manager daemon x restarted 2024-09-13T18:22:29.139 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:28 smithi134 ceph-mon[34283]: Activating manager daemon x 2024-09-13T18:22:29.140 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:28 smithi134 ceph-mon[34283]: osdmap e94: 12 total, 12 up, 12 in 2024-09-13T18:22:29.140 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:28 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T18:22:29.140 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:28 smithi134 ceph-mon[34283]: mgrmap e21: x(active, starting, since 0.0169593s), standbys: y, z 2024-09-13T18:22:29.140 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:28 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T18:22:29.140 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:28 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T18:22:29.140 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:28 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-13T18:22:29.140 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:28 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-13T18:22:29.140 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:28 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-13T18:22:29.140 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:28 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-13T18:22:29.140 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:28 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-13T18:22:29.140 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:28 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-13T18:22:29.140 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:28 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-13T18:22:29.141 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:28 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-13T18:22:29.141 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:28 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-13T18:22:29.141 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:28 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-13T18:22:29.141 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:28 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-13T18:22:29.141 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:28 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-13T18:22:29.141 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:28 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-13T18:22:29.141 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:28 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-13T18:22:29.141 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:28 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-13T18:22:29.141 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:28 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T18:22:29.141 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:28 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T18:22:29.141 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:28 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T18:22:29.141 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:28 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-13T18:22:29.141 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:28 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-13T18:22:29.141 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:28 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T18:22:29.142 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:28 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T18:22:29.142 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:28 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-13T18:22:29.142 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:28 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T18:22:29.142 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:28 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-13T18:22:29.142 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:28 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T18:22:29.142 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:28 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T18:22:29.142 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:28 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-13T18:22:29.142 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:28 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-13T18:22:29.142 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:28 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-13T18:22:29.142 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:28 smithi134 ceph-mon[34283]: Manager daemon x is now available 2024-09-13T18:22:29.142 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:28 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/2695892655' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-13T18:22:29.142 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:28 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:22:29.142 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:28 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-13T18:22:29.142 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:28 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-13T18:22:29.143 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:28 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-13T18:22:29.143 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:28 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-13T18:22:29.157 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-mon[30897]: mgrmap e20: x(active, since 9m), standbys: y, z 2024-09-13T18:22:29.157 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-mon[30897]: Active manager daemon x restarted 2024-09-13T18:22:29.157 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-mon[30897]: Activating manager daemon x 2024-09-13T18:22:29.157 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-mon[30897]: osdmap e94: 12 total, 12 up, 12 in 2024-09-13T18:22:29.157 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T18:22:29.158 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-mon[30897]: mgrmap e21: x(active, starting, since 0.0169593s), standbys: y, z 2024-09-13T18:22:29.158 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T18:22:29.158 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T18:22:29.158 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-13T18:22:29.158 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-13T18:22:29.158 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-13T18:22:29.158 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-13T18:22:29.158 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-13T18:22:29.158 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-13T18:22:29.158 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-13T18:22:29.158 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-13T18:22:29.158 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-13T18:22:29.158 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-13T18:22:29.158 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-13T18:22:29.159 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-13T18:22:29.159 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-13T18:22:29.159 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-13T18:22:29.159 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-13T18:22:29.159 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T18:22:29.159 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T18:22:29.159 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T18:22:29.159 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-13T18:22:29.159 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-13T18:22:29.159 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T18:22:29.159 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T18:22:29.159 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-13T18:22:29.159 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T18:22:29.159 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-13T18:22:29.160 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T18:22:29.160 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T18:22:29.160 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-13T18:22:29.160 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-13T18:22:29.160 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-13T18:22:29.160 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-mon[30897]: Manager daemon x is now available 2024-09-13T18:22:29.160 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2695892655' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-13T18:22:29.160 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:22:29.160 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-13T18:22:29.161 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-13T18:22:29.161 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-13T18:22:29.161 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:28 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-13T18:22:29.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:28 smithi204 ceph-mon[34777]: mgrmap e20: x(active, since 9m), standbys: y, z 2024-09-13T18:22:29.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:28 smithi204 ceph-mon[34777]: Active manager daemon x restarted 2024-09-13T18:22:29.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:28 smithi204 ceph-mon[34777]: Activating manager daemon x 2024-09-13T18:22:29.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:28 smithi204 ceph-mon[34777]: osdmap e94: 12 total, 12 up, 12 in 2024-09-13T18:22:29.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:28 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-13T18:22:29.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:28 smithi204 ceph-mon[34777]: mgrmap e21: x(active, starting, since 0.0169593s), standbys: y, z 2024-09-13T18:22:29.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:28 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-13T18:22:29.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:28 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-13T18:22:29.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:28 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-13T18:22:29.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:28 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-13T18:22:29.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:28 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-13T18:22:29.196 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:28 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-13T18:22:29.196 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:28 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-13T18:22:29.196 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:28 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-13T18:22:29.196 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:28 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-13T18:22:29.196 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:28 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-13T18:22:29.196 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:28 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-13T18:22:29.196 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:28 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-13T18:22:29.196 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:28 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-13T18:22:29.196 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:28 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-13T18:22:29.196 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:28 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-13T18:22:29.196 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:28 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-13T18:22:29.196 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:28 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-13T18:22:29.197 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:28 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-13T18:22:29.197 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:28 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-13T18:22:29.197 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:28 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-13T18:22:29.197 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:28 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-13T18:22:29.197 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:28 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-13T18:22:29.197 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:28 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-13T18:22:29.197 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:28 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-13T18:22:29.197 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:28 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-13T18:22:29.197 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:28 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-13T18:22:29.197 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:28 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-13T18:22:29.197 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:28 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-13T18:22:29.197 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:28 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-13T18:22:29.197 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:28 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-13T18:22:29.197 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:28 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-13T18:22:29.198 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:28 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-13T18:22:29.198 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:28 smithi204 ceph-mon[34777]: Manager daemon x is now available 2024-09-13T18:22:29.198 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:28 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/2695892655' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-13T18:22:29.198 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:28 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:22:29.198 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:28 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-13T18:22:29.198 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:28 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-13T18:22:29.198 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:28 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-13T18:22:29.198 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:28 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-13T18:22:29.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.236+0000 7f43c806b640 1 -- 172.21.15.114:0/4090773990 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43c0103150 msgr2=0x7f43c0105540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:29.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.236+0000 7f43c806b640 1 --2- 172.21.15.114:0/4090773990 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43c0103150 0x7f43c0105540 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f43b0009900 tx=0x7f43b002a410 comp rx=0 tx=0).stop 2024-09-13T18:22:29.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.236+0000 7f43c806b640 1 -- 172.21.15.114:0/4090773990 shutdown_connections 2024-09-13T18:22:29.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.236+0000 7f43c806b640 1 --2- 172.21.15.114:0/4090773990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43c0105a80 0x7f43c0107ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:29.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.236+0000 7f43c806b640 1 --2- 172.21.15.114:0/4090773990 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43c0103150 0x7f43c0105540 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:29.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.236+0000 7f43c806b640 1 --2- 172.21.15.114:0/4090773990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43c0100820 0x7f43c0102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:29.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.236+0000 7f43c806b640 1 -- 172.21.15.114:0/4090773990 >> 172.21.15.114:0/4090773990 conn(0x7f43c00fa440 msgr2=0x7f43c00fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:29.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.236+0000 7f43c806b640 1 -- 172.21.15.114:0/4090773990 shutdown_connections 2024-09-13T18:22:29.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.236+0000 7f43c806b640 1 -- 172.21.15.114:0/4090773990 wait complete. 2024-09-13T18:22:29.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.236+0000 7f43c806b640 1 Processor -- start 2024-09-13T18:22:29.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.236+0000 7f43c806b640 1 -- start start 2024-09-13T18:22:29.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.237+0000 7f43c806b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43c0100820 0x7f43c0195d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:29.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.237+0000 7f43c806b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43c0103150 0x7f43c0196260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:29.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.237+0000 7f43c806b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43c0105a80 0x7f43c019d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:29.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.237+0000 7f43c806b640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c019d800 con 0x7f43c0100820 2024-09-13T18:22:29.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.237+0000 7f43c806b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c019d970 con 0x7f43c0105a80 2024-09-13T18:22:29.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.237+0000 7f43c806b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c019db30 con 0x7f43c0103150 2024-09-13T18:22:29.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.237+0000 7f43c5e07640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43c0100820 0x7f43c0195d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:29.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.237+0000 7f43c5e07640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43c0100820 0x7f43c0195d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33202/0 (socket says 172.21.15.114:33202) 2024-09-13T18:22:29.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.237+0000 7f43c5e07640 1 -- 172.21.15.114:0/3021363049 learned_addr learned my addr 172.21.15.114:0/3021363049 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:29.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.237+0000 7f43c5606640 1 --2- 172.21.15.114:0/3021363049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43c0103150 0x7f43c0196260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:29.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.237+0000 7f43c6608640 1 --2- 172.21.15.114:0/3021363049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43c0105a80 0x7f43c019d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:29.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.237+0000 7f43c5e07640 1 -- 172.21.15.114:0/3021363049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43c0103150 msgr2=0x7f43c0196260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:29.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.237+0000 7f43c5e07640 1 --2- 172.21.15.114:0/3021363049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43c0103150 0x7f43c0196260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:29.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.237+0000 7f43c5e07640 1 -- 172.21.15.114:0/3021363049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43c0105a80 msgr2=0x7f43c019d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:29.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.237+0000 7f43c5e07640 1 --2- 172.21.15.114:0/3021363049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43c0105a80 0x7f43c019d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:29.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.237+0000 7f43c5e07640 1 -- 172.21.15.114:0/3021363049 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b0009560 con 0x7f43c0100820 2024-09-13T18:22:29.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.237+0000 7f43c5e07640 1 --2- 172.21.15.114:0/3021363049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43c0100820 0x7f43c0195d20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f43bc00d770 tx=0x7f43bc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:29.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.237+0000 7f43b6ffd640 1 -- 172.21.15.114:0/3021363049 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43bc009c30 con 0x7f43c0100820 2024-09-13T18:22:29.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.237+0000 7f43c806b640 1 -- 172.21.15.114:0/3021363049 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43c019de10 con 0x7f43c0100820 2024-09-13T18:22:29.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.237+0000 7f43b6ffd640 1 -- 172.21.15.114:0/3021363049 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f43bc00de80 con 0x7f43c0100820 2024-09-13T18:22:29.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.237+0000 7f43b6ffd640 1 -- 172.21.15.114:0/3021363049 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43bc004d40 con 0x7f43c0100820 2024-09-13T18:22:29.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.237+0000 7f43c806b640 1 -- 172.21.15.114:0/3021363049 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43c019e230 con 0x7f43c0100820 2024-09-13T18:22:29.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.238+0000 7f43c806b640 1 -- 172.21.15.114:0/3021363049 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43c00fbf30 con 0x7f43c0100820 2024-09-13T18:22:29.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.240+0000 7f43b6ffd640 1 -- 172.21.15.114:0/3021363049 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 22) v1 ==== 135685+0+0 (secure 0 0 0) 0x7f43bc004ea0 con 0x7f43c0100820 2024-09-13T18:22:29.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.240+0000 7f43b6ffd640 1 --2- 172.21.15.114:0/3021363049 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f43ac0a0e90 0x7f43ac0a3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:29.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.240+0000 7f43b6ffd640 1 -- 172.21.15.114:0/3021363049 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f43bc0cc7b0 con 0x7f43c0100820 2024-09-13T18:22:29.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.240+0000 7f43c5606640 1 --2- 172.21.15.114:0/3021363049 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f43ac0a0e90 0x7f43ac0a3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:29.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.241+0000 7f43c5606640 1 --2- 172.21.15.114:0/3021363049 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f43ac0a0e90 0x7f43ac0a3350 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f43b00057c0 tx=0x7f43b0005a20 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:29.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.244+0000 7f43b6ffd640 1 -- 172.21.15.114:0/3021363049 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f43bc095d30 con 0x7f43c0100820 2024-09-13T18:22:29.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.415+0000 7f43c806b640 1 -- 172.21.15.114:0/3021363049 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f43c010b540 con 0x7f43c0100820 2024-09-13T18:22:29.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.417+0000 7f43b6ffd640 1 -- 172.21.15.114:0/3021363049 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+25079 (secure 0 0 0) 0x7f43bc017070 con 0x7f43c0100820 2024-09-13T18:22:29.420 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:22:29.420 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":94,"fsid":"97e7edf2-71fb-11ef-bceb-c7b262605968","created":"2024-09-13T18:12:36.360625+0000","modified":"2024-09-13T18:22:28.057664+0000","last_up_change":"2024-09-13T18:19:12.350370+0000","last_in_change":"2024-09-13T18:18:52.321596+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:15:40.129659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:20:00.024407+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:20:01.085549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T18:20:04.334216+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9914f01a-9f6f-4878-b955-83570199c354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6803","nonce":3499525826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6805","nonce":3499525826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6809","nonce":3499525826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6807","nonce":3499525826}]},"public_addr":"172.21.15.114:6803/3499525826","cluster_addr":"172.21.15.114:6805/3499525826","heartbeat_back_addr":"172.21.15.114:6809/3499525826","heartbeat_front_addr":"172.21.15.114:6807/3499525826","state":["exists","up"]},{"osd":1,"uuid":"ef65e012-7469-4c20-b354-17022bca63c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6801","nonce":4264264866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6803","nonce":4264264866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6807","nonce":4264264866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6805","nonce":4264264866}]},"public_addr":"172.21.15.134:6801/4264264866","cluster_addr":"172.21.15.134:6803/4264264866","heartbeat_back_addr":"172.21.15.134:6807/4264264866","heartbeat_front_addr":"172.21.15.134:6805/4264264866","state":["exists","up"]},{"osd":2,"uuid":"578213fd-715f-4be7-91fa-254f3d0e3022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6801","nonce":2288807707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6803","nonce":2288807707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6807","nonce":2288807707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6805","nonce":2288807707}]},"public_addr":"172.21.15.204:6801/2288807707","cluster_addr":"172.21.15.204:6803/2288807707","heartbeat_back_addr":"172.21.15.204:6807/2288807707","heartbeat_front_addr":"172.21.15.204:6805/2288807707","state":["exists","up"]},{"osd":3,"uuid":"b64aab3f-1e52-4828-8397-74af5e05f2cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6811","nonce":2904204254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6813","nonce":2904204254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6817","nonce":2904204254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6815","nonce":2904204254}]},"public_addr":"172.21.15.114:6811/2904204254","cluster_addr":"172.21.15.114:6813/2904204254","heartbeat_back_addr":"172.21.15.114:6817/2904204254","heartbeat_front_addr":"172.21.15.114:6815/2904204254","state":["exists","up"]},{"osd":4,"uuid":"f38943d6-39c0-42e1-b427-cb6bb90ac38a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6809","nonce":1773945333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6811","nonce":1773945333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6815","nonce":1773945333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6813","nonce":1773945333}]},"public_addr":"172.21.15.134:6809/1773945333","cluster_addr":"172.21.15.134:6811/1773945333","heartbeat_back_addr":"172.21.15.134:6815/1773945333","heartbeat_front_addr":"172.21.15.134:6813/1773945333","state":["exists","up"]},{"osd":5,"uuid":"bb38f361-8e00-4178-9752-5ddbf3af20e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6809","nonce":2371778055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6811","nonce":2371778055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6815","nonce":2371778055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6813","nonce":2371778055}]},"public_addr":"172.21.15.204:6809/2371778055","cluster_addr":"172.21.15.204:6811/2371778055","heartbeat_back_addr":"172.21.15.204:6815/2371778055","heartbeat_front_addr":"172.21.15.204:6813/2371778055","state":["exists","up"]},{"osd":6,"uuid":"93d40a03-e7ff-4a3f-9efa-5c5403c9983d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6819","nonce":1484325044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6821","nonce":1484325044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6825","nonce":1484325044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6823","nonce":1484325044}]},"public_addr":"172.21.15.114:6819/1484325044","cluster_addr":"172.21.15.114:6821/1484325044","heartbeat_back_addr":"172.21.15.114:6825/1484325044","heartbeat_front_addr":"172.21.15.114:6823/1484325044","state":["exists","up"]},{"osd":7,"uuid":"982f629a-1b95-4be1-8e75-5ddce1ad77b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6817","nonce":799246614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6819","nonce":799246614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6823","nonce":799246614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6821","nonce":799246614}]},"public_addr":"172.21.15.134:6817/799246614","cluster_addr":"172.21.15.134:6819/799246614","heartbeat_back_addr":"172.21.15.134:6823/799246614","heartbeat_front_addr":"172.21.15.134:6821/799246614","state":["exists","up"]},{"osd":8,"uuid":"409beb9b-8227-4dad-a4a0-f0e5ce70dc5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6817","nonce":858331805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6819","nonce":858331805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6823","nonce":858331805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6821","nonce":858331805}]},"public_addr":"172.21.15.204:6817/858331805","cluster_addr":"172.21.15.204:6819/858331805","heartbeat_back_addr":"172.21.15.204:6823/858331805","heartbeat_front_addr":"172.21.15.204:6821/858331805","state":["exists","up"]},{"osd":9,"uuid":"a345a15c-b1fc-4877-b445-418eebf0d0ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6827","nonce":2172392854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6829","nonce":2172392854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6833","nonce":2172392854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6831","nonce":2172392854}]},"public_addr":"172.21.15.114:6827/2172392854","cluster_addr":"172.21.15.114:6829/2172392854","heartbeat_back_addr":"172.21.15.114:6833/2172392854","heartbeat_front_addr":"172.21.15.114:6831/2172392854","state":["exists","up"]},{"osd":10,"uuid":"464d995f-d90b-4caa-addb-1f7562bd74b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6825","nonce":3073513468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6827","nonce":3073513468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6831","nonce":3073513468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6829","nonce":3073513468}]},"public_addr":"172.21.15.134:6825/3073513468","cluster_addr":"172.21.15.134:6827/3073513468","heartbeat_back_addr":"172.21.15.134:6831/3073513468","heartbeat_front_addr":"172.21.15.134:6829/3073513468","state":["exists","up"]},{"osd":11,"uuid":"71dd794d-326c-477f-8d02-ee32bff1350d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6825","nonce":3239521765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6827","nonce":3239521765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6831","nonce":3239521765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6829","nonce":3239521765}]},"public_addr":"172.21.15.204:6825/3239521765","cluster_addr":"172.21.15.204:6827/3239521765","heartbeat_back_addr":"172.21.15.204:6831/3239521765","heartbeat_front_addr":"172.21.15.204:6829/3239521765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:14:54.542687+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:15:15.746055+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:15:38.371131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:01.383932+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:24.796237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:48.422214+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:11.284323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:34.570365+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:58.232855+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:18:21.609596+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:18:45.565859+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:19:10.017969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:6801/3396185541":"2024-09-14T18:22:28.057623+0000","172.21.15.114:6800/830738794":"2024-09-14T18:13:03.521116+0000","172.21.15.114:6801/830738794":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/1684954293":"2024-09-14T18:22:28.057623+0000","172.21.15.114:0/3544951322":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/722571999":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/2949422849":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2034742251":"2024-09-14T18:13:03.521116+0000","172.21.15.114:6800/4281866858":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/3407523320":"2024-09-14T18:22:28.057623+0000","172.21.15.114:6800/3396185541":"2024-09-14T18:22:28.057623+0000","172.21.15.114:0/4257828906":"2024-09-14T18:13:25.823027+0000","172.21.15.114:6801/4281866858":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2828364892":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2714091073":"2024-09-14T18:22:28.057623+0000","172.21.15.114:0/842237781":"2024-09-14T18:22:28.057623+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:22:29.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.420+0000 7f43c806b640 1 -- 172.21.15.114:0/3021363049 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f43ac0a0e90 msgr2=0x7f43ac0a3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:29.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.420+0000 7f43c806b640 1 --2- 172.21.15.114:0/3021363049 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f43ac0a0e90 0x7f43ac0a3350 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f43b00057c0 tx=0x7f43b0005a20 comp rx=0 tx=0).stop 2024-09-13T18:22:29.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.420+0000 7f43c806b640 1 -- 172.21.15.114:0/3021363049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43c0100820 msgr2=0x7f43c0195d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:29.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.420+0000 7f43c806b640 1 --2- 172.21.15.114:0/3021363049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43c0100820 0x7f43c0195d20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f43bc00d770 tx=0x7f43bc00dc30 comp rx=0 tx=0).stop 2024-09-13T18:22:29.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.420+0000 7f43c806b640 1 -- 172.21.15.114:0/3021363049 shutdown_connections 2024-09-13T18:22:29.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.420+0000 7f43c806b640 1 --2- 172.21.15.114:0/3021363049 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f43ac0a0e90 0x7f43ac0a3350 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:29.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.420+0000 7f43c806b640 1 --2- 172.21.15.114:0/3021363049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43c0105a80 0x7f43c019d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:29.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.420+0000 7f43c806b640 1 --2- 172.21.15.114:0/3021363049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43c0103150 0x7f43c0196260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:29.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.420+0000 7f43c806b640 1 --2- 172.21.15.114:0/3021363049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43c0100820 0x7f43c0195d20 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:29.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.420+0000 7f43c806b640 1 -- 172.21.15.114:0/3021363049 >> 172.21.15.114:0/3021363049 conn(0x7f43c00fa440 msgr2=0x7f43c0101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:29.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.420+0000 7f43c806b640 1 -- 172.21.15.114:0/3021363049 shutdown_connections 2024-09-13T18:22:29.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.420+0000 7f43c806b640 1 -- 172.21.15.114:0/3021363049 wait complete. 2024-09-13T18:22:29.425 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:29 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:22:29.215+0000 7fd91a50e640 -1 log_channel(cephadm) log [ERR] : [13/Sep/2024:18:22:29] ENGINE Error in HTTPServer.serve 2024-09-13T18:22:29.426 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:29 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: Traceback (most recent call last): 2024-09-13T18:22:29.426 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:29 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-13T18:22:29.426 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:29 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: self._connections.run(self.expiration_interval) 2024-09-13T18:22:29.426 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:29 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-13T18:22:29.426 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:29 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: self._run(expiration_interval) 2024-09-13T18:22:29.426 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:29 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-13T18:22:29.426 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:29 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: new_conn = self._from_server_socket(self.server.socket) 2024-09-13T18:22:29.426 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:29 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-13T18:22:29.426 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:29 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-13T18:22:29.426 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:29 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-13T18:22:29.426 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:29 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: s = self.context.wrap_socket( 2024-09-13T18:22:29.426 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:29 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-13T18:22:29.426 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:29 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: return self.sslsocket_class._create( 2024-09-13T18:22:29.427 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:29 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-13T18:22:29.427 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:29 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: self.do_handshake() 2024-09-13T18:22:29.427 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:29 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-13T18:22:29.427 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:29 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: self._sslobj.do_handshake() 2024-09-13T18:22:29.427 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:29 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-13T18:22:29.427 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:22:29 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:22:29.433 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T18:15:40.129659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T18:20:00.024407+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T18:20:01.085549+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T18:20:04.334216+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T18:22:29.434 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T18:22:29.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.577+0000 7f4ce35e2640 1 -- 172.21.15.114:0/2875370539 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4cdc103110 msgr2=0x7f4cdc105500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:29.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.577+0000 7f4ce35e2640 1 --2- 172.21.15.114:0/2875370539 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4cdc103110 0x7f4cdc105500 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc009900 tx=0x7f4ccc02a410 comp rx=0 tx=0).stop 2024-09-13T18:22:29.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.577+0000 7f4ce35e2640 1 -- 172.21.15.114:0/2875370539 shutdown_connections 2024-09-13T18:22:29.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.577+0000 7f4ce35e2640 1 --2- 172.21.15.114:0/2875370539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cdc105a40 0x7f4cdc107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:29.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.577+0000 7f4ce35e2640 1 --2- 172.21.15.114:0/2875370539 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4cdc103110 0x7f4cdc105500 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:29.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.577+0000 7f4ce35e2640 1 --2- 172.21.15.114:0/2875370539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4cdc1007e0 0x7f4cdc102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:29.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.577+0000 7f4ce35e2640 1 -- 172.21.15.114:0/2875370539 >> 172.21.15.114:0/2875370539 conn(0x7f4cdc0fa3e0 msgr2=0x7f4cdc0fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:29.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.577+0000 7f4ce35e2640 1 -- 172.21.15.114:0/2875370539 shutdown_connections 2024-09-13T18:22:29.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.577+0000 7f4ce35e2640 1 -- 172.21.15.114:0/2875370539 wait complete. 2024-09-13T18:22:29.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.578+0000 7f4ce35e2640 1 Processor -- start 2024-09-13T18:22:29.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.578+0000 7f4ce35e2640 1 -- start start 2024-09-13T18:22:29.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.578+0000 7f4ce35e2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4cdc1007e0 0x7f4cdc195cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:29.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.578+0000 7f4ce35e2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cdc103110 0x7f4cdc1961f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:29.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.578+0000 7f4ce35e2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4cdc105a40 0x7f4cdc19d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:29.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.578+0000 7f4ce35e2640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cdc19d7c0 con 0x7f4cdc1007e0 2024-09-13T18:22:29.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.578+0000 7f4ce35e2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cdc19d930 con 0x7f4cdc103110 2024-09-13T18:22:29.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.578+0000 7f4ce35e2640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cdc19daf0 con 0x7f4cdc105a40 2024-09-13T18:22:29.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.578+0000 7f4ce1b7f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4cdc105a40 0x7f4cdc19d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:29.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.578+0000 7f4ce137e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4cdc1007e0 0x7f4cdc195cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:29.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.578+0000 7f4ce1b7f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4cdc105a40 0x7f4cdc19d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:58400/0 (socket says 172.21.15.114:58400) 2024-09-13T18:22:29.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.578+0000 7f4ce1b7f640 1 -- 172.21.15.114:0/4284547997 learned_addr learned my addr 172.21.15.114:0/4284547997 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:29.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.578+0000 7f4ce0b7d640 1 --2- 172.21.15.114:0/4284547997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cdc103110 0x7f4cdc1961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:29.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.578+0000 7f4ce1b7f640 1 -- 172.21.15.114:0/4284547997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cdc103110 msgr2=0x7f4cdc1961f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:29.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.579+0000 7f4ce1b7f640 1 --2- 172.21.15.114:0/4284547997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cdc103110 0x7f4cdc1961f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:29.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.579+0000 7f4ce1b7f640 1 -- 172.21.15.114:0/4284547997 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4cdc1007e0 msgr2=0x7f4cdc195cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:29.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.579+0000 7f4ce1b7f640 1 --2- 172.21.15.114:0/4284547997 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4cdc1007e0 0x7f4cdc195cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:29.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.579+0000 7f4ce1b7f640 1 -- 172.21.15.114:0/4284547997 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ccc009560 con 0x7f4cdc105a40 2024-09-13T18:22:29.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.579+0000 7f4ce1b7f640 1 --2- 172.21.15.114:0/4284547997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4cdc105a40 0x7f4cdc19d250 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4cd400ccb0 tx=0x7f4cd400a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:29.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.579+0000 7f4cd27fc640 1 -- 172.21.15.114:0/4284547997 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd400f450 con 0x7f4cdc105a40 2024-09-13T18:22:29.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.579+0000 7f4cd27fc640 1 -- 172.21.15.114:0/4284547997 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4cd400fa40 con 0x7f4cdc105a40 2024-09-13T18:22:29.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.579+0000 7f4ce35e2640 1 -- 172.21.15.114:0/4284547997 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cdc19ddd0 con 0x7f4cdc105a40 2024-09-13T18:22:29.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.579+0000 7f4cd27fc640 1 -- 172.21.15.114:0/4284547997 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd4018970 con 0x7f4cdc105a40 2024-09-13T18:22:29.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.579+0000 7f4ce35e2640 1 -- 172.21.15.114:0/4284547997 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cdc19e200 con 0x7f4cdc105a40 2024-09-13T18:22:29.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.581+0000 7f4ce35e2640 1 -- 172.21.15.114:0/4284547997 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cdc0fbed0 con 0x7f4cdc105a40 2024-09-13T18:22:29.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.582+0000 7f4cd27fc640 1 -- 172.21.15.114:0/4284547997 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 22) v1 ==== 135685+0+0 (secure 0 0 0) 0x7f4cd400fbb0 con 0x7f4cdc105a40 2024-09-13T18:22:29.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.582+0000 7f4cd27fc640 1 --2- 172.21.15.114:0/4284547997 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f4cc80a0e90 0x7f4cc80a3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:29.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.583+0000 7f4cd27fc640 1 -- 172.21.15.114:0/4284547997 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f4cd40ccdb0 con 0x7f4cdc105a40 2024-09-13T18:22:29.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.583+0000 7f4ce137e640 1 --2- 172.21.15.114:0/4284547997 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f4cc80a0e90 0x7f4cc80a3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:29.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.583+0000 7f4ce137e640 1 --2- 172.21.15.114:0/4284547997 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f4cc80a0e90 0x7f4cc80a3350 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc196b80 tx=0x7f4cd8005f90 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:29.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.587+0000 7f4cd27fc640 1 -- 172.21.15.114:0/4284547997 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f4cd40963c0 con 0x7f4cdc105a40 2024-09-13T18:22:29.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.763+0000 7f4ce35e2640 1 -- 172.21.15.114:0/4284547997 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4cdc02c3e0 con 0x7f4cdc105a40 2024-09-13T18:22:29.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.764+0000 7f4cd27fc640 1 -- 172.21.15.114:0/4284547997 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4cd40999e0 con 0x7f4cdc105a40 2024-09-13T18:22:29.766 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 1 2024-09-13T18:22:29.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.766+0000 7f4ce35e2640 1 -- 172.21.15.114:0/4284547997 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f4cc80a0e90 msgr2=0x7f4cc80a3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:29.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.766+0000 7f4ce35e2640 1 --2- 172.21.15.114:0/4284547997 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f4cc80a0e90 0x7f4cc80a3350 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc196b80 tx=0x7f4cd8005f90 comp rx=0 tx=0).stop 2024-09-13T18:22:29.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.766+0000 7f4ce35e2640 1 -- 172.21.15.114:0/4284547997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4cdc105a40 msgr2=0x7f4cdc19d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:29.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.766+0000 7f4ce35e2640 1 --2- 172.21.15.114:0/4284547997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4cdc105a40 0x7f4cdc19d250 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4cd400ccb0 tx=0x7f4cd400a620 comp rx=0 tx=0).stop 2024-09-13T18:22:29.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.766+0000 7f4ce35e2640 1 -- 172.21.15.114:0/4284547997 shutdown_connections 2024-09-13T18:22:29.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.766+0000 7f4ce35e2640 1 --2- 172.21.15.114:0/4284547997 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f4cc80a0e90 0x7f4cc80a3350 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:29.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.766+0000 7f4ce35e2640 1 --2- 172.21.15.114:0/4284547997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4cdc105a40 0x7f4cdc19d250 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:29.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.766+0000 7f4ce35e2640 1 --2- 172.21.15.114:0/4284547997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cdc103110 0x7f4cdc1961f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:29.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.766+0000 7f4ce35e2640 1 --2- 172.21.15.114:0/4284547997 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4cdc1007e0 0x7f4cdc195cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:29.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.766+0000 7f4ce35e2640 1 -- 172.21.15.114:0/4284547997 >> 172.21.15.114:0/4284547997 conn(0x7f4cdc0fa3e0 msgr2=0x7f4cdc1011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:29.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.766+0000 7f4ce35e2640 1 -- 172.21.15.114:0/4284547997 shutdown_connections 2024-09-13T18:22:29.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.766+0000 7f4ce35e2640 1 -- 172.21.15.114:0/4284547997 wait complete. 2024-09-13T18:22:29.781 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T18:22:29.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.890+0000 7fd3ffa06640 1 -- 172.21.15.114:0/1677290960 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd3f80fe650 msgr2=0x7fd3f80fea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:29.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.890+0000 7fd3ffa06640 1 --2- 172.21.15.114:0/1677290960 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd3f80fe650 0x7fd3f80fea50 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4009900 tx=0x7fd3f402a480 comp rx=0 tx=0).stop 2024-09-13T18:22:29.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.890+0000 7fd3ffa06640 1 -- 172.21.15.114:0/1677290960 shutdown_connections 2024-09-13T18:22:29.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.890+0000 7fd3ffa06640 1 --2- 172.21.15.114:0/1677290960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3f80ff930 0x7fd3f80ffde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:29.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.890+0000 7fd3ffa06640 1 --2- 172.21.15.114:0/1677290960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd3f80fef90 0x7fd3f80ff3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:29.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.890+0000 7fd3ffa06640 1 --2- 172.21.15.114:0/1677290960 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd3f80fe650 0x7fd3f80fea50 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:29.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.890+0000 7fd3ffa06640 1 -- 172.21.15.114:0/1677290960 >> 172.21.15.114:0/1677290960 conn(0x7fd3f80fa3e0 msgr2=0x7fd3f80fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:29.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.890+0000 7fd3ffa06640 1 -- 172.21.15.114:0/1677290960 shutdown_connections 2024-09-13T18:22:29.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.890+0000 7fd3ffa06640 1 -- 172.21.15.114:0/1677290960 wait complete. 2024-09-13T18:22:29.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.891+0000 7fd3ffa06640 1 Processor -- start 2024-09-13T18:22:29.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.891+0000 7fd3ffa06640 1 -- start start 2024-09-13T18:22:29.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.891+0000 7fd3ffa06640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3f80fe650 0x7fd3f8197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:29.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.891+0000 7fd3ffa06640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd3f80fef90 0x7fd3f8198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:29.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.891+0000 7fd3ffa06640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd3f80ff930 0x7fd3f819f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:29.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.891+0000 7fd3ffa06640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f819f9f0 con 0x7fd3f80ff930 2024-09-13T18:22:29.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.891+0000 7fd3ffa06640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f819fb60 con 0x7fd3f80fef90 2024-09-13T18:22:29.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.891+0000 7fd3ffa06640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f819fd20 con 0x7fd3f80fe650 2024-09-13T18:22:29.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.891+0000 7fd3fdfa3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd3f80ff930 0x7fd3f819f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:29.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.891+0000 7fd3fdfa3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd3f80ff930 0x7fd3f819f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33216/0 (socket says 172.21.15.114:33216) 2024-09-13T18:22:29.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.891+0000 7fd3fdfa3640 1 -- 172.21.15.114:0/1432676717 learned_addr learned my addr 172.21.15.114:0/1432676717 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:29.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.891+0000 7fd3fcfa1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd3f80fef90 0x7fd3f8198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:29.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.891+0000 7fd3fd7a2640 1 --2- 172.21.15.114:0/1432676717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3f80fe650 0x7fd3f8197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:29.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.891+0000 7fd3fcfa1640 1 -- 172.21.15.114:0/1432676717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3f80fe650 msgr2=0x7fd3f8197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:29.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.891+0000 7fd3fcfa1640 1 --2- 172.21.15.114:0/1432676717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3f80fe650 0x7fd3f8197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:29.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.891+0000 7fd3fcfa1640 1 -- 172.21.15.114:0/1432676717 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd3f80ff930 msgr2=0x7fd3f819f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:29.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.891+0000 7fd3fcfa1640 1 --2- 172.21.15.114:0/1432676717 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd3f80ff930 0x7fd3f819f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:29.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.891+0000 7fd3fcfa1640 1 -- 172.21.15.114:0/1432676717 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f4009560 con 0x7fd3f80fef90 2024-09-13T18:22:29.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.891+0000 7fd3fdfa3640 1 --2- 172.21.15.114:0/1432676717 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd3f80ff930 0x7fd3f819f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:22:29.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.891+0000 7fd3fd7a2640 1 --2- 172.21.15.114:0/1432676717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3f80fe650 0x7fd3f8197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:22:29.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.892+0000 7fd3fcfa1640 1 --2- 172.21.15.114:0/1432676717 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd3f80fef90 0x7fd3f8198440 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd3e800e9f0 tx=0x7fd3e800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:29.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.892+0000 7fd3ee7fc640 1 -- 172.21.15.114:0/1432676717 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e800ccc0 con 0x7fd3f80fef90 2024-09-13T18:22:29.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.892+0000 7fd3ee7fc640 1 -- 172.21.15.114:0/1432676717 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd3e800ce20 con 0x7fd3f80fef90 2024-09-13T18:22:29.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.892+0000 7fd3ffa06640 1 -- 172.21.15.114:0/1432676717 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3f819ffa0 con 0x7fd3f80fef90 2024-09-13T18:22:29.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.892+0000 7fd3ee7fc640 1 -- 172.21.15.114:0/1432676717 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e8018750 con 0x7fd3f80fef90 2024-09-13T18:22:29.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.892+0000 7fd3ffa06640 1 -- 172.21.15.114:0/1432676717 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3f81a0480 con 0x7fd3f80fef90 2024-09-13T18:22:29.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.893+0000 7fd3ffa06640 1 -- 172.21.15.114:0/1432676717 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3f81093a0 con 0x7fd3f80fef90 2024-09-13T18:22:29.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.894+0000 7fd3ee7fc640 1 -- 172.21.15.114:0/1432676717 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 22) v1 ==== 135685+0+0 (secure 0 0 0) 0x7fd3e8010430 con 0x7fd3f80fef90 2024-09-13T18:22:29.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.895+0000 7fd3ee7fc640 1 --2- 172.21.15.114:0/1432676717 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fd3e40a0e90 0x7fd3e40a3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:29.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.895+0000 7fd3fd7a2640 1 --2- 172.21.15.114:0/1432676717 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fd3e40a0e90 0x7fd3e40a3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:29.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.895+0000 7fd3ee7fc640 1 -- 172.21.15.114:0/1432676717 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fd3e8014070 con 0x7fd3f80fef90 2024-09-13T18:22:29.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.895+0000 7fd3fd7a2640 1 --2- 172.21.15.114:0/1432676717 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fd3e40a0e90 0x7fd3e40a3350 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4005ec0 tx=0x7fd3f4000bc0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:29.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:29.899+0000 7fd3ee7fc640 1 -- 172.21.15.114:0/1432676717 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fd3e8052e20 con 0x7fd3f80fef90 2024-09-13T18:22:30.070 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:30 smithi114 ceph-mon[30897]: mgrmap e22: x(active, since 1.01911s), standbys: y, z 2024-09-13T18:22:30.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.095+0000 7fd3ffa06640 1 -- 172.21.15.114:0/1432676717 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd3f8108e10 con 0x7fd3f80fef90 2024-09-13T18:22:30.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.095+0000 7fd3ee7fc640 1 -- 172.21.15.114:0/1432676717 <== mon.2 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd3e80cce60 con 0x7fd3f80fef90 2024-09-13T18:22:30.098 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2024-09-13T18:22:30.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.097+0000 7fd3ffa06640 1 -- 172.21.15.114:0/1432676717 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fd3e40a0e90 msgr2=0x7fd3e40a3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:30.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.097+0000 7fd3ffa06640 1 --2- 172.21.15.114:0/1432676717 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fd3e40a0e90 0x7fd3e40a3350 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4005ec0 tx=0x7fd3f4000bc0 comp rx=0 tx=0).stop 2024-09-13T18:22:30.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.097+0000 7fd3ffa06640 1 -- 172.21.15.114:0/1432676717 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd3f80fef90 msgr2=0x7fd3f8198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:30.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.097+0000 7fd3ffa06640 1 --2- 172.21.15.114:0/1432676717 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd3f80fef90 0x7fd3f8198440 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd3e800e9f0 tx=0x7fd3e800eeb0 comp rx=0 tx=0).stop 2024-09-13T18:22:30.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.097+0000 7fd3ffa06640 1 -- 172.21.15.114:0/1432676717 shutdown_connections 2024-09-13T18:22:30.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.097+0000 7fd3ffa06640 1 --2- 172.21.15.114:0/1432676717 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fd3e40a0e90 0x7fd3e40a3350 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:30.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.097+0000 7fd3ffa06640 1 --2- 172.21.15.114:0/1432676717 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd3f80ff930 0x7fd3f819f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:30.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.097+0000 7fd3ffa06640 1 --2- 172.21.15.114:0/1432676717 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd3f80fef90 0x7fd3f8198440 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:30.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.097+0000 7fd3ffa06640 1 --2- 172.21.15.114:0/1432676717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3f80fe650 0x7fd3f8197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:30.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.097+0000 7fd3ffa06640 1 -- 172.21.15.114:0/1432676717 >> 172.21.15.114:0/1432676717 conn(0x7fd3f80fa3e0 msgr2=0x7fd3f80fbfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:30.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.097+0000 7fd3ffa06640 1 -- 172.21.15.114:0/1432676717 shutdown_connections 2024-09-13T18:22:30.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.097+0000 7fd3ffa06640 1 -- 172.21.15.114:0/1432676717 wait complete. 2024-09-13T18:22:30.112 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T18:22:30.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.238+0000 7f5ea1b4c640 1 -- 172.21.15.114:0/228757668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e9c100870 msgr2=0x7f5e9c102c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:30.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.238+0000 7f5ea1b4c640 1 --2- 172.21.15.114:0/228757668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e9c100870 0x7f5e9c102c60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c009860 tx=0x7f5e8c02a340 comp rx=0 tx=0).stop 2024-09-13T18:22:30.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.238+0000 7f5ea1b4c640 1 -- 172.21.15.114:0/228757668 shutdown_connections 2024-09-13T18:22:30.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.238+0000 7f5ea1b4c640 1 --2- 172.21.15.114:0/228757668 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e9c105ad0 0x7f5e9c107ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:30.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.238+0000 7f5ea1b4c640 1 --2- 172.21.15.114:0/228757668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e9c1031a0 0x7f5e9c105590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:30.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.238+0000 7f5ea1b4c640 1 --2- 172.21.15.114:0/228757668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e9c100870 0x7f5e9c102c60 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:30.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.238+0000 7f5ea1b4c640 1 -- 172.21.15.114:0/228757668 >> 172.21.15.114:0/228757668 conn(0x7f5e9c0fa430 msgr2=0x7f5e9c0fc850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:30.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.238+0000 7f5ea1b4c640 1 -- 172.21.15.114:0/228757668 shutdown_connections 2024-09-13T18:22:30.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.238+0000 7f5ea1b4c640 1 -- 172.21.15.114:0/228757668 wait complete. 2024-09-13T18:22:30.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.239+0000 7f5ea1b4c640 1 Processor -- start 2024-09-13T18:22:30.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.239+0000 7f5ea1b4c640 1 -- start start 2024-09-13T18:22:30.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.239+0000 7f5ea1b4c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e9c100870 0x7f5e9c195da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:30.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.239+0000 7f5ea1b4c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e9c1031a0 0x7f5e9c1962e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:30.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.239+0000 7f5ea1b4c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e9c105ad0 0x7f5e9c19d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:30.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.239+0000 7f5ea1b4c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9c19d840 con 0x7f5e9c105ad0 2024-09-13T18:22:30.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.239+0000 7f5ea1b4c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9c19d9b0 con 0x7f5e9c100870 2024-09-13T18:22:30.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.239+0000 7f5ea1b4c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9c19db70 con 0x7f5e9c1031a0 2024-09-13T18:22:30.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.239+0000 7f5e9affd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e9c1031a0 0x7f5e9c1962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:30.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.239+0000 7f5e9b7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e9c100870 0x7f5e9c195da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:30.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.239+0000 7f5e9bfff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e9c105ad0 0x7f5e9c19d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:30.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.239+0000 7f5e9affd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e9c1031a0 0x7f5e9c1962e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:58460/0 (socket says 172.21.15.114:58460) 2024-09-13T18:22:30.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.239+0000 7f5e9bfff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e9c105ad0 0x7f5e9c19d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33270/0 (socket says 172.21.15.114:33270) 2024-09-13T18:22:30.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.239+0000 7f5e9affd640 1 -- 172.21.15.114:0/1359951022 learned_addr learned my addr 172.21.15.114:0/1359951022 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:30.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.239+0000 7f5e9b7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e9c100870 0x7f5e9c195da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:53756/0 (socket says 172.21.15.114:53756) 2024-09-13T18:22:30.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.240+0000 7f5e9b7fe640 1 -- 172.21.15.114:0/1359951022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e9c1031a0 msgr2=0x7f5e9c1962e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:30.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.240+0000 7f5e9b7fe640 1 --2- 172.21.15.114:0/1359951022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e9c1031a0 0x7f5e9c1962e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:30.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.240+0000 7f5e9b7fe640 1 -- 172.21.15.114:0/1359951022 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e9c105ad0 msgr2=0x7f5e9c19d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:30.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.240+0000 7f5e9b7fe640 1 --2- 172.21.15.114:0/1359951022 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e9c105ad0 0x7f5e9c19d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:30.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.240+0000 7f5e9b7fe640 1 -- 172.21.15.114:0/1359951022 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e94009900 con 0x7f5e9c100870 2024-09-13T18:22:30.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.240+0000 7f5e9bfff640 1 --2- 172.21.15.114:0/1359951022 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e9c105ad0 0x7f5e9c19d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:22:30.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.240+0000 7f5e9b7fe640 1 --2- 172.21.15.114:0/1359951022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e9c100870 0x7f5e9c195da0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c02a860 tx=0x7f5e8c030890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:30.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.240+0000 7f5e98ff9640 1 -- 172.21.15.114:0/1359951022 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e8c032e50 con 0x7f5e9c100870 2024-09-13T18:22:30.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.240+0000 7f5e98ff9640 1 -- 172.21.15.114:0/1359951022 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5e8c0044d0 con 0x7f5e9c100870 2024-09-13T18:22:30.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.240+0000 7f5ea1b4c640 1 -- 172.21.15.114:0/1359951022 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e8c009490 con 0x7f5e9c100870 2024-09-13T18:22:30.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.240+0000 7f5e98ff9640 1 -- 172.21.15.114:0/1359951022 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e8c03b430 con 0x7f5e9c100870 2024-09-13T18:22:30.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.240+0000 7f5ea1b4c640 1 -- 172.21.15.114:0/1359951022 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e9c19e110 con 0x7f5e9c100870 2024-09-13T18:22:30.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.241+0000 7f5ea1b4c640 1 -- 172.21.15.114:0/1359951022 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e9c0fbf20 con 0x7f5e9c100870 2024-09-13T18:22:30.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.243+0000 7f5e98ff9640 1 -- 172.21.15.114:0/1359951022 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 22) v1 ==== 135685+0+0 (secure 0 0 0) 0x7f5e8c039040 con 0x7f5e9c100870 2024-09-13T18:22:30.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.244+0000 7f5e98ff9640 1 --2- 172.21.15.114:0/1359951022 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f5e840a0e90 0x7f5e840a3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:30.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.244+0000 7f5e9affd640 1 --2- 172.21.15.114:0/1359951022 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f5e840a0e90 0x7f5e840a3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:30.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.244+0000 7f5e98ff9640 1 -- 172.21.15.114:0/1359951022 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f5e8c0ed2f0 con 0x7f5e9c100870 2024-09-13T18:22:30.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.244+0000 7f5e9affd640 1 --2- 172.21.15.114:0/1359951022 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f5e840a0e90 0x7f5e840a3350 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c197260 tx=0x7f5e9000b480 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:30.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.246+0000 7f5e98ff9640 1 -- 172.21.15.114:0/1359951022 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f5e8c0b6900 con 0x7f5e9c100870 2024-09-13T18:22:30.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:30 smithi134 ceph-mon[34283]: mgrmap e22: x(active, since 1.01911s), standbys: y, z 2024-09-13T18:22:30.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:30 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/2695892655' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-13T18:22:30.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:30 smithi134 ceph-mon[34283]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-13T18:22:30.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:30 smithi134 ceph-mon[34283]: [13/Sep/2024:18:22:29] ENGINE Bus STARTING 2024-09-13T18:22:30.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:30 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/3021363049' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:22:30.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:30 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/4284547997' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T18:22:30.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:30 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:30.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:30 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:30.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:30 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:30.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:30 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:30.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:30 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:30.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:30 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:30.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.418+0000 7f5ea1b4c640 1 -- 172.21.15.114:0/1359951022 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5e9c10b550 con 0x7f5e9c100870 2024-09-13T18:22:30.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.419+0000 7f5e98ff9640 1 -- 172.21.15.114:0/1359951022 <== mon.2 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5e8c0a3e60 con 0x7f5e9c100870 2024-09-13T18:22:30.422 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 64 2024-09-13T18:22:30.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.421+0000 7f5ea1b4c640 1 -- 172.21.15.114:0/1359951022 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f5e840a0e90 msgr2=0x7f5e840a3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:30.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.421+0000 7f5ea1b4c640 1 --2- 172.21.15.114:0/1359951022 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f5e840a0e90 0x7f5e840a3350 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c197260 tx=0x7f5e9000b480 comp rx=0 tx=0).stop 2024-09-13T18:22:30.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.421+0000 7f5ea1b4c640 1 -- 172.21.15.114:0/1359951022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e9c100870 msgr2=0x7f5e9c195da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:30.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.421+0000 7f5ea1b4c640 1 --2- 172.21.15.114:0/1359951022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e9c100870 0x7f5e9c195da0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c02a860 tx=0x7f5e8c030890 comp rx=0 tx=0).stop 2024-09-13T18:22:30.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.421+0000 7f5ea1b4c640 1 -- 172.21.15.114:0/1359951022 shutdown_connections 2024-09-13T18:22:30.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.421+0000 7f5ea1b4c640 1 --2- 172.21.15.114:0/1359951022 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f5e840a0e90 0x7f5e840a3350 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:30.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.421+0000 7f5ea1b4c640 1 --2- 172.21.15.114:0/1359951022 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e9c105ad0 0x7f5e9c19d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:30.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.421+0000 7f5ea1b4c640 1 --2- 172.21.15.114:0/1359951022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e9c1031a0 0x7f5e9c1962e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:30.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.421+0000 7f5ea1b4c640 1 --2- 172.21.15.114:0/1359951022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e9c100870 0x7f5e9c195da0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:30.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.421+0000 7f5ea1b4c640 1 -- 172.21.15.114:0/1359951022 >> 172.21.15.114:0/1359951022 conn(0x7f5e9c0fa430 msgr2=0x7f5e9c101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:30.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.421+0000 7f5ea1b4c640 1 -- 172.21.15.114:0/1359951022 shutdown_connections 2024-09-13T18:22:30.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.421+0000 7f5ea1b4c640 1 -- 172.21.15.114:0/1359951022 wait complete. 2024-09-13T18:22:30.434 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:30 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2695892655' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-13T18:22:30.434 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:30 smithi114 ceph-mon[30897]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-13T18:22:30.434 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:30 smithi114 ceph-mon[30897]: [13/Sep/2024:18:22:29] ENGINE Bus STARTING 2024-09-13T18:22:30.435 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:30 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/3021363049' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:22:30.435 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:30 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/4284547997' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T18:22:30.435 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:30 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:30.435 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:30 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:30.435 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:30 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:30.435 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:30 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:30.435 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:30 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:30.435 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:30 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:30.436 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T18:22:30.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:30 smithi204 ceph-mon[34777]: mgrmap e22: x(active, since 1.01911s), standbys: y, z 2024-09-13T18:22:30.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:30 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/2695892655' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-13T18:22:30.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:30 smithi204 ceph-mon[34777]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-13T18:22:30.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:30 smithi204 ceph-mon[34777]: [13/Sep/2024:18:22:29] ENGINE Bus STARTING 2024-09-13T18:22:30.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:30 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/3021363049' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:22:30.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:30 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/4284547997' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T18:22:30.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:30 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:30.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:30 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:30.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:30 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:30.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:30 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:30.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:30 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:30.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:30 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:30.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.541+0000 7fcf3bd7d640 1 -- 172.21.15.114:0/4143702039 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf34103110 msgr2=0x7fcf34105500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:30.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.541+0000 7fcf3bd7d640 1 --2- 172.21.15.114:0/4143702039 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf34103110 0x7fcf34105500 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fcf24009900 tx=0x7fcf2402a410 comp rx=0 tx=0).stop 2024-09-13T18:22:30.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.541+0000 7fcf3bd7d640 1 -- 172.21.15.114:0/4143702039 shutdown_connections 2024-09-13T18:22:30.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.541+0000 7fcf3bd7d640 1 --2- 172.21.15.114:0/4143702039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf34105a40 0x7fcf34107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:30.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.541+0000 7fcf3bd7d640 1 --2- 172.21.15.114:0/4143702039 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf34103110 0x7fcf34105500 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:30.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.541+0000 7fcf3bd7d640 1 --2- 172.21.15.114:0/4143702039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf341007e0 0x7fcf34102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:30.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.541+0000 7fcf3bd7d640 1 -- 172.21.15.114:0/4143702039 >> 172.21.15.114:0/4143702039 conn(0x7fcf340fa3e0 msgr2=0x7fcf340fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:30.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.541+0000 7fcf3bd7d640 1 -- 172.21.15.114:0/4143702039 shutdown_connections 2024-09-13T18:22:30.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.541+0000 7fcf3bd7d640 1 -- 172.21.15.114:0/4143702039 wait complete. 2024-09-13T18:22:30.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.542+0000 7fcf3bd7d640 1 Processor -- start 2024-09-13T18:22:30.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.542+0000 7fcf3bd7d640 1 -- start start 2024-09-13T18:22:30.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.542+0000 7fcf3bd7d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf341007e0 0x7fcf34195d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:30.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.542+0000 7fcf3bd7d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf34103110 0x7fcf34196250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:30.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.542+0000 7fcf3bd7d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf34105a40 0x7fcf3419d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:30.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.542+0000 7fcf3bd7d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3419d7f0 con 0x7fcf341007e0 2024-09-13T18:22:30.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.542+0000 7fcf3bd7d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3419d960 con 0x7fcf34103110 2024-09-13T18:22:30.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.542+0000 7fcf3bd7d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3419db20 con 0x7fcf34105a40 2024-09-13T18:22:30.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.542+0000 7fcf39b19640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf341007e0 0x7fcf34195d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:30.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.542+0000 7fcf39b19640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf341007e0 0x7fcf34195d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33296/0 (socket says 172.21.15.114:33296) 2024-09-13T18:22:30.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.542+0000 7fcf39b19640 1 -- 172.21.15.114:0/2558828298 learned_addr learned my addr 172.21.15.114:0/2558828298 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:30.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.542+0000 7fcf39318640 1 --2- 172.21.15.114:0/2558828298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf34103110 0x7fcf34196250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:30.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.542+0000 7fcf3a31a640 1 --2- 172.21.15.114:0/2558828298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf34105a40 0x7fcf3419d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:30.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.542+0000 7fcf39b19640 1 -- 172.21.15.114:0/2558828298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf34105a40 msgr2=0x7fcf3419d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:30.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.542+0000 7fcf39b19640 1 --2- 172.21.15.114:0/2558828298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf34105a40 0x7fcf3419d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:30.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.542+0000 7fcf39b19640 1 -- 172.21.15.114:0/2558828298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf34103110 msgr2=0x7fcf34196250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:30.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.542+0000 7fcf39b19640 1 --2- 172.21.15.114:0/2558828298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf34103110 0x7fcf34196250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:30.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.542+0000 7fcf39b19640 1 -- 172.21.15.114:0/2558828298 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf24009560 con 0x7fcf341007e0 2024-09-13T18:22:30.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.543+0000 7fcf3a31a640 1 --2- 172.21.15.114:0/2558828298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf34105a40 0x7fcf3419d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:22:30.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.543+0000 7fcf39b19640 1 --2- 172.21.15.114:0/2558828298 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf341007e0 0x7fcf34195d10 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fcf3000e9f0 tx=0x7fcf3000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:30.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.543+0000 7fcf2affd640 1 -- 172.21.15.114:0/2558828298 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf30009700 con 0x7fcf341007e0 2024-09-13T18:22:30.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.543+0000 7fcf2affd640 1 -- 172.21.15.114:0/2558828298 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcf3000cd80 con 0x7fcf341007e0 2024-09-13T18:22:30.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.543+0000 7fcf3bd7d640 1 -- 172.21.15.114:0/2558828298 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf3419de00 con 0x7fcf341007e0 2024-09-13T18:22:30.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.543+0000 7fcf2affd640 1 -- 172.21.15.114:0/2558828298 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf300187e0 con 0x7fcf341007e0 2024-09-13T18:22:30.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.543+0000 7fcf3bd7d640 1 -- 172.21.15.114:0/2558828298 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf3419e220 con 0x7fcf341007e0 2024-09-13T18:22:30.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.544+0000 7fcf3bd7d640 1 -- 172.21.15.114:0/2558828298 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf341034f0 con 0x7fcf341007e0 2024-09-13T18:22:30.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.545+0000 7fcf2affd640 1 -- 172.21.15.114:0/2558828298 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 22) v1 ==== 135685+0+0 (secure 0 0 0) 0x7fcf30010430 con 0x7fcf341007e0 2024-09-13T18:22:30.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.545+0000 7fcf2affd640 1 --2- 172.21.15.114:0/2558828298 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fcf200a0e90 0x7fcf200a3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:30.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.546+0000 7fcf39318640 1 --2- 172.21.15.114:0/2558828298 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fcf200a0e90 0x7fcf200a3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:30.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.546+0000 7fcf2affd640 1 -- 172.21.15.114:0/2558828298 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fcf30014070 con 0x7fcf341007e0 2024-09-13T18:22:30.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.546+0000 7fcf39318640 1 --2- 172.21.15.114:0/2558828298 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fcf200a0e90 0x7fcf200a3350 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcf240057c0 tx=0x7fcf24005a20 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:30.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.550+0000 7fcf2affd640 1 -- 172.21.15.114:0/2558828298 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fcf30052e20 con 0x7fcf341007e0 2024-09-13T18:22:30.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.731+0000 7fcf3bd7d640 1 -- 172.21.15.114:0/2558828298 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fcf340fbed0 con 0x7fcf341007e0 2024-09-13T18:22:30.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.731+0000 7fcf2affd640 1 -- 172.21.15.114:0/2558828298 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) v1 ==== 103+0+11 (secure 0 0 0) 0x7fcf340fbed0 con 0x7fcf341007e0 2024-09-13T18:22:30.735 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2024-09-13T18:22:30.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.733+0000 7fcf3bd7d640 1 -- 172.21.15.114:0/2558828298 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fcf200a0e90 msgr2=0x7fcf200a3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:30.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.733+0000 7fcf3bd7d640 1 --2- 172.21.15.114:0/2558828298 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fcf200a0e90 0x7fcf200a3350 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcf240057c0 tx=0x7fcf24005a20 comp rx=0 tx=0).stop 2024-09-13T18:22:30.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.734+0000 7fcf3bd7d640 1 -- 172.21.15.114:0/2558828298 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf341007e0 msgr2=0x7fcf34195d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:30.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.734+0000 7fcf3bd7d640 1 --2- 172.21.15.114:0/2558828298 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf341007e0 0x7fcf34195d10 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fcf3000e9f0 tx=0x7fcf3000eeb0 comp rx=0 tx=0).stop 2024-09-13T18:22:30.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.734+0000 7fcf3bd7d640 1 -- 172.21.15.114:0/2558828298 shutdown_connections 2024-09-13T18:22:30.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.734+0000 7fcf3bd7d640 1 --2- 172.21.15.114:0/2558828298 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fcf200a0e90 0x7fcf200a3350 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:30.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.734+0000 7fcf3bd7d640 1 --2- 172.21.15.114:0/2558828298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf34105a40 0x7fcf3419d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:30.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.734+0000 7fcf3bd7d640 1 --2- 172.21.15.114:0/2558828298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf34103110 0x7fcf34196250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:30.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.734+0000 7fcf3bd7d640 1 --2- 172.21.15.114:0/2558828298 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf341007e0 0x7fcf34195d10 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:30.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.734+0000 7fcf3bd7d640 1 -- 172.21.15.114:0/2558828298 >> 172.21.15.114:0/2558828298 conn(0x7fcf340fa3e0 msgr2=0x7fcf34101140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:30.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.734+0000 7fcf3bd7d640 1 -- 172.21.15.114:0/2558828298 shutdown_connections 2024-09-13T18:22:30.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.734+0000 7fcf3bd7d640 1 -- 172.21.15.114:0/2558828298 wait complete. 2024-09-13T18:22:30.748 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:22:30.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.847+0000 7f7214125640 1 -- 172.21.15.114:0/1151106508 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f720c100280 msgr2=0x7f720c100680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:30.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.847+0000 7f7214125640 1 --2- 172.21.15.114:0/1151106508 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f720c100280 0x7f720c100680 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7208009900 tx=0x7f720802a410 comp rx=0 tx=0).stop 2024-09-13T18:22:30.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.847+0000 7f7214125640 1 -- 172.21.15.114:0/1151106508 shutdown_connections 2024-09-13T18:22:30.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.847+0000 7f7214125640 1 --2- 172.21.15.114:0/1151106508 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f720c0ff8e0 0x7f720c107180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:30.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.847+0000 7f7214125640 1 --2- 172.21.15.114:0/1151106508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f720c0fef20 0x7f720c0ff3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:30.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.847+0000 7f7214125640 1 --2- 172.21.15.114:0/1151106508 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f720c100280 0x7f720c100680 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:30.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.847+0000 7f7214125640 1 -- 172.21.15.114:0/1151106508 >> 172.21.15.114:0/1151106508 conn(0x7f720c0f9b80 msgr2=0x7f720c0fbfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:30.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.848+0000 7f7214125640 1 -- 172.21.15.114:0/1151106508 shutdown_connections 2024-09-13T18:22:30.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.848+0000 7f7214125640 1 -- 172.21.15.114:0/1151106508 wait complete. 2024-09-13T18:22:30.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.848+0000 7f7214125640 1 Processor -- start 2024-09-13T18:22:30.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.848+0000 7f7214125640 1 -- start start 2024-09-13T18:22:30.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.848+0000 7f7214125640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f720c0fef20 0x7f720c0fe980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:30.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.848+0000 7f7214125640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f720c0ff8e0 0x7f720c19aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:30.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.848+0000 7f7214125640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f720c100280 0x7f720c19d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:30.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.848+0000 7f7214125640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f720c19d8b0 con 0x7f720c0ff8e0 2024-09-13T18:22:30.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.848+0000 7f7214125640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f720c19da20 con 0x7f720c0fef20 2024-09-13T18:22:30.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.848+0000 7f7214125640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f720c19dbe0 con 0x7f720c100280 2024-09-13T18:22:30.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.848+0000 7f72116c0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f720c0ff8e0 0x7f720c19aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:30.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.848+0000 7f72116c0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f720c0ff8e0 0x7f720c19aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33302/0 (socket says 172.21.15.114:33302) 2024-09-13T18:22:30.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.848+0000 7f72116c0640 1 -- 172.21.15.114:0/696874907 learned_addr learned my addr 172.21.15.114:0/696874907 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:30.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.848+0000 7f7211ec1640 1 --2- 172.21.15.114:0/696874907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f720c0fef20 0x7f720c0fe980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:30.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.848+0000 7f72126c2640 1 --2- 172.21.15.114:0/696874907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f720c100280 0x7f720c19d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:30.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.849+0000 7f72116c0640 1 -- 172.21.15.114:0/696874907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f720c100280 msgr2=0x7f720c19d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:30.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.849+0000 7f72116c0640 1 --2- 172.21.15.114:0/696874907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f720c100280 0x7f720c19d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:30.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.849+0000 7f72116c0640 1 -- 172.21.15.114:0/696874907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f720c0fef20 msgr2=0x7f720c0fe980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:30.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.849+0000 7f72116c0640 1 --2- 172.21.15.114:0/696874907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f720c0fef20 0x7f720c0fe980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:30.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.849+0000 7f72116c0640 1 -- 172.21.15.114:0/696874907 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7208009560 con 0x7f720c0ff8e0 2024-09-13T18:22:30.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.849+0000 7f72116c0640 1 --2- 172.21.15.114:0/696874907 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f720c0ff8e0 0x7f720c19aab0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f71fc00e9f0 tx=0x7f71fc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:30.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.849+0000 7f7202ffd640 1 -- 172.21.15.114:0/696874907 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71fc009720 con 0x7f720c0ff8e0 2024-09-13T18:22:30.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.849+0000 7f7214125640 1 -- 172.21.15.114:0/696874907 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f720c19de60 con 0x7f720c0ff8e0 2024-09-13T18:22:30.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.849+0000 7f7202ffd640 1 -- 172.21.15.114:0/696874907 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f71fc00cd80 con 0x7f720c0ff8e0 2024-09-13T18:22:30.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.849+0000 7f7202ffd640 1 -- 172.21.15.114:0/696874907 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71fc010430 con 0x7f720c0ff8e0 2024-09-13T18:22:30.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.849+0000 7f7214125640 1 -- 172.21.15.114:0/696874907 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f720c19e290 con 0x7f720c0ff8e0 2024-09-13T18:22:30.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.850+0000 7f7214125640 1 -- 172.21.15.114:0/696874907 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f720c106310 con 0x7f720c0ff8e0 2024-09-13T18:22:30.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.851+0000 7f7202ffd640 1 -- 172.21.15.114:0/696874907 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 22) v1 ==== 135685+0+0 (secure 0 0 0) 0x7f71fc004f90 con 0x7f720c0ff8e0 2024-09-13T18:22:30.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.852+0000 7f7202ffd640 1 --2- 172.21.15.114:0/696874907 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f71f80a0e90 0x7f71f80a3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:30.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.852+0000 7f7211ec1640 1 --2- 172.21.15.114:0/696874907 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f71f80a0e90 0x7f71f80a3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:30.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.852+0000 7f7202ffd640 1 -- 172.21.15.114:0/696874907 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f71fc014070 con 0x7f720c0ff8e0 2024-09-13T18:22:30.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.853+0000 7f7211ec1640 1 --2- 172.21.15.114:0/696874907 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f71f80a0e90 0x7f71f80a3350 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7208006010 tx=0x7f7208000bc0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:30.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:30.859+0000 7f7202ffd640 1 -- 172.21.15.114:0/696874907 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f71fc0962c0 con 0x7f720c0ff8e0 2024-09-13T18:22:31.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.033+0000 7f7214125640 1 -- 172.21.15.114:0/696874907 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f720c105d80 con 0x7f720c0ff8e0 2024-09-13T18:22:31.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.034+0000 7f7202ffd640 1 -- 172.21.15.114:0/696874907 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+25079 (secure 0 0 0) 0x7f71fc0998e0 con 0x7f720c0ff8e0 2024-09-13T18:22:31.038 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:22:31.038 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":94,"fsid":"97e7edf2-71fb-11ef-bceb-c7b262605968","created":"2024-09-13T18:12:36.360625+0000","modified":"2024-09-13T18:22:28.057664+0000","last_up_change":"2024-09-13T18:19:12.350370+0000","last_in_change":"2024-09-13T18:18:52.321596+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:15:40.129659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:20:00.024407+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:20:01.085549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T18:20:04.334216+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9914f01a-9f6f-4878-b955-83570199c354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6803","nonce":3499525826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6805","nonce":3499525826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6809","nonce":3499525826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6807","nonce":3499525826}]},"public_addr":"172.21.15.114:6803/3499525826","cluster_addr":"172.21.15.114:6805/3499525826","heartbeat_back_addr":"172.21.15.114:6809/3499525826","heartbeat_front_addr":"172.21.15.114:6807/3499525826","state":["exists","up"]},{"osd":1,"uuid":"ef65e012-7469-4c20-b354-17022bca63c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6801","nonce":4264264866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6803","nonce":4264264866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6807","nonce":4264264866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6805","nonce":4264264866}]},"public_addr":"172.21.15.134:6801/4264264866","cluster_addr":"172.21.15.134:6803/4264264866","heartbeat_back_addr":"172.21.15.134:6807/4264264866","heartbeat_front_addr":"172.21.15.134:6805/4264264866","state":["exists","up"]},{"osd":2,"uuid":"578213fd-715f-4be7-91fa-254f3d0e3022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6801","nonce":2288807707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6803","nonce":2288807707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6807","nonce":2288807707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6805","nonce":2288807707}]},"public_addr":"172.21.15.204:6801/2288807707","cluster_addr":"172.21.15.204:6803/2288807707","heartbeat_back_addr":"172.21.15.204:6807/2288807707","heartbeat_front_addr":"172.21.15.204:6805/2288807707","state":["exists","up"]},{"osd":3,"uuid":"b64aab3f-1e52-4828-8397-74af5e05f2cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6811","nonce":2904204254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6813","nonce":2904204254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6817","nonce":2904204254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6815","nonce":2904204254}]},"public_addr":"172.21.15.114:6811/2904204254","cluster_addr":"172.21.15.114:6813/2904204254","heartbeat_back_addr":"172.21.15.114:6817/2904204254","heartbeat_front_addr":"172.21.15.114:6815/2904204254","state":["exists","up"]},{"osd":4,"uuid":"f38943d6-39c0-42e1-b427-cb6bb90ac38a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6809","nonce":1773945333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6811","nonce":1773945333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6815","nonce":1773945333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6813","nonce":1773945333}]},"public_addr":"172.21.15.134:6809/1773945333","cluster_addr":"172.21.15.134:6811/1773945333","heartbeat_back_addr":"172.21.15.134:6815/1773945333","heartbeat_front_addr":"172.21.15.134:6813/1773945333","state":["exists","up"]},{"osd":5,"uuid":"bb38f361-8e00-4178-9752-5ddbf3af20e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6809","nonce":2371778055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6811","nonce":2371778055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6815","nonce":2371778055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6813","nonce":2371778055}]},"public_addr":"172.21.15.204:6809/2371778055","cluster_addr":"172.21.15.204:6811/2371778055","heartbeat_back_addr":"172.21.15.204:6815/2371778055","heartbeat_front_addr":"172.21.15.204:6813/2371778055","state":["exists","up"]},{"osd":6,"uuid":"93d40a03-e7ff-4a3f-9efa-5c5403c9983d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6819","nonce":1484325044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6821","nonce":1484325044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6825","nonce":1484325044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6823","nonce":1484325044}]},"public_addr":"172.21.15.114:6819/1484325044","cluster_addr":"172.21.15.114:6821/1484325044","heartbeat_back_addr":"172.21.15.114:6825/1484325044","heartbeat_front_addr":"172.21.15.114:6823/1484325044","state":["exists","up"]},{"osd":7,"uuid":"982f629a-1b95-4be1-8e75-5ddce1ad77b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6817","nonce":799246614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6819","nonce":799246614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6823","nonce":799246614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6821","nonce":799246614}]},"public_addr":"172.21.15.134:6817/799246614","cluster_addr":"172.21.15.134:6819/799246614","heartbeat_back_addr":"172.21.15.134:6823/799246614","heartbeat_front_addr":"172.21.15.134:6821/799246614","state":["exists","up"]},{"osd":8,"uuid":"409beb9b-8227-4dad-a4a0-f0e5ce70dc5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6817","nonce":858331805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6819","nonce":858331805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6823","nonce":858331805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6821","nonce":858331805}]},"public_addr":"172.21.15.204:6817/858331805","cluster_addr":"172.21.15.204:6819/858331805","heartbeat_back_addr":"172.21.15.204:6823/858331805","heartbeat_front_addr":"172.21.15.204:6821/858331805","state":["exists","up"]},{"osd":9,"uuid":"a345a15c-b1fc-4877-b445-418eebf0d0ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6827","nonce":2172392854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6829","nonce":2172392854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6833","nonce":2172392854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6831","nonce":2172392854}]},"public_addr":"172.21.15.114:6827/2172392854","cluster_addr":"172.21.15.114:6829/2172392854","heartbeat_back_addr":"172.21.15.114:6833/2172392854","heartbeat_front_addr":"172.21.15.114:6831/2172392854","state":["exists","up"]},{"osd":10,"uuid":"464d995f-d90b-4caa-addb-1f7562bd74b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6825","nonce":3073513468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6827","nonce":3073513468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6831","nonce":3073513468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6829","nonce":3073513468}]},"public_addr":"172.21.15.134:6825/3073513468","cluster_addr":"172.21.15.134:6827/3073513468","heartbeat_back_addr":"172.21.15.134:6831/3073513468","heartbeat_front_addr":"172.21.15.134:6829/3073513468","state":["exists","up"]},{"osd":11,"uuid":"71dd794d-326c-477f-8d02-ee32bff1350d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6825","nonce":3239521765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6827","nonce":3239521765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6831","nonce":3239521765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6829","nonce":3239521765}]},"public_addr":"172.21.15.204:6825/3239521765","cluster_addr":"172.21.15.204:6827/3239521765","heartbeat_back_addr":"172.21.15.204:6831/3239521765","heartbeat_front_addr":"172.21.15.204:6829/3239521765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:14:54.542687+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:15:15.746055+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:15:38.371131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:01.383932+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:24.796237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:48.422214+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:11.284323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:34.570365+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:58.232855+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:18:21.609596+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:18:45.565859+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:19:10.017969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:6801/3396185541":"2024-09-14T18:22:28.057623+0000","172.21.15.114:6800/830738794":"2024-09-14T18:13:03.521116+0000","172.21.15.114:6801/830738794":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/1684954293":"2024-09-14T18:22:28.057623+0000","172.21.15.114:0/3544951322":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/722571999":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/2949422849":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2034742251":"2024-09-14T18:13:03.521116+0000","172.21.15.114:6800/4281866858":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/3407523320":"2024-09-14T18:22:28.057623+0000","172.21.15.114:6800/3396185541":"2024-09-14T18:22:28.057623+0000","172.21.15.114:0/4257828906":"2024-09-14T18:13:25.823027+0000","172.21.15.114:6801/4281866858":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2828364892":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2714091073":"2024-09-14T18:22:28.057623+0000","172.21.15.114:0/842237781":"2024-09-14T18:22:28.057623+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:22:31.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.037+0000 7f7214125640 1 -- 172.21.15.114:0/696874907 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f71f80a0e90 msgr2=0x7f71f80a3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:31.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.037+0000 7f7214125640 1 --2- 172.21.15.114:0/696874907 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f71f80a0e90 0x7f71f80a3350 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7208006010 tx=0x7f7208000bc0 comp rx=0 tx=0).stop 2024-09-13T18:22:31.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.037+0000 7f7214125640 1 -- 172.21.15.114:0/696874907 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f720c0ff8e0 msgr2=0x7f720c19aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:31.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.037+0000 7f7214125640 1 --2- 172.21.15.114:0/696874907 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f720c0ff8e0 0x7f720c19aab0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f71fc00e9f0 tx=0x7f71fc00edf0 comp rx=0 tx=0).stop 2024-09-13T18:22:31.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.038+0000 7f7214125640 1 -- 172.21.15.114:0/696874907 shutdown_connections 2024-09-13T18:22:31.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.038+0000 7f7214125640 1 --2- 172.21.15.114:0/696874907 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f71f80a0e90 0x7f71f80a3350 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:31.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.038+0000 7f7214125640 1 --2- 172.21.15.114:0/696874907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f720c100280 0x7f720c19d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:31.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.038+0000 7f7214125640 1 --2- 172.21.15.114:0/696874907 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f720c0ff8e0 0x7f720c19aab0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:31.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.038+0000 7f7214125640 1 --2- 172.21.15.114:0/696874907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f720c0fef20 0x7f720c0fe980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:31.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.038+0000 7f7214125640 1 -- 172.21.15.114:0/696874907 >> 172.21.15.114:0/696874907 conn(0x7f720c0f9b80 msgr2=0x7f720c1034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:31.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.038+0000 7f7214125640 1 -- 172.21.15.114:0/696874907 shutdown_connections 2024-09-13T18:22:31.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.038+0000 7f7214125640 1 -- 172.21.15.114:0/696874907 wait complete. 2024-09-13T18:22:31.060 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T18:15:40.129659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T18:20:00.024407+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T18:20:01.085549+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T18:20:04.334216+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T18:22:31.060 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T18:22:31.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.165+0000 7f2aeec96640 1 -- 172.21.15.114:0/475923835 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ae8100240 msgr2=0x7f2ae8100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:31.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.165+0000 7f2aeec96640 1 --2- 172.21.15.114:0/475923835 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ae8100240 0x7f2ae8100640 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f2adc009900 tx=0x7f2adc02a480 comp rx=0 tx=0).stop 2024-09-13T18:22:31.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.166+0000 7f2aeec96640 1 -- 172.21.15.114:0/475923835 shutdown_connections 2024-09-13T18:22:31.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.166+0000 7f2aeec96640 1 --2- 172.21.15.114:0/475923835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ae80ff850 0x7f2ae81070f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:31.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.166+0000 7f2aeec96640 1 --2- 172.21.15.114:0/475923835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ae80fee90 0x7f2ae80ff310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:31.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.166+0000 7f2aeec96640 1 --2- 172.21.15.114:0/475923835 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ae8100240 0x7f2ae8100640 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:31.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.166+0000 7f2aeec96640 1 -- 172.21.15.114:0/475923835 >> 172.21.15.114:0/475923835 conn(0x7f2ae80f9b70 msgr2=0x7f2ae80fbf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:31.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.166+0000 7f2aeec96640 1 -- 172.21.15.114:0/475923835 shutdown_connections 2024-09-13T18:22:31.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.166+0000 7f2aeec96640 1 -- 172.21.15.114:0/475923835 wait complete. 2024-09-13T18:22:31.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.166+0000 7f2aeec96640 1 Processor -- start 2024-09-13T18:22:31.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.166+0000 7f2aeec96640 1 -- start start 2024-09-13T18:22:31.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.166+0000 7f2aeec96640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ae80fee90 0x7f2ae80fe8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:31.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.166+0000 7f2aeec96640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ae80ff850 0x7f2ae819aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:31.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.166+0000 7f2aeec96640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ae8100240 0x7f2ae819d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:31.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.166+0000 7f2aeec96640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae819d870 con 0x7f2ae8100240 2024-09-13T18:22:31.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.166+0000 7f2aeec96640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae819d9e0 con 0x7f2ae80fee90 2024-09-13T18:22:31.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.166+0000 7f2aeec96640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae819dba0 con 0x7f2ae80ff850 2024-09-13T18:22:31.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.166+0000 7f2aed233640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ae8100240 0x7f2ae819d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:31.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.166+0000 7f2aeca32640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ae80fee90 0x7f2ae80fe8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:31.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.167+0000 7f2ae7fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ae80ff850 0x7f2ae819aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:31.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.167+0000 7f2aed233640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ae8100240 0x7f2ae819d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33308/0 (socket says 172.21.15.114:33308) 2024-09-13T18:22:31.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.167+0000 7f2aeca32640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ae80fee90 0x7f2ae80fe8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:53804/0 (socket says 172.21.15.114:53804) 2024-09-13T18:22:31.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.167+0000 7f2aed233640 1 -- 172.21.15.114:0/2521704552 learned_addr learned my addr 172.21.15.114:0/2521704552 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:31.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.167+0000 7f2aed233640 1 -- 172.21.15.114:0/2521704552 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ae80ff850 msgr2=0x7f2ae819aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:31.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.167+0000 7f2aed233640 1 --2- 172.21.15.114:0/2521704552 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ae80ff850 0x7f2ae819aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:31.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.167+0000 7f2aed233640 1 -- 172.21.15.114:0/2521704552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ae80fee90 msgr2=0x7f2ae80fe8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:31.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.167+0000 7f2aed233640 1 --2- 172.21.15.114:0/2521704552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ae80fee90 0x7f2ae80fe8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:31.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.167+0000 7f2aed233640 1 -- 172.21.15.114:0/2521704552 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad4009490 con 0x7f2ae8100240 2024-09-13T18:22:31.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.167+0000 7f2ae7fff640 1 --2- 172.21.15.114:0/2521704552 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ae80ff850 0x7f2ae819aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:22:31.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.167+0000 7f2aed233640 1 --2- 172.21.15.114:0/2521704552 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ae8100240 0x7f2ae819d090 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f2ae000efd0 tx=0x7f2ae000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:31.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.167+0000 7f2ae5ffb640 1 -- 172.21.15.114:0/2521704552 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae0009b40 con 0x7f2ae8100240 2024-09-13T18:22:31.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.167+0000 7f2ae5ffb640 1 -- 172.21.15.114:0/2521704552 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2ae0004020 con 0x7f2ae8100240 2024-09-13T18:22:31.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.167+0000 7f2aeec96640 1 -- 172.21.15.114:0/2521704552 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ae0009840 con 0x7f2ae8100240 2024-09-13T18:22:31.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.167+0000 7f2ae5ffb640 1 -- 172.21.15.114:0/2521704552 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae0010740 con 0x7f2ae8100240 2024-09-13T18:22:31.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.167+0000 7f2aeec96640 1 -- 172.21.15.114:0/2521704552 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2adc009560 con 0x7f2ae8100240 2024-09-13T18:22:31.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.168+0000 7f2aeec96640 1 -- 172.21.15.114:0/2521704552 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ae81062d0 con 0x7f2ae8100240 2024-09-13T18:22:31.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.169+0000 7f2ae5ffb640 1 -- 172.21.15.114:0/2521704552 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 23) v1 ==== 135994+0+0 (secure 0 0 0) 0x7f2ae0003b70 con 0x7f2ae8100240 2024-09-13T18:22:31.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.170+0000 7f2ae5ffb640 1 --2- 172.21.15.114:0/2521704552 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2ad80a1200 0x7f2ad80a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:31.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.170+0000 7f2ae5ffb640 1 -- 172.21.15.114:0/2521704552 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2ae0014070 con 0x7f2ae8100240 2024-09-13T18:22:31.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.170+0000 7f2aeca32640 1 --2- 172.21.15.114:0/2521704552 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2ad80a1200 0x7f2ad80a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:31.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.171+0000 7f2aeca32640 1 --2- 172.21.15.114:0/2521704552 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2ad80a1200 0x7f2ad80a36c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2adc006010 tx=0x7f2adc000bc0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:31.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.175+0000 7f2ae5ffb640 1 -- 172.21.15.114:0/2521704552 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f2ae0096760 con 0x7f2ae8100240 2024-09-13T18:22:31.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.350+0000 7f2aeec96640 1 -- 172.21.15.114:0/2521704552 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2ae810a720 con 0x7f2ae8100240 2024-09-13T18:22:31.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.350+0000 7f2ae5ffb640 1 -- 172.21.15.114:0/2521704552 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2ae0099d80 con 0x7f2ae8100240 2024-09-13T18:22:31.354 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 1 2024-09-13T18:22:31.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.352+0000 7f2aeec96640 1 -- 172.21.15.114:0/2521704552 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2ad80a1200 msgr2=0x7f2ad80a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:31.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.352+0000 7f2aeec96640 1 --2- 172.21.15.114:0/2521704552 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2ad80a1200 0x7f2ad80a36c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2adc006010 tx=0x7f2adc000bc0 comp rx=0 tx=0).stop 2024-09-13T18:22:31.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.352+0000 7f2aeec96640 1 -- 172.21.15.114:0/2521704552 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ae8100240 msgr2=0x7f2ae819d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:31.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.352+0000 7f2aeec96640 1 --2- 172.21.15.114:0/2521704552 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ae8100240 0x7f2ae819d090 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f2ae000efd0 tx=0x7f2ae000c6a0 comp rx=0 tx=0).stop 2024-09-13T18:22:31.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.353+0000 7f2aeec96640 1 -- 172.21.15.114:0/2521704552 shutdown_connections 2024-09-13T18:22:31.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.353+0000 7f2aeec96640 1 --2- 172.21.15.114:0/2521704552 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2ad80a1200 0x7f2ad80a36c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:31.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.353+0000 7f2aeec96640 1 --2- 172.21.15.114:0/2521704552 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ae8100240 0x7f2ae819d090 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:31.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.353+0000 7f2aeec96640 1 --2- 172.21.15.114:0/2521704552 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ae80ff850 0x7f2ae819aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:31.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.353+0000 7f2aeec96640 1 --2- 172.21.15.114:0/2521704552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ae80fee90 0x7f2ae80fe8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:31.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.353+0000 7f2aeec96640 1 -- 172.21.15.114:0/2521704552 >> 172.21.15.114:0/2521704552 conn(0x7f2ae80f9b70 msgr2=0x7f2ae81034a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:31.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.353+0000 7f2aeec96640 1 -- 172.21.15.114:0/2521704552 shutdown_connections 2024-09-13T18:22:31.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.353+0000 7f2aeec96640 1 -- 172.21.15.114:0/2521704552 wait complete. 2024-09-13T18:22:31.367 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:31 smithi114 ceph-mon[30897]: [13/Sep/2024:18:22:29] ENGINE Serving on https://172.21.15.114:7150 2024-09-13T18:22:31.367 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:31 smithi114 ceph-mon[30897]: [13/Sep/2024:18:22:29] ENGINE Error in HTTPServer.serve 2024-09-13T18:22:31.367 INFO:journalctl@ceph.mon.a.smithi114.stdout: Traceback (most recent call last): 2024-09-13T18:22:31.367 INFO:journalctl@ceph.mon.a.smithi114.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-13T18:22:31.367 INFO:journalctl@ceph.mon.a.smithi114.stdout: self._connections.run(self.expiration_interval) 2024-09-13T18:22:31.367 INFO:journalctl@ceph.mon.a.smithi114.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-13T18:22:31.367 INFO:journalctl@ceph.mon.a.smithi114.stdout: self._run(expiration_interval) 2024-09-13T18:22:31.367 INFO:journalctl@ceph.mon.a.smithi114.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-13T18:22:31.367 INFO:journalctl@ceph.mon.a.smithi114.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-13T18:22:31.367 INFO:journalctl@ceph.mon.a.smithi114.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-13T18:22:31.368 INFO:journalctl@ceph.mon.a.smithi114.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-13T18:22:31.368 INFO:journalctl@ceph.mon.a.smithi114.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-13T18:22:31.368 INFO:journalctl@ceph.mon.a.smithi114.stdout: s = self.context.wrap_socket( 2024-09-13T18:22:31.368 INFO:journalctl@ceph.mon.a.smithi114.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-13T18:22:31.368 INFO:journalctl@ceph.mon.a.smithi114.stdout: return self.sslsocket_class._create( 2024-09-13T18:22:31.368 INFO:journalctl@ceph.mon.a.smithi114.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-13T18:22:31.368 INFO:journalctl@ceph.mon.a.smithi114.stdout: self.do_handshake() 2024-09-13T18:22:31.368 INFO:journalctl@ceph.mon.a.smithi114.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-13T18:22:31.368 INFO:journalctl@ceph.mon.a.smithi114.stdout: self._sslobj.do_handshake() 2024-09-13T18:22:31.368 INFO:journalctl@ceph.mon.a.smithi114.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-13T18:22:31.368 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:31 smithi114 ceph-mon[30897]: [13/Sep/2024:18:22:29] ENGINE Serving on http://172.21.15.114:8765 2024-09-13T18:22:31.368 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:31 smithi114 ceph-mon[30897]: [13/Sep/2024:18:22:29] ENGINE Bus STARTED 2024-09-13T18:22:31.368 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:31 smithi114 ceph-mon[30897]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:22:31.369 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:31 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/1432676717' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T18:22:31.369 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:31 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/1359951022' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T18:22:31.369 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:31 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:31.369 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:31 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:31.369 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:31 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.369 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:31 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.369 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:31 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.369 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:31 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.369 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:31 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.369 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:31 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:31 smithi134 ceph-mon[34283]: [13/Sep/2024:18:22:29] ENGINE Serving on https://172.21.15.114:7150 2024-09-13T18:22:31.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:31 smithi134 ceph-mon[34283]: [13/Sep/2024:18:22:29] ENGINE Error in HTTPServer.serve 2024-09-13T18:22:31.370 INFO:journalctl@ceph.mon.b.smithi134.stdout: Traceback (most recent call last): 2024-09-13T18:22:31.370 INFO:journalctl@ceph.mon.b.smithi134.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-13T18:22:31.370 INFO:journalctl@ceph.mon.b.smithi134.stdout: self._connections.run(self.expiration_interval) 2024-09-13T18:22:31.371 INFO:journalctl@ceph.mon.b.smithi134.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-13T18:22:31.371 INFO:journalctl@ceph.mon.b.smithi134.stdout: self._run(expiration_interval) 2024-09-13T18:22:31.371 INFO:journalctl@ceph.mon.b.smithi134.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-13T18:22:31.371 INFO:journalctl@ceph.mon.b.smithi134.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-13T18:22:31.371 INFO:journalctl@ceph.mon.b.smithi134.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-13T18:22:31.371 INFO:journalctl@ceph.mon.b.smithi134.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-13T18:22:31.371 INFO:journalctl@ceph.mon.b.smithi134.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-13T18:22:31.371 INFO:journalctl@ceph.mon.b.smithi134.stdout: s = self.context.wrap_socket( 2024-09-13T18:22:31.372 INFO:journalctl@ceph.mon.b.smithi134.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-13T18:22:31.372 INFO:journalctl@ceph.mon.b.smithi134.stdout: return self.sslsocket_class._create( 2024-09-13T18:22:31.372 INFO:journalctl@ceph.mon.b.smithi134.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-13T18:22:31.372 INFO:journalctl@ceph.mon.b.smithi134.stdout: self.do_handshake() 2024-09-13T18:22:31.372 INFO:journalctl@ceph.mon.b.smithi134.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-13T18:22:31.372 INFO:journalctl@ceph.mon.b.smithi134.stdout: self._sslobj.do_handshake() 2024-09-13T18:22:31.372 INFO:journalctl@ceph.mon.b.smithi134.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-13T18:22:31.372 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:31 smithi134 ceph-mon[34283]: [13/Sep/2024:18:22:29] ENGINE Serving on http://172.21.15.114:8765 2024-09-13T18:22:31.372 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:31 smithi134 ceph-mon[34283]: [13/Sep/2024:18:22:29] ENGINE Bus STARTED 2024-09-13T18:22:31.372 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:31 smithi134 ceph-mon[34283]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:22:31.372 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:31 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/1432676717' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T18:22:31.372 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:31 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/1359951022' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T18:22:31.372 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:31 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:31.373 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:31 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:31.373 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:31 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.373 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:31 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.373 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:31 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.373 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:31 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.373 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:31 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.373 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:31 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.373 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:31 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.373 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:31 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.373 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:31 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:31.373 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:31 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:31.373 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:31 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.373 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:31 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.374 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:31 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.374 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:31 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.374 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:31 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.374 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:31 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.374 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:31 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.374 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:31 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.374 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:31 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:31.374 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:31 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:31.374 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:31 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.374 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:31 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.375 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:31 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.375 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:31 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.375 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:31 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.375 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:31 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.375 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:31 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.375 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:31 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.375 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:31 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:22:31.375 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:31 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:22:31.375 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:31 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/2558828298' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T18:22:31.375 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:31 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/696874907' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:22:31.375 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:31 smithi134 ceph-mon[34283]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-09-13T18:22:31.376 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T18:22:31.376 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:31 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.377 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:31 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.377 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:31 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:31.377 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:31 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:31.377 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:31 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.377 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:31 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.377 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:31 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.377 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:31 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.377 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:31 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.377 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:31 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.377 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:31 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.377 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:31 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.377 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:31 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:31.378 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:31 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:31.378 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:31 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.378 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:31 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.378 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:31 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.378 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:31 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.378 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:31 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.378 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:31 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.378 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:31 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.378 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:31 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.378 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:31 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:22:31.378 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:31 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:22:31.378 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:31 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2558828298' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T18:22:31.378 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:31 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/696874907' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:22:31.378 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:31 smithi114 ceph-mon[30897]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-09-13T18:22:31.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:31 smithi204 ceph-mon[34777]: [13/Sep/2024:18:22:29] ENGINE Serving on https://172.21.15.114:7150 2024-09-13T18:22:31.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:31 smithi204 ceph-mon[34777]: [13/Sep/2024:18:22:29] ENGINE Error in HTTPServer.serve 2024-09-13T18:22:31.445 INFO:journalctl@ceph.mon.c.smithi204.stdout: Traceback (most recent call last): 2024-09-13T18:22:31.445 INFO:journalctl@ceph.mon.c.smithi204.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-13T18:22:31.445 INFO:journalctl@ceph.mon.c.smithi204.stdout: self._connections.run(self.expiration_interval) 2024-09-13T18:22:31.445 INFO:journalctl@ceph.mon.c.smithi204.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-13T18:22:31.445 INFO:journalctl@ceph.mon.c.smithi204.stdout: self._run(expiration_interval) 2024-09-13T18:22:31.445 INFO:journalctl@ceph.mon.c.smithi204.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-13T18:22:31.445 INFO:journalctl@ceph.mon.c.smithi204.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-13T18:22:31.445 INFO:journalctl@ceph.mon.c.smithi204.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-13T18:22:31.445 INFO:journalctl@ceph.mon.c.smithi204.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-13T18:22:31.445 INFO:journalctl@ceph.mon.c.smithi204.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-13T18:22:31.445 INFO:journalctl@ceph.mon.c.smithi204.stdout: s = self.context.wrap_socket( 2024-09-13T18:22:31.445 INFO:journalctl@ceph.mon.c.smithi204.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-13T18:22:31.446 INFO:journalctl@ceph.mon.c.smithi204.stdout: return self.sslsocket_class._create( 2024-09-13T18:22:31.446 INFO:journalctl@ceph.mon.c.smithi204.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-13T18:22:31.446 INFO:journalctl@ceph.mon.c.smithi204.stdout: self.do_handshake() 2024-09-13T18:22:31.446 INFO:journalctl@ceph.mon.c.smithi204.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-13T18:22:31.446 INFO:journalctl@ceph.mon.c.smithi204.stdout: self._sslobj.do_handshake() 2024-09-13T18:22:31.446 INFO:journalctl@ceph.mon.c.smithi204.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-13T18:22:31.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:31 smithi204 ceph-mon[34777]: [13/Sep/2024:18:22:29] ENGINE Serving on http://172.21.15.114:8765 2024-09-13T18:22:31.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:31 smithi204 ceph-mon[34777]: [13/Sep/2024:18:22:29] ENGINE Bus STARTED 2024-09-13T18:22:31.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:31 smithi204 ceph-mon[34777]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:22:31.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:31 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/1432676717' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T18:22:31.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:31 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/1359951022' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T18:22:31.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:31 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:31.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:31 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:31.447 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:31 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.447 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:31 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.447 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:31 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.447 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:31 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.447 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:31 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.447 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:31 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.447 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:31 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.447 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:31 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.447 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:31 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:31.447 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:31 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:31.447 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:31 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.447 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:31 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.447 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:31 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.448 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:31 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.448 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:31 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.448 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:31 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.448 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:31 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.448 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:31 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.448 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:31 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:31.448 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:31 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:31.448 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:31 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.448 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:31 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.448 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:31 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.448 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:31 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.448 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:31 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.448 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:31 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.449 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:31 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.449 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:31 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:22:31.449 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:31 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:22:31.449 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:31 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:22:31.449 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:31 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/2558828298' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T18:22:31.449 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:31 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/696874907' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:22:31.449 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:31 smithi204 ceph-mon[34777]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-09-13T18:22:31.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.485+0000 7f7fe4cc9640 1 -- 172.21.15.114:0/2230751048 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7fe0103bf0 msgr2=0x7f7fe0103fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:31.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.485+0000 7f7fe4cc9640 1 --2- 172.21.15.114:0/2230751048 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7fe0103bf0 0x7f7fe0103fd0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0009900 tx=0x7f7fd002a410 comp rx=0 tx=0).stop 2024-09-13T18:22:31.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.486+0000 7f7fe4cc9640 1 -- 172.21.15.114:0/2230751048 shutdown_connections 2024-09-13T18:22:31.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.486+0000 7f7fe4cc9640 1 --2- 172.21.15.114:0/2230751048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7fe00ffd10 0x7f7fe010ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:31.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.486+0000 7f7fe4cc9640 1 --2- 172.21.15.114:0/2230751048 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7fe00ff390 0x7f7fe00ff7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:31.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.486+0000 7f7fe4cc9640 1 --2- 172.21.15.114:0/2230751048 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7fe0103bf0 0x7f7fe0103fd0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:31.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.486+0000 7f7fe4cc9640 1 -- 172.21.15.114:0/2230751048 >> 172.21.15.114:0/2230751048 conn(0x7f7fe00753f0 msgr2=0x7f7fe00fc740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:31.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.486+0000 7f7fe4cc9640 1 -- 172.21.15.114:0/2230751048 shutdown_connections 2024-09-13T18:22:31.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.486+0000 7f7fe4cc9640 1 -- 172.21.15.114:0/2230751048 wait complete. 2024-09-13T18:22:31.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.486+0000 7f7fe4cc9640 1 Processor -- start 2024-09-13T18:22:31.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.486+0000 7f7fe4cc9640 1 -- start start 2024-09-13T18:22:31.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.487+0000 7f7fe4cc9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7fe00ff390 0x7f7fe00fa800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:31.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.487+0000 7f7fe4cc9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7fe00ffd10 0x7f7fe00fad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:31.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.487+0000 7f7fe4cc9640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7fe0103bf0 0x7f7fe00fb280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:31.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.487+0000 7f7fe4cc9640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe01a45a0 con 0x7f7fe0103bf0 2024-09-13T18:22:31.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.487+0000 7f7fe4cc9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe01a4710 con 0x7f7fe00ff390 2024-09-13T18:22:31.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.487+0000 7f7fe4cc9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe01a48d0 con 0x7f7fe00ffd10 2024-09-13T18:22:31.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.487+0000 7f7fded9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7fe0103bf0 0x7f7fe00fb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:31.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.487+0000 7f7fded9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7fe0103bf0 0x7f7fe00fb280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33328/0 (socket says 172.21.15.114:33328) 2024-09-13T18:22:31.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.487+0000 7f7fde59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7fe00ff390 0x7f7fe00fa800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:31.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.487+0000 7f7fddd9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7fe00ffd10 0x7f7fe00fad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:31.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.487+0000 7f7fded9d640 1 -- 172.21.15.114:0/3827832745 learned_addr learned my addr 172.21.15.114:0/3827832745 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:31.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.487+0000 7f7fded9d640 1 -- 172.21.15.114:0/3827832745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7fe00ffd10 msgr2=0x7f7fe00fad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:31.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.487+0000 7f7fded9d640 1 --2- 172.21.15.114:0/3827832745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7fe00ffd10 0x7f7fe00fad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:31.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.487+0000 7f7fded9d640 1 -- 172.21.15.114:0/3827832745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7fe00ff390 msgr2=0x7f7fe00fa800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:31.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.487+0000 7f7fded9d640 1 --2- 172.21.15.114:0/3827832745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7fe00ff390 0x7f7fe00fa800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:31.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.487+0000 7f7fded9d640 1 -- 172.21.15.114:0/3827832745 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd0009560 con 0x7f7fe0103bf0 2024-09-13T18:22:31.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.487+0000 7f7fddd9b640 1 --2- 172.21.15.114:0/3827832745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7fe00ffd10 0x7f7fe00fad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:22:31.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.487+0000 7f7fded9d640 1 --2- 172.21.15.114:0/3827832745 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7fe0103bf0 0x7f7fe00fb280 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8009fd0 tx=0x7f7fd800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:31.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.487+0000 7f7fcf7fe640 1 -- 172.21.15.114:0/3827832745 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd800ce20 con 0x7f7fe0103bf0 2024-09-13T18:22:31.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.487+0000 7f7fcf7fe640 1 -- 172.21.15.114:0/3827832745 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7fd8005020 con 0x7f7fe0103bf0 2024-09-13T18:22:31.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.487+0000 7f7fcf7fe640 1 -- 172.21.15.114:0/3827832745 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd8010690 con 0x7f7fe0103bf0 2024-09-13T18:22:31.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.487+0000 7f7fe4cc9640 1 -- 172.21.15.114:0/3827832745 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fe01a4bb0 con 0x7f7fe0103bf0 2024-09-13T18:22:31.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.487+0000 7f7fe4cc9640 1 -- 172.21.15.114:0/3827832745 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fe01a5070 con 0x7f7fe0103bf0 2024-09-13T18:22:31.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.488+0000 7f7fe4cc9640 1 -- 172.21.15.114:0/3827832745 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fe01003a0 con 0x7f7fe0103bf0 2024-09-13T18:22:31.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.490+0000 7f7fcf7fe640 1 -- 172.21.15.114:0/3827832745 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 23) v1 ==== 135994+0+0 (secure 0 0 0) 0x7f7fd8005190 con 0x7f7fe0103bf0 2024-09-13T18:22:31.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.490+0000 7f7fcf7fe640 1 --2- 172.21.15.114:0/3827832745 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7fc80a1200 0x7f7fc80a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:31.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.490+0000 7f7fde59c640 1 --2- 172.21.15.114:0/3827832745 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7fc80a1200 0x7f7fc80a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:31.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.490+0000 7f7fcf7fe640 1 -- 172.21.15.114:0/3827832745 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f7fd8014070 con 0x7f7fe0103bf0 2024-09-13T18:22:31.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.491+0000 7f7fde59c640 1 --2- 172.21.15.114:0/3827832745 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7fc80a1200 0x7f7fc80a36c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7fd002aa00 tx=0x7f7fd0005a20 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:31.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.496+0000 7f7fcf7fe640 1 -- 172.21.15.114:0/3827832745 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f7fd8096a70 con 0x7f7fe0103bf0 2024-09-13T18:22:31.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.673+0000 7f7fe4cc9640 1 -- 172.21.15.114:0/3827832745 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7fe00ff7d0 con 0x7f7fe0103bf0 2024-09-13T18:22:31.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.673+0000 7f7fcf7fe640 1 -- 172.21.15.114:0/3827832745 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7fd809a090 con 0x7f7fe0103bf0 2024-09-13T18:22:31.677 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2024-09-13T18:22:31.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.675+0000 7f7fe4cc9640 1 -- 172.21.15.114:0/3827832745 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7fc80a1200 msgr2=0x7f7fc80a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:31.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.675+0000 7f7fe4cc9640 1 --2- 172.21.15.114:0/3827832745 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7fc80a1200 0x7f7fc80a36c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7fd002aa00 tx=0x7f7fd0005a20 comp rx=0 tx=0).stop 2024-09-13T18:22:31.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.675+0000 7f7fe4cc9640 1 -- 172.21.15.114:0/3827832745 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7fe0103bf0 msgr2=0x7f7fe00fb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:31.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.675+0000 7f7fe4cc9640 1 --2- 172.21.15.114:0/3827832745 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7fe0103bf0 0x7f7fe00fb280 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8009fd0 tx=0x7f7fd800ef10 comp rx=0 tx=0).stop 2024-09-13T18:22:31.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.675+0000 7f7fe4cc9640 1 -- 172.21.15.114:0/3827832745 shutdown_connections 2024-09-13T18:22:31.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.675+0000 7f7fe4cc9640 1 --2- 172.21.15.114:0/3827832745 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7fc80a1200 0x7f7fc80a36c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:31.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.675+0000 7f7fe4cc9640 1 --2- 172.21.15.114:0/3827832745 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7fe0103bf0 0x7f7fe00fb280 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:31.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.675+0000 7f7fe4cc9640 1 --2- 172.21.15.114:0/3827832745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7fe00ffd10 0x7f7fe00fad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:31.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.675+0000 7f7fe4cc9640 1 --2- 172.21.15.114:0/3827832745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7fe00ff390 0x7f7fe00fa800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:31.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.676+0000 7f7fe4cc9640 1 -- 172.21.15.114:0/3827832745 >> 172.21.15.114:0/3827832745 conn(0x7f7fe00753f0 msgr2=0x7f7fe00fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:31.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.676+0000 7f7fe4cc9640 1 -- 172.21.15.114:0/3827832745 shutdown_connections 2024-09-13T18:22:31.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.676+0000 7f7fe4cc9640 1 -- 172.21.15.114:0/3827832745 wait complete. 2024-09-13T18:22:31.689 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T18:22:31.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.790+0000 7f9fe874c640 1 -- 172.21.15.114:0/3415571181 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9fe0102180 msgr2=0x7f9fe0102560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:31.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.790+0000 7f9fe874c640 1 --2- 172.21.15.114:0/3415571181 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9fe0102180 0x7f9fe0102560 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4009930 tx=0x7f9fd402a420 comp rx=0 tx=0).stop 2024-09-13T18:22:31.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.791+0000 7f9fe874c640 1 -- 172.21.15.114:0/3415571181 shutdown_connections 2024-09-13T18:22:31.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.791+0000 7f9fe874c640 1 --2- 172.21.15.114:0/3415571181 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9fe01072d0 0x7f9fe010ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:31.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.791+0000 7f9fe874c640 1 --2- 172.21.15.114:0/3415571181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9fe0102b30 0x7f9fe0106b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:31.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.791+0000 7f9fe874c640 1 --2- 172.21.15.114:0/3415571181 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9fe0102180 0x7f9fe0102560 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:31.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.791+0000 7f9fe874c640 1 -- 172.21.15.114:0/3415571181 >> 172.21.15.114:0/3415571181 conn(0x7f9fe00fdb90 msgr2=0x7f9fe00fffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:31.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.791+0000 7f9fe874c640 1 -- 172.21.15.114:0/3415571181 shutdown_connections 2024-09-13T18:22:31.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.791+0000 7f9fe874c640 1 -- 172.21.15.114:0/3415571181 wait complete. 2024-09-13T18:22:31.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.791+0000 7f9fe874c640 1 Processor -- start 2024-09-13T18:22:31.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.791+0000 7f9fe874c640 1 -- start start 2024-09-13T18:22:31.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.791+0000 7f9fe874c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9fe0102180 0x7f9fe01984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:31.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.791+0000 7f9fe874c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9fe01072d0 0x7f9fe01989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:31.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.791+0000 7f9fe874c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9fe019cd90 0x7f9fe019d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:31.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.791+0000 7f9fe874c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe019d780 con 0x7f9fe0102180 2024-09-13T18:22:31.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.791+0000 7f9fe874c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe019d8f0 con 0x7f9fe01072d0 2024-09-13T18:22:31.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.791+0000 7f9fe874c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe019da60 con 0x7f9fe019cd90 2024-09-13T18:22:31.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.791+0000 7f9fe64e8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9fe0102180 0x7f9fe01984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:31.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.792+0000 7f9fe64e8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9fe0102180 0x7f9fe01984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33340/0 (socket says 172.21.15.114:33340) 2024-09-13T18:22:31.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.792+0000 7f9fe64e8640 1 -- 172.21.15.114:0/1309728805 learned_addr learned my addr 172.21.15.114:0/1309728805 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:31.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.792+0000 7f9fe5ce7640 1 --2- 172.21.15.114:0/1309728805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9fe01072d0 0x7f9fe01989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:31.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.792+0000 7f9fe6ce9640 1 --2- 172.21.15.114:0/1309728805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9fe019cd90 0x7f9fe019d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:31.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.792+0000 7f9fe64e8640 1 -- 172.21.15.114:0/1309728805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9fe019cd90 msgr2=0x7f9fe019d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:31.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.792+0000 7f9fe64e8640 1 --2- 172.21.15.114:0/1309728805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9fe019cd90 0x7f9fe019d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:31.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.792+0000 7f9fe64e8640 1 -- 172.21.15.114:0/1309728805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9fe01072d0 msgr2=0x7f9fe01989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:31.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.792+0000 7f9fe64e8640 1 --2- 172.21.15.114:0/1309728805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9fe01072d0 0x7f9fe01989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:31.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.792+0000 7f9fe64e8640 1 -- 172.21.15.114:0/1309728805 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd8009840 con 0x7f9fe0102180 2024-09-13T18:22:31.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.792+0000 7f9fe6ce9640 1 --2- 172.21.15.114:0/1309728805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9fe019cd90 0x7f9fe019d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:22:31.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.792+0000 7f9fe64e8640 1 --2- 172.21.15.114:0/1309728805 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9fe0102180 0x7f9fe01984b0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4005ab0 tx=0x7f9fd4032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:31.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.792+0000 7f9fd37fe640 1 -- 172.21.15.114:0/1309728805 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd40044f0 con 0x7f9fe0102180 2024-09-13T18:22:31.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.792+0000 7f9fd37fe640 1 -- 172.21.15.114:0/1309728805 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9fd4032e10 con 0x7f9fe0102180 2024-09-13T18:22:31.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.792+0000 7f9fe874c640 1 -- 172.21.15.114:0/1309728805 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fd4009560 con 0x7f9fe0102180 2024-09-13T18:22:31.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.792+0000 7f9fd37fe640 1 -- 172.21.15.114:0/1309728805 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd4003850 con 0x7f9fe0102180 2024-09-13T18:22:31.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.792+0000 7f9fe874c640 1 -- 172.21.15.114:0/1309728805 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fe01a5330 con 0x7f9fe0102180 2024-09-13T18:22:31.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.793+0000 7f9fe874c640 1 -- 172.21.15.114:0/1309728805 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fe0103650 con 0x7f9fe0102180 2024-09-13T18:22:31.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.794+0000 7f9fd37fe640 1 -- 172.21.15.114:0/1309728805 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 23) v1 ==== 135994+0+0 (secure 0 0 0) 0x7f9fd4005210 con 0x7f9fe0102180 2024-09-13T18:22:31.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.795+0000 7f9fd37fe640 1 --2- 172.21.15.114:0/1309728805 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f9fcc0a1280 0x7f9fcc0a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:31.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.795+0000 7f9fe5ce7640 1 --2- 172.21.15.114:0/1309728805 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f9fcc0a1280 0x7f9fcc0a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:31.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.795+0000 7f9fd37fe640 1 -- 172.21.15.114:0/1309728805 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f9fd40edb90 con 0x7f9fe0102180 2024-09-13T18:22:31.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.795+0000 7f9fe5ce7640 1 --2- 172.21.15.114:0/1309728805 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f9fcc0a1280 0x7f9fcc0a3740 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0199a00 tx=0x7f9fdc00b390 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:31.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.799+0000 7f9fd37fe640 1 -- 172.21.15.114:0/1309728805 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f9fd40b6c90 con 0x7f9fe0102180 2024-09-13T18:22:31.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.961+0000 7f9fe874c640 1 -- 172.21.15.114:0/1309728805 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9fe01997b0 con 0x7f9fe0102180 2024-09-13T18:22:31.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.962+0000 7f9fd37fe640 1 -- 172.21.15.114:0/1309728805 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9fd40ba2b0 con 0x7f9fe0102180 2024-09-13T18:22:31.965 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 64 2024-09-13T18:22:31.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.964+0000 7f9fe874c640 1 -- 172.21.15.114:0/1309728805 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f9fcc0a1280 msgr2=0x7f9fcc0a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:31.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.964+0000 7f9fe874c640 1 --2- 172.21.15.114:0/1309728805 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f9fcc0a1280 0x7f9fcc0a3740 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0199a00 tx=0x7f9fdc00b390 comp rx=0 tx=0).stop 2024-09-13T18:22:31.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.964+0000 7f9fe874c640 1 -- 172.21.15.114:0/1309728805 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9fe0102180 msgr2=0x7f9fe01984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:31.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.964+0000 7f9fe874c640 1 --2- 172.21.15.114:0/1309728805 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9fe0102180 0x7f9fe01984b0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4005ab0 tx=0x7f9fd4032c20 comp rx=0 tx=0).stop 2024-09-13T18:22:31.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.964+0000 7f9fe874c640 1 -- 172.21.15.114:0/1309728805 shutdown_connections 2024-09-13T18:22:31.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.964+0000 7f9fe874c640 1 --2- 172.21.15.114:0/1309728805 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f9fcc0a1280 0x7f9fcc0a3740 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:31.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.964+0000 7f9fe874c640 1 --2- 172.21.15.114:0/1309728805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9fe019cd90 0x7f9fe019d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:31.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.964+0000 7f9fe874c640 1 --2- 172.21.15.114:0/1309728805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9fe01072d0 0x7f9fe01989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:31.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.964+0000 7f9fe874c640 1 --2- 172.21.15.114:0/1309728805 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9fe0102180 0x7f9fe01984b0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:31.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.964+0000 7f9fe874c640 1 -- 172.21.15.114:0/1309728805 >> 172.21.15.114:0/1309728805 conn(0x7f9fe00fdb90 msgr2=0x7f9fe00fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:31.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.964+0000 7f9fe874c640 1 -- 172.21.15.114:0/1309728805 shutdown_connections 2024-09-13T18:22:31.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:31.964+0000 7f9fe874c640 1 -- 172.21.15.114:0/1309728805 wait complete. 2024-09-13T18:22:31.978 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T18:22:32.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.076+0000 7f310d0d7640 1 -- 172.21.15.114:0/3847355028 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31080fee60 msgr2=0x7f31080ff240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:32.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.076+0000 7f310d0d7640 1 --2- 172.21.15.114:0/3847355028 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31080fee60 0x7f31080ff240 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f30f8009930 tx=0x7f30f802a420 comp rx=0 tx=0).stop 2024-09-13T18:22:32.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.076+0000 7f310d0d7640 1 -- 172.21.15.114:0/3847355028 shutdown_connections 2024-09-13T18:22:32.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.076+0000 7f310d0d7640 1 --2- 172.21.15.114:0/3847355028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3108103fb0 0x7f310810bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:32.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.076+0000 7f310d0d7640 1 --2- 172.21.15.114:0/3847355028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31080ff810 0x7f3108103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:32.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.076+0000 7f310d0d7640 1 --2- 172.21.15.114:0/3847355028 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31080fee60 0x7f31080ff240 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:32.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.076+0000 7f310d0d7640 1 -- 172.21.15.114:0/3847355028 >> 172.21.15.114:0/3847355028 conn(0x7f3108075c60 msgr2=0x7f31080fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:32.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.077+0000 7f310d0d7640 1 -- 172.21.15.114:0/3847355028 shutdown_connections 2024-09-13T18:22:32.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.077+0000 7f310d0d7640 1 -- 172.21.15.114:0/3847355028 wait complete. 2024-09-13T18:22:32.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.077+0000 7f310d0d7640 1 Processor -- start 2024-09-13T18:22:32.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.077+0000 7f310d0d7640 1 -- start start 2024-09-13T18:22:32.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.078+0000 7f310d0d7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31080fee60 0x7f310810c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:32.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.078+0000 7f310d0d7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3108103fb0 0x7f310810c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:32.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.078+0000 7f310d0d7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f310810ce90 0x7f31081a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:32.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.078+0000 7f310d0d7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31081a49f0 con 0x7f310810ce90 2024-09-13T18:22:32.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.078+0000 7f310d0d7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31081a4b60 con 0x7f31080fee60 2024-09-13T18:22:32.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.078+0000 7f310d0d7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31081a4d20 con 0x7f3108103fb0 2024-09-13T18:22:32.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.078+0000 7f310759e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f310810ce90 0x7f31081a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:32.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.078+0000 7f310759e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f310810ce90 0x7f31081a4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33370/0 (socket says 172.21.15.114:33370) 2024-09-13T18:22:32.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.078+0000 7f310759e640 1 -- 172.21.15.114:0/2617702121 learned_addr learned my addr 172.21.15.114:0/2617702121 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:32.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.078+0000 7f310659c640 1 --2- 172.21.15.114:0/2617702121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3108103fb0 0x7f310810c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:32.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.078+0000 7f3106d9d640 1 --2- 172.21.15.114:0/2617702121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31080fee60 0x7f310810c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:32.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.078+0000 7f310759e640 1 -- 172.21.15.114:0/2617702121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3108103fb0 msgr2=0x7f310810c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:32.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.078+0000 7f310759e640 1 --2- 172.21.15.114:0/2617702121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3108103fb0 0x7f310810c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:32.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.078+0000 7f310759e640 1 -- 172.21.15.114:0/2617702121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31080fee60 msgr2=0x7f310810c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:32.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.078+0000 7f310759e640 1 --2- 172.21.15.114:0/2617702121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31080fee60 0x7f310810c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:32.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.078+0000 7f310759e640 1 -- 172.21.15.114:0/2617702121 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f8009560 con 0x7f310810ce90 2024-09-13T18:22:32.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.078+0000 7f310759e640 1 --2- 172.21.15.114:0/2617702121 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f310810ce90 0x7f31081a4420 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f310000ea60 tx=0x7f310000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:32.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.078+0000 7f30effff640 1 -- 172.21.15.114:0/2617702121 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310000ce40 con 0x7f310810ce90 2024-09-13T18:22:32.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.078+0000 7f30effff640 1 -- 172.21.15.114:0/2617702121 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3100005020 con 0x7f310810ce90 2024-09-13T18:22:32.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.078+0000 7f310d0d7640 1 -- 172.21.15.114:0/2617702121 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31081a5000 con 0x7f310810ce90 2024-09-13T18:22:32.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.078+0000 7f30effff640 1 -- 172.21.15.114:0/2617702121 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3100010690 con 0x7f310810ce90 2024-09-13T18:22:32.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.078+0000 7f310d0d7640 1 -- 172.21.15.114:0/2617702121 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31081a5490 con 0x7f310810ce90 2024-09-13T18:22:32.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.079+0000 7f310d0d7640 1 -- 172.21.15.114:0/2617702121 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3108100330 con 0x7f310810ce90 2024-09-13T18:22:32.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.080+0000 7f30effff640 1 -- 172.21.15.114:0/2617702121 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 23) v1 ==== 135994+0+0 (secure 0 0 0) 0x7f3100005190 con 0x7f310810ce90 2024-09-13T18:22:32.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.081+0000 7f30effff640 1 --2- 172.21.15.114:0/2617702121 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f30f00a12b0 0x7f30f00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:32.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.081+0000 7f3106d9d640 1 --2- 172.21.15.114:0/2617702121 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f30f00a12b0 0x7f30f00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:32.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.081+0000 7f30effff640 1 -- 172.21.15.114:0/2617702121 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f3100014070 con 0x7f310810ce90 2024-09-13T18:22:32.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.082+0000 7f3106d9d640 1 --2- 172.21.15.114:0/2617702121 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f30f00a12b0 0x7f30f00a3770 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f30f80057c0 tx=0x7f30f8005a20 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:32.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.085+0000 7f30effff640 1 -- 172.21.15.114:0/2617702121 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f3100096560 con 0x7f310810ce90 2024-09-13T18:22:32.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.246+0000 7f310d0d7640 1 -- 172.21.15.114:0/2617702121 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f3108107160 con 0x7f310810ce90 2024-09-13T18:22:32.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.247+0000 7f30effff640 1 -- 172.21.15.114:0/2617702121 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) v1 ==== 103+0+11 (secure 0 0 0) 0x7f3100099b80 con 0x7f310810ce90 2024-09-13T18:22:32.250 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2024-09-13T18:22:32.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.249+0000 7f310d0d7640 1 -- 172.21.15.114:0/2617702121 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f30f00a12b0 msgr2=0x7f30f00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:32.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.249+0000 7f310d0d7640 1 --2- 172.21.15.114:0/2617702121 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f30f00a12b0 0x7f30f00a3770 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f30f80057c0 tx=0x7f30f8005a20 comp rx=0 tx=0).stop 2024-09-13T18:22:32.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.249+0000 7f310d0d7640 1 -- 172.21.15.114:0/2617702121 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f310810ce90 msgr2=0x7f31081a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:32.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.249+0000 7f310d0d7640 1 --2- 172.21.15.114:0/2617702121 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f310810ce90 0x7f31081a4420 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f310000ea60 tx=0x7f310000ef20 comp rx=0 tx=0).stop 2024-09-13T18:22:32.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.249+0000 7f310d0d7640 1 -- 172.21.15.114:0/2617702121 shutdown_connections 2024-09-13T18:22:32.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.249+0000 7f310d0d7640 1 --2- 172.21.15.114:0/2617702121 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f30f00a12b0 0x7f30f00a3770 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:32.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.249+0000 7f310d0d7640 1 --2- 172.21.15.114:0/2617702121 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f310810ce90 0x7f31081a4420 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:32.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.249+0000 7f310d0d7640 1 --2- 172.21.15.114:0/2617702121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3108103fb0 0x7f310810c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:32.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.249+0000 7f310d0d7640 1 --2- 172.21.15.114:0/2617702121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31080fee60 0x7f310810c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:32.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.249+0000 7f310d0d7640 1 -- 172.21.15.114:0/2617702121 >> 172.21.15.114:0/2617702121 conn(0x7f3108075c60 msgr2=0x7f3108102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:32.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.249+0000 7f310d0d7640 1 -- 172.21.15.114:0/2617702121 shutdown_connections 2024-09-13T18:22:32.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.249+0000 7f310d0d7640 1 -- 172.21.15.114:0/2617702121 wait complete. 2024-09-13T18:22:32.262 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:32 smithi114 ceph-mon[30897]: Adjusting osd_memory_target on smithi204 to 176.0M 2024-09-13T18:22:32.262 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:32 smithi114 ceph-mon[30897]: Unable to set osd_memory_target on smithi204 to 184552345: error parsing value: Value '184552345' is below minimum 939524096 2024-09-13T18:22:32.262 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:32 smithi114 ceph-mon[30897]: Adjusting osd_memory_target on smithi134 to 176.0M 2024-09-13T18:22:32.262 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:32 smithi114 ceph-mon[30897]: Unable to set osd_memory_target on smithi134 to 184585318: error parsing value: Value '184585318' is below minimum 939524096 2024-09-13T18:22:32.263 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:32 smithi114 ceph-mon[30897]: Adjusting osd_memory_target on smithi114 to 176.0M 2024-09-13T18:22:32.263 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:32 smithi114 ceph-mon[30897]: Unable to set osd_memory_target on smithi114 to 184563814: error parsing value: Value '184563814' is below minimum 939524096 2024-09-13T18:22:32.263 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:32 smithi114 ceph-mon[30897]: Updating smithi114:/etc/ceph/ceph.conf 2024-09-13T18:22:32.263 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:32 smithi114 ceph-mon[30897]: Updating smithi134:/etc/ceph/ceph.conf 2024-09-13T18:22:32.263 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:32 smithi114 ceph-mon[30897]: Updating smithi204:/etc/ceph/ceph.conf 2024-09-13T18:22:32.263 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:32 smithi114 ceph-mon[30897]: Updating smithi114:/var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T18:22:32.263 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:32 smithi114 ceph-mon[30897]: Updating smithi204:/var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T18:22:32.263 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:32 smithi114 ceph-mon[30897]: Updating smithi134:/var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T18:22:32.263 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:32 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2521704552' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T18:22:32.263 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:32 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:32.263 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:32 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:32.264 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:32 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:32.264 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:32 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:32.264 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:32 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:32.264 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:32 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:32.264 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:32 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:32.264 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:32 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/3827832745' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T18:22:32.264 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:32 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:22:32.264 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:32 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:22:32.264 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:32 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:22:32.264 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:32 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:32.264 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:32 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/1309728805' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T18:22:32.265 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-13T18:22:32.265 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:22:32.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.361+0000 7f86e3c61640 1 -- 172.21.15.114:0/2423375191 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86dc101860 msgr2=0x7f86dc109d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:32.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.361+0000 7f86e3c61640 1 --2- 172.21.15.114:0/2423375191 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86dc101860 0x7f86dc109d40 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f86cc009900 tx=0x7f86cc02a480 comp rx=0 tx=0).stop 2024-09-13T18:22:32.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.361+0000 7f86e3c61640 1 -- 172.21.15.114:0/2423375191 shutdown_connections 2024-09-13T18:22:32.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.361+0000 7f86e3c61640 1 --2- 172.21.15.114:0/2423375191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86dc10a310 0x7f86dc10c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:32.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.361+0000 7f86e3c61640 1 --2- 172.21.15.114:0/2423375191 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86dc101860 0x7f86dc109d40 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:32.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.361+0000 7f86e3c61640 1 --2- 172.21.15.114:0/2423375191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86dc100f40 0x7f86dc101320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:32.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.361+0000 7f86e3c61640 1 -- 172.21.15.114:0/2423375191 >> 172.21.15.114:0/2423375191 conn(0x7f86dc075c60 msgr2=0x7f86dc0fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:32.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.361+0000 7f86e3c61640 1 -- 172.21.15.114:0/2423375191 shutdown_connections 2024-09-13T18:22:32.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.361+0000 7f86e3c61640 1 -- 172.21.15.114:0/2423375191 wait complete. 2024-09-13T18:22:32.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.362+0000 7f86e3c61640 1 Processor -- start 2024-09-13T18:22:32.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.362+0000 7f86e3c61640 1 -- start start 2024-09-13T18:22:32.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.362+0000 7f86e3c61640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86dc100f40 0x7f86dc196220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:32.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.362+0000 7f86e3c61640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86dc101860 0x7f86dc196760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:32.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.362+0000 7f86e3c61640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86dc10a310 0x7f86dc19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:32.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.362+0000 7f86e3c61640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc19b150 con 0x7f86dc100f40 2024-09-13T18:22:32.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.362+0000 7f86e3c61640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc19b2c0 con 0x7f86dc101860 2024-09-13T18:22:32.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.362+0000 7f86e3c61640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc19b480 con 0x7f86dc10a310 2024-09-13T18:22:32.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.362+0000 7f86e19fd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86dc100f40 0x7f86dc196220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:32.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.362+0000 7f86e19fd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86dc100f40 0x7f86dc196220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33400/0 (socket says 172.21.15.114:33400) 2024-09-13T18:22:32.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.362+0000 7f86e19fd640 1 -- 172.21.15.114:0/1850740749 learned_addr learned my addr 172.21.15.114:0/1850740749 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:32.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.362+0000 7f86e11fc640 1 --2- 172.21.15.114:0/1850740749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86dc101860 0x7f86dc196760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:32.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.363+0000 7f86e19fd640 1 -- 172.21.15.114:0/1850740749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86dc10a310 msgr2=0x7f86dc19ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:32.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.363+0000 7f86e19fd640 1 --2- 172.21.15.114:0/1850740749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86dc10a310 0x7f86dc19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:32.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.363+0000 7f86e19fd640 1 -- 172.21.15.114:0/1850740749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86dc101860 msgr2=0x7f86dc196760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:32.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.363+0000 7f86e19fd640 1 --2- 172.21.15.114:0/1850740749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86dc101860 0x7f86dc196760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:32.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.363+0000 7f86e19fd640 1 -- 172.21.15.114:0/1850740749 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86cc009560 con 0x7f86dc100f40 2024-09-13T18:22:32.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.363+0000 7f86e11fc640 1 --2- 172.21.15.114:0/1850740749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86dc101860 0x7f86dc196760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:22:32.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.363+0000 7f86e19fd640 1 --2- 172.21.15.114:0/1850740749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86dc100f40 0x7f86dc196220 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f86d800c970 tx=0x7f86d800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:32.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.363+0000 7f86d2ffd640 1 -- 172.21.15.114:0/1850740749 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d8007bb0 con 0x7f86dc100f40 2024-09-13T18:22:32.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.363+0000 7f86d2ffd640 1 -- 172.21.15.114:0/1850740749 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f86d8007d10 con 0x7f86dc100f40 2024-09-13T18:22:32.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.363+0000 7f86d2ffd640 1 -- 172.21.15.114:0/1850740749 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d8004d70 con 0x7f86dc100f40 2024-09-13T18:22:32.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.363+0000 7f86e3c61640 1 -- 172.21.15.114:0/1850740749 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86dc19b760 con 0x7f86dc100f40 2024-09-13T18:22:32.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.363+0000 7f86e3c61640 1 -- 172.21.15.114:0/1850740749 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86dc1a2fe0 con 0x7f86dc100f40 2024-09-13T18:22:32.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.364+0000 7f86e3c61640 1 -- 172.21.15.114:0/1850740749 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86dc067e20 con 0x7f86dc100f40 2024-09-13T18:22:32.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:32 smithi134 ceph-mon[34283]: Adjusting osd_memory_target on smithi204 to 176.0M 2024-09-13T18:22:32.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:32 smithi134 ceph-mon[34283]: Unable to set osd_memory_target on smithi204 to 184552345: error parsing value: Value '184552345' is below minimum 939524096 2024-09-13T18:22:32.371 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:32 smithi134 ceph-mon[34283]: Adjusting osd_memory_target on smithi134 to 176.0M 2024-09-13T18:22:32.371 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:32 smithi134 ceph-mon[34283]: Unable to set osd_memory_target on smithi134 to 184585318: error parsing value: Value '184585318' is below minimum 939524096 2024-09-13T18:22:32.371 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:32 smithi134 ceph-mon[34283]: Adjusting osd_memory_target on smithi114 to 176.0M 2024-09-13T18:22:32.371 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:32 smithi134 ceph-mon[34283]: Unable to set osd_memory_target on smithi114 to 184563814: error parsing value: Value '184563814' is below minimum 939524096 2024-09-13T18:22:32.371 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:32 smithi134 ceph-mon[34283]: Updating smithi114:/etc/ceph/ceph.conf 2024-09-13T18:22:32.371 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:32 smithi134 ceph-mon[34283]: Updating smithi134:/etc/ceph/ceph.conf 2024-09-13T18:22:32.371 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:32 smithi134 ceph-mon[34283]: Updating smithi204:/etc/ceph/ceph.conf 2024-09-13T18:22:32.371 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:32 smithi134 ceph-mon[34283]: Updating smithi114:/var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T18:22:32.371 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:32 smithi134 ceph-mon[34283]: Updating smithi204:/var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T18:22:32.371 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:32 smithi134 ceph-mon[34283]: Updating smithi134:/var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T18:22:32.371 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:32 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/2521704552' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T18:22:32.371 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:32 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:32.372 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:32 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:32.372 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:32 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:32.372 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:32 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:32.372 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:32 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:32.372 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:32 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:32.372 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:32 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:32.372 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:32 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/3827832745' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T18:22:32.372 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:32 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:22:32.372 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:32 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:22:32.372 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:32 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:22:32.372 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:32 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:32.372 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:32 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/1309728805' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T18:22:32.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.365+0000 7f86d2ffd640 1 -- 172.21.15.114:0/1850740749 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 23) v1 ==== 135994+0+0 (secure 0 0 0) 0x7f86d8004f30 con 0x7f86dc100f40 2024-09-13T18:22:32.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.366+0000 7f86d2ffd640 1 --2- 172.21.15.114:0/1850740749 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f86c80a1280 0x7f86c80a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:32.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.366+0000 7f86d2ffd640 1 -- 172.21.15.114:0/1850740749 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f86d80cd1a0 con 0x7f86dc100f40 2024-09-13T18:22:32.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.366+0000 7f86e11fc640 1 --2- 172.21.15.114:0/1850740749 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f86c80a1280 0x7f86c80a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:32.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.366+0000 7f86e11fc640 1 --2- 172.21.15.114:0/1850740749 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f86c80a1280 0x7f86c80a3740 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f86cc000c00 tx=0x7f86cc005e00 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:32.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.369+0000 7f86d2ffd640 1 -- 172.21.15.114:0/1850740749 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f86d8096320 con 0x7f86dc100f40 2024-09-13T18:22:32.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:32 smithi204 ceph-mon[34777]: Adjusting osd_memory_target on smithi204 to 176.0M 2024-09-13T18:22:32.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:32 smithi204 ceph-mon[34777]: Unable to set osd_memory_target on smithi204 to 184552345: error parsing value: Value '184552345' is below minimum 939524096 2024-09-13T18:22:32.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:32 smithi204 ceph-mon[34777]: Adjusting osd_memory_target on smithi134 to 176.0M 2024-09-13T18:22:32.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:32 smithi204 ceph-mon[34777]: Unable to set osd_memory_target on smithi134 to 184585318: error parsing value: Value '184585318' is below minimum 939524096 2024-09-13T18:22:32.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:32 smithi204 ceph-mon[34777]: Adjusting osd_memory_target on smithi114 to 176.0M 2024-09-13T18:22:32.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:32 smithi204 ceph-mon[34777]: Unable to set osd_memory_target on smithi114 to 184563814: error parsing value: Value '184563814' is below minimum 939524096 2024-09-13T18:22:32.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:32 smithi204 ceph-mon[34777]: Updating smithi114:/etc/ceph/ceph.conf 2024-09-13T18:22:32.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:32 smithi204 ceph-mon[34777]: Updating smithi134:/etc/ceph/ceph.conf 2024-09-13T18:22:32.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:32 smithi204 ceph-mon[34777]: Updating smithi204:/etc/ceph/ceph.conf 2024-09-13T18:22:32.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:32 smithi204 ceph-mon[34777]: Updating smithi114:/var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T18:22:32.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:32 smithi204 ceph-mon[34777]: Updating smithi204:/var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T18:22:32.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:32 smithi204 ceph-mon[34777]: Updating smithi134:/var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-13T18:22:32.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:32 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/2521704552' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T18:22:32.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:32 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:32.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:32 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:32.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:32 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:32.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:32 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:32.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:32 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:32.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:32 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:32.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:32 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:32.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:32 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/3827832745' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T18:22:32.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:32 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:22:32.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:32 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:22:32.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:32 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:22:32.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:32 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:32.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:32 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/1309728805' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T18:22:32.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.569+0000 7f86e3c61640 1 -- 172.21.15.114:0/1850740749 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f86dc101320 con 0x7f86dc100f40 2024-09-13T18:22:32.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.570+0000 7f86d2ffd640 1 -- 172.21.15.114:0/1850740749 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+10930 (secure 0 0 0) 0x7f86d801d070 con 0x7f86dc100f40 2024-09-13T18:22:32.573 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:22:32.573 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4162817497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4162817497},{"type":"v1","addr":"172.21.15.114:6835","nonce":4162817497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/2435428478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":2435428478},{"type":"v1","addr":"172.21.15.114:6837","nonce":2435428478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T18:20:01.895117+0000","modified":"2024-09-13T18:22:29.064231+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24482,"mds_2":24475,"mds_3":24458,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/2046516451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":2046516451},{"type":"v1","addr":"172.21.15.114:6839","nonce":2046516451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":10,"addr":"172.21.15.114:6841/1956923446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":1956923446},{"type":"v1","addr":"172.21.15.114:6841","nonce":1956923446}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24391":{"gid":24391,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/507301496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":507301496},{"type":"v1","addr":"172.21.15.204:6833","nonce":507301496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4028657219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4028657219},{"type":"v1","addr":"172.21.15.134:6833","nonce":4028657219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/57029119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":57029119},{"type":"v1","addr":"172.21.15.204:6835","nonce":57029119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3068880939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3068880939},{"type":"v1","addr":"172.21.15.204:6837","nonce":3068880939}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"e","rank":3,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.134:6835/1150003275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1150003275},{"type":"v1","addr":"172.21.15.134:6835","nonce":1150003275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/344883912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":344883912},{"type":"v1","addr":"172.21.15.204:6839","nonce":344883912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6837/891692210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":891692210},{"type":"v1","addr":"172.21.15.134:6837","nonce":891692210}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6839/3466309724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3466309724},{"type":"v1","addr":"172.21.15.134:6839","nonce":3466309724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:22:32.573 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 30 2024-09-13T18:22:32.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.572+0000 7f86e3c61640 1 -- 172.21.15.114:0/1850740749 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f86c80a1280 msgr2=0x7f86c80a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:32.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.572+0000 7f86e3c61640 1 --2- 172.21.15.114:0/1850740749 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f86c80a1280 0x7f86c80a3740 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f86cc000c00 tx=0x7f86cc005e00 comp rx=0 tx=0).stop 2024-09-13T18:22:32.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.572+0000 7f86e3c61640 1 -- 172.21.15.114:0/1850740749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86dc100f40 msgr2=0x7f86dc196220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:32.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.572+0000 7f86e3c61640 1 --2- 172.21.15.114:0/1850740749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86dc100f40 0x7f86dc196220 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f86d800c970 tx=0x7f86d800ce30 comp rx=0 tx=0).stop 2024-09-13T18:22:32.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.572+0000 7f86e3c61640 1 -- 172.21.15.114:0/1850740749 shutdown_connections 2024-09-13T18:22:32.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.572+0000 7f86e3c61640 1 --2- 172.21.15.114:0/1850740749 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f86c80a1280 0x7f86c80a3740 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:32.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.572+0000 7f86e3c61640 1 --2- 172.21.15.114:0/1850740749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86dc10a310 0x7f86dc19ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:32.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.572+0000 7f86e3c61640 1 --2- 172.21.15.114:0/1850740749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86dc101860 0x7f86dc196760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:32.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.572+0000 7f86e3c61640 1 --2- 172.21.15.114:0/1850740749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86dc100f40 0x7f86dc196220 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:32.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.572+0000 7f86e3c61640 1 -- 172.21.15.114:0/1850740749 >> 172.21.15.114:0/1850740749 conn(0x7f86dc075c60 msgr2=0x7f86dc0fb690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:32.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.572+0000 7f86e3c61640 1 -- 172.21.15.114:0/1850740749 shutdown_connections 2024-09-13T18:22:32.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.572+0000 7f86e3c61640 1 -- 172.21.15.114:0/1850740749 wait complete. 2024-09-13T18:22:32.586 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-13T18:22:32.586 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-13T18:22:32.586 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:22:32.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.682+0000 7fc0138be640 1 -- 172.21.15.114:0/4283302216 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc00c0fff80 msgr2=0x7fc00c100360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:32.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.682+0000 7fc0138be640 1 --2- 172.21.15.114:0/4283302216 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc00c0fff80 0x7fc00c100360 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc004009930 tx=0x7fc00402a420 comp rx=0 tx=0).stop 2024-09-13T18:22:32.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.682+0000 7fc0138be640 1 -- 172.21.15.114:0/4283302216 shutdown_connections 2024-09-13T18:22:32.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.682+0000 7fc0138be640 1 --2- 172.21.15.114:0/4283302216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc00c1050d0 0x7fc00c10cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:32.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.682+0000 7fc0138be640 1 --2- 172.21.15.114:0/4283302216 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc00c100930 0x7fc00c104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:32.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.682+0000 7fc0138be640 1 --2- 172.21.15.114:0/4283302216 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc00c0fff80 0x7fc00c100360 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:32.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.682+0000 7fc0138be640 1 -- 172.21.15.114:0/4283302216 >> 172.21.15.114:0/4283302216 conn(0x7fc00c0fba10 msgr2=0x7fc00c0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:32.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.682+0000 7fc0138be640 1 -- 172.21.15.114:0/4283302216 shutdown_connections 2024-09-13T18:22:32.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.682+0000 7fc0138be640 1 -- 172.21.15.114:0/4283302216 wait complete. 2024-09-13T18:22:32.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.683+0000 7fc0138be640 1 Processor -- start 2024-09-13T18:22:32.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.683+0000 7fc0138be640 1 -- start start 2024-09-13T18:22:32.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.683+0000 7fc0138be640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc00c0fff80 0x7fc00c196330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:32.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.683+0000 7fc0138be640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc00c1050d0 0x7fc00c196870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:32.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.683+0000 7fc0138be640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc00c19ac60 0x7fc00c19b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:32.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.683+0000 7fc0138be640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00c19b650 con 0x7fc00c1050d0 2024-09-13T18:22:32.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.683+0000 7fc0138be640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00c19b7c0 con 0x7fc00c0fff80 2024-09-13T18:22:32.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.683+0000 7fc0138be640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00c19b930 con 0x7fc00c19ac60 2024-09-13T18:22:32.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.683+0000 7fc010e59640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc00c1050d0 0x7fc00c196870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:32.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.683+0000 7fc010e59640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc00c1050d0 0x7fc00c196870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33424/0 (socket says 172.21.15.114:33424) 2024-09-13T18:22:32.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.683+0000 7fc010e59640 1 -- 172.21.15.114:0/2055052778 learned_addr learned my addr 172.21.15.114:0/2055052778 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:32.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.683+0000 7fc011e5b640 1 --2- 172.21.15.114:0/2055052778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc00c19ac60 0x7fc00c19b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:32.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.683+0000 7fc01165a640 1 --2- 172.21.15.114:0/2055052778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc00c0fff80 0x7fc00c196330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:32.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.683+0000 7fc010e59640 1 -- 172.21.15.114:0/2055052778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc00c19ac60 msgr2=0x7fc00c19b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:32.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.683+0000 7fc010e59640 1 --2- 172.21.15.114:0/2055052778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc00c19ac60 0x7fc00c19b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:32.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.683+0000 7fc010e59640 1 -- 172.21.15.114:0/2055052778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc00c0fff80 msgr2=0x7fc00c196330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:32.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.683+0000 7fc010e59640 1 --2- 172.21.15.114:0/2055052778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc00c0fff80 0x7fc00c196330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:32.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.683+0000 7fc010e59640 1 -- 172.21.15.114:0/2055052778 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc004009560 con 0x7fc00c1050d0 2024-09-13T18:22:32.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.683+0000 7fc010e59640 1 --2- 172.21.15.114:0/2055052778 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc00c1050d0 0x7fc00c196870 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc00000b450 tx=0x7fc00000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:32.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.684+0000 7fc00a7fc640 1 -- 172.21.15.114:0/2055052778 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc000011890 con 0x7fc00c1050d0 2024-09-13T18:22:32.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.684+0000 7fc0138be640 1 -- 172.21.15.114:0/2055052778 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc00c1a2f10 con 0x7fc00c1050d0 2024-09-13T18:22:32.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.684+0000 7fc0138be640 1 -- 172.21.15.114:0/2055052778 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc00c1a3340 con 0x7fc00c1050d0 2024-09-13T18:22:32.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.684+0000 7fc00a7fc640 1 -- 172.21.15.114:0/2055052778 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc000011e80 con 0x7fc00c1050d0 2024-09-13T18:22:32.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.684+0000 7fc00a7fc640 1 -- 172.21.15.114:0/2055052778 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00000f340 con 0x7fc00c1050d0 2024-09-13T18:22:32.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.684+0000 7fc0138be640 1 -- 172.21.15.114:0/2055052778 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc00c101450 con 0x7fc00c1050d0 2024-09-13T18:22:32.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.686+0000 7fc00a7fc640 1 -- 172.21.15.114:0/2055052778 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 23) v1 ==== 135994+0+0 (secure 0 0 0) 0x7fc0000119f0 con 0x7fc00c1050d0 2024-09-13T18:22:32.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.686+0000 7fc00a7fc640 1 --2- 172.21.15.114:0/2055052778 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbff40a1280 0x7fbff40a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:32.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.686+0000 7fc01165a640 1 --2- 172.21.15.114:0/2055052778 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbff40a1280 0x7fbff40a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:32.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.686+0000 7fc00a7fc640 1 -- 172.21.15.114:0/2055052778 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fc0000cd280 con 0x7fc00c1050d0 2024-09-13T18:22:32.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.687+0000 7fc01165a640 1 --2- 172.21.15.114:0/2055052778 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbff40a1280 0x7fbff40a3740 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc004006010 tx=0x7fc004005a20 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:32.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.690+0000 7fc00a7fc640 1 -- 172.21.15.114:0/2055052778 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fc000096490 con 0x7fc00c1050d0 2024-09-13T18:22:32.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.765+0000 7fc00a7fc640 1 -- 172.21.15.114:0/2055052778 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mgrmap(e 24) v1 ==== 136050+0+0 (secure 0 0 0) 0x7fc0000ca900 con 0x7fc00c1050d0 2024-09-13T18:22:32.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.854+0000 7fc0138be640 1 -- 172.21.15.114:0/2055052778 --> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc00c197720 con 0x7fbff40a1280 2024-09-13T18:22:32.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.865+0000 7fc00a7fc640 1 -- 172.21.15.114:0/2055052778 <== mgr.15555 v2:172.21.15.114:6800/2440721366 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427785 (secure 0 0 0) 0x7fc00c197720 con 0x7fbff40a1280 2024-09-13T18:22:32.874 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:22:32.877 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-13T18:22:32.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.869+0000 7fc0138be640 1 -- 172.21.15.114:0/2055052778 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbff40a1280 msgr2=0x7fbff40a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:32.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.869+0000 7fc0138be640 1 --2- 172.21.15.114:0/2055052778 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbff40a1280 0x7fbff40a3740 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc004006010 tx=0x7fc004005a20 comp rx=0 tx=0).stop 2024-09-13T18:22:32.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.869+0000 7fc0138be640 1 -- 172.21.15.114:0/2055052778 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc00c1050d0 msgr2=0x7fc00c196870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:32.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.869+0000 7fc0138be640 1 --2- 172.21.15.114:0/2055052778 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc00c1050d0 0x7fc00c196870 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc00000b450 tx=0x7fc00000b910 comp rx=0 tx=0).stop 2024-09-13T18:22:32.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.869+0000 7fc0138be640 1 -- 172.21.15.114:0/2055052778 shutdown_connections 2024-09-13T18:22:32.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.869+0000 7fc0138be640 1 --2- 172.21.15.114:0/2055052778 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbff40a1280 0x7fbff40a3740 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:32.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.869+0000 7fc0138be640 1 --2- 172.21.15.114:0/2055052778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc00c19ac60 0x7fc00c19b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:32.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.869+0000 7fc0138be640 1 --2- 172.21.15.114:0/2055052778 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc00c1050d0 0x7fc00c196870 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:32.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.869+0000 7fc0138be640 1 --2- 172.21.15.114:0/2055052778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc00c0fff80 0x7fc00c196330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:32.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.869+0000 7fc0138be640 1 -- 172.21.15.114:0/2055052778 >> 172.21.15.114:0/2055052778 conn(0x7fc00c0fba10 msgr2=0x7fc00c0fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:32.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.869+0000 7fc0138be640 1 -- 172.21.15.114:0/2055052778 shutdown_connections 2024-09-13T18:22:32.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:32.869+0000 7fc0138be640 1 -- 172.21.15.114:0/2055052778 wait complete. 2024-09-13T18:22:32.884 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-09-13T18:22:32.078907+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1194735,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1483,"num_read_kb":912,"num_write":548,"num_write_kb":4407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180875,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":938,"ondisk_log_size":938,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361600,"kb_used_data":12792,"kb_used_omap":18,"kb_used_meta":348653,"kb_avail":1124580224,"statfs":{"total":1151940427776,"available":1151570149376,"internally_reserved":0,"allocated":13099008,"data_stored":9351983,"data_compressed":474714,"data_compressed_allocated":647168,"data_compressed_original":3694592,"omap_allocated":19129,"internal_metadata":357020999},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.988041"},"pg_stats":[{"pgid":"2.1f","version":"93'14","reported_seq":157,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.139188+0000","last_change":"2024-09-13T18:20:01.577029+0000","last_active":"2024-09-13T18:22:28.139188+0000","last_peered":"2024-09-13T18:22:28.139188+0000","last_clean":"2024-09-13T18:22:28.139188+0000","last_became_active":"2024-09-13T18:20:01.576710+0000","last_became_peered":"2024-09-13T18:20:01.576710+0000","last_unstale":"2024-09-13T18:22:28.139188+0000","last_undegraded":"2024-09-13T18:22:28.139188+0000","last_fullsized":"2024-09-13T18:22:28.139188+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:25:11.977436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"93'5","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:03.100355+0000","last_change":"2024-09-13T18:20:01.922955+0000","last_active":"2024-09-13T18:22:03.100355+0000","last_peered":"2024-09-13T18:22:03.100355+0000","last_clean":"2024-09-13T18:22:03.100355+0000","last_became_active":"2024-09-13T18:20:01.922704+0000","last_became_peered":"2024-09-13T18:20:01.922704+0000","last_unstale":"2024-09-13T18:22:03.100355+0000","last_undegraded":"2024-09-13T18:22:03.100355+0000","last_fullsized":"2024-09-13T18:22:03.100355+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:44:20.672784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:28.310843+0000","last_change":"2024-09-13T18:20:27.300985+0000","last_active":"2024-09-13T18:21:28.310843+0000","last_peered":"2024-09-13T18:21:28.310843+0000","last_clean":"2024-09-13T18:21:28.310843+0000","last_became_active":"2024-09-13T18:20:27.300546+0000","last_became_peered":"2024-09-13T18:20:27.300546+0000","last_unstale":"2024-09-13T18:21:28.310843+0000","last_undegraded":"2024-09-13T18:21:28.310843+0000","last_fullsized":"2024-09-13T18:21:28.310843+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:00:11.750929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:03.100379+0000","last_change":"2024-09-13T18:20:01.578921+0000","last_active":"2024-09-13T18:22:03.100379+0000","last_peered":"2024-09-13T18:22:03.100379+0000","last_clean":"2024-09-13T18:22:03.100379+0000","last_became_active":"2024-09-13T18:20:01.578627+0000","last_became_peered":"2024-09-13T18:20:01.578627+0000","last_unstale":"2024-09-13T18:22:03.100379+0000","last_undegraded":"2024-09-13T18:22:03.100379+0000","last_fullsized":"2024-09-13T18:22:03.100379+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:43:32.919156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"93'6","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:07.284848+0000","last_change":"2024-09-13T18:20:01.922901+0000","last_active":"2024-09-13T18:22:07.284848+0000","last_peered":"2024-09-13T18:22:07.284848+0000","last_clean":"2024-09-13T18:22:07.284848+0000","last_became_active":"2024-09-13T18:20:01.922604+0000","last_became_peered":"2024-09-13T18:20:01.922604+0000","last_unstale":"2024-09-13T18:22:07.284848+0000","last_undegraded":"2024-09-13T18:22:07.284848+0000","last_fullsized":"2024-09-13T18:22:07.284848+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:27:30.650010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:28.151945+0000","last_change":"2024-09-13T18:20:06.135406+0000","last_active":"2024-09-13T18:21:28.151945+0000","last_peered":"2024-09-13T18:21:28.151945+0000","last_clean":"2024-09-13T18:21:28.151945+0000","last_became_active":"2024-09-13T18:20:06.134558+0000","last_became_peered":"2024-09-13T18:20:06.134558+0000","last_unstale":"2024-09-13T18:21:28.151945+0000","last_undegraded":"2024-09-13T18:21:28.151945+0000","last_fullsized":"2024-09-13T18:21:28.151945+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:10.901966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'12","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:28.153481+0000","last_change":"2024-09-13T18:20:01.580040+0000","last_active":"2024-09-13T18:21:28.153481+0000","last_peered":"2024-09-13T18:21:28.153481+0000","last_clean":"2024-09-13T18:21:28.153481+0000","last_became_active":"2024-09-13T18:20:01.579715+0000","last_became_peered":"2024-09-13T18:20:01.579715+0000","last_unstale":"2024-09-13T18:21:28.153481+0000","last_undegraded":"2024-09-13T18:21:28.153481+0000","last_fullsized":"2024-09-13T18:21:28.153481+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:57:03.588169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"93'9","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:55.925782+0000","last_change":"2024-09-13T18:20:01.928314+0000","last_active":"2024-09-13T18:21:55.925782+0000","last_peered":"2024-09-13T18:21:55.925782+0000","last_clean":"2024-09-13T18:21:55.925782+0000","last_became_active":"2024-09-13T18:20:01.925356+0000","last_became_peered":"2024-09-13T18:20:01.925356+0000","last_unstale":"2024-09-13T18:21:55.925782+0000","last_undegraded":"2024-09-13T18:21:55.925782+0000","last_fullsized":"2024-09-13T18:21:55.925782+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:43.906842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:28.152198+0000","last_change":"2024-09-13T18:20:06.135886+0000","last_active":"2024-09-13T18:21:28.152198+0000","last_peered":"2024-09-13T18:21:28.152198+0000","last_clean":"2024-09-13T18:21:28.152198+0000","last_became_active":"2024-09-13T18:20:06.134587+0000","last_became_peered":"2024-09-13T18:20:06.134587+0000","last_unstale":"2024-09-13T18:21:28.152198+0000","last_undegraded":"2024-09-13T18:21:28.152198+0000","last_fullsized":"2024-09-13T18:21:28.152198+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:44:42.434729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"91'5","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.140618+0000","last_change":"2024-09-13T18:20:01.576908+0000","last_active":"2024-09-13T18:22:28.140618+0000","last_peered":"2024-09-13T18:22:28.140618+0000","last_clean":"2024-09-13T18:22:28.140618+0000","last_became_active":"2024-09-13T18:20:01.576478+0000","last_became_peered":"2024-09-13T18:20:01.576478+0000","last_unstale":"2024-09-13T18:22:28.140618+0000","last_undegraded":"2024-09-13T18:22:28.140618+0000","last_fullsized":"2024-09-13T18:22:28.140618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:34:52.126901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"93'15","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:55.923587+0000","last_change":"2024-09-13T18:20:01.921960+0000","last_active":"2024-09-13T18:21:55.923587+0000","last_peered":"2024-09-13T18:21:55.923587+0000","last_clean":"2024-09-13T18:21:55.923587+0000","last_became_active":"2024-09-13T18:20:01.921652+0000","last_became_peered":"2024-09-13T18:20:01.921652+0000","last_unstale":"2024-09-13T18:21:55.923587+0000","last_undegraded":"2024-09-13T18:21:55.923587+0000","last_fullsized":"2024-09-13T18:21:55.923587+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:00:31.277613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:28.152004+0000","last_change":"2024-09-13T18:20:06.138983+0000","last_active":"2024-09-13T18:21:28.152004+0000","last_peered":"2024-09-13T18:21:28.152004+0000","last_clean":"2024-09-13T18:21:28.152004+0000","last_became_active":"2024-09-13T18:20:06.138708+0000","last_became_peered":"2024-09-13T18:20:06.138708+0000","last_unstale":"2024-09-13T18:21:28.152004+0000","last_undegraded":"2024-09-13T18:21:28.152004+0000","last_fullsized":"2024-09-13T18:21:28.152004+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:45:21.978217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"93'5","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:16.192564+0000","last_change":"2024-09-13T18:20:01.929817+0000","last_active":"2024-09-13T18:22:16.192564+0000","last_peered":"2024-09-13T18:22:16.192564+0000","last_clean":"2024-09-13T18:22:16.192564+0000","last_became_active":"2024-09-13T18:20:01.928907+0000","last_became_peered":"2024-09-13T18:20:01.928907+0000","last_unstale":"2024-09-13T18:22:16.192564+0000","last_undegraded":"2024-09-13T18:22:16.192564+0000","last_fullsized":"2024-09-13T18:22:16.192564+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:56:25.847230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:15.988721+0000","last_change":"2024-09-13T18:20:01.578968+0000","last_active":"2024-09-13T18:22:15.988721+0000","last_peered":"2024-09-13T18:22:15.988721+0000","last_clean":"2024-09-13T18:22:15.988721+0000","last_became_active":"2024-09-13T18:20:01.578623+0000","last_became_peered":"2024-09-13T18:20:01.578623+0000","last_unstale":"2024-09-13T18:22:15.988721+0000","last_undegraded":"2024-09-13T18:22:15.988721+0000","last_fullsized":"2024-09-13T18:22:15.988721+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:03:30.333107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:21.739811+0000","last_change":"2024-09-13T18:20:06.135977+0000","last_active":"2024-09-13T18:22:21.739811+0000","last_peered":"2024-09-13T18:22:21.739811+0000","last_clean":"2024-09-13T18:22:21.739811+0000","last_became_active":"2024-09-13T18:20:06.134844+0000","last_became_peered":"2024-09-13T18:20:06.134844+0000","last_unstale":"2024-09-13T18:22:21.739811+0000","last_undegraded":"2024-09-13T18:22:21.739811+0000","last_fullsized":"2024-09-13T18:22:21.739811+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:58:36.139240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"93'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:18.101943+0000","last_change":"2024-09-13T18:20:01.916675+0000","last_active":"2024-09-13T18:22:18.101943+0000","last_peered":"2024-09-13T18:22:18.101943+0000","last_clean":"2024-09-13T18:22:18.101943+0000","last_became_active":"2024-09-13T18:20:01.912902+0000","last_became_peered":"2024-09-13T18:20:01.912902+0000","last_unstale":"2024-09-13T18:22:18.101943+0000","last_undegraded":"2024-09-13T18:22:18.101943+0000","last_fullsized":"2024-09-13T18:22:18.101943+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:24:54.271569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"91'3","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:28.154557+0000","last_change":"2024-09-13T18:20:27.300845+0000","last_active":"2024-09-13T18:21:28.154557+0000","last_peered":"2024-09-13T18:21:28.154557+0000","last_clean":"2024-09-13T18:21:28.154557+0000","last_became_active":"2024-09-13T18:20:27.300417+0000","last_became_peered":"2024-09-13T18:20:27.300417+0000","last_unstale":"2024-09-13T18:21:28.154557+0000","last_undegraded":"2024-09-13T18:21:28.154557+0000","last_fullsized":"2024-09-13T18:21:28.154557+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:44:27.507140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:21.739768+0000","last_change":"2024-09-13T18:21:28.235707+0000","last_active":"2024-09-13T18:22:21.739768+0000","last_peered":"2024-09-13T18:22:21.739768+0000","last_clean":"2024-09-13T18:22:21.739768+0000","last_became_active":"2024-09-13T18:21:28.231794+0000","last_became_peered":"2024-09-13T18:21:28.231794+0000","last_unstale":"2024-09-13T18:22:21.739768+0000","last_undegraded":"2024-09-13T18:22:21.739768+0000","last_fullsized":"2024-09-13T18:22:21.739768+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:01:00.113821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"93'9","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:12.959281+0000","last_change":"2024-09-13T18:20:01.913634+0000","last_active":"2024-09-13T18:22:12.959281+0000","last_peered":"2024-09-13T18:22:12.959281+0000","last_clean":"2024-09-13T18:22:12.959281+0000","last_became_active":"2024-09-13T18:20:01.913128+0000","last_became_peered":"2024-09-13T18:20:01.913128+0000","last_unstale":"2024-09-13T18:22:12.959281+0000","last_undegraded":"2024-09-13T18:22:12.959281+0000","last_fullsized":"2024-09-13T18:22:12.959281+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:37:15.251756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"91'7","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:13.101465+0000","last_change":"2024-09-13T18:20:01.579385+0000","last_active":"2024-09-13T18:22:13.101465+0000","last_peered":"2024-09-13T18:22:13.101465+0000","last_clean":"2024-09-13T18:22:13.101465+0000","last_became_active":"2024-09-13T18:20:01.579102+0000","last_became_peered":"2024-09-13T18:20:01.579102+0000","last_unstale":"2024-09-13T18:22:13.101465+0000","last_undegraded":"2024-09-13T18:22:13.101465+0000","last_fullsized":"2024-09-13T18:22:13.101465+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:22:00.432052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.138888+0000","last_change":"2024-09-13T18:20:06.138110+0000","last_active":"2024-09-13T18:22:28.138888+0000","last_peered":"2024-09-13T18:22:28.138888+0000","last_clean":"2024-09-13T18:22:28.138888+0000","last_became_active":"2024-09-13T18:20:06.137763+0000","last_became_peered":"2024-09-13T18:20:06.137763+0000","last_unstale":"2024-09-13T18:22:28.138888+0000","last_undegraded":"2024-09-13T18:22:28.138888+0000","last_fullsized":"2024-09-13T18:22:28.138888+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:16:15.956216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"93'2","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:15.988679+0000","last_change":"2024-09-13T18:20:01.918318+0000","last_active":"2024-09-13T18:22:15.988679+0000","last_peered":"2024-09-13T18:22:15.988679+0000","last_clean":"2024-09-13T18:22:15.988679+0000","last_became_active":"2024-09-13T18:20:01.917977+0000","last_became_peered":"2024-09-13T18:20:01.917977+0000","last_unstale":"2024-09-13T18:22:15.988679+0000","last_undegraded":"2024-09-13T18:22:15.988679+0000","last_fullsized":"2024-09-13T18:22:15.988679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:23.368866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:13.276327+0000","last_change":"2024-09-13T18:20:01.572009+0000","last_active":"2024-09-13T18:22:13.276327+0000","last_peered":"2024-09-13T18:22:13.276327+0000","last_clean":"2024-09-13T18:22:13.276327+0000","last_became_active":"2024-09-13T18:20:01.571313+0000","last_became_peered":"2024-09-13T18:20:01.571313+0000","last_unstale":"2024-09-13T18:22:13.276327+0000","last_undegraded":"2024-09-13T18:22:13.276327+0000","last_fullsized":"2024-09-13T18:22:13.276327+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:32:59.340476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:21.739824+0000","last_change":"2024-09-13T18:20:06.132212+0000","last_active":"2024-09-13T18:22:21.739824+0000","last_peered":"2024-09-13T18:22:21.739824+0000","last_clean":"2024-09-13T18:22:21.739824+0000","last_became_active":"2024-09-13T18:20:06.131809+0000","last_became_peered":"2024-09-13T18:20:06.131809+0000","last_unstale":"2024-09-13T18:22:21.739824+0000","last_undegraded":"2024-09-13T18:22:21.739824+0000","last_fullsized":"2024-09-13T18:22:21.739824+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:18:11.667551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"93'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:12.285278+0000","last_change":"2024-09-13T18:20:27.301112+0000","last_active":"2024-09-13T18:22:12.285278+0000","last_peered":"2024-09-13T18:22:12.285278+0000","last_clean":"2024-09-13T18:22:12.285278+0000","last_became_active":"2024-09-13T18:20:27.300772+0000","last_became_peered":"2024-09-13T18:20:27.300772+0000","last_unstale":"2024-09-13T18:22:12.285278+0000","last_undegraded":"2024-09-13T18:22:12.285278+0000","last_fullsized":"2024-09-13T18:22:12.285278+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:59:57.606105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"91'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:11.738288+0000","last_change":"2024-09-13T18:20:01.572025+0000","last_active":"2024-09-13T18:22:11.738288+0000","last_peered":"2024-09-13T18:22:11.738288+0000","last_clean":"2024-09-13T18:22:11.738288+0000","last_became_active":"2024-09-13T18:20:01.571321+0000","last_became_peered":"2024-09-13T18:20:01.571321+0000","last_unstale":"2024-09-13T18:22:11.738288+0000","last_undegraded":"2024-09-13T18:22:11.738288+0000","last_fullsized":"2024-09-13T18:22:11.738288+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:09.374364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:10.788550+0000","last_change":"2024-09-13T18:20:06.130687+0000","last_active":"2024-09-13T18:22:10.788550+0000","last_peered":"2024-09-13T18:22:10.788550+0000","last_clean":"2024-09-13T18:22:10.788550+0000","last_became_active":"2024-09-13T18:20:06.129839+0000","last_became_peered":"2024-09-13T18:20:06.129839+0000","last_unstale":"2024-09-13T18:22:10.788550+0000","last_undegraded":"2024-09-13T18:22:10.788550+0000","last_fullsized":"2024-09-13T18:22:10.788550+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:12:52.334945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"93'5","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:13.101520+0000","last_change":"2024-09-13T18:20:01.917244+0000","last_active":"2024-09-13T18:22:13.101520+0000","last_peered":"2024-09-13T18:22:13.101520+0000","last_clean":"2024-09-13T18:22:13.101520+0000","last_became_active":"2024-09-13T18:20:01.916948+0000","last_became_peered":"2024-09-13T18:20:01.916948+0000","last_unstale":"2024-09-13T18:22:13.101520+0000","last_undegraded":"2024-09-13T18:22:13.101520+0000","last_fullsized":"2024-09-13T18:22:13.101520+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:17:11.967096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'44","reported_seq":89,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.252535+0000","last_change":"2024-09-13T18:20:01.569123+0000","last_active":"2024-09-13T18:22:28.252535+0000","last_peered":"2024-09-13T18:22:28.252535+0000","last_clean":"2024-09-13T18:22:28.252535+0000","last_became_active":"2024-09-13T18:20:01.568720+0000","last_became_peered":"2024-09-13T18:20:01.568720+0000","last_unstale":"2024-09-13T18:22:28.252535+0000","last_undegraded":"2024-09-13T18:22:28.252535+0000","last_fullsized":"2024-09-13T18:22:28.252535+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:04:29.561013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":76,"num_write_kb":833,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:08.100974+0000","last_change":"2024-09-13T18:20:06.141771+0000","last_active":"2024-09-13T18:22:08.100974+0000","last_peered":"2024-09-13T18:22:08.100974+0000","last_clean":"2024-09-13T18:22:08.100974+0000","last_became_active":"2024-09-13T18:20:06.141362+0000","last_became_peered":"2024-09-13T18:20:06.141362+0000","last_unstale":"2024-09-13T18:22:08.100974+0000","last_undegraded":"2024-09-13T18:22:08.100974+0000","last_fullsized":"2024-09-13T18:22:08.100974+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:09:58.597605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"93'9","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:08.482216+0000","last_change":"2024-09-13T18:20:01.911048+0000","last_active":"2024-09-13T18:22:08.482216+0000","last_peered":"2024-09-13T18:22:08.482216+0000","last_clean":"2024-09-13T18:22:08.482216+0000","last_became_active":"2024-09-13T18:20:01.910735+0000","last_became_peered":"2024-09-13T18:20:01.910735+0000","last_unstale":"2024-09-13T18:22:08.482216+0000","last_undegraded":"2024-09-13T18:22:08.482216+0000","last_fullsized":"2024-09-13T18:22:08.482216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:07:35.447676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"91'4","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:05.987451+0000","last_change":"2024-09-13T18:20:01.571627+0000","last_active":"2024-09-13T18:22:05.987451+0000","last_peered":"2024-09-13T18:22:05.987451+0000","last_clean":"2024-09-13T18:22:05.987451+0000","last_became_active":"2024-09-13T18:20:01.571250+0000","last_became_peered":"2024-09-13T18:20:01.571250+0000","last_unstale":"2024-09-13T18:22:05.987451+0000","last_undegraded":"2024-09-13T18:22:05.987451+0000","last_fullsized":"2024-09-13T18:22:05.987451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:34:03.479324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:12.959308+0000","last_change":"2024-09-13T18:20:06.132974+0000","last_active":"2024-09-13T18:22:12.959308+0000","last_peered":"2024-09-13T18:22:12.959308+0000","last_clean":"2024-09-13T18:22:12.959308+0000","last_became_active":"2024-09-13T18:20:06.131668+0000","last_became_peered":"2024-09-13T18:20:06.131668+0000","last_unstale":"2024-09-13T18:22:12.959308+0000","last_undegraded":"2024-09-13T18:22:12.959308+0000","last_fullsized":"2024-09-13T18:22:12.959308+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:36:30.329761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"93'4","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:08.478967+0000","last_change":"2024-09-13T18:20:01.904670+0000","last_active":"2024-09-13T18:22:08.478967+0000","last_peered":"2024-09-13T18:22:08.478967+0000","last_clean":"2024-09-13T18:22:08.478967+0000","last_became_active":"2024-09-13T18:20:01.904316+0000","last_became_peered":"2024-09-13T18:20:01.904316+0000","last_unstale":"2024-09-13T18:22:08.478967+0000","last_undegraded":"2024-09-13T18:22:08.478967+0000","last_fullsized":"2024-09-13T18:22:08.478967+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:08:17.687179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"91'1","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:09.591727+0000","last_change":"2024-09-13T18:20:01.564750+0000","last_active":"2024-09-13T18:22:09.591727+0000","last_peered":"2024-09-13T18:22:09.591727+0000","last_clean":"2024-09-13T18:22:09.591727+0000","last_became_active":"2024-09-13T18:20:01.564096+0000","last_became_peered":"2024-09-13T18:20:01.564096+0000","last_unstale":"2024-09-13T18:22:09.591727+0000","last_undegraded":"2024-09-13T18:22:09.591727+0000","last_fullsized":"2024-09-13T18:22:09.591727+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:42:19.588824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:10.988064+0000","last_change":"2024-09-13T18:20:06.133386+0000","last_active":"2024-09-13T18:22:10.988064+0000","last_peered":"2024-09-13T18:22:10.988064+0000","last_clean":"2024-09-13T18:22:10.988064+0000","last_became_active":"2024-09-13T18:20:06.133004+0000","last_became_peered":"2024-09-13T18:20:06.133004+0000","last_unstale":"2024-09-13T18:22:10.988064+0000","last_undegraded":"2024-09-13T18:22:10.988064+0000","last_fullsized":"2024-09-13T18:22:10.988064+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:54:46.440410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"93'7","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:08.478947+0000","last_change":"2024-09-13T18:20:01.912823+0000","last_active":"2024-09-13T18:22:08.478947+0000","last_peered":"2024-09-13T18:22:08.478947+0000","last_clean":"2024-09-13T18:22:08.478947+0000","last_became_active":"2024-09-13T18:20:01.912123+0000","last_became_peered":"2024-09-13T18:20:01.912123+0000","last_unstale":"2024-09-13T18:22:08.478947+0000","last_undegraded":"2024-09-13T18:22:08.478947+0000","last_fullsized":"2024-09-13T18:22:08.478947+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:32:33.544816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"94'8","reported_seq":153,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.424067+0000","last_change":"2024-09-13T18:20:01.577095+0000","last_active":"2024-09-13T18:22:28.424067+0000","last_peered":"2024-09-13T18:22:28.424067+0000","last_clean":"2024-09-13T18:22:28.424067+0000","last_became_active":"2024-09-13T18:20:01.576765+0000","last_became_peered":"2024-09-13T18:20:01.576765+0000","last_unstale":"2024-09-13T18:22:28.424067+0000","last_undegraded":"2024-09-13T18:22:28.424067+0000","last_fullsized":"2024-09-13T18:22:28.424067+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:44:08.750441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1818,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":123,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:13.482840+0000","last_change":"2024-09-13T18:20:06.131650+0000","last_active":"2024-09-13T18:22:13.482840+0000","last_peered":"2024-09-13T18:22:13.482840+0000","last_clean":"2024-09-13T18:22:13.482840+0000","last_became_active":"2024-09-13T18:20:06.130902+0000","last_became_peered":"2024-09-13T18:20:06.130902+0000","last_unstale":"2024-09-13T18:22:13.482840+0000","last_undegraded":"2024-09-13T18:22:13.482840+0000","last_fullsized":"2024-09-13T18:22:13.482840+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.366694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"93'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:06.033866+0000","last_change":"2024-09-13T18:20:01.918221+0000","last_active":"2024-09-13T18:22:06.033866+0000","last_peered":"2024-09-13T18:22:06.033866+0000","last_clean":"2024-09-13T18:22:06.033866+0000","last_became_active":"2024-09-13T18:20:01.917134+0000","last_became_peered":"2024-09-13T18:20:01.917134+0000","last_unstale":"2024-09-13T18:22:06.033866+0000","last_undegraded":"2024-09-13T18:22:06.033866+0000","last_fullsized":"2024-09-13T18:22:06.033866+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:19:06.269454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'6","reported_seq":150,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.292167+0000","last_change":"2024-09-13T18:20:01.572258+0000","last_active":"2024-09-13T18:22:28.292167+0000","last_peered":"2024-09-13T18:22:28.292167+0000","last_clean":"2024-09-13T18:22:28.292167+0000","last_became_active":"2024-09-13T18:20:01.571968+0000","last_became_peered":"2024-09-13T18:20:01.571968+0000","last_unstale":"2024-09-13T18:22:28.292167+0000","last_undegraded":"2024-09-13T18:22:28.292167+0000","last_fullsized":"2024-09-13T18:22:28.292167+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:40:59.143123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":122,"num_read_kb":122,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:18.483349+0000","last_change":"2024-09-13T18:20:06.131714+0000","last_active":"2024-09-13T18:22:18.483349+0000","last_peered":"2024-09-13T18:22:18.483349+0000","last_clean":"2024-09-13T18:22:18.483349+0000","last_became_active":"2024-09-13T18:20:06.131244+0000","last_became_peered":"2024-09-13T18:20:06.131244+0000","last_unstale":"2024-09-13T18:22:18.483349+0000","last_undegraded":"2024-09-13T18:22:18.483349+0000","last_fullsized":"2024-09-13T18:22:18.483349+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:37.946002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"93'5","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:08.482116+0000","last_change":"2024-09-13T18:20:01.917351+0000","last_active":"2024-09-13T18:22:08.482116+0000","last_peered":"2024-09-13T18:22:08.482116+0000","last_clean":"2024-09-13T18:22:08.482116+0000","last_became_active":"2024-09-13T18:20:01.917067+0000","last_became_peered":"2024-09-13T18:20:01.917067+0000","last_unstale":"2024-09-13T18:22:08.482116+0000","last_undegraded":"2024-09-13T18:22:08.482116+0000","last_fullsized":"2024-09-13T18:22:08.482116+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:35:37.723132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'4","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:01.190731+0000","last_change":"2024-09-13T18:20:01.573515+0000","last_active":"2024-09-13T18:22:01.190731+0000","last_peered":"2024-09-13T18:22:01.190731+0000","last_clean":"2024-09-13T18:22:01.190731+0000","last_became_active":"2024-09-13T18:20:01.573211+0000","last_became_peered":"2024-09-13T18:20:01.573211+0000","last_unstale":"2024-09-13T18:22:01.190731+0000","last_undegraded":"2024-09-13T18:22:01.190731+0000","last_fullsized":"2024-09-13T18:22:01.190731+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:05:30.912467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:30.983462+0000","last_change":"2024-09-13T18:21:28.231613+0000","last_active":"2024-09-13T18:21:30.983462+0000","last_peered":"2024-09-13T18:21:30.983462+0000","last_clean":"2024-09-13T18:21:30.983462+0000","last_became_active":"2024-09-13T18:21:28.231243+0000","last_became_peered":"2024-09-13T18:21:28.231243+0000","last_unstale":"2024-09-13T18:21:30.983462+0000","last_undegraded":"2024-09-13T18:21:30.983462+0000","last_fullsized":"2024-09-13T18:21:30.983462+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:58:46.066195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.2","version":"93'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:08.101059+0000","last_change":"2024-09-13T18:20:01.921597+0000","last_active":"2024-09-13T18:22:08.101059+0000","last_peered":"2024-09-13T18:22:08.101059+0000","last_clean":"2024-09-13T18:22:08.101059+0000","last_became_active":"2024-09-13T18:20:01.920200+0000","last_became_peered":"2024-09-13T18:20:01.920200+0000","last_unstale":"2024-09-13T18:22:08.101059+0000","last_undegraded":"2024-09-13T18:22:08.101059+0000","last_fullsized":"2024-09-13T18:22:08.101059+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:22:40.517941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'6","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:06.737540+0000","last_change":"2024-09-13T18:20:01.571313+0000","last_active":"2024-09-13T18:22:06.737540+0000","last_peered":"2024-09-13T18:22:06.737540+0000","last_clean":"2024-09-13T18:22:06.737540+0000","last_became_active":"2024-09-13T18:20:01.570977+0000","last_became_peered":"2024-09-13T18:20:01.570977+0000","last_unstale":"2024-09-13T18:22:06.737540+0000","last_undegraded":"2024-09-13T18:22:06.737540+0000","last_fullsized":"2024-09-13T18:22:06.737540+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:06.870320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":347,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:41.188167+0000","last_change":"2024-09-13T18:18:51.237409+0000","last_active":"2024-09-13T18:21:41.188167+0000","last_peered":"2024-09-13T18:21:41.188167+0000","last_clean":"2024-09-13T18:21:41.188167+0000","last_became_active":"2024-09-13T18:18:51.236986+0000","last_became_peered":"2024-09-13T18:18:51.236986+0000","last_unstale":"2024-09-13T18:21:41.188167+0000","last_undegraded":"2024-09-13T18:21:41.188167+0000","last_fullsized":"2024-09-13T18:21:41.188167+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:15:40.757299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:15:40.757299+0000","last_clean_scrub_stamp":"2024-09-13T18:15:40.757299+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:58:29.679404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":19,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:11.192032+0000","last_change":"2024-09-13T18:20:06.130803+0000","last_active":"2024-09-13T18:22:11.192032+0000","last_peered":"2024-09-13T18:22:11.192032+0000","last_clean":"2024-09-13T18:22:11.192032+0000","last_became_active":"2024-09-13T18:20:06.130433+0000","last_became_peered":"2024-09-13T18:20:06.130433+0000","last_unstale":"2024-09-13T18:22:11.192032+0000","last_undegraded":"2024-09-13T18:22:11.192032+0000","last_fullsized":"2024-09-13T18:22:11.192032+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:23:11.787084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"93'7","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:21.193214+0000","last_change":"2024-09-13T18:20:01.922702+0000","last_active":"2024-09-13T18:22:21.193214+0000","last_peered":"2024-09-13T18:22:21.193214+0000","last_clean":"2024-09-13T18:22:21.193214+0000","last_became_active":"2024-09-13T18:20:01.921109+0000","last_became_peered":"2024-09-13T18:20:01.921109+0000","last_unstale":"2024-09-13T18:22:21.193214+0000","last_undegraded":"2024-09-13T18:22:21.193214+0000","last_fullsized":"2024-09-13T18:22:21.193214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:03:34.618054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"93'6","reported_seq":152,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.067908+0000","last_change":"2024-09-13T18:20:01.571024+0000","last_active":"2024-09-13T18:22:28.067908+0000","last_peered":"2024-09-13T18:22:28.067908+0000","last_clean":"2024-09-13T18:22:28.067908+0000","last_became_active":"2024-09-13T18:20:01.570687+0000","last_became_peered":"2024-09-13T18:20:01.570687+0000","last_unstale":"2024-09-13T18:22:28.067908+0000","last_undegraded":"2024-09-13T18:22:28.067908+0000","last_fullsized":"2024-09-13T18:22:28.067908+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:10:25.126179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1616,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":124,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:18.277060+0000","last_change":"2024-09-13T18:20:06.129025+0000","last_active":"2024-09-13T18:22:18.277060+0000","last_peered":"2024-09-13T18:22:18.277060+0000","last_clean":"2024-09-13T18:22:18.277060+0000","last_became_active":"2024-09-13T18:20:06.128618+0000","last_became_peered":"2024-09-13T18:20:06.128618+0000","last_unstale":"2024-09-13T18:22:18.277060+0000","last_undegraded":"2024-09-13T18:22:18.277060+0000","last_fullsized":"2024-09-13T18:22:18.277060+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:20:08.891259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"93'6","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:21.193185+0000","last_change":"2024-09-13T18:20:01.921697+0000","last_active":"2024-09-13T18:22:21.193185+0000","last_peered":"2024-09-13T18:22:21.193185+0000","last_clean":"2024-09-13T18:22:21.193185+0000","last_became_active":"2024-09-13T18:20:01.920174+0000","last_became_peered":"2024-09-13T18:20:01.920174+0000","last_unstale":"2024-09-13T18:22:21.193185+0000","last_undegraded":"2024-09-13T18:22:21.193185+0000","last_fullsized":"2024-09-13T18:22:21.193185+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:57:03.922725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"94'41","reported_seq":71,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.250386+0000","last_change":"2024-09-13T18:20:01.582102+0000","last_active":"2024-09-13T18:22:28.250386+0000","last_peered":"2024-09-13T18:22:28.250386+0000","last_clean":"2024-09-13T18:22:28.250386+0000","last_became_active":"2024-09-13T18:20:01.581841+0000","last_became_peered":"2024-09-13T18:20:01.581841+0000","last_unstale":"2024-09-13T18:22:28.250386+0000","last_undegraded":"2024-09-13T18:22:28.250386+0000","last_fullsized":"2024-09-13T18:22:28.250386+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:59:57.929703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:22.286171+0000","last_change":"2024-09-13T18:20:06.133126+0000","last_active":"2024-09-13T18:22:22.286171+0000","last_peered":"2024-09-13T18:22:22.286171+0000","last_clean":"2024-09-13T18:22:22.286171+0000","last_became_active":"2024-09-13T18:20:06.132747+0000","last_became_peered":"2024-09-13T18:20:06.132747+0000","last_unstale":"2024-09-13T18:22:22.286171+0000","last_undegraded":"2024-09-13T18:22:22.286171+0000","last_fullsized":"2024-09-13T18:22:22.286171+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:54:21.843090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"93'9","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:21.193232+0000","last_change":"2024-09-13T18:20:01.916306+0000","last_active":"2024-09-13T18:22:21.193232+0000","last_peered":"2024-09-13T18:22:21.193232+0000","last_clean":"2024-09-13T18:22:21.193232+0000","last_became_active":"2024-09-13T18:20:01.915739+0000","last_became_peered":"2024-09-13T18:20:01.915739+0000","last_unstale":"2024-09-13T18:22:21.193232+0000","last_undegraded":"2024-09-13T18:22:21.193232+0000","last_fullsized":"2024-09-13T18:22:21.193232+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:19:09.689909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"91'5","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:18.483339+0000","last_change":"2024-09-13T18:20:01.565821+0000","last_active":"2024-09-13T18:22:18.483339+0000","last_peered":"2024-09-13T18:22:18.483339+0000","last_clean":"2024-09-13T18:22:18.483339+0000","last_became_active":"2024-09-13T18:20:01.565467+0000","last_became_peered":"2024-09-13T18:20:01.565467+0000","last_unstale":"2024-09-13T18:22:18.483339+0000","last_undegraded":"2024-09-13T18:22:18.483339+0000","last_fullsized":"2024-09-13T18:22:18.483339+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:11.609400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:16.035040+0000","last_change":"2024-09-13T18:20:06.131348+0000","last_active":"2024-09-13T18:22:16.035040+0000","last_peered":"2024-09-13T18:22:16.035040+0000","last_clean":"2024-09-13T18:22:16.035040+0000","last_became_active":"2024-09-13T18:20:06.131019+0000","last_became_peered":"2024-09-13T18:20:06.131019+0000","last_unstale":"2024-09-13T18:22:16.035040+0000","last_undegraded":"2024-09-13T18:22:16.035040+0000","last_fullsized":"2024-09-13T18:22:16.035040+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:13:49.717503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"93'5","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:17.959923+0000","last_change":"2024-09-13T18:20:01.922306+0000","last_active":"2024-09-13T18:22:17.959923+0000","last_peered":"2024-09-13T18:22:17.959923+0000","last_clean":"2024-09-13T18:22:17.959923+0000","last_became_active":"2024-09-13T18:20:01.921956+0000","last_became_peered":"2024-09-13T18:20:01.921956+0000","last_unstale":"2024-09-13T18:22:17.959923+0000","last_undegraded":"2024-09-13T18:22:17.959923+0000","last_fullsized":"2024-09-13T18:22:17.959923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:17:48.755318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"94'3","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.204515+0000","last_change":"2024-09-13T18:20:27.194085+0000","last_active":"2024-09-13T18:22:28.204515+0000","last_peered":"2024-09-13T18:22:28.204515+0000","last_clean":"2024-09-13T18:22:28.204515+0000","last_became_active":"2024-09-13T18:20:27.190136+0000","last_became_peered":"2024-09-13T18:20:27.190136+0000","last_unstale":"2024-09-13T18:22:28.204515+0000","last_undegraded":"2024-09-13T18:22:28.204515+0000","last_fullsized":"2024-09-13T18:22:28.204515+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:57:07.202901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:16.035090+0000","last_change":"2024-09-13T18:20:06.135498+0000","last_active":"2024-09-13T18:22:16.035090+0000","last_peered":"2024-09-13T18:22:16.035090+0000","last_clean":"2024-09-13T18:22:16.035090+0000","last_became_active":"2024-09-13T18:20:06.135095+0000","last_became_peered":"2024-09-13T18:20:06.135095+0000","last_unstale":"2024-09-13T18:22:16.035090+0000","last_undegraded":"2024-09-13T18:22:16.035090+0000","last_fullsized":"2024-09-13T18:22:16.035090+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:06:40.609718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"93'6","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:55.905745+0000","last_change":"2024-09-13T18:20:27.186825+0000","last_active":"2024-09-13T18:21:55.905745+0000","last_peered":"2024-09-13T18:21:55.905745+0000","last_clean":"2024-09-13T18:21:55.905745+0000","last_became_active":"2024-09-13T18:20:27.186523+0000","last_became_peered":"2024-09-13T18:20:27.186523+0000","last_unstale":"2024-09-13T18:21:55.905745+0000","last_undegraded":"2024-09-13T18:21:55.905745+0000","last_fullsized":"2024-09-13T18:21:55.905745+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:28:30.453945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.10","version":"91'17","reported_seq":61,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:20.989307+0000","last_change":"2024-09-13T18:20:27.200142+0000","last_active":"2024-09-13T18:22:20.989307+0000","last_peered":"2024-09-13T18:22:20.989307+0000","last_clean":"2024-09-13T18:22:20.989307+0000","last_became_active":"2024-09-13T18:20:27.189670+0000","last_became_peered":"2024-09-13T18:20:27.189670+0000","last_unstale":"2024-09-13T18:22:20.989307+0000","last_undegraded":"2024-09-13T18:22:20.989307+0000","last_fullsized":"2024-09-13T18:22:20.989307+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:16:02.934269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:28.150641+0000","last_change":"2024-09-13T18:20:06.136742+0000","last_active":"2024-09-13T18:21:28.150641+0000","last_peered":"2024-09-13T18:21:28.150641+0000","last_clean":"2024-09-13T18:21:28.150641+0000","last_became_active":"2024-09-13T18:20:06.136373+0000","last_became_peered":"2024-09-13T18:20:06.136373+0000","last_unstale":"2024-09-13T18:21:28.150641+0000","last_undegraded":"2024-09-13T18:21:28.150641+0000","last_fullsized":"2024-09-13T18:21:28.150641+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:49:11.782489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"93'6","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.288955+0000","last_change":"2024-09-13T18:20:01.922910+0000","last_active":"2024-09-13T18:22:28.288955+0000","last_peered":"2024-09-13T18:22:28.288955+0000","last_clean":"2024-09-13T18:22:28.288955+0000","last_became_active":"2024-09-13T18:20:01.921162+0000","last_became_peered":"2024-09-13T18:20:01.921162+0000","last_unstale":"2024-09-13T18:22:28.288955+0000","last_undegraded":"2024-09-13T18:22:28.288955+0000","last_fullsized":"2024-09-13T18:22:28.288955+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:14:47.243418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"94'8","reported_seq":155,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.416185+0000","last_change":"2024-09-13T18:20:01.575830+0000","last_active":"2024-09-13T18:22:28.416185+0000","last_peered":"2024-09-13T18:22:28.416185+0000","last_clean":"2024-09-13T18:22:28.416185+0000","last_became_active":"2024-09-13T18:20:01.575334+0000","last_became_peered":"2024-09-13T18:20:01.575334+0000","last_unstale":"2024-09-13T18:22:28.416185+0000","last_undegraded":"2024-09-13T18:22:28.416185+0000","last_fullsized":"2024-09-13T18:22:28.416185+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:13:00.935600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1818,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":126,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:28.154729+0000","last_change":"2024-09-13T18:20:06.138609+0000","last_active":"2024-09-13T18:21:28.154729+0000","last_peered":"2024-09-13T18:21:28.154729+0000","last_clean":"2024-09-13T18:21:28.154729+0000","last_became_active":"2024-09-13T18:20:06.138232+0000","last_became_peered":"2024-09-13T18:20:06.138232+0000","last_unstale":"2024-09-13T18:21:28.154729+0000","last_undegraded":"2024-09-13T18:21:28.154729+0000","last_fullsized":"2024-09-13T18:21:28.154729+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:46:06.227407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"93'15","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.289290+0000","last_change":"2024-09-13T18:20:01.928757+0000","last_active":"2024-09-13T18:22:28.289290+0000","last_peered":"2024-09-13T18:22:28.289290+0000","last_clean":"2024-09-13T18:22:28.289290+0000","last_became_active":"2024-09-13T18:20:01.928350+0000","last_became_peered":"2024-09-13T18:20:01.928350+0000","last_unstale":"2024-09-13T18:22:28.289290+0000","last_undegraded":"2024-09-13T18:22:28.289290+0000","last_fullsized":"2024-09-13T18:22:28.289290+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:17:43.208106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"91'5","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:21.035873+0000","last_change":"2024-09-13T18:20:01.579128+0000","last_active":"2024-09-13T18:22:21.035873+0000","last_peered":"2024-09-13T18:22:21.035873+0000","last_clean":"2024-09-13T18:22:21.035873+0000","last_became_active":"2024-09-13T18:20:01.578796+0000","last_became_peered":"2024-09-13T18:20:01.578796+0000","last_unstale":"2024-09-13T18:22:21.035873+0000","last_undegraded":"2024-09-13T18:22:21.035873+0000","last_fullsized":"2024-09-13T18:22:21.035873+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:30:04.836103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.204056+0000","last_change":"2024-09-13T18:20:06.131404+0000","last_active":"2024-09-13T18:22:28.204056+0000","last_peered":"2024-09-13T18:22:28.204056+0000","last_clean":"2024-09-13T18:22:28.204056+0000","last_became_active":"2024-09-13T18:20:06.130931+0000","last_became_peered":"2024-09-13T18:20:06.130931+0000","last_unstale":"2024-09-13T18:22:28.204056+0000","last_undegraded":"2024-09-13T18:22:28.204056+0000","last_fullsized":"2024-09-13T18:22:28.204056+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:55:54.568816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"93'9","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.292205+0000","last_change":"2024-09-13T18:20:01.916089+0000","last_active":"2024-09-13T18:22:28.292205+0000","last_peered":"2024-09-13T18:22:28.292205+0000","last_clean":"2024-09-13T18:22:28.292205+0000","last_became_active":"2024-09-13T18:20:01.915783+0000","last_became_peered":"2024-09-13T18:20:01.915783+0000","last_unstale":"2024-09-13T18:22:28.292205+0000","last_undegraded":"2024-09-13T18:22:28.292205+0000","last_fullsized":"2024-09-13T18:22:28.292205+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:43:25.886029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"91'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:20.789776+0000","last_change":"2024-09-13T18:20:01.577381+0000","last_active":"2024-09-13T18:22:20.789776+0000","last_peered":"2024-09-13T18:22:20.789776+0000","last_clean":"2024-09-13T18:22:20.789776+0000","last_became_active":"2024-09-13T18:20:01.577044+0000","last_became_peered":"2024-09-13T18:20:01.577044+0000","last_unstale":"2024-09-13T18:22:20.789776+0000","last_undegraded":"2024-09-13T18:22:20.789776+0000","last_fullsized":"2024-09-13T18:22:20.789776+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:00.907948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.146073+0000","last_change":"2024-09-13T18:20:06.139971+0000","last_active":"2024-09-13T18:22:28.146073+0000","last_peered":"2024-09-13T18:22:28.146073+0000","last_clean":"2024-09-13T18:22:28.146073+0000","last_became_active":"2024-09-13T18:20:06.139507+0000","last_became_peered":"2024-09-13T18:20:06.139507+0000","last_unstale":"2024-09-13T18:22:28.146073+0000","last_undegraded":"2024-09-13T18:22:28.146073+0000","last_fullsized":"2024-09-13T18:22:28.146073+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:40:21.037768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"93'6","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.140553+0000","last_change":"2024-09-13T18:20:01.915348+0000","last_active":"2024-09-13T18:22:28.140553+0000","last_peered":"2024-09-13T18:22:28.140553+0000","last_clean":"2024-09-13T18:22:28.140553+0000","last_became_active":"2024-09-13T18:20:01.915042+0000","last_became_peered":"2024-09-13T18:20:01.915042+0000","last_unstale":"2024-09-13T18:22:28.140553+0000","last_undegraded":"2024-09-13T18:22:28.140553+0000","last_fullsized":"2024-09-13T18:22:28.140553+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:31:33.712314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"94'41","reported_seq":315,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.421008+0000","last_change":"2024-09-13T18:20:01.578456+0000","last_active":"2024-09-13T18:22:28.421008+0000","last_peered":"2024-09-13T18:22:28.421008+0000","last_clean":"2024-09-13T18:22:28.421008+0000","last_became_active":"2024-09-13T18:20:01.578051+0000","last_became_peered":"2024-09-13T18:20:01.578051+0000","last_unstale":"2024-09-13T18:22:28.421008+0000","last_undegraded":"2024-09-13T18:22:28.421008+0000","last_fullsized":"2024-09-13T18:22:28.421008+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:34:17.651620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69151,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":252,"num_read_kb":194,"num_write":43,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.137141+0000","last_change":"2024-09-13T18:20:06.131410+0000","last_active":"2024-09-13T18:22:28.137141+0000","last_peered":"2024-09-13T18:22:28.137141+0000","last_clean":"2024-09-13T18:22:28.137141+0000","last_became_active":"2024-09-13T18:20:06.131060+0000","last_became_peered":"2024-09-13T18:20:06.131060+0000","last_unstale":"2024-09-13T18:22:28.137141+0000","last_undegraded":"2024-09-13T18:22:28.137141+0000","last_fullsized":"2024-09-13T18:22:28.137141+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:08:25.266971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"93'11","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.292104+0000","last_change":"2024-09-13T18:20:01.928839+0000","last_active":"2024-09-13T18:22:28.292104+0000","last_peered":"2024-09-13T18:22:28.292104+0000","last_clean":"2024-09-13T18:22:28.292104+0000","last_became_active":"2024-09-13T18:20:01.925774+0000","last_became_peered":"2024-09-13T18:20:01.925774+0000","last_unstale":"2024-09-13T18:22:28.292104+0000","last_undegraded":"2024-09-13T18:22:28.292104+0000","last_fullsized":"2024-09-13T18:22:28.292104+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:15:23.049671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"91'5","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.146218+0000","last_change":"2024-09-13T18:20:01.576332+0000","last_active":"2024-09-13T18:22:28.146218+0000","last_peered":"2024-09-13T18:22:28.146218+0000","last_clean":"2024-09-13T18:22:28.146218+0000","last_became_active":"2024-09-13T18:20:01.575862+0000","last_became_peered":"2024-09-13T18:20:01.575862+0000","last_unstale":"2024-09-13T18:22:28.146218+0000","last_undegraded":"2024-09-13T18:22:28.146218+0000","last_fullsized":"2024-09-13T18:22:28.146218+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:17:00.452480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.206402+0000","last_change":"2024-09-13T18:20:06.133142+0000","last_active":"2024-09-13T18:22:28.206402+0000","last_peered":"2024-09-13T18:22:28.206402+0000","last_clean":"2024-09-13T18:22:28.206402+0000","last_became_active":"2024-09-13T18:20:06.132740+0000","last_became_peered":"2024-09-13T18:20:06.132740+0000","last_unstale":"2024-09-13T18:22:28.206402+0000","last_undegraded":"2024-09-13T18:22:28.206402+0000","last_fullsized":"2024-09-13T18:22:28.206402+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:12:46.442481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"93'9","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.820616+0000","last_change":"2024-09-13T18:20:01.919673+0000","last_active":"2024-09-13T18:22:28.820616+0000","last_peered":"2024-09-13T18:22:28.820616+0000","last_clean":"2024-09-13T18:22:28.820616+0000","last_became_active":"2024-09-13T18:20:01.919358+0000","last_became_peered":"2024-09-13T18:20:01.919358+0000","last_unstale":"2024-09-13T18:22:28.820616+0000","last_undegraded":"2024-09-13T18:22:28.820616+0000","last_fullsized":"2024-09-13T18:22:28.820616+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:40:11.407643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"91'4","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.289505+0000","last_change":"2024-09-13T18:20:27.190451+0000","last_active":"2024-09-13T18:22:28.289505+0000","last_peered":"2024-09-13T18:22:28.289505+0000","last_clean":"2024-09-13T18:22:28.289505+0000","last_became_active":"2024-09-13T18:20:27.187091+0000","last_became_peered":"2024-09-13T18:20:27.187091+0000","last_unstale":"2024-09-13T18:22:28.289505+0000","last_undegraded":"2024-09-13T18:22:28.289505+0000","last_fullsized":"2024-09-13T18:22:28.289505+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:57:07.976008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.136965+0000","last_change":"2024-09-13T18:20:06.133475+0000","last_active":"2024-09-13T18:22:28.136965+0000","last_peered":"2024-09-13T18:22:28.136965+0000","last_clean":"2024-09-13T18:22:28.136965+0000","last_became_active":"2024-09-13T18:20:06.133109+0000","last_became_peered":"2024-09-13T18:20:06.133109+0000","last_unstale":"2024-09-13T18:22:28.136965+0000","last_undegraded":"2024-09-13T18:22:28.136965+0000","last_fullsized":"2024-09-13T18:22:28.136965+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:47:19.567115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"93'10","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.292323+0000","last_change":"2024-09-13T18:20:01.922660+0000","last_active":"2024-09-13T18:22:28.292323+0000","last_peered":"2024-09-13T18:22:28.292323+0000","last_clean":"2024-09-13T18:22:28.292323+0000","last_became_active":"2024-09-13T18:20:01.922370+0000","last_became_peered":"2024-09-13T18:20:01.922370+0000","last_unstale":"2024-09-13T18:22:28.292323+0000","last_undegraded":"2024-09-13T18:22:28.292323+0000","last_fullsized":"2024-09-13T18:22:28.292323+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:54:32.029704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"93'6","reported_seq":270,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.420091+0000","last_change":"2024-09-13T18:20:01.580975+0000","last_active":"2024-09-13T18:22:28.420091+0000","last_peered":"2024-09-13T18:22:28.420091+0000","last_clean":"2024-09-13T18:22:28.420091+0000","last_became_active":"2024-09-13T18:20:01.580647+0000","last_became_peered":"2024-09-13T18:20:01.580647+0000","last_unstale":"2024-09-13T18:22:28.420091+0000","last_undegraded":"2024-09-13T18:22:28.420091+0000","last_fullsized":"2024-09-13T18:22:28.420091+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:26:20.942915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":242,"num_read_kb":242,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.066097+0000","last_change":"2024-09-13T18:20:06.140003+0000","last_active":"2024-09-13T18:22:28.066097+0000","last_peered":"2024-09-13T18:22:28.066097+0000","last_clean":"2024-09-13T18:22:28.066097+0000","last_became_active":"2024-09-13T18:20:06.139663+0000","last_became_peered":"2024-09-13T18:20:06.139663+0000","last_unstale":"2024-09-13T18:22:28.066097+0000","last_undegraded":"2024-09-13T18:22:28.066097+0000","last_fullsized":"2024-09-13T18:22:28.066097+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:54:49.662401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:07.284799+0000","last_change":"2024-09-13T18:20:06.134448+0000","last_active":"2024-09-13T18:22:07.284799+0000","last_peered":"2024-09-13T18:22:07.284799+0000","last_clean":"2024-09-13T18:22:07.284799+0000","last_became_active":"2024-09-13T18:20:06.134084+0000","last_became_peered":"2024-09-13T18:20:06.134084+0000","last_unstale":"2024-09-13T18:22:07.284799+0000","last_undegraded":"2024-09-13T18:22:07.284799+0000","last_fullsized":"2024-09-13T18:22:07.284799+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:55:50.475665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"93'11","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:55.907316+0000","last_change":"2024-09-13T18:20:01.918955+0000","last_active":"2024-09-13T18:21:55.907316+0000","last_peered":"2024-09-13T18:21:55.907316+0000","last_clean":"2024-09-13T18:21:55.907316+0000","last_became_active":"2024-09-13T18:20:01.918666+0000","last_became_peered":"2024-09-13T18:20:01.918666+0000","last_unstale":"2024-09-13T18:21:55.907316+0000","last_undegraded":"2024-09-13T18:21:55.907316+0000","last_fullsized":"2024-09-13T18:21:55.907316+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:08:50.697710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"91'3","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.145838+0000","last_change":"2024-09-13T18:20:01.578789+0000","last_active":"2024-09-13T18:22:28.145838+0000","last_peered":"2024-09-13T18:22:28.145838+0000","last_clean":"2024-09-13T18:22:28.145838+0000","last_became_active":"2024-09-13T18:20:01.578476+0000","last_became_peered":"2024-09-13T18:20:01.578476+0000","last_unstale":"2024-09-13T18:22:28.145838+0000","last_undegraded":"2024-09-13T18:22:28.145838+0000","last_fullsized":"2024-09-13T18:22:28.145838+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:40:58.355492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:12.959377+0000","last_change":"2024-09-13T18:21:28.423633+0000","last_active":"2024-09-13T18:22:12.959377+0000","last_peered":"2024-09-13T18:22:12.959377+0000","last_clean":"2024-09-13T18:22:12.959377+0000","last_became_active":"2024-09-13T18:21:28.423128+0000","last_became_peered":"2024-09-13T18:21:28.423128+0000","last_unstale":"2024-09-13T18:22:12.959377+0000","last_undegraded":"2024-09-13T18:22:12.959377+0000","last_fullsized":"2024-09-13T18:22:12.959377+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:06:32.687874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"93'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.066097+0000","last_change":"2024-09-13T18:20:01.914804+0000","last_active":"2024-09-13T18:22:28.066097+0000","last_peered":"2024-09-13T18:22:28.066097+0000","last_clean":"2024-09-13T18:22:28.066097+0000","last_became_active":"2024-09-13T18:20:01.914517+0000","last_became_peered":"2024-09-13T18:20:01.914517+0000","last_unstale":"2024-09-13T18:22:28.066097+0000","last_undegraded":"2024-09-13T18:22:28.066097+0000","last_fullsized":"2024-09-13T18:22:28.066097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:57:10.734935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"91'3","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.292059+0000","last_change":"2024-09-13T18:20:01.569201+0000","last_active":"2024-09-13T18:22:28.292059+0000","last_peered":"2024-09-13T18:22:28.292059+0000","last_clean":"2024-09-13T18:22:28.292059+0000","last_became_active":"2024-09-13T18:20:01.568908+0000","last_became_peered":"2024-09-13T18:20:01.568908+0000","last_unstale":"2024-09-13T18:22:28.292059+0000","last_undegraded":"2024-09-13T18:22:28.292059+0000","last_fullsized":"2024-09-13T18:22:28.292059+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:16:34.069130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"91'7","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:28.230871+0000","last_change":"2024-09-13T18:20:01.579225+0000","last_active":"2024-09-13T18:21:28.230871+0000","last_peered":"2024-09-13T18:21:28.230871+0000","last_clean":"2024-09-13T18:21:28.230871+0000","last_became_active":"2024-09-13T18:20:01.578826+0000","last_became_peered":"2024-09-13T18:20:01.578826+0000","last_unstale":"2024-09-13T18:21:28.230871+0000","last_undegraded":"2024-09-13T18:21:28.230871+0000","last_fullsized":"2024-09-13T18:21:28.230871+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:48:07.632239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"93'8","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:55.916389+0000","last_change":"2024-09-13T18:20:01.914885+0000","last_active":"2024-09-13T18:21:55.916389+0000","last_peered":"2024-09-13T18:21:55.916389+0000","last_clean":"2024-09-13T18:21:55.916389+0000","last_became_active":"2024-09-13T18:20:01.914599+0000","last_became_peered":"2024-09-13T18:20:01.914599+0000","last_unstale":"2024-09-13T18:21:55.916389+0000","last_undegraded":"2024-09-13T18:21:55.916389+0000","last_fullsized":"2024-09-13T18:21:55.916389+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:41:30.888675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:28.311102+0000","last_change":"2024-09-13T18:20:06.129376+0000","last_active":"2024-09-13T18:21:28.311102+0000","last_peered":"2024-09-13T18:21:28.311102+0000","last_clean":"2024-09-13T18:21:28.311102+0000","last_became_active":"2024-09-13T18:20:06.129032+0000","last_became_peered":"2024-09-13T18:20:06.129032+0000","last_unstale":"2024-09-13T18:21:28.311102+0000","last_undegraded":"2024-09-13T18:21:28.311102+0000","last_fullsized":"2024-09-13T18:21:28.311102+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:21:53.511813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'7","reported_seq":154,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.421305+0000","last_change":"2024-09-13T18:20:01.576069+0000","last_active":"2024-09-13T18:22:28.421305+0000","last_peered":"2024-09-13T18:22:28.421305+0000","last_clean":"2024-09-13T18:22:28.421305+0000","last_became_active":"2024-09-13T18:20:01.575760+0000","last_became_peered":"2024-09-13T18:20:01.575760+0000","last_unstale":"2024-09-13T18:22:28.421305+0000","last_undegraded":"2024-09-13T18:22:28.421305+0000","last_fullsized":"2024-09-13T18:22:28.421305+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:05:47.677848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1460,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":125,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"93'7","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:55.900375+0000","last_change":"2024-09-13T18:20:01.917073+0000","last_active":"2024-09-13T18:21:55.900375+0000","last_peered":"2024-09-13T18:21:55.900375+0000","last_clean":"2024-09-13T18:21:55.900375+0000","last_became_active":"2024-09-13T18:20:01.916765+0000","last_became_peered":"2024-09-13T18:20:01.916765+0000","last_unstale":"2024-09-13T18:21:55.900375+0000","last_undegraded":"2024-09-13T18:21:55.900375+0000","last_fullsized":"2024-09-13T18:21:55.900375+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:22:07.788313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:28.311181+0000","last_change":"2024-09-13T18:20:06.131429+0000","last_active":"2024-09-13T18:21:28.311181+0000","last_peered":"2024-09-13T18:21:28.311181+0000","last_clean":"2024-09-13T18:21:28.311181+0000","last_became_active":"2024-09-13T18:20:06.131019+0000","last_became_peered":"2024-09-13T18:20:06.131019+0000","last_unstale":"2024-09-13T18:21:28.311181+0000","last_undegraded":"2024-09-13T18:21:28.311181+0000","last_fullsized":"2024-09-13T18:21:28.311181+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:44:37.892574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"93'9","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:07.958537+0000","last_change":"2024-09-13T18:20:01.914050+0000","last_active":"2024-09-13T18:22:07.958537+0000","last_peered":"2024-09-13T18:22:07.958537+0000","last_clean":"2024-09-13T18:22:07.958537+0000","last_became_active":"2024-09-13T18:20:01.913733+0000","last_became_peered":"2024-09-13T18:20:01.913733+0000","last_unstale":"2024-09-13T18:22:07.958537+0000","last_undegraded":"2024-09-13T18:22:07.958537+0000","last_fullsized":"2024-09-13T18:22:07.958537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:52:43.672096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"93'12","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:10.788565+0000","last_change":"2024-09-13T18:20:01.924290+0000","last_active":"2024-09-13T18:22:10.788565+0000","last_peered":"2024-09-13T18:22:10.788565+0000","last_clean":"2024-09-13T18:22:10.788565+0000","last_became_active":"2024-09-13T18:20:01.923883+0000","last_became_peered":"2024-09-13T18:20:01.923883+0000","last_unstale":"2024-09-13T18:22:10.788565+0000","last_undegraded":"2024-09-13T18:22:10.788565+0000","last_fullsized":"2024-09-13T18:22:10.788565+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:11:18.737501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"93'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:10.788577+0000","last_change":"2024-09-13T18:20:27.184256+0000","last_active":"2024-09-13T18:22:10.788577+0000","last_peered":"2024-09-13T18:22:10.788577+0000","last_clean":"2024-09-13T18:22:10.788577+0000","last_became_active":"2024-09-13T18:20:27.183790+0000","last_became_peered":"2024-09-13T18:20:27.183790+0000","last_unstale":"2024-09-13T18:22:10.788577+0000","last_undegraded":"2024-09-13T18:22:10.788577+0000","last_fullsized":"2024-09-13T18:22:10.788577+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:06:48.885753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"93'7","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:07.284763+0000","last_change":"2024-09-13T18:20:01.915799+0000","last_active":"2024-09-13T18:22:07.284763+0000","last_peered":"2024-09-13T18:22:07.284763+0000","last_clean":"2024-09-13T18:22:07.284763+0000","last_became_active":"2024-09-13T18:20:01.915519+0000","last_became_peered":"2024-09-13T18:20:01.915519+0000","last_unstale":"2024-09-13T18:22:07.284763+0000","last_undegraded":"2024-09-13T18:22:07.284763+0000","last_fullsized":"2024-09-13T18:22:07.284763+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:08:26.065052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"93'4","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:13.482862+0000","last_change":"2024-09-13T18:20:01.918547+0000","last_active":"2024-09-13T18:22:13.482862+0000","last_peered":"2024-09-13T18:22:13.482862+0000","last_clean":"2024-09-13T18:22:13.482862+0000","last_became_active":"2024-09-13T18:20:01.918053+0000","last_became_peered":"2024-09-13T18:20:01.918053+0000","last_unstale":"2024-09-13T18:22:13.482862+0000","last_undegraded":"2024-09-13T18:22:13.482862+0000","last_fullsized":"2024-09-13T18:22:13.482862+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:25:58.235101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"93'6","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:12.959339+0000","last_change":"2024-09-13T18:20:01.916205+0000","last_active":"2024-09-13T18:22:12.959339+0000","last_peered":"2024-09-13T18:22:12.959339+0000","last_clean":"2024-09-13T18:22:12.959339+0000","last_became_active":"2024-09-13T18:20:01.915909+0000","last_became_peered":"2024-09-13T18:20:01.915909+0000","last_unstale":"2024-09-13T18:22:12.959339+0000","last_undegraded":"2024-09-13T18:22:12.959339+0000","last_fullsized":"2024-09-13T18:22:12.959339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:23:03.976513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"93'15","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:15.789324+0000","last_change":"2024-09-13T18:20:01.916621+0000","last_active":"2024-09-13T18:22:15.789324+0000","last_peered":"2024-09-13T18:22:15.789324+0000","last_clean":"2024-09-13T18:22:15.789324+0000","last_became_active":"2024-09-13T18:20:01.916335+0000","last_became_peered":"2024-09-13T18:20:01.916335+0000","last_unstale":"2024-09-13T18:22:15.789324+0000","last_undegraded":"2024-09-13T18:22:15.789324+0000","last_fullsized":"2024-09-13T18:22:15.789324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:08:16.104571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"93'2","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:13.101502+0000","last_change":"2024-09-13T18:20:01.921195+0000","last_active":"2024-09-13T18:22:13.101502+0000","last_peered":"2024-09-13T18:22:13.101502+0000","last_clean":"2024-09-13T18:22:13.101502+0000","last_became_active":"2024-09-13T18:20:01.920895+0000","last_became_peered":"2024-09-13T18:20:01.920895+0000","last_unstale":"2024-09-13T18:22:13.101502+0000","last_undegraded":"2024-09-13T18:22:13.101502+0000","last_fullsized":"2024-09-13T18:22:13.101502+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:59:53.920047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"93'6","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:16.035058+0000","last_change":"2024-09-13T18:20:01.910889+0000","last_active":"2024-09-13T18:22:16.035058+0000","last_peered":"2024-09-13T18:22:16.035058+0000","last_clean":"2024-09-13T18:22:16.035058+0000","last_became_active":"2024-09-13T18:20:01.910451+0000","last_became_peered":"2024-09-13T18:20:01.910451+0000","last_unstale":"2024-09-13T18:22:16.035058+0000","last_undegraded":"2024-09-13T18:22:16.035058+0000","last_fullsized":"2024-09-13T18:22:16.035058+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:06:52.552785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"93'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:18.101915+0000","last_change":"2024-09-13T18:20:01.914041+0000","last_active":"2024-09-13T18:22:18.101915+0000","last_peered":"2024-09-13T18:22:18.101915+0000","last_clean":"2024-09-13T18:22:18.101915+0000","last_became_active":"2024-09-13T18:20:01.913240+0000","last_became_peered":"2024-09-13T18:20:01.913240+0000","last_unstale":"2024-09-13T18:22:18.101915+0000","last_undegraded":"2024-09-13T18:22:18.101915+0000","last_fullsized":"2024-09-13T18:22:18.101915+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:39:37.062544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"93'10","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:18.277086+0000","last_change":"2024-09-13T18:20:01.927322+0000","last_active":"2024-09-13T18:22:18.277086+0000","last_peered":"2024-09-13T18:22:18.277086+0000","last_clean":"2024-09-13T18:22:18.277086+0000","last_became_active":"2024-09-13T18:20:01.926767+0000","last_became_peered":"2024-09-13T18:20:01.926767+0000","last_unstale":"2024-09-13T18:22:18.277086+0000","last_undegraded":"2024-09-13T18:22:18.277086+0000","last_fullsized":"2024-09-13T18:22:18.277086+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:20.986089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"93'11","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:18.483303+0000","last_change":"2024-09-13T18:20:01.916314+0000","last_active":"2024-09-13T18:22:18.483303+0000","last_peered":"2024-09-13T18:22:18.483303+0000","last_clean":"2024-09-13T18:22:18.483303+0000","last_became_active":"2024-09-13T18:20:01.915790+0000","last_became_peered":"2024-09-13T18:20:01.915790+0000","last_unstale":"2024-09-13T18:22:18.483303+0000","last_undegraded":"2024-09-13T18:22:18.483303+0000","last_fullsized":"2024-09-13T18:22:18.483303+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:20:30.063814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"93'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:17.959902+0000","last_change":"2024-09-13T18:20:27.187172+0000","last_active":"2024-09-13T18:22:17.959902+0000","last_peered":"2024-09-13T18:22:17.959902+0000","last_clean":"2024-09-13T18:22:17.959902+0000","last_became_active":"2024-09-13T18:20:27.186765+0000","last_became_peered":"2024-09-13T18:20:27.186765+0000","last_unstale":"2024-09-13T18:22:17.959902+0000","last_undegraded":"2024-09-13T18:22:17.959902+0000","last_fullsized":"2024-09-13T18:22:17.959902+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:37:01.469560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"93'7","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:20.789864+0000","last_change":"2024-09-13T18:20:01.924074+0000","last_active":"2024-09-13T18:22:20.789864+0000","last_peered":"2024-09-13T18:22:20.789864+0000","last_clean":"2024-09-13T18:22:20.789864+0000","last_became_active":"2024-09-13T18:20:01.922364+0000","last_became_peered":"2024-09-13T18:20:01.922364+0000","last_unstale":"2024-09-13T18:22:20.789864+0000","last_undegraded":"2024-09-13T18:22:20.789864+0000","last_fullsized":"2024-09-13T18:22:20.789864+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:51:33.824362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"93'10","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.138025+0000","last_change":"2024-09-13T18:20:01.914606+0000","last_active":"2024-09-13T18:22:28.138025+0000","last_peered":"2024-09-13T18:22:28.138025+0000","last_clean":"2024-09-13T18:22:28.138025+0000","last_became_active":"2024-09-13T18:20:01.914281+0000","last_became_peered":"2024-09-13T18:20:01.914281+0000","last_unstale":"2024-09-13T18:22:28.138025+0000","last_undegraded":"2024-09-13T18:22:28.138025+0000","last_fullsized":"2024-09-13T18:22:28.138025+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:55:42.211959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"93'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:22.960471+0000","last_change":"2024-09-13T18:20:01.928876+0000","last_active":"2024-09-13T18:22:22.960471+0000","last_peered":"2024-09-13T18:22:22.960471+0000","last_clean":"2024-09-13T18:22:22.960471+0000","last_became_active":"2024-09-13T18:20:01.928534+0000","last_became_peered":"2024-09-13T18:20:01.928534+0000","last_unstale":"2024-09-13T18:22:22.960471+0000","last_undegraded":"2024-09-13T18:22:22.960471+0000","last_fullsized":"2024-09-13T18:22:22.960471+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:14:28.196746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"93'9","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.291764+0000","last_change":"2024-09-13T18:20:01.920642+0000","last_active":"2024-09-13T18:22:28.291764+0000","last_peered":"2024-09-13T18:22:28.291764+0000","last_clean":"2024-09-13T18:22:28.291764+0000","last_became_active":"2024-09-13T18:20:01.920335+0000","last_became_peered":"2024-09-13T18:20:01.920335+0000","last_unstale":"2024-09-13T18:22:28.291764+0000","last_undegraded":"2024-09-13T18:22:28.291764+0000","last_fullsized":"2024-09-13T18:22:28.291764+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:59:20.894563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"93'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:22.286145+0000","last_change":"2024-09-13T18:20:27.187529+0000","last_active":"2024-09-13T18:22:22.286145+0000","last_peered":"2024-09-13T18:22:22.286145+0000","last_clean":"2024-09-13T18:22:22.286145+0000","last_became_active":"2024-09-13T18:20:27.187123+0000","last_became_peered":"2024-09-13T18:20:27.187123+0000","last_unstale":"2024-09-13T18:22:22.286145+0000","last_undegraded":"2024-09-13T18:22:22.286145+0000","last_fullsized":"2024-09-13T18:22:22.286145+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:43:19.601235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"93'7","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:22.286160+0000","last_change":"2024-09-13T18:20:01.917111+0000","last_active":"2024-09-13T18:22:22.286160+0000","last_peered":"2024-09-13T18:22:22.286160+0000","last_clean":"2024-09-13T18:22:22.286160+0000","last_became_active":"2024-09-13T18:20:01.916826+0000","last_became_peered":"2024-09-13T18:20:01.916826+0000","last_unstale":"2024-09-13T18:22:22.286160+0000","last_undegraded":"2024-09-13T18:22:22.286160+0000","last_fullsized":"2024-09-13T18:22:22.286160+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:36:08.150028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"93'10","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.292386+0000","last_change":"2024-09-13T18:20:01.904496+0000","last_active":"2024-09-13T18:22:28.292386+0000","last_peered":"2024-09-13T18:22:28.292386+0000","last_clean":"2024-09-13T18:22:28.292386+0000","last_became_active":"2024-09-13T18:20:01.904199+0000","last_became_peered":"2024-09-13T18:20:01.904199+0000","last_unstale":"2024-09-13T18:22:28.292386+0000","last_undegraded":"2024-09-13T18:22:28.292386+0000","last_fullsized":"2024-09-13T18:22:28.292386+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:44.044162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"93'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.206304+0000","last_change":"2024-09-13T18:20:01.919023+0000","last_active":"2024-09-13T18:22:28.206304+0000","last_peered":"2024-09-13T18:22:28.206304+0000","last_clean":"2024-09-13T18:22:28.206304+0000","last_became_active":"2024-09-13T18:20:01.918661+0000","last_became_peered":"2024-09-13T18:20:01.918661+0000","last_unstale":"2024-09-13T18:22:28.206304+0000","last_undegraded":"2024-09-13T18:22:28.206304+0000","last_fullsized":"2024-09-13T18:22:28.206304+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:25:26.779029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"93'6","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:55.925241+0000","last_change":"2024-09-13T18:20:01.922833+0000","last_active":"2024-09-13T18:21:55.925241+0000","last_peered":"2024-09-13T18:21:55.925241+0000","last_clean":"2024-09-13T18:21:55.925241+0000","last_became_active":"2024-09-13T18:20:01.922533+0000","last_became_peered":"2024-09-13T18:20:01.922533+0000","last_unstale":"2024-09-13T18:21:55.925241+0000","last_undegraded":"2024-09-13T18:21:55.925241+0000","last_fullsized":"2024-09-13T18:21:55.925241+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:27:11.557419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"93'16","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.066646+0000","last_change":"2024-09-13T18:20:01.925916+0000","last_active":"2024-09-13T18:22:28.066646+0000","last_peered":"2024-09-13T18:22:28.066646+0000","last_clean":"2024-09-13T18:22:28.066646+0000","last_became_active":"2024-09-13T18:20:01.925568+0000","last_became_peered":"2024-09-13T18:20:01.925568+0000","last_unstale":"2024-09-13T18:22:28.066646+0000","last_undegraded":"2024-09-13T18:22:28.066646+0000","last_fullsized":"2024-09-13T18:22:28.066646+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:58:24.955273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"93'11","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.292300+0000","last_change":"2024-09-13T18:20:01.924328+0000","last_active":"2024-09-13T18:22:28.292300+0000","last_peered":"2024-09-13T18:22:28.292300+0000","last_clean":"2024-09-13T18:22:28.292300+0000","last_became_active":"2024-09-13T18:20:01.924004+0000","last_became_peered":"2024-09-13T18:20:01.924004+0000","last_unstale":"2024-09-13T18:22:28.292300+0000","last_undegraded":"2024-09-13T18:22:28.292300+0000","last_fullsized":"2024-09-13T18:22:28.292300+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:23:29.204157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"93'9","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:55.913636+0000","last_change":"2024-09-13T18:20:01.922234+0000","last_active":"2024-09-13T18:21:55.913636+0000","last_peered":"2024-09-13T18:21:55.913636+0000","last_clean":"2024-09-13T18:21:55.913636+0000","last_became_active":"2024-09-13T18:20:01.921196+0000","last_became_peered":"2024-09-13T18:20:01.921196+0000","last_unstale":"2024-09-13T18:21:55.913636+0000","last_undegraded":"2024-09-13T18:21:55.913636+0000","last_fullsized":"2024-09-13T18:21:55.913636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:24:10.144346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"93'5","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:55.917809+0000","last_change":"2024-09-13T18:20:01.921740+0000","last_active":"2024-09-13T18:21:55.917809+0000","last_peered":"2024-09-13T18:21:55.917809+0000","last_clean":"2024-09-13T18:21:55.917809+0000","last_became_active":"2024-09-13T18:20:01.919634+0000","last_became_peered":"2024-09-13T18:20:01.919634+0000","last_unstale":"2024-09-13T18:21:55.917809+0000","last_undegraded":"2024-09-13T18:21:55.917809+0000","last_fullsized":"2024-09-13T18:21:55.917809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:43:27.600637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"93'6","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:55.915209+0000","last_change":"2024-09-13T18:20:01.924806+0000","last_active":"2024-09-13T18:21:55.915209+0000","last_peered":"2024-09-13T18:21:55.915209+0000","last_clean":"2024-09-13T18:21:55.915209+0000","last_became_active":"2024-09-13T18:20:01.924514+0000","last_became_peered":"2024-09-13T18:20:01.924514+0000","last_unstale":"2024-09-13T18:21:55.915209+0000","last_undegraded":"2024-09-13T18:21:55.915209+0000","last_fullsized":"2024-09-13T18:21:55.915209+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:47.323362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"93'4","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:55.901946+0000","last_change":"2024-09-13T18:20:01.912402+0000","last_active":"2024-09-13T18:21:55.901946+0000","last_peered":"2024-09-13T18:21:55.901946+0000","last_clean":"2024-09-13T18:21:55.901946+0000","last_became_active":"2024-09-13T18:20:01.911534+0000","last_became_peered":"2024-09-13T18:20:01.911534+0000","last_unstale":"2024-09-13T18:21:55.901946+0000","last_undegraded":"2024-09-13T18:21:55.901946+0000","last_fullsized":"2024-09-13T18:21:55.901946+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:22:52.021688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"93'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:01.737020+0000","last_change":"2024-09-13T18:20:01.930248+0000","last_active":"2024-09-13T18:22:01.737020+0000","last_peered":"2024-09-13T18:22:01.737020+0000","last_clean":"2024-09-13T18:22:01.737020+0000","last_became_active":"2024-09-13T18:20:01.929931+0000","last_became_peered":"2024-09-13T18:20:01.929931+0000","last_unstale":"2024-09-13T18:22:01.737020+0000","last_undegraded":"2024-09-13T18:22:01.737020+0000","last_fullsized":"2024-09-13T18:22:01.737020+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:37:33.086722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"93'7","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:07.284781+0000","last_change":"2024-09-13T18:20:01.921913+0000","last_active":"2024-09-13T18:22:07.284781+0000","last_peered":"2024-09-13T18:22:07.284781+0000","last_clean":"2024-09-13T18:22:07.284781+0000","last_became_active":"2024-09-13T18:20:01.921395+0000","last_became_peered":"2024-09-13T18:20:01.921395+0000","last_unstale":"2024-09-13T18:22:07.284781+0000","last_undegraded":"2024-09-13T18:22:07.284781+0000","last_fullsized":"2024-09-13T18:22:07.284781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:54:50.816870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"93'4","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:07.284827+0000","last_change":"2024-09-13T18:20:01.912654+0000","last_active":"2024-09-13T18:22:07.284827+0000","last_peered":"2024-09-13T18:22:07.284827+0000","last_clean":"2024-09-13T18:22:07.284827+0000","last_became_active":"2024-09-13T18:20:01.912356+0000","last_became_peered":"2024-09-13T18:20:01.912356+0000","last_unstale":"2024-09-13T18:22:07.284827+0000","last_undegraded":"2024-09-13T18:22:07.284827+0000","last_fullsized":"2024-09-13T18:22:07.284827+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:25:18.099396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"93'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:07.958659+0000","last_change":"2024-09-13T18:20:01.923373+0000","last_active":"2024-09-13T18:22:07.958659+0000","last_peered":"2024-09-13T18:22:07.958659+0000","last_clean":"2024-09-13T18:22:07.958659+0000","last_became_active":"2024-09-13T18:20:01.923077+0000","last_became_peered":"2024-09-13T18:20:01.923077+0000","last_unstale":"2024-09-13T18:22:07.958659+0000","last_undegraded":"2024-09-13T18:22:07.958659+0000","last_fullsized":"2024-09-13T18:22:07.958659+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:07:52.955705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":604133,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1347,"num_read_kb":793,"num_write":365,"num_write_kb":1467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":712704,"data_stored":1812399,"data_compressed":80730,"data_compressed_allocated":110592,"data_compressed_original":1572864,"omap_allocated":0,"internal_metadata":0},"log_size":299,"ondisk_log_size":299,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107472,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383723,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":1044480,"data_stored":529645,"data_compressed":26479,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:22:20 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.51200000000000001,"15min":0.48799999999999999},"min":{"1min":0.36299999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.70899999999999996,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.48899999999999999,"15min":0.46899999999999997},"min":{"1min":0.35999999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.70299999999999996,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.33600000000000002}]},{"osd":1,"last update":"Fri Sep 13 18:22:20 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.59199999999999997,"15min":0.57399999999999995},"min":{"1min":0.46000000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.88100000000000001,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.57899999999999996,"15min":0.56899999999999995},"min":{"1min":0.40200000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.78700000000000003,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.38500000000000001}]},{"osd":2,"last update":"Fri Sep 13 18:22:20 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.45400000000000001,"15min":0.44900000000000001},"min":{"1min":0.34300000000000003,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.63100000000000001,"5min":1.238,"15min":1.238},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.44800000000000001,"15min":0.434},"min":{"1min":0.33700000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.70799999999999996,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.40000000000000002}]},{"osd":3,"last update":"Fri Sep 13 18:21:27 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.504},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.47199999999999998}]},{"osd":4,"last update":"Fri Sep 13 18:22:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.63100000000000001,"15min":0.60999999999999999},"min":{"1min":0.53600000000000003,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85899999999999999,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.58799999999999997,"15min":0.56899999999999995},"min":{"1min":0.47099999999999997,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.84199999999999997,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.42199999999999999}]},{"osd":5,"last update":"Fri Sep 13 18:22:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.51900000000000002,"15min":0.50600000000000001},"min":{"1min":0.36099999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.73899999999999999,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.624,"5min":0.54600000000000004,"15min":0.52200000000000002},"min":{"1min":0.47799999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.81699999999999995,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.439}]},{"osd":6,"last update":"Fri Sep 13 18:22:20 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57799999999999996,"15min":0.56899999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85499999999999998,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.61499999999999999,"15min":0.59799999999999998},"min":{"1min":0.54100000000000004,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.79300000000000004,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.45900000000000002}]},{"osd":7,"last update":"Fri Sep 13 18:22:20 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.67900000000000005,"15min":0.64700000000000002},"min":{"1min":0.59699999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.95599999999999996,"5min":1.123,"15min":1.123},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.66600000000000004,"15min":0.65000000000000002},"min":{"1min":0.53400000000000003,"5min":0.439,"15min":0.439},"max":{"1min":0.94099999999999995,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.48199999999999998}]},{"osd":8,"last update":"Fri Sep 13 18:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.65100000000000002,"15min":0.63800000000000001},"min":{"1min":0.55500000000000005,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.72299999999999998,"15min":0.71099999999999997},"min":{"1min":0.64000000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.64300000000000002}]},{"osd":9,"last update":"Fri Sep 13 18:22:20 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.70099999999999996,"15min":0.68899999999999995},"min":{"1min":0.60399999999999998,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.91200000000000003,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.71499999999999997,"15min":0.68700000000000006},"min":{"1min":0.624,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.95399999999999996,"5min":1.208,"15min":1.208},"last":0.68200000000000005}]},{"osd":10,"last update":"Fri Sep 13 18:22:20 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.69999999999999996,"15min":0.67900000000000005},"min":{"1min":0.621,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.91100000000000003,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.72399999999999998,"15min":0.70599999999999996},"min":{"1min":0.66500000000000004,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.91500000000000004,"5min":1.583,"15min":1.583},"last":0.57599999999999996}]}]},{"osd":10,"up_from":72,"seq":309237645359,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28208,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716944,"statfs":{"total":95995035648,"available":95966150656,"internally_reserved":0,"allocated":1212416,"data_stored":1321739,"data_compressed":70067,"data_compressed_allocated":94208,"data_compressed_original":794624,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:22:03 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.502,"15min":0.504},"min":{"1min":0.40300000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.66700000000000004,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":1.018,"15min":1.107},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.624,"5min":14.589,"15min":14.589},"last":0.373}]},{"osd":1,"last update":"Fri Sep 13 18:21:55 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.41599999999999998,"15min":0.41999999999999998},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.54700000000000004,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.41999999999999998,"15min":0.41699999999999998},"min":{"1min":0.309,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.57799999999999996,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.29499999999999998}]},{"osd":2,"last update":"Fri Sep 13 18:21:55 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55000000000000004,"15min":0.54400000000000004},"min":{"1min":0.33700000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.72599999999999998,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56499999999999995,"15min":0.55500000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.74199999999999999,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.53100000000000003}]},{"osd":3,"last update":"Fri Sep 13 18:21:55 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.56399999999999995,"15min":0.57399999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.70499999999999996,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.68200000000000005,"15min":0.61099999999999999},"min":{"1min":0.377,"5min":0.376,"15min":0.376},"max":{"1min":0.67300000000000004,"5min":14.505000000000001,"15min":14.505000000000001},"last":0.55100000000000005}]},{"osd":4,"last update":"Fri Sep 13 18:21:55 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.503,"15min":0.50800000000000001},"min":{"1min":0.34799999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.624,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.495,"15min":0.499},"min":{"1min":0.33800000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.68600000000000005,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.312}]},{"osd":5,"last update":"Fri Sep 13 18:21:55 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.60999999999999999,"15min":0.60199999999999998},"min":{"1min":0.45500000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.82799999999999996,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.626},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.71199999999999997,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.48599999999999999}]},{"osd":6,"last update":"Fri Sep 13 18:21:55 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.752,"15min":0.67700000000000005},"min":{"1min":0.496,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.77800000000000002,"5min":14.539999999999999,"15min":14.539999999999999},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.64200000000000002,"15min":0.64700000000000002},"min":{"1min":0.50800000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.92600000000000005,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.41699999999999998}]},{"osd":7,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.81599999999999995}]},{"osd":8,"last update":"Fri Sep 13 18:21:55 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.66700000000000004,"15min":0.67100000000000004},"min":{"1min":0.41999999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.90900000000000003,"5min":1.01,"15min":1.01},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.68899999999999995,"15min":0.69299999999999995},"min":{"1min":0.51300000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.81599999999999995,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.54200000000000004}]},{"osd":9,"last update":"Fri Sep 13 18:21:55 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.68400000000000005,"15min":0.69099999999999995},"min":{"1min":0.53600000000000003,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.89400000000000002,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.436},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.69999999999999996,"15min":0.71299999999999997},"min":{"1min":0.45800000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.81100000000000005,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.497}]},{"osd":11,"last update":"Fri Sep 13 18:22:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.72099999999999997,"15min":0.72899999999999998},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.84199999999999997,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.745,"5min":0.73799999999999999,"15min":0.73999999999999999},"min":{"1min":0.55400000000000005,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.89900000000000002,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.56200000000000006}]}]},{"osd":9,"up_from":67,"seq":287762808884,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1089536,"data_stored":922323,"data_compressed":45625,"data_compressed_allocated":61440,"data_compressed_original":466944,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:21:33 2024","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.36099999999999999,"15min":0.38100000000000001},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.45800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.41399999999999998,"15min":0.442},"min":{"1min":0.249,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.495,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.38700000000000001}]},{"osd":1,"last update":"Fri Sep 13 18:21:33 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.56299999999999994,"15min":0.58399999999999996},"min":{"1min":0.35699999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.67300000000000004,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.56399999999999995,"15min":0.58499999999999996},"min":{"1min":0.36399999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.67700000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.36599999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:21:33 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.55000000000000004,"15min":0.57399999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.65000000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.52600000000000002,"15min":0.52800000000000002},"min":{"1min":0.34899999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.42099999999999999}]},{"osd":3,"last update":"Fri Sep 13 18:21:33 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.44700000000000001,"15min":0.48099999999999998},"min":{"1min":0.28499999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.50900000000000001,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.37,"5min":0.437,"15min":0.46300000000000002},"min":{"1min":0.28000000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.496,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.32300000000000001}]},{"osd":4,"last update":"Fri Sep 13 18:21:33 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.55600000000000005,"15min":0.57299999999999995},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.67700000000000005,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.61299999999999999,"15min":0.64700000000000002},"min":{"1min":0.39000000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.65300000000000002,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.54200000000000004}]},{"osd":5,"last update":"Fri Sep 13 18:21:33 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.57199999999999995,"15min":0.59099999999999997},"min":{"1min":0.36599999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.59299999999999997,"15min":0.61099999999999999},"min":{"1min":0.40600000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.752,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.40400000000000003}]},{"osd":6,"last update":"Fri Sep 13 18:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.39800000000000002}]},{"osd":7,"last update":"Fri Sep 13 18:21:33 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.63600000000000001,"15min":0.67400000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.70899999999999996,"5min":0.871,"15min":0.871},"last":0.437},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.64700000000000002,"15min":0.68100000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.63900000000000001,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.42899999999999999}]},{"osd":8,"last update":"Fri Sep 13 18:21:33 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.63500000000000001,"15min":0.66500000000000004},"min":{"1min":0.40999999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.75,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.66500000000000004,"15min":0.69999999999999996},"min":{"1min":0.436,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.81899999999999995,"5min":1.02,"15min":1.02},"last":0.51900000000000002}]},{"osd":10,"last update":"Fri Sep 13 18:21:54 2024","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":0.73499999999999999,"15min":0.69299999999999995},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":9.3510000000000009,"5min":9.3510000000000009,"15min":9.3510000000000009},"last":0.505},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.65900000000000003,"15min":0.68300000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.79100000000000004,"5min":0.879,"15min":0.879},"last":0.498}]},{"osd":11,"last update":"Fri Sep 13 18:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.48199999999999998,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.72999999999999998,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.45900000000000002,"5min":0.432,"15min":0.432},"max":{"1min":0.65300000000000002,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.377}]}]},{"osd":8,"up_from":59,"seq":253403070521,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1081344,"data_stored":922679,"data_compressed":45625,"data_compressed_allocated":61440,"data_compressed_original":466944,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.51300000000000001,"15min":0.54000000000000004},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.72299999999999998,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.55300000000000005,"15min":0.56299999999999994},"min":{"1min":0.35199999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.65000000000000002,"5min":3.2989999999999999,"15min":3.2989999999999999},"last":0.44600000000000001}]},{"osd":1,"last update":"Fri Sep 13 18:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.57699999999999996,"15min":0.56499999999999995},"min":{"1min":0.36699999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.79100000000000004,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.58399999999999996,"15min":0.57599999999999996},"min":{"1min":0.42399999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72699999999999998,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.36199999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.42499999999999999,"15min":0.42899999999999999},"min":{"1min":0.25,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.58199999999999996,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.314},{"interface":"front","average":{"1min":0.442,"5min":0.439,"15min":0.437},"min":{"1min":0.307,"5min":0.224,"15min":0.224},"max":{"1min":0.68899999999999995,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.56699999999999995}]},{"osd":3,"last update":"Fri Sep 13 18:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53700000000000003,"15min":0.55500000000000005},"min":{"1min":0.35899999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.69999999999999996,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.53400000000000003,"15min":0.53800000000000003},"min":{"1min":0.37,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.68799999999999994,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.38800000000000001}]},{"osd":4,"last update":"Fri Sep 13 18:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.66400000000000003,"15min":0.63600000000000001},"min":{"1min":0.45100000000000001,"5min":0.37,"15min":0.37},"max":{"1min":5.3540000000000001,"5min":5.3540000000000001,"15min":5.3540000000000001},"last":0.68000000000000005}]},{"osd":5,"last update":"Fri Sep 13 18:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.52000000000000002,"15min":0.53400000000000003},"min":{"1min":0.33600000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.67500000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.51100000000000001,"15min":0.52100000000000002},"min":{"1min":0.33400000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.64900000000000002,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.40300000000000002}]},{"osd":6,"last update":"Fri Sep 13 18:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.61299999999999999,"15min":0.61499999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.77700000000000002,"5min":3.2719999999999998,"15min":3.2719999999999998},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.61499999999999999,"15min":0.64000000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.71999999999999997,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.48899999999999999}]},{"osd":7,"last update":"Fri Sep 13 18:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.65600000000000003,"15min":0.67200000000000004},"min":{"1min":0.45500000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.77900000000000003,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.66200000000000003,"15min":0.66500000000000004},"min":{"1min":0.51800000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.81999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.51400000000000001}]},{"osd":9,"last update":"Fri Sep 13 18:22:25 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63700000000000001,"15min":0.63300000000000001},"min":{"1min":0.50900000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73799999999999999,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.63500000000000001,"15min":0.64800000000000002},"min":{"1min":0.39400000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.82199999999999995,"5min":1.012,"15min":1.012},"last":0.505}]},{"osd":10,"last update":"Fri Sep 13 18:21:53 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.85499999999999998,"15min":0.93400000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.79800000000000004,"5min":5.3360000000000003,"15min":5.3360000000000003},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.67800000000000005,"15min":0.67400000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.80900000000000005,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.46999999999999997}]},{"osd":11,"last update":"Fri Sep 13 18:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.65500000000000003,"15min":0.65900000000000003},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.76000000000000001,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.499},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.71599999999999997,"15min":0.72199999999999998},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.79900000000000004,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.53000000000000003}]}]},{"osd":7,"up_from":51,"seq":219043332157,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1114112,"data_stored":1128889,"data_compressed":62303,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.94399999999999995,"15min":1.012},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.84299999999999997,"5min":12.068,"15min":12.068},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.46999999999999997,"15min":0.45900000000000002},"min":{"1min":0.35199999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.61599999999999999}]},{"osd":1,"last update":"Fri Sep 13 18:21:46 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.379,"15min":0.36099999999999999},"min":{"1min":0.27300000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.40300000000000002,"15min":0.38200000000000001},"min":{"1min":0.30399999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.54500000000000004,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.26500000000000001}]},{"osd":2,"last update":"Fri Sep 13 18:21:46 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.627,"15min":0.54900000000000004},"min":{"1min":0.40999999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.79100000000000004,"5min":7.6820000000000004,"15min":7.6820000000000004},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.54000000000000004,"15min":0.51500000000000001},"min":{"1min":0.45600000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.40500000000000003}]},{"osd":3,"last update":"Fri Sep 13 18:21:46 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.52900000000000003,"15min":0.49299999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.70599999999999996,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.52400000000000002,"15min":0.49399999999999999},"min":{"1min":0.35599999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.73699999999999999,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.38600000000000001}]},{"osd":4,"last update":"Fri Sep 13 18:21:46 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.45800000000000002,"15min":0.439},"min":{"1min":0.32000000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.57499999999999996,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.46100000000000002,"15min":0.45300000000000001},"min":{"1min":0.32500000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.66500000000000004,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.33000000000000002}]},{"osd":5,"last update":"Fri Sep 13 18:21:46 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.59299999999999997,"15min":0.56200000000000006},"min":{"1min":0.46600000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.76600000000000001,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.56899999999999995,"15min":0.54100000000000004},"min":{"1min":0.40999999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.748,"5min":0.754,"15min":0.754},"last":0.47299999999999998}]},{"osd":6,"last update":"Fri Sep 13 18:21:46 2024","interfaces":[{"interface":"back","average":{"1min":1.5700000000000001,"5min":0.77400000000000002,"15min":0.61499999999999999},"min":{"1min":0.40300000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":20.013000000000002,"5min":20.013000000000002,"15min":20.013000000000002},"last":0.438},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.60899999999999999,"15min":0.58699999999999997},"min":{"1min":0.38500000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72999999999999998,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.51000000000000001}]},{"osd":8,"last update":"Fri Sep 13 18:22:04 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60599999999999998,"15min":0.59199999999999997},"min":{"1min":0.497,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.70499999999999996,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.747,"15min":0.67000000000000004},"min":{"1min":0.505,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.79600000000000004,"5min":7.7149999999999999,"15min":7.7149999999999999},"last":0.53700000000000003}]},{"osd":9,"last update":"Fri Sep 13 18:21:29 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.66300000000000003,"15min":0.66100000000000003},"min":{"1min":0.40500000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.88,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.45600000000000002},{"interface":"front","average":{"1min":1.609,"5min":0.82999999999999996,"15min":0.70199999999999996},"min":{"1min":0.42699999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":20.056999999999999,"5min":20.056999999999999,"15min":20.056999999999999},"last":0.42399999999999999}]},{"osd":10,"last update":"Fri Sep 13 18:21:51 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.65100000000000002,"15min":0.65300000000000002},"min":{"1min":0.43099999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.79900000000000004,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.496},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.63800000000000001,"15min":0.64200000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.755,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.44700000000000001}]},{"osd":11,"last update":"Fri Sep 13 18:22:04 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.71899999999999997,"15min":0.72699999999999998},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.78500000000000003,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.502},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.73899999999999999,"15min":0.747},"min":{"1min":0.59999999999999998,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.80800000000000005,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.48799999999999999}]}]},{"osd":6,"up_from":46,"seq":197568495682,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":1040384,"data_stored":525841,"data_compressed":26479,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.37,"15min":0.36699999999999999},"min":{"1min":0.25900000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":1.2030000000000001},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.41399999999999998,"15min":0.432},"min":{"1min":0.28699999999999998,"5min":0.218,"15min":0.218},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":1.0940000000000001}]},{"osd":1,"last update":"Fri Sep 13 18:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.53700000000000003,"15min":0.52500000000000002},"min":{"1min":0.311,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.56399999999999995,"15min":0.54100000000000004},"min":{"1min":0.45300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":1.0669999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.42399999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78600000000000003,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.54700000000000004,"15min":0.55100000000000005},"min":{"1min":0.38800000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":1.2729999999999999}]},{"osd":3,"last update":"Fri Sep 13 18:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.41099999999999998,"15min":0.42199999999999999},"min":{"1min":0.28499999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":1.254},{"interface":"front","average":{"1min":0.499,"5min":0.46200000000000002,"15min":0.46999999999999997},"min":{"1min":0.34300000000000003,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":1.179}]},{"osd":4,"last update":"Fri Sep 13 18:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.53200000000000003,"15min":0.51800000000000002},"min":{"1min":0.35599999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.55400000000000005,"15min":0.55300000000000005},"min":{"1min":0.39400000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":1.2190000000000001}]},{"osd":5,"last update":"Fri Sep 13 18:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.59399999999999997,"15min":0.60599999999999998},"min":{"1min":0.48999999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":1.038},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.56299999999999994,"15min":0.58899999999999997},"min":{"1min":0.377,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.83999999999999997,"5min":0.874,"15min":0.874},"last":0.83999999999999997}]},{"osd":7,"last update":"Fri Sep 13 18:21:45 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57899999999999996,"15min":0.57499999999999996},"min":{"1min":0.41099999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.56299999999999994,"15min":0.56100000000000005},"min":{"1min":0.442,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.70399999999999996,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":1.2889999999999999}]},{"osd":8,"last update":"Fri Sep 13 18:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.61899999999999999,"15min":0.621},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.91800000000000004,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":1.01},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.58599999999999997,"15min":0.59599999999999997},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.71599999999999997,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":1.3340000000000001}]},{"osd":9,"last update":"Fri Sep 13 18:21:34 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.51500000000000001,"15min":0.53100000000000003},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.57199999999999995,"5min":0.871,"15min":0.871},"last":1.3089999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.59099999999999997,"15min":0.60499999999999998},"min":{"1min":0.41399999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.75700000000000001,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":1.153}]},{"osd":10,"last update":"Fri Sep 13 18:21:55 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.60599999999999998,"15min":0.61499999999999999},"min":{"1min":0.48099999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.71799999999999997,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.66700000000000004,"15min":0.67100000000000004},"min":{"1min":0.53300000000000003,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.78300000000000003,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.96799999999999997}]},{"osd":11,"last update":"Fri Sep 13 18:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.63900000000000001,"15min":0.64600000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.77400000000000002,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":1.3600000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68600000000000005,"15min":0.68700000000000006},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.90200000000000002,"5min":1.01,"15min":1.01},"last":1.127}]}]},{"osd":5,"up_from":38,"seq":163208757319,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1142784,"data_stored":1253611,"data_compressed":68459,"data_compressed_allocated":90112,"data_compressed_original":786432,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:22:00 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.497,"15min":0.495},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.65000000000000002,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.51200000000000001,"15min":0.51100000000000001},"min":{"1min":0.32300000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.73199999999999998,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.373}]},{"osd":1,"last update":"Fri Sep 13 18:22:00 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.54300000000000004,"15min":0.51100000000000001},"min":{"1min":0.33300000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.71599999999999997,"5min":0.878,"15min":0.878},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.53300000000000003,"15min":0.51400000000000001},"min":{"1min":0.45800000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.68899999999999995,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.45400000000000001}]},{"osd":2,"last update":"Fri Sep 13 18:22:00 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.39600000000000002,"15min":0.36799999999999999},"min":{"1min":0.27900000000000003,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.65100000000000002,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.315},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.39900000000000002,"15min":0.38700000000000001},"min":{"1min":0.29999999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.34399999999999997}]},{"osd":3,"last update":"Fri Sep 13 18:22:00 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.52300000000000002,"15min":0.503},"min":{"1min":0.34799999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.70199999999999996,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.52200000000000002,"15min":0.52000000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.46800000000000003}]},{"osd":4,"last update":"Fri Sep 13 18:22:00 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55900000000000005,"15min":0.54200000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.79000000000000004,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.79200000000000004,"15min":0.61799999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82999999999999996,"5min":18.571000000000002,"15min":18.571000000000002},"last":0.47699999999999998}]},{"osd":6,"last update":"Fri Sep 13 18:22:25 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57499999999999996,"15min":0.57399999999999995},"min":{"1min":0.47199999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.73299999999999998,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56000000000000005,"15min":0.55200000000000005},"min":{"1min":0.38400000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.53600000000000003}]},{"osd":7,"last update":"Fri Sep 13 18:21:45 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61599999999999999,"15min":0.60799999999999998},"min":{"1min":0.46300000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76400000000000001,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63100000000000001,"15min":0.61599999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.84999999999999998,"5min":0.88,"15min":0.88},"last":0.499}]},{"osd":8,"last update":"Fri Sep 13 18:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.52700000000000002,"15min":0.51600000000000001},"min":{"1min":0.40200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.56899999999999995,"15min":0.54500000000000004},"min":{"1min":0.40600000000000003,"5min":0.314,"15min":0.314},"max":{"1min":0.876,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.56499999999999995}]},{"osd":9,"last update":"Fri Sep 13 18:21:30 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.64500000000000002,"15min":0.627},"min":{"1min":0.48199999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.63900000000000001,"15min":0.628},"min":{"1min":0.48799999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.81299999999999994,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.57699999999999996}]},{"osd":10,"last update":"Fri Sep 13 18:21:54 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69199999999999995,"15min":0.69099999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.68000000000000005,"15min":0.67000000000000004},"min":{"1min":0.53300000000000003,"5min":0.37,"15min":0.37},"max":{"1min":0.89000000000000001,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.55900000000000005}]}]},{"osd":4,"up_from":33,"seq":141733920843,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":529915,"data_compressed":26479,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:19:33 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.48299999999999998,"15min":0.47699999999999998},"min":{"1min":0.35399999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.69399999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.433},{"interface":"front","average":{"1min":0.501,"5min":0.51200000000000001,"15min":0.47499999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.84599999999999997,"5min":4.2539999999999996,"15min":4.2539999999999996},"last":0.51900000000000002}]},{"osd":1,"last update":"Fri Sep 13 18:21:38 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.373,"15min":0.35199999999999998},"min":{"1min":0.25800000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.59199999999999997,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.379,"15min":0.35399999999999998},"min":{"1min":0.29199999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.68400000000000005,"5min":0.748,"15min":0.748},"last":0.39200000000000002}]},{"osd":2,"last update":"Fri Sep 13 18:21:38 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.60699999999999998,"15min":0.54900000000000004},"min":{"1min":0.40999999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":5.2930000000000001,"5min":5.2930000000000001,"15min":5.2930000000000001},"last":0.438},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55300000000000005,"15min":0.53800000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72499999999999998,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.51900000000000002}]},{"osd":3,"last update":"Fri Sep 13 18:21:38 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56000000000000005,"15min":0.505},"min":{"1min":0.434,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.82199999999999995,"5min":4.2679999999999998,"15min":4.2679999999999998},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.52100000000000002,"15min":0.496},"min":{"1min":0.46100000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.41399999999999998}]},{"osd":5,"last update":"Fri Sep 13 18:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.57099999999999995,"15min":0.54300000000000004},"min":{"1min":0.434,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.70399999999999996,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.58099999999999996,"15min":0.54700000000000004},"min":{"1min":0.40400000000000003,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.75900000000000001,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.42499999999999999}]},{"osd":6,"last update":"Fri Sep 13 18:22:21 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56399999999999995,"15min":0.52400000000000002},"min":{"1min":0.40400000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.754,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.56699999999999995,"15min":0.52300000000000002},"min":{"1min":0.47799999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.73899999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.50800000000000001}]},{"osd":7,"last update":"Fri Sep 13 18:21:48 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.46999999999999997,"15min":0.44600000000000001},"min":{"1min":0.376,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.76100000000000001,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.50900000000000001,"15min":0.48499999999999999},"min":{"1min":0.379,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.76100000000000001,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.48399999999999999}]},{"osd":8,"last update":"Fri Sep 13 18:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.64200000000000002,"15min":0.63},"min":{"1min":0.54900000000000004,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.76600000000000001,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.374},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.67000000000000004,"15min":0.65200000000000002},"min":{"1min":0.624,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.76900000000000002,"5min":0.874,"15min":0.874},"last":0.57499999999999996}]},{"osd":9,"last update":"Fri Sep 13 18:21:28 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.63300000000000001,"15min":0.61499999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.63700000000000001,"15min":0.629},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.83299999999999996,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.44700000000000001}]},{"osd":10,"last update":"Fri Sep 13 18:21:52 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.63400000000000001},"min":{"1min":0.438,"5min":0.377,"15min":0.377},"max":{"1min":0.81200000000000006,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.499},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.66100000000000003,"15min":0.65900000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.746,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.45600000000000002}]},{"osd":11,"last update":"Fri Sep 13 18:22:04 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.73299999999999998,"15min":0.73399999999999999},"min":{"1min":0.58099999999999996,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.89300000000000002,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.72499999999999998,"15min":0.72199999999999998},"min":{"1min":0.66300000000000003,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.81399999999999995,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.54800000000000004}]}]},{"osd":3,"up_from":25,"seq":107374182479,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":524038,"data_compressed":26479,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:22:18 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34699999999999998,"15min":0.313},"min":{"1min":0.216,"5min":0.183,"15min":0.183},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.34599999999999997,"15min":0.33000000000000002},"min":{"1min":0.23200000000000001,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.48099999999999998,"5min":0.58199999999999996,"15min":0.58199999999999996},"last":0.54700000000000004}]},{"osd":1,"last update":"Fri Sep 13 18:22:18 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.53700000000000003,"15min":0.497},"min":{"1min":0.41799999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.68400000000000005,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.622},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.54700000000000004,"15min":0.499},"min":{"1min":0.443,"5min":0.33700000000000002,"15min":0.313},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.68100000000000005}]},{"osd":2,"last update":"Fri Sep 13 18:22:18 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52500000000000002,"15min":0.52300000000000002},"min":{"1min":0.42199999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.54500000000000004,"15min":0.50800000000000001},"min":{"1min":0.42699999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.748,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.66700000000000004}]},{"osd":4,"last update":"Fri Sep 13 18:21:38 2024","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":0.622,"15min":0.52600000000000002},"min":{"1min":0.40500000000000003,"5min":0.312,"15min":0.312},"max":{"1min":9.3149999999999995,"5min":9.3149999999999995,"15min":9.3149999999999995},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.53900000000000003,"15min":0.51900000000000002},"min":{"1min":0.39900000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.72399999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.69499999999999995}]},{"osd":5,"last update":"Fri Sep 13 18:21:56 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.54800000000000004,"15min":0.53800000000000003},"min":{"1min":0.435,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.74199999999999999,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.54500000000000004,"15min":0.55000000000000004},"min":{"1min":0.38900000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.74199999999999999,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.60999999999999999}]},{"osd":6,"last update":"Fri Sep 13 18:22:28 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.434,"15min":0.44400000000000001},"min":{"1min":0.32800000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.749,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.749},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.48699999999999999,"15min":0.46899999999999997},"min":{"1min":0.307,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.76000000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.76000000000000001}]},{"osd":7,"last update":"Fri Sep 13 18:21:48 2024","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":0.69199999999999995,"15min":0.626},"min":{"1min":0.45700000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":9.3330000000000002,"5min":9.3330000000000002,"15min":9.3330000000000002},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57999999999999996,"15min":0.57399999999999995},"min":{"1min":0.39000000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.67700000000000005,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.70499999999999996}]},{"osd":8,"last update":"Fri Sep 13 18:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.69599999999999995}]},{"osd":9,"last update":"Fri Sep 13 18:21:31 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.52500000000000002,"15min":0.54900000000000004},"min":{"1min":0.33800000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.59999999999999998,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.57099999999999995,"15min":0.58499999999999996},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.73899999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.81000000000000005}]},{"osd":10,"last update":"Fri Sep 13 18:21:52 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.66700000000000004,"15min":0.67900000000000005},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.77900000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.63900000000000001,"15min":0.66200000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.69199999999999995,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.91200000000000003}]},{"osd":11,"last update":"Fri Sep 13 18:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.61099999999999999,"15min":0.60599999999999998},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64800000000000002,"15min":0.64900000000000002},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.76400000000000001,"5min":0.872,"15min":0.872},"last":0.85699999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1122304,"data_stored":596184,"data_compressed":28087,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:21:50 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.49299999999999999,"15min":0.48499999999999999},"min":{"1min":0.40600000000000003,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.503,"15min":0.48299999999999998},"min":{"1min":0.39600000000000002,"5min":0.28799999999999998,"15min":0.26300000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.45800000000000002}]},{"osd":1,"last update":"Fri Sep 13 18:21:50 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.54000000000000004,"15min":0.48799999999999999},"min":{"1min":0.46300000000000002,"5min":0.318,"15min":0.27400000000000002},"max":{"1min":0.81299999999999994,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.56899999999999995,"15min":0.49099999999999999},"min":{"1min":0.45000000000000001,"5min":0.32300000000000001,"15min":0.29099999999999998},"max":{"1min":0.79000000000000004,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.48899999999999999}]},{"osd":3,"last update":"Fri Sep 13 18:22:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55100000000000005,"15min":0.52400000000000002},"min":{"1min":0.41299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58199999999999996,"15min":0.56100000000000005},"min":{"1min":0.40799999999999997,"5min":0.34000000000000002,"15min":0.33300000000000002},"max":{"1min":0.79700000000000004,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.56799999999999995}]},{"osd":4,"last update":"Fri Sep 13 18:22:04 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.62,"15min":0.61699999999999999},"min":{"1min":0.495,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.75900000000000001,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.66900000000000004,"15min":0.66600000000000004},"min":{"1min":0.56899999999999995,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.82199999999999995,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.437}]},{"osd":5,"last update":"Fri Sep 13 18:21:53 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.42699999999999999,"15min":0.38900000000000001},"min":{"1min":0.27100000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.314},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.44600000000000001,"15min":0.40500000000000003},"min":{"1min":0.32100000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.67800000000000005,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.35299999999999998}]},{"osd":6,"last update":"Fri Sep 13 18:22:25 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.621,"15min":0.55900000000000005},"min":{"1min":0.52900000000000003,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.85099999999999998,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.626,"15min":0.57899999999999996},"min":{"1min":0.47799999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.82999999999999996,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.52400000000000002}]},{"osd":7,"last update":"Fri Sep 13 18:22:09 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.68899999999999995,"15min":0.68799999999999994},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.85899999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.67000000000000004,"15min":0.66800000000000004},"min":{"1min":0.495,"5min":0.434,"15min":0.434},"max":{"1min":0.86099999999999999,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.46700000000000003}]},{"osd":8,"last update":"Fri Sep 13 18:22:09 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.36299999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.79900000000000004,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.37},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.61599999999999999,"15min":0.59399999999999997},"min":{"1min":0.45800000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.86699999999999999,"5min":0.998,"15min":0.998},"last":0.28999999999999998}]},{"osd":9,"last update":"Fri Sep 13 18:21:28 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.63700000000000001,"15min":0.625},"min":{"1min":0.50900000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.86399999999999999,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.5},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.66900000000000004,"15min":0.65500000000000003},"min":{"1min":0.50900000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.96499999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.55000000000000004}]},{"osd":10,"last update":"Fri Sep 13 18:21:50 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.67600000000000005,"15min":0.63700000000000001},"min":{"1min":0.63900000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.93999999999999995,"5min":1.016,"15min":1.016},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.68899999999999995,"15min":0.65300000000000002},"min":{"1min":0.623,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.93600000000000005,"5min":1.042,"15min":1.042},"last":0.50900000000000001}]},{"osd":11,"last update":"Fri Sep 13 18:22:00 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.72499999999999998,"15min":0.71599999999999997},"min":{"1min":0.626,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.94699999999999995,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.73099999999999998,"15min":0.72299999999999998},"min":{"1min":0.54100000000000004,"5min":0.433,"15min":0.433},"max":{"1min":0.92000000000000004,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.54000000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1101824,"data_stored":580429,"data_compressed":25120,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:22:25 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.502,"15min":0.49299999999999999},"min":{"1min":0.376,"5min":0.30299999999999999,"15min":0.28399999999999997},"max":{"1min":0.67100000000000004,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55000000000000004,"15min":0.52100000000000002},"min":{"1min":0.40600000000000003,"5min":0.33200000000000002,"15min":0.31},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.40600000000000003}]},{"osd":2,"last update":"Fri Sep 13 18:21:53 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.56799999999999995,"15min":0.51200000000000001},"min":{"1min":0.39200000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.433},{"interface":"front","average":{"1min":0.622,"5min":0.52000000000000002,"15min":0.48599999999999999},"min":{"1min":0.33800000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.38700000000000001}]},{"osd":3,"last update":"Fri Sep 13 18:22:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55900000000000005,"15min":0.52700000000000002},"min":{"1min":0.374,"5min":0.31,"15min":0.31},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.503},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.60699999999999998,"15min":0.55800000000000005},"min":{"1min":0.47399999999999998,"5min":0.39700000000000002,"15min":0.34399999999999997},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.57099999999999995}]},{"osd":4,"last update":"Fri Sep 13 18:21:34 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.41099999999999998,"15min":0.371},"min":{"1min":0.29499999999999998,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.755,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.40500000000000003,"15min":0.36499999999999999},"min":{"1min":0.29599999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.65600000000000003,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.44700000000000001}]},{"osd":5,"last update":"Fri Sep 13 18:21:57 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.63400000000000001,"15min":0.58399999999999996},"min":{"1min":0.41899999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.60699999999999998,"15min":0.55500000000000005},"min":{"1min":0.47199999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85799999999999998,"5min":0.875,"15min":0.875},"last":0.41699999999999998}]},{"osd":6,"last update":"Fri Sep 13 18:22:25 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.59699999999999998,"15min":0.57399999999999995},"min":{"1min":0.46100000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.65400000000000003,"15min":0.622},"min":{"1min":0.54100000000000004,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.54800000000000004}]},{"osd":7,"last update":"Fri Sep 13 18:21:40 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.51700000000000002,"15min":0.495},"min":{"1min":0.312,"5min":0.246,"15min":0.246},"max":{"1min":0.74299999999999999,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.52200000000000002,"15min":0.50700000000000001},"min":{"1min":0.35499999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.82499999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.58699999999999997}]},{"osd":8,"last update":"Fri Sep 13 18:22:11 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.65400000000000003,"15min":0.626},"min":{"1min":0.48499999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.90300000000000002,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.747,"5min":0.66700000000000004,"15min":0.64200000000000002},"min":{"1min":0.57299999999999995,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.56000000000000005}]},{"osd":9,"last update":"Fri Sep 13 18:21:34 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.63600000000000001,"15min":0.64200000000000002},"min":{"1min":0.49399999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.84399999999999997,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.64000000000000001,"15min":0.64200000000000002},"min":{"1min":0.433,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.61599999999999999}]},{"osd":10,"last update":"Fri Sep 13 18:21:53 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.63100000000000001,"15min":0.60299999999999998},"min":{"1min":0.442,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.63300000000000001,"15min":0.61499999999999999},"min":{"1min":0.39800000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.52400000000000002}]},{"osd":11,"last update":"Fri Sep 13 18:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.80200000000000005,"15min":0.79700000000000004},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.623},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.57999999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":1052672,"data_stored":516690,"data_compressed":23512,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 18:21:28 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.50700000000000001,"15min":0.498},"min":{"1min":0.36799999999999999,"5min":0.318,"15min":0.30199999999999999},"max":{"1min":0.76900000000000002,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.497},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.52100000000000002,"15min":0.50900000000000001},"min":{"1min":0.434,"5min":0.307,"15min":0.307},"max":{"1min":0.73999999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.33000000000000002}]},{"osd":2,"last update":"Fri Sep 13 18:21:49 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.53400000000000003,"15min":0.52900000000000003},"min":{"1min":0.40699999999999997,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.70699999999999996,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.623},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.55700000000000005,"15min":0.52000000000000002},"min":{"1min":0.42899999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.69699999999999995,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.437}]},{"osd":3,"last update":"Fri Sep 13 18:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.40999999999999998,"15min":0.39200000000000002},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":2.9750000000000001,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.38200000000000001,"15min":0.38500000000000001},"min":{"1min":0.26500000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.61299999999999999,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.41599999999999998}]},{"osd":4,"last update":"Fri Sep 13 18:21:39 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52100000000000002,"15min":0.497},"min":{"1min":0.41299999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.54100000000000004,"15min":0.51500000000000001},"min":{"1min":0.36399999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76000000000000001,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.378}]},{"osd":5,"last update":"Fri Sep 13 18:21:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.61399999999999999,"15min":0.56599999999999995},"min":{"1min":0.35199999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.68300000000000005,"5min":7.194,"15min":7.194},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.53300000000000003},"min":{"1min":0.44500000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.84999999999999998,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.54300000000000004}]},{"osd":6,"last update":"Fri Sep 13 18:22:24 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.47699999999999998,"15min":0.45900000000000002},"min":{"1min":0.33000000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.437,"15min":0.45400000000000001},"min":{"1min":0.316,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.73899999999999999,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.40100000000000002}]},{"osd":7,"last update":"Fri Sep 13 18:22:09 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.59699999999999998,"15min":0.59399999999999997},"min":{"1min":0.44900000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.68799999999999994}]},{"osd":8,"last update":"Fri Sep 13 18:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58999999999999997,"15min":0.60399999999999998},"min":{"1min":0.41899999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.61799999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.65700000000000003}]},{"osd":9,"last update":"Fri Sep 13 18:21:33 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.58499999999999996,"15min":0.60699999999999998},"min":{"1min":0.33700000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78200000000000003,"5min":1.089,"15min":1.089},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.54100000000000004,"15min":0.57299999999999995},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.61799999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.61599999999999999}]},{"osd":10,"last update":"Fri Sep 13 18:21:49 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.65900000000000003,"15min":0.67700000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88800000000000001,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.749},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.68300000000000005,"15min":0.69799999999999995},"min":{"1min":0.51900000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.84599999999999997,"5min":1.048,"15min":1.048},"last":0.63800000000000001}]},{"osd":11,"last update":"Fri Sep 13 18:22:20 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.623,"15min":0.63100000000000001},"min":{"1min":0.44600000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68100000000000005,"15min":0.68300000000000005},"min":{"1min":0.45900000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.71999999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35824,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35824,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35824,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":72245,"data_compressed":1608,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":72295,"data_compressed":1608,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135258,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":6440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":394694,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":203717,"data_compressed":7764,"data_compressed_allocated":12288,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:22:32.898 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:22:33.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.022+0000 7f37e6d25640 1 -- 172.21.15.114:0/626963275 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37e00ff840 msgr2=0x7f37e0103880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:33.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.022+0000 7f37e6d25640 1 --2- 172.21.15.114:0/626963275 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37e00ff840 0x7f37e0103880 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f37cc009930 tx=0x7f37cc02a420 comp rx=0 tx=0).stop 2024-09-13T18:22:33.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.022+0000 7f37e6d25640 1 -- 172.21.15.114:0/626963275 shutdown_connections 2024-09-13T18:22:33.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.022+0000 7f37e6d25640 1 --2- 172.21.15.114:0/626963275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37e0103fe0 0x7f37e010bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:33.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.022+0000 7f37e6d25640 1 --2- 172.21.15.114:0/626963275 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37e00ff840 0x7f37e0103880 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:33.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.022+0000 7f37e6d25640 1 --2- 172.21.15.114:0/626963275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37e00fee90 0x7f37e00ff270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:33.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.022+0000 7f37e6d25640 1 -- 172.21.15.114:0/626963275 >> 172.21.15.114:0/626963275 conn(0x7f37e00fa900 msgr2=0x7f37e00fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:33.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.022+0000 7f37e6d25640 1 -- 172.21.15.114:0/626963275 shutdown_connections 2024-09-13T18:22:33.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.022+0000 7f37e6d25640 1 -- 172.21.15.114:0/626963275 wait complete. 2024-09-13T18:22:33.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.022+0000 7f37e6d25640 1 Processor -- start 2024-09-13T18:22:33.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.022+0000 7f37e6d25640 1 -- start start 2024-09-13T18:22:33.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.023+0000 7f37e6d25640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37e00fee90 0x7f37e0196240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:33.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.023+0000 7f37e6d25640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37e0103fe0 0x7f37e0196780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:33.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.023+0000 7f37e6d25640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37e019ab70 0x7f37e019b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:33.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.023+0000 7f37e6d25640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e019b560 con 0x7f37e019ab70 2024-09-13T18:22:33.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.023+0000 7f37e6d25640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e019b6d0 con 0x7f37e0103fe0 2024-09-13T18:22:33.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.023+0000 7f37e6d25640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e019b840 con 0x7f37e00fee90 2024-09-13T18:22:33.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.023+0000 7f37e52c2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37e019ab70 0x7f37e019b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:33.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.023+0000 7f37e4ac1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37e00fee90 0x7f37e0196240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:33.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.023+0000 7f37e52c2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37e019ab70 0x7f37e019b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33456/0 (socket says 172.21.15.114:33456) 2024-09-13T18:22:33.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.023+0000 7f37e52c2640 1 -- 172.21.15.114:0/4031248337 learned_addr learned my addr 172.21.15.114:0/4031248337 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:33.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.023+0000 7f37dffff640 1 --2- 172.21.15.114:0/4031248337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37e0103fe0 0x7f37e0196780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:33.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.023+0000 7f37e52c2640 1 -- 172.21.15.114:0/4031248337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37e00fee90 msgr2=0x7f37e0196240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:33.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.023+0000 7f37e52c2640 1 --2- 172.21.15.114:0/4031248337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37e00fee90 0x7f37e0196240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:33.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.023+0000 7f37e52c2640 1 -- 172.21.15.114:0/4031248337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37e0103fe0 msgr2=0x7f37e0196780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:33.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.023+0000 7f37e52c2640 1 --2- 172.21.15.114:0/4031248337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37e0103fe0 0x7f37e0196780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:33.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.023+0000 7f37e52c2640 1 -- 172.21.15.114:0/4031248337 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37cc009560 con 0x7f37e019ab70 2024-09-13T18:22:33.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.023+0000 7f37e4ac1640 1 --2- 172.21.15.114:0/4031248337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37e00fee90 0x7f37e0196240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:22:33.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.023+0000 7f37e52c2640 1 --2- 172.21.15.114:0/4031248337 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37e019ab70 0x7f37e019b020 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f37d800eb10 tx=0x7f37d800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:33.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.023+0000 7f37ddffb640 1 -- 172.21.15.114:0/4031248337 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d8004b20 con 0x7f37e019ab70 2024-09-13T18:22:33.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.023+0000 7f37ddffb640 1 -- 172.21.15.114:0/4031248337 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f37d8004c80 con 0x7f37e019ab70 2024-09-13T18:22:33.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.023+0000 7f37e6d25640 1 -- 172.21.15.114:0/4031248337 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37e01a2e20 con 0x7f37e019ab70 2024-09-13T18:22:33.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.024+0000 7f37ddffb640 1 -- 172.21.15.114:0/4031248337 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d8005a10 con 0x7f37e019ab70 2024-09-13T18:22:33.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.024+0000 7f37e6d25640 1 -- 172.21.15.114:0/4031248337 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37e01a32d0 con 0x7f37e019ab70 2024-09-13T18:22:33.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.025+0000 7f37e6d25640 1 -- 172.21.15.114:0/4031248337 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37e0100360 con 0x7f37e019ab70 2024-09-13T18:22:33.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.026+0000 7f37ddffb640 1 -- 172.21.15.114:0/4031248337 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 24) v1 ==== 136050+0+0 (secure 0 0 0) 0x7f37d8005020 con 0x7f37e019ab70 2024-09-13T18:22:33.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.026+0000 7f37ddffb640 1 --2- 172.21.15.114:0/4031248337 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f37d00a1310 0x7f37d00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:33.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.027+0000 7f37ddffb640 1 -- 172.21.15.114:0/4031248337 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f37d8014070 con 0x7f37e019ab70 2024-09-13T18:22:33.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.027+0000 7f37e4ac1640 1 --2- 172.21.15.114:0/4031248337 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f37d00a1310 0x7f37d00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:33.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.027+0000 7f37e4ac1640 1 --2- 172.21.15.114:0/4031248337 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f37d00a1310 0x7f37d00a37d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f37d4005d00 tx=0x7f37d400b410 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:33.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.030+0000 7f37ddffb640 1 -- 172.21.15.114:0/4031248337 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f37d80965e0 con 0x7f37e019ab70 2024-09-13T18:22:33.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.191+0000 7f37e6d25640 1 -- 172.21.15.114:0/4031248337 --> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f37e0197590 con 0x7f37d00a1310 2024-09-13T18:22:33.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.202+0000 7f37ddffb640 1 -- 172.21.15.114:0/4031248337 <== mgr.15555 v2:172.21.15.114:6800/2440721366 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427785 (secure 0 0 0) 0x7f37e0197590 con 0x7f37d00a1310 2024-09-13T18:22:33.210 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:22:33.213 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-13T18:22:33.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.206+0000 7f37e6d25640 1 -- 172.21.15.114:0/4031248337 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f37d00a1310 msgr2=0x7f37d00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:33.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.206+0000 7f37e6d25640 1 --2- 172.21.15.114:0/4031248337 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f37d00a1310 0x7f37d00a37d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f37d4005d00 tx=0x7f37d400b410 comp rx=0 tx=0).stop 2024-09-13T18:22:33.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.206+0000 7f37e6d25640 1 -- 172.21.15.114:0/4031248337 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37e019ab70 msgr2=0x7f37e019b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:33.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.206+0000 7f37e6d25640 1 --2- 172.21.15.114:0/4031248337 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37e019ab70 0x7f37e019b020 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f37d800eb10 tx=0x7f37d800c2d0 comp rx=0 tx=0).stop 2024-09-13T18:22:33.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.206+0000 7f37e6d25640 1 -- 172.21.15.114:0/4031248337 shutdown_connections 2024-09-13T18:22:33.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.206+0000 7f37e6d25640 1 --2- 172.21.15.114:0/4031248337 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f37d00a1310 0x7f37d00a37d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:33.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.206+0000 7f37e6d25640 1 --2- 172.21.15.114:0/4031248337 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37e019ab70 0x7f37e019b020 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:33.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.206+0000 7f37e6d25640 1 --2- 172.21.15.114:0/4031248337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37e0103fe0 0x7f37e0196780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:33.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.206+0000 7f37e6d25640 1 --2- 172.21.15.114:0/4031248337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37e00fee90 0x7f37e0196240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:33.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.206+0000 7f37e6d25640 1 -- 172.21.15.114:0/4031248337 >> 172.21.15.114:0/4031248337 conn(0x7f37e00fa900 msgr2=0x7f37e00fbd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:33.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.206+0000 7f37e6d25640 1 -- 172.21.15.114:0/4031248337 shutdown_connections 2024-09-13T18:22:33.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.206+0000 7f37e6d25640 1 -- 172.21.15.114:0/4031248337 wait complete. 2024-09-13T18:22:33.219 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:33 smithi114 ceph-mon[30897]: Updating smithi114:/etc/ceph/ceph.client.admin.keyring 2024-09-13T18:22:33.219 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:33 smithi114 ceph-mon[30897]: Updating smithi134:/etc/ceph/ceph.client.admin.keyring 2024-09-13T18:22:33.219 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:33 smithi114 ceph-mon[30897]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-09-13T18:22:33.219 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:33 smithi114 ceph-mon[30897]: Updating smithi114:/var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-13T18:22:33.219 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:33 smithi114 ceph-mon[30897]: Updating smithi134:/var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-13T18:22:33.219 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:33 smithi114 ceph-mon[30897]: Updating smithi204:/var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-13T18:22:33.219 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:33 smithi114 ceph-mon[30897]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:22:33.219 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:33 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2617702121' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T18:22:33.219 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:33 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/1850740749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:22:33.219 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:33 smithi114 ceph-mon[30897]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-09-13T18:22:33.220 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-09-13T18:22:32.078907+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1194735,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1483,"num_read_kb":912,"num_write":548,"num_write_kb":4407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180875,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":938,"ondisk_log_size":938,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361600,"kb_used_data":12792,"kb_used_omap":18,"kb_used_meta":348653,"kb_avail":1124580224,"statfs":{"total":1151940427776,"available":1151570149376,"internally_reserved":0,"allocated":13099008,"data_stored":9351983,"data_compressed":474714,"data_compressed_allocated":647168,"data_compressed_original":3694592,"omap_allocated":19129,"internal_metadata":357020999},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.988041"},"pg_stats":[{"pgid":"2.1f","version":"93'14","reported_seq":157,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.139188+0000","last_change":"2024-09-13T18:20:01.577029+0000","last_active":"2024-09-13T18:22:28.139188+0000","last_peered":"2024-09-13T18:22:28.139188+0000","last_clean":"2024-09-13T18:22:28.139188+0000","last_became_active":"2024-09-13T18:20:01.576710+0000","last_became_peered":"2024-09-13T18:20:01.576710+0000","last_unstale":"2024-09-13T18:22:28.139188+0000","last_undegraded":"2024-09-13T18:22:28.139188+0000","last_fullsized":"2024-09-13T18:22:28.139188+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:25:11.977436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"93'5","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:03.100355+0000","last_change":"2024-09-13T18:20:01.922955+0000","last_active":"2024-09-13T18:22:03.100355+0000","last_peered":"2024-09-13T18:22:03.100355+0000","last_clean":"2024-09-13T18:22:03.100355+0000","last_became_active":"2024-09-13T18:20:01.922704+0000","last_became_peered":"2024-09-13T18:20:01.922704+0000","last_unstale":"2024-09-13T18:22:03.100355+0000","last_undegraded":"2024-09-13T18:22:03.100355+0000","last_fullsized":"2024-09-13T18:22:03.100355+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:44:20.672784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:28.310843+0000","last_change":"2024-09-13T18:20:27.300985+0000","last_active":"2024-09-13T18:21:28.310843+0000","last_peered":"2024-09-13T18:21:28.310843+0000","last_clean":"2024-09-13T18:21:28.310843+0000","last_became_active":"2024-09-13T18:20:27.300546+0000","last_became_peered":"2024-09-13T18:20:27.300546+0000","last_unstale":"2024-09-13T18:21:28.310843+0000","last_undegraded":"2024-09-13T18:21:28.310843+0000","last_fullsized":"2024-09-13T18:21:28.310843+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:00:11.750929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:03.100379+0000","last_change":"2024-09-13T18:20:01.578921+0000","last_active":"2024-09-13T18:22:03.100379+0000","last_peered":"2024-09-13T18:22:03.100379+0000","last_clean":"2024-09-13T18:22:03.100379+0000","last_became_active":"2024-09-13T18:20:01.578627+0000","last_became_peered":"2024-09-13T18:20:01.578627+0000","last_unstale":"2024-09-13T18:22:03.100379+0000","last_undegraded":"2024-09-13T18:22:03.100379+0000","last_fullsized":"2024-09-13T18:22:03.100379+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:43:32.919156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"93'6","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:07.284848+0000","last_change":"2024-09-13T18:20:01.922901+0000","last_active":"2024-09-13T18:22:07.284848+0000","last_peered":"2024-09-13T18:22:07.284848+0000","last_clean":"2024-09-13T18:22:07.284848+0000","last_became_active":"2024-09-13T18:20:01.922604+0000","last_became_peered":"2024-09-13T18:20:01.922604+0000","last_unstale":"2024-09-13T18:22:07.284848+0000","last_undegraded":"2024-09-13T18:22:07.284848+0000","last_fullsized":"2024-09-13T18:22:07.284848+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:27:30.650010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:28.151945+0000","last_change":"2024-09-13T18:20:06.135406+0000","last_active":"2024-09-13T18:21:28.151945+0000","last_peered":"2024-09-13T18:21:28.151945+0000","last_clean":"2024-09-13T18:21:28.151945+0000","last_became_active":"2024-09-13T18:20:06.134558+0000","last_became_peered":"2024-09-13T18:20:06.134558+0000","last_unstale":"2024-09-13T18:21:28.151945+0000","last_undegraded":"2024-09-13T18:21:28.151945+0000","last_fullsized":"2024-09-13T18:21:28.151945+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:10.901966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'12","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:28.153481+0000","last_change":"2024-09-13T18:20:01.580040+0000","last_active":"2024-09-13T18:21:28.153481+0000","last_peered":"2024-09-13T18:21:28.153481+0000","last_clean":"2024-09-13T18:21:28.153481+0000","last_became_active":"2024-09-13T18:20:01.579715+0000","last_became_peered":"2024-09-13T18:20:01.579715+0000","last_unstale":"2024-09-13T18:21:28.153481+0000","last_undegraded":"2024-09-13T18:21:28.153481+0000","last_fullsized":"2024-09-13T18:21:28.153481+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:57:03.588169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"93'9","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:55.925782+0000","last_change":"2024-09-13T18:20:01.928314+0000","last_active":"2024-09-13T18:21:55.925782+0000","last_peered":"2024-09-13T18:21:55.925782+0000","last_clean":"2024-09-13T18:21:55.925782+0000","last_became_active":"2024-09-13T18:20:01.925356+0000","last_became_peered":"2024-09-13T18:20:01.925356+0000","last_unstale":"2024-09-13T18:21:55.925782+0000","last_undegraded":"2024-09-13T18:21:55.925782+0000","last_fullsized":"2024-09-13T18:21:55.925782+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:43.906842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:28.152198+0000","last_change":"2024-09-13T18:20:06.135886+0000","last_active":"2024-09-13T18:21:28.152198+0000","last_peered":"2024-09-13T18:21:28.152198+0000","last_clean":"2024-09-13T18:21:28.152198+0000","last_became_active":"2024-09-13T18:20:06.134587+0000","last_became_peered":"2024-09-13T18:20:06.134587+0000","last_unstale":"2024-09-13T18:21:28.152198+0000","last_undegraded":"2024-09-13T18:21:28.152198+0000","last_fullsized":"2024-09-13T18:21:28.152198+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:44:42.434729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"91'5","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.140618+0000","last_change":"2024-09-13T18:20:01.576908+0000","last_active":"2024-09-13T18:22:28.140618+0000","last_peered":"2024-09-13T18:22:28.140618+0000","last_clean":"2024-09-13T18:22:28.140618+0000","last_became_active":"2024-09-13T18:20:01.576478+0000","last_became_peered":"2024-09-13T18:20:01.576478+0000","last_unstale":"2024-09-13T18:22:28.140618+0000","last_undegraded":"2024-09-13T18:22:28.140618+0000","last_fullsized":"2024-09-13T18:22:28.140618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:34:52.126901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"93'15","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:55.923587+0000","last_change":"2024-09-13T18:20:01.921960+0000","last_active":"2024-09-13T18:21:55.923587+0000","last_peered":"2024-09-13T18:21:55.923587+0000","last_clean":"2024-09-13T18:21:55.923587+0000","last_became_active":"2024-09-13T18:20:01.921652+0000","last_became_peered":"2024-09-13T18:20:01.921652+0000","last_unstale":"2024-09-13T18:21:55.923587+0000","last_undegraded":"2024-09-13T18:21:55.923587+0000","last_fullsized":"2024-09-13T18:21:55.923587+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:00:31.277613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:28.152004+0000","last_change":"2024-09-13T18:20:06.138983+0000","last_active":"2024-09-13T18:21:28.152004+0000","last_peered":"2024-09-13T18:21:28.152004+0000","last_clean":"2024-09-13T18:21:28.152004+0000","last_became_active":"2024-09-13T18:20:06.138708+0000","last_became_peered":"2024-09-13T18:20:06.138708+0000","last_unstale":"2024-09-13T18:21:28.152004+0000","last_undegraded":"2024-09-13T18:21:28.152004+0000","last_fullsized":"2024-09-13T18:21:28.152004+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:45:21.978217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"93'5","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:16.192564+0000","last_change":"2024-09-13T18:20:01.929817+0000","last_active":"2024-09-13T18:22:16.192564+0000","last_peered":"2024-09-13T18:22:16.192564+0000","last_clean":"2024-09-13T18:22:16.192564+0000","last_became_active":"2024-09-13T18:20:01.928907+0000","last_became_peered":"2024-09-13T18:20:01.928907+0000","last_unstale":"2024-09-13T18:22:16.192564+0000","last_undegraded":"2024-09-13T18:22:16.192564+0000","last_fullsized":"2024-09-13T18:22:16.192564+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:56:25.847230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:15.988721+0000","last_change":"2024-09-13T18:20:01.578968+0000","last_active":"2024-09-13T18:22:15.988721+0000","last_peered":"2024-09-13T18:22:15.988721+0000","last_clean":"2024-09-13T18:22:15.988721+0000","last_became_active":"2024-09-13T18:20:01.578623+0000","last_became_peered":"2024-09-13T18:20:01.578623+0000","last_unstale":"2024-09-13T18:22:15.988721+0000","last_undegraded":"2024-09-13T18:22:15.988721+0000","last_fullsized":"2024-09-13T18:22:15.988721+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:03:30.333107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:21.739811+0000","last_change":"2024-09-13T18:20:06.135977+0000","last_active":"2024-09-13T18:22:21.739811+0000","last_peered":"2024-09-13T18:22:21.739811+0000","last_clean":"2024-09-13T18:22:21.739811+0000","last_became_active":"2024-09-13T18:20:06.134844+0000","last_became_peered":"2024-09-13T18:20:06.134844+0000","last_unstale":"2024-09-13T18:22:21.739811+0000","last_undegraded":"2024-09-13T18:22:21.739811+0000","last_fullsized":"2024-09-13T18:22:21.739811+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:58:36.139240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"93'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:18.101943+0000","last_change":"2024-09-13T18:20:01.916675+0000","last_active":"2024-09-13T18:22:18.101943+0000","last_peered":"2024-09-13T18:22:18.101943+0000","last_clean":"2024-09-13T18:22:18.101943+0000","last_became_active":"2024-09-13T18:20:01.912902+0000","last_became_peered":"2024-09-13T18:20:01.912902+0000","last_unstale":"2024-09-13T18:22:18.101943+0000","last_undegraded":"2024-09-13T18:22:18.101943+0000","last_fullsized":"2024-09-13T18:22:18.101943+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:24:54.271569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"91'3","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:28.154557+0000","last_change":"2024-09-13T18:20:27.300845+0000","last_active":"2024-09-13T18:21:28.154557+0000","last_peered":"2024-09-13T18:21:28.154557+0000","last_clean":"2024-09-13T18:21:28.154557+0000","last_became_active":"2024-09-13T18:20:27.300417+0000","last_became_peered":"2024-09-13T18:20:27.300417+0000","last_unstale":"2024-09-13T18:21:28.154557+0000","last_undegraded":"2024-09-13T18:21:28.154557+0000","last_fullsized":"2024-09-13T18:21:28.154557+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:44:27.507140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:21.739768+0000","last_change":"2024-09-13T18:21:28.235707+0000","last_active":"2024-09-13T18:22:21.739768+0000","last_peered":"2024-09-13T18:22:21.739768+0000","last_clean":"2024-09-13T18:22:21.739768+0000","last_became_active":"2024-09-13T18:21:28.231794+0000","last_became_peered":"2024-09-13T18:21:28.231794+0000","last_unstale":"2024-09-13T18:22:21.739768+0000","last_undegraded":"2024-09-13T18:22:21.739768+0000","last_fullsized":"2024-09-13T18:22:21.739768+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:01:00.113821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"93'9","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:12.959281+0000","last_change":"2024-09-13T18:20:01.913634+0000","last_active":"2024-09-13T18:22:12.959281+0000","last_peered":"2024-09-13T18:22:12.959281+0000","last_clean":"2024-09-13T18:22:12.959281+0000","last_became_active":"2024-09-13T18:20:01.913128+0000","last_became_peered":"2024-09-13T18:20:01.913128+0000","last_unstale":"2024-09-13T18:22:12.959281+0000","last_undegraded":"2024-09-13T18:22:12.959281+0000","last_fullsized":"2024-09-13T18:22:12.959281+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:37:15.251756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"91'7","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:13.101465+0000","last_change":"2024-09-13T18:20:01.579385+0000","last_active":"2024-09-13T18:22:13.101465+0000","last_peered":"2024-09-13T18:22:13.101465+0000","last_clean":"2024-09-13T18:22:13.101465+0000","last_became_active":"2024-09-13T18:20:01.579102+0000","last_became_peered":"2024-09-13T18:20:01.579102+0000","last_unstale":"2024-09-13T18:22:13.101465+0000","last_undegraded":"2024-09-13T18:22:13.101465+0000","last_fullsized":"2024-09-13T18:22:13.101465+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:22:00.432052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.138888+0000","last_change":"2024-09-13T18:20:06.138110+0000","last_active":"2024-09-13T18:22:28.138888+0000","last_peered":"2024-09-13T18:22:28.138888+0000","last_clean":"2024-09-13T18:22:28.138888+0000","last_became_active":"2024-09-13T18:20:06.137763+0000","last_became_peered":"2024-09-13T18:20:06.137763+0000","last_unstale":"2024-09-13T18:22:28.138888+0000","last_undegraded":"2024-09-13T18:22:28.138888+0000","last_fullsized":"2024-09-13T18:22:28.138888+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:16:15.956216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"93'2","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:15.988679+0000","last_change":"2024-09-13T18:20:01.918318+0000","last_active":"2024-09-13T18:22:15.988679+0000","last_peered":"2024-09-13T18:22:15.988679+0000","last_clean":"2024-09-13T18:22:15.988679+0000","last_became_active":"2024-09-13T18:20:01.917977+0000","last_became_peered":"2024-09-13T18:20:01.917977+0000","last_unstale":"2024-09-13T18:22:15.988679+0000","last_undegraded":"2024-09-13T18:22:15.988679+0000","last_fullsized":"2024-09-13T18:22:15.988679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:23.368866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:13.276327+0000","last_change":"2024-09-13T18:20:01.572009+0000","last_active":"2024-09-13T18:22:13.276327+0000","last_peered":"2024-09-13T18:22:13.276327+0000","last_clean":"2024-09-13T18:22:13.276327+0000","last_became_active":"2024-09-13T18:20:01.571313+0000","last_became_peered":"2024-09-13T18:20:01.571313+0000","last_unstale":"2024-09-13T18:22:13.276327+0000","last_undegraded":"2024-09-13T18:22:13.276327+0000","last_fullsized":"2024-09-13T18:22:13.276327+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:32:59.340476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:21.739824+0000","last_change":"2024-09-13T18:20:06.132212+0000","last_active":"2024-09-13T18:22:21.739824+0000","last_peered":"2024-09-13T18:22:21.739824+0000","last_clean":"2024-09-13T18:22:21.739824+0000","last_became_active":"2024-09-13T18:20:06.131809+0000","last_became_peered":"2024-09-13T18:20:06.131809+0000","last_unstale":"2024-09-13T18:22:21.739824+0000","last_undegraded":"2024-09-13T18:22:21.739824+0000","last_fullsized":"2024-09-13T18:22:21.739824+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:18:11.667551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"93'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:12.285278+0000","last_change":"2024-09-13T18:20:27.301112+0000","last_active":"2024-09-13T18:22:12.285278+0000","last_peered":"2024-09-13T18:22:12.285278+0000","last_clean":"2024-09-13T18:22:12.285278+0000","last_became_active":"2024-09-13T18:20:27.300772+0000","last_became_peered":"2024-09-13T18:20:27.300772+0000","last_unstale":"2024-09-13T18:22:12.285278+0000","last_undegraded":"2024-09-13T18:22:12.285278+0000","last_fullsized":"2024-09-13T18:22:12.285278+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:59:57.606105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"91'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:11.738288+0000","last_change":"2024-09-13T18:20:01.572025+0000","last_active":"2024-09-13T18:22:11.738288+0000","last_peered":"2024-09-13T18:22:11.738288+0000","last_clean":"2024-09-13T18:22:11.738288+0000","last_became_active":"2024-09-13T18:20:01.571321+0000","last_became_peered":"2024-09-13T18:20:01.571321+0000","last_unstale":"2024-09-13T18:22:11.738288+0000","last_undegraded":"2024-09-13T18:22:11.738288+0000","last_fullsized":"2024-09-13T18:22:11.738288+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:09.374364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:10.788550+0000","last_change":"2024-09-13T18:20:06.130687+0000","last_active":"2024-09-13T18:22:10.788550+0000","last_peered":"2024-09-13T18:22:10.788550+0000","last_clean":"2024-09-13T18:22:10.788550+0000","last_became_active":"2024-09-13T18:20:06.129839+0000","last_became_peered":"2024-09-13T18:20:06.129839+0000","last_unstale":"2024-09-13T18:22:10.788550+0000","last_undegraded":"2024-09-13T18:22:10.788550+0000","last_fullsized":"2024-09-13T18:22:10.788550+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:12:52.334945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"93'5","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:13.101520+0000","last_change":"2024-09-13T18:20:01.917244+0000","last_active":"2024-09-13T18:22:13.101520+0000","last_peered":"2024-09-13T18:22:13.101520+0000","last_clean":"2024-09-13T18:22:13.101520+0000","last_became_active":"2024-09-13T18:20:01.916948+0000","last_became_peered":"2024-09-13T18:20:01.916948+0000","last_unstale":"2024-09-13T18:22:13.101520+0000","last_undegraded":"2024-09-13T18:22:13.101520+0000","last_fullsized":"2024-09-13T18:22:13.101520+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:17:11.967096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'44","reported_seq":89,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.252535+0000","last_change":"2024-09-13T18:20:01.569123+0000","last_active":"2024-09-13T18:22:28.252535+0000","last_peered":"2024-09-13T18:22:28.252535+0000","last_clean":"2024-09-13T18:22:28.252535+0000","last_became_active":"2024-09-13T18:20:01.568720+0000","last_became_peered":"2024-09-13T18:20:01.568720+0000","last_unstale":"2024-09-13T18:22:28.252535+0000","last_undegraded":"2024-09-13T18:22:28.252535+0000","last_fullsized":"2024-09-13T18:22:28.252535+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:04:29.561013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":76,"num_write_kb":833,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:08.100974+0000","last_change":"2024-09-13T18:20:06.141771+0000","last_active":"2024-09-13T18:22:08.100974+0000","last_peered":"2024-09-13T18:22:08.100974+0000","last_clean":"2024-09-13T18:22:08.100974+0000","last_became_active":"2024-09-13T18:20:06.141362+0000","last_became_peered":"2024-09-13T18:20:06.141362+0000","last_unstale":"2024-09-13T18:22:08.100974+0000","last_undegraded":"2024-09-13T18:22:08.100974+0000","last_fullsized":"2024-09-13T18:22:08.100974+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:09:58.597605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"93'9","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:08.482216+0000","last_change":"2024-09-13T18:20:01.911048+0000","last_active":"2024-09-13T18:22:08.482216+0000","last_peered":"2024-09-13T18:22:08.482216+0000","last_clean":"2024-09-13T18:22:08.482216+0000","last_became_active":"2024-09-13T18:20:01.910735+0000","last_became_peered":"2024-09-13T18:20:01.910735+0000","last_unstale":"2024-09-13T18:22:08.482216+0000","last_undegraded":"2024-09-13T18:22:08.482216+0000","last_fullsized":"2024-09-13T18:22:08.482216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:07:35.447676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"91'4","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:05.987451+0000","last_change":"2024-09-13T18:20:01.571627+0000","last_active":"2024-09-13T18:22:05.987451+0000","last_peered":"2024-09-13T18:22:05.987451+0000","last_clean":"2024-09-13T18:22:05.987451+0000","last_became_active":"2024-09-13T18:20:01.571250+0000","last_became_peered":"2024-09-13T18:20:01.571250+0000","last_unstale":"2024-09-13T18:22:05.987451+0000","last_undegraded":"2024-09-13T18:22:05.987451+0000","last_fullsized":"2024-09-13T18:22:05.987451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:34:03.479324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:12.959308+0000","last_change":"2024-09-13T18:20:06.132974+0000","last_active":"2024-09-13T18:22:12.959308+0000","last_peered":"2024-09-13T18:22:12.959308+0000","last_clean":"2024-09-13T18:22:12.959308+0000","last_became_active":"2024-09-13T18:20:06.131668+0000","last_became_peered":"2024-09-13T18:20:06.131668+0000","last_unstale":"2024-09-13T18:22:12.959308+0000","last_undegraded":"2024-09-13T18:22:12.959308+0000","last_fullsized":"2024-09-13T18:22:12.959308+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:36:30.329761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"93'4","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:08.478967+0000","last_change":"2024-09-13T18:20:01.904670+0000","last_active":"2024-09-13T18:22:08.478967+0000","last_peered":"2024-09-13T18:22:08.478967+0000","last_clean":"2024-09-13T18:22:08.478967+0000","last_became_active":"2024-09-13T18:20:01.904316+0000","last_became_peered":"2024-09-13T18:20:01.904316+0000","last_unstale":"2024-09-13T18:22:08.478967+0000","last_undegraded":"2024-09-13T18:22:08.478967+0000","last_fullsized":"2024-09-13T18:22:08.478967+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:08:17.687179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"91'1","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:09.591727+0000","last_change":"2024-09-13T18:20:01.564750+0000","last_active":"2024-09-13T18:22:09.591727+0000","last_peered":"2024-09-13T18:22:09.591727+0000","last_clean":"2024-09-13T18:22:09.591727+0000","last_became_active":"2024-09-13T18:20:01.564096+0000","last_became_peered":"2024-09-13T18:20:01.564096+0000","last_unstale":"2024-09-13T18:22:09.591727+0000","last_undegraded":"2024-09-13T18:22:09.591727+0000","last_fullsized":"2024-09-13T18:22:09.591727+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:42:19.588824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:10.988064+0000","last_change":"2024-09-13T18:20:06.133386+0000","last_active":"2024-09-13T18:22:10.988064+0000","last_peered":"2024-09-13T18:22:10.988064+0000","last_clean":"2024-09-13T18:22:10.988064+0000","last_became_active":"2024-09-13T18:20:06.133004+0000","last_became_peered":"2024-09-13T18:20:06.133004+0000","last_unstale":"2024-09-13T18:22:10.988064+0000","last_undegraded":"2024-09-13T18:22:10.988064+0000","last_fullsized":"2024-09-13T18:22:10.988064+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:54:46.440410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"93'7","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:08.478947+0000","last_change":"2024-09-13T18:20:01.912823+0000","last_active":"2024-09-13T18:22:08.478947+0000","last_peered":"2024-09-13T18:22:08.478947+0000","last_clean":"2024-09-13T18:22:08.478947+0000","last_became_active":"2024-09-13T18:20:01.912123+0000","last_became_peered":"2024-09-13T18:20:01.912123+0000","last_unstale":"2024-09-13T18:22:08.478947+0000","last_undegraded":"2024-09-13T18:22:08.478947+0000","last_fullsized":"2024-09-13T18:22:08.478947+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:32:33.544816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"94'8","reported_seq":153,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.424067+0000","last_change":"2024-09-13T18:20:01.577095+0000","last_active":"2024-09-13T18:22:28.424067+0000","last_peered":"2024-09-13T18:22:28.424067+0000","last_clean":"2024-09-13T18:22:28.424067+0000","last_became_active":"2024-09-13T18:20:01.576765+0000","last_became_peered":"2024-09-13T18:20:01.576765+0000","last_unstale":"2024-09-13T18:22:28.424067+0000","last_undegraded":"2024-09-13T18:22:28.424067+0000","last_fullsized":"2024-09-13T18:22:28.424067+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:44:08.750441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1818,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":123,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:13.482840+0000","last_change":"2024-09-13T18:20:06.131650+0000","last_active":"2024-09-13T18:22:13.482840+0000","last_peered":"2024-09-13T18:22:13.482840+0000","last_clean":"2024-09-13T18:22:13.482840+0000","last_became_active":"2024-09-13T18:20:06.130902+0000","last_became_peered":"2024-09-13T18:20:06.130902+0000","last_unstale":"2024-09-13T18:22:13.482840+0000","last_undegraded":"2024-09-13T18:22:13.482840+0000","last_fullsized":"2024-09-13T18:22:13.482840+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.366694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"93'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:06.033866+0000","last_change":"2024-09-13T18:20:01.918221+0000","last_active":"2024-09-13T18:22:06.033866+0000","last_peered":"2024-09-13T18:22:06.033866+0000","last_clean":"2024-09-13T18:22:06.033866+0000","last_became_active":"2024-09-13T18:20:01.917134+0000","last_became_peered":"2024-09-13T18:20:01.917134+0000","last_unstale":"2024-09-13T18:22:06.033866+0000","last_undegraded":"2024-09-13T18:22:06.033866+0000","last_fullsized":"2024-09-13T18:22:06.033866+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:19:06.269454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'6","reported_seq":150,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.292167+0000","last_change":"2024-09-13T18:20:01.572258+0000","last_active":"2024-09-13T18:22:28.292167+0000","last_peered":"2024-09-13T18:22:28.292167+0000","last_clean":"2024-09-13T18:22:28.292167+0000","last_became_active":"2024-09-13T18:20:01.571968+0000","last_became_peered":"2024-09-13T18:20:01.571968+0000","last_unstale":"2024-09-13T18:22:28.292167+0000","last_undegraded":"2024-09-13T18:22:28.292167+0000","last_fullsized":"2024-09-13T18:22:28.292167+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:40:59.143123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":122,"num_read_kb":122,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:18.483349+0000","last_change":"2024-09-13T18:20:06.131714+0000","last_active":"2024-09-13T18:22:18.483349+0000","last_peered":"2024-09-13T18:22:18.483349+0000","last_clean":"2024-09-13T18:22:18.483349+0000","last_became_active":"2024-09-13T18:20:06.131244+0000","last_became_peered":"2024-09-13T18:20:06.131244+0000","last_unstale":"2024-09-13T18:22:18.483349+0000","last_undegraded":"2024-09-13T18:22:18.483349+0000","last_fullsized":"2024-09-13T18:22:18.483349+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:37.946002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"93'5","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:08.482116+0000","last_change":"2024-09-13T18:20:01.917351+0000","last_active":"2024-09-13T18:22:08.482116+0000","last_peered":"2024-09-13T18:22:08.482116+0000","last_clean":"2024-09-13T18:22:08.482116+0000","last_became_active":"2024-09-13T18:20:01.917067+0000","last_became_peered":"2024-09-13T18:20:01.917067+0000","last_unstale":"2024-09-13T18:22:08.482116+0000","last_undegraded":"2024-09-13T18:22:08.482116+0000","last_fullsized":"2024-09-13T18:22:08.482116+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:35:37.723132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'4","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:01.190731+0000","last_change":"2024-09-13T18:20:01.573515+0000","last_active":"2024-09-13T18:22:01.190731+0000","last_peered":"2024-09-13T18:22:01.190731+0000","last_clean":"2024-09-13T18:22:01.190731+0000","last_became_active":"2024-09-13T18:20:01.573211+0000","last_became_peered":"2024-09-13T18:20:01.573211+0000","last_unstale":"2024-09-13T18:22:01.190731+0000","last_undegraded":"2024-09-13T18:22:01.190731+0000","last_fullsized":"2024-09-13T18:22:01.190731+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:05:30.912467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:30.983462+0000","last_change":"2024-09-13T18:21:28.231613+0000","last_active":"2024-09-13T18:21:30.983462+0000","last_peered":"2024-09-13T18:21:30.983462+0000","last_clean":"2024-09-13T18:21:30.983462+0000","last_became_active":"2024-09-13T18:21:28.231243+0000","last_became_peered":"2024-09-13T18:21:28.231243+0000","last_unstale":"2024-09-13T18:21:30.983462+0000","last_undegraded":"2024-09-13T18:21:30.983462+0000","last_fullsized":"2024-09-13T18:21:30.983462+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:58:46.066195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.2","version":"93'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:08.101059+0000","last_change":"2024-09-13T18:20:01.921597+0000","last_active":"2024-09-13T18:22:08.101059+0000","last_peered":"2024-09-13T18:22:08.101059+0000","last_clean":"2024-09-13T18:22:08.101059+0000","last_became_active":"2024-09-13T18:20:01.920200+0000","last_became_peered":"2024-09-13T18:20:01.920200+0000","last_unstale":"2024-09-13T18:22:08.101059+0000","last_undegraded":"2024-09-13T18:22:08.101059+0000","last_fullsized":"2024-09-13T18:22:08.101059+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:22:40.517941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'6","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:06.737540+0000","last_change":"2024-09-13T18:20:01.571313+0000","last_active":"2024-09-13T18:22:06.737540+0000","last_peered":"2024-09-13T18:22:06.737540+0000","last_clean":"2024-09-13T18:22:06.737540+0000","last_became_active":"2024-09-13T18:20:01.570977+0000","last_became_peered":"2024-09-13T18:20:01.570977+0000","last_unstale":"2024-09-13T18:22:06.737540+0000","last_undegraded":"2024-09-13T18:22:06.737540+0000","last_fullsized":"2024-09-13T18:22:06.737540+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:06.870320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":347,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:41.188167+0000","last_change":"2024-09-13T18:18:51.237409+0000","last_active":"2024-09-13T18:21:41.188167+0000","last_peered":"2024-09-13T18:21:41.188167+0000","last_clean":"2024-09-13T18:21:41.188167+0000","last_became_active":"2024-09-13T18:18:51.236986+0000","last_became_peered":"2024-09-13T18:18:51.236986+0000","last_unstale":"2024-09-13T18:21:41.188167+0000","last_undegraded":"2024-09-13T18:21:41.188167+0000","last_fullsized":"2024-09-13T18:21:41.188167+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:15:40.757299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:15:40.757299+0000","last_clean_scrub_stamp":"2024-09-13T18:15:40.757299+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:58:29.679404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":19,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:11.192032+0000","last_change":"2024-09-13T18:20:06.130803+0000","last_active":"2024-09-13T18:22:11.192032+0000","last_peered":"2024-09-13T18:22:11.192032+0000","last_clean":"2024-09-13T18:22:11.192032+0000","last_became_active":"2024-09-13T18:20:06.130433+0000","last_became_peered":"2024-09-13T18:20:06.130433+0000","last_unstale":"2024-09-13T18:22:11.192032+0000","last_undegraded":"2024-09-13T18:22:11.192032+0000","last_fullsized":"2024-09-13T18:22:11.192032+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:23:11.787084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"93'7","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:21.193214+0000","last_change":"2024-09-13T18:20:01.922702+0000","last_active":"2024-09-13T18:22:21.193214+0000","last_peered":"2024-09-13T18:22:21.193214+0000","last_clean":"2024-09-13T18:22:21.193214+0000","last_became_active":"2024-09-13T18:20:01.921109+0000","last_became_peered":"2024-09-13T18:20:01.921109+0000","last_unstale":"2024-09-13T18:22:21.193214+0000","last_undegraded":"2024-09-13T18:22:21.193214+0000","last_fullsized":"2024-09-13T18:22:21.193214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:03:34.618054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"93'6","reported_seq":152,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.067908+0000","last_change":"2024-09-13T18:20:01.571024+0000","last_active":"2024-09-13T18:22:28.067908+0000","last_peered":"2024-09-13T18:22:28.067908+0000","last_clean":"2024-09-13T18:22:28.067908+0000","last_became_active":"2024-09-13T18:20:01.570687+0000","last_became_peered":"2024-09-13T18:20:01.570687+0000","last_unstale":"2024-09-13T18:22:28.067908+0000","last_undegraded":"2024-09-13T18:22:28.067908+0000","last_fullsized":"2024-09-13T18:22:28.067908+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:10:25.126179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1616,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":124,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:18.277060+0000","last_change":"2024-09-13T18:20:06.129025+0000","last_active":"2024-09-13T18:22:18.277060+0000","last_peered":"2024-09-13T18:22:18.277060+0000","last_clean":"2024-09-13T18:22:18.277060+0000","last_became_active":"2024-09-13T18:20:06.128618+0000","last_became_peered":"2024-09-13T18:20:06.128618+0000","last_unstale":"2024-09-13T18:22:18.277060+0000","last_undegraded":"2024-09-13T18:22:18.277060+0000","last_fullsized":"2024-09-13T18:22:18.277060+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:20:08.891259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"93'6","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:21.193185+0000","last_change":"2024-09-13T18:20:01.921697+0000","last_active":"2024-09-13T18:22:21.193185+0000","last_peered":"2024-09-13T18:22:21.193185+0000","last_clean":"2024-09-13T18:22:21.193185+0000","last_became_active":"2024-09-13T18:20:01.920174+0000","last_became_peered":"2024-09-13T18:20:01.920174+0000","last_unstale":"2024-09-13T18:22:21.193185+0000","last_undegraded":"2024-09-13T18:22:21.193185+0000","last_fullsized":"2024-09-13T18:22:21.193185+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:57:03.922725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"94'41","reported_seq":71,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.250386+0000","last_change":"2024-09-13T18:20:01.582102+0000","last_active":"2024-09-13T18:22:28.250386+0000","last_peered":"2024-09-13T18:22:28.250386+0000","last_clean":"2024-09-13T18:22:28.250386+0000","last_became_active":"2024-09-13T18:20:01.581841+0000","last_became_peered":"2024-09-13T18:20:01.581841+0000","last_unstale":"2024-09-13T18:22:28.250386+0000","last_undegraded":"2024-09-13T18:22:28.250386+0000","last_fullsized":"2024-09-13T18:22:28.250386+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:59:57.929703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:22.286171+0000","last_change":"2024-09-13T18:20:06.133126+0000","last_active":"2024-09-13T18:22:22.286171+0000","last_peered":"2024-09-13T18:22:22.286171+0000","last_clean":"2024-09-13T18:22:22.286171+0000","last_became_active":"2024-09-13T18:20:06.132747+0000","last_became_peered":"2024-09-13T18:20:06.132747+0000","last_unstale":"2024-09-13T18:22:22.286171+0000","last_undegraded":"2024-09-13T18:22:22.286171+0000","last_fullsized":"2024-09-13T18:22:22.286171+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:54:21.843090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"93'9","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:21.193232+0000","last_change":"2024-09-13T18:20:01.916306+0000","last_active":"2024-09-13T18:22:21.193232+0000","last_peered":"2024-09-13T18:22:21.193232+0000","last_clean":"2024-09-13T18:22:21.193232+0000","last_became_active":"2024-09-13T18:20:01.915739+0000","last_became_peered":"2024-09-13T18:20:01.915739+0000","last_unstale":"2024-09-13T18:22:21.193232+0000","last_undegraded":"2024-09-13T18:22:21.193232+0000","last_fullsized":"2024-09-13T18:22:21.193232+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:19:09.689909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"91'5","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:18.483339+0000","last_change":"2024-09-13T18:20:01.565821+0000","last_active":"2024-09-13T18:22:18.483339+0000","last_peered":"2024-09-13T18:22:18.483339+0000","last_clean":"2024-09-13T18:22:18.483339+0000","last_became_active":"2024-09-13T18:20:01.565467+0000","last_became_peered":"2024-09-13T18:20:01.565467+0000","last_unstale":"2024-09-13T18:22:18.483339+0000","last_undegraded":"2024-09-13T18:22:18.483339+0000","last_fullsized":"2024-09-13T18:22:18.483339+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:11.609400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:16.035040+0000","last_change":"2024-09-13T18:20:06.131348+0000","last_active":"2024-09-13T18:22:16.035040+0000","last_peered":"2024-09-13T18:22:16.035040+0000","last_clean":"2024-09-13T18:22:16.035040+0000","last_became_active":"2024-09-13T18:20:06.131019+0000","last_became_peered":"2024-09-13T18:20:06.131019+0000","last_unstale":"2024-09-13T18:22:16.035040+0000","last_undegraded":"2024-09-13T18:22:16.035040+0000","last_fullsized":"2024-09-13T18:22:16.035040+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:13:49.717503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"93'5","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:17.959923+0000","last_change":"2024-09-13T18:20:01.922306+0000","last_active":"2024-09-13T18:22:17.959923+0000","last_peered":"2024-09-13T18:22:17.959923+0000","last_clean":"2024-09-13T18:22:17.959923+0000","last_became_active":"2024-09-13T18:20:01.921956+0000","last_became_peered":"2024-09-13T18:20:01.921956+0000","last_unstale":"2024-09-13T18:22:17.959923+0000","last_undegraded":"2024-09-13T18:22:17.959923+0000","last_fullsized":"2024-09-13T18:22:17.959923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:17:48.755318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"94'3","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.204515+0000","last_change":"2024-09-13T18:20:27.194085+0000","last_active":"2024-09-13T18:22:28.204515+0000","last_peered":"2024-09-13T18:22:28.204515+0000","last_clean":"2024-09-13T18:22:28.204515+0000","last_became_active":"2024-09-13T18:20:27.190136+0000","last_became_peered":"2024-09-13T18:20:27.190136+0000","last_unstale":"2024-09-13T18:22:28.204515+0000","last_undegraded":"2024-09-13T18:22:28.204515+0000","last_fullsized":"2024-09-13T18:22:28.204515+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:57:07.202901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:16.035090+0000","last_change":"2024-09-13T18:20:06.135498+0000","last_active":"2024-09-13T18:22:16.035090+0000","last_peered":"2024-09-13T18:22:16.035090+0000","last_clean":"2024-09-13T18:22:16.035090+0000","last_became_active":"2024-09-13T18:20:06.135095+0000","last_became_peered":"2024-09-13T18:20:06.135095+0000","last_unstale":"2024-09-13T18:22:16.035090+0000","last_undegraded":"2024-09-13T18:22:16.035090+0000","last_fullsized":"2024-09-13T18:22:16.035090+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:06:40.609718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"93'6","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:55.905745+0000","last_change":"2024-09-13T18:20:27.186825+0000","last_active":"2024-09-13T18:21:55.905745+0000","last_peered":"2024-09-13T18:21:55.905745+0000","last_clean":"2024-09-13T18:21:55.905745+0000","last_became_active":"2024-09-13T18:20:27.186523+0000","last_became_peered":"2024-09-13T18:20:27.186523+0000","last_unstale":"2024-09-13T18:21:55.905745+0000","last_undegraded":"2024-09-13T18:21:55.905745+0000","last_fullsized":"2024-09-13T18:21:55.905745+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:28:30.453945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.10","version":"91'17","reported_seq":61,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:20.989307+0000","last_change":"2024-09-13T18:20:27.200142+0000","last_active":"2024-09-13T18:22:20.989307+0000","last_peered":"2024-09-13T18:22:20.989307+0000","last_clean":"2024-09-13T18:22:20.989307+0000","last_became_active":"2024-09-13T18:20:27.189670+0000","last_became_peered":"2024-09-13T18:20:27.189670+0000","last_unstale":"2024-09-13T18:22:20.989307+0000","last_undegraded":"2024-09-13T18:22:20.989307+0000","last_fullsized":"2024-09-13T18:22:20.989307+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:16:02.934269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:28.150641+0000","last_change":"2024-09-13T18:20:06.136742+0000","last_active":"2024-09-13T18:21:28.150641+0000","last_peered":"2024-09-13T18:21:28.150641+0000","last_clean":"2024-09-13T18:21:28.150641+0000","last_became_active":"2024-09-13T18:20:06.136373+0000","last_became_peered":"2024-09-13T18:20:06.136373+0000","last_unstale":"2024-09-13T18:21:28.150641+0000","last_undegraded":"2024-09-13T18:21:28.150641+0000","last_fullsized":"2024-09-13T18:21:28.150641+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:49:11.782489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"93'6","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.288955+0000","last_change":"2024-09-13T18:20:01.922910+0000","last_active":"2024-09-13T18:22:28.288955+0000","last_peered":"2024-09-13T18:22:28.288955+0000","last_clean":"2024-09-13T18:22:28.288955+0000","last_became_active":"2024-09-13T18:20:01.921162+0000","last_became_peered":"2024-09-13T18:20:01.921162+0000","last_unstale":"2024-09-13T18:22:28.288955+0000","last_undegraded":"2024-09-13T18:22:28.288955+0000","last_fullsized":"2024-09-13T18:22:28.288955+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:14:47.243418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"94'8","reported_seq":155,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.416185+0000","last_change":"2024-09-13T18:20:01.575830+0000","last_active":"2024-09-13T18:22:28.416185+0000","last_peered":"2024-09-13T18:22:28.416185+0000","last_clean":"2024-09-13T18:22:28.416185+0000","last_became_active":"2024-09-13T18:20:01.575334+0000","last_became_peered":"2024-09-13T18:20:01.575334+0000","last_unstale":"2024-09-13T18:22:28.416185+0000","last_undegraded":"2024-09-13T18:22:28.416185+0000","last_fullsized":"2024-09-13T18:22:28.416185+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:13:00.935600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1818,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":126,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:28.154729+0000","last_change":"2024-09-13T18:20:06.138609+0000","last_active":"2024-09-13T18:21:28.154729+0000","last_peered":"2024-09-13T18:21:28.154729+0000","last_clean":"2024-09-13T18:21:28.154729+0000","last_became_active":"2024-09-13T18:20:06.138232+0000","last_became_peered":"2024-09-13T18:20:06.138232+0000","last_unstale":"2024-09-13T18:21:28.154729+0000","last_undegraded":"2024-09-13T18:21:28.154729+0000","last_fullsized":"2024-09-13T18:21:28.154729+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:46:06.227407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"93'15","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.289290+0000","last_change":"2024-09-13T18:20:01.928757+0000","last_active":"2024-09-13T18:22:28.289290+0000","last_peered":"2024-09-13T18:22:28.289290+0000","last_clean":"2024-09-13T18:22:28.289290+0000","last_became_active":"2024-09-13T18:20:01.928350+0000","last_became_peered":"2024-09-13T18:20:01.928350+0000","last_unstale":"2024-09-13T18:22:28.289290+0000","last_undegraded":"2024-09-13T18:22:28.289290+0000","last_fullsized":"2024-09-13T18:22:28.289290+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:17:43.208106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"91'5","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:21.035873+0000","last_change":"2024-09-13T18:20:01.579128+0000","last_active":"2024-09-13T18:22:21.035873+0000","last_peered":"2024-09-13T18:22:21.035873+0000","last_clean":"2024-09-13T18:22:21.035873+0000","last_became_active":"2024-09-13T18:20:01.578796+0000","last_became_peered":"2024-09-13T18:20:01.578796+0000","last_unstale":"2024-09-13T18:22:21.035873+0000","last_undegraded":"2024-09-13T18:22:21.035873+0000","last_fullsized":"2024-09-13T18:22:21.035873+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:30:04.836103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.204056+0000","last_change":"2024-09-13T18:20:06.131404+0000","last_active":"2024-09-13T18:22:28.204056+0000","last_peered":"2024-09-13T18:22:28.204056+0000","last_clean":"2024-09-13T18:22:28.204056+0000","last_became_active":"2024-09-13T18:20:06.130931+0000","last_became_peered":"2024-09-13T18:20:06.130931+0000","last_unstale":"2024-09-13T18:22:28.204056+0000","last_undegraded":"2024-09-13T18:22:28.204056+0000","last_fullsized":"2024-09-13T18:22:28.204056+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:55:54.568816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"93'9","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.292205+0000","last_change":"2024-09-13T18:20:01.916089+0000","last_active":"2024-09-13T18:22:28.292205+0000","last_peered":"2024-09-13T18:22:28.292205+0000","last_clean":"2024-09-13T18:22:28.292205+0000","last_became_active":"2024-09-13T18:20:01.915783+0000","last_became_peered":"2024-09-13T18:20:01.915783+0000","last_unstale":"2024-09-13T18:22:28.292205+0000","last_undegraded":"2024-09-13T18:22:28.292205+0000","last_fullsized":"2024-09-13T18:22:28.292205+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:43:25.886029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"91'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:20.789776+0000","last_change":"2024-09-13T18:20:01.577381+0000","last_active":"2024-09-13T18:22:20.789776+0000","last_peered":"2024-09-13T18:22:20.789776+0000","last_clean":"2024-09-13T18:22:20.789776+0000","last_became_active":"2024-09-13T18:20:01.577044+0000","last_became_peered":"2024-09-13T18:20:01.577044+0000","last_unstale":"2024-09-13T18:22:20.789776+0000","last_undegraded":"2024-09-13T18:22:20.789776+0000","last_fullsized":"2024-09-13T18:22:20.789776+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:00.907948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.146073+0000","last_change":"2024-09-13T18:20:06.139971+0000","last_active":"2024-09-13T18:22:28.146073+0000","last_peered":"2024-09-13T18:22:28.146073+0000","last_clean":"2024-09-13T18:22:28.146073+0000","last_became_active":"2024-09-13T18:20:06.139507+0000","last_became_peered":"2024-09-13T18:20:06.139507+0000","last_unstale":"2024-09-13T18:22:28.146073+0000","last_undegraded":"2024-09-13T18:22:28.146073+0000","last_fullsized":"2024-09-13T18:22:28.146073+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:40:21.037768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"93'6","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.140553+0000","last_change":"2024-09-13T18:20:01.915348+0000","last_active":"2024-09-13T18:22:28.140553+0000","last_peered":"2024-09-13T18:22:28.140553+0000","last_clean":"2024-09-13T18:22:28.140553+0000","last_became_active":"2024-09-13T18:20:01.915042+0000","last_became_peered":"2024-09-13T18:20:01.915042+0000","last_unstale":"2024-09-13T18:22:28.140553+0000","last_undegraded":"2024-09-13T18:22:28.140553+0000","last_fullsized":"2024-09-13T18:22:28.140553+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:31:33.712314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"94'41","reported_seq":315,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.421008+0000","last_change":"2024-09-13T18:20:01.578456+0000","last_active":"2024-09-13T18:22:28.421008+0000","last_peered":"2024-09-13T18:22:28.421008+0000","last_clean":"2024-09-13T18:22:28.421008+0000","last_became_active":"2024-09-13T18:20:01.578051+0000","last_became_peered":"2024-09-13T18:20:01.578051+0000","last_unstale":"2024-09-13T18:22:28.421008+0000","last_undegraded":"2024-09-13T18:22:28.421008+0000","last_fullsized":"2024-09-13T18:22:28.421008+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:34:17.651620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69151,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":252,"num_read_kb":194,"num_write":43,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.137141+0000","last_change":"2024-09-13T18:20:06.131410+0000","last_active":"2024-09-13T18:22:28.137141+0000","last_peered":"2024-09-13T18:22:28.137141+0000","last_clean":"2024-09-13T18:22:28.137141+0000","last_became_active":"2024-09-13T18:20:06.131060+0000","last_became_peered":"2024-09-13T18:20:06.131060+0000","last_unstale":"2024-09-13T18:22:28.137141+0000","last_undegraded":"2024-09-13T18:22:28.137141+0000","last_fullsized":"2024-09-13T18:22:28.137141+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:08:25.266971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"93'11","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.292104+0000","last_change":"2024-09-13T18:20:01.928839+0000","last_active":"2024-09-13T18:22:28.292104+0000","last_peered":"2024-09-13T18:22:28.292104+0000","last_clean":"2024-09-13T18:22:28.292104+0000","last_became_active":"2024-09-13T18:20:01.925774+0000","last_became_peered":"2024-09-13T18:20:01.925774+0000","last_unstale":"2024-09-13T18:22:28.292104+0000","last_undegraded":"2024-09-13T18:22:28.292104+0000","last_fullsized":"2024-09-13T18:22:28.292104+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:15:23.049671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"91'5","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.146218+0000","last_change":"2024-09-13T18:20:01.576332+0000","last_active":"2024-09-13T18:22:28.146218+0000","last_peered":"2024-09-13T18:22:28.146218+0000","last_clean":"2024-09-13T18:22:28.146218+0000","last_became_active":"2024-09-13T18:20:01.575862+0000","last_became_peered":"2024-09-13T18:20:01.575862+0000","last_unstale":"2024-09-13T18:22:28.146218+0000","last_undegraded":"2024-09-13T18:22:28.146218+0000","last_fullsized":"2024-09-13T18:22:28.146218+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:17:00.452480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.206402+0000","last_change":"2024-09-13T18:20:06.133142+0000","last_active":"2024-09-13T18:22:28.206402+0000","last_peered":"2024-09-13T18:22:28.206402+0000","last_clean":"2024-09-13T18:22:28.206402+0000","last_became_active":"2024-09-13T18:20:06.132740+0000","last_became_peered":"2024-09-13T18:20:06.132740+0000","last_unstale":"2024-09-13T18:22:28.206402+0000","last_undegraded":"2024-09-13T18:22:28.206402+0000","last_fullsized":"2024-09-13T18:22:28.206402+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:12:46.442481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"93'9","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.820616+0000","last_change":"2024-09-13T18:20:01.919673+0000","last_active":"2024-09-13T18:22:28.820616+0000","last_peered":"2024-09-13T18:22:28.820616+0000","last_clean":"2024-09-13T18:22:28.820616+0000","last_became_active":"2024-09-13T18:20:01.919358+0000","last_became_peered":"2024-09-13T18:20:01.919358+0000","last_unstale":"2024-09-13T18:22:28.820616+0000","last_undegraded":"2024-09-13T18:22:28.820616+0000","last_fullsized":"2024-09-13T18:22:28.820616+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:40:11.407643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"91'4","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.289505+0000","last_change":"2024-09-13T18:20:27.190451+0000","last_active":"2024-09-13T18:22:28.289505+0000","last_peered":"2024-09-13T18:22:28.289505+0000","last_clean":"2024-09-13T18:22:28.289505+0000","last_became_active":"2024-09-13T18:20:27.187091+0000","last_became_peered":"2024-09-13T18:20:27.187091+0000","last_unstale":"2024-09-13T18:22:28.289505+0000","last_undegraded":"2024-09-13T18:22:28.289505+0000","last_fullsized":"2024-09-13T18:22:28.289505+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:57:07.976008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.136965+0000","last_change":"2024-09-13T18:20:06.133475+0000","last_active":"2024-09-13T18:22:28.136965+0000","last_peered":"2024-09-13T18:22:28.136965+0000","last_clean":"2024-09-13T18:22:28.136965+0000","last_became_active":"2024-09-13T18:20:06.133109+0000","last_became_peered":"2024-09-13T18:20:06.133109+0000","last_unstale":"2024-09-13T18:22:28.136965+0000","last_undegraded":"2024-09-13T18:22:28.136965+0000","last_fullsized":"2024-09-13T18:22:28.136965+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:47:19.567115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"93'10","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.292323+0000","last_change":"2024-09-13T18:20:01.922660+0000","last_active":"2024-09-13T18:22:28.292323+0000","last_peered":"2024-09-13T18:22:28.292323+0000","last_clean":"2024-09-13T18:22:28.292323+0000","last_became_active":"2024-09-13T18:20:01.922370+0000","last_became_peered":"2024-09-13T18:20:01.922370+0000","last_unstale":"2024-09-13T18:22:28.292323+0000","last_undegraded":"2024-09-13T18:22:28.292323+0000","last_fullsized":"2024-09-13T18:22:28.292323+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:54:32.029704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"93'6","reported_seq":270,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.420091+0000","last_change":"2024-09-13T18:20:01.580975+0000","last_active":"2024-09-13T18:22:28.420091+0000","last_peered":"2024-09-13T18:22:28.420091+0000","last_clean":"2024-09-13T18:22:28.420091+0000","last_became_active":"2024-09-13T18:20:01.580647+0000","last_became_peered":"2024-09-13T18:20:01.580647+0000","last_unstale":"2024-09-13T18:22:28.420091+0000","last_undegraded":"2024-09-13T18:22:28.420091+0000","last_fullsized":"2024-09-13T18:22:28.420091+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:26:20.942915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":242,"num_read_kb":242,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.066097+0000","last_change":"2024-09-13T18:20:06.140003+0000","last_active":"2024-09-13T18:22:28.066097+0000","last_peered":"2024-09-13T18:22:28.066097+0000","last_clean":"2024-09-13T18:22:28.066097+0000","last_became_active":"2024-09-13T18:20:06.139663+0000","last_became_peered":"2024-09-13T18:20:06.139663+0000","last_unstale":"2024-09-13T18:22:28.066097+0000","last_undegraded":"2024-09-13T18:22:28.066097+0000","last_fullsized":"2024-09-13T18:22:28.066097+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:54:49.662401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:07.284799+0000","last_change":"2024-09-13T18:20:06.134448+0000","last_active":"2024-09-13T18:22:07.284799+0000","last_peered":"2024-09-13T18:22:07.284799+0000","last_clean":"2024-09-13T18:22:07.284799+0000","last_became_active":"2024-09-13T18:20:06.134084+0000","last_became_peered":"2024-09-13T18:20:06.134084+0000","last_unstale":"2024-09-13T18:22:07.284799+0000","last_undegraded":"2024-09-13T18:22:07.284799+0000","last_fullsized":"2024-09-13T18:22:07.284799+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:55:50.475665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"93'11","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:55.907316+0000","last_change":"2024-09-13T18:20:01.918955+0000","last_active":"2024-09-13T18:21:55.907316+0000","last_peered":"2024-09-13T18:21:55.907316+0000","last_clean":"2024-09-13T18:21:55.907316+0000","last_became_active":"2024-09-13T18:20:01.918666+0000","last_became_peered":"2024-09-13T18:20:01.918666+0000","last_unstale":"2024-09-13T18:21:55.907316+0000","last_undegraded":"2024-09-13T18:21:55.907316+0000","last_fullsized":"2024-09-13T18:21:55.907316+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:08:50.697710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"91'3","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.145838+0000","last_change":"2024-09-13T18:20:01.578789+0000","last_active":"2024-09-13T18:22:28.145838+0000","last_peered":"2024-09-13T18:22:28.145838+0000","last_clean":"2024-09-13T18:22:28.145838+0000","last_became_active":"2024-09-13T18:20:01.578476+0000","last_became_peered":"2024-09-13T18:20:01.578476+0000","last_unstale":"2024-09-13T18:22:28.145838+0000","last_undegraded":"2024-09-13T18:22:28.145838+0000","last_fullsized":"2024-09-13T18:22:28.145838+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:40:58.355492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:12.959377+0000","last_change":"2024-09-13T18:21:28.423633+0000","last_active":"2024-09-13T18:22:12.959377+0000","last_peered":"2024-09-13T18:22:12.959377+0000","last_clean":"2024-09-13T18:22:12.959377+0000","last_became_active":"2024-09-13T18:21:28.423128+0000","last_became_peered":"2024-09-13T18:21:28.423128+0000","last_unstale":"2024-09-13T18:22:12.959377+0000","last_undegraded":"2024-09-13T18:22:12.959377+0000","last_fullsized":"2024-09-13T18:22:12.959377+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:06:32.687874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"93'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.066097+0000","last_change":"2024-09-13T18:20:01.914804+0000","last_active":"2024-09-13T18:22:28.066097+0000","last_peered":"2024-09-13T18:22:28.066097+0000","last_clean":"2024-09-13T18:22:28.066097+0000","last_became_active":"2024-09-13T18:20:01.914517+0000","last_became_peered":"2024-09-13T18:20:01.914517+0000","last_unstale":"2024-09-13T18:22:28.066097+0000","last_undegraded":"2024-09-13T18:22:28.066097+0000","last_fullsized":"2024-09-13T18:22:28.066097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:57:10.734935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"91'3","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.292059+0000","last_change":"2024-09-13T18:20:01.569201+0000","last_active":"2024-09-13T18:22:28.292059+0000","last_peered":"2024-09-13T18:22:28.292059+0000","last_clean":"2024-09-13T18:22:28.292059+0000","last_became_active":"2024-09-13T18:20:01.568908+0000","last_became_peered":"2024-09-13T18:20:01.568908+0000","last_unstale":"2024-09-13T18:22:28.292059+0000","last_undegraded":"2024-09-13T18:22:28.292059+0000","last_fullsized":"2024-09-13T18:22:28.292059+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:16:34.069130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"91'7","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:28.230871+0000","last_change":"2024-09-13T18:20:01.579225+0000","last_active":"2024-09-13T18:21:28.230871+0000","last_peered":"2024-09-13T18:21:28.230871+0000","last_clean":"2024-09-13T18:21:28.230871+0000","last_became_active":"2024-09-13T18:20:01.578826+0000","last_became_peered":"2024-09-13T18:20:01.578826+0000","last_unstale":"2024-09-13T18:21:28.230871+0000","last_undegraded":"2024-09-13T18:21:28.230871+0000","last_fullsized":"2024-09-13T18:21:28.230871+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:48:07.632239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"93'8","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:55.916389+0000","last_change":"2024-09-13T18:20:01.914885+0000","last_active":"2024-09-13T18:21:55.916389+0000","last_peered":"2024-09-13T18:21:55.916389+0000","last_clean":"2024-09-13T18:21:55.916389+0000","last_became_active":"2024-09-13T18:20:01.914599+0000","last_became_peered":"2024-09-13T18:20:01.914599+0000","last_unstale":"2024-09-13T18:21:55.916389+0000","last_undegraded":"2024-09-13T18:21:55.916389+0000","last_fullsized":"2024-09-13T18:21:55.916389+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:41:30.888675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:28.311102+0000","last_change":"2024-09-13T18:20:06.129376+0000","last_active":"2024-09-13T18:21:28.311102+0000","last_peered":"2024-09-13T18:21:28.311102+0000","last_clean":"2024-09-13T18:21:28.311102+0000","last_became_active":"2024-09-13T18:20:06.129032+0000","last_became_peered":"2024-09-13T18:20:06.129032+0000","last_unstale":"2024-09-13T18:21:28.311102+0000","last_undegraded":"2024-09-13T18:21:28.311102+0000","last_fullsized":"2024-09-13T18:21:28.311102+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:21:53.511813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'7","reported_seq":154,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.421305+0000","last_change":"2024-09-13T18:20:01.576069+0000","last_active":"2024-09-13T18:22:28.421305+0000","last_peered":"2024-09-13T18:22:28.421305+0000","last_clean":"2024-09-13T18:22:28.421305+0000","last_became_active":"2024-09-13T18:20:01.575760+0000","last_became_peered":"2024-09-13T18:20:01.575760+0000","last_unstale":"2024-09-13T18:22:28.421305+0000","last_undegraded":"2024-09-13T18:22:28.421305+0000","last_fullsized":"2024-09-13T18:22:28.421305+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:05:47.677848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1460,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":125,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"93'7","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:55.900375+0000","last_change":"2024-09-13T18:20:01.917073+0000","last_active":"2024-09-13T18:21:55.900375+0000","last_peered":"2024-09-13T18:21:55.900375+0000","last_clean":"2024-09-13T18:21:55.900375+0000","last_became_active":"2024-09-13T18:20:01.916765+0000","last_became_peered":"2024-09-13T18:20:01.916765+0000","last_unstale":"2024-09-13T18:21:55.900375+0000","last_undegraded":"2024-09-13T18:21:55.900375+0000","last_fullsized":"2024-09-13T18:21:55.900375+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:22:07.788313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:28.311181+0000","last_change":"2024-09-13T18:20:06.131429+0000","last_active":"2024-09-13T18:21:28.311181+0000","last_peered":"2024-09-13T18:21:28.311181+0000","last_clean":"2024-09-13T18:21:28.311181+0000","last_became_active":"2024-09-13T18:20:06.131019+0000","last_became_peered":"2024-09-13T18:20:06.131019+0000","last_unstale":"2024-09-13T18:21:28.311181+0000","last_undegraded":"2024-09-13T18:21:28.311181+0000","last_fullsized":"2024-09-13T18:21:28.311181+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:44:37.892574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"93'9","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:07.958537+0000","last_change":"2024-09-13T18:20:01.914050+0000","last_active":"2024-09-13T18:22:07.958537+0000","last_peered":"2024-09-13T18:22:07.958537+0000","last_clean":"2024-09-13T18:22:07.958537+0000","last_became_active":"2024-09-13T18:20:01.913733+0000","last_became_peered":"2024-09-13T18:20:01.913733+0000","last_unstale":"2024-09-13T18:22:07.958537+0000","last_undegraded":"2024-09-13T18:22:07.958537+0000","last_fullsized":"2024-09-13T18:22:07.958537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:52:43.672096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"93'12","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:10.788565+0000","last_change":"2024-09-13T18:20:01.924290+0000","last_active":"2024-09-13T18:22:10.788565+0000","last_peered":"2024-09-13T18:22:10.788565+0000","last_clean":"2024-09-13T18:22:10.788565+0000","last_became_active":"2024-09-13T18:20:01.923883+0000","last_became_peered":"2024-09-13T18:20:01.923883+0000","last_unstale":"2024-09-13T18:22:10.788565+0000","last_undegraded":"2024-09-13T18:22:10.788565+0000","last_fullsized":"2024-09-13T18:22:10.788565+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:11:18.737501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"93'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:10.788577+0000","last_change":"2024-09-13T18:20:27.184256+0000","last_active":"2024-09-13T18:22:10.788577+0000","last_peered":"2024-09-13T18:22:10.788577+0000","last_clean":"2024-09-13T18:22:10.788577+0000","last_became_active":"2024-09-13T18:20:27.183790+0000","last_became_peered":"2024-09-13T18:20:27.183790+0000","last_unstale":"2024-09-13T18:22:10.788577+0000","last_undegraded":"2024-09-13T18:22:10.788577+0000","last_fullsized":"2024-09-13T18:22:10.788577+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:06:48.885753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"93'7","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:07.284763+0000","last_change":"2024-09-13T18:20:01.915799+0000","last_active":"2024-09-13T18:22:07.284763+0000","last_peered":"2024-09-13T18:22:07.284763+0000","last_clean":"2024-09-13T18:22:07.284763+0000","last_became_active":"2024-09-13T18:20:01.915519+0000","last_became_peered":"2024-09-13T18:20:01.915519+0000","last_unstale":"2024-09-13T18:22:07.284763+0000","last_undegraded":"2024-09-13T18:22:07.284763+0000","last_fullsized":"2024-09-13T18:22:07.284763+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:08:26.065052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"93'4","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:13.482862+0000","last_change":"2024-09-13T18:20:01.918547+0000","last_active":"2024-09-13T18:22:13.482862+0000","last_peered":"2024-09-13T18:22:13.482862+0000","last_clean":"2024-09-13T18:22:13.482862+0000","last_became_active":"2024-09-13T18:20:01.918053+0000","last_became_peered":"2024-09-13T18:20:01.918053+0000","last_unstale":"2024-09-13T18:22:13.482862+0000","last_undegraded":"2024-09-13T18:22:13.482862+0000","last_fullsized":"2024-09-13T18:22:13.482862+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:25:58.235101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"93'6","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:12.959339+0000","last_change":"2024-09-13T18:20:01.916205+0000","last_active":"2024-09-13T18:22:12.959339+0000","last_peered":"2024-09-13T18:22:12.959339+0000","last_clean":"2024-09-13T18:22:12.959339+0000","last_became_active":"2024-09-13T18:20:01.915909+0000","last_became_peered":"2024-09-13T18:20:01.915909+0000","last_unstale":"2024-09-13T18:22:12.959339+0000","last_undegraded":"2024-09-13T18:22:12.959339+0000","last_fullsized":"2024-09-13T18:22:12.959339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:23:03.976513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"93'15","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:15.789324+0000","last_change":"2024-09-13T18:20:01.916621+0000","last_active":"2024-09-13T18:22:15.789324+0000","last_peered":"2024-09-13T18:22:15.789324+0000","last_clean":"2024-09-13T18:22:15.789324+0000","last_became_active":"2024-09-13T18:20:01.916335+0000","last_became_peered":"2024-09-13T18:20:01.916335+0000","last_unstale":"2024-09-13T18:22:15.789324+0000","last_undegraded":"2024-09-13T18:22:15.789324+0000","last_fullsized":"2024-09-13T18:22:15.789324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:08:16.104571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"93'2","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:13.101502+0000","last_change":"2024-09-13T18:20:01.921195+0000","last_active":"2024-09-13T18:22:13.101502+0000","last_peered":"2024-09-13T18:22:13.101502+0000","last_clean":"2024-09-13T18:22:13.101502+0000","last_became_active":"2024-09-13T18:20:01.920895+0000","last_became_peered":"2024-09-13T18:20:01.920895+0000","last_unstale":"2024-09-13T18:22:13.101502+0000","last_undegraded":"2024-09-13T18:22:13.101502+0000","last_fullsized":"2024-09-13T18:22:13.101502+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:59:53.920047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"93'6","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:16.035058+0000","last_change":"2024-09-13T18:20:01.910889+0000","last_active":"2024-09-13T18:22:16.035058+0000","last_peered":"2024-09-13T18:22:16.035058+0000","last_clean":"2024-09-13T18:22:16.035058+0000","last_became_active":"2024-09-13T18:20:01.910451+0000","last_became_peered":"2024-09-13T18:20:01.910451+0000","last_unstale":"2024-09-13T18:22:16.035058+0000","last_undegraded":"2024-09-13T18:22:16.035058+0000","last_fullsized":"2024-09-13T18:22:16.035058+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:06:52.552785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"93'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:18.101915+0000","last_change":"2024-09-13T18:20:01.914041+0000","last_active":"2024-09-13T18:22:18.101915+0000","last_peered":"2024-09-13T18:22:18.101915+0000","last_clean":"2024-09-13T18:22:18.101915+0000","last_became_active":"2024-09-13T18:20:01.913240+0000","last_became_peered":"2024-09-13T18:20:01.913240+0000","last_unstale":"2024-09-13T18:22:18.101915+0000","last_undegraded":"2024-09-13T18:22:18.101915+0000","last_fullsized":"2024-09-13T18:22:18.101915+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:39:37.062544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"93'10","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:18.277086+0000","last_change":"2024-09-13T18:20:01.927322+0000","last_active":"2024-09-13T18:22:18.277086+0000","last_peered":"2024-09-13T18:22:18.277086+0000","last_clean":"2024-09-13T18:22:18.277086+0000","last_became_active":"2024-09-13T18:20:01.926767+0000","last_became_peered":"2024-09-13T18:20:01.926767+0000","last_unstale":"2024-09-13T18:22:18.277086+0000","last_undegraded":"2024-09-13T18:22:18.277086+0000","last_fullsized":"2024-09-13T18:22:18.277086+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:20.986089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"93'11","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:18.483303+0000","last_change":"2024-09-13T18:20:01.916314+0000","last_active":"2024-09-13T18:22:18.483303+0000","last_peered":"2024-09-13T18:22:18.483303+0000","last_clean":"2024-09-13T18:22:18.483303+0000","last_became_active":"2024-09-13T18:20:01.915790+0000","last_became_peered":"2024-09-13T18:20:01.915790+0000","last_unstale":"2024-09-13T18:22:18.483303+0000","last_undegraded":"2024-09-13T18:22:18.483303+0000","last_fullsized":"2024-09-13T18:22:18.483303+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:20:30.063814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"93'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:17.959902+0000","last_change":"2024-09-13T18:20:27.187172+0000","last_active":"2024-09-13T18:22:17.959902+0000","last_peered":"2024-09-13T18:22:17.959902+0000","last_clean":"2024-09-13T18:22:17.959902+0000","last_became_active":"2024-09-13T18:20:27.186765+0000","last_became_peered":"2024-09-13T18:20:27.186765+0000","last_unstale":"2024-09-13T18:22:17.959902+0000","last_undegraded":"2024-09-13T18:22:17.959902+0000","last_fullsized":"2024-09-13T18:22:17.959902+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:37:01.469560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"93'7","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:20.789864+0000","last_change":"2024-09-13T18:20:01.924074+0000","last_active":"2024-09-13T18:22:20.789864+0000","last_peered":"2024-09-13T18:22:20.789864+0000","last_clean":"2024-09-13T18:22:20.789864+0000","last_became_active":"2024-09-13T18:20:01.922364+0000","last_became_peered":"2024-09-13T18:20:01.922364+0000","last_unstale":"2024-09-13T18:22:20.789864+0000","last_undegraded":"2024-09-13T18:22:20.789864+0000","last_fullsized":"2024-09-13T18:22:20.789864+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:51:33.824362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"93'10","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.138025+0000","last_change":"2024-09-13T18:20:01.914606+0000","last_active":"2024-09-13T18:22:28.138025+0000","last_peered":"2024-09-13T18:22:28.138025+0000","last_clean":"2024-09-13T18:22:28.138025+0000","last_became_active":"2024-09-13T18:20:01.914281+0000","last_became_peered":"2024-09-13T18:20:01.914281+0000","last_unstale":"2024-09-13T18:22:28.138025+0000","last_undegraded":"2024-09-13T18:22:28.138025+0000","last_fullsized":"2024-09-13T18:22:28.138025+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:55:42.211959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"93'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:22.960471+0000","last_change":"2024-09-13T18:20:01.928876+0000","last_active":"2024-09-13T18:22:22.960471+0000","last_peered":"2024-09-13T18:22:22.960471+0000","last_clean":"2024-09-13T18:22:22.960471+0000","last_became_active":"2024-09-13T18:20:01.928534+0000","last_became_peered":"2024-09-13T18:20:01.928534+0000","last_unstale":"2024-09-13T18:22:22.960471+0000","last_undegraded":"2024-09-13T18:22:22.960471+0000","last_fullsized":"2024-09-13T18:22:22.960471+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:14:28.196746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"93'9","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.291764+0000","last_change":"2024-09-13T18:20:01.920642+0000","last_active":"2024-09-13T18:22:28.291764+0000","last_peered":"2024-09-13T18:22:28.291764+0000","last_clean":"2024-09-13T18:22:28.291764+0000","last_became_active":"2024-09-13T18:20:01.920335+0000","last_became_peered":"2024-09-13T18:20:01.920335+0000","last_unstale":"2024-09-13T18:22:28.291764+0000","last_undegraded":"2024-09-13T18:22:28.291764+0000","last_fullsized":"2024-09-13T18:22:28.291764+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:59:20.894563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"93'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:22.286145+0000","last_change":"2024-09-13T18:20:27.187529+0000","last_active":"2024-09-13T18:22:22.286145+0000","last_peered":"2024-09-13T18:22:22.286145+0000","last_clean":"2024-09-13T18:22:22.286145+0000","last_became_active":"2024-09-13T18:20:27.187123+0000","last_became_peered":"2024-09-13T18:20:27.187123+0000","last_unstale":"2024-09-13T18:22:22.286145+0000","last_undegraded":"2024-09-13T18:22:22.286145+0000","last_fullsized":"2024-09-13T18:22:22.286145+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:43:19.601235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"93'7","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:22.286160+0000","last_change":"2024-09-13T18:20:01.917111+0000","last_active":"2024-09-13T18:22:22.286160+0000","last_peered":"2024-09-13T18:22:22.286160+0000","last_clean":"2024-09-13T18:22:22.286160+0000","last_became_active":"2024-09-13T18:20:01.916826+0000","last_became_peered":"2024-09-13T18:20:01.916826+0000","last_unstale":"2024-09-13T18:22:22.286160+0000","last_undegraded":"2024-09-13T18:22:22.286160+0000","last_fullsized":"2024-09-13T18:22:22.286160+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:36:08.150028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"93'10","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.292386+0000","last_change":"2024-09-13T18:20:01.904496+0000","last_active":"2024-09-13T18:22:28.292386+0000","last_peered":"2024-09-13T18:22:28.292386+0000","last_clean":"2024-09-13T18:22:28.292386+0000","last_became_active":"2024-09-13T18:20:01.904199+0000","last_became_peered":"2024-09-13T18:20:01.904199+0000","last_unstale":"2024-09-13T18:22:28.292386+0000","last_undegraded":"2024-09-13T18:22:28.292386+0000","last_fullsized":"2024-09-13T18:22:28.292386+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:44.044162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"93'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.206304+0000","last_change":"2024-09-13T18:20:01.919023+0000","last_active":"2024-09-13T18:22:28.206304+0000","last_peered":"2024-09-13T18:22:28.206304+0000","last_clean":"2024-09-13T18:22:28.206304+0000","last_became_active":"2024-09-13T18:20:01.918661+0000","last_became_peered":"2024-09-13T18:20:01.918661+0000","last_unstale":"2024-09-13T18:22:28.206304+0000","last_undegraded":"2024-09-13T18:22:28.206304+0000","last_fullsized":"2024-09-13T18:22:28.206304+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:25:26.779029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"93'6","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:55.925241+0000","last_change":"2024-09-13T18:20:01.922833+0000","last_active":"2024-09-13T18:21:55.925241+0000","last_peered":"2024-09-13T18:21:55.925241+0000","last_clean":"2024-09-13T18:21:55.925241+0000","last_became_active":"2024-09-13T18:20:01.922533+0000","last_became_peered":"2024-09-13T18:20:01.922533+0000","last_unstale":"2024-09-13T18:21:55.925241+0000","last_undegraded":"2024-09-13T18:21:55.925241+0000","last_fullsized":"2024-09-13T18:21:55.925241+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:27:11.557419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"93'16","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.066646+0000","last_change":"2024-09-13T18:20:01.925916+0000","last_active":"2024-09-13T18:22:28.066646+0000","last_peered":"2024-09-13T18:22:28.066646+0000","last_clean":"2024-09-13T18:22:28.066646+0000","last_became_active":"2024-09-13T18:20:01.925568+0000","last_became_peered":"2024-09-13T18:20:01.925568+0000","last_unstale":"2024-09-13T18:22:28.066646+0000","last_undegraded":"2024-09-13T18:22:28.066646+0000","last_fullsized":"2024-09-13T18:22:28.066646+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:58:24.955273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"93'11","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.292300+0000","last_change":"2024-09-13T18:20:01.924328+0000","last_active":"2024-09-13T18:22:28.292300+0000","last_peered":"2024-09-13T18:22:28.292300+0000","last_clean":"2024-09-13T18:22:28.292300+0000","last_became_active":"2024-09-13T18:20:01.924004+0000","last_became_peered":"2024-09-13T18:20:01.924004+0000","last_unstale":"2024-09-13T18:22:28.292300+0000","last_undegraded":"2024-09-13T18:22:28.292300+0000","last_fullsized":"2024-09-13T18:22:28.292300+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:23:29.204157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"93'9","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:55.913636+0000","last_change":"2024-09-13T18:20:01.922234+0000","last_active":"2024-09-13T18:21:55.913636+0000","last_peered":"2024-09-13T18:21:55.913636+0000","last_clean":"2024-09-13T18:21:55.913636+0000","last_became_active":"2024-09-13T18:20:01.921196+0000","last_became_peered":"2024-09-13T18:20:01.921196+0000","last_unstale":"2024-09-13T18:21:55.913636+0000","last_undegraded":"2024-09-13T18:21:55.913636+0000","last_fullsized":"2024-09-13T18:21:55.913636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:24:10.144346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"93'5","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:55.917809+0000","last_change":"2024-09-13T18:20:01.921740+0000","last_active":"2024-09-13T18:21:55.917809+0000","last_peered":"2024-09-13T18:21:55.917809+0000","last_clean":"2024-09-13T18:21:55.917809+0000","last_became_active":"2024-09-13T18:20:01.919634+0000","last_became_peered":"2024-09-13T18:20:01.919634+0000","last_unstale":"2024-09-13T18:21:55.917809+0000","last_undegraded":"2024-09-13T18:21:55.917809+0000","last_fullsized":"2024-09-13T18:21:55.917809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:43:27.600637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"93'6","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:55.915209+0000","last_change":"2024-09-13T18:20:01.924806+0000","last_active":"2024-09-13T18:21:55.915209+0000","last_peered":"2024-09-13T18:21:55.915209+0000","last_clean":"2024-09-13T18:21:55.915209+0000","last_became_active":"2024-09-13T18:20:01.924514+0000","last_became_peered":"2024-09-13T18:20:01.924514+0000","last_unstale":"2024-09-13T18:21:55.915209+0000","last_undegraded":"2024-09-13T18:21:55.915209+0000","last_fullsized":"2024-09-13T18:21:55.915209+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:47.323362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"93'4","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:55.901946+0000","last_change":"2024-09-13T18:20:01.912402+0000","last_active":"2024-09-13T18:21:55.901946+0000","last_peered":"2024-09-13T18:21:55.901946+0000","last_clean":"2024-09-13T18:21:55.901946+0000","last_became_active":"2024-09-13T18:20:01.911534+0000","last_became_peered":"2024-09-13T18:20:01.911534+0000","last_unstale":"2024-09-13T18:21:55.901946+0000","last_undegraded":"2024-09-13T18:21:55.901946+0000","last_fullsized":"2024-09-13T18:21:55.901946+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:22:52.021688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"93'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:01.737020+0000","last_change":"2024-09-13T18:20:01.930248+0000","last_active":"2024-09-13T18:22:01.737020+0000","last_peered":"2024-09-13T18:22:01.737020+0000","last_clean":"2024-09-13T18:22:01.737020+0000","last_became_active":"2024-09-13T18:20:01.929931+0000","last_became_peered":"2024-09-13T18:20:01.929931+0000","last_unstale":"2024-09-13T18:22:01.737020+0000","last_undegraded":"2024-09-13T18:22:01.737020+0000","last_fullsized":"2024-09-13T18:22:01.737020+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:37:33.086722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"93'7","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:07.284781+0000","last_change":"2024-09-13T18:20:01.921913+0000","last_active":"2024-09-13T18:22:07.284781+0000","last_peered":"2024-09-13T18:22:07.284781+0000","last_clean":"2024-09-13T18:22:07.284781+0000","last_became_active":"2024-09-13T18:20:01.921395+0000","last_became_peered":"2024-09-13T18:20:01.921395+0000","last_unstale":"2024-09-13T18:22:07.284781+0000","last_undegraded":"2024-09-13T18:22:07.284781+0000","last_fullsized":"2024-09-13T18:22:07.284781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:54:50.816870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"93'4","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:07.284827+0000","last_change":"2024-09-13T18:20:01.912654+0000","last_active":"2024-09-13T18:22:07.284827+0000","last_peered":"2024-09-13T18:22:07.284827+0000","last_clean":"2024-09-13T18:22:07.284827+0000","last_became_active":"2024-09-13T18:20:01.912356+0000","last_became_peered":"2024-09-13T18:20:01.912356+0000","last_unstale":"2024-09-13T18:22:07.284827+0000","last_undegraded":"2024-09-13T18:22:07.284827+0000","last_fullsized":"2024-09-13T18:22:07.284827+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:25:18.099396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"93'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:07.958659+0000","last_change":"2024-09-13T18:20:01.923373+0000","last_active":"2024-09-13T18:22:07.958659+0000","last_peered":"2024-09-13T18:22:07.958659+0000","last_clean":"2024-09-13T18:22:07.958659+0000","last_became_active":"2024-09-13T18:20:01.923077+0000","last_became_peered":"2024-09-13T18:20:01.923077+0000","last_unstale":"2024-09-13T18:22:07.958659+0000","last_undegraded":"2024-09-13T18:22:07.958659+0000","last_fullsized":"2024-09-13T18:22:07.958659+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:07:52.955705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":604133,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1347,"num_read_kb":793,"num_write":365,"num_write_kb":1467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":712704,"data_stored":1812399,"data_compressed":80730,"data_compressed_allocated":110592,"data_compressed_original":1572864,"omap_allocated":0,"internal_metadata":0},"log_size":299,"ondisk_log_size":299,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107472,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383723,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":1044480,"data_stored":529645,"data_compressed":26479,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:22:20 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.51200000000000001,"15min":0.48799999999999999},"min":{"1min":0.36299999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.70899999999999996,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.48899999999999999,"15min":0.46899999999999997},"min":{"1min":0.35999999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.70299999999999996,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.33600000000000002}]},{"osd":1,"last update":"Fri Sep 13 18:22:20 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.59199999999999997,"15min":0.57399999999999995},"min":{"1min":0.46000000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.88100000000000001,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.57899999999999996,"15min":0.56899999999999995},"min":{"1min":0.40200000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.78700000000000003,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.38500000000000001}]},{"osd":2,"last update":"Fri Sep 13 18:22:20 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.45400000000000001,"15min":0.44900000000000001},"min":{"1min":0.34300000000000003,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.63100000000000001,"5min":1.238,"15min":1.238},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.44800000000000001,"15min":0.434},"min":{"1min":0.33700000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.70799999999999996,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.40000000000000002}]},{"osd":3,"last update":"Fri Sep 13 18:21:27 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.504},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.47199999999999998}]},{"osd":4,"last update":"Fri Sep 13 18:22:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.63100000000000001,"15min":0.60999999999999999},"min":{"1min":0.53600000000000003,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85899999999999999,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.58799999999999997,"15min":0.56899999999999995},"min":{"1min":0.47099999999999997,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.84199999999999997,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.42199999999999999}]},{"osd":5,"last update":"Fri Sep 13 18:22:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.51900000000000002,"15min":0.50600000000000001},"min":{"1min":0.36099999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.73899999999999999,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.624,"5min":0.54600000000000004,"15min":0.52200000000000002},"min":{"1min":0.47799999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.81699999999999995,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.439}]},{"osd":6,"last update":"Fri Sep 13 18:22:20 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57799999999999996,"15min":0.56899999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85499999999999998,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.61499999999999999,"15min":0.59799999999999998},"min":{"1min":0.54100000000000004,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.79300000000000004,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.45900000000000002}]},{"osd":7,"last update":"Fri Sep 13 18:22:20 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.67900000000000005,"15min":0.64700000000000002},"min":{"1min":0.59699999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.95599999999999996,"5min":1.123,"15min":1.123},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.66600000000000004,"15min":0.65000000000000002},"min":{"1min":0.53400000000000003,"5min":0.439,"15min":0.439},"max":{"1min":0.94099999999999995,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.48199999999999998}]},{"osd":8,"last update":"Fri Sep 13 18:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.65100000000000002,"15min":0.63800000000000001},"min":{"1min":0.55500000000000005,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.72299999999999998,"15min":0.71099999999999997},"min":{"1min":0.64000000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.64300000000000002}]},{"osd":9,"last update":"Fri Sep 13 18:22:20 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.70099999999999996,"15min":0.68899999999999995},"min":{"1min":0.60399999999999998,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.91200000000000003,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.71499999999999997,"15min":0.68700000000000006},"min":{"1min":0.624,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.95399999999999996,"5min":1.208,"15min":1.208},"last":0.68200000000000005}]},{"osd":10,"last update":"Fri Sep 13 18:22:20 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.69999999999999996,"15min":0.67900000000000005},"min":{"1min":0.621,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.91100000000000003,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.72399999999999998,"15min":0.70599999999999996},"min":{"1min":0.66500000000000004,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.91500000000000004,"5min":1.583,"15min":1.583},"last":0.57599999999999996}]}]},{"osd":10,"up_from":72,"seq":309237645359,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28208,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716944,"statfs":{"total":95995035648,"available":95966150656,"internally_reserved":0,"allocated":1212416,"data_stored":1321739,"data_compressed":70067,"data_compressed_allocated":94208,"data_compressed_original":794624,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:22:03 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.502,"15min":0.504},"min":{"1min":0.40300000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.66700000000000004,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":1.018,"15min":1.107},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.624,"5min":14.589,"15min":14.589},"last":0.373}]},{"osd":1,"last update":"Fri Sep 13 18:21:55 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.41599999999999998,"15min":0.41999999999999998},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.54700000000000004,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.41999999999999998,"15min":0.41699999999999998},"min":{"1min":0.309,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.57799999999999996,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.29499999999999998}]},{"osd":2,"last update":"Fri Sep 13 18:21:55 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55000000000000004,"15min":0.54400000000000004},"min":{"1min":0.33700000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.72599999999999998,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56499999999999995,"15min":0.55500000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.74199999999999999,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.53100000000000003}]},{"osd":3,"last update":"Fri Sep 13 18:21:55 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.56399999999999995,"15min":0.57399999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.70499999999999996,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.68200000000000005,"15min":0.61099999999999999},"min":{"1min":0.377,"5min":0.376,"15min":0.376},"max":{"1min":0.67300000000000004,"5min":14.505000000000001,"15min":14.505000000000001},"last":0.55100000000000005}]},{"osd":4,"last update":"Fri Sep 13 18:21:55 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.503,"15min":0.50800000000000001},"min":{"1min":0.34799999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.624,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.495,"15min":0.499},"min":{"1min":0.33800000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.68600000000000005,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.312}]},{"osd":5,"last update":"Fri Sep 13 18:21:55 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.60999999999999999,"15min":0.60199999999999998},"min":{"1min":0.45500000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.82799999999999996,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.626},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.71199999999999997,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.48599999999999999}]},{"osd":6,"last update":"Fri Sep 13 18:21:55 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.752,"15min":0.67700000000000005},"min":{"1min":0.496,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.77800000000000002,"5min":14.539999999999999,"15min":14.539999999999999},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.64200000000000002,"15min":0.64700000000000002},"min":{"1min":0.50800000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.92600000000000005,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.41699999999999998}]},{"osd":7,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.81599999999999995}]},{"osd":8,"last update":"Fri Sep 13 18:21:55 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.66700000000000004,"15min":0.67100000000000004},"min":{"1min":0.41999999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.90900000000000003,"5min":1.01,"15min":1.01},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.68899999999999995,"15min":0.69299999999999995},"min":{"1min":0.51300000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.81599999999999995,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.54200000000000004}]},{"osd":9,"last update":"Fri Sep 13 18:21:55 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.68400000000000005,"15min":0.69099999999999995},"min":{"1min":0.53600000000000003,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.89400000000000002,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.436},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.69999999999999996,"15min":0.71299999999999997},"min":{"1min":0.45800000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.81100000000000005,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.497}]},{"osd":11,"last update":"Fri Sep 13 18:22:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.72099999999999997,"15min":0.72899999999999998},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.84199999999999997,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.745,"5min":0.73799999999999999,"15min":0.73999999999999999},"min":{"1min":0.55400000000000005,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.89900000000000002,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.56200000000000006}]}]},{"osd":9,"up_from":67,"seq":287762808884,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1089536,"data_stored":922323,"data_compressed":45625,"data_compressed_allocated":61440,"data_compressed_original":466944,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:21:33 2024","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.36099999999999999,"15min":0.38100000000000001},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.45800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.41399999999999998,"15min":0.442},"min":{"1min":0.249,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.495,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.38700000000000001}]},{"osd":1,"last update":"Fri Sep 13 18:21:33 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.56299999999999994,"15min":0.58399999999999996},"min":{"1min":0.35699999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.67300000000000004,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.56399999999999995,"15min":0.58499999999999996},"min":{"1min":0.36399999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.67700000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.36599999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:21:33 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.55000000000000004,"15min":0.57399999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.65000000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.52600000000000002,"15min":0.52800000000000002},"min":{"1min":0.34899999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.42099999999999999}]},{"osd":3,"last update":"Fri Sep 13 18:21:33 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.44700000000000001,"15min":0.48099999999999998},"min":{"1min":0.28499999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.50900000000000001,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.37,"5min":0.437,"15min":0.46300000000000002},"min":{"1min":0.28000000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.496,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.32300000000000001}]},{"osd":4,"last update":"Fri Sep 13 18:21:33 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.55600000000000005,"15min":0.57299999999999995},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.67700000000000005,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.61299999999999999,"15min":0.64700000000000002},"min":{"1min":0.39000000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.65300000000000002,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.54200000000000004}]},{"osd":5,"last update":"Fri Sep 13 18:21:33 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.57199999999999995,"15min":0.59099999999999997},"min":{"1min":0.36599999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.59299999999999997,"15min":0.61099999999999999},"min":{"1min":0.40600000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.752,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.40400000000000003}]},{"osd":6,"last update":"Fri Sep 13 18:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.39800000000000002}]},{"osd":7,"last update":"Fri Sep 13 18:21:33 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.63600000000000001,"15min":0.67400000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.70899999999999996,"5min":0.871,"15min":0.871},"last":0.437},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.64700000000000002,"15min":0.68100000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.63900000000000001,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.42899999999999999}]},{"osd":8,"last update":"Fri Sep 13 18:21:33 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.63500000000000001,"15min":0.66500000000000004},"min":{"1min":0.40999999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.75,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.66500000000000004,"15min":0.69999999999999996},"min":{"1min":0.436,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.81899999999999995,"5min":1.02,"15min":1.02},"last":0.51900000000000002}]},{"osd":10,"last update":"Fri Sep 13 18:21:54 2024","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":0.73499999999999999,"15min":0.69299999999999995},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":9.3510000000000009,"5min":9.3510000000000009,"15min":9.3510000000000009},"last":0.505},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.65900000000000003,"15min":0.68300000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.79100000000000004,"5min":0.879,"15min":0.879},"last":0.498}]},{"osd":11,"last update":"Fri Sep 13 18:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.48199999999999998,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.72999999999999998,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.45900000000000002,"5min":0.432,"15min":0.432},"max":{"1min":0.65300000000000002,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.377}]}]},{"osd":8,"up_from":59,"seq":253403070521,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1081344,"data_stored":922679,"data_compressed":45625,"data_compressed_allocated":61440,"data_compressed_original":466944,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.51300000000000001,"15min":0.54000000000000004},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.72299999999999998,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.55300000000000005,"15min":0.56299999999999994},"min":{"1min":0.35199999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.65000000000000002,"5min":3.2989999999999999,"15min":3.2989999999999999},"last":0.44600000000000001}]},{"osd":1,"last update":"Fri Sep 13 18:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.57699999999999996,"15min":0.56499999999999995},"min":{"1min":0.36699999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.79100000000000004,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.58399999999999996,"15min":0.57599999999999996},"min":{"1min":0.42399999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72699999999999998,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.36199999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.42499999999999999,"15min":0.42899999999999999},"min":{"1min":0.25,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.58199999999999996,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.314},{"interface":"front","average":{"1min":0.442,"5min":0.439,"15min":0.437},"min":{"1min":0.307,"5min":0.224,"15min":0.224},"max":{"1min":0.68899999999999995,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.56699999999999995}]},{"osd":3,"last update":"Fri Sep 13 18:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53700000000000003,"15min":0.55500000000000005},"min":{"1min":0.35899999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.69999999999999996,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.53400000000000003,"15min":0.53800000000000003},"min":{"1min":0.37,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.68799999999999994,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.38800000000000001}]},{"osd":4,"last update":"Fri Sep 13 18:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.66400000000000003,"15min":0.63600000000000001},"min":{"1min":0.45100000000000001,"5min":0.37,"15min":0.37},"max":{"1min":5.3540000000000001,"5min":5.3540000000000001,"15min":5.3540000000000001},"last":0.68000000000000005}]},{"osd":5,"last update":"Fri Sep 13 18:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.52000000000000002,"15min":0.53400000000000003},"min":{"1min":0.33600000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.67500000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.51100000000000001,"15min":0.52100000000000002},"min":{"1min":0.33400000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.64900000000000002,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.40300000000000002}]},{"osd":6,"last update":"Fri Sep 13 18:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.61299999999999999,"15min":0.61499999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.77700000000000002,"5min":3.2719999999999998,"15min":3.2719999999999998},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.61499999999999999,"15min":0.64000000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.71999999999999997,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.48899999999999999}]},{"osd":7,"last update":"Fri Sep 13 18:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.65600000000000003,"15min":0.67200000000000004},"min":{"1min":0.45500000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.77900000000000003,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.66200000000000003,"15min":0.66500000000000004},"min":{"1min":0.51800000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.81999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.51400000000000001}]},{"osd":9,"last update":"Fri Sep 13 18:22:25 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63700000000000001,"15min":0.63300000000000001},"min":{"1min":0.50900000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73799999999999999,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.63500000000000001,"15min":0.64800000000000002},"min":{"1min":0.39400000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.82199999999999995,"5min":1.012,"15min":1.012},"last":0.505}]},{"osd":10,"last update":"Fri Sep 13 18:21:53 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.85499999999999998,"15min":0.93400000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.79800000000000004,"5min":5.3360000000000003,"15min":5.3360000000000003},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.67800000000000005,"15min":0.67400000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.80900000000000005,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.46999999999999997}]},{"osd":11,"last update":"Fri Sep 13 18:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.65500000000000003,"15min":0.65900000000000003},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.76000000000000001,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.499},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.71599999999999997,"15min":0.72199999999999998},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.79900000000000004,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.53000000000000003}]}]},{"osd":7,"up_from":51,"seq":219043332157,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1114112,"data_stored":1128889,"data_compressed":62303,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.94399999999999995,"15min":1.012},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.84299999999999997,"5min":12.068,"15min":12.068},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.46999999999999997,"15min":0.45900000000000002},"min":{"1min":0.35199999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.61599999999999999}]},{"osd":1,"last update":"Fri Sep 13 18:21:46 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.379,"15min":0.36099999999999999},"min":{"1min":0.27300000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.40300000000000002,"15min":0.38200000000000001},"min":{"1min":0.30399999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.54500000000000004,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.26500000000000001}]},{"osd":2,"last update":"Fri Sep 13 18:21:46 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.627,"15min":0.54900000000000004},"min":{"1min":0.40999999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.79100000000000004,"5min":7.6820000000000004,"15min":7.6820000000000004},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.54000000000000004,"15min":0.51500000000000001},"min":{"1min":0.45600000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.40500000000000003}]},{"osd":3,"last update":"Fri Sep 13 18:21:46 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.52900000000000003,"15min":0.49299999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.70599999999999996,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.52400000000000002,"15min":0.49399999999999999},"min":{"1min":0.35599999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.73699999999999999,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.38600000000000001}]},{"osd":4,"last update":"Fri Sep 13 18:21:46 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.45800000000000002,"15min":0.439},"min":{"1min":0.32000000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.57499999999999996,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.46100000000000002,"15min":0.45300000000000001},"min":{"1min":0.32500000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.66500000000000004,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.33000000000000002}]},{"osd":5,"last update":"Fri Sep 13 18:21:46 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.59299999999999997,"15min":0.56200000000000006},"min":{"1min":0.46600000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.76600000000000001,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.56899999999999995,"15min":0.54100000000000004},"min":{"1min":0.40999999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.748,"5min":0.754,"15min":0.754},"last":0.47299999999999998}]},{"osd":6,"last update":"Fri Sep 13 18:21:46 2024","interfaces":[{"interface":"back","average":{"1min":1.5700000000000001,"5min":0.77400000000000002,"15min":0.61499999999999999},"min":{"1min":0.40300000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":20.013000000000002,"5min":20.013000000000002,"15min":20.013000000000002},"last":0.438},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.60899999999999999,"15min":0.58699999999999997},"min":{"1min":0.38500000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72999999999999998,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.51000000000000001}]},{"osd":8,"last update":"Fri Sep 13 18:22:04 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60599999999999998,"15min":0.59199999999999997},"min":{"1min":0.497,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.70499999999999996,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.747,"15min":0.67000000000000004},"min":{"1min":0.505,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.79600000000000004,"5min":7.7149999999999999,"15min":7.7149999999999999},"last":0.53700000000000003}]},{"osd":9,"last update":"Fri Sep 13 18:21:29 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.66300000000000003,"15min":0.66100000000000003},"min":{"1min":0.40500000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.88,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.45600000000000002},{"interface":"front","average":{"1min":1.609,"5min":0.82999999999999996,"15min":0.70199999999999996},"min":{"1min":0.42699999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":20.056999999999999,"5min":20.056999999999999,"15min":20.056999999999999},"last":0.42399999999999999}]},{"osd":10,"last update":"Fri Sep 13 18:21:51 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.65100000000000002,"15min":0.65300000000000002},"min":{"1min":0.43099999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.79900000000000004,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.496},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.63800000000000001,"15min":0.64200000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.755,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.44700000000000001}]},{"osd":11,"last update":"Fri Sep 13 18:22:04 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.71899999999999997,"15min":0.72699999999999998},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.78500000000000003,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.502},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.73899999999999999,"15min":0.747},"min":{"1min":0.59999999999999998,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.80800000000000005,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.48799999999999999}]}]},{"osd":6,"up_from":46,"seq":197568495682,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":1040384,"data_stored":525841,"data_compressed":26479,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.37,"15min":0.36699999999999999},"min":{"1min":0.25900000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":1.2030000000000001},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.41399999999999998,"15min":0.432},"min":{"1min":0.28699999999999998,"5min":0.218,"15min":0.218},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":1.0940000000000001}]},{"osd":1,"last update":"Fri Sep 13 18:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.53700000000000003,"15min":0.52500000000000002},"min":{"1min":0.311,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.56399999999999995,"15min":0.54100000000000004},"min":{"1min":0.45300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":1.0669999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.42399999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78600000000000003,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.54700000000000004,"15min":0.55100000000000005},"min":{"1min":0.38800000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":1.2729999999999999}]},{"osd":3,"last update":"Fri Sep 13 18:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.41099999999999998,"15min":0.42199999999999999},"min":{"1min":0.28499999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":1.254},{"interface":"front","average":{"1min":0.499,"5min":0.46200000000000002,"15min":0.46999999999999997},"min":{"1min":0.34300000000000003,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":1.179}]},{"osd":4,"last update":"Fri Sep 13 18:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.53200000000000003,"15min":0.51800000000000002},"min":{"1min":0.35599999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.55400000000000005,"15min":0.55300000000000005},"min":{"1min":0.39400000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":1.2190000000000001}]},{"osd":5,"last update":"Fri Sep 13 18:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.59399999999999997,"15min":0.60599999999999998},"min":{"1min":0.48999999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":1.038},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.56299999999999994,"15min":0.58899999999999997},"min":{"1min":0.377,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.83999999999999997,"5min":0.874,"15min":0.874},"last":0.83999999999999997}]},{"osd":7,"last update":"Fri Sep 13 18:21:45 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57899999999999996,"15min":0.57499999999999996},"min":{"1min":0.41099999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.56299999999999994,"15min":0.56100000000000005},"min":{"1min":0.442,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.70399999999999996,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":1.2889999999999999}]},{"osd":8,"last update":"Fri Sep 13 18:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.61899999999999999,"15min":0.621},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.91800000000000004,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":1.01},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.58599999999999997,"15min":0.59599999999999997},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.71599999999999997,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":1.3340000000000001}]},{"osd":9,"last update":"Fri Sep 13 18:21:34 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.51500000000000001,"15min":0.53100000000000003},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.57199999999999995,"5min":0.871,"15min":0.871},"last":1.3089999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.59099999999999997,"15min":0.60499999999999998},"min":{"1min":0.41399999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.75700000000000001,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":1.153}]},{"osd":10,"last update":"Fri Sep 13 18:21:55 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.60599999999999998,"15min":0.61499999999999999},"min":{"1min":0.48099999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.71799999999999997,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.66700000000000004,"15min":0.67100000000000004},"min":{"1min":0.53300000000000003,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.78300000000000003,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.96799999999999997}]},{"osd":11,"last update":"Fri Sep 13 18:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.63900000000000001,"15min":0.64600000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.77400000000000002,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":1.3600000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68600000000000005,"15min":0.68700000000000006},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.90200000000000002,"5min":1.01,"15min":1.01},"last":1.127}]}]},{"osd":5,"up_from":38,"seq":163208757319,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1142784,"data_stored":1253611,"data_compressed":68459,"data_compressed_allocated":90112,"data_compressed_original":786432,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:22:00 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.497,"15min":0.495},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.65000000000000002,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.51200000000000001,"15min":0.51100000000000001},"min":{"1min":0.32300000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.73199999999999998,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.373}]},{"osd":1,"last update":"Fri Sep 13 18:22:00 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.54300000000000004,"15min":0.51100000000000001},"min":{"1min":0.33300000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.71599999999999997,"5min":0.878,"15min":0.878},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.53300000000000003,"15min":0.51400000000000001},"min":{"1min":0.45800000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.68899999999999995,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.45400000000000001}]},{"osd":2,"last update":"Fri Sep 13 18:22:00 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.39600000000000002,"15min":0.36799999999999999},"min":{"1min":0.27900000000000003,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.65100000000000002,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.315},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.39900000000000002,"15min":0.38700000000000001},"min":{"1min":0.29999999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.34399999999999997}]},{"osd":3,"last update":"Fri Sep 13 18:22:00 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.52300000000000002,"15min":0.503},"min":{"1min":0.34799999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.70199999999999996,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.52200000000000002,"15min":0.52000000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.46800000000000003}]},{"osd":4,"last update":"Fri Sep 13 18:22:00 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55900000000000005,"15min":0.54200000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.79000000000000004,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.79200000000000004,"15min":0.61799999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82999999999999996,"5min":18.571000000000002,"15min":18.571000000000002},"last":0.47699999999999998}]},{"osd":6,"last update":"Fri Sep 13 18:22:25 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57499999999999996,"15min":0.57399999999999995},"min":{"1min":0.47199999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.73299999999999998,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56000000000000005,"15min":0.55200000000000005},"min":{"1min":0.38400000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.53600000000000003}]},{"osd":7,"last update":"Fri Sep 13 18:21:45 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61599999999999999,"15min":0.60799999999999998},"min":{"1min":0.46300000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76400000000000001,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63100000000000001,"15min":0.61599999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.84999999999999998,"5min":0.88,"15min":0.88},"last":0.499}]},{"osd":8,"last update":"Fri Sep 13 18:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.52700000000000002,"15min":0.51600000000000001},"min":{"1min":0.40200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.56899999999999995,"15min":0.54500000000000004},"min":{"1min":0.40600000000000003,"5min":0.314,"15min":0.314},"max":{"1min":0.876,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.56499999999999995}]},{"osd":9,"last update":"Fri Sep 13 18:21:30 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.64500000000000002,"15min":0.627},"min":{"1min":0.48199999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.63900000000000001,"15min":0.628},"min":{"1min":0.48799999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.81299999999999994,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.57699999999999996}]},{"osd":10,"last update":"Fri Sep 13 18:21:54 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69199999999999995,"15min":0.69099999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.68000000000000005,"15min":0.67000000000000004},"min":{"1min":0.53300000000000003,"5min":0.37,"15min":0.37},"max":{"1min":0.89000000000000001,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.55900000000000005}]}]},{"osd":4,"up_from":33,"seq":141733920843,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":529915,"data_compressed":26479,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:19:33 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.48299999999999998,"15min":0.47699999999999998},"min":{"1min":0.35399999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.69399999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.433},{"interface":"front","average":{"1min":0.501,"5min":0.51200000000000001,"15min":0.47499999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.84599999999999997,"5min":4.2539999999999996,"15min":4.2539999999999996},"last":0.51900000000000002}]},{"osd":1,"last update":"Fri Sep 13 18:21:38 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.373,"15min":0.35199999999999998},"min":{"1min":0.25800000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.59199999999999997,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.379,"15min":0.35399999999999998},"min":{"1min":0.29199999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.68400000000000005,"5min":0.748,"15min":0.748},"last":0.39200000000000002}]},{"osd":2,"last update":"Fri Sep 13 18:21:38 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.60699999999999998,"15min":0.54900000000000004},"min":{"1min":0.40999999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":5.2930000000000001,"5min":5.2930000000000001,"15min":5.2930000000000001},"last":0.438},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55300000000000005,"15min":0.53800000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72499999999999998,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.51900000000000002}]},{"osd":3,"last update":"Fri Sep 13 18:21:38 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56000000000000005,"15min":0.505},"min":{"1min":0.434,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.82199999999999995,"5min":4.2679999999999998,"15min":4.2679999999999998},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.52100000000000002,"15min":0.496},"min":{"1min":0.46100000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.41399999999999998}]},{"osd":5,"last update":"Fri Sep 13 18:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.57099999999999995,"15min":0.54300000000000004},"min":{"1min":0.434,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.70399999999999996,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.58099999999999996,"15min":0.54700000000000004},"min":{"1min":0.40400000000000003,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.75900000000000001,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.42499999999999999}]},{"osd":6,"last update":"Fri Sep 13 18:22:21 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56399999999999995,"15min":0.52400000000000002},"min":{"1min":0.40400000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.754,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.56699999999999995,"15min":0.52300000000000002},"min":{"1min":0.47799999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.73899999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.50800000000000001}]},{"osd":7,"last update":"Fri Sep 13 18:21:48 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.46999999999999997,"15min":0.44600000000000001},"min":{"1min":0.376,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.76100000000000001,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.50900000000000001,"15min":0.48499999999999999},"min":{"1min":0.379,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.76100000000000001,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.48399999999999999}]},{"osd":8,"last update":"Fri Sep 13 18:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.64200000000000002,"15min":0.63},"min":{"1min":0.54900000000000004,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.76600000000000001,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.374},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.67000000000000004,"15min":0.65200000000000002},"min":{"1min":0.624,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.76900000000000002,"5min":0.874,"15min":0.874},"last":0.57499999999999996}]},{"osd":9,"last update":"Fri Sep 13 18:21:28 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.63300000000000001,"15min":0.61499999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.63700000000000001,"15min":0.629},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.83299999999999996,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.44700000000000001}]},{"osd":10,"last update":"Fri Sep 13 18:21:52 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.63400000000000001},"min":{"1min":0.438,"5min":0.377,"15min":0.377},"max":{"1min":0.81200000000000006,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.499},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.66100000000000003,"15min":0.65900000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.746,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.45600000000000002}]},{"osd":11,"last update":"Fri Sep 13 18:22:04 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.73299999999999998,"15min":0.73399999999999999},"min":{"1min":0.58099999999999996,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.89300000000000002,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.72499999999999998,"15min":0.72199999999999998},"min":{"1min":0.66300000000000003,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.81399999999999995,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.54800000000000004}]}]},{"osd":3,"up_from":25,"seq":107374182479,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":524038,"data_compressed":26479,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:22:18 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34699999999999998,"15min":0.313},"min":{"1min":0.216,"5min":0.183,"15min":0.183},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.34599999999999997,"15min":0.33000000000000002},"min":{"1min":0.23200000000000001,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.48099999999999998,"5min":0.58199999999999996,"15min":0.58199999999999996},"last":0.54700000000000004}]},{"osd":1,"last update":"Fri Sep 13 18:22:18 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.53700000000000003,"15min":0.497},"min":{"1min":0.41799999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.68400000000000005,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.622},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.54700000000000004,"15min":0.499},"min":{"1min":0.443,"5min":0.33700000000000002,"15min":0.313},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.68100000000000005}]},{"osd":2,"last update":"Fri Sep 13 18:22:18 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52500000000000002,"15min":0.52300000000000002},"min":{"1min":0.42199999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.54500000000000004,"15min":0.50800000000000001},"min":{"1min":0.42699999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.748,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.66700000000000004}]},{"osd":4,"last update":"Fri Sep 13 18:21:38 2024","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":0.622,"15min":0.52600000000000002},"min":{"1min":0.40500000000000003,"5min":0.312,"15min":0.312},"max":{"1min":9.3149999999999995,"5min":9.3149999999999995,"15min":9.3149999999999995},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.53900000000000003,"15min":0.51900000000000002},"min":{"1min":0.39900000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.72399999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.69499999999999995}]},{"osd":5,"last update":"Fri Sep 13 18:21:56 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.54800000000000004,"15min":0.53800000000000003},"min":{"1min":0.435,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.74199999999999999,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.54500000000000004,"15min":0.55000000000000004},"min":{"1min":0.38900000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.74199999999999999,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.60999999999999999}]},{"osd":6,"last update":"Fri Sep 13 18:22:28 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.434,"15min":0.44400000000000001},"min":{"1min":0.32800000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.749,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.749},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.48699999999999999,"15min":0.46899999999999997},"min":{"1min":0.307,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.76000000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.76000000000000001}]},{"osd":7,"last update":"Fri Sep 13 18:21:48 2024","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":0.69199999999999995,"15min":0.626},"min":{"1min":0.45700000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":9.3330000000000002,"5min":9.3330000000000002,"15min":9.3330000000000002},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57999999999999996,"15min":0.57399999999999995},"min":{"1min":0.39000000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.67700000000000005,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.70499999999999996}]},{"osd":8,"last update":"Fri Sep 13 18:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.69599999999999995}]},{"osd":9,"last update":"Fri Sep 13 18:21:31 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.52500000000000002,"15min":0.54900000000000004},"min":{"1min":0.33800000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.59999999999999998,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.57099999999999995,"15min":0.58499999999999996},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.73899999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.81000000000000005}]},{"osd":10,"last update":"Fri Sep 13 18:21:52 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.66700000000000004,"15min":0.67900000000000005},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.77900000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.63900000000000001,"15min":0.66200000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.69199999999999995,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.91200000000000003}]},{"osd":11,"last update":"Fri Sep 13 18:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.61099999999999999,"15min":0.60599999999999998},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64800000000000002,"15min":0.64900000000000002},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.76400000000000001,"5min":0.872,"15min":0.872},"last":0.85699999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1122304,"data_stored":596184,"data_compressed":28087,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:21:50 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.49299999999999999,"15min":0.48499999999999999},"min":{"1min":0.40600000000000003,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.503,"15min":0.48299999999999998},"min":{"1min":0.39600000000000002,"5min":0.28799999999999998,"15min":0.26300000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.45800000000000002}]},{"osd":1,"last update":"Fri Sep 13 18:21:50 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.54000000000000004,"15min":0.48799999999999999},"min":{"1min":0.46300000000000002,"5min":0.318,"15min":0.27400000000000002},"max":{"1min":0.81299999999999994,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.56899999999999995,"15min":0.49099999999999999},"min":{"1min":0.45000000000000001,"5min":0.32300000000000001,"15min":0.29099999999999998},"max":{"1min":0.79000000000000004,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.48899999999999999}]},{"osd":3,"last update":"Fri Sep 13 18:22:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55100000000000005,"15min":0.52400000000000002},"min":{"1min":0.41299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58199999999999996,"15min":0.56100000000000005},"min":{"1min":0.40799999999999997,"5min":0.34000000000000002,"15min":0.33300000000000002},"max":{"1min":0.79700000000000004,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.56799999999999995}]},{"osd":4,"last update":"Fri Sep 13 18:22:04 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.62,"15min":0.61699999999999999},"min":{"1min":0.495,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.75900000000000001,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.66900000000000004,"15min":0.66600000000000004},"min":{"1min":0.56899999999999995,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.82199999999999995,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.437}]},{"osd":5,"last update":"Fri Sep 13 18:21:53 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.42699999999999999,"15min":0.38900000000000001},"min":{"1min":0.27100000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.314},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.44600000000000001,"15min":0.40500000000000003},"min":{"1min":0.32100000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.67800000000000005,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.35299999999999998}]},{"osd":6,"last update":"Fri Sep 13 18:22:25 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.621,"15min":0.55900000000000005},"min":{"1min":0.52900000000000003,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.85099999999999998,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.626,"15min":0.57899999999999996},"min":{"1min":0.47799999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.82999999999999996,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.52400000000000002}]},{"osd":7,"last update":"Fri Sep 13 18:22:09 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.68899999999999995,"15min":0.68799999999999994},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.85899999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.67000000000000004,"15min":0.66800000000000004},"min":{"1min":0.495,"5min":0.434,"15min":0.434},"max":{"1min":0.86099999999999999,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.46700000000000003}]},{"osd":8,"last update":"Fri Sep 13 18:22:09 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.36299999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.79900000000000004,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.37},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.61599999999999999,"15min":0.59399999999999997},"min":{"1min":0.45800000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.86699999999999999,"5min":0.998,"15min":0.998},"last":0.28999999999999998}]},{"osd":9,"last update":"Fri Sep 13 18:21:28 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.63700000000000001,"15min":0.625},"min":{"1min":0.50900000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.86399999999999999,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.5},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.66900000000000004,"15min":0.65500000000000003},"min":{"1min":0.50900000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.96499999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.55000000000000004}]},{"osd":10,"last update":"Fri Sep 13 18:21:50 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.67600000000000005,"15min":0.63700000000000001},"min":{"1min":0.63900000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.93999999999999995,"5min":1.016,"15min":1.016},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.68899999999999995,"15min":0.65300000000000002},"min":{"1min":0.623,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.93600000000000005,"5min":1.042,"15min":1.042},"last":0.50900000000000001}]},{"osd":11,"last update":"Fri Sep 13 18:22:00 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.72499999999999998,"15min":0.71599999999999997},"min":{"1min":0.626,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.94699999999999995,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.73099999999999998,"15min":0.72299999999999998},"min":{"1min":0.54100000000000004,"5min":0.433,"15min":0.433},"max":{"1min":0.92000000000000004,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.54000000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1101824,"data_stored":580429,"data_compressed":25120,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:22:25 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.502,"15min":0.49299999999999999},"min":{"1min":0.376,"5min":0.30299999999999999,"15min":0.28399999999999997},"max":{"1min":0.67100000000000004,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55000000000000004,"15min":0.52100000000000002},"min":{"1min":0.40600000000000003,"5min":0.33200000000000002,"15min":0.31},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.40600000000000003}]},{"osd":2,"last update":"Fri Sep 13 18:21:53 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.56799999999999995,"15min":0.51200000000000001},"min":{"1min":0.39200000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.433},{"interface":"front","average":{"1min":0.622,"5min":0.52000000000000002,"15min":0.48599999999999999},"min":{"1min":0.33800000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.38700000000000001}]},{"osd":3,"last update":"Fri Sep 13 18:22:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55900000000000005,"15min":0.52700000000000002},"min":{"1min":0.374,"5min":0.31,"15min":0.31},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.503},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.60699999999999998,"15min":0.55800000000000005},"min":{"1min":0.47399999999999998,"5min":0.39700000000000002,"15min":0.34399999999999997},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.57099999999999995}]},{"osd":4,"last update":"Fri Sep 13 18:21:34 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.41099999999999998,"15min":0.371},"min":{"1min":0.29499999999999998,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.755,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.40500000000000003,"15min":0.36499999999999999},"min":{"1min":0.29599999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.65600000000000003,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.44700000000000001}]},{"osd":5,"last update":"Fri Sep 13 18:21:57 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.63400000000000001,"15min":0.58399999999999996},"min":{"1min":0.41899999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.60699999999999998,"15min":0.55500000000000005},"min":{"1min":0.47199999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85799999999999998,"5min":0.875,"15min":0.875},"last":0.41699999999999998}]},{"osd":6,"last update":"Fri Sep 13 18:22:25 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.59699999999999998,"15min":0.57399999999999995},"min":{"1min":0.46100000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.65400000000000003,"15min":0.622},"min":{"1min":0.54100000000000004,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.54800000000000004}]},{"osd":7,"last update":"Fri Sep 13 18:21:40 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.51700000000000002,"15min":0.495},"min":{"1min":0.312,"5min":0.246,"15min":0.246},"max":{"1min":0.74299999999999999,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.52200000000000002,"15min":0.50700000000000001},"min":{"1min":0.35499999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.82499999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.58699999999999997}]},{"osd":8,"last update":"Fri Sep 13 18:22:11 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.65400000000000003,"15min":0.626},"min":{"1min":0.48499999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.90300000000000002,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.747,"5min":0.66700000000000004,"15min":0.64200000000000002},"min":{"1min":0.57299999999999995,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.56000000000000005}]},{"osd":9,"last update":"Fri Sep 13 18:21:34 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.63600000000000001,"15min":0.64200000000000002},"min":{"1min":0.49399999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.84399999999999997,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.64000000000000001,"15min":0.64200000000000002},"min":{"1min":0.433,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.61599999999999999}]},{"osd":10,"last update":"Fri Sep 13 18:21:53 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.63100000000000001,"15min":0.60299999999999998},"min":{"1min":0.442,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.63300000000000001,"15min":0.61499999999999999},"min":{"1min":0.39800000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.52400000000000002}]},{"osd":11,"last update":"Fri Sep 13 18:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.80200000000000005,"15min":0.79700000000000004},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.623},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.57999999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":1052672,"data_stored":516690,"data_compressed":23512,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 18:21:28 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.50700000000000001,"15min":0.498},"min":{"1min":0.36799999999999999,"5min":0.318,"15min":0.30199999999999999},"max":{"1min":0.76900000000000002,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.497},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.52100000000000002,"15min":0.50900000000000001},"min":{"1min":0.434,"5min":0.307,"15min":0.307},"max":{"1min":0.73999999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.33000000000000002}]},{"osd":2,"last update":"Fri Sep 13 18:21:49 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.53400000000000003,"15min":0.52900000000000003},"min":{"1min":0.40699999999999997,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.70699999999999996,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.623},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.55700000000000005,"15min":0.52000000000000002},"min":{"1min":0.42899999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.69699999999999995,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.437}]},{"osd":3,"last update":"Fri Sep 13 18:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.40999999999999998,"15min":0.39200000000000002},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":2.9750000000000001,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.38200000000000001,"15min":0.38500000000000001},"min":{"1min":0.26500000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.61299999999999999,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.41599999999999998}]},{"osd":4,"last update":"Fri Sep 13 18:21:39 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52100000000000002,"15min":0.497},"min":{"1min":0.41299999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.54100000000000004,"15min":0.51500000000000001},"min":{"1min":0.36399999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76000000000000001,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.378}]},{"osd":5,"last update":"Fri Sep 13 18:21:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.61399999999999999,"15min":0.56599999999999995},"min":{"1min":0.35199999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.68300000000000005,"5min":7.194,"15min":7.194},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.53300000000000003},"min":{"1min":0.44500000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.84999999999999998,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.54300000000000004}]},{"osd":6,"last update":"Fri Sep 13 18:22:24 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.47699999999999998,"15min":0.45900000000000002},"min":{"1min":0.33000000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.437,"15min":0.45400000000000001},"min":{"1min":0.316,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.73899999999999999,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.40100000000000002}]},{"osd":7,"last update":"Fri Sep 13 18:22:09 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.59699999999999998,"15min":0.59399999999999997},"min":{"1min":0.44900000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.68799999999999994}]},{"osd":8,"last update":"Fri Sep 13 18:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58999999999999997,"15min":0.60399999999999998},"min":{"1min":0.41899999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.61799999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.65700000000000003}]},{"osd":9,"last update":"Fri Sep 13 18:21:33 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.58499999999999996,"15min":0.60699999999999998},"min":{"1min":0.33700000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78200000000000003,"5min":1.089,"15min":1.089},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.54100000000000004,"15min":0.57299999999999995},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.61799999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.61599999999999999}]},{"osd":10,"last update":"Fri Sep 13 18:21:49 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.65900000000000003,"15min":0.67700000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88800000000000001,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.749},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.68300000000000005,"15min":0.69799999999999995},"min":{"1min":0.51900000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.84599999999999997,"5min":1.048,"15min":1.048},"last":0.63800000000000001}]},{"osd":11,"last update":"Fri Sep 13 18:22:20 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.623,"15min":0.63100000000000001},"min":{"1min":0.44600000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68100000000000005,"15min":0.68300000000000005},"min":{"1min":0.45900000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.71999999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35824,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35824,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35824,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":72245,"data_compressed":1608,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":72295,"data_compressed":1608,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135258,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":6440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":394694,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":203717,"data_compressed":7764,"data_compressed_allocated":12288,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:22:33.227 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-13T18:22:33.228 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:22:33.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.358+0000 7f3ea1a87640 1 -- 172.21.15.114:0/2986579558 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e9c10a320 msgr2=0x7f3e9c10c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:33.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.358+0000 7f3ea1a87640 1 --2- 172.21.15.114:0/2986579558 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e9c10a320 0x7f3e9c10c7c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f3e94009900 tx=0x7f3e9402a480 comp rx=0 tx=0).stop 2024-09-13T18:22:33.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.358+0000 7f3ea1a87640 1 -- 172.21.15.114:0/2986579558 shutdown_connections 2024-09-13T18:22:33.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.358+0000 7f3ea1a87640 1 --2- 172.21.15.114:0/2986579558 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e9c10a320 0x7f3e9c10c7c0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:33.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.358+0000 7f3ea1a87640 1 --2- 172.21.15.114:0/2986579558 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e9c1018b0 0x7f3e9c109d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:33.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.358+0000 7f3ea1a87640 1 --2- 172.21.15.114:0/2986579558 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e9c100f90 0x7f3e9c101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:33.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.358+0000 7f3ea1a87640 1 -- 172.21.15.114:0/2986579558 >> 172.21.15.114:0/2986579558 conn(0x7f3e9c0fa900 msgr2=0x7f3e9c0fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:33.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.358+0000 7f3ea1a87640 1 -- 172.21.15.114:0/2986579558 shutdown_connections 2024-09-13T18:22:33.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.358+0000 7f3ea1a87640 1 -- 172.21.15.114:0/2986579558 wait complete. 2024-09-13T18:22:33.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.359+0000 7f3ea1a87640 1 Processor -- start 2024-09-13T18:22:33.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.359+0000 7f3ea1a87640 1 -- start start 2024-09-13T18:22:33.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.359+0000 7f3ea1a87640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e9c100f90 0x7f3e9c1961b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:33.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.359+0000 7f3ea1a87640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e9c1018b0 0x7f3e9c1966f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:33.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.359+0000 7f3ea1a87640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e9c10a320 0x7f3e9c19aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:33.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.359+0000 7f3ea1a87640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9c19b0e0 con 0x7f3e9c1018b0 2024-09-13T18:22:33.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.359+0000 7f3ea1a87640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9c19b250 con 0x7f3e9c10a320 2024-09-13T18:22:33.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.359+0000 7f3ea1a87640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9c19b410 con 0x7f3e9c100f90 2024-09-13T18:22:33.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.359+0000 7f3e9affd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e9c1018b0 0x7f3e9c1966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:33.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.359+0000 7f3e9affd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e9c1018b0 0x7f3e9c1966f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50806/0 (socket says 172.21.15.114:50806) 2024-09-13T18:22:33.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.359+0000 7f3e9affd640 1 -- 172.21.15.114:0/2397506715 learned_addr learned my addr 172.21.15.114:0/2397506715 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:33.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.360+0000 7f3e9b7fe640 1 --2- 172.21.15.114:0/2397506715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e9c100f90 0x7f3e9c1961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:33.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.360+0000 7f3e9affd640 1 -- 172.21.15.114:0/2397506715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e9c100f90 msgr2=0x7f3e9c1961b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:33.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.360+0000 7f3e9affd640 1 --2- 172.21.15.114:0/2397506715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e9c100f90 0x7f3e9c1961b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:33.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.360+0000 7f3e9bfff640 1 --2- 172.21.15.114:0/2397506715 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e9c10a320 0x7f3e9c19aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:33.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.360+0000 7f3e9affd640 1 -- 172.21.15.114:0/2397506715 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e9c10a320 msgr2=0x7f3e9c19aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:33.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.360+0000 7f3e9affd640 1 --2- 172.21.15.114:0/2397506715 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e9c10a320 0x7f3e9c19aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:33.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.360+0000 7f3e9affd640 1 -- 172.21.15.114:0/2397506715 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e94009560 con 0x7f3e9c1018b0 2024-09-13T18:22:33.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.360+0000 7f3e9bfff640 1 --2- 172.21.15.114:0/2397506715 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e9c10a320 0x7f3e9c19aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:22:33.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.360+0000 7f3e9affd640 1 --2- 172.21.15.114:0/2397506715 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e9c1018b0 0x7f3e9c1966f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f3e9000c890 tx=0x7f3e9000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:33.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.360+0000 7f3e98ff9640 1 -- 172.21.15.114:0/2397506715 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e90004220 con 0x7f3e9c1018b0 2024-09-13T18:22:33.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.360+0000 7f3e98ff9640 1 -- 172.21.15.114:0/2397506715 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3e90007e40 con 0x7f3e9c1018b0 2024-09-13T18:22:33.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.360+0000 7f3ea1a87640 1 -- 172.21.15.114:0/2397506715 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e9c19b6f0 con 0x7f3e9c1018b0 2024-09-13T18:22:33.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.360+0000 7f3e98ff9640 1 -- 172.21.15.114:0/2397506715 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e90004a00 con 0x7f3e9c1018b0 2024-09-13T18:22:33.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.360+0000 7f3ea1a87640 1 -- 172.21.15.114:0/2397506715 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e9c1a2f70 con 0x7f3e9c1018b0 2024-09-13T18:22:33.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.361+0000 7f3ea1a87640 1 -- 172.21.15.114:0/2397506715 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e9c068e30 con 0x7f3e9c1018b0 2024-09-13T18:22:33.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.362+0000 7f3e98ff9640 1 -- 172.21.15.114:0/2397506715 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 24) v1 ==== 136050+0+0 (secure 0 0 0) 0x7f3e900079b0 con 0x7f3e9c1018b0 2024-09-13T18:22:33.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.363+0000 7f3e98ff9640 1 --2- 172.21.15.114:0/2397506715 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f3e840a1310 0x7f3e840a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:33.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.363+0000 7f3e9b7fe640 1 --2- 172.21.15.114:0/2397506715 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f3e840a1310 0x7f3e840a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:33.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.363+0000 7f3e98ff9640 1 -- 172.21.15.114:0/2397506715 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f3e900cd1e0 con 0x7f3e9c1018b0 2024-09-13T18:22:33.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.363+0000 7f3e9b7fe640 1 --2- 172.21.15.114:0/2397506715 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f3e840a1310 0x7f3e840a37d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c005c40 tx=0x7f3e8c00a390 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:33.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:33 smithi134 ceph-mon[34283]: Updating smithi114:/etc/ceph/ceph.client.admin.keyring 2024-09-13T18:22:33.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:33 smithi134 ceph-mon[34283]: Updating smithi134:/etc/ceph/ceph.client.admin.keyring 2024-09-13T18:22:33.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:33 smithi134 ceph-mon[34283]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-09-13T18:22:33.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:33 smithi134 ceph-mon[34283]: Updating smithi114:/var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-13T18:22:33.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:33 smithi134 ceph-mon[34283]: Updating smithi134:/var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-13T18:22:33.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:33 smithi134 ceph-mon[34283]: Updating smithi204:/var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-13T18:22:33.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:33 smithi134 ceph-mon[34283]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:22:33.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:33 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/2617702121' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T18:22:33.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:33 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/1850740749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:22:33.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:33 smithi134 ceph-mon[34283]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-09-13T18:22:33.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.367+0000 7f3e98ff9640 1 -- 172.21.15.114:0/2397506715 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f3e90096170 con 0x7f3e9c1018b0 2024-09-13T18:22:33.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:33 smithi204 ceph-mon[34777]: Updating smithi114:/etc/ceph/ceph.client.admin.keyring 2024-09-13T18:22:33.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:33 smithi204 ceph-mon[34777]: Updating smithi134:/etc/ceph/ceph.client.admin.keyring 2024-09-13T18:22:33.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:33 smithi204 ceph-mon[34777]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-09-13T18:22:33.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:33 smithi204 ceph-mon[34777]: Updating smithi114:/var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-13T18:22:33.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:33 smithi204 ceph-mon[34777]: Updating smithi134:/var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-13T18:22:33.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:33 smithi204 ceph-mon[34777]: Updating smithi204:/var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-13T18:22:33.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:33 smithi204 ceph-mon[34777]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:22:33.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:33 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/2617702121' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T18:22:33.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:33 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/1850740749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:22:33.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:33 smithi204 ceph-mon[34777]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-09-13T18:22:33.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.526+0000 7f3ea1a87640 1 -- 172.21.15.114:0/2397506715 --> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3e9c10fc30 con 0x7f3e840a1310 2024-09-13T18:22:33.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.537+0000 7f3e98ff9640 1 -- 172.21.15.114:0/2397506715 <== mgr.15555 v2:172.21.15.114:6800/2440721366 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427785 (secure 0 0 0) 0x7f3e9c10fc30 con 0x7f3e840a1310 2024-09-13T18:22:33.548 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:22:33.551 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-13T18:22:33.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.540+0000 7f3ea1a87640 1 -- 172.21.15.114:0/2397506715 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f3e840a1310 msgr2=0x7f3e840a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:33.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.540+0000 7f3ea1a87640 1 --2- 172.21.15.114:0/2397506715 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f3e840a1310 0x7f3e840a37d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c005c40 tx=0x7f3e8c00a390 comp rx=0 tx=0).stop 2024-09-13T18:22:33.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.540+0000 7f3ea1a87640 1 -- 172.21.15.114:0/2397506715 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e9c1018b0 msgr2=0x7f3e9c1966f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:33.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.540+0000 7f3ea1a87640 1 --2- 172.21.15.114:0/2397506715 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e9c1018b0 0x7f3e9c1966f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f3e9000c890 tx=0x7f3e9000cd50 comp rx=0 tx=0).stop 2024-09-13T18:22:33.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.540+0000 7f3ea1a87640 1 -- 172.21.15.114:0/2397506715 shutdown_connections 2024-09-13T18:22:33.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.540+0000 7f3ea1a87640 1 --2- 172.21.15.114:0/2397506715 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f3e840a1310 0x7f3e840a37d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:33.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.540+0000 7f3ea1a87640 1 --2- 172.21.15.114:0/2397506715 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e9c10a320 0x7f3e9c19aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:33.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.540+0000 7f3ea1a87640 1 --2- 172.21.15.114:0/2397506715 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e9c1018b0 0x7f3e9c1966f0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:33.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.540+0000 7f3ea1a87640 1 --2- 172.21.15.114:0/2397506715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e9c100f90 0x7f3e9c1961b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:33.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.541+0000 7f3ea1a87640 1 -- 172.21.15.114:0/2397506715 >> 172.21.15.114:0/2397506715 conn(0x7f3e9c0fa900 msgr2=0x7f3e9c0fc200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:33.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.541+0000 7f3ea1a87640 1 -- 172.21.15.114:0/2397506715 shutdown_connections 2024-09-13T18:22:33.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.541+0000 7f3ea1a87640 1 -- 172.21.15.114:0/2397506715 wait complete. 2024-09-13T18:22:33.554 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-09-13T18:22:32.078907+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1194735,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1483,"num_read_kb":912,"num_write":548,"num_write_kb":4407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180875,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":938,"ondisk_log_size":938,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361600,"kb_used_data":12792,"kb_used_omap":18,"kb_used_meta":348653,"kb_avail":1124580224,"statfs":{"total":1151940427776,"available":1151570149376,"internally_reserved":0,"allocated":13099008,"data_stored":9351983,"data_compressed":474714,"data_compressed_allocated":647168,"data_compressed_original":3694592,"omap_allocated":19129,"internal_metadata":357020999},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.988041"},"pg_stats":[{"pgid":"2.1f","version":"93'14","reported_seq":157,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.139188+0000","last_change":"2024-09-13T18:20:01.577029+0000","last_active":"2024-09-13T18:22:28.139188+0000","last_peered":"2024-09-13T18:22:28.139188+0000","last_clean":"2024-09-13T18:22:28.139188+0000","last_became_active":"2024-09-13T18:20:01.576710+0000","last_became_peered":"2024-09-13T18:20:01.576710+0000","last_unstale":"2024-09-13T18:22:28.139188+0000","last_undegraded":"2024-09-13T18:22:28.139188+0000","last_fullsized":"2024-09-13T18:22:28.139188+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:25:11.977436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"93'5","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:03.100355+0000","last_change":"2024-09-13T18:20:01.922955+0000","last_active":"2024-09-13T18:22:03.100355+0000","last_peered":"2024-09-13T18:22:03.100355+0000","last_clean":"2024-09-13T18:22:03.100355+0000","last_became_active":"2024-09-13T18:20:01.922704+0000","last_became_peered":"2024-09-13T18:20:01.922704+0000","last_unstale":"2024-09-13T18:22:03.100355+0000","last_undegraded":"2024-09-13T18:22:03.100355+0000","last_fullsized":"2024-09-13T18:22:03.100355+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:44:20.672784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:28.310843+0000","last_change":"2024-09-13T18:20:27.300985+0000","last_active":"2024-09-13T18:21:28.310843+0000","last_peered":"2024-09-13T18:21:28.310843+0000","last_clean":"2024-09-13T18:21:28.310843+0000","last_became_active":"2024-09-13T18:20:27.300546+0000","last_became_peered":"2024-09-13T18:20:27.300546+0000","last_unstale":"2024-09-13T18:21:28.310843+0000","last_undegraded":"2024-09-13T18:21:28.310843+0000","last_fullsized":"2024-09-13T18:21:28.310843+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:00:11.750929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:03.100379+0000","last_change":"2024-09-13T18:20:01.578921+0000","last_active":"2024-09-13T18:22:03.100379+0000","last_peered":"2024-09-13T18:22:03.100379+0000","last_clean":"2024-09-13T18:22:03.100379+0000","last_became_active":"2024-09-13T18:20:01.578627+0000","last_became_peered":"2024-09-13T18:20:01.578627+0000","last_unstale":"2024-09-13T18:22:03.100379+0000","last_undegraded":"2024-09-13T18:22:03.100379+0000","last_fullsized":"2024-09-13T18:22:03.100379+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:43:32.919156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"93'6","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:07.284848+0000","last_change":"2024-09-13T18:20:01.922901+0000","last_active":"2024-09-13T18:22:07.284848+0000","last_peered":"2024-09-13T18:22:07.284848+0000","last_clean":"2024-09-13T18:22:07.284848+0000","last_became_active":"2024-09-13T18:20:01.922604+0000","last_became_peered":"2024-09-13T18:20:01.922604+0000","last_unstale":"2024-09-13T18:22:07.284848+0000","last_undegraded":"2024-09-13T18:22:07.284848+0000","last_fullsized":"2024-09-13T18:22:07.284848+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:27:30.650010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:28.151945+0000","last_change":"2024-09-13T18:20:06.135406+0000","last_active":"2024-09-13T18:21:28.151945+0000","last_peered":"2024-09-13T18:21:28.151945+0000","last_clean":"2024-09-13T18:21:28.151945+0000","last_became_active":"2024-09-13T18:20:06.134558+0000","last_became_peered":"2024-09-13T18:20:06.134558+0000","last_unstale":"2024-09-13T18:21:28.151945+0000","last_undegraded":"2024-09-13T18:21:28.151945+0000","last_fullsized":"2024-09-13T18:21:28.151945+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:10.901966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'12","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:28.153481+0000","last_change":"2024-09-13T18:20:01.580040+0000","last_active":"2024-09-13T18:21:28.153481+0000","last_peered":"2024-09-13T18:21:28.153481+0000","last_clean":"2024-09-13T18:21:28.153481+0000","last_became_active":"2024-09-13T18:20:01.579715+0000","last_became_peered":"2024-09-13T18:20:01.579715+0000","last_unstale":"2024-09-13T18:21:28.153481+0000","last_undegraded":"2024-09-13T18:21:28.153481+0000","last_fullsized":"2024-09-13T18:21:28.153481+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:57:03.588169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"93'9","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:55.925782+0000","last_change":"2024-09-13T18:20:01.928314+0000","last_active":"2024-09-13T18:21:55.925782+0000","last_peered":"2024-09-13T18:21:55.925782+0000","last_clean":"2024-09-13T18:21:55.925782+0000","last_became_active":"2024-09-13T18:20:01.925356+0000","last_became_peered":"2024-09-13T18:20:01.925356+0000","last_unstale":"2024-09-13T18:21:55.925782+0000","last_undegraded":"2024-09-13T18:21:55.925782+0000","last_fullsized":"2024-09-13T18:21:55.925782+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:43.906842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:28.152198+0000","last_change":"2024-09-13T18:20:06.135886+0000","last_active":"2024-09-13T18:21:28.152198+0000","last_peered":"2024-09-13T18:21:28.152198+0000","last_clean":"2024-09-13T18:21:28.152198+0000","last_became_active":"2024-09-13T18:20:06.134587+0000","last_became_peered":"2024-09-13T18:20:06.134587+0000","last_unstale":"2024-09-13T18:21:28.152198+0000","last_undegraded":"2024-09-13T18:21:28.152198+0000","last_fullsized":"2024-09-13T18:21:28.152198+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:44:42.434729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"91'5","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.140618+0000","last_change":"2024-09-13T18:20:01.576908+0000","last_active":"2024-09-13T18:22:28.140618+0000","last_peered":"2024-09-13T18:22:28.140618+0000","last_clean":"2024-09-13T18:22:28.140618+0000","last_became_active":"2024-09-13T18:20:01.576478+0000","last_became_peered":"2024-09-13T18:20:01.576478+0000","last_unstale":"2024-09-13T18:22:28.140618+0000","last_undegraded":"2024-09-13T18:22:28.140618+0000","last_fullsized":"2024-09-13T18:22:28.140618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:34:52.126901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"93'15","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:55.923587+0000","last_change":"2024-09-13T18:20:01.921960+0000","last_active":"2024-09-13T18:21:55.923587+0000","last_peered":"2024-09-13T18:21:55.923587+0000","last_clean":"2024-09-13T18:21:55.923587+0000","last_became_active":"2024-09-13T18:20:01.921652+0000","last_became_peered":"2024-09-13T18:20:01.921652+0000","last_unstale":"2024-09-13T18:21:55.923587+0000","last_undegraded":"2024-09-13T18:21:55.923587+0000","last_fullsized":"2024-09-13T18:21:55.923587+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:00:31.277613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:28.152004+0000","last_change":"2024-09-13T18:20:06.138983+0000","last_active":"2024-09-13T18:21:28.152004+0000","last_peered":"2024-09-13T18:21:28.152004+0000","last_clean":"2024-09-13T18:21:28.152004+0000","last_became_active":"2024-09-13T18:20:06.138708+0000","last_became_peered":"2024-09-13T18:20:06.138708+0000","last_unstale":"2024-09-13T18:21:28.152004+0000","last_undegraded":"2024-09-13T18:21:28.152004+0000","last_fullsized":"2024-09-13T18:21:28.152004+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:45:21.978217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"93'5","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:16.192564+0000","last_change":"2024-09-13T18:20:01.929817+0000","last_active":"2024-09-13T18:22:16.192564+0000","last_peered":"2024-09-13T18:22:16.192564+0000","last_clean":"2024-09-13T18:22:16.192564+0000","last_became_active":"2024-09-13T18:20:01.928907+0000","last_became_peered":"2024-09-13T18:20:01.928907+0000","last_unstale":"2024-09-13T18:22:16.192564+0000","last_undegraded":"2024-09-13T18:22:16.192564+0000","last_fullsized":"2024-09-13T18:22:16.192564+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:56:25.847230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:15.988721+0000","last_change":"2024-09-13T18:20:01.578968+0000","last_active":"2024-09-13T18:22:15.988721+0000","last_peered":"2024-09-13T18:22:15.988721+0000","last_clean":"2024-09-13T18:22:15.988721+0000","last_became_active":"2024-09-13T18:20:01.578623+0000","last_became_peered":"2024-09-13T18:20:01.578623+0000","last_unstale":"2024-09-13T18:22:15.988721+0000","last_undegraded":"2024-09-13T18:22:15.988721+0000","last_fullsized":"2024-09-13T18:22:15.988721+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:03:30.333107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:21.739811+0000","last_change":"2024-09-13T18:20:06.135977+0000","last_active":"2024-09-13T18:22:21.739811+0000","last_peered":"2024-09-13T18:22:21.739811+0000","last_clean":"2024-09-13T18:22:21.739811+0000","last_became_active":"2024-09-13T18:20:06.134844+0000","last_became_peered":"2024-09-13T18:20:06.134844+0000","last_unstale":"2024-09-13T18:22:21.739811+0000","last_undegraded":"2024-09-13T18:22:21.739811+0000","last_fullsized":"2024-09-13T18:22:21.739811+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:58:36.139240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"93'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:18.101943+0000","last_change":"2024-09-13T18:20:01.916675+0000","last_active":"2024-09-13T18:22:18.101943+0000","last_peered":"2024-09-13T18:22:18.101943+0000","last_clean":"2024-09-13T18:22:18.101943+0000","last_became_active":"2024-09-13T18:20:01.912902+0000","last_became_peered":"2024-09-13T18:20:01.912902+0000","last_unstale":"2024-09-13T18:22:18.101943+0000","last_undegraded":"2024-09-13T18:22:18.101943+0000","last_fullsized":"2024-09-13T18:22:18.101943+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:24:54.271569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"91'3","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:28.154557+0000","last_change":"2024-09-13T18:20:27.300845+0000","last_active":"2024-09-13T18:21:28.154557+0000","last_peered":"2024-09-13T18:21:28.154557+0000","last_clean":"2024-09-13T18:21:28.154557+0000","last_became_active":"2024-09-13T18:20:27.300417+0000","last_became_peered":"2024-09-13T18:20:27.300417+0000","last_unstale":"2024-09-13T18:21:28.154557+0000","last_undegraded":"2024-09-13T18:21:28.154557+0000","last_fullsized":"2024-09-13T18:21:28.154557+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:44:27.507140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:21.739768+0000","last_change":"2024-09-13T18:21:28.235707+0000","last_active":"2024-09-13T18:22:21.739768+0000","last_peered":"2024-09-13T18:22:21.739768+0000","last_clean":"2024-09-13T18:22:21.739768+0000","last_became_active":"2024-09-13T18:21:28.231794+0000","last_became_peered":"2024-09-13T18:21:28.231794+0000","last_unstale":"2024-09-13T18:22:21.739768+0000","last_undegraded":"2024-09-13T18:22:21.739768+0000","last_fullsized":"2024-09-13T18:22:21.739768+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:01:00.113821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"93'9","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:12.959281+0000","last_change":"2024-09-13T18:20:01.913634+0000","last_active":"2024-09-13T18:22:12.959281+0000","last_peered":"2024-09-13T18:22:12.959281+0000","last_clean":"2024-09-13T18:22:12.959281+0000","last_became_active":"2024-09-13T18:20:01.913128+0000","last_became_peered":"2024-09-13T18:20:01.913128+0000","last_unstale":"2024-09-13T18:22:12.959281+0000","last_undegraded":"2024-09-13T18:22:12.959281+0000","last_fullsized":"2024-09-13T18:22:12.959281+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:37:15.251756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"91'7","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:13.101465+0000","last_change":"2024-09-13T18:20:01.579385+0000","last_active":"2024-09-13T18:22:13.101465+0000","last_peered":"2024-09-13T18:22:13.101465+0000","last_clean":"2024-09-13T18:22:13.101465+0000","last_became_active":"2024-09-13T18:20:01.579102+0000","last_became_peered":"2024-09-13T18:20:01.579102+0000","last_unstale":"2024-09-13T18:22:13.101465+0000","last_undegraded":"2024-09-13T18:22:13.101465+0000","last_fullsized":"2024-09-13T18:22:13.101465+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:22:00.432052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.138888+0000","last_change":"2024-09-13T18:20:06.138110+0000","last_active":"2024-09-13T18:22:28.138888+0000","last_peered":"2024-09-13T18:22:28.138888+0000","last_clean":"2024-09-13T18:22:28.138888+0000","last_became_active":"2024-09-13T18:20:06.137763+0000","last_became_peered":"2024-09-13T18:20:06.137763+0000","last_unstale":"2024-09-13T18:22:28.138888+0000","last_undegraded":"2024-09-13T18:22:28.138888+0000","last_fullsized":"2024-09-13T18:22:28.138888+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:16:15.956216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"93'2","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:15.988679+0000","last_change":"2024-09-13T18:20:01.918318+0000","last_active":"2024-09-13T18:22:15.988679+0000","last_peered":"2024-09-13T18:22:15.988679+0000","last_clean":"2024-09-13T18:22:15.988679+0000","last_became_active":"2024-09-13T18:20:01.917977+0000","last_became_peered":"2024-09-13T18:20:01.917977+0000","last_unstale":"2024-09-13T18:22:15.988679+0000","last_undegraded":"2024-09-13T18:22:15.988679+0000","last_fullsized":"2024-09-13T18:22:15.988679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:23.368866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:13.276327+0000","last_change":"2024-09-13T18:20:01.572009+0000","last_active":"2024-09-13T18:22:13.276327+0000","last_peered":"2024-09-13T18:22:13.276327+0000","last_clean":"2024-09-13T18:22:13.276327+0000","last_became_active":"2024-09-13T18:20:01.571313+0000","last_became_peered":"2024-09-13T18:20:01.571313+0000","last_unstale":"2024-09-13T18:22:13.276327+0000","last_undegraded":"2024-09-13T18:22:13.276327+0000","last_fullsized":"2024-09-13T18:22:13.276327+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:32:59.340476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:21.739824+0000","last_change":"2024-09-13T18:20:06.132212+0000","last_active":"2024-09-13T18:22:21.739824+0000","last_peered":"2024-09-13T18:22:21.739824+0000","last_clean":"2024-09-13T18:22:21.739824+0000","last_became_active":"2024-09-13T18:20:06.131809+0000","last_became_peered":"2024-09-13T18:20:06.131809+0000","last_unstale":"2024-09-13T18:22:21.739824+0000","last_undegraded":"2024-09-13T18:22:21.739824+0000","last_fullsized":"2024-09-13T18:22:21.739824+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:18:11.667551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"93'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:12.285278+0000","last_change":"2024-09-13T18:20:27.301112+0000","last_active":"2024-09-13T18:22:12.285278+0000","last_peered":"2024-09-13T18:22:12.285278+0000","last_clean":"2024-09-13T18:22:12.285278+0000","last_became_active":"2024-09-13T18:20:27.300772+0000","last_became_peered":"2024-09-13T18:20:27.300772+0000","last_unstale":"2024-09-13T18:22:12.285278+0000","last_undegraded":"2024-09-13T18:22:12.285278+0000","last_fullsized":"2024-09-13T18:22:12.285278+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:59:57.606105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"91'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:11.738288+0000","last_change":"2024-09-13T18:20:01.572025+0000","last_active":"2024-09-13T18:22:11.738288+0000","last_peered":"2024-09-13T18:22:11.738288+0000","last_clean":"2024-09-13T18:22:11.738288+0000","last_became_active":"2024-09-13T18:20:01.571321+0000","last_became_peered":"2024-09-13T18:20:01.571321+0000","last_unstale":"2024-09-13T18:22:11.738288+0000","last_undegraded":"2024-09-13T18:22:11.738288+0000","last_fullsized":"2024-09-13T18:22:11.738288+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:09.374364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:10.788550+0000","last_change":"2024-09-13T18:20:06.130687+0000","last_active":"2024-09-13T18:22:10.788550+0000","last_peered":"2024-09-13T18:22:10.788550+0000","last_clean":"2024-09-13T18:22:10.788550+0000","last_became_active":"2024-09-13T18:20:06.129839+0000","last_became_peered":"2024-09-13T18:20:06.129839+0000","last_unstale":"2024-09-13T18:22:10.788550+0000","last_undegraded":"2024-09-13T18:22:10.788550+0000","last_fullsized":"2024-09-13T18:22:10.788550+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:12:52.334945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"93'5","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:13.101520+0000","last_change":"2024-09-13T18:20:01.917244+0000","last_active":"2024-09-13T18:22:13.101520+0000","last_peered":"2024-09-13T18:22:13.101520+0000","last_clean":"2024-09-13T18:22:13.101520+0000","last_became_active":"2024-09-13T18:20:01.916948+0000","last_became_peered":"2024-09-13T18:20:01.916948+0000","last_unstale":"2024-09-13T18:22:13.101520+0000","last_undegraded":"2024-09-13T18:22:13.101520+0000","last_fullsized":"2024-09-13T18:22:13.101520+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:17:11.967096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'44","reported_seq":89,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.252535+0000","last_change":"2024-09-13T18:20:01.569123+0000","last_active":"2024-09-13T18:22:28.252535+0000","last_peered":"2024-09-13T18:22:28.252535+0000","last_clean":"2024-09-13T18:22:28.252535+0000","last_became_active":"2024-09-13T18:20:01.568720+0000","last_became_peered":"2024-09-13T18:20:01.568720+0000","last_unstale":"2024-09-13T18:22:28.252535+0000","last_undegraded":"2024-09-13T18:22:28.252535+0000","last_fullsized":"2024-09-13T18:22:28.252535+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:04:29.561013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":76,"num_write_kb":833,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:08.100974+0000","last_change":"2024-09-13T18:20:06.141771+0000","last_active":"2024-09-13T18:22:08.100974+0000","last_peered":"2024-09-13T18:22:08.100974+0000","last_clean":"2024-09-13T18:22:08.100974+0000","last_became_active":"2024-09-13T18:20:06.141362+0000","last_became_peered":"2024-09-13T18:20:06.141362+0000","last_unstale":"2024-09-13T18:22:08.100974+0000","last_undegraded":"2024-09-13T18:22:08.100974+0000","last_fullsized":"2024-09-13T18:22:08.100974+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:09:58.597605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"93'9","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:08.482216+0000","last_change":"2024-09-13T18:20:01.911048+0000","last_active":"2024-09-13T18:22:08.482216+0000","last_peered":"2024-09-13T18:22:08.482216+0000","last_clean":"2024-09-13T18:22:08.482216+0000","last_became_active":"2024-09-13T18:20:01.910735+0000","last_became_peered":"2024-09-13T18:20:01.910735+0000","last_unstale":"2024-09-13T18:22:08.482216+0000","last_undegraded":"2024-09-13T18:22:08.482216+0000","last_fullsized":"2024-09-13T18:22:08.482216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:07:35.447676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"91'4","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:05.987451+0000","last_change":"2024-09-13T18:20:01.571627+0000","last_active":"2024-09-13T18:22:05.987451+0000","last_peered":"2024-09-13T18:22:05.987451+0000","last_clean":"2024-09-13T18:22:05.987451+0000","last_became_active":"2024-09-13T18:20:01.571250+0000","last_became_peered":"2024-09-13T18:20:01.571250+0000","last_unstale":"2024-09-13T18:22:05.987451+0000","last_undegraded":"2024-09-13T18:22:05.987451+0000","last_fullsized":"2024-09-13T18:22:05.987451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:34:03.479324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:12.959308+0000","last_change":"2024-09-13T18:20:06.132974+0000","last_active":"2024-09-13T18:22:12.959308+0000","last_peered":"2024-09-13T18:22:12.959308+0000","last_clean":"2024-09-13T18:22:12.959308+0000","last_became_active":"2024-09-13T18:20:06.131668+0000","last_became_peered":"2024-09-13T18:20:06.131668+0000","last_unstale":"2024-09-13T18:22:12.959308+0000","last_undegraded":"2024-09-13T18:22:12.959308+0000","last_fullsized":"2024-09-13T18:22:12.959308+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:36:30.329761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"93'4","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:08.478967+0000","last_change":"2024-09-13T18:20:01.904670+0000","last_active":"2024-09-13T18:22:08.478967+0000","last_peered":"2024-09-13T18:22:08.478967+0000","last_clean":"2024-09-13T18:22:08.478967+0000","last_became_active":"2024-09-13T18:20:01.904316+0000","last_became_peered":"2024-09-13T18:20:01.904316+0000","last_unstale":"2024-09-13T18:22:08.478967+0000","last_undegraded":"2024-09-13T18:22:08.478967+0000","last_fullsized":"2024-09-13T18:22:08.478967+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:08:17.687179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"91'1","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:09.591727+0000","last_change":"2024-09-13T18:20:01.564750+0000","last_active":"2024-09-13T18:22:09.591727+0000","last_peered":"2024-09-13T18:22:09.591727+0000","last_clean":"2024-09-13T18:22:09.591727+0000","last_became_active":"2024-09-13T18:20:01.564096+0000","last_became_peered":"2024-09-13T18:20:01.564096+0000","last_unstale":"2024-09-13T18:22:09.591727+0000","last_undegraded":"2024-09-13T18:22:09.591727+0000","last_fullsized":"2024-09-13T18:22:09.591727+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:42:19.588824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:10.988064+0000","last_change":"2024-09-13T18:20:06.133386+0000","last_active":"2024-09-13T18:22:10.988064+0000","last_peered":"2024-09-13T18:22:10.988064+0000","last_clean":"2024-09-13T18:22:10.988064+0000","last_became_active":"2024-09-13T18:20:06.133004+0000","last_became_peered":"2024-09-13T18:20:06.133004+0000","last_unstale":"2024-09-13T18:22:10.988064+0000","last_undegraded":"2024-09-13T18:22:10.988064+0000","last_fullsized":"2024-09-13T18:22:10.988064+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:54:46.440410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"93'7","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:08.478947+0000","last_change":"2024-09-13T18:20:01.912823+0000","last_active":"2024-09-13T18:22:08.478947+0000","last_peered":"2024-09-13T18:22:08.478947+0000","last_clean":"2024-09-13T18:22:08.478947+0000","last_became_active":"2024-09-13T18:20:01.912123+0000","last_became_peered":"2024-09-13T18:20:01.912123+0000","last_unstale":"2024-09-13T18:22:08.478947+0000","last_undegraded":"2024-09-13T18:22:08.478947+0000","last_fullsized":"2024-09-13T18:22:08.478947+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:32:33.544816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"94'8","reported_seq":153,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.424067+0000","last_change":"2024-09-13T18:20:01.577095+0000","last_active":"2024-09-13T18:22:28.424067+0000","last_peered":"2024-09-13T18:22:28.424067+0000","last_clean":"2024-09-13T18:22:28.424067+0000","last_became_active":"2024-09-13T18:20:01.576765+0000","last_became_peered":"2024-09-13T18:20:01.576765+0000","last_unstale":"2024-09-13T18:22:28.424067+0000","last_undegraded":"2024-09-13T18:22:28.424067+0000","last_fullsized":"2024-09-13T18:22:28.424067+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:44:08.750441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1818,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":123,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:13.482840+0000","last_change":"2024-09-13T18:20:06.131650+0000","last_active":"2024-09-13T18:22:13.482840+0000","last_peered":"2024-09-13T18:22:13.482840+0000","last_clean":"2024-09-13T18:22:13.482840+0000","last_became_active":"2024-09-13T18:20:06.130902+0000","last_became_peered":"2024-09-13T18:20:06.130902+0000","last_unstale":"2024-09-13T18:22:13.482840+0000","last_undegraded":"2024-09-13T18:22:13.482840+0000","last_fullsized":"2024-09-13T18:22:13.482840+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.366694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"93'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:06.033866+0000","last_change":"2024-09-13T18:20:01.918221+0000","last_active":"2024-09-13T18:22:06.033866+0000","last_peered":"2024-09-13T18:22:06.033866+0000","last_clean":"2024-09-13T18:22:06.033866+0000","last_became_active":"2024-09-13T18:20:01.917134+0000","last_became_peered":"2024-09-13T18:20:01.917134+0000","last_unstale":"2024-09-13T18:22:06.033866+0000","last_undegraded":"2024-09-13T18:22:06.033866+0000","last_fullsized":"2024-09-13T18:22:06.033866+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:19:06.269454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'6","reported_seq":150,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.292167+0000","last_change":"2024-09-13T18:20:01.572258+0000","last_active":"2024-09-13T18:22:28.292167+0000","last_peered":"2024-09-13T18:22:28.292167+0000","last_clean":"2024-09-13T18:22:28.292167+0000","last_became_active":"2024-09-13T18:20:01.571968+0000","last_became_peered":"2024-09-13T18:20:01.571968+0000","last_unstale":"2024-09-13T18:22:28.292167+0000","last_undegraded":"2024-09-13T18:22:28.292167+0000","last_fullsized":"2024-09-13T18:22:28.292167+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:40:59.143123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":122,"num_read_kb":122,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:18.483349+0000","last_change":"2024-09-13T18:20:06.131714+0000","last_active":"2024-09-13T18:22:18.483349+0000","last_peered":"2024-09-13T18:22:18.483349+0000","last_clean":"2024-09-13T18:22:18.483349+0000","last_became_active":"2024-09-13T18:20:06.131244+0000","last_became_peered":"2024-09-13T18:20:06.131244+0000","last_unstale":"2024-09-13T18:22:18.483349+0000","last_undegraded":"2024-09-13T18:22:18.483349+0000","last_fullsized":"2024-09-13T18:22:18.483349+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:37.946002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"93'5","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:08.482116+0000","last_change":"2024-09-13T18:20:01.917351+0000","last_active":"2024-09-13T18:22:08.482116+0000","last_peered":"2024-09-13T18:22:08.482116+0000","last_clean":"2024-09-13T18:22:08.482116+0000","last_became_active":"2024-09-13T18:20:01.917067+0000","last_became_peered":"2024-09-13T18:20:01.917067+0000","last_unstale":"2024-09-13T18:22:08.482116+0000","last_undegraded":"2024-09-13T18:22:08.482116+0000","last_fullsized":"2024-09-13T18:22:08.482116+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:35:37.723132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'4","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:01.190731+0000","last_change":"2024-09-13T18:20:01.573515+0000","last_active":"2024-09-13T18:22:01.190731+0000","last_peered":"2024-09-13T18:22:01.190731+0000","last_clean":"2024-09-13T18:22:01.190731+0000","last_became_active":"2024-09-13T18:20:01.573211+0000","last_became_peered":"2024-09-13T18:20:01.573211+0000","last_unstale":"2024-09-13T18:22:01.190731+0000","last_undegraded":"2024-09-13T18:22:01.190731+0000","last_fullsized":"2024-09-13T18:22:01.190731+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:05:30.912467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:30.983462+0000","last_change":"2024-09-13T18:21:28.231613+0000","last_active":"2024-09-13T18:21:30.983462+0000","last_peered":"2024-09-13T18:21:30.983462+0000","last_clean":"2024-09-13T18:21:30.983462+0000","last_became_active":"2024-09-13T18:21:28.231243+0000","last_became_peered":"2024-09-13T18:21:28.231243+0000","last_unstale":"2024-09-13T18:21:30.983462+0000","last_undegraded":"2024-09-13T18:21:30.983462+0000","last_fullsized":"2024-09-13T18:21:30.983462+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:58:46.066195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.2","version":"93'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:08.101059+0000","last_change":"2024-09-13T18:20:01.921597+0000","last_active":"2024-09-13T18:22:08.101059+0000","last_peered":"2024-09-13T18:22:08.101059+0000","last_clean":"2024-09-13T18:22:08.101059+0000","last_became_active":"2024-09-13T18:20:01.920200+0000","last_became_peered":"2024-09-13T18:20:01.920200+0000","last_unstale":"2024-09-13T18:22:08.101059+0000","last_undegraded":"2024-09-13T18:22:08.101059+0000","last_fullsized":"2024-09-13T18:22:08.101059+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:22:40.517941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'6","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:06.737540+0000","last_change":"2024-09-13T18:20:01.571313+0000","last_active":"2024-09-13T18:22:06.737540+0000","last_peered":"2024-09-13T18:22:06.737540+0000","last_clean":"2024-09-13T18:22:06.737540+0000","last_became_active":"2024-09-13T18:20:01.570977+0000","last_became_peered":"2024-09-13T18:20:01.570977+0000","last_unstale":"2024-09-13T18:22:06.737540+0000","last_undegraded":"2024-09-13T18:22:06.737540+0000","last_fullsized":"2024-09-13T18:22:06.737540+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:06.870320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":347,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:41.188167+0000","last_change":"2024-09-13T18:18:51.237409+0000","last_active":"2024-09-13T18:21:41.188167+0000","last_peered":"2024-09-13T18:21:41.188167+0000","last_clean":"2024-09-13T18:21:41.188167+0000","last_became_active":"2024-09-13T18:18:51.236986+0000","last_became_peered":"2024-09-13T18:18:51.236986+0000","last_unstale":"2024-09-13T18:21:41.188167+0000","last_undegraded":"2024-09-13T18:21:41.188167+0000","last_fullsized":"2024-09-13T18:21:41.188167+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:15:40.757299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:15:40.757299+0000","last_clean_scrub_stamp":"2024-09-13T18:15:40.757299+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:58:29.679404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":19,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:11.192032+0000","last_change":"2024-09-13T18:20:06.130803+0000","last_active":"2024-09-13T18:22:11.192032+0000","last_peered":"2024-09-13T18:22:11.192032+0000","last_clean":"2024-09-13T18:22:11.192032+0000","last_became_active":"2024-09-13T18:20:06.130433+0000","last_became_peered":"2024-09-13T18:20:06.130433+0000","last_unstale":"2024-09-13T18:22:11.192032+0000","last_undegraded":"2024-09-13T18:22:11.192032+0000","last_fullsized":"2024-09-13T18:22:11.192032+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:23:11.787084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"93'7","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:21.193214+0000","last_change":"2024-09-13T18:20:01.922702+0000","last_active":"2024-09-13T18:22:21.193214+0000","last_peered":"2024-09-13T18:22:21.193214+0000","last_clean":"2024-09-13T18:22:21.193214+0000","last_became_active":"2024-09-13T18:20:01.921109+0000","last_became_peered":"2024-09-13T18:20:01.921109+0000","last_unstale":"2024-09-13T18:22:21.193214+0000","last_undegraded":"2024-09-13T18:22:21.193214+0000","last_fullsized":"2024-09-13T18:22:21.193214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:03:34.618054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"93'6","reported_seq":152,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.067908+0000","last_change":"2024-09-13T18:20:01.571024+0000","last_active":"2024-09-13T18:22:28.067908+0000","last_peered":"2024-09-13T18:22:28.067908+0000","last_clean":"2024-09-13T18:22:28.067908+0000","last_became_active":"2024-09-13T18:20:01.570687+0000","last_became_peered":"2024-09-13T18:20:01.570687+0000","last_unstale":"2024-09-13T18:22:28.067908+0000","last_undegraded":"2024-09-13T18:22:28.067908+0000","last_fullsized":"2024-09-13T18:22:28.067908+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:10:25.126179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1616,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":124,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:18.277060+0000","last_change":"2024-09-13T18:20:06.129025+0000","last_active":"2024-09-13T18:22:18.277060+0000","last_peered":"2024-09-13T18:22:18.277060+0000","last_clean":"2024-09-13T18:22:18.277060+0000","last_became_active":"2024-09-13T18:20:06.128618+0000","last_became_peered":"2024-09-13T18:20:06.128618+0000","last_unstale":"2024-09-13T18:22:18.277060+0000","last_undegraded":"2024-09-13T18:22:18.277060+0000","last_fullsized":"2024-09-13T18:22:18.277060+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:20:08.891259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"93'6","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:21.193185+0000","last_change":"2024-09-13T18:20:01.921697+0000","last_active":"2024-09-13T18:22:21.193185+0000","last_peered":"2024-09-13T18:22:21.193185+0000","last_clean":"2024-09-13T18:22:21.193185+0000","last_became_active":"2024-09-13T18:20:01.920174+0000","last_became_peered":"2024-09-13T18:20:01.920174+0000","last_unstale":"2024-09-13T18:22:21.193185+0000","last_undegraded":"2024-09-13T18:22:21.193185+0000","last_fullsized":"2024-09-13T18:22:21.193185+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:57:03.922725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"94'41","reported_seq":71,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.250386+0000","last_change":"2024-09-13T18:20:01.582102+0000","last_active":"2024-09-13T18:22:28.250386+0000","last_peered":"2024-09-13T18:22:28.250386+0000","last_clean":"2024-09-13T18:22:28.250386+0000","last_became_active":"2024-09-13T18:20:01.581841+0000","last_became_peered":"2024-09-13T18:20:01.581841+0000","last_unstale":"2024-09-13T18:22:28.250386+0000","last_undegraded":"2024-09-13T18:22:28.250386+0000","last_fullsized":"2024-09-13T18:22:28.250386+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:59:57.929703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:22.286171+0000","last_change":"2024-09-13T18:20:06.133126+0000","last_active":"2024-09-13T18:22:22.286171+0000","last_peered":"2024-09-13T18:22:22.286171+0000","last_clean":"2024-09-13T18:22:22.286171+0000","last_became_active":"2024-09-13T18:20:06.132747+0000","last_became_peered":"2024-09-13T18:20:06.132747+0000","last_unstale":"2024-09-13T18:22:22.286171+0000","last_undegraded":"2024-09-13T18:22:22.286171+0000","last_fullsized":"2024-09-13T18:22:22.286171+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:54:21.843090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"93'9","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:21.193232+0000","last_change":"2024-09-13T18:20:01.916306+0000","last_active":"2024-09-13T18:22:21.193232+0000","last_peered":"2024-09-13T18:22:21.193232+0000","last_clean":"2024-09-13T18:22:21.193232+0000","last_became_active":"2024-09-13T18:20:01.915739+0000","last_became_peered":"2024-09-13T18:20:01.915739+0000","last_unstale":"2024-09-13T18:22:21.193232+0000","last_undegraded":"2024-09-13T18:22:21.193232+0000","last_fullsized":"2024-09-13T18:22:21.193232+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:19:09.689909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"91'5","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:18.483339+0000","last_change":"2024-09-13T18:20:01.565821+0000","last_active":"2024-09-13T18:22:18.483339+0000","last_peered":"2024-09-13T18:22:18.483339+0000","last_clean":"2024-09-13T18:22:18.483339+0000","last_became_active":"2024-09-13T18:20:01.565467+0000","last_became_peered":"2024-09-13T18:20:01.565467+0000","last_unstale":"2024-09-13T18:22:18.483339+0000","last_undegraded":"2024-09-13T18:22:18.483339+0000","last_fullsized":"2024-09-13T18:22:18.483339+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:11.609400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:16.035040+0000","last_change":"2024-09-13T18:20:06.131348+0000","last_active":"2024-09-13T18:22:16.035040+0000","last_peered":"2024-09-13T18:22:16.035040+0000","last_clean":"2024-09-13T18:22:16.035040+0000","last_became_active":"2024-09-13T18:20:06.131019+0000","last_became_peered":"2024-09-13T18:20:06.131019+0000","last_unstale":"2024-09-13T18:22:16.035040+0000","last_undegraded":"2024-09-13T18:22:16.035040+0000","last_fullsized":"2024-09-13T18:22:16.035040+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:13:49.717503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"93'5","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:17.959923+0000","last_change":"2024-09-13T18:20:01.922306+0000","last_active":"2024-09-13T18:22:17.959923+0000","last_peered":"2024-09-13T18:22:17.959923+0000","last_clean":"2024-09-13T18:22:17.959923+0000","last_became_active":"2024-09-13T18:20:01.921956+0000","last_became_peered":"2024-09-13T18:20:01.921956+0000","last_unstale":"2024-09-13T18:22:17.959923+0000","last_undegraded":"2024-09-13T18:22:17.959923+0000","last_fullsized":"2024-09-13T18:22:17.959923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:17:48.755318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"94'3","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.204515+0000","last_change":"2024-09-13T18:20:27.194085+0000","last_active":"2024-09-13T18:22:28.204515+0000","last_peered":"2024-09-13T18:22:28.204515+0000","last_clean":"2024-09-13T18:22:28.204515+0000","last_became_active":"2024-09-13T18:20:27.190136+0000","last_became_peered":"2024-09-13T18:20:27.190136+0000","last_unstale":"2024-09-13T18:22:28.204515+0000","last_undegraded":"2024-09-13T18:22:28.204515+0000","last_fullsized":"2024-09-13T18:22:28.204515+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:57:07.202901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:16.035090+0000","last_change":"2024-09-13T18:20:06.135498+0000","last_active":"2024-09-13T18:22:16.035090+0000","last_peered":"2024-09-13T18:22:16.035090+0000","last_clean":"2024-09-13T18:22:16.035090+0000","last_became_active":"2024-09-13T18:20:06.135095+0000","last_became_peered":"2024-09-13T18:20:06.135095+0000","last_unstale":"2024-09-13T18:22:16.035090+0000","last_undegraded":"2024-09-13T18:22:16.035090+0000","last_fullsized":"2024-09-13T18:22:16.035090+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:06:40.609718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"93'6","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:55.905745+0000","last_change":"2024-09-13T18:20:27.186825+0000","last_active":"2024-09-13T18:21:55.905745+0000","last_peered":"2024-09-13T18:21:55.905745+0000","last_clean":"2024-09-13T18:21:55.905745+0000","last_became_active":"2024-09-13T18:20:27.186523+0000","last_became_peered":"2024-09-13T18:20:27.186523+0000","last_unstale":"2024-09-13T18:21:55.905745+0000","last_undegraded":"2024-09-13T18:21:55.905745+0000","last_fullsized":"2024-09-13T18:21:55.905745+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:28:30.453945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.10","version":"91'17","reported_seq":61,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:20.989307+0000","last_change":"2024-09-13T18:20:27.200142+0000","last_active":"2024-09-13T18:22:20.989307+0000","last_peered":"2024-09-13T18:22:20.989307+0000","last_clean":"2024-09-13T18:22:20.989307+0000","last_became_active":"2024-09-13T18:20:27.189670+0000","last_became_peered":"2024-09-13T18:20:27.189670+0000","last_unstale":"2024-09-13T18:22:20.989307+0000","last_undegraded":"2024-09-13T18:22:20.989307+0000","last_fullsized":"2024-09-13T18:22:20.989307+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:16:02.934269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:28.150641+0000","last_change":"2024-09-13T18:20:06.136742+0000","last_active":"2024-09-13T18:21:28.150641+0000","last_peered":"2024-09-13T18:21:28.150641+0000","last_clean":"2024-09-13T18:21:28.150641+0000","last_became_active":"2024-09-13T18:20:06.136373+0000","last_became_peered":"2024-09-13T18:20:06.136373+0000","last_unstale":"2024-09-13T18:21:28.150641+0000","last_undegraded":"2024-09-13T18:21:28.150641+0000","last_fullsized":"2024-09-13T18:21:28.150641+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:49:11.782489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"93'6","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.288955+0000","last_change":"2024-09-13T18:20:01.922910+0000","last_active":"2024-09-13T18:22:28.288955+0000","last_peered":"2024-09-13T18:22:28.288955+0000","last_clean":"2024-09-13T18:22:28.288955+0000","last_became_active":"2024-09-13T18:20:01.921162+0000","last_became_peered":"2024-09-13T18:20:01.921162+0000","last_unstale":"2024-09-13T18:22:28.288955+0000","last_undegraded":"2024-09-13T18:22:28.288955+0000","last_fullsized":"2024-09-13T18:22:28.288955+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:14:47.243418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"94'8","reported_seq":155,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.416185+0000","last_change":"2024-09-13T18:20:01.575830+0000","last_active":"2024-09-13T18:22:28.416185+0000","last_peered":"2024-09-13T18:22:28.416185+0000","last_clean":"2024-09-13T18:22:28.416185+0000","last_became_active":"2024-09-13T18:20:01.575334+0000","last_became_peered":"2024-09-13T18:20:01.575334+0000","last_unstale":"2024-09-13T18:22:28.416185+0000","last_undegraded":"2024-09-13T18:22:28.416185+0000","last_fullsized":"2024-09-13T18:22:28.416185+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:13:00.935600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1818,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":126,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:28.154729+0000","last_change":"2024-09-13T18:20:06.138609+0000","last_active":"2024-09-13T18:21:28.154729+0000","last_peered":"2024-09-13T18:21:28.154729+0000","last_clean":"2024-09-13T18:21:28.154729+0000","last_became_active":"2024-09-13T18:20:06.138232+0000","last_became_peered":"2024-09-13T18:20:06.138232+0000","last_unstale":"2024-09-13T18:21:28.154729+0000","last_undegraded":"2024-09-13T18:21:28.154729+0000","last_fullsized":"2024-09-13T18:21:28.154729+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:46:06.227407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"93'15","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.289290+0000","last_change":"2024-09-13T18:20:01.928757+0000","last_active":"2024-09-13T18:22:28.289290+0000","last_peered":"2024-09-13T18:22:28.289290+0000","last_clean":"2024-09-13T18:22:28.289290+0000","last_became_active":"2024-09-13T18:20:01.928350+0000","last_became_peered":"2024-09-13T18:20:01.928350+0000","last_unstale":"2024-09-13T18:22:28.289290+0000","last_undegraded":"2024-09-13T18:22:28.289290+0000","last_fullsized":"2024-09-13T18:22:28.289290+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:17:43.208106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"91'5","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:21.035873+0000","last_change":"2024-09-13T18:20:01.579128+0000","last_active":"2024-09-13T18:22:21.035873+0000","last_peered":"2024-09-13T18:22:21.035873+0000","last_clean":"2024-09-13T18:22:21.035873+0000","last_became_active":"2024-09-13T18:20:01.578796+0000","last_became_peered":"2024-09-13T18:20:01.578796+0000","last_unstale":"2024-09-13T18:22:21.035873+0000","last_undegraded":"2024-09-13T18:22:21.035873+0000","last_fullsized":"2024-09-13T18:22:21.035873+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:30:04.836103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.204056+0000","last_change":"2024-09-13T18:20:06.131404+0000","last_active":"2024-09-13T18:22:28.204056+0000","last_peered":"2024-09-13T18:22:28.204056+0000","last_clean":"2024-09-13T18:22:28.204056+0000","last_became_active":"2024-09-13T18:20:06.130931+0000","last_became_peered":"2024-09-13T18:20:06.130931+0000","last_unstale":"2024-09-13T18:22:28.204056+0000","last_undegraded":"2024-09-13T18:22:28.204056+0000","last_fullsized":"2024-09-13T18:22:28.204056+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:55:54.568816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"93'9","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.292205+0000","last_change":"2024-09-13T18:20:01.916089+0000","last_active":"2024-09-13T18:22:28.292205+0000","last_peered":"2024-09-13T18:22:28.292205+0000","last_clean":"2024-09-13T18:22:28.292205+0000","last_became_active":"2024-09-13T18:20:01.915783+0000","last_became_peered":"2024-09-13T18:20:01.915783+0000","last_unstale":"2024-09-13T18:22:28.292205+0000","last_undegraded":"2024-09-13T18:22:28.292205+0000","last_fullsized":"2024-09-13T18:22:28.292205+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:43:25.886029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"91'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:20.789776+0000","last_change":"2024-09-13T18:20:01.577381+0000","last_active":"2024-09-13T18:22:20.789776+0000","last_peered":"2024-09-13T18:22:20.789776+0000","last_clean":"2024-09-13T18:22:20.789776+0000","last_became_active":"2024-09-13T18:20:01.577044+0000","last_became_peered":"2024-09-13T18:20:01.577044+0000","last_unstale":"2024-09-13T18:22:20.789776+0000","last_undegraded":"2024-09-13T18:22:20.789776+0000","last_fullsized":"2024-09-13T18:22:20.789776+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:45:00.907948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.146073+0000","last_change":"2024-09-13T18:20:06.139971+0000","last_active":"2024-09-13T18:22:28.146073+0000","last_peered":"2024-09-13T18:22:28.146073+0000","last_clean":"2024-09-13T18:22:28.146073+0000","last_became_active":"2024-09-13T18:20:06.139507+0000","last_became_peered":"2024-09-13T18:20:06.139507+0000","last_unstale":"2024-09-13T18:22:28.146073+0000","last_undegraded":"2024-09-13T18:22:28.146073+0000","last_fullsized":"2024-09-13T18:22:28.146073+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:40:21.037768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"93'6","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.140553+0000","last_change":"2024-09-13T18:20:01.915348+0000","last_active":"2024-09-13T18:22:28.140553+0000","last_peered":"2024-09-13T18:22:28.140553+0000","last_clean":"2024-09-13T18:22:28.140553+0000","last_became_active":"2024-09-13T18:20:01.915042+0000","last_became_peered":"2024-09-13T18:20:01.915042+0000","last_unstale":"2024-09-13T18:22:28.140553+0000","last_undegraded":"2024-09-13T18:22:28.140553+0000","last_fullsized":"2024-09-13T18:22:28.140553+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:31:33.712314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"94'41","reported_seq":315,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.421008+0000","last_change":"2024-09-13T18:20:01.578456+0000","last_active":"2024-09-13T18:22:28.421008+0000","last_peered":"2024-09-13T18:22:28.421008+0000","last_clean":"2024-09-13T18:22:28.421008+0000","last_became_active":"2024-09-13T18:20:01.578051+0000","last_became_peered":"2024-09-13T18:20:01.578051+0000","last_unstale":"2024-09-13T18:22:28.421008+0000","last_undegraded":"2024-09-13T18:22:28.421008+0000","last_fullsized":"2024-09-13T18:22:28.421008+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:34:17.651620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69151,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":252,"num_read_kb":194,"num_write":43,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.137141+0000","last_change":"2024-09-13T18:20:06.131410+0000","last_active":"2024-09-13T18:22:28.137141+0000","last_peered":"2024-09-13T18:22:28.137141+0000","last_clean":"2024-09-13T18:22:28.137141+0000","last_became_active":"2024-09-13T18:20:06.131060+0000","last_became_peered":"2024-09-13T18:20:06.131060+0000","last_unstale":"2024-09-13T18:22:28.137141+0000","last_undegraded":"2024-09-13T18:22:28.137141+0000","last_fullsized":"2024-09-13T18:22:28.137141+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:08:25.266971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"93'11","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.292104+0000","last_change":"2024-09-13T18:20:01.928839+0000","last_active":"2024-09-13T18:22:28.292104+0000","last_peered":"2024-09-13T18:22:28.292104+0000","last_clean":"2024-09-13T18:22:28.292104+0000","last_became_active":"2024-09-13T18:20:01.925774+0000","last_became_peered":"2024-09-13T18:20:01.925774+0000","last_unstale":"2024-09-13T18:22:28.292104+0000","last_undegraded":"2024-09-13T18:22:28.292104+0000","last_fullsized":"2024-09-13T18:22:28.292104+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:15:23.049671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"91'5","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.146218+0000","last_change":"2024-09-13T18:20:01.576332+0000","last_active":"2024-09-13T18:22:28.146218+0000","last_peered":"2024-09-13T18:22:28.146218+0000","last_clean":"2024-09-13T18:22:28.146218+0000","last_became_active":"2024-09-13T18:20:01.575862+0000","last_became_peered":"2024-09-13T18:20:01.575862+0000","last_unstale":"2024-09-13T18:22:28.146218+0000","last_undegraded":"2024-09-13T18:22:28.146218+0000","last_fullsized":"2024-09-13T18:22:28.146218+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:17:00.452480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.206402+0000","last_change":"2024-09-13T18:20:06.133142+0000","last_active":"2024-09-13T18:22:28.206402+0000","last_peered":"2024-09-13T18:22:28.206402+0000","last_clean":"2024-09-13T18:22:28.206402+0000","last_became_active":"2024-09-13T18:20:06.132740+0000","last_became_peered":"2024-09-13T18:20:06.132740+0000","last_unstale":"2024-09-13T18:22:28.206402+0000","last_undegraded":"2024-09-13T18:22:28.206402+0000","last_fullsized":"2024-09-13T18:22:28.206402+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:12:46.442481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"93'9","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.820616+0000","last_change":"2024-09-13T18:20:01.919673+0000","last_active":"2024-09-13T18:22:28.820616+0000","last_peered":"2024-09-13T18:22:28.820616+0000","last_clean":"2024-09-13T18:22:28.820616+0000","last_became_active":"2024-09-13T18:20:01.919358+0000","last_became_peered":"2024-09-13T18:20:01.919358+0000","last_unstale":"2024-09-13T18:22:28.820616+0000","last_undegraded":"2024-09-13T18:22:28.820616+0000","last_fullsized":"2024-09-13T18:22:28.820616+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:40:11.407643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"91'4","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.289505+0000","last_change":"2024-09-13T18:20:27.190451+0000","last_active":"2024-09-13T18:22:28.289505+0000","last_peered":"2024-09-13T18:22:28.289505+0000","last_clean":"2024-09-13T18:22:28.289505+0000","last_became_active":"2024-09-13T18:20:27.187091+0000","last_became_peered":"2024-09-13T18:20:27.187091+0000","last_unstale":"2024-09-13T18:22:28.289505+0000","last_undegraded":"2024-09-13T18:22:28.289505+0000","last_fullsized":"2024-09-13T18:22:28.289505+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:57:07.976008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.136965+0000","last_change":"2024-09-13T18:20:06.133475+0000","last_active":"2024-09-13T18:22:28.136965+0000","last_peered":"2024-09-13T18:22:28.136965+0000","last_clean":"2024-09-13T18:22:28.136965+0000","last_became_active":"2024-09-13T18:20:06.133109+0000","last_became_peered":"2024-09-13T18:20:06.133109+0000","last_unstale":"2024-09-13T18:22:28.136965+0000","last_undegraded":"2024-09-13T18:22:28.136965+0000","last_fullsized":"2024-09-13T18:22:28.136965+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:47:19.567115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"93'10","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.292323+0000","last_change":"2024-09-13T18:20:01.922660+0000","last_active":"2024-09-13T18:22:28.292323+0000","last_peered":"2024-09-13T18:22:28.292323+0000","last_clean":"2024-09-13T18:22:28.292323+0000","last_became_active":"2024-09-13T18:20:01.922370+0000","last_became_peered":"2024-09-13T18:20:01.922370+0000","last_unstale":"2024-09-13T18:22:28.292323+0000","last_undegraded":"2024-09-13T18:22:28.292323+0000","last_fullsized":"2024-09-13T18:22:28.292323+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:54:32.029704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"93'6","reported_seq":270,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.420091+0000","last_change":"2024-09-13T18:20:01.580975+0000","last_active":"2024-09-13T18:22:28.420091+0000","last_peered":"2024-09-13T18:22:28.420091+0000","last_clean":"2024-09-13T18:22:28.420091+0000","last_became_active":"2024-09-13T18:20:01.580647+0000","last_became_peered":"2024-09-13T18:20:01.580647+0000","last_unstale":"2024-09-13T18:22:28.420091+0000","last_undegraded":"2024-09-13T18:22:28.420091+0000","last_fullsized":"2024-09-13T18:22:28.420091+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:26:20.942915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":242,"num_read_kb":242,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.066097+0000","last_change":"2024-09-13T18:20:06.140003+0000","last_active":"2024-09-13T18:22:28.066097+0000","last_peered":"2024-09-13T18:22:28.066097+0000","last_clean":"2024-09-13T18:22:28.066097+0000","last_became_active":"2024-09-13T18:20:06.139663+0000","last_became_peered":"2024-09-13T18:20:06.139663+0000","last_unstale":"2024-09-13T18:22:28.066097+0000","last_undegraded":"2024-09-13T18:22:28.066097+0000","last_fullsized":"2024-09-13T18:22:28.066097+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:54:49.662401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:07.284799+0000","last_change":"2024-09-13T18:20:06.134448+0000","last_active":"2024-09-13T18:22:07.284799+0000","last_peered":"2024-09-13T18:22:07.284799+0000","last_clean":"2024-09-13T18:22:07.284799+0000","last_became_active":"2024-09-13T18:20:06.134084+0000","last_became_peered":"2024-09-13T18:20:06.134084+0000","last_unstale":"2024-09-13T18:22:07.284799+0000","last_undegraded":"2024-09-13T18:22:07.284799+0000","last_fullsized":"2024-09-13T18:22:07.284799+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:55:50.475665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"93'11","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:55.907316+0000","last_change":"2024-09-13T18:20:01.918955+0000","last_active":"2024-09-13T18:21:55.907316+0000","last_peered":"2024-09-13T18:21:55.907316+0000","last_clean":"2024-09-13T18:21:55.907316+0000","last_became_active":"2024-09-13T18:20:01.918666+0000","last_became_peered":"2024-09-13T18:20:01.918666+0000","last_unstale":"2024-09-13T18:21:55.907316+0000","last_undegraded":"2024-09-13T18:21:55.907316+0000","last_fullsized":"2024-09-13T18:21:55.907316+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:08:50.697710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"91'3","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.145838+0000","last_change":"2024-09-13T18:20:01.578789+0000","last_active":"2024-09-13T18:22:28.145838+0000","last_peered":"2024-09-13T18:22:28.145838+0000","last_clean":"2024-09-13T18:22:28.145838+0000","last_became_active":"2024-09-13T18:20:01.578476+0000","last_became_peered":"2024-09-13T18:20:01.578476+0000","last_unstale":"2024-09-13T18:22:28.145838+0000","last_undegraded":"2024-09-13T18:22:28.145838+0000","last_fullsized":"2024-09-13T18:22:28.145838+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:40:58.355492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:12.959377+0000","last_change":"2024-09-13T18:21:28.423633+0000","last_active":"2024-09-13T18:22:12.959377+0000","last_peered":"2024-09-13T18:22:12.959377+0000","last_clean":"2024-09-13T18:22:12.959377+0000","last_became_active":"2024-09-13T18:21:28.423128+0000","last_became_peered":"2024-09-13T18:21:28.423128+0000","last_unstale":"2024-09-13T18:22:12.959377+0000","last_undegraded":"2024-09-13T18:22:12.959377+0000","last_fullsized":"2024-09-13T18:22:12.959377+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:06:32.687874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"93'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.066097+0000","last_change":"2024-09-13T18:20:01.914804+0000","last_active":"2024-09-13T18:22:28.066097+0000","last_peered":"2024-09-13T18:22:28.066097+0000","last_clean":"2024-09-13T18:22:28.066097+0000","last_became_active":"2024-09-13T18:20:01.914517+0000","last_became_peered":"2024-09-13T18:20:01.914517+0000","last_unstale":"2024-09-13T18:22:28.066097+0000","last_undegraded":"2024-09-13T18:22:28.066097+0000","last_fullsized":"2024-09-13T18:22:28.066097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:57:10.734935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"91'3","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.292059+0000","last_change":"2024-09-13T18:20:01.569201+0000","last_active":"2024-09-13T18:22:28.292059+0000","last_peered":"2024-09-13T18:22:28.292059+0000","last_clean":"2024-09-13T18:22:28.292059+0000","last_became_active":"2024-09-13T18:20:01.568908+0000","last_became_peered":"2024-09-13T18:20:01.568908+0000","last_unstale":"2024-09-13T18:22:28.292059+0000","last_undegraded":"2024-09-13T18:22:28.292059+0000","last_fullsized":"2024-09-13T18:22:28.292059+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:16:34.069130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"91'7","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:28.230871+0000","last_change":"2024-09-13T18:20:01.579225+0000","last_active":"2024-09-13T18:21:28.230871+0000","last_peered":"2024-09-13T18:21:28.230871+0000","last_clean":"2024-09-13T18:21:28.230871+0000","last_became_active":"2024-09-13T18:20:01.578826+0000","last_became_peered":"2024-09-13T18:20:01.578826+0000","last_unstale":"2024-09-13T18:21:28.230871+0000","last_undegraded":"2024-09-13T18:21:28.230871+0000","last_fullsized":"2024-09-13T18:21:28.230871+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:48:07.632239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"93'8","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:55.916389+0000","last_change":"2024-09-13T18:20:01.914885+0000","last_active":"2024-09-13T18:21:55.916389+0000","last_peered":"2024-09-13T18:21:55.916389+0000","last_clean":"2024-09-13T18:21:55.916389+0000","last_became_active":"2024-09-13T18:20:01.914599+0000","last_became_peered":"2024-09-13T18:20:01.914599+0000","last_unstale":"2024-09-13T18:21:55.916389+0000","last_undegraded":"2024-09-13T18:21:55.916389+0000","last_fullsized":"2024-09-13T18:21:55.916389+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:41:30.888675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:28.311102+0000","last_change":"2024-09-13T18:20:06.129376+0000","last_active":"2024-09-13T18:21:28.311102+0000","last_peered":"2024-09-13T18:21:28.311102+0000","last_clean":"2024-09-13T18:21:28.311102+0000","last_became_active":"2024-09-13T18:20:06.129032+0000","last_became_peered":"2024-09-13T18:20:06.129032+0000","last_unstale":"2024-09-13T18:21:28.311102+0000","last_undegraded":"2024-09-13T18:21:28.311102+0000","last_fullsized":"2024-09-13T18:21:28.311102+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:21:53.511813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'7","reported_seq":154,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.421305+0000","last_change":"2024-09-13T18:20:01.576069+0000","last_active":"2024-09-13T18:22:28.421305+0000","last_peered":"2024-09-13T18:22:28.421305+0000","last_clean":"2024-09-13T18:22:28.421305+0000","last_became_active":"2024-09-13T18:20:01.575760+0000","last_became_peered":"2024-09-13T18:20:01.575760+0000","last_unstale":"2024-09-13T18:22:28.421305+0000","last_undegraded":"2024-09-13T18:22:28.421305+0000","last_fullsized":"2024-09-13T18:22:28.421305+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:00.549260+0000","last_clean_scrub_stamp":"2024-09-13T18:20:00.549260+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:05:47.677848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1460,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":125,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"93'7","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:55.900375+0000","last_change":"2024-09-13T18:20:01.917073+0000","last_active":"2024-09-13T18:21:55.900375+0000","last_peered":"2024-09-13T18:21:55.900375+0000","last_clean":"2024-09-13T18:21:55.900375+0000","last_became_active":"2024-09-13T18:20:01.916765+0000","last_became_peered":"2024-09-13T18:20:01.916765+0000","last_unstale":"2024-09-13T18:21:55.900375+0000","last_undegraded":"2024-09-13T18:21:55.900375+0000","last_fullsized":"2024-09-13T18:21:55.900375+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:22:07.788313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:28.311181+0000","last_change":"2024-09-13T18:20:06.131429+0000","last_active":"2024-09-13T18:21:28.311181+0000","last_peered":"2024-09-13T18:21:28.311181+0000","last_clean":"2024-09-13T18:21:28.311181+0000","last_became_active":"2024-09-13T18:20:06.131019+0000","last_became_peered":"2024-09-13T18:20:06.131019+0000","last_unstale":"2024-09-13T18:21:28.311181+0000","last_undegraded":"2024-09-13T18:21:28.311181+0000","last_fullsized":"2024-09-13T18:21:28.311181+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:05.096160+0000","last_clean_scrub_stamp":"2024-09-13T18:20:05.096160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:44:37.892574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"93'9","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:07.958537+0000","last_change":"2024-09-13T18:20:01.914050+0000","last_active":"2024-09-13T18:22:07.958537+0000","last_peered":"2024-09-13T18:22:07.958537+0000","last_clean":"2024-09-13T18:22:07.958537+0000","last_became_active":"2024-09-13T18:20:01.913733+0000","last_became_peered":"2024-09-13T18:20:01.913733+0000","last_unstale":"2024-09-13T18:22:07.958537+0000","last_undegraded":"2024-09-13T18:22:07.958537+0000","last_fullsized":"2024-09-13T18:22:07.958537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:52:43.672096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"93'12","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:10.788565+0000","last_change":"2024-09-13T18:20:01.924290+0000","last_active":"2024-09-13T18:22:10.788565+0000","last_peered":"2024-09-13T18:22:10.788565+0000","last_clean":"2024-09-13T18:22:10.788565+0000","last_became_active":"2024-09-13T18:20:01.923883+0000","last_became_peered":"2024-09-13T18:20:01.923883+0000","last_unstale":"2024-09-13T18:22:10.788565+0000","last_undegraded":"2024-09-13T18:22:10.788565+0000","last_fullsized":"2024-09-13T18:22:10.788565+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:11:18.737501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"93'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:10.788577+0000","last_change":"2024-09-13T18:20:27.184256+0000","last_active":"2024-09-13T18:22:10.788577+0000","last_peered":"2024-09-13T18:22:10.788577+0000","last_clean":"2024-09-13T18:22:10.788577+0000","last_became_active":"2024-09-13T18:20:27.183790+0000","last_became_peered":"2024-09-13T18:20:27.183790+0000","last_unstale":"2024-09-13T18:22:10.788577+0000","last_undegraded":"2024-09-13T18:22:10.788577+0000","last_fullsized":"2024-09-13T18:22:10.788577+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:06:48.885753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"93'7","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:07.284763+0000","last_change":"2024-09-13T18:20:01.915799+0000","last_active":"2024-09-13T18:22:07.284763+0000","last_peered":"2024-09-13T18:22:07.284763+0000","last_clean":"2024-09-13T18:22:07.284763+0000","last_became_active":"2024-09-13T18:20:01.915519+0000","last_became_peered":"2024-09-13T18:20:01.915519+0000","last_unstale":"2024-09-13T18:22:07.284763+0000","last_undegraded":"2024-09-13T18:22:07.284763+0000","last_fullsized":"2024-09-13T18:22:07.284763+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:08:26.065052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"93'4","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:13.482862+0000","last_change":"2024-09-13T18:20:01.918547+0000","last_active":"2024-09-13T18:22:13.482862+0000","last_peered":"2024-09-13T18:22:13.482862+0000","last_clean":"2024-09-13T18:22:13.482862+0000","last_became_active":"2024-09-13T18:20:01.918053+0000","last_became_peered":"2024-09-13T18:20:01.918053+0000","last_unstale":"2024-09-13T18:22:13.482862+0000","last_undegraded":"2024-09-13T18:22:13.482862+0000","last_fullsized":"2024-09-13T18:22:13.482862+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:25:58.235101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"93'6","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:12.959339+0000","last_change":"2024-09-13T18:20:01.916205+0000","last_active":"2024-09-13T18:22:12.959339+0000","last_peered":"2024-09-13T18:22:12.959339+0000","last_clean":"2024-09-13T18:22:12.959339+0000","last_became_active":"2024-09-13T18:20:01.915909+0000","last_became_peered":"2024-09-13T18:20:01.915909+0000","last_unstale":"2024-09-13T18:22:12.959339+0000","last_undegraded":"2024-09-13T18:22:12.959339+0000","last_fullsized":"2024-09-13T18:22:12.959339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:23:03.976513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"93'15","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:15.789324+0000","last_change":"2024-09-13T18:20:01.916621+0000","last_active":"2024-09-13T18:22:15.789324+0000","last_peered":"2024-09-13T18:22:15.789324+0000","last_clean":"2024-09-13T18:22:15.789324+0000","last_became_active":"2024-09-13T18:20:01.916335+0000","last_became_peered":"2024-09-13T18:20:01.916335+0000","last_unstale":"2024-09-13T18:22:15.789324+0000","last_undegraded":"2024-09-13T18:22:15.789324+0000","last_fullsized":"2024-09-13T18:22:15.789324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:08:16.104571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"93'2","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:13.101502+0000","last_change":"2024-09-13T18:20:01.921195+0000","last_active":"2024-09-13T18:22:13.101502+0000","last_peered":"2024-09-13T18:22:13.101502+0000","last_clean":"2024-09-13T18:22:13.101502+0000","last_became_active":"2024-09-13T18:20:01.920895+0000","last_became_peered":"2024-09-13T18:20:01.920895+0000","last_unstale":"2024-09-13T18:22:13.101502+0000","last_undegraded":"2024-09-13T18:22:13.101502+0000","last_fullsized":"2024-09-13T18:22:13.101502+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:59:53.920047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"93'6","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:16.035058+0000","last_change":"2024-09-13T18:20:01.910889+0000","last_active":"2024-09-13T18:22:16.035058+0000","last_peered":"2024-09-13T18:22:16.035058+0000","last_clean":"2024-09-13T18:22:16.035058+0000","last_became_active":"2024-09-13T18:20:01.910451+0000","last_became_peered":"2024-09-13T18:20:01.910451+0000","last_unstale":"2024-09-13T18:22:16.035058+0000","last_undegraded":"2024-09-13T18:22:16.035058+0000","last_fullsized":"2024-09-13T18:22:16.035058+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:06:52.552785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"93'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:18.101915+0000","last_change":"2024-09-13T18:20:01.914041+0000","last_active":"2024-09-13T18:22:18.101915+0000","last_peered":"2024-09-13T18:22:18.101915+0000","last_clean":"2024-09-13T18:22:18.101915+0000","last_became_active":"2024-09-13T18:20:01.913240+0000","last_became_peered":"2024-09-13T18:20:01.913240+0000","last_unstale":"2024-09-13T18:22:18.101915+0000","last_undegraded":"2024-09-13T18:22:18.101915+0000","last_fullsized":"2024-09-13T18:22:18.101915+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:39:37.062544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"93'10","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:18.277086+0000","last_change":"2024-09-13T18:20:01.927322+0000","last_active":"2024-09-13T18:22:18.277086+0000","last_peered":"2024-09-13T18:22:18.277086+0000","last_clean":"2024-09-13T18:22:18.277086+0000","last_became_active":"2024-09-13T18:20:01.926767+0000","last_became_peered":"2024-09-13T18:20:01.926767+0000","last_unstale":"2024-09-13T18:22:18.277086+0000","last_undegraded":"2024-09-13T18:22:18.277086+0000","last_fullsized":"2024-09-13T18:22:18.277086+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:20.986089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"93'11","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:18.483303+0000","last_change":"2024-09-13T18:20:01.916314+0000","last_active":"2024-09-13T18:22:18.483303+0000","last_peered":"2024-09-13T18:22:18.483303+0000","last_clean":"2024-09-13T18:22:18.483303+0000","last_became_active":"2024-09-13T18:20:01.915790+0000","last_became_peered":"2024-09-13T18:20:01.915790+0000","last_unstale":"2024-09-13T18:22:18.483303+0000","last_undegraded":"2024-09-13T18:22:18.483303+0000","last_fullsized":"2024-09-13T18:22:18.483303+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:20:30.063814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"93'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:17.959902+0000","last_change":"2024-09-13T18:20:27.187172+0000","last_active":"2024-09-13T18:22:17.959902+0000","last_peered":"2024-09-13T18:22:17.959902+0000","last_clean":"2024-09-13T18:22:17.959902+0000","last_became_active":"2024-09-13T18:20:27.186765+0000","last_became_peered":"2024-09-13T18:20:27.186765+0000","last_unstale":"2024-09-13T18:22:17.959902+0000","last_undegraded":"2024-09-13T18:22:17.959902+0000","last_fullsized":"2024-09-13T18:22:17.959902+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:37:01.469560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"93'7","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:20.789864+0000","last_change":"2024-09-13T18:20:01.924074+0000","last_active":"2024-09-13T18:22:20.789864+0000","last_peered":"2024-09-13T18:22:20.789864+0000","last_clean":"2024-09-13T18:22:20.789864+0000","last_became_active":"2024-09-13T18:20:01.922364+0000","last_became_peered":"2024-09-13T18:20:01.922364+0000","last_unstale":"2024-09-13T18:22:20.789864+0000","last_undegraded":"2024-09-13T18:22:20.789864+0000","last_fullsized":"2024-09-13T18:22:20.789864+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:51:33.824362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"93'10","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.138025+0000","last_change":"2024-09-13T18:20:01.914606+0000","last_active":"2024-09-13T18:22:28.138025+0000","last_peered":"2024-09-13T18:22:28.138025+0000","last_clean":"2024-09-13T18:22:28.138025+0000","last_became_active":"2024-09-13T18:20:01.914281+0000","last_became_peered":"2024-09-13T18:20:01.914281+0000","last_unstale":"2024-09-13T18:22:28.138025+0000","last_undegraded":"2024-09-13T18:22:28.138025+0000","last_fullsized":"2024-09-13T18:22:28.138025+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:55:42.211959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"93'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:22.960471+0000","last_change":"2024-09-13T18:20:01.928876+0000","last_active":"2024-09-13T18:22:22.960471+0000","last_peered":"2024-09-13T18:22:22.960471+0000","last_clean":"2024-09-13T18:22:22.960471+0000","last_became_active":"2024-09-13T18:20:01.928534+0000","last_became_peered":"2024-09-13T18:20:01.928534+0000","last_unstale":"2024-09-13T18:22:22.960471+0000","last_undegraded":"2024-09-13T18:22:22.960471+0000","last_fullsized":"2024-09-13T18:22:22.960471+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:14:28.196746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"93'9","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.291764+0000","last_change":"2024-09-13T18:20:01.920642+0000","last_active":"2024-09-13T18:22:28.291764+0000","last_peered":"2024-09-13T18:22:28.291764+0000","last_clean":"2024-09-13T18:22:28.291764+0000","last_became_active":"2024-09-13T18:20:01.920335+0000","last_became_peered":"2024-09-13T18:20:01.920335+0000","last_unstale":"2024-09-13T18:22:28.291764+0000","last_undegraded":"2024-09-13T18:22:28.291764+0000","last_fullsized":"2024-09-13T18:22:28.291764+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:59:20.894563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"93'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:22.286145+0000","last_change":"2024-09-13T18:20:27.187529+0000","last_active":"2024-09-13T18:22:22.286145+0000","last_peered":"2024-09-13T18:22:22.286145+0000","last_clean":"2024-09-13T18:22:22.286145+0000","last_became_active":"2024-09-13T18:20:27.187123+0000","last_became_peered":"2024-09-13T18:20:27.187123+0000","last_unstale":"2024-09-13T18:22:22.286145+0000","last_undegraded":"2024-09-13T18:22:22.286145+0000","last_fullsized":"2024-09-13T18:22:22.286145+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:43:19.601235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"93'7","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:22.286160+0000","last_change":"2024-09-13T18:20:01.917111+0000","last_active":"2024-09-13T18:22:22.286160+0000","last_peered":"2024-09-13T18:22:22.286160+0000","last_clean":"2024-09-13T18:22:22.286160+0000","last_became_active":"2024-09-13T18:20:01.916826+0000","last_became_peered":"2024-09-13T18:20:01.916826+0000","last_unstale":"2024-09-13T18:22:22.286160+0000","last_undegraded":"2024-09-13T18:22:22.286160+0000","last_fullsized":"2024-09-13T18:22:22.286160+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:36:08.150028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"93'10","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.292386+0000","last_change":"2024-09-13T18:20:01.904496+0000","last_active":"2024-09-13T18:22:28.292386+0000","last_peered":"2024-09-13T18:22:28.292386+0000","last_clean":"2024-09-13T18:22:28.292386+0000","last_became_active":"2024-09-13T18:20:01.904199+0000","last_became_peered":"2024-09-13T18:20:01.904199+0000","last_unstale":"2024-09-13T18:22:28.292386+0000","last_undegraded":"2024-09-13T18:22:28.292386+0000","last_fullsized":"2024-09-13T18:22:28.292386+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:44.044162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"93'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.206304+0000","last_change":"2024-09-13T18:20:01.919023+0000","last_active":"2024-09-13T18:22:28.206304+0000","last_peered":"2024-09-13T18:22:28.206304+0000","last_clean":"2024-09-13T18:22:28.206304+0000","last_became_active":"2024-09-13T18:20:01.918661+0000","last_became_peered":"2024-09-13T18:20:01.918661+0000","last_unstale":"2024-09-13T18:22:28.206304+0000","last_undegraded":"2024-09-13T18:22:28.206304+0000","last_fullsized":"2024-09-13T18:22:28.206304+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:25:26.779029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"93'6","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:55.925241+0000","last_change":"2024-09-13T18:20:01.922833+0000","last_active":"2024-09-13T18:21:55.925241+0000","last_peered":"2024-09-13T18:21:55.925241+0000","last_clean":"2024-09-13T18:21:55.925241+0000","last_became_active":"2024-09-13T18:20:01.922533+0000","last_became_peered":"2024-09-13T18:20:01.922533+0000","last_unstale":"2024-09-13T18:21:55.925241+0000","last_undegraded":"2024-09-13T18:21:55.925241+0000","last_fullsized":"2024-09-13T18:21:55.925241+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:27:11.557419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"93'16","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.066646+0000","last_change":"2024-09-13T18:20:01.925916+0000","last_active":"2024-09-13T18:22:28.066646+0000","last_peered":"2024-09-13T18:22:28.066646+0000","last_clean":"2024-09-13T18:22:28.066646+0000","last_became_active":"2024-09-13T18:20:01.925568+0000","last_became_peered":"2024-09-13T18:20:01.925568+0000","last_unstale":"2024-09-13T18:22:28.066646+0000","last_undegraded":"2024-09-13T18:22:28.066646+0000","last_fullsized":"2024-09-13T18:22:28.066646+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:58:24.955273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"93'11","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T18:22:28.292300+0000","last_change":"2024-09-13T18:20:01.924328+0000","last_active":"2024-09-13T18:22:28.292300+0000","last_peered":"2024-09-13T18:22:28.292300+0000","last_clean":"2024-09-13T18:22:28.292300+0000","last_became_active":"2024-09-13T18:20:01.924004+0000","last_became_peered":"2024-09-13T18:20:01.924004+0000","last_unstale":"2024-09-13T18:22:28.292300+0000","last_undegraded":"2024-09-13T18:22:28.292300+0000","last_fullsized":"2024-09-13T18:22:28.292300+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:23:29.204157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"93'9","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:55.913636+0000","last_change":"2024-09-13T18:20:01.922234+0000","last_active":"2024-09-13T18:21:55.913636+0000","last_peered":"2024-09-13T18:21:55.913636+0000","last_clean":"2024-09-13T18:21:55.913636+0000","last_became_active":"2024-09-13T18:20:01.921196+0000","last_became_peered":"2024-09-13T18:20:01.921196+0000","last_unstale":"2024-09-13T18:21:55.913636+0000","last_undegraded":"2024-09-13T18:21:55.913636+0000","last_fullsized":"2024-09-13T18:21:55.913636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:24:10.144346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"93'5","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:55.917809+0000","last_change":"2024-09-13T18:20:01.921740+0000","last_active":"2024-09-13T18:21:55.917809+0000","last_peered":"2024-09-13T18:21:55.917809+0000","last_clean":"2024-09-13T18:21:55.917809+0000","last_became_active":"2024-09-13T18:20:01.919634+0000","last_became_peered":"2024-09-13T18:20:01.919634+0000","last_unstale":"2024-09-13T18:21:55.917809+0000","last_undegraded":"2024-09-13T18:21:55.917809+0000","last_fullsized":"2024-09-13T18:21:55.917809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:43:27.600637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"93'6","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:55.915209+0000","last_change":"2024-09-13T18:20:01.924806+0000","last_active":"2024-09-13T18:21:55.915209+0000","last_peered":"2024-09-13T18:21:55.915209+0000","last_clean":"2024-09-13T18:21:55.915209+0000","last_became_active":"2024-09-13T18:20:01.924514+0000","last_became_peered":"2024-09-13T18:20:01.924514+0000","last_unstale":"2024-09-13T18:21:55.915209+0000","last_undegraded":"2024-09-13T18:21:55.915209+0000","last_fullsized":"2024-09-13T18:21:55.915209+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:47.323362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"93'4","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:21:55.901946+0000","last_change":"2024-09-13T18:20:01.912402+0000","last_active":"2024-09-13T18:21:55.901946+0000","last_peered":"2024-09-13T18:21:55.901946+0000","last_clean":"2024-09-13T18:21:55.901946+0000","last_became_active":"2024-09-13T18:20:01.911534+0000","last_became_peered":"2024-09-13T18:20:01.911534+0000","last_unstale":"2024-09-13T18:21:55.901946+0000","last_undegraded":"2024-09-13T18:21:55.901946+0000","last_fullsized":"2024-09-13T18:21:55.901946+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:22:52.021688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"93'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:01.737020+0000","last_change":"2024-09-13T18:20:01.930248+0000","last_active":"2024-09-13T18:22:01.737020+0000","last_peered":"2024-09-13T18:22:01.737020+0000","last_clean":"2024-09-13T18:22:01.737020+0000","last_became_active":"2024-09-13T18:20:01.929931+0000","last_became_peered":"2024-09-13T18:20:01.929931+0000","last_unstale":"2024-09-13T18:22:01.737020+0000","last_undegraded":"2024-09-13T18:22:01.737020+0000","last_fullsized":"2024-09-13T18:22:01.737020+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:37:33.086722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"93'7","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:07.284781+0000","last_change":"2024-09-13T18:20:01.921913+0000","last_active":"2024-09-13T18:22:07.284781+0000","last_peered":"2024-09-13T18:22:07.284781+0000","last_clean":"2024-09-13T18:22:07.284781+0000","last_became_active":"2024-09-13T18:20:01.921395+0000","last_became_peered":"2024-09-13T18:20:01.921395+0000","last_unstale":"2024-09-13T18:22:07.284781+0000","last_undegraded":"2024-09-13T18:22:07.284781+0000","last_fullsized":"2024-09-13T18:22:07.284781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:54:50.816870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"93'4","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:07.284827+0000","last_change":"2024-09-13T18:20:01.912654+0000","last_active":"2024-09-13T18:22:07.284827+0000","last_peered":"2024-09-13T18:22:07.284827+0000","last_clean":"2024-09-13T18:22:07.284827+0000","last_became_active":"2024-09-13T18:20:01.912356+0000","last_became_peered":"2024-09-13T18:20:01.912356+0000","last_unstale":"2024-09-13T18:22:07.284827+0000","last_undegraded":"2024-09-13T18:22:07.284827+0000","last_fullsized":"2024-09-13T18:22:07.284827+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:25:18.099396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"93'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T18:22:07.958659+0000","last_change":"2024-09-13T18:20:01.923373+0000","last_active":"2024-09-13T18:22:07.958659+0000","last_peered":"2024-09-13T18:22:07.958659+0000","last_clean":"2024-09-13T18:22:07.958659+0000","last_became_active":"2024-09-13T18:20:01.923077+0000","last_became_peered":"2024-09-13T18:20:01.923077+0000","last_unstale":"2024-09-13T18:22:07.958659+0000","last_undegraded":"2024-09-13T18:22:07.958659+0000","last_fullsized":"2024-09-13T18:22:07.958659+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:20:01.553959+0000","last_clean_scrub_stamp":"2024-09-13T18:20:01.553959+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:07:52.955705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":604133,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1347,"num_read_kb":793,"num_write":365,"num_write_kb":1467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":712704,"data_stored":1812399,"data_compressed":80730,"data_compressed_allocated":110592,"data_compressed_original":1572864,"omap_allocated":0,"internal_metadata":0},"log_size":299,"ondisk_log_size":299,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107472,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383723,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":1044480,"data_stored":529645,"data_compressed":26479,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:22:20 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.51200000000000001,"15min":0.48799999999999999},"min":{"1min":0.36299999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.70899999999999996,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.48899999999999999,"15min":0.46899999999999997},"min":{"1min":0.35999999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.70299999999999996,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.33600000000000002}]},{"osd":1,"last update":"Fri Sep 13 18:22:20 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.59199999999999997,"15min":0.57399999999999995},"min":{"1min":0.46000000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.88100000000000001,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.57899999999999996,"15min":0.56899999999999995},"min":{"1min":0.40200000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.78700000000000003,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.38500000000000001}]},{"osd":2,"last update":"Fri Sep 13 18:22:20 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.45400000000000001,"15min":0.44900000000000001},"min":{"1min":0.34300000000000003,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.63100000000000001,"5min":1.238,"15min":1.238},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.44800000000000001,"15min":0.434},"min":{"1min":0.33700000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.70799999999999996,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.40000000000000002}]},{"osd":3,"last update":"Fri Sep 13 18:21:27 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.504},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.47199999999999998}]},{"osd":4,"last update":"Fri Sep 13 18:22:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.63100000000000001,"15min":0.60999999999999999},"min":{"1min":0.53600000000000003,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85899999999999999,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.58799999999999997,"15min":0.56899999999999995},"min":{"1min":0.47099999999999997,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.84199999999999997,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.42199999999999999}]},{"osd":5,"last update":"Fri Sep 13 18:22:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.51900000000000002,"15min":0.50600000000000001},"min":{"1min":0.36099999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.73899999999999999,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.624,"5min":0.54600000000000004,"15min":0.52200000000000002},"min":{"1min":0.47799999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.81699999999999995,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.439}]},{"osd":6,"last update":"Fri Sep 13 18:22:20 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57799999999999996,"15min":0.56899999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85499999999999998,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.61499999999999999,"15min":0.59799999999999998},"min":{"1min":0.54100000000000004,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.79300000000000004,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.45900000000000002}]},{"osd":7,"last update":"Fri Sep 13 18:22:20 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.67900000000000005,"15min":0.64700000000000002},"min":{"1min":0.59699999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.95599999999999996,"5min":1.123,"15min":1.123},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.66600000000000004,"15min":0.65000000000000002},"min":{"1min":0.53400000000000003,"5min":0.439,"15min":0.439},"max":{"1min":0.94099999999999995,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.48199999999999998}]},{"osd":8,"last update":"Fri Sep 13 18:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.65100000000000002,"15min":0.63800000000000001},"min":{"1min":0.55500000000000005,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.72299999999999998,"15min":0.71099999999999997},"min":{"1min":0.64000000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.64300000000000002}]},{"osd":9,"last update":"Fri Sep 13 18:22:20 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.70099999999999996,"15min":0.68899999999999995},"min":{"1min":0.60399999999999998,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.91200000000000003,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.71499999999999997,"15min":0.68700000000000006},"min":{"1min":0.624,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.95399999999999996,"5min":1.208,"15min":1.208},"last":0.68200000000000005}]},{"osd":10,"last update":"Fri Sep 13 18:22:20 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.69999999999999996,"15min":0.67900000000000005},"min":{"1min":0.621,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.91100000000000003,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.72399999999999998,"15min":0.70599999999999996},"min":{"1min":0.66500000000000004,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.91500000000000004,"5min":1.583,"15min":1.583},"last":0.57599999999999996}]}]},{"osd":10,"up_from":72,"seq":309237645359,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28208,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716944,"statfs":{"total":95995035648,"available":95966150656,"internally_reserved":0,"allocated":1212416,"data_stored":1321739,"data_compressed":70067,"data_compressed_allocated":94208,"data_compressed_original":794624,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:22:03 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.502,"15min":0.504},"min":{"1min":0.40300000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.66700000000000004,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":1.018,"15min":1.107},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.624,"5min":14.589,"15min":14.589},"last":0.373}]},{"osd":1,"last update":"Fri Sep 13 18:21:55 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.41599999999999998,"15min":0.41999999999999998},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.54700000000000004,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.41999999999999998,"15min":0.41699999999999998},"min":{"1min":0.309,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.57799999999999996,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.29499999999999998}]},{"osd":2,"last update":"Fri Sep 13 18:21:55 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55000000000000004,"15min":0.54400000000000004},"min":{"1min":0.33700000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.72599999999999998,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56499999999999995,"15min":0.55500000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.74199999999999999,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.53100000000000003}]},{"osd":3,"last update":"Fri Sep 13 18:21:55 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.56399999999999995,"15min":0.57399999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.70499999999999996,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.68200000000000005,"15min":0.61099999999999999},"min":{"1min":0.377,"5min":0.376,"15min":0.376},"max":{"1min":0.67300000000000004,"5min":14.505000000000001,"15min":14.505000000000001},"last":0.55100000000000005}]},{"osd":4,"last update":"Fri Sep 13 18:21:55 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.503,"15min":0.50800000000000001},"min":{"1min":0.34799999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.624,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.495,"15min":0.499},"min":{"1min":0.33800000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.68600000000000005,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.312}]},{"osd":5,"last update":"Fri Sep 13 18:21:55 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.60999999999999999,"15min":0.60199999999999998},"min":{"1min":0.45500000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.82799999999999996,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.626},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.71199999999999997,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.48599999999999999}]},{"osd":6,"last update":"Fri Sep 13 18:21:55 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.752,"15min":0.67700000000000005},"min":{"1min":0.496,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.77800000000000002,"5min":14.539999999999999,"15min":14.539999999999999},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.64200000000000002,"15min":0.64700000000000002},"min":{"1min":0.50800000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.92600000000000005,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.41699999999999998}]},{"osd":7,"last update":"Fri Sep 13 18:19:48 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.81599999999999995}]},{"osd":8,"last update":"Fri Sep 13 18:21:55 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.66700000000000004,"15min":0.67100000000000004},"min":{"1min":0.41999999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.90900000000000003,"5min":1.01,"15min":1.01},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.68899999999999995,"15min":0.69299999999999995},"min":{"1min":0.51300000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.81599999999999995,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.54200000000000004}]},{"osd":9,"last update":"Fri Sep 13 18:21:55 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.68400000000000005,"15min":0.69099999999999995},"min":{"1min":0.53600000000000003,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.89400000000000002,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.436},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.69999999999999996,"15min":0.71299999999999997},"min":{"1min":0.45800000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.81100000000000005,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.497}]},{"osd":11,"last update":"Fri Sep 13 18:22:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.72099999999999997,"15min":0.72899999999999998},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.84199999999999997,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.745,"5min":0.73799999999999999,"15min":0.73999999999999999},"min":{"1min":0.55400000000000005,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.89900000000000002,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.56200000000000006}]}]},{"osd":9,"up_from":67,"seq":287762808884,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1089536,"data_stored":922323,"data_compressed":45625,"data_compressed_allocated":61440,"data_compressed_original":466944,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:21:33 2024","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.36099999999999999,"15min":0.38100000000000001},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.45800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.41399999999999998,"15min":0.442},"min":{"1min":0.249,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.495,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.38700000000000001}]},{"osd":1,"last update":"Fri Sep 13 18:21:33 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.56299999999999994,"15min":0.58399999999999996},"min":{"1min":0.35699999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.67300000000000004,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.56399999999999995,"15min":0.58499999999999996},"min":{"1min":0.36399999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.67700000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.36599999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:21:33 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.55000000000000004,"15min":0.57399999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.65000000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.52600000000000002,"15min":0.52800000000000002},"min":{"1min":0.34899999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.42099999999999999}]},{"osd":3,"last update":"Fri Sep 13 18:21:33 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.44700000000000001,"15min":0.48099999999999998},"min":{"1min":0.28499999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.50900000000000001,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.37,"5min":0.437,"15min":0.46300000000000002},"min":{"1min":0.28000000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.496,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.32300000000000001}]},{"osd":4,"last update":"Fri Sep 13 18:21:33 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.55600000000000005,"15min":0.57299999999999995},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.67700000000000005,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.61299999999999999,"15min":0.64700000000000002},"min":{"1min":0.39000000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.65300000000000002,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.54200000000000004}]},{"osd":5,"last update":"Fri Sep 13 18:21:33 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.57199999999999995,"15min":0.59099999999999997},"min":{"1min":0.36599999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.59299999999999997,"15min":0.61099999999999999},"min":{"1min":0.40600000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.752,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.40400000000000003}]},{"osd":6,"last update":"Fri Sep 13 18:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.39800000000000002}]},{"osd":7,"last update":"Fri Sep 13 18:21:33 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.63600000000000001,"15min":0.67400000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.70899999999999996,"5min":0.871,"15min":0.871},"last":0.437},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.64700000000000002,"15min":0.68100000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.63900000000000001,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.42899999999999999}]},{"osd":8,"last update":"Fri Sep 13 18:21:33 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.63500000000000001,"15min":0.66500000000000004},"min":{"1min":0.40999999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.75,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.66500000000000004,"15min":0.69999999999999996},"min":{"1min":0.436,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.81899999999999995,"5min":1.02,"15min":1.02},"last":0.51900000000000002}]},{"osd":10,"last update":"Fri Sep 13 18:21:54 2024","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":0.73499999999999999,"15min":0.69299999999999995},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":9.3510000000000009,"5min":9.3510000000000009,"15min":9.3510000000000009},"last":0.505},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.65900000000000003,"15min":0.68300000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.79100000000000004,"5min":0.879,"15min":0.879},"last":0.498}]},{"osd":11,"last update":"Fri Sep 13 18:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.48199999999999998,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.72999999999999998,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.45900000000000002,"5min":0.432,"15min":0.432},"max":{"1min":0.65300000000000002,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.377}]}]},{"osd":8,"up_from":59,"seq":253403070521,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1081344,"data_stored":922679,"data_compressed":45625,"data_compressed_allocated":61440,"data_compressed_original":466944,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.51300000000000001,"15min":0.54000000000000004},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.72299999999999998,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.55300000000000005,"15min":0.56299999999999994},"min":{"1min":0.35199999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.65000000000000002,"5min":3.2989999999999999,"15min":3.2989999999999999},"last":0.44600000000000001}]},{"osd":1,"last update":"Fri Sep 13 18:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.57699999999999996,"15min":0.56499999999999995},"min":{"1min":0.36699999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.79100000000000004,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.58399999999999996,"15min":0.57599999999999996},"min":{"1min":0.42399999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72699999999999998,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.36199999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.42499999999999999,"15min":0.42899999999999999},"min":{"1min":0.25,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.58199999999999996,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.314},{"interface":"front","average":{"1min":0.442,"5min":0.439,"15min":0.437},"min":{"1min":0.307,"5min":0.224,"15min":0.224},"max":{"1min":0.68899999999999995,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.56699999999999995}]},{"osd":3,"last update":"Fri Sep 13 18:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53700000000000003,"15min":0.55500000000000005},"min":{"1min":0.35899999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.69999999999999996,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.53400000000000003,"15min":0.53800000000000003},"min":{"1min":0.37,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.68799999999999994,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.38800000000000001}]},{"osd":4,"last update":"Fri Sep 13 18:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.66400000000000003,"15min":0.63600000000000001},"min":{"1min":0.45100000000000001,"5min":0.37,"15min":0.37},"max":{"1min":5.3540000000000001,"5min":5.3540000000000001,"15min":5.3540000000000001},"last":0.68000000000000005}]},{"osd":5,"last update":"Fri Sep 13 18:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.52000000000000002,"15min":0.53400000000000003},"min":{"1min":0.33600000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.67500000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.51100000000000001,"15min":0.52100000000000002},"min":{"1min":0.33400000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.64900000000000002,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.40300000000000002}]},{"osd":6,"last update":"Fri Sep 13 18:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.61299999999999999,"15min":0.61499999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.77700000000000002,"5min":3.2719999999999998,"15min":3.2719999999999998},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.61499999999999999,"15min":0.64000000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.71999999999999997,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.48899999999999999}]},{"osd":7,"last update":"Fri Sep 13 18:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.65600000000000003,"15min":0.67200000000000004},"min":{"1min":0.45500000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.77900000000000003,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.66200000000000003,"15min":0.66500000000000004},"min":{"1min":0.51800000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.81999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.51400000000000001}]},{"osd":9,"last update":"Fri Sep 13 18:22:25 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63700000000000001,"15min":0.63300000000000001},"min":{"1min":0.50900000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73799999999999999,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.63500000000000001,"15min":0.64800000000000002},"min":{"1min":0.39400000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.82199999999999995,"5min":1.012,"15min":1.012},"last":0.505}]},{"osd":10,"last update":"Fri Sep 13 18:21:53 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.85499999999999998,"15min":0.93400000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.79800000000000004,"5min":5.3360000000000003,"15min":5.3360000000000003},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.67800000000000005,"15min":0.67400000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.80900000000000005,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.46999999999999997}]},{"osd":11,"last update":"Fri Sep 13 18:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.65500000000000003,"15min":0.65900000000000003},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.76000000000000001,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.499},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.71599999999999997,"15min":0.72199999999999998},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.79900000000000004,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.53000000000000003}]}]},{"osd":7,"up_from":51,"seq":219043332157,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1114112,"data_stored":1128889,"data_compressed":62303,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:19:42 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.94399999999999995,"15min":1.012},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.84299999999999997,"5min":12.068,"15min":12.068},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.46999999999999997,"15min":0.45900000000000002},"min":{"1min":0.35199999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.61599999999999999}]},{"osd":1,"last update":"Fri Sep 13 18:21:46 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.379,"15min":0.36099999999999999},"min":{"1min":0.27300000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.40300000000000002,"15min":0.38200000000000001},"min":{"1min":0.30399999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.54500000000000004,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.26500000000000001}]},{"osd":2,"last update":"Fri Sep 13 18:21:46 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.627,"15min":0.54900000000000004},"min":{"1min":0.40999999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.79100000000000004,"5min":7.6820000000000004,"15min":7.6820000000000004},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.54000000000000004,"15min":0.51500000000000001},"min":{"1min":0.45600000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.40500000000000003}]},{"osd":3,"last update":"Fri Sep 13 18:21:46 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.52900000000000003,"15min":0.49299999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.70599999999999996,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.52400000000000002,"15min":0.49399999999999999},"min":{"1min":0.35599999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.73699999999999999,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.38600000000000001}]},{"osd":4,"last update":"Fri Sep 13 18:21:46 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.45800000000000002,"15min":0.439},"min":{"1min":0.32000000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.57499999999999996,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.46100000000000002,"15min":0.45300000000000001},"min":{"1min":0.32500000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.66500000000000004,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.33000000000000002}]},{"osd":5,"last update":"Fri Sep 13 18:21:46 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.59299999999999997,"15min":0.56200000000000006},"min":{"1min":0.46600000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.76600000000000001,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.56899999999999995,"15min":0.54100000000000004},"min":{"1min":0.40999999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.748,"5min":0.754,"15min":0.754},"last":0.47299999999999998}]},{"osd":6,"last update":"Fri Sep 13 18:21:46 2024","interfaces":[{"interface":"back","average":{"1min":1.5700000000000001,"5min":0.77400000000000002,"15min":0.61499999999999999},"min":{"1min":0.40300000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":20.013000000000002,"5min":20.013000000000002,"15min":20.013000000000002},"last":0.438},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.60899999999999999,"15min":0.58699999999999997},"min":{"1min":0.38500000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72999999999999998,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.51000000000000001}]},{"osd":8,"last update":"Fri Sep 13 18:22:04 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60599999999999998,"15min":0.59199999999999997},"min":{"1min":0.497,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.70499999999999996,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.747,"15min":0.67000000000000004},"min":{"1min":0.505,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.79600000000000004,"5min":7.7149999999999999,"15min":7.7149999999999999},"last":0.53700000000000003}]},{"osd":9,"last update":"Fri Sep 13 18:21:29 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.66300000000000003,"15min":0.66100000000000003},"min":{"1min":0.40500000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.88,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.45600000000000002},{"interface":"front","average":{"1min":1.609,"5min":0.82999999999999996,"15min":0.70199999999999996},"min":{"1min":0.42699999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":20.056999999999999,"5min":20.056999999999999,"15min":20.056999999999999},"last":0.42399999999999999}]},{"osd":10,"last update":"Fri Sep 13 18:21:51 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.65100000000000002,"15min":0.65300000000000002},"min":{"1min":0.43099999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.79900000000000004,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.496},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.63800000000000001,"15min":0.64200000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.755,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.44700000000000001}]},{"osd":11,"last update":"Fri Sep 13 18:22:04 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.71899999999999997,"15min":0.72699999999999998},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.78500000000000003,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.502},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.73899999999999999,"15min":0.747},"min":{"1min":0.59999999999999998,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.80800000000000005,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.48799999999999999}]}]},{"osd":6,"up_from":46,"seq":197568495682,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":1040384,"data_stored":525841,"data_compressed":26479,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.37,"15min":0.36699999999999999},"min":{"1min":0.25900000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":1.2030000000000001},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.41399999999999998,"15min":0.432},"min":{"1min":0.28699999999999998,"5min":0.218,"15min":0.218},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":1.0940000000000001}]},{"osd":1,"last update":"Fri Sep 13 18:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.53700000000000003,"15min":0.52500000000000002},"min":{"1min":0.311,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.56399999999999995,"15min":0.54100000000000004},"min":{"1min":0.45300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":1.0669999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.42399999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78600000000000003,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.54700000000000004,"15min":0.55100000000000005},"min":{"1min":0.38800000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":1.2729999999999999}]},{"osd":3,"last update":"Fri Sep 13 18:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.41099999999999998,"15min":0.42199999999999999},"min":{"1min":0.28499999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":1.254},{"interface":"front","average":{"1min":0.499,"5min":0.46200000000000002,"15min":0.46999999999999997},"min":{"1min":0.34300000000000003,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":1.179}]},{"osd":4,"last update":"Fri Sep 13 18:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.53200000000000003,"15min":0.51800000000000002},"min":{"1min":0.35599999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.55400000000000005,"15min":0.55300000000000005},"min":{"1min":0.39400000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":1.2190000000000001}]},{"osd":5,"last update":"Fri Sep 13 18:22:26 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.59399999999999997,"15min":0.60599999999999998},"min":{"1min":0.48999999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":1.038},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.56299999999999994,"15min":0.58899999999999997},"min":{"1min":0.377,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.83999999999999997,"5min":0.874,"15min":0.874},"last":0.83999999999999997}]},{"osd":7,"last update":"Fri Sep 13 18:21:45 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57899999999999996,"15min":0.57499999999999996},"min":{"1min":0.41099999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.56299999999999994,"15min":0.56100000000000005},"min":{"1min":0.442,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.70399999999999996,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":1.2889999999999999}]},{"osd":8,"last update":"Fri Sep 13 18:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.61899999999999999,"15min":0.621},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.91800000000000004,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":1.01},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.58599999999999997,"15min":0.59599999999999997},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.71599999999999997,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":1.3340000000000001}]},{"osd":9,"last update":"Fri Sep 13 18:21:34 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.51500000000000001,"15min":0.53100000000000003},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.57199999999999995,"5min":0.871,"15min":0.871},"last":1.3089999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.59099999999999997,"15min":0.60499999999999998},"min":{"1min":0.41399999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.75700000000000001,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":1.153}]},{"osd":10,"last update":"Fri Sep 13 18:21:55 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.60599999999999998,"15min":0.61499999999999999},"min":{"1min":0.48099999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.71799999999999997,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.66700000000000004,"15min":0.67100000000000004},"min":{"1min":0.53300000000000003,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.78300000000000003,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.96799999999999997}]},{"osd":11,"last update":"Fri Sep 13 18:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.63900000000000001,"15min":0.64600000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.77400000000000002,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":1.3600000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68600000000000005,"15min":0.68700000000000006},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.90200000000000002,"5min":1.01,"15min":1.01},"last":1.127}]}]},{"osd":5,"up_from":38,"seq":163208757319,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1142784,"data_stored":1253611,"data_compressed":68459,"data_compressed_allocated":90112,"data_compressed_original":786432,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:22:00 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.497,"15min":0.495},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.65000000000000002,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.51200000000000001,"15min":0.51100000000000001},"min":{"1min":0.32300000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.73199999999999998,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.373}]},{"osd":1,"last update":"Fri Sep 13 18:22:00 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.54300000000000004,"15min":0.51100000000000001},"min":{"1min":0.33300000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.71599999999999997,"5min":0.878,"15min":0.878},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.53300000000000003,"15min":0.51400000000000001},"min":{"1min":0.45800000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.68899999999999995,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.45400000000000001}]},{"osd":2,"last update":"Fri Sep 13 18:22:00 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.39600000000000002,"15min":0.36799999999999999},"min":{"1min":0.27900000000000003,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.65100000000000002,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.315},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.39900000000000002,"15min":0.38700000000000001},"min":{"1min":0.29999999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.34399999999999997}]},{"osd":3,"last update":"Fri Sep 13 18:22:00 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.52300000000000002,"15min":0.503},"min":{"1min":0.34799999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.70199999999999996,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.52200000000000002,"15min":0.52000000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.46800000000000003}]},{"osd":4,"last update":"Fri Sep 13 18:22:00 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55900000000000005,"15min":0.54200000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.79000000000000004,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.79200000000000004,"15min":0.61799999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82999999999999996,"5min":18.571000000000002,"15min":18.571000000000002},"last":0.47699999999999998}]},{"osd":6,"last update":"Fri Sep 13 18:22:25 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57499999999999996,"15min":0.57399999999999995},"min":{"1min":0.47199999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.73299999999999998,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56000000000000005,"15min":0.55200000000000005},"min":{"1min":0.38400000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.53600000000000003}]},{"osd":7,"last update":"Fri Sep 13 18:21:45 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61599999999999999,"15min":0.60799999999999998},"min":{"1min":0.46300000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76400000000000001,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63100000000000001,"15min":0.61599999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.84999999999999998,"5min":0.88,"15min":0.88},"last":0.499}]},{"osd":8,"last update":"Fri Sep 13 18:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.52700000000000002,"15min":0.51600000000000001},"min":{"1min":0.40200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.56899999999999995,"15min":0.54500000000000004},"min":{"1min":0.40600000000000003,"5min":0.314,"15min":0.314},"max":{"1min":0.876,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.56499999999999995}]},{"osd":9,"last update":"Fri Sep 13 18:21:30 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.64500000000000002,"15min":0.627},"min":{"1min":0.48199999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.63900000000000001,"15min":0.628},"min":{"1min":0.48799999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.81299999999999994,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.57699999999999996}]},{"osd":10,"last update":"Fri Sep 13 18:21:54 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69199999999999995,"15min":0.69099999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.68000000000000005,"15min":0.67000000000000004},"min":{"1min":0.53300000000000003,"5min":0.37,"15min":0.37},"max":{"1min":0.89000000000000001,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.55900000000000005}]}]},{"osd":4,"up_from":33,"seq":141733920843,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":529915,"data_compressed":26479,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:19:33 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.48299999999999998,"15min":0.47699999999999998},"min":{"1min":0.35399999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.69399999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.433},{"interface":"front","average":{"1min":0.501,"5min":0.51200000000000001,"15min":0.47499999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.84599999999999997,"5min":4.2539999999999996,"15min":4.2539999999999996},"last":0.51900000000000002}]},{"osd":1,"last update":"Fri Sep 13 18:21:38 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.373,"15min":0.35199999999999998},"min":{"1min":0.25800000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.59199999999999997,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.379,"15min":0.35399999999999998},"min":{"1min":0.29199999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.68400000000000005,"5min":0.748,"15min":0.748},"last":0.39200000000000002}]},{"osd":2,"last update":"Fri Sep 13 18:21:38 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.60699999999999998,"15min":0.54900000000000004},"min":{"1min":0.40999999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":5.2930000000000001,"5min":5.2930000000000001,"15min":5.2930000000000001},"last":0.438},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55300000000000005,"15min":0.53800000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72499999999999998,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.51900000000000002}]},{"osd":3,"last update":"Fri Sep 13 18:21:38 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56000000000000005,"15min":0.505},"min":{"1min":0.434,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.82199999999999995,"5min":4.2679999999999998,"15min":4.2679999999999998},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.52100000000000002,"15min":0.496},"min":{"1min":0.46100000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.41399999999999998}]},{"osd":5,"last update":"Fri Sep 13 18:21:59 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.57099999999999995,"15min":0.54300000000000004},"min":{"1min":0.434,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.70399999999999996,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.58099999999999996,"15min":0.54700000000000004},"min":{"1min":0.40400000000000003,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.75900000000000001,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.42499999999999999}]},{"osd":6,"last update":"Fri Sep 13 18:22:21 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56399999999999995,"15min":0.52400000000000002},"min":{"1min":0.40400000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.754,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.56699999999999995,"15min":0.52300000000000002},"min":{"1min":0.47799999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.73899999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.50800000000000001}]},{"osd":7,"last update":"Fri Sep 13 18:21:48 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.46999999999999997,"15min":0.44600000000000001},"min":{"1min":0.376,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.76100000000000001,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.50900000000000001,"15min":0.48499999999999999},"min":{"1min":0.379,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.76100000000000001,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.48399999999999999}]},{"osd":8,"last update":"Fri Sep 13 18:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.64200000000000002,"15min":0.63},"min":{"1min":0.54900000000000004,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.76600000000000001,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.374},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.67000000000000004,"15min":0.65200000000000002},"min":{"1min":0.624,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.76900000000000002,"5min":0.874,"15min":0.874},"last":0.57499999999999996}]},{"osd":9,"last update":"Fri Sep 13 18:21:28 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.63300000000000001,"15min":0.61499999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.63700000000000001,"15min":0.629},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.83299999999999996,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.44700000000000001}]},{"osd":10,"last update":"Fri Sep 13 18:21:52 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.63400000000000001},"min":{"1min":0.438,"5min":0.377,"15min":0.377},"max":{"1min":0.81200000000000006,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.499},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.66100000000000003,"15min":0.65900000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.746,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.45600000000000002}]},{"osd":11,"last update":"Fri Sep 13 18:22:04 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.73299999999999998,"15min":0.73399999999999999},"min":{"1min":0.58099999999999996,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.89300000000000002,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.72499999999999998,"15min":0.72199999999999998},"min":{"1min":0.66300000000000003,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.81399999999999995,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.54800000000000004}]}]},{"osd":3,"up_from":25,"seq":107374182479,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":524038,"data_compressed":26479,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:22:18 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34699999999999998,"15min":0.313},"min":{"1min":0.216,"5min":0.183,"15min":0.183},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.34599999999999997,"15min":0.33000000000000002},"min":{"1min":0.23200000000000001,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.48099999999999998,"5min":0.58199999999999996,"15min":0.58199999999999996},"last":0.54700000000000004}]},{"osd":1,"last update":"Fri Sep 13 18:22:18 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.53700000000000003,"15min":0.497},"min":{"1min":0.41799999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.68400000000000005,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.622},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.54700000000000004,"15min":0.499},"min":{"1min":0.443,"5min":0.33700000000000002,"15min":0.313},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.68100000000000005}]},{"osd":2,"last update":"Fri Sep 13 18:22:18 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52500000000000002,"15min":0.52300000000000002},"min":{"1min":0.42199999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.54500000000000004,"15min":0.50800000000000001},"min":{"1min":0.42699999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.748,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.66700000000000004}]},{"osd":4,"last update":"Fri Sep 13 18:21:38 2024","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":0.622,"15min":0.52600000000000002},"min":{"1min":0.40500000000000003,"5min":0.312,"15min":0.312},"max":{"1min":9.3149999999999995,"5min":9.3149999999999995,"15min":9.3149999999999995},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.53900000000000003,"15min":0.51900000000000002},"min":{"1min":0.39900000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.72399999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.69499999999999995}]},{"osd":5,"last update":"Fri Sep 13 18:21:56 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.54800000000000004,"15min":0.53800000000000003},"min":{"1min":0.435,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.74199999999999999,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.54500000000000004,"15min":0.55000000000000004},"min":{"1min":0.38900000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.74199999999999999,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.60999999999999999}]},{"osd":6,"last update":"Fri Sep 13 18:22:28 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.434,"15min":0.44400000000000001},"min":{"1min":0.32800000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.749,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.749},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.48699999999999999,"15min":0.46899999999999997},"min":{"1min":0.307,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.76000000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.76000000000000001}]},{"osd":7,"last update":"Fri Sep 13 18:21:48 2024","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":0.69199999999999995,"15min":0.626},"min":{"1min":0.45700000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":9.3330000000000002,"5min":9.3330000000000002,"15min":9.3330000000000002},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57999999999999996,"15min":0.57399999999999995},"min":{"1min":0.39000000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.67700000000000005,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.70499999999999996}]},{"osd":8,"last update":"Fri Sep 13 18:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.69599999999999995}]},{"osd":9,"last update":"Fri Sep 13 18:21:31 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.52500000000000002,"15min":0.54900000000000004},"min":{"1min":0.33800000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.59999999999999998,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.57099999999999995,"15min":0.58499999999999996},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.73899999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.81000000000000005}]},{"osd":10,"last update":"Fri Sep 13 18:21:52 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.66700000000000004,"15min":0.67900000000000005},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.77900000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.63900000000000001,"15min":0.66200000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.69199999999999995,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.91200000000000003}]},{"osd":11,"last update":"Fri Sep 13 18:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.61099999999999999,"15min":0.60599999999999998},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64800000000000002,"15min":0.64900000000000002},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.76400000000000001,"5min":0.872,"15min":0.872},"last":0.85699999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1122304,"data_stored":596184,"data_compressed":28087,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:21:50 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.49299999999999999,"15min":0.48499999999999999},"min":{"1min":0.40600000000000003,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.503,"15min":0.48299999999999998},"min":{"1min":0.39600000000000002,"5min":0.28799999999999998,"15min":0.26300000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.45800000000000002}]},{"osd":1,"last update":"Fri Sep 13 18:21:50 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.54000000000000004,"15min":0.48799999999999999},"min":{"1min":0.46300000000000002,"5min":0.318,"15min":0.27400000000000002},"max":{"1min":0.81299999999999994,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.56899999999999995,"15min":0.49099999999999999},"min":{"1min":0.45000000000000001,"5min":0.32300000000000001,"15min":0.29099999999999998},"max":{"1min":0.79000000000000004,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.48899999999999999}]},{"osd":3,"last update":"Fri Sep 13 18:22:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55100000000000005,"15min":0.52400000000000002},"min":{"1min":0.41299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58199999999999996,"15min":0.56100000000000005},"min":{"1min":0.40799999999999997,"5min":0.34000000000000002,"15min":0.33300000000000002},"max":{"1min":0.79700000000000004,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.56799999999999995}]},{"osd":4,"last update":"Fri Sep 13 18:22:04 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.62,"15min":0.61699999999999999},"min":{"1min":0.495,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.75900000000000001,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.66900000000000004,"15min":0.66600000000000004},"min":{"1min":0.56899999999999995,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.82199999999999995,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.437}]},{"osd":5,"last update":"Fri Sep 13 18:21:53 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.42699999999999999,"15min":0.38900000000000001},"min":{"1min":0.27100000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.314},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.44600000000000001,"15min":0.40500000000000003},"min":{"1min":0.32100000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.67800000000000005,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.35299999999999998}]},{"osd":6,"last update":"Fri Sep 13 18:22:25 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.621,"15min":0.55900000000000005},"min":{"1min":0.52900000000000003,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.85099999999999998,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.626,"15min":0.57899999999999996},"min":{"1min":0.47799999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.82999999999999996,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.52400000000000002}]},{"osd":7,"last update":"Fri Sep 13 18:22:09 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.68899999999999995,"15min":0.68799999999999994},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.85899999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.67000000000000004,"15min":0.66800000000000004},"min":{"1min":0.495,"5min":0.434,"15min":0.434},"max":{"1min":0.86099999999999999,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.46700000000000003}]},{"osd":8,"last update":"Fri Sep 13 18:22:09 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.36299999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.79900000000000004,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.37},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.61599999999999999,"15min":0.59399999999999997},"min":{"1min":0.45800000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.86699999999999999,"5min":0.998,"15min":0.998},"last":0.28999999999999998}]},{"osd":9,"last update":"Fri Sep 13 18:21:28 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.63700000000000001,"15min":0.625},"min":{"1min":0.50900000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.86399999999999999,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.5},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.66900000000000004,"15min":0.65500000000000003},"min":{"1min":0.50900000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.96499999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.55000000000000004}]},{"osd":10,"last update":"Fri Sep 13 18:21:50 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.67600000000000005,"15min":0.63700000000000001},"min":{"1min":0.63900000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.93999999999999995,"5min":1.016,"15min":1.016},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.68899999999999995,"15min":0.65300000000000002},"min":{"1min":0.623,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.93600000000000005,"5min":1.042,"15min":1.042},"last":0.50900000000000001}]},{"osd":11,"last update":"Fri Sep 13 18:22:00 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.72499999999999998,"15min":0.71599999999999997},"min":{"1min":0.626,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.94699999999999995,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.73099999999999998,"15min":0.72299999999999998},"min":{"1min":0.54100000000000004,"5min":0.433,"15min":0.433},"max":{"1min":0.92000000000000004,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.54000000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1101824,"data_stored":580429,"data_compressed":25120,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:22:25 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.502,"15min":0.49299999999999999},"min":{"1min":0.376,"5min":0.30299999999999999,"15min":0.28399999999999997},"max":{"1min":0.67100000000000004,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55000000000000004,"15min":0.52100000000000002},"min":{"1min":0.40600000000000003,"5min":0.33200000000000002,"15min":0.31},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.40600000000000003}]},{"osd":2,"last update":"Fri Sep 13 18:21:53 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.56799999999999995,"15min":0.51200000000000001},"min":{"1min":0.39200000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.433},{"interface":"front","average":{"1min":0.622,"5min":0.52000000000000002,"15min":0.48599999999999999},"min":{"1min":0.33800000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.38700000000000001}]},{"osd":3,"last update":"Fri Sep 13 18:22:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55900000000000005,"15min":0.52700000000000002},"min":{"1min":0.374,"5min":0.31,"15min":0.31},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.503},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.60699999999999998,"15min":0.55800000000000005},"min":{"1min":0.47399999999999998,"5min":0.39700000000000002,"15min":0.34399999999999997},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.57099999999999995}]},{"osd":4,"last update":"Fri Sep 13 18:21:34 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.41099999999999998,"15min":0.371},"min":{"1min":0.29499999999999998,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.755,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.40500000000000003,"15min":0.36499999999999999},"min":{"1min":0.29599999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.65600000000000003,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.44700000000000001}]},{"osd":5,"last update":"Fri Sep 13 18:21:57 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.63400000000000001,"15min":0.58399999999999996},"min":{"1min":0.41899999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.60699999999999998,"15min":0.55500000000000005},"min":{"1min":0.47199999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85799999999999998,"5min":0.875,"15min":0.875},"last":0.41699999999999998}]},{"osd":6,"last update":"Fri Sep 13 18:22:25 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.59699999999999998,"15min":0.57399999999999995},"min":{"1min":0.46100000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.65400000000000003,"15min":0.622},"min":{"1min":0.54100000000000004,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.54800000000000004}]},{"osd":7,"last update":"Fri Sep 13 18:21:40 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.51700000000000002,"15min":0.495},"min":{"1min":0.312,"5min":0.246,"15min":0.246},"max":{"1min":0.74299999999999999,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.52200000000000002,"15min":0.50700000000000001},"min":{"1min":0.35499999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.82499999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.58699999999999997}]},{"osd":8,"last update":"Fri Sep 13 18:22:11 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.65400000000000003,"15min":0.626},"min":{"1min":0.48499999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.90300000000000002,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.747,"5min":0.66700000000000004,"15min":0.64200000000000002},"min":{"1min":0.57299999999999995,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.56000000000000005}]},{"osd":9,"last update":"Fri Sep 13 18:21:34 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.63600000000000001,"15min":0.64200000000000002},"min":{"1min":0.49399999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.84399999999999997,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.64000000000000001,"15min":0.64200000000000002},"min":{"1min":0.433,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.61599999999999999}]},{"osd":10,"last update":"Fri Sep 13 18:21:53 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.63100000000000001,"15min":0.60299999999999998},"min":{"1min":0.442,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.63300000000000001,"15min":0.61499999999999999},"min":{"1min":0.39800000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.52400000000000002}]},{"osd":11,"last update":"Fri Sep 13 18:22:06 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.80200000000000005,"15min":0.79700000000000004},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.623},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.57999999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":1052672,"data_stored":516690,"data_compressed":23512,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 18:21:28 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.50700000000000001,"15min":0.498},"min":{"1min":0.36799999999999999,"5min":0.318,"15min":0.30199999999999999},"max":{"1min":0.76900000000000002,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.497},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.52100000000000002,"15min":0.50900000000000001},"min":{"1min":0.434,"5min":0.307,"15min":0.307},"max":{"1min":0.73999999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.33000000000000002}]},{"osd":2,"last update":"Fri Sep 13 18:21:49 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.53400000000000003,"15min":0.52900000000000003},"min":{"1min":0.40699999999999997,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.70699999999999996,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.623},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.55700000000000005,"15min":0.52000000000000002},"min":{"1min":0.42899999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.69699999999999995,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.437}]},{"osd":3,"last update":"Fri Sep 13 18:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.40999999999999998,"15min":0.39200000000000002},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":2.9750000000000001,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.38200000000000001,"15min":0.38500000000000001},"min":{"1min":0.26500000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.61299999999999999,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.41599999999999998}]},{"osd":4,"last update":"Fri Sep 13 18:21:39 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52100000000000002,"15min":0.497},"min":{"1min":0.41299999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.54100000000000004,"15min":0.51500000000000001},"min":{"1min":0.36399999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76000000000000001,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.378}]},{"osd":5,"last update":"Fri Sep 13 18:21:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.61399999999999999,"15min":0.56599999999999995},"min":{"1min":0.35199999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.68300000000000005,"5min":7.194,"15min":7.194},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.53300000000000003},"min":{"1min":0.44500000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.84999999999999998,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.54300000000000004}]},{"osd":6,"last update":"Fri Sep 13 18:22:24 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.47699999999999998,"15min":0.45900000000000002},"min":{"1min":0.33000000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.437,"15min":0.45400000000000001},"min":{"1min":0.316,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.73899999999999999,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.40100000000000002}]},{"osd":7,"last update":"Fri Sep 13 18:22:09 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.59699999999999998,"15min":0.59399999999999997},"min":{"1min":0.44900000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.68799999999999994}]},{"osd":8,"last update":"Fri Sep 13 18:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58999999999999997,"15min":0.60399999999999998},"min":{"1min":0.41899999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.61799999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.65700000000000003}]},{"osd":9,"last update":"Fri Sep 13 18:21:33 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.58499999999999996,"15min":0.60699999999999998},"min":{"1min":0.33700000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78200000000000003,"5min":1.089,"15min":1.089},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.54100000000000004,"15min":0.57299999999999995},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.61799999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.61599999999999999}]},{"osd":10,"last update":"Fri Sep 13 18:21:49 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.65900000000000003,"15min":0.67700000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88800000000000001,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.749},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.68300000000000005,"15min":0.69799999999999995},"min":{"1min":0.51900000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.84599999999999997,"5min":1.048,"15min":1.048},"last":0.63800000000000001}]},{"osd":11,"last update":"Fri Sep 13 18:22:20 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.623,"15min":0.63100000000000001},"min":{"1min":0.44600000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68100000000000005,"15min":0.68300000000000005},"min":{"1min":0.45900000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.71999999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35824,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35824,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35824,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":72245,"data_compressed":1608,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":72295,"data_compressed":1608,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135258,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":6440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":394694,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":203717,"data_compressed":7764,"data_compressed_allocated":12288,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:22:33.563 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:22:33.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.690+0000 7fc110e72640 1 -- 172.21.15.114:0/2898484696 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc10c100f10 msgr2=0x7fc10c1012f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:33.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.690+0000 7fc110e72640 1 --2- 172.21.15.114:0/2898484696 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc10c100f10 0x7fc10c1012f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc009900 tx=0x7fc0fc02a480 comp rx=0 tx=0).stop 2024-09-13T18:22:33.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.690+0000 7fc110e72640 1 -- 172.21.15.114:0/2898484696 shutdown_connections 2024-09-13T18:22:33.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.690+0000 7fc110e72640 1 --2- 172.21.15.114:0/2898484696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc10c10a360 0x7fc10c10c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:33.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.690+0000 7fc110e72640 1 --2- 172.21.15.114:0/2898484696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc10c101830 0x7fc10c109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:33.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.690+0000 7fc110e72640 1 --2- 172.21.15.114:0/2898484696 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc10c100f10 0x7fc10c1012f0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:33.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.690+0000 7fc110e72640 1 -- 172.21.15.114:0/2898484696 >> 172.21.15.114:0/2898484696 conn(0x7fc10c0fa900 msgr2=0x7fc10c0fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:33.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.690+0000 7fc110e72640 1 -- 172.21.15.114:0/2898484696 shutdown_connections 2024-09-13T18:22:33.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.690+0000 7fc110e72640 1 -- 172.21.15.114:0/2898484696 wait complete. 2024-09-13T18:22:33.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.691+0000 7fc110e72640 1 Processor -- start 2024-09-13T18:22:33.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.691+0000 7fc110e72640 1 -- start start 2024-09-13T18:22:33.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.691+0000 7fc110e72640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc10c100f10 0x7fc10c196190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:33.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.691+0000 7fc110e72640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc10c101830 0x7fc10c1966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:33.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.691+0000 7fc110e72640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc10c10a360 0x7fc10c19aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:33.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.691+0000 7fc110e72640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc10c19b0c0 con 0x7fc10c101830 2024-09-13T18:22:33.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.691+0000 7fc110e72640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc10c19b230 con 0x7fc10c10a360 2024-09-13T18:22:33.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.691+0000 7fc110e72640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc10c19b3f0 con 0x7fc10c100f10 2024-09-13T18:22:33.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.691+0000 7fc109d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc10c101830 0x7fc10c1966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:33.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.692+0000 7fc109d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc10c101830 0x7fc10c1966d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50816/0 (socket says 172.21.15.114:50816) 2024-09-13T18:22:33.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.692+0000 7fc109d9b640 1 -- 172.21.15.114:0/3092377467 learned_addr learned my addr 172.21.15.114:0/3092377467 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:33.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.692+0000 7fc10a59c640 1 --2- 172.21.15.114:0/3092377467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc10c100f10 0x7fc10c196190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:33.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.692+0000 7fc10ad9d640 1 --2- 172.21.15.114:0/3092377467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc10c10a360 0x7fc10c19aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:33.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.692+0000 7fc109d9b640 1 -- 172.21.15.114:0/3092377467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc10c100f10 msgr2=0x7fc10c196190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:33.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.692+0000 7fc109d9b640 1 --2- 172.21.15.114:0/3092377467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc10c100f10 0x7fc10c196190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:33.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.692+0000 7fc109d9b640 1 -- 172.21.15.114:0/3092377467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc10c10a360 msgr2=0x7fc10c19aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:33.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.692+0000 7fc109d9b640 1 --2- 172.21.15.114:0/3092377467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc10c10a360 0x7fc10c19aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:33.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.692+0000 7fc109d9b640 1 -- 172.21.15.114:0/3092377467 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0fc009560 con 0x7fc10c101830 2024-09-13T18:22:33.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.692+0000 7fc10a59c640 1 --2- 172.21.15.114:0/3092377467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc10c100f10 0x7fc10c196190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:22:33.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.692+0000 7fc10ad9d640 1 --2- 172.21.15.114:0/3092377467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc10c10a360 0x7fc10c19aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:22:33.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.692+0000 7fc109d9b640 1 --2- 172.21.15.114:0/3092377467 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc10c101830 0x7fc10c1966d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc10000d750 tx=0x7fc10000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:33.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.692+0000 7fc0fb7fe640 1 -- 172.21.15.114:0/3092377467 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc100009a30 con 0x7fc10c101830 2024-09-13T18:22:33.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.692+0000 7fc110e72640 1 -- 172.21.15.114:0/3092377467 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc10c19b6d0 con 0x7fc10c101830 2024-09-13T18:22:33.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.692+0000 7fc0fb7fe640 1 -- 172.21.15.114:0/3092377467 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc1000054b0 con 0x7fc10c101830 2024-09-13T18:22:33.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.692+0000 7fc0fb7fe640 1 -- 172.21.15.114:0/3092377467 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc100004cf0 con 0x7fc10c101830 2024-09-13T18:22:33.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.692+0000 7fc110e72640 1 -- 172.21.15.114:0/3092377467 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc10c1a2f50 con 0x7fc10c101830 2024-09-13T18:22:33.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.693+0000 7fc110e72640 1 -- 172.21.15.114:0/3092377467 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc10c068ed0 con 0x7fc10c101830 2024-09-13T18:22:33.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.694+0000 7fc0fb7fe640 1 -- 172.21.15.114:0/3092377467 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 24) v1 ==== 136050+0+0 (secure 0 0 0) 0x7fc100009b90 con 0x7fc10c101830 2024-09-13T18:22:33.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.695+0000 7fc0fb7fe640 1 --2- 172.21.15.114:0/3092377467 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc0f40a1290 0x7fc0f40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:33.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.695+0000 7fc10a59c640 1 --2- 172.21.15.114:0/3092377467 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc0f40a1290 0x7fc0f40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:33.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.695+0000 7fc0fb7fe640 1 -- 172.21.15.114:0/3092377467 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fc1000cd030 con 0x7fc10c101830 2024-09-13T18:22:33.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.695+0000 7fc10a59c640 1 --2- 172.21.15.114:0/3092377467 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc0f40a1290 0x7fc0f40a3750 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc000c00 tx=0x7fc0fc005e00 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:33.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.698+0000 7fc0fb7fe640 1 -- 172.21.15.114:0/3092377467 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fc100095fc0 con 0x7fc10c101830 2024-09-13T18:22:33.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.861+0000 7fc110e72640 1 -- 172.21.15.114:0/3092377467 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc10c1053d0 con 0x7fc10c101830 2024-09-13T18:22:33.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.862+0000 7fc0fb7fe640 1 -- 172.21.15.114:0/3092377467 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+25079 (secure 0 0 0) 0x7fc100017020 con 0x7fc10c101830 2024-09-13T18:22:33.866 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:22:33.866 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":94,"fsid":"97e7edf2-71fb-11ef-bceb-c7b262605968","created":"2024-09-13T18:12:36.360625+0000","modified":"2024-09-13T18:22:28.057664+0000","last_up_change":"2024-09-13T18:19:12.350370+0000","last_in_change":"2024-09-13T18:18:52.321596+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:15:40.129659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:20:00.024407+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:20:01.085549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T18:20:04.334216+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9914f01a-9f6f-4878-b955-83570199c354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6803","nonce":3499525826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6805","nonce":3499525826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6809","nonce":3499525826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6807","nonce":3499525826}]},"public_addr":"172.21.15.114:6803/3499525826","cluster_addr":"172.21.15.114:6805/3499525826","heartbeat_back_addr":"172.21.15.114:6809/3499525826","heartbeat_front_addr":"172.21.15.114:6807/3499525826","state":["exists","up"]},{"osd":1,"uuid":"ef65e012-7469-4c20-b354-17022bca63c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6801","nonce":4264264866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6803","nonce":4264264866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6807","nonce":4264264866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6805","nonce":4264264866}]},"public_addr":"172.21.15.134:6801/4264264866","cluster_addr":"172.21.15.134:6803/4264264866","heartbeat_back_addr":"172.21.15.134:6807/4264264866","heartbeat_front_addr":"172.21.15.134:6805/4264264866","state":["exists","up"]},{"osd":2,"uuid":"578213fd-715f-4be7-91fa-254f3d0e3022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6801","nonce":2288807707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6803","nonce":2288807707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6807","nonce":2288807707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6805","nonce":2288807707}]},"public_addr":"172.21.15.204:6801/2288807707","cluster_addr":"172.21.15.204:6803/2288807707","heartbeat_back_addr":"172.21.15.204:6807/2288807707","heartbeat_front_addr":"172.21.15.204:6805/2288807707","state":["exists","up"]},{"osd":3,"uuid":"b64aab3f-1e52-4828-8397-74af5e05f2cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6811","nonce":2904204254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6813","nonce":2904204254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6817","nonce":2904204254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6815","nonce":2904204254}]},"public_addr":"172.21.15.114:6811/2904204254","cluster_addr":"172.21.15.114:6813/2904204254","heartbeat_back_addr":"172.21.15.114:6817/2904204254","heartbeat_front_addr":"172.21.15.114:6815/2904204254","state":["exists","up"]},{"osd":4,"uuid":"f38943d6-39c0-42e1-b427-cb6bb90ac38a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6809","nonce":1773945333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6811","nonce":1773945333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6815","nonce":1773945333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6813","nonce":1773945333}]},"public_addr":"172.21.15.134:6809/1773945333","cluster_addr":"172.21.15.134:6811/1773945333","heartbeat_back_addr":"172.21.15.134:6815/1773945333","heartbeat_front_addr":"172.21.15.134:6813/1773945333","state":["exists","up"]},{"osd":5,"uuid":"bb38f361-8e00-4178-9752-5ddbf3af20e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6809","nonce":2371778055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6811","nonce":2371778055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6815","nonce":2371778055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6813","nonce":2371778055}]},"public_addr":"172.21.15.204:6809/2371778055","cluster_addr":"172.21.15.204:6811/2371778055","heartbeat_back_addr":"172.21.15.204:6815/2371778055","heartbeat_front_addr":"172.21.15.204:6813/2371778055","state":["exists","up"]},{"osd":6,"uuid":"93d40a03-e7ff-4a3f-9efa-5c5403c9983d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6819","nonce":1484325044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6821","nonce":1484325044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6825","nonce":1484325044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6823","nonce":1484325044}]},"public_addr":"172.21.15.114:6819/1484325044","cluster_addr":"172.21.15.114:6821/1484325044","heartbeat_back_addr":"172.21.15.114:6825/1484325044","heartbeat_front_addr":"172.21.15.114:6823/1484325044","state":["exists","up"]},{"osd":7,"uuid":"982f629a-1b95-4be1-8e75-5ddce1ad77b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6817","nonce":799246614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6819","nonce":799246614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6823","nonce":799246614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6821","nonce":799246614}]},"public_addr":"172.21.15.134:6817/799246614","cluster_addr":"172.21.15.134:6819/799246614","heartbeat_back_addr":"172.21.15.134:6823/799246614","heartbeat_front_addr":"172.21.15.134:6821/799246614","state":["exists","up"]},{"osd":8,"uuid":"409beb9b-8227-4dad-a4a0-f0e5ce70dc5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6817","nonce":858331805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6819","nonce":858331805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6823","nonce":858331805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6821","nonce":858331805}]},"public_addr":"172.21.15.204:6817/858331805","cluster_addr":"172.21.15.204:6819/858331805","heartbeat_back_addr":"172.21.15.204:6823/858331805","heartbeat_front_addr":"172.21.15.204:6821/858331805","state":["exists","up"]},{"osd":9,"uuid":"a345a15c-b1fc-4877-b445-418eebf0d0ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6827","nonce":2172392854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6829","nonce":2172392854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6833","nonce":2172392854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6831","nonce":2172392854}]},"public_addr":"172.21.15.114:6827/2172392854","cluster_addr":"172.21.15.114:6829/2172392854","heartbeat_back_addr":"172.21.15.114:6833/2172392854","heartbeat_front_addr":"172.21.15.114:6831/2172392854","state":["exists","up"]},{"osd":10,"uuid":"464d995f-d90b-4caa-addb-1f7562bd74b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6825","nonce":3073513468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6827","nonce":3073513468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6831","nonce":3073513468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6829","nonce":3073513468}]},"public_addr":"172.21.15.134:6825/3073513468","cluster_addr":"172.21.15.134:6827/3073513468","heartbeat_back_addr":"172.21.15.134:6831/3073513468","heartbeat_front_addr":"172.21.15.134:6829/3073513468","state":["exists","up"]},{"osd":11,"uuid":"71dd794d-326c-477f-8d02-ee32bff1350d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6825","nonce":3239521765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6827","nonce":3239521765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6831","nonce":3239521765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6829","nonce":3239521765}]},"public_addr":"172.21.15.204:6825/3239521765","cluster_addr":"172.21.15.204:6827/3239521765","heartbeat_back_addr":"172.21.15.204:6831/3239521765","heartbeat_front_addr":"172.21.15.204:6829/3239521765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:14:54.542687+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:15:15.746055+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:15:38.371131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:01.383932+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:24.796237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:48.422214+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:11.284323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:34.570365+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:58.232855+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:18:21.609596+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:18:45.565859+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:19:10.017969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:6801/3396185541":"2024-09-14T18:22:28.057623+0000","172.21.15.114:6800/830738794":"2024-09-14T18:13:03.521116+0000","172.21.15.114:6801/830738794":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/1684954293":"2024-09-14T18:22:28.057623+0000","172.21.15.114:0/3544951322":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/722571999":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/2949422849":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2034742251":"2024-09-14T18:13:03.521116+0000","172.21.15.114:6800/4281866858":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/3407523320":"2024-09-14T18:22:28.057623+0000","172.21.15.114:6800/3396185541":"2024-09-14T18:22:28.057623+0000","172.21.15.114:0/4257828906":"2024-09-14T18:13:25.823027+0000","172.21.15.114:6801/4281866858":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2828364892":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2714091073":"2024-09-14T18:22:28.057623+0000","172.21.15.114:0/842237781":"2024-09-14T18:22:28.057623+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:22:33.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.864+0000 7fc110e72640 1 -- 172.21.15.114:0/3092377467 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc0f40a1290 msgr2=0x7fc0f40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:33.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.864+0000 7fc110e72640 1 --2- 172.21.15.114:0/3092377467 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc0f40a1290 0x7fc0f40a3750 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc000c00 tx=0x7fc0fc005e00 comp rx=0 tx=0).stop 2024-09-13T18:22:33.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.864+0000 7fc110e72640 1 -- 172.21.15.114:0/3092377467 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc10c101830 msgr2=0x7fc10c1966d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:33.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.864+0000 7fc110e72640 1 --2- 172.21.15.114:0/3092377467 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc10c101830 0x7fc10c1966d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc10000d750 tx=0x7fc10000dc10 comp rx=0 tx=0).stop 2024-09-13T18:22:33.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.864+0000 7fc110e72640 1 -- 172.21.15.114:0/3092377467 shutdown_connections 2024-09-13T18:22:33.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.864+0000 7fc110e72640 1 --2- 172.21.15.114:0/3092377467 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc0f40a1290 0x7fc0f40a3750 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:33.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.864+0000 7fc110e72640 1 --2- 172.21.15.114:0/3092377467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc10c10a360 0x7fc10c19aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:33.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.864+0000 7fc110e72640 1 --2- 172.21.15.114:0/3092377467 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc10c101830 0x7fc10c1966d0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:33.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.864+0000 7fc110e72640 1 --2- 172.21.15.114:0/3092377467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc10c100f10 0x7fc10c196190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:33.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.864+0000 7fc110e72640 1 -- 172.21.15.114:0/3092377467 >> 172.21.15.114:0/3092377467 conn(0x7fc10c0fa900 msgr2=0x7fc10c0fc1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:33.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.864+0000 7fc110e72640 1 -- 172.21.15.114:0/3092377467 shutdown_connections 2024-09-13T18:22:33.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.864+0000 7fc110e72640 1 -- 172.21.15.114:0/3092377467 wait complete. 2024-09-13T18:22:33.878 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T18:15:40.129659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T18:20:00.024407+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T18:20:01.085549+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T18:20:04.334216+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T18:22:33.879 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T18:22:33.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.981+0000 7f6f2da02640 1 -- 172.21.15.114:0/3916931846 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f28103f40 msgr2=0x7f6f2810ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:33.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.981+0000 7f6f2da02640 1 --2- 172.21.15.114:0/3916931846 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f28103f40 0x7f6f2810ba90 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6f20009cd0 tx=0x7f6f2002a8e0 comp rx=0 tx=0).stop 2024-09-13T18:22:33.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.981+0000 7f6f2da02640 1 -- 172.21.15.114:0/3916931846 shutdown_connections 2024-09-13T18:22:33.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.981+0000 7f6f2da02640 1 --2- 172.21.15.114:0/3916931846 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f28103f40 0x7f6f2810ba90 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:33.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.981+0000 7f6f2da02640 1 --2- 172.21.15.114:0/3916931846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f280ff7a0 0x7f6f281037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:33.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.981+0000 7f6f2da02640 1 --2- 172.21.15.114:0/3916931846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f280fedf0 0x7f6f280ff1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:33.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.981+0000 7f6f2da02640 1 -- 172.21.15.114:0/3916931846 >> 172.21.15.114:0/3916931846 conn(0x7f6f280fa840 msgr2=0x7f6f280fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:33.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.981+0000 7f6f2da02640 1 -- 172.21.15.114:0/3916931846 shutdown_connections 2024-09-13T18:22:33.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.981+0000 7f6f2da02640 1 -- 172.21.15.114:0/3916931846 wait complete. 2024-09-13T18:22:33.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.981+0000 7f6f2da02640 1 Processor -- start 2024-09-13T18:22:33.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.982+0000 7f6f2da02640 1 -- start start 2024-09-13T18:22:33.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.982+0000 7f6f2da02640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f280fedf0 0x7f6f28196160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:33.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.982+0000 7f6f2da02640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f28103f40 0x7f6f281966a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:33.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.982+0000 7f6f2da02640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f2819aa90 0x7f6f2819af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:33.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.982+0000 7f6f2da02640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2819b480 con 0x7f6f280fedf0 2024-09-13T18:22:33.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.982+0000 7f6f2da02640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2819b5f0 con 0x7f6f28103f40 2024-09-13T18:22:33.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.982+0000 7f6f2da02640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2819b760 con 0x7f6f2819aa90 2024-09-13T18:22:33.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.982+0000 7f6f26ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f280fedf0 0x7f6f28196160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:33.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.982+0000 7f6f26ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f280fedf0 0x7f6f28196160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50828/0 (socket says 172.21.15.114:50828) 2024-09-13T18:22:33.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.982+0000 7f6f26ffd640 1 -- 172.21.15.114:0/4250266003 learned_addr learned my addr 172.21.15.114:0/4250266003 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:33.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.982+0000 7f6f277fe640 1 --2- 172.21.15.114:0/4250266003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f2819aa90 0x7f6f2819af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:33.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.982+0000 7f6f267fc640 1 --2- 172.21.15.114:0/4250266003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f28103f40 0x7f6f281966a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:33.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.982+0000 7f6f26ffd640 1 -- 172.21.15.114:0/4250266003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f2819aa90 msgr2=0x7f6f2819af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:33.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.982+0000 7f6f26ffd640 1 --2- 172.21.15.114:0/4250266003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f2819aa90 0x7f6f2819af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:33.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.982+0000 7f6f26ffd640 1 -- 172.21.15.114:0/4250266003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f28103f40 msgr2=0x7f6f281966a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:33.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.982+0000 7f6f26ffd640 1 --2- 172.21.15.114:0/4250266003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f28103f40 0x7f6f281966a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:33.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.982+0000 7f6f26ffd640 1 -- 172.21.15.114:0/4250266003 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f20009900 con 0x7f6f280fedf0 2024-09-13T18:22:33.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.982+0000 7f6f26ffd640 1 --2- 172.21.15.114:0/4250266003 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f280fedf0 0x7f6f28196160 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f6f1800c890 tx=0x7f6f1800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:33.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.983+0000 7f6f2ca00640 1 -- 172.21.15.114:0/4250266003 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f18004220 con 0x7f6f280fedf0 2024-09-13T18:22:33.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.983+0000 7f6f2da02640 1 -- 172.21.15.114:0/4250266003 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f281a2d40 con 0x7f6f280fedf0 2024-09-13T18:22:33.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.983+0000 7f6f2ca00640 1 -- 172.21.15.114:0/4250266003 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6f18007e40 con 0x7f6f280fedf0 2024-09-13T18:22:33.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.983+0000 7f6f2ca00640 1 -- 172.21.15.114:0/4250266003 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f18004a00 con 0x7f6f280fedf0 2024-09-13T18:22:33.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.983+0000 7f6f2da02640 1 -- 172.21.15.114:0/4250266003 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f281a31a0 con 0x7f6f280fedf0 2024-09-13T18:22:33.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.984+0000 7f6f2da02640 1 -- 172.21.15.114:0/4250266003 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f281002c0 con 0x7f6f280fedf0 2024-09-13T18:22:33.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.985+0000 7f6f2ca00640 1 -- 172.21.15.114:0/4250266003 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 24) v1 ==== 136050+0+0 (secure 0 0 0) 0x7f6f180079b0 con 0x7f6f280fedf0 2024-09-13T18:22:33.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.985+0000 7f6f2ca00640 1 --2- 172.21.15.114:0/4250266003 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f6f140a1290 0x7f6f140a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:33.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.986+0000 7f6f2ca00640 1 -- 172.21.15.114:0/4250266003 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f6f180cd210 con 0x7f6f280fedf0 2024-09-13T18:22:33.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.986+0000 7f6f267fc640 1 --2- 172.21.15.114:0/4250266003 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f6f140a1290 0x7f6f140a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:33.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.986+0000 7f6f267fc640 1 --2- 172.21.15.114:0/4250266003 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f6f140a1290 0x7f6f140a3750 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6f28197700 tx=0x7f6f10008040 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:33.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:33.988+0000 7f6f2ca00640 1 -- 172.21.15.114:0/4250266003 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f6f180961a0 con 0x7f6f280fedf0 2024-09-13T18:22:34.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.155+0000 7f6f2da02640 1 -- 172.21.15.114:0/4250266003 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6f280ff1d0 con 0x7f6f280fedf0 2024-09-13T18:22:34.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.155+0000 7f6f2ca00640 1 -- 172.21.15.114:0/4250266003 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6f1801d0a0 con 0x7f6f280fedf0 2024-09-13T18:22:34.158 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 1 2024-09-13T18:22:34.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.157+0000 7f6f2da02640 1 -- 172.21.15.114:0/4250266003 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f6f140a1290 msgr2=0x7f6f140a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:34.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.157+0000 7f6f2da02640 1 --2- 172.21.15.114:0/4250266003 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f6f140a1290 0x7f6f140a3750 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6f28197700 tx=0x7f6f10008040 comp rx=0 tx=0).stop 2024-09-13T18:22:34.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.157+0000 7f6f2da02640 1 -- 172.21.15.114:0/4250266003 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f280fedf0 msgr2=0x7f6f28196160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:34.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.157+0000 7f6f2da02640 1 --2- 172.21.15.114:0/4250266003 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f280fedf0 0x7f6f28196160 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f6f1800c890 tx=0x7f6f1800cd50 comp rx=0 tx=0).stop 2024-09-13T18:22:34.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.157+0000 7f6f2da02640 1 -- 172.21.15.114:0/4250266003 shutdown_connections 2024-09-13T18:22:34.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.157+0000 7f6f2da02640 1 --2- 172.21.15.114:0/4250266003 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f6f140a1290 0x7f6f140a3750 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:34.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.157+0000 7f6f2da02640 1 --2- 172.21.15.114:0/4250266003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f2819aa90 0x7f6f2819af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:34.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.157+0000 7f6f2da02640 1 --2- 172.21.15.114:0/4250266003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f28103f40 0x7f6f281966a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:34.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.157+0000 7f6f2da02640 1 --2- 172.21.15.114:0/4250266003 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f280fedf0 0x7f6f28196160 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:34.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.157+0000 7f6f2da02640 1 -- 172.21.15.114:0/4250266003 >> 172.21.15.114:0/4250266003 conn(0x7f6f280fa840 msgr2=0x7f6f280fbd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:34.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.157+0000 7f6f2da02640 1 -- 172.21.15.114:0/4250266003 shutdown_connections 2024-09-13T18:22:34.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.157+0000 7f6f2da02640 1 -- 172.21.15.114:0/4250266003 wait complete. 2024-09-13T18:22:34.170 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:34 smithi114 ceph-mon[30897]: from='client.15684 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T18:22:34.170 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:34 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/3092377467' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:22:34.171 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T18:22:34.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.270+0000 7f698b126640 1 -- 172.21.15.114:0/3741499933 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69840ff530 msgr2=0x7f6984103130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:34.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.270+0000 7f698b126640 1 --2- 172.21.15.114:0/3741499933 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69840ff530 0x7f6984103130 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f6974009900 tx=0x7f697402a400 comp rx=0 tx=0).stop 2024-09-13T18:22:34.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.270+0000 7f698b126640 1 -- 172.21.15.114:0/3741499933 shutdown_connections 2024-09-13T18:22:34.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.270+0000 7f698b126640 1 --2- 172.21.15.114:0/3741499933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6984103840 0x7f698410b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:34.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.270+0000 7f698b126640 1 --2- 172.21.15.114:0/3741499933 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69840ff530 0x7f6984103130 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:34.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.270+0000 7f698b126640 1 --2- 172.21.15.114:0/3741499933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69840feb80 0x7f69840fef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:34.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.270+0000 7f698b126640 1 -- 172.21.15.114:0/3741499933 >> 172.21.15.114:0/3741499933 conn(0x7f6984074110 msgr2=0x7f6984074520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:34.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.270+0000 7f698b126640 1 -- 172.21.15.114:0/3741499933 shutdown_connections 2024-09-13T18:22:34.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.270+0000 7f698b126640 1 -- 172.21.15.114:0/3741499933 wait complete. 2024-09-13T18:22:34.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.271+0000 7f698b126640 1 Processor -- start 2024-09-13T18:22:34.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.271+0000 7f698b126640 1 -- start start 2024-09-13T18:22:34.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.271+0000 7f698b126640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69840feb80 0x7f6984196360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:34.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.271+0000 7f698b126640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6984103840 0x7f69841968a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:34.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.271+0000 7f698b126640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f698419ac90 0x7f698419b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:34.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.271+0000 7f698b126640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f698419b680 con 0x7f6984103840 2024-09-13T18:22:34.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.271+0000 7f698b126640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f698419b7f0 con 0x7f698419ac90 2024-09-13T18:22:34.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.271+0000 7f698b126640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f698419b960 con 0x7f69840feb80 2024-09-13T18:22:34.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.271+0000 7f697bfff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6984103840 0x7f69841968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:34.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.271+0000 7f697bfff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6984103840 0x7f69841968a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50840/0 (socket says 172.21.15.114:50840) 2024-09-13T18:22:34.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.271+0000 7f697bfff640 1 -- 172.21.15.114:0/2743356852 learned_addr learned my addr 172.21.15.114:0/2743356852 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:34.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.272+0000 7f69896c3640 1 --2- 172.21.15.114:0/2743356852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f698419ac90 0x7f698419b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:34.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.272+0000 7f697bfff640 1 -- 172.21.15.114:0/2743356852 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69840feb80 msgr2=0x7f6984196360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:34.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.272+0000 7f697bfff640 1 --2- 172.21.15.114:0/2743356852 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69840feb80 0x7f6984196360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:34.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.272+0000 7f697bfff640 1 -- 172.21.15.114:0/2743356852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f698419ac90 msgr2=0x7f698419b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:34.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.272+0000 7f697bfff640 1 --2- 172.21.15.114:0/2743356852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f698419ac90 0x7f698419b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:34.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.272+0000 7f697bfff640 1 -- 172.21.15.114:0/2743356852 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6974009560 con 0x7f6984103840 2024-09-13T18:22:34.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.272+0000 7f69896c3640 1 --2- 172.21.15.114:0/2743356852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f698419ac90 0x7f698419b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:22:34.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.272+0000 7f697bfff640 1 --2- 172.21.15.114:0/2743356852 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6984103840 0x7f69841968a0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f697402a920 tx=0x7f69740048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:34.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.272+0000 7f6979ffb640 1 -- 172.21.15.114:0/2743356852 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69740035f0 con 0x7f6984103840 2024-09-13T18:22:34.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.272+0000 7f6979ffb640 1 -- 172.21.15.114:0/2743356852 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6974003be0 con 0x7f6984103840 2024-09-13T18:22:34.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.272+0000 7f6979ffb640 1 -- 172.21.15.114:0/2743356852 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697403cb30 con 0x7f6984103840 2024-09-13T18:22:34.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.272+0000 7f698b126640 1 -- 172.21.15.114:0/2743356852 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69841a2f40 con 0x7f6984103840 2024-09-13T18:22:34.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.272+0000 7f698b126640 1 -- 172.21.15.114:0/2743356852 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69841a32b0 con 0x7f6984103840 2024-09-13T18:22:34.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.273+0000 7f698b126640 1 -- 172.21.15.114:0/2743356852 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69840ffbf0 con 0x7f6984103840 2024-09-13T18:22:34.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.274+0000 7f6979ffb640 1 -- 172.21.15.114:0/2743356852 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 24) v1 ==== 136050+0+0 (secure 0 0 0) 0x7f6974003750 con 0x7f6984103840 2024-09-13T18:22:34.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.275+0000 7f6979ffb640 1 --2- 172.21.15.114:0/2743356852 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f696c0a1310 0x7f696c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:34.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.275+0000 7f6979ffb640 1 -- 172.21.15.114:0/2743356852 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f69740eeab0 con 0x7f6984103840 2024-09-13T18:22:34.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.275+0000 7f6988ec2640 1 --2- 172.21.15.114:0/2743356852 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f696c0a1310 0x7f696c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:34.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.275+0000 7f6988ec2640 1 --2- 172.21.15.114:0/2743356852 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f696c0a1310 0x7f696c0a37d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6980009710 tx=0x7f6980006c90 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:34.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.278+0000 7f6979ffb640 1 -- 172.21.15.114:0/2743356852 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f69740b7a40 con 0x7f6984103840 2024-09-13T18:22:34.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:34 smithi134 ceph-mon[34283]: from='client.15684 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T18:22:34.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:34 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/3092377467' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:22:34.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:34 smithi204 ceph-mon[34777]: from='client.15684 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T18:22:34.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:34 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/3092377467' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:22:34.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.442+0000 7f698b126640 1 -- 172.21.15.114:0/2743356852 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f69841976b0 con 0x7f6984103840 2024-09-13T18:22:34.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.442+0000 7f6979ffb640 1 -- 172.21.15.114:0/2743356852 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+11 (secure 0 0 0) 0x7f69740bb060 con 0x7f6984103840 2024-09-13T18:22:34.445 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2024-09-13T18:22:34.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.444+0000 7f698b126640 1 -- 172.21.15.114:0/2743356852 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f696c0a1310 msgr2=0x7f696c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:34.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.444+0000 7f698b126640 1 --2- 172.21.15.114:0/2743356852 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f696c0a1310 0x7f696c0a37d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6980009710 tx=0x7f6980006c90 comp rx=0 tx=0).stop 2024-09-13T18:22:34.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.444+0000 7f698b126640 1 -- 172.21.15.114:0/2743356852 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6984103840 msgr2=0x7f69841968a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:34.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.444+0000 7f698b126640 1 --2- 172.21.15.114:0/2743356852 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6984103840 0x7f69841968a0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f697402a920 tx=0x7f69740048f0 comp rx=0 tx=0).stop 2024-09-13T18:22:34.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.444+0000 7f698b126640 1 -- 172.21.15.114:0/2743356852 shutdown_connections 2024-09-13T18:22:34.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.444+0000 7f698b126640 1 --2- 172.21.15.114:0/2743356852 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f696c0a1310 0x7f696c0a37d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:34.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.444+0000 7f698b126640 1 --2- 172.21.15.114:0/2743356852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f698419ac90 0x7f698419b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:34.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.444+0000 7f698b126640 1 --2- 172.21.15.114:0/2743356852 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6984103840 0x7f69841968a0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:34.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.444+0000 7f698b126640 1 --2- 172.21.15.114:0/2743356852 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69840feb80 0x7f6984196360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:34.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.444+0000 7f698b126640 1 -- 172.21.15.114:0/2743356852 >> 172.21.15.114:0/2743356852 conn(0x7f6984074110 msgr2=0x7f698410c080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:34.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.445+0000 7f698b126640 1 -- 172.21.15.114:0/2743356852 shutdown_connections 2024-09-13T18:22:34.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.445+0000 7f698b126640 1 -- 172.21.15.114:0/2743356852 wait complete. 2024-09-13T18:22:34.458 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T18:22:34.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.557+0000 7efcfea96640 1 -- 172.21.15.114:0/4196269971 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcf8103fb0 msgr2=0x7efcf810bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:34.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.557+0000 7efcfea96640 1 --2- 172.21.15.114:0/4196269971 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcf8103fb0 0x7efcf810bb00 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7efcf0009cd0 tx=0x7efcf002a8e0 comp rx=0 tx=0).stop 2024-09-13T18:22:34.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.557+0000 7efcfea96640 1 -- 172.21.15.114:0/4196269971 shutdown_connections 2024-09-13T18:22:34.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.557+0000 7efcfea96640 1 --2- 172.21.15.114:0/4196269971 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcf8103fb0 0x7efcf810bb00 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:34.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.557+0000 7efcfea96640 1 --2- 172.21.15.114:0/4196269971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efcf80ff810 0x7efcf8103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:34.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.557+0000 7efcfea96640 1 --2- 172.21.15.114:0/4196269971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcf80fee60 0x7efcf80ff240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:34.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.557+0000 7efcfea96640 1 -- 172.21.15.114:0/4196269971 >> 172.21.15.114:0/4196269971 conn(0x7efcf8075c60 msgr2=0x7efcf80fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:34.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.557+0000 7efcfea96640 1 -- 172.21.15.114:0/4196269971 shutdown_connections 2024-09-13T18:22:34.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.557+0000 7efcfea96640 1 -- 172.21.15.114:0/4196269971 wait complete. 2024-09-13T18:22:34.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.557+0000 7efcfea96640 1 Processor -- start 2024-09-13T18:22:34.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.557+0000 7efcfea96640 1 -- start start 2024-09-13T18:22:34.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.557+0000 7efcfea96640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efcf80fee60 0x7efcf8198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:34.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.557+0000 7efcfea96640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcf8103fb0 0x7efcf8198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:34.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.557+0000 7efcfea96640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcf819cd70 0x7efcf819d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:34.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.558+0000 7efcfea96640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf819d760 con 0x7efcf8103fb0 2024-09-13T18:22:34.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.558+0000 7efcfea96640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf819d8d0 con 0x7efcf819cd70 2024-09-13T18:22:34.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.558+0000 7efcfea96640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf819da40 con 0x7efcf80fee60 2024-09-13T18:22:34.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.558+0000 7efcf7fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcf8103fb0 0x7efcf8198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:34.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.558+0000 7efcf7fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcf8103fb0 0x7efcf8198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50850/0 (socket says 172.21.15.114:50850) 2024-09-13T18:22:34.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.558+0000 7efcf7fff640 1 -- 172.21.15.114:0/3027855658 learned_addr learned my addr 172.21.15.114:0/3027855658 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:34.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.558+0000 7efcfc832640 1 --2- 172.21.15.114:0/3027855658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efcf80fee60 0x7efcf8198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:34.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.558+0000 7efcf7fff640 1 -- 172.21.15.114:0/3027855658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efcf80fee60 msgr2=0x7efcf8198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:34.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.558+0000 7efcfd033640 1 --2- 172.21.15.114:0/3027855658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcf819cd70 0x7efcf819d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:34.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.558+0000 7efcf7fff640 1 --2- 172.21.15.114:0/3027855658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efcf80fee60 0x7efcf8198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:34.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.558+0000 7efcf7fff640 1 -- 172.21.15.114:0/3027855658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcf819cd70 msgr2=0x7efcf819d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:34.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.558+0000 7efcf7fff640 1 --2- 172.21.15.114:0/3027855658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcf819cd70 0x7efcf819d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:34.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.558+0000 7efcf7fff640 1 -- 172.21.15.114:0/3027855658 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf0009900 con 0x7efcf8103fb0 2024-09-13T18:22:34.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.558+0000 7efcfd033640 1 --2- 172.21.15.114:0/3027855658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcf819cd70 0x7efcf819d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:22:34.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.558+0000 7efcf7fff640 1 --2- 172.21.15.114:0/3027855658 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcf8103fb0 0x7efcf8198980 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7efce400d770 tx=0x7efce400db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:34.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.558+0000 7efcf5ffb640 1 -- 172.21.15.114:0/3027855658 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce4009a20 con 0x7efcf8103fb0 2024-09-13T18:22:34.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.558+0000 7efcf5ffb640 1 -- 172.21.15.114:0/3027855658 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efce4009b80 con 0x7efcf8103fb0 2024-09-13T18:22:34.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.558+0000 7efcfea96640 1 -- 172.21.15.114:0/3027855658 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcf81a5020 con 0x7efcf8103fb0 2024-09-13T18:22:34.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.558+0000 7efcf5ffb640 1 -- 172.21.15.114:0/3027855658 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce400f620 con 0x7efcf8103fb0 2024-09-13T18:22:34.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.558+0000 7efcfea96640 1 -- 172.21.15.114:0/3027855658 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcf81a54d0 con 0x7efcf8103fb0 2024-09-13T18:22:34.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.559+0000 7efcfea96640 1 -- 172.21.15.114:0/3027855658 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcf8100330 con 0x7efcf8103fb0 2024-09-13T18:22:34.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.561+0000 7efcf5ffb640 1 -- 172.21.15.114:0/3027855658 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 24) v1 ==== 136050+0+0 (secure 0 0 0) 0x7efce4010460 con 0x7efcf8103fb0 2024-09-13T18:22:34.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.562+0000 7efcf5ffb640 1 --2- 172.21.15.114:0/3027855658 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7efce80a1280 0x7efce80a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:34.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.562+0000 7efcfc832640 1 --2- 172.21.15.114:0/3027855658 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7efce80a1280 0x7efce80a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:34.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.562+0000 7efcf5ffb640 1 -- 172.21.15.114:0/3027855658 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7efce40ccf90 con 0x7efcf8103fb0 2024-09-13T18:22:34.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.562+0000 7efcfc832640 1 --2- 172.21.15.114:0/3027855658 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7efce80a1280 0x7efce80a3740 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efcec009990 tx=0x7efcec009110 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:34.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.565+0000 7efcf5ffb640 1 -- 172.21.15.114:0/3027855658 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7efce4095f20 con 0x7efcf8103fb0 2024-09-13T18:22:34.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.725+0000 7efcfea96640 1 -- 172.21.15.114:0/3027855658 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efcf8107160 con 0x7efcf8103fb0 2024-09-13T18:22:34.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.726+0000 7efcf5ffb640 1 -- 172.21.15.114:0/3027855658 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+11 (secure 0 0 0) 0x7efce40170a0 con 0x7efcf8103fb0 2024-09-13T18:22:34.729 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 64 2024-09-13T18:22:34.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.728+0000 7efcfea96640 1 -- 172.21.15.114:0/3027855658 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7efce80a1280 msgr2=0x7efce80a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:34.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.728+0000 7efcfea96640 1 --2- 172.21.15.114:0/3027855658 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7efce80a1280 0x7efce80a3740 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efcec009990 tx=0x7efcec009110 comp rx=0 tx=0).stop 2024-09-13T18:22:34.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.728+0000 7efcfea96640 1 -- 172.21.15.114:0/3027855658 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcf8103fb0 msgr2=0x7efcf8198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:34.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.728+0000 7efcfea96640 1 --2- 172.21.15.114:0/3027855658 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcf8103fb0 0x7efcf8198980 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7efce400d770 tx=0x7efce400db70 comp rx=0 tx=0).stop 2024-09-13T18:22:34.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.728+0000 7efcfea96640 1 -- 172.21.15.114:0/3027855658 shutdown_connections 2024-09-13T18:22:34.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.728+0000 7efcfea96640 1 --2- 172.21.15.114:0/3027855658 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7efce80a1280 0x7efce80a3740 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:34.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.728+0000 7efcfea96640 1 --2- 172.21.15.114:0/3027855658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcf819cd70 0x7efcf819d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:34.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.728+0000 7efcfea96640 1 --2- 172.21.15.114:0/3027855658 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcf8103fb0 0x7efcf8198980 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:34.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.728+0000 7efcfea96640 1 --2- 172.21.15.114:0/3027855658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efcf80fee60 0x7efcf8198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:34.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.728+0000 7efcfea96640 1 -- 172.21.15.114:0/3027855658 >> 172.21.15.114:0/3027855658 conn(0x7efcf8075c60 msgr2=0x7efcf8102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:34.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.728+0000 7efcfea96640 1 -- 172.21.15.114:0/3027855658 shutdown_connections 2024-09-13T18:22:34.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.728+0000 7efcfea96640 1 -- 172.21.15.114:0/3027855658 wait complete. 2024-09-13T18:22:34.742 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T18:22:34.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.842+0000 7fb5952f7640 1 -- 172.21.15.114:0/3155519964 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb590103fe0 msgr2=0x7fb59010bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:34.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.842+0000 7fb5952f7640 1 --2- 172.21.15.114:0/3155519964 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb590103fe0 0x7fb59010bb30 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb580009cd0 tx=0x7fb58002a8e0 comp rx=0 tx=0).stop 2024-09-13T18:22:34.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.842+0000 7fb5952f7640 1 -- 172.21.15.114:0/3155519964 shutdown_connections 2024-09-13T18:22:34.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.842+0000 7fb5952f7640 1 --2- 172.21.15.114:0/3155519964 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb590103fe0 0x7fb59010bb30 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:34.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.842+0000 7fb5952f7640 1 --2- 172.21.15.114:0/3155519964 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5900ff840 0x7fb590103880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:34.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.842+0000 7fb5952f7640 1 --2- 172.21.15.114:0/3155519964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5900fee90 0x7fb5900ff270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:34.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.842+0000 7fb5952f7640 1 -- 172.21.15.114:0/3155519964 >> 172.21.15.114:0/3155519964 conn(0x7fb5900fa900 msgr2=0x7fb5900fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:34.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.842+0000 7fb5952f7640 1 -- 172.21.15.114:0/3155519964 shutdown_connections 2024-09-13T18:22:34.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.842+0000 7fb5952f7640 1 -- 172.21.15.114:0/3155519964 wait complete. 2024-09-13T18:22:34.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.842+0000 7fb5952f7640 1 Processor -- start 2024-09-13T18:22:34.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.843+0000 7fb5952f7640 1 -- start start 2024-09-13T18:22:34.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.843+0000 7fb5952f7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5900fee90 0x7fb5901951e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:34.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.843+0000 7fb5952f7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb590103fe0 0x7fb590195720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:34.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.843+0000 7fb5952f7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb590199b10 0x7fb590199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:34.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.843+0000 7fb5952f7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59019a500 con 0x7fb590103fe0 2024-09-13T18:22:34.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.843+0000 7fb5952f7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59019a670 con 0x7fb5900fee90 2024-09-13T18:22:34.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.843+0000 7fb5952f7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59019a7e0 con 0x7fb590199b10 2024-09-13T18:22:34.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.843+0000 7fb58e7fc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb590103fe0 0x7fb590195720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:34.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.843+0000 7fb58e7fc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb590103fe0 0x7fb590195720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50870/0 (socket says 172.21.15.114:50870) 2024-09-13T18:22:34.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.843+0000 7fb58e7fc640 1 -- 172.21.15.114:0/3917576444 learned_addr learned my addr 172.21.15.114:0/3917576444 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:34.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.843+0000 7fb58f7fe640 1 --2- 172.21.15.114:0/3917576444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb590199b10 0x7fb590199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:34.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.843+0000 7fb58effd640 1 --2- 172.21.15.114:0/3917576444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5900fee90 0x7fb5901951e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:34.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.843+0000 7fb58e7fc640 1 -- 172.21.15.114:0/3917576444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb590199b10 msgr2=0x7fb590199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:34.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.843+0000 7fb58e7fc640 1 --2- 172.21.15.114:0/3917576444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb590199b10 0x7fb590199fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:34.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.843+0000 7fb58e7fc640 1 -- 172.21.15.114:0/3917576444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5900fee90 msgr2=0x7fb5901951e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:34.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.843+0000 7fb58e7fc640 1 --2- 172.21.15.114:0/3917576444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5900fee90 0x7fb5901951e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:34.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.843+0000 7fb58e7fc640 1 -- 172.21.15.114:0/3917576444 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb580009900 con 0x7fb590103fe0 2024-09-13T18:22:34.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.843+0000 7fb58f7fe640 1 --2- 172.21.15.114:0/3917576444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb590199b10 0x7fb590199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:22:34.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.843+0000 7fb58effd640 1 --2- 172.21.15.114:0/3917576444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5900fee90 0x7fb5901951e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:22:34.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.843+0000 7fb58e7fc640 1 --2- 172.21.15.114:0/3917576444 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb590103fe0 0x7fb590195720 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb57800b450 tx=0x7fb57800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:34.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.843+0000 7fb587fff640 1 -- 172.21.15.114:0/3917576444 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb578011890 con 0x7fb590103fe0 2024-09-13T18:22:34.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.843+0000 7fb5952f7640 1 -- 172.21.15.114:0/3917576444 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5901a1dc0 con 0x7fb590103fe0 2024-09-13T18:22:34.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.843+0000 7fb587fff640 1 -- 172.21.15.114:0/3917576444 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb578011e80 con 0x7fb590103fe0 2024-09-13T18:22:34.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.843+0000 7fb587fff640 1 -- 172.21.15.114:0/3917576444 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57800f340 con 0x7fb590103fe0 2024-09-13T18:22:34.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.844+0000 7fb5952f7640 1 -- 172.21.15.114:0/3917576444 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5901a2210 con 0x7fb590103fe0 2024-09-13T18:22:34.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.844+0000 7fb5952f7640 1 -- 172.21.15.114:0/3917576444 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb590100360 con 0x7fb590103fe0 2024-09-13T18:22:34.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.845+0000 7fb587fff640 1 -- 172.21.15.114:0/3917576444 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 24) v1 ==== 136050+0+0 (secure 0 0 0) 0x7fb5780119f0 con 0x7fb590103fe0 2024-09-13T18:22:34.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.846+0000 7fb587fff640 1 --2- 172.21.15.114:0/3917576444 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fb57c0a1290 0x7fb57c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:34.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.846+0000 7fb58effd640 1 --2- 172.21.15.114:0/3917576444 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fb57c0a1290 0x7fb57c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:34.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.846+0000 7fb587fff640 1 -- 172.21.15.114:0/3917576444 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fb5780ccf90 con 0x7fb590103fe0 2024-09-13T18:22:34.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.846+0000 7fb58effd640 1 --2- 172.21.15.114:0/3917576444 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fb57c0a1290 0x7fb57c0a3750 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb588005d00 tx=0x7fb588009550 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:34.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:34.850+0000 7fb587fff640 1 -- 172.21.15.114:0/3917576444 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fb578100e20 con 0x7fb590103fe0 2024-09-13T18:22:35.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.014+0000 7fb5952f7640 1 -- 172.21.15.114:0/3917576444 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb590107190 con 0x7fb590103fe0 2024-09-13T18:22:35.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.014+0000 7fb587fff640 1 -- 172.21.15.114:0/3917576444 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb5780150a0 con 0x7fb590103fe0 2024-09-13T18:22:35.017 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2024-09-13T18:22:35.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.016+0000 7fb5952f7640 1 -- 172.21.15.114:0/3917576444 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fb57c0a1290 msgr2=0x7fb57c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:35.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.016+0000 7fb5952f7640 1 --2- 172.21.15.114:0/3917576444 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fb57c0a1290 0x7fb57c0a3750 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb588005d00 tx=0x7fb588009550 comp rx=0 tx=0).stop 2024-09-13T18:22:35.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.016+0000 7fb5952f7640 1 -- 172.21.15.114:0/3917576444 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb590103fe0 msgr2=0x7fb590195720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:35.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.016+0000 7fb5952f7640 1 --2- 172.21.15.114:0/3917576444 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb590103fe0 0x7fb590195720 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb57800b450 tx=0x7fb57800b910 comp rx=0 tx=0).stop 2024-09-13T18:22:35.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.016+0000 7fb5952f7640 1 -- 172.21.15.114:0/3917576444 shutdown_connections 2024-09-13T18:22:35.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.016+0000 7fb5952f7640 1 --2- 172.21.15.114:0/3917576444 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fb57c0a1290 0x7fb57c0a3750 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.016+0000 7fb5952f7640 1 --2- 172.21.15.114:0/3917576444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb590199b10 0x7fb590199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.016+0000 7fb5952f7640 1 --2- 172.21.15.114:0/3917576444 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb590103fe0 0x7fb590195720 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.016+0000 7fb5952f7640 1 --2- 172.21.15.114:0/3917576444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5900fee90 0x7fb5901951e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.016+0000 7fb5952f7640 1 -- 172.21.15.114:0/3917576444 >> 172.21.15.114:0/3917576444 conn(0x7fb5900fa900 msgr2=0x7fb5900fbd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:35.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.016+0000 7fb5952f7640 1 -- 172.21.15.114:0/3917576444 shutdown_connections 2024-09-13T18:22:35.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.016+0000 7fb5952f7640 1 -- 172.21.15.114:0/3917576444 wait complete. 2024-09-13T18:22:35.031 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:22:35.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.127+0000 7f76b0701640 1 -- 172.21.15.114:0/981287842 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76a80fef00 msgr2=0x7f76a80ff2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:35.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.127+0000 7f76b0701640 1 --2- 172.21.15.114:0/981287842 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76a80fef00 0x7f76a80ff2e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f76a0009930 tx=0x7f76a002a420 comp rx=0 tx=0).stop 2024-09-13T18:22:35.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.127+0000 7f76b0701640 1 -- 172.21.15.114:0/981287842 shutdown_connections 2024-09-13T18:22:35.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.127+0000 7f76b0701640 1 --2- 172.21.15.114:0/981287842 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76a8104050 0x7f76a810bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.127+0000 7f76b0701640 1 --2- 172.21.15.114:0/981287842 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76a80ff8b0 0x7f76a81038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.127+0000 7f76b0701640 1 --2- 172.21.15.114:0/981287842 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76a80fef00 0x7f76a80ff2e0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.127+0000 7f76b0701640 1 -- 172.21.15.114:0/981287842 >> 172.21.15.114:0/981287842 conn(0x7f76a80fa950 msgr2=0x7f76a80fcd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:35.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.127+0000 7f76b0701640 1 -- 172.21.15.114:0/981287842 shutdown_connections 2024-09-13T18:22:35.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.127+0000 7f76b0701640 1 -- 172.21.15.114:0/981287842 wait complete. 2024-09-13T18:22:35.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.127+0000 7f76b0701640 1 Processor -- start 2024-09-13T18:22:35.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.128+0000 7f76b0701640 1 -- start start 2024-09-13T18:22:35.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.128+0000 7f76b0701640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76a80fef00 0x7f76a81971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:35.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.128+0000 7f76b0701640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76a8104050 0x7f76a8197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:35.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.128+0000 7f76b0701640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76a8197d00 0x7f76a81a2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:35.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.128+0000 7f76b0701640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a81a29a0 con 0x7f76a80fef00 2024-09-13T18:22:35.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.128+0000 7f76b0701640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a81a2b10 con 0x7f76a8104050 2024-09-13T18:22:35.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.128+0000 7f76b0701640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a81a2cd0 con 0x7f76a8197d00 2024-09-13T18:22:35.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.128+0000 7f76ae49d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76a80fef00 0x7f76a81971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:35.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.128+0000 7f76ae49d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76a80fef00 0x7f76a81971e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50880/0 (socket says 172.21.15.114:50880) 2024-09-13T18:22:35.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.128+0000 7f76ae49d640 1 -- 172.21.15.114:0/2481894019 learned_addr learned my addr 172.21.15.114:0/2481894019 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:35.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.128+0000 7f76adc9c640 1 --2- 172.21.15.114:0/2481894019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76a8104050 0x7f76a8197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:35.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.128+0000 7f76aec9e640 1 --2- 172.21.15.114:0/2481894019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76a8197d00 0x7f76a81a2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:35.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.128+0000 7f76ae49d640 1 -- 172.21.15.114:0/2481894019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76a8197d00 msgr2=0x7f76a81a2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:35.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.128+0000 7f76ae49d640 1 --2- 172.21.15.114:0/2481894019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76a8197d00 0x7f76a81a2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.128+0000 7f76ae49d640 1 -- 172.21.15.114:0/2481894019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76a8104050 msgr2=0x7f76a8197720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:35.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.128+0000 7f76ae49d640 1 --2- 172.21.15.114:0/2481894019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76a8104050 0x7f76a8197720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.128+0000 7f76ae49d640 1 -- 172.21.15.114:0/2481894019 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a0009560 con 0x7f76a80fef00 2024-09-13T18:22:35.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.128+0000 7f76ae49d640 1 --2- 172.21.15.114:0/2481894019 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76a80fef00 0x7f76a81971e0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f76a002a940 tx=0x7f76a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:35.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.128+0000 7f76a77fe640 1 -- 172.21.15.114:0/2481894019 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a0004cf0 con 0x7f76a80fef00 2024-09-13T18:22:35.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.128+0000 7f76a77fe640 1 -- 172.21.15.114:0/2481894019 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f76a0003a80 con 0x7f76a80fef00 2024-09-13T18:22:35.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.128+0000 7f76b0701640 1 -- 172.21.15.114:0/2481894019 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76a81a2f50 con 0x7f76a80fef00 2024-09-13T18:22:35.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.129+0000 7f76a77fe640 1 -- 172.21.15.114:0/2481894019 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a003ca70 con 0x7f76a80fef00 2024-09-13T18:22:35.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.129+0000 7f76b0701640 1 -- 172.21.15.114:0/2481894019 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76a81a3370 con 0x7f76a80fef00 2024-09-13T18:22:35.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.129+0000 7f76b0701640 1 -- 172.21.15.114:0/2481894019 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76a81003d0 con 0x7f76a80fef00 2024-09-13T18:22:35.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.130+0000 7f76a77fe640 1 -- 172.21.15.114:0/2481894019 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 24) v1 ==== 136050+0+0 (secure 0 0 0) 0x7f76a0003bf0 con 0x7f76a80fef00 2024-09-13T18:22:35.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.131+0000 7f76a77fe640 1 --2- 172.21.15.114:0/2481894019 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f76900a1290 0x7f76900a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:35.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.131+0000 7f76adc9c640 1 --2- 172.21.15.114:0/2481894019 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f76900a1290 0x7f76900a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:35.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.131+0000 7f76a77fe640 1 -- 172.21.15.114:0/2481894019 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f76a00ee470 con 0x7f76a80fef00 2024-09-13T18:22:35.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.132+0000 7f76adc9c640 1 --2- 172.21.15.114:0/2481894019 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f76900a1290 0x7f76900a3750 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f76a80691b0 tx=0x7f769c00b390 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:35.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.134+0000 7f76a77fe640 1 -- 172.21.15.114:0/2481894019 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f76a00b7520 con 0x7f76a80fef00 2024-09-13T18:22:35.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.331+0000 7f76b0701640 1 -- 172.21.15.114:0/2481894019 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f76a80ff2e0 con 0x7f76a80fef00 2024-09-13T18:22:35.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.331+0000 7f76a77fe640 1 -- 172.21.15.114:0/2481894019 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+10930 (secure 0 0 0) 0x7f76a0040090 con 0x7f76a80fef00 2024-09-13T18:22:35.335 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:22:35.335 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4162817497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4162817497},{"type":"v1","addr":"172.21.15.114:6835","nonce":4162817497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/2435428478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":2435428478},{"type":"v1","addr":"172.21.15.114:6837","nonce":2435428478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T18:20:01.895117+0000","modified":"2024-09-13T18:22:29.064231+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24482,"mds_2":24475,"mds_3":24458,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/2046516451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":2046516451},{"type":"v1","addr":"172.21.15.114:6839","nonce":2046516451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":10,"addr":"172.21.15.114:6841/1956923446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":1956923446},{"type":"v1","addr":"172.21.15.114:6841","nonce":1956923446}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24391":{"gid":24391,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/507301496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":507301496},{"type":"v1","addr":"172.21.15.204:6833","nonce":507301496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4028657219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4028657219},{"type":"v1","addr":"172.21.15.134:6833","nonce":4028657219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/57029119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":57029119},{"type":"v1","addr":"172.21.15.204:6835","nonce":57029119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3068880939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3068880939},{"type":"v1","addr":"172.21.15.204:6837","nonce":3068880939}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"e","rank":3,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.134:6835/1150003275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1150003275},{"type":"v1","addr":"172.21.15.134:6835","nonce":1150003275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/344883912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":344883912},{"type":"v1","addr":"172.21.15.204:6839","nonce":344883912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6837/891692210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":891692210},{"type":"v1","addr":"172.21.15.134:6837","nonce":891692210}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6839/3466309724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3466309724},{"type":"v1","addr":"172.21.15.134:6839","nonce":3466309724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:22:35.335 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 30 2024-09-13T18:22:35.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.333+0000 7f76b0701640 1 -- 172.21.15.114:0/2481894019 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f76900a1290 msgr2=0x7f76900a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:35.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.333+0000 7f76b0701640 1 --2- 172.21.15.114:0/2481894019 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f76900a1290 0x7f76900a3750 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f76a80691b0 tx=0x7f769c00b390 comp rx=0 tx=0).stop 2024-09-13T18:22:35.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.333+0000 7f76b0701640 1 -- 172.21.15.114:0/2481894019 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76a80fef00 msgr2=0x7f76a81971e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:35.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.333+0000 7f76b0701640 1 --2- 172.21.15.114:0/2481894019 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76a80fef00 0x7f76a81971e0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f76a002a940 tx=0x7f76a00046c0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.333+0000 7f76b0701640 1 -- 172.21.15.114:0/2481894019 shutdown_connections 2024-09-13T18:22:35.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.333+0000 7f76b0701640 1 --2- 172.21.15.114:0/2481894019 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f76900a1290 0x7f76900a3750 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.333+0000 7f76b0701640 1 --2- 172.21.15.114:0/2481894019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76a8197d00 0x7f76a81a2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.333+0000 7f76b0701640 1 --2- 172.21.15.114:0/2481894019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76a8104050 0x7f76a8197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.333+0000 7f76b0701640 1 --2- 172.21.15.114:0/2481894019 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76a80fef00 0x7f76a81971e0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.333+0000 7f76b0701640 1 -- 172.21.15.114:0/2481894019 >> 172.21.15.114:0/2481894019 conn(0x7f76a80fa950 msgr2=0x7f76a80fbe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:35.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.333+0000 7f76b0701640 1 -- 172.21.15.114:0/2481894019 shutdown_connections 2024-09-13T18:22:35.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.334+0000 7f76b0701640 1 -- 172.21.15.114:0/2481894019 wait complete. 2024-09-13T18:22:35.346 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:35 smithi114 ceph-mon[30897]: from='client.15690 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T18:22:35.346 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:35 smithi114 ceph-mon[30897]: from='client.15696 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T18:22:35.347 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:35 smithi114 ceph-mon[30897]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-13T18:22:35.347 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:35 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/4250266003' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T18:22:35.347 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:35 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2743356852' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T18:22:35.347 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:35 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/3027855658' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T18:22:35.347 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:35 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/3917576444' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T18:22:35.348 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:22:35.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:35 smithi134 ceph-mon[34283]: from='client.15690 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T18:22:35.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:35 smithi134 ceph-mon[34283]: from='client.15696 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T18:22:35.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:35 smithi134 ceph-mon[34283]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-13T18:22:35.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:35 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/4250266003' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T18:22:35.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:35 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/2743356852' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T18:22:35.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:35 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/3027855658' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T18:22:35.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:35 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/3917576444' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T18:22:35.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:35 smithi204 ceph-mon[34777]: from='client.15690 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T18:22:35.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:35 smithi204 ceph-mon[34777]: from='client.15696 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-13T18:22:35.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:35 smithi204 ceph-mon[34777]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-13T18:22:35.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:35 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/4250266003' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T18:22:35.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:35 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/2743356852' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T18:22:35.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:35 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/3027855658' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T18:22:35.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:35 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/3917576444' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T18:22:35.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.444+0000 7f010b718640 1 -- 172.21.15.114:0/2676299521 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01040fee70 msgr2=0x7f01040ff250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:35.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.444+0000 7f010b718640 1 --2- 172.21.15.114:0/2676299521 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01040fee70 0x7f01040ff250 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f0100009930 tx=0x7f010002a420 comp rx=0 tx=0).stop 2024-09-13T18:22:35.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.444+0000 7f010b718640 1 -- 172.21.15.114:0/2676299521 shutdown_connections 2024-09-13T18:22:35.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.444+0000 7f010b718640 1 --2- 172.21.15.114:0/2676299521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0104103fc0 0x7f010410bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.444+0000 7f010b718640 1 --2- 172.21.15.114:0/2676299521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01040ff820 0x7f0104103860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.444+0000 7f010b718640 1 --2- 172.21.15.114:0/2676299521 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01040fee70 0x7f01040ff250 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.444+0000 7f010b718640 1 -- 172.21.15.114:0/2676299521 >> 172.21.15.114:0/2676299521 conn(0x7f01040fa920 msgr2=0x7f01040fcd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:35.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.444+0000 7f010b718640 1 -- 172.21.15.114:0/2676299521 shutdown_connections 2024-09-13T18:22:35.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.444+0000 7f010b718640 1 -- 172.21.15.114:0/2676299521 wait complete. 2024-09-13T18:22:35.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.445+0000 7f010b718640 1 Processor -- start 2024-09-13T18:22:35.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.445+0000 7f010b718640 1 -- start start 2024-09-13T18:22:35.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.445+0000 7f010b718640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01040fee70 0x7f0104195060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:35.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.445+0000 7f010b718640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0104103fc0 0x7f01041955a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:35.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.445+0000 7f010b718640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0104199990 0x7f0104199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:35.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.445+0000 7f010b718640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010419a380 con 0x7f0104199990 2024-09-13T18:22:35.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.445+0000 7f010b718640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010419a4f0 con 0x7f01040fee70 2024-09-13T18:22:35.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.445+0000 7f010b718640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010419a660 con 0x7f0104103fc0 2024-09-13T18:22:35.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.445+0000 7f0109cb5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0104199990 0x7f0104199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:35.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.445+0000 7f0109cb5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0104199990 0x7f0104199e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50894/0 (socket says 172.21.15.114:50894) 2024-09-13T18:22:35.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.445+0000 7f0109cb5640 1 -- 172.21.15.114:0/1216293605 learned_addr learned my addr 172.21.15.114:0/1216293605 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:35.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.445+0000 7f01094b4640 1 --2- 172.21.15.114:0/1216293605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01040fee70 0x7f0104195060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:35.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.445+0000 7f0108cb3640 1 --2- 172.21.15.114:0/1216293605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0104103fc0 0x7f01041955a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:35.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.445+0000 7f0109cb5640 1 -- 172.21.15.114:0/1216293605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0104103fc0 msgr2=0x7f01041955a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:35.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.445+0000 7f0109cb5640 1 --2- 172.21.15.114:0/1216293605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0104103fc0 0x7f01041955a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.445+0000 7f0109cb5640 1 -- 172.21.15.114:0/1216293605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01040fee70 msgr2=0x7f0104195060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:35.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.445+0000 7f0109cb5640 1 --2- 172.21.15.114:0/1216293605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01040fee70 0x7f0104195060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.445+0000 7f0109cb5640 1 -- 172.21.15.114:0/1216293605 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0100009560 con 0x7f0104199990 2024-09-13T18:22:35.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.445+0000 7f0109cb5640 1 --2- 172.21.15.114:0/1216293605 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0104199990 0x7f0104199e40 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f00fc009fd0 tx=0x7f00fc00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:35.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.446+0000 7f00f67fc640 1 -- 172.21.15.114:0/1216293605 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00fc00bea0 con 0x7f0104199990 2024-09-13T18:22:35.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.446+0000 7f00f67fc640 1 -- 172.21.15.114:0/1216293605 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f00fc009940 con 0x7f0104199990 2024-09-13T18:22:35.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.446+0000 7f010b718640 1 -- 172.21.15.114:0/1216293605 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01041a1c40 con 0x7f0104199990 2024-09-13T18:22:35.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.446+0000 7f00f67fc640 1 -- 172.21.15.114:0/1216293605 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00fc010480 con 0x7f0104199990 2024-09-13T18:22:35.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.446+0000 7f010b718640 1 -- 172.21.15.114:0/1216293605 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01041a20f0 con 0x7f0104199990 2024-09-13T18:22:35.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.447+0000 7f010b718640 1 -- 172.21.15.114:0/1216293605 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0104100340 con 0x7f0104199990 2024-09-13T18:22:35.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.448+0000 7f00f67fc640 1 -- 172.21.15.114:0/1216293605 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 24) v1 ==== 136050+0+0 (secure 0 0 0) 0x7f00fc003970 con 0x7f0104199990 2024-09-13T18:22:35.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.448+0000 7f00f67fc640 1 --2- 172.21.15.114:0/1216293605 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f00f00a1310 0x7f00f00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:35.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.448+0000 7f01094b4640 1 --2- 172.21.15.114:0/1216293605 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f00f00a1310 0x7f00f00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:35.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.448+0000 7f00f67fc640 1 -- 172.21.15.114:0/1216293605 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f00fc0ccdd0 con 0x7f0104199990 2024-09-13T18:22:35.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.449+0000 7f01094b4640 1 --2- 172.21.15.114:0/1216293605 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f00f00a1310 0x7f00f00a37d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f01000057c0 tx=0x7f0100005a20 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:35.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.452+0000 7f00f67fc640 1 -- 172.21.15.114:0/1216293605 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f00fc095d60 con 0x7f0104199990 2024-09-13T18:22:35.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.616+0000 7f010b718640 1 -- 172.21.15.114:0/1216293605 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f01040ff250 con 0x7f0104199990 2024-09-13T18:22:35.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.617+0000 7f00f67fc640 1 -- 172.21.15.114:0/1216293605 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+25079 (secure 0 0 0) 0x7f00fc020090 con 0x7f0104199990 2024-09-13T18:22:35.621 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:22:35.621 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":94,"fsid":"97e7edf2-71fb-11ef-bceb-c7b262605968","created":"2024-09-13T18:12:36.360625+0000","modified":"2024-09-13T18:22:28.057664+0000","last_up_change":"2024-09-13T18:19:12.350370+0000","last_in_change":"2024-09-13T18:18:52.321596+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:15:40.129659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:20:00.024407+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:20:01.085549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T18:20:04.334216+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9914f01a-9f6f-4878-b955-83570199c354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6803","nonce":3499525826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6805","nonce":3499525826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6809","nonce":3499525826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6807","nonce":3499525826}]},"public_addr":"172.21.15.114:6803/3499525826","cluster_addr":"172.21.15.114:6805/3499525826","heartbeat_back_addr":"172.21.15.114:6809/3499525826","heartbeat_front_addr":"172.21.15.114:6807/3499525826","state":["exists","up"]},{"osd":1,"uuid":"ef65e012-7469-4c20-b354-17022bca63c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6801","nonce":4264264866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6803","nonce":4264264866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6807","nonce":4264264866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6805","nonce":4264264866}]},"public_addr":"172.21.15.134:6801/4264264866","cluster_addr":"172.21.15.134:6803/4264264866","heartbeat_back_addr":"172.21.15.134:6807/4264264866","heartbeat_front_addr":"172.21.15.134:6805/4264264866","state":["exists","up"]},{"osd":2,"uuid":"578213fd-715f-4be7-91fa-254f3d0e3022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6801","nonce":2288807707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6803","nonce":2288807707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6807","nonce":2288807707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6805","nonce":2288807707}]},"public_addr":"172.21.15.204:6801/2288807707","cluster_addr":"172.21.15.204:6803/2288807707","heartbeat_back_addr":"172.21.15.204:6807/2288807707","heartbeat_front_addr":"172.21.15.204:6805/2288807707","state":["exists","up"]},{"osd":3,"uuid":"b64aab3f-1e52-4828-8397-74af5e05f2cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6811","nonce":2904204254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6813","nonce":2904204254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6817","nonce":2904204254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6815","nonce":2904204254}]},"public_addr":"172.21.15.114:6811/2904204254","cluster_addr":"172.21.15.114:6813/2904204254","heartbeat_back_addr":"172.21.15.114:6817/2904204254","heartbeat_front_addr":"172.21.15.114:6815/2904204254","state":["exists","up"]},{"osd":4,"uuid":"f38943d6-39c0-42e1-b427-cb6bb90ac38a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6809","nonce":1773945333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6811","nonce":1773945333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6815","nonce":1773945333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6813","nonce":1773945333}]},"public_addr":"172.21.15.134:6809/1773945333","cluster_addr":"172.21.15.134:6811/1773945333","heartbeat_back_addr":"172.21.15.134:6815/1773945333","heartbeat_front_addr":"172.21.15.134:6813/1773945333","state":["exists","up"]},{"osd":5,"uuid":"bb38f361-8e00-4178-9752-5ddbf3af20e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6809","nonce":2371778055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6811","nonce":2371778055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6815","nonce":2371778055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6813","nonce":2371778055}]},"public_addr":"172.21.15.204:6809/2371778055","cluster_addr":"172.21.15.204:6811/2371778055","heartbeat_back_addr":"172.21.15.204:6815/2371778055","heartbeat_front_addr":"172.21.15.204:6813/2371778055","state":["exists","up"]},{"osd":6,"uuid":"93d40a03-e7ff-4a3f-9efa-5c5403c9983d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6819","nonce":1484325044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6821","nonce":1484325044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6825","nonce":1484325044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6823","nonce":1484325044}]},"public_addr":"172.21.15.114:6819/1484325044","cluster_addr":"172.21.15.114:6821/1484325044","heartbeat_back_addr":"172.21.15.114:6825/1484325044","heartbeat_front_addr":"172.21.15.114:6823/1484325044","state":["exists","up"]},{"osd":7,"uuid":"982f629a-1b95-4be1-8e75-5ddce1ad77b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6817","nonce":799246614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6819","nonce":799246614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6823","nonce":799246614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6821","nonce":799246614}]},"public_addr":"172.21.15.134:6817/799246614","cluster_addr":"172.21.15.134:6819/799246614","heartbeat_back_addr":"172.21.15.134:6823/799246614","heartbeat_front_addr":"172.21.15.134:6821/799246614","state":["exists","up"]},{"osd":8,"uuid":"409beb9b-8227-4dad-a4a0-f0e5ce70dc5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6817","nonce":858331805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6819","nonce":858331805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6823","nonce":858331805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6821","nonce":858331805}]},"public_addr":"172.21.15.204:6817/858331805","cluster_addr":"172.21.15.204:6819/858331805","heartbeat_back_addr":"172.21.15.204:6823/858331805","heartbeat_front_addr":"172.21.15.204:6821/858331805","state":["exists","up"]},{"osd":9,"uuid":"a345a15c-b1fc-4877-b445-418eebf0d0ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6827","nonce":2172392854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6829","nonce":2172392854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6833","nonce":2172392854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6831","nonce":2172392854}]},"public_addr":"172.21.15.114:6827/2172392854","cluster_addr":"172.21.15.114:6829/2172392854","heartbeat_back_addr":"172.21.15.114:6833/2172392854","heartbeat_front_addr":"172.21.15.114:6831/2172392854","state":["exists","up"]},{"osd":10,"uuid":"464d995f-d90b-4caa-addb-1f7562bd74b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6825","nonce":3073513468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6827","nonce":3073513468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6831","nonce":3073513468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6829","nonce":3073513468}]},"public_addr":"172.21.15.134:6825/3073513468","cluster_addr":"172.21.15.134:6827/3073513468","heartbeat_back_addr":"172.21.15.134:6831/3073513468","heartbeat_front_addr":"172.21.15.134:6829/3073513468","state":["exists","up"]},{"osd":11,"uuid":"71dd794d-326c-477f-8d02-ee32bff1350d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6825","nonce":3239521765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6827","nonce":3239521765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6831","nonce":3239521765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6829","nonce":3239521765}]},"public_addr":"172.21.15.204:6825/3239521765","cluster_addr":"172.21.15.204:6827/3239521765","heartbeat_back_addr":"172.21.15.204:6831/3239521765","heartbeat_front_addr":"172.21.15.204:6829/3239521765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:14:54.542687+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:15:15.746055+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:15:38.371131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:01.383932+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:24.796237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:48.422214+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:11.284323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:34.570365+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:58.232855+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:18:21.609596+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:18:45.565859+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:19:10.017969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:6801/3396185541":"2024-09-14T18:22:28.057623+0000","172.21.15.114:6800/830738794":"2024-09-14T18:13:03.521116+0000","172.21.15.114:6801/830738794":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/1684954293":"2024-09-14T18:22:28.057623+0000","172.21.15.114:0/3544951322":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/722571999":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/2949422849":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2034742251":"2024-09-14T18:13:03.521116+0000","172.21.15.114:6800/4281866858":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/3407523320":"2024-09-14T18:22:28.057623+0000","172.21.15.114:6800/3396185541":"2024-09-14T18:22:28.057623+0000","172.21.15.114:0/4257828906":"2024-09-14T18:13:25.823027+0000","172.21.15.114:6801/4281866858":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2828364892":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2714091073":"2024-09-14T18:22:28.057623+0000","172.21.15.114:0/842237781":"2024-09-14T18:22:28.057623+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:22:35.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.619+0000 7f010b718640 1 -- 172.21.15.114:0/1216293605 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f00f00a1310 msgr2=0x7f00f00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:35.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.619+0000 7f010b718640 1 --2- 172.21.15.114:0/1216293605 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f00f00a1310 0x7f00f00a37d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f01000057c0 tx=0x7f0100005a20 comp rx=0 tx=0).stop 2024-09-13T18:22:35.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.619+0000 7f010b718640 1 -- 172.21.15.114:0/1216293605 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0104199990 msgr2=0x7f0104199e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:35.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.619+0000 7f010b718640 1 --2- 172.21.15.114:0/1216293605 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0104199990 0x7f0104199e40 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f00fc009fd0 tx=0x7f00fc00b260 comp rx=0 tx=0).stop 2024-09-13T18:22:35.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.619+0000 7f010b718640 1 -- 172.21.15.114:0/1216293605 shutdown_connections 2024-09-13T18:22:35.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.619+0000 7f010b718640 1 --2- 172.21.15.114:0/1216293605 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f00f00a1310 0x7f00f00a37d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.619+0000 7f010b718640 1 --2- 172.21.15.114:0/1216293605 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0104199990 0x7f0104199e40 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.619+0000 7f010b718640 1 --2- 172.21.15.114:0/1216293605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0104103fc0 0x7f01041955a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.619+0000 7f010b718640 1 --2- 172.21.15.114:0/1216293605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01040fee70 0x7f0104195060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.619+0000 7f010b718640 1 -- 172.21.15.114:0/1216293605 >> 172.21.15.114:0/1216293605 conn(0x7f01040fa920 msgr2=0x7f01040fbd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:35.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.619+0000 7f010b718640 1 -- 172.21.15.114:0/1216293605 shutdown_connections 2024-09-13T18:22:35.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.620+0000 7f010b718640 1 -- 172.21.15.114:0/1216293605 wait complete. 2024-09-13T18:22:35.634 DEBUG:tasks.fwd_scrub:Yielding 2024-09-13T18:22:35.634 INFO:teuthology.run_tasks:Running task exec... 2024-09-13T18:22:35.644 INFO:teuthology.task.exec:Executing custom commands... 2024-09-13T18:22:35.644 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi114.front.sepia.ceph.com 2024-09-13T18:22:35.644 DEBUG:teuthology.orchestra.run.smithi114:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-09-13T18:22:35.644 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-13T18:22:35.645 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-13T18:22:35.645 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-09-13T18:22:35.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.743+0000 7f6ad91d5640 1 -- 172.21.15.114:0/4220330575 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6ad4104c30 msgr2=0x7f6ad4105010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:35.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.743+0000 7f6ad91d5640 1 --2- 172.21.15.114:0/4220330575 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6ad4104c30 0x7f6ad4105010 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4009930 tx=0x7f6ac402a420 comp rx=0 tx=0).stop 2024-09-13T18:22:35.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.743+0000 7f6ad91d5640 1 -- 172.21.15.114:0/4220330575 shutdown_connections 2024-09-13T18:22:35.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.743+0000 7f6ad91d5640 1 --2- 172.21.15.114:0/4220330575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ad40ff970 0x7f6ad410ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.743+0000 7f6ad91d5640 1 --2- 172.21.15.114:0/4220330575 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ad40feff0 0x7f6ad40ff430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.743+0000 7f6ad91d5640 1 --2- 172.21.15.114:0/4220330575 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6ad4104c30 0x7f6ad4105010 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.743+0000 7f6ad91d5640 1 -- 172.21.15.114:0/4220330575 >> 172.21.15.114:0/4220330575 conn(0x7f6ad4074940 msgr2=0x7f6ad4074d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:35.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.743+0000 7f6ad91d5640 1 -- 172.21.15.114:0/4220330575 shutdown_connections 2024-09-13T18:22:35.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.743+0000 7f6ad91d5640 1 -- 172.21.15.114:0/4220330575 wait complete. 2024-09-13T18:22:35.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.744+0000 7f6ad91d5640 1 Processor -- start 2024-09-13T18:22:35.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.744+0000 7f6ad91d5640 1 -- start start 2024-09-13T18:22:35.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.744+0000 7f6ad91d5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ad40feff0 0x7f6ad4196130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:35.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.744+0000 7f6ad91d5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6ad40ff970 0x7f6ad4196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:35.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.744+0000 7f6ad91d5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ad4104c30 0x7f6ad419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:35.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.744+0000 7f6ad91d5640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad419b060 con 0x7f6ad40ff970 2024-09-13T18:22:35.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.744+0000 7f6ad91d5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad419b1d0 con 0x7f6ad40feff0 2024-09-13T18:22:35.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.744+0000 7f6ad91d5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad419b390 con 0x7f6ad4104c30 2024-09-13T18:22:35.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.744+0000 7f6acbfff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6ad40ff970 0x7f6ad4196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:35.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.744+0000 7f6acbfff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6ad40ff970 0x7f6ad4196670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50928/0 (socket says 172.21.15.114:50928) 2024-09-13T18:22:35.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.744+0000 7f6acbfff640 1 -- 172.21.15.114:0/3871354009 learned_addr learned my addr 172.21.15.114:0/3871354009 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:35.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.744+0000 7f6ad2d9d640 1 --2- 172.21.15.114:0/3871354009 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ad40feff0 0x7f6ad4196130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:35.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.744+0000 7f6ad359e640 1 --2- 172.21.15.114:0/3871354009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ad4104c30 0x7f6ad419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:35.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.745+0000 7f6acbfff640 1 -- 172.21.15.114:0/3871354009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ad4104c30 msgr2=0x7f6ad419aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:35.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.745+0000 7f6acbfff640 1 --2- 172.21.15.114:0/3871354009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ad4104c30 0x7f6ad419aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.745+0000 7f6acbfff640 1 -- 172.21.15.114:0/3871354009 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ad40feff0 msgr2=0x7f6ad4196130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:35.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.745+0000 7f6acbfff640 1 --2- 172.21.15.114:0/3871354009 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ad40feff0 0x7f6ad4196130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.745+0000 7f6acbfff640 1 -- 172.21.15.114:0/3871354009 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ac4009560 con 0x7f6ad40ff970 2024-09-13T18:22:35.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.745+0000 7f6acbfff640 1 --2- 172.21.15.114:0/3871354009 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6ad40ff970 0x7f6ad4196670 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f6abc00c970 tx=0x7f6abc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:35.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.745+0000 7f6ad0d99640 1 -- 172.21.15.114:0/3871354009 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6abc007bb0 con 0x7f6ad40ff970 2024-09-13T18:22:35.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.745+0000 7f6ad0d99640 1 -- 172.21.15.114:0/3871354009 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6abc007d10 con 0x7f6ad40ff970 2024-09-13T18:22:35.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.745+0000 7f6ad91d5640 1 -- 172.21.15.114:0/3871354009 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ad419b670 con 0x7f6ad40ff970 2024-09-13T18:22:35.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.745+0000 7f6ad0d99640 1 -- 172.21.15.114:0/3871354009 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6abc004d70 con 0x7f6ad40ff970 2024-09-13T18:22:35.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.745+0000 7f6ad91d5640 1 -- 172.21.15.114:0/3871354009 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ad41a2ef0 con 0x7f6ad40ff970 2024-09-13T18:22:35.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.745+0000 7f43c3e90640 1 -- 172.21.15.114:0/515665784 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43bc100900 msgr2=0x7f43bc104940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:35.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.745+0000 7f43c3e90640 1 --2- 172.21.15.114:0/515665784 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43bc100900 0x7f43bc104940 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f43ac009930 tx=0x7f43ac02a420 comp rx=0 tx=0).stop 2024-09-13T18:22:35.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.745+0000 7f43c3e90640 1 -- 172.21.15.114:0/515665784 shutdown_connections 2024-09-13T18:22:35.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.745+0000 7f43c3e90640 1 --2- 172.21.15.114:0/515665784 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43bc1050a0 0x7f43bc10cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.745+0000 7f43c3e90640 1 --2- 172.21.15.114:0/515665784 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43bc100900 0x7f43bc104940 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.745+0000 7f43c3e90640 1 --2- 172.21.15.114:0/515665784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43bc0fff50 0x7f43bc100330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.745+0000 7f43c3e90640 1 -- 172.21.15.114:0/515665784 >> 172.21.15.114:0/515665784 conn(0x7f43bc0fb9c0 msgr2=0x7f43bc0fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:35.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.745+0000 7f43c3e90640 1 -- 172.21.15.114:0/515665784 shutdown_connections 2024-09-13T18:22:35.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.745+0000 7f43c3e90640 1 -- 172.21.15.114:0/515665784 wait complete. 2024-09-13T18:22:35.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.746+0000 7f43c3e90640 1 Processor -- start 2024-09-13T18:22:35.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.746+0000 7f43c3e90640 1 -- start start 2024-09-13T18:22:35.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.746+0000 7f43c3e90640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43bc0fff50 0x7f43bc196220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:35.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.746+0000 7f43c3e90640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43bc1050a0 0x7f43bc196760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:35.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.746+0000 7f43c3e90640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43bc19ab50 0x7f43bc19b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:35.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.746+0000 7f43c3e90640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43bc19b540 con 0x7f43bc1050a0 2024-09-13T18:22:35.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.746+0000 7f43c3e90640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43bc19b6b0 con 0x7f43bc19ab50 2024-09-13T18:22:35.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.746+0000 7f43c3e90640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43bc19b820 con 0x7f43bc0fff50 2024-09-13T18:22:35.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.746+0000 7f6ad91d5640 1 -- 172.21.15.114:0/3871354009 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ad41002b0 con 0x7f6ad40ff970 2024-09-13T18:22:35.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.746+0000 7f43c142b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43bc1050a0 0x7f43bc196760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:35.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.747+0000 7f43c1c2c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43bc0fff50 0x7f43bc196220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:35.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.747+0000 7f43c142b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43bc1050a0 0x7f43bc196760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50934/0 (socket says 172.21.15.114:50934) 2024-09-13T18:22:35.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.747+0000 7f43c142b640 1 -- 172.21.15.114:0/137066389 learned_addr learned my addr 172.21.15.114:0/137066389 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:35.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.747+0000 7f43c242d640 1 --2- 172.21.15.114:0/137066389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43bc19ab50 0x7f43bc19b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:35.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.747+0000 7f43c142b640 1 -- 172.21.15.114:0/137066389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43bc0fff50 msgr2=0x7f43bc196220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:35.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.747+0000 7f43c142b640 1 --2- 172.21.15.114:0/137066389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43bc0fff50 0x7f43bc196220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.747+0000 7f43c142b640 1 -- 172.21.15.114:0/137066389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43bc19ab50 msgr2=0x7f43bc19b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:35.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.747+0000 7f43c142b640 1 --2- 172.21.15.114:0/137066389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43bc19ab50 0x7f43bc19b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.747+0000 7f43c142b640 1 -- 172.21.15.114:0/137066389 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43ac009560 con 0x7f43bc1050a0 2024-09-13T18:22:35.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.747+0000 7f43c242d640 1 --2- 172.21.15.114:0/137066389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43bc19ab50 0x7f43bc19b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:22:35.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.747+0000 7f43c142b640 1 --2- 172.21.15.114:0/137066389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43bc1050a0 0x7f43bc196760 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f43ac005ab0 tx=0x7f43ac032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:35.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.747+0000 7f43b2ffd640 1 -- 172.21.15.114:0/137066389 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43ac032e10 con 0x7f43bc1050a0 2024-09-13T18:22:35.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.747+0000 7f43b2ffd640 1 -- 172.21.15.114:0/137066389 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f43ac0046c0 con 0x7f43bc1050a0 2024-09-13T18:22:35.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.747+0000 7f43c3e90640 1 -- 172.21.15.114:0/137066389 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43bc1a2e00 con 0x7f43bc1050a0 2024-09-13T18:22:35.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.747+0000 7f43b2ffd640 1 -- 172.21.15.114:0/137066389 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43ac003850 con 0x7f43bc1050a0 2024-09-13T18:22:35.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.747+0000 7f43c3e90640 1 -- 172.21.15.114:0/137066389 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43bc1a3220 con 0x7f43bc1050a0 2024-09-13T18:22:35.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.748+0000 7f6ad0d99640 1 -- 172.21.15.114:0/3871354009 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 24) v1 ==== 136050+0+0 (secure 0 0 0) 0x7f6abc004f50 con 0x7f6ad40ff970 2024-09-13T18:22:35.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.748+0000 7f6ad0d99640 1 --2- 172.21.15.114:0/3871354009 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f6ac00a1310 0x7f6ac00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:35.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.749+0000 7f6ad2d9d640 1 --2- 172.21.15.114:0/3871354009 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f6ac00a1310 0x7f6ac00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:35.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.749+0000 7f6ad0d99640 1 -- 172.21.15.114:0/3871354009 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f6abc0cce00 con 0x7f6ad40ff970 2024-09-13T18:22:35.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.749+0000 7f6ad2d9d640 1 --2- 172.21.15.114:0/3871354009 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f6ac00a1310 0x7f6ac00a37d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4009900 tx=0x7f6ac4005a20 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:35.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.750+0000 7f43b2ffd640 1 -- 172.21.15.114:0/137066389 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 24) v1 ==== 136050+0+0 (secure 0 0 0) 0x7f43ac004830 con 0x7f43bc1050a0 2024-09-13T18:22:35.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.750+0000 7f43c6c8b740 1 Processor -- start 2024-09-13T18:22:35.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.750+0000 7f43c6c8b740 1 -- start start 2024-09-13T18:22:35.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.750+0000 7f43b2ffd640 1 --2- 172.21.15.114:0/137066389 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f43a80a1310 0x7f43a80a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:35.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.751+0000 7f43c6c8b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b600ebb7c0 0x55b600ebbba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:35.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.751+0000 7f43c6c8b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b600ebc0e0 0x55b600ec6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:35.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.751+0000 7f43c6c8b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b600ec6980 0x55b600ec8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:35.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.751+0000 7f43c6c8b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b600e26c60 con 0x55b600ec6980 2024-09-13T18:22:35.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.751+0000 7f43c6c8b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b600e243f0 con 0x55b600ebb7c0 2024-09-13T18:22:35.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.751+0000 7f43c6c8b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b600e1fce0 con 0x55b600ebc0e0 2024-09-13T18:22:35.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.751+0000 7f43c242d640 1 --2- 172.21.15.114:0/137066389 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f43a80a1310 0x7f43a80a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:35.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.751+0000 7f43b2ffd640 1 -- 172.21.15.114:0/137066389 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f43ac0edda0 con 0x7f43bc1050a0 2024-09-13T18:22:35.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.751+0000 7f43c242d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b600ec6980 0x55b600ec8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:35.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.751+0000 7f43c242d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b600ec6980 0x55b600ec8d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50938/0 (socket says 172.21.15.114:50938) 2024-09-13T18:22:35.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.751+0000 7f43c242d640 1 -- 172.21.15.114:0/2796279887 learned_addr learned my addr 172.21.15.114:0/2796279887 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:35.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.751+0000 7f43c142b640 1 --2- 172.21.15.114:0/2796279887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b600ebc0e0 0x55b600ec6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:35.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.751+0000 7f43c1c2c640 1 --2- 172.21.15.114:0/2796279887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b600ebb7c0 0x55b600ebbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:35.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.751+0000 7f43c242d640 1 --2- 172.21.15.114:0/137066389 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f43a80a1310 0x7f43a80a37d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f43b40147a0 tx=0x7f43b4014ba0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:35.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.751+0000 7f43c242d640 1 -- 172.21.15.114:0/2796279887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b600ebc0e0 msgr2=0x55b600ec6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:35.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.751+0000 7f43c242d640 1 --2- 172.21.15.114:0/2796279887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b600ebc0e0 0x55b600ec6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.751+0000 7f43c242d640 1 -- 172.21.15.114:0/2796279887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b600ebb7c0 msgr2=0x55b600ebbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:35.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.751+0000 7f43c242d640 1 --2- 172.21.15.114:0/2796279887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b600ebb7c0 0x55b600ebbba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.751+0000 7f43c242d640 1 -- 172.21.15.114:0/2796279887 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b600ddb970 con 0x55b600ec6980 2024-09-13T18:22:35.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.751+0000 7f43c242d640 1 --2- 172.21.15.114:0/2796279887 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b600ec6980 0x55b600ec8d70 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f43b4012760 tx=0x7f43b401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d217fb8580a76d15 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:35.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.753+0000 7f43a7fff640 1 -- 172.21.15.114:0/2796279887 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b400eae0 con 0x55b600ec6980 2024-09-13T18:22:35.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.753+0000 7f43a7fff640 1 -- 172.21.15.114:0/2796279887 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f43b401a070 con 0x55b600ec6980 2024-09-13T18:22:35.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.753+0000 7f43a7fff640 1 -- 172.21.15.114:0/2796279887 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b401f3d0 con 0x55b600ec6980 2024-09-13T18:22:35.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.753+0000 7f6ad0d99640 1 -- 172.21.15.114:0/3871354009 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f6abc095e40 con 0x7f6ad40ff970 2024-09-13T18:22:35.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.753+0000 7f43c6c8b740 1 -- 172.21.15.114:0/2796279887 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b600ec6980 msgr2=0x55b600ec8d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:35.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.753+0000 7f43c6c8b740 1 --2- 172.21.15.114:0/2796279887 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b600ec6980 0x55b600ec8d70 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f43b4012760 tx=0x7f43b401e000 comp rx=0 tx=0).stop 2024-09-13T18:22:35.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.753+0000 7f43c6c8b740 1 -- 172.21.15.114:0/2796279887 shutdown_connections 2024-09-13T18:22:35.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.753+0000 7f43c6c8b740 1 --2- 172.21.15.114:0/2796279887 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b600ec6980 0x55b600ec8d70 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.753+0000 7f43c6c8b740 1 --2- 172.21.15.114:0/2796279887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b600ebc0e0 0x55b600ec6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.753+0000 7f43c6c8b740 1 --2- 172.21.15.114:0/2796279887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b600ebb7c0 0x55b600ebbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.753+0000 7f43c6c8b740 1 -- 172.21.15.114:0/2796279887 >> 172.21.15.114:0/2796279887 conn(0x55b600eb7d50 msgr2=0x55b600eb8130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:35.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.753+0000 7f43c6c8b740 1 -- 172.21.15.114:0/2796279887 shutdown_connections 2024-09-13T18:22:35.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.753+0000 7f43c6c8b740 1 -- 172.21.15.114:0/2796279887 wait complete. 2024-09-13T18:22:35.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.754+0000 7f43c6c8b740 1 -- start start 2024-09-13T18:22:35.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.754+0000 7f43c6c8b740 1 Processor -- start 2024-09-13T18:22:35.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.754+0000 7f43c6c8b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b600ebb7c0 0x55b600ec6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:35.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.754+0000 7f43c6c8b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b600ebc0e0 0x55b600ec3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:35.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.754+0000 7f43c6c8b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b600ec6980 0x55b600ec3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:35.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.754+0000 7f43c6c8b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b600dddf80 con 0x55b600ec6980 2024-09-13T18:22:35.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.754+0000 7f43c6c8b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b600dddc20 con 0x55b600ebc0e0 2024-09-13T18:22:35.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.754+0000 7f43c6c8b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b600daaab0 con 0x55b600ebb7c0 2024-09-13T18:22:35.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.754+0000 7f43c6c8b740 10 client.0 _ll_register_callbacks cb 0x55b600e19350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:22:35.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.754+0000 7f43c242d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b600ec6980 0x55b600ec3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:35.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.754+0000 7f43c242d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b600ec6980 0x55b600ec3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50942/0 (socket says 172.21.15.114:50942) 2024-09-13T18:22:35.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.754+0000 7f43c242d640 1 -- 172.21.15.114:0/3120112206 learned_addr learned my addr 172.21.15.114:0/3120112206 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:35.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.755+0000 7f43c142b640 1 --2- 172.21.15.114:0/3120112206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b600ebc0e0 0x55b600ec3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:35.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.755+0000 7f43c242d640 1 -- 172.21.15.114:0/3120112206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b600ebb7c0 msgr2=0x55b600ec6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:35.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.755+0000 7f43c1c2c640 1 --2- 172.21.15.114:0/3120112206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b600ebb7c0 0x55b600ec6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:35.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.755+0000 7f43c242d640 1 --2- 172.21.15.114:0/3120112206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b600ebb7c0 0x55b600ec6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.755+0000 7f43c242d640 1 -- 172.21.15.114:0/3120112206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b600ebc0e0 msgr2=0x55b600ec3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:35.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.755+0000 7f43c242d640 1 --2- 172.21.15.114:0/3120112206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b600ebc0e0 0x55b600ec3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.755+0000 7f43c242d640 1 -- 172.21.15.114:0/3120112206 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43b4014df0 con 0x55b600ec6980 2024-09-13T18:22:35.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.755+0000 7f43c142b640 1 --2- 172.21.15.114:0/3120112206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b600ebc0e0 0x55b600ec3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:22:35.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.755+0000 7f43c1c2c640 1 --2- 172.21.15.114:0/3120112206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b600ebb7c0 0x55b600ec6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:22:35.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.755+0000 7f43c242d640 1 --2- 172.21.15.114:0/3120112206 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b600ec6980 0x55b600ec3cf0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f43b4009620 tx=0x7f43b4020b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2427fe3aeaa814cb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:35.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.755+0000 7f43a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:22:35.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.755+0000 7f43a57fa640 1 -- 172.21.15.114:0/3120112206 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b402d070 con 0x55b600ec6980 2024-09-13T18:22:35.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.755+0000 7f43a57fa640 1 -- 172.21.15.114:0/3120112206 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f43b4009e50 con 0x55b600ec6980 2024-09-13T18:22:35.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.755+0000 7f43c6c8b740 1 -- 172.21.15.114:0/3120112206 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b600b13590 con 0x55b600ec6980 2024-09-13T18:22:35.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.755+0000 7f43a57fa640 1 -- 172.21.15.114:0/3120112206 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43b4020650 con 0x55b600ec6980 2024-09-13T18:22:35.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.755+0000 7f43c6c8b740 10 client.15756 fetch_fsmap learned FSMap version 30 2024-09-13T18:22:35.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.755+0000 7f43c6c8b740 1 -- 172.21.15.114:0/3120112206 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55b600e1e360 con 0x55b600ec6980 2024-09-13T18:22:35.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.755+0000 7f43a57fa640 1 -- 172.21.15.114:0/3120112206 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b600b13590 con 0x55b600ec6980 2024-09-13T18:22:35.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.756+0000 7f43a57fa640 1 -- 172.21.15.114:0/3120112206 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f43b4020d20 con 0x55b600ec6980 2024-09-13T18:22:35.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.756+0000 7f43c6c8b740 10 client.15756 fetch_fsmap finished waiting for FSMap version 30 2024-09-13T18:22:35.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.756+0000 7f43c6c8b740 10 client.15756 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:22:35.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.756+0000 7f43c6c8b740 20 client.15756 populate_metadata read hostname 'smithi114' 2024-09-13T18:22:35.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.756+0000 7f43c6c8b740 1 --2- 172.21.15.114:0/3120112206 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55b600ed6670 0x55b600ed8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:35.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.756+0000 7f43c6c8b740 4 client.15756 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:22:35.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.756+0000 7f43c6c8b740 1 -- 172.21.15.114:0/3120112206 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b600b24860 con 0x55b600ed6670 2024-09-13T18:22:35.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.756+0000 7f43c1c2c640 1 --2- 172.21.15.114:0/3120112206 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55b600ed6670 0x55b600ed8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:35.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.756+0000 7f43a57fa640 10 client.15756 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:22:35.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.756+0000 7f43c1c2c640 1 --2- 172.21.15.114:0/3120112206 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55b600ed6670 0x55b600ed8a90 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da2a883154002315 server_cookie=765e3edf716c4cc4 in_seq=0 out_seq=0 2024-09-13T18:22:35.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.757+0000 7f43a57fa640 1 -- 172.21.15.114:0/3120112206 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55b600b24860 con 0x55b600ed6670 2024-09-13T18:22:35.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.757+0000 7f43a57fa640 10 client.15756 handle_command_reply: tid=0 2024-09-13T18:22:35.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.757+0000 7f43c6c8b740 1 client.15756 shutdown 2024-09-13T18:22:35.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.757+0000 7f43c6c8b740 1 -- 172.21.15.114:0/3120112206 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b600ec6980 msgr2=0x55b600ec3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:35.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.757+0000 7f43c6c8b740 1 --2- 172.21.15.114:0/3120112206 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b600ec6980 0x55b600ec3cf0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f43b4009620 tx=0x7f43b4020b80 comp rx=0 tx=0).stop 2024-09-13T18:22:35.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.757+0000 7f43c6c8b740 1 -- 172.21.15.114:0/3120112206 shutdown_connections 2024-09-13T18:22:35.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.757+0000 7f43c6c8b740 1 --2- 172.21.15.114:0/3120112206 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55b600ed6670 0x55b600ed8a90 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.757+0000 7f43c6c8b740 1 --2- 172.21.15.114:0/3120112206 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b600ec6980 0x55b600ec3cf0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.757+0000 7f43a57fa640 0 client.15756 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:22:35.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.757+0000 7f43c6c8b740 1 --2- 172.21.15.114:0/3120112206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b600ebc0e0 0x55b600ec3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.757+0000 7f43c6c8b740 1 --2- 172.21.15.114:0/3120112206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b600ebb7c0 0x55b600ec6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.757+0000 7f43c6c8b740 1 -- 172.21.15.114:0/3120112206 >> 172.21.15.114:0/3120112206 conn(0x55b600eb7d50 msgr2=0x55b600ebd1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:35.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.757+0000 7f43c6c8b740 1 -- 172.21.15.114:0/3120112206 shutdown_connections 2024-09-13T18:22:35.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.757+0000 7f43c6c8b740 1 -- 172.21.15.114:0/3120112206 wait complete. 2024-09-13T18:22:35.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.758+0000 7f43c6c8b740 20 client.15756 trim_cache size 0 max 16384 2024-09-13T18:22:35.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.772+0000 7f43c6c8b740 1 Processor -- start 2024-09-13T18:22:35.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.772+0000 7f43c6c8b740 1 -- start start 2024-09-13T18:22:35.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.772+0000 7f43c6c8b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b600eb7d50 0x55b600ecd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:35.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.772+0000 7f43c6c8b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b600ebb7c0 0x55b600ecdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:35.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.772+0000 7f43c6c8b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b600ebc0e0 0x55b600ebd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:35.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.772+0000 7f43c6c8b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b600ec9430 con 0x55b600ebb7c0 2024-09-13T18:22:35.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.772+0000 7f43c6c8b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b600eb8b40 con 0x55b600ebc0e0 2024-09-13T18:22:35.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.772+0000 7f43c6c8b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b600ebb610 con 0x55b600eb7d50 2024-09-13T18:22:35.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.772+0000 7f43c142b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b600ebb7c0 0x55b600ecdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:35.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.772+0000 7f43c142b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b600ebb7c0 0x55b600ecdbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50956/0 (socket says 172.21.15.114:50956) 2024-09-13T18:22:35.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.772+0000 7f43c142b640 1 -- 172.21.15.114:0/784274038 learned_addr learned my addr 172.21.15.114:0/784274038 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:35.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.772+0000 7f43c1c2c640 1 --2- 172.21.15.114:0/784274038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b600eb7d50 0x55b600ecd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:35.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.772+0000 7f43c142b640 1 -- 172.21.15.114:0/784274038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b600eb7d50 msgr2=0x55b600ecd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:35.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.772+0000 7f43c142b640 1 --2- 172.21.15.114:0/784274038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b600eb7d50 0x55b600ecd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.772+0000 7f43c142b640 1 -- 172.21.15.114:0/784274038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b600ebc0e0 msgr2=0x55b600ebd1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:35.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.772+0000 7f43c242d640 1 --2- 172.21.15.114:0/784274038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b600ebc0e0 0x55b600ebd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:35.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.772+0000 7f43c142b640 1 --2- 172.21.15.114:0/784274038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b600ebc0e0 0x55b600ebd1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.772+0000 7f43c142b640 1 -- 172.21.15.114:0/784274038 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b600dc01c0 con 0x55b600ebb7c0 2024-09-13T18:22:35.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.772+0000 7f43c1c2c640 1 --2- 172.21.15.114:0/784274038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b600eb7d50 0x55b600ecd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:22:35.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.772+0000 7f43c142b640 1 --2- 172.21.15.114:0/784274038 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b600ebb7c0 0x55b600ecdbc0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f43ac0eb170 tx=0x7f43ac005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44c8f5fab7f79710 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:35.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.773+0000 7f43b0ff9640 1 -- 172.21.15.114:0/784274038 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43ac0321e0 con 0x55b600ebb7c0 2024-09-13T18:22:35.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.773+0000 7f43b0ff9640 1 -- 172.21.15.114:0/784274038 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f43ac045410 con 0x55b600ebb7c0 2024-09-13T18:22:35.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.773+0000 7f43b0ff9640 1 -- 172.21.15.114:0/784274038 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43ac0325a0 con 0x55b600ebb7c0 2024-09-13T18:22:35.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.774+0000 7f43c6c8b740 1 -- 172.21.15.114:0/784274038 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b600ebb7c0 msgr2=0x55b600ecdbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:35.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.774+0000 7f43c6c8b740 1 --2- 172.21.15.114:0/784274038 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b600ebb7c0 0x55b600ecdbc0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f43ac0eb170 tx=0x7f43ac005e40 comp rx=0 tx=0).stop 2024-09-13T18:22:35.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.774+0000 7f43c6c8b740 1 -- 172.21.15.114:0/784274038 shutdown_connections 2024-09-13T18:22:35.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.774+0000 7f43c6c8b740 1 --2- 172.21.15.114:0/784274038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b600ebc0e0 0x55b600ebd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.774+0000 7f43c6c8b740 1 --2- 172.21.15.114:0/784274038 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b600ebb7c0 0x55b600ecdbc0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.774+0000 7f43c6c8b740 1 --2- 172.21.15.114:0/784274038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b600eb7d50 0x55b600ecd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.774+0000 7f43c6c8b740 1 -- 172.21.15.114:0/784274038 >> 172.21.15.114:0/784274038 conn(0x55b600ec64d0 msgr2=0x55b600ec1f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:35.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.774+0000 7f43c6c8b740 1 -- 172.21.15.114:0/784274038 shutdown_connections 2024-09-13T18:22:35.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.774+0000 7f43c6c8b740 1 -- 172.21.15.114:0/784274038 wait complete. 2024-09-13T18:22:35.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.774+0000 7f43c6c8b740 1 -- start start 2024-09-13T18:22:35.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.774+0000 7f43c6c8b740 1 Processor -- start 2024-09-13T18:22:35.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.775+0000 7f43c6c8b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b600eb7d50 0x55b600f261c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:35.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.775+0000 7f43c6c8b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b600ebb7c0 0x55b600f26700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:35.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.775+0000 7f43c6c8b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b600ebc0e0 0x55b600eccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:35.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.775+0000 7f43c6c8b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b600ecd310 con 0x55b600eb7d50 2024-09-13T18:22:35.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.775+0000 7f43c6c8b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b600ecd480 con 0x55b600ebb7c0 2024-09-13T18:22:35.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.775+0000 7f43c6c8b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b600ecd640 con 0x55b600ebc0e0 2024-09-13T18:22:35.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.775+0000 7f43c6c8b740 10 client.0 _ll_register_callbacks cb 0x7f43b8004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:22:35.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.775+0000 7f43c1c2c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b600eb7d50 0x55b600f261c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:35.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.775+0000 7f43c1c2c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b600eb7d50 0x55b600f261c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50960/0 (socket says 172.21.15.114:50960) 2024-09-13T18:22:35.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.775+0000 7f43c1c2c640 1 -- 172.21.15.114:0/3422477388 learned_addr learned my addr 172.21.15.114:0/3422477388 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:35.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.775+0000 7f43c142b640 1 --2- 172.21.15.114:0/3422477388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b600ebb7c0 0x55b600f26700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:35.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.775+0000 7f43c242d640 1 --2- 172.21.15.114:0/3422477388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b600ebc0e0 0x55b600eccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:35.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.775+0000 7f43c1c2c640 1 -- 172.21.15.114:0/3422477388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b600ebc0e0 msgr2=0x55b600eccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:35.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.775+0000 7f43c1c2c640 1 --2- 172.21.15.114:0/3422477388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b600ebc0e0 0x55b600eccda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.775+0000 7f43c1c2c640 1 -- 172.21.15.114:0/3422477388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b600ebb7c0 msgr2=0x55b600f26700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:35.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.775+0000 7f43c1c2c640 1 --2- 172.21.15.114:0/3422477388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b600ebb7c0 0x55b600f26700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.775+0000 7f43c1c2c640 1 -- 172.21.15.114:0/3422477388 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43bc1a3220 con 0x55b600eb7d50 2024-09-13T18:22:35.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.775+0000 7f43c242d640 1 --2- 172.21.15.114:0/3422477388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b600ebc0e0 0x55b600eccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:22:35.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.775+0000 7f43c1c2c640 1 --2- 172.21.15.114:0/3422477388 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b600eb7d50 0x55b600f261c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f43b8005f00 tx=0x7f43b800be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c044d03dca76fffc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:35.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.775+0000 7f43a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:22:35.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.775+0000 7f43a57fa640 1 -- 172.21.15.114:0/3422477388 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b8017040 con 0x55b600eb7d50 2024-09-13T18:22:35.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.775+0000 7f43a57fa640 1 -- 172.21.15.114:0/3422477388 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f43b801c3a0 con 0x55b600eb7d50 2024-09-13T18:22:35.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.775+0000 7f43c6c8b740 1 -- 172.21.15.114:0/3422477388 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f43b401a070 con 0x55b600eb7d50 2024-09-13T18:22:35.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.776+0000 7f43a57fa640 1 -- 172.21.15.114:0/3422477388 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43b801c560 con 0x55b600eb7d50 2024-09-13T18:22:35.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.776+0000 7f43c6c8b740 10 client.15762 fetch_fsmap learned FSMap version 30 2024-09-13T18:22:35.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.776+0000 7f43c6c8b740 1 -- 172.21.15.114:0/3422477388 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55b600de5990 con 0x55b600eb7d50 2024-09-13T18:22:35.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.776+0000 7f43a57fa640 1 -- 172.21.15.114:0/3422477388 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f43b401a070 con 0x55b600eb7d50 2024-09-13T18:22:35.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.776+0000 7f43a57fa640 1 -- 172.21.15.114:0/3422477388 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f43b8011c00 con 0x55b600eb7d50 2024-09-13T18:22:35.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.776+0000 7f43c6c8b740 10 client.15762 fetch_fsmap finished waiting for FSMap version 30 2024-09-13T18:22:35.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.776+0000 7f43c6c8b740 10 client.15762 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:22:35.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.776+0000 7f43c6c8b740 20 client.15762 populate_metadata read hostname 'smithi114' 2024-09-13T18:22:35.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.776+0000 7f43c6c8b740 1 --2- 172.21.15.114:0/3422477388 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55b600ec6980 0x55b600f31190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:35.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.776+0000 7f43c6c8b740 4 client.15762 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-13T18:22:35.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.776+0000 7f43c6c8b740 1 -- 172.21.15.114:0/3422477388 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55b600b77eb0 con 0x55b600ec6980 2024-09-13T18:22:35.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.776+0000 7f43c142b640 1 --2- 172.21.15.114:0/3422477388 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55b600ec6980 0x55b600f31190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:35.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.777+0000 7f43a57fa640 10 client.15762 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:22:35.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.777+0000 7f43c142b640 1 --2- 172.21.15.114:0/3422477388 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55b600ec6980 0x55b600f31190 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89d6e76a508c7b92 server_cookie=3521af50187356ec in_seq=0 out_seq=0 2024-09-13T18:22:35.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.778+0000 7f43a57fa640 1 -- 172.21.15.114:0/3422477388 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55b600b77eb0 con 0x55b600ec6980 2024-09-13T18:22:35.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.778+0000 7f43a57fa640 10 client.15762 handle_command_reply: tid=0 2024-09-13T18:22:35.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.778+0000 7f43c6c8b740 1 client.15762 shutdown 2024-09-13T18:22:35.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.778+0000 7f43c6c8b740 1 -- 172.21.15.114:0/3422477388 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b600eb7d50 msgr2=0x55b600f261c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:35.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.778+0000 7f43c6c8b740 1 --2- 172.21.15.114:0/3422477388 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b600eb7d50 0x55b600f261c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f43b8005f00 tx=0x7f43b800be00 comp rx=0 tx=0).stop 2024-09-13T18:22:35.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.778+0000 7f43c6c8b740 1 -- 172.21.15.114:0/3422477388 shutdown_connections 2024-09-13T18:22:35.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.778+0000 7f43c6c8b740 1 --2- 172.21.15.114:0/3422477388 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55b600ec6980 0x55b600f31190 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.778+0000 7f43a57fa640 0 client.15762 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:22:35.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.778+0000 7f43c6c8b740 1 --2- 172.21.15.114:0/3422477388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b600ebc0e0 0x55b600eccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.778+0000 7f43c6c8b740 1 --2- 172.21.15.114:0/3422477388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b600ebb7c0 0x55b600f26700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.778+0000 7f43c6c8b740 1 --2- 172.21.15.114:0/3422477388 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b600eb7d50 0x55b600f261c0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.778+0000 7f43c6c8b740 1 -- 172.21.15.114:0/3422477388 >> 172.21.15.114:0/3422477388 conn(0x55b600ec64d0 msgr2=0x55b600ebd1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:35.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.778+0000 7f43c6c8b740 1 -- 172.21.15.114:0/3422477388 shutdown_connections 2024-09-13T18:22:35.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.778+0000 7f43c6c8b740 1 -- 172.21.15.114:0/3422477388 wait complete. 2024-09-13T18:22:35.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.778+0000 7f43c6c8b740 20 client.15762 trim_cache size 0 max 16384 2024-09-13T18:22:35.793 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:22:35.793 INFO:teuthology.orchestra.run.smithi114.stdout: "return_code": 0, 2024-09-13T18:22:35.793 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_tag": "f7ef14df-4102-4786-8e08-4da08feaa73e", 2024-09-13T18:22:35.793 INFO:teuthology.orchestra.run.smithi114.stdout: "mode": "asynchronous" 2024-09-13T18:22:35.793 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:22:35.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.779+0000 7f43c3e90640 1 -- 172.21.15.114:0/137066389 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f43a80a1310 msgr2=0x7f43a80a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:35.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.779+0000 7f43c3e90640 1 --2- 172.21.15.114:0/137066389 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f43a80a1310 0x7f43a80a37d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f43b40147a0 tx=0x7f43b4014ba0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.779+0000 7f43c3e90640 1 -- 172.21.15.114:0/137066389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43bc1050a0 msgr2=0x7f43bc196760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:35.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.779+0000 7f43c3e90640 1 --2- 172.21.15.114:0/137066389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43bc1050a0 0x7f43bc196760 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f43ac005ab0 tx=0x7f43ac032c20 comp rx=0 tx=0).stop 2024-09-13T18:22:35.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.779+0000 7f43c3e90640 1 -- 172.21.15.114:0/137066389 shutdown_connections 2024-09-13T18:22:35.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.779+0000 7f43c3e90640 1 --2- 172.21.15.114:0/137066389 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f43a80a1310 0x7f43a80a37d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.779+0000 7f43c3e90640 1 --2- 172.21.15.114:0/137066389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43bc19ab50 0x7f43bc19b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.779+0000 7f43c3e90640 1 --2- 172.21.15.114:0/137066389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43bc1050a0 0x7f43bc196760 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.779+0000 7f43c3e90640 1 --2- 172.21.15.114:0/137066389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43bc0fff50 0x7f43bc196220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.779+0000 7f43c3e90640 1 -- 172.21.15.114:0/137066389 >> 172.21.15.114:0/137066389 conn(0x7f43bc0fb9c0 msgr2=0x7f43bc0fce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:35.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.779+0000 7f43c3e90640 1 -- 172.21.15.114:0/137066389 shutdown_connections 2024-09-13T18:22:35.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.779+0000 7f43c3e90640 1 -- 172.21.15.114:0/137066389 wait complete. 2024-09-13T18:22:35.797 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T18:22:35.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.896+0000 7fc198053640 1 -- 172.21.15.114:0/1994662274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc190100f40 msgr2=0x7fc190101320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:35.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.896+0000 7fc198053640 1 --2- 172.21.15.114:0/1994662274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc190100f40 0x7fc190101320 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fc180009900 tx=0x7fc18002a480 comp rx=0 tx=0).stop 2024-09-13T18:22:35.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.896+0000 7fc198053640 1 -- 172.21.15.114:0/1994662274 shutdown_connections 2024-09-13T18:22:35.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.896+0000 7fc198053640 1 --2- 172.21.15.114:0/1994662274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc19010a310 0x7fc19010c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.896+0000 7fc198053640 1 --2- 172.21.15.114:0/1994662274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc190101860 0x7fc190109d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.896+0000 7fc198053640 1 --2- 172.21.15.114:0/1994662274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc190100f40 0x7fc190101320 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.896+0000 7fc198053640 1 -- 172.21.15.114:0/1994662274 >> 172.21.15.114:0/1994662274 conn(0x7fc190075c60 msgr2=0x7fc1900fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:35.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.896+0000 7fc198053640 1 -- 172.21.15.114:0/1994662274 shutdown_connections 2024-09-13T18:22:35.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.896+0000 7fc198053640 1 -- 172.21.15.114:0/1994662274 wait complete. 2024-09-13T18:22:35.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.896+0000 7fc198053640 1 Processor -- start 2024-09-13T18:22:35.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.896+0000 7fc198053640 1 -- start start 2024-09-13T18:22:35.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.896+0000 7fc198053640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc190100f40 0x7fc1901961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:35.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.896+0000 7fc198053640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc190101860 0x7fc190196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:35.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.896+0000 7fc198053640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc19010a310 0x7fc19019aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:35.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.897+0000 7fc198053640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19019b0c0 con 0x7fc190100f40 2024-09-13T18:22:35.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.897+0000 7fc198053640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19019b230 con 0x7fc19010a310 2024-09-13T18:22:35.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.897+0000 7fc198053640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19019b3f0 con 0x7fc190101860 2024-09-13T18:22:35.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.897+0000 7fc195def640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc190100f40 0x7fc1901961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:35.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.897+0000 7fc195def640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc190100f40 0x7fc1901961e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50970/0 (socket says 172.21.15.114:50970) 2024-09-13T18:22:35.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.897+0000 7fc195def640 1 -- 172.21.15.114:0/3612960243 learned_addr learned my addr 172.21.15.114:0/3612960243 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:35.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.897+0000 7fc1955ee640 1 --2- 172.21.15.114:0/3612960243 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc190101860 0x7fc190196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:35.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.897+0000 7fc195def640 1 -- 172.21.15.114:0/3612960243 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc190101860 msgr2=0x7fc190196720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:35.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.897+0000 7fc195def640 1 --2- 172.21.15.114:0/3612960243 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc190101860 0x7fc190196720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.897+0000 7fc195def640 1 -- 172.21.15.114:0/3612960243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc19010a310 msgr2=0x7fc19019aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:35.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.897+0000 7fc1965f0640 1 --2- 172.21.15.114:0/3612960243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc19010a310 0x7fc19019aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:35.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.897+0000 7fc195def640 1 --2- 172.21.15.114:0/3612960243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc19010a310 0x7fc19019aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.897+0000 7fc195def640 1 -- 172.21.15.114:0/3612960243 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc180009560 con 0x7fc190100f40 2024-09-13T18:22:35.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.897+0000 7fc195def640 1 --2- 172.21.15.114:0/3612960243 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc190100f40 0x7fc1901961e0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fc1800057c0 tx=0x7fc180032be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:35.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.897+0000 7fc186ffd640 1 -- 172.21.15.114:0/3612960243 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc180005000 con 0x7fc190100f40 2024-09-13T18:22:35.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.897+0000 7fc186ffd640 1 -- 172.21.15.114:0/3612960243 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc1800046c0 con 0x7fc190100f40 2024-09-13T18:22:35.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.897+0000 7fc198053640 1 -- 172.21.15.114:0/3612960243 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc19019b670 con 0x7fc190100f40 2024-09-13T18:22:35.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.897+0000 7fc186ffd640 1 -- 172.21.15.114:0/3612960243 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc180003850 con 0x7fc190100f40 2024-09-13T18:22:35.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.897+0000 7fc198053640 1 -- 172.21.15.114:0/3612960243 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1901a2f50 con 0x7fc190100f40 2024-09-13T18:22:35.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.899+0000 7fc186ffd640 1 -- 172.21.15.114:0/3612960243 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 24) v1 ==== 136050+0+0 (secure 0 0 0) 0x7fc180005160 con 0x7fc190100f40 2024-09-13T18:22:35.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.900+0000 7fc19ae52740 1 Processor -- start 2024-09-13T18:22:35.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.900+0000 7fc19ae52740 1 -- start start 2024-09-13T18:22:35.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.900+0000 7fc19ae52740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb0036d10 0x55fbb00370f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:35.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.900+0000 7fc19ae52740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb0037630 0x55fbb0045ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:35.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.900+0000 7fc19ae52740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbb00463e0 0x55fbb00487d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:35.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.900+0000 7fc19ae52740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbaffa1f20 con 0x55fbb0036d10 2024-09-13T18:22:35.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.900+0000 7fc19ae52740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbaff9f6b0 con 0x55fbb0037630 2024-09-13T18:22:35.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.900+0000 7fc19ae52740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbaff9afa0 con 0x55fbb00463e0 2024-09-13T18:22:35.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.900+0000 7fc1965f0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb0036d10 0x55fbb00370f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:35.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.900+0000 7fc1965f0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb0036d10 0x55fbb00370f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50984/0 (socket says 172.21.15.114:50984) 2024-09-13T18:22:35.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.900+0000 7fc1965f0640 1 -- 172.21.15.114:0/1529503974 learned_addr learned my addr 172.21.15.114:0/1529503974 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:35.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.900+0000 7fc186ffd640 1 --2- 172.21.15.114:0/3612960243 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc17c0a1310 0x7fc17c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:35.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.900+0000 7fc195def640 1 --2- 172.21.15.114:0/1529503974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb0037630 0x55fbb0045ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:35.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.900+0000 7fc1955ee640 1 --2- 172.21.15.114:0/1529503974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbb00463e0 0x55fbb00487d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:35.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.901+0000 7fc1965f0640 1 -- 172.21.15.114:0/1529503974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbb00463e0 msgr2=0x55fbb00487d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:35.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.901+0000 7fc1965f0640 1 --2- 172.21.15.114:0/1529503974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbb00463e0 0x55fbb00487d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.901+0000 7fc1965f0640 1 -- 172.21.15.114:0/1529503974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb0037630 msgr2=0x55fbb0045ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:35.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.901+0000 7fc1965f0640 1 --2- 172.21.15.114:0/1529503974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb0037630 0x55fbb0045ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.901+0000 7fc1965f0640 1 -- 172.21.15.114:0/1529503974 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fbaff55840 con 0x55fbb0036d10 2024-09-13T18:22:35.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.901+0000 7fc195def640 1 --2- 172.21.15.114:0/1529503974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb0037630 0x55fbb0045ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:22:35.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.901+0000 7fc1955ee640 1 --2- 172.21.15.114:0/1529503974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbb00463e0 0x55fbb00487d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:22:35.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.901+0000 7fc186ffd640 1 -- 172.21.15.114:0/3612960243 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fc1800ee0c0 con 0x7fc190100f40 2024-09-13T18:22:35.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.901+0000 7fc1965f0640 1 --2- 172.21.15.114:0/3612960243 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc17c0a1310 0x7fc17c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:35.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.901+0000 7fc1965f0640 1 --2- 172.21.15.114:0/1529503974 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb0036d10 0x55fbb00370f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fc188009f70 tx=0x7fc18800f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46fbeb226e25eee2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:35.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.901+0000 7fc17bfff640 1 -- 172.21.15.114:0/1529503974 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc188020030 con 0x55fbb0036d10 2024-09-13T18:22:35.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.901+0000 7fc17bfff640 1 -- 172.21.15.114:0/1529503974 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc1880098a0 con 0x55fbb0036d10 2024-09-13T18:22:35.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.901+0000 7fc17bfff640 1 -- 172.21.15.114:0/1529503974 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc188010610 con 0x55fbb0036d10 2024-09-13T18:22:35.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.902+0000 7fc1965f0640 1 --2- 172.21.15.114:0/3612960243 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc17c0a1310 0x7fc17c0a37d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc188020000 tx=0x7fc18801d000 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:35.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.902+0000 7fc19ae52740 1 -- 172.21.15.114:0/1529503974 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb0036d10 msgr2=0x55fbb00370f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:35.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.902+0000 7fc19ae52740 1 --2- 172.21.15.114:0/1529503974 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb0036d10 0x55fbb00370f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fc188009f70 tx=0x7fc18800f7f0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.902+0000 7fc19ae52740 1 -- 172.21.15.114:0/1529503974 shutdown_connections 2024-09-13T18:22:35.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.902+0000 7fc19ae52740 1 --2- 172.21.15.114:0/1529503974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbb00463e0 0x55fbb00487d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.902+0000 7fc19ae52740 1 --2- 172.21.15.114:0/1529503974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb0037630 0x55fbb0045ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.902+0000 7fc19ae52740 1 --2- 172.21.15.114:0/1529503974 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb0036d10 0x55fbb00370f0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.902+0000 7fc19ae52740 1 -- 172.21.15.114:0/1529503974 >> 172.21.15.114:0/1529503974 conn(0x55fbb0033380 msgr2=0x55fbb0033760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:35.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.902+0000 7fc19ae52740 1 -- 172.21.15.114:0/1529503974 shutdown_connections 2024-09-13T18:22:35.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.902+0000 7fc19ae52740 1 -- 172.21.15.114:0/1529503974 wait complete. 2024-09-13T18:22:35.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.902+0000 7fc19ae52740 1 -- start start 2024-09-13T18:22:35.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.903+0000 7fc19ae52740 1 Processor -- start 2024-09-13T18:22:35.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.903+0000 7fc19ae52740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb0036d10 0x55fbb0045dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:35.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.903+0000 7fc19ae52740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb0037630 0x55fbb0042ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:35.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.903+0000 7fc19ae52740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbb00463e0 0x55fbb00435c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:35.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.903+0000 7fc19ae52740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbaff3c2a0 con 0x55fbb0036d10 2024-09-13T18:22:35.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.903+0000 7fc19ae52740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbafc73430 con 0x55fbb0037630 2024-09-13T18:22:35.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.903+0000 7fc19ae52740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbaff1e680 con 0x55fbb00463e0 2024-09-13T18:22:35.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.903+0000 7fc19ae52740 10 client.0 _ll_register_callbacks cb 0x55fbaff94610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:22:35.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.903+0000 7fc195def640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb0036d10 0x55fbb0045dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:35.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.903+0000 7fc195def640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb0036d10 0x55fbb0045dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50998/0 (socket says 172.21.15.114:50998) 2024-09-13T18:22:35.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.903+0000 7fc195def640 1 -- 172.21.15.114:0/516711601 learned_addr learned my addr 172.21.15.114:0/516711601 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:35.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.903+0000 7fc1955ee640 1 --2- 172.21.15.114:0/516711601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb0037630 0x55fbb0042ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:35.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.903+0000 7fc1965f0640 1 --2- 172.21.15.114:0/516711601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbb00463e0 0x55fbb00435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:35.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.903+0000 7fc195def640 1 -- 172.21.15.114:0/516711601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbb00463e0 msgr2=0x55fbb00435c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:35.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.903+0000 7fc195def640 1 --2- 172.21.15.114:0/516711601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbb00463e0 0x55fbb00435c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.903+0000 7fc195def640 1 -- 172.21.15.114:0/516711601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb0037630 msgr2=0x55fbb0042ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:35.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.903+0000 7fc195def640 1 --2- 172.21.15.114:0/516711601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb0037630 0x55fbb0042ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.903+0000 7fc195def640 1 -- 172.21.15.114:0/516711601 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc188009a70 con 0x55fbb0036d10 2024-09-13T18:22:35.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.904+0000 7fc195def640 1 --2- 172.21.15.114:0/516711601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb0036d10 0x55fbb0045dc0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fc1800b7780 tx=0x7fc1800c8e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b40600981c0cb9c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:35.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.904+0000 7fc1797fa640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:22:35.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.904+0000 7fc1797fa640 1 -- 172.21.15.114:0/516711601 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1800eb8b0 con 0x55fbb0036d10 2024-09-13T18:22:35.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.904+0000 7fc1797fa640 1 -- 172.21.15.114:0/516711601 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fc180005410 con 0x55fbb0036d10 2024-09-13T18:22:35.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.904+0000 7fc19ae52740 1 -- 172.21.15.114:0/516711601 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fbafc9f7e0 con 0x55fbb0036d10 2024-09-13T18:22:35.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.904+0000 7fc1797fa640 1 -- 172.21.15.114:0/516711601 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1800beb40 con 0x55fbb0036d10 2024-09-13T18:22:35.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.904+0000 7fc19ae52740 10 client.15774 fetch_fsmap learned FSMap version 30 2024-09-13T18:22:35.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.904+0000 7fc19ae52740 1 -- 172.21.15.114:0/516711601 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55fbaff99620 con 0x55fbb0036d10 2024-09-13T18:22:35.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.904+0000 7fc1797fa640 1 -- 172.21.15.114:0/516711601 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55fbafc9f7e0 con 0x55fbb0036d10 2024-09-13T18:22:35.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.904+0000 7fc1797fa640 1 -- 172.21.15.114:0/516711601 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc18004d070 con 0x55fbb0036d10 2024-09-13T18:22:35.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.904+0000 7fc19ae52740 10 client.15774 fetch_fsmap finished waiting for FSMap version 30 2024-09-13T18:22:35.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.904+0000 7fc19ae52740 10 client.15774 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:22:35.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.904+0000 7fc19ae52740 20 client.15774 populate_metadata read hostname 'smithi114' 2024-09-13T18:22:35.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.904+0000 7fc19ae52740 1 --2- 172.21.15.114:0/516711601 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55fbb0055f90 0x55fbb0058380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:35.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.904+0000 7fc19ae52740 4 client.15774 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:22:35.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.904+0000 7fc19ae52740 1 -- 172.21.15.114:0/516711601 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fbafcf2e30 con 0x55fbb0055f90 2024-09-13T18:22:35.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.905+0000 7fc1955ee640 1 --2- 172.21.15.114:0/516711601 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55fbb0055f90 0x55fbb0058380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:35.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.905+0000 7fc1797fa640 10 client.15774 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:22:35.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.905+0000 7fc1955ee640 1 --2- 172.21.15.114:0/516711601 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55fbb0055f90 0x55fbb0058380 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39258e81bcc8757f server_cookie=664890dcb31ce9c in_seq=0 out_seq=0 2024-09-13T18:22:35.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.905+0000 7fc1797fa640 1 -- 172.21.15.114:0/516711601 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55fbafcf2e30 con 0x55fbb0055f90 2024-09-13T18:22:35.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.906+0000 7fc1797fa640 10 client.15774 handle_command_reply: tid=0 2024-09-13T18:22:35.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.906+0000 7fc19ae52740 1 client.15774 shutdown 2024-09-13T18:22:35.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.906+0000 7fc19ae52740 1 -- 172.21.15.114:0/516711601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb0036d10 msgr2=0x55fbb0045dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:35.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.906+0000 7fc19ae52740 1 --2- 172.21.15.114:0/516711601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb0036d10 0x55fbb0045dc0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fc1800b7780 tx=0x7fc1800c8e50 comp rx=0 tx=0).stop 2024-09-13T18:22:35.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.906+0000 7fc19ae52740 1 -- 172.21.15.114:0/516711601 shutdown_connections 2024-09-13T18:22:35.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.906+0000 7fc19ae52740 1 --2- 172.21.15.114:0/516711601 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55fbb0055f90 0x55fbb0058380 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.906+0000 7fc19ae52740 1 --2- 172.21.15.114:0/516711601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbb00463e0 0x55fbb00435c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.906+0000 7fc19ae52740 1 --2- 172.21.15.114:0/516711601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb0037630 0x55fbb0042ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.906+0000 7fc19ae52740 1 --2- 172.21.15.114:0/516711601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb0036d10 0x55fbb0045dc0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.906+0000 7fc1797fa640 0 client.15774 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:22:35.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.906+0000 7fc19ae52740 1 -- 172.21.15.114:0/516711601 >> 172.21.15.114:0/516711601 conn(0x55fbb0033380 msgr2=0x55fbb00390e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:35.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.906+0000 7fc19ae52740 1 -- 172.21.15.114:0/516711601 shutdown_connections 2024-09-13T18:22:35.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.906+0000 7fc19ae52740 1 -- 172.21.15.114:0/516711601 wait complete. 2024-09-13T18:22:35.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.906+0000 7fc19ae52740 20 client.15774 trim_cache size 0 max 16384 2024-09-13T18:22:35.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.920+0000 7fc19ae52740 1 Processor -- start 2024-09-13T18:22:35.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.920+0000 7fc19ae52740 1 -- start start 2024-09-13T18:22:35.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.920+0000 7fc19ae52740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb0036d10 0x55fbb00458d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:35.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.920+0000 7fc19ae52740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbb0037630 0x55fbb0033760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:35.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.920+0000 7fc19ae52740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb00463e0 0x55fbb0033ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:35.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.920+0000 7fc19ae52740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbb00380e0 con 0x55fbb0036d10 2024-09-13T18:22:35.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.920+0000 7fc19ae52740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbb00520f0 con 0x55fbb00463e0 2024-09-13T18:22:35.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.920+0000 7fc19ae52740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbb003de40 con 0x55fbb0037630 2024-09-13T18:22:35.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.920+0000 7fc195def640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb0036d10 0x55fbb00458d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:35.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.920+0000 7fc195def640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb0036d10 0x55fbb00458d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51014/0 (socket says 172.21.15.114:51014) 2024-09-13T18:22:35.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.920+0000 7fc195def640 1 -- 172.21.15.114:0/476493392 learned_addr learned my addr 172.21.15.114:0/476493392 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:35.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.920+0000 7fc1965f0640 1 --2- 172.21.15.114:0/476493392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb00463e0 0x55fbb0033ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:35.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.920+0000 7fc195def640 1 -- 172.21.15.114:0/476493392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbb0037630 msgr2=0x55fbb0033760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:35.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.920+0000 7fc195def640 1 --2- 172.21.15.114:0/476493392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbb0037630 0x55fbb0033760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.920+0000 7fc195def640 1 -- 172.21.15.114:0/476493392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb00463e0 msgr2=0x55fbb0033ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:35.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.920+0000 7fc195def640 1 --2- 172.21.15.114:0/476493392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb00463e0 0x55fbb0033ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.920+0000 7fc195def640 1 -- 172.21.15.114:0/476493392 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fbaff3b790 con 0x55fbb0036d10 2024-09-13T18:22:35.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.920+0000 7fc1965f0640 1 --2- 172.21.15.114:0/476493392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb00463e0 0x55fbb0033ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:22:35.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.921+0000 7fc195def640 1 --2- 172.21.15.114:0/476493392 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb0036d10 0x55fbb00458d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fc1800ebac0 tx=0x7fc1800c8ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc4845f868b531a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:35.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.921+0000 7fc184ff9640 1 -- 172.21.15.114:0/476493392 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1800eb8b0 con 0x55fbb0036d10 2024-09-13T18:22:35.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.921+0000 7fc184ff9640 1 -- 172.21.15.114:0/476493392 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc1800eb340 con 0x55fbb0036d10 2024-09-13T18:22:35.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.921+0000 7fc184ff9640 1 -- 172.21.15.114:0/476493392 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1800c2f60 con 0x55fbb0036d10 2024-09-13T18:22:35.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.921+0000 7fc19ae52740 1 -- 172.21.15.114:0/476493392 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb0036d10 msgr2=0x55fbb00458d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:35.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.921+0000 7fc19ae52740 1 --2- 172.21.15.114:0/476493392 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb0036d10 0x55fbb00458d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fc1800ebac0 tx=0x7fc1800c8ec0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.921+0000 7fc19ae52740 1 -- 172.21.15.114:0/476493392 shutdown_connections 2024-09-13T18:22:35.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.921+0000 7fc19ae52740 1 --2- 172.21.15.114:0/476493392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb00463e0 0x55fbb0033ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.921+0000 7fc19ae52740 1 --2- 172.21.15.114:0/476493392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbb0037630 0x55fbb0033760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.921+0000 7fc19ae52740 1 --2- 172.21.15.114:0/476493392 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb0036d10 0x55fbb00458d0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.921+0000 7fc19ae52740 1 -- 172.21.15.114:0/476493392 >> 172.21.15.114:0/476493392 conn(0x55fbb0033380 msgr2=0x55fbb003e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:35.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.921+0000 7fc19ae52740 1 -- 172.21.15.114:0/476493392 shutdown_connections 2024-09-13T18:22:35.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.921+0000 7fc19ae52740 1 -- 172.21.15.114:0/476493392 wait complete. 2024-09-13T18:22:35.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.922+0000 7fc19ae52740 1 -- start start 2024-09-13T18:22:35.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.922+0000 7fc19ae52740 1 Processor -- start 2024-09-13T18:22:35.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.922+0000 7fc19ae52740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb0036d10 0x55fbb00a4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:35.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.922+0000 7fc19ae52740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb0037630 0x55fbb00a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:35.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.922+0000 7fc19ae52740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbb00463e0 0x55fbb004cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:35.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.922+0000 7fc19ae52740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbb004d180 con 0x55fbb0037630 2024-09-13T18:22:35.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.922+0000 7fc19ae52740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbb004d2f0 con 0x55fbb0036d10 2024-09-13T18:22:35.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.922+0000 7fc19ae52740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbb004d4b0 con 0x55fbb00463e0 2024-09-13T18:22:35.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.922+0000 7fc19ae52740 10 client.0 _ll_register_callbacks cb 0x7fc1800eb290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:22:35.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.922+0000 7fc1955ee640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb0037630 0x55fbb00a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:35.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.922+0000 7fc1955ee640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb0037630 0x55fbb00a4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51020/0 (socket says 172.21.15.114:51020) 2024-09-13T18:22:35.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.922+0000 7fc1955ee640 1 -- 172.21.15.114:0/3022561247 learned_addr learned my addr 172.21.15.114:0/3022561247 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:35.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.922+0000 7fc1965f0640 1 --2- 172.21.15.114:0/3022561247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbb00463e0 0x55fbb004cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:35.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.922+0000 7fc195def640 1 --2- 172.21.15.114:0/3022561247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb0036d10 0x55fbb00a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:35.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.922+0000 7fc1955ee640 1 -- 172.21.15.114:0/3022561247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbb00463e0 msgr2=0x55fbb004cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:35.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.922+0000 7fc1955ee640 1 --2- 172.21.15.114:0/3022561247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbb00463e0 0x55fbb004cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.922+0000 7fc1955ee640 1 -- 172.21.15.114:0/3022561247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb0036d10 msgr2=0x55fbb00a4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:35.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.922+0000 7fc1955ee640 1 --2- 172.21.15.114:0/3022561247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb0036d10 0x55fbb00a4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.923+0000 7fc1955ee640 1 -- 172.21.15.114:0/3022561247 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fbaff99620 con 0x55fbb0037630 2024-09-13T18:22:35.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.923+0000 7fc1955ee640 1 --2- 172.21.15.114:0/3022561247 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb0037630 0x55fbb00a4b40 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x55fbb0038890 tx=0x7fc18c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58482af1f2d7a11b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:35.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.923+0000 7fc1797fa640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:22:35.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.923+0000 7fc1797fa640 1 -- 172.21.15.114:0/3022561247 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c017040 con 0x55fbb0037630 2024-09-13T18:22:35.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.923+0000 7fc1797fa640 1 -- 172.21.15.114:0/3022561247 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fc18c01c870 con 0x55fbb0037630 2024-09-13T18:22:35.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.923+0000 7fc19ae52740 1 -- 172.21.15.114:0/3022561247 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fbb00ad190 con 0x55fbb0037630 2024-09-13T18:22:35.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.923+0000 7fc1797fa640 1 -- 172.21.15.114:0/3022561247 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc18c005020 con 0x55fbb0037630 2024-09-13T18:22:35.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.923+0000 7fc19ae52740 10 client.15780 fetch_fsmap learned FSMap version 30 2024-09-13T18:22:35.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.923+0000 7fc19ae52740 1 -- 172.21.15.114:0/3022561247 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55fbaff60ea0 con 0x55fbb0037630 2024-09-13T18:22:35.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.923+0000 7fc1797fa640 1 -- 172.21.15.114:0/3022561247 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55fbb00ad190 con 0x55fbb0037630 2024-09-13T18:22:35.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.923+0000 7fc1797fa640 1 -- 172.21.15.114:0/3022561247 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc18c00f420 con 0x55fbb0037630 2024-09-13T18:22:35.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.923+0000 7fc19ae52740 10 client.15780 fetch_fsmap finished waiting for FSMap version 30 2024-09-13T18:22:35.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.923+0000 7fc19ae52740 10 client.15780 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:22:35.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.923+0000 7fc19ae52740 20 client.15780 populate_metadata read hostname 'smithi114' 2024-09-13T18:22:35.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.923+0000 7fc19ae52740 1 --2- 172.21.15.114:0/3022561247 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55fbb00ad840 0x55fbb00afc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:35.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.923+0000 7fc19ae52740 4 client.15780 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T18:22:35.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.923+0000 7fc19ae52740 1 -- 172.21.15.114:0/3022561247 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55fbb00b0420 con 0x55fbb00ad840 2024-09-13T18:22:35.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.924+0000 7fc195def640 1 --2- 172.21.15.114:0/3022561247 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55fbb00ad840 0x55fbb00afc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:35.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.924+0000 7fc195def640 1 --2- 172.21.15.114:0/3022561247 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55fbb00ad840 0x55fbb00afc60 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e0cdb19e3d6560d server_cookie=541c7f8f6584aadb in_seq=0 out_seq=0 2024-09-13T18:22:35.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.924+0000 7fc1797fa640 10 client.15780 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:22:35.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.924+0000 7fc1797fa640 1 -- 172.21.15.114:0/3022561247 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55fbb00b0420 con 0x55fbb00ad840 2024-09-13T18:22:35.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.924+0000 7fc1797fa640 10 client.15780 handle_command_reply: tid=0 2024-09-13T18:22:35.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.924+0000 7fc19ae52740 1 client.15780 shutdown 2024-09-13T18:22:35.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.924+0000 7fc19ae52740 1 -- 172.21.15.114:0/3022561247 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb0037630 msgr2=0x55fbb00a4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:35.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.924+0000 7fc19ae52740 1 --2- 172.21.15.114:0/3022561247 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb0037630 0x55fbb00a4b40 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x55fbb0038890 tx=0x7fc18c011b00 comp rx=0 tx=0).stop 2024-09-13T18:22:35.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.924+0000 7fc19ae52740 1 -- 172.21.15.114:0/3022561247 shutdown_connections 2024-09-13T18:22:35.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.924+0000 7fc19ae52740 1 --2- 172.21.15.114:0/3022561247 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55fbb00ad840 0x55fbb00afc60 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.924+0000 7fc19ae52740 1 --2- 172.21.15.114:0/3022561247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbb00463e0 0x55fbb004cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.924+0000 7fc19ae52740 1 --2- 172.21.15.114:0/3022561247 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb0037630 0x55fbb00a4b40 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.924+0000 7fc1797fa640 0 client.15780 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:22:35.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.924+0000 7fc19ae52740 1 --2- 172.21.15.114:0/3022561247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb0036d10 0x55fbb00a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.924+0000 7fc19ae52740 1 -- 172.21.15.114:0/3022561247 >> 172.21.15.114:0/3022561247 conn(0x55fbb0033380 msgr2=0x55fbb003e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:35.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.924+0000 7fc19ae52740 1 -- 172.21.15.114:0/3022561247 shutdown_connections 2024-09-13T18:22:35.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.924+0000 7fc19ae52740 1 -- 172.21.15.114:0/3022561247 wait complete. 2024-09-13T18:22:35.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.925+0000 7fc19ae52740 20 client.15780 trim_cache size 0 max 16384 2024-09-13T18:22:35.936 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:22:35.936 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-13T18:22:35.936 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": { 2024-09-13T18:22:35.936 INFO:teuthology.orchestra.run.smithi114.stdout: "f7ef14df-4102-4786-8e08-4da08feaa73e": { 2024-09-13T18:22:35.936 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-13T18:22:35.936 INFO:teuthology.orchestra.run.smithi114.stdout: "tag": "f7ef14df-4102-4786-8e08-4da08feaa73e", 2024-09-13T18:22:35.936 INFO:teuthology.orchestra.run.smithi114.stdout: "options": "recursive,force" 2024-09-13T18:22:35.936 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:22:35.936 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:22:35.936 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:22:35.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.925+0000 7fc198053640 1 -- 172.21.15.114:0/3612960243 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc17c0a1310 msgr2=0x7fc17c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:35.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.925+0000 7fc198053640 1 --2- 172.21.15.114:0/3612960243 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc17c0a1310 0x7fc17c0a37d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc188020000 tx=0x7fc18801d000 comp rx=0 tx=0).stop 2024-09-13T18:22:35.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.925+0000 7fc198053640 1 -- 172.21.15.114:0/3612960243 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc190100f40 msgr2=0x7fc1901961e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:35.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.925+0000 7fc198053640 1 --2- 172.21.15.114:0/3612960243 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc190100f40 0x7fc1901961e0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fc1800057c0 tx=0x7fc180032be0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.925+0000 7fc198053640 1 -- 172.21.15.114:0/3612960243 shutdown_connections 2024-09-13T18:22:35.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.925+0000 7fc198053640 1 --2- 172.21.15.114:0/3612960243 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc17c0a1310 0x7fc17c0a37d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.925+0000 7fc198053640 1 --2- 172.21.15.114:0/3612960243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc19010a310 0x7fc19019aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.925+0000 7fc198053640 1 --2- 172.21.15.114:0/3612960243 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc190101860 0x7fc190196720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.925+0000 7fc198053640 1 --2- 172.21.15.114:0/3612960243 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc190100f40 0x7fc1901961e0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.925+0000 7fc198053640 1 -- 172.21.15.114:0/3612960243 >> 172.21.15.114:0/3612960243 conn(0x7fc190075c60 msgr2=0x7fc1900fcfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:35.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.925+0000 7fc198053640 1 -- 172.21.15.114:0/3612960243 shutdown_connections 2024-09-13T18:22:35.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.925+0000 7fc198053640 1 -- 172.21.15.114:0/3612960243 wait complete. 2024-09-13T18:22:35.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.930+0000 7f6ad91d5640 1 -- 172.21.15.114:0/3871354009 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) v1 -- 0x7f6ad4197480 con 0x7f6ad40ff970 2024-09-13T18:22:35.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.935+0000 7f6ad0d99640 1 -- 172.21.15.114:0/3871354009 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v29)=0 v29) v1 ==== 129+0+0 (secure 0 0 0) 0x7f6abc01d020 con 0x7f6ad40ff970 2024-09-13T18:22:35.939 INFO:tasks.cephfs.filesystem:scrub status for tag:f7ef14df-4102-4786-8e08-4da08feaa73e - {'path': '/', 'tag': 'f7ef14df-4102-4786-8e08-4da08feaa73e', 'options': 'recursive,force'} 2024-09-13T18:22:35.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.937+0000 7f6ad91d5640 1 -- 172.21.15.114:0/3871354009 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f6ac00a1310 msgr2=0x7f6ac00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:35.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.937+0000 7f6ad91d5640 1 --2- 172.21.15.114:0/3871354009 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f6ac00a1310 0x7f6ac00a37d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4009900 tx=0x7f6ac4005a20 comp rx=0 tx=0).stop 2024-09-13T18:22:35.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.937+0000 7f6ad91d5640 1 -- 172.21.15.114:0/3871354009 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6ad40ff970 msgr2=0x7f6ad4196670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:35.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.937+0000 7f6ad91d5640 1 --2- 172.21.15.114:0/3871354009 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6ad40ff970 0x7f6ad4196670 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f6abc00c970 tx=0x7f6abc00ce30 comp rx=0 tx=0).stop 2024-09-13T18:22:35.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.937+0000 7f6ad91d5640 1 -- 172.21.15.114:0/3871354009 shutdown_connections 2024-09-13T18:22:35.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.937+0000 7f6ad91d5640 1 --2- 172.21.15.114:0/3871354009 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f6ac00a1310 0x7f6ac00a37d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.937+0000 7f6ad91d5640 1 --2- 172.21.15.114:0/3871354009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ad4104c30 0x7f6ad419aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.937+0000 7f6ad91d5640 1 --2- 172.21.15.114:0/3871354009 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6ad40ff970 0x7f6ad4196670 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.937+0000 7f6ad91d5640 1 --2- 172.21.15.114:0/3871354009 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ad40feff0 0x7f6ad4196130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:35.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.937+0000 7f6ad91d5640 1 -- 172.21.15.114:0/3871354009 >> 172.21.15.114:0/3871354009 conn(0x7f6ad4074940 msgr2=0x7f6ad4067a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:35.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.937+0000 7f6ad91d5640 1 -- 172.21.15.114:0/3871354009 shutdown_connections 2024-09-13T18:22:35.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:35.937+0000 7f6ad91d5640 1 -- 172.21.15.114:0/3871354009 wait complete. 2024-09-13T18:22:35.952 DEBUG:teuthology.orchestra.run.smithi114:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-09-13T18:22:36.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.089+0000 7f751171f640 1 -- 172.21.15.114:0/559981172 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f750c0fede0 msgr2=0x7f750c0ff1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:36.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.089+0000 7f751171f640 1 --2- 172.21.15.114:0/559981172 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f750c0fede0 0x7f750c0ff1c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f74fc009930 tx=0x7f74fc02a420 comp rx=0 tx=0).stop 2024-09-13T18:22:36.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.089+0000 7f751171f640 1 -- 172.21.15.114:0/559981172 shutdown_connections 2024-09-13T18:22:36.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.089+0000 7f751171f640 1 --2- 172.21.15.114:0/559981172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f750c103f30 0x7f750c10ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:36.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.089+0000 7f751171f640 1 --2- 172.21.15.114:0/559981172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f750c0ff790 0x7f750c1037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:36.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.089+0000 7f751171f640 1 --2- 172.21.15.114:0/559981172 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f750c0fede0 0x7f750c0ff1c0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:36.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.089+0000 7f751171f640 1 -- 172.21.15.114:0/559981172 >> 172.21.15.114:0/559981172 conn(0x7f750c0fa870 msgr2=0x7f750c0fcc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:36.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.089+0000 7f751171f640 1 -- 172.21.15.114:0/559981172 shutdown_connections 2024-09-13T18:22:36.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.089+0000 7f751171f640 1 -- 172.21.15.114:0/559981172 wait complete. 2024-09-13T18:22:36.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.090+0000 7f751171f640 1 Processor -- start 2024-09-13T18:22:36.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.090+0000 7f751171f640 1 -- start start 2024-09-13T18:22:36.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.090+0000 7f751171f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f750c0fede0 0x7f750c196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:36.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.090+0000 7f751171f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f750c103f30 0x7f750c1974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:36.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.090+0000 7f751171f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f750c197a30 0x7f750c1a2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:36.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.090+0000 7f751171f640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f750c1a26d0 con 0x7f750c0fede0 2024-09-13T18:22:36.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.090+0000 7f751171f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f750c1a2840 con 0x7f750c103f30 2024-09-13T18:22:36.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.090+0000 7f751171f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f750c1a2a00 con 0x7f750c197a30 2024-09-13T18:22:36.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.090+0000 7f750affd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f750c0fede0 0x7f750c196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:36.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.090+0000 7f750affd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f750c0fede0 0x7f750c196fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51040/0 (socket says 172.21.15.114:51040) 2024-09-13T18:22:36.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.090+0000 7f750affd640 1 -- 172.21.15.114:0/818194881 learned_addr learned my addr 172.21.15.114:0/818194881 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:36.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.090+0000 7f7503fff640 1 --2- 172.21.15.114:0/818194881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f750c103f30 0x7f750c1974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:36.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.090+0000 7f750b7fe640 1 --2- 172.21.15.114:0/818194881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f750c197a30 0x7f750c1a2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:36.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.090+0000 7f750affd640 1 -- 172.21.15.114:0/818194881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f750c197a30 msgr2=0x7f750c1a2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:36.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.090+0000 7f750affd640 1 --2- 172.21.15.114:0/818194881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f750c197a30 0x7f750c1a2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:36.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.090+0000 7f750affd640 1 -- 172.21.15.114:0/818194881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f750c103f30 msgr2=0x7f750c1974f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:36.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.090+0000 7f750affd640 1 --2- 172.21.15.114:0/818194881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f750c103f30 0x7f750c1974f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:36.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.090+0000 7f750affd640 1 -- 172.21.15.114:0/818194881 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74fc009560 con 0x7f750c0fede0 2024-09-13T18:22:36.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.090+0000 7f750b7fe640 1 --2- 172.21.15.114:0/818194881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f750c197a30 0x7f750c1a2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:22:36.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.090+0000 7f750affd640 1 --2- 172.21.15.114:0/818194881 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f750c0fede0 0x7f750c196fb0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f74fc02aa10 tx=0x7f74fc032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:36.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.091+0000 7f7508ff9640 1 -- 172.21.15.114:0/818194881 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74fc005210 con 0x7f750c0fede0 2024-09-13T18:22:36.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.091+0000 7f7508ff9640 1 -- 172.21.15.114:0/818194881 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f74fc005370 con 0x7f750c0fede0 2024-09-13T18:22:36.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.091+0000 7f751171f640 1 -- 172.21.15.114:0/818194881 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f750c1a2c80 con 0x7f750c0fede0 2024-09-13T18:22:36.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.091+0000 7f7508ff9640 1 -- 172.21.15.114:0/818194881 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74fc0038e0 con 0x7f750c0fede0 2024-09-13T18:22:36.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.091+0000 7f751171f640 1 -- 172.21.15.114:0/818194881 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f750c1a3130 con 0x7f750c0fede0 2024-09-13T18:22:36.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.092+0000 7f751171f640 1 -- 172.21.15.114:0/818194881 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f750c1002b0 con 0x7f750c0fede0 2024-09-13T18:22:36.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.093+0000 7f7508ff9640 1 -- 172.21.15.114:0/818194881 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 24) v1 ==== 136050+0+0 (secure 0 0 0) 0x7f74fc003a40 con 0x7f750c0fede0 2024-09-13T18:22:36.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.094+0000 7f7508ff9640 1 --2- 172.21.15.114:0/818194881 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f74f80a1290 0x7f74f80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:36.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.094+0000 7f7503fff640 1 --2- 172.21.15.114:0/818194881 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f74f80a1290 0x7f74f80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:36.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.094+0000 7f7508ff9640 1 -- 172.21.15.114:0/818194881 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f74fc0ee2a0 con 0x7f750c0fede0 2024-09-13T18:22:36.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.094+0000 7f7503fff640 1 --2- 172.21.15.114:0/818194881 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f74f80a1290 0x7f74f80a3750 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f750c068df0 tx=0x7f74f400a250 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:36.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.096+0000 7f7508ff9640 1 -- 172.21.15.114:0/818194881 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f74fc0b7230 con 0x7f750c0fede0 2024-09-13T18:22:36.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:36 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2481894019' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:22:36.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:36 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/1216293605' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:22:36.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:36 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/3871354009' entity='client.admin' 2024-09-13T18:22:36.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:36 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:22:36.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:36 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:22:36.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:36 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:22:36.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:36 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:36.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.264+0000 7f751171f640 1 -- 172.21.15.114:0/818194881 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f750c068ba0 con 0x7f750c0fede0 2024-09-13T18:22:36.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.270+0000 7f7508ff9640 1 -- 172.21.15.114:0/818194881 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v30)=0 v30) v1 ==== 138+0+0 (secure 0 0 0) 0x7f74fc04a0d0 con 0x7f750c0fede0 2024-09-13T18:22:36.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.272+0000 7f751171f640 1 -- 172.21.15.114:0/818194881 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f74f80a1290 msgr2=0x7f74f80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:36.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.272+0000 7f751171f640 1 --2- 172.21.15.114:0/818194881 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f74f80a1290 0x7f74f80a3750 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f750c068df0 tx=0x7f74f400a250 comp rx=0 tx=0).stop 2024-09-13T18:22:36.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.272+0000 7f751171f640 1 -- 172.21.15.114:0/818194881 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f750c0fede0 msgr2=0x7f750c196fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:36.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.272+0000 7f751171f640 1 --2- 172.21.15.114:0/818194881 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f750c0fede0 0x7f750c196fb0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f74fc02aa10 tx=0x7f74fc032c20 comp rx=0 tx=0).stop 2024-09-13T18:22:36.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.272+0000 7f751171f640 1 -- 172.21.15.114:0/818194881 shutdown_connections 2024-09-13T18:22:36.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.272+0000 7f751171f640 1 --2- 172.21.15.114:0/818194881 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f74f80a1290 0x7f74f80a3750 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:36.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.272+0000 7f751171f640 1 --2- 172.21.15.114:0/818194881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f750c197a30 0x7f750c1a2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:36.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.272+0000 7f751171f640 1 --2- 172.21.15.114:0/818194881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f750c103f30 0x7f750c1974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:36.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.272+0000 7f751171f640 1 --2- 172.21.15.114:0/818194881 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f750c0fede0 0x7f750c196fb0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:36.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.273+0000 7f751171f640 1 -- 172.21.15.114:0/818194881 >> 172.21.15.114:0/818194881 conn(0x7f750c0fa870 msgr2=0x7f750c0fb800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:36.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.273+0000 7f751171f640 1 -- 172.21.15.114:0/818194881 shutdown_connections 2024-09-13T18:22:36.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.273+0000 7f751171f640 1 -- 172.21.15.114:0/818194881 wait complete. 2024-09-13T18:22:36.288 DEBUG:teuthology.orchestra.run.smithi114:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-09-13T18:22:36.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:36 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/2481894019' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:22:36.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:36 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/1216293605' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:22:36.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:36 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/3871354009' entity='client.admin' 2024-09-13T18:22:36.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:36 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:22:36.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:36 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:22:36.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:36 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:22:36.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:36 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:36.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.387+0000 7f28a328f640 1 -- 172.21.15.114:0/403315515 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f289c0ffef0 msgr2=0x7f289c1002d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:36.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.387+0000 7f28a328f640 1 --2- 172.21.15.114:0/403315515 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f289c0ffef0 0x7f289c1002d0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f2894009930 tx=0x7f289402a420 comp rx=0 tx=0).stop 2024-09-13T18:22:36.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.387+0000 7f28a328f640 1 -- 172.21.15.114:0/403315515 shutdown_connections 2024-09-13T18:22:36.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.387+0000 7f28a328f640 1 --2- 172.21.15.114:0/403315515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f289c105040 0x7f289c10cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:36.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.387+0000 7f28a328f640 1 --2- 172.21.15.114:0/403315515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f289c1008a0 0x7f289c1048e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:36.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.387+0000 7f28a328f640 1 --2- 172.21.15.114:0/403315515 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f289c0ffef0 0x7f289c1002d0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:36.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.387+0000 7f28a328f640 1 -- 172.21.15.114:0/403315515 >> 172.21.15.114:0/403315515 conn(0x7f289c0fb960 msgr2=0x7f289c0fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:36.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.387+0000 7f28a328f640 1 -- 172.21.15.114:0/403315515 shutdown_connections 2024-09-13T18:22:36.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.387+0000 7f28a328f640 1 -- 172.21.15.114:0/403315515 wait complete. 2024-09-13T18:22:36.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.388+0000 7f28a328f640 1 Processor -- start 2024-09-13T18:22:36.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.388+0000 7f28a328f640 1 -- start start 2024-09-13T18:22:36.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.388+0000 7f28a328f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f289c0ffef0 0x7f289c1961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:36.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.388+0000 7f28a328f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f289c105040 0x7f289c196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:36.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.388+0000 7f28a328f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f289c19ab10 0x7f289c19afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:36.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.388+0000 7f28a328f640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289c19b500 con 0x7f289c0ffef0 2024-09-13T18:22:36.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.388+0000 7f28a328f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289c19b670 con 0x7f289c19ab10 2024-09-13T18:22:36.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.388+0000 7f28a328f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289c19b7e0 con 0x7f289c105040 2024-09-13T18:22:36.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.388+0000 7f28a102b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f289c0ffef0 0x7f289c1961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:36.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.388+0000 7f28a102b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f289c0ffef0 0x7f289c1961e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51046/0 (socket says 172.21.15.114:51046) 2024-09-13T18:22:36.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.388+0000 7f28a102b640 1 -- 172.21.15.114:0/2366049253 learned_addr learned my addr 172.21.15.114:0/2366049253 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:36.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.388+0000 7f28a082a640 1 --2- 172.21.15.114:0/2366049253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f289c105040 0x7f289c196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:36.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.388+0000 7f28a182c640 1 --2- 172.21.15.114:0/2366049253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f289c19ab10 0x7f289c19afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:36.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.388+0000 7f28a102b640 1 -- 172.21.15.114:0/2366049253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f289c105040 msgr2=0x7f289c196720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:36.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.388+0000 7f28a102b640 1 --2- 172.21.15.114:0/2366049253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f289c105040 0x7f289c196720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:36.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.388+0000 7f28a102b640 1 -- 172.21.15.114:0/2366049253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f289c19ab10 msgr2=0x7f289c19afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:36.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.388+0000 7f28a102b640 1 --2- 172.21.15.114:0/2366049253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f289c19ab10 0x7f289c19afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:36.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.388+0000 7f28a102b640 1 -- 172.21.15.114:0/2366049253 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2894009560 con 0x7f289c0ffef0 2024-09-13T18:22:36.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.388+0000 7f28a082a640 1 --2- 172.21.15.114:0/2366049253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f289c105040 0x7f289c196720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:22:36.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.388+0000 7f28a182c640 1 --2- 172.21.15.114:0/2366049253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f289c19ab10 0x7f289c19afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:22:36.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.388+0000 7f28a102b640 1 --2- 172.21.15.114:0/2366049253 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f289c0ffef0 0x7f289c1961e0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f2894005ab0 tx=0x7f2894032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:36.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.388+0000 7f289a7fc640 1 -- 172.21.15.114:0/2366049253 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2894032e10 con 0x7f289c0ffef0 2024-09-13T18:22:36.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.389+0000 7f289a7fc640 1 -- 172.21.15.114:0/2366049253 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f28940046c0 con 0x7f289c0ffef0 2024-09-13T18:22:36.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.389+0000 7f28a328f640 1 -- 172.21.15.114:0/2366049253 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f289c1a2dc0 con 0x7f289c0ffef0 2024-09-13T18:22:36.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.389+0000 7f289a7fc640 1 -- 172.21.15.114:0/2366049253 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2894003850 con 0x7f289c0ffef0 2024-09-13T18:22:36.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.389+0000 7f28a328f640 1 -- 172.21.15.114:0/2366049253 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f289c1a31e0 con 0x7f289c0ffef0 2024-09-13T18:22:36.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.389+0000 7f28a328f640 1 -- 172.21.15.114:0/2366049253 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f289c1013c0 con 0x7f289c0ffef0 2024-09-13T18:22:36.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.391+0000 7f289a7fc640 1 -- 172.21.15.114:0/2366049253 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 24) v1 ==== 136050+0+0 (secure 0 0 0) 0x7f2894004830 con 0x7f289c0ffef0 2024-09-13T18:22:36.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.391+0000 7f289a7fc640 1 --2- 172.21.15.114:0/2366049253 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f28840a1310 0x7f28840a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:36.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.391+0000 7f28a082a640 1 --2- 172.21.15.114:0/2366049253 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f28840a1310 0x7f28840a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:36.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.392+0000 7f289a7fc640 1 -- 172.21.15.114:0/2366049253 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f28940ee200 con 0x7f289c0ffef0 2024-09-13T18:22:36.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.392+0000 7f28a082a640 1 --2- 172.21.15.114:0/2366049253 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f28840a1310 0x7f28840a37d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f289c197780 tx=0x7f289000b5f0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:36.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.394+0000 7f289a7fc640 1 -- 172.21.15.114:0/2366049253 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f28940b7190 con 0x7f289c0ffef0 2024-09-13T18:22:36.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:36 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/2481894019' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:22:36.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:36 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/1216293605' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:22:36.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:36 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/3871354009' entity='client.admin' 2024-09-13T18:22:36.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:36 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:22:36.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:36 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:22:36.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:36 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:22:36.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:36 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:36.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.558+0000 7f28a328f640 1 -- 172.21.15.114:0/2366049253 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f289c1002d0 con 0x7f289c0ffef0 2024-09-13T18:22:36.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.562+0000 7f289a7fc640 1 -- 172.21.15.114:0/2366049253 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v31)=0 v31) v1 ==== 133+0+0 (secure 0 0 0) 0x7f28940400c0 con 0x7f289c0ffef0 2024-09-13T18:22:36.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.565+0000 7f28a328f640 1 -- 172.21.15.114:0/2366049253 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f28840a1310 msgr2=0x7f28840a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:36.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.565+0000 7f28a328f640 1 --2- 172.21.15.114:0/2366049253 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f28840a1310 0x7f28840a37d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f289c197780 tx=0x7f289000b5f0 comp rx=0 tx=0).stop 2024-09-13T18:22:36.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.565+0000 7f28a328f640 1 -- 172.21.15.114:0/2366049253 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f289c0ffef0 msgr2=0x7f289c1961e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:36.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.565+0000 7f28a328f640 1 --2- 172.21.15.114:0/2366049253 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f289c0ffef0 0x7f289c1961e0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f2894005ab0 tx=0x7f2894032c20 comp rx=0 tx=0).stop 2024-09-13T18:22:36.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.565+0000 7f28a328f640 1 -- 172.21.15.114:0/2366049253 shutdown_connections 2024-09-13T18:22:36.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.565+0000 7f28a328f640 1 --2- 172.21.15.114:0/2366049253 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f28840a1310 0x7f28840a37d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:36.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.565+0000 7f28a328f640 1 --2- 172.21.15.114:0/2366049253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f289c19ab10 0x7f289c19afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:36.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.565+0000 7f28a328f640 1 --2- 172.21.15.114:0/2366049253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f289c105040 0x7f289c196720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:36.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.565+0000 7f28a328f640 1 --2- 172.21.15.114:0/2366049253 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f289c0ffef0 0x7f289c1961e0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:36.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.565+0000 7f28a328f640 1 -- 172.21.15.114:0/2366049253 >> 172.21.15.114:0/2366049253 conn(0x7f289c0fb960 msgr2=0x7f289c0fce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:36.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.565+0000 7f28a328f640 1 -- 172.21.15.114:0/2366049253 shutdown_connections 2024-09-13T18:22:36.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.565+0000 7f28a328f640 1 -- 172.21.15.114:0/2366049253 wait complete. 2024-09-13T18:22:36.579 DEBUG:teuthology.orchestra.run.smithi114:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-09-13T18:22:36.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.678+0000 7f1e6f90f640 1 -- 172.21.15.114:0/1461873677 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e68067830 msgr2=0x7f1e68067c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:36.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.678+0000 7f1e6f90f640 1 --2- 172.21.15.114:0/1461873677 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e68067830 0x7f1e68067c90 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f1e58009900 tx=0x7f1e5802a410 comp rx=0 tx=0).stop 2024-09-13T18:22:36.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.679+0000 7f1e6f90f640 1 -- 172.21.15.114:0/1461873677 shutdown_connections 2024-09-13T18:22:36.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.679+0000 7f1e6f90f640 1 --2- 172.21.15.114:0/1461873677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e680681d0 0x7f1e6810cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:36.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.679+0000 7f1e6f90f640 1 --2- 172.21.15.114:0/1461873677 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e68067830 0x7f1e68067c90 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:36.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.679+0000 7f1e6f90f640 1 --2- 172.21.15.114:0/1461873677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e68104ce0 0x7f1e681050c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:36.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.679+0000 7f1e6f90f640 1 -- 172.21.15.114:0/1461873677 >> 172.21.15.114:0/1461873677 conn(0x7f1e68074940 msgr2=0x7f1e68074d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:36.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.679+0000 7f1e6f90f640 1 -- 172.21.15.114:0/1461873677 shutdown_connections 2024-09-13T18:22:36.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.679+0000 7f1e6f90f640 1 -- 172.21.15.114:0/1461873677 wait complete. 2024-09-13T18:22:36.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.679+0000 7f1e6f90f640 1 Processor -- start 2024-09-13T18:22:36.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.679+0000 7f1e6f90f640 1 -- start start 2024-09-13T18:22:36.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.679+0000 7f1e6f90f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e68067830 0x7f1e68196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:36.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.679+0000 7f1e6f90f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e680681d0 0x7f1e681967b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:36.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.679+0000 7f1e6f90f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e68104ce0 0x7f1e6819aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:36.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.679+0000 7f1e6f90f640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e6819b1a0 con 0x7f1e68104ce0 2024-09-13T18:22:36.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.679+0000 7f1e6f90f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e6819b310 con 0x7f1e68067830 2024-09-13T18:22:36.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.679+0000 7f1e6f90f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e6819b4d0 con 0x7f1e680681d0 2024-09-13T18:22:36.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.679+0000 7f1e6deac640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e68104ce0 0x7f1e6819aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:36.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.680+0000 7f1e6deac640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e68104ce0 0x7f1e6819aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51058/0 (socket says 172.21.15.114:51058) 2024-09-13T18:22:36.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.680+0000 7f1e6deac640 1 -- 172.21.15.114:0/2591609670 learned_addr learned my addr 172.21.15.114:0/2591609670 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:36.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.680+0000 7f1e6ceaa640 1 --2- 172.21.15.114:0/2591609670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e680681d0 0x7f1e681967b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:36.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.680+0000 7f1e6d6ab640 1 --2- 172.21.15.114:0/2591609670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e68067830 0x7f1e68196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:36.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.680+0000 7f1e6deac640 1 -- 172.21.15.114:0/2591609670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e680681d0 msgr2=0x7f1e681967b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:36.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.680+0000 7f1e6deac640 1 --2- 172.21.15.114:0/2591609670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e680681d0 0x7f1e681967b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:36.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.680+0000 7f1e6deac640 1 -- 172.21.15.114:0/2591609670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e68067830 msgr2=0x7f1e68196270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:36.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.680+0000 7f1e6deac640 1 --2- 172.21.15.114:0/2591609670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e68067830 0x7f1e68196270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:36.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.680+0000 7f1e6deac640 1 -- 172.21.15.114:0/2591609670 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e58009560 con 0x7f1e68104ce0 2024-09-13T18:22:36.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.680+0000 7f1e6ceaa640 1 --2- 172.21.15.114:0/2591609670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e680681d0 0x7f1e681967b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:22:36.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.680+0000 7f1e6deac640 1 --2- 172.21.15.114:0/2591609670 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e68104ce0 0x7f1e6819aba0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f1e6000cad0 tx=0x7f1e6000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:36.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.680+0000 7f1e5e7fc640 1 -- 172.21.15.114:0/2591609670 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6000ae20 con 0x7f1e68104ce0 2024-09-13T18:22:36.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.680+0000 7f1e6f90f640 1 -- 172.21.15.114:0/2591609670 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e6819b7b0 con 0x7f1e68104ce0 2024-09-13T18:22:36.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.680+0000 7f1e5e7fc640 1 -- 172.21.15.114:0/2591609670 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1e60005020 con 0x7f1e68104ce0 2024-09-13T18:22:36.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.680+0000 7f1e5e7fc640 1 -- 172.21.15.114:0/2591609670 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6000f6b0 con 0x7f1e68104ce0 2024-09-13T18:22:36.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.680+0000 7f1e6f90f640 1 -- 172.21.15.114:0/2591609670 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e681a3030 con 0x7f1e68104ce0 2024-09-13T18:22:36.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.681+0000 7f1e6f90f640 1 -- 172.21.15.114:0/2591609670 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e68101410 con 0x7f1e68104ce0 2024-09-13T18:22:36.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.682+0000 7f1e5e7fc640 1 -- 172.21.15.114:0/2591609670 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 24) v1 ==== 136050+0+0 (secure 0 0 0) 0x7f1e60005190 con 0x7f1e68104ce0 2024-09-13T18:22:36.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.683+0000 7f1e5e7fc640 1 --2- 172.21.15.114:0/2591609670 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f1e540a1290 0x7f1e540a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:36.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.683+0000 7f1e6d6ab640 1 --2- 172.21.15.114:0/2591609670 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f1e540a1290 0x7f1e540a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:36.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.683+0000 7f1e5e7fc640 1 -- 172.21.15.114:0/2591609670 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f1e600cd2c0 con 0x7f1e68104ce0 2024-09-13T18:22:36.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.683+0000 7f1e6d6ab640 1 --2- 172.21.15.114:0/2591609670 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f1e540a1290 0x7f1e540a3750 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1e64005e00 tx=0x7f1e64005c50 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:36.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.687+0000 7f1e5e7fc640 1 -- 172.21.15.114:0/2591609670 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f1e60096250 con 0x7f1e68104ce0 2024-09-13T18:22:36.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.868+0000 7f1e6f90f640 1 -- 172.21.15.114:0/2591609670 --> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f1e681975c0 con 0x7f1e540a1290 2024-09-13T18:22:36.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.927+0000 7f1e5e7fc640 1 -- 172.21.15.114:0/2591609670 <== mgr.15555 v2:172.21.15.114:6800/2440721366 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f1e681975c0 con 0x7f1e540a1290 2024-09-13T18:22:36.930 INFO:teuthology.orchestra.run.smithi114.stdout:Schedule set for path / 2024-09-13T18:22:36.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.929+0000 7f1e6f90f640 1 -- 172.21.15.114:0/2591609670 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f1e540a1290 msgr2=0x7f1e540a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:36.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.929+0000 7f1e6f90f640 1 --2- 172.21.15.114:0/2591609670 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f1e540a1290 0x7f1e540a3750 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1e64005e00 tx=0x7f1e64005c50 comp rx=0 tx=0).stop 2024-09-13T18:22:36.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.929+0000 7f1e6f90f640 1 -- 172.21.15.114:0/2591609670 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e68104ce0 msgr2=0x7f1e6819aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:36.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.929+0000 7f1e6f90f640 1 --2- 172.21.15.114:0/2591609670 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e68104ce0 0x7f1e6819aba0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f1e6000cad0 tx=0x7f1e6000cf90 comp rx=0 tx=0).stop 2024-09-13T18:22:36.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.929+0000 7f1e6f90f640 1 -- 172.21.15.114:0/2591609670 shutdown_connections 2024-09-13T18:22:36.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.929+0000 7f1e6f90f640 1 --2- 172.21.15.114:0/2591609670 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f1e540a1290 0x7f1e540a3750 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:36.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.929+0000 7f1e6f90f640 1 --2- 172.21.15.114:0/2591609670 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e68104ce0 0x7f1e6819aba0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:36.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.929+0000 7f1e6f90f640 1 --2- 172.21.15.114:0/2591609670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e680681d0 0x7f1e681967b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:36.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.929+0000 7f1e6f90f640 1 --2- 172.21.15.114:0/2591609670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e68067830 0x7f1e68196270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:36.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.929+0000 7f1e6f90f640 1 -- 172.21.15.114:0/2591609670 >> 172.21.15.114:0/2591609670 conn(0x7f1e68074940 msgr2=0x7f1e680fd1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:36.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.929+0000 7f1e6f90f640 1 -- 172.21.15.114:0/2591609670 shutdown_connections 2024-09-13T18:22:36.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:36.929+0000 7f1e6f90f640 1 -- 172.21.15.114:0/2591609670 wait complete. 2024-09-13T18:22:36.942 DEBUG:teuthology.orchestra.run.smithi114:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-09-13T18:22:37.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.041+0000 7fe634dc7640 1 -- 172.21.15.114:0/3081768594 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe630104c30 msgr2=0x7fe630105010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:37.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.041+0000 7fe634dc7640 1 --2- 172.21.15.114:0/3081768594 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe630104c30 0x7fe630105010 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe620009cd0 tx=0x7fe62002a8e0 comp rx=0 tx=0).stop 2024-09-13T18:22:37.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.041+0000 7fe634dc7640 1 -- 172.21.15.114:0/3081768594 shutdown_connections 2024-09-13T18:22:37.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.041+0000 7fe634dc7640 1 --2- 172.21.15.114:0/3081768594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe6300ff970 0x7fe63010ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:37.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.041+0000 7fe634dc7640 1 --2- 172.21.15.114:0/3081768594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6300feff0 0x7fe6300ff430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:37.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.041+0000 7fe634dc7640 1 --2- 172.21.15.114:0/3081768594 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe630104c30 0x7fe630105010 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:37.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.041+0000 7fe634dc7640 1 -- 172.21.15.114:0/3081768594 >> 172.21.15.114:0/3081768594 conn(0x7fe630074940 msgr2=0x7fe630074d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:37.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.041+0000 7fe634dc7640 1 -- 172.21.15.114:0/3081768594 shutdown_connections 2024-09-13T18:22:37.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.041+0000 7fe634dc7640 1 -- 172.21.15.114:0/3081768594 wait complete. 2024-09-13T18:22:37.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.041+0000 7fe634dc7640 1 Processor -- start 2024-09-13T18:22:37.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.041+0000 7fe634dc7640 1 -- start start 2024-09-13T18:22:37.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.042+0000 7fe634dc7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe6300feff0 0x7fe6301960c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:37.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.042+0000 7fe634dc7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6300ff970 0x7fe630196600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:37.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.042+0000 7fe634dc7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe630104c30 0x7fe63019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:37.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.042+0000 7fe634dc7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe63019aff0 con 0x7fe6300feff0 2024-09-13T18:22:37.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.042+0000 7fe634dc7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe63019b160 con 0x7fe6300ff970 2024-09-13T18:22:37.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.042+0000 7fe634dc7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe63019b320 con 0x7fe630104c30 2024-09-13T18:22:37.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.042+0000 7fe62e59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe6300feff0 0x7fe6301960c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:37.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.042+0000 7fe62e59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe6300feff0 0x7fe6301960c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51074/0 (socket says 172.21.15.114:51074) 2024-09-13T18:22:37.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.042+0000 7fe62e59c640 1 -- 172.21.15.114:0/662553099 learned_addr learned my addr 172.21.15.114:0/662553099 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:37.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.042+0000 7fe62ed9d640 1 --2- 172.21.15.114:0/662553099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe630104c30 0x7fe63019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:37.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.042+0000 7fe62dd9b640 1 --2- 172.21.15.114:0/662553099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6300ff970 0x7fe630196600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:37.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.042+0000 7fe62e59c640 1 -- 172.21.15.114:0/662553099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe630104c30 msgr2=0x7fe63019a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:37.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.042+0000 7fe62e59c640 1 --2- 172.21.15.114:0/662553099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe630104c30 0x7fe63019a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:37.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.042+0000 7fe62e59c640 1 -- 172.21.15.114:0/662553099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6300ff970 msgr2=0x7fe630196600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:37.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.042+0000 7fe62e59c640 1 --2- 172.21.15.114:0/662553099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6300ff970 0x7fe630196600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:37.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.042+0000 7fe62e59c640 1 -- 172.21.15.114:0/662553099 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe620009900 con 0x7fe6300feff0 2024-09-13T18:22:37.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.042+0000 7fe62e59c640 1 --2- 172.21.15.114:0/662553099 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe6300feff0 0x7fe6301960c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe620004af0 tx=0x7fe620030e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:37.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.042+0000 7fe61f7fe640 1 -- 172.21.15.114:0/662553099 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe620031b80 con 0x7fe6300feff0 2024-09-13T18:22:37.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.042+0000 7fe634dc7640 1 -- 172.21.15.114:0/662553099 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe63019b5a0 con 0x7fe6300feff0 2024-09-13T18:22:37.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.042+0000 7fe61f7fe640 1 -- 172.21.15.114:0/662553099 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe620003e00 con 0x7fe6300feff0 2024-09-13T18:22:37.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.042+0000 7fe61f7fe640 1 -- 172.21.15.114:0/662553099 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe620045410 con 0x7fe6300feff0 2024-09-13T18:22:37.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.043+0000 7fe634dc7640 1 -- 172.21.15.114:0/662553099 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6301a2e80 con 0x7fe6300feff0 2024-09-13T18:22:37.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.044+0000 7fe634dc7640 1 -- 172.21.15.114:0/662553099 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6301002b0 con 0x7fe6300feff0 2024-09-13T18:22:37.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.044+0000 7fe61f7fe640 1 -- 172.21.15.114:0/662553099 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 24) v1 ==== 136050+0+0 (secure 0 0 0) 0x7fe62000a040 con 0x7fe6300feff0 2024-09-13T18:22:37.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.045+0000 7fe61f7fe640 1 --2- 172.21.15.114:0/662553099 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe6180a1290 0x7fe6180a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:37.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.045+0000 7fe62dd9b640 1 --2- 172.21.15.114:0/662553099 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe6180a1290 0x7fe6180a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:37.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.045+0000 7fe61f7fe640 1 -- 172.21.15.114:0/662553099 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fe6200ee8a0 con 0x7fe6300feff0 2024-09-13T18:22:37.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.046+0000 7fe62dd9b640 1 --2- 172.21.15.114:0/662553099 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe6180a1290 0x7fe6180a3750 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe624005e10 tx=0x7fe624005da0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:37.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.049+0000 7fe61f7fe640 1 -- 172.21.15.114:0/662553099 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fe6200b7830 con 0x7fe6300feff0 2024-09-13T18:22:37.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:37 smithi114 ceph-mon[30897]: scrub queued for path: / 2024-09-13T18:22:37.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:37 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:22:37.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:37 smithi114 ceph-mon[30897]: scrub summary: active paths [/] 2024-09-13T18:22:37.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:37 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:22:37.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:37 smithi114 ceph-mon[30897]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 293 B/s wr, 25 op/s 2024-09-13T18:22:37.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:37 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/818194881' entity='client.admin' 2024-09-13T18:22:37.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:37 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:22:37.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:37 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:22:37.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:37 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:22:37.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:37 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:37.222 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:37 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2366049253' entity='client.admin' 2024-09-13T18:22:37.222 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:37 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:22:37.222 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:37 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:22:37.222 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:37 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:22:37.222 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:37 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:37.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.233+0000 7fe634dc7640 1 -- 172.21.15.114:0/662553099 --> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) v1 -- 0x7fe630197410 con 0x7fe6180a1290 2024-09-13T18:22:37.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.290+0000 7fe61f7fe640 1 -- 172.21.15.114:0/662553099 <== mgr.15555 v2:172.21.15.114:6800/2440721366 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7fe630197410 con 0x7fe6180a1290 2024-09-13T18:22:37.293 INFO:teuthology.orchestra.run.smithi114.stdout:Retention added to path / 2024-09-13T18:22:37.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.292+0000 7fe634dc7640 1 -- 172.21.15.114:0/662553099 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe6180a1290 msgr2=0x7fe6180a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:37.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.292+0000 7fe634dc7640 1 --2- 172.21.15.114:0/662553099 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe6180a1290 0x7fe6180a3750 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe624005e10 tx=0x7fe624005da0 comp rx=0 tx=0).stop 2024-09-13T18:22:37.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.292+0000 7fe634dc7640 1 -- 172.21.15.114:0/662553099 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe6300feff0 msgr2=0x7fe6301960c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:37.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.292+0000 7fe634dc7640 1 --2- 172.21.15.114:0/662553099 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe6300feff0 0x7fe6301960c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe620004af0 tx=0x7fe620030e50 comp rx=0 tx=0).stop 2024-09-13T18:22:37.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.292+0000 7fe634dc7640 1 -- 172.21.15.114:0/662553099 shutdown_connections 2024-09-13T18:22:37.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.292+0000 7fe634dc7640 1 --2- 172.21.15.114:0/662553099 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe6180a1290 0x7fe6180a3750 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:37.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.292+0000 7fe634dc7640 1 --2- 172.21.15.114:0/662553099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe630104c30 0x7fe63019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:37.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.292+0000 7fe634dc7640 1 --2- 172.21.15.114:0/662553099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6300ff970 0x7fe630196600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:37.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.292+0000 7fe634dc7640 1 --2- 172.21.15.114:0/662553099 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe6300feff0 0x7fe6301960c0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:37.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.292+0000 7fe634dc7640 1 -- 172.21.15.114:0/662553099 >> 172.21.15.114:0/662553099 conn(0x7fe630074940 msgr2=0x7fe6300fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:37.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.292+0000 7fe634dc7640 1 -- 172.21.15.114:0/662553099 shutdown_connections 2024-09-13T18:22:37.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.292+0000 7fe634dc7640 1 -- 172.21.15.114:0/662553099 wait complete. 2024-09-13T18:22:37.305 DEBUG:teuthology.orchestra.run.smithi114:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-13T18:22:37.363 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:37 smithi204 ceph-mon[34777]: scrub queued for path: / 2024-09-13T18:22:37.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:37 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:22:37.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:37 smithi204 ceph-mon[34777]: scrub summary: active paths [/] 2024-09-13T18:22:37.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:37 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:22:37.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:37 smithi204 ceph-mon[34777]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 293 B/s wr, 25 op/s 2024-09-13T18:22:37.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:37 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/818194881' entity='client.admin' 2024-09-13T18:22:37.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:37 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:22:37.364 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:37 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:22:37.365 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:37 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:22:37.365 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:37 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:37.365 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:37 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/2366049253' entity='client.admin' 2024-09-13T18:22:37.365 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:37 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:22:37.365 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:37 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:22:37.365 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:37 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:22:37.365 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:37 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:37.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:37 smithi134 ceph-mon[34283]: scrub queued for path: / 2024-09-13T18:22:37.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:37 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:22:37.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:37 smithi134 ceph-mon[34283]: scrub summary: active paths [/] 2024-09-13T18:22:37.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:37 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:22:37.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:37 smithi134 ceph-mon[34283]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 293 B/s wr, 25 op/s 2024-09-13T18:22:37.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:37 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/818194881' entity='client.admin' 2024-09-13T18:22:37.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:37 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:22:37.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:37 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:22:37.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:37 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:22:37.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:37 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:37.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:37 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/2366049253' entity='client.admin' 2024-09-13T18:22:37.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:37 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:22:37.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:37 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:22:37.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:37 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:22:37.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:37 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:22:37.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.402+0000 7f6e01441640 1 -- 172.21.15.114:0/2929624038 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6dfc067830 msgr2=0x7f6dfc067c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:37.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.402+0000 7f6e01441640 1 --2- 172.21.15.114:0/2929624038 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6dfc067830 0x7f6dfc067c90 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f6df0009900 tx=0x7f6df002a410 comp rx=0 tx=0).stop 2024-09-13T18:22:37.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.402+0000 7f6e01441640 1 -- 172.21.15.114:0/2929624038 shutdown_connections 2024-09-13T18:22:37.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.402+0000 7f6e01441640 1 --2- 172.21.15.114:0/2929624038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6dfc0681d0 0x7f6dfc10cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:37.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.402+0000 7f6e01441640 1 --2- 172.21.15.114:0/2929624038 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6dfc067830 0x7f6dfc067c90 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:37.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.402+0000 7f6e01441640 1 --2- 172.21.15.114:0/2929624038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6dfc104ce0 0x7f6dfc1050c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:37.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.402+0000 7f6e01441640 1 -- 172.21.15.114:0/2929624038 >> 172.21.15.114:0/2929624038 conn(0x7f6dfc074940 msgr2=0x7f6dfc074d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:37.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.402+0000 7f6e01441640 1 -- 172.21.15.114:0/2929624038 shutdown_connections 2024-09-13T18:22:37.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.402+0000 7f6e01441640 1 -- 172.21.15.114:0/2929624038 wait complete. 2024-09-13T18:22:37.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.402+0000 7f6e01441640 1 Processor -- start 2024-09-13T18:22:37.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.402+0000 7f6e01441640 1 -- start start 2024-09-13T18:22:37.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.403+0000 7f6e01441640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6dfc067830 0x7f6dfc196190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:37.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.403+0000 7f6e01441640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6dfc0681d0 0x7f6dfc1966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:37.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.403+0000 7f6e01441640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6dfc104ce0 0x7f6dfc19aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:37.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.403+0000 7f6e01441640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dfc19b0c0 con 0x7f6dfc104ce0 2024-09-13T18:22:37.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.403+0000 7f6e01441640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dfc19b230 con 0x7f6dfc067830 2024-09-13T18:22:37.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.403+0000 7f6e01441640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dfc19b3f0 con 0x7f6dfc0681d0 2024-09-13T18:22:37.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.403+0000 7f6dfb7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6dfc104ce0 0x7f6dfc19aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:37.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.403+0000 7f6dfb7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6dfc104ce0 0x7f6dfc19aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51092/0 (socket says 172.21.15.114:51092) 2024-09-13T18:22:37.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.403+0000 7f6dfb7fe640 1 -- 172.21.15.114:0/3456745016 learned_addr learned my addr 172.21.15.114:0/3456745016 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:37.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.403+0000 7f6dfa7fc640 1 --2- 172.21.15.114:0/3456745016 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6dfc0681d0 0x7f6dfc1966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:37.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.403+0000 7f6dfaffd640 1 --2- 172.21.15.114:0/3456745016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6dfc067830 0x7f6dfc196190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:37.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.403+0000 7f6dfb7fe640 1 -- 172.21.15.114:0/3456745016 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6dfc0681d0 msgr2=0x7f6dfc1966d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:37.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.403+0000 7f6dfb7fe640 1 --2- 172.21.15.114:0/3456745016 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6dfc0681d0 0x7f6dfc1966d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:37.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.403+0000 7f6dfb7fe640 1 -- 172.21.15.114:0/3456745016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6dfc067830 msgr2=0x7f6dfc196190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:37.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.403+0000 7f6dfb7fe640 1 --2- 172.21.15.114:0/3456745016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6dfc067830 0x7f6dfc196190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:37.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.403+0000 7f6dfb7fe640 1 -- 172.21.15.114:0/3456745016 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df0009560 con 0x7f6dfc104ce0 2024-09-13T18:22:37.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.403+0000 7f6dfb7fe640 1 --2- 172.21.15.114:0/3456745016 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6dfc104ce0 0x7f6dfc19aac0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f6df400d750 tx=0x7f6df400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:37.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.403+0000 7f6de3fff640 1 -- 172.21.15.114:0/3456745016 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df4009bd0 con 0x7f6dfc104ce0 2024-09-13T18:22:37.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.403+0000 7f6de3fff640 1 -- 172.21.15.114:0/3456745016 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6df400de60 con 0x7f6dfc104ce0 2024-09-13T18:22:37.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.403+0000 7f6de3fff640 1 -- 172.21.15.114:0/3456745016 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df4004cf0 con 0x7f6dfc104ce0 2024-09-13T18:22:37.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.403+0000 7f6e01441640 1 -- 172.21.15.114:0/3456745016 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dfc19b6d0 con 0x7f6dfc104ce0 2024-09-13T18:22:37.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.403+0000 7f6e01441640 1 -- 172.21.15.114:0/3456745016 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dfc1a2f50 con 0x7f6dfc104ce0 2024-09-13T18:22:37.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.405+0000 7f6e01441640 1 -- 172.21.15.114:0/3456745016 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dfc101410 con 0x7f6dfc104ce0 2024-09-13T18:22:37.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.405+0000 7f6de3fff640 1 -- 172.21.15.114:0/3456745016 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 24) v1 ==== 136050+0+0 (secure 0 0 0) 0x7f6df4004e50 con 0x7f6dfc104ce0 2024-09-13T18:22:37.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.406+0000 7f6de3fff640 1 --2- 172.21.15.114:0/3456745016 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f6de40a1290 0x7f6de40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:37.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.406+0000 7f6dfaffd640 1 --2- 172.21.15.114:0/3456745016 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f6de40a1290 0x7f6de40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:37.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.407+0000 7f6de3fff640 1 -- 172.21.15.114:0/3456745016 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f6df40ccba0 con 0x7f6dfc104ce0 2024-09-13T18:22:37.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.407+0000 7f6dfaffd640 1 --2- 172.21.15.114:0/3456745016 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f6de40a1290 0x7f6de40a3750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6dec005fd0 tx=0x7f6dec0098b0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:37.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.411+0000 7f6de3fff640 1 -- 172.21.15.114:0/3456745016 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f6df4095b30 con 0x7f6dfc104ce0 2024-09-13T18:22:37.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.594+0000 7f6e01441640 1 -- 172.21.15.114:0/3456745016 --> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f6dfc1974e0 con 0x7f6de40a1290 2024-09-13T18:22:37.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.600+0000 7f6de3fff640 1 -- 172.21.15.114:0/3456745016 <== mgr.15555 v2:172.21.15.114:6800/2440721366 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+298 (secure 0 0 0) 0x7f6dfc1974e0 con 0x7f6de40a1290 2024-09-13T18:22:37.603 INFO:teuthology.orchestra.run.smithi114.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-13T00:00:00", "created": "2024-09-13T18:22:36", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-09-13T18:22:37.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.602+0000 7f6e01441640 1 -- 172.21.15.114:0/3456745016 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f6de40a1290 msgr2=0x7f6de40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:37.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.602+0000 7f6e01441640 1 --2- 172.21.15.114:0/3456745016 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f6de40a1290 0x7f6de40a3750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6dec005fd0 tx=0x7f6dec0098b0 comp rx=0 tx=0).stop 2024-09-13T18:22:37.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.602+0000 7f6e01441640 1 -- 172.21.15.114:0/3456745016 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6dfc104ce0 msgr2=0x7f6dfc19aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:37.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.602+0000 7f6e01441640 1 --2- 172.21.15.114:0/3456745016 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6dfc104ce0 0x7f6dfc19aac0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f6df400d750 tx=0x7f6df400dc10 comp rx=0 tx=0).stop 2024-09-13T18:22:37.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.602+0000 7f6e01441640 1 -- 172.21.15.114:0/3456745016 shutdown_connections 2024-09-13T18:22:37.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.602+0000 7f6e01441640 1 --2- 172.21.15.114:0/3456745016 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f6de40a1290 0x7f6de40a3750 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:37.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.602+0000 7f6e01441640 1 --2- 172.21.15.114:0/3456745016 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6dfc104ce0 0x7f6dfc19aac0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:37.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.602+0000 7f6e01441640 1 --2- 172.21.15.114:0/3456745016 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6dfc0681d0 0x7f6dfc1966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:37.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.602+0000 7f6e01441640 1 --2- 172.21.15.114:0/3456745016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6dfc067830 0x7f6dfc196190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:37.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.602+0000 7f6e01441640 1 -- 172.21.15.114:0/3456745016 >> 172.21.15.114:0/3456745016 conn(0x7f6dfc074940 msgr2=0x7f6dfc0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:37.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.602+0000 7f6e01441640 1 -- 172.21.15.114:0/3456745016 shutdown_connections 2024-09-13T18:22:37.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.602+0000 7f6e01441640 1 -- 172.21.15.114:0/3456745016 wait complete. 2024-09-13T18:22:37.615 DEBUG:teuthology.orchestra.run.smithi114:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-13T18:22:37.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.712+0000 7f92a8290640 1 -- 172.21.15.114:0/2027205810 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92a00ff360 msgr2=0x7f92a00ff7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:37.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.712+0000 7f92a8290640 1 --2- 172.21.15.114:0/2027205810 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92a00ff360 0x7f92a00ff7a0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f9290009900 tx=0x7f929002a410 comp rx=0 tx=0).stop 2024-09-13T18:22:37.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.713+0000 7f92a8290640 1 -- 172.21.15.114:0/2027205810 shutdown_connections 2024-09-13T18:22:37.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.713+0000 7f92a8290640 1 --2- 172.21.15.114:0/2027205810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92a00ffce0 0x7f92a010ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:37.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.713+0000 7f92a8290640 1 --2- 172.21.15.114:0/2027205810 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92a00ff360 0x7f92a00ff7a0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:37.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.713+0000 7f92a8290640 1 --2- 172.21.15.114:0/2027205810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92a0103bc0 0x7f92a0103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:37.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.713+0000 7f92a8290640 1 -- 172.21.15.114:0/2027205810 >> 172.21.15.114:0/2027205810 conn(0x7f92a0075350 msgr2=0x7f92a00fc6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:37.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.713+0000 7f92a8290640 1 -- 172.21.15.114:0/2027205810 shutdown_connections 2024-09-13T18:22:37.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.713+0000 7f92a8290640 1 -- 172.21.15.114:0/2027205810 wait complete. 2024-09-13T18:22:37.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.713+0000 7f92a8290640 1 Processor -- start 2024-09-13T18:22:37.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.713+0000 7f92a8290640 1 -- start start 2024-09-13T18:22:37.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.713+0000 7f92a8290640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92a00ff360 0x7f92a010c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:37.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.713+0000 7f92a8290640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92a00ffce0 0x7f92a010c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:37.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.713+0000 7f92a8290640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92a0103bc0 0x7f92a010cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:37.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.713+0000 7f92a8290640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a01a4520 con 0x7f92a0103bc0 2024-09-13T18:22:37.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.713+0000 7f92a8290640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a01a4690 con 0x7f92a00ffce0 2024-09-13T18:22:37.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.713+0000 7f92a8290640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a01a4850 con 0x7f92a00ff360 2024-09-13T18:22:37.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.713+0000 7f92a682d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92a0103bc0 0x7f92a010cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:37.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.713+0000 7f92a682d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92a0103bc0 0x7f92a010cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51124/0 (socket says 172.21.15.114:51124) 2024-09-13T18:22:37.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.713+0000 7f92a682d640 1 -- 172.21.15.114:0/4147770299 learned_addr learned my addr 172.21.15.114:0/4147770299 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:37.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.714+0000 7f92a602c640 1 --2- 172.21.15.114:0/4147770299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92a00ff360 0x7f92a010c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:37.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.714+0000 7f92a582b640 1 --2- 172.21.15.114:0/4147770299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92a00ffce0 0x7f92a010c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:37.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.714+0000 7f92a682d640 1 -- 172.21.15.114:0/4147770299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92a00ff360 msgr2=0x7f92a010c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:37.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.714+0000 7f92a682d640 1 --2- 172.21.15.114:0/4147770299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92a00ff360 0x7f92a010c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:37.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.714+0000 7f92a682d640 1 -- 172.21.15.114:0/4147770299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92a00ffce0 msgr2=0x7f92a010c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:37.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.714+0000 7f92a682d640 1 --2- 172.21.15.114:0/4147770299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92a00ffce0 0x7f92a010c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:37.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.714+0000 7f92a682d640 1 -- 172.21.15.114:0/4147770299 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9290009560 con 0x7f92a0103bc0 2024-09-13T18:22:37.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.714+0000 7f92a682d640 1 --2- 172.21.15.114:0/4147770299 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92a0103bc0 0x7f92a010cf30 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f929800e9f0 tx=0x7f929800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:37.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.714+0000 7f92977fe640 1 -- 172.21.15.114:0/4147770299 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929800cd80 con 0x7f92a0103bc0 2024-09-13T18:22:37.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.714+0000 7f92a8290640 1 -- 172.21.15.114:0/4147770299 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92a01a4b30 con 0x7f92a0103bc0 2024-09-13T18:22:37.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.714+0000 7f92977fe640 1 -- 172.21.15.114:0/4147770299 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9298005020 con 0x7f92a0103bc0 2024-09-13T18:22:37.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.714+0000 7f92977fe640 1 -- 172.21.15.114:0/4147770299 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92980116d0 con 0x7f92a0103bc0 2024-09-13T18:22:37.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.714+0000 7f92a8290640 1 -- 172.21.15.114:0/4147770299 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92a01a5010 con 0x7f92a0103bc0 2024-09-13T18:22:37.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.715+0000 7f92a8290640 1 -- 172.21.15.114:0/4147770299 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92a0100370 con 0x7f92a0103bc0 2024-09-13T18:22:37.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.716+0000 7f92977fe640 1 -- 172.21.15.114:0/4147770299 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 24) v1 ==== 136050+0+0 (secure 0 0 0) 0x7f9298005190 con 0x7f92a0103bc0 2024-09-13T18:22:37.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.717+0000 7f92977fe640 1 --2- 172.21.15.114:0/4147770299 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f928c0a1290 0x7f928c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:37.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.717+0000 7f92a602c640 1 --2- 172.21.15.114:0/4147770299 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f928c0a1290 0x7f928c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:37.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.717+0000 7f92977fe640 1 -- 172.21.15.114:0/4147770299 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f92980cd320 con 0x7f92a0103bc0 2024-09-13T18:22:37.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.717+0000 7f92a602c640 1 --2- 172.21.15.114:0/4147770299 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f928c0a1290 0x7f928c0a3750 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f929c005d00 tx=0x7f929c00b410 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:37.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.721+0000 7f92977fe640 1 -- 172.21.15.114:0/4147770299 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f9298096360 con 0x7f92a0103bc0 2024-09-13T18:22:37.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.902+0000 7f92a8290640 1 -- 172.21.15.114:0/4147770299 --> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f92a010fc90 con 0x7f928c0a1290 2024-09-13T18:22:37.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.908+0000 7f92977fe640 1 -- 172.21.15.114:0/4147770299 <== mgr.15555 v2:172.21.15.114:6800/2440721366 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f92a010fc90 con 0x7f928c0a1290 2024-09-13T18:22:37.911 INFO:teuthology.orchestra.run.smithi114.stdout:/ 1m 6m3h 2024-09-13T18:22:37.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.910+0000 7f92a8290640 1 -- 172.21.15.114:0/4147770299 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f928c0a1290 msgr2=0x7f928c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:37.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.910+0000 7f92a8290640 1 --2- 172.21.15.114:0/4147770299 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f928c0a1290 0x7f928c0a3750 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f929c005d00 tx=0x7f929c00b410 comp rx=0 tx=0).stop 2024-09-13T18:22:37.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.910+0000 7f92a8290640 1 -- 172.21.15.114:0/4147770299 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92a0103bc0 msgr2=0x7f92a010cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:37.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.910+0000 7f92a8290640 1 --2- 172.21.15.114:0/4147770299 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92a0103bc0 0x7f92a010cf30 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f929800e9f0 tx=0x7f929800eeb0 comp rx=0 tx=0).stop 2024-09-13T18:22:37.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.910+0000 7f92a8290640 1 -- 172.21.15.114:0/4147770299 shutdown_connections 2024-09-13T18:22:37.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.910+0000 7f92a8290640 1 --2- 172.21.15.114:0/4147770299 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f928c0a1290 0x7f928c0a3750 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:37.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.910+0000 7f92a8290640 1 --2- 172.21.15.114:0/4147770299 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92a0103bc0 0x7f92a010cf30 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:37.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.910+0000 7f92a8290640 1 --2- 172.21.15.114:0/4147770299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92a00ffce0 0x7f92a010c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:37.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.910+0000 7f92a8290640 1 --2- 172.21.15.114:0/4147770299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92a00ff360 0x7f92a010c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:37.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.910+0000 7f92a8290640 1 -- 172.21.15.114:0/4147770299 >> 172.21.15.114:0/4147770299 conn(0x7f92a0075350 msgr2=0x7f92a00fc6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:37.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.910+0000 7f92a8290640 1 -- 172.21.15.114:0/4147770299 shutdown_connections 2024-09-13T18:22:37.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:22:37.910+0000 7f92a8290640 1 -- 172.21.15.114:0/4147770299 wait complete. 2024-09-13T18:22:37.924 DEBUG:teuthology.orchestra.run.smithi114:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-09-13T18:22:37.950 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-09-13T18:22:37.961 INFO:teuthology.run_tasks:Running task background_exec... 2024-09-13T18:22:37.971 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi114.front.sepia.ceph.com 2024-09-13T18:22:37.971 DEBUG:teuthology.orchestra.run.smithi114:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-09-13T18:22:37.993 INFO:teuthology.run_tasks:Running task workunit... 2024-09-13T18:22:38.007 INFO:tasks.workunit:Pulling workunits from ref 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T18:22:38.008 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-13T18:22:38.008 INFO:tasks.workunit:timeout=3h 2024-09-13T18:22:38.008 INFO:tasks.workunit:cleanup=True 2024-09-13T18:22:38.008 DEBUG:teuthology.orchestra.run.smithi114:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-13T18:22:38.024 INFO:teuthology.orchestra.run.smithi114.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-13T18:22:38.024 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-09-13T18:22:38.024 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 78h/120d Inode: 1 Links: 2 2024-09-13T18:22:38.024 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T18:22:38.024 INFO:teuthology.orchestra.run.smithi114.stdout:Context: system_u:object_r:fusefs_t:s0 2024-09-13T18:22:38.025 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2024-09-13 18:20:28.538057573 +0000 2024-09-13T18:22:38.025 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-09-13 18:20:28.538057573 +0000 2024-09-13T18:22:38.025 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-09-13 18:22:27.725778260 +0000 2024-09-13T18:22:38.025 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: - 2024-09-13T18:22:38.025 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-13T18:22:38.025 DEBUG:teuthology.orchestra.run.smithi114:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-13T18:22:38.096 DEBUG:teuthology.orchestra.run.smithi114:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T18:22:38.152 INFO:tasks.workunit.client.0.smithi114.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-13T18:22:38.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:38 smithi114 ceph-mon[30897]: from='client.15798 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:22:38.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:38 smithi134 ceph-mon[34283]: from='client.15798 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:22:38.447 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:38 smithi204 ceph-mon[34777]: from='client.15798 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:22:39.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:39 smithi134 ceph-mon[34283]: from='client.15804 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:22:39.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:39 smithi134 ceph-mon[34283]: from='client.15810 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:22:39.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:39 smithi134 ceph-mon[34283]: from='client.15816 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:22:39.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:39 smithi134 ceph-mon[34283]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 227 B/s wr, 19 op/s 2024-09-13T18:22:39.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:39 smithi204 ceph-mon[34777]: from='client.15804 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:22:39.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:39 smithi204 ceph-mon[34777]: from='client.15810 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:22:39.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:39 smithi204 ceph-mon[34777]: from='client.15816 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:22:39.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:39 smithi204 ceph-mon[34777]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 227 B/s wr, 19 op/s 2024-09-13T18:22:39.473 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:39 smithi114 ceph-mon[30897]: from='client.15804 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:22:39.473 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:39 smithi114 ceph-mon[30897]: from='client.15810 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:22:39.473 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:39 smithi114 ceph-mon[30897]: from='client.15816 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:22:39.473 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:39 smithi114 ceph-mon[30897]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 227 B/s wr, 19 op/s 2024-09-13T18:22:41.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:41 smithi134 ceph-mon[34283]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-09-13T18:22:41.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:41 smithi204 ceph-mon[34777]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-09-13T18:22:41.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:41 smithi114 ceph-mon[30897]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-09-13T18:22:43.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:43 smithi134 ceph-mon[34283]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-09-13T18:22:43.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:43 smithi204 ceph-mon[34777]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-09-13T18:22:43.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:43 smithi114 ceph-mon[30897]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-09-13T18:22:44.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:44 smithi204 ceph-mon[34777]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-09-13T18:22:44.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:44 smithi114 ceph-mon[30897]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-09-13T18:22:44.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:44 smithi134 ceph-mon[34283]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-09-13T18:22:46.422 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:46 smithi134 ceph-mon[34283]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 77 KiB/s wr, 54 op/s 2024-09-13T18:22:46.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:46 smithi204 ceph-mon[34777]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 77 KiB/s wr, 54 op/s 2024-09-13T18:22:46.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:46 smithi114 ceph-mon[30897]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 77 KiB/s wr, 54 op/s 2024-09-13T18:22:47.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:47 smithi204 ceph-mon[34777]: scrub summary: idle 2024-09-13T18:22:47.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:47 smithi114 ceph-mon[30897]: scrub summary: idle 2024-09-13T18:22:47.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:47 smithi134 ceph-mon[34283]: scrub summary: idle 2024-09-13T18:22:48.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:48 smithi204 ceph-mon[34777]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-09-13T18:22:48.469 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:48 smithi114 ceph-mon[30897]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-09-13T18:22:48.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:48 smithi134 ceph-mon[34283]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-09-13T18:22:50.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:50 smithi204 ceph-mon[34777]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-09-13T18:22:50.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:50 smithi114 ceph-mon[30897]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-09-13T18:22:50.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:50 smithi134 ceph-mon[34283]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-09-13T18:22:51.132 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.126+0000 7f6c77087640 1 -- 172.21.15.114:0/2152734881 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c70101800 msgr2=0x7f6c70109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:51.132 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.126+0000 7f6c77087640 1 --2- 172.21.15.114:0/2152734881 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c70101800 0x7f6c70109ce0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c009900 tx=0x7f6c5c02a480 comp rx=0 tx=0).stop 2024-09-13T18:22:51.132 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.126+0000 7f6c77087640 1 -- 172.21.15.114:0/2152734881 shutdown_connections 2024-09-13T18:22:51.132 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.126+0000 7f6c77087640 1 --2- 172.21.15.114:0/2152734881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c7010a270 0x7f6c7010c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:51.132 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.126+0000 7f6c77087640 1 --2- 172.21.15.114:0/2152734881 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c70101800 0x7f6c70109ce0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:51.132 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.126+0000 7f6c77087640 1 --2- 172.21.15.114:0/2152734881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c70100ee0 0x7f6c701012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:51.132 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.126+0000 7f6c77087640 1 -- 172.21.15.114:0/2152734881 >> 172.21.15.114:0/2152734881 conn(0x7f6c700fa890 msgr2=0x7f6c700fccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:51.132 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.126+0000 7f6c77087640 1 -- 172.21.15.114:0/2152734881 shutdown_connections 2024-09-13T18:22:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.126+0000 7f6c77087640 1 -- 172.21.15.114:0/2152734881 wait complete. 2024-09-13T18:22:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.127+0000 7f6c77087640 1 Processor -- start 2024-09-13T18:22:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.127+0000 7f6c77087640 1 -- start start 2024-09-13T18:22:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.127+0000 7f6c77087640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c70100ee0 0x7f6c70196140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.127+0000 7f6c77087640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c70101800 0x7f6c70196680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.127+0000 7f6c77087640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c7010a270 0x7f6c7019aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.127+0000 7f6c77087640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7019b070 con 0x7f6c70100ee0 2024-09-13T18:22:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.127+0000 7f6c77087640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7019b1e0 con 0x7f6c70101800 2024-09-13T18:22:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.127+0000 7f6c77087640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7019b3a0 con 0x7f6c7010a270 2024-09-13T18:22:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.127+0000 7f6c74e23640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c70100ee0 0x7f6c70196140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.127+0000 7f6c74e23640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c70100ee0 0x7f6c70196140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44054/0 (socket says 172.21.15.114:44054) 2024-09-13T18:22:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.127+0000 7f6c74e23640 1 -- 172.21.15.114:0/196871743 learned_addr learned my addr 172.21.15.114:0/196871743 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.128+0000 7f6c75624640 1 --2- 172.21.15.114:0/196871743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c7010a270 0x7f6c7019aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.128+0000 7f6c6ffff640 1 --2- 172.21.15.114:0/196871743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c70101800 0x7f6c70196680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.128+0000 7f6c74e23640 1 -- 172.21.15.114:0/196871743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c7010a270 msgr2=0x7f6c7019aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.128+0000 7f6c74e23640 1 --2- 172.21.15.114:0/196871743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c7010a270 0x7f6c7019aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.128+0000 7f6c74e23640 1 -- 172.21.15.114:0/196871743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c70101800 msgr2=0x7f6c70196680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.128+0000 7f6c74e23640 1 --2- 172.21.15.114:0/196871743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c70101800 0x7f6c70196680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.128+0000 7f6c74e23640 1 -- 172.21.15.114:0/196871743 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c5c009560 con 0x7f6c70100ee0 2024-09-13T18:22:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.128+0000 7f6c6ffff640 1 --2- 172.21.15.114:0/196871743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c70101800 0x7f6c70196680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:22:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.128+0000 7f6c75624640 1 --2- 172.21.15.114:0/196871743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c7010a270 0x7f6c7019aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:22:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.128+0000 7f6c74e23640 1 --2- 172.21.15.114:0/196871743 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c70100ee0 0x7f6c70196140 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f6c6400c970 tx=0x7f6c6400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.128+0000 7f6c6dffb640 1 -- 172.21.15.114:0/196871743 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c64007bb0 con 0x7f6c70100ee0 2024-09-13T18:22:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.128+0000 7f6c6dffb640 1 -- 172.21.15.114:0/196871743 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6c64007d10 con 0x7f6c70100ee0 2024-09-13T18:22:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.128+0000 7f6c6dffb640 1 -- 172.21.15.114:0/196871743 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c64004d70 con 0x7f6c70100ee0 2024-09-13T18:22:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.128+0000 7f6c77087640 1 -- 172.21.15.114:0/196871743 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c7019b680 con 0x7f6c70100ee0 2024-09-13T18:22:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.128+0000 7f6c77087640 1 -- 172.21.15.114:0/196871743 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c701a2f00 con 0x7f6c70100ee0 2024-09-13T18:22:51.143 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.130+0000 7f6c6dffb640 1 -- 172.21.15.114:0/196871743 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 24) v1 ==== 136050+0+0 (secure 0 0 0) 0x7f6c64004f50 con 0x7f6c70100ee0 2024-09-13T18:22:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.131+0000 7f6c79e86740 1 Processor -- start 2024-09-13T18:22:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.131+0000 7f6c79e86740 1 -- start start 2024-09-13T18:22:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.131+0000 7f6c6dffb640 1 --2- 172.21.15.114:0/196871743 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f6c600a1310 0x7f6c600a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.131+0000 7f6c79e86740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564c3ed2e330 0x564c3ed2e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.131+0000 7f6c79e86740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c3ed2ec50 0x564c3ed38fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.131+0000 7f6c79e86740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c3ed394f0 0x564c3ed3b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.131+0000 7f6c79e86740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c3ec99c30 con 0x564c3ed2e330 2024-09-13T18:22:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.131+0000 7f6c79e86740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c3ec973c0 con 0x564c3ed394f0 2024-09-13T18:22:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.131+0000 7f6c79e86740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c3ec92cb0 con 0x564c3ed2ec50 2024-09-13T18:22:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.131+0000 7f6c75624640 1 --2- 172.21.15.114:0/196871743 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f6c600a1310 0x7f6c600a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:51.146 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.131+0000 7f6c74e23640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564c3ed2e330 0x564c3ed2e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:51.146 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.131+0000 7f6c74e23640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564c3ed2e330 0x564c3ed2e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44060/0 (socket says 172.21.15.114:44060) 2024-09-13T18:22:51.146 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.131+0000 7f6c6dffb640 1 -- 172.21.15.114:0/196871743 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f6c640ccfc0 con 0x7f6c70100ee0 2024-09-13T18:22:51.146 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.131+0000 7f6c74e23640 1 -- 172.21.15.114:0/354449766 learned_addr learned my addr 172.21.15.114:0/354449766 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:51.146 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.132+0000 7f6c6ffff640 1 --2- 172.21.15.114:0/354449766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c3ed2ec50 0x564c3ed38fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:51.146 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.132+0000 7f6c75624640 1 --2- 172.21.15.114:0/354449766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c3ed394f0 0x564c3ed3b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:51.146 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.132+0000 7f6c74e23640 1 -- 172.21.15.114:0/354449766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c3ed2ec50 msgr2=0x564c3ed38fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:51.146 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.132+0000 7f6c74e23640 1 --2- 172.21.15.114:0/354449766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c3ed2ec50 0x564c3ed38fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:51.146 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.132+0000 7f6c74e23640 1 -- 172.21.15.114:0/354449766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c3ed394f0 msgr2=0x564c3ed3b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:51.146 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.132+0000 7f6c74e23640 1 --2- 172.21.15.114:0/354449766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c3ed394f0 0x564c3ed3b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:51.146 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.132+0000 7f6c74e23640 1 -- 172.21.15.114:0/354449766 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c3ec39910 con 0x564c3ed2e330 2024-09-13T18:22:51.146 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.132+0000 7f6c75624640 1 --2- 172.21.15.114:0/354449766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c3ed394f0 0x564c3ed3b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:22:51.147 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.132+0000 7f6c6ffff640 1 --2- 172.21.15.114:0/354449766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c3ed2ec50 0x564c3ed38fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:22:51.147 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.132+0000 7f6c74e23640 1 --2- 172.21.15.114:0/354449766 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564c3ed2e330 0x564c3ed2e710 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6c640a1b00 tx=0x7f6c64099820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96fd07421fdd457f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:51.147 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.132+0000 7f6c75624640 1 --2- 172.21.15.114:0/196871743 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f6c600a1310 0x7f6c600a37d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x564c3ed2bc40 tx=0x7f6c68008730 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:51.147 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.132+0000 7f6c5affd640 1 -- 172.21.15.114:0/354449766 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c64077490 con 0x564c3ed2e330 2024-09-13T18:22:51.147 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.132+0000 7f6c5affd640 1 -- 172.21.15.114:0/354449766 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6c640046c0 con 0x564c3ed2e330 2024-09-13T18:22:51.147 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.132+0000 7f6c5affd640 1 -- 172.21.15.114:0/354449766 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c640962c0 con 0x564c3ed2e330 2024-09-13T18:22:51.147 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.133+0000 7f6c79e86740 1 -- 172.21.15.114:0/354449766 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564c3ed2e330 msgr2=0x564c3ed2e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:51.147 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.133+0000 7f6c79e86740 1 --2- 172.21.15.114:0/354449766 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564c3ed2e330 0x564c3ed2e710 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6c640a1b00 tx=0x7f6c64099820 comp rx=0 tx=0).stop 2024-09-13T18:22:51.147 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.133+0000 7f6c79e86740 1 -- 172.21.15.114:0/354449766 shutdown_connections 2024-09-13T18:22:51.147 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.133+0000 7f6c79e86740 1 --2- 172.21.15.114:0/354449766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c3ed394f0 0x564c3ed3b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:51.147 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.133+0000 7f6c79e86740 1 --2- 172.21.15.114:0/354449766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c3ed2ec50 0x564c3ed38fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:51.147 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.133+0000 7f6c79e86740 1 --2- 172.21.15.114:0/354449766 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564c3ed2e330 0x564c3ed2e710 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:51.148 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.133+0000 7f6c79e86740 1 -- 172.21.15.114:0/354449766 >> 172.21.15.114:0/354449766 conn(0x564c3ed2a8c0 msgr2=0x564c3ed2aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:51.148 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.133+0000 7f6c79e86740 1 -- 172.21.15.114:0/354449766 shutdown_connections 2024-09-13T18:22:51.148 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.133+0000 7f6c79e86740 1 -- 172.21.15.114:0/354449766 wait complete. 2024-09-13T18:22:51.148 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.133+0000 7f6c79e86740 1 -- start start 2024-09-13T18:22:51.148 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.133+0000 7f6c79e86740 1 Processor -- start 2024-09-13T18:22:51.148 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.134+0000 7f6c79e86740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564c3ed2e330 0x564c3ed38f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:51.148 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.134+0000 7f6c79e86740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c3ed2ec50 0x564c3ed36260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:51.148 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.134+0000 7f6c79e86740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c3ed394f0 0x564c3ed369f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:51.148 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.134+0000 7f6c79e86740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c3ec50e80 con 0x564c3ed2e330 2024-09-13T18:22:51.148 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.134+0000 7f6c79e86740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c3ec50b20 con 0x564c3ed2ec50 2024-09-13T18:22:51.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.134+0000 7f6c79e86740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c3ec16260 con 0x564c3ed394f0 2024-09-13T18:22:51.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.134+0000 7f6c79e86740 10 client.0 _ll_register_callbacks cb 0x564c3ec4bdf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:22:51.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.134+0000 7f6c74e23640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564c3ed2e330 0x564c3ed38f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:51.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.134+0000 7f6c74e23640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564c3ed2e330 0x564c3ed38f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44062/0 (socket says 172.21.15.114:44062) 2024-09-13T18:22:51.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.134+0000 7f6c74e23640 1 -- 172.21.15.114:0/1851898693 learned_addr learned my addr 172.21.15.114:0/1851898693 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:51.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.134+0000 7f6c75624640 1 --2- 172.21.15.114:0/1851898693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c3ed394f0 0x564c3ed369f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:51.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.134+0000 7f6c6ffff640 1 --2- 172.21.15.114:0/1851898693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c3ed2ec50 0x564c3ed36260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:51.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.134+0000 7f6c74e23640 1 -- 172.21.15.114:0/1851898693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c3ed394f0 msgr2=0x564c3ed369f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:51.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.134+0000 7f6c74e23640 1 --2- 172.21.15.114:0/1851898693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c3ed394f0 0x564c3ed369f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:51.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.134+0000 7f6c74e23640 1 -- 172.21.15.114:0/1851898693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c3ed2ec50 msgr2=0x564c3ed36260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:51.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.134+0000 7f6c74e23640 1 --2- 172.21.15.114:0/1851898693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c3ed2ec50 0x564c3ed36260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:51.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.134+0000 7f6c74e23640 1 -- 172.21.15.114:0/1851898693 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c701a2f00 con 0x564c3ed2e330 2024-09-13T18:22:51.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.134+0000 7f6c74e23640 1 --2- 172.21.15.114:0/1851898693 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564c3ed2e330 0x564c3ed38f20 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f6c640960b0 tx=0x7f6c64099bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34fa55cf6a46b61f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:51.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.134+0000 7f6c4ffff640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:22:51.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.134+0000 7f6c4ffff640 1 -- 172.21.15.114:0/1851898693 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6409a330 con 0x564c3ed2e330 2024-09-13T18:22:51.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.134+0000 7f6c4ffff640 1 -- 172.21.15.114:0/1851898693 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f6c6409d9e0 con 0x564c3ed2e330 2024-09-13T18:22:51.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.134+0000 7f6c79e86740 1 -- 172.21.15.114:0/1851898693 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564c3ecbb680 con 0x564c3ed2e330 2024-09-13T18:22:51.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.135+0000 7f6c4ffff640 1 -- 172.21.15.114:0/1851898693 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c640a1db0 con 0x564c3ed2e330 2024-09-13T18:22:51.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.135+0000 7f6c79e86740 10 client.15828 fetch_fsmap learned FSMap version 30 2024-09-13T18:22:51.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.135+0000 7f6c79e86740 1 -- 172.21.15.114:0/1851898693 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x564c3ec91330 con 0x564c3ed2e330 2024-09-13T18:22:51.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.135+0000 7f6c4ffff640 1 -- 172.21.15.114:0/1851898693 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6c640b6080 con 0x564c3ed2e330 2024-09-13T18:22:51.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.135+0000 7f6c4ffff640 1 -- 172.21.15.114:0/1851898693 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6c64016030 con 0x564c3ed2e330 2024-09-13T18:22:51.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.135+0000 7f6c79e86740 10 client.15828 fetch_fsmap finished waiting for FSMap version 30 2024-09-13T18:22:51.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.135+0000 7f6c79e86740 10 client.15828 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:22:51.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.135+0000 7f6c79e86740 20 client.15828 populate_metadata read hostname 'smithi114' 2024-09-13T18:22:51.151 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.135+0000 7f6c79e86740 1 --2- 172.21.15.114:0/1851898693 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x564c3ed48a30 0x564c3ed4ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:51.151 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.135+0000 7f6c79e86740 4 client.15828 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:22:51.151 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.135+0000 7f6c79e86740 1 -- 172.21.15.114:0/1851898693 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564c3ec865d0 con 0x564c3ed48a30 2024-09-13T18:22:51.151 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.135+0000 7f6c6ffff640 1 --2- 172.21.15.114:0/1851898693 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x564c3ed48a30 0x564c3ed4ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:51.151 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.136+0000 7f6c6ffff640 1 --2- 172.21.15.114:0/1851898693 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x564c3ed48a30 0x564c3ed4ae20 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37816197c057ba2c server_cookie=f26e764f1a7c6aa5 in_seq=0 out_seq=0 2024-09-13T18:22:51.151 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.136+0000 7f6c4ffff640 10 client.15828 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:22:51.151 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.136+0000 7f6c4ffff640 1 -- 172.21.15.114:0/1851898693 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x564c3ec865d0 con 0x564c3ed48a30 2024-09-13T18:22:51.151 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.136+0000 7f6c4ffff640 10 client.15828 handle_command_reply: tid=0 2024-09-13T18:22:51.151 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.136+0000 7f6c79e86740 1 client.15828 shutdown 2024-09-13T18:22:51.151 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.136+0000 7f6c79e86740 1 -- 172.21.15.114:0/1851898693 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564c3ed2e330 msgr2=0x564c3ed38f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:51.151 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.136+0000 7f6c79e86740 1 --2- 172.21.15.114:0/1851898693 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564c3ed2e330 0x564c3ed38f20 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f6c640960b0 tx=0x7f6c64099bb0 comp rx=0 tx=0).stop 2024-09-13T18:22:51.151 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.136+0000 7f6c79e86740 1 -- 172.21.15.114:0/1851898693 shutdown_connections 2024-09-13T18:22:51.151 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.136+0000 7f6c79e86740 1 --2- 172.21.15.114:0/1851898693 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x564c3ed48a30 0x564c3ed4ae20 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:51.152 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.136+0000 7f6c79e86740 1 --2- 172.21.15.114:0/1851898693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c3ed394f0 0x564c3ed369f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:51.152 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.137+0000 7f6c79e86740 1 --2- 172.21.15.114:0/1851898693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c3ed2ec50 0x564c3ed36260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:51.152 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.137+0000 7f6c79e86740 1 --2- 172.21.15.114:0/1851898693 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564c3ed2e330 0x564c3ed38f20 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:51.152 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.137+0000 7f6c4ffff640 0 client.15828 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:22:51.152 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.137+0000 7f6c79e86740 1 -- 172.21.15.114:0/1851898693 >> 172.21.15.114:0/1851898693 conn(0x564c3ed2a8c0 msgr2=0x564c3ed29b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:51.152 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.137+0000 7f6c79e86740 1 -- 172.21.15.114:0/1851898693 shutdown_connections 2024-09-13T18:22:51.152 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.137+0000 7f6c79e86740 1 -- 172.21.15.114:0/1851898693 wait complete. 2024-09-13T18:22:51.152 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.137+0000 7f6c79e86740 20 client.15828 trim_cache size 0 max 16384 2024-09-13T18:22:51.159 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.150+0000 7f6c79e86740 1 Processor -- start 2024-09-13T18:22:51.159 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.150+0000 7f6c79e86740 1 -- start start 2024-09-13T18:22:51.159 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.150+0000 7f6c79e86740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c3ed2e330 0x564c3ed389c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:51.159 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.150+0000 7f6c79e86740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564c3ed2ec50 0x564c3ed38f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:51.159 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.150+0000 7f6c79e86740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c3ed394f0 0x564c3ed25a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:51.159 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.150+0000 7f6c79e86740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c3ed34790 con 0x564c3ed2ec50 2024-09-13T18:22:51.159 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.150+0000 7f6c79e86740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c3ed3bc10 con 0x564c3ed394f0 2024-09-13T18:22:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.150+0000 7f6c79e86740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c3ed2b4c0 con 0x564c3ed2e330 2024-09-13T18:22:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.150+0000 7f6c6ffff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564c3ed2ec50 0x564c3ed38f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.150+0000 7f6c6ffff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564c3ed2ec50 0x564c3ed38f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44068/0 (socket says 172.21.15.114:44068) 2024-09-13T18:22:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.150+0000 7f6c6ffff640 1 -- 172.21.15.114:0/564242944 learned_addr learned my addr 172.21.15.114:0/564242944 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.150+0000 7f6c75624640 1 --2- 172.21.15.114:0/564242944 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c3ed394f0 0x564c3ed25a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.150+0000 7f6c74e23640 1 --2- 172.21.15.114:0/564242944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c3ed2e330 0x564c3ed389c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.150+0000 7f6c6ffff640 1 -- 172.21.15.114:0/564242944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c3ed2e330 msgr2=0x564c3ed389c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.150+0000 7f6c6ffff640 1 --2- 172.21.15.114:0/564242944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c3ed2e330 0x564c3ed389c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.150+0000 7f6c6ffff640 1 -- 172.21.15.114:0/564242944 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c3ed394f0 msgr2=0x564c3ed25a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.150+0000 7f6c6ffff640 1 --2- 172.21.15.114:0/564242944 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c3ed394f0 0x564c3ed25a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.150+0000 7f6c6ffff640 1 -- 172.21.15.114:0/564242944 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c3ec33300 con 0x564c3ed2ec50 2024-09-13T18:22:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.150+0000 7f6c75624640 1 --2- 172.21.15.114:0/564242944 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c3ed394f0 0x564c3ed25a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:22:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.150+0000 7f6c74e23640 1 --2- 172.21.15.114:0/564242944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c3ed2e330 0x564c3ed389c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:22:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.150+0000 7f6c6ffff640 1 --2- 172.21.15.114:0/564242944 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564c3ed2ec50 0x564c3ed38f00 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c0092a0 tx=0x7f6c5c005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b85876298e0c2ec7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.150+0000 7f6c5b7fe640 1 -- 172.21.15.114:0/564242944 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c041070 con 0x564c3ed2ec50 2024-09-13T18:22:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.150+0000 7f6c5b7fe640 1 -- 172.21.15.114:0/564242944 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6c5c045070 con 0x564c3ed2ec50 2024-09-13T18:22:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.150+0000 7f6c5b7fe640 1 -- 172.21.15.114:0/564242944 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c003ab0 con 0x564c3ed2ec50 2024-09-13T18:22:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.151+0000 7f6c79e86740 1 -- 172.21.15.114:0/564242944 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564c3ed2ec50 msgr2=0x564c3ed38f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.151+0000 7f6c79e86740 1 --2- 172.21.15.114:0/564242944 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564c3ed2ec50 0x564c3ed38f00 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c0092a0 tx=0x7f6c5c005e00 comp rx=0 tx=0).stop 2024-09-13T18:22:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.151+0000 7f6c79e86740 1 -- 172.21.15.114:0/564242944 shutdown_connections 2024-09-13T18:22:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.151+0000 7f6c79e86740 1 --2- 172.21.15.114:0/564242944 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c3ed394f0 0x564c3ed25a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.151+0000 7f6c79e86740 1 --2- 172.21.15.114:0/564242944 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564c3ed2ec50 0x564c3ed38f00 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.151+0000 7f6c79e86740 1 --2- 172.21.15.114:0/564242944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c3ed2e330 0x564c3ed389c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.151+0000 7f6c79e86740 1 -- 172.21.15.114:0/564242944 >> 172.21.15.114:0/564242944 conn(0x564c3ed2a8c0 msgr2=0x564c3ed34b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.151+0000 7f6c79e86740 1 -- 172.21.15.114:0/564242944 shutdown_connections 2024-09-13T18:22:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.151+0000 7f6c79e86740 1 -- 172.21.15.114:0/564242944 wait complete. 2024-09-13T18:22:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.151+0000 7f6c79e86740 1 -- start start 2024-09-13T18:22:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.151+0000 7f6c79e86740 1 Processor -- start 2024-09-13T18:22:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.151+0000 7f6c79e86740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564c3ed2e330 0x564c3ed406d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.151+0000 7f6c79e86740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c3ed2ec50 0x564c3ed3caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.151+0000 7f6c79e86740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c3ed394f0 0x564c3ed3d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.152+0000 7f6c79e86740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c3ed3d8e0 con 0x564c3ed2e330 2024-09-13T18:22:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.152+0000 7f6c79e86740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c3ed9eea0 con 0x564c3ed394f0 2024-09-13T18:22:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.152+0000 7f6c79e86740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c3ed9efe0 con 0x564c3ed2ec50 2024-09-13T18:22:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.152+0000 7f6c79e86740 10 client.0 _ll_register_callbacks cb 0x7f6c640ca130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:22:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.152+0000 7f6c74e23640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564c3ed2e330 0x564c3ed406d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.152+0000 7f6c74e23640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564c3ed2e330 0x564c3ed406d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44074/0 (socket says 172.21.15.114:44074) 2024-09-13T18:22:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.152+0000 7f6c74e23640 1 -- 172.21.15.114:0/706114622 learned_addr learned my addr 172.21.15.114:0/706114622 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:22:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.152+0000 7f6c6ffff640 1 --2- 172.21.15.114:0/706114622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c3ed2ec50 0x564c3ed3caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.152+0000 7f6c74e23640 1 -- 172.21.15.114:0/706114622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c3ed2ec50 msgr2=0x564c3ed3caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.152+0000 7f6c75624640 1 --2- 172.21.15.114:0/706114622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c3ed394f0 0x564c3ed3d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.152+0000 7f6c74e23640 1 --2- 172.21.15.114:0/706114622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c3ed2ec50 0x564c3ed3caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.152+0000 7f6c74e23640 1 -- 172.21.15.114:0/706114622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c3ed394f0 msgr2=0x564c3ed3d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.152+0000 7f6c74e23640 1 --2- 172.21.15.114:0/706114622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c3ed394f0 0x564c3ed3d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.152+0000 7f6c74e23640 1 -- 172.21.15.114:0/706114622 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c7010d3b0 con 0x564c3ed2e330 2024-09-13T18:22:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.152+0000 7f6c6ffff640 1 --2- 172.21.15.114:0/706114622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c3ed2ec50 0x564c3ed3caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:22:51.164 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.152+0000 7f6c74e23640 1 --2- 172.21.15.114:0/706114622 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564c3ed2e330 0x564c3ed406d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f6c640058c0 tx=0x7f6c64074320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d2330f18e386be8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:51.164 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.152+0000 7f6c4ffff640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:22:51.164 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.152+0000 7f6c4ffff640 1 -- 172.21.15.114:0/706114622 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c640cacb0 con 0x564c3ed2e330 2024-09-13T18:22:51.164 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.152+0000 7f6c4ffff640 1 -- 172.21.15.114:0/706114622 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f6c640a1db0 con 0x564c3ed2e330 2024-09-13T18:22:51.164 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.152+0000 7f6c79e86740 1 -- 172.21.15.114:0/706114622 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6c640046c0 con 0x564c3ed2e330 2024-09-13T18:22:51.164 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.152+0000 7f6c4ffff640 1 -- 172.21.15.114:0/706114622 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c64095ef0 con 0x564c3ed2e330 2024-09-13T18:22:51.164 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.152+0000 7f6c79e86740 10 client.15834 fetch_fsmap learned FSMap version 30 2024-09-13T18:22:51.164 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.152+0000 7f6c79e86740 1 -- 172.21.15.114:0/706114622 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x564c3ec588e0 con 0x564c3ed2e330 2024-09-13T18:22:51.164 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.153+0000 7f6c4ffff640 1 -- 172.21.15.114:0/706114622 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6c640046c0 con 0x564c3ed2e330 2024-09-13T18:22:51.164 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.153+0000 7f6c4ffff640 1 -- 172.21.15.114:0/706114622 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6c64016070 con 0x564c3ed2e330 2024-09-13T18:22:51.164 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.153+0000 7f6c79e86740 10 client.15834 fetch_fsmap finished waiting for FSMap version 30 2024-09-13T18:22:51.164 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.153+0000 7f6c79e86740 10 client.15834 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:22:51.165 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.153+0000 7f6c79e86740 20 client.15834 populate_metadata read hostname 'smithi114' 2024-09-13T18:22:51.165 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.153+0000 7f6c79e86740 1 --2- 172.21.15.114:0/706114622 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x564c3eda01b0 0x564c3eda25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:51.165 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.153+0000 7f6c79e86740 4 client.15834 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T18:22:51.165 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.153+0000 7f6c79e86740 1 -- 172.21.15.114:0/706114622 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564c3e986300 con 0x564c3eda01b0 2024-09-13T18:22:51.165 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.153+0000 7f6c6ffff640 1 --2- 172.21.15.114:0/706114622 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x564c3eda01b0 0x564c3eda25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:51.165 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.153+0000 7f6c4ffff640 10 client.15834 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:22:51.165 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.153+0000 7f6c6ffff640 1 --2- 172.21.15.114:0/706114622 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x564c3eda01b0 0x564c3eda25a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48341e9a8abb38d2 server_cookie=74b94daa905072c2 in_seq=0 out_seq=0 2024-09-13T18:22:51.169 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.165+0000 7f6c4ffff640 1 -- 172.21.15.114:0/706114622 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564c3e986300 con 0x564c3eda01b0 2024-09-13T18:22:51.169 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.165+0000 7f6c4ffff640 10 client.15834 handle_command_reply: tid=0 2024-09-13T18:22:51.169 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.165+0000 7f6c79e86740 1 client.15834 shutdown 2024-09-13T18:22:51.169 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.165+0000 7f6c79e86740 1 -- 172.21.15.114:0/706114622 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564c3ed2e330 msgr2=0x564c3ed406d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:51.169 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.165+0000 7f6c79e86740 1 --2- 172.21.15.114:0/706114622 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564c3ed2e330 0x564c3ed406d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f6c640058c0 tx=0x7f6c64074320 comp rx=0 tx=0).stop 2024-09-13T18:22:51.169 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.165+0000 7f6c79e86740 1 -- 172.21.15.114:0/706114622 shutdown_connections 2024-09-13T18:22:51.169 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.165+0000 7f6c79e86740 1 --2- 172.21.15.114:0/706114622 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x564c3eda01b0 0x564c3eda25a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:51.169 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.165+0000 7f6c79e86740 1 --2- 172.21.15.114:0/706114622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c3ed394f0 0x564c3ed3d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:51.169 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.165+0000 7f6c79e86740 1 --2- 172.21.15.114:0/706114622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c3ed2ec50 0x564c3ed3caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.165+0000 7f6c79e86740 1 --2- 172.21.15.114:0/706114622 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564c3ed2e330 0x564c3ed406d0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:51.171 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.165+0000 7f6c4ffff640 0 client.15834 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:22:51.171 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.165+0000 7f6c79e86740 1 -- 172.21.15.114:0/706114622 >> 172.21.15.114:0/706114622 conn(0x564c3ed2a8c0 msgr2=0x564c3ed34a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.165+0000 7f6c79e86740 1 -- 172.21.15.114:0/706114622 shutdown_connections 2024-09-13T18:22:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.165+0000 7f6c79e86740 1 -- 172.21.15.114:0/706114622 wait complete. 2024-09-13T18:22:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.166+0000 7f6c79e86740 20 client.15834 trim_cache size 0 max 16384 2024-09-13T18:22:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-13T18:22:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-13T18:22:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-13T18:22:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-13T18:22:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.166+0000 7f6c77087640 1 -- 172.21.15.114:0/196871743 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f6c600a1310 msgr2=0x7f6c600a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.166+0000 7f6c77087640 1 --2- 172.21.15.114:0/196871743 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f6c600a1310 0x7f6c600a37d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x564c3ed2bc40 tx=0x7f6c68008730 comp rx=0 tx=0).stop 2024-09-13T18:22:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.166+0000 7f6c77087640 1 -- 172.21.15.114:0/196871743 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c70100ee0 msgr2=0x7f6c70196140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.166+0000 7f6c77087640 1 --2- 172.21.15.114:0/196871743 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c70100ee0 0x7f6c70196140 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f6c6400c970 tx=0x7f6c6400ce30 comp rx=0 tx=0).stop 2024-09-13T18:22:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.166+0000 7f6c77087640 1 -- 172.21.15.114:0/196871743 shutdown_connections 2024-09-13T18:22:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.166+0000 7f6c77087640 1 --2- 172.21.15.114:0/196871743 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f6c600a1310 0x7f6c600a37d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.166+0000 7f6c77087640 1 --2- 172.21.15.114:0/196871743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c7010a270 0x7f6c7019aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.166+0000 7f6c77087640 1 --2- 172.21.15.114:0/196871743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c70101800 0x7f6c70196680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.166+0000 7f6c77087640 1 --2- 172.21.15.114:0/196871743 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c70100ee0 0x7f6c70196140 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.166+0000 7f6c77087640 1 -- 172.21.15.114:0/196871743 >> 172.21.15.114:0/196871743 conn(0x7f6c700fa890 msgr2=0x7f6c700fc330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.166+0000 7f6c77087640 1 -- 172.21.15.114:0/196871743 shutdown_connections 2024-09-13T18:22:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:22:51.166+0000 7f6c77087640 1 -- 172.21.15.114:0/196871743 wait complete. 2024-09-13T18:22:52.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:52 smithi204 ceph-mon[34777]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-13T18:22:52.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:52 smithi114 ceph-mon[30897]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-13T18:22:52.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:52 smithi134 ceph-mon[34283]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-13T18:22:54.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:54 smithi204 ceph-mon[34777]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-13T18:22:54.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:54 smithi114 ceph-mon[30897]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-13T18:22:54.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:54 smithi134 ceph-mon[34283]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-13T18:22:56.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:56 smithi204 ceph-mon[34777]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.2 KiB/s wr, 17 op/s 2024-09-13T18:22:56.469 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:56 smithi114 ceph-mon[30897]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.2 KiB/s wr, 17 op/s 2024-09-13T18:22:56.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:56 smithi134 ceph-mon[34283]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.2 KiB/s wr, 17 op/s 2024-09-13T18:22:58.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:22:58 smithi204 ceph-mon[34777]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.1 KiB/s wr, 13 op/s 2024-09-13T18:22:58.469 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:22:58 smithi114 ceph-mon[30897]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.1 KiB/s wr, 13 op/s 2024-09-13T18:22:58.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:22:58 smithi134 ceph-mon[34283]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.1 KiB/s wr, 13 op/s 2024-09-13T18:23:00.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:23:00 smithi204 ceph-mon[34777]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.1 KiB/s wr, 17 op/s 2024-09-13T18:23:00.469 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:23:00 smithi114 ceph-mon[30897]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.1 KiB/s wr, 17 op/s 2024-09-13T18:23:00.470 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:23:00 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:23:00.301+0000 7fd90abaf640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T18:23:00.470 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:23:00 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:23:00.301+0000 7fd90abaf640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T18:23:00.470 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:23:00 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:23:00.301+0000 7fd90abaf640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T18:23:00.470 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:23:00 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:23:00.301+0000 7fd90abaf640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T18:23:00.470 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:23:00 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x[31126]: 2024-09-13T18:23:00.301+0000 7fd90abaf640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T18:23:00.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:23:00 smithi134 ceph-mon[34283]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.1 KiB/s wr, 17 op/s 2024-09-13T18:23:02.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:23:02 smithi204 ceph-mon[34777]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-09-13T18:23:02.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:23:02 smithi204 ceph-mon[34777]: mgrmap e25: x(active, since 34s), standbys: y, z 2024-09-13T18:23:02.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:23:02 smithi114 ceph-mon[30897]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-09-13T18:23:02.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:23:02 smithi114 ceph-mon[30897]: mgrmap e25: x(active, since 34s), standbys: y, z 2024-09-13T18:23:02.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:23:02 smithi134 ceph-mon[34283]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-09-13T18:23:02.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:23:02 smithi134 ceph-mon[34283]: mgrmap e25: x(active, since 34s), standbys: y, z 2024-09-13T18:23:04.260 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.255+0000 7f1c374be640 1 -- 172.21.15.114:0/908939674 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c301008f0 msgr2=0x7f1c30104930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:04.260 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.255+0000 7f1c374be640 1 --2- 172.21.15.114:0/908939674 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c301008f0 0x7f1c30104930 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f1c20009930 tx=0x7f1c2002a420 comp rx=0 tx=0).stop 2024-09-13T18:23:04.260 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.255+0000 7f1c374be640 1 -- 172.21.15.114:0/908939674 shutdown_connections 2024-09-13T18:23:04.260 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.255+0000 7f1c374be640 1 --2- 172.21.15.114:0/908939674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c30105090 0x7f1c3010cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:04.260 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.255+0000 7f1c374be640 1 --2- 172.21.15.114:0/908939674 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c301008f0 0x7f1c30104930 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:04.260 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.255+0000 7f1c374be640 1 --2- 172.21.15.114:0/908939674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c300fff40 0x7f1c30100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:04.260 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.255+0000 7f1c374be640 1 -- 172.21.15.114:0/908939674 >> 172.21.15.114:0/908939674 conn(0x7f1c300fb9d0 msgr2=0x7f1c300fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:04.260 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.255+0000 7f1c374be640 1 -- 172.21.15.114:0/908939674 shutdown_connections 2024-09-13T18:23:04.260 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.255+0000 7f1c374be640 1 -- 172.21.15.114:0/908939674 wait complete. 2024-09-13T18:23:04.260 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.256+0000 7f1c374be640 1 Processor -- start 2024-09-13T18:23:04.261 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.256+0000 7f1c374be640 1 -- start start 2024-09-13T18:23:04.261 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.256+0000 7f1c374be640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c300fff40 0x7f1c300727b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:04.261 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.256+0000 7f1c374be640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c30105090 0x7f1c30072cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:04.261 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.256+0000 7f1c374be640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c30073230 0x7f1c301a43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:04.261 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.256+0000 7f1c374be640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c301a4990 con 0x7f1c30105090 2024-09-13T18:23:04.261 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.256+0000 7f1c374be640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c301a4b00 con 0x7f1c300fff40 2024-09-13T18:23:04.261 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.256+0000 7f1c374be640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c301a4cc0 con 0x7f1c30073230 2024-09-13T18:23:04.261 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.256+0000 7f1c34a59640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c30105090 0x7f1c30072cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:04.261 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.256+0000 7f1c34a59640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c30105090 0x7f1c30072cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36480/0 (socket says 172.21.15.114:36480) 2024-09-13T18:23:04.261 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.256+0000 7f1c34a59640 1 -- 172.21.15.114:0/3787356543 learned_addr learned my addr 172.21.15.114:0/3787356543 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:04.261 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.256+0000 7f1c35a5b640 1 --2- 172.21.15.114:0/3787356543 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c30073230 0x7f1c301a43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:04.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.256+0000 7f1c3525a640 1 --2- 172.21.15.114:0/3787356543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c300fff40 0x7f1c300727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:04.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.256+0000 7f1c34a59640 1 -- 172.21.15.114:0/3787356543 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c30073230 msgr2=0x7f1c301a43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:04.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.256+0000 7f1c34a59640 1 --2- 172.21.15.114:0/3787356543 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c30073230 0x7f1c301a43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:04.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.257+0000 7f1c34a59640 1 -- 172.21.15.114:0/3787356543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c300fff40 msgr2=0x7f1c300727b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:04.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.257+0000 7f1c34a59640 1 --2- 172.21.15.114:0/3787356543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c300fff40 0x7f1c300727b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:04.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.257+0000 7f1c34a59640 1 -- 172.21.15.114:0/3787356543 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c20009560 con 0x7f1c30105090 2024-09-13T18:23:04.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.257+0000 7f1c34a59640 1 --2- 172.21.15.114:0/3787356543 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c30105090 0x7f1c30072cf0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f1c20005ab0 tx=0x7f1c20032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:04.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.257+0000 7f1c267fc640 1 -- 172.21.15.114:0/3787356543 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c200044d0 con 0x7f1c30105090 2024-09-13T18:23:04.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.257+0000 7f1c267fc640 1 -- 172.21.15.114:0/3787356543 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1c20032e10 con 0x7f1c30105090 2024-09-13T18:23:04.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.257+0000 7f1c374be640 1 -- 172.21.15.114:0/3787356543 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c301a4f40 con 0x7f1c30105090 2024-09-13T18:23:04.263 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.257+0000 7f1c267fc640 1 -- 172.21.15.114:0/3787356543 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c20003850 con 0x7f1c30105090 2024-09-13T18:23:04.263 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.257+0000 7f1c374be640 1 -- 172.21.15.114:0/3787356543 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c301a5370 con 0x7f1c30105090 2024-09-13T18:23:04.270 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.260+0000 7f1c3a1e0740 1 Processor -- start 2024-09-13T18:23:04.270 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.260+0000 7f1c3a1e0740 1 -- start start 2024-09-13T18:23:04.271 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.260+0000 7f1c3a1e0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb34afd330 0x55fb34afd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:04.271 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.260+0000 7f1c3a1e0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb34afdc50 0x55fb34b07fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:04.271 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.260+0000 7f1c3a1e0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb34b084f0 0x55fb34b0a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:04.271 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.260+0000 7f1c3a1e0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb34a68c30 con 0x55fb34afd330 2024-09-13T18:23:04.271 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.260+0000 7f1c3a1e0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb34a663c0 con 0x55fb34afdc50 2024-09-13T18:23:04.271 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.260+0000 7f1c3a1e0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb34a61cb0 con 0x55fb34b084f0 2024-09-13T18:23:04.271 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.260+0000 7f1c267fc640 1 -- 172.21.15.114:0/3787356543 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) v1 ==== 136107+0+0 (secure 0 0 0) 0x7f1c20004850 con 0x7f1c30105090 2024-09-13T18:23:04.271 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.260+0000 7f1c35a5b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb34afd330 0x55fb34afd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:04.271 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.260+0000 7f1c35a5b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb34afd330 0x55fb34afd710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36490/0 (socket says 172.21.15.114:36490) 2024-09-13T18:23:04.271 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.260+0000 7f1c35a5b640 1 -- 172.21.15.114:0/2678224194 learned_addr learned my addr 172.21.15.114:0/2678224194 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:04.271 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.260+0000 7f1c3525a640 1 --2- 172.21.15.114:0/2678224194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb34afdc50 0x55fb34b07fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:04.271 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.260+0000 7f1c35a5b640 1 -- 172.21.15.114:0/2678224194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb34b084f0 msgr2=0x55fb34b0a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.260+0000 7f1c35a5b640 1 --2- 172.21.15.114:0/2678224194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb34b084f0 0x55fb34b0a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.260+0000 7f1c35a5b640 1 -- 172.21.15.114:0/2678224194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb34afdc50 msgr2=0x55fb34b07fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.260+0000 7f1c35a5b640 1 --2- 172.21.15.114:0/2678224194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb34afdc50 0x55fb34b07fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.260+0000 7f1c35a5b640 1 -- 172.21.15.114:0/2678224194 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb34a08910 con 0x55fb34afd330 2024-09-13T18:23:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.261+0000 7f1c35a5b640 1 --2- 172.21.15.114:0/2678224194 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb34afd330 0x55fb34afd710 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f1c28009e60 tx=0x7f1c28010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f82d1772d022c06 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.261+0000 7f1c1b7fe640 1 -- 172.21.15.114:0/2678224194 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c28004d20 con 0x55fb34afd330 2024-09-13T18:23:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.261+0000 7f1c1b7fe640 1 -- 172.21.15.114:0/2678224194 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1c28004e80 con 0x55fb34afd330 2024-09-13T18:23:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.261+0000 7f1c1b7fe640 1 -- 172.21.15.114:0/2678224194 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c280127d0 con 0x55fb34afd330 2024-09-13T18:23:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.261+0000 7f1c267fc640 1 --2- 172.21.15.114:0/3787356543 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f1c1c0a13d0 0x7f1c1c0a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.261+0000 7f1c3525a640 1 --2- 172.21.15.114:0/3787356543 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f1c1c0a13d0 0x7f1c1c0a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.261+0000 7f1c267fc640 1 -- 172.21.15.114:0/3787356543 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f1c200ede00 con 0x7f1c30105090 2024-09-13T18:23:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.262+0000 7f1c3a1e0740 1 -- 172.21.15.114:0/2678224194 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb34afd330 msgr2=0x55fb34afd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.262+0000 7f1c3525a640 1 --2- 172.21.15.114:0/3787356543 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f1c1c0a13d0 0x7f1c1c0a3890 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x55fb34af93c0 tx=0x7f1c2c005f90 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.262+0000 7f1c3a1e0740 1 --2- 172.21.15.114:0/2678224194 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb34afd330 0x55fb34afd710 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f1c28009e60 tx=0x7f1c28010740 comp rx=0 tx=0).stop 2024-09-13T18:23:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.262+0000 7f1c3a1e0740 1 -- 172.21.15.114:0/2678224194 shutdown_connections 2024-09-13T18:23:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.262+0000 7f1c3a1e0740 1 --2- 172.21.15.114:0/2678224194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb34b084f0 0x55fb34b0a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.262+0000 7f1c3a1e0740 1 --2- 172.21.15.114:0/2678224194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb34afdc50 0x55fb34b07fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.262+0000 7f1c3a1e0740 1 --2- 172.21.15.114:0/2678224194 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb34afd330 0x55fb34afd710 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.262+0000 7f1c3a1e0740 1 -- 172.21.15.114:0/2678224194 >> 172.21.15.114:0/2678224194 conn(0x55fb34af98c0 msgr2=0x55fb34af9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.262+0000 7f1c3a1e0740 1 -- 172.21.15.114:0/2678224194 shutdown_connections 2024-09-13T18:23:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.262+0000 7f1c3a1e0740 1 -- 172.21.15.114:0/2678224194 wait complete. 2024-09-13T18:23:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.262+0000 7f1c3a1e0740 1 -- start start 2024-09-13T18:23:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.262+0000 7f1c3a1e0740 1 Processor -- start 2024-09-13T18:23:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.263+0000 7f1c3a1e0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb34afd330 0x55fb34b07f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.263+0000 7f1c3a1e0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb34afdc50 0x55fb34b05260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.263+0000 7f1c3a1e0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb34b084f0 0x55fb34b059f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.263+0000 7f1c3a1e0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb34a1fe80 con 0x55fb34afd330 2024-09-13T18:23:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.263+0000 7f1c3a1e0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb34a1fb20 con 0x55fb34afdc50 2024-09-13T18:23:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.263+0000 7f1c3a1e0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb349e5260 con 0x55fb34b084f0 2024-09-13T18:23:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.263+0000 7f1c3a1e0740 10 client.0 _ll_register_callbacks cb 0x55fb34a1adf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:23:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.263+0000 7f1c3525a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb34afd330 0x55fb34b07f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.263+0000 7f1c3525a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb34afd330 0x55fb34b07f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36500/0 (socket says 172.21.15.114:36500) 2024-09-13T18:23:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.263+0000 7f1c3525a640 1 -- 172.21.15.114:0/597951574 learned_addr learned my addr 172.21.15.114:0/597951574 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:04.275 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.263+0000 7f1c34a59640 1 --2- 172.21.15.114:0/597951574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb34afdc50 0x55fb34b05260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:04.275 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.263+0000 7f1c35a5b640 1 --2- 172.21.15.114:0/597951574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb34b084f0 0x55fb34b059f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:04.275 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.263+0000 7f1c3525a640 1 -- 172.21.15.114:0/597951574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb34b084f0 msgr2=0x55fb34b059f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:04.275 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.263+0000 7f1c3525a640 1 --2- 172.21.15.114:0/597951574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb34b084f0 0x55fb34b059f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:04.275 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.263+0000 7f1c3525a640 1 -- 172.21.15.114:0/597951574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb34afdc50 msgr2=0x55fb34b05260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:04.275 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.263+0000 7f1c3525a640 1 --2- 172.21.15.114:0/597951574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb34afdc50 0x55fb34b05260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:04.275 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.263+0000 7f1c3525a640 1 -- 172.21.15.114:0/597951574 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c28009ac0 con 0x55fb34afd330 2024-09-13T18:23:04.275 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.263+0000 7f1c3525a640 1 --2- 172.21.15.114:0/597951574 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb34afd330 0x55fb34b07f40 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c0045f0 tx=0x7f1c2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d62fccc05fbd38a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:04.275 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.263+0000 7f1c18ff9640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:23:04.275 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.263+0000 7f1c18ff9640 1 -- 172.21.15.114:0/597951574 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c2c011920 con 0x55fb34afd330 2024-09-13T18:23:04.275 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.264+0000 7f1c18ff9640 1 -- 172.21.15.114:0/597951574 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f1c2c011a80 con 0x55fb34afd330 2024-09-13T18:23:04.275 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.264+0000 7f1c3a1e0740 1 -- 172.21.15.114:0/597951574 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fb34a555d0 con 0x55fb34afd330 2024-09-13T18:23:04.275 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.264+0000 7f1c18ff9640 1 -- 172.21.15.114:0/597951574 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c2c011c40 con 0x55fb34afd330 2024-09-13T18:23:04.276 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.264+0000 7f1c3a1e0740 10 client.15852 fetch_fsmap learned FSMap version 30 2024-09-13T18:23:04.276 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.264+0000 7f1c3a1e0740 1 -- 172.21.15.114:0/597951574 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55fb34a60330 con 0x55fb34afd330 2024-09-13T18:23:04.276 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.264+0000 7f1c18ff9640 1 -- 172.21.15.114:0/597951574 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55fb34a555d0 con 0x55fb34afd330 2024-09-13T18:23:04.276 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.264+0000 7f1c18ff9640 1 -- 172.21.15.114:0/597951574 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1c2c026020 con 0x55fb34afd330 2024-09-13T18:23:04.276 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.265+0000 7f1c3a1e0740 10 client.15852 fetch_fsmap finished waiting for FSMap version 30 2024-09-13T18:23:04.276 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.265+0000 7f1c3a1e0740 10 client.15852 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:23:04.276 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.265+0000 7f1c3a1e0740 20 client.15852 populate_metadata read hostname 'smithi114' 2024-09-13T18:23:04.276 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.265+0000 7f1c3a1e0740 1 --2- 172.21.15.114:0/597951574 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55fb34b17f40 0x55fb34b1a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:04.276 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.265+0000 7f1c3a1e0740 4 client.15852 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:23:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.265+0000 7f1c3a1e0740 1 -- 172.21.15.114:0/597951574 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fb34755300 con 0x55fb34b17f40 2024-09-13T18:23:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.265+0000 7f1c34a59640 1 --2- 172.21.15.114:0/597951574 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55fb34b17f40 0x55fb34b1a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.265+0000 7f1c18ff9640 10 client.15852 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:23:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.265+0000 7f1c34a59640 1 --2- 172.21.15.114:0/597951574 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55fb34b17f40 0x55fb34b1a330 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a299a64da09b347 server_cookie=ce2b2d1c3093c20f in_seq=0 out_seq=0 2024-09-13T18:23:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.266+0000 7f1c18ff9640 1 -- 172.21.15.114:0/597951574 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55fb34755300 con 0x55fb34b17f40 2024-09-13T18:23:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.266+0000 7f1c18ff9640 10 client.15852 handle_command_reply: tid=0 2024-09-13T18:23:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.266+0000 7f1c3a1e0740 1 client.15852 shutdown 2024-09-13T18:23:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.266+0000 7f1c3a1e0740 1 -- 172.21.15.114:0/597951574 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb34afd330 msgr2=0x55fb34b07f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.266+0000 7f1c3a1e0740 1 --2- 172.21.15.114:0/597951574 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb34afd330 0x55fb34b07f40 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c0045f0 tx=0x7f1c2c0046c0 comp rx=0 tx=0).stop 2024-09-13T18:23:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.266+0000 7f1c3a1e0740 1 -- 172.21.15.114:0/597951574 shutdown_connections 2024-09-13T18:23:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.266+0000 7f1c3a1e0740 1 --2- 172.21.15.114:0/597951574 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55fb34b17f40 0x55fb34b1a330 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.266+0000 7f1c3a1e0740 1 --2- 172.21.15.114:0/597951574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb34b084f0 0x55fb34b059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.266+0000 7f1c3a1e0740 1 --2- 172.21.15.114:0/597951574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb34afdc50 0x55fb34b05260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:04.278 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.266+0000 7f1c18ff9640 0 client.15852 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:23:04.278 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.266+0000 7f1c3a1e0740 1 --2- 172.21.15.114:0/597951574 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb34afd330 0x55fb34b07f40 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:04.278 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.266+0000 7f1c3a1e0740 1 -- 172.21.15.114:0/597951574 >> 172.21.15.114:0/597951574 conn(0x55fb34af98c0 msgr2=0x55fb34af8b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:04.278 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.266+0000 7f1c3a1e0740 1 -- 172.21.15.114:0/597951574 shutdown_connections 2024-09-13T18:23:04.278 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.266+0000 7f1c3a1e0740 1 -- 172.21.15.114:0/597951574 wait complete. 2024-09-13T18:23:04.278 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.267+0000 7f1c3a1e0740 20 client.15852 trim_cache size 0 max 16384 2024-09-13T18:23:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.279+0000 7f1c3a1e0740 1 Processor -- start 2024-09-13T18:23:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.279+0000 7f1c3a1e0740 1 -- start start 2024-09-13T18:23:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.279+0000 7f1c3a1e0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb34afd330 0x55fb34af5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.279+0000 7f1c3a1e0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb34afdc50 0x55fb34b07aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.279+0000 7f1c3a1e0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb34b084f0 0x55fb34b0f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.279+0000 7f1c3a1e0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb34b05d70 con 0x55fb34b084f0 2024-09-13T18:23:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.279+0000 7f1c3a1e0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb34b00d10 con 0x55fb34afdc50 2024-09-13T18:23:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.279+0000 7f1c3a1e0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb34b00e80 con 0x55fb34afd330 2024-09-13T18:23:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.279+0000 7f1c35a5b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb34b084f0 0x55fb34b0f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.279+0000 7f1c35a5b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb34b084f0 0x55fb34b0f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36510/0 (socket says 172.21.15.114:36510) 2024-09-13T18:23:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.279+0000 7f1c35a5b640 1 -- 172.21.15.114:0/3857282474 learned_addr learned my addr 172.21.15.114:0/3857282474 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.279+0000 7f1c3525a640 1 --2- 172.21.15.114:0/3857282474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb34afd330 0x55fb34af5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.279+0000 7f1c34a59640 1 --2- 172.21.15.114:0/3857282474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb34afdc50 0x55fb34b07aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.279+0000 7f1c35a5b640 1 -- 172.21.15.114:0/3857282474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb34afd330 msgr2=0x55fb34af5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.279+0000 7f1c35a5b640 1 --2- 172.21.15.114:0/3857282474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb34afd330 0x55fb34af5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.279+0000 7f1c35a5b640 1 -- 172.21.15.114:0/3857282474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb34afdc50 msgr2=0x55fb34b07aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.279+0000 7f1c35a5b640 1 --2- 172.21.15.114:0/3857282474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb34afdc50 0x55fb34b07aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.279+0000 7f1c35a5b640 1 -- 172.21.15.114:0/3857282474 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb34a02300 con 0x55fb34b084f0 2024-09-13T18:23:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.279+0000 7f1c3525a640 1 --2- 172.21.15.114:0/3857282474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb34afd330 0x55fb34af5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:23:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.279+0000 7f1c34a59640 1 --2- 172.21.15.114:0/3857282474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb34afdc50 0x55fb34b07aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:23:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.279+0000 7f1c35a5b640 1 --2- 172.21.15.114:0/3857282474 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb34b084f0 0x55fb34b0f1b0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f1c28009e60 tx=0x7f1c28010fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb8c0984df5c9d03 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.280+0000 7f1c1bfff640 1 -- 172.21.15.114:0/3857282474 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c28005200 con 0x55fb34b084f0 2024-09-13T18:23:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.280+0000 7f1c1bfff640 1 -- 172.21.15.114:0/3857282474 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1c28004c90 con 0x55fb34b084f0 2024-09-13T18:23:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.280+0000 7f1c1bfff640 1 -- 172.21.15.114:0/3857282474 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c28012ba0 con 0x55fb34b084f0 2024-09-13T18:23:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.280+0000 7f1c3a1e0740 1 -- 172.21.15.114:0/3857282474 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb34b084f0 msgr2=0x55fb34b0f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.280+0000 7f1c3a1e0740 1 --2- 172.21.15.114:0/3857282474 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb34b084f0 0x55fb34b0f1b0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f1c28009e60 tx=0x7f1c28010fd0 comp rx=0 tx=0).stop 2024-09-13T18:23:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.280+0000 7f1c3a1e0740 1 -- 172.21.15.114:0/3857282474 shutdown_connections 2024-09-13T18:23:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.280+0000 7f1c3a1e0740 1 --2- 172.21.15.114:0/3857282474 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb34b084f0 0x55fb34b0f1b0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.280+0000 7f1c3a1e0740 1 --2- 172.21.15.114:0/3857282474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb34afdc50 0x55fb34b07aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.280+0000 7f1c3a1e0740 1 --2- 172.21.15.114:0/3857282474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb34afd330 0x55fb34af5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.280+0000 7f1c3a1e0740 1 -- 172.21.15.114:0/3857282474 >> 172.21.15.114:0/3857282474 conn(0x55fb34af98c0 msgr2=0x55fb34af4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.280+0000 7f1c3a1e0740 1 -- 172.21.15.114:0/3857282474 shutdown_connections 2024-09-13T18:23:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.280+0000 7f1c3a1e0740 1 -- 172.21.15.114:0/3857282474 wait complete. 2024-09-13T18:23:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.281+0000 7f1c3a1e0740 1 -- start start 2024-09-13T18:23:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.281+0000 7f1c3a1e0740 1 Processor -- start 2024-09-13T18:23:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.281+0000 7f1c3a1e0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb34afd330 0x55fb34b0bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.281+0000 7f1c3a1e0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb34afdc50 0x55fb34b0c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.281+0000 7f1c3a1e0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb34b084f0 0x55fb34b0c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:04.289 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.281+0000 7f1c3a1e0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb34b6e570 con 0x55fb34b084f0 2024-09-13T18:23:04.289 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.281+0000 7f1c3a1e0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb34b6e6e0 con 0x55fb34afd330 2024-09-13T18:23:04.289 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.281+0000 7f1c3a1e0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb34b6e8a0 con 0x55fb34afdc50 2024-09-13T18:23:04.289 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.281+0000 7f1c3a1e0740 10 client.0 _ll_register_callbacks cb 0x7f1c28009600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:23:04.289 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.281+0000 7f1c35a5b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb34b084f0 0x55fb34b0c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:04.289 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.281+0000 7f1c35a5b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb34b084f0 0x55fb34b0c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36526/0 (socket says 172.21.15.114:36526) 2024-09-13T18:23:04.289 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.281+0000 7f1c35a5b640 1 -- 172.21.15.114:0/113417706 learned_addr learned my addr 172.21.15.114:0/113417706 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:04.289 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.281+0000 7f1c3525a640 1 --2- 172.21.15.114:0/113417706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb34afd330 0x55fb34b0bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:04.289 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.282+0000 7f1c34a59640 1 --2- 172.21.15.114:0/113417706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb34afdc50 0x55fb34b0c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:04.289 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.282+0000 7f1c35a5b640 1 -- 172.21.15.114:0/113417706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb34afdc50 msgr2=0x55fb34b0c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:04.289 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.282+0000 7f1c35a5b640 1 --2- 172.21.15.114:0/113417706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb34afdc50 0x55fb34b0c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:04.289 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.282+0000 7f1c35a5b640 1 -- 172.21.15.114:0/113417706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb34afd330 msgr2=0x55fb34b0bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:04.289 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.282+0000 7f1c35a5b640 1 --2- 172.21.15.114:0/113417706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb34afd330 0x55fb34b0bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.282+0000 7f1c35a5b640 1 -- 172.21.15.114:0/113417706 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fb34a08910 con 0x55fb34b084f0 2024-09-13T18:23:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.282+0000 7f1c3525a640 1 --2- 172.21.15.114:0/113417706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb34afd330 0x55fb34b0bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:23:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.282+0000 7f1c35a5b640 1 --2- 172.21.15.114:0/113417706 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb34b084f0 0x55fb34b0c990 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f1c28010c40 tx=0x7f1c28012a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3ccca3724b054b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.282+0000 7f1c18ff9640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:23:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.282+0000 7f1c18ff9640 1 -- 172.21.15.114:0/113417706 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c28005200 con 0x55fb34b084f0 2024-09-13T18:23:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.282+0000 7f1c18ff9640 1 -- 172.21.15.114:0/113417706 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f1c28020070 con 0x55fb34b084f0 2024-09-13T18:23:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.282+0000 7f1c3a1e0740 1 -- 172.21.15.114:0/113417706 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fb34a8a680 con 0x55fb34b084f0 2024-09-13T18:23:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.282+0000 7f1c18ff9640 1 -- 172.21.15.114:0/113417706 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c28012e10 con 0x55fb34b084f0 2024-09-13T18:23:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.282+0000 7f1c3a1e0740 10 client.15858 fetch_fsmap learned FSMap version 30 2024-09-13T18:23:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.282+0000 7f1c3a1e0740 1 -- 172.21.15.114:0/113417706 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55fb34a278e0 con 0x55fb34b084f0 2024-09-13T18:23:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.282+0000 7f1c18ff9640 1 -- 172.21.15.114:0/113417706 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55fb34a8a680 con 0x55fb34b084f0 2024-09-13T18:23:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.283+0000 7f1c18ff9640 1 -- 172.21.15.114:0/113417706 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1c2802a070 con 0x55fb34b084f0 2024-09-13T18:23:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.283+0000 7f1c3a1e0740 10 client.15858 fetch_fsmap finished waiting for FSMap version 30 2024-09-13T18:23:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.283+0000 7f1c3a1e0740 10 client.15858 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:23:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.283+0000 7f1c3a1e0740 20 client.15858 populate_metadata read hostname 'smithi114' 2024-09-13T18:23:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.283+0000 7f1c3a1e0740 1 --2- 172.21.15.114:0/113417706 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55fb34b6fb40 0x55fb34b71f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.283+0000 7f1c3a1e0740 4 client.15858 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T18:23:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.283+0000 7f1c3a1e0740 1 -- 172.21.15.114:0/113417706 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fb34766600 con 0x55fb34b6fb40 2024-09-13T18:23:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.283+0000 7f1c3525a640 1 --2- 172.21.15.114:0/113417706 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55fb34b6fb40 0x55fb34b71f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.283+0000 7f1c18ff9640 10 client.15858 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:23:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.283+0000 7f1c3525a640 1 --2- 172.21.15.114:0/113417706 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55fb34b6fb40 0x55fb34b71f30 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cded55eb7995bc4b server_cookie=34aa6f4a703a534 in_seq=0 out_seq=0 2024-09-13T18:23:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.289+0000 7f1c18ff9640 1 -- 172.21.15.114:0/113417706 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fb34766600 con 0x55fb34b6fb40 2024-09-13T18:23:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.289+0000 7f1c18ff9640 10 client.15858 handle_command_reply: tid=0 2024-09-13T18:23:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.289+0000 7f1c3a1e0740 1 client.15858 shutdown 2024-09-13T18:23:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.289+0000 7f1c3a1e0740 1 -- 172.21.15.114:0/113417706 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb34b084f0 msgr2=0x55fb34b0c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.289+0000 7f1c3a1e0740 1 --2- 172.21.15.114:0/113417706 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb34b084f0 0x55fb34b0c990 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f1c28010c40 tx=0x7f1c28012a10 comp rx=0 tx=0).stop 2024-09-13T18:23:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.289+0000 7f1c3a1e0740 1 -- 172.21.15.114:0/113417706 shutdown_connections 2024-09-13T18:23:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.289+0000 7f1c3a1e0740 1 --2- 172.21.15.114:0/113417706 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55fb34b6fb40 0x55fb34b71f30 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.289+0000 7f1c3a1e0740 1 --2- 172.21.15.114:0/113417706 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb34b084f0 0x55fb34b0c990 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.289+0000 7f1c18ff9640 0 client.15858 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:23:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.289+0000 7f1c3a1e0740 1 --2- 172.21.15.114:0/113417706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb34afdc50 0x55fb34b0c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.289+0000 7f1c3a1e0740 1 --2- 172.21.15.114:0/113417706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb34afd330 0x55fb34b0bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.289+0000 7f1c3a1e0740 1 -- 172.21.15.114:0/113417706 >> 172.21.15.114:0/113417706 conn(0x55fb34af98c0 msgr2=0x55fb34af82f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.290+0000 7f1c3a1e0740 1 -- 172.21.15.114:0/113417706 shutdown_connections 2024-09-13T18:23:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.290+0000 7f1c3a1e0740 1 -- 172.21.15.114:0/113417706 wait complete. 2024-09-13T18:23:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.290+0000 7f1c3a1e0740 20 client.15858 trim_cache size 0 max 16384 2024-09-13T18:23:04.294 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-13T18:23:04.294 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-13T18:23:04.294 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-13T18:23:04.294 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-13T18:23:04.295 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.290+0000 7f1c374be640 1 -- 172.21.15.114:0/3787356543 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f1c1c0a13d0 msgr2=0x7f1c1c0a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:04.295 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.290+0000 7f1c374be640 1 --2- 172.21.15.114:0/3787356543 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f1c1c0a13d0 0x7f1c1c0a3890 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x55fb34af93c0 tx=0x7f1c2c005f90 comp rx=0 tx=0).stop 2024-09-13T18:23:04.295 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.290+0000 7f1c374be640 1 -- 172.21.15.114:0/3787356543 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c30105090 msgr2=0x7f1c30072cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:04.295 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.290+0000 7f1c374be640 1 --2- 172.21.15.114:0/3787356543 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c30105090 0x7f1c30072cf0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f1c20005ab0 tx=0x7f1c20032c20 comp rx=0 tx=0).stop 2024-09-13T18:23:04.295 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.290+0000 7f1c374be640 1 -- 172.21.15.114:0/3787356543 shutdown_connections 2024-09-13T18:23:04.295 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.290+0000 7f1c374be640 1 --2- 172.21.15.114:0/3787356543 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f1c1c0a13d0 0x7f1c1c0a3890 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:04.295 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.290+0000 7f1c374be640 1 --2- 172.21.15.114:0/3787356543 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c30073230 0x7f1c301a43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.290+0000 7f1c374be640 1 --2- 172.21.15.114:0/3787356543 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c30105090 0x7f1c30072cf0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.290+0000 7f1c374be640 1 --2- 172.21.15.114:0/3787356543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c300fff40 0x7f1c300727b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.290+0000 7f1c374be640 1 -- 172.21.15.114:0/3787356543 >> 172.21.15.114:0/3787356543 conn(0x7f1c300fb9d0 msgr2=0x7f1c300fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.290+0000 7f1c374be640 1 -- 172.21.15.114:0/3787356543 shutdown_connections 2024-09-13T18:23:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:04.290+0000 7f1c374be640 1 -- 172.21.15.114:0/3787356543 wait complete. 2024-09-13T18:23:04.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:23:04 smithi204 ceph-mon[34777]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2024-09-13T18:23:04.469 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:23:04 smithi114 ceph-mon[30897]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2024-09-13T18:23:04.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:23:04 smithi134 ceph-mon[34283]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2024-09-13T18:23:05.941 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T18:23:06.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.051+0000 7fe675f53640 1 -- 172.21.15.114:0/3473767000 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe6700ff810 msgr2=0x7fe670103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:06.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.051+0000 7fe675f53640 1 --2- 172.21.15.114:0/3473767000 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe6700ff810 0x7fe670103850 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fe658009930 tx=0x7fe65802a420 comp rx=0 tx=0).stop 2024-09-13T18:23:06.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.051+0000 7fe675f53640 1 -- 172.21.15.114:0/3473767000 shutdown_connections 2024-09-13T18:23:06.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.051+0000 7fe675f53640 1 --2- 172.21.15.114:0/3473767000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe670103fb0 0x7fe67010bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.051+0000 7fe675f53640 1 --2- 172.21.15.114:0/3473767000 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe6700ff810 0x7fe670103850 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.051+0000 7fe675f53640 1 --2- 172.21.15.114:0/3473767000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe6700fee60 0x7fe6700ff240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.051+0000 7fe675f53640 1 -- 172.21.15.114:0/3473767000 >> 172.21.15.114:0/3473767000 conn(0x7fe670075c60 msgr2=0x7fe6700fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:06.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.051+0000 7fe675f53640 1 -- 172.21.15.114:0/3473767000 shutdown_connections 2024-09-13T18:23:06.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.051+0000 7fe675f53640 1 -- 172.21.15.114:0/3473767000 wait complete. 2024-09-13T18:23:06.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.051+0000 7fe675f53640 1 Processor -- start 2024-09-13T18:23:06.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.051+0000 7fe675f53640 1 -- start start 2024-09-13T18:23:06.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.052+0000 7fe675f53640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe6700fee60 0x7fe670197060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.052+0000 7fe675f53640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe670103fb0 0x7fe6701975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.052+0000 7fe675f53640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe670197ae0 0x7fe6701a20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.052+0000 7fe675f53640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6701a2780 con 0x7fe670197ae0 2024-09-13T18:23:06.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.052+0000 7fe675f53640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6701a28f0 con 0x7fe670103fb0 2024-09-13T18:23:06.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.052+0000 7fe675f53640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6701a2ab0 con 0x7fe6700fee60 2024-09-13T18:23:06.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.052+0000 7fe66ffff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe670197ae0 0x7fe6701a20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.052+0000 7fe66ffff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe670197ae0 0x7fe6701a20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36536/0 (socket says 172.21.15.114:36536) 2024-09-13T18:23:06.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.052+0000 7fe66ffff640 1 -- 172.21.15.114:0/473802352 learned_addr learned my addr 172.21.15.114:0/473802352 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:06.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.052+0000 7fe66f7fe640 1 --2- 172.21.15.114:0/473802352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe6700fee60 0x7fe670197060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.052+0000 7fe667fff640 1 --2- 172.21.15.114:0/473802352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe670103fb0 0x7fe6701975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.052+0000 7fe66ffff640 1 -- 172.21.15.114:0/473802352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe6700fee60 msgr2=0x7fe670197060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:06.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.052+0000 7fe66ffff640 1 --2- 172.21.15.114:0/473802352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe6700fee60 0x7fe670197060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.052+0000 7fe66ffff640 1 -- 172.21.15.114:0/473802352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe670103fb0 msgr2=0x7fe6701975a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:06.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.052+0000 7fe66ffff640 1 --2- 172.21.15.114:0/473802352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe670103fb0 0x7fe6701975a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.052+0000 7fe66ffff640 1 -- 172.21.15.114:0/473802352 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe658009560 con 0x7fe670197ae0 2024-09-13T18:23:06.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.052+0000 7fe66ffff640 1 --2- 172.21.15.114:0/473802352 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe670197ae0 0x7fe6701a20e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fe66800e960 tx=0x7fe66800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:06.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.052+0000 7fe66d7fa640 1 -- 172.21.15.114:0/473802352 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe668004d20 con 0x7fe670197ae0 2024-09-13T18:23:06.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.052+0000 7fe66d7fa640 1 -- 172.21.15.114:0/473802352 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe668004e80 con 0x7fe670197ae0 2024-09-13T18:23:06.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.052+0000 7fe675f53640 1 -- 172.21.15.114:0/473802352 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6701a2d90 con 0x7fe670197ae0 2024-09-13T18:23:06.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.053+0000 7fe66d7fa640 1 -- 172.21.15.114:0/473802352 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe668011470 con 0x7fe670197ae0 2024-09-13T18:23:06.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.053+0000 7fe675f53640 1 -- 172.21.15.114:0/473802352 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6701a3220 con 0x7fe670197ae0 2024-09-13T18:23:06.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.055+0000 7fe66d7fa640 1 -- 172.21.15.114:0/473802352 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) v1 ==== 136107+0+0 (secure 0 0 0) 0x7fe66800cb20 con 0x7fe670197ae0 2024-09-13T18:23:06.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.055+0000 7fe678c71740 1 Processor -- start 2024-09-13T18:23:06.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.055+0000 7fe678c71740 1 -- start start 2024-09-13T18:23:06.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.055+0000 7fe678c71740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a5708a0e10 0x55a5708a11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.055+0000 7fe678c71740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5708a1730 0x55a5708aba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.055+0000 7fe678c71740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5708abfd0 0x55a5708ae3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.055+0000 7fe678c71740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a57080bf20 con 0x55a5708a0e10 2024-09-13T18:23:06.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.055+0000 7fe678c71740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5708096b0 con 0x55a5708a1730 2024-09-13T18:23:06.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.055+0000 7fe678c71740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a570804fa0 con 0x55a5708abfd0 2024-09-13T18:23:06.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.056+0000 7fe667fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a5708a0e10 0x55a5708a11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.056+0000 7fe667fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a5708a0e10 0x55a5708a11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36542/0 (socket says 172.21.15.114:36542) 2024-09-13T18:23:06.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.056+0000 7fe667fff640 1 -- 172.21.15.114:0/2069381693 learned_addr learned my addr 172.21.15.114:0/2069381693 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:06.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.056+0000 7fe66f7fe640 1 --2- 172.21.15.114:0/2069381693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5708a1730 0x55a5708aba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.056+0000 7fe66d7fa640 1 --2- 172.21.15.114:0/473802352 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe65c0a1320 0x7fe65c0a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.056+0000 7fe667fff640 1 --2- 172.21.15.114:0/2069381693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5708abfd0 0x55a5708ae3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.056+0000 7fe66d7fa640 1 -- 172.21.15.114:0/473802352 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fe6680cd7c0 con 0x7fe670197ae0 2024-09-13T18:23:06.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.056+0000 7fe66ffff640 1 --2- 172.21.15.114:0/473802352 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe65c0a1320 0x7fe65c0a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.056+0000 7fe667fff640 1 -- 172.21.15.114:0/2069381693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5708abfd0 msgr2=0x55a5708ae3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:06.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.056+0000 7fe667fff640 1 --2- 172.21.15.114:0/2069381693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5708abfd0 0x55a5708ae3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.056+0000 7fe667fff640 1 -- 172.21.15.114:0/2069381693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5708a1730 msgr2=0x55a5708aba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:06.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.056+0000 7fe667fff640 1 --2- 172.21.15.114:0/2069381693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5708a1730 0x55a5708aba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.056+0000 7fe667fff640 1 -- 172.21.15.114:0/2069381693 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5707bf840 con 0x55a5708a0e10 2024-09-13T18:23:06.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.056+0000 7fe66f7fe640 1 --2- 172.21.15.114:0/2069381693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5708a1730 0x55a5708aba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:23:06.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.056+0000 7fe667fff640 1 --2- 172.21.15.114:0/2069381693 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a5708a0e10 0x55a5708a11f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fe658009930 tx=0x7fe658005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cc73588c1faf3c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:06.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.056+0000 7fe66ffff640 1 --2- 172.21.15.114:0/473802352 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe65c0a1320 0x7fe65c0a37e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe6680caaa0 tx=0x7fe66800ef50 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:06.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.057+0000 7fe665ffb640 1 -- 172.21.15.114:0/2069381693 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe658044070 con 0x55a5708a0e10 2024-09-13T18:23:06.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.057+0000 7fe665ffb640 1 -- 172.21.15.114:0/2069381693 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe658003a80 con 0x55a5708a0e10 2024-09-13T18:23:06.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.057+0000 7fe665ffb640 1 -- 172.21.15.114:0/2069381693 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe658031980 con 0x55a5708a0e10 2024-09-13T18:23:06.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.057+0000 7fe678c71740 1 -- 172.21.15.114:0/2069381693 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a5708a0e10 msgr2=0x55a5708a11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:06.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.057+0000 7fe678c71740 1 --2- 172.21.15.114:0/2069381693 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a5708a0e10 0x55a5708a11f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fe658009930 tx=0x7fe658005b50 comp rx=0 tx=0).stop 2024-09-13T18:23:06.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.057+0000 7fe678c71740 1 -- 172.21.15.114:0/2069381693 shutdown_connections 2024-09-13T18:23:06.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.057+0000 7fe678c71740 1 --2- 172.21.15.114:0/2069381693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5708abfd0 0x55a5708ae3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.057+0000 7fe678c71740 1 --2- 172.21.15.114:0/2069381693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5708a1730 0x55a5708aba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.057+0000 7fe678c71740 1 --2- 172.21.15.114:0/2069381693 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a5708a0e10 0x55a5708a11f0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.057+0000 7fe678c71740 1 -- 172.21.15.114:0/2069381693 >> 172.21.15.114:0/2069381693 conn(0x55a57089d380 msgr2=0x55a57089d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:06.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.057+0000 7fe678c71740 1 -- 172.21.15.114:0/2069381693 shutdown_connections 2024-09-13T18:23:06.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.057+0000 7fe678c71740 1 -- 172.21.15.114:0/2069381693 wait complete. 2024-09-13T18:23:06.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.058+0000 7fe678c71740 1 -- start start 2024-09-13T18:23:06.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.058+0000 7fe678c71740 1 Processor -- start 2024-09-13T18:23:06.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.058+0000 7fe678c71740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5708a0e10 0x55a5708ab9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.058+0000 7fe678c71740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a5708a1730 0x55a5708a8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.058+0000 7fe678c71740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5708abfd0 0x55a5708a9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.058+0000 7fe678c71740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5707a62a0 con 0x55a5708a1730 2024-09-13T18:23:06.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.058+0000 7fe678c71740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5704dd430 con 0x55a5708a0e10 2024-09-13T18:23:06.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.058+0000 7fe678c71740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a570788680 con 0x55a5708abfd0 2024-09-13T18:23:06.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.058+0000 7fe678c71740 10 client.0 _ll_register_callbacks cb 0x55a5707fe610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:23:06.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.058+0000 7fe66f7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a5708a1730 0x55a5708a8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.058+0000 7fe66f7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a5708a1730 0x55a5708a8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36552/0 (socket says 172.21.15.114:36552) 2024-09-13T18:23:06.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.058+0000 7fe66f7fe640 1 -- 172.21.15.114:0/3472378779 learned_addr learned my addr 172.21.15.114:0/3472378779 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:06.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.058+0000 7fe667fff640 1 --2- 172.21.15.114:0/3472378779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5708a0e10 0x55a5708ab9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.059+0000 7fe66f7fe640 1 -- 172.21.15.114:0/3472378779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5708abfd0 msgr2=0x55a5708a9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:06.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.059+0000 7fe66f7fe640 1 --2- 172.21.15.114:0/3472378779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5708abfd0 0x55a5708a9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.059+0000 7fe66f7fe640 1 -- 172.21.15.114:0/3472378779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5708a0e10 msgr2=0x55a5708ab9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:06.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.059+0000 7fe66f7fe640 1 --2- 172.21.15.114:0/3472378779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5708a0e10 0x55a5708ab9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.059+0000 7fe66f7fe640 1 -- 172.21.15.114:0/3472378779 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe67010c340 con 0x55a5708a1730 2024-09-13T18:23:06.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.059+0000 7fe667fff640 1 --2- 172.21.15.114:0/3472378779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5708a0e10 0x55a5708ab9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:23:06.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.059+0000 7fe66f7fe640 1 --2- 172.21.15.114:0/3472378779 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a5708a1730 0x55a5708a8c40 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fe66000baa0 tx=0x7fe66000bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d64192b97d0cc262 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:06.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.059+0000 7fe64effd640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:23:06.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.059+0000 7fe64effd640 1 -- 172.21.15.114:0/3472378779 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe660016070 con 0x55a5708a1730 2024-09-13T18:23:06.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.059+0000 7fe64effd640 1 -- 172.21.15.114:0/3472378779 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fe660011440 con 0x55a5708a1730 2024-09-13T18:23:06.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.059+0000 7fe678c71740 1 -- 172.21.15.114:0/3472378779 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a5704f84e0 con 0x55a5708a1730 2024-09-13T18:23:06.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.059+0000 7fe64effd640 1 -- 172.21.15.114:0/3472378779 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe660011600 con 0x55a5708a1730 2024-09-13T18:23:06.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.059+0000 7fe678c71740 10 client.15870 fetch_fsmap learned FSMap version 30 2024-09-13T18:23:06.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.059+0000 7fe678c71740 1 -- 172.21.15.114:0/3472378779 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55a570803620 con 0x55a5708a1730 2024-09-13T18:23:06.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.059+0000 7fe64effd640 1 -- 172.21.15.114:0/3472378779 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a5704f84e0 con 0x55a5708a1730 2024-09-13T18:23:06.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.059+0000 7fe64effd640 1 -- 172.21.15.114:0/3472378779 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe66001b6b0 con 0x55a5708a1730 2024-09-13T18:23:06.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.060+0000 7fe678c71740 10 client.15870 fetch_fsmap finished waiting for FSMap version 30 2024-09-13T18:23:06.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.060+0000 7fe678c71740 10 client.15870 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:23:06.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.060+0000 7fe678c71740 20 client.15870 populate_metadata read hostname 'smithi114' 2024-09-13T18:23:06.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.060+0000 7fe678c71740 1 --2- 172.21.15.114:0/3472378779 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55a5708bbea0 0x55a5708be2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.060+0000 7fe678c71740 4 client.15870 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:23:06.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.060+0000 7fe678c71740 1 -- 172.21.15.114:0/3472378779 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a5705097e0 con 0x55a5708bbea0 2024-09-13T18:23:06.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.060+0000 7fe667fff640 1 --2- 172.21.15.114:0/3472378779 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55a5708bbea0 0x55a5708be2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.060+0000 7fe64effd640 10 client.15870 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:23:06.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.060+0000 7fe667fff640 1 --2- 172.21.15.114:0/3472378779 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55a5708bbea0 0x55a5708be2c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67ddf75c9bbf8906 server_cookie=ae13d80bf94d5738 in_seq=0 out_seq=0 2024-09-13T18:23:06.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.061+0000 7fe64effd640 1 -- 172.21.15.114:0/3472378779 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55a5705097e0 con 0x55a5708bbea0 2024-09-13T18:23:06.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.061+0000 7fe64effd640 10 client.15870 handle_command_reply: tid=0 2024-09-13T18:23:06.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.061+0000 7fe678c71740 1 client.15870 shutdown 2024-09-13T18:23:06.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.061+0000 7fe678c71740 1 -- 172.21.15.114:0/3472378779 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a5708a1730 msgr2=0x55a5708a8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:06.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.061+0000 7fe678c71740 1 --2- 172.21.15.114:0/3472378779 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a5708a1730 0x55a5708a8c40 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fe66000baa0 tx=0x7fe66000bf60 comp rx=0 tx=0).stop 2024-09-13T18:23:06.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.061+0000 7fe678c71740 1 -- 172.21.15.114:0/3472378779 shutdown_connections 2024-09-13T18:23:06.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.061+0000 7fe678c71740 1 --2- 172.21.15.114:0/3472378779 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55a5708bbea0 0x55a5708be2c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.061+0000 7fe678c71740 1 --2- 172.21.15.114:0/3472378779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5708abfd0 0x55a5708a9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.061+0000 7fe678c71740 1 --2- 172.21.15.114:0/3472378779 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a5708a1730 0x55a5708a8c40 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.061+0000 7fe678c71740 1 --2- 172.21.15.114:0/3472378779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5708a0e10 0x55a5708ab9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.061+0000 7fe64effd640 0 client.15870 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:23:06.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.061+0000 7fe678c71740 1 -- 172.21.15.114:0/3472378779 >> 172.21.15.114:0/3472378779 conn(0x55a57089d380 msgr2=0x55a5708a30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:06.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.061+0000 7fe678c71740 1 -- 172.21.15.114:0/3472378779 shutdown_connections 2024-09-13T18:23:06.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.061+0000 7fe678c71740 1 -- 172.21.15.114:0/3472378779 wait complete. 2024-09-13T18:23:06.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.061+0000 7fe678c71740 20 client.15870 trim_cache size 0 max 16384 2024-09-13T18:23:06.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.076+0000 7fe678c71740 1 Processor -- start 2024-09-13T18:23:06.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.076+0000 7fe678c71740 1 -- start start 2024-09-13T18:23:06.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.076+0000 7fe678c71740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5708a0e10 0x55a5708b2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.076+0000 7fe678c71740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5708a1730 0x55a5708b3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.076+0000 7fe678c71740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a5708abfd0 0x55a57089d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.076+0000 7fe678c71740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5704f84e0 con 0x55a5708abfd0 2024-09-13T18:23:06.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.076+0000 7fe678c71740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5708ba680 con 0x55a5708a0e10 2024-09-13T18:23:06.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.076+0000 7fe678c71740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a57089b9c0 con 0x55a5708a1730 2024-09-13T18:23:06.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.076+0000 7fe667fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a5708abfd0 0x55a57089d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.076+0000 7fe667fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a5708abfd0 0x55a57089d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36562/0 (socket says 172.21.15.114:36562) 2024-09-13T18:23:06.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.076+0000 7fe667fff640 1 -- 172.21.15.114:0/95881933 learned_addr learned my addr 172.21.15.114:0/95881933 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:06.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.077+0000 7fe66f7fe640 1 --2- 172.21.15.114:0/95881933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5708a1730 0x55a5708b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.077+0000 7fe667fff640 1 -- 172.21.15.114:0/95881933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5708a1730 msgr2=0x55a5708b3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:06.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.077+0000 7fe667fff640 1 --2- 172.21.15.114:0/95881933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5708a1730 0x55a5708b3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.077+0000 7fe667fff640 1 -- 172.21.15.114:0/95881933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5708a0e10 msgr2=0x55a5708b2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:06.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.077+0000 7fe667fff640 1 --2- 172.21.15.114:0/95881933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5708a0e10 0x55a5708b2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.077+0000 7fe667fff640 1 -- 172.21.15.114:0/95881933 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5707a5790 con 0x55a5708abfd0 2024-09-13T18:23:06.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.077+0000 7fe667fff640 1 --2- 172.21.15.114:0/95881933 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a5708abfd0 0x55a57089d760 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fe658009930 tx=0x7fe658031620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=546bc4ac0ccc3ae9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:06.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.077+0000 7fe6667fc640 1 -- 172.21.15.114:0/95881933 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe658044040 con 0x55a5708abfd0 2024-09-13T18:23:06.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.077+0000 7fe6667fc640 1 -- 172.21.15.114:0/95881933 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe658003eb0 con 0x55a5708abfd0 2024-09-13T18:23:06.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.077+0000 7fe6667fc640 1 -- 172.21.15.114:0/95881933 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe658031960 con 0x55a5708abfd0 2024-09-13T18:23:06.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.078+0000 7fe678c71740 1 -- 172.21.15.114:0/95881933 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a5708abfd0 msgr2=0x55a57089d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:06.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.078+0000 7fe678c71740 1 --2- 172.21.15.114:0/95881933 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a5708abfd0 0x55a57089d760 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fe658009930 tx=0x7fe658031620 comp rx=0 tx=0).stop 2024-09-13T18:23:06.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.078+0000 7fe678c71740 1 -- 172.21.15.114:0/95881933 shutdown_connections 2024-09-13T18:23:06.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.078+0000 7fe678c71740 1 --2- 172.21.15.114:0/95881933 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a5708abfd0 0x55a57089d760 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.078+0000 7fe678c71740 1 --2- 172.21.15.114:0/95881933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5708a1730 0x55a5708b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.078+0000 7fe678c71740 1 --2- 172.21.15.114:0/95881933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5708a0e10 0x55a5708b2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.078+0000 7fe678c71740 1 -- 172.21.15.114:0/95881933 >> 172.21.15.114:0/95881933 conn(0x55a57089d380 msgr2=0x55a5708c2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:06.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.078+0000 7fe678c71740 1 -- 172.21.15.114:0/95881933 shutdown_connections 2024-09-13T18:23:06.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.078+0000 7fe678c71740 1 -- 172.21.15.114:0/95881933 wait complete. 2024-09-13T18:23:06.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.078+0000 7fe678c71740 1 -- start start 2024-09-13T18:23:06.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.078+0000 7fe678c71740 1 Processor -- start 2024-09-13T18:23:06.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.079+0000 7fe678c71740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5708a0e10 0x55a5708aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.079+0000 7fe678c71740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5708a1730 0x55a5708af410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.079+0000 7fe678c71740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a5708abfd0 0x55a570911350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.079+0000 7fe678c71740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5708afb70 con 0x55a5708abfd0 2024-09-13T18:23:06.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.079+0000 7fe678c71740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a570911890 con 0x55a5708a1730 2024-09-13T18:23:06.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.079+0000 7fe678c71740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a570911a50 con 0x55a5708a0e10 2024-09-13T18:23:06.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.079+0000 7fe678c71740 10 client.0 _ll_register_callbacks cb 0x7fe6600040c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:23:06.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.079+0000 7fe667fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a5708abfd0 0x55a570911350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.079+0000 7fe667fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a5708abfd0 0x55a570911350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36576/0 (socket says 172.21.15.114:36576) 2024-09-13T18:23:06.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.079+0000 7fe667fff640 1 -- 172.21.15.114:0/2837028556 learned_addr learned my addr 172.21.15.114:0/2837028556 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:06.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.079+0000 7fe66f7fe640 1 --2- 172.21.15.114:0/2837028556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5708a1730 0x55a5708af410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.079+0000 7fe667fff640 1 --2- 172.21.15.114:0/2837028556 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5708a0e10 0x55a5708aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.079+0000 7fe667fff640 1 -- 172.21.15.114:0/2837028556 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5708a0e10 msgr2=0x55a5708aeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:06.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.079+0000 7fe667fff640 1 --2- 172.21.15.114:0/2837028556 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5708a0e10 0x55a5708aeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.079+0000 7fe667fff640 1 -- 172.21.15.114:0/2837028556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5708a1730 msgr2=0x55a5708af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:06.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.079+0000 7fe667fff640 1 --2- 172.21.15.114:0/2837028556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5708a1730 0x55a5708af410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.079+0000 7fe667fff640 1 -- 172.21.15.114:0/2837028556 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a5707bf840 con 0x55a5708abfd0 2024-09-13T18:23:06.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.079+0000 7fe667fff640 1 --2- 172.21.15.114:0/2837028556 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a5708abfd0 0x55a570911350 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe65802aa10 tx=0x7fe65804e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9ccba275873863a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:06.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.079+0000 7fe64effd640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:23:06.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.079+0000 7fe64effd640 1 -- 172.21.15.114:0/2837028556 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe658044040 con 0x55a5708abfd0 2024-09-13T18:23:06.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.079+0000 7fe64effd640 1 -- 172.21.15.114:0/2837028556 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fe658031900 con 0x55a5708abfd0 2024-09-13T18:23:06.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.079+0000 7fe678c71740 1 -- 172.21.15.114:0/2837028556 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe658003a80 con 0x55a5708abfd0 2024-09-13T18:23:06.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.080+0000 7fe64effd640 1 -- 172.21.15.114:0/2837028556 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe658031b00 con 0x55a5708abfd0 2024-09-13T18:23:06.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.080+0000 7fe678c71740 10 client.15876 fetch_fsmap learned FSMap version 30 2024-09-13T18:23:06.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.080+0000 7fe678c71740 1 -- 172.21.15.114:0/2837028556 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55a5707caea0 con 0x55a5708abfd0 2024-09-13T18:23:06.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.080+0000 7fe64effd640 1 -- 172.21.15.114:0/2837028556 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe658003a80 con 0x55a5708abfd0 2024-09-13T18:23:06.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.080+0000 7fe64effd640 1 -- 172.21.15.114:0/2837028556 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe65805ca50 con 0x55a5708abfd0 2024-09-13T18:23:06.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.080+0000 7fe678c71740 10 client.15876 fetch_fsmap finished waiting for FSMap version 30 2024-09-13T18:23:06.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.080+0000 7fe678c71740 10 client.15876 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:23:06.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.080+0000 7fe678c71740 20 client.15876 populate_metadata read hostname 'smithi114' 2024-09-13T18:23:06.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.080+0000 7fe678c71740 1 --2- 172.21.15.114:0/2837028556 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55a570912d50 0x55a570915140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.080+0000 7fe678c71740 4 client.15876 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T18:23:06.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.080+0000 7fe678c71740 1 -- 172.21.15.114:0/2837028556 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a57055ce30 con 0x55a570912d50 2024-09-13T18:23:06.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.080+0000 7fe66f7fe640 1 --2- 172.21.15.114:0/2837028556 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55a570912d50 0x55a570915140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.081+0000 7fe64effd640 10 client.15876 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:23:06.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.081+0000 7fe66f7fe640 1 --2- 172.21.15.114:0/2837028556 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55a570912d50 0x55a570915140 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38143c2510a10e84 server_cookie=5af5930af53c8614 in_seq=0 out_seq=0 2024-09-13T18:23:06.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.081+0000 7fe64effd640 1 -- 172.21.15.114:0/2837028556 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a57055ce30 con 0x55a570912d50 2024-09-13T18:23:06.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.081+0000 7fe64effd640 10 client.15876 handle_command_reply: tid=0 2024-09-13T18:23:06.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.081+0000 7fe678c71740 1 client.15876 shutdown 2024-09-13T18:23:06.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.081+0000 7fe678c71740 1 -- 172.21.15.114:0/2837028556 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a5708abfd0 msgr2=0x55a570911350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:06.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.081+0000 7fe678c71740 1 --2- 172.21.15.114:0/2837028556 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a5708abfd0 0x55a570911350 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe65802aa10 tx=0x7fe65804e040 comp rx=0 tx=0).stop 2024-09-13T18:23:06.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.081+0000 7fe678c71740 1 -- 172.21.15.114:0/2837028556 shutdown_connections 2024-09-13T18:23:06.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.081+0000 7fe678c71740 1 --2- 172.21.15.114:0/2837028556 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55a570912d50 0x55a570915140 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.081+0000 7fe678c71740 1 --2- 172.21.15.114:0/2837028556 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a5708abfd0 0x55a570911350 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.081+0000 7fe64effd640 0 client.15876 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:23:06.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.081+0000 7fe678c71740 1 --2- 172.21.15.114:0/2837028556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5708a1730 0x55a5708af410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.081+0000 7fe678c71740 1 --2- 172.21.15.114:0/2837028556 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5708a0e10 0x55a5708aeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.081+0000 7fe678c71740 1 -- 172.21.15.114:0/2837028556 >> 172.21.15.114:0/2837028556 conn(0x55a57089d380 msgr2=0x55a57089d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:06.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.081+0000 7fe678c71740 1 -- 172.21.15.114:0/2837028556 shutdown_connections 2024-09-13T18:23:06.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.081+0000 7fe678c71740 1 -- 172.21.15.114:0/2837028556 wait complete. 2024-09-13T18:23:06.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.081+0000 7fe678c71740 20 client.15876 trim_cache size 0 max 16384 2024-09-13T18:23:06.090 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:23:06.090 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "no active scrubs running", 2024-09-13T18:23:06.090 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": {} 2024-09-13T18:23:06.090 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:23:06.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.082+0000 7fe675f53640 1 -- 172.21.15.114:0/473802352 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe65c0a1320 msgr2=0x7fe65c0a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:06.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.082+0000 7fe675f53640 1 --2- 172.21.15.114:0/473802352 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe65c0a1320 0x7fe65c0a37e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe6680caaa0 tx=0x7fe66800ef50 comp rx=0 tx=0).stop 2024-09-13T18:23:06.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.082+0000 7fe675f53640 1 -- 172.21.15.114:0/473802352 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe670197ae0 msgr2=0x7fe6701a20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:06.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.082+0000 7fe675f53640 1 --2- 172.21.15.114:0/473802352 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe670197ae0 0x7fe6701a20e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fe66800e960 tx=0x7fe66800ee20 comp rx=0 tx=0).stop 2024-09-13T18:23:06.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.082+0000 7fe675f53640 1 -- 172.21.15.114:0/473802352 shutdown_connections 2024-09-13T18:23:06.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.082+0000 7fe675f53640 1 --2- 172.21.15.114:0/473802352 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe65c0a1320 0x7fe65c0a37e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.082+0000 7fe675f53640 1 --2- 172.21.15.114:0/473802352 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe670197ae0 0x7fe6701a20e0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.082+0000 7fe675f53640 1 --2- 172.21.15.114:0/473802352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe670103fb0 0x7fe6701975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.082+0000 7fe675f53640 1 --2- 172.21.15.114:0/473802352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe6700fee60 0x7fe670197060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.082+0000 7fe675f53640 1 -- 172.21.15.114:0/473802352 >> 172.21.15.114:0/473802352 conn(0x7fe670075c60 msgr2=0x7fe670102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:06.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.082+0000 7fe675f53640 1 -- 172.21.15.114:0/473802352 shutdown_connections 2024-09-13T18:23:06.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.082+0000 7fe675f53640 1 -- 172.21.15.114:0/473802352 wait complete. 2024-09-13T18:23:06.094 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-13T18:23:06.095 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:23:06.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.195+0000 7fa484a41640 1 -- 172.21.15.114:0/125854700 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4800ff830 msgr2=0x7fa480103870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:06.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.195+0000 7fa484a41640 1 --2- 172.21.15.114:0/125854700 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4800ff830 0x7fa480103870 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fa474009930 tx=0x7fa47402a420 comp rx=0 tx=0).stop 2024-09-13T18:23:06.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.195+0000 7fa484a41640 1 -- 172.21.15.114:0/125854700 shutdown_connections 2024-09-13T18:23:06.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.195+0000 7fa484a41640 1 --2- 172.21.15.114:0/125854700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa480103fd0 0x7fa48010bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.195+0000 7fa484a41640 1 --2- 172.21.15.114:0/125854700 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4800ff830 0x7fa480103870 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.195+0000 7fa484a41640 1 --2- 172.21.15.114:0/125854700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4800fee80 0x7fa4800ff260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.195+0000 7fa484a41640 1 -- 172.21.15.114:0/125854700 >> 172.21.15.114:0/125854700 conn(0x7fa480075c60 msgr2=0x7fa4800fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:06.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.195+0000 7fa484a41640 1 -- 172.21.15.114:0/125854700 shutdown_connections 2024-09-13T18:23:06.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.195+0000 7fa484a41640 1 -- 172.21.15.114:0/125854700 wait complete. 2024-09-13T18:23:06.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.196+0000 7fa484a41640 1 Processor -- start 2024-09-13T18:23:06.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.196+0000 7fa484a41640 1 -- start start 2024-09-13T18:23:06.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.196+0000 7fa484a41640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4800fee80 0x7fa480196200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.196+0000 7fa484a41640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa480103fd0 0x7fa480196740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.196+0000 7fa484a41640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa48019ab30 0x7fa48019afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.196+0000 7fa484a41640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48019b520 con 0x7fa48019ab30 2024-09-13T18:23:06.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.196+0000 7fa484a41640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48019b690 con 0x7fa480103fd0 2024-09-13T18:23:06.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.196+0000 7fa484a41640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48019b800 con 0x7fa4800fee80 2024-09-13T18:23:06.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.196+0000 7fa47ed9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa48019ab30 0x7fa48019afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.196+0000 7fa47ed9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa48019ab30 0x7fa48019afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36584/0 (socket says 172.21.15.114:36584) 2024-09-13T18:23:06.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.196+0000 7fa47ed9d640 1 -- 172.21.15.114:0/3504797702 learned_addr learned my addr 172.21.15.114:0/3504797702 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:06.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.196+0000 7fa47dd9b640 1 --2- 172.21.15.114:0/3504797702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa480103fd0 0x7fa480196740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.197+0000 7fa47e59c640 1 --2- 172.21.15.114:0/3504797702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4800fee80 0x7fa480196200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.197+0000 7fa47ed9d640 1 -- 172.21.15.114:0/3504797702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4800fee80 msgr2=0x7fa480196200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:06.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.197+0000 7fa47ed9d640 1 --2- 172.21.15.114:0/3504797702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4800fee80 0x7fa480196200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.197+0000 7fa47ed9d640 1 -- 172.21.15.114:0/3504797702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa480103fd0 msgr2=0x7fa480196740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:06.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.197+0000 7fa47ed9d640 1 --2- 172.21.15.114:0/3504797702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa480103fd0 0x7fa480196740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.197+0000 7fa47ed9d640 1 -- 172.21.15.114:0/3504797702 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa474009560 con 0x7fa48019ab30 2024-09-13T18:23:06.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.197+0000 7fa47ed9d640 1 --2- 172.21.15.114:0/3504797702 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa48019ab30 0x7fa48019afe0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fa47800eaa0 tx=0x7fa47800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:06.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.197+0000 7fa46f7fe640 1 -- 172.21.15.114:0/3504797702 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47800cdb0 con 0x7fa48019ab30 2024-09-13T18:23:06.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.197+0000 7fa46f7fe640 1 -- 172.21.15.114:0/3504797702 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa478005020 con 0x7fa48019ab30 2024-09-13T18:23:06.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.197+0000 7fa484a41640 1 -- 172.21.15.114:0/3504797702 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4801a2de0 con 0x7fa48019ab30 2024-09-13T18:23:06.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.197+0000 7fa46f7fe640 1 -- 172.21.15.114:0/3504797702 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4780116d0 con 0x7fa48019ab30 2024-09-13T18:23:06.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.197+0000 7fa484a41640 1 -- 172.21.15.114:0/3504797702 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4801a3260 con 0x7fa48019ab30 2024-09-13T18:23:06.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.198+0000 7fa484a41640 1 -- 172.21.15.114:0/3504797702 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa480100350 con 0x7fa48019ab30 2024-09-13T18:23:06.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.199+0000 7fa46f7fe640 1 -- 172.21.15.114:0/3504797702 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) v1 ==== 136107+0+0 (secure 0 0 0) 0x7fa478005190 con 0x7fa48019ab30 2024-09-13T18:23:06.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.200+0000 7fa46f7fe640 1 --2- 172.21.15.114:0/3504797702 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa4680a1320 0x7fa4680a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.200+0000 7fa46f7fe640 1 -- 172.21.15.114:0/3504797702 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fa4780cd750 con 0x7fa48019ab30 2024-09-13T18:23:06.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.200+0000 7fa47e59c640 1 --2- 172.21.15.114:0/3504797702 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa4680a1320 0x7fa4680a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.200+0000 7fa47e59c640 1 --2- 172.21.15.114:0/3504797702 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa4680a1320 0x7fa4680a37e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa470009bd0 tx=0x7fa470005f90 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:06.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.204+0000 7fa46f7fe640 1 -- 172.21.15.114:0/3504797702 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fa4780966b0 con 0x7fa48019ab30 2024-09-13T18:23:06.219 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:23:06 smithi114 ceph-mon[30897]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 31 op/s 2024-09-13T18:23:06.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:23:06 smithi204 ceph-mon[34777]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 31 op/s 2024-09-13T18:23:06.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.448+0000 7fa484a41640 1 -- 172.21.15.114:0/3504797702 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa4800ff260 con 0x7fa48019ab30 2024-09-13T18:23:06.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.449+0000 7fa46f7fe640 1 -- 172.21.15.114:0/3504797702 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+10930 (secure 0 0 0) 0x7fa47801f090 con 0x7fa48019ab30 2024-09-13T18:23:06.451 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:23:06.451 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4162817497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4162817497},{"type":"v1","addr":"172.21.15.114:6835","nonce":4162817497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/2435428478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":2435428478},{"type":"v1","addr":"172.21.15.114:6837","nonce":2435428478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T18:20:01.895117+0000","modified":"2024-09-13T18:22:29.064231+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24482,"mds_2":24475,"mds_3":24458,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/2046516451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":2046516451},{"type":"v1","addr":"172.21.15.114:6839","nonce":2046516451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":10,"addr":"172.21.15.114:6841/1956923446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":1956923446},{"type":"v1","addr":"172.21.15.114:6841","nonce":1956923446}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24391":{"gid":24391,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/507301496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":507301496},{"type":"v1","addr":"172.21.15.204:6833","nonce":507301496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4028657219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4028657219},{"type":"v1","addr":"172.21.15.134:6833","nonce":4028657219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/57029119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":57029119},{"type":"v1","addr":"172.21.15.204:6835","nonce":57029119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3068880939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3068880939},{"type":"v1","addr":"172.21.15.204:6837","nonce":3068880939}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"e","rank":3,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.134:6835/1150003275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1150003275},{"type":"v1","addr":"172.21.15.134:6835","nonce":1150003275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/344883912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":344883912},{"type":"v1","addr":"172.21.15.204:6839","nonce":344883912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6837/891692210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":891692210},{"type":"v1","addr":"172.21.15.134:6837","nonce":891692210}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6839/3466309724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3466309724},{"type":"v1","addr":"172.21.15.134:6839","nonce":3466309724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:23:06.451 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 30 2024-09-13T18:23:06.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.451+0000 7fa484a41640 1 -- 172.21.15.114:0/3504797702 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa4680a1320 msgr2=0x7fa4680a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:06.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.451+0000 7fa484a41640 1 --2- 172.21.15.114:0/3504797702 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa4680a1320 0x7fa4680a37e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa470009bd0 tx=0x7fa470005f90 comp rx=0 tx=0).stop 2024-09-13T18:23:06.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.451+0000 7fa484a41640 1 -- 172.21.15.114:0/3504797702 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa48019ab30 msgr2=0x7fa48019afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:06.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.451+0000 7fa484a41640 1 --2- 172.21.15.114:0/3504797702 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa48019ab30 0x7fa48019afe0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fa47800eaa0 tx=0x7fa47800ef60 comp rx=0 tx=0).stop 2024-09-13T18:23:06.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.451+0000 7fa484a41640 1 -- 172.21.15.114:0/3504797702 shutdown_connections 2024-09-13T18:23:06.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.451+0000 7fa484a41640 1 --2- 172.21.15.114:0/3504797702 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa4680a1320 0x7fa4680a37e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.451+0000 7fa484a41640 1 --2- 172.21.15.114:0/3504797702 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa48019ab30 0x7fa48019afe0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.451+0000 7fa484a41640 1 --2- 172.21.15.114:0/3504797702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa480103fd0 0x7fa480196740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.451+0000 7fa484a41640 1 --2- 172.21.15.114:0/3504797702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4800fee80 0x7fa480196200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.451+0000 7fa484a41640 1 -- 172.21.15.114:0/3504797702 >> 172.21.15.114:0/3504797702 conn(0x7fa480075c60 msgr2=0x7fa4800676e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:06.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.451+0000 7fa484a41640 1 -- 172.21.15.114:0/3504797702 shutdown_connections 2024-09-13T18:23:06.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.451+0000 7fa484a41640 1 -- 172.21.15.114:0/3504797702 wait complete. 2024-09-13T18:23:06.464 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-09-13T18:23:06.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.565+0000 7fdac4673640 1 -- 172.21.15.114:0/2175951409 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdabc0fee60 msgr2=0x7fdabc0ff240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:06.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.565+0000 7fdac4673640 1 --2- 172.21.15.114:0/2175951409 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdabc0fee60 0x7fdabc0ff240 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fdab8009930 tx=0x7fdab802a420 comp rx=0 tx=0).stop 2024-09-13T18:23:06.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.565+0000 7fdac4673640 1 -- 172.21.15.114:0/2175951409 shutdown_connections 2024-09-13T18:23:06.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.565+0000 7fdac4673640 1 --2- 172.21.15.114:0/2175951409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdabc103fb0 0x7fdabc10bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.565+0000 7fdac4673640 1 --2- 172.21.15.114:0/2175951409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdabc0ff810 0x7fdabc103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.565+0000 7fdac4673640 1 --2- 172.21.15.114:0/2175951409 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdabc0fee60 0x7fdabc0ff240 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.565+0000 7fdac4673640 1 -- 172.21.15.114:0/2175951409 >> 172.21.15.114:0/2175951409 conn(0x7fdabc075c60 msgr2=0x7fdabc0fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:06.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.565+0000 7fdac4673640 1 -- 172.21.15.114:0/2175951409 shutdown_connections 2024-09-13T18:23:06.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.565+0000 7fdac4673640 1 -- 172.21.15.114:0/2175951409 wait complete. 2024-09-13T18:23:06.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.566+0000 7fdac4673640 1 Processor -- start 2024-09-13T18:23:06.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.566+0000 7fdac4673640 1 -- start start 2024-09-13T18:23:06.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.566+0000 7fdac4673640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdabc0fee60 0x7fdabc1961c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.566+0000 7fdac4673640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdabc103fb0 0x7fdabc196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.566+0000 7fdac4673640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdabc19aaf0 0x7fdabc19afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.566+0000 7fdac4673640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdabc19b4e0 con 0x7fdabc0fee60 2024-09-13T18:23:06.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.566+0000 7fdac4673640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdabc19b650 con 0x7fdabc103fb0 2024-09-13T18:23:06.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.566+0000 7fdac4673640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdabc19b7c0 con 0x7fdabc19aaf0 2024-09-13T18:23:06.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.566+0000 7fdac240f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdabc0fee60 0x7fdabc1961c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.567+0000 7fdac240f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdabc0fee60 0x7fdabc1961c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36598/0 (socket says 172.21.15.114:36598) 2024-09-13T18:23:06.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.567+0000 7fdac240f640 1 -- 172.21.15.114:0/4175888461 learned_addr learned my addr 172.21.15.114:0/4175888461 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:06.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.567+0000 7fdac1c0e640 1 --2- 172.21.15.114:0/4175888461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdabc103fb0 0x7fdabc196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.567+0000 7fdac2c10640 1 --2- 172.21.15.114:0/4175888461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdabc19aaf0 0x7fdabc19afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.567+0000 7fdac240f640 1 -- 172.21.15.114:0/4175888461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdabc19aaf0 msgr2=0x7fdabc19afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:06.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.567+0000 7fdac240f640 1 --2- 172.21.15.114:0/4175888461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdabc19aaf0 0x7fdabc19afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.567+0000 7fdac240f640 1 -- 172.21.15.114:0/4175888461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdabc103fb0 msgr2=0x7fdabc196700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:06.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.567+0000 7fdac240f640 1 --2- 172.21.15.114:0/4175888461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdabc103fb0 0x7fdabc196700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.567+0000 7fdac240f640 1 -- 172.21.15.114:0/4175888461 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdab8009560 con 0x7fdabc0fee60 2024-09-13T18:23:06.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.567+0000 7fdac240f640 1 --2- 172.21.15.114:0/4175888461 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdabc0fee60 0x7fdabc1961c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fdab8005ab0 tx=0x7fdab8032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:06.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.567+0000 7fdab37fe640 1 -- 172.21.15.114:0/4175888461 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab8032e10 con 0x7fdabc0fee60 2024-09-13T18:23:06.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.567+0000 7fdac4673640 1 -- 172.21.15.114:0/4175888461 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdabc1a2da0 con 0x7fdabc0fee60 2024-09-13T18:23:06.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.567+0000 7fdab37fe640 1 -- 172.21.15.114:0/4175888461 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdab80046c0 con 0x7fdabc0fee60 2024-09-13T18:23:06.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.567+0000 7fdab37fe640 1 -- 172.21.15.114:0/4175888461 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab8003850 con 0x7fdabc0fee60 2024-09-13T18:23:06.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.567+0000 7fdac4673640 1 -- 172.21.15.114:0/4175888461 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdabc1a31d0 con 0x7fdabc0fee60 2024-09-13T18:23:06.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.570+0000 7fdab37fe640 1 -- 172.21.15.114:0/4175888461 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) v1 ==== 136107+0+0 (secure 0 0 0) 0x7fdab8004830 con 0x7fdabc0fee60 2024-09-13T18:23:06.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.570+0000 7fdac7468740 1 Processor -- start 2024-09-13T18:23:06.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.570+0000 7fdac7468740 1 -- start start 2024-09-13T18:23:06.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.571+0000 7fdac7468740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c63dc7c790 0x55c63dc7cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.571+0000 7fdac7468740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c63dc7d0b0 0x55c63dc87410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.571+0000 7fdac7468740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c63dc87950 0x55c63dc89d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.571+0000 7fdac7468740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c63dbe6cc0 con 0x55c63dc7c790 2024-09-13T18:23:06.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.571+0000 7fdac7468740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c63dbe4450 con 0x55c63dc87950 2024-09-13T18:23:06.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.571+0000 7fdac7468740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c63dbdfd40 con 0x55c63dc7d0b0 2024-09-13T18:23:06.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.571+0000 7fdab37fe640 1 --2- 172.21.15.114:0/4175888461 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fdaa80a13a0 0x7fdaa80a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.571+0000 7fdac2c10640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c63dc7c790 0x55c63dc7cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.571+0000 7fdac2c10640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c63dc7c790 0x55c63dc7cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36610/0 (socket says 172.21.15.114:36610) 2024-09-13T18:23:06.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.571+0000 7fdac2c10640 1 -- 172.21.15.114:0/2662965649 learned_addr learned my addr 172.21.15.114:0/2662965649 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:06.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.571+0000 7fdac1c0e640 1 --2- 172.21.15.114:0/2662965649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c63dc87950 0x55c63dc89d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.571+0000 7fdab37fe640 1 -- 172.21.15.114:0/4175888461 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fdab80eddb0 con 0x7fdabc0fee60 2024-09-13T18:23:06.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.571+0000 7fdac2c10640 1 --2- 172.21.15.114:0/4175888461 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fdaa80a13a0 0x7fdaa80a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.571+0000 7fdac240f640 1 --2- 172.21.15.114:0/2662965649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c63dc7d0b0 0x55c63dc87410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.571+0000 7fdac2c10640 1 -- 172.21.15.114:0/2662965649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c63dc7d0b0 msgr2=0x55c63dc87410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:06.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.571+0000 7fdac2c10640 1 --2- 172.21.15.114:0/2662965649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c63dc7d0b0 0x55c63dc87410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.571+0000 7fdac2c10640 1 -- 172.21.15.114:0/2662965649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c63dc87950 msgr2=0x55c63dc89d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:06.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.571+0000 7fdac2c10640 1 --2- 172.21.15.114:0/2662965649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c63dc87950 0x55c63dc89d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.571+0000 7fdac2c10640 1 -- 172.21.15.114:0/2662965649 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c63dba7620 con 0x55c63dc7c790 2024-09-13T18:23:06.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.571+0000 7fdac1c0e640 1 --2- 172.21.15.114:0/2662965649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c63dc87950 0x55c63dc89d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:23:06.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.572+0000 7fdac2c10640 1 --2- 172.21.15.114:0/2662965649 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c63dc7c790 0x55c63dc7cb70 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fdab4012a50 tx=0x7fdab4012f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16ce9c4651b198ed server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:06.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.572+0000 7fdab0ff9640 1 -- 172.21.15.114:0/2662965649 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab4012370 con 0x55c63dc7c790 2024-09-13T18:23:06.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.572+0000 7fdab0ff9640 1 -- 172.21.15.114:0/2662965649 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdab401dc60 con 0x55c63dc7c790 2024-09-13T18:23:06.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.572+0000 7fdab0ff9640 1 -- 172.21.15.114:0/2662965649 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab400f950 con 0x55c63dc7c790 2024-09-13T18:23:06.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.572+0000 7fdac7468740 1 -- 172.21.15.114:0/2662965649 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c63dc7c790 msgr2=0x55c63dc7cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:06.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.572+0000 7fdac2c10640 1 --2- 172.21.15.114:0/4175888461 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fdaa80a13a0 0x7fdaa80a3860 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdab4021000 tx=0x7fdab401f000 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:06.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.572+0000 7fdac7468740 1 --2- 172.21.15.114:0/2662965649 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c63dc7c790 0x55c63dc7cb70 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fdab4012a50 tx=0x7fdab4012f10 comp rx=0 tx=0).stop 2024-09-13T18:23:06.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.573+0000 7fdac7468740 1 -- 172.21.15.114:0/2662965649 shutdown_connections 2024-09-13T18:23:06.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.573+0000 7fdac7468740 1 --2- 172.21.15.114:0/2662965649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c63dc87950 0x55c63dc89d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.573+0000 7fdac7468740 1 --2- 172.21.15.114:0/2662965649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c63dc7d0b0 0x55c63dc87410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.573+0000 7fdac7468740 1 --2- 172.21.15.114:0/2662965649 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c63dc7c790 0x55c63dc7cb70 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.573+0000 7fdac7468740 1 -- 172.21.15.114:0/2662965649 >> 172.21.15.114:0/2662965649 conn(0x55c63dc78d00 msgr2=0x55c63dc790e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:06.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.573+0000 7fdac7468740 1 -- 172.21.15.114:0/2662965649 shutdown_connections 2024-09-13T18:23:06.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.573+0000 7fdac7468740 1 -- 172.21.15.114:0/2662965649 wait complete. 2024-09-13T18:23:06.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.573+0000 7fdac7468740 1 -- start start 2024-09-13T18:23:06.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.573+0000 7fdac7468740 1 Processor -- start 2024-09-13T18:23:06.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.573+0000 7fdac7468740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c63dc7c790 0x55c63dc86f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.573+0000 7fdac7468740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c63dc7d0b0 0x55c63dc843a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.573+0000 7fdac7468740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c63dc87950 0x55c63dc848e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.573+0000 7fdac7468740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c63db9e180 con 0x55c63dc7c790 2024-09-13T18:23:06.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.573+0000 7fdac7468740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c63d8b8430 con 0x55c63dc87950 2024-09-13T18:23:06.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.573+0000 7fdac7468740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c63db635e0 con 0x55c63dc7d0b0 2024-09-13T18:23:06.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.574+0000 7fdac7468740 10 client.0 _ll_register_callbacks cb 0x55c63dbcfaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:23:06.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.574+0000 7fdac240f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c63dc7c790 0x55c63dc86f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.574+0000 7fdac240f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c63dc7c790 0x55c63dc86f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36614/0 (socket says 172.21.15.114:36614) 2024-09-13T18:23:06.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.574+0000 7fdac240f640 1 -- 172.21.15.114:0/162564127 learned_addr learned my addr 172.21.15.114:0/162564127 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:06.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.574+0000 7fdac1c0e640 1 --2- 172.21.15.114:0/162564127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c63dc7d0b0 0x55c63dc843a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.574+0000 7fdac240f640 1 -- 172.21.15.114:0/162564127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c63dc7d0b0 msgr2=0x55c63dc843a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:06.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.574+0000 7fdac240f640 1 --2- 172.21.15.114:0/162564127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c63dc7d0b0 0x55c63dc843a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.574+0000 7fdac240f640 1 -- 172.21.15.114:0/162564127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c63dc87950 msgr2=0x55c63dc848e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:06.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.574+0000 7fdac240f640 1 --2- 172.21.15.114:0/162564127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c63dc87950 0x55c63dc848e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.574+0000 7fdac240f640 1 -- 172.21.15.114:0/162564127 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdab4012630 con 0x55c63dc7c790 2024-09-13T18:23:06.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.574+0000 7fdac240f640 1 --2- 172.21.15.114:0/162564127 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c63dc7c790 0x55c63dc86f40 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fdab80b6f40 tx=0x7fdab80ba460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d46fcb059d37ea7c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:06.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.574+0000 7fdaa5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:23:06.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.574+0000 7fdaa5ffb640 1 -- 172.21.15.114:0/162564127 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab80ba130 con 0x55c63dc7c790 2024-09-13T18:23:06.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.574+0000 7fdaa5ffb640 1 -- 172.21.15.114:0/162564127 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fdab80c2e10 con 0x55c63dc7c790 2024-09-13T18:23:06.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.574+0000 7fdac7468740 1 -- 172.21.15.114:0/162564127 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c63dc09b80 con 0x55c63dc7c790 2024-09-13T18:23:06.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.574+0000 7fdaa5ffb640 1 -- 172.21.15.114:0/162564127 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdab80c2980 con 0x55c63dc7c790 2024-09-13T18:23:06.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.574+0000 7fdac7468740 10 client.15894 fetch_fsmap learned FSMap version 30 2024-09-13T18:23:06.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.574+0000 7fdac7468740 1 -- 172.21.15.114:0/162564127 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c63dbde3c0 con 0x55c63dc7c790 2024-09-13T18:23:06.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.574+0000 7fdaa5ffb640 1 -- 172.21.15.114:0/162564127 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c63dc09b80 con 0x55c63dc7c790 2024-09-13T18:23:06.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.575+0000 7fdaa5ffb640 1 -- 172.21.15.114:0/162564127 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdab80eb340 con 0x55c63dc7c790 2024-09-13T18:23:06.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.575+0000 7fdac7468740 10 client.15894 fetch_fsmap finished waiting for FSMap version 30 2024-09-13T18:23:06.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.575+0000 7fdac7468740 10 client.15894 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-13T18:23:06.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.575+0000 7fdac7468740 20 client.15894 populate_metadata read hostname 'smithi114' 2024-09-13T18:23:06.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.575+0000 7fdac7468740 1 --2- 172.21.15.114:0/162564127 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55c63dc96ff0 0x55c63dc99410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.575+0000 7fdac7468740 4 client.15894 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:23:06.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.575+0000 7fdac7468740 1 -- 172.21.15.114:0/162564127 --> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c63dbd21f0 con 0x55c63dc96ff0 2024-09-13T18:23:06.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.575+0000 7fdac1c0e640 1 --2- 172.21.15.114:0/162564127 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55c63dc96ff0 0x55c63dc99410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.575+0000 7fdaa5ffb640 10 client.15894 ms_handle_connect on v2:172.21.15.114:6840/1956923446 2024-09-13T18:23:06.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.575+0000 7fdac1c0e640 1 --2- 172.21.15.114:0/162564127 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55c63dc96ff0 0x55c63dc99410 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=74ffbee8b0a2798 server_cookie=ca7b61582a7d8b9f in_seq=0 out_seq=0 2024-09-13T18:23:06.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.576+0000 7fdaa5ffb640 1 -- 172.21.15.114:0/162564127 <== mds.4 v2:172.21.15.114:6840/1956923446 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55c63dbd21f0 con 0x55c63dc96ff0 2024-09-13T18:23:06.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.576+0000 7fdaa5ffb640 10 client.15894 handle_command_reply: tid=0 2024-09-13T18:23:06.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.576+0000 7fdac7468740 1 client.15894 shutdown 2024-09-13T18:23:06.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.576+0000 7fdac7468740 1 -- 172.21.15.114:0/162564127 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c63dc7c790 msgr2=0x55c63dc86f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:06.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.576+0000 7fdac7468740 1 --2- 172.21.15.114:0/162564127 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c63dc7c790 0x55c63dc86f40 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fdab80b6f40 tx=0x7fdab80ba460 comp rx=0 tx=0).stop 2024-09-13T18:23:06.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.576+0000 7fdac7468740 1 -- 172.21.15.114:0/162564127 shutdown_connections 2024-09-13T18:23:06.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.576+0000 7fdac7468740 1 --2- 172.21.15.114:0/162564127 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55c63dc96ff0 0x55c63dc99410 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.576+0000 7fdac7468740 1 --2- 172.21.15.114:0/162564127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c63dc87950 0x55c63dc848e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.576+0000 7fdaa5ffb640 0 client.15894 ms_handle_reset on v2:172.21.15.114:6840/1956923446 2024-09-13T18:23:06.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.576+0000 7fdac7468740 1 --2- 172.21.15.114:0/162564127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c63dc7d0b0 0x55c63dc843a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.576+0000 7fdac7468740 1 --2- 172.21.15.114:0/162564127 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c63dc7c790 0x55c63dc86f40 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.576+0000 7fdac7468740 1 -- 172.21.15.114:0/162564127 >> 172.21.15.114:0/162564127 conn(0x55c63dc78d00 msgr2=0x55c63dc77eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:06.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.576+0000 7fdac7468740 1 -- 172.21.15.114:0/162564127 shutdown_connections 2024-09-13T18:23:06.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.576+0000 7fdac7468740 1 -- 172.21.15.114:0/162564127 wait complete. 2024-09-13T18:23:06.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.577+0000 7fdac7468740 20 client.15894 trim_cache size 0 max 16384 2024-09-13T18:23:06.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.595+0000 7fdac7468740 1 Processor -- start 2024-09-13T18:23:06.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.595+0000 7fdac7468740 1 -- start start 2024-09-13T18:23:06.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.595+0000 7fdac7468740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c63dc7c790 0x55c63dc8e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.595+0000 7fdac7468740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c63dc7d0b0 0x55c63dc8eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.595+0000 7fdac7468740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c63dc87950 0x55c63dc7f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.595+0000 7fdac7468740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c63dc9e3f0 con 0x55c63dc7d0b0 2024-09-13T18:23:06.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.595+0000 7fdac7468740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c63dc8a400 con 0x55c63dc7c790 2024-09-13T18:23:06.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.595+0000 7fdac7468740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c63dc09b80 con 0x55c63dc87950 2024-09-13T18:23:06.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.595+0000 7fdac1c0e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c63dc7d0b0 0x55c63dc8eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.595+0000 7fdac1c0e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c63dc7d0b0 0x55c63dc8eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36624/0 (socket says 172.21.15.114:36624) 2024-09-13T18:23:06.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.595+0000 7fdac1c0e640 1 -- 172.21.15.114:0/4253552624 learned_addr learned my addr 172.21.15.114:0/4253552624 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:06.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.596+0000 7fdac240f640 1 --2- 172.21.15.114:0/4253552624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c63dc7c790 0x55c63dc8e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.596+0000 7fdac2c10640 1 --2- 172.21.15.114:0/4253552624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c63dc87950 0x55c63dc7f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.596+0000 7fdac1c0e640 1 -- 172.21.15.114:0/4253552624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c63dc87950 msgr2=0x55c63dc7f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:06.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.596+0000 7fdac1c0e640 1 --2- 172.21.15.114:0/4253552624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c63dc87950 0x55c63dc7f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.596+0000 7fdac1c0e640 1 -- 172.21.15.114:0/4253552624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c63dc7c790 msgr2=0x55c63dc8e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:06.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.596+0000 7fdac1c0e640 1 --2- 172.21.15.114:0/4253552624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c63dc7c790 0x55c63dc8e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.596+0000 7fdac1c0e640 1 -- 172.21.15.114:0/4253552624 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c63db80770 con 0x55c63dc7d0b0 2024-09-13T18:23:06.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.596+0000 7fdac1c0e640 1 --2- 172.21.15.114:0/4253552624 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c63dc7d0b0 0x55c63dc8eb50 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x55c63dc7f330 tx=0x7fdaac00bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a03a3c0e9d93793c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:06.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.596+0000 7fdab17fa640 1 -- 172.21.15.114:0/4253552624 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaac017040 con 0x55c63dc7d0b0 2024-09-13T18:23:06.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.596+0000 7fdab17fa640 1 -- 172.21.15.114:0/4253552624 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdaac01c830 con 0x55c63dc7d0b0 2024-09-13T18:23:06.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.596+0000 7fdab17fa640 1 -- 172.21.15.114:0/4253552624 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaac011b20 con 0x55c63dc7d0b0 2024-09-13T18:23:06.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.597+0000 7fdac7468740 1 -- 172.21.15.114:0/4253552624 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c63dc7d0b0 msgr2=0x55c63dc8eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:06.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.597+0000 7fdac7468740 1 --2- 172.21.15.114:0/4253552624 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c63dc7d0b0 0x55c63dc8eb50 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x55c63dc7f330 tx=0x7fdaac00bd90 comp rx=0 tx=0).stop 2024-09-13T18:23:06.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.597+0000 7fdac7468740 1 -- 172.21.15.114:0/4253552624 shutdown_connections 2024-09-13T18:23:06.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.597+0000 7fdac7468740 1 --2- 172.21.15.114:0/4253552624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c63dc87950 0x55c63dc7f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.597+0000 7fdac7468740 1 --2- 172.21.15.114:0/4253552624 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c63dc7d0b0 0x55c63dc8eb50 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.597+0000 7fdac7468740 1 --2- 172.21.15.114:0/4253552624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c63dc7c790 0x55c63dc8e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.597+0000 7fdac7468740 1 -- 172.21.15.114:0/4253552624 >> 172.21.15.114:0/4253552624 conn(0x55c63dc78d00 msgr2=0x55c63dc82c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:06.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.597+0000 7fdac7468740 1 -- 172.21.15.114:0/4253552624 shutdown_connections 2024-09-13T18:23:06.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.597+0000 7fdac7468740 1 -- 172.21.15.114:0/4253552624 wait complete. 2024-09-13T18:23:06.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.598+0000 7fdac7468740 1 -- start start 2024-09-13T18:23:06.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.598+0000 7fdac7468740 1 Processor -- start 2024-09-13T18:23:06.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.598+0000 7fdac7468740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c63dc7c790 0x55c63dc80f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.598+0000 7fdac7468740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c63dc7d0b0 0x55c63dc814a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.598+0000 7fdac7468740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c63dc87950 0x55c63dc819e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.598+0000 7fdac7468740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c63dced4f0 con 0x55c63dc87950 2024-09-13T18:23:06.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.598+0000 7fdac7468740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c63dced630 con 0x55c63dc7c790 2024-09-13T18:23:06.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.598+0000 7fdac7468740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c63dced770 con 0x55c63dc7d0b0 2024-09-13T18:23:06.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.598+0000 7fdac2c10640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c63dc87950 0x55c63dc819e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.598+0000 7fdac7468740 10 client.0 _ll_register_callbacks cb 0x7fdab4009a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:23:06.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.598+0000 7fdac2c10640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c63dc87950 0x55c63dc819e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36630/0 (socket says 172.21.15.114:36630) 2024-09-13T18:23:06.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.598+0000 7fdac2c10640 1 -- 172.21.15.114:0/1951263208 learned_addr learned my addr 172.21.15.114:0/1951263208 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:06.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.599+0000 7fdac240f640 1 --2- 172.21.15.114:0/1951263208 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c63dc7c790 0x55c63dc80f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.599+0000 7fdac1c0e640 1 --2- 172.21.15.114:0/1951263208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c63dc7d0b0 0x55c63dc814a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.599+0000 7fdac2c10640 1 -- 172.21.15.114:0/1951263208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c63dc7d0b0 msgr2=0x55c63dc814a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:06.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.599+0000 7fdac2c10640 1 --2- 172.21.15.114:0/1951263208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c63dc7d0b0 0x55c63dc814a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.599+0000 7fdac2c10640 1 -- 172.21.15.114:0/1951263208 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c63dc7c790 msgr2=0x55c63dc80f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:06.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.599+0000 7fdac2c10640 1 --2- 172.21.15.114:0/1951263208 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c63dc7c790 0x55c63dc80f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.599+0000 7fdac2c10640 1 -- 172.21.15.114:0/1951263208 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaac0041c0 con 0x55c63dc87950 2024-09-13T18:23:06.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.599+0000 7fdac240f640 1 --2- 172.21.15.114:0/1951263208 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c63dc7c790 0x55c63dc80f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:23:06.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.599+0000 7fdac2c10640 1 --2- 172.21.15.114:0/1951263208 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c63dc87950 0x55c63dc819e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fdab400e8e0 tx=0x7fdab4009480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67a71539f62a8355 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:06.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.599+0000 7fdaa5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:23:06.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.599+0000 7fdaa5ffb640 1 -- 172.21.15.114:0/1951263208 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab402d040 con 0x55c63dc87950 2024-09-13T18:23:06.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.599+0000 7fdaa5ffb640 1 -- 172.21.15.114:0/1951263208 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fdab400fe40 con 0x55c63dc87950 2024-09-13T18:23:06.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.599+0000 7fdac7468740 1 -- 172.21.15.114:0/1951263208 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdab401dc60 con 0x55c63dc87950 2024-09-13T18:23:06.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.599+0000 7fdaa5ffb640 1 -- 172.21.15.114:0/1951263208 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdab400f930 con 0x55c63dc87950 2024-09-13T18:23:06.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.600+0000 7fdac7468740 10 client.15900 fetch_fsmap learned FSMap version 30 2024-09-13T18:23:06.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.600+0000 7fdac7468740 1 -- 172.21.15.114:0/1951263208 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c63dba5e80 con 0x55c63dc87950 2024-09-13T18:23:06.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.600+0000 7fdaa5ffb640 1 -- 172.21.15.114:0/1951263208 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdab401dc60 con 0x55c63dc87950 2024-09-13T18:23:06.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.600+0000 7fdaa5ffb640 1 -- 172.21.15.114:0/1951263208 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdab4004980 con 0x55c63dc87950 2024-09-13T18:23:06.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.600+0000 7fdac7468740 10 client.15900 fetch_fsmap finished waiting for FSMap version 30 2024-09-13T18:23:06.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.600+0000 7fdac7468740 10 client.15900 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-13T18:23:06.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.600+0000 7fdac7468740 20 client.15900 populate_metadata read hostname 'smithi114' 2024-09-13T18:23:06.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.600+0000 7fdac7468740 1 --2- 172.21.15.114:0/1951263208 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55c63dc96ff0 0x55c63dcf0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.600+0000 7fdac7468740 4 client.15900 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:23:06.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.600+0000 7fdac7468740 1 -- 172.21.15.114:0/1951263208 --> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c63d8d34e0 con 0x55c63dc96ff0 2024-09-13T18:23:06.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.600+0000 7fdac240f640 1 --2- 172.21.15.114:0/1951263208 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55c63dc96ff0 0x55c63dcf0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.601+0000 7fdaa5ffb640 10 client.15900 ms_handle_connect on v2:172.21.15.114:6840/1956923446 2024-09-13T18:23:06.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.601+0000 7fdac240f640 1 --2- 172.21.15.114:0/1951263208 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55c63dc96ff0 0x55c63dcf0fd0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f1cf484ff7efa14c server_cookie=9a7a1aa922f2db7f in_seq=0 out_seq=0 2024-09-13T18:23:06.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.601+0000 7fdaa5ffb640 1 -- 172.21.15.114:0/1951263208 <== mds.4 v2:172.21.15.114:6840/1956923446 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c63d8d34e0 con 0x55c63dc96ff0 2024-09-13T18:23:06.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.601+0000 7fdaa5ffb640 10 client.15900 handle_command_reply: tid=0 2024-09-13T18:23:06.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.601+0000 7fdac7468740 1 client.15900 shutdown 2024-09-13T18:23:06.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.601+0000 7fdac7468740 1 -- 172.21.15.114:0/1951263208 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c63dc87950 msgr2=0x55c63dc819e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:06.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.601+0000 7fdac7468740 1 --2- 172.21.15.114:0/1951263208 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c63dc87950 0x55c63dc819e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fdab400e8e0 tx=0x7fdab4009480 comp rx=0 tx=0).stop 2024-09-13T18:23:06.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.601+0000 7fdac7468740 1 -- 172.21.15.114:0/1951263208 shutdown_connections 2024-09-13T18:23:06.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.601+0000 7fdac7468740 1 --2- 172.21.15.114:0/1951263208 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55c63dc96ff0 0x55c63dcf0fd0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.601+0000 7fdac7468740 1 --2- 172.21.15.114:0/1951263208 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c63dc87950 0x55c63dc819e0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.601+0000 7fdac7468740 1 --2- 172.21.15.114:0/1951263208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c63dc7d0b0 0x55c63dc814a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.601+0000 7fdaa5ffb640 0 client.15900 ms_handle_reset on v2:172.21.15.114:6840/1956923446 2024-09-13T18:23:06.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.601+0000 7fdac7468740 1 --2- 172.21.15.114:0/1951263208 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c63dc7c790 0x55c63dc80f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.601+0000 7fdac7468740 1 -- 172.21.15.114:0/1951263208 >> 172.21.15.114:0/1951263208 conn(0x55c63dc78d00 msgr2=0x55c63dc7e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:06.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.601+0000 7fdac7468740 1 -- 172.21.15.114:0/1951263208 shutdown_connections 2024-09-13T18:23:06.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.601+0000 7fdac7468740 1 -- 172.21.15.114:0/1951263208 wait complete. 2024-09-13T18:23:06.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.601+0000 7fdac7468740 20 client.15900 trim_cache size 0 max 16384 2024-09-13T18:23:06.610 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:23:06.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.602+0000 7fdac4673640 1 -- 172.21.15.114:0/4175888461 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fdaa80a13a0 msgr2=0x7fdaa80a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:06.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.602+0000 7fdac4673640 1 --2- 172.21.15.114:0/4175888461 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fdaa80a13a0 0x7fdaa80a3860 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdab4021000 tx=0x7fdab401f000 comp rx=0 tx=0).stop 2024-09-13T18:23:06.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.602+0000 7fdac4673640 1 -- 172.21.15.114:0/4175888461 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdabc0fee60 msgr2=0x7fdabc1961c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:06.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.602+0000 7fdac4673640 1 --2- 172.21.15.114:0/4175888461 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdabc0fee60 0x7fdabc1961c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fdab8005ab0 tx=0x7fdab8032c20 comp rx=0 tx=0).stop 2024-09-13T18:23:06.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.602+0000 7fdac4673640 1 -- 172.21.15.114:0/4175888461 shutdown_connections 2024-09-13T18:23:06.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.602+0000 7fdac4673640 1 --2- 172.21.15.114:0/4175888461 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fdaa80a13a0 0x7fdaa80a3860 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.602+0000 7fdac4673640 1 --2- 172.21.15.114:0/4175888461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdabc19aaf0 0x7fdabc19afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.602+0000 7fdac4673640 1 --2- 172.21.15.114:0/4175888461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdabc103fb0 0x7fdabc196700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.602+0000 7fdac4673640 1 --2- 172.21.15.114:0/4175888461 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdabc0fee60 0x7fdabc1961c0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.602+0000 7fdac4673640 1 -- 172.21.15.114:0/4175888461 >> 172.21.15.114:0/4175888461 conn(0x7fdabc075c60 msgr2=0x7fdabc067740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:06.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.602+0000 7fdac4673640 1 -- 172.21.15.114:0/4175888461 shutdown_connections 2024-09-13T18:23:06.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.602+0000 7fdac4673640 1 -- 172.21.15.114:0/4175888461 wait complete. 2024-09-13T18:23:06.617 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-09-13T18:23:06.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:23:06 smithi134 ceph-mon[34283]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 31 op/s 2024-09-13T18:23:06.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.717+0000 7f07ab311640 1 -- 172.21.15.114:0/1050279746 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07a4101860 msgr2=0x7f07a4109d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:06.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.717+0000 7f07ab311640 1 --2- 172.21.15.114:0/1050279746 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07a4101860 0x7f07a4109d40 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f0794009900 tx=0x7f079402a480 comp rx=0 tx=0).stop 2024-09-13T18:23:06.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.717+0000 7f07ab311640 1 -- 172.21.15.114:0/1050279746 shutdown_connections 2024-09-13T18:23:06.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.717+0000 7f07ab311640 1 --2- 172.21.15.114:0/1050279746 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f07a410a310 0x7f07a410c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.717+0000 7f07ab311640 1 --2- 172.21.15.114:0/1050279746 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07a4101860 0x7f07a4109d40 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.717+0000 7f07ab311640 1 --2- 172.21.15.114:0/1050279746 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f07a4100f40 0x7f07a4101320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.717+0000 7f07ab311640 1 -- 172.21.15.114:0/1050279746 >> 172.21.15.114:0/1050279746 conn(0x7f07a4075c60 msgr2=0x7f07a40fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:06.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.718+0000 7f07ab311640 1 -- 172.21.15.114:0/1050279746 shutdown_connections 2024-09-13T18:23:06.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.718+0000 7f07ab311640 1 -- 172.21.15.114:0/1050279746 wait complete. 2024-09-13T18:23:06.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.718+0000 7f07ab311640 1 Processor -- start 2024-09-13T18:23:06.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.718+0000 7f07ab311640 1 -- start start 2024-09-13T18:23:06.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.719+0000 7f07ab311640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f07a4100f40 0x7f07a40feb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.719+0000 7f07ab311640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f07a4101860 0x7f07a40ff0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.719+0000 7f07ab311640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07a410a310 0x7f07a40ff610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.719+0000 7f07ab311640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a41a44b0 con 0x7f07a410a310 2024-09-13T18:23:06.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.719+0000 7f07ab311640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a41a4620 con 0x7f07a4101860 2024-09-13T18:23:06.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.719+0000 7f07ab311640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a41a47e0 con 0x7f07a4100f40 2024-09-13T18:23:06.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.719+0000 7f07a98ae640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07a410a310 0x7f07a40ff610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.719+0000 7f07a90ad640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f07a4100f40 0x7f07a40feb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.719+0000 7f07a98ae640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07a410a310 0x7f07a40ff610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36640/0 (socket says 172.21.15.114:36640) 2024-09-13T18:23:06.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.719+0000 7f07a98ae640 1 -- 172.21.15.114:0/2824358552 learned_addr learned my addr 172.21.15.114:0/2824358552 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:06.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.719+0000 7f07a88ac640 1 --2- 172.21.15.114:0/2824358552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f07a4101860 0x7f07a40ff0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.719+0000 7f07a90ad640 1 -- 172.21.15.114:0/2824358552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f07a4101860 msgr2=0x7f07a40ff0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:06.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.719+0000 7f07a90ad640 1 --2- 172.21.15.114:0/2824358552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f07a4101860 0x7f07a40ff0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.719+0000 7f07a90ad640 1 -- 172.21.15.114:0/2824358552 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07a410a310 msgr2=0x7f07a40ff610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:06.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.719+0000 7f07a90ad640 1 --2- 172.21.15.114:0/2824358552 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07a410a310 0x7f07a40ff610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.719+0000 7f07a90ad640 1 -- 172.21.15.114:0/2824358552 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0794009560 con 0x7f07a4100f40 2024-09-13T18:23:06.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.719+0000 7f07a98ae640 1 --2- 172.21.15.114:0/2824358552 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07a410a310 0x7f07a40ff610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:23:06.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.720+0000 7f07a90ad640 1 --2- 172.21.15.114:0/2824358552 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f07a4100f40 0x7f07a40feb90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f07a000c970 tx=0x7f07a000ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:06.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.720+0000 7f079a7fc640 1 -- 172.21.15.114:0/2824358552 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a0007bb0 con 0x7f07a4100f40 2024-09-13T18:23:06.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.720+0000 7f07ab311640 1 -- 172.21.15.114:0/2824358552 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07a41a4ac0 con 0x7f07a4100f40 2024-09-13T18:23:06.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.720+0000 7f079a7fc640 1 -- 172.21.15.114:0/2824358552 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f07a0007d10 con 0x7f07a4100f40 2024-09-13T18:23:06.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.720+0000 7f079a7fc640 1 -- 172.21.15.114:0/2824358552 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a0018660 con 0x7f07a4100f40 2024-09-13T18:23:06.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.720+0000 7f07ab311640 1 -- 172.21.15.114:0/2824358552 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07a41a4ff0 con 0x7f07a4100f40 2024-09-13T18:23:06.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.723+0000 7f079a7fc640 1 -- 172.21.15.114:0/2824358552 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 136107+0+0 (secure 0 0 0) 0x7f07a000f450 con 0x7f07a4100f40 2024-09-13T18:23:06.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.723+0000 7f07ae032740 1 Processor -- start 2024-09-13T18:23:06.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.724+0000 7f07ae032740 1 -- start start 2024-09-13T18:23:06.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.724+0000 7f07ae032740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558c71ae790 0x5558c71aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.724+0000 7f07ae032740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5558c71af0b0 0x5558c71b9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.724+0000 7f079a7fc640 1 --2- 172.21.15.114:0/2824358552 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f07900a1320 0x7f07900a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.724+0000 7f07ae032740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558c71b9950 0x5558c71bbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.724+0000 7f07ae032740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558c7118cc0 con 0x5558c71af0b0 2024-09-13T18:23:06.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.724+0000 7f07ae032740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558c7116450 con 0x5558c71b9950 2024-09-13T18:23:06.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.724+0000 7f07ae032740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558c7111d40 con 0x5558c71ae790 2024-09-13T18:23:06.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.724+0000 7f07a88ac640 1 --2- 172.21.15.114:0/2824358552 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f07900a1320 0x7f07900a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.724+0000 7f079a7fc640 1 -- 172.21.15.114:0/2824358552 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f07a00cd2b0 con 0x7f07a4100f40 2024-09-13T18:23:06.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.724+0000 7f07a98ae640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558c71ae790 0x5558c71aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.724+0000 7f07a98ae640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558c71ae790 0x5558c71aeb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:32880/0 (socket says 172.21.15.114:32880) 2024-09-13T18:23:06.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.724+0000 7f07a98ae640 1 -- 172.21.15.114:0/1592907064 learned_addr learned my addr 172.21.15.114:0/1592907064 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:06.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.724+0000 7f07a90ad640 1 --2- 172.21.15.114:0/1592907064 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5558c71af0b0 0x5558c71b9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.724+0000 7f07a98ae640 1 --2- 172.21.15.114:0/1592907064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558c71b9950 0x5558c71bbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.725+0000 7f07a98ae640 1 -- 172.21.15.114:0/1592907064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558c71b9950 msgr2=0x5558c71bbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:06.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.725+0000 7f07a98ae640 1 --2- 172.21.15.114:0/1592907064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558c71b9950 0x5558c71bbd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.725+0000 7f07a98ae640 1 -- 172.21.15.114:0/1592907064 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5558c71af0b0 msgr2=0x5558c71b9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:06.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.725+0000 7f07a98ae640 1 --2- 172.21.15.114:0/1592907064 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5558c71af0b0 0x5558c71b9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.725+0000 7f07a98ae640 1 -- 172.21.15.114:0/1592907064 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558c70d9620 con 0x5558c71ae790 2024-09-13T18:23:06.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.725+0000 7f07a88ac640 1 --2- 172.21.15.114:0/2824358552 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f07900a1320 0x7f07900a37e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f079402aa70 tx=0x7f0794005a20 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:06.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.725+0000 7f07a98ae640 1 --2- 172.21.15.114:0/1592907064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558c71ae790 0x5558c71aeb70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f079c013f30 tx=0x7f079c0113e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=503b48dc876d3e22 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:06.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.725+0000 7f078f7fe640 1 -- 172.21.15.114:0/1592907064 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079c011da0 con 0x5558c71ae790 2024-09-13T18:23:06.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.725+0000 7f078f7fe640 1 -- 172.21.15.114:0/1592907064 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f079c0208b0 con 0x5558c71ae790 2024-09-13T18:23:06.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.725+0000 7f078f7fe640 1 -- 172.21.15.114:0/1592907064 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079c00f450 con 0x5558c71ae790 2024-09-13T18:23:06.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.726+0000 7f07ae032740 1 -- 172.21.15.114:0/1592907064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558c71ae790 msgr2=0x5558c71aeb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:06.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.726+0000 7f07ae032740 1 --2- 172.21.15.114:0/1592907064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558c71ae790 0x5558c71aeb70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f079c013f30 tx=0x7f079c0113e0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.726+0000 7f07ae032740 1 -- 172.21.15.114:0/1592907064 shutdown_connections 2024-09-13T18:23:06.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.726+0000 7f07ae032740 1 --2- 172.21.15.114:0/1592907064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558c71b9950 0x5558c71bbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.726+0000 7f07ae032740 1 --2- 172.21.15.114:0/1592907064 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5558c71af0b0 0x5558c71b9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.726+0000 7f07ae032740 1 --2- 172.21.15.114:0/1592907064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558c71ae790 0x5558c71aeb70 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.726+0000 7f07ae032740 1 -- 172.21.15.114:0/1592907064 >> 172.21.15.114:0/1592907064 conn(0x5558c71aad00 msgr2=0x5558c71ab0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:06.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.726+0000 7f07ae032740 1 -- 172.21.15.114:0/1592907064 shutdown_connections 2024-09-13T18:23:06.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.726+0000 7f07ae032740 1 -- 172.21.15.114:0/1592907064 wait complete. 2024-09-13T18:23:06.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.727+0000 7f07ae032740 1 -- start start 2024-09-13T18:23:06.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.727+0000 7f07ae032740 1 Processor -- start 2024-09-13T18:23:06.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.727+0000 7f07ae032740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558c71ae790 0x5558c71b8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.727+0000 7f07ae032740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558c71af0b0 0x5558c71b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.727+0000 7f07ae032740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5558c71b9950 0x5558c71b6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.727+0000 7f07ae032740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558c70d0180 con 0x5558c71b9950 2024-09-13T18:23:06.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.727+0000 7f07ae032740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558c6dea430 con 0x5558c71af0b0 2024-09-13T18:23:06.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.727+0000 7f07ae032740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558c70955e0 con 0x5558c71ae790 2024-09-13T18:23:06.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.727+0000 7f07ae032740 10 client.0 _ll_register_callbacks cb 0x5558c7101aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:23:06.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.727+0000 7f07a90ad640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558c71ae790 0x5558c71b8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.728+0000 7f07a90ad640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558c71ae790 0x5558c71b8f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:32882/0 (socket says 172.21.15.114:32882) 2024-09-13T18:23:06.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.728+0000 7f07a90ad640 1 -- 172.21.15.114:0/4187809739 learned_addr learned my addr 172.21.15.114:0/4187809739 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:06.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.728+0000 7f07a98ae640 1 --2- 172.21.15.114:0/4187809739 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5558c71b9950 0x5558c71b6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.728+0000 7f07a88ac640 1 --2- 172.21.15.114:0/4187809739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558c71af0b0 0x5558c71b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.728+0000 7f07a90ad640 1 -- 172.21.15.114:0/4187809739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558c71af0b0 msgr2=0x5558c71b61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:06.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.728+0000 7f07a90ad640 1 --2- 172.21.15.114:0/4187809739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558c71af0b0 0x5558c71b61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.728+0000 7f07a90ad640 1 -- 172.21.15.114:0/4187809739 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5558c71b9950 msgr2=0x5558c71b6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:06.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.728+0000 7f07a90ad640 1 --2- 172.21.15.114:0/4187809739 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5558c71b9950 0x5558c71b6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.728+0000 7f07a90ad640 1 -- 172.21.15.114:0/4187809739 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f079c01b040 con 0x5558c71ae790 2024-09-13T18:23:06.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.728+0000 7f07a98ae640 1 --2- 172.21.15.114:0/4187809739 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5558c71b9950 0x5558c71b6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:23:06.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.728+0000 7f07a90ad640 1 --2- 172.21.15.114:0/4187809739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558c71ae790 0x5558c71b8f20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f07a00cacb0 tx=0x7f07a0099a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47ddcfea478427f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:06.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.728+0000 7f078cff9640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:23:06.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.728+0000 7f078cff9640 1 -- 172.21.15.114:0/4187809739 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a0099e20 con 0x5558c71ae790 2024-09-13T18:23:06.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.728+0000 7f078cff9640 1 -- 172.21.15.114:0/4187809739 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f07a009dc50 con 0x5558c71ae790 2024-09-13T18:23:06.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.728+0000 7f07ae032740 1 -- 172.21.15.114:0/4187809739 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5558c713bb80 con 0x5558c71ae790 2024-09-13T18:23:06.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.728+0000 7f078cff9640 1 -- 172.21.15.114:0/4187809739 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07a00a22a0 con 0x5558c71ae790 2024-09-13T18:23:06.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.728+0000 7f07ae032740 10 client.25375 fetch_fsmap learned FSMap version 30 2024-09-13T18:23:06.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.729+0000 7f07ae032740 1 -- 172.21.15.114:0/4187809739 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5558c71103c0 con 0x5558c71ae790 2024-09-13T18:23:06.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.729+0000 7f078cff9640 1 -- 172.21.15.114:0/4187809739 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f07a00b1030 con 0x5558c71ae790 2024-09-13T18:23:06.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.729+0000 7f078cff9640 1 -- 172.21.15.114:0/4187809739 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f07a0016020 con 0x5558c71ae790 2024-09-13T18:23:06.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.729+0000 7f07ae032740 10 client.25375 fetch_fsmap finished waiting for FSMap version 30 2024-09-13T18:23:06.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.729+0000 7f07ae032740 10 client.25375 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-13T18:23:06.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.729+0000 7f07ae032740 20 client.25375 populate_metadata read hostname 'smithi114' 2024-09-13T18:23:06.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.729+0000 7f07ae032740 1 --2- 172.21.15.114:0/4187809739 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x5558c71c9160 0x5558c71cb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.729+0000 7f07ae032740 4 client.25375 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:23:06.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.729+0000 7f07ae032740 1 -- 172.21.15.114:0/4187809739 --> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5558c71041f0 con 0x5558c71c9160 2024-09-13T18:23:06.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.729+0000 7f07a88ac640 1 --2- 172.21.15.114:0/4187809739 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x5558c71c9160 0x5558c71cb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.730+0000 7f078cff9640 10 client.25375 ms_handle_connect on v2:172.21.15.134:6834/1150003275 2024-09-13T18:23:06.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.730+0000 7f07a88ac640 1 --2- 172.21.15.114:0/4187809739 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x5558c71c9160 0x5558c71cb550 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3ace7c5a29530148 server_cookie=c5b75c1cfb26d2ae in_seq=0 out_seq=0 2024-09-13T18:23:06.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.730+0000 7f078cff9640 1 -- 172.21.15.114:0/4187809739 <== mds.3 v2:172.21.15.134:6834/1150003275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5558c71041f0 con 0x5558c71c9160 2024-09-13T18:23:06.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.730+0000 7f078cff9640 10 client.25375 handle_command_reply: tid=0 2024-09-13T18:23:06.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.730+0000 7f07ae032740 1 client.25375 shutdown 2024-09-13T18:23:06.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.730+0000 7f07ae032740 1 -- 172.21.15.114:0/4187809739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558c71ae790 msgr2=0x5558c71b8f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:06.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.730+0000 7f07ae032740 1 --2- 172.21.15.114:0/4187809739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558c71ae790 0x5558c71b8f20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f07a00cacb0 tx=0x7f07a0099a20 comp rx=0 tx=0).stop 2024-09-13T18:23:06.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.731+0000 7f07ae032740 1 -- 172.21.15.114:0/4187809739 shutdown_connections 2024-09-13T18:23:06.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.731+0000 7f07ae032740 1 --2- 172.21.15.114:0/4187809739 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x5558c71c9160 0x5558c71cb550 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.731+0000 7f07ae032740 1 --2- 172.21.15.114:0/4187809739 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5558c71b9950 0x5558c71b6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.731+0000 7f078cff9640 0 client.25375 ms_handle_reset on v2:172.21.15.134:6834/1150003275 2024-09-13T18:23:06.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.731+0000 7f07ae032740 1 --2- 172.21.15.114:0/4187809739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558c71af0b0 0x5558c71b61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.731+0000 7f07ae032740 1 --2- 172.21.15.114:0/4187809739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558c71ae790 0x5558c71b8f20 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.731+0000 7f07ae032740 1 -- 172.21.15.114:0/4187809739 >> 172.21.15.114:0/4187809739 conn(0x5558c71aad00 msgr2=0x5558c71b0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:06.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.731+0000 7f07ae032740 1 -- 172.21.15.114:0/4187809739 shutdown_connections 2024-09-13T18:23:06.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.731+0000 7f07ae032740 1 -- 172.21.15.114:0/4187809739 wait complete. 2024-09-13T18:23:06.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.731+0000 7f07ae032740 20 client.25375 trim_cache size 0 max 16384 2024-09-13T18:23:06.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.752+0000 7f07ae032740 1 Processor -- start 2024-09-13T18:23:06.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.752+0000 7f07ae032740 1 -- start start 2024-09-13T18:23:06.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.752+0000 7f07ae032740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558c71ae790 0x5558c71cfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.752+0000 7f07ae032740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5558c71af0b0 0x5558c71b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.752+0000 7f07ae032740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558c71b9950 0x5558c71b66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.752+0000 7f07ae032740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558c71a6440 con 0x5558c71af0b0 2024-09-13T18:23:06.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.752+0000 7f07ae032740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558c71b5a90 con 0x5558c71b9950 2024-09-13T18:23:06.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.752+0000 7f07ae032740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558c71b5c50 con 0x5558c71ae790 2024-09-13T18:23:06.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.752+0000 7f07a88ac640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5558c71af0b0 0x5558c71b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.752+0000 7f07a88ac640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5558c71af0b0 0x5558c71b61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36674/0 (socket says 172.21.15.114:36674) 2024-09-13T18:23:06.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.752+0000 7f07a88ac640 1 -- 172.21.15.114:0/3901837052 learned_addr learned my addr 172.21.15.114:0/3901837052 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:06.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.752+0000 7f07a90ad640 1 --2- 172.21.15.114:0/3901837052 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558c71ae790 0x5558c71cfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.752+0000 7f07a98ae640 1 --2- 172.21.15.114:0/3901837052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558c71b9950 0x5558c71b66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.752+0000 7f07a88ac640 1 -- 172.21.15.114:0/3901837052 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558c71ae790 msgr2=0x5558c71cfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:06.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.752+0000 7f07a88ac640 1 --2- 172.21.15.114:0/3901837052 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558c71ae790 0x5558c71cfe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.752+0000 7f07a88ac640 1 -- 172.21.15.114:0/3901837052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558c71b9950 msgr2=0x5558c71b66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:06.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.752+0000 7f07a88ac640 1 --2- 172.21.15.114:0/3901837052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558c71b9950 0x5558c71b66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.752+0000 7f07a88ac640 1 -- 172.21.15.114:0/3901837052 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558c70b2770 con 0x5558c71af0b0 2024-09-13T18:23:06.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.753+0000 7f07a88ac640 1 --2- 172.21.15.114:0/3901837052 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5558c71af0b0 0x5558c71b61b0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f0794006010 tx=0x7f0794041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c03c63db789143fa server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:06.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.753+0000 7f078ffff640 1 -- 172.21.15.114:0/3901837052 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0794051070 con 0x5558c71af0b0 2024-09-13T18:23:06.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.753+0000 7f078ffff640 1 -- 172.21.15.114:0/3901837052 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0794031d70 con 0x5558c71af0b0 2024-09-13T18:23:06.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.753+0000 7f078ffff640 1 -- 172.21.15.114:0/3901837052 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07940035f0 con 0x5558c71af0b0 2024-09-13T18:23:06.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.753+0000 7f07ae032740 1 -- 172.21.15.114:0/3901837052 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5558c71af0b0 msgr2=0x5558c71b61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:06.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.753+0000 7f07ae032740 1 --2- 172.21.15.114:0/3901837052 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5558c71af0b0 0x5558c71b61b0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f0794006010 tx=0x7f0794041000 comp rx=0 tx=0).stop 2024-09-13T18:23:06.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.754+0000 7f07ae032740 1 -- 172.21.15.114:0/3901837052 shutdown_connections 2024-09-13T18:23:06.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.754+0000 7f07ae032740 1 --2- 172.21.15.114:0/3901837052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558c71b9950 0x5558c71b66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.754+0000 7f07ae032740 1 --2- 172.21.15.114:0/3901837052 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5558c71af0b0 0x5558c71b61b0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.754+0000 7f07ae032740 1 --2- 172.21.15.114:0/3901837052 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558c71ae790 0x5558c71cfe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.754+0000 7f07ae032740 1 -- 172.21.15.114:0/3901837052 >> 172.21.15.114:0/3901837052 conn(0x5558c71aad00 msgr2=0x5558c71af930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:06.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.754+0000 7f07ae032740 1 -- 172.21.15.114:0/3901837052 shutdown_connections 2024-09-13T18:23:06.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.754+0000 7f07ae032740 1 -- 172.21.15.114:0/3901837052 wait complete. 2024-09-13T18:23:06.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.754+0000 7f07ae032740 1 -- start start 2024-09-13T18:23:06.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.754+0000 7f07ae032740 1 Processor -- start 2024-09-13T18:23:06.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.755+0000 7f07ae032740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558c71ae790 0x5558c71c09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.755+0000 7f07ae032740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5558c71b9950 0x5558c71bccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.755+0000 7f07ae032740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558c71bd200 0x5558c71bd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.755+0000 7f07ae032740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558c721f650 con 0x5558c71b9950 2024-09-13T18:23:06.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.755+0000 7f07ae032740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558c721f7c0 con 0x5558c71bd200 2024-09-13T18:23:06.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.755+0000 7f07ae032740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558c721f980 con 0x5558c71ae790 2024-09-13T18:23:06.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.755+0000 7f07ae032740 10 client.0 _ll_register_callbacks cb 0x7f079c0136e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:23:06.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.755+0000 7f07a88ac640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5558c71b9950 0x5558c71bccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.755+0000 7f07a88ac640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5558c71b9950 0x5558c71bccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36686/0 (socket says 172.21.15.114:36686) 2024-09-13T18:23:06.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.755+0000 7f07a88ac640 1 -- 172.21.15.114:0/1159085759 learned_addr learned my addr 172.21.15.114:0/1159085759 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:06.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.755+0000 7f07a90ad640 1 --2- 172.21.15.114:0/1159085759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558c71ae790 0x5558c71c09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.755+0000 7f07a98ae640 1 --2- 172.21.15.114:0/1159085759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558c71bd200 0x5558c71bd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.755+0000 7f07a90ad640 1 -- 172.21.15.114:0/1159085759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558c71bd200 msgr2=0x5558c71bd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:06.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.755+0000 7f07a90ad640 1 --2- 172.21.15.114:0/1159085759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558c71bd200 0x5558c71bd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.755+0000 7f07a90ad640 1 -- 172.21.15.114:0/1159085759 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5558c71b9950 msgr2=0x5558c71bccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:06.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.755+0000 7f07a90ad640 1 --2- 172.21.15.114:0/1159085759 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5558c71b9950 0x5558c71bccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.755+0000 7f07a90ad640 1 -- 172.21.15.114:0/1159085759 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07a410d410 con 0x5558c71ae790 2024-09-13T18:23:06.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.755+0000 7f07a88ac640 1 --2- 172.21.15.114:0/1159085759 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5558c71b9950 0x5558c71bccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:23:06.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.755+0000 7f07a90ad640 1 --2- 172.21.15.114:0/1159085759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558c71ae790 0x5558c71c09e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f07a00ca640 tx=0x7f07a009a5e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e0f50160669cf70 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:06.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.755+0000 7f078cff9640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:23:06.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.756+0000 7f078cff9640 1 -- 172.21.15.114:0/1159085759 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a0099bf0 con 0x5558c71ae790 2024-09-13T18:23:06.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.756+0000 7f07ae032740 1 -- 172.21.15.114:0/1159085759 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f079c0208b0 con 0x5558c71ae790 2024-09-13T18:23:06.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.756+0000 7f078cff9640 1 -- 172.21.15.114:0/1159085759 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f07a009dc50 con 0x5558c71ae790 2024-09-13T18:23:06.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.756+0000 7f078cff9640 1 -- 172.21.15.114:0/1159085759 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07a0096160 con 0x5558c71ae790 2024-09-13T18:23:06.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.756+0000 7f07ae032740 10 client.25381 fetch_fsmap learned FSMap version 30 2024-09-13T18:23:06.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.756+0000 7f07ae032740 1 -- 172.21.15.114:0/1159085759 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5558c70d7e80 con 0x5558c71ae790 2024-09-13T18:23:06.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.756+0000 7f078cff9640 1 -- 172.21.15.114:0/1159085759 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f079c0208b0 con 0x5558c71ae790 2024-09-13T18:23:06.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.756+0000 7f078cff9640 1 -- 172.21.15.114:0/1159085759 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f07a0016070 con 0x5558c71ae790 2024-09-13T18:23:06.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.756+0000 7f07ae032740 10 client.25381 fetch_fsmap finished waiting for FSMap version 30 2024-09-13T18:23:06.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.756+0000 7f07ae032740 10 client.25381 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-13T18:23:06.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.756+0000 7f07ae032740 20 client.25381 populate_metadata read hostname 'smithi114' 2024-09-13T18:23:06.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.756+0000 7f07ae032740 1 --2- 172.21.15.114:0/1159085759 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x5558c7220c00 0x5558c7222ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.756+0000 7f07ae032740 4 client.25381 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:23:06.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.756+0000 7f07ae032740 1 -- 172.21.15.114:0/1159085759 --> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5558c6e054e0 con 0x5558c7220c00 2024-09-13T18:23:06.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.757+0000 7f07a88ac640 1 --2- 172.21.15.114:0/1159085759 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x5558c7220c00 0x5558c7222ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.757+0000 7f078cff9640 10 client.25381 ms_handle_connect on v2:172.21.15.134:6834/1150003275 2024-09-13T18:23:06.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.757+0000 7f07a88ac640 1 --2- 172.21.15.114:0/1159085759 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x5558c7220c00 0x5558c7222ff0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f54e8660977a0727 server_cookie=fc072d088622329c in_seq=0 out_seq=0 2024-09-13T18:23:06.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.757+0000 7f078cff9640 1 -- 172.21.15.114:0/1159085759 <== mds.3 v2:172.21.15.134:6834/1150003275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5558c6e054e0 con 0x5558c7220c00 2024-09-13T18:23:06.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.757+0000 7f078cff9640 10 client.25381 handle_command_reply: tid=0 2024-09-13T18:23:06.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.757+0000 7f07ae032740 1 client.25381 shutdown 2024-09-13T18:23:06.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.757+0000 7f07ae032740 1 -- 172.21.15.114:0/1159085759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558c71ae790 msgr2=0x5558c71c09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:06.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.757+0000 7f07ae032740 1 --2- 172.21.15.114:0/1159085759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558c71ae790 0x5558c71c09e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f07a00ca640 tx=0x7f07a009a5e0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.757+0000 7f07ae032740 1 -- 172.21.15.114:0/1159085759 shutdown_connections 2024-09-13T18:23:06.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.757+0000 7f07ae032740 1 --2- 172.21.15.114:0/1159085759 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x5558c7220c00 0x5558c7222ff0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.758+0000 7f07ae032740 1 --2- 172.21.15.114:0/1159085759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558c71bd200 0x5558c71bd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.758+0000 7f078cff9640 0 client.25381 ms_handle_reset on v2:172.21.15.134:6834/1150003275 2024-09-13T18:23:06.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.758+0000 7f07ae032740 1 --2- 172.21.15.114:0/1159085759 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5558c71b9950 0x5558c71bccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.758+0000 7f07ae032740 1 --2- 172.21.15.114:0/1159085759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558c71ae790 0x5558c71c09e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.758+0000 7f07ae032740 1 -- 172.21.15.114:0/1159085759 >> 172.21.15.114:0/1159085759 conn(0x5558c71aad00 msgr2=0x5558c71b6190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:06.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.758+0000 7f07ae032740 1 -- 172.21.15.114:0/1159085759 shutdown_connections 2024-09-13T18:23:06.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.758+0000 7f07ae032740 1 -- 172.21.15.114:0/1159085759 wait complete. 2024-09-13T18:23:06.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.758+0000 7f07ae032740 20 client.25381 trim_cache size 0 max 16384 2024-09-13T18:23:06.777 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:23:06.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.758+0000 7f07ab311640 1 -- 172.21.15.114:0/2824358552 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f07900a1320 msgr2=0x7f07900a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:06.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.758+0000 7f07ab311640 1 --2- 172.21.15.114:0/2824358552 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f07900a1320 0x7f07900a37e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f079402aa70 tx=0x7f0794005a20 comp rx=0 tx=0).stop 2024-09-13T18:23:06.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.758+0000 7f07ab311640 1 -- 172.21.15.114:0/2824358552 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f07a4100f40 msgr2=0x7f07a40feb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:06.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.758+0000 7f07ab311640 1 --2- 172.21.15.114:0/2824358552 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f07a4100f40 0x7f07a40feb90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f07a000c970 tx=0x7f07a000ce30 comp rx=0 tx=0).stop 2024-09-13T18:23:06.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.758+0000 7f07ab311640 1 -- 172.21.15.114:0/2824358552 shutdown_connections 2024-09-13T18:23:06.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.758+0000 7f07ab311640 1 --2- 172.21.15.114:0/2824358552 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f07900a1320 0x7f07900a37e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.758+0000 7f07ab311640 1 --2- 172.21.15.114:0/2824358552 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07a410a310 0x7f07a40ff610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.758+0000 7f07ab311640 1 --2- 172.21.15.114:0/2824358552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f07a4101860 0x7f07a40ff0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.758+0000 7f07ab311640 1 --2- 172.21.15.114:0/2824358552 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f07a4100f40 0x7f07a40feb90 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.758+0000 7f07ab311640 1 -- 172.21.15.114:0/2824358552 >> 172.21.15.114:0/2824358552 conn(0x7f07a4075c60 msgr2=0x7f07a40fb530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:06.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.759+0000 7f07ab311640 1 -- 172.21.15.114:0/2824358552 shutdown_connections 2024-09-13T18:23:06.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.759+0000 7f07ab311640 1 -- 172.21.15.114:0/2824358552 wait complete. 2024-09-13T18:23:06.779 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-09-13T18:23:06.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.912+0000 7f2c2f42b640 1 -- 172.21.15.114:0/4211854691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c28101800 msgr2=0x7f2c28109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:06.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.912+0000 7f2c2f42b640 1 --2- 172.21.15.114:0/4211854691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c28101800 0x7f2c28109ce0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f2c18009900 tx=0x7f2c1802a410 comp rx=0 tx=0).stop 2024-09-13T18:23:06.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.913+0000 7f2c2f42b640 1 -- 172.21.15.114:0/4211854691 shutdown_connections 2024-09-13T18:23:06.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.913+0000 7f2c2f42b640 1 --2- 172.21.15.114:0/4211854691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c2810a350 0x7f2c2810c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.913+0000 7f2c2f42b640 1 --2- 172.21.15.114:0/4211854691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c28101800 0x7f2c28109ce0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.913+0000 7f2c2f42b640 1 --2- 172.21.15.114:0/4211854691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c28100e50 0x7f2c28101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.913+0000 7f2c2f42b640 1 -- 172.21.15.114:0/4211854691 >> 172.21.15.114:0/4211854691 conn(0x7f2c280fa930 msgr2=0x7f2c280fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:06.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.913+0000 7f2c2f42b640 1 -- 172.21.15.114:0/4211854691 shutdown_connections 2024-09-13T18:23:06.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.913+0000 7f2c2f42b640 1 -- 172.21.15.114:0/4211854691 wait complete. 2024-09-13T18:23:06.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.913+0000 7f2c2f42b640 1 Processor -- start 2024-09-13T18:23:06.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.914+0000 7f2c2f42b640 1 -- start start 2024-09-13T18:23:06.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.914+0000 7f2c2f42b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c28100e50 0x7f2c28196130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.914+0000 7f2c2f42b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c28101800 0x7f2c28196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.914+0000 7f2c2f42b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c2810a350 0x7f2c2819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.914+0000 7f2c2c9c6640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c28101800 0x7f2c28196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.914+0000 7f2c2f42b640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2819b060 con 0x7f2c28101800 2024-09-13T18:23:06.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.914+0000 7f2c2c9c6640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c28101800 0x7f2c28196670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36704/0 (socket says 172.21.15.114:36704) 2024-09-13T18:23:06.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.914+0000 7f2c2c9c6640 1 -- 172.21.15.114:0/2691318008 learned_addr learned my addr 172.21.15.114:0/2691318008 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:06.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.914+0000 7f2c2f42b640 1 -- 172.21.15.114:0/2691318008 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2819b1d0 con 0x7f2c2810a350 2024-09-13T18:23:06.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.914+0000 7f2c2f42b640 1 -- 172.21.15.114:0/2691318008 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2819b390 con 0x7f2c28100e50 2024-09-13T18:23:06.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.914+0000 7f2c2d1c7640 1 --2- 172.21.15.114:0/2691318008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c28100e50 0x7f2c28196130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.914+0000 7f2c2d9c8640 1 --2- 172.21.15.114:0/2691318008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c2810a350 0x7f2c2819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.914+0000 7f2c2c9c6640 1 -- 172.21.15.114:0/2691318008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c28100e50 msgr2=0x7f2c28196130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:06.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.914+0000 7f2c2c9c6640 1 --2- 172.21.15.114:0/2691318008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c28100e50 0x7f2c28196130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.914+0000 7f2c2c9c6640 1 -- 172.21.15.114:0/2691318008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c2810a350 msgr2=0x7f2c2819aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:06.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.914+0000 7f2c2c9c6640 1 --2- 172.21.15.114:0/2691318008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c2810a350 0x7f2c2819aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.914+0000 7f2c2c9c6640 1 -- 172.21.15.114:0/2691318008 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c18009560 con 0x7f2c28101800 2024-09-13T18:23:06.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.915+0000 7f2c2c9c6640 1 --2- 172.21.15.114:0/2691318008 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c28101800 0x7f2c28196670 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f2c180057c0 tx=0x7f2c18032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:06.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.915+0000 7f2c1e7fc640 1 -- 172.21.15.114:0/2691318008 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c18032e10 con 0x7f2c28101800 2024-09-13T18:23:06.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.915+0000 7f2c2f42b640 1 -- 172.21.15.114:0/2691318008 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c2819b610 con 0x7f2c28101800 2024-09-13T18:23:06.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.915+0000 7f2c1e7fc640 1 -- 172.21.15.114:0/2691318008 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2c180046c0 con 0x7f2c28101800 2024-09-13T18:23:06.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.915+0000 7f2c1e7fc640 1 -- 172.21.15.114:0/2691318008 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c18003850 con 0x7f2c28101800 2024-09-13T18:23:06.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.915+0000 7f2c2f42b640 1 -- 172.21.15.114:0/2691318008 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c2819b8e0 con 0x7f2c28101800 2024-09-13T18:23:06.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.918+0000 7f2c1e7fc640 1 -- 172.21.15.114:0/2691318008 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) v1 ==== 136107+0+0 (secure 0 0 0) 0x7f2c18005210 con 0x7f2c28101800 2024-09-13T18:23:06.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.919+0000 7f2c32226740 1 Processor -- start 2024-09-13T18:23:06.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.919+0000 7f2c32226740 1 -- start start 2024-09-13T18:23:06.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.919+0000 7f2c32226740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56388462d790 0x56388462db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.919+0000 7f2c32226740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56388462e0b0 0x563884638410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.919+0000 7f2c32226740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563884638950 0x56388463ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.919+0000 7f2c32226740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x563884597cc0 con 0x563884638950 2024-09-13T18:23:06.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.919+0000 7f2c32226740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563884595450 con 0x56388462e0b0 2024-09-13T18:23:06.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.919+0000 7f2c32226740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563884590d40 con 0x56388462d790 2024-09-13T18:23:06.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.919+0000 7f2c1e7fc640 1 --2- 172.21.15.114:0/2691318008 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2c140a1320 0x7f2c140a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.919+0000 7f2c2d9c8640 1 --2- 172.21.15.114:0/2691318008 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2c140a1320 0x7f2c140a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.919+0000 7f2c2d9c8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56388462d790 0x56388462db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.919+0000 7f2c2d9c8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56388462d790 0x56388462db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:32942/0 (socket says 172.21.15.114:32942) 2024-09-13T18:23:06.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.919+0000 7f2c2d9c8640 1 -- 172.21.15.114:0/2446270513 learned_addr learned my addr 172.21.15.114:0/2446270513 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:06.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.919+0000 7f2c1e7fc640 1 -- 172.21.15.114:0/2691318008 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2c180ee190 con 0x7f2c28101800 2024-09-13T18:23:06.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.919+0000 7f2c2c9c6640 1 --2- 172.21.15.114:0/2446270513 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563884638950 0x56388463ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.919+0000 7f2c2d1c7640 1 --2- 172.21.15.114:0/2446270513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56388462e0b0 0x563884638410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.920+0000 7f2c2d9c8640 1 -- 172.21.15.114:0/2446270513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56388462e0b0 msgr2=0x563884638410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:06.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.920+0000 7f2c2d9c8640 1 --2- 172.21.15.114:0/2446270513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56388462e0b0 0x563884638410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.920+0000 7f2c2d9c8640 1 -- 172.21.15.114:0/2446270513 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563884638950 msgr2=0x56388463ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:06.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.920+0000 7f2c2d9c8640 1 --2- 172.21.15.114:0/2446270513 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563884638950 0x56388463ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.920+0000 7f2c2d9c8640 1 -- 172.21.15.114:0/2446270513 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563884558620 con 0x56388462d790 2024-09-13T18:23:06.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.920+0000 7f2c2d9c8640 1 --2- 172.21.15.114:0/2446270513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56388462d790 0x56388462db70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2c20012d10 tx=0x7f2c2000f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30e1c027269635b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:06.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.920+0000 7f2c2d9c8640 1 --2- 172.21.15.114:0/2691318008 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2c140a1320 0x7f2c140a37e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2c2000fc30 tx=0x7f2c20017000 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:06.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.920+0000 7f2c137fe640 1 -- 172.21.15.114:0/2446270513 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c20012e00 con 0x56388462d790 2024-09-13T18:23:06.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.920+0000 7f2c137fe640 1 -- 172.21.15.114:0/2446270513 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2c20009d10 con 0x56388462d790 2024-09-13T18:23:06.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.920+0000 7f2c137fe640 1 -- 172.21.15.114:0/2446270513 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2001fb70 con 0x56388462d790 2024-09-13T18:23:06.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.921+0000 7f2c32226740 1 -- 172.21.15.114:0/2446270513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56388462d790 msgr2=0x56388462db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:06.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.921+0000 7f2c32226740 1 --2- 172.21.15.114:0/2446270513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56388462d790 0x56388462db70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2c20012d10 tx=0x7f2c2000f730 comp rx=0 tx=0).stop 2024-09-13T18:23:06.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.921+0000 7f2c32226740 1 -- 172.21.15.114:0/2446270513 shutdown_connections 2024-09-13T18:23:06.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.921+0000 7f2c32226740 1 --2- 172.21.15.114:0/2446270513 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563884638950 0x56388463ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.921+0000 7f2c32226740 1 --2- 172.21.15.114:0/2446270513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56388462e0b0 0x563884638410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.921+0000 7f2c32226740 1 --2- 172.21.15.114:0/2446270513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56388462d790 0x56388462db70 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.921+0000 7f2c32226740 1 -- 172.21.15.114:0/2446270513 >> 172.21.15.114:0/2446270513 conn(0x563884629d00 msgr2=0x56388462a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:06.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.921+0000 7f2c32226740 1 -- 172.21.15.114:0/2446270513 shutdown_connections 2024-09-13T18:23:06.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.921+0000 7f2c32226740 1 -- 172.21.15.114:0/2446270513 wait complete. 2024-09-13T18:23:06.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.921+0000 7f2c32226740 1 -- start start 2024-09-13T18:23:06.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.921+0000 7f2c32226740 1 Processor -- start 2024-09-13T18:23:06.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.922+0000 7f2c32226740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56388462d790 0x563884637ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.922+0000 7f2c32226740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56388462e0b0 0x5638846353e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.922+0000 7f2c32226740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563884638950 0x563884635920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.922+0000 7f2c32226740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56388454f180 con 0x56388462e0b0 2024-09-13T18:23:06.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.922+0000 7f2c32226740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563884269430 con 0x56388462d790 2024-09-13T18:23:06.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.922+0000 7f2c32226740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638845145e0 con 0x563884638950 2024-09-13T18:23:06.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.922+0000 7f2c32226740 10 client.0 _ll_register_callbacks cb 0x563884580aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:23:06.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.922+0000 7f2c2c9c6640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56388462e0b0 0x5638846353e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.922+0000 7f2c2c9c6640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56388462e0b0 0x5638846353e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36724/0 (socket says 172.21.15.114:36724) 2024-09-13T18:23:06.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.922+0000 7f2c2c9c6640 1 -- 172.21.15.114:0/4047487536 learned_addr learned my addr 172.21.15.114:0/4047487536 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:06.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.922+0000 7f2c2d9c8640 1 --2- 172.21.15.114:0/4047487536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563884638950 0x563884635920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.922+0000 7f2c2d1c7640 1 --2- 172.21.15.114:0/4047487536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56388462d790 0x563884637ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.922+0000 7f2c2c9c6640 1 -- 172.21.15.114:0/4047487536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563884638950 msgr2=0x563884635920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:06.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.922+0000 7f2c2c9c6640 1 --2- 172.21.15.114:0/4047487536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563884638950 0x563884635920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.922+0000 7f2c2c9c6640 1 -- 172.21.15.114:0/4047487536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56388462d790 msgr2=0x563884637ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:06.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.922+0000 7f2c2c9c6640 1 --2- 172.21.15.114:0/4047487536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56388462d790 0x563884637ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.922+0000 7f2c2c9c6640 1 -- 172.21.15.114:0/4047487536 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c2001a040 con 0x56388462e0b0 2024-09-13T18:23:06.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.922+0000 7f2c2d1c7640 1 --2- 172.21.15.114:0/4047487536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56388462d790 0x563884637ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:23:06.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.922+0000 7f2c2c9c6640 1 --2- 172.21.15.114:0/4047487536 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56388462e0b0 0x5638846353e0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f2c180bad60 tx=0x7f2c180ba7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c55b0708a10fdec server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:06.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.922+0000 7f2c10ff9640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:23:06.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.923+0000 7f2c10ff9640 1 -- 172.21.15.114:0/4047487536 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c180badd0 con 0x56388462e0b0 2024-09-13T18:23:06.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.923+0000 7f2c32226740 1 -- 172.21.15.114:0/4047487536 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5638845bab80 con 0x56388462e0b0 2024-09-13T18:23:06.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.923+0000 7f2c10ff9640 1 -- 172.21.15.114:0/4047487536 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2c180bec00 con 0x56388462e0b0 2024-09-13T18:23:06.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.923+0000 7f2c10ff9640 1 -- 172.21.15.114:0/4047487536 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c180c3250 con 0x56388462e0b0 2024-09-13T18:23:06.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.923+0000 7f2c32226740 10 client.15930 fetch_fsmap learned FSMap version 30 2024-09-13T18:23:06.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.923+0000 7f2c32226740 1 -- 172.21.15.114:0/4047487536 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56388458f3c0 con 0x56388462e0b0 2024-09-13T18:23:06.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.923+0000 7f2c10ff9640 1 -- 172.21.15.114:0/4047487536 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5638845bab80 con 0x56388462e0b0 2024-09-13T18:23:06.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.923+0000 7f2c10ff9640 1 -- 172.21.15.114:0/4047487536 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2c1804d070 con 0x56388462e0b0 2024-09-13T18:23:06.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.923+0000 7f2c32226740 10 client.15930 fetch_fsmap finished waiting for FSMap version 30 2024-09-13T18:23:06.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.923+0000 7f2c32226740 10 client.15930 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-13T18:23:06.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.923+0000 7f2c32226740 20 client.15930 populate_metadata read hostname 'smithi114' 2024-09-13T18:23:06.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.923+0000 7f2c32226740 1 --2- 172.21.15.114:0/4047487536 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x563884648060 0x56388464a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.924+0000 7f2c32226740 4 client.15930 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:23:06.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.924+0000 7f2c32226740 1 -- 172.21.15.114:0/4047487536 --> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5638845831f0 con 0x563884648060 2024-09-13T18:23:06.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.924+0000 7f2c2d1c7640 1 --2- 172.21.15.114:0/4047487536 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x563884648060 0x56388464a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.924+0000 7f2c2d1c7640 1 --2- 172.21.15.114:0/4047487536 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x563884648060 0x56388464a480 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4561945cfd21c7a6 server_cookie=82545c5f57a9a1b0 in_seq=0 out_seq=0 2024-09-13T18:23:06.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.924+0000 7f2c10ff9640 10 client.15930 ms_handle_connect on v2:172.21.15.204:6838/344883912 2024-09-13T18:23:06.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.925+0000 7f2c10ff9640 1 -- 172.21.15.114:0/4047487536 <== mds.2 v2:172.21.15.204:6838/344883912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5638845831f0 con 0x563884648060 2024-09-13T18:23:06.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.925+0000 7f2c10ff9640 10 client.15930 handle_command_reply: tid=0 2024-09-13T18:23:06.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.925+0000 7f2c32226740 1 client.15930 shutdown 2024-09-13T18:23:06.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.925+0000 7f2c32226740 1 -- 172.21.15.114:0/4047487536 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56388462e0b0 msgr2=0x5638846353e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:06.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.925+0000 7f2c32226740 1 --2- 172.21.15.114:0/4047487536 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56388462e0b0 0x5638846353e0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f2c180bad60 tx=0x7f2c180ba7f0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.925+0000 7f2c32226740 1 -- 172.21.15.114:0/4047487536 shutdown_connections 2024-09-13T18:23:06.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.925+0000 7f2c32226740 1 --2- 172.21.15.114:0/4047487536 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x563884648060 0x56388464a480 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.925+0000 7f2c32226740 1 --2- 172.21.15.114:0/4047487536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563884638950 0x563884635920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.925+0000 7f2c10ff9640 0 client.15930 ms_handle_reset on v2:172.21.15.204:6838/344883912 2024-09-13T18:23:06.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.925+0000 7f2c32226740 1 --2- 172.21.15.114:0/4047487536 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56388462e0b0 0x5638846353e0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.925+0000 7f2c32226740 1 --2- 172.21.15.114:0/4047487536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56388462d790 0x563884637ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.925+0000 7f2c32226740 1 -- 172.21.15.114:0/4047487536 >> 172.21.15.114:0/4047487536 conn(0x563884629d00 msgr2=0x56388462fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:06.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.925+0000 7f2c32226740 1 -- 172.21.15.114:0/4047487536 shutdown_connections 2024-09-13T18:23:06.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.925+0000 7f2c32226740 1 -- 172.21.15.114:0/4047487536 wait complete. 2024-09-13T18:23:06.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.926+0000 7f2c32226740 20 client.15930 trim_cache size 0 max 16384 2024-09-13T18:23:06.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.945+0000 7f2c32226740 1 Processor -- start 2024-09-13T18:23:06.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.945+0000 7f2c32226740 1 -- start start 2024-09-13T18:23:06.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.946+0000 7f2c32226740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563884629d00 0x5638846440d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.946+0000 7f2c32226740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56388462d790 0x563884644610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.946+0000 7f2c32226740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563884638950 0x563884633c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.946+0000 7f2c32226740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x563884635240 con 0x563884629d00 2024-09-13T18:23:06.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.946+0000 7f2c32226740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56388462d5f0 con 0x563884638950 2024-09-13T18:23:06.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.946+0000 7f2c32226740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56388462ac60 con 0x56388462d790 2024-09-13T18:23:06.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.946+0000 7f2c2d1c7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563884629d00 0x5638846440d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.946+0000 7f2c2d1c7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563884629d00 0x5638846440d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36730/0 (socket says 172.21.15.114:36730) 2024-09-13T18:23:06.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.946+0000 7f2c2d1c7640 1 -- 172.21.15.114:0/876178930 learned_addr learned my addr 172.21.15.114:0/876178930 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:06.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.946+0000 7f2c2c9c6640 1 --2- 172.21.15.114:0/876178930 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56388462d790 0x563884644610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.946+0000 7f2c2d9c8640 1 --2- 172.21.15.114:0/876178930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563884638950 0x563884633c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.946+0000 7f2c2d1c7640 1 -- 172.21.15.114:0/876178930 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56388462d790 msgr2=0x563884644610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:06.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.946+0000 7f2c2d1c7640 1 --2- 172.21.15.114:0/876178930 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56388462d790 0x563884644610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.946+0000 7f2c2d1c7640 1 -- 172.21.15.114:0/876178930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563884638950 msgr2=0x563884633c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:06.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.946+0000 7f2c2d1c7640 1 --2- 172.21.15.114:0/876178930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563884638950 0x563884633c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.946+0000 7f2c2d1c7640 1 -- 172.21.15.114:0/876178930 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563884531770 con 0x563884629d00 2024-09-13T18:23:06.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.946+0000 7f2c2d1c7640 1 --2- 172.21.15.114:0/876178930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563884629d00 0x5638846440d0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f2c24000c00 tx=0x7f2c2400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39ffdfd4b381715d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:06.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.947+0000 7f2c13fff640 1 -- 172.21.15.114:0/876178930 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c24017040 con 0x563884629d00 2024-09-13T18:23:06.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.947+0000 7f2c13fff640 1 -- 172.21.15.114:0/876178930 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2c24013e60 con 0x563884629d00 2024-09-13T18:23:06.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.947+0000 7f2c13fff640 1 -- 172.21.15.114:0/876178930 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c24019510 con 0x563884629d00 2024-09-13T18:23:06.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.948+0000 7f2c32226740 1 -- 172.21.15.114:0/876178930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563884629d00 msgr2=0x5638846440d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:06.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.948+0000 7f2c32226740 1 --2- 172.21.15.114:0/876178930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563884629d00 0x5638846440d0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f2c24000c00 tx=0x7f2c2400e720 comp rx=0 tx=0).stop 2024-09-13T18:23:06.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.948+0000 7f2c32226740 1 -- 172.21.15.114:0/876178930 shutdown_connections 2024-09-13T18:23:06.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.948+0000 7f2c32226740 1 --2- 172.21.15.114:0/876178930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563884638950 0x563884633c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.948+0000 7f2c32226740 1 --2- 172.21.15.114:0/876178930 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56388462d790 0x563884644610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.948+0000 7f2c32226740 1 --2- 172.21.15.114:0/876178930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563884629d00 0x5638846440d0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.948+0000 7f2c32226740 1 -- 172.21.15.114:0/876178930 >> 172.21.15.114:0/876178930 conn(0x563884638150 msgr2=0x5638846353e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:06.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.948+0000 7f2c32226740 1 -- 172.21.15.114:0/876178930 shutdown_connections 2024-09-13T18:23:06.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.948+0000 7f2c32226740 1 -- 172.21.15.114:0/876178930 wait complete. 2024-09-13T18:23:06.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.948+0000 7f2c32226740 1 -- start start 2024-09-13T18:23:06.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.948+0000 7f2c32226740 1 Processor -- start 2024-09-13T18:23:06.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.949+0000 7f2c32226740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563884629d00 0x563884632340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.949+0000 7f2c32226740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56388462d790 0x563884632880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.949+0000 7f2c32226740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563884638950 0x563884632dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.949+0000 7f2c32226740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56388469e680 con 0x563884629d00 2024-09-13T18:23:06.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.949+0000 7f2c32226740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56388469e7f0 con 0x563884638950 2024-09-13T18:23:06.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.949+0000 7f2c32226740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56388469e9b0 con 0x56388462d790 2024-09-13T18:23:06.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.949+0000 7f2c32226740 10 client.0 _ll_register_callbacks cb 0x7f2c200123f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:23:06.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.949+0000 7f2c2d1c7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563884629d00 0x563884632340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.949+0000 7f2c2d1c7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563884629d00 0x563884632340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36738/0 (socket says 172.21.15.114:36738) 2024-09-13T18:23:06.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.949+0000 7f2c2d1c7640 1 -- 172.21.15.114:0/315886789 learned_addr learned my addr 172.21.15.114:0/315886789 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:06.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.949+0000 7f2c2d9c8640 1 --2- 172.21.15.114:0/315886789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563884638950 0x563884632dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.949+0000 7f2c2c9c6640 1 --2- 172.21.15.114:0/315886789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56388462d790 0x563884632880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.949+0000 7f2c2d1c7640 1 -- 172.21.15.114:0/315886789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56388462d790 msgr2=0x563884632880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:06.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.949+0000 7f2c2d1c7640 1 --2- 172.21.15.114:0/315886789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56388462d790 0x563884632880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.949+0000 7f2c2d1c7640 1 -- 172.21.15.114:0/315886789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563884638950 msgr2=0x563884632dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:06.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.949+0000 7f2c2d1c7640 1 --2- 172.21.15.114:0/315886789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563884638950 0x563884632dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.949+0000 7f2c2d1c7640 1 -- 172.21.15.114:0/315886789 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c2400ee50 con 0x563884629d00 2024-09-13T18:23:06.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.949+0000 7f2c2d1c7640 1 --2- 172.21.15.114:0/315886789 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563884629d00 0x563884632340 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f2c2400e660 tx=0x7f2c240192b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e83a2f4067036a7d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:06.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.949+0000 7f2c10ff9640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:23:06.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.949+0000 7f2c10ff9640 1 -- 172.21.15.114:0/315886789 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c240194d0 con 0x563884629d00 2024-09-13T18:23:06.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.949+0000 7f2c10ff9640 1 -- 172.21.15.114:0/315886789 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2c240139d0 con 0x563884629d00 2024-09-13T18:23:06.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.949+0000 7f2c32226740 1 -- 172.21.15.114:0/315886789 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2c20009d10 con 0x563884629d00 2024-09-13T18:23:06.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.950+0000 7f2c10ff9640 1 -- 172.21.15.114:0/315886789 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c24013bd0 con 0x563884629d00 2024-09-13T18:23:06.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.950+0000 7f2c32226740 10 client.15936 fetch_fsmap learned FSMap version 30 2024-09-13T18:23:06.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.950+0000 7f2c32226740 1 -- 172.21.15.114:0/315886789 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x563884556e80 con 0x563884629d00 2024-09-13T18:23:06.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.950+0000 7f2c10ff9640 1 -- 172.21.15.114:0/315886789 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2c20009d10 con 0x563884629d00 2024-09-13T18:23:06.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.950+0000 7f2c10ff9640 1 -- 172.21.15.114:0/315886789 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2c24004aa0 con 0x563884629d00 2024-09-13T18:23:06.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.950+0000 7f2c32226740 10 client.15936 fetch_fsmap finished waiting for FSMap version 30 2024-09-13T18:23:06.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.950+0000 7f2c32226740 10 client.15936 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-13T18:23:06.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.950+0000 7f2c32226740 20 client.15936 populate_metadata read hostname 'smithi114' 2024-09-13T18:23:06.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.950+0000 7f2c32226740 1 --2- 172.21.15.114:0/315886789 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x563884648060 0x5638846a2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:06.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.950+0000 7f2c32226740 4 client.15936 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:23:06.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.950+0000 7f2c32226740 1 -- 172.21.15.114:0/315886789 --> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5638842844e0 con 0x563884648060 2024-09-13T18:23:06.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.950+0000 7f2c2c9c6640 1 --2- 172.21.15.114:0/315886789 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x563884648060 0x5638846a2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:06.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.951+0000 7f2c10ff9640 10 client.15936 ms_handle_connect on v2:172.21.15.204:6838/344883912 2024-09-13T18:23:06.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.951+0000 7f2c2c9c6640 1 --2- 172.21.15.114:0/315886789 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x563884648060 0x5638846a2160 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5027c07182f90e20 server_cookie=4784763574c91da in_seq=0 out_seq=0 2024-09-13T18:23:06.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.951+0000 7f2c10ff9640 1 -- 172.21.15.114:0/315886789 <== mds.2 v2:172.21.15.204:6838/344883912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5638842844e0 con 0x563884648060 2024-09-13T18:23:06.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.951+0000 7f2c10ff9640 10 client.15936 handle_command_reply: tid=0 2024-09-13T18:23:06.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.951+0000 7f2c32226740 1 client.15936 shutdown 2024-09-13T18:23:06.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.951+0000 7f2c32226740 1 -- 172.21.15.114:0/315886789 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563884629d00 msgr2=0x563884632340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:06.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.951+0000 7f2c32226740 1 --2- 172.21.15.114:0/315886789 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563884629d00 0x563884632340 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f2c2400e660 tx=0x7f2c240192b0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.951+0000 7f2c32226740 1 -- 172.21.15.114:0/315886789 shutdown_connections 2024-09-13T18:23:06.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.951+0000 7f2c32226740 1 --2- 172.21.15.114:0/315886789 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x563884648060 0x5638846a2160 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.951+0000 7f2c32226740 1 --2- 172.21.15.114:0/315886789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563884638950 0x563884632dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.951+0000 7f2c10ff9640 0 client.15936 ms_handle_reset on v2:172.21.15.204:6838/344883912 2024-09-13T18:23:06.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.951+0000 7f2c32226740 1 --2- 172.21.15.114:0/315886789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56388462d790 0x563884632880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.951+0000 7f2c32226740 1 --2- 172.21.15.114:0/315886789 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563884629d00 0x563884632340 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.951+0000 7f2c32226740 1 -- 172.21.15.114:0/315886789 >> 172.21.15.114:0/315886789 conn(0x563884638150 msgr2=0x5638846353e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:06.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.951+0000 7f2c32226740 1 -- 172.21.15.114:0/315886789 shutdown_connections 2024-09-13T18:23:06.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.951+0000 7f2c32226740 1 -- 172.21.15.114:0/315886789 wait complete. 2024-09-13T18:23:06.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.951+0000 7f2c32226740 20 client.15936 trim_cache size 0 max 16384 2024-09-13T18:23:06.961 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:23:06.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.952+0000 7f2c2f42b640 1 -- 172.21.15.114:0/2691318008 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2c140a1320 msgr2=0x7f2c140a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:06.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.952+0000 7f2c2f42b640 1 --2- 172.21.15.114:0/2691318008 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2c140a1320 0x7f2c140a37e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2c2000fc30 tx=0x7f2c20017000 comp rx=0 tx=0).stop 2024-09-13T18:23:06.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.952+0000 7f2c2f42b640 1 -- 172.21.15.114:0/2691318008 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c28101800 msgr2=0x7f2c28196670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:06.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.952+0000 7f2c2f42b640 1 --2- 172.21.15.114:0/2691318008 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c28101800 0x7f2c28196670 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f2c180057c0 tx=0x7f2c18032c20 comp rx=0 tx=0).stop 2024-09-13T18:23:06.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.952+0000 7f2c2f42b640 1 -- 172.21.15.114:0/2691318008 shutdown_connections 2024-09-13T18:23:06.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.952+0000 7f2c2f42b640 1 --2- 172.21.15.114:0/2691318008 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2c140a1320 0x7f2c140a37e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.952+0000 7f2c2f42b640 1 --2- 172.21.15.114:0/2691318008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c2810a350 0x7f2c2819aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.952+0000 7f2c2f42b640 1 --2- 172.21.15.114:0/2691318008 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c28101800 0x7f2c28196670 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.952+0000 7f2c2f42b640 1 --2- 172.21.15.114:0/2691318008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c28100e50 0x7f2c28196130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:06.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.952+0000 7f2c2f42b640 1 -- 172.21.15.114:0/2691318008 >> 172.21.15.114:0/2691318008 conn(0x7f2c280fa930 msgr2=0x7f2c280fc290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:06.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.952+0000 7f2c2f42b640 1 -- 172.21.15.114:0/2691318008 shutdown_connections 2024-09-13T18:23:06.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:06.952+0000 7f2c2f42b640 1 -- 172.21.15.114:0/2691318008 wait complete. 2024-09-13T18:23:06.968 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-09-13T18:23:07.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.069+0000 7fdff97e9640 1 -- 172.21.15.114:0/1776563554 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdff40ff810 msgr2=0x7fdff4103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:07.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.069+0000 7fdff97e9640 1 --2- 172.21.15.114:0/1776563554 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdff40ff810 0x7fdff4103850 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8009930 tx=0x7fdfe802a420 comp rx=0 tx=0).stop 2024-09-13T18:23:07.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.070+0000 7fdff97e9640 1 -- 172.21.15.114:0/1776563554 shutdown_connections 2024-09-13T18:23:07.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.070+0000 7fdff97e9640 1 --2- 172.21.15.114:0/1776563554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdff4103fb0 0x7fdff410bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.070+0000 7fdff97e9640 1 --2- 172.21.15.114:0/1776563554 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdff40ff810 0x7fdff4103850 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.070+0000 7fdff97e9640 1 --2- 172.21.15.114:0/1776563554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdff40fee60 0x7fdff40ff240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.070+0000 7fdff97e9640 1 -- 172.21.15.114:0/1776563554 >> 172.21.15.114:0/1776563554 conn(0x7fdff4075c60 msgr2=0x7fdff40fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:07.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.070+0000 7fdff97e9640 1 -- 172.21.15.114:0/1776563554 shutdown_connections 2024-09-13T18:23:07.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.070+0000 7fdff97e9640 1 -- 172.21.15.114:0/1776563554 wait complete. 2024-09-13T18:23:07.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.070+0000 7fdff97e9640 1 Processor -- start 2024-09-13T18:23:07.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.070+0000 7fdff97e9640 1 -- start start 2024-09-13T18:23:07.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.071+0000 7fdff97e9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdff40fee60 0x7fdff4197030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:07.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.071+0000 7fdff97e9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdff4103fb0 0x7fdff4197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:07.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.071+0000 7fdff97e9640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdff4197ab0 0x7fdff41a20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:07.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.071+0000 7fdff97e9640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff41a2750 con 0x7fdff4197ab0 2024-09-13T18:23:07.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.071+0000 7fdff97e9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff41a28c0 con 0x7fdff40fee60 2024-09-13T18:23:07.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.071+0000 7fdff97e9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff41a2a80 con 0x7fdff4103fb0 2024-09-13T18:23:07.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.071+0000 7fdff27fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdff4103fb0 0x7fdff4197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:07.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.071+0000 7fdff37fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdff4197ab0 0x7fdff41a20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:07.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.071+0000 7fdff27fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdff4103fb0 0x7fdff4197570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:32986/0 (socket says 172.21.15.114:32986) 2024-09-13T18:23:07.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.071+0000 7fdff27fc640 1 -- 172.21.15.114:0/2381371833 learned_addr learned my addr 172.21.15.114:0/2381371833 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:07.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.071+0000 7fdff2ffd640 1 --2- 172.21.15.114:0/2381371833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdff40fee60 0x7fdff4197030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:07.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.071+0000 7fdff37fe640 1 -- 172.21.15.114:0/2381371833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdff4103fb0 msgr2=0x7fdff4197570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:07.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.071+0000 7fdff37fe640 1 --2- 172.21.15.114:0/2381371833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdff4103fb0 0x7fdff4197570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.071+0000 7fdff37fe640 1 -- 172.21.15.114:0/2381371833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdff40fee60 msgr2=0x7fdff4197030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:07.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.071+0000 7fdff37fe640 1 --2- 172.21.15.114:0/2381371833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdff40fee60 0x7fdff4197030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.071+0000 7fdff37fe640 1 -- 172.21.15.114:0/2381371833 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe8009560 con 0x7fdff4197ab0 2024-09-13T18:23:07.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.071+0000 7fdff37fe640 1 --2- 172.21.15.114:0/2381371833 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdff4197ab0 0x7fdff41a20b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fdfec00d9c0 tx=0x7fdfec00de80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:07.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.072+0000 7fdfdbfff640 1 -- 172.21.15.114:0/2381371833 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfec00bd10 con 0x7fdff4197ab0 2024-09-13T18:23:07.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.072+0000 7fdfdbfff640 1 -- 172.21.15.114:0/2381371833 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdfec00be70 con 0x7fdff4197ab0 2024-09-13T18:23:07.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.072+0000 7fdff97e9640 1 -- 172.21.15.114:0/2381371833 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdff41a2d60 con 0x7fdff4197ab0 2024-09-13T18:23:07.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.072+0000 7fdfdbfff640 1 -- 172.21.15.114:0/2381371833 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfec010460 con 0x7fdff4197ab0 2024-09-13T18:23:07.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.072+0000 7fdff97e9640 1 -- 172.21.15.114:0/2381371833 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdff41a3240 con 0x7fdff4197ab0 2024-09-13T18:23:07.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.074+0000 7fdffc5e5740 1 Processor -- start 2024-09-13T18:23:07.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.075+0000 7fdfdbfff640 1 -- 172.21.15.114:0/2381371833 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) v1 ==== 136107+0+0 (secure 0 0 0) 0x7fdfec004000 con 0x7fdff4197ab0 2024-09-13T18:23:07.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.075+0000 7fdffc5e5740 1 -- start start 2024-09-13T18:23:07.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.075+0000 7fdffc5e5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8974ed790 0x55e8974edb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:07.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.075+0000 7fdffc5e5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e8974ee0b0 0x55e8974f8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:07.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.075+0000 7fdffc5e5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8974f8950 0x55e8974fad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:07.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.075+0000 7fdffc5e5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e897457cc0 con 0x55e8974ee0b0 2024-09-13T18:23:07.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.075+0000 7fdffc5e5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e897455450 con 0x55e8974f8950 2024-09-13T18:23:07.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.075+0000 7fdffc5e5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e897450d40 con 0x55e8974ed790 2024-09-13T18:23:07.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.075+0000 7fdff2ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e8974ee0b0 0x55e8974f8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:07.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.075+0000 7fdff2ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e8974ee0b0 0x55e8974f8410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36772/0 (socket says 172.21.15.114:36772) 2024-09-13T18:23:07.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.075+0000 7fdff2ffd640 1 -- 172.21.15.114:0/635282947 learned_addr learned my addr 172.21.15.114:0/635282947 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:07.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.076+0000 7fdff27fc640 1 --2- 172.21.15.114:0/635282947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8974ed790 0x55e8974edb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:07.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.076+0000 7fdff27fc640 1 --2- 172.21.15.114:0/635282947 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8974f8950 0x55e8974fad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:07.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.076+0000 7fdfdbfff640 1 --2- 172.21.15.114:0/2381371833 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fdfdc0a1470 0x7fdfdc0a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:07.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.076+0000 7fdff2ffd640 1 -- 172.21.15.114:0/635282947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8974ed790 msgr2=0x55e8974edb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:07.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.076+0000 7fdff2ffd640 1 --2- 172.21.15.114:0/635282947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8974ed790 0x55e8974edb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.076+0000 7fdff2ffd640 1 -- 172.21.15.114:0/635282947 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8974f8950 msgr2=0x55e8974fad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:07.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.076+0000 7fdff2ffd640 1 --2- 172.21.15.114:0/635282947 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8974f8950 0x55e8974fad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.076+0000 7fdff2ffd640 1 -- 172.21.15.114:0/635282947 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e897418620 con 0x55e8974ee0b0 2024-09-13T18:23:07.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.076+0000 7fdff37fe640 1 --2- 172.21.15.114:0/2381371833 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fdfdc0a1470 0x7fdfdc0a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:07.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.076+0000 7fdff2ffd640 1 --2- 172.21.15.114:0/635282947 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e8974ee0b0 0x55e8974f8410 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fdfe400bd10 tx=0x7fdfe40107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f09381990609b8a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:07.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.076+0000 7fdfdbfff640 1 -- 172.21.15.114:0/2381371833 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fdfec0ccea0 con 0x7fdff4197ab0 2024-09-13T18:23:07.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.076+0000 7fdfd97fa640 1 -- 172.21.15.114:0/635282947 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe4004030 con 0x55e8974ee0b0 2024-09-13T18:23:07.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.076+0000 7fdfd97fa640 1 -- 172.21.15.114:0/635282947 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdfe4015070 con 0x55e8974ee0b0 2024-09-13T18:23:07.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.076+0000 7fdfd97fa640 1 -- 172.21.15.114:0/635282947 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe40116a0 con 0x55e8974ee0b0 2024-09-13T18:23:07.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.077+0000 7fdffc5e5740 1 -- 172.21.15.114:0/635282947 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e8974ee0b0 msgr2=0x55e8974f8410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:07.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.077+0000 7fdff37fe640 1 --2- 172.21.15.114:0/2381371833 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fdfdc0a1470 0x7fdfdc0a3930 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdfec0ca120 tx=0x7fdfec004a40 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:07.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.077+0000 7fdffc5e5740 1 --2- 172.21.15.114:0/635282947 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e8974ee0b0 0x55e8974f8410 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fdfe400bd10 tx=0x7fdfe40107a0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.077+0000 7fdffc5e5740 1 -- 172.21.15.114:0/635282947 shutdown_connections 2024-09-13T18:23:07.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.077+0000 7fdffc5e5740 1 --2- 172.21.15.114:0/635282947 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8974f8950 0x55e8974fad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.077+0000 7fdffc5e5740 1 --2- 172.21.15.114:0/635282947 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e8974ee0b0 0x55e8974f8410 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.077+0000 7fdffc5e5740 1 --2- 172.21.15.114:0/635282947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8974ed790 0x55e8974edb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.077+0000 7fdffc5e5740 1 -- 172.21.15.114:0/635282947 >> 172.21.15.114:0/635282947 conn(0x55e8974e9d00 msgr2=0x55e8974ea0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:07.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.077+0000 7fdffc5e5740 1 -- 172.21.15.114:0/635282947 shutdown_connections 2024-09-13T18:23:07.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.077+0000 7fdffc5e5740 1 -- 172.21.15.114:0/635282947 wait complete. 2024-09-13T18:23:07.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.077+0000 7fdffc5e5740 1 -- start start 2024-09-13T18:23:07.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.078+0000 7fdffc5e5740 1 Processor -- start 2024-09-13T18:23:07.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.078+0000 7fdffc5e5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e8974ed790 0x55e8974f7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:07.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.078+0000 7fdffc5e5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8974ee0b0 0x55e8974f5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:07.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.078+0000 7fdffc5e5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8974f8950 0x55e8974f57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:07.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.078+0000 7fdffc5e5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e89740f180 con 0x55e8974ed790 2024-09-13T18:23:07.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.078+0000 7fdffc5e5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e897129430 con 0x55e8974f8950 2024-09-13T18:23:07.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.078+0000 7fdffc5e5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8973d45e0 con 0x55e8974ee0b0 2024-09-13T18:23:07.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.078+0000 7fdffc5e5740 10 client.0 _ll_register_callbacks cb 0x55e897440aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:23:07.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.078+0000 7fdff27fc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e8974ed790 0x55e8974f7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:07.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.078+0000 7fdff27fc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e8974ed790 0x55e8974f7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36782/0 (socket says 172.21.15.114:36782) 2024-09-13T18:23:07.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.078+0000 7fdff27fc640 1 -- 172.21.15.114:0/2785054362 learned_addr learned my addr 172.21.15.114:0/2785054362 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:07.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.078+0000 7fdff27fc640 1 --2- 172.21.15.114:0/2785054362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8974f8950 0x55e8974f57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:07.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.078+0000 7fdff2ffd640 1 --2- 172.21.15.114:0/2785054362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8974ee0b0 0x55e8974f5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:07.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.078+0000 7fdff27fc640 1 -- 172.21.15.114:0/2785054362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8974ee0b0 msgr2=0x55e8974f5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:07.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.078+0000 7fdff27fc640 1 --2- 172.21.15.114:0/2785054362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8974ee0b0 0x55e8974f5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.078+0000 7fdff27fc640 1 -- 172.21.15.114:0/2785054362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8974f8950 msgr2=0x55e8974f57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:07.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.078+0000 7fdff27fc640 1 --2- 172.21.15.114:0/2785054362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8974f8950 0x55e8974f57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.078+0000 7fdff27fc640 1 -- 172.21.15.114:0/2785054362 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfe400b940 con 0x55e8974ed790 2024-09-13T18:23:07.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.078+0000 7fdff2ffd640 1 --2- 172.21.15.114:0/2785054362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8974ee0b0 0x55e8974f5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:23:07.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.078+0000 7fdff27fc640 1 --2- 172.21.15.114:0/2785054362 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e8974ed790 0x55e8974f7ef0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fdfe802aa10 tx=0x7fdfe8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fab493b5dfcca7e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:07.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.078+0000 7fdfd6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:23:07.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.079+0000 7fdfd6ff5640 1 -- 172.21.15.114:0/2785054362 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe80035f0 con 0x55e8974ed790 2024-09-13T18:23:07.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.079+0000 7fdfd6ff5640 1 -- 172.21.15.114:0/2785054362 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fdfe8003750 con 0x55e8974ed790 2024-09-13T18:23:07.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.079+0000 7fdffc5e5740 1 -- 172.21.15.114:0/2785054362 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e89747ab80 con 0x55e8974ed790 2024-09-13T18:23:07.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.079+0000 7fdfd6ff5640 1 -- 172.21.15.114:0/2785054362 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdfe8003910 con 0x55e8974ed790 2024-09-13T18:23:07.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.079+0000 7fdffc5e5740 10 client.15948 fetch_fsmap learned FSMap version 30 2024-09-13T18:23:07.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.079+0000 7fdffc5e5740 1 -- 172.21.15.114:0/2785054362 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55e89744f3c0 con 0x55e8974ed790 2024-09-13T18:23:07.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.079+0000 7fdfd6ff5640 1 -- 172.21.15.114:0/2785054362 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e89747ab80 con 0x55e8974ed790 2024-09-13T18:23:07.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.079+0000 7fdfd6ff5640 1 -- 172.21.15.114:0/2785054362 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdfe803d950 con 0x55e8974ed790 2024-09-13T18:23:07.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.079+0000 7fdffc5e5740 10 client.15948 fetch_fsmap finished waiting for FSMap version 30 2024-09-13T18:23:07.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.079+0000 7fdffc5e5740 10 client.15948 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-13T18:23:07.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.079+0000 7fdffc5e5740 20 client.15948 populate_metadata read hostname 'smithi114' 2024-09-13T18:23:07.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.079+0000 7fdffc5e5740 1 --2- 172.21.15.114:0/2785054362 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55e897508210 0x55e89750a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:07.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.079+0000 7fdffc5e5740 4 client.15948 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:23:07.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.079+0000 7fdffc5e5740 1 -- 172.21.15.114:0/2785054362 --> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e8974431f0 con 0x55e897508210 2024-09-13T18:23:07.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.080+0000 7fdff2ffd640 1 --2- 172.21.15.114:0/2785054362 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55e897508210 0x55e89750a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:07.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.080+0000 7fdfd6ff5640 10 client.15948 ms_handle_connect on v2:172.21.15.134:6836/891692210 2024-09-13T18:23:07.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.080+0000 7fdff2ffd640 1 --2- 172.21.15.114:0/2785054362 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55e897508210 0x55e89750a630 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d7e3dd46821c77d1 server_cookie=7097e279c4cda101 in_seq=0 out_seq=0 2024-09-13T18:23:07.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.081+0000 7fdfd6ff5640 1 -- 172.21.15.114:0/2785054362 <== mds.1 v2:172.21.15.134:6836/891692210 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55e8974431f0 con 0x55e897508210 2024-09-13T18:23:07.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.081+0000 7fdfd6ff5640 10 client.15948 handle_command_reply: tid=0 2024-09-13T18:23:07.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.081+0000 7fdffc5e5740 1 client.15948 shutdown 2024-09-13T18:23:07.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.081+0000 7fdffc5e5740 1 -- 172.21.15.114:0/2785054362 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e8974ed790 msgr2=0x55e8974f7ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:07.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.081+0000 7fdffc5e5740 1 --2- 172.21.15.114:0/2785054362 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e8974ed790 0x55e8974f7ef0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fdfe802aa10 tx=0x7fdfe8009260 comp rx=0 tx=0).stop 2024-09-13T18:23:07.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.081+0000 7fdffc5e5740 1 -- 172.21.15.114:0/2785054362 shutdown_connections 2024-09-13T18:23:07.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.081+0000 7fdffc5e5740 1 --2- 172.21.15.114:0/2785054362 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55e897508210 0x55e89750a630 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.081+0000 7fdfd6ff5640 0 client.15948 ms_handle_reset on v2:172.21.15.134:6836/891692210 2024-09-13T18:23:07.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.081+0000 7fdffc5e5740 1 --2- 172.21.15.114:0/2785054362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8974f8950 0x55e8974f57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.081+0000 7fdffc5e5740 1 --2- 172.21.15.114:0/2785054362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8974ee0b0 0x55e8974f5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.081+0000 7fdffc5e5740 1 --2- 172.21.15.114:0/2785054362 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e8974ed790 0x55e8974f7ef0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.081+0000 7fdffc5e5740 1 -- 172.21.15.114:0/2785054362 >> 172.21.15.114:0/2785054362 conn(0x55e8974e9d00 msgr2=0x55e8974efa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:07.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.081+0000 7fdffc5e5740 1 -- 172.21.15.114:0/2785054362 shutdown_connections 2024-09-13T18:23:07.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.081+0000 7fdffc5e5740 1 -- 172.21.15.114:0/2785054362 wait complete. 2024-09-13T18:23:07.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.081+0000 7fdffc5e5740 20 client.15948 trim_cache size 0 max 16384 2024-09-13T18:23:07.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.092+0000 7fdffc5e5740 1 Processor -- start 2024-09-13T18:23:07.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.092+0000 7fdffc5e5740 1 -- start start 2024-09-13T18:23:07.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.092+0000 7fdffc5e5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8974ed790 0x55e8974f79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:07.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.092+0000 7fdffc5e5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8974ee0b0 0x55e8974ff610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:07.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.092+0000 7fdffc5e5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e8974f8950 0x55e8974ffb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:07.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.092+0000 7fdffc5e5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e89747abb0 con 0x55e8974f8950 2024-09-13T18:23:07.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.092+0000 7fdffc5e5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e897508070 con 0x55e8974ed790 2024-09-13T18:23:07.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.092+0000 7fdffc5e5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8974f5b10 con 0x55e8974ee0b0 2024-09-13T18:23:07.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.092+0000 7fdff27fc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e8974f8950 0x55e8974ffb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:07.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.092+0000 7fdff27fc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e8974f8950 0x55e8974ffb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36790/0 (socket says 172.21.15.114:36790) 2024-09-13T18:23:07.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.092+0000 7fdff27fc640 1 -- 172.21.15.114:0/2090874873 learned_addr learned my addr 172.21.15.114:0/2090874873 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:07.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.093+0000 7fdff2ffd640 1 --2- 172.21.15.114:0/2090874873 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8974ee0b0 0x55e8974ff610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:07.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.093+0000 7fdff27fc640 1 --2- 172.21.15.114:0/2090874873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8974ed790 0x55e8974f79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:07.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.093+0000 7fdff27fc640 1 -- 172.21.15.114:0/2090874873 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8974ee0b0 msgr2=0x55e8974ff610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:07.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.093+0000 7fdff27fc640 1 --2- 172.21.15.114:0/2090874873 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8974ee0b0 0x55e8974ff610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.093+0000 7fdff27fc640 1 -- 172.21.15.114:0/2090874873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8974ed790 msgr2=0x55e8974f79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:07.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.093+0000 7fdff27fc640 1 --2- 172.21.15.114:0/2090874873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8974ed790 0x55e8974f79e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.093+0000 7fdff27fc640 1 -- 172.21.15.114:0/2090874873 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8973f1770 con 0x55e8974f8950 2024-09-13T18:23:07.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.093+0000 7fdff27fc640 1 --2- 172.21.15.114:0/2090874873 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e8974f8950 0x55e8974ffb50 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fdfe80057c0 tx=0x7fdfe8041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2647996db4360cb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:07.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.093+0000 7fdfd9ffb640 1 -- 172.21.15.114:0/2090874873 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe8031780 con 0x55e8974f8950 2024-09-13T18:23:07.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.093+0000 7fdfd9ffb640 1 -- 172.21.15.114:0/2090874873 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdfe8031da0 con 0x55e8974f8950 2024-09-13T18:23:07.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.093+0000 7fdfd9ffb640 1 -- 172.21.15.114:0/2090874873 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe8032030 con 0x55e8974f8950 2024-09-13T18:23:07.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.094+0000 7fdffc5e5740 1 -- 172.21.15.114:0/2090874873 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e8974f8950 msgr2=0x55e8974ffb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:07.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.094+0000 7fdffc5e5740 1 --2- 172.21.15.114:0/2090874873 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e8974f8950 0x55e8974ffb50 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fdfe80057c0 tx=0x7fdfe8041040 comp rx=0 tx=0).stop 2024-09-13T18:23:07.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.094+0000 7fdffc5e5740 1 -- 172.21.15.114:0/2090874873 shutdown_connections 2024-09-13T18:23:07.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.094+0000 7fdffc5e5740 1 --2- 172.21.15.114:0/2090874873 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e8974f8950 0x55e8974ffb50 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.094+0000 7fdffc5e5740 1 --2- 172.21.15.114:0/2090874873 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8974ee0b0 0x55e8974ff610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.094+0000 7fdffc5e5740 1 --2- 172.21.15.114:0/2090874873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8974ed790 0x55e8974f79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.094+0000 7fdffc5e5740 1 -- 172.21.15.114:0/2090874873 >> 172.21.15.114:0/2090874873 conn(0x55e8974e9d00 msgr2=0x55e8974f3b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:07.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.094+0000 7fdffc5e5740 1 -- 172.21.15.114:0/2090874873 shutdown_connections 2024-09-13T18:23:07.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.094+0000 7fdffc5e5740 1 -- 172.21.15.114:0/2090874873 wait complete. 2024-09-13T18:23:07.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.094+0000 7fdffc5e5740 1 -- start start 2024-09-13T18:23:07.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.094+0000 7fdffc5e5740 1 Processor -- start 2024-09-13T18:23:07.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.095+0000 7fdffc5e5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8974ed790 0x55e8974ff620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:07.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.095+0000 7fdffc5e5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e8974ee0b0 0x55e8974f16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:07.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.095+0000 7fdffc5e5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8974f8950 0x55e8974f1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:07.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.095+0000 7fdffc5e5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8974ffc30 con 0x55e8974ee0b0 2024-09-13T18:23:07.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.095+0000 7fdff2ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e8974ee0b0 0x55e8974f16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:07.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.095+0000 7fdff2ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e8974ee0b0 0x55e8974f16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36806/0 (socket says 172.21.15.114:36806) 2024-09-13T18:23:07.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.095+0000 7fdff2ffd640 1 -- 172.21.15.114:0/1703871353 learned_addr learned my addr 172.21.15.114:0/1703871353 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:07.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.095+0000 7fdffc5e5740 1 -- 172.21.15.114:0/1703871353 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8974ffda0 con 0x55e8974f8950 2024-09-13T18:23:07.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.095+0000 7fdffc5e5740 1 -- 172.21.15.114:0/1703871353 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8974f2170 con 0x55e8974ed790 2024-09-13T18:23:07.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.095+0000 7fdff27fc640 1 --2- 172.21.15.114:0/1703871353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8974ed790 0x55e8974ff620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:07.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.095+0000 7fdffc5e5740 10 client.0 _ll_register_callbacks cb 0x7fdfe400b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:23:07.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.095+0000 7fdff27fc640 1 --2- 172.21.15.114:0/1703871353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8974f8950 0x55e8974f1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:07.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.095+0000 7fdff2ffd640 1 -- 172.21.15.114:0/1703871353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8974ed790 msgr2=0x55e8974ff620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:07.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.095+0000 7fdff2ffd640 1 --2- 172.21.15.114:0/1703871353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8974ed790 0x55e8974ff620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.095+0000 7fdff2ffd640 1 -- 172.21.15.114:0/1703871353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8974f8950 msgr2=0x55e8974f1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:07.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.095+0000 7fdff2ffd640 1 --2- 172.21.15.114:0/1703871353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8974f8950 0x55e8974f1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.095+0000 7fdff2ffd640 1 -- 172.21.15.114:0/1703871353 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e89744f3c0 con 0x55e8974ee0b0 2024-09-13T18:23:07.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.095+0000 7fdff27fc640 1 --2- 172.21.15.114:0/1703871353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8974f8950 0x55e8974f1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:23:07.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.095+0000 7fdff2ffd640 1 --2- 172.21.15.114:0/1703871353 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e8974ee0b0 0x55e8974f16f0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4000c00 tx=0x7fdfe400bc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=958196100741f586 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:07.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.095+0000 7fdfd6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:23:07.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.095+0000 7fdfd6ff5640 1 -- 172.21.15.114:0/1703871353 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe402c070 con 0x55e8974ee0b0 2024-09-13T18:23:07.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.095+0000 7fdffc5e5740 1 -- 172.21.15.114:0/1703871353 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdfe4015070 con 0x55e8974ee0b0 2024-09-13T18:23:07.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.095+0000 7fdfd6ff5640 1 -- 172.21.15.114:0/1703871353 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fdfe4017070 con 0x55e8974ee0b0 2024-09-13T18:23:07.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.096+0000 7fdfd6ff5640 1 -- 172.21.15.114:0/1703871353 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdfe4003800 con 0x55e8974ee0b0 2024-09-13T18:23:07.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.096+0000 7fdffc5e5740 10 client.15954 fetch_fsmap learned FSMap version 30 2024-09-13T18:23:07.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.096+0000 7fdffc5e5740 1 -- 172.21.15.114:0/1703871353 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55e897416e80 con 0x55e8974ee0b0 2024-09-13T18:23:07.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.096+0000 7fdfd6ff5640 1 -- 172.21.15.114:0/1703871353 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdfe4015070 con 0x55e8974ee0b0 2024-09-13T18:23:07.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.096+0000 7fdfd6ff5640 1 -- 172.21.15.114:0/1703871353 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdfe40050d0 con 0x55e8974ee0b0 2024-09-13T18:23:07.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.096+0000 7fdffc5e5740 10 client.15954 fetch_fsmap finished waiting for FSMap version 30 2024-09-13T18:23:07.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.096+0000 7fdffc5e5740 10 client.15954 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-13T18:23:07.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.096+0000 7fdffc5e5740 20 client.15954 populate_metadata read hostname 'smithi114' 2024-09-13T18:23:07.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.096+0000 7fdffc5e5740 1 --2- 172.21.15.114:0/1703871353 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55e897508210 0x55e897561900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:07.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.096+0000 7fdffc5e5740 4 client.15954 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:23:07.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.096+0000 7fdffc5e5740 1 -- 172.21.15.114:0/1703871353 --> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e8971444e0 con 0x55e897508210 2024-09-13T18:23:07.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.096+0000 7fdff27fc640 1 --2- 172.21.15.114:0/1703871353 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55e897508210 0x55e897561900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:07.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.097+0000 7fdff27fc640 1 --2- 172.21.15.114:0/1703871353 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55e897508210 0x55e897561900 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c8b16dc5ce7929f0 server_cookie=68fd9c942070b2b8 in_seq=0 out_seq=0 2024-09-13T18:23:07.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.097+0000 7fdfd6ff5640 10 client.15954 ms_handle_connect on v2:172.21.15.134:6836/891692210 2024-09-13T18:23:07.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.097+0000 7fdfd6ff5640 1 -- 172.21.15.114:0/1703871353 <== mds.1 v2:172.21.15.134:6836/891692210 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e8971444e0 con 0x55e897508210 2024-09-13T18:23:07.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.097+0000 7fdfd6ff5640 10 client.15954 handle_command_reply: tid=0 2024-09-13T18:23:07.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.097+0000 7fdffc5e5740 1 client.15954 shutdown 2024-09-13T18:23:07.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.097+0000 7fdffc5e5740 1 -- 172.21.15.114:0/1703871353 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e8974ee0b0 msgr2=0x55e8974f16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:07.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.097+0000 7fdffc5e5740 1 --2- 172.21.15.114:0/1703871353 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e8974ee0b0 0x55e8974f16f0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4000c00 tx=0x7fdfe400bc70 comp rx=0 tx=0).stop 2024-09-13T18:23:07.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.097+0000 7fdffc5e5740 1 -- 172.21.15.114:0/1703871353 shutdown_connections 2024-09-13T18:23:07.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.097+0000 7fdffc5e5740 1 --2- 172.21.15.114:0/1703871353 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55e897508210 0x55e897561900 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.097+0000 7fdfd6ff5640 0 client.15954 ms_handle_reset on v2:172.21.15.134:6836/891692210 2024-09-13T18:23:07.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.097+0000 7fdffc5e5740 1 --2- 172.21.15.114:0/1703871353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8974f8950 0x55e8974f1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.097+0000 7fdffc5e5740 1 --2- 172.21.15.114:0/1703871353 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e8974ee0b0 0x55e8974f16f0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.097+0000 7fdffc5e5740 1 --2- 172.21.15.114:0/1703871353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8974ed790 0x55e8974ff620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.097+0000 7fdffc5e5740 1 -- 172.21.15.114:0/1703871353 >> 172.21.15.114:0/1703871353 conn(0x55e8974e9d00 msgr2=0x55e8974f4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:07.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.097+0000 7fdffc5e5740 1 -- 172.21.15.114:0/1703871353 shutdown_connections 2024-09-13T18:23:07.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.097+0000 7fdffc5e5740 1 -- 172.21.15.114:0/1703871353 wait complete. 2024-09-13T18:23:07.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.097+0000 7fdffc5e5740 20 client.15954 trim_cache size 0 max 16384 2024-09-13T18:23:07.107 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:23:07.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.098+0000 7fdff97e9640 1 -- 172.21.15.114:0/2381371833 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fdfdc0a1470 msgr2=0x7fdfdc0a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:07.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.098+0000 7fdff97e9640 1 --2- 172.21.15.114:0/2381371833 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fdfdc0a1470 0x7fdfdc0a3930 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdfec0ca120 tx=0x7fdfec004a40 comp rx=0 tx=0).stop 2024-09-13T18:23:07.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.098+0000 7fdff97e9640 1 -- 172.21.15.114:0/2381371833 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdff4197ab0 msgr2=0x7fdff41a20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:07.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.098+0000 7fdff97e9640 1 --2- 172.21.15.114:0/2381371833 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdff4197ab0 0x7fdff41a20b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fdfec00d9c0 tx=0x7fdfec00de80 comp rx=0 tx=0).stop 2024-09-13T18:23:07.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.098+0000 7fdff97e9640 1 -- 172.21.15.114:0/2381371833 shutdown_connections 2024-09-13T18:23:07.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.098+0000 7fdff97e9640 1 --2- 172.21.15.114:0/2381371833 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fdfdc0a1470 0x7fdfdc0a3930 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.098+0000 7fdff97e9640 1 --2- 172.21.15.114:0/2381371833 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdff4197ab0 0x7fdff41a20b0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.098+0000 7fdff97e9640 1 --2- 172.21.15.114:0/2381371833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdff4103fb0 0x7fdff4197570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.098+0000 7fdff97e9640 1 --2- 172.21.15.114:0/2381371833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdff40fee60 0x7fdff4197030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.098+0000 7fdff97e9640 1 -- 172.21.15.114:0/2381371833 >> 172.21.15.114:0/2381371833 conn(0x7fdff4075c60 msgr2=0x7fdff4102390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:07.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.098+0000 7fdff97e9640 1 -- 172.21.15.114:0/2381371833 shutdown_connections 2024-09-13T18:23:07.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.098+0000 7fdff97e9640 1 -- 172.21.15.114:0/2381371833 wait complete. 2024-09-13T18:23:07.111 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-09-13T18:23:07.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.212+0000 7f878adf4640 1 -- 172.21.15.114:0/3799661714 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8784103fb0 msgr2=0x7f878410bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:07.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.212+0000 7f878adf4640 1 --2- 172.21.15.114:0/3799661714 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8784103fb0 0x7f878410bb00 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f877c009cd0 tx=0x7f877c02a8e0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.212+0000 7f878adf4640 1 -- 172.21.15.114:0/3799661714 shutdown_connections 2024-09-13T18:23:07.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.212+0000 7f878adf4640 1 --2- 172.21.15.114:0/3799661714 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8784103fb0 0x7f878410bb00 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.212+0000 7f878adf4640 1 --2- 172.21.15.114:0/3799661714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87840ff810 0x7f8784103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.212+0000 7f878adf4640 1 --2- 172.21.15.114:0/3799661714 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87840fee60 0x7f87840ff240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.212+0000 7f878adf4640 1 -- 172.21.15.114:0/3799661714 >> 172.21.15.114:0/3799661714 conn(0x7f8784075c60 msgr2=0x7f87840fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:07.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.212+0000 7f878adf4640 1 -- 172.21.15.114:0/3799661714 shutdown_connections 2024-09-13T18:23:07.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.212+0000 7f878adf4640 1 -- 172.21.15.114:0/3799661714 wait complete. 2024-09-13T18:23:07.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.213+0000 7f878adf4640 1 Processor -- start 2024-09-13T18:23:07.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.213+0000 7f878adf4640 1 -- start start 2024-09-13T18:23:07.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.213+0000 7f878adf4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87840fee60 0x7f8784195050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:07.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.213+0000 7f878adf4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8784103fb0 0x7f8784195590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:07.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.214+0000 7f878adf4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8784199980 0x7f8784199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:07.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.214+0000 7f878adf4640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f878419a370 con 0x7f8784103fb0 2024-09-13T18:23:07.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.214+0000 7f878adf4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f878419a4e0 con 0x7f8784199980 2024-09-13T18:23:07.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.214+0000 7f878adf4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f878419a650 con 0x7f87840fee60 2024-09-13T18:23:07.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.214+0000 7f8783fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8784103fb0 0x7f8784195590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:07.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.214+0000 7f8783fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8784103fb0 0x7f8784195590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36816/0 (socket says 172.21.15.114:36816) 2024-09-13T18:23:07.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.214+0000 7f8783fff640 1 -- 172.21.15.114:0/799478592 learned_addr learned my addr 172.21.15.114:0/799478592 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:07.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.214+0000 7f8788b90640 1 --2- 172.21.15.114:0/799478592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87840fee60 0x7f8784195050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:07.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.214+0000 7f8789391640 1 --2- 172.21.15.114:0/799478592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8784199980 0x7f8784199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:07.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.214+0000 7f8783fff640 1 -- 172.21.15.114:0/799478592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87840fee60 msgr2=0x7f8784195050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:07.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.214+0000 7f8783fff640 1 --2- 172.21.15.114:0/799478592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87840fee60 0x7f8784195050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.214+0000 7f8783fff640 1 -- 172.21.15.114:0/799478592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8784199980 msgr2=0x7f8784199e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:07.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.214+0000 7f8783fff640 1 --2- 172.21.15.114:0/799478592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8784199980 0x7f8784199e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.214+0000 7f8783fff640 1 -- 172.21.15.114:0/799478592 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f877c009900 con 0x7f8784103fb0 2024-09-13T18:23:07.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.214+0000 7f8788b90640 1 --2- 172.21.15.114:0/799478592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87840fee60 0x7f8784195050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:23:07.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.214+0000 7f8783fff640 1 --2- 172.21.15.114:0/799478592 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8784103fb0 0x7f8784195590 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f877000e9f0 tx=0x7f877000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:07.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.214+0000 7f8781ffb640 1 -- 172.21.15.114:0/799478592 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87700096c0 con 0x7f8784103fb0 2024-09-13T18:23:07.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.215+0000 7f878adf4640 1 -- 172.21.15.114:0/799478592 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87840675a0 con 0x7f8784103fb0 2024-09-13T18:23:07.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.215+0000 7f8781ffb640 1 -- 172.21.15.114:0/799478592 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f877000ccc0 con 0x7f8784103fb0 2024-09-13T18:23:07.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.215+0000 7f8781ffb640 1 -- 172.21.15.114:0/799478592 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8770004e60 con 0x7f8784103fb0 2024-09-13T18:23:07.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.215+0000 7f878adf4640 1 -- 172.21.15.114:0/799478592 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8784067a50 con 0x7f8784103fb0 2024-09-13T18:23:07.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.218+0000 7f878dbf2740 1 Processor -- start 2024-09-13T18:23:07.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.218+0000 7f878dbf2740 1 -- start start 2024-09-13T18:23:07.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.218+0000 7f878dbf2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557ad38c9790 0x557ad38c9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:07.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.218+0000 7f8781ffb640 1 -- 172.21.15.114:0/799478592 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) v1 ==== 136107+0+0 (secure 0 0 0) 0x7f8770004fe0 con 0x7f8784103fb0 2024-09-13T18:23:07.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.218+0000 7f878dbf2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ad38ca0b0 0x557ad38d4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:07.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.218+0000 7f878dbf2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ad38d4950 0x557ad38d6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:07.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.218+0000 7f878dbf2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ad3833cc0 con 0x557ad38c9790 2024-09-13T18:23:07.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.218+0000 7f878dbf2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ad3831450 con 0x557ad38ca0b0 2024-09-13T18:23:07.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.218+0000 7f878dbf2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ad382cd40 con 0x557ad38d4950 2024-09-13T18:23:07.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.218+0000 7f8789391640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557ad38c9790 0x557ad38c9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:07.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.218+0000 7f8789391640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557ad38c9790 0x557ad38c9b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36820/0 (socket says 172.21.15.114:36820) 2024-09-13T18:23:07.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.218+0000 7f8789391640 1 -- 172.21.15.114:0/3127232003 learned_addr learned my addr 172.21.15.114:0/3127232003 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:07.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.218+0000 7f8783fff640 1 --2- 172.21.15.114:0/3127232003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ad38d4950 0x557ad38d6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:07.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.218+0000 7f8788b90640 1 --2- 172.21.15.114:0/3127232003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ad38ca0b0 0x557ad38d4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:07.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.219+0000 7f8789391640 1 -- 172.21.15.114:0/3127232003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ad38d4950 msgr2=0x557ad38d6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:07.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.219+0000 7f8789391640 1 --2- 172.21.15.114:0/3127232003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ad38d4950 0x557ad38d6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.219+0000 7f8789391640 1 -- 172.21.15.114:0/3127232003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ad38ca0b0 msgr2=0x557ad38d4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:07.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.219+0000 7f8789391640 1 --2- 172.21.15.114:0/3127232003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ad38ca0b0 0x557ad38d4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.219+0000 7f8789391640 1 -- 172.21.15.114:0/3127232003 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ad37f4620 con 0x557ad38c9790 2024-09-13T18:23:07.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.219+0000 7f8788b90640 1 --2- 172.21.15.114:0/3127232003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ad38ca0b0 0x557ad38d4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:23:07.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.219+0000 7f8789391640 1 --2- 172.21.15.114:0/3127232003 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557ad38c9790 0x557ad38c9b70 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f877c005e30 tx=0x7f877c005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3866bfc4e1d5fb19 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:07.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.219+0000 7f8781ffb640 1 --2- 172.21.15.114:0/799478592 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f87740a13a0 0x7f87740a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:07.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.219+0000 7f8781ffb640 1 -- 172.21.15.114:0/799478592 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f8770014070 con 0x7f8784103fb0 2024-09-13T18:23:07.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.219+0000 7f8788b90640 1 --2- 172.21.15.114:0/799478592 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f87740a13a0 0x7f87740a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:07.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.219+0000 7f876effd640 1 -- 172.21.15.114:0/3127232003 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877c0307a0 con 0x557ad38c9790 2024-09-13T18:23:07.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.219+0000 7f876effd640 1 -- 172.21.15.114:0/3127232003 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f877c03f070 con 0x557ad38c9790 2024-09-13T18:23:07.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.219+0000 7f876effd640 1 -- 172.21.15.114:0/3127232003 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877c003bd0 con 0x557ad38c9790 2024-09-13T18:23:07.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.220+0000 7f878dbf2740 1 -- 172.21.15.114:0/3127232003 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557ad38c9790 msgr2=0x557ad38c9b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:07.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.220+0000 7f8788b90640 1 --2- 172.21.15.114:0/799478592 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f87740a13a0 0x7f87740a3860 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8778009fd0 tx=0x7f8778005f90 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:07.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.220+0000 7f878dbf2740 1 --2- 172.21.15.114:0/3127232003 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557ad38c9790 0x557ad38c9b70 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f877c005e30 tx=0x7f877c005f70 comp rx=0 tx=0).stop 2024-09-13T18:23:07.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.220+0000 7f878dbf2740 1 -- 172.21.15.114:0/3127232003 shutdown_connections 2024-09-13T18:23:07.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.220+0000 7f878dbf2740 1 --2- 172.21.15.114:0/3127232003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ad38d4950 0x557ad38d6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.220+0000 7f878dbf2740 1 --2- 172.21.15.114:0/3127232003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ad38ca0b0 0x557ad38d4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.220+0000 7f878dbf2740 1 --2- 172.21.15.114:0/3127232003 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557ad38c9790 0x557ad38c9b70 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.220+0000 7f878dbf2740 1 -- 172.21.15.114:0/3127232003 >> 172.21.15.114:0/3127232003 conn(0x557ad38c5d00 msgr2=0x557ad38c60e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:07.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.220+0000 7f878dbf2740 1 -- 172.21.15.114:0/3127232003 shutdown_connections 2024-09-13T18:23:07.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.220+0000 7f878dbf2740 1 -- 172.21.15.114:0/3127232003 wait complete. 2024-09-13T18:23:07.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.221+0000 7f878dbf2740 1 -- start start 2024-09-13T18:23:07.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.221+0000 7f878dbf2740 1 Processor -- start 2024-09-13T18:23:07.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.221+0000 7f878dbf2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ad38c9790 0x557ad38d3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:07.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.221+0000 7f878dbf2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557ad38ca0b0 0x557ad38d12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:07.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.221+0000 7f878dbf2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ad38d4950 0x557ad38d1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:07.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.221+0000 7f878dbf2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ad37eb180 con 0x557ad38ca0b0 2024-09-13T18:23:07.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.221+0000 7f8783fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557ad38ca0b0 0x557ad38d12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:07.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.221+0000 7f8783fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557ad38ca0b0 0x557ad38d12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36822/0 (socket says 172.21.15.114:36822) 2024-09-13T18:23:07.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.221+0000 7f8783fff640 1 -- 172.21.15.114:0/1442684215 learned_addr learned my addr 172.21.15.114:0/1442684215 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:07.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.221+0000 7f878dbf2740 1 -- 172.21.15.114:0/1442684215 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ad3505430 con 0x557ad38c9790 2024-09-13T18:23:07.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.221+0000 7f878dbf2740 1 -- 172.21.15.114:0/1442684215 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ad37b05e0 con 0x557ad38d4950 2024-09-13T18:23:07.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.221+0000 7f878dbf2740 10 client.0 _ll_register_callbacks cb 0x557ad381caa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:23:07.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.221+0000 7f8788b90640 1 --2- 172.21.15.114:0/1442684215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ad38c9790 0x557ad38d3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:07.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.221+0000 7f8789391640 1 --2- 172.21.15.114:0/1442684215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ad38d4950 0x557ad38d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:07.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.222+0000 7f8783fff640 1 -- 172.21.15.114:0/1442684215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ad38d4950 msgr2=0x557ad38d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:07.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.222+0000 7f8783fff640 1 --2- 172.21.15.114:0/1442684215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ad38d4950 0x557ad38d1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.222+0000 7f8783fff640 1 -- 172.21.15.114:0/1442684215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ad38c9790 msgr2=0x557ad38d3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:07.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.222+0000 7f8783fff640 1 --2- 172.21.15.114:0/1442684215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ad38c9790 0x557ad38d3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.222+0000 7f8783fff640 1 -- 172.21.15.114:0/1442684215 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f878410c340 con 0x557ad38ca0b0 2024-09-13T18:23:07.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.222+0000 7f8783fff640 1 --2- 172.21.15.114:0/1442684215 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557ad38ca0b0 0x557ad38d12f0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f877000e9c0 tx=0x7f8770004780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8bc8ed4039b23ad server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:07.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.222+0000 7f8763fff640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:23:07.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.222+0000 7f8763fff640 1 -- 172.21.15.114:0/1442684215 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8770099f80 con 0x557ad38ca0b0 2024-09-13T18:23:07.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.222+0000 7f8763fff640 1 -- 172.21.15.114:0/1442684215 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f87700cc030 con 0x557ad38ca0b0 2024-09-13T18:23:07.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.222+0000 7f878dbf2740 1 -- 172.21.15.114:0/1442684215 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557ad3856b80 con 0x557ad38ca0b0 2024-09-13T18:23:07.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.222+0000 7f8763fff640 1 -- 172.21.15.114:0/1442684215 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8770099cf0 con 0x557ad38ca0b0 2024-09-13T18:23:07.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.222+0000 7f878dbf2740 10 client.15966 fetch_fsmap learned FSMap version 30 2024-09-13T18:23:07.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.222+0000 7f878dbf2740 1 -- 172.21.15.114:0/1442684215 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x557ad382b3c0 con 0x557ad38ca0b0 2024-09-13T18:23:07.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.222+0000 7f8763fff640 1 -- 172.21.15.114:0/1442684215 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557ad3856b80 con 0x557ad38ca0b0 2024-09-13T18:23:07.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.222+0000 7f8763fff640 1 -- 172.21.15.114:0/1442684215 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f877001f060 con 0x557ad38ca0b0 2024-09-13T18:23:07.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.223+0000 7f878dbf2740 10 client.15966 fetch_fsmap finished waiting for FSMap version 30 2024-09-13T18:23:07.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.223+0000 7f878dbf2740 10 client.15966 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:23:07.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.223+0000 7f878dbf2740 20 client.15966 populate_metadata read hostname 'smithi114' 2024-09-13T18:23:07.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.223+0000 7f878dbf2740 1 --2- 172.21.15.114:0/1442684215 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x557ad38e4270 0x557ad38e6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:07.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.223+0000 7f878dbf2740 4 client.15966 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:23:07.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.223+0000 7f878dbf2740 1 -- 172.21.15.114:0/1442684215 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557ad381f1f0 con 0x557ad38e4270 2024-09-13T18:23:07.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.223+0000 7f8788b90640 1 --2- 172.21.15.114:0/1442684215 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x557ad38e4270 0x557ad38e6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:07.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.223+0000 7f8763fff640 10 client.15966 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:23:07.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.223+0000 7f8788b90640 1 --2- 172.21.15.114:0/1442684215 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x557ad38e4270 0x557ad38e6660 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd51840df0272c06 server_cookie=5bfc1638963efc6e in_seq=0 out_seq=0 2024-09-13T18:23:07.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.224+0000 7f8763fff640 1 -- 172.21.15.114:0/1442684215 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x557ad381f1f0 con 0x557ad38e4270 2024-09-13T18:23:07.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.224+0000 7f8763fff640 10 client.15966 handle_command_reply: tid=0 2024-09-13T18:23:07.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.224+0000 7f878dbf2740 1 client.15966 shutdown 2024-09-13T18:23:07.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.224+0000 7f878dbf2740 1 -- 172.21.15.114:0/1442684215 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557ad38ca0b0 msgr2=0x557ad38d12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:07.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.224+0000 7f878dbf2740 1 --2- 172.21.15.114:0/1442684215 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557ad38ca0b0 0x557ad38d12f0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f877000e9c0 tx=0x7f8770004780 comp rx=0 tx=0).stop 2024-09-13T18:23:07.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.224+0000 7f878dbf2740 1 -- 172.21.15.114:0/1442684215 shutdown_connections 2024-09-13T18:23:07.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.224+0000 7f878dbf2740 1 --2- 172.21.15.114:0/1442684215 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x557ad38e4270 0x557ad38e6660 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.224+0000 7f878dbf2740 1 --2- 172.21.15.114:0/1442684215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ad38d4950 0x557ad38d1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.224+0000 7f878dbf2740 1 --2- 172.21.15.114:0/1442684215 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557ad38ca0b0 0x557ad38d12f0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.224+0000 7f8763fff640 0 client.15966 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:23:07.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.224+0000 7f878dbf2740 1 --2- 172.21.15.114:0/1442684215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ad38c9790 0x557ad38d3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.224+0000 7f878dbf2740 1 -- 172.21.15.114:0/1442684215 >> 172.21.15.114:0/1442684215 conn(0x557ad38c5d00 msgr2=0x557ad38cba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:07.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.224+0000 7f878dbf2740 1 -- 172.21.15.114:0/1442684215 shutdown_connections 2024-09-13T18:23:07.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.224+0000 7f878dbf2740 1 -- 172.21.15.114:0/1442684215 wait complete. 2024-09-13T18:23:07.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.224+0000 7f878dbf2740 20 client.15966 trim_cache size 0 max 16384 2024-09-13T18:23:07.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.235+0000 7f878dbf2740 1 Processor -- start 2024-09-13T18:23:07.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.235+0000 7f878dbf2740 1 -- start start 2024-09-13T18:23:07.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.235+0000 7f878dbf2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557ad38c9790 0x557ad38c56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:07.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.235+0000 7f878dbf2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ad38ca0b0 0x557ad38db610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:07.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.235+0000 7f878dbf2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ad38d4950 0x557ad38dbb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:07.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.235+0000 7f878dbf2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ad3856b80 con 0x557ad38c9790 2024-09-13T18:23:07.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.235+0000 7f878dbf2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ad38c69f0 con 0x557ad38ca0b0 2024-09-13T18:23:07.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.235+0000 7f878dbf2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ad38c03b0 con 0x557ad38d4950 2024-09-13T18:23:07.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.235+0000 7f8788b90640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557ad38c9790 0x557ad38c56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:07.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.236+0000 7f8788b90640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557ad38c9790 0x557ad38c56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36828/0 (socket says 172.21.15.114:36828) 2024-09-13T18:23:07.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.236+0000 7f8788b90640 1 -- 172.21.15.114:0/3338794974 learned_addr learned my addr 172.21.15.114:0/3338794974 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:07.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.236+0000 7f8783fff640 1 --2- 172.21.15.114:0/3338794974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ad38ca0b0 0x557ad38db610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:07.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.236+0000 7f8788b90640 1 -- 172.21.15.114:0/3338794974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ad38d4950 msgr2=0x557ad38dbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:07.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.236+0000 7f8789391640 1 --2- 172.21.15.114:0/3338794974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ad38d4950 0x557ad38dbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:07.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.236+0000 7f8788b90640 1 --2- 172.21.15.114:0/3338794974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ad38d4950 0x557ad38dbb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.236+0000 7f8788b90640 1 -- 172.21.15.114:0/3338794974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ad38ca0b0 msgr2=0x557ad38db610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:07.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.236+0000 7f8788b90640 1 --2- 172.21.15.114:0/3338794974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ad38ca0b0 0x557ad38db610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.236+0000 7f8788b90640 1 -- 172.21.15.114:0/3338794974 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ad37cd770 con 0x557ad38c9790 2024-09-13T18:23:07.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.236+0000 7f8788b90640 1 --2- 172.21.15.114:0/3338794974 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557ad38c9790 0x557ad38c56a0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x557ad38c6b60 tx=0x7f8778017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12ef47336a9a4476 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:07.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.236+0000 7f876f7fe640 1 -- 172.21.15.114:0/3338794974 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8778025040 con 0x557ad38c9790 2024-09-13T18:23:07.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.236+0000 7f876f7fe640 1 -- 172.21.15.114:0/3338794974 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f87780044c0 con 0x557ad38c9790 2024-09-13T18:23:07.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.236+0000 7f876f7fe640 1 -- 172.21.15.114:0/3338794974 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8778017070 con 0x557ad38c9790 2024-09-13T18:23:07.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.237+0000 7f878dbf2740 1 -- 172.21.15.114:0/3338794974 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557ad38c9790 msgr2=0x557ad38c56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:07.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.237+0000 7f878dbf2740 1 --2- 172.21.15.114:0/3338794974 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557ad38c9790 0x557ad38c56a0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x557ad38c6b60 tx=0x7f8778017000 comp rx=0 tx=0).stop 2024-09-13T18:23:07.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.237+0000 7f878dbf2740 1 -- 172.21.15.114:0/3338794974 shutdown_connections 2024-09-13T18:23:07.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.237+0000 7f878dbf2740 1 --2- 172.21.15.114:0/3338794974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ad38d4950 0x557ad38dbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.237+0000 7f878dbf2740 1 --2- 172.21.15.114:0/3338794974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ad38ca0b0 0x557ad38db610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.237+0000 7f878dbf2740 1 --2- 172.21.15.114:0/3338794974 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557ad38c9790 0x557ad38c56a0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.237+0000 7f878dbf2740 1 -- 172.21.15.114:0/3338794974 >> 172.21.15.114:0/3338794974 conn(0x557ad38c5d00 msgr2=0x557ad38cfba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:07.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.237+0000 7f878dbf2740 1 -- 172.21.15.114:0/3338794974 shutdown_connections 2024-09-13T18:23:07.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.237+0000 7f878dbf2740 1 -- 172.21.15.114:0/3338794974 wait complete. 2024-09-13T18:23:07.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.237+0000 7f878dbf2740 1 -- start start 2024-09-13T18:23:07.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.237+0000 7f878dbf2740 1 Processor -- start 2024-09-13T18:23:07.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.238+0000 7f878dbf2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ad38c9790 0x557ad38db940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:07.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.238+0000 7f878dbf2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ad38ca0b0 0x557ad38d7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:07.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.238+0000 7f878dbf2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557ad38d4950 0x557ad38d8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:07.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.238+0000 7f878dbf2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ad38d8900 con 0x557ad38d4950 2024-09-13T18:23:07.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.238+0000 7f878dbf2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ad38d8a70 con 0x557ad38c9790 2024-09-13T18:23:07.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.238+0000 7f878dbf2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ad393a740 con 0x557ad38ca0b0 2024-09-13T18:23:07.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.238+0000 7f878dbf2740 10 client.0 _ll_register_callbacks cb 0x7f8784069b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:23:07.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.238+0000 7f8789391640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557ad38d4950 0x557ad38d8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:07.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.238+0000 7f8783fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ad38ca0b0 0x557ad38d7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:07.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.238+0000 7f8783fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ad38ca0b0 0x557ad38d7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:33036/0 (socket says 172.21.15.114:33036) 2024-09-13T18:23:07.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.238+0000 7f8789391640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557ad38d4950 0x557ad38d8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36836/0 (socket says 172.21.15.114:36836) 2024-09-13T18:23:07.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.238+0000 7f8783fff640 1 -- 172.21.15.114:0/1633739084 learned_addr learned my addr 172.21.15.114:0/1633739084 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:07.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.238+0000 7f8788b90640 1 --2- 172.21.15.114:0/1633739084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ad38c9790 0x557ad38db940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:07.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.238+0000 7f8783fff640 1 -- 172.21.15.114:0/1633739084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ad38c9790 msgr2=0x557ad38db940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:07.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.238+0000 7f8783fff640 1 --2- 172.21.15.114:0/1633739084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ad38c9790 0x557ad38db940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.238+0000 7f8783fff640 1 -- 172.21.15.114:0/1633739084 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557ad38d4950 msgr2=0x557ad38d8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:07.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.238+0000 7f8783fff640 1 --2- 172.21.15.114:0/1633739084 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557ad38d4950 0x557ad38d8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.238+0000 7f8783fff640 1 -- 172.21.15.114:0/1633739084 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8778012e70 con 0x557ad38ca0b0 2024-09-13T18:23:07.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.238+0000 7f8783fff640 1 --2- 172.21.15.114:0/1633739084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ad38ca0b0 0x557ad38d7d10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8770096c20 tx=0x7f8770005780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2dea49da29849969 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:07.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.238+0000 7f8763fff640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:23:07.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.239+0000 7f8763fff640 1 -- 172.21.15.114:0/1633739084 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8770080070 con 0x557ad38ca0b0 2024-09-13T18:23:07.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.239+0000 7f878dbf2740 1 -- 172.21.15.114:0/1633739084 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f877c03f070 con 0x557ad38ca0b0 2024-09-13T18:23:07.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.239+0000 7f8763fff640 1 -- 172.21.15.114:0/1633739084 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f87700cc040 con 0x557ad38ca0b0 2024-09-13T18:23:07.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.239+0000 7f8763fff640 1 -- 172.21.15.114:0/1633739084 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87700b84a0 con 0x557ad38ca0b0 2024-09-13T18:23:07.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.239+0000 7f878dbf2740 10 client.25426 fetch_fsmap learned FSMap version 30 2024-09-13T18:23:07.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.239+0000 7f878dbf2740 1 -- 172.21.15.114:0/1633739084 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x557ad37f2e80 con 0x557ad38ca0b0 2024-09-13T18:23:07.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.239+0000 7f8763fff640 1 -- 172.21.15.114:0/1633739084 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f877c03f070 con 0x557ad38ca0b0 2024-09-13T18:23:07.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.239+0000 7f8763fff640 1 -- 172.21.15.114:0/1633739084 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f877001f040 con 0x557ad38ca0b0 2024-09-13T18:23:07.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.239+0000 7f878dbf2740 10 client.25426 fetch_fsmap finished waiting for FSMap version 30 2024-09-13T18:23:07.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.239+0000 7f878dbf2740 10 client.25426 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:23:07.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.239+0000 7f878dbf2740 20 client.25426 populate_metadata read hostname 'smithi114' 2024-09-13T18:23:07.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.239+0000 7f878dbf2740 1 --2- 172.21.15.114:0/1633739084 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x557ad38e4270 0x557ad393d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:07.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.239+0000 7f878dbf2740 4 client.25426 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:23:07.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.239+0000 7f878dbf2740 1 -- 172.21.15.114:0/1633739084 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557ad35204e0 con 0x557ad38e4270 2024-09-13T18:23:07.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.239+0000 7f8788b90640 1 --2- 172.21.15.114:0/1633739084 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x557ad38e4270 0x557ad393d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:07.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.240+0000 7f8763fff640 10 client.25426 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:23:07.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.240+0000 7f8788b90640 1 --2- 172.21.15.114:0/1633739084 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x557ad38e4270 0x557ad393d8e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acef43c26cd8dc4d server_cookie=214c8b4d4b6950b6 in_seq=0 out_seq=0 2024-09-13T18:23:07.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.240+0000 7f8763fff640 1 -- 172.21.15.114:0/1633739084 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557ad35204e0 con 0x557ad38e4270 2024-09-13T18:23:07.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.240+0000 7f8763fff640 10 client.25426 handle_command_reply: tid=0 2024-09-13T18:23:07.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.240+0000 7f878dbf2740 1 client.25426 shutdown 2024-09-13T18:23:07.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.240+0000 7f878dbf2740 1 -- 172.21.15.114:0/1633739084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ad38ca0b0 msgr2=0x557ad38d7d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:07.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.240+0000 7f878dbf2740 1 --2- 172.21.15.114:0/1633739084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ad38ca0b0 0x557ad38d7d10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8770096c20 tx=0x7f8770005780 comp rx=0 tx=0).stop 2024-09-13T18:23:07.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.240+0000 7f878dbf2740 1 -- 172.21.15.114:0/1633739084 shutdown_connections 2024-09-13T18:23:07.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.240+0000 7f878dbf2740 1 --2- 172.21.15.114:0/1633739084 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x557ad38e4270 0x557ad393d8e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.240+0000 7f878dbf2740 1 --2- 172.21.15.114:0/1633739084 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557ad38d4950 0x557ad38d8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.240+0000 7f878dbf2740 1 --2- 172.21.15.114:0/1633739084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ad38ca0b0 0x557ad38d7d10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.240+0000 7f878dbf2740 1 --2- 172.21.15.114:0/1633739084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ad38c9790 0x557ad38db940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.240+0000 7f8763fff640 0 client.25426 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:23:07.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.240+0000 7f878dbf2740 1 -- 172.21.15.114:0/1633739084 >> 172.21.15.114:0/1633739084 conn(0x557ad38c5d00 msgr2=0x557ad38d2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:07.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.241+0000 7f878dbf2740 1 -- 172.21.15.114:0/1633739084 shutdown_connections 2024-09-13T18:23:07.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.241+0000 7f878dbf2740 1 -- 172.21.15.114:0/1633739084 wait complete. 2024-09-13T18:23:07.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.241+0000 7f878dbf2740 20 client.25426 trim_cache size 0 max 16384 2024-09-13T18:23:07.249 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:23:07.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.241+0000 7f878adf4640 1 -- 172.21.15.114:0/799478592 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f87740a13a0 msgr2=0x7f87740a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:07.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.241+0000 7f878adf4640 1 --2- 172.21.15.114:0/799478592 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f87740a13a0 0x7f87740a3860 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8778009fd0 tx=0x7f8778005f90 comp rx=0 tx=0).stop 2024-09-13T18:23:07.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.241+0000 7f878adf4640 1 -- 172.21.15.114:0/799478592 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8784103fb0 msgr2=0x7f8784195590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:07.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.241+0000 7f878adf4640 1 --2- 172.21.15.114:0/799478592 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8784103fb0 0x7f8784195590 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f877000e9f0 tx=0x7f877000eeb0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.241+0000 7f878adf4640 1 -- 172.21.15.114:0/799478592 shutdown_connections 2024-09-13T18:23:07.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.241+0000 7f878adf4640 1 --2- 172.21.15.114:0/799478592 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f87740a13a0 0x7f87740a3860 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.241+0000 7f878adf4640 1 --2- 172.21.15.114:0/799478592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8784199980 0x7f8784199e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.241+0000 7f878adf4640 1 --2- 172.21.15.114:0/799478592 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8784103fb0 0x7f8784195590 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.241+0000 7f878adf4640 1 --2- 172.21.15.114:0/799478592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87840fee60 0x7f8784195050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:07.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.241+0000 7f878adf4640 1 -- 172.21.15.114:0/799478592 >> 172.21.15.114:0/799478592 conn(0x7f8784075c60 msgr2=0x7f87840fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:07.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.241+0000 7f878adf4640 1 -- 172.21.15.114:0/799478592 shutdown_connections 2024-09-13T18:23:07.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:07.241+0000 7f878adf4640 1 -- 172.21.15.114:0/799478592 wait complete. 2024-09-13T18:23:07.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:23:07 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/3504797702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:23:07.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:23:07 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/3504797702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:23:07.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:23:07 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/3504797702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:23:08.255 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-13T18:23:08.255 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-09-13T18:23:08.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.365+0000 7f818db1f640 1 -- 172.21.15.114:0/1267590646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f81880feac0 msgr2=0x7f81880feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:08.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.365+0000 7f818db1f640 1 --2- 172.21.15.114:0/1267590646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f81880feac0 0x7f81880feea0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f8178009900 tx=0x7f817802a410 comp rx=0 tx=0).stop 2024-09-13T18:23:08.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.365+0000 7f818db1f640 1 -- 172.21.15.114:0/1267590646 shutdown_connections 2024-09-13T18:23:08.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.365+0000 7f818db1f640 1 --2- 172.21.15.114:0/1267590646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f818810c670 0x7f818810ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.365+0000 7f818db1f640 1 --2- 172.21.15.114:0/1267590646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81880ff470 0x7f818810c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.365+0000 7f818db1f640 1 --2- 172.21.15.114:0/1267590646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f81880feac0 0x7f81880feea0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.365+0000 7f818db1f640 1 -- 172.21.15.114:0/1267590646 >> 172.21.15.114:0/1267590646 conn(0x7f8188075c60 msgr2=0x7f81880fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:08.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.365+0000 7f818db1f640 1 -- 172.21.15.114:0/1267590646 shutdown_connections 2024-09-13T18:23:08.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.365+0000 7f818db1f640 1 -- 172.21.15.114:0/1267590646 wait complete. 2024-09-13T18:23:08.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.366+0000 7f818db1f640 1 Processor -- start 2024-09-13T18:23:08.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.366+0000 7f818db1f640 1 -- start start 2024-09-13T18:23:08.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.366+0000 7f818db1f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81880feac0 0x7f8188198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:08.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.366+0000 7f818db1f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81880ff470 0x7f8188198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:08.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.366+0000 7f818db1f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f818810c670 0x7f818819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:08.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.366+0000 7f818db1f640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818819d320 con 0x7f818810c670 2024-09-13T18:23:08.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.366+0000 7f818db1f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818819d490 con 0x7f81880feac0 2024-09-13T18:23:08.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.366+0000 7f818db1f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818819d650 con 0x7f81880ff470 2024-09-13T18:23:08.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.367+0000 7f8187fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f818810c670 0x7f818819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:08.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.367+0000 7f8187fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f818810c670 0x7f818819cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36844/0 (socket says 172.21.15.114:36844) 2024-09-13T18:23:08.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.367+0000 7f8187fff640 1 -- 172.21.15.114:0/414470909 learned_addr learned my addr 172.21.15.114:0/414470909 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:08.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.367+0000 7f81877fe640 1 --2- 172.21.15.114:0/414470909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81880feac0 0x7f8188198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:08.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.367+0000 7f8186ffd640 1 --2- 172.21.15.114:0/414470909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81880ff470 0x7f8188198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:08.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.367+0000 7f8187fff640 1 -- 172.21.15.114:0/414470909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81880ff470 msgr2=0x7f8188198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:08.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.367+0000 7f8187fff640 1 --2- 172.21.15.114:0/414470909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81880ff470 0x7f8188198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.367+0000 7f8187fff640 1 -- 172.21.15.114:0/414470909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81880feac0 msgr2=0x7f8188198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:08.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.367+0000 7f8187fff640 1 --2- 172.21.15.114:0/414470909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81880feac0 0x7f8188198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.367+0000 7f8187fff640 1 -- 172.21.15.114:0/414470909 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8178009560 con 0x7f818810c670 2024-09-13T18:23:08.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.367+0000 7f8186ffd640 1 --2- 172.21.15.114:0/414470909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81880ff470 0x7f8188198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:23:08.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.367+0000 7f8187fff640 1 --2- 172.21.15.114:0/414470909 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f818810c670 0x7f818819cd20 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f818000ea80 tx=0x7f818000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:08.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.367+0000 7f8184ff9640 1 -- 172.21.15.114:0/414470909 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818000ce20 con 0x7f818810c670 2024-09-13T18:23:08.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.367+0000 7f8184ff9640 1 -- 172.21.15.114:0/414470909 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8180005020 con 0x7f818810c670 2024-09-13T18:23:08.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.367+0000 7f818db1f640 1 -- 172.21.15.114:0/414470909 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f818819d930 con 0x7f818810c670 2024-09-13T18:23:08.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.367+0000 7f8184ff9640 1 -- 172.21.15.114:0/414470909 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8180010690 con 0x7f818810c670 2024-09-13T18:23:08.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.367+0000 7f818db1f640 1 -- 172.21.15.114:0/414470909 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81881a51b0 con 0x7f818810c670 2024-09-13T18:23:08.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.370+0000 7f8190847740 1 Processor -- start 2024-09-13T18:23:08.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.371+0000 7f8184ff9640 1 -- 172.21.15.114:0/414470909 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) v1 ==== 136107+0+0 (secure 0 0 0) 0x7f8180005190 con 0x7f818810c670 2024-09-13T18:23:08.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.371+0000 7f8190847740 1 -- start start 2024-09-13T18:23:08.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.371+0000 7f8190847740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56523b1047c0 0x56523b104ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:08.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.371+0000 7f8190847740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56523b1050e0 0x56523b10f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:08.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.371+0000 7f8190847740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56523b10f980 0x56523b111d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:08.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.371+0000 7f8190847740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56523b06fc60 con 0x56523b10f980 2024-09-13T18:23:08.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.371+0000 7f8190847740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56523b06d3f0 con 0x56523b1047c0 2024-09-13T18:23:08.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.371+0000 7f8190847740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56523b068ce0 con 0x56523b1050e0 2024-09-13T18:23:08.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.371+0000 7f8186ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56523b1047c0 0x56523b104ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:08.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.371+0000 7f8186ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56523b10f980 0x56523b111d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:08.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.371+0000 7f8186ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56523b1047c0 0x56523b104ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:47506/0 (socket says 172.21.15.114:47506) 2024-09-13T18:23:08.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.371+0000 7f8186ffd640 1 -- 172.21.15.114:0/4062006767 learned_addr learned my addr 172.21.15.114:0/4062006767 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:08.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.371+0000 7f81877fe640 1 --2- 172.21.15.114:0/4062006767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56523b1050e0 0x56523b10f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:08.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.371+0000 7f8186ffd640 1 -- 172.21.15.114:0/4062006767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56523b1050e0 msgr2=0x56523b10f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:08.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.371+0000 7f8186ffd640 1 --2- 172.21.15.114:0/4062006767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56523b1050e0 0x56523b10f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.371+0000 7f8186ffd640 1 -- 172.21.15.114:0/4062006767 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56523b10f980 msgr2=0x56523b111d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:08.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.371+0000 7f8186ffd640 1 --2- 172.21.15.114:0/4062006767 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56523b10f980 0x56523b111d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.371+0000 7f8186ffd640 1 -- 172.21.15.114:0/4062006767 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56523b024970 con 0x56523b1047c0 2024-09-13T18:23:08.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.372+0000 7f8184ff9640 1 --2- 172.21.15.114:0/414470909 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f81700a1320 0x7f81700a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:08.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.372+0000 7f8186ffd640 1 --2- 172.21.15.114:0/4062006767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56523b1047c0 0x56523b104ba0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x56523b101e40 tx=0x7f817c0095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bb0a75b46b5458a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:08.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.372+0000 7f8184ff9640 1 -- 172.21.15.114:0/414470909 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f8180014070 con 0x7f818810c670 2024-09-13T18:23:08.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.372+0000 7f81877fe640 1 --2- 172.21.15.114:0/414470909 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f81700a1320 0x7f81700a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:08.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.372+0000 7f816dffb640 1 -- 172.21.15.114:0/4062006767 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817c0120d0 con 0x56523b1047c0 2024-09-13T18:23:08.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.372+0000 7f816dffb640 1 -- 172.21.15.114:0/4062006767 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f817c012230 con 0x56523b1047c0 2024-09-13T18:23:08.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.372+0000 7f816dffb640 1 -- 172.21.15.114:0/4062006767 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817c004af0 con 0x56523b1047c0 2024-09-13T18:23:08.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.373+0000 7f8190847740 1 -- 172.21.15.114:0/4062006767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56523b1047c0 msgr2=0x56523b104ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:08.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.373+0000 7f81877fe640 1 --2- 172.21.15.114:0/414470909 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f81700a1320 0x7f81700a37e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f817802aa00 tx=0x7f8178005a20 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:08.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.373+0000 7f8190847740 1 --2- 172.21.15.114:0/4062006767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56523b1047c0 0x56523b104ba0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x56523b101e40 tx=0x7f817c0095c0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.373+0000 7f8190847740 1 -- 172.21.15.114:0/4062006767 shutdown_connections 2024-09-13T18:23:08.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.373+0000 7f8190847740 1 --2- 172.21.15.114:0/4062006767 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56523b10f980 0x56523b111d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.373+0000 7f8190847740 1 --2- 172.21.15.114:0/4062006767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56523b1050e0 0x56523b10f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.373+0000 7f8190847740 1 --2- 172.21.15.114:0/4062006767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56523b1047c0 0x56523b104ba0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.373+0000 7f8190847740 1 -- 172.21.15.114:0/4062006767 >> 172.21.15.114:0/4062006767 conn(0x56523b100d50 msgr2=0x56523b101130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:08.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.373+0000 7f8190847740 1 -- 172.21.15.114:0/4062006767 shutdown_connections 2024-09-13T18:23:08.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.373+0000 7f8190847740 1 -- 172.21.15.114:0/4062006767 wait complete. 2024-09-13T18:23:08.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.373+0000 7f8190847740 1 -- start start 2024-09-13T18:23:08.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.373+0000 7f8190847740 1 Processor -- start 2024-09-13T18:23:08.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.374+0000 7f8190847740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56523b1047c0 0x56523b10f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:08.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.374+0000 7f8190847740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56523b1050e0 0x56523b10c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:08.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.374+0000 7f8190847740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56523b10f980 0x56523b10cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:08.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.374+0000 7f8190847740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56523b026f80 con 0x56523b10f980 2024-09-13T18:23:08.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.374+0000 7f8190847740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56523b026c20 con 0x56523b1050e0 2024-09-13T18:23:08.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.374+0000 7f8190847740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56523aff3ab0 con 0x56523b1047c0 2024-09-13T18:23:08.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.374+0000 7f8190847740 10 client.0 _ll_register_callbacks cb 0x56523b062350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:23:08.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.374+0000 7f81877fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56523b1047c0 0x56523b10f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:08.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.374+0000 7f81877fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56523b1047c0 0x56523b10f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:33078/0 (socket says 172.21.15.114:33078) 2024-09-13T18:23:08.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.374+0000 7f81877fe640 1 -- 172.21.15.114:0/1440154723 learned_addr learned my addr 172.21.15.114:0/1440154723 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:08.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.374+0000 7f8186ffd640 1 --2- 172.21.15.114:0/1440154723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56523b1050e0 0x56523b10c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:08.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.374+0000 7f8187fff640 1 --2- 172.21.15.114:0/1440154723 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56523b10f980 0x56523b10cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:08.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.374+0000 7f81877fe640 1 -- 172.21.15.114:0/1440154723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56523b1050e0 msgr2=0x56523b10c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:08.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.374+0000 7f81877fe640 1 --2- 172.21.15.114:0/1440154723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56523b1050e0 0x56523b10c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.374+0000 7f81877fe640 1 -- 172.21.15.114:0/1440154723 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56523b10f980 msgr2=0x56523b10cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:08.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.374+0000 7f81877fe640 1 --2- 172.21.15.114:0/1440154723 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56523b10f980 0x56523b10cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.374+0000 7f81877fe640 1 -- 172.21.15.114:0/1440154723 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f817c01a040 con 0x56523b1047c0 2024-09-13T18:23:08.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.374+0000 7f8187fff640 1 --2- 172.21.15.114:0/1440154723 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56523b10f980 0x56523b10cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:23:08.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.375+0000 7f81877fe640 1 --2- 172.21.15.114:0/1440154723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56523b1047c0 0x56523b10f340 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8178006010 tx=0x7f8178009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8aa0e942e9e3101c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:08.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.375+0000 7f8166ffd640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:23:08.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.375+0000 7f8166ffd640 1 -- 172.21.15.114:0/1440154723 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81780035f0 con 0x56523b1047c0 2024-09-13T18:23:08.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.375+0000 7f8166ffd640 1 -- 172.21.15.114:0/1440154723 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f8178003750 con 0x56523b1047c0 2024-09-13T18:23:08.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.375+0000 7f8190847740 1 -- 172.21.15.114:0/1440154723 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56523ad5c590 con 0x56523b1047c0 2024-09-13T18:23:08.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.375+0000 7f8166ffd640 1 -- 172.21.15.114:0/1440154723 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8178003910 con 0x56523b1047c0 2024-09-13T18:23:08.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.375+0000 7f8190847740 10 client.25438 fetch_fsmap learned FSMap version 30 2024-09-13T18:23:08.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.375+0000 7f8190847740 1 -- 172.21.15.114:0/1440154723 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56523b067360 con 0x56523b1047c0 2024-09-13T18:23:08.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.375+0000 7f8166ffd640 1 -- 172.21.15.114:0/1440154723 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56523ad5c590 con 0x56523b1047c0 2024-09-13T18:23:08.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.376+0000 7f8166ffd640 1 -- 172.21.15.114:0/1440154723 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8178047a50 con 0x56523b1047c0 2024-09-13T18:23:08.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.376+0000 7f8190847740 10 client.25438 fetch_fsmap finished waiting for FSMap version 30 2024-09-13T18:23:08.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.376+0000 7f8190847740 10 client.25438 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:23:08.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.376+0000 7f8190847740 20 client.25438 populate_metadata read hostname 'smithi114' 2024-09-13T18:23:08.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.376+0000 7f8190847740 1 --2- 172.21.15.114:0/1440154723 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56523b11f660 0x56523b121a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:08.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.376+0000 7f8190847740 4 client.25438 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:23:08.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.376+0000 7f8190847740 1 -- 172.21.15.114:0/1440154723 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56523ad6d860 con 0x56523b11f660 2024-09-13T18:23:08.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.376+0000 7f8186ffd640 1 --2- 172.21.15.114:0/1440154723 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56523b11f660 0x56523b121a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:08.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.376+0000 7f8166ffd640 10 client.25438 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:23:08.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.376+0000 7f8186ffd640 1 --2- 172.21.15.114:0/1440154723 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56523b11f660 0x56523b121a80 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b948a4e5bb6427bb server_cookie=1565adfe376ddeab in_seq=0 out_seq=0 2024-09-13T18:23:08.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.377+0000 7f8166ffd640 1 -- 172.21.15.114:0/1440154723 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x56523ad6d860 con 0x56523b11f660 2024-09-13T18:23:08.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.377+0000 7f8166ffd640 10 client.25438 handle_command_reply: tid=0 2024-09-13T18:23:08.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.377+0000 7f8190847740 1 client.25438 shutdown 2024-09-13T18:23:08.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.377+0000 7f8190847740 1 -- 172.21.15.114:0/1440154723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56523b1047c0 msgr2=0x56523b10f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:08.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.377+0000 7f8190847740 1 --2- 172.21.15.114:0/1440154723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56523b1047c0 0x56523b10f340 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8178006010 tx=0x7f8178009260 comp rx=0 tx=0).stop 2024-09-13T18:23:08.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.377+0000 7f8190847740 1 -- 172.21.15.114:0/1440154723 shutdown_connections 2024-09-13T18:23:08.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.377+0000 7f8190847740 1 --2- 172.21.15.114:0/1440154723 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56523b11f660 0x56523b121a80 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.377+0000 7f8190847740 1 --2- 172.21.15.114:0/1440154723 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56523b10f980 0x56523b10cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.377+0000 7f8166ffd640 0 client.25438 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:23:08.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.377+0000 7f8190847740 1 --2- 172.21.15.114:0/1440154723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56523b1050e0 0x56523b10c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.377+0000 7f8190847740 1 --2- 172.21.15.114:0/1440154723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56523b1047c0 0x56523b10f340 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.377+0000 7f8190847740 1 -- 172.21.15.114:0/1440154723 >> 172.21.15.114:0/1440154723 conn(0x56523b100d50 msgr2=0x56523b1061a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:08.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.377+0000 7f8190847740 1 -- 172.21.15.114:0/1440154723 shutdown_connections 2024-09-13T18:23:08.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.377+0000 7f8190847740 1 -- 172.21.15.114:0/1440154723 wait complete. 2024-09-13T18:23:08.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.377+0000 7f8190847740 20 client.25438 trim_cache size 0 max 16384 2024-09-13T18:23:08.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.391+0000 7f8190847740 1 Processor -- start 2024-09-13T18:23:08.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.391+0000 7f8190847740 1 -- start start 2024-09-13T18:23:08.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.392+0000 7f8190847740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56523b1047c0 0x56523b1068b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:08.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.392+0000 7f8190847740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56523b1050e0 0x56523b116650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:08.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.392+0000 7f8190847740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56523b10f980 0x56523b116b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:08.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.392+0000 7f8190847740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56523ad5c590 con 0x56523b10f980 2024-09-13T18:23:08.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.392+0000 7f8190847740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56523b101ab0 con 0x56523b1047c0 2024-09-13T18:23:08.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.392+0000 7f8190847740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56523b105d20 con 0x56523b1050e0 2024-09-13T18:23:08.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.392+0000 7f8187fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56523b10f980 0x56523b116b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:08.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.392+0000 7f8186ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56523b1050e0 0x56523b116650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:08.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.392+0000 7f8187fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56523b10f980 0x56523b116b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36872/0 (socket says 172.21.15.114:36872) 2024-09-13T18:23:08.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.392+0000 7f8187fff640 1 -- 172.21.15.114:0/4090181798 learned_addr learned my addr 172.21.15.114:0/4090181798 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:08.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.392+0000 7f81877fe640 1 --2- 172.21.15.114:0/4090181798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56523b1047c0 0x56523b1068b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:08.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.392+0000 7f8187fff640 1 -- 172.21.15.114:0/4090181798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56523b1050e0 msgr2=0x56523b116650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:08.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.392+0000 7f8187fff640 1 --2- 172.21.15.114:0/4090181798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56523b1050e0 0x56523b116650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.392+0000 7f8187fff640 1 -- 172.21.15.114:0/4090181798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56523b1047c0 msgr2=0x56523b1068b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:08.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.392+0000 7f8187fff640 1 --2- 172.21.15.114:0/4090181798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56523b1047c0 0x56523b1068b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.392+0000 7f8187fff640 1 -- 172.21.15.114:0/4090181798 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56523b0091c0 con 0x56523b10f980 2024-09-13T18:23:08.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.392+0000 7f8186ffd640 1 --2- 172.21.15.114:0/4090181798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56523b1050e0 0x56523b116650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:23:08.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.392+0000 7f8187fff640 1 --2- 172.21.15.114:0/4090181798 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56523b10f980 0x56523b116b90 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f818009a240 tx=0x7f81800ba120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ad875f9e6f25ba3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:08.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.392+0000 7f816e7fc640 1 -- 172.21.15.114:0/4090181798 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818009e170 con 0x56523b10f980 2024-09-13T18:23:08.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.393+0000 7f816e7fc640 1 -- 172.21.15.114:0/4090181798 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f81800b5070 con 0x56523b10f980 2024-09-13T18:23:08.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.393+0000 7f816e7fc640 1 -- 172.21.15.114:0/4090181798 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81800a29c0 con 0x56523b10f980 2024-09-13T18:23:08.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.393+0000 7f8190847740 1 -- 172.21.15.114:0/4090181798 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56523b10f980 msgr2=0x56523b116b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:08.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.393+0000 7f8190847740 1 --2- 172.21.15.114:0/4090181798 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56523b10f980 0x56523b116b90 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f818009a240 tx=0x7f81800ba120 comp rx=0 tx=0).stop 2024-09-13T18:23:08.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.393+0000 7f8190847740 1 -- 172.21.15.114:0/4090181798 shutdown_connections 2024-09-13T18:23:08.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.393+0000 7f8190847740 1 --2- 172.21.15.114:0/4090181798 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56523b10f980 0x56523b116b90 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.393+0000 7f8190847740 1 --2- 172.21.15.114:0/4090181798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56523b1050e0 0x56523b116650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.393+0000 7f8190847740 1 --2- 172.21.15.114:0/4090181798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56523b1047c0 0x56523b1068b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.393+0000 7f8190847740 1 -- 172.21.15.114:0/4090181798 >> 172.21.15.114:0/4090181798 conn(0x56523b100d50 msgr2=0x56523b1061a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:08.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.393+0000 7f8190847740 1 -- 172.21.15.114:0/4090181798 shutdown_connections 2024-09-13T18:23:08.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.393+0000 7f8190847740 1 -- 172.21.15.114:0/4090181798 wait complete. 2024-09-13T18:23:08.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.394+0000 7f8190847740 1 -- start start 2024-09-13T18:23:08.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.394+0000 7f8190847740 1 Processor -- start 2024-09-13T18:23:08.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.394+0000 7f8190847740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56523b1047c0 0x56523b115e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:08.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.394+0000 7f8190847740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56523b1050e0 0x56523b116380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:08.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.394+0000 7f8190847740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56523b10f980 0x56523b16ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:08.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.394+0000 7f8190847740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56523b16f2d0 con 0x56523b1050e0 2024-09-13T18:23:08.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.394+0000 7f8190847740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56523b16f440 con 0x56523b10f980 2024-09-13T18:23:08.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.394+0000 7f8190847740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56523b16f600 con 0x56523b1047c0 2024-09-13T18:23:08.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.394+0000 7f8190847740 10 client.0 _ll_register_callbacks cb 0x7f8178009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:23:08.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.394+0000 7f8186ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56523b1050e0 0x56523b116380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:08.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.394+0000 7f8186ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56523b1050e0 0x56523b116380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36884/0 (socket says 172.21.15.114:36884) 2024-09-13T18:23:08.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.394+0000 7f8186ffd640 1 -- 172.21.15.114:0/2725602856 learned_addr learned my addr 172.21.15.114:0/2725602856 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:08.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.394+0000 7f81877fe640 1 --2- 172.21.15.114:0/2725602856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56523b1047c0 0x56523b115e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:08.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.394+0000 7f8187fff640 1 --2- 172.21.15.114:0/2725602856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56523b10f980 0x56523b16ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:08.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.395+0000 7f8186ffd640 1 -- 172.21.15.114:0/2725602856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56523b1047c0 msgr2=0x56523b115e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:08.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.395+0000 7f8186ffd640 1 --2- 172.21.15.114:0/2725602856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56523b1047c0 0x56523b115e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.395+0000 7f8186ffd640 1 -- 172.21.15.114:0/2725602856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56523b10f980 msgr2=0x56523b16ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:08.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.395+0000 7f8186ffd640 1 --2- 172.21.15.114:0/2725602856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56523b10f980 0x56523b16ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.395+0000 7f8186ffd640 1 -- 172.21.15.114:0/2725602856 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56523b024970 con 0x56523b1050e0 2024-09-13T18:23:08.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.395+0000 7f8186ffd640 1 --2- 172.21.15.114:0/2725602856 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56523b1050e0 0x56523b116380 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f817c000c00 tx=0x7f817c009630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=964b8627177d21fb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:08.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.395+0000 7f8166ffd640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:23:08.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.395+0000 7f8166ffd640 1 -- 172.21.15.114:0/2725602856 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817c016040 con 0x56523b1050e0 2024-09-13T18:23:08.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.395+0000 7f8166ffd640 1 -- 172.21.15.114:0/2725602856 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f817c0123a0 con 0x56523b1050e0 2024-09-13T18:23:08.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.395+0000 7f8190847740 1 -- 172.21.15.114:0/2725602856 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f817c012230 con 0x56523b1050e0 2024-09-13T18:23:08.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.395+0000 7f8166ffd640 1 -- 172.21.15.114:0/2725602856 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f817c0120b0 con 0x56523b1050e0 2024-09-13T18:23:08.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.395+0000 7f8190847740 10 client.15990 fetch_fsmap learned FSMap version 30 2024-09-13T18:23:08.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.395+0000 7f8190847740 1 -- 172.21.15.114:0/2725602856 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f81881a51b0 con 0x56523b1050e0 2024-09-13T18:23:08.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.395+0000 7f8166ffd640 1 -- 172.21.15.114:0/2725602856 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f817c012230 con 0x56523b1050e0 2024-09-13T18:23:08.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.396+0000 7f8166ffd640 1 -- 172.21.15.114:0/2725602856 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f817c0272f0 con 0x56523b1050e0 2024-09-13T18:23:08.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.396+0000 7f8190847740 10 client.15990 fetch_fsmap finished waiting for FSMap version 30 2024-09-13T18:23:08.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.396+0000 7f8190847740 10 client.15990 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:23:08.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.396+0000 7f8190847740 20 client.15990 populate_metadata read hostname 'smithi114' 2024-09-13T18:23:08.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.396+0000 7f8190847740 1 --2- 172.21.15.114:0/2725602856 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56523b178050 0x56523b17a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:08.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.396+0000 7f8190847740 4 client.15990 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-13T18:23:08.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.396+0000 7f8190847740 1 -- 172.21.15.114:0/2725602856 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56523adc0eb0 con 0x56523b178050 2024-09-13T18:23:08.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.396+0000 7f81877fe640 1 --2- 172.21.15.114:0/2725602856 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56523b178050 0x56523b17a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:08.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.396+0000 7f81877fe640 1 --2- 172.21.15.114:0/2725602856 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56523b178050 0x56523b17a470 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce07fc53a4710e76 server_cookie=458bc7e7a1b48d5a in_seq=0 out_seq=0 2024-09-13T18:23:08.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.396+0000 7f8166ffd640 10 client.15990 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:23:08.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.397+0000 7f8166ffd640 1 -- 172.21.15.114:0/2725602856 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56523adc0eb0 con 0x56523b178050 2024-09-13T18:23:08.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.397+0000 7f8166ffd640 10 client.15990 handle_command_reply: tid=0 2024-09-13T18:23:08.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.397+0000 7f8190847740 1 client.15990 shutdown 2024-09-13T18:23:08.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.398+0000 7f8190847740 1 -- 172.21.15.114:0/2725602856 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56523b1050e0 msgr2=0x56523b116380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:08.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.398+0000 7f8190847740 1 --2- 172.21.15.114:0/2725602856 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56523b1050e0 0x56523b116380 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f817c000c00 tx=0x7f817c009630 comp rx=0 tx=0).stop 2024-09-13T18:23:08.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.398+0000 7f8190847740 1 -- 172.21.15.114:0/2725602856 shutdown_connections 2024-09-13T18:23:08.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.398+0000 7f8190847740 1 --2- 172.21.15.114:0/2725602856 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56523b178050 0x56523b17a470 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.398+0000 7f8166ffd640 0 client.15990 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:23:08.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.398+0000 7f8190847740 1 --2- 172.21.15.114:0/2725602856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56523b10f980 0x56523b16ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.398+0000 7f8190847740 1 --2- 172.21.15.114:0/2725602856 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56523b1050e0 0x56523b116380 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.398+0000 7f8190847740 1 --2- 172.21.15.114:0/2725602856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56523b1047c0 0x56523b115e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.398+0000 7f8190847740 1 -- 172.21.15.114:0/2725602856 >> 172.21.15.114:0/2725602856 conn(0x56523b100d50 msgr2=0x56523b0ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:08.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.398+0000 7f8190847740 1 -- 172.21.15.114:0/2725602856 shutdown_connections 2024-09-13T18:23:08.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.398+0000 7f8190847740 1 -- 172.21.15.114:0/2725602856 wait complete. 2024-09-13T18:23:08.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.398+0000 7f8190847740 20 client.15990 trim_cache size 0 max 16384 2024-09-13T18:23:08.405 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:23:08.405 INFO:teuthology.orchestra.run.smithi114.stdout: "return_code": 0, 2024-09-13T18:23:08.405 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_tag": "e98642d0-00e3-423e-8ff3-881b3d1500ce", 2024-09-13T18:23:08.406 INFO:teuthology.orchestra.run.smithi114.stdout: "mode": "asynchronous" 2024-09-13T18:23:08.406 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:23:08.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.398+0000 7f818db1f640 1 -- 172.21.15.114:0/414470909 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f81700a1320 msgr2=0x7f81700a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:08.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.398+0000 7f818db1f640 1 --2- 172.21.15.114:0/414470909 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f81700a1320 0x7f81700a37e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f817802aa00 tx=0x7f8178005a20 comp rx=0 tx=0).stop 2024-09-13T18:23:08.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.398+0000 7f818db1f640 1 -- 172.21.15.114:0/414470909 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f818810c670 msgr2=0x7f818819cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:08.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.398+0000 7f818db1f640 1 --2- 172.21.15.114:0/414470909 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f818810c670 0x7f818819cd20 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f818000ea80 tx=0x7f818000ef40 comp rx=0 tx=0).stop 2024-09-13T18:23:08.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.399+0000 7f818db1f640 1 -- 172.21.15.114:0/414470909 shutdown_connections 2024-09-13T18:23:08.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.399+0000 7f818db1f640 1 --2- 172.21.15.114:0/414470909 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f81700a1320 0x7f81700a37e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.399+0000 7f818db1f640 1 --2- 172.21.15.114:0/414470909 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f818810c670 0x7f818819cd20 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.399+0000 7f818db1f640 1 --2- 172.21.15.114:0/414470909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81880ff470 0x7f8188198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.399+0000 7f818db1f640 1 --2- 172.21.15.114:0/414470909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81880feac0 0x7f8188198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.399+0000 7f818db1f640 1 -- 172.21.15.114:0/414470909 >> 172.21.15.114:0/414470909 conn(0x7f8188075c60 msgr2=0x7f81880fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:08.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.399+0000 7f818db1f640 1 -- 172.21.15.114:0/414470909 shutdown_connections 2024-09-13T18:23:08.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.399+0000 7f818db1f640 1 -- 172.21.15.114:0/414470909 wait complete. 2024-09-13T18:23:08.411 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T18:23:08.432 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:23:08 smithi114 ceph-mon[30897]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-13T18:23:08.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:23:08 smithi204 ceph-mon[34777]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-13T18:23:08.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.528+0000 7f2e568c9640 1 -- 172.21.15.114:0/3397920928 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e500ff830 msgr2=0x7f2e50103870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:08.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.528+0000 7f2e568c9640 1 --2- 172.21.15.114:0/3397920928 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e500ff830 0x7f2e50103870 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f2e44009930 tx=0x7f2e4402a420 comp rx=0 tx=0).stop 2024-09-13T18:23:08.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.528+0000 7f2e568c9640 1 -- 172.21.15.114:0/3397920928 shutdown_connections 2024-09-13T18:23:08.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.528+0000 7f2e568c9640 1 --2- 172.21.15.114:0/3397920928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e50103fd0 0x7f2e5010bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.528+0000 7f2e568c9640 1 --2- 172.21.15.114:0/3397920928 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e500ff830 0x7f2e50103870 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.528+0000 7f2e568c9640 1 --2- 172.21.15.114:0/3397920928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e500fee80 0x7f2e500ff260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.528+0000 7f2e568c9640 1 -- 172.21.15.114:0/3397920928 >> 172.21.15.114:0/3397920928 conn(0x7f2e500fa8d0 msgr2=0x7f2e500fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:08.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.528+0000 7f2e568c9640 1 -- 172.21.15.114:0/3397920928 shutdown_connections 2024-09-13T18:23:08.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.528+0000 7f2e568c9640 1 -- 172.21.15.114:0/3397920928 wait complete. 2024-09-13T18:23:08.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.529+0000 7f2e568c9640 1 Processor -- start 2024-09-13T18:23:08.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.529+0000 7f2e568c9640 1 -- start start 2024-09-13T18:23:08.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.529+0000 7f2e568c9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e500fee80 0x7f2e501962e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:08.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.529+0000 7f2e568c9640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e50103fd0 0x7f2e50196820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:08.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.529+0000 7f2e568c9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e5019ac10 0x7f2e5019b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:08.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.529+0000 7f2e568c9640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5019b600 con 0x7f2e50103fd0 2024-09-13T18:23:08.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.529+0000 7f2e568c9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5019b770 con 0x7f2e5019ac10 2024-09-13T18:23:08.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.529+0000 7f2e568c9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5019b8e0 con 0x7f2e500fee80 2024-09-13T18:23:08.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.529+0000 7f2e4f7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e50103fd0 0x7f2e50196820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:08.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.530+0000 7f2e4f7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e50103fd0 0x7f2e50196820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36904/0 (socket says 172.21.15.114:36904) 2024-09-13T18:23:08.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.530+0000 7f2e4f7fe640 1 -- 172.21.15.114:0/2803430486 learned_addr learned my addr 172.21.15.114:0/2803430486 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:08.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.530+0000 7f2e4ffff640 1 --2- 172.21.15.114:0/2803430486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e500fee80 0x7f2e501962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:08.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.530+0000 7f2e54e66640 1 --2- 172.21.15.114:0/2803430486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e5019ac10 0x7f2e5019b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:08.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.530+0000 7f2e4f7fe640 1 -- 172.21.15.114:0/2803430486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e500fee80 msgr2=0x7f2e501962e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:08.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.530+0000 7f2e4f7fe640 1 --2- 172.21.15.114:0/2803430486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e500fee80 0x7f2e501962e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.530+0000 7f2e4f7fe640 1 -- 172.21.15.114:0/2803430486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e5019ac10 msgr2=0x7f2e5019b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:08.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.530+0000 7f2e4f7fe640 1 --2- 172.21.15.114:0/2803430486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e5019ac10 0x7f2e5019b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.530+0000 7f2e4f7fe640 1 -- 172.21.15.114:0/2803430486 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e44009560 con 0x7f2e50103fd0 2024-09-13T18:23:08.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.530+0000 7f2e54e66640 1 --2- 172.21.15.114:0/2803430486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e5019ac10 0x7f2e5019b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:23:08.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.530+0000 7f2e4f7fe640 1 --2- 172.21.15.114:0/2803430486 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e50103fd0 0x7f2e50196820 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f2e44005ab0 tx=0x7f2e44032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:08.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.530+0000 7f2e4d7fa640 1 -- 172.21.15.114:0/2803430486 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e440044d0 con 0x7f2e50103fd0 2024-09-13T18:23:08.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.530+0000 7f2e4d7fa640 1 -- 172.21.15.114:0/2803430486 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2e44032e10 con 0x7f2e50103fd0 2024-09-13T18:23:08.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.530+0000 7f2e568c9640 1 -- 172.21.15.114:0/2803430486 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e501a2ec0 con 0x7f2e50103fd0 2024-09-13T18:23:08.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.530+0000 7f2e4d7fa640 1 -- 172.21.15.114:0/2803430486 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e44003850 con 0x7f2e50103fd0 2024-09-13T18:23:08.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.530+0000 7f2e568c9640 1 -- 172.21.15.114:0/2803430486 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e501a3230 con 0x7f2e50103fd0 2024-09-13T18:23:08.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.533+0000 7f2e595e9740 1 Processor -- start 2024-09-13T18:23:08.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.533+0000 7f2e595e9740 1 -- start start 2024-09-13T18:23:08.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.534+0000 7f2e595e9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b92d269e10 0x55b92d26a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:08.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.534+0000 7f2e595e9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b92d26a730 0x55b92d274a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:08.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.534+0000 7f2e595e9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b92d274fd0 0x55b92d2773c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:08.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.534+0000 7f2e595e9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b92d1d4f20 con 0x55b92d26a730 2024-09-13T18:23:08.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.534+0000 7f2e595e9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b92d1d26b0 con 0x55b92d274fd0 2024-09-13T18:23:08.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.534+0000 7f2e595e9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b92d1cdfa0 con 0x55b92d269e10 2024-09-13T18:23:08.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.534+0000 7f2e4d7fa640 1 -- 172.21.15.114:0/2803430486 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) v1 ==== 136107+0+0 (secure 0 0 0) 0x7f2e44004850 con 0x7f2e50103fd0 2024-09-13T18:23:08.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.534+0000 7f2e4ffff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b92d26a730 0x55b92d274a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:08.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.534+0000 7f2e4ffff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b92d26a730 0x55b92d274a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36918/0 (socket says 172.21.15.114:36918) 2024-09-13T18:23:08.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.534+0000 7f2e4ffff640 1 -- 172.21.15.114:0/797361694 learned_addr learned my addr 172.21.15.114:0/797361694 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:08.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.534+0000 7f2e54e66640 1 --2- 172.21.15.114:0/797361694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b92d269e10 0x55b92d26a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:08.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.534+0000 7f2e4ffff640 1 -- 172.21.15.114:0/797361694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b92d269e10 msgr2=0x55b92d26a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:08.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.534+0000 7f2e4ffff640 1 --2- 172.21.15.114:0/797361694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b92d269e10 0x55b92d26a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.534+0000 7f2e4ffff640 1 -- 172.21.15.114:0/797361694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b92d274fd0 msgr2=0x55b92d2773c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:08.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.534+0000 7f2e4ffff640 1 --2- 172.21.15.114:0/797361694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b92d274fd0 0x55b92d2773c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.534+0000 7f2e4ffff640 1 -- 172.21.15.114:0/797361694 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b92d188840 con 0x55b92d26a730 2024-09-13T18:23:08.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.534+0000 7f2e54e66640 1 --2- 172.21.15.114:0/797361694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b92d269e10 0x55b92d26a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:23:08.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.534+0000 7f2e4ffff640 1 --2- 172.21.15.114:0/797361694 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b92d26a730 0x55b92d274a90 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f2e4000bcd0 tx=0x7f2e400107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f191b98b6652d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:08.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.535+0000 7f2e4d7fa640 1 --2- 172.21.15.114:0/2803430486 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2e380a1320 0x7f2e380a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:08.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.535+0000 7f2e367fc640 1 -- 172.21.15.114:0/797361694 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e40005220 con 0x55b92d26a730 2024-09-13T18:23:08.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.535+0000 7f2e54e66640 1 --2- 172.21.15.114:0/2803430486 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2e380a1320 0x7f2e380a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:08.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.535+0000 7f2e367fc640 1 -- 172.21.15.114:0/797361694 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2e40015070 con 0x55b92d26a730 2024-09-13T18:23:08.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.535+0000 7f2e4d7fa640 1 -- 172.21.15.114:0/2803430486 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2e440ee190 con 0x7f2e50103fd0 2024-09-13T18:23:08.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.535+0000 7f2e367fc640 1 -- 172.21.15.114:0/797361694 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e40011460 con 0x55b92d26a730 2024-09-13T18:23:08.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.536+0000 7f2e595e9740 1 -- 172.21.15.114:0/797361694 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b92d26a730 msgr2=0x55b92d274a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:08.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.536+0000 7f2e54e66640 1 --2- 172.21.15.114:0/2803430486 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2e380a1320 0x7f2e380a37e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2e48005fd0 tx=0x7f2e48009500 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:08.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.536+0000 7f2e595e9740 1 --2- 172.21.15.114:0/797361694 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b92d26a730 0x55b92d274a90 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f2e4000bcd0 tx=0x7f2e400107a0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.536+0000 7f2e595e9740 1 -- 172.21.15.114:0/797361694 shutdown_connections 2024-09-13T18:23:08.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.536+0000 7f2e595e9740 1 --2- 172.21.15.114:0/797361694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b92d274fd0 0x55b92d2773c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.536+0000 7f2e595e9740 1 --2- 172.21.15.114:0/797361694 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b92d26a730 0x55b92d274a90 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.536+0000 7f2e595e9740 1 --2- 172.21.15.114:0/797361694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b92d269e10 0x55b92d26a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.536+0000 7f2e595e9740 1 -- 172.21.15.114:0/797361694 >> 172.21.15.114:0/797361694 conn(0x55b92d266380 msgr2=0x55b92d266760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:08.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.536+0000 7f2e595e9740 1 -- 172.21.15.114:0/797361694 shutdown_connections 2024-09-13T18:23:08.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.536+0000 7f2e595e9740 1 -- 172.21.15.114:0/797361694 wait complete. 2024-09-13T18:23:08.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.536+0000 7f2e595e9740 1 -- start start 2024-09-13T18:23:08.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.537+0000 7f2e595e9740 1 Processor -- start 2024-09-13T18:23:08.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.537+0000 7f2e595e9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b92d269e10 0x55b92d26deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:08.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.537+0000 7f2e595e9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b92d274fd0 0x55b92d26e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:08.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.537+0000 7f2e595e9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b92d26e930 0x55b92d2726f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:08.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.537+0000 7f2e595e9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b92d16f2a0 con 0x55b92d26e930 2024-09-13T18:23:08.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.537+0000 7f2e595e9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b92cea6430 con 0x55b92d274fd0 2024-09-13T18:23:08.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.537+0000 7f2e595e9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b92d151680 con 0x55b92d269e10 2024-09-13T18:23:08.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.537+0000 7f2e595e9740 10 client.0 _ll_register_callbacks cb 0x55b92d1c7610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:23:08.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.537+0000 7f2e4f7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b92d274fd0 0x55b92d26e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:08.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.537+0000 7f2e4f7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b92d274fd0 0x55b92d26e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:47552/0 (socket says 172.21.15.114:47552) 2024-09-13T18:23:08.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.537+0000 7f2e4f7fe640 1 -- 172.21.15.114:0/1885444061 learned_addr learned my addr 172.21.15.114:0/1885444061 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:08.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.537+0000 7f2e54e66640 1 --2- 172.21.15.114:0/1885444061 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b92d26e930 0x55b92d2726f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:08.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.537+0000 7f2e4ffff640 1 --2- 172.21.15.114:0/1885444061 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b92d269e10 0x55b92d26deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:08.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.537+0000 7f2e4f7fe640 1 -- 172.21.15.114:0/1885444061 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b92d269e10 msgr2=0x55b92d26deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:08.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.537+0000 7f2e4f7fe640 1 --2- 172.21.15.114:0/1885444061 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b92d269e10 0x55b92d26deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.537+0000 7f2e4f7fe640 1 -- 172.21.15.114:0/1885444061 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b92d26e930 msgr2=0x55b92d2726f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:08.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.538+0000 7f2e4f7fe640 1 --2- 172.21.15.114:0/1885444061 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b92d26e930 0x55b92d2726f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.538+0000 7f2e4f7fe640 1 -- 172.21.15.114:0/1885444061 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e4000b900 con 0x55b92d274fd0 2024-09-13T18:23:08.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.538+0000 7f2e54e66640 1 --2- 172.21.15.114:0/1885444061 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b92d26e930 0x55b92d2726f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:23:08.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.538+0000 7f2e4f7fe640 1 --2- 172.21.15.114:0/1885444061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b92d274fd0 0x55b92d26e3f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2e440ebb90 tx=0x7f2e440ba730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e12a2542a74385d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:08.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.538+0000 7f2e2f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:23:08.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.538+0000 7f2e2f7fe640 1 -- 172.21.15.114:0/1885444061 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e440bb230 con 0x55b92d274fd0 2024-09-13T18:23:08.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.538+0000 7f2e2f7fe640 1 -- 172.21.15.114:0/1885444061 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2e440badd0 con 0x55b92d274fd0 2024-09-13T18:23:08.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.538+0000 7f2e595e9740 1 -- 172.21.15.114:0/1885444061 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b92cec14e0 con 0x55b92d274fd0 2024-09-13T18:23:08.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.538+0000 7f2e2f7fe640 1 -- 172.21.15.114:0/1885444061 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e440bec00 con 0x55b92d274fd0 2024-09-13T18:23:08.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.538+0000 7f2e595e9740 10 client.25412 fetch_fsmap learned FSMap version 30 2024-09-13T18:23:08.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.539+0000 7f2e595e9740 1 -- 172.21.15.114:0/1885444061 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55b92d1cc620 con 0x55b92d274fd0 2024-09-13T18:23:08.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.539+0000 7f2e2f7fe640 1 -- 172.21.15.114:0/1885444061 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2e440c3250 con 0x55b92d274fd0 2024-09-13T18:23:08.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.539+0000 7f2e2f7fe640 1 -- 172.21.15.114:0/1885444061 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2e4404d070 con 0x55b92d274fd0 2024-09-13T18:23:08.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.540+0000 7f2e595e9740 10 client.25412 fetch_fsmap finished waiting for FSMap version 30 2024-09-13T18:23:08.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.540+0000 7f2e595e9740 10 client.25412 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:23:08.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.540+0000 7f2e595e9740 20 client.25412 populate_metadata read hostname 'smithi114' 2024-09-13T18:23:08.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.540+0000 7f2e595e9740 1 --2- 172.21.15.114:0/1885444061 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55b92d285280 0x55b92d2876b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:08.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.540+0000 7f2e595e9740 4 client.25412 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:23:08.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.540+0000 7f2e595e9740 1 -- 172.21.15.114:0/1885444061 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b92cec14e0 con 0x55b92d285280 2024-09-13T18:23:08.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.540+0000 7f2e4ffff640 1 --2- 172.21.15.114:0/1885444061 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55b92d285280 0x55b92d2876b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:08.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.540+0000 7f2e2f7fe640 10 client.25412 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:23:08.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.540+0000 7f2e4ffff640 1 --2- 172.21.15.114:0/1885444061 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55b92d285280 0x55b92d2876b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=164358f5e5fdf199 server_cookie=8a44789a018403e3 in_seq=0 out_seq=0 2024-09-13T18:23:08.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.541+0000 7f2e2f7fe640 1 -- 172.21.15.114:0/1885444061 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55b92cec14e0 con 0x55b92d285280 2024-09-13T18:23:08.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.541+0000 7f2e2f7fe640 10 client.25412 handle_command_reply: tid=0 2024-09-13T18:23:08.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.541+0000 7f2e595e9740 1 client.25412 shutdown 2024-09-13T18:23:08.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.542+0000 7f2e595e9740 1 -- 172.21.15.114:0/1885444061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b92d274fd0 msgr2=0x55b92d26e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:08.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.542+0000 7f2e595e9740 1 --2- 172.21.15.114:0/1885444061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b92d274fd0 0x55b92d26e3f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2e440ebb90 tx=0x7f2e440ba730 comp rx=0 tx=0).stop 2024-09-13T18:23:08.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.542+0000 7f2e595e9740 1 -- 172.21.15.114:0/1885444061 shutdown_connections 2024-09-13T18:23:08.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.542+0000 7f2e595e9740 1 --2- 172.21.15.114:0/1885444061 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55b92d285280 0x55b92d2876b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.542+0000 7f2e595e9740 1 --2- 172.21.15.114:0/1885444061 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b92d26e930 0x55b92d2726f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.542+0000 7f2e595e9740 1 --2- 172.21.15.114:0/1885444061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b92d274fd0 0x55b92d26e3f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.542+0000 7f2e595e9740 1 --2- 172.21.15.114:0/1885444061 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b92d269e10 0x55b92d26deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.542+0000 7f2e595e9740 1 -- 172.21.15.114:0/1885444061 >> 172.21.15.114:0/1885444061 conn(0x55b92d266380 msgr2=0x55b92d26c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:08.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.542+0000 7f2e2f7fe640 0 client.25412 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:23:08.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.542+0000 7f2e595e9740 1 -- 172.21.15.114:0/1885444061 shutdown_connections 2024-09-13T18:23:08.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.542+0000 7f2e595e9740 1 -- 172.21.15.114:0/1885444061 wait complete. 2024-09-13T18:23:08.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.542+0000 7f2e595e9740 20 client.25412 trim_cache size 0 max 16384 2024-09-13T18:23:08.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.568+0000 7f2e595e9740 1 Processor -- start 2024-09-13T18:23:08.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.568+0000 7f2e595e9740 1 -- start start 2024-09-13T18:23:08.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.568+0000 7f2e595e9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b92d269e10 0x55b92d2726e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:08.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.569+0000 7f2e595e9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b92d26e930 0x55b92d261310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:08.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.569+0000 7f2e595e9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b92d274fd0 0x55b92d26ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:08.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.569+0000 7f2e595e9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b92d277960 con 0x55b92d269e10 2024-09-13T18:23:08.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.569+0000 7f2e595e9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b92d260710 con 0x55b92d26e930 2024-09-13T18:23:08.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.569+0000 7f2e595e9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b92d267410 con 0x55b92d274fd0 2024-09-13T18:23:08.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.569+0000 7f2e4ffff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b92d269e10 0x55b92d2726e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:08.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.569+0000 7f2e4ffff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b92d269e10 0x55b92d2726e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36946/0 (socket says 172.21.15.114:36946) 2024-09-13T18:23:08.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.569+0000 7f2e4ffff640 1 -- 172.21.15.114:0/1891843630 learned_addr learned my addr 172.21.15.114:0/1891843630 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:08.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.569+0000 7f2e54e66640 1 --2- 172.21.15.114:0/1891843630 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b92d274fd0 0x55b92d26ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:08.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.569+0000 7f2e4f7fe640 1 --2- 172.21.15.114:0/1891843630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b92d26e930 0x55b92d261310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:08.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.569+0000 7f2e4ffff640 1 -- 172.21.15.114:0/1891843630 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b92d274fd0 msgr2=0x55b92d26ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:08.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.569+0000 7f2e4ffff640 1 --2- 172.21.15.114:0/1891843630 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b92d274fd0 0x55b92d26ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.569+0000 7f2e4ffff640 1 -- 172.21.15.114:0/1891843630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b92d26e930 msgr2=0x55b92d261310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:08.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.569+0000 7f2e4ffff640 1 --2- 172.21.15.114:0/1891843630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b92d26e930 0x55b92d261310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.569+0000 7f2e4ffff640 1 -- 172.21.15.114:0/1891843630 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b92d16e790 con 0x55b92d269e10 2024-09-13T18:23:08.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.569+0000 7f2e4f7fe640 1 --2- 172.21.15.114:0/1891843630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b92d26e930 0x55b92d261310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:23:08.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.569+0000 7f2e4ffff640 1 --2- 172.21.15.114:0/1891843630 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b92d269e10 0x55b92d2726e0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f2e40010ca0 tx=0x7f2e40005ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e790850b63df177d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:08.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.569+0000 7f2e36ffd640 1 -- 172.21.15.114:0/1891843630 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e4000be80 con 0x55b92d269e10 2024-09-13T18:23:08.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.570+0000 7f2e36ffd640 1 -- 172.21.15.114:0/1891843630 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2e40020040 con 0x55b92d269e10 2024-09-13T18:23:08.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.570+0000 7f2e36ffd640 1 -- 172.21.15.114:0/1891843630 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e400048c0 con 0x55b92d269e10 2024-09-13T18:23:08.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.570+0000 7f2e595e9740 1 -- 172.21.15.114:0/1891843630 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b92d269e10 msgr2=0x55b92d2726e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:08.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.570+0000 7f2e595e9740 1 --2- 172.21.15.114:0/1891843630 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b92d269e10 0x55b92d2726e0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f2e40010ca0 tx=0x7f2e40005ed0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.570+0000 7f2e595e9740 1 -- 172.21.15.114:0/1891843630 shutdown_connections 2024-09-13T18:23:08.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.570+0000 7f2e595e9740 1 --2- 172.21.15.114:0/1891843630 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b92d274fd0 0x55b92d26ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.570+0000 7f2e595e9740 1 --2- 172.21.15.114:0/1891843630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b92d26e930 0x55b92d261310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.570+0000 7f2e595e9740 1 --2- 172.21.15.114:0/1891843630 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b92d269e10 0x55b92d2726e0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.570+0000 7f2e595e9740 1 -- 172.21.15.114:0/1891843630 >> 172.21.15.114:0/1891843630 conn(0x55b92d266380 msgr2=0x55b92d2783f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:08.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.570+0000 7f2e595e9740 1 -- 172.21.15.114:0/1891843630 shutdown_connections 2024-09-13T18:23:08.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.570+0000 7f2e595e9740 1 -- 172.21.15.114:0/1891843630 wait complete. 2024-09-13T18:23:08.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.571+0000 7f2e595e9740 1 -- start start 2024-09-13T18:23:08.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.571+0000 7f2e595e9740 1 Processor -- start 2024-09-13T18:23:08.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.571+0000 7f2e595e9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b92d269e10 0x55b92d281870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:08.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.571+0000 7f2e595e9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b92d26e930 0x55b92d27cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:08.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.571+0000 7f2e595e9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b92d274fd0 0x55b92d27d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:08.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.571+0000 7f2e595e9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b92d27d870 con 0x55b92d26e930 2024-09-13T18:23:08.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.571+0000 7f2e595e9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b92d27d9e0 con 0x55b92d274fd0 2024-09-13T18:23:08.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.571+0000 7f2e595e9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b92d2da780 con 0x55b92d269e10 2024-09-13T18:23:08.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.571+0000 7f2e595e9740 10 client.0 _ll_register_callbacks cb 0x7f2e44009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:23:08.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.571+0000 7f2e4f7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b92d26e930 0x55b92d27cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:08.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.571+0000 7f2e4f7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b92d26e930 0x55b92d27cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36958/0 (socket says 172.21.15.114:36958) 2024-09-13T18:23:08.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.571+0000 7f2e4f7fe640 1 -- 172.21.15.114:0/118410748 learned_addr learned my addr 172.21.15.114:0/118410748 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:08.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.571+0000 7f2e4ffff640 1 --2- 172.21.15.114:0/118410748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b92d269e10 0x55b92d281870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:08.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.571+0000 7f2e54e66640 1 --2- 172.21.15.114:0/118410748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b92d274fd0 0x55b92d27d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:08.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.572+0000 7f2e4f7fe640 1 -- 172.21.15.114:0/118410748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b92d269e10 msgr2=0x55b92d281870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:08.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.572+0000 7f2e4f7fe640 1 --2- 172.21.15.114:0/118410748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b92d269e10 0x55b92d281870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.572+0000 7f2e4f7fe640 1 -- 172.21.15.114:0/118410748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b92d274fd0 msgr2=0x55b92d27d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:08.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.572+0000 7f2e4f7fe640 1 --2- 172.21.15.114:0/118410748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b92d274fd0 0x55b92d27d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.572+0000 7f2e4f7fe640 1 -- 172.21.15.114:0/118410748 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b92d188840 con 0x55b92d26e930 2024-09-13T18:23:08.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.572+0000 7f2e54e66640 1 --2- 172.21.15.114:0/118410748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b92d274fd0 0x55b92d27d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:23:08.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.572+0000 7f2e4f7fe640 1 --2- 172.21.15.114:0/118410748 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b92d26e930 0x55b92d27cc30 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f2e440b7840 tx=0x7f2e440c2ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fda9b735abd36df1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:08.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.572+0000 7f2e2f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:23:08.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.572+0000 7f2e2f7fe640 1 -- 172.21.15.114:0/118410748 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e440d91b0 con 0x55b92d26e930 2024-09-13T18:23:08.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.572+0000 7f2e2f7fe640 1 -- 172.21.15.114:0/118410748 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2e440badd0 con 0x55b92d26e930 2024-09-13T18:23:08.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.572+0000 7f2e595e9740 1 -- 172.21.15.114:0/118410748 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2e40015070 con 0x55b92d26e930 2024-09-13T18:23:08.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.572+0000 7f2e2f7fe640 1 -- 172.21.15.114:0/118410748 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e440baba0 con 0x55b92d26e930 2024-09-13T18:23:08.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.572+0000 7f2e595e9740 10 client.16008 fetch_fsmap learned FSMap version 30 2024-09-13T18:23:08.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.572+0000 7f2e595e9740 1 -- 172.21.15.114:0/118410748 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55b92d193ea0 con 0x55b92d26e930 2024-09-13T18:23:08.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.572+0000 7f2e2f7fe640 1 -- 172.21.15.114:0/118410748 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2e40015070 con 0x55b92d26e930 2024-09-13T18:23:08.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.573+0000 7f2e2f7fe640 1 -- 172.21.15.114:0/118410748 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2e4404d070 con 0x55b92d26e930 2024-09-13T18:23:08.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.573+0000 7f2e595e9740 10 client.16008 fetch_fsmap finished waiting for FSMap version 30 2024-09-13T18:23:08.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.573+0000 7f2e595e9740 10 client.16008 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:23:08.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.573+0000 7f2e595e9740 20 client.16008 populate_metadata read hostname 'smithi114' 2024-09-13T18:23:08.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.573+0000 7f2e595e9740 1 --2- 172.21.15.114:0/118410748 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55b92d2db980 0x55b92d2ddd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:08.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.573+0000 7f2e595e9740 4 client.16008 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T18:23:08.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.573+0000 7f2e595e9740 1 -- 172.21.15.114:0/118410748 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b92ced27e0 con 0x55b92d2db980 2024-09-13T18:23:08.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.573+0000 7f2e4ffff640 1 --2- 172.21.15.114:0/118410748 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55b92d2db980 0x55b92d2ddd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:08.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.573+0000 7f2e2f7fe640 10 client.16008 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:23:08.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.573+0000 7f2e4ffff640 1 --2- 172.21.15.114:0/118410748 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55b92d2db980 0x55b92d2ddd70 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a75272f10ea9457c server_cookie=e624528c1cc13eac in_seq=0 out_seq=0 2024-09-13T18:23:08.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.574+0000 7f2e2f7fe640 1 -- 172.21.15.114:0/118410748 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55b92ced27e0 con 0x55b92d2db980 2024-09-13T18:23:08.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.574+0000 7f2e2f7fe640 10 client.16008 handle_command_reply: tid=0 2024-09-13T18:23:08.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.574+0000 7f2e595e9740 1 client.16008 shutdown 2024-09-13T18:23:08.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.574+0000 7f2e595e9740 1 -- 172.21.15.114:0/118410748 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b92d26e930 msgr2=0x55b92d27cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:08.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.574+0000 7f2e595e9740 1 --2- 172.21.15.114:0/118410748 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b92d26e930 0x55b92d27cc30 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f2e440b7840 tx=0x7f2e440c2ff0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.574+0000 7f2e595e9740 1 -- 172.21.15.114:0/118410748 shutdown_connections 2024-09-13T18:23:08.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.574+0000 7f2e595e9740 1 --2- 172.21.15.114:0/118410748 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55b92d2db980 0x55b92d2ddd70 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.574+0000 7f2e595e9740 1 --2- 172.21.15.114:0/118410748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b92d274fd0 0x55b92d27d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.574+0000 7f2e2f7fe640 0 client.16008 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:23:08.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.574+0000 7f2e595e9740 1 --2- 172.21.15.114:0/118410748 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b92d26e930 0x55b92d27cc30 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.574+0000 7f2e595e9740 1 --2- 172.21.15.114:0/118410748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b92d269e10 0x55b92d281870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.574+0000 7f2e595e9740 1 -- 172.21.15.114:0/118410748 >> 172.21.15.114:0/118410748 conn(0x55b92d266380 msgr2=0x55b92d26c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:08.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.574+0000 7f2e595e9740 1 -- 172.21.15.114:0/118410748 shutdown_connections 2024-09-13T18:23:08.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.574+0000 7f2e595e9740 1 -- 172.21.15.114:0/118410748 wait complete. 2024-09-13T18:23:08.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.574+0000 7f2e595e9740 20 client.16008 trim_cache size 0 max 16384 2024-09-13T18:23:08.582 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:23:08.582 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-13T18:23:08.582 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": { 2024-09-13T18:23:08.582 INFO:teuthology.orchestra.run.smithi114.stdout: "e98642d0-00e3-423e-8ff3-881b3d1500ce": { 2024-09-13T18:23:08.582 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-13T18:23:08.582 INFO:teuthology.orchestra.run.smithi114.stdout: "tag": "e98642d0-00e3-423e-8ff3-881b3d1500ce", 2024-09-13T18:23:08.582 INFO:teuthology.orchestra.run.smithi114.stdout: "options": "recursive,force" 2024-09-13T18:23:08.582 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:23:08.583 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:23:08.583 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:23:08.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.575+0000 7f2e568c9640 1 -- 172.21.15.114:0/2803430486 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2e380a1320 msgr2=0x7f2e380a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:08.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.575+0000 7f2e568c9640 1 --2- 172.21.15.114:0/2803430486 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2e380a1320 0x7f2e380a37e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2e48005fd0 tx=0x7f2e48009500 comp rx=0 tx=0).stop 2024-09-13T18:23:08.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.575+0000 7f2e568c9640 1 -- 172.21.15.114:0/2803430486 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e50103fd0 msgr2=0x7f2e50196820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:08.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.575+0000 7f2e568c9640 1 --2- 172.21.15.114:0/2803430486 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e50103fd0 0x7f2e50196820 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f2e44005ab0 tx=0x7f2e44032c20 comp rx=0 tx=0).stop 2024-09-13T18:23:08.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.575+0000 7f2e568c9640 1 -- 172.21.15.114:0/2803430486 shutdown_connections 2024-09-13T18:23:08.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.575+0000 7f2e568c9640 1 --2- 172.21.15.114:0/2803430486 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2e380a1320 0x7f2e380a37e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.575+0000 7f2e568c9640 1 --2- 172.21.15.114:0/2803430486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e5019ac10 0x7f2e5019b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.575+0000 7f2e568c9640 1 --2- 172.21.15.114:0/2803430486 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e50103fd0 0x7f2e50196820 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.575+0000 7f2e568c9640 1 --2- 172.21.15.114:0/2803430486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e500fee80 0x7f2e501962e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:08.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.575+0000 7f2e568c9640 1 -- 172.21.15.114:0/2803430486 >> 172.21.15.114:0/2803430486 conn(0x7f2e500fa8d0 msgr2=0x7f2e500683e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:08.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.575+0000 7f2e568c9640 1 -- 172.21.15.114:0/2803430486 shutdown_connections 2024-09-13T18:23:08.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:08.575+0000 7f2e568c9640 1 -- 172.21.15.114:0/2803430486 wait complete. 2024-09-13T18:23:08.590 INFO:tasks.cephfs.filesystem:scrub status for tag:e98642d0-00e3-423e-8ff3-881b3d1500ce - {'path': '/', 'tag': 'e98642d0-00e3-423e-8ff3-881b3d1500ce', 'options': 'recursive,force'} 2024-09-13T18:23:08.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:23:08 smithi134 ceph-mon[34283]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-13T18:23:09.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:23:09 smithi134 ceph-mon[34283]: scrub queued for path: / 2024-09-13T18:23:09.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:23:09 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:23:09.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:23:09 smithi134 ceph-mon[34283]: scrub summary: active paths [/] 2024-09-13T18:23:09.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:23:09 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:23:09.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:23:09 smithi204 ceph-mon[34777]: scrub queued for path: / 2024-09-13T18:23:09.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:23:09 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:23:09.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:23:09 smithi204 ceph-mon[34777]: scrub summary: active paths [/] 2024-09-13T18:23:09.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:23:09 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:23:09.719 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:23:09 smithi114 ceph-mon[30897]: scrub queued for path: / 2024-09-13T18:23:09.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:23:09 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:23:09.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:23:09 smithi114 ceph-mon[30897]: scrub summary: active paths [/] 2024-09-13T18:23:09.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:23:09 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:23:10.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:23:10 smithi134 ceph-mon[34283]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-09-13T18:23:10.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:23:10 smithi204 ceph-mon[34777]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-09-13T18:23:10.719 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:23:10 smithi114 ceph-mon[30897]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-09-13T18:23:11.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:23:11 smithi134 ceph-mon[34283]: scrub summary: idle 2024-09-13T18:23:11.656 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:23:11 smithi114 ceph-mon[30897]: scrub summary: idle 2024-09-13T18:23:11.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:23:11 smithi204 ceph-mon[34777]: scrub summary: idle 2024-09-13T18:23:12.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:23:12 smithi134 ceph-mon[34283]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-09-13T18:23:12.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:23:12 smithi204 ceph-mon[34777]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-09-13T18:23:12.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:23:12 smithi114 ceph-mon[30897]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-09-13T18:23:14.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:23:14 smithi204 ceph-mon[34777]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-09-13T18:23:14.469 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:23:14 smithi114 ceph-mon[30897]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-09-13T18:23:14.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:23:14 smithi134 ceph-mon[34283]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-09-13T18:23:16.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:23:16 smithi204 ceph-mon[34777]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-09-13T18:23:16.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:23:16 smithi114 ceph-mon[30897]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-09-13T18:23:16.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:23:16 smithi134 ceph-mon[34283]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-09-13T18:23:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.422+0000 7fa045318640 1 -- 172.21.15.114:0/2598748252 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa0400fe7a0 msgr2=0x7fa0400febe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.422+0000 7fa045318640 1 --2- 172.21.15.114:0/2598748252 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa0400fe7a0 0x7fa0400febe0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fa028009900 tx=0x7fa02802a410 comp rx=0 tx=0).stop 2024-09-13T18:23:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.422+0000 7fa045318640 1 -- 172.21.15.114:0/2598748252 shutdown_connections 2024-09-13T18:23:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.422+0000 7fa045318640 1 --2- 172.21.15.114:0/2598748252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa0400ff120 0x7fa04010b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.422+0000 7fa045318640 1 --2- 172.21.15.114:0/2598748252 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa0400fe7a0 0x7fa0400febe0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.422+0000 7fa045318640 1 --2- 172.21.15.114:0/2598748252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0401043e0 0x7fa0401047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.422+0000 7fa045318640 1 -- 172.21.15.114:0/2598748252 >> 172.21.15.114:0/2598748252 conn(0x7fa040075380 msgr2=0x7fa040075790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.422+0000 7fa045318640 1 -- 172.21.15.114:0/2598748252 shutdown_connections 2024-09-13T18:23:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.422+0000 7fa045318640 1 -- 172.21.15.114:0/2598748252 wait complete. 2024-09-13T18:23:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.423+0000 7fa045318640 1 Processor -- start 2024-09-13T18:23:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.423+0000 7fa045318640 1 -- start start 2024-09-13T18:23:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.424+0000 7fa045318640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa0400fe7a0 0x7fa040196160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.424+0000 7fa045318640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa0400ff120 0x7fa0401966a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.424+0000 7fa045318640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0401043e0 0x7fa04019aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.424+0000 7fa045318640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa04019b090 con 0x7fa0400ff120 2024-09-13T18:23:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.424+0000 7fa045318640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa04019b200 con 0x7fa0401043e0 2024-09-13T18:23:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.424+0000 7fa045318640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa04019b3c0 con 0x7fa0400fe7a0 2024-09-13T18:23:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.424+0000 7fa037fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa0400ff120 0x7fa0401966a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.424+0000 7fa037fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa0400ff120 0x7fa0401966a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33522/0 (socket says 172.21.15.114:33522) 2024-09-13T18:23:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.424+0000 7fa037fff640 1 -- 172.21.15.114:0/3747420725 learned_addr learned my addr 172.21.15.114:0/3747420725 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.424+0000 7fa03effd640 1 --2- 172.21.15.114:0/3747420725 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa0400fe7a0 0x7fa040196160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.424+0000 7fa03f7fe640 1 --2- 172.21.15.114:0/3747420725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0401043e0 0x7fa04019aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.425+0000 7fa037fff640 1 -- 172.21.15.114:0/3747420725 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa0400fe7a0 msgr2=0x7fa040196160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.425+0000 7fa037fff640 1 --2- 172.21.15.114:0/3747420725 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa0400fe7a0 0x7fa040196160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.425+0000 7fa037fff640 1 -- 172.21.15.114:0/3747420725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0401043e0 msgr2=0x7fa04019aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.425+0000 7fa037fff640 1 --2- 172.21.15.114:0/3747420725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0401043e0 0x7fa04019aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.425+0000 7fa037fff640 1 -- 172.21.15.114:0/3747420725 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa028009560 con 0x7fa0400ff120 2024-09-13T18:23:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.425+0000 7fa037fff640 1 --2- 172.21.15.114:0/3747420725 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa0400ff120 0x7fa0401966a0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fa028006010 tx=0x7fa028004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.425+0000 7fa03cff9640 1 -- 172.21.15.114:0/3747420725 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa028004d70 con 0x7fa0400ff120 2024-09-13T18:23:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.425+0000 7fa03cff9640 1 -- 172.21.15.114:0/3747420725 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa028003a80 con 0x7fa0400ff120 2024-09-13T18:23:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.425+0000 7fa03cff9640 1 -- 172.21.15.114:0/3747420725 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02803cad0 con 0x7fa0400ff120 2024-09-13T18:23:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.425+0000 7fa045318640 1 -- 172.21.15.114:0/3747420725 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa04019b640 con 0x7fa0400ff120 2024-09-13T18:23:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.425+0000 7fa045318640 1 -- 172.21.15.114:0/3747420725 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa04010c840 con 0x7fa0400ff120 2024-09-13T18:23:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.429+0000 7fa03cff9640 1 -- 172.21.15.114:0/3747420725 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) v1 ==== 136107+0+0 (secure 0 0 0) 0x7fa028003bf0 con 0x7fa0400ff120 2024-09-13T18:23:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.429+0000 7fa04803b740 1 Processor -- start 2024-09-13T18:23:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.429+0000 7fa04803b740 1 -- start start 2024-09-13T18:23:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.429+0000 7fa04803b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556de0bcf330 0x556de0bcf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.429+0000 7fa04803b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556de0bcfc50 0x556de0bd9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.429+0000 7fa04803b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556de0bda4f0 0x556de0bdc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.429+0000 7fa04803b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x556de0b3ac30 con 0x556de0bcfc50 2024-09-13T18:23:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.429+0000 7fa04803b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556de0b383c0 con 0x556de0bcf330 2024-09-13T18:23:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.429+0000 7fa04803b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556de0b33cb0 con 0x556de0bda4f0 2024-09-13T18:23:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.430+0000 7fa03f7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556de0bcf330 0x556de0bcf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.430+0000 7fa03effd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556de0bcfc50 0x556de0bd9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.430+0000 7fa03f7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556de0bcf330 0x556de0bcf710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:50646/0 (socket says 172.21.15.114:50646) 2024-09-13T18:23:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.430+0000 7fa03f7fe640 1 -- 172.21.15.114:0/1834310598 learned_addr learned my addr 172.21.15.114:0/1834310598 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.430+0000 7fa037fff640 1 --2- 172.21.15.114:0/1834310598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556de0bda4f0 0x556de0bdc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.430+0000 7fa03f7fe640 1 -- 172.21.15.114:0/1834310598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556de0bda4f0 msgr2=0x556de0bdc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.430+0000 7fa03f7fe640 1 --2- 172.21.15.114:0/1834310598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556de0bda4f0 0x556de0bdc8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.430+0000 7fa03f7fe640 1 -- 172.21.15.114:0/1834310598 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556de0bcfc50 msgr2=0x556de0bd9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.430+0000 7fa03f7fe640 1 --2- 172.21.15.114:0/1834310598 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556de0bcfc50 0x556de0bd9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.430+0000 7fa03f7fe640 1 -- 172.21.15.114:0/1834310598 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556de0ada910 con 0x556de0bcf330 2024-09-13T18:23:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.430+0000 7fa03cff9640 1 --2- 172.21.15.114:0/3747420725 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa02c0a57b0 0x7fa02c0a7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.430+0000 7fa03effd640 1 --2- 172.21.15.114:0/1834310598 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556de0bcfc50 0x556de0bd9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:23:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.430+0000 7fa03cff9640 1 -- 172.21.15.114:0/3747420725 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fa0280ee160 con 0x7fa0400ff120 2024-09-13T18:23:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.431+0000 7fa03effd640 1 --2- 172.21.15.114:0/3747420725 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa02c0a57b0 0x7fa02c0a7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.431+0000 7fa03f7fe640 1 --2- 172.21.15.114:0/1834310598 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556de0bcf330 0x556de0bcf710 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa03800a430 tx=0x7fa0380095a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31b3ab154e7cf00 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.431+0000 7fa0357fa640 1 -- 172.21.15.114:0/1834310598 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa038010040 con 0x556de0bcf330 2024-09-13T18:23:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.431+0000 7fa0357fa640 1 -- 172.21.15.114:0/1834310598 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa038013c20 con 0x556de0bcf330 2024-09-13T18:23:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.431+0000 7fa0357fa640 1 -- 172.21.15.114:0/1834310598 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa038014c00 con 0x556de0bcf330 2024-09-13T18:23:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.432+0000 7fa04803b740 1 -- 172.21.15.114:0/1834310598 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556de0bcf330 msgr2=0x556de0bcf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.432+0000 7fa03effd640 1 --2- 172.21.15.114:0/3747420725 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa02c0a57b0 0x7fa02c0a7c70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x556de0bcb3c0 tx=0x7fa03000a2d0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.432+0000 7fa04803b740 1 --2- 172.21.15.114:0/1834310598 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556de0bcf330 0x556de0bcf710 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa03800a430 tx=0x7fa0380095a0 comp rx=0 tx=0).stop 2024-09-13T18:23:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.432+0000 7fa04803b740 1 -- 172.21.15.114:0/1834310598 shutdown_connections 2024-09-13T18:23:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.432+0000 7fa04803b740 1 --2- 172.21.15.114:0/1834310598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556de0bda4f0 0x556de0bdc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.432+0000 7fa04803b740 1 --2- 172.21.15.114:0/1834310598 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556de0bcfc50 0x556de0bd9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.432+0000 7fa04803b740 1 --2- 172.21.15.114:0/1834310598 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556de0bcf330 0x556de0bcf710 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.432+0000 7fa04803b740 1 -- 172.21.15.114:0/1834310598 >> 172.21.15.114:0/1834310598 conn(0x556de0bcb8c0 msgr2=0x556de0bcbca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.432+0000 7fa04803b740 1 -- 172.21.15.114:0/1834310598 shutdown_connections 2024-09-13T18:23:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.432+0000 7fa04803b740 1 -- 172.21.15.114:0/1834310598 wait complete. 2024-09-13T18:23:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.433+0000 7fa04803b740 1 -- start start 2024-09-13T18:23:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.433+0000 7fa04803b740 1 Processor -- start 2024-09-13T18:23:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.433+0000 7fa04803b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556de0bcf330 0x556de0bd9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.433+0000 7fa04803b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556de0bcfc50 0x556de0bd70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.433+0000 7fa03effd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556de0bcf330 0x556de0bd9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.433+0000 7fa03effd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556de0bcf330 0x556de0bd9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33528/0 (socket says 172.21.15.114:33528) 2024-09-13T18:23:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.433+0000 7fa03effd640 1 -- 172.21.15.114:0/123306288 learned_addr learned my addr 172.21.15.114:0/123306288 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.433+0000 7fa04803b740 1 --2- 172.21.15.114:0/123306288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556de0bda4f0 0x556de0bd7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.434+0000 7fa04803b740 1 -- 172.21.15.114:0/123306288 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x556de0af1e80 con 0x556de0bcf330 2024-09-13T18:23:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.434+0000 7fa04803b740 1 -- 172.21.15.114:0/123306288 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556de0af1b20 con 0x556de0bcfc50 2024-09-13T18:23:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.434+0000 7fa04803b740 1 -- 172.21.15.114:0/123306288 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556de0ab7260 con 0x556de0bda4f0 2024-09-13T18:23:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.434+0000 7fa037fff640 1 --2- 172.21.15.114:0/123306288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556de0bcfc50 0x556de0bd70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.434+0000 7fa03effd640 1 -- 172.21.15.114:0/123306288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556de0bda4f0 msgr2=0x556de0bd7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.434+0000 7fa03effd640 1 --2- 172.21.15.114:0/123306288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556de0bda4f0 0x556de0bd7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.434+0000 7fa03effd640 1 -- 172.21.15.114:0/123306288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556de0bcfc50 msgr2=0x556de0bd70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.434+0000 7fa03effd640 1 --2- 172.21.15.114:0/123306288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556de0bcfc50 0x556de0bd70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.434+0000 7fa03effd640 1 -- 172.21.15.114:0/123306288 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa038009c50 con 0x556de0bcf330 2024-09-13T18:23:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.434+0000 7fa03effd640 1 --2- 172.21.15.114:0/123306288 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556de0bcf330 0x556de0bd9e10 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fa030005940 tx=0x7fa030004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=916ec20fb75467b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.434+0000 7fa0267fc640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:23:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.434+0000 7fa04803b740 10 client.0 _ll_register_callbacks cb 0x556de0aecdf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:23:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.434+0000 7fa0267fc640 1 -- 172.21.15.114:0/123306288 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa030021070 con 0x556de0bcf330 2024-09-13T18:23:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.434+0000 7fa0267fc640 1 -- 172.21.15.114:0/123306288 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fa030027420 con 0x556de0bcf330 2024-09-13T18:23:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.435+0000 7fa04803b740 1 -- 172.21.15.114:0/123306288 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556de0b5c680 con 0x556de0bcf330 2024-09-13T18:23:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.435+0000 7fa0267fc640 1 -- 172.21.15.114:0/123306288 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa030015070 con 0x556de0bcf330 2024-09-13T18:23:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.435+0000 7fa04803b740 10 client.16020 fetch_fsmap learned FSMap version 30 2024-09-13T18:23:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.435+0000 7fa04803b740 1 -- 172.21.15.114:0/123306288 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({fsmap=30}) v3 -- 0x556de0b32330 con 0x556de0bcf330 2024-09-13T18:23:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.435+0000 7fa0267fc640 1 -- 172.21.15.114:0/123306288 <== mon.0 v2:172.21.15.114:3300/0 4 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa0300275e0 con 0x556de0bcf330 2024-09-13T18:23:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.435+0000 7fa04803b740 10 client.16020 fetch_fsmap finished waiting for FSMap version 30 2024-09-13T18:23:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.435+0000 7fa04803b740 10 client.16020 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:23:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.436+0000 7fa04803b740 20 client.16020 populate_metadata read hostname 'smithi114' 2024-09-13T18:23:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.436+0000 7fa04803b740 1 --2- 172.21.15.114:0/123306288 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x556de0be9b80 0x556de0bec040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.436+0000 7fa04803b740 4 client.16020 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:23:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.436+0000 7fa04803b740 1 -- 172.21.15.114:0/123306288 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556de0b5c680 con 0x556de0be9b80 2024-09-13T18:23:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.436+0000 7fa037fff640 1 --2- 172.21.15.114:0/123306288 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x556de0be9b80 0x556de0bec040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.436+0000 7fa0267fc640 10 client.16020 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:23:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.436+0000 7fa037fff640 1 --2- 172.21.15.114:0/123306288 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x556de0be9b80 0x556de0bec040 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e78bca6f771545d server_cookie=802bbbf4edd4c130 in_seq=0 out_seq=0 2024-09-13T18:23:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.437+0000 7fa0267fc640 1 -- 172.21.15.114:0/123306288 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x556de0b5c680 con 0x556de0be9b80 2024-09-13T18:23:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.437+0000 7fa0267fc640 10 client.16020 handle_command_reply: tid=0 2024-09-13T18:23:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.437+0000 7fa04803b740 1 client.16020 shutdown 2024-09-13T18:23:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.437+0000 7fa04803b740 1 -- 172.21.15.114:0/123306288 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556de0bcf330 msgr2=0x556de0bd9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.437+0000 7fa04803b740 1 --2- 172.21.15.114:0/123306288 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556de0bcf330 0x556de0bd9e10 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fa030005940 tx=0x7fa030004730 comp rx=0 tx=0).stop 2024-09-13T18:23:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.437+0000 7fa04803b740 1 -- 172.21.15.114:0/123306288 shutdown_connections 2024-09-13T18:23:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.437+0000 7fa04803b740 1 --2- 172.21.15.114:0/123306288 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x556de0be9b80 0x556de0bec040 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.437+0000 7fa04803b740 1 --2- 172.21.15.114:0/123306288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556de0bda4f0 0x556de0bd7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.437+0000 7fa04803b740 1 --2- 172.21.15.114:0/123306288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556de0bcfc50 0x556de0bd70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.437+0000 7fa04803b740 1 --2- 172.21.15.114:0/123306288 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556de0bcf330 0x556de0bd9e10 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.437+0000 7fa04803b740 1 -- 172.21.15.114:0/123306288 >> 172.21.15.114:0/123306288 conn(0x556de0bcb8c0 msgr2=0x556de0bcab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.437+0000 7fa0267fc640 0 client.16020 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:23:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.437+0000 7fa04803b740 1 -- 172.21.15.114:0/123306288 shutdown_connections 2024-09-13T18:23:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.437+0000 7fa04803b740 1 -- 172.21.15.114:0/123306288 wait complete. 2024-09-13T18:23:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.438+0000 7fa04803b740 20 client.16020 trim_cache size 0 max 16384 2024-09-13T18:23:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.462+0000 7fa04803b740 1 Processor -- start 2024-09-13T18:23:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.462+0000 7fa04803b740 1 -- start start 2024-09-13T18:23:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.462+0000 7fa04803b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556de0bcf330 0x556de0bd9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.462+0000 7fa04803b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556de0bcfc50 0x556de0be11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.462+0000 7fa04803b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556de0bda4f0 0x556de0be1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.462+0000 7fa03f7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556de0bda4f0 0x556de0be1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.462+0000 7fa03f7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556de0bda4f0 0x556de0be1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33530/0 (socket says 172.21.15.114:33530) 2024-09-13T18:23:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.462+0000 7fa03f7fe640 1 -- 172.21.15.114:0/3543443315 learned_addr learned my addr 172.21.15.114:0/3543443315 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.462+0000 7fa03effd640 1 --2- 172.21.15.114:0/3543443315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556de0bcf330 0x556de0bd9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.462+0000 7fa04803b740 1 -- 172.21.15.114:0/3543443315 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x556de0b5c680 con 0x556de0bda4f0 2024-09-13T18:23:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.462+0000 7fa04803b740 1 -- 172.21.15.114:0/3543443315 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556de0be6b40 con 0x556de0bcfc50 2024-09-13T18:23:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.462+0000 7fa04803b740 1 -- 172.21.15.114:0/3543443315 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556de0bd02a0 con 0x556de0bcf330 2024-09-13T18:23:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.462+0000 7fa037fff640 1 --2- 172.21.15.114:0/3543443315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556de0bcfc50 0x556de0be11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.463+0000 7fa037fff640 1 -- 172.21.15.114:0/3543443315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556de0bcf330 msgr2=0x556de0bd9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.463+0000 7fa037fff640 1 --2- 172.21.15.114:0/3543443315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556de0bcf330 0x556de0bd9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.463+0000 7fa037fff640 1 -- 172.21.15.114:0/3543443315 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556de0bda4f0 msgr2=0x556de0be1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.463+0000 7fa037fff640 1 --2- 172.21.15.114:0/3543443315 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556de0bda4f0 0x556de0be1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.463+0000 7fa037fff640 1 -- 172.21.15.114:0/3543443315 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556de0ad4300 con 0x556de0bcfc50 2024-09-13T18:23:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.463+0000 7fa03effd640 1 --2- 172.21.15.114:0/3543443315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556de0bcf330 0x556de0bd9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:23:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.463+0000 7fa03f7fe640 1 --2- 172.21.15.114:0/3543443315 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556de0bda4f0 0x556de0be1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:23:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.463+0000 7fa037fff640 1 --2- 172.21.15.114:0/3543443315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556de0bcfc50 0x556de0be11c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa0280eb3e0 tx=0x7fa0280f0630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=581feb134e864f21 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.463+0000 7fa035ffb640 1 -- 172.21.15.114:0/3543443315 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0280ba700 con 0x556de0bcfc50 2024-09-13T18:23:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.463+0000 7fa035ffb640 1 -- 172.21.15.114:0/3543443315 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa0280bb4a0 con 0x556de0bcfc50 2024-09-13T18:23:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.463+0000 7fa035ffb640 1 -- 172.21.15.114:0/3543443315 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0280b74b0 con 0x556de0bcfc50 2024-09-13T18:23:17.470 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.464+0000 7fa04803b740 1 -- 172.21.15.114:0/3543443315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556de0bcfc50 msgr2=0x556de0be11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.464+0000 7fa04803b740 1 --2- 172.21.15.114:0/3543443315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556de0bcfc50 0x556de0be11c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa0280eb3e0 tx=0x7fa0280f0630 comp rx=0 tx=0).stop 2024-09-13T18:23:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.465+0000 7fa04803b740 1 -- 172.21.15.114:0/3543443315 shutdown_connections 2024-09-13T18:23:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.465+0000 7fa04803b740 1 --2- 172.21.15.114:0/3543443315 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556de0bda4f0 0x556de0be1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.465+0000 7fa04803b740 1 --2- 172.21.15.114:0/3543443315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556de0bcfc50 0x556de0be11c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.465+0000 7fa04803b740 1 --2- 172.21.15.114:0/3543443315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556de0bcf330 0x556de0bd9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.465+0000 7fa04803b740 1 -- 172.21.15.114:0/3543443315 >> 172.21.15.114:0/3543443315 conn(0x556de0bcb8c0 msgr2=0x556de0bd23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.465+0000 7fa04803b740 1 -- 172.21.15.114:0/3543443315 shutdown_connections 2024-09-13T18:23:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.465+0000 7fa04803b740 1 -- 172.21.15.114:0/3543443315 wait complete. 2024-09-13T18:23:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.465+0000 7fa04803b740 1 -- start start 2024-09-13T18:23:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.466+0000 7fa04803b740 1 Processor -- start 2024-09-13T18:23:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.466+0000 7fa04803b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556de0bcf330 0x556de0bdfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.466+0000 7fa04803b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556de0bcfc50 0x556de0be01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:17.472 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.466+0000 7fa04803b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556de0bda4f0 0x556de0be0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:17.472 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.466+0000 7fa04803b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x556de0c40130 con 0x556de0bcfc50 2024-09-13T18:23:17.472 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.466+0000 7fa037fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556de0bcfc50 0x556de0be01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:17.472 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.467+0000 7fa03effd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556de0bcf330 0x556de0bdfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:17.472 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.467+0000 7fa04803b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556de0c40270 con 0x556de0bcf330 2024-09-13T18:23:17.472 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.467+0000 7fa04803b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556de0c403e0 con 0x556de0bda4f0 2024-09-13T18:23:17.472 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.467+0000 7fa037fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556de0bcfc50 0x556de0be01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33544/0 (socket says 172.21.15.114:33544) 2024-09-13T18:23:17.472 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.467+0000 7fa037fff640 1 -- 172.21.15.114:0/2176933231 learned_addr learned my addr 172.21.15.114:0/2176933231 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:17.472 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.467+0000 7fa03f7fe640 1 --2- 172.21.15.114:0/2176933231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556de0bda4f0 0x556de0be0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:17.472 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.467+0000 7fa04803b740 10 client.0 _ll_register_callbacks cb 0x7fa038009a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:23:17.472 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.467+0000 7fa03effd640 1 -- 172.21.15.114:0/2176933231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556de0bda4f0 msgr2=0x556de0be0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:17.472 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.467+0000 7fa03effd640 1 --2- 172.21.15.114:0/2176933231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556de0bda4f0 0x556de0be0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:17.472 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.467+0000 7fa03effd640 1 -- 172.21.15.114:0/2176933231 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556de0bcfc50 msgr2=0x556de0be01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:17.473 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.467+0000 7fa03effd640 1 --2- 172.21.15.114:0/2176933231 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556de0bcfc50 0x556de0be01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:17.473 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.467+0000 7fa03effd640 1 -- 172.21.15.114:0/2176933231 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556de0b32330 con 0x556de0bcf330 2024-09-13T18:23:17.473 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.467+0000 7fa04803b740 1 -- 172.21.15.114:0/2176933231 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa038013c20 con 0x556de0bcf330 2024-09-13T18:23:17.473 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.467+0000 7fa03effd640 1 --2- 172.21.15.114:0/2176933231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556de0bcf330 0x556de0bdfcb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa030010c90 tx=0x7fa030017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62720052cb1ffc92 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:17.473 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.467+0000 7fa0267fc640 10 client.25433 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:23:17.473 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.467+0000 7fa0267fc640 1 -- 172.21.15.114:0/2176933231 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa030015040 con 0x556de0bcf330 2024-09-13T18:23:17.473 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.467+0000 7fa0267fc640 1 -- 172.21.15.114:0/2176933231 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fa0300276f0 con 0x556de0bcf330 2024-09-13T18:23:17.473 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.468+0000 7fa0267fc640 1 -- 172.21.15.114:0/2176933231 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa03000c330 con 0x556de0bcf330 2024-09-13T18:23:17.473 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.468+0000 7fa04803b740 10 client.25433 fetch_fsmap learned FSMap version 30 2024-09-13T18:23:17.473 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.468+0000 7fa04803b740 1 -- 172.21.15.114:0/2176933231 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({fsmap=30}) v3 -- 0x556de0ada910 con 0x556de0bcf330 2024-09-13T18:23:17.474 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.468+0000 7fa0267fc640 1 -- 172.21.15.114:0/2176933231 <== mon.2 v2:172.21.15.134:3300/0 4 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa030041060 con 0x556de0bcf330 2024-09-13T18:23:17.475 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.468+0000 7fa04803b740 10 client.25433 fetch_fsmap finished waiting for FSMap version 30 2024-09-13T18:23:17.475 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.468+0000 7fa04803b740 10 client.25433 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:23:17.475 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.468+0000 7fa04803b740 20 client.25433 populate_metadata read hostname 'smithi114' 2024-09-13T18:23:17.475 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.468+0000 7fa04803b740 1 --2- 172.21.15.114:0/2176933231 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x556de0c41910 0x556de0c43d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:17.475 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.468+0000 7fa04803b740 4 client.25433 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T18:23:17.475 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.468+0000 7fa04803b740 1 -- 172.21.15.114:0/2176933231 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556de0b275d0 con 0x556de0c41910 2024-09-13T18:23:17.475 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.469+0000 7fa037fff640 1 --2- 172.21.15.114:0/2176933231 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x556de0c41910 0x556de0c43d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:17.475 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.469+0000 7fa037fff640 1 --2- 172.21.15.114:0/2176933231 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x556de0c41910 0x556de0c43d00 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdfda2bcd59b256b server_cookie=67dadcd30446c91 in_seq=0 out_seq=0 2024-09-13T18:23:17.475 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.469+0000 7fa0267fc640 10 client.25433 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:23:17.476 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.473+0000 7fa0267fc640 1 -- 172.21.15.114:0/2176933231 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556de0b275d0 con 0x556de0c41910 2024-09-13T18:23:17.476 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.473+0000 7fa0267fc640 10 client.25433 handle_command_reply: tid=0 2024-09-13T18:23:17.477 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.473+0000 7fa04803b740 1 client.25433 shutdown 2024-09-13T18:23:17.477 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.473+0000 7fa04803b740 1 -- 172.21.15.114:0/2176933231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556de0bcf330 msgr2=0x556de0bdfcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:17.477 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.473+0000 7fa04803b740 1 --2- 172.21.15.114:0/2176933231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556de0bcf330 0x556de0bdfcb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa030010c90 tx=0x7fa030017000 comp rx=0 tx=0).stop 2024-09-13T18:23:17.477 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.473+0000 7fa04803b740 1 -- 172.21.15.114:0/2176933231 shutdown_connections 2024-09-13T18:23:17.477 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.473+0000 7fa04803b740 1 --2- 172.21.15.114:0/2176933231 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x556de0c41910 0x556de0c43d00 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:17.477 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.473+0000 7fa04803b740 1 --2- 172.21.15.114:0/2176933231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556de0bda4f0 0x556de0be0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:17.477 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.473+0000 7fa04803b740 1 --2- 172.21.15.114:0/2176933231 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556de0bcfc50 0x556de0be01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:17.477 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.473+0000 7fa04803b740 1 --2- 172.21.15.114:0/2176933231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556de0bcf330 0x556de0bdfcb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:17.477 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.473+0000 7fa04803b740 1 -- 172.21.15.114:0/2176933231 >> 172.21.15.114:0/2176933231 conn(0x556de0bcb8c0 msgr2=0x556de0bd5950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:17.477 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.473+0000 7fa0267fc640 0 client.25433 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:23:17.477 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.473+0000 7fa04803b740 1 -- 172.21.15.114:0/2176933231 shutdown_connections 2024-09-13T18:23:17.477 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.473+0000 7fa04803b740 1 -- 172.21.15.114:0/2176933231 wait complete. 2024-09-13T18:23:17.478 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.473+0000 7fa04803b740 20 client.25433 trim_cache size 0 max 16384 2024-09-13T18:23:17.478 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-13T18:23:17.478 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-13T18:23:17.478 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-13T18:23:17.479 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-13T18:23:17.479 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.474+0000 7fa045318640 1 -- 172.21.15.114:0/3747420725 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa02c0a57b0 msgr2=0x7fa02c0a7c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:17.479 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.474+0000 7fa045318640 1 --2- 172.21.15.114:0/3747420725 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa02c0a57b0 0x7fa02c0a7c70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x556de0bcb3c0 tx=0x7fa03000a2d0 comp rx=0 tx=0).stop 2024-09-13T18:23:17.479 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.474+0000 7fa045318640 1 -- 172.21.15.114:0/3747420725 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa0400ff120 msgr2=0x7fa0401966a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:17.480 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.474+0000 7fa045318640 1 --2- 172.21.15.114:0/3747420725 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa0400ff120 0x7fa0401966a0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fa028006010 tx=0x7fa028004740 comp rx=0 tx=0).stop 2024-09-13T18:23:17.480 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.474+0000 7fa045318640 1 -- 172.21.15.114:0/3747420725 shutdown_connections 2024-09-13T18:23:17.480 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.474+0000 7fa045318640 1 --2- 172.21.15.114:0/3747420725 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa02c0a57b0 0x7fa02c0a7c70 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:17.480 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.474+0000 7fa045318640 1 --2- 172.21.15.114:0/3747420725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0401043e0 0x7fa04019aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:17.480 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.474+0000 7fa045318640 1 --2- 172.21.15.114:0/3747420725 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa0400ff120 0x7fa0401966a0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:17.480 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.474+0000 7fa045318640 1 --2- 172.21.15.114:0/3747420725 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa0400fe7a0 0x7fa040196160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:17.480 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.474+0000 7fa045318640 1 -- 172.21.15.114:0/3747420725 >> 172.21.15.114:0/3747420725 conn(0x7fa040075380 msgr2=0x7fa0400fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:17.480 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.474+0000 7fa045318640 1 -- 172.21.15.114:0/3747420725 shutdown_connections 2024-09-13T18:23:17.480 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:17.474+0000 7fa045318640 1 -- 172.21.15.114:0/3747420725 wait complete. 2024-09-13T18:23:18.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:23:18 smithi204 ceph-mon[34777]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-13T18:23:18.469 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:23:18 smithi114 ceph-mon[30897]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-13T18:23:18.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:23:18 smithi134 ceph-mon[34283]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-13T18:23:20.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:23:20 smithi204 ceph-mon[34777]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 767 B/s wr, 14 op/s 2024-09-13T18:23:20.469 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:23:20 smithi114 ceph-mon[30897]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 767 B/s wr, 14 op/s 2024-09-13T18:23:20.622 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:23:20 smithi134 ceph-mon[34283]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 767 B/s wr, 14 op/s 2024-09-13T18:23:22.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:23:22 smithi204 ceph-mon[34777]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-13T18:23:22.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:23:22 smithi114 ceph-mon[30897]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-13T18:23:22.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:23:22 smithi134 ceph-mon[34283]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-13T18:23:24.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:23:24 smithi204 ceph-mon[34777]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-13T18:23:24.469 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:23:24 smithi114 ceph-mon[30897]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-13T18:23:24.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:23:24 smithi134 ceph-mon[34283]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-13T18:23:26.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:23:26 smithi204 ceph-mon[34777]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-13T18:23:26.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:23:26 smithi114 ceph-mon[30897]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-13T18:23:26.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:23:26 smithi134 ceph-mon[34283]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-13T18:23:28.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:23:28 smithi204 ceph-mon[34777]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-13T18:23:28.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:23:28 smithi114 ceph-mon[30897]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-13T18:23:28.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:23:28 smithi134 ceph-mon[34283]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-13T18:23:30.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:23:30 smithi204 ceph-mon[34777]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-13T18:23:30.473 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:23:30 smithi114 ceph-mon[30897]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-13T18:23:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.589+0000 7f3b3cc96640 1 -- 172.21.15.114:0/3896170831 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3b380feff0 msgr2=0x7f3b380ff430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.589+0000 7f3b3cc96640 1 --2- 172.21.15.114:0/3896170831 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3b380feff0 0x7f3b380ff430 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c009930 tx=0x7f3b2c02a420 comp rx=0 tx=0).stop 2024-09-13T18:23:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.589+0000 7f3b3cc96640 1 -- 172.21.15.114:0/3896170831 shutdown_connections 2024-09-13T18:23:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.589+0000 7f3b3cc96640 1 --2- 172.21.15.114:0/3896170831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b380ff970 0x7f3b3810ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.589+0000 7f3b3cc96640 1 --2- 172.21.15.114:0/3896170831 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3b380feff0 0x7f3b380ff430 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.589+0000 7f3b3cc96640 1 --2- 172.21.15.114:0/3896170831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b38104c30 0x7f3b38105010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.589+0000 7f3b3cc96640 1 -- 172.21.15.114:0/3896170831 >> 172.21.15.114:0/3896170831 conn(0x7f3b38074940 msgr2=0x7f3b38074d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.589+0000 7f3b3cc96640 1 -- 172.21.15.114:0/3896170831 shutdown_connections 2024-09-13T18:23:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.589+0000 7f3b3cc96640 1 -- 172.21.15.114:0/3896170831 wait complete. 2024-09-13T18:23:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.590+0000 7f3b3cc96640 1 Processor -- start 2024-09-13T18:23:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.590+0000 7f3b3cc96640 1 -- start start 2024-09-13T18:23:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.590+0000 7f3b3cc96640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3b380feff0 0x7f3b381950e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.590+0000 7f3b3cc96640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b380ff970 0x7f3b38195620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.590+0000 7f3b3cc96640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b38104c30 0x7f3b38199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.590+0000 7f3b3cc96640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3819a010 con 0x7f3b380feff0 2024-09-13T18:23:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.590+0000 7f3b3cc96640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3819a180 con 0x7f3b38104c30 2024-09-13T18:23:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.590+0000 7f3b3cc96640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3819a340 con 0x7f3b380ff970 2024-09-13T18:23:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.590+0000 7f3b3659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3b380feff0 0x7f3b381950e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.590+0000 7f3b3659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3b380feff0 0x7f3b381950e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33100/0 (socket says 172.21.15.114:33100) 2024-09-13T18:23:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.590+0000 7f3b3659c640 1 -- 172.21.15.114:0/37326459 learned_addr learned my addr 172.21.15.114:0/37326459 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.590+0000 7f3b35d9b640 1 --2- 172.21.15.114:0/37326459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b380ff970 0x7f3b38195620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.590+0000 7f3b36d9d640 1 --2- 172.21.15.114:0/37326459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b38104c30 0x7f3b38199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.591+0000 7f3b3659c640 1 -- 172.21.15.114:0/37326459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b380ff970 msgr2=0x7f3b38195620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.591+0000 7f3b3659c640 1 --2- 172.21.15.114:0/37326459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b380ff970 0x7f3b38195620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.591+0000 7f3b3659c640 1 -- 172.21.15.114:0/37326459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b38104c30 msgr2=0x7f3b38199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.591+0000 7f3b3659c640 1 --2- 172.21.15.114:0/37326459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b38104c30 0x7f3b38199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.591+0000 7f3b3659c640 1 -- 172.21.15.114:0/37326459 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b2c009560 con 0x7f3b380feff0 2024-09-13T18:23:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.591+0000 7f3b36d9d640 1 --2- 172.21.15.114:0/37326459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b38104c30 0x7f3b38199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:23:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.591+0000 7f3b3659c640 1 --2- 172.21.15.114:0/37326459 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3b380feff0 0x7f3b381950e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f3b2800c890 tx=0x7f3b2800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.591+0000 7f3b277fe640 1 -- 172.21.15.114:0/37326459 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b28004220 con 0x7f3b380feff0 2024-09-13T18:23:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.591+0000 7f3b3cc96640 1 -- 172.21.15.114:0/37326459 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b3819a620 con 0x7f3b380feff0 2024-09-13T18:23:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.591+0000 7f3b277fe640 1 -- 172.21.15.114:0/37326459 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3b28007e40 con 0x7f3b380feff0 2024-09-13T18:23:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.591+0000 7f3b277fe640 1 -- 172.21.15.114:0/37326459 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b28004a00 con 0x7f3b380feff0 2024-09-13T18:23:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.591+0000 7f3b3cc96640 1 -- 172.21.15.114:0/37326459 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b381a1ea0 con 0x7f3b380feff0 2024-09-13T18:23:30.605 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.594+0000 7f3b277fe640 1 -- 172.21.15.114:0/37326459 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) v1 ==== 136107+0+0 (secure 0 0 0) 0x7f3b280079b0 con 0x7f3b380feff0 2024-09-13T18:23:30.605 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.594+0000 7f3b3f5fd740 1 Processor -- start 2024-09-13T18:23:30.605 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.595+0000 7f3b3f5fd740 1 -- start start 2024-09-13T18:23:30.605 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.595+0000 7f3b3f5fd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563855083bf0 0x563855082400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:30.605 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.595+0000 7f3b3f5fd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638550829d0 0x563855082e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:30.605 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.595+0000 7f3b3f5fd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56385508f5b0 0x5638550919c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:30.606 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.595+0000 7f3b3f5fd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x563854febc30 con 0x56385508f5b0 2024-09-13T18:23:30.606 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.595+0000 7f3b3f5fd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563854fe93c0 con 0x5638550829d0 2024-09-13T18:23:30.606 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.595+0000 7f3b3f5fd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563854fe4cb0 con 0x563855083bf0 2024-09-13T18:23:30.606 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.595+0000 7f3b36d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563855083bf0 0x563855082400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:30.606 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.595+0000 7f3b35d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56385508f5b0 0x5638550919c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:30.606 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.595+0000 7f3b36d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563855083bf0 0x563855082400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:50992/0 (socket says 172.21.15.114:50992) 2024-09-13T18:23:30.606 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.595+0000 7f3b36d9d640 1 -- 172.21.15.114:0/24889342 learned_addr learned my addr 172.21.15.114:0/24889342 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:30.606 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.595+0000 7f3b277fe640 1 --2- 172.21.15.114:0/37326459 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f3b200a1320 0x7f3b200a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:30.606 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.595+0000 7f3b3659c640 1 --2- 172.21.15.114:0/24889342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638550829d0 0x563855082e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:30.606 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.595+0000 7f3b36d9d640 1 --2- 172.21.15.114:0/37326459 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f3b200a1320 0x7f3b200a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:30.606 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.595+0000 7f3b277fe640 1 -- 172.21.15.114:0/37326459 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f3b280cd300 con 0x7f3b380feff0 2024-09-13T18:23:30.606 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.596+0000 7f3b35d9b640 1 -- 172.21.15.114:0/24889342 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563855083bf0 msgr2=0x563855082400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:30.606 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.596+0000 7f3b35d9b640 1 --2- 172.21.15.114:0/24889342 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563855083bf0 0x563855082400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.596+0000 7f3b35d9b640 1 -- 172.21.15.114:0/24889342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638550829d0 msgr2=0x563855082e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.596+0000 7f3b35d9b640 1 --2- 172.21.15.114:0/24889342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638550829d0 0x563855082e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.596+0000 7f3b35d9b640 1 -- 172.21.15.114:0/24889342 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563854f8b910 con 0x56385508f5b0 2024-09-13T18:23:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.596+0000 7f3b36d9d640 1 --2- 172.21.15.114:0/24889342 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563855083bf0 0x563855082400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:23:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.596+0000 7f3b35d9b640 1 --2- 172.21.15.114:0/24889342 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56385508f5b0 0x5638550919c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c02aa10 tx=0x7f3b2c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70f5c4f46ad5c84a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.596+0000 7f3b36d9d640 1 --2- 172.21.15.114:0/37326459 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f3b200a1320 0x7f3b200a37e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3b30014c00 tx=0x7f3b30004ab0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.596+0000 7f3b24ff9640 1 -- 172.21.15.114:0/24889342 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c004230 con 0x56385508f5b0 2024-09-13T18:23:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.596+0000 7f3b24ff9640 1 -- 172.21.15.114:0/24889342 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3b2c031df0 con 0x56385508f5b0 2024-09-13T18:23:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.596+0000 7f3b24ff9640 1 -- 172.21.15.114:0/24889342 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c003ca0 con 0x56385508f5b0 2024-09-13T18:23:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.597+0000 7f3b3f5fd740 1 -- 172.21.15.114:0/24889342 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56385508f5b0 msgr2=0x5638550919c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.597+0000 7f3b3f5fd740 1 --2- 172.21.15.114:0/24889342 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56385508f5b0 0x5638550919c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c02aa10 tx=0x7f3b2c005fd0 comp rx=0 tx=0).stop 2024-09-13T18:23:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.597+0000 7f3b3f5fd740 1 -- 172.21.15.114:0/24889342 shutdown_connections 2024-09-13T18:23:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.597+0000 7f3b3f5fd740 1 --2- 172.21.15.114:0/24889342 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56385508f5b0 0x5638550919c0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.597+0000 7f3b3f5fd740 1 --2- 172.21.15.114:0/24889342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638550829d0 0x563855082e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.597+0000 7f3b3f5fd740 1 --2- 172.21.15.114:0/24889342 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563855083bf0 0x563855082400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.597+0000 7f3b3f5fd740 1 -- 172.21.15.114:0/24889342 >> 172.21.15.114:0/24889342 conn(0x56385507ead0 msgr2=0x56385507eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.597+0000 7f3b3f5fd740 1 -- 172.21.15.114:0/24889342 shutdown_connections 2024-09-13T18:23:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.597+0000 7f3b3f5fd740 1 -- 172.21.15.114:0/24889342 wait complete. 2024-09-13T18:23:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.597+0000 7f3b3f5fd740 1 -- start start 2024-09-13T18:23:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.597+0000 7f3b3f5fd740 1 Processor -- start 2024-09-13T18:23:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.598+0000 7f3b3f5fd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5638550829d0 0x56385508efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.598+0000 7f3b3f5fd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563855083bf0 0x56385508c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.598+0000 7f3b3f5fd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56385508f5b0 0x56385508c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.598+0000 7f3b3f5fd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x563854fa2e80 con 0x5638550829d0 2024-09-13T18:23:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.598+0000 7f3b3f5fd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563854fa2b20 con 0x563855083bf0 2024-09-13T18:23:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.598+0000 7f3b3f5fd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563854f68260 con 0x56385508f5b0 2024-09-13T18:23:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.598+0000 7f3b3f5fd740 10 client.0 _ll_register_callbacks cb 0x563854f9ddf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:23:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.598+0000 7f3b3659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5638550829d0 0x56385508efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.598+0000 7f3b36d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56385508f5b0 0x56385508c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.598+0000 7f3b3659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5638550829d0 0x56385508efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33118/0 (socket says 172.21.15.114:33118) 2024-09-13T18:23:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.598+0000 7f3b3659c640 1 -- 172.21.15.114:0/1508946282 learned_addr learned my addr 172.21.15.114:0/1508946282 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.598+0000 7f3b36d9d640 1 --2- 172.21.15.114:0/1508946282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56385508f5b0 0x56385508c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:51000/0 (socket says 172.21.15.114:51000) 2024-09-13T18:23:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.598+0000 7f3b35d9b640 1 --2- 172.21.15.114:0/1508946282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563855083bf0 0x56385508c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.598+0000 7f3b36d9d640 1 -- 172.21.15.114:0/1508946282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563855083bf0 msgr2=0x56385508c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.598+0000 7f3b36d9d640 1 --2- 172.21.15.114:0/1508946282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563855083bf0 0x56385508c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.598+0000 7f3b36d9d640 1 -- 172.21.15.114:0/1508946282 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5638550829d0 msgr2=0x56385508efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.598+0000 7f3b36d9d640 1 --2- 172.21.15.114:0/1508946282 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5638550829d0 0x56385508efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.598+0000 7f3b36d9d640 1 -- 172.21.15.114:0/1508946282 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b3810c340 con 0x56385508f5b0 2024-09-13T18:23:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.598+0000 7f3b3659c640 1 --2- 172.21.15.114:0/1508946282 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5638550829d0 0x56385508efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:23:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.598+0000 7f3b36d9d640 1 --2- 172.21.15.114:0/1508946282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56385508f5b0 0x56385508c920 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3b30005fd0 tx=0x7f3b30004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d72eecceb6c1ce0d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.598+0000 7f3b1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:23:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.599+0000 7f3b1dffb640 1 -- 172.21.15.114:0/1508946282 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b300138d0 con 0x56385508f5b0 2024-09-13T18:23:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.599+0000 7f3b3f5fd740 1 -- 172.21.15.114:0/1508946282 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56385500d680 con 0x56385508f5b0 2024-09-13T18:23:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.599+0000 7f3b1dffb640 1 -- 172.21.15.114:0/1508946282 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f3b30013a30 con 0x56385508f5b0 2024-09-13T18:23:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.599+0000 7f3b1dffb640 1 -- 172.21.15.114:0/1508946282 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b30013bf0 con 0x56385508f5b0 2024-09-13T18:23:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.599+0000 7f3b3f5fd740 10 client.25486 fetch_fsmap learned FSMap version 30 2024-09-13T18:23:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.599+0000 7f3b3f5fd740 1 -- 172.21.15.114:0/1508946282 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x563854fe3330 con 0x56385508f5b0 2024-09-13T18:23:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.599+0000 7f3b1dffb640 1 -- 172.21.15.114:0/1508946282 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3b3002e030 con 0x56385508f5b0 2024-09-13T18:23:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.599+0000 7f3b1dffb640 1 -- 172.21.15.114:0/1508946282 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3b30010400 con 0x56385508f5b0 2024-09-13T18:23:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.599+0000 7f3b3f5fd740 10 client.25486 fetch_fsmap finished waiting for FSMap version 30 2024-09-13T18:23:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.599+0000 7f3b3f5fd740 10 client.25486 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:23:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.600+0000 7f3b3f5fd740 20 client.25486 populate_metadata read hostname 'smithi114' 2024-09-13T18:23:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.600+0000 7f3b3f5fd740 1 --2- 172.21.15.114:0/1508946282 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56385509f3b0 0x5638550a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.600+0000 7f3b3f5fd740 4 client.25486 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:23:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.600+0000 7f3b3f5fd740 1 -- 172.21.15.114:0/1508946282 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563854fd85d0 con 0x56385509f3b0 2024-09-13T18:23:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.600+0000 7f3b3659c640 1 --2- 172.21.15.114:0/1508946282 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56385509f3b0 0x5638550a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.600+0000 7f3b1dffb640 10 client.25486 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:23:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.600+0000 7f3b3659c640 1 --2- 172.21.15.114:0/1508946282 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56385509f3b0 0x5638550a17d0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7fd2abd3c1123ca server_cookie=c0daf294c96f72bc in_seq=0 out_seq=0 2024-09-13T18:23:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.601+0000 7f3b1dffb640 1 -- 172.21.15.114:0/1508946282 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x563854fd85d0 con 0x56385509f3b0 2024-09-13T18:23:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.601+0000 7f3b1dffb640 10 client.25486 handle_command_reply: tid=0 2024-09-13T18:23:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.601+0000 7f3b3f5fd740 1 client.25486 shutdown 2024-09-13T18:23:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.601+0000 7f3b3f5fd740 1 -- 172.21.15.114:0/1508946282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56385508f5b0 msgr2=0x56385508c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.601+0000 7f3b3f5fd740 1 --2- 172.21.15.114:0/1508946282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56385508f5b0 0x56385508c920 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3b30005fd0 tx=0x7f3b30004a40 comp rx=0 tx=0).stop 2024-09-13T18:23:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.601+0000 7f3b3f5fd740 1 -- 172.21.15.114:0/1508946282 shutdown_connections 2024-09-13T18:23:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.601+0000 7f3b3f5fd740 1 --2- 172.21.15.114:0/1508946282 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56385509f3b0 0x5638550a17d0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.601+0000 7f3b3f5fd740 1 --2- 172.21.15.114:0/1508946282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56385508f5b0 0x56385508c920 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.601+0000 7f3b1dffb640 0 client.25486 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:23:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.601+0000 7f3b3f5fd740 1 --2- 172.21.15.114:0/1508946282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563855083bf0 0x56385508c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.601+0000 7f3b3f5fd740 1 --2- 172.21.15.114:0/1508946282 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5638550829d0 0x56385508efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.601+0000 7f3b3f5fd740 1 -- 172.21.15.114:0/1508946282 >> 172.21.15.114:0/1508946282 conn(0x56385507ead0 msgr2=0x5638550847f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.601+0000 7f3b3f5fd740 1 -- 172.21.15.114:0/1508946282 shutdown_connections 2024-09-13T18:23:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.601+0000 7f3b3f5fd740 1 -- 172.21.15.114:0/1508946282 wait complete. 2024-09-13T18:23:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.601+0000 7f3b3f5fd740 20 client.25486 trim_cache size 0 max 16384 2024-09-13T18:23:30.618 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.613+0000 7f3b3f5fd740 1 Processor -- start 2024-09-13T18:23:30.618 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.613+0000 7f3b3f5fd740 1 -- start start 2024-09-13T18:23:30.618 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.614+0000 7f3b3f5fd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638550829d0 0x56385507e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:30.618 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.614+0000 7f3b3f5fd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563855083bf0 0x563855096580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:30.618 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.614+0000 7f3b3f5fd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56385508f5b0 0x563855096ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:30.618 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.614+0000 7f3b3f5fd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x563855083820 con 0x56385508f5b0 2024-09-13T18:23:30.618 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.614+0000 7f3b3f5fd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56385509b850 con 0x5638550829d0 2024-09-13T18:23:30.619 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.614+0000 7f3b3f5fd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56385508a7f0 con 0x563855083bf0 2024-09-13T18:23:30.619 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.614+0000 7f3b36d9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56385508f5b0 0x563855096ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:30.619 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.614+0000 7f3b36d9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56385508f5b0 0x563855096ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33124/0 (socket says 172.21.15.114:33124) 2024-09-13T18:23:30.619 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.614+0000 7f3b36d9d640 1 -- 172.21.15.114:0/1300865556 learned_addr learned my addr 172.21.15.114:0/1300865556 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:30.619 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.614+0000 7f3b3659c640 1 --2- 172.21.15.114:0/1300865556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638550829d0 0x56385507e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:30.619 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.614+0000 7f3b35d9b640 1 --2- 172.21.15.114:0/1300865556 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563855083bf0 0x563855096580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:30.619 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.614+0000 7f3b36d9d640 1 -- 172.21.15.114:0/1300865556 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563855083bf0 msgr2=0x563855096580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:30.619 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.614+0000 7f3b36d9d640 1 --2- 172.21.15.114:0/1300865556 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563855083bf0 0x563855096580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:30.619 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.614+0000 7f3b36d9d640 1 -- 172.21.15.114:0/1300865556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638550829d0 msgr2=0x56385507e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:30.619 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.614+0000 7f3b36d9d640 1 --2- 172.21.15.114:0/1300865556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638550829d0 0x56385507e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:30.620 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.614+0000 7f3b36d9d640 1 -- 172.21.15.114:0/1300865556 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563854f85300 con 0x56385508f5b0 2024-09-13T18:23:30.620 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.614+0000 7f3b35d9b640 1 --2- 172.21.15.114:0/1300865556 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563855083bf0 0x563855096580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:23:30.620 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.614+0000 7f3b36d9d640 1 --2- 172.21.15.114:0/1300865556 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56385508f5b0 0x563855096ac0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f3b30010760 tx=0x7f3b30009a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe6b576678c48e3c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:30.620 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.614+0000 7f3b257fa640 1 -- 172.21.15.114:0/1300865556 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b30014650 con 0x56385508f5b0 2024-09-13T18:23:30.620 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.615+0000 7f3b257fa640 1 -- 172.21.15.114:0/1300865556 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3b3002e070 con 0x56385508f5b0 2024-09-13T18:23:30.620 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.615+0000 7f3b257fa640 1 -- 172.21.15.114:0/1300865556 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b30013a30 con 0x56385508f5b0 2024-09-13T18:23:30.621 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:23:30 smithi134 ceph-mon[34283]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-13T18:23:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.615+0000 7f3b3f5fd740 1 -- 172.21.15.114:0/1300865556 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56385508f5b0 msgr2=0x563855096ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.615+0000 7f3b3f5fd740 1 --2- 172.21.15.114:0/1300865556 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56385508f5b0 0x563855096ac0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f3b30010760 tx=0x7f3b30009a30 comp rx=0 tx=0).stop 2024-09-13T18:23:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.615+0000 7f3b3f5fd740 1 -- 172.21.15.114:0/1300865556 shutdown_connections 2024-09-13T18:23:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.615+0000 7f3b3f5fd740 1 --2- 172.21.15.114:0/1300865556 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56385508f5b0 0x563855096ac0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.615+0000 7f3b3f5fd740 1 --2- 172.21.15.114:0/1300865556 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563855083bf0 0x563855096580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.615+0000 7f3b3f5fd740 1 --2- 172.21.15.114:0/1300865556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638550829d0 0x56385507e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.615+0000 7f3b3f5fd740 1 -- 172.21.15.114:0/1300865556 >> 172.21.15.114:0/1300865556 conn(0x56385507ead0 msgr2=0x56385507c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.615+0000 7f3b3f5fd740 1 -- 172.21.15.114:0/1300865556 shutdown_connections 2024-09-13T18:23:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.616+0000 7f3b3f5fd740 1 -- 172.21.15.114:0/1300865556 wait complete. 2024-09-13T18:23:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.616+0000 7f3b3f5fd740 1 -- start start 2024-09-13T18:23:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.616+0000 7f3b3f5fd740 1 Processor -- start 2024-09-13T18:23:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.616+0000 7f3b3f5fd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5638550829d0 0x563855096780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.616+0000 7f3b3f5fd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563855083bf0 0x563855094b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.616+0000 7f3b3f5fd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56385508f5b0 0x563855095060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.616+0000 7f3b3f5fd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x563855095670 con 0x5638550829d0 2024-09-13T18:23:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.616+0000 7f3b3659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5638550829d0 0x563855096780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.616+0000 7f3b3f5fd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638550957e0 con 0x56385508f5b0 2024-09-13T18:23:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.616+0000 7f3b3f5fd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638550959a0 con 0x563855083bf0 2024-09-13T18:23:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.616+0000 7f3b3659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5638550829d0 0x563855096780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33134/0 (socket says 172.21.15.114:33134) 2024-09-13T18:23:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.616+0000 7f3b3659c640 1 -- 172.21.15.114:0/4284931106 learned_addr learned my addr 172.21.15.114:0/4284931106 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.616+0000 7f3b3f5fd740 10 client.0 _ll_register_callbacks cb 0x7f3b2c031400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:23:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.617+0000 7f3b35d9b640 1 --2- 172.21.15.114:0/4284931106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563855083bf0 0x563855094b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.617+0000 7f3b3659c640 1 -- 172.21.15.114:0/4284931106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563855083bf0 msgr2=0x563855094b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.617+0000 7f3b3659c640 1 --2- 172.21.15.114:0/4284931106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563855083bf0 0x563855094b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.617+0000 7f3b3659c640 1 -- 172.21.15.114:0/4284931106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56385508f5b0 msgr2=0x563855095060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.617+0000 7f3b36d9d640 1 --2- 172.21.15.114:0/4284931106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56385508f5b0 0x563855095060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.617+0000 7f3b3659c640 1 --2- 172.21.15.114:0/4284931106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56385508f5b0 0x563855095060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.617+0000 7f3b3659c640 1 -- 172.21.15.114:0/4284931106 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563854fe3330 con 0x5638550829d0 2024-09-13T18:23:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.617+0000 7f3b3659c640 1 --2- 172.21.15.114:0/4284931106 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5638550829d0 0x563855096780 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f3b28096890 tx=0x7f3b28005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3aa3cf06c672271 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.617+0000 7f3b1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:23:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.617+0000 7f3b1dffb640 1 -- 172.21.15.114:0/4284931106 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b280cd120 con 0x5638550829d0 2024-09-13T18:23:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.617+0000 7f3b1dffb640 1 -- 172.21.15.114:0/4284931106 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f3b2809dca0 con 0x5638550829d0 2024-09-13T18:23:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.617+0000 7f3b3f5fd740 1 -- 172.21.15.114:0/4284931106 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3b2c031df0 con 0x5638550829d0 2024-09-13T18:23:30.627 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.617+0000 7f3b1dffb640 1 -- 172.21.15.114:0/4284931106 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b280b8250 con 0x5638550829d0 2024-09-13T18:23:30.627 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.617+0000 7f3b3f5fd740 10 client.16044 fetch_fsmap learned FSMap version 30 2024-09-13T18:23:30.627 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.617+0000 7f3b3f5fd740 1 -- 172.21.15.114:0/4284931106 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x563854faa8e0 con 0x5638550829d0 2024-09-13T18:23:30.627 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.618+0000 7f3b1dffb640 1 -- 172.21.15.114:0/4284931106 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3b280a22f0 con 0x5638550829d0 2024-09-13T18:23:30.627 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.618+0000 7f3b1dffb640 1 -- 172.21.15.114:0/4284931106 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3b28020070 con 0x5638550829d0 2024-09-13T18:23:30.627 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.618+0000 7f3b3f5fd740 10 client.16044 fetch_fsmap finished waiting for FSMap version 30 2024-09-13T18:23:30.627 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.618+0000 7f3b3f5fd740 10 client.16044 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:23:30.627 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.618+0000 7f3b3f5fd740 20 client.16044 populate_metadata read hostname 'smithi114' 2024-09-13T18:23:30.627 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.618+0000 7f3b3f5fd740 1 --2- 172.21.15.114:0/4284931106 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56385509f3b0 0x5638550f8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:30.627 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.618+0000 7f3b3f5fd740 4 client.16044 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T18:23:30.627 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.618+0000 7f3b3f5fd740 1 -- 172.21.15.114:0/4284931106 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7f3b2c031df0 con 0x56385509f3b0 2024-09-13T18:23:30.627 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.618+0000 7f3b35d9b640 1 --2- 172.21.15.114:0/4284931106 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56385509f3b0 0x5638550f8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:30.627 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.618+0000 7f3b1dffb640 10 client.16044 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:23:30.628 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.618+0000 7f3b35d9b640 1 --2- 172.21.15.114:0/4284931106 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56385509f3b0 0x5638550f8ed0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e29554fb69aabc1b server_cookie=e20a397e92641410 in_seq=0 out_seq=0 2024-09-13T18:23:30.629 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.622+0000 7f3b1dffb640 1 -- 172.21.15.114:0/4284931106 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f3b2c031df0 con 0x56385509f3b0 2024-09-13T18:23:30.629 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.622+0000 7f3b1dffb640 10 client.16044 handle_command_reply: tid=0 2024-09-13T18:23:30.629 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.622+0000 7f3b3f5fd740 1 client.16044 shutdown 2024-09-13T18:23:30.629 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.622+0000 7f3b3f5fd740 1 -- 172.21.15.114:0/4284931106 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5638550829d0 msgr2=0x563855096780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:30.629 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.622+0000 7f3b3f5fd740 1 --2- 172.21.15.114:0/4284931106 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5638550829d0 0x563855096780 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f3b28096890 tx=0x7f3b28005900 comp rx=0 tx=0).stop 2024-09-13T18:23:30.629 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.622+0000 7f3b3f5fd740 1 -- 172.21.15.114:0/4284931106 shutdown_connections 2024-09-13T18:23:30.629 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.622+0000 7f3b3f5fd740 1 --2- 172.21.15.114:0/4284931106 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56385509f3b0 0x5638550f8ed0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.622+0000 7f3b1dffb640 0 client.16044 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:23:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.622+0000 7f3b3f5fd740 1 --2- 172.21.15.114:0/4284931106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56385508f5b0 0x563855095060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.622+0000 7f3b3f5fd740 1 --2- 172.21.15.114:0/4284931106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563855083bf0 0x563855094b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.622+0000 7f3b3f5fd740 1 --2- 172.21.15.114:0/4284931106 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5638550829d0 0x563855096780 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.622+0000 7f3b3f5fd740 1 -- 172.21.15.114:0/4284931106 >> 172.21.15.114:0/4284931106 conn(0x56385507ead0 msgr2=0x563855077a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.622+0000 7f3b3f5fd740 1 -- 172.21.15.114:0/4284931106 shutdown_connections 2024-09-13T18:23:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.622+0000 7f3b3f5fd740 1 -- 172.21.15.114:0/4284931106 wait complete. 2024-09-13T18:23:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.622+0000 7f3b3f5fd740 20 client.16044 trim_cache size 0 max 16384 2024-09-13T18:23:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-13T18:23:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-13T18:23:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-13T18:23:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-13T18:23:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.623+0000 7f3b3cc96640 1 -- 172.21.15.114:0/37326459 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f3b200a1320 msgr2=0x7f3b200a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.623+0000 7f3b3cc96640 1 --2- 172.21.15.114:0/37326459 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f3b200a1320 0x7f3b200a37e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3b30014c00 tx=0x7f3b30004ab0 comp rx=0 tx=0).stop 2024-09-13T18:23:30.632 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.623+0000 7f3b3cc96640 1 -- 172.21.15.114:0/37326459 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3b380feff0 msgr2=0x7f3b381950e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:30.632 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.623+0000 7f3b3cc96640 1 --2- 172.21.15.114:0/37326459 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3b380feff0 0x7f3b381950e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f3b2800c890 tx=0x7f3b2800cd50 comp rx=0 tx=0).stop 2024-09-13T18:23:30.632 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.623+0000 7f3b3cc96640 1 -- 172.21.15.114:0/37326459 shutdown_connections 2024-09-13T18:23:30.632 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.623+0000 7f3b3cc96640 1 --2- 172.21.15.114:0/37326459 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f3b200a1320 0x7f3b200a37e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:30.632 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.623+0000 7f3b3cc96640 1 --2- 172.21.15.114:0/37326459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b38104c30 0x7f3b38199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:30.632 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.623+0000 7f3b3cc96640 1 --2- 172.21.15.114:0/37326459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b380ff970 0x7f3b38195620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:30.632 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.623+0000 7f3b3cc96640 1 --2- 172.21.15.114:0/37326459 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3b380feff0 0x7f3b381950e0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:30.632 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.623+0000 7f3b3cc96640 1 -- 172.21.15.114:0/37326459 >> 172.21.15.114:0/37326459 conn(0x7f3b38074940 msgr2=0x7f3b380fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:30.632 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.623+0000 7f3b3cc96640 1 -- 172.21.15.114:0/37326459 shutdown_connections 2024-09-13T18:23:30.632 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:30.623+0000 7f3b3cc96640 1 -- 172.21.15.114:0/37326459 wait complete. 2024-09-13T18:23:32.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:23:32 smithi114 ceph-mon[30897]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-13T18:23:32.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:23:32 smithi204 ceph-mon[34777]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-13T18:23:32.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:23:32 smithi134 ceph-mon[34283]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-13T18:23:34.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:23:34 smithi204 ceph-mon[34777]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-13T18:23:34.469 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:23:34 smithi114 ceph-mon[30897]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-13T18:23:34.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:23:34 smithi134 ceph-mon[34283]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-13T18:23:36.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:23:36 smithi204 ceph-mon[34777]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-13T18:23:36.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:23:36 smithi114 ceph-mon[30897]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-13T18:23:36.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:23:36 smithi134 ceph-mon[34283]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-13T18:23:37.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:23:37 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:23:37.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:23:37 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:23:37.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:23:37 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:23:37.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:23:37 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:23:37.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:23:37 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:23:37.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:23:37 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:23:37.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:23:37 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:23:37.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:23:37 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:23:37.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:23:37 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:23:38.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:23:38 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:23:38.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:23:38 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:23:38.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:23:38 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:23:38.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:23:38 smithi204 ceph-mon[34777]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-13T18:23:38.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:23:38 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:23:38.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:23:38 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:23:38.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:23:38 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:23:38.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:23:38 smithi114 ceph-mon[30897]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-13T18:23:38.592 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T18:23:38.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:23:38 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:23:38.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:23:38 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:23:38.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:23:38 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:23:38.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:23:38 smithi134 ceph-mon[34283]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-13T18:23:38.622 INFO:tasks.workunit.client.0.smithi114.stderr:Updating files: 99% (12586/12697) Updating files: 100% (12697/12697) Updating files: 100% (12697/12697), done. 2024-09-13T18:23:38.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.698+0000 7fe39105a640 1 -- 172.21.15.114:0/251715743 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe38c10a310 msgr2=0x7fe38c10c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:38.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.698+0000 7fe39105a640 1 --2- 172.21.15.114:0/251715743 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe38c10a310 0x7fe38c10c790 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fe384009ca0 tx=0x7fe38402a950 comp rx=0 tx=0).stop 2024-09-13T18:23:38.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.698+0000 7fe39105a640 1 -- 172.21.15.114:0/251715743 shutdown_connections 2024-09-13T18:23:38.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.698+0000 7fe39105a640 1 --2- 172.21.15.114:0/251715743 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe38c10a310 0x7fe38c10c790 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:38.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.698+0000 7fe39105a640 1 --2- 172.21.15.114:0/251715743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe38c101860 0x7fe38c109d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:38.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.698+0000 7fe39105a640 1 --2- 172.21.15.114:0/251715743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe38c100f40 0x7fe38c101320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:38.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.698+0000 7fe39105a640 1 -- 172.21.15.114:0/251715743 >> 172.21.15.114:0/251715743 conn(0x7fe38c075c60 msgr2=0x7fe38c0fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:38.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.698+0000 7fe39105a640 1 -- 172.21.15.114:0/251715743 shutdown_connections 2024-09-13T18:23:38.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.698+0000 7fe39105a640 1 -- 172.21.15.114:0/251715743 wait complete. 2024-09-13T18:23:38.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.699+0000 7fe39105a640 1 Processor -- start 2024-09-13T18:23:38.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.699+0000 7fe39105a640 1 -- start start 2024-09-13T18:23:38.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.699+0000 7fe39105a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe38c100f40 0x7fe38c1961a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:38.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.699+0000 7fe39105a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe38c101860 0x7fe38c1966e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:38.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.699+0000 7fe39105a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe38c10a310 0x7fe38c19aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:38.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.699+0000 7fe39105a640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38c19b0d0 con 0x7fe38c10a310 2024-09-13T18:23:38.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.699+0000 7fe39105a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38c19b240 con 0x7fe38c100f40 2024-09-13T18:23:38.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.699+0000 7fe39105a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38c19b400 con 0x7fe38c101860 2024-09-13T18:23:38.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.699+0000 7fe38b59e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe38c10a310 0x7fe38c19aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:38.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.700+0000 7fe38b59e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe38c10a310 0x7fe38c19aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51254/0 (socket says 172.21.15.114:51254) 2024-09-13T18:23:38.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.700+0000 7fe38b59e640 1 -- 172.21.15.114:0/2867903977 learned_addr learned my addr 172.21.15.114:0/2867903977 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:38.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.700+0000 7fe38ad9d640 1 --2- 172.21.15.114:0/2867903977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe38c100f40 0x7fe38c1961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:38.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.700+0000 7fe38a59c640 1 --2- 172.21.15.114:0/2867903977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe38c101860 0x7fe38c1966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:38.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.700+0000 7fe38b59e640 1 -- 172.21.15.114:0/2867903977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe38c101860 msgr2=0x7fe38c1966e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:38.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.700+0000 7fe38b59e640 1 --2- 172.21.15.114:0/2867903977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe38c101860 0x7fe38c1966e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:38.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.700+0000 7fe38b59e640 1 -- 172.21.15.114:0/2867903977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe38c100f40 msgr2=0x7fe38c1961a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:38.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.700+0000 7fe38b59e640 1 --2- 172.21.15.114:0/2867903977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe38c100f40 0x7fe38c1961a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:38.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.700+0000 7fe38b59e640 1 -- 172.21.15.114:0/2867903977 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe384009900 con 0x7fe38c10a310 2024-09-13T18:23:38.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.700+0000 7fe38a59c640 1 --2- 172.21.15.114:0/2867903977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe38c101860 0x7fe38c1966e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:23:38.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.700+0000 7fe38b59e640 1 --2- 172.21.15.114:0/2867903977 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe38c10a310 0x7fe38c19aad0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fe38402af40 tx=0x7fe38402fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:38.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.700+0000 7fe383fff640 1 -- 172.21.15.114:0/2867903977 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe384030440 con 0x7fe38c10a310 2024-09-13T18:23:38.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.700+0000 7fe39105a640 1 -- 172.21.15.114:0/2867903977 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe38c19b680 con 0x7fe38c10a310 2024-09-13T18:23:38.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.700+0000 7fe383fff640 1 -- 172.21.15.114:0/2867903977 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe384030a30 con 0x7fe38c10a310 2024-09-13T18:23:38.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.700+0000 7fe383fff640 1 -- 172.21.15.114:0/2867903977 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe384039450 con 0x7fe38c10a310 2024-09-13T18:23:38.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.700+0000 7fe39105a640 1 -- 172.21.15.114:0/2867903977 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe38c1a2f60 con 0x7fe38c10a310 2024-09-13T18:23:38.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.702+0000 7fe383fff640 1 -- 172.21.15.114:0/2867903977 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) v1 ==== 136107+0+0 (secure 0 0 0) 0x7fe38402e050 con 0x7fe38c10a310 2024-09-13T18:23:38.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.703+0000 7fe393e57740 1 Processor -- start 2024-09-13T18:23:38.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.703+0000 7fe393e57740 1 -- start start 2024-09-13T18:23:38.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.703+0000 7fe393e57740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdb1250e10 0x55bdb12511f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:38.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.703+0000 7fe393e57740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdb1251730 0x55bdb125ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:38.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.703+0000 7fe393e57740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdb125bfd0 0x55bdb125e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:38.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.703+0000 7fe393e57740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdb11bbf20 con 0x55bdb1250e10 2024-09-13T18:23:38.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.703+0000 7fe393e57740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdb11b96b0 con 0x55bdb1251730 2024-09-13T18:23:38.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.703+0000 7fe393e57740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdb11b4fa0 con 0x55bdb125bfd0 2024-09-13T18:23:38.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.703+0000 7fe383fff640 1 --2- 172.21.15.114:0/2867903977 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe3780a1320 0x7fe3780a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:38.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.703+0000 7fe38a59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdb1250e10 0x55bdb12511f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:38.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.703+0000 7fe38a59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdb1250e10 0x55bdb12511f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51260/0 (socket says 172.21.15.114:51260) 2024-09-13T18:23:38.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.703+0000 7fe38a59c640 1 -- 172.21.15.114:0/873143748 learned_addr learned my addr 172.21.15.114:0/873143748 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:38.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.703+0000 7fe38b59e640 1 --2- 172.21.15.114:0/2867903977 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe3780a1320 0x7fe3780a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:38.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.703+0000 7fe383fff640 1 -- 172.21.15.114:0/2867903977 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fe3840ee320 con 0x7fe38c10a310 2024-09-13T18:23:38.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.704+0000 7fe38ad9d640 1 --2- 172.21.15.114:0/873143748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdb1251730 0x55bdb125ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:38.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.704+0000 7fe38a59c640 1 --2- 172.21.15.114:0/873143748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdb125bfd0 0x55bdb125e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:38.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.704+0000 7fe38a59c640 1 -- 172.21.15.114:0/873143748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdb125bfd0 msgr2=0x55bdb125e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:38.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.704+0000 7fe38a59c640 1 --2- 172.21.15.114:0/873143748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdb125bfd0 0x55bdb125e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:38.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.704+0000 7fe38a59c640 1 -- 172.21.15.114:0/873143748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdb1251730 msgr2=0x55bdb125ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:38.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.704+0000 7fe38a59c640 1 --2- 172.21.15.114:0/873143748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdb1251730 0x55bdb125ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:38.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.704+0000 7fe38a59c640 1 -- 172.21.15.114:0/873143748 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bdb116f840 con 0x55bdb1250e10 2024-09-13T18:23:38.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.704+0000 7fe38b59e640 1 --2- 172.21.15.114:0/2867903977 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe3780a1320 0x7fe3780a37e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe3840eb600 tx=0x7fe38403e000 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:38.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.704+0000 7fe38a59c640 1 --2- 172.21.15.114:0/873143748 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdb1250e10 0x55bdb12511f0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fe3740144f0 tx=0x7fe37400b530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57d9c8ddd6bf0af server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:38.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.704+0000 7fe3817fa640 1 -- 172.21.15.114:0/873143748 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe374012710 con 0x55bdb1250e10 2024-09-13T18:23:38.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.704+0000 7fe3817fa640 1 -- 172.21.15.114:0/873143748 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe374018070 con 0x55bdb1250e10 2024-09-13T18:23:38.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.704+0000 7fe3817fa640 1 -- 172.21.15.114:0/873143748 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3740118e0 con 0x55bdb1250e10 2024-09-13T18:23:38.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.705+0000 7fe393e57740 1 -- 172.21.15.114:0/873143748 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdb1250e10 msgr2=0x55bdb12511f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:38.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.705+0000 7fe393e57740 1 --2- 172.21.15.114:0/873143748 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdb1250e10 0x55bdb12511f0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fe3740144f0 tx=0x7fe37400b530 comp rx=0 tx=0).stop 2024-09-13T18:23:38.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.705+0000 7fe393e57740 1 -- 172.21.15.114:0/873143748 shutdown_connections 2024-09-13T18:23:38.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.705+0000 7fe393e57740 1 --2- 172.21.15.114:0/873143748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdb125bfd0 0x55bdb125e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:38.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.705+0000 7fe393e57740 1 --2- 172.21.15.114:0/873143748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdb1251730 0x55bdb125ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:38.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.705+0000 7fe393e57740 1 --2- 172.21.15.114:0/873143748 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdb1250e10 0x55bdb12511f0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:38.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.705+0000 7fe393e57740 1 -- 172.21.15.114:0/873143748 >> 172.21.15.114:0/873143748 conn(0x55bdb124d380 msgr2=0x55bdb124d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:38.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.705+0000 7fe393e57740 1 -- 172.21.15.114:0/873143748 shutdown_connections 2024-09-13T18:23:38.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.705+0000 7fe393e57740 1 -- 172.21.15.114:0/873143748 wait complete. 2024-09-13T18:23:38.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.705+0000 7fe393e57740 1 -- start start 2024-09-13T18:23:38.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.705+0000 7fe393e57740 1 Processor -- start 2024-09-13T18:23:38.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.706+0000 7fe393e57740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdb1250e10 0x55bdb125b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:38.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.706+0000 7fe393e57740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdb1251730 0x55bdb1258c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:38.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.706+0000 7fe393e57740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdb125bfd0 0x55bdb1259220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:38.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.706+0000 7fe393e57740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdb11562a0 con 0x55bdb1251730 2024-09-13T18:23:38.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.706+0000 7fe393e57740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdb0e8d430 con 0x55bdb1250e10 2024-09-13T18:23:38.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.706+0000 7fe393e57740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdb1138680 con 0x55bdb125bfd0 2024-09-13T18:23:38.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.706+0000 7fe393e57740 10 client.0 _ll_register_callbacks cb 0x55bdb11ae610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:23:38.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.706+0000 7fe38ad9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdb1251730 0x55bdb1258c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:38.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.706+0000 7fe38ad9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdb1251730 0x55bdb1258c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51268/0 (socket says 172.21.15.114:51268) 2024-09-13T18:23:38.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.706+0000 7fe38ad9d640 1 -- 172.21.15.114:0/4108475482 learned_addr learned my addr 172.21.15.114:0/4108475482 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:38.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.706+0000 7fe38a59c640 1 --2- 172.21.15.114:0/4108475482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdb125bfd0 0x55bdb1259220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:38.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.706+0000 7fe38a59c640 1 --2- 172.21.15.114:0/4108475482 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdb1250e10 0x55bdb125b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:38.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.706+0000 7fe38ad9d640 1 -- 172.21.15.114:0/4108475482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdb125bfd0 msgr2=0x55bdb1259220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:38.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.706+0000 7fe38ad9d640 1 --2- 172.21.15.114:0/4108475482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdb125bfd0 0x55bdb1259220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:38.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.706+0000 7fe38ad9d640 1 -- 172.21.15.114:0/4108475482 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdb1250e10 msgr2=0x55bdb125b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:38.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.706+0000 7fe38ad9d640 1 --2- 172.21.15.114:0/4108475482 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdb1250e10 0x55bdb125b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:38.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.706+0000 7fe38ad9d640 1 -- 172.21.15.114:0/4108475482 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe37400bc60 con 0x55bdb1251730 2024-09-13T18:23:38.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.706+0000 7fe38a59c640 1 --2- 172.21.15.114:0/4108475482 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdb1250e10 0x55bdb125b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:23:38.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.706+0000 7fe38ad9d640 1 --2- 172.21.15.114:0/4108475482 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdb1251730 0x55bdb1258c50 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fe37c00bfc0 tx=0x7fe37c00f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2a1c13ca8a73988 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:38.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.706+0000 7fe3727fc640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:23:38.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.706+0000 7fe3727fc640 1 -- 172.21.15.114:0/4108475482 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37c014870 con 0x55bdb1251730 2024-09-13T18:23:38.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.706+0000 7fe3727fc640 1 -- 172.21.15.114:0/4108475482 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fe37c0149d0 con 0x55bdb1251730 2024-09-13T18:23:38.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.706+0000 7fe393e57740 1 -- 172.21.15.114:0/4108475482 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bdb0ea84e0 con 0x55bdb1251730 2024-09-13T18:23:38.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.706+0000 7fe3727fc640 1 -- 172.21.15.114:0/4108475482 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe37c00fea0 con 0x55bdb1251730 2024-09-13T18:23:38.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.706+0000 7fe393e57740 10 client.16056 fetch_fsmap learned FSMap version 30 2024-09-13T18:23:38.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.707+0000 7fe393e57740 1 -- 172.21.15.114:0/4108475482 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55bdb11b3620 con 0x55bdb1251730 2024-09-13T18:23:38.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.707+0000 7fe3727fc640 1 -- 172.21.15.114:0/4108475482 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55bdb0ea84e0 con 0x55bdb1251730 2024-09-13T18:23:38.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.707+0000 7fe3727fc640 1 -- 172.21.15.114:0/4108475482 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe37c013bd0 con 0x55bdb1251730 2024-09-13T18:23:38.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.707+0000 7fe393e57740 10 client.16056 fetch_fsmap finished waiting for FSMap version 30 2024-09-13T18:23:38.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.707+0000 7fe393e57740 10 client.16056 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:23:38.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.707+0000 7fe393e57740 20 client.16056 populate_metadata read hostname 'smithi114' 2024-09-13T18:23:38.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.707+0000 7fe393e57740 1 --2- 172.21.15.114:0/4108475482 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55bdb126bee0 0x55bdb126e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:38.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.707+0000 7fe393e57740 4 client.16056 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:23:38.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.707+0000 7fe393e57740 1 -- 172.21.15.114:0/4108475482 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bdb0eb97e0 con 0x55bdb126bee0 2024-09-13T18:23:38.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.707+0000 7fe38a59c640 1 --2- 172.21.15.114:0/4108475482 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55bdb126bee0 0x55bdb126e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:38.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.708+0000 7fe3727fc640 10 client.16056 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:23:38.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.708+0000 7fe38a59c640 1 --2- 172.21.15.114:0/4108475482 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55bdb126bee0 0x55bdb126e300 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba5132c2e87541ac server_cookie=417777e10ee213ca in_seq=0 out_seq=0 2024-09-13T18:23:38.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.708+0000 7fe3727fc640 1 -- 172.21.15.114:0/4108475482 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55bdb0eb97e0 con 0x55bdb126bee0 2024-09-13T18:23:38.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.708+0000 7fe3727fc640 10 client.16056 handle_command_reply: tid=0 2024-09-13T18:23:38.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.708+0000 7fe393e57740 1 client.16056 shutdown 2024-09-13T18:23:38.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.708+0000 7fe393e57740 1 -- 172.21.15.114:0/4108475482 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdb1251730 msgr2=0x55bdb1258c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:38.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.708+0000 7fe393e57740 1 --2- 172.21.15.114:0/4108475482 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdb1251730 0x55bdb1258c50 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fe37c00bfc0 tx=0x7fe37c00f670 comp rx=0 tx=0).stop 2024-09-13T18:23:38.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.708+0000 7fe393e57740 1 -- 172.21.15.114:0/4108475482 shutdown_connections 2024-09-13T18:23:38.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.708+0000 7fe393e57740 1 --2- 172.21.15.114:0/4108475482 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55bdb126bee0 0x55bdb126e300 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:38.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.708+0000 7fe393e57740 1 --2- 172.21.15.114:0/4108475482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdb125bfd0 0x55bdb1259220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:38.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.708+0000 7fe3727fc640 0 client.16056 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:23:38.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.708+0000 7fe393e57740 1 --2- 172.21.15.114:0/4108475482 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdb1251730 0x55bdb1258c50 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:38.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.708+0000 7fe393e57740 1 --2- 172.21.15.114:0/4108475482 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdb1250e10 0x55bdb125b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:38.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.708+0000 7fe393e57740 1 -- 172.21.15.114:0/4108475482 >> 172.21.15.114:0/4108475482 conn(0x55bdb124d380 msgr2=0x55bdb12530e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:38.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.708+0000 7fe393e57740 1 -- 172.21.15.114:0/4108475482 shutdown_connections 2024-09-13T18:23:38.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.708+0000 7fe393e57740 1 -- 172.21.15.114:0/4108475482 wait complete. 2024-09-13T18:23:38.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.709+0000 7fe393e57740 20 client.16056 trim_cache size 0 max 16384 2024-09-13T18:23:38.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.723+0000 7fe393e57740 1 Processor -- start 2024-09-13T18:23:38.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.723+0000 7fe393e57740 1 -- start start 2024-09-13T18:23:38.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.723+0000 7fe393e57740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdb1250e10 0x55bdb1262d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:38.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.723+0000 7fe393e57740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdb1251730 0x55bdb12632a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:38.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.723+0000 7fe393e57740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdb125bfd0 0x55bdb1253ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:38.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.723+0000 7fe393e57740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdb1248760 con 0x55bdb125bfd0 2024-09-13T18:23:38.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.723+0000 7fe393e57740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdb1268aa0 con 0x55bdb1251730 2024-09-13T18:23:38.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.723+0000 7fe393e57740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdb1257180 con 0x55bdb1250e10 2024-09-13T18:23:38.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.723+0000 7fe38a59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdb125bfd0 0x55bdb1253ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:38.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.723+0000 7fe38a59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdb125bfd0 0x55bdb1253ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51270/0 (socket says 172.21.15.114:51270) 2024-09-13T18:23:38.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.723+0000 7fe38a59c640 1 -- 172.21.15.114:0/1530684489 learned_addr learned my addr 172.21.15.114:0/1530684489 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:38.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.724+0000 7fe38a59c640 1 --2- 172.21.15.114:0/1530684489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdb1250e10 0x55bdb1262d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:38.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.724+0000 7fe38ad9d640 1 --2- 172.21.15.114:0/1530684489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdb1251730 0x55bdb12632a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:38.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.724+0000 7fe38a59c640 1 -- 172.21.15.114:0/1530684489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdb1250e10 msgr2=0x55bdb1262d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:38.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.724+0000 7fe38a59c640 1 --2- 172.21.15.114:0/1530684489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdb1250e10 0x55bdb1262d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:38.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.724+0000 7fe38a59c640 1 -- 172.21.15.114:0/1530684489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdb1251730 msgr2=0x55bdb12632a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:38.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.724+0000 7fe38a59c640 1 --2- 172.21.15.114:0/1530684489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdb1251730 0x55bdb12632a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:38.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.724+0000 7fe38a59c640 1 -- 172.21.15.114:0/1530684489 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bdb1155790 con 0x55bdb125bfd0 2024-09-13T18:23:38.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.724+0000 7fe38a59c640 1 --2- 172.21.15.114:0/1530684489 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdb125bfd0 0x55bdb1253ef0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fe374011550 tx=0x7fe37401f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9222ded3161cc36f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:38.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.724+0000 7fe381ffb640 1 -- 172.21.15.114:0/1530684489 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe374024040 con 0x55bdb125bfd0 2024-09-13T18:23:38.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.724+0000 7fe381ffb640 1 -- 172.21.15.114:0/1530684489 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe374011760 con 0x55bdb125bfd0 2024-09-13T18:23:38.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.724+0000 7fe381ffb640 1 -- 172.21.15.114:0/1530684489 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe374012770 con 0x55bdb125bfd0 2024-09-13T18:23:38.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.725+0000 7fe393e57740 1 -- 172.21.15.114:0/1530684489 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdb125bfd0 msgr2=0x55bdb1253ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:38.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.725+0000 7fe393e57740 1 --2- 172.21.15.114:0/1530684489 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdb125bfd0 0x55bdb1253ef0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fe374011550 tx=0x7fe37401f040 comp rx=0 tx=0).stop 2024-09-13T18:23:38.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.725+0000 7fe393e57740 1 -- 172.21.15.114:0/1530684489 shutdown_connections 2024-09-13T18:23:38.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.725+0000 7fe393e57740 1 --2- 172.21.15.114:0/1530684489 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdb125bfd0 0x55bdb1253ef0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:38.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.725+0000 7fe393e57740 1 --2- 172.21.15.114:0/1530684489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdb1251730 0x55bdb12632a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:38.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.725+0000 7fe393e57740 1 --2- 172.21.15.114:0/1530684489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdb1250e10 0x55bdb1262d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:38.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.725+0000 7fe393e57740 1 -- 172.21.15.114:0/1530684489 >> 172.21.15.114:0/1530684489 conn(0x55bdb124d380 msgr2=0x55bdb1272c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:38.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.725+0000 7fe393e57740 1 -- 172.21.15.114:0/1530684489 shutdown_connections 2024-09-13T18:23:38.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.725+0000 7fe393e57740 1 -- 172.21.15.114:0/1530684489 wait complete. 2024-09-13T18:23:38.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.725+0000 7fe393e57740 1 -- start start 2024-09-13T18:23:38.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.725+0000 7fe393e57740 1 Processor -- start 2024-09-13T18:23:38.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.726+0000 7fe393e57740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdb1250e10 0x55bdb1262df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:38.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.726+0000 7fe393e57740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdb1251730 0x55bdb125f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:38.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.726+0000 7fe393e57740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdb125bfd0 0x55bdb125f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:38.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.726+0000 7fe393e57740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdb125fda0 con 0x55bdb1250e10 2024-09-13T18:23:38.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.726+0000 7fe393e57740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdb125ff10 con 0x55bdb1251730 2024-09-13T18:23:38.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.726+0000 7fe393e57740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdb12600d0 con 0x55bdb125bfd0 2024-09-13T18:23:38.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.726+0000 7fe393e57740 10 client.0 _ll_register_callbacks cb 0x7fe37c00b2e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:23:38.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.726+0000 7fe38a59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdb1250e10 0x55bdb1262df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:38.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.726+0000 7fe38a59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdb1250e10 0x55bdb1262df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51272/0 (socket says 172.21.15.114:51272) 2024-09-13T18:23:38.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.726+0000 7fe38a59c640 1 -- 172.21.15.114:0/2053322634 learned_addr learned my addr 172.21.15.114:0/2053322634 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:38.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.726+0000 7fe38a59c640 1 --2- 172.21.15.114:0/2053322634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdb125bfd0 0x55bdb125f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:38.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.726+0000 7fe38a59c640 1 -- 172.21.15.114:0/2053322634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdb125bfd0 msgr2=0x55bdb125f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:38.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.726+0000 7fe38a59c640 1 --2- 172.21.15.114:0/2053322634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdb125bfd0 0x55bdb125f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:38.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.726+0000 7fe38a59c640 1 -- 172.21.15.114:0/2053322634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdb1251730 msgr2=0x55bdb125f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:38.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.726+0000 7fe38ad9d640 1 --2- 172.21.15.114:0/2053322634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdb1251730 0x55bdb125f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:38.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.726+0000 7fe38a59c640 1 --2- 172.21.15.114:0/2053322634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdb1251730 0x55bdb125f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:38.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.726+0000 7fe38a59c640 1 -- 172.21.15.114:0/2053322634 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bdb116f840 con 0x55bdb1250e10 2024-09-13T18:23:38.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.726+0000 7fe38a59c640 1 --2- 172.21.15.114:0/2053322634 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdb1250e10 0x55bdb1262df0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fe374012010 tx=0x7fe374011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40840ed4b9c44fb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:38.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.726+0000 7fe3727fc640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:23:38.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.726+0000 7fe3727fc640 1 -- 172.21.15.114:0/2053322634 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe374024030 con 0x55bdb1250e10 2024-09-13T18:23:38.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.726+0000 7fe3727fc640 1 -- 172.21.15.114:0/2053322634 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fe374000e40 con 0x55bdb1250e10 2024-09-13T18:23:38.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.726+0000 7fe393e57740 1 -- 172.21.15.114:0/2053322634 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe374018070 con 0x55bdb1250e10 2024-09-13T18:23:38.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.726+0000 7fe3727fc640 1 -- 172.21.15.114:0/2053322634 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe374011c50 con 0x55bdb1250e10 2024-09-13T18:23:38.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.727+0000 7fe393e57740 10 client.16062 fetch_fsmap learned FSMap version 30 2024-09-13T18:23:38.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.727+0000 7fe393e57740 1 -- 172.21.15.114:0/2053322634 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55bdb117aea0 con 0x55bdb1250e10 2024-09-13T18:23:38.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.727+0000 7fe3727fc640 1 -- 172.21.15.114:0/2053322634 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe374018070 con 0x55bdb1250e10 2024-09-13T18:23:38.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.727+0000 7fe3727fc640 1 -- 172.21.15.114:0/2053322634 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe374012920 con 0x55bdb1250e10 2024-09-13T18:23:38.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.727+0000 7fe393e57740 10 client.16062 fetch_fsmap finished waiting for FSMap version 30 2024-09-13T18:23:38.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.727+0000 7fe393e57740 10 client.16062 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:23:38.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.727+0000 7fe393e57740 20 client.16062 populate_metadata read hostname 'smithi114' 2024-09-13T18:23:38.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.727+0000 7fe393e57740 1 --2- 172.21.15.114:0/2053322634 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55bdb126bee0 0x55bdb12c5300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:38.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.727+0000 7fe393e57740 4 client.16062 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T18:23:38.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.727+0000 7fe393e57740 1 -- 172.21.15.114:0/2053322634 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bdb0f0ce30 con 0x55bdb126bee0 2024-09-13T18:23:38.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.727+0000 7fe38ad9d640 1 --2- 172.21.15.114:0/2053322634 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55bdb126bee0 0x55bdb12c5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:38.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.727+0000 7fe3727fc640 10 client.16062 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:23:38.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.727+0000 7fe38ad9d640 1 --2- 172.21.15.114:0/2053322634 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55bdb126bee0 0x55bdb12c5300 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdeee322454b718d server_cookie=878dae862f455b52 in_seq=0 out_seq=0 2024-09-13T18:23:38.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.728+0000 7fe3727fc640 1 -- 172.21.15.114:0/2053322634 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55bdb0f0ce30 con 0x55bdb126bee0 2024-09-13T18:23:38.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.728+0000 7fe3727fc640 10 client.16062 handle_command_reply: tid=0 2024-09-13T18:23:38.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.728+0000 7fe393e57740 1 client.16062 shutdown 2024-09-13T18:23:38.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.728+0000 7fe393e57740 1 -- 172.21.15.114:0/2053322634 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdb1250e10 msgr2=0x55bdb1262df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:38.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.728+0000 7fe393e57740 1 --2- 172.21.15.114:0/2053322634 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdb1250e10 0x55bdb1262df0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fe374012010 tx=0x7fe374011440 comp rx=0 tx=0).stop 2024-09-13T18:23:38.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.728+0000 7fe393e57740 1 -- 172.21.15.114:0/2053322634 shutdown_connections 2024-09-13T18:23:38.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.728+0000 7fe393e57740 1 --2- 172.21.15.114:0/2053322634 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55bdb126bee0 0x55bdb12c5300 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:38.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.728+0000 7fe393e57740 1 --2- 172.21.15.114:0/2053322634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bdb125bfd0 0x55bdb125f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:38.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.728+0000 7fe3727fc640 0 client.16062 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:23:38.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.728+0000 7fe393e57740 1 --2- 172.21.15.114:0/2053322634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdb1251730 0x55bdb125f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:38.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.728+0000 7fe393e57740 1 --2- 172.21.15.114:0/2053322634 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdb1250e10 0x55bdb1262df0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:38.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.728+0000 7fe393e57740 1 -- 172.21.15.114:0/2053322634 >> 172.21.15.114:0/2053322634 conn(0x55bdb124d380 msgr2=0x55bdb1272c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:38.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.728+0000 7fe393e57740 1 -- 172.21.15.114:0/2053322634 shutdown_connections 2024-09-13T18:23:38.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.728+0000 7fe393e57740 1 -- 172.21.15.114:0/2053322634 wait complete. 2024-09-13T18:23:38.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.728+0000 7fe393e57740 20 client.16062 trim_cache size 0 max 16384 2024-09-13T18:23:38.737 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:23:38.737 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "no active scrubs running", 2024-09-13T18:23:38.738 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": {} 2024-09-13T18:23:38.738 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:23:38.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.728+0000 7fe39105a640 1 -- 172.21.15.114:0/2867903977 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe3780a1320 msgr2=0x7fe3780a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:38.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.728+0000 7fe39105a640 1 --2- 172.21.15.114:0/2867903977 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe3780a1320 0x7fe3780a37e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe3840eb600 tx=0x7fe38403e000 comp rx=0 tx=0).stop 2024-09-13T18:23:38.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.728+0000 7fe39105a640 1 -- 172.21.15.114:0/2867903977 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe38c10a310 msgr2=0x7fe38c19aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:38.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.728+0000 7fe39105a640 1 --2- 172.21.15.114:0/2867903977 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe38c10a310 0x7fe38c19aad0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fe38402af40 tx=0x7fe38402fd10 comp rx=0 tx=0).stop 2024-09-13T18:23:38.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.729+0000 7fe39105a640 1 -- 172.21.15.114:0/2867903977 shutdown_connections 2024-09-13T18:23:38.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.729+0000 7fe39105a640 1 --2- 172.21.15.114:0/2867903977 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe3780a1320 0x7fe3780a37e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:38.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.729+0000 7fe39105a640 1 --2- 172.21.15.114:0/2867903977 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe38c10a310 0x7fe38c19aad0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:38.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.729+0000 7fe39105a640 1 --2- 172.21.15.114:0/2867903977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe38c101860 0x7fe38c1966e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:38.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.729+0000 7fe39105a640 1 --2- 172.21.15.114:0/2867903977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe38c100f40 0x7fe38c1961a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:38.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.729+0000 7fe39105a640 1 -- 172.21.15.114:0/2867903977 >> 172.21.15.114:0/2867903977 conn(0x7fe38c075c60 msgr2=0x7fe38c0fb590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:38.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.729+0000 7fe39105a640 1 -- 172.21.15.114:0/2867903977 shutdown_connections 2024-09-13T18:23:38.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.729+0000 7fe39105a640 1 -- 172.21.15.114:0/2867903977 wait complete. 2024-09-13T18:23:38.742 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-13T18:23:38.742 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:23:38.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.880+0000 7f8cfa525640 1 -- 172.21.15.114:0/3630903880 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8cf40fee80 msgr2=0x7f8cf40ff260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:38.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.880+0000 7f8cfa525640 1 --2- 172.21.15.114:0/3630903880 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8cf40fee80 0x7f8cf40ff260 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4009930 tx=0x7f8ce402a420 comp rx=0 tx=0).stop 2024-09-13T18:23:38.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.880+0000 7f8cfa525640 1 -- 172.21.15.114:0/3630903880 shutdown_connections 2024-09-13T18:23:38.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.880+0000 7f8cfa525640 1 --2- 172.21.15.114:0/3630903880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cf4103fd0 0x7f8cf410bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:38.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.880+0000 7f8cfa525640 1 --2- 172.21.15.114:0/3630903880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8cf40ff830 0x7f8cf4103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:38.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.880+0000 7f8cfa525640 1 --2- 172.21.15.114:0/3630903880 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8cf40fee80 0x7f8cf40ff260 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:38.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.880+0000 7f8cfa525640 1 -- 172.21.15.114:0/3630903880 >> 172.21.15.114:0/3630903880 conn(0x7f8cf4075c60 msgr2=0x7f8cf40fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:38.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.880+0000 7f8cfa525640 1 -- 172.21.15.114:0/3630903880 shutdown_connections 2024-09-13T18:23:38.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.880+0000 7f8cfa525640 1 -- 172.21.15.114:0/3630903880 wait complete. 2024-09-13T18:23:38.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.880+0000 7f8cfa525640 1 Processor -- start 2024-09-13T18:23:38.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.880+0000 7f8cfa525640 1 -- start start 2024-09-13T18:23:38.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.880+0000 7f8cfa525640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8cf40fee80 0x7f8cf4197010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:38.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.880+0000 7f8cfa525640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8cf4103fd0 0x7f8cf4197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:38.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.881+0000 7f8cfa525640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cf4197a90 0x7f8cf41a2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:38.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.881+0000 7f8cfa525640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf41a2730 con 0x7f8cf4103fd0 2024-09-13T18:23:38.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.881+0000 7f8cfa525640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf41a28a0 con 0x7f8cf4197a90 2024-09-13T18:23:38.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.881+0000 7f8cfa525640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf41a2a60 con 0x7f8cf40fee80 2024-09-13T18:23:38.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.881+0000 7f8cebfff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8cf4103fd0 0x7f8cf4197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:38.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.881+0000 7f8cebfff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8cf4103fd0 0x7f8cf4197550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51286/0 (socket says 172.21.15.114:51286) 2024-09-13T18:23:38.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.881+0000 7f8cebfff640 1 -- 172.21.15.114:0/1707115762 learned_addr learned my addr 172.21.15.114:0/1707115762 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:38.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.881+0000 7f8cf8ac2640 1 --2- 172.21.15.114:0/1707115762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cf4197a90 0x7f8cf41a2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:38.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.881+0000 7f8cf3fff640 1 --2- 172.21.15.114:0/1707115762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8cf40fee80 0x7f8cf4197010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:38.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.881+0000 7f8cebfff640 1 -- 172.21.15.114:0/1707115762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8cf40fee80 msgr2=0x7f8cf4197010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:38.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.881+0000 7f8cebfff640 1 --2- 172.21.15.114:0/1707115762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8cf40fee80 0x7f8cf4197010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:38.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.881+0000 7f8cebfff640 1 -- 172.21.15.114:0/1707115762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cf4197a90 msgr2=0x7f8cf41a2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:38.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.881+0000 7f8cebfff640 1 --2- 172.21.15.114:0/1707115762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cf4197a90 0x7f8cf41a2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:38.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.881+0000 7f8cebfff640 1 -- 172.21.15.114:0/1707115762 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ce4009560 con 0x7f8cf4103fd0 2024-09-13T18:23:38.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.881+0000 7f8cebfff640 1 --2- 172.21.15.114:0/1707115762 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8cf4103fd0 0x7f8cf4197550 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc00e9f0 tx=0x7f8cdc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:38.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.881+0000 7f8cf1ffb640 1 -- 172.21.15.114:0/1707115762 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc0096c0 con 0x7f8cf4103fd0 2024-09-13T18:23:38.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.881+0000 7f8cfa525640 1 -- 172.21.15.114:0/1707115762 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cf41a2d40 con 0x7f8cf4103fd0 2024-09-13T18:23:38.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.881+0000 7f8cf1ffb640 1 -- 172.21.15.114:0/1707115762 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8cdc00ccc0 con 0x7f8cf4103fd0 2024-09-13T18:23:38.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.881+0000 7f8cf1ffb640 1 -- 172.21.15.114:0/1707115762 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc004e60 con 0x7f8cf4103fd0 2024-09-13T18:23:38.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.881+0000 7f8cfa525640 1 -- 172.21.15.114:0/1707115762 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cf41a31f0 con 0x7f8cf4103fd0 2024-09-13T18:23:38.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.882+0000 7f8cfa525640 1 -- 172.21.15.114:0/1707115762 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cf4100350 con 0x7f8cf4103fd0 2024-09-13T18:23:38.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.884+0000 7f8cf1ffb640 1 -- 172.21.15.114:0/1707115762 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) v1 ==== 136107+0+0 (secure 0 0 0) 0x7f8cdc005000 con 0x7f8cf4103fd0 2024-09-13T18:23:38.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.884+0000 7f8cf1ffb640 1 --2- 172.21.15.114:0/1707115762 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f8ce00a1320 0x7f8ce00a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:38.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.885+0000 7f8cf3fff640 1 --2- 172.21.15.114:0/1707115762 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f8ce00a1320 0x7f8ce00a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:38.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.885+0000 7f8cf1ffb640 1 -- 172.21.15.114:0/1707115762 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f8cdc014070 con 0x7f8cf4103fd0 2024-09-13T18:23:38.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.885+0000 7f8cf3fff640 1 --2- 172.21.15.114:0/1707115762 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f8ce00a1320 0x7f8ce00a37e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8ce40057c0 tx=0x7f8ce4005a20 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:38.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:38.888+0000 7f8cf1ffb640 1 -- 172.21.15.114:0/1707115762 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f8cdc096720 con 0x7f8cf4103fd0 2024-09-13T18:23:39.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.084+0000 7f8cfa525640 1 -- 172.21.15.114:0/1707115762 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8cf4107220 con 0x7f8cf4103fd0 2024-09-13T18:23:39.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.085+0000 7f8cf1ffb640 1 -- 172.21.15.114:0/1707115762 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+10930 (secure 0 0 0) 0x7f8cdc099d40 con 0x7f8cf4103fd0 2024-09-13T18:23:39.087 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:23:39.087 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4162817497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4162817497},{"type":"v1","addr":"172.21.15.114:6835","nonce":4162817497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/2435428478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":2435428478},{"type":"v1","addr":"172.21.15.114:6837","nonce":2435428478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T18:20:01.895117+0000","modified":"2024-09-13T18:22:29.064231+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24482,"mds_2":24475,"mds_3":24458,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/2046516451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":2046516451},{"type":"v1","addr":"172.21.15.114:6839","nonce":2046516451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":10,"addr":"172.21.15.114:6841/1956923446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":1956923446},{"type":"v1","addr":"172.21.15.114:6841","nonce":1956923446}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24391":{"gid":24391,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/507301496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":507301496},{"type":"v1","addr":"172.21.15.204:6833","nonce":507301496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4028657219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4028657219},{"type":"v1","addr":"172.21.15.134:6833","nonce":4028657219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/57029119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":57029119},{"type":"v1","addr":"172.21.15.204:6835","nonce":57029119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3068880939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3068880939},{"type":"v1","addr":"172.21.15.204:6837","nonce":3068880939}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"e","rank":3,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.134:6835/1150003275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1150003275},{"type":"v1","addr":"172.21.15.134:6835","nonce":1150003275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/344883912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":344883912},{"type":"v1","addr":"172.21.15.204:6839","nonce":344883912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6837/891692210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":891692210},{"type":"v1","addr":"172.21.15.134:6837","nonce":891692210}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6839/3466309724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3466309724},{"type":"v1","addr":"172.21.15.134:6839","nonce":3466309724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:23:39.088 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 30 2024-09-13T18:23:39.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.087+0000 7f8cfa525640 1 -- 172.21.15.114:0/1707115762 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f8ce00a1320 msgr2=0x7f8ce00a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:39.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.087+0000 7f8cfa525640 1 --2- 172.21.15.114:0/1707115762 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f8ce00a1320 0x7f8ce00a37e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8ce40057c0 tx=0x7f8ce4005a20 comp rx=0 tx=0).stop 2024-09-13T18:23:39.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.087+0000 7f8cfa525640 1 -- 172.21.15.114:0/1707115762 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8cf4103fd0 msgr2=0x7f8cf4197550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:39.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.087+0000 7f8cfa525640 1 --2- 172.21.15.114:0/1707115762 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8cf4103fd0 0x7f8cf4197550 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc00e9f0 tx=0x7f8cdc00eeb0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.087+0000 7f8cfa525640 1 -- 172.21.15.114:0/1707115762 shutdown_connections 2024-09-13T18:23:39.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.087+0000 7f8cfa525640 1 --2- 172.21.15.114:0/1707115762 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f8ce00a1320 0x7f8ce00a37e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.087+0000 7f8cfa525640 1 --2- 172.21.15.114:0/1707115762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cf4197a90 0x7f8cf41a2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.087+0000 7f8cfa525640 1 --2- 172.21.15.114:0/1707115762 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8cf4103fd0 0x7f8cf4197550 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.087+0000 7f8cfa525640 1 --2- 172.21.15.114:0/1707115762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8cf40fee80 0x7f8cf4197010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.087+0000 7f8cfa525640 1 -- 172.21.15.114:0/1707115762 >> 172.21.15.114:0/1707115762 conn(0x7f8cf4075c60 msgr2=0x7f8cf41023d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:39.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.087+0000 7f8cfa525640 1 -- 172.21.15.114:0/1707115762 shutdown_connections 2024-09-13T18:23:39.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.087+0000 7f8cfa525640 1 -- 172.21.15.114:0/1707115762 wait complete. 2024-09-13T18:23:39.101 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-09-13T18:23:39.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.201+0000 7fc7f5b0f640 1 -- 172.21.15.114:0/1307268301 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7f0100940 msgr2=0x7fc7f0104980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:39.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.201+0000 7fc7f5b0f640 1 --2- 172.21.15.114:0/1307268301 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7f0100940 0x7fc7f0104980 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8009930 tx=0x7fc7d802a420 comp rx=0 tx=0).stop 2024-09-13T18:23:39.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.201+0000 7fc7f5b0f640 1 -- 172.21.15.114:0/1307268301 shutdown_connections 2024-09-13T18:23:39.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.201+0000 7fc7f5b0f640 1 --2- 172.21.15.114:0/1307268301 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7f01050e0 0x7fc7f010cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.201+0000 7fc7f5b0f640 1 --2- 172.21.15.114:0/1307268301 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7f0100940 0x7fc7f0104980 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.201+0000 7fc7f5b0f640 1 --2- 172.21.15.114:0/1307268301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7f00fff90 0x7fc7f0100370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.201+0000 7fc7f5b0f640 1 -- 172.21.15.114:0/1307268301 >> 172.21.15.114:0/1307268301 conn(0x7fc7f00fba20 msgr2=0x7fc7f00fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:39.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.201+0000 7fc7f5b0f640 1 -- 172.21.15.114:0/1307268301 shutdown_connections 2024-09-13T18:23:39.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.201+0000 7fc7f5b0f640 1 -- 172.21.15.114:0/1307268301 wait complete. 2024-09-13T18:23:39.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.201+0000 7fc7f5b0f640 1 Processor -- start 2024-09-13T18:23:39.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.201+0000 7fc7f5b0f640 1 -- start start 2024-09-13T18:23:39.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.202+0000 7fc7f5b0f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7f00fff90 0x7fc7f00727b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.202+0000 7fc7f5b0f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7f01050e0 0x7fc7f0072cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.202+0000 7fc7f5b0f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7f0073230 0x7fc7f01a43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.202+0000 7fc7f5b0f640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f01a49a0 con 0x7fc7f01050e0 2024-09-13T18:23:39.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.202+0000 7fc7f5b0f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f01a4b10 con 0x7fc7f00fff90 2024-09-13T18:23:39.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.202+0000 7fc7f5b0f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f01a4cd0 con 0x7fc7f0073230 2024-09-13T18:23:39.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.202+0000 7fc7eeffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7f01050e0 0x7fc7f0072cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.202+0000 7fc7eeffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7f01050e0 0x7fc7f0072cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51310/0 (socket says 172.21.15.114:51310) 2024-09-13T18:23:39.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.202+0000 7fc7eeffd640 1 -- 172.21.15.114:0/1037710108 learned_addr learned my addr 172.21.15.114:0/1037710108 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:39.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.202+0000 7fc7ef7fe640 1 --2- 172.21.15.114:0/1037710108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7f00fff90 0x7fc7f00727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.202+0000 7fc7effff640 1 --2- 172.21.15.114:0/1037710108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7f0073230 0x7fc7f01a43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.202+0000 7fc7eeffd640 1 -- 172.21.15.114:0/1037710108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7f0073230 msgr2=0x7fc7f01a43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:39.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.202+0000 7fc7eeffd640 1 --2- 172.21.15.114:0/1037710108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7f0073230 0x7fc7f01a43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.202+0000 7fc7eeffd640 1 -- 172.21.15.114:0/1037710108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7f00fff90 msgr2=0x7fc7f00727b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:39.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.202+0000 7fc7eeffd640 1 --2- 172.21.15.114:0/1037710108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7f00fff90 0x7fc7f00727b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.202+0000 7fc7eeffd640 1 -- 172.21.15.114:0/1037710108 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d8009560 con 0x7fc7f01050e0 2024-09-13T18:23:39.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.202+0000 7fc7ef7fe640 1 --2- 172.21.15.114:0/1037710108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7f00fff90 0x7fc7f00727b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:23:39.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.202+0000 7fc7effff640 1 --2- 172.21.15.114:0/1037710108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7f0073230 0x7fc7f01a43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:23:39.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.202+0000 7fc7eeffd640 1 --2- 172.21.15.114:0/1037710108 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7f01050e0 0x7fc7f0072cf0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8005ab0 tx=0x7fc7d8032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:39.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.202+0000 7fc7ecff9640 1 -- 172.21.15.114:0/1037710108 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d80044d0 con 0x7fc7f01050e0 2024-09-13T18:23:39.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.202+0000 7fc7ecff9640 1 -- 172.21.15.114:0/1037710108 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc7d8032e10 con 0x7fc7f01050e0 2024-09-13T18:23:39.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.202+0000 7fc7f5b0f640 1 -- 172.21.15.114:0/1037710108 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7f01a4f50 con 0x7fc7f01050e0 2024-09-13T18:23:39.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.202+0000 7fc7ecff9640 1 -- 172.21.15.114:0/1037710108 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d8003850 con 0x7fc7f01050e0 2024-09-13T18:23:39.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.202+0000 7fc7f5b0f640 1 -- 172.21.15.114:0/1037710108 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7f01a5370 con 0x7fc7f01050e0 2024-09-13T18:23:39.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.205+0000 7fc7ecff9640 1 -- 172.21.15.114:0/1037710108 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) v1 ==== 136107+0+0 (secure 0 0 0) 0x7fc7d8004850 con 0x7fc7f01050e0 2024-09-13T18:23:39.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.205+0000 7fc7f8830740 1 Processor -- start 2024-09-13T18:23:39.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.205+0000 7fc7f8830740 1 -- start start 2024-09-13T18:23:39.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.205+0000 7fc7f8830740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c88376790 0x557c88376b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.205+0000 7fc7f8830740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c883770b0 0x557c88381410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.205+0000 7fc7f8830740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c88381950 0x557c88383d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.205+0000 7fc7ecff9640 1 --2- 172.21.15.114:0/1037710108 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc7dc0a1350 0x7fc7dc0a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.205+0000 7fc7f8830740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c882e0cc0 con 0x557c883770b0 2024-09-13T18:23:39.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.205+0000 7fc7f8830740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c882de450 con 0x557c88381950 2024-09-13T18:23:39.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.205+0000 7fc7f8830740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c882d9d40 con 0x557c88376790 2024-09-13T18:23:39.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.205+0000 7fc7ef7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c883770b0 0x557c88381410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.206+0000 7fc7effff640 1 --2- 172.21.15.114:0/1037710108 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc7dc0a1350 0x7fc7dc0a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.206+0000 7fc7ef7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c883770b0 0x557c88381410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51316/0 (socket says 172.21.15.114:51316) 2024-09-13T18:23:39.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.206+0000 7fc7ef7fe640 1 -- 172.21.15.114:0/952998462 learned_addr learned my addr 172.21.15.114:0/952998462 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:39.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.206+0000 7fc7ecff9640 1 -- 172.21.15.114:0/1037710108 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fc7d80ede70 con 0x7fc7f01050e0 2024-09-13T18:23:39.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.206+0000 7fc7effff640 1 --2- 172.21.15.114:0/952998462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c88376790 0x557c88376b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.206+0000 7fc7eeffd640 1 --2- 172.21.15.114:0/952998462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c88381950 0x557c88383d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.206+0000 7fc7ef7fe640 1 -- 172.21.15.114:0/952998462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c88376790 msgr2=0x557c88376b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:39.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.206+0000 7fc7ef7fe640 1 --2- 172.21.15.114:0/952998462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c88376790 0x557c88376b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.206+0000 7fc7ef7fe640 1 -- 172.21.15.114:0/952998462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c88381950 msgr2=0x557c88383d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:39.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.206+0000 7fc7ef7fe640 1 --2- 172.21.15.114:0/952998462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c88381950 0x557c88383d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.206+0000 7fc7ef7fe640 1 -- 172.21.15.114:0/952998462 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c882a1620 con 0x557c883770b0 2024-09-13T18:23:39.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.206+0000 7fc7effff640 1 --2- 172.21.15.114:0/952998462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c88376790 0x557c88376b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:23:39.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.206+0000 7fc7eeffd640 1 --2- 172.21.15.114:0/952998462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c88381950 0x557c88383d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:23:39.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.206+0000 7fc7ef7fe640 1 --2- 172.21.15.114:0/952998462 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c883770b0 0x557c88381410 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fc7e000f3e0 tx=0x7fc7e000f8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17686cff33f69a05 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:39.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.206+0000 7fc7effff640 1 --2- 172.21.15.114:0/1037710108 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc7dc0a1350 0x7fc7dc0a3810 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8009460 tx=0x7fc7e8013830 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:39.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.206+0000 7fc7e5ffb640 1 -- 172.21.15.114:0/952998462 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e0004d20 con 0x557c883770b0 2024-09-13T18:23:39.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.206+0000 7fc7e5ffb640 1 -- 172.21.15.114:0/952998462 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc7e0004e80 con 0x557c883770b0 2024-09-13T18:23:39.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.206+0000 7fc7e5ffb640 1 -- 172.21.15.114:0/952998462 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e0005360 con 0x557c883770b0 2024-09-13T18:23:39.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.207+0000 7fc7f8830740 1 -- 172.21.15.114:0/952998462 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c883770b0 msgr2=0x557c88381410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:39.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.207+0000 7fc7f8830740 1 --2- 172.21.15.114:0/952998462 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c883770b0 0x557c88381410 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fc7e000f3e0 tx=0x7fc7e000f8a0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.207+0000 7fc7f8830740 1 -- 172.21.15.114:0/952998462 shutdown_connections 2024-09-13T18:23:39.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.207+0000 7fc7f8830740 1 --2- 172.21.15.114:0/952998462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c88381950 0x557c88383d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.207+0000 7fc7f8830740 1 --2- 172.21.15.114:0/952998462 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c883770b0 0x557c88381410 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.207+0000 7fc7f8830740 1 --2- 172.21.15.114:0/952998462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c88376790 0x557c88376b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.207+0000 7fc7f8830740 1 -- 172.21.15.114:0/952998462 >> 172.21.15.114:0/952998462 conn(0x557c88372d00 msgr2=0x557c883730e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:39.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.207+0000 7fc7f8830740 1 -- 172.21.15.114:0/952998462 shutdown_connections 2024-09-13T18:23:39.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.207+0000 7fc7f8830740 1 -- 172.21.15.114:0/952998462 wait complete. 2024-09-13T18:23:39.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.207+0000 7fc7f8830740 1 -- start start 2024-09-13T18:23:39.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.208+0000 7fc7f8830740 1 Processor -- start 2024-09-13T18:23:39.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.208+0000 7fc7f8830740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c88376790 0x557c88380eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.208+0000 7fc7f8830740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c883770b0 0x557c883813f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.208+0000 7fc7f8830740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c88381950 0x557c8837e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.208+0000 7fc7f8830740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c88298180 con 0x557c88381950 2024-09-13T18:23:39.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.208+0000 7fc7f8830740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c87fb2430 con 0x557c88376790 2024-09-13T18:23:39.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.208+0000 7fc7f8830740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c8825d5e0 con 0x557c883770b0 2024-09-13T18:23:39.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.208+0000 7fc7f8830740 10 client.0 _ll_register_callbacks cb 0x557c882c9aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:23:39.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.208+0000 7fc7effff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c88381950 0x557c8837e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.208+0000 7fc7effff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c88381950 0x557c8837e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51318/0 (socket says 172.21.15.114:51318) 2024-09-13T18:23:39.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.208+0000 7fc7effff640 1 -- 172.21.15.114:0/280103242 learned_addr learned my addr 172.21.15.114:0/280103242 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:39.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.208+0000 7fc7eeffd640 1 --2- 172.21.15.114:0/280103242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c883770b0 0x557c883813f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.208+0000 7fc7ef7fe640 1 --2- 172.21.15.114:0/280103242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c88376790 0x557c88380eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.208+0000 7fc7effff640 1 -- 172.21.15.114:0/280103242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c883770b0 msgr2=0x557c883813f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:39.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.208+0000 7fc7effff640 1 --2- 172.21.15.114:0/280103242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c883770b0 0x557c883813f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.208+0000 7fc7effff640 1 -- 172.21.15.114:0/280103242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c88376790 msgr2=0x557c88380eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:39.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.208+0000 7fc7effff640 1 --2- 172.21.15.114:0/280103242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c88376790 0x557c88380eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.208+0000 7fc7effff640 1 -- 172.21.15.114:0/280103242 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7e0009ce0 con 0x557c88381950 2024-09-13T18:23:39.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.208+0000 7fc7effff640 1 --2- 172.21.15.114:0/280103242 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c88381950 0x557c8837e420 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8003840 tx=0x7fc7e8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3290292662ecce0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:39.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.208+0000 7fc7ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:23:39.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.208+0000 7fc7ceffd640 1 -- 172.21.15.114:0/280103242 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e800fb50 con 0x557c88381950 2024-09-13T18:23:39.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.209+0000 7fc7ceffd640 1 -- 172.21.15.114:0/280103242 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fc7e800fcb0 con 0x557c88381950 2024-09-13T18:23:39.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.209+0000 7fc7f8830740 1 -- 172.21.15.114:0/280103242 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557c882cc1f0 con 0x557c88381950 2024-09-13T18:23:39.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.209+0000 7fc7ceffd640 1 -- 172.21.15.114:0/280103242 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7e8003e50 con 0x557c88381950 2024-09-13T18:23:39.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.209+0000 7fc7f8830740 10 client.16080 fetch_fsmap learned FSMap version 30 2024-09-13T18:23:39.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.209+0000 7fc7f8830740 1 -- 172.21.15.114:0/280103242 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x557c882d83c0 con 0x557c88381950 2024-09-13T18:23:39.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.209+0000 7fc7ceffd640 1 -- 172.21.15.114:0/280103242 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557c882cc1f0 con 0x557c88381950 2024-09-13T18:23:39.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.209+0000 7fc7ceffd640 1 -- 172.21.15.114:0/280103242 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc7e80113e0 con 0x557c88381950 2024-09-13T18:23:39.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.209+0000 7fc7f8830740 10 client.16080 fetch_fsmap finished waiting for FSMap version 30 2024-09-13T18:23:39.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.209+0000 7fc7f8830740 10 client.16080 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-13T18:23:39.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.209+0000 7fc7f8830740 20 client.16080 populate_metadata read hostname 'smithi114' 2024-09-13T18:23:39.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.209+0000 7fc7f8830740 1 --2- 172.21.15.114:0/280103242 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x557c88391100 0x557c88393520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.209+0000 7fc7f8830740 4 client.16080 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:23:39.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.209+0000 7fc7f8830740 1 -- 172.21.15.114:0/280103242 --> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557c87fcd4e0 con 0x557c88391100 2024-09-13T18:23:39.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.209+0000 7fc7ef7fe640 1 --2- 172.21.15.114:0/280103242 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x557c88391100 0x557c88393520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.210+0000 7fc7ceffd640 10 client.16080 ms_handle_connect on v2:172.21.15.114:6840/1956923446 2024-09-13T18:23:39.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.210+0000 7fc7ef7fe640 1 --2- 172.21.15.114:0/280103242 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x557c88391100 0x557c88393520 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7d704da43a3367f7 server_cookie=21d117d2c5a24055 in_seq=0 out_seq=0 2024-09-13T18:23:39.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.210+0000 7fc7ceffd640 1 -- 172.21.15.114:0/280103242 <== mds.4 v2:172.21.15.114:6840/1956923446 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x557c87fcd4e0 con 0x557c88391100 2024-09-13T18:23:39.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.210+0000 7fc7ceffd640 10 client.16080 handle_command_reply: tid=0 2024-09-13T18:23:39.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.210+0000 7fc7f8830740 1 client.16080 shutdown 2024-09-13T18:23:39.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.210+0000 7fc7f8830740 1 -- 172.21.15.114:0/280103242 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c88381950 msgr2=0x557c8837e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:39.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.210+0000 7fc7f8830740 1 --2- 172.21.15.114:0/280103242 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c88381950 0x557c8837e420 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8003840 tx=0x7fc7e8005f90 comp rx=0 tx=0).stop 2024-09-13T18:23:39.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.210+0000 7fc7f8830740 1 -- 172.21.15.114:0/280103242 shutdown_connections 2024-09-13T18:23:39.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.210+0000 7fc7f8830740 1 --2- 172.21.15.114:0/280103242 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x557c88391100 0x557c88393520 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.210+0000 7fc7f8830740 1 --2- 172.21.15.114:0/280103242 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c88381950 0x557c8837e420 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.210+0000 7fc7f8830740 1 --2- 172.21.15.114:0/280103242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c883770b0 0x557c883813f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.210+0000 7fc7ceffd640 0 client.16080 ms_handle_reset on v2:172.21.15.114:6840/1956923446 2024-09-13T18:23:39.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.210+0000 7fc7f8830740 1 --2- 172.21.15.114:0/280103242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c88376790 0x557c88380eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.210+0000 7fc7f8830740 1 -- 172.21.15.114:0/280103242 >> 172.21.15.114:0/280103242 conn(0x557c88372d00 msgr2=0x557c88378a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:39.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.210+0000 7fc7f8830740 1 -- 172.21.15.114:0/280103242 shutdown_connections 2024-09-13T18:23:39.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.211+0000 7fc7f8830740 1 -- 172.21.15.114:0/280103242 wait complete. 2024-09-13T18:23:39.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.211+0000 7fc7f8830740 20 client.16080 trim_cache size 0 max 16384 2024-09-13T18:23:39.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.222+0000 7fc7f8830740 1 Processor -- start 2024-09-13T18:23:39.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.222+0000 7fc7f8830740 1 -- start start 2024-09-13T18:23:39.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.222+0000 7fc7f8830740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c88376790 0x557c88388620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.222+0000 7fc7f8830740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c883770b0 0x557c88388b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.222+0000 7fc7f8830740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c88381950 0x557c88379330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.222+0000 7fc7f8830740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c8837ea20 con 0x557c88381950 2024-09-13T18:23:39.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.222+0000 7fc7f8830740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c883765f0 con 0x557c883770b0 2024-09-13T18:23:39.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.222+0000 7fc7f8830740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c88377750 con 0x557c88376790 2024-09-13T18:23:39.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.222+0000 7fc7effff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c88381950 0x557c88379330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.222+0000 7fc7effff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c88381950 0x557c88379330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51324/0 (socket says 172.21.15.114:51324) 2024-09-13T18:23:39.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.222+0000 7fc7effff640 1 -- 172.21.15.114:0/4234388864 learned_addr learned my addr 172.21.15.114:0/4234388864 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:39.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.222+0000 7fc7ef7fe640 1 --2- 172.21.15.114:0/4234388864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c88376790 0x557c88388620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.222+0000 7fc7eeffd640 1 --2- 172.21.15.114:0/4234388864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c883770b0 0x557c88388b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.222+0000 7fc7effff640 1 -- 172.21.15.114:0/4234388864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c88376790 msgr2=0x557c88388620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:39.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.222+0000 7fc7effff640 1 --2- 172.21.15.114:0/4234388864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c88376790 0x557c88388620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.222+0000 7fc7effff640 1 -- 172.21.15.114:0/4234388864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c883770b0 msgr2=0x557c88388b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:39.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.222+0000 7fc7effff640 1 --2- 172.21.15.114:0/4234388864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c883770b0 0x557c88388b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.222+0000 7fc7effff640 1 -- 172.21.15.114:0/4234388864 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c8827a770 con 0x557c88381950 2024-09-13T18:23:39.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.222+0000 7fc7ef7fe640 1 --2- 172.21.15.114:0/4234388864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c88376790 0x557c88388620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:23:39.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.223+0000 7fc7effff640 1 --2- 172.21.15.114:0/4234388864 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c88381950 0x557c88379330 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8015010 tx=0x7fc7e8009a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2357a52a10dae3b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:39.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.223+0000 7fc7e67fc640 1 -- 172.21.15.114:0/4234388864 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e8003e50 con 0x557c88381950 2024-09-13T18:23:39.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.223+0000 7fc7e67fc640 1 -- 172.21.15.114:0/4234388864 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc7e8011a60 con 0x557c88381950 2024-09-13T18:23:39.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.223+0000 7fc7e67fc640 1 -- 172.21.15.114:0/4234388864 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e800f7a0 con 0x557c88381950 2024-09-13T18:23:39.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.224+0000 7fc7f8830740 1 -- 172.21.15.114:0/4234388864 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c88381950 msgr2=0x557c88379330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:39.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.224+0000 7fc7f8830740 1 --2- 172.21.15.114:0/4234388864 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c88381950 0x557c88379330 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8015010 tx=0x7fc7e8009a90 comp rx=0 tx=0).stop 2024-09-13T18:23:39.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.224+0000 7fc7f8830740 1 -- 172.21.15.114:0/4234388864 shutdown_connections 2024-09-13T18:23:39.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.224+0000 7fc7f8830740 1 --2- 172.21.15.114:0/4234388864 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c88381950 0x557c88379330 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.224+0000 7fc7f8830740 1 --2- 172.21.15.114:0/4234388864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c883770b0 0x557c88388b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.224+0000 7fc7f8830740 1 --2- 172.21.15.114:0/4234388864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c88376790 0x557c88388620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.224+0000 7fc7f8830740 1 -- 172.21.15.114:0/4234388864 >> 172.21.15.114:0/4234388864 conn(0x557c88372d00 msgr2=0x557c88397ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:39.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.224+0000 7fc7f8830740 1 -- 172.21.15.114:0/4234388864 shutdown_connections 2024-09-13T18:23:39.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.224+0000 7fc7f8830740 1 -- 172.21.15.114:0/4234388864 wait complete. 2024-09-13T18:23:39.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.224+0000 7fc7f8830740 1 -- start start 2024-09-13T18:23:39.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.224+0000 7fc7f8830740 1 Processor -- start 2024-09-13T18:23:39.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.225+0000 7fc7f8830740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c88376790 0x557c88387360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.225+0000 7fc7f8830740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c883770b0 0x557c883878a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.225+0000 7fc7f8830740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c88381950 0x557c88387de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.225+0000 7fc7f8830740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c883effb0 con 0x557c88376790 2024-09-13T18:23:39.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.225+0000 7fc7f8830740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c883f0120 con 0x557c883770b0 2024-09-13T18:23:39.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.225+0000 7fc7f8830740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c883f02e0 con 0x557c88381950 2024-09-13T18:23:39.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.225+0000 7fc7f8830740 10 client.0 _ll_register_callbacks cb 0x7fc7e0009840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:23:39.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.225+0000 7fc7ef7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c88376790 0x557c88387360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.225+0000 7fc7ef7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c88376790 0x557c88387360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51340/0 (socket says 172.21.15.114:51340) 2024-09-13T18:23:39.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.225+0000 7fc7ef7fe640 1 -- 172.21.15.114:0/733782102 learned_addr learned my addr 172.21.15.114:0/733782102 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:39.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.225+0000 7fc7eeffd640 1 --2- 172.21.15.114:0/733782102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c883770b0 0x557c883878a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.225+0000 7fc7effff640 1 --2- 172.21.15.114:0/733782102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c88381950 0x557c88387de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.225+0000 7fc7ef7fe640 1 -- 172.21.15.114:0/733782102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c88381950 msgr2=0x557c88387de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:39.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.225+0000 7fc7ef7fe640 1 --2- 172.21.15.114:0/733782102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c88381950 0x557c88387de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.225+0000 7fc7ef7fe640 1 -- 172.21.15.114:0/733782102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c883770b0 msgr2=0x557c883878a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:39.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.225+0000 7fc7ef7fe640 1 --2- 172.21.15.114:0/733782102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c883770b0 0x557c883878a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.225+0000 7fc7ef7fe640 1 -- 172.21.15.114:0/733782102 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557c882d83c0 con 0x557c88376790 2024-09-13T18:23:39.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.225+0000 7fc7eeffd640 1 --2- 172.21.15.114:0/733782102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c883770b0 0x557c883878a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:23:39.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.225+0000 7fc7ef7fe640 1 --2- 172.21.15.114:0/733782102 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c88376790 0x557c88387360 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fc7e000f3e0 tx=0x7fc7e0011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4db2d391f7f63480 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:39.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.225+0000 7fc7ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:23:39.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.225+0000 7fc7ceffd640 1 -- 172.21.15.114:0/733782102 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e0026630 con 0x557c88376790 2024-09-13T18:23:39.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.225+0000 7fc7ceffd640 1 -- 172.21.15.114:0/733782102 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fc7e0026790 con 0x557c88376790 2024-09-13T18:23:39.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.225+0000 7fc7f8830740 1 -- 172.21.15.114:0/733782102 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557c88303b80 con 0x557c88376790 2024-09-13T18:23:39.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.226+0000 7fc7ceffd640 1 -- 172.21.15.114:0/733782102 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7e0003970 con 0x557c88376790 2024-09-13T18:23:39.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.226+0000 7fc7f8830740 10 client.16086 fetch_fsmap learned FSMap version 30 2024-09-13T18:23:39.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.226+0000 7fc7f8830740 1 -- 172.21.15.114:0/733782102 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x557c8829fe80 con 0x557c88376790 2024-09-13T18:23:39.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.226+0000 7fc7ceffd640 1 -- 172.21.15.114:0/733782102 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557c88303b80 con 0x557c88376790 2024-09-13T18:23:39.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.226+0000 7fc7ceffd640 1 -- 172.21.15.114:0/733782102 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc7e0004040 con 0x557c88376790 2024-09-13T18:23:39.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.226+0000 7fc7f8830740 10 client.16086 fetch_fsmap finished waiting for FSMap version 30 2024-09-13T18:23:39.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.226+0000 7fc7f8830740 10 client.16086 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-13T18:23:39.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.226+0000 7fc7f8830740 20 client.16086 populate_metadata read hostname 'smithi114' 2024-09-13T18:23:39.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.226+0000 7fc7f8830740 1 --2- 172.21.15.114:0/733782102 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x557c883f1ac0 0x557c883f3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.226+0000 7fc7f8830740 4 client.16086 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:23:39.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.226+0000 7fc7f8830740 1 -- 172.21.15.114:0/733782102 --> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557c87fde7e0 con 0x557c883f1ac0 2024-09-13T18:23:39.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.226+0000 7fc7eeffd640 1 --2- 172.21.15.114:0/733782102 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x557c883f1ac0 0x557c883f3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.226+0000 7fc7ceffd640 10 client.16086 ms_handle_connect on v2:172.21.15.114:6840/1956923446 2024-09-13T18:23:39.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.226+0000 7fc7eeffd640 1 --2- 172.21.15.114:0/733782102 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x557c883f1ac0 0x557c883f3ee0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b39b32abd5cd6f24 server_cookie=97fd24bbf8fb39ca in_seq=0 out_seq=0 2024-09-13T18:23:39.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.227+0000 7fc7ceffd640 1 -- 172.21.15.114:0/733782102 <== mds.4 v2:172.21.15.114:6840/1956923446 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557c87fde7e0 con 0x557c883f1ac0 2024-09-13T18:23:39.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.227+0000 7fc7ceffd640 10 client.16086 handle_command_reply: tid=0 2024-09-13T18:23:39.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.227+0000 7fc7f8830740 1 client.16086 shutdown 2024-09-13T18:23:39.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.227+0000 7fc7f8830740 1 -- 172.21.15.114:0/733782102 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c88376790 msgr2=0x557c88387360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:39.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.227+0000 7fc7f8830740 1 --2- 172.21.15.114:0/733782102 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c88376790 0x557c88387360 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fc7e000f3e0 tx=0x7fc7e0011e50 comp rx=0 tx=0).stop 2024-09-13T18:23:39.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.227+0000 7fc7f8830740 1 -- 172.21.15.114:0/733782102 shutdown_connections 2024-09-13T18:23:39.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.227+0000 7fc7f8830740 1 --2- 172.21.15.114:0/733782102 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x557c883f1ac0 0x557c883f3ee0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.227+0000 7fc7f8830740 1 --2- 172.21.15.114:0/733782102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c88381950 0x557c88387de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.227+0000 7fc7f8830740 1 --2- 172.21.15.114:0/733782102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c883770b0 0x557c883878a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.227+0000 7fc7ceffd640 0 client.16086 ms_handle_reset on v2:172.21.15.114:6840/1956923446 2024-09-13T18:23:39.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.227+0000 7fc7f8830740 1 --2- 172.21.15.114:0/733782102 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c88376790 0x557c88387360 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.227+0000 7fc7f8830740 1 -- 172.21.15.114:0/733782102 >> 172.21.15.114:0/733782102 conn(0x557c88372d00 msgr2=0x557c88379330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:39.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.227+0000 7fc7f8830740 1 -- 172.21.15.114:0/733782102 shutdown_connections 2024-09-13T18:23:39.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.227+0000 7fc7f8830740 1 -- 172.21.15.114:0/733782102 wait complete. 2024-09-13T18:23:39.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.227+0000 7fc7f8830740 20 client.16086 trim_cache size 0 max 16384 2024-09-13T18:23:39.241 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:23:39.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.228+0000 7fc7f5b0f640 1 -- 172.21.15.114:0/1037710108 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc7dc0a1350 msgr2=0x7fc7dc0a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:39.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.228+0000 7fc7f5b0f640 1 --2- 172.21.15.114:0/1037710108 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc7dc0a1350 0x7fc7dc0a3810 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8009460 tx=0x7fc7e8013830 comp rx=0 tx=0).stop 2024-09-13T18:23:39.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.228+0000 7fc7f5b0f640 1 -- 172.21.15.114:0/1037710108 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7f01050e0 msgr2=0x7fc7f0072cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:39.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.228+0000 7fc7f5b0f640 1 --2- 172.21.15.114:0/1037710108 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7f01050e0 0x7fc7f0072cf0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8005ab0 tx=0x7fc7d8032c20 comp rx=0 tx=0).stop 2024-09-13T18:23:39.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.228+0000 7fc7f5b0f640 1 -- 172.21.15.114:0/1037710108 shutdown_connections 2024-09-13T18:23:39.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.228+0000 7fc7f5b0f640 1 --2- 172.21.15.114:0/1037710108 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc7dc0a1350 0x7fc7dc0a3810 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.228+0000 7fc7f5b0f640 1 --2- 172.21.15.114:0/1037710108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7f0073230 0x7fc7f01a43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.228+0000 7fc7f5b0f640 1 --2- 172.21.15.114:0/1037710108 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7f01050e0 0x7fc7f0072cf0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.228+0000 7fc7f5b0f640 1 --2- 172.21.15.114:0/1037710108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7f00fff90 0x7fc7f00727b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.228+0000 7fc7f5b0f640 1 -- 172.21.15.114:0/1037710108 >> 172.21.15.114:0/1037710108 conn(0x7fc7f00fba20 msgr2=0x7fc7f00fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:39.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.228+0000 7fc7f5b0f640 1 -- 172.21.15.114:0/1037710108 shutdown_connections 2024-09-13T18:23:39.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.228+0000 7fc7f5b0f640 1 -- 172.21.15.114:0/1037710108 wait complete. 2024-09-13T18:23:39.244 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-09-13T18:23:39.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.346+0000 7fcfbb3f4640 1 -- 172.21.15.114:0/3869366928 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcfb410a340 msgr2=0x7fcfb410c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:39.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.346+0000 7fcfbb3f4640 1 --2- 172.21.15.114:0/3869366928 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcfb410a340 0x7fcfb410c7b0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fcfac009900 tx=0x7fcfac02a480 comp rx=0 tx=0).stop 2024-09-13T18:23:39.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.346+0000 7fcfbb3f4640 1 -- 172.21.15.114:0/3869366928 shutdown_connections 2024-09-13T18:23:39.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.346+0000 7fcfbb3f4640 1 --2- 172.21.15.114:0/3869366928 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcfb410a340 0x7fcfb410c7b0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.346+0000 7fcfbb3f4640 1 --2- 172.21.15.114:0/3869366928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcfb4101890 0x7fcfb4109d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.346+0000 7fcfbb3f4640 1 --2- 172.21.15.114:0/3869366928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcfb4100f70 0x7fcfb4101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.346+0000 7fcfbb3f4640 1 -- 172.21.15.114:0/3869366928 >> 172.21.15.114:0/3869366928 conn(0x7fcfb40fa900 msgr2=0x7fcfb40fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:39.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.346+0000 7fcfbb3f4640 1 -- 172.21.15.114:0/3869366928 shutdown_connections 2024-09-13T18:23:39.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.346+0000 7fcfbb3f4640 1 -- 172.21.15.114:0/3869366928 wait complete. 2024-09-13T18:23:39.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.347+0000 7fcfbb3f4640 1 Processor -- start 2024-09-13T18:23:39.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.347+0000 7fcfbb3f4640 1 -- start start 2024-09-13T18:23:39.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.347+0000 7fcfbb3f4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcfb4100f70 0x7fcfb41961c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.347+0000 7fcfbb3f4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcfb4101890 0x7fcfb4196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.347+0000 7fcfbb3f4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcfb410a340 0x7fcfb419aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.347+0000 7fcfbb3f4640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb419b0f0 con 0x7fcfb4101890 2024-09-13T18:23:39.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.347+0000 7fcfbb3f4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb419b260 con 0x7fcfb4100f70 2024-09-13T18:23:39.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.347+0000 7fcfbb3f4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb419b420 con 0x7fcfb410a340 2024-09-13T18:23:39.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.347+0000 7fcfb898f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcfb4101890 0x7fcfb4196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.347+0000 7fcfb898f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcfb4101890 0x7fcfb4196700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51366/0 (socket says 172.21.15.114:51366) 2024-09-13T18:23:39.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.347+0000 7fcfb898f640 1 -- 172.21.15.114:0/2266518600 learned_addr learned my addr 172.21.15.114:0/2266518600 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:39.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.347+0000 7fcfb9190640 1 --2- 172.21.15.114:0/2266518600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcfb4100f70 0x7fcfb41961c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.347+0000 7fcfb9991640 1 --2- 172.21.15.114:0/2266518600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcfb410a340 0x7fcfb419aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.348+0000 7fcfb898f640 1 -- 172.21.15.114:0/2266518600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcfb410a340 msgr2=0x7fcfb419aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:39.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.348+0000 7fcfb898f640 1 --2- 172.21.15.114:0/2266518600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcfb410a340 0x7fcfb419aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.348+0000 7fcfb898f640 1 -- 172.21.15.114:0/2266518600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcfb4100f70 msgr2=0x7fcfb41961c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:39.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.348+0000 7fcfb898f640 1 --2- 172.21.15.114:0/2266518600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcfb4100f70 0x7fcfb41961c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.348+0000 7fcfb898f640 1 -- 172.21.15.114:0/2266518600 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfac009560 con 0x7fcfb4101890 2024-09-13T18:23:39.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.348+0000 7fcfb9991640 1 --2- 172.21.15.114:0/2266518600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcfb410a340 0x7fcfb419aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:23:39.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.348+0000 7fcfb898f640 1 --2- 172.21.15.114:0/2266518600 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcfb4101890 0x7fcfb4196700 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fcfb000eab0 tx=0x7fcfb000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:39.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.348+0000 7fcfaa7fc640 1 -- 172.21.15.114:0/2266518600 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb000ce40 con 0x7fcfb4101890 2024-09-13T18:23:39.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.348+0000 7fcfaa7fc640 1 -- 172.21.15.114:0/2266518600 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcfb0011900 con 0x7fcfb4101890 2024-09-13T18:23:39.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.348+0000 7fcfbb3f4640 1 -- 172.21.15.114:0/2266518600 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfb419b700 con 0x7fcfb4101890 2024-09-13T18:23:39.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.348+0000 7fcfaa7fc640 1 -- 172.21.15.114:0/2266518600 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb00108c0 con 0x7fcfb4101890 2024-09-13T18:23:39.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.348+0000 7fcfbb3f4640 1 -- 172.21.15.114:0/2266518600 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfb41a2f80 con 0x7fcfb4101890 2024-09-13T18:23:39.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.350+0000 7fcfaa7fc640 1 -- 172.21.15.114:0/2266518600 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) v1 ==== 136107+0+0 (secure 0 0 0) 0x7fcfb0011470 con 0x7fcfb4101890 2024-09-13T18:23:39.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.351+0000 7fcfbe1f1740 1 Processor -- start 2024-09-13T18:23:39.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.351+0000 7fcfbe1f1740 1 -- start start 2024-09-13T18:23:39.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.351+0000 7fcfbe1f1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625feb6b790 0x5625feb6bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.351+0000 7fcfaa7fc640 1 --2- 172.21.15.114:0/2266518600 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fcfa00a1320 0x7fcfa00a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.351+0000 7fcfbe1f1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625feb6c0b0 0x5625feb76410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.351+0000 7fcfbe1f1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625feb76950 0x5625feb78d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.351+0000 7fcfbe1f1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625fead5cc0 con 0x5625feb6b790 2024-09-13T18:23:39.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.351+0000 7fcfbe1f1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625fead3450 con 0x5625feb76950 2024-09-13T18:23:39.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.351+0000 7fcfbe1f1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625feaced40 con 0x5625feb6c0b0 2024-09-13T18:23:39.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.351+0000 7fcfb9190640 1 --2- 172.21.15.114:0/2266518600 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fcfa00a1320 0x7fcfa00a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.351+0000 7fcfb9991640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625feb6b790 0x5625feb6bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.351+0000 7fcfb9991640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625feb6b790 0x5625feb6bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51376/0 (socket says 172.21.15.114:51376) 2024-09-13T18:23:39.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.351+0000 7fcfb9991640 1 -- 172.21.15.114:0/2215244092 learned_addr learned my addr 172.21.15.114:0/2215244092 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:39.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.351+0000 7fcfaa7fc640 1 -- 172.21.15.114:0/2266518600 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fcfb00cd850 con 0x7fcfb4101890 2024-09-13T18:23:39.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.351+0000 7fcfb898f640 1 --2- 172.21.15.114:0/2215244092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625feb6c0b0 0x5625feb76410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.351+0000 7fcfb9991640 1 --2- 172.21.15.114:0/2215244092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625feb76950 0x5625feb78d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.352+0000 7fcfb898f640 1 -- 172.21.15.114:0/2215244092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625feb76950 msgr2=0x5625feb78d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:39.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.352+0000 7fcfb898f640 1 --2- 172.21.15.114:0/2215244092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625feb76950 0x5625feb78d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.352+0000 7fcfb898f640 1 -- 172.21.15.114:0/2215244092 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625feb6b790 msgr2=0x5625feb6bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:39.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.352+0000 7fcfb898f640 1 --2- 172.21.15.114:0/2215244092 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625feb6b790 0x5625feb6bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.352+0000 7fcfb898f640 1 -- 172.21.15.114:0/2215244092 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625fea96620 con 0x5625feb6c0b0 2024-09-13T18:23:39.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.352+0000 7fcfb9991640 1 --2- 172.21.15.114:0/2215244092 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625feb6b790 0x5625feb6bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:23:39.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.352+0000 7fcfb9190640 1 --2- 172.21.15.114:0/2266518600 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fcfa00a1320 0x7fcfa00a37e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4009bd0 tx=0x7fcfa4005f90 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:39.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.352+0000 7fcfb898f640 1 --2- 172.21.15.114:0/2215244092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625feb6c0b0 0x5625feb76410 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcfb00caad0 tx=0x7fcfb009a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad2dd8720505a5c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:39.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.352+0000 7fcf9f7fe640 1 -- 172.21.15.114:0/2215244092 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb00acc70 con 0x5625feb6c0b0 2024-09-13T18:23:39.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.352+0000 7fcf9f7fe640 1 -- 172.21.15.114:0/2215244092 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcfb00aca90 con 0x5625feb6c0b0 2024-09-13T18:23:39.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.352+0000 7fcf9f7fe640 1 -- 172.21.15.114:0/2215244092 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb009aac0 con 0x5625feb6c0b0 2024-09-13T18:23:39.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.353+0000 7fcfbe1f1740 1 -- 172.21.15.114:0/2215244092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625feb6c0b0 msgr2=0x5625feb76410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:39.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.353+0000 7fcfbe1f1740 1 --2- 172.21.15.114:0/2215244092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625feb6c0b0 0x5625feb76410 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcfb00caad0 tx=0x7fcfb009a250 comp rx=0 tx=0).stop 2024-09-13T18:23:39.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.353+0000 7fcfbe1f1740 1 -- 172.21.15.114:0/2215244092 shutdown_connections 2024-09-13T18:23:39.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.353+0000 7fcfbe1f1740 1 --2- 172.21.15.114:0/2215244092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625feb76950 0x5625feb78d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.353+0000 7fcfbe1f1740 1 --2- 172.21.15.114:0/2215244092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625feb6c0b0 0x5625feb76410 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.353+0000 7fcfbe1f1740 1 --2- 172.21.15.114:0/2215244092 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625feb6b790 0x5625feb6bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.353+0000 7fcfbe1f1740 1 -- 172.21.15.114:0/2215244092 >> 172.21.15.114:0/2215244092 conn(0x5625feb67d00 msgr2=0x5625feb680e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:39.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.353+0000 7fcfbe1f1740 1 -- 172.21.15.114:0/2215244092 shutdown_connections 2024-09-13T18:23:39.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.353+0000 7fcfbe1f1740 1 -- 172.21.15.114:0/2215244092 wait complete. 2024-09-13T18:23:39.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.353+0000 7fcfbe1f1740 1 -- start start 2024-09-13T18:23:39.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.353+0000 7fcfbe1f1740 1 Processor -- start 2024-09-13T18:23:39.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.354+0000 7fcfbe1f1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625feb6b790 0x5625feb75e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.354+0000 7fcfbe1f1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625feb6c0b0 0x5625feb76370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.354+0000 7fcfbe1f1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625feb76950 0x5625feb73270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.354+0000 7fcfbe1f1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625fea8d180 con 0x5625feb6b790 2024-09-13T18:23:39.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.354+0000 7fcfbe1f1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625fe7a7430 con 0x5625feb6c0b0 2024-09-13T18:23:39.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.354+0000 7fcfbe1f1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625fea525e0 con 0x5625feb76950 2024-09-13T18:23:39.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.354+0000 7fcfbe1f1740 10 client.0 _ll_register_callbacks cb 0x5625feabeaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:23:39.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.354+0000 7fcfb9190640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625feb6b790 0x5625feb75e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.354+0000 7fcfb9190640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625feb6b790 0x5625feb75e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51390/0 (socket says 172.21.15.114:51390) 2024-09-13T18:23:39.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.354+0000 7fcfb9190640 1 -- 172.21.15.114:0/3311900820 learned_addr learned my addr 172.21.15.114:0/3311900820 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:39.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.354+0000 7fcfb9991640 1 --2- 172.21.15.114:0/3311900820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625feb76950 0x5625feb73270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.354+0000 7fcfb898f640 1 --2- 172.21.15.114:0/3311900820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625feb6c0b0 0x5625feb76370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.354+0000 7fcfb9190640 1 -- 172.21.15.114:0/3311900820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625feb76950 msgr2=0x5625feb73270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:39.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.354+0000 7fcfb9190640 1 --2- 172.21.15.114:0/3311900820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625feb76950 0x5625feb73270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.354+0000 7fcfb9190640 1 -- 172.21.15.114:0/3311900820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625feb6c0b0 msgr2=0x5625feb76370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:39.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.354+0000 7fcfb9190640 1 --2- 172.21.15.114:0/3311900820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625feb6c0b0 0x5625feb76370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.354+0000 7fcfb9190640 1 -- 172.21.15.114:0/3311900820 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfb41a2f80 con 0x5625feb6b790 2024-09-13T18:23:39.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.354+0000 7fcfb9190640 1 --2- 172.21.15.114:0/3311900820 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625feb6b790 0x5625feb75e30 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fcfa400ecd0 tx=0x7fcfa4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4afacf02046f3016 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:39.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.354+0000 7fcf9cff9640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:23:39.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.354+0000 7fcf9cff9640 1 -- 172.21.15.114:0/3311900820 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa401ccb0 con 0x5625feb6b790 2024-09-13T18:23:39.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.354+0000 7fcf9cff9640 1 -- 172.21.15.114:0/3311900820 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fcfa4023070 con 0x5625feb6b790 2024-09-13T18:23:39.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.354+0000 7fcfbe1f1740 1 -- 172.21.15.114:0/3311900820 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5625feaf8b80 con 0x5625feb6b790 2024-09-13T18:23:39.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.355+0000 7fcf9cff9640 1 -- 172.21.15.114:0/3311900820 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcfa400ee80 con 0x5625feb6b790 2024-09-13T18:23:39.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.355+0000 7fcfbe1f1740 10 client.16098 fetch_fsmap learned FSMap version 30 2024-09-13T18:23:39.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.355+0000 7fcfbe1f1740 1 -- 172.21.15.114:0/3311900820 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5625feacd3c0 con 0x5625feb6b790 2024-09-13T18:23:39.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.355+0000 7fcf9cff9640 1 -- 172.21.15.114:0/3311900820 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5625feaf8b80 con 0x5625feb6b790 2024-09-13T18:23:39.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.355+0000 7fcf9cff9640 1 -- 172.21.15.114:0/3311900820 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcfa401d410 con 0x5625feb6b790 2024-09-13T18:23:39.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.355+0000 7fcfbe1f1740 10 client.16098 fetch_fsmap finished waiting for FSMap version 30 2024-09-13T18:23:39.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.355+0000 7fcfbe1f1740 10 client.16098 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-13T18:23:39.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.355+0000 7fcfbe1f1740 20 client.16098 populate_metadata read hostname 'smithi114' 2024-09-13T18:23:39.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.355+0000 7fcfbe1f1740 1 --2- 172.21.15.114:0/3311900820 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x5625feb861b0 0x5625feb885d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.355+0000 7fcfbe1f1740 4 client.16098 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:23:39.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.355+0000 7fcfbe1f1740 1 -- 172.21.15.114:0/3311900820 --> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5625feac11f0 con 0x5625feb861b0 2024-09-13T18:23:39.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.355+0000 7fcfb898f640 1 --2- 172.21.15.114:0/3311900820 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x5625feb861b0 0x5625feb885d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.356+0000 7fcfb898f640 1 --2- 172.21.15.114:0/3311900820 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x5625feb861b0 0x5625feb885d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5880a7c36b0aea64 server_cookie=67cba13e346108a2 in_seq=0 out_seq=0 2024-09-13T18:23:39.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.356+0000 7fcf9cff9640 10 client.16098 ms_handle_connect on v2:172.21.15.134:6834/1150003275 2024-09-13T18:23:39.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.356+0000 7fcf9cff9640 1 -- 172.21.15.114:0/3311900820 <== mds.3 v2:172.21.15.134:6834/1150003275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5625feac11f0 con 0x5625feb861b0 2024-09-13T18:23:39.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.356+0000 7fcf9cff9640 10 client.16098 handle_command_reply: tid=0 2024-09-13T18:23:39.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.356+0000 7fcfbe1f1740 1 client.16098 shutdown 2024-09-13T18:23:39.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.357+0000 7fcfbe1f1740 1 -- 172.21.15.114:0/3311900820 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625feb6b790 msgr2=0x5625feb75e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:39.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.357+0000 7fcfbe1f1740 1 --2- 172.21.15.114:0/3311900820 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625feb6b790 0x5625feb75e30 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fcfa400ecd0 tx=0x7fcfa4004730 comp rx=0 tx=0).stop 2024-09-13T18:23:39.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.357+0000 7fcfbe1f1740 1 -- 172.21.15.114:0/3311900820 shutdown_connections 2024-09-13T18:23:39.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.357+0000 7fcfbe1f1740 1 --2- 172.21.15.114:0/3311900820 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x5625feb861b0 0x5625feb885d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.357+0000 7fcf9cff9640 0 client.16098 ms_handle_reset on v2:172.21.15.134:6834/1150003275 2024-09-13T18:23:39.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.357+0000 7fcfbe1f1740 1 --2- 172.21.15.114:0/3311900820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625feb76950 0x5625feb73270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.357+0000 7fcfbe1f1740 1 --2- 172.21.15.114:0/3311900820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625feb6c0b0 0x5625feb76370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.357+0000 7fcfbe1f1740 1 --2- 172.21.15.114:0/3311900820 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625feb6b790 0x5625feb75e30 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.357+0000 7fcfbe1f1740 1 -- 172.21.15.114:0/3311900820 >> 172.21.15.114:0/3311900820 conn(0x5625feb67d00 msgr2=0x5625feb6da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:39.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.357+0000 7fcfbe1f1740 1 -- 172.21.15.114:0/3311900820 shutdown_connections 2024-09-13T18:23:39.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.357+0000 7fcfbe1f1740 1 -- 172.21.15.114:0/3311900820 wait complete. 2024-09-13T18:23:39.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.357+0000 7fcfbe1f1740 20 client.16098 trim_cache size 0 max 16384 2024-09-13T18:23:39.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.368+0000 7fcfbe1f1740 1 Processor -- start 2024-09-13T18:23:39.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.368+0000 7fcfbe1f1740 1 -- start start 2024-09-13T18:23:39.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.368+0000 7fcfbe1f1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625feb6b790 0x5625feb7d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.368+0000 7fcfbe1f1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625feb6c0b0 0x5625feb7db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.368+0000 7fcfbe1f1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625feb76950 0x5625feb6e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.369+0000 7fcfbe1f1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625feb74130 con 0x5625feb6c0b0 2024-09-13T18:23:39.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.369+0000 7fcfbe1f1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625feb84a90 con 0x5625feb76950 2024-09-13T18:23:39.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.369+0000 7fcfbe1f1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625feb637c0 con 0x5625feb6b790 2024-09-13T18:23:39.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.369+0000 7fcfb898f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625feb6c0b0 0x5625feb7db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.369+0000 7fcfb898f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625feb6c0b0 0x5625feb7db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51398/0 (socket says 172.21.15.114:51398) 2024-09-13T18:23:39.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.369+0000 7fcfb898f640 1 -- 172.21.15.114:0/1327234075 learned_addr learned my addr 172.21.15.114:0/1327234075 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:39.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.369+0000 7fcfb9190640 1 --2- 172.21.15.114:0/1327234075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625feb6b790 0x5625feb7d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.369+0000 7fcfb9991640 1 --2- 172.21.15.114:0/1327234075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625feb76950 0x5625feb6e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.369+0000 7fcfb898f640 1 -- 172.21.15.114:0/1327234075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625feb6b790 msgr2=0x5625feb7d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:39.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.369+0000 7fcfb898f640 1 --2- 172.21.15.114:0/1327234075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625feb6b790 0x5625feb7d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.369+0000 7fcfb898f640 1 -- 172.21.15.114:0/1327234075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625feb76950 msgr2=0x5625feb6e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:39.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.369+0000 7fcfb898f640 1 --2- 172.21.15.114:0/1327234075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625feb76950 0x5625feb6e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.369+0000 7fcfb898f640 1 -- 172.21.15.114:0/1327234075 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625fea6f770 con 0x5625feb6c0b0 2024-09-13T18:23:39.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.369+0000 7fcfb898f640 1 --2- 172.21.15.114:0/1327234075 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625feb6c0b0 0x5625feb7db60 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fcfb000c700 tx=0x7fcfb0000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf6323462db86973 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:39.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.370+0000 7fcf9ffff640 1 -- 172.21.15.114:0/1327234075 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb009aac0 con 0x5625feb6c0b0 2024-09-13T18:23:39.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.370+0000 7fcf9ffff640 1 -- 172.21.15.114:0/1327234075 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcfb009a3c0 con 0x5625feb6c0b0 2024-09-13T18:23:39.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.370+0000 7fcf9ffff640 1 -- 172.21.15.114:0/1327234075 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb00acc00 con 0x5625feb6c0b0 2024-09-13T18:23:39.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.371+0000 7fcfbe1f1740 1 -- 172.21.15.114:0/1327234075 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625feb6c0b0 msgr2=0x5625feb7db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:39.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.371+0000 7fcfbe1f1740 1 --2- 172.21.15.114:0/1327234075 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625feb6c0b0 0x5625feb7db60 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fcfb000c700 tx=0x7fcfb0000f40 comp rx=0 tx=0).stop 2024-09-13T18:23:39.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.371+0000 7fcfbe1f1740 1 -- 172.21.15.114:0/1327234075 shutdown_connections 2024-09-13T18:23:39.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.371+0000 7fcfbe1f1740 1 --2- 172.21.15.114:0/1327234075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625feb76950 0x5625feb6e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.371+0000 7fcfbe1f1740 1 --2- 172.21.15.114:0/1327234075 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625feb6c0b0 0x5625feb7db60 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.371+0000 7fcfbe1f1740 1 --2- 172.21.15.114:0/1327234075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625feb6b790 0x5625feb7d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.371+0000 7fcfbe1f1740 1 -- 172.21.15.114:0/1327234075 >> 172.21.15.114:0/1327234075 conn(0x5625feb67d00 msgr2=0x5625feb680e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:39.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.371+0000 7fcfbe1f1740 1 -- 172.21.15.114:0/1327234075 shutdown_connections 2024-09-13T18:23:39.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.371+0000 7fcfbe1f1740 1 -- 172.21.15.114:0/1327234075 wait complete. 2024-09-13T18:23:39.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.371+0000 7fcfbe1f1740 1 -- start start 2024-09-13T18:23:39.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.371+0000 7fcfbe1f1740 1 Processor -- start 2024-09-13T18:23:39.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.372+0000 7fcfbe1f1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625feb6b790 0x5625feb7b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.372+0000 7fcfbe1f1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625feb6c0b0 0x5625feb7bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.372+0000 7fcfbe1f1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625feb76950 0x5625febdb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.372+0000 7fcfbe1f1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625febdbbf0 con 0x5625feb6b790 2024-09-13T18:23:39.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.372+0000 7fcfbe1f1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625febdbd60 con 0x5625feb76950 2024-09-13T18:23:39.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.372+0000 7fcfbe1f1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625febdbf20 con 0x5625feb6c0b0 2024-09-13T18:23:39.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.372+0000 7fcfbe1f1740 10 client.0 _ll_register_callbacks cb 0x7fcfa4009680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:23:39.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.372+0000 7fcfb9190640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625feb6b790 0x5625feb7b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.372+0000 7fcfb9190640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625feb6b790 0x5625feb7b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51404/0 (socket says 172.21.15.114:51404) 2024-09-13T18:23:39.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.372+0000 7fcfb9190640 1 -- 172.21.15.114:0/3343510392 learned_addr learned my addr 172.21.15.114:0/3343510392 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:39.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.372+0000 7fcfb898f640 1 --2- 172.21.15.114:0/3343510392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625feb6c0b0 0x5625feb7bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.372+0000 7fcfb9991640 1 --2- 172.21.15.114:0/3343510392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625feb76950 0x5625febdb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.372+0000 7fcfb9190640 1 -- 172.21.15.114:0/3343510392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625feb6c0b0 msgr2=0x5625feb7bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:39.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.372+0000 7fcfb9190640 1 --2- 172.21.15.114:0/3343510392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625feb6c0b0 0x5625feb7bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.372+0000 7fcfb9190640 1 -- 172.21.15.114:0/3343510392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625feb76950 msgr2=0x5625febdb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:39.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.372+0000 7fcfb9190640 1 --2- 172.21.15.114:0/3343510392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625feb76950 0x5625febdb6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.372+0000 7fcfb9190640 1 -- 172.21.15.114:0/3343510392 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5625fea96620 con 0x5625feb6b790 2024-09-13T18:23:39.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.372+0000 7fcfb9190640 1 --2- 172.21.15.114:0/3343510392 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625feb6b790 0x5625feb7b9d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4016040 tx=0x7fcfa4011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be79b6b1d7aeb606 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:39.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.372+0000 7fcf9cff9640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:23:39.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.373+0000 7fcf9cff9640 1 -- 172.21.15.114:0/3343510392 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa401d620 con 0x5625feb6b790 2024-09-13T18:23:39.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.373+0000 7fcf9cff9640 1 -- 172.21.15.114:0/3343510392 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fcfa4023040 con 0x5625feb6b790 2024-09-13T18:23:39.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.373+0000 7fcfbe1f1740 1 -- 172.21.15.114:0/3343510392 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5625feb828a0 con 0x5625feb6b790 2024-09-13T18:23:39.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.373+0000 7fcf9cff9640 1 -- 172.21.15.114:0/3343510392 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcfa401d9f0 con 0x5625feb6b790 2024-09-13T18:23:39.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.373+0000 7fcfbe1f1740 10 client.16104 fetch_fsmap learned FSMap version 30 2024-09-13T18:23:39.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.373+0000 7fcfbe1f1740 1 -- 172.21.15.114:0/3343510392 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5625fea94e80 con 0x5625feb6b790 2024-09-13T18:23:39.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.373+0000 7fcf9cff9640 1 -- 172.21.15.114:0/3343510392 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5625feb828a0 con 0x5625feb6b790 2024-09-13T18:23:39.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.373+0000 7fcf9cff9640 1 -- 172.21.15.114:0/3343510392 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcfa40035f0 con 0x5625feb6b790 2024-09-13T18:23:39.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.373+0000 7fcfbe1f1740 10 client.16104 fetch_fsmap finished waiting for FSMap version 30 2024-09-13T18:23:39.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.373+0000 7fcfbe1f1740 10 client.16104 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-13T18:23:39.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.373+0000 7fcfbe1f1740 20 client.16104 populate_metadata read hostname 'smithi114' 2024-09-13T18:23:39.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.373+0000 7fcfbe1f1740 1 --2- 172.21.15.114:0/3343510392 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x5625feb861b0 0x5625febdf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.373+0000 7fcfbe1f1740 4 client.16104 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:23:39.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.373+0000 7fcfbe1f1740 1 -- 172.21.15.114:0/3343510392 --> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fcfb00aca90 con 0x5625feb861b0 2024-09-13T18:23:39.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.373+0000 7fcfb898f640 1 --2- 172.21.15.114:0/3343510392 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x5625feb861b0 0x5625febdf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.374+0000 7fcfb898f640 1 --2- 172.21.15.114:0/3343510392 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x5625feb861b0 0x5625febdf6c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f23cb3dd959a2123 server_cookie=bc51fca864001091 in_seq=0 out_seq=0 2024-09-13T18:23:39.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.374+0000 7fcf9cff9640 10 client.16104 ms_handle_connect on v2:172.21.15.134:6834/1150003275 2024-09-13T18:23:39.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.374+0000 7fcf9cff9640 1 -- 172.21.15.114:0/3343510392 <== mds.3 v2:172.21.15.134:6834/1150003275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fcfb00aca90 con 0x5625feb861b0 2024-09-13T18:23:39.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.374+0000 7fcf9cff9640 10 client.16104 handle_command_reply: tid=0 2024-09-13T18:23:39.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.374+0000 7fcfbe1f1740 1 client.16104 shutdown 2024-09-13T18:23:39.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.374+0000 7fcfbe1f1740 1 -- 172.21.15.114:0/3343510392 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625feb6b790 msgr2=0x5625feb7b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:39.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.374+0000 7fcfbe1f1740 1 --2- 172.21.15.114:0/3343510392 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625feb6b790 0x5625feb7b9d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4016040 tx=0x7fcfa4011920 comp rx=0 tx=0).stop 2024-09-13T18:23:39.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.374+0000 7fcfbe1f1740 1 -- 172.21.15.114:0/3343510392 shutdown_connections 2024-09-13T18:23:39.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.374+0000 7fcfbe1f1740 1 --2- 172.21.15.114:0/3343510392 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x5625feb861b0 0x5625febdf6c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.374+0000 7fcfbe1f1740 1 --2- 172.21.15.114:0/3343510392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625feb76950 0x5625febdb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.374+0000 7fcf9cff9640 0 client.16104 ms_handle_reset on v2:172.21.15.134:6834/1150003275 2024-09-13T18:23:39.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.374+0000 7fcfbe1f1740 1 --2- 172.21.15.114:0/3343510392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625feb6c0b0 0x5625feb7bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.374+0000 7fcfbe1f1740 1 --2- 172.21.15.114:0/3343510392 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625feb6b790 0x5625feb7b9d0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.374+0000 7fcfbe1f1740 1 -- 172.21.15.114:0/3343510392 >> 172.21.15.114:0/3343510392 conn(0x5625feb67d00 msgr2=0x5625feb6e360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:39.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.374+0000 7fcfbe1f1740 1 -- 172.21.15.114:0/3343510392 shutdown_connections 2024-09-13T18:23:39.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.374+0000 7fcfbe1f1740 1 -- 172.21.15.114:0/3343510392 wait complete. 2024-09-13T18:23:39.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.374+0000 7fcfbe1f1740 20 client.16104 trim_cache size 0 max 16384 2024-09-13T18:23:39.385 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:23:39.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.375+0000 7fcfbb3f4640 1 -- 172.21.15.114:0/2266518600 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fcfa00a1320 msgr2=0x7fcfa00a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:39.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.375+0000 7fcfbb3f4640 1 --2- 172.21.15.114:0/2266518600 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fcfa00a1320 0x7fcfa00a37e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4009bd0 tx=0x7fcfa4005f90 comp rx=0 tx=0).stop 2024-09-13T18:23:39.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.375+0000 7fcfbb3f4640 1 -- 172.21.15.114:0/2266518600 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcfb4101890 msgr2=0x7fcfb4196700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:39.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.375+0000 7fcfbb3f4640 1 --2- 172.21.15.114:0/2266518600 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcfb4101890 0x7fcfb4196700 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fcfb000eab0 tx=0x7fcfb000ef70 comp rx=0 tx=0).stop 2024-09-13T18:23:39.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.375+0000 7fcfbb3f4640 1 -- 172.21.15.114:0/2266518600 shutdown_connections 2024-09-13T18:23:39.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.375+0000 7fcfbb3f4640 1 --2- 172.21.15.114:0/2266518600 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fcfa00a1320 0x7fcfa00a37e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.375+0000 7fcfbb3f4640 1 --2- 172.21.15.114:0/2266518600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcfb410a340 0x7fcfb419aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.375+0000 7fcfbb3f4640 1 --2- 172.21.15.114:0/2266518600 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcfb4101890 0x7fcfb4196700 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.375+0000 7fcfbb3f4640 1 --2- 172.21.15.114:0/2266518600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcfb4100f70 0x7fcfb41961c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.375+0000 7fcfbb3f4640 1 -- 172.21.15.114:0/2266518600 >> 172.21.15.114:0/2266518600 conn(0x7fcfb40fa900 msgr2=0x7fcfb40fc390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:39.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.375+0000 7fcfbb3f4640 1 -- 172.21.15.114:0/2266518600 shutdown_connections 2024-09-13T18:23:39.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.375+0000 7fcfbb3f4640 1 -- 172.21.15.114:0/2266518600 wait complete. 2024-09-13T18:23:39.390 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-09-13T18:23:39.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.493+0000 7fb6a16c9640 1 -- 172.21.15.114:0/3615257003 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb69c103fa0 msgr2=0x7fb69c10baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:39.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.493+0000 7fb6a16c9640 1 --2- 172.21.15.114:0/3615257003 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb69c103fa0 0x7fb69c10baf0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fb694009cd0 tx=0x7fb69402a8e0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.493+0000 7fb6a16c9640 1 -- 172.21.15.114:0/3615257003 shutdown_connections 2024-09-13T18:23:39.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.493+0000 7fb6a16c9640 1 --2- 172.21.15.114:0/3615257003 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb69c103fa0 0x7fb69c10baf0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.493+0000 7fb6a16c9640 1 --2- 172.21.15.114:0/3615257003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb69c0ff800 0x7fb69c103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.493+0000 7fb6a16c9640 1 --2- 172.21.15.114:0/3615257003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb69c0fee50 0x7fb69c0ff230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.493+0000 7fb6a16c9640 1 -- 172.21.15.114:0/3615257003 >> 172.21.15.114:0/3615257003 conn(0x7fb69c0fa8e0 msgr2=0x7fb69c0fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:39.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.493+0000 7fb6a16c9640 1 -- 172.21.15.114:0/3615257003 shutdown_connections 2024-09-13T18:23:39.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.493+0000 7fb6a16c9640 1 -- 172.21.15.114:0/3615257003 wait complete. 2024-09-13T18:23:39.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.494+0000 7fb6a16c9640 1 Processor -- start 2024-09-13T18:23:39.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.494+0000 7fb6a16c9640 1 -- start start 2024-09-13T18:23:39.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.494+0000 7fb6a16c9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb69c0fee50 0x7fb69c10c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.494+0000 7fb6a16c9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb69c103fa0 0x7fb69c10c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.494+0000 7fb6a16c9640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb69c10ce80 0x7fb69c1a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.494+0000 7fb6a16c9640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb69c1a4920 con 0x7fb69c10ce80 2024-09-13T18:23:39.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.494+0000 7fb6a16c9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb69c1a4a90 con 0x7fb69c0fee50 2024-09-13T18:23:39.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.494+0000 7fb6a16c9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb69c1a4c50 con 0x7fb69c103fa0 2024-09-13T18:23:39.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.494+0000 7fb69b7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb69c10ce80 0x7fb69c1a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.494+0000 7fb69b7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb69c10ce80 0x7fb69c1a4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51426/0 (socket says 172.21.15.114:51426) 2024-09-13T18:23:39.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.494+0000 7fb69b7fe640 1 -- 172.21.15.114:0/2797847447 learned_addr learned my addr 172.21.15.114:0/2797847447 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:39.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.494+0000 7fb69affd640 1 --2- 172.21.15.114:0/2797847447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb69c0fee50 0x7fb69c10c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.494+0000 7fb69a7fc640 1 --2- 172.21.15.114:0/2797847447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb69c103fa0 0x7fb69c10c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.494+0000 7fb69b7fe640 1 -- 172.21.15.114:0/2797847447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb69c103fa0 msgr2=0x7fb69c10c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:39.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.494+0000 7fb69b7fe640 1 --2- 172.21.15.114:0/2797847447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb69c103fa0 0x7fb69c10c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.494+0000 7fb69b7fe640 1 -- 172.21.15.114:0/2797847447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb69c0fee50 msgr2=0x7fb69c10c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:39.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.494+0000 7fb69b7fe640 1 --2- 172.21.15.114:0/2797847447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb69c0fee50 0x7fb69c10c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.495+0000 7fb69b7fe640 1 -- 172.21.15.114:0/2797847447 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb694009900 con 0x7fb69c10ce80 2024-09-13T18:23:39.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.495+0000 7fb69affd640 1 --2- 172.21.15.114:0/2797847447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb69c0fee50 0x7fb69c10c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:23:39.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.495+0000 7fb69b7fe640 1 --2- 172.21.15.114:0/2797847447 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb69c10ce80 0x7fb69c1a4350 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fb694004af0 tx=0x7fb694030e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:39.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.495+0000 7fb693fff640 1 -- 172.21.15.114:0/2797847447 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb694031b80 con 0x7fb69c10ce80 2024-09-13T18:23:39.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.495+0000 7fb6a16c9640 1 -- 172.21.15.114:0/2797847447 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb69c1a4ed0 con 0x7fb69c10ce80 2024-09-13T18:23:39.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.495+0000 7fb693fff640 1 -- 172.21.15.114:0/2797847447 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb694003e00 con 0x7fb69c10ce80 2024-09-13T18:23:39.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.495+0000 7fb693fff640 1 -- 172.21.15.114:0/2797847447 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb694045410 con 0x7fb69c10ce80 2024-09-13T18:23:39.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.495+0000 7fb6a16c9640 1 -- 172.21.15.114:0/2797847447 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb69c1a5400 con 0x7fb69c10ce80 2024-09-13T18:23:39.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.497+0000 7fb693fff640 1 -- 172.21.15.114:0/2797847447 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) v1 ==== 136107+0+0 (secure 0 0 0) 0x7fb69400a040 con 0x7fb69c10ce80 2024-09-13T18:23:39.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.498+0000 7fb6a43ec740 1 Processor -- start 2024-09-13T18:23:39.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.498+0000 7fb6a43ec740 1 -- start start 2024-09-13T18:23:39.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.498+0000 7fb693fff640 1 --2- 172.21.15.114:0/2797847447 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fb6880a1350 0x7fb6880a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.498+0000 7fb6a43ec740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561781a4a790 0x561781a4ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.498+0000 7fb6a43ec740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561781a4b0b0 0x561781a55410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.498+0000 7fb6a43ec740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561781a55950 0x561781a57d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.498+0000 7fb6a43ec740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617819b4cc0 con 0x561781a4b0b0 2024-09-13T18:23:39.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.498+0000 7fb6a43ec740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617819b2450 con 0x561781a4a790 2024-09-13T18:23:39.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.498+0000 7fb6a43ec740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617819add40 con 0x561781a55950 2024-09-13T18:23:39.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.498+0000 7fb69a7fc640 1 --2- 172.21.15.114:0/2797847447 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fb6880a1350 0x7fb6880a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.498+0000 7fb693fff640 1 -- 172.21.15.114:0/2797847447 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fb6940ee6f0 con 0x7fb69c10ce80 2024-09-13T18:23:39.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.498+0000 7fb69a7fc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561781a4b0b0 0x561781a55410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.498+0000 7fb69affd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561781a4a790 0x561781a4ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.498+0000 7fb69affd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561781a4a790 0x561781a4ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:51018/0 (socket says 172.21.15.114:51018) 2024-09-13T18:23:39.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.498+0000 7fb69affd640 1 -- 172.21.15.114:0/2668534916 learned_addr learned my addr 172.21.15.114:0/2668534916 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:39.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.498+0000 7fb69b7fe640 1 --2- 172.21.15.114:0/2668534916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561781a55950 0x561781a57d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.499+0000 7fb69b7fe640 1 -- 172.21.15.114:0/2668534916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561781a4a790 msgr2=0x561781a4ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:39.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.499+0000 7fb69b7fe640 1 --2- 172.21.15.114:0/2668534916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561781a4a790 0x561781a4ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.499+0000 7fb69b7fe640 1 -- 172.21.15.114:0/2668534916 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561781a4b0b0 msgr2=0x561781a55410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:39.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.499+0000 7fb69b7fe640 1 --2- 172.21.15.114:0/2668534916 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561781a4b0b0 0x561781a55410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.499+0000 7fb69b7fe640 1 -- 172.21.15.114:0/2668534916 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561781975620 con 0x561781a55950 2024-09-13T18:23:39.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.499+0000 7fb69a7fc640 1 --2- 172.21.15.114:0/2797847447 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fb6880a1350 0x7fb6880a3810 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb68c0058c0 tx=0x7fb68c00b960 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:39.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.499+0000 7fb69b7fe640 1 --2- 172.21.15.114:0/2668534916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561781a55950 0x561781a57d40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb6940eb970 tx=0x7fb6940bae60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f409a8a9fc1d8b99 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:39.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.499+0000 7fb6917fa640 1 -- 172.21.15.114:0/2668534916 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6940a61e0 con 0x561781a55950 2024-09-13T18:23:39.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.499+0000 7fb6917fa640 1 -- 172.21.15.114:0/2668534916 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb6940bb960 con 0x561781a55950 2024-09-13T18:23:39.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.499+0000 7fb6917fa640 1 -- 172.21.15.114:0/2668534916 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6940c36e0 con 0x561781a55950 2024-09-13T18:23:39.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.500+0000 7fb6a43ec740 1 -- 172.21.15.114:0/2668534916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561781a55950 msgr2=0x561781a57d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:39.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.500+0000 7fb6a43ec740 1 --2- 172.21.15.114:0/2668534916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561781a55950 0x561781a57d40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb6940eb970 tx=0x7fb6940bae60 comp rx=0 tx=0).stop 2024-09-13T18:23:39.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.500+0000 7fb6a43ec740 1 -- 172.21.15.114:0/2668534916 shutdown_connections 2024-09-13T18:23:39.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.500+0000 7fb6a43ec740 1 --2- 172.21.15.114:0/2668534916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561781a55950 0x561781a57d40 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.500+0000 7fb6a43ec740 1 --2- 172.21.15.114:0/2668534916 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561781a4b0b0 0x561781a55410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.500+0000 7fb6a43ec740 1 --2- 172.21.15.114:0/2668534916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561781a4a790 0x561781a4ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.500+0000 7fb6a43ec740 1 -- 172.21.15.114:0/2668534916 >> 172.21.15.114:0/2668534916 conn(0x561781a46d00 msgr2=0x561781a470e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:39.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.500+0000 7fb6a43ec740 1 -- 172.21.15.114:0/2668534916 shutdown_connections 2024-09-13T18:23:39.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.500+0000 7fb6a43ec740 1 -- 172.21.15.114:0/2668534916 wait complete. 2024-09-13T18:23:39.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.500+0000 7fb6a43ec740 1 -- start start 2024-09-13T18:23:39.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.500+0000 7fb6a43ec740 1 Processor -- start 2024-09-13T18:23:39.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.500+0000 7fb6a43ec740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561781a4a790 0x561781a52270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.500+0000 7fb6a43ec740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561781a4b0b0 0x561781a527b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.500+0000 7fb6a43ec740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561781a55950 0x561781a548c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.500+0000 7fb6a43ec740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56178196c180 con 0x561781a4b0b0 2024-09-13T18:23:39.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.500+0000 7fb6a43ec740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561781686430 con 0x561781a55950 2024-09-13T18:23:39.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.500+0000 7fb6a43ec740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617819315e0 con 0x561781a4a790 2024-09-13T18:23:39.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.501+0000 7fb6a43ec740 10 client.0 _ll_register_callbacks cb 0x56178199daa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:23:39.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.501+0000 7fb69a7fc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561781a4b0b0 0x561781a527b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.501+0000 7fb69a7fc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561781a4b0b0 0x561781a527b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51438/0 (socket says 172.21.15.114:51438) 2024-09-13T18:23:39.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.501+0000 7fb69a7fc640 1 -- 172.21.15.114:0/1559201906 learned_addr learned my addr 172.21.15.114:0/1559201906 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:39.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.501+0000 7fb69affd640 1 --2- 172.21.15.114:0/1559201906 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561781a4a790 0x561781a52270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.501+0000 7fb69b7fe640 1 --2- 172.21.15.114:0/1559201906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561781a55950 0x561781a548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.501+0000 7fb69a7fc640 1 -- 172.21.15.114:0/1559201906 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561781a4a790 msgr2=0x561781a52270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:39.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.501+0000 7fb69a7fc640 1 --2- 172.21.15.114:0/1559201906 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561781a4a790 0x561781a52270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.501+0000 7fb69a7fc640 1 -- 172.21.15.114:0/1559201906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561781a55950 msgr2=0x561781a548c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:39.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.501+0000 7fb69a7fc640 1 --2- 172.21.15.114:0/1559201906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561781a55950 0x561781a548c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.501+0000 7fb69a7fc640 1 -- 172.21.15.114:0/1559201906 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6940b5e70 con 0x561781a4b0b0 2024-09-13T18:23:39.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.501+0000 7fb69b7fe640 1 --2- 172.21.15.114:0/1559201906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561781a55950 0x561781a548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:23:39.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.501+0000 7fb69a7fc640 1 --2- 172.21.15.114:0/1559201906 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561781a4b0b0 0x561781a527b0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fb68c0108c0 tx=0x7fb68c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b88740f58d5931ef server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:39.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.501+0000 7fb6827fc640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:23:39.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.501+0000 7fb6827fc640 1 -- 172.21.15.114:0/1559201906 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68c011d70 con 0x561781a4b0b0 2024-09-13T18:23:39.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.501+0000 7fb6827fc640 1 -- 172.21.15.114:0/1559201906 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fb68c02a410 con 0x561781a4b0b0 2024-09-13T18:23:39.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.501+0000 7fb6a43ec740 1 -- 172.21.15.114:0/1559201906 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5617819d7b80 con 0x561781a4b0b0 2024-09-13T18:23:39.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.501+0000 7fb6827fc640 1 -- 172.21.15.114:0/1559201906 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb68c011ed0 con 0x561781a4b0b0 2024-09-13T18:23:39.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.501+0000 7fb6a43ec740 10 client.16116 fetch_fsmap learned FSMap version 30 2024-09-13T18:23:39.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.501+0000 7fb6a43ec740 1 -- 172.21.15.114:0/1559201906 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5617819ac3c0 con 0x561781a4b0b0 2024-09-13T18:23:39.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.502+0000 7fb6827fc640 1 -- 172.21.15.114:0/1559201906 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5617819d7b80 con 0x561781a4b0b0 2024-09-13T18:23:39.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.502+0000 7fb6827fc640 1 -- 172.21.15.114:0/1559201906 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb68c0203f0 con 0x561781a4b0b0 2024-09-13T18:23:39.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.502+0000 7fb6a43ec740 10 client.16116 fetch_fsmap finished waiting for FSMap version 30 2024-09-13T18:23:39.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.502+0000 7fb6a43ec740 10 client.16116 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-13T18:23:39.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.502+0000 7fb6a43ec740 20 client.16116 populate_metadata read hostname 'smithi114' 2024-09-13T18:23:39.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.502+0000 7fb6a43ec740 1 --2- 172.21.15.114:0/1559201906 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x561781a64f20 0x561781a67340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.502+0000 7fb6a43ec740 4 client.16116 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:23:39.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.502+0000 7fb6a43ec740 1 -- 172.21.15.114:0/1559201906 --> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5617819a01f0 con 0x561781a64f20 2024-09-13T18:23:39.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.502+0000 7fb69affd640 1 --2- 172.21.15.114:0/1559201906 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x561781a64f20 0x561781a67340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.503+0000 7fb69affd640 1 --2- 172.21.15.114:0/1559201906 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x561781a64f20 0x561781a67340 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a66b24da6ed72e8e server_cookie=b56c7cdbf54372e1 in_seq=0 out_seq=0 2024-09-13T18:23:39.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.503+0000 7fb6827fc640 10 client.16116 ms_handle_connect on v2:172.21.15.204:6838/344883912 2024-09-13T18:23:39.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.503+0000 7fb6827fc640 1 -- 172.21.15.114:0/1559201906 <== mds.2 v2:172.21.15.204:6838/344883912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5617819a01f0 con 0x561781a64f20 2024-09-13T18:23:39.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.503+0000 7fb6827fc640 10 client.16116 handle_command_reply: tid=0 2024-09-13T18:23:39.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.503+0000 7fb6a43ec740 1 client.16116 shutdown 2024-09-13T18:23:39.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.503+0000 7fb6a43ec740 1 -- 172.21.15.114:0/1559201906 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561781a4b0b0 msgr2=0x561781a527b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:39.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.503+0000 7fb6a43ec740 1 --2- 172.21.15.114:0/1559201906 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561781a4b0b0 0x561781a527b0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fb68c0108c0 tx=0x7fb68c005850 comp rx=0 tx=0).stop 2024-09-13T18:23:39.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.503+0000 7fb6a43ec740 1 -- 172.21.15.114:0/1559201906 shutdown_connections 2024-09-13T18:23:39.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.503+0000 7fb6a43ec740 1 --2- 172.21.15.114:0/1559201906 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x561781a64f20 0x561781a67340 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.503+0000 7fb6a43ec740 1 --2- 172.21.15.114:0/1559201906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561781a55950 0x561781a548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.503+0000 7fb6827fc640 0 client.16116 ms_handle_reset on v2:172.21.15.204:6838/344883912 2024-09-13T18:23:39.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.503+0000 7fb6a43ec740 1 --2- 172.21.15.114:0/1559201906 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561781a4b0b0 0x561781a527b0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.503+0000 7fb6a43ec740 1 --2- 172.21.15.114:0/1559201906 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561781a4a790 0x561781a52270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.503+0000 7fb6a43ec740 1 -- 172.21.15.114:0/1559201906 >> 172.21.15.114:0/1559201906 conn(0x561781a46d00 msgr2=0x561781a45eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:39.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.503+0000 7fb6a43ec740 1 -- 172.21.15.114:0/1559201906 shutdown_connections 2024-09-13T18:23:39.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.504+0000 7fb6a43ec740 1 -- 172.21.15.114:0/1559201906 wait complete. 2024-09-13T18:23:39.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.504+0000 7fb6a43ec740 20 client.16116 trim_cache size 0 max 16384 2024-09-13T18:23:39.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.515+0000 7fb6a43ec740 1 Processor -- start 2024-09-13T18:23:39.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.515+0000 7fb6a43ec740 1 -- start start 2024-09-13T18:23:39.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.515+0000 7fb6a43ec740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561781a4a790 0x561781a54850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.515+0000 7fb6a43ec740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561781a4b0b0 0x561781a5c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.515+0000 7fb6a43ec740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561781a55950 0x561781a5c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.515+0000 7fb6a43ec740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x561781a4b6d0 con 0x561781a4a790 2024-09-13T18:23:39.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.515+0000 7fb6a43ec740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561781a53220 con 0x561781a4b0b0 2024-09-13T18:23:39.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.515+0000 7fb6a43ec740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561781a50610 con 0x561781a55950 2024-09-13T18:23:39.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.515+0000 7fb69affd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561781a4a790 0x561781a54850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.515+0000 7fb69affd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561781a4a790 0x561781a54850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51442/0 (socket says 172.21.15.114:51442) 2024-09-13T18:23:39.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.515+0000 7fb69affd640 1 -- 172.21.15.114:0/512349592 learned_addr learned my addr 172.21.15.114:0/512349592 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:39.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.515+0000 7fb69b7fe640 1 --2- 172.21.15.114:0/512349592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561781a55950 0x561781a5c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.515+0000 7fb69a7fc640 1 --2- 172.21.15.114:0/512349592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561781a4b0b0 0x561781a5c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.516+0000 7fb69affd640 1 -- 172.21.15.114:0/512349592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561781a55950 msgr2=0x561781a5c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:39.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.516+0000 7fb69affd640 1 --2- 172.21.15.114:0/512349592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561781a55950 0x561781a5c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.516+0000 7fb69affd640 1 -- 172.21.15.114:0/512349592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561781a4b0b0 msgr2=0x561781a5c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:39.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.516+0000 7fb69affd640 1 --2- 172.21.15.114:0/512349592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561781a4b0b0 0x561781a5c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.516+0000 7fb69affd640 1 -- 172.21.15.114:0/512349592 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56178194e770 con 0x561781a4a790 2024-09-13T18:23:39.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.516+0000 7fb69b7fe640 1 --2- 172.21.15.114:0/512349592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561781a55950 0x561781a5c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:23:39.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.516+0000 7fb69affd640 1 --2- 172.21.15.114:0/512349592 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561781a4a790 0x561781a54850 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fb6840098c0 tx=0x7fb6840039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98c1a0583d080b51 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:39.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.516+0000 7fb691ffb640 1 -- 172.21.15.114:0/512349592 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb684018040 con 0x561781a4a790 2024-09-13T18:23:39.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.516+0000 7fb691ffb640 1 -- 172.21.15.114:0/512349592 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb684009e90 con 0x561781a4a790 2024-09-13T18:23:39.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.516+0000 7fb691ffb640 1 -- 172.21.15.114:0/512349592 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb684012d40 con 0x561781a4a790 2024-09-13T18:23:39.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.517+0000 7fb6a43ec740 1 -- 172.21.15.114:0/512349592 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561781a4a790 msgr2=0x561781a54850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:39.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.517+0000 7fb6a43ec740 1 --2- 172.21.15.114:0/512349592 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561781a4a790 0x561781a54850 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fb6840098c0 tx=0x7fb6840039c0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.517+0000 7fb6a43ec740 1 -- 172.21.15.114:0/512349592 shutdown_connections 2024-09-13T18:23:39.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.517+0000 7fb6a43ec740 1 --2- 172.21.15.114:0/512349592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561781a55950 0x561781a5c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.517+0000 7fb6a43ec740 1 --2- 172.21.15.114:0/512349592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561781a4b0b0 0x561781a5c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.517+0000 7fb6a43ec740 1 --2- 172.21.15.114:0/512349592 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561781a4a790 0x561781a54850 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.517+0000 7fb6a43ec740 1 -- 172.21.15.114:0/512349592 >> 172.21.15.114:0/512349592 conn(0x561781a46d00 msgr2=0x561781a4d360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:39.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.517+0000 7fb6a43ec740 1 -- 172.21.15.114:0/512349592 shutdown_connections 2024-09-13T18:23:39.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.517+0000 7fb6a43ec740 1 -- 172.21.15.114:0/512349592 wait complete. 2024-09-13T18:23:39.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.517+0000 7fb6a43ec740 1 -- start start 2024-09-13T18:23:39.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.517+0000 7fb6a43ec740 1 Processor -- start 2024-09-13T18:23:39.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.517+0000 7fb6a43ec740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561781a4a790 0x561781a5b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.517+0000 7fb6a43ec740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561781a4b0b0 0x561781a5b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.517+0000 7fb6a43ec740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561781a55950 0x561781a5bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.517+0000 7fb6a43ec740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x561781abb500 con 0x561781a4b0b0 2024-09-13T18:23:39.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.517+0000 7fb6a43ec740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561781abb670 con 0x561781a4a790 2024-09-13T18:23:39.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.517+0000 7fb6a43ec740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561781abb830 con 0x561781a55950 2024-09-13T18:23:39.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.517+0000 7fb6a43ec740 10 client.0 _ll_register_callbacks cb 0x7fb68c00bb70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:23:39.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.517+0000 7fb69a7fc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561781a4b0b0 0x561781a5b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.518+0000 7fb69a7fc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561781a4b0b0 0x561781a5b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51458/0 (socket says 172.21.15.114:51458) 2024-09-13T18:23:39.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.518+0000 7fb69a7fc640 1 -- 172.21.15.114:0/537765480 learned_addr learned my addr 172.21.15.114:0/537765480 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:39.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.518+0000 7fb69b7fe640 1 --2- 172.21.15.114:0/537765480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561781a55950 0x561781a5bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.518+0000 7fb69affd640 1 --2- 172.21.15.114:0/537765480 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561781a4a790 0x561781a5b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.518+0000 7fb69a7fc640 1 -- 172.21.15.114:0/537765480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561781a55950 msgr2=0x561781a5bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:39.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.518+0000 7fb69a7fc640 1 --2- 172.21.15.114:0/537765480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561781a55950 0x561781a5bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.518+0000 7fb69a7fc640 1 -- 172.21.15.114:0/537765480 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561781a4a790 msgr2=0x561781a5b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:39.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.518+0000 7fb69a7fc640 1 --2- 172.21.15.114:0/537765480 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561781a4a790 0x561781a5b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.518+0000 7fb69a7fc640 1 -- 172.21.15.114:0/537765480 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561781975620 con 0x561781a4b0b0 2024-09-13T18:23:39.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.518+0000 7fb69a7fc640 1 --2- 172.21.15.114:0/537765480 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561781a4b0b0 0x561781a5b630 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fb68c010490 tx=0x7fb68c0135a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc981239554b7fc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:39.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.518+0000 7fb6827fc640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:23:39.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.518+0000 7fb6827fc640 1 -- 172.21.15.114:0/537765480 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68c011d70 con 0x561781a4b0b0 2024-09-13T18:23:39.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.518+0000 7fb6a43ec740 1 -- 172.21.15.114:0/537765480 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb6940bb960 con 0x561781a4b0b0 2024-09-13T18:23:39.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.518+0000 7fb6827fc640 1 -- 172.21.15.114:0/537765480 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fb68c00b380 con 0x561781a4b0b0 2024-09-13T18:23:39.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.518+0000 7fb6827fc640 1 -- 172.21.15.114:0/537765480 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb68c011ed0 con 0x561781a4b0b0 2024-09-13T18:23:39.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.518+0000 7fb6a43ec740 10 client.16122 fetch_fsmap learned FSMap version 30 2024-09-13T18:23:39.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.518+0000 7fb6a43ec740 1 -- 172.21.15.114:0/537765480 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7fb6840099c0 con 0x561781a4b0b0 2024-09-13T18:23:39.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.518+0000 7fb6827fc640 1 -- 172.21.15.114:0/537765480 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb6940bb960 con 0x561781a4b0b0 2024-09-13T18:23:39.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.519+0000 7fb6827fc640 1 -- 172.21.15.114:0/537765480 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb68c03e020 con 0x561781a4b0b0 2024-09-13T18:23:39.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.519+0000 7fb6a43ec740 10 client.16122 fetch_fsmap finished waiting for FSMap version 30 2024-09-13T18:23:39.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.519+0000 7fb6a43ec740 10 client.16122 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-13T18:23:39.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.519+0000 7fb6a43ec740 20 client.16122 populate_metadata read hostname 'smithi114' 2024-09-13T18:23:39.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.519+0000 7fb6a43ec740 1 --2- 172.21.15.114:0/537765480 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x561781abcee0 0x561781abf2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.519+0000 7fb6a43ec740 4 client.16122 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:23:39.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.519+0000 7fb6a43ec740 1 -- 172.21.15.114:0/537765480 --> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5617816a14e0 con 0x561781abcee0 2024-09-13T18:23:39.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.519+0000 7fb69affd640 1 --2- 172.21.15.114:0/537765480 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x561781abcee0 0x561781abf2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.519+0000 7fb6827fc640 10 client.16122 ms_handle_connect on v2:172.21.15.204:6838/344883912 2024-09-13T18:23:39.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.519+0000 7fb69affd640 1 --2- 172.21.15.114:0/537765480 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x561781abcee0 0x561781abf2d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ed6e291900e371e8 server_cookie=ba120d9b86314d80 in_seq=0 out_seq=0 2024-09-13T18:23:39.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.519+0000 7fb6827fc640 1 -- 172.21.15.114:0/537765480 <== mds.2 v2:172.21.15.204:6838/344883912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5617816a14e0 con 0x561781abcee0 2024-09-13T18:23:39.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.519+0000 7fb6827fc640 10 client.16122 handle_command_reply: tid=0 2024-09-13T18:23:39.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.519+0000 7fb6a43ec740 1 client.16122 shutdown 2024-09-13T18:23:39.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.520+0000 7fb6a43ec740 1 -- 172.21.15.114:0/537765480 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561781a4b0b0 msgr2=0x561781a5b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:39.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.520+0000 7fb6a43ec740 1 --2- 172.21.15.114:0/537765480 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561781a4b0b0 0x561781a5b630 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fb68c010490 tx=0x7fb68c0135a0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.520+0000 7fb6a43ec740 1 -- 172.21.15.114:0/537765480 shutdown_connections 2024-09-13T18:23:39.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.520+0000 7fb6a43ec740 1 --2- 172.21.15.114:0/537765480 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x561781abcee0 0x561781abf2d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.520+0000 7fb6a43ec740 1 --2- 172.21.15.114:0/537765480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561781a55950 0x561781a5bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.520+0000 7fb6827fc640 0 client.16122 ms_handle_reset on v2:172.21.15.204:6838/344883912 2024-09-13T18:23:39.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.520+0000 7fb6a43ec740 1 --2- 172.21.15.114:0/537765480 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561781a4b0b0 0x561781a5b630 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.520+0000 7fb6a43ec740 1 --2- 172.21.15.114:0/537765480 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561781a4a790 0x561781a5b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.520+0000 7fb6a43ec740 1 -- 172.21.15.114:0/537765480 >> 172.21.15.114:0/537765480 conn(0x561781a46d00 msgr2=0x561781a52020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:39.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.520+0000 7fb6a43ec740 1 -- 172.21.15.114:0/537765480 shutdown_connections 2024-09-13T18:23:39.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.520+0000 7fb6a43ec740 1 -- 172.21.15.114:0/537765480 wait complete. 2024-09-13T18:23:39.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.520+0000 7fb6a43ec740 20 client.16122 trim_cache size 0 max 16384 2024-09-13T18:23:39.535 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:23:39.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.520+0000 7fb6a16c9640 1 -- 172.21.15.114:0/2797847447 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fb6880a1350 msgr2=0x7fb6880a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:39.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.520+0000 7fb6a16c9640 1 --2- 172.21.15.114:0/2797847447 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fb6880a1350 0x7fb6880a3810 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb68c0058c0 tx=0x7fb68c00b960 comp rx=0 tx=0).stop 2024-09-13T18:23:39.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.520+0000 7fb6a16c9640 1 -- 172.21.15.114:0/2797847447 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb69c10ce80 msgr2=0x7fb69c1a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:39.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.520+0000 7fb6a16c9640 1 --2- 172.21.15.114:0/2797847447 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb69c10ce80 0x7fb69c1a4350 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fb694004af0 tx=0x7fb694030e50 comp rx=0 tx=0).stop 2024-09-13T18:23:39.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.520+0000 7fb6a16c9640 1 -- 172.21.15.114:0/2797847447 shutdown_connections 2024-09-13T18:23:39.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.520+0000 7fb6a16c9640 1 --2- 172.21.15.114:0/2797847447 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fb6880a1350 0x7fb6880a3810 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.520+0000 7fb6a16c9640 1 --2- 172.21.15.114:0/2797847447 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb69c10ce80 0x7fb69c1a4350 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.520+0000 7fb6a16c9640 1 --2- 172.21.15.114:0/2797847447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb69c103fa0 0x7fb69c10c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.520+0000 7fb6a16c9640 1 --2- 172.21.15.114:0/2797847447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb69c0fee50 0x7fb69c10c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.520+0000 7fb6a16c9640 1 -- 172.21.15.114:0/2797847447 >> 172.21.15.114:0/2797847447 conn(0x7fb69c0fa8e0 msgr2=0x7fb69c0fbcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:39.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.520+0000 7fb6a16c9640 1 -- 172.21.15.114:0/2797847447 shutdown_connections 2024-09-13T18:23:39.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.520+0000 7fb6a16c9640 1 -- 172.21.15.114:0/2797847447 wait complete. 2024-09-13T18:23:39.538 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-09-13T18:23:39.589 INFO:tasks.workunit.client.0.smithi114.stderr:Note: switching to '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3'. 2024-09-13T18:23:39.589 INFO:tasks.workunit.client.0.smithi114.stderr: 2024-09-13T18:23:39.589 INFO:tasks.workunit.client.0.smithi114.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-13T18:23:39.589 INFO:tasks.workunit.client.0.smithi114.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-13T18:23:39.589 INFO:tasks.workunit.client.0.smithi114.stderr:state without impacting any branches by switching back to a branch. 2024-09-13T18:23:39.589 INFO:tasks.workunit.client.0.smithi114.stderr: 2024-09-13T18:23:39.589 INFO:tasks.workunit.client.0.smithi114.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-13T18:23:39.589 INFO:tasks.workunit.client.0.smithi114.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-13T18:23:39.590 INFO:tasks.workunit.client.0.smithi114.stderr: 2024-09-13T18:23:39.590 INFO:tasks.workunit.client.0.smithi114.stderr: git switch -c 2024-09-13T18:23:39.590 INFO:tasks.workunit.client.0.smithi114.stderr: 2024-09-13T18:23:39.590 INFO:tasks.workunit.client.0.smithi114.stderr:Or undo this operation with: 2024-09-13T18:23:39.590 INFO:tasks.workunit.client.0.smithi114.stderr: 2024-09-13T18:23:39.590 INFO:tasks.workunit.client.0.smithi114.stderr: git switch - 2024-09-13T18:23:39.590 INFO:tasks.workunit.client.0.smithi114.stderr: 2024-09-13T18:23:39.590 INFO:tasks.workunit.client.0.smithi114.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-13T18:23:39.590 INFO:tasks.workunit.client.0.smithi114.stderr: 2024-09-13T18:23:39.590 INFO:tasks.workunit.client.0.smithi114.stderr:HEAD is now at 6c6b083f148 Merge PR #56690 into wip-rishabh-testing-20240910.100730-reef 2024-09-13T18:23:39.595 DEBUG:teuthology.orchestra.run.smithi114:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-13T18:23:39.619 INFO:tasks.workunit.client.0.smithi114.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-13T18:23:39.621 INFO:tasks.workunit.client.0.smithi114.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-13T18:23:39.621 INFO:tasks.workunit.client.0.smithi114.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-13T18:23:39.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.640+0000 7f7e45725640 1 -- 172.21.15.114:0/147068493 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7e40106420 msgr2=0x7f7e4010cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:39.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.640+0000 7f7e45725640 1 --2- 172.21.15.114:0/147068493 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7e40106420 0x7f7e4010cd70 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f7e38009d00 tx=0x7f7e3802a9a0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.640+0000 7f7e45725640 1 -- 172.21.15.114:0/147068493 shutdown_connections 2024-09-13T18:23:39.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.640+0000 7f7e45725640 1 --2- 172.21.15.114:0/147068493 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7e40106420 0x7f7e4010cd70 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.640+0000 7f7e45725640 1 --2- 172.21.15.114:0/147068493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e400ff480 0x7f7e40105cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.640+0000 7f7e45725640 1 --2- 172.21.15.114:0/147068493 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e400feb60 0x7f7e400fef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.640+0000 7f7e45725640 1 -- 172.21.15.114:0/147068493 >> 172.21.15.114:0/147068493 conn(0x7f7e400741b0 msgr2=0x7f7e400745c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:39.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.640+0000 7f7e45725640 1 -- 172.21.15.114:0/147068493 shutdown_connections 2024-09-13T18:23:39.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.640+0000 7f7e45725640 1 -- 172.21.15.114:0/147068493 wait complete. 2024-09-13T18:23:39.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.641+0000 7f7e45725640 1 Processor -- start 2024-09-13T18:23:39.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.641+0000 7f7e45725640 1 -- start start 2024-09-13T18:23:39.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.641+0000 7f7e45725640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7e400feb60 0x7f7e400ffad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.641+0000 7f7e45725640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e400ff480 0x7f7e40100010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.641+0000 7f7e45725640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e40106420 0x7f7e40100550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.641+0000 7f7e45725640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e401a45d0 con 0x7f7e400feb60 2024-09-13T18:23:39.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.641+0000 7f7e45725640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e401a4740 con 0x7f7e400ff480 2024-09-13T18:23:39.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.641+0000 7f7e45725640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e401a4900 con 0x7f7e40106420 2024-09-13T18:23:39.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.641+0000 7f7e3effd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7e400feb60 0x7f7e400ffad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.641+0000 7f7e3effd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7e400feb60 0x7f7e400ffad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51482/0 (socket says 172.21.15.114:51482) 2024-09-13T18:23:39.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.641+0000 7f7e3effd640 1 -- 172.21.15.114:0/3230612252 learned_addr learned my addr 172.21.15.114:0/3230612252 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:39.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.641+0000 7f7e3e7fc640 1 --2- 172.21.15.114:0/3230612252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e400ff480 0x7f7e40100010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.642+0000 7f7e3effd640 1 -- 172.21.15.114:0/3230612252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e40106420 msgr2=0x7f7e40100550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:39.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.642+0000 7f7e3effd640 1 --2- 172.21.15.114:0/3230612252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e40106420 0x7f7e40100550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.642+0000 7f7e3effd640 1 -- 172.21.15.114:0/3230612252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e400ff480 msgr2=0x7f7e40100010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:39.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.642+0000 7f7e3effd640 1 --2- 172.21.15.114:0/3230612252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e400ff480 0x7f7e40100010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.642+0000 7f7e3effd640 1 -- 172.21.15.114:0/3230612252 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e38009900 con 0x7f7e400feb60 2024-09-13T18:23:39.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.642+0000 7f7e3effd640 1 --2- 172.21.15.114:0/3230612252 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7e400feb60 0x7f7e400ffad0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f7e2800c970 tx=0x7f7e2800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:39.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.642+0000 7f7e37fff640 1 -- 172.21.15.114:0/3230612252 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e28007bb0 con 0x7f7e400feb60 2024-09-13T18:23:39.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.642+0000 7f7e45725640 1 -- 172.21.15.114:0/3230612252 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e401a4be0 con 0x7f7e400feb60 2024-09-13T18:23:39.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.642+0000 7f7e37fff640 1 -- 172.21.15.114:0/3230612252 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7e28007d10 con 0x7f7e400feb60 2024-09-13T18:23:39.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.642+0000 7f7e37fff640 1 -- 172.21.15.114:0/3230612252 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e28004d70 con 0x7f7e400feb60 2024-09-13T18:23:39.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.642+0000 7f7e45725640 1 -- 172.21.15.114:0/3230612252 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e401a50a0 con 0x7f7e400feb60 2024-09-13T18:23:39.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.644+0000 7f7e37fff640 1 -- 172.21.15.114:0/3230612252 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) v1 ==== 136107+0+0 (secure 0 0 0) 0x7f7e28004f10 con 0x7f7e400feb60 2024-09-13T18:23:39.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.644+0000 7f7e48451740 1 Processor -- start 2024-09-13T18:23:39.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.645+0000 7f7e48451740 1 -- start start 2024-09-13T18:23:39.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.645+0000 7f7e48451740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dff6ae9790 0x55dff6ae9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.645+0000 7f7e48451740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dff6aea0b0 0x55dff6af4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.645+0000 7f7e48451740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff6af4950 0x55dff6af6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.645+0000 7f7e48451740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dff6a53cc0 con 0x55dff6ae9790 2024-09-13T18:23:39.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.645+0000 7f7e48451740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dff6a51450 con 0x55dff6af4950 2024-09-13T18:23:39.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.645+0000 7f7e48451740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dff6a4cd40 con 0x55dff6aea0b0 2024-09-13T18:23:39.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.645+0000 7f7e3f7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dff6ae9790 0x55dff6ae9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.645+0000 7f7e3f7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dff6ae9790 0x55dff6ae9b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51494/0 (socket says 172.21.15.114:51494) 2024-09-13T18:23:39.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.645+0000 7f7e3f7fe640 1 -- 172.21.15.114:0/2440693882 learned_addr learned my addr 172.21.15.114:0/2440693882 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:39.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.645+0000 7f7e3effd640 1 --2- 172.21.15.114:0/2440693882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dff6aea0b0 0x55dff6af4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.645+0000 7f7e37fff640 1 --2- 172.21.15.114:0/3230612252 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7e2c0a1350 0x7f7e2c0a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.645+0000 7f7e3e7fc640 1 --2- 172.21.15.114:0/2440693882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff6af4950 0x55dff6af6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.645+0000 7f7e3f7fe640 1 -- 172.21.15.114:0/2440693882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dff6aea0b0 msgr2=0x55dff6af4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:39.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.645+0000 7f7e3f7fe640 1 --2- 172.21.15.114:0/2440693882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dff6aea0b0 0x55dff6af4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.645+0000 7f7e3f7fe640 1 -- 172.21.15.114:0/2440693882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff6af4950 msgr2=0x55dff6af6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:39.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.645+0000 7f7e3f7fe640 1 --2- 172.21.15.114:0/2440693882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff6af4950 0x55dff6af6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.645+0000 7f7e3f7fe640 1 -- 172.21.15.114:0/2440693882 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dff6a14620 con 0x55dff6ae9790 2024-09-13T18:23:39.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.645+0000 7f7e3e7fc640 1 --2- 172.21.15.114:0/2440693882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff6af4950 0x55dff6af6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:23:39.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.645+0000 7f7e37fff640 1 -- 172.21.15.114:0/3230612252 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f7e280cd0a0 con 0x7f7e400feb60 2024-09-13T18:23:39.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.646+0000 7f7e3f7fe640 1 --2- 172.21.15.114:0/3230612252 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7e2c0a1350 0x7f7e2c0a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.646+0000 7f7e3f7fe640 1 --2- 172.21.15.114:0/2440693882 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dff6ae9790 0x55dff6ae9b70 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f7e380098d0 tx=0x7f7e3802e3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f409fe1fa813bd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:39.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.646+0000 7f7e357fa640 1 -- 172.21.15.114:0/2440693882 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e38030ab0 con 0x55dff6ae9790 2024-09-13T18:23:39.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.646+0000 7f7e3f7fe640 1 --2- 172.21.15.114:0/3230612252 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7e2c0a1350 0x7f7e2c0a3810 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7e3804c000 tx=0x7f7e3804f000 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:39.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.646+0000 7f7e357fa640 1 -- 172.21.15.114:0/2440693882 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7e38030c10 con 0x55dff6ae9790 2024-09-13T18:23:39.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.646+0000 7f7e357fa640 1 -- 172.21.15.114:0/2440693882 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e38039650 con 0x55dff6ae9790 2024-09-13T18:23:39.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.647+0000 7f7e48451740 1 -- 172.21.15.114:0/2440693882 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dff6ae9790 msgr2=0x55dff6ae9b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:39.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.647+0000 7f7e48451740 1 --2- 172.21.15.114:0/2440693882 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dff6ae9790 0x55dff6ae9b70 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f7e380098d0 tx=0x7f7e3802e3e0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.647+0000 7f7e48451740 1 -- 172.21.15.114:0/2440693882 shutdown_connections 2024-09-13T18:23:39.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.647+0000 7f7e48451740 1 --2- 172.21.15.114:0/2440693882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff6af4950 0x55dff6af6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.647+0000 7f7e48451740 1 --2- 172.21.15.114:0/2440693882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dff6aea0b0 0x55dff6af4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.647+0000 7f7e48451740 1 --2- 172.21.15.114:0/2440693882 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dff6ae9790 0x55dff6ae9b70 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.647+0000 7f7e48451740 1 -- 172.21.15.114:0/2440693882 >> 172.21.15.114:0/2440693882 conn(0x55dff6ae5d00 msgr2=0x55dff6ae60e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:39.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.647+0000 7f7e48451740 1 -- 172.21.15.114:0/2440693882 shutdown_connections 2024-09-13T18:23:39.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.647+0000 7f7e48451740 1 -- 172.21.15.114:0/2440693882 wait complete. 2024-09-13T18:23:39.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.647+0000 7f7e48451740 1 -- start start 2024-09-13T18:23:39.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.647+0000 7f7e48451740 1 Processor -- start 2024-09-13T18:23:39.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.647+0000 7f7e48451740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff6ae9790 0x55dff6af3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.647+0000 7f7e48451740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dff6aea0b0 0x55dff6af13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.647+0000 7f7e48451740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dff6af4950 0x55dff6af18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.647+0000 7f7e48451740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dff6a0b180 con 0x55dff6aea0b0 2024-09-13T18:23:39.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.647+0000 7f7e48451740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dff6725430 con 0x55dff6ae9790 2024-09-13T18:23:39.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.647+0000 7f7e48451740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dff69d05e0 con 0x55dff6af4950 2024-09-13T18:23:39.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.647+0000 7f7e48451740 10 client.0 _ll_register_callbacks cb 0x55dff6a3caa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:23:39.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.647+0000 7f7e3e7fc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dff6aea0b0 0x55dff6af13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.648+0000 7f7e3e7fc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dff6aea0b0 0x55dff6af13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51500/0 (socket says 172.21.15.114:51500) 2024-09-13T18:23:39.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.648+0000 7f7e3e7fc640 1 -- 172.21.15.114:0/3289951796 learned_addr learned my addr 172.21.15.114:0/3289951796 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:39.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.648+0000 7f7e3effd640 1 --2- 172.21.15.114:0/3289951796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff6ae9790 0x55dff6af3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.648+0000 7f7e3f7fe640 1 --2- 172.21.15.114:0/3289951796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dff6af4950 0x55dff6af18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.648+0000 7f7e3e7fc640 1 -- 172.21.15.114:0/3289951796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dff6af4950 msgr2=0x55dff6af18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:39.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.648+0000 7f7e3e7fc640 1 --2- 172.21.15.114:0/3289951796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dff6af4950 0x55dff6af18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.648+0000 7f7e3e7fc640 1 -- 172.21.15.114:0/3289951796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff6ae9790 msgr2=0x55dff6af3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:39.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.648+0000 7f7e3e7fc640 1 --2- 172.21.15.114:0/3289951796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff6ae9790 0x55dff6af3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.648+0000 7f7e3e7fc640 1 -- 172.21.15.114:0/3289951796 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e4010d580 con 0x55dff6aea0b0 2024-09-13T18:23:39.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.648+0000 7f7e3e7fc640 1 --2- 172.21.15.114:0/3289951796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dff6aea0b0 0x55dff6af13a0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f7e300107f0 tx=0x7f7e30010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=586a7e8a0d8a932e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:39.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.648+0000 7f7e267fc640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:23:39.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.648+0000 7f7e267fc640 1 -- 172.21.15.114:0/3289951796 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3000cb10 con 0x55dff6aea0b0 2024-09-13T18:23:39.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.648+0000 7f7e267fc640 1 -- 172.21.15.114:0/3289951796 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f7e3000cc70 con 0x55dff6aea0b0 2024-09-13T18:23:39.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.648+0000 7f7e48451740 1 -- 172.21.15.114:0/3289951796 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dff6a76b80 con 0x55dff6aea0b0 2024-09-13T18:23:39.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.648+0000 7f7e267fc640 1 -- 172.21.15.114:0/3289951796 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e3000ce30 con 0x55dff6aea0b0 2024-09-13T18:23:39.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.648+0000 7f7e48451740 10 client.16134 fetch_fsmap learned FSMap version 30 2024-09-13T18:23:39.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.648+0000 7f7e48451740 1 -- 172.21.15.114:0/3289951796 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55dff6a4b3c0 con 0x55dff6aea0b0 2024-09-13T18:23:39.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.648+0000 7f7e267fc640 1 -- 172.21.15.114:0/3289951796 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55dff6a76b80 con 0x55dff6aea0b0 2024-09-13T18:23:39.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.648+0000 7f7e267fc640 1 -- 172.21.15.114:0/3289951796 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7e3001a4c0 con 0x55dff6aea0b0 2024-09-13T18:23:39.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.649+0000 7f7e48451740 10 client.16134 fetch_fsmap finished waiting for FSMap version 30 2024-09-13T18:23:39.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.649+0000 7f7e48451740 10 client.16134 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-13T18:23:39.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.649+0000 7f7e48451740 20 client.16134 populate_metadata read hostname 'smithi114' 2024-09-13T18:23:39.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.649+0000 7f7e48451740 1 --2- 172.21.15.114:0/3289951796 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55dff6b043d0 0x55dff6b067f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.649+0000 7f7e48451740 4 client.16134 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:23:39.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.649+0000 7f7e48451740 1 -- 172.21.15.114:0/3289951796 --> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dff6a3f1f0 con 0x55dff6b043d0 2024-09-13T18:23:39.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.649+0000 7f7e3effd640 1 --2- 172.21.15.114:0/3289951796 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55dff6b043d0 0x55dff6b067f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.649+0000 7f7e267fc640 10 client.16134 ms_handle_connect on v2:172.21.15.134:6836/891692210 2024-09-13T18:23:39.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.649+0000 7f7e3effd640 1 --2- 172.21.15.114:0/3289951796 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55dff6b043d0 0x55dff6b067f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=226e83ec8174ce7b server_cookie=d3193759eca0770a in_seq=0 out_seq=0 2024-09-13T18:23:39.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.650+0000 7f7e267fc640 1 -- 172.21.15.114:0/3289951796 <== mds.1 v2:172.21.15.134:6836/891692210 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55dff6a3f1f0 con 0x55dff6b043d0 2024-09-13T18:23:39.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.650+0000 7f7e267fc640 10 client.16134 handle_command_reply: tid=0 2024-09-13T18:23:39.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.650+0000 7f7e48451740 1 client.16134 shutdown 2024-09-13T18:23:39.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.650+0000 7f7e48451740 1 -- 172.21.15.114:0/3289951796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dff6aea0b0 msgr2=0x55dff6af13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:39.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.650+0000 7f7e48451740 1 --2- 172.21.15.114:0/3289951796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dff6aea0b0 0x55dff6af13a0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f7e300107f0 tx=0x7f7e30010cb0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.650+0000 7f7e48451740 1 -- 172.21.15.114:0/3289951796 shutdown_connections 2024-09-13T18:23:39.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.650+0000 7f7e48451740 1 --2- 172.21.15.114:0/3289951796 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55dff6b043d0 0x55dff6b067f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.650+0000 7f7e48451740 1 --2- 172.21.15.114:0/3289951796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dff6af4950 0x55dff6af18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.650+0000 7f7e48451740 1 --2- 172.21.15.114:0/3289951796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dff6aea0b0 0x55dff6af13a0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.650+0000 7f7e48451740 1 --2- 172.21.15.114:0/3289951796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff6ae9790 0x55dff6af3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.650+0000 7f7e48451740 1 -- 172.21.15.114:0/3289951796 >> 172.21.15.114:0/3289951796 conn(0x55dff6ae5d00 msgr2=0x55dff6ae4eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:39.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.650+0000 7f7e267fc640 0 client.16134 ms_handle_reset on v2:172.21.15.134:6836/891692210 2024-09-13T18:23:39.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.650+0000 7f7e48451740 1 -- 172.21.15.114:0/3289951796 shutdown_connections 2024-09-13T18:23:39.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.650+0000 7f7e48451740 1 -- 172.21.15.114:0/3289951796 wait complete. 2024-09-13T18:23:39.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.650+0000 7f7e48451740 20 client.16134 trim_cache size 0 max 16384 2024-09-13T18:23:39.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.661+0000 7f7e48451740 1 Processor -- start 2024-09-13T18:23:39.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.661+0000 7f7e48451740 1 -- start start 2024-09-13T18:23:39.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.661+0000 7f7e48451740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dff6ae5d00 0x55dff6ae56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.661+0000 7f7e48451740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dff6ae9790 0x55dff6af13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.661+0000 7f7e48451740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff6aea0b0 0x55dff6af18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.661+0000 7f7e48451740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dff6aea750 con 0x55dff6ae5d00 2024-09-13T18:23:39.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.661+0000 7f7e48451740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dff6aef8b0 con 0x55dff6aea0b0 2024-09-13T18:23:39.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.661+0000 7f7e48451740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dff6ae9580 con 0x55dff6ae9790 2024-09-13T18:23:39.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.661+0000 7f7e3effd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dff6ae5d00 0x55dff6ae56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.661+0000 7f7e3effd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dff6ae5d00 0x55dff6ae56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51510/0 (socket says 172.21.15.114:51510) 2024-09-13T18:23:39.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.661+0000 7f7e3effd640 1 -- 172.21.15.114:0/620786626 learned_addr learned my addr 172.21.15.114:0/620786626 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:39.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.662+0000 7f7e3effd640 1 -- 172.21.15.114:0/620786626 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dff6ae9790 msgr2=0x55dff6af13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:39.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.662+0000 7f7e3effd640 1 --2- 172.21.15.114:0/620786626 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dff6ae9790 0x55dff6af13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.662+0000 7f7e3effd640 1 -- 172.21.15.114:0/620786626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff6aea0b0 msgr2=0x55dff6af18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:39.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.662+0000 7f7e3effd640 1 --2- 172.21.15.114:0/620786626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff6aea0b0 0x55dff6af18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.662+0000 7f7e3effd640 1 -- 172.21.15.114:0/620786626 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dff69ed770 con 0x55dff6ae5d00 2024-09-13T18:23:39.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.662+0000 7f7e3effd640 1 --2- 172.21.15.114:0/620786626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dff6ae5d00 0x55dff6ae56a0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f7e28099ba0 tx=0x7f7e2808d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bca556b12351eec server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:39.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.662+0000 7f7e35ffb640 1 -- 172.21.15.114:0/620786626 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e280046c0 con 0x55dff6ae5d00 2024-09-13T18:23:39.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.662+0000 7f7e35ffb640 1 -- 172.21.15.114:0/620786626 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7e2809a310 con 0x55dff6ae5d00 2024-09-13T18:23:39.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.662+0000 7f7e35ffb640 1 -- 172.21.15.114:0/620786626 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e28095a00 con 0x55dff6ae5d00 2024-09-13T18:23:39.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.663+0000 7f7e48451740 1 -- 172.21.15.114:0/620786626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dff6ae5d00 msgr2=0x55dff6ae56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:39.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.663+0000 7f7e48451740 1 --2- 172.21.15.114:0/620786626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dff6ae5d00 0x55dff6ae56a0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f7e28099ba0 tx=0x7f7e2808d040 comp rx=0 tx=0).stop 2024-09-13T18:23:39.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.663+0000 7f7e48451740 1 -- 172.21.15.114:0/620786626 shutdown_connections 2024-09-13T18:23:39.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.663+0000 7f7e48451740 1 --2- 172.21.15.114:0/620786626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff6aea0b0 0x55dff6af18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.663+0000 7f7e48451740 1 --2- 172.21.15.114:0/620786626 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dff6ae9790 0x55dff6af13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.663+0000 7f7e48451740 1 --2- 172.21.15.114:0/620786626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dff6ae5d00 0x55dff6ae56a0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.663+0000 7f7e48451740 1 -- 172.21.15.114:0/620786626 >> 172.21.15.114:0/620786626 conn(0x55dff6af40c0 msgr2=0x55dff6b0b150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:39.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.663+0000 7f7e48451740 1 -- 172.21.15.114:0/620786626 shutdown_connections 2024-09-13T18:23:39.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.663+0000 7f7e48451740 1 -- 172.21.15.114:0/620786626 wait complete. 2024-09-13T18:23:39.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.663+0000 7f7e48451740 1 -- start start 2024-09-13T18:23:39.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.663+0000 7f7e48451740 1 Processor -- start 2024-09-13T18:23:39.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.663+0000 7f7e48451740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dff6ae5d00 0x55dff6afbba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.663+0000 7f7e48451740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dff6ae9790 0x55dff6aedfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.663+0000 7f7e48451740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff6aea0b0 0x55dff6aee6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.663+0000 7f7e48451740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dff6aeedf0 con 0x55dff6ae9790 2024-09-13T18:23:39.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.663+0000 7f7e48451740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dff6b5a8d0 con 0x55dff6aea0b0 2024-09-13T18:23:39.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.663+0000 7f7e48451740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dff6b5aa40 con 0x55dff6ae5d00 2024-09-13T18:23:39.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.663+0000 7f7e48451740 10 client.0 _ll_register_callbacks cb 0x7f7e38009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:23:39.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.663+0000 7f7e3e7fc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dff6ae9790 0x55dff6aedfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.663+0000 7f7e3e7fc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dff6ae9790 0x55dff6aedfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51520/0 (socket says 172.21.15.114:51520) 2024-09-13T18:23:39.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.663+0000 7f7e3e7fc640 1 -- 172.21.15.114:0/4288620129 learned_addr learned my addr 172.21.15.114:0/4288620129 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:39.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.664+0000 7f7e3f7fe640 1 --2- 172.21.15.114:0/4288620129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff6aea0b0 0x55dff6aee6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.664+0000 7f7e3effd640 1 --2- 172.21.15.114:0/4288620129 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dff6ae5d00 0x55dff6afbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.664+0000 7f7e3e7fc640 1 -- 172.21.15.114:0/4288620129 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dff6ae5d00 msgr2=0x55dff6afbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:39.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.664+0000 7f7e3e7fc640 1 --2- 172.21.15.114:0/4288620129 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dff6ae5d00 0x55dff6afbba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.664+0000 7f7e3e7fc640 1 -- 172.21.15.114:0/4288620129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff6aea0b0 msgr2=0x55dff6aee6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:39.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.664+0000 7f7e3e7fc640 1 --2- 172.21.15.114:0/4288620129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff6aea0b0 0x55dff6aee6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.664+0000 7f7e3e7fc640 1 -- 172.21.15.114:0/4288620129 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e401a50a0 con 0x55dff6ae9790 2024-09-13T18:23:39.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.664+0000 7f7e3f7fe640 1 --2- 172.21.15.114:0/4288620129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff6aea0b0 0x55dff6aee6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:23:39.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.664+0000 7f7e3e7fc640 1 --2- 172.21.15.114:0/4288620129 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dff6ae9790 0x55dff6aedfb0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7e3001a970 tx=0x7f7e3001a6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d9a1d276b7a111f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:39.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.664+0000 7f7e267fc640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:23:39.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.664+0000 7f7e267fc640 1 -- 172.21.15.114:0/4288620129 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3001aea0 con 0x55dff6ae9790 2024-09-13T18:23:39.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.664+0000 7f7e267fc640 1 -- 172.21.15.114:0/4288620129 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f7e3000c910 con 0x55dff6ae9790 2024-09-13T18:23:39.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.664+0000 7f7e48451740 1 -- 172.21.15.114:0/4288620129 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7e38030c10 con 0x55dff6ae9790 2024-09-13T18:23:39.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.664+0000 7f7e267fc640 1 -- 172.21.15.114:0/4288620129 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e3000c700 con 0x55dff6ae9790 2024-09-13T18:23:39.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.664+0000 7f7e48451740 10 client.16140 fetch_fsmap learned FSMap version 30 2024-09-13T18:23:39.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.664+0000 7f7e48451740 1 -- 172.21.15.114:0/4288620129 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55dff6a12e80 con 0x55dff6ae9790 2024-09-13T18:23:39.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.664+0000 7f7e267fc640 1 -- 172.21.15.114:0/4288620129 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7e38030c10 con 0x55dff6ae9790 2024-09-13T18:23:39.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.664+0000 7f7e267fc640 1 -- 172.21.15.114:0/4288620129 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7e300035f0 con 0x55dff6ae9790 2024-09-13T18:23:39.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.664+0000 7f7e48451740 10 client.16140 fetch_fsmap finished waiting for FSMap version 30 2024-09-13T18:23:39.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.664+0000 7f7e48451740 10 client.16140 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-13T18:23:39.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.665+0000 7f7e48451740 20 client.16140 populate_metadata read hostname 'smithi114' 2024-09-13T18:23:39.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.665+0000 7f7e48451740 1 --2- 172.21.15.114:0/4288620129 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55dff6af4950 0x55dff6b5e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.665+0000 7f7e48451740 4 client.16140 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:23:39.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.665+0000 7f7e48451740 1 -- 172.21.15.114:0/4288620129 --> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dff67404e0 con 0x55dff6af4950 2024-09-13T18:23:39.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.665+0000 7f7e3effd640 1 --2- 172.21.15.114:0/4288620129 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55dff6af4950 0x55dff6b5e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.665+0000 7f7e267fc640 10 client.16140 ms_handle_connect on v2:172.21.15.134:6836/891692210 2024-09-13T18:23:39.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.665+0000 7f7e3effd640 1 --2- 172.21.15.114:0/4288620129 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55dff6af4950 0x55dff6b5e2e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e7f94cca205f5079 server_cookie=35fe982ae376b81d in_seq=0 out_seq=0 2024-09-13T18:23:39.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.665+0000 7f7e267fc640 1 -- 172.21.15.114:0/4288620129 <== mds.1 v2:172.21.15.134:6836/891692210 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dff67404e0 con 0x55dff6af4950 2024-09-13T18:23:39.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.665+0000 7f7e267fc640 10 client.16140 handle_command_reply: tid=0 2024-09-13T18:23:39.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.665+0000 7f7e48451740 1 client.16140 shutdown 2024-09-13T18:23:39.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.665+0000 7f7e48451740 1 -- 172.21.15.114:0/4288620129 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dff6ae9790 msgr2=0x55dff6aedfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:39.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.665+0000 7f7e48451740 1 --2- 172.21.15.114:0/4288620129 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dff6ae9790 0x55dff6aedfb0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7e3001a970 tx=0x7f7e3001a6e0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.665+0000 7f7e48451740 1 -- 172.21.15.114:0/4288620129 shutdown_connections 2024-09-13T18:23:39.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.665+0000 7f7e48451740 1 --2- 172.21.15.114:0/4288620129 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55dff6af4950 0x55dff6b5e2e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.665+0000 7f7e48451740 1 --2- 172.21.15.114:0/4288620129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff6aea0b0 0x55dff6aee6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.665+0000 7f7e267fc640 0 client.16140 ms_handle_reset on v2:172.21.15.134:6836/891692210 2024-09-13T18:23:39.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.665+0000 7f7e48451740 1 --2- 172.21.15.114:0/4288620129 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dff6ae9790 0x55dff6aedfb0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.665+0000 7f7e48451740 1 --2- 172.21.15.114:0/4288620129 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dff6ae5d00 0x55dff6afbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.666+0000 7f7e48451740 1 -- 172.21.15.114:0/4288620129 >> 172.21.15.114:0/4288620129 conn(0x55dff6af40c0 msgr2=0x55dff6af1380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:39.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.666+0000 7f7e48451740 1 -- 172.21.15.114:0/4288620129 shutdown_connections 2024-09-13T18:23:39.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.666+0000 7f7e48451740 1 -- 172.21.15.114:0/4288620129 wait complete. 2024-09-13T18:23:39.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.666+0000 7f7e48451740 20 client.16140 trim_cache size 0 max 16384 2024-09-13T18:23:39.684 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:23:39.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.666+0000 7f7e45725640 1 -- 172.21.15.114:0/3230612252 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7e2c0a1350 msgr2=0x7f7e2c0a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:39.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.666+0000 7f7e45725640 1 --2- 172.21.15.114:0/3230612252 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7e2c0a1350 0x7f7e2c0a3810 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7e3804c000 tx=0x7f7e3804f000 comp rx=0 tx=0).stop 2024-09-13T18:23:39.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.666+0000 7f7e45725640 1 -- 172.21.15.114:0/3230612252 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7e400feb60 msgr2=0x7f7e400ffad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:39.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.666+0000 7f7e45725640 1 --2- 172.21.15.114:0/3230612252 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7e400feb60 0x7f7e400ffad0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f7e2800c970 tx=0x7f7e2800ce30 comp rx=0 tx=0).stop 2024-09-13T18:23:39.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.666+0000 7f7e45725640 1 -- 172.21.15.114:0/3230612252 shutdown_connections 2024-09-13T18:23:39.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.666+0000 7f7e45725640 1 --2- 172.21.15.114:0/3230612252 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7e2c0a1350 0x7f7e2c0a3810 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.666+0000 7f7e45725640 1 --2- 172.21.15.114:0/3230612252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e40106420 0x7f7e40100550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.666+0000 7f7e45725640 1 --2- 172.21.15.114:0/3230612252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e400ff480 0x7f7e40100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.666+0000 7f7e45725640 1 --2- 172.21.15.114:0/3230612252 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7e400feb60 0x7f7e400ffad0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.666+0000 7f7e45725640 1 -- 172.21.15.114:0/3230612252 >> 172.21.15.114:0/3230612252 conn(0x7f7e400741b0 msgr2=0x7f7e40103f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:39.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.666+0000 7f7e45725640 1 -- 172.21.15.114:0/3230612252 shutdown_connections 2024-09-13T18:23:39.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.666+0000 7f7e45725640 1 -- 172.21.15.114:0/3230612252 wait complete. 2024-09-13T18:23:39.686 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:23:39 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/1707115762' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:23:39.687 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-09-13T18:23:39.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:23:39 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/1707115762' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:23:39.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.828+0000 7fd920f11640 1 -- 172.21.15.114:0/2267041974 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd91c0fee80 msgr2=0x7fd91c0ff260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:39.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.828+0000 7fd920f11640 1 --2- 172.21.15.114:0/2267041974 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd91c0fee80 0x7fd91c0ff260 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fd914009cd0 tx=0x7fd91402a8e0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.828+0000 7fd920f11640 1 -- 172.21.15.114:0/2267041974 shutdown_connections 2024-09-13T18:23:39.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.828+0000 7fd920f11640 1 --2- 172.21.15.114:0/2267041974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd91c103fd0 0x7fd91c10bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.828+0000 7fd920f11640 1 --2- 172.21.15.114:0/2267041974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd91c0ff830 0x7fd91c103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.828+0000 7fd920f11640 1 --2- 172.21.15.114:0/2267041974 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd91c0fee80 0x7fd91c0ff260 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.828+0000 7fd920f11640 1 -- 172.21.15.114:0/2267041974 >> 172.21.15.114:0/2267041974 conn(0x7fd91c0fa930 msgr2=0x7fd91c0fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:39.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.828+0000 7fd920f11640 1 -- 172.21.15.114:0/2267041974 shutdown_connections 2024-09-13T18:23:39.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.828+0000 7fd920f11640 1 -- 172.21.15.114:0/2267041974 wait complete. 2024-09-13T18:23:39.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.829+0000 7fd920f11640 1 Processor -- start 2024-09-13T18:23:39.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.829+0000 7fd920f11640 1 -- start start 2024-09-13T18:23:39.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.829+0000 7fd920f11640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd91c0fee80 0x7fd91c1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.829+0000 7fd920f11640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd91c103fd0 0x7fd91c198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.829+0000 7fd920f11640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd91c19cd20 0x7fd91c19d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.829+0000 7fd920f11640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c19d710 con 0x7fd91c0fee80 2024-09-13T18:23:39.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.829+0000 7fd920f11640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c19d880 con 0x7fd91c19cd20 2024-09-13T18:23:39.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.829+0000 7fd920f11640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c19d9f0 con 0x7fd91c103fd0 2024-09-13T18:23:39.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.829+0000 7fd91a59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd91c0fee80 0x7fd91c1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.829+0000 7fd91a59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd91c0fee80 0x7fd91c1983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51542/0 (socket says 172.21.15.114:51542) 2024-09-13T18:23:39.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.829+0000 7fd91a59c640 1 -- 172.21.15.114:0/3613621194 learned_addr learned my addr 172.21.15.114:0/3613621194 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:39.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.829+0000 7fd91ad9d640 1 --2- 172.21.15.114:0/3613621194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd91c19cd20 0x7fd91c19d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.830+0000 7fd91a59c640 1 -- 172.21.15.114:0/3613621194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd91c103fd0 msgr2=0x7fd91c198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:39.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.830+0000 7fd91a59c640 1 --2- 172.21.15.114:0/3613621194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd91c103fd0 0x7fd91c198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.830+0000 7fd91a59c640 1 -- 172.21.15.114:0/3613621194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd91c19cd20 msgr2=0x7fd91c19d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:39.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.830+0000 7fd91a59c640 1 --2- 172.21.15.114:0/3613621194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd91c19cd20 0x7fd91c19d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.830+0000 7fd91a59c640 1 -- 172.21.15.114:0/3613621194 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd914009900 con 0x7fd91c0fee80 2024-09-13T18:23:39.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.830+0000 7fd91a59c640 1 --2- 172.21.15.114:0/3613621194 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd91c0fee80 0x7fd91c1983f0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fd914004af0 tx=0x7fd914030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:39.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.830+0000 7fd9137fe640 1 -- 172.21.15.114:0/3613621194 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd914003970 con 0x7fd91c0fee80 2024-09-13T18:23:39.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.830+0000 7fd9137fe640 1 -- 172.21.15.114:0/3613621194 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd914004040 con 0x7fd91c0fee80 2024-09-13T18:23:39.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.830+0000 7fd920f11640 1 -- 172.21.15.114:0/3613621194 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd91c1a4fd0 con 0x7fd91c0fee80 2024-09-13T18:23:39.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.830+0000 7fd9137fe640 1 -- 172.21.15.114:0/3613621194 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd914045650 con 0x7fd91c0fee80 2024-09-13T18:23:39.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.830+0000 7fd920f11640 1 -- 172.21.15.114:0/3613621194 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd91c1a5390 con 0x7fd91c0fee80 2024-09-13T18:23:39.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.832+0000 7fd9137fe640 1 -- 172.21.15.114:0/3613621194 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) v1 ==== 136107+0+0 (secure 0 0 0) 0x7fd91400a040 con 0x7fd91c0fee80 2024-09-13T18:23:39.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.832+0000 7fd923c34740 1 Processor -- start 2024-09-13T18:23:39.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.833+0000 7fd923c34740 1 -- start start 2024-09-13T18:23:39.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.833+0000 7fd923c34740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55785b112790 0x55785b112b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.833+0000 7fd923c34740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55785b1130b0 0x55785b11d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.833+0000 7fd923c34740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55785b11d950 0x55785b11fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.833+0000 7fd923c34740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55785b07ccc0 con 0x55785b1130b0 2024-09-13T18:23:39.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.833+0000 7fd923c34740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55785b07a450 con 0x55785b11d950 2024-09-13T18:23:39.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.833+0000 7fd923c34740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55785b075d40 con 0x55785b112790 2024-09-13T18:23:39.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.833+0000 7fd9137fe640 1 --2- 172.21.15.114:0/3613621194 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fd9080a1320 0x7fd9080a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.833+0000 7fd91a59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55785b1130b0 0x55785b11d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.833+0000 7fd91ad9d640 1 --2- 172.21.15.114:0/3613621194 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fd9080a1320 0x7fd9080a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.833+0000 7fd91a59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55785b1130b0 0x55785b11d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51544/0 (socket says 172.21.15.114:51544) 2024-09-13T18:23:39.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.833+0000 7fd91a59c640 1 -- 172.21.15.114:0/939643754 learned_addr learned my addr 172.21.15.114:0/939643754 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:39.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.833+0000 7fd91ad9d640 1 --2- 172.21.15.114:0/939643754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55785b112790 0x55785b112b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.833+0000 7fd919d9b640 1 --2- 172.21.15.114:0/939643754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55785b11d950 0x55785b11fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.833+0000 7fd9137fe640 1 -- 172.21.15.114:0/3613621194 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fd9140eeb00 con 0x7fd91c0fee80 2024-09-13T18:23:39.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.833+0000 7fd91a59c640 1 -- 172.21.15.114:0/939643754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55785b112790 msgr2=0x55785b112b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:39.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.833+0000 7fd91a59c640 1 --2- 172.21.15.114:0/939643754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55785b112790 0x55785b112b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.833+0000 7fd91a59c640 1 -- 172.21.15.114:0/939643754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55785b11d950 msgr2=0x55785b11fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:39.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.833+0000 7fd91a59c640 1 --2- 172.21.15.114:0/939643754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55785b11d950 0x55785b11fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.833+0000 7fd91a59c640 1 -- 172.21.15.114:0/939643754 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55785b03d620 con 0x55785b1130b0 2024-09-13T18:23:39.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.834+0000 7fd91ad9d640 1 --2- 172.21.15.114:0/3613621194 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fd9080a1320 0x7fd9080a37e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd90c005d50 tx=0x7fd90c00e9b0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:39.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.834+0000 7fd91a59c640 1 --2- 172.21.15.114:0/939643754 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55785b1130b0 0x55785b11d410 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fd91402aed0 tx=0x7fd91402a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ba908867e3bb597 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:39.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.834+0000 7fd910ff9640 1 -- 172.21.15.114:0/939643754 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9140bb670 con 0x55785b1130b0 2024-09-13T18:23:39.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.834+0000 7fd910ff9640 1 -- 172.21.15.114:0/939643754 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd9140bf4a0 con 0x55785b1130b0 2024-09-13T18:23:39.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.834+0000 7fd910ff9640 1 -- 172.21.15.114:0/939643754 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9140044b0 con 0x55785b1130b0 2024-09-13T18:23:39.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.834+0000 7fd923c34740 1 -- 172.21.15.114:0/939643754 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55785b1130b0 msgr2=0x55785b11d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:39.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.834+0000 7fd923c34740 1 --2- 172.21.15.114:0/939643754 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55785b1130b0 0x55785b11d410 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fd91402aed0 tx=0x7fd91402a8e0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.835+0000 7fd923c34740 1 -- 172.21.15.114:0/939643754 shutdown_connections 2024-09-13T18:23:39.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.835+0000 7fd923c34740 1 --2- 172.21.15.114:0/939643754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55785b11d950 0x55785b11fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.835+0000 7fd923c34740 1 --2- 172.21.15.114:0/939643754 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55785b1130b0 0x55785b11d410 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.835+0000 7fd923c34740 1 --2- 172.21.15.114:0/939643754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55785b112790 0x55785b112b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.835+0000 7fd923c34740 1 -- 172.21.15.114:0/939643754 >> 172.21.15.114:0/939643754 conn(0x55785b10ed00 msgr2=0x55785b10f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:39.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.835+0000 7fd923c34740 1 -- 172.21.15.114:0/939643754 shutdown_connections 2024-09-13T18:23:39.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.835+0000 7fd923c34740 1 -- 172.21.15.114:0/939643754 wait complete. 2024-09-13T18:23:39.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.835+0000 7fd923c34740 1 -- start start 2024-09-13T18:23:39.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.835+0000 7fd923c34740 1 Processor -- start 2024-09-13T18:23:39.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.835+0000 7fd923c34740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55785b112790 0x55785b11cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.835+0000 7fd923c34740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55785b1130b0 0x55785b11a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.835+0000 7fd923c34740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55785b11d950 0x55785b11a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.835+0000 7fd923c34740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55785b034180 con 0x55785b1130b0 2024-09-13T18:23:39.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.835+0000 7fd923c34740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55785ad4e430 con 0x55785b11d950 2024-09-13T18:23:39.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.835+0000 7fd923c34740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55785aff95e0 con 0x55785b112790 2024-09-13T18:23:39.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.835+0000 7fd923c34740 10 client.0 _ll_register_callbacks cb 0x55785b065aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:23:39.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.835+0000 7fd919d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55785b1130b0 0x55785b11a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.835+0000 7fd919d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55785b1130b0 0x55785b11a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51558/0 (socket says 172.21.15.114:51558) 2024-09-13T18:23:39.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.835+0000 7fd919d9b640 1 -- 172.21.15.114:0/4200668307 learned_addr learned my addr 172.21.15.114:0/4200668307 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:39.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.836+0000 7fd91ad9d640 1 --2- 172.21.15.114:0/4200668307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55785b11d950 0x55785b11a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.836+0000 7fd91a59c640 1 --2- 172.21.15.114:0/4200668307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55785b112790 0x55785b11cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.836+0000 7fd919d9b640 1 -- 172.21.15.114:0/4200668307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55785b112790 msgr2=0x55785b11cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:39.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.836+0000 7fd919d9b640 1 --2- 172.21.15.114:0/4200668307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55785b112790 0x55785b11cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.836+0000 7fd919d9b640 1 -- 172.21.15.114:0/4200668307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55785b11d950 msgr2=0x55785b11a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:39.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.836+0000 7fd919d9b640 1 --2- 172.21.15.114:0/4200668307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55785b11d950 0x55785b11a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.836+0000 7fd919d9b640 1 -- 172.21.15.114:0/4200668307 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd91c1a5390 con 0x55785b1130b0 2024-09-13T18:23:39.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.836+0000 7fd91ad9d640 1 --2- 172.21.15.114:0/4200668307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55785b11d950 0x55785b11a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:23:39.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.836+0000 7fd91a59c640 1 --2- 172.21.15.114:0/4200668307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55785b112790 0x55785b11cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:23:39.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.836+0000 7fd919d9b640 1 --2- 172.21.15.114:0/4200668307 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55785b1130b0 0x55785b11a2e0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fd904003870 tx=0x7fd904003c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a83e41387d63493 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:39.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.836+0000 7fd901ffb640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:23:39.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.836+0000 7fd901ffb640 1 -- 172.21.15.114:0/4200668307 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90401a470 con 0x55785b1130b0 2024-09-13T18:23:39.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.836+0000 7fd901ffb640 1 -- 172.21.15.114:0/4200668307 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fd90401a5d0 con 0x55785b1130b0 2024-09-13T18:23:39.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.836+0000 7fd923c34740 1 -- 172.21.15.114:0/4200668307 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55785b09fb80 con 0x55785b1130b0 2024-09-13T18:23:39.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.836+0000 7fd901ffb640 1 -- 172.21.15.114:0/4200668307 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd90401a790 con 0x55785b1130b0 2024-09-13T18:23:39.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.836+0000 7fd923c34740 10 client.16152 fetch_fsmap learned FSMap version 30 2024-09-13T18:23:39.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.836+0000 7fd923c34740 1 -- 172.21.15.114:0/4200668307 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55785b0743c0 con 0x55785b1130b0 2024-09-13T18:23:39.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.836+0000 7fd901ffb640 1 -- 172.21.15.114:0/4200668307 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55785b09fb80 con 0x55785b1130b0 2024-09-13T18:23:39.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.836+0000 7fd901ffb640 1 -- 172.21.15.114:0/4200668307 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd9040118e0 con 0x55785b1130b0 2024-09-13T18:23:39.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.836+0000 7fd923c34740 10 client.16152 fetch_fsmap finished waiting for FSMap version 30 2024-09-13T18:23:39.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.836+0000 7fd923c34740 10 client.16152 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:23:39.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.836+0000 7fd923c34740 20 client.16152 populate_metadata read hostname 'smithi114' 2024-09-13T18:23:39.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.836+0000 7fd923c34740 1 --2- 172.21.15.114:0/4200668307 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55785b12d0f0 0x55785b12f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.836+0000 7fd923c34740 4 client.16152 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:23:39.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.837+0000 7fd923c34740 1 -- 172.21.15.114:0/4200668307 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55785b0681f0 con 0x55785b12d0f0 2024-09-13T18:23:39.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.837+0000 7fd91a59c640 1 --2- 172.21.15.114:0/4200668307 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55785b12d0f0 0x55785b12f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.837+0000 7fd91a59c640 1 --2- 172.21.15.114:0/4200668307 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55785b12d0f0 0x55785b12f510 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4491938c50cdfb8 server_cookie=13149b108f4e997e in_seq=0 out_seq=0 2024-09-13T18:23:39.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.837+0000 7fd901ffb640 10 client.16152 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:23:39.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.838+0000 7fd901ffb640 1 -- 172.21.15.114:0/4200668307 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55785b0681f0 con 0x55785b12d0f0 2024-09-13T18:23:39.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.838+0000 7fd901ffb640 10 client.16152 handle_command_reply: tid=0 2024-09-13T18:23:39.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.838+0000 7fd923c34740 1 client.16152 shutdown 2024-09-13T18:23:39.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.838+0000 7fd923c34740 1 -- 172.21.15.114:0/4200668307 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55785b1130b0 msgr2=0x55785b11a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:39.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.838+0000 7fd923c34740 1 --2- 172.21.15.114:0/4200668307 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55785b1130b0 0x55785b11a2e0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fd904003870 tx=0x7fd904003c70 comp rx=0 tx=0).stop 2024-09-13T18:23:39.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.838+0000 7fd923c34740 1 -- 172.21.15.114:0/4200668307 shutdown_connections 2024-09-13T18:23:39.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.838+0000 7fd923c34740 1 --2- 172.21.15.114:0/4200668307 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55785b12d0f0 0x55785b12f510 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.838+0000 7fd923c34740 1 --2- 172.21.15.114:0/4200668307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55785b11d950 0x55785b11a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.838+0000 7fd923c34740 1 --2- 172.21.15.114:0/4200668307 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55785b1130b0 0x55785b11a2e0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.838+0000 7fd901ffb640 0 client.16152 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:23:39.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.838+0000 7fd923c34740 1 --2- 172.21.15.114:0/4200668307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55785b112790 0x55785b11cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.838+0000 7fd923c34740 1 -- 172.21.15.114:0/4200668307 >> 172.21.15.114:0/4200668307 conn(0x55785b10ed00 msgr2=0x55785b10deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:39.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.838+0000 7fd923c34740 1 -- 172.21.15.114:0/4200668307 shutdown_connections 2024-09-13T18:23:39.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.838+0000 7fd923c34740 1 -- 172.21.15.114:0/4200668307 wait complete. 2024-09-13T18:23:39.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.838+0000 7fd923c34740 20 client.16152 trim_cache size 0 max 16384 2024-09-13T18:23:39.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.849+0000 7fd923c34740 1 Processor -- start 2024-09-13T18:23:39.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.849+0000 7fd923c34740 1 -- start start 2024-09-13T18:23:39.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.849+0000 7fd923c34740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55785b10ed00 0x55785b10e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.849+0000 7fd923c34740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55785b112790 0x55785b1243d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.849+0000 7fd923c34740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55785b1130b0 0x55785b124910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.849+0000 7fd923c34740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55785b113930 con 0x55785b10ed00 2024-09-13T18:23:39.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.849+0000 7fd923c34740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55785b109300 con 0x55785b112790 2024-09-13T18:23:39.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.849+0000 7fd923c34740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55785b112580 con 0x55785b1130b0 2024-09-13T18:23:39.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.849+0000 7fd91a59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55785b10ed00 0x55785b10e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.849+0000 7fd91a59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55785b10ed00 0x55785b10e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51566/0 (socket says 172.21.15.114:51566) 2024-09-13T18:23:39.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.849+0000 7fd91a59c640 1 -- 172.21.15.114:0/1074508859 learned_addr learned my addr 172.21.15.114:0/1074508859 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:39.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.849+0000 7fd91a59c640 1 -- 172.21.15.114:0/1074508859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55785b1130b0 msgr2=0x55785b124910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:39.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.849+0000 7fd91a59c640 1 --2- 172.21.15.114:0/1074508859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55785b1130b0 0x55785b124910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.849+0000 7fd91a59c640 1 -- 172.21.15.114:0/1074508859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55785b112790 msgr2=0x55785b1243d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:39.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.849+0000 7fd91a59c640 1 --2- 172.21.15.114:0/1074508859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55785b112790 0x55785b1243d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.849+0000 7fd91a59c640 1 -- 172.21.15.114:0/1074508859 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55785b016770 con 0x55785b10ed00 2024-09-13T18:23:39.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.850+0000 7fd91a59c640 1 --2- 172.21.15.114:0/1074508859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55785b10ed00 0x55785b10e6a0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fd9140bb600 tx=0x7fd9140e2040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aafd31cefdb00f6b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:39.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.850+0000 7fd9117fa640 1 -- 172.21.15.114:0/1074508859 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9140e0070 con 0x55785b10ed00 2024-09-13T18:23:39.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.850+0000 7fd9117fa640 1 -- 172.21.15.114:0/1074508859 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd9140d3070 con 0x55785b10ed00 2024-09-13T18:23:39.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.850+0000 7fd9117fa640 1 -- 172.21.15.114:0/1074508859 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9140db0d0 con 0x55785b10ed00 2024-09-13T18:23:39.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.850+0000 7fd923c34740 1 -- 172.21.15.114:0/1074508859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55785b10ed00 msgr2=0x55785b10e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:39.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.850+0000 7fd923c34740 1 --2- 172.21.15.114:0/1074508859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55785b10ed00 0x55785b10e6a0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fd9140bb600 tx=0x7fd9140e2040 comp rx=0 tx=0).stop 2024-09-13T18:23:39.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.850+0000 7fd923c34740 1 -- 172.21.15.114:0/1074508859 shutdown_connections 2024-09-13T18:23:39.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.850+0000 7fd923c34740 1 --2- 172.21.15.114:0/1074508859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55785b1130b0 0x55785b124910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.850+0000 7fd923c34740 1 --2- 172.21.15.114:0/1074508859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55785b112790 0x55785b1243d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.850+0000 7fd923c34740 1 --2- 172.21.15.114:0/1074508859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55785b10ed00 0x55785b10e6a0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.850+0000 7fd923c34740 1 -- 172.21.15.114:0/1074508859 >> 172.21.15.114:0/1074508859 conn(0x55785b11d080 msgr2=0x55785b128e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:39.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.850+0000 7fd923c34740 1 -- 172.21.15.114:0/1074508859 shutdown_connections 2024-09-13T18:23:39.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.850+0000 7fd923c34740 1 -- 172.21.15.114:0/1074508859 wait complete. 2024-09-13T18:23:39.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.850+0000 7fd923c34740 1 -- start start 2024-09-13T18:23:39.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.851+0000 7fd923c34740 1 Processor -- start 2024-09-13T18:23:39.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.851+0000 7fd923c34740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55785b10ed00 0x55785b1245d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.851+0000 7fd923c34740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55785b112790 0x55785b116af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.851+0000 7fd923c34740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55785b1130b0 0x55785b117030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.851+0000 7fd923c34740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55785b1175c0 con 0x55785b1130b0 2024-09-13T18:23:39.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.851+0000 7fd923c34740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55785b117730 con 0x55785b112790 2024-09-13T18:23:39.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.851+0000 7fd923c34740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55785b1178f0 con 0x55785b10ed00 2024-09-13T18:23:39.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.851+0000 7fd923c34740 10 client.0 _ll_register_callbacks cb 0x7fd9140bb440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:23:39.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.851+0000 7fd91ad9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55785b1130b0 0x55785b117030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.851+0000 7fd91ad9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55785b1130b0 0x55785b117030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51570/0 (socket says 172.21.15.114:51570) 2024-09-13T18:23:39.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.851+0000 7fd91ad9d640 1 -- 172.21.15.114:0/3829485921 learned_addr learned my addr 172.21.15.114:0/3829485921 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:39.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.851+0000 7fd919d9b640 1 --2- 172.21.15.114:0/3829485921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55785b112790 0x55785b116af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.851+0000 7fd91a59c640 1 --2- 172.21.15.114:0/3829485921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55785b10ed00 0x55785b1245d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.851+0000 7fd91ad9d640 1 -- 172.21.15.114:0/3829485921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55785b10ed00 msgr2=0x55785b1245d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:39.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.851+0000 7fd91ad9d640 1 --2- 172.21.15.114:0/3829485921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55785b10ed00 0x55785b1245d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.851+0000 7fd91ad9d640 1 -- 172.21.15.114:0/3829485921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55785b112790 msgr2=0x55785b116af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:39.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.851+0000 7fd91ad9d640 1 --2- 172.21.15.114:0/3829485921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55785b112790 0x55785b116af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.851+0000 7fd91ad9d640 1 -- 172.21.15.114:0/3829485921 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55785b03d620 con 0x55785b1130b0 2024-09-13T18:23:39.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.851+0000 7fd919d9b640 1 --2- 172.21.15.114:0/3829485921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55785b112790 0x55785b116af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:23:39.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.851+0000 7fd91ad9d640 1 --2- 172.21.15.114:0/3829485921 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55785b1130b0 0x55785b117030 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fd90c010be0 tx=0x7fd90c005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed1a84e19bad9307 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:39.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.851+0000 7fd901ffb640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:23:39.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.851+0000 7fd901ffb640 1 -- 172.21.15.114:0/3829485921 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c011d90 con 0x55785b1130b0 2024-09-13T18:23:39.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.852+0000 7fd901ffb640 1 -- 172.21.15.114:0/3829485921 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fd90c029460 con 0x55785b1130b0 2024-09-13T18:23:39.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.852+0000 7fd923c34740 1 -- 172.21.15.114:0/3829485921 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd9140bf4a0 con 0x55785b1130b0 2024-09-13T18:23:39.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.852+0000 7fd901ffb640 1 -- 172.21.15.114:0/3829485921 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd90c029620 con 0x55785b1130b0 2024-09-13T18:23:39.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.852+0000 7fd923c34740 10 client.16158 fetch_fsmap learned FSMap version 30 2024-09-13T18:23:39.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.852+0000 7fd923c34740 1 -- 172.21.15.114:0/3829485921 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55785b03be80 con 0x55785b1130b0 2024-09-13T18:23:39.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.852+0000 7fd901ffb640 1 -- 172.21.15.114:0/3829485921 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd9140bf4a0 con 0x55785b1130b0 2024-09-13T18:23:39.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.852+0000 7fd901ffb640 1 -- 172.21.15.114:0/3829485921 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd90c020680 con 0x55785b1130b0 2024-09-13T18:23:39.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.852+0000 7fd923c34740 10 client.16158 fetch_fsmap finished waiting for FSMap version 30 2024-09-13T18:23:39.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.852+0000 7fd923c34740 10 client.16158 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:23:39.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.852+0000 7fd923c34740 20 client.16158 populate_metadata read hostname 'smithi114' 2024-09-13T18:23:39.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.852+0000 7fd923c34740 1 --2- 172.21.15.114:0/3829485921 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55785b11d950 0x55785b186c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:39.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.852+0000 7fd923c34740 4 client.16158 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:23:39.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.852+0000 7fd923c34740 1 -- 172.21.15.114:0/3829485921 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55785ad694e0 con 0x55785b11d950 2024-09-13T18:23:39.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.852+0000 7fd91a59c640 1 --2- 172.21.15.114:0/3829485921 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55785b11d950 0x55785b186c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:39.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.853+0000 7fd91a59c640 1 --2- 172.21.15.114:0/3829485921 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55785b11d950 0x55785b186c30 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d21421cd0724281 server_cookie=4c38b2ed0bf4544a in_seq=0 out_seq=0 2024-09-13T18:23:39.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.853+0000 7fd901ffb640 10 client.16158 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:23:39.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.853+0000 7fd901ffb640 1 -- 172.21.15.114:0/3829485921 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55785ad694e0 con 0x55785b11d950 2024-09-13T18:23:39.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.853+0000 7fd901ffb640 10 client.16158 handle_command_reply: tid=0 2024-09-13T18:23:39.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.853+0000 7fd923c34740 1 client.16158 shutdown 2024-09-13T18:23:39.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.853+0000 7fd923c34740 1 -- 172.21.15.114:0/3829485921 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55785b1130b0 msgr2=0x55785b117030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:39.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.853+0000 7fd923c34740 1 --2- 172.21.15.114:0/3829485921 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55785b1130b0 0x55785b117030 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fd90c010be0 tx=0x7fd90c005ce0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.853+0000 7fd923c34740 1 -- 172.21.15.114:0/3829485921 shutdown_connections 2024-09-13T18:23:39.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.853+0000 7fd923c34740 1 --2- 172.21.15.114:0/3829485921 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55785b11d950 0x55785b186c30 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.853+0000 7fd923c34740 1 --2- 172.21.15.114:0/3829485921 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55785b1130b0 0x55785b117030 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.853+0000 7fd923c34740 1 --2- 172.21.15.114:0/3829485921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55785b112790 0x55785b116af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.853+0000 7fd901ffb640 0 client.16158 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:23:39.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.853+0000 7fd923c34740 1 --2- 172.21.15.114:0/3829485921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55785b10ed00 0x55785b1245d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.853+0000 7fd923c34740 1 -- 172.21.15.114:0/3829485921 >> 172.21.15.114:0/3829485921 conn(0x55785b11d080 msgr2=0x55785b11b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:39.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.853+0000 7fd923c34740 1 -- 172.21.15.114:0/3829485921 shutdown_connections 2024-09-13T18:23:39.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.853+0000 7fd923c34740 1 -- 172.21.15.114:0/3829485921 wait complete. 2024-09-13T18:23:39.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.853+0000 7fd923c34740 20 client.16158 trim_cache size 0 max 16384 2024-09-13T18:23:39.863 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:23:39.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.854+0000 7fd920f11640 1 -- 172.21.15.114:0/3613621194 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fd9080a1320 msgr2=0x7fd9080a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:39.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.854+0000 7fd920f11640 1 --2- 172.21.15.114:0/3613621194 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fd9080a1320 0x7fd9080a37e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd90c005d50 tx=0x7fd90c00e9b0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.854+0000 7fd920f11640 1 -- 172.21.15.114:0/3613621194 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd91c0fee80 msgr2=0x7fd91c1983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:39.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.854+0000 7fd920f11640 1 --2- 172.21.15.114:0/3613621194 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd91c0fee80 0x7fd91c1983f0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fd914004af0 tx=0x7fd914030ec0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.854+0000 7fd920f11640 1 -- 172.21.15.114:0/3613621194 shutdown_connections 2024-09-13T18:23:39.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.854+0000 7fd920f11640 1 --2- 172.21.15.114:0/3613621194 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fd9080a1320 0x7fd9080a37e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.854+0000 7fd920f11640 1 --2- 172.21.15.114:0/3613621194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd91c19cd20 0x7fd91c19d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.854+0000 7fd920f11640 1 --2- 172.21.15.114:0/3613621194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd91c103fd0 0x7fd91c198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.854+0000 7fd920f11640 1 --2- 172.21.15.114:0/3613621194 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd91c0fee80 0x7fd91c1983f0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:39.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.854+0000 7fd920f11640 1 -- 172.21.15.114:0/3613621194 >> 172.21.15.114:0/3613621194 conn(0x7fd91c0fa930 msgr2=0x7fd91c0fb9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:39.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.854+0000 7fd920f11640 1 -- 172.21.15.114:0/3613621194 shutdown_connections 2024-09-13T18:23:39.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:39.854+0000 7fd920f11640 1 -- 172.21.15.114:0/3613621194 wait complete. 2024-09-13T18:23:39.868 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:23:39 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/1707115762' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:23:40.355 INFO:tasks.workunit.client.0.smithi114.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-13T18:23:40.424 INFO:tasks.workunit.client.0.smithi114.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-13T18:23:40.463 INFO:tasks.workunit.client.0.smithi114.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-13T18:23:40.465 INFO:tasks.workunit.client.0.smithi114.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-13T18:23:40.465 INFO:tasks.workunit.client.0.smithi114.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-13T18:23:40.506 INFO:tasks.workunit.client.0.smithi114.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-13T18:23:40.508 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-13T18:23:40.509 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-13T18:23:40.525 INFO:tasks.workunit:Running workunits matching suites/iogen.sh on client.0... 2024-09-13T18:23:40.527 INFO:tasks.workunit:Running workunit suites/iogen.sh... 2024-09-13T18:23:40.527 DEBUG:teuthology.orchestra.run.smithi114:workunit test suites/iogen.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iogen.sh 2024-09-13T18:23:40.595 INFO:tasks.workunit.client.0.smithi114.stdout:getting iogen 2024-09-13T18:23:40.622 INFO:tasks.workunit.client.0.smithi114.stderr:--2024-09-13 18:23:40-- http://download.ceph.com/qa/iogen_3.1p0.tar 2024-09-13T18:23:40.623 INFO:tasks.workunit.client.0.smithi114.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-09-13T18:23:40.644 INFO:tasks.workunit.client.0.smithi114.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-09-13T18:23:40.665 INFO:tasks.workunit.client.0.smithi114.stderr:HTTP request sent, awaiting response... 200 OK 2024-09-13T18:23:40.665 INFO:tasks.workunit.client.0.smithi114.stderr:Length: 8071 (7.9K) [application/octet-stream] 2024-09-13T18:23:40.666 INFO:tasks.workunit.client.0.smithi114.stderr:Saving to: ‘iogen_3.1p0.tar’ 2024-09-13T18:23:40.670 INFO:tasks.workunit.client.0.smithi114.stderr: 2024-09-13T18:23:40.670 INFO:tasks.workunit.client.0.smithi114.stderr: 0K ....... 100% 1021M=0s 2024-09-13T18:23:40.670 INFO:tasks.workunit.client.0.smithi114.stderr: 2024-09-13T18:23:40.670 INFO:tasks.workunit.client.0.smithi114.stderr:2024-09-13 18:23:40 (1021 MB/s) - ‘iogen_3.1p0.tar’ saved [8071/8071] 2024-09-13T18:23:40.671 INFO:tasks.workunit.client.0.smithi114.stderr: 2024-09-13T18:23:40.693 INFO:tasks.workunit.client.0.smithi114.stdout:iogen_3.1p0 2024-09-13T18:23:40.694 INFO:tasks.workunit.client.0.smithi114.stdout:iogen_3.1p0/src 2024-09-13T18:23:40.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:23:40 smithi204 ceph-mon[34777]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-13T18:23:40.695 INFO:tasks.workunit.client.0.smithi114.stdout:iogen_3.1p0/src/iogen.8 2024-09-13T18:23:40.697 INFO:tasks.workunit.client.0.smithi114.stdout:iogen_3.1p0/src/iogen.c 2024-09-13T18:23:40.700 INFO:tasks.workunit.client.0.smithi114.stdout:iogen_3.1p0/README 2024-09-13T18:23:40.701 INFO:tasks.workunit.client.0.smithi114.stdout:iogen_3.1p0/Makefile.bsd 2024-09-13T18:23:40.703 INFO:tasks.workunit.client.0.smithi114.stdout:iogen_3.1p0/Makefile 2024-09-13T18:23:40.707 INFO:tasks.workunit.client.0.smithi114.stdout:making iogen 2024-09-13T18:23:40.717 INFO:tasks.workunit.client.0.smithi114.stdout:gcc -D_FILE_OFFSET_BITS=64 -DVERSION=\"3.1p0\" -Dstrlcpy=strncpy -Darc4random=rand -O2 src/iogen.c -o iogen 2024-09-13T18:23:40.719 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:23:40 smithi114 ceph-mon[30897]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-13T18:23:40.867 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-13T18:23:40.867 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-09-13T18:23:40.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:23:40 smithi134 ceph-mon[34283]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-13T18:23:40.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.971+0000 7fbf28753640 1 -- 172.21.15.114:0/2953594963 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf200fff50 msgr2=0x7fbf20103a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:40.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.971+0000 7fbf28753640 1 --2- 172.21.15.114:0/2953594963 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf200fff50 0x7fbf20103a70 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fbf10009930 tx=0x7fbf1002a420 comp rx=0 tx=0).stop 2024-09-13T18:23:40.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.971+0000 7fbf28753640 1 -- 172.21.15.114:0/2953594963 shutdown_connections 2024-09-13T18:23:40.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.971+0000 7fbf28753640 1 --2- 172.21.15.114:0/2953594963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf201041d0 0x7fbf2010bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:40.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.971+0000 7fbf28753640 1 --2- 172.21.15.114:0/2953594963 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf200fff50 0x7fbf20103a70 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:40.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.971+0000 7fbf28753640 1 --2- 172.21.15.114:0/2953594963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf200ff5a0 0x7fbf200ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:40.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.971+0000 7fbf28753640 1 -- 172.21.15.114:0/2953594963 >> 172.21.15.114:0/2953594963 conn(0x7fbf200753f0 msgr2=0x7fbf200fc740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:40.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.971+0000 7fbf28753640 1 -- 172.21.15.114:0/2953594963 shutdown_connections 2024-09-13T18:23:40.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.971+0000 7fbf28753640 1 -- 172.21.15.114:0/2953594963 wait complete. 2024-09-13T18:23:40.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.972+0000 7fbf28753640 1 Processor -- start 2024-09-13T18:23:40.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.972+0000 7fbf28753640 1 -- start start 2024-09-13T18:23:40.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.972+0000 7fbf28753640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf200ff5a0 0x7fbf200fa800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:40.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.972+0000 7fbf28753640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf201041d0 0x7fbf200fad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:40.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.972+0000 7fbf28753640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf200fb280 0x7fbf201a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:40.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.972+0000 7fbf28753640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf201a4bd0 con 0x7fbf200ff5a0 2024-09-13T18:23:40.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.972+0000 7fbf28753640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf201a4d40 con 0x7fbf201041d0 2024-09-13T18:23:40.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.972+0000 7fbf28753640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf201a4f00 con 0x7fbf200fb280 2024-09-13T18:23:40.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.972+0000 7fbf264ef640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf200ff5a0 0x7fbf200fa800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:40.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.972+0000 7fbf264ef640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf200ff5a0 0x7fbf200fa800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51578/0 (socket says 172.21.15.114:51578) 2024-09-13T18:23:40.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.972+0000 7fbf264ef640 1 -- 172.21.15.114:0/825549145 learned_addr learned my addr 172.21.15.114:0/825549145 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:40.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.972+0000 7fbf25cee640 1 --2- 172.21.15.114:0/825549145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf201041d0 0x7fbf200fad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:40.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.972+0000 7fbf264ef640 1 -- 172.21.15.114:0/825549145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf200fb280 msgr2=0x7fbf201a4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:40.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.972+0000 7fbf264ef640 1 --2- 172.21.15.114:0/825549145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf200fb280 0x7fbf201a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:40.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.972+0000 7fbf264ef640 1 -- 172.21.15.114:0/825549145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf201041d0 msgr2=0x7fbf200fad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:40.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.972+0000 7fbf264ef640 1 --2- 172.21.15.114:0/825549145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf201041d0 0x7fbf200fad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:40.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.972+0000 7fbf264ef640 1 -- 172.21.15.114:0/825549145 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf10009560 con 0x7fbf200ff5a0 2024-09-13T18:23:40.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.972+0000 7fbf264ef640 1 --2- 172.21.15.114:0/825549145 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf200ff5a0 0x7fbf200fa800 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c00c890 tx=0x7fbf1c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:40.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.973+0000 7fbf177fe640 1 -- 172.21.15.114:0/825549145 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c004220 con 0x7fbf200ff5a0 2024-09-13T18:23:40.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.973+0000 7fbf28753640 1 -- 172.21.15.114:0/825549145 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf201a51e0 con 0x7fbf200ff5a0 2024-09-13T18:23:40.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.973+0000 7fbf177fe640 1 -- 172.21.15.114:0/825549145 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbf1c007e40 con 0x7fbf200ff5a0 2024-09-13T18:23:40.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.973+0000 7fbf177fe640 1 -- 172.21.15.114:0/825549145 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c004a00 con 0x7fbf200ff5a0 2024-09-13T18:23:40.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.973+0000 7fbf28753640 1 -- 172.21.15.114:0/825549145 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf201a5710 con 0x7fbf200ff5a0 2024-09-13T18:23:40.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.975+0000 7fbf177fe640 1 -- 172.21.15.114:0/825549145 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) v1 ==== 136107+0+0 (secure 0 0 0) 0x7fbf1c0079b0 con 0x7fbf200ff5a0 2024-09-13T18:23:40.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.975+0000 7fbf2b472740 1 Processor -- start 2024-09-13T18:23:40.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.975+0000 7fbf2b472740 1 -- start start 2024-09-13T18:23:40.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.976+0000 7fbf2b472740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55717a95d7c0 0x55717a95dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:40.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.976+0000 7fbf2b472740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55717a95e0e0 0x55717a968440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:40.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.976+0000 7fbf2b472740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55717a968980 0x55717a96ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:40.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.976+0000 7fbf2b472740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55717a8c8c60 con 0x55717a95d7c0 2024-09-13T18:23:40.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.976+0000 7fbf2b472740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55717a8c63f0 con 0x55717a968980 2024-09-13T18:23:40.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.976+0000 7fbf2b472740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55717a8c1ce0 con 0x55717a95e0e0 2024-09-13T18:23:40.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.976+0000 7fbf177fe640 1 --2- 172.21.15.114:0/825549145 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbf0c0a1350 0x7fbf0c0a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:40.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.976+0000 7fbf26cf0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55717a95d7c0 0x55717a95dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:40.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.976+0000 7fbf26cf0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55717a95d7c0 0x55717a95dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51588/0 (socket says 172.21.15.114:51588) 2024-09-13T18:23:40.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.976+0000 7fbf26cf0640 1 -- 172.21.15.114:0/4158814203 learned_addr learned my addr 172.21.15.114:0/4158814203 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:40.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.976+0000 7fbf264ef640 1 --2- 172.21.15.114:0/4158814203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55717a95e0e0 0x55717a968440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:40.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.976+0000 7fbf25cee640 1 --2- 172.21.15.114:0/4158814203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55717a968980 0x55717a96ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:40.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.976+0000 7fbf26cf0640 1 --2- 172.21.15.114:0/825549145 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbf0c0a1350 0x7fbf0c0a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:40.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.976+0000 7fbf177fe640 1 -- 172.21.15.114:0/825549145 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fbf1c0cd080 con 0x7fbf200ff5a0 2024-09-13T18:23:40.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.976+0000 7fbf26cf0640 1 -- 172.21.15.114:0/4158814203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55717a95e0e0 msgr2=0x55717a968440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:40.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.976+0000 7fbf26cf0640 1 --2- 172.21.15.114:0/4158814203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55717a95e0e0 0x55717a968440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:40.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.976+0000 7fbf26cf0640 1 -- 172.21.15.114:0/4158814203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55717a968980 msgr2=0x55717a96ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:40.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.976+0000 7fbf26cf0640 1 --2- 172.21.15.114:0/4158814203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55717a968980 0x55717a96ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:40.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.976+0000 7fbf26cf0640 1 -- 172.21.15.114:0/4158814203 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55717a87d970 con 0x55717a95d7c0 2024-09-13T18:23:40.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.976+0000 7fbf25cee640 1 --2- 172.21.15.114:0/4158814203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55717a968980 0x55717a96ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:23:40.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.977+0000 7fbf26cf0640 1 --2- 172.21.15.114:0/4158814203 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55717a95d7c0 0x55717a95dba0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fbf180148d0 tx=0x7fbf18014d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d3b758f605ad6ce server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:40.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.977+0000 7fbf14ff9640 1 -- 172.21.15.114:0/4158814203 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf18014050 con 0x55717a95d7c0 2024-09-13T18:23:40.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.977+0000 7fbf14ff9640 1 -- 172.21.15.114:0/4158814203 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbf180141b0 con 0x55717a95d7c0 2024-09-13T18:23:40.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.977+0000 7fbf14ff9640 1 -- 172.21.15.114:0/4158814203 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1801d680 con 0x55717a95d7c0 2024-09-13T18:23:40.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.977+0000 7fbf2b472740 1 -- 172.21.15.114:0/4158814203 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55717a95d7c0 msgr2=0x55717a95dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:40.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.977+0000 7fbf26cf0640 1 --2- 172.21.15.114:0/825549145 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbf0c0a1350 0x7fbf0c0a3810 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbf1801e000 tx=0x7fbf1801b000 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:40.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.977+0000 7fbf2b472740 1 --2- 172.21.15.114:0/4158814203 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55717a95d7c0 0x55717a95dba0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fbf180148d0 tx=0x7fbf18014d90 comp rx=0 tx=0).stop 2024-09-13T18:23:40.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.978+0000 7fbf2b472740 1 -- 172.21.15.114:0/4158814203 shutdown_connections 2024-09-13T18:23:40.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.978+0000 7fbf2b472740 1 --2- 172.21.15.114:0/4158814203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55717a968980 0x55717a96ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:40.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.978+0000 7fbf2b472740 1 --2- 172.21.15.114:0/4158814203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55717a95e0e0 0x55717a968440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:40.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.978+0000 7fbf2b472740 1 --2- 172.21.15.114:0/4158814203 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55717a95d7c0 0x55717a95dba0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:40.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.978+0000 7fbf2b472740 1 -- 172.21.15.114:0/4158814203 >> 172.21.15.114:0/4158814203 conn(0x55717a959d50 msgr2=0x55717a95a130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:40.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.978+0000 7fbf2b472740 1 -- 172.21.15.114:0/4158814203 shutdown_connections 2024-09-13T18:23:40.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.978+0000 7fbf2b472740 1 -- 172.21.15.114:0/4158814203 wait complete. 2024-09-13T18:23:40.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.978+0000 7fbf2b472740 1 -- start start 2024-09-13T18:23:40.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.978+0000 7fbf2b472740 1 Processor -- start 2024-09-13T18:23:40.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.978+0000 7fbf2b472740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55717a95d7c0 0x55717a968350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:40.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.978+0000 7fbf2b472740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55717a95e0e0 0x55717a965640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:40.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.978+0000 7fbf2b472740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55717a968980 0x55717a965c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:40.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.978+0000 7fbf2b472740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55717a87ff80 con 0x55717a95d7c0 2024-09-13T18:23:40.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.978+0000 7fbf2b472740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55717a87fc20 con 0x55717a95e0e0 2024-09-13T18:23:40.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.978+0000 7fbf2b472740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55717a84cab0 con 0x55717a968980 2024-09-13T18:23:40.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.978+0000 7fbf2b472740 10 client.0 _ll_register_callbacks cb 0x55717a8bb350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:23:40.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.978+0000 7fbf264ef640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55717a95d7c0 0x55717a968350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:40.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.978+0000 7fbf264ef640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55717a95d7c0 0x55717a968350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51596/0 (socket says 172.21.15.114:51596) 2024-09-13T18:23:40.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.978+0000 7fbf264ef640 1 -- 172.21.15.114:0/539108810 learned_addr learned my addr 172.21.15.114:0/539108810 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:40.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.979+0000 7fbf25cee640 1 --2- 172.21.15.114:0/539108810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55717a95e0e0 0x55717a965640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:40.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.979+0000 7fbf26cf0640 1 --2- 172.21.15.114:0/539108810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55717a968980 0x55717a965c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:40.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.979+0000 7fbf264ef640 1 -- 172.21.15.114:0/539108810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55717a968980 msgr2=0x55717a965c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:40.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.979+0000 7fbf264ef640 1 --2- 172.21.15.114:0/539108810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55717a968980 0x55717a965c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:40.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.979+0000 7fbf264ef640 1 -- 172.21.15.114:0/539108810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55717a95e0e0 msgr2=0x55717a965640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:40.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.979+0000 7fbf264ef640 1 --2- 172.21.15.114:0/539108810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55717a95e0e0 0x55717a965640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:40.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.979+0000 7fbf264ef640 1 -- 172.21.15.114:0/539108810 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf180144b0 con 0x55717a95d7c0 2024-09-13T18:23:40.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.979+0000 7fbf26cf0640 1 --2- 172.21.15.114:0/539108810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55717a968980 0x55717a965c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:23:40.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.979+0000 7fbf264ef640 1 --2- 172.21.15.114:0/539108810 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55717a95d7c0 0x55717a968350 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c096090 tx=0x7fbf1c068e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b36053ada4acbdd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:40.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.979+0000 7fbf09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:23:40.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.979+0000 7fbf09ffb640 1 -- 172.21.15.114:0/539108810 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c099280 con 0x55717a95d7c0 2024-09-13T18:23:40.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.979+0000 7fbf09ffb640 1 -- 172.21.15.114:0/539108810 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fbf1c099b90 con 0x55717a95d7c0 2024-09-13T18:23:40.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.979+0000 7fbf2b472740 1 -- 172.21.15.114:0/539108810 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55717a5b5590 con 0x55717a95d7c0 2024-09-13T18:23:40.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.979+0000 7fbf09ffb640 1 -- 172.21.15.114:0/539108810 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf1c09d9c0 con 0x55717a95d7c0 2024-09-13T18:23:40.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.979+0000 7fbf2b472740 10 client.16170 fetch_fsmap learned FSMap version 30 2024-09-13T18:23:40.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.979+0000 7fbf2b472740 1 -- 172.21.15.114:0/539108810 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55717a8c0360 con 0x55717a95d7c0 2024-09-13T18:23:40.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.979+0000 7fbf09ffb640 1 -- 172.21.15.114:0/539108810 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbf1c0b1070 con 0x55717a95d7c0 2024-09-13T18:23:40.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.980+0000 7fbf09ffb640 1 -- 172.21.15.114:0/539108810 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbf1c016030 con 0x55717a95d7c0 2024-09-13T18:23:40.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.980+0000 7fbf2b472740 10 client.16170 fetch_fsmap finished waiting for FSMap version 30 2024-09-13T18:23:40.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.980+0000 7fbf2b472740 10 client.16170 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:23:40.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.980+0000 7fbf2b472740 20 client.16170 populate_metadata read hostname 'smithi114' 2024-09-13T18:23:40.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.980+0000 7fbf2b472740 1 --2- 172.21.15.114:0/539108810 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55717a9785c0 0x55717a97a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:40.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.980+0000 7fbf2b472740 4 client.16170 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:23:40.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.980+0000 7fbf2b472740 1 -- 172.21.15.114:0/539108810 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55717a5c6860 con 0x55717a9785c0 2024-09-13T18:23:40.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.980+0000 7fbf25cee640 1 --2- 172.21.15.114:0/539108810 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55717a9785c0 0x55717a97a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:40.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.980+0000 7fbf09ffb640 10 client.16170 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:23:40.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.980+0000 7fbf25cee640 1 --2- 172.21.15.114:0/539108810 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55717a9785c0 0x55717a97a9b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=580b6a6ccac3eb6b server_cookie=f6b926c0905d845b in_seq=0 out_seq=0 2024-09-13T18:23:40.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.981+0000 7fbf09ffb640 1 -- 172.21.15.114:0/539108810 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55717a5c6860 con 0x55717a9785c0 2024-09-13T18:23:40.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.981+0000 7fbf09ffb640 10 client.16170 handle_command_reply: tid=0 2024-09-13T18:23:40.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.981+0000 7fbf2b472740 1 client.16170 shutdown 2024-09-13T18:23:40.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.981+0000 7fbf2b472740 1 -- 172.21.15.114:0/539108810 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55717a95d7c0 msgr2=0x55717a968350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:40.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.981+0000 7fbf2b472740 1 --2- 172.21.15.114:0/539108810 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55717a95d7c0 0x55717a968350 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c096090 tx=0x7fbf1c068e50 comp rx=0 tx=0).stop 2024-09-13T18:23:40.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.981+0000 7fbf2b472740 1 -- 172.21.15.114:0/539108810 shutdown_connections 2024-09-13T18:23:40.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.981+0000 7fbf2b472740 1 --2- 172.21.15.114:0/539108810 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55717a9785c0 0x55717a97a9b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:40.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.981+0000 7fbf2b472740 1 --2- 172.21.15.114:0/539108810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55717a968980 0x55717a965c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:40.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.981+0000 7fbf09ffb640 0 client.16170 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:23:40.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.981+0000 7fbf2b472740 1 --2- 172.21.15.114:0/539108810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55717a95e0e0 0x55717a965640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:40.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.981+0000 7fbf2b472740 1 --2- 172.21.15.114:0/539108810 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55717a95d7c0 0x55717a968350 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:40.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.981+0000 7fbf2b472740 1 -- 172.21.15.114:0/539108810 >> 172.21.15.114:0/539108810 conn(0x55717a959d50 msgr2=0x55717a95f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:40.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.981+0000 7fbf2b472740 1 -- 172.21.15.114:0/539108810 shutdown_connections 2024-09-13T18:23:40.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.981+0000 7fbf2b472740 1 -- 172.21.15.114:0/539108810 wait complete. 2024-09-13T18:23:40.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.981+0000 7fbf2b472740 20 client.16170 trim_cache size 0 max 16384 2024-09-13T18:23:40.997 INFO:tasks.workunit.client.0.smithi114.stdout:nroff -mandoc -Tascii src/iogen.8 > iogen.cat8 2024-09-13T18:23:41.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.998+0000 7fbf2b472740 1 Processor -- start 2024-09-13T18:23:41.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.998+0000 7fbf2b472740 1 -- start start 2024-09-13T18:23:41.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.998+0000 7fbf2b472740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55717a95d7c0 0x55717a96f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:41.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.998+0000 7fbf2b472740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55717a95e0e0 0x55717a96fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:41.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.998+0000 7fbf2b472740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55717a968980 0x55717a963ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:41.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.998+0000 7fbf2b472740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55717a95cfd0 con 0x55717a968980 2024-09-13T18:23:41.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.998+0000 7fbf2b472740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55717a95ded0 con 0x55717a95d7c0 2024-09-13T18:23:41.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.998+0000 7fbf2b472740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55717a974cc0 con 0x55717a95e0e0 2024-09-13T18:23:41.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.998+0000 7fbf26cf0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55717a968980 0x55717a963ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:41.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.999+0000 7fbf25cee640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55717a95e0e0 0x55717a96fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:41.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.999+0000 7fbf264ef640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55717a95d7c0 0x55717a96f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:41.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.999+0000 7fbf25cee640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55717a95e0e0 0x55717a96fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:56178/0 (socket says 172.21.15.114:56178) 2024-09-13T18:23:41.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.999+0000 7fbf25cee640 1 -- 172.21.15.114:0/3841834717 learned_addr learned my addr 172.21.15.114:0/3841834717 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:41.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.999+0000 7fbf264ef640 1 --2- 172.21.15.114:0/3841834717 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55717a95d7c0 0x55717a96f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:51210/0 (socket says 172.21.15.114:51210) 2024-09-13T18:23:41.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.999+0000 7fbf25cee640 1 -- 172.21.15.114:0/3841834717 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55717a95d7c0 msgr2=0x55717a96f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:41.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.999+0000 7fbf25cee640 1 --2- 172.21.15.114:0/3841834717 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55717a95d7c0 0x55717a96f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:41.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.999+0000 7fbf25cee640 1 -- 172.21.15.114:0/3841834717 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55717a968980 msgr2=0x55717a963ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:41.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.999+0000 7fbf25cee640 1 --2- 172.21.15.114:0/3841834717 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55717a968980 0x55717a963ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:41.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.999+0000 7fbf25cee640 1 -- 172.21.15.114:0/3841834717 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55717a8621c0 con 0x55717a95e0e0 2024-09-13T18:23:41.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.999+0000 7fbf264ef640 1 --2- 172.21.15.114:0/3841834717 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55717a95d7c0 0x55717a96f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:23:41.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.999+0000 7fbf25cee640 1 --2- 172.21.15.114:0/3841834717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55717a95e0e0 0x55717a96fb90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbf10005ab0 tx=0x7fbf10005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f76ee2fec439eaa7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:41.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:40.999+0000 7fbf157fa640 1 -- 172.21.15.114:0/3841834717 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf10043040 con 0x55717a95e0e0 2024-09-13T18:23:41.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.000+0000 7fbf157fa640 1 -- 172.21.15.114:0/3841834717 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbf10045e30 con 0x55717a95e0e0 2024-09-13T18:23:41.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.000+0000 7fbf157fa640 1 -- 172.21.15.114:0/3841834717 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf100035f0 con 0x55717a95e0e0 2024-09-13T18:23:41.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.000+0000 7fbf2b472740 1 -- 172.21.15.114:0/3841834717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55717a95e0e0 msgr2=0x55717a96fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:41.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.000+0000 7fbf2b472740 1 --2- 172.21.15.114:0/3841834717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55717a95e0e0 0x55717a96fb90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbf10005ab0 tx=0x7fbf10005a20 comp rx=0 tx=0).stop 2024-09-13T18:23:41.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.000+0000 7fbf2b472740 1 -- 172.21.15.114:0/3841834717 shutdown_connections 2024-09-13T18:23:41.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.000+0000 7fbf2b472740 1 --2- 172.21.15.114:0/3841834717 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55717a968980 0x55717a963ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:41.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.000+0000 7fbf2b472740 1 --2- 172.21.15.114:0/3841834717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55717a95e0e0 0x55717a96fb90 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:41.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.000+0000 7fbf2b472740 1 --2- 172.21.15.114:0/3841834717 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55717a95d7c0 0x55717a96f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:41.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.000+0000 7fbf2b472740 1 -- 172.21.15.114:0/3841834717 >> 172.21.15.114:0/3841834717 conn(0x55717a959d50 msgr2=0x55717a9608e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:41.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.000+0000 7fbf2b472740 1 -- 172.21.15.114:0/3841834717 shutdown_connections 2024-09-13T18:23:41.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.000+0000 7fbf2b472740 1 -- 172.21.15.114:0/3841834717 wait complete. 2024-09-13T18:23:41.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.001+0000 7fbf2b472740 1 -- start start 2024-09-13T18:23:41.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.001+0000 7fbf2b472740 1 Processor -- start 2024-09-13T18:23:41.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.001+0000 7fbf2b472740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55717a95d7c0 0x55717a96e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:41.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.001+0000 7fbf2b472740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55717a95e0e0 0x55717a96ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:41.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.001+0000 7fbf2b472740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55717a968980 0x55717a9cea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:41.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.001+0000 7fbf2b472740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55717a9cef80 con 0x55717a968980 2024-09-13T18:23:41.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.001+0000 7fbf2b472740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55717a9cf0c0 con 0x55717a95d7c0 2024-09-13T18:23:41.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.001+0000 7fbf2b472740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55717a9cf200 con 0x55717a95e0e0 2024-09-13T18:23:41.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.001+0000 7fbf2b472740 10 client.0 _ll_register_callbacks cb 0x7fbf1c099960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:23:41.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.001+0000 7fbf26cf0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55717a968980 0x55717a9cea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:41.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.001+0000 7fbf26cf0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55717a968980 0x55717a9cea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51620/0 (socket says 172.21.15.114:51620) 2024-09-13T18:23:41.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.001+0000 7fbf26cf0640 1 -- 172.21.15.114:0/4101292917 learned_addr learned my addr 172.21.15.114:0/4101292917 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:41.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.001+0000 7fbf264ef640 1 --2- 172.21.15.114:0/4101292917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55717a95d7c0 0x55717a96e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:41.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.001+0000 7fbf25cee640 1 --2- 172.21.15.114:0/4101292917 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55717a95e0e0 0x55717a96ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:41.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.002+0000 7fbf26cf0640 1 -- 172.21.15.114:0/4101292917 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55717a95e0e0 msgr2=0x55717a96ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:41.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.002+0000 7fbf26cf0640 1 --2- 172.21.15.114:0/4101292917 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55717a95e0e0 0x55717a96ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:41.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.002+0000 7fbf26cf0640 1 -- 172.21.15.114:0/4101292917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55717a95d7c0 msgr2=0x55717a96e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:41.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.002+0000 7fbf26cf0640 1 --2- 172.21.15.114:0/4101292917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55717a95d7c0 0x55717a96e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:41.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.002+0000 7fbf26cf0640 1 -- 172.21.15.114:0/4101292917 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55717a8c0360 con 0x55717a968980 2024-09-13T18:23:41.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.002+0000 7fbf264ef640 1 --2- 172.21.15.114:0/4101292917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55717a95d7c0 0x55717a96e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:23:41.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.002+0000 7fbf26cf0640 1 --2- 172.21.15.114:0/4101292917 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55717a968980 0x55717a9cea40 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fbf180148d0 tx=0x7fbf18004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e30b9bed12291489 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:41.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.002+0000 7fbf09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:23:41.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.002+0000 7fbf09ffb640 1 -- 172.21.15.114:0/4101292917 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1802a040 con 0x55717a968980 2024-09-13T18:23:41.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.002+0000 7fbf09ffb640 1 -- 172.21.15.114:0/4101292917 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fbf180099a0 con 0x55717a968980 2024-09-13T18:23:41.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.002+0000 7fbf2b472740 1 -- 172.21.15.114:0/4101292917 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbf180141b0 con 0x55717a968980 2024-09-13T18:23:41.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.002+0000 7fbf09ffb640 1 -- 172.21.15.114:0/4101292917 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf18014050 con 0x55717a968980 2024-09-13T18:23:41.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.002+0000 7fbf2b472740 10 client.16176 fetch_fsmap learned FSMap version 30 2024-09-13T18:23:41.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.002+0000 7fbf2b472740 1 -- 172.21.15.114:0/4101292917 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7fbf10004310 con 0x55717a968980 2024-09-13T18:23:41.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.002+0000 7fbf09ffb640 1 -- 172.21.15.114:0/4101292917 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbf180141b0 con 0x55717a968980 2024-09-13T18:23:41.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.002+0000 7fbf09ffb640 1 -- 172.21.15.114:0/4101292917 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbf1802d070 con 0x55717a968980 2024-09-13T18:23:41.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.003+0000 7fbf2b472740 10 client.16176 fetch_fsmap finished waiting for FSMap version 30 2024-09-13T18:23:41.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.003+0000 7fbf2b472740 10 client.16176 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:23:41.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.003+0000 7fbf2b472740 20 client.16176 populate_metadata read hostname 'smithi114' 2024-09-13T18:23:41.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.003+0000 7fbf2b472740 1 --2- 172.21.15.114:0/4101292917 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55717a9785c0 0x55717a9d2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:41.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.003+0000 7fbf2b472740 4 client.16176 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-13T18:23:41.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.003+0000 7fbf2b472740 1 -- 172.21.15.114:0/4101292917 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55717a619eb0 con 0x55717a9785c0 2024-09-13T18:23:41.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.003+0000 7fbf264ef640 1 --2- 172.21.15.114:0/4101292917 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55717a9785c0 0x55717a9d2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:41.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.003+0000 7fbf09ffb640 10 client.16176 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:23:41.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.003+0000 7fbf264ef640 1 --2- 172.21.15.114:0/4101292917 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55717a9785c0 0x55717a9d2930 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a19f35bd308c905 server_cookie=baf5ccc6d2d9d37d in_seq=0 out_seq=0 2024-09-13T18:23:41.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.004+0000 7fbf09ffb640 1 -- 172.21.15.114:0/4101292917 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55717a619eb0 con 0x55717a9785c0 2024-09-13T18:23:41.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.004+0000 7fbf09ffb640 10 client.16176 handle_command_reply: tid=0 2024-09-13T18:23:41.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.004+0000 7fbf2b472740 1 client.16176 shutdown 2024-09-13T18:23:41.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.004+0000 7fbf2b472740 1 -- 172.21.15.114:0/4101292917 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55717a968980 msgr2=0x55717a9cea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:41.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.004+0000 7fbf2b472740 1 --2- 172.21.15.114:0/4101292917 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55717a968980 0x55717a9cea40 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fbf180148d0 tx=0x7fbf18004f90 comp rx=0 tx=0).stop 2024-09-13T18:23:41.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.004+0000 7fbf2b472740 1 -- 172.21.15.114:0/4101292917 shutdown_connections 2024-09-13T18:23:41.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.004+0000 7fbf2b472740 1 --2- 172.21.15.114:0/4101292917 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55717a9785c0 0x55717a9d2930 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:41.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.004+0000 7fbf2b472740 1 --2- 172.21.15.114:0/4101292917 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55717a968980 0x55717a9cea40 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:41.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.004+0000 7fbf09ffb640 0 client.16176 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:23:41.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.004+0000 7fbf2b472740 1 --2- 172.21.15.114:0/4101292917 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55717a95e0e0 0x55717a96ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:41.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.004+0000 7fbf2b472740 1 --2- 172.21.15.114:0/4101292917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55717a95d7c0 0x55717a96e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:41.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.004+0000 7fbf2b472740 1 -- 172.21.15.114:0/4101292917 >> 172.21.15.114:0/4101292917 conn(0x55717a959d50 msgr2=0x55717a963d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:41.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.004+0000 7fbf2b472740 1 -- 172.21.15.114:0/4101292917 shutdown_connections 2024-09-13T18:23:41.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.005+0000 7fbf2b472740 1 -- 172.21.15.114:0/4101292917 wait complete. 2024-09-13T18:23:41.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.005+0000 7fbf2b472740 20 client.16176 trim_cache size 0 max 16384 2024-09-13T18:23:41.013 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:23:41.014 INFO:teuthology.orchestra.run.smithi114.stdout: "return_code": 0, 2024-09-13T18:23:41.014 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_tag": "3fc5a7f0-d4bf-4907-9af5-af1128853e75", 2024-09-13T18:23:41.014 INFO:teuthology.orchestra.run.smithi114.stdout: "mode": "asynchronous" 2024-09-13T18:23:41.014 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:23:41.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.005+0000 7fbf28753640 1 -- 172.21.15.114:0/825549145 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbf0c0a1350 msgr2=0x7fbf0c0a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:41.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.005+0000 7fbf28753640 1 --2- 172.21.15.114:0/825549145 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbf0c0a1350 0x7fbf0c0a3810 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbf1801e000 tx=0x7fbf1801b000 comp rx=0 tx=0).stop 2024-09-13T18:23:41.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.005+0000 7fbf28753640 1 -- 172.21.15.114:0/825549145 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf200ff5a0 msgr2=0x7fbf200fa800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:41.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.005+0000 7fbf28753640 1 --2- 172.21.15.114:0/825549145 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf200ff5a0 0x7fbf200fa800 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c00c890 tx=0x7fbf1c00cd50 comp rx=0 tx=0).stop 2024-09-13T18:23:41.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.005+0000 7fbf28753640 1 -- 172.21.15.114:0/825549145 shutdown_connections 2024-09-13T18:23:41.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.005+0000 7fbf28753640 1 --2- 172.21.15.114:0/825549145 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbf0c0a1350 0x7fbf0c0a3810 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:41.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.005+0000 7fbf28753640 1 --2- 172.21.15.114:0/825549145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf200fb280 0x7fbf201a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:41.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.005+0000 7fbf28753640 1 --2- 172.21.15.114:0/825549145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf201041d0 0x7fbf200fad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:41.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.005+0000 7fbf28753640 1 --2- 172.21.15.114:0/825549145 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf200ff5a0 0x7fbf200fa800 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:41.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.005+0000 7fbf28753640 1 -- 172.21.15.114:0/825549145 >> 172.21.15.114:0/825549145 conn(0x7fbf200753f0 msgr2=0x7fbf200fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:41.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.005+0000 7fbf28753640 1 -- 172.21.15.114:0/825549145 shutdown_connections 2024-09-13T18:23:41.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.005+0000 7fbf28753640 1 -- 172.21.15.114:0/825549145 wait complete. 2024-09-13T18:23:41.018 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T18:23:41.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.118+0000 7f5825308640 1 -- 172.21.15.114:0/3151056804 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58201050a0 msgr2=0x7f582010cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:41.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.118+0000 7f5825308640 1 --2- 172.21.15.114:0/3151056804 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58201050a0 0x7f582010cbf0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f5818009cd0 tx=0x7f581802a8e0 comp rx=0 tx=0).stop 2024-09-13T18:23:41.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.118+0000 7f5825308640 1 -- 172.21.15.114:0/3151056804 shutdown_connections 2024-09-13T18:23:41.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.118+0000 7f5825308640 1 --2- 172.21.15.114:0/3151056804 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58201050a0 0x7f582010cbf0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:41.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.118+0000 7f5825308640 1 --2- 172.21.15.114:0/3151056804 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5820100900 0x7f5820104940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:41.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.118+0000 7f5825308640 1 --2- 172.21.15.114:0/3151056804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58200fff50 0x7f5820100330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:41.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.118+0000 7f5825308640 1 -- 172.21.15.114:0/3151056804 >> 172.21.15.114:0/3151056804 conn(0x7f58200fb9e0 msgr2=0x7f58200fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:41.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.118+0000 7f5825308640 1 -- 172.21.15.114:0/3151056804 shutdown_connections 2024-09-13T18:23:41.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.118+0000 7f5825308640 1 -- 172.21.15.114:0/3151056804 wait complete. 2024-09-13T18:23:41.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.118+0000 7f5825308640 1 Processor -- start 2024-09-13T18:23:41.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.118+0000 7f5825308640 1 -- start start 2024-09-13T18:23:41.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.119+0000 7f5825308640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58200fff50 0x7f5820196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:41.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.119+0000 7f5825308640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58201050a0 0x7f58201967c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:41.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.119+0000 7f5825308640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f582019abb0 0x7f582019b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:41.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.119+0000 7f5825308640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582019b5a0 con 0x7f58200fff50 2024-09-13T18:23:41.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.119+0000 7f5825308640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582019b710 con 0x7f58201050a0 2024-09-13T18:23:41.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.119+0000 7f5825308640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582019b880 con 0x7f582019abb0 2024-09-13T18:23:41.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.119+0000 7f581effd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58200fff50 0x7f5820196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:41.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.119+0000 7f581effd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58200fff50 0x7f5820196280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51628/0 (socket says 172.21.15.114:51628) 2024-09-13T18:23:41.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.119+0000 7f581effd640 1 -- 172.21.15.114:0/2426787578 learned_addr learned my addr 172.21.15.114:0/2426787578 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:41.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.119+0000 7f581f7fe640 1 --2- 172.21.15.114:0/2426787578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f582019abb0 0x7f582019b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:41.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.119+0000 7f581e7fc640 1 --2- 172.21.15.114:0/2426787578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58201050a0 0x7f58201967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:41.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.119+0000 7f581effd640 1 -- 172.21.15.114:0/2426787578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f582019abb0 msgr2=0x7f582019b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:41.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.119+0000 7f581effd640 1 --2- 172.21.15.114:0/2426787578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f582019abb0 0x7f582019b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:41.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.119+0000 7f581effd640 1 -- 172.21.15.114:0/2426787578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58201050a0 msgr2=0x7f58201967c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:41.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.119+0000 7f581effd640 1 --2- 172.21.15.114:0/2426787578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58201050a0 0x7f58201967c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:41.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.119+0000 7f581effd640 1 -- 172.21.15.114:0/2426787578 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5818009900 con 0x7f58200fff50 2024-09-13T18:23:41.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.119+0000 7f581e7fc640 1 --2- 172.21.15.114:0/2426787578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58201050a0 0x7f58201967c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:23:41.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.119+0000 7f581effd640 1 --2- 172.21.15.114:0/2426787578 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58200fff50 0x7f5820196280 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f581000c970 tx=0x7f581000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:41.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.119+0000 7f5807fff640 1 -- 172.21.15.114:0/2426787578 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5810007bb0 con 0x7f58200fff50 2024-09-13T18:23:41.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.119+0000 7f5807fff640 1 -- 172.21.15.114:0/2426787578 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5810007d10 con 0x7f58200fff50 2024-09-13T18:23:41.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.119+0000 7f5825308640 1 -- 172.21.15.114:0/2426787578 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58201a2e60 con 0x7f58200fff50 2024-09-13T18:23:41.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.119+0000 7f5807fff640 1 -- 172.21.15.114:0/2426787578 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5810004d70 con 0x7f58200fff50 2024-09-13T18:23:41.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.119+0000 7f5825308640 1 -- 172.21.15.114:0/2426787578 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58201a3230 con 0x7f58200fff50 2024-09-13T18:23:41.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.122+0000 7f5807fff640 1 -- 172.21.15.114:0/2426787578 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) v1 ==== 136107+0+0 (secure 0 0 0) 0x7f5810005020 con 0x7f58200fff50 2024-09-13T18:23:41.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.122+0000 7f5828029740 1 Processor -- start 2024-09-13T18:23:41.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.122+0000 7f5828029740 1 -- start start 2024-09-13T18:23:41.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.122+0000 7f5828029740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556919454e10 0x5569194551f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:41.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.122+0000 7f5828029740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556919455730 0x55691945fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:41.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.122+0000 7f5828029740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55691945ffd0 0x5569194623c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:41.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.122+0000 7f5828029740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569193bff20 con 0x556919454e10 2024-09-13T18:23:41.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.122+0000 7f5828029740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569193bd6b0 con 0x556919455730 2024-09-13T18:23:41.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.122+0000 7f5828029740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569193b8fa0 con 0x55691945ffd0 2024-09-13T18:23:41.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.122+0000 7f581f7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556919454e10 0x5569194551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:41.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.122+0000 7f581f7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556919454e10 0x5569194551f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51638/0 (socket says 172.21.15.114:51638) 2024-09-13T18:23:41.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.122+0000 7f581f7fe640 1 -- 172.21.15.114:0/2010685220 learned_addr learned my addr 172.21.15.114:0/2010685220 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:41.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.122+0000 7f5807fff640 1 --2- 172.21.15.114:0/2426787578 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f58080a1320 0x7f58080a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:41.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.122+0000 7f581e7fc640 1 --2- 172.21.15.114:0/2010685220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55691945ffd0 0x5569194623c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:41.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.122+0000 7f581f7fe640 1 -- 172.21.15.114:0/2010685220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55691945ffd0 msgr2=0x5569194623c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:41.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.122+0000 7f581f7fe640 1 --2- 172.21.15.114:0/2010685220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55691945ffd0 0x5569194623c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:41.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.122+0000 7f581effd640 1 --2- 172.21.15.114:0/2010685220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556919455730 0x55691945fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:41.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.122+0000 7f581f7fe640 1 -- 172.21.15.114:0/2010685220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556919455730 msgr2=0x55691945fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:41.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.123+0000 7f581f7fe640 1 --2- 172.21.15.114:0/2010685220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556919455730 0x55691945fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:41.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.123+0000 7f581f7fe640 1 -- 172.21.15.114:0/2010685220 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556919373840 con 0x556919454e10 2024-09-13T18:23:41.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.123+0000 7f581effd640 1 --2- 172.21.15.114:0/2010685220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556919455730 0x55691945fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:23:41.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.123+0000 7f5807fff640 1 -- 172.21.15.114:0/2426787578 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f58100cd030 con 0x7f58200fff50 2024-09-13T18:23:41.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.123+0000 7f581f7fe640 1 --2- 172.21.15.114:0/2426787578 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f58080a1320 0x7f58080a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:41.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.123+0000 7f581f7fe640 1 --2- 172.21.15.114:0/2010685220 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556919454e10 0x5569194551f0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f5818009430 tx=0x7f5818031580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efdce04f5ca54da8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:41.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.123+0000 7f58057fa640 1 -- 172.21.15.114:0/2010685220 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581804d070 con 0x556919454e10 2024-09-13T18:23:41.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.123+0000 7f58057fa640 1 -- 172.21.15.114:0/2010685220 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5818030e20 con 0x556919454e10 2024-09-13T18:23:41.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.123+0000 7f58057fa640 1 -- 172.21.15.114:0/2010685220 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5818004040 con 0x556919454e10 2024-09-13T18:23:41.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.124+0000 7f5828029740 1 -- 172.21.15.114:0/2010685220 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556919454e10 msgr2=0x5569194551f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:41.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.124+0000 7f581f7fe640 1 --2- 172.21.15.114:0/2426787578 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f58080a1320 0x7f58080a37e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5818006010 tx=0x7f5818049000 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:41.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.124+0000 7f5828029740 1 --2- 172.21.15.114:0/2010685220 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556919454e10 0x5569194551f0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f5818009430 tx=0x7f5818031580 comp rx=0 tx=0).stop 2024-09-13T18:23:41.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.124+0000 7f5828029740 1 -- 172.21.15.114:0/2010685220 shutdown_connections 2024-09-13T18:23:41.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.124+0000 7f5828029740 1 --2- 172.21.15.114:0/2010685220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55691945ffd0 0x5569194623c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:41.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.124+0000 7f5828029740 1 --2- 172.21.15.114:0/2010685220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556919455730 0x55691945fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:41.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.124+0000 7f5828029740 1 --2- 172.21.15.114:0/2010685220 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556919454e10 0x5569194551f0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:41.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.124+0000 7f5828029740 1 -- 172.21.15.114:0/2010685220 >> 172.21.15.114:0/2010685220 conn(0x556919451380 msgr2=0x556919451760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:41.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.124+0000 7f5828029740 1 -- 172.21.15.114:0/2010685220 shutdown_connections 2024-09-13T18:23:41.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.124+0000 7f5828029740 1 -- 172.21.15.114:0/2010685220 wait complete. 2024-09-13T18:23:41.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.124+0000 7f5828029740 1 -- start start 2024-09-13T18:23:41.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.124+0000 7f5828029740 1 Processor -- start 2024-09-13T18:23:41.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.124+0000 7f5828029740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556919454e10 0x55691945f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:41.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.124+0000 7f5828029740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556919455730 0x55691945cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:41.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.124+0000 7f5828029740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55691945ffd0 0x55691945d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:41.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.124+0000 7f5828029740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55691935a2a0 con 0x55691945ffd0 2024-09-13T18:23:41.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.124+0000 7f5828029740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556919091430 con 0x556919454e10 2024-09-13T18:23:41.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.124+0000 7f5828029740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55691933c680 con 0x556919455730 2024-09-13T18:23:41.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.124+0000 7f5828029740 10 client.0 _ll_register_callbacks cb 0x5569193b2610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:23:41.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.125+0000 7f581f7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55691945ffd0 0x55691945d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:41.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.125+0000 7f581f7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55691945ffd0 0x55691945d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51642/0 (socket says 172.21.15.114:51642) 2024-09-13T18:23:41.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.125+0000 7f581f7fe640 1 -- 172.21.15.114:0/787220236 learned_addr learned my addr 172.21.15.114:0/787220236 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:41.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.125+0000 7f581effd640 1 --2- 172.21.15.114:0/787220236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556919454e10 0x55691945f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:41.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.125+0000 7f581e7fc640 1 --2- 172.21.15.114:0/787220236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556919455730 0x55691945cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:41.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.125+0000 7f581f7fe640 1 -- 172.21.15.114:0/787220236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556919455730 msgr2=0x55691945cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:41.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.125+0000 7f581f7fe640 1 --2- 172.21.15.114:0/787220236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556919455730 0x55691945cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:41.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.125+0000 7f581f7fe640 1 -- 172.21.15.114:0/787220236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556919454e10 msgr2=0x55691945f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:41.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.125+0000 7f581f7fe640 1 --2- 172.21.15.114:0/787220236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556919454e10 0x55691945f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:41.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.125+0000 7f581f7fe640 1 -- 172.21.15.114:0/787220236 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f582010d430 con 0x55691945ffd0 2024-09-13T18:23:41.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.125+0000 7f581e7fc640 1 --2- 172.21.15.114:0/787220236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556919455730 0x55691945cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:23:41.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.125+0000 7f581f7fe640 1 --2- 172.21.15.114:0/787220236 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55691945ffd0 0x55691945d230 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f5818005e30 tx=0x7f5818003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5de82bf0739a2bf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:41.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.125+0000 7f5802ff5640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:23:41.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.125+0000 7f5802ff5640 1 -- 172.21.15.114:0/787220236 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581804d040 con 0x55691945ffd0 2024-09-13T18:23:41.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.125+0000 7f5802ff5640 1 -- 172.21.15.114:0/787220236 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f5818044030 con 0x55691945ffd0 2024-09-13T18:23:41.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.125+0000 7f5828029740 1 -- 172.21.15.114:0/787220236 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5569190ac4e0 con 0x55691945ffd0 2024-09-13T18:23:41.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.125+0000 7f5802ff5640 1 -- 172.21.15.114:0/787220236 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5818031b90 con 0x55691945ffd0 2024-09-13T18:23:41.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.125+0000 7f5828029740 10 client.16188 fetch_fsmap learned FSMap version 30 2024-09-13T18:23:41.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.125+0000 7f5828029740 1 -- 172.21.15.114:0/787220236 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5569193b7620 con 0x55691945ffd0 2024-09-13T18:23:41.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.126+0000 7f5802ff5640 1 -- 172.21.15.114:0/787220236 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5569190ac4e0 con 0x55691945ffd0 2024-09-13T18:23:41.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.126+0000 7f5802ff5640 1 -- 172.21.15.114:0/787220236 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5818052910 con 0x55691945ffd0 2024-09-13T18:23:41.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.126+0000 7f5828029740 10 client.16188 fetch_fsmap finished waiting for FSMap version 30 2024-09-13T18:23:41.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.126+0000 7f5828029740 10 client.16188 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:23:41.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.126+0000 7f5828029740 20 client.16188 populate_metadata read hostname 'smithi114' 2024-09-13T18:23:41.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.126+0000 7f5828029740 1 --2- 172.21.15.114:0/787220236 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55691946f9c0 0x556919471de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:41.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.126+0000 7f5828029740 4 client.16188 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:23:41.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.126+0000 7f5828029740 1 -- 172.21.15.114:0/787220236 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5569190bd7e0 con 0x55691946f9c0 2024-09-13T18:23:41.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.126+0000 7f581effd640 1 --2- 172.21.15.114:0/787220236 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55691946f9c0 0x556919471de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:41.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.126+0000 7f5802ff5640 10 client.16188 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:23:41.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.126+0000 7f581effd640 1 --2- 172.21.15.114:0/787220236 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55691946f9c0 0x556919471de0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1a93e7c55e836db server_cookie=f2549d86d91568bb in_seq=0 out_seq=0 2024-09-13T18:23:41.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.127+0000 7f5802ff5640 1 -- 172.21.15.114:0/787220236 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5569190bd7e0 con 0x55691946f9c0 2024-09-13T18:23:41.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.127+0000 7f5802ff5640 10 client.16188 handle_command_reply: tid=0 2024-09-13T18:23:41.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.127+0000 7f5828029740 1 client.16188 shutdown 2024-09-13T18:23:41.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.127+0000 7f5828029740 1 -- 172.21.15.114:0/787220236 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55691945ffd0 msgr2=0x55691945d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:41.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.127+0000 7f5828029740 1 --2- 172.21.15.114:0/787220236 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55691945ffd0 0x55691945d230 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f5818005e30 tx=0x7f5818003c00 comp rx=0 tx=0).stop 2024-09-13T18:23:41.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.127+0000 7f5828029740 1 -- 172.21.15.114:0/787220236 shutdown_connections 2024-09-13T18:23:41.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.127+0000 7f5828029740 1 --2- 172.21.15.114:0/787220236 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55691946f9c0 0x556919471de0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:41.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.127+0000 7f5828029740 1 --2- 172.21.15.114:0/787220236 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55691945ffd0 0x55691945d230 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:41.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.127+0000 7f5828029740 1 --2- 172.21.15.114:0/787220236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556919455730 0x55691945cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:41.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.127+0000 7f5828029740 1 --2- 172.21.15.114:0/787220236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556919454e10 0x55691945f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:41.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.127+0000 7f5802ff5640 0 client.16188 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:23:41.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.127+0000 7f5828029740 1 -- 172.21.15.114:0/787220236 >> 172.21.15.114:0/787220236 conn(0x556919451380 msgr2=0x5569194570e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:41.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.127+0000 7f5828029740 1 -- 172.21.15.114:0/787220236 shutdown_connections 2024-09-13T18:23:41.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.127+0000 7f5828029740 1 -- 172.21.15.114:0/787220236 wait complete. 2024-09-13T18:23:41.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.127+0000 7f5828029740 20 client.16188 trim_cache size 0 max 16384 2024-09-13T18:23:41.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.141+0000 7f5828029740 1 Processor -- start 2024-09-13T18:23:41.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.141+0000 7f5828029740 1 -- start start 2024-09-13T18:23:41.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.141+0000 7f5828029740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556919454e10 0x556919450d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:41.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.141+0000 7f5828029740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556919455730 0x556919466d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:41.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.141+0000 7f5828029740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55691945ffd0 0x556919467270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:41.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.141+0000 7f5828029740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569190ac4e0 con 0x55691945ffd0 2024-09-13T18:23:41.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.141+0000 7f5828029740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55691944c760 con 0x556919454e10 2024-09-13T18:23:41.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.141+0000 7f5828029740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569194579e0 con 0x556919455730 2024-09-13T18:23:41.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.141+0000 7f581f7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55691945ffd0 0x556919467270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:41.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.141+0000 7f581f7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55691945ffd0 0x556919467270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51644/0 (socket says 172.21.15.114:51644) 2024-09-13T18:23:41.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.141+0000 7f581f7fe640 1 -- 172.21.15.114:0/3328641434 learned_addr learned my addr 172.21.15.114:0/3328641434 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:41.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.142+0000 7f581effd640 1 --2- 172.21.15.114:0/3328641434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556919454e10 0x556919450d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:41.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.142+0000 7f581f7fe640 1 -- 172.21.15.114:0/3328641434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556919455730 msgr2=0x556919466d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:41.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.142+0000 7f581f7fe640 1 --2- 172.21.15.114:0/3328641434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556919455730 0x556919466d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:41.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.142+0000 7f581f7fe640 1 -- 172.21.15.114:0/3328641434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556919454e10 msgr2=0x556919450d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:41.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.142+0000 7f581f7fe640 1 --2- 172.21.15.114:0/3328641434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556919454e10 0x556919450d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:41.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.142+0000 7f581f7fe640 1 -- 172.21.15.114:0/3328641434 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556919359790 con 0x55691945ffd0 2024-09-13T18:23:41.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.142+0000 7f581effd640 1 --2- 172.21.15.114:0/3328641434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556919454e10 0x556919450d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:23:41.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.142+0000 7f581f7fe640 1 --2- 172.21.15.114:0/3328641434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55691945ffd0 0x556919467270 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f581802ae00 tx=0x7f5818030830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f50da76dd2a37d4c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:41.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.142+0000 7f5805ffb640 1 -- 172.21.15.114:0/3328641434 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581804d030 con 0x55691945ffd0 2024-09-13T18:23:41.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.142+0000 7f5805ffb640 1 -- 172.21.15.114:0/3328641434 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5818003a30 con 0x55691945ffd0 2024-09-13T18:23:41.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.142+0000 7f5805ffb640 1 -- 172.21.15.114:0/3328641434 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5818004580 con 0x55691945ffd0 2024-09-13T18:23:41.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.142+0000 7f5828029740 1 -- 172.21.15.114:0/3328641434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55691945ffd0 msgr2=0x556919467270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:41.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.142+0000 7f5828029740 1 --2- 172.21.15.114:0/3328641434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55691945ffd0 0x556919467270 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f581802ae00 tx=0x7f5818030830 comp rx=0 tx=0).stop 2024-09-13T18:23:41.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.142+0000 7f5828029740 1 -- 172.21.15.114:0/3328641434 shutdown_connections 2024-09-13T18:23:41.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.142+0000 7f5828029740 1 --2- 172.21.15.114:0/3328641434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55691945ffd0 0x556919467270 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:41.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.142+0000 7f5828029740 1 --2- 172.21.15.114:0/3328641434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556919455730 0x556919466d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:41.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.142+0000 7f5828029740 1 --2- 172.21.15.114:0/3328641434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556919454e10 0x556919450d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:41.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.142+0000 7f5828029740 1 -- 172.21.15.114:0/3328641434 >> 172.21.15.114:0/3328641434 conn(0x556919451380 msgr2=0x556919451760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:41.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.142+0000 7f5828029740 1 -- 172.21.15.114:0/3328641434 shutdown_connections 2024-09-13T18:23:41.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.143+0000 7f5828029740 1 -- 172.21.15.114:0/3328641434 wait complete. 2024-09-13T18:23:41.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.143+0000 7f5828029740 1 -- start start 2024-09-13T18:23:41.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.143+0000 7f5828029740 1 Processor -- start 2024-09-13T18:23:41.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.143+0000 7f5828029740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556919454e10 0x556919462f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:41.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.143+0000 7f5828029740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556919455730 0x556919463460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:41.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.143+0000 7f5828029740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55691945ffd0 0x5569194c4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:41.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.143+0000 7f5828029740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x556919463bc0 con 0x556919454e10 2024-09-13T18:23:41.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.143+0000 7f5828029740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569194c5410 con 0x55691945ffd0 2024-09-13T18:23:41.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.143+0000 7f5828029740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569194c55d0 con 0x556919455730 2024-09-13T18:23:41.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.143+0000 7f5828029740 10 client.0 _ll_register_callbacks cb 0x7f581804b020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:23:41.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.143+0000 7f581effd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556919454e10 0x556919462f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:41.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.143+0000 7f581effd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556919454e10 0x556919462f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51660/0 (socket says 172.21.15.114:51660) 2024-09-13T18:23:41.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.143+0000 7f581effd640 1 -- 172.21.15.114:0/1927686308 learned_addr learned my addr 172.21.15.114:0/1927686308 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:41.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.143+0000 7f581e7fc640 1 --2- 172.21.15.114:0/1927686308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556919455730 0x556919463460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:41.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.143+0000 7f581f7fe640 1 --2- 172.21.15.114:0/1927686308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55691945ffd0 0x5569194c4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:41.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.143+0000 7f581effd640 1 -- 172.21.15.114:0/1927686308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556919455730 msgr2=0x556919463460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:41.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.143+0000 7f581effd640 1 --2- 172.21.15.114:0/1927686308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556919455730 0x556919463460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:41.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.143+0000 7f581effd640 1 -- 172.21.15.114:0/1927686308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55691945ffd0 msgr2=0x5569194c4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:41.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.143+0000 7f581effd640 1 --2- 172.21.15.114:0/1927686308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55691945ffd0 0x5569194c4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:41.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.143+0000 7f581effd640 1 -- 172.21.15.114:0/1927686308 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5569193b7620 con 0x556919454e10 2024-09-13T18:23:41.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.143+0000 7f581f7fe640 1 --2- 172.21.15.114:0/1927686308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55691945ffd0 0x5569194c4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:23:41.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.144+0000 7f581effd640 1 --2- 172.21.15.114:0/1927686308 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556919454e10 0x556919462f20 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f58100960a0 tx=0x7f581001d7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85d781e777c75840 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:41.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.144+0000 7f5802ff5640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:23:41.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.144+0000 7f5802ff5640 1 -- 172.21.15.114:0/1927686308 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581009d9d0 con 0x556919454e10 2024-09-13T18:23:41.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.144+0000 7f5802ff5640 1 -- 172.21.15.114:0/1927686308 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f58100b9040 con 0x556919454e10 2024-09-13T18:23:41.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.144+0000 7f5828029740 1 -- 172.21.15.114:0/1927686308 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5818030e20 con 0x556919454e10 2024-09-13T18:23:41.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.144+0000 7f5802ff5640 1 -- 172.21.15.114:0/1927686308 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58100a2020 con 0x556919454e10 2024-09-13T18:23:41.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.144+0000 7f5828029740 10 client.16194 fetch_fsmap learned FSMap version 30 2024-09-13T18:23:41.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.144+0000 7f5828029740 1 -- 172.21.15.114:0/1927686308 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55691937eea0 con 0x556919454e10 2024-09-13T18:23:41.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.144+0000 7f5802ff5640 1 -- 172.21.15.114:0/1927686308 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5818030e20 con 0x556919454e10 2024-09-13T18:23:41.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.144+0000 7f5802ff5640 1 -- 172.21.15.114:0/1927686308 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5810016030 con 0x556919454e10 2024-09-13T18:23:41.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.144+0000 7f5828029740 10 client.16194 fetch_fsmap finished waiting for FSMap version 30 2024-09-13T18:23:41.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.144+0000 7f5828029740 10 client.16194 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:23:41.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.144+0000 7f5828029740 20 client.16194 populate_metadata read hostname 'smithi114' 2024-09-13T18:23:41.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.144+0000 7f5828029740 1 --2- 172.21.15.114:0/1927686308 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5569194c6cd0 0x5569194c90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:41.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.144+0000 7f5828029740 4 client.16194 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T18:23:41.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.144+0000 7f5828029740 1 -- 172.21.15.114:0/1927686308 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556919110e30 con 0x5569194c6cd0 2024-09-13T18:23:41.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.145+0000 7f581e7fc640 1 --2- 172.21.15.114:0/1927686308 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5569194c6cd0 0x5569194c90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:41.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.145+0000 7f581e7fc640 1 --2- 172.21.15.114:0/1927686308 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5569194c6cd0 0x5569194c90f0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33618038beccf37b server_cookie=ca1ea4729b058069 in_seq=0 out_seq=0 2024-09-13T18:23:41.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.145+0000 7f5802ff5640 10 client.16194 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:23:41.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.145+0000 7f5802ff5640 1 -- 172.21.15.114:0/1927686308 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x556919110e30 con 0x5569194c6cd0 2024-09-13T18:23:41.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.145+0000 7f5802ff5640 10 client.16194 handle_command_reply: tid=0 2024-09-13T18:23:41.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.145+0000 7f5828029740 1 client.16194 shutdown 2024-09-13T18:23:41.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.145+0000 7f5828029740 1 -- 172.21.15.114:0/1927686308 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556919454e10 msgr2=0x556919462f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:41.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.145+0000 7f5828029740 1 --2- 172.21.15.114:0/1927686308 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556919454e10 0x556919462f20 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f58100960a0 tx=0x7f581001d7f0 comp rx=0 tx=0).stop 2024-09-13T18:23:41.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.145+0000 7f5828029740 1 -- 172.21.15.114:0/1927686308 shutdown_connections 2024-09-13T18:23:41.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.145+0000 7f5828029740 1 --2- 172.21.15.114:0/1927686308 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5569194c6cd0 0x5569194c90f0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:41.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.145+0000 7f5828029740 1 --2- 172.21.15.114:0/1927686308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55691945ffd0 0x5569194c4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:41.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.145+0000 7f5828029740 1 --2- 172.21.15.114:0/1927686308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556919455730 0x556919463460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:41.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.145+0000 7f5802ff5640 0 client.16194 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:23:41.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.145+0000 7f5828029740 1 --2- 172.21.15.114:0/1927686308 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556919454e10 0x556919462f20 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:41.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.145+0000 7f5828029740 1 -- 172.21.15.114:0/1927686308 >> 172.21.15.114:0/1927686308 conn(0x556919451380 msgr2=0x556919457ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:41.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.145+0000 7f5828029740 1 -- 172.21.15.114:0/1927686308 shutdown_connections 2024-09-13T18:23:41.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.146+0000 7f5828029740 1 -- 172.21.15.114:0/1927686308 wait complete. 2024-09-13T18:23:41.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.146+0000 7f5828029740 20 client.16194 trim_cache size 0 max 16384 2024-09-13T18:23:41.157 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:23:41.157 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-13T18:23:41.158 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": { 2024-09-13T18:23:41.158 INFO:teuthology.orchestra.run.smithi114.stdout: "3fc5a7f0-d4bf-4907-9af5-af1128853e75": { 2024-09-13T18:23:41.158 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-13T18:23:41.158 INFO:teuthology.orchestra.run.smithi114.stdout: "tag": "3fc5a7f0-d4bf-4907-9af5-af1128853e75", 2024-09-13T18:23:41.158 INFO:teuthology.orchestra.run.smithi114.stdout: "options": "recursive,force" 2024-09-13T18:23:41.158 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:23:41.158 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:23:41.158 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:23:41.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.146+0000 7f5825308640 1 -- 172.21.15.114:0/2426787578 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f58080a1320 msgr2=0x7f58080a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:41.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.146+0000 7f5825308640 1 --2- 172.21.15.114:0/2426787578 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f58080a1320 0x7f58080a37e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5818006010 tx=0x7f5818049000 comp rx=0 tx=0).stop 2024-09-13T18:23:41.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.146+0000 7f5825308640 1 -- 172.21.15.114:0/2426787578 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58200fff50 msgr2=0x7f5820196280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:41.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.146+0000 7f5825308640 1 --2- 172.21.15.114:0/2426787578 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58200fff50 0x7f5820196280 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f581000c970 tx=0x7f581000ce30 comp rx=0 tx=0).stop 2024-09-13T18:23:41.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.146+0000 7f5825308640 1 -- 172.21.15.114:0/2426787578 shutdown_connections 2024-09-13T18:23:41.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.146+0000 7f5825308640 1 --2- 172.21.15.114:0/2426787578 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f58080a1320 0x7f58080a37e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:41.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.146+0000 7f5825308640 1 --2- 172.21.15.114:0/2426787578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f582019abb0 0x7f582019b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:41.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.146+0000 7f5825308640 1 --2- 172.21.15.114:0/2426787578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58201050a0 0x7f58201967c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:41.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.146+0000 7f5825308640 1 --2- 172.21.15.114:0/2426787578 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58200fff50 0x7f5820196280 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:41.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.146+0000 7f5825308640 1 -- 172.21.15.114:0/2426787578 >> 172.21.15.114:0/2426787578 conn(0x7f58200fb9e0 msgr2=0x7f58200fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:41.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.146+0000 7f5825308640 1 -- 172.21.15.114:0/2426787578 shutdown_connections 2024-09-13T18:23:41.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:23:41.146+0000 7f5825308640 1 -- 172.21.15.114:0/2426787578 wait complete. 2024-09-13T18:23:41.161 INFO:tasks.cephfs.filesystem:scrub status for tag:3fc5a7f0-d4bf-4907-9af5-af1128853e75 - {'path': '/', 'tag': '3fc5a7f0-d4bf-4907-9af5-af1128853e75', 'options': 'recursive,force'} 2024-09-13T18:23:41.193 INFO:tasks.workunit.client.0.smithi114.stdout:running iogen 2024-09-13T18:23:41.194 INFO:tasks.workunit.client.0.smithi114.stderr:child spawned 88635 2024-09-13T18:23:41.194 INFO:tasks.workunit.client.0.smithi114.stderr:child spawned 88636 2024-09-13T18:23:41.195 INFO:tasks.workunit.client.0.smithi114.stderr:child spawned 88637 2024-09-13T18:23:41.195 INFO:tasks.workunit.client.0.smithi114.stderr:child spawned 88638 2024-09-13T18:23:41.195 INFO:tasks.workunit.client.0.smithi114.stderr:child spawned 88639 2024-09-13T18:23:41.195 INFO:tasks.workunit.client.0.smithi114.stdout:sleep for 10 min 2024-09-13T18:23:41.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:23:41 smithi114 ceph-mon[30897]: scrub queued for path: / 2024-09-13T18:23:41.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:23:41 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:23:41.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:23:41 smithi114 ceph-mon[30897]: scrub summary: active paths [/] 2024-09-13T18:23:41.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:23:41 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:23:41.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:23:41 smithi114 ceph-mon[30897]: scrub summary: idle 2024-09-13T18:23:41.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:23:41 smithi204 ceph-mon[34777]: scrub queued for path: / 2024-09-13T18:23:41.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:23:41 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:23:41.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:23:41 smithi204 ceph-mon[34777]: scrub summary: active paths [/] 2024-09-13T18:23:41.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:23:41 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:23:41.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:23:41 smithi204 ceph-mon[34777]: scrub summary: idle 2024-09-13T18:23:41.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:23:41 smithi134 ceph-mon[34283]: scrub queued for path: / 2024-09-13T18:23:41.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:23:41 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:23:41.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:23:41 smithi134 ceph-mon[34283]: scrub summary: active paths [/] 2024-09-13T18:23:41.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:23:41 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:23:41.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:23:41 smithi134 ceph-mon[34283]: scrub summary: idle 2024-09-13T18:23:42.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:23:42 smithi204 ceph-mon[34777]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-13T18:23:42.719 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:23:42 smithi114 ceph-mon[30897]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-13T18:23:42.868 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:23:42 smithi134 ceph-mon[34283]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-13T18:23:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.749+0000 7f20d3287640 1 -- 172.21.15.114:0/4244458414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20cc1062a0 msgr2=0x7f20cc10cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.749+0000 7f20d3287640 1 --2- 172.21.15.114:0/4244458414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20cc1062a0 0x7f20cc10cc20 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f20c4009cd0 tx=0x7f20c402a8e0 comp rx=0 tx=0).stop 2024-09-13T18:23:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.749+0000 7f20d3287640 1 -- 172.21.15.114:0/4244458414 shutdown_connections 2024-09-13T18:23:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.749+0000 7f20d3287640 1 --2- 172.21.15.114:0/4244458414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20cc1062a0 0x7f20cc10cc20 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.749+0000 7f20d3287640 1 --2- 172.21.15.114:0/4244458414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20cc0ff3c0 0x7f20cc105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.749+0000 7f20d3287640 1 --2- 172.21.15.114:0/4244458414 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20cc0fea10 0x7f20cc0fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.749+0000 7f20d3287640 1 -- 172.21.15.114:0/4244458414 >> 172.21.15.114:0/4244458414 conn(0x7f20cc075380 msgr2=0x7f20cc075790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.749+0000 7f20d3287640 1 -- 172.21.15.114:0/4244458414 shutdown_connections 2024-09-13T18:23:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.749+0000 7f20d3287640 1 -- 172.21.15.114:0/4244458414 wait complete. 2024-09-13T18:23:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.750+0000 7f20d3287640 1 Processor -- start 2024-09-13T18:23:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.750+0000 7f20d3287640 1 -- start start 2024-09-13T18:23:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.750+0000 7f20d3287640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20cc0fea10 0x7f20cc196130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.750+0000 7f20d3287640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20cc1062a0 0x7f20cc196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.750+0000 7f20d3287640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20cc19aa10 0x7f20cc19aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.750+0000 7f20d3287640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20cc19b400 con 0x7f20cc19aa10 2024-09-13T18:23:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.750+0000 7f20d3287640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20cc19b570 con 0x7f20cc0fea10 2024-09-13T18:23:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.750+0000 7f20d3287640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20cc19b6e0 con 0x7f20cc1062a0 2024-09-13T18:23:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.751+0000 7f20d1023640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20cc0fea10 0x7f20cc196130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.751+0000 7f20d1023640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20cc0fea10 0x7f20cc196130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:38920/0 (socket says 172.21.15.114:38920) 2024-09-13T18:23:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.751+0000 7f20d1023640 1 -- 172.21.15.114:0/2549221510 learned_addr learned my addr 172.21.15.114:0/2549221510 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.751+0000 7f20d1824640 1 --2- 172.21.15.114:0/2549221510 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20cc19aa10 0x7f20cc19aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.751+0000 7f20d0822640 1 --2- 172.21.15.114:0/2549221510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20cc1062a0 0x7f20cc196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.751+0000 7f20d1824640 1 -- 172.21.15.114:0/2549221510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20cc1062a0 msgr2=0x7f20cc196670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.751+0000 7f20d1824640 1 --2- 172.21.15.114:0/2549221510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20cc1062a0 0x7f20cc196670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.751+0000 7f20d1824640 1 -- 172.21.15.114:0/2549221510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20cc0fea10 msgr2=0x7f20cc196130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.751+0000 7f20d1824640 1 --2- 172.21.15.114:0/2549221510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20cc0fea10 0x7f20cc196130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.751+0000 7f20d1824640 1 -- 172.21.15.114:0/2549221510 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c4009900 con 0x7f20cc19aa10 2024-09-13T18:23:43.756 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.751+0000 7f20d1824640 1 --2- 172.21.15.114:0/2549221510 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20cc19aa10 0x7f20cc19aec0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f20c4009ca0 tx=0x7f20c4030dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:43.756 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.751+0000 7f20c27fc640 1 -- 172.21.15.114:0/2549221510 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c4031b80 con 0x7f20cc19aa10 2024-09-13T18:23:43.756 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.752+0000 7f20c27fc640 1 -- 172.21.15.114:0/2549221510 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f20c4003e00 con 0x7f20cc19aa10 2024-09-13T18:23:43.756 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.752+0000 7f20d3287640 1 -- 172.21.15.114:0/2549221510 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20cc1a2cc0 con 0x7f20cc19aa10 2024-09-13T18:23:43.756 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.752+0000 7f20c27fc640 1 -- 172.21.15.114:0/2549221510 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c4004740 con 0x7f20cc19aa10 2024-09-13T18:23:43.756 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.752+0000 7f20d3287640 1 -- 172.21.15.114:0/2549221510 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20cc1a3140 con 0x7f20cc19aa10 2024-09-13T18:23:43.757 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.755+0000 7f20c27fc640 1 -- 172.21.15.114:0/2549221510 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) v1 ==== 136107+0+0 (secure 0 0 0) 0x7f20c400a040 con 0x7f20cc19aa10 2024-09-13T18:23:43.757 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.755+0000 7f20d6083740 1 Processor -- start 2024-09-13T18:23:43.757 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.755+0000 7f20d6083740 1 -- start start 2024-09-13T18:23:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.755+0000 7f20d6083740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b06ebd2330 0x55b06ebd2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.755+0000 7f20d6083740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b06ebd2c50 0x55b06ebdcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.755+0000 7f20d6083740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b06ebdd4f0 0x55b06ebdf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.755+0000 7f20d6083740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b06eb3dc30 con 0x55b06ebdd4f0 2024-09-13T18:23:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.755+0000 7f20d6083740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b06eb3b3c0 con 0x55b06ebd2c50 2024-09-13T18:23:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.755+0000 7f20d6083740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b06eb36cb0 con 0x55b06ebd2330 2024-09-13T18:23:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.755+0000 7f20d0822640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b06ebdd4f0 0x55b06ebdf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.755+0000 7f20d0822640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b06ebdd4f0 0x55b06ebdf8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:53676/0 (socket says 172.21.15.114:53676) 2024-09-13T18:23:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.755+0000 7f20d0822640 1 -- 172.21.15.114:0/3892568567 learned_addr learned my addr 172.21.15.114:0/3892568567 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.755+0000 7f20d1023640 1 --2- 172.21.15.114:0/3892568567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b06ebd2c50 0x55b06ebdcfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.756+0000 7f20d0822640 1 --2- 172.21.15.114:0/3892568567 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b06ebd2330 0x55b06ebd2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.756+0000 7f20c27fc640 1 --2- 172.21.15.114:0/2549221510 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f20b80a13a0 0x7f20b80a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.756+0000 7f20d0822640 1 -- 172.21.15.114:0/3892568567 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b06ebd2330 msgr2=0x55b06ebd2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.756+0000 7f20d0822640 1 --2- 172.21.15.114:0/3892568567 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b06ebd2330 0x55b06ebd2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.756+0000 7f20d0822640 1 -- 172.21.15.114:0/3892568567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b06ebd2c50 msgr2=0x55b06ebdcfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.756+0000 7f20d0822640 1 --2- 172.21.15.114:0/3892568567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b06ebd2c50 0x55b06ebdcfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.756+0000 7f20d1824640 1 --2- 172.21.15.114:0/2549221510 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f20b80a13a0 0x7f20b80a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.756+0000 7f20d0822640 1 -- 172.21.15.114:0/3892568567 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b06eadd910 con 0x55b06ebdd4f0 2024-09-13T18:23:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.756+0000 7f20d1023640 1 --2- 172.21.15.114:0/3892568567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b06ebd2c50 0x55b06ebdcfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:23:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.756+0000 7f20c27fc640 1 -- 172.21.15.114:0/2549221510 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f20c40ee0b0 con 0x7f20cc19aa10 2024-09-13T18:23:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.756+0000 7f20d0822640 1 --2- 172.21.15.114:0/3892568567 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b06ebdd4f0 0x55b06ebdf8e0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f20bc011700 tx=0x7f20bc00a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a5f22a92e3ed954 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.756+0000 7f20b77fe640 1 -- 172.21.15.114:0/3892568567 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20bc011e90 con 0x55b06ebdd4f0 2024-09-13T18:23:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.756+0000 7f20b77fe640 1 -- 172.21.15.114:0/3892568567 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f20bc0054b0 con 0x55b06ebdd4f0 2024-09-13T18:23:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.756+0000 7f20b77fe640 1 -- 172.21.15.114:0/3892568567 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20bc00f7c0 con 0x55b06ebdd4f0 2024-09-13T18:23:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.757+0000 7f20d6083740 1 -- 172.21.15.114:0/3892568567 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b06ebdd4f0 msgr2=0x55b06ebdf8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.757+0000 7f20d6083740 1 --2- 172.21.15.114:0/3892568567 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b06ebdd4f0 0x55b06ebdf8e0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f20bc011700 tx=0x7f20bc00a680 comp rx=0 tx=0).stop 2024-09-13T18:23:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.757+0000 7f20d1824640 1 --2- 172.21.15.114:0/2549221510 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f20b80a13a0 0x7f20b80a3860 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f20c40eb390 tx=0x7f20c4086040 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.757+0000 7f20d6083740 1 -- 172.21.15.114:0/3892568567 shutdown_connections 2024-09-13T18:23:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.757+0000 7f20d6083740 1 --2- 172.21.15.114:0/3892568567 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b06ebdd4f0 0x55b06ebdf8e0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.757+0000 7f20d6083740 1 --2- 172.21.15.114:0/3892568567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b06ebd2c50 0x55b06ebdcfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.757+0000 7f20d6083740 1 --2- 172.21.15.114:0/3892568567 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b06ebd2330 0x55b06ebd2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.757+0000 7f20d6083740 1 -- 172.21.15.114:0/3892568567 >> 172.21.15.114:0/3892568567 conn(0x55b06ebce8c0 msgr2=0x55b06ebceca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.757+0000 7f20d6083740 1 -- 172.21.15.114:0/3892568567 shutdown_connections 2024-09-13T18:23:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.757+0000 7f20d6083740 1 -- 172.21.15.114:0/3892568567 wait complete. 2024-09-13T18:23:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.758+0000 7f20d6083740 1 -- start start 2024-09-13T18:23:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.758+0000 7f20d6083740 1 Processor -- start 2024-09-13T18:23:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.758+0000 7f20d6083740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b06ebd2330 0x55b06ebdcf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.758+0000 7f20d6083740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b06ebd2c50 0x55b06ebda200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.758+0000 7f20d6083740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b06ebdd4f0 0x55b06ebda810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.758+0000 7f20d6083740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b06eaf4e80 con 0x55b06ebd2330 2024-09-13T18:23:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.758+0000 7f20d6083740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b06eaf4b20 con 0x55b06ebdd4f0 2024-09-13T18:23:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.758+0000 7f20d6083740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b06eaba260 con 0x55b06ebd2c50 2024-09-13T18:23:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.758+0000 7f20d0822640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b06ebd2330 0x55b06ebdcf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.758+0000 7f20d6083740 10 client.0 _ll_register_callbacks cb 0x55b06eaefdf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:23:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.758+0000 7f20d0822640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b06ebd2330 0x55b06ebdcf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:53688/0 (socket says 172.21.15.114:53688) 2024-09-13T18:23:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.758+0000 7f20d0822640 1 -- 172.21.15.114:0/454992724 learned_addr learned my addr 172.21.15.114:0/454992724 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.759+0000 7f20d1023640 1 --2- 172.21.15.114:0/454992724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b06ebd2c50 0x55b06ebda200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.759+0000 7f20d0822640 1 -- 172.21.15.114:0/454992724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b06ebd2c50 msgr2=0x55b06ebda200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.759+0000 7f20d0822640 1 --2- 172.21.15.114:0/454992724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b06ebd2c50 0x55b06ebda200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.759+0000 7f20d0822640 1 -- 172.21.15.114:0/454992724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b06ebdd4f0 msgr2=0x55b06ebda810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.759+0000 7f20d0822640 1 --2- 172.21.15.114:0/454992724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b06ebdd4f0 0x55b06ebda810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.759+0000 7f20d0822640 1 -- 172.21.15.114:0/454992724 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20bc019040 con 0x55b06ebd2330 2024-09-13T18:23:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.759+0000 7f20d1023640 1 --2- 172.21.15.114:0/454992724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b06ebd2c50 0x55b06ebda200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:23:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.759+0000 7f20d0822640 1 --2- 172.21.15.114:0/454992724 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b06ebd2330 0x55b06ebdcf10 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f20bc004fb0 tx=0x7f20bc0114d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a3134505ffeff59 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.759+0000 7f20b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:23:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.759+0000 7f20b4ff9640 1 -- 172.21.15.114:0/454992724 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20bc00f7c0 con 0x55b06ebd2330 2024-09-13T18:23:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.759+0000 7f20d6083740 1 -- 172.21.15.114:0/454992724 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b06eb5f680 con 0x55b06ebd2330 2024-09-13T18:23:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.759+0000 7f20b4ff9640 1 -- 172.21.15.114:0/454992724 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f20bc000e40 con 0x55b06ebd2330 2024-09-13T18:23:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.759+0000 7f20b4ff9640 1 -- 172.21.15.114:0/454992724 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20bc00f920 con 0x55b06ebd2330 2024-09-13T18:23:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.760+0000 7f20d6083740 10 client.16206 fetch_fsmap learned FSMap version 30 2024-09-13T18:23:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.760+0000 7f20d6083740 1 -- 172.21.15.114:0/454992724 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55b06eb35330 con 0x55b06ebd2330 2024-09-13T18:23:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.760+0000 7f20b4ff9640 1 -- 172.21.15.114:0/454992724 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b06eb5f680 con 0x55b06ebd2330 2024-09-13T18:23:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.760+0000 7f20b4ff9640 1 -- 172.21.15.114:0/454992724 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f20bc024710 con 0x55b06ebd2330 2024-09-13T18:23:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.760+0000 7f20d6083740 10 client.16206 fetch_fsmap finished waiting for FSMap version 30 2024-09-13T18:23:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.760+0000 7f20d6083740 10 client.16206 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:23:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.760+0000 7f20d6083740 20 client.16206 populate_metadata read hostname 'smithi114' 2024-09-13T18:23:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.760+0000 7f20d6083740 1 --2- 172.21.15.114:0/454992724 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55b06ebece20 0x55b06ebef240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.760+0000 7f20d6083740 4 client.16206 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:23:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.760+0000 7f20d6083740 1 -- 172.21.15.114:0/454992724 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b06eb2a5d0 con 0x55b06ebece20 2024-09-13T18:23:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.761+0000 7f20d1023640 1 --2- 172.21.15.114:0/454992724 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55b06ebece20 0x55b06ebef240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.761+0000 7f20b4ff9640 10 client.16206 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:23:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.761+0000 7f20d1023640 1 --2- 172.21.15.114:0/454992724 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55b06ebece20 0x55b06ebef240 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4989cd74c5f95334 server_cookie=249f37640f5ed7c5 in_seq=0 out_seq=0 2024-09-13T18:23:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.762+0000 7f20b4ff9640 1 -- 172.21.15.114:0/454992724 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55b06eb2a5d0 con 0x55b06ebece20 2024-09-13T18:23:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.762+0000 7f20b4ff9640 10 client.16206 handle_command_reply: tid=0 2024-09-13T18:23:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.762+0000 7f20d6083740 1 client.16206 shutdown 2024-09-13T18:23:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.762+0000 7f20d6083740 1 -- 172.21.15.114:0/454992724 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b06ebd2330 msgr2=0x55b06ebdcf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.762+0000 7f20d6083740 1 --2- 172.21.15.114:0/454992724 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b06ebd2330 0x55b06ebdcf10 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f20bc004fb0 tx=0x7f20bc0114d0 comp rx=0 tx=0).stop 2024-09-13T18:23:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.762+0000 7f20d6083740 1 -- 172.21.15.114:0/454992724 shutdown_connections 2024-09-13T18:23:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.762+0000 7f20d6083740 1 --2- 172.21.15.114:0/454992724 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55b06ebece20 0x55b06ebef240 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.762+0000 7f20d6083740 1 --2- 172.21.15.114:0/454992724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b06ebdd4f0 0x55b06ebda810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.762+0000 7f20d6083740 1 --2- 172.21.15.114:0/454992724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b06ebd2c50 0x55b06ebda200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.762+0000 7f20d6083740 1 --2- 172.21.15.114:0/454992724 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b06ebd2330 0x55b06ebdcf10 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.762+0000 7f20b4ff9640 0 client.16206 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:23:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.762+0000 7f20d6083740 1 -- 172.21.15.114:0/454992724 >> 172.21.15.114:0/454992724 conn(0x55b06ebce8c0 msgr2=0x55b06ebcdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.762+0000 7f20d6083740 1 -- 172.21.15.114:0/454992724 shutdown_connections 2024-09-13T18:23:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.762+0000 7f20d6083740 1 -- 172.21.15.114:0/454992724 wait complete. 2024-09-13T18:23:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.763+0000 7f20d6083740 20 client.16206 trim_cache size 0 max 16384 2024-09-13T18:23:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.783+0000 7f20d6083740 1 Processor -- start 2024-09-13T18:23:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.783+0000 7f20d6083740 1 -- start start 2024-09-13T18:23:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.783+0000 7f20d6083740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b06ebd2330 0x55b06ebdca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.783+0000 7f20d6083740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b06ebd2c50 0x55b06ebd8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.783+0000 7f20d6083740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b06ebdd4f0 0x55b06ebd8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.783+0000 7f20d6083740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b06ebdffa0 con 0x55b06ebd2330 2024-09-13T18:23:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.783+0000 7f20d6083740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b06ebe9c50 con 0x55b06ebd2c50 2024-09-13T18:23:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.783+0000 7f20d6083740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b06ebda000 con 0x55b06ebdd4f0 2024-09-13T18:23:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.783+0000 7f20d0822640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b06ebd2330 0x55b06ebdca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.783+0000 7f20d0822640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b06ebd2330 0x55b06ebdca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:53702/0 (socket says 172.21.15.114:53702) 2024-09-13T18:23:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.783+0000 7f20d0822640 1 -- 172.21.15.114:0/602978831 learned_addr learned my addr 172.21.15.114:0/602978831 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.784+0000 7f20d1023640 1 --2- 172.21.15.114:0/602978831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b06ebd2c50 0x55b06ebd8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.784+0000 7f20d0822640 1 -- 172.21.15.114:0/602978831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b06ebdd4f0 msgr2=0x55b06ebd8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.784+0000 7f20d0822640 1 --2- 172.21.15.114:0/602978831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b06ebdd4f0 0x55b06ebd8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.784+0000 7f20d0822640 1 -- 172.21.15.114:0/602978831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b06ebd2c50 msgr2=0x55b06ebd8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.784+0000 7f20d0822640 1 --2- 172.21.15.114:0/602978831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b06ebd2c50 0x55b06ebd8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.784+0000 7f20d0822640 1 -- 172.21.15.114:0/602978831 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b06ead7300 con 0x55b06ebd2330 2024-09-13T18:23:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.784+0000 7f20d1023640 1 --2- 172.21.15.114:0/602978831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b06ebd2c50 0x55b06ebd8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:23:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.784+0000 7f20d0822640 1 --2- 172.21.15.114:0/602978831 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b06ebd2330 0x55b06ebdca20 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f20bc011540 tx=0x7f20bc005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6234d4d3ec2f6c9d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.784+0000 7f20b7fff640 1 -- 172.21.15.114:0/602978831 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20bc024d10 con 0x55b06ebd2330 2024-09-13T18:23:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.784+0000 7f20b7fff640 1 -- 172.21.15.114:0/602978831 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f20bc015040 con 0x55b06ebd2330 2024-09-13T18:23:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.784+0000 7f20b7fff640 1 -- 172.21.15.114:0/602978831 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20bc0048d0 con 0x55b06ebd2330 2024-09-13T18:23:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.785+0000 7f20d6083740 1 -- 172.21.15.114:0/602978831 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b06ebd2330 msgr2=0x55b06ebdca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.785+0000 7f20d6083740 1 --2- 172.21.15.114:0/602978831 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b06ebd2330 0x55b06ebdca20 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f20bc011540 tx=0x7f20bc005e40 comp rx=0 tx=0).stop 2024-09-13T18:23:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.785+0000 7f20d6083740 1 -- 172.21.15.114:0/602978831 shutdown_connections 2024-09-13T18:23:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.785+0000 7f20d6083740 1 --2- 172.21.15.114:0/602978831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b06ebdd4f0 0x55b06ebd8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.785+0000 7f20d6083740 1 --2- 172.21.15.114:0/602978831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b06ebd2c50 0x55b06ebd8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.785+0000 7f20d6083740 1 --2- 172.21.15.114:0/602978831 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b06ebd2330 0x55b06ebdca20 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.785+0000 7f20d6083740 1 -- 172.21.15.114:0/602978831 >> 172.21.15.114:0/602978831 conn(0x55b06ebce8c0 msgr2=0x55b06ebe8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.786+0000 7f20d6083740 1 -- 172.21.15.114:0/602978831 shutdown_connections 2024-09-13T18:23:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.786+0000 7f20d6083740 1 -- 172.21.15.114:0/602978831 wait complete. 2024-09-13T18:23:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.786+0000 7f20d6083740 1 -- start start 2024-09-13T18:23:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.786+0000 7f20d6083740 1 Processor -- start 2024-09-13T18:23:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.787+0000 7f20d6083740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b06ebd2330 0x55b06ebe4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.787+0000 7f20d6083740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b06ebd2c50 0x55b06ebe07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.787+0000 7f20d6083740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b06ebdd4f0 0x55b06ebe0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.787+0000 7f20d6083740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b06ebe1220 con 0x55b06ebd2330 2024-09-13T18:23:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.787+0000 7f20d6083740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b06ebe1390 con 0x55b06ebd2c50 2024-09-13T18:23:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.787+0000 7f20d6083740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b06ebe1550 con 0x55b06ebdd4f0 2024-09-13T18:23:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.787+0000 7f20d6083740 10 client.0 _ll_register_callbacks cb 0x7f20bc00a9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:23:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.787+0000 7f20d0822640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b06ebd2330 0x55b06ebe4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:43.792 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.787+0000 7f20d1023640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b06ebd2c50 0x55b06ebe07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:43.792 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.787+0000 7f20d1023640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b06ebd2c50 0x55b06ebe07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:38970/0 (socket says 172.21.15.114:38970) 2024-09-13T18:23:43.792 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.787+0000 7f20d1023640 1 -- 172.21.15.114:0/2831427701 learned_addr learned my addr 172.21.15.114:0/2831427701 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:43.792 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.787+0000 7f20d0822640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b06ebd2330 0x55b06ebe4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:53710/0 (socket says 172.21.15.114:53710) 2024-09-13T18:23:43.792 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.788+0000 7f20d0822640 1 --2- 172.21.15.114:0/2831427701 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b06ebdd4f0 0x55b06ebe0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:43.792 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.788+0000 7f20d1023640 1 -- 172.21.15.114:0/2831427701 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b06ebdd4f0 msgr2=0x55b06ebe0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:43.792 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.788+0000 7f20d1023640 1 --2- 172.21.15.114:0/2831427701 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b06ebdd4f0 0x55b06ebe0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:43.792 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.788+0000 7f20d1023640 1 -- 172.21.15.114:0/2831427701 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b06ebd2330 msgr2=0x55b06ebe4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:43.792 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.788+0000 7f20d1023640 1 --2- 172.21.15.114:0/2831427701 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b06ebd2330 0x55b06ebe4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:43.792 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.788+0000 7f20d1023640 1 -- 172.21.15.114:0/2831427701 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b06eb35330 con 0x55b06ebd2c50 2024-09-13T18:23:43.792 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.788+0000 7f20d0822640 1 --2- 172.21.15.114:0/2831427701 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b06ebdd4f0 0x55b06ebe0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:23:43.792 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.788+0000 7f20d1023640 1 --2- 172.21.15.114:0/2831427701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b06ebd2c50 0x55b06ebe07a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f20c800b8f0 tx=0x7f20c800bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=638082aa06047173 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:43.792 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.788+0000 7f20b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:23:43.792 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.788+0000 7f20b4ff9640 1 -- 172.21.15.114:0/2831427701 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c801e600 con 0x55b06ebd2c50 2024-09-13T18:23:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.788+0000 7f20b4ff9640 1 -- 172.21.15.114:0/2831427701 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f20c8011e20 con 0x55b06ebd2c50 2024-09-13T18:23:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.788+0000 7f20d6083740 1 -- 172.21.15.114:0/2831427701 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f20bc0054b0 con 0x55b06ebd2c50 2024-09-13T18:23:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.789+0000 7f20b4ff9640 1 -- 172.21.15.114:0/2831427701 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20c801e760 con 0x55b06ebd2c50 2024-09-13T18:23:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.789+0000 7f20d6083740 10 client.25556 fetch_fsmap learned FSMap version 30 2024-09-13T18:23:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.789+0000 7f20d6083740 1 -- 172.21.15.114:0/2831427701 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55b06eafc8e0 con 0x55b06ebd2c50 2024-09-13T18:23:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.789+0000 7f20b4ff9640 1 -- 172.21.15.114:0/2831427701 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f20bc0054b0 con 0x55b06ebd2c50 2024-09-13T18:23:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.789+0000 7f20b4ff9640 1 -- 172.21.15.114:0/2831427701 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f20c8010430 con 0x55b06ebd2c50 2024-09-13T18:23:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.789+0000 7f20d6083740 10 client.25556 fetch_fsmap finished waiting for FSMap version 30 2024-09-13T18:23:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.789+0000 7f20d6083740 10 client.25556 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:23:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.789+0000 7f20d6083740 20 client.25556 populate_metadata read hostname 'smithi114' 2024-09-13T18:23:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.789+0000 7f20d6083740 1 --2- 172.21.15.114:0/2831427701 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55b06ec4cfd0 0x55b06ec4f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.789+0000 7f20d6083740 4 client.25556 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T18:23:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.789+0000 7f20d6083740 1 -- 172.21.15.114:0/2831427701 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b06e82a300 con 0x55b06ec4cfd0 2024-09-13T18:23:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.789+0000 7f20d0822640 1 --2- 172.21.15.114:0/2831427701 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55b06ec4cfd0 0x55b06ec4f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.790+0000 7f20b4ff9640 10 client.25556 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:23:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.790+0000 7f20d0822640 1 --2- 172.21.15.114:0/2831427701 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55b06ec4cfd0 0x55b06ec4f3f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f109540cf2b8bc23 server_cookie=d90cc73747a66bac in_seq=0 out_seq=0 2024-09-13T18:23:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.807+0000 7f20b4ff9640 1 -- 172.21.15.114:0/2831427701 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b06e82a300 con 0x55b06ec4cfd0 2024-09-13T18:23:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.807+0000 7f20b4ff9640 10 client.25556 handle_command_reply: tid=0 2024-09-13T18:23:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.807+0000 7f20d6083740 1 client.25556 shutdown 2024-09-13T18:23:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.808+0000 7f20d6083740 1 -- 172.21.15.114:0/2831427701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b06ebd2c50 msgr2=0x55b06ebe07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.808+0000 7f20d6083740 1 --2- 172.21.15.114:0/2831427701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b06ebd2c50 0x55b06ebe07a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f20c800b8f0 tx=0x7f20c800bf60 comp rx=0 tx=0).stop 2024-09-13T18:23:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.808+0000 7f20d6083740 1 -- 172.21.15.114:0/2831427701 shutdown_connections 2024-09-13T18:23:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.808+0000 7f20d6083740 1 --2- 172.21.15.114:0/2831427701 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55b06ec4cfd0 0x55b06ec4f3f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.808+0000 7f20d6083740 1 --2- 172.21.15.114:0/2831427701 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b06ebdd4f0 0x55b06ebe0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.808+0000 7f20d6083740 1 --2- 172.21.15.114:0/2831427701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b06ebd2c50 0x55b06ebe07a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.808+0000 7f20d6083740 1 --2- 172.21.15.114:0/2831427701 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b06ebd2330 0x55b06ebe4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.808+0000 7f20d6083740 1 -- 172.21.15.114:0/2831427701 >> 172.21.15.114:0/2831427701 conn(0x55b06ebce8c0 msgr2=0x55b06ebc9a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.808+0000 7f20b4ff9640 0 client.25556 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:23:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.808+0000 7f20d6083740 1 -- 172.21.15.114:0/2831427701 shutdown_connections 2024-09-13T18:23:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.808+0000 7f20d6083740 1 -- 172.21.15.114:0/2831427701 wait complete. 2024-09-13T18:23:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.808+0000 7f20d6083740 20 client.25556 trim_cache size 0 max 16384 2024-09-13T18:23:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-13T18:23:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-13T18:23:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-13T18:23:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-13T18:23:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.809+0000 7f20d3287640 1 -- 172.21.15.114:0/2549221510 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f20b80a13a0 msgr2=0x7f20b80a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.809+0000 7f20d3287640 1 --2- 172.21.15.114:0/2549221510 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f20b80a13a0 0x7f20b80a3860 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f20c40eb390 tx=0x7f20c4086040 comp rx=0 tx=0).stop 2024-09-13T18:23:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.809+0000 7f20d3287640 1 -- 172.21.15.114:0/2549221510 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20cc19aa10 msgr2=0x7f20cc19aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.809+0000 7f20d3287640 1 --2- 172.21.15.114:0/2549221510 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20cc19aa10 0x7f20cc19aec0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f20c4009ca0 tx=0x7f20c4030dd0 comp rx=0 tx=0).stop 2024-09-13T18:23:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.809+0000 7f20d3287640 1 -- 172.21.15.114:0/2549221510 shutdown_connections 2024-09-13T18:23:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.809+0000 7f20d3287640 1 --2- 172.21.15.114:0/2549221510 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f20b80a13a0 0x7f20b80a3860 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.809+0000 7f20d3287640 1 --2- 172.21.15.114:0/2549221510 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20cc19aa10 0x7f20cc19aec0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.809+0000 7f20d3287640 1 --2- 172.21.15.114:0/2549221510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20cc1062a0 0x7f20cc196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.809+0000 7f20d3287640 1 --2- 172.21.15.114:0/2549221510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20cc0fea10 0x7f20cc196130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.809+0000 7f20d3287640 1 -- 172.21.15.114:0/2549221510 >> 172.21.15.114:0/2549221510 conn(0x7f20cc075380 msgr2=0x7f20cc103dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.809+0000 7f20d3287640 1 -- 172.21.15.114:0/2549221510 shutdown_connections 2024-09-13T18:23:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:43.809+0000 7f20d3287640 1 -- 172.21.15.114:0/2549221510 wait complete. 2024-09-13T18:23:44.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:23:44 smithi204 ceph-mon[34777]: pgmap v41: 129 pgs: 129 active+clean; 398 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 33 MiB/s wr, 36 op/s 2024-09-13T18:23:44.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:23:44 smithi114 ceph-mon[30897]: pgmap v41: 129 pgs: 129 active+clean; 398 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 33 MiB/s wr, 36 op/s 2024-09-13T18:23:44.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:23:44 smithi134 ceph-mon[34283]: pgmap v41: 129 pgs: 129 active+clean; 398 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 33 MiB/s wr, 36 op/s 2024-09-13T18:23:46.436 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:23:46 smithi134 ceph-mon[34283]: pgmap v42: 129 pgs: 129 active+clean; 498 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 41 MiB/s wr, 46 op/s 2024-09-13T18:23:46.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:23:46 smithi204 ceph-mon[34777]: pgmap v42: 129 pgs: 129 active+clean; 498 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 41 MiB/s wr, 46 op/s 2024-09-13T18:23:46.469 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:23:46 smithi114 ceph-mon[30897]: pgmap v42: 129 pgs: 129 active+clean; 498 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 41 MiB/s wr, 46 op/s 2024-09-13T18:23:48.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:23:48 smithi204 ceph-mon[34777]: pgmap v43: 129 pgs: 129 active+clean; 498 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 41 MiB/s wr, 43 op/s 2024-09-13T18:23:48.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:23:48 smithi114 ceph-mon[30897]: pgmap v43: 129 pgs: 129 active+clean; 498 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 41 MiB/s wr, 43 op/s 2024-09-13T18:23:48.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:23:48 smithi134 ceph-mon[34283]: pgmap v43: 129 pgs: 129 active+clean; 498 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 41 MiB/s wr, 43 op/s 2024-09-13T18:23:50.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:23:50 smithi204 ceph-mon[34777]: pgmap v44: 129 pgs: 129 active+clean; 1.4 GiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 117 MiB/s wr, 87 op/s 2024-09-13T18:23:50.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:23:50 smithi114 ceph-mon[30897]: pgmap v44: 129 pgs: 129 active+clean; 1.4 GiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 117 MiB/s wr, 87 op/s 2024-09-13T18:23:50.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:23:50 smithi134 ceph-mon[34283]: pgmap v44: 129 pgs: 129 active+clean; 1.4 GiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 117 MiB/s wr, 87 op/s 2024-09-13T18:23:52.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:23:52 smithi204 ceph-mon[34777]: pgmap v45: 129 pgs: 129 active+clean; 1.4 GiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 117 MiB/s wr, 83 op/s 2024-09-13T18:23:52.469 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:23:52 smithi114 ceph-mon[30897]: pgmap v45: 129 pgs: 129 active+clean; 1.4 GiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 117 MiB/s wr, 83 op/s 2024-09-13T18:23:52.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:23:52 smithi134 ceph-mon[34283]: pgmap v45: 129 pgs: 129 active+clean; 1.4 GiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 117 MiB/s wr, 83 op/s 2024-09-13T18:23:54.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:23:54 smithi204 ceph-mon[34777]: pgmap v46: 129 pgs: 129 active+clean; 2.3 GiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 194 MiB/s wr, 127 op/s 2024-09-13T18:23:54.469 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:23:54 smithi114 ceph-mon[30897]: pgmap v46: 129 pgs: 129 active+clean; 2.3 GiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 194 MiB/s wr, 127 op/s 2024-09-13T18:23:54.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:23:54 smithi134 ceph-mon[34283]: pgmap v46: 129 pgs: 129 active+clean; 2.3 GiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 194 MiB/s wr, 127 op/s 2024-09-13T18:23:56.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:23:56 smithi204 ceph-mon[34777]: pgmap v47: 129 pgs: 129 active+clean; 2.3 GiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 161 MiB/s wr, 99 op/s 2024-09-13T18:23:56.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:23:56 smithi114 ceph-mon[30897]: pgmap v47: 129 pgs: 129 active+clean; 2.3 GiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 161 MiB/s wr, 99 op/s 2024-09-13T18:23:56.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:23:56 smithi134 ceph-mon[34283]: pgmap v47: 129 pgs: 129 active+clean; 2.3 GiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 161 MiB/s wr, 99 op/s 2024-09-13T18:23:56.945 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.942+0000 7feebb220640 1 -- 172.21.15.114:0/189645465 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feeb40ff210 msgr2=0x7feeb40ff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:56.945 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.942+0000 7feebb220640 1 --2- 172.21.15.114:0/189645465 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feeb40ff210 0x7feeb40ff5f0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7feea800d940 tx=0x7feea802c3b0 comp rx=0 tx=0).stop 2024-09-13T18:23:56.945 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.942+0000 7feebb220640 1 -- 172.21.15.114:0/189645465 shutdown_connections 2024-09-13T18:23:56.945 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.942+0000 7feebb220640 1 --2- 172.21.15.114:0/189645465 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feeb4103f20 0x7feeb410ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:56.945 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.942+0000 7feebb220640 1 --2- 172.21.15.114:0/189645465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feeb40ffbc0 0x7feeb41037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:56.945 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.942+0000 7feebb220640 1 --2- 172.21.15.114:0/189645465 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feeb40ff210 0x7feeb40ff5f0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:56.945 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.942+0000 7feebb220640 1 -- 172.21.15.114:0/189645465 >> 172.21.15.114:0/189645465 conn(0x7feeb4074930 msgr2=0x7feeb4074d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:56.946 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.942+0000 7feebb220640 1 -- 172.21.15.114:0/189645465 shutdown_connections 2024-09-13T18:23:56.946 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.942+0000 7feebb220640 1 -- 172.21.15.114:0/189645465 wait complete. 2024-09-13T18:23:56.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.943+0000 7feebb220640 1 Processor -- start 2024-09-13T18:23:56.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.943+0000 7feebb220640 1 -- start start 2024-09-13T18:23:56.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.944+0000 7feebb220640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feeb40ff210 0x7feeb419c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:56.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.944+0000 7feebb220640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feeb4103f20 0x7feeb419cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:56.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.944+0000 7feebb220640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feeb41a0000 0x7feeb41a04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:56.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.944+0000 7feebb220640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb41a09f0 con 0x7feeb40ff210 2024-09-13T18:23:56.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.944+0000 7feebb220640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb41a0b60 con 0x7feeb41a0000 2024-09-13T18:23:56.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.944+0000 7feebb220640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb41a0d20 con 0x7feeb4103f20 2024-09-13T18:23:56.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.944+0000 7feeb8fbc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feeb40ff210 0x7feeb419c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:56.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.944+0000 7feeb3fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feeb4103f20 0x7feeb419cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:56.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.944+0000 7feeb3fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feeb4103f20 0x7feeb419cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:32846/0 (socket says 172.21.15.114:32846) 2024-09-13T18:23:56.949 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.944+0000 7feeb97bd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feeb41a0000 0x7feeb41a04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:56.949 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.944+0000 7feeb3fff640 1 -- 172.21.15.114:0/1491086419 learned_addr learned my addr 172.21.15.114:0/1491086419 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:56.949 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.944+0000 7feeb97bd640 1 -- 172.21.15.114:0/1491086419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feeb4103f20 msgr2=0x7feeb419cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:56.949 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.944+0000 7feeb97bd640 1 --2- 172.21.15.114:0/1491086419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feeb4103f20 0x7feeb419cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:56.949 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.944+0000 7feeb97bd640 1 -- 172.21.15.114:0/1491086419 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feeb40ff210 msgr2=0x7feeb419c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:56.949 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.944+0000 7feeb97bd640 1 --2- 172.21.15.114:0/1491086419 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feeb40ff210 0x7feeb419c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:56.949 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.944+0000 7feeb97bd640 1 -- 172.21.15.114:0/1491086419 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea0009490 con 0x7feeb41a0000 2024-09-13T18:23:56.949 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.944+0000 7feeb97bd640 1 --2- 172.21.15.114:0/1491086419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feeb41a0000 0x7feeb41a04b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7feeac009fd0 tx=0x7feeac00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:56.949 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.945+0000 7feeb1ffb640 1 -- 172.21.15.114:0/1491086419 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeac00ce20 con 0x7feeb41a0000 2024-09-13T18:23:56.949 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.945+0000 7feeb1ffb640 1 -- 172.21.15.114:0/1491086419 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feeac005020 con 0x7feeb41a0000 2024-09-13T18:23:56.949 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.945+0000 7feebb220640 1 -- 172.21.15.114:0/1491086419 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feea800d570 con 0x7feeb41a0000 2024-09-13T18:23:56.949 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.945+0000 7feeb1ffb640 1 -- 172.21.15.114:0/1491086419 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeac010690 con 0x7feeb41a0000 2024-09-13T18:23:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.945+0000 7feebb220640 1 -- 172.21.15.114:0/1491086419 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeb41a9700 con 0x7feeb41a0000 2024-09-13T18:23:56.951 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.948+0000 7feeb1ffb640 1 -- 172.21.15.114:0/1491086419 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 25) v1 ==== 136107+0+0 (secure 0 0 0) 0x7feeac004b20 con 0x7feeb41a0000 2024-09-13T18:23:56.953 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.950+0000 7feeb1ffb640 1 --2- 172.21.15.114:0/1491086419 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7feea40a1420 0x7feea40a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:56.953 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.950+0000 7feebe01e740 1 Processor -- start 2024-09-13T18:23:56.954 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.950+0000 7feeb1ffb640 1 -- 172.21.15.114:0/1491086419 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7feeac014070 con 0x7feeb41a0000 2024-09-13T18:23:56.954 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.950+0000 7feeb8fbc640 1 --2- 172.21.15.114:0/1491086419 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7feea40a1420 0x7feea40a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:56.954 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.950+0000 7feebe01e740 1 -- start start 2024-09-13T18:23:56.954 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.950+0000 7feebe01e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8703d2330 0x55a8703d2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:56.954 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.950+0000 7feebe01e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a8703d2c50 0x55a8703dcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:56.954 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.950+0000 7feebe01e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8703dd4f0 0x55a8703df8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:56.954 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.950+0000 7feeb8fbc640 1 --2- 172.21.15.114:0/1491086419 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7feea40a1420 0x7feea40a38e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7feea8005ab0 tx=0x7feea8005a20 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:56.954 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.951+0000 7feebe01e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a87033dc30 con 0x55a8703dd4f0 2024-09-13T18:23:56.954 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.951+0000 7feebe01e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a87033b3c0 con 0x55a8703d2330 2024-09-13T18:23:56.954 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.951+0000 7feebe01e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a870336cb0 con 0x55a8703d2c50 2024-09-13T18:23:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.951+0000 7feeb97bd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8703dd4f0 0x55a8703df8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.951+0000 7feeb97bd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8703dd4f0 0x55a8703df8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:58456/0 (socket says 172.21.15.114:58456) 2024-09-13T18:23:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.951+0000 7feeb97bd640 1 -- 172.21.15.114:0/587700467 learned_addr learned my addr 172.21.15.114:0/587700467 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.951+0000 7feeb8fbc640 1 --2- 172.21.15.114:0/587700467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8703d2330 0x55a8703d2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.951+0000 7feeb3fff640 1 --2- 172.21.15.114:0/587700467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a8703d2c50 0x55a8703dcfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.951+0000 7feeb8fbc640 1 -- 172.21.15.114:0/587700467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a8703d2c50 msgr2=0x55a8703dcfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.951+0000 7feeb8fbc640 1 --2- 172.21.15.114:0/587700467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a8703d2c50 0x55a8703dcfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.951+0000 7feeb8fbc640 1 -- 172.21.15.114:0/587700467 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8703dd4f0 msgr2=0x55a8703df8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.951+0000 7feeb8fbc640 1 --2- 172.21.15.114:0/587700467 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8703dd4f0 0x55a8703df8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.951+0000 7feeb8fbc640 1 -- 172.21.15.114:0/587700467 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8702dd910 con 0x55a8703d2330 2024-09-13T18:23:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.951+0000 7feeb8fbc640 1 --2- 172.21.15.114:0/587700467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8703d2330 0x55a8703d2710 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7feea802c9a0 tx=0x7feea803d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b537712fc10785b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.951+0000 7fee9effd640 1 -- 172.21.15.114:0/587700467 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea804f070 con 0x55a8703d2330 2024-09-13T18:23:56.956 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.951+0000 7fee9effd640 1 -- 172.21.15.114:0/587700467 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feea802eef0 con 0x55a8703d2330 2024-09-13T18:23:56.956 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.951+0000 7fee9effd640 1 -- 172.21.15.114:0/587700467 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea8045040 con 0x55a8703d2330 2024-09-13T18:23:56.960 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.952+0000 7feebe01e740 1 -- 172.21.15.114:0/587700467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8703d2330 msgr2=0x55a8703d2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:56.960 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.952+0000 7feebe01e740 1 --2- 172.21.15.114:0/587700467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8703d2330 0x55a8703d2710 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7feea802c9a0 tx=0x7feea803d000 comp rx=0 tx=0).stop 2024-09-13T18:23:56.960 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.952+0000 7feebe01e740 1 -- 172.21.15.114:0/587700467 shutdown_connections 2024-09-13T18:23:56.960 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.952+0000 7feebe01e740 1 --2- 172.21.15.114:0/587700467 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8703dd4f0 0x55a8703df8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:56.960 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.952+0000 7feebe01e740 1 --2- 172.21.15.114:0/587700467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a8703d2c50 0x55a8703dcfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:56.960 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.952+0000 7feebe01e740 1 --2- 172.21.15.114:0/587700467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8703d2330 0x55a8703d2710 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:56.960 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.952+0000 7feebe01e740 1 -- 172.21.15.114:0/587700467 >> 172.21.15.114:0/587700467 conn(0x55a8703ce8c0 msgr2=0x55a8703ceca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:56.960 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.952+0000 7feebe01e740 1 -- 172.21.15.114:0/587700467 shutdown_connections 2024-09-13T18:23:56.960 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.952+0000 7feebe01e740 1 -- 172.21.15.114:0/587700467 wait complete. 2024-09-13T18:23:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.953+0000 7feebe01e740 1 -- start start 2024-09-13T18:23:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.953+0000 7feebe01e740 1 Processor -- start 2024-09-13T18:23:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.953+0000 7feebe01e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a8703d2330 0x55a8703dcec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.953+0000 7feebe01e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8703d2c50 0x55a8703da1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.953+0000 7feebe01e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8703dd4f0 0x55a8703da7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.953+0000 7feebe01e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8702f4e80 con 0x55a8703dd4f0 2024-09-13T18:23:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.953+0000 7feebe01e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8702f4b20 con 0x55a8703d2c50 2024-09-13T18:23:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.953+0000 7feebe01e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8702ba260 con 0x55a8703d2330 2024-09-13T18:23:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.954+0000 7feebe01e740 10 client.0 _ll_register_callbacks cb 0x55a8702efdf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:23:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.954+0000 7feeb97bd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8703dd4f0 0x55a8703da7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.954+0000 7feeb97bd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8703dd4f0 0x55a8703da7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:58466/0 (socket says 172.21.15.114:58466) 2024-09-13T18:23:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.954+0000 7feeb97bd640 1 -- 172.21.15.114:0/3028054926 learned_addr learned my addr 172.21.15.114:0/3028054926 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.954+0000 7feeb3fff640 1 --2- 172.21.15.114:0/3028054926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8703d2c50 0x55a8703da1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.954+0000 7feeb8fbc640 1 --2- 172.21.15.114:0/3028054926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a8703d2330 0x55a8703dcec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.954+0000 7feeb3fff640 1 -- 172.21.15.114:0/3028054926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a8703d2330 msgr2=0x55a8703dcec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.954+0000 7feeb3fff640 1 --2- 172.21.15.114:0/3028054926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a8703d2330 0x55a8703dcec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.954+0000 7feeb3fff640 1 -- 172.21.15.114:0/3028054926 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8703dd4f0 msgr2=0x55a8703da7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.954+0000 7feeb3fff640 1 --2- 172.21.15.114:0/3028054926 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8703dd4f0 0x55a8703da7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.954+0000 7feeb3fff640 1 -- 172.21.15.114:0/3028054926 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feea8038040 con 0x55a8703d2c50 2024-09-13T18:23:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.954+0000 7feeb97bd640 1 --2- 172.21.15.114:0/3028054926 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8703dd4f0 0x55a8703da7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:23:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.954+0000 7feeb3fff640 1 --2- 172.21.15.114:0/3028054926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8703d2c50 0x55a8703da1b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7feeb419dd20 tx=0x7feea0012c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ba34de84e1445c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.954+0000 7fee93fff640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:23:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.954+0000 7fee93fff640 1 -- 172.21.15.114:0/3028054926 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea0017070 con 0x55a8703d2c50 2024-09-13T18:23:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.954+0000 7feebe01e740 1 -- 172.21.15.114:0/3028054926 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a87032a5d0 con 0x55a8703d2c50 2024-09-13T18:23:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.954+0000 7fee93fff640 1 -- 172.21.15.114:0/3028054926 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7feea0004020 con 0x55a8703d2c50 2024-09-13T18:23:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.955+0000 7fee93fff640 1 -- 172.21.15.114:0/3028054926 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7feea00041e0 con 0x55a8703d2c50 2024-09-13T18:23:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.955+0000 7feebe01e740 10 client.25568 fetch_fsmap learned FSMap version 30 2024-09-13T18:23:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.955+0000 7feebe01e740 1 -- 172.21.15.114:0/3028054926 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55a870335330 con 0x55a8703d2c50 2024-09-13T18:23:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.955+0000 7fee93fff640 1 -- 172.21.15.114:0/3028054926 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feea000fd80 con 0x55a8703d2c50 2024-09-13T18:23:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.955+0000 7fee93fff640 1 -- 172.21.15.114:0/3028054926 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7feea0003930 con 0x55a8703d2c50 2024-09-13T18:23:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.955+0000 7feebe01e740 10 client.25568 fetch_fsmap finished waiting for FSMap version 30 2024-09-13T18:23:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.955+0000 7feebe01e740 10 client.25568 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:23:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.955+0000 7feebe01e740 20 client.25568 populate_metadata read hostname 'smithi114' 2024-09-13T18:23:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.955+0000 7feebe01e740 1 --2- 172.21.15.114:0/3028054926 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55a8703ecf40 0x55a8703ef360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.955+0000 7feebe01e740 4 client.25568 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:23:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.955+0000 7feebe01e740 1 -- 172.21.15.114:0/3028054926 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a87032a5d0 con 0x55a8703ecf40 2024-09-13T18:23:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.955+0000 7feeb8fbc640 1 --2- 172.21.15.114:0/3028054926 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55a8703ecf40 0x55a8703ef360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.956+0000 7fee93fff640 10 client.25568 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:23:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.956+0000 7feeb8fbc640 1 --2- 172.21.15.114:0/3028054926 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55a8703ecf40 0x55a8703ef360 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2ebc1055bef6767 server_cookie=d1f2130e8fbd5588 in_seq=0 out_seq=0 2024-09-13T18:23:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.956+0000 7fee93fff640 1 -- 172.21.15.114:0/3028054926 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55a87032a5d0 con 0x55a8703ecf40 2024-09-13T18:23:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.956+0000 7fee93fff640 10 client.25568 handle_command_reply: tid=0 2024-09-13T18:23:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.956+0000 7feebe01e740 1 client.25568 shutdown 2024-09-13T18:23:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.957+0000 7feebe01e740 1 -- 172.21.15.114:0/3028054926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8703d2c50 msgr2=0x55a8703da1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.957+0000 7feebe01e740 1 --2- 172.21.15.114:0/3028054926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8703d2c50 0x55a8703da1b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7feeb419dd20 tx=0x7feea0012c70 comp rx=0 tx=0).stop 2024-09-13T18:23:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.957+0000 7feebe01e740 1 -- 172.21.15.114:0/3028054926 shutdown_connections 2024-09-13T18:23:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.957+0000 7feebe01e740 1 --2- 172.21.15.114:0/3028054926 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55a8703ecf40 0x55a8703ef360 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.957+0000 7feebe01e740 1 --2- 172.21.15.114:0/3028054926 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8703dd4f0 0x55a8703da7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.957+0000 7feebe01e740 1 --2- 172.21.15.114:0/3028054926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8703d2c50 0x55a8703da1b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.957+0000 7feebe01e740 1 --2- 172.21.15.114:0/3028054926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a8703d2330 0x55a8703dcec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.957+0000 7feebe01e740 1 -- 172.21.15.114:0/3028054926 >> 172.21.15.114:0/3028054926 conn(0x55a8703ce8c0 msgr2=0x55a8703cdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.957+0000 7fee93fff640 0 client.25568 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:23:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.957+0000 7feebe01e740 1 -- 172.21.15.114:0/3028054926 shutdown_connections 2024-09-13T18:23:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.957+0000 7feebe01e740 1 -- 172.21.15.114:0/3028054926 wait complete. 2024-09-13T18:23:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.957+0000 7feebe01e740 20 client.25568 trim_cache size 0 max 16384 2024-09-13T18:23:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.975+0000 7feebe01e740 1 Processor -- start 2024-09-13T18:23:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.975+0000 7feebe01e740 1 -- start start 2024-09-13T18:23:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.975+0000 7feebe01e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8703d2330 0x55a8703ce300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.975+0000 7feebe01e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a8703d2c50 0x55a8703d54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.975+0000 7feebe01e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8703dd4f0 0x55a8703d59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.975+0000 7feebe01e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8702efd60 con 0x55a8703d2330 2024-09-13T18:23:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.975+0000 7feebe01e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8703d2a40 con 0x55a8703dd4f0 2024-09-13T18:23:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.975+0000 7feebe01e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8703e9490 con 0x55a8703d2c50 2024-09-13T18:23:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.975+0000 7feeb8fbc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8703d2330 0x55a8703ce300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.975+0000 7feeb8fbc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8703d2330 0x55a8703ce300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:58478/0 (socket says 172.21.15.114:58478) 2024-09-13T18:23:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.975+0000 7feeb8fbc640 1 -- 172.21.15.114:0/3544223479 learned_addr learned my addr 172.21.15.114:0/3544223479 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.976+0000 7feeb97bd640 1 --2- 172.21.15.114:0/3544223479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8703dd4f0 0x55a8703d59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.976+0000 7feeb3fff640 1 --2- 172.21.15.114:0/3544223479 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a8703d2c50 0x55a8703d54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.976+0000 7feeb8fbc640 1 -- 172.21.15.114:0/3544223479 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a8703d2c50 msgr2=0x55a8703d54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.976+0000 7feeb8fbc640 1 --2- 172.21.15.114:0/3544223479 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a8703d2c50 0x55a8703d54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.976+0000 7feeb8fbc640 1 -- 172.21.15.114:0/3544223479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8703dd4f0 msgr2=0x55a8703d59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.976+0000 7feeb8fbc640 1 --2- 172.21.15.114:0/3544223479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8703dd4f0 0x55a8703d59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.976+0000 7feeb8fbc640 1 -- 172.21.15.114:0/3544223479 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8702d7300 con 0x55a8703d2330 2024-09-13T18:23:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.976+0000 7feeb8fbc640 1 --2- 172.21.15.114:0/3544223479 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8703d2330 0x55a8703ce300 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7feea8004950 tx=0x7feea802ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a04c31335341cf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.976+0000 7fee9f7fe640 1 -- 172.21.15.114:0/3544223479 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea804f040 con 0x55a8703d2330 2024-09-13T18:23:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.976+0000 7fee9f7fe640 1 -- 172.21.15.114:0/3544223479 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feea8050040 con 0x55a8703d2330 2024-09-13T18:23:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.977+0000 7fee9f7fe640 1 -- 172.21.15.114:0/3544223479 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea8058630 con 0x55a8703d2330 2024-09-13T18:23:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.977+0000 7feebe01e740 1 -- 172.21.15.114:0/3544223479 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8703d2330 msgr2=0x55a8703ce300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.977+0000 7feebe01e740 1 --2- 172.21.15.114:0/3544223479 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8703d2330 0x55a8703ce300 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7feea8004950 tx=0x7feea802ed90 comp rx=0 tx=0).stop 2024-09-13T18:23:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.978+0000 7feebe01e740 1 -- 172.21.15.114:0/3544223479 shutdown_connections 2024-09-13T18:23:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.978+0000 7feebe01e740 1 --2- 172.21.15.114:0/3544223479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8703dd4f0 0x55a8703d59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.978+0000 7feebe01e740 1 --2- 172.21.15.114:0/3544223479 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a8703d2c50 0x55a8703d54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.978+0000 7feebe01e740 1 --2- 172.21.15.114:0/3544223479 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8703d2330 0x55a8703ce300 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.978+0000 7feebe01e740 1 -- 172.21.15.114:0/3544223479 >> 172.21.15.114:0/3544223479 conn(0x55a8703ce8c0 msgr2=0x55a8703f3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.978+0000 7feebe01e740 1 -- 172.21.15.114:0/3544223479 shutdown_connections 2024-09-13T18:23:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.978+0000 7feebe01e740 1 -- 172.21.15.114:0/3544223479 wait complete. 2024-09-13T18:23:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.978+0000 7feebe01e740 1 -- start start 2024-09-13T18:23:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.978+0000 7feebe01e740 1 Processor -- start 2024-09-13T18:23:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.979+0000 7feebe01e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a8703d2330 0x55a8703e43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.979+0000 7feebe01e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8703d2c50 0x55a8703d6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.979+0000 7feebe01e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8703dd4f0 0x55a8703d6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.979+0000 7feebe01e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8703d74e0 con 0x55a8703d2c50 2024-09-13T18:23:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.979+0000 7feebe01e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8703d7650 con 0x55a8703dd4f0 2024-09-13T18:23:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.979+0000 7feebe01e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8703d7810 con 0x55a8703d2330 2024-09-13T18:23:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.979+0000 7feeb3fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8703d2c50 0x55a8703d6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.979+0000 7feeb3fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8703d2c50 0x55a8703d6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:58486/0 (socket says 172.21.15.114:58486) 2024-09-13T18:23:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.979+0000 7feeb8fbc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a8703d2330 0x55a8703e43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.979+0000 7feeb3fff640 1 -- 172.21.15.114:0/3903562075 learned_addr learned my addr 172.21.15.114:0/3903562075 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:23:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.979+0000 7feeb97bd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8703dd4f0 0x55a8703d6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.979+0000 7feebe01e740 10 client.0 _ll_register_callbacks cb 0x7feeac0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:23:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.979+0000 7feeb97bd640 1 -- 172.21.15.114:0/3903562075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a8703d2330 msgr2=0x55a8703e43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.979+0000 7feeb97bd640 1 --2- 172.21.15.114:0/3903562075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a8703d2330 0x55a8703e43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.979+0000 7feeb97bd640 1 -- 172.21.15.114:0/3903562075 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8703d2c50 msgr2=0x55a8703d6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:23:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.979+0000 7feeb97bd640 1 --2- 172.21.15.114:0/3903562075 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8703d2c50 0x55a8703d6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.979+0000 7feeb97bd640 1 -- 172.21.15.114:0/3903562075 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a8702dd910 con 0x55a8703dd4f0 2024-09-13T18:23:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.979+0000 7feeb8fbc640 1 --2- 172.21.15.114:0/3903562075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a8703d2330 0x55a8703e43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:23:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.979+0000 7feeb3fff640 1 --2- 172.21.15.114:0/3903562075 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8703d2c50 0x55a8703d6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:23:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.980+0000 7feeb97bd640 1 --2- 172.21.15.114:0/3903562075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8703dd4f0 0x55a8703d6fa0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7feeac00c760 tx=0x7feeac005530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56afd8680a932594 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:23:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.980+0000 7fee93fff640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:23:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.980+0000 7fee93fff640 1 -- 172.21.15.114:0/3903562075 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeac09e180 con 0x55a8703dd4f0 2024-09-13T18:23:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.980+0000 7fee93fff640 1 -- 172.21.15.114:0/3903562075 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7feeac0a27d0 con 0x55a8703dd4f0 2024-09-13T18:23:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.980+0000 7feebe01e740 1 -- 172.21.15.114:0/3903562075 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a87035f680 con 0x55a8703dd4f0 2024-09-13T18:23:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.980+0000 7fee93fff640 1 -- 172.21.15.114:0/3903562075 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7feeac09a5b0 con 0x55a8703dd4f0 2024-09-13T18:23:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.980+0000 7feebe01e740 10 client.25574 fetch_fsmap learned FSMap version 30 2024-09-13T18:23:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.980+0000 7feebe01e740 1 -- 172.21.15.114:0/3903562075 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55a8702fc8e0 con 0x55a8703dd4f0 2024-09-13T18:23:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.980+0000 7fee93fff640 1 -- 172.21.15.114:0/3903562075 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a87035f680 con 0x55a8703dd4f0 2024-09-13T18:23:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.980+0000 7fee93fff640 1 -- 172.21.15.114:0/3903562075 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7feeac080070 con 0x55a8703dd4f0 2024-09-13T18:23:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.980+0000 7feebe01e740 10 client.25574 fetch_fsmap finished waiting for FSMap version 30 2024-09-13T18:23:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.980+0000 7feebe01e740 10 client.25574 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:23:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.981+0000 7feebe01e740 20 client.25574 populate_metadata read hostname 'smithi114' 2024-09-13T18:23:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.981+0000 7feebe01e740 1 --2- 172.21.15.114:0/3903562075 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55a870444980 0x55a870446d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:23:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.981+0000 7feebe01e740 4 client.25574 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T18:23:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.981+0000 7feebe01e740 1 -- 172.21.15.114:0/3903562075 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a87002a300 con 0x55a870444980 2024-09-13T18:23:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.981+0000 7feeb8fbc640 1 --2- 172.21.15.114:0/3903562075 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55a870444980 0x55a870446d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:23:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.981+0000 7fee93fff640 10 client.25574 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:23:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.981+0000 7feeb8fbc640 1 --2- 172.21.15.114:0/3903562075 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55a870444980 0x55a870446d70 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f070312ce9a6071 server_cookie=2c96976e7306027a in_seq=0 out_seq=0 2024-09-13T18:23:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.991+0000 7fee93fff640 1 -- 172.21.15.114:0/3903562075 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a87002a300 con 0x55a870444980 2024-09-13T18:23:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.992+0000 7fee93fff640 10 client.25574 handle_command_reply: tid=0 2024-09-13T18:23:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.992+0000 7feebe01e740 1 client.25574 shutdown 2024-09-13T18:23:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.992+0000 7feebe01e740 1 -- 172.21.15.114:0/3903562075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8703dd4f0 msgr2=0x55a8703d6fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.992+0000 7feebe01e740 1 --2- 172.21.15.114:0/3903562075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8703dd4f0 0x55a8703d6fa0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7feeac00c760 tx=0x7feeac005530 comp rx=0 tx=0).stop 2024-09-13T18:23:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.992+0000 7feebe01e740 1 -- 172.21.15.114:0/3903562075 shutdown_connections 2024-09-13T18:23:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.992+0000 7feebe01e740 1 --2- 172.21.15.114:0/3903562075 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55a870444980 0x55a870446d70 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.992+0000 7feebe01e740 1 --2- 172.21.15.114:0/3903562075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8703dd4f0 0x55a8703d6fa0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.992+0000 7fee93fff640 0 client.25574 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:23:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.992+0000 7feebe01e740 1 --2- 172.21.15.114:0/3903562075 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8703d2c50 0x55a8703d6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.992+0000 7feebe01e740 1 --2- 172.21.15.114:0/3903562075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a8703d2330 0x55a8703e43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.992+0000 7feebe01e740 1 -- 172.21.15.114:0/3903562075 >> 172.21.15.114:0/3903562075 conn(0x55a8703ce8c0 msgr2=0x55a8703d54b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.992+0000 7feebe01e740 1 -- 172.21.15.114:0/3903562075 shutdown_connections 2024-09-13T18:23:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.992+0000 7feebe01e740 1 -- 172.21.15.114:0/3903562075 wait complete. 2024-09-13T18:23:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.992+0000 7feebe01e740 20 client.25574 trim_cache size 0 max 16384 2024-09-13T18:23:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-13T18:23:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-13T18:23:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-13T18:23:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-13T18:23:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.993+0000 7feebb220640 1 -- 172.21.15.114:0/1491086419 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7feea40a1420 msgr2=0x7feea40a38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.993+0000 7feebb220640 1 --2- 172.21.15.114:0/1491086419 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7feea40a1420 0x7feea40a38e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7feea8005ab0 tx=0x7feea8005a20 comp rx=0 tx=0).stop 2024-09-13T18:23:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.993+0000 7feebb220640 1 -- 172.21.15.114:0/1491086419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feeb41a0000 msgr2=0x7feeb41a04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:23:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.993+0000 7feebb220640 1 --2- 172.21.15.114:0/1491086419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feeb41a0000 0x7feeb41a04b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7feeac009fd0 tx=0x7feeac00ef10 comp rx=0 tx=0).stop 2024-09-13T18:23:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.993+0000 7feebb220640 1 -- 172.21.15.114:0/1491086419 shutdown_connections 2024-09-13T18:23:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.993+0000 7feebb220640 1 --2- 172.21.15.114:0/1491086419 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7feea40a1420 0x7feea40a38e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.993+0000 7feebb220640 1 --2- 172.21.15.114:0/1491086419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feeb41a0000 0x7feeb41a04b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.993+0000 7feebb220640 1 --2- 172.21.15.114:0/1491086419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feeb4103f20 0x7feeb419cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.993+0000 7feebb220640 1 --2- 172.21.15.114:0/1491086419 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feeb40ff210 0x7feeb419c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:23:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.993+0000 7feebb220640 1 -- 172.21.15.114:0/1491086419 >> 172.21.15.114:0/1491086419 conn(0x7feeb4074930 msgr2=0x7feeb40fd280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:23:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.993+0000 7feebb220640 1 -- 172.21.15.114:0/1491086419 shutdown_connections 2024-09-13T18:23:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:23:56.993+0000 7feebb220640 1 -- 172.21.15.114:0/1491086419 wait complete. 2024-09-13T18:23:58.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:23:58 smithi204 ceph-mon[34777]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-13T18:23:58.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:23:58 smithi204 ceph-mon[34777]: pgmap v48: 129 pgs: 129 active+clean; 2.3 GiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 153 MiB/s wr, 87 op/s 2024-09-13T18:23:58.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:23:58 smithi114 ceph-mon[30897]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-13T18:23:58.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:23:58 smithi114 ceph-mon[30897]: pgmap v48: 129 pgs: 129 active+clean; 2.3 GiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 153 MiB/s wr, 87 op/s 2024-09-13T18:23:58.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:23:58 smithi134 ceph-mon[34283]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-13T18:23:58.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:23:58 smithi134 ceph-mon[34283]: pgmap v48: 129 pgs: 129 active+clean; 2.3 GiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 153 MiB/s wr, 87 op/s 2024-09-13T18:24:00.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:00 smithi204 ceph-mon[34777]: pgmap v49: 129 pgs: 129 active+clean; 3.2 GiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 228 MiB/s wr, 134 op/s 2024-09-13T18:24:00.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:00 smithi204 ceph-mon[34777]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-09-13T18:24:00.469 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:00 smithi114 ceph-mon[30897]: pgmap v49: 129 pgs: 129 active+clean; 3.2 GiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 228 MiB/s wr, 134 op/s 2024-09-13T18:24:00.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:00 smithi114 ceph-mon[30897]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-09-13T18:24:00.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:00 smithi134 ceph-mon[34283]: pgmap v49: 129 pgs: 129 active+clean; 3.2 GiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 228 MiB/s wr, 134 op/s 2024-09-13T18:24:00.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:00 smithi134 ceph-mon[34283]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-09-13T18:24:02.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:02 smithi134 ceph-mon[34283]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-13T18:24:02.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:02 smithi134 ceph-mon[34283]: pgmap v50: 129 pgs: 129 active+clean; 3.2 GiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 152 MiB/s wr, 90 op/s 2024-09-13T18:24:02.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:02 smithi204 ceph-mon[34777]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-13T18:24:02.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:02 smithi204 ceph-mon[34777]: pgmap v50: 129 pgs: 129 active+clean; 3.2 GiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 152 MiB/s wr, 90 op/s 2024-09-13T18:24:02.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:02 smithi114 ceph-mon[30897]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-13T18:24:02.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:02 smithi114 ceph-mon[30897]: pgmap v50: 129 pgs: 129 active+clean; 3.2 GiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 152 MiB/s wr, 90 op/s 2024-09-13T18:24:04.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:04 smithi204 ceph-mon[34777]: pgmap v51: 129 pgs: 129 active+clean; 4.0 GiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 225 MiB/s wr, 146 op/s 2024-09-13T18:24:04.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:04 smithi114 ceph-mon[30897]: pgmap v51: 129 pgs: 129 active+clean; 4.0 GiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 225 MiB/s wr, 146 op/s 2024-09-13T18:24:04.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:04 smithi134 ceph-mon[34283]: pgmap v51: 129 pgs: 129 active+clean; 4.0 GiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 225 MiB/s wr, 146 op/s 2024-09-13T18:24:06.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:06 smithi204 ceph-mon[34777]: pgmap v52: 129 pgs: 129 active+clean; 4.0 GiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 149 MiB/s wr, 103 op/s 2024-09-13T18:24:06.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:06 smithi114 ceph-mon[30897]: pgmap v52: 129 pgs: 129 active+clean; 4.0 GiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 149 MiB/s wr, 103 op/s 2024-09-13T18:24:06.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:06 smithi134 ceph-mon[34283]: pgmap v52: 129 pgs: 129 active+clean; 4.0 GiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 149 MiB/s wr, 103 op/s 2024-09-13T18:24:08.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:08 smithi204 ceph-mon[34777]: pgmap v53: 129 pgs: 129 active+clean; 4.0 GiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 149 MiB/s wr, 103 op/s 2024-09-13T18:24:08.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:08 smithi114 ceph-mon[30897]: pgmap v53: 129 pgs: 129 active+clean; 4.0 GiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 149 MiB/s wr, 103 op/s 2024-09-13T18:24:08.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:08 smithi134 ceph-mon[34283]: pgmap v53: 129 pgs: 129 active+clean; 4.0 GiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 149 MiB/s wr, 103 op/s 2024-09-13T18:24:10.123 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.119+0000 7fbe81e27640 1 -- 172.21.15.114:0/3919780564 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe7c10c580 msgr2=0x7fbe7c10e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:10.123 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.119+0000 7fbe81e27640 1 --2- 172.21.15.114:0/3919780564 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe7c10c580 0x7fbe7c10e9d0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fbe74009ca0 tx=0x7fbe7402a8c0 comp rx=0 tx=0).stop 2024-09-13T18:24:10.123 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.119+0000 7fbe81e27640 1 -- 172.21.15.114:0/3919780564 shutdown_connections 2024-09-13T18:24:10.123 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.119+0000 7fbe81e27640 1 --2- 172.21.15.114:0/3919780564 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe7c10c580 0x7fbe7c10e9d0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:10.123 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.119+0000 7fbe81e27640 1 --2- 172.21.15.114:0/3919780564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe7c0ff330 0x7fbe7c10bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:10.123 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.119+0000 7fbe81e27640 1 --2- 172.21.15.114:0/3919780564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe7c0fe980 0x7fbe7c0fed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:10.123 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.119+0000 7fbe81e27640 1 -- 172.21.15.114:0/3919780564 >> 172.21.15.114:0/3919780564 conn(0x7fbe7c075380 msgr2=0x7fbe7c075790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.119+0000 7fbe81e27640 1 -- 172.21.15.114:0/3919780564 shutdown_connections 2024-09-13T18:24:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.119+0000 7fbe81e27640 1 -- 172.21.15.114:0/3919780564 wait complete. 2024-09-13T18:24:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.120+0000 7fbe81e27640 1 Processor -- start 2024-09-13T18:24:10.126 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.120+0000 7fbe81e27640 1 -- start start 2024-09-13T18:24:10.126 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.121+0000 7fbe81e27640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe7c0fe980 0x7fbe7c198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:10.126 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.121+0000 7fbe81e27640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe7c0ff330 0x7fbe7c198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:10.126 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.121+0000 7fbe81e27640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe7c10c580 0x7fbe7c19cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:10.126 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.121+0000 7fbe81e27640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7c19d230 con 0x7fbe7c0ff330 2024-09-13T18:24:10.126 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.121+0000 7fbe81e27640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7c19d3a0 con 0x7fbe7c10c580 2024-09-13T18:24:10.126 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.121+0000 7fbe81e27640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7c19d560 con 0x7fbe7c0fe980 2024-09-13T18:24:10.126 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.121+0000 7fbe7b7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe7c0fe980 0x7fbe7c198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:10.126 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.121+0000 7fbe7affd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe7c0ff330 0x7fbe7c198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:10.126 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.121+0000 7fbe7b7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe7c0fe980 0x7fbe7c198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:43322/0 (socket says 172.21.15.114:43322) 2024-09-13T18:24:10.127 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.121+0000 7fbe7b7fe640 1 -- 172.21.15.114:0/3908245802 learned_addr learned my addr 172.21.15.114:0/3908245802 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:10.127 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.121+0000 7fbe7bfff640 1 --2- 172.21.15.114:0/3908245802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe7c10c580 0x7fbe7c19cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:10.127 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.121+0000 7fbe7b7fe640 1 -- 172.21.15.114:0/3908245802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe7c10c580 msgr2=0x7fbe7c19cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:10.127 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.121+0000 7fbe7b7fe640 1 --2- 172.21.15.114:0/3908245802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe7c10c580 0x7fbe7c19cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:10.127 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.121+0000 7fbe7b7fe640 1 -- 172.21.15.114:0/3908245802 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe7c0ff330 msgr2=0x7fbe7c198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:10.127 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.121+0000 7fbe7b7fe640 1 --2- 172.21.15.114:0/3908245802 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe7c0ff330 0x7fbe7c198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:10.127 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.121+0000 7fbe7b7fe640 1 -- 172.21.15.114:0/3908245802 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe74009900 con 0x7fbe7c0fe980 2024-09-13T18:24:10.127 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.121+0000 7fbe7affd640 1 --2- 172.21.15.114:0/3908245802 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe7c0ff330 0x7fbe7c198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:24:10.127 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.121+0000 7fbe7b7fe640 1 --2- 172.21.15.114:0/3908245802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe7c0fe980 0x7fbe7c198300 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c00e9f0 tx=0x7fbe6c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:10.127 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.122+0000 7fbe78ff9640 1 -- 172.21.15.114:0/3908245802 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe6c009720 con 0x7fbe7c0fe980 2024-09-13T18:24:10.127 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.122+0000 7fbe81e27640 1 -- 172.21.15.114:0/3908245802 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe7c19d840 con 0x7fbe7c0fe980 2024-09-13T18:24:10.127 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.122+0000 7fbe78ff9640 1 -- 172.21.15.114:0/3908245802 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbe6c00cd80 con 0x7fbe7c0fe980 2024-09-13T18:24:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.122+0000 7fbe78ff9640 1 -- 172.21.15.114:0/3908245802 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe6c010430 con 0x7fbe7c0fe980 2024-09-13T18:24:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.122+0000 7fbe81e27640 1 -- 172.21.15.114:0/3908245802 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe7c1a50c0 con 0x7fbe7c0fe980 2024-09-13T18:24:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.125+0000 7fbe78ff9640 1 -- 172.21.15.114:0/3908245802 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fbe6c004fb0 con 0x7fbe7c0fe980 2024-09-13T18:24:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.125+0000 7fbe84c24740 1 Processor -- start 2024-09-13T18:24:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.126+0000 7fbe84c24740 1 -- start start 2024-09-13T18:24:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.126+0000 7fbe84c24740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617ff90d330 0x5617ff90d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.126+0000 7fbe84c24740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5617ff90dc50 0x5617ff917fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.126+0000 7fbe84c24740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5617ff9184f0 0x5617ff91a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.126+0000 7fbe84c24740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617ff878c30 con 0x5617ff9184f0 2024-09-13T18:24:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.126+0000 7fbe84c24740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617ff8763c0 con 0x5617ff90d330 2024-09-13T18:24:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.126+0000 7fbe84c24740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617ff871cb0 con 0x5617ff90dc50 2024-09-13T18:24:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.126+0000 7fbe7affd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5617ff9184f0 0x5617ff91a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.126+0000 7fbe7affd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5617ff9184f0 0x5617ff91a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43286/0 (socket says 172.21.15.114:43286) 2024-09-13T18:24:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.126+0000 7fbe7affd640 1 -- 172.21.15.114:0/3343482169 learned_addr learned my addr 172.21.15.114:0/3343482169 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.126+0000 7fbe7bfff640 1 --2- 172.21.15.114:0/3343482169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617ff90d330 0x5617ff90d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.126+0000 7fbe7b7fe640 1 --2- 172.21.15.114:0/3343482169 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5617ff90dc50 0x5617ff917fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:10.132 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.127+0000 7fbe7affd640 1 -- 172.21.15.114:0/3343482169 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5617ff90dc50 msgr2=0x5617ff917fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:10.132 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.127+0000 7fbe7affd640 1 --2- 172.21.15.114:0/3343482169 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5617ff90dc50 0x5617ff917fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:10.132 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.127+0000 7fbe7affd640 1 -- 172.21.15.114:0/3343482169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617ff90d330 msgr2=0x5617ff90d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:10.132 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.127+0000 7fbe7affd640 1 --2- 172.21.15.114:0/3343482169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617ff90d330 0x5617ff90d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:10.132 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.127+0000 7fbe7affd640 1 -- 172.21.15.114:0/3343482169 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617ff818910 con 0x5617ff9184f0 2024-09-13T18:24:10.132 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.127+0000 7fbe7affd640 1 --2- 172.21.15.114:0/3343482169 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5617ff9184f0 0x5617ff91a8e0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fbe700114f0 tx=0x7fbe700119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6937c56f7428d4e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:10.132 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.127+0000 7fbe78ff9640 1 --2- 172.21.15.114:0/3908245802 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbe640a1280 0x7fbe640a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:10.132 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.127+0000 7fbe61ffb640 1 -- 172.21.15.114:0/3343482169 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe70005020 con 0x5617ff9184f0 2024-09-13T18:24:10.132 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.127+0000 7fbe61ffb640 1 -- 172.21.15.114:0/3343482169 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbe70005610 con 0x5617ff9184f0 2024-09-13T18:24:10.132 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.127+0000 7fbe61ffb640 1 -- 172.21.15.114:0/3343482169 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7000f880 con 0x5617ff9184f0 2024-09-13T18:24:10.132 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.127+0000 7fbe7bfff640 1 --2- 172.21.15.114:0/3908245802 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbe640a1280 0x7fbe640a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:10.132 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.128+0000 7fbe78ff9640 1 -- 172.21.15.114:0/3908245802 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fbe6c014070 con 0x7fbe7c0fe980 2024-09-13T18:24:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.128+0000 7fbe84c24740 1 -- 172.21.15.114:0/3343482169 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5617ff9184f0 msgr2=0x5617ff91a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.128+0000 7fbe84c24740 1 --2- 172.21.15.114:0/3343482169 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5617ff9184f0 0x5617ff91a8e0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fbe700114f0 tx=0x7fbe700119b0 comp rx=0 tx=0).stop 2024-09-13T18:24:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.128+0000 7fbe7bfff640 1 --2- 172.21.15.114:0/3908245802 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbe640a1280 0x7fbe640a3740 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbe74006010 tx=0x7fbe74005d80 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.128+0000 7fbe84c24740 1 -- 172.21.15.114:0/3343482169 shutdown_connections 2024-09-13T18:24:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.128+0000 7fbe84c24740 1 --2- 172.21.15.114:0/3343482169 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5617ff9184f0 0x5617ff91a8e0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.128+0000 7fbe84c24740 1 --2- 172.21.15.114:0/3343482169 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5617ff90dc50 0x5617ff917fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.128+0000 7fbe84c24740 1 --2- 172.21.15.114:0/3343482169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617ff90d330 0x5617ff90d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.128+0000 7fbe84c24740 1 -- 172.21.15.114:0/3343482169 >> 172.21.15.114:0/3343482169 conn(0x5617ff9098c0 msgr2=0x5617ff909ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.128+0000 7fbe84c24740 1 -- 172.21.15.114:0/3343482169 shutdown_connections 2024-09-13T18:24:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.128+0000 7fbe84c24740 1 -- 172.21.15.114:0/3343482169 wait complete. 2024-09-13T18:24:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.129+0000 7fbe84c24740 1 -- start start 2024-09-13T18:24:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.129+0000 7fbe84c24740 1 Processor -- start 2024-09-13T18:24:10.139 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.129+0000 7fbe84c24740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5617ff90d330 0x5617ff917e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:10.139 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.129+0000 7fbe84c24740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5617ff90dc50 0x5617ff915180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:10.139 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.129+0000 7fbe84c24740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617ff9184f0 0x5617ff915790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:10.139 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.129+0000 7fbe84c24740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617ff82fe80 con 0x5617ff90dc50 2024-09-13T18:24:10.139 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.129+0000 7fbe84c24740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617ff82fb20 con 0x5617ff9184f0 2024-09-13T18:24:10.139 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.129+0000 7fbe84c24740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617ff7f5260 con 0x5617ff90d330 2024-09-13T18:24:10.139 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.129+0000 7fbe84c24740 10 client.0 _ll_register_callbacks cb 0x5617ff82adf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:24:10.139 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.130+0000 7fbe7bfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617ff9184f0 0x5617ff915790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:10.139 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.130+0000 7fbe7affd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5617ff90dc50 0x5617ff915180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:10.139 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.130+0000 7fbe7bfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617ff9184f0 0x5617ff915790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:51610/0 (socket says 172.21.15.114:51610) 2024-09-13T18:24:10.139 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.130+0000 7fbe7affd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5617ff90dc50 0x5617ff915180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43298/0 (socket says 172.21.15.114:43298) 2024-09-13T18:24:10.140 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.130+0000 7fbe7bfff640 1 -- 172.21.15.114:0/3206901055 learned_addr learned my addr 172.21.15.114:0/3206901055 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:10.140 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.130+0000 7fbe7b7fe640 1 --2- 172.21.15.114:0/3206901055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5617ff90d330 0x5617ff917e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:10.140 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.130+0000 7fbe7bfff640 1 -- 172.21.15.114:0/3206901055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5617ff90d330 msgr2=0x5617ff917e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:10.140 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.130+0000 7fbe7bfff640 1 --2- 172.21.15.114:0/3206901055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5617ff90d330 0x5617ff917e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:10.140 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.130+0000 7fbe7bfff640 1 -- 172.21.15.114:0/3206901055 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5617ff90dc50 msgr2=0x5617ff915180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:10.140 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.130+0000 7fbe7bfff640 1 --2- 172.21.15.114:0/3206901055 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5617ff90dc50 0x5617ff915180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:10.140 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.130+0000 7fbe7bfff640 1 -- 172.21.15.114:0/3206901055 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe70009c80 con 0x5617ff9184f0 2024-09-13T18:24:10.140 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.130+0000 7fbe7bfff640 1 --2- 172.21.15.114:0/3206901055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617ff9184f0 0x5617ff915790 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbe74035000 tx=0x7fbe7402a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5beb5f0883ac36d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:10.140 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.130+0000 7fbe5affd640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:24:10.140 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.130+0000 7fbe5affd640 1 -- 172.21.15.114:0/3206901055 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe74042070 con 0x5617ff9184f0 2024-09-13T18:24:10.140 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.130+0000 7fbe84c24740 1 -- 172.21.15.114:0/3206901055 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5617ff89a680 con 0x5617ff9184f0 2024-09-13T18:24:10.140 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.130+0000 7fbe5affd640 1 -- 172.21.15.114:0/3206901055 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fbe740305c0 con 0x5617ff9184f0 2024-09-13T18:24:10.140 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.130+0000 7fbe5affd640 1 -- 172.21.15.114:0/3206901055 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe74030780 con 0x5617ff9184f0 2024-09-13T18:24:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.131+0000 7fbe84c24740 10 client.25586 fetch_fsmap learned FSMap version 32 2024-09-13T18:24:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.131+0000 7fbe84c24740 1 -- 172.21.15.114:0/3206901055 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5617ff870330 con 0x5617ff9184f0 2024-09-13T18:24:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.131+0000 7fbe5affd640 1 -- 172.21.15.114:0/3206901055 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbe7405d080 con 0x5617ff9184f0 2024-09-13T18:24:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.131+0000 7fbe5affd640 1 -- 172.21.15.114:0/3206901055 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fbe7402fd40 con 0x5617ff9184f0 2024-09-13T18:24:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.131+0000 7fbe84c24740 10 client.25586 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:24:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.131+0000 7fbe84c24740 10 client.25586 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:24:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.131+0000 7fbe84c24740 20 client.25586 populate_metadata read hostname 'smithi114' 2024-09-13T18:24:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.131+0000 7fbe84c24740 1 --2- 172.21.15.114:0/3206901055 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5617ff927f70 0x5617ff92a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.131+0000 7fbe84c24740 4 client.25586 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:24:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.131+0000 7fbe84c24740 1 -- 172.21.15.114:0/3206901055 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5617ff8655d0 con 0x5617ff927f70 2024-09-13T18:24:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.131+0000 7fbe7b7fe640 1 --2- 172.21.15.114:0/3206901055 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5617ff927f70 0x5617ff92a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.132+0000 7fbe7b7fe640 1 --2- 172.21.15.114:0/3206901055 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5617ff927f70 0x5617ff92a390 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26f474b95a6513bb server_cookie=ebcab4926302edd9 in_seq=0 out_seq=0 2024-09-13T18:24:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.132+0000 7fbe5affd640 10 client.25586 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:24:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.132+0000 7fbe5affd640 1 -- 172.21.15.114:0/3206901055 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5617ff8655d0 con 0x5617ff927f70 2024-09-13T18:24:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.132+0000 7fbe5affd640 10 client.25586 handle_command_reply: tid=0 2024-09-13T18:24:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.132+0000 7fbe84c24740 1 client.25586 shutdown 2024-09-13T18:24:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.133+0000 7fbe84c24740 1 -- 172.21.15.114:0/3206901055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617ff9184f0 msgr2=0x5617ff915790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.133+0000 7fbe84c24740 1 --2- 172.21.15.114:0/3206901055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617ff9184f0 0x5617ff915790 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbe74035000 tx=0x7fbe7402a8c0 comp rx=0 tx=0).stop 2024-09-13T18:24:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.133+0000 7fbe84c24740 1 -- 172.21.15.114:0/3206901055 shutdown_connections 2024-09-13T18:24:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.133+0000 7fbe84c24740 1 --2- 172.21.15.114:0/3206901055 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5617ff927f70 0x5617ff92a390 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.133+0000 7fbe84c24740 1 --2- 172.21.15.114:0/3206901055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617ff9184f0 0x5617ff915790 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.133+0000 7fbe84c24740 1 --2- 172.21.15.114:0/3206901055 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5617ff90dc50 0x5617ff915180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.133+0000 7fbe5affd640 0 client.25586 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:24:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.133+0000 7fbe84c24740 1 --2- 172.21.15.114:0/3206901055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5617ff90d330 0x5617ff917e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.133+0000 7fbe84c24740 1 -- 172.21.15.114:0/3206901055 >> 172.21.15.114:0/3206901055 conn(0x5617ff9098c0 msgr2=0x5617ff908b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.133+0000 7fbe84c24740 1 -- 172.21.15.114:0/3206901055 shutdown_connections 2024-09-13T18:24:10.142 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.133+0000 7fbe84c24740 1 -- 172.21.15.114:0/3206901055 wait complete. 2024-09-13T18:24:10.143 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.133+0000 7fbe84c24740 20 client.25586 trim_cache size 0 max 16384 2024-09-13T18:24:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.152+0000 7fbe84c24740 1 Processor -- start 2024-09-13T18:24:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.152+0000 7fbe84c24740 1 -- start start 2024-09-13T18:24:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.153+0000 7fbe84c24740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617ff90d330 0x5617ff909300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.153+0000 7fbe84c24740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5617ff90dc50 0x5617ff91f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:10.158 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.153+0000 7fbe84c24740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5617ff9184f0 0x5617ff91f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:10.158 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.153+0000 7fbe84c24740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617ff905460 con 0x5617ff9184f0 2024-09-13T18:24:10.158 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.153+0000 7fbe84c24740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617ff926720 con 0x5617ff90d330 2024-09-13T18:24:10.158 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.153+0000 7fbe84c24740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617ff917a50 con 0x5617ff90dc50 2024-09-13T18:24:10.158 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.153+0000 7fbe7bfff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5617ff9184f0 0x5617ff91f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:10.158 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.153+0000 7fbe7bfff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5617ff9184f0 0x5617ff91f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43304/0 (socket says 172.21.15.114:43304) 2024-09-13T18:24:10.158 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.153+0000 7fbe7bfff640 1 -- 172.21.15.114:0/1286790302 learned_addr learned my addr 172.21.15.114:0/1286790302 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:10.158 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.153+0000 7fbe7b7fe640 1 --2- 172.21.15.114:0/1286790302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617ff90d330 0x5617ff909300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:10.158 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.153+0000 7fbe7affd640 1 --2- 172.21.15.114:0/1286790302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5617ff90dc50 0x5617ff91f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:10.158 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.153+0000 7fbe7bfff640 1 -- 172.21.15.114:0/1286790302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5617ff90dc50 msgr2=0x5617ff91f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.153+0000 7fbe7bfff640 1 --2- 172.21.15.114:0/1286790302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5617ff90dc50 0x5617ff91f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.153+0000 7fbe7bfff640 1 -- 172.21.15.114:0/1286790302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617ff90d330 msgr2=0x5617ff909300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.153+0000 7fbe7bfff640 1 --2- 172.21.15.114:0/1286790302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617ff90d330 0x5617ff909300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.153+0000 7fbe7bfff640 1 -- 172.21.15.114:0/1286790302 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617ff812300 con 0x5617ff9184f0 2024-09-13T18:24:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.153+0000 7fbe7b7fe640 1 --2- 172.21.15.114:0/1286790302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617ff90d330 0x5617ff909300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:24:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.153+0000 7fbe7bfff640 1 --2- 172.21.15.114:0/1286790302 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5617ff9184f0 0x5617ff91f720 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fbe7402ade0 tx=0x7fbe74031f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3309a1e2aecabaa server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.154+0000 7fbe627fc640 1 -- 172.21.15.114:0/1286790302 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe74054660 con 0x5617ff9184f0 2024-09-13T18:24:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.154+0000 7fbe627fc640 1 -- 172.21.15.114:0/1286790302 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5617ff89a680 con 0x5617ff9184f0 2024-09-13T18:24:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.154+0000 7fbe627fc640 1 -- 172.21.15.114:0/1286790302 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7402f5d0 con 0x5617ff9184f0 2024-09-13T18:24:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.155+0000 7fbe84c24740 1 -- 172.21.15.114:0/1286790302 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5617ff9184f0 msgr2=0x5617ff91f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.155+0000 7fbe84c24740 1 --2- 172.21.15.114:0/1286790302 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5617ff9184f0 0x5617ff91f720 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fbe7402ade0 tx=0x7fbe74031f20 comp rx=0 tx=0).stop 2024-09-13T18:24:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.155+0000 7fbe84c24740 1 -- 172.21.15.114:0/1286790302 shutdown_connections 2024-09-13T18:24:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.155+0000 7fbe84c24740 1 --2- 172.21.15.114:0/1286790302 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5617ff9184f0 0x5617ff91f720 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.155+0000 7fbe84c24740 1 --2- 172.21.15.114:0/1286790302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5617ff90dc50 0x5617ff91f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.155+0000 7fbe84c24740 1 --2- 172.21.15.114:0/1286790302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617ff90d330 0x5617ff909300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.155+0000 7fbe84c24740 1 -- 172.21.15.114:0/1286790302 >> 172.21.15.114:0/1286790302 conn(0x5617ff9098c0 msgr2=0x5617ff910470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.156+0000 7fbe84c24740 1 -- 172.21.15.114:0/1286790302 shutdown_connections 2024-09-13T18:24:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.156+0000 7fbe84c24740 1 -- 172.21.15.114:0/1286790302 wait complete. 2024-09-13T18:24:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.156+0000 7fbe84c24740 1 -- start start 2024-09-13T18:24:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.156+0000 7fbe84c24740 1 Processor -- start 2024-09-13T18:24:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.157+0000 7fbe84c24740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5617ff90d330 0x5617ff91f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.157+0000 7fbe84c24740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5617ff90dc50 0x5617ff91b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.157+0000 7fbe84c24740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617ff9184f0 0x5617ff91bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.157+0000 7fbe84c24740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617ff91c5f0 con 0x5617ff90dc50 2024-09-13T18:24:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.157+0000 7fbe84c24740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617ff91c760 con 0x5617ff9184f0 2024-09-13T18:24:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.157+0000 7fbe84c24740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617ff97e470 con 0x5617ff90d330 2024-09-13T18:24:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.157+0000 7fbe84c24740 10 client.0 _ll_register_callbacks cb 0x7fbe70009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:24:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.157+0000 7fbe7affd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5617ff90dc50 0x5617ff91b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.157+0000 7fbe7b7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5617ff90d330 0x5617ff91f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.157+0000 7fbe7affd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5617ff90dc50 0x5617ff91b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43310/0 (socket says 172.21.15.114:43310) 2024-09-13T18:24:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.157+0000 7fbe7affd640 1 -- 172.21.15.114:0/491652353 learned_addr learned my addr 172.21.15.114:0/491652353 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.157+0000 7fbe7bfff640 1 --2- 172.21.15.114:0/491652353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617ff9184f0 0x5617ff91bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.157+0000 7fbe7b7fe640 1 -- 172.21.15.114:0/491652353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617ff9184f0 msgr2=0x5617ff91bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.157+0000 7fbe7b7fe640 1 --2- 172.21.15.114:0/491652353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617ff9184f0 0x5617ff91bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.157+0000 7fbe7b7fe640 1 -- 172.21.15.114:0/491652353 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5617ff90dc50 msgr2=0x5617ff91b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.157+0000 7fbe7b7fe640 1 --2- 172.21.15.114:0/491652353 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5617ff90dc50 0x5617ff91b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.157+0000 7fbe7b7fe640 1 -- 172.21.15.114:0/491652353 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe7c1a50c0 con 0x5617ff90d330 2024-09-13T18:24:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.157+0000 7fbe7affd640 1 --2- 172.21.15.114:0/491652353 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5617ff90dc50 0x5617ff91b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:24:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.157+0000 7fbe7b7fe640 1 --2- 172.21.15.114:0/491652353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5617ff90d330 0x5617ff91f5f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c0ca7e0 tx=0x7fbe6c00cfc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f760338c3c339bff server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.157+0000 7fbe5affd640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:24:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.158+0000 7fbe5affd640 1 -- 172.21.15.114:0/491652353 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe6c09a8a0 con 0x5617ff90d330 2024-09-13T18:24:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.158+0000 7fbe5affd640 1 -- 172.21.15.114:0/491652353 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fbe6c09a1a0 con 0x5617ff90d330 2024-09-13T18:24:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.158+0000 7fbe84c24740 1 -- 172.21.15.114:0/491652353 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbe70005610 con 0x5617ff90d330 2024-09-13T18:24:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.158+0000 7fbe5affd640 1 -- 172.21.15.114:0/491652353 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe6c09dfd0 con 0x5617ff90d330 2024-09-13T18:24:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.158+0000 7fbe84c24740 10 client.25633 fetch_fsmap learned FSMap version 32 2024-09-13T18:24:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.158+0000 7fbe84c24740 1 -- 172.21.15.114:0/491652353 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5617ff870330 con 0x5617ff90d330 2024-09-13T18:24:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.159+0000 7fbe5affd640 1 -- 172.21.15.114:0/491652353 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbe6c0a2620 con 0x5617ff90d330 2024-09-13T18:24:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.159+0000 7fbe5affd640 1 -- 172.21.15.114:0/491652353 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fbe6c01f060 con 0x5617ff90d330 2024-09-13T18:24:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.159+0000 7fbe84c24740 10 client.25633 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:24:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.159+0000 7fbe84c24740 10 client.25633 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:24:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.159+0000 7fbe84c24740 20 client.25633 populate_metadata read hostname 'smithi114' 2024-09-13T18:24:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.159+0000 7fbe84c24740 1 --2- 172.21.15.114:0/491652353 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5617ff927f70 0x5617ff981c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.159+0000 7fbe84c24740 4 client.25633 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T18:24:10.168 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.159+0000 7fbe84c24740 1 -- 172.21.15.114:0/491652353 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7fbe70005610 con 0x5617ff927f70 2024-09-13T18:24:10.168 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.160+0000 7fbe7affd640 1 --2- 172.21.15.114:0/491652353 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5617ff927f70 0x5617ff981c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:10.168 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.160+0000 7fbe5affd640 10 client.25633 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:24:10.168 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.160+0000 7fbe7affd640 1 --2- 172.21.15.114:0/491652353 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5617ff927f70 0x5617ff981c30 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3faa2846f9ccbe20 server_cookie=85446dc089f38f63 in_seq=0 out_seq=0 2024-09-13T18:24:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.171+0000 7fbe5affd640 1 -- 172.21.15.114:0/491652353 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7fbe70005610 con 0x5617ff927f70 2024-09-13T18:24:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.171+0000 7fbe5affd640 10 client.25633 handle_command_reply: tid=0 2024-09-13T18:24:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.171+0000 7fbe84c24740 1 client.25633 shutdown 2024-09-13T18:24:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.171+0000 7fbe84c24740 1 -- 172.21.15.114:0/491652353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5617ff90d330 msgr2=0x5617ff91f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.171+0000 7fbe84c24740 1 --2- 172.21.15.114:0/491652353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5617ff90d330 0x5617ff91f5f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c0ca7e0 tx=0x7fbe6c00cfc0 comp rx=0 tx=0).stop 2024-09-13T18:24:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.171+0000 7fbe84c24740 1 -- 172.21.15.114:0/491652353 shutdown_connections 2024-09-13T18:24:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.171+0000 7fbe84c24740 1 --2- 172.21.15.114:0/491652353 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5617ff927f70 0x5617ff981c30 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.171+0000 7fbe84c24740 1 --2- 172.21.15.114:0/491652353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617ff9184f0 0x5617ff91bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.171+0000 7fbe84c24740 1 --2- 172.21.15.114:0/491652353 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5617ff90dc50 0x5617ff91b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.171+0000 7fbe84c24740 1 --2- 172.21.15.114:0/491652353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5617ff90d330 0x5617ff91f5f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.171+0000 7fbe5affd640 0 client.25633 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:24:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.171+0000 7fbe84c24740 1 -- 172.21.15.114:0/491652353 >> 172.21.15.114:0/491652353 conn(0x5617ff9098c0 msgr2=0x5617ff9139d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.171+0000 7fbe84c24740 1 -- 172.21.15.114:0/491652353 shutdown_connections 2024-09-13T18:24:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.171+0000 7fbe84c24740 1 -- 172.21.15.114:0/491652353 wait complete. 2024-09-13T18:24:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.172+0000 7fbe84c24740 20 client.25633 trim_cache size 0 max 16384 2024-09-13T18:24:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-13T18:24:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-13T18:24:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-13T18:24:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-13T18:24:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.172+0000 7fbe81e27640 1 -- 172.21.15.114:0/3908245802 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbe640a1280 msgr2=0x7fbe640a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.172+0000 7fbe81e27640 1 --2- 172.21.15.114:0/3908245802 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbe640a1280 0x7fbe640a3740 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbe74006010 tx=0x7fbe74005d80 comp rx=0 tx=0).stop 2024-09-13T18:24:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.172+0000 7fbe81e27640 1 -- 172.21.15.114:0/3908245802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe7c0fe980 msgr2=0x7fbe7c198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.172+0000 7fbe81e27640 1 --2- 172.21.15.114:0/3908245802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe7c0fe980 0x7fbe7c198300 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c00e9f0 tx=0x7fbe6c00edf0 comp rx=0 tx=0).stop 2024-09-13T18:24:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.172+0000 7fbe81e27640 1 -- 172.21.15.114:0/3908245802 shutdown_connections 2024-09-13T18:24:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.172+0000 7fbe81e27640 1 --2- 172.21.15.114:0/3908245802 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbe640a1280 0x7fbe640a3740 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.172+0000 7fbe81e27640 1 --2- 172.21.15.114:0/3908245802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe7c10c580 0x7fbe7c19cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.172+0000 7fbe81e27640 1 --2- 172.21.15.114:0/3908245802 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe7c0ff330 0x7fbe7c198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.172+0000 7fbe81e27640 1 --2- 172.21.15.114:0/3908245802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe7c0fe980 0x7fbe7c198300 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.172+0000 7fbe81e27640 1 -- 172.21.15.114:0/3908245802 >> 172.21.15.114:0/3908245802 conn(0x7fbe7c075380 msgr2=0x7fbe7c10cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.172+0000 7fbe81e27640 1 -- 172.21.15.114:0/3908245802 shutdown_connections 2024-09-13T18:24:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:10.172+0000 7fbe81e27640 1 -- 172.21.15.114:0/3908245802 wait complete. 2024-09-13T18:24:10.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:10 smithi204 ceph-mon[34777]: pgmap v54: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 223 MiB/s wr, 145 op/s 2024-09-13T18:24:10.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:10 smithi114 ceph-mon[30897]: pgmap v54: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 223 MiB/s wr, 145 op/s 2024-09-13T18:24:10.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:10 smithi134 ceph-mon[34283]: pgmap v54: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 223 MiB/s wr, 145 op/s 2024-09-13T18:24:11.162 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T18:24:11.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.311+0000 7f60fdf45640 1 -- 172.21.15.114:0/3254858711 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60f810c400 msgr2=0x7f60f810c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:11.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.311+0000 7f60fdf45640 1 --2- 172.21.15.114:0/3254858711 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60f810c400 0x7f60f810c7e0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f60e8009930 tx=0x7f60e802a4d0 comp rx=0 tx=0).stop 2024-09-13T18:24:11.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.311+0000 7f60fdf45640 1 -- 172.21.15.114:0/3254858711 shutdown_connections 2024-09-13T18:24:11.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.311+0000 7f60fdf45640 1 --2- 172.21.15.114:0/3254858711 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60f8106cc0 0x7f60f81131a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:11.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.311+0000 7f60fdf45640 1 --2- 172.21.15.114:0/3254858711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60f8106340 0x7f60f8106780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:11.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.311+0000 7f60fdf45640 1 --2- 172.21.15.114:0/3254858711 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60f810c400 0x7f60f810c7e0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:11.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.311+0000 7f60fdf45640 1 -- 172.21.15.114:0/3254858711 >> 172.21.15.114:0/3254858711 conn(0x7f60f8075c80 msgr2=0x7f60f80780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:11.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.311+0000 7f60fdf45640 1 -- 172.21.15.114:0/3254858711 shutdown_connections 2024-09-13T18:24:11.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.311+0000 7f60fdf45640 1 -- 172.21.15.114:0/3254858711 wait complete. 2024-09-13T18:24:11.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.312+0000 7f60fdf45640 1 Processor -- start 2024-09-13T18:24:11.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.312+0000 7f60fdf45640 1 -- start start 2024-09-13T18:24:11.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.312+0000 7f60fdf45640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60f8106340 0x7f60f819c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:11.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.312+0000 7f60fdf45640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60f8106cc0 0x7f60f819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:11.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.312+0000 7f60fdf45640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60f810c400 0x7f60f819ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:11.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.312+0000 7f60fdf45640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f81a0580 con 0x7f60f810c400 2024-09-13T18:24:11.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.312+0000 7f60fdf45640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f81a06f0 con 0x7f60f8106cc0 2024-09-13T18:24:11.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.312+0000 7f60fdf45640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f81a08b0 con 0x7f60f8106340 2024-09-13T18:24:11.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.312+0000 7f60f7fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60f810c400 0x7f60f819ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:11.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.312+0000 7f60f7fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60f810c400 0x7f60f819ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43334/0 (socket says 172.21.15.114:43334) 2024-09-13T18:24:11.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.312+0000 7f60f7fff640 1 -- 172.21.15.114:0/3105242104 learned_addr learned my addr 172.21.15.114:0/3105242104 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:11.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.312+0000 7f60eeffd640 1 --2- 172.21.15.114:0/3105242104 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60f8106cc0 0x7f60f819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:11.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.313+0000 7f60f77fe640 1 --2- 172.21.15.114:0/3105242104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60f8106340 0x7f60f819c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:11.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.313+0000 7f60f7fff640 1 -- 172.21.15.114:0/3105242104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60f8106340 msgr2=0x7f60f819c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:11.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.313+0000 7f60f7fff640 1 --2- 172.21.15.114:0/3105242104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60f8106340 0x7f60f819c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:11.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.313+0000 7f60f7fff640 1 -- 172.21.15.114:0/3105242104 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60f8106cc0 msgr2=0x7f60f819cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:11.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.313+0000 7f60f7fff640 1 --2- 172.21.15.114:0/3105242104 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60f8106cc0 0x7f60f819cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:11.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.313+0000 7f60f7fff640 1 -- 172.21.15.114:0/3105242104 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60e8009560 con 0x7f60f810c400 2024-09-13T18:24:11.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.313+0000 7f60f7fff640 1 --2- 172.21.15.114:0/3105242104 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60f810c400 0x7f60f819ff80 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f60f000cce0 tx=0x7f60f000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:11.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.313+0000 7f60f57fa640 1 -- 172.21.15.114:0/3105242104 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f000f450 con 0x7f60f810c400 2024-09-13T18:24:11.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.313+0000 7f60f57fa640 1 -- 172.21.15.114:0/3105242104 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f60f000fa40 con 0x7f60f810c400 2024-09-13T18:24:11.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.313+0000 7f60fdf45640 1 -- 172.21.15.114:0/3105242104 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60f81a0b90 con 0x7f60f810c400 2024-09-13T18:24:11.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.313+0000 7f60f57fa640 1 -- 172.21.15.114:0/3105242104 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f0018900 con 0x7f60f810c400 2024-09-13T18:24:11.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.314+0000 7f60fdf45640 1 -- 172.21.15.114:0/3105242104 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60f81a94f0 con 0x7f60f810c400 2024-09-13T18:24:11.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.317+0000 7f60f57fa640 1 -- 172.21.15.114:0/3105242104 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f60f000fbb0 con 0x7f60f810c400 2024-09-13T18:24:11.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.317+0000 7f6100c66740 1 Processor -- start 2024-09-13T18:24:11.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.317+0000 7f6100c66740 1 -- start start 2024-09-13T18:24:11.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.317+0000 7f60f57fa640 1 --2- 172.21.15.114:0/3105242104 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f60d80a1280 0x7f60d80a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:11.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.317+0000 7f6100c66740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56261921fe10 0x5626192201f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:11.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.317+0000 7f6100c66740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562619220730 0x56261922aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:11.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.318+0000 7f60f57fa640 1 -- 172.21.15.114:0/3105242104 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f60f00cd450 con 0x7f60f810c400 2024-09-13T18:24:11.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.318+0000 7f6100c66740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56261922afd0 0x56261922d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:11.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.318+0000 7f6100c66740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56261918af20 con 0x562619220730 2024-09-13T18:24:11.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.318+0000 7f6100c66740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626191886b0 con 0x56261921fe10 2024-09-13T18:24:11.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.318+0000 7f6100c66740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562619183fa0 con 0x56261922afd0 2024-09-13T18:24:11.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.318+0000 7f60eeffd640 1 --2- 172.21.15.114:0/3105242104 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f60d80a1280 0x7f60d80a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:11.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.318+0000 7f60f77fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56261921fe10 0x5626192201f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:11.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.318+0000 7f60f77fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56261921fe10 0x5626192201f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:51650/0 (socket says 172.21.15.114:51650) 2024-09-13T18:24:11.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.318+0000 7f60f77fe640 1 -- 172.21.15.114:0/2374388978 learned_addr learned my addr 172.21.15.114:0/2374388978 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:11.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.318+0000 7f60f7fff640 1 --2- 172.21.15.114:0/2374388978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56261922afd0 0x56261922d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:11.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.318+0000 7f60eeffd640 1 --2- 172.21.15.114:0/2374388978 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562619220730 0x56261922aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:11.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.318+0000 7f60f77fe640 1 -- 172.21.15.114:0/2374388978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56261922afd0 msgr2=0x56261922d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:11.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.318+0000 7f60f77fe640 1 --2- 172.21.15.114:0/2374388978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56261922afd0 0x56261922d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:11.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.318+0000 7f60f77fe640 1 -- 172.21.15.114:0/2374388978 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562619220730 msgr2=0x56261922aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:11.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.318+0000 7f60f77fe640 1 --2- 172.21.15.114:0/2374388978 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562619220730 0x56261922aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:11.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.318+0000 7f60f77fe640 1 -- 172.21.15.114:0/2374388978 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56261913e840 con 0x56261921fe10 2024-09-13T18:24:11.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.318+0000 7f60f77fe640 1 --2- 172.21.15.114:0/2374388978 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56261921fe10 0x5626192201f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f60e8009900 tx=0x7f60e8004350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d070a7bde67d174 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:11.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.319+0000 7f60eeffd640 1 --2- 172.21.15.114:0/3105242104 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f60d80a1280 0x7f60d80a3740 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f60f819dca0 tx=0x7f60e0009910 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:11.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.319+0000 7f60edffb640 1 -- 172.21.15.114:0/2374388978 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e8005200 con 0x56261921fe10 2024-09-13T18:24:11.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.319+0000 7f60edffb640 1 -- 172.21.15.114:0/2374388978 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f60e8005360 con 0x56261921fe10 2024-09-13T18:24:11.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.319+0000 7f60edffb640 1 -- 172.21.15.114:0/2374388978 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e8003950 con 0x56261921fe10 2024-09-13T18:24:11.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.320+0000 7f6100c66740 1 -- 172.21.15.114:0/2374388978 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56261921fe10 msgr2=0x5626192201f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:11.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.320+0000 7f6100c66740 1 --2- 172.21.15.114:0/2374388978 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56261921fe10 0x5626192201f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f60e8009900 tx=0x7f60e8004350 comp rx=0 tx=0).stop 2024-09-13T18:24:11.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.320+0000 7f6100c66740 1 -- 172.21.15.114:0/2374388978 shutdown_connections 2024-09-13T18:24:11.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.320+0000 7f6100c66740 1 --2- 172.21.15.114:0/2374388978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56261922afd0 0x56261922d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:11.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.320+0000 7f6100c66740 1 --2- 172.21.15.114:0/2374388978 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562619220730 0x56261922aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:11.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.320+0000 7f6100c66740 1 --2- 172.21.15.114:0/2374388978 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56261921fe10 0x5626192201f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:11.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.320+0000 7f6100c66740 1 -- 172.21.15.114:0/2374388978 >> 172.21.15.114:0/2374388978 conn(0x56261921c380 msgr2=0x56261921c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:11.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.320+0000 7f6100c66740 1 -- 172.21.15.114:0/2374388978 shutdown_connections 2024-09-13T18:24:11.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.320+0000 7f6100c66740 1 -- 172.21.15.114:0/2374388978 wait complete. 2024-09-13T18:24:11.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.321+0000 7f6100c66740 1 -- start start 2024-09-13T18:24:11.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.321+0000 7f6100c66740 1 Processor -- start 2024-09-13T18:24:11.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.321+0000 7f6100c66740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56261921fe10 0x56261922a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:11.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.321+0000 7f6100c66740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562619220730 0x562619227cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:11.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.322+0000 7f6100c66740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56261922afd0 0x5626192282a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:11.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.322+0000 7f6100c66740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626191252a0 con 0x56261921fe10 2024-09-13T18:24:11.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.322+0000 7f6100c66740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562618e5c430 con 0x562619220730 2024-09-13T18:24:11.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.322+0000 7f6100c66740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562619107680 con 0x56261922afd0 2024-09-13T18:24:11.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.322+0000 7f60eeffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562619220730 0x562619227cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:11.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.322+0000 7f6100c66740 10 client.0 _ll_register_callbacks cb 0x56261917d610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:24:11.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.322+0000 7f60eeffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562619220730 0x562619227cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:51658/0 (socket says 172.21.15.114:51658) 2024-09-13T18:24:11.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.322+0000 7f60eeffd640 1 -- 172.21.15.114:0/3063100073 learned_addr learned my addr 172.21.15.114:0/3063100073 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:11.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.322+0000 7f60f77fe640 1 --2- 172.21.15.114:0/3063100073 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56261921fe10 0x56261922a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:11.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.322+0000 7f60f7fff640 1 --2- 172.21.15.114:0/3063100073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56261922afd0 0x5626192282a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:11.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.322+0000 7f60f7fff640 1 -- 172.21.15.114:0/3063100073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562619220730 msgr2=0x562619227cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:11.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.322+0000 7f60f7fff640 1 --2- 172.21.15.114:0/3063100073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562619220730 0x562619227cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:11.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.322+0000 7f60f7fff640 1 -- 172.21.15.114:0/3063100073 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56261921fe10 msgr2=0x56261922a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:11.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.322+0000 7f60f7fff640 1 --2- 172.21.15.114:0/3063100073 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56261921fe10 0x56261922a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:11.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.322+0000 7f60f7fff640 1 -- 172.21.15.114:0/3063100073 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60f8113aa0 con 0x56261922afd0 2024-09-13T18:24:11.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.322+0000 7f60f7fff640 1 --2- 172.21.15.114:0/3063100073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56261922afd0 0x5626192282a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f60f00ca850 tx=0x7f60f009a9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d27cfe74ce7fc2a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:11.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.322+0000 7f60e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:24:11.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.323+0000 7f60e6ffd640 1 -- 172.21.15.114:0/3063100073 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f00a2690 con 0x56261922afd0 2024-09-13T18:24:11.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.323+0000 7f6100c66740 1 -- 172.21.15.114:0/3063100073 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562618e774e0 con 0x56261922afd0 2024-09-13T18:24:11.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.323+0000 7f60e6ffd640 1 -- 172.21.15.114:0/3063100073 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f60f0099700 con 0x56261922afd0 2024-09-13T18:24:11.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.324+0000 7f60e6ffd640 1 -- 172.21.15.114:0/3063100073 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60f00998c0 con 0x56261922afd0 2024-09-13T18:24:11.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.324+0000 7f6100c66740 10 client.25642 fetch_fsmap learned FSMap version 32 2024-09-13T18:24:11.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.324+0000 7f6100c66740 1 -- 172.21.15.114:0/3063100073 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x562619182620 con 0x56261922afd0 2024-09-13T18:24:11.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.324+0000 7f60e6ffd640 1 -- 172.21.15.114:0/3063100073 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562618e774e0 con 0x56261922afd0 2024-09-13T18:24:11.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.324+0000 7f60e6ffd640 1 -- 172.21.15.114:0/3063100073 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f60f0016020 con 0x56261922afd0 2024-09-13T18:24:11.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.325+0000 7f6100c66740 10 client.25642 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:24:11.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.325+0000 7f6100c66740 10 client.25642 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:24:11.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.325+0000 7f6100c66740 20 client.25642 populate_metadata read hostname 'smithi114' 2024-09-13T18:24:11.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.325+0000 7f6100c66740 1 --2- 172.21.15.114:0/3063100073 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56261923aff0 0x56261923d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:11.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.325+0000 7f6100c66740 4 client.25642 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:24:11.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.325+0000 7f6100c66740 1 -- 172.21.15.114:0/3063100073 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562618e887e0 con 0x56261923aff0 2024-09-13T18:24:11.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.325+0000 7f60f77fe640 1 --2- 172.21.15.114:0/3063100073 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56261923aff0 0x56261923d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:11.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.326+0000 7f60f77fe640 1 --2- 172.21.15.114:0/3063100073 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56261923aff0 0x56261923d410 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44128b812ac3334 server_cookie=c8f4941f3e8419d in_seq=0 out_seq=0 2024-09-13T18:24:11.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.326+0000 7f60e6ffd640 10 client.25642 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:24:11.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.326+0000 7f60e6ffd640 1 -- 172.21.15.114:0/3063100073 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x562618e887e0 con 0x56261923aff0 2024-09-13T18:24:11.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.326+0000 7f60e6ffd640 10 client.25642 handle_command_reply: tid=0 2024-09-13T18:24:11.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.326+0000 7f6100c66740 1 client.25642 shutdown 2024-09-13T18:24:11.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.327+0000 7f6100c66740 1 -- 172.21.15.114:0/3063100073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56261922afd0 msgr2=0x5626192282a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:11.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.327+0000 7f6100c66740 1 --2- 172.21.15.114:0/3063100073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56261922afd0 0x5626192282a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f60f00ca850 tx=0x7f60f009a9d0 comp rx=0 tx=0).stop 2024-09-13T18:24:11.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.327+0000 7f6100c66740 1 -- 172.21.15.114:0/3063100073 shutdown_connections 2024-09-13T18:24:11.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.327+0000 7f6100c66740 1 --2- 172.21.15.114:0/3063100073 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56261923aff0 0x56261923d410 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:11.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.327+0000 7f6100c66740 1 --2- 172.21.15.114:0/3063100073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56261922afd0 0x5626192282a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:11.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.327+0000 7f6100c66740 1 --2- 172.21.15.114:0/3063100073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562619220730 0x562619227cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:11.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.327+0000 7f6100c66740 1 --2- 172.21.15.114:0/3063100073 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56261921fe10 0x56261922a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:11.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.327+0000 7f6100c66740 1 -- 172.21.15.114:0/3063100073 >> 172.21.15.114:0/3063100073 conn(0x56261921c380 msgr2=0x5626192220e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:11.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.327+0000 7f60e6ffd640 0 client.25642 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:24:11.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.327+0000 7f6100c66740 1 -- 172.21.15.114:0/3063100073 shutdown_connections 2024-09-13T18:24:11.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.327+0000 7f6100c66740 1 -- 172.21.15.114:0/3063100073 wait complete. 2024-09-13T18:24:11.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.327+0000 7f6100c66740 20 client.25642 trim_cache size 0 max 16384 2024-09-13T18:24:11.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.349+0000 7f6100c66740 1 Processor -- start 2024-09-13T18:24:11.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.349+0000 7f6100c66740 1 -- start start 2024-09-13T18:24:11.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.350+0000 7f6100c66740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56261921fe10 0x56261921bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:11.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.350+0000 7f6100c66740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562619220730 0x56261922a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:11.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.350+0000 7f6100c66740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56261922afd0 0x562619231d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:11.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.350+0000 7f6100c66740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56261921c240 con 0x562619220730 2024-09-13T18:24:11.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.350+0000 7f6100c66740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562619217790 con 0x56261921fe10 2024-09-13T18:24:11.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.350+0000 7f6100c66740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562619239680 con 0x56261922afd0 2024-09-13T18:24:11.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.350+0000 7f60eeffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562619220730 0x56261922a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:11.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.350+0000 7f60f7fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56261922afd0 0x562619231d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:11.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.350+0000 7f60f7fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56261922afd0 0x562619231d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:43386/0 (socket says 172.21.15.114:43386) 2024-09-13T18:24:11.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.350+0000 7f60eeffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562619220730 0x56261922a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43366/0 (socket says 172.21.15.114:43366) 2024-09-13T18:24:11.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.350+0000 7f60f7fff640 1 -- 172.21.15.114:0/1756030258 learned_addr learned my addr 172.21.15.114:0/1756030258 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:11.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.350+0000 7f60f77fe640 1 --2- 172.21.15.114:0/1756030258 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56261921fe10 0x56261921bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:11.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.350+0000 7f60eeffd640 1 -- 172.21.15.114:0/1756030258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56261922afd0 msgr2=0x562619231d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:11.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.350+0000 7f60eeffd640 1 --2- 172.21.15.114:0/1756030258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56261922afd0 0x562619231d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:11.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.350+0000 7f60eeffd640 1 -- 172.21.15.114:0/1756030258 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56261921fe10 msgr2=0x56261921bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:11.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.350+0000 7f60eeffd640 1 --2- 172.21.15.114:0/1756030258 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56261921fe10 0x56261921bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:11.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.350+0000 7f60eeffd640 1 -- 172.21.15.114:0/1756030258 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562619124790 con 0x562619220730 2024-09-13T18:24:11.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.350+0000 7f60f7fff640 1 --2- 172.21.15.114:0/1756030258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56261922afd0 0x562619231d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:24:11.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.350+0000 7f60f77fe640 1 --2- 172.21.15.114:0/1756030258 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56261921fe10 0x56261921bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:24:11.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.351+0000 7f60eeffd640 1 --2- 172.21.15.114:0/1756030258 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562619220730 0x56261922a4d0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x5626192292a0 tx=0x7f60e001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a15d4ae5e24dc98c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:11.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.351+0000 7f60ee7fc640 1 -- 172.21.15.114:0/1756030258 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e0005bd0 con 0x562619220730 2024-09-13T18:24:11.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.351+0000 7f60ee7fc640 1 -- 172.21.15.114:0/1756030258 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f60e0005d30 con 0x562619220730 2024-09-13T18:24:11.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.351+0000 7f60ee7fc640 1 -- 172.21.15.114:0/1756030258 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e0024550 con 0x562619220730 2024-09-13T18:24:11.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.352+0000 7f6100c66740 1 -- 172.21.15.114:0/1756030258 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562619220730 msgr2=0x56261922a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:11.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.352+0000 7f6100c66740 1 --2- 172.21.15.114:0/1756030258 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562619220730 0x56261922a4d0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x5626192292a0 tx=0x7f60e001c000 comp rx=0 tx=0).stop 2024-09-13T18:24:11.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.352+0000 7f6100c66740 1 -- 172.21.15.114:0/1756030258 shutdown_connections 2024-09-13T18:24:11.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.352+0000 7f6100c66740 1 --2- 172.21.15.114:0/1756030258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56261922afd0 0x562619231d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:11.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.352+0000 7f6100c66740 1 --2- 172.21.15.114:0/1756030258 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562619220730 0x56261922a4d0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:11.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.352+0000 7f6100c66740 1 --2- 172.21.15.114:0/1756030258 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56261921fe10 0x56261921bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:11.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.352+0000 7f6100c66740 1 -- 172.21.15.114:0/1756030258 >> 172.21.15.114:0/1756030258 conn(0x56261921c380 msgr2=0x562619241d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:11.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.352+0000 7f6100c66740 1 -- 172.21.15.114:0/1756030258 shutdown_connections 2024-09-13T18:24:11.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.352+0000 7f6100c66740 1 -- 172.21.15.114:0/1756030258 wait complete. 2024-09-13T18:24:11.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.352+0000 7f6100c66740 1 -- start start 2024-09-13T18:24:11.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.353+0000 7f6100c66740 1 Processor -- start 2024-09-13T18:24:11.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.353+0000 7f6100c66740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56261921fe10 0x56261922dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:11.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.353+0000 7f6100c66740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562619220730 0x56261922e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:11.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.353+0000 7f6100c66740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56261922afd0 0x56261922e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:11.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.353+0000 7f6100c66740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626192904e0 con 0x562619220730 2024-09-13T18:24:11.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.353+0000 7f6100c66740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562619290620 con 0x56261921fe10 2024-09-13T18:24:11.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.353+0000 7f6100c66740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562619290790 con 0x56261922afd0 2024-09-13T18:24:11.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.353+0000 7f60f77fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56261921fe10 0x56261922dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:11.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.353+0000 7f60eeffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562619220730 0x56261922e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:11.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.353+0000 7f60f7fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56261922afd0 0x56261922e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:11.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.353+0000 7f60eeffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562619220730 0x56261922e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43368/0 (socket says 172.21.15.114:43368) 2024-09-13T18:24:11.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.353+0000 7f60eeffd640 1 -- 172.21.15.114:0/3464967791 learned_addr learned my addr 172.21.15.114:0/3464967791 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:11.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.353+0000 7f6100c66740 10 client.0 _ll_register_callbacks cb 0x7f60f0099b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:24:11.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.354+0000 7f60f7fff640 1 -- 172.21.15.114:0/3464967791 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56261921fe10 msgr2=0x56261922dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:11.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.354+0000 7f60f7fff640 1 --2- 172.21.15.114:0/3464967791 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56261921fe10 0x56261922dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:11.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.354+0000 7f60f7fff640 1 -- 172.21.15.114:0/3464967791 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562619220730 msgr2=0x56261922e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:11.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.354+0000 7f60f7fff640 1 --2- 172.21.15.114:0/3464967791 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562619220730 0x56261922e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:11.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.354+0000 7f60f7fff640 1 -- 172.21.15.114:0/3464967791 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56261913e840 con 0x56261922afd0 2024-09-13T18:24:11.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.354+0000 7f60f7fff640 1 --2- 172.21.15.114:0/3464967791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56261922afd0 0x56261922e790 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f60f000ccb0 tx=0x7f60f0099c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e969e2c827aba75f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:11.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.354+0000 7f60e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:24:11.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.354+0000 7f60e6ffd640 1 -- 172.21.15.114:0/3464967791 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f0099900 con 0x56261922afd0 2024-09-13T18:24:11.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.354+0000 7f6100c66740 1 -- 172.21.15.114:0/3464967791 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f60e8005360 con 0x56261922afd0 2024-09-13T18:24:11.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.354+0000 7f60e6ffd640 1 -- 172.21.15.114:0/3464967791 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f60f0096240 con 0x56261922afd0 2024-09-13T18:24:11.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.354+0000 7f60e6ffd640 1 -- 172.21.15.114:0/3464967791 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60f00a2690 con 0x56261922afd0 2024-09-13T18:24:11.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.354+0000 7f6100c66740 10 client.25648 fetch_fsmap learned FSMap version 32 2024-09-13T18:24:11.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.354+0000 7f6100c66740 1 -- 172.21.15.114:0/3464967791 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x562619182620 con 0x56261922afd0 2024-09-13T18:24:11.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.355+0000 7f60e6ffd640 1 -- 172.21.15.114:0/3464967791 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f60f00b8250 con 0x56261922afd0 2024-09-13T18:24:11.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.355+0000 7f60e6ffd640 1 -- 172.21.15.114:0/3464967791 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f60f0099170 con 0x56261922afd0 2024-09-13T18:24:11.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.355+0000 7f6100c66740 10 client.25648 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:24:11.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.355+0000 7f6100c66740 10 client.25648 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:24:11.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.355+0000 7f6100c66740 20 client.25648 populate_metadata read hostname 'smithi114' 2024-09-13T18:24:11.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.355+0000 7f6100c66740 1 --2- 172.21.15.114:0/3464967791 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x562619291bf0 0x562619294010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:11.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.355+0000 7f6100c66740 4 client.25648 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T18:24:11.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.355+0000 7f6100c66740 1 -- 172.21.15.114:0/3464967791 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f60e8005360 con 0x562619291bf0 2024-09-13T18:24:11.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.355+0000 7f60f77fe640 1 --2- 172.21.15.114:0/3464967791 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x562619291bf0 0x562619294010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:11.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.355+0000 7f60e6ffd640 10 client.25648 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:24:11.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.355+0000 7f60f77fe640 1 --2- 172.21.15.114:0/3464967791 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x562619291bf0 0x562619294010 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f88ce463719cc059 server_cookie=c2b4c41ac89d975e in_seq=0 out_seq=0 2024-09-13T18:24:11.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.356+0000 7f60e6ffd640 1 -- 172.21.15.114:0/3464967791 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x7f60e8005360 con 0x562619291bf0 2024-09-13T18:24:11.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.356+0000 7f60e6ffd640 10 client.25648 handle_command_reply: tid=0 2024-09-13T18:24:11.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.356+0000 7f6100c66740 1 client.25648 shutdown 2024-09-13T18:24:11.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.356+0000 7f6100c66740 1 -- 172.21.15.114:0/3464967791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56261922afd0 msgr2=0x56261922e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:11.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.356+0000 7f6100c66740 1 --2- 172.21.15.114:0/3464967791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56261922afd0 0x56261922e790 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f60f000ccb0 tx=0x7f60f0099c40 comp rx=0 tx=0).stop 2024-09-13T18:24:11.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.356+0000 7f6100c66740 1 -- 172.21.15.114:0/3464967791 shutdown_connections 2024-09-13T18:24:11.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.356+0000 7f6100c66740 1 --2- 172.21.15.114:0/3464967791 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x562619291bf0 0x562619294010 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:11.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.356+0000 7f6100c66740 1 --2- 172.21.15.114:0/3464967791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56261922afd0 0x56261922e790 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:11.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.356+0000 7f6100c66740 1 --2- 172.21.15.114:0/3464967791 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562619220730 0x56261922e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:11.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.356+0000 7f6100c66740 1 --2- 172.21.15.114:0/3464967791 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56261921fe10 0x56261922dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:11.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.356+0000 7f6100c66740 1 -- 172.21.15.114:0/3464967791 >> 172.21.15.114:0/3464967791 conn(0x56261921c380 msgr2=0x562619222f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:11.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.356+0000 7f60e6ffd640 0 client.25648 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:24:11.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.356+0000 7f6100c66740 1 -- 172.21.15.114:0/3464967791 shutdown_connections 2024-09-13T18:24:11.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.356+0000 7f6100c66740 1 -- 172.21.15.114:0/3464967791 wait complete. 2024-09-13T18:24:11.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.356+0000 7f6100c66740 20 client.25648 trim_cache size 0 max 16384 2024-09-13T18:24:11.365 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:24:11.365 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "no active scrubs running", 2024-09-13T18:24:11.365 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": {} 2024-09-13T18:24:11.365 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:24:11.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.357+0000 7f60fdf45640 1 -- 172.21.15.114:0/3105242104 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f60d80a1280 msgr2=0x7f60d80a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:11.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.357+0000 7f60fdf45640 1 --2- 172.21.15.114:0/3105242104 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f60d80a1280 0x7f60d80a3740 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f60f819dca0 tx=0x7f60e0009910 comp rx=0 tx=0).stop 2024-09-13T18:24:11.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.357+0000 7f60fdf45640 1 -- 172.21.15.114:0/3105242104 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60f810c400 msgr2=0x7f60f819ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:11.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.357+0000 7f60fdf45640 1 --2- 172.21.15.114:0/3105242104 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60f810c400 0x7f60f819ff80 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f60f000cce0 tx=0x7f60f000a620 comp rx=0 tx=0).stop 2024-09-13T18:24:11.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.357+0000 7f60fdf45640 1 -- 172.21.15.114:0/3105242104 shutdown_connections 2024-09-13T18:24:11.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.357+0000 7f60fdf45640 1 --2- 172.21.15.114:0/3105242104 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f60d80a1280 0x7f60d80a3740 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:11.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.357+0000 7f60fdf45640 1 --2- 172.21.15.114:0/3105242104 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60f810c400 0x7f60f819ff80 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:11.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.357+0000 7f60fdf45640 1 --2- 172.21.15.114:0/3105242104 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60f8106cc0 0x7f60f819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:11.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.357+0000 7f60fdf45640 1 --2- 172.21.15.114:0/3105242104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60f8106340 0x7f60f819c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:11.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.357+0000 7f60fdf45640 1 -- 172.21.15.114:0/3105242104 >> 172.21.15.114:0/3105242104 conn(0x7f60f8075c80 msgr2=0x7f60f80775b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:11.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.357+0000 7f60fdf45640 1 -- 172.21.15.114:0/3105242104 shutdown_connections 2024-09-13T18:24:11.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.357+0000 7f60fdf45640 1 -- 172.21.15.114:0/3105242104 wait complete. 2024-09-13T18:24:11.376 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-13T18:24:11.376 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:24:11.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.521+0000 7fb95ffff640 1 -- 172.21.15.114:0/941607247 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb958031770 con 0x7fb9680ff830 2024-09-13T18:24:11.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.522+0000 7fb96f63a640 1 -- 172.21.15.114:0/941607247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9680ff830 msgr2=0x7fb968103870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:11.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.522+0000 7fb96f63a640 1 --2- 172.21.15.114:0/941607247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9680ff830 0x7fb968103870 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb958009900 tx=0x7fb95802a410 comp rx=0 tx=0).stop 2024-09-13T18:24:11.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.522+0000 7fb96f63a640 1 -- 172.21.15.114:0/941607247 shutdown_connections 2024-09-13T18:24:11.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.522+0000 7fb96f63a640 1 --2- 172.21.15.114:0/941607247 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb968103fd0 0x7fb96810bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:11.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.522+0000 7fb96f63a640 1 --2- 172.21.15.114:0/941607247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9680ff830 0x7fb968103870 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:11.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.522+0000 7fb96f63a640 1 --2- 172.21.15.114:0/941607247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9680fee80 0x7fb9680ff260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:11.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.522+0000 7fb96f63a640 1 -- 172.21.15.114:0/941607247 >> 172.21.15.114:0/941607247 conn(0x7fb968075c60 msgr2=0x7fb9680fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:11.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.523+0000 7fb96f63a640 1 -- 172.21.15.114:0/941607247 shutdown_connections 2024-09-13T18:24:11.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.523+0000 7fb96f63a640 1 -- 172.21.15.114:0/941607247 wait complete. 2024-09-13T18:24:11.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.524+0000 7fb96f63a640 1 Processor -- start 2024-09-13T18:24:11.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.524+0000 7fb96f63a640 1 -- start start 2024-09-13T18:24:11.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.524+0000 7fb96f63a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9680fee80 0x7fb968198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:11.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.524+0000 7fb96f63a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb968103fd0 0x7fb968198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:11.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.524+0000 7fb96f63a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb96819cd30 0x7fb96819d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:11.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.524+0000 7fb96f63a640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb96819d720 con 0x7fb968103fd0 2024-09-13T18:24:11.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.524+0000 7fb96f63a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb96819d890 con 0x7fb9680fee80 2024-09-13T18:24:11.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.524+0000 7fb96f63a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb96819da00 con 0x7fb96819cd30 2024-09-13T18:24:11.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.524+0000 7fb96cbd5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb968103fd0 0x7fb968198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:11.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.524+0000 7fb96cbd5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb968103fd0 0x7fb968198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43388/0 (socket says 172.21.15.114:43388) 2024-09-13T18:24:11.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.524+0000 7fb96d3d6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9680fee80 0x7fb968198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:11.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.525+0000 7fb96cbd5640 1 -- 172.21.15.114:0/3590890566 learned_addr learned my addr 172.21.15.114:0/3590890566 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:11.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.525+0000 7fb96dbd7640 1 --2- 172.21.15.114:0/3590890566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb96819cd30 0x7fb96819d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:11.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.525+0000 7fb96d3d6640 1 -- 172.21.15.114:0/3590890566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb96819cd30 msgr2=0x7fb96819d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:11.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.525+0000 7fb96d3d6640 1 --2- 172.21.15.114:0/3590890566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb96819cd30 0x7fb96819d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:11.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.525+0000 7fb96d3d6640 1 -- 172.21.15.114:0/3590890566 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb968103fd0 msgr2=0x7fb968198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:11.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.525+0000 7fb96d3d6640 1 --2- 172.21.15.114:0/3590890566 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb968103fd0 0x7fb968198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:11.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.525+0000 7fb96d3d6640 1 -- 172.21.15.114:0/3590890566 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb958009560 con 0x7fb9680fee80 2024-09-13T18:24:11.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.525+0000 7fb96cbd5640 1 --2- 172.21.15.114:0/3590890566 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb968103fd0 0x7fb968198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:24:11.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.525+0000 7fb96d3d6640 1 --2- 172.21.15.114:0/3590890566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9680fee80 0x7fb968198400 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb96400e9f0 tx=0x7fb96400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:11.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.525+0000 7fb95e7fc640 1 -- 172.21.15.114:0/3590890566 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96400ccc0 con 0x7fb9680fee80 2024-09-13T18:24:11.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.525+0000 7fb95e7fc640 1 -- 172.21.15.114:0/3590890566 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb9640044b0 con 0x7fb9680fee80 2024-09-13T18:24:11.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.525+0000 7fb96f63a640 1 -- 172.21.15.114:0/3590890566 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9681a4fe0 con 0x7fb9680fee80 2024-09-13T18:24:11.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.525+0000 7fb95e7fc640 1 -- 172.21.15.114:0/3590890566 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb964004a00 con 0x7fb9680fee80 2024-09-13T18:24:11.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.525+0000 7fb96f63a640 1 -- 172.21.15.114:0/3590890566 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9681a53b0 con 0x7fb9680fee80 2024-09-13T18:24:11.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.527+0000 7fb96f63a640 1 -- 172.21.15.114:0/3590890566 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb968100350 con 0x7fb9680fee80 2024-09-13T18:24:11.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.528+0000 7fb95e7fc640 1 -- 172.21.15.114:0/3590890566 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fb964004b60 con 0x7fb9680fee80 2024-09-13T18:24:11.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.529+0000 7fb95e7fc640 1 --2- 172.21.15.114:0/3590890566 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fb9540a1240 0x7fb9540a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:11.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.529+0000 7fb96cbd5640 1 --2- 172.21.15.114:0/3590890566 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fb9540a1240 0x7fb9540a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:11.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.529+0000 7fb95e7fc640 1 -- 172.21.15.114:0/3590890566 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fb964014070 con 0x7fb9680fee80 2024-09-13T18:24:11.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.529+0000 7fb96cbd5640 1 --2- 172.21.15.114:0/3590890566 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fb9540a1240 0x7fb9540a3700 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb9681999a0 tx=0x7fb95802a910 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:11.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.533+0000 7fb95e7fc640 1 -- 172.21.15.114:0/3590890566 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fb964096080 con 0x7fb9680fee80 2024-09-13T18:24:11.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.878+0000 7fb96f63a640 1 -- 172.21.15.114:0/3590890566 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb968107210 con 0x7fb9680fee80 2024-09-13T18:24:11.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.878+0000 7fb95e7fc640 1 -- 172.21.15.114:0/3590890566 <== mon.2 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+10937 (secure 0 0 0) 0x7fb9640996a0 con 0x7fb9680fee80 2024-09-13T18:24:11.882 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:24:11.882 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4162817497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4162817497},{"type":"v1","addr":"172.21.15.114:6835","nonce":4162817497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/2435428478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":2435428478},{"type":"v1","addr":"172.21.15.114:6837","nonce":2435428478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T18:20:01.895117+0000","modified":"2024-09-13T18:24:01.324695+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24482,"mds_2":24475,"mds_3":24458,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/2046516451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":2046516451},{"type":"v1","addr":"172.21.15.114:6839","nonce":2046516451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":10,"addr":"172.21.15.114:6841/1956923446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":1956923446},{"type":"v1","addr":"172.21.15.114:6841","nonce":1956923446}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24391":{"gid":24391,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/507301496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":507301496},{"type":"v1","addr":"172.21.15.204:6833","nonce":507301496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4028657219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4028657219},{"type":"v1","addr":"172.21.15.134:6833","nonce":4028657219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/57029119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":57029119},{"type":"v1","addr":"172.21.15.204:6835","nonce":57029119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3068880939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3068880939},{"type":"v1","addr":"172.21.15.204:6837","nonce":3068880939}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"e","rank":3,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.134:6835/1150003275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1150003275},{"type":"v1","addr":"172.21.15.134:6835","nonce":1150003275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/344883912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":344883912},{"type":"v1","addr":"172.21.15.204:6839","nonce":344883912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6837/891692210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":891692210},{"type":"v1","addr":"172.21.15.134:6837","nonce":891692210}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6839/3466309724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3466309724},{"type":"v1","addr":"172.21.15.134:6839","nonce":3466309724}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:24:11.882 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 32 2024-09-13T18:24:11.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.881+0000 7fb96f63a640 1 -- 172.21.15.114:0/3590890566 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fb9540a1240 msgr2=0x7fb9540a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:11.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.881+0000 7fb96f63a640 1 --2- 172.21.15.114:0/3590890566 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fb9540a1240 0x7fb9540a3700 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb9681999a0 tx=0x7fb95802a910 comp rx=0 tx=0).stop 2024-09-13T18:24:11.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.881+0000 7fb96f63a640 1 -- 172.21.15.114:0/3590890566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9680fee80 msgr2=0x7fb968198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:11.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.881+0000 7fb96f63a640 1 --2- 172.21.15.114:0/3590890566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9680fee80 0x7fb968198400 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb96400e9f0 tx=0x7fb96400eeb0 comp rx=0 tx=0).stop 2024-09-13T18:24:11.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.881+0000 7fb96f63a640 1 -- 172.21.15.114:0/3590890566 shutdown_connections 2024-09-13T18:24:11.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.881+0000 7fb96f63a640 1 --2- 172.21.15.114:0/3590890566 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fb9540a1240 0x7fb9540a3700 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:11.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.881+0000 7fb96f63a640 1 --2- 172.21.15.114:0/3590890566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb96819cd30 0x7fb96819d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:11.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.881+0000 7fb96f63a640 1 --2- 172.21.15.114:0/3590890566 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb968103fd0 0x7fb968198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:11.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.881+0000 7fb96f63a640 1 --2- 172.21.15.114:0/3590890566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9680fee80 0x7fb968198400 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:11.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.881+0000 7fb96f63a640 1 -- 172.21.15.114:0/3590890566 >> 172.21.15.114:0/3590890566 conn(0x7fb968075c60 msgr2=0x7fb9681022a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:11.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.881+0000 7fb96f63a640 1 -- 172.21.15.114:0/3590890566 shutdown_connections 2024-09-13T18:24:11.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:11.881+0000 7fb96f63a640 1 -- 172.21.15.114:0/3590890566 wait complete. 2024-09-13T18:24:11.901 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-09-13T18:24:12.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.047+0000 7f46a7a5a640 1 -- 172.21.15.114:0/2422361825 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46a01041d0 msgr2=0x7f46a010bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:12.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.047+0000 7f46a7a5a640 1 --2- 172.21.15.114:0/2422361825 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46a01041d0 0x7f46a010bd20 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f4698009cd0 tx=0x7f469802a8e0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.047+0000 7f46a7a5a640 1 -- 172.21.15.114:0/2422361825 shutdown_connections 2024-09-13T18:24:12.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.047+0000 7f46a7a5a640 1 --2- 172.21.15.114:0/2422361825 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46a01041d0 0x7f46a010bd20 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.047+0000 7f46a7a5a640 1 --2- 172.21.15.114:0/2422361825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46a00fff50 0x7f46a0103a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.047+0000 7f46a7a5a640 1 --2- 172.21.15.114:0/2422361825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46a00ff5a0 0x7f46a00ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.047+0000 7f46a7a5a640 1 -- 172.21.15.114:0/2422361825 >> 172.21.15.114:0/2422361825 conn(0x7f46a00753f0 msgr2=0x7f46a00fc740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:12.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.047+0000 7f46a7a5a640 1 -- 172.21.15.114:0/2422361825 shutdown_connections 2024-09-13T18:24:12.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.047+0000 7f46a7a5a640 1 -- 172.21.15.114:0/2422361825 wait complete. 2024-09-13T18:24:12.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.048+0000 7f46a7a5a640 1 Processor -- start 2024-09-13T18:24:12.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.048+0000 7f46a7a5a640 1 -- start start 2024-09-13T18:24:12.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.048+0000 7f46a7a5a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46a00ff5a0 0x7f46a00fa800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.048+0000 7f46a7a5a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46a01041d0 0x7f46a00fad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.048+0000 7f46a7a5a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46a00fb280 0x7f46a01a4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.048+0000 7f46a7a5a640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a01a4c20 con 0x7f46a00ff5a0 2024-09-13T18:24:12.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.048+0000 7f46a7a5a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a01a4d90 con 0x7f46a00fb280 2024-09-13T18:24:12.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.048+0000 7f46a7a5a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a01a4f50 con 0x7f46a01041d0 2024-09-13T18:24:12.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.049+0000 7f46a57f6640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46a00ff5a0 0x7f46a00fa800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.049+0000 7f46a5ff7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46a00fb280 0x7f46a01a4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.049+0000 7f46a5ff7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46a00fb280 0x7f46a01a4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:51716/0 (socket says 172.21.15.114:51716) 2024-09-13T18:24:12.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.049+0000 7f46a5ff7640 1 -- 172.21.15.114:0/1494692597 learned_addr learned my addr 172.21.15.114:0/1494692597 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:12.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.049+0000 7f46a4ff5640 1 --2- 172.21.15.114:0/1494692597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46a01041d0 0x7f46a00fad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.049+0000 7f46a5ff7640 1 -- 172.21.15.114:0/1494692597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46a01041d0 msgr2=0x7f46a00fad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:12.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.049+0000 7f46a5ff7640 1 --2- 172.21.15.114:0/1494692597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46a01041d0 0x7f46a00fad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.049+0000 7f46a5ff7640 1 -- 172.21.15.114:0/1494692597 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46a00ff5a0 msgr2=0x7f46a00fa800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:12.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.049+0000 7f46a5ff7640 1 --2- 172.21.15.114:0/1494692597 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46a00ff5a0 0x7f46a00fa800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.049+0000 7f46a5ff7640 1 -- 172.21.15.114:0/1494692597 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4698009900 con 0x7f46a00fb280 2024-09-13T18:24:12.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.049+0000 7f46a5ff7640 1 --2- 172.21.15.114:0/1494692597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46a00fb280 0x7f46a01a4650 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4698005e30 tx=0x7f4698030dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:12.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.050+0000 7f46927fc640 1 -- 172.21.15.114:0/1494692597 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4698031970 con 0x7f46a00fb280 2024-09-13T18:24:12.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.050+0000 7f46a7a5a640 1 -- 172.21.15.114:0/1494692597 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46a01a51d0 con 0x7f46a00fb280 2024-09-13T18:24:12.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.050+0000 7f46927fc640 1 -- 172.21.15.114:0/1494692597 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4698003e00 con 0x7f46a00fb280 2024-09-13T18:24:12.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.050+0000 7f46927fc640 1 -- 172.21.15.114:0/1494692597 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4698004760 con 0x7f46a00fb280 2024-09-13T18:24:12.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.050+0000 7f46a7a5a640 1 -- 172.21.15.114:0/1494692597 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46a01a5730 con 0x7f46a00fb280 2024-09-13T18:24:12.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.052+0000 7f46927fc640 1 -- 172.21.15.114:0/1494692597 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f469800a040 con 0x7f46a00fb280 2024-09-13T18:24:12.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.053+0000 7f46aa85c740 1 Processor -- start 2024-09-13T18:24:12.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.053+0000 7f46aa85c740 1 -- start start 2024-09-13T18:24:12.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.053+0000 7f46927fc640 1 --2- 172.21.15.114:0/1494692597 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f468c0a1280 0x7f468c0a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.054+0000 7f46a4ff5640 1 --2- 172.21.15.114:0/1494692597 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f468c0a1280 0x7f468c0a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.054+0000 7f46aa85c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f05be5790 0x561f05be5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.054+0000 7f46aa85c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f05be60b0 0x561f05bf0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.054+0000 7f46927fc640 1 -- 172.21.15.114:0/1494692597 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f46980ee370 con 0x7f46a00fb280 2024-09-13T18:24:12.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.054+0000 7f46aa85c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f05bf0950 0x561f05bf2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.054+0000 7f46aa85c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f05b4fcc0 con 0x561f05bf0950 2024-09-13T18:24:12.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.054+0000 7f46aa85c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f05b4d450 con 0x561f05be5790 2024-09-13T18:24:12.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.054+0000 7f46aa85c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f05b48d40 con 0x561f05be60b0 2024-09-13T18:24:12.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.054+0000 7f46a57f6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f05be5790 0x561f05be5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.054+0000 7f46a5ff7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f05bf0950 0x561f05bf2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.054+0000 7f46a57f6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f05be5790 0x561f05be5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:51724/0 (socket says 172.21.15.114:51724) 2024-09-13T18:24:12.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.054+0000 7f46a57f6640 1 -- 172.21.15.114:0/4246308196 learned_addr learned my addr 172.21.15.114:0/4246308196 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:12.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.054+0000 7f46a4ff5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f05be60b0 0x561f05bf0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.054+0000 7f46a4ff5640 1 --2- 172.21.15.114:0/1494692597 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f468c0a1280 0x7f468c0a3740 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4694005e00 tx=0x7f469400bf40 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:12.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.054+0000 7f46a5ff7640 1 -- 172.21.15.114:0/4246308196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f05be60b0 msgr2=0x561f05bf0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:12.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.054+0000 7f46a5ff7640 1 --2- 172.21.15.114:0/4246308196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f05be60b0 0x561f05bf0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.054+0000 7f46a5ff7640 1 -- 172.21.15.114:0/4246308196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f05be5790 msgr2=0x561f05be5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:12.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.054+0000 7f46a5ff7640 1 --2- 172.21.15.114:0/4246308196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f05be5790 0x561f05be5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.054+0000 7f46a5ff7640 1 -- 172.21.15.114:0/4246308196 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f05b10620 con 0x561f05bf0950 2024-09-13T18:24:12.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.055+0000 7f46a5ff7640 1 --2- 172.21.15.114:0/4246308196 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f05bf0950 0x561f05bf2d40 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f46980eb770 tx=0x7f46980bab60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1e251169aedadd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:12.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.055+0000 7f468b7fe640 1 -- 172.21.15.114:0/4246308196 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46980bef60 con 0x561f05bf0950 2024-09-13T18:24:12.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.055+0000 7f468b7fe640 1 -- 172.21.15.114:0/4246308196 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f46980bd070 con 0x561f05bf0950 2024-09-13T18:24:12.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.055+0000 7f468b7fe640 1 -- 172.21.15.114:0/4246308196 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46980ee070 con 0x561f05bf0950 2024-09-13T18:24:12.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.056+0000 7f46aa85c740 1 -- 172.21.15.114:0/4246308196 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f05bf0950 msgr2=0x561f05bf2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:12.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.056+0000 7f46aa85c740 1 --2- 172.21.15.114:0/4246308196 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f05bf0950 0x561f05bf2d40 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f46980eb770 tx=0x7f46980bab60 comp rx=0 tx=0).stop 2024-09-13T18:24:12.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.056+0000 7f46aa85c740 1 -- 172.21.15.114:0/4246308196 shutdown_connections 2024-09-13T18:24:12.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.056+0000 7f46aa85c740 1 --2- 172.21.15.114:0/4246308196 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f05bf0950 0x561f05bf2d40 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.056+0000 7f46aa85c740 1 --2- 172.21.15.114:0/4246308196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f05be60b0 0x561f05bf0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.056+0000 7f46aa85c740 1 --2- 172.21.15.114:0/4246308196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f05be5790 0x561f05be5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.056+0000 7f46aa85c740 1 -- 172.21.15.114:0/4246308196 >> 172.21.15.114:0/4246308196 conn(0x561f05be1d00 msgr2=0x561f05be20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:12.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.056+0000 7f46aa85c740 1 -- 172.21.15.114:0/4246308196 shutdown_connections 2024-09-13T18:24:12.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.056+0000 7f46aa85c740 1 -- 172.21.15.114:0/4246308196 wait complete. 2024-09-13T18:24:12.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.056+0000 7f46aa85c740 1 -- start start 2024-09-13T18:24:12.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.056+0000 7f46aa85c740 1 Processor -- start 2024-09-13T18:24:12.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.057+0000 7f46aa85c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f05be5790 0x561f05beff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.057+0000 7f46aa85c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f05be60b0 0x561f05bed340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.057+0000 7f46aa85c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f05bf0950 0x561f05bed880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.057+0000 7f46aa85c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f05b07180 con 0x561f05be60b0 2024-09-13T18:24:12.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.057+0000 7f46a4ff5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f05be60b0 0x561f05bed340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.057+0000 7f46a57f6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f05be5790 0x561f05beff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.057+0000 7f46a4ff5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f05be60b0 0x561f05bed340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43430/0 (socket says 172.21.15.114:43430) 2024-09-13T18:24:12.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.057+0000 7f46a57f6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f05be5790 0x561f05beff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:43430/0 (socket says 172.21.15.114:43430) 2024-09-13T18:24:12.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.057+0000 7f46a4ff5640 1 -- 172.21.15.114:0/2671277407 learned_addr learned my addr 172.21.15.114:0/2671277407 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:12.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.058+0000 7f46aa85c740 1 -- 172.21.15.114:0/2671277407 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f05821430 con 0x561f05bf0950 2024-09-13T18:24:12.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.058+0000 7f46aa85c740 1 -- 172.21.15.114:0/2671277407 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f05acc5e0 con 0x561f05be5790 2024-09-13T18:24:12.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.058+0000 7f46aa85c740 10 client.0 _ll_register_callbacks cb 0x561f05b38aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:24:12.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.058+0000 7f46a5ff7640 1 --2- 172.21.15.114:0/2671277407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f05bf0950 0x561f05bed880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.058+0000 7f46a57f6640 1 -- 172.21.15.114:0/2671277407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f05bf0950 msgr2=0x561f05bed880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:12.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.058+0000 7f46a57f6640 1 --2- 172.21.15.114:0/2671277407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f05bf0950 0x561f05bed880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.058+0000 7f46a57f6640 1 -- 172.21.15.114:0/2671277407 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f05be60b0 msgr2=0x561f05bed340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:12.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.058+0000 7f46a57f6640 1 --2- 172.21.15.114:0/2671277407 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f05be60b0 0x561f05bed340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.058+0000 7f46a57f6640 1 -- 172.21.15.114:0/2671277407 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46a01a5730 con 0x561f05be5790 2024-09-13T18:24:12.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.058+0000 7f46a4ff5640 1 --2- 172.21.15.114:0/2671277407 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f05be60b0 0x561f05bed340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:24:12.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.058+0000 7f46a5ff7640 1 --2- 172.21.15.114:0/2671277407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f05bf0950 0x561f05bed880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:24:12.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.058+0000 7f46a57f6640 1 --2- 172.21.15.114:0/2671277407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f05be5790 0x561f05beff50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f469c005880 tx=0x7f469c00be00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64403c615978c61d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:12.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.058+0000 7f4688ff9640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:24:12.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.059+0000 7f4688ff9640 1 -- 172.21.15.114:0/2671277407 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469c016070 con 0x561f05be5790 2024-09-13T18:24:12.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.059+0000 7f4688ff9640 1 -- 172.21.15.114:0/2671277407 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f469c003e20 con 0x561f05be5790 2024-09-13T18:24:12.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.059+0000 7f46aa85c740 1 -- 172.21.15.114:0/2671277407 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561f05b72b80 con 0x561f05be5790 2024-09-13T18:24:12.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.059+0000 7f4688ff9640 1 -- 172.21.15.114:0/2671277407 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f469c005020 con 0x561f05be5790 2024-09-13T18:24:12.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.059+0000 7f46aa85c740 10 client.25666 fetch_fsmap learned FSMap version 32 2024-09-13T18:24:12.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.059+0000 7f46aa85c740 1 -- 172.21.15.114:0/2671277407 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x561f05b473c0 con 0x561f05be5790 2024-09-13T18:24:12.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.060+0000 7f4688ff9640 1 -- 172.21.15.114:0/2671277407 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561f05b72b80 con 0x561f05be5790 2024-09-13T18:24:12.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.060+0000 7f4688ff9640 1 -- 172.21.15.114:0/2671277407 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f469c01b470 con 0x561f05be5790 2024-09-13T18:24:12.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.060+0000 7f46aa85c740 10 client.25666 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:24:12.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.060+0000 7f46aa85c740 10 client.25666 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-13T18:24:12.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.060+0000 7f46aa85c740 20 client.25666 populate_metadata read hostname 'smithi114' 2024-09-13T18:24:12.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.060+0000 7f46aa85c740 1 --2- 172.21.15.114:0/2671277407 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x561f05c000a0 0x561f05c024c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.060+0000 7f46aa85c740 4 client.25666 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:24:12.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.060+0000 7f46aa85c740 1 -- 172.21.15.114:0/2671277407 --> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561f05b3b1f0 con 0x561f05c000a0 2024-09-13T18:24:12.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.060+0000 7f46a4ff5640 1 --2- 172.21.15.114:0/2671277407 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x561f05c000a0 0x561f05c024c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.061+0000 7f4688ff9640 10 client.25666 ms_handle_connect on v2:172.21.15.114:6840/1956923446 2024-09-13T18:24:12.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.061+0000 7f46a4ff5640 1 --2- 172.21.15.114:0/2671277407 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x561f05c000a0 0x561f05c024c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=815fa9d0598ed65f server_cookie=db72920d0fde5058 in_seq=0 out_seq=0 2024-09-13T18:24:12.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.062+0000 7f4688ff9640 1 -- 172.21.15.114:0/2671277407 <== mds.4 v2:172.21.15.114:6840/1956923446 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x561f05b3b1f0 con 0x561f05c000a0 2024-09-13T18:24:12.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.062+0000 7f4688ff9640 10 client.25666 handle_command_reply: tid=0 2024-09-13T18:24:12.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.062+0000 7f46aa85c740 1 client.25666 shutdown 2024-09-13T18:24:12.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.063+0000 7f46aa85c740 1 -- 172.21.15.114:0/2671277407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f05be5790 msgr2=0x561f05beff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:12.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.063+0000 7f46aa85c740 1 --2- 172.21.15.114:0/2671277407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f05be5790 0x561f05beff50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f469c005880 tx=0x7f469c00be00 comp rx=0 tx=0).stop 2024-09-13T18:24:12.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.063+0000 7f46aa85c740 1 -- 172.21.15.114:0/2671277407 shutdown_connections 2024-09-13T18:24:12.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.063+0000 7f46aa85c740 1 --2- 172.21.15.114:0/2671277407 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x561f05c000a0 0x561f05c024c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.063+0000 7f46aa85c740 1 --2- 172.21.15.114:0/2671277407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f05bf0950 0x561f05bed880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.063+0000 7f46aa85c740 1 --2- 172.21.15.114:0/2671277407 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f05be60b0 0x561f05bed340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.063+0000 7f46aa85c740 1 --2- 172.21.15.114:0/2671277407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f05be5790 0x561f05beff50 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.063+0000 7f46aa85c740 1 -- 172.21.15.114:0/2671277407 >> 172.21.15.114:0/2671277407 conn(0x561f05be1d00 msgr2=0x561f05be0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:12.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.063+0000 7f4688ff9640 0 client.25666 ms_handle_reset on v2:172.21.15.114:6840/1956923446 2024-09-13T18:24:12.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.063+0000 7f46aa85c740 1 -- 172.21.15.114:0/2671277407 shutdown_connections 2024-09-13T18:24:12.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.063+0000 7f46aa85c740 1 -- 172.21.15.114:0/2671277407 wait complete. 2024-09-13T18:24:12.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.064+0000 7f46aa85c740 20 client.25666 trim_cache size 0 max 16384 2024-09-13T18:24:12.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.080+0000 7f46aa85c740 1 Processor -- start 2024-09-13T18:24:12.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.080+0000 7f46aa85c740 1 -- start start 2024-09-13T18:24:12.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.080+0000 7f46aa85c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f05be5790 0x561f05be16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.080+0000 7f46aa85c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f05be60b0 0x561f05bf73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.080+0000 7f46aa85c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f05bf0950 0x561f05bf7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.080+0000 7f46aa85c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f05b72b80 con 0x561f05bf0950 2024-09-13T18:24:12.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.080+0000 7f46aa85c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f05bfe930 con 0x561f05be5790 2024-09-13T18:24:12.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.080+0000 7f46aa85c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f05be6750 con 0x561f05be60b0 2024-09-13T18:24:12.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.081+0000 7f46a5ff7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f05bf0950 0x561f05bf7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.081+0000 7f46a5ff7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f05bf0950 0x561f05bf7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43434/0 (socket says 172.21.15.114:43434) 2024-09-13T18:24:12.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.081+0000 7f46a5ff7640 1 -- 172.21.15.114:0/451129185 learned_addr learned my addr 172.21.15.114:0/451129185 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:12.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.081+0000 7f46a57f6640 1 --2- 172.21.15.114:0/451129185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f05be5790 0x561f05be16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.081+0000 7f46a4ff5640 1 --2- 172.21.15.114:0/451129185 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f05be60b0 0x561f05bf73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.081+0000 7f46a5ff7640 1 -- 172.21.15.114:0/451129185 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f05be60b0 msgr2=0x561f05bf73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:12.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.081+0000 7f46a5ff7640 1 --2- 172.21.15.114:0/451129185 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f05be60b0 0x561f05bf73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.081+0000 7f46a5ff7640 1 -- 172.21.15.114:0/451129185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f05be5790 msgr2=0x561f05be16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:12.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.081+0000 7f46a5ff7640 1 --2- 172.21.15.114:0/451129185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f05be5790 0x561f05be16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.081+0000 7f46a5ff7640 1 -- 172.21.15.114:0/451129185 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f05ae9770 con 0x561f05bf0950 2024-09-13T18:24:12.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.081+0000 7f46a57f6640 1 --2- 172.21.15.114:0/451129185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f05be5790 0x561f05be16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:24:12.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.081+0000 7f46a5ff7640 1 --2- 172.21.15.114:0/451129185 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f05bf0950 0x561f05bf7910 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f46980b7630 tx=0x7f46980bb030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9fed049e841837d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:12.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.081+0000 7f468bfff640 1 -- 172.21.15.114:0/451129185 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46980bef60 con 0x561f05bf0950 2024-09-13T18:24:12.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.081+0000 7f468bfff640 1 -- 172.21.15.114:0/451129185 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f46980c35b0 con 0x561f05bf0950 2024-09-13T18:24:12.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.081+0000 7f468bfff640 1 -- 172.21.15.114:0/451129185 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46980b95a0 con 0x561f05bf0950 2024-09-13T18:24:12.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.082+0000 7f46aa85c740 1 -- 172.21.15.114:0/451129185 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f05bf0950 msgr2=0x561f05bf7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:12.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.082+0000 7f46aa85c740 1 --2- 172.21.15.114:0/451129185 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f05bf0950 0x561f05bf7910 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f46980b7630 tx=0x7f46980bb030 comp rx=0 tx=0).stop 2024-09-13T18:24:12.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.082+0000 7f46aa85c740 1 -- 172.21.15.114:0/451129185 shutdown_connections 2024-09-13T18:24:12.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.082+0000 7f46aa85c740 1 --2- 172.21.15.114:0/451129185 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f05bf0950 0x561f05bf7910 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.082+0000 7f46aa85c740 1 --2- 172.21.15.114:0/451129185 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f05be60b0 0x561f05bf73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.082+0000 7f46aa85c740 1 --2- 172.21.15.114:0/451129185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f05be5790 0x561f05be16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.082+0000 7f46aa85c740 1 -- 172.21.15.114:0/451129185 >> 172.21.15.114:0/451129185 conn(0x561f05be1d00 msgr2=0x561f05be20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:12.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.082+0000 7f46aa85c740 1 -- 172.21.15.114:0/451129185 shutdown_connections 2024-09-13T18:24:12.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.082+0000 7f46aa85c740 1 -- 172.21.15.114:0/451129185 wait complete. 2024-09-13T18:24:12.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.083+0000 7f46aa85c740 1 -- start start 2024-09-13T18:24:12.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.083+0000 7f46aa85c740 1 Processor -- start 2024-09-13T18:24:12.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.083+0000 7f46aa85c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f05be5790 0x561f05bf33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.083+0000 7f46aa85c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f05be60b0 0x561f05bf38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.083+0000 7f46aa85c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f05bf0950 0x561f05c555d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.084+0000 7f46aa85c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f05bf4110 con 0x561f05be5790 2024-09-13T18:24:12.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.084+0000 7f46aa85c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f05bf4280 con 0x561f05bf0950 2024-09-13T18:24:12.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.084+0000 7f46aa85c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f05c55b60 con 0x561f05be60b0 2024-09-13T18:24:12.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.084+0000 7f46aa85c740 10 client.0 _ll_register_callbacks cb 0x7f46980b7470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:24:12.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.084+0000 7f46a57f6640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f05be5790 0x561f05bf33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.084+0000 7f46a57f6640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f05be5790 0x561f05bf33b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43446/0 (socket says 172.21.15.114:43446) 2024-09-13T18:24:12.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.084+0000 7f46a57f6640 1 -- 172.21.15.114:0/3066397122 learned_addr learned my addr 172.21.15.114:0/3066397122 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:12.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.084+0000 7f46a5ff7640 1 --2- 172.21.15.114:0/3066397122 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f05bf0950 0x561f05c555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.084+0000 7f46a57f6640 1 -- 172.21.15.114:0/3066397122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f05be60b0 msgr2=0x561f05bf38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:12.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.084+0000 7f46a57f6640 1 --2- 172.21.15.114:0/3066397122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f05be60b0 0x561f05bf38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.084+0000 7f46a57f6640 1 -- 172.21.15.114:0/3066397122 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f05bf0950 msgr2=0x561f05c555d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:12.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.084+0000 7f46a57f6640 1 --2- 172.21.15.114:0/3066397122 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f05bf0950 0x561f05c555d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.084+0000 7f46a57f6640 1 -- 172.21.15.114:0/3066397122 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561f05b473c0 con 0x561f05be5790 2024-09-13T18:24:12.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.084+0000 7f46a57f6640 1 --2- 172.21.15.114:0/3066397122 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f05be5790 0x561f05bf33b0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f469c005880 tx=0x7f469c003e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb72320e3ea30338 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:12.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.084+0000 7f4688ff9640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:24:12.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.085+0000 7f4688ff9640 1 -- 172.21.15.114:0/3066397122 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469c016040 con 0x561f05be5790 2024-09-13T18:24:12.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.085+0000 7f4688ff9640 1 -- 172.21.15.114:0/3066397122 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f469c000e40 con 0x561f05be5790 2024-09-13T18:24:12.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.085+0000 7f46aa85c740 1 -- 172.21.15.114:0/3066397122 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f46980bd070 con 0x561f05be5790 2024-09-13T18:24:12.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.085+0000 7f4688ff9640 1 -- 172.21.15.114:0/3066397122 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f469c01bc00 con 0x561f05be5790 2024-09-13T18:24:12.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.085+0000 7f46aa85c740 10 client.16290 fetch_fsmap learned FSMap version 32 2024-09-13T18:24:12.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.085+0000 7f46aa85c740 1 -- 172.21.15.114:0/3066397122 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x561f05b10620 con 0x561f05be5790 2024-09-13T18:24:12.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.086+0000 7f4688ff9640 1 -- 172.21.15.114:0/3066397122 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f469c011ad0 con 0x561f05be5790 2024-09-13T18:24:12.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.086+0000 7f4688ff9640 1 -- 172.21.15.114:0/3066397122 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f469c029900 con 0x561f05be5790 2024-09-13T18:24:12.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.086+0000 7f46aa85c740 10 client.16290 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:24:12.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.086+0000 7f46aa85c740 10 client.16290 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-13T18:24:12.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.086+0000 7f46aa85c740 20 client.16290 populate_metadata read hostname 'smithi114' 2024-09-13T18:24:12.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.086+0000 7f46aa85c740 1 --2- 172.21.15.114:0/3066397122 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x561f05c572c0 0x561f05c596e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.086+0000 7f46aa85c740 4 client.16290 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:24:12.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.086+0000 7f46aa85c740 1 -- 172.21.15.114:0/3066397122 --> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f46980bd070 con 0x561f05c572c0 2024-09-13T18:24:12.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.086+0000 7f46a4ff5640 1 --2- 172.21.15.114:0/3066397122 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x561f05c572c0 0x561f05c596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.086+0000 7f4688ff9640 10 client.16290 ms_handle_connect on v2:172.21.15.114:6840/1956923446 2024-09-13T18:24:12.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.087+0000 7f46a4ff5640 1 --2- 172.21.15.114:0/3066397122 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x561f05c572c0 0x561f05c596e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8e834b26616dde3 server_cookie=8ef0bcef02474840 in_seq=0 out_seq=0 2024-09-13T18:24:12.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.087+0000 7f4688ff9640 1 -- 172.21.15.114:0/3066397122 <== mds.4 v2:172.21.15.114:6840/1956923446 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f46980bd070 con 0x561f05c572c0 2024-09-13T18:24:12.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.087+0000 7f4688ff9640 10 client.16290 handle_command_reply: tid=0 2024-09-13T18:24:12.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.087+0000 7f46aa85c740 1 client.16290 shutdown 2024-09-13T18:24:12.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.087+0000 7f46aa85c740 1 -- 172.21.15.114:0/3066397122 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f05be5790 msgr2=0x561f05bf33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:12.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.087+0000 7f46aa85c740 1 --2- 172.21.15.114:0/3066397122 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f05be5790 0x561f05bf33b0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f469c005880 tx=0x7f469c003e20 comp rx=0 tx=0).stop 2024-09-13T18:24:12.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.087+0000 7f46aa85c740 1 -- 172.21.15.114:0/3066397122 shutdown_connections 2024-09-13T18:24:12.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.087+0000 7f46aa85c740 1 --2- 172.21.15.114:0/3066397122 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x561f05c572c0 0x561f05c596e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.087+0000 7f46aa85c740 1 --2- 172.21.15.114:0/3066397122 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f05bf0950 0x561f05c555d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.087+0000 7f46aa85c740 1 --2- 172.21.15.114:0/3066397122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561f05be60b0 0x561f05bf38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.087+0000 7f4688ff9640 0 client.16290 ms_handle_reset on v2:172.21.15.114:6840/1956923446 2024-09-13T18:24:12.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.087+0000 7f46aa85c740 1 --2- 172.21.15.114:0/3066397122 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f05be5790 0x561f05bf33b0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.087+0000 7f46aa85c740 1 -- 172.21.15.114:0/3066397122 >> 172.21.15.114:0/3066397122 conn(0x561f05be1d00 msgr2=0x561f05bebb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:12.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.087+0000 7f46aa85c740 1 -- 172.21.15.114:0/3066397122 shutdown_connections 2024-09-13T18:24:12.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.087+0000 7f46aa85c740 1 -- 172.21.15.114:0/3066397122 wait complete. 2024-09-13T18:24:12.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.088+0000 7f46aa85c740 20 client.16290 trim_cache size 0 max 16384 2024-09-13T18:24:12.096 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:24:12.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.088+0000 7f46a7a5a640 1 -- 172.21.15.114:0/1494692597 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f468c0a1280 msgr2=0x7f468c0a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:12.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.088+0000 7f46a7a5a640 1 --2- 172.21.15.114:0/1494692597 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f468c0a1280 0x7f468c0a3740 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4694005e00 tx=0x7f469400bf40 comp rx=0 tx=0).stop 2024-09-13T18:24:12.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.088+0000 7f46a7a5a640 1 -- 172.21.15.114:0/1494692597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46a00fb280 msgr2=0x7f46a01a4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:12.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.088+0000 7f46a7a5a640 1 --2- 172.21.15.114:0/1494692597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46a00fb280 0x7f46a01a4650 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4698005e30 tx=0x7f4698030dd0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.088+0000 7f46a7a5a640 1 -- 172.21.15.114:0/1494692597 shutdown_connections 2024-09-13T18:24:12.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.088+0000 7f46a7a5a640 1 --2- 172.21.15.114:0/1494692597 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f468c0a1280 0x7f468c0a3740 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.088+0000 7f46a7a5a640 1 --2- 172.21.15.114:0/1494692597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46a00fb280 0x7f46a01a4650 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.088+0000 7f46a7a5a640 1 --2- 172.21.15.114:0/1494692597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46a01041d0 0x7f46a00fad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.088+0000 7f46a7a5a640 1 --2- 172.21.15.114:0/1494692597 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46a00ff5a0 0x7f46a00fa800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.088+0000 7f46a7a5a640 1 -- 172.21.15.114:0/1494692597 >> 172.21.15.114:0/1494692597 conn(0x7f46a00753f0 msgr2=0x7f46a00fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:12.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.088+0000 7f46a7a5a640 1 -- 172.21.15.114:0/1494692597 shutdown_connections 2024-09-13T18:24:12.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.088+0000 7f46a7a5a640 1 -- 172.21.15.114:0/1494692597 wait complete. 2024-09-13T18:24:12.105 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-09-13T18:24:12.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:11 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/3590890566' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:24:12.219 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:11 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/3590890566' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:24:12.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.250+0000 7f0ab9b01640 1 -- 172.21.15.114:0/192073963 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ab40ff5a0 msgr2=0x7f0ab40ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:12.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.250+0000 7f0ab9b01640 1 --2- 172.21.15.114:0/192073963 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ab40ff5a0 0x7f0ab40ff980 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4009930 tx=0x7f0aa402a420 comp rx=0 tx=0).stop 2024-09-13T18:24:12.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.250+0000 7f0ab9b01640 1 -- 172.21.15.114:0/192073963 shutdown_connections 2024-09-13T18:24:12.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.250+0000 7f0ab9b01640 1 --2- 172.21.15.114:0/192073963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ab41041d0 0x7f0ab410bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.250+0000 7f0ab9b01640 1 --2- 172.21.15.114:0/192073963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ab40fff50 0x7f0ab4103a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.250+0000 7f0ab9b01640 1 --2- 172.21.15.114:0/192073963 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ab40ff5a0 0x7f0ab40ff980 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.250+0000 7f0ab9b01640 1 -- 172.21.15.114:0/192073963 >> 172.21.15.114:0/192073963 conn(0x7f0ab40753f0 msgr2=0x7f0ab40fc740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:12.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.250+0000 7f0ab9b01640 1 -- 172.21.15.114:0/192073963 shutdown_connections 2024-09-13T18:24:12.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.250+0000 7f0ab9b01640 1 -- 172.21.15.114:0/192073963 wait complete. 2024-09-13T18:24:12.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.250+0000 7f0ab9b01640 1 Processor -- start 2024-09-13T18:24:12.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.251+0000 7f0ab9b01640 1 -- start start 2024-09-13T18:24:12.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.251+0000 7f0ab9b01640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ab40ff5a0 0x7f0ab4196480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.251+0000 7f0ab9b01640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ab41041d0 0x7f0ab41969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.251+0000 7f0ab9b01640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ab419adb0 0x7f0ab419b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.251+0000 7f0ab9b01640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab419b7a0 con 0x7f0ab419adb0 2024-09-13T18:24:12.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.251+0000 7f0ab9b01640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab419b910 con 0x7f0ab40ff5a0 2024-09-13T18:24:12.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.251+0000 7f0ab9b01640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab419ba80 con 0x7f0ab41041d0 2024-09-13T18:24:12.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.251+0000 7f0ab3fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ab419adb0 0x7f0ab419b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.251+0000 7f0ab2ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ab41041d0 0x7f0ab41969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.251+0000 7f0ab3fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ab419adb0 0x7f0ab419b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43458/0 (socket says 172.21.15.114:43458) 2024-09-13T18:24:12.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.251+0000 7f0ab3fff640 1 -- 172.21.15.114:0/1927246309 learned_addr learned my addr 172.21.15.114:0/1927246309 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:12.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.251+0000 7f0ab37fe640 1 --2- 172.21.15.114:0/1927246309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ab40ff5a0 0x7f0ab4196480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.252+0000 7f0ab3fff640 1 -- 172.21.15.114:0/1927246309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ab41041d0 msgr2=0x7f0ab41969c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:12.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.252+0000 7f0ab3fff640 1 --2- 172.21.15.114:0/1927246309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ab41041d0 0x7f0ab41969c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.252+0000 7f0ab3fff640 1 -- 172.21.15.114:0/1927246309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ab40ff5a0 msgr2=0x7f0ab4196480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:12.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.252+0000 7f0ab3fff640 1 --2- 172.21.15.114:0/1927246309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ab40ff5a0 0x7f0ab4196480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.252+0000 7f0ab3fff640 1 -- 172.21.15.114:0/1927246309 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa4009560 con 0x7f0ab419adb0 2024-09-13T18:24:12.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.252+0000 7f0ab2ffd640 1 --2- 172.21.15.114:0/1927246309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ab41041d0 0x7f0ab41969c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:24:12.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.252+0000 7f0ab37fe640 1 --2- 172.21.15.114:0/1927246309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ab40ff5a0 0x7f0ab4196480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:24:12.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.252+0000 7f0ab3fff640 1 --2- 172.21.15.114:0/1927246309 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ab419adb0 0x7f0ab419b260 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f0aac00ea60 tx=0x7f0aac00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:12.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.252+0000 7f0ab0ff9640 1 -- 172.21.15.114:0/1927246309 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aac00cd80 con 0x7f0ab419adb0 2024-09-13T18:24:12.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.252+0000 7f0ab9b01640 1 -- 172.21.15.114:0/1927246309 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ab41a3060 con 0x7f0ab419adb0 2024-09-13T18:24:12.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.252+0000 7f0ab0ff9640 1 -- 172.21.15.114:0/1927246309 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0aac005020 con 0x7f0ab419adb0 2024-09-13T18:24:12.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.252+0000 7f0ab0ff9640 1 -- 172.21.15.114:0/1927246309 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aac010690 con 0x7f0ab419adb0 2024-09-13T18:24:12.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.252+0000 7f0ab9b01640 1 -- 172.21.15.114:0/1927246309 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ab41a3510 con 0x7f0ab419adb0 2024-09-13T18:24:12.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.255+0000 7f0abc81e740 1 Processor -- start 2024-09-13T18:24:12.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.255+0000 7f0ab0ff9640 1 -- 172.21.15.114:0/1927246309 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f0aac005190 con 0x7f0ab419adb0 2024-09-13T18:24:12.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.255+0000 7f0abc81e740 1 -- start start 2024-09-13T18:24:12.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.255+0000 7f0abc81e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb5906140 0x55fbb5904aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.255+0000 7f0abc81e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb5904fe0 0x55fbb5905440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.255+0000 7f0abc81e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbb5913ff0 0x55fbb59163e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.255+0000 7f0abc81e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbb586acc0 con 0x55fbb5906140 2024-09-13T18:24:12.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.255+0000 7f0abc81e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbb5868450 con 0x55fbb5904fe0 2024-09-13T18:24:12.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.255+0000 7f0abc81e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbb5863d40 con 0x55fbb5913ff0 2024-09-13T18:24:12.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.256+0000 7f0ab2ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb5906140 0x55fbb5904aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.256+0000 7f0ab37fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb5904fe0 0x55fbb5905440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.256+0000 7f0ab2ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb5906140 0x55fbb5904aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43468/0 (socket says 172.21.15.114:43468) 2024-09-13T18:24:12.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.256+0000 7f0ab2ffd640 1 -- 172.21.15.114:0/2723032049 learned_addr learned my addr 172.21.15.114:0/2723032049 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:12.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.256+0000 7f0ab2ffd640 1 --2- 172.21.15.114:0/2723032049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbb5913ff0 0x55fbb59163e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.256+0000 7f0ab2ffd640 1 -- 172.21.15.114:0/2723032049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbb5913ff0 msgr2=0x55fbb59163e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:12.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.256+0000 7f0ab2ffd640 1 --2- 172.21.15.114:0/2723032049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbb5913ff0 0x55fbb59163e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.256+0000 7f0ab2ffd640 1 -- 172.21.15.114:0/2723032049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb5904fe0 msgr2=0x55fbb5905440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:12.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.256+0000 7f0ab2ffd640 1 --2- 172.21.15.114:0/2723032049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb5904fe0 0x55fbb5905440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.256+0000 7f0ab2ffd640 1 -- 172.21.15.114:0/2723032049 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fbb582b620 con 0x55fbb5906140 2024-09-13T18:24:12.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.256+0000 7f0ab2ffd640 1 --2- 172.21.15.114:0/2723032049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb5906140 0x55fbb5904aa0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8009330 tx=0x7f0aa8009960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6de48efeb93f4a08 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:12.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.256+0000 7f0a99ffb640 1 -- 172.21.15.114:0/2723032049 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa80107c0 con 0x55fbb5906140 2024-09-13T18:24:12.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.256+0000 7f0a99ffb640 1 -- 172.21.15.114:0/2723032049 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0aa8010db0 con 0x55fbb5906140 2024-09-13T18:24:12.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.256+0000 7f0a99ffb640 1 -- 172.21.15.114:0/2723032049 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa8017d60 con 0x55fbb5906140 2024-09-13T18:24:12.264 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.257+0000 7f0ab0ff9640 1 --2- 172.21.15.114:0/1927246309 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0a9c0a1280 0x7f0a9c0a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.264 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.257+0000 7f0abc81e740 1 -- 172.21.15.114:0/2723032049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb5906140 msgr2=0x55fbb5904aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:12.264 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.257+0000 7f0abc81e740 1 --2- 172.21.15.114:0/2723032049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb5906140 0x55fbb5904aa0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8009330 tx=0x7f0aa8009960 comp rx=0 tx=0).stop 2024-09-13T18:24:12.264 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.257+0000 7f0ab37fe640 1 --2- 172.21.15.114:0/1927246309 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0a9c0a1280 0x7f0a9c0a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.264 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.257+0000 7f0ab0ff9640 1 -- 172.21.15.114:0/1927246309 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f0aac014070 con 0x7f0ab419adb0 2024-09-13T18:24:12.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.258+0000 7f0abc81e740 1 -- 172.21.15.114:0/2723032049 shutdown_connections 2024-09-13T18:24:12.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.258+0000 7f0abc81e740 1 --2- 172.21.15.114:0/2723032049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbb5913ff0 0x55fbb59163e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.258+0000 7f0abc81e740 1 --2- 172.21.15.114:0/2723032049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb5904fe0 0x55fbb5905440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.258+0000 7f0abc81e740 1 --2- 172.21.15.114:0/2723032049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb5906140 0x55fbb5904aa0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.258+0000 7f0abc81e740 1 -- 172.21.15.114:0/2723032049 >> 172.21.15.114:0/2723032049 conn(0x55fbb5901110 msgr2=0x55fbb59014f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:12.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.258+0000 7f0abc81e740 1 -- 172.21.15.114:0/2723032049 shutdown_connections 2024-09-13T18:24:12.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.258+0000 7f0abc81e740 1 -- 172.21.15.114:0/2723032049 wait complete. 2024-09-13T18:24:12.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.258+0000 7f0ab37fe640 1 --2- 172.21.15.114:0/1927246309 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0a9c0a1280 0x7f0a9c0a3740 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0aa40057c0 tx=0x7f0aa4005a20 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:12.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.258+0000 7f0abc81e740 1 -- start start 2024-09-13T18:24:12.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.258+0000 7f0abc81e740 1 Processor -- start 2024-09-13T18:24:12.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.259+0000 7f0abc81e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb5904fe0 0x55fbb58ffdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.259+0000 7f0abc81e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbb5906140 0x55fbb58fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.259+0000 7f0abc81e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb5913ff0 0x55fbb58fd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.259+0000 7f0abc81e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbb5822180 con 0x55fbb5904fe0 2024-09-13T18:24:12.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.259+0000 7f0abc81e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbb553c430 con 0x55fbb5913ff0 2024-09-13T18:24:12.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.259+0000 7f0abc81e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbb57e75e0 con 0x55fbb5906140 2024-09-13T18:24:12.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.259+0000 7f0abc81e740 10 client.0 _ll_register_callbacks cb 0x55fbb5853aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:24:12.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.259+0000 7f0ab37fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb5904fe0 0x55fbb58ffdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.259+0000 7f0ab37fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb5904fe0 0x55fbb58ffdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43476/0 (socket says 172.21.15.114:43476) 2024-09-13T18:24:12.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.259+0000 7f0ab37fe640 1 -- 172.21.15.114:0/3146558638 learned_addr learned my addr 172.21.15.114:0/3146558638 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:12.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.259+0000 7f0ab2ffd640 1 --2- 172.21.15.114:0/3146558638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbb5906140 0x55fbb58fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.259+0000 7f0ab3fff640 1 --2- 172.21.15.114:0/3146558638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb5913ff0 0x55fbb58fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.259+0000 7f0ab37fe640 1 -- 172.21.15.114:0/3146558638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbb5906140 msgr2=0x55fbb58fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:12.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.259+0000 7f0ab37fe640 1 --2- 172.21.15.114:0/3146558638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbb5906140 0x55fbb58fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.259+0000 7f0ab37fe640 1 -- 172.21.15.114:0/3146558638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb5913ff0 msgr2=0x55fbb58fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:12.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.259+0000 7f0ab37fe640 1 --2- 172.21.15.114:0/3146558638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb5913ff0 0x55fbb58fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.259+0000 7f0ab37fe640 1 -- 172.21.15.114:0/3146558638 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aa8015300 con 0x55fbb5904fe0 2024-09-13T18:24:12.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.259+0000 7f0ab3fff640 1 --2- 172.21.15.114:0/3146558638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb5913ff0 0x55fbb58fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:24:12.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.260+0000 7f0ab37fe640 1 --2- 172.21.15.114:0/3146558638 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb5904fe0 0x55fbb58ffdb0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4006010 tx=0x7f0aa4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30085e08aa8a3c86 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:12.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.260+0000 7f0a92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:24:12.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.260+0000 7f0a92ffd640 1 -- 172.21.15.114:0/3146558638 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa4051470 con 0x55fbb5904fe0 2024-09-13T18:24:12.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.260+0000 7f0abc81e740 1 -- 172.21.15.114:0/3146558638 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fbb588db80 con 0x55fbb5904fe0 2024-09-13T18:24:12.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.260+0000 7f0a92ffd640 1 -- 172.21.15.114:0/3146558638 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f0aa40515d0 con 0x55fbb5904fe0 2024-09-13T18:24:12.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.260+0000 7f0a92ffd640 1 -- 172.21.15.114:0/3146558638 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0aa4051790 con 0x55fbb5904fe0 2024-09-13T18:24:12.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.260+0000 7f0abc81e740 10 client.16302 fetch_fsmap learned FSMap version 32 2024-09-13T18:24:12.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.260+0000 7f0abc81e740 1 -- 172.21.15.114:0/3146558638 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55fbb58623c0 con 0x55fbb5904fe0 2024-09-13T18:24:12.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.260+0000 7f0a92ffd640 1 -- 172.21.15.114:0/3146558638 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55fbb588db80 con 0x55fbb5904fe0 2024-09-13T18:24:12.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.261+0000 7f0a92ffd640 1 -- 172.21.15.114:0/3146558638 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f0aa4047850 con 0x55fbb5904fe0 2024-09-13T18:24:12.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.261+0000 7f0abc81e740 10 client.16302 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:24:12.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.261+0000 7f0abc81e740 10 client.16302 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-13T18:24:12.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.261+0000 7f0abc81e740 20 client.16302 populate_metadata read hostname 'smithi114' 2024-09-13T18:24:12.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.261+0000 7f0abc81e740 1 --2- 172.21.15.114:0/3146558638 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55fbb5923b20 0x55fbb5925f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.261+0000 7f0abc81e740 4 client.16302 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:24:12.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.261+0000 7f0abc81e740 1 -- 172.21.15.114:0/3146558638 --> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fbb58561f0 con 0x55fbb5923b20 2024-09-13T18:24:12.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.261+0000 7f0ab2ffd640 1 --2- 172.21.15.114:0/3146558638 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55fbb5923b20 0x55fbb5925f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.261+0000 7f0a92ffd640 10 client.16302 ms_handle_connect on v2:172.21.15.134:6834/1150003275 2024-09-13T18:24:12.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.261+0000 7f0ab2ffd640 1 --2- 172.21.15.114:0/3146558638 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55fbb5923b20 0x55fbb5925f40 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4a6c7499830593cf server_cookie=509ad2b2069356bb in_seq=0 out_seq=0 2024-09-13T18:24:12.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.262+0000 7f0a92ffd640 1 -- 172.21.15.114:0/3146558638 <== mds.3 v2:172.21.15.134:6834/1150003275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55fbb58561f0 con 0x55fbb5923b20 2024-09-13T18:24:12.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.262+0000 7f0a92ffd640 10 client.16302 handle_command_reply: tid=0 2024-09-13T18:24:12.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.262+0000 7f0abc81e740 1 client.16302 shutdown 2024-09-13T18:24:12.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.262+0000 7f0abc81e740 1 -- 172.21.15.114:0/3146558638 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb5904fe0 msgr2=0x55fbb58ffdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:12.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.262+0000 7f0abc81e740 1 --2- 172.21.15.114:0/3146558638 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb5904fe0 0x55fbb58ffdb0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4006010 tx=0x7f0aa4009260 comp rx=0 tx=0).stop 2024-09-13T18:24:12.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.263+0000 7f0abc81e740 1 -- 172.21.15.114:0/3146558638 shutdown_connections 2024-09-13T18:24:12.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.263+0000 7f0abc81e740 1 --2- 172.21.15.114:0/3146558638 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55fbb5923b20 0x55fbb5925f40 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.263+0000 7f0abc81e740 1 --2- 172.21.15.114:0/3146558638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb5913ff0 0x55fbb58fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.263+0000 7f0a92ffd640 0 client.16302 ms_handle_reset on v2:172.21.15.134:6834/1150003275 2024-09-13T18:24:12.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.263+0000 7f0abc81e740 1 --2- 172.21.15.114:0/3146558638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbb5906140 0x55fbb58fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.263+0000 7f0abc81e740 1 --2- 172.21.15.114:0/3146558638 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb5904fe0 0x55fbb58ffdb0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.263+0000 7f0abc81e740 1 -- 172.21.15.114:0/3146558638 >> 172.21.15.114:0/3146558638 conn(0x55fbb5901110 msgr2=0x55fbb59014f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:12.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.263+0000 7f0abc81e740 1 -- 172.21.15.114:0/3146558638 shutdown_connections 2024-09-13T18:24:12.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.263+0000 7f0abc81e740 1 -- 172.21.15.114:0/3146558638 wait complete. 2024-09-13T18:24:12.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.263+0000 7f0abc81e740 20 client.16302 trim_cache size 0 max 16384 2024-09-13T18:24:12.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.279+0000 7f0abc81e740 1 Processor -- start 2024-09-13T18:24:12.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.279+0000 7f0abc81e740 1 -- start start 2024-09-13T18:24:12.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.279+0000 7f0abc81e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb5906140 0x55fbb591ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.279+0000 7f0abc81e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb5913ff0 0x55fbb591b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.279+0000 7f0abc81e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbb58ff850 0x55fbb58ffcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.279+0000 7f0abc81e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbb58fe120 con 0x55fbb5906140 2024-09-13T18:24:12.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.279+0000 7f0abc81e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbb5901d10 con 0x55fbb5913ff0 2024-09-13T18:24:12.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.279+0000 7f0abc81e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbb58fda70 con 0x55fbb58ff850 2024-09-13T18:24:12.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.279+0000 7f0ab37fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb5906140 0x55fbb591ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.280+0000 7f0ab37fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb5906140 0x55fbb591ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43488/0 (socket says 172.21.15.114:43488) 2024-09-13T18:24:12.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.280+0000 7f0ab3fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbb58ff850 0x55fbb58ffcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.280+0000 7f0ab37fe640 1 -- 172.21.15.114:0/2442709167 learned_addr learned my addr 172.21.15.114:0/2442709167 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:12.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.280+0000 7f0ab2ffd640 1 --2- 172.21.15.114:0/2442709167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb5913ff0 0x55fbb591b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.280+0000 7f0ab3fff640 1 -- 172.21.15.114:0/2442709167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb5913ff0 msgr2=0x55fbb591b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:12.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.280+0000 7f0ab3fff640 1 --2- 172.21.15.114:0/2442709167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb5913ff0 0x55fbb591b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.280+0000 7f0ab3fff640 1 -- 172.21.15.114:0/2442709167 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb5906140 msgr2=0x55fbb591ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:12.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.280+0000 7f0ab3fff640 1 --2- 172.21.15.114:0/2442709167 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb5906140 0x55fbb591ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.280+0000 7f0ab3fff640 1 -- 172.21.15.114:0/2442709167 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fbb5804770 con 0x55fbb58ff850 2024-09-13T18:24:12.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.280+0000 7f0ab2ffd640 1 --2- 172.21.15.114:0/2442709167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb5913ff0 0x55fbb591b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:24:12.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.280+0000 7f0ab37fe640 1 --2- 172.21.15.114:0/2442709167 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb5906140 0x55fbb591ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:24:12.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.280+0000 7f0ab3fff640 1 --2- 172.21.15.114:0/2442709167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbb58ff850 0x55fbb58ffcd0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0aac0ca940 tx=0x7f0aac00cee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=397c90d25c1e529d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:12.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.280+0000 7f0a9a7fc640 1 -- 172.21.15.114:0/2442709167 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aac09a300 con 0x55fbb58ff850 2024-09-13T18:24:12.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.281+0000 7f0a9a7fc640 1 -- 172.21.15.114:0/2442709167 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0aac09e130 con 0x55fbb58ff850 2024-09-13T18:24:12.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.281+0000 7f0a9a7fc640 1 -- 172.21.15.114:0/2442709167 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aac0997f0 con 0x55fbb58ff850 2024-09-13T18:24:12.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.281+0000 7f0abc81e740 1 -- 172.21.15.114:0/2442709167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbb58ff850 msgr2=0x55fbb58ffcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:12.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.281+0000 7f0abc81e740 1 --2- 172.21.15.114:0/2442709167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbb58ff850 0x55fbb58ffcd0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0aac0ca940 tx=0x7f0aac00cee0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.281+0000 7f0abc81e740 1 -- 172.21.15.114:0/2442709167 shutdown_connections 2024-09-13T18:24:12.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.281+0000 7f0abc81e740 1 --2- 172.21.15.114:0/2442709167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbb58ff850 0x55fbb58ffcd0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.281+0000 7f0abc81e740 1 --2- 172.21.15.114:0/2442709167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb5913ff0 0x55fbb591b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.281+0000 7f0abc81e740 1 --2- 172.21.15.114:0/2442709167 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb5906140 0x55fbb591ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.281+0000 7f0abc81e740 1 -- 172.21.15.114:0/2442709167 >> 172.21.15.114:0/2442709167 conn(0x55fbb5901110 msgr2=0x55fbb592a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:12.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.282+0000 7f0abc81e740 1 -- 172.21.15.114:0/2442709167 shutdown_connections 2024-09-13T18:24:12.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.282+0000 7f0abc81e740 1 -- 172.21.15.114:0/2442709167 wait complete. 2024-09-13T18:24:12.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.282+0000 7f0abc81e740 1 -- start start 2024-09-13T18:24:12.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.282+0000 7f0abc81e740 1 Processor -- start 2024-09-13T18:24:12.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.283+0000 7f0abc81e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb58ff850 0x55fbb591ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.283+0000 7f0abc81e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb5906140 0x55fbb591b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.283+0000 7f0abc81e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbb5913ff0 0x55fbb5919640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.283+0000 7f0abc81e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbb5919d10 con 0x55fbb5906140 2024-09-13T18:24:12.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.283+0000 7f0abc81e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbb5919e80 con 0x55fbb58ff850 2024-09-13T18:24:12.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.283+0000 7f0abc81e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbb591a040 con 0x55fbb5913ff0 2024-09-13T18:24:12.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.283+0000 7f0abc81e740 10 client.0 _ll_register_callbacks cb 0x7f0aa8009870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:24:12.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.283+0000 7f0ab2ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb5906140 0x55fbb591b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.283+0000 7f0ab2ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb5906140 0x55fbb591b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43502/0 (socket says 172.21.15.114:43502) 2024-09-13T18:24:12.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.283+0000 7f0ab2ffd640 1 -- 172.21.15.114:0/3920696101 learned_addr learned my addr 172.21.15.114:0/3920696101 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:12.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.283+0000 7f0ab3fff640 1 --2- 172.21.15.114:0/3920696101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbb5913ff0 0x55fbb5919640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.283+0000 7f0ab37fe640 1 --2- 172.21.15.114:0/3920696101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb58ff850 0x55fbb591ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.283+0000 7f0ab2ffd640 1 -- 172.21.15.114:0/3920696101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbb5913ff0 msgr2=0x55fbb5919640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:12.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.283+0000 7f0ab2ffd640 1 --2- 172.21.15.114:0/3920696101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbb5913ff0 0x55fbb5919640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.283+0000 7f0ab2ffd640 1 -- 172.21.15.114:0/3920696101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb58ff850 msgr2=0x55fbb591ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:12.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.283+0000 7f0ab2ffd640 1 --2- 172.21.15.114:0/3920696101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb58ff850 0x55fbb591ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.283+0000 7f0ab2ffd640 1 -- 172.21.15.114:0/3920696101 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fbb58623c0 con 0x55fbb5906140 2024-09-13T18:24:12.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.283+0000 7f0ab3fff640 1 --2- 172.21.15.114:0/3920696101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbb5913ff0 0x55fbb5919640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:24:12.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.284+0000 7f0ab2ffd640 1 --2- 172.21.15.114:0/3920696101 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb5906140 0x55fbb591b370 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8015f30 tx=0x7f0aa80047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82f3d4e7207011a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:12.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.284+0000 7f0a92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:24:12.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.284+0000 7f0a92ffd640 1 -- 172.21.15.114:0/3920696101 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa8025040 con 0x55fbb5906140 2024-09-13T18:24:12.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.284+0000 7f0abc81e740 1 -- 172.21.15.114:0/3920696101 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0aa8010db0 con 0x55fbb5906140 2024-09-13T18:24:12.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.284+0000 7f0a92ffd640 1 -- 172.21.15.114:0/3920696101 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f0aa8017ce0 con 0x55fbb5906140 2024-09-13T18:24:12.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.284+0000 7f0a92ffd640 1 -- 172.21.15.114:0/3920696101 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0aa8017ea0 con 0x55fbb5906140 2024-09-13T18:24:12.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.284+0000 7f0abc81e740 10 client.16308 fetch_fsmap learned FSMap version 32 2024-09-13T18:24:12.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.284+0000 7f0abc81e740 1 -- 172.21.15.114:0/3920696101 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55fbb582b620 con 0x55fbb5906140 2024-09-13T18:24:12.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.284+0000 7f0a92ffd640 1 -- 172.21.15.114:0/3920696101 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0aa8010db0 con 0x55fbb5906140 2024-09-13T18:24:12.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.285+0000 7f0a92ffd640 1 -- 172.21.15.114:0/3920696101 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f0aa8034420 con 0x55fbb5906140 2024-09-13T18:24:12.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.285+0000 7f0abc81e740 10 client.16308 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:24:12.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.285+0000 7f0abc81e740 10 client.16308 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-13T18:24:12.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.285+0000 7f0abc81e740 20 client.16308 populate_metadata read hostname 'smithi114' 2024-09-13T18:24:12.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.285+0000 7f0abc81e740 1 --2- 172.21.15.114:0/3920696101 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55fbb597b410 0x55fbb597d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.285+0000 7f0abc81e740 4 client.16308 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:24:12.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.285+0000 7f0abc81e740 1 -- 172.21.15.114:0/3920696101 --> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fbb55574e0 con 0x55fbb597b410 2024-09-13T18:24:12.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.285+0000 7f0ab37fe640 1 --2- 172.21.15.114:0/3920696101 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55fbb597b410 0x55fbb597d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.285+0000 7f0a92ffd640 10 client.16308 ms_handle_connect on v2:172.21.15.134:6834/1150003275 2024-09-13T18:24:12.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.285+0000 7f0ab37fe640 1 --2- 172.21.15.114:0/3920696101 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55fbb597b410 0x55fbb597d830 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1b8d731b58cab65d server_cookie=b46152c7f1810c07 in_seq=0 out_seq=0 2024-09-13T18:24:12.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.286+0000 7f0a92ffd640 1 -- 172.21.15.114:0/3920696101 <== mds.3 v2:172.21.15.134:6834/1150003275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fbb55574e0 con 0x55fbb597b410 2024-09-13T18:24:12.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.286+0000 7f0a92ffd640 10 client.16308 handle_command_reply: tid=0 2024-09-13T18:24:12.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.286+0000 7f0abc81e740 1 client.16308 shutdown 2024-09-13T18:24:12.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.286+0000 7f0abc81e740 1 -- 172.21.15.114:0/3920696101 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb5906140 msgr2=0x55fbb591b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:12.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.286+0000 7f0abc81e740 1 --2- 172.21.15.114:0/3920696101 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb5906140 0x55fbb591b370 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8015f30 tx=0x7f0aa80047d0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.286+0000 7f0abc81e740 1 -- 172.21.15.114:0/3920696101 shutdown_connections 2024-09-13T18:24:12.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.287+0000 7f0abc81e740 1 --2- 172.21.15.114:0/3920696101 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55fbb597b410 0x55fbb597d830 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.287+0000 7f0abc81e740 1 --2- 172.21.15.114:0/3920696101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fbb5913ff0 0x55fbb5919640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.287+0000 7f0abc81e740 1 --2- 172.21.15.114:0/3920696101 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fbb5906140 0x55fbb591b370 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.287+0000 7f0abc81e740 1 --2- 172.21.15.114:0/3920696101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb58ff850 0x55fbb591ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.287+0000 7f0abc81e740 1 -- 172.21.15.114:0/3920696101 >> 172.21.15.114:0/3920696101 conn(0x55fbb5901110 msgr2=0x55fbb58fba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:12.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.287+0000 7f0a92ffd640 0 client.16308 ms_handle_reset on v2:172.21.15.134:6834/1150003275 2024-09-13T18:24:12.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.287+0000 7f0abc81e740 1 -- 172.21.15.114:0/3920696101 shutdown_connections 2024-09-13T18:24:12.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.287+0000 7f0abc81e740 1 -- 172.21.15.114:0/3920696101 wait complete. 2024-09-13T18:24:12.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.287+0000 7f0abc81e740 20 client.16308 trim_cache size 0 max 16384 2024-09-13T18:24:12.295 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:24:12.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.288+0000 7f0ab9b01640 1 -- 172.21.15.114:0/1927246309 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0a9c0a1280 msgr2=0x7f0a9c0a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:12.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.288+0000 7f0ab9b01640 1 --2- 172.21.15.114:0/1927246309 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0a9c0a1280 0x7f0a9c0a3740 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0aa40057c0 tx=0x7f0aa4005a20 comp rx=0 tx=0).stop 2024-09-13T18:24:12.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.288+0000 7f0ab9b01640 1 -- 172.21.15.114:0/1927246309 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ab419adb0 msgr2=0x7f0ab419b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:12.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.288+0000 7f0ab9b01640 1 --2- 172.21.15.114:0/1927246309 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ab419adb0 0x7f0ab419b260 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f0aac00ea60 tx=0x7f0aac00ee60 comp rx=0 tx=0).stop 2024-09-13T18:24:12.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.288+0000 7f0ab9b01640 1 -- 172.21.15.114:0/1927246309 shutdown_connections 2024-09-13T18:24:12.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.288+0000 7f0ab9b01640 1 --2- 172.21.15.114:0/1927246309 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0a9c0a1280 0x7f0a9c0a3740 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.288+0000 7f0ab9b01640 1 --2- 172.21.15.114:0/1927246309 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ab419adb0 0x7f0ab419b260 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.288+0000 7f0ab9b01640 1 --2- 172.21.15.114:0/1927246309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ab41041d0 0x7f0ab41969c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.288+0000 7f0ab9b01640 1 --2- 172.21.15.114:0/1927246309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ab40ff5a0 0x7f0ab4196480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.288+0000 7f0ab9b01640 1 -- 172.21.15.114:0/1927246309 >> 172.21.15.114:0/1927246309 conn(0x7f0ab40753f0 msgr2=0x7f0ab40fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:12.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.288+0000 7f0ab9b01640 1 -- 172.21.15.114:0/1927246309 shutdown_connections 2024-09-13T18:24:12.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.288+0000 7f0ab9b01640 1 -- 172.21.15.114:0/1927246309 wait complete. 2024-09-13T18:24:12.306 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-09-13T18:24:12.368 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:11 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/3590890566' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:24:12.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.453+0000 7f166afe2640 1 -- 172.21.15.114:0/1580626033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16640fef00 msgr2=0x7f16640ff2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:12.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.453+0000 7f166afe2640 1 --2- 172.21.15.114:0/1580626033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16640fef00 0x7f16640ff2e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1658009900 tx=0x7f165802a410 comp rx=0 tx=0).stop 2024-09-13T18:24:12.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.453+0000 7f166afe2640 1 -- 172.21.15.114:0/1580626033 shutdown_connections 2024-09-13T18:24:12.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.453+0000 7f166afe2640 1 --2- 172.21.15.114:0/1580626033 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1664104050 0x7f166410bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.453+0000 7f166afe2640 1 --2- 172.21.15.114:0/1580626033 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16640ff8b0 0x7f16641038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.453+0000 7f166afe2640 1 --2- 172.21.15.114:0/1580626033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16640fef00 0x7f16640ff2e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.453+0000 7f166afe2640 1 -- 172.21.15.114:0/1580626033 >> 172.21.15.114:0/1580626033 conn(0x7f16640fa990 msgr2=0x7f16640fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:12.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.453+0000 7f166afe2640 1 -- 172.21.15.114:0/1580626033 shutdown_connections 2024-09-13T18:24:12.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.453+0000 7f166afe2640 1 -- 172.21.15.114:0/1580626033 wait complete. 2024-09-13T18:24:12.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.454+0000 7f166afe2640 1 Processor -- start 2024-09-13T18:24:12.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.454+0000 7f166afe2640 1 -- start start 2024-09-13T18:24:12.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.455+0000 7f166afe2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16640fef00 0x7f166410c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.455+0000 7f166afe2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1664104050 0x7f166410c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.455+0000 7f166afe2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f166410cd70 0x7f16641a4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.455+0000 7f166afe2640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16641a4ad0 con 0x7f16640fef00 2024-09-13T18:24:12.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.455+0000 7f166afe2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16641a4c40 con 0x7f1664104050 2024-09-13T18:24:12.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.455+0000 7f166afe2640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16641a4e00 con 0x7f166410cd70 2024-09-13T18:24:12.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.455+0000 7f1668d7e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16640fef00 0x7f166410c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.455+0000 7f1668d7e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16640fef00 0x7f166410c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43518/0 (socket says 172.21.15.114:43518) 2024-09-13T18:24:12.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.455+0000 7f1668d7e640 1 -- 172.21.15.114:0/1407662319 learned_addr learned my addr 172.21.15.114:0/1407662319 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:12.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.455+0000 7f1663fff640 1 --2- 172.21.15.114:0/1407662319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1664104050 0x7f166410c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.455+0000 7f166957f640 1 --2- 172.21.15.114:0/1407662319 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f166410cd70 0x7f16641a4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.455+0000 7f1668d7e640 1 -- 172.21.15.114:0/1407662319 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f166410cd70 msgr2=0x7f16641a4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:12.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.455+0000 7f1668d7e640 1 --2- 172.21.15.114:0/1407662319 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f166410cd70 0x7f16641a4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.455+0000 7f1668d7e640 1 -- 172.21.15.114:0/1407662319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1664104050 msgr2=0x7f166410c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:12.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.455+0000 7f1668d7e640 1 --2- 172.21.15.114:0/1407662319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1664104050 0x7f166410c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.455+0000 7f1668d7e640 1 -- 172.21.15.114:0/1407662319 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1658009560 con 0x7f16640fef00 2024-09-13T18:24:12.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.455+0000 7f1663fff640 1 --2- 172.21.15.114:0/1407662319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1664104050 0x7f166410c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:24:12.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.455+0000 7f166957f640 1 --2- 172.21.15.114:0/1407662319 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f166410cd70 0x7f16641a4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:24:12.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.455+0000 7f1668d7e640 1 --2- 172.21.15.114:0/1407662319 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16640fef00 0x7f166410c2f0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f1658009a20 tx=0x7f16580045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:12.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.456+0000 7f1661ffb640 1 -- 172.21.15.114:0/1407662319 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1658004c50 con 0x7f16640fef00 2024-09-13T18:24:12.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.456+0000 7f1661ffb640 1 -- 172.21.15.114:0/1407662319 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1658004db0 con 0x7f16640fef00 2024-09-13T18:24:12.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.456+0000 7f166afe2640 1 -- 172.21.15.114:0/1407662319 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16641a5080 con 0x7f16640fef00 2024-09-13T18:24:12.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.456+0000 7f1661ffb640 1 -- 172.21.15.114:0/1407662319 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f165803ca40 con 0x7f16640fef00 2024-09-13T18:24:12.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.456+0000 7f166afe2640 1 -- 172.21.15.114:0/1407662319 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16641a5580 con 0x7f16640fef00 2024-09-13T18:24:12.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.459+0000 7f1661ffb640 1 -- 172.21.15.114:0/1407662319 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f165803cba0 con 0x7f16640fef00 2024-09-13T18:24:12.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.459+0000 7f166ddda740 1 Processor -- start 2024-09-13T18:24:12.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.459+0000 7f166ddda740 1 -- start start 2024-09-13T18:24:12.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.460+0000 7f166ddda740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5557b5711790 0x5557b5711b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.460+0000 7f166ddda740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557b57120b0 0x5557b571c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.460+0000 7f166ddda740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557b571c950 0x5557b571ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.460+0000 7f166ddda740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557b567bcc0 con 0x5557b5711790 2024-09-13T18:24:12.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.460+0000 7f166ddda740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557b5679450 con 0x5557b571c950 2024-09-13T18:24:12.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.460+0000 7f166ddda740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557b5674d40 con 0x5557b57120b0 2024-09-13T18:24:12.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.460+0000 7f166957f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5557b5711790 0x5557b5711b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.460+0000 7f1668d7e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557b57120b0 0x5557b571c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.460+0000 7f1663fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557b571c950 0x5557b571ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.460+0000 7f166957f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5557b5711790 0x5557b5711b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43522/0 (socket says 172.21.15.114:43522) 2024-09-13T18:24:12.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.460+0000 7f166957f640 1 -- 172.21.15.114:0/2914723517 learned_addr learned my addr 172.21.15.114:0/2914723517 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:12.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.460+0000 7f1668d7e640 1 -- 172.21.15.114:0/2914723517 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557b571c950 msgr2=0x5557b571ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:12.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.460+0000 7f1668d7e640 1 --2- 172.21.15.114:0/2914723517 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557b571c950 0x5557b571ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.460+0000 7f1668d7e640 1 -- 172.21.15.114:0/2914723517 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5557b5711790 msgr2=0x5557b5711b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:12.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.460+0000 7f1668d7e640 1 --2- 172.21.15.114:0/2914723517 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5557b5711790 0x5557b5711b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.461+0000 7f1668d7e640 1 -- 172.21.15.114:0/2914723517 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557b563c620 con 0x5557b57120b0 2024-09-13T18:24:12.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.461+0000 7f1663fff640 1 --2- 172.21.15.114:0/2914723517 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557b571c950 0x5557b571ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:24:12.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.461+0000 7f166957f640 1 --2- 172.21.15.114:0/2914723517 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5557b5711790 0x5557b5711b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:24:12.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.461+0000 7f1668d7e640 1 --2- 172.21.15.114:0/2914723517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557b57120b0 0x5557b571c410 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f16580eb340 tx=0x7f16580f60e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33cc5bc90c0b1b88 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:12.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.461+0000 7f164effd640 1 -- 172.21.15.114:0/2914723517 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16580f6920 con 0x5557b57120b0 2024-09-13T18:24:12.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.461+0000 7f164effd640 1 -- 172.21.15.114:0/2914723517 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f16580fc400 con 0x5557b57120b0 2024-09-13T18:24:12.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.461+0000 7f164effd640 1 -- 172.21.15.114:0/2914723517 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16580ee610 con 0x5557b57120b0 2024-09-13T18:24:12.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.462+0000 7f1661ffb640 1 --2- 172.21.15.114:0/1407662319 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f16540a1280 0x7f16540a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.462+0000 7f166957f640 1 --2- 172.21.15.114:0/1407662319 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f16540a1280 0x7f16540a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.462+0000 7f166ddda740 1 -- 172.21.15.114:0/2914723517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557b57120b0 msgr2=0x5557b571c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:12.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.462+0000 7f166ddda740 1 --2- 172.21.15.114:0/2914723517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557b57120b0 0x5557b571c410 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f16580eb340 tx=0x7f16580f60e0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.462+0000 7f1661ffb640 1 -- 172.21.15.114:0/1407662319 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f16580edf40 con 0x7f16640fef00 2024-09-13T18:24:12.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.462+0000 7f166ddda740 1 -- 172.21.15.114:0/2914723517 shutdown_connections 2024-09-13T18:24:12.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.462+0000 7f166ddda740 1 --2- 172.21.15.114:0/2914723517 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557b571c950 0x5557b571ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.462+0000 7f166ddda740 1 --2- 172.21.15.114:0/2914723517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557b57120b0 0x5557b571c410 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.462+0000 7f166ddda740 1 --2- 172.21.15.114:0/2914723517 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5557b5711790 0x5557b5711b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.462+0000 7f166ddda740 1 -- 172.21.15.114:0/2914723517 >> 172.21.15.114:0/2914723517 conn(0x5557b570dd00 msgr2=0x5557b570e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:12.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.462+0000 7f166ddda740 1 -- 172.21.15.114:0/2914723517 shutdown_connections 2024-09-13T18:24:12.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.462+0000 7f166ddda740 1 -- 172.21.15.114:0/2914723517 wait complete. 2024-09-13T18:24:12.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.463+0000 7f166957f640 1 --2- 172.21.15.114:0/1407662319 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f16540a1280 0x7f16540a3740 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f165c0135d0 tx=0x7f165c009820 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:12.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.463+0000 7f166ddda740 1 -- start start 2024-09-13T18:24:12.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.463+0000 7f166ddda740 1 Processor -- start 2024-09-13T18:24:12.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.463+0000 7f166ddda740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557b5711790 0x5557b571be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.463+0000 7f166ddda740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5557b57120b0 0x5557b571c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.463+0000 7f166ddda740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557b571c950 0x5557b5719400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.463+0000 7f166ddda740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557b5633180 con 0x5557b57120b0 2024-09-13T18:24:12.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.463+0000 7f166ddda740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557b534d430 con 0x5557b571c950 2024-09-13T18:24:12.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.463+0000 7f166ddda740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557b55f85e0 con 0x5557b5711790 2024-09-13T18:24:12.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.463+0000 7f166ddda740 10 client.0 _ll_register_callbacks cb 0x5557b5664aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:24:12.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.463+0000 7f1668d7e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557b5711790 0x5557b571be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.463+0000 7f1668d7e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557b5711790 0x5557b571be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:43564/0 (socket says 172.21.15.114:43564) 2024-09-13T18:24:12.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.463+0000 7f1668d7e640 1 -- 172.21.15.114:0/2387856305 learned_addr learned my addr 172.21.15.114:0/2387856305 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:12.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.463+0000 7f1663fff640 1 --2- 172.21.15.114:0/2387856305 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5557b57120b0 0x5557b571c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.464+0000 7f166957f640 1 --2- 172.21.15.114:0/2387856305 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557b571c950 0x5557b5719400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.464+0000 7f1663fff640 1 -- 172.21.15.114:0/2387856305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557b5711790 msgr2=0x5557b571be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:12.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.464+0000 7f1663fff640 1 --2- 172.21.15.114:0/2387856305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557b5711790 0x5557b571be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.464+0000 7f1663fff640 1 -- 172.21.15.114:0/2387856305 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557b571c950 msgr2=0x5557b5719400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:12.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.464+0000 7f1663fff640 1 --2- 172.21.15.114:0/2387856305 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557b571c950 0x5557b5719400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.464+0000 7f1663fff640 1 -- 172.21.15.114:0/2387856305 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16641a5580 con 0x5557b57120b0 2024-09-13T18:24:12.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.464+0000 7f1668d7e640 1 --2- 172.21.15.114:0/2387856305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557b5711790 0x5557b571be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:24:12.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.464+0000 7f1663fff640 1 --2- 172.21.15.114:0/2387856305 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5557b57120b0 0x5557b571c3b0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x5557b5714330 tx=0x7f165000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3d3f22ab033298e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:12.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.464+0000 7f1643fff640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:24:12.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.464+0000 7f1643fff640 1 -- 172.21.15.114:0/2387856305 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1650017070 con 0x5557b57120b0 2024-09-13T18:24:12.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.464+0000 7f1643fff640 1 -- 172.21.15.114:0/2387856305 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f16500120d0 con 0x5557b57120b0 2024-09-13T18:24:12.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.464+0000 7f166ddda740 1 -- 172.21.15.114:0/2387856305 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5557b569eb80 con 0x5557b57120b0 2024-09-13T18:24:12.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.465+0000 7f1643fff640 1 -- 172.21.15.114:0/2387856305 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1650012290 con 0x5557b57120b0 2024-09-13T18:24:12.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.465+0000 7f166ddda740 10 client.16320 fetch_fsmap learned FSMap version 32 2024-09-13T18:24:12.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.465+0000 7f166ddda740 1 -- 172.21.15.114:0/2387856305 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5557b56733c0 con 0x5557b57120b0 2024-09-13T18:24:12.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.465+0000 7f1643fff640 1 -- 172.21.15.114:0/2387856305 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5557b569eb80 con 0x5557b57120b0 2024-09-13T18:24:12.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.465+0000 7f1643fff640 1 -- 172.21.15.114:0/2387856305 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f165001c460 con 0x5557b57120b0 2024-09-13T18:24:12.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.465+0000 7f166ddda740 10 client.16320 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:24:12.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.465+0000 7f166ddda740 10 client.16320 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-13T18:24:12.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.465+0000 7f166ddda740 20 client.16320 populate_metadata read hostname 'smithi114' 2024-09-13T18:24:12.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.465+0000 7f166ddda740 1 --2- 172.21.15.114:0/2387856305 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x5557b5734760 0x5557b5736b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.466+0000 7f166ddda740 4 client.16320 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:24:12.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.466+0000 7f166ddda740 1 -- 172.21.15.114:0/2387856305 --> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5557b56671f0 con 0x5557b5734760 2024-09-13T18:24:12.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.466+0000 7f1668d7e640 1 --2- 172.21.15.114:0/2387856305 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x5557b5734760 0x5557b5736b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.466+0000 7f1643fff640 10 client.16320 ms_handle_connect on v2:172.21.15.204:6838/344883912 2024-09-13T18:24:12.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.466+0000 7f1668d7e640 1 --2- 172.21.15.114:0/2387856305 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x5557b5734760 0x5557b5736b80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6ed929bc2b22fe4a server_cookie=cca8648b5423d53e in_seq=0 out_seq=0 2024-09-13T18:24:12.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.467+0000 7f1643fff640 1 -- 172.21.15.114:0/2387856305 <== mds.2 v2:172.21.15.204:6838/344883912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5557b56671f0 con 0x5557b5734760 2024-09-13T18:24:12.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.467+0000 7f1643fff640 10 client.16320 handle_command_reply: tid=0 2024-09-13T18:24:12.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.467+0000 7f166ddda740 1 client.16320 shutdown 2024-09-13T18:24:12.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.467+0000 7f166ddda740 1 -- 172.21.15.114:0/2387856305 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5557b57120b0 msgr2=0x5557b571c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:12.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.467+0000 7f166ddda740 1 --2- 172.21.15.114:0/2387856305 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5557b57120b0 0x5557b571c3b0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x5557b5714330 tx=0x7f165000f790 comp rx=0 tx=0).stop 2024-09-13T18:24:12.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.467+0000 7f166ddda740 1 -- 172.21.15.114:0/2387856305 shutdown_connections 2024-09-13T18:24:12.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.467+0000 7f166ddda740 1 --2- 172.21.15.114:0/2387856305 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x5557b5734760 0x5557b5736b80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.467+0000 7f166ddda740 1 --2- 172.21.15.114:0/2387856305 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557b571c950 0x5557b5719400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.467+0000 7f166ddda740 1 --2- 172.21.15.114:0/2387856305 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5557b57120b0 0x5557b571c3b0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.467+0000 7f166ddda740 1 --2- 172.21.15.114:0/2387856305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557b5711790 0x5557b571be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.467+0000 7f166ddda740 1 -- 172.21.15.114:0/2387856305 >> 172.21.15.114:0/2387856305 conn(0x5557b570dd00 msgr2=0x5557b5713a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:12.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.467+0000 7f1643fff640 0 client.16320 ms_handle_reset on v2:172.21.15.204:6838/344883912 2024-09-13T18:24:12.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.467+0000 7f166ddda740 1 -- 172.21.15.114:0/2387856305 shutdown_connections 2024-09-13T18:24:12.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.467+0000 7f166ddda740 1 -- 172.21.15.114:0/2387856305 wait complete. 2024-09-13T18:24:12.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.467+0000 7f166ddda740 20 client.16320 trim_cache size 0 max 16384 2024-09-13T18:24:12.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.486+0000 7f166ddda740 1 Processor -- start 2024-09-13T18:24:12.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.486+0000 7f166ddda740 1 -- start start 2024-09-13T18:24:12.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.487+0000 7f166ddda740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557b5711790 0x5557b573b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.487+0000 7f166ddda740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557b57120b0 0x5557b572bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.487+0000 7f166ddda740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5557b571c950 0x5557b572c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.487+0000 7f166ddda740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557b5667220 con 0x5557b571c950 2024-09-13T18:24:12.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.487+0000 7f166ddda740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557b569ebb0 con 0x5557b57120b0 2024-09-13T18:24:12.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.487+0000 7f166ddda740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557b5733060 con 0x5557b5711790 2024-09-13T18:24:12.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.487+0000 7f166957f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5557b571c950 0x5557b572c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.487+0000 7f166957f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5557b571c950 0x5557b572c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43538/0 (socket says 172.21.15.114:43538) 2024-09-13T18:24:12.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.487+0000 7f166957f640 1 -- 172.21.15.114:0/3543614213 learned_addr learned my addr 172.21.15.114:0/3543614213 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:12.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.487+0000 7f1668d7e640 1 --2- 172.21.15.114:0/3543614213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557b5711790 0x5557b573b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.487+0000 7f1663fff640 1 --2- 172.21.15.114:0/3543614213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557b57120b0 0x5557b572bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.487+0000 7f1668d7e640 1 -- 172.21.15.114:0/3543614213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557b57120b0 msgr2=0x5557b572bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:12.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.487+0000 7f1668d7e640 1 --2- 172.21.15.114:0/3543614213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557b57120b0 0x5557b572bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.487+0000 7f1668d7e640 1 -- 172.21.15.114:0/3543614213 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5557b571c950 msgr2=0x5557b572c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:12.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.487+0000 7f1668d7e640 1 --2- 172.21.15.114:0/3543614213 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5557b571c950 0x5557b572c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.487+0000 7f1668d7e640 1 -- 172.21.15.114:0/3543614213 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557b5615770 con 0x5557b5711790 2024-09-13T18:24:12.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.487+0000 7f166957f640 1 --2- 172.21.15.114:0/3543614213 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5557b571c950 0x5557b572c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:24:12.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.488+0000 7f1668d7e640 1 --2- 172.21.15.114:0/3543614213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557b5711790 0x5557b573b4e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f16580b7770 tx=0x7f165802a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aff1469552dfa961 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:12.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.488+0000 7f164f7fe640 1 -- 172.21.15.114:0/3543614213 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16580ee200 con 0x5557b5711790 2024-09-13T18:24:12.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.488+0000 7f164f7fe640 1 -- 172.21.15.114:0/3543614213 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f16580b7040 con 0x5557b5711790 2024-09-13T18:24:12.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.488+0000 7f164f7fe640 1 -- 172.21.15.114:0/3543614213 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16580bad00 con 0x5557b5711790 2024-09-13T18:24:12.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.489+0000 7f166ddda740 1 -- 172.21.15.114:0/3543614213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557b5711790 msgr2=0x5557b573b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:12.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.489+0000 7f166ddda740 1 --2- 172.21.15.114:0/3543614213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557b5711790 0x5557b573b4e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f16580b7770 tx=0x7f165802a410 comp rx=0 tx=0).stop 2024-09-13T18:24:12.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.489+0000 7f166ddda740 1 -- 172.21.15.114:0/3543614213 shutdown_connections 2024-09-13T18:24:12.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.489+0000 7f166ddda740 1 --2- 172.21.15.114:0/3543614213 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5557b571c950 0x5557b572c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.489+0000 7f166ddda740 1 --2- 172.21.15.114:0/3543614213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557b57120b0 0x5557b572bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.489+0000 7f166ddda740 1 --2- 172.21.15.114:0/3543614213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557b5711790 0x5557b573b4e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.489+0000 7f166ddda740 1 -- 172.21.15.114:0/3543614213 >> 172.21.15.114:0/3543614213 conn(0x5557b570dd00 msgr2=0x5557b570e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:12.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.489+0000 7f166ddda740 1 -- 172.21.15.114:0/3543614213 shutdown_connections 2024-09-13T18:24:12.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.489+0000 7f166ddda740 1 -- 172.21.15.114:0/3543614213 wait complete. 2024-09-13T18:24:12.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.489+0000 7f166ddda740 1 -- start start 2024-09-13T18:24:12.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.489+0000 7f166ddda740 1 Processor -- start 2024-09-13T18:24:12.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.490+0000 7f166ddda740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5557b5711790 0x5557b572bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.490+0000 7f166ddda740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557b57120b0 0x5557b572c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.490+0000 7f166ddda740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557b571c950 0x5557b572a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.490+0000 7f166ddda740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557b572a7c0 con 0x5557b5711790 2024-09-13T18:24:12.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.490+0000 7f166ddda740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557b572a930 con 0x5557b571c950 2024-09-13T18:24:12.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.490+0000 7f166ddda740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557b572aaf0 con 0x5557b57120b0 2024-09-13T18:24:12.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.490+0000 7f166ddda740 10 client.0 _ll_register_callbacks cb 0x7f16580f1920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:24:12.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.490+0000 7f1668d7e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5557b5711790 0x5557b572bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.490+0000 7f1663fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557b57120b0 0x5557b572c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.490+0000 7f1668d7e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5557b5711790 0x5557b572bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43546/0 (socket says 172.21.15.114:43546) 2024-09-13T18:24:12.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.490+0000 7f1668d7e640 1 -- 172.21.15.114:0/4263050393 learned_addr learned my addr 172.21.15.114:0/4263050393 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:12.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.490+0000 7f166957f640 1 --2- 172.21.15.114:0/4263050393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557b571c950 0x5557b572a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.490+0000 7f1668d7e640 1 -- 172.21.15.114:0/4263050393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557b57120b0 msgr2=0x5557b572c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:12.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.490+0000 7f1668d7e640 1 --2- 172.21.15.114:0/4263050393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557b57120b0 0x5557b572c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.490+0000 7f1668d7e640 1 -- 172.21.15.114:0/4263050393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557b571c950 msgr2=0x5557b572a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:12.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.490+0000 7f1668d7e640 1 --2- 172.21.15.114:0/4263050393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557b571c950 0x5557b572a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.490+0000 7f1668d7e640 1 -- 172.21.15.114:0/4263050393 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f165c019070 con 0x5557b5711790 2024-09-13T18:24:12.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.490+0000 7f1663fff640 1 --2- 172.21.15.114:0/4263050393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557b57120b0 0x5557b572c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:24:12.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.490+0000 7f166957f640 1 --2- 172.21.15.114:0/4263050393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557b571c950 0x5557b572a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:24:12.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.491+0000 7f1668d7e640 1 --2- 172.21.15.114:0/4263050393 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5557b5711790 0x5557b572bb30 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f1658000c00 tx=0x7f16580f1b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c9fe09fd6298fbc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:12.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.491+0000 7f1643fff640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:24:12.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.491+0000 7f1643fff640 1 -- 172.21.15.114:0/4263050393 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16580bb470 con 0x5557b5711790 2024-09-13T18:24:12.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.491+0000 7f1643fff640 1 -- 172.21.15.114:0/4263050393 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f16580beb30 con 0x5557b5711790 2024-09-13T18:24:12.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.491+0000 7f166ddda740 1 -- 172.21.15.114:0/4263050393 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f16580fc400 con 0x5557b5711790 2024-09-13T18:24:12.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.491+0000 7f1643fff640 1 -- 172.21.15.114:0/4263050393 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16580baad0 con 0x5557b5711790 2024-09-13T18:24:12.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.491+0000 7f166ddda740 10 client.16326 fetch_fsmap learned FSMap version 32 2024-09-13T18:24:12.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.491+0000 7f166ddda740 1 -- 172.21.15.114:0/4263050393 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5557b563c620 con 0x5557b5711790 2024-09-13T18:24:12.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.491+0000 7f1643fff640 1 -- 172.21.15.114:0/4263050393 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f16580fc400 con 0x5557b5711790 2024-09-13T18:24:12.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.492+0000 7f1643fff640 1 -- 172.21.15.114:0/4263050393 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f16580eb5c0 con 0x5557b5711790 2024-09-13T18:24:12.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.492+0000 7f166ddda740 10 client.16326 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:24:12.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.492+0000 7f166ddda740 10 client.16326 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-13T18:24:12.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.492+0000 7f166ddda740 20 client.16326 populate_metadata read hostname 'smithi114' 2024-09-13T18:24:12.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.492+0000 7f166ddda740 1 --2- 172.21.15.114:0/4263050393 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x5557b578ba90 0x5557b578de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.492+0000 7f166ddda740 4 client.16326 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:24:12.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.492+0000 7f166ddda740 1 -- 172.21.15.114:0/4263050393 --> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5557b53684e0 con 0x5557b578ba90 2024-09-13T18:24:12.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.492+0000 7f1663fff640 1 --2- 172.21.15.114:0/4263050393 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x5557b578ba90 0x5557b578de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.493+0000 7f1663fff640 1 --2- 172.21.15.114:0/4263050393 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x5557b578ba90 0x5557b578de80 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=167587df2280bc26 server_cookie=c3dcd06a9c7e596 in_seq=0 out_seq=0 2024-09-13T18:24:12.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.493+0000 7f1643fff640 10 client.16326 ms_handle_connect on v2:172.21.15.204:6838/344883912 2024-09-13T18:24:12.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.494+0000 7f1643fff640 1 -- 172.21.15.114:0/4263050393 <== mds.2 v2:172.21.15.204:6838/344883912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5557b53684e0 con 0x5557b578ba90 2024-09-13T18:24:12.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.494+0000 7f1643fff640 10 client.16326 handle_command_reply: tid=0 2024-09-13T18:24:12.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.494+0000 7f166ddda740 1 client.16326 shutdown 2024-09-13T18:24:12.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.494+0000 7f166ddda740 1 -- 172.21.15.114:0/4263050393 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5557b5711790 msgr2=0x5557b572bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:12.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.494+0000 7f166ddda740 1 --2- 172.21.15.114:0/4263050393 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5557b5711790 0x5557b572bb30 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f1658000c00 tx=0x7f16580f1b90 comp rx=0 tx=0).stop 2024-09-13T18:24:12.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.494+0000 7f166ddda740 1 -- 172.21.15.114:0/4263050393 shutdown_connections 2024-09-13T18:24:12.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.494+0000 7f166ddda740 1 --2- 172.21.15.114:0/4263050393 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x5557b578ba90 0x5557b578de80 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.494+0000 7f166ddda740 1 --2- 172.21.15.114:0/4263050393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557b571c950 0x5557b572a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.494+0000 7f166ddda740 1 --2- 172.21.15.114:0/4263050393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5557b57120b0 0x5557b572c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.494+0000 7f166ddda740 1 --2- 172.21.15.114:0/4263050393 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5557b5711790 0x5557b572bb30 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.494+0000 7f1643fff640 0 client.16326 ms_handle_reset on v2:172.21.15.204:6838/344883912 2024-09-13T18:24:12.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.494+0000 7f166ddda740 1 -- 172.21.15.114:0/4263050393 >> 172.21.15.114:0/4263050393 conn(0x5557b570dd00 msgr2=0x5557b570e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:12.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.494+0000 7f166ddda740 1 -- 172.21.15.114:0/4263050393 shutdown_connections 2024-09-13T18:24:12.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.494+0000 7f166ddda740 1 -- 172.21.15.114:0/4263050393 wait complete. 2024-09-13T18:24:12.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.495+0000 7f166ddda740 20 client.16326 trim_cache size 0 max 16384 2024-09-13T18:24:12.501 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:24:12.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.495+0000 7f166afe2640 1 -- 172.21.15.114:0/1407662319 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f16540a1280 msgr2=0x7f16540a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:12.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.495+0000 7f166afe2640 1 --2- 172.21.15.114:0/1407662319 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f16540a1280 0x7f16540a3740 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f165c0135d0 tx=0x7f165c009820 comp rx=0 tx=0).stop 2024-09-13T18:24:12.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.495+0000 7f166afe2640 1 -- 172.21.15.114:0/1407662319 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16640fef00 msgr2=0x7f166410c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:12.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.495+0000 7f166afe2640 1 --2- 172.21.15.114:0/1407662319 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16640fef00 0x7f166410c2f0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f1658009a20 tx=0x7f16580045e0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.495+0000 7f166afe2640 1 -- 172.21.15.114:0/1407662319 shutdown_connections 2024-09-13T18:24:12.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.495+0000 7f166afe2640 1 --2- 172.21.15.114:0/1407662319 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f16540a1280 0x7f16540a3740 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.495+0000 7f166afe2640 1 --2- 172.21.15.114:0/1407662319 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f166410cd70 0x7f16641a4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.495+0000 7f166afe2640 1 --2- 172.21.15.114:0/1407662319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1664104050 0x7f166410c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.495+0000 7f166afe2640 1 --2- 172.21.15.114:0/1407662319 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16640fef00 0x7f166410c2f0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.495+0000 7f166afe2640 1 -- 172.21.15.114:0/1407662319 >> 172.21.15.114:0/1407662319 conn(0x7f16640fa990 msgr2=0x7f16640fbe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:12.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.495+0000 7f166afe2640 1 -- 172.21.15.114:0/1407662319 shutdown_connections 2024-09-13T18:24:12.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.496+0000 7f166afe2640 1 -- 172.21.15.114:0/1407662319 wait complete. 2024-09-13T18:24:12.514 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-09-13T18:24:12.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.663+0000 7f53b53d4640 1 -- 172.21.15.114:0/631745611 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53b0111b70 msgr2=0x7f53b0113f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:12.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.663+0000 7f53b53d4640 1 --2- 172.21.15.114:0/631745611 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53b0111b70 0x7f53b0113f60 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f53a8009ca0 tx=0x7f53a802a8c0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.663+0000 7f53b53d4640 1 -- 172.21.15.114:0/631745611 shutdown_connections 2024-09-13T18:24:12.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.664+0000 7f53b53d4640 1 --2- 172.21.15.114:0/631745611 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53b0111b70 0x7f53b0113f60 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.664+0000 7f53b53d4640 1 --2- 172.21.15.114:0/631745611 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53b0108f70 0x7f53b0111630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.664+0000 7f53b53d4640 1 --2- 172.21.15.114:0/631745611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53b01085c0 0x7f53b01089a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.664+0000 7f53b53d4640 1 -- 172.21.15.114:0/631745611 >> 172.21.15.114:0/631745611 conn(0x7f53b006b440 msgr2=0x7f53b006b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:12.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.664+0000 7f53b53d4640 1 -- 172.21.15.114:0/631745611 shutdown_connections 2024-09-13T18:24:12.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.664+0000 7f53b53d4640 1 -- 172.21.15.114:0/631745611 wait complete. 2024-09-13T18:24:12.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.664+0000 7f53b53d4640 1 Processor -- start 2024-09-13T18:24:12.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.664+0000 7f53b53d4640 1 -- start start 2024-09-13T18:24:12.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.665+0000 7f53b53d4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53b01085c0 0x7f53b01a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.665+0000 7f53b53d4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53b0108f70 0x7f53b01a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.665+0000 7f53b53d4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53b0111b70 0x7f53b019e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.665+0000 7f53b53d4640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b019ef40 con 0x7f53b0111b70 2024-09-13T18:24:12.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.665+0000 7f53b53d4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b019f0b0 con 0x7f53b0108f70 2024-09-13T18:24:12.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.665+0000 7f53b53d4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b019f270 con 0x7f53b01085c0 2024-09-13T18:24:12.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.665+0000 7f53af7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53b0111b70 0x7f53b019e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.665+0000 7f53ae7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53b0108f70 0x7f53b01a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.665+0000 7f53ae7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53b0108f70 0x7f53b01a4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:51864/0 (socket says 172.21.15.114:51864) 2024-09-13T18:24:12.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.665+0000 7f53ae7fc640 1 -- 172.21.15.114:0/883305540 learned_addr learned my addr 172.21.15.114:0/883305540 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:12.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.665+0000 7f53aeffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53b01085c0 0x7f53b01a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.665+0000 7f53af7fe640 1 --2- 172.21.15.114:0/883305540 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53b0111b70 0x7f53b019e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43564/0 (socket says 172.21.15.114:43564) 2024-09-13T18:24:12.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.665+0000 7f53ae7fc640 1 -- 172.21.15.114:0/883305540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53b01085c0 msgr2=0x7f53b01a46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:12.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.665+0000 7f53ae7fc640 1 --2- 172.21.15.114:0/883305540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53b01085c0 0x7f53b01a46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.665+0000 7f53ae7fc640 1 -- 172.21.15.114:0/883305540 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53b0111b70 msgr2=0x7f53b019e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:12.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.665+0000 7f53ae7fc640 1 --2- 172.21.15.114:0/883305540 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53b0111b70 0x7f53b019e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.665+0000 7f53ae7fc640 1 -- 172.21.15.114:0/883305540 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53a8009900 con 0x7f53b0108f70 2024-09-13T18:24:12.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.666+0000 7f53ae7fc640 1 --2- 172.21.15.114:0/883305540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53b0108f70 0x7f53b01a4c10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f53a400e9f0 tx=0x7f53a400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:12.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.666+0000 7f5397fff640 1 -- 172.21.15.114:0/883305540 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a4009700 con 0x7f53b0108f70 2024-09-13T18:24:12.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.666+0000 7f53b53d4640 1 -- 172.21.15.114:0/883305540 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53b019f550 con 0x7f53b0108f70 2024-09-13T18:24:12.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.666+0000 7f5397fff640 1 -- 172.21.15.114:0/883305540 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f53a400cd80 con 0x7f53b0108f70 2024-09-13T18:24:12.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.666+0000 7f5397fff640 1 -- 172.21.15.114:0/883305540 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a40187e0 con 0x7f53b0108f70 2024-09-13T18:24:12.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.666+0000 7f53b53d4640 1 -- 172.21.15.114:0/883305540 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53b006ebc0 con 0x7f53b0108f70 2024-09-13T18:24:12.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.669+0000 7f53b81d1740 1 Processor -- start 2024-09-13T18:24:12.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.669+0000 7f53b81d1740 1 -- start start 2024-09-13T18:24:12.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.669+0000 7f5397fff640 1 -- 172.21.15.114:0/883305540 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f53a4010430 con 0x7f53b0108f70 2024-09-13T18:24:12.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.669+0000 7f53b81d1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dff1ad7790 0x55dff1ad7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.669+0000 7f53b81d1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff1ad80b0 0x55dff1ae2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.669+0000 7f53b81d1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dff1ae2950 0x55dff1ae4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.669+0000 7f53b81d1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dff1a41cc0 con 0x55dff1ad7790 2024-09-13T18:24:12.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.669+0000 7f53b81d1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dff1a3f450 con 0x55dff1ad80b0 2024-09-13T18:24:12.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.669+0000 7f53b81d1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dff1a3ad40 con 0x55dff1ae2950 2024-09-13T18:24:12.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.670+0000 7f53af7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dff1ad7790 0x55dff1ad7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.670+0000 7f53af7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dff1ad7790 0x55dff1ad7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43566/0 (socket says 172.21.15.114:43566) 2024-09-13T18:24:12.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.670+0000 7f53af7fe640 1 -- 172.21.15.114:0/1723416483 learned_addr learned my addr 172.21.15.114:0/1723416483 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:12.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.670+0000 7f53ae7fc640 1 --2- 172.21.15.114:0/1723416483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dff1ae2950 0x55dff1ae4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.670+0000 7f53af7fe640 1 -- 172.21.15.114:0/1723416483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dff1ae2950 msgr2=0x55dff1ae4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:12.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.670+0000 7f53af7fe640 1 --2- 172.21.15.114:0/1723416483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dff1ae2950 0x55dff1ae4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.670+0000 7f53af7fe640 1 -- 172.21.15.114:0/1723416483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff1ad80b0 msgr2=0x55dff1ae2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:12.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.670+0000 7f53af7fe640 1 --2- 172.21.15.114:0/1723416483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff1ad80b0 0x55dff1ae2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.670+0000 7f53af7fe640 1 -- 172.21.15.114:0/1723416483 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dff1a02620 con 0x55dff1ad7790 2024-09-13T18:24:12.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.670+0000 7f53af7fe640 1 --2- 172.21.15.114:0/1723416483 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dff1ad7790 0x55dff1ad7b70 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f53b01095a0 tx=0x7f53a80093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e34e8b1665e431a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:12.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.670+0000 7f5397fff640 1 --2- 172.21.15.114:0/883305540 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f53980a1280 0x7f53980a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.671+0000 7f53957fa640 1 -- 172.21.15.114:0/1723416483 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a8031cb0 con 0x55dff1ad7790 2024-09-13T18:24:12.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.671+0000 7f53957fa640 1 -- 172.21.15.114:0/1723416483 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f53a80318e0 con 0x55dff1ad7790 2024-09-13T18:24:12.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.671+0000 7f5397fff640 1 -- 172.21.15.114:0/883305540 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f53a4014070 con 0x7f53b0108f70 2024-09-13T18:24:12.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.671+0000 7f53957fa640 1 -- 172.21.15.114:0/1723416483 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a80307c0 con 0x55dff1ad7790 2024-09-13T18:24:12.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.671+0000 7f53aeffd640 1 --2- 172.21.15.114:0/883305540 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f53980a1280 0x7f53980a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.672+0000 7f53b81d1740 1 -- 172.21.15.114:0/1723416483 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dff1ad7790 msgr2=0x55dff1ad7b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:12.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.672+0000 7f53aeffd640 1 --2- 172.21.15.114:0/883305540 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f53980a1280 0x7f53980a3740 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x55dff1ad4b60 tx=0x7f53a0009840 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:12.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.672+0000 7f53b81d1740 1 --2- 172.21.15.114:0/1723416483 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dff1ad7790 0x55dff1ad7b70 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f53b01095a0 tx=0x7f53a80093e0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.672+0000 7f53b81d1740 1 -- 172.21.15.114:0/1723416483 shutdown_connections 2024-09-13T18:24:12.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.672+0000 7f53b81d1740 1 --2- 172.21.15.114:0/1723416483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dff1ae2950 0x55dff1ae4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.672+0000 7f53b81d1740 1 --2- 172.21.15.114:0/1723416483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff1ad80b0 0x55dff1ae2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.672+0000 7f53b81d1740 1 --2- 172.21.15.114:0/1723416483 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dff1ad7790 0x55dff1ad7b70 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.672+0000 7f53b81d1740 1 -- 172.21.15.114:0/1723416483 >> 172.21.15.114:0/1723416483 conn(0x55dff1ad3d00 msgr2=0x55dff1ad40e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:12.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.672+0000 7f53b81d1740 1 -- 172.21.15.114:0/1723416483 shutdown_connections 2024-09-13T18:24:12.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.672+0000 7f53b81d1740 1 -- 172.21.15.114:0/1723416483 wait complete. 2024-09-13T18:24:12.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.672+0000 7f53b81d1740 1 -- start start 2024-09-13T18:24:12.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.672+0000 7f53b81d1740 1 Processor -- start 2024-09-13T18:24:12.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.673+0000 7f53b81d1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dff1ad7790 0x55dff1ae1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.673+0000 7f53b81d1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff1ad80b0 0x55dff1ae2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.673+0000 7f53b81d1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dff1ae2950 0x55dff1adf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.673+0000 7f53b81d1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dff19f9180 con 0x55dff1ad7790 2024-09-13T18:24:12.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.673+0000 7f53b81d1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dff1713430 con 0x55dff1ad80b0 2024-09-13T18:24:12.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.673+0000 7f53b81d1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dff19be5e0 con 0x55dff1ae2950 2024-09-13T18:24:12.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.673+0000 7f53ae7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff1ad80b0 0x55dff1ae2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.673+0000 7f53aeffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dff1ad7790 0x55dff1ae1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.673+0000 7f53ae7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff1ad80b0 0x55dff1ae2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:51876/0 (socket says 172.21.15.114:51876) 2024-09-13T18:24:12.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.673+0000 7f53ae7fc640 1 -- 172.21.15.114:0/3439812391 learned_addr learned my addr 172.21.15.114:0/3439812391 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:12.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.673+0000 7f53af7fe640 1 --2- 172.21.15.114:0/3439812391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dff1ae2950 0x55dff1adf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.673+0000 7f53b81d1740 10 client.0 _ll_register_callbacks cb 0x55dff1a2aaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:24:12.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.673+0000 7f53ae7fc640 1 -- 172.21.15.114:0/3439812391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dff1ae2950 msgr2=0x55dff1adf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:12.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.673+0000 7f53ae7fc640 1 --2- 172.21.15.114:0/3439812391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dff1ae2950 0x55dff1adf450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.673+0000 7f53ae7fc640 1 -- 172.21.15.114:0/3439812391 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dff1ad7790 msgr2=0x55dff1ae1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:12.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.673+0000 7f53ae7fc640 1 --2- 172.21.15.114:0/3439812391 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dff1ad7790 0x55dff1ae1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.673+0000 7f53ae7fc640 1 -- 172.21.15.114:0/3439812391 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53b0114b90 con 0x55dff1ad80b0 2024-09-13T18:24:12.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.673+0000 7f53ae7fc640 1 --2- 172.21.15.114:0/3439812391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff1ad80b0 0x55dff1ae2380 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f53a409a050 tx=0x7f53a4099f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc9b71014fc5e1b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:12.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.673+0000 7f5392ff5640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:24:12.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.674+0000 7f5392ff5640 1 -- 172.21.15.114:0/3439812391 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a40cd170 con 0x55dff1ad80b0 2024-09-13T18:24:12.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.674+0000 7f53b81d1740 1 -- 172.21.15.114:0/3439812391 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dff1a64b80 con 0x55dff1ad80b0 2024-09-13T18:24:12.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.674+0000 7f5392ff5640 1 -- 172.21.15.114:0/3439812391 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f53a409de40 con 0x55dff1ad80b0 2024-09-13T18:24:12.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.674+0000 7f5392ff5640 1 -- 172.21.15.114:0/3439812391 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53a40a2490 con 0x55dff1ad80b0 2024-09-13T18:24:12.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.674+0000 7f53b81d1740 10 client.25673 fetch_fsmap learned FSMap version 32 2024-09-13T18:24:12.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.674+0000 7f53b81d1740 1 -- 172.21.15.114:0/3439812391 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55dff1a393c0 con 0x55dff1ad80b0 2024-09-13T18:24:12.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.675+0000 7f5392ff5640 1 -- 172.21.15.114:0/3439812391 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f53a40b6080 con 0x55dff1ad80b0 2024-09-13T18:24:12.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.675+0000 7f5392ff5640 1 -- 172.21.15.114:0/3439812391 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f53a40cf180 con 0x55dff1ad80b0 2024-09-13T18:24:12.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.675+0000 7f53b81d1740 10 client.25673 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:24:12.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.675+0000 7f53b81d1740 10 client.25673 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-13T18:24:12.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.675+0000 7f53b81d1740 20 client.25673 populate_metadata read hostname 'smithi114' 2024-09-13T18:24:12.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.675+0000 7f53b81d1740 1 --2- 172.21.15.114:0/3439812391 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55dff1af2370 0x55dff1af4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.675+0000 7f53b81d1740 4 client.25673 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:24:12.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.675+0000 7f53b81d1740 1 -- 172.21.15.114:0/3439812391 --> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dff1a2d1f0 con 0x55dff1af2370 2024-09-13T18:24:12.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.676+0000 7f53aeffd640 1 --2- 172.21.15.114:0/3439812391 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55dff1af2370 0x55dff1af4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.676+0000 7f53aeffd640 1 --2- 172.21.15.114:0/3439812391 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55dff1af2370 0x55dff1af4790 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2ebac3999414ca1a server_cookie=ef939869c7ac1486 in_seq=0 out_seq=0 2024-09-13T18:24:12.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.676+0000 7f5392ff5640 10 client.25673 ms_handle_connect on v2:172.21.15.134:6836/891692210 2024-09-13T18:24:12.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.677+0000 7f5392ff5640 1 -- 172.21.15.114:0/3439812391 <== mds.1 v2:172.21.15.134:6836/891692210 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55dff1a2d1f0 con 0x55dff1af2370 2024-09-13T18:24:12.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.677+0000 7f5392ff5640 10 client.25673 handle_command_reply: tid=0 2024-09-13T18:24:12.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.677+0000 7f53b81d1740 1 client.25673 shutdown 2024-09-13T18:24:12.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.677+0000 7f53b81d1740 1 -- 172.21.15.114:0/3439812391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff1ad80b0 msgr2=0x55dff1ae2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:12.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.677+0000 7f53b81d1740 1 --2- 172.21.15.114:0/3439812391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff1ad80b0 0x55dff1ae2380 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f53a409a050 tx=0x7f53a4099f50 comp rx=0 tx=0).stop 2024-09-13T18:24:12.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.677+0000 7f53b81d1740 1 -- 172.21.15.114:0/3439812391 shutdown_connections 2024-09-13T18:24:12.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.677+0000 7f53b81d1740 1 --2- 172.21.15.114:0/3439812391 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55dff1af2370 0x55dff1af4790 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.677+0000 7f53b81d1740 1 --2- 172.21.15.114:0/3439812391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dff1ae2950 0x55dff1adf450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.677+0000 7f53b81d1740 1 --2- 172.21.15.114:0/3439812391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff1ad80b0 0x55dff1ae2380 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.677+0000 7f53b81d1740 1 --2- 172.21.15.114:0/3439812391 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dff1ad7790 0x55dff1ae1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.677+0000 7f53b81d1740 1 -- 172.21.15.114:0/3439812391 >> 172.21.15.114:0/3439812391 conn(0x55dff1ad3d00 msgr2=0x55dff1ad9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:12.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.677+0000 7f5392ff5640 0 client.25673 ms_handle_reset on v2:172.21.15.134:6836/891692210 2024-09-13T18:24:12.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.677+0000 7f53b81d1740 1 -- 172.21.15.114:0/3439812391 shutdown_connections 2024-09-13T18:24:12.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.677+0000 7f53b81d1740 1 -- 172.21.15.114:0/3439812391 wait complete. 2024-09-13T18:24:12.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.678+0000 7f53b81d1740 20 client.25673 trim_cache size 0 max 16384 2024-09-13T18:24:12.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.694+0000 7f53b81d1740 1 Processor -- start 2024-09-13T18:24:12.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.694+0000 7f53b81d1740 1 -- start start 2024-09-13T18:24:12.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.694+0000 7f53b81d1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dff1ad7790 0x55dff1ae9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.694+0000 7f53b81d1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dff1ad80b0 0x55dff1ae9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.694+0000 7f53b81d1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff1ae2950 0x55dff1addab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.694+0000 7f53b81d1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dff1adf0c0 con 0x55dff1ad7790 2024-09-13T18:24:12.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.694+0000 7f53b81d1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dff1af0b20 con 0x55dff1ae2950 2024-09-13T18:24:12.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.694+0000 7f53b81d1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dff1acf910 con 0x55dff1ad80b0 2024-09-13T18:24:12.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.694+0000 7f53aeffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dff1ad7790 0x55dff1ae9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.694+0000 7f53aeffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dff1ad7790 0x55dff1ae9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43584/0 (socket says 172.21.15.114:43584) 2024-09-13T18:24:12.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.694+0000 7f53aeffd640 1 -- 172.21.15.114:0/3035571439 learned_addr learned my addr 172.21.15.114:0/3035571439 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:12.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.695+0000 7f53ae7fc640 1 --2- 172.21.15.114:0/3035571439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dff1ad80b0 0x55dff1ae9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.695+0000 7f53aeffd640 1 -- 172.21.15.114:0/3035571439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dff1ad80b0 msgr2=0x55dff1ae9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:12.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.695+0000 7f53aeffd640 1 --2- 172.21.15.114:0/3035571439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dff1ad80b0 0x55dff1ae9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.695+0000 7f53aeffd640 1 -- 172.21.15.114:0/3035571439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff1ae2950 msgr2=0x55dff1addab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:12.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.695+0000 7f53aeffd640 1 --2- 172.21.15.114:0/3035571439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff1ae2950 0x55dff1addab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.695+0000 7f53aeffd640 1 -- 172.21.15.114:0/3035571439 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dff19db770 con 0x55dff1ad7790 2024-09-13T18:24:12.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.695+0000 7f53aeffd640 1 --2- 172.21.15.114:0/3035571439 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dff1ad7790 0x55dff1ae9670 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f53a0004700 tx=0x7f53a0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f6d018e5d6b379 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:12.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.695+0000 7f5395ffb640 1 -- 172.21.15.114:0/3035571439 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a0022040 con 0x55dff1ad7790 2024-09-13T18:24:12.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.695+0000 7f5395ffb640 1 -- 172.21.15.114:0/3035571439 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f53a001dd70 con 0x55dff1ad7790 2024-09-13T18:24:12.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.695+0000 7f5395ffb640 1 -- 172.21.15.114:0/3035571439 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a0005540 con 0x55dff1ad7790 2024-09-13T18:24:12.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.696+0000 7f53b81d1740 1 -- 172.21.15.114:0/3035571439 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dff1ad7790 msgr2=0x55dff1ae9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:12.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.696+0000 7f53b81d1740 1 --2- 172.21.15.114:0/3035571439 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dff1ad7790 0x55dff1ae9670 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f53a0004700 tx=0x7f53a0014000 comp rx=0 tx=0).stop 2024-09-13T18:24:12.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.696+0000 7f53b81d1740 1 -- 172.21.15.114:0/3035571439 shutdown_connections 2024-09-13T18:24:12.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.696+0000 7f53b81d1740 1 --2- 172.21.15.114:0/3035571439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff1ae2950 0x55dff1addab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.696+0000 7f53b81d1740 1 --2- 172.21.15.114:0/3035571439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dff1ad80b0 0x55dff1ae9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.696+0000 7f53b81d1740 1 --2- 172.21.15.114:0/3035571439 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dff1ad7790 0x55dff1ae9670 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.696+0000 7f53b81d1740 1 -- 172.21.15.114:0/3035571439 >> 172.21.15.114:0/3035571439 conn(0x55dff1ad3d00 msgr2=0x55dff1af90f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:12.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.696+0000 7f53b81d1740 1 -- 172.21.15.114:0/3035571439 shutdown_connections 2024-09-13T18:24:12.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.696+0000 7f53b81d1740 1 -- 172.21.15.114:0/3035571439 wait complete. 2024-09-13T18:24:12.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.697+0000 7f53b81d1740 1 -- start start 2024-09-13T18:24:12.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.697+0000 7f53b81d1740 1 Processor -- start 2024-09-13T18:24:12.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.697+0000 7f53b81d1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dff1ad7790 0x55dff1ae9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.697+0000 7f53b81d1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff1ad80b0 0x55dff1ae9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.697+0000 7f53b81d1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dff1ae2950 0x55dff1ae7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.697+0000 7f53b81d1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dff1ae7ee0 con 0x55dff1ae2950 2024-09-13T18:24:12.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.697+0000 7f53b81d1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dff1ae8050 con 0x55dff1ad80b0 2024-09-13T18:24:12.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.697+0000 7f53b81d1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dff1ae8210 con 0x55dff1ad7790 2024-09-13T18:24:12.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.697+0000 7f53b81d1740 10 client.0 _ll_register_callbacks cb 0x7f53a0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:24:12.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.697+0000 7f53af7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dff1ae2950 0x55dff1ae7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.697+0000 7f53ae7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff1ad80b0 0x55dff1ae9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.697+0000 7f53af7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dff1ae2950 0x55dff1ae7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43594/0 (socket says 172.21.15.114:43594) 2024-09-13T18:24:12.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.697+0000 7f53af7fe640 1 -- 172.21.15.114:0/406334765 learned_addr learned my addr 172.21.15.114:0/406334765 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:12.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.697+0000 7f53aeffd640 1 --2- 172.21.15.114:0/406334765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dff1ad7790 0x55dff1ae9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.698+0000 7f53af7fe640 1 -- 172.21.15.114:0/406334765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dff1ad7790 msgr2=0x55dff1ae9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:12.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.698+0000 7f53af7fe640 1 --2- 172.21.15.114:0/406334765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dff1ad7790 0x55dff1ae9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.698+0000 7f53af7fe640 1 -- 172.21.15.114:0/406334765 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff1ad80b0 msgr2=0x55dff1ae9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:12.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.698+0000 7f53af7fe640 1 --2- 172.21.15.114:0/406334765 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff1ad80b0 0x55dff1ae9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.698+0000 7f53af7fe640 1 -- 172.21.15.114:0/406334765 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53a0017070 con 0x55dff1ae2950 2024-09-13T18:24:12.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.698+0000 7f53aeffd640 1 --2- 172.21.15.114:0/406334765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dff1ad7790 0x55dff1ae9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:24:12.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.698+0000 7f53af7fe640 1 --2- 172.21.15.114:0/406334765 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dff1ae2950 0x55dff1ae7920 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f53a8005cc0 tx=0x7f53a8003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68441e0cff1309f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:12.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.698+0000 7f5392ff5640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:24:12.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.698+0000 7f5392ff5640 1 -- 172.21.15.114:0/406334765 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a8031d70 con 0x55dff1ae2950 2024-09-13T18:24:12.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.698+0000 7f5392ff5640 1 -- 172.21.15.114:0/406334765 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f53a80305c0 con 0x55dff1ae2950 2024-09-13T18:24:12.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.698+0000 7f53b81d1740 1 -- 172.21.15.114:0/406334765 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f53a80318e0 con 0x55dff1ae2950 2024-09-13T18:24:12.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.699+0000 7f5392ff5640 1 -- 172.21.15.114:0/406334765 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53a8030780 con 0x55dff1ae2950 2024-09-13T18:24:12.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.699+0000 7f53b81d1740 10 client.16344 fetch_fsmap learned FSMap version 32 2024-09-13T18:24:12.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.699+0000 7f53b81d1740 1 -- 172.21.15.114:0/406334765 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55dff1a00e80 con 0x55dff1ae2950 2024-09-13T18:24:12.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.699+0000 7f5392ff5640 1 -- 172.21.15.114:0/406334765 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f53a80318e0 con 0x55dff1ae2950 2024-09-13T18:24:12.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.699+0000 7f5392ff5640 1 -- 172.21.15.114:0/406334765 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f53a802fcc0 con 0x55dff1ae2950 2024-09-13T18:24:12.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.700+0000 7f53b81d1740 10 client.16344 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:24:12.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.700+0000 7f53b81d1740 10 client.16344 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-13T18:24:12.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.700+0000 7f53b81d1740 20 client.16344 populate_metadata read hostname 'smithi114' 2024-09-13T18:24:12.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.700+0000 7f53b81d1740 1 --2- 172.21.15.114:0/406334765 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55dff1b497e0 0x55dff1b4bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.700+0000 7f53b81d1740 4 client.16344 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:24:12.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.700+0000 7f53b81d1740 1 -- 172.21.15.114:0/406334765 --> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dff172e4e0 con 0x55dff1b497e0 2024-09-13T18:24:12.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.700+0000 7f53aeffd640 1 --2- 172.21.15.114:0/406334765 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55dff1b497e0 0x55dff1b4bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.701+0000 7f53aeffd640 1 --2- 172.21.15.114:0/406334765 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55dff1b497e0 0x55dff1b4bc00 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3835c69ec42f83eb server_cookie=f5e6140c50c7ff7b in_seq=0 out_seq=0 2024-09-13T18:24:12.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.701+0000 7f5392ff5640 10 client.16344 ms_handle_connect on v2:172.21.15.134:6836/891692210 2024-09-13T18:24:12.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.701+0000 7f5392ff5640 1 -- 172.21.15.114:0/406334765 <== mds.1 v2:172.21.15.134:6836/891692210 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dff172e4e0 con 0x55dff1b497e0 2024-09-13T18:24:12.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.701+0000 7f5392ff5640 10 client.16344 handle_command_reply: tid=0 2024-09-13T18:24:12.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.701+0000 7f53b81d1740 1 client.16344 shutdown 2024-09-13T18:24:12.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.701+0000 7f53b81d1740 1 -- 172.21.15.114:0/406334765 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dff1ae2950 msgr2=0x55dff1ae7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:12.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.701+0000 7f53b81d1740 1 --2- 172.21.15.114:0/406334765 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dff1ae2950 0x55dff1ae7920 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f53a8005cc0 tx=0x7f53a8003ba0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.701+0000 7f53b81d1740 1 -- 172.21.15.114:0/406334765 shutdown_connections 2024-09-13T18:24:12.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.701+0000 7f53b81d1740 1 --2- 172.21.15.114:0/406334765 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55dff1b497e0 0x55dff1b4bc00 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.701+0000 7f53b81d1740 1 --2- 172.21.15.114:0/406334765 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dff1ae2950 0x55dff1ae7920 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.701+0000 7f5392ff5640 0 client.16344 ms_handle_reset on v2:172.21.15.134:6836/891692210 2024-09-13T18:24:12.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.701+0000 7f53b81d1740 1 --2- 172.21.15.114:0/406334765 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff1ad80b0 0x55dff1ae9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.701+0000 7f53b81d1740 1 --2- 172.21.15.114:0/406334765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dff1ad7790 0x55dff1ae9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.701+0000 7f53b81d1740 1 -- 172.21.15.114:0/406334765 >> 172.21.15.114:0/406334765 conn(0x55dff1ad3d00 msgr2=0x55dff1add9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:12.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.701+0000 7f53b81d1740 1 -- 172.21.15.114:0/406334765 shutdown_connections 2024-09-13T18:24:12.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.701+0000 7f53b81d1740 1 -- 172.21.15.114:0/406334765 wait complete. 2024-09-13T18:24:12.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.702+0000 7f53b81d1740 20 client.16344 trim_cache size 0 max 16384 2024-09-13T18:24:12.709 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:24:12.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.702+0000 7f53b53d4640 1 -- 172.21.15.114:0/883305540 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f53980a1280 msgr2=0x7f53980a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:12.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.702+0000 7f53b53d4640 1 --2- 172.21.15.114:0/883305540 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f53980a1280 0x7f53980a3740 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x55dff1ad4b60 tx=0x7f53a0009840 comp rx=0 tx=0).stop 2024-09-13T18:24:12.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.702+0000 7f53b53d4640 1 -- 172.21.15.114:0/883305540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53b0108f70 msgr2=0x7f53b01a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:12.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.702+0000 7f53b53d4640 1 --2- 172.21.15.114:0/883305540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53b0108f70 0x7f53b01a4c10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f53a400e9f0 tx=0x7f53a400eeb0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.702+0000 7f53b53d4640 1 -- 172.21.15.114:0/883305540 shutdown_connections 2024-09-13T18:24:12.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.702+0000 7f53b53d4640 1 --2- 172.21.15.114:0/883305540 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f53980a1280 0x7f53980a3740 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.702+0000 7f53b53d4640 1 --2- 172.21.15.114:0/883305540 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53b0111b70 0x7f53b019e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.702+0000 7f53b53d4640 1 --2- 172.21.15.114:0/883305540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53b0108f70 0x7f53b01a4c10 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.702+0000 7f53b53d4640 1 --2- 172.21.15.114:0/883305540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53b01085c0 0x7f53b01a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.702+0000 7f53b53d4640 1 -- 172.21.15.114:0/883305540 >> 172.21.15.114:0/883305540 conn(0x7f53b006b440 msgr2=0x7f53b0107890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:12.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.703+0000 7f53b53d4640 1 -- 172.21.15.114:0/883305540 shutdown_connections 2024-09-13T18:24:12.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.703+0000 7f53b53d4640 1 -- 172.21.15.114:0/883305540 wait complete. 2024-09-13T18:24:12.721 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-09-13T18:24:12.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.871+0000 7fe89e55a640 1 -- 172.21.15.114:0/3829038287 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8980ff810 msgr2=0x7fe898103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:12.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.871+0000 7fe89e55a640 1 --2- 172.21.15.114:0/3829038287 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8980ff810 0x7fe898103850 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fe88c00d940 tx=0x7fe88c02c3b0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.871+0000 7fe89e55a640 1 -- 172.21.15.114:0/3829038287 shutdown_connections 2024-09-13T18:24:12.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.871+0000 7fe89e55a640 1 --2- 172.21.15.114:0/3829038287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe898103fb0 0x7fe89810bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.871+0000 7fe89e55a640 1 --2- 172.21.15.114:0/3829038287 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8980ff810 0x7fe898103850 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.871+0000 7fe89e55a640 1 --2- 172.21.15.114:0/3829038287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8980fee60 0x7fe8980ff240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.871+0000 7fe89e55a640 1 -- 172.21.15.114:0/3829038287 >> 172.21.15.114:0/3829038287 conn(0x7fe898075c60 msgr2=0x7fe8980fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:12.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.871+0000 7fe89e55a640 1 -- 172.21.15.114:0/3829038287 shutdown_connections 2024-09-13T18:24:12.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.871+0000 7fe89e55a640 1 -- 172.21.15.114:0/3829038287 wait complete. 2024-09-13T18:24:12.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.872+0000 7fe89e55a640 1 Processor -- start 2024-09-13T18:24:12.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.872+0000 7fe89e55a640 1 -- start start 2024-09-13T18:24:12.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.872+0000 7fe89e55a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8980fee60 0x7fe898193ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.872+0000 7fe89e55a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe898103fb0 0x7fe898194530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.872+0000 7fe89e55a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe898198920 0x7fe898198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.872+0000 7fe89e55a640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe898199310 con 0x7fe8980fee60 2024-09-13T18:24:12.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.872+0000 7fe89e55a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe898199480 con 0x7fe898103fb0 2024-09-13T18:24:12.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.872+0000 7fe89e55a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8981995f0 con 0x7fe898198920 2024-09-13T18:24:12.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.873+0000 7fe897fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8980fee60 0x7fe898193ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.873+0000 7fe897fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8980fee60 0x7fe898193ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43616/0 (socket says 172.21.15.114:43616) 2024-09-13T18:24:12.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.873+0000 7fe897fff640 1 -- 172.21.15.114:0/2746496608 learned_addr learned my addr 172.21.15.114:0/2746496608 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:12.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.873+0000 7fe8977fe640 1 --2- 172.21.15.114:0/2746496608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe898103fb0 0x7fe898194530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.873+0000 7fe89caf7640 1 --2- 172.21.15.114:0/2746496608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe898198920 0x7fe898198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.873+0000 7fe897fff640 1 -- 172.21.15.114:0/2746496608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe898198920 msgr2=0x7fe898198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:12.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.873+0000 7fe897fff640 1 --2- 172.21.15.114:0/2746496608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe898198920 0x7fe898198dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.873+0000 7fe897fff640 1 -- 172.21.15.114:0/2746496608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe898103fb0 msgr2=0x7fe898194530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:12.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.873+0000 7fe897fff640 1 --2- 172.21.15.114:0/2746496608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe898103fb0 0x7fe898194530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.873+0000 7fe897fff640 1 -- 172.21.15.114:0/2746496608 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe88c00d570 con 0x7fe8980fee60 2024-09-13T18:24:12.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.873+0000 7fe8977fe640 1 --2- 172.21.15.114:0/2746496608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe898103fb0 0x7fe898194530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:24:12.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.873+0000 7fe897fff640 1 --2- 172.21.15.114:0/2746496608 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8980fee60 0x7fe898193ff0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fe88800e9f0 tx=0x7fe88800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:12.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.873+0000 7fe8957fa640 1 -- 172.21.15.114:0/2746496608 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe888009700 con 0x7fe8980fee60 2024-09-13T18:24:12.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.874+0000 7fe89e55a640 1 -- 172.21.15.114:0/2746496608 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8981a4fe0 con 0x7fe8980fee60 2024-09-13T18:24:12.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.874+0000 7fe8957fa640 1 -- 172.21.15.114:0/2746496608 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe88800cd80 con 0x7fe8980fee60 2024-09-13T18:24:12.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.874+0000 7fe8957fa640 1 -- 172.21.15.114:0/2746496608 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8880187e0 con 0x7fe8980fee60 2024-09-13T18:24:12.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.874+0000 7fe89e55a640 1 -- 172.21.15.114:0/2746496608 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8981a5400 con 0x7fe8980fee60 2024-09-13T18:24:12.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.877+0000 7fe8957fa640 1 -- 172.21.15.114:0/2746496608 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fe888018940 con 0x7fe8980fee60 2024-09-13T18:24:12.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.877+0000 7fe8a1278740 1 Processor -- start 2024-09-13T18:24:12.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.877+0000 7fe8a1278740 1 -- start start 2024-09-13T18:24:12.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.878+0000 7fe8a1278740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4786bd790 0x55c4786bdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.878+0000 7fe8a1278740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4786be0b0 0x55c4786c8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.878+0000 7fe8a1278740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c4786c8950 0x55c4786cad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.878+0000 7fe8a1278740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c478627cc0 con 0x55c4786c8950 2024-09-13T18:24:12.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.878+0000 7fe8a1278740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c478625450 con 0x55c4786be0b0 2024-09-13T18:24:12.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.878+0000 7fe8a1278740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c478620d40 con 0x55c4786bd790 2024-09-13T18:24:12.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.878+0000 7fe8977fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c4786c8950 0x55c4786cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.878+0000 7fe89caf7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4786bd790 0x55c4786bdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.878+0000 7fe8957fa640 1 --2- 172.21.15.114:0/2746496608 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe8800a1300 0x7fe8800a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.878+0000 7fe8977fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c4786c8950 0x55c4786cad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43626/0 (socket says 172.21.15.114:43626) 2024-09-13T18:24:12.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.878+0000 7fe8977fe640 1 -- 172.21.15.114:0/3410853287 learned_addr learned my addr 172.21.15.114:0/3410853287 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:12.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.878+0000 7fe897fff640 1 --2- 172.21.15.114:0/3410853287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4786be0b0 0x55c4786c8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.878+0000 7fe89caf7640 1 --2- 172.21.15.114:0/2746496608 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe8800a1300 0x7fe8800a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.878+0000 7fe8957fa640 1 -- 172.21.15.114:0/2746496608 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fe888014070 con 0x7fe8980fee60 2024-09-13T18:24:12.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.878+0000 7fe8977fe640 1 -- 172.21.15.114:0/3410853287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4786bd790 msgr2=0x55c4786bdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:12.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.878+0000 7fe8977fe640 1 --2- 172.21.15.114:0/3410853287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4786bd790 0x55c4786bdb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.878+0000 7fe8977fe640 1 -- 172.21.15.114:0/3410853287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4786be0b0 msgr2=0x55c4786c8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:12.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.878+0000 7fe8977fe640 1 --2- 172.21.15.114:0/3410853287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4786be0b0 0x55c4786c8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.878+0000 7fe8977fe640 1 -- 172.21.15.114:0/3410853287 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4785e8620 con 0x55c4786c8950 2024-09-13T18:24:12.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.879+0000 7fe8977fe640 1 --2- 172.21.15.114:0/3410853287 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c4786c8950 0x55c4786cad40 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe88c005ab0 tx=0x7fe88c005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cc28fb0bfc73672 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:12.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.879+0000 7fe89caf7640 1 --2- 172.21.15.114:0/2746496608 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe8800a1300 0x7fe8800a37c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe890005fd0 tx=0x7fe890004ab0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:12.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.879+0000 7fe87e7fc640 1 -- 172.21.15.114:0/3410853287 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88c004740 con 0x55c4786c8950 2024-09-13T18:24:12.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.879+0000 7fe87e7fc640 1 -- 172.21.15.114:0/3410853287 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe88c004e40 con 0x55c4786c8950 2024-09-13T18:24:12.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.880+0000 7fe87e7fc640 1 -- 172.21.15.114:0/3410853287 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88c03e9d0 con 0x55c4786c8950 2024-09-13T18:24:12.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.880+0000 7fe8a1278740 1 -- 172.21.15.114:0/3410853287 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c4786c8950 msgr2=0x55c4786cad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:12.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.880+0000 7fe8a1278740 1 --2- 172.21.15.114:0/3410853287 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c4786c8950 0x55c4786cad40 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe88c005ab0 tx=0x7fe88c005620 comp rx=0 tx=0).stop 2024-09-13T18:24:12.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.880+0000 7fe8a1278740 1 -- 172.21.15.114:0/3410853287 shutdown_connections 2024-09-13T18:24:12.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.880+0000 7fe8a1278740 1 --2- 172.21.15.114:0/3410853287 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c4786c8950 0x55c4786cad40 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.880+0000 7fe8a1278740 1 --2- 172.21.15.114:0/3410853287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4786be0b0 0x55c4786c8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.880+0000 7fe8a1278740 1 --2- 172.21.15.114:0/3410853287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4786bd790 0x55c4786bdb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.880+0000 7fe8a1278740 1 -- 172.21.15.114:0/3410853287 >> 172.21.15.114:0/3410853287 conn(0x55c4786b9d00 msgr2=0x55c4786ba0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:12.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.880+0000 7fe8a1278740 1 -- 172.21.15.114:0/3410853287 shutdown_connections 2024-09-13T18:24:12.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.880+0000 7fe8a1278740 1 -- 172.21.15.114:0/3410853287 wait complete. 2024-09-13T18:24:12.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.881+0000 7fe8a1278740 1 -- start start 2024-09-13T18:24:12.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.881+0000 7fe8a1278740 1 Processor -- start 2024-09-13T18:24:12.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.881+0000 7fe8a1278740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4786bd790 0x55c4786c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.881+0000 7fe8a1278740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c4786be0b0 0x55c4786c52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.881+0000 7fe8a1278740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4786c8950 0x55c4786c57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.882+0000 7fe8a1278740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4785df180 con 0x55c4786be0b0 2024-09-13T18:24:12.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.882+0000 7fe8a1278740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4782f9430 con 0x55c4786c8950 2024-09-13T18:24:12.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.882+0000 7fe8a1278740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4785a45e0 con 0x55c4786bd790 2024-09-13T18:24:12.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.882+0000 7fe897fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4786bd790 0x55c4786c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.882+0000 7fe897fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4786bd790 0x55c4786c7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:43666/0 (socket says 172.21.15.114:43666) 2024-09-13T18:24:12.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.882+0000 7fe8977fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c4786be0b0 0x55c4786c52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.882+0000 7fe897fff640 1 -- 172.21.15.114:0/1801267551 learned_addr learned my addr 172.21.15.114:0/1801267551 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:12.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.882+0000 7fe8a1278740 10 client.0 _ll_register_callbacks cb 0x55c478610aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:24:12.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.882+0000 7fe89caf7640 1 --2- 172.21.15.114:0/1801267551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4786c8950 0x55c4786c57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.882+0000 7fe897fff640 1 -- 172.21.15.114:0/1801267551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4786c8950 msgr2=0x55c4786c57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:12.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.882+0000 7fe897fff640 1 --2- 172.21.15.114:0/1801267551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4786c8950 0x55c4786c57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.882+0000 7fe897fff640 1 -- 172.21.15.114:0/1801267551 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c4786be0b0 msgr2=0x55c4786c52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:12.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.882+0000 7fe897fff640 1 --2- 172.21.15.114:0/1801267551 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c4786be0b0 0x55c4786c52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.882+0000 7fe897fff640 1 -- 172.21.15.114:0/1801267551 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe88c038040 con 0x55c4786bd790 2024-09-13T18:24:12.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.882+0000 7fe89caf7640 1 --2- 172.21.15.114:0/1801267551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4786c8950 0x55c4786c57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:24:12.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.882+0000 7fe897fff640 1 --2- 172.21.15.114:0/1801267551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4786bd790 0x55c4786c7f20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe888099e70 tx=0x7fe88801cea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=169837f4eca2e4a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:12.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.882+0000 7fe8777fe640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:24:12.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.882+0000 7fe8777fe640 1 -- 172.21.15.114:0/1801267551 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88809d070 con 0x55c4786bd790 2024-09-13T18:24:12.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.882+0000 7fe8777fe640 1 -- 172.21.15.114:0/1801267551 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fe888014eb0 con 0x55c4786bd790 2024-09-13T18:24:12.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.882+0000 7fe8a1278740 1 -- 172.21.15.114:0/1801267551 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c47864ab80 con 0x55c4786bd790 2024-09-13T18:24:12.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.882+0000 7fe8777fe640 1 -- 172.21.15.114:0/1801267551 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe88806a210 con 0x55c4786bd790 2024-09-13T18:24:12.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.883+0000 7fe8a1278740 10 client.25732 fetch_fsmap learned FSMap version 32 2024-09-13T18:24:12.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.883+0000 7fe8a1278740 1 -- 172.21.15.114:0/1801267551 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55c47861f3c0 con 0x55c4786bd790 2024-09-13T18:24:12.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.883+0000 7fe8777fe640 1 -- 172.21.15.114:0/1801267551 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe888093080 con 0x55c4786bd790 2024-09-13T18:24:12.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.883+0000 7fe8777fe640 1 -- 172.21.15.114:0/1801267551 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fe888096880 con 0x55c4786bd790 2024-09-13T18:24:12.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.883+0000 7fe8a1278740 10 client.25732 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:24:12.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.883+0000 7fe8a1278740 10 client.25732 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:24:12.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.883+0000 7fe8a1278740 20 client.25732 populate_metadata read hostname 'smithi114' 2024-09-13T18:24:12.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.883+0000 7fe8a1278740 1 --2- 172.21.15.114:0/1801267551 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55c4786d83f0 0x55c4786da810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.883+0000 7fe8a1278740 4 client.25732 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:24:12.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.883+0000 7fe8a1278740 1 -- 172.21.15.114:0/1801267551 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c4786131f0 con 0x55c4786d83f0 2024-09-13T18:24:12.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.883+0000 7fe8977fe640 1 --2- 172.21.15.114:0/1801267551 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55c4786d83f0 0x55c4786da810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.884+0000 7fe8977fe640 1 --2- 172.21.15.114:0/1801267551 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55c4786d83f0 0x55c4786da810 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d369a5bf2c5e0f8e server_cookie=14a2e128d04d67ac in_seq=0 out_seq=0 2024-09-13T18:24:12.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.884+0000 7fe8777fe640 10 client.25732 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:24:12.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.884+0000 7fe8777fe640 1 -- 172.21.15.114:0/1801267551 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55c4786131f0 con 0x55c4786d83f0 2024-09-13T18:24:12.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.884+0000 7fe8777fe640 10 client.25732 handle_command_reply: tid=0 2024-09-13T18:24:12.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.884+0000 7fe8a1278740 1 client.25732 shutdown 2024-09-13T18:24:12.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.884+0000 7fe8a1278740 1 -- 172.21.15.114:0/1801267551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4786bd790 msgr2=0x55c4786c7f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:12.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.884+0000 7fe8a1278740 1 --2- 172.21.15.114:0/1801267551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4786bd790 0x55c4786c7f20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe888099e70 tx=0x7fe88801cea0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.885+0000 7fe8a1278740 1 -- 172.21.15.114:0/1801267551 shutdown_connections 2024-09-13T18:24:12.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.885+0000 7fe8a1278740 1 --2- 172.21.15.114:0/1801267551 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55c4786d83f0 0x55c4786da810 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.885+0000 7fe8a1278740 1 --2- 172.21.15.114:0/1801267551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4786c8950 0x55c4786c57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.885+0000 7fe8a1278740 1 --2- 172.21.15.114:0/1801267551 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c4786be0b0 0x55c4786c52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.885+0000 7fe8777fe640 0 client.25732 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:24:12.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.885+0000 7fe8a1278740 1 --2- 172.21.15.114:0/1801267551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4786bd790 0x55c4786c7f20 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.885+0000 7fe8a1278740 1 -- 172.21.15.114:0/1801267551 >> 172.21.15.114:0/1801267551 conn(0x55c4786b9d00 msgr2=0x55c4786bfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:12.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.885+0000 7fe8a1278740 1 -- 172.21.15.114:0/1801267551 shutdown_connections 2024-09-13T18:24:12.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.885+0000 7fe8a1278740 1 -- 172.21.15.114:0/1801267551 wait complete. 2024-09-13T18:24:12.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.885+0000 7fe8a1278740 20 client.25732 trim_cache size 0 max 16384 2024-09-13T18:24:12.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.904+0000 7fe8a1278740 1 Processor -- start 2024-09-13T18:24:12.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.904+0000 7fe8a1278740 1 -- start start 2024-09-13T18:24:12.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.904+0000 7fe8a1278740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c4786bd790 0x55c4786c79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.904+0000 7fe8a1278740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4786be0b0 0x55c4786c7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.904+0000 7fe8a1278740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4786c8950 0x55c4786ba0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.904+0000 7fe8a1278740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4786bae90 con 0x55c4786bd790 2024-09-13T18:24:12.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.905+0000 7fe8a1278740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4786c37e0 con 0x55c4786be0b0 2024-09-13T18:24:12.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.905+0000 7fe8a1278740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4786b5440 con 0x55c4786c8950 2024-09-13T18:24:12.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.905+0000 7fe897fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c4786bd790 0x55c4786c79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.905+0000 7fe897fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c4786bd790 0x55c4786c79c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43634/0 (socket says 172.21.15.114:43634) 2024-09-13T18:24:12.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.905+0000 7fe897fff640 1 -- 172.21.15.114:0/4247245250 learned_addr learned my addr 172.21.15.114:0/4247245250 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:12.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.905+0000 7fe8977fe640 1 --2- 172.21.15.114:0/4247245250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4786be0b0 0x55c4786c7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.905+0000 7fe89caf7640 1 --2- 172.21.15.114:0/4247245250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4786c8950 0x55c4786ba0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.905+0000 7fe897fff640 1 -- 172.21.15.114:0/4247245250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4786c8950 msgr2=0x55c4786ba0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:12.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.905+0000 7fe897fff640 1 --2- 172.21.15.114:0/4247245250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4786c8950 0x55c4786ba0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.905+0000 7fe897fff640 1 -- 172.21.15.114:0/4247245250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4786be0b0 msgr2=0x55c4786c7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:12.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.905+0000 7fe897fff640 1 --2- 172.21.15.114:0/4247245250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4786be0b0 0x55c4786c7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.905+0000 7fe897fff640 1 -- 172.21.15.114:0/4247245250 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4785c1770 con 0x55c4786bd790 2024-09-13T18:24:12.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.905+0000 7fe8977fe640 1 --2- 172.21.15.114:0/4247245250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4786be0b0 0x55c4786c7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:24:12.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.905+0000 7fe897fff640 1 --2- 172.21.15.114:0/4247245250 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c4786bd790 0x55c4786c79c0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fe8880cd2d0 tx=0x7fe8880999f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=354648c95d90e0b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:12.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.905+0000 7fe87effd640 1 -- 172.21.15.114:0/4247245250 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88809d070 con 0x55c4786bd790 2024-09-13T18:24:12.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.906+0000 7fe87effd640 1 -- 172.21.15.114:0/4247245250 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c47864ab80 con 0x55c4786bd790 2024-09-13T18:24:12.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.906+0000 7fe87effd640 1 -- 172.21.15.114:0/4247245250 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88809def0 con 0x55c4786bd790 2024-09-13T18:24:12.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.906+0000 7fe8a1278740 1 -- 172.21.15.114:0/4247245250 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c4786bd790 msgr2=0x55c4786c79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:12.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.906+0000 7fe8a1278740 1 --2- 172.21.15.114:0/4247245250 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c4786bd790 0x55c4786c79c0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fe8880cd2d0 tx=0x7fe8880999f0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.906+0000 7fe8a1278740 1 -- 172.21.15.114:0/4247245250 shutdown_connections 2024-09-13T18:24:12.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.906+0000 7fe8a1278740 1 --2- 172.21.15.114:0/4247245250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4786c8950 0x55c4786ba0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.906+0000 7fe8a1278740 1 --2- 172.21.15.114:0/4247245250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4786be0b0 0x55c4786c7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.906+0000 7fe8a1278740 1 --2- 172.21.15.114:0/4247245250 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c4786bd790 0x55c4786c79c0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.906+0000 7fe8a1278740 1 -- 172.21.15.114:0/4247245250 >> 172.21.15.114:0/4247245250 conn(0x55c4786b9d00 msgr2=0x55c4786c5270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:12.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.907+0000 7fe8a1278740 1 -- 172.21.15.114:0/4247245250 shutdown_connections 2024-09-13T18:24:12.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.907+0000 7fe8a1278740 1 -- 172.21.15.114:0/4247245250 wait complete. 2024-09-13T18:24:12.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.907+0000 7fe8a1278740 1 -- start start 2024-09-13T18:24:12.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.907+0000 7fe8a1278740 1 Processor -- start 2024-09-13T18:24:12.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.907+0000 7fe8a1278740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4786bd790 0x55c4786cba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.907+0000 7fe8a1278740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4786be0b0 0x55c4786cbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.908+0000 7fe8a1278740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c4786c8950 0x55c47872d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.908+0000 7fe8a1278740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c47872de40 con 0x55c4786c8950 2024-09-13T18:24:12.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.908+0000 7fe8a1278740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c47872dfb0 con 0x55c4786be0b0 2024-09-13T18:24:12.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.908+0000 7fe8a1278740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c47872e170 con 0x55c4786bd790 2024-09-13T18:24:12.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.908+0000 7fe8a1278740 10 client.0 _ll_register_callbacks cb 0x7fe8900093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:24:12.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.908+0000 7fe89caf7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c4786c8950 0x55c47872d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.908+0000 7fe89caf7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c4786c8950 0x55c47872d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43636/0 (socket says 172.21.15.114:43636) 2024-09-13T18:24:12.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.908+0000 7fe89caf7640 1 -- 172.21.15.114:0/413004304 learned_addr learned my addr 172.21.15.114:0/413004304 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:12.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.908+0000 7fe897fff640 1 --2- 172.21.15.114:0/413004304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4786bd790 0x55c4786cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.908+0000 7fe8977fe640 1 --2- 172.21.15.114:0/413004304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4786be0b0 0x55c4786cbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.908+0000 7fe89caf7640 1 -- 172.21.15.114:0/413004304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4786bd790 msgr2=0x55c4786cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:12.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.908+0000 7fe89caf7640 1 --2- 172.21.15.114:0/413004304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4786bd790 0x55c4786cba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.908+0000 7fe89caf7640 1 -- 172.21.15.114:0/413004304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4786be0b0 msgr2=0x55c4786cbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:12.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.908+0000 7fe89caf7640 1 --2- 172.21.15.114:0/413004304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4786be0b0 0x55c4786cbf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.908+0000 7fe89caf7640 1 -- 172.21.15.114:0/413004304 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c4785e8620 con 0x55c4786c8950 2024-09-13T18:24:12.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.908+0000 7fe897fff640 1 --2- 172.21.15.114:0/413004304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4786bd790 0x55c4786cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:24:12.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.908+0000 7fe89caf7640 1 --2- 172.21.15.114:0/413004304 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c4786c8950 0x55c47872d900 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fe890013fd0 tx=0x7fe890004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ec0c58788d01fb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:12.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.908+0000 7fe8777fe640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:24:12.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.909+0000 7fe8777fe640 1 -- 172.21.15.114:0/413004304 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe890026070 con 0x55c4786c8950 2024-09-13T18:24:12.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.909+0000 7fe8777fe640 1 -- 172.21.15.114:0/413004304 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fe890004b20 con 0x55c4786c8950 2024-09-13T18:24:12.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.909+0000 7fe8a1278740 1 -- 172.21.15.114:0/413004304 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe88c004e40 con 0x55c4786c8950 2024-09-13T18:24:12.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.909+0000 7fe8777fe640 1 -- 172.21.15.114:0/413004304 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe890004ce0 con 0x55c4786c8950 2024-09-13T18:24:12.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.909+0000 7fe8a1278740 10 client.16362 fetch_fsmap learned FSMap version 32 2024-09-13T18:24:12.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.909+0000 7fe8a1278740 1 -- 172.21.15.114:0/413004304 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55c47861f3c0 con 0x55c4786c8950 2024-09-13T18:24:12.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.909+0000 7fe8777fe640 1 -- 172.21.15.114:0/413004304 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe88c004e40 con 0x55c4786c8950 2024-09-13T18:24:12.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.909+0000 7fe8777fe640 1 -- 172.21.15.114:0/413004304 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fe89002b470 con 0x55c4786c8950 2024-09-13T18:24:12.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.909+0000 7fe8a1278740 10 client.16362 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:24:12.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.909+0000 7fe8a1278740 10 client.16362 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:24:12.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.909+0000 7fe8a1278740 20 client.16362 populate_metadata read hostname 'smithi114' 2024-09-13T18:24:12.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.909+0000 7fe8a1278740 1 --2- 172.21.15.114:0/413004304 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55c47872fa30 0x55c478731e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:12.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.910+0000 7fe8a1278740 4 client.16362 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:24:12.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.910+0000 7fe8a1278740 1 -- 172.21.15.114:0/413004304 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c4783144e0 con 0x55c47872fa30 2024-09-13T18:24:12.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.910+0000 7fe897fff640 1 --2- 172.21.15.114:0/413004304 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55c47872fa30 0x55c478731e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:12.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.911+0000 7fe897fff640 1 --2- 172.21.15.114:0/413004304 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55c47872fa30 0x55c478731e50 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f4e8975bd0d8d10 server_cookie=c19b03c249beb328 in_seq=0 out_seq=0 2024-09-13T18:24:12.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.911+0000 7fe8777fe640 10 client.16362 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:24:12.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.911+0000 7fe8777fe640 1 -- 172.21.15.114:0/413004304 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c4783144e0 con 0x55c47872fa30 2024-09-13T18:24:12.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.911+0000 7fe8777fe640 10 client.16362 handle_command_reply: tid=0 2024-09-13T18:24:12.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.911+0000 7fe8a1278740 1 client.16362 shutdown 2024-09-13T18:24:12.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.911+0000 7fe8a1278740 1 -- 172.21.15.114:0/413004304 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c4786c8950 msgr2=0x55c47872d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:12.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.911+0000 7fe8a1278740 1 --2- 172.21.15.114:0/413004304 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c4786c8950 0x55c47872d900 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fe890013fd0 tx=0x7fe890004a40 comp rx=0 tx=0).stop 2024-09-13T18:24:12.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.912+0000 7fe8a1278740 1 -- 172.21.15.114:0/413004304 shutdown_connections 2024-09-13T18:24:12.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.912+0000 7fe8a1278740 1 --2- 172.21.15.114:0/413004304 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55c47872fa30 0x55c478731e50 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.912+0000 7fe8a1278740 1 --2- 172.21.15.114:0/413004304 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c4786c8950 0x55c47872d900 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.912+0000 7fe8a1278740 1 --2- 172.21.15.114:0/413004304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4786be0b0 0x55c4786cbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.912+0000 7fe8777fe640 0 client.16362 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:24:12.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.912+0000 7fe8a1278740 1 --2- 172.21.15.114:0/413004304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4786bd790 0x55c4786cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.912+0000 7fe8a1278740 1 -- 172.21.15.114:0/413004304 >> 172.21.15.114:0/413004304 conn(0x55c4786b9d00 msgr2=0x55c4786c0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:12.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.912+0000 7fe8a1278740 1 -- 172.21.15.114:0/413004304 shutdown_connections 2024-09-13T18:24:12.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.912+0000 7fe8a1278740 1 -- 172.21.15.114:0/413004304 wait complete. 2024-09-13T18:24:12.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.912+0000 7fe8a1278740 20 client.16362 trim_cache size 0 max 16384 2024-09-13T18:24:12.919 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:24:12.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.912+0000 7fe89e55a640 1 -- 172.21.15.114:0/2746496608 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe8800a1300 msgr2=0x7fe8800a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:12.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.912+0000 7fe89e55a640 1 --2- 172.21.15.114:0/2746496608 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe8800a1300 0x7fe8800a37c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe890005fd0 tx=0x7fe890004ab0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.912+0000 7fe89e55a640 1 -- 172.21.15.114:0/2746496608 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8980fee60 msgr2=0x7fe898193ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:12.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.912+0000 7fe89e55a640 1 --2- 172.21.15.114:0/2746496608 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8980fee60 0x7fe898193ff0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fe88800e9f0 tx=0x7fe88800eeb0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.913+0000 7fe89e55a640 1 -- 172.21.15.114:0/2746496608 shutdown_connections 2024-09-13T18:24:12.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.913+0000 7fe89e55a640 1 --2- 172.21.15.114:0/2746496608 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe8800a1300 0x7fe8800a37c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.913+0000 7fe89e55a640 1 --2- 172.21.15.114:0/2746496608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe898198920 0x7fe898198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.913+0000 7fe89e55a640 1 --2- 172.21.15.114:0/2746496608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe898103fb0 0x7fe898194530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.913+0000 7fe89e55a640 1 --2- 172.21.15.114:0/2746496608 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8980fee60 0x7fe898193ff0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:12.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.913+0000 7fe89e55a640 1 -- 172.21.15.114:0/2746496608 >> 172.21.15.114:0/2746496608 conn(0x7fe898075c60 msgr2=0x7fe898102440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:12.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.913+0000 7fe89e55a640 1 -- 172.21.15.114:0/2746496608 shutdown_connections 2024-09-13T18:24:12.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:12.913+0000 7fe89e55a640 1 -- 172.21.15.114:0/2746496608 wait complete. 2024-09-13T18:24:13.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:12 smithi204 ceph-mon[34777]: pgmap v55: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 148 MiB/s wr, 99 op/s 2024-09-13T18:24:13.219 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:12 smithi114 ceph-mon[30897]: pgmap v55: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 148 MiB/s wr, 99 op/s 2024-09-13T18:24:13.368 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:12 smithi134 ceph-mon[34283]: pgmap v55: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 148 MiB/s wr, 99 op/s 2024-09-13T18:24:13.931 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-13T18:24:13.931 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-09-13T18:24:14.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.085+0000 7ff553957640 1 -- 172.21.15.114:0/383899160 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff54c0fee60 msgr2=0x7ff54c0ff240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:14.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.085+0000 7ff553957640 1 --2- 172.21.15.114:0/383899160 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff54c0fee60 0x7ff54c0ff240 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7ff548009930 tx=0x7ff54802a440 comp rx=0 tx=0).stop 2024-09-13T18:24:14.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.085+0000 7ff553957640 1 -- 172.21.15.114:0/383899160 shutdown_connections 2024-09-13T18:24:14.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.085+0000 7ff553957640 1 --2- 172.21.15.114:0/383899160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff54c103fb0 0x7ff54c10bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.085+0000 7ff553957640 1 --2- 172.21.15.114:0/383899160 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff54c0ff810 0x7ff54c103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.085+0000 7ff553957640 1 --2- 172.21.15.114:0/383899160 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff54c0fee60 0x7ff54c0ff240 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.085+0000 7ff553957640 1 -- 172.21.15.114:0/383899160 >> 172.21.15.114:0/383899160 conn(0x7ff54c075c60 msgr2=0x7ff54c0fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:14.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.086+0000 7ff553957640 1 -- 172.21.15.114:0/383899160 shutdown_connections 2024-09-13T18:24:14.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.086+0000 7ff553957640 1 -- 172.21.15.114:0/383899160 wait complete. 2024-09-13T18:24:14.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.087+0000 7ff553957640 1 Processor -- start 2024-09-13T18:24:14.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.087+0000 7ff553957640 1 -- start start 2024-09-13T18:24:14.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.088+0000 7ff553957640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff54c0fee60 0x7ff54c19cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:14.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.088+0000 7ff553957640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff54c103fb0 0x7ff54c19d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:14.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.088+0000 7ff553957640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff54c10ac70 0x7ff54c10b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:14.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.088+0000 7ff553957640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c10b880 con 0x7ff54c0fee60 2024-09-13T18:24:14.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.088+0000 7ff553957640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c10b9f0 con 0x7ff54c103fb0 2024-09-13T18:24:14.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.088+0000 7ff553957640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c198a50 con 0x7ff54c10ac70 2024-09-13T18:24:14.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.088+0000 7ff550ef2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff54c103fb0 0x7ff54c19d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:14.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.088+0000 7ff551ef4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff54c10ac70 0x7ff54c10b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:14.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.088+0000 7ff550ef2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff54c103fb0 0x7ff54c19d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:45436/0 (socket says 172.21.15.114:45436) 2024-09-13T18:24:14.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.088+0000 7ff550ef2640 1 -- 172.21.15.114:0/2037040024 learned_addr learned my addr 172.21.15.114:0/2037040024 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:14.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.088+0000 7ff5516f3640 1 --2- 172.21.15.114:0/2037040024 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff54c0fee60 0x7ff54c19cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:14.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.088+0000 7ff550ef2640 1 -- 172.21.15.114:0/2037040024 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff54c10ac70 msgr2=0x7ff54c10b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:14.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.088+0000 7ff550ef2640 1 --2- 172.21.15.114:0/2037040024 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff54c10ac70 0x7ff54c10b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.088+0000 7ff550ef2640 1 -- 172.21.15.114:0/2037040024 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff54c0fee60 msgr2=0x7ff54c19cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:14.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.088+0000 7ff550ef2640 1 --2- 172.21.15.114:0/2037040024 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff54c0fee60 0x7ff54c19cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.088+0000 7ff550ef2640 1 -- 172.21.15.114:0/2037040024 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff548009560 con 0x7ff54c103fb0 2024-09-13T18:24:14.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.088+0000 7ff5516f3640 1 --2- 172.21.15.114:0/2037040024 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff54c0fee60 0x7ff54c19cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:24:14.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.088+0000 7ff551ef4640 1 --2- 172.21.15.114:0/2037040024 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff54c10ac70 0x7ff54c10b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:24:14.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.089+0000 7ff550ef2640 1 --2- 172.21.15.114:0/2037040024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff54c103fb0 0x7ff54c19d460 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff53c00b470 tx=0x7ff53c00b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:14.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.089+0000 7ff5427fc640 1 -- 172.21.15.114:0/2037040024 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53c011890 con 0x7ff54c103fb0 2024-09-13T18:24:14.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.089+0000 7ff553957640 1 -- 172.21.15.114:0/2037040024 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff54c198d30 con 0x7ff54c103fb0 2024-09-13T18:24:14.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.089+0000 7ff5427fc640 1 -- 172.21.15.114:0/2037040024 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff53c011e80 con 0x7ff54c103fb0 2024-09-13T18:24:14.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.089+0000 7ff5427fc640 1 -- 172.21.15.114:0/2037040024 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53c00f340 con 0x7ff54c103fb0 2024-09-13T18:24:14.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.089+0000 7ff553957640 1 -- 172.21.15.114:0/2037040024 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff54c199240 con 0x7ff54c103fb0 2024-09-13T18:24:14.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.092+0000 7ff5427fc640 1 -- 172.21.15.114:0/2037040024 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7ff53c0119f0 con 0x7ff54c103fb0 2024-09-13T18:24:14.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.094+0000 7ff55667b740 1 Processor -- start 2024-09-13T18:24:14.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.094+0000 7ff55667b740 1 -- start start 2024-09-13T18:24:14.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.094+0000 7ff5427fc640 1 --2- 172.21.15.114:0/2037040024 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff5380a12c0 0x7ff5380a3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:14.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.094+0000 7ff55667b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596a0d7a7c0 0x5596a0d7aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:14.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.094+0000 7ff55667b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596a0d7b0e0 0x5596a0d85440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:14.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.094+0000 7ff5427fc640 1 -- 172.21.15.114:0/2037040024 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ff53c0ccdc0 con 0x7ff54c103fb0 2024-09-13T18:24:14.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.094+0000 7ff55667b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596a0d85980 0x5596a0d87d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:14.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.094+0000 7ff55667b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596a0ce5c60 con 0x5596a0d7a7c0 2024-09-13T18:24:14.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.094+0000 7ff55667b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596a0ce33f0 con 0x5596a0d7b0e0 2024-09-13T18:24:14.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.094+0000 7ff55667b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596a0cdece0 con 0x5596a0d85980 2024-09-13T18:24:14.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.094+0000 7ff551ef4640 1 --2- 172.21.15.114:0/2037040024 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff5380a12c0 0x7ff5380a3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:14.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.094+0000 7ff550ef2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596a0d7b0e0 0x5596a0d85440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:14.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.094+0000 7ff550ef2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596a0d7b0e0 0x5596a0d85440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:45444/0 (socket says 172.21.15.114:45444) 2024-09-13T18:24:14.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.094+0000 7ff550ef2640 1 -- 172.21.15.114:0/3354745098 learned_addr learned my addr 172.21.15.114:0/3354745098 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:14.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.095+0000 7ff5516f3640 1 --2- 172.21.15.114:0/3354745098 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596a0d7a7c0 0x5596a0d7aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:14.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.095+0000 7ff550ef2640 1 -- 172.21.15.114:0/3354745098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596a0d85980 msgr2=0x5596a0d87d70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:24:14.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.095+0000 7ff550ef2640 1 --2- 172.21.15.114:0/3354745098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596a0d85980 0x5596a0d87d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.095+0000 7ff550ef2640 1 -- 172.21.15.114:0/3354745098 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596a0d7a7c0 msgr2=0x5596a0d7aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:14.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.095+0000 7ff550ef2640 1 --2- 172.21.15.114:0/3354745098 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596a0d7a7c0 0x5596a0d7aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.095+0000 7ff550ef2640 1 -- 172.21.15.114:0/3354745098 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596a0c9a970 con 0x5596a0d7b0e0 2024-09-13T18:24:14.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.095+0000 7ff5516f3640 1 --2- 172.21.15.114:0/3354745098 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596a0d7a7c0 0x5596a0d7aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:24:14.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.095+0000 7ff551ef4640 1 --2- 172.21.15.114:0/2037040024 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff5380a12c0 0x7ff5380a3780 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff54400c470 tx=0x7ff544009590 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:14.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.095+0000 7ff550ef2640 1 --2- 172.21.15.114:0/3354745098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596a0d7b0e0 0x5596a0d85440 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff53c016000 tx=0x7ff53c099890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=247da34e56e2c746 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:14.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.095+0000 7ff5377fe640 1 -- 172.21.15.114:0/3354745098 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53c09a390 con 0x5596a0d7b0e0 2024-09-13T18:24:14.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.095+0000 7ff5377fe640 1 -- 172.21.15.114:0/3354745098 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff53c0103e0 con 0x5596a0d7b0e0 2024-09-13T18:24:14.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.095+0000 7ff5377fe640 1 -- 172.21.15.114:0/3354745098 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53c0963a0 con 0x5596a0d7b0e0 2024-09-13T18:24:14.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.096+0000 7ff55667b740 1 -- 172.21.15.114:0/3354745098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596a0d7b0e0 msgr2=0x5596a0d85440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:14.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.096+0000 7ff55667b740 1 --2- 172.21.15.114:0/3354745098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596a0d7b0e0 0x5596a0d85440 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff53c016000 tx=0x7ff53c099890 comp rx=0 tx=0).stop 2024-09-13T18:24:14.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.096+0000 7ff55667b740 1 -- 172.21.15.114:0/3354745098 shutdown_connections 2024-09-13T18:24:14.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.096+0000 7ff55667b740 1 --2- 172.21.15.114:0/3354745098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596a0d85980 0x5596a0d87d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.096+0000 7ff55667b740 1 --2- 172.21.15.114:0/3354745098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596a0d7b0e0 0x5596a0d85440 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.096+0000 7ff55667b740 1 --2- 172.21.15.114:0/3354745098 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596a0d7a7c0 0x5596a0d7aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.096+0000 7ff55667b740 1 -- 172.21.15.114:0/3354745098 >> 172.21.15.114:0/3354745098 conn(0x5596a0d76d50 msgr2=0x5596a0d77130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:14.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.096+0000 7ff55667b740 1 -- 172.21.15.114:0/3354745098 shutdown_connections 2024-09-13T18:24:14.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.096+0000 7ff55667b740 1 -- 172.21.15.114:0/3354745098 wait complete. 2024-09-13T18:24:14.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.097+0000 7ff55667b740 1 -- start start 2024-09-13T18:24:14.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.097+0000 7ff55667b740 1 Processor -- start 2024-09-13T18:24:14.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.097+0000 7ff55667b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596a0d7a7c0 0x5596a0d85430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:14.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.097+0000 7ff55667b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596a0d7b0e0 0x5596a0d82720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:14.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.097+0000 7ff55667b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596a0d85980 0x5596a0d82eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:14.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.097+0000 7ff55667b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596a0c9cf80 con 0x5596a0d85980 2024-09-13T18:24:14.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.097+0000 7ff55667b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596a0c9cc20 con 0x5596a0d7a7c0 2024-09-13T18:24:14.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.097+0000 7ff55667b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596a0c69ab0 con 0x5596a0d7b0e0 2024-09-13T18:24:14.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.097+0000 7ff55667b740 10 client.0 _ll_register_callbacks cb 0x5596a0cd8350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:24:14.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.098+0000 7ff551ef4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596a0d85980 0x5596a0d82eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:14.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.098+0000 7ff551ef4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596a0d85980 0x5596a0d82eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:54228/0 (socket says 172.21.15.114:54228) 2024-09-13T18:24:14.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.098+0000 7ff551ef4640 1 -- 172.21.15.114:0/312002918 learned_addr learned my addr 172.21.15.114:0/312002918 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:14.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.098+0000 7ff550ef2640 1 --2- 172.21.15.114:0/312002918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596a0d7b0e0 0x5596a0d82720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:14.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.098+0000 7ff5516f3640 1 --2- 172.21.15.114:0/312002918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596a0d7a7c0 0x5596a0d85430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:14.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.098+0000 7ff551ef4640 1 -- 172.21.15.114:0/312002918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596a0d7b0e0 msgr2=0x5596a0d82720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:14.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.098+0000 7ff551ef4640 1 --2- 172.21.15.114:0/312002918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596a0d7b0e0 0x5596a0d82720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.098+0000 7ff551ef4640 1 -- 172.21.15.114:0/312002918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596a0d7a7c0 msgr2=0x5596a0d85430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:14.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.098+0000 7ff551ef4640 1 --2- 172.21.15.114:0/312002918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596a0d7a7c0 0x5596a0d85430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.098+0000 7ff551ef4640 1 -- 172.21.15.114:0/312002918 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff54c199240 con 0x5596a0d85980 2024-09-13T18:24:14.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.098+0000 7ff5516f3640 1 --2- 172.21.15.114:0/312002918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596a0d7a7c0 0x5596a0d85430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:24:14.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.098+0000 7ff551ef4640 1 --2- 172.21.15.114:0/312002918 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596a0d85980 0x5596a0d82eb0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7ff544018bb0 tx=0x7ff544004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f55a15948519083b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:14.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.098+0000 7ff534ff9640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:24:14.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.098+0000 7ff534ff9640 1 -- 172.21.15.114:0/312002918 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54401d070 con 0x5596a0d85980 2024-09-13T18:24:14.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.098+0000 7ff55667b740 1 -- 172.21.15.114:0/312002918 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5596a09d2590 con 0x5596a0d85980 2024-09-13T18:24:14.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.098+0000 7ff534ff9640 1 -- 172.21.15.114:0/312002918 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ff544018d60 con 0x5596a0d85980 2024-09-13T18:24:14.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.099+0000 7ff534ff9640 1 -- 172.21.15.114:0/312002918 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff544011070 con 0x5596a0d85980 2024-09-13T18:24:14.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.099+0000 7ff55667b740 10 client.16371 fetch_fsmap learned FSMap version 32 2024-09-13T18:24:14.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.099+0000 7ff55667b740 1 -- 172.21.15.114:0/312002918 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5596a0cdd360 con 0x5596a0d85980 2024-09-13T18:24:14.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.099+0000 7ff534ff9640 1 -- 172.21.15.114:0/312002918 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5596a09d2590 con 0x5596a0d85980 2024-09-13T18:24:14.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.099+0000 7ff534ff9640 1 -- 172.21.15.114:0/312002918 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7ff54402b070 con 0x5596a0d85980 2024-09-13T18:24:14.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.099+0000 7ff55667b740 10 client.16371 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:24:14.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.099+0000 7ff55667b740 10 client.16371 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:24:14.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.099+0000 7ff55667b740 20 client.16371 populate_metadata read hostname 'smithi114' 2024-09-13T18:24:14.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.099+0000 7ff55667b740 1 --2- 172.21.15.114:0/312002918 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5596a0d957e0 0x5596a0d97c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:14.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.099+0000 7ff55667b740 4 client.16371 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:24:14.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.099+0000 7ff55667b740 1 -- 172.21.15.114:0/312002918 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5596a09e3860 con 0x5596a0d957e0 2024-09-13T18:24:14.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.100+0000 7ff5516f3640 1 --2- 172.21.15.114:0/312002918 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5596a0d957e0 0x5596a0d97c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:14.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.100+0000 7ff534ff9640 10 client.16371 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:24:14.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.100+0000 7ff5516f3640 1 --2- 172.21.15.114:0/312002918 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5596a0d957e0 0x5596a0d97c00 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=203c2d4c1c93a055 server_cookie=687afb92224a219e in_seq=0 out_seq=0 2024-09-13T18:24:14.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.101+0000 7ff534ff9640 1 -- 172.21.15.114:0/312002918 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5596a09e3860 con 0x5596a0d957e0 2024-09-13T18:24:14.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.101+0000 7ff534ff9640 10 client.16371 handle_command_reply: tid=0 2024-09-13T18:24:14.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.101+0000 7ff55667b740 1 client.16371 shutdown 2024-09-13T18:24:14.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.101+0000 7ff55667b740 1 -- 172.21.15.114:0/312002918 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596a0d85980 msgr2=0x5596a0d82eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:14.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.101+0000 7ff55667b740 1 --2- 172.21.15.114:0/312002918 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596a0d85980 0x5596a0d82eb0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7ff544018bb0 tx=0x7ff544004b50 comp rx=0 tx=0).stop 2024-09-13T18:24:14.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.101+0000 7ff55667b740 1 -- 172.21.15.114:0/312002918 shutdown_connections 2024-09-13T18:24:14.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.101+0000 7ff55667b740 1 --2- 172.21.15.114:0/312002918 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5596a0d957e0 0x5596a0d97c00 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.101+0000 7ff55667b740 1 --2- 172.21.15.114:0/312002918 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596a0d85980 0x5596a0d82eb0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.101+0000 7ff55667b740 1 --2- 172.21.15.114:0/312002918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596a0d7b0e0 0x5596a0d82720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.101+0000 7ff55667b740 1 --2- 172.21.15.114:0/312002918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596a0d7a7c0 0x5596a0d85430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.101+0000 7ff55667b740 1 -- 172.21.15.114:0/312002918 >> 172.21.15.114:0/312002918 conn(0x5596a0d76d50 msgr2=0x5596a0d7c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:14.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.101+0000 7ff534ff9640 0 client.16371 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:24:14.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.101+0000 7ff55667b740 1 -- 172.21.15.114:0/312002918 shutdown_connections 2024-09-13T18:24:14.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.101+0000 7ff55667b740 1 -- 172.21.15.114:0/312002918 wait complete. 2024-09-13T18:24:14.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.101+0000 7ff55667b740 20 client.16371 trim_cache size 0 max 16384 2024-09-13T18:24:14.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.125+0000 7ff55667b740 1 Processor -- start 2024-09-13T18:24:14.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.125+0000 7ff55667b740 1 -- start start 2024-09-13T18:24:14.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.125+0000 7ff55667b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596a0d7a7c0 0x5596a0d8c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:14.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.125+0000 7ff55667b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596a0d7b0e0 0x5596a0d8cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:14.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.125+0000 7ff55667b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596a0d85980 0x5596a0d7d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:14.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.125+0000 7ff55667b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596a0d95530 con 0x5596a0d85980 2024-09-13T18:24:14.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.125+0000 7ff55667b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596a0d85820 con 0x5596a0d7a7c0 2024-09-13T18:24:14.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.125+0000 7ff55667b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596a0d924b0 con 0x5596a0d7b0e0 2024-09-13T18:24:14.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.125+0000 7ff551ef4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596a0d85980 0x5596a0d7d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:14.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.125+0000 7ff551ef4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596a0d85980 0x5596a0d7d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:54236/0 (socket says 172.21.15.114:54236) 2024-09-13T18:24:14.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.125+0000 7ff551ef4640 1 -- 172.21.15.114:0/1198964481 learned_addr learned my addr 172.21.15.114:0/1198964481 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:14.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.125+0000 7ff551ef4640 1 -- 172.21.15.114:0/1198964481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596a0d7b0e0 msgr2=0x5596a0d8cb90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:24:14.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.126+0000 7ff551ef4640 1 --2- 172.21.15.114:0/1198964481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596a0d7b0e0 0x5596a0d8cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.126+0000 7ff551ef4640 1 -- 172.21.15.114:0/1198964481 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596a0d7a7c0 msgr2=0x5596a0d8c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:14.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.126+0000 7ff551ef4640 1 --2- 172.21.15.114:0/1198964481 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596a0d7a7c0 0x5596a0d8c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.126+0000 7ff551ef4640 1 -- 172.21.15.114:0/1198964481 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596a0c7f1c0 con 0x5596a0d85980 2024-09-13T18:24:14.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.126+0000 7ff551ef4640 1 --2- 172.21.15.114:0/1198964481 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596a0d85980 0x5596a0d7d9e0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7ff5440097c0 tx=0x7ff5440097f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=341adc9e4ea99a10 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:14.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.126+0000 7ff537fff640 1 -- 172.21.15.114:0/1198964481 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54401d040 con 0x5596a0d85980 2024-09-13T18:24:14.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.126+0000 7ff537fff640 1 -- 172.21.15.114:0/1198964481 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff5440187c0 con 0x5596a0d85980 2024-09-13T18:24:14.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.126+0000 7ff537fff640 1 -- 172.21.15.114:0/1198964481 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff544016e30 con 0x5596a0d85980 2024-09-13T18:24:14.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.127+0000 7ff55667b740 1 -- 172.21.15.114:0/1198964481 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596a0d85980 msgr2=0x5596a0d7d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:14.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.127+0000 7ff55667b740 1 --2- 172.21.15.114:0/1198964481 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596a0d85980 0x5596a0d7d9e0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7ff5440097c0 tx=0x7ff5440097f0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.127+0000 7ff55667b740 1 -- 172.21.15.114:0/1198964481 shutdown_connections 2024-09-13T18:24:14.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.127+0000 7ff55667b740 1 --2- 172.21.15.114:0/1198964481 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596a0d85980 0x5596a0d7d9e0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.127+0000 7ff55667b740 1 --2- 172.21.15.114:0/1198964481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596a0d7b0e0 0x5596a0d8cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.127+0000 7ff55667b740 1 --2- 172.21.15.114:0/1198964481 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596a0d7a7c0 0x5596a0d8c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.127+0000 7ff55667b740 1 -- 172.21.15.114:0/1198964481 >> 172.21.15.114:0/1198964481 conn(0x5596a0d76d50 msgr2=0x5596a0d9c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:14.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.127+0000 7ff55667b740 1 -- 172.21.15.114:0/1198964481 shutdown_connections 2024-09-13T18:24:14.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.127+0000 7ff55667b740 1 -- 172.21.15.114:0/1198964481 wait complete. 2024-09-13T18:24:14.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.128+0000 7ff55667b740 1 -- start start 2024-09-13T18:24:14.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.128+0000 7ff55667b740 1 Processor -- start 2024-09-13T18:24:14.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.128+0000 7ff55667b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596a0d7a7c0 0x5596a0d8c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:14.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.128+0000 7ff55667b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596a0d7b0e0 0x5596a0d8ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:14.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.128+0000 7ff55667b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596a0d85980 0x5596a0d8b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:14.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.128+0000 7ff55667b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596a0d8b970 con 0x5596a0d7a7c0 2024-09-13T18:24:14.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.128+0000 7ff55667b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596a0d8bae0 con 0x5596a0d85980 2024-09-13T18:24:14.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.128+0000 7ff55667b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596a0debcf0 con 0x5596a0d7b0e0 2024-09-13T18:24:14.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.128+0000 7ff55667b740 10 client.0 _ll_register_callbacks cb 0x7ff54c069b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:24:14.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.128+0000 7ff5516f3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596a0d7a7c0 0x5596a0d8c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:14.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.128+0000 7ff551ef4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596a0d85980 0x5596a0d8b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:14.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.128+0000 7ff551ef4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596a0d85980 0x5596a0d8b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:45480/0 (socket says 172.21.15.114:45480) 2024-09-13T18:24:14.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.128+0000 7ff551ef4640 1 -- 172.21.15.114:0/2600174215 learned_addr learned my addr 172.21.15.114:0/2600174215 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:14.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.129+0000 7ff550ef2640 1 --2- 172.21.15.114:0/2600174215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596a0d7b0e0 0x5596a0d8ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:14.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.129+0000 7ff551ef4640 1 -- 172.21.15.114:0/2600174215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596a0d7b0e0 msgr2=0x5596a0d8ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:14.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.129+0000 7ff551ef4640 1 --2- 172.21.15.114:0/2600174215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596a0d7b0e0 0x5596a0d8ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.129+0000 7ff551ef4640 1 -- 172.21.15.114:0/2600174215 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596a0d7a7c0 msgr2=0x5596a0d8c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:14.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.129+0000 7ff551ef4640 1 --2- 172.21.15.114:0/2600174215 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596a0d7a7c0 0x5596a0d8c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.129+0000 7ff551ef4640 1 -- 172.21.15.114:0/2600174215 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5596a0cdd360 con 0x5596a0d85980 2024-09-13T18:24:14.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.129+0000 7ff551ef4640 1 --2- 172.21.15.114:0/2600174215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596a0d85980 0x5596a0d8b270 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff544004b90 tx=0x7ff544023650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be01da2adf2fe185 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:14.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.129+0000 7ff534ff9640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:24:14.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.129+0000 7ff534ff9640 1 -- 172.21.15.114:0/2600174215 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54401d040 con 0x5596a0d85980 2024-09-13T18:24:14.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.129+0000 7ff534ff9640 1 -- 172.21.15.114:0/2600174215 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ff544023c50 con 0x5596a0d85980 2024-09-13T18:24:14.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.129+0000 7ff55667b740 1 -- 172.21.15.114:0/2600174215 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff53c0103e0 con 0x5596a0d85980 2024-09-13T18:24:14.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.130+0000 7ff534ff9640 1 -- 172.21.15.114:0/2600174215 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff544011040 con 0x5596a0d85980 2024-09-13T18:24:14.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.130+0000 7ff55667b740 10 client.25703 fetch_fsmap learned FSMap version 32 2024-09-13T18:24:14.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.130+0000 7ff55667b740 1 -- 172.21.15.114:0/2600174215 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5596a0ca4990 con 0x5596a0d85980 2024-09-13T18:24:14.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.130+0000 7ff534ff9640 1 -- 172.21.15.114:0/2600174215 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff53c0103e0 con 0x5596a0d85980 2024-09-13T18:24:14.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.130+0000 7ff534ff9640 1 -- 172.21.15.114:0/2600174215 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7ff54402b040 con 0x5596a0d85980 2024-09-13T18:24:14.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.130+0000 7ff55667b740 10 client.25703 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:24:14.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.130+0000 7ff55667b740 10 client.25703 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:24:14.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.130+0000 7ff55667b740 20 client.25703 populate_metadata read hostname 'smithi114' 2024-09-13T18:24:14.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.130+0000 7ff55667b740 1 --2- 172.21.15.114:0/2600174215 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5596a0decf40 0x5596a0def360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:14.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.130+0000 7ff55667b740 4 client.25703 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-13T18:24:14.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.130+0000 7ff55667b740 1 -- 172.21.15.114:0/2600174215 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5596a0a36eb0 con 0x5596a0decf40 2024-09-13T18:24:14.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.130+0000 7ff5516f3640 1 --2- 172.21.15.114:0/2600174215 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5596a0decf40 0x5596a0def360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:14.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.131+0000 7ff534ff9640 10 client.25703 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:24:14.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.131+0000 7ff5516f3640 1 --2- 172.21.15.114:0/2600174215 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5596a0decf40 0x5596a0def360 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7073d9f84a08e6ed server_cookie=8760b0761c41ba84 in_seq=0 out_seq=0 2024-09-13T18:24:14.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.132+0000 7ff534ff9640 1 -- 172.21.15.114:0/2600174215 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5596a0a36eb0 con 0x5596a0decf40 2024-09-13T18:24:14.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.132+0000 7ff534ff9640 10 client.25703 handle_command_reply: tid=0 2024-09-13T18:24:14.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.132+0000 7ff55667b740 1 client.25703 shutdown 2024-09-13T18:24:14.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.132+0000 7ff55667b740 1 -- 172.21.15.114:0/2600174215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596a0d85980 msgr2=0x5596a0d8b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:14.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.132+0000 7ff55667b740 1 --2- 172.21.15.114:0/2600174215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596a0d85980 0x5596a0d8b270 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff544004b90 tx=0x7ff544023650 comp rx=0 tx=0).stop 2024-09-13T18:24:14.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.132+0000 7ff55667b740 1 -- 172.21.15.114:0/2600174215 shutdown_connections 2024-09-13T18:24:14.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.132+0000 7ff55667b740 1 --2- 172.21.15.114:0/2600174215 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5596a0decf40 0x5596a0def360 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.132+0000 7ff55667b740 1 --2- 172.21.15.114:0/2600174215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596a0d85980 0x5596a0d8b270 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.132+0000 7ff55667b740 1 --2- 172.21.15.114:0/2600174215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596a0d7b0e0 0x5596a0d8ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.132+0000 7ff55667b740 1 --2- 172.21.15.114:0/2600174215 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596a0d7a7c0 0x5596a0d8c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.132+0000 7ff534ff9640 0 client.25703 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:24:14.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.132+0000 7ff55667b740 1 -- 172.21.15.114:0/2600174215 >> 172.21.15.114:0/2600174215 conn(0x5596a0d76d50 msgr2=0x5596a0d7d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:14.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.132+0000 7ff55667b740 1 -- 172.21.15.114:0/2600174215 shutdown_connections 2024-09-13T18:24:14.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.133+0000 7ff55667b740 1 -- 172.21.15.114:0/2600174215 wait complete. 2024-09-13T18:24:14.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.133+0000 7ff55667b740 20 client.25703 trim_cache size 0 max 16384 2024-09-13T18:24:14.141 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:24:14.142 INFO:teuthology.orchestra.run.smithi114.stdout: "return_code": 0, 2024-09-13T18:24:14.142 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_tag": "7d4aea73-3e2f-4a57-bcba-035cf9ad7f33", 2024-09-13T18:24:14.142 INFO:teuthology.orchestra.run.smithi114.stdout: "mode": "asynchronous" 2024-09-13T18:24:14.142 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:24:14.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.133+0000 7ff553957640 1 -- 172.21.15.114:0/2037040024 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff5380a12c0 msgr2=0x7ff5380a3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:14.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.133+0000 7ff553957640 1 --2- 172.21.15.114:0/2037040024 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff5380a12c0 0x7ff5380a3780 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff54400c470 tx=0x7ff544009590 comp rx=0 tx=0).stop 2024-09-13T18:24:14.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.133+0000 7ff553957640 1 -- 172.21.15.114:0/2037040024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff54c103fb0 msgr2=0x7ff54c19d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:14.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.133+0000 7ff553957640 1 --2- 172.21.15.114:0/2037040024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff54c103fb0 0x7ff54c19d460 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff53c00b470 tx=0x7ff53c00b930 comp rx=0 tx=0).stop 2024-09-13T18:24:14.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.134+0000 7ff553957640 1 -- 172.21.15.114:0/2037040024 shutdown_connections 2024-09-13T18:24:14.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.134+0000 7ff553957640 1 --2- 172.21.15.114:0/2037040024 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff5380a12c0 0x7ff5380a3780 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.134+0000 7ff553957640 1 --2- 172.21.15.114:0/2037040024 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff54c10ac70 0x7ff54c10b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.134+0000 7ff553957640 1 --2- 172.21.15.114:0/2037040024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff54c103fb0 0x7ff54c19d460 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.134+0000 7ff553957640 1 --2- 172.21.15.114:0/2037040024 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff54c0fee60 0x7ff54c19cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.134+0000 7ff553957640 1 -- 172.21.15.114:0/2037040024 >> 172.21.15.114:0/2037040024 conn(0x7ff54c075c60 msgr2=0x7ff54c1020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:14.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.134+0000 7ff553957640 1 -- 172.21.15.114:0/2037040024 shutdown_connections 2024-09-13T18:24:14.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.134+0000 7ff553957640 1 -- 172.21.15.114:0/2037040024 wait complete. 2024-09-13T18:24:14.153 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T18:24:14.219 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:14 smithi114 ceph-mon[30897]: pgmap v56: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 221 MiB/s wr, 142 op/s 2024-09-13T18:24:14.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.302+0000 7f296a725640 1 -- 172.21.15.114:0/3068990377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2964100f40 msgr2=0x7f2964101320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:14.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.302+0000 7f296a725640 1 --2- 172.21.15.114:0/3068990377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2964100f40 0x7f2964101320 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2954009900 tx=0x7f295402a480 comp rx=0 tx=0).stop 2024-09-13T18:24:14.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.302+0000 7f296a725640 1 -- 172.21.15.114:0/3068990377 shutdown_connections 2024-09-13T18:24:14.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.302+0000 7f296a725640 1 --2- 172.21.15.114:0/3068990377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f296410a310 0x7f296410c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.302+0000 7f296a725640 1 --2- 172.21.15.114:0/3068990377 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2964101860 0x7f2964109d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.302+0000 7f296a725640 1 --2- 172.21.15.114:0/3068990377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2964100f40 0x7f2964101320 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.302+0000 7f296a725640 1 -- 172.21.15.114:0/3068990377 >> 172.21.15.114:0/3068990377 conn(0x7f2964075c60 msgr2=0x7f29640fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:14.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.302+0000 7f296a725640 1 -- 172.21.15.114:0/3068990377 shutdown_connections 2024-09-13T18:24:14.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.302+0000 7f296a725640 1 -- 172.21.15.114:0/3068990377 wait complete. 2024-09-13T18:24:14.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.303+0000 7f296a725640 1 Processor -- start 2024-09-13T18:24:14.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.303+0000 7f296a725640 1 -- start start 2024-09-13T18:24:14.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.303+0000 7f296a725640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2964100f40 0x7f29641a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:14.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.303+0000 7f296a725640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2964101860 0x7f29641a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:14.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.303+0000 7f296a725640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f296410a310 0x7f2964108ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:14.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.304+0000 7f296a725640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29641094a0 con 0x7f296410a310 2024-09-13T18:24:14.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.304+0000 7f296a725640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2964109610 con 0x7f2964100f40 2024-09-13T18:24:14.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.304+0000 7f296a725640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29641097d0 con 0x7f2964101860 2024-09-13T18:24:14.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.304+0000 7f2968cc2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f296410a310 0x7f2964108ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:14.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.304+0000 7f2968cc2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f296410a310 0x7f2964108ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:54252/0 (socket says 172.21.15.114:54252) 2024-09-13T18:24:14.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.304+0000 7f2968cc2640 1 -- 172.21.15.114:0/2775540278 learned_addr learned my addr 172.21.15.114:0/2775540278 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:14.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.304+0000 7f2963fff640 1 --2- 172.21.15.114:0/2775540278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2964100f40 0x7f29641a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:14.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.304+0000 7f29637fe640 1 --2- 172.21.15.114:0/2775540278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2964101860 0x7f29641a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:14.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.304+0000 7f2968cc2640 1 -- 172.21.15.114:0/2775540278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2964101860 msgr2=0x7f29641a1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:14.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.304+0000 7f2968cc2640 1 --2- 172.21.15.114:0/2775540278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2964101860 0x7f29641a1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.304+0000 7f2968cc2640 1 -- 172.21.15.114:0/2775540278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2964100f40 msgr2=0x7f29641a1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:14.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.304+0000 7f2968cc2640 1 --2- 172.21.15.114:0/2775540278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2964100f40 0x7f29641a1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.304+0000 7f2968cc2640 1 -- 172.21.15.114:0/2775540278 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2958009490 con 0x7f296410a310 2024-09-13T18:24:14.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.304+0000 7f2968cc2640 1 --2- 172.21.15.114:0/2775540278 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f296410a310 0x7f2964108ea0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f295c00cab0 tx=0x7f295c00cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:14.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.305+0000 7f29617fa640 1 -- 172.21.15.114:0/2775540278 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295c00adc0 con 0x7f296410a310 2024-09-13T18:24:14.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.305+0000 7f29617fa640 1 -- 172.21.15.114:0/2775540278 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f295c00f8e0 con 0x7f296410a310 2024-09-13T18:24:14.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.305+0000 7f29617fa640 1 -- 172.21.15.114:0/2775540278 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295c018830 con 0x7f296410a310 2024-09-13T18:24:14.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.305+0000 7f296a725640 1 -- 172.21.15.114:0/2775540278 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2954009560 con 0x7f296410a310 2024-09-13T18:24:14.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.305+0000 7f296a725640 1 -- 172.21.15.114:0/2775540278 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29641a93b0 con 0x7f296410a310 2024-09-13T18:24:14.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.308+0000 7f29617fa640 1 -- 172.21.15.114:0/2775540278 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f295c00fd30 con 0x7f296410a310 2024-09-13T18:24:14.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.308+0000 7f296d44b740 1 Processor -- start 2024-09-13T18:24:14.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.308+0000 7f296d44b740 1 -- start start 2024-09-13T18:24:14.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.308+0000 7f296d44b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564194780e10 0x5641947811f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:14.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.308+0000 7f296d44b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564194781730 0x56419478ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:14.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.308+0000 7f296d44b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56419478bfd0 0x56419478e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:14.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.308+0000 7f296d44b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641946ebf20 con 0x56419478bfd0 2024-09-13T18:24:14.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.308+0000 7f296d44b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641946e96b0 con 0x564194781730 2024-09-13T18:24:14.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.308+0000 7f296d44b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641946e4fa0 con 0x564194780e10 2024-09-13T18:24:14.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.309+0000 7f29637fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564194780e10 0x5641947811f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:14.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.309+0000 7f29637fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564194780e10 0x5641947811f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:51772/0 (socket says 172.21.15.114:51772) 2024-09-13T18:24:14.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.309+0000 7f29637fe640 1 -- 172.21.15.114:0/893606683 learned_addr learned my addr 172.21.15.114:0/893606683 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:14.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.309+0000 7f29637fe640 1 --2- 172.21.15.114:0/893606683 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56419478bfd0 0x56419478e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:14.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.309+0000 7f29617fa640 1 --2- 172.21.15.114:0/2775540278 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f294c0a1280 0x7f294c0a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:14.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.309+0000 7f2968cc2640 1 --2- 172.21.15.114:0/2775540278 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f294c0a1280 0x7f294c0a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:14.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.309+0000 7f2963fff640 1 --2- 172.21.15.114:0/893606683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564194781730 0x56419478ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:14.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.309+0000 7f29637fe640 1 -- 172.21.15.114:0/893606683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564194781730 msgr2=0x56419478ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:14.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.309+0000 7f29637fe640 1 --2- 172.21.15.114:0/893606683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564194781730 0x56419478ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.309+0000 7f29637fe640 1 -- 172.21.15.114:0/893606683 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56419478bfd0 msgr2=0x56419478e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:14.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.309+0000 7f29637fe640 1 --2- 172.21.15.114:0/893606683 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56419478bfd0 0x56419478e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.309+0000 7f29637fe640 1 -- 172.21.15.114:0/893606683 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56419469f840 con 0x564194780e10 2024-09-13T18:24:14.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.309+0000 7f2963fff640 1 --2- 172.21.15.114:0/893606683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564194781730 0x56419478ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:24:14.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.309+0000 7f29617fa640 1 -- 172.21.15.114:0/2775540278 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f295c0cd380 con 0x7f296410a310 2024-09-13T18:24:14.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.309+0000 7f29637fe640 1 --2- 172.21.15.114:0/893606683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564194780e10 0x5641947811f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f29580128c0 tx=0x7f2958009820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1a3825584f8309e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:14.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.310+0000 7f294a7fc640 1 -- 172.21.15.114:0/893606683 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2958012e40 con 0x564194780e10 2024-09-13T18:24:14.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.310+0000 7f294a7fc640 1 -- 172.21.15.114:0/893606683 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f29580054b0 con 0x564194780e10 2024-09-13T18:24:14.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.310+0000 7f294a7fc640 1 -- 172.21.15.114:0/893606683 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29580107d0 con 0x564194780e10 2024-09-13T18:24:14.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.311+0000 7f296d44b740 1 -- 172.21.15.114:0/893606683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564194780e10 msgr2=0x5641947811f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:14.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.311+0000 7f296d44b740 1 --2- 172.21.15.114:0/893606683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564194780e10 0x5641947811f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f29580128c0 tx=0x7f2958009820 comp rx=0 tx=0).stop 2024-09-13T18:24:14.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.311+0000 7f2968cc2640 1 --2- 172.21.15.114:0/2775540278 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f294c0a1280 0x7f294c0a3740 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f295c096bb0 tx=0x7f295c06d2d0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:14.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.311+0000 7f296d44b740 1 -- 172.21.15.114:0/893606683 shutdown_connections 2024-09-13T18:24:14.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.311+0000 7f296d44b740 1 --2- 172.21.15.114:0/893606683 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56419478bfd0 0x56419478e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.311+0000 7f296d44b740 1 --2- 172.21.15.114:0/893606683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564194781730 0x56419478ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.311+0000 7f296d44b740 1 --2- 172.21.15.114:0/893606683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564194780e10 0x5641947811f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.311+0000 7f296d44b740 1 -- 172.21.15.114:0/893606683 >> 172.21.15.114:0/893606683 conn(0x56419477d380 msgr2=0x56419477d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:14.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.311+0000 7f296d44b740 1 -- 172.21.15.114:0/893606683 shutdown_connections 2024-09-13T18:24:14.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.311+0000 7f296d44b740 1 -- 172.21.15.114:0/893606683 wait complete. 2024-09-13T18:24:14.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.311+0000 7f296d44b740 1 -- start start 2024-09-13T18:24:14.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.311+0000 7f296d44b740 1 Processor -- start 2024-09-13T18:24:14.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.312+0000 7f296d44b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564194780e10 0x56419478ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:14.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.312+0000 7f296d44b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564194781730 0x564194788ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:14.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.312+0000 7f296d44b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56419478bfd0 0x564194789430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:14.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.312+0000 7f296d44b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641946862a0 con 0x564194781730 2024-09-13T18:24:14.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.312+0000 7f296d44b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641943bd430 con 0x564194780e10 2024-09-13T18:24:14.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.312+0000 7f296d44b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564194668680 con 0x56419478bfd0 2024-09-13T18:24:14.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.312+0000 7f2963fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564194781730 0x564194788ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:14.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.312+0000 7f29637fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56419478bfd0 0x564194789430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:14.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.312+0000 7f29637fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56419478bfd0 0x564194789430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:51780/0 (socket says 172.21.15.114:51780) 2024-09-13T18:24:14.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.312+0000 7f29637fe640 1 -- 172.21.15.114:0/1811302557 learned_addr learned my addr 172.21.15.114:0/1811302557 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:14.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.312+0000 7f296d44b740 10 client.0 _ll_register_callbacks cb 0x5641946de610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:24:14.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.312+0000 7f29637fe640 1 --2- 172.21.15.114:0/1811302557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564194780e10 0x56419478ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:14.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.312+0000 7f29637fe640 1 -- 172.21.15.114:0/1811302557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564194780e10 msgr2=0x56419478ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:14.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.312+0000 7f29637fe640 1 --2- 172.21.15.114:0/1811302557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564194780e10 0x56419478ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.312+0000 7f29637fe640 1 -- 172.21.15.114:0/1811302557 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564194781730 msgr2=0x564194788ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:14.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.312+0000 7f29637fe640 1 --2- 172.21.15.114:0/1811302557 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564194781730 0x564194788ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.312+0000 7f29637fe640 1 -- 172.21.15.114:0/1811302557 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f295801a040 con 0x56419478bfd0 2024-09-13T18:24:14.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.312+0000 7f2963fff640 1 --2- 172.21.15.114:0/1811302557 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564194781730 0x564194788ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:24:14.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.312+0000 7f29637fe640 1 --2- 172.21.15.114:0/1811302557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56419478bfd0 0x564194789430 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2958005740 tx=0x7f2958004c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8759657ddec8e527 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:14.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.312+0000 7f29437fe640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:24:14.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.313+0000 7f29437fe640 1 -- 172.21.15.114:0/1811302557 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2958023070 con 0x56419478bfd0 2024-09-13T18:24:14.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.313+0000 7f29437fe640 1 -- 172.21.15.114:0/1811302557 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f29580109e0 con 0x56419478bfd0 2024-09-13T18:24:14.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.313+0000 7f296d44b740 1 -- 172.21.15.114:0/1811302557 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5641943d84e0 con 0x56419478bfd0 2024-09-13T18:24:14.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.313+0000 7f29437fe640 1 -- 172.21.15.114:0/1811302557 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2958010ba0 con 0x56419478bfd0 2024-09-13T18:24:14.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.313+0000 7f296d44b740 10 client.25756 fetch_fsmap learned FSMap version 32 2024-09-13T18:24:14.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.313+0000 7f296d44b740 1 -- 172.21.15.114:0/1811302557 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5641946e3620 con 0x56419478bfd0 2024-09-13T18:24:14.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.313+0000 7f29437fe640 1 -- 172.21.15.114:0/1811302557 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5641943d84e0 con 0x56419478bfd0 2024-09-13T18:24:14.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.313+0000 7f29437fe640 1 -- 172.21.15.114:0/1811302557 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f295801f9b0 con 0x56419478bfd0 2024-09-13T18:24:14.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.313+0000 7f296d44b740 10 client.25756 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:24:14.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.313+0000 7f296d44b740 10 client.25756 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:24:14.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.313+0000 7f296d44b740 20 client.25756 populate_metadata read hostname 'smithi114' 2024-09-13T18:24:14.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.313+0000 7f296d44b740 1 --2- 172.21.15.114:0/1811302557 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56419479bc20 0x56419479e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:14.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.313+0000 7f296d44b740 4 client.25756 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:24:14.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.313+0000 7f296d44b740 1 -- 172.21.15.114:0/1811302557 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5641943e97e0 con 0x56419479bc20 2024-09-13T18:24:14.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.314+0000 7f2963fff640 1 --2- 172.21.15.114:0/1811302557 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56419479bc20 0x56419479e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:14.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.314+0000 7f29437fe640 10 client.25756 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:24:14.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.314+0000 7f2963fff640 1 --2- 172.21.15.114:0/1811302557 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56419479bc20 0x56419479e040 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=303ace7488a5fff2 server_cookie=14ac80643f90e169 in_seq=0 out_seq=0 2024-09-13T18:24:14.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.315+0000 7f29437fe640 1 -- 172.21.15.114:0/1811302557 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5641943e97e0 con 0x56419479bc20 2024-09-13T18:24:14.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.315+0000 7f29437fe640 10 client.25756 handle_command_reply: tid=0 2024-09-13T18:24:14.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.315+0000 7f296d44b740 1 client.25756 shutdown 2024-09-13T18:24:14.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.315+0000 7f296d44b740 1 -- 172.21.15.114:0/1811302557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56419478bfd0 msgr2=0x564194789430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:14.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.315+0000 7f296d44b740 1 --2- 172.21.15.114:0/1811302557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56419478bfd0 0x564194789430 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2958005740 tx=0x7f2958004c30 comp rx=0 tx=0).stop 2024-09-13T18:24:14.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.315+0000 7f296d44b740 1 -- 172.21.15.114:0/1811302557 shutdown_connections 2024-09-13T18:24:14.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.315+0000 7f296d44b740 1 --2- 172.21.15.114:0/1811302557 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56419479bc20 0x56419479e040 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.315+0000 7f296d44b740 1 --2- 172.21.15.114:0/1811302557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56419478bfd0 0x564194789430 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.315+0000 7f29437fe640 0 client.25756 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:24:14.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.315+0000 7f296d44b740 1 --2- 172.21.15.114:0/1811302557 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564194781730 0x564194788ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.315+0000 7f296d44b740 1 --2- 172.21.15.114:0/1811302557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564194780e10 0x56419478ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.315+0000 7f296d44b740 1 -- 172.21.15.114:0/1811302557 >> 172.21.15.114:0/1811302557 conn(0x56419477d380 msgr2=0x564194783090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:14.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.315+0000 7f296d44b740 1 -- 172.21.15.114:0/1811302557 shutdown_connections 2024-09-13T18:24:14.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.315+0000 7f296d44b740 1 -- 172.21.15.114:0/1811302557 wait complete. 2024-09-13T18:24:14.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.315+0000 7f296d44b740 20 client.25756 trim_cache size 0 max 16384 2024-09-13T18:24:14.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.337+0000 7f296d44b740 1 Processor -- start 2024-09-13T18:24:14.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.337+0000 7f296d44b740 1 -- start start 2024-09-13T18:24:14.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.337+0000 7f296d44b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564194780e10 0x56419477ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:14.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.337+0000 7f296d44b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564194781730 0x56419478e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:14.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.337+0000 7f296d44b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56419478bfd0 0x564194788510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:14.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.337+0000 7f296d44b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641947984d0 con 0x56419478bfd0 2024-09-13T18:24:14.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.337+0000 7f296d44b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564194788a50 con 0x564194781730 2024-09-13T18:24:14.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.337+0000 7f296d44b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564194789bb0 con 0x564194780e10 2024-09-13T18:24:14.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.337+0000 7f29637fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56419478bfd0 0x564194788510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:14.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.337+0000 7f29637fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56419478bfd0 0x564194788510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:54280/0 (socket says 172.21.15.114:54280) 2024-09-13T18:24:14.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.338+0000 7f29637fe640 1 -- 172.21.15.114:0/740782434 learned_addr learned my addr 172.21.15.114:0/740782434 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:14.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.338+0000 7f29637fe640 1 --2- 172.21.15.114:0/740782434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564194780e10 0x56419477ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:14.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.338+0000 7f2963fff640 1 --2- 172.21.15.114:0/740782434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564194781730 0x56419478e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:14.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.338+0000 7f29637fe640 1 -- 172.21.15.114:0/740782434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564194780e10 msgr2=0x56419477ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:14.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.338+0000 7f29637fe640 1 --2- 172.21.15.114:0/740782434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564194780e10 0x56419477ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.338+0000 7f29637fe640 1 -- 172.21.15.114:0/740782434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564194781730 msgr2=0x56419478e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:14.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.338+0000 7f29637fe640 1 --2- 172.21.15.114:0/740782434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564194781730 0x56419478e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.338+0000 7f29637fe640 1 -- 172.21.15.114:0/740782434 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564194685790 con 0x56419478bfd0 2024-09-13T18:24:14.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.338+0000 7f29637fe640 1 --2- 172.21.15.114:0/740782434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56419478bfd0 0x564194788510 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f295801fde0 tx=0x7f2958017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e16c759fa98e57f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:14.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.338+0000 7f294affd640 1 -- 172.21.15.114:0/740782434 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2958023070 con 0x56419478bfd0 2024-09-13T18:24:14.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.339+0000 7f294affd640 1 -- 172.21.15.114:0/740782434 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2958024040 con 0x56419478bfd0 2024-09-13T18:24:14.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.339+0000 7f294affd640 1 -- 172.21.15.114:0/740782434 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295802d4b0 con 0x56419478bfd0 2024-09-13T18:24:14.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.339+0000 7f296d44b740 1 -- 172.21.15.114:0/740782434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56419478bfd0 msgr2=0x564194788510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:14.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.339+0000 7f296d44b740 1 --2- 172.21.15.114:0/740782434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56419478bfd0 0x564194788510 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f295801fde0 tx=0x7f2958017040 comp rx=0 tx=0).stop 2024-09-13T18:24:14.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.339+0000 7f296d44b740 1 -- 172.21.15.114:0/740782434 shutdown_connections 2024-09-13T18:24:14.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.340+0000 7f296d44b740 1 --2- 172.21.15.114:0/740782434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56419478bfd0 0x564194788510 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.340+0000 7f296d44b740 1 --2- 172.21.15.114:0/740782434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564194781730 0x56419478e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.340+0000 7f296d44b740 1 --2- 172.21.15.114:0/740782434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564194780e10 0x56419477ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.340+0000 7f296d44b740 1 -- 172.21.15.114:0/740782434 >> 172.21.15.114:0/740782434 conn(0x56419477d380 msgr2=0x56419478b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:14.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.340+0000 7f296d44b740 1 -- 172.21.15.114:0/740782434 shutdown_connections 2024-09-13T18:24:14.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.340+0000 7f296d44b740 1 -- 172.21.15.114:0/740782434 wait complete. 2024-09-13T18:24:14.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.340+0000 7f296d44b740 1 -- start start 2024-09-13T18:24:14.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.340+0000 7f296d44b740 1 Processor -- start 2024-09-13T18:24:14.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.341+0000 7f296d44b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564194780e10 0x56419478ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:14.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.341+0000 7f296d44b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564194781730 0x56419478f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:14.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.341+0000 7f296d44b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56419478bfd0 0x56419478f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:14.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.341+0000 7f296d44b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641947f1110 con 0x564194780e10 2024-09-13T18:24:14.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.341+0000 7f296d44b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641947f1250 con 0x564194781730 2024-09-13T18:24:14.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.341+0000 7f296d44b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641947f1390 con 0x56419478bfd0 2024-09-13T18:24:14.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.341+0000 7f29637fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564194780e10 0x56419478ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:14.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.341+0000 7f2963fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564194781730 0x56419478f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:14.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.341+0000 7f296d44b740 10 client.0 _ll_register_callbacks cb 0x7f2958009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:24:14.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.341+0000 7f29637fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564194780e10 0x56419478ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:54296/0 (socket says 172.21.15.114:54296) 2024-09-13T18:24:14.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.341+0000 7f29637fe640 1 -- 172.21.15.114:0/349906088 learned_addr learned my addr 172.21.15.114:0/349906088 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:14.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.341+0000 7f29637fe640 1 --2- 172.21.15.114:0/349906088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56419478bfd0 0x56419478f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:14.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.341+0000 7f29637fe640 1 -- 172.21.15.114:0/349906088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56419478bfd0 msgr2=0x56419478f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:14.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.341+0000 7f29637fe640 1 --2- 172.21.15.114:0/349906088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56419478bfd0 0x56419478f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.341+0000 7f29637fe640 1 -- 172.21.15.114:0/349906088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564194781730 msgr2=0x56419478f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:14.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.341+0000 7f29637fe640 1 --2- 172.21.15.114:0/349906088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564194781730 0x56419478f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.341+0000 7f29637fe640 1 -- 172.21.15.114:0/349906088 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5641946e3620 con 0x564194780e10 2024-09-13T18:24:14.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.342+0000 7f29637fe640 1 --2- 172.21.15.114:0/349906088 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564194780e10 0x56419478ed10 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f2958004c70 tx=0x7f2958012ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85e1225e67ac5e0a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:14.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.342+0000 7f29437fe640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:24:14.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.342+0000 7f29437fe640 1 -- 172.21.15.114:0/349906088 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2958023030 con 0x564194780e10 2024-09-13T18:24:14.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.342+0000 7f29437fe640 1 -- 172.21.15.114:0/349906088 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2958010bf0 con 0x564194780e10 2024-09-13T18:24:14.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.342+0000 7f296d44b740 1 -- 172.21.15.114:0/349906088 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f29580054b0 con 0x564194780e10 2024-09-13T18:24:14.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.342+0000 7f29437fe640 1 -- 172.21.15.114:0/349906088 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f295802d4b0 con 0x564194780e10 2024-09-13T18:24:14.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.342+0000 7f296d44b740 10 client.16395 fetch_fsmap learned FSMap version 32 2024-09-13T18:24:14.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.342+0000 7f296d44b740 1 -- 172.21.15.114:0/349906088 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5641946aaea0 con 0x564194780e10 2024-09-13T18:24:14.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.342+0000 7f29437fe640 1 -- 172.21.15.114:0/349906088 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f29580054b0 con 0x564194780e10 2024-09-13T18:24:14.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.343+0000 7f29437fe640 1 -- 172.21.15.114:0/349906088 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f295803a040 con 0x564194780e10 2024-09-13T18:24:14.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.343+0000 7f296d44b740 10 client.16395 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:24:14.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.343+0000 7f296d44b740 10 client.16395 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:24:14.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.343+0000 7f296d44b740 20 client.16395 populate_metadata read hostname 'smithi114' 2024-09-13T18:24:14.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.343+0000 7f296d44b740 1 --2- 172.21.15.114:0/349906088 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56419479bc20 0x5641947f46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:14.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.343+0000 7f296d44b740 4 client.16395 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T18:24:14.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.343+0000 7f296d44b740 1 -- 172.21.15.114:0/349906088 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56419443ce30 con 0x56419479bc20 2024-09-13T18:24:14.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.343+0000 7f2963fff640 1 --2- 172.21.15.114:0/349906088 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56419479bc20 0x5641947f46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:14.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.343+0000 7f29437fe640 10 client.16395 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:24:14.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.344+0000 7f2963fff640 1 --2- 172.21.15.114:0/349906088 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56419479bc20 0x5641947f46f0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd11cd2143203f50 server_cookie=e44ebc6c4931778a in_seq=0 out_seq=0 2024-09-13T18:24:14.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.344+0000 7f29437fe640 1 -- 172.21.15.114:0/349906088 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56419443ce30 con 0x56419479bc20 2024-09-13T18:24:14.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.344+0000 7f29437fe640 10 client.16395 handle_command_reply: tid=0 2024-09-13T18:24:14.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.344+0000 7f296d44b740 1 client.16395 shutdown 2024-09-13T18:24:14.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.344+0000 7f296d44b740 1 -- 172.21.15.114:0/349906088 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564194780e10 msgr2=0x56419478ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:14.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.344+0000 7f296d44b740 1 --2- 172.21.15.114:0/349906088 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564194780e10 0x56419478ed10 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f2958004c70 tx=0x7f2958012ee0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.344+0000 7f296d44b740 1 -- 172.21.15.114:0/349906088 shutdown_connections 2024-09-13T18:24:14.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.344+0000 7f296d44b740 1 --2- 172.21.15.114:0/349906088 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56419479bc20 0x5641947f46f0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.344+0000 7f296d44b740 1 --2- 172.21.15.114:0/349906088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56419478bfd0 0x56419478f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.344+0000 7f296d44b740 1 --2- 172.21.15.114:0/349906088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564194781730 0x56419478f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.344+0000 7f296d44b740 1 --2- 172.21.15.114:0/349906088 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564194780e10 0x56419478ed10 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.344+0000 7f296d44b740 1 -- 172.21.15.114:0/349906088 >> 172.21.15.114:0/349906088 conn(0x56419477d380 msgr2=0x5641947a29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:14.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.344+0000 7f29437fe640 0 client.16395 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:24:14.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.344+0000 7f296d44b740 1 -- 172.21.15.114:0/349906088 shutdown_connections 2024-09-13T18:24:14.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.344+0000 7f296d44b740 1 -- 172.21.15.114:0/349906088 wait complete. 2024-09-13T18:24:14.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.345+0000 7f296d44b740 20 client.16395 trim_cache size 0 max 16384 2024-09-13T18:24:14.355 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:24:14.355 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-13T18:24:14.355 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": { 2024-09-13T18:24:14.355 INFO:teuthology.orchestra.run.smithi114.stdout: "7d4aea73-3e2f-4a57-bcba-035cf9ad7f33": { 2024-09-13T18:24:14.355 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-13T18:24:14.355 INFO:teuthology.orchestra.run.smithi114.stdout: "tag": "7d4aea73-3e2f-4a57-bcba-035cf9ad7f33", 2024-09-13T18:24:14.355 INFO:teuthology.orchestra.run.smithi114.stdout: "options": "recursive,force" 2024-09-13T18:24:14.355 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:24:14.355 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:24:14.355 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:24:14.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.345+0000 7f296a725640 1 -- 172.21.15.114:0/2775540278 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f294c0a1280 msgr2=0x7f294c0a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:14.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.345+0000 7f296a725640 1 --2- 172.21.15.114:0/2775540278 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f294c0a1280 0x7f294c0a3740 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f295c096bb0 tx=0x7f295c06d2d0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.345+0000 7f296a725640 1 -- 172.21.15.114:0/2775540278 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f296410a310 msgr2=0x7f2964108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:14.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.345+0000 7f296a725640 1 --2- 172.21.15.114:0/2775540278 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f296410a310 0x7f2964108ea0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f295c00cab0 tx=0x7f295c00cf70 comp rx=0 tx=0).stop 2024-09-13T18:24:14.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.345+0000 7f296a725640 1 -- 172.21.15.114:0/2775540278 shutdown_connections 2024-09-13T18:24:14.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.345+0000 7f296a725640 1 --2- 172.21.15.114:0/2775540278 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f294c0a1280 0x7f294c0a3740 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.345+0000 7f296a725640 1 --2- 172.21.15.114:0/2775540278 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f296410a310 0x7f2964108ea0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.345+0000 7f296a725640 1 --2- 172.21.15.114:0/2775540278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2964101860 0x7f29641a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.345+0000 7f296a725640 1 --2- 172.21.15.114:0/2775540278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2964100f40 0x7f29641a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:14.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.345+0000 7f296a725640 1 -- 172.21.15.114:0/2775540278 >> 172.21.15.114:0/2775540278 conn(0x7f2964075c60 msgr2=0x7f29640fb420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:14.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.345+0000 7f296a725640 1 -- 172.21.15.114:0/2775540278 shutdown_connections 2024-09-13T18:24:14.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:14.346+0000 7f296a725640 1 -- 172.21.15.114:0/2775540278 wait complete. 2024-09-13T18:24:14.361 INFO:tasks.cephfs.filesystem:scrub status for tag:7d4aea73-3e2f-4a57-bcba-035cf9ad7f33 - {'path': '/', 'tag': '7d4aea73-3e2f-4a57-bcba-035cf9ad7f33', 'options': 'recursive,force'} 2024-09-13T18:24:14.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:14 smithi204 ceph-mon[34777]: pgmap v56: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 221 MiB/s wr, 142 op/s 2024-09-13T18:24:14.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:14 smithi134 ceph-mon[34283]: pgmap v56: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 221 MiB/s wr, 142 op/s 2024-09-13T18:24:15.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:15 smithi204 ceph-mon[34777]: scrub queued for path: / 2024-09-13T18:24:15.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:15 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:24:15.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:15 smithi204 ceph-mon[34777]: scrub summary: active paths [/] 2024-09-13T18:24:15.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:15 smithi204 ceph-mon[34777]: scrub summary: active paths [/] 2024-09-13T18:24:15.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:15 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:24:15.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:15 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:24:15.469 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:15 smithi114 ceph-mon[30897]: scrub queued for path: / 2024-09-13T18:24:15.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:15 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:24:15.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:15 smithi114 ceph-mon[30897]: scrub summary: active paths [/] 2024-09-13T18:24:15.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:15 smithi114 ceph-mon[30897]: scrub summary: active paths [/] 2024-09-13T18:24:15.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:15 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:24:15.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:15 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:24:15.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:15 smithi134 ceph-mon[34283]: scrub queued for path: / 2024-09-13T18:24:15.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:15 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:24:15.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:15 smithi134 ceph-mon[34283]: scrub summary: active paths [/] 2024-09-13T18:24:15.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:15 smithi134 ceph-mon[34283]: scrub summary: active paths [/] 2024-09-13T18:24:15.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:15 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:24:15.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:15 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:24:16.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:16 smithi204 ceph-mon[34777]: pgmap v57: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 147 MiB/s wr, 85 op/s 2024-09-13T18:24:16.469 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:16 smithi114 ceph-mon[30897]: pgmap v57: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 147 MiB/s wr, 85 op/s 2024-09-13T18:24:16.519 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:16 smithi134 ceph-mon[34283]: pgmap v57: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 147 MiB/s wr, 85 op/s 2024-09-13T18:24:18.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:18 smithi204 ceph-mon[34777]: pgmap v58: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 147 MiB/s wr, 85 op/s 2024-09-13T18:24:18.469 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:18 smithi114 ceph-mon[30897]: pgmap v58: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 147 MiB/s wr, 85 op/s 2024-09-13T18:24:18.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:18 smithi134 ceph-mon[34283]: pgmap v58: 129 pgs: 129 active+clean; 5.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 147 MiB/s wr, 85 op/s 2024-09-13T18:24:20.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:20 smithi204 ceph-mon[34777]: pgmap v59: 129 pgs: 129 active+clean; 6.6 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 223 MiB/s wr, 131 op/s 2024-09-13T18:24:20.469 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:20 smithi114 ceph-mon[30897]: pgmap v59: 129 pgs: 129 active+clean; 6.6 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 223 MiB/s wr, 131 op/s 2024-09-13T18:24:20.621 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:20 smithi134 ceph-mon[34283]: pgmap v59: 129 pgs: 129 active+clean; 6.6 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 223 MiB/s wr, 131 op/s 2024-09-13T18:24:21.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:21 smithi134 ceph-mon[34283]: scrub summary: idle 2024-09-13T18:24:21.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:21 smithi204 ceph-mon[34777]: scrub summary: idle 2024-09-13T18:24:21.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:21 smithi114 ceph-mon[30897]: scrub summary: idle 2024-09-13T18:24:22.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:22 smithi134 ceph-mon[34283]: scrub summary: idle 2024-09-13T18:24:22.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:22 smithi134 ceph-mon[34283]: pgmap v60: 129 pgs: 129 active+clean; 6.6 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 148 MiB/s wr, 88 op/s 2024-09-13T18:24:22.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:22 smithi204 ceph-mon[34777]: scrub summary: idle 2024-09-13T18:24:22.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:22 smithi204 ceph-mon[34777]: pgmap v60: 129 pgs: 129 active+clean; 6.6 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 148 MiB/s wr, 88 op/s 2024-09-13T18:24:22.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:22 smithi114 ceph-mon[30897]: scrub summary: idle 2024-09-13T18:24:22.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:22 smithi114 ceph-mon[30897]: pgmap v60: 129 pgs: 129 active+clean; 6.6 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 148 MiB/s wr, 88 op/s 2024-09-13T18:24:23.305 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.302+0000 7f7cf93aa640 1 -- 172.21.15.114:0/295744997 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cf40feff0 msgr2=0x7f7cf40ff430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:23.305 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.302+0000 7f7cf93aa640 1 --2- 172.21.15.114:0/295744997 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cf40feff0 0x7f7cf40ff430 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f7ce8009900 tx=0x7f7ce802a3f0 comp rx=0 tx=0).stop 2024-09-13T18:24:23.305 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.302+0000 7f7cf93aa640 1 -- 172.21.15.114:0/295744997 shutdown_connections 2024-09-13T18:24:23.305 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.302+0000 7f7cf93aa640 1 --2- 172.21.15.114:0/295744997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7cf40ff970 0x7f7cf410ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:23.305 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.302+0000 7f7cf93aa640 1 --2- 172.21.15.114:0/295744997 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cf40feff0 0x7f7cf40ff430 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:23.305 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.302+0000 7f7cf93aa640 1 --2- 172.21.15.114:0/295744997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7cf4104c30 0x7f7cf4105010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:23.306 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.303+0000 7f7cf93aa640 1 -- 172.21.15.114:0/295744997 >> 172.21.15.114:0/295744997 conn(0x7f7cf4074940 msgr2=0x7f7cf4074d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:23.306 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.303+0000 7f7cf93aa640 1 -- 172.21.15.114:0/295744997 shutdown_connections 2024-09-13T18:24:23.306 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.303+0000 7f7cf93aa640 1 -- 172.21.15.114:0/295744997 wait complete. 2024-09-13T18:24:23.307 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.303+0000 7f7cf93aa640 1 Processor -- start 2024-09-13T18:24:23.307 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.304+0000 7f7cf93aa640 1 -- start start 2024-09-13T18:24:23.307 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.304+0000 7f7cf93aa640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cf40feff0 0x7f7cf410c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:23.308 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.304+0000 7f7cf93aa640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7cf40ff970 0x7f7cf410c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:23.308 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.304+0000 7f7cf93aa640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7cf4104c30 0x7f7cf410cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:23.308 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.304+0000 7f7cf93aa640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf410e280 con 0x7f7cf40feff0 2024-09-13T18:24:23.308 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.304+0000 7f7cf93aa640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf41a8ab0 con 0x7f7cf4104c30 2024-09-13T18:24:23.308 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.304+0000 7f7cf93aa640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf41a8c70 con 0x7f7cf40ff970 2024-09-13T18:24:23.308 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.304+0000 7f7cf2ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cf40feff0 0x7f7cf410c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:23.308 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.304+0000 7f7cf2ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cf40feff0 0x7f7cf410c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37486/0 (socket says 172.21.15.114:37486) 2024-09-13T18:24:23.308 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.304+0000 7f7cf2ffd640 1 -- 172.21.15.114:0/2030029688 learned_addr learned my addr 172.21.15.114:0/2030029688 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:23.308 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.304+0000 7f7cf37fe640 1 --2- 172.21.15.114:0/2030029688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7cf4104c30 0x7f7cf410cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:23.308 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.304+0000 7f7cf27fc640 1 --2- 172.21.15.114:0/2030029688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7cf40ff970 0x7f7cf410c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:23.308 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.305+0000 7f7cf27fc640 1 -- 172.21.15.114:0/2030029688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7cf4104c30 msgr2=0x7f7cf410cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.305+0000 7f7cf27fc640 1 --2- 172.21.15.114:0/2030029688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7cf4104c30 0x7f7cf410cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.305+0000 7f7cf27fc640 1 -- 172.21.15.114:0/2030029688 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cf40feff0 msgr2=0x7f7cf410c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.305+0000 7f7cf27fc640 1 --2- 172.21.15.114:0/2030029688 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cf40feff0 0x7f7cf410c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.305+0000 7f7cf27fc640 1 -- 172.21.15.114:0/2030029688 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce8009560 con 0x7f7cf40ff970 2024-09-13T18:24:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.305+0000 7f7cf2ffd640 1 --2- 172.21.15.114:0/2030029688 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cf40feff0 0x7f7cf410c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:24:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.305+0000 7f7cf27fc640 1 --2- 172.21.15.114:0/2030029688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7cf40ff970 0x7f7cf410c810 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7ce802a910 tx=0x7f7ce8005610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.306+0000 7f7cdbfff640 1 -- 172.21.15.114:0/2030029688 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce8004c50 con 0x7f7cf40ff970 2024-09-13T18:24:23.310 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.306+0000 7f7cdbfff640 1 -- 172.21.15.114:0/2030029688 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7ce8004db0 con 0x7f7cf40ff970 2024-09-13T18:24:23.310 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.306+0000 7f7cdbfff640 1 -- 172.21.15.114:0/2030029688 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce803c960 con 0x7f7cf40ff970 2024-09-13T18:24:23.310 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.306+0000 7f7cf93aa640 1 -- 172.21.15.114:0/2030029688 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cf41a8ef0 con 0x7f7cf40ff970 2024-09-13T18:24:23.310 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.306+0000 7f7cf93aa640 1 -- 172.21.15.114:0/2030029688 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cf41a9260 con 0x7f7cf40ff970 2024-09-13T18:24:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.309+0000 7f7cdbfff640 1 -- 172.21.15.114:0/2030029688 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f7ce803cac0 con 0x7f7cf40ff970 2024-09-13T18:24:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.309+0000 7f7cfc0ca740 1 Processor -- start 2024-09-13T18:24:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.309+0000 7f7cfc0ca740 1 -- start start 2024-09-13T18:24:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.310+0000 7f7cfc0ca740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb8066d330 0x55bb8066d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.310+0000 7f7cfc0ca740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb8066dc50 0x55bb80677fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.310+0000 7f7cfc0ca740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bb806784f0 0x55bb8067a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.310+0000 7f7cfc0ca740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb805d8c30 con 0x55bb806784f0 2024-09-13T18:24:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.310+0000 7f7cfc0ca740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb805d63c0 con 0x55bb8066dc50 2024-09-13T18:24:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.310+0000 7f7cfc0ca740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb805d1cb0 con 0x55bb8066d330 2024-09-13T18:24:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.310+0000 7f7cf37fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb8066d330 0x55bb8066d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.310+0000 7f7cf2ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb8066dc50 0x55bb80677fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.310+0000 7f7cf37fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb8066d330 0x55bb8066d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:57330/0 (socket says 172.21.15.114:57330) 2024-09-13T18:24:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.310+0000 7f7cf37fe640 1 -- 172.21.15.114:0/3203307337 learned_addr learned my addr 172.21.15.114:0/3203307337 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.310+0000 7f7cf27fc640 1 --2- 172.21.15.114:0/3203307337 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bb806784f0 0x55bb8067a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.310+0000 7f7cdbfff640 1 --2- 172.21.15.114:0/2030029688 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7cdc0a1280 0x7f7cdc0a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.310+0000 7f7cf2ffd640 1 -- 172.21.15.114:0/3203307337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb8066d330 msgr2=0x55bb8066d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.310+0000 7f7cf2ffd640 1 --2- 172.21.15.114:0/3203307337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb8066d330 0x55bb8066d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.310+0000 7f7cf2ffd640 1 -- 172.21.15.114:0/3203307337 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bb806784f0 msgr2=0x55bb8067a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.310+0000 7f7cf2ffd640 1 --2- 172.21.15.114:0/3203307337 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bb806784f0 0x55bb8067a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.310+0000 7f7cf2ffd640 1 -- 172.21.15.114:0/3203307337 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb80578910 con 0x55bb8066dc50 2024-09-13T18:24:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.310+0000 7f7cdbfff640 1 -- 172.21.15.114:0/2030029688 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f7ce80ee0a0 con 0x7f7cf40ff970 2024-09-13T18:24:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.311+0000 7f7cf37fe640 1 --2- 172.21.15.114:0/2030029688 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7cdc0a1280 0x7f7cdc0a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.311+0000 7f7cf2ffd640 1 --2- 172.21.15.114:0/3203307337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb8066dc50 0x55bb80677fb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4011440 tx=0x7f7ce4011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f393fd232364090 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.311+0000 7f7cd97fa640 1 -- 172.21.15.114:0/3203307337 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce4004030 con 0x55bb8066dc50 2024-09-13T18:24:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.311+0000 7f7cd97fa640 1 -- 172.21.15.114:0/3203307337 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7ce40035f0 con 0x55bb8066dc50 2024-09-13T18:24:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.311+0000 7f7cd97fa640 1 -- 172.21.15.114:0/3203307337 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce400f860 con 0x55bb8066dc50 2024-09-13T18:24:23.318 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.312+0000 7f7cfc0ca740 1 -- 172.21.15.114:0/3203307337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb8066dc50 msgr2=0x55bb80677fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:23.318 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.312+0000 7f7cfc0ca740 1 --2- 172.21.15.114:0/3203307337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb8066dc50 0x55bb80677fb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4011440 tx=0x7f7ce4011900 comp rx=0 tx=0).stop 2024-09-13T18:24:23.319 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.312+0000 7f7cf37fe640 1 --2- 172.21.15.114:0/2030029688 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7cdc0a1280 0x7f7cdc0a3740 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7cec007fd0 tx=0x7f7cec007e40 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:23.319 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.312+0000 7f7cfc0ca740 1 -- 172.21.15.114:0/3203307337 shutdown_connections 2024-09-13T18:24:23.319 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.312+0000 7f7cfc0ca740 1 --2- 172.21.15.114:0/3203307337 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bb806784f0 0x55bb8067a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:23.319 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.312+0000 7f7cfc0ca740 1 --2- 172.21.15.114:0/3203307337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb8066dc50 0x55bb80677fb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:23.319 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.312+0000 7f7cfc0ca740 1 --2- 172.21.15.114:0/3203307337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb8066d330 0x55bb8066d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:23.319 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.312+0000 7f7cfc0ca740 1 -- 172.21.15.114:0/3203307337 >> 172.21.15.114:0/3203307337 conn(0x55bb806698c0 msgr2=0x55bb80669ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:23.319 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.312+0000 7f7cfc0ca740 1 -- 172.21.15.114:0/3203307337 shutdown_connections 2024-09-13T18:24:23.319 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.312+0000 7f7cfc0ca740 1 -- 172.21.15.114:0/3203307337 wait complete. 2024-09-13T18:24:23.319 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.312+0000 7f7cfc0ca740 1 -- start start 2024-09-13T18:24:23.319 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.312+0000 7f7cfc0ca740 1 Processor -- start 2024-09-13T18:24:23.319 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.312+0000 7f7cfc0ca740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bb8066d330 0x55bb80677da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:23.319 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.313+0000 7f7cfc0ca740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb8066dc50 0x55bb80675020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.313+0000 7f7cfc0ca740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb806784f0 0x55bb80675560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.313+0000 7f7cfc0ca740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb8058fe80 con 0x55bb8066d330 2024-09-13T18:24:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.313+0000 7f7cf2ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bb8066d330 0x55bb80677da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.313+0000 7f7cfc0ca740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb8058fb20 con 0x55bb806784f0 2024-09-13T18:24:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.313+0000 7f7cfc0ca740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb80555260 con 0x55bb8066dc50 2024-09-13T18:24:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.313+0000 7f7cf2ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bb8066d330 0x55bb80677da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37504/0 (socket says 172.21.15.114:37504) 2024-09-13T18:24:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.313+0000 7f7cf2ffd640 1 -- 172.21.15.114:0/422588099 learned_addr learned my addr 172.21.15.114:0/422588099 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.313+0000 7f7cfc0ca740 10 client.0 _ll_register_callbacks cb 0x55bb8058adf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:24:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.313+0000 7f7cf37fe640 1 --2- 172.21.15.114:0/422588099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb806784f0 0x55bb80675560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.313+0000 7f7cf2ffd640 1 -- 172.21.15.114:0/422588099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb8066dc50 msgr2=0x55bb80675020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.313+0000 7f7cf2ffd640 1 --2- 172.21.15.114:0/422588099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb8066dc50 0x55bb80675020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.313+0000 7f7cf2ffd640 1 -- 172.21.15.114:0/422588099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb806784f0 msgr2=0x55bb80675560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.313+0000 7f7cf2ffd640 1 --2- 172.21.15.114:0/422588099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb806784f0 0x55bb80675560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.313+0000 7f7cf2ffd640 1 -- 172.21.15.114:0/422588099 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ce4009c50 con 0x55bb8066d330 2024-09-13T18:24:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.313+0000 7f7cf37fe640 1 --2- 172.21.15.114:0/422588099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb806784f0 0x55bb80675560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:24:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.313+0000 7f7cf2ffd640 1 --2- 172.21.15.114:0/422588099 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bb8066d330 0x55bb80677da0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f7ce401b040 tx=0x7f7ce4004200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=656529ab93927e53 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.313+0000 7f7cd6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:24:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.314+0000 7f7cd6ff5640 1 -- 172.21.15.114:0/422588099 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce4003d20 con 0x55bb8066d330 2024-09-13T18:24:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.314+0000 7f7cd6ff5640 1 -- 172.21.15.114:0/422588099 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f7ce4005030 con 0x55bb8066d330 2024-09-13T18:24:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.314+0000 7f7cfc0ca740 1 -- 172.21.15.114:0/422588099 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bb805fa680 con 0x55bb8066d330 2024-09-13T18:24:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.314+0000 7f7cd6ff5640 1 -- 172.21.15.114:0/422588099 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ce40051f0 con 0x55bb8066d330 2024-09-13T18:24:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.314+0000 7f7cfc0ca740 10 client.16407 fetch_fsmap learned FSMap version 32 2024-09-13T18:24:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.314+0000 7f7cfc0ca740 1 -- 172.21.15.114:0/422588099 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55bb805d0330 con 0x55bb8066d330 2024-09-13T18:24:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.314+0000 7f7cd6ff5640 1 -- 172.21.15.114:0/422588099 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55bb805fa680 con 0x55bb8066d330 2024-09-13T18:24:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.314+0000 7f7cd6ff5640 1 -- 172.21.15.114:0/422588099 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f7ce4021070 con 0x55bb8066d330 2024-09-13T18:24:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.314+0000 7f7cfc0ca740 10 client.16407 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:24:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.314+0000 7f7cfc0ca740 10 client.16407 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:24:23.322 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.314+0000 7f7cfc0ca740 20 client.16407 populate_metadata read hostname 'smithi114' 2024-09-13T18:24:23.322 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.314+0000 7f7cfc0ca740 1 --2- 172.21.15.114:0/422588099 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55bb80687e50 0x55bb8068a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:23.322 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.314+0000 7f7cfc0ca740 4 client.16407 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:24:23.322 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.314+0000 7f7cfc0ca740 1 -- 172.21.15.114:0/422588099 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bb805c55d0 con 0x55bb80687e50 2024-09-13T18:24:23.322 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.315+0000 7f7cf27fc640 1 --2- 172.21.15.114:0/422588099 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55bb80687e50 0x55bb8068a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:23.322 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.315+0000 7f7cd6ff5640 10 client.16407 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:24:23.322 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.315+0000 7f7cf27fc640 1 --2- 172.21.15.114:0/422588099 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55bb80687e50 0x55bb8068a270 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f87f110a5a771d91 server_cookie=1eff286907700c72 in_seq=0 out_seq=0 2024-09-13T18:24:23.322 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.316+0000 7f7cd6ff5640 1 -- 172.21.15.114:0/422588099 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55bb805c55d0 con 0x55bb80687e50 2024-09-13T18:24:23.322 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.316+0000 7f7cd6ff5640 10 client.16407 handle_command_reply: tid=0 2024-09-13T18:24:23.322 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.316+0000 7f7cfc0ca740 1 client.16407 shutdown 2024-09-13T18:24:23.322 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.316+0000 7f7cfc0ca740 1 -- 172.21.15.114:0/422588099 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bb8066d330 msgr2=0x55bb80677da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:23.322 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.316+0000 7f7cfc0ca740 1 --2- 172.21.15.114:0/422588099 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bb8066d330 0x55bb80677da0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f7ce401b040 tx=0x7f7ce4004200 comp rx=0 tx=0).stop 2024-09-13T18:24:23.322 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.316+0000 7f7cfc0ca740 1 -- 172.21.15.114:0/422588099 shutdown_connections 2024-09-13T18:24:23.322 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.316+0000 7f7cfc0ca740 1 --2- 172.21.15.114:0/422588099 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55bb80687e50 0x55bb8068a270 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.316+0000 7f7cfc0ca740 1 --2- 172.21.15.114:0/422588099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb806784f0 0x55bb80675560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.316+0000 7f7cfc0ca740 1 --2- 172.21.15.114:0/422588099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb8066dc50 0x55bb80675020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.316+0000 7f7cfc0ca740 1 --2- 172.21.15.114:0/422588099 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bb8066d330 0x55bb80677da0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.316+0000 7f7cd6ff5640 0 client.16407 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:24:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.316+0000 7f7cfc0ca740 1 -- 172.21.15.114:0/422588099 >> 172.21.15.114:0/422588099 conn(0x55bb806698c0 msgr2=0x55bb80668b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.316+0000 7f7cfc0ca740 1 -- 172.21.15.114:0/422588099 shutdown_connections 2024-09-13T18:24:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.316+0000 7f7cfc0ca740 1 -- 172.21.15.114:0/422588099 wait complete. 2024-09-13T18:24:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.316+0000 7f7cfc0ca740 20 client.16407 trim_cache size 0 max 16384 2024-09-13T18:24:23.336 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.334+0000 7f7cfc0ca740 1 Processor -- start 2024-09-13T18:24:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.334+0000 7f7cfc0ca740 1 -- start start 2024-09-13T18:24:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.334+0000 7f7cfc0ca740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb8066d330 0x55bb80677900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.334+0000 7f7cfc0ca740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb8066dc50 0x55bb806722a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.334+0000 7f7cfc0ca740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bb806784f0 0x55bb806727e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.334+0000 7f7cfc0ca740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb806758d0 con 0x55bb806784f0 2024-09-13T18:24:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.334+0000 7f7cfc0ca740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb80675de0 con 0x55bb8066dc50 2024-09-13T18:24:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.334+0000 7f7cfc0ca740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb80673570 con 0x55bb8066d330 2024-09-13T18:24:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.334+0000 7f7cf37fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bb806784f0 0x55bb806727e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:23.338 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.335+0000 7f7cf2ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb8066d330 0x55bb80677900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:23.338 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.335+0000 7f7cf2ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb8066d330 0x55bb80677900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:57348/0 (socket says 172.21.15.114:57348) 2024-09-13T18:24:23.338 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.335+0000 7f7cf2ffd640 1 -- 172.21.15.114:0/883570010 learned_addr learned my addr 172.21.15.114:0/883570010 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:23.338 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.335+0000 7f7cf27fc640 1 --2- 172.21.15.114:0/883570010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb8066dc50 0x55bb806722a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:23.338 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.335+0000 7f7cf2ffd640 1 -- 172.21.15.114:0/883570010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb8066dc50 msgr2=0x55bb806722a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.335+0000 7f7cf2ffd640 1 --2- 172.21.15.114:0/883570010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb8066dc50 0x55bb806722a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.335+0000 7f7cf2ffd640 1 -- 172.21.15.114:0/883570010 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bb806784f0 msgr2=0x55bb806727e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.335+0000 7f7cf2ffd640 1 --2- 172.21.15.114:0/883570010 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bb806784f0 0x55bb806727e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.335+0000 7f7cf2ffd640 1 -- 172.21.15.114:0/883570010 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb80572300 con 0x55bb8066d330 2024-09-13T18:24:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.335+0000 7f7cf37fe640 1 --2- 172.21.15.114:0/883570010 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bb806784f0 0x55bb806727e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:24:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.336+0000 7f7cf2ffd640 1 --2- 172.21.15.114:0/883570010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb8066d330 0x55bb80677900 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4009620 tx=0x7f7ce4011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a51df89210f6de7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.336+0000 7f7cd9ffb640 1 -- 172.21.15.114:0/883570010 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce400f870 con 0x55bb8066d330 2024-09-13T18:24:23.340 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.337+0000 7f7cd9ffb640 1 -- 172.21.15.114:0/883570010 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7ce4003c80 con 0x55bb8066d330 2024-09-13T18:24:23.340 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.337+0000 7f7cd9ffb640 1 -- 172.21.15.114:0/883570010 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce4021590 con 0x55bb8066d330 2024-09-13T18:24:23.340 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.338+0000 7f7cfc0ca740 1 -- 172.21.15.114:0/883570010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb8066d330 msgr2=0x55bb80677900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:23.341 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.338+0000 7f7cfc0ca740 1 --2- 172.21.15.114:0/883570010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb8066d330 0x55bb80677900 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4009620 tx=0x7f7ce4011900 comp rx=0 tx=0).stop 2024-09-13T18:24:23.341 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.338+0000 7f7cfc0ca740 1 -- 172.21.15.114:0/883570010 shutdown_connections 2024-09-13T18:24:23.341 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.338+0000 7f7cfc0ca740 1 --2- 172.21.15.114:0/883570010 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bb806784f0 0x55bb806727e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:23.341 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.338+0000 7f7cfc0ca740 1 --2- 172.21.15.114:0/883570010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb8066dc50 0x55bb806722a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:23.341 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.338+0000 7f7cfc0ca740 1 --2- 172.21.15.114:0/883570010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb8066d330 0x55bb80677900 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:23.341 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.338+0000 7f7cfc0ca740 1 -- 172.21.15.114:0/883570010 >> 172.21.15.114:0/883570010 conn(0x55bb806698c0 msgr2=0x55bb806738f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:23.341 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.338+0000 7f7cfc0ca740 1 -- 172.21.15.114:0/883570010 shutdown_connections 2024-09-13T18:24:23.341 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.338+0000 7f7cfc0ca740 1 -- 172.21.15.114:0/883570010 wait complete. 2024-09-13T18:24:23.341 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.338+0000 7f7cfc0ca740 1 -- start start 2024-09-13T18:24:23.341 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.339+0000 7f7cfc0ca740 1 Processor -- start 2024-09-13T18:24:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.339+0000 7f7cfc0ca740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb8066d330 0x55bb8067dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.339+0000 7f7cfc0ca740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb8066dc50 0x55bb8067e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.339+0000 7f7cfc0ca740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bb806784f0 0x55bb8067e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.339+0000 7f7cfc0ca740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb806de3a0 con 0x55bb806784f0 2024-09-13T18:24:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.339+0000 7f7cfc0ca740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb806de4e0 con 0x55bb8066dc50 2024-09-13T18:24:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.339+0000 7f7cfc0ca740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb806de620 con 0x55bb8066d330 2024-09-13T18:24:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.339+0000 7f7cfc0ca740 10 client.0 _ll_register_callbacks cb 0x7f7ce4009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:24:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.339+0000 7f7cf27fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb8066dc50 0x55bb8067e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.339+0000 7f7cf27fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb8066dc50 0x55bb8067e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:43154/0 (socket says 172.21.15.114:43154) 2024-09-13T18:24:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.339+0000 7f7cf27fc640 1 -- 172.21.15.114:0/3814489714 learned_addr learned my addr 172.21.15.114:0/3814489714 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.339+0000 7f7cf2ffd640 1 --2- 172.21.15.114:0/3814489714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb8066d330 0x55bb8067dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.339+0000 7f7cf37fe640 1 --2- 172.21.15.114:0/3814489714 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bb806784f0 0x55bb8067e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.339+0000 7f7cf27fc640 1 -- 172.21.15.114:0/3814489714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb8066d330 msgr2=0x55bb8067dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.339+0000 7f7cf27fc640 1 --2- 172.21.15.114:0/3814489714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb8066d330 0x55bb8067dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.339+0000 7f7cf27fc640 1 -- 172.21.15.114:0/3814489714 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bb806784f0 msgr2=0x55bb8067e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.339+0000 7f7cf27fc640 1 --2- 172.21.15.114:0/3814489714 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bb806784f0 0x55bb8067e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.339+0000 7f7cf27fc640 1 -- 172.21.15.114:0/3814489714 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cec013d40 con 0x55bb8066dc50 2024-09-13T18:24:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.339+0000 7f7cf37fe640 1 --2- 172.21.15.114:0/3814489714 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bb806784f0 0x55bb8067e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:24:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.340+0000 7f7cf27fc640 1 --2- 172.21.15.114:0/3814489714 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb8066dc50 0x55bb8067e190 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7ce80b7980 tx=0x7f7ce80bada0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74960ee9fd03cd79 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.340+0000 7f7cd6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:24:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.340+0000 7f7cd6ff5640 1 -- 172.21.15.114:0/3814489714 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce80baf10 con 0x55bb8066dc50 2024-09-13T18:24:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.340+0000 7f7cd6ff5640 1 -- 172.21.15.114:0/3814489714 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f7ce80bec90 con 0x55bb8066dc50 2024-09-13T18:24:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.340+0000 7f7cfc0ca740 1 -- 172.21.15.114:0/3814489714 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7ce40035f0 con 0x55bb8066dc50 2024-09-13T18:24:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.340+0000 7f7cd6ff5640 1 -- 172.21.15.114:0/3814489714 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ce80c32e0 con 0x55bb8066dc50 2024-09-13T18:24:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.340+0000 7f7cfc0ca740 10 client.25733 fetch_fsmap learned FSMap version 32 2024-09-13T18:24:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.340+0000 7f7cfc0ca740 1 -- 172.21.15.114:0/3814489714 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55bb805d0330 con 0x55bb8066dc50 2024-09-13T18:24:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.340+0000 7f7cd6ff5640 1 -- 172.21.15.114:0/3814489714 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7ce80d2020 con 0x55bb8066dc50 2024-09-13T18:24:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.340+0000 7f7cd6ff5640 1 -- 172.21.15.114:0/3814489714 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f7ce804d070 con 0x55bb8066dc50 2024-09-13T18:24:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.341+0000 7f7cfc0ca740 10 client.25733 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:24:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.341+0000 7f7cfc0ca740 10 client.25733 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:24:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.341+0000 7f7cfc0ca740 20 client.25733 populate_metadata read hostname 'smithi114' 2024-09-13T18:24:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.341+0000 7f7cfc0ca740 1 --2- 172.21.15.114:0/3814489714 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55bb806df9c0 0x55bb806e1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.341+0000 7f7cfc0ca740 4 client.25733 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T18:24:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.341+0000 7f7cfc0ca740 1 -- 172.21.15.114:0/3814489714 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7f7ce40035f0 con 0x55bb806df9c0 2024-09-13T18:24:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.341+0000 7f7cf2ffd640 1 --2- 172.21.15.114:0/3814489714 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55bb806df9c0 0x55bb806e1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.341+0000 7f7cd6ff5640 10 client.25733 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:24:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.341+0000 7f7cf2ffd640 1 --2- 172.21.15.114:0/3814489714 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55bb806df9c0 0x55bb806e1de0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ee00a6f921bbc4b server_cookie=f0a252c99f4cd3b6 in_seq=0 out_seq=0 2024-09-13T18:24:23.356 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.353+0000 7f7cd6ff5640 1 -- 172.21.15.114:0/3814489714 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f7ce40035f0 con 0x55bb806df9c0 2024-09-13T18:24:23.356 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.353+0000 7f7cd6ff5640 10 client.25733 handle_command_reply: tid=0 2024-09-13T18:24:23.356 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.353+0000 7f7cfc0ca740 1 client.25733 shutdown 2024-09-13T18:24:23.356 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.353+0000 7f7cfc0ca740 1 -- 172.21.15.114:0/3814489714 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb8066dc50 msgr2=0x55bb8067e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:23.356 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.353+0000 7f7cfc0ca740 1 --2- 172.21.15.114:0/3814489714 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb8066dc50 0x55bb8067e190 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7ce80b7980 tx=0x7f7ce80bada0 comp rx=0 tx=0).stop 2024-09-13T18:24:23.356 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.353+0000 7f7cfc0ca740 1 -- 172.21.15.114:0/3814489714 shutdown_connections 2024-09-13T18:24:23.356 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.353+0000 7f7cfc0ca740 1 --2- 172.21.15.114:0/3814489714 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55bb806df9c0 0x55bb806e1de0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:23.356 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.353+0000 7f7cfc0ca740 1 --2- 172.21.15.114:0/3814489714 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bb806784f0 0x55bb8067e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:23.356 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.353+0000 7f7cd6ff5640 0 client.25733 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:24:23.356 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.353+0000 7f7cfc0ca740 1 --2- 172.21.15.114:0/3814489714 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb8066dc50 0x55bb8067e190 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:23.356 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.353+0000 7f7cfc0ca740 1 --2- 172.21.15.114:0/3814489714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb8066d330 0x55bb8067dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:23.357 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.353+0000 7f7cfc0ca740 1 -- 172.21.15.114:0/3814489714 >> 172.21.15.114:0/3814489714 conn(0x55bb806698c0 msgr2=0x55bb80674870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:23.357 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.353+0000 7f7cfc0ca740 1 -- 172.21.15.114:0/3814489714 shutdown_connections 2024-09-13T18:24:23.357 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.353+0000 7f7cfc0ca740 1 -- 172.21.15.114:0/3814489714 wait complete. 2024-09-13T18:24:23.357 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.353+0000 7f7cfc0ca740 20 client.25733 trim_cache size 0 max 16384 2024-09-13T18:24:23.357 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-13T18:24:23.357 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-13T18:24:23.358 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-13T18:24:23.358 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-13T18:24:23.358 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.354+0000 7f7cf93aa640 1 -- 172.21.15.114:0/2030029688 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7cdc0a1280 msgr2=0x7f7cdc0a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:23.358 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.354+0000 7f7cf93aa640 1 --2- 172.21.15.114:0/2030029688 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7cdc0a1280 0x7f7cdc0a3740 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7cec007fd0 tx=0x7f7cec007e40 comp rx=0 tx=0).stop 2024-09-13T18:24:23.359 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.354+0000 7f7cf93aa640 1 -- 172.21.15.114:0/2030029688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7cf40ff970 msgr2=0x7f7cf410c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:23.359 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.354+0000 7f7cf93aa640 1 --2- 172.21.15.114:0/2030029688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7cf40ff970 0x7f7cf410c810 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7ce802a910 tx=0x7f7ce8005610 comp rx=0 tx=0).stop 2024-09-13T18:24:23.359 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.354+0000 7f7cf93aa640 1 -- 172.21.15.114:0/2030029688 shutdown_connections 2024-09-13T18:24:23.359 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.354+0000 7f7cf93aa640 1 --2- 172.21.15.114:0/2030029688 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7cdc0a1280 0x7f7cdc0a3740 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:23.360 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.354+0000 7f7cf93aa640 1 --2- 172.21.15.114:0/2030029688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7cf4104c30 0x7f7cf410cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:23.360 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.354+0000 7f7cf93aa640 1 --2- 172.21.15.114:0/2030029688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7cf40ff970 0x7f7cf410c810 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:23.360 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.354+0000 7f7cf93aa640 1 --2- 172.21.15.114:0/2030029688 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cf40feff0 0x7f7cf410c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:23.360 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.354+0000 7f7cf93aa640 1 -- 172.21.15.114:0/2030029688 >> 172.21.15.114:0/2030029688 conn(0x7f7cf4074940 msgr2=0x7f7cf40fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:23.360 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.355+0000 7f7cf93aa640 1 -- 172.21.15.114:0/2030029688 shutdown_connections 2024-09-13T18:24:23.360 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:23.355+0000 7f7cf93aa640 1 -- 172.21.15.114:0/2030029688 wait complete. 2024-09-13T18:24:24.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:24 smithi204 ceph-mon[34777]: pgmap v61: 129 pgs: 129 active+clean; 7.5 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 224 MiB/s wr, 134 op/s 2024-09-13T18:24:24.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:24 smithi114 ceph-mon[30897]: pgmap v61: 129 pgs: 129 active+clean; 7.5 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 224 MiB/s wr, 134 op/s 2024-09-13T18:24:24.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:24 smithi134 ceph-mon[34283]: pgmap v61: 129 pgs: 129 active+clean; 7.5 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 224 MiB/s wr, 134 op/s 2024-09-13T18:24:26.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:26 smithi204 ceph-mon[34777]: pgmap v62: 129 pgs: 129 active+clean; 7.5 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 152 MiB/s wr, 91 op/s 2024-09-13T18:24:26.469 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:26 smithi114 ceph-mon[30897]: pgmap v62: 129 pgs: 129 active+clean; 7.5 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 152 MiB/s wr, 91 op/s 2024-09-13T18:24:26.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:26 smithi134 ceph-mon[34283]: pgmap v62: 129 pgs: 129 active+clean; 7.5 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 152 MiB/s wr, 91 op/s 2024-09-13T18:24:28.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:28 smithi204 ceph-mon[34777]: pgmap v63: 129 pgs: 129 active+clean; 7.5 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 152 MiB/s wr, 91 op/s 2024-09-13T18:24:28.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:28 smithi114 ceph-mon[30897]: pgmap v63: 129 pgs: 129 active+clean; 7.5 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 152 MiB/s wr, 91 op/s 2024-09-13T18:24:28.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:28 smithi134 ceph-mon[34283]: pgmap v63: 129 pgs: 129 active+clean; 7.5 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 152 MiB/s wr, 91 op/s 2024-09-13T18:24:30.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:30 smithi204 ceph-mon[34777]: pgmap v64: 129 pgs: 129 active+clean; 8.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 228 MiB/s wr, 133 op/s 2024-09-13T18:24:30.469 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:30 smithi114 ceph-mon[30897]: pgmap v64: 129 pgs: 129 active+clean; 8.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 228 MiB/s wr, 133 op/s 2024-09-13T18:24:30.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:30 smithi134 ceph-mon[34283]: pgmap v64: 129 pgs: 129 active+clean; 8.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 228 MiB/s wr, 133 op/s 2024-09-13T18:24:32.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:32 smithi204 ceph-mon[34777]: pgmap v65: 129 pgs: 129 active+clean; 8.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 152 MiB/s wr, 88 op/s 2024-09-13T18:24:32.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:32 smithi114 ceph-mon[30897]: pgmap v65: 129 pgs: 129 active+clean; 8.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 152 MiB/s wr, 88 op/s 2024-09-13T18:24:32.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:32 smithi134 ceph-mon[34283]: pgmap v65: 129 pgs: 129 active+clean; 8.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 152 MiB/s wr, 88 op/s 2024-09-13T18:24:34.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:34 smithi204 ceph-mon[34777]: pgmap v66: 129 pgs: 129 active+clean; 9.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 227 MiB/s wr, 130 op/s 2024-09-13T18:24:34.469 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:34 smithi114 ceph-mon[30897]: pgmap v66: 129 pgs: 129 active+clean; 9.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 227 MiB/s wr, 130 op/s 2024-09-13T18:24:34.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:34 smithi134 ceph-mon[34283]: pgmap v66: 129 pgs: 129 active+clean; 9.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 227 MiB/s wr, 130 op/s 2024-09-13T18:24:36.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:36 smithi204 ceph-mon[34777]: pgmap v67: 129 pgs: 129 active+clean; 9.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 151 MiB/s wr, 84 op/s 2024-09-13T18:24:36.469 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:36 smithi114 ceph-mon[30897]: pgmap v67: 129 pgs: 129 active+clean; 9.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 151 MiB/s wr, 84 op/s 2024-09-13T18:24:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.485+0000 7f3a7cff4640 1 -- 172.21.15.114:0/2060172165 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a780fea10 msgr2=0x7f3a780fedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.485+0000 7f3a7cff4640 1 --2- 172.21.15.114:0/2060172165 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a780fea10 0x7f3a780fedf0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f3a68009930 tx=0x7f3a6802a420 comp rx=0 tx=0).stop 2024-09-13T18:24:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.485+0000 7f3a7cff4640 1 -- 172.21.15.114:0/2060172165 shutdown_connections 2024-09-13T18:24:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.485+0000 7f3a7cff4640 1 --2- 172.21.15.114:0/2060172165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a781062a0 0x7f3a7810cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.485+0000 7f3a7cff4640 1 --2- 172.21.15.114:0/2060172165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a780ff3c0 0x7f3a78105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.485+0000 7f3a7cff4640 1 --2- 172.21.15.114:0/2060172165 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a780fea10 0x7f3a780fedf0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.485+0000 7f3a7cff4640 1 -- 172.21.15.114:0/2060172165 >> 172.21.15.114:0/2060172165 conn(0x7f3a78075340 msgr2=0x7f3a78075750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.485+0000 7f3a7cff4640 1 -- 172.21.15.114:0/2060172165 shutdown_connections 2024-09-13T18:24:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.485+0000 7f3a7cff4640 1 -- 172.21.15.114:0/2060172165 wait complete. 2024-09-13T18:24:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.486+0000 7f3a7cff4640 1 Processor -- start 2024-09-13T18:24:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.486+0000 7f3a7cff4640 1 -- start start 2024-09-13T18:24:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.486+0000 7f3a7cff4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a780fea10 0x7f3a78196160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.486+0000 7f3a7cff4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a781062a0 0x7f3a781966a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.486+0000 7f3a7cff4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a7819aa90 0x7f3a7819af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.487+0000 7f3a7cff4640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7819b480 con 0x7f3a780fea10 2024-09-13T18:24:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.487+0000 7f3a7cff4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7819b5f0 con 0x7f3a781062a0 2024-09-13T18:24:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.487+0000 7f3a7cff4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7819b760 con 0x7f3a7819aa90 2024-09-13T18:24:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.487+0000 7f3a7659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a780fea10 0x7f3a78196160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.487+0000 7f3a75d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a781062a0 0x7f3a781966a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.487+0000 7f3a7659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a780fea10 0x7f3a78196160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48748/0 (socket says 172.21.15.114:48748) 2024-09-13T18:24:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.487+0000 7f3a7659c640 1 -- 172.21.15.114:0/2577851432 learned_addr learned my addr 172.21.15.114:0/2577851432 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.487+0000 7f3a76d9d640 1 --2- 172.21.15.114:0/2577851432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a7819aa90 0x7f3a7819af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.487+0000 7f3a75d9b640 1 -- 172.21.15.114:0/2577851432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a7819aa90 msgr2=0x7f3a7819af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:36.492 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.487+0000 7f3a75d9b640 1 --2- 172.21.15.114:0/2577851432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a7819aa90 0x7f3a7819af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:36.492 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.487+0000 7f3a75d9b640 1 -- 172.21.15.114:0/2577851432 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a780fea10 msgr2=0x7f3a78196160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:36.492 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.487+0000 7f3a75d9b640 1 --2- 172.21.15.114:0/2577851432 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a780fea10 0x7f3a78196160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:36.492 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.487+0000 7f3a75d9b640 1 -- 172.21.15.114:0/2577851432 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a68009560 con 0x7f3a781062a0 2024-09-13T18:24:36.492 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.487+0000 7f3a76d9d640 1 --2- 172.21.15.114:0/2577851432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a7819aa90 0x7f3a7819af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:24:36.492 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.487+0000 7f3a7659c640 1 --2- 172.21.15.114:0/2577851432 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a780fea10 0x7f3a78196160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:24:36.492 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.487+0000 7f3a75d9b640 1 --2- 172.21.15.114:0/2577851432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a781062a0 0x7f3a781966a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3a78076560 tx=0x7f3a6c00ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:36.492 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.488+0000 7f3a677fe640 1 -- 172.21.15.114:0/2577851432 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c00cc40 con 0x7f3a781062a0 2024-09-13T18:24:36.492 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.488+0000 7f3a7cff4640 1 -- 172.21.15.114:0/2577851432 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a780ff8b0 con 0x7f3a781062a0 2024-09-13T18:24:36.492 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.488+0000 7f3a677fe640 1 -- 172.21.15.114:0/2577851432 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3a6c00cda0 con 0x7f3a781062a0 2024-09-13T18:24:36.492 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.488+0000 7f3a677fe640 1 -- 172.21.15.114:0/2577851432 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c018680 con 0x7f3a781062a0 2024-09-13T18:24:36.492 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.488+0000 7f3a7cff4640 1 -- 172.21.15.114:0/2577851432 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a780ffd60 con 0x7f3a781062a0 2024-09-13T18:24:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.491+0000 7f3a677fe640 1 -- 172.21.15.114:0/2577851432 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f3a6c010430 con 0x7f3a781062a0 2024-09-13T18:24:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.491+0000 7f3a7fdf1740 1 Processor -- start 2024-09-13T18:24:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.491+0000 7f3a7fdf1740 1 -- start start 2024-09-13T18:24:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.492+0000 7f3a7fdf1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b53547330 0x561b53547710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.492+0000 7f3a7fdf1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b53547c50 0x561b53551fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.492+0000 7f3a7fdf1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b535524f0 0x561b535548e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.492+0000 7f3a7fdf1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b534b2c30 con 0x561b53547330 2024-09-13T18:24:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.492+0000 7f3a7fdf1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b534b03c0 con 0x561b535524f0 2024-09-13T18:24:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.492+0000 7f3a7fdf1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b534abcb0 con 0x561b53547c50 2024-09-13T18:24:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.492+0000 7f3a76d9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b53547330 0x561b53547710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.492+0000 7f3a76d9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b53547330 0x561b53547710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48764/0 (socket says 172.21.15.114:48764) 2024-09-13T18:24:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.492+0000 7f3a76d9d640 1 -- 172.21.15.114:0/1368348905 learned_addr learned my addr 172.21.15.114:0/1368348905 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.492+0000 7f3a7659c640 1 --2- 172.21.15.114:0/1368348905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b53547c50 0x561b53551fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.492+0000 7f3a75d9b640 1 --2- 172.21.15.114:0/1368348905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b535524f0 0x561b535548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.492+0000 7f3a677fe640 1 --2- 172.21.15.114:0/2577851432 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f3a600a1200 0x7f3a600a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.492+0000 7f3a7659c640 1 -- 172.21.15.114:0/1368348905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b535524f0 msgr2=0x561b535548e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.492+0000 7f3a7659c640 1 --2- 172.21.15.114:0/1368348905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b535524f0 0x561b535548e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.492+0000 7f3a7659c640 1 -- 172.21.15.114:0/1368348905 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b53547330 msgr2=0x561b53547710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.492+0000 7f3a7659c640 1 --2- 172.21.15.114:0/1368348905 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b53547330 0x561b53547710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.492+0000 7f3a7659c640 1 -- 172.21.15.114:0/1368348905 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b53452910 con 0x561b53547c50 2024-09-13T18:24:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.493+0000 7f3a76d9d640 1 --2- 172.21.15.114:0/1368348905 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b53547330 0x561b53547710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:24:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.493+0000 7f3a677fe640 1 -- 172.21.15.114:0/2577851432 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f3a6c014070 con 0x7f3a781062a0 2024-09-13T18:24:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.493+0000 7f3a76d9d640 1 --2- 172.21.15.114:0/2577851432 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f3a600a1200 0x7f3a600a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.493+0000 7f3a7659c640 1 --2- 172.21.15.114:0/1368348905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b53547c50 0x561b53551fb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3a68009900 tx=0x7f3a68004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd2f9f278e394f68 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.493+0000 7f3a64ff9640 1 -- 172.21.15.114:0/1368348905 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a680044f0 con 0x561b53547c50 2024-09-13T18:24:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.493+0000 7f3a76d9d640 1 --2- 172.21.15.114:0/2577851432 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f3a600a1200 0x7f3a600a36c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3a700092f0 tx=0x7f3a70004a40 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.493+0000 7f3a64ff9640 1 -- 172.21.15.114:0/1368348905 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3a68031df0 con 0x561b53547c50 2024-09-13T18:24:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.493+0000 7f3a64ff9640 1 -- 172.21.15.114:0/1368348905 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6803d450 con 0x561b53547c50 2024-09-13T18:24:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.494+0000 7f3a7fdf1740 1 -- 172.21.15.114:0/1368348905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b53547c50 msgr2=0x561b53551fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.494+0000 7f3a7fdf1740 1 --2- 172.21.15.114:0/1368348905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b53547c50 0x561b53551fb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3a68009900 tx=0x7f3a68004040 comp rx=0 tx=0).stop 2024-09-13T18:24:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.494+0000 7f3a7fdf1740 1 -- 172.21.15.114:0/1368348905 shutdown_connections 2024-09-13T18:24:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.494+0000 7f3a7fdf1740 1 --2- 172.21.15.114:0/1368348905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b535524f0 0x561b535548e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.494+0000 7f3a7fdf1740 1 --2- 172.21.15.114:0/1368348905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b53547c50 0x561b53551fb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.494+0000 7f3a7fdf1740 1 --2- 172.21.15.114:0/1368348905 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b53547330 0x561b53547710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.494+0000 7f3a7fdf1740 1 -- 172.21.15.114:0/1368348905 >> 172.21.15.114:0/1368348905 conn(0x561b535438c0 msgr2=0x561b53543ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.494+0000 7f3a7fdf1740 1 -- 172.21.15.114:0/1368348905 shutdown_connections 2024-09-13T18:24:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.494+0000 7f3a7fdf1740 1 -- 172.21.15.114:0/1368348905 wait complete. 2024-09-13T18:24:36.501 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.494+0000 7f3a7fdf1740 1 -- start start 2024-09-13T18:24:36.501 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.495+0000 7f3a7fdf1740 1 Processor -- start 2024-09-13T18:24:36.501 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.495+0000 7f3a7fdf1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b53547330 0x561b53551e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:36.501 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.495+0000 7f3a7fdf1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b53547c50 0x561b5354f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:36.502 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.495+0000 7f3a7fdf1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b535524f0 0x561b5354f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:36.502 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.495+0000 7f3a7fdf1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b53469e80 con 0x561b535524f0 2024-09-13T18:24:36.502 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.495+0000 7f3a7fdf1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b53469b20 con 0x561b53547330 2024-09-13T18:24:36.502 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.495+0000 7f3a7fdf1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b5342f260 con 0x561b53547c50 2024-09-13T18:24:36.502 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.495+0000 7f3a7fdf1740 10 client.0 _ll_register_callbacks cb 0x561b53464df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:24:36.502 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.495+0000 7f3a76d9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b535524f0 0x561b5354f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:36.502 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.495+0000 7f3a76d9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b535524f0 0x561b5354f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48772/0 (socket says 172.21.15.114:48772) 2024-09-13T18:24:36.502 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.495+0000 7f3a76d9d640 1 -- 172.21.15.114:0/3707706258 learned_addr learned my addr 172.21.15.114:0/3707706258 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:36.502 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.496+0000 7f3a7659c640 1 --2- 172.21.15.114:0/3707706258 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b53547330 0x561b53551e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:36.502 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.496+0000 7f3a75d9b640 1 --2- 172.21.15.114:0/3707706258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b53547c50 0x561b5354f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:36.502 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.496+0000 7f3a76d9d640 1 -- 172.21.15.114:0/3707706258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b53547c50 msgr2=0x561b5354f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:36.502 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.496+0000 7f3a76d9d640 1 --2- 172.21.15.114:0/3707706258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b53547c50 0x561b5354f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:36.502 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.496+0000 7f3a76d9d640 1 -- 172.21.15.114:0/3707706258 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b53547330 msgr2=0x561b53551e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:36.503 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.496+0000 7f3a76d9d640 1 --2- 172.21.15.114:0/3707706258 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b53547330 0x561b53551e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:36.503 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.496+0000 7f3a76d9d640 1 -- 172.21.15.114:0/3707706258 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a7810d430 con 0x561b535524f0 2024-09-13T18:24:36.503 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.496+0000 7f3a76d9d640 1 --2- 172.21.15.114:0/3707706258 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b535524f0 0x561b5354f680 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f3a7000eef0 tx=0x7f3a70004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43df9e570cab965e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:36.503 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.496+0000 7f3a5dffb640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:24:36.503 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.496+0000 7f3a5dffb640 1 -- 172.21.15.114:0/3707706258 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7001fdb0 con 0x561b535524f0 2024-09-13T18:24:36.503 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.496+0000 7f3a5dffb640 1 -- 172.21.15.114:0/3707706258 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f3a70013070 con 0x561b535524f0 2024-09-13T18:24:36.503 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.496+0000 7f3a7fdf1740 1 -- 172.21.15.114:0/3707706258 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561b534d4680 con 0x561b535524f0 2024-09-13T18:24:36.503 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.497+0000 7f3a5dffb640 1 -- 172.21.15.114:0/3707706258 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a70013230 con 0x561b535524f0 2024-09-13T18:24:36.503 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.497+0000 7f3a7fdf1740 10 client.16425 fetch_fsmap learned FSMap version 32 2024-09-13T18:24:36.503 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.497+0000 7f3a7fdf1740 1 -- 172.21.15.114:0/3707706258 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x561b534aa330 con 0x561b535524f0 2024-09-13T18:24:36.503 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.497+0000 7f3a5dffb640 1 -- 172.21.15.114:0/3707706258 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561b534d4680 con 0x561b535524f0 2024-09-13T18:24:36.503 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.497+0000 7f3a5dffb640 1 -- 172.21.15.114:0/3707706258 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f3a70028c80 con 0x561b535524f0 2024-09-13T18:24:36.503 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.497+0000 7f3a7fdf1740 10 client.16425 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:24:36.504 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.497+0000 7f3a7fdf1740 10 client.16425 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:24:36.504 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.497+0000 7f3a7fdf1740 20 client.16425 populate_metadata read hostname 'smithi114' 2024-09-13T18:24:36.504 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.497+0000 7f3a7fdf1740 1 --2- 172.21.15.114:0/3707706258 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x561b5356a810 0x561b5356cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:36.504 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.497+0000 7f3a7fdf1740 4 client.16425 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:24:36.504 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.497+0000 7f3a7fdf1740 1 -- 172.21.15.114:0/3707706258 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561b5349f5d0 con 0x561b5356a810 2024-09-13T18:24:36.504 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.497+0000 7f3a7659c640 1 --2- 172.21.15.114:0/3707706258 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x561b5356a810 0x561b5356cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:36.504 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.498+0000 7f3a5dffb640 10 client.16425 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:24:36.504 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.498+0000 7f3a7659c640 1 --2- 172.21.15.114:0/3707706258 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x561b5356a810 0x561b5356cc30 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f52606e8528ac64e server_cookie=51075095aa5b256d in_seq=0 out_seq=0 2024-09-13T18:24:36.505 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.500+0000 7f3a5dffb640 1 -- 172.21.15.114:0/3707706258 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x561b5349f5d0 con 0x561b5356a810 2024-09-13T18:24:36.505 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.500+0000 7f3a5dffb640 10 client.16425 handle_command_reply: tid=0 2024-09-13T18:24:36.505 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.501+0000 7f3a7fdf1740 1 client.16425 shutdown 2024-09-13T18:24:36.505 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.501+0000 7f3a7fdf1740 1 -- 172.21.15.114:0/3707706258 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b535524f0 msgr2=0x561b5354f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:36.505 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.501+0000 7f3a7fdf1740 1 --2- 172.21.15.114:0/3707706258 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b535524f0 0x561b5354f680 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f3a7000eef0 tx=0x7f3a70004ab0 comp rx=0 tx=0).stop 2024-09-13T18:24:36.505 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.501+0000 7f3a7fdf1740 1 -- 172.21.15.114:0/3707706258 shutdown_connections 2024-09-13T18:24:36.505 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.501+0000 7f3a7fdf1740 1 --2- 172.21.15.114:0/3707706258 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x561b5356a810 0x561b5356cc30 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:36.506 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.501+0000 7f3a7fdf1740 1 --2- 172.21.15.114:0/3707706258 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b535524f0 0x561b5354f680 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:36.506 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.501+0000 7f3a7fdf1740 1 --2- 172.21.15.114:0/3707706258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b53547c50 0x561b5354f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:36.506 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.501+0000 7f3a7fdf1740 1 --2- 172.21.15.114:0/3707706258 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b53547330 0x561b53551e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:36.506 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.501+0000 7f3a7fdf1740 1 -- 172.21.15.114:0/3707706258 >> 172.21.15.114:0/3707706258 conn(0x561b535438c0 msgr2=0x561b53542b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:36.506 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.502+0000 7f3a5dffb640 0 client.16425 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:24:36.506 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.502+0000 7f3a7fdf1740 1 -- 172.21.15.114:0/3707706258 shutdown_connections 2024-09-13T18:24:36.506 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.502+0000 7f3a7fdf1740 1 -- 172.21.15.114:0/3707706258 wait complete. 2024-09-13T18:24:36.506 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.502+0000 7f3a7fdf1740 20 client.16425 trim_cache size 0 max 16384 2024-09-13T18:24:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.522+0000 7f3a7fdf1740 1 Processor -- start 2024-09-13T18:24:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.522+0000 7f3a7fdf1740 1 -- start start 2024-09-13T18:24:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.522+0000 7f3a7fdf1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b53547330 0x561b53551960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.523+0000 7f3a7fdf1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b53547c50 0x561b53561ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.523+0000 7f3a7fdf1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b535524f0 0x561b53562020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.523+0000 7f3a7fdf1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b535488b0 con 0x561b53547c50 2024-09-13T18:24:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.523+0000 7f3a7fdf1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b53568f90 con 0x561b53547330 2024-09-13T18:24:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.523+0000 7f3a7fdf1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b5354d680 con 0x561b535524f0 2024-09-13T18:24:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.523+0000 7f3a75d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b53547c50 0x561b53561ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.523+0000 7f3a7659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b53547330 0x561b53551960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.523+0000 7f3a7659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b53547330 0x561b53551960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:33384/0 (socket says 172.21.15.114:33384) 2024-09-13T18:24:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.523+0000 7f3a75d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b53547c50 0x561b53561ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48778/0 (socket says 172.21.15.114:48778) 2024-09-13T18:24:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.523+0000 7f3a7659c640 1 -- 172.21.15.114:0/888661007 learned_addr learned my addr 172.21.15.114:0/888661007 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.523+0000 7f3a76d9d640 1 --2- 172.21.15.114:0/888661007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b535524f0 0x561b53562020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.523+0000 7f3a7659c640 1 -- 172.21.15.114:0/888661007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b535524f0 msgr2=0x561b53562020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.523+0000 7f3a7659c640 1 --2- 172.21.15.114:0/888661007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b535524f0 0x561b53562020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.523+0000 7f3a7659c640 1 -- 172.21.15.114:0/888661007 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b53547c50 msgr2=0x561b53561ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.523+0000 7f3a7659c640 1 --2- 172.21.15.114:0/888661007 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b53547c50 0x561b53561ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.523+0000 7f3a7659c640 1 -- 172.21.15.114:0/888661007 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b5344c300 con 0x561b53547330 2024-09-13T18:24:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.523+0000 7f3a75d9b640 1 --2- 172.21.15.114:0/888661007 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b53547c50 0x561b53561ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:24:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.523+0000 7f3a7659c640 1 --2- 172.21.15.114:0/888661007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b53547330 0x561b53551960 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3a6802aa10 tx=0x7f3a68005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df4b73f1a827cf7d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.523+0000 7f3a657fa640 1 -- 172.21.15.114:0/888661007 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a680042f0 con 0x561b53547330 2024-09-13T18:24:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.524+0000 7f3a657fa640 1 -- 172.21.15.114:0/888661007 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3a6804d040 con 0x561b53547330 2024-09-13T18:24:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.524+0000 7f3a657fa640 1 -- 172.21.15.114:0/888661007 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a68003850 con 0x561b53547330 2024-09-13T18:24:36.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.524+0000 7f3a7fdf1740 1 -- 172.21.15.114:0/888661007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b53547330 msgr2=0x561b53551960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:36.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.524+0000 7f3a7fdf1740 1 --2- 172.21.15.114:0/888661007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b53547330 0x561b53551960 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3a6802aa10 tx=0x7f3a68005fd0 comp rx=0 tx=0).stop 2024-09-13T18:24:36.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.524+0000 7f3a7fdf1740 1 -- 172.21.15.114:0/888661007 shutdown_connections 2024-09-13T18:24:36.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.524+0000 7f3a7fdf1740 1 --2- 172.21.15.114:0/888661007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b535524f0 0x561b53562020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:36.532 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.524+0000 7f3a7fdf1740 1 --2- 172.21.15.114:0/888661007 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b53547c50 0x561b53561ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:36.532 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.524+0000 7f3a7fdf1740 1 --2- 172.21.15.114:0/888661007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b53547330 0x561b53551960 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:36.532 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.524+0000 7f3a7fdf1740 1 -- 172.21.15.114:0/888661007 >> 172.21.15.114:0/888661007 conn(0x561b535438c0 msgr2=0x561b5354d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:36.532 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.524+0000 7f3a7fdf1740 1 -- 172.21.15.114:0/888661007 shutdown_connections 2024-09-13T18:24:36.532 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.525+0000 7f3a7fdf1740 1 -- 172.21.15.114:0/888661007 wait complete. 2024-09-13T18:24:36.532 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.525+0000 7f3a7fdf1740 1 -- start start 2024-09-13T18:24:36.532 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.525+0000 7f3a7fdf1740 1 Processor -- start 2024-09-13T18:24:36.532 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.525+0000 7f3a7fdf1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b53547330 0x561b53561d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:36.532 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.525+0000 7f3a7fdf1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b53547c50 0x561b5355e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:36.532 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.525+0000 7f3a7fdf1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b535524f0 0x561b5355e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:36.532 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.525+0000 7f3a7fdf1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b5355edc0 con 0x561b535524f0 2024-09-13T18:24:36.532 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.525+0000 7f3a7fdf1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b5355ef30 con 0x561b53547330 2024-09-13T18:24:36.533 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.525+0000 7f3a7fdf1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b535c0d50 con 0x561b53547c50 2024-09-13T18:24:36.533 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.525+0000 7f3a7fdf1740 10 client.0 _ll_register_callbacks cb 0x7f3a68009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:24:36.533 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.525+0000 7f3a76d9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b535524f0 0x561b5355e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:36.533 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.526+0000 7f3a76d9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b535524f0 0x561b5355e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48784/0 (socket says 172.21.15.114:48784) 2024-09-13T18:24:36.533 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.526+0000 7f3a76d9d640 1 -- 172.21.15.114:0/67581721 learned_addr learned my addr 172.21.15.114:0/67581721 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:36.533 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.526+0000 7f3a7659c640 1 --2- 172.21.15.114:0/67581721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b53547330 0x561b53561d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:36.533 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.526+0000 7f3a75d9b640 1 --2- 172.21.15.114:0/67581721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b53547c50 0x561b5355e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:36.533 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.526+0000 7f3a76d9d640 1 -- 172.21.15.114:0/67581721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b53547c50 msgr2=0x561b5355e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:36.533 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.526+0000 7f3a76d9d640 1 --2- 172.21.15.114:0/67581721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b53547c50 0x561b5355e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:36.533 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.526+0000 7f3a76d9d640 1 -- 172.21.15.114:0/67581721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b53547330 msgr2=0x561b53561d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:36.533 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.526+0000 7f3a76d9d640 1 --2- 172.21.15.114:0/67581721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b53547330 0x561b53561d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:36.533 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.526+0000 7f3a76d9d640 1 -- 172.21.15.114:0/67581721 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a780ffd60 con 0x561b535524f0 2024-09-13T18:24:36.533 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.526+0000 7f3a75d9b640 1 --2- 172.21.15.114:0/67581721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b53547c50 0x561b5355e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:24:36.533 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.526+0000 7f3a76d9d640 1 --2- 172.21.15.114:0/67581721 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b535524f0 0x561b5355e710 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f3a70012440 tx=0x7f3a70004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f42efa3e1fa5dc87 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:36.534 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.526+0000 7f3a5dffb640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:24:36.534 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.526+0000 7f3a5dffb640 1 -- 172.21.15.114:0/67581721 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a70013450 con 0x561b535524f0 2024-09-13T18:24:36.534 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.526+0000 7f3a5dffb640 1 -- 172.21.15.114:0/67581721 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f3a700135b0 con 0x561b535524f0 2024-09-13T18:24:36.534 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.526+0000 7f3a7fdf1740 1 -- 172.21.15.114:0/67581721 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3a68031df0 con 0x561b535524f0 2024-09-13T18:24:36.534 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.527+0000 7f3a5dffb640 1 -- 172.21.15.114:0/67581721 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a7001f7f0 con 0x561b535524f0 2024-09-13T18:24:36.534 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.527+0000 7f3a7fdf1740 10 client.16431 fetch_fsmap learned FSMap version 32 2024-09-13T18:24:36.534 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.527+0000 7f3a7fdf1740 1 -- 172.21.15.114:0/67581721 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x561b53452910 con 0x561b535524f0 2024-09-13T18:24:36.534 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.527+0000 7f3a5dffb640 1 -- 172.21.15.114:0/67581721 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3a68031df0 con 0x561b535524f0 2024-09-13T18:24:36.534 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.527+0000 7f3a5dffb640 1 -- 172.21.15.114:0/67581721 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f3a7003d020 con 0x561b535524f0 2024-09-13T18:24:36.534 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.527+0000 7f3a7fdf1740 10 client.16431 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:24:36.534 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.527+0000 7f3a7fdf1740 10 client.16431 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:24:36.534 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.527+0000 7f3a7fdf1740 20 client.16431 populate_metadata read hostname 'smithi114' 2024-09-13T18:24:36.535 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.527+0000 7f3a7fdf1740 1 --2- 172.21.15.114:0/67581721 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x561b535c2370 0x561b535c4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:36.535 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.528+0000 7f3a7fdf1740 4 client.16431 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T18:24:36.535 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.528+0000 7f3a7fdf1740 1 -- 172.21.15.114:0/67581721 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561b5319f300 con 0x561b535c2370 2024-09-13T18:24:36.535 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.528+0000 7f3a7659c640 1 --2- 172.21.15.114:0/67581721 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x561b535c2370 0x561b535c4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:36.535 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.529+0000 7f3a7659c640 1 --2- 172.21.15.114:0/67581721 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x561b535c2370 0x561b535c4790 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60616586331c95d7 server_cookie=42df5cbba1060be4 in_seq=0 out_seq=0 2024-09-13T18:24:36.535 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.529+0000 7f3a5dffb640 10 client.16431 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:24:36.538 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.536+0000 7f3a5dffb640 1 -- 172.21.15.114:0/67581721 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561b5319f300 con 0x561b535c2370 2024-09-13T18:24:36.538 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.536+0000 7f3a5dffb640 10 client.16431 handle_command_reply: tid=0 2024-09-13T18:24:36.539 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.536+0000 7f3a7fdf1740 1 client.16431 shutdown 2024-09-13T18:24:36.539 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.536+0000 7f3a7fdf1740 1 -- 172.21.15.114:0/67581721 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b535524f0 msgr2=0x561b5355e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:36.539 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.536+0000 7f3a7fdf1740 1 --2- 172.21.15.114:0/67581721 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b535524f0 0x561b5355e710 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f3a70012440 tx=0x7f3a70004db0 comp rx=0 tx=0).stop 2024-09-13T18:24:36.539 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.536+0000 7f3a7fdf1740 1 -- 172.21.15.114:0/67581721 shutdown_connections 2024-09-13T18:24:36.539 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.536+0000 7f3a7fdf1740 1 --2- 172.21.15.114:0/67581721 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x561b535c2370 0x561b535c4790 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:36.539 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.536+0000 7f3a5dffb640 0 client.16431 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:24:36.539 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.536+0000 7f3a7fdf1740 1 --2- 172.21.15.114:0/67581721 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b535524f0 0x561b5355e710 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:36.539 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.536+0000 7f3a7fdf1740 1 --2- 172.21.15.114:0/67581721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b53547c50 0x561b5355e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:36.539 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.536+0000 7f3a7fdf1740 1 --2- 172.21.15.114:0/67581721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b53547330 0x561b53561d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:36.539 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.536+0000 7f3a7fdf1740 1 -- 172.21.15.114:0/67581721 >> 172.21.15.114:0/67581721 conn(0x561b535438c0 msgr2=0x561b5353ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:36.539 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.536+0000 7f3a7fdf1740 1 -- 172.21.15.114:0/67581721 shutdown_connections 2024-09-13T18:24:36.539 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.536+0000 7f3a7fdf1740 1 -- 172.21.15.114:0/67581721 wait complete. 2024-09-13T18:24:36.539 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.536+0000 7f3a7fdf1740 20 client.16431 trim_cache size 0 max 16384 2024-09-13T18:24:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-13T18:24:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-13T18:24:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-13T18:24:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-13T18:24:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.537+0000 7f3a7cff4640 1 -- 172.21.15.114:0/2577851432 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f3a600a1200 msgr2=0x7f3a600a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.537+0000 7f3a7cff4640 1 --2- 172.21.15.114:0/2577851432 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f3a600a1200 0x7f3a600a36c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3a700092f0 tx=0x7f3a70004a40 comp rx=0 tx=0).stop 2024-09-13T18:24:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.537+0000 7f3a7cff4640 1 -- 172.21.15.114:0/2577851432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a781062a0 msgr2=0x7f3a781966a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.537+0000 7f3a7cff4640 1 --2- 172.21.15.114:0/2577851432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a781062a0 0x7f3a781966a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3a78076560 tx=0x7f3a6c00ee50 comp rx=0 tx=0).stop 2024-09-13T18:24:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.537+0000 7f3a7cff4640 1 -- 172.21.15.114:0/2577851432 shutdown_connections 2024-09-13T18:24:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.537+0000 7f3a7cff4640 1 --2- 172.21.15.114:0/2577851432 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f3a600a1200 0x7f3a600a36c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.537+0000 7f3a7cff4640 1 --2- 172.21.15.114:0/2577851432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a7819aa90 0x7f3a7819af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.537+0000 7f3a7cff4640 1 --2- 172.21.15.114:0/2577851432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a781062a0 0x7f3a781966a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.537+0000 7f3a7cff4640 1 --2- 172.21.15.114:0/2577851432 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a780fea10 0x7f3a78196160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.537+0000 7f3a7cff4640 1 -- 172.21.15.114:0/2577851432 >> 172.21.15.114:0/2577851432 conn(0x7f3a78075340 msgr2=0x7f3a78103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.537+0000 7f3a7cff4640 1 -- 172.21.15.114:0/2577851432 shutdown_connections 2024-09-13T18:24:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:36.537+0000 7f3a7cff4640 1 -- 172.21.15.114:0/2577851432 wait complete. 2024-09-13T18:24:36.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:36 smithi134 ceph-mon[34283]: pgmap v67: 129 pgs: 129 active+clean; 9.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 151 MiB/s wr, 84 op/s 2024-09-13T18:24:37.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:37 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:24:37.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:37 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:24:37.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:37 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:24:39.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:38 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:24:39.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:38 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:24:39.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:38 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:24:39.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:38 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:24:39.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:38 smithi134 ceph-mon[34283]: pgmap v68: 129 pgs: 129 active+clean; 9.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 151 MiB/s wr, 84 op/s 2024-09-13T18:24:39.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:38 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:24:39.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:38 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:24:39.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:38 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:24:39.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:38 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:24:39.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:38 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:24:39.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:38 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:24:39.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:38 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:24:39.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:38 smithi204 ceph-mon[34777]: pgmap v68: 129 pgs: 129 active+clean; 9.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 151 MiB/s wr, 84 op/s 2024-09-13T18:24:39.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:38 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:24:39.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:38 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:24:39.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:38 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:24:39.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:38 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:24:39.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:38 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:24:39.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:38 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:24:39.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:38 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:24:39.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:38 smithi114 ceph-mon[30897]: pgmap v68: 129 pgs: 129 active+clean; 9.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 151 MiB/s wr, 84 op/s 2024-09-13T18:24:39.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:38 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:24:39.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:38 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:24:39.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:38 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:24:40.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:40 smithi204 ceph-mon[34777]: pgmap v69: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 217 MiB/s wr, 197 op/s 2024-09-13T18:24:40.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:40 smithi114 ceph-mon[30897]: pgmap v69: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 217 MiB/s wr, 197 op/s 2024-09-13T18:24:40.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:40 smithi134 ceph-mon[34283]: pgmap v69: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 217 MiB/s wr, 197 op/s 2024-09-13T18:24:42.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:42 smithi204 ceph-mon[34777]: pgmap v70: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 140 MiB/s wr, 154 op/s 2024-09-13T18:24:42.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:42 smithi114 ceph-mon[30897]: pgmap v70: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 140 MiB/s wr, 154 op/s 2024-09-13T18:24:42.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:42 smithi134 ceph-mon[34283]: pgmap v70: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 140 MiB/s wr, 154 op/s 2024-09-13T18:24:44.362 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T18:24:44.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:44 smithi204 ceph-mon[34777]: pgmap v71: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 167 MiB/s wr, 589 op/s 2024-09-13T18:24:44.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:44 smithi114 ceph-mon[30897]: pgmap v71: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 167 MiB/s wr, 589 op/s 2024-09-13T18:24:44.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.527+0000 7f2fbe525640 1 -- 172.21.15.114:0/4086840831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2fb80fff60 msgr2=0x7f2fb8100340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:44.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.527+0000 7f2fbe525640 1 --2- 172.21.15.114:0/4086840831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2fb80fff60 0x7f2fb8100340 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0009ca0 tx=0x7f2fb002e9a0 comp rx=0 tx=0).stop 2024-09-13T18:24:44.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.527+0000 7f2fbe525640 1 -- 172.21.15.114:0/4086840831 shutdown_connections 2024-09-13T18:24:44.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.527+0000 7f2fbe525640 1 --2- 172.21.15.114:0/4086840831 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2fb81050b0 0x7f2fb810cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:44.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.527+0000 7f2fbe525640 1 --2- 172.21.15.114:0/4086840831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fb8100910 0x7f2fb8104950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:44.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.527+0000 7f2fbe525640 1 --2- 172.21.15.114:0/4086840831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2fb80fff60 0x7f2fb8100340 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:44.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.527+0000 7f2fbe525640 1 -- 172.21.15.114:0/4086840831 >> 172.21.15.114:0/4086840831 conn(0x7f2fb80fba10 msgr2=0x7f2fb80fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:44.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.528+0000 7f2fbe525640 1 -- 172.21.15.114:0/4086840831 shutdown_connections 2024-09-13T18:24:44.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.528+0000 7f2fbe525640 1 -- 172.21.15.114:0/4086840831 wait complete. 2024-09-13T18:24:44.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.529+0000 7f2fbe525640 1 Processor -- start 2024-09-13T18:24:44.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.529+0000 7f2fbe525640 1 -- start start 2024-09-13T18:24:44.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.529+0000 7f2fbe525640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fb80fff60 0x7f2fb8194040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:44.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.529+0000 7f2fbe525640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2fb81050b0 0x7f2fb8194580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:44.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.529+0000 7f2fb7fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fb80fff60 0x7f2fb8194040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:44.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.530+0000 7f2fb7fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fb80fff60 0x7f2fb8194040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:57266/0 (socket says 172.21.15.114:57266) 2024-09-13T18:24:44.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.530+0000 7f2fb7fff640 1 -- 172.21.15.114:0/1766798714 learned_addr learned my addr 172.21.15.114:0/1766798714 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:44.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.530+0000 7f2fb77fe640 1 --2- 172.21.15.114:0/1766798714 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2fb81050b0 0x7f2fb8194580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:44.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.530+0000 7f2fbe525640 1 --2- 172.21.15.114:0/1766798714 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2fb8198970 0x7f2fb8198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:44.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.530+0000 7f2fbe525640 1 -- 172.21.15.114:0/1766798714 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb8199360 con 0x7f2fb8198970 2024-09-13T18:24:44.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.530+0000 7f2fbe525640 1 -- 172.21.15.114:0/1766798714 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb81994d0 con 0x7f2fb81050b0 2024-09-13T18:24:44.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.530+0000 7f2fbe525640 1 -- 172.21.15.114:0/1766798714 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb8199640 con 0x7f2fb80fff60 2024-09-13T18:24:44.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.530+0000 7f2fbcac2640 1 --2- 172.21.15.114:0/1766798714 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2fb8198970 0x7f2fb8198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:44.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.531+0000 7f2fb77fe640 1 -- 172.21.15.114:0/1766798714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fb80fff60 msgr2=0x7f2fb8194040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:44.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.531+0000 7f2fb77fe640 1 --2- 172.21.15.114:0/1766798714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fb80fff60 0x7f2fb8194040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:44.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.531+0000 7f2fb77fe640 1 -- 172.21.15.114:0/1766798714 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2fb8198970 msgr2=0x7f2fb8198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:44.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.531+0000 7f2fb77fe640 1 --2- 172.21.15.114:0/1766798714 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2fb8198970 0x7f2fb8198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:44.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.531+0000 7f2fb77fe640 1 -- 172.21.15.114:0/1766798714 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb0009900 con 0x7f2fb81050b0 2024-09-13T18:24:44.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.531+0000 7f2fb77fe640 1 --2- 172.21.15.114:0/1766798714 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2fb81050b0 0x7f2fb8194580 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2fac00e9c0 tx=0x7f2fac00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:44.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.532+0000 7f2fb57fa640 1 -- 172.21.15.114:0/1766798714 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fac00ccc0 con 0x7f2fb81050b0 2024-09-13T18:24:44.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.532+0000 7f2fb57fa640 1 -- 172.21.15.114:0/1766798714 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2fac00ce20 con 0x7f2fb81050b0 2024-09-13T18:24:44.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.532+0000 7f2fb57fa640 1 -- 172.21.15.114:0/1766798714 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fac018660 con 0x7f2fb81050b0 2024-09-13T18:24:44.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.532+0000 7f2fbe525640 1 -- 172.21.15.114:0/1766798714 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fb81a0c20 con 0x7f2fb81050b0 2024-09-13T18:24:44.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.532+0000 7f2fbe525640 1 -- 172.21.15.114:0/1766798714 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fb81a10d0 con 0x7f2fb81050b0 2024-09-13T18:24:44.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.537+0000 7f2fb57fa640 1 -- 172.21.15.114:0/1766798714 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f2fac010430 con 0x7f2fb81050b0 2024-09-13T18:24:44.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.537+0000 7f2fc124b740 1 Processor -- start 2024-09-13T18:24:44.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.538+0000 7f2fb57fa640 1 --2- 172.21.15.114:0/1766798714 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2fa00a1410 0x7f2fa00a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:44.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.538+0000 7f2fc124b740 1 -- start start 2024-09-13T18:24:44.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.538+0000 7f2fb57fa640 1 -- 172.21.15.114:0/1766798714 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2fac014070 con 0x7f2fb81050b0 2024-09-13T18:24:44.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.538+0000 7f2fc124b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559504e1f620 0x559504e1fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:44.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.538+0000 7f2fc124b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559504e1ff40 0x559504e2a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:44.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.538+0000 7f2fc124b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559504e2a7e0 0x559504e2cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:44.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.538+0000 7f2fc124b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x559504d81f20 con 0x559504e1f620 2024-09-13T18:24:44.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.538+0000 7f2fc124b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559504d7f6b0 con 0x559504e1ff40 2024-09-13T18:24:44.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.538+0000 7f2fc124b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559504d7afa0 con 0x559504e2a7e0 2024-09-13T18:24:44.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.538+0000 7f2fbcac2640 1 --2- 172.21.15.114:0/1766798714 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2fa00a1410 0x7f2fa00a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:44.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.538+0000 7f2fbcac2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559504e2a7e0 0x559504e2cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:44.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.538+0000 7f2fbcac2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559504e2a7e0 0x559504e2cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:57276/0 (socket says 172.21.15.114:57276) 2024-09-13T18:24:44.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.538+0000 7f2fbcac2640 1 -- 172.21.15.114:0/358549124 learned_addr learned my addr 172.21.15.114:0/358549124 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:44.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.538+0000 7f2fb7fff640 1 --2- 172.21.15.114:0/358549124 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559504e1f620 0x559504e1fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:44.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.539+0000 7f2fb77fe640 1 --2- 172.21.15.114:0/358549124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559504e1ff40 0x559504e2a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:44.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.539+0000 7f2fb77fe640 1 -- 172.21.15.114:0/358549124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559504e2a7e0 msgr2=0x559504e2cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:44.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.539+0000 7f2fb77fe640 1 --2- 172.21.15.114:0/358549124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559504e2a7e0 0x559504e2cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:44.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.539+0000 7f2fb77fe640 1 -- 172.21.15.114:0/358549124 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559504e1f620 msgr2=0x559504e1fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:44.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.539+0000 7f2fb77fe640 1 --2- 172.21.15.114:0/358549124 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559504e1f620 0x559504e1fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:44.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.539+0000 7f2fb77fe640 1 -- 172.21.15.114:0/358549124 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559504d35840 con 0x559504e1ff40 2024-09-13T18:24:44.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.539+0000 7f2fb77fe640 1 --2- 172.21.15.114:0/358549124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559504e1ff40 0x559504e2a2a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2fac0ca600 tx=0x7f2fac099a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da313b9fc3c18c8d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:44.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.539+0000 7f2f9e7fc640 1 -- 172.21.15.114:0/358549124 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fac0109c0 con 0x559504e1ff40 2024-09-13T18:24:44.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.540+0000 7f2fbcac2640 1 --2- 172.21.15.114:0/1766798714 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2fa00a1410 0x7f2fa00a38d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x559504e1cf10 tx=0x7f2fa80098c0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:44.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.540+0000 7f2f9e7fc640 1 -- 172.21.15.114:0/358549124 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2fac0ade90 con 0x559504e1ff40 2024-09-13T18:24:44.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.540+0000 7f2f9e7fc640 1 -- 172.21.15.114:0/358549124 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fac0ca880 con 0x559504e1ff40 2024-09-13T18:24:44.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.540+0000 7f2fc124b740 1 -- 172.21.15.114:0/358549124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559504e1ff40 msgr2=0x559504e2a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:44.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.540+0000 7f2fc124b740 1 --2- 172.21.15.114:0/358549124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559504e1ff40 0x559504e2a2a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2fac0ca600 tx=0x7f2fac099a90 comp rx=0 tx=0).stop 2024-09-13T18:24:44.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.541+0000 7f2fc124b740 1 -- 172.21.15.114:0/358549124 shutdown_connections 2024-09-13T18:24:44.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.541+0000 7f2fc124b740 1 --2- 172.21.15.114:0/358549124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559504e2a7e0 0x559504e2cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:44.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.541+0000 7f2fc124b740 1 --2- 172.21.15.114:0/358549124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559504e1ff40 0x559504e2a2a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:44.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.541+0000 7f2fc124b740 1 --2- 172.21.15.114:0/358549124 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559504e1f620 0x559504e1fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:44.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.541+0000 7f2fc124b740 1 -- 172.21.15.114:0/358549124 >> 172.21.15.114:0/358549124 conn(0x559504e1bb90 msgr2=0x559504e1bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:44.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.541+0000 7f2fc124b740 1 -- 172.21.15.114:0/358549124 shutdown_connections 2024-09-13T18:24:44.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.541+0000 7f2fc124b740 1 -- 172.21.15.114:0/358549124 wait complete. 2024-09-13T18:24:44.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.541+0000 7f2fc124b740 1 -- start start 2024-09-13T18:24:44.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.541+0000 7f2fc124b740 1 Processor -- start 2024-09-13T18:24:44.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.542+0000 7f2fc124b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559504e1f620 0x559504e2a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:44.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.542+0000 7f2fc124b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559504e1ff40 0x559504e274a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:44.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.542+0000 7f2fc124b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559504e2a7e0 0x559504e27c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:44.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.542+0000 7f2fc124b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x559504d1c2a0 con 0x559504e1f620 2024-09-13T18:24:44.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.542+0000 7f2fc124b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559504a53430 con 0x559504e2a7e0 2024-09-13T18:24:44.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.542+0000 7f2fc124b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559504cfe680 con 0x559504e1ff40 2024-09-13T18:24:44.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.542+0000 7f2fc124b740 10 client.0 _ll_register_callbacks cb 0x559504d74610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:24:44.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.542+0000 7f2fb77fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559504e1ff40 0x559504e274a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:44.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.542+0000 7f2fb7fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559504e1f620 0x559504e2a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:44.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.542+0000 7f2fb77fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559504e1ff40 0x559504e274a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:57284/0 (socket says 172.21.15.114:57284) 2024-09-13T18:24:44.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.542+0000 7f2fb77fe640 1 -- 172.21.15.114:0/1732404486 learned_addr learned my addr 172.21.15.114:0/1732404486 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:44.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.542+0000 7f2fbcac2640 1 --2- 172.21.15.114:0/1732404486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559504e2a7e0 0x559504e27c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:44.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.543+0000 7f2fb77fe640 1 -- 172.21.15.114:0/1732404486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559504e2a7e0 msgr2=0x559504e27c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:44.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.543+0000 7f2fb77fe640 1 --2- 172.21.15.114:0/1732404486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559504e2a7e0 0x559504e27c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:44.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.543+0000 7f2fb77fe640 1 -- 172.21.15.114:0/1732404486 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559504e1f620 msgr2=0x559504e2a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:44.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.543+0000 7f2fb77fe640 1 --2- 172.21.15.114:0/1732404486 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559504e1f620 0x559504e2a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:44.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.543+0000 7f2fb77fe640 1 -- 172.21.15.114:0/1732404486 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fb81a10d0 con 0x559504e1ff40 2024-09-13T18:24:44.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.543+0000 7f2fb7fff640 1 --2- 172.21.15.114:0/1732404486 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559504e1f620 0x559504e2a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:24:44.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.543+0000 7f2fb77fe640 1 --2- 172.21.15.114:0/1732404486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559504e1ff40 0x559504e274a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2fac00c6a0 tx=0x7f2fac0cd160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4e58f2de2a3061b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:44.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.543+0000 7f2f977fe640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:24:44.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.543+0000 7f2f977fe640 1 -- 172.21.15.114:0/1732404486 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fac0106e0 con 0x559504e1ff40 2024-09-13T18:24:44.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.544+0000 7f2fc124b740 1 -- 172.21.15.114:0/1732404486 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559504a6e4e0 con 0x559504e1ff40 2024-09-13T18:24:44.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.544+0000 7f2f977fe640 1 -- 172.21.15.114:0/1732404486 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2fac0a1fb0 con 0x559504e1ff40 2024-09-13T18:24:44.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.544+0000 7f2f977fe640 1 -- 172.21.15.114:0/1732404486 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fac010840 con 0x559504e1ff40 2024-09-13T18:24:44.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.544+0000 7f2fc124b740 10 client.25804 fetch_fsmap learned FSMap version 32 2024-09-13T18:24:44.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.544+0000 7f2fc124b740 1 -- 172.21.15.114:0/1732404486 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x559504d79620 con 0x559504e1ff40 2024-09-13T18:24:44.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.544+0000 7f2f977fe640 1 -- 172.21.15.114:0/1732404486 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559504a6e4e0 con 0x559504e1ff40 2024-09-13T18:24:44.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.545+0000 7f2f977fe640 1 -- 172.21.15.114:0/1732404486 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f2fac098fd0 con 0x559504e1ff40 2024-09-13T18:24:44.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.545+0000 7f2fc124b740 10 client.25804 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:24:44.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.545+0000 7f2fc124b740 10 client.25804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:24:44.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.545+0000 7f2fc124b740 20 client.25804 populate_metadata read hostname 'smithi114' 2024-09-13T18:24:44.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.545+0000 7f2fc124b740 1 --2- 172.21.15.114:0/1732404486 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559504e168c0 0x559504e18cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:44.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.545+0000 7f2fc124b740 4 client.25804 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:24:44.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.545+0000 7f2fc124b740 1 -- 172.21.15.114:0/1732404486 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559504a7f7e0 con 0x559504e168c0 2024-09-13T18:24:44.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.546+0000 7f2fb7fff640 1 --2- 172.21.15.114:0/1732404486 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559504e168c0 0x559504e18cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:44.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.547+0000 7f2fb7fff640 1 --2- 172.21.15.114:0/1732404486 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559504e168c0 0x559504e18cb0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=313c3c4b8e44c620 server_cookie=5a9eb15de6a3c2b9 in_seq=0 out_seq=0 2024-09-13T18:24:44.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.547+0000 7f2f977fe640 10 client.25804 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:24:44.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.548+0000 7f2f977fe640 1 -- 172.21.15.114:0/1732404486 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x559504a7f7e0 con 0x559504e168c0 2024-09-13T18:24:44.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.548+0000 7f2f977fe640 10 client.25804 handle_command_reply: tid=0 2024-09-13T18:24:44.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.548+0000 7f2fc124b740 1 client.25804 shutdown 2024-09-13T18:24:44.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.549+0000 7f2fc124b740 1 -- 172.21.15.114:0/1732404486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559504e1ff40 msgr2=0x559504e274a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:44.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.549+0000 7f2fc124b740 1 --2- 172.21.15.114:0/1732404486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559504e1ff40 0x559504e274a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2fac00c6a0 tx=0x7f2fac0cd160 comp rx=0 tx=0).stop 2024-09-13T18:24:44.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.549+0000 7f2fc124b740 1 -- 172.21.15.114:0/1732404486 shutdown_connections 2024-09-13T18:24:44.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.549+0000 7f2fc124b740 1 --2- 172.21.15.114:0/1732404486 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559504e168c0 0x559504e18cb0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:44.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.549+0000 7f2fc124b740 1 --2- 172.21.15.114:0/1732404486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559504e2a7e0 0x559504e27c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:44.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.549+0000 7f2fc124b740 1 --2- 172.21.15.114:0/1732404486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559504e1ff40 0x559504e274a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:44.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.549+0000 7f2fc124b740 1 --2- 172.21.15.114:0/1732404486 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559504e1f620 0x559504e2a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:44.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.549+0000 7f2fc124b740 1 -- 172.21.15.114:0/1732404486 >> 172.21.15.114:0/1732404486 conn(0x559504e1bb90 msgr2=0x559504e218f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:44.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.549+0000 7f2f977fe640 0 client.25804 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:24:44.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.549+0000 7f2fc124b740 1 -- 172.21.15.114:0/1732404486 shutdown_connections 2024-09-13T18:24:44.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.549+0000 7f2fc124b740 1 -- 172.21.15.114:0/1732404486 wait complete. 2024-09-13T18:24:44.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.549+0000 7f2fc124b740 20 client.25804 trim_cache size 0 max 16384 2024-09-13T18:24:44.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.578+0000 7f2fc124b740 1 Processor -- start 2024-09-13T18:24:44.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.578+0000 7f2fc124b740 1 -- start start 2024-09-13T18:24:44.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.578+0000 7f2fc124b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559504e1f620 0x559504e4b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:44.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.578+0000 7f2fc124b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559504e1ff40 0x559504e227a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:44.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.578+0000 7f2fc124b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559504e2a7e0 0x559504e22ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:44.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.578+0000 7f2fc124b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x559504a6e4e0 con 0x559504e1ff40 2024-09-13T18:24:44.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.579+0000 7f2fc124b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559504e28450 con 0x559504e1f620 2024-09-13T18:24:44.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.579+0000 7f2fc124b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559504e1f410 con 0x559504e2a7e0 2024-09-13T18:24:44.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.579+0000 7f2fbcac2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559504e2a7e0 0x559504e22ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:44.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.579+0000 7f2fbcac2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559504e2a7e0 0x559504e22ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:57290/0 (socket says 172.21.15.114:57290) 2024-09-13T18:24:44.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.579+0000 7f2fbcac2640 1 -- 172.21.15.114:0/2706039235 learned_addr learned my addr 172.21.15.114:0/2706039235 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:44.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.579+0000 7f2fb7fff640 1 --2- 172.21.15.114:0/2706039235 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559504e1f620 0x559504e4b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:44.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.579+0000 7f2fb77fe640 1 --2- 172.21.15.114:0/2706039235 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559504e1ff40 0x559504e227a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:44.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.579+0000 7f2fbcac2640 1 -- 172.21.15.114:0/2706039235 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559504e1f620 msgr2=0x559504e4b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:44.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.579+0000 7f2fbcac2640 1 --2- 172.21.15.114:0/2706039235 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559504e1f620 0x559504e4b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:44.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.579+0000 7f2fbcac2640 1 -- 172.21.15.114:0/2706039235 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559504e1ff40 msgr2=0x559504e227a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:44.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.579+0000 7f2fbcac2640 1 --2- 172.21.15.114:0/2706039235 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559504e1ff40 0x559504e227a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:44.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.579+0000 7f2fbcac2640 1 -- 172.21.15.114:0/2706039235 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559504d1b790 con 0x559504e2a7e0 2024-09-13T18:24:44.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.580+0000 7f2fbcac2640 1 --2- 172.21.15.114:0/2706039235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559504e2a7e0 0x559504e22ce0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8005920 tx=0x7f2fa8005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65c4741ff237a1e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:44.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.580+0000 7f2f9effd640 1 -- 172.21.15.114:0/2706039235 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa801f440 con 0x559504e2a7e0 2024-09-13T18:24:44.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.580+0000 7f2f9effd640 1 -- 172.21.15.114:0/2706039235 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2fa8016070 con 0x559504e2a7e0 2024-09-13T18:24:44.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.580+0000 7f2f9effd640 1 -- 172.21.15.114:0/2706039235 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa8028860 con 0x559504e2a7e0 2024-09-13T18:24:44.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.581+0000 7f2fc124b740 1 -- 172.21.15.114:0/2706039235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559504e2a7e0 msgr2=0x559504e22ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:44.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.581+0000 7f2fc124b740 1 --2- 172.21.15.114:0/2706039235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559504e2a7e0 0x559504e22ce0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8005920 tx=0x7f2fa8005f90 comp rx=0 tx=0).stop 2024-09-13T18:24:44.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.581+0000 7f2fc124b740 1 -- 172.21.15.114:0/2706039235 shutdown_connections 2024-09-13T18:24:44.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.581+0000 7f2fc124b740 1 --2- 172.21.15.114:0/2706039235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559504e2a7e0 0x559504e22ce0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:44.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.581+0000 7f2fc124b740 1 --2- 172.21.15.114:0/2706039235 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559504e1ff40 0x559504e227a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:44.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.581+0000 7f2fc124b740 1 --2- 172.21.15.114:0/2706039235 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559504e1f620 0x559504e4b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:44.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.581+0000 7f2fc124b740 1 -- 172.21.15.114:0/2706039235 >> 172.21.15.114:0/2706039235 conn(0x559504e1bb90 msgr2=0x559504e1bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:44.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.581+0000 7f2fc124b740 1 -- 172.21.15.114:0/2706039235 shutdown_connections 2024-09-13T18:24:44.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.581+0000 7f2fc124b740 1 -- 172.21.15.114:0/2706039235 wait complete. 2024-09-13T18:24:44.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.582+0000 7f2fc124b740 1 -- start start 2024-09-13T18:24:44.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.582+0000 7f2fc124b740 1 Processor -- start 2024-09-13T18:24:44.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.582+0000 7f2fc124b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559504e1f620 0x559504e31550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:44.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.582+0000 7f2fc124b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559504e1ff40 0x559504e31a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:44.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.582+0000 7f2fc124b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559504e2a7e0 0x559504e23ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:44.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.582+0000 7f2fc124b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x559504e241c0 con 0x559504e1f620 2024-09-13T18:24:44.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.582+0000 7f2fc124b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559504e24330 con 0x559504e1ff40 2024-09-13T18:24:44.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.582+0000 7f2fc124b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559504e244f0 con 0x559504e2a7e0 2024-09-13T18:24:44.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.582+0000 7f2fc124b740 10 client.0 _ll_register_callbacks cb 0x7f2fac005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:24:44.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.583+0000 7f2fb77fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559504e1ff40 0x559504e31a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:44.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.583+0000 7f2fb77fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559504e1ff40 0x559504e31a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:39102/0 (socket says 172.21.15.114:39102) 2024-09-13T18:24:44.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.583+0000 7f2fb77fe640 1 -- 172.21.15.114:0/3126954648 learned_addr learned my addr 172.21.15.114:0/3126954648 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:44.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.583+0000 7f2fb7fff640 1 --2- 172.21.15.114:0/3126954648 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559504e1f620 0x559504e31550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:44.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.583+0000 7f2fbcac2640 1 --2- 172.21.15.114:0/3126954648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559504e2a7e0 0x559504e23ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:44.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.583+0000 7f2fb77fe640 1 -- 172.21.15.114:0/3126954648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559504e2a7e0 msgr2=0x559504e23ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:44.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.583+0000 7f2fb77fe640 1 --2- 172.21.15.114:0/3126954648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559504e2a7e0 0x559504e23ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:44.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.583+0000 7f2fb77fe640 1 -- 172.21.15.114:0/3126954648 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559504e1f620 msgr2=0x559504e31550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:44.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.583+0000 7f2fb77fe640 1 --2- 172.21.15.114:0/3126954648 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559504e1f620 0x559504e31550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:44.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.583+0000 7f2fb77fe640 1 -- 172.21.15.114:0/3126954648 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fa8019040 con 0x559504e1ff40 2024-09-13T18:24:44.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.583+0000 7f2fb7fff640 1 --2- 172.21.15.114:0/3126954648 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559504e1f620 0x559504e31550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:24:44.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.583+0000 7f2fbcac2640 1 --2- 172.21.15.114:0/3126954648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559504e2a7e0 0x559504e23ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:24:44.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.583+0000 7f2fb77fe640 1 --2- 172.21.15.114:0/3126954648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559504e1ff40 0x559504e31a90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2fac099330 tx=0x7f2fac00c7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=170e906f1e1ee663 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:44.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.583+0000 7f2f977fe640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:24:44.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.583+0000 7f2f977fe640 1 -- 172.21.15.114:0/3126954648 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fac0b8250 con 0x559504e1ff40 2024-09-13T18:24:44.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.583+0000 7f2f977fe640 1 -- 172.21.15.114:0/3126954648 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2fac0a1fb0 con 0x559504e1ff40 2024-09-13T18:24:44.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.583+0000 7f2fc124b740 1 -- 172.21.15.114:0/3126954648 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2fac0ade90 con 0x559504e1ff40 2024-09-13T18:24:44.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.584+0000 7f2f977fe640 1 -- 172.21.15.114:0/3126954648 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fac0b83b0 con 0x559504e1ff40 2024-09-13T18:24:44.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.584+0000 7f2fc124b740 10 client.25766 fetch_fsmap learned FSMap version 32 2024-09-13T18:24:44.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.584+0000 7f2fc124b740 1 -- 172.21.15.114:0/3126954648 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x559504d40ea0 con 0x559504e1ff40 2024-09-13T18:24:44.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.584+0000 7f2f977fe640 1 -- 172.21.15.114:0/3126954648 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2fac0ade90 con 0x559504e1ff40 2024-09-13T18:24:44.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.584+0000 7f2f977fe640 1 -- 172.21.15.114:0/3126954648 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f2fac005020 con 0x559504e1ff40 2024-09-13T18:24:44.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.584+0000 7f2fc124b740 10 client.25766 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:24:44.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.584+0000 7f2fc124b740 10 client.25766 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:24:44.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.584+0000 7f2fc124b740 20 client.25766 populate_metadata read hostname 'smithi114' 2024-09-13T18:24:44.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.584+0000 7f2fc124b740 1 --2- 172.21.15.114:0/3126954648 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559504e99b60 0x559504e9bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:44.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.584+0000 7f2fc124b740 4 client.25766 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T18:24:44.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.584+0000 7f2fc124b740 1 -- 172.21.15.114:0/3126954648 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559504ad2e30 con 0x559504e99b60 2024-09-13T18:24:44.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.584+0000 7f2fb7fff640 1 --2- 172.21.15.114:0/3126954648 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559504e99b60 0x559504e9bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:44.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.585+0000 7f2fb7fff640 1 --2- 172.21.15.114:0/3126954648 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559504e99b60 0x559504e9bf80 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e966be5d0b24383 server_cookie=7fc9e8a5426bbbaa in_seq=0 out_seq=0 2024-09-13T18:24:44.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.585+0000 7f2f977fe640 10 client.25766 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:24:44.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.585+0000 7f2f977fe640 1 -- 172.21.15.114:0/3126954648 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x559504ad2e30 con 0x559504e99b60 2024-09-13T18:24:44.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.585+0000 7f2f977fe640 10 client.25766 handle_command_reply: tid=0 2024-09-13T18:24:44.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.585+0000 7f2fc124b740 1 client.25766 shutdown 2024-09-13T18:24:44.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.585+0000 7f2fc124b740 1 -- 172.21.15.114:0/3126954648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559504e1ff40 msgr2=0x559504e31a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:44.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.585+0000 7f2fc124b740 1 --2- 172.21.15.114:0/3126954648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559504e1ff40 0x559504e31a90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2fac099330 tx=0x7f2fac00c7d0 comp rx=0 tx=0).stop 2024-09-13T18:24:44.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.585+0000 7f2fc124b740 1 -- 172.21.15.114:0/3126954648 shutdown_connections 2024-09-13T18:24:44.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.585+0000 7f2fc124b740 1 --2- 172.21.15.114:0/3126954648 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559504e99b60 0x559504e9bf80 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:44.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.585+0000 7f2fc124b740 1 --2- 172.21.15.114:0/3126954648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559504e2a7e0 0x559504e23ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:44.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.585+0000 7f2fc124b740 1 --2- 172.21.15.114:0/3126954648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559504e1ff40 0x559504e31a90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:44.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.585+0000 7f2f977fe640 0 client.25766 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:24:44.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.585+0000 7f2fc124b740 1 --2- 172.21.15.114:0/3126954648 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559504e1f620 0x559504e31550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:44.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.585+0000 7f2fc124b740 1 -- 172.21.15.114:0/3126954648 >> 172.21.15.114:0/3126954648 conn(0x559504e1bb90 msgr2=0x559504e227a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:44.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.586+0000 7f2fc124b740 1 -- 172.21.15.114:0/3126954648 shutdown_connections 2024-09-13T18:24:44.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.586+0000 7f2fc124b740 1 -- 172.21.15.114:0/3126954648 wait complete. 2024-09-13T18:24:44.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.586+0000 7f2fc124b740 20 client.25766 trim_cache size 0 max 16384 2024-09-13T18:24:44.597 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:24:44.597 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "no active scrubs running", 2024-09-13T18:24:44.597 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": {} 2024-09-13T18:24:44.597 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:24:44.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.587+0000 7f2fbe525640 1 -- 172.21.15.114:0/1766798714 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2fa00a1410 msgr2=0x7f2fa00a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:44.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.587+0000 7f2fbe525640 1 --2- 172.21.15.114:0/1766798714 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2fa00a1410 0x7f2fa00a38d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x559504e1cf10 tx=0x7f2fa80098c0 comp rx=0 tx=0).stop 2024-09-13T18:24:44.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.587+0000 7f2fbe525640 1 -- 172.21.15.114:0/1766798714 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2fb81050b0 msgr2=0x7f2fb8194580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:44.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.587+0000 7f2fbe525640 1 --2- 172.21.15.114:0/1766798714 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2fb81050b0 0x7f2fb8194580 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2fac00e9c0 tx=0x7f2fac00ee80 comp rx=0 tx=0).stop 2024-09-13T18:24:44.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.587+0000 7f2fbe525640 1 -- 172.21.15.114:0/1766798714 shutdown_connections 2024-09-13T18:24:44.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.587+0000 7f2fbe525640 1 --2- 172.21.15.114:0/1766798714 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2fa00a1410 0x7f2fa00a38d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:44.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.587+0000 7f2fbe525640 1 --2- 172.21.15.114:0/1766798714 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2fb8198970 0x7f2fb8198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:44.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.587+0000 7f2fbe525640 1 --2- 172.21.15.114:0/1766798714 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2fb81050b0 0x7f2fb8194580 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:44.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.587+0000 7f2fbe525640 1 --2- 172.21.15.114:0/1766798714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fb80fff60 0x7f2fb8194040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:44.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.587+0000 7f2fbe525640 1 -- 172.21.15.114:0/1766798714 >> 172.21.15.114:0/1766798714 conn(0x7f2fb80fba10 msgr2=0x7f2fb80fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:44.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.588+0000 7f2fbe525640 1 -- 172.21.15.114:0/1766798714 shutdown_connections 2024-09-13T18:24:44.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.588+0000 7f2fbe525640 1 -- 172.21.15.114:0/1766798714 wait complete. 2024-09-13T18:24:44.609 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-13T18:24:44.609 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:24:44.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:44 smithi134 ceph-mon[34283]: pgmap v71: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 167 MiB/s wr, 589 op/s 2024-09-13T18:24:44.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.778+0000 7f1e2effd640 1 -- 172.21.15.114:0/314379354 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c030ad0 con 0x7f1e2809a540 2024-09-13T18:24:44.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.778+0000 7f1e351c9640 1 -- 172.21.15.114:0/314379354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e2809a540 msgr2=0x7f1e2809a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:44.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.778+0000 7f1e351c9640 1 --2- 172.21.15.114:0/314379354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e2809a540 0x7f1e2809a920 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c009860 tx=0x7f1e1c02a3d0 comp rx=0 tx=0).stop 2024-09-13T18:24:44.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.779+0000 7f1e351c9640 1 -- 172.21.15.114:0/314379354 shutdown_connections 2024-09-13T18:24:44.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.779+0000 7f1e351c9640 1 --2- 172.21.15.114:0/314379354 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e28005820 0x7f1e28098010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:44.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.779+0000 7f1e351c9640 1 --2- 172.21.15.114:0/314379354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e28004c80 0x7f1e280050c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:44.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.779+0000 7f1e351c9640 1 --2- 172.21.15.114:0/314379354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e2809a540 0x7f1e2809a920 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:44.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.779+0000 7f1e351c9640 1 -- 172.21.15.114:0/314379354 >> 172.21.15.114:0/314379354 conn(0x7f1e2800a1c0 msgr2=0x7f1e2800a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:44.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.780+0000 7f1e351c9640 1 -- 172.21.15.114:0/314379354 shutdown_connections 2024-09-13T18:24:44.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.780+0000 7f1e351c9640 1 -- 172.21.15.114:0/314379354 wait complete. 2024-09-13T18:24:44.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.780+0000 7f1e351c9640 1 Processor -- start 2024-09-13T18:24:44.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.781+0000 7f1e351c9640 1 -- start start 2024-09-13T18:24:44.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.781+0000 7f1e351c9640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e28004c80 0x7f1e28143170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:44.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.781+0000 7f1e351c9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e28005820 0x7f1e281436b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:44.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.781+0000 7f1e351c9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e2809a540 0x7f1e28147aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:44.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.781+0000 7f1e351c9640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e281480a0 con 0x7f1e28004c80 2024-09-13T18:24:44.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.781+0000 7f1e351c9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e28148210 con 0x7f1e28005820 2024-09-13T18:24:44.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.781+0000 7f1e351c9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e281483d0 con 0x7f1e2809a540 2024-09-13T18:24:44.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.782+0000 7f1e2ffff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e28004c80 0x7f1e28143170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:44.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.782+0000 7f1e2f7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e28005820 0x7f1e281436b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:44.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.782+0000 7f1e2ffff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e28004c80 0x7f1e28143170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51492/0 (socket says 172.21.15.114:51492) 2024-09-13T18:24:44.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.782+0000 7f1e2ffff640 1 -- 172.21.15.114:0/676524118 learned_addr learned my addr 172.21.15.114:0/676524118 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:44.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.782+0000 7f1e349c8640 1 --2- 172.21.15.114:0/676524118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e2809a540 0x7f1e28147aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:44.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.782+0000 7f1e2f7fe640 1 -- 172.21.15.114:0/676524118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e2809a540 msgr2=0x7f1e28147aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:44.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.782+0000 7f1e2f7fe640 1 --2- 172.21.15.114:0/676524118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e2809a540 0x7f1e28147aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:44.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.782+0000 7f1e2f7fe640 1 -- 172.21.15.114:0/676524118 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e28004c80 msgr2=0x7f1e28143170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:44.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.782+0000 7f1e2f7fe640 1 --2- 172.21.15.114:0/676524118 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e28004c80 0x7f1e28143170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:44.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.782+0000 7f1e2f7fe640 1 -- 172.21.15.114:0/676524118 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e1c009490 con 0x7f1e28005820 2024-09-13T18:24:44.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.782+0000 7f1e349c8640 1 --2- 172.21.15.114:0/676524118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e2809a540 0x7f1e28147aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:24:44.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.782+0000 7f1e2f7fe640 1 --2- 172.21.15.114:0/676524118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e28005820 0x7f1e281436b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1e2400e9f0 tx=0x7f1e2400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:44.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.783+0000 7f1e2d7fa640 1 -- 172.21.15.114:0/676524118 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2400ccc0 con 0x7f1e28005820 2024-09-13T18:24:44.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.783+0000 7f1e2d7fa640 1 -- 172.21.15.114:0/676524118 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1e2400ce20 con 0x7f1e28005820 2024-09-13T18:24:44.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.783+0000 7f1e2d7fa640 1 -- 172.21.15.114:0/676524118 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e24018750 con 0x7f1e28005820 2024-09-13T18:24:44.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.783+0000 7f1e351c9640 1 -- 172.21.15.114:0/676524118 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e281486b0 con 0x7f1e28005820 2024-09-13T18:24:44.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.783+0000 7f1e351c9640 1 -- 172.21.15.114:0/676524118 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e2814ff30 con 0x7f1e28005820 2024-09-13T18:24:44.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.785+0000 7f1e351c9640 1 -- 172.21.15.114:0/676524118 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e28094af0 con 0x7f1e28005820 2024-09-13T18:24:44.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.787+0000 7f1e2d7fa640 1 -- 172.21.15.114:0/676524118 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f1e24010430 con 0x7f1e28005820 2024-09-13T18:24:44.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.787+0000 7f1e2d7fa640 1 --2- 172.21.15.114:0/676524118 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f1e200a1410 0x7f1e200a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:44.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.788+0000 7f1e2d7fa640 1 -- 172.21.15.114:0/676524118 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f1e24014070 con 0x7f1e28005820 2024-09-13T18:24:44.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.788+0000 7f1e2ffff640 1 --2- 172.21.15.114:0/676524118 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f1e200a1410 0x7f1e200a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:44.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.789+0000 7f1e2ffff640 1 --2- 172.21.15.114:0/676524118 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f1e200a1410 0x7f1e200a38d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c0057c0 tx=0x7f1e1c004120 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:44.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:44.794+0000 7f1e2d7fa640 1 -- 172.21.15.114:0/676524118 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f1e24052e20 con 0x7f1e28005820 2024-09-13T18:24:45.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.202+0000 7f1e351c9640 1 -- 172.21.15.114:0/676524118 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1e28098010 con 0x7f1e28005820 2024-09-13T18:24:45.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.202+0000 7f1e2d7fa640 1 -- 172.21.15.114:0/676524118 <== mon.2 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+10937 (secure 0 0 0) 0x7f1e240963d0 con 0x7f1e28005820 2024-09-13T18:24:45.207 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:24:45.207 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4162817497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4162817497},{"type":"v1","addr":"172.21.15.114:6835","nonce":4162817497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/2435428478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":2435428478},{"type":"v1","addr":"172.21.15.114:6837","nonce":2435428478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T18:20:01.895117+0000","modified":"2024-09-13T18:24:01.324695+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24482,"mds_2":24475,"mds_3":24458,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/2046516451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":2046516451},{"type":"v1","addr":"172.21.15.114:6839","nonce":2046516451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":10,"addr":"172.21.15.114:6841/1956923446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":1956923446},{"type":"v1","addr":"172.21.15.114:6841","nonce":1956923446}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24391":{"gid":24391,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/507301496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":507301496},{"type":"v1","addr":"172.21.15.204:6833","nonce":507301496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4028657219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4028657219},{"type":"v1","addr":"172.21.15.134:6833","nonce":4028657219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/57029119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":57029119},{"type":"v1","addr":"172.21.15.204:6835","nonce":57029119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3068880939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3068880939},{"type":"v1","addr":"172.21.15.204:6837","nonce":3068880939}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"e","rank":3,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.134:6835/1150003275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1150003275},{"type":"v1","addr":"172.21.15.134:6835","nonce":1150003275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/344883912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":344883912},{"type":"v1","addr":"172.21.15.204:6839","nonce":344883912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6837/891692210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":891692210},{"type":"v1","addr":"172.21.15.134:6837","nonce":891692210}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6839/3466309724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3466309724},{"type":"v1","addr":"172.21.15.134:6839","nonce":3466309724}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:24:45.207 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 32 2024-09-13T18:24:45.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.206+0000 7f1e351c9640 1 -- 172.21.15.114:0/676524118 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f1e200a1410 msgr2=0x7f1e200a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:45.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.206+0000 7f1e351c9640 1 --2- 172.21.15.114:0/676524118 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f1e200a1410 0x7f1e200a38d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c0057c0 tx=0x7f1e1c004120 comp rx=0 tx=0).stop 2024-09-13T18:24:45.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.206+0000 7f1e351c9640 1 -- 172.21.15.114:0/676524118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e28005820 msgr2=0x7f1e281436b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:45.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.206+0000 7f1e351c9640 1 --2- 172.21.15.114:0/676524118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e28005820 0x7f1e281436b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1e2400e9f0 tx=0x7f1e2400eeb0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.206+0000 7f1e351c9640 1 -- 172.21.15.114:0/676524118 shutdown_connections 2024-09-13T18:24:45.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.206+0000 7f1e351c9640 1 --2- 172.21.15.114:0/676524118 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f1e200a1410 0x7f1e200a38d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.206+0000 7f1e351c9640 1 --2- 172.21.15.114:0/676524118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e2809a540 0x7f1e28147aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.206+0000 7f1e351c9640 1 --2- 172.21.15.114:0/676524118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e28005820 0x7f1e281436b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.206+0000 7f1e351c9640 1 --2- 172.21.15.114:0/676524118 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e28004c80 0x7f1e28143170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.206+0000 7f1e351c9640 1 -- 172.21.15.114:0/676524118 >> 172.21.15.114:0/676524118 conn(0x7f1e2800a1c0 msgr2=0x7f1e28094fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:45.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.206+0000 7f1e351c9640 1 -- 172.21.15.114:0/676524118 shutdown_connections 2024-09-13T18:24:45.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.206+0000 7f1e351c9640 1 -- 172.21.15.114:0/676524118 wait complete. 2024-09-13T18:24:45.224 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-09-13T18:24:45.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.405+0000 7f0a1087f640 1 -- 172.21.15.114:0/3632443835 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc031780 con 0x7f0a0c1074b0 2024-09-13T18:24:45.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.406+0000 7f0a13ae5640 1 -- 172.21.15.114:0/3632443835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a0c1074b0 msgr2=0x7f0a0c10e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:45.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.406+0000 7f0a13ae5640 1 --2- 172.21.15.114:0/3632443835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a0c1074b0 0x7f0a0c10e510 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f09fc009930 tx=0x7f09fc02a440 comp rx=0 tx=0).stop 2024-09-13T18:24:45.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.406+0000 7f0a13ae5640 1 -- 172.21.15.114:0/3632443835 shutdown_connections 2024-09-13T18:24:45.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.406+0000 7f0a13ae5640 1 --2- 172.21.15.114:0/3632443835 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a0c119400 0x7f0a0c11b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.406+0000 7f0a13ae5640 1 --2- 172.21.15.114:0/3632443835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a0c107890 0x7f0a0c107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.406+0000 7f0a13ae5640 1 --2- 172.21.15.114:0/3632443835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a0c1074b0 0x7f0a0c10e510 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.406+0000 7f0a13ae5640 1 -- 172.21.15.114:0/3632443835 >> 172.21.15.114:0/3632443835 conn(0x7f0a0c06af70 msgr2=0x7f0a0c06b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:45.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.406+0000 7f0a13ae5640 1 -- 172.21.15.114:0/3632443835 shutdown_connections 2024-09-13T18:24:45.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.406+0000 7f0a13ae5640 1 -- 172.21.15.114:0/3632443835 wait complete. 2024-09-13T18:24:45.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.407+0000 7f0a13ae5640 1 Processor -- start 2024-09-13T18:24:45.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.408+0000 7f0a13ae5640 1 -- start start 2024-09-13T18:24:45.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.408+0000 7f0a13ae5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a0c1074b0 0x7f0a0c1a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:45.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.408+0000 7f0a13ae5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a0c107890 0x7f0a0c1a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:45.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.408+0000 7f0a13ae5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a0c119400 0x7f0a0c1a98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:45.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.408+0000 7f0a13ae5640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c1a9eb0 con 0x7f0a0c107890 2024-09-13T18:24:45.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.408+0000 7f0a13ae5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c1aa020 con 0x7f0a0c119400 2024-09-13T18:24:45.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.408+0000 7f0a13ae5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c1aa1e0 con 0x7f0a0c1074b0 2024-09-13T18:24:45.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.409+0000 7f0a11080640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a0c107890 0x7f0a0c1a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:45.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.409+0000 7f0a12082640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a0c119400 0x7f0a0c1a98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:45.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.409+0000 7f0a11080640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a0c107890 0x7f0a0c1a54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51498/0 (socket says 172.21.15.114:51498) 2024-09-13T18:24:45.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.409+0000 7f0a12082640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a0c119400 0x7f0a0c1a98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:39126/0 (socket says 172.21.15.114:39126) 2024-09-13T18:24:45.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.409+0000 7f0a11080640 1 -- 172.21.15.114:0/24266263 learned_addr learned my addr 172.21.15.114:0/24266263 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:45.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.409+0000 7f0a11881640 1 --2- 172.21.15.114:0/24266263 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a0c1074b0 0x7f0a0c1a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:45.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.409+0000 7f0a11080640 1 -- 172.21.15.114:0/24266263 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a0c1074b0 msgr2=0x7f0a0c1a4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:45.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.409+0000 7f0a11080640 1 --2- 172.21.15.114:0/24266263 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a0c1074b0 0x7f0a0c1a4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.409+0000 7f0a11080640 1 -- 172.21.15.114:0/24266263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a0c119400 msgr2=0x7f0a0c1a98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:45.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.409+0000 7f0a11080640 1 --2- 172.21.15.114:0/24266263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a0c119400 0x7f0a0c1a98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.409+0000 7f0a11080640 1 -- 172.21.15.114:0/24266263 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09fc009560 con 0x7f0a0c107890 2024-09-13T18:24:45.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.409+0000 7f0a12082640 1 --2- 172.21.15.114:0/24266263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a0c119400 0x7f0a0c1a98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:24:45.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.410+0000 7f0a11080640 1 --2- 172.21.15.114:0/24266263 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a0c107890 0x7f0a0c1a54c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f0a0800e9f0 tx=0x7f0a0800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:45.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.410+0000 7f0a02ffd640 1 -- 172.21.15.114:0/24266263 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a08009700 con 0x7f0a0c107890 2024-09-13T18:24:45.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.410+0000 7f0a02ffd640 1 -- 172.21.15.114:0/24266263 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0a0800cd80 con 0x7f0a0c107890 2024-09-13T18:24:45.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.410+0000 7f0a02ffd640 1 -- 172.21.15.114:0/24266263 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a08018750 con 0x7f0a0c107890 2024-09-13T18:24:45.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.410+0000 7f0a13ae5640 1 -- 172.21.15.114:0/24266263 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a0c1aa4c0 con 0x7f0a0c107890 2024-09-13T18:24:45.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.410+0000 7f0a13ae5640 1 -- 172.21.15.114:0/24266263 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a0c1b1d40 con 0x7f0a0c107890 2024-09-13T18:24:45.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.413+0000 7f0a02ffd640 1 -- 172.21.15.114:0/24266263 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f0a08010430 con 0x7f0a0c107890 2024-09-13T18:24:45.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.414+0000 7f0a16802740 1 Processor -- start 2024-09-13T18:24:45.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.414+0000 7f0a16802740 1 -- start start 2024-09-13T18:24:45.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.414+0000 7f0a02ffd640 1 --2- 172.21.15.114:0/24266263 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f09f80a1350 0x7f09f80a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:45.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.414+0000 7f0a16802740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe06e3b970 0x55fe06e3bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:45.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.414+0000 7f0a16802740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe06e3c290 0x55fe06e465f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:45.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.414+0000 7f0a16802740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe06e46b30 0x55fe06e48f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:45.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.414+0000 7f0a16802740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe06da5cc0 con 0x55fe06e46b30 2024-09-13T18:24:45.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.414+0000 7f0a16802740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe06da3450 con 0x55fe06e3c290 2024-09-13T18:24:45.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.414+0000 7f0a16802740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe06d9ed40 con 0x55fe06e3b970 2024-09-13T18:24:45.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.414+0000 7f0a02ffd640 1 -- 172.21.15.114:0/24266263 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f0a08014070 con 0x7f0a0c107890 2024-09-13T18:24:45.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.415+0000 7f0a11080640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe06e3c290 0x55fe06e465f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:45.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.415+0000 7f0a11080640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe06e3c290 0x55fe06e465f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:39142/0 (socket says 172.21.15.114:39142) 2024-09-13T18:24:45.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.415+0000 7f0a11080640 1 -- 172.21.15.114:0/2290020103 learned_addr learned my addr 172.21.15.114:0/2290020103 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:45.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.415+0000 7f0a11881640 1 --2- 172.21.15.114:0/2290020103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe06e3b970 0x55fe06e3bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:45.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.415+0000 7f0a12082640 1 --2- 172.21.15.114:0/24266263 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f09f80a1350 0x7f09f80a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:45.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.415+0000 7f0a11080640 1 -- 172.21.15.114:0/2290020103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe06e3b970 msgr2=0x55fe06e3bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:45.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.415+0000 7f0a11080640 1 --2- 172.21.15.114:0/2290020103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe06e3b970 0x55fe06e3bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.415+0000 7f0a11080640 1 -- 172.21.15.114:0/2290020103 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe06e46b30 msgr2=0x55fe06e48f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:45.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.415+0000 7f0a11080640 1 --2- 172.21.15.114:0/2290020103 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe06e46b30 0x55fe06e48f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.415+0000 7f0a11080640 1 -- 172.21.15.114:0/2290020103 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe06d66620 con 0x55fe06e3c290 2024-09-13T18:24:45.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.415+0000 7f0a11881640 1 --2- 172.21.15.114:0/2290020103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe06e3b970 0x55fe06e3bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:24:45.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.415+0000 7f0a11080640 1 --2- 172.21.15.114:0/2290020103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe06e3c290 0x55fe06e465f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0a0800c6a0 tx=0x7f0a0809a860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98b48e0e08e4a23a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:45.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.416+0000 7f09f7fff640 1 -- 172.21.15.114:0/2290020103 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a080a25e0 con 0x55fe06e3c290 2024-09-13T18:24:45.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.416+0000 7f09f7fff640 1 -- 172.21.15.114:0/2290020103 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0a080cd160 con 0x55fe06e3c290 2024-09-13T18:24:45.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.416+0000 7f09f7fff640 1 -- 172.21.15.114:0/2290020103 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a080106e0 con 0x55fe06e3c290 2024-09-13T18:24:45.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.417+0000 7f0a12082640 1 --2- 172.21.15.114:0/24266263 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f09f80a1350 0x7f09f80a3810 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x55fe06e39260 tx=0x7f0a040091e0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:45.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.417+0000 7f0a16802740 1 -- 172.21.15.114:0/2290020103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe06e3c290 msgr2=0x55fe06e465f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:45.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.417+0000 7f0a16802740 1 --2- 172.21.15.114:0/2290020103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe06e3c290 0x55fe06e465f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0a0800c6a0 tx=0x7f0a0809a860 comp rx=0 tx=0).stop 2024-09-13T18:24:45.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.417+0000 7f0a16802740 1 -- 172.21.15.114:0/2290020103 shutdown_connections 2024-09-13T18:24:45.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.417+0000 7f0a16802740 1 --2- 172.21.15.114:0/2290020103 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe06e46b30 0x55fe06e48f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.417+0000 7f0a16802740 1 --2- 172.21.15.114:0/2290020103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe06e3c290 0x55fe06e465f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.417+0000 7f0a16802740 1 --2- 172.21.15.114:0/2290020103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe06e3b970 0x55fe06e3bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.417+0000 7f0a16802740 1 -- 172.21.15.114:0/2290020103 >> 172.21.15.114:0/2290020103 conn(0x55fe06e37ee0 msgr2=0x55fe06e382c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:45.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.417+0000 7f0a16802740 1 -- 172.21.15.114:0/2290020103 shutdown_connections 2024-09-13T18:24:45.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.417+0000 7f0a16802740 1 -- 172.21.15.114:0/2290020103 wait complete. 2024-09-13T18:24:45.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.417+0000 7f0a16802740 1 -- start start 2024-09-13T18:24:45.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.417+0000 7f0a16802740 1 Processor -- start 2024-09-13T18:24:45.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.418+0000 7f0a16802740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe06e3b970 0x55fe06e46150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:45.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.418+0000 7f0a16802740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe06e3c290 0x55fe06e43420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:45.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.418+0000 7f0a16802740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe06e46b30 0x55fe06e43960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:45.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.418+0000 7f0a16802740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe06d5d180 con 0x55fe06e3b970 2024-09-13T18:24:45.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.418+0000 7f0a16802740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe06a77430 con 0x55fe06e46b30 2024-09-13T18:24:45.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.418+0000 7f0a16802740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe06d225e0 con 0x55fe06e3c290 2024-09-13T18:24:45.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.418+0000 7f0a16802740 10 client.0 _ll_register_callbacks cb 0x55fe06d8eaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:24:45.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.418+0000 7f0a11881640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe06e3b970 0x55fe06e46150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:45.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.418+0000 7f0a11881640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe06e3b970 0x55fe06e46150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51514/0 (socket says 172.21.15.114:51514) 2024-09-13T18:24:45.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.418+0000 7f0a11881640 1 -- 172.21.15.114:0/915554629 learned_addr learned my addr 172.21.15.114:0/915554629 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:45.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.418+0000 7f0a12082640 1 --2- 172.21.15.114:0/915554629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe06e46b30 0x55fe06e43960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:45.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.418+0000 7f0a11080640 1 --2- 172.21.15.114:0/915554629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe06e3c290 0x55fe06e43420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:45.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.418+0000 7f0a12082640 1 -- 172.21.15.114:0/915554629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe06e3c290 msgr2=0x55fe06e43420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:45.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.418+0000 7f0a12082640 1 --2- 172.21.15.114:0/915554629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe06e3c290 0x55fe06e43420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.418+0000 7f0a12082640 1 -- 172.21.15.114:0/915554629 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe06e3b970 msgr2=0x55fe06e46150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:45.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.419+0000 7f0a12082640 1 --2- 172.21.15.114:0/915554629 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe06e3b970 0x55fe06e46150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.419+0000 7f0a12082640 1 -- 172.21.15.114:0/915554629 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a0c1b1d40 con 0x55fe06e46b30 2024-09-13T18:24:45.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.419+0000 7f0a11881640 1 --2- 172.21.15.114:0/915554629 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe06e3b970 0x55fe06e46150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:24:45.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.419+0000 7f0a12082640 1 --2- 172.21.15.114:0/915554629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe06e46b30 0x55fe06e43960 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x55fe06e3e510 tx=0x7f0a04009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f9ab587cddb00 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:45.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.419+0000 7f09f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:24:45.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.419+0000 7f09f57fa640 1 -- 172.21.15.114:0/915554629 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a04009900 con 0x55fe06e46b30 2024-09-13T18:24:45.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.419+0000 7f09f57fa640 1 -- 172.21.15.114:0/915554629 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f0a04022070 con 0x55fe06e46b30 2024-09-13T18:24:45.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.419+0000 7f0a16802740 1 -- 172.21.15.114:0/915554629 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fe06dc8b80 con 0x55fe06e46b30 2024-09-13T18:24:45.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.419+0000 7f09f57fa640 1 -- 172.21.15.114:0/915554629 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a0402b5a0 con 0x55fe06e46b30 2024-09-13T18:24:45.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.419+0000 7f0a16802740 10 client.25784 fetch_fsmap learned FSMap version 32 2024-09-13T18:24:45.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.419+0000 7f0a16802740 1 -- 172.21.15.114:0/915554629 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55fe06d9d3c0 con 0x55fe06e46b30 2024-09-13T18:24:45.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.420+0000 7f09f57fa640 1 -- 172.21.15.114:0/915554629 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55fe06dc8b80 con 0x55fe06e46b30 2024-09-13T18:24:45.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.420+0000 7f09f57fa640 1 -- 172.21.15.114:0/915554629 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f0a04010650 con 0x55fe06e46b30 2024-09-13T18:24:45.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.420+0000 7f0a16802740 10 client.25784 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:24:45.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.420+0000 7f0a16802740 10 client.25784 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-13T18:24:45.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.420+0000 7f0a16802740 20 client.25784 populate_metadata read hostname 'smithi114' 2024-09-13T18:24:45.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.420+0000 7f0a16802740 1 --2- 172.21.15.114:0/915554629 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55fe06e564e0 0x55fe06e588d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:45.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.420+0000 7f0a16802740 4 client.25784 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:24:45.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.420+0000 7f0a16802740 1 -- 172.21.15.114:0/915554629 --> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fe06d911f0 con 0x55fe06e564e0 2024-09-13T18:24:45.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.420+0000 7f0a11881640 1 --2- 172.21.15.114:0/915554629 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55fe06e564e0 0x55fe06e588d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:45.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.421+0000 7f09f57fa640 10 client.25784 ms_handle_connect on v2:172.21.15.114:6840/1956923446 2024-09-13T18:24:45.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.422+0000 7f0a11881640 1 --2- 172.21.15.114:0/915554629 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55fe06e564e0 0x55fe06e588d0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e1c34cd132c6e120 server_cookie=165e4278334338ee in_seq=0 out_seq=0 2024-09-13T18:24:45.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.423+0000 7f09f57fa640 1 -- 172.21.15.114:0/915554629 <== mds.4 v2:172.21.15.114:6840/1956923446 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55fe06d911f0 con 0x55fe06e564e0 2024-09-13T18:24:45.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.423+0000 7f09f57fa640 10 client.25784 handle_command_reply: tid=0 2024-09-13T18:24:45.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.423+0000 7f0a16802740 1 client.25784 shutdown 2024-09-13T18:24:45.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.423+0000 7f0a16802740 1 -- 172.21.15.114:0/915554629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe06e46b30 msgr2=0x55fe06e43960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:45.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.423+0000 7f0a16802740 1 --2- 172.21.15.114:0/915554629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe06e46b30 0x55fe06e43960 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x55fe06e3e510 tx=0x7f0a04009500 comp rx=0 tx=0).stop 2024-09-13T18:24:45.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.423+0000 7f0a16802740 1 -- 172.21.15.114:0/915554629 shutdown_connections 2024-09-13T18:24:45.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.423+0000 7f0a16802740 1 --2- 172.21.15.114:0/915554629 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55fe06e564e0 0x55fe06e588d0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.423+0000 7f0a16802740 1 --2- 172.21.15.114:0/915554629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe06e46b30 0x55fe06e43960 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.423+0000 7f0a16802740 1 --2- 172.21.15.114:0/915554629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe06e3c290 0x55fe06e43420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.423+0000 7f0a16802740 1 --2- 172.21.15.114:0/915554629 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe06e3b970 0x55fe06e46150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.423+0000 7f0a16802740 1 -- 172.21.15.114:0/915554629 >> 172.21.15.114:0/915554629 conn(0x55fe06e37ee0 msgr2=0x55fe06e37230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:45.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.423+0000 7f09f57fa640 0 client.25784 ms_handle_reset on v2:172.21.15.114:6840/1956923446 2024-09-13T18:24:45.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.424+0000 7f0a16802740 1 -- 172.21.15.114:0/915554629 shutdown_connections 2024-09-13T18:24:45.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.424+0000 7f0a16802740 1 -- 172.21.15.114:0/915554629 wait complete. 2024-09-13T18:24:45.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.424+0000 7f0a16802740 20 client.25784 trim_cache size 0 max 16384 2024-09-13T18:24:45.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.448+0000 7f0a16802740 1 Processor -- start 2024-09-13T18:24:45.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.448+0000 7f0a16802740 1 -- start start 2024-09-13T18:24:45.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.449+0000 7f0a16802740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe06e3b970 0x55fe06e3e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:45.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.449+0000 7f0a16802740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe06e3c290 0x55fe06e3ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:45.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.449+0000 7f0a16802740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe06e46b30 0x55fe06e40680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:45.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.449+0000 7f0a16802740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe06d911f0 con 0x55fe06e3b970 2024-09-13T18:24:45.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.449+0000 7f0a16802740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe06dc8b80 con 0x55fe06e46b30 2024-09-13T18:24:45.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.449+0000 7f0a16802740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe06e467a0 con 0x55fe06e3c290 2024-09-13T18:24:45.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.449+0000 7f0a11080640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe06e3c290 0x55fe06e3ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:45.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.449+0000 7f0a11080640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe06e3c290 0x55fe06e3ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:57376/0 (socket says 172.21.15.114:57376) 2024-09-13T18:24:45.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.449+0000 7f0a11080640 1 -- 172.21.15.114:0/3571013737 learned_addr learned my addr 172.21.15.114:0/3571013737 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:45.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.449+0000 7f0a12082640 1 --2- 172.21.15.114:0/3571013737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe06e46b30 0x55fe06e40680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:45.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.450+0000 7f0a11881640 1 --2- 172.21.15.114:0/3571013737 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe06e3b970 0x55fe06e3e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:45.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.450+0000 7f0a11080640 1 -- 172.21.15.114:0/3571013737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe06e46b30 msgr2=0x55fe06e40680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:45.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.450+0000 7f0a11080640 1 --2- 172.21.15.114:0/3571013737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe06e46b30 0x55fe06e40680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.450+0000 7f0a11080640 1 -- 172.21.15.114:0/3571013737 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe06e3b970 msgr2=0x55fe06e3e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:45.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.450+0000 7f0a11080640 1 --2- 172.21.15.114:0/3571013737 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe06e3b970 0x55fe06e3e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.450+0000 7f0a11080640 1 -- 172.21.15.114:0/3571013737 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe06d3f770 con 0x55fe06e3c290 2024-09-13T18:24:45.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.450+0000 7f0a12082640 1 --2- 172.21.15.114:0/3571013737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe06e46b30 0x55fe06e40680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:24:45.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.450+0000 7f0a11080640 1 --2- 172.21.15.114:0/3571013737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe06e3c290 0x55fe06e3ea80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0a08010fd0 tx=0x7f0a08095e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2de29137b3ce12e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:45.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.451+0000 7f0a00e05640 1 -- 172.21.15.114:0/3571013737 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a08010a30 con 0x55fe06e3c290 2024-09-13T18:24:45.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.452+0000 7f0a00e05640 1 -- 172.21.15.114:0/3571013737 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0a08098500 con 0x55fe06e3c290 2024-09-13T18:24:45.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.452+0000 7f0a00e05640 1 -- 172.21.15.114:0/3571013737 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a08010870 con 0x55fe06e3c290 2024-09-13T18:24:45.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.453+0000 7f0a16802740 1 -- 172.21.15.114:0/3571013737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe06e3c290 msgr2=0x55fe06e3ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:45.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.453+0000 7f0a16802740 1 --2- 172.21.15.114:0/3571013737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe06e3c290 0x55fe06e3ea80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0a08010fd0 tx=0x7f0a08095e70 comp rx=0 tx=0).stop 2024-09-13T18:24:45.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.454+0000 7f0a16802740 1 -- 172.21.15.114:0/3571013737 shutdown_connections 2024-09-13T18:24:45.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.454+0000 7f0a16802740 1 --2- 172.21.15.114:0/3571013737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe06e46b30 0x55fe06e40680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.454+0000 7f0a16802740 1 --2- 172.21.15.114:0/3571013737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe06e3c290 0x55fe06e3ea80 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.454+0000 7f0a16802740 1 --2- 172.21.15.114:0/3571013737 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe06e3b970 0x55fe06e3e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.454+0000 7f0a16802740 1 -- 172.21.15.114:0/3571013737 >> 172.21.15.114:0/3571013737 conn(0x55fe06e37ee0 msgr2=0x55fe06e41cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:45.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.455+0000 7f0a16802740 1 -- 172.21.15.114:0/3571013737 shutdown_connections 2024-09-13T18:24:45.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.455+0000 7f0a16802740 1 -- 172.21.15.114:0/3571013737 wait complete. 2024-09-13T18:24:45.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.455+0000 7f0a16802740 1 -- start start 2024-09-13T18:24:45.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.456+0000 7f0a16802740 1 Processor -- start 2024-09-13T18:24:45.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.456+0000 7f0a16802740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe06e3b970 0x55fe06e4da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:45.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.456+0000 7f0a16802740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe06e3c290 0x55fe06e4bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:45.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.456+0000 7f0a16802740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe06e46b30 0x55fe06e4c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:45.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.456+0000 7f0a16802740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe06e4c9b0 con 0x55fe06e3c290 2024-09-13T18:24:45.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.456+0000 7f0a16802740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe06e4cb20 con 0x55fe06e46b30 2024-09-13T18:24:45.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.456+0000 7f0a16802740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe06e4cce0 con 0x55fe06e3b970 2024-09-13T18:24:45.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.456+0000 7f0a16802740 10 client.0 _ll_register_callbacks cb 0x7f0a08096540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:24:45.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.457+0000 7f0a11080640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe06e3c290 0x55fe06e4bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:45.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.457+0000 7f0a11080640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe06e3c290 0x55fe06e4bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51546/0 (socket says 172.21.15.114:51546) 2024-09-13T18:24:45.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.457+0000 7f0a11080640 1 -- 172.21.15.114:0/518640459 learned_addr learned my addr 172.21.15.114:0/518640459 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:45.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.457+0000 7f0a12082640 1 --2- 172.21.15.114:0/518640459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe06e46b30 0x55fe06e4c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:45.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.457+0000 7f0a11881640 1 --2- 172.21.15.114:0/518640459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe06e3b970 0x55fe06e4da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:45.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.457+0000 7f0a11080640 1 -- 172.21.15.114:0/518640459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe06e3b970 msgr2=0x55fe06e4da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:45.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.457+0000 7f0a11080640 1 --2- 172.21.15.114:0/518640459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe06e3b970 0x55fe06e4da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.457+0000 7f0a11080640 1 -- 172.21.15.114:0/518640459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe06e46b30 msgr2=0x55fe06e4c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:45.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.458+0000 7f0a11080640 1 --2- 172.21.15.114:0/518640459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe06e46b30 0x55fe06e4c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.458+0000 7f0a11080640 1 -- 172.21.15.114:0/518640459 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fe06d9d3c0 con 0x55fe06e3c290 2024-09-13T18:24:45.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.458+0000 7f0a12082640 1 --2- 172.21.15.114:0/518640459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe06e46b30 0x55fe06e4c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:24:45.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.458+0000 7f0a11080640 1 --2- 172.21.15.114:0/518640459 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe06e3c290 0x55fe06e4bf00 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f0a08099ae0 tx=0x7f0a08010c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dfffed38b673a3e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:45.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.458+0000 7f09f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:24:45.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.458+0000 7f09f57fa640 1 -- 172.21.15.114:0/518640459 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a08010850 con 0x55fe06e3c290 2024-09-13T18:24:45.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.458+0000 7f09f57fa640 1 -- 172.21.15.114:0/518640459 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f0a080a2150 con 0x55fe06e3c290 2024-09-13T18:24:45.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.458+0000 7f0a16802740 1 -- 172.21.15.114:0/518640459 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0a080cd160 con 0x55fe06e3c290 2024-09-13T18:24:45.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.459+0000 7f09f57fa640 1 -- 172.21.15.114:0/518640459 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a080a2310 con 0x55fe06e3c290 2024-09-13T18:24:45.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.459+0000 7f0a16802740 10 client.16470 fetch_fsmap learned FSMap version 32 2024-09-13T18:24:45.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.459+0000 7f0a16802740 1 -- 172.21.15.114:0/518640459 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55fe06d66620 con 0x55fe06e3c290 2024-09-13T18:24:45.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.459+0000 7f09f57fa640 1 -- 172.21.15.114:0/518640459 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0a080cd160 con 0x55fe06e3c290 2024-09-13T18:24:45.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.460+0000 7f09f57fa640 1 -- 172.21.15.114:0/518640459 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f0a080990c0 con 0x55fe06e3c290 2024-09-13T18:24:45.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.460+0000 7f0a16802740 10 client.16470 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:24:45.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.460+0000 7f0a16802740 10 client.16470 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-13T18:24:45.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.460+0000 7f0a16802740 20 client.16470 populate_metadata read hostname 'smithi114' 2024-09-13T18:24:45.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.460+0000 7f0a16802740 1 --2- 172.21.15.114:0/518640459 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55fe06eada50 0x55fe06eafe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:45.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.460+0000 7f0a16802740 4 client.16470 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:24:45.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.460+0000 7f0a16802740 1 -- 172.21.15.114:0/518640459 --> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fe06a924e0 con 0x55fe06eada50 2024-09-13T18:24:45.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.460+0000 7f0a11881640 1 --2- 172.21.15.114:0/518640459 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55fe06eada50 0x55fe06eafe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:45.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.461+0000 7f0a11881640 1 --2- 172.21.15.114:0/518640459 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55fe06eada50 0x55fe06eafe40 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1e182f8893513920 server_cookie=7ff56f5f94f21d40 in_seq=0 out_seq=0 2024-09-13T18:24:45.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.461+0000 7f09f57fa640 10 client.16470 ms_handle_connect on v2:172.21.15.114:6840/1956923446 2024-09-13T18:24:45.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.461+0000 7f09f57fa640 1 -- 172.21.15.114:0/518640459 <== mds.4 v2:172.21.15.114:6840/1956923446 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fe06a924e0 con 0x55fe06eada50 2024-09-13T18:24:45.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.461+0000 7f09f57fa640 10 client.16470 handle_command_reply: tid=0 2024-09-13T18:24:45.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.461+0000 7f0a16802740 1 client.16470 shutdown 2024-09-13T18:24:45.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.462+0000 7f0a16802740 1 -- 172.21.15.114:0/518640459 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe06e3c290 msgr2=0x55fe06e4bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:45.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.462+0000 7f0a16802740 1 --2- 172.21.15.114:0/518640459 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe06e3c290 0x55fe06e4bf00 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f0a08099ae0 tx=0x7f0a08010c60 comp rx=0 tx=0).stop 2024-09-13T18:24:45.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.462+0000 7f0a16802740 1 -- 172.21.15.114:0/518640459 shutdown_connections 2024-09-13T18:24:45.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.462+0000 7f0a16802740 1 --2- 172.21.15.114:0/518640459 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55fe06eada50 0x55fe06eafe40 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.462+0000 7f0a16802740 1 --2- 172.21.15.114:0/518640459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe06e46b30 0x55fe06e4c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.462+0000 7f0a16802740 1 --2- 172.21.15.114:0/518640459 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe06e3c290 0x55fe06e4bf00 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.462+0000 7f09f57fa640 0 client.16470 ms_handle_reset on v2:172.21.15.114:6840/1956923446 2024-09-13T18:24:45.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.462+0000 7f0a16802740 1 --2- 172.21.15.114:0/518640459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe06e3b970 0x55fe06e4da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.462+0000 7f0a16802740 1 -- 172.21.15.114:0/518640459 >> 172.21.15.114:0/518640459 conn(0x55fe06e37ee0 msgr2=0x55fe06e41b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:45.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.463+0000 7f0a16802740 1 -- 172.21.15.114:0/518640459 shutdown_connections 2024-09-13T18:24:45.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.463+0000 7f0a16802740 1 -- 172.21.15.114:0/518640459 wait complete. 2024-09-13T18:24:45.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.463+0000 7f0a16802740 20 client.16470 trim_cache size 0 max 16384 2024-09-13T18:24:45.468 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:24:45.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.464+0000 7f0a13ae5640 1 -- 172.21.15.114:0/24266263 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f09f80a1350 msgr2=0x7f09f80a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:45.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.464+0000 7f0a13ae5640 1 --2- 172.21.15.114:0/24266263 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f09f80a1350 0x7f09f80a3810 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x55fe06e39260 tx=0x7f0a040091e0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.464+0000 7f0a13ae5640 1 -- 172.21.15.114:0/24266263 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a0c107890 msgr2=0x7f0a0c1a54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:45.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.465+0000 7f0a13ae5640 1 --2- 172.21.15.114:0/24266263 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a0c107890 0x7f0a0c1a54c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f0a0800e9f0 tx=0x7f0a0800eeb0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.465+0000 7f0a13ae5640 1 -- 172.21.15.114:0/24266263 shutdown_connections 2024-09-13T18:24:45.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.465+0000 7f0a13ae5640 1 --2- 172.21.15.114:0/24266263 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f09f80a1350 0x7f09f80a3810 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.465+0000 7f0a13ae5640 1 --2- 172.21.15.114:0/24266263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a0c119400 0x7f0a0c1a98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.465+0000 7f0a13ae5640 1 --2- 172.21.15.114:0/24266263 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a0c107890 0x7f0a0c1a54c0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.465+0000 7f0a13ae5640 1 --2- 172.21.15.114:0/24266263 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a0c1074b0 0x7f0a0c1a4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.465+0000 7f0a13ae5640 1 -- 172.21.15.114:0/24266263 >> 172.21.15.114:0/24266263 conn(0x7f0a0c06af70 msgr2=0x7f0a0c119d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:45.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.465+0000 7f0a13ae5640 1 -- 172.21.15.114:0/24266263 shutdown_connections 2024-09-13T18:24:45.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.466+0000 7f0a13ae5640 1 -- 172.21.15.114:0/24266263 wait complete. 2024-09-13T18:24:45.487 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:45 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/676524118' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:24:45.488 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-09-13T18:24:45.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:45 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/676524118' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:24:45.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.674+0000 7f6a3dec3640 1 -- 172.21.15.114:0/3274757250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a38106c40 msgr2=0x7f6a38113150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:45.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.674+0000 7f6a3dec3640 1 --2- 172.21.15.114:0/3274757250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a38106c40 0x7f6a38113150 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6a30009cd0 tx=0x7f6a3002a8f0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.674+0000 7f6a3dec3640 1 -- 172.21.15.114:0/3274757250 shutdown_connections 2024-09-13T18:24:45.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.674+0000 7f6a3dec3640 1 --2- 172.21.15.114:0/3274757250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a38106c40 0x7f6a38113150 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.674+0000 7f6a3dec3640 1 --2- 172.21.15.114:0/3274757250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a381062e0 0x7f6a38106700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.674+0000 7f6a3dec3640 1 --2- 172.21.15.114:0/3274757250 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6a3810c320 0x7f6a3810c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.674+0000 7f6a3dec3640 1 -- 172.21.15.114:0/3274757250 >> 172.21.15.114:0/3274757250 conn(0x7f6a38075c50 msgr2=0x7f6a38078070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:45.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.675+0000 7f6a3dec3640 1 -- 172.21.15.114:0/3274757250 shutdown_connections 2024-09-13T18:24:45.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.676+0000 7f6a3dec3640 1 -- 172.21.15.114:0/3274757250 wait complete. 2024-09-13T18:24:45.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.677+0000 7f6a3dec3640 1 Processor -- start 2024-09-13T18:24:45.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.677+0000 7f6a3dec3640 1 -- start start 2024-09-13T18:24:45.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.677+0000 7f6a3dec3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a381062e0 0x7f6a3819c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:45.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.677+0000 7f6a3dec3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a38106c40 0x7f6a3819cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:45.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.678+0000 7f6a3dec3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6a3810c320 0x7f6a381a1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:45.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.678+0000 7f6a3dec3640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a381a1600 con 0x7f6a3810c320 2024-09-13T18:24:45.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.678+0000 7f6a3dec3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a381a1770 con 0x7f6a381062e0 2024-09-13T18:24:45.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.678+0000 7f6a3dec3640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a381a1930 con 0x7f6a38106c40 2024-09-13T18:24:45.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.678+0000 7f6a3d6c2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6a3810c320 0x7f6a381a1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:45.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.678+0000 7f6a37fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a38106c40 0x7f6a3819cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:45.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.678+0000 7f6a3d6c2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6a3810c320 0x7f6a381a1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51566/0 (socket says 172.21.15.114:51566) 2024-09-13T18:24:45.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.678+0000 7f6a3d6c2640 1 -- 172.21.15.114:0/3570602045 learned_addr learned my addr 172.21.15.114:0/3570602045 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:45.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.678+0000 7f6a3cec1640 1 --2- 172.21.15.114:0/3570602045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a381062e0 0x7f6a3819c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:45.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.678+0000 7f6a3d6c2640 1 -- 172.21.15.114:0/3570602045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a38106c40 msgr2=0x7f6a3819cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:45.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.678+0000 7f6a3d6c2640 1 --2- 172.21.15.114:0/3570602045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a38106c40 0x7f6a3819cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.678+0000 7f6a3d6c2640 1 -- 172.21.15.114:0/3570602045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a381062e0 msgr2=0x7f6a3819c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:45.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.678+0000 7f6a3d6c2640 1 --2- 172.21.15.114:0/3570602045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a381062e0 0x7f6a3819c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.678+0000 7f6a3d6c2640 1 -- 172.21.15.114:0/3570602045 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a30009900 con 0x7f6a3810c320 2024-09-13T18:24:45.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.678+0000 7f6a3cec1640 1 --2- 172.21.15.114:0/3570602045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a381062e0 0x7f6a3819c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:24:45.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.679+0000 7f6a3d6c2640 1 --2- 172.21.15.114:0/3570602045 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6a3810c320 0x7f6a381a1000 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f6a30005e30 tx=0x7f6a300093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:45.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.680+0000 7f6a35ffb640 1 -- 172.21.15.114:0/3570602045 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a30004040 con 0x7f6a3810c320 2024-09-13T18:24:45.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.680+0000 7f6a35ffb640 1 -- 172.21.15.114:0/3570602045 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6a30004630 con 0x7f6a3810c320 2024-09-13T18:24:45.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.680+0000 7f6a3dec3640 1 -- 172.21.15.114:0/3570602045 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a381a1bb0 con 0x7f6a3810c320 2024-09-13T18:24:45.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.680+0000 7f6a3dec3640 1 -- 172.21.15.114:0/3570602045 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a381a9490 con 0x7f6a3810c320 2024-09-13T18:24:45.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.681+0000 7f6a35ffb640 1 -- 172.21.15.114:0/3570602045 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a30004040 con 0x7f6a3810c320 2024-09-13T18:24:45.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.684+0000 7f6a41e4a740 1 Processor -- start 2024-09-13T18:24:45.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.685+0000 7f6a35ffb640 1 -- 172.21.15.114:0/3570602045 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f6a30042070 con 0x7f6a3810c320 2024-09-13T18:24:45.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.685+0000 7f6a41e4a740 1 -- start start 2024-09-13T18:24:45.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.685+0000 7f6a41e4a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637b48e0970 0x5637b48e0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:45.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.686+0000 7f6a41e4a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637b48e1290 0x5637b48eb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:45.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.686+0000 7f6a41e4a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5637b48ebb30 0x5637b48edf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:45.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.686+0000 7f6a41e4a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637b484acc0 con 0x5637b48ebb30 2024-09-13T18:24:45.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.686+0000 7f6a41e4a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637b4848450 con 0x5637b48e0970 2024-09-13T18:24:45.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.686+0000 7f6a41e4a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637b4843d40 con 0x5637b48e1290 2024-09-13T18:24:45.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.686+0000 7f6a37fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637b48e0970 0x5637b48e0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:45.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.686+0000 7f6a35ffb640 1 --2- 172.21.15.114:0/3570602045 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f6a280a12c0 0x7f6a280a3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:45.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.686+0000 7f6a37fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637b48e0970 0x5637b48e0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:39192/0 (socket says 172.21.15.114:39192) 2024-09-13T18:24:45.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.686+0000 7f6a37fff640 1 -- 172.21.15.114:0/1752994291 learned_addr learned my addr 172.21.15.114:0/1752994291 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:45.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.686+0000 7f6a3cec1640 1 --2- 172.21.15.114:0/1752994291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637b48e1290 0x5637b48eb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:45.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.686+0000 7f6a35ffb640 1 -- 172.21.15.114:0/3570602045 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f6a300ed5a0 con 0x7f6a3810c320 2024-09-13T18:24:45.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.686+0000 7f6a37fff640 1 -- 172.21.15.114:0/1752994291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637b48e1290 msgr2=0x5637b48eb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:45.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.686+0000 7f6a37fff640 1 --2- 172.21.15.114:0/1752994291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637b48e1290 0x5637b48eb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.686+0000 7f6a37fff640 1 -- 172.21.15.114:0/1752994291 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5637b48ebb30 msgr2=0x5637b48edf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:45.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.686+0000 7f6a37fff640 1 --2- 172.21.15.114:0/1752994291 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5637b48ebb30 0x5637b48edf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.686+0000 7f6a37fff640 1 -- 172.21.15.114:0/1752994291 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637b480b620 con 0x5637b48e0970 2024-09-13T18:24:45.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.686+0000 7f6a3d6c2640 1 --2- 172.21.15.114:0/3570602045 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f6a280a12c0 0x7f6a280a3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:45.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.687+0000 7f6a3d6c2640 1 --2- 172.21.15.114:0/3570602045 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f6a280a12c0 0x7f6a280a3780 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6a300eab60 tx=0x7f6a3003e040 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:45.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.687+0000 7f6a37fff640 1 --2- 172.21.15.114:0/1752994291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637b48e0970 0x5637b48e0d50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6a2400bd50 tx=0x7f6a240127c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6629fb4103b694d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:45.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.687+0000 7f6a22ffd640 1 -- 172.21.15.114:0/1752994291 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a240120d0 con 0x5637b48e0970 2024-09-13T18:24:45.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.688+0000 7f6a22ffd640 1 -- 172.21.15.114:0/1752994291 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6a24012230 con 0x5637b48e0970 2024-09-13T18:24:45.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.688+0000 7f6a22ffd640 1 -- 172.21.15.114:0/1752994291 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2400f740 con 0x5637b48e0970 2024-09-13T18:24:45.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.688+0000 7f6a41e4a740 1 -- 172.21.15.114:0/1752994291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637b48e0970 msgr2=0x5637b48e0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:45.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.688+0000 7f6a41e4a740 1 --2- 172.21.15.114:0/1752994291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637b48e0970 0x5637b48e0d50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6a2400bd50 tx=0x7f6a240127c0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.689+0000 7f6a41e4a740 1 -- 172.21.15.114:0/1752994291 shutdown_connections 2024-09-13T18:24:45.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.689+0000 7f6a41e4a740 1 --2- 172.21.15.114:0/1752994291 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5637b48ebb30 0x5637b48edf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.689+0000 7f6a41e4a740 1 --2- 172.21.15.114:0/1752994291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637b48e1290 0x5637b48eb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.689+0000 7f6a41e4a740 1 --2- 172.21.15.114:0/1752994291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637b48e0970 0x5637b48e0d50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.689+0000 7f6a41e4a740 1 -- 172.21.15.114:0/1752994291 >> 172.21.15.114:0/1752994291 conn(0x5637b48dcee0 msgr2=0x5637b48dd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:45.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.689+0000 7f6a41e4a740 1 -- 172.21.15.114:0/1752994291 shutdown_connections 2024-09-13T18:24:45.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.689+0000 7f6a41e4a740 1 -- 172.21.15.114:0/1752994291 wait complete. 2024-09-13T18:24:45.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.689+0000 7f6a41e4a740 1 -- start start 2024-09-13T18:24:45.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.689+0000 7f6a41e4a740 1 Processor -- start 2024-09-13T18:24:45.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.690+0000 7f6a41e4a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637b48e0970 0x5637b48eb250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:45.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.690+0000 7f6a41e4a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637b48e1290 0x5637b48e8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:45.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.690+0000 7f6a37fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637b48e0970 0x5637b48eb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:45.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.690+0000 7f6a37fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637b48e0970 0x5637b48eb250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:39204/0 (socket says 172.21.15.114:39204) 2024-09-13T18:24:45.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.690+0000 7f6a37fff640 1 -- 172.21.15.114:0/3319791094 learned_addr learned my addr 172.21.15.114:0/3319791094 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:45.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.690+0000 7f6a41e4a740 1 --2- 172.21.15.114:0/3319791094 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5637b48ebb30 0x5637b48e8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:45.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.691+0000 7f6a41e4a740 1 -- 172.21.15.114:0/3319791094 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637b4802180 con 0x5637b48ebb30 2024-09-13T18:24:45.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.691+0000 7f6a3cec1640 1 --2- 172.21.15.114:0/3319791094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637b48e1290 0x5637b48e8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:45.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.691+0000 7f6a41e4a740 1 -- 172.21.15.114:0/3319791094 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637b451c430 con 0x5637b48e0970 2024-09-13T18:24:45.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.691+0000 7f6a41e4a740 1 -- 172.21.15.114:0/3319791094 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637b47c75e0 con 0x5637b48e1290 2024-09-13T18:24:45.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.691+0000 7f6a37fff640 1 --2- 172.21.15.114:0/3319791094 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5637b48ebb30 0x5637b48e8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:45.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.691+0000 7f6a37fff640 1 -- 172.21.15.114:0/3319791094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637b48e1290 msgr2=0x5637b48e8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:45.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.691+0000 7f6a37fff640 1 --2- 172.21.15.114:0/3319791094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637b48e1290 0x5637b48e8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.691+0000 7f6a37fff640 1 -- 172.21.15.114:0/3319791094 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5637b48ebb30 msgr2=0x5637b48e8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:45.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.691+0000 7f6a37fff640 1 --2- 172.21.15.114:0/3319791094 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5637b48ebb30 0x5637b48e8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.691+0000 7f6a37fff640 1 -- 172.21.15.114:0/3319791094 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a2400ba30 con 0x5637b48e0970 2024-09-13T18:24:45.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.691+0000 7f6a3cec1640 1 --2- 172.21.15.114:0/3319791094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637b48e1290 0x5637b48e8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:24:45.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.691+0000 7f6a41e4a740 10 client.0 _ll_register_callbacks cb 0x5637b4833aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:24:45.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.691+0000 7f6a37fff640 1 --2- 172.21.15.114:0/3319791094 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637b48e0970 0x5637b48eb250 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6a240123c0 tx=0x7f6a240056d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acb76f44caf8f5a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:45.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.691+0000 7f6a17fff640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:24:45.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.691+0000 7f6a41e4a740 1 -- 172.21.15.114:0/3319791094 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5637b486db80 con 0x5637b48e0970 2024-09-13T18:24:45.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.692+0000 7f6a17fff640 1 -- 172.21.15.114:0/3319791094 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2401b470 con 0x5637b48e0970 2024-09-13T18:24:45.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.692+0000 7f6a17fff640 1 -- 172.21.15.114:0/3319791094 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f6a2401b5d0 con 0x5637b48e0970 2024-09-13T18:24:45.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.692+0000 7f6a17fff640 1 -- 172.21.15.114:0/3319791094 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a2401b7d0 con 0x5637b48e0970 2024-09-13T18:24:45.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.692+0000 7f6a41e4a740 10 client.25799 fetch_fsmap learned FSMap version 32 2024-09-13T18:24:45.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.692+0000 7f6a41e4a740 1 -- 172.21.15.114:0/3319791094 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({fsmap=32}) v3 -- 0x5637b48423c0 con 0x5637b48e0970 2024-09-13T18:24:45.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.692+0000 7f6a17fff640 1 -- 172.21.15.114:0/3319791094 <== mon.2 v2:172.21.15.134:3300/0 4 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f6a24026070 con 0x5637b48e0970 2024-09-13T18:24:45.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.692+0000 7f6a41e4a740 10 client.25799 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:24:45.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.692+0000 7f6a41e4a740 10 client.25799 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-13T18:24:45.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.692+0000 7f6a41e4a740 20 client.25799 populate_metadata read hostname 'smithi114' 2024-09-13T18:24:45.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.692+0000 7f6a41e4a740 1 --2- 172.21.15.114:0/3319791094 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x5637b48fb340 0x5637b48fd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:45.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.692+0000 7f6a41e4a740 4 client.25799 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:24:45.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.692+0000 7f6a41e4a740 1 -- 172.21.15.114:0/3319791094 --> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5637b48361f0 con 0x5637b48fb340 2024-09-13T18:24:45.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.693+0000 7f6a3cec1640 1 --2- 172.21.15.114:0/3319791094 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x5637b48fb340 0x5637b48fd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:45.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.693+0000 7f6a17fff640 10 client.25799 ms_handle_connect on v2:172.21.15.134:6834/1150003275 2024-09-13T18:24:45.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.693+0000 7f6a3cec1640 1 --2- 172.21.15.114:0/3319791094 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x5637b48fb340 0x5637b48fd770 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=95f9b49ca19cb01e server_cookie=c24013b8f0a07492 in_seq=0 out_seq=0 2024-09-13T18:24:45.703 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:45 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/676524118' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:24:45.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.694+0000 7f6a17fff640 1 -- 172.21.15.114:0/3319791094 <== mds.3 v2:172.21.15.134:6834/1150003275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5637b48361f0 con 0x5637b48fb340 2024-09-13T18:24:45.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.694+0000 7f6a17fff640 10 client.25799 handle_command_reply: tid=0 2024-09-13T18:24:45.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.694+0000 7f6a41e4a740 1 client.25799 shutdown 2024-09-13T18:24:45.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.695+0000 7f6a41e4a740 1 -- 172.21.15.114:0/3319791094 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637b48e0970 msgr2=0x5637b48eb250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:45.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.695+0000 7f6a41e4a740 1 --2- 172.21.15.114:0/3319791094 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637b48e0970 0x5637b48eb250 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6a240123c0 tx=0x7f6a240056d0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.695+0000 7f6a41e4a740 1 -- 172.21.15.114:0/3319791094 shutdown_connections 2024-09-13T18:24:45.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.695+0000 7f6a41e4a740 1 --2- 172.21.15.114:0/3319791094 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x5637b48fb340 0x5637b48fd770 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.695+0000 7f6a41e4a740 1 --2- 172.21.15.114:0/3319791094 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5637b48ebb30 0x5637b48e8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.695+0000 7f6a41e4a740 1 --2- 172.21.15.114:0/3319791094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637b48e1290 0x5637b48e8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.695+0000 7f6a17fff640 0 client.25799 ms_handle_reset on v2:172.21.15.134:6834/1150003275 2024-09-13T18:24:45.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.695+0000 7f6a41e4a740 1 --2- 172.21.15.114:0/3319791094 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637b48e0970 0x5637b48eb250 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.695+0000 7f6a41e4a740 1 -- 172.21.15.114:0/3319791094 >> 172.21.15.114:0/3319791094 conn(0x5637b48dcee0 msgr2=0x5637b48dc230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:45.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.695+0000 7f6a41e4a740 1 -- 172.21.15.114:0/3319791094 shutdown_connections 2024-09-13T18:24:45.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.695+0000 7f6a41e4a740 1 -- 172.21.15.114:0/3319791094 wait complete. 2024-09-13T18:24:45.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.695+0000 7f6a41e4a740 20 client.25799 trim_cache size 0 max 16384 2024-09-13T18:24:45.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.716+0000 7f6a41e4a740 1 Processor -- start 2024-09-13T18:24:45.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.717+0000 7f6a41e4a740 1 -- start start 2024-09-13T18:24:45.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.717+0000 7f6a41e4a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5637b48e0970 0x5637b48f2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:45.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.717+0000 7f6a41e4a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637b48e1290 0x5637b48f2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:45.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.717+0000 7f6a41e4a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637b48ebb30 0x5637b48e84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:45.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.717+0000 7f6a41e4a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637b48e0190 con 0x5637b48e0970 2024-09-13T18:24:45.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.717+0000 7f6a41e4a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637b48e9500 con 0x5637b48ebb30 2024-09-13T18:24:45.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.717+0000 7f6a41e4a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637b48f82c0 con 0x5637b48e1290 2024-09-13T18:24:45.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.718+0000 7f6a37fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637b48ebb30 0x5637b48e84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:45.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.718+0000 7f6a37fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637b48ebb30 0x5637b48e84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:39214/0 (socket says 172.21.15.114:39214) 2024-09-13T18:24:45.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.718+0000 7f6a37fff640 1 -- 172.21.15.114:0/337647272 learned_addr learned my addr 172.21.15.114:0/337647272 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:45.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.718+0000 7f6a37fff640 1 --2- 172.21.15.114:0/337647272 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5637b48e0970 0x5637b48f2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:45.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.718+0000 7f6a37fff640 1 -- 172.21.15.114:0/337647272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637b48e1290 msgr2=0x5637b48f2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:45.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.718+0000 7f6a37fff640 1 --2- 172.21.15.114:0/337647272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637b48e1290 0x5637b48f2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.718+0000 7f6a37fff640 1 -- 172.21.15.114:0/337647272 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5637b48e0970 msgr2=0x5637b48f2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:45.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.718+0000 7f6a37fff640 1 --2- 172.21.15.114:0/337647272 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5637b48e0970 0x5637b48f2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.718+0000 7f6a37fff640 1 -- 172.21.15.114:0/337647272 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637b47e4770 con 0x5637b48ebb30 2024-09-13T18:24:45.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.718+0000 7f6a37fff640 1 --2- 172.21.15.114:0/337647272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637b48ebb30 0x5637b48e84e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6a2400b4e0 tx=0x7f6a24014070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d395523b9241a138 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:45.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.718+0000 7f6a237fe640 1 -- 172.21.15.114:0/337647272 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a24037040 con 0x5637b48ebb30 2024-09-13T18:24:45.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.719+0000 7f6a237fe640 1 -- 172.21.15.114:0/337647272 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5637b486db80 con 0x5637b48ebb30 2024-09-13T18:24:45.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.719+0000 7f6a237fe640 1 -- 172.21.15.114:0/337647272 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a24019b90 con 0x5637b48ebb30 2024-09-13T18:24:45.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.720+0000 7f6a41e4a740 1 -- 172.21.15.114:0/337647272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637b48ebb30 msgr2=0x5637b48e84e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:45.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.720+0000 7f6a41e4a740 1 --2- 172.21.15.114:0/337647272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637b48ebb30 0x5637b48e84e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6a2400b4e0 tx=0x7f6a24014070 comp rx=0 tx=0).stop 2024-09-13T18:24:45.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.720+0000 7f6a41e4a740 1 -- 172.21.15.114:0/337647272 shutdown_connections 2024-09-13T18:24:45.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.720+0000 7f6a41e4a740 1 --2- 172.21.15.114:0/337647272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637b48ebb30 0x5637b48e84e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.720+0000 7f6a41e4a740 1 --2- 172.21.15.114:0/337647272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637b48e1290 0x5637b48f2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.720+0000 7f6a41e4a740 1 --2- 172.21.15.114:0/337647272 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5637b48e0970 0x5637b48f2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.720+0000 7f6a41e4a740 1 -- 172.21.15.114:0/337647272 >> 172.21.15.114:0/337647272 conn(0x5637b48dcee0 msgr2=0x5637b48f72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:45.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.720+0000 7f6a41e4a740 1 -- 172.21.15.114:0/337647272 shutdown_connections 2024-09-13T18:24:45.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.721+0000 7f6a41e4a740 1 -- 172.21.15.114:0/337647272 wait complete. 2024-09-13T18:24:45.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.721+0000 7f6a41e4a740 1 -- start start 2024-09-13T18:24:45.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.721+0000 7f6a41e4a740 1 Processor -- start 2024-09-13T18:24:45.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.722+0000 7f6a41e4a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637b48e0970 0x5637b48f2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:45.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.722+0000 7f6a41e4a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5637b48e1290 0x5637b48eef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:45.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.722+0000 7f6a41e4a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637b48ebb30 0x5637b48ef470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:45.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.722+0000 7f6a41e4a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637b48efb70 con 0x5637b48e1290 2024-09-13T18:24:45.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.722+0000 7f6a41e4a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637b48efce0 con 0x5637b48e0970 2024-09-13T18:24:45.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.722+0000 7f6a41e4a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637b4951890 con 0x5637b48ebb30 2024-09-13T18:24:45.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.722+0000 7f6a41e4a740 10 client.0 _ll_register_callbacks cb 0x7f6a2400b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:24:45.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.722+0000 7f6a3cec1640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5637b48e1290 0x5637b48eef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:45.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.722+0000 7f6a3cec1640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5637b48e1290 0x5637b48eef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51614/0 (socket says 172.21.15.114:51614) 2024-09-13T18:24:45.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.722+0000 7f6a3cec1640 1 -- 172.21.15.114:0/2082645560 learned_addr learned my addr 172.21.15.114:0/2082645560 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:45.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.722+0000 7f6a37fff640 1 --2- 172.21.15.114:0/2082645560 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637b48e0970 0x5637b48f2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:45.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.722+0000 7f6a37fff640 1 --2- 172.21.15.114:0/2082645560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637b48ebb30 0x5637b48ef470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:45.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.723+0000 7f6a3cec1640 1 -- 172.21.15.114:0/2082645560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637b48ebb30 msgr2=0x5637b48ef470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:45.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.723+0000 7f6a3cec1640 1 --2- 172.21.15.114:0/2082645560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637b48ebb30 0x5637b48ef470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.723+0000 7f6a3cec1640 1 -- 172.21.15.114:0/2082645560 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637b48e0970 msgr2=0x5637b48f2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:45.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.723+0000 7f6a3cec1640 1 --2- 172.21.15.114:0/2082645560 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637b48e0970 0x5637b48f2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.723+0000 7f6a3cec1640 1 -- 172.21.15.114:0/2082645560 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5637b48423c0 con 0x5637b48e1290 2024-09-13T18:24:45.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.723+0000 7f6a37fff640 1 --2- 172.21.15.114:0/2082645560 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637b48e0970 0x5637b48f2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:24:45.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.723+0000 7f6a3cec1640 1 --2- 172.21.15.114:0/2082645560 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5637b48e1290 0x5637b48eef30 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c005d80 tx=0x7f6a2c005410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b7c048e082eaaf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:45.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.723+0000 7f6a17fff640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:24:45.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.723+0000 7f6a17fff640 1 -- 172.21.15.114:0/2082645560 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c018040 con 0x5637b48e1290 2024-09-13T18:24:45.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.723+0000 7f6a17fff640 1 -- 172.21.15.114:0/2082645560 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f6a2c004d80 con 0x5637b48e1290 2024-09-13T18:24:45.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.723+0000 7f6a41e4a740 1 -- 172.21.15.114:0/2082645560 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6a24012230 con 0x5637b48e1290 2024-09-13T18:24:45.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.724+0000 7f6a17fff640 1 -- 172.21.15.114:0/2082645560 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a2c004020 con 0x5637b48e1290 2024-09-13T18:24:45.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.724+0000 7f6a41e4a740 10 client.16485 fetch_fsmap learned FSMap version 32 2024-09-13T18:24:45.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.724+0000 7f6a41e4a740 1 -- 172.21.15.114:0/2082645560 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5637b4809e80 con 0x5637b48e1290 2024-09-13T18:24:45.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.724+0000 7f6a17fff640 1 -- 172.21.15.114:0/2082645560 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6a24012230 con 0x5637b48e1290 2024-09-13T18:24:45.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.725+0000 7f6a17fff640 1 -- 172.21.15.114:0/2082645560 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f6a2c0035f0 con 0x5637b48e1290 2024-09-13T18:24:45.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.725+0000 7f6a41e4a740 10 client.16485 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:24:45.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.725+0000 7f6a41e4a740 10 client.16485 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-13T18:24:45.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.725+0000 7f6a41e4a740 20 client.16485 populate_metadata read hostname 'smithi114' 2024-09-13T18:24:45.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.725+0000 7f6a41e4a740 1 --2- 172.21.15.114:0/2082645560 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x5637b4952ee0 0x5637b4955300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:45.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.725+0000 7f6a41e4a740 4 client.16485 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:24:45.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.725+0000 7f6a41e4a740 1 -- 172.21.15.114:0/2082645560 --> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5637b45374e0 con 0x5637b4952ee0 2024-09-13T18:24:45.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.725+0000 7f6a37fff640 1 --2- 172.21.15.114:0/2082645560 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x5637b4952ee0 0x5637b4955300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:45.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.726+0000 7f6a17fff640 10 client.16485 ms_handle_connect on v2:172.21.15.134:6834/1150003275 2024-09-13T18:24:45.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.726+0000 7f6a37fff640 1 --2- 172.21.15.114:0/2082645560 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x5637b4952ee0 0x5637b4955300 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c8d1298a94bb4003 server_cookie=74d15a58e5c168cc in_seq=0 out_seq=0 2024-09-13T18:24:45.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.727+0000 7f6a17fff640 1 -- 172.21.15.114:0/2082645560 <== mds.3 v2:172.21.15.134:6834/1150003275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5637b45374e0 con 0x5637b4952ee0 2024-09-13T18:24:45.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.727+0000 7f6a17fff640 10 client.16485 handle_command_reply: tid=0 2024-09-13T18:24:45.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.727+0000 7f6a41e4a740 1 client.16485 shutdown 2024-09-13T18:24:45.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.727+0000 7f6a41e4a740 1 -- 172.21.15.114:0/2082645560 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5637b48e1290 msgr2=0x5637b48eef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:45.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.727+0000 7f6a41e4a740 1 --2- 172.21.15.114:0/2082645560 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5637b48e1290 0x5637b48eef30 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c005d80 tx=0x7f6a2c005410 comp rx=0 tx=0).stop 2024-09-13T18:24:45.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.727+0000 7f6a41e4a740 1 -- 172.21.15.114:0/2082645560 shutdown_connections 2024-09-13T18:24:45.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.727+0000 7f6a41e4a740 1 --2- 172.21.15.114:0/2082645560 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x5637b4952ee0 0x5637b4955300 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.727+0000 7f6a41e4a740 1 --2- 172.21.15.114:0/2082645560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637b48ebb30 0x5637b48ef470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.727+0000 7f6a41e4a740 1 --2- 172.21.15.114:0/2082645560 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5637b48e1290 0x5637b48eef30 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.727+0000 7f6a17fff640 0 client.16485 ms_handle_reset on v2:172.21.15.134:6834/1150003275 2024-09-13T18:24:45.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.727+0000 7f6a41e4a740 1 --2- 172.21.15.114:0/2082645560 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637b48e0970 0x5637b48f2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.727+0000 7f6a41e4a740 1 -- 172.21.15.114:0/2082645560 >> 172.21.15.114:0/2082645560 conn(0x5637b48dcee0 msgr2=0x5637b48e6d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:45.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.728+0000 7f6a41e4a740 1 -- 172.21.15.114:0/2082645560 shutdown_connections 2024-09-13T18:24:45.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.728+0000 7f6a41e4a740 1 -- 172.21.15.114:0/2082645560 wait complete. 2024-09-13T18:24:45.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.728+0000 7f6a41e4a740 20 client.16485 trim_cache size 0 max 16384 2024-09-13T18:24:45.740 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:24:45.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.729+0000 7f6a3dec3640 1 -- 172.21.15.114:0/3570602045 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f6a280a12c0 msgr2=0x7f6a280a3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:45.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.729+0000 7f6a3dec3640 1 --2- 172.21.15.114:0/3570602045 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f6a280a12c0 0x7f6a280a3780 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6a300eab60 tx=0x7f6a3003e040 comp rx=0 tx=0).stop 2024-09-13T18:24:45.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.730+0000 7f6a3dec3640 1 -- 172.21.15.114:0/3570602045 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6a3810c320 msgr2=0x7f6a381a1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:45.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.730+0000 7f6a3dec3640 1 --2- 172.21.15.114:0/3570602045 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6a3810c320 0x7f6a381a1000 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f6a30005e30 tx=0x7f6a300093d0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.730+0000 7f6a3dec3640 1 -- 172.21.15.114:0/3570602045 shutdown_connections 2024-09-13T18:24:45.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.730+0000 7f6a3dec3640 1 --2- 172.21.15.114:0/3570602045 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f6a280a12c0 0x7f6a280a3780 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.730+0000 7f6a3dec3640 1 --2- 172.21.15.114:0/3570602045 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6a3810c320 0x7f6a381a1000 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.730+0000 7f6a3dec3640 1 --2- 172.21.15.114:0/3570602045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a38106c40 0x7f6a3819cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.730+0000 7f6a3dec3640 1 --2- 172.21.15.114:0/3570602045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a381062e0 0x7f6a3819c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.730+0000 7f6a3dec3640 1 -- 172.21.15.114:0/3570602045 >> 172.21.15.114:0/3570602045 conn(0x7f6a38075c50 msgr2=0x7f6a38077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:45.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.730+0000 7f6a3dec3640 1 -- 172.21.15.114:0/3570602045 shutdown_connections 2024-09-13T18:24:45.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.731+0000 7f6a3dec3640 1 -- 172.21.15.114:0/3570602045 wait complete. 2024-09-13T18:24:45.752 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-09-13T18:24:45.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.929+0000 7f6626a41640 1 -- 172.21.15.114:0/2935816257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6620101800 msgr2=0x7f6620109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:45.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.929+0000 7f6626a41640 1 --2- 172.21.15.114:0/2935816257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6620101800 0x7f6620109ce0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6614009900 tx=0x7f661402a480 comp rx=0 tx=0).stop 2024-09-13T18:24:45.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.929+0000 7f6626a41640 1 -- 172.21.15.114:0/2935816257 shutdown_connections 2024-09-13T18:24:45.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.929+0000 7f6626a41640 1 --2- 172.21.15.114:0/2935816257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f662010a330 0x7f662010c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.929+0000 7f6626a41640 1 --2- 172.21.15.114:0/2935816257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6620101800 0x7f6620109ce0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.929+0000 7f6626a41640 1 --2- 172.21.15.114:0/2935816257 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6620100ee0 0x7f66201012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.929+0000 7f6626a41640 1 -- 172.21.15.114:0/2935816257 >> 172.21.15.114:0/2935816257 conn(0x7f66200fa8d0 msgr2=0x7f66200fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:45.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.930+0000 7f6626a41640 1 -- 172.21.15.114:0/2935816257 shutdown_connections 2024-09-13T18:24:45.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.930+0000 7f6626a41640 1 -- 172.21.15.114:0/2935816257 wait complete. 2024-09-13T18:24:45.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.930+0000 7f6626a41640 1 Processor -- start 2024-09-13T18:24:45.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.930+0000 7f6626a41640 1 -- start start 2024-09-13T18:24:45.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.931+0000 7f6626a41640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6620100ee0 0x7f6620109470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:45.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.931+0000 7f6626a41640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6620101800 0x7f66201099b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:45.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.931+0000 7f6626a41640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f662010a330 0x7f66201034d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:45.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.931+0000 7f6626a41640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6620103bc0 con 0x7f6620101800 2024-09-13T18:24:45.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.931+0000 7f661ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6620100ee0 0x7f6620109470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:45.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.931+0000 7f661ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6620100ee0 0x7f6620109470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:39226/0 (socket says 172.21.15.114:39226) 2024-09-13T18:24:45.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.931+0000 7f661ffff640 1 -- 172.21.15.114:0/415158621 learned_addr learned my addr 172.21.15.114:0/415158621 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:45.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.931+0000 7f6626a41640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6620103d30 con 0x7f6620100ee0 2024-09-13T18:24:45.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.931+0000 7f6626a41640 1 -- 172.21.15.114:0/415158621 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6620103ef0 con 0x7f662010a330 2024-09-13T18:24:45.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.932+0000 7f6624fde640 1 --2- 172.21.15.114:0/415158621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f662010a330 0x7f66201034d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:45.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.932+0000 7f661f7fe640 1 --2- 172.21.15.114:0/415158621 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6620101800 0x7f66201099b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:45.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.932+0000 7f661ffff640 1 -- 172.21.15.114:0/415158621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f662010a330 msgr2=0x7f66201034d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:45.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.932+0000 7f661ffff640 1 --2- 172.21.15.114:0/415158621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f662010a330 0x7f66201034d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.932+0000 7f661ffff640 1 -- 172.21.15.114:0/415158621 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6620101800 msgr2=0x7f66201099b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:45.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.932+0000 7f661ffff640 1 --2- 172.21.15.114:0/415158621 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6620101800 0x7f66201099b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.932+0000 7f661ffff640 1 -- 172.21.15.114:0/415158621 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6614009560 con 0x7f6620100ee0 2024-09-13T18:24:45.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.932+0000 7f661ffff640 1 --2- 172.21.15.114:0/415158621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6620100ee0 0x7f6620109470 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f661800beb0 tx=0x7f66180095b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:45.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.932+0000 7f661d7fa640 1 -- 172.21.15.114:0/415158621 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661800de80 con 0x7f6620100ee0 2024-09-13T18:24:45.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.933+0000 7f661d7fa640 1 -- 172.21.15.114:0/415158621 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6618009c60 con 0x7f6620100ee0 2024-09-13T18:24:45.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.933+0000 7f661d7fa640 1 -- 172.21.15.114:0/415158621 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6618015620 con 0x7f6620100ee0 2024-09-13T18:24:45.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.933+0000 7f6626a41640 1 -- 172.21.15.114:0/415158621 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66201041d0 con 0x7f6620100ee0 2024-09-13T18:24:45.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.933+0000 7f6626a41640 1 -- 172.21.15.114:0/415158621 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66201a95b0 con 0x7f6620100ee0 2024-09-13T18:24:45.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.936+0000 7f661d7fa640 1 -- 172.21.15.114:0/415158621 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f6618014070 con 0x7f6620100ee0 2024-09-13T18:24:45.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.937+0000 7f6629843740 1 Processor -- start 2024-09-13T18:24:45.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.937+0000 7f6629843740 1 -- start start 2024-09-13T18:24:45.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.937+0000 7f6629843740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b92334790 0x561b92334b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:45.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.937+0000 7f6629843740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b923350b0 0x561b9233f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:45.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.938+0000 7f6629843740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b9233f950 0x561b92341d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:45.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.938+0000 7f6629843740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b9229ecc0 con 0x561b92334790 2024-09-13T18:24:45.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.938+0000 7f6629843740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b9229c450 con 0x561b923350b0 2024-09-13T18:24:45.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.938+0000 7f6629843740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b92297d40 con 0x561b9233f950 2024-09-13T18:24:45.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.938+0000 7f661ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b923350b0 0x561b9233f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:45.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.938+0000 7f661f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b9233f950 0x561b92341d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:45.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.938+0000 7f661ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b923350b0 0x561b9233f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:39242/0 (socket says 172.21.15.114:39242) 2024-09-13T18:24:45.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.938+0000 7f661ffff640 1 -- 172.21.15.114:0/233410679 learned_addr learned my addr 172.21.15.114:0/233410679 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:45.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.938+0000 7f661ffff640 1 -- 172.21.15.114:0/233410679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b9233f950 msgr2=0x561b92341d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:45.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.938+0000 7f661ffff640 1 --2- 172.21.15.114:0/233410679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b9233f950 0x561b92341d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.938+0000 7f661ffff640 1 -- 172.21.15.114:0/233410679 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b92334790 msgr2=0x561b92334b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:45.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.938+0000 7f661ffff640 1 --2- 172.21.15.114:0/233410679 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b92334790 0x561b92334b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.938+0000 7f661ffff640 1 -- 172.21.15.114:0/233410679 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b9225f620 con 0x561b923350b0 2024-09-13T18:24:45.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.939+0000 7f661d7fa640 1 --2- 172.21.15.114:0/415158621 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f66080a1300 0x7f66080a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:45.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.940+0000 7f661d7fa640 1 -- 172.21.15.114:0/415158621 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f66180cceb0 con 0x7f6620100ee0 2024-09-13T18:24:45.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.940+0000 7f6624fde640 1 --2- 172.21.15.114:0/415158621 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f66080a1300 0x7f66080a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:45.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.940+0000 7f661ffff640 1 --2- 172.21.15.114:0/233410679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b923350b0 0x561b9233f410 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f66180ca190 tx=0x7f66180d3830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45a3a5e7df34b972 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:45.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.941+0000 7f66067fc640 1 -- 172.21.15.114:0/233410679 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66180d8070 con 0x561b923350b0 2024-09-13T18:24:45.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.941+0000 7f6624fde640 1 --2- 172.21.15.114:0/415158621 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f66080a1300 0x7f66080a37c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6610009d60 tx=0x7f6610009900 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:45.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.941+0000 7f66067fc640 1 -- 172.21.15.114:0/233410679 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6618095f50 con 0x561b923350b0 2024-09-13T18:24:45.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.942+0000 7f6629843740 1 -- 172.21.15.114:0/233410679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b923350b0 msgr2=0x561b9233f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:45.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.942+0000 7f6629843740 1 --2- 172.21.15.114:0/233410679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b923350b0 0x561b9233f410 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f66180ca190 tx=0x7f66180d3830 comp rx=0 tx=0).stop 2024-09-13T18:24:45.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.942+0000 7f6629843740 1 -- 172.21.15.114:0/233410679 shutdown_connections 2024-09-13T18:24:45.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.942+0000 7f6629843740 1 --2- 172.21.15.114:0/233410679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b9233f950 0x561b92341d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.942+0000 7f6629843740 1 --2- 172.21.15.114:0/233410679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b923350b0 0x561b9233f410 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.942+0000 7f6629843740 1 --2- 172.21.15.114:0/233410679 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b92334790 0x561b92334b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.942+0000 7f6629843740 1 -- 172.21.15.114:0/233410679 >> 172.21.15.114:0/233410679 conn(0x561b92330d00 msgr2=0x561b923310e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:45.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.942+0000 7f6629843740 1 -- 172.21.15.114:0/233410679 shutdown_connections 2024-09-13T18:24:45.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.943+0000 7f6629843740 1 -- 172.21.15.114:0/233410679 wait complete. 2024-09-13T18:24:45.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.943+0000 7f6629843740 1 -- start start 2024-09-13T18:24:45.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.944+0000 7f6629843740 1 Processor -- start 2024-09-13T18:24:45.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.944+0000 7f6629843740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b92334790 0x561b9233c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:45.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.944+0000 7f6629843740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b9233f950 0x561b9233c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:45.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.944+0000 7f6629843740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b923447f0 0x561b92344c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:45.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.944+0000 7f6629843740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b92256180 con 0x561b923447f0 2024-09-13T18:24:45.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.944+0000 7f6629843740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b91f70430 con 0x561b92334790 2024-09-13T18:24:45.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.944+0000 7f6629843740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b9221b5e0 con 0x561b9233f950 2024-09-13T18:24:45.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.944+0000 7f6629843740 10 client.0 _ll_register_callbacks cb 0x561b92287aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:24:45.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.944+0000 7f661f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b9233f950 0x561b9233c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:45.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.944+0000 7f6624fde640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b923447f0 0x561b92344c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:45.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.944+0000 7f661f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b9233f950 0x561b9233c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:57468/0 (socket says 172.21.15.114:57468) 2024-09-13T18:24:45.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.944+0000 7f661f7fe640 1 -- 172.21.15.114:0/1189601046 learned_addr learned my addr 172.21.15.114:0/1189601046 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:45.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.945+0000 7f6624fde640 1 -- 172.21.15.114:0/1189601046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b9233f950 msgr2=0x561b9233c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:45.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.945+0000 7f6624fde640 1 --2- 172.21.15.114:0/1189601046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b9233f950 0x561b9233c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.945+0000 7f6624fde640 1 -- 172.21.15.114:0/1189601046 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b92334790 msgr2=0x561b9233c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:45.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.945+0000 7f6624fde640 1 --2- 172.21.15.114:0/1189601046 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b92334790 0x561b9233c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.945+0000 7f6624fde640 1 -- 172.21.15.114:0/1189601046 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66201a95b0 con 0x561b923447f0 2024-09-13T18:24:45.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.945+0000 7f6624fde640 1 --2- 172.21.15.114:0/1189601046 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b923447f0 0x561b92344c70 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f661001cb60 tx=0x7f6610005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc3123618b51ece9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:45.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.945+0000 7f65ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:24:45.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.946+0000 7f65ff7fe640 1 -- 172.21.15.114:0/1189601046 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661001ce50 con 0x561b923447f0 2024-09-13T18:24:45.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.946+0000 7f65ff7fe640 1 -- 172.21.15.114:0/1189601046 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f6610023030 con 0x561b923447f0 2024-09-13T18:24:45.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.946+0000 7f6629843740 1 -- 172.21.15.114:0/1189601046 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561b9228a1f0 con 0x561b923447f0 2024-09-13T18:24:45.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.947+0000 7f65ff7fe640 1 -- 172.21.15.114:0/1189601046 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6610004030 con 0x561b923447f0 2024-09-13T18:24:45.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.947+0000 7f6629843740 10 client.16494 fetch_fsmap learned FSMap version 32 2024-09-13T18:24:45.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.947+0000 7f6629843740 1 -- 172.21.15.114:0/1189601046 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x561b922963c0 con 0x561b923447f0 2024-09-13T18:24:45.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.947+0000 7f65ff7fe640 1 -- 172.21.15.114:0/1189601046 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561b9228a1f0 con 0x561b923447f0 2024-09-13T18:24:45.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.948+0000 7f65ff7fe640 1 -- 172.21.15.114:0/1189601046 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f66100038a0 con 0x561b923447f0 2024-09-13T18:24:45.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.948+0000 7f6629843740 10 client.16494 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:24:45.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.948+0000 7f6629843740 10 client.16494 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-13T18:24:45.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.948+0000 7f6629843740 20 client.16494 populate_metadata read hostname 'smithi114' 2024-09-13T18:24:45.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.948+0000 7f6629843740 1 --2- 172.21.15.114:0/1189601046 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x561b9234f630 0x561b92351a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:45.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.948+0000 7f6629843740 4 client.16494 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:24:45.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.948+0000 7f6629843740 1 -- 172.21.15.114:0/1189601046 --> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561b91f8b4e0 con 0x561b9234f630 2024-09-13T18:24:45.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.948+0000 7f661ffff640 1 --2- 172.21.15.114:0/1189601046 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x561b9234f630 0x561b92351a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:45.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.949+0000 7f65ff7fe640 10 client.16494 ms_handle_connect on v2:172.21.15.204:6838/344883912 2024-09-13T18:24:45.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.949+0000 7f661ffff640 1 --2- 172.21.15.114:0/1189601046 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x561b9234f630 0x561b92351a50 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c57fa7ad69b75a14 server_cookie=9f646992750eeae3 in_seq=0 out_seq=0 2024-09-13T18:24:45.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.950+0000 7f65ff7fe640 1 -- 172.21.15.114:0/1189601046 <== mds.2 v2:172.21.15.204:6838/344883912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x561b91f8b4e0 con 0x561b9234f630 2024-09-13T18:24:45.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.950+0000 7f65ff7fe640 10 client.16494 handle_command_reply: tid=0 2024-09-13T18:24:45.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.950+0000 7f6629843740 1 client.16494 shutdown 2024-09-13T18:24:45.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.951+0000 7f6629843740 1 -- 172.21.15.114:0/1189601046 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b923447f0 msgr2=0x561b92344c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:45.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.951+0000 7f6629843740 1 --2- 172.21.15.114:0/1189601046 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b923447f0 0x561b92344c70 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f661001cb60 tx=0x7f6610005f90 comp rx=0 tx=0).stop 2024-09-13T18:24:45.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.951+0000 7f6629843740 1 -- 172.21.15.114:0/1189601046 shutdown_connections 2024-09-13T18:24:45.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.951+0000 7f6629843740 1 --2- 172.21.15.114:0/1189601046 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x561b9234f630 0x561b92351a50 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.951+0000 7f6629843740 1 --2- 172.21.15.114:0/1189601046 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b923447f0 0x561b92344c70 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.951+0000 7f6629843740 1 --2- 172.21.15.114:0/1189601046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b9233f950 0x561b9233c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.951+0000 7f6629843740 1 --2- 172.21.15.114:0/1189601046 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b92334790 0x561b9233c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.951+0000 7f6629843740 1 -- 172.21.15.114:0/1189601046 >> 172.21.15.114:0/1189601046 conn(0x561b92330d00 msgr2=0x561b92336240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:45.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.951+0000 7f65ff7fe640 0 client.16494 ms_handle_reset on v2:172.21.15.204:6838/344883912 2024-09-13T18:24:45.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.951+0000 7f6629843740 1 -- 172.21.15.114:0/1189601046 shutdown_connections 2024-09-13T18:24:45.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.951+0000 7f6629843740 1 -- 172.21.15.114:0/1189601046 wait complete. 2024-09-13T18:24:45.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.952+0000 7f6629843740 20 client.16494 trim_cache size 0 max 16384 2024-09-13T18:24:45.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.973+0000 7f6629843740 1 Processor -- start 2024-09-13T18:24:45.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.973+0000 7f6629843740 1 -- start start 2024-09-13T18:24:45.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.973+0000 7f6629843740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b92334790 0x561b9234b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:45.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.973+0000 7f6629843740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b9233f950 0x561b923563f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:45.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.973+0000 7f6629843740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b923447f0 0x561b92337270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:45.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.973+0000 7f6629843740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b92345030 con 0x561b92334790 2024-09-13T18:24:45.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.973+0000 7f6629843740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b92351cc0 con 0x561b9233f950 2024-09-13T18:24:45.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.973+0000 7f6629843740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b92335280 con 0x561b923447f0 2024-09-13T18:24:45.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.974+0000 7f661ffff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b92334790 0x561b9234b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:45.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.974+0000 7f661f7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b9233f950 0x561b923563f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:45.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.974+0000 7f661ffff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b92334790 0x561b9234b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51664/0 (socket says 172.21.15.114:51664) 2024-09-13T18:24:45.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.974+0000 7f661ffff640 1 -- 172.21.15.114:0/3368717960 learned_addr learned my addr 172.21.15.114:0/3368717960 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:45.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.974+0000 7f661f7fe640 1 --2- 172.21.15.114:0/3368717960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b9233f950 0x561b923563f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:39254/0 (socket says 172.21.15.114:39254) 2024-09-13T18:24:45.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.974+0000 7f6624fde640 1 --2- 172.21.15.114:0/3368717960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b923447f0 0x561b92337270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:45.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.974+0000 7f661f7fe640 1 -- 172.21.15.114:0/3368717960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b923447f0 msgr2=0x561b92337270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:45.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.974+0000 7f661f7fe640 1 --2- 172.21.15.114:0/3368717960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b923447f0 0x561b92337270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.974+0000 7f661f7fe640 1 -- 172.21.15.114:0/3368717960 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b92334790 msgr2=0x561b9234b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:45.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.974+0000 7f661f7fe640 1 --2- 172.21.15.114:0/3368717960 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b92334790 0x561b9234b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.974+0000 7f661f7fe640 1 -- 172.21.15.114:0/3368717960 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b92238770 con 0x561b9233f950 2024-09-13T18:24:45.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.975+0000 7f661f7fe640 1 --2- 172.21.15.114:0/3368717960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b9233f950 0x561b923563f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x561b9232c6d0 tx=0x7f66140040d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cce621a4fde4723 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:45.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.975+0000 7f6606ffd640 1 -- 172.21.15.114:0/3368717960 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6614004500 con 0x561b9233f950 2024-09-13T18:24:45.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.975+0000 7f6606ffd640 1 -- 172.21.15.114:0/3368717960 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6614003a80 con 0x561b9233f950 2024-09-13T18:24:45.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.975+0000 7f6606ffd640 1 -- 172.21.15.114:0/3368717960 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661403ca10 con 0x561b9233f950 2024-09-13T18:24:45.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.976+0000 7f6629843740 1 -- 172.21.15.114:0/3368717960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b9233f950 msgr2=0x561b923563f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:45.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.976+0000 7f6629843740 1 --2- 172.21.15.114:0/3368717960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b9233f950 0x561b923563f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x561b9232c6d0 tx=0x7f66140040d0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.977+0000 7f6629843740 1 -- 172.21.15.114:0/3368717960 shutdown_connections 2024-09-13T18:24:45.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.977+0000 7f6629843740 1 --2- 172.21.15.114:0/3368717960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b923447f0 0x561b92337270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.978+0000 7f6629843740 1 --2- 172.21.15.114:0/3368717960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b9233f950 0x561b923563f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.978+0000 7f6629843740 1 --2- 172.21.15.114:0/3368717960 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b92334790 0x561b9234b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.978+0000 7f6629843740 1 -- 172.21.15.114:0/3368717960 >> 172.21.15.114:0/3368717960 conn(0x561b92330d00 msgr2=0x561b92334b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:45.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.978+0000 7f6629843740 1 -- 172.21.15.114:0/3368717960 shutdown_connections 2024-09-13T18:24:45.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.979+0000 7f6629843740 1 -- 172.21.15.114:0/3368717960 wait complete. 2024-09-13T18:24:45.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.979+0000 7f6629843740 1 -- start start 2024-09-13T18:24:45.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.980+0000 7f6629843740 1 Processor -- start 2024-09-13T18:24:45.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.981+0000 7f6629843740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b92334790 0x561b92348ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:45.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.981+0000 7f6629843740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b9233f950 0x561b92349220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:45.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.981+0000 7f6629843740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b923447f0 0x561b92349760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:45.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.981+0000 7f6629843740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b923a4b70 con 0x561b9233f950 2024-09-13T18:24:45.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.981+0000 7f6629843740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b923a4cb0 con 0x561b92334790 2024-09-13T18:24:45.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.981+0000 7f6624fde640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b923447f0 0x561b92349760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:45.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.981+0000 7f661f7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b9233f950 0x561b92349220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:45.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.981+0000 7f6624fde640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b923447f0 0x561b92349760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:57486/0 (socket says 172.21.15.114:57486) 2024-09-13T18:24:45.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.981+0000 7f661ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b92334790 0x561b92348ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:45.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.981+0000 7f6624fde640 1 -- 172.21.15.114:0/1668061212 learned_addr learned my addr 172.21.15.114:0/1668061212 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:45.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.981+0000 7f6629843740 1 -- 172.21.15.114:0/1668061212 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b923a4e20 con 0x561b923447f0 2024-09-13T18:24:45.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.982+0000 7f6629843740 10 client.0 _ll_register_callbacks cb 0x7f66180d0770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:24:45.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.982+0000 7f6624fde640 1 -- 172.21.15.114:0/1668061212 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b92334790 msgr2=0x561b92348ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:45.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.982+0000 7f6624fde640 1 --2- 172.21.15.114:0/1668061212 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b92334790 0x561b92348ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.982+0000 7f6624fde640 1 -- 172.21.15.114:0/1668061212 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b9233f950 msgr2=0x561b92349220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:45.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.982+0000 7f6624fde640 1 --2- 172.21.15.114:0/1668061212 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b9233f950 0x561b92349220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.982+0000 7f6624fde640 1 -- 172.21.15.114:0/1668061212 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f662010d430 con 0x561b923447f0 2024-09-13T18:24:45.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.982+0000 7f661f7fe640 1 --2- 172.21.15.114:0/1668061212 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b9233f950 0x561b92349220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:24:45.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.982+0000 7f6624fde640 1 --2- 172.21.15.114:0/1668061212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b923447f0 0x561b92349760 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6610011ca0 tx=0x7f661001cf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=761de9fe7364a078 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:45.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.982+0000 7f65ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:24:45.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.983+0000 7f65ff7fe640 1 -- 172.21.15.114:0/1668061212 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6610003a00 con 0x561b923447f0 2024-09-13T18:24:45.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.983+0000 7f65ff7fe640 1 -- 172.21.15.114:0/1668061212 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f6610023050 con 0x561b923447f0 2024-09-13T18:24:45.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.983+0000 7f6629843740 1 -- 172.21.15.114:0/1668061212 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561b922c1b80 con 0x561b923447f0 2024-09-13T18:24:45.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.984+0000 7f65ff7fe640 1 -- 172.21.15.114:0/1668061212 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66100050d0 con 0x561b923447f0 2024-09-13T18:24:45.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.984+0000 7f6629843740 10 client.25864 fetch_fsmap learned FSMap version 32 2024-09-13T18:24:45.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.984+0000 7f6629843740 1 -- 172.21.15.114:0/1668061212 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x561b9225de80 con 0x561b923447f0 2024-09-13T18:24:45.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.984+0000 7f65ff7fe640 1 -- 172.21.15.114:0/1668061212 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f661000f840 con 0x561b923447f0 2024-09-13T18:24:45.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.984+0000 7f65ff7fe640 1 -- 172.21.15.114:0/1668061212 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f6610028610 con 0x561b923447f0 2024-09-13T18:24:45.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.985+0000 7f6629843740 10 client.25864 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:24:45.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.985+0000 7f6629843740 10 client.25864 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-13T18:24:45.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.985+0000 7f6629843740 20 client.25864 populate_metadata read hostname 'smithi114' 2024-09-13T18:24:45.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.985+0000 7f6629843740 1 --2- 172.21.15.114:0/1668061212 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x561b923a6480 0x561b923a88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:45.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.985+0000 7f6629843740 4 client.25864 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:24:45.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.985+0000 7f6629843740 1 -- 172.21.15.114:0/1668061212 --> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561b922c1b80 con 0x561b923a6480 2024-09-13T18:24:45.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.985+0000 7f661ffff640 1 --2- 172.21.15.114:0/1668061212 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x561b923a6480 0x561b923a88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:45.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.986+0000 7f661ffff640 1 --2- 172.21.15.114:0/1668061212 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x561b923a6480 0x561b923a88a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=51f3174a953135b1 server_cookie=5b85022f8495aa3 in_seq=0 out_seq=0 2024-09-13T18:24:45.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.986+0000 7f65ff7fe640 10 client.25864 ms_handle_connect on v2:172.21.15.204:6838/344883912 2024-09-13T18:24:45.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.986+0000 7f65ff7fe640 1 -- 172.21.15.114:0/1668061212 <== mds.2 v2:172.21.15.204:6838/344883912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561b922c1b80 con 0x561b923a6480 2024-09-13T18:24:45.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.986+0000 7f65ff7fe640 10 client.25864 handle_command_reply: tid=0 2024-09-13T18:24:45.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.986+0000 7f6629843740 1 client.25864 shutdown 2024-09-13T18:24:45.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.988+0000 7f6629843740 1 -- 172.21.15.114:0/1668061212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b923447f0 msgr2=0x561b92349760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:45.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.988+0000 7f6629843740 1 --2- 172.21.15.114:0/1668061212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b923447f0 0x561b92349760 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6610011ca0 tx=0x7f661001cf80 comp rx=0 tx=0).stop 2024-09-13T18:24:45.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.988+0000 7f6629843740 1 -- 172.21.15.114:0/1668061212 shutdown_connections 2024-09-13T18:24:45.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.988+0000 7f6629843740 1 --2- 172.21.15.114:0/1668061212 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x561b923a6480 0x561b923a88a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.988+0000 7f6629843740 1 --2- 172.21.15.114:0/1668061212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b923447f0 0x561b92349760 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.988+0000 7f6629843740 1 --2- 172.21.15.114:0/1668061212 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b9233f950 0x561b92349220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.988+0000 7f65ff7fe640 0 client.25864 ms_handle_reset on v2:172.21.15.204:6838/344883912 2024-09-13T18:24:45.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.988+0000 7f6629843740 1 --2- 172.21.15.114:0/1668061212 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b92334790 0x561b92348ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.988+0000 7f6629843740 1 -- 172.21.15.114:0/1668061212 >> 172.21.15.114:0/1668061212 conn(0x561b92330d00 msgr2=0x561b923563f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:45.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.989+0000 7f6629843740 1 -- 172.21.15.114:0/1668061212 shutdown_connections 2024-09-13T18:24:45.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.990+0000 7f6629843740 1 -- 172.21.15.114:0/1668061212 wait complete. 2024-09-13T18:24:45.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.991+0000 7f6629843740 20 client.25864 trim_cache size 0 max 16384 2024-09-13T18:24:45.994 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:24:45.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.992+0000 7f6626a41640 1 -- 172.21.15.114:0/415158621 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f66080a1300 msgr2=0x7f66080a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:45.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.992+0000 7f6626a41640 1 --2- 172.21.15.114:0/415158621 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f66080a1300 0x7f66080a37c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6610009d60 tx=0x7f6610009900 comp rx=0 tx=0).stop 2024-09-13T18:24:45.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.992+0000 7f6626a41640 1 -- 172.21.15.114:0/415158621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6620100ee0 msgr2=0x7f6620109470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:45.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.992+0000 7f6626a41640 1 --2- 172.21.15.114:0/415158621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6620100ee0 0x7f6620109470 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f661800beb0 tx=0x7f66180095b0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.993+0000 7f6626a41640 1 -- 172.21.15.114:0/415158621 shutdown_connections 2024-09-13T18:24:45.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.993+0000 7f6626a41640 1 --2- 172.21.15.114:0/415158621 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f66080a1300 0x7f66080a37c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.993+0000 7f6626a41640 1 --2- 172.21.15.114:0/415158621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f662010a330 0x7f66201034d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.993+0000 7f6626a41640 1 --2- 172.21.15.114:0/415158621 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6620101800 0x7f66201099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.993+0000 7f6626a41640 1 --2- 172.21.15.114:0/415158621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6620100ee0 0x7f6620109470 secure :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f661800beb0 tx=0x7f66180095b0 comp rx=0 tx=0).stop 2024-09-13T18:24:45.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.993+0000 7f6626a41640 1 -- 172.21.15.114:0/415158621 >> 172.21.15.114:0/415158621 conn(0x7f66200fa8d0 msgr2=0x7f66200fc360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:45.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.995+0000 7f6626a41640 1 -- 172.21.15.114:0/415158621 shutdown_connections 2024-09-13T18:24:45.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:45.995+0000 7f6626a41640 1 -- 172.21.15.114:0/415158621 wait complete. 2024-09-13T18:24:46.020 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-09-13T18:24:46.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.193+0000 7fb49c653640 1 -- 172.21.15.114:0/2494926375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4940fec20 msgr2=0x7fb4940ff000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:46.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.193+0000 7fb49c653640 1 --2- 172.21.15.114:0/2494926375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4940fec20 0x7fb4940ff000 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb484009860 tx=0x7fb48402a340 comp rx=0 tx=0).stop 2024-09-13T18:24:46.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.193+0000 7fb49c653640 1 -- 172.21.15.114:0/2494926375 shutdown_connections 2024-09-13T18:24:46.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.193+0000 7fb49c653640 1 --2- 172.21.15.114:0/2494926375 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb4941041a0 0x7fb494104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.193+0000 7fb49c653640 1 --2- 172.21.15.114:0/2494926375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4940ff5d0 0x7fb494103a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.193+0000 7fb49c653640 1 --2- 172.21.15.114:0/2494926375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4940fec20 0x7fb4940ff000 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.193+0000 7fb49c653640 1 -- 172.21.15.114:0/2494926375 >> 172.21.15.114:0/2494926375 conn(0x7fb4940fa8d0 msgr2=0x7fb4940fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:46.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.193+0000 7fb49c653640 1 -- 172.21.15.114:0/2494926375 shutdown_connections 2024-09-13T18:24:46.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.193+0000 7fb49c653640 1 -- 172.21.15.114:0/2494926375 wait complete. 2024-09-13T18:24:46.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.194+0000 7fb49c653640 1 Processor -- start 2024-09-13T18:24:46.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.194+0000 7fb49c653640 1 -- start start 2024-09-13T18:24:46.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.194+0000 7fb49c653640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4940fec20 0x7fb4941a25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:46.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.194+0000 7fb49c653640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb4940ff5d0 0x7fb4941a2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:46.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.195+0000 7fb49c653640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4941041a0 0x7fb49419c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:46.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.195+0000 7fb49c653640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49419cce0 con 0x7fb4940ff5d0 2024-09-13T18:24:46.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.195+0000 7fb49c653640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49419ce50 con 0x7fb4940fec20 2024-09-13T18:24:46.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.195+0000 7fb49c653640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49419d010 con 0x7fb4941041a0 2024-09-13T18:24:46.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.195+0000 7fb49a3ef640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4940fec20 0x7fb4941a25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:46.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.195+0000 7fb499bee640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb4940ff5d0 0x7fb4941a2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:46.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.195+0000 7fb49a3ef640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4940fec20 0x7fb4941a25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:39274/0 (socket says 172.21.15.114:39274) 2024-09-13T18:24:46.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.195+0000 7fb49a3ef640 1 -- 172.21.15.114:0/3986345516 learned_addr learned my addr 172.21.15.114:0/3986345516 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:46.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.195+0000 7fb49a3ef640 1 -- 172.21.15.114:0/3986345516 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4941041a0 msgr2=0x7fb49419c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:46.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.195+0000 7fb49a3ef640 1 --2- 172.21.15.114:0/3986345516 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4941041a0 0x7fb49419c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.195+0000 7fb49a3ef640 1 -- 172.21.15.114:0/3986345516 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb4940ff5d0 msgr2=0x7fb4941a2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:46.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.195+0000 7fb49a3ef640 1 --2- 172.21.15.114:0/3986345516 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb4940ff5d0 0x7fb4941a2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.195+0000 7fb49a3ef640 1 -- 172.21.15.114:0/3986345516 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb48c009900 con 0x7fb4940fec20 2024-09-13T18:24:46.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.195+0000 7fb49a3ef640 1 --2- 172.21.15.114:0/3986345516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4940fec20 0x7fb4941a25e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb484035040 tx=0x7fb484033c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:46.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.195+0000 7fb4937fe640 1 -- 172.21.15.114:0/3986345516 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb484030ed0 con 0x7fb4940fec20 2024-09-13T18:24:46.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.196+0000 7fb49c653640 1 -- 172.21.15.114:0/3986345516 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb484009490 con 0x7fb4940fec20 2024-09-13T18:24:46.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.196+0000 7fb49c653640 1 -- 172.21.15.114:0/3986345516 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4941a93f0 con 0x7fb4940fec20 2024-09-13T18:24:46.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.196+0000 7fb4937fe640 1 -- 172.21.15.114:0/3986345516 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb484033e70 con 0x7fb4940fec20 2024-09-13T18:24:46.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.196+0000 7fb4937fe640 1 -- 172.21.15.114:0/3986345516 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb484003d60 con 0x7fb4940fec20 2024-09-13T18:24:46.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.198+0000 7fb4937fe640 1 -- 172.21.15.114:0/3986345516 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fb484039040 con 0x7fb4940fec20 2024-09-13T18:24:46.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.199+0000 7fb49f44f740 1 Processor -- start 2024-09-13T18:24:46.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.200+0000 7fb4937fe640 1 --2- 172.21.15.114:0/3986345516 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fb47c0a1300 0x7fb47c0a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:46.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.200+0000 7fb4937fe640 1 -- 172.21.15.114:0/3986345516 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fb4840ed6e0 con 0x7fb4940fec20 2024-09-13T18:24:46.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.201+0000 7fb49f44f740 1 -- start start 2024-09-13T18:24:46.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.201+0000 7fb49f44f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561853c9d790 0x561853c9db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:46.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.201+0000 7fb49f44f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561853c9e0b0 0x561853ca8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:46.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.201+0000 7fb49f44f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561853ca8950 0x561853caad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:46.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.201+0000 7fb49f44f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x561853c07cc0 con 0x561853c9d790 2024-09-13T18:24:46.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.201+0000 7fb49f44f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561853c05450 con 0x561853c9e0b0 2024-09-13T18:24:46.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.201+0000 7fb49f44f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561853c00d40 con 0x561853ca8950 2024-09-13T18:24:46.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.201+0000 7fb49abf0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561853ca8950 0x561853caad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:46.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.201+0000 7fb499bee640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561853c9e0b0 0x561853ca8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:46.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.201+0000 7fb499bee640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561853c9e0b0 0x561853ca8410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:39288/0 (socket says 172.21.15.114:39288) 2024-09-13T18:24:46.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.201+0000 7fb499bee640 1 -- 172.21.15.114:0/3106154725 learned_addr learned my addr 172.21.15.114:0/3106154725 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:46.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.202+0000 7fb49abf0640 1 --2- 172.21.15.114:0/3986345516 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fb47c0a1300 0x7fb47c0a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:46.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.202+0000 7fb49a3ef640 1 --2- 172.21.15.114:0/3106154725 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561853c9d790 0x561853c9db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:46.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.202+0000 7fb499bee640 1 -- 172.21.15.114:0/3106154725 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561853ca8950 msgr2=0x561853caad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:46.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.202+0000 7fb499bee640 1 --2- 172.21.15.114:0/3106154725 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561853ca8950 0x561853caad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.202+0000 7fb499bee640 1 -- 172.21.15.114:0/3106154725 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561853c9d790 msgr2=0x561853c9db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:46.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.202+0000 7fb499bee640 1 --2- 172.21.15.114:0/3106154725 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561853c9d790 0x561853c9db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.202+0000 7fb499bee640 1 -- 172.21.15.114:0/3106154725 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561853bc8620 con 0x561853c9e0b0 2024-09-13T18:24:46.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.202+0000 7fb499bee640 1 --2- 172.21.15.114:0/3106154725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561853c9e0b0 0x561853ca8410 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb488004630 tx=0x7fb48800bba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5ac5e94f83d01bc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:46.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.203+0000 7fb490ff9640 1 -- 172.21.15.114:0/3106154725 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb488003bd0 con 0x561853c9e0b0 2024-09-13T18:24:46.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.203+0000 7fb490ff9640 1 -- 172.21.15.114:0/3106154725 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb488003d30 con 0x561853c9e0b0 2024-09-13T18:24:46.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.204+0000 7fb490ff9640 1 -- 172.21.15.114:0/3106154725 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb488004dd0 con 0x561853c9e0b0 2024-09-13T18:24:46.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.204+0000 7fb49f44f740 1 -- 172.21.15.114:0/3106154725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561853c9e0b0 msgr2=0x561853ca8410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:46.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.204+0000 7fb49abf0640 1 --2- 172.21.15.114:0/3986345516 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fb47c0a1300 0x7fb47c0a37c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x561853c9b080 tx=0x7fb48c0095c0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:46.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.204+0000 7fb49f44f740 1 --2- 172.21.15.114:0/3106154725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561853c9e0b0 0x561853ca8410 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb488004630 tx=0x7fb48800bba0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.205+0000 7fb49f44f740 1 -- 172.21.15.114:0/3106154725 shutdown_connections 2024-09-13T18:24:46.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.205+0000 7fb49f44f740 1 --2- 172.21.15.114:0/3106154725 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561853ca8950 0x561853caad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.205+0000 7fb49f44f740 1 --2- 172.21.15.114:0/3106154725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561853c9e0b0 0x561853ca8410 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.205+0000 7fb49f44f740 1 --2- 172.21.15.114:0/3106154725 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561853c9d790 0x561853c9db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.205+0000 7fb49f44f740 1 -- 172.21.15.114:0/3106154725 >> 172.21.15.114:0/3106154725 conn(0x561853c99d00 msgr2=0x561853c9a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:46.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.205+0000 7fb49f44f740 1 -- 172.21.15.114:0/3106154725 shutdown_connections 2024-09-13T18:24:46.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.205+0000 7fb49f44f740 1 -- 172.21.15.114:0/3106154725 wait complete. 2024-09-13T18:24:46.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.205+0000 7fb49f44f740 1 -- start start 2024-09-13T18:24:46.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.205+0000 7fb49f44f740 1 Processor -- start 2024-09-13T18:24:46.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.206+0000 7fb49f44f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561853c9d790 0x561853ca7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:46.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.206+0000 7fb49f44f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561853c9e0b0 0x561853ca8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:46.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.206+0000 7fb49f44f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561853ca8950 0x561853ca5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:46.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.206+0000 7fb49f44f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x561853bbf180 con 0x561853c9e0b0 2024-09-13T18:24:46.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.206+0000 7fb49f44f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618538d9430 con 0x561853c9d790 2024-09-13T18:24:46.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.206+0000 7fb49f44f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561853b845e0 con 0x561853ca8950 2024-09-13T18:24:46.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.206+0000 7fb49f44f740 10 client.0 _ll_register_callbacks cb 0x561853bf0aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:24:46.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.206+0000 7fb499bee640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561853c9e0b0 0x561853ca8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:46.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.206+0000 7fb499bee640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561853c9e0b0 0x561853ca8340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51724/0 (socket says 172.21.15.114:51724) 2024-09-13T18:24:46.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.206+0000 7fb499bee640 1 -- 172.21.15.114:0/2608558031 learned_addr learned my addr 172.21.15.114:0/2608558031 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:46.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.206+0000 7fb49abf0640 1 --2- 172.21.15.114:0/2608558031 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561853ca8950 0x561853ca5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:46.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.206+0000 7fb49a3ef640 1 --2- 172.21.15.114:0/2608558031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561853c9d790 0x561853ca7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:46.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.207+0000 7fb499bee640 1 -- 172.21.15.114:0/2608558031 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561853ca8950 msgr2=0x561853ca5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:46.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.207+0000 7fb499bee640 1 --2- 172.21.15.114:0/2608558031 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561853ca8950 0x561853ca5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.207+0000 7fb499bee640 1 -- 172.21.15.114:0/2608558031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561853c9d790 msgr2=0x561853ca7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:46.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.207+0000 7fb499bee640 1 --2- 172.21.15.114:0/2608558031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561853c9d790 0x561853ca7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.207+0000 7fb499bee640 1 -- 172.21.15.114:0/2608558031 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb488004310 con 0x561853c9e0b0 2024-09-13T18:24:46.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.207+0000 7fb499bee640 1 --2- 172.21.15.114:0/2608558031 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561853c9e0b0 0x561853ca8340 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fb4880049b0 tx=0x7fb488003c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f59e3246737869f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:46.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.207+0000 7fb479ffb640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:24:46.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.208+0000 7fb479ffb640 1 -- 172.21.15.114:0/2608558031 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb488004d20 con 0x561853c9e0b0 2024-09-13T18:24:46.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.208+0000 7fb479ffb640 1 -- 172.21.15.114:0/2608558031 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fb488004e80 con 0x561853c9e0b0 2024-09-13T18:24:46.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.208+0000 7fb49f44f740 1 -- 172.21.15.114:0/2608558031 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561853c2ab80 con 0x561853c9e0b0 2024-09-13T18:24:46.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.209+0000 7fb479ffb640 1 -- 172.21.15.114:0/2608558031 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb488005040 con 0x561853c9e0b0 2024-09-13T18:24:46.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.209+0000 7fb49f44f740 10 client.16512 fetch_fsmap learned FSMap version 32 2024-09-13T18:24:46.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.209+0000 7fb49f44f740 1 -- 172.21.15.114:0/2608558031 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x561853bff3c0 con 0x561853c9e0b0 2024-09-13T18:24:46.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.210+0000 7fb479ffb640 1 -- 172.21.15.114:0/2608558031 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561853c2ab80 con 0x561853c9e0b0 2024-09-13T18:24:46.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.210+0000 7fb479ffb640 1 -- 172.21.15.114:0/2608558031 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fb48804e050 con 0x561853c9e0b0 2024-09-13T18:24:46.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.210+0000 7fb49f44f740 10 client.16512 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:24:46.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.210+0000 7fb49f44f740 10 client.16512 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-13T18:24:46.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.210+0000 7fb49f44f740 20 client.16512 populate_metadata read hostname 'smithi114' 2024-09-13T18:24:46.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.210+0000 7fb49f44f740 1 --2- 172.21.15.114:0/2608558031 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x561853cb8260 0x561853cba680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:46.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.210+0000 7fb49f44f740 4 client.16512 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:24:46.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.210+0000 7fb49f44f740 1 -- 172.21.15.114:0/2608558031 --> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561853bf31f0 con 0x561853cb8260 2024-09-13T18:24:46.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.210+0000 7fb49a3ef640 1 --2- 172.21.15.114:0/2608558031 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x561853cb8260 0x561853cba680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:46.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.211+0000 7fb49a3ef640 1 --2- 172.21.15.114:0/2608558031 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x561853cb8260 0x561853cba680 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a33d2caafd8d1f10 server_cookie=d740f407f5aa0f99 in_seq=0 out_seq=0 2024-09-13T18:24:46.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.211+0000 7fb479ffb640 10 client.16512 ms_handle_connect on v2:172.21.15.134:6836/891692210 2024-09-13T18:24:46.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.212+0000 7fb479ffb640 1 -- 172.21.15.114:0/2608558031 <== mds.1 v2:172.21.15.134:6836/891692210 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x561853bf31f0 con 0x561853cb8260 2024-09-13T18:24:46.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.212+0000 7fb479ffb640 10 client.16512 handle_command_reply: tid=0 2024-09-13T18:24:46.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.212+0000 7fb49f44f740 1 client.16512 shutdown 2024-09-13T18:24:46.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.213+0000 7fb49f44f740 1 -- 172.21.15.114:0/2608558031 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561853c9e0b0 msgr2=0x561853ca8340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:46.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.213+0000 7fb49f44f740 1 --2- 172.21.15.114:0/2608558031 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561853c9e0b0 0x561853ca8340 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fb4880049b0 tx=0x7fb488003c40 comp rx=0 tx=0).stop 2024-09-13T18:24:46.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.213+0000 7fb49f44f740 1 -- 172.21.15.114:0/2608558031 shutdown_connections 2024-09-13T18:24:46.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.213+0000 7fb49f44f740 1 --2- 172.21.15.114:0/2608558031 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x561853cb8260 0x561853cba680 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.213+0000 7fb49f44f740 1 --2- 172.21.15.114:0/2608558031 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561853ca8950 0x561853ca5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.213+0000 7fb49f44f740 1 --2- 172.21.15.114:0/2608558031 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561853c9e0b0 0x561853ca8340 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.213+0000 7fb49f44f740 1 --2- 172.21.15.114:0/2608558031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561853c9d790 0x561853ca7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.213+0000 7fb49f44f740 1 -- 172.21.15.114:0/2608558031 >> 172.21.15.114:0/2608558031 conn(0x561853c99d00 msgr2=0x561853c9fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:46.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.213+0000 7fb479ffb640 0 client.16512 ms_handle_reset on v2:172.21.15.134:6836/891692210 2024-09-13T18:24:46.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.213+0000 7fb49f44f740 1 -- 172.21.15.114:0/2608558031 shutdown_connections 2024-09-13T18:24:46.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.213+0000 7fb49f44f740 1 -- 172.21.15.114:0/2608558031 wait complete. 2024-09-13T18:24:46.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.214+0000 7fb49f44f740 20 client.16512 trim_cache size 0 max 16384 2024-09-13T18:24:46.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.236+0000 7fb49f44f740 1 Processor -- start 2024-09-13T18:24:46.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.236+0000 7fb49f44f740 1 -- start start 2024-09-13T18:24:46.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.236+0000 7fb49f44f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561853c9d790 0x561853c9a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:46.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.236+0000 7fb49f44f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561853c9e0b0 0x561853c9a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:46.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.236+0000 7fb49f44f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561853ca8950 0x561853ca0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:46.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.236+0000 7fb49f44f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x561853c9ae90 con 0x561853ca8950 2024-09-13T18:24:46.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.236+0000 7fb49f44f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561853ca5840 con 0x561853c9d790 2024-09-13T18:24:46.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.236+0000 7fb49f44f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561853ca3690 con 0x561853c9e0b0 2024-09-13T18:24:46.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.237+0000 7fb49a3ef640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561853c9d790 0x561853c9a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:46.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.237+0000 7fb49a3ef640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561853c9d790 0x561853c9a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:39302/0 (socket says 172.21.15.114:39302) 2024-09-13T18:24:46.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.237+0000 7fb49a3ef640 1 -- 172.21.15.114:0/4079621855 learned_addr learned my addr 172.21.15.114:0/4079621855 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:46.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.237+0000 7fb499bee640 1 --2- 172.21.15.114:0/4079621855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561853c9e0b0 0x561853c9a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:46.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.237+0000 7fb49abf0640 1 --2- 172.21.15.114:0/4079621855 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561853ca8950 0x561853ca0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:46.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.237+0000 7fb499bee640 1 -- 172.21.15.114:0/4079621855 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561853c9d790 msgr2=0x561853c9a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:46.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.237+0000 7fb499bee640 1 --2- 172.21.15.114:0/4079621855 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561853c9d790 0x561853c9a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.237+0000 7fb499bee640 1 -- 172.21.15.114:0/4079621855 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561853ca8950 msgr2=0x561853ca0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:46.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.237+0000 7fb499bee640 1 --2- 172.21.15.114:0/4079621855 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561853ca8950 0x561853ca0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.237+0000 7fb499bee640 1 -- 172.21.15.114:0/4079621855 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561853ba1770 con 0x561853c9e0b0 2024-09-13T18:24:46.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.237+0000 7fb49a3ef640 1 --2- 172.21.15.114:0/4079621855 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561853c9d790 0x561853c9a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:24:46.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.237+0000 7fb499bee640 1 --2- 172.21.15.114:0/4079621855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561853c9e0b0 0x561853c9a620 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb488046e30 tx=0x7fb488004000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f745d65436ea3d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:46.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.238+0000 7fb4917fa640 1 -- 172.21.15.114:0/4079621855 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb488051d30 con 0x561853c9e0b0 2024-09-13T18:24:46.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.238+0000 7fb4917fa640 1 -- 172.21.15.114:0/4079621855 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb488005e50 con 0x561853c9e0b0 2024-09-13T18:24:46.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.238+0000 7fb4917fa640 1 -- 172.21.15.114:0/4079621855 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4880463e0 con 0x561853c9e0b0 2024-09-13T18:24:46.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.239+0000 7fb49f44f740 1 -- 172.21.15.114:0/4079621855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561853c9e0b0 msgr2=0x561853c9a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:46.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.239+0000 7fb49f44f740 1 --2- 172.21.15.114:0/4079621855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561853c9e0b0 0x561853c9a620 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb488046e30 tx=0x7fb488004000 comp rx=0 tx=0).stop 2024-09-13T18:24:46.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.239+0000 7fb49f44f740 1 -- 172.21.15.114:0/4079621855 shutdown_connections 2024-09-13T18:24:46.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.239+0000 7fb49f44f740 1 --2- 172.21.15.114:0/4079621855 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561853ca8950 0x561853ca0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.239+0000 7fb49f44f740 1 --2- 172.21.15.114:0/4079621855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561853c9e0b0 0x561853c9a620 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.239+0000 7fb49f44f740 1 --2- 172.21.15.114:0/4079621855 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561853c9d790 0x561853c9a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.239+0000 7fb49f44f740 1 -- 172.21.15.114:0/4079621855 >> 172.21.15.114:0/4079621855 conn(0x561853c99d00 msgr2=0x561853cc0310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:46.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.239+0000 7fb49f44f740 1 -- 172.21.15.114:0/4079621855 shutdown_connections 2024-09-13T18:24:46.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.239+0000 7fb49f44f740 1 -- 172.21.15.114:0/4079621855 wait complete. 2024-09-13T18:24:46.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.240+0000 7fb49f44f740 1 -- start start 2024-09-13T18:24:46.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.240+0000 7fb49f44f740 1 Processor -- start 2024-09-13T18:24:46.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.240+0000 7fb49f44f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561853c9d790 0x561853ca1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:46.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.240+0000 7fb49f44f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561853c9e0b0 0x561853ca2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:46.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.240+0000 7fb49f44f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561853ca8950 0x561853ca2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:46.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.240+0000 7fb49f44f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x561853d0dba0 con 0x561853ca8950 2024-09-13T18:24:46.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.240+0000 7fb49f44f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561853d0dd10 con 0x561853c9e0b0 2024-09-13T18:24:46.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.240+0000 7fb49f44f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561853d0ded0 con 0x561853c9d790 2024-09-13T18:24:46.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.240+0000 7fb49f44f740 10 client.0 _ll_register_callbacks cb 0x7fb4880040e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:24:46.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.240+0000 7fb49abf0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561853ca8950 0x561853ca2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:46.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.240+0000 7fb49a3ef640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561853c9d790 0x561853ca1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:46.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.240+0000 7fb49abf0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561853ca8950 0x561853ca2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51754/0 (socket says 172.21.15.114:51754) 2024-09-13T18:24:46.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.240+0000 7fb49a3ef640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561853c9d790 0x561853ca1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:57534/0 (socket says 172.21.15.114:57534) 2024-09-13T18:24:46.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.240+0000 7fb49a3ef640 1 -- 172.21.15.114:0/4062903498 learned_addr learned my addr 172.21.15.114:0/4062903498 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:46.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.241+0000 7fb499bee640 1 --2- 172.21.15.114:0/4062903498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561853c9e0b0 0x561853ca2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:46.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.241+0000 7fb49a3ef640 1 -- 172.21.15.114:0/4062903498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561853c9e0b0 msgr2=0x561853ca2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:46.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.241+0000 7fb49a3ef640 1 --2- 172.21.15.114:0/4062903498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561853c9e0b0 0x561853ca2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.241+0000 7fb49a3ef640 1 -- 172.21.15.114:0/4062903498 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561853ca8950 msgr2=0x561853ca2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:46.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.241+0000 7fb49a3ef640 1 --2- 172.21.15.114:0/4062903498 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561853ca8950 0x561853ca2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.241+0000 7fb49a3ef640 1 -- 172.21.15.114:0/4062903498 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4840b6040 con 0x561853c9d790 2024-09-13T18:24:46.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.241+0000 7fb49a3ef640 1 --2- 172.21.15.114:0/4062903498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561853c9d790 0x561853ca1fe0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb4840a5010 tx=0x7fb4840da040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57370f3962b6332e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:46.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.241+0000 7fb479ffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:24:46.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.241+0000 7fb479ffb640 1 -- 172.21.15.114:0/4062903498 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4840be380 con 0x561853c9d790 2024-09-13T18:24:46.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.241+0000 7fb49f44f740 1 -- 172.21.15.114:0/4062903498 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb488003d30 con 0x561853c9d790 2024-09-13T18:24:46.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.241+0000 7fb479ffb640 1 -- 172.21.15.114:0/4062903498 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fb4840c29d0 con 0x561853c9d790 2024-09-13T18:24:46.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.242+0000 7fb479ffb640 1 -- 172.21.15.114:0/4062903498 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4840be120 con 0x561853c9d790 2024-09-13T18:24:46.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.242+0000 7fb49f44f740 10 client.25879 fetch_fsmap learned FSMap version 32 2024-09-13T18:24:46.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.242+0000 7fb49f44f740 1 -- 172.21.15.114:0/4062903498 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x561853bff3c0 con 0x561853c9d790 2024-09-13T18:24:46.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.242+0000 7fb479ffb640 1 -- 172.21.15.114:0/4062903498 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb488003d30 con 0x561853c9d790 2024-09-13T18:24:46.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.242+0000 7fb479ffb640 1 -- 172.21.15.114:0/4062903498 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fb4840a0020 con 0x561853c9d790 2024-09-13T18:24:46.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.242+0000 7fb49f44f740 10 client.25879 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:24:46.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.242+0000 7fb49f44f740 10 client.25879 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-13T18:24:46.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.242+0000 7fb49f44f740 20 client.25879 populate_metadata read hostname 'smithi114' 2024-09-13T18:24:46.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.242+0000 7fb49f44f740 1 --2- 172.21.15.114:0/4062903498 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x561853d0f4a0 0x561853d118c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:46.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.242+0000 7fb49f44f740 4 client.25879 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:24:46.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.242+0000 7fb49f44f740 1 -- 172.21.15.114:0/4062903498 --> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5618538f44e0 con 0x561853d0f4a0 2024-09-13T18:24:46.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.243+0000 7fb499bee640 1 --2- 172.21.15.114:0/4062903498 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x561853d0f4a0 0x561853d118c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:46.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.243+0000 7fb479ffb640 10 client.25879 ms_handle_connect on v2:172.21.15.134:6836/891692210 2024-09-13T18:24:46.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.243+0000 7fb499bee640 1 --2- 172.21.15.114:0/4062903498 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x561853d0f4a0 0x561853d118c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=84e861a95552de server_cookie=11f83398b42647aa in_seq=0 out_seq=0 2024-09-13T18:24:46.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.243+0000 7fb479ffb640 1 -- 172.21.15.114:0/4062903498 <== mds.1 v2:172.21.15.134:6836/891692210 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5618538f44e0 con 0x561853d0f4a0 2024-09-13T18:24:46.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.243+0000 7fb479ffb640 10 client.25879 handle_command_reply: tid=0 2024-09-13T18:24:46.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.243+0000 7fb49f44f740 1 client.25879 shutdown 2024-09-13T18:24:46.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.243+0000 7fb49f44f740 1 -- 172.21.15.114:0/4062903498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561853c9d790 msgr2=0x561853ca1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:46.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.243+0000 7fb49f44f740 1 --2- 172.21.15.114:0/4062903498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561853c9d790 0x561853ca1fe0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb4840a5010 tx=0x7fb4840da040 comp rx=0 tx=0).stop 2024-09-13T18:24:46.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.243+0000 7fb49f44f740 1 -- 172.21.15.114:0/4062903498 shutdown_connections 2024-09-13T18:24:46.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.243+0000 7fb49f44f740 1 --2- 172.21.15.114:0/4062903498 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x561853d0f4a0 0x561853d118c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.243+0000 7fb49f44f740 1 --2- 172.21.15.114:0/4062903498 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561853ca8950 0x561853ca2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.243+0000 7fb479ffb640 0 client.25879 ms_handle_reset on v2:172.21.15.134:6836/891692210 2024-09-13T18:24:46.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.243+0000 7fb49f44f740 1 --2- 172.21.15.114:0/4062903498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561853c9e0b0 0x561853ca2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.243+0000 7fb49f44f740 1 --2- 172.21.15.114:0/4062903498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561853c9d790 0x561853ca1fe0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.243+0000 7fb49f44f740 1 -- 172.21.15.114:0/4062903498 >> 172.21.15.114:0/4062903498 conn(0x561853c99d00 msgr2=0x561853c9a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:46.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.244+0000 7fb49f44f740 1 -- 172.21.15.114:0/4062903498 shutdown_connections 2024-09-13T18:24:46.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.244+0000 7fb49f44f740 1 -- 172.21.15.114:0/4062903498 wait complete. 2024-09-13T18:24:46.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.244+0000 7fb49f44f740 20 client.25879 trim_cache size 0 max 16384 2024-09-13T18:24:46.253 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:24:46.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.244+0000 7fb49c653640 1 -- 172.21.15.114:0/3986345516 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fb47c0a1300 msgr2=0x7fb47c0a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:46.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.244+0000 7fb49c653640 1 --2- 172.21.15.114:0/3986345516 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fb47c0a1300 0x7fb47c0a37c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x561853c9b080 tx=0x7fb48c0095c0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.244+0000 7fb49c653640 1 -- 172.21.15.114:0/3986345516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4940fec20 msgr2=0x7fb4941a25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:46.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.244+0000 7fb49c653640 1 --2- 172.21.15.114:0/3986345516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4940fec20 0x7fb4941a25e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb484035040 tx=0x7fb484033c80 comp rx=0 tx=0).stop 2024-09-13T18:24:46.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.245+0000 7fb49c653640 1 -- 172.21.15.114:0/3986345516 shutdown_connections 2024-09-13T18:24:46.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.245+0000 7fb49c653640 1 --2- 172.21.15.114:0/3986345516 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fb47c0a1300 0x7fb47c0a37c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.245+0000 7fb49c653640 1 --2- 172.21.15.114:0/3986345516 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4941041a0 0x7fb49419c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.245+0000 7fb49c653640 1 --2- 172.21.15.114:0/3986345516 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb4940ff5d0 0x7fb4941a2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.245+0000 7fb49c653640 1 --2- 172.21.15.114:0/3986345516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4940fec20 0x7fb4941a25e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.245+0000 7fb49c653640 1 -- 172.21.15.114:0/3986345516 >> 172.21.15.114:0/3986345516 conn(0x7fb4940fa8d0 msgr2=0x7fb4940fbb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:46.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.245+0000 7fb49c653640 1 -- 172.21.15.114:0/3986345516 shutdown_connections 2024-09-13T18:24:46.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.245+0000 7fb49c653640 1 -- 172.21.15.114:0/3986345516 wait complete. 2024-09-13T18:24:46.269 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-09-13T18:24:46.304 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:46 smithi114 ceph-mon[30897]: pgmap v72: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 92 MiB/s wr, 547 op/s 2024-09-13T18:24:46.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.438+0000 7f5efa8c9640 1 -- 172.21.15.114:0/434096646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5eec09a5f0 msgr2=0x7f5eec09a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:46.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.438+0000 7f5efa8c9640 1 --2- 172.21.15.114:0/434096646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5eec09a5f0 0x7f5eec09a9d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8009900 tx=0x7f5ee802a3f0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.439+0000 7f5efa8c9640 1 -- 172.21.15.114:0/434096646 shutdown_connections 2024-09-13T18:24:46.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.439+0000 7f5efa8c9640 1 --2- 172.21.15.114:0/434096646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5eec006f50 0x7f5eec0980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.439+0000 7f5efa8c9640 1 --2- 172.21.15.114:0/434096646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5eec0064e0 0x7f5eec006900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.439+0000 7f5efa8c9640 1 --2- 172.21.15.114:0/434096646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5eec09a5f0 0x7f5eec09a9d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.439+0000 7f5efa8c9640 1 -- 172.21.15.114:0/434096646 >> 172.21.15.114:0/434096646 conn(0x7f5eec00a130 msgr2=0x7f5eec00a540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:46.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.439+0000 7f5efa8c9640 1 -- 172.21.15.114:0/434096646 shutdown_connections 2024-09-13T18:24:46.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.439+0000 7f5efa8c9640 1 -- 172.21.15.114:0/434096646 wait complete. 2024-09-13T18:24:46.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.440+0000 7f5efa8c9640 1 Processor -- start 2024-09-13T18:24:46.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.440+0000 7f5efa8c9640 1 -- start start 2024-09-13T18:24:46.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.441+0000 7f5efa8c9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5eec0064e0 0x7f5eec132050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:46.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.441+0000 7f5efa8c9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5eec006f50 0x7f5eec132590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:46.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.441+0000 7f5efa8c9640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5eec09a5f0 0x7f5eec136930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:46.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.441+0000 7f5efa8c9640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec136f30 con 0x7f5eec09a5f0 2024-09-13T18:24:46.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.441+0000 7f5efa8c9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec1370a0 con 0x7f5eec006f50 2024-09-13T18:24:46.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.441+0000 7f5efa8c9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec137260 con 0x7f5eec0064e0 2024-09-13T18:24:46.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.441+0000 7f5ef8e66640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5eec09a5f0 0x7f5eec136930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:46.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.441+0000 7f5ef3fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5eec0064e0 0x7f5eec132050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:46.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.441+0000 7f5ef37fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5eec006f50 0x7f5eec132590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:46.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.441+0000 7f5ef8e66640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5eec09a5f0 0x7f5eec136930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51770/0 (socket says 172.21.15.114:51770) 2024-09-13T18:24:46.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.441+0000 7f5ef3fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5eec0064e0 0x7f5eec132050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:57562/0 (socket says 172.21.15.114:57562) 2024-09-13T18:24:46.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.441+0000 7f5ef8e66640 1 -- 172.21.15.114:0/3191458712 learned_addr learned my addr 172.21.15.114:0/3191458712 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:46.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.441+0000 7f5ef8e66640 1 -- 172.21.15.114:0/3191458712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5eec0064e0 msgr2=0x7f5eec132050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:46.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.441+0000 7f5ef8e66640 1 --2- 172.21.15.114:0/3191458712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5eec0064e0 0x7f5eec132050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.441+0000 7f5ef8e66640 1 -- 172.21.15.114:0/3191458712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5eec006f50 msgr2=0x7f5eec132590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:46.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.441+0000 7f5ef8e66640 1 --2- 172.21.15.114:0/3191458712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5eec006f50 0x7f5eec132590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.441+0000 7f5ef8e66640 1 -- 172.21.15.114:0/3191458712 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee8009560 con 0x7f5eec09a5f0 2024-09-13T18:24:46.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.442+0000 7f5ef37fe640 1 --2- 172.21.15.114:0/3191458712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5eec006f50 0x7f5eec132590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:24:46.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.442+0000 7f5ef3fff640 1 --2- 172.21.15.114:0/3191458712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5eec0064e0 0x7f5eec132050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:24:46.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.442+0000 7f5ef8e66640 1 --2- 172.21.15.114:0/3191458712 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5eec09a5f0 0x7f5eec136930 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f5ef404efc0 tx=0x7f5ef40696c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:46.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.442+0000 7f5ef17fa640 1 -- 172.21.15.114:0/3191458712 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef4066cd0 con 0x7f5eec09a5f0 2024-09-13T18:24:46.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.442+0000 7f5efa8c9640 1 -- 172.21.15.114:0/3191458712 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eec137540 con 0x7f5eec09a5f0 2024-09-13T18:24:46.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.442+0000 7f5ef17fa640 1 -- 172.21.15.114:0/3191458712 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5ef407b040 con 0x7f5eec09a5f0 2024-09-13T18:24:46.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.442+0000 7f5ef17fa640 1 -- 172.21.15.114:0/3191458712 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef407f790 con 0x7f5eec09a5f0 2024-09-13T18:24:46.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.442+0000 7f5efa8c9640 1 -- 172.21.15.114:0/3191458712 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eec13edc0 con 0x7f5eec09a5f0 2024-09-13T18:24:46.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.446+0000 7f5efd5eb740 1 Processor -- start 2024-09-13T18:24:46.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.446+0000 7f5ef17fa640 1 -- 172.21.15.114:0/3191458712 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f5ef4068110 con 0x7f5eec09a5f0 2024-09-13T18:24:46.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.447+0000 7f5efd5eb740 1 -- start start 2024-09-13T18:24:46.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.447+0000 7f5efd5eb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d9742c970 0x556d9742cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:46.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.447+0000 7f5efd5eb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d9742d290 0x556d974375f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:46.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.447+0000 7f5efd5eb740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d97437b30 0x556d97439f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:46.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.447+0000 7f5efd5eb740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d97396cc0 con 0x556d97437b30 2024-09-13T18:24:46.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.447+0000 7f5efd5eb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d97394450 con 0x556d9742d290 2024-09-13T18:24:46.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.447+0000 7f5efd5eb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d9738fd40 con 0x556d9742c970 2024-09-13T18:24:46.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.448+0000 7f5ef3fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d9742d290 0x556d974375f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:46.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.448+0000 7f5ef3fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d9742d290 0x556d974375f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:39330/0 (socket says 172.21.15.114:39330) 2024-09-13T18:24:46.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.448+0000 7f5ef3fff640 1 -- 172.21.15.114:0/1960479521 learned_addr learned my addr 172.21.15.114:0/1960479521 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:46.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.448+0000 7f5ef37fe640 1 --2- 172.21.15.114:0/1960479521 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d97437b30 0x556d97439f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:46.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.448+0000 7f5ef17fa640 1 --2- 172.21.15.114:0/3191458712 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f5ee40a1410 0x7f5ee40a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:46.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.448+0000 7f5ef17fa640 1 -- 172.21.15.114:0/3191458712 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f5ef4136a40 con 0x7f5eec09a5f0 2024-09-13T18:24:46.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.448+0000 7f5ef3fff640 1 -- 172.21.15.114:0/1960479521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d9742c970 msgr2=0x556d9742cd50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:24:46.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.448+0000 7f5ef3fff640 1 --2- 172.21.15.114:0/1960479521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d9742c970 0x556d9742cd50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.448+0000 7f5ef3fff640 1 -- 172.21.15.114:0/1960479521 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d97437b30 msgr2=0x556d97439f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:46.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.448+0000 7f5ef3fff640 1 --2- 172.21.15.114:0/1960479521 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d97437b30 0x556d97439f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.448+0000 7f5ef3fff640 1 -- 172.21.15.114:0/1960479521 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d97357620 con 0x556d9742d290 2024-09-13T18:24:46.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.449+0000 7f5ef37fe640 1 --2- 172.21.15.114:0/1960479521 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d97437b30 0x556d97439f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:24:46.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.449+0000 7f5ef8e66640 1 --2- 172.21.15.114:0/3191458712 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f5ee40a1410 0x7f5ee40a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:46.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.449+0000 7f5ef3fff640 1 --2- 172.21.15.114:0/1960479521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d9742d290 0x556d974375f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8006010 tx=0x7f5ee80045d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ea473ad22bbaa26 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:46.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.449+0000 7f5ede7fc640 1 -- 172.21.15.114:0/1960479521 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee8003a30 con 0x556d9742d290 2024-09-13T18:24:46.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.449+0000 7f5ede7fc640 1 -- 172.21.15.114:0/1960479521 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5ee8046460 con 0x556d9742d290 2024-09-13T18:24:46.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.449+0000 7f5ede7fc640 1 -- 172.21.15.114:0/1960479521 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee803d6b0 con 0x556d9742d290 2024-09-13T18:24:46.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.450+0000 7f5efd5eb740 1 -- 172.21.15.114:0/1960479521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d9742d290 msgr2=0x556d974375f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:46.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.450+0000 7f5efd5eb740 1 --2- 172.21.15.114:0/1960479521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d9742d290 0x556d974375f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8006010 tx=0x7f5ee80045d0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.450+0000 7f5ef8e66640 1 --2- 172.21.15.114:0/3191458712 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f5ee40a1410 0x7f5ee40a38d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4100210 tx=0x7f5ef4077040 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:46.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.451+0000 7f5efd5eb740 1 -- 172.21.15.114:0/1960479521 shutdown_connections 2024-09-13T18:24:46.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.451+0000 7f5efd5eb740 1 --2- 172.21.15.114:0/1960479521 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d97437b30 0x556d97439f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.451+0000 7f5efd5eb740 1 --2- 172.21.15.114:0/1960479521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d9742d290 0x556d974375f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.451+0000 7f5efd5eb740 1 --2- 172.21.15.114:0/1960479521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d9742c970 0x556d9742cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.451+0000 7f5efd5eb740 1 -- 172.21.15.114:0/1960479521 >> 172.21.15.114:0/1960479521 conn(0x556d97428ee0 msgr2=0x556d974292c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:46.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.451+0000 7f5efd5eb740 1 -- 172.21.15.114:0/1960479521 shutdown_connections 2024-09-13T18:24:46.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.451+0000 7f5efd5eb740 1 -- 172.21.15.114:0/1960479521 wait complete. 2024-09-13T18:24:46.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.451+0000 7f5efd5eb740 1 -- start start 2024-09-13T18:24:46.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.452+0000 7f5efd5eb740 1 Processor -- start 2024-09-13T18:24:46.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.452+0000 7f5efd5eb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d9742c970 0x556d97437170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:46.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.452+0000 7f5efd5eb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d9742d290 0x556d97434500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:46.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.452+0000 7f5efd5eb740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d97437b30 0x556d97434a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:46.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.452+0000 7f5efd5eb740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d9734e180 con 0x556d97437b30 2024-09-13T18:24:46.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.452+0000 7f5efd5eb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d97068430 con 0x556d9742d290 2024-09-13T18:24:46.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.452+0000 7f5efd5eb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d973135e0 con 0x556d9742c970 2024-09-13T18:24:46.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.452+0000 7f5efd5eb740 10 client.0 _ll_register_callbacks cb 0x556d9737faa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:24:46.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.452+0000 7f5ef3fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d9742d290 0x556d97434500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:46.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.452+0000 7f5ef3fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d9742d290 0x556d97434500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:39340/0 (socket says 172.21.15.114:39340) 2024-09-13T18:24:46.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.452+0000 7f5ef3fff640 1 -- 172.21.15.114:0/1215295769 learned_addr learned my addr 172.21.15.114:0/1215295769 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:46.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.452+0000 7f5ef37fe640 1 --2- 172.21.15.114:0/1215295769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d9742c970 0x556d97437170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:46.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.452+0000 7f5ef37fe640 1 --2- 172.21.15.114:0/1215295769 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d97437b30 0x556d97434a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:46.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.452+0000 7f5ef3fff640 1 -- 172.21.15.114:0/1215295769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d9742c970 msgr2=0x556d97437170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:46.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.452+0000 7f5ef3fff640 1 --2- 172.21.15.114:0/1215295769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d9742c970 0x556d97437170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.453+0000 7f5ef3fff640 1 -- 172.21.15.114:0/1215295769 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d97437b30 msgr2=0x556d97434a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:46.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.453+0000 7f5ef3fff640 1 --2- 172.21.15.114:0/1215295769 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d97437b30 0x556d97434a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.453+0000 7f5ef3fff640 1 -- 172.21.15.114:0/1215295769 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eec098c10 con 0x556d9742d290 2024-09-13T18:24:46.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.453+0000 7f5ef3fff640 1 --2- 172.21.15.114:0/1215295769 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d9742d290 0x556d97434500 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8005ab0 tx=0x7f5ee8046b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd8bd8311a11468d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:46.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.453+0000 7f5ed37fe640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:24:46.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.453+0000 7f5ed37fe640 1 -- 172.21.15.114:0/1215295769 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee804a040 con 0x556d9742d290 2024-09-13T18:24:46.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.453+0000 7f5efd5eb740 1 -- 172.21.15.114:0/1215295769 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556d973b9b80 con 0x556d9742d290 2024-09-13T18:24:46.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.453+0000 7f5ed37fe640 1 -- 172.21.15.114:0/1215295769 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f5ee8003c40 con 0x556d9742d290 2024-09-13T18:24:46.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.453+0000 7f5ed37fe640 1 -- 172.21.15.114:0/1215295769 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ee8003e00 con 0x556d9742d290 2024-09-13T18:24:46.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.453+0000 7f5efd5eb740 10 client.25850 fetch_fsmap learned FSMap version 32 2024-09-13T18:24:46.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.453+0000 7f5efd5eb740 1 -- 172.21.15.114:0/1215295769 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x556d9738e3c0 con 0x556d9742d290 2024-09-13T18:24:46.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.454+0000 7f5ed37fe640 1 -- 172.21.15.114:0/1215295769 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x556d973b9b80 con 0x556d9742d290 2024-09-13T18:24:46.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.454+0000 7f5ed37fe640 1 -- 172.21.15.114:0/1215295769 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f5ee8031400 con 0x556d9742d290 2024-09-13T18:24:46.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.454+0000 7f5efd5eb740 10 client.25850 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:24:46.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.454+0000 7f5efd5eb740 10 client.25850 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:24:46.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.454+0000 7f5efd5eb740 20 client.25850 populate_metadata read hostname 'smithi114' 2024-09-13T18:24:46.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.454+0000 7f5efd5eb740 1 --2- 172.21.15.114:0/1215295769 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x556d974472c0 0x556d974496e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:46.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.454+0000 7f5efd5eb740 4 client.25850 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:24:46.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.454+0000 7f5efd5eb740 1 -- 172.21.15.114:0/1215295769 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556d973821f0 con 0x556d974472c0 2024-09-13T18:24:46.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.454+0000 7f5ef37fe640 1 --2- 172.21.15.114:0/1215295769 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x556d974472c0 0x556d974496e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:46.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.455+0000 7f5ef37fe640 1 --2- 172.21.15.114:0/1215295769 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x556d974472c0 0x556d974496e0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=601d649652f98031 server_cookie=6b1de41e426100ca in_seq=0 out_seq=0 2024-09-13T18:24:46.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.455+0000 7f5ed37fe640 10 client.25850 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:24:46.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.456+0000 7f5ed37fe640 1 -- 172.21.15.114:0/1215295769 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x556d973821f0 con 0x556d974472c0 2024-09-13T18:24:46.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.456+0000 7f5ed37fe640 10 client.25850 handle_command_reply: tid=0 2024-09-13T18:24:46.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.456+0000 7f5efd5eb740 1 client.25850 shutdown 2024-09-13T18:24:46.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.456+0000 7f5efd5eb740 1 -- 172.21.15.114:0/1215295769 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d9742d290 msgr2=0x556d97434500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:46.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.456+0000 7f5efd5eb740 1 --2- 172.21.15.114:0/1215295769 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d9742d290 0x556d97434500 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8005ab0 tx=0x7f5ee8046b00 comp rx=0 tx=0).stop 2024-09-13T18:24:46.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.457+0000 7f5efd5eb740 1 -- 172.21.15.114:0/1215295769 shutdown_connections 2024-09-13T18:24:46.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.457+0000 7f5efd5eb740 1 --2- 172.21.15.114:0/1215295769 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x556d974472c0 0x556d974496e0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.457+0000 7f5efd5eb740 1 --2- 172.21.15.114:0/1215295769 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d97437b30 0x556d97434a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.457+0000 7f5efd5eb740 1 --2- 172.21.15.114:0/1215295769 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d9742d290 0x556d97434500 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.457+0000 7f5efd5eb740 1 --2- 172.21.15.114:0/1215295769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d9742c970 0x556d97437170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.457+0000 7f5efd5eb740 1 -- 172.21.15.114:0/1215295769 >> 172.21.15.114:0/1215295769 conn(0x556d97428ee0 msgr2=0x556d97428230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:46.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.457+0000 7f5ed37fe640 0 client.25850 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:24:46.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.457+0000 7f5efd5eb740 1 -- 172.21.15.114:0/1215295769 shutdown_connections 2024-09-13T18:24:46.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.457+0000 7f5efd5eb740 1 -- 172.21.15.114:0/1215295769 wait complete. 2024-09-13T18:24:46.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.457+0000 7f5efd5eb740 20 client.25850 trim_cache size 0 max 16384 2024-09-13T18:24:46.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.479+0000 7f5efd5eb740 1 Processor -- start 2024-09-13T18:24:46.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.479+0000 7f5efd5eb740 1 -- start start 2024-09-13T18:24:46.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.479+0000 7f5efd5eb740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d9742c970 0x556d97432db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:46.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.479+0000 7f5efd5eb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d9742d290 0x556d974332f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:46.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.479+0000 7f5efd5eb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d97437b30 0x556d9742f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:46.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.479+0000 7f5efd5eb740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d974379a0 con 0x556d9742c970 2024-09-13T18:24:46.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.479+0000 7f5efd5eb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d9743a5e0 con 0x556d97437b30 2024-09-13T18:24:46.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.479+0000 7f5efd5eb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d97444250 con 0x556d9742d290 2024-09-13T18:24:46.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.480+0000 7f5ef37fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d97437b30 0x556d9742f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:46.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.480+0000 7f5ef3fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d9742d290 0x556d974332f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:46.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.480+0000 7f5ef37fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d97437b30 0x556d9742f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:39346/0 (socket says 172.21.15.114:39346) 2024-09-13T18:24:46.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.480+0000 7f5ef37fe640 1 -- 172.21.15.114:0/3310243940 learned_addr learned my addr 172.21.15.114:0/3310243940 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:46.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.480+0000 7f5ef37fe640 1 --2- 172.21.15.114:0/3310243940 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d9742c970 0x556d97432db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:46.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.480+0000 7f5ef37fe640 1 -- 172.21.15.114:0/3310243940 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d9742d290 msgr2=0x556d974332f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:46.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.480+0000 7f5ef37fe640 1 --2- 172.21.15.114:0/3310243940 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d9742d290 0x556d974332f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.480+0000 7f5ef37fe640 1 -- 172.21.15.114:0/3310243940 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d9742c970 msgr2=0x556d97432db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:46.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.480+0000 7f5ef37fe640 1 --2- 172.21.15.114:0/3310243940 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d9742c970 0x556d97432db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.480+0000 7f5ef37fe640 1 -- 172.21.15.114:0/3310243940 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d97330770 con 0x556d97437b30 2024-09-13T18:24:46.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.480+0000 7f5ef3fff640 1 --2- 172.21.15.114:0/3310243940 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d9742d290 0x556d974332f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:24:46.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.481+0000 7f5ef37fe640 1 --2- 172.21.15.114:0/3310243940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d97437b30 0x556d9742f540 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0005780 tx=0x7f5ee00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86352e46020f8771 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:46.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.481+0000 7f5edeffd640 1 -- 172.21.15.114:0/3310243940 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0029040 con 0x556d97437b30 2024-09-13T18:24:46.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.481+0000 7f5edeffd640 1 -- 172.21.15.114:0/3310243940 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5ee0039070 con 0x556d97437b30 2024-09-13T18:24:46.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.481+0000 7f5edeffd640 1 -- 172.21.15.114:0/3310243940 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0022b20 con 0x556d97437b30 2024-09-13T18:24:46.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.482+0000 7f5efd5eb740 1 -- 172.21.15.114:0/3310243940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d97437b30 msgr2=0x556d9742f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:46.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.482+0000 7f5efd5eb740 1 --2- 172.21.15.114:0/3310243940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d97437b30 0x556d9742f540 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0005780 tx=0x7f5ee00046c0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.482+0000 7f5efd5eb740 1 -- 172.21.15.114:0/3310243940 shutdown_connections 2024-09-13T18:24:46.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.482+0000 7f5efd5eb740 1 --2- 172.21.15.114:0/3310243940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d97437b30 0x556d9742f540 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.482+0000 7f5efd5eb740 1 --2- 172.21.15.114:0/3310243940 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d9742d290 0x556d974332f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.482+0000 7f5efd5eb740 1 --2- 172.21.15.114:0/3310243940 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d9742c970 0x556d97432db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.482+0000 7f5efd5eb740 1 -- 172.21.15.114:0/3310243940 >> 172.21.15.114:0/3310243940 conn(0x556d97428ee0 msgr2=0x556d974292c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:46.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.482+0000 7f5efd5eb740 1 -- 172.21.15.114:0/3310243940 shutdown_connections 2024-09-13T18:24:46.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.482+0000 7f5efd5eb740 1 -- 172.21.15.114:0/3310243940 wait complete. 2024-09-13T18:24:46.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.482+0000 7f5efd5eb740 1 -- start start 2024-09-13T18:24:46.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.483+0000 7f5efd5eb740 1 Processor -- start 2024-09-13T18:24:46.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.483+0000 7f5efd5eb740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d9742c970 0x556d9743ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:46.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.483+0000 7f5efd5eb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d9742d290 0x556d9743ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:46.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.483+0000 7f5efd5eb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d97437b30 0x556d9743b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:46.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.483+0000 7f5efd5eb740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d9743b8b0 con 0x556d9742c970 2024-09-13T18:24:46.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.483+0000 7f5efd5eb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d9743ba20 con 0x556d9742d290 2024-09-13T18:24:46.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.483+0000 7f5efd5eb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d9743bbe0 con 0x556d97437b30 2024-09-13T18:24:46.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.483+0000 7f5efd5eb740 10 client.0 _ll_register_callbacks cb 0x7f5ee8005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:24:46.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.483+0000 7f5ef3fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d9742d290 0x556d9743ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:46.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.483+0000 7f5ef3fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d9742d290 0x556d9743ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:39354/0 (socket says 172.21.15.114:39354) 2024-09-13T18:24:46.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.483+0000 7f5ef3fff640 1 -- 172.21.15.114:0/1156675460 learned_addr learned my addr 172.21.15.114:0/1156675460 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:46.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.484+0000 7f5ef37fe640 1 --2- 172.21.15.114:0/1156675460 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d97437b30 0x556d9743b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:46.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.484+0000 7f5ef37fe640 1 --2- 172.21.15.114:0/1156675460 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d9742c970 0x556d9743ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:46.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.484+0000 7f5ef3fff640 1 -- 172.21.15.114:0/1156675460 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d97437b30 msgr2=0x556d9743b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:46.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.484+0000 7f5ef3fff640 1 --2- 172.21.15.114:0/1156675460 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d97437b30 0x556d9743b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.484+0000 7f5ef3fff640 1 -- 172.21.15.114:0/1156675460 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d9742c970 msgr2=0x556d9743ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:46.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.484+0000 7f5ef3fff640 1 --2- 172.21.15.114:0/1156675460 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d9742c970 0x556d9743ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.484+0000 7f5ef3fff640 1 -- 172.21.15.114:0/1156675460 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556d9738e3c0 con 0x556d9742d290 2024-09-13T18:24:46.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.484+0000 7f5ef37fe640 1 --2- 172.21.15.114:0/1156675460 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d9742c970 0x556d9743ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:24:46.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.484+0000 7f5ef3fff640 1 --2- 172.21.15.114:0/1156675460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d9742d290 0x556d9743ae00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5ee802a430 tx=0x7f5ee803ae90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=985f652cdc5b6c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:46.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.484+0000 7f5ed37fe640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:24:46.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.485+0000 7f5ed37fe640 1 -- 172.21.15.114:0/1156675460 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee804a040 con 0x556d9742d290 2024-09-13T18:24:46.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.485+0000 7f5ed37fe640 1 -- 172.21.15.114:0/1156675460 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f5ee8031a30 con 0x556d9742d290 2024-09-13T18:24:46.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.485+0000 7f5efd5eb740 1 -- 172.21.15.114:0/1156675460 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5ee8046460 con 0x556d9742d290 2024-09-13T18:24:46.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.485+0000 7f5ed37fe640 1 -- 172.21.15.114:0/1156675460 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ee8004500 con 0x556d9742d290 2024-09-13T18:24:46.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.485+0000 7f5efd5eb740 10 client.25856 fetch_fsmap learned FSMap version 32 2024-09-13T18:24:46.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.485+0000 7f5efd5eb740 1 -- 172.21.15.114:0/1156675460 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f5ee002c040 con 0x556d9742d290 2024-09-13T18:24:46.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.486+0000 7f5ed37fe640 1 -- 172.21.15.114:0/1156675460 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5ee8046460 con 0x556d9742d290 2024-09-13T18:24:46.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.486+0000 7f5ed37fe640 1 -- 172.21.15.114:0/1156675460 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f5ee8058070 con 0x556d9742d290 2024-09-13T18:24:46.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.486+0000 7f5efd5eb740 10 client.25856 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:24:46.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.486+0000 7f5efd5eb740 10 client.25856 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:24:46.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.486+0000 7f5efd5eb740 20 client.25856 populate_metadata read hostname 'smithi114' 2024-09-13T18:24:46.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.486+0000 7f5efd5eb740 1 --2- 172.21.15.114:0/1156675460 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x556d9749e910 0x556d974a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:46.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.486+0000 7f5efd5eb740 4 client.25856 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:24:46.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.486+0000 7f5efd5eb740 1 -- 172.21.15.114:0/1156675460 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556d970834e0 con 0x556d9749e910 2024-09-13T18:24:46.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.487+0000 7f5ef37fe640 1 --2- 172.21.15.114:0/1156675460 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x556d9749e910 0x556d974a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:46.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.488+0000 7f5ef37fe640 1 --2- 172.21.15.114:0/1156675460 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x556d9749e910 0x556d974a0d30 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94a9ed29ed7d702d server_cookie=b5c7d601ce49aa3b in_seq=0 out_seq=0 2024-09-13T18:24:46.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.488+0000 7f5ed37fe640 10 client.25856 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:24:46.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.488+0000 7f5ed37fe640 1 -- 172.21.15.114:0/1156675460 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556d970834e0 con 0x556d9749e910 2024-09-13T18:24:46.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.488+0000 7f5ed37fe640 10 client.25856 handle_command_reply: tid=0 2024-09-13T18:24:46.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.488+0000 7f5efd5eb740 1 client.25856 shutdown 2024-09-13T18:24:46.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.488+0000 7f5efd5eb740 1 -- 172.21.15.114:0/1156675460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d9742d290 msgr2=0x556d9743ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:46.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.488+0000 7f5efd5eb740 1 --2- 172.21.15.114:0/1156675460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d9742d290 0x556d9743ae00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5ee802a430 tx=0x7f5ee803ae90 comp rx=0 tx=0).stop 2024-09-13T18:24:46.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.488+0000 7f5efd5eb740 1 -- 172.21.15.114:0/1156675460 shutdown_connections 2024-09-13T18:24:46.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.488+0000 7f5efd5eb740 1 --2- 172.21.15.114:0/1156675460 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x556d9749e910 0x556d974a0d30 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.488+0000 7f5efd5eb740 1 --2- 172.21.15.114:0/1156675460 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d97437b30 0x556d9743b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.488+0000 7f5efd5eb740 1 --2- 172.21.15.114:0/1156675460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d9742d290 0x556d9743ae00 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.488+0000 7f5efd5eb740 1 --2- 172.21.15.114:0/1156675460 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d9742c970 0x556d9743ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.488+0000 7f5efd5eb740 1 -- 172.21.15.114:0/1156675460 >> 172.21.15.114:0/1156675460 conn(0x556d97428ee0 msgr2=0x556d974432e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:46.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.489+0000 7f5ed37fe640 0 client.25856 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:24:46.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.489+0000 7f5efd5eb740 1 -- 172.21.15.114:0/1156675460 shutdown_connections 2024-09-13T18:24:46.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.489+0000 7f5efd5eb740 1 -- 172.21.15.114:0/1156675460 wait complete. 2024-09-13T18:24:46.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.490+0000 7f5efd5eb740 20 client.25856 trim_cache size 0 max 16384 2024-09-13T18:24:46.495 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:24:46.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.490+0000 7f5efa8c9640 1 -- 172.21.15.114:0/3191458712 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f5ee40a1410 msgr2=0x7f5ee40a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:46.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.490+0000 7f5efa8c9640 1 --2- 172.21.15.114:0/3191458712 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f5ee40a1410 0x7f5ee40a38d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4100210 tx=0x7f5ef4077040 comp rx=0 tx=0).stop 2024-09-13T18:24:46.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.490+0000 7f5efa8c9640 1 -- 172.21.15.114:0/3191458712 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5eec09a5f0 msgr2=0x7f5eec136930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:46.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.491+0000 7f5efa8c9640 1 --2- 172.21.15.114:0/3191458712 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5eec09a5f0 0x7f5eec136930 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f5ef404efc0 tx=0x7f5ef40696c0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.491+0000 7f5efa8c9640 1 -- 172.21.15.114:0/3191458712 shutdown_connections 2024-09-13T18:24:46.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.491+0000 7f5efa8c9640 1 --2- 172.21.15.114:0/3191458712 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f5ee40a1410 0x7f5ee40a38d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.491+0000 7f5efa8c9640 1 --2- 172.21.15.114:0/3191458712 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5eec09a5f0 0x7f5eec136930 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.491+0000 7f5efa8c9640 1 --2- 172.21.15.114:0/3191458712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5eec006f50 0x7f5eec132590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.491+0000 7f5efa8c9640 1 --2- 172.21.15.114:0/3191458712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5eec0064e0 0x7f5eec132050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:46.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.491+0000 7f5efa8c9640 1 -- 172.21.15.114:0/3191458712 >> 172.21.15.114:0/3191458712 conn(0x7f5eec00a130 msgr2=0x7f5eec091f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:46.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.491+0000 7f5efa8c9640 1 -- 172.21.15.114:0/3191458712 shutdown_connections 2024-09-13T18:24:46.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:46.491+0000 7f5efa8c9640 1 -- 172.21.15.114:0/3191458712 wait complete. 2024-09-13T18:24:46.520 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:46 smithi134 ceph-mon[34283]: pgmap v72: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 92 MiB/s wr, 547 op/s 2024-09-13T18:24:46.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:46 smithi204 ceph-mon[34777]: pgmap v72: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 92 MiB/s wr, 547 op/s 2024-09-13T18:24:47.512 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-13T18:24:47.513 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-09-13T18:24:47.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.690+0000 7f88b4a4e640 1 -- 172.21.15.114:0/214694260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88a8006eb0 msgr2=0x7f88a8007310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:47.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.690+0000 7f88b4a4e640 1 --2- 172.21.15.114:0/214694260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88a8006eb0 0x7f88a8007310 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8898009900 tx=0x7f889802a400 comp rx=0 tx=0).stop 2024-09-13T18:24:47.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.690+0000 7f88b4a4e640 1 -- 172.21.15.114:0/214694260 shutdown_connections 2024-09-13T18:24:47.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.690+0000 7f88b4a4e640 1 --2- 172.21.15.114:0/214694260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88a80a1ec0 0x7f88a80a42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:47.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.690+0000 7f88b4a4e640 1 --2- 172.21.15.114:0/214694260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88a8006eb0 0x7f88a8007310 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:47.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.690+0000 7f88b4a4e640 1 --2- 172.21.15.114:0/214694260 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88a80086c0 0x7f88a80068e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:47.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.690+0000 7f88b4a4e640 1 -- 172.21.15.114:0/214694260 >> 172.21.15.114:0/214694260 conn(0x7f88a800d6f0 msgr2=0x7f88a800db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:47.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.691+0000 7f88b4a4e640 1 -- 172.21.15.114:0/214694260 shutdown_connections 2024-09-13T18:24:47.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.691+0000 7f88b4a4e640 1 -- 172.21.15.114:0/214694260 wait complete. 2024-09-13T18:24:47.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.692+0000 7f88b4a4e640 1 Processor -- start 2024-09-13T18:24:47.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.692+0000 7f88b4a4e640 1 -- start start 2024-09-13T18:24:47.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.693+0000 7f88b4a4e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88a8006eb0 0x7f88a813b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:47.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.693+0000 7f88b4a4e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88a80086c0 0x7f88a813b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:47.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.693+0000 7f88b4a4e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88a80a1ec0 0x7f88a813ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:47.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.693+0000 7f88b4a4e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a813f590 con 0x7f88a80086c0 2024-09-13T18:24:47.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.693+0000 7f88b4a4e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a813f700 con 0x7f88a8006eb0 2024-09-13T18:24:47.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.693+0000 7f88b4a4e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a813f8c0 con 0x7f88a80a1ec0 2024-09-13T18:24:47.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.693+0000 7f88ae59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88a80086c0 0x7f88a813b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:47.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.693+0000 7f88ae59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88a80086c0 0x7f88a813b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51820/0 (socket says 172.21.15.114:51820) 2024-09-13T18:24:47.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.693+0000 7f88ae59c640 1 -- 172.21.15.114:0/3603166289 learned_addr learned my addr 172.21.15.114:0/3603166289 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:47.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.693+0000 7f88ae59c640 1 -- 172.21.15.114:0/3603166289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88a80a1ec0 msgr2=0x7f88a813ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:47.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.693+0000 7f88ae59c640 1 --2- 172.21.15.114:0/3603166289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88a80a1ec0 0x7f88a813ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:47.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.693+0000 7f88ae59c640 1 -- 172.21.15.114:0/3603166289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88a8006eb0 msgr2=0x7f88a813b020 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:24:47.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.693+0000 7f88ae59c640 1 --2- 172.21.15.114:0/3603166289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88a8006eb0 0x7f88a813b020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:47.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.693+0000 7f88ae59c640 1 -- 172.21.15.114:0/3603166289 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8898009560 con 0x7f88a80086c0 2024-09-13T18:24:47.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.694+0000 7f88ae59c640 1 --2- 172.21.15.114:0/3603166289 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88a80086c0 0x7f88a813b560 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f8898009850 tx=0x7f8898004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:47.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.694+0000 7f88a7fff640 1 -- 172.21.15.114:0/3603166289 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8898004520 con 0x7f88a80086c0 2024-09-13T18:24:47.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.694+0000 7f88a7fff640 1 -- 172.21.15.114:0/3603166289 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8898003a80 con 0x7f88a80086c0 2024-09-13T18:24:47.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.695+0000 7f88b4a4e640 1 -- 172.21.15.114:0/3603166289 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88a813fb40 con 0x7f88a80086c0 2024-09-13T18:24:47.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.695+0000 7f88a7fff640 1 -- 172.21.15.114:0/3603166289 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889803ca90 con 0x7f88a80086c0 2024-09-13T18:24:47.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.695+0000 7f88b4a4e640 1 -- 172.21.15.114:0/3603166289 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88a8147460 con 0x7f88a80086c0 2024-09-13T18:24:47.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.698+0000 7f88b804b740 1 Processor -- start 2024-09-13T18:24:47.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.699+0000 7f88b804b740 1 -- start start 2024-09-13T18:24:47.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.699+0000 7f88b804b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2064649a0 0x55f206464d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:47.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.699+0000 7f88b804b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2064652c0 0x55f20646f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:47.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.699+0000 7f88b804b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f20646fb60 0x55f206471f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:47.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.699+0000 7f88b804b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2063cfc60 con 0x55f20646fb60 2024-09-13T18:24:47.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.699+0000 7f88b804b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2063cd3f0 con 0x55f2064649a0 2024-09-13T18:24:47.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.699+0000 7f88b804b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2063c8ce0 con 0x55f2064652c0 2024-09-13T18:24:47.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.699+0000 7f88aed9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2064652c0 0x55f20646f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:47.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.699+0000 7f88aed9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2064652c0 0x55f20646f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:57622/0 (socket says 172.21.15.114:57622) 2024-09-13T18:24:47.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.699+0000 7f88aed9d640 1 -- 172.21.15.114:0/739467814 learned_addr learned my addr 172.21.15.114:0/739467814 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:47.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.700+0000 7f88a7fff640 1 -- 172.21.15.114:0/3603166289 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f889803cbf0 con 0x7f88a80086c0 2024-09-13T18:24:47.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.700+0000 7f88af59e640 1 --2- 172.21.15.114:0/739467814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2064649a0 0x55f206464d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:47.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.700+0000 7f88aed9d640 1 -- 172.21.15.114:0/739467814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2064649a0 msgr2=0x55f206464d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:47.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.700+0000 7f88aed9d640 1 --2- 172.21.15.114:0/739467814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2064649a0 0x55f206464d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:47.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.700+0000 7f88aed9d640 1 -- 172.21.15.114:0/739467814 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f20646fb60 msgr2=0x55f206471f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:47.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.700+0000 7f88aed9d640 1 --2- 172.21.15.114:0/739467814 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f20646fb60 0x55f206471f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:47.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.700+0000 7f88aed9d640 1 -- 172.21.15.114:0/739467814 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f206384970 con 0x55f2064652c0 2024-09-13T18:24:47.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.700+0000 7f88af59e640 1 --2- 172.21.15.114:0/739467814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2064649a0 0x55f206464d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:24:47.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.700+0000 7f88aed9d640 1 --2- 172.21.15.114:0/739467814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2064652c0 0x55f20646f620 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f88a0009bd0 tx=0x7f88a000e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98b799a398b37bf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:47.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.701+0000 7f88a57fa640 1 -- 172.21.15.114:0/739467814 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a0009720 con 0x55f2064652c0 2024-09-13T18:24:47.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.701+0000 7f88a57fa640 1 -- 172.21.15.114:0/739467814 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f88a00054b0 con 0x55f2064652c0 2024-09-13T18:24:47.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.701+0000 7f88a57fa640 1 -- 172.21.15.114:0/739467814 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a0010430 con 0x55f2064652c0 2024-09-13T18:24:47.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.701+0000 7f88a7fff640 1 --2- 172.21.15.114:0/3603166289 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f889c0a1280 0x7f889c0a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:47.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.701+0000 7f88a7fff640 1 -- 172.21.15.114:0/3603166289 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f88980ee300 con 0x7f88a80086c0 2024-09-13T18:24:47.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.702+0000 7f88af59e640 1 --2- 172.21.15.114:0/3603166289 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f889c0a1280 0x7f889c0a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:47.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.702+0000 7f88b804b740 1 -- 172.21.15.114:0/739467814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2064652c0 msgr2=0x55f20646f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:47.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.702+0000 7f88b804b740 1 --2- 172.21.15.114:0/739467814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2064652c0 0x55f20646f620 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f88a0009bd0 tx=0x7f88a000e720 comp rx=0 tx=0).stop 2024-09-13T18:24:47.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.702+0000 7f88b804b740 1 -- 172.21.15.114:0/739467814 shutdown_connections 2024-09-13T18:24:47.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.702+0000 7f88b804b740 1 --2- 172.21.15.114:0/739467814 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f20646fb60 0x55f206471f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:47.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.702+0000 7f88b804b740 1 --2- 172.21.15.114:0/739467814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2064652c0 0x55f20646f620 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:47.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.702+0000 7f88b804b740 1 --2- 172.21.15.114:0/739467814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2064649a0 0x55f206464d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:47.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.702+0000 7f88b804b740 1 -- 172.21.15.114:0/739467814 >> 172.21.15.114:0/739467814 conn(0x55f206460f30 msgr2=0x55f206461310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:47.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.703+0000 7f88b804b740 1 -- 172.21.15.114:0/739467814 shutdown_connections 2024-09-13T18:24:47.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.703+0000 7f88af59e640 1 --2- 172.21.15.114:0/3603166289 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f889c0a1280 0x7f889c0a3740 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f88b0063850 tx=0x7f88b006a040 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:47.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.703+0000 7f88b804b740 1 -- 172.21.15.114:0/739467814 wait complete. 2024-09-13T18:24:47.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.703+0000 7f88b804b740 1 -- start start 2024-09-13T18:24:47.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.704+0000 7f88b804b740 1 Processor -- start 2024-09-13T18:24:47.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.704+0000 7f88b804b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2064649a0 0x55f20646f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:47.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.704+0000 7f88b804b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2064652c0 0x55f20646c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:47.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.705+0000 7f88aed9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2064649a0 0x55f20646f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:47.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.705+0000 7f88aed9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2064649a0 0x55f20646f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:57628/0 (socket says 172.21.15.114:57628) 2024-09-13T18:24:47.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.705+0000 7f88aed9d640 1 -- 172.21.15.114:0/2875730273 learned_addr learned my addr 172.21.15.114:0/2875730273 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:47.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.705+0000 7f88b804b740 1 --2- 172.21.15.114:0/2875730273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f20646fb60 0x55f20646cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:47.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.705+0000 7f88ae59c640 1 --2- 172.21.15.114:0/2875730273 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2064652c0 0x55f20646c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:47.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.705+0000 7f88af59e640 1 --2- 172.21.15.114:0/2875730273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f20646fb60 0x55f20646cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:47.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.705+0000 7f88b804b740 1 -- 172.21.15.114:0/2875730273 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f206386f80 con 0x55f2064652c0 2024-09-13T18:24:47.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.705+0000 7f88b804b740 1 -- 172.21.15.114:0/2875730273 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f206386c20 con 0x55f20646fb60 2024-09-13T18:24:47.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.705+0000 7f88b804b740 1 -- 172.21.15.114:0/2875730273 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f206353ab0 con 0x55f2064649a0 2024-09-13T18:24:47.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.706+0000 7f88b804b740 10 client.0 _ll_register_callbacks cb 0x55f2063c2350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:24:47.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.706+0000 7f88ae59c640 1 -- 172.21.15.114:0/2875730273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2064649a0 msgr2=0x55f20646f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:47.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.706+0000 7f88ae59c640 1 --2- 172.21.15.114:0/2875730273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2064649a0 0x55f20646f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:47.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.706+0000 7f88ae59c640 1 -- 172.21.15.114:0/2875730273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f20646fb60 msgr2=0x55f20646cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:47.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.706+0000 7f88ae59c640 1 --2- 172.21.15.114:0/2875730273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f20646fb60 0x55f20646cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:47.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.706+0000 7f88ae59c640 1 -- 172.21.15.114:0/2875730273 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88a0009880 con 0x55f2064652c0 2024-09-13T18:24:47.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.707+0000 7f88ae59c640 1 --2- 172.21.15.114:0/2875730273 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2064652c0 0x55f20646c740 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f88980eb6a0 tx=0x7f88980bac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fd9a2f5134d1dca server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:47.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.707+0000 7f88967fc640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:24:47.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.708+0000 7f88967fc640 1 -- 172.21.15.114:0/2875730273 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88980bb060 con 0x55f2064652c0 2024-09-13T18:24:47.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.708+0000 7f88967fc640 1 -- 172.21.15.114:0/2875730273 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f88980bee90 con 0x55f2064652c0 2024-09-13T18:24:47.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.708+0000 7f88b804b740 1 -- 172.21.15.114:0/2875730273 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f2060bc590 con 0x55f2064652c0 2024-09-13T18:24:47.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.708+0000 7f88967fc640 1 -- 172.21.15.114:0/2875730273 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88980c34e0 con 0x55f2064652c0 2024-09-13T18:24:47.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.708+0000 7f88b804b740 10 client.16542 fetch_fsmap learned FSMap version 32 2024-09-13T18:24:47.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.708+0000 7f88b804b740 1 -- 172.21.15.114:0/2875730273 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55f2063c7360 con 0x55f2064652c0 2024-09-13T18:24:47.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.708+0000 7f88967fc640 1 -- 172.21.15.114:0/2875730273 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f2060bc590 con 0x55f2064652c0 2024-09-13T18:24:47.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.708+0000 7f88967fc640 1 -- 172.21.15.114:0/2875730273 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f8898003bf0 con 0x55f2064652c0 2024-09-13T18:24:47.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.709+0000 7f88b804b740 10 client.16542 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:24:47.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.709+0000 7f88b804b740 10 client.16542 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:24:47.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.709+0000 7f88b804b740 20 client.16542 populate_metadata read hostname 'smithi114' 2024-09-13T18:24:47.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.709+0000 7f88b804b740 1 --2- 172.21.15.114:0/2875730273 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f20647f780 0x55f206481ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:47.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.709+0000 7f88b804b740 4 client.16542 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:24:47.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.709+0000 7f88b804b740 1 -- 172.21.15.114:0/2875730273 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f2060cd860 con 0x55f20647f780 2024-09-13T18:24:47.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.709+0000 7f88aed9d640 1 --2- 172.21.15.114:0/2875730273 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f20647f780 0x55f206481ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:47.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.709+0000 7f88aed9d640 1 --2- 172.21.15.114:0/2875730273 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f20647f780 0x55f206481ba0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7a297b228a2a898 server_cookie=21d52cc75d896c86 in_seq=0 out_seq=0 2024-09-13T18:24:47.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.709+0000 7f88967fc640 10 client.16542 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:24:47.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.710+0000 7f88967fc640 1 -- 172.21.15.114:0/2875730273 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55f2060cd860 con 0x55f20647f780 2024-09-13T18:24:47.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.710+0000 7f88967fc640 10 client.16542 handle_command_reply: tid=0 2024-09-13T18:24:47.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.710+0000 7f88b804b740 1 client.16542 shutdown 2024-09-13T18:24:47.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.710+0000 7f88b804b740 1 -- 172.21.15.114:0/2875730273 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2064652c0 msgr2=0x55f20646c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:47.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.710+0000 7f88b804b740 1 --2- 172.21.15.114:0/2875730273 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2064652c0 0x55f20646c740 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f88980eb6a0 tx=0x7f88980bac60 comp rx=0 tx=0).stop 2024-09-13T18:24:47.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.711+0000 7f88b804b740 1 -- 172.21.15.114:0/2875730273 shutdown_connections 2024-09-13T18:24:47.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.711+0000 7f88b804b740 1 --2- 172.21.15.114:0/2875730273 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f20647f780 0x55f206481ba0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:47.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.711+0000 7f88b804b740 1 --2- 172.21.15.114:0/2875730273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f20646fb60 0x55f20646cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:47.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.711+0000 7f88b804b740 1 --2- 172.21.15.114:0/2875730273 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2064652c0 0x55f20646c740 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:47.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.711+0000 7f88b804b740 1 --2- 172.21.15.114:0/2875730273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2064649a0 0x55f20646f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:47.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.711+0000 7f88967fc640 0 client.16542 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:24:47.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.711+0000 7f88b804b740 1 -- 172.21.15.114:0/2875730273 >> 172.21.15.114:0/2875730273 conn(0x55f206460f30 msgr2=0x55f206460160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:47.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.711+0000 7f88b804b740 1 -- 172.21.15.114:0/2875730273 shutdown_connections 2024-09-13T18:24:47.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.711+0000 7f88b804b740 1 -- 172.21.15.114:0/2875730273 wait complete. 2024-09-13T18:24:47.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.711+0000 7f88b804b740 20 client.16542 trim_cache size 0 max 16384 2024-09-13T18:24:47.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.739+0000 7f88b804b740 1 Processor -- start 2024-09-13T18:24:47.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.740+0000 7f88b804b740 1 -- start start 2024-09-13T18:24:47.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.740+0000 7f88b804b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2064649a0 0x55f20646c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:47.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.741+0000 7f88b804b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2064652c0 0x55f206460950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:47.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.741+0000 7f88b804b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f20646fb60 0x55f206461310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:47.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.741+0000 7f88aed9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2064649a0 0x55f20646c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:47.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.741+0000 7f88aed9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2064649a0 0x55f20646c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51846/0 (socket says 172.21.15.114:51846) 2024-09-13T18:24:47.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.741+0000 7f88aed9d640 1 -- 172.21.15.114:0/3073764824 learned_addr learned my addr 172.21.15.114:0/3073764824 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:47.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.741+0000 7f88b804b740 1 -- 172.21.15.114:0/3073764824 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f206465f30 con 0x55f2064649a0 2024-09-13T18:24:47.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.741+0000 7f88b804b740 1 -- 172.21.15.114:0/3073764824 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2064657e0 con 0x55f20646fb60 2024-09-13T18:24:47.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.741+0000 7f88b804b740 1 -- 172.21.15.114:0/3073764824 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f206461850 con 0x55f2064652c0 2024-09-13T18:24:47.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.741+0000 7f88ae59c640 1 --2- 172.21.15.114:0/3073764824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2064652c0 0x55f206460950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:47.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.741+0000 7f88af59e640 1 --2- 172.21.15.114:0/3073764824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f20646fb60 0x55f206461310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:47.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.741+0000 7f88ae59c640 1 -- 172.21.15.114:0/3073764824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f20646fb60 msgr2=0x55f206461310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:47.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.741+0000 7f88ae59c640 1 --2- 172.21.15.114:0/3073764824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f20646fb60 0x55f206461310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:47.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.741+0000 7f88ae59c640 1 -- 172.21.15.114:0/3073764824 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2064649a0 msgr2=0x55f20646c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:47.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.741+0000 7f88ae59c640 1 --2- 172.21.15.114:0/3073764824 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2064649a0 0x55f20646c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:47.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.741+0000 7f88ae59c640 1 -- 172.21.15.114:0/3073764824 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2063691c0 con 0x55f2064652c0 2024-09-13T18:24:47.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.742+0000 7f88ae59c640 1 --2- 172.21.15.114:0/3073764824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2064652c0 0x55f206460950 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x55f20646dd60 tx=0x7f88980bacd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b78a1c743a7ea0a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:47.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.742+0000 7f88a5ffb640 1 -- 172.21.15.114:0/3073764824 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889803cea0 con 0x55f2064652c0 2024-09-13T18:24:47.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.742+0000 7f88a5ffb640 1 -- 172.21.15.114:0/3073764824 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f88980ba550 con 0x55f2064652c0 2024-09-13T18:24:47.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.742+0000 7f88a5ffb640 1 -- 172.21.15.114:0/3073764824 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88980b9400 con 0x55f2064652c0 2024-09-13T18:24:47.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.743+0000 7f88b804b740 1 -- 172.21.15.114:0/3073764824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2064652c0 msgr2=0x55f206460950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:47.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.743+0000 7f88b804b740 1 --2- 172.21.15.114:0/3073764824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2064652c0 0x55f206460950 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x55f20646dd60 tx=0x7f88980bacd0 comp rx=0 tx=0).stop 2024-09-13T18:24:47.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.744+0000 7f88b804b740 1 -- 172.21.15.114:0/3073764824 shutdown_connections 2024-09-13T18:24:47.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.744+0000 7f88b804b740 1 --2- 172.21.15.114:0/3073764824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f20646fb60 0x55f206461310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:47.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.744+0000 7f88b804b740 1 --2- 172.21.15.114:0/3073764824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2064652c0 0x55f206460950 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:47.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.744+0000 7f88b804b740 1 --2- 172.21.15.114:0/3073764824 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2064649a0 0x55f20646c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:47.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.744+0000 7f88b804b740 1 -- 172.21.15.114:0/3073764824 >> 172.21.15.114:0/3073764824 conn(0x55f206460f30 msgr2=0x55f20646bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:47.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.744+0000 7f88b804b740 1 -- 172.21.15.114:0/3073764824 shutdown_connections 2024-09-13T18:24:47.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.744+0000 7f88b804b740 1 -- 172.21.15.114:0/3073764824 wait complete. 2024-09-13T18:24:47.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.745+0000 7f88b804b740 1 -- start start 2024-09-13T18:24:47.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.745+0000 7f88b804b740 1 Processor -- start 2024-09-13T18:24:47.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.745+0000 7f88b804b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2064649a0 0x55f206469680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:47.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.745+0000 7f88b804b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2064652c0 0x55f206469bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:47.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.745+0000 7f88b804b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f20646fb60 0x55f20646a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:47.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.745+0000 7f88b804b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2064d5ce0 con 0x55f2064649a0 2024-09-13T18:24:47.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.745+0000 7f88b804b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2064d5e20 con 0x55f20646fb60 2024-09-13T18:24:47.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.745+0000 7f88b804b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2064d5fe0 con 0x55f2064652c0 2024-09-13T18:24:47.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.745+0000 7f88b804b740 10 client.0 _ll_register_callbacks cb 0x7f8898005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:24:47.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.745+0000 7f88aed9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2064649a0 0x55f206469680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:47.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.746+0000 7f88aed9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2064649a0 0x55f206469680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51850/0 (socket says 172.21.15.114:51850) 2024-09-13T18:24:47.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.746+0000 7f88aed9d640 1 -- 172.21.15.114:0/724847984 learned_addr learned my addr 172.21.15.114:0/724847984 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:47.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.746+0000 7f88ae59c640 1 --2- 172.21.15.114:0/724847984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2064652c0 0x55f206469bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:47.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.746+0000 7f88af59e640 1 --2- 172.21.15.114:0/724847984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f20646fb60 0x55f20646a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:47.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.746+0000 7f88aed9d640 1 -- 172.21.15.114:0/724847984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2064652c0 msgr2=0x55f206469bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:47.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.746+0000 7f88aed9d640 1 --2- 172.21.15.114:0/724847984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2064652c0 0x55f206469bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:47.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.746+0000 7f88aed9d640 1 -- 172.21.15.114:0/724847984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f20646fb60 msgr2=0x55f20646a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:47.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.746+0000 7f88aed9d640 1 --2- 172.21.15.114:0/724847984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f20646fb60 0x55f20646a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:47.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.746+0000 7f88aed9d640 1 -- 172.21.15.114:0/724847984 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f2063c7360 con 0x55f2064649a0 2024-09-13T18:24:47.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.746+0000 7f88ae59c640 1 --2- 172.21.15.114:0/724847984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2064652c0 0x55f206469bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:24:47.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.746+0000 7f88af59e640 1 --2- 172.21.15.114:0/724847984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f20646fb60 0x55f20646a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:24:47.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.746+0000 7f88aed9d640 1 --2- 172.21.15.114:0/724847984 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2064649a0 0x55f206469680 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f88a0009ba0 tx=0x7f88a0025670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5109d91ff6f57cf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:47.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.746+0000 7f88967fc640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:24:47.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.747+0000 7f88967fc640 1 -- 172.21.15.114:0/724847984 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a0005310 con 0x55f2064649a0 2024-09-13T18:24:47.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.747+0000 7f88967fc640 1 -- 172.21.15.114:0/724847984 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f88a0014070 con 0x55f2064649a0 2024-09-13T18:24:47.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.747+0000 7f88b804b740 1 -- 172.21.15.114:0/724847984 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f88a00054b0 con 0x55f2064649a0 2024-09-13T18:24:47.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.747+0000 7f88967fc640 1 -- 172.21.15.114:0/724847984 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88a00257e0 con 0x55f2064649a0 2024-09-13T18:24:47.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.747+0000 7f88b804b740 10 client.16548 fetch_fsmap learned FSMap version 32 2024-09-13T18:24:47.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.747+0000 7f88b804b740 1 -- 172.21.15.114:0/724847984 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55f20638e990 con 0x55f2064649a0 2024-09-13T18:24:47.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.747+0000 7f88967fc640 1 -- 172.21.15.114:0/724847984 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f88a00054b0 con 0x55f2064649a0 2024-09-13T18:24:47.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.747+0000 7f88967fc640 1 -- 172.21.15.114:0/724847984 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f88a0028610 con 0x55f2064649a0 2024-09-13T18:24:47.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.747+0000 7f88b804b740 10 client.16548 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:24:47.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.747+0000 7f88b804b740 10 client.16548 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:24:47.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.747+0000 7f88b804b740 20 client.16548 populate_metadata read hostname 'smithi114' 2024-09-13T18:24:47.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.748+0000 7f88b804b740 1 --2- 172.21.15.114:0/724847984 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f2064d7900 0x55f2064d9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:47.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.748+0000 7f88b804b740 4 client.16548 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-13T18:24:47.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.748+0000 7f88b804b740 1 -- 172.21.15.114:0/724847984 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55f206120eb0 con 0x55f2064d7900 2024-09-13T18:24:47.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.748+0000 7f88ae59c640 1 --2- 172.21.15.114:0/724847984 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f2064d7900 0x55f2064d9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:47.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.749+0000 7f88ae59c640 1 --2- 172.21.15.114:0/724847984 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f2064d7900 0x55f2064d9d20 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34058ccbe0a7cfa1 server_cookie=4451c39196b7493c in_seq=0 out_seq=0 2024-09-13T18:24:47.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.749+0000 7f88967fc640 10 client.16548 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:24:47.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.750+0000 7f88967fc640 1 -- 172.21.15.114:0/724847984 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f206120eb0 con 0x55f2064d7900 2024-09-13T18:24:47.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.750+0000 7f88967fc640 10 client.16548 handle_command_reply: tid=0 2024-09-13T18:24:47.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.750+0000 7f88b804b740 1 client.16548 shutdown 2024-09-13T18:24:47.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.750+0000 7f88b804b740 1 -- 172.21.15.114:0/724847984 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2064649a0 msgr2=0x55f206469680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:47.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.751+0000 7f88b804b740 1 --2- 172.21.15.114:0/724847984 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2064649a0 0x55f206469680 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f88a0009ba0 tx=0x7f88a0025670 comp rx=0 tx=0).stop 2024-09-13T18:24:47.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.751+0000 7f88b804b740 1 -- 172.21.15.114:0/724847984 shutdown_connections 2024-09-13T18:24:47.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.751+0000 7f88b804b740 1 --2- 172.21.15.114:0/724847984 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f2064d7900 0x55f2064d9d20 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:47.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.751+0000 7f88b804b740 1 --2- 172.21.15.114:0/724847984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f20646fb60 0x55f20646a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:47.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.751+0000 7f88b804b740 1 --2- 172.21.15.114:0/724847984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2064652c0 0x55f206469bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:47.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.751+0000 7f88b804b740 1 --2- 172.21.15.114:0/724847984 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2064649a0 0x55f206469680 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:47.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.751+0000 7f88b804b740 1 -- 172.21.15.114:0/724847984 >> 172.21.15.114:0/724847984 conn(0x55f206460f30 msgr2=0x55f20646e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:47.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.751+0000 7f88967fc640 0 client.16548 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:24:47.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.751+0000 7f88b804b740 1 -- 172.21.15.114:0/724847984 shutdown_connections 2024-09-13T18:24:47.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.751+0000 7f88b804b740 1 -- 172.21.15.114:0/724847984 wait complete. 2024-09-13T18:24:47.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.752+0000 7f88b804b740 20 client.16548 trim_cache size 0 max 16384 2024-09-13T18:24:47.758 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:24:47.758 INFO:teuthology.orchestra.run.smithi114.stdout: "return_code": 0, 2024-09-13T18:24:47.758 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_tag": "181907a9-a1bd-420d-ad64-5c2aa352ef01", 2024-09-13T18:24:47.758 INFO:teuthology.orchestra.run.smithi114.stdout: "mode": "asynchronous" 2024-09-13T18:24:47.758 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:24:47.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.754+0000 7f88b4a4e640 1 -- 172.21.15.114:0/3603166289 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f889c0a1280 msgr2=0x7f889c0a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:47.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.754+0000 7f88b4a4e640 1 --2- 172.21.15.114:0/3603166289 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f889c0a1280 0x7f889c0a3740 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f88b0063850 tx=0x7f88b006a040 comp rx=0 tx=0).stop 2024-09-13T18:24:47.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.754+0000 7f88b4a4e640 1 -- 172.21.15.114:0/3603166289 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88a80086c0 msgr2=0x7f88a813b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:47.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.754+0000 7f88b4a4e640 1 --2- 172.21.15.114:0/3603166289 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88a80086c0 0x7f88a813b560 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f8898009850 tx=0x7f8898004ce0 comp rx=0 tx=0).stop 2024-09-13T18:24:47.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.754+0000 7f88b4a4e640 1 -- 172.21.15.114:0/3603166289 shutdown_connections 2024-09-13T18:24:47.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.754+0000 7f88b4a4e640 1 --2- 172.21.15.114:0/3603166289 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f889c0a1280 0x7f889c0a3740 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:47.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.754+0000 7f88b4a4e640 1 --2- 172.21.15.114:0/3603166289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88a80a1ec0 0x7f88a813ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:47.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.754+0000 7f88b4a4e640 1 --2- 172.21.15.114:0/3603166289 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88a80086c0 0x7f88a813b560 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:47.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.754+0000 7f88b4a4e640 1 --2- 172.21.15.114:0/3603166289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88a8006eb0 0x7f88a813b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:47.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.754+0000 7f88b4a4e640 1 -- 172.21.15.114:0/3603166289 >> 172.21.15.114:0/3603166289 conn(0x7f88a800d6f0 msgr2=0x7f88a80929a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:47.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.755+0000 7f88b4a4e640 1 -- 172.21.15.114:0/3603166289 shutdown_connections 2024-09-13T18:24:47.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.755+0000 7f88b4a4e640 1 -- 172.21.15.114:0/3603166289 wait complete. 2024-09-13T18:24:47.777 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T18:24:47.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.956+0000 7efdf8bb3640 1 -- 172.21.15.114:0/2367800363 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf0031770 con 0x7efdf40fff50 2024-09-13T18:24:47.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.957+0000 7efdfbe19640 1 -- 172.21.15.114:0/2367800363 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efdf40fff50 msgr2=0x7efdf4100330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:47.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.957+0000 7efdfbe19640 1 --2- 172.21.15.114:0/2367800363 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efdf40fff50 0x7efdf4100330 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7efdf0009900 tx=0x7efdf002a410 comp rx=0 tx=0).stop 2024-09-13T18:24:47.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.957+0000 7efdfbe19640 1 -- 172.21.15.114:0/2367800363 shutdown_connections 2024-09-13T18:24:47.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.957+0000 7efdfbe19640 1 --2- 172.21.15.114:0/2367800363 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efdf41050a0 0x7efdf410cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:47.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.957+0000 7efdfbe19640 1 --2- 172.21.15.114:0/2367800363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efdf4100900 0x7efdf4104940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:47.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.957+0000 7efdfbe19640 1 --2- 172.21.15.114:0/2367800363 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efdf40fff50 0x7efdf4100330 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:47.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.957+0000 7efdfbe19640 1 -- 172.21.15.114:0/2367800363 >> 172.21.15.114:0/2367800363 conn(0x7efdf40fb9e0 msgr2=0x7efdf40fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:47.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.957+0000 7efdfbe19640 1 -- 172.21.15.114:0/2367800363 shutdown_connections 2024-09-13T18:24:47.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.957+0000 7efdfbe19640 1 -- 172.21.15.114:0/2367800363 wait complete. 2024-09-13T18:24:47.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.958+0000 7efdfbe19640 1 Processor -- start 2024-09-13T18:24:47.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.958+0000 7efdfbe19640 1 -- start start 2024-09-13T18:24:47.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.958+0000 7efdfbe19640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efdf40fff50 0x7efdf410cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:47.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.958+0000 7efdfbe19640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efdf41050a0 0x7efdf4109c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:47.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.958+0000 7efdfbe19640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efdf410a310 0x7efdf4070f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:47.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.958+0000 7efdfbe19640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf410a9e0 con 0x7efdf41050a0 2024-09-13T18:24:47.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.958+0000 7efdfbe19640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf4071460 con 0x7efdf40fff50 2024-09-13T18:24:47.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.958+0000 7efdfbe19640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf4071620 con 0x7efdf410a310 2024-09-13T18:24:47.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.959+0000 7efdf9bb5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efdf40fff50 0x7efdf410cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:47.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.959+0000 7efdfa3b6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efdf410a310 0x7efdf4070f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:47.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.959+0000 7efdfa3b6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efdf410a310 0x7efdf4070f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:57658/0 (socket says 172.21.15.114:57658) 2024-09-13T18:24:47.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.959+0000 7efdfa3b6640 1 -- 172.21.15.114:0/1287753865 learned_addr learned my addr 172.21.15.114:0/1287753865 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:47.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.959+0000 7efdf93b4640 1 --2- 172.21.15.114:0/1287753865 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efdf41050a0 0x7efdf4109c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:47.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.959+0000 7efdfa3b6640 1 -- 172.21.15.114:0/1287753865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efdf40fff50 msgr2=0x7efdf410cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:47.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.959+0000 7efdfa3b6640 1 --2- 172.21.15.114:0/1287753865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efdf40fff50 0x7efdf410cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:47.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.959+0000 7efdfa3b6640 1 -- 172.21.15.114:0/1287753865 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efdf41050a0 msgr2=0x7efdf4109c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:47.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.959+0000 7efdfa3b6640 1 --2- 172.21.15.114:0/1287753865 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efdf41050a0 0x7efdf4109c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:47.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.959+0000 7efdfa3b6640 1 -- 172.21.15.114:0/1287753865 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf0009560 con 0x7efdf410a310 2024-09-13T18:24:47.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.959+0000 7efdf93b4640 1 --2- 172.21.15.114:0/1287753865 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efdf41050a0 0x7efdf4109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:24:47.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.959+0000 7efdfa3b6640 1 --2- 172.21.15.114:0/1287753865 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efdf410a310 0x7efdf4070f20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7efdec00eaa0 tx=0x7efdec00ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:47.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.960+0000 7efdeaffd640 1 -- 172.21.15.114:0/1287753865 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdec00cdb0 con 0x7efdf410a310 2024-09-13T18:24:47.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.960+0000 7efdeaffd640 1 -- 172.21.15.114:0/1287753865 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efdec005020 con 0x7efdf410a310 2024-09-13T18:24:47.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.960+0000 7efdfbe19640 1 -- 172.21.15.114:0/1287753865 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdf4071900 con 0x7efdf410a310 2024-09-13T18:24:47.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.960+0000 7efdfbe19640 1 -- 172.21.15.114:0/1287753865 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdf4071e10 con 0x7efdf410a310 2024-09-13T18:24:47.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.961+0000 7efdeaffd640 1 -- 172.21.15.114:0/1287753865 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdec0116d0 con 0x7efdf410a310 2024-09-13T18:24:47.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.964+0000 7efdeaffd640 1 -- 172.21.15.114:0/1287753865 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7efdec005190 con 0x7efdf410a310 2024-09-13T18:24:47.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.964+0000 7efdfec1c740 1 Processor -- start 2024-09-13T18:24:47.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.964+0000 7efdfec1c740 1 -- start start 2024-09-13T18:24:47.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.965+0000 7efdfec1c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2a6f22e10 0x55e2a6f231f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:47.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.965+0000 7efdfec1c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e2a6f23730 0x55e2a6f2da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:47.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.965+0000 7efdfec1c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2a6f2dfd0 0x55e2a6f303c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:47.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.965+0000 7efdfec1c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2a6e8df20 con 0x55e2a6f23730 2024-09-13T18:24:47.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.965+0000 7efdfec1c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2a6e8b6b0 con 0x55e2a6f22e10 2024-09-13T18:24:47.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.965+0000 7efdfec1c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2a6e86fa0 con 0x55e2a6f2dfd0 2024-09-13T18:24:47.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.965+0000 7efdf93b4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2a6f22e10 0x55e2a6f231f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:47.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.965+0000 7efdf9bb5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e2a6f23730 0x55e2a6f2da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:47.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.965+0000 7efdf9bb5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e2a6f23730 0x55e2a6f2da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51878/0 (socket says 172.21.15.114:51878) 2024-09-13T18:24:47.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.965+0000 7efdf9bb5640 1 -- 172.21.15.114:0/3013915377 learned_addr learned my addr 172.21.15.114:0/3013915377 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:47.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.965+0000 7efdf93b4640 1 --2- 172.21.15.114:0/3013915377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2a6f2dfd0 0x55e2a6f303c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:47.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.965+0000 7efdeaffd640 1 --2- 172.21.15.114:0/1287753865 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7efde00a1390 0x7efde00a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:47.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.965+0000 7efdf9bb5640 1 -- 172.21.15.114:0/3013915377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2a6f2dfd0 msgr2=0x55e2a6f303c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:47.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.965+0000 7efdf9bb5640 1 --2- 172.21.15.114:0/3013915377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2a6f2dfd0 0x55e2a6f303c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:48.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.965+0000 7efdf9bb5640 1 -- 172.21.15.114:0/3013915377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2a6f22e10 msgr2=0x55e2a6f231f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:48.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.965+0000 7efdf9bb5640 1 --2- 172.21.15.114:0/3013915377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2a6f22e10 0x55e2a6f231f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:48.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.965+0000 7efdf9bb5640 1 -- 172.21.15.114:0/3013915377 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2a6e41840 con 0x55e2a6f23730 2024-09-13T18:24:48.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.966+0000 7efdfa3b6640 1 --2- 172.21.15.114:0/1287753865 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7efde00a1390 0x7efde00a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:48.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.966+0000 7efdf93b4640 1 --2- 172.21.15.114:0/3013915377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2a6f2dfd0 0x55e2a6f303c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:24:48.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.966+0000 7efdeaffd640 1 -- 172.21.15.114:0/1287753865 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7efdec0cd120 con 0x7efdf410a310 2024-09-13T18:24:48.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.966+0000 7efdf9bb5640 1 --2- 172.21.15.114:0/3013915377 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e2a6f23730 0x55e2a6f2da90 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7efdf0000c00 tx=0x7efdf0004130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f952f5bd85e236b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:48.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.966+0000 7efdfa3b6640 1 --2- 172.21.15.114:0/1287753865 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7efde00a1390 0x7efde00a3850 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7efdec06d060 tx=0x7efdec0c43c0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:48.496 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:48 smithi204 ceph-mon[34777]: pgmap v73: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 92 MiB/s wr, 547 op/s 2024-09-13T18:24:48.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.966+0000 7efddffff640 1 -- 172.21.15.114:0/3013915377 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf00035f0 con 0x55e2a6f23730 2024-09-13T18:24:48.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.967+0000 7efddffff640 1 -- 172.21.15.114:0/3013915377 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efdf0003be0 con 0x55e2a6f23730 2024-09-13T18:24:48.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.967+0000 7efddffff640 1 -- 172.21.15.114:0/3013915377 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf003db00 con 0x55e2a6f23730 2024-09-13T18:24:48.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.968+0000 7efdfec1c740 1 -- 172.21.15.114:0/3013915377 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e2a6f23730 msgr2=0x55e2a6f2da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:48.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.968+0000 7efdfec1c740 1 --2- 172.21.15.114:0/3013915377 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e2a6f23730 0x55e2a6f2da90 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7efdf0000c00 tx=0x7efdf0004130 comp rx=0 tx=0).stop 2024-09-13T18:24:48.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.968+0000 7efdfec1c740 1 -- 172.21.15.114:0/3013915377 shutdown_connections 2024-09-13T18:24:48.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.969+0000 7efdfec1c740 1 --2- 172.21.15.114:0/3013915377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2a6f2dfd0 0x55e2a6f303c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:48.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.969+0000 7efdfec1c740 1 --2- 172.21.15.114:0/3013915377 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e2a6f23730 0x55e2a6f2da90 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:48.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.969+0000 7efdfec1c740 1 --2- 172.21.15.114:0/3013915377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2a6f22e10 0x55e2a6f231f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:48.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.969+0000 7efdfec1c740 1 -- 172.21.15.114:0/3013915377 >> 172.21.15.114:0/3013915377 conn(0x55e2a6f1f380 msgr2=0x55e2a6f1f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:48.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.969+0000 7efdfec1c740 1 -- 172.21.15.114:0/3013915377 shutdown_connections 2024-09-13T18:24:48.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.969+0000 7efdfec1c740 1 -- 172.21.15.114:0/3013915377 wait complete. 2024-09-13T18:24:48.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.969+0000 7efdfec1c740 1 -- start start 2024-09-13T18:24:48.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.969+0000 7efdfec1c740 1 Processor -- start 2024-09-13T18:24:48.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.970+0000 7efdfec1c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2a6f22e10 0x55e2a6f2d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:48.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.970+0000 7efdfec1c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e2a6f23730 0x55e2a6f2ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:48.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.970+0000 7efdfec1c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2a6f2dfd0 0x55e2a6f2b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:48.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.970+0000 7efdf93b4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2a6f22e10 0x55e2a6f2d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:48.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.970+0000 7efdf93b4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2a6f22e10 0x55e2a6f2d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:39428/0 (socket says 172.21.15.114:39428) 2024-09-13T18:24:48.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.970+0000 7efdf93b4640 1 -- 172.21.15.114:0/1993241770 learned_addr learned my addr 172.21.15.114:0/1993241770 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:48.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.970+0000 7efdfec1c740 1 -- 172.21.15.114:0/1993241770 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2a6e282a0 con 0x55e2a6f23730 2024-09-13T18:24:48.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.970+0000 7efdfec1c740 1 -- 172.21.15.114:0/1993241770 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2a6b5f430 con 0x55e2a6f22e10 2024-09-13T18:24:48.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.970+0000 7efdfec1c740 1 -- 172.21.15.114:0/1993241770 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2a6e0a680 con 0x55e2a6f2dfd0 2024-09-13T18:24:48.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.970+0000 7efdf93b4640 1 --2- 172.21.15.114:0/1993241770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2a6f2dfd0 0x55e2a6f2b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:48.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.970+0000 7efdf9bb5640 1 --2- 172.21.15.114:0/1993241770 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e2a6f23730 0x55e2a6f2ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:48.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.971+0000 7efdfec1c740 10 client.0 _ll_register_callbacks cb 0x55e2a6e80610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:24:48.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.971+0000 7efdf93b4640 1 -- 172.21.15.114:0/1993241770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2a6f2dfd0 msgr2=0x55e2a6f2b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:48.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.971+0000 7efdf93b4640 1 --2- 172.21.15.114:0/1993241770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2a6f2dfd0 0x55e2a6f2b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:48.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.971+0000 7efdf93b4640 1 -- 172.21.15.114:0/1993241770 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e2a6f23730 msgr2=0x55e2a6f2ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:48.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.971+0000 7efdf93b4640 1 --2- 172.21.15.114:0/1993241770 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e2a6f23730 0x55e2a6f2ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:48.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.971+0000 7efdf93b4640 1 -- 172.21.15.114:0/1993241770 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdf410d430 con 0x55e2a6f22e10 2024-09-13T18:24:48.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.971+0000 7efdf93b4640 1 --2- 172.21.15.114:0/1993241770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2a6f22e10 0x55e2a6f2d9c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7efdf40fdf90 tx=0x7efde4005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da4559b494f6bc0e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:48.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.971+0000 7efddd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:24:48.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.971+0000 7efddd7fa640 1 -- 172.21.15.114:0/1993241770 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde4015110 con 0x55e2a6f22e10 2024-09-13T18:24:48.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.971+0000 7efddd7fa640 1 -- 172.21.15.114:0/1993241770 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7efde4012bd0 con 0x55e2a6f22e10 2024-09-13T18:24:48.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.972+0000 7efdfec1c740 1 -- 172.21.15.114:0/1993241770 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e2a6b7a4e0 con 0x55e2a6f22e10 2024-09-13T18:24:48.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.972+0000 7efddd7fa640 1 -- 172.21.15.114:0/1993241770 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7efde4015270 con 0x55e2a6f22e10 2024-09-13T18:24:48.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.972+0000 7efdfec1c740 10 client.25883 fetch_fsmap learned FSMap version 32 2024-09-13T18:24:48.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.972+0000 7efdfec1c740 1 -- 172.21.15.114:0/1993241770 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55e2a6e85620 con 0x55e2a6f22e10 2024-09-13T18:24:48.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.972+0000 7efddd7fa640 1 -- 172.21.15.114:0/1993241770 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e2a6b7a4e0 con 0x55e2a6f22e10 2024-09-13T18:24:48.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.972+0000 7efddd7fa640 1 -- 172.21.15.114:0/1993241770 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7efde40037b0 con 0x55e2a6f22e10 2024-09-13T18:24:48.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.973+0000 7efdfec1c740 10 client.25883 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:24:48.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.973+0000 7efdfec1c740 10 client.25883 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:24:48.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.973+0000 7efdfec1c740 20 client.25883 populate_metadata read hostname 'smithi114' 2024-09-13T18:24:48.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.973+0000 7efdfec1c740 1 --2- 172.21.15.114:0/1993241770 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55e2a6f3de10 0x55e2a6f40230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:48.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.973+0000 7efdfec1c740 4 client.25883 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:24:48.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.973+0000 7efdfec1c740 1 -- 172.21.15.114:0/1993241770 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e2a6b8b7e0 con 0x55e2a6f3de10 2024-09-13T18:24:48.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.973+0000 7efdf9bb5640 1 --2- 172.21.15.114:0/1993241770 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55e2a6f3de10 0x55e2a6f40230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:48.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.974+0000 7efdf9bb5640 1 --2- 172.21.15.114:0/1993241770 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55e2a6f3de10 0x55e2a6f40230 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2491a8609b44b634 server_cookie=6ac5bdda8d51666d in_seq=0 out_seq=0 2024-09-13T18:24:48.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.974+0000 7efddd7fa640 10 client.25883 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:24:48.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.975+0000 7efddd7fa640 1 -- 172.21.15.114:0/1993241770 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55e2a6b8b7e0 con 0x55e2a6f3de10 2024-09-13T18:24:48.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.975+0000 7efddd7fa640 10 client.25883 handle_command_reply: tid=0 2024-09-13T18:24:48.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.975+0000 7efdfec1c740 1 client.25883 shutdown 2024-09-13T18:24:48.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.975+0000 7efdfec1c740 1 -- 172.21.15.114:0/1993241770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2a6f22e10 msgr2=0x55e2a6f2d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:48.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.975+0000 7efdfec1c740 1 --2- 172.21.15.114:0/1993241770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2a6f22e10 0x55e2a6f2d9c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7efdf40fdf90 tx=0x7efde4005f90 comp rx=0 tx=0).stop 2024-09-13T18:24:48.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.976+0000 7efdfec1c740 1 -- 172.21.15.114:0/1993241770 shutdown_connections 2024-09-13T18:24:48.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.976+0000 7efdfec1c740 1 --2- 172.21.15.114:0/1993241770 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55e2a6f3de10 0x55e2a6f40230 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:48.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.976+0000 7efddd7fa640 0 client.25883 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:24:48.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.976+0000 7efdfec1c740 1 --2- 172.21.15.114:0/1993241770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2a6f2dfd0 0x55e2a6f2b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:48.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.976+0000 7efdfec1c740 1 --2- 172.21.15.114:0/1993241770 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e2a6f23730 0x55e2a6f2ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:48.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.976+0000 7efdfec1c740 1 --2- 172.21.15.114:0/1993241770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2a6f22e10 0x55e2a6f2d9c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:48.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.976+0000 7efdfec1c740 1 -- 172.21.15.114:0/1993241770 >> 172.21.15.114:0/1993241770 conn(0x55e2a6f1f380 msgr2=0x55e2a6f250e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:48.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.976+0000 7efdfec1c740 1 -- 172.21.15.114:0/1993241770 shutdown_connections 2024-09-13T18:24:48.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.976+0000 7efdfec1c740 1 -- 172.21.15.114:0/1993241770 wait complete. 2024-09-13T18:24:48.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:47.976+0000 7efdfec1c740 20 client.25883 trim_cache size 0 max 16384 2024-09-13T18:24:48.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.007+0000 7efdfec1c740 1 Processor -- start 2024-09-13T18:24:48.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.008+0000 7efdfec1c740 1 -- start start 2024-09-13T18:24:48.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.008+0000 7efdfec1c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2a6f22e10 0x55e2a6f2d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:48.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.008+0000 7efdfec1c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2a6f23730 0x55e2a6f44bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:48.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.008+0000 7efdfec1c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e2a6f2dfd0 0x55e2a6f34db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:48.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.008+0000 7efdfec1c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2a6f20060 con 0x55e2a6f2dfd0 2024-09-13T18:24:48.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.008+0000 7efdfec1c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2a6f3c640 con 0x55e2a6f22e10 2024-09-13T18:24:48.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.008+0000 7efdf93b4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2a6f22e10 0x55e2a6f2d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:48.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.008+0000 7efdf93b4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2a6f22e10 0x55e2a6f2d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:39444/0 (socket says 172.21.15.114:39444) 2024-09-13T18:24:48.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.008+0000 7efdfec1c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2a6f22620 con 0x55e2a6f23730 2024-09-13T18:24:48.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.008+0000 7efdf93b4640 1 -- 172.21.15.114:0/3316168010 learned_addr learned my addr 172.21.15.114:0/3316168010 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:48.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.008+0000 7efdf9bb5640 1 --2- 172.21.15.114:0/3316168010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2a6f23730 0x55e2a6f44bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:48.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.009+0000 7efdf93b4640 1 --2- 172.21.15.114:0/3316168010 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e2a6f2dfd0 0x55e2a6f34db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:48.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.009+0000 7efdf93b4640 1 -- 172.21.15.114:0/3316168010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2a6f23730 msgr2=0x55e2a6f44bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:48.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.009+0000 7efdf93b4640 1 --2- 172.21.15.114:0/3316168010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2a6f23730 0x55e2a6f44bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:48.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.009+0000 7efdf93b4640 1 -- 172.21.15.114:0/3316168010 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e2a6f2dfd0 msgr2=0x55e2a6f34db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:48.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.009+0000 7efdf93b4640 1 --2- 172.21.15.114:0/3316168010 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e2a6f2dfd0 0x55e2a6f34db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:48.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.009+0000 7efdf93b4640 1 -- 172.21.15.114:0/3316168010 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2a6e27790 con 0x55e2a6f22e10 2024-09-13T18:24:48.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.009+0000 7efdf93b4640 1 --2- 172.21.15.114:0/3316168010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2a6f22e10 0x55e2a6f2d620 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efde4005900 tx=0x7efde4003e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebf9528f4b54ce85 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:48.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.009+0000 7efde8ff9640 1 -- 172.21.15.114:0/3316168010 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde4015110 con 0x55e2a6f22e10 2024-09-13T18:24:48.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.010+0000 7efde8ff9640 1 -- 172.21.15.114:0/3316168010 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efde401a070 con 0x55e2a6f22e10 2024-09-13T18:24:48.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.010+0000 7efde8ff9640 1 -- 172.21.15.114:0/3316168010 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde4005320 con 0x55e2a6f22e10 2024-09-13T18:24:48.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.010+0000 7efdfec1c740 1 -- 172.21.15.114:0/3316168010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2a6f22e10 msgr2=0x55e2a6f2d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:48.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.010+0000 7efdfec1c740 1 --2- 172.21.15.114:0/3316168010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2a6f22e10 0x55e2a6f2d620 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efde4005900 tx=0x7efde4003e90 comp rx=0 tx=0).stop 2024-09-13T18:24:48.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.010+0000 7efdfec1c740 1 -- 172.21.15.114:0/3316168010 shutdown_connections 2024-09-13T18:24:48.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.010+0000 7efdfec1c740 1 --2- 172.21.15.114:0/3316168010 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e2a6f2dfd0 0x55e2a6f34db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:48.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.010+0000 7efdfec1c740 1 --2- 172.21.15.114:0/3316168010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2a6f23730 0x55e2a6f44bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:48.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.010+0000 7efdfec1c740 1 --2- 172.21.15.114:0/3316168010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2a6f22e10 0x55e2a6f2d620 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:48.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.010+0000 7efdfec1c740 1 -- 172.21.15.114:0/3316168010 >> 172.21.15.114:0/3316168010 conn(0x55e2a6f1f380 msgr2=0x55e2a6f23fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:48.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.011+0000 7efdfec1c740 1 -- 172.21.15.114:0/3316168010 shutdown_connections 2024-09-13T18:24:48.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.011+0000 7efdfec1c740 1 -- 172.21.15.114:0/3316168010 wait complete. 2024-09-13T18:24:48.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.011+0000 7efdfec1c740 1 -- start start 2024-09-13T18:24:48.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.011+0000 7efdfec1c740 1 Processor -- start 2024-09-13T18:24:48.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.012+0000 7efdfec1c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2a6f22e10 0x55e2a6f35250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:48.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.012+0000 7efdfec1c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e2a6f23730 0x55e2a6f28200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:48.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.012+0000 7efdfec1c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2a6f2dfd0 0x55e2a6f28880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:48.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.012+0000 7efdfec1c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2a6f26f80 con 0x55e2a6f23730 2024-09-13T18:24:48.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.012+0000 7efdf93b4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2a6f22e10 0x55e2a6f35250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:48.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.012+0000 7efdf93b4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2a6f22e10 0x55e2a6f35250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:57684/0 (socket says 172.21.15.114:57684) 2024-09-13T18:24:48.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.012+0000 7efdf93b4640 1 -- 172.21.15.114:0/1249080221 learned_addr learned my addr 172.21.15.114:0/1249080221 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:48.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.012+0000 7efdf9bb5640 1 --2- 172.21.15.114:0/1249080221 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e2a6f23730 0x55e2a6f28200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:48.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.012+0000 7efdfec1c740 1 -- 172.21.15.114:0/1249080221 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2a6f270f0 con 0x55e2a6f2dfd0 2024-09-13T18:24:48.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.012+0000 7efdfec1c740 1 -- 172.21.15.114:0/1249080221 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2a6f272b0 con 0x55e2a6f22e10 2024-09-13T18:24:48.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.012+0000 7efdfec1c740 10 client.0 _ll_register_callbacks cb 0x7efde4009ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:24:48.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.012+0000 7efdf93b4640 1 --2- 172.21.15.114:0/1249080221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2a6f2dfd0 0x55e2a6f28880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:48.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.012+0000 7efdf9bb5640 1 -- 172.21.15.114:0/1249080221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2a6f22e10 msgr2=0x55e2a6f35250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:48.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.012+0000 7efdf9bb5640 1 --2- 172.21.15.114:0/1249080221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2a6f22e10 0x55e2a6f35250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:48.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.012+0000 7efdf9bb5640 1 -- 172.21.15.114:0/1249080221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2a6f2dfd0 msgr2=0x55e2a6f28880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:48.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.012+0000 7efdf9bb5640 1 --2- 172.21.15.114:0/1249080221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2a6f2dfd0 0x55e2a6f28880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:48.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.013+0000 7efdf9bb5640 1 -- 172.21.15.114:0/1249080221 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e2a6e85620 con 0x55e2a6f23730 2024-09-13T18:24:48.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.013+0000 7efdf9bb5640 1 --2- 172.21.15.114:0/1249080221 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e2a6f23730 0x55e2a6f28200 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7efdf0005c40 tx=0x7efdf0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be6ccf26b977a6cb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:48.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.013+0000 7efddd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:24:48.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.013+0000 7efddd7fa640 1 -- 172.21.15.114:0/1249080221 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf004b040 con 0x55e2a6f23730 2024-09-13T18:24:48.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.013+0000 7efddd7fa640 1 -- 172.21.15.114:0/1249080221 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7efdf0003a00 con 0x55e2a6f23730 2024-09-13T18:24:48.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.014+0000 7efdfec1c740 1 -- 172.21.15.114:0/1249080221 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efdf0003be0 con 0x55e2a6f23730 2024-09-13T18:24:48.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.014+0000 7efddd7fa640 1 -- 172.21.15.114:0/1249080221 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7efdf005a750 con 0x55e2a6f23730 2024-09-13T18:24:48.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.014+0000 7efdfec1c740 10 client.16563 fetch_fsmap learned FSMap version 32 2024-09-13T18:24:48.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.014+0000 7efdfec1c740 1 -- 172.21.15.114:0/1249080221 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55e2a6e4cea0 con 0x55e2a6f23730 2024-09-13T18:24:48.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.015+0000 7efddd7fa640 1 -- 172.21.15.114:0/1249080221 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efdf0003be0 con 0x55e2a6f23730 2024-09-13T18:24:48.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.015+0000 7efddd7fa640 1 -- 172.21.15.114:0/1249080221 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7efdf0052790 con 0x55e2a6f23730 2024-09-13T18:24:48.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.015+0000 7efdfec1c740 10 client.16563 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:24:48.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.015+0000 7efdfec1c740 10 client.16563 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:24:48.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.015+0000 7efdfec1c740 20 client.16563 populate_metadata read hostname 'smithi114' 2024-09-13T18:24:48.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.015+0000 7efdfec1c740 1 --2- 172.21.15.114:0/1249080221 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55e2a6f94ca0 0x55e2a6f970c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:48.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.015+0000 7efdfec1c740 4 client.16563 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T18:24:48.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.015+0000 7efdfec1c740 1 -- 172.21.15.114:0/1249080221 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e2a6bdee30 con 0x55e2a6f94ca0 2024-09-13T18:24:48.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.015+0000 7efdf93b4640 1 --2- 172.21.15.114:0/1249080221 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55e2a6f94ca0 0x55e2a6f970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:48.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.016+0000 7efddd7fa640 10 client.16563 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:24:48.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.016+0000 7efdf93b4640 1 --2- 172.21.15.114:0/1249080221 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55e2a6f94ca0 0x55e2a6f970c0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2eab364d1a47cc6d server_cookie=86d6f19af7c67da9 in_seq=0 out_seq=0 2024-09-13T18:24:48.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.016+0000 7efddd7fa640 1 -- 172.21.15.114:0/1249080221 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55e2a6bdee30 con 0x55e2a6f94ca0 2024-09-13T18:24:48.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.016+0000 7efddd7fa640 10 client.16563 handle_command_reply: tid=0 2024-09-13T18:24:48.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.016+0000 7efdfec1c740 1 client.16563 shutdown 2024-09-13T18:24:48.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.016+0000 7efdfec1c740 1 -- 172.21.15.114:0/1249080221 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e2a6f23730 msgr2=0x55e2a6f28200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:48.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.016+0000 7efdfec1c740 1 --2- 172.21.15.114:0/1249080221 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e2a6f23730 0x55e2a6f28200 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7efdf0005c40 tx=0x7efdf0009260 comp rx=0 tx=0).stop 2024-09-13T18:24:48.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.016+0000 7efdfec1c740 1 -- 172.21.15.114:0/1249080221 shutdown_connections 2024-09-13T18:24:48.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.017+0000 7efdfec1c740 1 --2- 172.21.15.114:0/1249080221 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55e2a6f94ca0 0x55e2a6f970c0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:48.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.017+0000 7efdfec1c740 1 --2- 172.21.15.114:0/1249080221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2a6f2dfd0 0x55e2a6f28880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:48.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.017+0000 7efdfec1c740 1 --2- 172.21.15.114:0/1249080221 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e2a6f23730 0x55e2a6f28200 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:48.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.017+0000 7efdfec1c740 1 --2- 172.21.15.114:0/1249080221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2a6f22e10 0x55e2a6f35250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:48.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.017+0000 7efdfec1c740 1 -- 172.21.15.114:0/1249080221 >> 172.21.15.114:0/1249080221 conn(0x55e2a6f1f380 msgr2=0x55e2a6f2d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:48.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.017+0000 7efddd7fa640 0 client.16563 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:24:48.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.017+0000 7efdfec1c740 1 -- 172.21.15.114:0/1249080221 shutdown_connections 2024-09-13T18:24:48.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.017+0000 7efdfec1c740 1 -- 172.21.15.114:0/1249080221 wait complete. 2024-09-13T18:24:48.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.017+0000 7efdfec1c740 20 client.16563 trim_cache size 0 max 16384 2024-09-13T18:24:48.517 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:24:48.517 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-13T18:24:48.517 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": { 2024-09-13T18:24:48.517 INFO:teuthology.orchestra.run.smithi114.stdout: "181907a9-a1bd-420d-ad64-5c2aa352ef01": { 2024-09-13T18:24:48.517 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-13T18:24:48.517 INFO:teuthology.orchestra.run.smithi114.stdout: "tag": "181907a9-a1bd-420d-ad64-5c2aa352ef01", 2024-09-13T18:24:48.517 INFO:teuthology.orchestra.run.smithi114.stdout: "options": "recursive,force" 2024-09-13T18:24:48.518 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:24:48.518 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:24:48.518 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:24:48.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.018+0000 7efdfbe19640 1 -- 172.21.15.114:0/1287753865 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7efde00a1390 msgr2=0x7efde00a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:48.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.018+0000 7efdfbe19640 1 --2- 172.21.15.114:0/1287753865 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7efde00a1390 0x7efde00a3850 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7efdec06d060 tx=0x7efdec0c43c0 comp rx=0 tx=0).stop 2024-09-13T18:24:48.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.018+0000 7efdfbe19640 1 -- 172.21.15.114:0/1287753865 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efdf410a310 msgr2=0x7efdf4070f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:48.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.018+0000 7efdfbe19640 1 --2- 172.21.15.114:0/1287753865 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efdf410a310 0x7efdf4070f20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7efdec00eaa0 tx=0x7efdec00ef60 comp rx=0 tx=0).stop 2024-09-13T18:24:48.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.018+0000 7efdfbe19640 1 -- 172.21.15.114:0/1287753865 shutdown_connections 2024-09-13T18:24:48.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.018+0000 7efdfbe19640 1 --2- 172.21.15.114:0/1287753865 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7efde00a1390 0x7efde00a3850 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:48.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.018+0000 7efdfbe19640 1 --2- 172.21.15.114:0/1287753865 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efdf410a310 0x7efdf4070f20 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:48.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.018+0000 7efdfbe19640 1 --2- 172.21.15.114:0/1287753865 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efdf41050a0 0x7efdf4109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:48.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.018+0000 7efdfbe19640 1 --2- 172.21.15.114:0/1287753865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efdf40fff50 0x7efdf410cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:48.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.018+0000 7efdfbe19640 1 -- 172.21.15.114:0/1287753865 >> 172.21.15.114:0/1287753865 conn(0x7efdf40fb9e0 msgr2=0x7efdf40fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:48.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.018+0000 7efdfbe19640 1 -- 172.21.15.114:0/1287753865 shutdown_connections 2024-09-13T18:24:48.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:24:48.018+0000 7efdfbe19640 1 -- 172.21.15.114:0/1287753865 wait complete. 2024-09-13T18:24:48.520 INFO:tasks.cephfs.filesystem:scrub status for tag:181907a9-a1bd-420d-ad64-5c2aa352ef01 - {'path': '/', 'tag': '181907a9-a1bd-420d-ad64-5c2aa352ef01', 'options': 'recursive,force'} 2024-09-13T18:24:48.521 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:48 smithi114 ceph-mon[30897]: pgmap v73: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 92 MiB/s wr, 547 op/s 2024-09-13T18:24:48.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:48 smithi134 ceph-mon[34283]: pgmap v73: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 92 MiB/s wr, 547 op/s 2024-09-13T18:24:49.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:49 smithi204 ceph-mon[34777]: scrub queued for path: / 2024-09-13T18:24:49.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:49 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:24:49.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:49 smithi204 ceph-mon[34777]: scrub summary: active paths [/] 2024-09-13T18:24:49.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:49 smithi204 ceph-mon[34777]: scrub summary: active paths [/] 2024-09-13T18:24:49.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:49 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:24:49.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:49 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:24:49.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:49 smithi114 ceph-mon[30897]: scrub queued for path: / 2024-09-13T18:24:49.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:49 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:24:49.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:49 smithi114 ceph-mon[30897]: scrub summary: active paths [/] 2024-09-13T18:24:49.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:49 smithi114 ceph-mon[30897]: scrub summary: active paths [/] 2024-09-13T18:24:49.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:49 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:24:49.472 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:49 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:24:49.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:49 smithi134 ceph-mon[34283]: scrub queued for path: / 2024-09-13T18:24:49.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:49 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:24:49.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:49 smithi134 ceph-mon[34283]: scrub summary: active paths [/] 2024-09-13T18:24:49.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:49 smithi134 ceph-mon[34283]: scrub summary: active paths [/] 2024-09-13T18:24:49.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:49 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:24:49.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:49 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:24:49.690 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.686+0000 7f3d9fc19640 1 -- 172.21.15.114:0/3354245737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d98072850 msgr2=0x7f3d98072c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:49.690 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.686+0000 7f3d9fc19640 1 --2- 172.21.15.114:0/3354245737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d98072850 0x7f3d98072c30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3d94009900 tx=0x7f3d9402a480 comp rx=0 tx=0).stop 2024-09-13T18:24:49.690 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.686+0000 7f3d9fc19640 1 -- 172.21.15.114:0/3354245737 shutdown_connections 2024-09-13T18:24:49.690 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.686+0000 7f3d9fc19640 1 --2- 172.21.15.114:0/3354245737 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d98110cd0 0x7f3d981130c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:49.690 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.686+0000 7f3d9fc19640 1 --2- 172.21.15.114:0/3354245737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d98073170 0x7f3d98110790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:49.690 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.686+0000 7f3d9fc19640 1 --2- 172.21.15.114:0/3354245737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d98072850 0x7f3d98072c30 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:49.690 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.686+0000 7f3d9fc19640 1 -- 172.21.15.114:0/3354245737 >> 172.21.15.114:0/3354245737 conn(0x7f3d980fb9c0 msgr2=0x7f3d980fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:49.690 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.686+0000 7f3d9fc19640 1 -- 172.21.15.114:0/3354245737 shutdown_connections 2024-09-13T18:24:49.690 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.686+0000 7f3d9fc19640 1 -- 172.21.15.114:0/3354245737 wait complete. 2024-09-13T18:24:49.691 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.687+0000 7f3d9fc19640 1 Processor -- start 2024-09-13T18:24:49.691 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.687+0000 7f3d9fc19640 1 -- start start 2024-09-13T18:24:49.691 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.688+0000 7f3d9fc19640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d98072850 0x7f3d9819c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:49.691 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.688+0000 7f3d9fc19640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d98073170 0x7f3d9819cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:49.692 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.688+0000 7f3d9fc19640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d98110cd0 0x7f3d981a1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:49.692 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.688+0000 7f3d9fc19640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d981a1600 con 0x7f3d98073170 2024-09-13T18:24:49.692 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.688+0000 7f3d9fc19640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d981a1770 con 0x7f3d98110cd0 2024-09-13T18:24:49.692 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.688+0000 7f3d9fc19640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d981a1930 con 0x7f3d98072850 2024-09-13T18:24:49.692 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.688+0000 7f3d9e1b6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d98110cd0 0x7f3d981a1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:49.692 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.688+0000 7f3d9e1b6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d98110cd0 0x7f3d981a1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:39472/0 (socket says 172.21.15.114:39472) 2024-09-13T18:24:49.692 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.688+0000 7f3d9e1b6640 1 -- 172.21.15.114:0/2729770551 learned_addr learned my addr 172.21.15.114:0/2729770551 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:49.692 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.688+0000 7f3d9d9b5640 1 --2- 172.21.15.114:0/2729770551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d98072850 0x7f3d9819c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:49.692 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.688+0000 7f3d9d1b4640 1 --2- 172.21.15.114:0/2729770551 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d98073170 0x7f3d9819cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:49.693 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.689+0000 7f3d9e1b6640 1 -- 172.21.15.114:0/2729770551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d98072850 msgr2=0x7f3d9819c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:49.693 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.689+0000 7f3d9e1b6640 1 --2- 172.21.15.114:0/2729770551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d98072850 0x7f3d9819c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:49.693 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.689+0000 7f3d9e1b6640 1 -- 172.21.15.114:0/2729770551 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d98073170 msgr2=0x7f3d9819cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:49.693 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.689+0000 7f3d9e1b6640 1 --2- 172.21.15.114:0/2729770551 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d98073170 0x7f3d9819cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:49.693 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.689+0000 7f3d9e1b6640 1 -- 172.21.15.114:0/2729770551 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d94009560 con 0x7f3d98110cd0 2024-09-13T18:24:49.693 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.689+0000 7f3d9d1b4640 1 --2- 172.21.15.114:0/2729770551 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d98073170 0x7f3d9819cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:24:49.694 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.689+0000 7f3d9e1b6640 1 --2- 172.21.15.114:0/2729770551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d98110cd0 0x7f3d981a1000 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3d9000cce0 tx=0x7f3d9000a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:49.694 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.690+0000 7f3d8effd640 1 -- 172.21.15.114:0/2729770551 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9000f450 con 0x7f3d98110cd0 2024-09-13T18:24:49.694 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.690+0000 7f3d8effd640 1 -- 172.21.15.114:0/2729770551 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3d9000fa40 con 0x7f3d98110cd0 2024-09-13T18:24:49.694 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.690+0000 7f3d9fc19640 1 -- 172.21.15.114:0/2729770551 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d981a1c10 con 0x7f3d98110cd0 2024-09-13T18:24:49.694 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.690+0000 7f3d8effd640 1 -- 172.21.15.114:0/2729770551 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d900189a0 con 0x7f3d98110cd0 2024-09-13T18:24:49.694 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.690+0000 7f3d9fc19640 1 -- 172.21.15.114:0/2729770551 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d981a9490 con 0x7f3d98110cd0 2024-09-13T18:24:49.698 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.695+0000 7f3d8effd640 1 -- 172.21.15.114:0/2729770551 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f3d9000f5b0 con 0x7f3d98110cd0 2024-09-13T18:24:49.699 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.696+0000 7f3da2a1a740 1 Processor -- start 2024-09-13T18:24:49.699 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.697+0000 7f3da2a1a740 1 -- start start 2024-09-13T18:24:49.700 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.697+0000 7f3d8effd640 1 --2- 172.21.15.114:0/2729770551 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f3d840a13d0 0x7f3d840a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:49.700 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.697+0000 7f3da2a1a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649b0640330 0x5649b0640710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:49.701 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.697+0000 7f3d8effd640 1 -- 172.21.15.114:0/2729770551 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f3d900cd3a0 con 0x7f3d98110cd0 2024-09-13T18:24:49.701 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.697+0000 7f3d9d1b4640 1 --2- 172.21.15.114:0/2729770551 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f3d840a13d0 0x7f3d840a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:49.701 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.697+0000 7f3d9d9b5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649b0640330 0x5649b0640710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:49.701 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.697+0000 7f3d9d9b5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649b0640330 0x5649b0640710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:57700/0 (socket says 172.21.15.114:57700) 2024-09-13T18:24:49.701 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.697+0000 7f3da2a1a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649b0640c50 0x5649b064afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:49.701 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.697+0000 7f3d9d9b5640 1 -- 172.21.15.114:0/3216414228 learned_addr learned my addr 172.21.15.114:0/3216414228 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:49.701 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.698+0000 7f3da2a1a740 1 --2- 172.21.15.114:0/3216414228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649b064b4f0 0x5649b064d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:49.701 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.698+0000 7f3da2a1a740 1 -- 172.21.15.114:0/3216414228 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649b05abc30 con 0x5649b0640c50 2024-09-13T18:24:49.701 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.698+0000 7f3da2a1a740 1 -- 172.21.15.114:0/3216414228 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649b05a93c0 con 0x5649b064b4f0 2024-09-13T18:24:49.701 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.698+0000 7f3da2a1a740 1 -- 172.21.15.114:0/3216414228 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649b05a4cb0 con 0x5649b0640330 2024-09-13T18:24:49.702 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.698+0000 7f3d9e1b6640 1 --2- 172.21.15.114:0/3216414228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649b064b4f0 0x5649b064d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:49.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.698+0000 7f3d9d1b4640 1 --2- 172.21.15.114:0/3216414228 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649b0640c50 0x5649b064afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:49.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.698+0000 7f3d9e1b6640 1 -- 172.21.15.114:0/3216414228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649b0640330 msgr2=0x5649b0640710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:49.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.698+0000 7f3d9e1b6640 1 --2- 172.21.15.114:0/3216414228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649b0640330 0x5649b0640710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:49.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.698+0000 7f3d9e1b6640 1 -- 172.21.15.114:0/3216414228 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649b0640c50 msgr2=0x5649b064afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:49.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.698+0000 7f3d9e1b6640 1 --2- 172.21.15.114:0/3216414228 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649b0640c50 0x5649b064afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:49.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.698+0000 7f3d9e1b6640 1 -- 172.21.15.114:0/3216414228 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649b054b910 con 0x5649b064b4f0 2024-09-13T18:24:49.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.698+0000 7f3d9d1b4640 1 --2- 172.21.15.114:0/3216414228 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649b0640c50 0x5649b064afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:24:49.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.698+0000 7f3d9e1b6640 1 --2- 172.21.15.114:0/3216414228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649b064b4f0 0x5649b064d8e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3d900ca740 tx=0x7f3d90099d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=630e897e1290e646 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:49.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.699+0000 7f3d9d1b4640 1 --2- 172.21.15.114:0/2729770551 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f3d840a13d0 0x7f3d840a3890 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3d9819dc70 tx=0x7f3d880127c0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:49.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.699+0000 7f3d83fff640 1 -- 172.21.15.114:0/3216414228 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9009a800 con 0x5649b064b4f0 2024-09-13T18:24:49.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.699+0000 7f3d83fff640 1 -- 172.21.15.114:0/3216414228 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3d900cae90 con 0x5649b064b4f0 2024-09-13T18:24:49.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.699+0000 7f3d83fff640 1 -- 172.21.15.114:0/3216414228 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d900a2580 con 0x5649b064b4f0 2024-09-13T18:24:49.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.700+0000 7f3da2a1a740 1 -- 172.21.15.114:0/3216414228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649b064b4f0 msgr2=0x5649b064d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.700+0000 7f3da2a1a740 1 --2- 172.21.15.114:0/3216414228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649b064b4f0 0x5649b064d8e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3d900ca740 tx=0x7f3d90099d00 comp rx=0 tx=0).stop 2024-09-13T18:24:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.700+0000 7f3da2a1a740 1 -- 172.21.15.114:0/3216414228 shutdown_connections 2024-09-13T18:24:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.700+0000 7f3da2a1a740 1 --2- 172.21.15.114:0/3216414228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649b064b4f0 0x5649b064d8e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.700+0000 7f3da2a1a740 1 --2- 172.21.15.114:0/3216414228 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649b0640c50 0x5649b064afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.700+0000 7f3da2a1a740 1 --2- 172.21.15.114:0/3216414228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649b0640330 0x5649b0640710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.700+0000 7f3da2a1a740 1 -- 172.21.15.114:0/3216414228 >> 172.21.15.114:0/3216414228 conn(0x5649b063c8c0 msgr2=0x5649b063cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.700+0000 7f3da2a1a740 1 -- 172.21.15.114:0/3216414228 shutdown_connections 2024-09-13T18:24:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.700+0000 7f3da2a1a740 1 -- 172.21.15.114:0/3216414228 wait complete. 2024-09-13T18:24:49.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.701+0000 7f3da2a1a740 1 -- start start 2024-09-13T18:24:49.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.701+0000 7f3da2a1a740 1 Processor -- start 2024-09-13T18:24:49.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.701+0000 7f3da2a1a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649b0640330 0x5649b064af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:49.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.702+0000 7f3da2a1a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649b0640c50 0x5649b0648220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:49.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.702+0000 7f3da2a1a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649b064b4f0 0x5649b0648990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:49.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.702+0000 7f3da2a1a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649b0562e80 con 0x5649b0640c50 2024-09-13T18:24:49.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.702+0000 7f3da2a1a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649b0562b20 con 0x5649b0640330 2024-09-13T18:24:49.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.702+0000 7f3da2a1a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649b0528260 con 0x5649b064b4f0 2024-09-13T18:24:49.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.702+0000 7f3da2a1a740 10 client.0 _ll_register_callbacks cb 0x5649b055ddf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:24:49.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.702+0000 7f3d9d9b5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649b0640330 0x5649b064af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:49.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.702+0000 7f3d9d1b4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649b0640c50 0x5649b0648220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:49.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.702+0000 7f3d9d9b5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649b0640330 0x5649b064af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:39488/0 (socket says 172.21.15.114:39488) 2024-09-13T18:24:49.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.702+0000 7f3d9d9b5640 1 -- 172.21.15.114:0/2229534228 learned_addr learned my addr 172.21.15.114:0/2229534228 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:49.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.702+0000 7f3d9e1b6640 1 --2- 172.21.15.114:0/2229534228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649b064b4f0 0x5649b0648990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:49.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.702+0000 7f3d9e1b6640 1 -- 172.21.15.114:0/2229534228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649b0640330 msgr2=0x5649b064af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:49.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.702+0000 7f3d9e1b6640 1 --2- 172.21.15.114:0/2229534228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649b0640330 0x5649b064af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:49.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.702+0000 7f3d9e1b6640 1 -- 172.21.15.114:0/2229534228 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649b0640c50 msgr2=0x5649b0648220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:49.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.702+0000 7f3d9e1b6640 1 --2- 172.21.15.114:0/2229534228 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649b0640c50 0x5649b0648220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:49.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.702+0000 7f3d9e1b6640 1 -- 172.21.15.114:0/2229534228 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d981a9490 con 0x5649b064b4f0 2024-09-13T18:24:49.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.702+0000 7f3d9d9b5640 1 --2- 172.21.15.114:0/2229534228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649b0640330 0x5649b064af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:24:49.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.703+0000 7f3d9e1b6640 1 --2- 172.21.15.114:0/2229534228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649b064b4f0 0x5649b0648990 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3d9000ffb0 tx=0x7f3d900cabf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a126557132d03827 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:49.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.703+0000 7f3d817fa640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:24:49.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.703+0000 7f3d817fa640 1 -- 172.21.15.114:0/2229534228 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d900a8070 con 0x5649b064b4f0 2024-09-13T18:24:49.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.703+0000 7f3d817fa640 1 -- 172.21.15.114:0/2229534228 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f3d9009d070 con 0x5649b064b4f0 2024-09-13T18:24:49.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.703+0000 7f3da2a1a740 1 -- 172.21.15.114:0/2229534228 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5649b05cd680 con 0x5649b064b4f0 2024-09-13T18:24:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.704+0000 7f3d817fa640 1 -- 172.21.15.114:0/2229534228 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d9009a800 con 0x5649b064b4f0 2024-09-13T18:24:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.704+0000 7f3da2a1a740 10 client.25936 fetch_fsmap learned FSMap version 32 2024-09-13T18:24:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.704+0000 7f3da2a1a740 1 -- 172.21.15.114:0/2229534228 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5649b05a3330 con 0x5649b064b4f0 2024-09-13T18:24:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.705+0000 7f3d817fa640 1 -- 172.21.15.114:0/2229534228 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3d9009a100 con 0x5649b064b4f0 2024-09-13T18:24:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.705+0000 7f3d817fa640 1 -- 172.21.15.114:0/2229534228 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f3d900ab030 con 0x5649b064b4f0 2024-09-13T18:24:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.705+0000 7f3da2a1a740 10 client.25936 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:24:49.714 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.705+0000 7f3da2a1a740 10 client.25936 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:24:49.714 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.706+0000 7f3da2a1a740 20 client.25936 populate_metadata read hostname 'smithi114' 2024-09-13T18:24:49.714 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.706+0000 7f3da2a1a740 1 --2- 172.21.15.114:0/2229534228 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5649b065a9d0 0x5649b065cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:49.714 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.706+0000 7f3da2a1a740 4 client.25936 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:24:49.714 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.706+0000 7f3da2a1a740 1 -- 172.21.15.114:0/2229534228 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5649b05cd680 con 0x5649b065a9d0 2024-09-13T18:24:49.714 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.706+0000 7f3d9d9b5640 1 --2- 172.21.15.114:0/2229534228 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5649b065a9d0 0x5649b065cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:49.714 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.707+0000 7f3d817fa640 10 client.25936 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:24:49.714 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.707+0000 7f3d9d9b5640 1 --2- 172.21.15.114:0/2229534228 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5649b065a9d0 0x5649b065cdc0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ddf3d849f04a0ce server_cookie=44b97ff2ca1960b0 in_seq=0 out_seq=0 2024-09-13T18:24:49.714 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.707+0000 7f3d817fa640 1 -- 172.21.15.114:0/2229534228 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5649b05cd680 con 0x5649b065a9d0 2024-09-13T18:24:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.707+0000 7f3d817fa640 10 client.25936 handle_command_reply: tid=0 2024-09-13T18:24:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.707+0000 7f3da2a1a740 1 client.25936 shutdown 2024-09-13T18:24:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.708+0000 7f3da2a1a740 1 -- 172.21.15.114:0/2229534228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649b064b4f0 msgr2=0x5649b0648990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.708+0000 7f3da2a1a740 1 --2- 172.21.15.114:0/2229534228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649b064b4f0 0x5649b0648990 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3d9000ffb0 tx=0x7f3d900cabf0 comp rx=0 tx=0).stop 2024-09-13T18:24:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.708+0000 7f3da2a1a740 1 -- 172.21.15.114:0/2229534228 shutdown_connections 2024-09-13T18:24:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.708+0000 7f3da2a1a740 1 --2- 172.21.15.114:0/2229534228 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5649b065a9d0 0x5649b065cdc0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.708+0000 7f3da2a1a740 1 --2- 172.21.15.114:0/2229534228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649b064b4f0 0x5649b0648990 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.708+0000 7f3da2a1a740 1 --2- 172.21.15.114:0/2229534228 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649b0640c50 0x5649b0648220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.708+0000 7f3da2a1a740 1 --2- 172.21.15.114:0/2229534228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649b0640330 0x5649b064af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:49.716 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.708+0000 7f3da2a1a740 1 -- 172.21.15.114:0/2229534228 >> 172.21.15.114:0/2229534228 conn(0x5649b063c8c0 msgr2=0x5649b063bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:49.716 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.708+0000 7f3d817fa640 0 client.25936 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:24:49.716 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.708+0000 7f3da2a1a740 1 -- 172.21.15.114:0/2229534228 shutdown_connections 2024-09-13T18:24:49.716 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.708+0000 7f3da2a1a740 1 -- 172.21.15.114:0/2229534228 wait complete. 2024-09-13T18:24:49.716 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.709+0000 7f3da2a1a740 20 client.25936 trim_cache size 0 max 16384 2024-09-13T18:24:49.743 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.739+0000 7f3da2a1a740 1 Processor -- start 2024-09-13T18:24:49.743 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.739+0000 7f3da2a1a740 1 -- start start 2024-09-13T18:24:49.743 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.740+0000 7f3da2a1a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649b0640330 0x5649b0647480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:49.744 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.740+0000 7f3da2a1a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649b0640c50 0x5649b06520c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:49.744 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.740+0000 7f3da2a1a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649b064b4f0 0x5649b0652600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:49.744 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.740+0000 7f3da2a1a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649b064deb0 con 0x5649b0640c50 2024-09-13T18:24:49.744 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.740+0000 7f3da2a1a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649b0638490 con 0x5649b0640330 2024-09-13T18:24:49.744 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.740+0000 7f3da2a1a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649b0642de0 con 0x5649b064b4f0 2024-09-13T18:24:49.744 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.740+0000 7f3d9d1b4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649b0640c50 0x5649b06520c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:49.744 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.740+0000 7f3d9e1b6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649b064b4f0 0x5649b0652600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:49.744 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.740+0000 7f3d9d9b5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649b0640330 0x5649b0647480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:49.744 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.740+0000 7f3d9d1b4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649b0640c50 0x5649b06520c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51960/0 (socket says 172.21.15.114:51960) 2024-09-13T18:24:49.744 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.740+0000 7f3d9d9b5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649b0640330 0x5649b0647480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:39500/0 (socket says 172.21.15.114:39500) 2024-09-13T18:24:49.744 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.740+0000 7f3d9d1b4640 1 -- 172.21.15.114:0/2965967263 learned_addr learned my addr 172.21.15.114:0/2965967263 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.740+0000 7f3d9d9b5640 1 -- 172.21.15.114:0/2965967263 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649b064b4f0 msgr2=0x5649b0652600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.740+0000 7f3d9d9b5640 1 --2- 172.21.15.114:0/2965967263 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649b064b4f0 0x5649b0652600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.740+0000 7f3d9d9b5640 1 -- 172.21.15.114:0/2965967263 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649b0640c50 msgr2=0x5649b06520c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.740+0000 7f3d9d9b5640 1 --2- 172.21.15.114:0/2965967263 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649b0640c50 0x5649b06520c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.740+0000 7f3d9d9b5640 1 -- 172.21.15.114:0/2965967263 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649b0545300 con 0x5649b0640330 2024-09-13T18:24:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.741+0000 7f3d9d9b5640 1 --2- 172.21.15.114:0/2965967263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649b0640330 0x5649b0647480 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x5649b065d6b0 tx=0x7f3d94005db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8a07ed1ff4e319 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.741+0000 7f3d8ce05640 1 -- 172.21.15.114:0/2965967263 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d940483b0 con 0x5649b0640330 2024-09-13T18:24:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.741+0000 7f3d8ce05640 1 -- 172.21.15.114:0/2965967263 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3d94043040 con 0x5649b0640330 2024-09-13T18:24:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.741+0000 7f3d8ce05640 1 -- 172.21.15.114:0/2965967263 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d94004ec0 con 0x5649b0640330 2024-09-13T18:24:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.742+0000 7f3da2a1a740 1 -- 172.21.15.114:0/2965967263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649b0640330 msgr2=0x5649b0647480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.742+0000 7f3da2a1a740 1 --2- 172.21.15.114:0/2965967263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649b0640330 0x5649b0647480 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x5649b065d6b0 tx=0x7f3d94005db0 comp rx=0 tx=0).stop 2024-09-13T18:24:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.742+0000 7f3da2a1a740 1 -- 172.21.15.114:0/2965967263 shutdown_connections 2024-09-13T18:24:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.743+0000 7f3da2a1a740 1 --2- 172.21.15.114:0/2965967263 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649b064b4f0 0x5649b0652600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.743+0000 7f3da2a1a740 1 --2- 172.21.15.114:0/2965967263 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649b0640c50 0x5649b06520c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.743+0000 7f3da2a1a740 1 --2- 172.21.15.114:0/2965967263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649b0640330 0x5649b0647480 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.743+0000 7f3da2a1a740 1 -- 172.21.15.114:0/2965967263 >> 172.21.15.114:0/2965967263 conn(0x5649b063c8c0 msgr2=0x5649b063cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.743+0000 7f3da2a1a740 1 -- 172.21.15.114:0/2965967263 shutdown_connections 2024-09-13T18:24:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.743+0000 7f3da2a1a740 1 -- 172.21.15.114:0/2965967263 wait complete. 2024-09-13T18:24:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.743+0000 7f3da2a1a740 1 -- start start 2024-09-13T18:24:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.743+0000 7f3da2a1a740 1 Processor -- start 2024-09-13T18:24:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.744+0000 7f3da2a1a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649b0640330 0x5649b06522c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.744+0000 7f3da2a1a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649b0640c50 0x5649b0644a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.744+0000 7f3da2a1a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649b064b4f0 0x5649b0644fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.745+0000 7f3d9d9b5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649b0640330 0x5649b06522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.745+0000 7f3d9d1b4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649b0640c50 0x5649b0644a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.745+0000 7f3d9e1b6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649b064b4f0 0x5649b0644fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.745+0000 7f3d9d9b5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649b0640330 0x5649b06522c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51974/0 (socket says 172.21.15.114:51974) 2024-09-13T18:24:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.745+0000 7f3d9e1b6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649b064b4f0 0x5649b0644fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:57724/0 (socket says 172.21.15.114:57724) 2024-09-13T18:24:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.745+0000 7f3d9d9b5640 1 -- 172.21.15.114:0/96542332 learned_addr learned my addr 172.21.15.114:0/96542332 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:24:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.745+0000 7f3da2a1a740 1 -- 172.21.15.114:0/96542332 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649b0645540 con 0x5649b0640330 2024-09-13T18:24:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.745+0000 7f3da2a1a740 1 -- 172.21.15.114:0/96542332 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649b06456b0 con 0x5649b0640c50 2024-09-13T18:24:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.745+0000 7f3da2a1a740 1 -- 172.21.15.114:0/96542332 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649b0645870 con 0x5649b064b4f0 2024-09-13T18:24:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.745+0000 7f3d9e1b6640 1 -- 172.21.15.114:0/96542332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649b0640c50 msgr2=0x5649b0644a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.745+0000 7f3d9e1b6640 1 --2- 172.21.15.114:0/96542332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649b0640c50 0x5649b0644a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.745+0000 7f3d9e1b6640 1 -- 172.21.15.114:0/96542332 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649b0640330 msgr2=0x5649b06522c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:24:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.745+0000 7f3d9e1b6640 1 --2- 172.21.15.114:0/96542332 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649b0640330 0x5649b06522c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.745+0000 7f3d9e1b6640 1 -- 172.21.15.114:0/96542332 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d981139f0 con 0x5649b064b4f0 2024-09-13T18:24:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.745+0000 7f3da2a1a740 10 client.0 _ll_register_callbacks cb 0x7f3d9000fe90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:24:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.745+0000 7f3da2a1a740 1 -- 172.21.15.114:0/96542332 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3d900cae90 con 0x5649b064b4f0 2024-09-13T18:24:49.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.745+0000 7f3d9e1b6640 1 --2- 172.21.15.114:0/96542332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649b064b4f0 0x5649b0644fb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3d9000ccb0 tx=0x7f3d900b8420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7aa88c9ab4adbbde server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:24:49.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.745+0000 7f3d817fa640 10 client.25942 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:24:49.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.746+0000 7f3d817fa640 1 -- 172.21.15.114:0/96542332 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d900a8050 con 0x5649b064b4f0 2024-09-13T18:24:49.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.746+0000 7f3d817fa640 1 -- 172.21.15.114:0/96542332 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f3d9009d040 con 0x5649b064b4f0 2024-09-13T18:24:49.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.746+0000 7f3d817fa640 1 -- 172.21.15.114:0/96542332 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d900998b0 con 0x5649b064b4f0 2024-09-13T18:24:49.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.746+0000 7f3da2a1a740 10 client.25942 fetch_fsmap learned FSMap version 32 2024-09-13T18:24:49.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.746+0000 7f3da2a1a740 1 -- 172.21.15.114:0/96542332 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({fsmap=32}) v3 -- 0x5649b056a8e0 con 0x5649b064b4f0 2024-09-13T18:24:49.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.746+0000 7f3d817fa640 1 -- 172.21.15.114:0/96542332 <== mon.1 v2:172.21.15.204:3300/0 4 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f3d900ab060 con 0x5649b064b4f0 2024-09-13T18:24:49.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.746+0000 7f3da2a1a740 10 client.25942 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:24:49.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.746+0000 7f3da2a1a740 10 client.25942 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:24:49.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.746+0000 7f3da2a1a740 20 client.25942 populate_metadata read hostname 'smithi114' 2024-09-13T18:24:49.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.746+0000 7f3da2a1a740 1 --2- 172.21.15.114:0/96542332 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5649b06b1d90 0x5649b06b41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:24:49.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.746+0000 7f3da2a1a740 4 client.25942 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T18:24:49.754 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.747+0000 7f3da2a1a740 1 -- 172.21.15.114:0/96542332 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5649b05985d0 con 0x5649b06b1d90 2024-09-13T18:24:49.754 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.747+0000 7f3d9d9b5640 1 --2- 172.21.15.114:0/96542332 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5649b06b1d90 0x5649b06b41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:24:49.754 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.747+0000 7f3d9d9b5640 1 --2- 172.21.15.114:0/96542332 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5649b06b1d90 0x5649b06b41b0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db8e7748de3c47cd server_cookie=3eeafa396df884f5 in_seq=0 out_seq=0 2024-09-13T18:24:49.754 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.747+0000 7f3d817fa640 10 client.25942 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:24:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.759+0000 7f3d817fa640 1 -- 172.21.15.114:0/96542332 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5649b05985d0 con 0x5649b06b1d90 2024-09-13T18:24:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.759+0000 7f3d817fa640 10 client.25942 handle_command_reply: tid=0 2024-09-13T18:24:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.759+0000 7f3da2a1a740 1 client.25942 shutdown 2024-09-13T18:24:49.766 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.761+0000 7f3da2a1a740 1 -- 172.21.15.114:0/96542332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649b064b4f0 msgr2=0x5649b0644fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:49.766 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.761+0000 7f3da2a1a740 1 --2- 172.21.15.114:0/96542332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649b064b4f0 0x5649b0644fb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3d9000ccb0 tx=0x7f3d900b8420 comp rx=0 tx=0).stop 2024-09-13T18:24:49.766 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.761+0000 7f3da2a1a740 1 -- 172.21.15.114:0/96542332 shutdown_connections 2024-09-13T18:24:49.766 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.761+0000 7f3da2a1a740 1 --2- 172.21.15.114:0/96542332 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5649b06b1d90 0x5649b06b41b0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:49.766 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.761+0000 7f3da2a1a740 1 --2- 172.21.15.114:0/96542332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649b064b4f0 0x5649b0644fb0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:49.766 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.761+0000 7f3d817fa640 0 client.25942 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:24:49.766 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.761+0000 7f3da2a1a740 1 --2- 172.21.15.114:0/96542332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649b0640c50 0x5649b0644a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:49.766 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.761+0000 7f3da2a1a740 1 --2- 172.21.15.114:0/96542332 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649b0640330 0x5649b06522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:49.767 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.762+0000 7f3da2a1a740 1 -- 172.21.15.114:0/96542332 >> 172.21.15.114:0/96542332 conn(0x5649b063c8c0 msgr2=0x5649b06434f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:49.767 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.762+0000 7f3da2a1a740 1 -- 172.21.15.114:0/96542332 shutdown_connections 2024-09-13T18:24:49.767 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.762+0000 7f3da2a1a740 1 -- 172.21.15.114:0/96542332 wait complete. 2024-09-13T18:24:49.767 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.763+0000 7f3da2a1a740 20 client.25942 trim_cache size 0 max 16384 2024-09-13T18:24:49.767 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-13T18:24:49.767 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-13T18:24:49.767 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-13T18:24:49.767 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-13T18:24:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.764+0000 7f3d9fc19640 1 -- 172.21.15.114:0/2729770551 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f3d840a13d0 msgr2=0x7f3d840a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.764+0000 7f3d9fc19640 1 --2- 172.21.15.114:0/2729770551 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f3d840a13d0 0x7f3d840a3890 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3d9819dc70 tx=0x7f3d880127c0 comp rx=0 tx=0).stop 2024-09-13T18:24:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.764+0000 7f3d9fc19640 1 -- 172.21.15.114:0/2729770551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d98110cd0 msgr2=0x7f3d981a1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:24:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.764+0000 7f3d9fc19640 1 --2- 172.21.15.114:0/2729770551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d98110cd0 0x7f3d981a1000 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3d9000cce0 tx=0x7f3d9000a620 comp rx=0 tx=0).stop 2024-09-13T18:24:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.764+0000 7f3d9fc19640 1 -- 172.21.15.114:0/2729770551 shutdown_connections 2024-09-13T18:24:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.764+0000 7f3d9fc19640 1 --2- 172.21.15.114:0/2729770551 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f3d840a13d0 0x7f3d840a3890 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.764+0000 7f3d9fc19640 1 --2- 172.21.15.114:0/2729770551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d98110cd0 0x7f3d981a1000 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.764+0000 7f3d9fc19640 1 --2- 172.21.15.114:0/2729770551 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3d98073170 0x7f3d9819cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.764+0000 7f3d9fc19640 1 --2- 172.21.15.114:0/2729770551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d98072850 0x7f3d9819c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:24:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.764+0000 7f3d9fc19640 1 -- 172.21.15.114:0/2729770551 >> 172.21.15.114:0/2729770551 conn(0x7f3d980fb9c0 msgr2=0x7f3d980fd410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:24:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.765+0000 7f3d9fc19640 1 -- 172.21.15.114:0/2729770551 shutdown_connections 2024-09-13T18:24:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:24:49.765+0000 7f3d9fc19640 1 -- 172.21.15.114:0/2729770551 wait complete. 2024-09-13T18:24:50.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:50 smithi204 ceph-mon[34777]: pgmap v74: 129 pgs: 129 active+clean; 10 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 118 MiB/s wr, 966 op/s 2024-09-13T18:24:50.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:50 smithi114 ceph-mon[30897]: pgmap v74: 129 pgs: 129 active+clean; 10 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 118 MiB/s wr, 966 op/s 2024-09-13T18:24:50.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:50 smithi134 ceph-mon[34283]: pgmap v74: 129 pgs: 129 active+clean; 10 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 118 MiB/s wr, 966 op/s 2024-09-13T18:24:52.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:52 smithi204 ceph-mon[34777]: pgmap v75: 129 pgs: 129 active+clean; 10 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 53 MiB/s wr, 853 op/s 2024-09-13T18:24:52.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:52 smithi114 ceph-mon[30897]: pgmap v75: 129 pgs: 129 active+clean; 10 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 53 MiB/s wr, 853 op/s 2024-09-13T18:24:52.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:52 smithi134 ceph-mon[34283]: pgmap v75: 129 pgs: 129 active+clean; 10 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 53 MiB/s wr, 853 op/s 2024-09-13T18:24:54.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:54 smithi204 ceph-mon[34777]: pgmap v76: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 64 MiB/s wr, 1.04k op/s 2024-09-13T18:24:54.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:54 smithi114 ceph-mon[30897]: pgmap v76: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 64 MiB/s wr, 1.04k op/s 2024-09-13T18:24:54.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:54 smithi134 ceph-mon[34283]: pgmap v76: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 64 MiB/s wr, 1.04k op/s 2024-09-13T18:24:56.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:56 smithi204 ceph-mon[34777]: pgmap v77: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 37 MiB/s wr, 610 op/s 2024-09-13T18:24:56.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:56 smithi114 ceph-mon[30897]: pgmap v77: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 37 MiB/s wr, 610 op/s 2024-09-13T18:24:56.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:56 smithi134 ceph-mon[34283]: pgmap v77: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 37 MiB/s wr, 610 op/s 2024-09-13T18:24:57.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:57 smithi204 ceph-mon[34777]: scrub summary: idle 2024-09-13T18:24:57.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:57 smithi204 ceph-mon[34777]: scrub summary: idle 2024-09-13T18:24:57.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:57 smithi114 ceph-mon[30897]: scrub summary: idle 2024-09-13T18:24:57.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:57 smithi114 ceph-mon[30897]: scrub summary: idle 2024-09-13T18:24:57.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:57 smithi134 ceph-mon[34283]: scrub summary: idle 2024-09-13T18:24:57.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:57 smithi134 ceph-mon[34283]: scrub summary: idle 2024-09-13T18:24:58.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:24:58 smithi114 ceph-mon[30897]: pgmap v78: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 37 MiB/s wr, 610 op/s 2024-09-13T18:24:58.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:24:58 smithi134 ceph-mon[34283]: pgmap v78: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 37 MiB/s wr, 610 op/s 2024-09-13T18:24:58.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:24:58 smithi204 ceph-mon[34777]: pgmap v78: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 37 MiB/s wr, 610 op/s 2024-09-13T18:25:00.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:00 smithi204 ceph-mon[34777]: pgmap v79: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 38 MiB/s wr, 624 op/s 2024-09-13T18:25:00.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:00 smithi114 ceph-mon[30897]: pgmap v79: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 38 MiB/s wr, 624 op/s 2024-09-13T18:25:00.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:00 smithi134 ceph-mon[34283]: pgmap v79: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 38 MiB/s wr, 624 op/s 2024-09-13T18:25:02.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:02 smithi204 ceph-mon[34777]: pgmap v80: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 12 MiB/s wr, 206 op/s 2024-09-13T18:25:02.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:02 smithi114 ceph-mon[30897]: pgmap v80: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 12 MiB/s wr, 206 op/s 2024-09-13T18:25:02.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:02 smithi134 ceph-mon[34283]: pgmap v80: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 12 MiB/s wr, 206 op/s 2024-09-13T18:25:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.869+0000 7f4a79184640 1 -- 172.21.15.114:0/3304353761 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4a74100e80 msgr2=0x7f4a74101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.869+0000 7f4a79184640 1 --2- 172.21.15.114:0/3304353761 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4a74100e80 0x7f4a74101260 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f4a64009900 tx=0x7f4a6402a480 comp rx=0 tx=0).stop 2024-09-13T18:25:02.873 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.869+0000 7f4a79184640 1 -- 172.21.15.114:0/3304353761 shutdown_connections 2024-09-13T18:25:02.873 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.869+0000 7f4a79184640 1 --2- 172.21.15.114:0/3304353761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a7410a2a0 0x7f4a7410c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:02.873 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.869+0000 7f4a79184640 1 --2- 172.21.15.114:0/3304353761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a741017a0 0x7f4a74109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:02.873 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.869+0000 7f4a79184640 1 --2- 172.21.15.114:0/3304353761 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4a74100e80 0x7f4a74101260 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:02.873 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.869+0000 7f4a79184640 1 -- 172.21.15.114:0/3304353761 >> 172.21.15.114:0/3304353761 conn(0x7f4a740fa8f0 msgr2=0x7f4a740fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:02.873 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.870+0000 7f4a79184640 1 -- 172.21.15.114:0/3304353761 shutdown_connections 2024-09-13T18:25:02.873 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.870+0000 7f4a79184640 1 -- 172.21.15.114:0/3304353761 wait complete. 2024-09-13T18:25:02.873 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.870+0000 7f4a79184640 1 Processor -- start 2024-09-13T18:25:02.876 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.871+0000 7f4a79184640 1 -- start start 2024-09-13T18:25:02.876 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.871+0000 7f4a79184640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a74100e80 0x7f4a74196090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:02.876 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.871+0000 7f4a79184640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4a741017a0 0x7f4a741965d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:02.876 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.871+0000 7f4a79184640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a7410a2a0 0x7f4a7419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:02.876 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.871+0000 7f4a79184640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7419afc0 con 0x7f4a741017a0 2024-09-13T18:25:02.876 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.871+0000 7f4a79184640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7419b130 con 0x7f4a74100e80 2024-09-13T18:25:02.876 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.871+0000 7f4a79184640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7419b2f0 con 0x7f4a7410a2a0 2024-09-13T18:25:02.876 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.871+0000 7f4a7259c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4a741017a0 0x7f4a741965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:02.876 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.871+0000 7f4a7259c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4a741017a0 0x7f4a741965d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36750/0 (socket says 172.21.15.114:36750) 2024-09-13T18:25:02.876 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.871+0000 7f4a7259c640 1 -- 172.21.15.114:0/1536662378 learned_addr learned my addr 172.21.15.114:0/1536662378 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:02.877 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.871+0000 7f4a72d9d640 1 --2- 172.21.15.114:0/1536662378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a74100e80 0x7f4a74196090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:02.877 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.871+0000 7f4a7359e640 1 --2- 172.21.15.114:0/1536662378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a7410a2a0 0x7f4a7419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:02.877 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.871+0000 7f4a7259c640 1 -- 172.21.15.114:0/1536662378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a7410a2a0 msgr2=0x7f4a7419a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:02.877 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.871+0000 7f4a7259c640 1 --2- 172.21.15.114:0/1536662378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a7410a2a0 0x7f4a7419a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:02.877 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.871+0000 7f4a7259c640 1 -- 172.21.15.114:0/1536662378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a74100e80 msgr2=0x7f4a74196090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:02.877 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.871+0000 7f4a7259c640 1 --2- 172.21.15.114:0/1536662378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a74100e80 0x7f4a74196090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:02.877 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.871+0000 7f4a7259c640 1 -- 172.21.15.114:0/1536662378 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a64009560 con 0x7f4a741017a0 2024-09-13T18:25:02.877 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.871+0000 7f4a72d9d640 1 --2- 172.21.15.114:0/1536662378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a74100e80 0x7f4a74196090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:25:02.877 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.872+0000 7f4a7259c640 1 --2- 172.21.15.114:0/1536662378 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4a741017a0 0x7f4a741965d0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f4a6800d950 tx=0x7f4a6800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.872+0000 7f4a5bfff640 1 -- 172.21.15.114:0/1536662378 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a68009c30 con 0x7f4a741017a0 2024-09-13T18:25:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.872+0000 7f4a5bfff640 1 -- 172.21.15.114:0/1536662378 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4a68009d90 con 0x7f4a741017a0 2024-09-13T18:25:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.872+0000 7f4a79184640 1 -- 172.21.15.114:0/1536662378 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a7419b5d0 con 0x7f4a741017a0 2024-09-13T18:25:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.872+0000 7f4a5bfff640 1 -- 172.21.15.114:0/1536662378 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a6800f620 con 0x7f4a741017a0 2024-09-13T18:25:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.872+0000 7f4a79184640 1 -- 172.21.15.114:0/1536662378 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a741a2e50 con 0x7f4a741017a0 2024-09-13T18:25:02.885 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.874+0000 7f4a5bfff640 1 -- 172.21.15.114:0/1536662378 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f4a680109f0 con 0x7f4a741017a0 2024-09-13T18:25:02.885 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.875+0000 7f4a7bea4740 1 Processor -- start 2024-09-13T18:25:02.885 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.875+0000 7f4a7bea4740 1 -- start start 2024-09-13T18:25:02.885 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.875+0000 7f4a7bea4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4ee38a1f0 0x55e4ee38a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:02.885 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.875+0000 7f4a7bea4740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4ee38ab10 0x55e4ee3993c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:02.885 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.875+0000 7f4a7bea4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4ee399900 0x55e4ee39bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:02.885 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.875+0000 7f4a7bea4740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4ee2f5c30 con 0x55e4ee38ab10 2024-09-13T18:25:02.886 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.875+0000 7f4a7bea4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4ee2f33c0 con 0x55e4ee399900 2024-09-13T18:25:02.886 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.875+0000 7f4a7bea4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4ee2eecb0 con 0x55e4ee38a1f0 2024-09-13T18:25:02.886 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.875+0000 7f4a5bfff640 1 --2- 172.21.15.114:0/1536662378 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f4a5c0a1200 0x7f4a5c0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:02.886 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.875+0000 7f4a72d9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4ee38ab10 0x55e4ee3993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:02.886 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.875+0000 7f4a72d9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4ee38ab10 0x55e4ee3993c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36762/0 (socket says 172.21.15.114:36762) 2024-09-13T18:25:02.886 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.875+0000 7f4a72d9d640 1 -- 172.21.15.114:0/968486230 learned_addr learned my addr 172.21.15.114:0/968486230 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:02.886 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.875+0000 7f4a7359e640 1 --2- 172.21.15.114:0/1536662378 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f4a5c0a1200 0x7f4a5c0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:02.886 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.875+0000 7f4a7359e640 1 --2- 172.21.15.114:0/968486230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4ee38a1f0 0x55e4ee38a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:02.886 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.876+0000 7f4a5bfff640 1 -- 172.21.15.114:0/1536662378 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f4a680ccd40 con 0x7f4a741017a0 2024-09-13T18:25:02.886 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.876+0000 7f4a7259c640 1 --2- 172.21.15.114:0/968486230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4ee399900 0x55e4ee39bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:02.886 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.876+0000 7f4a72d9d640 1 -- 172.21.15.114:0/968486230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4ee38a1f0 msgr2=0x55e4ee38a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:02.886 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.876+0000 7f4a72d9d640 1 --2- 172.21.15.114:0/968486230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4ee38a1f0 0x55e4ee38a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:02.886 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.876+0000 7f4a72d9d640 1 -- 172.21.15.114:0/968486230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4ee399900 msgr2=0x55e4ee39bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:02.887 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.876+0000 7f4a72d9d640 1 --2- 172.21.15.114:0/968486230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4ee399900 0x55e4ee39bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:02.887 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.876+0000 7f4a72d9d640 1 -- 172.21.15.114:0/968486230 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4ee295910 con 0x55e4ee38ab10 2024-09-13T18:25:02.887 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.876+0000 7f4a7359e640 1 --2- 172.21.15.114:0/968486230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4ee38a1f0 0x55e4ee38a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:25:02.887 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.876+0000 7f4a72d9d640 1 --2- 172.21.15.114:0/968486230 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4ee38ab10 0x55e4ee3993c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f4a64009900 tx=0x7f4a64031620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a950531d0bc2150 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:02.887 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.876+0000 7f4a7359e640 1 --2- 172.21.15.114:0/1536662378 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f4a5c0a1200 0x7f4a5c0a36c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c005cc0 tx=0x7f4a6c00b300 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:02.887 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.876+0000 7f4a597fa640 1 -- 172.21.15.114:0/968486230 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a64004230 con 0x55e4ee38ab10 2024-09-13T18:25:02.887 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.876+0000 7f4a597fa640 1 -- 172.21.15.114:0/968486230 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4a64031db0 con 0x55e4ee38ab10 2024-09-13T18:25:02.887 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.876+0000 7f4a597fa640 1 -- 172.21.15.114:0/968486230 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a64003ca0 con 0x55e4ee38ab10 2024-09-13T18:25:02.887 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.877+0000 7f4a7bea4740 1 -- 172.21.15.114:0/968486230 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4ee38ab10 msgr2=0x55e4ee3993c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.877+0000 7f4a7bea4740 1 --2- 172.21.15.114:0/968486230 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4ee38ab10 0x55e4ee3993c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f4a64009900 tx=0x7f4a64031620 comp rx=0 tx=0).stop 2024-09-13T18:25:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.877+0000 7f4a7bea4740 1 -- 172.21.15.114:0/968486230 shutdown_connections 2024-09-13T18:25:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.877+0000 7f4a7bea4740 1 --2- 172.21.15.114:0/968486230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4ee399900 0x55e4ee39bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.877+0000 7f4a7bea4740 1 --2- 172.21.15.114:0/968486230 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4ee38ab10 0x55e4ee3993c0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.877+0000 7f4a7bea4740 1 --2- 172.21.15.114:0/968486230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4ee38a1f0 0x55e4ee38a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.877+0000 7f4a7bea4740 1 -- 172.21.15.114:0/968486230 >> 172.21.15.114:0/968486230 conn(0x55e4ee3868c0 msgr2=0x55e4ee386ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.877+0000 7f4a7bea4740 1 -- 172.21.15.114:0/968486230 shutdown_connections 2024-09-13T18:25:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.877+0000 7f4a7bea4740 1 -- 172.21.15.114:0/968486230 wait complete. 2024-09-13T18:25:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.877+0000 7f4a7bea4740 1 -- start start 2024-09-13T18:25:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.877+0000 7f4a7bea4740 1 Processor -- start 2024-09-13T18:25:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.877+0000 7f4a7bea4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4ee38a1f0 0x55e4ee399340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.878+0000 7f4a7bea4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4ee38ab10 0x55e4ee3965c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.878+0000 7f4a7bea4740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4ee399900 0x55e4ee396d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.878+0000 7f4a7bea4740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4ee2ace80 con 0x55e4ee399900 2024-09-13T18:25:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.878+0000 7f4a7bea4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4ee2acb20 con 0x55e4ee38a1f0 2024-09-13T18:25:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.878+0000 7f4a7bea4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4ee272260 con 0x55e4ee38ab10 2024-09-13T18:25:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.878+0000 7f4a7bea4740 10 client.0 _ll_register_callbacks cb 0x55e4ee2a7df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:25:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.878+0000 7f4a7359e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4ee399900 0x55e4ee396d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.878+0000 7f4a7359e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4ee399900 0x55e4ee396d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36768/0 (socket says 172.21.15.114:36768) 2024-09-13T18:25:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.878+0000 7f4a7359e640 1 -- 172.21.15.114:0/1130700019 learned_addr learned my addr 172.21.15.114:0/1130700019 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.878+0000 7f4a7259c640 1 --2- 172.21.15.114:0/1130700019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4ee38ab10 0x55e4ee3965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.878+0000 7f4a72d9d640 1 --2- 172.21.15.114:0/1130700019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4ee38a1f0 0x55e4ee399340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.878+0000 7f4a7359e640 1 -- 172.21.15.114:0/1130700019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4ee38ab10 msgr2=0x55e4ee3965c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.878+0000 7f4a7359e640 1 --2- 172.21.15.114:0/1130700019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4ee38ab10 0x55e4ee3965c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.878+0000 7f4a7359e640 1 -- 172.21.15.114:0/1130700019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4ee38a1f0 msgr2=0x55e4ee399340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.878+0000 7f4a7359e640 1 --2- 172.21.15.114:0/1130700019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4ee38a1f0 0x55e4ee399340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.878+0000 7f4a7359e640 1 -- 172.21.15.114:0/1130700019 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a7410d3e0 con 0x55e4ee399900 2024-09-13T18:25:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.878+0000 7f4a72d9d640 1 --2- 172.21.15.114:0/1130700019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4ee38a1f0 0x55e4ee399340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:25:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.878+0000 7f4a7359e640 1 --2- 172.21.15.114:0/1130700019 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4ee399900 0x55e4ee396d50 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c007cf0 tx=0x7f4a6c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52cb96021c9dbe1a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.878+0000 7f4a56ff5640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:25:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.878+0000 7f4a56ff5640 1 -- 172.21.15.114:0/1130700019 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c00f650 con 0x55e4ee399900 2024-09-13T18:25:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.878+0000 7f4a56ff5640 1 -- 172.21.15.114:0/1130700019 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f4a6c013e70 con 0x55e4ee399900 2024-09-13T18:25:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.878+0000 7f4a7bea4740 1 -- 172.21.15.114:0/1130700019 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e4ee317680 con 0x55e4ee399900 2024-09-13T18:25:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.879+0000 7f4a56ff5640 1 -- 172.21.15.114:0/1130700019 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a6c012e60 con 0x55e4ee399900 2024-09-13T18:25:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.879+0000 7f4a7bea4740 10 client.16587 fetch_fsmap learned FSMap version 32 2024-09-13T18:25:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.879+0000 7f4a7bea4740 1 -- 172.21.15.114:0/1130700019 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55e4ee2ed330 con 0x55e4ee399900 2024-09-13T18:25:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.879+0000 7f4a56ff5640 1 -- 172.21.15.114:0/1130700019 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e4ee317680 con 0x55e4ee399900 2024-09-13T18:25:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.879+0000 7f4a56ff5640 1 -- 172.21.15.114:0/1130700019 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f4a6c01f690 con 0x55e4ee399900 2024-09-13T18:25:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.879+0000 7f4a7bea4740 10 client.16587 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:25:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.879+0000 7f4a7bea4740 10 client.16587 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:25:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.879+0000 7f4a7bea4740 20 client.16587 populate_metadata read hostname 'smithi114' 2024-09-13T18:25:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.879+0000 7f4a7bea4740 1 --2- 172.21.15.114:0/1130700019 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55e4ee3a9400 0x55e4ee3ab820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.879+0000 7f4a7bea4740 4 client.16587 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:25:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.879+0000 7f4a7bea4740 1 -- 172.21.15.114:0/1130700019 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e4ee2e25d0 con 0x55e4ee3a9400 2024-09-13T18:25:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.879+0000 7f4a72d9d640 1 --2- 172.21.15.114:0/1130700019 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55e4ee3a9400 0x55e4ee3ab820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.880+0000 7f4a56ff5640 10 client.16587 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:25:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.880+0000 7f4a72d9d640 1 --2- 172.21.15.114:0/1130700019 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55e4ee3a9400 0x55e4ee3ab820 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d68bf298a19b9624 server_cookie=74a96e276c59cdd4 in_seq=0 out_seq=0 2024-09-13T18:25:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.880+0000 7f4a56ff5640 1 -- 172.21.15.114:0/1130700019 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55e4ee2e25d0 con 0x55e4ee3a9400 2024-09-13T18:25:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.880+0000 7f4a56ff5640 10 client.16587 handle_command_reply: tid=0 2024-09-13T18:25:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.880+0000 7f4a7bea4740 1 client.16587 shutdown 2024-09-13T18:25:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.880+0000 7f4a7bea4740 1 -- 172.21.15.114:0/1130700019 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4ee399900 msgr2=0x55e4ee396d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.880+0000 7f4a7bea4740 1 --2- 172.21.15.114:0/1130700019 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4ee399900 0x55e4ee396d50 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c007cf0 tx=0x7f4a6c004a40 comp rx=0 tx=0).stop 2024-09-13T18:25:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.880+0000 7f4a7bea4740 1 -- 172.21.15.114:0/1130700019 shutdown_connections 2024-09-13T18:25:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.881+0000 7f4a7bea4740 1 --2- 172.21.15.114:0/1130700019 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55e4ee3a9400 0x55e4ee3ab820 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.881+0000 7f4a7bea4740 1 --2- 172.21.15.114:0/1130700019 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4ee399900 0x55e4ee396d50 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.881+0000 7f4a7bea4740 1 --2- 172.21.15.114:0/1130700019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4ee38ab10 0x55e4ee3965c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.881+0000 7f4a56ff5640 0 client.16587 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:25:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.881+0000 7f4a7bea4740 1 --2- 172.21.15.114:0/1130700019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4ee38a1f0 0x55e4ee399340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.881+0000 7f4a7bea4740 1 -- 172.21.15.114:0/1130700019 >> 172.21.15.114:0/1130700019 conn(0x55e4ee3868c0 msgr2=0x55e4ee38c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.881+0000 7f4a7bea4740 1 -- 172.21.15.114:0/1130700019 shutdown_connections 2024-09-13T18:25:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.881+0000 7f4a7bea4740 1 -- 172.21.15.114:0/1130700019 wait complete. 2024-09-13T18:25:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.881+0000 7f4a7bea4740 20 client.16587 trim_cache size 0 max 16384 2024-09-13T18:25:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.893+0000 7f4a7bea4740 1 Processor -- start 2024-09-13T18:25:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.893+0000 7f4a7bea4740 1 -- start start 2024-09-13T18:25:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.893+0000 7f4a7bea4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4ee38a1f0 0x55e4ee3918c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.893+0000 7f4a7bea4740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4ee38ab10 0x55e4ee391e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.893+0000 7f4a7bea4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4ee399900 0x55e4ee395e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.893+0000 7f4a7bea4740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4ee386720 con 0x55e4ee38ab10 2024-09-13T18:25:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.893+0000 7f4a7bea4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4ee387820 con 0x55e4ee399900 2024-09-13T18:25:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.893+0000 7f4a7bea4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4ee387320 con 0x55e4ee38a1f0 2024-09-13T18:25:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.893+0000 7f4a7259c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4ee38ab10 0x55e4ee391e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.893+0000 7f4a7259c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4ee38ab10 0x55e4ee391e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36770/0 (socket says 172.21.15.114:36770) 2024-09-13T18:25:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.893+0000 7f4a7259c640 1 -- 172.21.15.114:0/1302173896 learned_addr learned my addr 172.21.15.114:0/1302173896 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.894+0000 7f4a72d9d640 1 --2- 172.21.15.114:0/1302173896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4ee38a1f0 0x55e4ee3918c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.894+0000 7f4a7259c640 1 -- 172.21.15.114:0/1302173896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4ee38a1f0 msgr2=0x55e4ee3918c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.894+0000 7f4a7259c640 1 --2- 172.21.15.114:0/1302173896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4ee38a1f0 0x55e4ee3918c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.894+0000 7f4a7359e640 1 --2- 172.21.15.114:0/1302173896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4ee399900 0x55e4ee395e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.894+0000 7f4a7259c640 1 -- 172.21.15.114:0/1302173896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4ee399900 msgr2=0x55e4ee395e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.894+0000 7f4a7259c640 1 --2- 172.21.15.114:0/1302173896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4ee399900 0x55e4ee395e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.894+0000 7f4a7259c640 1 -- 172.21.15.114:0/1302173896 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4ee28f300 con 0x55e4ee38ab10 2024-09-13T18:25:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.894+0000 7f4a7259c640 1 --2- 172.21.15.114:0/1302173896 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4ee38ab10 0x55e4ee391e00 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f4a6800dfe0 tx=0x7f4a68009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d6f0255f04a675b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.894+0000 7f4a59ffb640 1 -- 172.21.15.114:0/1302173896 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a68099aa0 con 0x55e4ee38ab10 2024-09-13T18:25:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.894+0000 7f4a59ffb640 1 -- 172.21.15.114:0/1302173896 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4a6809d8d0 con 0x55e4ee38ab10 2024-09-13T18:25:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.894+0000 7f4a59ffb640 1 -- 172.21.15.114:0/1302173896 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a680b2210 con 0x55e4ee38ab10 2024-09-13T18:25:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.895+0000 7f4a7bea4740 1 -- 172.21.15.114:0/1302173896 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4ee38ab10 msgr2=0x55e4ee391e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.895+0000 7f4a7bea4740 1 --2- 172.21.15.114:0/1302173896 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4ee38ab10 0x55e4ee391e00 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f4a6800dfe0 tx=0x7f4a68009f00 comp rx=0 tx=0).stop 2024-09-13T18:25:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.895+0000 7f4a7bea4740 1 -- 172.21.15.114:0/1302173896 shutdown_connections 2024-09-13T18:25:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.895+0000 7f4a7bea4740 1 --2- 172.21.15.114:0/1302173896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4ee399900 0x55e4ee395e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.895+0000 7f4a7bea4740 1 --2- 172.21.15.114:0/1302173896 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4ee38ab10 0x55e4ee391e00 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.895+0000 7f4a7bea4740 1 --2- 172.21.15.114:0/1302173896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4ee38a1f0 0x55e4ee3918c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:02.903 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.895+0000 7f4a7bea4740 1 -- 172.21.15.114:0/1302173896 >> 172.21.15.114:0/1302173896 conn(0x55e4ee3868c0 msgr2=0x55e4ee3b01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:02.903 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.895+0000 7f4a7bea4740 1 -- 172.21.15.114:0/1302173896 shutdown_connections 2024-09-13T18:25:02.903 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.895+0000 7f4a7bea4740 1 -- 172.21.15.114:0/1302173896 wait complete. 2024-09-13T18:25:02.903 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.895+0000 7f4a7bea4740 1 -- start start 2024-09-13T18:25:02.903 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.895+0000 7f4a7bea4740 1 Processor -- start 2024-09-13T18:25:02.903 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.895+0000 7f4a7bea4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4ee38a1f0 0x55e4ee393180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:02.903 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.895+0000 7f4a7bea4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4ee38ab10 0x55e4ee3936c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:02.903 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.895+0000 7f4a7bea4740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4ee399900 0x55e4ee393c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:02.903 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.895+0000 7f4a7bea4740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4ee3a0e90 con 0x55e4ee399900 2024-09-13T18:25:02.903 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.895+0000 7f4a7bea4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4ee3ff920 con 0x55e4ee38a1f0 2024-09-13T18:25:02.903 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.895+0000 7f4a7bea4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4ee3ffa90 con 0x55e4ee38ab10 2024-09-13T18:25:02.903 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.895+0000 7f4a7bea4740 10 client.0 _ll_register_callbacks cb 0x7f4a64009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:25:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.895+0000 7f4a7359e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4ee399900 0x55e4ee393c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.895+0000 7f4a7359e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4ee399900 0x55e4ee393c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36776/0 (socket says 172.21.15.114:36776) 2024-09-13T18:25:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.895+0000 7f4a7359e640 1 -- 172.21.15.114:0/2043984459 learned_addr learned my addr 172.21.15.114:0/2043984459 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.895+0000 7f4a7259c640 1 --2- 172.21.15.114:0/2043984459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4ee38ab10 0x55e4ee3936c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.896+0000 7f4a72d9d640 1 --2- 172.21.15.114:0/2043984459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4ee38a1f0 0x55e4ee393180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.896+0000 7f4a7359e640 1 -- 172.21.15.114:0/2043984459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4ee38ab10 msgr2=0x55e4ee3936c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.896+0000 7f4a7359e640 1 --2- 172.21.15.114:0/2043984459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4ee38ab10 0x55e4ee3936c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.896+0000 7f4a7359e640 1 -- 172.21.15.114:0/2043984459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4ee38a1f0 msgr2=0x55e4ee393180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.896+0000 7f4a7359e640 1 --2- 172.21.15.114:0/2043984459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4ee38a1f0 0x55e4ee393180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.896+0000 7f4a7359e640 1 -- 172.21.15.114:0/2043984459 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a741a2e50 con 0x55e4ee399900 2024-09-13T18:25:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.896+0000 7f4a7359e640 1 --2- 172.21.15.114:0/2043984459 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4ee399900 0x55e4ee393c00 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c007cc0 tx=0x7f4a6c00f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=130592ff59d59185 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.896+0000 7f4a56ff5640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:25:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.896+0000 7f4a56ff5640 1 -- 172.21.15.114:0/2043984459 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c01fd60 con 0x55e4ee399900 2024-09-13T18:25:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.896+0000 7f4a56ff5640 1 -- 172.21.15.114:0/2043984459 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f4a6c00bab0 con 0x55e4ee399900 2024-09-13T18:25:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.896+0000 7f4a7bea4740 1 -- 172.21.15.114:0/2043984459 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4a64031db0 con 0x55e4ee399900 2024-09-13T18:25:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.896+0000 7f4a56ff5640 1 -- 172.21.15.114:0/2043984459 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a6c00bc70 con 0x55e4ee399900 2024-09-13T18:25:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.896+0000 7f4a7bea4740 10 client.16593 fetch_fsmap learned FSMap version 32 2024-09-13T18:25:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.896+0000 7f4a7bea4740 1 -- 172.21.15.114:0/2043984459 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55e4ee2b48e0 con 0x55e4ee399900 2024-09-13T18:25:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.896+0000 7f4a56ff5640 1 -- 172.21.15.114:0/2043984459 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4a64031db0 con 0x55e4ee399900 2024-09-13T18:25:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.896+0000 7f4a56ff5640 1 -- 172.21.15.114:0/2043984459 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f4a6c004040 con 0x55e4ee399900 2024-09-13T18:25:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.896+0000 7f4a7bea4740 10 client.16593 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:25:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.896+0000 7f4a7bea4740 10 client.16593 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:25:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.896+0000 7f4a7bea4740 20 client.16593 populate_metadata read hostname 'smithi114' 2024-09-13T18:25:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.896+0000 7f4a7bea4740 1 --2- 172.21.15.114:0/2043984459 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55e4ee4012d0 0x55e4ee4036f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.896+0000 7f4a7bea4740 4 client.16593 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T18:25:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.896+0000 7f4a7bea4740 1 -- 172.21.15.114:0/2043984459 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e4edfe2300 con 0x55e4ee4012d0 2024-09-13T18:25:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.897+0000 7f4a72d9d640 1 --2- 172.21.15.114:0/2043984459 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55e4ee4012d0 0x55e4ee4036f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.897+0000 7f4a56ff5640 10 client.16593 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:25:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.897+0000 7f4a72d9d640 1 --2- 172.21.15.114:0/2043984459 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55e4ee4012d0 0x55e4ee4036f0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c455cd3f28d7eb17 server_cookie=d7fb49f1057d20b2 in_seq=0 out_seq=0 2024-09-13T18:25:02.917 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.914+0000 7f4a56ff5640 1 -- 172.21.15.114:0/2043984459 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e4edfe2300 con 0x55e4ee4012d0 2024-09-13T18:25:02.918 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.914+0000 7f4a56ff5640 10 client.16593 handle_command_reply: tid=0 2024-09-13T18:25:02.918 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.914+0000 7f4a7bea4740 1 client.16593 shutdown 2024-09-13T18:25:02.918 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.915+0000 7f4a7bea4740 1 -- 172.21.15.114:0/2043984459 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4ee399900 msgr2=0x55e4ee393c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:02.918 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.915+0000 7f4a7bea4740 1 --2- 172.21.15.114:0/2043984459 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4ee399900 0x55e4ee393c00 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c007cc0 tx=0x7f4a6c00f000 comp rx=0 tx=0).stop 2024-09-13T18:25:02.918 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.915+0000 7f4a7bea4740 1 -- 172.21.15.114:0/2043984459 shutdown_connections 2024-09-13T18:25:02.918 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.915+0000 7f4a7bea4740 1 --2- 172.21.15.114:0/2043984459 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55e4ee4012d0 0x55e4ee4036f0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:02.918 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.915+0000 7f4a7bea4740 1 --2- 172.21.15.114:0/2043984459 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4ee399900 0x55e4ee393c00 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:02.919 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.915+0000 7f4a56ff5640 0 client.16593 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:25:02.919 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.915+0000 7f4a7bea4740 1 --2- 172.21.15.114:0/2043984459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4ee38ab10 0x55e4ee3936c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:02.919 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.915+0000 7f4a7bea4740 1 --2- 172.21.15.114:0/2043984459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4ee38a1f0 0x55e4ee393180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:02.919 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.915+0000 7f4a7bea4740 1 -- 172.21.15.114:0/2043984459 >> 172.21.15.114:0/2043984459 conn(0x55e4ee3868c0 msgr2=0x55e4ee3918c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:02.919 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.915+0000 7f4a7bea4740 1 -- 172.21.15.114:0/2043984459 shutdown_connections 2024-09-13T18:25:02.919 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.915+0000 7f4a7bea4740 1 -- 172.21.15.114:0/2043984459 wait complete. 2024-09-13T18:25:02.919 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.915+0000 7f4a7bea4740 20 client.16593 trim_cache size 0 max 16384 2024-09-13T18:25:02.920 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-13T18:25:02.920 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-13T18:25:02.920 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-13T18:25:02.920 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-13T18:25:02.921 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.915+0000 7f4a79184640 1 -- 172.21.15.114:0/1536662378 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f4a5c0a1200 msgr2=0x7f4a5c0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:02.921 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.915+0000 7f4a79184640 1 --2- 172.21.15.114:0/1536662378 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f4a5c0a1200 0x7f4a5c0a36c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c005cc0 tx=0x7f4a6c00b300 comp rx=0 tx=0).stop 2024-09-13T18:25:02.921 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.915+0000 7f4a79184640 1 -- 172.21.15.114:0/1536662378 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4a741017a0 msgr2=0x7f4a741965d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:02.921 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.915+0000 7f4a79184640 1 --2- 172.21.15.114:0/1536662378 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4a741017a0 0x7f4a741965d0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f4a6800d950 tx=0x7f4a6800de10 comp rx=0 tx=0).stop 2024-09-13T18:25:02.921 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.916+0000 7f4a79184640 1 -- 172.21.15.114:0/1536662378 shutdown_connections 2024-09-13T18:25:02.921 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.916+0000 7f4a79184640 1 --2- 172.21.15.114:0/1536662378 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f4a5c0a1200 0x7f4a5c0a36c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.916+0000 7f4a79184640 1 --2- 172.21.15.114:0/1536662378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a7410a2a0 0x7f4a7419a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.916+0000 7f4a79184640 1 --2- 172.21.15.114:0/1536662378 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4a741017a0 0x7f4a741965d0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.916+0000 7f4a79184640 1 --2- 172.21.15.114:0/1536662378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a74100e80 0x7f4a74196090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.916+0000 7f4a79184640 1 -- 172.21.15.114:0/1536662378 >> 172.21.15.114:0/1536662378 conn(0x7f4a740fa8f0 msgr2=0x7f4a740fc250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.916+0000 7f4a79184640 1 -- 172.21.15.114:0/1536662378 shutdown_connections 2024-09-13T18:25:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:02.916+0000 7f4a79184640 1 -- 172.21.15.114:0/1536662378 wait complete. 2024-09-13T18:25:04.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:04 smithi204 ceph-mon[34777]: pgmap v81: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 18 MiB/s wr, 312 op/s 2024-09-13T18:25:04.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:04 smithi114 ceph-mon[30897]: pgmap v81: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 18 MiB/s wr, 312 op/s 2024-09-13T18:25:04.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:04 smithi134 ceph-mon[34283]: pgmap v81: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 18 MiB/s wr, 312 op/s 2024-09-13T18:25:06.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:06 smithi204 ceph-mon[34777]: pgmap v82: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.0 MiB/s wr, 120 op/s 2024-09-13T18:25:06.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:06 smithi114 ceph-mon[30897]: pgmap v82: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.0 MiB/s wr, 120 op/s 2024-09-13T18:25:06.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:06 smithi134 ceph-mon[34283]: pgmap v82: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.0 MiB/s wr, 120 op/s 2024-09-13T18:25:08.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:08 smithi204 ceph-mon[34777]: pgmap v83: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.0 MiB/s wr, 120 op/s 2024-09-13T18:25:08.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:08 smithi114 ceph-mon[30897]: pgmap v83: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.0 MiB/s wr, 120 op/s 2024-09-13T18:25:08.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:08 smithi134 ceph-mon[34283]: pgmap v83: 129 pgs: 129 active+clean; 10 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.0 MiB/s wr, 120 op/s 2024-09-13T18:25:10.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:10 smithi204 ceph-mon[34777]: pgmap v84: 129 pgs: 129 active+clean; 11 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 31 MiB/s wr, 529 op/s 2024-09-13T18:25:10.469 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:10 smithi114 ceph-mon[30897]: pgmap v84: 129 pgs: 129 active+clean; 11 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 31 MiB/s wr, 529 op/s 2024-09-13T18:25:10.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:10 smithi134 ceph-mon[34283]: pgmap v84: 129 pgs: 129 active+clean; 11 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 31 MiB/s wr, 529 op/s 2024-09-13T18:25:12.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:12 smithi204 ceph-mon[34777]: pgmap v85: 129 pgs: 129 active+clean; 11 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 31 MiB/s wr, 514 op/s 2024-09-13T18:25:12.469 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:12 smithi114 ceph-mon[30897]: pgmap v85: 129 pgs: 129 active+clean; 11 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 31 MiB/s wr, 514 op/s 2024-09-13T18:25:12.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:12 smithi134 ceph-mon[34283]: pgmap v85: 129 pgs: 129 active+clean; 11 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 31 MiB/s wr, 514 op/s 2024-09-13T18:25:14.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:14 smithi204 ceph-mon[34777]: pgmap v86: 129 pgs: 129 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 56 MiB/s wr, 930 op/s 2024-09-13T18:25:14.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:14 smithi114 ceph-mon[30897]: pgmap v86: 129 pgs: 129 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 56 MiB/s wr, 930 op/s 2024-09-13T18:25:14.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:14 smithi134 ceph-mon[34283]: pgmap v86: 129 pgs: 129 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 56 MiB/s wr, 930 op/s 2024-09-13T18:25:16.058 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.055+0000 7f2e24195640 1 -- 172.21.15.114:0/715311982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e1c0feff0 msgr2=0x7f2e1c0ff430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:16.058 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.055+0000 7f2e24195640 1 --2- 172.21.15.114:0/715311982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e1c0feff0 0x7f2e1c0ff430 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c009930 tx=0x7f2e0c02a420 comp rx=0 tx=0).stop 2024-09-13T18:25:16.058 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.055+0000 7f2e24195640 1 -- 172.21.15.114:0/715311982 shutdown_connections 2024-09-13T18:25:16.058 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.055+0000 7f2e24195640 1 --2- 172.21.15.114:0/715311982 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e1c0ff970 0x7f2e1c10ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:16.058 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.055+0000 7f2e24195640 1 --2- 172.21.15.114:0/715311982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e1c0feff0 0x7f2e1c0ff430 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:16.058 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.055+0000 7f2e24195640 1 --2- 172.21.15.114:0/715311982 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e1c104c30 0x7f2e1c105010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:16.058 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.055+0000 7f2e24195640 1 -- 172.21.15.114:0/715311982 >> 172.21.15.114:0/715311982 conn(0x7f2e1c074940 msgr2=0x7f2e1c074d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:16.058 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.055+0000 7f2e24195640 1 -- 172.21.15.114:0/715311982 shutdown_connections 2024-09-13T18:25:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.056+0000 7f2e24195640 1 -- 172.21.15.114:0/715311982 wait complete. 2024-09-13T18:25:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.057+0000 7f2e24195640 1 Processor -- start 2024-09-13T18:25:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.057+0000 7f2e24195640 1 -- start start 2024-09-13T18:25:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.057+0000 7f2e24195640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e1c0feff0 0x7f2e1c193fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.057+0000 7f2e24195640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e1c0ff970 0x7f2e1c1944f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.057+0000 7f2e24195640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e1c104c30 0x7f2e1c198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.057+0000 7f2e21f31640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e1c0feff0 0x7f2e1c193fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.057+0000 7f2e21730640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e1c0ff970 0x7f2e1c1944f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.057+0000 7f2e21f31640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e1c0feff0 0x7f2e1c193fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:43592/0 (socket says 172.21.15.114:43592) 2024-09-13T18:25:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.057+0000 7f2e21730640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e1c0ff970 0x7f2e1c1944f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:37960/0 (socket says 172.21.15.114:37960) 2024-09-13T18:25:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.057+0000 7f2e24195640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1c198e90 con 0x7f2e1c104c30 2024-09-13T18:25:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.057+0000 7f2e24195640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1c199000 con 0x7f2e1c0feff0 2024-09-13T18:25:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.057+0000 7f2e24195640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1c1991c0 con 0x7f2e1c0ff970 2024-09-13T18:25:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.057+0000 7f2e21f31640 1 -- 172.21.15.114:0/253117820 learned_addr learned my addr 172.21.15.114:0/253117820 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.058+0000 7f2e22732640 1 --2- 172.21.15.114:0/253117820 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e1c104c30 0x7f2e1c198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.059+0000 7f2e21f31640 1 -- 172.21.15.114:0/253117820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e1c0ff970 msgr2=0x7f2e1c1944f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.059+0000 7f2e21f31640 1 --2- 172.21.15.114:0/253117820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e1c0ff970 0x7f2e1c1944f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.059+0000 7f2e21f31640 1 -- 172.21.15.114:0/253117820 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e1c104c30 msgr2=0x7f2e1c198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.059+0000 7f2e21f31640 1 --2- 172.21.15.114:0/253117820 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e1c104c30 0x7f2e1c198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.059+0000 7f2e21f31640 1 -- 172.21.15.114:0/253117820 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e0c009560 con 0x7f2e1c0feff0 2024-09-13T18:25:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.059+0000 7f2e21f31640 1 --2- 172.21.15.114:0/253117820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e1c0feff0 0x7f2e1c193fb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2e18009490 tx=0x7f2e18009950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:16.063 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.060+0000 7f2e12ffd640 1 -- 172.21.15.114:0/253117820 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1801c520 con 0x7f2e1c0feff0 2024-09-13T18:25:16.063 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.060+0000 7f2e12ffd640 1 -- 172.21.15.114:0/253117820 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2e1801cb10 con 0x7f2e1c0feff0 2024-09-13T18:25:16.063 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.060+0000 7f2e12ffd640 1 -- 172.21.15.114:0/253117820 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1801ec00 con 0x7f2e1c0feff0 2024-09-13T18:25:16.063 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.060+0000 7f2e24195640 1 -- 172.21.15.114:0/253117820 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e1c1994a0 con 0x7f2e1c0feff0 2024-09-13T18:25:16.063 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.060+0000 7f2e24195640 1 -- 172.21.15.114:0/253117820 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e1c1a0d20 con 0x7f2e1c0feff0 2024-09-13T18:25:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.063+0000 7f2e26ebb740 1 Processor -- start 2024-09-13T18:25:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.064+0000 7f2e26ebb740 1 -- start start 2024-09-13T18:25:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.064+0000 7f2e26ebb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fe0925330 0x560fe0925710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.064+0000 7f2e26ebb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fe0925c50 0x560fe092ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.064+0000 7f2e26ebb740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fe09304f0 0x560fe09328e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.065+0000 7f2e26ebb740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fe0890c30 con 0x560fe09304f0 2024-09-13T18:25:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.065+0000 7f2e26ebb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fe088e3c0 con 0x560fe0925c50 2024-09-13T18:25:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.065+0000 7f2e26ebb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fe0889cb0 con 0x560fe0925330 2024-09-13T18:25:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.065+0000 7f2e12ffd640 1 -- 172.21.15.114:0/253117820 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f2e1801c680 con 0x7f2e1c0feff0 2024-09-13T18:25:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.065+0000 7f2e21f31640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fe0925c50 0x560fe092ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.065+0000 7f2e22732640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fe0925330 0x560fe0925710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.065+0000 7f2e22732640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fe0925330 0x560fe0925710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:37972/0 (socket says 172.21.15.114:37972) 2024-09-13T18:25:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.065+0000 7f2e22732640 1 -- 172.21.15.114:0/3979015726 learned_addr learned my addr 172.21.15.114:0/3979015726 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.065+0000 7f2e21730640 1 --2- 172.21.15.114:0/3979015726 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fe09304f0 0x560fe09328e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.065+0000 7f2e22732640 1 -- 172.21.15.114:0/3979015726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fe0925c50 msgr2=0x560fe092ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.065+0000 7f2e22732640 1 --2- 172.21.15.114:0/3979015726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fe0925c50 0x560fe092ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.065+0000 7f2e22732640 1 -- 172.21.15.114:0/3979015726 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fe09304f0 msgr2=0x560fe09328e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.065+0000 7f2e22732640 1 --2- 172.21.15.114:0/3979015726 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fe09304f0 0x560fe09328e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.065+0000 7f2e22732640 1 -- 172.21.15.114:0/3979015726 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560fe0830910 con 0x560fe0925330 2024-09-13T18:25:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.066+0000 7f2e21f31640 1 --2- 172.21.15.114:0/3979015726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fe0925c50 0x560fe092ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:25:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.066+0000 7f2e22732640 1 --2- 172.21.15.114:0/3979015726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fe0925330 0x560fe0925710 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2e14009ca0 tx=0x7f2e14011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2ea10395df88fc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.066+0000 7f2e07fff640 1 -- 172.21.15.114:0/3979015726 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e14019630 con 0x560fe0925330 2024-09-13T18:25:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.066+0000 7f2e07fff640 1 -- 172.21.15.114:0/3979015726 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2e14009e50 con 0x560fe0925330 2024-09-13T18:25:16.070 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.066+0000 7f2e07fff640 1 -- 172.21.15.114:0/3979015726 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1400ea60 con 0x560fe0925330 2024-09-13T18:25:16.070 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.067+0000 7f2e12ffd640 1 --2- 172.21.15.114:0/253117820 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2e080a1280 0x7f2e080a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:16.070 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.067+0000 7f2e26ebb740 1 -- 172.21.15.114:0/3979015726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fe0925330 msgr2=0x560fe0925710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:16.071 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.067+0000 7f2e26ebb740 1 --2- 172.21.15.114:0/3979015726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fe0925330 0x560fe0925710 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2e14009ca0 tx=0x7f2e14011810 comp rx=0 tx=0).stop 2024-09-13T18:25:16.071 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.067+0000 7f2e12ffd640 1 -- 172.21.15.114:0/253117820 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2e180d4ec0 con 0x7f2e1c0feff0 2024-09-13T18:25:16.071 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.067+0000 7f2e26ebb740 1 -- 172.21.15.114:0/3979015726 shutdown_connections 2024-09-13T18:25:16.071 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.067+0000 7f2e26ebb740 1 --2- 172.21.15.114:0/3979015726 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fe09304f0 0x560fe09328e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:16.071 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.067+0000 7f2e26ebb740 1 --2- 172.21.15.114:0/3979015726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fe0925c50 0x560fe092ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:16.071 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.067+0000 7f2e26ebb740 1 --2- 172.21.15.114:0/3979015726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fe0925330 0x560fe0925710 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:16.071 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.067+0000 7f2e26ebb740 1 -- 172.21.15.114:0/3979015726 >> 172.21.15.114:0/3979015726 conn(0x560fe09218c0 msgr2=0x560fe0921ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:16.071 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.068+0000 7f2e21f31640 1 --2- 172.21.15.114:0/253117820 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2e080a1280 0x7f2e080a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:16.071 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.068+0000 7f2e26ebb740 1 -- 172.21.15.114:0/3979015726 shutdown_connections 2024-09-13T18:25:16.071 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.068+0000 7f2e26ebb740 1 -- 172.21.15.114:0/3979015726 wait complete. 2024-09-13T18:25:16.071 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.068+0000 7f2e26ebb740 1 -- start start 2024-09-13T18:25:16.071 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.068+0000 7f2e26ebb740 1 Processor -- start 2024-09-13T18:25:16.073 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.068+0000 7f2e21f31640 1 --2- 172.21.15.114:0/253117820 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2e080a1280 0x7f2e080a3740 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2e180d2370 tx=0x7f2e180d8920 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:16.073 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.069+0000 7f2e26ebb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fe0925330 0x560fe092fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:16.073 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.069+0000 7f2e26ebb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fe0925c50 0x560fe092d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:16.073 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.069+0000 7f2e26ebb740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fe09304f0 0x560fe092d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:16.073 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.069+0000 7f2e26ebb740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fe0847e80 con 0x560fe09304f0 2024-09-13T18:25:16.073 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.069+0000 7f2e26ebb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fe0847b20 con 0x560fe0925c50 2024-09-13T18:25:16.073 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.069+0000 7f2e26ebb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fe080d260 con 0x560fe0925330 2024-09-13T18:25:16.073 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.069+0000 7f2e26ebb740 10 client.0 _ll_register_callbacks cb 0x560fe0842df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:25:16.073 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.069+0000 7f2e21730640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fe0925c50 0x560fe092d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:16.073 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.069+0000 7f2e21730640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fe0925c50 0x560fe092d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:43614/0 (socket says 172.21.15.114:43614) 2024-09-13T18:25:16.074 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.070+0000 7f2e21730640 1 -- 172.21.15.114:0/851516102 learned_addr learned my addr 172.21.15.114:0/851516102 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:16.074 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.070+0000 7f2e22732640 1 --2- 172.21.15.114:0/851516102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fe0925330 0x560fe092fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:16.074 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.070+0000 7f2e22732640 1 --2- 172.21.15.114:0/851516102 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fe09304f0 0x560fe092d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:16.074 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.070+0000 7f2e21730640 1 -- 172.21.15.114:0/851516102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fe0925330 msgr2=0x560fe092fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:16.074 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.070+0000 7f2e21730640 1 --2- 172.21.15.114:0/851516102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fe0925330 0x560fe092fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:16.074 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.070+0000 7f2e21730640 1 -- 172.21.15.114:0/851516102 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fe09304f0 msgr2=0x560fe092d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:16.074 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.070+0000 7f2e21730640 1 --2- 172.21.15.114:0/851516102 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fe09304f0 0x560fe092d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:16.074 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.070+0000 7f2e21730640 1 -- 172.21.15.114:0/851516102 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e14009950 con 0x560fe0925c50 2024-09-13T18:25:16.074 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.070+0000 7f2e21730640 1 --2- 172.21.15.114:0/851516102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fe0925c50 0x560fe092d160 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c009930 tx=0x7f2e0c0044f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da317654e1dd23df server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:16.074 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.070+0000 7f2e057fa640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:25:16.074 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.071+0000 7f2e057fa640 1 -- 172.21.15.114:0/851516102 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c003ac0 con 0x560fe0925c50 2024-09-13T18:25:16.074 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.071+0000 7f2e057fa640 1 -- 172.21.15.114:0/851516102 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2e0c003c20 con 0x560fe0925c50 2024-09-13T18:25:16.074 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.071+0000 7f2e26ebb740 1 -- 172.21.15.114:0/851516102 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560fe08b2680 con 0x560fe0925c50 2024-09-13T18:25:16.075 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.071+0000 7f2e057fa640 1 -- 172.21.15.114:0/851516102 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e0c003de0 con 0x560fe0925c50 2024-09-13T18:25:16.075 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.071+0000 7f2e26ebb740 10 client.25925 fetch_fsmap learned FSMap version 32 2024-09-13T18:25:16.075 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.071+0000 7f2e26ebb740 1 -- 172.21.15.114:0/851516102 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x560fe0888330 con 0x560fe0925c50 2024-09-13T18:25:16.076 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.071+0000 7f2e057fa640 1 -- 172.21.15.114:0/851516102 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560fe08b2680 con 0x560fe0925c50 2024-09-13T18:25:16.076 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.071+0000 7f2e057fa640 1 -- 172.21.15.114:0/851516102 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f2e0c03d5e0 con 0x560fe0925c50 2024-09-13T18:25:16.076 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.071+0000 7f2e26ebb740 10 client.25925 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:25:16.076 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.071+0000 7f2e26ebb740 10 client.25925 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:25:16.076 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.071+0000 7f2e26ebb740 20 client.25925 populate_metadata read hostname 'smithi114' 2024-09-13T18:25:16.076 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.071+0000 7f2e26ebb740 1 --2- 172.21.15.114:0/851516102 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x560fe093ffa0 0x560fe09423c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:16.076 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.072+0000 7f2e26ebb740 4 client.25925 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:25:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.072+0000 7f2e26ebb740 1 -- 172.21.15.114:0/851516102 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560fe087d5d0 con 0x560fe093ffa0 2024-09-13T18:25:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.072+0000 7f2e22732640 1 --2- 172.21.15.114:0/851516102 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x560fe093ffa0 0x560fe09423c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.073+0000 7f2e22732640 1 --2- 172.21.15.114:0/851516102 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x560fe093ffa0 0x560fe09423c0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9ca8e9cfb8aefdc server_cookie=6bbf5de8126093d4 in_seq=0 out_seq=0 2024-09-13T18:25:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.073+0000 7f2e057fa640 10 client.25925 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:25:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.073+0000 7f2e057fa640 1 -- 172.21.15.114:0/851516102 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x560fe087d5d0 con 0x560fe093ffa0 2024-09-13T18:25:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.073+0000 7f2e057fa640 10 client.25925 handle_command_reply: tid=0 2024-09-13T18:25:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.074+0000 7f2e26ebb740 1 client.25925 shutdown 2024-09-13T18:25:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.074+0000 7f2e26ebb740 1 -- 172.21.15.114:0/851516102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fe0925c50 msgr2=0x560fe092d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.074+0000 7f2e26ebb740 1 --2- 172.21.15.114:0/851516102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fe0925c50 0x560fe092d160 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c009930 tx=0x7f2e0c0044f0 comp rx=0 tx=0).stop 2024-09-13T18:25:16.078 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.075+0000 7f2e26ebb740 1 -- 172.21.15.114:0/851516102 shutdown_connections 2024-09-13T18:25:16.078 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.075+0000 7f2e26ebb740 1 --2- 172.21.15.114:0/851516102 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x560fe093ffa0 0x560fe09423c0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:16.078 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.075+0000 7f2e26ebb740 1 --2- 172.21.15.114:0/851516102 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fe09304f0 0x560fe092d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:16.078 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.075+0000 7f2e26ebb740 1 --2- 172.21.15.114:0/851516102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fe0925c50 0x560fe092d160 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:16.078 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.075+0000 7f2e26ebb740 1 --2- 172.21.15.114:0/851516102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fe0925330 0x560fe092fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:16.078 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.075+0000 7f2e26ebb740 1 -- 172.21.15.114:0/851516102 >> 172.21.15.114:0/851516102 conn(0x560fe09218c0 msgr2=0x560fe0920b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:16.078 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.076+0000 7f2e057fa640 0 client.25925 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:25:16.078 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.076+0000 7f2e26ebb740 1 -- 172.21.15.114:0/851516102 shutdown_connections 2024-09-13T18:25:16.079 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.076+0000 7f2e26ebb740 1 -- 172.21.15.114:0/851516102 wait complete. 2024-09-13T18:25:16.079 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.076+0000 7f2e26ebb740 20 client.25925 trim_cache size 0 max 16384 2024-09-13T18:25:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.101+0000 7f2e26ebb740 1 Processor -- start 2024-09-13T18:25:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.101+0000 7f2e26ebb740 1 -- start start 2024-09-13T18:25:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.101+0000 7f2e26ebb740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fe0925330 0x560fe0921300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.101+0000 7f2e26ebb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fe0925c50 0x560fe092f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.101+0000 7f2e26ebb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fe09304f0 0x560fe0937250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.101+0000 7f2e26ebb740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fe092da10 con 0x560fe0925330 2024-09-13T18:25:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.101+0000 7f2e26ebb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fe0932fa0 con 0x560fe09304f0 2024-09-13T18:25:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.101+0000 7f2e26ebb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fe091bf90 con 0x560fe0925c50 2024-09-13T18:25:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.101+0000 7f2e21730640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fe0925c50 0x560fe092f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.101+0000 7f2e21730640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fe0925c50 0x560fe092f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:37992/0 (socket says 172.21.15.114:37992) 2024-09-13T18:25:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.101+0000 7f2e21730640 1 -- 172.21.15.114:0/203491543 learned_addr learned my addr 172.21.15.114:0/203491543 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.101+0000 7f2e22732640 1 --2- 172.21.15.114:0/203491543 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fe0925330 0x560fe0921300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.101+0000 7f2e22732640 1 --2- 172.21.15.114:0/203491543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fe09304f0 0x560fe0937250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.102+0000 7f2e22732640 1 -- 172.21.15.114:0/203491543 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fe0925c50 msgr2=0x560fe092f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.102+0000 7f2e22732640 1 --2- 172.21.15.114:0/203491543 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fe0925c50 0x560fe092f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.102+0000 7f2e22732640 1 -- 172.21.15.114:0/203491543 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fe0925330 msgr2=0x560fe0921300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.102+0000 7f2e22732640 1 --2- 172.21.15.114:0/203491543 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fe0925330 0x560fe0921300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.102+0000 7f2e22732640 1 -- 172.21.15.114:0/203491543 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560fe082a300 con 0x560fe09304f0 2024-09-13T18:25:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.102+0000 7f2e22732640 1 --2- 172.21.15.114:0/203491543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fe09304f0 0x560fe0937250 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2e1401c040 tx=0x7f2e14000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=990c8b43a8e67631 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:16.106 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.103+0000 7f2e10ff9640 1 -- 172.21.15.114:0/203491543 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e14031070 con 0x560fe09304f0 2024-09-13T18:25:16.106 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.103+0000 7f2e10ff9640 1 -- 172.21.15.114:0/203491543 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2e14032040 con 0x560fe09304f0 2024-09-13T18:25:16.106 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.103+0000 7f2e10ff9640 1 -- 172.21.15.114:0/203491543 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e140038e0 con 0x560fe09304f0 2024-09-13T18:25:16.107 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.104+0000 7f2e26ebb740 1 -- 172.21.15.114:0/203491543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fe09304f0 msgr2=0x560fe0937250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:16.107 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.104+0000 7f2e26ebb740 1 --2- 172.21.15.114:0/203491543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fe09304f0 0x560fe0937250 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2e1401c040 tx=0x7f2e14000bc0 comp rx=0 tx=0).stop 2024-09-13T18:25:16.107 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.104+0000 7f2e26ebb740 1 -- 172.21.15.114:0/203491543 shutdown_connections 2024-09-13T18:25:16.107 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.104+0000 7f2e26ebb740 1 --2- 172.21.15.114:0/203491543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fe09304f0 0x560fe0937250 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:16.107 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.104+0000 7f2e26ebb740 1 --2- 172.21.15.114:0/203491543 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fe0925c50 0x560fe092f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:16.108 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.105+0000 7f2e26ebb740 1 --2- 172.21.15.114:0/203491543 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fe0925330 0x560fe0921300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:16.108 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.105+0000 7f2e26ebb740 1 -- 172.21.15.114:0/203491543 >> 172.21.15.114:0/203491543 conn(0x560fe09218c0 msgr2=0x560fe092ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:16.108 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.105+0000 7f2e26ebb740 1 -- 172.21.15.114:0/203491543 shutdown_connections 2024-09-13T18:25:16.108 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.105+0000 7f2e26ebb740 1 -- 172.21.15.114:0/203491543 wait complete. 2024-09-13T18:25:16.108 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.105+0000 7f2e26ebb740 1 -- start start 2024-09-13T18:25:16.108 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.105+0000 7f2e26ebb740 1 Processor -- start 2024-09-13T18:25:16.108 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.106+0000 7f2e26ebb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fe0925330 0x560fe09376a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:16.109 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.107+0000 7f2e26ebb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fe0925c50 0x560fe0933b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:16.109 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.107+0000 7f2e26ebb740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fe09304f0 0x560fe09340e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:16.109 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.107+0000 7f2e26ebb740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fe09347e0 con 0x560fe09304f0 2024-09-13T18:25:16.109 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.107+0000 7f2e26ebb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fe0934950 con 0x560fe0925330 2024-09-13T18:25:16.109 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.107+0000 7f2e26ebb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fe0996460 con 0x560fe0925c50 2024-09-13T18:25:16.109 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.107+0000 7f2e22732640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fe0925330 0x560fe09376a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:16.109 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.107+0000 7f2e26ebb740 10 client.0 _ll_register_callbacks cb 0x7f2e14009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:25:16.109 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.107+0000 7f2e22732640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fe0925330 0x560fe09376a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:43632/0 (socket says 172.21.15.114:43632) 2024-09-13T18:25:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.107+0000 7f2e22732640 1 -- 172.21.15.114:0/3009201865 learned_addr learned my addr 172.21.15.114:0/3009201865 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.107+0000 7f2e21730640 1 --2- 172.21.15.114:0/3009201865 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fe0925c50 0x560fe0933b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.107+0000 7f2e22732640 1 --2- 172.21.15.114:0/3009201865 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fe09304f0 0x560fe09340e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.108+0000 7f2e21730640 1 -- 172.21.15.114:0/3009201865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fe0925330 msgr2=0x560fe09376a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.108+0000 7f2e21730640 1 --2- 172.21.15.114:0/3009201865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fe0925330 0x560fe09376a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.108+0000 7f2e21730640 1 -- 172.21.15.114:0/3009201865 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fe09304f0 msgr2=0x560fe09340e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.108+0000 7f2e21730640 1 --2- 172.21.15.114:0/3009201865 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fe09304f0 0x560fe09340e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.108+0000 7f2e21730640 1 -- 172.21.15.114:0/3009201865 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560fe0830910 con 0x560fe0925c50 2024-09-13T18:25:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.108+0000 7f2e21730640 1 --2- 172.21.15.114:0/3009201865 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fe0925c50 0x560fe0933b40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c02a940 tx=0x7f2e0c004210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7524583d75e1183 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.108+0000 7f2e057fa640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:25:16.112 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.110+0000 7f2e057fa640 1 -- 172.21.15.114:0/3009201865 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c03d640 con 0x560fe0925c50 2024-09-13T18:25:16.112 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.110+0000 7f2e057fa640 1 -- 172.21.15.114:0/3009201865 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2e0c000e40 con 0x560fe0925c50 2024-09-13T18:25:16.112 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.110+0000 7f2e26ebb740 1 -- 172.21.15.114:0/3009201865 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2e14009e50 con 0x560fe0925c50 2024-09-13T18:25:16.112 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.110+0000 7f2e057fa640 1 -- 172.21.15.114:0/3009201865 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e0c03d7a0 con 0x560fe0925c50 2024-09-13T18:25:16.112 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.110+0000 7f2e26ebb740 10 client.25978 fetch_fsmap learned FSMap version 32 2024-09-13T18:25:16.113 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.110+0000 7f2e26ebb740 1 -- 172.21.15.114:0/3009201865 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x560fe084f8e0 con 0x560fe0925c50 2024-09-13T18:25:16.113 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.111+0000 7f2e057fa640 1 -- 172.21.15.114:0/3009201865 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2e0c03dc10 con 0x560fe0925c50 2024-09-13T18:25:16.113 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.111+0000 7f2e057fa640 1 -- 172.21.15.114:0/3009201865 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f2e0c05b070 con 0x560fe0925c50 2024-09-13T18:25:16.113 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.111+0000 7f2e26ebb740 10 client.25978 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:25:16.113 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.111+0000 7f2e26ebb740 10 client.25978 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:25:16.113 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.111+0000 7f2e26ebb740 20 client.25978 populate_metadata read hostname 'smithi114' 2024-09-13T18:25:16.113 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.111+0000 7f2e26ebb740 1 --2- 172.21.15.114:0/3009201865 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x560fe0997a10 0x560fe0999e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:16.113 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.111+0000 7f2e26ebb740 4 client.25978 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T18:25:16.113 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.111+0000 7f2e26ebb740 1 -- 172.21.15.114:0/3009201865 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7f2e14009e50 con 0x560fe0997a10 2024-09-13T18:25:16.114 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.112+0000 7f2e22732640 1 --2- 172.21.15.114:0/3009201865 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x560fe0997a10 0x560fe0999e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:16.115 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.113+0000 7f2e057fa640 10 client.25978 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:25:16.116 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.114+0000 7f2e22732640 1 --2- 172.21.15.114:0/3009201865 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x560fe0997a10 0x560fe0999e30 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f797a37b2a81444e server_cookie=c5bcd4e8c98d285 in_seq=0 out_seq=0 2024-09-13T18:25:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.122+0000 7f2e057fa640 1 -- 172.21.15.114:0/3009201865 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f2e14009e50 con 0x560fe0997a10 2024-09-13T18:25:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.122+0000 7f2e057fa640 10 client.25978 handle_command_reply: tid=0 2024-09-13T18:25:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.122+0000 7f2e26ebb740 1 client.25978 shutdown 2024-09-13T18:25:16.126 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.123+0000 7f2e26ebb740 1 -- 172.21.15.114:0/3009201865 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fe0925c50 msgr2=0x560fe0933b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:16.126 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.123+0000 7f2e26ebb740 1 --2- 172.21.15.114:0/3009201865 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fe0925c50 0x560fe0933b40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c02a940 tx=0x7f2e0c004210 comp rx=0 tx=0).stop 2024-09-13T18:25:16.126 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.123+0000 7f2e26ebb740 1 -- 172.21.15.114:0/3009201865 shutdown_connections 2024-09-13T18:25:16.126 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.123+0000 7f2e26ebb740 1 --2- 172.21.15.114:0/3009201865 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x560fe0997a10 0x560fe0999e30 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:16.126 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.123+0000 7f2e26ebb740 1 --2- 172.21.15.114:0/3009201865 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fe09304f0 0x560fe09340e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:16.126 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.124+0000 7f2e057fa640 0 client.25978 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:25:16.127 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.124+0000 7f2e26ebb740 1 --2- 172.21.15.114:0/3009201865 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fe0925c50 0x560fe0933b40 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:16.127 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.124+0000 7f2e26ebb740 1 --2- 172.21.15.114:0/3009201865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fe0925330 0x560fe09376a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:16.127 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.124+0000 7f2e26ebb740 1 -- 172.21.15.114:0/3009201865 >> 172.21.15.114:0/3009201865 conn(0x560fe09218c0 msgr2=0x560fe092c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:16.127 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.124+0000 7f2e26ebb740 1 -- 172.21.15.114:0/3009201865 shutdown_connections 2024-09-13T18:25:16.127 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.124+0000 7f2e26ebb740 1 -- 172.21.15.114:0/3009201865 wait complete. 2024-09-13T18:25:16.127 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-13T18:25:16.127 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-13T18:25:16.128 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-13T18:25:16.128 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-13T18:25:16.128 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.124+0000 7f2e26ebb740 20 client.25978 trim_cache size 0 max 16384 2024-09-13T18:25:16.128 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.125+0000 7f2e24195640 1 -- 172.21.15.114:0/253117820 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2e080a1280 msgr2=0x7f2e080a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:16.128 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.125+0000 7f2e24195640 1 --2- 172.21.15.114:0/253117820 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2e080a1280 0x7f2e080a3740 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2e180d2370 tx=0x7f2e180d8920 comp rx=0 tx=0).stop 2024-09-13T18:25:16.128 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.125+0000 7f2e24195640 1 -- 172.21.15.114:0/253117820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e1c0feff0 msgr2=0x7f2e1c193fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:16.128 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.125+0000 7f2e24195640 1 --2- 172.21.15.114:0/253117820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e1c0feff0 0x7f2e1c193fb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2e18009490 tx=0x7f2e18009950 comp rx=0 tx=0).stop 2024-09-13T18:25:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.126+0000 7f2e24195640 1 -- 172.21.15.114:0/253117820 shutdown_connections 2024-09-13T18:25:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.126+0000 7f2e24195640 1 --2- 172.21.15.114:0/253117820 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2e080a1280 0x7f2e080a3740 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.126+0000 7f2e24195640 1 --2- 172.21.15.114:0/253117820 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e1c104c30 0x7f2e1c198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.126+0000 7f2e24195640 1 --2- 172.21.15.114:0/253117820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e1c0ff970 0x7f2e1c1944f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.126+0000 7f2e24195640 1 --2- 172.21.15.114:0/253117820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e1c0feff0 0x7f2e1c193fb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.126+0000 7f2e24195640 1 -- 172.21.15.114:0/253117820 >> 172.21.15.114:0/253117820 conn(0x7f2e1c074940 msgr2=0x7f2e1c0fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.127+0000 7f2e24195640 1 -- 172.21.15.114:0/253117820 shutdown_connections 2024-09-13T18:25:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:16.127+0000 7f2e24195640 1 -- 172.21.15.114:0/253117820 wait complete. 2024-09-13T18:25:16.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:16 smithi204 ceph-mon[34777]: pgmap v87: 129 pgs: 129 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 51 MiB/s wr, 824 op/s 2024-09-13T18:25:16.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:16 smithi114 ceph-mon[30897]: pgmap v87: 129 pgs: 129 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 51 MiB/s wr, 824 op/s 2024-09-13T18:25:16.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:16 smithi134 ceph-mon[34283]: pgmap v87: 129 pgs: 129 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 51 MiB/s wr, 824 op/s 2024-09-13T18:25:18.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:18 smithi204 ceph-mon[34777]: pgmap v88: 129 pgs: 129 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 51 MiB/s wr, 824 op/s 2024-09-13T18:25:18.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:18 smithi114 ceph-mon[30897]: pgmap v88: 129 pgs: 129 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 51 MiB/s wr, 824 op/s 2024-09-13T18:25:18.522 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T18:25:18.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:18 smithi134 ceph-mon[34283]: pgmap v88: 129 pgs: 129 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 51 MiB/s wr, 824 op/s 2024-09-13T18:25:18.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.706+0000 7fa3fa7fc640 1 -- 172.21.15.114:0/981204041 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e4031730 con 0x7fa3fc0fee60 2024-09-13T18:25:18.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.706+0000 7fa402087640 1 -- 172.21.15.114:0/981204041 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3fc0fee60 msgr2=0x7fa3fc0ff240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:18.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.706+0000 7fa402087640 1 --2- 172.21.15.114:0/981204041 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3fc0fee60 0x7fa3fc0ff240 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4009900 tx=0x7fa3e402a410 comp rx=0 tx=0).stop 2024-09-13T18:25:18.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.707+0000 7fa402087640 1 -- 172.21.15.114:0/981204041 shutdown_connections 2024-09-13T18:25:18.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.707+0000 7fa402087640 1 --2- 172.21.15.114:0/981204041 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3fc103fb0 0x7fa3fc10bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:18.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.707+0000 7fa402087640 1 --2- 172.21.15.114:0/981204041 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3fc0ff810 0x7fa3fc103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:18.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.707+0000 7fa402087640 1 --2- 172.21.15.114:0/981204041 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3fc0fee60 0x7fa3fc0ff240 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:18.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.707+0000 7fa402087640 1 -- 172.21.15.114:0/981204041 >> 172.21.15.114:0/981204041 conn(0x7fa3fc075c60 msgr2=0x7fa3fc0fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:18.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.709+0000 7fa402087640 1 -- 172.21.15.114:0/981204041 shutdown_connections 2024-09-13T18:25:18.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.712+0000 7fa402087640 1 -- 172.21.15.114:0/981204041 wait complete. 2024-09-13T18:25:18.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.713+0000 7fa402087640 1 Processor -- start 2024-09-13T18:25:18.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.714+0000 7fa402087640 1 -- start start 2024-09-13T18:25:18.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.714+0000 7fa402087640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3fc0fee60 0x7fa3fc10c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:18.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.714+0000 7fa402087640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3fc103fb0 0x7fa3fc10c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:18.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.714+0000 7fa402087640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3fc10cac0 0x7fa3fc070cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:18.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.714+0000 7fa402087640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3fc0712b0 con 0x7fa3fc103fb0 2024-09-13T18:25:18.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.714+0000 7fa402087640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3fc0713f0 con 0x7fa3fc10cac0 2024-09-13T18:25:18.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.714+0000 7fa402087640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3fc0715b0 con 0x7fa3fc0fee60 2024-09-13T18:25:18.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.715+0000 7fa3fbfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3fc10cac0 0x7fa3fc070cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:18.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.715+0000 7fa3fbfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3fc10cac0 0x7fa3fc070cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:43648/0 (socket says 172.21.15.114:43648) 2024-09-13T18:25:18.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.715+0000 7fa3fbfff640 1 -- 172.21.15.114:0/2867433428 learned_addr learned my addr 172.21.15.114:0/2867433428 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:18.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.715+0000 7fa3faffd640 1 --2- 172.21.15.114:0/2867433428 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3fc103fb0 0x7fa3fc10c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:18.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.716+0000 7fa3fb7fe640 1 --2- 172.21.15.114:0/2867433428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3fc0fee60 0x7fa3fc10c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:18.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.716+0000 7fa3faffd640 1 -- 172.21.15.114:0/2867433428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3fc0fee60 msgr2=0x7fa3fc10c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:18.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.716+0000 7fa3faffd640 1 --2- 172.21.15.114:0/2867433428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3fc0fee60 0x7fa3fc10c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:18.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.716+0000 7fa3faffd640 1 -- 172.21.15.114:0/2867433428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3fc10cac0 msgr2=0x7fa3fc070cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:18.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.716+0000 7fa3faffd640 1 --2- 172.21.15.114:0/2867433428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3fc10cac0 0x7fa3fc070cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:18.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.716+0000 7fa3faffd640 1 -- 172.21.15.114:0/2867433428 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e4009560 con 0x7fa3fc103fb0 2024-09-13T18:25:18.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.717+0000 7fa3faffd640 1 --2- 172.21.15.114:0/2867433428 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3fc103fb0 0x7fa3fc10c580 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec00e9c0 tx=0x7fa3ec00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:18.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.717+0000 7fa3fbfff640 1 --2- 172.21.15.114:0/2867433428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3fc10cac0 0x7fa3fc070cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:25:18.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.718+0000 7fa3f8ff9640 1 -- 172.21.15.114:0/2867433428 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec00ccc0 con 0x7fa3fc103fb0 2024-09-13T18:25:18.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.718+0000 7fa3f8ff9640 1 -- 172.21.15.114:0/2867433428 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa3ec00ce20 con 0x7fa3fc103fb0 2024-09-13T18:25:18.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.718+0000 7fa3f8ff9640 1 -- 172.21.15.114:0/2867433428 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec018680 con 0x7fa3fc103fb0 2024-09-13T18:25:18.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.718+0000 7fa402087640 1 -- 172.21.15.114:0/2867433428 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3fc071890 con 0x7fa3fc103fb0 2024-09-13T18:25:18.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.718+0000 7fa402087640 1 -- 172.21.15.114:0/2867433428 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3fc071d50 con 0x7fa3fc103fb0 2024-09-13T18:25:18.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.722+0000 7fa404e86740 1 Processor -- start 2024-09-13T18:25:18.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.722+0000 7fa404e86740 1 -- start start 2024-09-13T18:25:18.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.722+0000 7fa404e86740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650e2ef2e10 0x5650e2ef31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:18.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.722+0000 7fa404e86740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650e2ef3730 0x5650e2efda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:18.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.722+0000 7fa404e86740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650e2efdfd0 0x5650e2f003c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:18.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.722+0000 7fa404e86740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650e2e5df20 con 0x5650e2ef2e10 2024-09-13T18:25:18.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.722+0000 7fa404e86740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650e2e5b6b0 con 0x5650e2ef3730 2024-09-13T18:25:18.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.722+0000 7fa404e86740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650e2e56fa0 con 0x5650e2efdfd0 2024-09-13T18:25:18.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.723+0000 7fa3fb7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650e2ef3730 0x5650e2efda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:18.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.723+0000 7fa3fb7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650e2ef3730 0x5650e2efda90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:43658/0 (socket says 172.21.15.114:43658) 2024-09-13T18:25:18.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.723+0000 7fa3fb7fe640 1 -- 172.21.15.114:0/1374414255 learned_addr learned my addr 172.21.15.114:0/1374414255 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:18.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.723+0000 7fa3fbfff640 1 --2- 172.21.15.114:0/1374414255 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650e2ef2e10 0x5650e2ef31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:18.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.723+0000 7fa3fb7fe640 1 -- 172.21.15.114:0/1374414255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650e2efdfd0 msgr2=0x5650e2f003c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:25:18.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.723+0000 7fa3fb7fe640 1 --2- 172.21.15.114:0/1374414255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650e2efdfd0 0x5650e2f003c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:18.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.723+0000 7fa3fb7fe640 1 -- 172.21.15.114:0/1374414255 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650e2ef2e10 msgr2=0x5650e2ef31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:18.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.723+0000 7fa3fb7fe640 1 --2- 172.21.15.114:0/1374414255 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650e2ef2e10 0x5650e2ef31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:18.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.723+0000 7fa3fb7fe640 1 -- 172.21.15.114:0/1374414255 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650e2e11840 con 0x5650e2ef3730 2024-09-13T18:25:18.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.724+0000 7fa3fb7fe640 1 --2- 172.21.15.114:0/1374414255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650e2ef3730 0x5650e2efda90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4009230 tx=0x7fa3e4004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e372c44179a4a5c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:18.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.724+0000 7fa3f1ffb640 1 -- 172.21.15.114:0/1374414255 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e40043b0 con 0x5650e2ef3730 2024-09-13T18:25:18.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.724+0000 7fa3f1ffb640 1 -- 172.21.15.114:0/1374414255 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa3e4004510 con 0x5650e2ef3730 2024-09-13T18:25:18.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.724+0000 7fa3f1ffb640 1 -- 172.21.15.114:0/1374414255 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e40459b0 con 0x5650e2ef3730 2024-09-13T18:25:18.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.725+0000 7fa3f8ff9640 1 -- 172.21.15.114:0/2867433428 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fa3ec010430 con 0x7fa3fc103fb0 2024-09-13T18:25:18.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.725+0000 7fa404e86740 1 -- 172.21.15.114:0/1374414255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650e2ef3730 msgr2=0x5650e2efda90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:18.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.725+0000 7fa404e86740 1 --2- 172.21.15.114:0/1374414255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650e2ef3730 0x5650e2efda90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4009230 tx=0x7fa3e4004040 comp rx=0 tx=0).stop 2024-09-13T18:25:18.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.725+0000 7fa404e86740 1 -- 172.21.15.114:0/1374414255 shutdown_connections 2024-09-13T18:25:18.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.725+0000 7fa404e86740 1 --2- 172.21.15.114:0/1374414255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650e2efdfd0 0x5650e2f003c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:18.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.725+0000 7fa404e86740 1 --2- 172.21.15.114:0/1374414255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650e2ef3730 0x5650e2efda90 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:18.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.725+0000 7fa404e86740 1 --2- 172.21.15.114:0/1374414255 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650e2ef2e10 0x5650e2ef31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:18.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.725+0000 7fa404e86740 1 -- 172.21.15.114:0/1374414255 >> 172.21.15.114:0/1374414255 conn(0x5650e2eef380 msgr2=0x5650e2eef760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:18.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.726+0000 7fa404e86740 1 -- 172.21.15.114:0/1374414255 shutdown_connections 2024-09-13T18:25:18.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.726+0000 7fa404e86740 1 -- 172.21.15.114:0/1374414255 wait complete. 2024-09-13T18:25:18.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.726+0000 7fa3f8ff9640 1 --2- 172.21.15.114:0/2867433428 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa3e80a1320 0x7fa3e80a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:18.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.726+0000 7fa3f8ff9640 1 -- 172.21.15.114:0/2867433428 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fa3ec014070 con 0x7fa3fc103fb0 2024-09-13T18:25:18.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.726+0000 7fa3fb7fe640 1 --2- 172.21.15.114:0/2867433428 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa3e80a1320 0x7fa3e80a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:18.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.726+0000 7fa404e86740 1 -- start start 2024-09-13T18:25:18.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.727+0000 7fa404e86740 1 Processor -- start 2024-09-13T18:25:18.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.727+0000 7fa3fb7fe640 1 --2- 172.21.15.114:0/2867433428 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa3e80a1320 0x7fa3e80a37e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4045690 tx=0x7fa3e4005fd0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:18.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.727+0000 7fa404e86740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650e2ef2e10 0x5650e2efd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:18.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.727+0000 7fa404e86740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650e2ef3730 0x5650e2eface0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:18.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.727+0000 7fa404e86740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650e2efdfd0 0x5650e2efb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:18.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.727+0000 7fa404e86740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650e2df82a0 con 0x5650e2efdfd0 2024-09-13T18:25:18.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.727+0000 7fa404e86740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650e2b2f430 con 0x5650e2ef3730 2024-09-13T18:25:18.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.727+0000 7fa404e86740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650e2dda680 con 0x5650e2ef2e10 2024-09-13T18:25:18.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.727+0000 7fa404e86740 10 client.0 _ll_register_callbacks cb 0x5650e2e50610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:25:18.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.727+0000 7fa3faffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650e2ef3730 0x5650e2eface0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:18.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.727+0000 7fa3fb7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650e2ef2e10 0x5650e2efd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:18.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.727+0000 7fa3fb7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650e2ef2e10 0x5650e2efd9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:38010/0 (socket says 172.21.15.114:38010) 2024-09-13T18:25:18.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.727+0000 7fa3fb7fe640 1 -- 172.21.15.114:0/3007007646 learned_addr learned my addr 172.21.15.114:0/3007007646 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:18.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.728+0000 7fa3faffd640 1 -- 172.21.15.114:0/3007007646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650e2ef2e10 msgr2=0x5650e2efd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:18.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.728+0000 7fa3faffd640 1 --2- 172.21.15.114:0/3007007646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650e2ef2e10 0x5650e2efd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:18.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.728+0000 7fa3faffd640 1 -- 172.21.15.114:0/3007007646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650e2efdfd0 msgr2=0x5650e2efb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:18.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.728+0000 7fa3faffd640 1 --2- 172.21.15.114:0/3007007646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650e2efdfd0 0x5650e2efb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:18.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.728+0000 7fa3faffd640 1 -- 172.21.15.114:0/3007007646 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3fc067920 con 0x5650e2ef3730 2024-09-13T18:25:18.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.728+0000 7fa3faffd640 1 --2- 172.21.15.114:0/3007007646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650e2ef3730 0x5650e2eface0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec0ca8f0 tx=0x7fa3ec099cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f37922d15dd910f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:18.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.728+0000 7fa3daffd640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:25:18.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.728+0000 7fa3daffd640 1 -- 172.21.15.114:0/3007007646 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec0cd340 con 0x5650e2ef3730 2024-09-13T18:25:18.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.728+0000 7fa404e86740 1 -- 172.21.15.114:0/3007007646 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5650e2b4a4e0 con 0x5650e2ef3730 2024-09-13T18:25:18.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.728+0000 7fa3daffd640 1 -- 172.21.15.114:0/3007007646 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fa3ec0a2730 con 0x5650e2ef3730 2024-09-13T18:25:18.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.729+0000 7fa3daffd640 1 -- 172.21.15.114:0/3007007646 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3ec0cad80 con 0x5650e2ef3730 2024-09-13T18:25:18.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.729+0000 7fa404e86740 10 client.25943 fetch_fsmap learned FSMap version 32 2024-09-13T18:25:18.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.729+0000 7fa404e86740 1 -- 172.21.15.114:0/3007007646 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5650e2e55620 con 0x5650e2ef3730 2024-09-13T18:25:18.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.729+0000 7fa3daffd640 1 -- 172.21.15.114:0/3007007646 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5650e2b4a4e0 con 0x5650e2ef3730 2024-09-13T18:25:18.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.729+0000 7fa3daffd640 1 -- 172.21.15.114:0/3007007646 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fa3ec099210 con 0x5650e2ef3730 2024-09-13T18:25:18.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.729+0000 7fa404e86740 10 client.25943 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:25:18.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.729+0000 7fa404e86740 10 client.25943 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:25:18.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.729+0000 7fa404e86740 20 client.25943 populate_metadata read hostname 'smithi114' 2024-09-13T18:25:18.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.729+0000 7fa404e86740 1 --2- 172.21.15.114:0/3007007646 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5650e2f0def0 0x5650e2f10310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:18.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.729+0000 7fa404e86740 4 client.25943 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:25:18.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.729+0000 7fa404e86740 1 -- 172.21.15.114:0/3007007646 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5650e2b5b7e0 con 0x5650e2f0def0 2024-09-13T18:25:18.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.730+0000 7fa3fb7fe640 1 --2- 172.21.15.114:0/3007007646 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5650e2f0def0 0x5650e2f10310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:18.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.730+0000 7fa3fb7fe640 1 --2- 172.21.15.114:0/3007007646 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5650e2f0def0 0x5650e2f10310 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7689e327d31ae366 server_cookie=1491b1e18024266f in_seq=0 out_seq=0 2024-09-13T18:25:18.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.730+0000 7fa3daffd640 10 client.25943 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:25:18.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.731+0000 7fa3daffd640 1 -- 172.21.15.114:0/3007007646 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5650e2b5b7e0 con 0x5650e2f0def0 2024-09-13T18:25:18.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.731+0000 7fa3daffd640 10 client.25943 handle_command_reply: tid=0 2024-09-13T18:25:18.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.731+0000 7fa404e86740 1 client.25943 shutdown 2024-09-13T18:25:18.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.731+0000 7fa404e86740 1 -- 172.21.15.114:0/3007007646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650e2ef3730 msgr2=0x5650e2eface0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:18.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.731+0000 7fa404e86740 1 --2- 172.21.15.114:0/3007007646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650e2ef3730 0x5650e2eface0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec0ca8f0 tx=0x7fa3ec099cd0 comp rx=0 tx=0).stop 2024-09-13T18:25:18.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.731+0000 7fa404e86740 1 -- 172.21.15.114:0/3007007646 shutdown_connections 2024-09-13T18:25:18.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.731+0000 7fa404e86740 1 --2- 172.21.15.114:0/3007007646 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5650e2f0def0 0x5650e2f10310 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:18.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.732+0000 7fa404e86740 1 --2- 172.21.15.114:0/3007007646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650e2efdfd0 0x5650e2efb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:18.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.732+0000 7fa404e86740 1 --2- 172.21.15.114:0/3007007646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650e2ef3730 0x5650e2eface0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:18.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.732+0000 7fa404e86740 1 --2- 172.21.15.114:0/3007007646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650e2ef2e10 0x5650e2efd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:18.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.732+0000 7fa3daffd640 0 client.25943 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:25:18.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.732+0000 7fa404e86740 1 -- 172.21.15.114:0/3007007646 >> 172.21.15.114:0/3007007646 conn(0x5650e2eef380 msgr2=0x5650e2ef50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:18.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.732+0000 7fa404e86740 1 -- 172.21.15.114:0/3007007646 shutdown_connections 2024-09-13T18:25:18.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.732+0000 7fa404e86740 1 -- 172.21.15.114:0/3007007646 wait complete. 2024-09-13T18:25:18.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.733+0000 7fa404e86740 20 client.25943 trim_cache size 0 max 16384 2024-09-13T18:25:18.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.762+0000 7fa404e86740 1 Processor -- start 2024-09-13T18:25:18.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.762+0000 7fa404e86740 1 -- start start 2024-09-13T18:25:18.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.762+0000 7fa404e86740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650e2ef2e10 0x5650e2eef760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:18.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.762+0000 7fa404e86740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650e2ef3730 0x5650e2eefca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:18.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.762+0000 7fa404e86740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650e2efdfd0 0x5650e2ef5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:18.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.762+0000 7fa404e86740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650e2efab40 con 0x5650e2ef3730 2024-09-13T18:25:18.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.762+0000 7fa404e86740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650e2ef04f0 con 0x5650e2ef2e10 2024-09-13T18:25:18.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.762+0000 7fa404e86740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650e2f10970 con 0x5650e2efdfd0 2024-09-13T18:25:18.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.763+0000 7fa3faffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650e2ef3730 0x5650e2eefca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:18.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.763+0000 7fa3faffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650e2ef3730 0x5650e2eefca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43786/0 (socket says 172.21.15.114:43786) 2024-09-13T18:25:18.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.763+0000 7fa3faffd640 1 -- 172.21.15.114:0/4052516126 learned_addr learned my addr 172.21.15.114:0/4052516126 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:18.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.763+0000 7fa3fb7fe640 1 --2- 172.21.15.114:0/4052516126 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650e2ef2e10 0x5650e2eef760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:18.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.764+0000 7fa3faffd640 1 -- 172.21.15.114:0/4052516126 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650e2efdfd0 msgr2=0x5650e2ef5fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:25:18.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.764+0000 7fa3faffd640 1 --2- 172.21.15.114:0/4052516126 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650e2efdfd0 0x5650e2ef5fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:18.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.764+0000 7fa3faffd640 1 -- 172.21.15.114:0/4052516126 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650e2ef2e10 msgr2=0x5650e2eef760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:18.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.764+0000 7fa3faffd640 1 --2- 172.21.15.114:0/4052516126 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650e2ef2e10 0x5650e2eef760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:18.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.764+0000 7fa3faffd640 1 -- 172.21.15.114:0/4052516126 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650e2df7790 con 0x5650e2ef3730 2024-09-13T18:25:18.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.764+0000 7fa3faffd640 1 --2- 172.21.15.114:0/4052516126 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650e2ef3730 0x5650e2eefca0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec096d20 tx=0x7fa3ec099d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28a7d76aa3c07149 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:18.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.764+0000 7fa3f27fc640 1 -- 172.21.15.114:0/4052516126 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec0cad80 con 0x5650e2ef3730 2024-09-13T18:25:18.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.765+0000 7fa3f27fc640 1 -- 172.21.15.114:0/4052516126 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa3ec0a2730 con 0x5650e2ef3730 2024-09-13T18:25:18.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.765+0000 7fa3f27fc640 1 -- 172.21.15.114:0/4052516126 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec0a2500 con 0x5650e2ef3730 2024-09-13T18:25:18.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.766+0000 7fa404e86740 1 -- 172.21.15.114:0/4052516126 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650e2ef3730 msgr2=0x5650e2eefca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:18.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.766+0000 7fa404e86740 1 --2- 172.21.15.114:0/4052516126 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650e2ef3730 0x5650e2eefca0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec096d20 tx=0x7fa3ec099d40 comp rx=0 tx=0).stop 2024-09-13T18:25:18.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.766+0000 7fa404e86740 1 -- 172.21.15.114:0/4052516126 shutdown_connections 2024-09-13T18:25:18.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.766+0000 7fa404e86740 1 --2- 172.21.15.114:0/4052516126 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650e2efdfd0 0x5650e2ef5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:18.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.766+0000 7fa404e86740 1 --2- 172.21.15.114:0/4052516126 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650e2ef3730 0x5650e2eefca0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:18.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.766+0000 7fa404e86740 1 --2- 172.21.15.114:0/4052516126 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650e2ef2e10 0x5650e2eef760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:18.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.766+0000 7fa404e86740 1 -- 172.21.15.114:0/4052516126 >> 172.21.15.114:0/4052516126 conn(0x5650e2eef380 msgr2=0x5650e2f14c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:18.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.766+0000 7fa404e86740 1 -- 172.21.15.114:0/4052516126 shutdown_connections 2024-09-13T18:25:18.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.766+0000 7fa404e86740 1 -- 172.21.15.114:0/4052516126 wait complete. 2024-09-13T18:25:18.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.767+0000 7fa404e86740 1 -- start start 2024-09-13T18:25:18.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.768+0000 7fa404e86740 1 Processor -- start 2024-09-13T18:25:18.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.768+0000 7fa404e86740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650e2ef2e10 0x5650e2f05250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:18.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.768+0000 7fa404e86740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650e2ef3730 0x5650e2ef8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:18.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.768+0000 7fa404e86740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650e2efdfd0 0x5650e2ef88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:18.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.768+0000 7fa404e86740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650e2ef6f50 con 0x5650e2efdfd0 2024-09-13T18:25:18.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.768+0000 7fa404e86740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650e2ef70c0 con 0x5650e2ef3730 2024-09-13T18:25:18.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.768+0000 7fa404e86740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650e2ef7280 con 0x5650e2ef2e10 2024-09-13T18:25:18.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.768+0000 7fa404e86740 10 client.0 _ll_register_callbacks cb 0x7fa3ec010c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:25:18.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.769+0000 7fa3faffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650e2ef3730 0x5650e2ef8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:18.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.769+0000 7fa3faffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650e2ef3730 0x5650e2ef8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:43688/0 (socket says 172.21.15.114:43688) 2024-09-13T18:25:18.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.769+0000 7fa3faffd640 1 -- 172.21.15.114:0/90098314 learned_addr learned my addr 172.21.15.114:0/90098314 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:18.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.769+0000 7fa3fbfff640 1 --2- 172.21.15.114:0/90098314 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650e2efdfd0 0x5650e2ef88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:18.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.769+0000 7fa3fb7fe640 1 --2- 172.21.15.114:0/90098314 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650e2ef2e10 0x5650e2f05250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:18.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.769+0000 7fa3fbfff640 1 -- 172.21.15.114:0/90098314 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650e2ef2e10 msgr2=0x5650e2f05250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:18.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.769+0000 7fa3fbfff640 1 --2- 172.21.15.114:0/90098314 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650e2ef2e10 0x5650e2f05250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:18.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.769+0000 7fa3fbfff640 1 -- 172.21.15.114:0/90098314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650e2ef3730 msgr2=0x5650e2ef8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:18.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.769+0000 7fa3fbfff640 1 --2- 172.21.15.114:0/90098314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650e2ef3730 0x5650e2ef8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:18.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.769+0000 7fa3fbfff640 1 -- 172.21.15.114:0/90098314 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5650e2e55620 con 0x5650e2efdfd0 2024-09-13T18:25:18.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.769+0000 7fa3fbfff640 1 --2- 172.21.15.114:0/90098314 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650e2efdfd0 0x5650e2ef88b0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x5650e2ef0220 tx=0x7fa3f4013be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4a777b72aaeaa84 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:18.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.770+0000 7fa3daffd640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:25:18.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.770+0000 7fa3daffd640 1 -- 172.21.15.114:0/90098314 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f4011d20 con 0x5650e2efdfd0 2024-09-13T18:25:18.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.770+0000 7fa3daffd640 1 -- 172.21.15.114:0/90098314 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fa3f4011e80 con 0x5650e2efdfd0 2024-09-13T18:25:18.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.770+0000 7fa404e86740 1 -- 172.21.15.114:0/90098314 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa3e4004510 con 0x5650e2efdfd0 2024-09-13T18:25:18.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.770+0000 7fa3daffd640 1 -- 172.21.15.114:0/90098314 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3f401c440 con 0x5650e2efdfd0 2024-09-13T18:25:18.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.771+0000 7fa404e86740 10 client.16626 fetch_fsmap learned FSMap version 32 2024-09-13T18:25:18.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.771+0000 7fa404e86740 1 -- 172.21.15.114:0/90098314 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5650e2e1cea0 con 0x5650e2efdfd0 2024-09-13T18:25:18.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.772+0000 7fa3daffd640 1 -- 172.21.15.114:0/90098314 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa3e4004510 con 0x5650e2efdfd0 2024-09-13T18:25:18.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.772+0000 7fa3daffd640 1 -- 172.21.15.114:0/90098314 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fa3f4026840 con 0x5650e2efdfd0 2024-09-13T18:25:18.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.773+0000 7fa404e86740 10 client.16626 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:25:18.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.773+0000 7fa404e86740 10 client.16626 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:25:18.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.773+0000 7fa404e86740 20 client.16626 populate_metadata read hostname 'smithi114' 2024-09-13T18:25:18.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.773+0000 7fa404e86740 1 --2- 172.21.15.114:0/90098314 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5650e2f64ed0 0x5650e2f672f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:18.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.773+0000 7fa404e86740 4 client.16626 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T18:25:18.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.773+0000 7fa404e86740 1 -- 172.21.15.114:0/90098314 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5650e2baee30 con 0x5650e2f64ed0 2024-09-13T18:25:18.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.773+0000 7fa3fb7fe640 1 --2- 172.21.15.114:0/90098314 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5650e2f64ed0 0x5650e2f672f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:18.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.774+0000 7fa3fb7fe640 1 --2- 172.21.15.114:0/90098314 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5650e2f64ed0 0x5650e2f672f0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa89bc9ce75b81b server_cookie=31fb2baa79f3955a in_seq=0 out_seq=0 2024-09-13T18:25:18.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.774+0000 7fa3daffd640 10 client.16626 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:25:18.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.774+0000 7fa3daffd640 1 -- 172.21.15.114:0/90098314 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5650e2baee30 con 0x5650e2f64ed0 2024-09-13T18:25:18.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.774+0000 7fa3daffd640 10 client.16626 handle_command_reply: tid=0 2024-09-13T18:25:18.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.774+0000 7fa404e86740 1 client.16626 shutdown 2024-09-13T18:25:18.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.774+0000 7fa404e86740 1 -- 172.21.15.114:0/90098314 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650e2efdfd0 msgr2=0x5650e2ef88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:18.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.774+0000 7fa404e86740 1 --2- 172.21.15.114:0/90098314 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650e2efdfd0 0x5650e2ef88b0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x5650e2ef0220 tx=0x7fa3f4013be0 comp rx=0 tx=0).stop 2024-09-13T18:25:18.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.776+0000 7fa404e86740 1 -- 172.21.15.114:0/90098314 shutdown_connections 2024-09-13T18:25:18.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.776+0000 7fa404e86740 1 --2- 172.21.15.114:0/90098314 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5650e2f64ed0 0x5650e2f672f0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:18.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.776+0000 7fa404e86740 1 --2- 172.21.15.114:0/90098314 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650e2efdfd0 0x5650e2ef88b0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:18.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.776+0000 7fa3daffd640 0 client.16626 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:25:18.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.776+0000 7fa404e86740 1 --2- 172.21.15.114:0/90098314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5650e2ef3730 0x5650e2ef8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:18.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.776+0000 7fa404e86740 1 --2- 172.21.15.114:0/90098314 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650e2ef2e10 0x5650e2f05250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:18.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.776+0000 7fa404e86740 1 -- 172.21.15.114:0/90098314 >> 172.21.15.114:0/90098314 conn(0x5650e2eef380 msgr2=0x5650e2eef760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:18.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.776+0000 7fa404e86740 1 -- 172.21.15.114:0/90098314 shutdown_connections 2024-09-13T18:25:18.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.776+0000 7fa404e86740 1 -- 172.21.15.114:0/90098314 wait complete. 2024-09-13T18:25:18.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.777+0000 7fa404e86740 20 client.16626 trim_cache size 0 max 16384 2024-09-13T18:25:18.780 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:25:18.780 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "no active scrubs running", 2024-09-13T18:25:18.780 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": {} 2024-09-13T18:25:18.780 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:25:18.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.778+0000 7fa402087640 1 -- 172.21.15.114:0/2867433428 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa3e80a1320 msgr2=0x7fa3e80a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:18.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.778+0000 7fa402087640 1 --2- 172.21.15.114:0/2867433428 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa3e80a1320 0x7fa3e80a37e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4045690 tx=0x7fa3e4005fd0 comp rx=0 tx=0).stop 2024-09-13T18:25:18.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.778+0000 7fa402087640 1 -- 172.21.15.114:0/2867433428 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3fc103fb0 msgr2=0x7fa3fc10c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:18.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.778+0000 7fa402087640 1 --2- 172.21.15.114:0/2867433428 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3fc103fb0 0x7fa3fc10c580 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec00e9c0 tx=0x7fa3ec00ee80 comp rx=0 tx=0).stop 2024-09-13T18:25:18.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.778+0000 7fa402087640 1 -- 172.21.15.114:0/2867433428 shutdown_connections 2024-09-13T18:25:18.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.778+0000 7fa402087640 1 --2- 172.21.15.114:0/2867433428 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa3e80a1320 0x7fa3e80a37e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:18.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.778+0000 7fa402087640 1 --2- 172.21.15.114:0/2867433428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3fc10cac0 0x7fa3fc070cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:18.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.778+0000 7fa402087640 1 --2- 172.21.15.114:0/2867433428 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3fc103fb0 0x7fa3fc10c580 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:18.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.778+0000 7fa402087640 1 --2- 172.21.15.114:0/2867433428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3fc0fee60 0x7fa3fc10c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:18.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.778+0000 7fa402087640 1 -- 172.21.15.114:0/2867433428 >> 172.21.15.114:0/2867433428 conn(0x7fa3fc075c60 msgr2=0x7fa3fc1023d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:18.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.778+0000 7fa402087640 1 -- 172.21.15.114:0/2867433428 shutdown_connections 2024-09-13T18:25:18.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.778+0000 7fa402087640 1 -- 172.21.15.114:0/2867433428 wait complete. 2024-09-13T18:25:18.799 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-13T18:25:18.799 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:25:18.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.971+0000 7fc4b6c55640 1 -- 172.21.15.114:0/2782314347 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4a809be50 msgr2=0x7fc4a809c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:18.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.971+0000 7fc4b6c55640 1 --2- 172.21.15.114:0/2782314347 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4a809be50 0x7fc4a809c230 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac009900 tx=0x7fc4ac02a400 comp rx=0 tx=0).stop 2024-09-13T18:25:18.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.971+0000 7fc4b6c55640 1 -- 172.21.15.114:0/2782314347 shutdown_connections 2024-09-13T18:25:18.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.971+0000 7fc4b6c55640 1 --2- 172.21.15.114:0/2782314347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4a80a0f50 0x7fc4a80a8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:18.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.971+0000 7fc4b6c55640 1 --2- 172.21.15.114:0/2782314347 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4a809c800 0x7fc4a80a07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:18.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.971+0000 7fc4b6c55640 1 --2- 172.21.15.114:0/2782314347 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4a809be50 0x7fc4a809c230 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:18.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.971+0000 7fc4b6c55640 1 -- 172.21.15.114:0/2782314347 >> 172.21.15.114:0/2782314347 conn(0x7fc4a8008fe0 msgr2=0x7fc4a809a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:18.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.971+0000 7fc4b6c55640 1 -- 172.21.15.114:0/2782314347 shutdown_connections 2024-09-13T18:25:18.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.972+0000 7fc4b6c55640 1 -- 172.21.15.114:0/2782314347 wait complete. 2024-09-13T18:25:18.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.972+0000 7fc4b6c55640 1 Processor -- start 2024-09-13T18:25:18.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.972+0000 7fc4b6c55640 1 -- start start 2024-09-13T18:25:18.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.973+0000 7fc4b6c55640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4a809be50 0x7fc4a81430b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:18.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.973+0000 7fc4b6c55640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4a80a0f50 0x7fc4a81435f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:18.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.973+0000 7fc4b6c55640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4a81479e0 0x7fc4a8147e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:18.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.973+0000 7fc4b6c55640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a81483d0 con 0x7fc4a80a0f50 2024-09-13T18:25:18.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.973+0000 7fc4b6c55640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a8148540 con 0x7fc4a809be50 2024-09-13T18:25:18.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.973+0000 7fc4b6c55640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a81486b0 con 0x7fc4a81479e0 2024-09-13T18:25:18.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.973+0000 7fc4b6454640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4a81479e0 0x7fc4a8147e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:18.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.973+0000 7fc4b6454640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4a81479e0 0x7fc4a8147e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:38048/0 (socket says 172.21.15.114:38048) 2024-09-13T18:25:18.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.973+0000 7fc4b6454640 1 -- 172.21.15.114:0/2746027394 learned_addr learned my addr 172.21.15.114:0/2746027394 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:18.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.973+0000 7fc4b5c53640 1 --2- 172.21.15.114:0/2746027394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4a809be50 0x7fc4a81430b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:18.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.974+0000 7fc4b6454640 1 -- 172.21.15.114:0/2746027394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4a809be50 msgr2=0x7fc4a81430b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:18.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.974+0000 7fc4b6454640 1 --2- 172.21.15.114:0/2746027394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4a809be50 0x7fc4a81430b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:18.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.974+0000 7fc4b6454640 1 -- 172.21.15.114:0/2746027394 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4a80a0f50 msgr2=0x7fc4a81435f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:18.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.974+0000 7fc4b6454640 1 --2- 172.21.15.114:0/2746027394 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4a80a0f50 0x7fc4a81435f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:18.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.974+0000 7fc4b6454640 1 -- 172.21.15.114:0/2746027394 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4ac009560 con 0x7fc4a81479e0 2024-09-13T18:25:18.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.974+0000 7fc4b6454640 1 --2- 172.21.15.114:0/2746027394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4a81479e0 0x7fc4a8147e90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc4b00667e0 tx=0x7fc4b006b780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:18.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.975+0000 7fc4a6ffd640 1 -- 172.21.15.114:0/2746027394 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b006d040 con 0x7fc4a81479e0 2024-09-13T18:25:18.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.975+0000 7fc4a6ffd640 1 -- 172.21.15.114:0/2746027394 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc4b006bea0 con 0x7fc4a81479e0 2024-09-13T18:25:18.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.975+0000 7fc4a6ffd640 1 -- 172.21.15.114:0/2746027394 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b0073e80 con 0x7fc4a81479e0 2024-09-13T18:25:18.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.975+0000 7fc4b6c55640 1 -- 172.21.15.114:0/2746027394 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4a8137140 con 0x7fc4a81479e0 2024-09-13T18:25:18.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.977+0000 7fc4b6c55640 1 -- 172.21.15.114:0/2746027394 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4a8137570 con 0x7fc4a81479e0 2024-09-13T18:25:18.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.978+0000 7fc4b6c55640 1 -- 172.21.15.114:0/2746027394 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4a809d2d0 con 0x7fc4a81479e0 2024-09-13T18:25:18.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.981+0000 7fc4a6ffd640 1 -- 172.21.15.114:0/2746027394 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fc4b0075110 con 0x7fc4a81479e0 2024-09-13T18:25:18.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.982+0000 7fc4a6ffd640 1 --2- 172.21.15.114:0/2746027394 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc49c0a13d0 0x7fc49c0a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:18.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.982+0000 7fc4a6ffd640 1 -- 172.21.15.114:0/2746027394 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fc4b012eda0 con 0x7fc4a81479e0 2024-09-13T18:25:18.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.982+0000 7fc4b5c53640 1 --2- 172.21.15.114:0/2746027394 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc49c0a13d0 0x7fc49c0a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:18.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.983+0000 7fc4b5c53640 1 --2- 172.21.15.114:0/2746027394 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc49c0a13d0 0x7fc49c0a3890 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac000c00 tx=0x7fc4ac005e80 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:18.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:18.987+0000 7fc4a6ffd640 1 -- 172.21.15.114:0/2746027394 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fc4b00f7f50 con 0x7fc4a81479e0 2024-09-13T18:25:19.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.398+0000 7fc4b6c55640 1 -- 172.21.15.114:0/2746027394 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc4a809c230 con 0x7fc4a81479e0 2024-09-13T18:25:19.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.399+0000 7fc4a6ffd640 1 -- 172.21.15.114:0/2746027394 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+10937 (secure 0 0 0) 0x7fc4b00fb570 con 0x7fc4a81479e0 2024-09-13T18:25:19.401 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:25:19.402 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4162817497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4162817497},{"type":"v1","addr":"172.21.15.114:6835","nonce":4162817497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/2435428478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":2435428478},{"type":"v1","addr":"172.21.15.114:6837","nonce":2435428478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T18:20:01.895117+0000","modified":"2024-09-13T18:24:01.324695+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24482,"mds_2":24475,"mds_3":24458,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/2046516451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":2046516451},{"type":"v1","addr":"172.21.15.114:6839","nonce":2046516451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":10,"addr":"172.21.15.114:6841/1956923446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":1956923446},{"type":"v1","addr":"172.21.15.114:6841","nonce":1956923446}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24391":{"gid":24391,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/507301496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":507301496},{"type":"v1","addr":"172.21.15.204:6833","nonce":507301496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4028657219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4028657219},{"type":"v1","addr":"172.21.15.134:6833","nonce":4028657219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/57029119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":57029119},{"type":"v1","addr":"172.21.15.204:6835","nonce":57029119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3068880939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3068880939},{"type":"v1","addr":"172.21.15.204:6837","nonce":3068880939}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"e","rank":3,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.134:6835/1150003275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1150003275},{"type":"v1","addr":"172.21.15.134:6835","nonce":1150003275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/344883912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":344883912},{"type":"v1","addr":"172.21.15.204:6839","nonce":344883912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6837/891692210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":891692210},{"type":"v1","addr":"172.21.15.134:6837","nonce":891692210}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6839/3466309724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3466309724},{"type":"v1","addr":"172.21.15.134:6839","nonce":3466309724}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:25:19.402 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 32 2024-09-13T18:25:19.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.402+0000 7fc4b6c55640 1 -- 172.21.15.114:0/2746027394 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc49c0a13d0 msgr2=0x7fc49c0a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:19.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.402+0000 7fc4b6c55640 1 --2- 172.21.15.114:0/2746027394 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc49c0a13d0 0x7fc49c0a3890 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac000c00 tx=0x7fc4ac005e80 comp rx=0 tx=0).stop 2024-09-13T18:25:19.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.402+0000 7fc4b6c55640 1 -- 172.21.15.114:0/2746027394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4a81479e0 msgr2=0x7fc4a8147e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:19.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.402+0000 7fc4b6c55640 1 --2- 172.21.15.114:0/2746027394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4a81479e0 0x7fc4a8147e90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc4b00667e0 tx=0x7fc4b006b780 comp rx=0 tx=0).stop 2024-09-13T18:25:19.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.403+0000 7fc4b6c55640 1 -- 172.21.15.114:0/2746027394 shutdown_connections 2024-09-13T18:25:19.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.403+0000 7fc4b6c55640 1 --2- 172.21.15.114:0/2746027394 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc49c0a13d0 0x7fc49c0a3890 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.403+0000 7fc4b6c55640 1 --2- 172.21.15.114:0/2746027394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4a81479e0 0x7fc4a8147e90 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.403+0000 7fc4b6c55640 1 --2- 172.21.15.114:0/2746027394 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4a80a0f50 0x7fc4a81435f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.403+0000 7fc4b6c55640 1 --2- 172.21.15.114:0/2746027394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4a809be50 0x7fc4a81430b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.403+0000 7fc4b6c55640 1 -- 172.21.15.114:0/2746027394 >> 172.21.15.114:0/2746027394 conn(0x7fc4a8008fe0 msgr2=0x7fc4a8009820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:19.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.403+0000 7fc4b6c55640 1 -- 172.21.15.114:0/2746027394 shutdown_connections 2024-09-13T18:25:19.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.403+0000 7fc4b6c55640 1 -- 172.21.15.114:0/2746027394 wait complete. 2024-09-13T18:25:19.426 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-09-13T18:25:19.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.599+0000 7fde3cc80640 1 -- 172.21.15.114:0/1589362176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde38107720 msgr2=0x7fde38107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:19.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.599+0000 7fde3cc80640 1 --2- 172.21.15.114:0/1589362176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde38107720 0x7fde38107b00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fde280085a0 tx=0x7fde2800ac00 comp rx=0 tx=0).stop 2024-09-13T18:25:19.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.599+0000 7fde3cc80640 1 -- 172.21.15.114:0/1589362176 shutdown_connections 2024-09-13T18:25:19.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.599+0000 7fde3cc80640 1 --2- 172.21.15.114:0/1589362176 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fde3810ccd0 0x7fde38113620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.599+0000 7fde3cc80640 1 --2- 172.21.15.114:0/1589362176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde381080d0 0x7fde3810c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.599+0000 7fde3cc80640 1 --2- 172.21.15.114:0/1589362176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde38107720 0x7fde38107b00 secure :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fde280085a0 tx=0x7fde2800ac00 comp rx=0 tx=0).stop 2024-09-13T18:25:19.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.599+0000 7fde3cc80640 1 -- 172.21.15.114:0/1589362176 >> 172.21.15.114:0/1589362176 conn(0x7fde3806b3c0 msgr2=0x7fde3806b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:19.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.600+0000 7fde3cc80640 1 -- 172.21.15.114:0/1589362176 shutdown_connections 2024-09-13T18:25:19.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.600+0000 7fde3cc80640 1 -- 172.21.15.114:0/1589362176 wait complete. 2024-09-13T18:25:19.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.601+0000 7fde3cc80640 1 Processor -- start 2024-09-13T18:25:19.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.601+0000 7fde3cc80640 1 -- start start 2024-09-13T18:25:19.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.601+0000 7fde3cc80640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde3810ccd0 0x7fde381ab120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:19.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.601+0000 7fde3cc80640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fde381ab660 0x7fde381120c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:19.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.601+0000 7fde3cc80640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde38112600 0x7fde38112ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:19.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.601+0000 7fde3cc80640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde381abcb0 con 0x7fde381ab660 2024-09-13T18:25:19.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.601+0000 7fde3cc80640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde381b1f00 con 0x7fde3810ccd0 2024-09-13T18:25:19.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.601+0000 7fde3cc80640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde381b20c0 con 0x7fde38112600 2024-09-13T18:25:19.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.602+0000 7fde36d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde38112600 0x7fde38112ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:19.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.602+0000 7fde36d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde38112600 0x7fde38112ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:38062/0 (socket says 172.21.15.114:38062) 2024-09-13T18:25:19.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.602+0000 7fde36d9d640 1 -- 172.21.15.114:0/54853648 learned_addr learned my addr 172.21.15.114:0/54853648 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:19.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.602+0000 7fde35d9b640 1 --2- 172.21.15.114:0/54853648 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fde381ab660 0x7fde381120c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:19.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.602+0000 7fde3659c640 1 --2- 172.21.15.114:0/54853648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde3810ccd0 0x7fde381ab120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:19.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.602+0000 7fde36d9d640 1 -- 172.21.15.114:0/54853648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde3810ccd0 msgr2=0x7fde381ab120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:19.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.602+0000 7fde36d9d640 1 --2- 172.21.15.114:0/54853648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde3810ccd0 0x7fde381ab120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.602+0000 7fde36d9d640 1 -- 172.21.15.114:0/54853648 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fde381ab660 msgr2=0x7fde381120c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:19.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.602+0000 7fde36d9d640 1 --2- 172.21.15.114:0/54853648 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fde381ab660 0x7fde381120c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.602+0000 7fde36d9d640 1 -- 172.21.15.114:0/54853648 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde280081d0 con 0x7fde38112600 2024-09-13T18:25:19.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.602+0000 7fde36d9d640 1 --2- 172.21.15.114:0/54853648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde38112600 0x7fde38112ab0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fde3000b970 tx=0x7fde3000be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:19.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.603+0000 7fde277fe640 1 -- 172.21.15.114:0/54853648 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3000cb40 con 0x7fde38112600 2024-09-13T18:25:19.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.603+0000 7fde3cc80640 1 -- 172.21.15.114:0/54853648 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde381b23a0 con 0x7fde38112600 2024-09-13T18:25:19.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.603+0000 7fde3cc80640 1 -- 172.21.15.114:0/54853648 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde381b28b0 con 0x7fde38112600 2024-09-13T18:25:19.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.603+0000 7fde277fe640 1 -- 172.21.15.114:0/54853648 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fde3000cca0 con 0x7fde38112600 2024-09-13T18:25:19.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.604+0000 7fde277fe640 1 -- 172.21.15.114:0/54853648 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde30007610 con 0x7fde38112600 2024-09-13T18:25:19.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.606+0000 7fde277fe640 1 -- 172.21.15.114:0/54853648 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fde300078b0 con 0x7fde38112600 2024-09-13T18:25:19.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.607+0000 7fde3fa7a740 1 Processor -- start 2024-09-13T18:25:19.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.607+0000 7fde3fa7a740 1 -- start start 2024-09-13T18:25:19.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.607+0000 7fde3fa7a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568d0473970 0x5568d0473d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:19.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.607+0000 7fde3fa7a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568d0474290 0x5568d047e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:19.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.607+0000 7fde3fa7a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568d047eb30 0x5568d0480f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:19.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.607+0000 7fde3fa7a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568d03ddcc0 con 0x5568d047eb30 2024-09-13T18:25:19.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.607+0000 7fde3fa7a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568d03db450 con 0x5568d0474290 2024-09-13T18:25:19.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.607+0000 7fde3fa7a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568d03d6d40 con 0x5568d0473970 2024-09-13T18:25:19.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.608+0000 7fde35d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568d0473970 0x5568d0473d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:19.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.608+0000 7fde35d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568d0473970 0x5568d0473d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:38068/0 (socket says 172.21.15.114:38068) 2024-09-13T18:25:19.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.608+0000 7fde35d9b640 1 -- 172.21.15.114:0/2372053886 learned_addr learned my addr 172.21.15.114:0/2372053886 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:19.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.608+0000 7fde3659c640 1 --2- 172.21.15.114:0/2372053886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568d0474290 0x5568d047e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:19.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.608+0000 7fde35d9b640 1 --2- 172.21.15.114:0/2372053886 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568d047eb30 0x5568d0480f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:19.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.608+0000 7fde35d9b640 1 -- 172.21.15.114:0/2372053886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568d0474290 msgr2=0x5568d047e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:19.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.608+0000 7fde35d9b640 1 --2- 172.21.15.114:0/2372053886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568d0474290 0x5568d047e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.608+0000 7fde35d9b640 1 -- 172.21.15.114:0/2372053886 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568d047eb30 msgr2=0x5568d0480f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:19.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.608+0000 7fde35d9b640 1 --2- 172.21.15.114:0/2372053886 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568d047eb30 0x5568d0480f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.608+0000 7fde35d9b640 1 -- 172.21.15.114:0/2372053886 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568d039e620 con 0x5568d0473970 2024-09-13T18:25:19.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.608+0000 7fde277fe640 1 --2- 172.21.15.114:0/54853648 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fde200a1390 0x7fde200a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:19.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.608+0000 7fde277fe640 1 -- 172.21.15.114:0/54853648 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fde300cc560 con 0x7fde38112600 2024-09-13T18:25:19.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.608+0000 7fde3659c640 1 --2- 172.21.15.114:0/54853648 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fde200a1390 0x7fde200a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:19.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.608+0000 7fde35d9b640 1 --2- 172.21.15.114:0/2372053886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568d0473970 0x5568d0473d50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fde2c018510 tx=0x7fde2c005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e8248ae1cdabf4d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:19.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.609+0000 7fde24ff9640 1 -- 172.21.15.114:0/2372053886 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2c01e070 con 0x5568d0473970 2024-09-13T18:25:19.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.609+0000 7fde24ff9640 1 -- 172.21.15.114:0/2372053886 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fde2c01a880 con 0x5568d0473970 2024-09-13T18:25:19.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.609+0000 7fde24ff9640 1 -- 172.21.15.114:0/2372053886 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2c010030 con 0x5568d0473970 2024-09-13T18:25:19.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.610+0000 7fde3fa7a740 1 -- 172.21.15.114:0/2372053886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568d0473970 msgr2=0x5568d0473d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:19.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.610+0000 7fde3fa7a740 1 --2- 172.21.15.114:0/2372053886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568d0473970 0x5568d0473d50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fde2c018510 tx=0x7fde2c005f90 comp rx=0 tx=0).stop 2024-09-13T18:25:19.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.610+0000 7fde3659c640 1 --2- 172.21.15.114:0/54853648 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fde200a1390 0x7fde200a3850 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fde2800a040 tx=0x7fde28000bc0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:19.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.610+0000 7fde3fa7a740 1 -- 172.21.15.114:0/2372053886 shutdown_connections 2024-09-13T18:25:19.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.610+0000 7fde3fa7a740 1 --2- 172.21.15.114:0/2372053886 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568d047eb30 0x5568d0480f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.610+0000 7fde3fa7a740 1 --2- 172.21.15.114:0/2372053886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568d0474290 0x5568d047e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.610+0000 7fde3fa7a740 1 --2- 172.21.15.114:0/2372053886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568d0473970 0x5568d0473d50 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.610+0000 7fde3fa7a740 1 -- 172.21.15.114:0/2372053886 >> 172.21.15.114:0/2372053886 conn(0x5568d046fee0 msgr2=0x5568d04702c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:19.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.610+0000 7fde3fa7a740 1 -- 172.21.15.114:0/2372053886 shutdown_connections 2024-09-13T18:25:19.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.610+0000 7fde3fa7a740 1 -- 172.21.15.114:0/2372053886 wait complete. 2024-09-13T18:25:19.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.610+0000 7fde3fa7a740 1 -- start start 2024-09-13T18:25:19.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.610+0000 7fde3fa7a740 1 Processor -- start 2024-09-13T18:25:19.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.611+0000 7fde3fa7a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568d0473970 0x5568d047e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:19.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.611+0000 7fde3fa7a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568d0474290 0x5568d047e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:19.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.611+0000 7fde3fa7a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568d047eb30 0x5568d047b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:19.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.611+0000 7fde3fa7a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568d0395180 con 0x5568d047eb30 2024-09-13T18:25:19.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.611+0000 7fde3fa7a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568d00af430 con 0x5568d0474290 2024-09-13T18:25:19.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.611+0000 7fde3fa7a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568d035a5e0 con 0x5568d0473970 2024-09-13T18:25:19.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.611+0000 7fde3fa7a740 10 client.0 _ll_register_callbacks cb 0x5568d03c6aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:25:19.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.611+0000 7fde3659c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568d0473970 0x5568d047e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:19.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.611+0000 7fde3659c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568d0473970 0x5568d047e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:38076/0 (socket says 172.21.15.114:38076) 2024-09-13T18:25:19.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.611+0000 7fde3659c640 1 -- 172.21.15.114:0/1134289375 learned_addr learned my addr 172.21.15.114:0/1134289375 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:19.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.611+0000 7fde35d9b640 1 --2- 172.21.15.114:0/1134289375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568d0474290 0x5568d047e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:19.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.612+0000 7fde36d9d640 1 --2- 172.21.15.114:0/1134289375 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568d047eb30 0x5568d047b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:19.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.612+0000 7fde3659c640 1 -- 172.21.15.114:0/1134289375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568d0474290 msgr2=0x5568d047e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:19.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.612+0000 7fde3659c640 1 --2- 172.21.15.114:0/1134289375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568d0474290 0x5568d047e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.612+0000 7fde3659c640 1 -- 172.21.15.114:0/1134289375 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568d047eb30 msgr2=0x5568d047b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:19.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.612+0000 7fde3659c640 1 --2- 172.21.15.114:0/1134289375 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568d047eb30 0x5568d047b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.612+0000 7fde3659c640 1 -- 172.21.15.114:0/1134289375 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde2c00bdb0 con 0x5568d0473970 2024-09-13T18:25:19.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.612+0000 7fde3659c640 1 --2- 172.21.15.114:0/1134289375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568d0473970 0x5568d047e0b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fde2802a500 tx=0x7fde2800ac00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a074f96c3d9b030 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:19.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.612+0000 7fde1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:25:19.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.612+0000 7fde1dffb640 1 -- 172.21.15.114:0/1134289375 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde280038c0 con 0x5568d0473970 2024-09-13T18:25:19.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.613+0000 7fde1dffb640 1 -- 172.21.15.114:0/1134289375 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fde28003a20 con 0x5568d0473970 2024-09-13T18:25:19.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.613+0000 7fde3fa7a740 1 -- 172.21.15.114:0/1134289375 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5568d0400b80 con 0x5568d0473970 2024-09-13T18:25:19.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.613+0000 7fde1dffb640 1 -- 172.21.15.114:0/1134289375 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde28003be0 con 0x5568d0473970 2024-09-13T18:25:19.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.613+0000 7fde3fa7a740 10 client.26008 fetch_fsmap learned FSMap version 32 2024-09-13T18:25:19.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.613+0000 7fde3fa7a740 1 -- 172.21.15.114:0/1134289375 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5568d03d53c0 con 0x5568d0473970 2024-09-13T18:25:19.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.614+0000 7fde1dffb640 1 -- 172.21.15.114:0/1134289375 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5568d0400b80 con 0x5568d0473970 2024-09-13T18:25:19.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.614+0000 7fde1dffb640 1 -- 172.21.15.114:0/1134289375 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fde2804a470 con 0x5568d0473970 2024-09-13T18:25:19.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.614+0000 7fde3fa7a740 10 client.26008 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:25:19.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.614+0000 7fde3fa7a740 10 client.26008 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-13T18:25:19.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.614+0000 7fde3fa7a740 20 client.26008 populate_metadata read hostname 'smithi114' 2024-09-13T18:25:19.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.615+0000 7fde3fa7a740 1 --2- 172.21.15.114:0/1134289375 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x5568d048e5d0 0x5568d04909f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:19.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.615+0000 7fde3fa7a740 4 client.26008 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:25:19.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.615+0000 7fde3fa7a740 1 -- 172.21.15.114:0/1134289375 --> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5568d03c91f0 con 0x5568d048e5d0 2024-09-13T18:25:19.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.615+0000 7fde35d9b640 1 --2- 172.21.15.114:0/1134289375 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x5568d048e5d0 0x5568d04909f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:19.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.615+0000 7fde35d9b640 1 --2- 172.21.15.114:0/1134289375 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x5568d048e5d0 0x5568d04909f0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=27f0062e5e8bff8f server_cookie=cdfac9b25fb808df in_seq=0 out_seq=0 2024-09-13T18:25:19.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.615+0000 7fde1dffb640 10 client.26008 ms_handle_connect on v2:172.21.15.114:6840/1956923446 2024-09-13T18:25:19.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.616+0000 7fde1dffb640 1 -- 172.21.15.114:0/1134289375 <== mds.4 v2:172.21.15.114:6840/1956923446 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5568d03c91f0 con 0x5568d048e5d0 2024-09-13T18:25:19.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.616+0000 7fde1dffb640 10 client.26008 handle_command_reply: tid=0 2024-09-13T18:25:19.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.616+0000 7fde3fa7a740 1 client.26008 shutdown 2024-09-13T18:25:19.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.617+0000 7fde3fa7a740 1 -- 172.21.15.114:0/1134289375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568d0473970 msgr2=0x5568d047e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:19.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.617+0000 7fde3fa7a740 1 --2- 172.21.15.114:0/1134289375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568d0473970 0x5568d047e0b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fde2802a500 tx=0x7fde2800ac00 comp rx=0 tx=0).stop 2024-09-13T18:25:19.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.617+0000 7fde3fa7a740 1 -- 172.21.15.114:0/1134289375 shutdown_connections 2024-09-13T18:25:19.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.617+0000 7fde3fa7a740 1 --2- 172.21.15.114:0/1134289375 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x5568d048e5d0 0x5568d04909f0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.617+0000 7fde3fa7a740 1 --2- 172.21.15.114:0/1134289375 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568d047eb30 0x5568d047b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.617+0000 7fde3fa7a740 1 --2- 172.21.15.114:0/1134289375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568d0474290 0x5568d047e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.617+0000 7fde3fa7a740 1 --2- 172.21.15.114:0/1134289375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568d0473970 0x5568d047e0b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.617+0000 7fde3fa7a740 1 -- 172.21.15.114:0/1134289375 >> 172.21.15.114:0/1134289375 conn(0x5568d046fee0 msgr2=0x5568d046f230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:19.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.617+0000 7fde1dffb640 0 client.26008 ms_handle_reset on v2:172.21.15.114:6840/1956923446 2024-09-13T18:25:19.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.617+0000 7fde3fa7a740 1 -- 172.21.15.114:0/1134289375 shutdown_connections 2024-09-13T18:25:19.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.617+0000 7fde3fa7a740 1 -- 172.21.15.114:0/1134289375 wait complete. 2024-09-13T18:25:19.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.617+0000 7fde3fa7a740 20 client.26008 trim_cache size 0 max 16384 2024-09-13T18:25:19.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.638+0000 7fde3fa7a740 1 Processor -- start 2024-09-13T18:25:19.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.639+0000 7fde3fa7a740 1 -- start start 2024-09-13T18:25:19.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.639+0000 7fde3fa7a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568d0473970 0x5568d04763e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:19.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.639+0000 7fde3fa7a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568d0474290 0x5568d0476920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:19.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.639+0000 7fde3fa7a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568d047eb30 0x5568d0478750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:19.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.639+0000 7fde3fa7a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568d047c860 con 0x5568d0474290 2024-09-13T18:25:19.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.639+0000 7fde3fa7a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568d046b8c0 con 0x5568d047eb30 2024-09-13T18:25:19.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.640+0000 7fde36d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568d047eb30 0x5568d0478750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:19.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.640+0000 7fde36d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568d047eb30 0x5568d0478750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:43750/0 (socket says 172.21.15.114:43750) 2024-09-13T18:25:19.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.640+0000 7fde36d9d640 1 -- 172.21.15.114:0/2465455595 learned_addr learned my addr 172.21.15.114:0/2465455595 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:19.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.640+0000 7fde3fa7a740 1 -- 172.21.15.114:0/2465455595 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568d048b200 con 0x5568d0473970 2024-09-13T18:25:19.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.640+0000 7fde35d9b640 1 --2- 172.21.15.114:0/2465455595 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568d0474290 0x5568d0476920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:19.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.640+0000 7fde36d9d640 1 -- 172.21.15.114:0/2465455595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568d0473970 msgr2=0x5568d04763e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:25:19.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.640+0000 7fde36d9d640 1 --2- 172.21.15.114:0/2465455595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568d0473970 0x5568d04763e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.640+0000 7fde36d9d640 1 -- 172.21.15.114:0/2465455595 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568d0474290 msgr2=0x5568d0476920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:19.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.640+0000 7fde36d9d640 1 --2- 172.21.15.114:0/2465455595 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568d0474290 0x5568d0476920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.640+0000 7fde36d9d640 1 -- 172.21.15.114:0/2465455595 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568d0377770 con 0x5568d047eb30 2024-09-13T18:25:19.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.641+0000 7fde36d9d640 1 --2- 172.21.15.114:0/2465455595 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568d047eb30 0x5568d0478750 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fde300c9ad0 tx=0x7fde30095220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4379ad4ac50d8af server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:19.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.641+0000 7fde257fa640 1 -- 172.21.15.114:0/2465455595 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde300994e0 con 0x5568d047eb30 2024-09-13T18:25:19.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.641+0000 7fde257fa640 1 -- 172.21.15.114:0/2465455595 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fde3009d260 con 0x5568d047eb30 2024-09-13T18:25:19.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.641+0000 7fde257fa640 1 -- 172.21.15.114:0/2465455595 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde300c9d70 con 0x5568d047eb30 2024-09-13T18:25:19.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.642+0000 7fde3fa7a740 1 -- 172.21.15.114:0/2465455595 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568d047eb30 msgr2=0x5568d0478750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:19.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.642+0000 7fde3fa7a740 1 --2- 172.21.15.114:0/2465455595 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568d047eb30 0x5568d0478750 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fde300c9ad0 tx=0x7fde30095220 comp rx=0 tx=0).stop 2024-09-13T18:25:19.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.643+0000 7fde3fa7a740 1 -- 172.21.15.114:0/2465455595 shutdown_connections 2024-09-13T18:25:19.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.643+0000 7fde3fa7a740 1 --2- 172.21.15.114:0/2465455595 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568d047eb30 0x5568d0478750 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.643+0000 7fde3fa7a740 1 --2- 172.21.15.114:0/2465455595 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568d0474290 0x5568d0476920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.643+0000 7fde3fa7a740 1 --2- 172.21.15.114:0/2465455595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568d0473970 0x5568d04763e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.643+0000 7fde3fa7a740 1 -- 172.21.15.114:0/2465455595 >> 172.21.15.114:0/2465455595 conn(0x5568d046fee0 msgr2=0x5568d04702c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:19.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.644+0000 7fde3fa7a740 1 -- 172.21.15.114:0/2465455595 shutdown_connections 2024-09-13T18:25:19.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.644+0000 7fde3fa7a740 1 -- 172.21.15.114:0/2465455595 wait complete. 2024-09-13T18:25:19.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.645+0000 7fde3fa7a740 1 -- start start 2024-09-13T18:25:19.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.645+0000 7fde3fa7a740 1 Processor -- start 2024-09-13T18:25:19.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.645+0000 7fde3fa7a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568d0473970 0x5568d04824b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:19.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.645+0000 7fde3fa7a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568d0474290 0x5568d04829f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:19.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.645+0000 7fde3fa7a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568d047eb30 0x5568d0482f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:19.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.645+0000 7fde3fa7a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568d04e4b90 con 0x5568d0473970 2024-09-13T18:25:19.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.645+0000 7fde3fa7a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568d04e4cd0 con 0x5568d047eb30 2024-09-13T18:25:19.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.645+0000 7fde3fa7a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568d04e4e90 con 0x5568d0474290 2024-09-13T18:25:19.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.646+0000 7fde3fa7a740 10 client.0 _ll_register_callbacks cb 0x7fde2c00b850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:25:19.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.646+0000 7fde3659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568d0473970 0x5568d04824b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:19.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.646+0000 7fde3659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568d0473970 0x5568d04824b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43858/0 (socket says 172.21.15.114:43858) 2024-09-13T18:25:19.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.646+0000 7fde36d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568d047eb30 0x5568d0482f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:19.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.646+0000 7fde3659c640 1 -- 172.21.15.114:0/370827003 learned_addr learned my addr 172.21.15.114:0/370827003 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:19.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.646+0000 7fde35d9b640 1 --2- 172.21.15.114:0/370827003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568d0474290 0x5568d04829f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:19.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.647+0000 7fde3659c640 1 -- 172.21.15.114:0/370827003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568d0474290 msgr2=0x5568d04829f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:19.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.647+0000 7fde3659c640 1 --2- 172.21.15.114:0/370827003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568d0474290 0x5568d04829f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.647+0000 7fde3659c640 1 -- 172.21.15.114:0/370827003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568d047eb30 msgr2=0x5568d0482f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:19.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.647+0000 7fde3659c640 1 --2- 172.21.15.114:0/370827003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568d047eb30 0x5568d0482f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.647+0000 7fde3659c640 1 -- 172.21.15.114:0/370827003 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde381b28b0 con 0x5568d0473970 2024-09-13T18:25:19.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.648+0000 7fde3659c640 1 --2- 172.21.15.114:0/370827003 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568d0473970 0x5568d04824b0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fde280084c0 tx=0x7fde2804bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34d89ab99dca5fa0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:19.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.648+0000 7fde1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:25:19.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.648+0000 7fde1dffb640 1 -- 172.21.15.114:0/370827003 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2804a9c0 con 0x5568d0473970 2024-09-13T18:25:19.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.648+0000 7fde3fa7a740 1 -- 172.21.15.114:0/370827003 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fde2c01a880 con 0x5568d0473970 2024-09-13T18:25:19.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.649+0000 7fde1dffb640 1 -- 172.21.15.114:0/370827003 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fde2804ab20 con 0x5568d0473970 2024-09-13T18:25:19.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.649+0000 7fde1dffb640 1 -- 172.21.15.114:0/370827003 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde280038c0 con 0x5568d0473970 2024-09-13T18:25:19.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.649+0000 7fde3fa7a740 10 client.16641 fetch_fsmap learned FSMap version 32 2024-09-13T18:25:19.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.649+0000 7fde3fa7a740 1 -- 172.21.15.114:0/370827003 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5568d039ce80 con 0x5568d0473970 2024-09-13T18:25:19.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.650+0000 7fde1dffb640 1 -- 172.21.15.114:0/370827003 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fde2c01a880 con 0x5568d0473970 2024-09-13T18:25:19.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.650+0000 7fde1dffb640 1 -- 172.21.15.114:0/370827003 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fde2800a6e0 con 0x5568d0473970 2024-09-13T18:25:19.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.650+0000 7fde3fa7a740 10 client.16641 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:25:19.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.650+0000 7fde3fa7a740 10 client.16641 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-13T18:25:19.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.650+0000 7fde3fa7a740 20 client.16641 populate_metadata read hostname 'smithi114' 2024-09-13T18:25:19.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.650+0000 7fde3fa7a740 1 --2- 172.21.15.114:0/370827003 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x5568d04e6370 0x5568d04e8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:19.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.650+0000 7fde3fa7a740 4 client.16641 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:25:19.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.650+0000 7fde3fa7a740 1 -- 172.21.15.114:0/370827003 --> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5568d00ca4e0 con 0x5568d04e6370 2024-09-13T18:25:19.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.653+0000 7fde35d9b640 1 --2- 172.21.15.114:0/370827003 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x5568d04e6370 0x5568d04e8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:19.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.654+0000 7fde35d9b640 1 --2- 172.21.15.114:0/370827003 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x5568d04e6370 0x5568d04e8790 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8863258d28e25933 server_cookie=f6ec70815dee995a in_seq=0 out_seq=0 2024-09-13T18:25:19.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.654+0000 7fde1dffb640 10 client.16641 ms_handle_connect on v2:172.21.15.114:6840/1956923446 2024-09-13T18:25:19.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.654+0000 7fde1dffb640 1 -- 172.21.15.114:0/370827003 <== mds.4 v2:172.21.15.114:6840/1956923446 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5568d00ca4e0 con 0x5568d04e6370 2024-09-13T18:25:19.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.654+0000 7fde1dffb640 10 client.16641 handle_command_reply: tid=0 2024-09-13T18:25:19.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.654+0000 7fde3fa7a740 1 client.16641 shutdown 2024-09-13T18:25:19.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.654+0000 7fde3fa7a740 1 -- 172.21.15.114:0/370827003 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568d0473970 msgr2=0x5568d04824b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:19.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.654+0000 7fde3fa7a740 1 --2- 172.21.15.114:0/370827003 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568d0473970 0x5568d04824b0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fde280084c0 tx=0x7fde2804bf90 comp rx=0 tx=0).stop 2024-09-13T18:25:19.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.655+0000 7fde3fa7a740 1 -- 172.21.15.114:0/370827003 shutdown_connections 2024-09-13T18:25:19.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.655+0000 7fde3fa7a740 1 --2- 172.21.15.114:0/370827003 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x5568d04e6370 0x5568d04e8790 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.655+0000 7fde3fa7a740 1 --2- 172.21.15.114:0/370827003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568d047eb30 0x5568d0482f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.655+0000 7fde3fa7a740 1 --2- 172.21.15.114:0/370827003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568d0474290 0x5568d04829f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.655+0000 7fde3fa7a740 1 --2- 172.21.15.114:0/370827003 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568d0473970 0x5568d04824b0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.655+0000 7fde1dffb640 0 client.16641 ms_handle_reset on v2:172.21.15.114:6840/1956923446 2024-09-13T18:25:19.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.655+0000 7fde3fa7a740 1 -- 172.21.15.114:0/370827003 >> 172.21.15.114:0/370827003 conn(0x5568d046fee0 msgr2=0x5568d046ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:19.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.655+0000 7fde3fa7a740 1 -- 172.21.15.114:0/370827003 shutdown_connections 2024-09-13T18:25:19.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.655+0000 7fde3fa7a740 1 -- 172.21.15.114:0/370827003 wait complete. 2024-09-13T18:25:19.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.656+0000 7fde3fa7a740 20 client.16641 trim_cache size 0 max 16384 2024-09-13T18:25:19.659 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:25:19.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.656+0000 7fde3cc80640 1 -- 172.21.15.114:0/54853648 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fde200a1390 msgr2=0x7fde200a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:19.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.656+0000 7fde3cc80640 1 --2- 172.21.15.114:0/54853648 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fde200a1390 0x7fde200a3850 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fde2800a040 tx=0x7fde28000bc0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.656+0000 7fde3cc80640 1 -- 172.21.15.114:0/54853648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde38112600 msgr2=0x7fde38112ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:19.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.656+0000 7fde3cc80640 1 --2- 172.21.15.114:0/54853648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde38112600 0x7fde38112ab0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fde3000b970 tx=0x7fde3000be30 comp rx=0 tx=0).stop 2024-09-13T18:25:19.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.657+0000 7fde3cc80640 1 -- 172.21.15.114:0/54853648 shutdown_connections 2024-09-13T18:25:19.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.657+0000 7fde3cc80640 1 --2- 172.21.15.114:0/54853648 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fde200a1390 0x7fde200a3850 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.657+0000 7fde3cc80640 1 --2- 172.21.15.114:0/54853648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde38112600 0x7fde38112ab0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.657+0000 7fde3cc80640 1 --2- 172.21.15.114:0/54853648 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fde381ab660 0x7fde381120c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.657+0000 7fde3cc80640 1 --2- 172.21.15.114:0/54853648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde3810ccd0 0x7fde381ab120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.657+0000 7fde3cc80640 1 -- 172.21.15.114:0/54853648 >> 172.21.15.114:0/54853648 conn(0x7fde3806b3c0 msgr2=0x7fde380711a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:19.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.657+0000 7fde3cc80640 1 -- 172.21.15.114:0/54853648 shutdown_connections 2024-09-13T18:25:19.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.657+0000 7fde3cc80640 1 -- 172.21.15.114:0/54853648 wait complete. 2024-09-13T18:25:19.674 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:19 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2746027394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:25:19.676 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-09-13T18:25:19.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.849+0000 7fe9c89c3640 1 -- 172.21.15.114:0/1122463935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9c4106ca0 msgr2=0x7fe9c4113180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:19.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.849+0000 7fe9c89c3640 1 --2- 172.21.15.114:0/1122463935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9c4106ca0 0x7fe9c4113180 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc009ca0 tx=0x7fe9bc02a950 comp rx=0 tx=0).stop 2024-09-13T18:25:19.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:19 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/2746027394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:25:19.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.851+0000 7fe9c89c3640 1 -- 172.21.15.114:0/1122463935 shutdown_connections 2024-09-13T18:25:19.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.851+0000 7fe9c89c3640 1 --2- 172.21.15.114:0/1122463935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9c4106ca0 0x7fe9c4113180 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.851+0000 7fe9c89c3640 1 --2- 172.21.15.114:0/1122463935 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe9c4106320 0x7fe9c4106760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.851+0000 7fe9c89c3640 1 --2- 172.21.15.114:0/1122463935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9c410c3e0 0x7fe9c410c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.851+0000 7fe9c89c3640 1 -- 172.21.15.114:0/1122463935 >> 172.21.15.114:0/1122463935 conn(0x7fe9c4075c60 msgr2=0x7fe9c4078080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:19.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.852+0000 7fe9c89c3640 1 -- 172.21.15.114:0/1122463935 shutdown_connections 2024-09-13T18:25:19.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.852+0000 7fe9c89c3640 1 -- 172.21.15.114:0/1122463935 wait complete. 2024-09-13T18:25:19.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.853+0000 7fe9c89c3640 1 Processor -- start 2024-09-13T18:25:19.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.853+0000 7fe9c89c3640 1 -- start start 2024-09-13T18:25:19.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.854+0000 7fe9c89c3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9c4106320 0x7fe9c4080970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:19.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.854+0000 7fe9c89c3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe9c4106ca0 0x7fe9c4080ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:19.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.854+0000 7fe9c89c3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9c410c3e0 0x7fe9c4079b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:19.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.854+0000 7fe9c259c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9c4106320 0x7fe9c4080970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:19.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.854+0000 7fe9c89c3640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c407a2f0 con 0x7fe9c4106ca0 2024-09-13T18:25:19.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.854+0000 7fe9c89c3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c407a460 con 0x7fe9c4106320 2024-09-13T18:25:19.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.854+0000 7fe9c89c3640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c407a620 con 0x7fe9c410c3e0 2024-09-13T18:25:19.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.854+0000 7fe9c259c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9c4106320 0x7fe9c4080970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:43776/0 (socket says 172.21.15.114:43776) 2024-09-13T18:25:19.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.854+0000 7fe9c259c640 1 -- 172.21.15.114:0/4275991846 learned_addr learned my addr 172.21.15.114:0/4275991846 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:19.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.854+0000 7fe9c1d9b640 1 --2- 172.21.15.114:0/4275991846 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe9c4106ca0 0x7fe9c4080ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:19.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.855+0000 7fe9c259c640 1 -- 172.21.15.114:0/4275991846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9c410c3e0 msgr2=0x7fe9c4079b60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:25:19.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.855+0000 7fe9c259c640 1 --2- 172.21.15.114:0/4275991846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9c410c3e0 0x7fe9c4079b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.855+0000 7fe9c259c640 1 -- 172.21.15.114:0/4275991846 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe9c4106ca0 msgr2=0x7fe9c4080ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:19.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.855+0000 7fe9c259c640 1 --2- 172.21.15.114:0/4275991846 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe9c4106ca0 0x7fe9c4080ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.855+0000 7fe9c259c640 1 -- 172.21.15.114:0/4275991846 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9bc009900 con 0x7fe9c4106320 2024-09-13T18:25:19.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.855+0000 7fe9c259c640 1 --2- 172.21.15.114:0/4275991846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9c4106320 0x7fe9c4080970 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe9b400d950 tx=0x7fe9b400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:19.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.855+0000 7fe9b37fe640 1 -- 172.21.15.114:0/4275991846 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b4009c30 con 0x7fe9c4106320 2024-09-13T18:25:19.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.856+0000 7fe9c89c3640 1 -- 172.21.15.114:0/4275991846 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9c407e500 con 0x7fe9c4106320 2024-09-13T18:25:19.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.856+0000 7fe9b37fe640 1 -- 172.21.15.114:0/4275991846 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe9b4009d90 con 0x7fe9c4106320 2024-09-13T18:25:19.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.856+0000 7fe9c89c3640 1 -- 172.21.15.114:0/4275991846 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9c407ea10 con 0x7fe9c4106320 2024-09-13T18:25:19.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.857+0000 7fe9b37fe640 1 -- 172.21.15.114:0/4275991846 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b400f620 con 0x7fe9c4106320 2024-09-13T18:25:19.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.859+0000 7fe9cb7bf740 1 Processor -- start 2024-09-13T18:25:19.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.860+0000 7fe9cb7bf740 1 -- start start 2024-09-13T18:25:19.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.860+0000 7fe9cb7bf740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe56f8e790 0x55fe56f8eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:19.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.860+0000 7fe9cb7bf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe56f8f0b0 0x55fe56f99410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:19.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.860+0000 7fe9cb7bf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe56f99950 0x55fe56f9bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:19.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.860+0000 7fe9cb7bf740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe56ef8cc0 con 0x55fe56f8e790 2024-09-13T18:25:19.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.860+0000 7fe9cb7bf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe56ef6450 con 0x55fe56f8f0b0 2024-09-13T18:25:19.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.860+0000 7fe9cb7bf740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe56ef1d40 con 0x55fe56f99950 2024-09-13T18:25:19.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.861+0000 7fe9c2d9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe56f8e790 0x55fe56f8eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:19.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.861+0000 7fe9c2d9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe56f8e790 0x55fe56f8eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43896/0 (socket says 172.21.15.114:43896) 2024-09-13T18:25:19.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.861+0000 7fe9c2d9d640 1 -- 172.21.15.114:0/2698255465 learned_addr learned my addr 172.21.15.114:0/2698255465 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:19.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.861+0000 7fe9c1d9b640 1 --2- 172.21.15.114:0/2698255465 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe56f99950 0x55fe56f9bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:19.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.862+0000 7fe9c259c640 1 --2- 172.21.15.114:0/2698255465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe56f8f0b0 0x55fe56f99410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:19.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.862+0000 7fe9b37fe640 1 -- 172.21.15.114:0/4275991846 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fe9b4010980 con 0x7fe9c4106320 2024-09-13T18:25:19.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.863+0000 7fe9c1d9b640 1 -- 172.21.15.114:0/2698255465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe56f8f0b0 msgr2=0x55fe56f99410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:19.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.863+0000 7fe9c1d9b640 1 --2- 172.21.15.114:0/2698255465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe56f8f0b0 0x55fe56f99410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.863+0000 7fe9c1d9b640 1 -- 172.21.15.114:0/2698255465 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe56f8e790 msgr2=0x55fe56f8eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:19.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.863+0000 7fe9c1d9b640 1 --2- 172.21.15.114:0/2698255465 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe56f8e790 0x55fe56f8eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.863+0000 7fe9c1d9b640 1 -- 172.21.15.114:0/2698255465 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe56eb9620 con 0x55fe56f99950 2024-09-13T18:25:19.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.864+0000 7fe9c1d9b640 1 --2- 172.21.15.114:0/2698255465 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe56f99950 0x55fe56f9bd40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe9b800b730 tx=0x7fe9b800bbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6526675763fd1425 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:19.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.864+0000 7fe9b0ff9640 1 -- 172.21.15.114:0/2698255465 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b800be40 con 0x55fe56f99950 2024-09-13T18:25:19.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.864+0000 7fe9b37fe640 1 --2- 172.21.15.114:0/4275991846 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe9ac0a1310 0x7fe9ac0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:19.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.865+0000 7fe9b37fe640 1 -- 172.21.15.114:0/4275991846 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fe9b40ceb30 con 0x7fe9c4106320 2024-09-13T18:25:19.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.866+0000 7fe9c2d9d640 1 --2- 172.21.15.114:0/4275991846 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe9ac0a1310 0x7fe9ac0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:19.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.866+0000 7fe9c259c640 1 --2- 172.21.15.114:0/2698255465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe56f8f0b0 0x55fe56f99410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:25:19.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.866+0000 7fe9b0ff9640 1 -- 172.21.15.114:0/2698255465 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe9b800dde0 con 0x55fe56f99950 2024-09-13T18:25:19.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.866+0000 7fe9b0ff9640 1 -- 172.21.15.114:0/2698255465 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b8014e80 con 0x55fe56f99950 2024-09-13T18:25:19.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.867+0000 7fe9cb7bf740 1 -- 172.21.15.114:0/2698255465 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe56f99950 msgr2=0x55fe56f9bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:19.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.867+0000 7fe9cb7bf740 1 --2- 172.21.15.114:0/2698255465 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe56f99950 0x55fe56f9bd40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe9b800b730 tx=0x7fe9b800bbf0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.867+0000 7fe9cb7bf740 1 -- 172.21.15.114:0/2698255465 shutdown_connections 2024-09-13T18:25:19.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.867+0000 7fe9cb7bf740 1 --2- 172.21.15.114:0/2698255465 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe56f99950 0x55fe56f9bd40 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.867+0000 7fe9cb7bf740 1 --2- 172.21.15.114:0/2698255465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe56f8f0b0 0x55fe56f99410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.867+0000 7fe9cb7bf740 1 --2- 172.21.15.114:0/2698255465 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe56f8e790 0x55fe56f8eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.867+0000 7fe9cb7bf740 1 -- 172.21.15.114:0/2698255465 >> 172.21.15.114:0/2698255465 conn(0x55fe56f8ad00 msgr2=0x55fe56f8b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:19.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.867+0000 7fe9c2d9d640 1 --2- 172.21.15.114:0/4275991846 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe9ac0a1310 0x7fe9ac0a37d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc006010 tx=0x7fe9bc03d040 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:19.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.867+0000 7fe9cb7bf740 1 -- 172.21.15.114:0/2698255465 shutdown_connections 2024-09-13T18:25:19.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.868+0000 7fe9cb7bf740 1 -- 172.21.15.114:0/2698255465 wait complete. 2024-09-13T18:25:19.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.868+0000 7fe9cb7bf740 1 -- start start 2024-09-13T18:25:19.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.869+0000 7fe9cb7bf740 1 Processor -- start 2024-09-13T18:25:19.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.873+0000 7fe9cb7bf740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe56f8e790 0x55fe56f98fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:19.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.873+0000 7fe9cb7bf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe56f8f0b0 0x55fe56f963c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:19.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.873+0000 7fe9cb7bf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe56f99950 0x55fe56f96900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:19.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.873+0000 7fe9cb7bf740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe56eb0180 con 0x55fe56f8e790 2024-09-13T18:25:19.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.873+0000 7fe9cb7bf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe56bca430 con 0x55fe56f99950 2024-09-13T18:25:19.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.873+0000 7fe9cb7bf740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe56e755e0 con 0x55fe56f8f0b0 2024-09-13T18:25:19.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.873+0000 7fe9cb7bf740 10 client.0 _ll_register_callbacks cb 0x55fe56ee1aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:25:19.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.874+0000 7fe9c1d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe56f8f0b0 0x55fe56f963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:19.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.874+0000 7fe9c1d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe56f8f0b0 0x55fe56f963c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:38132/0 (socket says 172.21.15.114:38132) 2024-09-13T18:25:19.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.874+0000 7fe9c1d9b640 1 -- 172.21.15.114:0/3713038387 learned_addr learned my addr 172.21.15.114:0/3713038387 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:19.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.874+0000 7fe9c2d9d640 1 --2- 172.21.15.114:0/3713038387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe56f99950 0x55fe56f96900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:19.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.875+0000 7fe9c1d9b640 1 -- 172.21.15.114:0/3713038387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe56f99950 msgr2=0x55fe56f96900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:19.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.875+0000 7fe9c1d9b640 1 --2- 172.21.15.114:0/3713038387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe56f99950 0x55fe56f96900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.875+0000 7fe9c1d9b640 1 -- 172.21.15.114:0/3713038387 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe56f8e790 msgr2=0x55fe56f98fe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:25:19.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.875+0000 7fe9c1d9b640 1 --2- 172.21.15.114:0/3713038387 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe56f8e790 0x55fe56f98fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.875+0000 7fe9c1d9b640 1 -- 172.21.15.114:0/3713038387 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9b800e040 con 0x55fe56f8f0b0 2024-09-13T18:25:19.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.875+0000 7fe9c1d9b640 1 --2- 172.21.15.114:0/3713038387 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe56f8f0b0 0x55fe56f963c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8014a00 tx=0x7fe9b8014660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e76fc567185f7e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:19.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.875+0000 7fe9a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:25:19.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.876+0000 7fe9a9ffb640 1 -- 172.21.15.114:0/3713038387 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b800db60 con 0x55fe56f8f0b0 2024-09-13T18:25:19.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.876+0000 7fe9a9ffb640 1 -- 172.21.15.114:0/3713038387 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fe9b801d420 con 0x55fe56f8f0b0 2024-09-13T18:25:19.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.876+0000 7fe9cb7bf740 1 -- 172.21.15.114:0/3713038387 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fe56f1bb80 con 0x55fe56f8f0b0 2024-09-13T18:25:19.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.876+0000 7fe9a9ffb640 1 -- 172.21.15.114:0/3713038387 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9b801d5e0 con 0x55fe56f8f0b0 2024-09-13T18:25:19.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.876+0000 7fe9cb7bf740 10 client.26017 fetch_fsmap learned FSMap version 32 2024-09-13T18:25:19.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.877+0000 7fe9cb7bf740 1 -- 172.21.15.114:0/3713038387 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55fe56ef03c0 con 0x55fe56f8f0b0 2024-09-13T18:25:19.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.878+0000 7fe9a9ffb640 1 -- 172.21.15.114:0/3713038387 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55fe56f1bb80 con 0x55fe56f8f0b0 2024-09-13T18:25:19.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.878+0000 7fe9a9ffb640 1 -- 172.21.15.114:0/3713038387 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fe9b802a070 con 0x55fe56f8f0b0 2024-09-13T18:25:19.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.878+0000 7fe9cb7bf740 10 client.26017 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:25:19.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.878+0000 7fe9cb7bf740 10 client.26017 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-13T18:25:19.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.879+0000 7fe9cb7bf740 20 client.26017 populate_metadata read hostname 'smithi114' 2024-09-13T18:25:19.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.879+0000 7fe9cb7bf740 1 --2- 172.21.15.114:0/3713038387 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55fe56fba100 0x55fe56fbc520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:19.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.879+0000 7fe9cb7bf740 4 client.26017 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:25:19.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.879+0000 7fe9cb7bf740 1 -- 172.21.15.114:0/3713038387 --> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fe56ee41f0 con 0x55fe56fba100 2024-09-13T18:25:19.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.879+0000 7fe9c259c640 1 --2- 172.21.15.114:0/3713038387 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55fe56fba100 0x55fe56fbc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:19.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.880+0000 7fe9a9ffb640 10 client.26017 ms_handle_connect on v2:172.21.15.134:6834/1150003275 2024-09-13T18:25:19.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.881+0000 7fe9c259c640 1 --2- 172.21.15.114:0/3713038387 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55fe56fba100 0x55fe56fbc520 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=95c4dce687f4334c server_cookie=51a76293c1d59523 in_seq=0 out_seq=0 2024-09-13T18:25:19.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.882+0000 7fe9a9ffb640 1 -- 172.21.15.114:0/3713038387 <== mds.3 v2:172.21.15.134:6834/1150003275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55fe56ee41f0 con 0x55fe56fba100 2024-09-13T18:25:19.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.882+0000 7fe9a9ffb640 10 client.26017 handle_command_reply: tid=0 2024-09-13T18:25:19.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.883+0000 7fe9cb7bf740 1 client.26017 shutdown 2024-09-13T18:25:19.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.884+0000 7fe9cb7bf740 1 -- 172.21.15.114:0/3713038387 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe56f8f0b0 msgr2=0x55fe56f963c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:19.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.884+0000 7fe9cb7bf740 1 --2- 172.21.15.114:0/3713038387 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe56f8f0b0 0x55fe56f963c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8014a00 tx=0x7fe9b8014660 comp rx=0 tx=0).stop 2024-09-13T18:25:19.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.884+0000 7fe9cb7bf740 1 -- 172.21.15.114:0/3713038387 shutdown_connections 2024-09-13T18:25:19.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.884+0000 7fe9cb7bf740 1 --2- 172.21.15.114:0/3713038387 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55fe56fba100 0x55fe56fbc520 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.884+0000 7fe9cb7bf740 1 --2- 172.21.15.114:0/3713038387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe56f99950 0x55fe56f96900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.884+0000 7fe9cb7bf740 1 --2- 172.21.15.114:0/3713038387 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe56f8f0b0 0x55fe56f963c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.884+0000 7fe9cb7bf740 1 --2- 172.21.15.114:0/3713038387 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe56f8e790 0x55fe56f98fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.884+0000 7fe9cb7bf740 1 -- 172.21.15.114:0/3713038387 >> 172.21.15.114:0/3713038387 conn(0x55fe56f8ad00 msgr2=0x55fe56f90a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:19.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.885+0000 7fe9a9ffb640 0 client.26017 ms_handle_reset on v2:172.21.15.134:6834/1150003275 2024-09-13T18:25:19.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.885+0000 7fe9cb7bf740 1 -- 172.21.15.114:0/3713038387 shutdown_connections 2024-09-13T18:25:19.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.885+0000 7fe9cb7bf740 1 -- 172.21.15.114:0/3713038387 wait complete. 2024-09-13T18:25:19.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.885+0000 7fe9cb7bf740 20 client.26017 trim_cache size 0 max 16384 2024-09-13T18:25:19.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.904+0000 7fe9cb7bf740 1 Processor -- start 2024-09-13T18:25:19.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.904+0000 7fe9cb7bf740 1 -- start start 2024-09-13T18:25:19.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.905+0000 7fe9cb7bf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe56f8e790 0x55fe56fb1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:19.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.905+0000 7fe9cb7bf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe56f8f0b0 0x55fe56fb1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:19.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.905+0000 7fe9cb7bf740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe56f99950 0x55fe56f98a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:19.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.905+0000 7fe9cb7bf740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe56fb9f30 con 0x55fe56f99950 2024-09-13T18:25:19.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.905+0000 7fe9c2d9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe56f99950 0x55fe56f98a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:19.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.905+0000 7fe9c2d9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe56f99950 0x55fe56f98a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43910/0 (socket says 172.21.15.114:43910) 2024-09-13T18:25:19.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.905+0000 7fe9c2d9d640 1 -- 172.21.15.114:0/3145731782 learned_addr learned my addr 172.21.15.114:0/3145731782 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:19.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.905+0000 7fe9cb7bf740 1 -- 172.21.15.114:0/3145731782 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe56fbc8e0 con 0x55fe56f8f0b0 2024-09-13T18:25:19.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.905+0000 7fe9cb7bf740 1 -- 172.21.15.114:0/3145731782 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe56fad410 con 0x55fe56f8e790 2024-09-13T18:25:19.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.906+0000 7fe9c1d9b640 1 --2- 172.21.15.114:0/3145731782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe56f8f0b0 0x55fe56fb1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:19.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.906+0000 7fe9c2d9d640 1 -- 172.21.15.114:0/3145731782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe56f8e790 msgr2=0x55fe56fb1650 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:25:19.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.906+0000 7fe9c2d9d640 1 --2- 172.21.15.114:0/3145731782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe56f8e790 0x55fe56fb1650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.906+0000 7fe9c2d9d640 1 -- 172.21.15.114:0/3145731782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe56f8f0b0 msgr2=0x55fe56fb1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:19.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.906+0000 7fe9c2d9d640 1 --2- 172.21.15.114:0/3145731782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe56f8f0b0 0x55fe56fb1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.906+0000 7fe9c2d9d640 1 -- 172.21.15.114:0/3145731782 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe56e92770 con 0x55fe56f99950 2024-09-13T18:25:19.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.906+0000 7fe9c1d9b640 1 --2- 172.21.15.114:0/3145731782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe56f8f0b0 0x55fe56fb1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:25:19.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.906+0000 7fe9c2d9d640 1 --2- 172.21.15.114:0/3145731782 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe56f99950 0x55fe56f98a80 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc02aec0 tx=0x7fe9bc041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=260c3894c07d02c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:19.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.907+0000 7fe9b17fa640 1 -- 172.21.15.114:0/3145731782 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc04d070 con 0x55fe56f99950 2024-09-13T18:25:19.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.907+0000 7fe9b17fa640 1 -- 172.21.15.114:0/3145731782 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe9bc02f930 con 0x55fe56f99950 2024-09-13T18:25:19.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.907+0000 7fe9b17fa640 1 -- 172.21.15.114:0/3145731782 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc040040 con 0x55fe56f99950 2024-09-13T18:25:19.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.908+0000 7fe9cb7bf740 1 -- 172.21.15.114:0/3145731782 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe56f99950 msgr2=0x55fe56f98a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:19.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.908+0000 7fe9cb7bf740 1 --2- 172.21.15.114:0/3145731782 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe56f99950 0x55fe56f98a80 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc02aec0 tx=0x7fe9bc041000 comp rx=0 tx=0).stop 2024-09-13T18:25:19.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.909+0000 7fe9cb7bf740 1 -- 172.21.15.114:0/3145731782 shutdown_connections 2024-09-13T18:25:19.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.909+0000 7fe9cb7bf740 1 --2- 172.21.15.114:0/3145731782 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe56f99950 0x55fe56f98a80 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.909+0000 7fe9cb7bf740 1 --2- 172.21.15.114:0/3145731782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe56f8f0b0 0x55fe56fb1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.909+0000 7fe9cb7bf740 1 --2- 172.21.15.114:0/3145731782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe56f8e790 0x55fe56fb1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.909+0000 7fe9cb7bf740 1 -- 172.21.15.114:0/3145731782 >> 172.21.15.114:0/3145731782 conn(0x55fe56f8ad00 msgr2=0x55fe56f94c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:19.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.911+0000 7fe9cb7bf740 1 -- 172.21.15.114:0/3145731782 shutdown_connections 2024-09-13T18:25:19.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.911+0000 7fe9cb7bf740 1 -- 172.21.15.114:0/3145731782 wait complete. 2024-09-13T18:25:19.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.911+0000 7fe9cb7bf740 1 -- start start 2024-09-13T18:25:19.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.911+0000 7fe9cb7bf740 1 Processor -- start 2024-09-13T18:25:19.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.912+0000 7fe9cb7bf740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe56f8e790 0x55fe56faddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:19.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.912+0000 7fe9cb7bf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe56f8f0b0 0x55fe56fae300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:19.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.912+0000 7fe9cb7bf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe56f99950 0x55fe570085f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:19.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.912+0000 7fe9cb7bf740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe57008c00 con 0x55fe56f8e790 2024-09-13T18:25:19.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.912+0000 7fe9cb7bf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe57008d70 con 0x55fe56f8f0b0 2024-09-13T18:25:19.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.912+0000 7fe9cb7bf740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe57008f30 con 0x55fe56f99950 2024-09-13T18:25:19.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.912+0000 7fe9cb7bf740 10 client.0 _ll_register_callbacks cb 0x7fe9b800b180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:25:19.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.912+0000 7fe9c2d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe56f99950 0x55fe570085f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:19.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.912+0000 7fe9c2d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe56f99950 0x55fe570085f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:38150/0 (socket says 172.21.15.114:38150) 2024-09-13T18:25:19.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.912+0000 7fe9c2d9d640 1 -- 172.21.15.114:0/2932619742 learned_addr learned my addr 172.21.15.114:0/2932619742 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:19.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.913+0000 7fe9c2d9d640 1 -- 172.21.15.114:0/2932619742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe56f8f0b0 msgr2=0x55fe56fae300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:25:19.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.913+0000 7fe9c2d9d640 1 --2- 172.21.15.114:0/2932619742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe56f8f0b0 0x55fe56fae300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.913+0000 7fe9c2d9d640 1 -- 172.21.15.114:0/2932619742 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe56f8e790 msgr2=0x55fe56faddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:19.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.913+0000 7fe9c2d9d640 1 --2- 172.21.15.114:0/2932619742 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe56f8e790 0x55fe56faddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.913+0000 7fe9c2d9d640 1 -- 172.21.15.114:0/2932619742 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9c4113a80 con 0x55fe56f99950 2024-09-13T18:25:19.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.915+0000 7fe9c2d9d640 1 --2- 172.21.15.114:0/2932619742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe56f99950 0x55fe570085f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc009430 tx=0x7fe9bc02ae50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d79139c1208be265 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:19.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.915+0000 7fe9a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:25:19.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.915+0000 7fe9a9ffb640 1 -- 172.21.15.114:0/2932619742 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc04d040 con 0x55fe56f99950 2024-09-13T18:25:19.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.915+0000 7fe9a9ffb640 1 -- 172.21.15.114:0/2932619742 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fe9bc003e60 con 0x55fe56f99950 2024-09-13T18:25:19.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.916+0000 7fe9cb7bf740 1 -- 172.21.15.114:0/2932619742 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe9b800dde0 con 0x55fe56f99950 2024-09-13T18:25:19.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.916+0000 7fe9a9ffb640 1 -- 172.21.15.114:0/2932619742 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9bc030440 con 0x55fe56f99950 2024-09-13T18:25:19.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.916+0000 7fe9cb7bf740 10 client.26020 fetch_fsmap learned FSMap version 32 2024-09-13T18:25:19.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.916+0000 7fe9cb7bf740 1 -- 172.21.15.114:0/2932619742 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55fe56eb7e80 con 0x55fe56f99950 2024-09-13T18:25:19.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.916+0000 7fe9a9ffb640 1 -- 172.21.15.114:0/2932619742 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe9bc009b20 con 0x55fe56f99950 2024-09-13T18:25:19.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.916+0000 7fe9a9ffb640 1 -- 172.21.15.114:0/2932619742 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fe9bc03b870 con 0x55fe56f99950 2024-09-13T18:25:19.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.917+0000 7fe9cb7bf740 10 client.26020 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:25:19.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.917+0000 7fe9cb7bf740 10 client.26020 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-13T18:25:19.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.917+0000 7fe9cb7bf740 20 client.26020 populate_metadata read hostname 'smithi114' 2024-09-13T18:25:19.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.917+0000 7fe9cb7bf740 1 --2- 172.21.15.114:0/2932619742 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55fe570115f0 0x55fe57013a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:19.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.917+0000 7fe9cb7bf740 4 client.26020 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:25:19.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.917+0000 7fe9cb7bf740 1 -- 172.21.15.114:0/2932619742 --> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fe9b800dde0 con 0x55fe570115f0 2024-09-13T18:25:19.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.918+0000 7fe9c259c640 1 --2- 172.21.15.114:0/2932619742 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55fe570115f0 0x55fe57013a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:19.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.920+0000 7fe9c259c640 1 --2- 172.21.15.114:0/2932619742 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55fe570115f0 0x55fe57013a10 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8783644768507a4 server_cookie=a40c0ea416fcdcde in_seq=0 out_seq=0 2024-09-13T18:25:19.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.920+0000 7fe9a9ffb640 10 client.26020 ms_handle_connect on v2:172.21.15.134:6834/1150003275 2024-09-13T18:25:19.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.920+0000 7fe9a9ffb640 1 -- 172.21.15.114:0/2932619742 <== mds.3 v2:172.21.15.134:6834/1150003275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fe9b800dde0 con 0x55fe570115f0 2024-09-13T18:25:19.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.920+0000 7fe9a9ffb640 10 client.26020 handle_command_reply: tid=0 2024-09-13T18:25:19.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.920+0000 7fe9cb7bf740 1 client.26020 shutdown 2024-09-13T18:25:19.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.923+0000 7fe9cb7bf740 1 -- 172.21.15.114:0/2932619742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe56f99950 msgr2=0x55fe570085f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:19.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.923+0000 7fe9cb7bf740 1 --2- 172.21.15.114:0/2932619742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe56f99950 0x55fe570085f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc009430 tx=0x7fe9bc02ae50 comp rx=0 tx=0).stop 2024-09-13T18:25:19.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.923+0000 7fe9cb7bf740 1 -- 172.21.15.114:0/2932619742 shutdown_connections 2024-09-13T18:25:19.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.923+0000 7fe9cb7bf740 1 --2- 172.21.15.114:0/2932619742 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55fe570115f0 0x55fe57013a10 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.923+0000 7fe9cb7bf740 1 --2- 172.21.15.114:0/2932619742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe56f99950 0x55fe570085f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.923+0000 7fe9cb7bf740 1 --2- 172.21.15.114:0/2932619742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe56f8f0b0 0x55fe56fae300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.923+0000 7fe9cb7bf740 1 --2- 172.21.15.114:0/2932619742 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe56f8e790 0x55fe56faddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.923+0000 7fe9cb7bf740 1 -- 172.21.15.114:0/2932619742 >> 172.21.15.114:0/2932619742 conn(0x55fe56f8ad00 msgr2=0x55fe56f89690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:19.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.923+0000 7fe9a9ffb640 0 client.26020 ms_handle_reset on v2:172.21.15.134:6834/1150003275 2024-09-13T18:25:19.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.924+0000 7fe9cb7bf740 1 -- 172.21.15.114:0/2932619742 shutdown_connections 2024-09-13T18:25:19.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.925+0000 7fe9cb7bf740 1 -- 172.21.15.114:0/2932619742 wait complete. 2024-09-13T18:25:19.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.925+0000 7fe9cb7bf740 20 client.26020 trim_cache size 0 max 16384 2024-09-13T18:25:19.927 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:25:19.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.926+0000 7fe9c89c3640 1 -- 172.21.15.114:0/4275991846 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe9ac0a1310 msgr2=0x7fe9ac0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:19.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.926+0000 7fe9c89c3640 1 --2- 172.21.15.114:0/4275991846 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe9ac0a1310 0x7fe9ac0a37d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc006010 tx=0x7fe9bc03d040 comp rx=0 tx=0).stop 2024-09-13T18:25:19.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.926+0000 7fe9c89c3640 1 -- 172.21.15.114:0/4275991846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9c4106320 msgr2=0x7fe9c4080970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:19.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.926+0000 7fe9c89c3640 1 --2- 172.21.15.114:0/4275991846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9c4106320 0x7fe9c4080970 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe9b400d950 tx=0x7fe9b400de10 comp rx=0 tx=0).stop 2024-09-13T18:25:19.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.927+0000 7fe9c89c3640 1 -- 172.21.15.114:0/4275991846 shutdown_connections 2024-09-13T18:25:19.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.927+0000 7fe9c89c3640 1 --2- 172.21.15.114:0/4275991846 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe9ac0a1310 0x7fe9ac0a37d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.927+0000 7fe9c89c3640 1 --2- 172.21.15.114:0/4275991846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9c410c3e0 0x7fe9c4079b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.927+0000 7fe9c89c3640 1 --2- 172.21.15.114:0/4275991846 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe9c4106ca0 0x7fe9c4080ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.927+0000 7fe9c89c3640 1 --2- 172.21.15.114:0/4275991846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9c4106320 0x7fe9c4080970 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:19.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.927+0000 7fe9c89c3640 1 -- 172.21.15.114:0/4275991846 >> 172.21.15.114:0/4275991846 conn(0x7fe9c4075c60 msgr2=0x7fe9c4077660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:19.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.928+0000 7fe9c89c3640 1 -- 172.21.15.114:0/4275991846 shutdown_connections 2024-09-13T18:25:19.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:19.928+0000 7fe9c89c3640 1 -- 172.21.15.114:0/4275991846 wait complete. 2024-09-13T18:25:19.946 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:19 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/2746027394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:25:19.955 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-09-13T18:25:20.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.130+0000 7fcd8a4b6640 1 -- 172.21.15.114:0/3790110606 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd84105010 msgr2=0x7fcd8410cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:20.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.130+0000 7fcd8a4b6640 1 --2- 172.21.15.114:0/3790110606 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd84105010 0x7fcd8410cb60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c009c10 tx=0x7fcd7c02a6e0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.130+0000 7fcd8a4b6640 1 -- 172.21.15.114:0/3790110606 shutdown_connections 2024-09-13T18:25:20.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.130+0000 7fcd8a4b6640 1 --2- 172.21.15.114:0/3790110606 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd84105010 0x7fcd8410cb60 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.130+0000 7fcd8a4b6640 1 --2- 172.21.15.114:0/3790110606 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd84100870 0x7fcd841048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.130+0000 7fcd8a4b6640 1 --2- 172.21.15.114:0/3790110606 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd840ffec0 0x7fcd841002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.130+0000 7fcd8a4b6640 1 -- 172.21.15.114:0/3790110606 >> 172.21.15.114:0/3790110606 conn(0x7fcd840fb950 msgr2=0x7fcd840fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:20.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.131+0000 7fcd8a4b6640 1 -- 172.21.15.114:0/3790110606 shutdown_connections 2024-09-13T18:25:20.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.131+0000 7fcd8a4b6640 1 -- 172.21.15.114:0/3790110606 wait complete. 2024-09-13T18:25:20.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.131+0000 7fcd8a4b6640 1 Processor -- start 2024-09-13T18:25:20.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.132+0000 7fcd8a4b6640 1 -- start start 2024-09-13T18:25:20.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.132+0000 7fcd8a4b6640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd840ffec0 0x7fcd8406bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:20.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.132+0000 7fcd8a4b6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd84105010 0x7fcd8406c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:20.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.132+0000 7fcd8a4b6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd84070470 0x7fcd84070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:20.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.132+0000 7fcd8a4b6640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd84070e60 con 0x7fcd840ffec0 2024-09-13T18:25:20.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.132+0000 7fcd8a4b6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd84070fd0 con 0x7fcd84070470 2024-09-13T18:25:20.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.132+0000 7fcd8a4b6640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd84071190 con 0x7fcd84105010 2024-09-13T18:25:20.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.132+0000 7fcd89cb5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd84070470 0x7fcd84070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:20.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.132+0000 7fcd89cb5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd84070470 0x7fcd84070920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:43808/0 (socket says 172.21.15.114:43808) 2024-09-13T18:25:20.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.132+0000 7fcd89cb5640 1 -- 172.21.15.114:0/670455956 learned_addr learned my addr 172.21.15.114:0/670455956 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:20.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.133+0000 7fcd88cb3640 1 --2- 172.21.15.114:0/670455956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd84105010 0x7fcd8406c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:20.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.133+0000 7fcd894b4640 1 --2- 172.21.15.114:0/670455956 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd840ffec0 0x7fcd8406bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:20.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.133+0000 7fcd89cb5640 1 -- 172.21.15.114:0/670455956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd84105010 msgr2=0x7fcd8406c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:20.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.133+0000 7fcd89cb5640 1 --2- 172.21.15.114:0/670455956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd84105010 0x7fcd8406c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.133+0000 7fcd89cb5640 1 -- 172.21.15.114:0/670455956 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd840ffec0 msgr2=0x7fcd8406bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:20.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.133+0000 7fcd89cb5640 1 --2- 172.21.15.114:0/670455956 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd840ffec0 0x7fcd8406bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.133+0000 7fcd89cb5640 1 -- 172.21.15.114:0/670455956 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd7c009840 con 0x7fcd84070470 2024-09-13T18:25:20.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.135+0000 7fcd89cb5640 1 --2- 172.21.15.114:0/670455956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd84070470 0x7fcd84070920 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c009560 tx=0x7fcd7c003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:20.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.136+0000 7fcd7a7fc640 1 -- 172.21.15.114:0/670455956 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c004040 con 0x7fcd84070470 2024-09-13T18:25:20.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.136+0000 7fcd7a7fc640 1 -- 172.21.15.114:0/670455956 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcd7c004630 con 0x7fcd84070470 2024-09-13T18:25:20.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.136+0000 7fcd7a7fc640 1 -- 172.21.15.114:0/670455956 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c031be0 con 0x7fcd84070470 2024-09-13T18:25:20.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.136+0000 7fcd8a4b6640 1 -- 172.21.15.114:0/670455956 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd841ba340 con 0x7fcd84070470 2024-09-13T18:25:20.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.137+0000 7fcd8a4b6640 1 -- 172.21.15.114:0/670455956 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd841ba6b0 con 0x7fcd84070470 2024-09-13T18:25:20.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.141+0000 7fcd7a7fc640 1 -- 172.21.15.114:0/670455956 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fcd7c0041a0 con 0x7fcd84070470 2024-09-13T18:25:20.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.141+0000 7fcd8e437740 1 Processor -- start 2024-09-13T18:25:20.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.141+0000 7fcd8e437740 1 -- start start 2024-09-13T18:25:20.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.141+0000 7fcd8e437740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f063546970 0x55f063546d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:20.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.141+0000 7fcd8e437740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f063547290 0x55f0635515f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:20.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.142+0000 7fcd8e437740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f063551b30 0x55f063553f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:20.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.142+0000 7fcd8e437740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0634b0cc0 con 0x55f063547290 2024-09-13T18:25:20.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.142+0000 7fcd8e437740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0634ae450 con 0x55f063551b30 2024-09-13T18:25:20.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.142+0000 7fcd8e437740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0634a9d40 con 0x55f063546970 2024-09-13T18:25:20.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.142+0000 7fcd88cb3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f063546970 0x55f063546d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:20.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.142+0000 7fcd88cb3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f063546970 0x55f063546d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:38170/0 (socket says 172.21.15.114:38170) 2024-09-13T18:25:20.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.142+0000 7fcd88cb3640 1 -- 172.21.15.114:0/202664624 learned_addr learned my addr 172.21.15.114:0/202664624 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:20.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.142+0000 7fcd88cb3640 1 --2- 172.21.15.114:0/202664624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f063551b30 0x55f063553f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:20.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.142+0000 7fcd88cb3640 1 -- 172.21.15.114:0/202664624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f063551b30 msgr2=0x55f063553f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:20.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.142+0000 7fcd88cb3640 1 --2- 172.21.15.114:0/202664624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f063551b30 0x55f063553f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.142+0000 7fcd88cb3640 1 -- 172.21.15.114:0/202664624 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f063547290 msgr2=0x55f0635515f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:20.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.142+0000 7fcd88cb3640 1 --2- 172.21.15.114:0/202664624 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f063547290 0x55f0635515f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.143+0000 7fcd88cb3640 1 -- 172.21.15.114:0/202664624 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f063471620 con 0x55f063546970 2024-09-13T18:25:20.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.143+0000 7fcd7a7fc640 1 --2- 172.21.15.114:0/670455956 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fcd700a1410 0x7fcd700a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:20.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.143+0000 7fcd88cb3640 1 --2- 172.21.15.114:0/202664624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f063546970 0x55f063546d50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcd74009730 tx=0x7fcd740096c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b1b8d476f44d7d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:20.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.143+0000 7fcd894b4640 1 --2- 172.21.15.114:0/670455956 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fcd700a1410 0x7fcd700a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:20.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.143+0000 7fcd7a7fc640 1 -- 172.21.15.114:0/670455956 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fcd7c0eda60 con 0x7fcd84070470 2024-09-13T18:25:20.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.143+0000 7fcd6f7fe640 1 -- 172.21.15.114:0/202664624 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd74004030 con 0x55f063546970 2024-09-13T18:25:20.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.143+0000 7fcd6f7fe640 1 -- 172.21.15.114:0/202664624 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcd74005020 con 0x55f063546970 2024-09-13T18:25:20.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.144+0000 7fcd6f7fe640 1 -- 172.21.15.114:0/202664624 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd74004a00 con 0x55f063546970 2024-09-13T18:25:20.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.144+0000 7fcd8e437740 1 -- 172.21.15.114:0/202664624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f063546970 msgr2=0x55f063546d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:20.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.144+0000 7fcd8e437740 1 --2- 172.21.15.114:0/202664624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f063546970 0x55f063546d50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcd74009730 tx=0x7fcd740096c0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.144+0000 7fcd8e437740 1 -- 172.21.15.114:0/202664624 shutdown_connections 2024-09-13T18:25:20.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.144+0000 7fcd8e437740 1 --2- 172.21.15.114:0/202664624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f063551b30 0x55f063553f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.144+0000 7fcd8e437740 1 --2- 172.21.15.114:0/202664624 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f063547290 0x55f0635515f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.144+0000 7fcd8e437740 1 --2- 172.21.15.114:0/202664624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f063546970 0x55f063546d50 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.144+0000 7fcd8e437740 1 -- 172.21.15.114:0/202664624 >> 172.21.15.114:0/202664624 conn(0x55f063542ee0 msgr2=0x55f0635432c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:20.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.145+0000 7fcd894b4640 1 --2- 172.21.15.114:0/670455956 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fcd700a1410 0x7fcd700a38d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x55f063543d40 tx=0x7fcd80004180 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:20.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.145+0000 7fcd8e437740 1 -- 172.21.15.114:0/202664624 shutdown_connections 2024-09-13T18:25:20.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.145+0000 7fcd8e437740 1 -- 172.21.15.114:0/202664624 wait complete. 2024-09-13T18:25:20.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.145+0000 7fcd8e437740 1 -- start start 2024-09-13T18:25:20.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.145+0000 7fcd8e437740 1 Processor -- start 2024-09-13T18:25:20.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.146+0000 7fcd8e437740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f063546970 0x55f063551130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:20.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.146+0000 7fcd8e437740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f063547290 0x55f06354e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:20.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.146+0000 7fcd8e437740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f063551b30 0x55f06354e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:20.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.146+0000 7fcd8e437740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f063468180 con 0x55f063551b30 2024-09-13T18:25:20.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.146+0000 7fcd8e437740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f063182430 con 0x55f063547290 2024-09-13T18:25:20.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.146+0000 7fcd8e437740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f06342d5e0 con 0x55f063546970 2024-09-13T18:25:20.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.146+0000 7fcd8e437740 10 client.0 _ll_register_callbacks cb 0x55f063499aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:25:20.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.146+0000 7fcd88cb3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f063547290 0x55f06354e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:20.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.146+0000 7fcd88cb3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f063547290 0x55f06354e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:43816/0 (socket says 172.21.15.114:43816) 2024-09-13T18:25:20.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.146+0000 7fcd88cb3640 1 -- 172.21.15.114:0/8256267 learned_addr learned my addr 172.21.15.114:0/8256267 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:20.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.147+0000 7fcd89cb5640 1 --2- 172.21.15.114:0/8256267 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f063551b30 0x55f06354e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:20.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.147+0000 7fcd894b4640 1 --2- 172.21.15.114:0/8256267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f063546970 0x55f063551130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:20.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.147+0000 7fcd88cb3640 1 -- 172.21.15.114:0/8256267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f063546970 msgr2=0x55f063551130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:20.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.147+0000 7fcd88cb3640 1 --2- 172.21.15.114:0/8256267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f063546970 0x55f063551130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.147+0000 7fcd88cb3640 1 -- 172.21.15.114:0/8256267 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f063551b30 msgr2=0x55f06354e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:20.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.147+0000 7fcd88cb3640 1 --2- 172.21.15.114:0/8256267 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f063551b30 0x55f06354e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.147+0000 7fcd88cb3640 1 -- 172.21.15.114:0/8256267 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd74019040 con 0x55f063547290 2024-09-13T18:25:20.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.147+0000 7fcd88cb3640 1 --2- 172.21.15.114:0/8256267 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f063547290 0x55f06354e4a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcd74005bc0 tx=0x7fcd7400f7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8785096ac0488ed6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:20.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.147+0000 7fcd6cff9640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:25:20.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.148+0000 7fcd6cff9640 1 -- 172.21.15.114:0/8256267 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd74024070 con 0x55f063547290 2024-09-13T18:25:20.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.148+0000 7fcd6cff9640 1 -- 172.21.15.114:0/8256267 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fcd7400fb70 con 0x55f063547290 2024-09-13T18:25:20.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.148+0000 7fcd8e437740 1 -- 172.21.15.114:0/8256267 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f0634d3b80 con 0x55f063547290 2024-09-13T18:25:20.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.148+0000 7fcd6cff9640 1 -- 172.21.15.114:0/8256267 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd7400fd30 con 0x55f063547290 2024-09-13T18:25:20.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.148+0000 7fcd8e437740 10 client.25994 fetch_fsmap learned FSMap version 32 2024-09-13T18:25:20.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.148+0000 7fcd8e437740 1 -- 172.21.15.114:0/8256267 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55f0634a83c0 con 0x55f063547290 2024-09-13T18:25:20.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.149+0000 7fcd6cff9640 1 -- 172.21.15.114:0/8256267 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f0634d3b80 con 0x55f063547290 2024-09-13T18:25:20.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.149+0000 7fcd6cff9640 1 -- 172.21.15.114:0/8256267 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fcd7401d070 con 0x55f063547290 2024-09-13T18:25:20.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.149+0000 7fcd8e437740 10 client.25994 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:25:20.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.149+0000 7fcd8e437740 10 client.25994 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-13T18:25:20.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.149+0000 7fcd8e437740 20 client.25994 populate_metadata read hostname 'smithi114' 2024-09-13T18:25:20.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.149+0000 7fcd8e437740 1 --2- 172.21.15.114:0/8256267 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55f0635613b0 0x55f0635637d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:20.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.149+0000 7fcd8e437740 4 client.25994 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:25:20.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.149+0000 7fcd8e437740 1 -- 172.21.15.114:0/8256267 --> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f06349c1f0 con 0x55f0635613b0 2024-09-13T18:25:20.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.150+0000 7fcd894b4640 1 --2- 172.21.15.114:0/8256267 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55f0635613b0 0x55f0635637d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:20.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.150+0000 7fcd894b4640 1 --2- 172.21.15.114:0/8256267 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55f0635613b0 0x55f0635637d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7fe3d62d0e76fd15 server_cookie=8cc6b9696254f0f1 in_seq=0 out_seq=0 2024-09-13T18:25:20.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.150+0000 7fcd6cff9640 10 client.25994 ms_handle_connect on v2:172.21.15.204:6838/344883912 2024-09-13T18:25:20.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.151+0000 7fcd6cff9640 1 -- 172.21.15.114:0/8256267 <== mds.2 v2:172.21.15.204:6838/344883912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55f06349c1f0 con 0x55f0635613b0 2024-09-13T18:25:20.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.151+0000 7fcd6cff9640 10 client.25994 handle_command_reply: tid=0 2024-09-13T18:25:20.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.151+0000 7fcd8e437740 1 client.25994 shutdown 2024-09-13T18:25:20.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.151+0000 7fcd8e437740 1 -- 172.21.15.114:0/8256267 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f063547290 msgr2=0x55f06354e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:20.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.151+0000 7fcd8e437740 1 --2- 172.21.15.114:0/8256267 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f063547290 0x55f06354e4a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcd74005bc0 tx=0x7fcd7400f7c0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.152+0000 7fcd8e437740 1 -- 172.21.15.114:0/8256267 shutdown_connections 2024-09-13T18:25:20.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.152+0000 7fcd8e437740 1 --2- 172.21.15.114:0/8256267 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55f0635613b0 0x55f0635637d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.152+0000 7fcd8e437740 1 --2- 172.21.15.114:0/8256267 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f063551b30 0x55f06354e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.152+0000 7fcd8e437740 1 --2- 172.21.15.114:0/8256267 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f063547290 0x55f06354e4a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.152+0000 7fcd8e437740 1 --2- 172.21.15.114:0/8256267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f063546970 0x55f063551130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.152+0000 7fcd8e437740 1 -- 172.21.15.114:0/8256267 >> 172.21.15.114:0/8256267 conn(0x55f063542ee0 msgr2=0x55f063542230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:20.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.152+0000 7fcd6cff9640 0 client.25994 ms_handle_reset on v2:172.21.15.204:6838/344883912 2024-09-13T18:25:20.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.152+0000 7fcd8e437740 1 -- 172.21.15.114:0/8256267 shutdown_connections 2024-09-13T18:25:20.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.152+0000 7fcd8e437740 1 -- 172.21.15.114:0/8256267 wait complete. 2024-09-13T18:25:20.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.153+0000 7fcd8e437740 20 client.25994 trim_cache size 0 max 16384 2024-09-13T18:25:20.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.172+0000 7fcd8e437740 1 Processor -- start 2024-09-13T18:25:20.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.172+0000 7fcd8e437740 1 -- start start 2024-09-13T18:25:20.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.172+0000 7fcd8e437740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f063546970 0x55f0635633c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:20.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.172+0000 7fcd8e437740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f063547290 0x55f063558890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:20.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.172+0000 7fcd8e437740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f063551b30 0x55f063558dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:20.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.172+0000 7fcd8e437740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f063547b10 con 0x55f063547290 2024-09-13T18:25:20.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.172+0000 7fcd8e437740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f063563900 con 0x55f063546970 2024-09-13T18:25:20.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.172+0000 7fcd8e437740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0635519b0 con 0x55f063551b30 2024-09-13T18:25:20.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.173+0000 7fcd89cb5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f063551b30 0x55f063558dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:20.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.173+0000 7fcd89cb5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f063551b30 0x55f063558dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:38190/0 (socket says 172.21.15.114:38190) 2024-09-13T18:25:20.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.173+0000 7fcd89cb5640 1 -- 172.21.15.114:0/3716456678 learned_addr learned my addr 172.21.15.114:0/3716456678 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:20.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.173+0000 7fcd88cb3640 1 --2- 172.21.15.114:0/3716456678 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f063547290 0x55f063558890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:20.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.173+0000 7fcd894b4640 1 --2- 172.21.15.114:0/3716456678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f063546970 0x55f0635633c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:20.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.173+0000 7fcd89cb5640 1 -- 172.21.15.114:0/3716456678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f063546970 msgr2=0x55f0635633c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:20.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.173+0000 7fcd89cb5640 1 --2- 172.21.15.114:0/3716456678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f063546970 0x55f0635633c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.173+0000 7fcd89cb5640 1 -- 172.21.15.114:0/3716456678 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f063547290 msgr2=0x55f063558890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:20.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.173+0000 7fcd89cb5640 1 --2- 172.21.15.114:0/3716456678 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f063547290 0x55f063558890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.173+0000 7fcd89cb5640 1 -- 172.21.15.114:0/3716456678 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f06344a770 con 0x55f063551b30 2024-09-13T18:25:20.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.174+0000 7fcd89cb5640 1 --2- 172.21.15.114:0/3716456678 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f063551b30 0x55f063558dd0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c0b6d70 tx=0x7fcd7c0ba290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=290cff1f044f495e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:20.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.174+0000 7fcd6ffff640 1 -- 172.21.15.114:0/3716456678 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c0be5f0 con 0x55f063551b30 2024-09-13T18:25:20.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.174+0000 7fcd6ffff640 1 -- 172.21.15.114:0/3716456678 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcd7c0c2c40 con 0x55f063551b30 2024-09-13T18:25:20.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.174+0000 7fcd6ffff640 1 -- 172.21.15.114:0/3716456678 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c0b6f80 con 0x55f063551b30 2024-09-13T18:25:20.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.176+0000 7fcd8e437740 1 -- 172.21.15.114:0/3716456678 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f063551b30 msgr2=0x55f063558dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:20.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.176+0000 7fcd8e437740 1 --2- 172.21.15.114:0/3716456678 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f063551b30 0x55f063558dd0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c0b6d70 tx=0x7fcd7c0ba290 comp rx=0 tx=0).stop 2024-09-13T18:25:20.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.177+0000 7fcd8e437740 1 -- 172.21.15.114:0/3716456678 shutdown_connections 2024-09-13T18:25:20.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.177+0000 7fcd8e437740 1 --2- 172.21.15.114:0/3716456678 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f063551b30 0x55f063558dd0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.177+0000 7fcd8e437740 1 --2- 172.21.15.114:0/3716456678 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f063547290 0x55f063558890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.177+0000 7fcd8e437740 1 --2- 172.21.15.114:0/3716456678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f063546970 0x55f0635633c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.177+0000 7fcd8e437740 1 -- 172.21.15.114:0/3716456678 >> 172.21.15.114:0/3716456678 conn(0x55f063542ee0 msgr2=0x55f06355d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:20.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.178+0000 7fcd8e437740 1 -- 172.21.15.114:0/3716456678 shutdown_connections 2024-09-13T18:25:20.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.178+0000 7fcd8e437740 1 -- 172.21.15.114:0/3716456678 wait complete. 2024-09-13T18:25:20.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.179+0000 7fcd8e437740 1 -- start start 2024-09-13T18:25:20.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.179+0000 7fcd8e437740 1 Processor -- start 2024-09-13T18:25:20.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.180+0000 7fcd8e437740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f063546970 0x55f063555290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:20.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.180+0000 7fcd8e437740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f063547290 0x55f0635557d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:20.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.180+0000 7fcd8e437740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f063551b30 0x55f063555d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:20.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.180+0000 7fcd8e437740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0635591b0 con 0x55f063551b30 2024-09-13T18:25:20.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.180+0000 7fcd8e437740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0635b78b0 con 0x55f063546970 2024-09-13T18:25:20.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.180+0000 7fcd8e437740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0635b79f0 con 0x55f063547290 2024-09-13T18:25:20.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.180+0000 7fcd8e437740 10 client.0 _ll_register_callbacks cb 0x7fcd740099d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:25:20.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.181+0000 7fcd89cb5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f063551b30 0x55f063555d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:20.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.181+0000 7fcd89cb5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f063551b30 0x55f063555d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43968/0 (socket says 172.21.15.114:43968) 2024-09-13T18:25:20.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.181+0000 7fcd89cb5640 1 -- 172.21.15.114:0/2823587953 learned_addr learned my addr 172.21.15.114:0/2823587953 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:20.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.181+0000 7fcd88cb3640 1 --2- 172.21.15.114:0/2823587953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f063547290 0x55f0635557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:20.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.181+0000 7fcd89cb5640 1 -- 172.21.15.114:0/2823587953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f063547290 msgr2=0x55f0635557d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:20.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.181+0000 7fcd89cb5640 1 --2- 172.21.15.114:0/2823587953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f063547290 0x55f0635557d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.181+0000 7fcd89cb5640 1 -- 172.21.15.114:0/2823587953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f063546970 msgr2=0x55f063555290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:20.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.181+0000 7fcd894b4640 1 --2- 172.21.15.114:0/2823587953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f063546970 0x55f063555290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:20.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.181+0000 7fcd89cb5640 1 --2- 172.21.15.114:0/2823587953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f063546970 0x55f063555290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.181+0000 7fcd89cb5640 1 -- 172.21.15.114:0/2823587953 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd841ba6b0 con 0x55f063551b30 2024-09-13T18:25:20.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.182+0000 7fcd88cb3640 1 --2- 172.21.15.114:0/2823587953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f063547290 0x55f0635557d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:25:20.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.182+0000 7fcd89cb5640 1 --2- 172.21.15.114:0/2823587953 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f063551b30 0x55f063555d10 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c0b8bc0 tx=0x7fcd7c0ba300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67259b920f7c6040 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:20.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.182+0000 7fcd6cff9640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:25:20.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.182+0000 7fcd6cff9640 1 -- 172.21.15.114:0/2823587953 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c0da070 con 0x55f063551b30 2024-09-13T18:25:20.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.182+0000 7fcd6cff9640 1 -- 172.21.15.114:0/2823587953 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fcd7c0da1d0 con 0x55f063551b30 2024-09-13T18:25:20.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.182+0000 7fcd8e437740 1 -- 172.21.15.114:0/2823587953 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcd74005020 con 0x55f063551b30 2024-09-13T18:25:20.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.183+0000 7fcd6cff9640 1 -- 172.21.15.114:0/2823587953 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd7c0eb080 con 0x55f063551b30 2024-09-13T18:25:20.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.183+0000 7fcd8e437740 10 client.16665 fetch_fsmap learned FSMap version 32 2024-09-13T18:25:20.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.183+0000 7fcd8e437740 1 -- 172.21.15.114:0/2823587953 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55f06346fe80 con 0x55f063551b30 2024-09-13T18:25:20.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.184+0000 7fcd6cff9640 1 -- 172.21.15.114:0/2823587953 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcd7c0c2e90 con 0x55f063551b30 2024-09-13T18:25:20.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.184+0000 7fcd6cff9640 1 -- 172.21.15.114:0/2823587953 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fcd7c0c27b0 con 0x55f063551b30 2024-09-13T18:25:20.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.184+0000 7fcd8e437740 10 client.16665 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:25:20.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.184+0000 7fcd8e437740 10 client.16665 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-13T18:25:20.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.184+0000 7fcd8e437740 20 client.16665 populate_metadata read hostname 'smithi114' 2024-09-13T18:25:20.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.184+0000 7fcd8e437740 1 --2- 172.21.15.114:0/2823587953 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55f0635b8ca0 0x55f0635bb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:20.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.184+0000 7fcd8e437740 4 client.16665 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:25:20.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.184+0000 7fcd8e437740 1 -- 172.21.15.114:0/2823587953 --> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fcd74005020 con 0x55f0635b8ca0 2024-09-13T18:25:20.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.185+0000 7fcd894b4640 1 --2- 172.21.15.114:0/2823587953 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55f0635b8ca0 0x55f0635bb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:20.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.187+0000 7fcd894b4640 1 --2- 172.21.15.114:0/2823587953 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55f0635b8ca0 0x55f0635bb090 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b8fb33be841a3b80 server_cookie=f05276a72a9189eb in_seq=0 out_seq=0 2024-09-13T18:25:20.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.187+0000 7fcd6cff9640 10 client.16665 ms_handle_connect on v2:172.21.15.204:6838/344883912 2024-09-13T18:25:20.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.187+0000 7fcd6cff9640 1 -- 172.21.15.114:0/2823587953 <== mds.2 v2:172.21.15.204:6838/344883912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fcd74005020 con 0x55f0635b8ca0 2024-09-13T18:25:20.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.187+0000 7fcd6cff9640 10 client.16665 handle_command_reply: tid=0 2024-09-13T18:25:20.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.187+0000 7fcd8e437740 1 client.16665 shutdown 2024-09-13T18:25:20.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.191+0000 7fcd8e437740 1 -- 172.21.15.114:0/2823587953 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f063551b30 msgr2=0x55f063555d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:20.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.191+0000 7fcd8e437740 1 --2- 172.21.15.114:0/2823587953 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f063551b30 0x55f063555d10 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c0b8bc0 tx=0x7fcd7c0ba300 comp rx=0 tx=0).stop 2024-09-13T18:25:20.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.192+0000 7fcd8e437740 1 -- 172.21.15.114:0/2823587953 shutdown_connections 2024-09-13T18:25:20.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.192+0000 7fcd8e437740 1 --2- 172.21.15.114:0/2823587953 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55f0635b8ca0 0x55f0635bb090 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.192+0000 7fcd8e437740 1 --2- 172.21.15.114:0/2823587953 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f063551b30 0x55f063555d10 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.192+0000 7fcd8e437740 1 --2- 172.21.15.114:0/2823587953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f063547290 0x55f0635557d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.192+0000 7fcd8e437740 1 --2- 172.21.15.114:0/2823587953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f063546970 0x55f063555290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.192+0000 7fcd8e437740 1 -- 172.21.15.114:0/2823587953 >> 172.21.15.114:0/2823587953 conn(0x55f063542ee0 msgr2=0x55f06354fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:20.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.193+0000 7fcd6cff9640 0 client.16665 ms_handle_reset on v2:172.21.15.204:6838/344883912 2024-09-13T18:25:20.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.193+0000 7fcd8e437740 1 -- 172.21.15.114:0/2823587953 shutdown_connections 2024-09-13T18:25:20.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.193+0000 7fcd8e437740 1 -- 172.21.15.114:0/2823587953 wait complete. 2024-09-13T18:25:20.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.193+0000 7fcd8e437740 20 client.16665 trim_cache size 0 max 16384 2024-09-13T18:25:20.200 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:25:20.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.194+0000 7fcd8a4b6640 1 -- 172.21.15.114:0/670455956 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fcd700a1410 msgr2=0x7fcd700a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:20.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.194+0000 7fcd8a4b6640 1 --2- 172.21.15.114:0/670455956 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fcd700a1410 0x7fcd700a38d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x55f063543d40 tx=0x7fcd80004180 comp rx=0 tx=0).stop 2024-09-13T18:25:20.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.194+0000 7fcd8a4b6640 1 -- 172.21.15.114:0/670455956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd84070470 msgr2=0x7fcd84070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:20.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.194+0000 7fcd8a4b6640 1 --2- 172.21.15.114:0/670455956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd84070470 0x7fcd84070920 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c009560 tx=0x7fcd7c003c10 comp rx=0 tx=0).stop 2024-09-13T18:25:20.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.194+0000 7fcd8a4b6640 1 -- 172.21.15.114:0/670455956 shutdown_connections 2024-09-13T18:25:20.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.194+0000 7fcd8a4b6640 1 --2- 172.21.15.114:0/670455956 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fcd700a1410 0x7fcd700a38d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.194+0000 7fcd8a4b6640 1 --2- 172.21.15.114:0/670455956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd84070470 0x7fcd84070920 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.194+0000 7fcd8a4b6640 1 --2- 172.21.15.114:0/670455956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd84105010 0x7fcd8406c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.194+0000 7fcd8a4b6640 1 --2- 172.21.15.114:0/670455956 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd840ffec0 0x7fcd8406bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.194+0000 7fcd8a4b6640 1 -- 172.21.15.114:0/670455956 >> 172.21.15.114:0/670455956 conn(0x7fcd840fb950 msgr2=0x7fcd840fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:20.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.195+0000 7fcd8a4b6640 1 -- 172.21.15.114:0/670455956 shutdown_connections 2024-09-13T18:25:20.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.195+0000 7fcd8a4b6640 1 -- 172.21.15.114:0/670455956 wait complete. 2024-09-13T18:25:20.217 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-09-13T18:25:20.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.410+0000 7f7b007b5640 1 -- 172.21.15.114:0/783485052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7af810a710 msgr2=0x7f7af810ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:20.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.410+0000 7f7b007b5640 1 --2- 172.21.15.114:0/783485052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7af810a710 0x7f7af810ab70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7aec009900 tx=0x7f7aec02a420 comp rx=0 tx=0).stop 2024-09-13T18:25:20.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.411+0000 7f7b007b5640 1 -- 172.21.15.114:0/783485052 shutdown_connections 2024-09-13T18:25:20.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.411+0000 7f7b007b5640 1 --2- 172.21.15.114:0/783485052 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7af810b0b0 0x7f7af81175d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.411+0000 7f7b007b5640 1 --2- 172.21.15.114:0/783485052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7af810a710 0x7f7af810ab70 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.411+0000 7f7b007b5640 1 --2- 172.21.15.114:0/783485052 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7af8110710 0x7f7af8110af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.411+0000 7f7b007b5640 1 -- 172.21.15.114:0/783485052 >> 172.21.15.114:0/783485052 conn(0x7f7af806aaa0 msgr2=0x7f7af806aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:20.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.412+0000 7f7b007b5640 1 -- 172.21.15.114:0/783485052 shutdown_connections 2024-09-13T18:25:20.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.412+0000 7f7b007b5640 1 -- 172.21.15.114:0/783485052 wait complete. 2024-09-13T18:25:20.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.413+0000 7f7b007b5640 1 Processor -- start 2024-09-13T18:25:20.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.413+0000 7f7b007b5640 1 -- start start 2024-09-13T18:25:20.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.413+0000 7f7b007b5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7af810a710 0x7f7af8115ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:20.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.413+0000 7f7b007b5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7af810b0b0 0x7f7af8116530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:20.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.413+0000 7f7b007b5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7af8110710 0x7f7af8116a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:20.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.413+0000 7f7b007b5640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af81b9b00 con 0x7f7af8110710 2024-09-13T18:25:20.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.413+0000 7f7b007b5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af81b9c40 con 0x7f7af810a710 2024-09-13T18:25:20.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.413+0000 7f7b007b5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af81b9e00 con 0x7f7af810b0b0 2024-09-13T18:25:20.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.414+0000 7f7afdd50640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7af810b0b0 0x7f7af8116530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:20.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.414+0000 7f7afe551640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7af810a710 0x7f7af8115ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:20.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.414+0000 7f7afdd50640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7af810b0b0 0x7f7af8116530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:38218/0 (socket says 172.21.15.114:38218) 2024-09-13T18:25:20.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.414+0000 7f7afdd50640 1 -- 172.21.15.114:0/850075055 learned_addr learned my addr 172.21.15.114:0/850075055 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:20.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.414+0000 7f7afed52640 1 --2- 172.21.15.114:0/850075055 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7af8110710 0x7f7af8116a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:20.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.414+0000 7f7afdd50640 1 -- 172.21.15.114:0/850075055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7af810a710 msgr2=0x7f7af8115ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:20.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.414+0000 7f7afdd50640 1 --2- 172.21.15.114:0/850075055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7af810a710 0x7f7af8115ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.414+0000 7f7afdd50640 1 -- 172.21.15.114:0/850075055 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7af8110710 msgr2=0x7f7af8116a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:20.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.414+0000 7f7afdd50640 1 --2- 172.21.15.114:0/850075055 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7af8110710 0x7f7af8116a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.414+0000 7f7afdd50640 1 -- 172.21.15.114:0/850075055 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aec009560 con 0x7f7af810b0b0 2024-09-13T18:25:20.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.414+0000 7f7afdd50640 1 --2- 172.21.15.114:0/850075055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7af810b0b0 0x7f7af8116530 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7aec02aa10 tx=0x7f7aec0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:20.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.415+0000 7f7af77fe640 1 -- 172.21.15.114:0/850075055 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec004d80 con 0x7f7af810b0b0 2024-09-13T18:25:20.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.415+0000 7f7af77fe640 1 -- 172.21.15.114:0/850075055 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7aec003a80 con 0x7f7af810b0b0 2024-09-13T18:25:20.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.415+0000 7f7b007b5640 1 -- 172.21.15.114:0/850075055 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7af81ba080 con 0x7f7af810b0b0 2024-09-13T18:25:20.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.415+0000 7f7af77fe640 1 -- 172.21.15.114:0/850075055 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec03c990 con 0x7f7af810b0b0 2024-09-13T18:25:20.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.415+0000 7f7b007b5640 1 -- 172.21.15.114:0/850075055 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7af81ba580 con 0x7f7af810b0b0 2024-09-13T18:25:20.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.419+0000 7f7b034d8740 1 Processor -- start 2024-09-13T18:25:20.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.419+0000 7f7b034d8740 1 -- start start 2024-09-13T18:25:20.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.420+0000 7f7b034d8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56370a248790 0x56370a248b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:20.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.420+0000 7f7b034d8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56370a2490b0 0x56370a253410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:20.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.420+0000 7f7b034d8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56370a253950 0x56370a255d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:20.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.420+0000 7f7b034d8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56370a1b2cc0 con 0x56370a248790 2024-09-13T18:25:20.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.420+0000 7f7b034d8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56370a1b0450 con 0x56370a253950 2024-09-13T18:25:20.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.420+0000 7f7b034d8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56370a1abd40 con 0x56370a2490b0 2024-09-13T18:25:20.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.420+0000 7f7afe551640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56370a2490b0 0x56370a253410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:20.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.420+0000 7f7afed52640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56370a248790 0x56370a248b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:20.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.420+0000 7f7afed52640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56370a248790 0x56370a248b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44004/0 (socket says 172.21.15.114:44004) 2024-09-13T18:25:20.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.420+0000 7f7afed52640 1 -- 172.21.15.114:0/778017567 learned_addr learned my addr 172.21.15.114:0/778017567 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:20.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.421+0000 7f7af77fe640 1 -- 172.21.15.114:0/850075055 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f7aec003bf0 con 0x7f7af810b0b0 2024-09-13T18:25:20.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.421+0000 7f7afdd50640 1 --2- 172.21.15.114:0/778017567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56370a253950 0x56370a255d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:20.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.422+0000 7f7af77fe640 1 --2- 172.21.15.114:0/850075055 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7ae00a1340 0x7f7ae00a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:20.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.422+0000 7f7afed52640 1 -- 172.21.15.114:0/778017567 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56370a2490b0 msgr2=0x56370a253410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:20.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.422+0000 7f7afed52640 1 --2- 172.21.15.114:0/778017567 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56370a2490b0 0x56370a253410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.422+0000 7f7afed52640 1 -- 172.21.15.114:0/778017567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56370a253950 msgr2=0x56370a255d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:20.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.422+0000 7f7afed52640 1 --2- 172.21.15.114:0/778017567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56370a253950 0x56370a255d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.422+0000 7f7afed52640 1 -- 172.21.15.114:0/778017567 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56370a173620 con 0x56370a248790 2024-09-13T18:25:20.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.422+0000 7f7af77fe640 1 -- 172.21.15.114:0/850075055 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f7aec0ee140 con 0x7f7af810b0b0 2024-09-13T18:25:20.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.422+0000 7f7afed52640 1 --2- 172.21.15.114:0/850075055 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7ae00a1340 0x7f7ae00a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:20.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.423+0000 7f7afed52640 1 --2- 172.21.15.114:0/778017567 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56370a248790 0x56370a248b70 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f7ae800b770 tx=0x7f7ae800bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae271c62dad79d72 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:20.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.423+0000 7f7af4ff9640 1 -- 172.21.15.114:0/778017567 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae800be80 con 0x56370a248790 2024-09-13T18:25:20.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.423+0000 7f7af4ff9640 1 -- 172.21.15.114:0/778017567 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7ae8015d60 con 0x56370a248790 2024-09-13T18:25:20.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.424+0000 7f7af4ff9640 1 -- 172.21.15.114:0/778017567 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae801fbe0 con 0x56370a248790 2024-09-13T18:25:20.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.424+0000 7f7b034d8740 1 -- 172.21.15.114:0/778017567 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56370a248790 msgr2=0x56370a248b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:20.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.424+0000 7f7b034d8740 1 --2- 172.21.15.114:0/778017567 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56370a248790 0x56370a248b70 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f7ae800b770 tx=0x7f7ae800bc30 comp rx=0 tx=0).stop 2024-09-13T18:25:20.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.424+0000 7f7afed52640 1 --2- 172.21.15.114:0/850075055 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7ae00a1340 0x7f7ae00a3800 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7ae801e000 tx=0x7f7ae8021000 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:20.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.425+0000 7f7b034d8740 1 -- 172.21.15.114:0/778017567 shutdown_connections 2024-09-13T18:25:20.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.425+0000 7f7b034d8740 1 --2- 172.21.15.114:0/778017567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56370a253950 0x56370a255d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.425+0000 7f7b034d8740 1 --2- 172.21.15.114:0/778017567 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56370a2490b0 0x56370a253410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.425+0000 7f7b034d8740 1 --2- 172.21.15.114:0/778017567 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56370a248790 0x56370a248b70 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.425+0000 7f7b034d8740 1 -- 172.21.15.114:0/778017567 >> 172.21.15.114:0/778017567 conn(0x56370a244d00 msgr2=0x56370a2450e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:20.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.425+0000 7f7b034d8740 1 -- 172.21.15.114:0/778017567 shutdown_connections 2024-09-13T18:25:20.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.425+0000 7f7b034d8740 1 -- 172.21.15.114:0/778017567 wait complete. 2024-09-13T18:25:20.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.425+0000 7f7b034d8740 1 -- start start 2024-09-13T18:25:20.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.426+0000 7f7b034d8740 1 Processor -- start 2024-09-13T18:25:20.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.426+0000 7f7b034d8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56370a248790 0x56370a252f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:20.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.426+0000 7f7b034d8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56370a2490b0 0x56370a250270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:20.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.426+0000 7f7b034d8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56370a253950 0x56370a2507b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:20.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.426+0000 7f7b034d8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56370a16a180 con 0x56370a2490b0 2024-09-13T18:25:20.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.426+0000 7f7b034d8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563709e84430 con 0x56370a248790 2024-09-13T18:25:20.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.426+0000 7f7b034d8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56370a12f5e0 con 0x56370a253950 2024-09-13T18:25:20.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.426+0000 7f7b034d8740 10 client.0 _ll_register_callbacks cb 0x56370a19baa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:25:20.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.426+0000 7f7afdd50640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56370a2490b0 0x56370a250270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:20.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.426+0000 7f7afdd50640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56370a2490b0 0x56370a250270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44010/0 (socket says 172.21.15.114:44010) 2024-09-13T18:25:20.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.426+0000 7f7afdd50640 1 -- 172.21.15.114:0/4027004739 learned_addr learned my addr 172.21.15.114:0/4027004739 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:20.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.427+0000 7f7afe551640 1 --2- 172.21.15.114:0/4027004739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56370a248790 0x56370a252f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:20.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.427+0000 7f7afed52640 1 --2- 172.21.15.114:0/4027004739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56370a253950 0x56370a2507b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:20.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.427+0000 7f7afdd50640 1 -- 172.21.15.114:0/4027004739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56370a253950 msgr2=0x56370a2507b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:20.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.427+0000 7f7afdd50640 1 --2- 172.21.15.114:0/4027004739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56370a253950 0x56370a2507b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.427+0000 7f7afdd50640 1 -- 172.21.15.114:0/4027004739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56370a248790 msgr2=0x56370a252f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:20.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.427+0000 7f7afdd50640 1 --2- 172.21.15.114:0/4027004739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56370a248790 0x56370a252f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.427+0000 7f7afdd50640 1 -- 172.21.15.114:0/4027004739 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ae800e040 con 0x56370a2490b0 2024-09-13T18:25:20.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.427+0000 7f7afdd50640 1 --2- 172.21.15.114:0/4027004739 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56370a2490b0 0x56370a250270 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f7aec0040b0 tx=0x7f7aec003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ba6cd24b9324e3f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:20.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.427+0000 7f7addffb640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:25:20.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.428+0000 7f7addffb640 1 -- 172.21.15.114:0/4027004739 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec0becd0 con 0x56370a2490b0 2024-09-13T18:25:20.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.428+0000 7f7addffb640 1 -- 172.21.15.114:0/4027004739 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f7aec0c3320 con 0x56370a2490b0 2024-09-13T18:25:20.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.428+0000 7f7b034d8740 1 -- 172.21.15.114:0/4027004739 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56370a1d5b80 con 0x56370a2490b0 2024-09-13T18:25:20.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.428+0000 7f7addffb640 1 -- 172.21.15.114:0/4027004739 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7aec0ba390 con 0x56370a2490b0 2024-09-13T18:25:20.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.428+0000 7f7b034d8740 10 client.16674 fetch_fsmap learned FSMap version 32 2024-09-13T18:25:20.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.428+0000 7f7b034d8740 1 -- 172.21.15.114:0/4027004739 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56370a1aa3c0 con 0x56370a2490b0 2024-09-13T18:25:20.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.429+0000 7f7addffb640 1 -- 172.21.15.114:0/4027004739 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56370a1d5b80 con 0x56370a2490b0 2024-09-13T18:25:20.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.429+0000 7f7addffb640 1 -- 172.21.15.114:0/4027004739 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f7aec0f0110 con 0x56370a2490b0 2024-09-13T18:25:20.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.429+0000 7f7b034d8740 10 client.16674 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:25:20.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.429+0000 7f7b034d8740 10 client.16674 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-13T18:25:20.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.429+0000 7f7b034d8740 20 client.16674 populate_metadata read hostname 'smithi114' 2024-09-13T18:25:20.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.429+0000 7f7b034d8740 1 --2- 172.21.15.114:0/4027004739 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x56370a262ec0 0x56370a2652e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:20.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.429+0000 7f7b034d8740 4 client.16674 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:25:20.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.429+0000 7f7b034d8740 1 -- 172.21.15.114:0/4027004739 --> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56370a19e1f0 con 0x56370a262ec0 2024-09-13T18:25:20.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.430+0000 7f7afe551640 1 --2- 172.21.15.114:0/4027004739 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x56370a262ec0 0x56370a2652e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:20.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.430+0000 7f7afe551640 1 --2- 172.21.15.114:0/4027004739 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x56370a262ec0 0x56370a2652e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=274047d7bd58b65a server_cookie=8e01787ccd172857 in_seq=0 out_seq=0 2024-09-13T18:25:20.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.430+0000 7f7addffb640 10 client.16674 ms_handle_connect on v2:172.21.15.134:6836/891692210 2024-09-13T18:25:20.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.431+0000 7f7addffb640 1 -- 172.21.15.114:0/4027004739 <== mds.1 v2:172.21.15.134:6836/891692210 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x56370a19e1f0 con 0x56370a262ec0 2024-09-13T18:25:20.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.432+0000 7f7addffb640 10 client.16674 handle_command_reply: tid=0 2024-09-13T18:25:20.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.432+0000 7f7b034d8740 1 client.16674 shutdown 2024-09-13T18:25:20.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.432+0000 7f7b034d8740 1 -- 172.21.15.114:0/4027004739 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56370a2490b0 msgr2=0x56370a250270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:20.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.432+0000 7f7b034d8740 1 --2- 172.21.15.114:0/4027004739 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56370a2490b0 0x56370a250270 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f7aec0040b0 tx=0x7f7aec003f90 comp rx=0 tx=0).stop 2024-09-13T18:25:20.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.432+0000 7f7b034d8740 1 -- 172.21.15.114:0/4027004739 shutdown_connections 2024-09-13T18:25:20.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.432+0000 7f7b034d8740 1 --2- 172.21.15.114:0/4027004739 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x56370a262ec0 0x56370a2652e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.432+0000 7f7b034d8740 1 --2- 172.21.15.114:0/4027004739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56370a253950 0x56370a2507b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.432+0000 7f7b034d8740 1 --2- 172.21.15.114:0/4027004739 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56370a2490b0 0x56370a250270 secure :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f7aec0040b0 tx=0x7f7aec003f90 comp rx=0 tx=0).stop 2024-09-13T18:25:20.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.432+0000 7f7b034d8740 1 --2- 172.21.15.114:0/4027004739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56370a248790 0x56370a252f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.432+0000 7f7b034d8740 1 -- 172.21.15.114:0/4027004739 >> 172.21.15.114:0/4027004739 conn(0x56370a244d00 msgr2=0x56370a243eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:20.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.432+0000 7f7addffb640 0 client.16674 ms_handle_reset on v2:172.21.15.134:6836/891692210 2024-09-13T18:25:20.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.433+0000 7f7b034d8740 1 -- 172.21.15.114:0/4027004739 shutdown_connections 2024-09-13T18:25:20.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.433+0000 7f7b034d8740 1 -- 172.21.15.114:0/4027004739 wait complete. 2024-09-13T18:25:20.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.433+0000 7f7b034d8740 20 client.16674 trim_cache size 0 max 16384 2024-09-13T18:25:20.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.453+0000 7f7b034d8740 1 Processor -- start 2024-09-13T18:25:20.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.453+0000 7f7b034d8740 1 -- start start 2024-09-13T18:25:20.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.453+0000 7f7b034d8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56370a244d00 0x56370a249b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:20.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.453+0000 7f7b034d8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56370a248790 0x56370a25a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:20.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.454+0000 7f7b034d8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56370a253950 0x56370a25ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:20.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.454+0000 7f7afe551640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56370a244d00 0x56370a249b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:20.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.454+0000 7f7afe551640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56370a244d00 0x56370a249b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:38254/0 (socket says 172.21.15.114:38254) 2024-09-13T18:25:20.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.454+0000 7f7afdd50640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56370a248790 0x56370a25a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:20.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.454+0000 7f7afdd50640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56370a248790 0x56370a25a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:43874/0 (socket says 172.21.15.114:43874) 2024-09-13T18:25:20.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.454+0000 7f7afe551640 1 -- 172.21.15.114:0/4254942810 learned_addr learned my addr 172.21.15.114:0/4254942810 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:20.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.454+0000 7f7b034d8740 1 -- 172.21.15.114:0/4254942810 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56370a19e1f0 con 0x56370a253950 2024-09-13T18:25:20.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.454+0000 7f7b034d8740 1 -- 172.21.15.114:0/4254942810 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56370a1d5b80 con 0x56370a248790 2024-09-13T18:25:20.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.454+0000 7f7b034d8740 1 -- 172.21.15.114:0/4254942810 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56370a2459f0 con 0x56370a244d00 2024-09-13T18:25:20.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.455+0000 7f7afe551640 1 -- 172.21.15.114:0/4254942810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56370a248790 msgr2=0x56370a25a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:20.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.455+0000 7f7afe551640 1 --2- 172.21.15.114:0/4254942810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56370a248790 0x56370a25a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.455+0000 7f7afe551640 1 -- 172.21.15.114:0/4254942810 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56370a253950 msgr2=0x56370a25ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:20.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.455+0000 7f7afe551640 1 --2- 172.21.15.114:0/4254942810 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56370a253950 0x56370a25ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.455+0000 7f7afe551640 1 -- 172.21.15.114:0/4254942810 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56370a14c770 con 0x56370a244d00 2024-09-13T18:25:20.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.455+0000 7f7afe551640 1 --2- 172.21.15.114:0/4254942810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56370a244d00 0x56370a249b70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x56370a245b60 tx=0x7f7af00096b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9dd1a4fbe65957 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:20.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.456+0000 7f7af57fa640 1 -- 172.21.15.114:0/4254942810 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af001b410 con 0x56370a244d00 2024-09-13T18:25:20.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.457+0000 7f7af57fa640 1 -- 172.21.15.114:0/4254942810 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7af001f070 con 0x56370a244d00 2024-09-13T18:25:20.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.457+0000 7f7af57fa640 1 -- 172.21.15.114:0/4254942810 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af0024430 con 0x56370a244d00 2024-09-13T18:25:20.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.459+0000 7f7b034d8740 1 -- 172.21.15.114:0/4254942810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56370a244d00 msgr2=0x56370a249b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:20.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.459+0000 7f7b034d8740 1 --2- 172.21.15.114:0/4254942810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56370a244d00 0x56370a249b70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x56370a245b60 tx=0x7f7af00096b0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.460+0000 7f7b034d8740 1 -- 172.21.15.114:0/4254942810 shutdown_connections 2024-09-13T18:25:20.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.460+0000 7f7b034d8740 1 --2- 172.21.15.114:0/4254942810 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56370a253950 0x56370a25ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.460+0000 7f7b034d8740 1 --2- 172.21.15.114:0/4254942810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56370a248790 0x56370a25a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.460+0000 7f7b034d8740 1 --2- 172.21.15.114:0/4254942810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56370a244d00 0x56370a249b70 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.460+0000 7f7b034d8740 1 -- 172.21.15.114:0/4254942810 >> 172.21.15.114:0/4254942810 conn(0x56370a253060 msgr2=0x56370a24eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:20.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.461+0000 7f7b034d8740 1 -- 172.21.15.114:0/4254942810 shutdown_connections 2024-09-13T18:25:20.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.461+0000 7f7b034d8740 1 -- 172.21.15.114:0/4254942810 wait complete. 2024-09-13T18:25:20.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.462+0000 7f7b034d8740 1 -- start start 2024-09-13T18:25:20.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.462+0000 7f7b034d8740 1 Processor -- start 2024-09-13T18:25:20.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.462+0000 7f7b034d8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56370a244d00 0x56370a25a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:20.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.462+0000 7f7b034d8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56370a248790 0x56370a256d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:20.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.463+0000 7f7b034d8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56370a253950 0x56370a2572a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:20.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.463+0000 7f7b034d8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56370a257950 con 0x56370a253950 2024-09-13T18:25:20.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.463+0000 7f7b034d8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56370a257ac0 con 0x56370a244d00 2024-09-13T18:25:20.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.463+0000 7f7afdd50640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56370a248790 0x56370a256d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:20.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.463+0000 7f7afed52640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56370a253950 0x56370a2572a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:20.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.463+0000 7f7b034d8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56370a2b93c0 con 0x56370a248790 2024-09-13T18:25:20.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.463+0000 7f7afed52640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56370a253950 0x56370a2572a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44030/0 (socket says 172.21.15.114:44030) 2024-09-13T18:25:20.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.463+0000 7f7afdd50640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56370a248790 0x56370a256d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:38260/0 (socket says 172.21.15.114:38260) 2024-09-13T18:25:20.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.463+0000 7f7afed52640 1 -- 172.21.15.114:0/3397862012 learned_addr learned my addr 172.21.15.114:0/3397862012 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:20.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.463+0000 7f7b034d8740 10 client.0 _ll_register_callbacks cb 0x7f7af8071960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:25:20.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.463+0000 7f7afdd50640 1 -- 172.21.15.114:0/3397862012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56370a244d00 msgr2=0x56370a25a910 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:25:20.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.464+0000 7f7afdd50640 1 --2- 172.21.15.114:0/3397862012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56370a244d00 0x56370a25a910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.464+0000 7f7afdd50640 1 -- 172.21.15.114:0/3397862012 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56370a253950 msgr2=0x56370a2572a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:20.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.464+0000 7f7afdd50640 1 --2- 172.21.15.114:0/3397862012 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56370a253950 0x56370a2572a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.464+0000 7f7afdd50640 1 -- 172.21.15.114:0/3397862012 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7af0009e50 con 0x56370a248790 2024-09-13T18:25:20.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.464+0000 7f7afdd50640 1 --2- 172.21.15.114:0/3397862012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56370a248790 0x56370a256d60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x56370a251330 tx=0x7f7aec0c30c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a410effc17c671e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:20.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.464+0000 7f7addffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:25:20.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.464+0000 7f7addffb640 1 -- 172.21.15.114:0/3397862012 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec0a48a0 con 0x56370a248790 2024-09-13T18:25:20.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.464+0000 7f7addffb640 1 -- 172.21.15.114:0/3397862012 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f7aec0c3320 con 0x56370a248790 2024-09-13T18:25:20.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.464+0000 7f7b034d8740 1 -- 172.21.15.114:0/3397862012 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7ae8015d60 con 0x56370a248790 2024-09-13T18:25:20.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.464+0000 7f7addffb640 1 -- 172.21.15.114:0/3397862012 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7aec0eb970 con 0x56370a248790 2024-09-13T18:25:20.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.464+0000 7f7b034d8740 10 client.26047 fetch_fsmap learned FSMap version 32 2024-09-13T18:25:20.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.465+0000 7f7b034d8740 1 -- 172.21.15.114:0/3397862012 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56370a171e80 con 0x56370a248790 2024-09-13T18:25:20.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.465+0000 7f7addffb640 1 -- 172.21.15.114:0/3397862012 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7aec0ba800 con 0x56370a248790 2024-09-13T18:25:20.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.465+0000 7f7addffb640 1 -- 172.21.15.114:0/3397862012 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f7aec0ba4b0 con 0x56370a248790 2024-09-13T18:25:20.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.466+0000 7f7b034d8740 10 client.26047 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:25:20.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.466+0000 7f7b034d8740 10 client.26047 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-13T18:25:20.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.466+0000 7f7b034d8740 20 client.26047 populate_metadata read hostname 'smithi114' 2024-09-13T18:25:20.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.466+0000 7f7b034d8740 1 --2- 172.21.15.114:0/3397862012 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x56370a2ba530 0x56370a2bc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:20.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.466+0000 7f7b034d8740 4 client.26047 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:25:20.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.466+0000 7f7b034d8740 1 -- 172.21.15.114:0/3397862012 --> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f7ae8015d60 con 0x56370a2ba530 2024-09-13T18:25:20.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.466+0000 7f7afe551640 1 --2- 172.21.15.114:0/3397862012 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x56370a2ba530 0x56370a2bc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:20.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:20 smithi114 ceph-mon[30897]: pgmap v89: 129 pgs: 129 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 77 MiB/s wr, 1.24k op/s 2024-09-13T18:25:20.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.468+0000 7f7afe551640 1 --2- 172.21.15.114:0/3397862012 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x56370a2ba530 0x56370a2bc920 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a4fbbddf02f0b5e2 server_cookie=9b5ddb25f04835e5 in_seq=0 out_seq=0 2024-09-13T18:25:20.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.468+0000 7f7addffb640 10 client.26047 ms_handle_connect on v2:172.21.15.134:6836/891692210 2024-09-13T18:25:20.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.468+0000 7f7addffb640 1 -- 172.21.15.114:0/3397862012 <== mds.1 v2:172.21.15.134:6836/891692210 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f7ae8015d60 con 0x56370a2ba530 2024-09-13T18:25:20.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.468+0000 7f7addffb640 10 client.26047 handle_command_reply: tid=0 2024-09-13T18:25:20.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.469+0000 7f7b034d8740 1 client.26047 shutdown 2024-09-13T18:25:20.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.469+0000 7f7b034d8740 1 -- 172.21.15.114:0/3397862012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56370a248790 msgr2=0x56370a256d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:20.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.469+0000 7f7b034d8740 1 --2- 172.21.15.114:0/3397862012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56370a248790 0x56370a256d60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x56370a251330 tx=0x7f7aec0c30c0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.470+0000 7f7b034d8740 1 -- 172.21.15.114:0/3397862012 shutdown_connections 2024-09-13T18:25:20.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.470+0000 7f7b034d8740 1 --2- 172.21.15.114:0/3397862012 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x56370a2ba530 0x56370a2bc920 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.470+0000 7f7b034d8740 1 --2- 172.21.15.114:0/3397862012 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56370a253950 0x56370a2572a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.470+0000 7f7b034d8740 1 --2- 172.21.15.114:0/3397862012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56370a248790 0x56370a256d60 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.470+0000 7f7addffb640 0 client.26047 ms_handle_reset on v2:172.21.15.134:6836/891692210 2024-09-13T18:25:20.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.470+0000 7f7b034d8740 1 --2- 172.21.15.114:0/3397862012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56370a244d00 0x56370a25a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.470+0000 7f7b034d8740 1 -- 172.21.15.114:0/3397862012 >> 172.21.15.114:0/3397862012 conn(0x56370a253060 msgr2=0x56370a250330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:20.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.470+0000 7f7b034d8740 1 -- 172.21.15.114:0/3397862012 shutdown_connections 2024-09-13T18:25:20.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.470+0000 7f7b034d8740 1 -- 172.21.15.114:0/3397862012 wait complete. 2024-09-13T18:25:20.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.470+0000 7f7b034d8740 20 client.26047 trim_cache size 0 max 16384 2024-09-13T18:25:20.475 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:25:20.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.471+0000 7f7b007b5640 1 -- 172.21.15.114:0/850075055 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7ae00a1340 msgr2=0x7f7ae00a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:20.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.471+0000 7f7b007b5640 1 --2- 172.21.15.114:0/850075055 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7ae00a1340 0x7f7ae00a3800 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7ae801e000 tx=0x7f7ae8021000 comp rx=0 tx=0).stop 2024-09-13T18:25:20.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.471+0000 7f7b007b5640 1 -- 172.21.15.114:0/850075055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7af810b0b0 msgr2=0x7f7af8116530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:20.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.471+0000 7f7b007b5640 1 --2- 172.21.15.114:0/850075055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7af810b0b0 0x7f7af8116530 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7aec02aa10 tx=0x7f7aec0046c0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.472+0000 7f7b007b5640 1 -- 172.21.15.114:0/850075055 shutdown_connections 2024-09-13T18:25:20.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.472+0000 7f7b007b5640 1 --2- 172.21.15.114:0/850075055 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7ae00a1340 0x7f7ae00a3800 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.472+0000 7f7b007b5640 1 --2- 172.21.15.114:0/850075055 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7af8110710 0x7f7af8116a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.472+0000 7f7b007b5640 1 --2- 172.21.15.114:0/850075055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7af810b0b0 0x7f7af8116530 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.472+0000 7f7b007b5640 1 --2- 172.21.15.114:0/850075055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7af810a710 0x7f7af8115ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.472+0000 7f7b007b5640 1 -- 172.21.15.114:0/850075055 >> 172.21.15.114:0/850075055 conn(0x7f7af806aaa0 msgr2=0x7f7af806e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:20.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.472+0000 7f7b007b5640 1 -- 172.21.15.114:0/850075055 shutdown_connections 2024-09-13T18:25:20.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.472+0000 7f7b007b5640 1 -- 172.21.15.114:0/850075055 wait complete. 2024-09-13T18:25:20.492 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-09-13T18:25:20.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.662+0000 7f3ddf7fe640 1 -- 172.21.15.114:0/1571242985 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd8031640 con 0x7f3de01050d0 2024-09-13T18:25:20.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.662+0000 7f3de5a83640 1 -- 172.21.15.114:0/1571242985 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3de01050d0 msgr2=0x7f3de010ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:20.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.662+0000 7f3de5a83640 1 --2- 172.21.15.114:0/1571242985 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3de01050d0 0x7f3de010ba40 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8009cd0 tx=0x7f3dd802a8f0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.662+0000 7f3de5a83640 1 -- 172.21.15.114:0/1571242985 shutdown_connections 2024-09-13T18:25:20.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.662+0000 7f3de5a83640 1 --2- 172.21.15.114:0/1571242985 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3de01050d0 0x7f3de010ba40 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.662+0000 7f3de5a83640 1 --2- 172.21.15.114:0/1571242985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3de01004d0 0x7f3de0104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.662+0000 7f3de5a83640 1 --2- 172.21.15.114:0/1571242985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3de00ffb20 0x7f3de00fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.662+0000 7f3de5a83640 1 -- 172.21.15.114:0/1571242985 >> 172.21.15.114:0/1571242985 conn(0x7f3de0075380 msgr2=0x7f3de0075790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:20.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.663+0000 7f3de5a83640 1 -- 172.21.15.114:0/1571242985 shutdown_connections 2024-09-13T18:25:20.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.663+0000 7f3de5a83640 1 -- 172.21.15.114:0/1571242985 wait complete. 2024-09-13T18:25:20.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.663+0000 7f3de5a83640 1 Processor -- start 2024-09-13T18:25:20.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.663+0000 7f3de5a83640 1 -- start start 2024-09-13T18:25:20.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.664+0000 7f3de5a83640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3de00ffb20 0x7f3de01982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:20.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.664+0000 7f3de5a83640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3de01050d0 0x7f3de01987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:20.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.664+0000 7f3de5a83640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3de019cbe0 0x7f3de019d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:20.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.664+0000 7f3de5a83640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de019d5d0 con 0x7f3de01050d0 2024-09-13T18:25:20.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.664+0000 7f3de5a83640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de019d740 con 0x7f3de00ffb20 2024-09-13T18:25:20.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.664+0000 7f3de5a83640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de019d8b0 con 0x7f3de019cbe0 2024-09-13T18:25:20.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.664+0000 7f3de4a81640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3de00ffb20 0x7f3de01982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:20.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.664+0000 7f3de4a81640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3de00ffb20 0x7f3de01982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:43900/0 (socket says 172.21.15.114:43900) 2024-09-13T18:25:20.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.664+0000 7f3de4a81640 1 -- 172.21.15.114:0/797913739 learned_addr learned my addr 172.21.15.114:0/797913739 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:20.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.664+0000 7f3ddffff640 1 --2- 172.21.15.114:0/797913739 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3de01050d0 0x7f3de01987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:20.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.664+0000 7f3de5282640 1 --2- 172.21.15.114:0/797913739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3de019cbe0 0x7f3de019d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:20.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.664+0000 7f3de4a81640 1 -- 172.21.15.114:0/797913739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3de019cbe0 msgr2=0x7f3de019d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:20.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.664+0000 7f3de4a81640 1 --2- 172.21.15.114:0/797913739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3de019cbe0 0x7f3de019d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.664+0000 7f3de4a81640 1 -- 172.21.15.114:0/797913739 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3de01050d0 msgr2=0x7f3de01987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:20.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.664+0000 7f3de4a81640 1 --2- 172.21.15.114:0/797913739 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3de01050d0 0x7f3de01987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.664+0000 7f3de4a81640 1 -- 172.21.15.114:0/797913739 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd8009900 con 0x7f3de00ffb20 2024-09-13T18:25:20.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.665+0000 7f3de4a81640 1 --2- 172.21.15.114:0/797913739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3de00ffb20 0x7f3de01982b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3dc800e9f0 tx=0x7f3dc800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:20.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.665+0000 7f3dddffb640 1 -- 172.21.15.114:0/797913739 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc8009720 con 0x7f3de00ffb20 2024-09-13T18:25:20.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.665+0000 7f3dddffb640 1 -- 172.21.15.114:0/797913739 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3dc80054b0 con 0x7f3de00ffb20 2024-09-13T18:25:20.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.665+0000 7f3de5a83640 1 -- 172.21.15.114:0/797913739 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3de01a4e90 con 0x7f3de00ffb20 2024-09-13T18:25:20.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.666+0000 7f3de5a83640 1 -- 172.21.15.114:0/797913739 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3de01a52b0 con 0x7f3de00ffb20 2024-09-13T18:25:20.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.666+0000 7f3dddffb640 1 -- 172.21.15.114:0/797913739 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc8004e80 con 0x7f3de00ffb20 2024-09-13T18:25:20.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.669+0000 7f3de9a07740 1 Processor -- start 2024-09-13T18:25:20.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.669+0000 7f3de9a07740 1 -- start start 2024-09-13T18:25:20.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.670+0000 7f3de9a07740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652d6f8f890 0x5652d6f8fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:20.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.670+0000 7f3de9a07740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5652d6f901b0 0x5652d6fa2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:20.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.670+0000 7f3de9a07740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652d6fa3340 0x5652d6fa5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:20.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.670+0000 7f3de9a07740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652d6ef9cc0 con 0x5652d6f901b0 2024-09-13T18:25:20.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.670+0000 7f3de9a07740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652d6ef7450 con 0x5652d6f8f890 2024-09-13T18:25:20.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.670+0000 7f3de9a07740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652d6ef2d40 con 0x5652d6fa3340 2024-09-13T18:25:20.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.670+0000 7f3ddffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652d6fa3340 0x5652d6fa5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:20.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.670+0000 7f3ddffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652d6fa3340 0x5652d6fa5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:38284/0 (socket says 172.21.15.114:38284) 2024-09-13T18:25:20.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.670+0000 7f3ddffff640 1 -- 172.21.15.114:0/809668412 learned_addr learned my addr 172.21.15.114:0/809668412 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:20.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.670+0000 7f3de5282640 1 --2- 172.21.15.114:0/809668412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652d6f8f890 0x5652d6f8fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:20.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.671+0000 7f3ddffff640 1 -- 172.21.15.114:0/809668412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652d6f8f890 msgr2=0x5652d6f8fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:20.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.672+0000 7f3dddffb640 1 -- 172.21.15.114:0/797913739 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f3dc800ccc0 con 0x7f3de00ffb20 2024-09-13T18:25:20.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.672+0000 7f3de4a81640 1 --2- 172.21.15.114:0/809668412 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5652d6f901b0 0x5652d6fa2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:20.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.672+0000 7f3ddffff640 1 --2- 172.21.15.114:0/809668412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652d6f8f890 0x5652d6f8fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.672+0000 7f3ddffff640 1 -- 172.21.15.114:0/809668412 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5652d6f901b0 msgr2=0x5652d6fa2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:20.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.672+0000 7f3ddffff640 1 --2- 172.21.15.114:0/809668412 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5652d6f901b0 0x5652d6fa2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.672+0000 7f3ddffff640 1 -- 172.21.15.114:0/809668412 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652d6eba620 con 0x5652d6fa3340 2024-09-13T18:25:20.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.672+0000 7f3de5282640 1 --2- 172.21.15.114:0/809668412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652d6f8f890 0x5652d6f8fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:25:20.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.673+0000 7f3de4a81640 1 --2- 172.21.15.114:0/809668412 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5652d6f901b0 0x5652d6fa2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:25:20.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.673+0000 7f3ddffff640 1 --2- 172.21.15.114:0/809668412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652d6fa3340 0x5652d6fa5730 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3dd000abf0 tx=0x7f3dd000f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61c3683f40d60bbc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:20.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.673+0000 7f3dddffb640 1 --2- 172.21.15.114:0/797913739 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f3dcc0a1160 0x7f3dcc0a3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:20.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.673+0000 7f3dd6ffd640 1 -- 172.21.15.114:0/809668412 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd0018640 con 0x5652d6fa3340 2024-09-13T18:25:20.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.673+0000 7f3dddffb640 1 -- 172.21.15.114:0/797913739 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f3dc8014070 con 0x7f3de00ffb20 2024-09-13T18:25:20.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.673+0000 7f3dd6ffd640 1 -- 172.21.15.114:0/809668412 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3dd0018c30 con 0x5652d6fa3340 2024-09-13T18:25:20.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.674+0000 7f3dd6ffd640 1 -- 172.21.15.114:0/809668412 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd0010b60 con 0x5652d6fa3340 2024-09-13T18:25:20.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.674+0000 7f3de5282640 1 --2- 172.21.15.114:0/797913739 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f3dcc0a1160 0x7f3dcc0a3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:20.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.675+0000 7f3de9a07740 1 -- 172.21.15.114:0/809668412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652d6fa3340 msgr2=0x5652d6fa5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:20.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.675+0000 7f3de9a07740 1 --2- 172.21.15.114:0/809668412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652d6fa3340 0x5652d6fa5730 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3dd000abf0 tx=0x7f3dd000f790 comp rx=0 tx=0).stop 2024-09-13T18:25:20.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.675+0000 7f3de9a07740 1 -- 172.21.15.114:0/809668412 shutdown_connections 2024-09-13T18:25:20.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.675+0000 7f3de9a07740 1 --2- 172.21.15.114:0/809668412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652d6fa3340 0x5652d6fa5730 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.675+0000 7f3de9a07740 1 --2- 172.21.15.114:0/809668412 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5652d6f901b0 0x5652d6fa2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.675+0000 7f3de9a07740 1 --2- 172.21.15.114:0/809668412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652d6f8f890 0x5652d6f8fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.675+0000 7f3de9a07740 1 -- 172.21.15.114:0/809668412 >> 172.21.15.114:0/809668412 conn(0x5652d6f8bee0 msgr2=0x5652d6f8c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:20.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.675+0000 7f3de9a07740 1 -- 172.21.15.114:0/809668412 shutdown_connections 2024-09-13T18:25:20.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.675+0000 7f3de5282640 1 --2- 172.21.15.114:0/797913739 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f3dcc0a1160 0x7f3dcc0a3620 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8009430 tx=0x7f3dd8009360 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:20.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.675+0000 7f3de9a07740 1 -- 172.21.15.114:0/809668412 wait complete. 2024-09-13T18:25:20.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.676+0000 7f3de9a07740 1 -- start start 2024-09-13T18:25:20.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.676+0000 7f3de9a07740 1 Processor -- start 2024-09-13T18:25:20.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.677+0000 7f3de9a07740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652d6f8f890 0x5652d6fa2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:20.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.677+0000 7f3de9a07740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5652d6f901b0 0x5652d6f9fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:20.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.677+0000 7f3de9a07740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652d6fa3340 0x5652d6fa0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:20.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.677+0000 7f3de9a07740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652d6eb1180 con 0x5652d6f901b0 2024-09-13T18:25:20.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.677+0000 7f3de9a07740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652d6bcb430 con 0x5652d6f8f890 2024-09-13T18:25:20.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.677+0000 7f3de5282640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652d6fa3340 0x5652d6fa0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:20.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.677+0000 7f3de5282640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652d6fa3340 0x5652d6fa0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:38288/0 (socket says 172.21.15.114:38288) 2024-09-13T18:25:20.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.677+0000 7f3de5282640 1 -- 172.21.15.114:0/3264074243 learned_addr learned my addr 172.21.15.114:0/3264074243 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:20.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.677+0000 7f3de4a81640 1 --2- 172.21.15.114:0/3264074243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652d6f8f890 0x5652d6fa2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:20.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.677+0000 7f3de9a07740 1 -- 172.21.15.114:0/3264074243 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652d6e765e0 con 0x5652d6fa3340 2024-09-13T18:25:20.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.678+0000 7f3ddffff640 1 --2- 172.21.15.114:0/3264074243 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5652d6f901b0 0x5652d6f9fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:20.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.678+0000 7f3de9a07740 10 client.0 _ll_register_callbacks cb 0x5652d6ee2aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:25:20.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.678+0000 7f3de5282640 1 -- 172.21.15.114:0/3264074243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652d6f8f890 msgr2=0x5652d6fa2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:20.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.678+0000 7f3de5282640 1 --2- 172.21.15.114:0/3264074243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652d6f8f890 0x5652d6fa2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.678+0000 7f3de5282640 1 -- 172.21.15.114:0/3264074243 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5652d6f901b0 msgr2=0x5652d6f9fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:20.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.678+0000 7f3de5282640 1 --2- 172.21.15.114:0/3264074243 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5652d6f901b0 0x5652d6f9fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.678+0000 7f3de5282640 1 -- 172.21.15.114:0/3264074243 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dd000a820 con 0x5652d6fa3340 2024-09-13T18:25:20.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.678+0000 7f3de5282640 1 --2- 172.21.15.114:0/3264074243 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652d6fa3340 0x5652d6fa0190 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8030980 tx=0x7f3dd802a8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a646ef565d20cf7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:20.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.678+0000 7f3dbffff640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:25:20.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.679+0000 7f3dbffff640 1 -- 172.21.15.114:0/3264074243 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd8041070 con 0x5652d6fa3340 2024-09-13T18:25:20.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.679+0000 7f3de9a07740 1 -- 172.21.15.114:0/3264074243 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5652d6f1cb80 con 0x5652d6fa3340 2024-09-13T18:25:20.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.679+0000 7f3dbffff640 1 -- 172.21.15.114:0/3264074243 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f3dd8004250 con 0x5652d6fa3340 2024-09-13T18:25:20.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.679+0000 7f3dbffff640 1 -- 172.21.15.114:0/3264074243 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3dd8004410 con 0x5652d6fa3340 2024-09-13T18:25:20.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.679+0000 7f3de9a07740 10 client.26059 fetch_fsmap learned FSMap version 32 2024-09-13T18:25:20.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.679+0000 7f3de9a07740 1 -- 172.21.15.114:0/3264074243 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5652d6ef13c0 con 0x5652d6fa3340 2024-09-13T18:25:20.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.680+0000 7f3dbffff640 1 -- 172.21.15.114:0/3264074243 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5652d6f1cb80 con 0x5652d6fa3340 2024-09-13T18:25:20.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.680+0000 7f3dbffff640 1 -- 172.21.15.114:0/3264074243 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f3dd8053b10 con 0x5652d6fa3340 2024-09-13T18:25:20.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.680+0000 7f3de9a07740 10 client.26059 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:25:20.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.680+0000 7f3de9a07740 10 client.26059 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:25:20.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.680+0000 7f3de9a07740 20 client.26059 populate_metadata read hostname 'smithi114' 2024-09-13T18:25:20.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.680+0000 7f3de9a07740 1 --2- 172.21.15.114:0/3264074243 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5652d6fb2d00 0x5652d6fb5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:20.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.680+0000 7f3de9a07740 4 client.26059 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:25:20.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.680+0000 7f3de9a07740 1 -- 172.21.15.114:0/3264074243 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5652d6ee51f0 con 0x5652d6fb2d00 2024-09-13T18:25:20.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.681+0000 7f3de4a81640 1 --2- 172.21.15.114:0/3264074243 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5652d6fb2d00 0x5652d6fb5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:20.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.681+0000 7f3de4a81640 1 --2- 172.21.15.114:0/3264074243 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5652d6fb2d00 0x5652d6fb5120 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7ac1df50b18c798 server_cookie=32381e973be1151b in_seq=0 out_seq=0 2024-09-13T18:25:20.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.681+0000 7f3dbffff640 10 client.26059 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:25:20.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.682+0000 7f3dbffff640 1 -- 172.21.15.114:0/3264074243 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5652d6ee51f0 con 0x5652d6fb2d00 2024-09-13T18:25:20.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.682+0000 7f3dbffff640 10 client.26059 handle_command_reply: tid=0 2024-09-13T18:25:20.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.682+0000 7f3de9a07740 1 client.26059 shutdown 2024-09-13T18:25:20.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.682+0000 7f3de9a07740 1 -- 172.21.15.114:0/3264074243 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652d6fa3340 msgr2=0x5652d6fa0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:20.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.682+0000 7f3de9a07740 1 --2- 172.21.15.114:0/3264074243 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652d6fa3340 0x5652d6fa0190 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8030980 tx=0x7f3dd802a8f0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.682+0000 7f3de9a07740 1 -- 172.21.15.114:0/3264074243 shutdown_connections 2024-09-13T18:25:20.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.682+0000 7f3de9a07740 1 --2- 172.21.15.114:0/3264074243 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5652d6fb2d00 0x5652d6fb5120 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.682+0000 7f3de9a07740 1 --2- 172.21.15.114:0/3264074243 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652d6fa3340 0x5652d6fa0190 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.682+0000 7f3de9a07740 1 --2- 172.21.15.114:0/3264074243 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5652d6f901b0 0x5652d6f9fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.682+0000 7f3de9a07740 1 --2- 172.21.15.114:0/3264074243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652d6f8f890 0x5652d6fa2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.682+0000 7f3de9a07740 1 -- 172.21.15.114:0/3264074243 >> 172.21.15.114:0/3264074243 conn(0x5652d6f8bee0 msgr2=0x5652d6f8b230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:20.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.682+0000 7f3dbffff640 0 client.26059 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:25:20.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.685+0000 7f3de9a07740 1 -- 172.21.15.114:0/3264074243 shutdown_connections 2024-09-13T18:25:20.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.685+0000 7f3de9a07740 1 -- 172.21.15.114:0/3264074243 wait complete. 2024-09-13T18:25:20.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.686+0000 7f3de9a07740 20 client.26059 trim_cache size 0 max 16384 2024-09-13T18:25:20.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.707+0000 7f3de9a07740 1 Processor -- start 2024-09-13T18:25:20.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.707+0000 7f3de9a07740 1 -- start start 2024-09-13T18:25:20.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.707+0000 7f3de9a07740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652d6f8bee0 0x5652d6faa040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:20.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.707+0000 7f3de9a07740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652d6f8f890 0x5652d6faa580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:20.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.707+0000 7f3de9a07740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5652d6f901b0 0x5652d6f9e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:20.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.707+0000 7f3de5282640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5652d6f901b0 0x5652d6f9e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:20.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.708+0000 7f3de5282640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5652d6f901b0 0x5652d6f9e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44076/0 (socket says 172.21.15.114:44076) 2024-09-13T18:25:20.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.708+0000 7f3de4a81640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652d6f8bee0 0x5652d6faa040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:20.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.708+0000 7f3de5282640 1 -- 172.21.15.114:0/3686036173 learned_addr learned my addr 172.21.15.114:0/3686036173 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:20.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.708+0000 7f3de9a07740 1 -- 172.21.15.114:0/3686036173 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652d6f9fa50 con 0x5652d6f901b0 2024-09-13T18:25:20.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.708+0000 7f3de9a07740 1 -- 172.21.15.114:0/3686036173 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652d6f8f160 con 0x5652d6f8f890 2024-09-13T18:25:20.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.708+0000 7f3de9a07740 1 -- 172.21.15.114:0/3686036173 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652d6fba330 con 0x5652d6f8bee0 2024-09-13T18:25:20.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.708+0000 7f3ddffff640 1 --2- 172.21.15.114:0/3686036173 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652d6f8f890 0x5652d6faa580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:20.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.708+0000 7f3de5282640 1 -- 172.21.15.114:0/3686036173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652d6f8bee0 msgr2=0x5652d6faa040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:20.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.708+0000 7f3de5282640 1 --2- 172.21.15.114:0/3686036173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652d6f8bee0 0x5652d6faa040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.708+0000 7f3de5282640 1 -- 172.21.15.114:0/3686036173 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652d6f8f890 msgr2=0x5652d6faa580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:20.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.708+0000 7f3de5282640 1 --2- 172.21.15.114:0/3686036173 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652d6f8f890 0x5652d6faa580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.708+0000 7f3de5282640 1 -- 172.21.15.114:0/3686036173 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652d6e93770 con 0x5652d6f901b0 2024-09-13T18:25:20.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.709+0000 7f3de5282640 1 --2- 172.21.15.114:0/3686036173 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5652d6f901b0 0x5652d6f9e500 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8053f10 tx=0x7f3dd8030f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=203cf91905187098 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:20.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.709+0000 7f3dd77fe640 1 -- 172.21.15.114:0/3686036173 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd8030520 con 0x5652d6f901b0 2024-09-13T18:25:20.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.709+0000 7f3dd77fe640 1 -- 172.21.15.114:0/3686036173 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3dd8049850 con 0x5652d6f901b0 2024-09-13T18:25:20.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.709+0000 7f3dd77fe640 1 -- 172.21.15.114:0/3686036173 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd8004490 con 0x5652d6f901b0 2024-09-13T18:25:20.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.710+0000 7f3de9a07740 1 -- 172.21.15.114:0/3686036173 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5652d6f901b0 msgr2=0x5652d6f9e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:20.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.710+0000 7f3de9a07740 1 --2- 172.21.15.114:0/3686036173 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5652d6f901b0 0x5652d6f9e500 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8053f10 tx=0x7f3dd8030f40 comp rx=0 tx=0).stop 2024-09-13T18:25:20.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.710+0000 7f3de9a07740 1 -- 172.21.15.114:0/3686036173 shutdown_connections 2024-09-13T18:25:20.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.710+0000 7f3de9a07740 1 --2- 172.21.15.114:0/3686036173 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5652d6f901b0 0x5652d6f9e500 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.711+0000 7f3de9a07740 1 --2- 172.21.15.114:0/3686036173 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652d6f8f890 0x5652d6faa580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.711+0000 7f3de9a07740 1 --2- 172.21.15.114:0/3686036173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652d6f8bee0 0x5652d6faa040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.711+0000 7f3de9a07740 1 -- 172.21.15.114:0/3686036173 >> 172.21.15.114:0/3686036173 conn(0x5652d6fa2a90 msgr2=0x5652d6f8c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:20.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.711+0000 7f3de9a07740 1 -- 172.21.15.114:0/3686036173 shutdown_connections 2024-09-13T18:25:20.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.711+0000 7f3de9a07740 1 -- 172.21.15.114:0/3686036173 wait complete. 2024-09-13T18:25:20.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.712+0000 7f3de9a07740 1 -- start start 2024-09-13T18:25:20.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.712+0000 7f3de9a07740 1 Processor -- start 2024-09-13T18:25:20.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.712+0000 7f3de9a07740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652d6f8bee0 0x5652d6faa360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:20.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.712+0000 7f3de9a07740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5652d6f8f890 0x5652d6fa87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:20.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.712+0000 7f3de9a07740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652d6f901b0 0x5652d6fa8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:20.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.713+0000 7f3de9a07740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652d6fa93c0 con 0x5652d6f8f890 2024-09-13T18:25:20.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.713+0000 7f3de9a07740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652d6fa9530 con 0x5652d6f8bee0 2024-09-13T18:25:20.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.713+0000 7f3de9a07740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652d7009210 con 0x5652d6f901b0 2024-09-13T18:25:20.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.713+0000 7f3de9a07740 10 client.0 _ll_register_callbacks cb 0x7f3dd000a630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:25:20.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.713+0000 7f3de4a81640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652d6f8bee0 0x5652d6faa360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:20.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.713+0000 7f3de5282640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652d6f901b0 0x5652d6fa8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:20.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.713+0000 7f3de4a81640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652d6f8bee0 0x5652d6faa360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:43938/0 (socket says 172.21.15.114:43938) 2024-09-13T18:25:20.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.713+0000 7f3de5282640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652d6f901b0 0x5652d6fa8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:38302/0 (socket says 172.21.15.114:38302) 2024-09-13T18:25:20.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.713+0000 7f3de4a81640 1 -- 172.21.15.114:0/2313633731 learned_addr learned my addr 172.21.15.114:0/2313633731 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:20.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.713+0000 7f3ddffff640 1 --2- 172.21.15.114:0/2313633731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5652d6f8f890 0x5652d6fa87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:20.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.713+0000 7f3de4a81640 1 -- 172.21.15.114:0/2313633731 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652d6f901b0 msgr2=0x5652d6fa8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:20.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.713+0000 7f3de4a81640 1 --2- 172.21.15.114:0/2313633731 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652d6f901b0 0x5652d6fa8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.713+0000 7f3de4a81640 1 -- 172.21.15.114:0/2313633731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5652d6f8f890 msgr2=0x5652d6fa87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:20.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.713+0000 7f3de4a81640 1 --2- 172.21.15.114:0/2313633731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5652d6f8f890 0x5652d6fa87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.713+0000 7f3de4a81640 1 -- 172.21.15.114:0/2313633731 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5652d6ef13c0 con 0x5652d6f8bee0 2024-09-13T18:25:20.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.713+0000 7f3de5282640 1 --2- 172.21.15.114:0/2313633731 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652d6f901b0 0x5652d6fa8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:25:20.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.714+0000 7f3de4a81640 1 --2- 172.21.15.114:0/2313633731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652d6f8bee0 0x5652d6faa360 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3dc800c6a0 tx=0x7f3dc800c7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ede51f630374342 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:20.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.714+0000 7f3dbffff640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:25:20.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.714+0000 7f3dbffff640 1 -- 172.21.15.114:0/2313633731 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc809dfc0 con 0x5652d6f8bee0 2024-09-13T18:25:20.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.714+0000 7f3dbffff640 1 -- 172.21.15.114:0/2313633731 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f3dc80a2610 con 0x5652d6f8bee0 2024-09-13T18:25:20.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.714+0000 7f3de9a07740 1 -- 172.21.15.114:0/2313633731 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3dd0018c30 con 0x5652d6f8bee0 2024-09-13T18:25:20.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.714+0000 7f3dbffff640 1 -- 172.21.15.114:0/2313633731 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3dc80968a0 con 0x5652d6f8bee0 2024-09-13T18:25:20.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.714+0000 7f3de9a07740 10 client.26024 fetch_fsmap learned FSMap version 32 2024-09-13T18:25:20.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.714+0000 7f3de9a07740 1 -- 172.21.15.114:0/2313633731 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5652d6eb8e80 con 0x5652d6f8bee0 2024-09-13T18:25:20.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.715+0000 7f3dbffff640 1 -- 172.21.15.114:0/2313633731 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3dd0018c30 con 0x5652d6f8bee0 2024-09-13T18:25:20.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.715+0000 7f3dbffff640 1 -- 172.21.15.114:0/2313633731 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f3dc801f060 con 0x5652d6f8bee0 2024-09-13T18:25:20.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.715+0000 7f3de9a07740 10 client.26024 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:25:20.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.715+0000 7f3de9a07740 10 client.26024 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:25:20.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.715+0000 7f3de9a07740 20 client.26024 populate_metadata read hostname 'smithi114' 2024-09-13T18:25:20.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.715+0000 7f3de9a07740 1 --2- 172.21.15.114:0/2313633731 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5652d6fa3340 0x5652d700c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:20.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.715+0000 7f3de9a07740 4 client.26024 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:25:20.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.715+0000 7f3de9a07740 1 -- 172.21.15.114:0/2313633731 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5652d6be64e0 con 0x5652d6fa3340 2024-09-13T18:25:20.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.715+0000 7f3ddffff640 1 --2- 172.21.15.114:0/2313633731 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5652d6fa3340 0x5652d700c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:20.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.716+0000 7f3dbffff640 10 client.26024 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:25:20.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.716+0000 7f3ddffff640 1 --2- 172.21.15.114:0/2313633731 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5652d6fa3340 0x5652d700c9d0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3db552641f940126 server_cookie=bb160c3ddca61104 in_seq=0 out_seq=0 2024-09-13T18:25:20.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.716+0000 7f3dbffff640 1 -- 172.21.15.114:0/2313633731 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5652d6be64e0 con 0x5652d6fa3340 2024-09-13T18:25:20.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.716+0000 7f3dbffff640 10 client.26024 handle_command_reply: tid=0 2024-09-13T18:25:20.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.716+0000 7f3de9a07740 1 client.26024 shutdown 2024-09-13T18:25:20.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.717+0000 7f3de9a07740 1 -- 172.21.15.114:0/2313633731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652d6f8bee0 msgr2=0x5652d6faa360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:20.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.717+0000 7f3de9a07740 1 --2- 172.21.15.114:0/2313633731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652d6f8bee0 0x5652d6faa360 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3dc800c6a0 tx=0x7f3dc800c7d0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.717+0000 7f3de9a07740 1 -- 172.21.15.114:0/2313633731 shutdown_connections 2024-09-13T18:25:20.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.717+0000 7f3de9a07740 1 --2- 172.21.15.114:0/2313633731 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5652d6fa3340 0x5652d700c9d0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.717+0000 7f3de9a07740 1 --2- 172.21.15.114:0/2313633731 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652d6f901b0 0x5652d6fa8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.717+0000 7f3de9a07740 1 --2- 172.21.15.114:0/2313633731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5652d6f8f890 0x5652d6fa87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.717+0000 7f3de9a07740 1 --2- 172.21.15.114:0/2313633731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652d6f8bee0 0x5652d6faa360 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.717+0000 7f3de9a07740 1 -- 172.21.15.114:0/2313633731 >> 172.21.15.114:0/2313633731 conn(0x5652d6fa2a90 msgr2=0x5652d6f9ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:20.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.718+0000 7f3dbffff640 0 client.26024 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:25:20.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.718+0000 7f3de9a07740 1 -- 172.21.15.114:0/2313633731 shutdown_connections 2024-09-13T18:25:20.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.718+0000 7f3de9a07740 1 -- 172.21.15.114:0/2313633731 wait complete. 2024-09-13T18:25:20.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.718+0000 7f3de9a07740 20 client.26024 trim_cache size 0 max 16384 2024-09-13T18:25:20.723 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:25:20.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.719+0000 7f3de5a83640 1 -- 172.21.15.114:0/797913739 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f3dcc0a1160 msgr2=0x7f3dcc0a3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:20.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.719+0000 7f3de5a83640 1 --2- 172.21.15.114:0/797913739 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f3dcc0a1160 0x7f3dcc0a3620 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8009430 tx=0x7f3dd8009360 comp rx=0 tx=0).stop 2024-09-13T18:25:20.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.719+0000 7f3de5a83640 1 -- 172.21.15.114:0/797913739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3de00ffb20 msgr2=0x7f3de01982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:20.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.719+0000 7f3de5a83640 1 --2- 172.21.15.114:0/797913739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3de00ffb20 0x7f3de01982b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3dc800e9f0 tx=0x7f3dc800edf0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.719+0000 7f3de5a83640 1 -- 172.21.15.114:0/797913739 shutdown_connections 2024-09-13T18:25:20.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.719+0000 7f3de5a83640 1 --2- 172.21.15.114:0/797913739 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f3dcc0a1160 0x7f3dcc0a3620 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.719+0000 7f3de5a83640 1 --2- 172.21.15.114:0/797913739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3de019cbe0 0x7f3de019d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.719+0000 7f3de5a83640 1 --2- 172.21.15.114:0/797913739 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3de01050d0 0x7f3de01987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.719+0000 7f3de5a83640 1 --2- 172.21.15.114:0/797913739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3de00ffb20 0x7f3de01982b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:20.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.719+0000 7f3de5a83640 1 -- 172.21.15.114:0/797913739 >> 172.21.15.114:0/797913739 conn(0x7f3de0075380 msgr2=0x7f3de00fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:20.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.719+0000 7f3de5a83640 1 -- 172.21.15.114:0/797913739 shutdown_connections 2024-09-13T18:25:20.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:20.719+0000 7f3de5a83640 1 -- 172.21.15.114:0/797913739 wait complete. 2024-09-13T18:25:20.868 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:20 smithi134 ceph-mon[34283]: pgmap v89: 129 pgs: 129 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 77 MiB/s wr, 1.24k op/s 2024-09-13T18:25:20.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:20 smithi204 ceph-mon[34777]: pgmap v89: 129 pgs: 129 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 77 MiB/s wr, 1.24k op/s 2024-09-13T18:25:21.747 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-13T18:25:21.747 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-09-13T18:25:21.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.924+0000 7f82d7c2b640 1 -- 172.21.15.114:0/73021892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82d00fe5e0 msgr2=0x7f82d00fe9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:21.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.924+0000 7f82d7c2b640 1 --2- 172.21.15.114:0/73021892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82d00fe5e0 0x7f82d00fe9c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f82c4009900 tx=0x7f82c402a410 comp rx=0 tx=0).stop 2024-09-13T18:25:21.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.925+0000 7f82d7c2b640 1 -- 172.21.15.114:0/73021892 shutdown_connections 2024-09-13T18:25:21.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.925+0000 7f82d7c2b640 1 --2- 172.21.15.114:0/73021892 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82d0103730 0x7f82d010b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:21.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.925+0000 7f82d7c2b640 1 --2- 172.21.15.114:0/73021892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82d00fef90 0x7f82d0102fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:21.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.925+0000 7f82d7c2b640 1 --2- 172.21.15.114:0/73021892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82d00fe5e0 0x7f82d00fe9c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:21.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.925+0000 7f82d7c2b640 1 -- 172.21.15.114:0/73021892 >> 172.21.15.114:0/73021892 conn(0x7f82d00753f0 msgr2=0x7f82d00fc740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:21.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.925+0000 7f82d7c2b640 1 -- 172.21.15.114:0/73021892 shutdown_connections 2024-09-13T18:25:21.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.926+0000 7f82d7c2b640 1 -- 172.21.15.114:0/73021892 wait complete. 2024-09-13T18:25:21.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.927+0000 7f82d7c2b640 1 Processor -- start 2024-09-13T18:25:21.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.927+0000 7f82d7c2b640 1 -- start start 2024-09-13T18:25:21.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.927+0000 7f82d7c2b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82d00fe5e0 0x7f82d0113d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:21.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.927+0000 7f82d7c2b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82d0103730 0x7f82d01142d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:21.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.927+0000 7f82d7c2b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82d010ded0 0x7f82d010e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:21.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.927+0000 7f82d7c2b640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d010eae0 con 0x7f82d0103730 2024-09-13T18:25:21.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.927+0000 7f82d7c2b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d010ec20 con 0x7f82d010ded0 2024-09-13T18:25:21.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.927+0000 7f82d7c2b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d0069ed0 con 0x7f82d00fe5e0 2024-09-13T18:25:21.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.928+0000 7f82d59c7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82d00fe5e0 0x7f82d0113d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:21.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.928+0000 7f82d51c6640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82d0103730 0x7f82d01142d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:21.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.928+0000 7f82d51c6640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82d0103730 0x7f82d01142d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44110/0 (socket says 172.21.15.114:44110) 2024-09-13T18:25:21.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.928+0000 7f82d51c6640 1 -- 172.21.15.114:0/878496337 learned_addr learned my addr 172.21.15.114:0/878496337 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:21.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.928+0000 7f82d61c8640 1 --2- 172.21.15.114:0/878496337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82d010ded0 0x7f82d010e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:21.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.928+0000 7f82d61c8640 1 -- 172.21.15.114:0/878496337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82d00fe5e0 msgr2=0x7f82d0113d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:21.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.928+0000 7f82d61c8640 1 --2- 172.21.15.114:0/878496337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82d00fe5e0 0x7f82d0113d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:21.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.928+0000 7f82d61c8640 1 -- 172.21.15.114:0/878496337 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82d0103730 msgr2=0x7f82d01142d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:21.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.928+0000 7f82d61c8640 1 --2- 172.21.15.114:0/878496337 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82d0103730 0x7f82d01142d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:21.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.928+0000 7f82d61c8640 1 -- 172.21.15.114:0/878496337 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c4009560 con 0x7f82d010ded0 2024-09-13T18:25:21.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.929+0000 7f82d61c8640 1 --2- 172.21.15.114:0/878496337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82d010ded0 0x7f82d010e350 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f82c800e9d0 tx=0x7f82c800ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:21.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.929+0000 7f82c2ffd640 1 -- 172.21.15.114:0/878496337 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c800cde0 con 0x7f82d010ded0 2024-09-13T18:25:21.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.929+0000 7f82d7c2b640 1 -- 172.21.15.114:0/878496337 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82d006a1b0 con 0x7f82d010ded0 2024-09-13T18:25:21.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.929+0000 7f82d7c2b640 1 -- 172.21.15.114:0/878496337 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82d006a6c0 con 0x7f82d010ded0 2024-09-13T18:25:21.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.929+0000 7f82c2ffd640 1 -- 172.21.15.114:0/878496337 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f82c8005020 con 0x7f82d010ded0 2024-09-13T18:25:21.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.929+0000 7f82c2ffd640 1 -- 172.21.15.114:0/878496337 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c8010690 con 0x7f82d010ded0 2024-09-13T18:25:21.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.932+0000 7f82c2ffd640 1 -- 172.21.15.114:0/878496337 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f82c8004b20 con 0x7f82d010ded0 2024-09-13T18:25:21.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.933+0000 7f82daa29740 1 Processor -- start 2024-09-13T18:25:21.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.933+0000 7f82daa29740 1 -- start start 2024-09-13T18:25:21.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.933+0000 7f82daa29740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d1ee937c0 0x555d1ee93ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:21.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.933+0000 7f82daa29740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555d1ee940e0 0x555d1ee9e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:21.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.934+0000 7f82daa29740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1ee9e980 0x555d1eea0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:21.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.934+0000 7f82daa29740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d1edfec60 con 0x555d1ee940e0 2024-09-13T18:25:21.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.934+0000 7f82daa29740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d1edfc3f0 con 0x555d1ee937c0 2024-09-13T18:25:21.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.934+0000 7f82daa29740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d1edf7ce0 con 0x555d1ee9e980 2024-09-13T18:25:21.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.934+0000 7f82d51c6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d1ee937c0 0x555d1ee93ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:21.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.934+0000 7f82d51c6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d1ee937c0 0x555d1ee93ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:43962/0 (socket says 172.21.15.114:43962) 2024-09-13T18:25:21.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.934+0000 7f82d51c6640 1 -- 172.21.15.114:0/884320806 learned_addr learned my addr 172.21.15.114:0/884320806 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:21.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.934+0000 7f82c2ffd640 1 --2- 172.21.15.114:0/878496337 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f82bc0a1410 0x7f82bc0a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:21.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.934+0000 7f82d51c6640 1 --2- 172.21.15.114:0/884320806 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1ee9e980 0x555d1eea0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:21.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.934+0000 7f82d59c7640 1 --2- 172.21.15.114:0/884320806 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555d1ee940e0 0x555d1ee9e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:21.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.934+0000 7f82c2ffd640 1 -- 172.21.15.114:0/878496337 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f82c8014070 con 0x7f82d010ded0 2024-09-13T18:25:21.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.934+0000 7f82d51c6640 1 -- 172.21.15.114:0/884320806 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d1ee937c0 msgr2=0x555d1ee93ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:21.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.934+0000 7f82d51c6640 1 --2- 172.21.15.114:0/884320806 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d1ee937c0 0x555d1ee93ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:21.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.934+0000 7f82d51c6640 1 -- 172.21.15.114:0/884320806 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555d1ee940e0 msgr2=0x555d1ee9e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:21.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.934+0000 7f82d51c6640 1 --2- 172.21.15.114:0/884320806 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555d1ee940e0 0x555d1ee9e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:21.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.934+0000 7f82d51c6640 1 -- 172.21.15.114:0/884320806 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d1edb3970 con 0x555d1ee9e980 2024-09-13T18:25:21.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.934+0000 7f82d61c8640 1 --2- 172.21.15.114:0/878496337 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f82bc0a1410 0x7f82bc0a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:21.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.935+0000 7f82d51c6640 1 --2- 172.21.15.114:0/884320806 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1ee9e980 0x555d1eea0d70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f82cc017700 tx=0x7f82cc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cd9076d858ae60c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:21.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.935+0000 7f82bbfff640 1 -- 172.21.15.114:0/884320806 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82cc015960 con 0x555d1ee9e980 2024-09-13T18:25:21.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.935+0000 7f82bbfff640 1 -- 172.21.15.114:0/884320806 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f82cc0183f0 con 0x555d1ee9e980 2024-09-13T18:25:21.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.935+0000 7f82bbfff640 1 -- 172.21.15.114:0/884320806 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82cc018ea0 con 0x555d1ee9e980 2024-09-13T18:25:21.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.936+0000 7f82d61c8640 1 --2- 172.21.15.114:0/878496337 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f82bc0a1410 0x7f82bc0a38d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f82c801a040 tx=0x7f82c8005f70 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:21.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.936+0000 7f82daa29740 1 -- 172.21.15.114:0/884320806 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1ee9e980 msgr2=0x555d1eea0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:21.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.936+0000 7f82daa29740 1 --2- 172.21.15.114:0/884320806 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1ee9e980 0x555d1eea0d70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f82cc017700 tx=0x7f82cc0046c0 comp rx=0 tx=0).stop 2024-09-13T18:25:21.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.936+0000 7f82daa29740 1 -- 172.21.15.114:0/884320806 shutdown_connections 2024-09-13T18:25:21.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.936+0000 7f82daa29740 1 --2- 172.21.15.114:0/884320806 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1ee9e980 0x555d1eea0d70 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:21.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.936+0000 7f82daa29740 1 --2- 172.21.15.114:0/884320806 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555d1ee940e0 0x555d1ee9e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:21.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.936+0000 7f82daa29740 1 --2- 172.21.15.114:0/884320806 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d1ee937c0 0x555d1ee93ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:21.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.936+0000 7f82daa29740 1 -- 172.21.15.114:0/884320806 >> 172.21.15.114:0/884320806 conn(0x555d1ee8fd50 msgr2=0x555d1ee90130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:21.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.936+0000 7f82daa29740 1 -- 172.21.15.114:0/884320806 shutdown_connections 2024-09-13T18:25:21.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.936+0000 7f82daa29740 1 -- 172.21.15.114:0/884320806 wait complete. 2024-09-13T18:25:21.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.937+0000 7f82daa29740 1 -- start start 2024-09-13T18:25:21.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.937+0000 7f82daa29740 1 Processor -- start 2024-09-13T18:25:21.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.937+0000 7f82daa29740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1ee937c0 0x555d1ee9b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:21.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.938+0000 7f82daa29740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d1ee940e0 0x555d1ee9bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:21.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.938+0000 7f82daa29740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555d1ee9e980 0x555d1ee9c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:21.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.938+0000 7f82d51c6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1ee937c0 0x555d1ee9b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:21.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.938+0000 7f82daa29740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d1edb5f80 con 0x555d1ee9e980 2024-09-13T18:25:21.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.938+0000 7f82daa29740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d1edb5c20 con 0x555d1ee940e0 2024-09-13T18:25:21.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.938+0000 7f82daa29740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d1ed82ab0 con 0x555d1ee937c0 2024-09-13T18:25:21.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.938+0000 7f82d51c6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1ee937c0 0x555d1ee9b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:38340/0 (socket says 172.21.15.114:38340) 2024-09-13T18:25:21.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.938+0000 7f82d51c6640 1 -- 172.21.15.114:0/571135319 learned_addr learned my addr 172.21.15.114:0/571135319 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:21.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.938+0000 7f82d59c7640 1 --2- 172.21.15.114:0/571135319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d1ee940e0 0x555d1ee9bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:21.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.938+0000 7f82daa29740 10 client.0 _ll_register_callbacks cb 0x555d1edf1350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:25:21.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.938+0000 7f82d51c6640 1 --2- 172.21.15.114:0/571135319 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555d1ee9e980 0x555d1ee9c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:21.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.938+0000 7f82d51c6640 1 -- 172.21.15.114:0/571135319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d1ee940e0 msgr2=0x555d1ee9bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:21.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.938+0000 7f82d51c6640 1 --2- 172.21.15.114:0/571135319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d1ee940e0 0x555d1ee9bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:21.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.938+0000 7f82d51c6640 1 -- 172.21.15.114:0/571135319 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555d1ee9e980 msgr2=0x555d1ee9c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:21.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.938+0000 7f82d51c6640 1 --2- 172.21.15.114:0/571135319 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555d1ee9e980 0x555d1ee9c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:21.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.938+0000 7f82d51c6640 1 -- 172.21.15.114:0/571135319 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82cc0173e0 con 0x555d1ee937c0 2024-09-13T18:25:21.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.939+0000 7f82d51c6640 1 --2- 172.21.15.114:0/571135319 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1ee937c0 0x555d1ee9b750 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f82cc004810 tx=0x7f82cc018d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e587d558db8a43c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:21.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.939+0000 7f82b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:25:21.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.939+0000 7f82b97fa640 1 -- 172.21.15.114:0/571135319 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82cc02b070 con 0x555d1ee937c0 2024-09-13T18:25:21.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.939+0000 7f82b97fa640 1 -- 172.21.15.114:0/571135319 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f82cc000e40 con 0x555d1ee937c0 2024-09-13T18:25:21.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.939+0000 7f82daa29740 1 -- 172.21.15.114:0/571135319 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555d1eaeb590 con 0x555d1ee937c0 2024-09-13T18:25:21.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.939+0000 7f82b97fa640 1 -- 172.21.15.114:0/571135319 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82cc018770 con 0x555d1ee937c0 2024-09-13T18:25:21.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.939+0000 7f82daa29740 10 client.26074 fetch_fsmap learned FSMap version 32 2024-09-13T18:25:21.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.940+0000 7f82daa29740 1 -- 172.21.15.114:0/571135319 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x555d1edf6360 con 0x555d1ee937c0 2024-09-13T18:25:21.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.940+0000 7f82b97fa640 1 -- 172.21.15.114:0/571135319 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x555d1eaeb590 con 0x555d1ee937c0 2024-09-13T18:25:21.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.940+0000 7f82b97fa640 1 -- 172.21.15.114:0/571135319 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f82cc021580 con 0x555d1ee937c0 2024-09-13T18:25:21.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.940+0000 7f82daa29740 10 client.26074 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:25:21.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.940+0000 7f82daa29740 10 client.26074 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:25:21.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.940+0000 7f82daa29740 20 client.26074 populate_metadata read hostname 'smithi114' 2024-09-13T18:25:21.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.940+0000 7f82daa29740 1 --2- 172.21.15.114:0/571135319 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x555d1eeae670 0x555d1eeb0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:21.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.940+0000 7f82daa29740 4 client.26074 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:25:21.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.940+0000 7f82daa29740 1 -- 172.21.15.114:0/571135319 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555d1eafc860 con 0x555d1eeae670 2024-09-13T18:25:21.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.940+0000 7f82d59c7640 1 --2- 172.21.15.114:0/571135319 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x555d1eeae670 0x555d1eeb0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:21.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.941+0000 7f82d59c7640 1 --2- 172.21.15.114:0/571135319 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x555d1eeae670 0x555d1eeb0a90 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95ad8bf812ccc2c3 server_cookie=7677bb4983748182 in_seq=0 out_seq=0 2024-09-13T18:25:21.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.941+0000 7f82b97fa640 10 client.26074 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:25:21.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.942+0000 7f82b97fa640 1 -- 172.21.15.114:0/571135319 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x555d1eafc860 con 0x555d1eeae670 2024-09-13T18:25:21.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.942+0000 7f82b97fa640 10 client.26074 handle_command_reply: tid=0 2024-09-13T18:25:21.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.942+0000 7f82daa29740 1 client.26074 shutdown 2024-09-13T18:25:21.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.943+0000 7f82daa29740 1 -- 172.21.15.114:0/571135319 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1ee937c0 msgr2=0x555d1ee9b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:21.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.943+0000 7f82daa29740 1 --2- 172.21.15.114:0/571135319 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1ee937c0 0x555d1ee9b750 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f82cc004810 tx=0x7f82cc018d80 comp rx=0 tx=0).stop 2024-09-13T18:25:21.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.943+0000 7f82daa29740 1 -- 172.21.15.114:0/571135319 shutdown_connections 2024-09-13T18:25:21.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.943+0000 7f82daa29740 1 --2- 172.21.15.114:0/571135319 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x555d1eeae670 0x555d1eeb0a90 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:21.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.943+0000 7f82daa29740 1 --2- 172.21.15.114:0/571135319 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555d1ee9e980 0x555d1ee9c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:21.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.943+0000 7f82daa29740 1 --2- 172.21.15.114:0/571135319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d1ee940e0 0x555d1ee9bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:21.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.943+0000 7f82daa29740 1 --2- 172.21.15.114:0/571135319 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1ee937c0 0x555d1ee9b750 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:21.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.943+0000 7f82daa29740 1 -- 172.21.15.114:0/571135319 >> 172.21.15.114:0/571135319 conn(0x555d1ee8fd50 msgr2=0x555d1ee8f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:21.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.943+0000 7f82b97fa640 0 client.26074 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:25:21.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.943+0000 7f82daa29740 1 -- 172.21.15.114:0/571135319 shutdown_connections 2024-09-13T18:25:21.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.943+0000 7f82daa29740 1 -- 172.21.15.114:0/571135319 wait complete. 2024-09-13T18:25:21.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.943+0000 7f82daa29740 20 client.26074 trim_cache size 0 max 16384 2024-09-13T18:25:21.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.973+0000 7f82daa29740 1 Processor -- start 2024-09-13T18:25:21.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.973+0000 7f82daa29740 1 -- start start 2024-09-13T18:25:21.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.973+0000 7f82daa29740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d1ee937c0 0x555d1eea56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:21.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.973+0000 7f82daa29740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555d1ee940e0 0x555d1eea5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:21.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.973+0000 7f82daa29740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1ee9e980 0x555d1ee98990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:21.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.973+0000 7f82daa29740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d1ee93610 con 0x555d1ee940e0 2024-09-13T18:25:21.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.973+0000 7f82daa29740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d1eea1430 con 0x555d1ee937c0 2024-09-13T18:25:21.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.973+0000 7f82daa29740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d1ee93ed0 con 0x555d1ee9e980 2024-09-13T18:25:21.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.973+0000 7f82d51c6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d1ee937c0 0x555d1eea56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:21.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.973+0000 7f82d51c6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d1ee937c0 0x555d1eea56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:43986/0 (socket says 172.21.15.114:43986) 2024-09-13T18:25:21.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.973+0000 7f82d51c6640 1 -- 172.21.15.114:0/2504339420 learned_addr learned my addr 172.21.15.114:0/2504339420 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:21.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.973+0000 7f82d51c6640 1 --2- 172.21.15.114:0/2504339420 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1ee9e980 0x555d1ee98990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:21.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.974+0000 7f82d51c6640 1 -- 172.21.15.114:0/2504339420 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1ee9e980 msgr2=0x555d1ee98990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:21.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.974+0000 7f82d51c6640 1 --2- 172.21.15.114:0/2504339420 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1ee9e980 0x555d1ee98990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:21.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.974+0000 7f82d51c6640 1 -- 172.21.15.114:0/2504339420 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555d1ee940e0 msgr2=0x555d1eea5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:21.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.974+0000 7f82d51c6640 1 --2- 172.21.15.114:0/2504339420 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555d1ee940e0 0x555d1eea5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:21.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.974+0000 7f82d51c6640 1 -- 172.21.15.114:0/2504339420 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d1ed981c0 con 0x555d1ee937c0 2024-09-13T18:25:21.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.974+0000 7f82d51c6640 1 --2- 172.21.15.114:0/2504339420 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d1ee937c0 0x555d1eea56a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f82cc021b50 tx=0x7f82cc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=199fbfd027557fb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:21.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.975+0000 7f82c0e05640 1 -- 172.21.15.114:0/2504339420 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82cc02b070 con 0x555d1ee937c0 2024-09-13T18:25:21.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.975+0000 7f82c0e05640 1 -- 172.21.15.114:0/2504339420 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f82cc02c040 con 0x555d1ee937c0 2024-09-13T18:25:21.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.975+0000 7f82c0e05640 1 -- 172.21.15.114:0/2504339420 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82cc021780 con 0x555d1ee937c0 2024-09-13T18:25:21.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.976+0000 7f82daa29740 1 -- 172.21.15.114:0/2504339420 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d1ee937c0 msgr2=0x555d1eea56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:21.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.976+0000 7f82daa29740 1 --2- 172.21.15.114:0/2504339420 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d1ee937c0 0x555d1eea56a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f82cc021b50 tx=0x7f82cc0046c0 comp rx=0 tx=0).stop 2024-09-13T18:25:21.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.977+0000 7f82daa29740 1 -- 172.21.15.114:0/2504339420 shutdown_connections 2024-09-13T18:25:21.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.977+0000 7f82daa29740 1 --2- 172.21.15.114:0/2504339420 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1ee9e980 0x555d1ee98990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:21.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.977+0000 7f82daa29740 1 --2- 172.21.15.114:0/2504339420 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555d1ee940e0 0x555d1eea5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:21.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.977+0000 7f82daa29740 1 --2- 172.21.15.114:0/2504339420 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d1ee937c0 0x555d1eea56a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:21.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.977+0000 7f82daa29740 1 -- 172.21.15.114:0/2504339420 >> 172.21.15.114:0/2504339420 conn(0x555d1ee8fd50 msgr2=0x555d1ee9a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:21.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.977+0000 7f82daa29740 1 -- 172.21.15.114:0/2504339420 shutdown_connections 2024-09-13T18:25:21.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.977+0000 7f82daa29740 1 -- 172.21.15.114:0/2504339420 wait complete. 2024-09-13T18:25:21.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.978+0000 7f82daa29740 1 -- start start 2024-09-13T18:25:21.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.978+0000 7f82daa29740 1 Processor -- start 2024-09-13T18:25:21.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.980+0000 7f82daa29740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d1ee937c0 0x555d1eefee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:21.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.980+0000 7f82daa29740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555d1ee940e0 0x555d1eeff3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:21.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.980+0000 7f82daa29740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1ee9e980 0x555d1eea28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:21.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.980+0000 7f82d59c7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555d1ee940e0 0x555d1eeff3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:21.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.980+0000 7f82d59c7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555d1ee940e0 0x555d1eeff3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44144/0 (socket says 172.21.15.114:44144) 2024-09-13T18:25:21.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.980+0000 7f82d51c6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d1ee937c0 0x555d1eefee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:21.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.980+0000 7f82d51c6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d1ee937c0 0x555d1eefee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:44000/0 (socket says 172.21.15.114:44000) 2024-09-13T18:25:21.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.980+0000 7f82d59c7640 1 -- 172.21.15.114:0/1720543949 learned_addr learned my addr 172.21.15.114:0/1720543949 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:21.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.981+0000 7f82daa29740 1 -- 172.21.15.114:0/1720543949 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d1eea2eb0 con 0x555d1ee940e0 2024-09-13T18:25:21.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.981+0000 7f82daa29740 1 -- 172.21.15.114:0/1720543949 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d1eea3020 con 0x555d1ee937c0 2024-09-13T18:25:21.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.981+0000 7f82daa29740 1 -- 172.21.15.114:0/1720543949 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d1eea31e0 con 0x555d1ee9e980 2024-09-13T18:25:21.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.981+0000 7f82daa29740 10 client.0 _ll_register_callbacks cb 0x7f82c4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:25:21.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.981+0000 7f82d51c6640 1 --2- 172.21.15.114:0/1720543949 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1ee9e980 0x555d1eea28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:21.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.981+0000 7f82d59c7640 1 -- 172.21.15.114:0/1720543949 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1ee9e980 msgr2=0x555d1eea28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:21.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.981+0000 7f82d59c7640 1 --2- 172.21.15.114:0/1720543949 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1ee9e980 0x555d1eea28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:21.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.981+0000 7f82d59c7640 1 -- 172.21.15.114:0/1720543949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d1ee937c0 msgr2=0x555d1eefee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:21.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.981+0000 7f82d59c7640 1 --2- 172.21.15.114:0/1720543949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d1ee937c0 0x555d1eefee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:21.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.981+0000 7f82d59c7640 1 -- 172.21.15.114:0/1720543949 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555d1edf6360 con 0x555d1ee940e0 2024-09-13T18:25:21.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.982+0000 7f82d51c6640 1 --2- 172.21.15.114:0/1720543949 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1ee9e980 0x555d1eea28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:25:21.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.983+0000 7f82d59c7640 1 --2- 172.21.15.114:0/1720543949 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555d1ee940e0 0x555d1eeff3b0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f82c4009230 tx=0x7f82c40092f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ce82c2c29b7e9ab server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:21.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.983+0000 7f82b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:25:21.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.983+0000 7f82b97fa640 1 -- 172.21.15.114:0/1720543949 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c4042040 con 0x555d1ee940e0 2024-09-13T18:25:21.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.983+0000 7f82b97fa640 1 -- 172.21.15.114:0/1720543949 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f82c4046880 con 0x555d1ee940e0 2024-09-13T18:25:21.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.983+0000 7f82daa29740 1 -- 172.21.15.114:0/1720543949 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f82cc0183f0 con 0x555d1ee940e0 2024-09-13T18:25:21.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.983+0000 7f82b97fa640 1 -- 172.21.15.114:0/1720543949 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82c4046a90 con 0x555d1ee940e0 2024-09-13T18:25:21.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.983+0000 7f82daa29740 10 client.16707 fetch_fsmap learned FSMap version 32 2024-09-13T18:25:21.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.983+0000 7f82daa29740 1 -- 172.21.15.114:0/1720543949 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x555d1edbd990 con 0x555d1ee940e0 2024-09-13T18:25:21.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.984+0000 7f82b97fa640 1 -- 172.21.15.114:0/1720543949 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f82cc0183f0 con 0x555d1ee940e0 2024-09-13T18:25:21.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.984+0000 7f82b97fa640 1 -- 172.21.15.114:0/1720543949 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f82c4045030 con 0x555d1ee940e0 2024-09-13T18:25:21.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.984+0000 7f82daa29740 10 client.16707 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:25:21.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.984+0000 7f82daa29740 10 client.16707 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:25:21.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.984+0000 7f82daa29740 20 client.16707 populate_metadata read hostname 'smithi114' 2024-09-13T18:25:21.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.984+0000 7f82daa29740 1 --2- 172.21.15.114:0/1720543949 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x555d1ef06de0 0x555d1ef09200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:21.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.984+0000 7f82daa29740 4 client.16707 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-13T18:25:21.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.984+0000 7f82daa29740 1 -- 172.21.15.114:0/1720543949 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x555d1eb4feb0 con 0x555d1ef06de0 2024-09-13T18:25:21.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.984+0000 7f82d51c6640 1 --2- 172.21.15.114:0/1720543949 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x555d1ef06de0 0x555d1ef09200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:21.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.985+0000 7f82d51c6640 1 --2- 172.21.15.114:0/1720543949 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x555d1ef06de0 0x555d1ef09200 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94e34fb6db82142a server_cookie=1a2c9f0c6649bb9d in_seq=0 out_seq=0 2024-09-13T18:25:21.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.985+0000 7f82b97fa640 10 client.16707 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:25:21.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.986+0000 7f82b97fa640 1 -- 172.21.15.114:0/1720543949 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x555d1eb4feb0 con 0x555d1ef06de0 2024-09-13T18:25:21.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.986+0000 7f82b97fa640 10 client.16707 handle_command_reply: tid=0 2024-09-13T18:25:21.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.986+0000 7f82daa29740 1 client.16707 shutdown 2024-09-13T18:25:21.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.986+0000 7f82daa29740 1 -- 172.21.15.114:0/1720543949 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555d1ee940e0 msgr2=0x555d1eeff3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:21.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.986+0000 7f82daa29740 1 --2- 172.21.15.114:0/1720543949 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555d1ee940e0 0x555d1eeff3b0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f82c4009230 tx=0x7f82c40092f0 comp rx=0 tx=0).stop 2024-09-13T18:25:21.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.987+0000 7f82daa29740 1 -- 172.21.15.114:0/1720543949 shutdown_connections 2024-09-13T18:25:21.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.987+0000 7f82daa29740 1 --2- 172.21.15.114:0/1720543949 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x555d1ef06de0 0x555d1ef09200 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:21.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.987+0000 7f82daa29740 1 --2- 172.21.15.114:0/1720543949 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1ee9e980 0x555d1eea28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:21.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.987+0000 7f82b97fa640 0 client.16707 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:25:21.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.987+0000 7f82daa29740 1 --2- 172.21.15.114:0/1720543949 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555d1ee940e0 0x555d1eeff3b0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:21.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.987+0000 7f82daa29740 1 --2- 172.21.15.114:0/1720543949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d1ee937c0 0x555d1eefee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:21.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.987+0000 7f82daa29740 1 -- 172.21.15.114:0/1720543949 >> 172.21.15.114:0/1720543949 conn(0x555d1ee8fd50 msgr2=0x555d1ee95140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:21.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.987+0000 7f82daa29740 1 -- 172.21.15.114:0/1720543949 shutdown_connections 2024-09-13T18:25:21.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.987+0000 7f82daa29740 1 -- 172.21.15.114:0/1720543949 wait complete. 2024-09-13T18:25:21.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.988+0000 7f82daa29740 20 client.16707 trim_cache size 0 max 16384 2024-09-13T18:25:21.991 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:25:21.991 INFO:teuthology.orchestra.run.smithi114.stdout: "return_code": 0, 2024-09-13T18:25:21.991 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_tag": "8c9c32f6-25f3-45a8-b3d0-0d974760d7bd", 2024-09-13T18:25:21.991 INFO:teuthology.orchestra.run.smithi114.stdout: "mode": "asynchronous" 2024-09-13T18:25:21.991 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:25:21.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.989+0000 7f82d7c2b640 1 -- 172.21.15.114:0/878496337 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f82bc0a1410 msgr2=0x7f82bc0a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:21.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.989+0000 7f82d7c2b640 1 --2- 172.21.15.114:0/878496337 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f82bc0a1410 0x7f82bc0a38d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f82c801a040 tx=0x7f82c8005f70 comp rx=0 tx=0).stop 2024-09-13T18:25:21.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.990+0000 7f82d7c2b640 1 -- 172.21.15.114:0/878496337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82d010ded0 msgr2=0x7f82d010e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:21.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.990+0000 7f82d7c2b640 1 --2- 172.21.15.114:0/878496337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82d010ded0 0x7f82d010e350 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f82c800e9d0 tx=0x7f82c800ee90 comp rx=0 tx=0).stop 2024-09-13T18:25:21.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.990+0000 7f82d7c2b640 1 -- 172.21.15.114:0/878496337 shutdown_connections 2024-09-13T18:25:21.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.990+0000 7f82d7c2b640 1 --2- 172.21.15.114:0/878496337 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f82bc0a1410 0x7f82bc0a38d0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:21.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.990+0000 7f82d7c2b640 1 --2- 172.21.15.114:0/878496337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82d010ded0 0x7f82d010e350 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:21.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.990+0000 7f82d7c2b640 1 --2- 172.21.15.114:0/878496337 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82d0103730 0x7f82d01142d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:21.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.990+0000 7f82d7c2b640 1 --2- 172.21.15.114:0/878496337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82d00fe5e0 0x7f82d0113d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:21.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.990+0000 7f82d7c2b640 1 -- 172.21.15.114:0/878496337 >> 172.21.15.114:0/878496337 conn(0x7f82d00753f0 msgr2=0x7f82d0101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:21.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.990+0000 7f82d7c2b640 1 -- 172.21.15.114:0/878496337 shutdown_connections 2024-09-13T18:25:21.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:21.990+0000 7f82d7c2b640 1 -- 172.21.15.114:0/878496337 wait complete. 2024-09-13T18:25:22.013 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T18:25:22.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.193+0000 7fb63c5ed640 1 -- 172.21.15.114:0/1582065516 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6340fee60 msgr2=0x7fb6340ff240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:22.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.194+0000 7fb63c5ed640 1 --2- 172.21.15.114:0/1582065516 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6340fee60 0x7fb6340ff240 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb62400b8b0 tx=0x7fb624005f90 comp rx=0 tx=0).stop 2024-09-13T18:25:22.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.194+0000 7fb63c5ed640 1 -- 172.21.15.114:0/1582065516 shutdown_connections 2024-09-13T18:25:22.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.194+0000 7fb63c5ed640 1 --2- 172.21.15.114:0/1582065516 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb634103fb0 0x7fb63410bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:22.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.194+0000 7fb63c5ed640 1 --2- 172.21.15.114:0/1582065516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6340ff810 0x7fb634103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:22.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.194+0000 7fb63c5ed640 1 --2- 172.21.15.114:0/1582065516 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6340fee60 0x7fb6340ff240 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:22.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.194+0000 7fb63c5ed640 1 -- 172.21.15.114:0/1582065516 >> 172.21.15.114:0/1582065516 conn(0x7fb634075c60 msgr2=0x7fb6340fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:22.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.194+0000 7fb63c5ed640 1 -- 172.21.15.114:0/1582065516 shutdown_connections 2024-09-13T18:25:22.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.194+0000 7fb63c5ed640 1 -- 172.21.15.114:0/1582065516 wait complete. 2024-09-13T18:25:22.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.195+0000 7fb63c5ed640 1 Processor -- start 2024-09-13T18:25:22.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.195+0000 7fb63c5ed640 1 -- start start 2024-09-13T18:25:22.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.196+0000 7fb63c5ed640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6340fee60 0x7fb6341a25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:22.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.196+0000 7fb63c5ed640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb634103fb0 0x7fb6341a2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:22.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.196+0000 7fb63c5ed640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb63419c680 0x7fb63419cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:22.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.196+0000 7fb63c5ed640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63419d100 con 0x7fb634103fb0 2024-09-13T18:25:22.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.196+0000 7fb63c5ed640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63419d270 con 0x7fb63419c680 2024-09-13T18:25:22.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.196+0000 7fb63c5ed640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63419d430 con 0x7fb6340fee60 2024-09-13T18:25:22.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.196+0000 7fb639b88640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb634103fb0 0x7fb6341a2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:22.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.196+0000 7fb63a389640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6340fee60 0x7fb6341a25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:22.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.196+0000 7fb639b88640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb634103fb0 0x7fb6341a2ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44152/0 (socket says 172.21.15.114:44152) 2024-09-13T18:25:22.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.196+0000 7fb639b88640 1 -- 172.21.15.114:0/2226308391 learned_addr learned my addr 172.21.15.114:0/2226308391 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:22.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.196+0000 7fb63a389640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6340fee60 0x7fb6341a25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:38370/0 (socket says 172.21.15.114:38370) 2024-09-13T18:25:22.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.196+0000 7fb63ab8a640 1 --2- 172.21.15.114:0/2226308391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb63419c680 0x7fb63419cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:22.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.197+0000 7fb63ab8a640 1 -- 172.21.15.114:0/2226308391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6340fee60 msgr2=0x7fb6341a25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:22.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.197+0000 7fb63ab8a640 1 --2- 172.21.15.114:0/2226308391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6340fee60 0x7fb6341a25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:22.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.197+0000 7fb63ab8a640 1 -- 172.21.15.114:0/2226308391 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb634103fb0 msgr2=0x7fb6341a2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:22.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.197+0000 7fb63ab8a640 1 --2- 172.21.15.114:0/2226308391 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb634103fb0 0x7fb6341a2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:22.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.197+0000 7fb63ab8a640 1 -- 172.21.15.114:0/2226308391 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb62c009900 con 0x7fb63419c680 2024-09-13T18:25:22.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.197+0000 7fb63a389640 1 --2- 172.21.15.114:0/2226308391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6340fee60 0x7fb6341a25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:25:22.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.197+0000 7fb63ab8a640 1 --2- 172.21.15.114:0/2226308391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb63419c680 0x7fb63419cb00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb62c009fc0 tx=0x7fb62c00c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:22.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.197+0000 7fb62b7fe640 1 -- 172.21.15.114:0/2226308391 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62c003b70 con 0x7fb63419c680 2024-09-13T18:25:22.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.197+0000 7fb62b7fe640 1 -- 172.21.15.114:0/2226308391 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb62c003cd0 con 0x7fb63419c680 2024-09-13T18:25:22.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.197+0000 7fb63c5ed640 1 -- 172.21.15.114:0/2226308391 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb62400b4b0 con 0x7fb63419c680 2024-09-13T18:25:22.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.197+0000 7fb63c5ed640 1 -- 172.21.15.114:0/2226308391 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb63406a260 con 0x7fb63419c680 2024-09-13T18:25:22.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.197+0000 7fb62b7fe640 1 -- 172.21.15.114:0/2226308391 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62c027790 con 0x7fb63419c680 2024-09-13T18:25:22.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.200+0000 7fb62b7fe640 1 -- 172.21.15.114:0/2226308391 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fb62c025070 con 0x7fb63419c680 2024-09-13T18:25:22.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.202+0000 7fb62b7fe640 1 --2- 172.21.15.114:0/2226308391 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fb6200a1240 0x7fb6200a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:22.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.202+0000 7fb63f3ea740 1 Processor -- start 2024-09-13T18:25:22.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.202+0000 7fb62b7fe640 1 -- 172.21.15.114:0/2226308391 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fb62c0de330 con 0x7fb63419c680 2024-09-13T18:25:22.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.202+0000 7fb63a389640 1 --2- 172.21.15.114:0/2226308391 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fb6200a1240 0x7fb6200a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:22.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.203+0000 7fb63f3ea740 1 -- start start 2024-09-13T18:25:22.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.203+0000 7fb63f3ea740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b152b03e10 0x55b152b041f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:22.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.203+0000 7fb63f3ea740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b152b04730 0x55b152b0ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:22.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.203+0000 7fb63a389640 1 --2- 172.21.15.114:0/2226308391 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fb6200a1240 0x7fb6200a3700 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb624000c00 tx=0x7fb62400a4a0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:22.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.203+0000 7fb63f3ea740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b152b0efd0 0x55b152b113c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:22.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.203+0000 7fb63f3ea740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b152a6ef20 con 0x55b152b04730 2024-09-13T18:25:22.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.203+0000 7fb63f3ea740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b152a6c6b0 con 0x55b152b0efd0 2024-09-13T18:25:22.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.203+0000 7fb63f3ea740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b152a67fa0 con 0x55b152b03e10 2024-09-13T18:25:22.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.203+0000 7fb63a389640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b152b03e10 0x55b152b041f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:22.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.203+0000 7fb63a389640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b152b03e10 0x55b152b041f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:38378/0 (socket says 172.21.15.114:38378) 2024-09-13T18:25:22.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.204+0000 7fb63a389640 1 -- 172.21.15.114:0/623443434 learned_addr learned my addr 172.21.15.114:0/623443434 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:22.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.204+0000 7fb63ab8a640 1 --2- 172.21.15.114:0/623443434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b152b0efd0 0x55b152b113c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:22.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.204+0000 7fb639b88640 1 --2- 172.21.15.114:0/623443434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b152b04730 0x55b152b0ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:22.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.204+0000 7fb63ab8a640 1 -- 172.21.15.114:0/623443434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b152b03e10 msgr2=0x55b152b041f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:22.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.204+0000 7fb63ab8a640 1 --2- 172.21.15.114:0/623443434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b152b03e10 0x55b152b041f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:22.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.204+0000 7fb63ab8a640 1 -- 172.21.15.114:0/623443434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b152b04730 msgr2=0x55b152b0ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:22.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.204+0000 7fb63ab8a640 1 --2- 172.21.15.114:0/623443434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b152b04730 0x55b152b0ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:22.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.204+0000 7fb63ab8a640 1 -- 172.21.15.114:0/623443434 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b152a22840 con 0x55b152b0efd0 2024-09-13T18:25:22.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.204+0000 7fb63ab8a640 1 --2- 172.21.15.114:0/623443434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b152b0efd0 0x55b152b113c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb62c0db730 tx=0x7fb62c0aaba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32c7fe4104eb792d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:22.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.206+0000 7fb628ff9640 1 -- 172.21.15.114:0/623443434 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62c0ab180 con 0x55b152b0efd0 2024-09-13T18:25:22.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.206+0000 7fb628ff9640 1 -- 172.21.15.114:0/623443434 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb62c0aefb0 con 0x55b152b0efd0 2024-09-13T18:25:22.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.206+0000 7fb628ff9640 1 -- 172.21.15.114:0/623443434 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62c0b1070 con 0x55b152b0efd0 2024-09-13T18:25:22.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.207+0000 7fb63f3ea740 1 -- 172.21.15.114:0/623443434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b152b0efd0 msgr2=0x55b152b113c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:22.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.207+0000 7fb63f3ea740 1 --2- 172.21.15.114:0/623443434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b152b0efd0 0x55b152b113c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb62c0db730 tx=0x7fb62c0aaba0 comp rx=0 tx=0).stop 2024-09-13T18:25:22.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.207+0000 7fb63f3ea740 1 -- 172.21.15.114:0/623443434 shutdown_connections 2024-09-13T18:25:22.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.207+0000 7fb63f3ea740 1 --2- 172.21.15.114:0/623443434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b152b0efd0 0x55b152b113c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:22.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.207+0000 7fb63f3ea740 1 --2- 172.21.15.114:0/623443434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b152b04730 0x55b152b0ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:22.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.207+0000 7fb63f3ea740 1 --2- 172.21.15.114:0/623443434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b152b03e10 0x55b152b041f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:22.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.207+0000 7fb63f3ea740 1 -- 172.21.15.114:0/623443434 >> 172.21.15.114:0/623443434 conn(0x55b152b00380 msgr2=0x55b152b00760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:22.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.207+0000 7fb63f3ea740 1 -- 172.21.15.114:0/623443434 shutdown_connections 2024-09-13T18:25:22.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.207+0000 7fb63f3ea740 1 -- 172.21.15.114:0/623443434 wait complete. 2024-09-13T18:25:22.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.208+0000 7fb63f3ea740 1 -- start start 2024-09-13T18:25:22.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.208+0000 7fb63f3ea740 1 Processor -- start 2024-09-13T18:25:22.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.208+0000 7fb63f3ea740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b152b03e10 0x55b152b0ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:22.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.208+0000 7fb63f3ea740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b152b04730 0x55b152b0bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:22.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.208+0000 7fb63f3ea740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b152b0efd0 0x55b152b0c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:22.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.208+0000 7fb63f3ea740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b152a092a0 con 0x55b152b0efd0 2024-09-13T18:25:22.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.208+0000 7fb63f3ea740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b152740430 con 0x55b152b03e10 2024-09-13T18:25:22.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.208+0000 7fb63f3ea740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1529eb680 con 0x55b152b04730 2024-09-13T18:25:22.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.208+0000 7fb63f3ea740 10 client.0 _ll_register_callbacks cb 0x55b152a61610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:25:22.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.209+0000 7fb63a389640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b152b03e10 0x55b152b0ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:22.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.209+0000 7fb639b88640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b152b04730 0x55b152b0bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:22.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.209+0000 7fb63a389640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b152b03e10 0x55b152b0ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:44030/0 (socket says 172.21.15.114:44030) 2024-09-13T18:25:22.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.209+0000 7fb63a389640 1 -- 172.21.15.114:0/610563970 learned_addr learned my addr 172.21.15.114:0/610563970 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:22.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.209+0000 7fb63ab8a640 1 --2- 172.21.15.114:0/610563970 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b152b0efd0 0x55b152b0c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:22.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.209+0000 7fb63a389640 1 -- 172.21.15.114:0/610563970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b152b04730 msgr2=0x55b152b0bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:22.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.209+0000 7fb63a389640 1 --2- 172.21.15.114:0/610563970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b152b04730 0x55b152b0bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:22.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.209+0000 7fb63a389640 1 -- 172.21.15.114:0/610563970 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b152b0efd0 msgr2=0x55b152b0c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:22.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.209+0000 7fb63a389640 1 --2- 172.21.15.114:0/610563970 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b152b0efd0 0x55b152b0c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:22.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.209+0000 7fb63a389640 1 -- 172.21.15.114:0/610563970 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb63406a260 con 0x55b152b03e10 2024-09-13T18:25:22.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.210+0000 7fb63a389640 1 --2- 172.21.15.114:0/610563970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b152b03e10 0x55b152b0ea80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb624005470 tx=0x7fb62400d040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=513cd9301ebbb1cd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:22.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.210+0000 7fb61dffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:25:22.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.210+0000 7fb61dffb640 1 -- 172.21.15.114:0/610563970 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6240326c0 con 0x55b152b03e10 2024-09-13T18:25:22.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.210+0000 7fb63f3ea740 1 -- 172.21.15.114:0/610563970 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b15275b4e0 con 0x55b152b03e10 2024-09-13T18:25:22.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.210+0000 7fb61dffb640 1 -- 172.21.15.114:0/610563970 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fb624044040 con 0x55b152b03e10 2024-09-13T18:25:22.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.210+0000 7fb61dffb640 1 -- 172.21.15.114:0/610563970 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb624032820 con 0x55b152b03e10 2024-09-13T18:25:22.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.210+0000 7fb63f3ea740 10 client.26054 fetch_fsmap learned FSMap version 32 2024-09-13T18:25:22.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.210+0000 7fb63f3ea740 1 -- 172.21.15.114:0/610563970 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55b152a66620 con 0x55b152b03e10 2024-09-13T18:25:22.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.211+0000 7fb61dffb640 1 -- 172.21.15.114:0/610563970 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b15275b4e0 con 0x55b152b03e10 2024-09-13T18:25:22.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.211+0000 7fb61dffb640 1 -- 172.21.15.114:0/610563970 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fb624004a70 con 0x55b152b03e10 2024-09-13T18:25:22.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.211+0000 7fb63f3ea740 10 client.26054 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:25:22.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.211+0000 7fb63f3ea740 10 client.26054 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:25:22.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.211+0000 7fb63f3ea740 20 client.26054 populate_metadata read hostname 'smithi114' 2024-09-13T18:25:22.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.211+0000 7fb63f3ea740 1 --2- 172.21.15.114:0/610563970 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55b152b1ee60 0x55b152b21280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:22.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.211+0000 7fb63f3ea740 4 client.26054 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:25:22.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.211+0000 7fb63f3ea740 1 -- 172.21.15.114:0/610563970 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b15276c7e0 con 0x55b152b1ee60 2024-09-13T18:25:22.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.211+0000 7fb639b88640 1 --2- 172.21.15.114:0/610563970 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55b152b1ee60 0x55b152b21280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:22.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.212+0000 7fb639b88640 1 --2- 172.21.15.114:0/610563970 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55b152b1ee60 0x55b152b21280 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6538a0aebc996e03 server_cookie=10c19471e6eb91cb in_seq=0 out_seq=0 2024-09-13T18:25:22.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.212+0000 7fb61dffb640 10 client.26054 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:25:22.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.213+0000 7fb61dffb640 1 -- 172.21.15.114:0/610563970 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55b15276c7e0 con 0x55b152b1ee60 2024-09-13T18:25:22.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.213+0000 7fb61dffb640 10 client.26054 handle_command_reply: tid=0 2024-09-13T18:25:22.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.213+0000 7fb63f3ea740 1 client.26054 shutdown 2024-09-13T18:25:22.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.213+0000 7fb63f3ea740 1 -- 172.21.15.114:0/610563970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b152b03e10 msgr2=0x55b152b0ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:22.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.213+0000 7fb63f3ea740 1 --2- 172.21.15.114:0/610563970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b152b03e10 0x55b152b0ea80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb624005470 tx=0x7fb62400d040 comp rx=0 tx=0).stop 2024-09-13T18:25:22.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.213+0000 7fb63f3ea740 1 -- 172.21.15.114:0/610563970 shutdown_connections 2024-09-13T18:25:22.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.214+0000 7fb63f3ea740 1 --2- 172.21.15.114:0/610563970 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55b152b1ee60 0x55b152b21280 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:22.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.214+0000 7fb63f3ea740 1 --2- 172.21.15.114:0/610563970 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b152b0efd0 0x55b152b0c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:22.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.214+0000 7fb63f3ea740 1 --2- 172.21.15.114:0/610563970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b152b04730 0x55b152b0bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:22.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.214+0000 7fb63f3ea740 1 --2- 172.21.15.114:0/610563970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b152b03e10 0x55b152b0ea80 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:22.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.214+0000 7fb63f3ea740 1 -- 172.21.15.114:0/610563970 >> 172.21.15.114:0/610563970 conn(0x55b152b00380 msgr2=0x55b152b060e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:22.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.214+0000 7fb61dffb640 0 client.26054 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:25:22.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.214+0000 7fb63f3ea740 1 -- 172.21.15.114:0/610563970 shutdown_connections 2024-09-13T18:25:22.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.214+0000 7fb63f3ea740 1 -- 172.21.15.114:0/610563970 wait complete. 2024-09-13T18:25:22.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.214+0000 7fb63f3ea740 20 client.26054 trim_cache size 0 max 16384 2024-09-13T18:25:22.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.250+0000 7fb63f3ea740 1 Processor -- start 2024-09-13T18:25:22.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.250+0000 7fb63f3ea740 1 -- start start 2024-09-13T18:25:22.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.251+0000 7fb63f3ea740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b152b03e10 0x55b152b0e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:22.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.251+0000 7fb63a389640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b152b03e10 0x55b152b0e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:22.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.251+0000 7fb63a389640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b152b03e10 0x55b152b0e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44182/0 (socket says 172.21.15.114:44182) 2024-09-13T18:25:22.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.251+0000 7fb63f3ea740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b152b04730 0x55b152b0a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:22.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.251+0000 7fb63f3ea740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b152b0efd0 0x55b152b0ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:22.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.251+0000 7fb63f3ea740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b15275b4e0 con 0x55b152b03e10 2024-09-13T18:25:22.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.251+0000 7fb63f3ea740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b152b03b90 con 0x55b152b0efd0 2024-09-13T18:25:22.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.251+0000 7fb63f3ea740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b152b0cab0 con 0x55b152b04730 2024-09-13T18:25:22.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.251+0000 7fb63a389640 1 -- 172.21.15.114:0/3686102811 learned_addr learned my addr 172.21.15.114:0/3686102811 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:22.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.251+0000 7fb63ab8a640 1 --2- 172.21.15.114:0/3686102811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b152b0efd0 0x55b152b0ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:22.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.251+0000 7fb639b88640 1 --2- 172.21.15.114:0/3686102811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b152b04730 0x55b152b0a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:22.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.252+0000 7fb63ab8a640 1 -- 172.21.15.114:0/3686102811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b152b04730 msgr2=0x55b152b0a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:22.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.252+0000 7fb63ab8a640 1 --2- 172.21.15.114:0/3686102811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b152b04730 0x55b152b0a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:22.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.252+0000 7fb63ab8a640 1 -- 172.21.15.114:0/3686102811 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b152b03e10 msgr2=0x55b152b0e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:22.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.252+0000 7fb63ab8a640 1 --2- 172.21.15.114:0/3686102811 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b152b03e10 0x55b152b0e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:22.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.252+0000 7fb63ab8a640 1 -- 172.21.15.114:0/3686102811 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b152a08790 con 0x55b152b0efd0 2024-09-13T18:25:22.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.252+0000 7fb63ab8a640 1 --2- 172.21.15.114:0/3686102811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b152b0efd0 0x55b152b0ab10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb62c0a7bf0 tx=0x7fb62c0aac10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e92a82802714fa4e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:22.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.252+0000 7fb6297fa640 1 -- 172.21.15.114:0/3686102811 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62c0dbb70 con 0x55b152b0efd0 2024-09-13T18:25:22.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.252+0000 7fb6297fa640 1 -- 172.21.15.114:0/3686102811 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb62c0ab180 con 0x55b152b0efd0 2024-09-13T18:25:22.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.252+0000 7fb6297fa640 1 -- 172.21.15.114:0/3686102811 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62c0a7a60 con 0x55b152b0efd0 2024-09-13T18:25:22.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.254+0000 7fb63f3ea740 1 -- 172.21.15.114:0/3686102811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b152b0efd0 msgr2=0x55b152b0ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:22.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.254+0000 7fb63f3ea740 1 --2- 172.21.15.114:0/3686102811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b152b0efd0 0x55b152b0ab10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb62c0a7bf0 tx=0x7fb62c0aac10 comp rx=0 tx=0).stop 2024-09-13T18:25:22.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.254+0000 7fb63f3ea740 1 -- 172.21.15.114:0/3686102811 shutdown_connections 2024-09-13T18:25:22.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.254+0000 7fb63f3ea740 1 --2- 172.21.15.114:0/3686102811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b152b0efd0 0x55b152b0ab10 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:22.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.254+0000 7fb63f3ea740 1 --2- 172.21.15.114:0/3686102811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b152b04730 0x55b152b0a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:22.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.254+0000 7fb63f3ea740 1 --2- 172.21.15.114:0/3686102811 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b152b03e10 0x55b152b0e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:22.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.254+0000 7fb63f3ea740 1 -- 172.21.15.114:0/3686102811 >> 172.21.15.114:0/3686102811 conn(0x55b152b00380 msgr2=0x55b152b07030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:22.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.254+0000 7fb63f3ea740 1 -- 172.21.15.114:0/3686102811 shutdown_connections 2024-09-13T18:25:22.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.254+0000 7fb63f3ea740 1 -- 172.21.15.114:0/3686102811 wait complete. 2024-09-13T18:25:22.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.255+0000 7fb63f3ea740 1 -- start start 2024-09-13T18:25:22.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.255+0000 7fb63f3ea740 1 Processor -- start 2024-09-13T18:25:22.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.255+0000 7fb63f3ea740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b152b03e10 0x55b152b11f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:22.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.256+0000 7fb63f3ea740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b152b04730 0x55b152b12490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:22.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.256+0000 7fb63f3ea740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b152b0efd0 0x55b152b74310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:22.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.256+0000 7fb63a389640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b152b03e10 0x55b152b11f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:22.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.256+0000 7fb63a389640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b152b03e10 0x55b152b11f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44194/0 (socket says 172.21.15.114:44194) 2024-09-13T18:25:22.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.256+0000 7fb639b88640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b152b04730 0x55b152b12490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:22.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.256+0000 7fb639b88640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b152b04730 0x55b152b12490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:38400/0 (socket says 172.21.15.114:38400) 2024-09-13T18:25:22.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.256+0000 7fb63a389640 1 -- 172.21.15.114:0/1895920333 learned_addr learned my addr 172.21.15.114:0/1895920333 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:22.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.256+0000 7fb63f3ea740 1 -- 172.21.15.114:0/1895920333 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b152b12bd0 con 0x55b152b03e10 2024-09-13T18:25:22.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.256+0000 7fb63f3ea740 1 -- 172.21.15.114:0/1895920333 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b152b74850 con 0x55b152b0efd0 2024-09-13T18:25:22.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.256+0000 7fb63f3ea740 1 -- 172.21.15.114:0/1895920333 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b152b749e0 con 0x55b152b04730 2024-09-13T18:25:22.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.256+0000 7fb63f3ea740 10 client.0 _ll_register_callbacks cb 0x7fb62400b6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:25:22.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.256+0000 7fb63ab8a640 1 --2- 172.21.15.114:0/1895920333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b152b0efd0 0x55b152b74310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:22.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.257+0000 7fb63a389640 1 -- 172.21.15.114:0/1895920333 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b152b04730 msgr2=0x55b152b12490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:22.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.257+0000 7fb63a389640 1 --2- 172.21.15.114:0/1895920333 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b152b04730 0x55b152b12490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:22.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.257+0000 7fb63a389640 1 -- 172.21.15.114:0/1895920333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b152b0efd0 msgr2=0x55b152b74310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:22.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.257+0000 7fb63a389640 1 --2- 172.21.15.114:0/1895920333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b152b0efd0 0x55b152b74310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:22.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.257+0000 7fb63a389640 1 -- 172.21.15.114:0/1895920333 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b152a22840 con 0x55b152b03e10 2024-09-13T18:25:22.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.257+0000 7fb63a389640 1 --2- 172.21.15.114:0/1895920333 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b152b03e10 0x55b152b11f50 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fb624005fd0 tx=0x7fb624005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cabf1b50e63bb8df server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:22.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.257+0000 7fb61dffb640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:25:22.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.258+0000 7fb61dffb640 1 -- 172.21.15.114:0/1895920333 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6240326c0 con 0x55b152b03e10 2024-09-13T18:25:22.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.258+0000 7fb61dffb640 1 -- 172.21.15.114:0/1895920333 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fb624032820 con 0x55b152b03e10 2024-09-13T18:25:22.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.258+0000 7fb63f3ea740 1 -- 172.21.15.114:0/1895920333 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb62c0aefb0 con 0x55b152b03e10 2024-09-13T18:25:22.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.258+0000 7fb61dffb640 1 -- 172.21.15.114:0/1895920333 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6240329e0 con 0x55b152b03e10 2024-09-13T18:25:22.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.258+0000 7fb63f3ea740 10 client.16725 fetch_fsmap learned FSMap version 32 2024-09-13T18:25:22.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.258+0000 7fb63f3ea740 1 -- 172.21.15.114:0/1895920333 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55b152a2dea0 con 0x55b152b03e10 2024-09-13T18:25:22.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.258+0000 7fb61dffb640 1 -- 172.21.15.114:0/1895920333 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb62406e020 con 0x55b152b03e10 2024-09-13T18:25:22.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.259+0000 7fb61dffb640 1 -- 172.21.15.114:0/1895920333 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fb62406b070 con 0x55b152b03e10 2024-09-13T18:25:22.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.259+0000 7fb63f3ea740 10 client.16725 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:25:22.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.259+0000 7fb63f3ea740 10 client.16725 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:25:22.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.259+0000 7fb63f3ea740 20 client.16725 populate_metadata read hostname 'smithi114' 2024-09-13T18:25:22.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.259+0000 7fb63f3ea740 1 --2- 172.21.15.114:0/1895920333 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55b152b75eb0 0x55b152b782d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:22.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.259+0000 7fb63f3ea740 4 client.16725 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T18:25:22.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.259+0000 7fb63f3ea740 1 -- 172.21.15.114:0/1895920333 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7fb62c0aefb0 con 0x55b152b75eb0 2024-09-13T18:25:22.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.259+0000 7fb639b88640 1 --2- 172.21.15.114:0/1895920333 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55b152b75eb0 0x55b152b782d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:22.264 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.260+0000 7fb61dffb640 10 client.16725 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:25:22.264 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.260+0000 7fb639b88640 1 --2- 172.21.15.114:0/1895920333 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55b152b75eb0 0x55b152b782d0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a617e8263eb7448b server_cookie=9e567dc2075d4faf in_seq=0 out_seq=0 2024-09-13T18:25:22.264 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.260+0000 7fb61dffb640 1 -- 172.21.15.114:0/1895920333 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x7fb62c0aefb0 con 0x55b152b75eb0 2024-09-13T18:25:22.264 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.260+0000 7fb61dffb640 10 client.16725 handle_command_reply: tid=0 2024-09-13T18:25:22.264 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.260+0000 7fb63f3ea740 1 client.16725 shutdown 2024-09-13T18:25:22.264 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.261+0000 7fb63f3ea740 1 -- 172.21.15.114:0/1895920333 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b152b03e10 msgr2=0x55b152b11f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:22.264 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.261+0000 7fb63f3ea740 1 --2- 172.21.15.114:0/1895920333 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b152b03e10 0x55b152b11f50 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fb624005fd0 tx=0x7fb624005b10 comp rx=0 tx=0).stop 2024-09-13T18:25:22.264 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.261+0000 7fb63f3ea740 1 -- 172.21.15.114:0/1895920333 shutdown_connections 2024-09-13T18:25:22.264 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.261+0000 7fb63f3ea740 1 --2- 172.21.15.114:0/1895920333 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55b152b75eb0 0x55b152b782d0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:22.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.261+0000 7fb63f3ea740 1 --2- 172.21.15.114:0/1895920333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b152b0efd0 0x55b152b74310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:22.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.261+0000 7fb63f3ea740 1 --2- 172.21.15.114:0/1895920333 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b152b04730 0x55b152b12490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:22.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.261+0000 7fb63f3ea740 1 --2- 172.21.15.114:0/1895920333 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b152b03e10 0x55b152b11f50 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:22.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.261+0000 7fb63f3ea740 1 -- 172.21.15.114:0/1895920333 >> 172.21.15.114:0/1895920333 conn(0x55b152b00380 msgr2=0x55b152b07030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:22.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.261+0000 7fb61dffb640 0 client.16725 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:25:22.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.261+0000 7fb63f3ea740 1 -- 172.21.15.114:0/1895920333 shutdown_connections 2024-09-13T18:25:22.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.261+0000 7fb63f3ea740 1 -- 172.21.15.114:0/1895920333 wait complete. 2024-09-13T18:25:22.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.262+0000 7fb63f3ea740 20 client.16725 trim_cache size 0 max 16384 2024-09-13T18:25:22.266 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:25:22.266 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-13T18:25:22.266 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": { 2024-09-13T18:25:22.266 INFO:teuthology.orchestra.run.smithi114.stdout: "8c9c32f6-25f3-45a8-b3d0-0d974760d7bd": { 2024-09-13T18:25:22.266 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-13T18:25:22.266 INFO:teuthology.orchestra.run.smithi114.stdout: "tag": "8c9c32f6-25f3-45a8-b3d0-0d974760d7bd", 2024-09-13T18:25:22.266 INFO:teuthology.orchestra.run.smithi114.stdout: "options": "recursive,force" 2024-09-13T18:25:22.266 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:25:22.266 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:25:22.266 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:25:22.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.263+0000 7fb63c5ed640 1 -- 172.21.15.114:0/2226308391 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fb6200a1240 msgr2=0x7fb6200a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:22.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.263+0000 7fb63c5ed640 1 --2- 172.21.15.114:0/2226308391 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fb6200a1240 0x7fb6200a3700 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb624000c00 tx=0x7fb62400a4a0 comp rx=0 tx=0).stop 2024-09-13T18:25:22.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.263+0000 7fb63c5ed640 1 -- 172.21.15.114:0/2226308391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb63419c680 msgr2=0x7fb63419cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:22.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.263+0000 7fb63c5ed640 1 --2- 172.21.15.114:0/2226308391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb63419c680 0x7fb63419cb00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb62c009fc0 tx=0x7fb62c00c740 comp rx=0 tx=0).stop 2024-09-13T18:25:22.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.263+0000 7fb63c5ed640 1 -- 172.21.15.114:0/2226308391 shutdown_connections 2024-09-13T18:25:22.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.263+0000 7fb63c5ed640 1 --2- 172.21.15.114:0/2226308391 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fb6200a1240 0x7fb6200a3700 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:22.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.263+0000 7fb63c5ed640 1 --2- 172.21.15.114:0/2226308391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb63419c680 0x7fb63419cb00 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:22.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.263+0000 7fb63c5ed640 1 --2- 172.21.15.114:0/2226308391 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb634103fb0 0x7fb6341a2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:22.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.263+0000 7fb63c5ed640 1 --2- 172.21.15.114:0/2226308391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6340fee60 0x7fb6341a25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:22.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.263+0000 7fb63c5ed640 1 -- 172.21.15.114:0/2226308391 >> 172.21.15.114:0/2226308391 conn(0x7fb634075c60 msgr2=0x7fb634102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:22.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.263+0000 7fb63c5ed640 1 -- 172.21.15.114:0/2226308391 shutdown_connections 2024-09-13T18:25:22.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:22.264+0000 7fb63c5ed640 1 -- 172.21.15.114:0/2226308391 wait complete. 2024-09-13T18:25:22.286 INFO:tasks.cephfs.filesystem:scrub status for tag:8c9c32f6-25f3-45a8-b3d0-0d974760d7bd - {'path': '/', 'tag': '8c9c32f6-25f3-45a8-b3d0-0d974760d7bd', 'options': 'recursive,force'} 2024-09-13T18:25:22.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:22 smithi204 ceph-mon[34777]: pgmap v90: 129 pgs: 129 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 52 MiB/s wr, 834 op/s 2024-09-13T18:25:22.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:22 smithi114 ceph-mon[30897]: pgmap v90: 129 pgs: 129 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 52 MiB/s wr, 834 op/s 2024-09-13T18:25:22.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:22 smithi134 ceph-mon[34283]: pgmap v90: 129 pgs: 129 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 52 MiB/s wr, 834 op/s 2024-09-13T18:25:23.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:23 smithi204 ceph-mon[34777]: scrub queued for path: / 2024-09-13T18:25:23.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:23 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:25:23.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:23 smithi204 ceph-mon[34777]: scrub summary: active paths [/] 2024-09-13T18:25:23.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:23 smithi204 ceph-mon[34777]: scrub summary: active paths [/] 2024-09-13T18:25:23.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:23 smithi204 ceph-mon[34777]: scrub summary: active paths [/] 2024-09-13T18:25:23.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:23 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:25:23.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:23 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:25:23.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:23 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:25:23.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:23 smithi114 ceph-mon[30897]: scrub queued for path: / 2024-09-13T18:25:23.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:23 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:25:23.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:23 smithi114 ceph-mon[30897]: scrub summary: active paths [/] 2024-09-13T18:25:23.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:23 smithi114 ceph-mon[30897]: scrub summary: active paths [/] 2024-09-13T18:25:23.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:23 smithi114 ceph-mon[30897]: scrub summary: active paths [/] 2024-09-13T18:25:23.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:23 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:25:23.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:23 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:25:23.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:23 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:25:23.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:23 smithi134 ceph-mon[34283]: scrub queued for path: / 2024-09-13T18:25:23.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:23 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:25:23.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:23 smithi134 ceph-mon[34283]: scrub summary: active paths [/] 2024-09-13T18:25:23.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:23 smithi134 ceph-mon[34283]: scrub summary: active paths [/] 2024-09-13T18:25:23.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:23 smithi134 ceph-mon[34283]: scrub summary: active paths [/] 2024-09-13T18:25:23.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:23 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:25:23.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:23 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:25:23.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:23 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:25:24.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:24 smithi114 ceph-mon[30897]: pgmap v91: 129 pgs: 129 active+clean; 11 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 76 MiB/s wr, 1.23k op/s 2024-09-13T18:25:24.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:24 smithi134 ceph-mon[34283]: pgmap v91: 129 pgs: 129 active+clean; 11 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 76 MiB/s wr, 1.23k op/s 2024-09-13T18:25:24.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:24 smithi204 ceph-mon[34777]: pgmap v91: 129 pgs: 129 active+clean; 11 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 76 MiB/s wr, 1.23k op/s 2024-09-13T18:25:26.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:26 smithi204 ceph-mon[34777]: pgmap v92: 129 pgs: 129 active+clean; 11 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 50 MiB/s wr, 817 op/s 2024-09-13T18:25:26.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:26 smithi114 ceph-mon[30897]: pgmap v92: 129 pgs: 129 active+clean; 11 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 50 MiB/s wr, 817 op/s 2024-09-13T18:25:26.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:26 smithi134 ceph-mon[34283]: pgmap v92: 129 pgs: 129 active+clean; 11 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 50 MiB/s wr, 817 op/s 2024-09-13T18:25:28.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:28 smithi204 ceph-mon[34777]: pgmap v93: 129 pgs: 129 active+clean; 11 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 50 MiB/s wr, 817 op/s 2024-09-13T18:25:28.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:28 smithi114 ceph-mon[30897]: pgmap v93: 129 pgs: 129 active+clean; 11 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 50 MiB/s wr, 817 op/s 2024-09-13T18:25:28.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:28 smithi134 ceph-mon[34283]: pgmap v93: 129 pgs: 129 active+clean; 11 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 50 MiB/s wr, 817 op/s 2024-09-13T18:25:29.281 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.277+0000 7f9dbb1be640 1 -- 172.21.15.114:0/153272225 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9dac0a0f60 msgr2=0x7f9dac0a8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:29.281 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.277+0000 7f9dbb1be640 1 --2- 172.21.15.114:0/153272225 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9dac0a0f60 0x7f9dac0a8ab0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f9db0009890 tx=0x7f9db002a380 comp rx=0 tx=0).stop 2024-09-13T18:25:29.281 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.278+0000 7f9dbb1be640 1 -- 172.21.15.114:0/153272225 shutdown_connections 2024-09-13T18:25:29.281 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.278+0000 7f9dbb1be640 1 --2- 172.21.15.114:0/153272225 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9dac0a0f60 0x7f9dac0a8ab0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:29.281 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.278+0000 7f9dbb1be640 1 --2- 172.21.15.114:0/153272225 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9dac09c7c0 0x7f9dac0a0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:29.282 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.278+0000 7f9dbb1be640 1 --2- 172.21.15.114:0/153272225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dac09be10 0x7f9dac09c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:29.282 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.278+0000 7f9dbb1be640 1 -- 172.21.15.114:0/153272225 >> 172.21.15.114:0/153272225 conn(0x7f9dac0093b0 msgr2=0x7f9dac099d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:29.282 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.278+0000 7f9dbb1be640 1 -- 172.21.15.114:0/153272225 shutdown_connections 2024-09-13T18:25:29.282 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.278+0000 7f9dbb1be640 1 -- 172.21.15.114:0/153272225 wait complete. 2024-09-13T18:25:29.283 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.279+0000 7f9dbb1be640 1 Processor -- start 2024-09-13T18:25:29.283 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.279+0000 7f9dbb1be640 1 -- start start 2024-09-13T18:25:29.283 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.279+0000 7f9dbb1be640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dac09be10 0x7f9dac132140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:29.283 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.279+0000 7f9dbb1be640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9dac0a0f60 0x7f9dac132680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:29.283 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.279+0000 7f9dbb1be640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9dac136a70 0x7f9dac136f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:29.283 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.279+0000 7f9dbb1be640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dac137460 con 0x7f9dac0a0f60 2024-09-13T18:25:29.283 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.279+0000 7f9dbb1be640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dac1375d0 con 0x7f9dac09be10 2024-09-13T18:25:29.283 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.279+0000 7f9dbb1be640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dac137740 con 0x7f9dac136a70 2024-09-13T18:25:29.283 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.279+0000 7f9db99bb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9dac0a0f60 0x7f9dac132680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:29.283 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.279+0000 7f9dba1bc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dac09be10 0x7f9dac132140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:29.283 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.280+0000 7f9dba1bc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dac09be10 0x7f9dac132140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:50380/0 (socket says 172.21.15.114:50380) 2024-09-13T18:25:29.283 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.280+0000 7f9dba1bc640 1 -- 172.21.15.114:0/3051665952 learned_addr learned my addr 172.21.15.114:0/3051665952 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:29.285 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.280+0000 7f9dba9bd640 1 --2- 172.21.15.114:0/3051665952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9dac136a70 0x7f9dac136f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:29.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.280+0000 7f9db99bb640 1 -- 172.21.15.114:0/3051665952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9dac136a70 msgr2=0x7f9dac136f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:29.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.280+0000 7f9db99bb640 1 --2- 172.21.15.114:0/3051665952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9dac136a70 0x7f9dac136f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:29.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.280+0000 7f9db99bb640 1 -- 172.21.15.114:0/3051665952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dac09be10 msgr2=0x7f9dac132140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:29.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.280+0000 7f9db99bb640 1 --2- 172.21.15.114:0/3051665952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dac09be10 0x7f9dac132140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:29.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.281+0000 7f9db99bb640 1 -- 172.21.15.114:0/3051665952 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db404f100 con 0x7f9dac0a0f60 2024-09-13T18:25:29.288 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.281+0000 7f9db99bb640 1 --2- 172.21.15.114:0/3051665952 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9dac0a0f60 0x7f9dac132680 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f9da4005fd0 tx=0x7f9da400ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:29.288 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.282+0000 7f9dab7fe640 1 -- 172.21.15.114:0/3051665952 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da400ccc0 con 0x7f9dac0a0f60 2024-09-13T18:25:29.288 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.282+0000 7f9dab7fe640 1 -- 172.21.15.114:0/3051665952 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9da400ee70 con 0x7f9dac0a0f60 2024-09-13T18:25:29.288 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.282+0000 7f9dab7fe640 1 -- 172.21.15.114:0/3051665952 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da4018680 con 0x7f9dac0a0f60 2024-09-13T18:25:29.288 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.282+0000 7f9dbb1be640 1 -- 172.21.15.114:0/3051665952 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9db0009490 con 0x7f9dac0a0f60 2024-09-13T18:25:29.288 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.282+0000 7f9dbb1be640 1 -- 172.21.15.114:0/3051665952 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dac13eff0 con 0x7f9dac0a0f60 2024-09-13T18:25:29.289 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.286+0000 7f9dbf21d740 1 Processor -- start 2024-09-13T18:25:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.287+0000 7f9dbf21d740 1 -- start start 2024-09-13T18:25:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.287+0000 7f9dbf21d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626a71c3500 0x5626a71c38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.287+0000 7f9dbf21d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626a71c3e20 0x5626a71ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.287+0000 7f9dbf21d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626a71ce6c0 0x5626a71d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.287+0000 7f9dbf21d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626a712ec30 con 0x5626a71c3e20 2024-09-13T18:25:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.287+0000 7f9dbf21d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626a712c3c0 con 0x5626a71c3500 2024-09-13T18:25:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.287+0000 7f9dbf21d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626a7127cb0 con 0x5626a71ce6c0 2024-09-13T18:25:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.287+0000 7f9dab7fe640 1 -- 172.21.15.114:0/3051665952 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f9da4010430 con 0x7f9dac0a0f60 2024-09-13T18:25:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.287+0000 7f9dba1bc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626a71c3e20 0x5626a71ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.287+0000 7f9dba1bc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626a71c3e20 0x5626a71ce180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42136/0 (socket says 172.21.15.114:42136) 2024-09-13T18:25:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.287+0000 7f9dba1bc640 1 -- 172.21.15.114:0/1770358005 learned_addr learned my addr 172.21.15.114:0/1770358005 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.288+0000 7f9dba9bd640 1 --2- 172.21.15.114:0/1770358005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626a71c3500 0x5626a71c38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.288+0000 7f9db99bb640 1 --2- 172.21.15.114:0/1770358005 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626a71ce6c0 0x5626a71d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.288+0000 7f9db99bb640 1 -- 172.21.15.114:0/1770358005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626a71c3500 msgr2=0x5626a71c38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.288+0000 7f9db99bb640 1 --2- 172.21.15.114:0/1770358005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626a71c3500 0x5626a71c38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.288+0000 7f9db99bb640 1 -- 172.21.15.114:0/1770358005 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626a71c3e20 msgr2=0x5626a71ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.288+0000 7f9db99bb640 1 --2- 172.21.15.114:0/1770358005 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626a71c3e20 0x5626a71ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.288+0000 7f9db99bb640 1 -- 172.21.15.114:0/1770358005 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626a70ce910 con 0x5626a71ce6c0 2024-09-13T18:25:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.289+0000 7f9db99bb640 1 --2- 172.21.15.114:0/1770358005 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626a71ce6c0 0x5626a71d0ab0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9da40ca490 tx=0x7f9da40d54b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40ce92f570eb22e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.289+0000 7f9dba1bc640 1 --2- 172.21.15.114:0/1770358005 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626a71c3e20 0x5626a71ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:25:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.289+0000 7f9dab7fe640 1 --2- 172.21.15.114:0/3051665952 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f9da00a1350 0x7f9da00a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.289+0000 7f9dab7fe640 1 -- 172.21.15.114:0/3051665952 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f9da4014070 con 0x7f9dac0a0f60 2024-09-13T18:25:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.289+0000 7f9da8e05640 1 -- 172.21.15.114:0/1770358005 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da4096240 con 0x5626a71ce6c0 2024-09-13T18:25:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.289+0000 7f9dba9bd640 1 --2- 172.21.15.114:0/3051665952 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f9da00a1350 0x7f9da00a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.290+0000 7f9da8e05640 1 -- 172.21.15.114:0/1770358005 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9da4099cd0 con 0x5626a71ce6c0 2024-09-13T18:25:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.290+0000 7f9da8e05640 1 -- 172.21.15.114:0/1770358005 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da4099860 con 0x5626a71ce6c0 2024-09-13T18:25:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.290+0000 7f9dbf21d740 1 -- 172.21.15.114:0/1770358005 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626a71ce6c0 msgr2=0x5626a71d0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.290+0000 7f9dbf21d740 1 --2- 172.21.15.114:0/1770358005 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626a71ce6c0 0x5626a71d0ab0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9da40ca490 tx=0x7f9da40d54b0 comp rx=0 tx=0).stop 2024-09-13T18:25:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.293+0000 7f9dbf21d740 1 -- 172.21.15.114:0/1770358005 shutdown_connections 2024-09-13T18:25:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.293+0000 7f9dbf21d740 1 --2- 172.21.15.114:0/1770358005 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626a71ce6c0 0x5626a71d0ab0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.293+0000 7f9dbf21d740 1 --2- 172.21.15.114:0/1770358005 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626a71c3e20 0x5626a71ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.293+0000 7f9dbf21d740 1 --2- 172.21.15.114:0/1770358005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626a71c3500 0x5626a71c38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.293+0000 7f9dbf21d740 1 -- 172.21.15.114:0/1770358005 >> 172.21.15.114:0/1770358005 conn(0x5626a71bfa90 msgr2=0x5626a71bfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:29.297 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.294+0000 7f9dba9bd640 1 --2- 172.21.15.114:0/3051665952 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f9da00a1350 0x7f9da00a3810 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9db00057c0 tx=0x7f9db00056d0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:29.300 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.295+0000 7f9dbf21d740 1 -- 172.21.15.114:0/1770358005 shutdown_connections 2024-09-13T18:25:29.301 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.297+0000 7f9dbf21d740 1 -- 172.21.15.114:0/1770358005 wait complete. 2024-09-13T18:25:29.301 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.297+0000 7f9dbf21d740 1 -- start start 2024-09-13T18:25:29.301 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.297+0000 7f9dbf21d740 1 Processor -- start 2024-09-13T18:25:29.303 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.298+0000 7f9dbf21d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626a71c3500 0x5626a71ce060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:29.303 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.298+0000 7f9dbf21d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626a71c3e20 0x5626a71cb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:29.303 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.298+0000 7f9dbf21d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626a71ce6c0 0x5626a71cb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:29.304 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.298+0000 7f9dbf21d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626a70e5e80 con 0x5626a71ce6c0 2024-09-13T18:25:29.304 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.298+0000 7f9dbf21d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626a70e5b20 con 0x5626a71c3500 2024-09-13T18:25:29.304 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.298+0000 7f9dbf21d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626a70ab260 con 0x5626a71c3e20 2024-09-13T18:25:29.304 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.299+0000 7f9dba1bc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626a71c3500 0x5626a71ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:29.304 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.299+0000 7f9dbf21d740 10 client.0 _ll_register_callbacks cb 0x5626a70e0df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:25:29.304 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.299+0000 7f9db99bb640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626a71c3e20 0x5626a71cb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:29.304 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.299+0000 7f9db99bb640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626a71c3e20 0x5626a71cb290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:45678/0 (socket says 172.21.15.114:45678) 2024-09-13T18:25:29.304 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.299+0000 7f9db99bb640 1 -- 172.21.15.114:0/486827877 learned_addr learned my addr 172.21.15.114:0/486827877 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:29.304 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.299+0000 7f9db99bb640 1 -- 172.21.15.114:0/486827877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626a71c3500 msgr2=0x5626a71ce060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:29.304 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.299+0000 7f9db99bb640 1 --2- 172.21.15.114:0/486827877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626a71c3500 0x5626a71ce060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:29.304 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.299+0000 7f9db99bb640 1 -- 172.21.15.114:0/486827877 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626a71ce6c0 msgr2=0x5626a71cb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:29.304 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.299+0000 7f9db99bb640 1 --2- 172.21.15.114:0/486827877 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626a71ce6c0 0x5626a71cb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.299+0000 7f9db99bb640 1 -- 172.21.15.114:0/486827877 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dac13eff0 con 0x5626a71c3e20 2024-09-13T18:25:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.300+0000 7f9db99bb640 1 --2- 172.21.15.114:0/486827877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626a71c3e20 0x5626a71cb290 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9da40caa90 tx=0x7f9da40cd200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed12416adec46f3e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.300+0000 7f9d9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:25:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.300+0000 7f9d9dffb640 1 -- 172.21.15.114:0/486827877 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da40d04f0 con 0x5626a71c3e20 2024-09-13T18:25:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.300+0000 7f9d9dffb640 1 -- 172.21.15.114:0/486827877 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f9da400ce20 con 0x5626a71c3e20 2024-09-13T18:25:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.300+0000 7f9dbf21d740 1 -- 172.21.15.114:0/486827877 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5626a7150680 con 0x5626a71c3e20 2024-09-13T18:25:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.300+0000 7f9d9dffb640 1 -- 172.21.15.114:0/486827877 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9da40108f0 con 0x5626a71c3e20 2024-09-13T18:25:29.306 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.301+0000 7f9dbf21d740 10 client.26107 fetch_fsmap learned FSMap version 32 2024-09-13T18:25:29.306 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.301+0000 7f9dbf21d740 1 -- 172.21.15.114:0/486827877 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5626a7126330 con 0x5626a71c3e20 2024-09-13T18:25:29.306 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.301+0000 7f9d9dffb640 1 -- 172.21.15.114:0/486827877 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9da40b2080 con 0x5626a71c3e20 2024-09-13T18:25:29.306 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.302+0000 7f9d9dffb640 1 -- 172.21.15.114:0/486827877 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f9da40add20 con 0x5626a71c3e20 2024-09-13T18:25:29.306 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.302+0000 7f9dbf21d740 10 client.26107 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:25:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.302+0000 7f9dbf21d740 10 client.26107 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:25:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.302+0000 7f9dbf21d740 20 client.26107 populate_metadata read hostname 'smithi114' 2024-09-13T18:25:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.302+0000 7f9dbf21d740 1 --2- 172.21.15.114:0/486827877 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5626a71ddf70 0x5626a71e0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.302+0000 7f9dbf21d740 4 client.26107 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:25:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.302+0000 7f9dbf21d740 1 -- 172.21.15.114:0/486827877 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5626a711b5d0 con 0x5626a71ddf70 2024-09-13T18:25:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.303+0000 7f9dba1bc640 1 --2- 172.21.15.114:0/486827877 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5626a71ddf70 0x5626a71e0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.303+0000 7f9dba1bc640 1 --2- 172.21.15.114:0/486827877 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5626a71ddf70 0x5626a71e0390 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de3047006e4d3c0b server_cookie=33ffb8a263673f1c in_seq=0 out_seq=0 2024-09-13T18:25:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.303+0000 7f9d9dffb640 10 client.26107 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:25:29.308 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.304+0000 7f9d9dffb640 1 -- 172.21.15.114:0/486827877 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5626a711b5d0 con 0x5626a71ddf70 2024-09-13T18:25:29.308 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.304+0000 7f9d9dffb640 10 client.26107 handle_command_reply: tid=0 2024-09-13T18:25:29.308 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.304+0000 7f9dbf21d740 1 client.26107 shutdown 2024-09-13T18:25:29.308 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.304+0000 7f9dbf21d740 1 -- 172.21.15.114:0/486827877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626a71c3e20 msgr2=0x5626a71cb290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:29.309 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.304+0000 7f9dbf21d740 1 --2- 172.21.15.114:0/486827877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626a71c3e20 0x5626a71cb290 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9da40caa90 tx=0x7f9da40cd200 comp rx=0 tx=0).stop 2024-09-13T18:25:29.309 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.304+0000 7f9dbf21d740 1 -- 172.21.15.114:0/486827877 shutdown_connections 2024-09-13T18:25:29.309 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.304+0000 7f9dbf21d740 1 --2- 172.21.15.114:0/486827877 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5626a71ddf70 0x5626a71e0390 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:29.309 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.304+0000 7f9dbf21d740 1 --2- 172.21.15.114:0/486827877 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626a71ce6c0 0x5626a71cb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:29.309 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.304+0000 7f9dbf21d740 1 --2- 172.21.15.114:0/486827877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626a71c3e20 0x5626a71cb290 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:29.309 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.304+0000 7f9dbf21d740 1 --2- 172.21.15.114:0/486827877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626a71c3500 0x5626a71ce060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:29.309 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.304+0000 7f9dbf21d740 1 -- 172.21.15.114:0/486827877 >> 172.21.15.114:0/486827877 conn(0x5626a71bfa90 msgr2=0x5626a71bed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:29.309 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.304+0000 7f9d9dffb640 0 client.26107 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:25:29.309 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.304+0000 7f9dbf21d740 1 -- 172.21.15.114:0/486827877 shutdown_connections 2024-09-13T18:25:29.310 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.305+0000 7f9dbf21d740 1 -- 172.21.15.114:0/486827877 wait complete. 2024-09-13T18:25:29.310 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.305+0000 7f9dbf21d740 20 client.26107 trim_cache size 0 max 16384 2024-09-13T18:25:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.330+0000 7f9dbf21d740 1 Processor -- start 2024-09-13T18:25:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.330+0000 7f9dbf21d740 1 -- start start 2024-09-13T18:25:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.331+0000 7f9dbf21d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626a71c3500 0x5626a71bf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.331+0000 7f9dbf21d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626a71c3e20 0x5626a71cdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.331+0000 7f9dbf21d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626a71ce6c0 0x5626a71d5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.331+0000 7f9dbf21d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626a71d1170 con 0x5626a71c3500 2024-09-13T18:25:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.331+0000 7f9dbf21d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626a71ba1e0 con 0x5626a71c3e20 2024-09-13T18:25:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.331+0000 7f9dbf21d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626a71c9800 con 0x5626a71ce6c0 2024-09-13T18:25:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.331+0000 7f9dba1bc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626a71c3500 0x5626a71bf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.331+0000 7f9dba1bc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626a71c3500 0x5626a71bf550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42154/0 (socket says 172.21.15.114:42154) 2024-09-13T18:25:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.331+0000 7f9dba1bc640 1 -- 172.21.15.114:0/426376934 learned_addr learned my addr 172.21.15.114:0/426376934 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.331+0000 7f9db99bb640 1 --2- 172.21.15.114:0/426376934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626a71c3e20 0x5626a71cdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.332+0000 7f9dba9bd640 1 --2- 172.21.15.114:0/426376934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626a71ce6c0 0x5626a71d5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.332+0000 7f9dba1bc640 1 -- 172.21.15.114:0/426376934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626a71ce6c0 msgr2=0x5626a71d5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.332+0000 7f9dba1bc640 1 --2- 172.21.15.114:0/426376934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626a71ce6c0 0x5626a71d5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.332+0000 7f9dba1bc640 1 -- 172.21.15.114:0/426376934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626a71c3e20 msgr2=0x5626a71cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.332+0000 7f9dba1bc640 1 --2- 172.21.15.114:0/426376934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626a71c3e20 0x5626a71cdb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.332+0000 7f9dba1bc640 1 -- 172.21.15.114:0/426376934 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626a70c8300 con 0x5626a71c3500 2024-09-13T18:25:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.333+0000 7f9dba1bc640 1 --2- 172.21.15.114:0/426376934 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626a71c3500 0x5626a71bf550 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f9db4066130 tx=0x7f9db4077040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6f9a0d9017a4231 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:29.337 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.333+0000 7f9da9606640 1 -- 172.21.15.114:0/426376934 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db4088070 con 0x5626a71c3500 2024-09-13T18:25:29.337 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.333+0000 7f9da9606640 1 -- 172.21.15.114:0/426376934 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9db407b040 con 0x5626a71c3500 2024-09-13T18:25:29.337 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.333+0000 7f9da9606640 1 -- 172.21.15.114:0/426376934 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db407a040 con 0x5626a71c3500 2024-09-13T18:25:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.334+0000 7f9dbf21d740 1 -- 172.21.15.114:0/426376934 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626a71c3500 msgr2=0x5626a71bf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.334+0000 7f9dbf21d740 1 --2- 172.21.15.114:0/426376934 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626a71c3500 0x5626a71bf550 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f9db4066130 tx=0x7f9db4077040 comp rx=0 tx=0).stop 2024-09-13T18:25:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.334+0000 7f9dbf21d740 1 -- 172.21.15.114:0/426376934 shutdown_connections 2024-09-13T18:25:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.335+0000 7f9dbf21d740 1 --2- 172.21.15.114:0/426376934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626a71ce6c0 0x5626a71d5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.335+0000 7f9dbf21d740 1 --2- 172.21.15.114:0/426376934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626a71c3e20 0x5626a71cdb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.335+0000 7f9dbf21d740 1 --2- 172.21.15.114:0/426376934 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626a71c3500 0x5626a71bf550 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.335+0000 7f9dbf21d740 1 -- 172.21.15.114:0/426376934 >> 172.21.15.114:0/426376934 conn(0x5626a71bfa90 msgr2=0x5626a71c65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.335+0000 7f9dbf21d740 1 -- 172.21.15.114:0/426376934 shutdown_connections 2024-09-13T18:25:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.335+0000 7f9dbf21d740 1 -- 172.21.15.114:0/426376934 wait complete. 2024-09-13T18:25:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.335+0000 7f9dbf21d740 1 -- start start 2024-09-13T18:25:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.335+0000 7f9dbf21d740 1 Processor -- start 2024-09-13T18:25:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.336+0000 7f9dbf21d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626a71c3500 0x5626a71d5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.336+0000 7f9dbf21d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626a71c3e20 0x5626a71c7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.336+0000 7f9dba1bc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626a71c3500 0x5626a71d5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.336+0000 7f9dba1bc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626a71c3500 0x5626a71d5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:45694/0 (socket says 172.21.15.114:45694) 2024-09-13T18:25:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.336+0000 7f9dba1bc640 1 -- 172.21.15.114:0/124486381 learned_addr learned my addr 172.21.15.114:0/124486381 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.336+0000 7f9dbf21d740 1 --2- 172.21.15.114:0/124486381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626a71ce6c0 0x5626a71c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.337+0000 7f9dbf21d740 1 -- 172.21.15.114:0/124486381 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626a71c88e0 con 0x5626a71c3e20 2024-09-13T18:25:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.337+0000 7f9dba9bd640 1 --2- 172.21.15.114:0/124486381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626a71ce6c0 0x5626a71c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.337+0000 7f9dbf21d740 1 -- 172.21.15.114:0/124486381 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626a71c8a50 con 0x5626a71ce6c0 2024-09-13T18:25:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.337+0000 7f9dbf21d740 1 -- 172.21.15.114:0/124486381 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626a7234470 con 0x5626a71c3500 2024-09-13T18:25:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.337+0000 7f9dbf21d740 10 client.0 _ll_register_callbacks cb 0x7f9da40d08b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:25:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.337+0000 7f9dba1bc640 1 -- 172.21.15.114:0/124486381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626a71ce6c0 msgr2=0x5626a71c81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.337+0000 7f9dba1bc640 1 --2- 172.21.15.114:0/124486381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626a71ce6c0 0x5626a71c81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.337+0000 7f9dba1bc640 1 -- 172.21.15.114:0/124486381 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626a71c3e20 msgr2=0x5626a71c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.337+0000 7f9dba1bc640 1 --2- 172.21.15.114:0/124486381 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626a71c3e20 0x5626a71c7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.337+0000 7f9dba1bc640 1 -- 172.21.15.114:0/124486381 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5626a70ed8e0 con 0x5626a71c3500 2024-09-13T18:25:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.338+0000 7f9dba1bc640 1 --2- 172.21.15.114:0/124486381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626a71c3500 0x5626a71d5720 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9db404fb10 tx=0x7f9db4089850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bbf6aef790edc70 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.338+0000 7f9dbf21d740 1 -- 172.21.15.114:0/124486381 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9da4099cd0 con 0x5626a71c3500 2024-09-13T18:25:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.341+0000 7f9d9dffb640 10 client.26113 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:25:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.341+0000 7f9d9dffb640 1 -- 172.21.15.114:0/124486381 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db4088040 con 0x5626a71c3500 2024-09-13T18:25:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.341+0000 7f9d9dffb640 1 -- 172.21.15.114:0/124486381 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f9db4073860 con 0x5626a71c3500 2024-09-13T18:25:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.341+0000 7f9d9dffb640 1 -- 172.21.15.114:0/124486381 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9db407a040 con 0x5626a71c3500 2024-09-13T18:25:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.342+0000 7f9dbf21d740 10 client.26113 fetch_fsmap learned FSMap version 32 2024-09-13T18:25:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.342+0000 7f9dbf21d740 1 -- 172.21.15.114:0/124486381 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({fsmap=32}) v3 -- 0x5626a70d4300 con 0x5626a71c3500 2024-09-13T18:25:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.342+0000 7f9d9dffb640 1 -- 172.21.15.114:0/124486381 <== mon.1 v2:172.21.15.204:3300/0 4 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f9db4080040 con 0x5626a71c3500 2024-09-13T18:25:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.342+0000 7f9dbf21d740 10 client.26113 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:25:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.342+0000 7f9dbf21d740 10 client.26113 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:25:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.342+0000 7f9dbf21d740 20 client.26113 populate_metadata read hostname 'smithi114' 2024-09-13T18:25:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.343+0000 7f9dbf21d740 1 --2- 172.21.15.114:0/124486381 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5626a72358c0 0x5626a7237d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.343+0000 7f9dbf21d740 4 client.26113 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T18:25:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.343+0000 7f9dbf21d740 1 -- 172.21.15.114:0/124486381 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7f9da4099cd0 con 0x5626a72358c0 2024-09-13T18:25:29.347 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.343+0000 7f9db99bb640 1 --2- 172.21.15.114:0/124486381 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5626a72358c0 0x5626a7237d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:29.347 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.344+0000 7f9db99bb640 1 --2- 172.21.15.114:0/124486381 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5626a72358c0 0x5626a7237d00 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2fe6fa750b2763f server_cookie=d5d830087115a7cc in_seq=0 out_seq=0 2024-09-13T18:25:29.347 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.344+0000 7f9d9dffb640 10 client.26113 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:25:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.352+0000 7f9d9dffb640 1 -- 172.21.15.114:0/124486381 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f9da4099cd0 con 0x5626a72358c0 2024-09-13T18:25:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.352+0000 7f9d9dffb640 10 client.26113 handle_command_reply: tid=0 2024-09-13T18:25:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.352+0000 7f9dbf21d740 1 client.26113 shutdown 2024-09-13T18:25:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.353+0000 7f9dbf21d740 1 -- 172.21.15.114:0/124486381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626a71c3500 msgr2=0x5626a71d5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.353+0000 7f9dbf21d740 1 --2- 172.21.15.114:0/124486381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626a71c3500 0x5626a71d5720 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9db404fb10 tx=0x7f9db4089850 comp rx=0 tx=0).stop 2024-09-13T18:25:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.353+0000 7f9dbf21d740 1 -- 172.21.15.114:0/124486381 shutdown_connections 2024-09-13T18:25:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.353+0000 7f9dbf21d740 1 --2- 172.21.15.114:0/124486381 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5626a72358c0 0x5626a7237d00 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.353+0000 7f9dbf21d740 1 --2- 172.21.15.114:0/124486381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626a71ce6c0 0x5626a71c81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.353+0000 7f9dbf21d740 1 --2- 172.21.15.114:0/124486381 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626a71c3e20 0x5626a71c7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.353+0000 7f9dbf21d740 1 --2- 172.21.15.114:0/124486381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626a71c3500 0x5626a71d5720 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:29.357 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.353+0000 7f9dbf21d740 1 -- 172.21.15.114:0/124486381 >> 172.21.15.114:0/124486381 conn(0x5626a71bfa90 msgr2=0x5626a71c9a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:29.357 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.353+0000 7f9d9dffb640 0 client.26113 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:25:29.357 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.353+0000 7f9dbf21d740 1 -- 172.21.15.114:0/124486381 shutdown_connections 2024-09-13T18:25:29.357 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.353+0000 7f9dbf21d740 1 -- 172.21.15.114:0/124486381 wait complete. 2024-09-13T18:25:29.358 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.353+0000 7f9dbf21d740 20 client.26113 trim_cache size 0 max 16384 2024-09-13T18:25:29.358 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-13T18:25:29.358 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-13T18:25:29.358 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-13T18:25:29.358 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-13T18:25:29.359 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.354+0000 7f9dbb1be640 1 -- 172.21.15.114:0/3051665952 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f9da00a1350 msgr2=0x7f9da00a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:29.359 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.354+0000 7f9dbb1be640 1 --2- 172.21.15.114:0/3051665952 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f9da00a1350 0x7f9da00a3810 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9db00057c0 tx=0x7f9db00056d0 comp rx=0 tx=0).stop 2024-09-13T18:25:29.359 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.354+0000 7f9dbb1be640 1 -- 172.21.15.114:0/3051665952 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9dac0a0f60 msgr2=0x7f9dac132680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:29.359 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.354+0000 7f9dbb1be640 1 --2- 172.21.15.114:0/3051665952 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9dac0a0f60 0x7f9dac132680 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f9da4005fd0 tx=0x7f9da400ec20 comp rx=0 tx=0).stop 2024-09-13T18:25:29.359 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.354+0000 7f9dbb1be640 1 -- 172.21.15.114:0/3051665952 shutdown_connections 2024-09-13T18:25:29.359 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.354+0000 7f9dbb1be640 1 --2- 172.21.15.114:0/3051665952 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f9da00a1350 0x7f9da00a3810 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:29.359 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.354+0000 7f9dbb1be640 1 --2- 172.21.15.114:0/3051665952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9dac136a70 0x7f9dac136f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.354+0000 7f9dbb1be640 1 --2- 172.21.15.114:0/3051665952 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9dac0a0f60 0x7f9dac132680 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.355+0000 7f9dbb1be640 1 --2- 172.21.15.114:0/3051665952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dac09be10 0x7f9dac132140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.355+0000 7f9dbb1be640 1 -- 172.21.15.114:0/3051665952 >> 172.21.15.114:0/3051665952 conn(0x7f9dac0093b0 msgr2=0x7f9dac009d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.355+0000 7f9dbb1be640 1 -- 172.21.15.114:0/3051665952 shutdown_connections 2024-09-13T18:25:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:29.355+0000 7f9dbb1be640 1 -- 172.21.15.114:0/3051665952 wait complete. 2024-09-13T18:25:30.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:30 smithi204 ceph-mon[34777]: pgmap v94: 129 pgs: 129 active+clean; 12 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 75 MiB/s wr, 1.21k op/s 2024-09-13T18:25:30.469 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:30 smithi114 ceph-mon[30897]: pgmap v94: 129 pgs: 129 active+clean; 12 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 75 MiB/s wr, 1.21k op/s 2024-09-13T18:25:30.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:30 smithi134 ceph-mon[34283]: pgmap v94: 129 pgs: 129 active+clean; 12 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 75 MiB/s wr, 1.21k op/s 2024-09-13T18:25:32.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:32 smithi204 ceph-mon[34777]: pgmap v95: 129 pgs: 129 active+clean; 12 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 49 MiB/s wr, 792 op/s 2024-09-13T18:25:32.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:32 smithi114 ceph-mon[30897]: pgmap v95: 129 pgs: 129 active+clean; 12 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 49 MiB/s wr, 792 op/s 2024-09-13T18:25:32.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:32 smithi134 ceph-mon[34283]: pgmap v95: 129 pgs: 129 active+clean; 12 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 49 MiB/s wr, 792 op/s 2024-09-13T18:25:34.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:34 smithi204 ceph-mon[34777]: pgmap v96: 129 pgs: 129 active+clean; 12 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 73 MiB/s wr, 1.19k op/s 2024-09-13T18:25:34.469 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:34 smithi114 ceph-mon[30897]: pgmap v96: 129 pgs: 129 active+clean; 12 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 73 MiB/s wr, 1.19k op/s 2024-09-13T18:25:34.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:34 smithi134 ceph-mon[34283]: pgmap v96: 129 pgs: 129 active+clean; 12 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 73 MiB/s wr, 1.19k op/s 2024-09-13T18:25:36.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:36 smithi204 ceph-mon[34777]: pgmap v97: 129 pgs: 129 active+clean; 12 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 49 MiB/s wr, 790 op/s 2024-09-13T18:25:36.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:36 smithi114 ceph-mon[30897]: pgmap v97: 129 pgs: 129 active+clean; 12 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 49 MiB/s wr, 790 op/s 2024-09-13T18:25:36.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:36 smithi134 ceph-mon[34283]: pgmap v97: 129 pgs: 129 active+clean; 12 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 49 MiB/s wr, 790 op/s 2024-09-13T18:25:37.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:37 smithi204 ceph-mon[34777]: scrub summary: idle 2024-09-13T18:25:37.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:37 smithi204 ceph-mon[34777]: scrub summary: idle 2024-09-13T18:25:37.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:37 smithi114 ceph-mon[30897]: scrub summary: idle 2024-09-13T18:25:37.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:37 smithi114 ceph-mon[30897]: scrub summary: idle 2024-09-13T18:25:37.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:37 smithi134 ceph-mon[34283]: scrub summary: idle 2024-09-13T18:25:37.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:37 smithi134 ceph-mon[34283]: scrub summary: idle 2024-09-13T18:25:38.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:38 smithi114 ceph-mon[30897]: scrub summary: idle 2024-09-13T18:25:38.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:38 smithi114 ceph-mon[30897]: pgmap v98: 129 pgs: 129 active+clean; 12 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 49 MiB/s wr, 790 op/s 2024-09-13T18:25:38.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:38 smithi134 ceph-mon[34283]: scrub summary: idle 2024-09-13T18:25:38.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:38 smithi134 ceph-mon[34283]: pgmap v98: 129 pgs: 129 active+clean; 12 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 49 MiB/s wr, 790 op/s 2024-09-13T18:25:38.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:38 smithi204 ceph-mon[34777]: scrub summary: idle 2024-09-13T18:25:38.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:38 smithi204 ceph-mon[34777]: pgmap v98: 129 pgs: 129 active+clean; 12 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 49 MiB/s wr, 790 op/s 2024-09-13T18:25:39.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:39 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:25:39.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:39 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:25:39.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:39 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:25:39.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:39 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:25:39.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:39 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:25:39.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:39 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:25:39.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:39 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:25:39.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:39 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:25:39.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:39 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:25:39.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:39 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:25:39.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:39 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:25:39.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:39 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:25:40.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:40 smithi114 ceph-mon[30897]: pgmap v99: 129 pgs: 129 active+clean; 12 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 73 MiB/s wr, 1.18k op/s 2024-09-13T18:25:40.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:40 smithi134 ceph-mon[34283]: pgmap v99: 129 pgs: 129 active+clean; 12 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 73 MiB/s wr, 1.18k op/s 2024-09-13T18:25:40.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:40 smithi204 ceph-mon[34777]: pgmap v99: 129 pgs: 129 active+clean; 12 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 73 MiB/s wr, 1.18k op/s 2024-09-13T18:25:42.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:42 smithi204 ceph-mon[34777]: pgmap v100: 129 pgs: 129 active+clean; 12 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 49 MiB/s wr, 784 op/s 2024-09-13T18:25:42.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:42 smithi114 ceph-mon[30897]: pgmap v100: 129 pgs: 129 active+clean; 12 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 49 MiB/s wr, 784 op/s 2024-09-13T18:25:42.511 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.508+0000 7fed8b7c7640 1 -- 172.21.15.114:0/3414617261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed84119400 msgr2=0x7fed8411b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:42.511 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.508+0000 7fed8b7c7640 1 --2- 172.21.15.114:0/3414617261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed84119400 0x7fed8411b7f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fed7c009cd0 tx=0x7fed7c02a930 comp rx=0 tx=0).stop 2024-09-13T18:25:42.512 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.508+0000 7fed8b7c7640 1 -- 172.21.15.114:0/3414617261 shutdown_connections 2024-09-13T18:25:42.512 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.508+0000 7fed8b7c7640 1 --2- 172.21.15.114:0/3414617261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed84119400 0x7fed8411b7f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:42.512 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.508+0000 7fed8b7c7640 1 --2- 172.21.15.114:0/3414617261 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed84107890 0x7fed84107cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:42.512 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.508+0000 7fed8b7c7640 1 --2- 172.21.15.114:0/3414617261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed841074b0 0x7fed8410e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:42.512 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.508+0000 7fed8b7c7640 1 -- 172.21.15.114:0/3414617261 >> 172.21.15.114:0/3414617261 conn(0x7fed84071d80 msgr2=0x7fed8406c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:42.512 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.509+0000 7fed8b7c7640 1 -- 172.21.15.114:0/3414617261 shutdown_connections 2024-09-13T18:25:42.512 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.509+0000 7fed8b7c7640 1 -- 172.21.15.114:0/3414617261 wait complete. 2024-09-13T18:25:42.512 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.510+0000 7fed8b7c7640 1 Processor -- start 2024-09-13T18:25:42.514 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.510+0000 7fed8b7c7640 1 -- start start 2024-09-13T18:25:42.514 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.511+0000 7fed8b7c7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed841074b0 0x7fed84116a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:42.514 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.511+0000 7fed8b7c7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed84107890 0x7fed8410fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:42.514 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.511+0000 7fed8b7c7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed84119400 0x7fed84110010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.511+0000 7fed8b7c7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed84110550 con 0x7fed84119400 2024-09-13T18:25:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.511+0000 7fed8b7c7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed841106c0 con 0x7fed841074b0 2024-09-13T18:25:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.511+0000 7fed8b7c7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed84110880 con 0x7fed84107890 2024-09-13T18:25:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.511+0000 7fed89d64640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed84119400 0x7fed84110010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.511+0000 7fed89d64640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed84119400 0x7fed84110010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43184/0 (socket says 172.21.15.114:43184) 2024-09-13T18:25:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.511+0000 7fed89d64640 1 -- 172.21.15.114:0/819027953 learned_addr learned my addr 172.21.15.114:0/819027953 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.511+0000 7fed89563640 1 --2- 172.21.15.114:0/819027953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed841074b0 0x7fed84116a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.511+0000 7fed88d62640 1 --2- 172.21.15.114:0/819027953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed84107890 0x7fed8410fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.512+0000 7fed89d64640 1 -- 172.21.15.114:0/819027953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed84107890 msgr2=0x7fed8410fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.512+0000 7fed89d64640 1 --2- 172.21.15.114:0/819027953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed84107890 0x7fed8410fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.512+0000 7fed89d64640 1 -- 172.21.15.114:0/819027953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed841074b0 msgr2=0x7fed84116a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.512+0000 7fed89d64640 1 --2- 172.21.15.114:0/819027953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed841074b0 0x7fed84116a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.512+0000 7fed89d64640 1 -- 172.21.15.114:0/819027953 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed7c009900 con 0x7fed84119400 2024-09-13T18:25:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.512+0000 7fed89d64640 1 --2- 172.21.15.114:0/819027953 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed84119400 0x7fed84110010 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fed7c005e30 tx=0x7fed7c030820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:42.517 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.513+0000 7fed767fc640 1 -- 172.21.15.114:0/819027953 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7c030da0 con 0x7fed84119400 2024-09-13T18:25:42.517 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.513+0000 7fed767fc640 1 -- 172.21.15.114:0/819027953 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fed7c03f070 con 0x7fed84119400 2024-09-13T18:25:42.517 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.513+0000 7fed8b7c7640 1 -- 172.21.15.114:0/819027953 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed841145f0 con 0x7fed84119400 2024-09-13T18:25:42.517 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.513+0000 7fed8b7c7640 1 -- 172.21.15.114:0/819027953 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed84114b50 con 0x7fed84119400 2024-09-13T18:25:42.517 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.513+0000 7fed767fc640 1 -- 172.21.15.114:0/819027953 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7c03b660 con 0x7fed84119400 2024-09-13T18:25:42.520 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.517+0000 7fed8e5c4740 1 Processor -- start 2024-09-13T18:25:42.523 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.517+0000 7fed767fc640 1 -- 172.21.15.114:0/819027953 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fed7c00a040 con 0x7fed84119400 2024-09-13T18:25:42.523 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.519+0000 7fed8e5c4740 1 -- start start 2024-09-13T18:25:42.523 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.519+0000 7fed767fc640 1 --2- 172.21.15.114:0/819027953 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fed700a1330 0x7fed700a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:42.523 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.519+0000 7fed8e5c4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a219a7330 0x555a219a7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:42.523 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.519+0000 7fed8e5c4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a219a7c50 0x555a219b1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:42.523 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.519+0000 7fed8e5c4740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a219b24f0 0x555a219b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:42.523 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.519+0000 7fed8e5c4740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a21912c30 con 0x555a219b24f0 2024-09-13T18:25:42.523 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.519+0000 7fed8e5c4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a219103c0 con 0x555a219a7330 2024-09-13T18:25:42.523 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.519+0000 7fed8e5c4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a2190bcb0 con 0x555a219a7c50 2024-09-13T18:25:42.524 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.519+0000 7fed767fc640 1 -- 172.21.15.114:0/819027953 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fed7c041070 con 0x7fed84119400 2024-09-13T18:25:42.524 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.520+0000 7fed88d62640 1 --2- 172.21.15.114:0/819027953 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fed700a1330 0x7fed700a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:42.524 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.520+0000 7fed88d62640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a219a7c50 0x555a219b1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:42.524 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.520+0000 7fed89563640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a219a7330 0x555a219a7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:42.524 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.520+0000 7fed88d62640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a219a7c50 0x555a219b1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:59880/0 (socket says 172.21.15.114:59880) 2024-09-13T18:25:42.524 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.520+0000 7fed88d62640 1 -- 172.21.15.114:0/2638791247 learned_addr learned my addr 172.21.15.114:0/2638791247 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:42.525 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.520+0000 7fed89d64640 1 --2- 172.21.15.114:0/2638791247 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a219b24f0 0x555a219b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:42.525 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.520+0000 7fed89563640 1 -- 172.21.15.114:0/2638791247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a219a7c50 msgr2=0x555a219b1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:42.525 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.520+0000 7fed89563640 1 --2- 172.21.15.114:0/2638791247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a219a7c50 0x555a219b1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:42.525 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.520+0000 7fed89563640 1 -- 172.21.15.114:0/2638791247 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a219b24f0 msgr2=0x555a219b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:42.525 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.520+0000 7fed89563640 1 --2- 172.21.15.114:0/2638791247 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a219b24f0 0x555a219b48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:42.525 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.520+0000 7fed89563640 1 -- 172.21.15.114:0/2638791247 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a218b2910 con 0x555a219a7330 2024-09-13T18:25:42.525 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.521+0000 7fed88d62640 1 --2- 172.21.15.114:0/819027953 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fed700a1330 0x7fed700a37f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x555a219a33c0 tx=0x7fed8000b5b0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:42.525 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.521+0000 7fed89563640 1 --2- 172.21.15.114:0/2638791247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a219a7330 0x555a219a7710 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fed7800bbf0 tx=0x7fed780107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62faf33c0703b11e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:42.525 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.521+0000 7fed6f7fe640 1 -- 172.21.15.114:0/2638791247 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed78004030 con 0x555a219a7330 2024-09-13T18:25:42.526 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.521+0000 7fed6f7fe640 1 -- 172.21.15.114:0/2638791247 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fed78015070 con 0x555a219a7330 2024-09-13T18:25:42.526 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.521+0000 7fed6f7fe640 1 -- 172.21.15.114:0/2638791247 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed78011440 con 0x555a219a7330 2024-09-13T18:25:42.529 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.522+0000 7fed8e5c4740 1 -- 172.21.15.114:0/2638791247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a219a7330 msgr2=0x555a219a7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:42.529 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.522+0000 7fed8e5c4740 1 --2- 172.21.15.114:0/2638791247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a219a7330 0x555a219a7710 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fed7800bbf0 tx=0x7fed780107a0 comp rx=0 tx=0).stop 2024-09-13T18:25:42.529 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.522+0000 7fed8e5c4740 1 -- 172.21.15.114:0/2638791247 shutdown_connections 2024-09-13T18:25:42.529 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.522+0000 7fed8e5c4740 1 --2- 172.21.15.114:0/2638791247 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a219b24f0 0x555a219b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:42.529 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.522+0000 7fed8e5c4740 1 --2- 172.21.15.114:0/2638791247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a219a7c50 0x555a219b1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:42.529 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.522+0000 7fed8e5c4740 1 --2- 172.21.15.114:0/2638791247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a219a7330 0x555a219a7710 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:42.529 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.522+0000 7fed8e5c4740 1 -- 172.21.15.114:0/2638791247 >> 172.21.15.114:0/2638791247 conn(0x555a219a38c0 msgr2=0x555a219a3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:42.529 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.522+0000 7fed8e5c4740 1 -- 172.21.15.114:0/2638791247 shutdown_connections 2024-09-13T18:25:42.529 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.522+0000 7fed8e5c4740 1 -- 172.21.15.114:0/2638791247 wait complete. 2024-09-13T18:25:42.530 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.523+0000 7fed8e5c4740 1 -- start start 2024-09-13T18:25:42.530 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.523+0000 7fed8e5c4740 1 Processor -- start 2024-09-13T18:25:42.530 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.523+0000 7fed8e5c4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a219a7330 0x555a219b1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:42.530 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.523+0000 7fed8e5c4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a219a7c50 0x555a219af160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:42.530 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.523+0000 7fed8e5c4740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a219b24f0 0x555a219af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:42.530 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.524+0000 7fed8e5c4740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a218c9e80 con 0x555a219b24f0 2024-09-13T18:25:42.530 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.524+0000 7fed8e5c4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a218c9b20 con 0x555a219a7c50 2024-09-13T18:25:42.530 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.524+0000 7fed8e5c4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a2188f260 con 0x555a219a7330 2024-09-13T18:25:42.530 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.524+0000 7fed8e5c4740 10 client.0 _ll_register_callbacks cb 0x555a218c4df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:25:42.530 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.524+0000 7fed89563640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a219a7330 0x555a219b1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:42.530 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.524+0000 7fed89563640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a219a7330 0x555a219b1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:59884/0 (socket says 172.21.15.114:59884) 2024-09-13T18:25:42.530 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.524+0000 7fed89563640 1 -- 172.21.15.114:0/31813047 learned_addr learned my addr 172.21.15.114:0/31813047 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.524+0000 7fed88d62640 1 --2- 172.21.15.114:0/31813047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a219a7c50 0x555a219af160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.524+0000 7fed89d64640 1 --2- 172.21.15.114:0/31813047 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a219b24f0 0x555a219af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.524+0000 7fed89563640 1 -- 172.21.15.114:0/31813047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a219a7c50 msgr2=0x555a219af160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.524+0000 7fed89563640 1 --2- 172.21.15.114:0/31813047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a219a7c50 0x555a219af160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.524+0000 7fed89563640 1 -- 172.21.15.114:0/31813047 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a219b24f0 msgr2=0x555a219af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.524+0000 7fed89563640 1 --2- 172.21.15.114:0/31813047 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a219b24f0 0x555a219af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.524+0000 7fed89563640 1 -- 172.21.15.114:0/31813047 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed7800b820 con 0x555a219a7330 2024-09-13T18:25:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.524+0000 7fed89563640 1 --2- 172.21.15.114:0/31813047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a219a7330 0x555a219b1e70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fed78011010 tx=0x7fed78003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=265473fa4bc31ee1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.524+0000 7fed6cff9640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:25:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.525+0000 7fed6cff9640 1 -- 172.21.15.114:0/31813047 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed78021070 con 0x555a219a7330 2024-09-13T18:25:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.525+0000 7fed6cff9640 1 -- 172.21.15.114:0/31813047 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fed78017070 con 0x555a219a7330 2024-09-13T18:25:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.525+0000 7fed8e5c4740 1 -- 172.21.15.114:0/31813047 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555a21934680 con 0x555a219a7330 2024-09-13T18:25:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.525+0000 7fed6cff9640 1 -- 172.21.15.114:0/31813047 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed78003990 con 0x555a219a7330 2024-09-13T18:25:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.525+0000 7fed8e5c4740 10 client.26125 fetch_fsmap learned FSMap version 32 2024-09-13T18:25:42.532 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.525+0000 7fed8e5c4740 1 -- 172.21.15.114:0/31813047 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x555a2190a330 con 0x555a219a7330 2024-09-13T18:25:42.532 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.525+0000 7fed6cff9640 1 -- 172.21.15.114:0/31813047 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x555a21934680 con 0x555a219a7330 2024-09-13T18:25:42.532 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.525+0000 7fed6cff9640 1 -- 172.21.15.114:0/31813047 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fed78004ca0 con 0x555a219a7330 2024-09-13T18:25:42.532 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.525+0000 7fed8e5c4740 10 client.26125 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:25:42.532 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.525+0000 7fed8e5c4740 10 client.26125 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:25:42.532 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.525+0000 7fed8e5c4740 20 client.26125 populate_metadata read hostname 'smithi114' 2024-09-13T18:25:42.532 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.525+0000 7fed8e5c4740 1 --2- 172.21.15.114:0/31813047 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x555a219c1fc0 0x555a219c43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:42.532 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.526+0000 7fed8e5c4740 4 client.26125 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:25:42.532 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.526+0000 7fed8e5c4740 1 -- 172.21.15.114:0/31813047 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555a218ff5d0 con 0x555a219c1fc0 2024-09-13T18:25:42.532 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.526+0000 7fed88d62640 1 --2- 172.21.15.114:0/31813047 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x555a219c1fc0 0x555a219c43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.526+0000 7fed88d62640 1 --2- 172.21.15.114:0/31813047 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x555a219c1fc0 0x555a219c43e0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c480d034b7ca66ba server_cookie=cfc54d50059baebc in_seq=0 out_seq=0 2024-09-13T18:25:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.526+0000 7fed6cff9640 10 client.26125 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:25:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.527+0000 7fed6cff9640 1 -- 172.21.15.114:0/31813047 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x555a218ff5d0 con 0x555a219c1fc0 2024-09-13T18:25:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.527+0000 7fed6cff9640 10 client.26125 handle_command_reply: tid=0 2024-09-13T18:25:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.527+0000 7fed8e5c4740 1 client.26125 shutdown 2024-09-13T18:25:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.528+0000 7fed8e5c4740 1 -- 172.21.15.114:0/31813047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a219a7330 msgr2=0x555a219b1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.528+0000 7fed8e5c4740 1 --2- 172.21.15.114:0/31813047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a219a7330 0x555a219b1e70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fed78011010 tx=0x7fed78003820 comp rx=0 tx=0).stop 2024-09-13T18:25:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.528+0000 7fed8e5c4740 1 -- 172.21.15.114:0/31813047 shutdown_connections 2024-09-13T18:25:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.528+0000 7fed8e5c4740 1 --2- 172.21.15.114:0/31813047 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x555a219c1fc0 0x555a219c43e0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.528+0000 7fed8e5c4740 1 --2- 172.21.15.114:0/31813047 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a219b24f0 0x555a219af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.528+0000 7fed8e5c4740 1 --2- 172.21.15.114:0/31813047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a219a7c50 0x555a219af160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.528+0000 7fed8e5c4740 1 --2- 172.21.15.114:0/31813047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a219a7330 0x555a219b1e70 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.528+0000 7fed8e5c4740 1 -- 172.21.15.114:0/31813047 >> 172.21.15.114:0/31813047 conn(0x555a219a38c0 msgr2=0x555a219a2b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.528+0000 7fed6cff9640 0 client.26125 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:25:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.528+0000 7fed8e5c4740 1 -- 172.21.15.114:0/31813047 shutdown_connections 2024-09-13T18:25:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.528+0000 7fed8e5c4740 1 -- 172.21.15.114:0/31813047 wait complete. 2024-09-13T18:25:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.528+0000 7fed8e5c4740 20 client.26125 trim_cache size 0 max 16384 2024-09-13T18:25:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.552+0000 7fed8e5c4740 1 Processor -- start 2024-09-13T18:25:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.552+0000 7fed8e5c4740 1 -- start start 2024-09-13T18:25:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.552+0000 7fed8e5c4740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a219a7330 0x555a219b1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.552+0000 7fed8e5c4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a219a7c50 0x555a219b9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.553+0000 7fed8e5c4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a219b24f0 0x555a219b9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.553+0000 7fed8e5c4740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a219ad690 con 0x555a219a7330 2024-09-13T18:25:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.553+0000 7fed8e5c4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a219a8550 con 0x555a219a7c50 2024-09-13T18:25:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.553+0000 7fed8e5c4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a219a7a40 con 0x555a219b24f0 2024-09-13T18:25:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.553+0000 7fed88d62640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a219a7c50 0x555a219b9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.553+0000 7fed88d62640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a219a7c50 0x555a219b9250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:43268/0 (socket says 172.21.15.114:43268) 2024-09-13T18:25:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.553+0000 7fed88d62640 1 -- 172.21.15.114:0/2351084014 learned_addr learned my addr 172.21.15.114:0/2351084014 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.553+0000 7fed89d64640 1 --2- 172.21.15.114:0/2351084014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a219b24f0 0x555a219b9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.553+0000 7fed89563640 1 --2- 172.21.15.114:0/2351084014 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a219a7330 0x555a219b1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.553+0000 7fed89d64640 1 -- 172.21.15.114:0/2351084014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a219a7c50 msgr2=0x555a219b9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.553+0000 7fed89d64640 1 --2- 172.21.15.114:0/2351084014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a219a7c50 0x555a219b9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.553+0000 7fed89d64640 1 -- 172.21.15.114:0/2351084014 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a219a7330 msgr2=0x555a219b1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.553+0000 7fed89d64640 1 --2- 172.21.15.114:0/2351084014 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a219a7330 0x555a219b1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.553+0000 7fed89d64640 1 -- 172.21.15.114:0/2351084014 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a218ac300 con 0x555a219b24f0 2024-09-13T18:25:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.554+0000 7fed89d64640 1 --2- 172.21.15.114:0/2351084014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a219b24f0 0x555a219b9790 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fed7c0eb6e0 tx=0x7fed7c03d040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0f525ef7948f660 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.554+0000 7fed6ffff640 1 -- 172.21.15.114:0/2351084014 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7c0bb700 con 0x555a219b24f0 2024-09-13T18:25:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.554+0000 7fed6ffff640 1 -- 172.21.15.114:0/2351084014 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fed7c0bb000 con 0x555a219b24f0 2024-09-13T18:25:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.555+0000 7fed8e5c4740 1 -- 172.21.15.114:0/2351084014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a219b24f0 msgr2=0x555a219b9790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.555+0000 7fed8e5c4740 1 --2- 172.21.15.114:0/2351084014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a219b24f0 0x555a219b9790 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fed7c0eb6e0 tx=0x7fed7c03d040 comp rx=0 tx=0).stop 2024-09-13T18:25:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.556+0000 7fed8e5c4740 1 -- 172.21.15.114:0/2351084014 shutdown_connections 2024-09-13T18:25:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.556+0000 7fed8e5c4740 1 --2- 172.21.15.114:0/2351084014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a219b24f0 0x555a219b9790 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.556+0000 7fed8e5c4740 1 --2- 172.21.15.114:0/2351084014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a219a7c50 0x555a219b9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.556+0000 7fed8e5c4740 1 --2- 172.21.15.114:0/2351084014 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a219a7330 0x555a219b1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.556+0000 7fed8e5c4740 1 -- 172.21.15.114:0/2351084014 >> 172.21.15.114:0/2351084014 conn(0x555a219a38c0 msgr2=0x555a219aa410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.556+0000 7fed8e5c4740 1 -- 172.21.15.114:0/2351084014 shutdown_connections 2024-09-13T18:25:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.556+0000 7fed8e5c4740 1 -- 172.21.15.114:0/2351084014 wait complete. 2024-09-13T18:25:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.556+0000 7fed8e5c4740 1 -- start start 2024-09-13T18:25:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.556+0000 7fed8e5c4740 1 Processor -- start 2024-09-13T18:25:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.557+0000 7fed8e5c4740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a219a7330 0x555a219b9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.557+0000 7fed8e5c4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a219a7c50 0x555a219b57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.557+0000 7fed8e5c4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a219b24f0 0x555a219b5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.557+0000 7fed8e5c4740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a219b6270 con 0x555a219a7330 2024-09-13T18:25:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.557+0000 7fed8e5c4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a219b63e0 con 0x555a219b24f0 2024-09-13T18:25:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.557+0000 7fed8e5c4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a219b65a0 con 0x555a219a7c50 2024-09-13T18:25:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.557+0000 7fed8e5c4740 10 client.0 _ll_register_callbacks cb 0x7fed7800b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:25:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.557+0000 7fed88d62640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a219a7c50 0x555a219b57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.557+0000 7fed88d62640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a219a7c50 0x555a219b57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:59908/0 (socket says 172.21.15.114:59908) 2024-09-13T18:25:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.557+0000 7fed88d62640 1 -- 172.21.15.114:0/3375806022 learned_addr learned my addr 172.21.15.114:0/3375806022 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.557+0000 7fed89563640 1 --2- 172.21.15.114:0/3375806022 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a219a7330 0x555a219b9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.557+0000 7fed89d64640 1 --2- 172.21.15.114:0/3375806022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a219b24f0 0x555a219b5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.558+0000 7fed88d62640 1 -- 172.21.15.114:0/3375806022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a219b24f0 msgr2=0x555a219b5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.558+0000 7fed88d62640 1 --2- 172.21.15.114:0/3375806022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a219b24f0 0x555a219b5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.558+0000 7fed88d62640 1 -- 172.21.15.114:0/3375806022 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a219a7330 msgr2=0x555a219b9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.558+0000 7fed88d62640 1 --2- 172.21.15.114:0/3375806022 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a219a7330 0x555a219b9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.558+0000 7fed88d62640 1 -- 172.21.15.114:0/3375806022 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed84114b50 con 0x555a219a7c50 2024-09-13T18:25:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.559+0000 7fed88d62640 1 --2- 172.21.15.114:0/3375806022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a219a7c50 0x555a219b57f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fed8000b570 tx=0x7fed80004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51016e8b25fc6ede server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.559+0000 7fed6cff9640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:25:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.559+0000 7fed6cff9640 1 -- 172.21.15.114:0/3375806022 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed8002b680 con 0x555a219a7c50 2024-09-13T18:25:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.559+0000 7fed6cff9640 1 -- 172.21.15.114:0/3375806022 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fed80024040 con 0x555a219a7c50 2024-09-13T18:25:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.559+0000 7fed8e5c4740 1 -- 172.21.15.114:0/3375806022 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555a215ff300 con 0x555a219a7c50 2024-09-13T18:25:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.560+0000 7fed6cff9640 1 -- 172.21.15.114:0/3375806022 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed8002b7e0 con 0x555a219a7c50 2024-09-13T18:25:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.560+0000 7fed8e5c4740 10 client.26131 fetch_fsmap learned FSMap version 32 2024-09-13T18:25:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.560+0000 7fed8e5c4740 1 -- 172.21.15.114:0/3375806022 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x555a218d18e0 con 0x555a219a7c50 2024-09-13T18:25:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.560+0000 7fed6cff9640 1 -- 172.21.15.114:0/3375806022 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x555a215ff300 con 0x555a219a7c50 2024-09-13T18:25:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.560+0000 7fed6cff9640 1 -- 172.21.15.114:0/3375806022 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fed80037820 con 0x555a219a7c50 2024-09-13T18:25:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.560+0000 7fed8e5c4740 10 client.26131 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:25:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.560+0000 7fed8e5c4740 10 client.26131 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:25:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.560+0000 7fed8e5c4740 20 client.26131 populate_metadata read hostname 'smithi114' 2024-09-13T18:25:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.560+0000 7fed8e5c4740 1 --2- 172.21.15.114:0/3375806022 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x555a21a19c10 0x555a21a1c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.560+0000 7fed8e5c4740 4 client.26131 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T18:25:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.560+0000 7fed8e5c4740 1 -- 172.21.15.114:0/3375806022 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555a21610600 con 0x555a21a19c10 2024-09-13T18:25:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.561+0000 7fed89563640 1 --2- 172.21.15.114:0/3375806022 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x555a21a19c10 0x555a21a1c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.564+0000 7fed89563640 1 --2- 172.21.15.114:0/3375806022 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x555a21a19c10 0x555a21a1c030 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccb00012baecdf4b server_cookie=b2f4c7ae7cf2c8b2 in_seq=0 out_seq=0 2024-09-13T18:25:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.564+0000 7fed6cff9640 10 client.26131 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:25:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.569+0000 7fed6cff9640 1 -- 172.21.15.114:0/3375806022 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555a21610600 con 0x555a21a19c10 2024-09-13T18:25:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.569+0000 7fed6cff9640 10 client.26131 handle_command_reply: tid=0 2024-09-13T18:25:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.569+0000 7fed8e5c4740 1 client.26131 shutdown 2024-09-13T18:25:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.569+0000 7fed8e5c4740 1 -- 172.21.15.114:0/3375806022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a219a7c50 msgr2=0x555a219b57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.569+0000 7fed8e5c4740 1 --2- 172.21.15.114:0/3375806022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a219a7c50 0x555a219b57f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fed8000b570 tx=0x7fed80004730 comp rx=0 tx=0).stop 2024-09-13T18:25:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.570+0000 7fed8e5c4740 1 -- 172.21.15.114:0/3375806022 shutdown_connections 2024-09-13T18:25:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.570+0000 7fed8e5c4740 1 --2- 172.21.15.114:0/3375806022 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x555a21a19c10 0x555a21a1c030 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.570+0000 7fed8e5c4740 1 --2- 172.21.15.114:0/3375806022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a219b24f0 0x555a219b5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.570+0000 7fed8e5c4740 1 --2- 172.21.15.114:0/3375806022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a219a7c50 0x555a219b57f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.570+0000 7fed8e5c4740 1 --2- 172.21.15.114:0/3375806022 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a219a7330 0x555a219b9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.570+0000 7fed8e5c4740 1 -- 172.21.15.114:0/3375806022 >> 172.21.15.114:0/3375806022 conn(0x555a219a38c0 msgr2=0x555a219ad950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.570+0000 7fed6cff9640 0 client.26131 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:25:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.570+0000 7fed8e5c4740 1 -- 172.21.15.114:0/3375806022 shutdown_connections 2024-09-13T18:25:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.570+0000 7fed8e5c4740 1 -- 172.21.15.114:0/3375806022 wait complete. 2024-09-13T18:25:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.570+0000 7fed8e5c4740 20 client.26131 trim_cache size 0 max 16384 2024-09-13T18:25:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-13T18:25:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-13T18:25:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-13T18:25:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-13T18:25:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.571+0000 7fed8b7c7640 1 -- 172.21.15.114:0/819027953 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fed700a1330 msgr2=0x7fed700a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.571+0000 7fed8b7c7640 1 --2- 172.21.15.114:0/819027953 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fed700a1330 0x7fed700a37f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x555a219a33c0 tx=0x7fed8000b5b0 comp rx=0 tx=0).stop 2024-09-13T18:25:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.571+0000 7fed8b7c7640 1 -- 172.21.15.114:0/819027953 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed84119400 msgr2=0x7fed84110010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.571+0000 7fed8b7c7640 1 --2- 172.21.15.114:0/819027953 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed84119400 0x7fed84110010 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fed7c005e30 tx=0x7fed7c030820 comp rx=0 tx=0).stop 2024-09-13T18:25:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.571+0000 7fed8b7c7640 1 -- 172.21.15.114:0/819027953 shutdown_connections 2024-09-13T18:25:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.571+0000 7fed8b7c7640 1 --2- 172.21.15.114:0/819027953 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fed700a1330 0x7fed700a37f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.571+0000 7fed8b7c7640 1 --2- 172.21.15.114:0/819027953 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed84119400 0x7fed84110010 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.571+0000 7fed8b7c7640 1 --2- 172.21.15.114:0/819027953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed84107890 0x7fed8410fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.571+0000 7fed8b7c7640 1 --2- 172.21.15.114:0/819027953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed841074b0 0x7fed84116a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.571+0000 7fed8b7c7640 1 -- 172.21.15.114:0/819027953 >> 172.21.15.114:0/819027953 conn(0x7fed84071d80 msgr2=0x7fed8411a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.571+0000 7fed8b7c7640 1 -- 172.21.15.114:0/819027953 shutdown_connections 2024-09-13T18:25:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:42.571+0000 7fed8b7c7640 1 -- 172.21.15.114:0/819027953 wait complete. 2024-09-13T18:25:42.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:42 smithi134 ceph-mon[34283]: pgmap v100: 129 pgs: 129 active+clean; 12 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 49 MiB/s wr, 784 op/s 2024-09-13T18:25:44.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:44 smithi114 ceph-mon[30897]: pgmap v101: 129 pgs: 129 active+clean; 12 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 70 MiB/s wr, 1.13k op/s 2024-09-13T18:25:44.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:44 smithi134 ceph-mon[34283]: pgmap v101: 129 pgs: 129 active+clean; 12 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 70 MiB/s wr, 1.13k op/s 2024-09-13T18:25:44.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:44 smithi204 ceph-mon[34777]: pgmap v101: 129 pgs: 129 active+clean; 12 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 70 MiB/s wr, 1.13k op/s 2024-09-13T18:25:46.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:46 smithi114 ceph-mon[30897]: pgmap v102: 129 pgs: 129 active+clean; 12 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 48 MiB/s wr, 778 op/s 2024-09-13T18:25:46.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:46 smithi134 ceph-mon[34283]: pgmap v102: 129 pgs: 129 active+clean; 12 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 48 MiB/s wr, 778 op/s 2024-09-13T18:25:46.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:46 smithi204 ceph-mon[34777]: pgmap v102: 129 pgs: 129 active+clean; 12 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 48 MiB/s wr, 778 op/s 2024-09-13T18:25:48.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:48 smithi114 ceph-mon[30897]: pgmap v103: 129 pgs: 129 active+clean; 12 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 48 MiB/s wr, 778 op/s 2024-09-13T18:25:48.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:48 smithi134 ceph-mon[34283]: pgmap v103: 129 pgs: 129 active+clean; 12 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 48 MiB/s wr, 778 op/s 2024-09-13T18:25:48.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:48 smithi204 ceph-mon[34777]: pgmap v103: 129 pgs: 129 active+clean; 12 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 48 MiB/s wr, 778 op/s 2024-09-13T18:25:50.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:50 smithi114 ceph-mon[30897]: pgmap v104: 129 pgs: 129 active+clean; 13 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 72 MiB/s wr, 1.16k op/s 2024-09-13T18:25:50.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:50 smithi134 ceph-mon[34283]: pgmap v104: 129 pgs: 129 active+clean; 13 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 72 MiB/s wr, 1.16k op/s 2024-09-13T18:25:50.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:50 smithi204 ceph-mon[34777]: pgmap v104: 129 pgs: 129 active+clean; 13 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 72 MiB/s wr, 1.16k op/s 2024-09-13T18:25:52.287 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T18:25:52.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:52 smithi114 ceph-mon[30897]: pgmap v105: 129 pgs: 129 active+clean; 13 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 48 MiB/s wr, 776 op/s 2024-09-13T18:25:52.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.496+0000 7f5b9673a640 1 -- 172.21.15.114:0/1021748135 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b901074b0 msgr2=0x7f5b9010e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:52.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.496+0000 7f5b9673a640 1 --2- 172.21.15.114:0/1021748135 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b901074b0 0x7f5b9010e510 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f5b88009ca0 tx=0x7f5b8802a8c0 comp rx=0 tx=0).stop 2024-09-13T18:25:52.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.496+0000 7f5b9673a640 1 -- 172.21.15.114:0/1021748135 shutdown_connections 2024-09-13T18:25:52.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.496+0000 7f5b9673a640 1 --2- 172.21.15.114:0/1021748135 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b90119400 0x7f5b9011b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:52.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.496+0000 7f5b9673a640 1 --2- 172.21.15.114:0/1021748135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b90107890 0x7f5b90107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:52.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.496+0000 7f5b9673a640 1 --2- 172.21.15.114:0/1021748135 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b901074b0 0x7f5b9010e510 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:52.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.496+0000 7f5b9673a640 1 -- 172.21.15.114:0/1021748135 >> 172.21.15.114:0/1021748135 conn(0x7f5b9006af40 msgr2=0x7f5b9006b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:52.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.497+0000 7f5b9673a640 1 -- 172.21.15.114:0/1021748135 shutdown_connections 2024-09-13T18:25:52.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.497+0000 7f5b9673a640 1 -- 172.21.15.114:0/1021748135 wait complete. 2024-09-13T18:25:52.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.497+0000 7f5b9673a640 1 Processor -- start 2024-09-13T18:25:52.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.497+0000 7f5b9673a640 1 -- start start 2024-09-13T18:25:52.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.498+0000 7f5b9673a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b901074b0 0x7f5b901a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:52.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.498+0000 7f5b9673a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b90107890 0x7f5b901a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:52.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.498+0000 7f5b9673a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b90119400 0x7f5b901a9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:52.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.498+0000 7f5b9673a640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b901a9e00 con 0x7f5b90107890 2024-09-13T18:25:52.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.498+0000 7f5b9673a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b901a9f70 con 0x7f5b901074b0 2024-09-13T18:25:52.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.498+0000 7f5b9673a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b901aa130 con 0x7f5b90119400 2024-09-13T18:25:52.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.498+0000 7f5b95738640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b901074b0 0x7f5b901a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:52.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.498+0000 7f5b95738640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b901074b0 0x7f5b901a4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:48350/0 (socket says 172.21.15.114:48350) 2024-09-13T18:25:52.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.498+0000 7f5b94f37640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b90107890 0x7f5b901a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:52.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.498+0000 7f5b95738640 1 -- 172.21.15.114:0/4280458876 learned_addr learned my addr 172.21.15.114:0/4280458876 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:52.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.498+0000 7f5b95f39640 1 --2- 172.21.15.114:0/4280458876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b90119400 0x7f5b901a9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:52.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.499+0000 7f5b95738640 1 -- 172.21.15.114:0/4280458876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b90119400 msgr2=0x7f5b901a9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:52.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.499+0000 7f5b95738640 1 --2- 172.21.15.114:0/4280458876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b90119400 0x7f5b901a9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:52.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.499+0000 7f5b95738640 1 -- 172.21.15.114:0/4280458876 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b90107890 msgr2=0x7f5b901a5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:52.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.499+0000 7f5b95738640 1 --2- 172.21.15.114:0/4280458876 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b90107890 0x7f5b901a5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:52.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.499+0000 7f5b95738640 1 -- 172.21.15.114:0/4280458876 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b88009900 con 0x7f5b901074b0 2024-09-13T18:25:52.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.500+0000 7f5b95738640 1 --2- 172.21.15.114:0/4280458876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b901074b0 0x7f5b901a4ed0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5b88005e30 tx=0x7f5b88031700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:52.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.500+0000 7f5b867fc640 1 -- 172.21.15.114:0/4280458876 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b88003b70 con 0x7f5b901074b0 2024-09-13T18:25:52.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.500+0000 7f5b867fc640 1 -- 172.21.15.114:0/4280458876 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5b88040070 con 0x7f5b901074b0 2024-09-13T18:25:52.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.500+0000 7f5b867fc640 1 -- 172.21.15.114:0/4280458876 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b880042a0 con 0x7f5b901074b0 2024-09-13T18:25:52.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.501+0000 7f5b9673a640 1 -- 172.21.15.114:0/4280458876 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b901aa3b0 con 0x7f5b901074b0 2024-09-13T18:25:52.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.501+0000 7f5b9673a640 1 -- 172.21.15.114:0/4280458876 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b901b1c90 con 0x7f5b901074b0 2024-09-13T18:25:52.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.504+0000 7f5b9a6c4740 1 Processor -- start 2024-09-13T18:25:52.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.504+0000 7f5b9a6c4740 1 -- start start 2024-09-13T18:25:52.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.505+0000 7f5b9a6c4740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dc5c2e3fe0 0x55dc5c2e43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:52.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.505+0000 7f5b9a6c4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc5c2e4900 0x55dc5c2eec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:52.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.505+0000 7f5b9a6c4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc5c2ef1a0 0x55dc5c2f1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:52.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.505+0000 7f5b9a6c4740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc5c24ef20 con 0x55dc5c2e3fe0 2024-09-13T18:25:52.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.505+0000 7f5b9a6c4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc5c24c6b0 con 0x55dc5c2e4900 2024-09-13T18:25:52.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.505+0000 7f5b9a6c4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc5c247fa0 con 0x55dc5c2ef1a0 2024-09-13T18:25:52.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.505+0000 7f5b95f39640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dc5c2e3fe0 0x55dc5c2e43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:52.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.505+0000 7f5b95f39640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dc5c2e3fe0 0x55dc5c2e43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43734/0 (socket says 172.21.15.114:43734) 2024-09-13T18:25:52.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.505+0000 7f5b95f39640 1 -- 172.21.15.114:0/3487768761 learned_addr learned my addr 172.21.15.114:0/3487768761 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:52.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.506+0000 7f5b95f39640 1 -- 172.21.15.114:0/3487768761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc5c2ef1a0 msgr2=0x55dc5c2f1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:52.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.506+0000 7f5b95f39640 1 --2- 172.21.15.114:0/3487768761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc5c2ef1a0 0x55dc5c2f1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:52.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.506+0000 7f5b95f39640 1 -- 172.21.15.114:0/3487768761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc5c2e4900 msgr2=0x55dc5c2eec60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:25:52.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.506+0000 7f5b95f39640 1 --2- 172.21.15.114:0/3487768761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc5c2e4900 0x55dc5c2eec60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:52.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.506+0000 7f5b95f39640 1 -- 172.21.15.114:0/3487768761 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc5c202840 con 0x55dc5c2e3fe0 2024-09-13T18:25:52.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.507+0000 7f5b95f39640 1 --2- 172.21.15.114:0/3487768761 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dc5c2e3fe0 0x55dc5c2e43c0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f5b80009c20 tx=0x7f5b80011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=994ee3682f2e9e82 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:52.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.507+0000 7f5b867fc640 1 -- 172.21.15.114:0/4280458876 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f5b88042030 con 0x7f5b901074b0 2024-09-13T18:25:52.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.508+0000 7f5b7b7fe640 1 -- 172.21.15.114:0/3487768761 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b80019630 con 0x55dc5c2e3fe0 2024-09-13T18:25:52.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.509+0000 7f5b7b7fe640 1 -- 172.21.15.114:0/3487768761 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5b80009e90 con 0x55dc5c2e3fe0 2024-09-13T18:25:52.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.509+0000 7f5b7b7fe640 1 -- 172.21.15.114:0/3487768761 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8000e940 con 0x55dc5c2e3fe0 2024-09-13T18:25:52.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.510+0000 7f5b867fc640 1 --2- 172.21.15.114:0/4280458876 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f5b7c0a13d0 0x7f5b7c0a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:52.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.510+0000 7f5b867fc640 1 -- 172.21.15.114:0/4280458876 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f5b880ee340 con 0x7f5b901074b0 2024-09-13T18:25:52.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.510+0000 7f5b9a6c4740 1 -- 172.21.15.114:0/3487768761 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dc5c2e3fe0 msgr2=0x55dc5c2e43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:52.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.510+0000 7f5b9a6c4740 1 --2- 172.21.15.114:0/3487768761 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dc5c2e3fe0 0x55dc5c2e43c0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f5b80009c20 tx=0x7f5b80011810 comp rx=0 tx=0).stop 2024-09-13T18:25:52.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.511+0000 7f5b95738640 1 --2- 172.21.15.114:0/4280458876 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f5b7c0a13d0 0x7f5b7c0a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:52.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.511+0000 7f5b9a6c4740 1 -- 172.21.15.114:0/3487768761 shutdown_connections 2024-09-13T18:25:52.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.511+0000 7f5b9a6c4740 1 --2- 172.21.15.114:0/3487768761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc5c2ef1a0 0x55dc5c2f1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:52.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.511+0000 7f5b9a6c4740 1 --2- 172.21.15.114:0/3487768761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc5c2e4900 0x55dc5c2eec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:52.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.511+0000 7f5b9a6c4740 1 --2- 172.21.15.114:0/3487768761 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dc5c2e3fe0 0x55dc5c2e43c0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:52.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.511+0000 7f5b9a6c4740 1 -- 172.21.15.114:0/3487768761 >> 172.21.15.114:0/3487768761 conn(0x55dc5c2e0570 msgr2=0x55dc5c2e0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:52.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.511+0000 7f5b9a6c4740 1 -- 172.21.15.114:0/3487768761 shutdown_connections 2024-09-13T18:25:52.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.512+0000 7f5b95738640 1 --2- 172.21.15.114:0/4280458876 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f5b7c0a13d0 0x7f5b7c0a3890 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x55dc5c2e0070 tx=0x7f5b8803e040 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:52.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.512+0000 7f5b9a6c4740 1 -- 172.21.15.114:0/3487768761 wait complete. 2024-09-13T18:25:52.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.512+0000 7f5b9a6c4740 1 -- start start 2024-09-13T18:25:52.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.512+0000 7f5b9a6c4740 1 Processor -- start 2024-09-13T18:25:52.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.513+0000 7f5b9a6c4740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dc5c2e3fe0 0x55dc5c2eeb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:52.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.513+0000 7f5b9a6c4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc5c2e4900 0x55dc5c2ebe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:52.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.513+0000 7f5b9a6c4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc5c2ef1a0 0x55dc5c2ec470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:52.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.513+0000 7f5b9a6c4740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc5c1e92a0 con 0x55dc5c2e3fe0 2024-09-13T18:25:52.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.513+0000 7f5b9a6c4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc5bf20430 con 0x55dc5c2e4900 2024-09-13T18:25:52.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.513+0000 7f5b9a6c4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc5c1cb680 con 0x55dc5c2ef1a0 2024-09-13T18:25:52.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.513+0000 7f5b9a6c4740 10 client.0 _ll_register_callbacks cb 0x55dc5c241610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:25:52.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.514+0000 7f5b95f39640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dc5c2e3fe0 0x55dc5c2eeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:52.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.514+0000 7f5b94f37640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc5c2e4900 0x55dc5c2ebe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:52.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.514+0000 7f5b95f39640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dc5c2e3fe0 0x55dc5c2eeb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43736/0 (socket says 172.21.15.114:43736) 2024-09-13T18:25:52.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.514+0000 7f5b95f39640 1 -- 172.21.15.114:0/804781421 learned_addr learned my addr 172.21.15.114:0/804781421 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:52.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.514+0000 7f5b95f39640 1 --2- 172.21.15.114:0/804781421 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc5c2ef1a0 0x55dc5c2ec470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:52.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.514+0000 7f5b94f37640 1 -- 172.21.15.114:0/804781421 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc5c2ef1a0 msgr2=0x55dc5c2ec470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:52.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.514+0000 7f5b94f37640 1 --2- 172.21.15.114:0/804781421 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc5c2ef1a0 0x55dc5c2ec470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:52.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.514+0000 7f5b94f37640 1 -- 172.21.15.114:0/804781421 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dc5c2e3fe0 msgr2=0x55dc5c2eeb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:52.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.514+0000 7f5b94f37640 1 --2- 172.21.15.114:0/804781421 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dc5c2e3fe0 0x55dc5c2eeb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:52.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.514+0000 7f5b94f37640 1 -- 172.21.15.114:0/804781421 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b800098d0 con 0x55dc5c2e4900 2024-09-13T18:25:52.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.514+0000 7f5b94f37640 1 --2- 172.21.15.114:0/804781421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc5c2e4900 0x55dc5c2ebe80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x55dc5c2e1660 tx=0x7f5b8c0107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5f472d9af7188d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:52.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.515+0000 7f5b78ff9640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:25:52.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.515+0000 7f5b78ff9640 1 -- 172.21.15.114:0/804781421 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c004230 con 0x55dc5c2e4900 2024-09-13T18:25:52.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.515+0000 7f5b78ff9640 1 -- 172.21.15.114:0/804781421 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f5b8c017070 con 0x55dc5c2e4900 2024-09-13T18:25:52.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.517+0000 7f5b9a6c4740 1 -- 172.21.15.114:0/804781421 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dc5bf3b4e0 con 0x55dc5c2e4900 2024-09-13T18:25:52.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.518+0000 7f5b78ff9640 1 -- 172.21.15.114:0/804781421 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b8c004390 con 0x55dc5c2e4900 2024-09-13T18:25:52.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.518+0000 7f5b9a6c4740 10 client.26105 fetch_fsmap learned FSMap version 32 2024-09-13T18:25:52.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.518+0000 7f5b9a6c4740 1 -- 172.21.15.114:0/804781421 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55dc5c246620 con 0x55dc5c2e4900 2024-09-13T18:25:52.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.519+0000 7f5b78ff9640 1 -- 172.21.15.114:0/804781421 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55dc5bf3b4e0 con 0x55dc5c2e4900 2024-09-13T18:25:52.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.519+0000 7f5b78ff9640 1 -- 172.21.15.114:0/804781421 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f5b8c011480 con 0x55dc5c2e4900 2024-09-13T18:25:52.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.519+0000 7f5b9a6c4740 10 client.26105 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:25:52.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.519+0000 7f5b9a6c4740 10 client.26105 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:25:52.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.519+0000 7f5b9a6c4740 20 client.26105 populate_metadata read hostname 'smithi114' 2024-09-13T18:25:52.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.519+0000 7f5b9a6c4740 1 --2- 172.21.15.114:0/804781421 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55dc5c2fef80 0x55dc5c3013a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:52.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.519+0000 7f5b9a6c4740 4 client.26105 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:25:52.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.519+0000 7f5b9a6c4740 1 -- 172.21.15.114:0/804781421 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dc5bf4c7e0 con 0x55dc5c2fef80 2024-09-13T18:25:52.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.521+0000 7f5b95f39640 1 --2- 172.21.15.114:0/804781421 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55dc5c2fef80 0x55dc5c3013a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:52.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.523+0000 7f5b95f39640 1 --2- 172.21.15.114:0/804781421 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55dc5c2fef80 0x55dc5c3013a0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bab67ee75bb3f221 server_cookie=9f7b15ae15aeff13 in_seq=0 out_seq=0 2024-09-13T18:25:52.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.523+0000 7f5b78ff9640 10 client.26105 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:25:52.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.524+0000 7f5b78ff9640 1 -- 172.21.15.114:0/804781421 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55dc5bf4c7e0 con 0x55dc5c2fef80 2024-09-13T18:25:52.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.524+0000 7f5b78ff9640 10 client.26105 handle_command_reply: tid=0 2024-09-13T18:25:52.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.524+0000 7f5b9a6c4740 1 client.26105 shutdown 2024-09-13T18:25:52.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.524+0000 7f5b9a6c4740 1 -- 172.21.15.114:0/804781421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc5c2e4900 msgr2=0x55dc5c2ebe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:52.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.524+0000 7f5b9a6c4740 1 --2- 172.21.15.114:0/804781421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc5c2e4900 0x55dc5c2ebe80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x55dc5c2e1660 tx=0x7f5b8c0107a0 comp rx=0 tx=0).stop 2024-09-13T18:25:52.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.525+0000 7f5b9a6c4740 1 -- 172.21.15.114:0/804781421 shutdown_connections 2024-09-13T18:25:52.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.525+0000 7f5b9a6c4740 1 --2- 172.21.15.114:0/804781421 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55dc5c2fef80 0x55dc5c3013a0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:52.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.525+0000 7f5b9a6c4740 1 --2- 172.21.15.114:0/804781421 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc5c2ef1a0 0x55dc5c2ec470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:52.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.525+0000 7f5b9a6c4740 1 --2- 172.21.15.114:0/804781421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc5c2e4900 0x55dc5c2ebe80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:52.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.525+0000 7f5b9a6c4740 1 --2- 172.21.15.114:0/804781421 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dc5c2e3fe0 0x55dc5c2eeb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:52.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.525+0000 7f5b9a6c4740 1 -- 172.21.15.114:0/804781421 >> 172.21.15.114:0/804781421 conn(0x55dc5c2e0570 msgr2=0x55dc5c2df760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:52.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.525+0000 7f5b78ff9640 0 client.26105 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:25:52.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.525+0000 7f5b9a6c4740 1 -- 172.21.15.114:0/804781421 shutdown_connections 2024-09-13T18:25:52.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.525+0000 7f5b9a6c4740 1 -- 172.21.15.114:0/804781421 wait complete. 2024-09-13T18:25:52.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.525+0000 7f5b9a6c4740 20 client.26105 trim_cache size 0 max 16384 2024-09-13T18:25:52.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.559+0000 7f5b9a6c4740 1 Processor -- start 2024-09-13T18:25:52.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.560+0000 7f5b9a6c4740 1 -- start start 2024-09-13T18:25:52.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.560+0000 7f5b9a6c4740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dc5c2e3fe0 0x55dc5c2f14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:52.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.560+0000 7f5b9a6c4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc5c2e4900 0x55dc5c305d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:52.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.561+0000 7f5b9a6c4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc5c2ef1a0 0x55dc5c2e7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:52.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.561+0000 7f5b9a6c4740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc5bf3b4e0 con 0x55dc5c2e3fe0 2024-09-13T18:25:52.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.561+0000 7f5b9a6c4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc5c2fbcd0 con 0x55dc5c2e4900 2024-09-13T18:25:52.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.561+0000 7f5b9a6c4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc5c2e3d80 con 0x55dc5c2ef1a0 2024-09-13T18:25:52.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.561+0000 7f5b94f37640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc5c2e4900 0x55dc5c305d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:52.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.561+0000 7f5b94f37640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc5c2e4900 0x55dc5c305d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:48366/0 (socket says 172.21.15.114:48366) 2024-09-13T18:25:52.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.561+0000 7f5b94f37640 1 -- 172.21.15.114:0/2433646738 learned_addr learned my addr 172.21.15.114:0/2433646738 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:52.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.561+0000 7f5b94f37640 1 -- 172.21.15.114:0/2433646738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc5c2ef1a0 msgr2=0x55dc5c2e7120 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:25:52.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.561+0000 7f5b94f37640 1 --2- 172.21.15.114:0/2433646738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc5c2ef1a0 0x55dc5c2e7120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:52.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.561+0000 7f5b94f37640 1 -- 172.21.15.114:0/2433646738 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dc5c2e3fe0 msgr2=0x55dc5c2f14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:52.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.561+0000 7f5b94f37640 1 --2- 172.21.15.114:0/2433646738 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dc5c2e3fe0 0x55dc5c2f14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:52.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.561+0000 7f5b94f37640 1 -- 172.21.15.114:0/2433646738 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc5c1e8790 con 0x55dc5c2e4900 2024-09-13T18:25:52.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.562+0000 7f5b94f37640 1 --2- 172.21.15.114:0/2433646738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc5c2e4900 0x55dc5c305d00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x55dc5c2ef170 tx=0x7f5b8c011960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70af98f86e0b1d2a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:52.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.562+0000 7f5b7bfff640 1 -- 172.21.15.114:0/2433646738 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c011ab0 con 0x55dc5c2e4900 2024-09-13T18:25:52.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.562+0000 7f5b7bfff640 1 -- 172.21.15.114:0/2433646738 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5b8c015070 con 0x55dc5c2e4900 2024-09-13T18:25:52.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.562+0000 7f5b7bfff640 1 -- 172.21.15.114:0/2433646738 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c005290 con 0x55dc5c2e4900 2024-09-13T18:25:52.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.564+0000 7f5b9a6c4740 1 -- 172.21.15.114:0/2433646738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc5c2e4900 msgr2=0x55dc5c305d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:52.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.564+0000 7f5b9a6c4740 1 --2- 172.21.15.114:0/2433646738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc5c2e4900 0x55dc5c305d00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x55dc5c2ef170 tx=0x7f5b8c011960 comp rx=0 tx=0).stop 2024-09-13T18:25:52.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.564+0000 7f5b9a6c4740 1 -- 172.21.15.114:0/2433646738 shutdown_connections 2024-09-13T18:25:52.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.564+0000 7f5b9a6c4740 1 --2- 172.21.15.114:0/2433646738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc5c2ef1a0 0x55dc5c2e7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:52.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.564+0000 7f5b9a6c4740 1 --2- 172.21.15.114:0/2433646738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc5c2e4900 0x55dc5c305d00 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:52.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.564+0000 7f5b9a6c4740 1 --2- 172.21.15.114:0/2433646738 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dc5c2e3fe0 0x55dc5c2f14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:52.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.564+0000 7f5b9a6c4740 1 -- 172.21.15.114:0/2433646738 >> 172.21.15.114:0/2433646738 conn(0x55dc5c2e0570 msgr2=0x55dc5c2ee730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:52.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.566+0000 7f5b9a6c4740 1 -- 172.21.15.114:0/2433646738 shutdown_connections 2024-09-13T18:25:52.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.569+0000 7f5b9a6c4740 1 -- 172.21.15.114:0/2433646738 wait complete. 2024-09-13T18:25:52.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.570+0000 7f5b9a6c4740 1 -- start start 2024-09-13T18:25:52.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.570+0000 7f5b9a6c4740 1 Processor -- start 2024-09-13T18:25:52.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.570+0000 7f5b9a6c4740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dc5c2e3fe0 0x55dc5c2f29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:52.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.570+0000 7f5b9a6c4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc5c2e4900 0x55dc5c2f2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:52.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.570+0000 7f5b9a6c4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc5c2ef1a0 0x55dc5c2f3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:52.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.570+0000 7f5b9a6c4740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc5c355480 con 0x55dc5c2e3fe0 2024-09-13T18:25:52.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.570+0000 7f5b9a6c4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc5c3555c0 con 0x55dc5c2e4900 2024-09-13T18:25:52.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.570+0000 7f5b9a6c4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc5c355700 con 0x55dc5c2ef1a0 2024-09-13T18:25:52.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.571+0000 7f5b9a6c4740 10 client.0 _ll_register_callbacks cb 0x7f5b8c00b9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:25:52.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.572+0000 7f5b95f39640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc5c2ef1a0 0x55dc5c2f3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:52.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.572+0000 7f5b95f39640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dc5c2e3fe0 0x55dc5c2f29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:52.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.572+0000 7f5b95f39640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dc5c2e3fe0 0x55dc5c2f29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43756/0 (socket says 172.21.15.114:43756) 2024-09-13T18:25:52.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.572+0000 7f5b95f39640 1 -- 172.21.15.114:0/2391705833 learned_addr learned my addr 172.21.15.114:0/2391705833 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:52.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.573+0000 7f5b94f37640 1 --2- 172.21.15.114:0/2391705833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc5c2e4900 0x55dc5c2f2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:52.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.574+0000 7f5b95f39640 1 -- 172.21.15.114:0/2391705833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc5c2e4900 msgr2=0x55dc5c2f2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:52.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.574+0000 7f5b95f39640 1 --2- 172.21.15.114:0/2391705833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc5c2e4900 0x55dc5c2f2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:52.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.574+0000 7f5b95f39640 1 -- 172.21.15.114:0/2391705833 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dc5c2e3fe0 msgr2=0x55dc5c2f29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:52.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.574+0000 7f5b95f39640 1 --2- 172.21.15.114:0/2391705833 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dc5c2e3fe0 0x55dc5c2f29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:52.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.574+0000 7f5b95f39640 1 -- 172.21.15.114:0/2391705833 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dc5c246620 con 0x55dc5c2ef1a0 2024-09-13T18:25:52.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.578+0000 7f5b95f39640 1 --2- 172.21.15.114:0/2391705833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc5c2ef1a0 0x55dc5c2f3440 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5b8000e670 tx=0x7f5b80012f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74704ccc1fc456a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:52.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.578+0000 7f5b78ff9640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:25:52.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.578+0000 7f5b78ff9640 1 -- 172.21.15.114:0/2391705833 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8000eb90 con 0x55dc5c2ef1a0 2024-09-13T18:25:52.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.578+0000 7f5b78ff9640 1 -- 172.21.15.114:0/2391705833 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f5b8000ecf0 con 0x55dc5c2ef1a0 2024-09-13T18:25:52.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.578+0000 7f5b9a6c4740 1 -- 172.21.15.114:0/2391705833 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5b80009e90 con 0x55dc5c2ef1a0 2024-09-13T18:25:52.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.580+0000 7f5b78ff9640 1 -- 172.21.15.114:0/2391705833 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b8000eeb0 con 0x55dc5c2ef1a0 2024-09-13T18:25:52.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.581+0000 7f5b9a6c4740 10 client.26143 fetch_fsmap learned FSMap version 32 2024-09-13T18:25:52.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.581+0000 7f5b9a6c4740 1 -- 172.21.15.114:0/2391705833 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55dc5c20dea0 con 0x55dc5c2ef1a0 2024-09-13T18:25:52.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.581+0000 7f5b78ff9640 1 -- 172.21.15.114:0/2391705833 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5b80009e90 con 0x55dc5c2ef1a0 2024-09-13T18:25:52.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.582+0000 7f5b78ff9640 1 -- 172.21.15.114:0/2391705833 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f5b80023040 con 0x55dc5c2ef1a0 2024-09-13T18:25:52.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.582+0000 7f5b9a6c4740 10 client.26143 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:25:52.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.582+0000 7f5b9a6c4740 10 client.26143 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:25:52.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.582+0000 7f5b9a6c4740 20 client.26143 populate_metadata read hostname 'smithi114' 2024-09-13T18:25:52.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.582+0000 7f5b9a6c4740 1 --2- 172.21.15.114:0/2391705833 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55dc5c356e90 0x55dc5c3592b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:52.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.582+0000 7f5b9a6c4740 4 client.26143 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T18:25:52.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.582+0000 7f5b9a6c4740 1 -- 172.21.15.114:0/2391705833 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55dc5bf9fe30 con 0x55dc5c356e90 2024-09-13T18:25:52.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.582+0000 7f5b94f37640 1 --2- 172.21.15.114:0/2391705833 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55dc5c356e90 0x55dc5c3592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:52.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.586+0000 7f5b94f37640 1 --2- 172.21.15.114:0/2391705833 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55dc5c356e90 0x55dc5c3592b0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1a139c1592b79d4 server_cookie=9ca8257853c9ea91 in_seq=0 out_seq=0 2024-09-13T18:25:52.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.586+0000 7f5b78ff9640 10 client.26143 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:25:52.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.586+0000 7f5b78ff9640 1 -- 172.21.15.114:0/2391705833 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55dc5bf9fe30 con 0x55dc5c356e90 2024-09-13T18:25:52.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.586+0000 7f5b78ff9640 10 client.26143 handle_command_reply: tid=0 2024-09-13T18:25:52.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.586+0000 7f5b9a6c4740 1 client.26143 shutdown 2024-09-13T18:25:52.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.590+0000 7f5b9a6c4740 1 -- 172.21.15.114:0/2391705833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc5c2ef1a0 msgr2=0x55dc5c2f3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:52.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.590+0000 7f5b9a6c4740 1 --2- 172.21.15.114:0/2391705833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc5c2ef1a0 0x55dc5c2f3440 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5b8000e670 tx=0x7f5b80012f90 comp rx=0 tx=0).stop 2024-09-13T18:25:52.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.591+0000 7f5b9a6c4740 1 -- 172.21.15.114:0/2391705833 shutdown_connections 2024-09-13T18:25:52.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.591+0000 7f5b9a6c4740 1 --2- 172.21.15.114:0/2391705833 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55dc5c356e90 0x55dc5c3592b0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:52.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.591+0000 7f5b9a6c4740 1 --2- 172.21.15.114:0/2391705833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc5c2ef1a0 0x55dc5c2f3440 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:52.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.591+0000 7f5b9a6c4740 1 --2- 172.21.15.114:0/2391705833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc5c2e4900 0x55dc5c2f2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:52.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.591+0000 7f5b9a6c4740 1 --2- 172.21.15.114:0/2391705833 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dc5c2e3fe0 0x55dc5c2f29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:52.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.591+0000 7f5b9a6c4740 1 -- 172.21.15.114:0/2391705833 >> 172.21.15.114:0/2391705833 conn(0x55dc5c2e0570 msgr2=0x55dc5c305d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:52.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.591+0000 7f5b78ff9640 0 client.26143 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:25:52.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.591+0000 7f5b9a6c4740 1 -- 172.21.15.114:0/2391705833 shutdown_connections 2024-09-13T18:25:52.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.591+0000 7f5b9a6c4740 1 -- 172.21.15.114:0/2391705833 wait complete. 2024-09-13T18:25:52.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.591+0000 7f5b9a6c4740 20 client.26143 trim_cache size 0 max 16384 2024-09-13T18:25:52.594 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:25:52.594 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "no active scrubs running", 2024-09-13T18:25:52.594 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": {} 2024-09-13T18:25:52.594 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:25:52.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.592+0000 7f5b9673a640 1 -- 172.21.15.114:0/4280458876 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f5b7c0a13d0 msgr2=0x7f5b7c0a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:52.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.592+0000 7f5b9673a640 1 --2- 172.21.15.114:0/4280458876 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f5b7c0a13d0 0x7f5b7c0a3890 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x55dc5c2e0070 tx=0x7f5b8803e040 comp rx=0 tx=0).stop 2024-09-13T18:25:52.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.592+0000 7f5b9673a640 1 -- 172.21.15.114:0/4280458876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b901074b0 msgr2=0x7f5b901a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:52.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.592+0000 7f5b9673a640 1 --2- 172.21.15.114:0/4280458876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b901074b0 0x7f5b901a4ed0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5b88005e30 tx=0x7f5b88031700 comp rx=0 tx=0).stop 2024-09-13T18:25:52.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.593+0000 7f5b9673a640 1 -- 172.21.15.114:0/4280458876 shutdown_connections 2024-09-13T18:25:52.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.594+0000 7f5b9673a640 1 --2- 172.21.15.114:0/4280458876 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f5b7c0a13d0 0x7f5b7c0a3890 secure :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0x55dc5c2e0070 tx=0x7f5b8803e040 comp rx=0 tx=0).stop 2024-09-13T18:25:52.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.594+0000 7f5b9673a640 1 --2- 172.21.15.114:0/4280458876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b90119400 0x7f5b901a9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:52.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.594+0000 7f5b9673a640 1 --2- 172.21.15.114:0/4280458876 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b90107890 0x7f5b901a5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:52.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.594+0000 7f5b9673a640 1 --2- 172.21.15.114:0/4280458876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b901074b0 0x7f5b901a4ed0 secure :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5b88005e30 tx=0x7f5b88031700 comp rx=0 tx=0).stop 2024-09-13T18:25:52.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.594+0000 7f5b9673a640 1 -- 172.21.15.114:0/4280458876 >> 172.21.15.114:0/4280458876 conn(0x7f5b9006af40 msgr2=0x7f5b90119d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:52.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.594+0000 7f5b9673a640 1 -- 172.21.15.114:0/4280458876 shutdown_connections 2024-09-13T18:25:52.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.594+0000 7f5b9673a640 1 -- 172.21.15.114:0/4280458876 wait complete. 2024-09-13T18:25:52.616 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-13T18:25:52.616 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:25:52.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:52 smithi134 ceph-mon[34283]: pgmap v105: 129 pgs: 129 active+clean; 13 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 48 MiB/s wr, 776 op/s 2024-09-13T18:25:52.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:52 smithi204 ceph-mon[34777]: pgmap v105: 129 pgs: 129 active+clean; 13 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 48 MiB/s wr, 776 op/s 2024-09-13T18:25:52.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.803+0000 7f94e67f4640 1 -- 172.21.15.114:0/1462747751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94e0101fe0 msgr2=0x7f94e01023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:52.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.803+0000 7f94e67f4640 1 --2- 172.21.15.114:0/1462747751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94e0101fe0 0x7f94e01023c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f94d8009ca0 tx=0x7f94d802a8c0 comp rx=0 tx=0).stop 2024-09-13T18:25:52.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.804+0000 7f94e67f4640 1 -- 172.21.15.114:0/1462747751 shutdown_connections 2024-09-13T18:25:52.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.804+0000 7f94e67f4640 1 --2- 172.21.15.114:0/1462747751 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f94e0107130 0x7f94e010ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:52.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.804+0000 7f94e67f4640 1 --2- 172.21.15.114:0/1462747751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94e0102990 0x7f94e01069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:52.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.804+0000 7f94e67f4640 1 --2- 172.21.15.114:0/1462747751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94e0101fe0 0x7f94e01023c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:52.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.804+0000 7f94e67f4640 1 -- 172.21.15.114:0/1462747751 >> 172.21.15.114:0/1462747751 conn(0x7f94e00fda10 msgr2=0x7f94e00ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:52.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.804+0000 7f94e67f4640 1 -- 172.21.15.114:0/1462747751 shutdown_connections 2024-09-13T18:25:52.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.804+0000 7f94e67f4640 1 -- 172.21.15.114:0/1462747751 wait complete. 2024-09-13T18:25:52.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.805+0000 7f94e67f4640 1 Processor -- start 2024-09-13T18:25:52.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.805+0000 7f94e67f4640 1 -- start start 2024-09-13T18:25:52.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.806+0000 7f94e67f4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f94e0101fe0 0x7f94e010ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:52.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.806+0000 7f94e67f4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94e0107130 0x7f94e010bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:52.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.806+0000 7f94e67f4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94e010c070 0x7f94e010c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:52.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.806+0000 7f94e67f4640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e01a8c70 con 0x7f94e0101fe0 2024-09-13T18:25:52.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.806+0000 7f94e67f4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e0069ed0 con 0x7f94e010c070 2024-09-13T18:25:52.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.806+0000 7f94e67f4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e006a090 con 0x7f94e0107130 2024-09-13T18:25:52.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.806+0000 7f94e4d91640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94e010c070 0x7f94e010c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:52.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.806+0000 7f94e4d91640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94e010c070 0x7f94e010c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:48378/0 (socket says 172.21.15.114:48378) 2024-09-13T18:25:52.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.806+0000 7f94e4d91640 1 -- 172.21.15.114:0/2519437687 learned_addr learned my addr 172.21.15.114:0/2519437687 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:52.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.807+0000 7f94dffff640 1 --2- 172.21.15.114:0/2519437687 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f94e0101fe0 0x7f94e010ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:52.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.807+0000 7f94df7fe640 1 --2- 172.21.15.114:0/2519437687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94e0107130 0x7f94e010bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:52.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.807+0000 7f94e4d91640 1 -- 172.21.15.114:0/2519437687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94e0107130 msgr2=0x7f94e010bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:52.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.807+0000 7f94e4d91640 1 --2- 172.21.15.114:0/2519437687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94e0107130 0x7f94e010bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:52.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.807+0000 7f94e4d91640 1 -- 172.21.15.114:0/2519437687 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f94e0101fe0 msgr2=0x7f94e010ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:52.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.807+0000 7f94e4d91640 1 --2- 172.21.15.114:0/2519437687 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f94e0101fe0 0x7f94e010ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:52.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.807+0000 7f94e4d91640 1 -- 172.21.15.114:0/2519437687 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d8009900 con 0x7f94e010c070 2024-09-13T18:25:52.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.807+0000 7f94df7fe640 1 --2- 172.21.15.114:0/2519437687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94e0107130 0x7f94e010bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:25:52.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.808+0000 7f94e4d91640 1 --2- 172.21.15.114:0/2519437687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94e010c070 0x7f94e010c4f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f94d0009fd0 tx=0x7f94d000b8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:52.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.808+0000 7f94dffff640 1 --2- 172.21.15.114:0/2519437687 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f94e0101fe0 0x7f94e010ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:25:52.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.808+0000 7f94dd7fa640 1 -- 172.21.15.114:0/2519437687 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d0011830 con 0x7f94e010c070 2024-09-13T18:25:52.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.808+0000 7f94e67f4640 1 -- 172.21.15.114:0/2519437687 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94e006a370 con 0x7f94e010c070 2024-09-13T18:25:52.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.808+0000 7f94e67f4640 1 -- 172.21.15.114:0/2519437687 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94e006a880 con 0x7f94e010c070 2024-09-13T18:25:52.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.809+0000 7f94dd7fa640 1 -- 172.21.15.114:0/2519437687 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f94d0011e20 con 0x7f94e010c070 2024-09-13T18:25:52.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.809+0000 7f94dd7fa640 1 -- 172.21.15.114:0/2519437687 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d0010e90 con 0x7f94e010c070 2024-09-13T18:25:52.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.810+0000 7f94e67f4640 1 -- 172.21.15.114:0/2519437687 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94cc005250 con 0x7f94e010c070 2024-09-13T18:25:52.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.812+0000 7f94dd7fa640 1 -- 172.21.15.114:0/2519437687 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f94d0011990 con 0x7f94e010c070 2024-09-13T18:25:52.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.813+0000 7f94dd7fa640 1 --2- 172.21.15.114:0/2519437687 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f94c80a1400 0x7f94c80a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:52.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.813+0000 7f94dd7fa640 1 -- 172.21.15.114:0/2519437687 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f94d00cc540 con 0x7f94e010c070 2024-09-13T18:25:52.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.813+0000 7f94dffff640 1 --2- 172.21.15.114:0/2519437687 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f94c80a1400 0x7f94c80a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:52.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.814+0000 7f94dffff640 1 --2- 172.21.15.114:0/2519437687 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f94c80a1400 0x7f94c80a38c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f94d8006010 tx=0x7f94d8032fc0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:52.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:52.815+0000 7f94dd7fa640 1 -- 172.21.15.114:0/2519437687 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f94d0099200 con 0x7f94e010c070 2024-09-13T18:25:53.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.279+0000 7f94e67f4640 1 -- 172.21.15.114:0/2519437687 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f94cc0050c0 con 0x7f94e010c070 2024-09-13T18:25:53.284 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:25:53.284 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4162817497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4162817497},{"type":"v1","addr":"172.21.15.114:6835","nonce":4162817497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/2435428478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":2435428478},{"type":"v1","addr":"172.21.15.114:6837","nonce":2435428478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T18:20:01.895117+0000","modified":"2024-09-13T18:24:01.324695+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24482,"mds_2":24475,"mds_3":24458,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/2046516451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":2046516451},{"type":"v1","addr":"172.21.15.114:6839","nonce":2046516451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":10,"addr":"172.21.15.114:6841/1956923446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":1956923446},{"type":"v1","addr":"172.21.15.114:6841","nonce":1956923446}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24391":{"gid":24391,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/507301496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":507301496},{"type":"v1","addr":"172.21.15.204:6833","nonce":507301496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4028657219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4028657219},{"type":"v1","addr":"172.21.15.134:6833","nonce":4028657219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/57029119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":57029119},{"type":"v1","addr":"172.21.15.204:6835","nonce":57029119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3068880939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3068880939},{"type":"v1","addr":"172.21.15.204:6837","nonce":3068880939}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"e","rank":3,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.134:6835/1150003275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1150003275},{"type":"v1","addr":"172.21.15.134:6835","nonce":1150003275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/344883912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":344883912},{"type":"v1","addr":"172.21.15.204:6839","nonce":344883912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6837/891692210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":891692210},{"type":"v1","addr":"172.21.15.134:6837","nonce":891692210}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6839/3466309724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3466309724},{"type":"v1","addr":"172.21.15.134:6839","nonce":3466309724}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:25:53.284 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 32 2024-09-13T18:25:53.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.281+0000 7f94dd7fa640 1 -- 172.21.15.114:0/2519437687 <== mon.2 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+10937 (secure 0 0 0) 0x7f94d0098d90 con 0x7f94e010c070 2024-09-13T18:25:53.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.284+0000 7f94e67f4640 1 -- 172.21.15.114:0/2519437687 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f94c80a1400 msgr2=0x7f94c80a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:53.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.284+0000 7f94e67f4640 1 --2- 172.21.15.114:0/2519437687 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f94c80a1400 0x7f94c80a38c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f94d8006010 tx=0x7f94d8032fc0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.284+0000 7f94e67f4640 1 -- 172.21.15.114:0/2519437687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94e010c070 msgr2=0x7f94e010c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:53.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.284+0000 7f94e67f4640 1 --2- 172.21.15.114:0/2519437687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94e010c070 0x7f94e010c4f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f94d0009fd0 tx=0x7f94d000b8e0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.285+0000 7f94e67f4640 1 -- 172.21.15.114:0/2519437687 shutdown_connections 2024-09-13T18:25:53.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.285+0000 7f94e67f4640 1 --2- 172.21.15.114:0/2519437687 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f94c80a1400 0x7f94c80a38c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.285+0000 7f94e67f4640 1 --2- 172.21.15.114:0/2519437687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94e010c070 0x7f94e010c4f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.285+0000 7f94e67f4640 1 --2- 172.21.15.114:0/2519437687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94e0107130 0x7f94e010bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.285+0000 7f94e67f4640 1 --2- 172.21.15.114:0/2519437687 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f94e0101fe0 0x7f94e010ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.285+0000 7f94e67f4640 1 -- 172.21.15.114:0/2519437687 >> 172.21.15.114:0/2519437687 conn(0x7f94e00fda10 msgr2=0x7f94e00feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:53.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.285+0000 7f94e67f4640 1 -- 172.21.15.114:0/2519437687 shutdown_connections 2024-09-13T18:25:53.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.285+0000 7f94e67f4640 1 -- 172.21.15.114:0/2519437687 wait complete. 2024-09-13T18:25:53.307 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-09-13T18:25:53.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.489+0000 7fd2b2ec9640 1 -- 172.21.15.114:0/910237115 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2a40964f0 msgr2=0x7fd2a4004710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:53.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.489+0000 7fd2b2ec9640 1 --2- 172.21.15.114:0/910237115 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2a40964f0 0x7fd2a4004710 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac02c970 tx=0x7fd2ac094990 comp rx=0 tx=0).stop 2024-09-13T18:25:53.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.489+0000 7fd2b2ec9640 1 -- 172.21.15.114:0/910237115 shutdown_connections 2024-09-13T18:25:53.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.489+0000 7fd2b2ec9640 1 --2- 172.21.15.114:0/910237115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2a40a1f40 0x7fd2a40a4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.489+0000 7fd2b2ec9640 1 --2- 172.21.15.114:0/910237115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2a4004c50 0x7fd2a4005090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.489+0000 7fd2b2ec9640 1 --2- 172.21.15.114:0/910237115 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2a40964f0 0x7fd2a4004710 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.489+0000 7fd2b2ec9640 1 -- 172.21.15.114:0/910237115 >> 172.21.15.114:0/910237115 conn(0x7fd2a400a1b0 msgr2=0x7fd2a400a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:53.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.490+0000 7fd2b2ec9640 1 -- 172.21.15.114:0/910237115 shutdown_connections 2024-09-13T18:25:53.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.491+0000 7fd2b2ec9640 1 -- 172.21.15.114:0/910237115 wait complete. 2024-09-13T18:25:53.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.491+0000 7fd2b2ec9640 1 Processor -- start 2024-09-13T18:25:53.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.492+0000 7fd2b2ec9640 1 -- start start 2024-09-13T18:25:53.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.492+0000 7fd2b2ec9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2a4004c50 0x7fd2a40a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:53.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.492+0000 7fd2b2ec9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2a40964f0 0x7fd2a409e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:53.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.492+0000 7fd2b2ec9640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2a40a1f40 0x7fd2a409ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:53.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.492+0000 7fd2b2ec9640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a409f520 con 0x7fd2a40a1f40 2024-09-13T18:25:53.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.492+0000 7fd2b2ec9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a409f690 con 0x7fd2a4004c50 2024-09-13T18:25:53.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.492+0000 7fd2b2ec9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a413e4c0 con 0x7fd2a40964f0 2024-09-13T18:25:53.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.492+0000 7fd2b1466640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2a40a1f40 0x7fd2a409ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:53.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.492+0000 7fd2b1466640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2a40a1f40 0x7fd2a409ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36132/0 (socket says 172.21.15.114:36132) 2024-09-13T18:25:53.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.492+0000 7fd2b1466640 1 -- 172.21.15.114:0/90713733 learned_addr learned my addr 172.21.15.114:0/90713733 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:53.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.492+0000 7fd2b0c65640 1 --2- 172.21.15.114:0/90713733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2a4004c50 0x7fd2a40a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:53.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.492+0000 7fd2abfff640 1 --2- 172.21.15.114:0/90713733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2a40964f0 0x7fd2a409e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:53.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.493+0000 7fd2b1466640 1 -- 172.21.15.114:0/90713733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2a40964f0 msgr2=0x7fd2a409e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:53.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.493+0000 7fd2b1466640 1 --2- 172.21.15.114:0/90713733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2a40964f0 0x7fd2a409e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.493+0000 7fd2b1466640 1 -- 172.21.15.114:0/90713733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2a4004c50 msgr2=0x7fd2a40a1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:53.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.493+0000 7fd2b1466640 1 --2- 172.21.15.114:0/90713733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2a4004c50 0x7fd2a40a1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.493+0000 7fd2b1466640 1 -- 172.21.15.114:0/90713733 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2ac04eac0 con 0x7fd2a40a1f40 2024-09-13T18:25:53.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.493+0000 7fd2b1466640 1 --2- 172.21.15.114:0/90713733 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2a40a1f40 0x7fd2a409ede0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fd2a000d950 tx=0x7fd2a000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:53.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.494+0000 7fd2a9ffb640 1 -- 172.21.15.114:0/90713733 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a0009bd0 con 0x7fd2a40a1f40 2024-09-13T18:25:53.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.494+0000 7fd2a9ffb640 1 -- 172.21.15.114:0/90713733 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd2a0009d30 con 0x7fd2a40a1f40 2024-09-13T18:25:53.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.494+0000 7fd2a9ffb640 1 -- 172.21.15.114:0/90713733 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a000f620 con 0x7fd2a40a1f40 2024-09-13T18:25:53.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.494+0000 7fd2b2ec9640 1 -- 172.21.15.114:0/90713733 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2a413e750 con 0x7fd2a40a1f40 2024-09-13T18:25:53.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.494+0000 7fd2b2ec9640 1 -- 172.21.15.114:0/90713733 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2a413ec10 con 0x7fd2a40a1f40 2024-09-13T18:25:53.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.498+0000 7fd2b5be9740 1 Processor -- start 2024-09-13T18:25:53.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.499+0000 7fd2a9ffb640 1 -- 172.21.15.114:0/90713733 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fd2a0010460 con 0x7fd2a40a1f40 2024-09-13T18:25:53.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.500+0000 7fd2a9ffb640 1 --2- 172.21.15.114:0/90713733 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fd29c0a1390 0x7fd29c0a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:53.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.500+0000 7fd2a9ffb640 1 -- 172.21.15.114:0/90713733 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fd2a00cccc0 con 0x7fd2a40a1f40 2024-09-13T18:25:53.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.500+0000 7fd2b5be9740 1 -- start start 2024-09-13T18:25:53.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.501+0000 7fd2b5be9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a4fac2970 0x562a4fac2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:53.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.501+0000 7fd2b5be9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562a4fac3290 0x562a4facd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:53.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.501+0000 7fd2b5be9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a4facdb30 0x562a4facff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:53.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.501+0000 7fd2b5be9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a4fa2ccc0 con 0x562a4fac3290 2024-09-13T18:25:53.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.501+0000 7fd2b5be9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a4fa2a450 con 0x562a4facdb30 2024-09-13T18:25:53.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.501+0000 7fd2b5be9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a4fa25d40 con 0x562a4fac2970 2024-09-13T18:25:53.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.501+0000 7fd2abfff640 1 --2- 172.21.15.114:0/90713733 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fd29c0a1390 0x7fd29c0a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:53.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.501+0000 7fd2b1466640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a4facdb30 0x562a4facff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:53.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.501+0000 7fd2b1466640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a4facdb30 0x562a4facff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:52570/0 (socket says 172.21.15.114:52570) 2024-09-13T18:25:53.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.501+0000 7fd2b1466640 1 -- 172.21.15.114:0/3801676931 learned_addr learned my addr 172.21.15.114:0/3801676931 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:53.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.502+0000 7fd2b0c65640 1 --2- 172.21.15.114:0/3801676931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a4fac2970 0x562a4fac2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:53.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.502+0000 7fd2b1466640 1 -- 172.21.15.114:0/3801676931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a4fac2970 msgr2=0x562a4fac2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:53.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.502+0000 7fd2b1466640 1 --2- 172.21.15.114:0/3801676931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a4fac2970 0x562a4fac2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.502+0000 7fd2b1466640 1 -- 172.21.15.114:0/3801676931 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562a4fac3290 msgr2=0x562a4facd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:53.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.502+0000 7fd2b1466640 1 --2- 172.21.15.114:0/3801676931 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562a4fac3290 0x562a4facd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.502+0000 7fd2b1466640 1 -- 172.21.15.114:0/3801676931 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a4f9ed620 con 0x562a4facdb30 2024-09-13T18:25:53.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.502+0000 7fd2b1466640 1 --2- 172.21.15.114:0/3801676931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a4facdb30 0x562a4facff20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0095f20 tx=0x7fd2a0005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7e18993909b08f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:53.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.503+0000 7fd2abfff640 1 --2- 172.21.15.114:0/90713733 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fd29c0a1390 0x7fd29c0a3850 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x562a4fabfd40 tx=0x7fd29800b6e0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:53.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.503+0000 7fd296ffd640 1 -- 172.21.15.114:0/3801676931 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a009d850 con 0x562a4facdb30 2024-09-13T18:25:53.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.504+0000 7fd296ffd640 1 -- 172.21.15.114:0/3801676931 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd2a00a1ea0 con 0x562a4facdb30 2024-09-13T18:25:53.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.504+0000 7fd296ffd640 1 -- 172.21.15.114:0/3801676931 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a0099170 con 0x562a4facdb30 2024-09-13T18:25:53.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.505+0000 7fd2b5be9740 1 -- 172.21.15.114:0/3801676931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a4facdb30 msgr2=0x562a4facff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:53.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.505+0000 7fd2b5be9740 1 --2- 172.21.15.114:0/3801676931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a4facdb30 0x562a4facff20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0095f20 tx=0x7fd2a0005c90 comp rx=0 tx=0).stop 2024-09-13T18:25:53.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.505+0000 7fd2b5be9740 1 -- 172.21.15.114:0/3801676931 shutdown_connections 2024-09-13T18:25:53.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.505+0000 7fd2b5be9740 1 --2- 172.21.15.114:0/3801676931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a4facdb30 0x562a4facff20 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.505+0000 7fd2b5be9740 1 --2- 172.21.15.114:0/3801676931 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562a4fac3290 0x562a4facd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.505+0000 7fd2b5be9740 1 --2- 172.21.15.114:0/3801676931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a4fac2970 0x562a4fac2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.505+0000 7fd2b5be9740 1 -- 172.21.15.114:0/3801676931 >> 172.21.15.114:0/3801676931 conn(0x562a4fabeee0 msgr2=0x562a4fabf2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:53.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.505+0000 7fd2b5be9740 1 -- 172.21.15.114:0/3801676931 shutdown_connections 2024-09-13T18:25:53.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.505+0000 7fd2b5be9740 1 -- 172.21.15.114:0/3801676931 wait complete. 2024-09-13T18:25:53.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.506+0000 7fd2b5be9740 1 -- start start 2024-09-13T18:25:53.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.506+0000 7fd2b5be9740 1 Processor -- start 2024-09-13T18:25:53.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.507+0000 7fd2b5be9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a4fac2970 0x562a4facd100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:53.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.507+0000 7fd2b5be9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562a4fac3290 0x562a4faca420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:53.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.507+0000 7fd2b5be9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a4facdb30 0x562a4faca960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:53.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.507+0000 7fd2b5be9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a4f9e4180 con 0x562a4fac3290 2024-09-13T18:25:53.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.507+0000 7fd2b5be9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a4f6fe430 con 0x562a4facdb30 2024-09-13T18:25:53.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.507+0000 7fd2b5be9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a4f9a95e0 con 0x562a4fac2970 2024-09-13T18:25:53.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.507+0000 7fd2b0c65640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a4fac2970 0x562a4facd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:53.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.507+0000 7fd2abfff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562a4fac3290 0x562a4faca420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:53.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.507+0000 7fd2b0c65640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a4fac2970 0x562a4facd100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:37750/0 (socket says 172.21.15.114:37750) 2024-09-13T18:25:53.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.507+0000 7fd2b0c65640 1 -- 172.21.15.114:0/38701016 learned_addr learned my addr 172.21.15.114:0/38701016 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:53.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.507+0000 7fd2b1466640 1 --2- 172.21.15.114:0/38701016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a4facdb30 0x562a4faca960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:53.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.508+0000 7fd2b0c65640 1 -- 172.21.15.114:0/38701016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a4facdb30 msgr2=0x562a4faca960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:53.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.508+0000 7fd2b0c65640 1 --2- 172.21.15.114:0/38701016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a4facdb30 0x562a4faca960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.508+0000 7fd2b0c65640 1 -- 172.21.15.114:0/38701016 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562a4fac3290 msgr2=0x562a4faca420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:53.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.508+0000 7fd2b0c65640 1 --2- 172.21.15.114:0/38701016 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562a4fac3290 0x562a4faca420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.508+0000 7fd2b5be9740 10 client.0 _ll_register_callbacks cb 0x562a4fa15aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:25:53.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.508+0000 7fd2b0c65640 1 -- 172.21.15.114:0/38701016 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2a413ec10 con 0x562a4fac2970 2024-09-13T18:25:53.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.508+0000 7fd2abfff640 1 --2- 172.21.15.114:0/38701016 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562a4fac3290 0x562a4faca420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:25:53.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.508+0000 7fd2b5be9740 1 -- 172.21.15.114:0/38701016 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562a4fa4fb80 con 0x562a4fac2970 2024-09-13T18:25:53.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.508+0000 7fd2b0c65640 1 --2- 172.21.15.114:0/38701016 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a4fac2970 0x562a4facd100 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac065d80 tx=0x7fd2ac066b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e85112bca42d874 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:53.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.508+0000 7fd28bfff640 10 client.26155 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:25:53.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.509+0000 7fd28bfff640 1 -- 172.21.15.114:0/38701016 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac070070 con 0x562a4fac2970 2024-09-13T18:25:53.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.509+0000 7fd28bfff640 1 -- 172.21.15.114:0/38701016 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fd2ac0722d0 con 0x562a4fac2970 2024-09-13T18:25:53.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.509+0000 7fd28bfff640 1 -- 172.21.15.114:0/38701016 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2ac0724d0 con 0x562a4fac2970 2024-09-13T18:25:53.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.509+0000 7fd2b5be9740 10 client.26155 fetch_fsmap learned FSMap version 32 2024-09-13T18:25:53.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.509+0000 7fd2b5be9740 1 -- 172.21.15.114:0/38701016 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({fsmap=32}) v3 -- 0x562a4fa243c0 con 0x562a4fac2970 2024-09-13T18:25:53.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.509+0000 7fd28bfff640 1 -- 172.21.15.114:0/38701016 <== mon.1 v2:172.21.15.204:3300/0 4 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fd2ac0608f0 con 0x562a4fac2970 2024-09-13T18:25:53.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.510+0000 7fd2b5be9740 10 client.26155 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:25:53.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.510+0000 7fd2b5be9740 10 client.26155 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-13T18:25:53.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.510+0000 7fd2b5be9740 20 client.26155 populate_metadata read hostname 'smithi114' 2024-09-13T18:25:53.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.510+0000 7fd2b5be9740 1 --2- 172.21.15.114:0/38701016 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x562a4faee360 0x562a4faf0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:53.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.510+0000 7fd2b5be9740 4 client.26155 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:25:53.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.510+0000 7fd2b5be9740 1 -- 172.21.15.114:0/38701016 --> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562a4fa181f0 con 0x562a4faee360 2024-09-13T18:25:53.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.510+0000 7fd2abfff640 1 --2- 172.21.15.114:0/38701016 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x562a4faee360 0x562a4faf0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:53.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.511+0000 7fd2abfff640 1 --2- 172.21.15.114:0/38701016 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x562a4faee360 0x562a4faf0820 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a129cfa77b5b6f6c server_cookie=f51abe47891d64b8 in_seq=0 out_seq=0 2024-09-13T18:25:53.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.511+0000 7fd28bfff640 10 client.26155 ms_handle_connect on v2:172.21.15.114:6840/1956923446 2024-09-13T18:25:53.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.512+0000 7fd28bfff640 1 -- 172.21.15.114:0/38701016 <== mds.4 v2:172.21.15.114:6840/1956923446 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x562a4fa181f0 con 0x562a4faee360 2024-09-13T18:25:53.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.512+0000 7fd28bfff640 10 client.26155 handle_command_reply: tid=0 2024-09-13T18:25:53.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.513+0000 7fd2b5be9740 1 client.26155 shutdown 2024-09-13T18:25:53.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.513+0000 7fd2b5be9740 1 -- 172.21.15.114:0/38701016 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a4fac2970 msgr2=0x562a4facd100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:53.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.513+0000 7fd2b5be9740 1 --2- 172.21.15.114:0/38701016 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a4fac2970 0x562a4facd100 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac065d80 tx=0x7fd2ac066b90 comp rx=0 tx=0).stop 2024-09-13T18:25:53.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.513+0000 7fd2b5be9740 1 -- 172.21.15.114:0/38701016 shutdown_connections 2024-09-13T18:25:53.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.513+0000 7fd2b5be9740 1 --2- 172.21.15.114:0/38701016 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x562a4faee360 0x562a4faf0820 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.513+0000 7fd2b5be9740 1 --2- 172.21.15.114:0/38701016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a4facdb30 0x562a4faca960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.513+0000 7fd2b5be9740 1 --2- 172.21.15.114:0/38701016 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562a4fac3290 0x562a4faca420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.513+0000 7fd28bfff640 0 client.26155 ms_handle_reset on v2:172.21.15.114:6840/1956923446 2024-09-13T18:25:53.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.513+0000 7fd2b5be9740 1 --2- 172.21.15.114:0/38701016 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a4fac2970 0x562a4facd100 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.513+0000 7fd2b5be9740 1 -- 172.21.15.114:0/38701016 >> 172.21.15.114:0/38701016 conn(0x562a4fabeee0 msgr2=0x562a4fabe230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:53.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.514+0000 7fd2b5be9740 1 -- 172.21.15.114:0/38701016 shutdown_connections 2024-09-13T18:25:53.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.514+0000 7fd2b5be9740 1 -- 172.21.15.114:0/38701016 wait complete. 2024-09-13T18:25:53.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.514+0000 7fd2b5be9740 20 client.26155 trim_cache size 0 max 16384 2024-09-13T18:25:53.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.537+0000 7fd2b5be9740 1 Processor -- start 2024-09-13T18:25:53.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.537+0000 7fd2b5be9740 1 -- start start 2024-09-13T18:25:53.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.537+0000 7fd2b5be9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a4fac2970 0x562a4fae5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:53.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.537+0000 7fd2b5be9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562a4fac3290 0x562a4fae5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:53.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.537+0000 7fd2b5be9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a4facdb30 0x562a4fac8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:53.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.537+0000 7fd2abfff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562a4fac3290 0x562a4fae5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:53.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.537+0000 7fd2abfff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562a4fac3290 0x562a4fae5dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36154/0 (socket says 172.21.15.114:36154) 2024-09-13T18:25:53.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.537+0000 7fd2abfff640 1 -- 172.21.15.114:0/183568251 learned_addr learned my addr 172.21.15.114:0/183568251 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:53.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.537+0000 7fd2b5be9740 1 -- 172.21.15.114:0/183568251 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a4facacd0 con 0x562a4fac3290 2024-09-13T18:25:53.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.537+0000 7fd2b5be9740 1 -- 172.21.15.114:0/183568251 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a4fae15f0 con 0x562a4fac2970 2024-09-13T18:25:53.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.537+0000 7fd2b5be9740 1 -- 172.21.15.114:0/183568251 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a4faeafa0 con 0x562a4facdb30 2024-09-13T18:25:53.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.537+0000 7fd2b0c65640 1 --2- 172.21.15.114:0/183568251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a4fac2970 0x562a4fae5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:53.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.537+0000 7fd2b1466640 1 --2- 172.21.15.114:0/183568251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a4facdb30 0x562a4fac8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:53.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.538+0000 7fd2b1466640 1 -- 172.21.15.114:0/183568251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a4fac2970 msgr2=0x562a4fae5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:53.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.538+0000 7fd2b1466640 1 --2- 172.21.15.114:0/183568251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a4fac2970 0x562a4fae5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.538+0000 7fd2b1466640 1 -- 172.21.15.114:0/183568251 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562a4fac3290 msgr2=0x562a4fae5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:53.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.538+0000 7fd2b1466640 1 --2- 172.21.15.114:0/183568251 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562a4fac3290 0x562a4fae5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.538+0000 7fd2b1466640 1 -- 172.21.15.114:0/183568251 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a4f9c6770 con 0x562a4facdb30 2024-09-13T18:25:53.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.538+0000 7fd2b0c65640 1 --2- 172.21.15.114:0/183568251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a4fac2970 0x562a4fae5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:25:53.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.538+0000 7fd2b1466640 1 --2- 172.21.15.114:0/183568251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a4facdb30 0x562a4fac8cd0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x562a4fac0260 tx=0x7fd2a000bec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b2eb04fdfb99d50 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:53.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.539+0000 7fd2977fe640 1 -- 172.21.15.114:0/183568251 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a0096130 con 0x562a4facdb30 2024-09-13T18:25:53.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.539+0000 7fd2977fe640 1 -- 172.21.15.114:0/183568251 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd2a009d850 con 0x562a4facdb30 2024-09-13T18:25:53.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.540+0000 7fd2b5be9740 1 -- 172.21.15.114:0/183568251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a4facdb30 msgr2=0x562a4fac8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:53.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.540+0000 7fd2b5be9740 1 --2- 172.21.15.114:0/183568251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a4facdb30 0x562a4fac8cd0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x562a4fac0260 tx=0x7fd2a000bec0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.540+0000 7fd2b5be9740 1 -- 172.21.15.114:0/183568251 shutdown_connections 2024-09-13T18:25:53.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.540+0000 7fd2b5be9740 1 --2- 172.21.15.114:0/183568251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a4facdb30 0x562a4fac8cd0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.540+0000 7fd2b5be9740 1 --2- 172.21.15.114:0/183568251 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562a4fac3290 0x562a4fae5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.540+0000 7fd2b5be9740 1 --2- 172.21.15.114:0/183568251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a4fac2970 0x562a4fae5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.540+0000 7fd2b5be9740 1 -- 172.21.15.114:0/183568251 >> 172.21.15.114:0/183568251 conn(0x562a4fabeee0 msgr2=0x562a4faccd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:53.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.540+0000 7fd2b5be9740 1 -- 172.21.15.114:0/183568251 shutdown_connections 2024-09-13T18:25:53.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.541+0000 7fd2b5be9740 1 -- 172.21.15.114:0/183568251 wait complete. 2024-09-13T18:25:53.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.541+0000 7fd2b5be9740 1 -- start start 2024-09-13T18:25:53.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.542+0000 7fd2b5be9740 1 Processor -- start 2024-09-13T18:25:53.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.542+0000 7fd2b5be9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a4fac2970 0x562a4fae3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:53.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.542+0000 7fd2b5be9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562a4fac3290 0x562a4fae4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:53.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.543+0000 7fd2b0c65640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a4fac2970 0x562a4fae3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:53.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.543+0000 7fd2b0c65640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a4fac2970 0x562a4fae3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:52594/0 (socket says 172.21.15.114:52594) 2024-09-13T18:25:53.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.543+0000 7fd2b0c65640 1 -- 172.21.15.114:0/3790846047 learned_addr learned my addr 172.21.15.114:0/3790846047 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:53.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.543+0000 7fd2b5be9740 1 --2- 172.21.15.114:0/3790846047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a4facdb30 0x562a4fb3d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:53.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.543+0000 7fd2b5be9740 1 -- 172.21.15.114:0/3790846047 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a4fb3de50 con 0x562a4fac3290 2024-09-13T18:25:53.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.543+0000 7fd2b1466640 1 --2- 172.21.15.114:0/3790846047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a4facdb30 0x562a4fb3d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:53.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.543+0000 7fd2b5be9740 1 -- 172.21.15.114:0/3790846047 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a4fb3dfc0 con 0x562a4fac2970 2024-09-13T18:25:53.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.543+0000 7fd2b5be9740 1 -- 172.21.15.114:0/3790846047 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a4fb3e180 con 0x562a4facdb30 2024-09-13T18:25:53.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.544+0000 7fd2b5be9740 10 client.0 _ll_register_callbacks cb 0x7fd2a0010b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:25:53.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.544+0000 7fd2b1466640 1 -- 172.21.15.114:0/3790846047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a4fac2970 msgr2=0x562a4fae3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:53.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.544+0000 7fd2b1466640 1 --2- 172.21.15.114:0/3790846047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a4fac2970 0x562a4fae3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.544+0000 7fd2b1466640 1 -- 172.21.15.114:0/3790846047 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562a4fac3290 msgr2=0x562a4fae4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:53.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.544+0000 7fd2b1466640 1 --2- 172.21.15.114:0/3790846047 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562a4fac3290 0x562a4fae4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.544+0000 7fd2b1466640 1 -- 172.21.15.114:0/3790846047 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562a4fa243c0 con 0x562a4facdb30 2024-09-13T18:25:53.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.544+0000 7fd2b1466640 1 --2- 172.21.15.114:0/3790846047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a4facdb30 0x562a4fb3d910 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0009610 tx=0x7fd2a00b24a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe06f561fd97eb08 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:53.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.544+0000 7fd2b5be9740 1 -- 172.21.15.114:0/3790846047 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd2a00a1ea0 con 0x562a4facdb30 2024-09-13T18:25:53.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.545+0000 7fd28bfff640 10 client.26161 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:25:53.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.545+0000 7fd28bfff640 1 -- 172.21.15.114:0/3790846047 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a0096130 con 0x562a4facdb30 2024-09-13T18:25:53.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.545+0000 7fd28bfff640 1 -- 172.21.15.114:0/3790846047 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fd2a00a1a10 con 0x562a4facdb30 2024-09-13T18:25:53.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.545+0000 7fd28bfff640 1 -- 172.21.15.114:0/3790846047 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2a00a1bd0 con 0x562a4facdb30 2024-09-13T18:25:53.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.545+0000 7fd2b5be9740 10 client.26161 fetch_fsmap learned FSMap version 32 2024-09-13T18:25:53.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.545+0000 7fd2b5be9740 1 -- 172.21.15.114:0/3790846047 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({fsmap=32}) v3 -- 0x562a4f9ed620 con 0x562a4facdb30 2024-09-13T18:25:53.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.546+0000 7fd28bfff640 1 -- 172.21.15.114:0/3790846047 <== mon.1 v2:172.21.15.204:3300/0 4 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fd2a0078b80 con 0x562a4facdb30 2024-09-13T18:25:53.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.546+0000 7fd2b5be9740 10 client.26161 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:25:53.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.546+0000 7fd2b5be9740 10 client.26161 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-13T18:25:53.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.546+0000 7fd2b5be9740 20 client.26161 populate_metadata read hostname 'smithi114' 2024-09-13T18:25:53.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.546+0000 7fd2b5be9740 1 --2- 172.21.15.114:0/3790846047 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x562a4fb453e0 0x562a4fb477d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:53.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.546+0000 7fd2b5be9740 4 client.26161 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:25:53.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.547+0000 7fd2b5be9740 1 -- 172.21.15.114:0/3790846047 --> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562a4f7194e0 con 0x562a4fb453e0 2024-09-13T18:25:53.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.547+0000 7fd2b0c65640 1 --2- 172.21.15.114:0/3790846047 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x562a4fb453e0 0x562a4fb477d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:53.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.547+0000 7fd2b0c65640 1 --2- 172.21.15.114:0/3790846047 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x562a4fb453e0 0x562a4fb477d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6973818b96af893c server_cookie=5dc5d134d53dff1b in_seq=0 out_seq=0 2024-09-13T18:25:53.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.547+0000 7fd28bfff640 10 client.26161 ms_handle_connect on v2:172.21.15.114:6840/1956923446 2024-09-13T18:25:53.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.548+0000 7fd28bfff640 1 -- 172.21.15.114:0/3790846047 <== mds.4 v2:172.21.15.114:6840/1956923446 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562a4f7194e0 con 0x562a4fb453e0 2024-09-13T18:25:53.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.548+0000 7fd28bfff640 10 client.26161 handle_command_reply: tid=0 2024-09-13T18:25:53.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.548+0000 7fd2b5be9740 1 client.26161 shutdown 2024-09-13T18:25:53.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.549+0000 7fd2b5be9740 1 -- 172.21.15.114:0/3790846047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a4facdb30 msgr2=0x562a4fb3d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:53.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.549+0000 7fd2b5be9740 1 --2- 172.21.15.114:0/3790846047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a4facdb30 0x562a4fb3d910 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0009610 tx=0x7fd2a00b24a0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.549+0000 7fd2b5be9740 1 -- 172.21.15.114:0/3790846047 shutdown_connections 2024-09-13T18:25:53.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.549+0000 7fd2b5be9740 1 --2- 172.21.15.114:0/3790846047 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x562a4fb453e0 0x562a4fb477d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.549+0000 7fd2b5be9740 1 --2- 172.21.15.114:0/3790846047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a4facdb30 0x562a4fb3d910 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.549+0000 7fd2b5be9740 1 --2- 172.21.15.114:0/3790846047 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562a4fac3290 0x562a4fae4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.549+0000 7fd2b5be9740 1 --2- 172.21.15.114:0/3790846047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a4fac2970 0x562a4fae3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.549+0000 7fd2b5be9740 1 -- 172.21.15.114:0/3790846047 >> 172.21.15.114:0/3790846047 conn(0x562a4fabeee0 msgr2=0x562a4fac8c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:53.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.549+0000 7fd28bfff640 0 client.26161 ms_handle_reset on v2:172.21.15.114:6840/1956923446 2024-09-13T18:25:53.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.552+0000 7fd2b5be9740 1 -- 172.21.15.114:0/3790846047 shutdown_connections 2024-09-13T18:25:53.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.552+0000 7fd2b5be9740 1 -- 172.21.15.114:0/3790846047 wait complete. 2024-09-13T18:25:53.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.553+0000 7fd2b5be9740 20 client.26161 trim_cache size 0 max 16384 2024-09-13T18:25:53.557 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:25:53.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.553+0000 7fd2b2ec9640 1 -- 172.21.15.114:0/90713733 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fd29c0a1390 msgr2=0x7fd29c0a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:53.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.554+0000 7fd2b2ec9640 1 --2- 172.21.15.114:0/90713733 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fd29c0a1390 0x7fd29c0a3850 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x562a4fabfd40 tx=0x7fd29800b6e0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.554+0000 7fd2b2ec9640 1 -- 172.21.15.114:0/90713733 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2a40a1f40 msgr2=0x7fd2a409ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:53.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.554+0000 7fd2b2ec9640 1 --2- 172.21.15.114:0/90713733 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2a40a1f40 0x7fd2a409ede0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fd2a000d950 tx=0x7fd2a000de10 comp rx=0 tx=0).stop 2024-09-13T18:25:53.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.554+0000 7fd2b2ec9640 1 -- 172.21.15.114:0/90713733 shutdown_connections 2024-09-13T18:25:53.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.554+0000 7fd2b2ec9640 1 --2- 172.21.15.114:0/90713733 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fd29c0a1390 0x7fd29c0a3850 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.554+0000 7fd2b2ec9640 1 --2- 172.21.15.114:0/90713733 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2a40a1f40 0x7fd2a409ede0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.554+0000 7fd2b2ec9640 1 --2- 172.21.15.114:0/90713733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2a40964f0 0x7fd2a409e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.554+0000 7fd2b2ec9640 1 --2- 172.21.15.114:0/90713733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2a4004c50 0x7fd2a40a1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.555+0000 7fd2b2ec9640 1 -- 172.21.15.114:0/90713733 >> 172.21.15.114:0/90713733 conn(0x7fd2a400a1b0 msgr2=0x7fd2a40a2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:53.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.556+0000 7fd2b2ec9640 1 -- 172.21.15.114:0/90713733 shutdown_connections 2024-09-13T18:25:53.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.556+0000 7fd2b2ec9640 1 -- 172.21.15.114:0/90713733 wait complete. 2024-09-13T18:25:53.577 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:53 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2519437687' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:25:53.578 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-09-13T18:25:53.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:53 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/2519437687' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:25:53.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:53 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/2519437687' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:25:53.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.737+0000 7f0bba819640 1 -- 172.21.15.114:0/636995445 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0bb40ff8d0 msgr2=0x7f0bb4103910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:53.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.737+0000 7f0bba819640 1 --2- 172.21.15.114:0/636995445 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0bb40ff8d0 0x7f0bb4103910 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c009930 tx=0x7f0b9c02a420 comp rx=0 tx=0).stop 2024-09-13T18:25:53.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.738+0000 7f0bba819640 1 -- 172.21.15.114:0/636995445 shutdown_connections 2024-09-13T18:25:53.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.738+0000 7f0bba819640 1 --2- 172.21.15.114:0/636995445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bb4104070 0x7f0bb410bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.738+0000 7f0bba819640 1 --2- 172.21.15.114:0/636995445 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0bb40ff8d0 0x7f0bb4103910 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.738+0000 7f0bba819640 1 --2- 172.21.15.114:0/636995445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0bb40fef20 0x7f0bb40ff300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.738+0000 7f0bba819640 1 -- 172.21.15.114:0/636995445 >> 172.21.15.114:0/636995445 conn(0x7f0bb40fa970 msgr2=0x7f0bb40fcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:53.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.738+0000 7f0bba819640 1 -- 172.21.15.114:0/636995445 shutdown_connections 2024-09-13T18:25:53.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.738+0000 7f0bba819640 1 -- 172.21.15.114:0/636995445 wait complete. 2024-09-13T18:25:53.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.739+0000 7f0bba819640 1 Processor -- start 2024-09-13T18:25:53.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.739+0000 7f0bba819640 1 -- start start 2024-09-13T18:25:53.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.739+0000 7f0bba819640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bb40fef20 0x7f0bb41009b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:53.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.740+0000 7f0bba819640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0bb4104070 0x7f0bb4100ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:53.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.740+0000 7f0bba819640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0bb4102670 0x7f0bb4101450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:53.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.740+0000 7f0bba819640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb41a9210 con 0x7f0bb4102670 2024-09-13T18:25:53.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.740+0000 7f0bba819640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb41a9380 con 0x7f0bb4104070 2024-09-13T18:25:53.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.740+0000 7f0bba819640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb41a9540 con 0x7f0bb40fef20 2024-09-13T18:25:53.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.740+0000 7f0bb8db6640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0bb4102670 0x7f0bb4101450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:53.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.740+0000 7f0babfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0bb4104070 0x7f0bb4100ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:53.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.740+0000 7f0bb8db6640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0bb4102670 0x7f0bb4101450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36182/0 (socket says 172.21.15.114:36182) 2024-09-13T18:25:53.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.740+0000 7f0bb8db6640 1 -- 172.21.15.114:0/2158602739 learned_addr learned my addr 172.21.15.114:0/2158602739 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:53.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.740+0000 7f0bb3fff640 1 --2- 172.21.15.114:0/2158602739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bb40fef20 0x7f0bb41009b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:53.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.741+0000 7f0bb3fff640 1 -- 172.21.15.114:0/2158602739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0bb4104070 msgr2=0x7f0bb4100ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:53.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.741+0000 7f0bb3fff640 1 --2- 172.21.15.114:0/2158602739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0bb4104070 0x7f0bb4100ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.741+0000 7f0bb3fff640 1 -- 172.21.15.114:0/2158602739 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0bb4102670 msgr2=0x7f0bb4101450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:53.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.741+0000 7f0bb3fff640 1 --2- 172.21.15.114:0/2158602739 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0bb4102670 0x7f0bb4101450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.741+0000 7f0bb3fff640 1 -- 172.21.15.114:0/2158602739 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b9c009560 con 0x7f0bb40fef20 2024-09-13T18:25:53.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.741+0000 7f0bb8db6640 1 --2- 172.21.15.114:0/2158602739 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0bb4102670 0x7f0bb4101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:25:53.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.741+0000 7f0bb3fff640 1 --2- 172.21.15.114:0/2158602739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bb40fef20 0x7f0bb41009b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0ba400b3f0 tx=0x7f0ba400b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:53.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.742+0000 7f0bb1ffb640 1 -- 172.21.15.114:0/2158602739 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba4011830 con 0x7f0bb40fef20 2024-09-13T18:25:53.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.742+0000 7f0bb1ffb640 1 -- 172.21.15.114:0/2158602739 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0ba4011e20 con 0x7f0bb40fef20 2024-09-13T18:25:53.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.742+0000 7f0bb1ffb640 1 -- 172.21.15.114:0/2158602739 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba4010df0 con 0x7f0bb40fef20 2024-09-13T18:25:53.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.742+0000 7f0bba819640 1 -- 172.21.15.114:0/2158602739 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bb41a9820 con 0x7f0bb40fef20 2024-09-13T18:25:53.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.742+0000 7f0bba819640 1 -- 172.21.15.114:0/2158602739 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bb41a9c50 con 0x7f0bb40fef20 2024-09-13T18:25:53.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.746+0000 7f0bbd61a740 1 Processor -- start 2024-09-13T18:25:53.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.746+0000 7f0bbd61a740 1 -- start start 2024-09-13T18:25:53.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.746+0000 7f0bbd61a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6596f9790 0x55e6596f9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:53.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.746+0000 7f0bbd61a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e6596fa0b0 0x55e659704410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:53.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.746+0000 7f0bbd61a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e659704950 0x55e659706d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:53.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.746+0000 7f0bbd61a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e659663cc0 con 0x55e6596fa0b0 2024-09-13T18:25:53.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.746+0000 7f0bbd61a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e659661450 con 0x55e659704950 2024-09-13T18:25:53.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.746+0000 7f0bbd61a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e65965cd40 con 0x55e6596f9790 2024-09-13T18:25:53.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.747+0000 7f0bb8db6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6596f9790 0x55e6596f9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:53.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.747+0000 7f0bb8db6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6596f9790 0x55e6596f9b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:37808/0 (socket says 172.21.15.114:37808) 2024-09-13T18:25:53.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.747+0000 7f0bb8db6640 1 -- 172.21.15.114:0/2219935583 learned_addr learned my addr 172.21.15.114:0/2219935583 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:53.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.747+0000 7f0babfff640 1 --2- 172.21.15.114:0/2219935583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e659704950 0x55e659706d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:53.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.747+0000 7f0bb8db6640 1 -- 172.21.15.114:0/2219935583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e659704950 msgr2=0x55e659706d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:53.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.747+0000 7f0bb8db6640 1 --2- 172.21.15.114:0/2219935583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e659704950 0x55e659706d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.747+0000 7f0bb8db6640 1 -- 172.21.15.114:0/2219935583 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e6596fa0b0 msgr2=0x55e659704410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:25:53.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.747+0000 7f0bb8db6640 1 --2- 172.21.15.114:0/2219935583 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e6596fa0b0 0x55e659704410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.747+0000 7f0bb8db6640 1 -- 172.21.15.114:0/2219935583 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e659624620 con 0x55e6596f9790 2024-09-13T18:25:53.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.747+0000 7f0bb1ffb640 1 -- 172.21.15.114:0/2158602739 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f0ba4011990 con 0x7f0bb40fef20 2024-09-13T18:25:53.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.748+0000 7f0bb8db6640 1 --2- 172.21.15.114:0/2219935583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6596f9790 0x55e6596f9b70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0bac0117b0 tx=0x7f0bac011c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b65533964e12aef server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:53.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.748+0000 7f0baa7fc640 1 -- 172.21.15.114:0/2219935583 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bac004d20 con 0x55e6596f9790 2024-09-13T18:25:53.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.748+0000 7f0baa7fc640 1 -- 172.21.15.114:0/2219935583 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0bac011ec0 con 0x55e6596f9790 2024-09-13T18:25:53.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.750+0000 7f0bbd61a740 1 -- 172.21.15.114:0/2219935583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6596f9790 msgr2=0x55e6596f9b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:53.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.750+0000 7f0bbd61a740 1 --2- 172.21.15.114:0/2219935583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6596f9790 0x55e6596f9b70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0bac0117b0 tx=0x7f0bac011c70 comp rx=0 tx=0).stop 2024-09-13T18:25:53.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.750+0000 7f0bb1ffb640 1 --2- 172.21.15.114:0/2158602739 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0ba00a13d0 0x7f0ba00a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:53.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.750+0000 7f0bbd61a740 1 -- 172.21.15.114:0/2219935583 shutdown_connections 2024-09-13T18:25:53.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.750+0000 7f0bb1ffb640 1 -- 172.21.15.114:0/2158602739 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f0ba40cc930 con 0x7f0bb40fef20 2024-09-13T18:25:53.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.750+0000 7f0bbd61a740 1 --2- 172.21.15.114:0/2219935583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e659704950 0x55e659706d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.750+0000 7f0bbd61a740 1 --2- 172.21.15.114:0/2219935583 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e6596fa0b0 0x55e659704410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.750+0000 7f0bbd61a740 1 --2- 172.21.15.114:0/2219935583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6596f9790 0x55e6596f9b70 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.750+0000 7f0bb3fff640 1 --2- 172.21.15.114:0/2158602739 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0ba00a13d0 0x7f0ba00a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:53.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.750+0000 7f0bbd61a740 1 -- 172.21.15.114:0/2219935583 >> 172.21.15.114:0/2219935583 conn(0x55e6596f5d00 msgr2=0x55e6596f60e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:53.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.751+0000 7f0bbd61a740 1 -- 172.21.15.114:0/2219935583 shutdown_connections 2024-09-13T18:25:53.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.751+0000 7f0bb3fff640 1 --2- 172.21.15.114:0/2158602739 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0ba00a13d0 0x7f0ba00a3890 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x55e6596f6b60 tx=0x7f0ba4011f90 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:53.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.751+0000 7f0bbd61a740 1 -- 172.21.15.114:0/2219935583 wait complete. 2024-09-13T18:25:53.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.751+0000 7f0bbd61a740 1 -- start start 2024-09-13T18:25:53.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.752+0000 7f0bbd61a740 1 Processor -- start 2024-09-13T18:25:53.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.752+0000 7f0bbd61a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6596f9790 0x55e659703ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:53.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.752+0000 7f0bbd61a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e6596fa0b0 0x55e6597043e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:53.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.754+0000 7f0bbd61a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e659704950 0x55e6597014e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:53.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.754+0000 7f0bbd61a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e65961b180 con 0x55e6596fa0b0 2024-09-13T18:25:53.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.754+0000 7f0bbd61a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e659335430 con 0x55e6596f9790 2024-09-13T18:25:53.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.754+0000 7f0bbd61a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6595e05e0 con 0x55e659704950 2024-09-13T18:25:53.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.754+0000 7f0bbd61a740 10 client.0 _ll_register_callbacks cb 0x55e65964caa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:25:53.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.754+0000 7f0babfff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e6596fa0b0 0x55e6597043e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:53.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.754+0000 7f0bb8db6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6596f9790 0x55e659703ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:53.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.754+0000 7f0bb8db6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6596f9790 0x55e659703ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:52644/0 (socket says 172.21.15.114:52644) 2024-09-13T18:25:53.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.754+0000 7f0bb8db6640 1 -- 172.21.15.114:0/4056340440 learned_addr learned my addr 172.21.15.114:0/4056340440 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:53.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.754+0000 7f0bb8db6640 1 --2- 172.21.15.114:0/4056340440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e659704950 0x55e6597014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:53.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.755+0000 7f0bb8db6640 1 -- 172.21.15.114:0/4056340440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e659704950 msgr2=0x55e6597014e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:53.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.755+0000 7f0bb8db6640 1 --2- 172.21.15.114:0/4056340440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e659704950 0x55e6597014e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.755+0000 7f0bb8db6640 1 -- 172.21.15.114:0/4056340440 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e6596fa0b0 msgr2=0x55e6597043e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:53.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.755+0000 7f0bb8db6640 1 --2- 172.21.15.114:0/4056340440 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e6596fa0b0 0x55e6597043e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.755+0000 7f0bb8db6640 1 -- 172.21.15.114:0/4056340440 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bac0113e0 con 0x55e6596f9790 2024-09-13T18:25:53.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.755+0000 7f0bb8db6640 1 --2- 172.21.15.114:0/4056340440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6596f9790 0x55e659703ea0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0bac0092f0 tx=0x7f0bac009900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12e853d45a0e4df9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:53.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.755+0000 7f0b937fe640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:25:53.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.755+0000 7f0b937fe640 1 -- 172.21.15.114:0/4056340440 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bac00fc40 con 0x55e6596f9790 2024-09-13T18:25:53.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.755+0000 7f0bbd61a740 1 -- 172.21.15.114:0/4056340440 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e65964f1f0 con 0x55e6596f9790 2024-09-13T18:25:53.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.756+0000 7f0b937fe640 1 -- 172.21.15.114:0/4056340440 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f0bac00fda0 con 0x55e6596f9790 2024-09-13T18:25:53.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.756+0000 7f0b937fe640 1 -- 172.21.15.114:0/4056340440 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0bac012450 con 0x55e6596f9790 2024-09-13T18:25:53.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.758+0000 7f0bbd61a740 10 client.26144 fetch_fsmap learned FSMap version 32 2024-09-13T18:25:53.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.758+0000 7f0bbd61a740 1 -- 172.21.15.114:0/4056340440 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55e65965b3c0 con 0x55e6596f9790 2024-09-13T18:25:53.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.761+0000 7f0b937fe640 1 -- 172.21.15.114:0/4056340440 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0bac012a40 con 0x55e6596f9790 2024-09-13T18:25:53.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.762+0000 7f0b937fe640 1 -- 172.21.15.114:0/4056340440 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f0bac00c070 con 0x55e6596f9790 2024-09-13T18:25:53.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.762+0000 7f0bbd61a740 10 client.26144 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:25:53.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.762+0000 7f0bbd61a740 10 client.26144 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-13T18:25:53.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.762+0000 7f0bbd61a740 20 client.26144 populate_metadata read hostname 'smithi114' 2024-09-13T18:25:53.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.762+0000 7f0bbd61a740 1 --2- 172.21.15.114:0/4056340440 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55e659714230 0x55e659716620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:53.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.762+0000 7f0bbd61a740 4 client.26144 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:25:53.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.762+0000 7f0bbd61a740 1 -- 172.21.15.114:0/4056340440 --> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e65964f1f0 con 0x55e659714230 2024-09-13T18:25:53.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.762+0000 7f0babfff640 1 --2- 172.21.15.114:0/4056340440 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55e659714230 0x55e659716620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:53.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.763+0000 7f0babfff640 1 --2- 172.21.15.114:0/4056340440 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55e659714230 0x55e659716620 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c0ffda46b6aa8256 server_cookie=1e1dbe296ce8950a in_seq=0 out_seq=0 2024-09-13T18:25:53.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.764+0000 7f0b937fe640 10 client.26144 ms_handle_connect on v2:172.21.15.134:6834/1150003275 2024-09-13T18:25:53.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.766+0000 7f0b937fe640 1 -- 172.21.15.114:0/4056340440 <== mds.3 v2:172.21.15.134:6834/1150003275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55e65964f1f0 con 0x55e659714230 2024-09-13T18:25:53.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.766+0000 7f0b937fe640 10 client.26144 handle_command_reply: tid=0 2024-09-13T18:25:53.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.766+0000 7f0bbd61a740 1 client.26144 shutdown 2024-09-13T18:25:53.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.766+0000 7f0bbd61a740 1 -- 172.21.15.114:0/4056340440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6596f9790 msgr2=0x55e659703ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:53.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.766+0000 7f0bbd61a740 1 --2- 172.21.15.114:0/4056340440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6596f9790 0x55e659703ea0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0bac0092f0 tx=0x7f0bac009900 comp rx=0 tx=0).stop 2024-09-13T18:25:53.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.767+0000 7f0bbd61a740 1 -- 172.21.15.114:0/4056340440 shutdown_connections 2024-09-13T18:25:53.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.767+0000 7f0bbd61a740 1 --2- 172.21.15.114:0/4056340440 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55e659714230 0x55e659716620 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.767+0000 7f0bbd61a740 1 --2- 172.21.15.114:0/4056340440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e659704950 0x55e6597014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.767+0000 7f0bbd61a740 1 --2- 172.21.15.114:0/4056340440 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e6596fa0b0 0x55e6597043e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.767+0000 7f0bbd61a740 1 --2- 172.21.15.114:0/4056340440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6596f9790 0x55e659703ea0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.767+0000 7f0b937fe640 0 client.26144 ms_handle_reset on v2:172.21.15.134:6834/1150003275 2024-09-13T18:25:53.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.767+0000 7f0bbd61a740 1 -- 172.21.15.114:0/4056340440 >> 172.21.15.114:0/4056340440 conn(0x55e6596f5d00 msgr2=0x55e6596fba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:53.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.767+0000 7f0bbd61a740 1 -- 172.21.15.114:0/4056340440 shutdown_connections 2024-09-13T18:25:53.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.767+0000 7f0bbd61a740 1 -- 172.21.15.114:0/4056340440 wait complete. 2024-09-13T18:25:53.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.767+0000 7f0bbd61a740 20 client.26144 trim_cache size 0 max 16384 2024-09-13T18:25:53.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.787+0000 7f0bbd61a740 1 Processor -- start 2024-09-13T18:25:53.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.787+0000 7f0bbd61a740 1 -- start start 2024-09-13T18:25:53.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.788+0000 7f0bbd61a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6596f9790 0x55e659700790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:53.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.788+0000 7f0bbd61a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6596fa0b0 0x55e659700cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:53.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.788+0000 7f0bb8db6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6596f9790 0x55e659700790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:53.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.788+0000 7f0bb8db6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6596f9790 0x55e659700790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:52658/0 (socket says 172.21.15.114:52658) 2024-09-13T18:25:53.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.788+0000 7f0bb8db6640 1 -- 172.21.15.114:0/3790014472 learned_addr learned my addr 172.21.15.114:0/3790014472 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:53.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.788+0000 7f0bbd61a740 1 --2- 172.21.15.114:0/3790014472 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e659704950 0x55e6596fc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:53.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.788+0000 7f0babfff640 1 --2- 172.21.15.114:0/3790014472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6596fa0b0 0x55e659700cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:53.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.789+0000 7f0bbd61a740 1 -- 172.21.15.114:0/3790014472 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e659702720 con 0x55e659704950 2024-09-13T18:25:53.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.789+0000 7f0bbd61a740 1 -- 172.21.15.114:0/3790014472 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6596fa9f0 con 0x55e6596f9790 2024-09-13T18:25:53.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.789+0000 7f0bbd61a740 1 -- 172.21.15.114:0/3790014472 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6596ff7a0 con 0x55e6596fa0b0 2024-09-13T18:25:53.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.789+0000 7f0bb8db6640 1 --2- 172.21.15.114:0/3790014472 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e659704950 0x55e6596fc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:53.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.789+0000 7f0bb8db6640 1 -- 172.21.15.114:0/3790014472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6596fa0b0 msgr2=0x55e659700cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:53.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.789+0000 7f0bb8db6640 1 --2- 172.21.15.114:0/3790014472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6596fa0b0 0x55e659700cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.789+0000 7f0bb8db6640 1 -- 172.21.15.114:0/3790014472 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e659704950 msgr2=0x55e6596fc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:53.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.789+0000 7f0bb8db6640 1 --2- 172.21.15.114:0/3790014472 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e659704950 0x55e6596fc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.789+0000 7f0bb8db6640 1 -- 172.21.15.114:0/3790014472 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6595fd770 con 0x55e6596f9790 2024-09-13T18:25:53.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.790+0000 7f0bb8db6640 1 --2- 172.21.15.114:0/3790014472 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6596f9790 0x55e659700790 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0bac005b40 tx=0x7f0bac011c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6406b5f0d90c57dd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:53.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.790+0000 7f0baaffd640 1 -- 172.21.15.114:0/3790014472 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bac029900 con 0x55e6596f9790 2024-09-13T18:25:53.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.790+0000 7f0baaffd640 1 -- 172.21.15.114:0/3790014472 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0bac029430 con 0x55e6596f9790 2024-09-13T18:25:53.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.790+0000 7f0baaffd640 1 -- 172.21.15.114:0/3790014472 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bac00fab0 con 0x55e6596f9790 2024-09-13T18:25:53.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.792+0000 7f0bbd61a740 1 -- 172.21.15.114:0/3790014472 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6596f9790 msgr2=0x55e659700790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:53.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.792+0000 7f0bbd61a740 1 --2- 172.21.15.114:0/3790014472 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6596f9790 0x55e659700790 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0bac005b40 tx=0x7f0bac011c70 comp rx=0 tx=0).stop 2024-09-13T18:25:53.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.792+0000 7f0bbd61a740 1 -- 172.21.15.114:0/3790014472 shutdown_connections 2024-09-13T18:25:53.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.792+0000 7f0bbd61a740 1 --2- 172.21.15.114:0/3790014472 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e659704950 0x55e6596fc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.792+0000 7f0bbd61a740 1 --2- 172.21.15.114:0/3790014472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6596fa0b0 0x55e659700cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.792+0000 7f0bbd61a740 1 --2- 172.21.15.114:0/3790014472 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6596f9790 0x55e659700790 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.792+0000 7f0bbd61a740 1 -- 172.21.15.114:0/3790014472 >> 172.21.15.114:0/3790014472 conn(0x55e6596f5d00 msgr2=0x55e65971af90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:53.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.792+0000 7f0bbd61a740 1 -- 172.21.15.114:0/3790014472 shutdown_connections 2024-09-13T18:25:53.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.792+0000 7f0bbd61a740 1 -- 172.21.15.114:0/3790014472 wait complete. 2024-09-13T18:25:53.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.793+0000 7f0bbd61a740 1 -- start start 2024-09-13T18:25:53.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.793+0000 7f0bbd61a740 1 Processor -- start 2024-09-13T18:25:53.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.794+0000 7f0bbd61a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e6596f9790 0x55e65970bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:53.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.794+0000 7f0bbd61a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6596fa0b0 0x55e659707e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:53.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.794+0000 7f0bbd61a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e659704950 0x55e659708410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:53.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.794+0000 7f0bb8db6640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e6596f9790 0x55e65970bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:53.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.794+0000 7f0bb8db6640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e6596f9790 0x55e65970bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36220/0 (socket says 172.21.15.114:36220) 2024-09-13T18:25:53.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.794+0000 7f0bb8db6640 1 -- 172.21.15.114:0/298804234 learned_addr learned my addr 172.21.15.114:0/298804234 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:53.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.794+0000 7f0bbd61a740 1 -- 172.21.15.114:0/298804234 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e659708b10 con 0x55e6596f9790 2024-09-13T18:25:53.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.794+0000 7f0bbd61a740 1 -- 172.21.15.114:0/298804234 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e659708c80 con 0x55e6596fa0b0 2024-09-13T18:25:53.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.794+0000 7f0bbd61a740 1 -- 172.21.15.114:0/298804234 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e65976a710 con 0x55e659704950 2024-09-13T18:25:53.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.794+0000 7f0bbd61a740 10 client.0 _ll_register_callbacks cb 0x7f0bac009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:25:53.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.794+0000 7f0babfff640 1 --2- 172.21.15.114:0/298804234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6596fa0b0 0x55e659707e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:53.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.795+0000 7f0babfff640 1 -- 172.21.15.114:0/298804234 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e659704950 msgr2=0x55e659708410 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:25:53.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.795+0000 7f0babfff640 1 --2- 172.21.15.114:0/298804234 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e659704950 0x55e659708410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.795+0000 7f0babfff640 1 -- 172.21.15.114:0/298804234 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e6596f9790 msgr2=0x55e65970bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:53.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.795+0000 7f0babfff640 1 --2- 172.21.15.114:0/298804234 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e6596f9790 0x55e65970bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.795+0000 7f0babfff640 1 -- 172.21.15.114:0/298804234 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e65965b3c0 con 0x55e6596fa0b0 2024-09-13T18:25:53.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.795+0000 7f0babfff640 1 --2- 172.21.15.114:0/298804234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6596fa0b0 0x55e659707e40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c02a940 tx=0x7f0b9c0041d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c187746dc1bf0727 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:53.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.795+0000 7f0b937fe640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:25:53.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.795+0000 7f0b937fe640 1 -- 172.21.15.114:0/298804234 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c043040 con 0x55e6596fa0b0 2024-09-13T18:25:53.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.795+0000 7f0b937fe640 1 -- 172.21.15.114:0/298804234 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f0b9c045b70 con 0x55e6596fa0b0 2024-09-13T18:25:53.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.795+0000 7f0bbd61a740 1 -- 172.21.15.114:0/298804234 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e659686b80 con 0x55e6596fa0b0 2024-09-13T18:25:53.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.796+0000 7f0b937fe640 1 -- 172.21.15.114:0/298804234 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b9c045d50 con 0x55e6596fa0b0 2024-09-13T18:25:53.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.796+0000 7f0bbd61a740 10 client.26150 fetch_fsmap learned FSMap version 32 2024-09-13T18:25:53.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.796+0000 7f0bbd61a740 1 -- 172.21.15.114:0/298804234 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55e659622e80 con 0x55e6596fa0b0 2024-09-13T18:25:53.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.796+0000 7f0b937fe640 1 -- 172.21.15.114:0/298804234 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e659686b80 con 0x55e6596fa0b0 2024-09-13T18:25:53.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.797+0000 7f0b937fe640 1 -- 172.21.15.114:0/298804234 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f0b9c045670 con 0x55e6596fa0b0 2024-09-13T18:25:53.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.797+0000 7f0bbd61a740 10 client.26150 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:25:53.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.797+0000 7f0bbd61a740 10 client.26150 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-13T18:25:53.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.797+0000 7f0bbd61a740 20 client.26150 populate_metadata read hostname 'smithi114' 2024-09-13T18:25:53.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.797+0000 7f0bbd61a740 1 --2- 172.21.15.114:0/298804234 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55e65976bf20 0x55e65976e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:53.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.797+0000 7f0bbd61a740 4 client.26150 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:25:53.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.797+0000 7f0bbd61a740 1 -- 172.21.15.114:0/298804234 --> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e6593504e0 con 0x55e65976bf20 2024-09-13T18:25:53.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.797+0000 7f0bb8db6640 1 --2- 172.21.15.114:0/298804234 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55e65976bf20 0x55e65976e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:53.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.798+0000 7f0bb8db6640 1 --2- 172.21.15.114:0/298804234 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55e65976bf20 0x55e65976e340 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=839101c13bd462e server_cookie=c9c1ef640f92be59 in_seq=0 out_seq=0 2024-09-13T18:25:53.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.798+0000 7f0b937fe640 10 client.26150 ms_handle_connect on v2:172.21.15.134:6834/1150003275 2024-09-13T18:25:53.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.798+0000 7f0b937fe640 1 -- 172.21.15.114:0/298804234 <== mds.3 v2:172.21.15.134:6834/1150003275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e6593504e0 con 0x55e65976bf20 2024-09-13T18:25:53.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.798+0000 7f0b937fe640 10 client.26150 handle_command_reply: tid=0 2024-09-13T18:25:53.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.798+0000 7f0bbd61a740 1 client.26150 shutdown 2024-09-13T18:25:53.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.798+0000 7f0bbd61a740 1 -- 172.21.15.114:0/298804234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6596fa0b0 msgr2=0x55e659707e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:53.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.798+0000 7f0bbd61a740 1 --2- 172.21.15.114:0/298804234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6596fa0b0 0x55e659707e40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c02a940 tx=0x7f0b9c0041d0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.799+0000 7f0bbd61a740 1 -- 172.21.15.114:0/298804234 shutdown_connections 2024-09-13T18:25:53.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.799+0000 7f0bbd61a740 1 --2- 172.21.15.114:0/298804234 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55e65976bf20 0x55e65976e340 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.799+0000 7f0bbd61a740 1 --2- 172.21.15.114:0/298804234 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e659704950 0x55e659708410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.799+0000 7f0bbd61a740 1 --2- 172.21.15.114:0/298804234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6596fa0b0 0x55e659707e40 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.799+0000 7f0bbd61a740 1 --2- 172.21.15.114:0/298804234 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e6596f9790 0x55e65970bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.799+0000 7f0bbd61a740 1 -- 172.21.15.114:0/298804234 >> 172.21.15.114:0/298804234 conn(0x55e6596f5d00 msgr2=0x55e659700790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:53.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.799+0000 7f0b937fe640 0 client.26150 ms_handle_reset on v2:172.21.15.134:6834/1150003275 2024-09-13T18:25:53.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.800+0000 7f0bbd61a740 1 -- 172.21.15.114:0/298804234 shutdown_connections 2024-09-13T18:25:53.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.800+0000 7f0bbd61a740 1 -- 172.21.15.114:0/298804234 wait complete. 2024-09-13T18:25:53.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.801+0000 7f0bbd61a740 20 client.26150 trim_cache size 0 max 16384 2024-09-13T18:25:53.804 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:25:53.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.804+0000 7f0bba819640 1 -- 172.21.15.114:0/2158602739 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0ba00a13d0 msgr2=0x7f0ba00a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:53.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.804+0000 7f0bba819640 1 --2- 172.21.15.114:0/2158602739 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0ba00a13d0 0x7f0ba00a3890 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x55e6596f6b60 tx=0x7f0ba4011f90 comp rx=0 tx=0).stop 2024-09-13T18:25:53.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.804+0000 7f0bba819640 1 -- 172.21.15.114:0/2158602739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bb40fef20 msgr2=0x7f0bb41009b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:53.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.804+0000 7f0bba819640 1 --2- 172.21.15.114:0/2158602739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bb40fef20 0x7f0bb41009b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0ba400b3f0 tx=0x7f0ba400b8b0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.804+0000 7f0bba819640 1 -- 172.21.15.114:0/2158602739 shutdown_connections 2024-09-13T18:25:53.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.804+0000 7f0bba819640 1 --2- 172.21.15.114:0/2158602739 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0ba00a13d0 0x7f0ba00a3890 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.804+0000 7f0bba819640 1 --2- 172.21.15.114:0/2158602739 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0bb4102670 0x7f0bb4101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.804+0000 7f0bba819640 1 --2- 172.21.15.114:0/2158602739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0bb4104070 0x7f0bb4100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.805+0000 7f0bba819640 1 --2- 172.21.15.114:0/2158602739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bb40fef20 0x7f0bb41009b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:53.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.805+0000 7f0bba819640 1 -- 172.21.15.114:0/2158602739 >> 172.21.15.114:0/2158602739 conn(0x7f0bb40fa970 msgr2=0x7f0bb40fbe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:53.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.806+0000 7f0bba819640 1 -- 172.21.15.114:0/2158602739 shutdown_connections 2024-09-13T18:25:53.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:53.806+0000 7f0bba819640 1 -- 172.21.15.114:0/2158602739 wait complete. 2024-09-13T18:25:53.832 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-09-13T18:25:54.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.042+0000 7f1d671a0640 1 -- 172.21.15.114:0/2833882856 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d60100870 msgr2=0x7f1d601048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:54.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.042+0000 7f1d671a0640 1 --2- 172.21.15.114:0/2833882856 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d60100870 0x7f1d601048b0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f1d50009890 tx=0x7f1d5002a380 comp rx=0 tx=0).stop 2024-09-13T18:25:54.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.043+0000 7f1d671a0640 1 -- 172.21.15.114:0/2833882856 shutdown_connections 2024-09-13T18:25:54.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.043+0000 7f1d671a0640 1 --2- 172.21.15.114:0/2833882856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d60105010 0x7f1d6010cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.044+0000 7f1d671a0640 1 --2- 172.21.15.114:0/2833882856 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d60100870 0x7f1d601048b0 secure :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f1d50009890 tx=0x7f1d5002a380 comp rx=0 tx=0).stop 2024-09-13T18:25:54.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.044+0000 7f1d671a0640 1 --2- 172.21.15.114:0/2833882856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d600ffec0 0x7f1d601002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.044+0000 7f1d671a0640 1 -- 172.21.15.114:0/2833882856 >> 172.21.15.114:0/2833882856 conn(0x7f1d600fb950 msgr2=0x7f1d600fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:54.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.045+0000 7f1d671a0640 1 -- 172.21.15.114:0/2833882856 shutdown_connections 2024-09-13T18:25:54.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.045+0000 7f1d671a0640 1 -- 172.21.15.114:0/2833882856 wait complete. 2024-09-13T18:25:54.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.046+0000 7f1d671a0640 1 Processor -- start 2024-09-13T18:25:54.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.046+0000 7f1d671a0640 1 -- start start 2024-09-13T18:25:54.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.046+0000 7f1d671a0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d600ffec0 0x7f1d601963f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:54.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.046+0000 7f1d671a0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d60105010 0x7f1d60196930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:54.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.046+0000 7f1d671a0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d6019ad20 0x7f1d6019b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:54.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.046+0000 7f1d671a0640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6019b710 con 0x7f1d60105010 2024-09-13T18:25:54.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.046+0000 7f1d671a0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6019b880 con 0x7f1d600ffec0 2024-09-13T18:25:54.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.046+0000 7f1d6619e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d600ffec0 0x7f1d601963f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:54.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.046+0000 7f1d6619e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d600ffec0 0x7f1d601963f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:52694/0 (socket says 172.21.15.114:52694) 2024-09-13T18:25:54.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.046+0000 7f1d671a0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6019b9f0 con 0x7f1d6019ad20 2024-09-13T18:25:54.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.046+0000 7f1d6619e640 1 -- 172.21.15.114:0/2585666917 learned_addr learned my addr 172.21.15.114:0/2585666917 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:54.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.046+0000 7f1d6599d640 1 --2- 172.21.15.114:0/2585666917 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d60105010 0x7f1d60196930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:54.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.047+0000 7f1d6699f640 1 --2- 172.21.15.114:0/2585666917 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d6019ad20 0x7f1d6019b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:54.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.047+0000 7f1d6699f640 1 -- 172.21.15.114:0/2585666917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d600ffec0 msgr2=0x7f1d601963f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:54.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.047+0000 7f1d6699f640 1 --2- 172.21.15.114:0/2585666917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d600ffec0 0x7f1d601963f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.047+0000 7f1d6699f640 1 -- 172.21.15.114:0/2585666917 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d60105010 msgr2=0x7f1d60196930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:54.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.047+0000 7f1d6699f640 1 --2- 172.21.15.114:0/2585666917 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d60105010 0x7f1d60196930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.047+0000 7f1d6699f640 1 -- 172.21.15.114:0/2585666917 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d50009490 con 0x7f1d6019ad20 2024-09-13T18:25:54.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.048+0000 7f1d6699f640 1 --2- 172.21.15.114:0/2585666917 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d6019ad20 0x7f1d6019b1d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1d5800ef00 tx=0x7f1d5800c740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:54.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.048+0000 7f1d577fe640 1 -- 172.21.15.114:0/2585666917 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d58004b20 con 0x7f1d6019ad20 2024-09-13T18:25:54.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.048+0000 7f1d577fe640 1 -- 172.21.15.114:0/2585666917 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1d58003e00 con 0x7f1d6019ad20 2024-09-13T18:25:54.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.048+0000 7f1d577fe640 1 -- 172.21.15.114:0/2585666917 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d58004700 con 0x7f1d6019ad20 2024-09-13T18:25:54.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.048+0000 7f1d671a0640 1 -- 172.21.15.114:0/2585666917 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d600726e0 con 0x7f1d6019ad20 2024-09-13T18:25:54.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.048+0000 7f1d671a0640 1 -- 172.21.15.114:0/2585666917 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d60072a50 con 0x7f1d6019ad20 2024-09-13T18:25:54.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.055+0000 7f1d6b205740 1 Processor -- start 2024-09-13T18:25:54.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.055+0000 7f1d577fe640 1 -- 172.21.15.114:0/2585666917 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f1d5800a040 con 0x7f1d6019ad20 2024-09-13T18:25:54.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.056+0000 7f1d6b205740 1 -- start start 2024-09-13T18:25:54.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.056+0000 7f1d6b205740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e8e55c180 0x563e8e55c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:54.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.056+0000 7f1d6b205740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563e8e55caa0 0x563e8e566e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:54.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.056+0000 7f1d6b205740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e8e567340 0x563e8e569730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:54.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.056+0000 7f1d6b205740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e8e4bdcc0 con 0x563e8e55caa0 2024-09-13T18:25:54.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.056+0000 7f1d6b205740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e8e4bb450 con 0x563e8e55c180 2024-09-13T18:25:54.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.056+0000 7f1d6b205740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e8e4b6d40 con 0x563e8e567340 2024-09-13T18:25:54.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.057+0000 7f1d6619e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563e8e55caa0 0x563e8e566e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:54.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.057+0000 7f1d6619e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563e8e55caa0 0x563e8e566e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36240/0 (socket says 172.21.15.114:36240) 2024-09-13T18:25:54.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.057+0000 7f1d6619e640 1 -- 172.21.15.114:0/991395792 learned_addr learned my addr 172.21.15.114:0/991395792 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:54.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.057+0000 7f1d577fe640 1 --2- 172.21.15.114:0/2585666917 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f1d4c0a1410 0x7f1d4c0a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:54.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.057+0000 7f1d577fe640 1 -- 172.21.15.114:0/2585666917 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f1d580ccdb0 con 0x7f1d6019ad20 2024-09-13T18:25:54.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.058+0000 7f1d6619e640 1 -- 172.21.15.114:0/991395792 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e8e567340 msgr2=0x563e8e569730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:25:54.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.058+0000 7f1d6619e640 1 --2- 172.21.15.114:0/991395792 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e8e567340 0x563e8e569730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.058+0000 7f1d6619e640 1 -- 172.21.15.114:0/991395792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e8e55c180 msgr2=0x563e8e55c560 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:25:54.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.058+0000 7f1d6619e640 1 --2- 172.21.15.114:0/991395792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e8e55c180 0x563e8e55c560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.058+0000 7f1d6619e640 1 -- 172.21.15.114:0/991395792 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e8e47e620 con 0x563e8e55caa0 2024-09-13T18:25:54.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.059+0000 7f1d6619e640 1 --2- 172.21.15.114:0/991395792 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563e8e55caa0 0x563e8e566e00 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c00bda0 tx=0x7f1d5c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1564612f8cbfbdd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:54.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.059+0000 7f1d54e05640 1 -- 172.21.15.114:0/991395792 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c01c680 con 0x563e8e55caa0 2024-09-13T18:25:54.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.059+0000 7f1d6699f640 1 --2- 172.21.15.114:0/2585666917 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f1d4c0a1410 0x7f1d4c0a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:54.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.059+0000 7f1d54e05640 1 -- 172.21.15.114:0/991395792 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1d5c015070 con 0x563e8e55caa0 2024-09-13T18:25:54.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.060+0000 7f1d6b205740 1 -- 172.21.15.114:0/991395792 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563e8e55caa0 msgr2=0x563e8e566e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:54.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.060+0000 7f1d6b205740 1 --2- 172.21.15.114:0/991395792 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563e8e55caa0 0x563e8e566e00 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c00bda0 tx=0x7f1d5c0107a0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.061+0000 7f1d6699f640 1 --2- 172.21.15.114:0/2585666917 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f1d4c0a1410 0x7f1d4c0a38d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1d58096010 tx=0x7f1d58009b70 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:54.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.061+0000 7f1d6b205740 1 -- 172.21.15.114:0/991395792 shutdown_connections 2024-09-13T18:25:54.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.061+0000 7f1d6b205740 1 --2- 172.21.15.114:0/991395792 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e8e567340 0x563e8e569730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.061+0000 7f1d6b205740 1 --2- 172.21.15.114:0/991395792 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563e8e55caa0 0x563e8e566e00 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.061+0000 7f1d6b205740 1 --2- 172.21.15.114:0/991395792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e8e55c180 0x563e8e55c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.061+0000 7f1d6b205740 1 -- 172.21.15.114:0/991395792 >> 172.21.15.114:0/991395792 conn(0x563e8e5586f0 msgr2=0x563e8e558ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:54.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.061+0000 7f1d6b205740 1 -- 172.21.15.114:0/991395792 shutdown_connections 2024-09-13T18:25:54.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.061+0000 7f1d6b205740 1 -- 172.21.15.114:0/991395792 wait complete. 2024-09-13T18:25:54.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.062+0000 7f1d6b205740 1 -- start start 2024-09-13T18:25:54.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.062+0000 7f1d6b205740 1 Processor -- start 2024-09-13T18:25:54.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.062+0000 7f1d6b205740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563e8e55caa0 0x563e8e566960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:54.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.062+0000 7f1d6b205740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e8e566ea0 0x563e8e563c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:54.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.062+0000 7f1d6b205740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e8e5642a0 0x563e8e564720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:54.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.062+0000 7f1d6b205740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e8e475180 con 0x563e8e55caa0 2024-09-13T18:25:54.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.062+0000 7f1d6b205740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e8e18f430 con 0x563e8e5642a0 2024-09-13T18:25:54.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.062+0000 7f1d6b205740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e8e43a5e0 con 0x563e8e566ea0 2024-09-13T18:25:54.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.062+0000 7f1d6b205740 10 client.0 _ll_register_callbacks cb 0x563e8e4a6aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:25:54.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.063+0000 7f1d6599d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e8e5642a0 0x563e8e564720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:54.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.063+0000 7f1d6599d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e8e5642a0 0x563e8e564720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:52702/0 (socket says 172.21.15.114:52702) 2024-09-13T18:25:54.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.063+0000 7f1d6599d640 1 -- 172.21.15.114:0/603243025 learned_addr learned my addr 172.21.15.114:0/603243025 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:54.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.063+0000 7f1d6619e640 1 --2- 172.21.15.114:0/603243025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e8e566ea0 0x563e8e563c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:54.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.063+0000 7f1d6599d640 1 -- 172.21.15.114:0/603243025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e8e566ea0 msgr2=0x563e8e563c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:54.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.063+0000 7f1d6599d640 1 --2- 172.21.15.114:0/603243025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e8e566ea0 0x563e8e563c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.063+0000 7f1d6599d640 1 -- 172.21.15.114:0/603243025 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563e8e55caa0 msgr2=0x563e8e566960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:54.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.063+0000 7f1d6599d640 1 --2- 172.21.15.114:0/603243025 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563e8e55caa0 0x563e8e566960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.063+0000 7f1d6599d640 1 -- 172.21.15.114:0/603243025 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d5c00b9d0 con 0x563e8e5642a0 2024-09-13T18:25:54.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.063+0000 7f1d6599d640 1 --2- 172.21.15.114:0/603243025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e8e5642a0 0x563e8e564720 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1d50009780 tx=0x7f1d50033c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34eb3509a821ead3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:54.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.064+0000 7f1d49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:25:54.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.064+0000 7f1d49ffb640 1 -- 172.21.15.114:0/603243025 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d50004230 con 0x563e8e5642a0 2024-09-13T18:25:54.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.064+0000 7f1d6b205740 1 -- 172.21.15.114:0/603243025 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563e8e4e0b80 con 0x563e8e5642a0 2024-09-13T18:25:54.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.064+0000 7f1d49ffb640 1 -- 172.21.15.114:0/603243025 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f1d50004390 con 0x563e8e5642a0 2024-09-13T18:25:54.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.064+0000 7f1d49ffb640 1 -- 172.21.15.114:0/603243025 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d50005070 con 0x563e8e5642a0 2024-09-13T18:25:54.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.064+0000 7f1d6b205740 10 client.26159 fetch_fsmap learned FSMap version 32 2024-09-13T18:25:54.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.064+0000 7f1d6b205740 1 -- 172.21.15.114:0/603243025 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x563e8e4b53c0 con 0x563e8e5642a0 2024-09-13T18:25:54.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.065+0000 7f1d49ffb640 1 -- 172.21.15.114:0/603243025 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563e8e4e0b80 con 0x563e8e5642a0 2024-09-13T18:25:54.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.065+0000 7f1d49ffb640 1 -- 172.21.15.114:0/603243025 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f1d50045070 con 0x563e8e5642a0 2024-09-13T18:25:54.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.065+0000 7f1d6b205740 10 client.26159 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:25:54.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.065+0000 7f1d6b205740 10 client.26159 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-13T18:25:54.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.065+0000 7f1d6b205740 20 client.26159 populate_metadata read hostname 'smithi114' 2024-09-13T18:25:54.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.065+0000 7f1d6b205740 1 --2- 172.21.15.114:0/603243025 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x563e8e588590 0x563e8e58a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:54.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.065+0000 7f1d6b205740 4 client.26159 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:25:54.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.065+0000 7f1d6b205740 1 -- 172.21.15.114:0/603243025 --> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563e8e4a91f0 con 0x563e8e588590 2024-09-13T18:25:54.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.065+0000 7f1d6619e640 1 --2- 172.21.15.114:0/603243025 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x563e8e588590 0x563e8e58a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:54.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.066+0000 7f1d6619e640 1 --2- 172.21.15.114:0/603243025 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x563e8e588590 0x563e8e58a9b0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3cfde8058f20e599 server_cookie=1888b1a68a6be872 in_seq=0 out_seq=0 2024-09-13T18:25:54.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.066+0000 7f1d49ffb640 10 client.26159 ms_handle_connect on v2:172.21.15.204:6838/344883912 2024-09-13T18:25:54.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.069+0000 7f1d49ffb640 1 -- 172.21.15.114:0/603243025 <== mds.2 v2:172.21.15.204:6838/344883912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x563e8e4a91f0 con 0x563e8e588590 2024-09-13T18:25:54.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.069+0000 7f1d49ffb640 10 client.26159 handle_command_reply: tid=0 2024-09-13T18:25:54.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.069+0000 7f1d6b205740 1 client.26159 shutdown 2024-09-13T18:25:54.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.069+0000 7f1d6b205740 1 -- 172.21.15.114:0/603243025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e8e5642a0 msgr2=0x563e8e564720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:54.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.069+0000 7f1d6b205740 1 --2- 172.21.15.114:0/603243025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e8e5642a0 0x563e8e564720 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1d50009780 tx=0x7f1d50033c80 comp rx=0 tx=0).stop 2024-09-13T18:25:54.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.070+0000 7f1d6b205740 1 -- 172.21.15.114:0/603243025 shutdown_connections 2024-09-13T18:25:54.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.070+0000 7f1d6b205740 1 --2- 172.21.15.114:0/603243025 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x563e8e588590 0x563e8e58a9b0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.070+0000 7f1d6b205740 1 --2- 172.21.15.114:0/603243025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e8e5642a0 0x563e8e564720 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.070+0000 7f1d6b205740 1 --2- 172.21.15.114:0/603243025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e8e566ea0 0x563e8e563c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.070+0000 7f1d6b205740 1 --2- 172.21.15.114:0/603243025 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563e8e55caa0 0x563e8e566960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.070+0000 7f1d6b205740 1 -- 172.21.15.114:0/603243025 >> 172.21.15.114:0/603243025 conn(0x563e8e5586f0 msgr2=0x563e8e55dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:54.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.070+0000 7f1d49ffb640 0 client.26159 ms_handle_reset on v2:172.21.15.204:6838/344883912 2024-09-13T18:25:54.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.070+0000 7f1d6b205740 1 -- 172.21.15.114:0/603243025 shutdown_connections 2024-09-13T18:25:54.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.070+0000 7f1d6b205740 1 -- 172.21.15.114:0/603243025 wait complete. 2024-09-13T18:25:54.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.072+0000 7f1d6b205740 20 client.26159 trim_cache size 0 max 16384 2024-09-13T18:25:54.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.098+0000 7f1d6b205740 1 Processor -- start 2024-09-13T18:25:54.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.099+0000 7f1d6b205740 1 -- start start 2024-09-13T18:25:54.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.099+0000 7f1d6b205740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e8e55caa0 0x563e8e58f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:54.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.099+0000 7f1d6b205740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563e8e5642a0 0x563e8e558ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:54.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.099+0000 7f1d6b205740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e8e566ea0 0x563e8e559010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:54.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.099+0000 7f1d6b205740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e8e585350 con 0x563e8e5642a0 2024-09-13T18:25:54.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.099+0000 7f1d6b205740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e8e54bd30 con 0x563e8e55caa0 2024-09-13T18:25:54.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.099+0000 7f1d6b205740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e8e55b940 con 0x563e8e566ea0 2024-09-13T18:25:54.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.100+0000 7f1d6619e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563e8e5642a0 0x563e8e558ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:54.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.100+0000 7f1d6619e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563e8e5642a0 0x563e8e558ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36260/0 (socket says 172.21.15.114:36260) 2024-09-13T18:25:54.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.100+0000 7f1d6619e640 1 -- 172.21.15.114:0/3770596552 learned_addr learned my addr 172.21.15.114:0/3770596552 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:54.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.101+0000 7f1d6619e640 1 -- 172.21.15.114:0/3770596552 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e8e566ea0 msgr2=0x563e8e559010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:25:54.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.101+0000 7f1d6619e640 1 --2- 172.21.15.114:0/3770596552 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e8e566ea0 0x563e8e559010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.101+0000 7f1d6619e640 1 -- 172.21.15.114:0/3770596552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e8e55caa0 msgr2=0x563e8e58f2d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:25:54.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.101+0000 7f1d6619e640 1 --2- 172.21.15.114:0/3770596552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e8e55caa0 0x563e8e58f2d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.101+0000 7f1d6619e640 1 -- 172.21.15.114:0/3770596552 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e8e457770 con 0x563e8e5642a0 2024-09-13T18:25:54.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.102+0000 7f1d6619e640 1 --2- 172.21.15.114:0/3770596552 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563e8e5642a0 0x563e8e558ad0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c000c00 tx=0x7f1d5c011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f780a5b06024012 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:54.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.102+0000 7f1d55606640 1 -- 172.21.15.114:0/3770596552 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c005130 con 0x563e8e5642a0 2024-09-13T18:25:54.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.102+0000 7f1d55606640 1 -- 172.21.15.114:0/3770596552 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1d5c02e070 con 0x563e8e5642a0 2024-09-13T18:25:54.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.102+0000 7f1d55606640 1 -- 172.21.15.114:0/3770596552 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c02a070 con 0x563e8e5642a0 2024-09-13T18:25:54.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.103+0000 7f1d6b205740 1 -- 172.21.15.114:0/3770596552 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563e8e5642a0 msgr2=0x563e8e558ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:54.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.103+0000 7f1d6b205740 1 --2- 172.21.15.114:0/3770596552 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563e8e5642a0 0x563e8e558ad0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c000c00 tx=0x7f1d5c011440 comp rx=0 tx=0).stop 2024-09-13T18:25:54.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.104+0000 7f1d6b205740 1 -- 172.21.15.114:0/3770596552 shutdown_connections 2024-09-13T18:25:54.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.104+0000 7f1d6b205740 1 --2- 172.21.15.114:0/3770596552 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e8e566ea0 0x563e8e559010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.104+0000 7f1d6b205740 1 --2- 172.21.15.114:0/3770596552 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563e8e5642a0 0x563e8e558ad0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.104+0000 7f1d6b205740 1 --2- 172.21.15.114:0/3770596552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e8e55caa0 0x563e8e58f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.104+0000 7f1d6b205740 1 -- 172.21.15.114:0/3770596552 >> 172.21.15.114:0/3770596552 conn(0x563e8e5586f0 msgr2=0x563e8e566500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:54.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.104+0000 7f1d6b205740 1 -- 172.21.15.114:0/3770596552 shutdown_connections 2024-09-13T18:25:54.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.104+0000 7f1d6b205740 1 -- 172.21.15.114:0/3770596552 wait complete. 2024-09-13T18:25:54.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.104+0000 7f1d6b205740 1 -- start start 2024-09-13T18:25:54.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.104+0000 7f1d6b205740 1 Processor -- start 2024-09-13T18:25:54.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.105+0000 7f1d6b205740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e8e55caa0 0x563e8e56c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:54.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.105+0000 7f1d6b205740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e8e5642a0 0x563e8e56cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:54.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.105+0000 7f1d6b205740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563e8e566ea0 0x563e8e5dda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:54.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.105+0000 7f1d6b205740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e8e56d2e0 con 0x563e8e566ea0 2024-09-13T18:25:54.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.105+0000 7f1d6b205740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e8e5ddf90 con 0x563e8e55caa0 2024-09-13T18:25:54.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.105+0000 7f1d6b205740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e8e5de0d0 con 0x563e8e5642a0 2024-09-13T18:25:54.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.105+0000 7f1d6b205740 10 client.0 _ll_register_callbacks cb 0x7f1d5c00b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:25:54.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.105+0000 7f1d6599d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e8e55caa0 0x563e8e56c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:54.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.105+0000 7f1d6599d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563e8e566ea0 0x563e8e5dda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:54.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.105+0000 7f1d6599d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e8e55caa0 0x563e8e56c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:52712/0 (socket says 172.21.15.114:52712) 2024-09-13T18:25:54.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.105+0000 7f1d6599d640 1 -- 172.21.15.114:0/1018288420 learned_addr learned my addr 172.21.15.114:0/1018288420 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:54.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.105+0000 7f1d6619e640 1 --2- 172.21.15.114:0/1018288420 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e8e5642a0 0x563e8e56cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:54.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.106+0000 7f1d6599d640 1 -- 172.21.15.114:0/1018288420 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e8e5642a0 msgr2=0x563e8e56cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:54.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.106+0000 7f1d6599d640 1 --2- 172.21.15.114:0/1018288420 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e8e5642a0 0x563e8e56cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.106+0000 7f1d6599d640 1 -- 172.21.15.114:0/1018288420 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563e8e566ea0 msgr2=0x563e8e5dda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:54.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.106+0000 7f1d6599d640 1 --2- 172.21.15.114:0/1018288420 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563e8e566ea0 0x563e8e5dda50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.106+0000 7f1d6599d640 1 -- 172.21.15.114:0/1018288420 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563e8e47e620 con 0x563e8e55caa0 2024-09-13T18:25:54.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.106+0000 7f1d6599d640 1 --2- 172.21.15.114:0/1018288420 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e8e55caa0 0x563e8e56c680 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1d500035f0 tx=0x7f1d5002c040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c35e0ef19afad73 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:54.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.107+0000 7f1d49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:25:54.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.107+0000 7f1d49ffb640 1 -- 172.21.15.114:0/1018288420 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d500054c0 con 0x563e8e55caa0 2024-09-13T18:25:54.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.107+0000 7f1d49ffb640 1 -- 172.21.15.114:0/1018288420 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f1d50005620 con 0x563e8e55caa0 2024-09-13T18:25:54.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.107+0000 7f1d6b205740 1 -- 172.21.15.114:0/1018288420 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1d5c015070 con 0x563e8e55caa0 2024-09-13T18:25:54.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.107+0000 7f1d49ffb640 1 -- 172.21.15.114:0/1018288420 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d500304e0 con 0x563e8e55caa0 2024-09-13T18:25:54.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.108+0000 7f1d6b205740 10 client.26162 fetch_fsmap learned FSMap version 32 2024-09-13T18:25:54.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.108+0000 7f1d6b205740 1 -- 172.21.15.114:0/1018288420 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x563e8e47ce80 con 0x563e8e55caa0 2024-09-13T18:25:54.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.108+0000 7f1d49ffb640 1 -- 172.21.15.114:0/1018288420 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1d5c015070 con 0x563e8e55caa0 2024-09-13T18:25:54.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.109+0000 7f1d49ffb640 1 -- 172.21.15.114:0/1018288420 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f1d50045050 con 0x563e8e55caa0 2024-09-13T18:25:54.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.109+0000 7f1d6b205740 10 client.26162 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:25:54.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.109+0000 7f1d6b205740 10 client.26162 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-13T18:25:54.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.109+0000 7f1d6b205740 20 client.26162 populate_metadata read hostname 'smithi114' 2024-09-13T18:25:54.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.109+0000 7f1d6b205740 1 --2- 172.21.15.114:0/1018288420 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x563e8e5df570 0x563e8e5e1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:54.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.109+0000 7f1d6b205740 4 client.26162 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:25:54.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.109+0000 7f1d6b205740 1 -- 172.21.15.114:0/1018288420 --> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563e8e1aa4e0 con 0x563e8e5df570 2024-09-13T18:25:54.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.109+0000 7f1d6619e640 1 --2- 172.21.15.114:0/1018288420 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x563e8e5df570 0x563e8e5e1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:54.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.110+0000 7f1d49ffb640 10 client.26162 ms_handle_connect on v2:172.21.15.204:6838/344883912 2024-09-13T18:25:54.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.110+0000 7f1d6619e640 1 --2- 172.21.15.114:0/1018288420 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x563e8e5df570 0x563e8e5e1990 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=53c1ccac61c2cbe0 server_cookie=1425845b0593299f in_seq=0 out_seq=0 2024-09-13T18:25:54.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.110+0000 7f1d49ffb640 1 -- 172.21.15.114:0/1018288420 <== mds.2 v2:172.21.15.204:6838/344883912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563e8e1aa4e0 con 0x563e8e5df570 2024-09-13T18:25:54.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.110+0000 7f1d49ffb640 10 client.26162 handle_command_reply: tid=0 2024-09-13T18:25:54.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.111+0000 7f1d6b205740 1 client.26162 shutdown 2024-09-13T18:25:54.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.111+0000 7f1d6b205740 1 -- 172.21.15.114:0/1018288420 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e8e55caa0 msgr2=0x563e8e56c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:54.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.111+0000 7f1d6b205740 1 --2- 172.21.15.114:0/1018288420 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e8e55caa0 0x563e8e56c680 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1d500035f0 tx=0x7f1d5002c040 comp rx=0 tx=0).stop 2024-09-13T18:25:54.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.112+0000 7f1d6b205740 1 -- 172.21.15.114:0/1018288420 shutdown_connections 2024-09-13T18:25:54.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.112+0000 7f1d6b205740 1 --2- 172.21.15.114:0/1018288420 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x563e8e5df570 0x563e8e5e1990 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.112+0000 7f1d6b205740 1 --2- 172.21.15.114:0/1018288420 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563e8e566ea0 0x563e8e5dda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.112+0000 7f1d6b205740 1 --2- 172.21.15.114:0/1018288420 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e8e5642a0 0x563e8e56cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.112+0000 7f1d6b205740 1 --2- 172.21.15.114:0/1018288420 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563e8e55caa0 0x563e8e56c680 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.112+0000 7f1d6b205740 1 -- 172.21.15.114:0/1018288420 >> 172.21.15.114:0/1018288420 conn(0x563e8e5586f0 msgr2=0x563e8e58f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:54.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.112+0000 7f1d49ffb640 0 client.26162 ms_handle_reset on v2:172.21.15.204:6838/344883912 2024-09-13T18:25:54.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.112+0000 7f1d6b205740 1 -- 172.21.15.114:0/1018288420 shutdown_connections 2024-09-13T18:25:54.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.113+0000 7f1d6b205740 1 -- 172.21.15.114:0/1018288420 wait complete. 2024-09-13T18:25:54.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.115+0000 7f1d6b205740 20 client.26162 trim_cache size 0 max 16384 2024-09-13T18:25:54.118 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:25:54.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.116+0000 7f1d671a0640 1 -- 172.21.15.114:0/2585666917 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f1d4c0a1410 msgr2=0x7f1d4c0a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:54.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.116+0000 7f1d671a0640 1 --2- 172.21.15.114:0/2585666917 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f1d4c0a1410 0x7f1d4c0a38d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1d58096010 tx=0x7f1d58009b70 comp rx=0 tx=0).stop 2024-09-13T18:25:54.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.116+0000 7f1d671a0640 1 -- 172.21.15.114:0/2585666917 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d6019ad20 msgr2=0x7f1d6019b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:54.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.116+0000 7f1d671a0640 1 --2- 172.21.15.114:0/2585666917 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d6019ad20 0x7f1d6019b1d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1d5800ef00 tx=0x7f1d5800c740 comp rx=0 tx=0).stop 2024-09-13T18:25:54.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.116+0000 7f1d671a0640 1 -- 172.21.15.114:0/2585666917 shutdown_connections 2024-09-13T18:25:54.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.116+0000 7f1d671a0640 1 --2- 172.21.15.114:0/2585666917 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f1d4c0a1410 0x7f1d4c0a38d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.116+0000 7f1d671a0640 1 --2- 172.21.15.114:0/2585666917 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d6019ad20 0x7f1d6019b1d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.116+0000 7f1d671a0640 1 --2- 172.21.15.114:0/2585666917 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d60105010 0x7f1d60196930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.116+0000 7f1d671a0640 1 --2- 172.21.15.114:0/2585666917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d600ffec0 0x7f1d601963f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.116+0000 7f1d671a0640 1 -- 172.21.15.114:0/2585666917 >> 172.21.15.114:0/2585666917 conn(0x7f1d600fb950 msgr2=0x7f1d600fcb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:54.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.118+0000 7f1d671a0640 1 -- 172.21.15.114:0/2585666917 shutdown_connections 2024-09-13T18:25:54.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.118+0000 7f1d671a0640 1 -- 172.21.15.114:0/2585666917 wait complete. 2024-09-13T18:25:54.148 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-09-13T18:25:54.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.360+0000 7fbe84101640 1 -- 172.21.15.114:0/1705545643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe7c107920 msgr2=0x7fbe7c107d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:54.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.360+0000 7fbe84101640 1 --2- 172.21.15.114:0/1705545643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe7c107920 0x7fbe7c107d60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c009ca0 tx=0x7fbe6c02a8f0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.361+0000 7fbe84101640 1 -- 172.21.15.114:0/1705545643 shutdown_connections 2024-09-13T18:25:54.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.361+0000 7fbe84101640 1 --2- 172.21.15.114:0/1705545643 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe7c119400 0x7fbe7c11b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.361+0000 7fbe84101640 1 --2- 172.21.15.114:0/1705545643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe7c107920 0x7fbe7c107d60 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.361+0000 7fbe84101640 1 --2- 172.21.15.114:0/1705545643 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe7c107540 0x7fbe7c10e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.361+0000 7fbe84101640 1 -- 172.21.15.114:0/1705545643 >> 172.21.15.114:0/1705545643 conn(0x7fbe7c06afe0 msgr2=0x7fbe7c06b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:54.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.361+0000 7fbe84101640 1 -- 172.21.15.114:0/1705545643 shutdown_connections 2024-09-13T18:25:54.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.361+0000 7fbe84101640 1 -- 172.21.15.114:0/1705545643 wait complete. 2024-09-13T18:25:54.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.362+0000 7fbe84101640 1 Processor -- start 2024-09-13T18:25:54.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.362+0000 7fbe84101640 1 -- start start 2024-09-13T18:25:54.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.362+0000 7fbe84101640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe7c107540 0x7fbe7c1a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:54.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.362+0000 7fbe84101640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe7c107920 0x7fbe7c1a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:54.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.362+0000 7fbe84101640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe7c119400 0x7fbe7c1a9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:54.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.362+0000 7fbe84101640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7c1aa0b0 con 0x7fbe7c107920 2024-09-13T18:25:54.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.362+0000 7fbe84101640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7c1aa220 con 0x7fbe7c107540 2024-09-13T18:25:54.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.362+0000 7fbe84101640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7c1aa3e0 con 0x7fbe7c119400 2024-09-13T18:25:54.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.363+0000 7fbe81e9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe7c107540 0x7fbe7c1a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:54.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.363+0000 7fbe8169c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe7c107920 0x7fbe7c1a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:54.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.363+0000 7fbe81e9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe7c107540 0x7fbe7c1a5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:52730/0 (socket says 172.21.15.114:52730) 2024-09-13T18:25:54.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.363+0000 7fbe81e9d640 1 -- 172.21.15.114:0/1162426240 learned_addr learned my addr 172.21.15.114:0/1162426240 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:54.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.363+0000 7fbe8269e640 1 --2- 172.21.15.114:0/1162426240 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe7c119400 0x7fbe7c1a9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:54.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.364+0000 7fbe81e9d640 1 -- 172.21.15.114:0/1162426240 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe7c119400 msgr2=0x7fbe7c1a9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:54.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.364+0000 7fbe81e9d640 1 --2- 172.21.15.114:0/1162426240 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe7c119400 0x7fbe7c1a9ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.364+0000 7fbe81e9d640 1 -- 172.21.15.114:0/1162426240 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe7c107920 msgr2=0x7fbe7c1a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:54.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.364+0000 7fbe81e9d640 1 --2- 172.21.15.114:0/1162426240 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe7c107920 0x7fbe7c1a56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.364+0000 7fbe81e9d640 1 -- 172.21.15.114:0/1162426240 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe6c009900 con 0x7fbe7c107540 2024-09-13T18:25:54.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.364+0000 7fbe81e9d640 1 --2- 172.21.15.114:0/1162426240 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe7c107540 0x7fbe7c1a5180 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbe7400e9f0 tx=0x7fbe7400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:54.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.365+0000 7fbe72ffd640 1 -- 172.21.15.114:0/1162426240 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe74009700 con 0x7fbe7c107540 2024-09-13T18:25:54.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.365+0000 7fbe72ffd640 1 -- 172.21.15.114:0/1162426240 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbe7400cd80 con 0x7fbe7c107540 2024-09-13T18:25:54.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.365+0000 7fbe72ffd640 1 -- 172.21.15.114:0/1162426240 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe740187e0 con 0x7fbe7c107540 2024-09-13T18:25:54.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.365+0000 7fbe84101640 1 -- 172.21.15.114:0/1162426240 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe7c1aa6c0 con 0x7fbe7c107540 2024-09-13T18:25:54.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.365+0000 7fbe84101640 1 -- 172.21.15.114:0/1162426240 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe7c0728c0 con 0x7fbe7c107540 2024-09-13T18:25:54.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.369+0000 7fbe86e21740 1 Processor -- start 2024-09-13T18:25:54.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.369+0000 7fbe86e21740 1 -- start start 2024-09-13T18:25:54.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.369+0000 7fbe86e21740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f10320790 0x555f10320b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:54.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.369+0000 7fbe86e21740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f103210b0 0x555f1032b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:54.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.369+0000 7fbe86e21740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f1032b950 0x555f1032dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:54.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.369+0000 7fbe86e21740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f1028acc0 con 0x555f103210b0 2024-09-13T18:25:54.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.369+0000 7fbe86e21740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f10288450 con 0x555f10320790 2024-09-13T18:25:54.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.369+0000 7fbe86e21740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f10283d40 con 0x555f1032b950 2024-09-13T18:25:54.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.370+0000 7fbe8169c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f1032b950 0x555f1032dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:54.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.370+0000 7fbe8169c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f1032b950 0x555f1032dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:37908/0 (socket says 172.21.15.114:37908) 2024-09-13T18:25:54.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.370+0000 7fbe8169c640 1 -- 172.21.15.114:0/2061528869 learned_addr learned my addr 172.21.15.114:0/2061528869 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:54.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.370+0000 7fbe8269e640 1 --2- 172.21.15.114:0/2061528869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f10320790 0x555f10320b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:54.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.371+0000 7fbe8169c640 1 -- 172.21.15.114:0/2061528869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f10320790 msgr2=0x555f10320b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:54.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.371+0000 7fbe8169c640 1 --2- 172.21.15.114:0/2061528869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f10320790 0x555f10320b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.371+0000 7fbe8169c640 1 -- 172.21.15.114:0/2061528869 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f103210b0 msgr2=0x555f1032b410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:25:54.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.371+0000 7fbe8169c640 1 --2- 172.21.15.114:0/2061528869 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f103210b0 0x555f1032b410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.371+0000 7fbe8169c640 1 -- 172.21.15.114:0/2061528869 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f1024b620 con 0x555f1032b950 2024-09-13T18:25:54.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.371+0000 7fbe8269e640 1 --2- 172.21.15.114:0/2061528869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f10320790 0x555f10320b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:25:54.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.371+0000 7fbe8169c640 1 --2- 172.21.15.114:0/2061528869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f1032b950 0x555f1032dd40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c009bf0 tx=0x7fbe6c02f320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6453c45f5db50f9d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:54.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.372+0000 7fbe63fff640 1 -- 172.21.15.114:0/2061528869 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe6c062070 con 0x555f1032b950 2024-09-13T18:25:54.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.372+0000 7fbe72ffd640 1 -- 172.21.15.114:0/1162426240 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fbe74010430 con 0x7fbe7c107540 2024-09-13T18:25:54.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.373+0000 7fbe63fff640 1 -- 172.21.15.114:0/2061528869 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbe6c031460 con 0x555f1032b950 2024-09-13T18:25:54.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.373+0000 7fbe63fff640 1 -- 172.21.15.114:0/2061528869 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe6c003e50 con 0x555f1032b950 2024-09-13T18:25:54.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.373+0000 7fbe72ffd640 1 --2- 172.21.15.114:0/1162426240 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbe640a1410 0x7fbe640a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:54.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.374+0000 7fbe72ffd640 1 -- 172.21.15.114:0/1162426240 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fbe74014070 con 0x7fbe7c107540 2024-09-13T18:25:54.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.374+0000 7fbe86e21740 1 -- 172.21.15.114:0/2061528869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f1032b950 msgr2=0x555f1032dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:54.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.374+0000 7fbe86e21740 1 --2- 172.21.15.114:0/2061528869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f1032b950 0x555f1032dd40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c009bf0 tx=0x7fbe6c02f320 comp rx=0 tx=0).stop 2024-09-13T18:25:54.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.374+0000 7fbe8269e640 1 --2- 172.21.15.114:0/1162426240 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbe640a1410 0x7fbe640a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:54.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.375+0000 7fbe86e21740 1 -- 172.21.15.114:0/2061528869 shutdown_connections 2024-09-13T18:25:54.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.375+0000 7fbe86e21740 1 --2- 172.21.15.114:0/2061528869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f1032b950 0x555f1032dd40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.375+0000 7fbe86e21740 1 --2- 172.21.15.114:0/2061528869 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f103210b0 0x555f1032b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.375+0000 7fbe86e21740 1 --2- 172.21.15.114:0/2061528869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f10320790 0x555f10320b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.375+0000 7fbe86e21740 1 -- 172.21.15.114:0/2061528869 >> 172.21.15.114:0/2061528869 conn(0x555f1031cd00 msgr2=0x555f1031d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:54.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.375+0000 7fbe8269e640 1 --2- 172.21.15.114:0/1162426240 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbe640a1410 0x7fbe640a38d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbe78005fd0 tx=0x7fbe7800b670 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:54.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.378+0000 7fbe86e21740 1 -- 172.21.15.114:0/2061528869 shutdown_connections 2024-09-13T18:25:54.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.378+0000 7fbe86e21740 1 -- 172.21.15.114:0/2061528869 wait complete. 2024-09-13T18:25:54.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.379+0000 7fbe86e21740 1 -- start start 2024-09-13T18:25:54.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.379+0000 7fbe86e21740 1 Processor -- start 2024-09-13T18:25:54.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.379+0000 7fbe86e21740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f10320790 0x555f1032afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:54.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.379+0000 7fbe86e21740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f103210b0 0x555f103282f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:54.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.379+0000 7fbe86e21740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f1032b950 0x555f10328830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:54.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.379+0000 7fbe86e21740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f10242180 con 0x555f103210b0 2024-09-13T18:25:54.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.379+0000 7fbe86e21740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f0ff5c430 con 0x555f1032b950 2024-09-13T18:25:54.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.379+0000 7fbe86e21740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f102075e0 con 0x555f10320790 2024-09-13T18:25:54.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.379+0000 7fbe86e21740 10 client.0 _ll_register_callbacks cb 0x555f10273aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:25:54.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.380+0000 7fbe8269e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f1032b950 0x555f10328830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:54.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.380+0000 7fbe8269e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f1032b950 0x555f10328830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:52750/0 (socket says 172.21.15.114:52750) 2024-09-13T18:25:54.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.380+0000 7fbe8269e640 1 -- 172.21.15.114:0/4168325074 learned_addr learned my addr 172.21.15.114:0/4168325074 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:54.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.380+0000 7fbe8169c640 1 --2- 172.21.15.114:0/4168325074 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f103210b0 0x555f103282f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:54.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.380+0000 7fbe81e9d640 1 --2- 172.21.15.114:0/4168325074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f10320790 0x555f1032afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:54.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.380+0000 7fbe81e9d640 1 -- 172.21.15.114:0/4168325074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f1032b950 msgr2=0x555f10328830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:54.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.380+0000 7fbe81e9d640 1 --2- 172.21.15.114:0/4168325074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f1032b950 0x555f10328830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.380+0000 7fbe81e9d640 1 -- 172.21.15.114:0/4168325074 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f103210b0 msgr2=0x555f103282f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:54.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.380+0000 7fbe81e9d640 1 --2- 172.21.15.114:0/4168325074 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f103210b0 0x555f103282f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.380+0000 7fbe81e9d640 1 -- 172.21.15.114:0/4168325074 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe7c11c2e0 con 0x555f10320790 2024-09-13T18:25:54.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.380+0000 7fbe8269e640 1 --2- 172.21.15.114:0/4168325074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f1032b950 0x555f10328830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:25:54.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.381+0000 7fbe81e9d640 1 --2- 172.21.15.114:0/4168325074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f10320790 0x555f1032afd0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbe740ca8a0 tx=0x7fbe74099cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a49ff5b77110629 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:54.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.381+0000 7fbe617fa640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:25:54.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.381+0000 7fbe617fa640 1 -- 172.21.15.114:0/4168325074 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe740af030 con 0x555f10320790 2024-09-13T18:25:54.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.381+0000 7fbe617fa640 1 -- 172.21.15.114:0/4168325074 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fbe740b0070 con 0x555f10320790 2024-09-13T18:25:54.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.381+0000 7fbe86e21740 1 -- 172.21.15.114:0/4168325074 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555f102adb80 con 0x555f10320790 2024-09-13T18:25:54.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.381+0000 7fbe617fa640 1 -- 172.21.15.114:0/4168325074 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe740cd310 con 0x555f10320790 2024-09-13T18:25:54.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.382+0000 7fbe86e21740 10 client.26200 fetch_fsmap learned FSMap version 32 2024-09-13T18:25:54.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.382+0000 7fbe86e21740 1 -- 172.21.15.114:0/4168325074 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x555f102823c0 con 0x555f10320790 2024-09-13T18:25:54.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.382+0000 7fbe617fa640 1 -- 172.21.15.114:0/4168325074 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x555f102adb80 con 0x555f10320790 2024-09-13T18:25:54.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.383+0000 7fbe617fa640 1 -- 172.21.15.114:0/4168325074 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fbe7401f060 con 0x555f10320790 2024-09-13T18:25:54.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.383+0000 7fbe86e21740 10 client.26200 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:25:54.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.383+0000 7fbe86e21740 10 client.26200 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-13T18:25:54.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.383+0000 7fbe86e21740 20 client.26200 populate_metadata read hostname 'smithi114' 2024-09-13T18:25:54.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.383+0000 7fbe86e21740 1 --2- 172.21.15.114:0/4168325074 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x555f10331490 0x555f10333880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:54.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.383+0000 7fbe86e21740 4 client.26200 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:25:54.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.383+0000 7fbe86e21740 1 -- 172.21.15.114:0/4168325074 --> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555f102761f0 con 0x555f10331490 2024-09-13T18:25:54.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.385+0000 7fbe8169c640 1 --2- 172.21.15.114:0/4168325074 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x555f10331490 0x555f10333880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:54.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.385+0000 7fbe8169c640 1 --2- 172.21.15.114:0/4168325074 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x555f10331490 0x555f10333880 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2d5793dfebfd108f server_cookie=f1de7e82c511651a in_seq=0 out_seq=0 2024-09-13T18:25:54.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.385+0000 7fbe617fa640 10 client.26200 ms_handle_connect on v2:172.21.15.134:6836/891692210 2024-09-13T18:25:54.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.386+0000 7fbe617fa640 1 -- 172.21.15.114:0/4168325074 <== mds.1 v2:172.21.15.134:6836/891692210 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x555f102761f0 con 0x555f10331490 2024-09-13T18:25:54.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.386+0000 7fbe617fa640 10 client.26200 handle_command_reply: tid=0 2024-09-13T18:25:54.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.386+0000 7fbe86e21740 1 client.26200 shutdown 2024-09-13T18:25:54.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.387+0000 7fbe86e21740 1 -- 172.21.15.114:0/4168325074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f10320790 msgr2=0x555f1032afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:54.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.387+0000 7fbe86e21740 1 --2- 172.21.15.114:0/4168325074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f10320790 0x555f1032afd0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbe740ca8a0 tx=0x7fbe74099cb0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.387+0000 7fbe86e21740 1 -- 172.21.15.114:0/4168325074 shutdown_connections 2024-09-13T18:25:54.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.387+0000 7fbe86e21740 1 --2- 172.21.15.114:0/4168325074 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x555f10331490 0x555f10333880 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.387+0000 7fbe86e21740 1 --2- 172.21.15.114:0/4168325074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f1032b950 0x555f10328830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.387+0000 7fbe86e21740 1 --2- 172.21.15.114:0/4168325074 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f103210b0 0x555f103282f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.387+0000 7fbe86e21740 1 --2- 172.21.15.114:0/4168325074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f10320790 0x555f1032afd0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.387+0000 7fbe86e21740 1 -- 172.21.15.114:0/4168325074 >> 172.21.15.114:0/4168325074 conn(0x555f1031cd00 msgr2=0x555f10322a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:54.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.389+0000 7fbe617fa640 0 client.26200 ms_handle_reset on v2:172.21.15.134:6836/891692210 2024-09-13T18:25:54.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.389+0000 7fbe86e21740 1 -- 172.21.15.114:0/4168325074 shutdown_connections 2024-09-13T18:25:54.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.389+0000 7fbe86e21740 1 -- 172.21.15.114:0/4168325074 wait complete. 2024-09-13T18:25:54.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.390+0000 7fbe86e21740 20 client.26200 trim_cache size 0 max 16384 2024-09-13T18:25:54.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.412+0000 7fbe86e21740 1 Processor -- start 2024-09-13T18:25:54.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.412+0000 7fbe86e21740 1 -- start start 2024-09-13T18:25:54.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.412+0000 7fbe86e21740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f10320790 0x555f10328840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:54.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.412+0000 7fbe86e21740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f103210b0 0x555f1031c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:54.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.412+0000 7fbe86e21740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f1032b950 0x555f103416c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:54.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.412+0000 7fbe86e21740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f1031de90 con 0x555f1032b950 2024-09-13T18:25:54.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.412+0000 7fbe86e21740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f103188e0 con 0x555f103210b0 2024-09-13T18:25:54.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.412+0000 7fbe86e21740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f10326840 con 0x555f10320790 2024-09-13T18:25:54.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.412+0000 7fbe81e9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f10320790 0x555f10328840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:54.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.412+0000 7fbe8269e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f1032b950 0x555f103416c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:54.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.412+0000 7fbe81e9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f10320790 0x555f10328840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:37916/0 (socket says 172.21.15.114:37916) 2024-09-13T18:25:54.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.412+0000 7fbe81e9d640 1 -- 172.21.15.114:0/3143315983 learned_addr learned my addr 172.21.15.114:0/3143315983 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:54.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.412+0000 7fbe8169c640 1 --2- 172.21.15.114:0/3143315983 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f103210b0 0x555f1031c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:54.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.413+0000 7fbe8269e640 1 -- 172.21.15.114:0/3143315983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f10320790 msgr2=0x555f10328840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:54.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.413+0000 7fbe8269e640 1 --2- 172.21.15.114:0/3143315983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f10320790 0x555f10328840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.413+0000 7fbe8269e640 1 -- 172.21.15.114:0/3143315983 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f103210b0 msgr2=0x555f1031c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:54.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.413+0000 7fbe8269e640 1 --2- 172.21.15.114:0/3143315983 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f103210b0 0x555f1031c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.413+0000 7fbe8269e640 1 -- 172.21.15.114:0/3143315983 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f10224770 con 0x555f1032b950 2024-09-13T18:25:54.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.413+0000 7fbe8269e640 1 --2- 172.21.15.114:0/3143315983 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f1032b950 0x555f103416c0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fbe7800b5b0 tx=0x7fbe78014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34c31ff499c21398 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:54.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.414+0000 7fbe70ff9640 1 -- 172.21.15.114:0/3143315983 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe780206f0 con 0x555f1032b950 2024-09-13T18:25:54.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.415+0000 7fbe70ff9640 1 -- 172.21.15.114:0/3143315983 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbe78016070 con 0x555f1032b950 2024-09-13T18:25:54.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.415+0000 7fbe70ff9640 1 -- 172.21.15.114:0/3143315983 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe78028b70 con 0x555f1032b950 2024-09-13T18:25:54.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.416+0000 7fbe86e21740 1 -- 172.21.15.114:0/3143315983 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f1032b950 msgr2=0x555f103416c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:54.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.416+0000 7fbe86e21740 1 --2- 172.21.15.114:0/3143315983 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f1032b950 0x555f103416c0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fbe7800b5b0 tx=0x7fbe78014000 comp rx=0 tx=0).stop 2024-09-13T18:25:54.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.417+0000 7fbe86e21740 1 -- 172.21.15.114:0/3143315983 shutdown_connections 2024-09-13T18:25:54.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.417+0000 7fbe86e21740 1 --2- 172.21.15.114:0/3143315983 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f1032b950 0x555f103416c0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.417+0000 7fbe86e21740 1 --2- 172.21.15.114:0/3143315983 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f103210b0 0x555f1031c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.417+0000 7fbe86e21740 1 --2- 172.21.15.114:0/3143315983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f10320790 0x555f10328840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.417+0000 7fbe86e21740 1 -- 172.21.15.114:0/3143315983 >> 172.21.15.114:0/3143315983 conn(0x555f1031cd00 msgr2=0x555f1031d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:54.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.418+0000 7fbe86e21740 1 -- 172.21.15.114:0/3143315983 shutdown_connections 2024-09-13T18:25:54.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.418+0000 7fbe86e21740 1 -- 172.21.15.114:0/3143315983 wait complete. 2024-09-13T18:25:54.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.418+0000 7fbe86e21740 1 -- start start 2024-09-13T18:25:54.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.418+0000 7fbe86e21740 1 Processor -- start 2024-09-13T18:25:54.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.419+0000 7fbe86e21740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f10320790 0x555f103416d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:54.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.419+0000 7fbe86e21740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f103210b0 0x555f10341c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:54.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.419+0000 7fbe81e9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f10320790 0x555f103416d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:54.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.419+0000 7fbe81e9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f10320790 0x555f103416d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:52776/0 (socket says 172.21.15.114:52776) 2024-09-13T18:25:54.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.419+0000 7fbe81e9d640 1 -- 172.21.15.114:0/2256296664 learned_addr learned my addr 172.21.15.114:0/2256296664 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:54.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.419+0000 7fbe86e21740 1 --2- 172.21.15.114:0/2256296664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f1032b950 0x555f10324620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:54.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.420+0000 7fbe8169c640 1 --2- 172.21.15.114:0/2256296664 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f103210b0 0x555f10341c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:54.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.420+0000 7fbe86e21740 1 -- 172.21.15.114:0/2256296664 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f10324b90 con 0x555f103210b0 2024-09-13T18:25:54.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.420+0000 7fbe86e21740 1 -- 172.21.15.114:0/2256296664 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f10324d00 con 0x555f10320790 2024-09-13T18:25:54.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.420+0000 7fbe86e21740 1 -- 172.21.15.114:0/2256296664 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f10324ec0 con 0x555f1032b950 2024-09-13T18:25:54.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.420+0000 7fbe86e21740 10 client.0 _ll_register_callbacks cb 0x7fbe6c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:25:54.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.420+0000 7fbe8269e640 1 --2- 172.21.15.114:0/2256296664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f1032b950 0x555f10324620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:54.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.420+0000 7fbe8269e640 1 -- 172.21.15.114:0/2256296664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f10320790 msgr2=0x555f103416d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:54.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.420+0000 7fbe8269e640 1 --2- 172.21.15.114:0/2256296664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f10320790 0x555f103416d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.420+0000 7fbe8269e640 1 -- 172.21.15.114:0/2256296664 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f103210b0 msgr2=0x555f10341c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:54.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.420+0000 7fbe8269e640 1 --2- 172.21.15.114:0/2256296664 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f103210b0 0x555f10341c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.420+0000 7fbe8269e640 1 -- 172.21.15.114:0/2256296664 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe7800be30 con 0x555f1032b950 2024-09-13T18:25:54.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.421+0000 7fbe8269e640 1 --2- 172.21.15.114:0/2256296664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f1032b950 0x555f10324620 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbe7800b5b0 tx=0x7fbe78005500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a92659be85052743 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:54.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.421+0000 7fbe617fa640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:25:54.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.421+0000 7fbe617fa640 1 -- 172.21.15.114:0/2256296664 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe78005670 con 0x555f1032b950 2024-09-13T18:25:54.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.421+0000 7fbe617fa640 1 -- 172.21.15.114:0/2256296664 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fbe78018070 con 0x555f1032b950 2024-09-13T18:25:54.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.421+0000 7fbe86e21740 1 -- 172.21.15.114:0/2256296664 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbe6c031460 con 0x555f1032b950 2024-09-13T18:25:54.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.422+0000 7fbe617fa640 1 -- 172.21.15.114:0/2256296664 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe780048d0 con 0x555f1032b950 2024-09-13T18:25:54.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.422+0000 7fbe86e21740 10 client.26206 fetch_fsmap learned FSMap version 32 2024-09-13T18:25:54.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.422+0000 7fbe86e21740 1 -- 172.21.15.114:0/2256296664 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x555f10249e80 con 0x555f1032b950 2024-09-13T18:25:54.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.422+0000 7fbe617fa640 1 -- 172.21.15.114:0/2256296664 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbe780204f0 con 0x555f1032b950 2024-09-13T18:25:54.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.422+0000 7fbe617fa640 1 -- 172.21.15.114:0/2256296664 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fbe78029440 con 0x555f1032b950 2024-09-13T18:25:54.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.422+0000 7fbe86e21740 10 client.26206 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:25:54.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.422+0000 7fbe86e21740 10 client.26206 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-13T18:25:54.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.422+0000 7fbe86e21740 20 client.26206 populate_metadata read hostname 'smithi114' 2024-09-13T18:25:54.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.422+0000 7fbe86e21740 1 --2- 172.21.15.114:0/2256296664 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x555f103b4570 0x555f103b6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:54.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.422+0000 7fbe86e21740 4 client.26206 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:25:54.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.422+0000 7fbe86e21740 1 -- 172.21.15.114:0/2256296664 --> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fbe6c031460 con 0x555f103b4570 2024-09-13T18:25:54.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.423+0000 7fbe81e9d640 1 --2- 172.21.15.114:0/2256296664 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x555f103b4570 0x555f103b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:54.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.424+0000 7fbe81e9d640 1 --2- 172.21.15.114:0/2256296664 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x555f103b4570 0x555f103b6990 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=355c15cd8bc7f4f server_cookie=1cf2695169f78567 in_seq=0 out_seq=0 2024-09-13T18:25:54.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.424+0000 7fbe617fa640 10 client.26206 ms_handle_connect on v2:172.21.15.134:6836/891692210 2024-09-13T18:25:54.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.425+0000 7fbe617fa640 1 -- 172.21.15.114:0/2256296664 <== mds.1 v2:172.21.15.134:6836/891692210 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fbe6c031460 con 0x555f103b4570 2024-09-13T18:25:54.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.425+0000 7fbe617fa640 10 client.26206 handle_command_reply: tid=0 2024-09-13T18:25:54.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.425+0000 7fbe86e21740 1 client.26206 shutdown 2024-09-13T18:25:54.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.426+0000 7fbe86e21740 1 -- 172.21.15.114:0/2256296664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f1032b950 msgr2=0x555f10324620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:54.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.426+0000 7fbe86e21740 1 --2- 172.21.15.114:0/2256296664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f1032b950 0x555f10324620 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbe7800b5b0 tx=0x7fbe78005500 comp rx=0 tx=0).stop 2024-09-13T18:25:54.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.426+0000 7fbe86e21740 1 -- 172.21.15.114:0/2256296664 shutdown_connections 2024-09-13T18:25:54.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.426+0000 7fbe86e21740 1 --2- 172.21.15.114:0/2256296664 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x555f103b4570 0x555f103b6990 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.426+0000 7fbe86e21740 1 --2- 172.21.15.114:0/2256296664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f1032b950 0x555f10324620 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.426+0000 7fbe86e21740 1 --2- 172.21.15.114:0/2256296664 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f103210b0 0x555f10341c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.426+0000 7fbe86e21740 1 --2- 172.21.15.114:0/2256296664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f10320790 0x555f103416d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.426+0000 7fbe86e21740 1 -- 172.21.15.114:0/2256296664 >> 172.21.15.114:0/2256296664 conn(0x555f1031cd00 msgr2=0x555f10323360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:54.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.426+0000 7fbe617fa640 0 client.26206 ms_handle_reset on v2:172.21.15.134:6836/891692210 2024-09-13T18:25:54.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.427+0000 7fbe86e21740 1 -- 172.21.15.114:0/2256296664 shutdown_connections 2024-09-13T18:25:54.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.427+0000 7fbe86e21740 1 -- 172.21.15.114:0/2256296664 wait complete. 2024-09-13T18:25:54.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.427+0000 7fbe86e21740 20 client.26206 trim_cache size 0 max 16384 2024-09-13T18:25:54.430 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:25:54.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.428+0000 7fbe84101640 1 -- 172.21.15.114:0/1162426240 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbe640a1410 msgr2=0x7fbe640a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:54.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.428+0000 7fbe84101640 1 --2- 172.21.15.114:0/1162426240 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbe640a1410 0x7fbe640a38d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbe78005fd0 tx=0x7fbe7800b670 comp rx=0 tx=0).stop 2024-09-13T18:25:54.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.428+0000 7fbe84101640 1 -- 172.21.15.114:0/1162426240 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe7c107540 msgr2=0x7fbe7c1a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:54.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.428+0000 7fbe84101640 1 --2- 172.21.15.114:0/1162426240 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe7c107540 0x7fbe7c1a5180 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbe7400e9f0 tx=0x7fbe7400eeb0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.428+0000 7fbe84101640 1 -- 172.21.15.114:0/1162426240 shutdown_connections 2024-09-13T18:25:54.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.428+0000 7fbe84101640 1 --2- 172.21.15.114:0/1162426240 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbe640a1410 0x7fbe640a38d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.428+0000 7fbe84101640 1 --2- 172.21.15.114:0/1162426240 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe7c119400 0x7fbe7c1a9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.428+0000 7fbe84101640 1 --2- 172.21.15.114:0/1162426240 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe7c107920 0x7fbe7c1a56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.429+0000 7fbe84101640 1 --2- 172.21.15.114:0/1162426240 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe7c107540 0x7fbe7c1a5180 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.429+0000 7fbe84101640 1 -- 172.21.15.114:0/1162426240 >> 172.21.15.114:0/1162426240 conn(0x7fbe7c06afe0 msgr2=0x7fbe7c119cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:54.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.429+0000 7fbe84101640 1 -- 172.21.15.114:0/1162426240 shutdown_connections 2024-09-13T18:25:54.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.429+0000 7fbe84101640 1 -- 172.21.15.114:0/1162426240 wait complete. 2024-09-13T18:25:54.457 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:54 smithi114 ceph-mon[30897]: pgmap v106: 129 pgs: 129 active+clean; 13 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 68 MiB/s wr, 1.09k op/s 2024-09-13T18:25:54.458 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-09-13T18:25:54.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:54 smithi134 ceph-mon[34283]: pgmap v106: 129 pgs: 129 active+clean; 13 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 68 MiB/s wr, 1.09k op/s 2024-09-13T18:25:54.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.648+0000 7f2d09dd4640 1 -- 172.21.15.114:0/4013673298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d04107540 msgr2=0x7f2d0410e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:54.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.648+0000 7f2d09dd4640 1 --2- 172.21.15.114:0/4013673298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d04107540 0x7f2d0410e510 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc009ca0 tx=0x7f2cfc02a8c0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.648+0000 7f2d09dd4640 1 -- 172.21.15.114:0/4013673298 shutdown_connections 2024-09-13T18:25:54.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.648+0000 7f2d09dd4640 1 --2- 172.21.15.114:0/4013673298 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2d04119400 0x7f2d0411b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.648+0000 7f2d09dd4640 1 --2- 172.21.15.114:0/4013673298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d04107920 0x7f2d04107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.648+0000 7f2d09dd4640 1 --2- 172.21.15.114:0/4013673298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d04107540 0x7f2d0410e510 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.648+0000 7f2d09dd4640 1 -- 172.21.15.114:0/4013673298 >> 172.21.15.114:0/4013673298 conn(0x7f2d0406afe0 msgr2=0x7f2d0406b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:54.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.649+0000 7f2d09dd4640 1 -- 172.21.15.114:0/4013673298 shutdown_connections 2024-09-13T18:25:54.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.649+0000 7f2d09dd4640 1 -- 172.21.15.114:0/4013673298 wait complete. 2024-09-13T18:25:54.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.649+0000 7f2d09dd4640 1 Processor -- start 2024-09-13T18:25:54.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.650+0000 7f2d09dd4640 1 -- start start 2024-09-13T18:25:54.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.650+0000 7f2d09dd4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d04107540 0x7f2d04118780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:54.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.650+0000 7f2d09dd4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2d04107920 0x7f2d04118cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:54.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.650+0000 7f2d09dd4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d04119400 0x7f2d041127d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:54.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.650+0000 7f2d09dd4640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d04112f60 con 0x7f2d04107920 2024-09-13T18:25:54.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.650+0000 7f2d09dd4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d041130d0 con 0x7f2d04107540 2024-09-13T18:25:54.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.650+0000 7f2d09dd4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d04113290 con 0x7f2d04119400 2024-09-13T18:25:54.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.650+0000 7f2d02ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2d04107920 0x7f2d04118cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:54.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.650+0000 7f2d037fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d04107540 0x7f2d04118780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:54.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.650+0000 7f2d02ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2d04107920 0x7f2d04118cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36362/0 (socket says 172.21.15.114:36362) 2024-09-13T18:25:54.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.650+0000 7f2d02ffd640 1 -- 172.21.15.114:0/2168273373 learned_addr learned my addr 172.21.15.114:0/2168273373 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:54.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.651+0000 7f2d03fff640 1 --2- 172.21.15.114:0/2168273373 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d04119400 0x7f2d041127d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:54.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.651+0000 7f2d02ffd640 1 -- 172.21.15.114:0/2168273373 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d04119400 msgr2=0x7f2d041127d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:54.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.651+0000 7f2d02ffd640 1 --2- 172.21.15.114:0/2168273373 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d04119400 0x7f2d041127d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.651+0000 7f2d02ffd640 1 -- 172.21.15.114:0/2168273373 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d04107540 msgr2=0x7f2d04118780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:54.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.651+0000 7f2d02ffd640 1 --2- 172.21.15.114:0/2168273373 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d04107540 0x7f2d04118780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.651+0000 7f2d02ffd640 1 -- 172.21.15.114:0/2168273373 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cfc009900 con 0x7f2d04107920 2024-09-13T18:25:54.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.651+0000 7f2d037fe640 1 --2- 172.21.15.114:0/2168273373 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d04107540 0x7f2d04118780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:25:54.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.652+0000 7f2d02ffd640 1 --2- 172.21.15.114:0/2168273373 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2d04107920 0x7f2d04118cc0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f2cf800c890 tx=0x7f2cf800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:54.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.652+0000 7f2d00ff9640 1 -- 172.21.15.114:0/2168273373 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf8004220 con 0x7f2d04107920 2024-09-13T18:25:54.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.652+0000 7f2d00ff9640 1 -- 172.21.15.114:0/2168273373 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2cf8007e40 con 0x7f2d04107920 2024-09-13T18:25:54.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.652+0000 7f2d00ff9640 1 -- 172.21.15.114:0/2168273373 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf8004a00 con 0x7f2d04107920 2024-09-13T18:25:54.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.652+0000 7f2d09dd4640 1 -- 172.21.15.114:0/2168273373 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d04113570 con 0x7f2d04107920 2024-09-13T18:25:54.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.652+0000 7f2d09dd4640 1 -- 172.21.15.114:0/2168273373 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d041ba570 con 0x7f2d04107920 2024-09-13T18:25:54.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.657+0000 7f2d0cbd3740 1 Processor -- start 2024-09-13T18:25:54.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.658+0000 7f2d00ff9640 1 -- 172.21.15.114:0/2168273373 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f2cf8004b60 con 0x7f2d04107920 2024-09-13T18:25:54.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.659+0000 7f2d00ff9640 1 --2- 172.21.15.114:0/2168273373 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2cec0a1410 0x7f2cec0a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:54.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.659+0000 7f2d00ff9640 1 -- 172.21.15.114:0/2168273373 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2cf80ccf10 con 0x7f2d04107920 2024-09-13T18:25:54.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.662+0000 7f2d0cbd3740 1 -- start start 2024-09-13T18:25:54.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.662+0000 7f2d0cbd3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558216cb9790 0x558216cb9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:54.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.662+0000 7f2d03fff640 1 --2- 172.21.15.114:0/2168273373 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2cec0a1410 0x7f2cec0a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:54.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.662+0000 7f2d0cbd3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558216cba0b0 0x558216cc4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:54.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.662+0000 7f2d0cbd3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558216cc4950 0x558216cc6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:54.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.662+0000 7f2d0cbd3740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x558216c23cc0 con 0x558216cba0b0 2024-09-13T18:25:54.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.662+0000 7f2d0cbd3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558216c21450 con 0x558216cb9790 2024-09-13T18:25:54.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.662+0000 7f2d0cbd3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558216c1cd40 con 0x558216cc4950 2024-09-13T18:25:54.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.662+0000 7f2d03fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558216cc4950 0x558216cc6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:54.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.662+0000 7f2d037fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558216cb9790 0x558216cb9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:54.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.662+0000 7f2d037fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558216cb9790 0x558216cb9b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:52802/0 (socket says 172.21.15.114:52802) 2024-09-13T18:25:54.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.662+0000 7f2d037fe640 1 -- 172.21.15.114:0/3605969182 learned_addr learned my addr 172.21.15.114:0/3605969182 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:54.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.662+0000 7f2d02ffd640 1 --2- 172.21.15.114:0/3605969182 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558216cba0b0 0x558216cc4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:54.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.663+0000 7f2d03fff640 1 -- 172.21.15.114:0/3605969182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558216cb9790 msgr2=0x558216cb9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:54.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.663+0000 7f2d03fff640 1 --2- 172.21.15.114:0/3605969182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558216cb9790 0x558216cb9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.663+0000 7f2d03fff640 1 -- 172.21.15.114:0/3605969182 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558216cba0b0 msgr2=0x558216cc4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:54.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.663+0000 7f2d03fff640 1 --2- 172.21.15.114:0/3605969182 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558216cba0b0 0x558216cc4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.663+0000 7f2d03fff640 1 -- 172.21.15.114:0/3605969182 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558216be4620 con 0x558216cc4950 2024-09-13T18:25:54.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.663+0000 7f2d037fe640 1 --2- 172.21.15.114:0/3605969182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558216cb9790 0x558216cb9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:25:54.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.663+0000 7f2d03fff640 1 --2- 172.21.15.114:0/2168273373 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2cec0a1410 0x7f2cec0a38d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2cf4009fd0 tx=0x7f2cf4009f40 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:54.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.663+0000 7f2d03fff640 1 --2- 172.21.15.114:0/3605969182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558216cc4950 0x558216cc6d40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2cf4011670 tx=0x7f2cf401d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6e1f8407f5e9385 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:54.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.663+0000 7f2ce9ffb640 1 -- 172.21.15.114:0/3605969182 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf4004220 con 0x558216cc4950 2024-09-13T18:25:54.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.664+0000 7f2ce9ffb640 1 -- 172.21.15.114:0/3605969182 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2cf4018070 con 0x558216cc4950 2024-09-13T18:25:54.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.664+0000 7f2ce9ffb640 1 -- 172.21.15.114:0/3605969182 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf400ea10 con 0x558216cc4950 2024-09-13T18:25:54.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.665+0000 7f2d0cbd3740 1 -- 172.21.15.114:0/3605969182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558216cc4950 msgr2=0x558216cc6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:54.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.665+0000 7f2d0cbd3740 1 --2- 172.21.15.114:0/3605969182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558216cc4950 0x558216cc6d40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2cf4011670 tx=0x7f2cf401d000 comp rx=0 tx=0).stop 2024-09-13T18:25:54.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.665+0000 7f2d0cbd3740 1 -- 172.21.15.114:0/3605969182 shutdown_connections 2024-09-13T18:25:54.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.665+0000 7f2d0cbd3740 1 --2- 172.21.15.114:0/3605969182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558216cc4950 0x558216cc6d40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.665+0000 7f2d0cbd3740 1 --2- 172.21.15.114:0/3605969182 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558216cba0b0 0x558216cc4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.665+0000 7f2d0cbd3740 1 --2- 172.21.15.114:0/3605969182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558216cb9790 0x558216cb9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.665+0000 7f2d0cbd3740 1 -- 172.21.15.114:0/3605969182 >> 172.21.15.114:0/3605969182 conn(0x558216cb5d00 msgr2=0x558216cb60e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:54.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.667+0000 7f2d0cbd3740 1 -- 172.21.15.114:0/3605969182 shutdown_connections 2024-09-13T18:25:54.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.667+0000 7f2d0cbd3740 1 -- 172.21.15.114:0/3605969182 wait complete. 2024-09-13T18:25:54.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.667+0000 7f2d0cbd3740 1 -- start start 2024-09-13T18:25:54.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.667+0000 7f2d0cbd3740 1 Processor -- start 2024-09-13T18:25:54.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.668+0000 7f2d0cbd3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558216cb9790 0x558216cc3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:54.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.668+0000 7f2d0cbd3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558216cba0b0 0x558216cc1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:54.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.668+0000 7f2d0cbd3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558216cc4950 0x558216cc17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:54.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.668+0000 7f2d0cbd3740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x558216bdb180 con 0x558216cba0b0 2024-09-13T18:25:54.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.668+0000 7f2d0cbd3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582168f5430 con 0x558216cb9790 2024-09-13T18:25:54.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.668+0000 7f2d0cbd3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558216ba05e0 con 0x558216cc4950 2024-09-13T18:25:54.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.668+0000 7f2d0cbd3740 10 client.0 _ll_register_callbacks cb 0x558216c0caa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:25:54.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.668+0000 7f2d037fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558216cb9790 0x558216cc3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:54.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.668+0000 7f2d037fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558216cb9790 0x558216cc3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:52812/0 (socket says 172.21.15.114:52812) 2024-09-13T18:25:54.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.668+0000 7f2d037fe640 1 -- 172.21.15.114:0/1075428162 learned_addr learned my addr 172.21.15.114:0/1075428162 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:54.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.668+0000 7f2d02ffd640 1 --2- 172.21.15.114:0/1075428162 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558216cba0b0 0x558216cc1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:54.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.668+0000 7f2d03fff640 1 --2- 172.21.15.114:0/1075428162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558216cc4950 0x558216cc17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:54.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.669+0000 7f2d037fe640 1 -- 172.21.15.114:0/1075428162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558216cc4950 msgr2=0x558216cc17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:54.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.669+0000 7f2d037fe640 1 --2- 172.21.15.114:0/1075428162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558216cc4950 0x558216cc17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.669+0000 7f2d037fe640 1 -- 172.21.15.114:0/1075428162 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558216cba0b0 msgr2=0x558216cc1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:54.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.669+0000 7f2d037fe640 1 --2- 172.21.15.114:0/1075428162 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558216cba0b0 0x558216cc1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.669+0000 7f2d037fe640 1 -- 172.21.15.114:0/1075428162 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cf4011e50 con 0x558216cb9790 2024-09-13T18:25:54.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.669+0000 7f2d03fff640 1 --2- 172.21.15.114:0/1075428162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558216cc4950 0x558216cc17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:25:54.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.669+0000 7f2d037fe640 1 --2- 172.21.15.114:0/1075428162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558216cb9790 0x558216cc3f30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc005e30 tx=0x7f2cfc003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e875694f46d79e6a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:54.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.669+0000 7f2ce2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:25:54.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.669+0000 7f2ce2ffd640 1 -- 172.21.15.114:0/1075428162 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc042070 con 0x558216cb9790 2024-09-13T18:25:54.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.669+0000 7f2d0cbd3740 1 -- 172.21.15.114:0/1075428162 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558216c46b80 con 0x558216cb9790 2024-09-13T18:25:54.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.670+0000 7f2ce2ffd640 1 -- 172.21.15.114:0/1075428162 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2cfc0305c0 con 0x558216cb9790 2024-09-13T18:25:54.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.670+0000 7f2ce2ffd640 1 -- 172.21.15.114:0/1075428162 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2cfc003e50 con 0x558216cb9790 2024-09-13T18:25:54.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.670+0000 7f2d0cbd3740 10 client.26192 fetch_fsmap learned FSMap version 32 2024-09-13T18:25:54.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.670+0000 7f2d0cbd3740 1 -- 172.21.15.114:0/1075428162 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x558216c1b3c0 con 0x558216cb9790 2024-09-13T18:25:54.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.670+0000 7f2ce2ffd640 1 -- 172.21.15.114:0/1075428162 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558216c46b80 con 0x558216cb9790 2024-09-13T18:25:54.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.670+0000 7f2ce2ffd640 1 -- 172.21.15.114:0/1075428162 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f2cfc004240 con 0x558216cb9790 2024-09-13T18:25:54.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.670+0000 7f2d0cbd3740 10 client.26192 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:25:54.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.670+0000 7f2d0cbd3740 10 client.26192 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:25:54.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.670+0000 7f2d0cbd3740 20 client.26192 populate_metadata read hostname 'smithi114' 2024-09-13T18:25:54.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.670+0000 7f2d0cbd3740 1 --2- 172.21.15.114:0/1075428162 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x558216cd4490 0x558216cd68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:54.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.670+0000 7f2d0cbd3740 4 client.26192 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:25:54.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.670+0000 7f2d0cbd3740 1 -- 172.21.15.114:0/1075428162 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558216c0f1f0 con 0x558216cd4490 2024-09-13T18:25:54.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.671+0000 7f2d02ffd640 1 --2- 172.21.15.114:0/1075428162 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x558216cd4490 0x558216cd68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:54.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.672+0000 7f2ce2ffd640 10 client.26192 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:25:54.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.672+0000 7f2d02ffd640 1 --2- 172.21.15.114:0/1075428162 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x558216cd4490 0x558216cd68b0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e537154971d48100 server_cookie=dd7517bf6f034f81 in_seq=0 out_seq=0 2024-09-13T18:25:54.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.672+0000 7f2ce2ffd640 1 -- 172.21.15.114:0/1075428162 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x558216c0f1f0 con 0x558216cd4490 2024-09-13T18:25:54.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.672+0000 7f2ce2ffd640 10 client.26192 handle_command_reply: tid=0 2024-09-13T18:25:54.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.672+0000 7f2d0cbd3740 1 client.26192 shutdown 2024-09-13T18:25:54.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.673+0000 7f2d0cbd3740 1 -- 172.21.15.114:0/1075428162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558216cb9790 msgr2=0x558216cc3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:54.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.673+0000 7f2d0cbd3740 1 --2- 172.21.15.114:0/1075428162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558216cb9790 0x558216cc3f30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc005e30 tx=0x7f2cfc003ba0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.673+0000 7f2d0cbd3740 1 -- 172.21.15.114:0/1075428162 shutdown_connections 2024-09-13T18:25:54.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.673+0000 7f2d0cbd3740 1 --2- 172.21.15.114:0/1075428162 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x558216cd4490 0x558216cd68b0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.673+0000 7f2d0cbd3740 1 --2- 172.21.15.114:0/1075428162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558216cc4950 0x558216cc17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.673+0000 7f2d0cbd3740 1 --2- 172.21.15.114:0/1075428162 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558216cba0b0 0x558216cc1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.673+0000 7f2d0cbd3740 1 --2- 172.21.15.114:0/1075428162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558216cb9790 0x558216cc3f30 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.673+0000 7f2d0cbd3740 1 -- 172.21.15.114:0/1075428162 >> 172.21.15.114:0/1075428162 conn(0x558216cb5d00 msgr2=0x558216cbba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:54.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.673+0000 7f2ce2ffd640 0 client.26192 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:25:54.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.674+0000 7f2d0cbd3740 1 -- 172.21.15.114:0/1075428162 shutdown_connections 2024-09-13T18:25:54.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.674+0000 7f2d0cbd3740 1 -- 172.21.15.114:0/1075428162 wait complete. 2024-09-13T18:25:54.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.674+0000 7f2d0cbd3740 20 client.26192 trim_cache size 0 max 16384 2024-09-13T18:25:54.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:54 smithi204 ceph-mon[34777]: pgmap v106: 129 pgs: 129 active+clean; 13 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 68 MiB/s wr, 1.09k op/s 2024-09-13T18:25:54.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.697+0000 7f2d0cbd3740 1 Processor -- start 2024-09-13T18:25:54.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.697+0000 7f2d0cbd3740 1 -- start start 2024-09-13T18:25:54.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.698+0000 7f2d0cbd3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558216cb9790 0x558216cb56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:54.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.698+0000 7f2d0cbd3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558216cba0b0 0x558216cc1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:54.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.698+0000 7f2d0cbd3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558216cc4950 0x558216cc1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:54.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.698+0000 7f2d02ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558216cba0b0 0x558216cc1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:54.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.698+0000 7f2d02ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558216cba0b0 0x558216cc1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:37964/0 (socket says 172.21.15.114:37964) 2024-09-13T18:25:54.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.698+0000 7f2d02ffd640 1 -- 172.21.15.114:0/1098529288 learned_addr learned my addr 172.21.15.114:0/1098529288 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:54.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.698+0000 7f2d0cbd3740 1 -- 172.21.15.114:0/1098529288 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x558216cc4780 con 0x558216cb9790 2024-09-13T18:25:54.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.698+0000 7f2d0cbd3740 1 -- 172.21.15.114:0/1098529288 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558216cb6e70 con 0x558216cc4950 2024-09-13T18:25:54.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.699+0000 7f2d0cbd3740 1 -- 172.21.15.114:0/1098529288 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558216cbf7a0 con 0x558216cba0b0 2024-09-13T18:25:54.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.699+0000 7f2d037fe640 1 --2- 172.21.15.114:0/1098529288 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558216cb9790 0x558216cb56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:54.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.699+0000 7f2d02ffd640 1 -- 172.21.15.114:0/1098529288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558216cc4950 msgr2=0x558216cc1770 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:25:54.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.699+0000 7f2d02ffd640 1 --2- 172.21.15.114:0/1098529288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558216cc4950 0x558216cc1770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.699+0000 7f2d02ffd640 1 -- 172.21.15.114:0/1098529288 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558216cb9790 msgr2=0x558216cb56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:54.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.699+0000 7f2d02ffd640 1 --2- 172.21.15.114:0/1098529288 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558216cb9790 0x558216cb56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.699+0000 7f2d02ffd640 1 -- 172.21.15.114:0/1098529288 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558216bbd770 con 0x558216cba0b0 2024-09-13T18:25:54.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.700+0000 7f2d02ffd640 1 --2- 172.21.15.114:0/1098529288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558216cba0b0 0x558216cc1230 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8007fd0 tx=0x7f2cf8096190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51383ebbc25db041 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:54.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.702+0000 7f2cea7fc640 1 -- 172.21.15.114:0/1098529288 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf8097e60 con 0x558216cba0b0 2024-09-13T18:25:54.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.702+0000 7f2cea7fc640 1 -- 172.21.15.114:0/1098529288 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2cf80980c0 con 0x558216cba0b0 2024-09-13T18:25:54.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.702+0000 7f2cea7fc640 1 -- 172.21.15.114:0/1098529288 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf8099210 con 0x558216cba0b0 2024-09-13T18:25:54.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.705+0000 7f2d0cbd3740 1 -- 172.21.15.114:0/1098529288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558216cba0b0 msgr2=0x558216cc1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:54.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.705+0000 7f2d0cbd3740 1 --2- 172.21.15.114:0/1098529288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558216cba0b0 0x558216cc1230 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8007fd0 tx=0x7f2cf8096190 comp rx=0 tx=0).stop 2024-09-13T18:25:54.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.708+0000 7f2d0cbd3740 1 -- 172.21.15.114:0/1098529288 shutdown_connections 2024-09-13T18:25:54.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.708+0000 7f2d0cbd3740 1 --2- 172.21.15.114:0/1098529288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558216cc4950 0x558216cc1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.708+0000 7f2d0cbd3740 1 --2- 172.21.15.114:0/1098529288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558216cba0b0 0x558216cc1230 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.708+0000 7f2d0cbd3740 1 --2- 172.21.15.114:0/1098529288 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558216cb9790 0x558216cb56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.708+0000 7f2d0cbd3740 1 -- 172.21.15.114:0/1098529288 >> 172.21.15.114:0/1098529288 conn(0x558216cb5d00 msgr2=0x558216cdb210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:54.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.711+0000 7f2d0cbd3740 1 -- 172.21.15.114:0/1098529288 shutdown_connections 2024-09-13T18:25:54.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.715+0000 7f2d0cbd3740 1 -- 172.21.15.114:0/1098529288 wait complete. 2024-09-13T18:25:54.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.716+0000 7f2d0cbd3740 1 -- start start 2024-09-13T18:25:54.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.716+0000 7f2d0cbd3740 1 Processor -- start 2024-09-13T18:25:54.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.717+0000 7f2d0cbd3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558216cb9790 0x558216cc7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:54.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.717+0000 7f2d0cbd3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558216cba0b0 0x558216cc8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:54.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.717+0000 7f2d0cbd3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558216cc4950 0x558216d239c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:54.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.717+0000 7f2d0cbd3740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x558216d23f80 con 0x558216cba0b0 2024-09-13T18:25:54.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.717+0000 7f2d0cbd3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558216d240f0 con 0x558216cc4950 2024-09-13T18:25:54.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.717+0000 7f2d0cbd3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558216d242b0 con 0x558216cb9790 2024-09-13T18:25:54.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.718+0000 7f2d0cbd3740 10 client.0 _ll_register_callbacks cb 0x7f2cf4009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:25:54.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.719+0000 7f2d02ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558216cba0b0 0x558216cc8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:54.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.719+0000 7f2d02ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558216cba0b0 0x558216cc8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36402/0 (socket says 172.21.15.114:36402) 2024-09-13T18:25:54.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.719+0000 7f2d02ffd640 1 -- 172.21.15.114:0/3275719796 learned_addr learned my addr 172.21.15.114:0/3275719796 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:54.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.719+0000 7f2d037fe640 1 --2- 172.21.15.114:0/3275719796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558216cb9790 0x558216cc7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:54.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.719+0000 7f2d03fff640 1 --2- 172.21.15.114:0/3275719796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558216cc4950 0x558216d239c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:54.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.719+0000 7f2d02ffd640 1 -- 172.21.15.114:0/3275719796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558216cb9790 msgr2=0x558216cc7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:54.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.720+0000 7f2d02ffd640 1 --2- 172.21.15.114:0/3275719796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558216cb9790 0x558216cc7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.720+0000 7f2d02ffd640 1 -- 172.21.15.114:0/3275719796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558216cc4950 msgr2=0x558216d239c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:54.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.720+0000 7f2d02ffd640 1 --2- 172.21.15.114:0/3275719796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558216cc4950 0x558216d239c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.720+0000 7f2d02ffd640 1 -- 172.21.15.114:0/3275719796 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d041ba570 con 0x558216cba0b0 2024-09-13T18:25:54.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.720+0000 7f2d03fff640 1 --2- 172.21.15.114:0/3275719796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558216cc4950 0x558216d239c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:25:54.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.720+0000 7f2d02ffd640 1 --2- 172.21.15.114:0/3275719796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558216cba0b0 0x558216cc8230 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8099a20 tx=0x7f2cf8099dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b9b779c29271db7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:54.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.720+0000 7f2ce2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:25:54.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.721+0000 7f2ce2ffd640 1 -- 172.21.15.114:0/3275719796 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf80ca530 con 0x558216cba0b0 2024-09-13T18:25:54.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.721+0000 7f2ce2ffd640 1 -- 172.21.15.114:0/3275719796 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2cf80ca690 con 0x558216cba0b0 2024-09-13T18:25:54.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.721+0000 7f2d0cbd3740 1 -- 172.21.15.114:0/3275719796 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2cf4018070 con 0x558216cba0b0 2024-09-13T18:25:54.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.721+0000 7f2ce2ffd640 1 -- 172.21.15.114:0/3275719796 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2cf80b8070 con 0x558216cba0b0 2024-09-13T18:25:54.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.721+0000 7f2d0cbd3740 10 client.16848 fetch_fsmap learned FSMap version 32 2024-09-13T18:25:54.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.721+0000 7f2d0cbd3740 1 -- 172.21.15.114:0/3275719796 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x558216be2e80 con 0x558216cba0b0 2024-09-13T18:25:54.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.722+0000 7f2ce2ffd640 1 -- 172.21.15.114:0/3275719796 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2cf4018070 con 0x558216cba0b0 2024-09-13T18:25:54.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.722+0000 7f2ce2ffd640 1 -- 172.21.15.114:0/3275719796 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f2cf8020070 con 0x558216cba0b0 2024-09-13T18:25:54.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.722+0000 7f2d0cbd3740 10 client.16848 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:25:54.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.722+0000 7f2d0cbd3740 10 client.16848 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:25:54.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.722+0000 7f2d0cbd3740 20 client.16848 populate_metadata read hostname 'smithi114' 2024-09-13T18:25:54.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.722+0000 7f2d0cbd3740 1 --2- 172.21.15.114:0/3275719796 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x558216d2b610 0x558216d2da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:54.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.722+0000 7f2d0cbd3740 4 client.16848 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:25:54.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.722+0000 7f2d0cbd3740 1 -- 172.21.15.114:0/3275719796 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5582169104e0 con 0x558216d2b610 2024-09-13T18:25:54.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.723+0000 7f2d037fe640 1 --2- 172.21.15.114:0/3275719796 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x558216d2b610 0x558216d2da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:54.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.723+0000 7f2ce2ffd640 10 client.16848 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:25:54.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.724+0000 7f2d037fe640 1 --2- 172.21.15.114:0/3275719796 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x558216d2b610 0x558216d2da00 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f29b1c838091a34 server_cookie=390645d8318eb7e0 in_seq=0 out_seq=0 2024-09-13T18:25:54.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.724+0000 7f2ce2ffd640 1 -- 172.21.15.114:0/3275719796 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5582169104e0 con 0x558216d2b610 2024-09-13T18:25:54.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.724+0000 7f2ce2ffd640 10 client.16848 handle_command_reply: tid=0 2024-09-13T18:25:54.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.724+0000 7f2d0cbd3740 1 client.16848 shutdown 2024-09-13T18:25:54.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.725+0000 7f2d0cbd3740 1 -- 172.21.15.114:0/3275719796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558216cba0b0 msgr2=0x558216cc8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:54.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.725+0000 7f2d0cbd3740 1 --2- 172.21.15.114:0/3275719796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558216cba0b0 0x558216cc8230 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8099a20 tx=0x7f2cf8099dd0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.727+0000 7f2d0cbd3740 1 -- 172.21.15.114:0/3275719796 shutdown_connections 2024-09-13T18:25:54.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.727+0000 7f2d0cbd3740 1 --2- 172.21.15.114:0/3275719796 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x558216d2b610 0x558216d2da00 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.727+0000 7f2d0cbd3740 1 --2- 172.21.15.114:0/3275719796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558216cc4950 0x558216d239c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.728+0000 7f2d0cbd3740 1 --2- 172.21.15.114:0/3275719796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558216cba0b0 0x558216cc8230 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.728+0000 7f2d0cbd3740 1 --2- 172.21.15.114:0/3275719796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558216cb9790 0x558216cc7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.728+0000 7f2d0cbd3740 1 -- 172.21.15.114:0/3275719796 >> 172.21.15.114:0/3275719796 conn(0x558216cb5d00 msgr2=0x558216cc1210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:54.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.728+0000 7f2ce2ffd640 0 client.16848 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:25:54.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.728+0000 7f2d0cbd3740 1 -- 172.21.15.114:0/3275719796 shutdown_connections 2024-09-13T18:25:54.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.728+0000 7f2d0cbd3740 1 -- 172.21.15.114:0/3275719796 wait complete. 2024-09-13T18:25:54.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.728+0000 7f2d0cbd3740 20 client.16848 trim_cache size 0 max 16384 2024-09-13T18:25:54.731 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:25:54.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.730+0000 7f2d09dd4640 1 -- 172.21.15.114:0/2168273373 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2cec0a1410 msgr2=0x7f2cec0a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:54.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.731+0000 7f2d09dd4640 1 --2- 172.21.15.114:0/2168273373 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2cec0a1410 0x7f2cec0a38d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2cf4009fd0 tx=0x7f2cf4009f40 comp rx=0 tx=0).stop 2024-09-13T18:25:54.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.731+0000 7f2d09dd4640 1 -- 172.21.15.114:0/2168273373 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2d04107920 msgr2=0x7f2d04118cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:54.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.731+0000 7f2d09dd4640 1 --2- 172.21.15.114:0/2168273373 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2d04107920 0x7f2d04118cc0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f2cf800c890 tx=0x7f2cf800cd50 comp rx=0 tx=0).stop 2024-09-13T18:25:54.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.731+0000 7f2d09dd4640 1 -- 172.21.15.114:0/2168273373 shutdown_connections 2024-09-13T18:25:54.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.731+0000 7f2d09dd4640 1 --2- 172.21.15.114:0/2168273373 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2cec0a1410 0x7f2cec0a38d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.731+0000 7f2d09dd4640 1 --2- 172.21.15.114:0/2168273373 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d04119400 0x7f2d041127d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.731+0000 7f2d09dd4640 1 --2- 172.21.15.114:0/2168273373 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2d04107920 0x7f2d04118cc0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.731+0000 7f2d09dd4640 1 --2- 172.21.15.114:0/2168273373 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d04107540 0x7f2d04118780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:54.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.731+0000 7f2d09dd4640 1 -- 172.21.15.114:0/2168273373 >> 172.21.15.114:0/2168273373 conn(0x7f2d0406afe0 msgr2=0x7f2d0406ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:54.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.732+0000 7f2d09dd4640 1 -- 172.21.15.114:0/2168273373 shutdown_connections 2024-09-13T18:25:54.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:54.733+0000 7f2d09dd4640 1 -- 172.21.15.114:0/2168273373 wait complete. 2024-09-13T18:25:55.735 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.732+0000 7f12531a0640 1 -- 172.21.15.114:0/87404840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f124c1074b0 msgr2=0x7f124c10e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:55.735 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.732+0000 7f12531a0640 1 --2- 172.21.15.114:0/87404840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f124c1074b0 0x7f124c10e510 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1244009cd0 tx=0x7f124402a930 comp rx=0 tx=0).stop 2024-09-13T18:25:55.735 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.732+0000 7f12531a0640 1 -- 172.21.15.114:0/87404840 shutdown_connections 2024-09-13T18:25:55.735 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.732+0000 7f12531a0640 1 --2- 172.21.15.114:0/87404840 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f124c119400 0x7f124c11b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:55.735 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.732+0000 7f12531a0640 1 --2- 172.21.15.114:0/87404840 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f124c107890 0x7f124c107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:55.735 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.732+0000 7f12531a0640 1 --2- 172.21.15.114:0/87404840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f124c1074b0 0x7f124c10e510 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:55.735 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.732+0000 7f12531a0640 1 -- 172.21.15.114:0/87404840 >> 172.21.15.114:0/87404840 conn(0x7f124c06adb0 msgr2=0x7f124c06b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:55.736 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.733+0000 7f12531a0640 1 -- 172.21.15.114:0/87404840 shutdown_connections 2024-09-13T18:25:55.736 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.733+0000 7f12531a0640 1 -- 172.21.15.114:0/87404840 wait complete. 2024-09-13T18:25:55.736 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.734+0000 7f12531a0640 1 Processor -- start 2024-09-13T18:25:55.736 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.734+0000 7f12531a0640 1 -- start start 2024-09-13T18:25:55.737 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.734+0000 7f12531a0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f124c1074b0 0x7f124c112b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:55.739 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.735+0000 7f12531a0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f124c107890 0x7f124c1130a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:55.739 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.735+0000 7f12531a0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f124c119400 0x7f124c1183e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:55.739 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.735+0000 7f12531a0640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124c118ae0 con 0x7f124c1074b0 2024-09-13T18:25:55.739 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.735+0000 7f12531a0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124c118c50 con 0x7f124c107890 2024-09-13T18:25:55.739 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.735+0000 7f12531a0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124c118e10 con 0x7f124c119400 2024-09-13T18:25:55.739 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.735+0000 7f125199d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f124c107890 0x7f124c1130a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:55.739 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.735+0000 7f125299f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f124c119400 0x7f124c1183e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.735+0000 7f125299f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f124c119400 0x7f124c1183e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:37992/0 (socket says 172.21.15.114:37992) 2024-09-13T18:25:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.735+0000 7f125299f640 1 -- 172.21.15.114:0/1002308232 learned_addr learned my addr 172.21.15.114:0/1002308232 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.735+0000 7f125299f640 1 -- 172.21.15.114:0/1002308232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f124c107890 msgr2=0x7f124c1130a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.735+0000 7f125299f640 1 --2- 172.21.15.114:0/1002308232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f124c107890 0x7f124c1130a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.735+0000 7f125299f640 1 -- 172.21.15.114:0/1002308232 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f124c1074b0 msgr2=0x7f124c112b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.735+0000 7f125299f640 1 --2- 172.21.15.114:0/1002308232 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f124c1074b0 0x7f124c112b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.735+0000 7f125299f640 1 -- 172.21.15.114:0/1002308232 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1244009900 con 0x7f124c119400 2024-09-13T18:25:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.736+0000 7f125299f640 1 --2- 172.21.15.114:0/1002308232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f124c119400 0x7f124c1183e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f124800d720 tx=0x7f124800dbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.736+0000 7f12437fe640 1 -- 172.21.15.114:0/1002308232 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1248009bd0 con 0x7f124c119400 2024-09-13T18:25:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.737+0000 7f12531a0640 1 -- 172.21.15.114:0/1002308232 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f124c1190f0 con 0x7f124c119400 2024-09-13T18:25:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.737+0000 7f12531a0640 1 -- 172.21.15.114:0/1002308232 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f124c1ba240 con 0x7f124c119400 2024-09-13T18:25:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.737+0000 7f12437fe640 1 -- 172.21.15.114:0/1002308232 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f124800de30 con 0x7f124c119400 2024-09-13T18:25:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.737+0000 7f12437fe640 1 -- 172.21.15.114:0/1002308232 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1248004d40 con 0x7f124c119400 2024-09-13T18:25:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.740+0000 7f1257200740 1 Processor -- start 2024-09-13T18:25:55.748 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.742+0000 7f1257200740 1 -- start start 2024-09-13T18:25:55.748 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.743+0000 7f1257200740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55712cf68500 0x55712cf688e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:55.748 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.743+0000 7f1257200740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55712cf68e20 0x55712cf73180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:55.748 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.743+0000 7f1257200740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55712cf736c0 0x55712cf75ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:55.748 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.743+0000 7f1257200740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55712ced3c30 con 0x55712cf68e20 2024-09-13T18:25:55.748 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.743+0000 7f1257200740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55712ced13c0 con 0x55712cf68500 2024-09-13T18:25:55.748 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.743+0000 7f1257200740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55712cecccb0 con 0x55712cf736c0 2024-09-13T18:25:55.749 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.744+0000 7f125219e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55712cf68e20 0x55712cf73180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:55.749 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.744+0000 7f125219e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55712cf68e20 0x55712cf73180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36440/0 (socket says 172.21.15.114:36440) 2024-09-13T18:25:55.749 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.744+0000 7f125219e640 1 -- 172.21.15.114:0/1605801951 learned_addr learned my addr 172.21.15.114:0/1605801951 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:55.749 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.744+0000 7f125219e640 1 -- 172.21.15.114:0/1605801951 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55712cf736c0 msgr2=0x55712cf75ab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:25:55.749 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.744+0000 7f125219e640 1 --2- 172.21.15.114:0/1605801951 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55712cf736c0 0x55712cf75ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:55.749 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.744+0000 7f125219e640 1 -- 172.21.15.114:0/1605801951 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55712cf68500 msgr2=0x55712cf688e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:25:55.749 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.744+0000 7f125219e640 1 --2- 172.21.15.114:0/1605801951 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55712cf68500 0x55712cf688e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:55.749 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.744+0000 7f125219e640 1 -- 172.21.15.114:0/1605801951 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55712ce73910 con 0x55712cf68e20 2024-09-13T18:25:55.749 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.745+0000 7f125219e640 1 --2- 172.21.15.114:0/1605801951 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55712cf68e20 0x55712cf73180 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f124402aea0 tx=0x7f1244031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71361398e5aab199 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:55.749 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.745+0000 7f12437fe640 1 -- 172.21.15.114:0/1002308232 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f1248004ea0 con 0x7f124c119400 2024-09-13T18:25:55.749 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.745+0000 7f1240e05640 1 -- 172.21.15.114:0/1605801951 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1244031b00 con 0x55712cf68e20 2024-09-13T18:25:55.749 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.745+0000 7f1240e05640 1 -- 172.21.15.114:0/1605801951 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f124403f070 con 0x55712cf68e20 2024-09-13T18:25:55.749 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.745+0000 7f1240e05640 1 -- 172.21.15.114:0/1605801951 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1244030b10 con 0x55712cf68e20 2024-09-13T18:25:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.746+0000 7f12437fe640 1 --2- 172.21.15.114:0/1002308232 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f12380a14d0 0x7f12380a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.746+0000 7f12437fe640 1 -- 172.21.15.114:0/1002308232 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f12480cc820 con 0x7f124c119400 2024-09-13T18:25:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.746+0000 7f1257200740 1 -- 172.21.15.114:0/1605801951 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55712cf68e20 msgr2=0x55712cf73180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.746+0000 7f1257200740 1 --2- 172.21.15.114:0/1605801951 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55712cf68e20 0x55712cf73180 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f124402aea0 tx=0x7f1244031000 comp rx=0 tx=0).stop 2024-09-13T18:25:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.747+0000 7f1257200740 1 -- 172.21.15.114:0/1605801951 shutdown_connections 2024-09-13T18:25:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.747+0000 7f1257200740 1 --2- 172.21.15.114:0/1605801951 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55712cf736c0 0x55712cf75ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.747+0000 7f1257200740 1 --2- 172.21.15.114:0/1605801951 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55712cf68e20 0x55712cf73180 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.747+0000 7f1257200740 1 --2- 172.21.15.114:0/1605801951 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55712cf68500 0x55712cf688e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.747+0000 7f1257200740 1 -- 172.21.15.114:0/1605801951 >> 172.21.15.114:0/1605801951 conn(0x55712cf64a90 msgr2=0x55712cf64e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.747+0000 7f1257200740 1 -- 172.21.15.114:0/1605801951 shutdown_connections 2024-09-13T18:25:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.747+0000 7f125199d640 1 --2- 172.21.15.114:0/1002308232 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f12380a14d0 0x7f12380a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.747+0000 7f1257200740 1 -- 172.21.15.114:0/1605801951 wait complete. 2024-09-13T18:25:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.748+0000 7f1257200740 1 -- start start 2024-09-13T18:25:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.748+0000 7f1257200740 1 Processor -- start 2024-09-13T18:25:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.748+0000 7f1257200740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55712cf68500 0x55712cf73000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.748+0000 7f1257200740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55712cf68e20 0x55712cf70320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.749+0000 7f1257200740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55712cf736c0 0x55712cf708a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.749+0000 7f1257200740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55712ce8ae80 con 0x55712cf68500 2024-09-13T18:25:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.749+0000 7f1257200740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55712ce8ab20 con 0x55712cf68e20 2024-09-13T18:25:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.749+0000 7f1257200740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55712ce50260 con 0x55712cf736c0 2024-09-13T18:25:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.749+0000 7f1257200740 10 client.0 _ll_register_callbacks cb 0x55712ce85df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:25:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.749+0000 7f125219e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55712cf68500 0x55712cf73000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.749+0000 7f125219e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55712cf68500 0x55712cf73000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36446/0 (socket says 172.21.15.114:36446) 2024-09-13T18:25:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.749+0000 7f125219e640 1 -- 172.21.15.114:0/2218642730 learned_addr learned my addr 172.21.15.114:0/2218642730 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.749+0000 7f125219e640 1 -- 172.21.15.114:0/2218642730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55712cf736c0 msgr2=0x55712cf708a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:25:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.749+0000 7f125219e640 1 --2- 172.21.15.114:0/2218642730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55712cf736c0 0x55712cf708a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.749+0000 7f125219e640 1 -- 172.21.15.114:0/2218642730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55712cf68e20 msgr2=0x55712cf70320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:25:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.749+0000 7f125219e640 1 --2- 172.21.15.114:0/2218642730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55712cf68e20 0x55712cf70320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.749+0000 7f125219e640 1 -- 172.21.15.114:0/2218642730 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f124c11c000 con 0x55712cf68500 2024-09-13T18:25:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.750+0000 7f125219e640 1 --2- 172.21.15.114:0/2218642730 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55712cf68500 0x55712cf73000 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f1244009bf0 tx=0x7f12440309c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e4c54893153e183 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.750+0000 7f1235ffb640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:25:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.750+0000 7f1235ffb640 1 -- 172.21.15.114:0/2218642730 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124404a070 con 0x55712cf68500 2024-09-13T18:25:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.750+0000 7f1235ffb640 1 -- 172.21.15.114:0/2218642730 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f1244041070 con 0x55712cf68500 2024-09-13T18:25:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.750+0000 7f1257200740 1 -- 172.21.15.114:0/2218642730 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55712cef5680 con 0x55712cf68500 2024-09-13T18:25:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.751+0000 7f1235ffb640 1 -- 172.21.15.114:0/2218642730 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1244030b70 con 0x55712cf68500 2024-09-13T18:25:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.751+0000 7f1257200740 10 client.16857 fetch_fsmap learned FSMap version 32 2024-09-13T18:25:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.751+0000 7f1257200740 1 -- 172.21.15.114:0/2218642730 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55712cecb330 con 0x55712cf68500 2024-09-13T18:25:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.753+0000 7f1235ffb640 1 -- 172.21.15.114:0/2218642730 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55712cef5680 con 0x55712cf68500 2024-09-13T18:25:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.754+0000 7f1235ffb640 1 -- 172.21.15.114:0/2218642730 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f124402fd00 con 0x55712cf68500 2024-09-13T18:25:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.754+0000 7f1257200740 10 client.16857 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:25:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.754+0000 7f1257200740 10 client.16857 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:25:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.754+0000 7f1257200740 20 client.16857 populate_metadata read hostname 'smithi114' 2024-09-13T18:25:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.754+0000 7f1257200740 1 --2- 172.21.15.114:0/2218642730 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55712cf82da0 0x55712cf851c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.754+0000 7f1257200740 4 client.16857 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:25:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.754+0000 7f1257200740 1 -- 172.21.15.114:0/2218642730 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55712cec05d0 con 0x55712cf82da0 2024-09-13T18:25:55.757 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-13T18:25:55.757 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-09-13T18:25:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.754+0000 7f125199d640 1 --2- 172.21.15.114:0/2218642730 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55712cf82da0 0x55712cf851c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.755+0000 7f125199d640 1 --2- 172.21.15.114:0/1002308232 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f12380a14d0 0x7f12380a3990 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x55712cf718a0 tx=0x7f123c009a40 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.755+0000 7f125199d640 1 --2- 172.21.15.114:0/2218642730 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55712cf82da0 0x55712cf851c0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d7ac7483979cf7a server_cookie=b6d05fb47937d063 in_seq=0 out_seq=0 2024-09-13T18:25:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.755+0000 7f1235ffb640 10 client.16857 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:25:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.756+0000 7f1235ffb640 1 -- 172.21.15.114:0/2218642730 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55712cec05d0 con 0x55712cf82da0 2024-09-13T18:25:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.756+0000 7f1235ffb640 10 client.16857 handle_command_reply: tid=0 2024-09-13T18:25:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.756+0000 7f1257200740 1 client.16857 shutdown 2024-09-13T18:25:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.757+0000 7f1257200740 1 -- 172.21.15.114:0/2218642730 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55712cf68500 msgr2=0x55712cf73000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.757+0000 7f1257200740 1 --2- 172.21.15.114:0/2218642730 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55712cf68500 0x55712cf73000 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f1244009bf0 tx=0x7f12440309c0 comp rx=0 tx=0).stop 2024-09-13T18:25:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.760+0000 7f1257200740 1 -- 172.21.15.114:0/2218642730 shutdown_connections 2024-09-13T18:25:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.760+0000 7f1257200740 1 --2- 172.21.15.114:0/2218642730 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55712cf82da0 0x55712cf851c0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.760+0000 7f1257200740 1 --2- 172.21.15.114:0/2218642730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55712cf736c0 0x55712cf708a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.760+0000 7f1257200740 1 --2- 172.21.15.114:0/2218642730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55712cf68e20 0x55712cf70320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.760+0000 7f1257200740 1 --2- 172.21.15.114:0/2218642730 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55712cf68500 0x55712cf73000 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.760+0000 7f1257200740 1 -- 172.21.15.114:0/2218642730 >> 172.21.15.114:0/2218642730 conn(0x55712cf64a90 msgr2=0x55712cf63d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.760+0000 7f1235ffb640 0 client.16857 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:25:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.760+0000 7f1257200740 1 -- 172.21.15.114:0/2218642730 shutdown_connections 2024-09-13T18:25:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.760+0000 7f1257200740 1 -- 172.21.15.114:0/2218642730 wait complete. 2024-09-13T18:25:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.761+0000 7f1257200740 20 client.16857 trim_cache size 0 max 16384 2024-09-13T18:25:55.787 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.784+0000 7f1257200740 1 Processor -- start 2024-09-13T18:25:55.788 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.786+0000 7f1257200740 1 -- start start 2024-09-13T18:25:55.788 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.786+0000 7f1257200740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55712cf68500 0x55712cf72b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:55.789 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.786+0000 7f1257200740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55712cf68e20 0x55712cf6f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:55.789 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.786+0000 7f1257200740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55712cf736c0 0x55712cf6fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:55.789 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.786+0000 7f1257200740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55712cf85840 con 0x55712cf68500 2024-09-13T18:25:55.789 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.786+0000 7f1257200740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55712cf6af60 con 0x55712cf68e20 2024-09-13T18:25:55.789 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.786+0000 7f1257200740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55712cf6b120 con 0x55712cf736c0 2024-09-13T18:25:55.790 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.787+0000 7f125219e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55712cf68500 0x55712cf72b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:55.790 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.787+0000 7f125219e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55712cf68500 0x55712cf72b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36454/0 (socket says 172.21.15.114:36454) 2024-09-13T18:25:55.791 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.787+0000 7f125219e640 1 -- 172.21.15.114:0/604011760 learned_addr learned my addr 172.21.15.114:0/604011760 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:55.791 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.787+0000 7f125199d640 1 --2- 172.21.15.114:0/604011760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55712cf68e20 0x55712cf6f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:55.791 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.787+0000 7f125299f640 1 --2- 172.21.15.114:0/604011760 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55712cf736c0 0x55712cf6fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:55.791 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.789+0000 7f125219e640 1 -- 172.21.15.114:0/604011760 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55712cf736c0 msgr2=0x55712cf6fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:55.791 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.789+0000 7f125219e640 1 --2- 172.21.15.114:0/604011760 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55712cf736c0 0x55712cf6fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:55.791 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.789+0000 7f125219e640 1 -- 172.21.15.114:0/604011760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55712cf68e20 msgr2=0x55712cf6f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:55.791 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.789+0000 7f125219e640 1 --2- 172.21.15.114:0/604011760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55712cf68e20 0x55712cf6f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:55.791 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.789+0000 7f125219e640 1 -- 172.21.15.114:0/604011760 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55712ce6d300 con 0x55712cf68500 2024-09-13T18:25:55.792 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.790+0000 7f125219e640 1 --2- 172.21.15.114:0/604011760 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55712cf68500 0x55712cf72b60 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f12440466f0 tx=0x7f1244031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c074e7b77479941 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.792+0000 7f1241606640 1 -- 172.21.15.114:0/604011760 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124404a070 con 0x55712cf68500 2024-09-13T18:25:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.792+0000 7f1241606640 1 -- 172.21.15.114:0/604011760 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1244005eb0 con 0x55712cf68500 2024-09-13T18:25:55.795 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.793+0000 7f1257200740 1 -- 172.21.15.114:0/604011760 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55712cf68500 msgr2=0x55712cf72b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:55.795 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.793+0000 7f1257200740 1 --2- 172.21.15.114:0/604011760 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55712cf68500 0x55712cf72b60 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f12440466f0 tx=0x7f1244031000 comp rx=0 tx=0).stop 2024-09-13T18:25:55.795 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.793+0000 7f1257200740 1 -- 172.21.15.114:0/604011760 shutdown_connections 2024-09-13T18:25:55.795 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.793+0000 7f1257200740 1 --2- 172.21.15.114:0/604011760 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55712cf736c0 0x55712cf6fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:55.796 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.793+0000 7f1257200740 1 --2- 172.21.15.114:0/604011760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55712cf68e20 0x55712cf6f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:55.796 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.793+0000 7f1257200740 1 --2- 172.21.15.114:0/604011760 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55712cf68500 0x55712cf72b60 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:55.796 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.793+0000 7f1257200740 1 -- 172.21.15.114:0/604011760 >> 172.21.15.114:0/604011760 conn(0x55712cf64a90 msgr2=0x55712cf89b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:55.799 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.797+0000 7f1257200740 1 -- 172.21.15.114:0/604011760 shutdown_connections 2024-09-13T18:25:55.800 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.797+0000 7f1257200740 1 -- 172.21.15.114:0/604011760 wait complete. 2024-09-13T18:25:55.800 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.798+0000 7f1257200740 1 -- start start 2024-09-13T18:25:55.800 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.798+0000 7f1257200740 1 Processor -- start 2024-09-13T18:25:55.806 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.799+0000 7f1257200740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55712cf68500 0x55712cf763f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:55.806 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.799+0000 7f1257200740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55712cf68e20 0x55712cf76930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:55.806 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.799+0000 7f1257200740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55712cf736c0 0x55712cfd8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:55.806 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.799+0000 7f1257200740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55712cf77100 con 0x55712cf68500 2024-09-13T18:25:55.806 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.799+0000 7f1257200740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55712cf77270 con 0x55712cf736c0 2024-09-13T18:25:55.806 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.799+0000 7f1257200740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55712cfd87d0 con 0x55712cf68e20 2024-09-13T18:25:55.806 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.799+0000 7f1257200740 10 client.0 _ll_register_callbacks cb 0x7f1244009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:25:55.806 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.799+0000 7f125199d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55712cf68e20 0x55712cf76930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:55.806 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.800+0000 7f125199d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55712cf68e20 0x55712cf76930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:38018/0 (socket says 172.21.15.114:38018) 2024-09-13T18:25:55.806 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.800+0000 7f125199d640 1 -- 172.21.15.114:0/1554651863 learned_addr learned my addr 172.21.15.114:0/1554651863 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:55.806 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.800+0000 7f125299f640 1 --2- 172.21.15.114:0/1554651863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55712cf736c0 0x55712cfd8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:55.806 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.800+0000 7f125219e640 1 --2- 172.21.15.114:0/1554651863 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55712cf68500 0x55712cf763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.800+0000 7f125199d640 1 -- 172.21.15.114:0/1554651863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55712cf736c0 msgr2=0x55712cfd8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.800+0000 7f125199d640 1 --2- 172.21.15.114:0/1554651863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55712cf736c0 0x55712cfd8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.800+0000 7f125199d640 1 -- 172.21.15.114:0/1554651863 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55712cf68500 msgr2=0x55712cf763f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.800+0000 7f125199d640 1 --2- 172.21.15.114:0/1554651863 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55712cf68500 0x55712cf763f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.800+0000 7f125199d640 1 -- 172.21.15.114:0/1554651863 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55712cecb330 con 0x55712cf68e20 2024-09-13T18:25:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.800+0000 7f125199d640 1 --2- 172.21.15.114:0/1554651863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55712cf68e20 0x55712cf76930 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f124c113f20 tx=0x7f123c0046e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2616d234b0528de server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.800+0000 7f1235ffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:25:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.801+0000 7f1235ffb640 1 -- 172.21.15.114:0/1554651863 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123c024720 con 0x55712cf68e20 2024-09-13T18:25:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.801+0000 7f1235ffb640 1 -- 172.21.15.114:0/1554651863 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f123c024880 con 0x55712cf68e20 2024-09-13T18:25:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.801+0000 7f1257200740 1 -- 172.21.15.114:0/1554651863 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f124403f070 con 0x55712cf68e20 2024-09-13T18:25:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.801+0000 7f1235ffb640 1 -- 172.21.15.114:0/1554651863 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f123c02d460 con 0x55712cf68e20 2024-09-13T18:25:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.801+0000 7f1257200740 10 client.26230 fetch_fsmap learned FSMap version 32 2024-09-13T18:25:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.801+0000 7f1257200740 1 -- 172.21.15.114:0/1554651863 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55712ce928e0 con 0x55712cf68e20 2024-09-13T18:25:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.802+0000 7f1235ffb640 1 -- 172.21.15.114:0/1554651863 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f124403f070 con 0x55712cf68e20 2024-09-13T18:25:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.802+0000 7f1235ffb640 1 -- 172.21.15.114:0/1554651863 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f123c023040 con 0x55712cf68e20 2024-09-13T18:25:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.802+0000 7f1257200740 10 client.26230 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:25:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.802+0000 7f1257200740 10 client.26230 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:25:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.802+0000 7f1257200740 20 client.26230 populate_metadata read hostname 'smithi114' 2024-09-13T18:25:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.802+0000 7f1257200740 1 --2- 172.21.15.114:0/1554651863 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55712cfda100 0x55712cfdc520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.802+0000 7f1257200740 4 client.26230 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T18:25:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.802+0000 7f1257200740 1 -- 172.21.15.114:0/1554651863 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55712cbc0300 con 0x55712cfda100 2024-09-13T18:25:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.805+0000 7f125219e640 1 --2- 172.21.15.114:0/1554651863 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55712cfda100 0x55712cfdc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.805+0000 7f1235ffb640 10 client.26230 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:25:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.805+0000 7f125219e640 1 --2- 172.21.15.114:0/1554651863 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55712cfda100 0x55712cfdc520 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8a0affaf88309b7 server_cookie=7ff3b3e43181b90b in_seq=0 out_seq=0 2024-09-13T18:25:55.813 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.811+0000 7f1235ffb640 1 -- 172.21.15.114:0/1554651863 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55712cbc0300 con 0x55712cfda100 2024-09-13T18:25:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.811+0000 7f1235ffb640 10 client.26230 handle_command_reply: tid=0 2024-09-13T18:25:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.811+0000 7f1257200740 1 client.26230 shutdown 2024-09-13T18:25:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.811+0000 7f1257200740 1 -- 172.21.15.114:0/1554651863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55712cf68e20 msgr2=0x55712cf76930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.811+0000 7f1257200740 1 --2- 172.21.15.114:0/1554651863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55712cf68e20 0x55712cf76930 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f124c113f20 tx=0x7f123c0046e0 comp rx=0 tx=0).stop 2024-09-13T18:25:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.811+0000 7f1257200740 1 -- 172.21.15.114:0/1554651863 shutdown_connections 2024-09-13T18:25:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.811+0000 7f1257200740 1 --2- 172.21.15.114:0/1554651863 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55712cfda100 0x55712cfdc520 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.812+0000 7f1257200740 1 --2- 172.21.15.114:0/1554651863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55712cf736c0 0x55712cfd8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.812+0000 7f1257200740 1 --2- 172.21.15.114:0/1554651863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55712cf68e20 0x55712cf76930 secure :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f124c113f20 tx=0x7f123c0046e0 comp rx=0 tx=0).stop 2024-09-13T18:25:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.812+0000 7f1257200740 1 --2- 172.21.15.114:0/1554651863 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55712cf68500 0x55712cf763f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.812+0000 7f1257200740 1 -- 172.21.15.114:0/1554651863 >> 172.21.15.114:0/1554651863 conn(0x55712cf64a90 msgr2=0x55712cf6b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.812+0000 7f1235ffb640 0 client.26230 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:25:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.813+0000 7f1257200740 1 -- 172.21.15.114:0/1554651863 shutdown_connections 2024-09-13T18:25:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.813+0000 7f1257200740 1 -- 172.21.15.114:0/1554651863 wait complete. 2024-09-13T18:25:55.816 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.813+0000 7f1257200740 20 client.26230 trim_cache size 0 max 16384 2024-09-13T18:25:55.816 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-13T18:25:55.816 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-13T18:25:55.816 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-13T18:25:55.816 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-13T18:25:55.817 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.815+0000 7f12531a0640 1 -- 172.21.15.114:0/1002308232 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f12380a14d0 msgr2=0x7f12380a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:55.817 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.815+0000 7f12531a0640 1 --2- 172.21.15.114:0/1002308232 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f12380a14d0 0x7f12380a3990 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x55712cf718a0 tx=0x7f123c009a40 comp rx=0 tx=0).stop 2024-09-13T18:25:55.818 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.815+0000 7f12531a0640 1 -- 172.21.15.114:0/1002308232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f124c119400 msgr2=0x7f124c1183e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:55.818 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.815+0000 7f12531a0640 1 --2- 172.21.15.114:0/1002308232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f124c119400 0x7f124c1183e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f124800d720 tx=0x7f124800dbe0 comp rx=0 tx=0).stop 2024-09-13T18:25:55.819 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.816+0000 7f12531a0640 1 -- 172.21.15.114:0/1002308232 shutdown_connections 2024-09-13T18:25:55.819 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.816+0000 7f12531a0640 1 --2- 172.21.15.114:0/1002308232 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f12380a14d0 0x7f12380a3990 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:55.819 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.816+0000 7f12531a0640 1 --2- 172.21.15.114:0/1002308232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f124c119400 0x7f124c1183e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:55.819 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.816+0000 7f12531a0640 1 --2- 172.21.15.114:0/1002308232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f124c107890 0x7f124c1130a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:55.819 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.816+0000 7f12531a0640 1 --2- 172.21.15.114:0/1002308232 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f124c1074b0 0x7f124c112b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:55.819 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.816+0000 7f12531a0640 1 -- 172.21.15.114:0/1002308232 >> 172.21.15.114:0/1002308232 conn(0x7f124c06adb0 msgr2=0x7f124c06ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:55.819 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.817+0000 7f12531a0640 1 -- 172.21.15.114:0/1002308232 shutdown_connections 2024-09-13T18:25:55.819 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:25:55.817+0000 7f12531a0640 1 -- 172.21.15.114:0/1002308232 wait complete. 2024-09-13T18:25:55.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.974+0000 7f27ba7fc640 1 -- 172.21.15.114:0/729213127 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27ac031730 con 0x7f27bc119400 2024-09-13T18:25:55.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.975+0000 7f27c1c12640 1 -- 172.21.15.114:0/729213127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27bc119400 msgr2=0x7f27bc11b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:55.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.975+0000 7f27c1c12640 1 --2- 172.21.15.114:0/729213127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27bc119400 0x7f27bc11b7f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f27ac009930 tx=0x7f27ac02a420 comp rx=0 tx=0).stop 2024-09-13T18:25:55.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.975+0000 7f27c1c12640 1 -- 172.21.15.114:0/729213127 shutdown_connections 2024-09-13T18:25:55.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.975+0000 7f27c1c12640 1 --2- 172.21.15.114:0/729213127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27bc119400 0x7f27bc11b7f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:55.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.975+0000 7f27c1c12640 1 --2- 172.21.15.114:0/729213127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27bc107890 0x7f27bc107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:55.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.975+0000 7f27c1c12640 1 --2- 172.21.15.114:0/729213127 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f27bc1074b0 0x7f27bc10e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:55.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.975+0000 7f27c1c12640 1 -- 172.21.15.114:0/729213127 >> 172.21.15.114:0/729213127 conn(0x7f27bc06af20 msgr2=0x7f27bc06b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:55.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.975+0000 7f27c1c12640 1 -- 172.21.15.114:0/729213127 shutdown_connections 2024-09-13T18:25:55.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.975+0000 7f27c1c12640 1 -- 172.21.15.114:0/729213127 wait complete. 2024-09-13T18:25:55.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.976+0000 7f27c1c12640 1 Processor -- start 2024-09-13T18:25:55.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.976+0000 7f27c1c12640 1 -- start start 2024-09-13T18:25:55.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.977+0000 7f27c1c12640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27bc1074b0 0x7f27bc1a4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:55.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.977+0000 7f27c1c12640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f27bc107890 0x7f27bc1a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:55.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.977+0000 7f27c1c12640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27bc119400 0x7f27bc1a97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:55.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.977+0000 7f27c1c12640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27bc1a9db0 con 0x7f27bc107890 2024-09-13T18:25:55.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.977+0000 7f27c1c12640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27bc1a9f20 con 0x7f27bc119400 2024-09-13T18:25:55.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.977+0000 7f27c1c12640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27bc1aa0e0 con 0x7f27bc1074b0 2024-09-13T18:25:55.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.977+0000 7f27bb7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27bc1074b0 0x7f27bc1a4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:55.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.977+0000 7f27bb7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27bc1074b0 0x7f27bc1a4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:38032/0 (socket says 172.21.15.114:38032) 2024-09-13T18:25:55.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.977+0000 7f27bb7fe640 1 -- 172.21.15.114:0/2667283677 learned_addr learned my addr 172.21.15.114:0/2667283677 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:55.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.977+0000 7f27baffd640 1 --2- 172.21.15.114:0/2667283677 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f27bc107890 0x7f27bc1a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:55.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.977+0000 7f27bbfff640 1 --2- 172.21.15.114:0/2667283677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27bc119400 0x7f27bc1a97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:55.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.978+0000 7f27bb7fe640 1 -- 172.21.15.114:0/2667283677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27bc119400 msgr2=0x7f27bc1a97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:55.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.978+0000 7f27bb7fe640 1 --2- 172.21.15.114:0/2667283677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27bc119400 0x7f27bc1a97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:55.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.978+0000 7f27bb7fe640 1 -- 172.21.15.114:0/2667283677 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f27bc107890 msgr2=0x7f27bc1a53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:55.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.978+0000 7f27bb7fe640 1 --2- 172.21.15.114:0/2667283677 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f27bc107890 0x7f27bc1a53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:55.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.978+0000 7f27bb7fe640 1 -- 172.21.15.114:0/2667283677 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b0009490 con 0x7f27bc1074b0 2024-09-13T18:25:55.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.978+0000 7f27baffd640 1 --2- 172.21.15.114:0/2667283677 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f27bc107890 0x7f27bc1a53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:25:55.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.978+0000 7f27bbfff640 1 --2- 172.21.15.114:0/2667283677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27bc119400 0x7f27bc1a97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:25:55.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.978+0000 7f27bb7fe640 1 --2- 172.21.15.114:0/2667283677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27bc1074b0 0x7f27bc1a4e80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f27b400ba50 tx=0x7f27b400bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:55.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.979+0000 7f27b8ff9640 1 -- 172.21.15.114:0/2667283677 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b400cbe0 con 0x7f27bc1074b0 2024-09-13T18:25:55.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.979+0000 7f27b8ff9640 1 -- 172.21.15.114:0/2667283677 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f27b400cd40 con 0x7f27bc1074b0 2024-09-13T18:25:55.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.979+0000 7f27c1c12640 1 -- 172.21.15.114:0/2667283677 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27ac009560 con 0x7f27bc1074b0 2024-09-13T18:25:55.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.979+0000 7f27b8ff9640 1 -- 172.21.15.114:0/2667283677 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b4007610 con 0x7f27bc1074b0 2024-09-13T18:25:55.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.979+0000 7f27c1c12640 1 -- 172.21.15.114:0/2667283677 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27bc1b1a60 con 0x7f27bc1074b0 2024-09-13T18:25:55.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.983+0000 7f27c4a11740 1 Processor -- start 2024-09-13T18:25:55.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.983+0000 7f27c4a11740 1 -- start start 2024-09-13T18:25:55.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.983+0000 7f27c4a11740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562a3c2a49a0 0x562a3c2a4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:55.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.983+0000 7f27c4a11740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a3c2a52c0 0x562a3c2af620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:55.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.983+0000 7f27b8ff9640 1 -- 172.21.15.114:0/2667283677 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f27b4014440 con 0x7f27bc1074b0 2024-09-13T18:25:55.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.983+0000 7f27c4a11740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a3c2afb60 0x562a3c2b1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:55.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.983+0000 7f27c4a11740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a3c20fc60 con 0x562a3c2a49a0 2024-09-13T18:25:55.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.983+0000 7f27c4a11740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a3c20d3f0 con 0x562a3c2a52c0 2024-09-13T18:25:55.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.983+0000 7f27c4a11740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a3c208ce0 con 0x562a3c2afb60 2024-09-13T18:25:55.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.984+0000 7f27bb7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a3c2a52c0 0x562a3c2af620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:55.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.984+0000 7f27bbfff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562a3c2a49a0 0x562a3c2a4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:55.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.984+0000 7f27bbfff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562a3c2a49a0 0x562a3c2a4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36478/0 (socket says 172.21.15.114:36478) 2024-09-13T18:25:55.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.984+0000 7f27bb7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a3c2a52c0 0x562a3c2af620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:52902/0 (socket says 172.21.15.114:52902) 2024-09-13T18:25:55.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.984+0000 7f27bbfff640 1 -- 172.21.15.114:0/2729503607 learned_addr learned my addr 172.21.15.114:0/2729503607 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:55.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.984+0000 7f27baffd640 1 --2- 172.21.15.114:0/2729503607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a3c2afb60 0x562a3c2b1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:55.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.985+0000 7f27baffd640 1 -- 172.21.15.114:0/2729503607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a3c2a52c0 msgr2=0x562a3c2af620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:55.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.985+0000 7f27baffd640 1 --2- 172.21.15.114:0/2729503607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a3c2a52c0 0x562a3c2af620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:55.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.985+0000 7f27baffd640 1 -- 172.21.15.114:0/2729503607 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562a3c2a49a0 msgr2=0x562a3c2a4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:55.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.985+0000 7f27baffd640 1 --2- 172.21.15.114:0/2729503607 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562a3c2a49a0 0x562a3c2a4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:55.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.985+0000 7f27baffd640 1 -- 172.21.15.114:0/2729503607 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a3c1c4970 con 0x562a3c2afb60 2024-09-13T18:25:55.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.985+0000 7f27baffd640 1 --2- 172.21.15.114:0/2729503607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a3c2afb60 0x562a3c2b1f50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f27b0011570 tx=0x7f27b0011a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3334bdc6ca93cc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:55.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.985+0000 7f27a1ffb640 1 -- 172.21.15.114:0/2729503607 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b0004020 con 0x562a3c2afb60 2024-09-13T18:25:55.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.985+0000 7f27a1ffb640 1 -- 172.21.15.114:0/2729503607 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f27b0005020 con 0x562a3c2afb60 2024-09-13T18:25:55.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.985+0000 7f27a1ffb640 1 -- 172.21.15.114:0/2729503607 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b0004ba0 con 0x562a3c2afb60 2024-09-13T18:25:55.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.985+0000 7f27b8ff9640 1 --2- 172.21.15.114:0/2667283677 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f27a40a1340 0x7f27a40a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:55.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.986+0000 7f27bbfff640 1 --2- 172.21.15.114:0/2667283677 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f27a40a1340 0x7f27a40a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:55.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.986+0000 7f27b8ff9640 1 -- 172.21.15.114:0/2667283677 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f27b40cc800 con 0x7f27bc1074b0 2024-09-13T18:25:55.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.986+0000 7f27c4a11740 1 -- 172.21.15.114:0/2729503607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a3c2afb60 msgr2=0x562a3c2b1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:55.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.986+0000 7f27c4a11740 1 --2- 172.21.15.114:0/2729503607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a3c2afb60 0x562a3c2b1f50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f27b0011570 tx=0x7f27b0011a30 comp rx=0 tx=0).stop 2024-09-13T18:25:55.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.987+0000 7f27c4a11740 1 -- 172.21.15.114:0/2729503607 shutdown_connections 2024-09-13T18:25:55.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.987+0000 7f27c4a11740 1 --2- 172.21.15.114:0/2729503607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a3c2afb60 0x562a3c2b1f50 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:55.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.987+0000 7f27c4a11740 1 --2- 172.21.15.114:0/2729503607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a3c2a52c0 0x562a3c2af620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:55.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.987+0000 7f27c4a11740 1 --2- 172.21.15.114:0/2729503607 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562a3c2a49a0 0x562a3c2a4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:55.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.987+0000 7f27c4a11740 1 -- 172.21.15.114:0/2729503607 >> 172.21.15.114:0/2729503607 conn(0x562a3c2a0f30 msgr2=0x562a3c2a1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:55.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.987+0000 7f27bbfff640 1 --2- 172.21.15.114:0/2667283677 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f27a40a1340 0x7f27a40a3800 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f27ac006010 tx=0x7f27ac02a920 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:55.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.990+0000 7f27c4a11740 1 -- 172.21.15.114:0/2729503607 shutdown_connections 2024-09-13T18:25:55.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.990+0000 7f27c4a11740 1 -- 172.21.15.114:0/2729503607 wait complete. 2024-09-13T18:25:55.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.991+0000 7f27c4a11740 1 -- start start 2024-09-13T18:25:55.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.991+0000 7f27c4a11740 1 Processor -- start 2024-09-13T18:25:55.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.991+0000 7f27c4a11740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a3c2a49a0 0x562a3c2af520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:55.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.992+0000 7f27c4a11740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562a3c2a52c0 0x562a3c2ac810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:55.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.992+0000 7f27c4a11740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a3c2afb60 0x562a3c2ace00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:55.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.992+0000 7f27c4a11740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a3c1c6f80 con 0x562a3c2a52c0 2024-09-13T18:25:55.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.992+0000 7f27c4a11740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a3c1c6c20 con 0x562a3c2afb60 2024-09-13T18:25:55.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.992+0000 7f27c4a11740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a3c193ab0 con 0x562a3c2a49a0 2024-09-13T18:25:55.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.992+0000 7f27c4a11740 10 client.0 _ll_register_callbacks cb 0x562a3c202350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:25:55.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.992+0000 7f27bb7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a3c2a49a0 0x562a3c2af520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:55.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.992+0000 7f27bb7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a3c2a49a0 0x562a3c2af520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:38046/0 (socket says 172.21.15.114:38046) 2024-09-13T18:25:55.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.992+0000 7f27bb7fe640 1 -- 172.21.15.114:0/4270799385 learned_addr learned my addr 172.21.15.114:0/4270799385 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:55.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.992+0000 7f27bbfff640 1 --2- 172.21.15.114:0/4270799385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a3c2afb60 0x562a3c2ace00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:55.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.993+0000 7f27bbfff640 1 -- 172.21.15.114:0/4270799385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a3c2a49a0 msgr2=0x562a3c2af520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:55.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.993+0000 7f27bbfff640 1 --2- 172.21.15.114:0/4270799385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a3c2a49a0 0x562a3c2af520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:55.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.993+0000 7f27baffd640 1 --2- 172.21.15.114:0/4270799385 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562a3c2a52c0 0x562a3c2ac810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:55.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.993+0000 7f27bbfff640 1 -- 172.21.15.114:0/4270799385 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562a3c2a52c0 msgr2=0x562a3c2ac810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:55.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.993+0000 7f27bbfff640 1 --2- 172.21.15.114:0/4270799385 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562a3c2a52c0 0x562a3c2ac810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:55.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.993+0000 7f27bbfff640 1 -- 172.21.15.114:0/4270799385 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27b0009d90 con 0x562a3c2afb60 2024-09-13T18:25:55.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.993+0000 7f27bb7fe640 1 --2- 172.21.15.114:0/4270799385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a3c2a49a0 0x562a3c2af520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:25:55.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.994+0000 7f27bbfff640 1 --2- 172.21.15.114:0/4270799385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a3c2afb60 0x562a3c2ace00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f27ac005ab0 tx=0x7f27ac02a420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7d9ef90ca998ec1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:55.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.995+0000 7f279affd640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:25:55.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.995+0000 7f279affd640 1 -- 172.21.15.114:0/4270799385 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27ac042070 con 0x562a3c2afb60 2024-09-13T18:25:55.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.995+0000 7f279affd640 1 -- 172.21.15.114:0/4270799385 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f27ac003800 con 0x562a3c2afb60 2024-09-13T18:25:55.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.995+0000 7f27baffd640 1 --2- 172.21.15.114:0/4270799385 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562a3c2a52c0 0x562a3c2ac810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:25:55.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.995+0000 7f27c4a11740 1 -- 172.21.15.114:0/4270799385 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562a3befc590 con 0x562a3c2afb60 2024-09-13T18:25:56.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.996+0000 7f279affd640 1 -- 172.21.15.114:0/4270799385 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27ac0039c0 con 0x562a3c2afb60 2024-09-13T18:25:56.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.996+0000 7f27c4a11740 10 client.26219 fetch_fsmap learned FSMap version 32 2024-09-13T18:25:56.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.996+0000 7f27c4a11740 1 -- 172.21.15.114:0/4270799385 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x562a3c207360 con 0x562a3c2afb60 2024-09-13T18:25:56.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.997+0000 7f279affd640 1 -- 172.21.15.114:0/4270799385 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562a3befc590 con 0x562a3c2afb60 2024-09-13T18:25:56.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.997+0000 7f279affd640 1 -- 172.21.15.114:0/4270799385 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f27ac054bf0 con 0x562a3c2afb60 2024-09-13T18:25:56.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.997+0000 7f27c4a11740 10 client.26219 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:25:56.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.997+0000 7f27c4a11740 10 client.26219 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:25:56.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.997+0000 7f27c4a11740 20 client.26219 populate_metadata read hostname 'smithi114' 2024-09-13T18:25:56.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.997+0000 7f27c4a11740 1 --2- 172.21.15.114:0/4270799385 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x562a3c2bf9f0 0x562a3c2c1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:56.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.997+0000 7f27c4a11740 4 client.26219 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:25:56.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:55.997+0000 7f27c4a11740 1 -- 172.21.15.114:0/4270799385 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562a3bf0d860 con 0x562a3c2bf9f0 2024-09-13T18:25:56.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.000+0000 7f27bb7fe640 1 --2- 172.21.15.114:0/4270799385 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x562a3c2bf9f0 0x562a3c2c1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:56.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.003+0000 7f27bb7fe640 1 --2- 172.21.15.114:0/4270799385 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x562a3c2bf9f0 0x562a3c2c1e10 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1841ec96bf4c606 server_cookie=4d4214ee25d12ca4 in_seq=0 out_seq=0 2024-09-13T18:25:56.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.003+0000 7f279affd640 10 client.26219 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:25:56.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.004+0000 7f279affd640 1 -- 172.21.15.114:0/4270799385 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x562a3bf0d860 con 0x562a3c2bf9f0 2024-09-13T18:25:56.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.004+0000 7f279affd640 10 client.26219 handle_command_reply: tid=0 2024-09-13T18:25:56.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.004+0000 7f27c4a11740 1 client.26219 shutdown 2024-09-13T18:25:56.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.005+0000 7f27c4a11740 1 -- 172.21.15.114:0/4270799385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a3c2afb60 msgr2=0x562a3c2ace00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:56.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.005+0000 7f27c4a11740 1 --2- 172.21.15.114:0/4270799385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a3c2afb60 0x562a3c2ace00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f27ac005ab0 tx=0x7f27ac02a420 comp rx=0 tx=0).stop 2024-09-13T18:25:56.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.005+0000 7f27c4a11740 1 -- 172.21.15.114:0/4270799385 shutdown_connections 2024-09-13T18:25:56.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.005+0000 7f27c4a11740 1 --2- 172.21.15.114:0/4270799385 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x562a3c2bf9f0 0x562a3c2c1e10 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:56.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.005+0000 7f27c4a11740 1 --2- 172.21.15.114:0/4270799385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a3c2afb60 0x562a3c2ace00 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:56.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.005+0000 7f279affd640 0 client.26219 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:25:56.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.005+0000 7f27c4a11740 1 --2- 172.21.15.114:0/4270799385 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562a3c2a52c0 0x562a3c2ac810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:56.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.005+0000 7f27c4a11740 1 --2- 172.21.15.114:0/4270799385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a3c2a49a0 0x562a3c2af520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:56.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.005+0000 7f27c4a11740 1 -- 172.21.15.114:0/4270799385 >> 172.21.15.114:0/4270799385 conn(0x562a3c2a0f30 msgr2=0x562a3c2a0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:56.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.006+0000 7f27c4a11740 1 -- 172.21.15.114:0/4270799385 shutdown_connections 2024-09-13T18:25:56.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.006+0000 7f27c4a11740 1 -- 172.21.15.114:0/4270799385 wait complete. 2024-09-13T18:25:56.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.006+0000 7f27c4a11740 20 client.26219 trim_cache size 0 max 16384 2024-09-13T18:25:56.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.033+0000 7f27c4a11740 1 Processor -- start 2024-09-13T18:25:56.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.034+0000 7f27c4a11740 1 -- start start 2024-09-13T18:25:56.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.035+0000 7f27c4a11740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562a3c2a49a0 0x562a3c2a0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:56.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.035+0000 7f27c4a11740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a3c2a52c0 0x562a3c2c67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:56.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.035+0000 7f27bb7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562a3c2a49a0 0x562a3c2a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:56.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.035+0000 7f27bb7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562a3c2a49a0 0x562a3c2a0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36500/0 (socket says 172.21.15.114:36500) 2024-09-13T18:25:56.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.035+0000 7f27bb7fe640 1 -- 172.21.15.114:0/1766319956 learned_addr learned my addr 172.21.15.114:0/1766319956 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:56.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.035+0000 7f27c4a11740 1 --2- 172.21.15.114:0/1766319956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a3c2afb60 0x562a3c2ac7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:56.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.036+0000 7f27c4a11740 1 -- 172.21.15.114:0/1766319956 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a3c2c6cf0 con 0x562a3c2a49a0 2024-09-13T18:25:56.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.036+0000 7f27c4a11740 1 -- 172.21.15.114:0/1766319956 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a3c2adb90 con 0x562a3c2a52c0 2024-09-13T18:25:56.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.036+0000 7f27c4a11740 1 -- 172.21.15.114:0/1766319956 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a3c2bbf00 con 0x562a3c2afb60 2024-09-13T18:25:56.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.036+0000 7f27bbfff640 1 --2- 172.21.15.114:0/1766319956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a3c2afb60 0x562a3c2ac7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:56.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.036+0000 7f27baffd640 1 --2- 172.21.15.114:0/1766319956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a3c2a52c0 0x562a3c2c67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:56.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.037+0000 7f27bbfff640 1 -- 172.21.15.114:0/1766319956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a3c2a52c0 msgr2=0x562a3c2c67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:56.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.037+0000 7f27bbfff640 1 --2- 172.21.15.114:0/1766319956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a3c2a52c0 0x562a3c2c67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:56.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.037+0000 7f27bbfff640 1 -- 172.21.15.114:0/1766319956 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562a3c2a49a0 msgr2=0x562a3c2a0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:56.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.037+0000 7f27bbfff640 1 --2- 172.21.15.114:0/1766319956 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562a3c2a49a0 0x562a3c2a0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:56.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.037+0000 7f27bbfff640 1 -- 172.21.15.114:0/1766319956 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a3c1a91c0 con 0x562a3c2afb60 2024-09-13T18:25:56.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.037+0000 7f27baffd640 1 --2- 172.21.15.114:0/1766319956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a3c2a52c0 0x562a3c2c67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:25:56.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.037+0000 7f27bbfff640 1 --2- 172.21.15.114:0/1766319956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a3c2afb60 0x562a3c2ac7d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f27ac044040 tx=0x7f27ac0044b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ebbef7d2de1eaf6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:56.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.038+0000 7f27a27fc640 1 -- 172.21.15.114:0/1766319956 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27ac04aa80 con 0x562a3c2afb60 2024-09-13T18:25:56.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.038+0000 7f27a27fc640 1 -- 172.21.15.114:0/1766319956 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f27ac0050b0 con 0x562a3c2afb60 2024-09-13T18:25:56.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.039+0000 7f27c4a11740 1 -- 172.21.15.114:0/1766319956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a3c2afb60 msgr2=0x562a3c2ac7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:56.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.039+0000 7f27c4a11740 1 --2- 172.21.15.114:0/1766319956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a3c2afb60 0x562a3c2ac7d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f27ac044040 tx=0x7f27ac0044b0 comp rx=0 tx=0).stop 2024-09-13T18:25:56.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.039+0000 7f27c4a11740 1 -- 172.21.15.114:0/1766319956 shutdown_connections 2024-09-13T18:25:56.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.039+0000 7f27c4a11740 1 --2- 172.21.15.114:0/1766319956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a3c2afb60 0x562a3c2ac7d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:56.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.039+0000 7f27c4a11740 1 --2- 172.21.15.114:0/1766319956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a3c2a52c0 0x562a3c2c67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:56.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.040+0000 7f27c4a11740 1 --2- 172.21.15.114:0/1766319956 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562a3c2a49a0 0x562a3c2a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:56.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.040+0000 7f27c4a11740 1 -- 172.21.15.114:0/1766319956 >> 172.21.15.114:0/1766319956 conn(0x562a3c2a0f30 msgr2=0x562a3c2a7b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:56.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.040+0000 7f27c4a11740 1 -- 172.21.15.114:0/1766319956 shutdown_connections 2024-09-13T18:25:56.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.041+0000 7f27c4a11740 1 -- 172.21.15.114:0/1766319956 wait complete. 2024-09-13T18:25:56.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.041+0000 7f27c4a11740 1 -- start start 2024-09-13T18:25:56.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.042+0000 7f27c4a11740 1 Processor -- start 2024-09-13T18:25:56.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.042+0000 7f27c4a11740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a3c2a49a0 0x562a3c2b6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:56.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.042+0000 7f27c4a11740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a3c2a52c0 0x562a3c2a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:56.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.042+0000 7f27c4a11740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562a3c2afb60 0x562a3c2a97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:56.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.043+0000 7f27bb7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a3c2a49a0 0x562a3c2b6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:56.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.043+0000 7f27bb7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a3c2a49a0 0x562a3c2b6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:38072/0 (socket says 172.21.15.114:38072) 2024-09-13T18:25:56.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.043+0000 7f27baffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a3c2a52c0 0x562a3c2a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:56.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.043+0000 7f27baffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a3c2a52c0 0x562a3c2a9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:52928/0 (socket says 172.21.15.114:52928) 2024-09-13T18:25:56.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.043+0000 7f27bb7fe640 1 -- 172.21.15.114:0/2894633557 learned_addr learned my addr 172.21.15.114:0/2894633557 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:56.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.043+0000 7f27c4a11740 1 -- 172.21.15.114:0/2894633557 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a3c2a9ec0 con 0x562a3c2afb60 2024-09-13T18:25:56.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.043+0000 7f27c4a11740 1 -- 172.21.15.114:0/2894633557 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a3c2aa030 con 0x562a3c2a52c0 2024-09-13T18:25:56.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.043+0000 7f27c4a11740 1 -- 172.21.15.114:0/2894633557 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a3c315f30 con 0x562a3c2a49a0 2024-09-13T18:25:56.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.043+0000 7f27c4a11740 10 client.0 _ll_register_callbacks cb 0x7f27ac005000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:25:56.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.044+0000 7f27baffd640 1 -- 172.21.15.114:0/2894633557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a3c2a49a0 msgr2=0x562a3c2b6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:56.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.044+0000 7f27baffd640 1 --2- 172.21.15.114:0/2894633557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a3c2a49a0 0x562a3c2b6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:56.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.044+0000 7f27baffd640 1 -- 172.21.15.114:0/2894633557 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562a3c2afb60 msgr2=0x562a3c2a97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:56.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.044+0000 7f27baffd640 1 --2- 172.21.15.114:0/2894633557 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562a3c2afb60 0x562a3c2a97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:56.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.044+0000 7f27baffd640 1 -- 172.21.15.114:0/2894633557 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562a3c1c4970 con 0x562a3c2a52c0 2024-09-13T18:25:56.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.048+0000 7f27baffd640 1 --2- 172.21.15.114:0/2894633557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a3c2a52c0 0x562a3c2a9280 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x562a3c2ac080 tx=0x7f27b00056f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e185b0959a05452e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:56.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.048+0000 7f279affd640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:25:56.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.048+0000 7f279affd640 1 -- 172.21.15.114:0/2894633557 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b0004680 con 0x562a3c2a52c0 2024-09-13T18:25:56.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.048+0000 7f279affd640 1 -- 172.21.15.114:0/2894633557 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f27b0004ab0 con 0x562a3c2a52c0 2024-09-13T18:25:56.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.049+0000 7f27c4a11740 1 -- 172.21.15.114:0/2894633557 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f27b0005020 con 0x562a3c2a52c0 2024-09-13T18:25:56.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.050+0000 7f279affd640 1 -- 172.21.15.114:0/2894633557 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27b0004c70 con 0x562a3c2a52c0 2024-09-13T18:25:56.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.051+0000 7f27c4a11740 10 client.26225 fetch_fsmap learned FSMap version 32 2024-09-13T18:25:56.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.051+0000 7f27c4a11740 1 -- 172.21.15.114:0/2894633557 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x562a3c207360 con 0x562a3c2a52c0 2024-09-13T18:25:56.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.054+0000 7f279affd640 1 -- 172.21.15.114:0/2894633557 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f27b0005020 con 0x562a3c2a52c0 2024-09-13T18:25:56.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.054+0000 7f279affd640 1 -- 172.21.15.114:0/2894633557 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f27b0023520 con 0x562a3c2a52c0 2024-09-13T18:25:56.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.054+0000 7f27c4a11740 10 client.26225 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:25:56.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.054+0000 7f27c4a11740 10 client.26225 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:25:56.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.054+0000 7f27c4a11740 20 client.26225 populate_metadata read hostname 'smithi114' 2024-09-13T18:25:56.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.054+0000 7f27c4a11740 1 --2- 172.21.15.114:0/2894633557 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x562a3c2bf9f0 0x562a3c3195b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:56.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.054+0000 7f27c4a11740 4 client.26225 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-13T18:25:56.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.054+0000 7f27c4a11740 1 -- 172.21.15.114:0/2894633557 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x562a3bf60eb0 con 0x562a3c2bf9f0 2024-09-13T18:25:56.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.055+0000 7f27bb7fe640 1 --2- 172.21.15.114:0/2894633557 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x562a3c2bf9f0 0x562a3c3195b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:56.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.055+0000 7f27bb7fe640 1 --2- 172.21.15.114:0/2894633557 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x562a3c2bf9f0 0x562a3c3195b0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ed61189b4de043 server_cookie=639f4f0bd1816a41 in_seq=0 out_seq=0 2024-09-13T18:25:56.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.055+0000 7f279affd640 10 client.26225 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:25:56.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.057+0000 7f279affd640 1 -- 172.21.15.114:0/2894633557 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x562a3bf60eb0 con 0x562a3c2bf9f0 2024-09-13T18:25:56.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.057+0000 7f279affd640 10 client.26225 handle_command_reply: tid=0 2024-09-13T18:25:56.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.057+0000 7f27c4a11740 1 client.26225 shutdown 2024-09-13T18:25:56.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.059+0000 7f27c4a11740 1 -- 172.21.15.114:0/2894633557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a3c2a52c0 msgr2=0x562a3c2a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:56.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.059+0000 7f27c4a11740 1 --2- 172.21.15.114:0/2894633557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a3c2a52c0 0x562a3c2a9280 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x562a3c2ac080 tx=0x7f27b00056f0 comp rx=0 tx=0).stop 2024-09-13T18:25:56.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.059+0000 7f27c4a11740 1 -- 172.21.15.114:0/2894633557 shutdown_connections 2024-09-13T18:25:56.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.059+0000 7f27c4a11740 1 --2- 172.21.15.114:0/2894633557 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x562a3c2bf9f0 0x562a3c3195b0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:56.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.059+0000 7f27c4a11740 1 --2- 172.21.15.114:0/2894633557 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562a3c2afb60 0x562a3c2a97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:56.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.059+0000 7f27c4a11740 1 --2- 172.21.15.114:0/2894633557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a3c2a52c0 0x562a3c2a9280 secure :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0x562a3c2ac080 tx=0x7f27b00056f0 comp rx=0 tx=0).stop 2024-09-13T18:25:56.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.059+0000 7f27c4a11740 1 --2- 172.21.15.114:0/2894633557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562a3c2a49a0 0x562a3c2b6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:56.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.059+0000 7f279affd640 0 client.26225 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:25:56.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.059+0000 7f27c4a11740 1 -- 172.21.15.114:0/2894633557 >> 172.21.15.114:0/2894633557 conn(0x562a3c2a0f30 msgr2=0x562a3c2ac7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:56.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.063+0000 7f27c4a11740 1 -- 172.21.15.114:0/2894633557 shutdown_connections 2024-09-13T18:25:56.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.063+0000 7f27c4a11740 1 -- 172.21.15.114:0/2894633557 wait complete. 2024-09-13T18:25:56.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.067+0000 7f27c4a11740 20 client.26225 trim_cache size 0 max 16384 2024-09-13T18:25:56.072 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:25:56.072 INFO:teuthology.orchestra.run.smithi114.stdout: "return_code": 0, 2024-09-13T18:25:56.072 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_tag": "944bda73-19dc-465f-a26c-5c13a1045f31", 2024-09-13T18:25:56.072 INFO:teuthology.orchestra.run.smithi114.stdout: "mode": "asynchronous" 2024-09-13T18:25:56.072 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:25:56.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.071+0000 7f27c1c12640 1 -- 172.21.15.114:0/2667283677 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f27a40a1340 msgr2=0x7f27a40a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:56.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.071+0000 7f27c1c12640 1 --2- 172.21.15.114:0/2667283677 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f27a40a1340 0x7f27a40a3800 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f27ac006010 tx=0x7f27ac02a920 comp rx=0 tx=0).stop 2024-09-13T18:25:56.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.071+0000 7f27c1c12640 1 -- 172.21.15.114:0/2667283677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27bc1074b0 msgr2=0x7f27bc1a4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:56.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.071+0000 7f27c1c12640 1 --2- 172.21.15.114:0/2667283677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27bc1074b0 0x7f27bc1a4e80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f27b400ba50 tx=0x7f27b400bf10 comp rx=0 tx=0).stop 2024-09-13T18:25:56.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.073+0000 7f27c1c12640 1 -- 172.21.15.114:0/2667283677 shutdown_connections 2024-09-13T18:25:56.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.073+0000 7f27c1c12640 1 --2- 172.21.15.114:0/2667283677 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f27a40a1340 0x7f27a40a3800 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:56.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.074+0000 7f27c1c12640 1 --2- 172.21.15.114:0/2667283677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27bc119400 0x7f27bc1a97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:56.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.074+0000 7f27c1c12640 1 --2- 172.21.15.114:0/2667283677 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f27bc107890 0x7f27bc1a53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:56.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.074+0000 7f27c1c12640 1 --2- 172.21.15.114:0/2667283677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27bc1074b0 0x7f27bc1a4e80 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:56.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.074+0000 7f27c1c12640 1 -- 172.21.15.114:0/2667283677 >> 172.21.15.114:0/2667283677 conn(0x7f27bc06af20 msgr2=0x7f27bc119a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:56.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.076+0000 7f27c1c12640 1 -- 172.21.15.114:0/2667283677 shutdown_connections 2024-09-13T18:25:56.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.076+0000 7f27c1c12640 1 -- 172.21.15.114:0/2667283677 wait complete. 2024-09-13T18:25:56.101 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T18:25:56.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.347+0000 7f94124c5640 1 -- 172.21.15.114:0/2194192012 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f940c103fb0 msgr2=0x7f940c10bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:56.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.347+0000 7f94124c5640 1 --2- 172.21.15.114:0/2194192012 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f940c103fb0 0x7f940c10bb00 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f93f4009930 tx=0x7f93f402a480 comp rx=0 tx=0).stop 2024-09-13T18:25:56.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.347+0000 7f94124c5640 1 -- 172.21.15.114:0/2194192012 shutdown_connections 2024-09-13T18:25:56.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.347+0000 7f94124c5640 1 --2- 172.21.15.114:0/2194192012 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f940c103fb0 0x7f940c10bb00 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:56.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.347+0000 7f94124c5640 1 --2- 172.21.15.114:0/2194192012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f940c0ff810 0x7f940c103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:56.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.347+0000 7f94124c5640 1 --2- 172.21.15.114:0/2194192012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f940c0fee60 0x7f940c0ff240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:56.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.347+0000 7f94124c5640 1 -- 172.21.15.114:0/2194192012 >> 172.21.15.114:0/2194192012 conn(0x7f940c075c60 msgr2=0x7f940c0fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:56.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.349+0000 7f94124c5640 1 -- 172.21.15.114:0/2194192012 shutdown_connections 2024-09-13T18:25:56.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.350+0000 7f94124c5640 1 -- 172.21.15.114:0/2194192012 wait complete. 2024-09-13T18:25:56.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.351+0000 7f94124c5640 1 Processor -- start 2024-09-13T18:25:56.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.351+0000 7f94124c5640 1 -- start start 2024-09-13T18:25:56.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.352+0000 7f94124c5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f940c0fee60 0x7f940c10b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:56.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.352+0000 7f94124c5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f940c103fb0 0x7f940c1087e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:56.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.352+0000 7f94124c5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f940c108d20 0x7f940c1091d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:56.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.352+0000 7f94124c5640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f940c10bea0 con 0x7f940c108d20 2024-09-13T18:25:56.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.352+0000 7f94124c5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f940c19e6c0 con 0x7f940c103fb0 2024-09-13T18:25:56.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.352+0000 7f94124c5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f940c19e850 con 0x7f940c0fee60 2024-09-13T18:25:56.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.352+0000 7f9410a62640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f940c108d20 0x7f940c1091d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:56.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.352+0000 7f940bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f940c0fee60 0x7f940c10b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:56.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.352+0000 7f9410a62640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f940c108d20 0x7f940c1091d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36518/0 (socket says 172.21.15.114:36518) 2024-09-13T18:25:56.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.352+0000 7f9410a62640 1 -- 172.21.15.114:0/24356554 learned_addr learned my addr 172.21.15.114:0/24356554 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:56.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.352+0000 7f940b7fe640 1 --2- 172.21.15.114:0/24356554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f940c103fb0 0x7f940c1087e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:56.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.353+0000 7f940b7fe640 1 -- 172.21.15.114:0/24356554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f940c0fee60 msgr2=0x7f940c10b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:56.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.353+0000 7f940b7fe640 1 --2- 172.21.15.114:0/24356554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f940c0fee60 0x7f940c10b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:56.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.353+0000 7f940b7fe640 1 -- 172.21.15.114:0/24356554 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f940c108d20 msgr2=0x7f940c1091d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:56.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.353+0000 7f940b7fe640 1 --2- 172.21.15.114:0/24356554 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f940c108d20 0x7f940c1091d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:56.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.353+0000 7f940b7fe640 1 -- 172.21.15.114:0/24356554 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f4009560 con 0x7f940c103fb0 2024-09-13T18:25:56.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.353+0000 7f940bfff640 1 --2- 172.21.15.114:0/24356554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f940c0fee60 0x7f940c10b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:25:56.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.353+0000 7f9410a62640 1 --2- 172.21.15.114:0/24356554 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f940c108d20 0x7f940c1091d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:25:56.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.353+0000 7f940b7fe640 1 --2- 172.21.15.114:0/24356554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f940c103fb0 0x7f940c1087e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f93fc00bdf0 tx=0x7f93fc009f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:56.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.354+0000 7f94097fa640 1 -- 172.21.15.114:0/24356554 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93fc014440 con 0x7f940c103fb0 2024-09-13T18:25:56.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.354+0000 7f94097fa640 1 -- 172.21.15.114:0/24356554 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f93fc014a30 con 0x7f940c103fb0 2024-09-13T18:25:56.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.354+0000 7f94097fa640 1 -- 172.21.15.114:0/24356554 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93fc0128d0 con 0x7f940c103fb0 2024-09-13T18:25:56.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.354+0000 7f94124c5640 1 -- 172.21.15.114:0/24356554 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f940c19eb30 con 0x7f940c103fb0 2024-09-13T18:25:56.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.354+0000 7f94124c5640 1 -- 172.21.15.114:0/24356554 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f940c19eff0 con 0x7f940c103fb0 2024-09-13T18:25:56.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.357+0000 7f94097fa640 1 -- 172.21.15.114:0/24356554 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f93fc014ba0 con 0x7f940c103fb0 2024-09-13T18:25:56.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.358+0000 7f94151e2740 1 Processor -- start 2024-09-13T18:25:56.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.358+0000 7f94151e2740 1 -- start start 2024-09-13T18:25:56.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.358+0000 7f94151e2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562d77c0e10 0x5562d77c11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:56.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.358+0000 7f94151e2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5562d77c1730 0x5562d77cba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:56.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.358+0000 7f94151e2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5562d77cbfd0 0x5562d77ce3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:56.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.358+0000 7f94151e2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562d772bf20 con 0x5562d77cbfd0 2024-09-13T18:25:56.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.358+0000 7f94151e2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562d77296b0 con 0x5562d77c1730 2024-09-13T18:25:56.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.358+0000 7f94151e2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562d7724fa0 con 0x5562d77c0e10 2024-09-13T18:25:56.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.358+0000 7f9410a62640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562d77c0e10 0x5562d77c11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:56.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.358+0000 7f940bfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5562d77c1730 0x5562d77cba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:56.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.358+0000 7f940bfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5562d77c1730 0x5562d77cba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:52950/0 (socket says 172.21.15.114:52950) 2024-09-13T18:25:56.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.358+0000 7f940bfff640 1 -- 172.21.15.114:0/17009395 learned_addr learned my addr 172.21.15.114:0/17009395 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:56.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.358+0000 7f940b7fe640 1 --2- 172.21.15.114:0/17009395 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5562d77cbfd0 0x5562d77ce3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:56.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.359+0000 7f94097fa640 1 --2- 172.21.15.114:0/24356554 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f93f80a1390 0x7f93f80a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:56.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.359+0000 7f9410a62640 1 -- 172.21.15.114:0/17009395 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5562d77c1730 msgr2=0x5562d77cba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:56.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.359+0000 7f9410a62640 1 --2- 172.21.15.114:0/17009395 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5562d77c1730 0x5562d77cba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:56.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.359+0000 7f9410a62640 1 -- 172.21.15.114:0/17009395 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5562d77cbfd0 msgr2=0x5562d77ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:56.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.359+0000 7f9410a62640 1 --2- 172.21.15.114:0/17009395 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5562d77cbfd0 0x5562d77ce3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:56.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.359+0000 7f9410a62640 1 -- 172.21.15.114:0/17009395 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5562d76df840 con 0x5562d77c0e10 2024-09-13T18:25:56.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.359+0000 7f94097fa640 1 -- 172.21.15.114:0/24356554 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f93fc0cc9c0 con 0x7f940c103fb0 2024-09-13T18:25:56.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.359+0000 7f9410a62640 1 --2- 172.21.15.114:0/24356554 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f93f80a1390 0x7f93f80a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:56.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.360+0000 7f9410a62640 1 --2- 172.21.15.114:0/17009395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562d77c0e10 0x5562d77c11f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f93f4006010 tx=0x7f93f4031de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee92f7f22b6048ce server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:56.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.360+0000 7f9410a62640 1 --2- 172.21.15.114:0/24356554 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f93f80a1390 0x7f93f80a3850 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f93f404c000 tx=0x7f93f404f000 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:56.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.360+0000 7f94027fc640 1 -- 172.21.15.114:0/17009395 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f404d070 con 0x5562d77c0e10 2024-09-13T18:25:56.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.360+0000 7f94027fc640 1 -- 172.21.15.114:0/17009395 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f93f4041070 con 0x5562d77c0e10 2024-09-13T18:25:56.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.360+0000 7f94027fc640 1 -- 172.21.15.114:0/17009395 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f403d820 con 0x5562d77c0e10 2024-09-13T18:25:56.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.361+0000 7f94151e2740 1 -- 172.21.15.114:0/17009395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562d77c0e10 msgr2=0x5562d77c11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:56.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.361+0000 7f94151e2740 1 --2- 172.21.15.114:0/17009395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562d77c0e10 0x5562d77c11f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f93f4006010 tx=0x7f93f4031de0 comp rx=0 tx=0).stop 2024-09-13T18:25:56.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.361+0000 7f94151e2740 1 -- 172.21.15.114:0/17009395 shutdown_connections 2024-09-13T18:25:56.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.361+0000 7f94151e2740 1 --2- 172.21.15.114:0/17009395 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5562d77cbfd0 0x5562d77ce3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:56.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.361+0000 7f94151e2740 1 --2- 172.21.15.114:0/17009395 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5562d77c1730 0x5562d77cba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:56.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.361+0000 7f94151e2740 1 --2- 172.21.15.114:0/17009395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562d77c0e10 0x5562d77c11f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:56.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.361+0000 7f94151e2740 1 -- 172.21.15.114:0/17009395 >> 172.21.15.114:0/17009395 conn(0x5562d77bd380 msgr2=0x5562d77bd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:56.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.361+0000 7f94151e2740 1 -- 172.21.15.114:0/17009395 shutdown_connections 2024-09-13T18:25:56.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.361+0000 7f94151e2740 1 -- 172.21.15.114:0/17009395 wait complete. 2024-09-13T18:25:56.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.361+0000 7f94151e2740 1 -- start start 2024-09-13T18:25:56.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.362+0000 7f94151e2740 1 Processor -- start 2024-09-13T18:25:56.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.362+0000 7f94151e2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562d77c0e10 0x5562d77cba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:56.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.362+0000 7f94151e2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5562d77c1730 0x5562d77c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:56.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.362+0000 7f94151e2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5562d77cbfd0 0x5562d77c9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:56.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.362+0000 7f94151e2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562d76c62a0 con 0x5562d77c1730 2024-09-13T18:25:56.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.362+0000 7f94151e2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562d73fd430 con 0x5562d77cbfd0 2024-09-13T18:25:56.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.362+0000 7f94151e2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562d76a8680 con 0x5562d77c0e10 2024-09-13T18:25:56.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.363+0000 7f94151e2740 10 client.0 _ll_register_callbacks cb 0x5562d771e610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:25:56.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.363+0000 7f940bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562d77c0e10 0x5562d77cba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:56.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.363+0000 7f940bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562d77c0e10 0x5562d77cba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:38092/0 (socket says 172.21.15.114:38092) 2024-09-13T18:25:56.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.363+0000 7f940b7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5562d77c1730 0x5562d77c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:56.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.363+0000 7f940bfff640 1 -- 172.21.15.114:0/3936905260 learned_addr learned my addr 172.21.15.114:0/3936905260 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:56.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.363+0000 7f940b7fe640 1 -- 172.21.15.114:0/3936905260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562d77c0e10 msgr2=0x5562d77cba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:56.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.363+0000 7f940b7fe640 1 --2- 172.21.15.114:0/3936905260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562d77c0e10 0x5562d77cba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:56.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.363+0000 7f940b7fe640 1 -- 172.21.15.114:0/3936905260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5562d77cbfd0 msgr2=0x5562d77c9400 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:25:56.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.363+0000 7f940b7fe640 1 --2- 172.21.15.114:0/3936905260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5562d77cbfd0 0x5562d77c9400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:56.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.363+0000 7f940b7fe640 1 -- 172.21.15.114:0/3936905260 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f940c0675a0 con 0x5562d77c1730 2024-09-13T18:25:56.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.364+0000 7f940b7fe640 1 --2- 172.21.15.114:0/3936905260 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5562d77c1730 0x5562d77c8cc0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f93fc0c9ed0 tx=0x7f93fc014f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baabc51f5b88e646 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:56.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.364+0000 7f93eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:25:56.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.364+0000 7f93eb7fe640 1 -- 172.21.15.114:0/3936905260 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93fc099da0 con 0x5562d77c1730 2024-09-13T18:25:56.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.364+0000 7f93eb7fe640 1 -- 172.21.15.114:0/3936905260 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f93fc0a1d10 con 0x5562d77c1730 2024-09-13T18:25:56.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.364+0000 7f94151e2740 1 -- 172.21.15.114:0/3936905260 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5562d74184e0 con 0x5562d77c1730 2024-09-13T18:25:56.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.365+0000 7f93eb7fe640 1 -- 172.21.15.114:0/3936905260 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93fc099940 con 0x5562d77c1730 2024-09-13T18:25:56.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.365+0000 7f94151e2740 10 client.16884 fetch_fsmap learned FSMap version 32 2024-09-13T18:25:56.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.365+0000 7f94151e2740 1 -- 172.21.15.114:0/3936905260 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5562d7723620 con 0x5562d77c1730 2024-09-13T18:25:56.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.365+0000 7f93eb7fe640 1 -- 172.21.15.114:0/3936905260 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f93fc09d6c0 con 0x5562d77c1730 2024-09-13T18:25:56.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.365+0000 7f93eb7fe640 1 -- 172.21.15.114:0/3936905260 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f93fc008030 con 0x5562d77c1730 2024-09-13T18:25:56.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.365+0000 7f94151e2740 10 client.16884 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:25:56.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.366+0000 7f94151e2740 10 client.16884 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:25:56.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.366+0000 7f94151e2740 20 client.16884 populate_metadata read hostname 'smithi114' 2024-09-13T18:25:56.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.366+0000 7f94151e2740 1 --2- 172.21.15.114:0/3936905260 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5562d77dbe00 0x5562d77de220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:56.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.366+0000 7f94151e2740 4 client.16884 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:25:56.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.366+0000 7f94151e2740 1 -- 172.21.15.114:0/3936905260 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5562d74184e0 con 0x5562d77dbe00 2024-09-13T18:25:56.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.368+0000 7f940bfff640 1 --2- 172.21.15.114:0/3936905260 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5562d77dbe00 0x5562d77de220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:56.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.369+0000 7f940bfff640 1 --2- 172.21.15.114:0/3936905260 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5562d77dbe00 0x5562d77de220 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4433fe7d5242d3b0 server_cookie=896fda6e771e07f6 in_seq=0 out_seq=0 2024-09-13T18:25:56.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.369+0000 7f93eb7fe640 10 client.16884 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:25:56.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.370+0000 7f93eb7fe640 1 -- 172.21.15.114:0/3936905260 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5562d74184e0 con 0x5562d77dbe00 2024-09-13T18:25:56.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.370+0000 7f93eb7fe640 10 client.16884 handle_command_reply: tid=0 2024-09-13T18:25:56.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.370+0000 7f94151e2740 1 client.16884 shutdown 2024-09-13T18:25:56.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.371+0000 7f94151e2740 1 -- 172.21.15.114:0/3936905260 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5562d77c1730 msgr2=0x5562d77c8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:56.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.372+0000 7f94151e2740 1 --2- 172.21.15.114:0/3936905260 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5562d77c1730 0x5562d77c8cc0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f93fc0c9ed0 tx=0x7f93fc014f40 comp rx=0 tx=0).stop 2024-09-13T18:25:56.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.372+0000 7f94151e2740 1 -- 172.21.15.114:0/3936905260 shutdown_connections 2024-09-13T18:25:56.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.372+0000 7f94151e2740 1 --2- 172.21.15.114:0/3936905260 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5562d77dbe00 0x5562d77de220 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:56.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.372+0000 7f94151e2740 1 --2- 172.21.15.114:0/3936905260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5562d77cbfd0 0x5562d77c9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:56.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.372+0000 7f94151e2740 1 --2- 172.21.15.114:0/3936905260 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5562d77c1730 0x5562d77c8cc0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:56.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.372+0000 7f93eb7fe640 0 client.16884 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:25:56.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.372+0000 7f94151e2740 1 --2- 172.21.15.114:0/3936905260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562d77c0e10 0x5562d77cba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:56.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.372+0000 7f94151e2740 1 -- 172.21.15.114:0/3936905260 >> 172.21.15.114:0/3936905260 conn(0x5562d77bd380 msgr2=0x5562d77c30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:56.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.373+0000 7f94151e2740 1 -- 172.21.15.114:0/3936905260 shutdown_connections 2024-09-13T18:25:56.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.374+0000 7f94151e2740 1 -- 172.21.15.114:0/3936905260 wait complete. 2024-09-13T18:25:56.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.374+0000 7f94151e2740 20 client.16884 trim_cache size 0 max 16384 2024-09-13T18:25:56.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.403+0000 7f94151e2740 1 Processor -- start 2024-09-13T18:25:56.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.403+0000 7f94151e2740 1 -- start start 2024-09-13T18:25:56.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.403+0000 7f94151e2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562d77c0e10 0x5562d77bcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:56.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.403+0000 7f94151e2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5562d77c1730 0x5562d77c8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:56.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.405+0000 7f940bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562d77c0e10 0x5562d77bcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:56.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.405+0000 7f940bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562d77c0e10 0x5562d77bcd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:38108/0 (socket says 172.21.15.114:38108) 2024-09-13T18:25:56.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.405+0000 7f940bfff640 1 -- 172.21.15.114:0/213763154 learned_addr learned my addr 172.21.15.114:0/213763154 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:56.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.405+0000 7f94151e2740 1 --2- 172.21.15.114:0/213763154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5562d77cbfd0 0x5562d77c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:56.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.405+0000 7f94151e2740 1 -- 172.21.15.114:0/213763154 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562d77c1dd0 con 0x5562d77c1730 2024-09-13T18:25:56.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.406+0000 7f9410a62640 1 --2- 172.21.15.114:0/213763154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5562d77cbfd0 0x5562d77c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:56.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.406+0000 7f94151e2740 1 -- 172.21.15.114:0/213763154 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562d77be070 con 0x5562d77cbfd0 2024-09-13T18:25:56.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.406+0000 7f94151e2740 1 -- 172.21.15.114:0/213763154 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562d77b7660 con 0x5562d77c0e10 2024-09-13T18:25:56.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.406+0000 7f940b7fe640 1 --2- 172.21.15.114:0/213763154 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5562d77c1730 0x5562d77c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:56.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.406+0000 7f940bfff640 1 -- 172.21.15.114:0/213763154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5562d77cbfd0 msgr2=0x5562d77c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:56.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.406+0000 7f940bfff640 1 --2- 172.21.15.114:0/213763154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5562d77cbfd0 0x5562d77c8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:56.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.406+0000 7f940bfff640 1 -- 172.21.15.114:0/213763154 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5562d77c1730 msgr2=0x5562d77c8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:56.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.406+0000 7f940bfff640 1 --2- 172.21.15.114:0/213763154 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5562d77c1730 0x5562d77c8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:56.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.406+0000 7f940bfff640 1 -- 172.21.15.114:0/213763154 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5562d76c5790 con 0x5562d77c0e10 2024-09-13T18:25:56.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.407+0000 7f940bfff640 1 --2- 172.21.15.114:0/213763154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562d77c0e10 0x5562d77bcd00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9404023700 tx=0x7f9404005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c09c8e714a2855b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:56.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.408+0000 7f9402ffd640 1 -- 172.21.15.114:0/213763154 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9404036030 con 0x5562d77c0e10 2024-09-13T18:25:56.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.408+0000 7f9402ffd640 1 -- 172.21.15.114:0/213763154 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5562d74184e0 con 0x5562d77c0e10 2024-09-13T18:25:56.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.408+0000 7f9402ffd640 1 -- 172.21.15.114:0/213763154 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9404028040 con 0x5562d77c0e10 2024-09-13T18:25:56.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.409+0000 7f94151e2740 1 -- 172.21.15.114:0/213763154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562d77c0e10 msgr2=0x5562d77bcd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:56.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.409+0000 7f94151e2740 1 --2- 172.21.15.114:0/213763154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562d77c0e10 0x5562d77bcd00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9404023700 tx=0x7f9404005f90 comp rx=0 tx=0).stop 2024-09-13T18:25:56.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.409+0000 7f94151e2740 1 -- 172.21.15.114:0/213763154 shutdown_connections 2024-09-13T18:25:56.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.409+0000 7f94151e2740 1 --2- 172.21.15.114:0/213763154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5562d77cbfd0 0x5562d77c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:56.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.409+0000 7f94151e2740 1 --2- 172.21.15.114:0/213763154 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5562d77c1730 0x5562d77c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:56.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.409+0000 7f94151e2740 1 --2- 172.21.15.114:0/213763154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562d77c0e10 0x5562d77bcd00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:56.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.409+0000 7f94151e2740 1 -- 172.21.15.114:0/213763154 >> 172.21.15.114:0/213763154 conn(0x5562d77bd380 msgr2=0x5562d77c3f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:56.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.409+0000 7f94151e2740 1 -- 172.21.15.114:0/213763154 shutdown_connections 2024-09-13T18:25:56.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.410+0000 7f94151e2740 1 -- 172.21.15.114:0/213763154 wait complete. 2024-09-13T18:25:56.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.410+0000 7f94151e2740 1 -- start start 2024-09-13T18:25:56.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.410+0000 7f94151e2740 1 Processor -- start 2024-09-13T18:25:56.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.411+0000 7f94151e2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562d77c0e10 0x5562d77d31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:56.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.411+0000 7f94151e2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5562d77c1730 0x5562d77c5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:56.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.411+0000 7f94151e2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5562d77cbfd0 0x5562d77c5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:56.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.411+0000 7f94151e2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562d77c6510 con 0x5562d77cbfd0 2024-09-13T18:25:56.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.411+0000 7f94151e2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562d77c6680 con 0x5562d77c1730 2024-09-13T18:25:56.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.411+0000 7f94151e2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562d78322e0 con 0x5562d77c0e10 2024-09-13T18:25:56.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.411+0000 7f94151e2740 10 client.0 _ll_register_callbacks cb 0x7f93fc0d2220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:25:56.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.411+0000 7f940b7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5562d77c1730 0x5562d77c5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:56.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.411+0000 7f9410a62640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5562d77cbfd0 0x5562d77c5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:56.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.411+0000 7f940b7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5562d77c1730 0x5562d77c5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:52992/0 (socket says 172.21.15.114:52992) 2024-09-13T18:25:56.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.411+0000 7f940b7fe640 1 -- 172.21.15.114:0/206139511 learned_addr learned my addr 172.21.15.114:0/206139511 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:25:56.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.411+0000 7f940bfff640 1 --2- 172.21.15.114:0/206139511 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562d77c0e10 0x5562d77d31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:56.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.412+0000 7f940b7fe640 1 -- 172.21.15.114:0/206139511 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562d77c0e10 msgr2=0x5562d77d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:56.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.412+0000 7f940b7fe640 1 --2- 172.21.15.114:0/206139511 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562d77c0e10 0x5562d77d31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:56.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.412+0000 7f940b7fe640 1 -- 172.21.15.114:0/206139511 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5562d77cbfd0 msgr2=0x5562d77c5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:25:56.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.412+0000 7f940b7fe640 1 --2- 172.21.15.114:0/206139511 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5562d77cbfd0 0x5562d77c5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:56.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.412+0000 7f940b7fe640 1 -- 172.21.15.114:0/206139511 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9404009df0 con 0x5562d77c1730 2024-09-13T18:25:56.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.412+0000 7f940bfff640 1 --2- 172.21.15.114:0/206139511 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562d77c0e10 0x5562d77d31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:25:56.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.412+0000 7f940b7fe640 1 --2- 172.21.15.114:0/206139511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5562d77c1730 0x5562d77c5820 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f93fc0cc170 tx=0x7f93fc0a1880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85c6d0f568681a95 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:25:56.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.412+0000 7f93eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:25:56.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.412+0000 7f93eb7fe640 1 -- 172.21.15.114:0/206139511 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93fc0a1d10 con 0x5562d77c1730 2024-09-13T18:25:56.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.412+0000 7f93eb7fe640 1 -- 172.21.15.114:0/206139511 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f93fc0a1a50 con 0x5562d77c1730 2024-09-13T18:25:56.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.412+0000 7f94151e2740 1 -- 172.21.15.114:0/206139511 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f93f4041070 con 0x5562d77c1730 2024-09-13T18:25:56.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.412+0000 7f93eb7fe640 1 -- 172.21.15.114:0/206139511 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93fc09d6c0 con 0x5562d77c1730 2024-09-13T18:25:56.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.413+0000 7f94151e2740 10 client.26240 fetch_fsmap learned FSMap version 32 2024-09-13T18:25:56.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.413+0000 7f94151e2740 1 -- 172.21.15.114:0/206139511 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5562d76eaea0 con 0x5562d77c1730 2024-09-13T18:25:56.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.413+0000 7f93eb7fe640 1 -- 172.21.15.114:0/206139511 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f93f4041070 con 0x5562d77c1730 2024-09-13T18:25:56.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.413+0000 7f93eb7fe640 1 -- 172.21.15.114:0/206139511 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f93fc0988a0 con 0x5562d77c1730 2024-09-13T18:25:56.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.413+0000 7f94151e2740 10 client.26240 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:25:56.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.413+0000 7f94151e2740 10 client.26240 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:25:56.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.413+0000 7f94151e2740 20 client.26240 populate_metadata read hostname 'smithi114' 2024-09-13T18:25:56.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.413+0000 7f94151e2740 1 --2- 172.21.15.114:0/206139511 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5562d77dbe00 0x5562d7835840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:25:56.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.413+0000 7f94151e2740 4 client.26240 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T18:25:56.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.413+0000 7f94151e2740 1 -- 172.21.15.114:0/206139511 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5562d74297e0 con 0x5562d77dbe00 2024-09-13T18:25:56.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.413+0000 7f940bfff640 1 --2- 172.21.15.114:0/206139511 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5562d77dbe00 0x5562d7835840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:25:56.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.414+0000 7f93eb7fe640 10 client.26240 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:25:56.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.414+0000 7f940bfff640 1 --2- 172.21.15.114:0/206139511 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5562d77dbe00 0x5562d7835840 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c4fd8c78b052c0c server_cookie=35ebddf848c50df1 in_seq=0 out_seq=0 2024-09-13T18:25:56.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.414+0000 7f93eb7fe640 1 -- 172.21.15.114:0/206139511 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5562d74297e0 con 0x5562d77dbe00 2024-09-13T18:25:56.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.414+0000 7f93eb7fe640 10 client.26240 handle_command_reply: tid=0 2024-09-13T18:25:56.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.414+0000 7f94151e2740 1 client.26240 shutdown 2024-09-13T18:25:56.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.414+0000 7f94151e2740 1 -- 172.21.15.114:0/206139511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5562d77c1730 msgr2=0x5562d77c5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:56.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.414+0000 7f94151e2740 1 --2- 172.21.15.114:0/206139511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5562d77c1730 0x5562d77c5820 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f93fc0cc170 tx=0x7f93fc0a1880 comp rx=0 tx=0).stop 2024-09-13T18:25:56.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.415+0000 7f94151e2740 1 -- 172.21.15.114:0/206139511 shutdown_connections 2024-09-13T18:25:56.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.415+0000 7f94151e2740 1 --2- 172.21.15.114:0/206139511 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5562d77dbe00 0x5562d7835840 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:56.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.415+0000 7f94151e2740 1 --2- 172.21.15.114:0/206139511 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5562d77cbfd0 0x5562d77c5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:56.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.415+0000 7f94151e2740 1 --2- 172.21.15.114:0/206139511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5562d77c1730 0x5562d77c5820 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:56.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.415+0000 7f94151e2740 1 --2- 172.21.15.114:0/206139511 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562d77c0e10 0x5562d77d31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:56.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.415+0000 7f94151e2740 1 -- 172.21.15.114:0/206139511 >> 172.21.15.114:0/206139511 conn(0x5562d77bd380 msgr2=0x5562d77c8560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:56.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.415+0000 7f93eb7fe640 0 client.26240 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:25:56.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.415+0000 7f94151e2740 1 -- 172.21.15.114:0/206139511 shutdown_connections 2024-09-13T18:25:56.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.415+0000 7f94151e2740 1 -- 172.21.15.114:0/206139511 wait complete. 2024-09-13T18:25:56.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.415+0000 7f94151e2740 20 client.26240 trim_cache size 0 max 16384 2024-09-13T18:25:56.423 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:25:56.423 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-13T18:25:56.423 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": { 2024-09-13T18:25:56.423 INFO:teuthology.orchestra.run.smithi114.stdout: "944bda73-19dc-465f-a26c-5c13a1045f31": { 2024-09-13T18:25:56.424 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-13T18:25:56.424 INFO:teuthology.orchestra.run.smithi114.stdout: "tag": "944bda73-19dc-465f-a26c-5c13a1045f31", 2024-09-13T18:25:56.424 INFO:teuthology.orchestra.run.smithi114.stdout: "options": "recursive,force" 2024-09-13T18:25:56.424 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:25:56.424 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:25:56.424 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:25:56.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.416+0000 7f94124c5640 1 -- 172.21.15.114:0/24356554 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f93f80a1390 msgr2=0x7f93f80a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:56.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.416+0000 7f94124c5640 1 --2- 172.21.15.114:0/24356554 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f93f80a1390 0x7f93f80a3850 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f93f404c000 tx=0x7f93f404f000 comp rx=0 tx=0).stop 2024-09-13T18:25:56.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.416+0000 7f94124c5640 1 -- 172.21.15.114:0/24356554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f940c103fb0 msgr2=0x7f940c1087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:25:56.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.416+0000 7f94124c5640 1 --2- 172.21.15.114:0/24356554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f940c103fb0 0x7f940c1087e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f93fc00bdf0 tx=0x7f93fc009f70 comp rx=0 tx=0).stop 2024-09-13T18:25:56.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.416+0000 7f94124c5640 1 -- 172.21.15.114:0/24356554 shutdown_connections 2024-09-13T18:25:56.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.416+0000 7f94124c5640 1 --2- 172.21.15.114:0/24356554 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f93f80a1390 0x7f93f80a3850 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:56.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.416+0000 7f94124c5640 1 --2- 172.21.15.114:0/24356554 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f940c108d20 0x7f940c1091d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:56.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.416+0000 7f94124c5640 1 --2- 172.21.15.114:0/24356554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f940c103fb0 0x7f940c1087e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:56.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.416+0000 7f94124c5640 1 --2- 172.21.15.114:0/24356554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f940c0fee60 0x7f940c10b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:25:56.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.416+0000 7f94124c5640 1 -- 172.21.15.114:0/24356554 >> 172.21.15.114:0/24356554 conn(0x7f940c075c60 msgr2=0x7f940c102610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:25:56.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.416+0000 7f94124c5640 1 -- 172.21.15.114:0/24356554 shutdown_connections 2024-09-13T18:25:56.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:25:56.416+0000 7f94124c5640 1 -- 172.21.15.114:0/24356554 wait complete. 2024-09-13T18:25:56.439 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:56 smithi114 ceph-mon[30897]: pgmap v107: 129 pgs: 129 active+clean; 13 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 50 MiB/s wr, 800 op/s 2024-09-13T18:25:56.440 INFO:tasks.cephfs.filesystem:scrub status for tag:944bda73-19dc-465f-a26c-5c13a1045f31 - {'path': '/', 'tag': '944bda73-19dc-465f-a26c-5c13a1045f31', 'options': 'recursive,force'} 2024-09-13T18:25:56.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:56 smithi134 ceph-mon[34283]: pgmap v107: 129 pgs: 129 active+clean; 13 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 50 MiB/s wr, 800 op/s 2024-09-13T18:25:56.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:56 smithi204 ceph-mon[34777]: pgmap v107: 129 pgs: 129 active+clean; 13 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 50 MiB/s wr, 800 op/s 2024-09-13T18:25:57.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:57 smithi114 ceph-mon[30897]: scrub queued for path: / 2024-09-13T18:25:57.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:57 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:25:57.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:57 smithi114 ceph-mon[30897]: scrub summary: active paths [/] 2024-09-13T18:25:57.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:57 smithi114 ceph-mon[30897]: scrub summary: active paths [/] 2024-09-13T18:25:57.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:57 smithi114 ceph-mon[30897]: scrub summary: active paths [/] 2024-09-13T18:25:57.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:57 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:25:57.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:57 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:25:57.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:57 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:25:57.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:57 smithi134 ceph-mon[34283]: scrub queued for path: / 2024-09-13T18:25:57.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:57 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:25:57.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:57 smithi134 ceph-mon[34283]: scrub summary: active paths [/] 2024-09-13T18:25:57.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:57 smithi134 ceph-mon[34283]: scrub summary: active paths [/] 2024-09-13T18:25:57.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:57 smithi134 ceph-mon[34283]: scrub summary: active paths [/] 2024-09-13T18:25:57.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:57 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:25:57.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:57 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:25:57.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:57 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:25:57.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:57 smithi204 ceph-mon[34777]: scrub queued for path: / 2024-09-13T18:25:57.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:57 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:25:57.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:57 smithi204 ceph-mon[34777]: scrub summary: active paths [/] 2024-09-13T18:25:57.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:57 smithi204 ceph-mon[34777]: scrub summary: active paths [/] 2024-09-13T18:25:57.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:57 smithi204 ceph-mon[34777]: scrub summary: active paths [/] 2024-09-13T18:25:57.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:57 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:25:57.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:57 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:25:57.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:57 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:25:58.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:25:58 smithi114 ceph-mon[30897]: pgmap v108: 129 pgs: 129 active+clean; 13 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 46 MiB/s wr, 751 op/s 2024-09-13T18:25:58.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:25:58 smithi134 ceph-mon[34283]: pgmap v108: 129 pgs: 129 active+clean; 13 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 46 MiB/s wr, 751 op/s 2024-09-13T18:25:58.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:25:58 smithi204 ceph-mon[34777]: pgmap v108: 129 pgs: 129 active+clean; 13 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 46 MiB/s wr, 751 op/s 2024-09-13T18:26:00.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:26:00 smithi114 ceph-mon[30897]: pgmap v109: 129 pgs: 129 active+clean; 13 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 70 MiB/s wr, 1.13k op/s 2024-09-13T18:26:00.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:26:00 smithi134 ceph-mon[34283]: pgmap v109: 129 pgs: 129 active+clean; 13 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 70 MiB/s wr, 1.13k op/s 2024-09-13T18:26:00.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:26:00 smithi204 ceph-mon[34777]: pgmap v109: 129 pgs: 129 active+clean; 13 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 70 MiB/s wr, 1.13k op/s 2024-09-13T18:26:02.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:26:02 smithi114 ceph-mon[30897]: pgmap v110: 129 pgs: 129 active+clean; 13 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 46 MiB/s wr, 741 op/s 2024-09-13T18:26:02.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:26:02 smithi134 ceph-mon[34283]: pgmap v110: 129 pgs: 129 active+clean; 13 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 46 MiB/s wr, 741 op/s 2024-09-13T18:26:02.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:26:02 smithi204 ceph-mon[34777]: pgmap v110: 129 pgs: 129 active+clean; 13 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 46 MiB/s wr, 741 op/s 2024-09-13T18:26:04.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:26:04 smithi114 ceph-mon[30897]: pgmap v111: 129 pgs: 129 active+clean; 13 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 62 MiB/s wr, 1.00k op/s 2024-09-13T18:26:04.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:26:04 smithi134 ceph-mon[34283]: pgmap v111: 129 pgs: 129 active+clean; 13 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 62 MiB/s wr, 1.00k op/s 2024-09-13T18:26:04.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:26:04 smithi204 ceph-mon[34777]: pgmap v111: 129 pgs: 129 active+clean; 13 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 62 MiB/s wr, 1.00k op/s 2024-09-13T18:26:06.476 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:26:06 smithi114 ceph-mon[30897]: pgmap v112: 129 pgs: 129 active+clean; 13 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 47 MiB/s wr, 781 op/s 2024-09-13T18:26:06.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:26:06 smithi134 ceph-mon[34283]: pgmap v112: 129 pgs: 129 active+clean; 13 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 47 MiB/s wr, 781 op/s 2024-09-13T18:26:06.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:26:06 smithi204 ceph-mon[34777]: pgmap v112: 129 pgs: 129 active+clean; 13 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 47 MiB/s wr, 781 op/s 2024-09-13T18:26:07.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:26:07 smithi114 ceph-mon[30897]: scrub summary: idle 2024-09-13T18:26:07.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:26:07 smithi114 ceph-mon[30897]: scrub summary: idle 2024-09-13T18:26:07.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:26:07 smithi134 ceph-mon[34283]: scrub summary: idle 2024-09-13T18:26:07.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:26:07 smithi134 ceph-mon[34283]: scrub summary: idle 2024-09-13T18:26:07.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:26:07 smithi204 ceph-mon[34777]: scrub summary: idle 2024-09-13T18:26:07.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:26:07 smithi204 ceph-mon[34777]: scrub summary: idle 2024-09-13T18:26:08.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:26:08 smithi114 ceph-mon[30897]: scrub summary: idle 2024-09-13T18:26:08.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:26:08 smithi114 ceph-mon[30897]: pgmap v113: 129 pgs: 129 active+clean; 13 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 44 MiB/s wr, 732 op/s 2024-09-13T18:26:08.595 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:26:08 smithi204 ceph-mon[34777]: scrub summary: idle 2024-09-13T18:26:08.595 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:26:08 smithi204 ceph-mon[34777]: pgmap v113: 129 pgs: 129 active+clean; 13 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 44 MiB/s wr, 732 op/s 2024-09-13T18:26:08.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:26:08 smithi134 ceph-mon[34283]: scrub summary: idle 2024-09-13T18:26:08.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:26:08 smithi134 ceph-mon[34283]: pgmap v113: 129 pgs: 129 active+clean; 13 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 44 MiB/s wr, 732 op/s 2024-09-13T18:26:08.989 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:08.986+0000 7fbd76af0640 1 -- 172.21.15.114:0/2157269477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd680a47c0 msgr2=0x7fbd680a4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:08.990 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:08.986+0000 7fbd76af0640 1 --2- 172.21.15.114:0/2157269477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd680a47c0 0x7fbd680a4ba0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c009860 tx=0x7fbd5c02a360 comp rx=0 tx=0).stop 2024-09-13T18:26:08.992 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:08.988+0000 7fbd76af0640 1 -- 172.21.15.114:0/2157269477 shutdown_connections 2024-09-13T18:26:08.992 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:08.988+0000 7fbd76af0640 1 --2- 172.21.15.114:0/2157269477 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbd680b74e0 0x7fbd680b98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:08.992 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:08.988+0000 7fbd76af0640 1 --2- 172.21.15.114:0/2157269477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd680a50e0 0x7fbd680b6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:08.992 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:08.988+0000 7fbd76af0640 1 --2- 172.21.15.114:0/2157269477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd680a47c0 0x7fbd680a4ba0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:08.992 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:08.988+0000 7fbd76af0640 1 -- 172.21.15.114:0/2157269477 >> 172.21.15.114:0/2157269477 conn(0x7fbd6801a0a0 msgr2=0x7fbd6801a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:08.992 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:08.989+0000 7fbd76af0640 1 -- 172.21.15.114:0/2157269477 shutdown_connections 2024-09-13T18:26:08.992 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:08.989+0000 7fbd76af0640 1 -- 172.21.15.114:0/2157269477 wait complete. 2024-09-13T18:26:08.993 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:08.989+0000 7fbd76af0640 1 Processor -- start 2024-09-13T18:26:08.993 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:08.989+0000 7fbd76af0640 1 -- start start 2024-09-13T18:26:08.994 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:08.991+0000 7fbd76af0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbd680a47c0 0x7fbd681431d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:08.994 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:08.991+0000 7fbd76af0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd680a50e0 0x7fbd68143710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:08.994 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:08.991+0000 7fbd76af0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd680b74e0 0x7fbd68147b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:08.994 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:08.991+0000 7fbd76af0640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd68148100 con 0x7fbd680a47c0 2024-09-13T18:26:08.994 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:08.991+0000 7fbd76af0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd68148270 con 0x7fbd680a50e0 2024-09-13T18:26:08.994 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:08.991+0000 7fbd76af0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd68148430 con 0x7fbd680b74e0 2024-09-13T18:26:08.995 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:08.991+0000 7fbd7488c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbd680a47c0 0x7fbd681431d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:08.995 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:08.991+0000 7fbd7508d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd680b74e0 0x7fbd68147b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:08.995 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:08.991+0000 7fbd7508d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd680b74e0 0x7fbd68147b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:47294/0 (socket says 172.21.15.114:47294) 2024-09-13T18:26:08.995 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:08.991+0000 7fbd7508d640 1 -- 172.21.15.114:0/1924179691 learned_addr learned my addr 172.21.15.114:0/1924179691 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:08.995 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:08.991+0000 7fbd6ffff640 1 --2- 172.21.15.114:0/1924179691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd680a50e0 0x7fbd68143710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:08.995 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:08.992+0000 7fbd7508d640 1 -- 172.21.15.114:0/1924179691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd680a50e0 msgr2=0x7fbd68143710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:08.995 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:08.992+0000 7fbd7508d640 1 --2- 172.21.15.114:0/1924179691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd680a50e0 0x7fbd68143710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:08.996 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:08.992+0000 7fbd7508d640 1 -- 172.21.15.114:0/1924179691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbd680a47c0 msgr2=0x7fbd681431d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:08.996 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:08.992+0000 7fbd7508d640 1 --2- 172.21.15.114:0/1924179691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbd680a47c0 0x7fbd681431d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:08.996 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:08.992+0000 7fbd7508d640 1 -- 172.21.15.114:0/1924179691 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd5c009490 con 0x7fbd680b74e0 2024-09-13T18:26:08.996 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:08.993+0000 7fbd7508d640 1 --2- 172.21.15.114:0/1924179691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd680b74e0 0x7fbd68147b00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbd70065d70 tx=0x7fbd7006f6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:08.994+0000 7fbd6dffb640 1 -- 172.21.15.114:0/1924179691 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd70073430 con 0x7fbd680b74e0 2024-09-13T18:26:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:08.994+0000 7fbd76af0640 1 -- 172.21.15.114:0/1924179691 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd68148710 con 0x7fbd680b74e0 2024-09-13T18:26:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:08.994+0000 7fbd76af0640 1 -- 172.21.15.114:0/1924179691 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd6814ff90 con 0x7fbd680b74e0 2024-09-13T18:26:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:08.994+0000 7fbd6dffb640 1 -- 172.21.15.114:0/1924179691 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbd70073a20 con 0x7fbd680b74e0 2024-09-13T18:26:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:08.995+0000 7fbd6dffb640 1 -- 172.21.15.114:0/1924179691 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd70072740 con 0x7fbd680b74e0 2024-09-13T18:26:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:08.999+0000 7fbd6dffb640 1 -- 172.21.15.114:0/1924179691 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fbd700728a0 con 0x7fbd680b74e0 2024-09-13T18:26:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:08.999+0000 7fbd79814740 1 Processor -- start 2024-09-13T18:26:09.006 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.002+0000 7fbd6dffb640 1 --2- 172.21.15.114:0/1924179691 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbd600a1410 0x7fbd600a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:09.006 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.003+0000 7fbd6dffb640 1 -- 172.21.15.114:0/1924179691 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fbd701306d0 con 0x7fbd680b74e0 2024-09-13T18:26:09.006 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.003+0000 7fbd79814740 1 -- start start 2024-09-13T18:26:09.007 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.003+0000 7fbd79814740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecfe685500 0x55ecfe6858e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:09.007 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.003+0000 7fbd79814740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ecfe685e20 0x55ecfe690180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:09.007 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.003+0000 7fbd79814740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ecfe6906c0 0x55ecfe692ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:09.007 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.003+0000 7fbd79814740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecfe5f0c30 con 0x55ecfe6906c0 2024-09-13T18:26:09.007 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.003+0000 7fbd79814740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecfe5ee3c0 con 0x55ecfe685e20 2024-09-13T18:26:09.007 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.003+0000 7fbd79814740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecfe5e9cb0 con 0x55ecfe685500 2024-09-13T18:26:09.007 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.003+0000 7fbd6ffff640 1 --2- 172.21.15.114:0/1924179691 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbd600a1410 0x7fbd600a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:09.007 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.004+0000 7fbd6ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ecfe685e20 0x55ecfe690180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.004+0000 7fbd6ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ecfe685e20 0x55ecfe690180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:38122/0 (socket says 172.21.15.114:38122) 2024-09-13T18:26:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.004+0000 7fbd6ffff640 1 -- 172.21.15.114:0/1667441683 learned_addr learned my addr 172.21.15.114:0/1667441683 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.004+0000 7fbd7508d640 1 --2- 172.21.15.114:0/1667441683 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ecfe6906c0 0x55ecfe692ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.005+0000 7fbd6ffff640 1 -- 172.21.15.114:0/1667441683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecfe685500 msgr2=0x55ecfe6858e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:26:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.005+0000 7fbd6ffff640 1 --2- 172.21.15.114:0/1667441683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecfe685500 0x55ecfe6858e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.005+0000 7fbd6ffff640 1 -- 172.21.15.114:0/1667441683 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ecfe6906c0 msgr2=0x55ecfe692ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.005+0000 7fbd6ffff640 1 --2- 172.21.15.114:0/1667441683 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ecfe6906c0 0x55ecfe692ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.005+0000 7fbd6ffff640 1 -- 172.21.15.114:0/1667441683 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ecfe590910 con 0x55ecfe685e20 2024-09-13T18:26:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.006+0000 7fbd6ffff640 1 --2- 172.21.15.114:0/1667441683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ecfe685e20 0x55ecfe690180 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbd64017950 tx=0x7fbd64017d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c4e8b415ff9314f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.006+0000 7fbd5affd640 1 -- 172.21.15.114:0/1667441683 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd64015d50 con 0x55ecfe685e20 2024-09-13T18:26:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.006+0000 7fbd5affd640 1 -- 172.21.15.114:0/1667441683 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbd6401c070 con 0x55ecfe685e20 2024-09-13T18:26:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.006+0000 7fbd5affd640 1 -- 172.21.15.114:0/1667441683 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd64013600 con 0x55ecfe685e20 2024-09-13T18:26:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.008+0000 7fbd79814740 1 -- 172.21.15.114:0/1667441683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ecfe685e20 msgr2=0x55ecfe690180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.008+0000 7fbd79814740 1 --2- 172.21.15.114:0/1667441683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ecfe685e20 0x55ecfe690180 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbd64017950 tx=0x7fbd64017d50 comp rx=0 tx=0).stop 2024-09-13T18:26:09.012 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.008+0000 7fbd79814740 1 -- 172.21.15.114:0/1667441683 shutdown_connections 2024-09-13T18:26:09.012 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.008+0000 7fbd79814740 1 --2- 172.21.15.114:0/1667441683 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ecfe6906c0 0x55ecfe692ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:09.012 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.008+0000 7fbd79814740 1 --2- 172.21.15.114:0/1667441683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ecfe685e20 0x55ecfe690180 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:09.012 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.008+0000 7fbd79814740 1 --2- 172.21.15.114:0/1667441683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecfe685500 0x55ecfe6858e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:09.012 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.008+0000 7fbd79814740 1 -- 172.21.15.114:0/1667441683 >> 172.21.15.114:0/1667441683 conn(0x55ecfe681a90 msgr2=0x55ecfe681e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:09.012 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.009+0000 7fbd6ffff640 1 --2- 172.21.15.114:0/1924179691 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbd600a1410 0x7fbd600a38d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbd6401f000 tx=0x7fbd64013c30 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:09.012 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.009+0000 7fbd79814740 1 -- 172.21.15.114:0/1667441683 shutdown_connections 2024-09-13T18:26:09.013 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.009+0000 7fbd79814740 1 -- 172.21.15.114:0/1667441683 wait complete. 2024-09-13T18:26:09.013 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.009+0000 7fbd79814740 1 -- start start 2024-09-13T18:26:09.013 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.009+0000 7fbd79814740 1 Processor -- start 2024-09-13T18:26:09.013 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.010+0000 7fbd79814740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ecfe685500 0x55ecfe690060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:09.013 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.010+0000 7fbd79814740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ecfe685e20 0x55ecfe68d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:09.013 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.010+0000 7fbd79814740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecfe6906c0 0x55ecfe68d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.010+0000 7fbd79814740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecfe5a7e80 con 0x55ecfe685500 2024-09-13T18:26:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.010+0000 7fbd79814740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecfe5a7b20 con 0x55ecfe685e20 2024-09-13T18:26:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.010+0000 7fbd79814740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecfe56d260 con 0x55ecfe6906c0 2024-09-13T18:26:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.010+0000 7fbd79814740 10 client.0 _ll_register_callbacks cb 0x55ecfe5a2df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:26:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.010+0000 7fbd6ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ecfe685e20 0x55ecfe68d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.010+0000 7fbd6ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ecfe685e20 0x55ecfe68d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:38136/0 (socket says 172.21.15.114:38136) 2024-09-13T18:26:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.010+0000 7fbd6ffff640 1 -- 172.21.15.114:0/3845206109 learned_addr learned my addr 172.21.15.114:0/3845206109 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.010+0000 7fbd7488c640 1 --2- 172.21.15.114:0/3845206109 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ecfe685500 0x55ecfe690060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.010+0000 7fbd7508d640 1 --2- 172.21.15.114:0/3845206109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecfe6906c0 0x55ecfe68d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.011+0000 7fbd6ffff640 1 -- 172.21.15.114:0/3845206109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecfe6906c0 msgr2=0x55ecfe68d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.011+0000 7fbd6ffff640 1 --2- 172.21.15.114:0/3845206109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecfe6906c0 0x55ecfe68d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.011+0000 7fbd6ffff640 1 -- 172.21.15.114:0/3845206109 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ecfe685500 msgr2=0x55ecfe690060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.011+0000 7fbd6ffff640 1 --2- 172.21.15.114:0/3845206109 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ecfe685500 0x55ecfe690060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.011+0000 7fbd6ffff640 1 -- 172.21.15.114:0/3845206109 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd64017500 con 0x55ecfe685e20 2024-09-13T18:26:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.011+0000 7fbd6ffff640 1 --2- 172.21.15.114:0/3845206109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ecfe685e20 0x55ecfe68d350 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbd64017450 tx=0x7fbd640046e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71cd1d345b3e0ba7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:09.016 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.013+0000 7fbd4ffff640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:26:09.016 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.013+0000 7fbd4ffff640 1 -- 172.21.15.114:0/3845206109 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd640133a0 con 0x55ecfe685e20 2024-09-13T18:26:09.016 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.013+0000 7fbd4ffff640 1 -- 172.21.15.114:0/3845206109 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fbd64013500 con 0x55ecfe685e20 2024-09-13T18:26:09.016 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.013+0000 7fbd79814740 1 -- 172.21.15.114:0/3845206109 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ecfe612680 con 0x55ecfe685e20 2024-09-13T18:26:09.016 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.014+0000 7fbd4ffff640 1 -- 172.21.15.114:0/3845206109 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd64015b50 con 0x55ecfe685e20 2024-09-13T18:26:09.017 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.014+0000 7fbd79814740 10 client.26252 fetch_fsmap learned FSMap version 32 2024-09-13T18:26:09.017 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.014+0000 7fbd79814740 1 -- 172.21.15.114:0/3845206109 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55ecfe5e8330 con 0x55ecfe685e20 2024-09-13T18:26:09.018 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.014+0000 7fbd4ffff640 1 -- 172.21.15.114:0/3845206109 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ecfe612680 con 0x55ecfe685e20 2024-09-13T18:26:09.018 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.015+0000 7fbd4ffff640 1 -- 172.21.15.114:0/3845206109 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fbd64018880 con 0x55ecfe685e20 2024-09-13T18:26:09.018 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.015+0000 7fbd79814740 10 client.26252 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:26:09.018 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.015+0000 7fbd79814740 10 client.26252 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:26:09.018 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.015+0000 7fbd79814740 20 client.26252 populate_metadata read hostname 'smithi114' 2024-09-13T18:26:09.018 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.015+0000 7fbd79814740 1 --2- 172.21.15.114:0/3845206109 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55ecfe6b11c0 0x55ecfe6b35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:09.018 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.015+0000 7fbd79814740 4 client.26252 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:26:09.018 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.015+0000 7fbd79814740 1 -- 172.21.15.114:0/3845206109 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ecfe5dd5d0 con 0x55ecfe6b11c0 2024-09-13T18:26:09.019 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.015+0000 7fbd7488c640 1 --2- 172.21.15.114:0/3845206109 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55ecfe6b11c0 0x55ecfe6b35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:09.019 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.016+0000 7fbd4ffff640 10 client.26252 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:26:09.019 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.016+0000 7fbd7488c640 1 --2- 172.21.15.114:0/3845206109 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55ecfe6b11c0 0x55ecfe6b35e0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d0e12da59b80496 server_cookie=942b90fc1fcb405f in_seq=0 out_seq=0 2024-09-13T18:26:09.020 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.017+0000 7fbd4ffff640 1 -- 172.21.15.114:0/3845206109 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55ecfe5dd5d0 con 0x55ecfe6b11c0 2024-09-13T18:26:09.020 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.017+0000 7fbd4ffff640 10 client.26252 handle_command_reply: tid=0 2024-09-13T18:26:09.020 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.017+0000 7fbd79814740 1 client.26252 shutdown 2024-09-13T18:26:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.018+0000 7fbd79814740 1 -- 172.21.15.114:0/3845206109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ecfe685e20 msgr2=0x55ecfe68d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.018+0000 7fbd79814740 1 --2- 172.21.15.114:0/3845206109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ecfe685e20 0x55ecfe68d350 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbd64017450 tx=0x7fbd640046e0 comp rx=0 tx=0).stop 2024-09-13T18:26:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.018+0000 7fbd79814740 1 -- 172.21.15.114:0/3845206109 shutdown_connections 2024-09-13T18:26:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.018+0000 7fbd79814740 1 --2- 172.21.15.114:0/3845206109 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55ecfe6b11c0 0x55ecfe6b35e0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.018+0000 7fbd79814740 1 --2- 172.21.15.114:0/3845206109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecfe6906c0 0x55ecfe68d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.018+0000 7fbd79814740 1 --2- 172.21.15.114:0/3845206109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ecfe685e20 0x55ecfe68d350 secure :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbd64017450 tx=0x7fbd640046e0 comp rx=0 tx=0).stop 2024-09-13T18:26:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.018+0000 7fbd79814740 1 --2- 172.21.15.114:0/3845206109 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ecfe685500 0x55ecfe690060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.018+0000 7fbd79814740 1 -- 172.21.15.114:0/3845206109 >> 172.21.15.114:0/3845206109 conn(0x55ecfe681a90 msgr2=0x55ecfe680d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.018+0000 7fbd4ffff640 0 client.26252 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:26:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.018+0000 7fbd79814740 1 -- 172.21.15.114:0/3845206109 shutdown_connections 2024-09-13T18:26:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.019+0000 7fbd79814740 1 -- 172.21.15.114:0/3845206109 wait complete. 2024-09-13T18:26:09.023 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.020+0000 7fbd79814740 20 client.26252 trim_cache size 0 max 16384 2024-09-13T18:26:09.046 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.043+0000 7fbd79814740 1 Processor -- start 2024-09-13T18:26:09.049 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.046+0000 7fbd79814740 1 -- start start 2024-09-13T18:26:09.050 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.046+0000 7fbd79814740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecfe681a90 0x55ecfe681550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:09.050 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.046+0000 7fbd79814740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ecfe685500 0x55ecfe688640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:09.050 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.046+0000 7fbd79814740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ecfe6906c0 0x55ecfe688b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:09.050 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.046+0000 7fbd79814740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecfe68fc80 con 0x55ecfe6906c0 2024-09-13T18:26:09.050 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.046+0000 7fbd79814740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecfe68b880 con 0x55ecfe685500 2024-09-13T18:26:09.050 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.046+0000 7fbd79814740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecfe68e200 con 0x55ecfe681a90 2024-09-13T18:26:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.047+0000 7fbd6ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ecfe685500 0x55ecfe688640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.047+0000 7fbd6ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ecfe685500 0x55ecfe688640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:38142/0 (socket says 172.21.15.114:38142) 2024-09-13T18:26:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.047+0000 7fbd6ffff640 1 -- 172.21.15.114:0/4026174860 learned_addr learned my addr 172.21.15.114:0/4026174860 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.047+0000 7fbd7508d640 1 --2- 172.21.15.114:0/4026174860 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ecfe6906c0 0x55ecfe688b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.047+0000 7fbd6ffff640 1 -- 172.21.15.114:0/4026174860 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecfe681a90 msgr2=0x55ecfe681550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.047+0000 7fbd6ffff640 1 --2- 172.21.15.114:0/4026174860 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecfe681a90 0x55ecfe681550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.047+0000 7fbd6ffff640 1 -- 172.21.15.114:0/4026174860 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ecfe6906c0 msgr2=0x55ecfe688b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.047+0000 7fbd6ffff640 1 --2- 172.21.15.114:0/4026174860 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ecfe6906c0 0x55ecfe688b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.047+0000 7fbd6ffff640 1 -- 172.21.15.114:0/4026174860 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ecfe58a300 con 0x55ecfe685500 2024-09-13T18:26:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.049+0000 7fbd6ffff640 1 --2- 172.21.15.114:0/4026174860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ecfe685500 0x55ecfe688640 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbd6402cc80 tx=0x7fbd64004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f549f77dc12efaa server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.049+0000 7fbd5b7fe640 1 -- 172.21.15.114:0/4026174860 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd64006d10 con 0x55ecfe685500 2024-09-13T18:26:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.049+0000 7fbd5b7fe640 1 -- 172.21.15.114:0/4026174860 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbd64020930 con 0x55ecfe685500 2024-09-13T18:26:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.050+0000 7fbd79814740 1 -- 172.21.15.114:0/4026174860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ecfe685500 msgr2=0x55ecfe688640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.050+0000 7fbd79814740 1 --2- 172.21.15.114:0/4026174860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ecfe685500 0x55ecfe688640 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbd6402cc80 tx=0x7fbd64004750 comp rx=0 tx=0).stop 2024-09-13T18:26:09.056 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.053+0000 7fbd79814740 1 -- 172.21.15.114:0/4026174860 shutdown_connections 2024-09-13T18:26:09.056 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.053+0000 7fbd79814740 1 --2- 172.21.15.114:0/4026174860 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ecfe6906c0 0x55ecfe688b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:09.056 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.053+0000 7fbd79814740 1 --2- 172.21.15.114:0/4026174860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ecfe685500 0x55ecfe688640 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:09.056 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.053+0000 7fbd79814740 1 --2- 172.21.15.114:0/4026174860 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecfe681a90 0x55ecfe681550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:09.056 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.053+0000 7fbd79814740 1 -- 172.21.15.114:0/4026174860 >> 172.21.15.114:0/4026174860 conn(0x55ecfe6901c0 msgr2=0x55ecfe681e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:09.061 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.058+0000 7fbd79814740 1 -- 172.21.15.114:0/4026174860 shutdown_connections 2024-09-13T18:26:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.059+0000 7fbd79814740 1 -- 172.21.15.114:0/4026174860 wait complete. 2024-09-13T18:26:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.060+0000 7fbd79814740 1 -- start start 2024-09-13T18:26:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.060+0000 7fbd79814740 1 Processor -- start 2024-09-13T18:26:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.061+0000 7fbd79814740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecfe681a90 0x55ecfe6a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.061+0000 7fbd79814740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ecfe685500 0x55ecfe6a5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.061+0000 7fbd79814740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ecfe6906c0 0x55ecfe7076a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.061+0000 7fbd79814740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecfe707be0 con 0x55ecfe6906c0 2024-09-13T18:26:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.061+0000 7fbd79814740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecfe707d20 con 0x55ecfe685500 2024-09-13T18:26:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.061+0000 7fbd79814740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecfe707e90 con 0x55ecfe681a90 2024-09-13T18:26:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.061+0000 7fbd79814740 10 client.0 _ll_register_callbacks cb 0x7fbd64009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:26:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.061+0000 7fbd6ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ecfe685500 0x55ecfe6a5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.061+0000 7fbd6ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ecfe685500 0x55ecfe6a5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:38152/0 (socket says 172.21.15.114:38152) 2024-09-13T18:26:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.061+0000 7fbd6ffff640 1 -- 172.21.15.114:0/1656046002 learned_addr learned my addr 172.21.15.114:0/1656046002 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.062+0000 7fbd7508d640 1 --2- 172.21.15.114:0/1656046002 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ecfe6906c0 0x55ecfe7076a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.062+0000 7fbd6ffff640 1 -- 172.21.15.114:0/1656046002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecfe681a90 msgr2=0x55ecfe6a55a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:26:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.062+0000 7fbd6ffff640 1 --2- 172.21.15.114:0/1656046002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecfe681a90 0x55ecfe6a55a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.062+0000 7fbd6ffff640 1 -- 172.21.15.114:0/1656046002 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ecfe6906c0 msgr2=0x55ecfe7076a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.062+0000 7fbd6ffff640 1 --2- 172.21.15.114:0/1656046002 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ecfe6906c0 0x55ecfe7076a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.062+0000 7fbd6ffff640 1 -- 172.21.15.114:0/1656046002 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ecfe5e8330 con 0x55ecfe685500 2024-09-13T18:26:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.062+0000 7fbd6ffff640 1 --2- 172.21.15.114:0/1656046002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ecfe685500 0x55ecfe6a5ae0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x55ecfe6863b0 tx=0x7fbd64015860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bb3cea8a4bbaced server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.062+0000 7fbd4ffff640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:26:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.063+0000 7fbd4ffff640 1 -- 172.21.15.114:0/1656046002 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd640135e0 con 0x55ecfe685500 2024-09-13T18:26:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.063+0000 7fbd79814740 1 -- 172.21.15.114:0/1656046002 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ecfe2dd300 con 0x55ecfe685500 2024-09-13T18:26:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.063+0000 7fbd4ffff640 1 -- 172.21.15.114:0/1656046002 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fbd64006d10 con 0x55ecfe685500 2024-09-13T18:26:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.063+0000 7fbd4ffff640 1 -- 172.21.15.114:0/1656046002 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd640133a0 con 0x55ecfe685500 2024-09-13T18:26:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.063+0000 7fbd79814740 10 client.26258 fetch_fsmap learned FSMap version 32 2024-09-13T18:26:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.063+0000 7fbd79814740 1 -- 172.21.15.114:0/1656046002 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55ecfe5af8e0 con 0x55ecfe685500 2024-09-13T18:26:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.064+0000 7fbd4ffff640 1 -- 172.21.15.114:0/1656046002 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ecfe2dd300 con 0x55ecfe685500 2024-09-13T18:26:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.064+0000 7fbd4ffff640 1 -- 172.21.15.114:0/1656046002 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fbd64042070 con 0x55ecfe685500 2024-09-13T18:26:09.069 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.065+0000 7fbd79814740 10 client.26258 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:26:09.069 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.065+0000 7fbd79814740 10 client.26258 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:26:09.069 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.065+0000 7fbd79814740 20 client.26258 populate_metadata read hostname 'smithi114' 2024-09-13T18:26:09.069 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.065+0000 7fbd79814740 1 --2- 172.21.15.114:0/1656046002 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55ecfe709300 0x55ecfe70b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:09.069 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.065+0000 7fbd79814740 4 client.26258 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T18:26:09.069 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.065+0000 7fbd79814740 1 -- 172.21.15.114:0/1656046002 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ecfe2ee600 con 0x55ecfe709300 2024-09-13T18:26:09.069 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.066+0000 7fbd7488c640 1 --2- 172.21.15.114:0/1656046002 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55ecfe709300 0x55ecfe70b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.079+0000 7fbd7488c640 1 --2- 172.21.15.114:0/1656046002 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55ecfe709300 0x55ecfe70b720 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eea420be7836b97d server_cookie=84f35dae774bb3dc in_seq=0 out_seq=0 2024-09-13T18:26:09.083 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.080+0000 7fbd4ffff640 10 client.26258 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:26:09.102 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.098+0000 7fbd4ffff640 1 -- 172.21.15.114:0/1656046002 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ecfe2ee600 con 0x55ecfe709300 2024-09-13T18:26:09.102 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.099+0000 7fbd4ffff640 10 client.26258 handle_command_reply: tid=0 2024-09-13T18:26:09.102 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.099+0000 7fbd79814740 1 client.26258 shutdown 2024-09-13T18:26:09.103 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.100+0000 7fbd79814740 1 -- 172.21.15.114:0/1656046002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ecfe685500 msgr2=0x55ecfe6a5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:09.103 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.100+0000 7fbd79814740 1 --2- 172.21.15.114:0/1656046002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ecfe685500 0x55ecfe6a5ae0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x55ecfe6863b0 tx=0x7fbd64015860 comp rx=0 tx=0).stop 2024-09-13T18:26:09.104 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.101+0000 7fbd79814740 1 -- 172.21.15.114:0/1656046002 shutdown_connections 2024-09-13T18:26:09.104 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.101+0000 7fbd79814740 1 --2- 172.21.15.114:0/1656046002 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55ecfe709300 0x55ecfe70b720 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:09.104 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.101+0000 7fbd79814740 1 --2- 172.21.15.114:0/1656046002 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ecfe6906c0 0x55ecfe7076a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:09.104 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.101+0000 7fbd79814740 1 --2- 172.21.15.114:0/1656046002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ecfe685500 0x55ecfe6a5ae0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:09.104 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.101+0000 7fbd79814740 1 --2- 172.21.15.114:0/1656046002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ecfe681a90 0x55ecfe6a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:09.105 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.101+0000 7fbd79814740 1 -- 172.21.15.114:0/1656046002 >> 172.21.15.114:0/1656046002 conn(0x55ecfe6901c0 msgr2=0x55ecfe6ace80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:09.105 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.101+0000 7fbd4ffff640 0 client.26258 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:26:09.110 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.104+0000 7fbd79814740 1 -- 172.21.15.114:0/1656046002 shutdown_connections 2024-09-13T18:26:09.110 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.104+0000 7fbd79814740 1 -- 172.21.15.114:0/1656046002 wait complete. 2024-09-13T18:26:09.110 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.107+0000 7fbd79814740 20 client.26258 trim_cache size 0 max 16384 2024-09-13T18:26:09.110 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-13T18:26:09.110 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-13T18:26:09.111 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-13T18:26:09.111 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-13T18:26:09.112 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.109+0000 7fbd76af0640 1 -- 172.21.15.114:0/1924179691 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbd600a1410 msgr2=0x7fbd600a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:09.112 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.109+0000 7fbd76af0640 1 --2- 172.21.15.114:0/1924179691 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbd600a1410 0x7fbd600a38d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbd6401f000 tx=0x7fbd64013c30 comp rx=0 tx=0).stop 2024-09-13T18:26:09.112 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.109+0000 7fbd76af0640 1 -- 172.21.15.114:0/1924179691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd680b74e0 msgr2=0x7fbd68147b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:09.112 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.109+0000 7fbd76af0640 1 --2- 172.21.15.114:0/1924179691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd680b74e0 0x7fbd68147b00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbd70065d70 tx=0x7fbd7006f6a0 comp rx=0 tx=0).stop 2024-09-13T18:26:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.109+0000 7fbd76af0640 1 -- 172.21.15.114:0/1924179691 shutdown_connections 2024-09-13T18:26:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.109+0000 7fbd76af0640 1 --2- 172.21.15.114:0/1924179691 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbd600a1410 0x7fbd600a38d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.109+0000 7fbd76af0640 1 --2- 172.21.15.114:0/1924179691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd680b74e0 0x7fbd68147b00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.109+0000 7fbd76af0640 1 --2- 172.21.15.114:0/1924179691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd680a50e0 0x7fbd68143710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.109+0000 7fbd76af0640 1 --2- 172.21.15.114:0/1924179691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbd680a47c0 0x7fbd681431d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.110+0000 7fbd76af0640 1 -- 172.21.15.114:0/1924179691 >> 172.21.15.114:0/1924179691 conn(0x7fbd6801a0a0 msgr2=0x7fbd680b5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.111+0000 7fbd76af0640 1 -- 172.21.15.114:0/1924179691 shutdown_connections 2024-09-13T18:26:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:09.111+0000 7fbd76af0640 1 -- 172.21.15.114:0/1924179691 wait complete. 2024-09-13T18:26:10.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:26:10 smithi114 ceph-mon[30897]: pgmap v114: 129 pgs: 129 active+clean; 14 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 66 MiB/s wr, 1.09k op/s 2024-09-13T18:26:10.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:26:10 smithi134 ceph-mon[34283]: pgmap v114: 129 pgs: 129 active+clean; 14 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 66 MiB/s wr, 1.09k op/s 2024-09-13T18:26:10.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:26:10 smithi204 ceph-mon[34777]: pgmap v114: 129 pgs: 129 active+clean; 14 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 66 MiB/s wr, 1.09k op/s 2024-09-13T18:26:12.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:26:12 smithi114 ceph-mon[30897]: pgmap v115: 129 pgs: 129 active+clean; 14 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 43 MiB/s wr, 715 op/s 2024-09-13T18:26:12.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:26:12 smithi134 ceph-mon[34283]: pgmap v115: 129 pgs: 129 active+clean; 14 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 43 MiB/s wr, 715 op/s 2024-09-13T18:26:12.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:26:12 smithi204 ceph-mon[34777]: pgmap v115: 129 pgs: 129 active+clean; 14 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 43 MiB/s wr, 715 op/s 2024-09-13T18:26:14.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:26:14 smithi114 ceph-mon[30897]: pgmap v116: 129 pgs: 129 active+clean; 14 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 60 MiB/s wr, 987 op/s 2024-09-13T18:26:14.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:26:14 smithi134 ceph-mon[34283]: pgmap v116: 129 pgs: 129 active+clean; 14 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 60 MiB/s wr, 987 op/s 2024-09-13T18:26:14.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:26:14 smithi204 ceph-mon[34777]: pgmap v116: 129 pgs: 129 active+clean; 14 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 60 MiB/s wr, 987 op/s 2024-09-13T18:26:16.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:26:16 smithi114 ceph-mon[30897]: pgmap v117: 129 pgs: 129 active+clean; 14 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 49 MiB/s wr, 810 op/s 2024-09-13T18:26:16.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:26:16 smithi134 ceph-mon[34283]: pgmap v117: 129 pgs: 129 active+clean; 14 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 49 MiB/s wr, 810 op/s 2024-09-13T18:26:16.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:26:16 smithi204 ceph-mon[34777]: pgmap v117: 129 pgs: 129 active+clean; 14 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 49 MiB/s wr, 810 op/s 2024-09-13T18:26:18.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:26:18 smithi114 ceph-mon[30897]: pgmap v118: 129 pgs: 129 active+clean; 14 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 44 MiB/s wr, 716 op/s 2024-09-13T18:26:18.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:26:18 smithi134 ceph-mon[34283]: pgmap v118: 129 pgs: 129 active+clean; 14 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 44 MiB/s wr, 716 op/s 2024-09-13T18:26:18.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:26:18 smithi204 ceph-mon[34777]: pgmap v118: 129 pgs: 129 active+clean; 14 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 44 MiB/s wr, 716 op/s 2024-09-13T18:26:20.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:26:20 smithi114 ceph-mon[30897]: pgmap v119: 129 pgs: 129 active+clean; 14 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 66 MiB/s wr, 1.07k op/s 2024-09-13T18:26:20.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:26:20 smithi134 ceph-mon[34283]: pgmap v119: 129 pgs: 129 active+clean; 14 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 66 MiB/s wr, 1.07k op/s 2024-09-13T18:26:20.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:26:20 smithi204 ceph-mon[34777]: pgmap v119: 129 pgs: 129 active+clean; 14 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 66 MiB/s wr, 1.07k op/s 2024-09-13T18:26:22.286 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.282+0000 7f496df4c640 1 -- 172.21.15.114:0/782372681 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49600ab1d0 msgr2=0x7f49600ab610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:22.286 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.282+0000 7f496df4c640 1 --2- 172.21.15.114:0/782372681 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49600ab1d0 0x7f49600ab610 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f4954009930 tx=0x7f495402a420 comp rx=0 tx=0).stop 2024-09-13T18:26:22.286 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.282+0000 7f496df4c640 1 -- 172.21.15.114:0/782372681 shutdown_connections 2024-09-13T18:26:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.282+0000 7f496df4c640 1 --2- 172.21.15.114:0/782372681 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49600abb50 0x7f49600b2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.282+0000 7f496df4c640 1 --2- 172.21.15.114:0/782372681 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49600ab1d0 0x7f49600ab610 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.282+0000 7f496df4c640 1 --2- 172.21.15.114:0/782372681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49600aa5f0 0x7f49600aa9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.282+0000 7f496df4c640 1 -- 172.21.15.114:0/782372681 >> 172.21.15.114:0/782372681 conn(0x7f496001a0a0 msgr2=0x7f496001a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.283+0000 7f496df4c640 1 -- 172.21.15.114:0/782372681 shutdown_connections 2024-09-13T18:26:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.283+0000 7f496df4c640 1 -- 172.21.15.114:0/782372681 wait complete. 2024-09-13T18:26:22.288 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.284+0000 7f496df4c640 1 Processor -- start 2024-09-13T18:26:22.288 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.285+0000 7f496df4c640 1 -- start start 2024-09-13T18:26:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.285+0000 7f496df4c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49600aa5f0 0x7f496013a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.285+0000 7f496df4c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49600ab1d0 0x7f496013ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.285+0000 7f496df4c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49600abb50 0x7f496013dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.285+0000 7f496df4c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f496013e5c0 con 0x7f49600aa5f0 2024-09-13T18:26:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.285+0000 7f496df4c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f496013e730 con 0x7f49600ab1d0 2024-09-13T18:26:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.285+0000 7f496df4c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f496013e8f0 con 0x7f49600abb50 2024-09-13T18:26:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.286+0000 7f4967fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49600abb50 0x7f496013dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.286+0000 7f4967fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49600abb50 0x7f496013dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:33662/0 (socket says 172.21.15.114:33662) 2024-09-13T18:26:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.286+0000 7f4967fff640 1 -- 172.21.15.114:0/4183280175 learned_addr learned my addr 172.21.15.114:0/4183280175 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.286+0000 7f49677fe640 1 --2- 172.21.15.114:0/4183280175 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49600aa5f0 0x7f496013a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.286+0000 7f4967fff640 1 -- 172.21.15.114:0/4183280175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49600ab1d0 msgr2=0x7f496013ac80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:26:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.286+0000 7f4967fff640 1 --2- 172.21.15.114:0/4183280175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49600ab1d0 0x7f496013ac80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.286+0000 7f4967fff640 1 -- 172.21.15.114:0/4183280175 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49600aa5f0 msgr2=0x7f496013a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.286+0000 7f4967fff640 1 --2- 172.21.15.114:0/4183280175 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49600aa5f0 0x7f496013a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.286+0000 7f4967fff640 1 -- 172.21.15.114:0/4183280175 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4954009560 con 0x7f49600abb50 2024-09-13T18:26:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.287+0000 7f4967fff640 1 --2- 172.21.15.114:0/4183280175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49600abb50 0x7f496013dfc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4968066a70 tx=0x7f496806f6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.287+0000 7f4964ff9640 1 -- 172.21.15.114:0/4183280175 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4968073430 con 0x7f49600abb50 2024-09-13T18:26:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.287+0000 7f4964ff9640 1 -- 172.21.15.114:0/4183280175 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4968073a20 con 0x7f49600abb50 2024-09-13T18:26:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.287+0000 7f4964ff9640 1 -- 172.21.15.114:0/4183280175 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49680728b0 con 0x7f49600abb50 2024-09-13T18:26:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.288+0000 7f496df4c640 1 -- 172.21.15.114:0/4183280175 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f496013ebd0 con 0x7f49600abb50 2024-09-13T18:26:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.288+0000 7f496df4c640 1 -- 172.21.15.114:0/4183280175 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4960147350 con 0x7f49600abb50 2024-09-13T18:26:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.293+0000 7f4970c6e740 1 Processor -- start 2024-09-13T18:26:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.293+0000 7f4964ff9640 1 -- 172.21.15.114:0/4183280175 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f4968072a10 con 0x7f49600abb50 2024-09-13T18:26:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.296+0000 7f4970c6e740 1 -- start start 2024-09-13T18:26:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.297+0000 7f4970c6e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620d6231d10 0x5620d62320f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.297+0000 7f4970c6e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620d6232630 0x5620d623c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.297+0000 7f4970c6e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620d623ced0 0x5620d623f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.297+0000 7f4970c6e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620d6194c30 con 0x5620d6232630 2024-09-13T18:26:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.297+0000 7f4970c6e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620d61923c0 con 0x5620d623ced0 2024-09-13T18:26:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.297+0000 7f4970c6e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620d618dcb0 con 0x5620d6231d10 2024-09-13T18:26:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.297+0000 7f4964ff9640 1 --2- 172.21.15.114:0/4183280175 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f49580a1300 0x7f49580a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.298+0000 7f4964ff9640 1 -- 172.21.15.114:0/4183280175 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f496812f6d0 con 0x7f49600abb50 2024-09-13T18:26:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.298+0000 7f4967fff640 1 --2- 172.21.15.114:0/4183280175 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f49580a1300 0x7f49580a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.299+0000 7f4966ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620d6231d10 0x5620d62320f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.299+0000 7f4966ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620d623ced0 0x5620d623f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.299+0000 7f4966ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620d6231d10 0x5620d62320f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:33668/0 (socket says 172.21.15.114:33668) 2024-09-13T18:26:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.299+0000 7f4966ffd640 1 -- 172.21.15.114:0/557837940 learned_addr learned my addr 172.21.15.114:0/557837940 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.300+0000 7f4966ffd640 1 -- 172.21.15.114:0/557837940 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620d6231d10 msgr2=0x5620d62320f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.300+0000 7f4966ffd640 1 --2- 172.21.15.114:0/557837940 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620d6231d10 0x5620d62320f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.300+0000 7f4966ffd640 1 -- 172.21.15.114:0/557837940 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620d6232630 msgr2=0x5620d623c990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:26:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.300+0000 7f4966ffd640 1 --2- 172.21.15.114:0/557837940 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620d6232630 0x5620d623c990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.300+0000 7f4966ffd640 1 -- 172.21.15.114:0/557837940 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620d6134910 con 0x5620d623ced0 2024-09-13T18:26:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.300+0000 7f4966ffd640 1 --2- 172.21.15.114:0/557837940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620d623ced0 0x5620d623f2c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f495402a940 tx=0x7f4954000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38fb80cc2b9c9e5d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.301+0000 7f4967fff640 1 --2- 172.21.15.114:0/4183280175 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f49580a1300 0x7f49580a37c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f496804efc0 tx=0x7f49680f9040 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.303+0000 7f4951ffb640 1 -- 172.21.15.114:0/557837940 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4954032220 con 0x5620d623ced0 2024-09-13T18:26:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.303+0000 7f4951ffb640 1 -- 172.21.15.114:0/557837940 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4954032660 con 0x5620d623ced0 2024-09-13T18:26:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.304+0000 7f4970c6e740 1 -- 172.21.15.114:0/557837940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620d623ced0 msgr2=0x5620d623f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.304+0000 7f4970c6e740 1 --2- 172.21.15.114:0/557837940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620d623ced0 0x5620d623f2c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f495402a940 tx=0x7f4954000bc0 comp rx=0 tx=0).stop 2024-09-13T18:26:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.306+0000 7f4970c6e740 1 -- 172.21.15.114:0/557837940 shutdown_connections 2024-09-13T18:26:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.306+0000 7f4970c6e740 1 --2- 172.21.15.114:0/557837940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620d623ced0 0x5620d623f2c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.306+0000 7f4970c6e740 1 --2- 172.21.15.114:0/557837940 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620d6232630 0x5620d623c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.306+0000 7f4970c6e740 1 --2- 172.21.15.114:0/557837940 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620d6231d10 0x5620d62320f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.306+0000 7f4970c6e740 1 -- 172.21.15.114:0/557837940 >> 172.21.15.114:0/557837940 conn(0x5620d622e2a0 msgr2=0x5620d622e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.307+0000 7f4970c6e740 1 -- 172.21.15.114:0/557837940 shutdown_connections 2024-09-13T18:26:22.311 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.307+0000 7f4970c6e740 1 -- 172.21.15.114:0/557837940 wait complete. 2024-09-13T18:26:22.311 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.308+0000 7f4970c6e740 1 -- start start 2024-09-13T18:26:22.311 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.308+0000 7f4970c6e740 1 Processor -- start 2024-09-13T18:26:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.309+0000 7f4970c6e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620d6231d10 0x5620d623c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.309+0000 7f4970c6e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620d6232630 0x5620d6239b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.309+0000 7f4970c6e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620d623ced0 0x5620d623a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.309+0000 7f4970c6e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620d614be80 con 0x5620d6231d10 2024-09-13T18:26:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.309+0000 7f4970c6e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620d614bb20 con 0x5620d623ced0 2024-09-13T18:26:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.309+0000 7f4970c6e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620d6111260 con 0x5620d6232630 2024-09-13T18:26:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.309+0000 7f4970c6e740 10 client.0 _ll_register_callbacks cb 0x5620d6146df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:26:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.309+0000 7f4966ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620d6231d10 0x5620d623c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.309+0000 7f4966ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620d6231d10 0x5620d623c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35694/0 (socket says 172.21.15.114:35694) 2024-09-13T18:26:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.309+0000 7f4966ffd640 1 -- 172.21.15.114:0/2473817280 learned_addr learned my addr 172.21.15.114:0/2473817280 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.309+0000 7f4966ffd640 1 --2- 172.21.15.114:0/2473817280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620d623ced0 0x5620d623a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.310+0000 7f49677fe640 1 --2- 172.21.15.114:0/2473817280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620d6232630 0x5620d6239b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.310+0000 7f4966ffd640 1 -- 172.21.15.114:0/2473817280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620d6232630 msgr2=0x5620d6239b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.310+0000 7f4966ffd640 1 --2- 172.21.15.114:0/2473817280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620d6232630 0x5620d6239b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.310+0000 7f4966ffd640 1 -- 172.21.15.114:0/2473817280 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620d6231d10 msgr2=0x5620d623c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.310+0000 7f4966ffd640 1 --2- 172.21.15.114:0/2473817280 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620d6231d10 0x5620d623c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.310+0000 7f4966ffd640 1 -- 172.21.15.114:0/2473817280 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49600b2d10 con 0x5620d623ced0 2024-09-13T18:26:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.310+0000 7f4966ffd640 1 --2- 172.21.15.114:0/2473817280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620d623ced0 0x5620d623a110 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4954001eb0 tx=0x7f495400b050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93e2d52fd2ade20 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.311+0000 7f4946ffd640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:26:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.311+0000 7f4946ffd640 1 -- 172.21.15.114:0/2473817280 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4954004230 con 0x5620d623ced0 2024-09-13T18:26:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.311+0000 7f4970c6e740 1 -- 172.21.15.114:0/2473817280 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5620d61815d0 con 0x5620d623ced0 2024-09-13T18:26:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.311+0000 7f4946ffd640 1 -- 172.21.15.114:0/2473817280 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f4954004390 con 0x5620d623ced0 2024-09-13T18:26:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.312+0000 7f4946ffd640 1 -- 172.21.15.114:0/2473817280 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49540323d0 con 0x5620d623ced0 2024-09-13T18:26:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.312+0000 7f4970c6e740 10 client.26267 fetch_fsmap learned FSMap version 32 2024-09-13T18:26:22.316 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.312+0000 7f4970c6e740 1 -- 172.21.15.114:0/2473817280 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5620d618c330 con 0x5620d623ced0 2024-09-13T18:26:22.317 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.313+0000 7f4946ffd640 1 -- 172.21.15.114:0/2473817280 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5620d61815d0 con 0x5620d623ced0 2024-09-13T18:26:22.317 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.313+0000 7f4946ffd640 1 -- 172.21.15.114:0/2473817280 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f495403d700 con 0x5620d623ced0 2024-09-13T18:26:22.317 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.313+0000 7f4970c6e740 10 client.26267 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:26:22.317 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.313+0000 7f4970c6e740 10 client.26267 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:26:22.317 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.313+0000 7f4970c6e740 20 client.26267 populate_metadata read hostname 'smithi114' 2024-09-13T18:26:22.317 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.313+0000 7f4970c6e740 1 --2- 172.21.15.114:0/2473817280 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5620d624c770 0x5620d624eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:22.317 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.313+0000 7f4970c6e740 4 client.26267 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:26:22.317 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.313+0000 7f4970c6e740 1 -- 172.21.15.114:0/2473817280 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5620d5e81300 con 0x5620d624c770 2024-09-13T18:26:22.319 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.316+0000 7f49677fe640 1 --2- 172.21.15.114:0/2473817280 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5620d624c770 0x5620d624eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:22.324 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.316+0000 7f49677fe640 1 --2- 172.21.15.114:0/2473817280 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5620d624c770 0x5620d624eb90 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=403b820684be31dd server_cookie=53f1519dc086ce60 in_seq=0 out_seq=0 2024-09-13T18:26:22.324 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.317+0000 7f4946ffd640 10 client.26267 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:26:22.324 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.317+0000 7f4946ffd640 1 -- 172.21.15.114:0/2473817280 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5620d5e81300 con 0x5620d624c770 2024-09-13T18:26:22.324 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.317+0000 7f4946ffd640 10 client.26267 handle_command_reply: tid=0 2024-09-13T18:26:22.324 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.317+0000 7f4970c6e740 1 client.26267 shutdown 2024-09-13T18:26:22.324 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.319+0000 7f4970c6e740 1 -- 172.21.15.114:0/2473817280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620d623ced0 msgr2=0x5620d623a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:22.324 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.319+0000 7f4970c6e740 1 --2- 172.21.15.114:0/2473817280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620d623ced0 0x5620d623a110 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4954001eb0 tx=0x7f495400b050 comp rx=0 tx=0).stop 2024-09-13T18:26:22.324 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.320+0000 7f4970c6e740 1 -- 172.21.15.114:0/2473817280 shutdown_connections 2024-09-13T18:26:22.324 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.320+0000 7f4970c6e740 1 --2- 172.21.15.114:0/2473817280 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5620d624c770 0x5620d624eb90 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:22.324 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.320+0000 7f4970c6e740 1 --2- 172.21.15.114:0/2473817280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620d623ced0 0x5620d623a110 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:22.325 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.320+0000 7f4970c6e740 1 --2- 172.21.15.114:0/2473817280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620d6232630 0x5620d6239b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:22.325 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.320+0000 7f4970c6e740 1 --2- 172.21.15.114:0/2473817280 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620d6231d10 0x5620d623c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:22.325 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.320+0000 7f4970c6e740 1 -- 172.21.15.114:0/2473817280 >> 172.21.15.114:0/2473817280 conn(0x5620d622e2a0 msgr2=0x5620d6233720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:22.325 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.320+0000 7f4946ffd640 0 client.26267 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:26:22.325 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.321+0000 7f4970c6e740 1 -- 172.21.15.114:0/2473817280 shutdown_connections 2024-09-13T18:26:22.325 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.322+0000 7f4970c6e740 1 -- 172.21.15.114:0/2473817280 wait complete. 2024-09-13T18:26:22.327 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.324+0000 7f4970c6e740 20 client.26267 trim_cache size 0 max 16384 2024-09-13T18:26:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.349+0000 7f4970c6e740 1 Processor -- start 2024-09-13T18:26:22.355 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.352+0000 7f4970c6e740 1 -- start start 2024-09-13T18:26:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.352+0000 7f4970c6e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620d6231d10 0x5620d6243cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.352+0000 7f4970c6e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620d6232630 0x5620d6244210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.352+0000 7f4970c6e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620d623ced0 0x5620d6234df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.352+0000 7f4970c6e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620d6238030 con 0x5620d6232630 2024-09-13T18:26:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.352+0000 7f4970c6e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620d623f5f0 con 0x5620d6231d10 2024-09-13T18:26:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.352+0000 7f4970c6e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620d62495c0 con 0x5620d623ced0 2024-09-13T18:26:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.353+0000 7f4966ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620d6231d10 0x5620d6243cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.353+0000 7f4966ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620d6231d10 0x5620d6243cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:34616/0 (socket says 172.21.15.114:34616) 2024-09-13T18:26:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.353+0000 7f4966ffd640 1 -- 172.21.15.114:0/1801259599 learned_addr learned my addr 172.21.15.114:0/1801259599 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.353+0000 7f49677fe640 1 --2- 172.21.15.114:0/1801259599 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620d6232630 0x5620d6244210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.353+0000 7f4966ffd640 1 --2- 172.21.15.114:0/1801259599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620d623ced0 0x5620d6234df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.354+0000 7f4966ffd640 1 -- 172.21.15.114:0/1801259599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620d623ced0 msgr2=0x5620d6234df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.354+0000 7f4966ffd640 1 --2- 172.21.15.114:0/1801259599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620d623ced0 0x5620d6234df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.354+0000 7f4966ffd640 1 -- 172.21.15.114:0/1801259599 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620d6232630 msgr2=0x5620d6244210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.354+0000 7f4966ffd640 1 --2- 172.21.15.114:0/1801259599 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620d6232630 0x5620d6244210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.354+0000 7f4966ffd640 1 -- 172.21.15.114:0/1801259599 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620d612e300 con 0x5620d6231d10 2024-09-13T18:26:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.355+0000 7f4966ffd640 1 --2- 172.21.15.114:0/1801259599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620d6231d10 0x5620d6243cd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4954033000 tx=0x7f4954040040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c0cec9e18d39adf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.356+0000 7f49527fc640 1 -- 172.21.15.114:0/1801259599 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49540323d0 con 0x5620d6231d10 2024-09-13T18:26:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.356+0000 7f49527fc640 1 -- 172.21.15.114:0/1801259599 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f49540325e0 con 0x5620d6231d10 2024-09-13T18:26:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.357+0000 7f4970c6e740 1 -- 172.21.15.114:0/1801259599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620d6231d10 msgr2=0x5620d6243cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.357+0000 7f4970c6e740 1 --2- 172.21.15.114:0/1801259599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620d6231d10 0x5620d6243cd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4954033000 tx=0x7f4954040040 comp rx=0 tx=0).stop 2024-09-13T18:26:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.361+0000 7f4970c6e740 1 -- 172.21.15.114:0/1801259599 shutdown_connections 2024-09-13T18:26:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.361+0000 7f4970c6e740 1 --2- 172.21.15.114:0/1801259599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620d623ced0 0x5620d6234df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.361+0000 7f4970c6e740 1 --2- 172.21.15.114:0/1801259599 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620d6232630 0x5620d6244210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.361+0000 7f4970c6e740 1 --2- 172.21.15.114:0/1801259599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620d6231d10 0x5620d6243cd0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.361+0000 7f4970c6e740 1 -- 172.21.15.114:0/1801259599 >> 172.21.15.114:0/1801259599 conn(0x5620d622e2a0 msgr2=0x5620d622e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:22.371 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.365+0000 7f4970c6e740 1 -- 172.21.15.114:0/1801259599 shutdown_connections 2024-09-13T18:26:22.371 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.368+0000 7f4970c6e740 1 -- 172.21.15.114:0/1801259599 wait complete. 2024-09-13T18:26:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.368+0000 7f4970c6e740 1 -- start start 2024-09-13T18:26:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.369+0000 7f4970c6e740 1 Processor -- start 2024-09-13T18:26:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.369+0000 7f4970c6e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620d6231d10 0x5620d62428e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.369+0000 7f4970c6e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620d6232630 0x5620d6242e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.369+0000 7f4970c6e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620d623ced0 0x5620d6243360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.369+0000 7f4970c6e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620d629ed90 con 0x5620d623ced0 2024-09-13T18:26:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.369+0000 7f4970c6e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620d629ef00 con 0x5620d6232630 2024-09-13T18:26:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.369+0000 7f4970c6e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620d629f0c0 con 0x5620d6231d10 2024-09-13T18:26:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.369+0000 7f4970c6e740 10 client.0 _ll_register_callbacks cb 0x7f495c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:26:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.370+0000 7f49677fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620d6232630 0x5620d6242e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.370+0000 7f49677fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620d6232630 0x5620d6242e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:34632/0 (socket says 172.21.15.114:34632) 2024-09-13T18:26:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.370+0000 7f49677fe640 1 -- 172.21.15.114:0/269851557 learned_addr learned my addr 172.21.15.114:0/269851557 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.370+0000 7f4966ffd640 1 --2- 172.21.15.114:0/269851557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620d6231d10 0x5620d62428e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.370+0000 7f4966ffd640 1 --2- 172.21.15.114:0/269851557 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620d623ced0 0x5620d6243360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.370+0000 7f49677fe640 1 -- 172.21.15.114:0/269851557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620d6231d10 msgr2=0x5620d62428e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.370+0000 7f49677fe640 1 --2- 172.21.15.114:0/269851557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620d6231d10 0x5620d62428e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.370+0000 7f49677fe640 1 -- 172.21.15.114:0/269851557 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620d623ced0 msgr2=0x5620d6243360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.370+0000 7f49677fe640 1 --2- 172.21.15.114:0/269851557 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620d623ced0 0x5620d6243360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.370+0000 7f49677fe640 1 -- 172.21.15.114:0/269851557 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5620d618c330 con 0x5620d6232630 2024-09-13T18:26:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.371+0000 7f49677fe640 1 --2- 172.21.15.114:0/269851557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620d6232630 0x5620d6242e20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f495c004700 tx=0x7f495c01af40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fce9dfd249273fe9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.371+0000 7f4946ffd640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:26:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.371+0000 7f4946ffd640 1 -- 172.21.15.114:0/269851557 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f495c018040 con 0x5620d6232630 2024-09-13T18:26:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.371+0000 7f4946ffd640 1 -- 172.21.15.114:0/269851557 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f495c012650 con 0x5620d6232630 2024-09-13T18:26:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.371+0000 7f4970c6e740 1 -- 172.21.15.114:0/269851557 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5620d61b6680 con 0x5620d6232630 2024-09-13T18:26:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.374+0000 7f4946ffd640 1 -- 172.21.15.114:0/269851557 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f495c009e50 con 0x5620d6232630 2024-09-13T18:26:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.374+0000 7f4970c6e740 10 client.26273 fetch_fsmap learned FSMap version 32 2024-09-13T18:26:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.374+0000 7f4970c6e740 1 -- 172.21.15.114:0/269851557 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5620d61538e0 con 0x5620d6232630 2024-09-13T18:26:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.375+0000 7f4946ffd640 1 -- 172.21.15.114:0/269851557 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5620d61b6680 con 0x5620d6232630 2024-09-13T18:26:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.375+0000 7f4946ffd640 1 -- 172.21.15.114:0/269851557 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f495c04e9e0 con 0x5620d6232630 2024-09-13T18:26:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.375+0000 7f4970c6e740 10 client.26273 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:26:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.375+0000 7f4970c6e740 10 client.26273 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:26:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.375+0000 7f4970c6e740 20 client.26273 populate_metadata read hostname 'smithi114' 2024-09-13T18:26:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.375+0000 7f4970c6e740 1 --2- 172.21.15.114:0/269851557 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5620d6228c00 0x5620d622b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.375+0000 7f4970c6e740 4 client.26273 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T18:26:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.375+0000 7f4970c6e740 1 -- 172.21.15.114:0/269851557 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5620d5e92600 con 0x5620d6228c00 2024-09-13T18:26:22.381 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.378+0000 7f4966ffd640 1 --2- 172.21.15.114:0/269851557 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5620d6228c00 0x5620d622b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.379+0000 7f4966ffd640 1 --2- 172.21.15.114:0/269851557 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5620d6228c00 0x5620d622b020 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=732bd610b96ff9 server_cookie=ee3da14f676015fe in_seq=0 out_seq=0 2024-09-13T18:26:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.379+0000 7f4946ffd640 10 client.26273 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:26:22.394 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.390+0000 7f4946ffd640 1 -- 172.21.15.114:0/269851557 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5620d5e92600 con 0x5620d6228c00 2024-09-13T18:26:22.394 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.390+0000 7f4946ffd640 10 client.26273 handle_command_reply: tid=0 2024-09-13T18:26:22.394 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.390+0000 7f4970c6e740 1 client.26273 shutdown 2024-09-13T18:26:22.396 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.391+0000 7f4970c6e740 1 -- 172.21.15.114:0/269851557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620d6232630 msgr2=0x5620d6242e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:22.396 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.391+0000 7f4970c6e740 1 --2- 172.21.15.114:0/269851557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620d6232630 0x5620d6242e20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f495c004700 tx=0x7f495c01af40 comp rx=0 tx=0).stop 2024-09-13T18:26:22.396 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.391+0000 7f4970c6e740 1 -- 172.21.15.114:0/269851557 shutdown_connections 2024-09-13T18:26:22.396 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.391+0000 7f4970c6e740 1 --2- 172.21.15.114:0/269851557 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5620d6228c00 0x5620d622b020 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:22.396 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.391+0000 7f4970c6e740 1 --2- 172.21.15.114:0/269851557 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620d623ced0 0x5620d6243360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:22.396 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.391+0000 7f4970c6e740 1 --2- 172.21.15.114:0/269851557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620d6232630 0x5620d6242e20 secure :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f495c004700 tx=0x7f495c01af40 comp rx=0 tx=0).stop 2024-09-13T18:26:22.396 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.391+0000 7f4970c6e740 1 --2- 172.21.15.114:0/269851557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620d6231d10 0x5620d62428e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:22.397 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.391+0000 7f4970c6e740 1 -- 172.21.15.114:0/269851557 >> 172.21.15.114:0/269851557 conn(0x5620d622e2a0 msgr2=0x5620d6220c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:22.397 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.391+0000 7f4946ffd640 0 client.26273 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:26:22.397 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.391+0000 7f4970c6e740 1 -- 172.21.15.114:0/269851557 shutdown_connections 2024-09-13T18:26:22.397 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.391+0000 7f4970c6e740 1 -- 172.21.15.114:0/269851557 wait complete. 2024-09-13T18:26:22.397 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.392+0000 7f4970c6e740 20 client.26273 trim_cache size 0 max 16384 2024-09-13T18:26:22.397 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-13T18:26:22.397 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-13T18:26:22.398 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-13T18:26:22.398 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-13T18:26:22.398 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.392+0000 7f496df4c640 1 -- 172.21.15.114:0/4183280175 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f49580a1300 msgr2=0x7f49580a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:22.398 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.392+0000 7f496df4c640 1 --2- 172.21.15.114:0/4183280175 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f49580a1300 0x7f49580a37c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f496804efc0 tx=0x7f49680f9040 comp rx=0 tx=0).stop 2024-09-13T18:26:22.398 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.392+0000 7f496df4c640 1 -- 172.21.15.114:0/4183280175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49600abb50 msgr2=0x7f496013dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:22.398 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.392+0000 7f496df4c640 1 --2- 172.21.15.114:0/4183280175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49600abb50 0x7f496013dfc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4968066a70 tx=0x7f496806f6a0 comp rx=0 tx=0).stop 2024-09-13T18:26:22.399 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.392+0000 7f496df4c640 1 -- 172.21.15.114:0/4183280175 shutdown_connections 2024-09-13T18:26:22.399 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.392+0000 7f496df4c640 1 --2- 172.21.15.114:0/4183280175 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f49580a1300 0x7f49580a37c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:22.399 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.393+0000 7f496df4c640 1 --2- 172.21.15.114:0/4183280175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49600abb50 0x7f496013dfc0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:22.399 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.393+0000 7f496df4c640 1 --2- 172.21.15.114:0/4183280175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49600ab1d0 0x7f496013ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:22.399 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.393+0000 7f496df4c640 1 --2- 172.21.15.114:0/4183280175 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49600aa5f0 0x7f496013a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:22.399 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.393+0000 7f496df4c640 1 -- 172.21.15.114:0/4183280175 >> 172.21.15.114:0/4183280175 conn(0x7f496001a0a0 msgr2=0x7f49600a3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:22.399 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.393+0000 7f496df4c640 1 -- 172.21.15.114:0/4183280175 shutdown_connections 2024-09-13T18:26:22.399 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:22.393+0000 7f496df4c640 1 -- 172.21.15.114:0/4183280175 wait complete. 2024-09-13T18:26:22.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:26:22 smithi114 ceph-mon[30897]: pgmap v120: 129 pgs: 129 active+clean; 14 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 44 MiB/s wr, 716 op/s 2024-09-13T18:26:22.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:26:22 smithi134 ceph-mon[34283]: pgmap v120: 129 pgs: 129 active+clean; 14 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 44 MiB/s wr, 716 op/s 2024-09-13T18:26:22.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:26:22 smithi204 ceph-mon[34777]: pgmap v120: 129 pgs: 129 active+clean; 14 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 44 MiB/s wr, 716 op/s 2024-09-13T18:26:25.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:26:25 smithi114 ceph-mon[30897]: pgmap v121: 129 pgs: 129 active+clean; 14 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 61 MiB/s wr, 989 op/s 2024-09-13T18:26:25.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:26:25 smithi134 ceph-mon[34283]: pgmap v121: 129 pgs: 129 active+clean; 14 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 61 MiB/s wr, 989 op/s 2024-09-13T18:26:25.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:26:25 smithi204 ceph-mon[34777]: pgmap v121: 129 pgs: 129 active+clean; 14 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 61 MiB/s wr, 989 op/s 2024-09-13T18:26:26.441 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T18:26:26.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.649+0000 7f393f39c640 1 -- 172.21.15.114:0/4241806705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3938107890 msgr2=0x7f3938107cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:26.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.649+0000 7f393f39c640 1 --2- 172.21.15.114:0/4241806705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3938107890 0x7f3938107cd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f393001acb0 tx=0x7f393003b920 comp rx=0 tx=0).stop 2024-09-13T18:26:26.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.649+0000 7f393f39c640 1 -- 172.21.15.114:0/4241806705 shutdown_connections 2024-09-13T18:26:26.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.649+0000 7f393f39c640 1 --2- 172.21.15.114:0/4241806705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3938119400 0x7f393811b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.650+0000 7f393f39c640 1 --2- 172.21.15.114:0/4241806705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3938107890 0x7f3938107cd0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.650+0000 7f393f39c640 1 --2- 172.21.15.114:0/4241806705 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39381074b0 0x7f393810e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.650+0000 7f393f39c640 1 -- 172.21.15.114:0/4241806705 >> 172.21.15.114:0/4241806705 conn(0x7f393806af20 msgr2=0x7f393806b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:26.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.650+0000 7f393f39c640 1 -- 172.21.15.114:0/4241806705 shutdown_connections 2024-09-13T18:26:26.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.650+0000 7f393f39c640 1 -- 172.21.15.114:0/4241806705 wait complete. 2024-09-13T18:26:26.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.651+0000 7f393f39c640 1 Processor -- start 2024-09-13T18:26:26.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.651+0000 7f393f39c640 1 -- start start 2024-09-13T18:26:26.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.651+0000 7f393f39c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39381074b0 0x7f3938114b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:26.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.651+0000 7f393f39c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3938107890 0x7f3938115070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:26.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.651+0000 7f393f39c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3938119400 0x7f39381a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:26.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.651+0000 7f393f39c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39381a1710 con 0x7f3938107890 2024-09-13T18:26:26.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.651+0000 7f393f39c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39381a1880 con 0x7f3938119400 2024-09-13T18:26:26.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.651+0000 7f393f39c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39381a1a40 con 0x7f39381074b0 2024-09-13T18:26:26.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.651+0000 7f393c937640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3938107890 0x7f3938115070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:26.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.651+0000 7f393d138640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39381074b0 0x7f3938114b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:26.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.651+0000 7f393d138640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39381074b0 0x7f3938114b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:52486/0 (socket says 172.21.15.114:52486) 2024-09-13T18:26:26.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.651+0000 7f393d138640 1 -- 172.21.15.114:0/2047224939 learned_addr learned my addr 172.21.15.114:0/2047224939 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:26.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.652+0000 7f393d939640 1 --2- 172.21.15.114:0/2047224939 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3938119400 0x7f39381a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:26.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.652+0000 7f393d138640 1 -- 172.21.15.114:0/2047224939 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3938119400 msgr2=0x7f39381a1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:26.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.652+0000 7f393d138640 1 --2- 172.21.15.114:0/2047224939 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3938119400 0x7f39381a1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.652+0000 7f393d138640 1 -- 172.21.15.114:0/2047224939 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3938107890 msgr2=0x7f3938115070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:26.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.652+0000 7f393d138640 1 --2- 172.21.15.114:0/2047224939 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3938107890 0x7f3938115070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.652+0000 7f393d138640 1 -- 172.21.15.114:0/2047224939 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f393001a910 con 0x7f39381074b0 2024-09-13T18:26:26.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.652+0000 7f393d138640 1 --2- 172.21.15.114:0/2047224939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39381074b0 0x7f3938114b30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f392800d950 tx=0x7f392800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:26.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.654+0000 7f392e7fc640 1 -- 172.21.15.114:0/2047224939 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3928009c30 con 0x7f39381074b0 2024-09-13T18:26:26.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.654+0000 7f392e7fc640 1 -- 172.21.15.114:0/2047224939 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3928009d90 con 0x7f39381074b0 2024-09-13T18:26:26.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.654+0000 7f392e7fc640 1 -- 172.21.15.114:0/2047224939 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f392800f620 con 0x7f39381074b0 2024-09-13T18:26:26.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.654+0000 7f393f39c640 1 -- 172.21.15.114:0/2047224939 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39381a1d20 con 0x7f39381074b0 2024-09-13T18:26:26.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.654+0000 7f393f39c640 1 -- 172.21.15.114:0/2047224939 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39381a92e0 con 0x7f39381074b0 2024-09-13T18:26:26.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.658+0000 7f392e7fc640 1 -- 172.21.15.114:0/2047224939 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f392800f780 con 0x7f39381074b0 2024-09-13T18:26:26.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.659+0000 7f392e7fc640 1 --2- 172.21.15.114:0/2047224939 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f39240a1410 0x7f39240a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:26.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.659+0000 7f392e7fc640 1 -- 172.21.15.114:0/2047224939 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f39280cda00 con 0x7f39381074b0 2024-09-13T18:26:26.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.660+0000 7f393c937640 1 --2- 172.21.15.114:0/2047224939 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f39240a1410 0x7f39240a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:26.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.661+0000 7f39420be740 1 Processor -- start 2024-09-13T18:26:26.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.661+0000 7f393c937640 1 --2- 172.21.15.114:0/2047224939 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f39240a1410 0x7f39240a38d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3930005e30 tx=0x7f393004c040 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:26.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.661+0000 7f39420be740 1 -- start start 2024-09-13T18:26:26.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.661+0000 7f39420be740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e5476bfe0 0x557e5476c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:26.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.661+0000 7f39420be740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e5476c900 0x557e54776c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:26.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.661+0000 7f39420be740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e547771a0 0x557e54779590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:26.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.661+0000 7f39420be740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e546d6f20 con 0x557e547771a0 2024-09-13T18:26:26.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.661+0000 7f39420be740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e546d46b0 con 0x557e5476c900 2024-09-13T18:26:26.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.661+0000 7f39420be740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e546cffa0 con 0x557e5476bfe0 2024-09-13T18:26:26.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.661+0000 7f393c937640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e5476c900 0x557e54776c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:26.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.661+0000 7f393c937640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e5476c900 0x557e54776c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:60476/0 (socket says 172.21.15.114:60476) 2024-09-13T18:26:26.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.661+0000 7f393c937640 1 -- 172.21.15.114:0/3456230146 learned_addr learned my addr 172.21.15.114:0/3456230146 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:26.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.662+0000 7f393d939640 1 --2- 172.21.15.114:0/3456230146 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e547771a0 0x557e54779590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:26.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.662+0000 7f393d138640 1 --2- 172.21.15.114:0/3456230146 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e5476bfe0 0x557e5476c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:26.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.662+0000 7f393c937640 1 -- 172.21.15.114:0/3456230146 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e5476bfe0 msgr2=0x557e5476c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:26.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.662+0000 7f393c937640 1 --2- 172.21.15.114:0/3456230146 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e5476bfe0 0x557e5476c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.662+0000 7f393c937640 1 -- 172.21.15.114:0/3456230146 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e547771a0 msgr2=0x557e54779590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:26.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.662+0000 7f393c937640 1 --2- 172.21.15.114:0/3456230146 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e547771a0 0x557e54779590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.662+0000 7f393c937640 1 -- 172.21.15.114:0/3456230146 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e5468a840 con 0x557e5476c900 2024-09-13T18:26:26.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.662+0000 7f393d939640 1 --2- 172.21.15.114:0/3456230146 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e547771a0 0x557e54779590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:26:26.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.662+0000 7f393c937640 1 --2- 172.21.15.114:0/3456230146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e5476c900 0x557e54776c60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f393001acb0 tx=0x7f3930050000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6b23bbae4186b96 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:26.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.662+0000 7f39237fe640 1 -- 172.21.15.114:0/3456230146 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393005a070 con 0x557e5476c900 2024-09-13T18:26:26.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.663+0000 7f39237fe640 1 -- 172.21.15.114:0/3456230146 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3930041620 con 0x557e5476c900 2024-09-13T18:26:26.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.663+0000 7f39237fe640 1 -- 172.21.15.114:0/3456230146 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393004f040 con 0x557e5476c900 2024-09-13T18:26:26.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.664+0000 7f39420be740 1 -- 172.21.15.114:0/3456230146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e5476c900 msgr2=0x557e54776c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:26.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.664+0000 7f39420be740 1 --2- 172.21.15.114:0/3456230146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e5476c900 0x557e54776c60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f393001acb0 tx=0x7f3930050000 comp rx=0 tx=0).stop 2024-09-13T18:26:26.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.664+0000 7f39420be740 1 -- 172.21.15.114:0/3456230146 shutdown_connections 2024-09-13T18:26:26.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.664+0000 7f39420be740 1 --2- 172.21.15.114:0/3456230146 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e547771a0 0x557e54779590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.664+0000 7f39420be740 1 --2- 172.21.15.114:0/3456230146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e5476c900 0x557e54776c60 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.664+0000 7f39420be740 1 --2- 172.21.15.114:0/3456230146 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e5476bfe0 0x557e5476c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.664+0000 7f39420be740 1 -- 172.21.15.114:0/3456230146 >> 172.21.15.114:0/3456230146 conn(0x557e54768570 msgr2=0x557e54768950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:26.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.664+0000 7f39420be740 1 -- 172.21.15.114:0/3456230146 shutdown_connections 2024-09-13T18:26:26.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.664+0000 7f39420be740 1 -- 172.21.15.114:0/3456230146 wait complete. 2024-09-13T18:26:26.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.665+0000 7f39420be740 1 -- start start 2024-09-13T18:26:26.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.665+0000 7f39420be740 1 Processor -- start 2024-09-13T18:26:26.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.666+0000 7f39420be740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e5476bfe0 0x557e54776a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:26.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.666+0000 7f39420be740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e5476c900 0x557e54773d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:26.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.666+0000 7f39420be740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e547771a0 0x557e547742d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:26.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.666+0000 7f39420be740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e546712a0 con 0x557e547771a0 2024-09-13T18:26:26.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.666+0000 7f39420be740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e543a8430 con 0x557e5476bfe0 2024-09-13T18:26:26.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.666+0000 7f39420be740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e54653680 con 0x557e5476c900 2024-09-13T18:26:26.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.666+0000 7f39420be740 10 client.0 _ll_register_callbacks cb 0x557e546c9610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:26:26.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.666+0000 7f393c937640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e5476c900 0x557e54773d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:26.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.666+0000 7f393c937640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e5476c900 0x557e54773d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:52500/0 (socket says 172.21.15.114:52500) 2024-09-13T18:26:26.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.666+0000 7f393c937640 1 -- 172.21.15.114:0/3778023912 learned_addr learned my addr 172.21.15.114:0/3778023912 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:26.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.667+0000 7f393d138640 1 --2- 172.21.15.114:0/3778023912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e5476bfe0 0x557e54776a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:26.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.667+0000 7f393d939640 1 --2- 172.21.15.114:0/3778023912 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e547771a0 0x557e547742d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:26.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.667+0000 7f393c937640 1 -- 172.21.15.114:0/3778023912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e5476bfe0 msgr2=0x557e54776a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:26.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.667+0000 7f393c937640 1 --2- 172.21.15.114:0/3778023912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e5476bfe0 0x557e54776a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.667+0000 7f393c937640 1 -- 172.21.15.114:0/3778023912 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e547771a0 msgr2=0x557e547742d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:26.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.667+0000 7f393c937640 1 --2- 172.21.15.114:0/3778023912 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e547771a0 0x557e547742d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.667+0000 7f393c937640 1 -- 172.21.15.114:0/3778023912 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f393811c140 con 0x557e5476c900 2024-09-13T18:26:26.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.667+0000 7f393d939640 1 --2- 172.21.15.114:0/3778023912 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e547771a0 0x557e547742d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:26:26.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.667+0000 7f393c937640 1 --2- 172.21.15.114:0/3778023912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e5476c900 0x557e54773d90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3930062040 tx=0x7f3930040e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca838183d54d1aba server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:26.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.667+0000 7f3920ff9640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:26:26.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.668+0000 7f3920ff9640 1 -- 172.21.15.114:0/3778023912 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393005a040 con 0x557e5476c900 2024-09-13T18:26:26.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.668+0000 7f3920ff9640 1 -- 172.21.15.114:0/3778023912 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f39300043d0 con 0x557e5476c900 2024-09-13T18:26:26.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.668+0000 7f39420be740 1 -- 172.21.15.114:0/3778023912 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557e543c34e0 con 0x557e5476c900 2024-09-13T18:26:26.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.668+0000 7f3920ff9640 1 -- 172.21.15.114:0/3778023912 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3930004590 con 0x557e5476c900 2024-09-13T18:26:26.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.669+0000 7f39420be740 10 client.26299 fetch_fsmap learned FSMap version 32 2024-09-13T18:26:26.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.669+0000 7f39420be740 1 -- 172.21.15.114:0/3778023912 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x557e546ce620 con 0x557e5476c900 2024-09-13T18:26:26.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.670+0000 7f3920ff9640 1 -- 172.21.15.114:0/3778023912 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3930076020 con 0x557e5476c900 2024-09-13T18:26:26.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.670+0000 7f3920ff9640 1 -- 172.21.15.114:0/3778023912 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f3930066030 con 0x557e5476c900 2024-09-13T18:26:26.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.670+0000 7f39420be740 10 client.26299 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:26:26.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.670+0000 7f39420be740 10 client.26299 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:26:26.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.670+0000 7f39420be740 20 client.26299 populate_metadata read hostname 'smithi114' 2024-09-13T18:26:26.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.670+0000 7f39420be740 1 --2- 172.21.15.114:0/3778023912 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x557e54786d60 0x557e54789180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:26.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.670+0000 7f39420be740 4 client.26299 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:26:26.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.670+0000 7f39420be740 1 -- 172.21.15.114:0/3778023912 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557e543d47e0 con 0x557e54786d60 2024-09-13T18:26:26.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.670+0000 7f393d138640 1 --2- 172.21.15.114:0/3778023912 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x557e54786d60 0x557e54789180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:26.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.671+0000 7f393d138640 1 --2- 172.21.15.114:0/3778023912 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x557e54786d60 0x557e54789180 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ead654258f11050 server_cookie=269f3e16a6ce62a in_seq=0 out_seq=0 2024-09-13T18:26:26.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.671+0000 7f3920ff9640 10 client.26299 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:26:26.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.672+0000 7f3920ff9640 1 -- 172.21.15.114:0/3778023912 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x557e543d47e0 con 0x557e54786d60 2024-09-13T18:26:26.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.672+0000 7f3920ff9640 10 client.26299 handle_command_reply: tid=0 2024-09-13T18:26:26.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.672+0000 7f39420be740 1 client.26299 shutdown 2024-09-13T18:26:26.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.673+0000 7f39420be740 1 -- 172.21.15.114:0/3778023912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e5476c900 msgr2=0x557e54773d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:26.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.673+0000 7f39420be740 1 --2- 172.21.15.114:0/3778023912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e5476c900 0x557e54773d90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3930062040 tx=0x7f3930040e50 comp rx=0 tx=0).stop 2024-09-13T18:26:26.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.673+0000 7f39420be740 1 -- 172.21.15.114:0/3778023912 shutdown_connections 2024-09-13T18:26:26.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.673+0000 7f39420be740 1 --2- 172.21.15.114:0/3778023912 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x557e54786d60 0x557e54789180 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.673+0000 7f39420be740 1 --2- 172.21.15.114:0/3778023912 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e547771a0 0x557e547742d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.673+0000 7f39420be740 1 --2- 172.21.15.114:0/3778023912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e5476c900 0x557e54773d90 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.673+0000 7f39420be740 1 --2- 172.21.15.114:0/3778023912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e5476bfe0 0x557e54776a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.673+0000 7f39420be740 1 -- 172.21.15.114:0/3778023912 >> 172.21.15.114:0/3778023912 conn(0x557e54768570 msgr2=0x557e54767760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:26.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.673+0000 7f3920ff9640 0 client.26299 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:26:26.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.674+0000 7f39420be740 1 -- 172.21.15.114:0/3778023912 shutdown_connections 2024-09-13T18:26:26.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.674+0000 7f39420be740 1 -- 172.21.15.114:0/3778023912 wait complete. 2024-09-13T18:26:26.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.674+0000 7f39420be740 20 client.26299 trim_cache size 0 max 16384 2024-09-13T18:26:26.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.710+0000 7f39420be740 1 Processor -- start 2024-09-13T18:26:26.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.710+0000 7f39420be740 1 -- start start 2024-09-13T18:26:26.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.711+0000 7f39420be740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e5476bfe0 0x557e54767f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:26.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.711+0000 7f39420be740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e5476c900 0x557e5477df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:26.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.711+0000 7f393d138640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e5476bfe0 0x557e54767f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:26.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.711+0000 7f393d138640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e5476bfe0 0x557e54767f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49280/0 (socket says 172.21.15.114:49280) 2024-09-13T18:26:26.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.711+0000 7f393d138640 1 -- 172.21.15.114:0/4290806772 learned_addr learned my addr 172.21.15.114:0/4290806772 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:26.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.711+0000 7f39420be740 1 --2- 172.21.15.114:0/4290806772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e547771a0 0x557e5477e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:26.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.711+0000 7f39420be740 1 -- 172.21.15.114:0/4290806772 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e54783c80 con 0x557e5476bfe0 2024-09-13T18:26:26.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.711+0000 7f39420be740 1 -- 172.21.15.114:0/4290806772 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e54763ee0 con 0x557e547771a0 2024-09-13T18:26:26.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.711+0000 7f39420be740 1 -- 172.21.15.114:0/4290806772 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e54774640 con 0x557e5476c900 2024-09-13T18:26:26.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.712+0000 7f393d939640 1 --2- 172.21.15.114:0/4290806772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e547771a0 0x557e5477e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:26.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.712+0000 7f393d939640 1 -- 172.21.15.114:0/4290806772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e5476c900 msgr2=0x557e5477df90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:26:26.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.712+0000 7f393d939640 1 --2- 172.21.15.114:0/4290806772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e5476c900 0x557e5477df90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.712+0000 7f393d939640 1 -- 172.21.15.114:0/4290806772 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e5476bfe0 msgr2=0x557e54767f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:26.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.712+0000 7f393d939640 1 --2- 172.21.15.114:0/4290806772 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e5476bfe0 0x557e54767f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.712+0000 7f393d939640 1 -- 172.21.15.114:0/4290806772 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e54670790 con 0x557e547771a0 2024-09-13T18:26:26.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.713+0000 7f393d939640 1 --2- 172.21.15.114:0/4290806772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e547771a0 0x557e5477e4d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x557e54769660 tx=0x7f3934010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64812803a4d10ad9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:26.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.713+0000 7f3923fff640 1 -- 172.21.15.114:0/4290806772 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3934004220 con 0x557e547771a0 2024-09-13T18:26:26.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.714+0000 7f3923fff640 1 -- 172.21.15.114:0/4290806772 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3934018040 con 0x557e547771a0 2024-09-13T18:26:26.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.714+0000 7f3923fff640 1 -- 172.21.15.114:0/4290806772 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393401c3f0 con 0x557e547771a0 2024-09-13T18:26:26.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.716+0000 7f39420be740 1 -- 172.21.15.114:0/4290806772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e547771a0 msgr2=0x557e5477e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:26.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.716+0000 7f39420be740 1 --2- 172.21.15.114:0/4290806772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e547771a0 0x557e5477e4d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x557e54769660 tx=0x7f3934010800 comp rx=0 tx=0).stop 2024-09-13T18:26:26.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.716+0000 7f39420be740 1 -- 172.21.15.114:0/4290806772 shutdown_connections 2024-09-13T18:26:26.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.716+0000 7f39420be740 1 --2- 172.21.15.114:0/4290806772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e547771a0 0x557e5477e4d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.716+0000 7f39420be740 1 --2- 172.21.15.114:0/4290806772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e5476c900 0x557e5477df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.716+0000 7f39420be740 1 --2- 172.21.15.114:0/4290806772 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e5476bfe0 0x557e54767f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.716+0000 7f39420be740 1 -- 172.21.15.114:0/4290806772 >> 172.21.15.114:0/4290806772 conn(0x557e54768570 msgr2=0x557e5478dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:26.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.718+0000 7f39420be740 1 -- 172.21.15.114:0/4290806772 shutdown_connections 2024-09-13T18:26:26.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.718+0000 7f39420be740 1 -- 172.21.15.114:0/4290806772 wait complete. 2024-09-13T18:26:26.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.718+0000 7f39420be740 1 -- start start 2024-09-13T18:26:26.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.719+0000 7f39420be740 1 Processor -- start 2024-09-13T18:26:26.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.719+0000 7f39420be740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e5476bfe0 0x557e5477e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:26.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.719+0000 7f39420be740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e5476c900 0x557e5477a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:26.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.719+0000 7f39420be740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e547771a0 0x557e5477aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:26.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.719+0000 7f39420be740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e5477afb0 con 0x557e5476c900 2024-09-13T18:26:26.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.719+0000 7f39420be740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e5477b120 con 0x557e547771a0 2024-09-13T18:26:26.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.719+0000 7f39420be740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e5477b2e0 con 0x557e5476bfe0 2024-09-13T18:26:26.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.719+0000 7f39420be740 10 client.0 _ll_register_callbacks cb 0x7f3930041f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:26:26.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.720+0000 7f393d939640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e547771a0 0x557e5477aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:26.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.720+0000 7f393d939640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e547771a0 0x557e5477aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:60508/0 (socket says 172.21.15.114:60508) 2024-09-13T18:26:26.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.720+0000 7f393d939640 1 -- 172.21.15.114:0/2505843938 learned_addr learned my addr 172.21.15.114:0/2505843938 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:26.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.720+0000 7f393d138640 1 --2- 172.21.15.114:0/2505843938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e5476bfe0 0x557e5477e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:26.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.720+0000 7f393d939640 1 -- 172.21.15.114:0/2505843938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e5476bfe0 msgr2=0x557e5477e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:26.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.720+0000 7f393d939640 1 --2- 172.21.15.114:0/2505843938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e5476bfe0 0x557e5477e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.720+0000 7f393d939640 1 -- 172.21.15.114:0/2505843938 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e5476c900 msgr2=0x557e5477a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:26.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.720+0000 7f393d939640 1 --2- 172.21.15.114:0/2505843938 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e5476c900 0x557e5477a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.720+0000 7f393d939640 1 -- 172.21.15.114:0/2505843938 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39340099b0 con 0x557e547771a0 2024-09-13T18:26:26.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.721+0000 7f393d939640 1 --2- 172.21.15.114:0/2505843938 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e547771a0 0x557e5477aa70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3934000c00 tx=0x7f3934003af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f2203450782fb2f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:26.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.721+0000 7f3920ff9640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:26:26.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.721+0000 7f3920ff9640 1 -- 172.21.15.114:0/2505843938 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3934004430 con 0x557e547771a0 2024-09-13T18:26:26.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.721+0000 7f3920ff9640 1 -- 172.21.15.114:0/2505843938 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f3934005040 con 0x557e547771a0 2024-09-13T18:26:26.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.721+0000 7f39420be740 1 -- 172.21.15.114:0/2505843938 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3930041620 con 0x557e547771a0 2024-09-13T18:26:26.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.722+0000 7f3920ff9640 1 -- 172.21.15.114:0/2505843938 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3934005200 con 0x557e547771a0 2024-09-13T18:26:26.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.722+0000 7f39420be740 10 client.26291 fetch_fsmap learned FSMap version 32 2024-09-13T18:26:26.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.722+0000 7f39420be740 1 -- 172.21.15.114:0/2505843938 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x557e54695ea0 con 0x557e547771a0 2024-09-13T18:26:26.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.723+0000 7f3920ff9640 1 -- 172.21.15.114:0/2505843938 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3934005360 con 0x557e547771a0 2024-09-13T18:26:26.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.723+0000 7f3920ff9640 1 -- 172.21.15.114:0/2505843938 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f3934024040 con 0x557e547771a0 2024-09-13T18:26:26.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.723+0000 7f39420be740 10 client.26291 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:26:26.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.723+0000 7f39420be740 10 client.26291 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:26:26.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.723+0000 7f39420be740 20 client.26291 populate_metadata read hostname 'smithi114' 2024-09-13T18:26:26.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.723+0000 7f39420be740 1 --2- 172.21.15.114:0/2505843938 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x557e547de730 0x557e547e0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:26.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.723+0000 7f39420be740 4 client.26291 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T18:26:26.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.723+0000 7f39420be740 1 -- 172.21.15.114:0/2505843938 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f3930041620 con 0x557e547de730 2024-09-13T18:26:26.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.724+0000 7f393d138640 1 --2- 172.21.15.114:0/2505843938 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x557e547de730 0x557e547e0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:26.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.725+0000 7f393d138640 1 --2- 172.21.15.114:0/2505843938 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x557e547de730 0x557e547e0b50 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b587ff99111df49f server_cookie=8c96c493ca2ea8e3 in_seq=0 out_seq=0 2024-09-13T18:26:26.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.725+0000 7f3920ff9640 10 client.26291 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:26:26.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.725+0000 7f3920ff9640 1 -- 172.21.15.114:0/2505843938 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x7f3930041620 con 0x557e547de730 2024-09-13T18:26:26.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.725+0000 7f3920ff9640 10 client.26291 handle_command_reply: tid=0 2024-09-13T18:26:26.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.725+0000 7f39420be740 1 client.26291 shutdown 2024-09-13T18:26:26.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.726+0000 7f39420be740 1 -- 172.21.15.114:0/2505843938 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e547771a0 msgr2=0x557e5477aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:26.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.726+0000 7f39420be740 1 --2- 172.21.15.114:0/2505843938 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e547771a0 0x557e5477aa70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3934000c00 tx=0x7f3934003af0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.728+0000 7f39420be740 1 -- 172.21.15.114:0/2505843938 shutdown_connections 2024-09-13T18:26:26.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.728+0000 7f39420be740 1 --2- 172.21.15.114:0/2505843938 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x557e547de730 0x557e547e0b50 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.728+0000 7f39420be740 1 --2- 172.21.15.114:0/2505843938 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e547771a0 0x557e5477aa70 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.728+0000 7f3920ff9640 0 client.26291 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:26:26.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.728+0000 7f39420be740 1 --2- 172.21.15.114:0/2505843938 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e5476c900 0x557e5477a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.728+0000 7f39420be740 1 --2- 172.21.15.114:0/2505843938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557e5476bfe0 0x557e5477e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.728+0000 7f39420be740 1 -- 172.21.15.114:0/2505843938 >> 172.21.15.114:0/2505843938 conn(0x557e54768570 msgr2=0x557e5476f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:26.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.731+0000 7f39420be740 1 -- 172.21.15.114:0/2505843938 shutdown_connections 2024-09-13T18:26:26.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.731+0000 7f39420be740 1 -- 172.21.15.114:0/2505843938 wait complete. 2024-09-13T18:26:26.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.732+0000 7f39420be740 20 client.26291 trim_cache size 0 max 16384 2024-09-13T18:26:26.739 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:26:26.739 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "no active scrubs running", 2024-09-13T18:26:26.740 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": {} 2024-09-13T18:26:26.740 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:26:26.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.738+0000 7f393f39c640 1 -- 172.21.15.114:0/2047224939 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f39240a1410 msgr2=0x7f39240a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:26.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.738+0000 7f393f39c640 1 --2- 172.21.15.114:0/2047224939 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f39240a1410 0x7f39240a38d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3930005e30 tx=0x7f393004c040 comp rx=0 tx=0).stop 2024-09-13T18:26:26.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.738+0000 7f393f39c640 1 -- 172.21.15.114:0/2047224939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39381074b0 msgr2=0x7f3938114b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:26.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.738+0000 7f393f39c640 1 --2- 172.21.15.114:0/2047224939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39381074b0 0x7f3938114b30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f392800d950 tx=0x7f392800de10 comp rx=0 tx=0).stop 2024-09-13T18:26:26.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.738+0000 7f393f39c640 1 -- 172.21.15.114:0/2047224939 shutdown_connections 2024-09-13T18:26:26.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.738+0000 7f393f39c640 1 --2- 172.21.15.114:0/2047224939 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f39240a1410 0x7f39240a38d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.738+0000 7f393f39c640 1 --2- 172.21.15.114:0/2047224939 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3938119400 0x7f39381a1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.738+0000 7f393f39c640 1 --2- 172.21.15.114:0/2047224939 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3938107890 0x7f3938115070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.738+0000 7f393f39c640 1 --2- 172.21.15.114:0/2047224939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39381074b0 0x7f3938114b30 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.738+0000 7f393f39c640 1 -- 172.21.15.114:0/2047224939 >> 172.21.15.114:0/2047224939 conn(0x7f393806af20 msgr2=0x7f39380701d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:26.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.738+0000 7f393f39c640 1 -- 172.21.15.114:0/2047224939 shutdown_connections 2024-09-13T18:26:26.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.738+0000 7f393f39c640 1 -- 172.21.15.114:0/2047224939 wait complete. 2024-09-13T18:26:26.773 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-13T18:26:26.774 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:26:26.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.915+0000 7fb21797d640 1 -- 172.21.15.114:0/3857260854 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2100ff860 msgr2=0x7fb2101038a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:26.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.915+0000 7fb21797d640 1 --2- 172.21.15.114:0/3857260854 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2100ff860 0x7fb2101038a0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fb200009930 tx=0x7fb20002a420 comp rx=0 tx=0).stop 2024-09-13T18:26:26.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.915+0000 7fb21797d640 1 -- 172.21.15.114:0/3857260854 shutdown_connections 2024-09-13T18:26:26.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.915+0000 7fb21797d640 1 --2- 172.21.15.114:0/3857260854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb210104000 0x7fb21010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.915+0000 7fb21797d640 1 --2- 172.21.15.114:0/3857260854 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2100ff860 0x7fb2101038a0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.915+0000 7fb21797d640 1 --2- 172.21.15.114:0/3857260854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2100feeb0 0x7fb2100ff290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.915+0000 7fb21797d640 1 -- 172.21.15.114:0/3857260854 >> 172.21.15.114:0/3857260854 conn(0x7fb2100fa900 msgr2=0x7fb2100fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:26.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.915+0000 7fb21797d640 1 -- 172.21.15.114:0/3857260854 shutdown_connections 2024-09-13T18:26:26.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.915+0000 7fb21797d640 1 -- 172.21.15.114:0/3857260854 wait complete. 2024-09-13T18:26:26.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.915+0000 7fb21797d640 1 Processor -- start 2024-09-13T18:26:26.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.915+0000 7fb21797d640 1 -- start start 2024-09-13T18:26:26.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.916+0000 7fb21797d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2100feeb0 0x7fb2101950c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:26.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.916+0000 7fb21797d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb210104000 0x7fb210195600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:26.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.916+0000 7fb21797d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2101999a0 0x7fb210199e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:26.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.916+0000 7fb21797d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb21019a390 con 0x7fb2100feeb0 2024-09-13T18:26:26.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.916+0000 7fb21797d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb21019a500 con 0x7fb2101999a0 2024-09-13T18:26:26.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.916+0000 7fb21797d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb21019a670 con 0x7fb210104000 2024-09-13T18:26:26.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.916+0000 7fb215719640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2100feeb0 0x7fb2101950c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:26.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.916+0000 7fb214f18640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb210104000 0x7fb210195600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:26.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.916+0000 7fb215719640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2100feeb0 0x7fb2101950c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49300/0 (socket says 172.21.15.114:49300) 2024-09-13T18:26:26.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.916+0000 7fb215719640 1 -- 172.21.15.114:0/2860698289 learned_addr learned my addr 172.21.15.114:0/2860698289 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:26.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.916+0000 7fb215f1a640 1 --2- 172.21.15.114:0/2860698289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2101999a0 0x7fb210199e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:26.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.916+0000 7fb215f1a640 1 -- 172.21.15.114:0/2860698289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb210104000 msgr2=0x7fb210195600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:26.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.916+0000 7fb215f1a640 1 --2- 172.21.15.114:0/2860698289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb210104000 0x7fb210195600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.916+0000 7fb215f1a640 1 -- 172.21.15.114:0/2860698289 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2100feeb0 msgr2=0x7fb2101950c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:26.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.916+0000 7fb215f1a640 1 --2- 172.21.15.114:0/2860698289 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2100feeb0 0x7fb2101950c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:26.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.916+0000 7fb215f1a640 1 -- 172.21.15.114:0/2860698289 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb200009560 con 0x7fb2101999a0 2024-09-13T18:26:26.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.917+0000 7fb215f1a640 1 --2- 172.21.15.114:0/2860698289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2101999a0 0x7fb210199e50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb20800e940 tx=0x7fb20800ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:26.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.917+0000 7fb2067fc640 1 -- 172.21.15.114:0/2860698289 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb208004d20 con 0x7fb2101999a0 2024-09-13T18:26:26.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.917+0000 7fb2067fc640 1 -- 172.21.15.114:0/2860698289 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb208004e80 con 0x7fb2101999a0 2024-09-13T18:26:26.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.917+0000 7fb2067fc640 1 -- 172.21.15.114:0/2860698289 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb208005280 con 0x7fb2101999a0 2024-09-13T18:26:26.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.917+0000 7fb21797d640 1 -- 172.21.15.114:0/2860698289 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb210068570 con 0x7fb2101999a0 2024-09-13T18:26:26.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.917+0000 7fb21797d640 1 -- 172.21.15.114:0/2860698289 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2100688e0 con 0x7fb2101999a0 2024-09-13T18:26:26.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.919+0000 7fb21797d640 1 -- 172.21.15.114:0/2860698289 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb210100380 con 0x7fb2101999a0 2024-09-13T18:26:26.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.920+0000 7fb2067fc640 1 -- 172.21.15.114:0/2860698289 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fb20800cb20 con 0x7fb2101999a0 2024-09-13T18:26:26.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.921+0000 7fb2067fc640 1 --2- 172.21.15.114:0/2860698289 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fb1fc0a13d0 0x7fb1fc0a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:26.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.922+0000 7fb2067fc640 1 -- 172.21.15.114:0/2860698289 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fb2080d0f10 con 0x7fb2101999a0 2024-09-13T18:26:26.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.924+0000 7fb215719640 1 --2- 172.21.15.114:0/2860698289 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fb1fc0a13d0 0x7fb1fc0a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:26.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.925+0000 7fb215719640 1 --2- 172.21.15.114:0/2860698289 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fb1fc0a13d0 0x7fb1fc0a3890 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb20c00c430 tx=0x7fb20c00e040 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:26.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:26.931+0000 7fb2067fc640 1 -- 172.21.15.114:0/2860698289 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fb20809a170 con 0x7fb2101999a0 2024-09-13T18:26:27.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:27.436+0000 7fb21797d640 1 -- 172.21.15.114:0/2860698289 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb210069380 con 0x7fb2101999a0 2024-09-13T18:26:27.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:27.439+0000 7fb2067fc640 1 -- 172.21.15.114:0/2860698289 <== mon.2 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+10937 (secure 0 0 0) 0x7fb20809d790 con 0x7fb2101999a0 2024-09-13T18:26:27.443 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:26:27.443 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4162817497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4162817497},{"type":"v1","addr":"172.21.15.114:6835","nonce":4162817497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/2435428478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":2435428478},{"type":"v1","addr":"172.21.15.114:6837","nonce":2435428478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T18:20:01.895117+0000","modified":"2024-09-13T18:24:01.324695+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24482,"mds_2":24475,"mds_3":24458,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/2046516451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":2046516451},{"type":"v1","addr":"172.21.15.114:6839","nonce":2046516451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":10,"addr":"172.21.15.114:6841/1956923446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":1956923446},{"type":"v1","addr":"172.21.15.114:6841","nonce":1956923446}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24391":{"gid":24391,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/507301496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":507301496},{"type":"v1","addr":"172.21.15.204:6833","nonce":507301496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4028657219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4028657219},{"type":"v1","addr":"172.21.15.134:6833","nonce":4028657219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/57029119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":57029119},{"type":"v1","addr":"172.21.15.204:6835","nonce":57029119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3068880939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3068880939},{"type":"v1","addr":"172.21.15.204:6837","nonce":3068880939}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"e","rank":3,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.134:6835/1150003275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1150003275},{"type":"v1","addr":"172.21.15.134:6835","nonce":1150003275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/344883912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":344883912},{"type":"v1","addr":"172.21.15.204:6839","nonce":344883912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6837/891692210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":891692210},{"type":"v1","addr":"172.21.15.134:6837","nonce":891692210}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6839/3466309724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3466309724},{"type":"v1","addr":"172.21.15.134:6839","nonce":3466309724}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:26:27.443 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 32 2024-09-13T18:26:27.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:27.443+0000 7fb21797d640 1 -- 172.21.15.114:0/2860698289 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fb1fc0a13d0 msgr2=0x7fb1fc0a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:27.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:27.443+0000 7fb21797d640 1 --2- 172.21.15.114:0/2860698289 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fb1fc0a13d0 0x7fb1fc0a3890 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb20c00c430 tx=0x7fb20c00e040 comp rx=0 tx=0).stop 2024-09-13T18:26:27.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:27.443+0000 7fb21797d640 1 -- 172.21.15.114:0/2860698289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2101999a0 msgr2=0x7fb210199e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:27.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:27.443+0000 7fb21797d640 1 --2- 172.21.15.114:0/2860698289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2101999a0 0x7fb210199e50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb20800e940 tx=0x7fb20800ee00 comp rx=0 tx=0).stop 2024-09-13T18:26:27.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:27.444+0000 7fb21797d640 1 -- 172.21.15.114:0/2860698289 shutdown_connections 2024-09-13T18:26:27.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:27.444+0000 7fb21797d640 1 --2- 172.21.15.114:0/2860698289 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fb1fc0a13d0 0x7fb1fc0a3890 secure :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb20c00c430 tx=0x7fb20c00e040 comp rx=0 tx=0).stop 2024-09-13T18:26:27.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:27.444+0000 7fb21797d640 1 --2- 172.21.15.114:0/2860698289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2101999a0 0x7fb210199e50 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:27.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:27.444+0000 7fb21797d640 1 --2- 172.21.15.114:0/2860698289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb210104000 0x7fb210195600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:27.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:27.444+0000 7fb21797d640 1 --2- 172.21.15.114:0/2860698289 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2100feeb0 0x7fb2101950c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:27.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:27.444+0000 7fb21797d640 1 -- 172.21.15.114:0/2860698289 >> 172.21.15.114:0/2860698289 conn(0x7fb2100fa900 msgr2=0x7fb2100fbdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:27.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:27.445+0000 7fb21797d640 1 -- 172.21.15.114:0/2860698289 shutdown_connections 2024-09-13T18:26:27.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:27.445+0000 7fb21797d640 1 -- 172.21.15.114:0/2860698289 wait complete. 2024-09-13T18:26:27.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:26:27 smithi114 ceph-mon[30897]: pgmap v122: 129 pgs: 129 active+clean; 14 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 49 MiB/s wr, 796 op/s 2024-09-13T18:26:27.475 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-09-13T18:26:27.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:26:27 smithi134 ceph-mon[34283]: pgmap v122: 129 pgs: 129 active+clean; 14 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 49 MiB/s wr, 796 op/s 2024-09-13T18:26:27.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:26:27 smithi204 ceph-mon[34777]: pgmap v122: 129 pgs: 129 active+clean; 14 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 49 MiB/s wr, 796 op/s 2024-09-13T18:26:28.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.394+0000 7f87b67fc640 1 -- 172.21.15.114:0/1821487874 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ac0308c0 con 0x7f87b8107890 2024-09-13T18:26:28.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.394+0000 7f87bcb9d640 1 -- 172.21.15.114:0/1821487874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87b8107890 msgr2=0x7f87b8107cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:28.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.394+0000 7f87bcb9d640 1 --2- 172.21.15.114:0/1821487874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87b8107890 0x7f87b8107cd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f87ac009860 tx=0x7f87ac02a360 comp rx=0 tx=0).stop 2024-09-13T18:26:28.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.396+0000 7f87bcb9d640 1 -- 172.21.15.114:0/1821487874 shutdown_connections 2024-09-13T18:26:28.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.396+0000 7f87bcb9d640 1 --2- 172.21.15.114:0/1821487874 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87b8119400 0x7f87b811b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.396+0000 7f87bcb9d640 1 --2- 172.21.15.114:0/1821487874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87b8107890 0x7f87b8107cd0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.396+0000 7f87bcb9d640 1 --2- 172.21.15.114:0/1821487874 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87b81074b0 0x7f87b810e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.396+0000 7f87bcb9d640 1 -- 172.21.15.114:0/1821487874 >> 172.21.15.114:0/1821487874 conn(0x7f87b806af40 msgr2=0x7f87b806b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:28.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.397+0000 7f87bcb9d640 1 -- 172.21.15.114:0/1821487874 shutdown_connections 2024-09-13T18:26:28.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.398+0000 7f87bcb9d640 1 -- 172.21.15.114:0/1821487874 wait complete. 2024-09-13T18:26:28.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.398+0000 7f87bcb9d640 1 Processor -- start 2024-09-13T18:26:28.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.399+0000 7f87bcb9d640 1 -- start start 2024-09-13T18:26:28.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.399+0000 7f87bcb9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87b81074b0 0x7f87b8116940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:28.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.399+0000 7f87bcb9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87b8107890 0x7f87b8116ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:28.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.399+0000 7f87bcb9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87b8119400 0x7f87b810fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:28.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.399+0000 7f87bcb9d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b81103d0 con 0x7f87b8119400 2024-09-13T18:26:28.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.399+0000 7f87bcb9d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b8110540 con 0x7f87b81074b0 2024-09-13T18:26:28.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.399+0000 7f87bcb9d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b8110700 con 0x7f87b8107890 2024-09-13T18:26:28.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.400+0000 7f87b77fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87b81074b0 0x7f87b8116940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:28.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.400+0000 7f87b7fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87b8119400 0x7f87b810fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:28.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.400+0000 7f87b6ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87b8107890 0x7f87b8116ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:28.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.400+0000 7f87b7fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87b8119400 0x7f87b810fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49318/0 (socket says 172.21.15.114:49318) 2024-09-13T18:26:28.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.400+0000 7f87b7fff640 1 -- 172.21.15.114:0/2999831595 learned_addr learned my addr 172.21.15.114:0/2999831595 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:28.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.400+0000 7f87b7fff640 1 -- 172.21.15.114:0/2999831595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87b8107890 msgr2=0x7f87b8116ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:28.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.400+0000 7f87b7fff640 1 --2- 172.21.15.114:0/2999831595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87b8107890 0x7f87b8116ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.400+0000 7f87b7fff640 1 -- 172.21.15.114:0/2999831595 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87b81074b0 msgr2=0x7f87b8116940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:28.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.400+0000 7f87b7fff640 1 --2- 172.21.15.114:0/2999831595 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87b81074b0 0x7f87b8116940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.400+0000 7f87b7fff640 1 -- 172.21.15.114:0/2999831595 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a8009560 con 0x7f87b8119400 2024-09-13T18:26:28.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.402+0000 7f87b7fff640 1 --2- 172.21.15.114:0/2999831595 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87b8119400 0x7f87b810fc40 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f87b000b990 tx=0x7f87b000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:28.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.402+0000 7f87b4ff9640 1 -- 172.21.15.114:0/2999831595 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b000cba0 con 0x7f87b8119400 2024-09-13T18:26:28.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.402+0000 7f87b4ff9640 1 -- 172.21.15.114:0/2999831595 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f87b000cd00 con 0x7f87b8119400 2024-09-13T18:26:28.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.402+0000 7f87b4ff9640 1 -- 172.21.15.114:0/2999831595 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b00075f0 con 0x7f87b8119400 2024-09-13T18:26:28.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.402+0000 7f87bcb9d640 1 -- 172.21.15.114:0/2999831595 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87ac009490 con 0x7f87b8119400 2024-09-13T18:26:28.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.402+0000 7f87bcb9d640 1 -- 172.21.15.114:0/2999831595 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b8114800 con 0x7f87b8119400 2024-09-13T18:26:28.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.406+0000 7f87b4ff9640 1 -- 172.21.15.114:0/2999831595 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f87b0007810 con 0x7f87b8119400 2024-09-13T18:26:28.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.408+0000 7f87b4ff9640 1 --2- 172.21.15.114:0/2999831595 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f87a00a1390 0x7f87a00a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:28.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.408+0000 7f87b4ff9640 1 -- 172.21.15.114:0/2999831595 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f87b00cd640 con 0x7f87b8119400 2024-09-13T18:26:28.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.409+0000 7f87b77fe640 1 --2- 172.21.15.114:0/2999831595 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f87a00a1390 0x7f87a00a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:28.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.488+0000 7f87b77fe640 1 --2- 172.21.15.114:0/2999831595 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f87a00a1390 0x7f87a00a3850 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f87a8009530 tx=0x7f87a8009260 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:28.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.488+0000 7f87c0bf6740 1 Processor -- start 2024-09-13T18:26:28.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.490+0000 7f87c0bf6740 1 -- start start 2024-09-13T18:26:28.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.490+0000 7f87c0bf6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b44492970 0x561b44492d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:28.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.490+0000 7f87c0bf6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b44493290 0x561b4449d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:28.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.490+0000 7f87c0bf6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b4449db30 0x561b4449ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:28.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.490+0000 7f87c0bf6740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b443fccc0 con 0x561b44493290 2024-09-13T18:26:28.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.490+0000 7f87c0bf6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b443fa450 con 0x561b4449db30 2024-09-13T18:26:28.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.490+0000 7f87c0bf6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b443f5d40 con 0x561b44492970 2024-09-13T18:26:28.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.490+0000 7f87b77fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b44492970 0x561b44492d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:28.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.490+0000 7f87b77fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b44492970 0x561b44492d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:52560/0 (socket says 172.21.15.114:52560) 2024-09-13T18:26:28.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.490+0000 7f87b77fe640 1 -- 172.21.15.114:0/191858893 learned_addr learned my addr 172.21.15.114:0/191858893 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:28.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.491+0000 7f87b7fff640 1 --2- 172.21.15.114:0/191858893 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b4449db30 0x561b4449ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:28.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.491+0000 7f87b77fe640 1 -- 172.21.15.114:0/191858893 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b4449db30 msgr2=0x561b4449ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:28.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.491+0000 7f87b77fe640 1 --2- 172.21.15.114:0/191858893 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b4449db30 0x561b4449ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.491+0000 7f87b77fe640 1 -- 172.21.15.114:0/191858893 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b44493290 msgr2=0x561b4449d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:28.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.491+0000 7f87b77fe640 1 --2- 172.21.15.114:0/191858893 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b44493290 0x561b4449d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.491+0000 7f87b77fe640 1 -- 172.21.15.114:0/191858893 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b443bd620 con 0x561b44492970 2024-09-13T18:26:28.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.491+0000 7f87b77fe640 1 --2- 172.21.15.114:0/191858893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b44492970 0x561b44492d50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f87a800e430 tx=0x7f87a8014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d2794172e3b420a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:28.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.491+0000 7f879dffb640 1 -- 172.21.15.114:0/191858893 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a800ee20 con 0x561b44492970 2024-09-13T18:26:28.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.491+0000 7f879dffb640 1 -- 172.21.15.114:0/191858893 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f87a8021070 con 0x561b44492970 2024-09-13T18:26:28.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.491+0000 7f879dffb640 1 -- 172.21.15.114:0/191858893 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a801d670 con 0x561b44492970 2024-09-13T18:26:28.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.492+0000 7f87c0bf6740 1 -- 172.21.15.114:0/191858893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b44492970 msgr2=0x561b44492d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:28.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.492+0000 7f87c0bf6740 1 --2- 172.21.15.114:0/191858893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b44492970 0x561b44492d50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f87a800e430 tx=0x7f87a8014000 comp rx=0 tx=0).stop 2024-09-13T18:26:28.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.497+0000 7f87c0bf6740 1 -- 172.21.15.114:0/191858893 shutdown_connections 2024-09-13T18:26:28.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.497+0000 7f87c0bf6740 1 --2- 172.21.15.114:0/191858893 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b4449db30 0x561b4449ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.497+0000 7f87c0bf6740 1 --2- 172.21.15.114:0/191858893 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b44493290 0x561b4449d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.497+0000 7f87c0bf6740 1 --2- 172.21.15.114:0/191858893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b44492970 0x561b44492d50 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.497+0000 7f87c0bf6740 1 -- 172.21.15.114:0/191858893 >> 172.21.15.114:0/191858893 conn(0x561b4448eee0 msgr2=0x561b4448f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:28.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.497+0000 7f87c0bf6740 1 -- 172.21.15.114:0/191858893 shutdown_connections 2024-09-13T18:26:28.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.498+0000 7f87c0bf6740 1 -- 172.21.15.114:0/191858893 wait complete. 2024-09-13T18:26:28.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.498+0000 7f87c0bf6740 1 -- start start 2024-09-13T18:26:28.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.499+0000 7f87c0bf6740 1 Processor -- start 2024-09-13T18:26:28.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.499+0000 7f87c0bf6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b44492970 0x561b4449d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:28.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.499+0000 7f87c0bf6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b44493290 0x561b4449d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:28.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.499+0000 7f87c0bf6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b4449db30 0x561b4449a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:28.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.499+0000 7f87c0bf6740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b443b4180 con 0x561b44492970 2024-09-13T18:26:28.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.499+0000 7f87c0bf6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b440ce430 con 0x561b44493290 2024-09-13T18:26:28.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.499+0000 7f87c0bf6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b443795e0 con 0x561b4449db30 2024-09-13T18:26:28.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.499+0000 7f87c0bf6740 10 client.0 _ll_register_callbacks cb 0x561b443e5aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:26:28.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.500+0000 7f87b77fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b44492970 0x561b4449d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:28.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.500+0000 7f87b6ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b44493290 0x561b4449d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:28.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.500+0000 7f87b6ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b44493290 0x561b4449d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:60550/0 (socket says 172.21.15.114:60550) 2024-09-13T18:26:28.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.500+0000 7f87b6ffd640 1 -- 172.21.15.114:0/1944851578 learned_addr learned my addr 172.21.15.114:0/1944851578 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:28.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.500+0000 7f87b7fff640 1 --2- 172.21.15.114:0/1944851578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b4449db30 0x561b4449a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:28.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.501+0000 7f87b77fe640 1 -- 172.21.15.114:0/1944851578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b4449db30 msgr2=0x561b4449a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:28.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.501+0000 7f87b77fe640 1 --2- 172.21.15.114:0/1944851578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b4449db30 0x561b4449a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.501+0000 7f87b77fe640 1 -- 172.21.15.114:0/1944851578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b44493290 msgr2=0x561b4449d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:28.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.501+0000 7f87b77fe640 1 --2- 172.21.15.114:0/1944851578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b44493290 0x561b4449d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.501+0000 7f87b77fe640 1 -- 172.21.15.114:0/1944851578 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87a8017070 con 0x561b44492970 2024-09-13T18:26:28.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.504+0000 7f87b77fe640 1 --2- 172.21.15.114:0/1944851578 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b44492970 0x561b4449d050 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f87a800e430 tx=0x7f87a80050c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a50b25c469afc03 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:28.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.504+0000 7f8796ffd640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:26:28.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.505+0000 7f8796ffd640 1 -- 172.21.15.114:0/1944851578 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a80054b0 con 0x561b44492970 2024-09-13T18:26:28.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.505+0000 7f87c0bf6740 1 -- 172.21.15.114:0/1944851578 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561b4441fb80 con 0x561b44492970 2024-09-13T18:26:28.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.505+0000 7f8796ffd640 1 -- 172.21.15.114:0/1944851578 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f87a8005610 con 0x561b44492970 2024-09-13T18:26:28.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.505+0000 7f8796ffd640 1 -- 172.21.15.114:0/1944851578 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87a80046c0 con 0x561b44492970 2024-09-13T18:26:28.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.506+0000 7f87c0bf6740 10 client.16953 fetch_fsmap learned FSMap version 32 2024-09-13T18:26:28.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.506+0000 7f87c0bf6740 1 -- 172.21.15.114:0/1944851578 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x561b443f43c0 con 0x561b44492970 2024-09-13T18:26:28.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.506+0000 7f8796ffd640 1 -- 172.21.15.114:0/1944851578 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561b4441fb80 con 0x561b44492970 2024-09-13T18:26:28.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.506+0000 7f8796ffd640 1 -- 172.21.15.114:0/1944851578 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f87a801c8e0 con 0x561b44492970 2024-09-13T18:26:28.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.506+0000 7f87c0bf6740 10 client.16953 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:26:28.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.506+0000 7f87c0bf6740 10 client.16953 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-13T18:26:28.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.506+0000 7f87c0bf6740 20 client.16953 populate_metadata read hostname 'smithi114' 2024-09-13T18:26:28.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.507+0000 7f87c0bf6740 1 --2- 172.21.15.114:0/1944851578 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x561b444cf560 0x561b444d1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:28.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.507+0000 7f87c0bf6740 4 client.16953 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:26:28.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.507+0000 7f87c0bf6740 1 -- 172.21.15.114:0/1944851578 --> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561b443e81f0 con 0x561b444cf560 2024-09-13T18:26:28.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.508+0000 7f87b6ffd640 1 --2- 172.21.15.114:0/1944851578 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x561b444cf560 0x561b444d1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:28.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.510+0000 7f87b6ffd640 1 --2- 172.21.15.114:0/1944851578 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x561b444cf560 0x561b444d1980 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b2562388a117e313 server_cookie=ad16eb1c42f208d4 in_seq=0 out_seq=0 2024-09-13T18:26:28.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.510+0000 7f8796ffd640 10 client.16953 ms_handle_connect on v2:172.21.15.114:6840/1956923446 2024-09-13T18:26:28.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.512+0000 7f8796ffd640 1 -- 172.21.15.114:0/1944851578 <== mds.4 v2:172.21.15.114:6840/1956923446 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x561b443e81f0 con 0x561b444cf560 2024-09-13T18:26:28.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.512+0000 7f8796ffd640 10 client.16953 handle_command_reply: tid=0 2024-09-13T18:26:28.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.513+0000 7f87c0bf6740 1 client.16953 shutdown 2024-09-13T18:26:28.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.518+0000 7f87c0bf6740 1 -- 172.21.15.114:0/1944851578 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b44492970 msgr2=0x561b4449d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:28.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.518+0000 7f87c0bf6740 1 --2- 172.21.15.114:0/1944851578 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b44492970 0x561b4449d050 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f87a800e430 tx=0x7f87a80050c0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.521+0000 7f87c0bf6740 1 -- 172.21.15.114:0/1944851578 shutdown_connections 2024-09-13T18:26:28.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.521+0000 7f87c0bf6740 1 --2- 172.21.15.114:0/1944851578 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x561b444cf560 0x561b444d1980 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.521+0000 7f87c0bf6740 1 --2- 172.21.15.114:0/1944851578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b4449db30 0x561b4449a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.521+0000 7f87c0bf6740 1 --2- 172.21.15.114:0/1944851578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b44493290 0x561b4449d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.521+0000 7f87c0bf6740 1 --2- 172.21.15.114:0/1944851578 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b44492970 0x561b4449d050 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.521+0000 7f87c0bf6740 1 -- 172.21.15.114:0/1944851578 >> 172.21.15.114:0/1944851578 conn(0x561b4448eee0 msgr2=0x561b4448e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:28.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.522+0000 7f8796ffd640 0 client.16953 ms_handle_reset on v2:172.21.15.114:6840/1956923446 2024-09-13T18:26:28.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.522+0000 7f87c0bf6740 1 -- 172.21.15.114:0/1944851578 shutdown_connections 2024-09-13T18:26:28.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.523+0000 7f87c0bf6740 1 -- 172.21.15.114:0/1944851578 wait complete. 2024-09-13T18:26:28.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.527+0000 7f87c0bf6740 20 client.16953 trim_cache size 0 max 16384 2024-09-13T18:26:28.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.552+0000 7f87c0bf6740 1 Processor -- start 2024-09-13T18:26:28.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.552+0000 7f87c0bf6740 1 -- start start 2024-09-13T18:26:28.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.552+0000 7f87c0bf6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b44492970 0x561b444d62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:28.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.552+0000 7f87c0bf6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b44493290 0x561b444a4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:28.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.552+0000 7f87c0bf6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b4449db30 0x561b444a4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:28.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.552+0000 7f87c0bf6740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b4448a7b0 con 0x561b44493290 2024-09-13T18:26:28.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.552+0000 7f87c0bf6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b44493920 con 0x561b44492970 2024-09-13T18:26:28.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.552+0000 7f87c0bf6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b444926f0 con 0x561b4449db30 2024-09-13T18:26:28.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.552+0000 7f87b7fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b4449db30 0x561b444a4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:28.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.552+0000 7f87b7fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b4449db30 0x561b444a4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:52580/0 (socket says 172.21.15.114:52580) 2024-09-13T18:26:28.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.552+0000 7f87b7fff640 1 -- 172.21.15.114:0/4041841181 learned_addr learned my addr 172.21.15.114:0/4041841181 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:28.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.553+0000 7f87b77fe640 1 --2- 172.21.15.114:0/4041841181 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b44492970 0x561b444d62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:28.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.553+0000 7f87b7fff640 1 -- 172.21.15.114:0/4041841181 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b44492970 msgr2=0x561b444d62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:28.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.553+0000 7f87b7fff640 1 --2- 172.21.15.114:0/4041841181 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b44492970 0x561b444d62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.553+0000 7f87b7fff640 1 -- 172.21.15.114:0/4041841181 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b44493290 msgr2=0x561b444a4830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:26:28.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.553+0000 7f87b7fff640 1 --2- 172.21.15.114:0/4041841181 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b44493290 0x561b444a4830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.553+0000 7f87b7fff640 1 -- 172.21.15.114:0/4041841181 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b44396770 con 0x561b4449db30 2024-09-13T18:26:28.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.554+0000 7f87b7fff640 1 --2- 172.21.15.114:0/4041841181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b4449db30 0x561b444a4d70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f87b000b960 tx=0x7f87b000e040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4896bdb31ea6bfcc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:28.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.554+0000 7f879e7fc640 1 -- 172.21.15.114:0/4041841181 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b009a5c0 con 0x561b4449db30 2024-09-13T18:26:28.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.554+0000 7f879e7fc640 1 -- 172.21.15.114:0/4041841181 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f87b009e340 con 0x561b4449db30 2024-09-13T18:26:28.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.554+0000 7f879e7fc640 1 -- 172.21.15.114:0/4041841181 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b0096cd0 con 0x561b4449db30 2024-09-13T18:26:28.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.556+0000 7f87c0bf6740 1 -- 172.21.15.114:0/4041841181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b4449db30 msgr2=0x561b444a4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:28.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.556+0000 7f87c0bf6740 1 --2- 172.21.15.114:0/4041841181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b4449db30 0x561b444a4d70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f87b000b960 tx=0x7f87b000e040 comp rx=0 tx=0).stop 2024-09-13T18:26:28.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.556+0000 7f87c0bf6740 1 -- 172.21.15.114:0/4041841181 shutdown_connections 2024-09-13T18:26:28.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.556+0000 7f87c0bf6740 1 --2- 172.21.15.114:0/4041841181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b4449db30 0x561b444a4d70 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.556+0000 7f87c0bf6740 1 --2- 172.21.15.114:0/4041841181 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b44493290 0x561b444a4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.556+0000 7f87c0bf6740 1 --2- 172.21.15.114:0/4041841181 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b44492970 0x561b444d62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.556+0000 7f87c0bf6740 1 -- 172.21.15.114:0/4041841181 >> 172.21.15.114:0/4041841181 conn(0x561b4448eee0 msgr2=0x561b444a92e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:28.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.556+0000 7f87c0bf6740 1 -- 172.21.15.114:0/4041841181 shutdown_connections 2024-09-13T18:26:28.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.556+0000 7f87c0bf6740 1 -- 172.21.15.114:0/4041841181 wait complete. 2024-09-13T18:26:28.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.556+0000 7f87c0bf6740 1 -- start start 2024-09-13T18:26:28.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.557+0000 7f87c0bf6740 1 Processor -- start 2024-09-13T18:26:28.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.557+0000 7f87c0bf6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b44492970 0x561b44519ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:28.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.557+0000 7f87c0bf6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b44493290 0x561b4451a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:28.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.558+0000 7f87c0bf6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b4449db30 0x561b4451eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:28.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.558+0000 7f87b6ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b44493290 0x561b4451a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:28.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.558+0000 7f87c0bf6740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b4451f050 con 0x561b44492970 2024-09-13T18:26:28.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.558+0000 7f87c0bf6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b4451f1c0 con 0x561b4449db30 2024-09-13T18:26:28.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.558+0000 7f87c0bf6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b4451f380 con 0x561b44493290 2024-09-13T18:26:28.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.558+0000 7f87c0bf6740 10 client.0 _ll_register_callbacks cb 0x7f87a8009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:26:28.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.558+0000 7f87b7fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b4449db30 0x561b4451eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:28.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.558+0000 7f87b6ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b44493290 0x561b4451a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:52596/0 (socket says 172.21.15.114:52596) 2024-09-13T18:26:28.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.558+0000 7f87b6ffd640 1 -- 172.21.15.114:0/3786861389 learned_addr learned my addr 172.21.15.114:0/3786861389 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:28.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.559+0000 7f87b77fe640 1 --2- 172.21.15.114:0/3786861389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b44492970 0x561b44519ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:28.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.560+0000 7f87b77fe640 1 -- 172.21.15.114:0/3786861389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b44493290 msgr2=0x561b4451a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:28.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.560+0000 7f87b77fe640 1 --2- 172.21.15.114:0/3786861389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b44493290 0x561b4451a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.560+0000 7f87b77fe640 1 -- 172.21.15.114:0/3786861389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b4449db30 msgr2=0x561b4451eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:28.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.560+0000 7f87b77fe640 1 --2- 172.21.15.114:0/3786861389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b4449db30 0x561b4451eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.560+0000 7f87b77fe640 1 -- 172.21.15.114:0/3786861389 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b8114800 con 0x561b44492970 2024-09-13T18:26:28.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.560+0000 7f87b77fe640 1 --2- 172.21.15.114:0/3786861389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b44492970 0x561b44519ba0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f87a8011d10 tx=0x7f87a80053c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b4d29996099fdb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:28.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.560+0000 7f8796ffd640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:26:28.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.561+0000 7f8796ffd640 1 -- 172.21.15.114:0/3786861389 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a800edb0 con 0x561b44492970 2024-09-13T18:26:28.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.561+0000 7f8796ffd640 1 -- 172.21.15.114:0/3786861389 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f87a802f480 con 0x561b44492970 2024-09-13T18:26:28.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.561+0000 7f87c0bf6740 1 -- 172.21.15.114:0/3786861389 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f87a8021070 con 0x561b44492970 2024-09-13T18:26:28.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.561+0000 7f8796ffd640 1 -- 172.21.15.114:0/3786861389 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87a801cde0 con 0x561b44492970 2024-09-13T18:26:28.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.561+0000 7f87c0bf6740 10 client.16956 fetch_fsmap learned FSMap version 32 2024-09-13T18:26:28.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.561+0000 7f87c0bf6740 1 -- 172.21.15.114:0/3786861389 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x561b443bbe80 con 0x561b44492970 2024-09-13T18:26:28.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.561+0000 7f8796ffd640 1 -- 172.21.15.114:0/3786861389 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f87a8021070 con 0x561b44492970 2024-09-13T18:26:28.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.561+0000 7f8796ffd640 1 -- 172.21.15.114:0/3786861389 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f87a802f910 con 0x561b44492970 2024-09-13T18:26:28.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.562+0000 7f87c0bf6740 10 client.16956 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:26:28.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.562+0000 7f87c0bf6740 10 client.16956 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-13T18:26:28.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.562+0000 7f87c0bf6740 20 client.16956 populate_metadata read hostname 'smithi114' 2024-09-13T18:26:28.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.562+0000 7f87c0bf6740 1 --2- 172.21.15.114:0/3786861389 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x561b44526790 0x561b44528be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:28.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.562+0000 7f87c0bf6740 4 client.16956 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:26:28.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.562+0000 7f87c0bf6740 1 -- 172.21.15.114:0/3786861389 --> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561b440e94e0 con 0x561b44526790 2024-09-13T18:26:28.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.563+0000 7f87b6ffd640 1 --2- 172.21.15.114:0/3786861389 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x561b44526790 0x561b44528be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:28.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.564+0000 7f87b6ffd640 1 --2- 172.21.15.114:0/3786861389 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x561b44526790 0x561b44528be0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3176b1d0894dcdce server_cookie=35105ef7b755156b in_seq=0 out_seq=0 2024-09-13T18:26:28.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.564+0000 7f8796ffd640 10 client.16956 ms_handle_connect on v2:172.21.15.114:6840/1956923446 2024-09-13T18:26:28.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.565+0000 7f8796ffd640 1 -- 172.21.15.114:0/3786861389 <== mds.4 v2:172.21.15.114:6840/1956923446 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561b440e94e0 con 0x561b44526790 2024-09-13T18:26:28.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.565+0000 7f8796ffd640 10 client.16956 handle_command_reply: tid=0 2024-09-13T18:26:28.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.565+0000 7f87c0bf6740 1 client.16956 shutdown 2024-09-13T18:26:28.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.567+0000 7f87c0bf6740 1 -- 172.21.15.114:0/3786861389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b44492970 msgr2=0x561b44519ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:28.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.567+0000 7f87c0bf6740 1 --2- 172.21.15.114:0/3786861389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b44492970 0x561b44519ba0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f87a8011d10 tx=0x7f87a80053c0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.568+0000 7f87c0bf6740 1 -- 172.21.15.114:0/3786861389 shutdown_connections 2024-09-13T18:26:28.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.568+0000 7f87c0bf6740 1 --2- 172.21.15.114:0/3786861389 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x561b44526790 0x561b44528be0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.568+0000 7f87c0bf6740 1 --2- 172.21.15.114:0/3786861389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b4449db30 0x561b4451eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.568+0000 7f87c0bf6740 1 --2- 172.21.15.114:0/3786861389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b44493290 0x561b4451a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.568+0000 7f87c0bf6740 1 --2- 172.21.15.114:0/3786861389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561b44492970 0x561b44519ba0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.569+0000 7f87c0bf6740 1 -- 172.21.15.114:0/3786861389 >> 172.21.15.114:0/3786861389 conn(0x561b4448eee0 msgr2=0x561b444d62a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:28.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.569+0000 7f8796ffd640 0 client.16956 ms_handle_reset on v2:172.21.15.114:6840/1956923446 2024-09-13T18:26:28.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.569+0000 7f87c0bf6740 1 -- 172.21.15.114:0/3786861389 shutdown_connections 2024-09-13T18:26:28.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.569+0000 7f87c0bf6740 1 -- 172.21.15.114:0/3786861389 wait complete. 2024-09-13T18:26:28.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.569+0000 7f87c0bf6740 20 client.16956 trim_cache size 0 max 16384 2024-09-13T18:26:28.573 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:26:28.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.570+0000 7f87bcb9d640 1 -- 172.21.15.114:0/2999831595 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f87a00a1390 msgr2=0x7f87a00a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:28.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.570+0000 7f87bcb9d640 1 --2- 172.21.15.114:0/2999831595 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f87a00a1390 0x7f87a00a3850 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f87a8009530 tx=0x7f87a8009260 comp rx=0 tx=0).stop 2024-09-13T18:26:28.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.570+0000 7f87bcb9d640 1 -- 172.21.15.114:0/2999831595 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87b8119400 msgr2=0x7f87b810fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:28.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.570+0000 7f87bcb9d640 1 --2- 172.21.15.114:0/2999831595 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87b8119400 0x7f87b810fc40 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f87b000b990 tx=0x7f87b000be50 comp rx=0 tx=0).stop 2024-09-13T18:26:28.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.571+0000 7f87bcb9d640 1 -- 172.21.15.114:0/2999831595 shutdown_connections 2024-09-13T18:26:28.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.571+0000 7f87bcb9d640 1 --2- 172.21.15.114:0/2999831595 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f87a00a1390 0x7f87a00a3850 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.571+0000 7f87bcb9d640 1 --2- 172.21.15.114:0/2999831595 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87b8119400 0x7f87b810fc40 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.571+0000 7f87bcb9d640 1 --2- 172.21.15.114:0/2999831595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87b8107890 0x7f87b8116ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.571+0000 7f87bcb9d640 1 --2- 172.21.15.114:0/2999831595 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87b81074b0 0x7f87b8116940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.572+0000 7f87bcb9d640 1 -- 172.21.15.114:0/2999831595 >> 172.21.15.114:0/2999831595 conn(0x7f87b806af40 msgr2=0x7f87b8119da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:28.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.573+0000 7f87bcb9d640 1 -- 172.21.15.114:0/2999831595 shutdown_connections 2024-09-13T18:26:28.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.573+0000 7f87bcb9d640 1 -- 172.21.15.114:0/2999831595 wait complete. 2024-09-13T18:26:28.598 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-09-13T18:26:28.599 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:26:28 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2860698289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:26:28.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:26:28 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/2860698289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:26:28.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:26:28 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/2860698289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:26:28.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.811+0000 7f9780bc7640 1 -- 172.21.15.114:0/4097674806 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f977c0ff5b0 msgr2=0x7f977c103a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:28.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.811+0000 7f9780bc7640 1 --2- 172.21.15.114:0/4097674806 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f977c0ff5b0 0x7f977c103a70 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f976c009960 tx=0x7f976c02a530 comp rx=0 tx=0).stop 2024-09-13T18:26:28.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.813+0000 7f9780bc7640 1 -- 172.21.15.114:0/4097674806 shutdown_connections 2024-09-13T18:26:28.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.813+0000 7f9780bc7640 1 --2- 172.21.15.114:0/4097674806 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f977c1040c0 0x7f977c104570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.813+0000 7f9780bc7640 1 --2- 172.21.15.114:0/4097674806 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f977c0ff5b0 0x7f977c103a70 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.813+0000 7f9780bc7640 1 --2- 172.21.15.114:0/4097674806 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f977c0fec90 0x7f977c0ff070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.813+0000 7f9780bc7640 1 -- 172.21.15.114:0/4097674806 >> 172.21.15.114:0/4097674806 conn(0x7f977c075bf0 msgr2=0x7f977c0fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:28.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.817+0000 7f9780bc7640 1 -- 172.21.15.114:0/4097674806 shutdown_connections 2024-09-13T18:26:28.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.817+0000 7f9780bc7640 1 -- 172.21.15.114:0/4097674806 wait complete. 2024-09-13T18:26:28.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.818+0000 7f9780bc7640 1 Processor -- start 2024-09-13T18:26:28.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.818+0000 7f9780bc7640 1 -- start start 2024-09-13T18:26:28.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.819+0000 7f9780bc7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f977c0fec90 0x7f977c1a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:28.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.819+0000 7f9780bc7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f977c0ff5b0 0x7f977c1a2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:28.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.819+0000 7f9780bc7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f977c1040c0 0x7f977c19c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:28.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.819+0000 7f9780bc7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f977c19cc80 con 0x7f977c0ff5b0 2024-09-13T18:26:28.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.819+0000 7f9780bc7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f977c19cdf0 con 0x7f977c1040c0 2024-09-13T18:26:28.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.819+0000 7f9780bc7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f977c19cfb0 con 0x7f977c0fec90 2024-09-13T18:26:28.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.820+0000 7f977ad9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f977c1040c0 0x7f977c19c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:28.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.820+0000 7f977ad9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f977c1040c0 0x7f977c19c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:60592/0 (socket says 172.21.15.114:60592) 2024-09-13T18:26:28.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.820+0000 7f977a59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f977c0fec90 0x7f977c1a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:28.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.820+0000 7f977a59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f977c0fec90 0x7f977c1a2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:52616/0 (socket says 172.21.15.114:52616) 2024-09-13T18:26:28.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.820+0000 7f977a59c640 1 -- 172.21.15.114:0/3175924400 learned_addr learned my addr 172.21.15.114:0/3175924400 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:28.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.820+0000 7f9779d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f977c0ff5b0 0x7f977c1a2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:28.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.821+0000 7f977a59c640 1 -- 172.21.15.114:0/3175924400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f977c1040c0 msgr2=0x7f977c19c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:28.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.821+0000 7f977a59c640 1 --2- 172.21.15.114:0/3175924400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f977c1040c0 0x7f977c19c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.821+0000 7f977a59c640 1 -- 172.21.15.114:0/3175924400 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f977c0ff5b0 msgr2=0x7f977c1a2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:28.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.821+0000 7f977a59c640 1 --2- 172.21.15.114:0/3175924400 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f977c0ff5b0 0x7f977c1a2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.821+0000 7f977a59c640 1 -- 172.21.15.114:0/3175924400 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f976c009560 con 0x7f977c0fec90 2024-09-13T18:26:28.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.821+0000 7f977ad9d640 1 --2- 172.21.15.114:0/3175924400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f977c1040c0 0x7f977c19c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:26:28.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.821+0000 7f977a59c640 1 --2- 172.21.15.114:0/3175924400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f977c0fec90 0x7f977c1a2580 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f977400ebc0 tx=0x7f977400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:28.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.822+0000 7f976b7fe640 1 -- 172.21.15.114:0/3175924400 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97740099b0 con 0x7f977c0fec90 2024-09-13T18:26:28.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.822+0000 7f9780bc7640 1 -- 172.21.15.114:0/3175924400 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f977c19d290 con 0x7f977c0fec90 2024-09-13T18:26:28.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.822+0000 7f9780bc7640 1 -- 172.21.15.114:0/3175924400 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f977c1a9570 con 0x7f977c0fec90 2024-09-13T18:26:28.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.822+0000 7f976b7fe640 1 -- 172.21.15.114:0/3175924400 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f977400eec0 con 0x7f977c0fec90 2024-09-13T18:26:28.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.823+0000 7f976b7fe640 1 -- 172.21.15.114:0/3175924400 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9774018a20 con 0x7f977c0fec90 2024-09-13T18:26:28.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.827+0000 7f97839c2740 1 Processor -- start 2024-09-13T18:26:28.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.829+0000 7f976b7fe640 1 -- 172.21.15.114:0/3175924400 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f9774018b80 con 0x7f977c0fec90 2024-09-13T18:26:28.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.830+0000 7f976b7fe640 1 --2- 172.21.15.114:0/3175924400 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f97640a1340 0x7f97640a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:28.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.830+0000 7f976b7fe640 1 -- 172.21.15.114:0/3175924400 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f9774014070 con 0x7f977c0fec90 2024-09-13T18:26:28.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.831+0000 7f977ad9d640 1 --2- 172.21.15.114:0/3175924400 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f97640a1340 0x7f97640a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:28.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.832+0000 7f97839c2740 1 -- start start 2024-09-13T18:26:28.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.833+0000 7f97839c2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568296d5970 0x5568296d5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:28.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.833+0000 7f97839c2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568296d6290 0x5568296e05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:28.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.833+0000 7f97839c2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568296e0b30 0x5568296e2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:28.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.833+0000 7f97839c2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55682963fcc0 con 0x5568296e0b30 2024-09-13T18:26:28.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.833+0000 7f97839c2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55682963d450 con 0x5568296d6290 2024-09-13T18:26:28.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.833+0000 7f97839c2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556829638d40 con 0x5568296d5970 2024-09-13T18:26:28.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.834+0000 7f977a59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568296d5970 0x5568296d5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:28.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.834+0000 7f977a59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568296d5970 0x5568296d5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:52630/0 (socket says 172.21.15.114:52630) 2024-09-13T18:26:28.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.834+0000 7f977a59c640 1 -- 172.21.15.114:0/4015817610 learned_addr learned my addr 172.21.15.114:0/4015817610 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:28.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.836+0000 7f977a59c640 1 -- 172.21.15.114:0/4015817610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568296d6290 msgr2=0x5568296e05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:28.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.836+0000 7f977a59c640 1 --2- 172.21.15.114:0/4015817610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568296d6290 0x5568296e05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.836+0000 7f977a59c640 1 -- 172.21.15.114:0/4015817610 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568296e0b30 msgr2=0x5568296e2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:28.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.836+0000 7f977a59c640 1 --2- 172.21.15.114:0/4015817610 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568296e0b30 0x5568296e2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.836+0000 7f977a59c640 1 -- 172.21.15.114:0/4015817610 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556829600620 con 0x5568296d5970 2024-09-13T18:26:28.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.837+0000 7f977a59c640 1 --2- 172.21.15.114:0/4015817610 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568296d5970 0x5568296d5d50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f977409a490 tx=0x7f977409a1b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=889a1e2059f96ce9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:28.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.842+0000 7f977ad9d640 1 --2- 172.21.15.114:0/3175924400 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f97640a1340 0x7f97640a3800 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x5568296d3260 tx=0x7f977000bb00 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:28.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.843+0000 7f9768ff9640 1 -- 172.21.15.114:0/4015817610 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97740ba0b0 con 0x5568296d5970 2024-09-13T18:26:28.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.843+0000 7f9768ff9640 1 -- 172.21.15.114:0/4015817610 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9774096ea0 con 0x5568296d5970 2024-09-13T18:26:28.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.843+0000 7f9768ff9640 1 -- 172.21.15.114:0/4015817610 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97740a26d0 con 0x5568296d5970 2024-09-13T18:26:28.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.844+0000 7f97839c2740 1 -- 172.21.15.114:0/4015817610 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568296d5970 msgr2=0x5568296d5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:28.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.844+0000 7f97839c2740 1 --2- 172.21.15.114:0/4015817610 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568296d5970 0x5568296d5d50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f977409a490 tx=0x7f977409a1b0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.849+0000 7f97839c2740 1 -- 172.21.15.114:0/4015817610 shutdown_connections 2024-09-13T18:26:28.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.849+0000 7f97839c2740 1 --2- 172.21.15.114:0/4015817610 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568296e0b30 0x5568296e2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.849+0000 7f97839c2740 1 --2- 172.21.15.114:0/4015817610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568296d6290 0x5568296e05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.849+0000 7f97839c2740 1 --2- 172.21.15.114:0/4015817610 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568296d5970 0x5568296d5d50 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.849+0000 7f97839c2740 1 -- 172.21.15.114:0/4015817610 >> 172.21.15.114:0/4015817610 conn(0x5568296d1ee0 msgr2=0x5568296d22c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:28.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.859+0000 7f97839c2740 1 -- 172.21.15.114:0/4015817610 shutdown_connections 2024-09-13T18:26:28.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.860+0000 7f97839c2740 1 -- 172.21.15.114:0/4015817610 wait complete. 2024-09-13T18:26:28.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.860+0000 7f97839c2740 1 -- start start 2024-09-13T18:26:28.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.860+0000 7f97839c2740 1 Processor -- start 2024-09-13T18:26:28.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.861+0000 7f97839c2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568296d5970 0x5568296e01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:28.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.861+0000 7f97839c2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568296d6290 0x5568296dd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:28.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.861+0000 7f97839c2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568296e0b30 0x5568296ddbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:28.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.861+0000 7f97839c2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568295f7180 con 0x5568296e0b30 2024-09-13T18:26:28.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.861+0000 7f97839c2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556829311430 con 0x5568296d6290 2024-09-13T18:26:28.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.861+0000 7f97839c2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568295bc5e0 con 0x5568296d5970 2024-09-13T18:26:28.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.861+0000 7f97839c2740 10 client.0 _ll_register_callbacks cb 0x556829628aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:26:28.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.861+0000 7f977a59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568296d5970 0x5568296e01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:28.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.861+0000 7f977a59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568296d5970 0x5568296e01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:52646/0 (socket says 172.21.15.114:52646) 2024-09-13T18:26:28.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.861+0000 7f977a59c640 1 -- 172.21.15.114:0/3846789683 learned_addr learned my addr 172.21.15.114:0/3846789683 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:28.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.861+0000 7f977ad9d640 1 --2- 172.21.15.114:0/3846789683 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568296e0b30 0x5568296ddbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:28.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.862+0000 7f977a59c640 1 -- 172.21.15.114:0/3846789683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568296d6290 msgr2=0x5568296dd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:28.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.862+0000 7f977a59c640 1 --2- 172.21.15.114:0/3846789683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568296d6290 0x5568296dd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.862+0000 7f977a59c640 1 -- 172.21.15.114:0/3846789683 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568296e0b30 msgr2=0x5568296ddbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:28.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.862+0000 7f977a59c640 1 --2- 172.21.15.114:0/3846789683 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568296e0b30 0x5568296ddbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.862+0000 7f977a59c640 1 -- 172.21.15.114:0/3846789683 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f977c1a9570 con 0x5568296d5970 2024-09-13T18:26:28.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.864+0000 7f977a59c640 1 --2- 172.21.15.114:0/3846789683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568296d5970 0x5568296e01f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9774018630 tx=0x7f97740a2900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdbff0bab1ae68f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:28.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.864+0000 7f9761ffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:26:28.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.864+0000 7f9761ffb640 1 -- 172.21.15.114:0/3846789683 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9774097010 con 0x5568296d5970 2024-09-13T18:26:28.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.864+0000 7f97839c2740 1 -- 172.21.15.114:0/3846789683 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556829662b80 con 0x5568296d5970 2024-09-13T18:26:28.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.864+0000 7f9761ffb640 1 -- 172.21.15.114:0/3846789683 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f9774098b30 con 0x5568296d5970 2024-09-13T18:26:28.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.864+0000 7f9761ffb640 1 -- 172.21.15.114:0/3846789683 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f977409e170 con 0x5568296d5970 2024-09-13T18:26:28.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.865+0000 7f97839c2740 10 client.26335 fetch_fsmap learned FSMap version 32 2024-09-13T18:26:28.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.865+0000 7f97839c2740 1 -- 172.21.15.114:0/3846789683 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5568296373c0 con 0x5568296d5970 2024-09-13T18:26:28.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.865+0000 7f9761ffb640 1 -- 172.21.15.114:0/3846789683 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x556829662b80 con 0x5568296d5970 2024-09-13T18:26:28.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.865+0000 7f9761ffb640 1 -- 172.21.15.114:0/3846789683 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f97740996f0 con 0x5568296d5970 2024-09-13T18:26:28.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.865+0000 7f97839c2740 10 client.26335 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:26:28.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.865+0000 7f97839c2740 10 client.26335 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-13T18:26:28.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.865+0000 7f97839c2740 20 client.26335 populate_metadata read hostname 'smithi114' 2024-09-13T18:26:28.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.865+0000 7f97839c2740 1 --2- 172.21.15.114:0/3846789683 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x556829712330 0x556829714750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:28.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.866+0000 7f97839c2740 4 client.26335 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:26:28.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.866+0000 7f97839c2740 1 -- 172.21.15.114:0/3846789683 --> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55682962b1f0 con 0x556829712330 2024-09-13T18:26:28.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.867+0000 7f9779d9b640 1 --2- 172.21.15.114:0/3846789683 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x556829712330 0x556829714750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:28.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.872+0000 7f9779d9b640 1 --2- 172.21.15.114:0/3846789683 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x556829712330 0x556829714750 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3e15b9b5105051f3 server_cookie=eec073407e9f192e in_seq=0 out_seq=0 2024-09-13T18:26:28.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.872+0000 7f9761ffb640 10 client.26335 ms_handle_connect on v2:172.21.15.134:6834/1150003275 2024-09-13T18:26:28.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.880+0000 7f9761ffb640 1 -- 172.21.15.114:0/3846789683 <== mds.3 v2:172.21.15.134:6834/1150003275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55682962b1f0 con 0x556829712330 2024-09-13T18:26:28.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.880+0000 7f9761ffb640 10 client.26335 handle_command_reply: tid=0 2024-09-13T18:26:28.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.880+0000 7f97839c2740 1 client.26335 shutdown 2024-09-13T18:26:28.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.880+0000 7f97839c2740 1 -- 172.21.15.114:0/3846789683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568296d5970 msgr2=0x5568296e01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:28.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.880+0000 7f97839c2740 1 --2- 172.21.15.114:0/3846789683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568296d5970 0x5568296e01f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9774018630 tx=0x7f97740a2900 comp rx=0 tx=0).stop 2024-09-13T18:26:28.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.880+0000 7f97839c2740 1 -- 172.21.15.114:0/3846789683 shutdown_connections 2024-09-13T18:26:28.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.880+0000 7f97839c2740 1 --2- 172.21.15.114:0/3846789683 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x556829712330 0x556829714750 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.880+0000 7f97839c2740 1 --2- 172.21.15.114:0/3846789683 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568296e0b30 0x5568296ddbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.880+0000 7f97839c2740 1 --2- 172.21.15.114:0/3846789683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568296d6290 0x5568296dd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.880+0000 7f97839c2740 1 --2- 172.21.15.114:0/3846789683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568296d5970 0x5568296e01f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.880+0000 7f97839c2740 1 -- 172.21.15.114:0/3846789683 >> 172.21.15.114:0/3846789683 conn(0x5568296d1ee0 msgr2=0x5568296d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:28.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.880+0000 7f9761ffb640 0 client.26335 ms_handle_reset on v2:172.21.15.134:6834/1150003275 2024-09-13T18:26:28.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.885+0000 7f97839c2740 1 -- 172.21.15.114:0/3846789683 shutdown_connections 2024-09-13T18:26:28.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.885+0000 7f97839c2740 1 -- 172.21.15.114:0/3846789683 wait complete. 2024-09-13T18:26:28.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.885+0000 7f97839c2740 20 client.26335 trim_cache size 0 max 16384 2024-09-13T18:26:28.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.906+0000 7f97839c2740 1 Processor -- start 2024-09-13T18:26:28.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.906+0000 7f97839c2740 1 -- start start 2024-09-13T18:26:28.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.906+0000 7f97839c2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568296d5970 0x5568296dfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:28.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.906+0000 7f97839c2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568296d6290 0x5568296e01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:28.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.906+0000 7f97839c2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568296e0b30 0x5568296da8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:28.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.906+0000 7f97839c2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568296d6080 con 0x5568296e0b30 2024-09-13T18:26:28.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.906+0000 7f97839c2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568296dc5b0 con 0x5568296d5970 2024-09-13T18:26:28.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.906+0000 7f97839c2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568297198d0 con 0x5568296d6290 2024-09-13T18:26:28.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.907+0000 7f977ad9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568296e0b30 0x5568296da8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:28.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.907+0000 7f977ad9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568296e0b30 0x5568296da8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49390/0 (socket says 172.21.15.114:49390) 2024-09-13T18:26:28.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.907+0000 7f977ad9d640 1 -- 172.21.15.114:0/1707039393 learned_addr learned my addr 172.21.15.114:0/1707039393 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:28.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.907+0000 7f977a59c640 1 --2- 172.21.15.114:0/1707039393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568296d5970 0x5568296dfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:28.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.907+0000 7f977ad9d640 1 -- 172.21.15.114:0/1707039393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568296d6290 msgr2=0x5568296e01e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:26:28.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.907+0000 7f977ad9d640 1 --2- 172.21.15.114:0/1707039393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568296d6290 0x5568296e01e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.907+0000 7f977ad9d640 1 -- 172.21.15.114:0/1707039393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568296d5970 msgr2=0x5568296dfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:28.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.907+0000 7f977ad9d640 1 --2- 172.21.15.114:0/1707039393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568296d5970 0x5568296dfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.907+0000 7f977ad9d640 1 -- 172.21.15.114:0/1707039393 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568295d9770 con 0x5568296e0b30 2024-09-13T18:26:28.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.908+0000 7f977ad9d640 1 --2- 172.21.15.114:0/1707039393 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568296e0b30 0x5568296da8e0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f9770004380 tx=0x7f9770013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=538547a7d8330ad5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:28.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.908+0000 7f97697fa640 1 -- 172.21.15.114:0/1707039393 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9770011440 con 0x5568296e0b30 2024-09-13T18:26:28.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.908+0000 7f97697fa640 1 -- 172.21.15.114:0/1707039393 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9770011a30 con 0x5568296e0b30 2024-09-13T18:26:28.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.909+0000 7f97839c2740 1 -- 172.21.15.114:0/1707039393 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568296e0b30 msgr2=0x5568296da8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:28.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.909+0000 7f97839c2740 1 --2- 172.21.15.114:0/1707039393 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568296e0b30 0x5568296da8e0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f9770004380 tx=0x7f9770013000 comp rx=0 tx=0).stop 2024-09-13T18:26:28.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.912+0000 7f97839c2740 1 -- 172.21.15.114:0/1707039393 shutdown_connections 2024-09-13T18:26:28.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.912+0000 7f97839c2740 1 --2- 172.21.15.114:0/1707039393 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568296e0b30 0x5568296da8e0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.912+0000 7f97839c2740 1 --2- 172.21.15.114:0/1707039393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568296d6290 0x5568296e01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.912+0000 7f97839c2740 1 --2- 172.21.15.114:0/1707039393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568296d5970 0x5568296dfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.912+0000 7f97839c2740 1 -- 172.21.15.114:0/1707039393 >> 172.21.15.114:0/1707039393 conn(0x5568296d1ee0 msgr2=0x5568296d22c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:28.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.916+0000 7f97839c2740 1 -- 172.21.15.114:0/1707039393 shutdown_connections 2024-09-13T18:26:28.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.916+0000 7f97839c2740 1 -- 172.21.15.114:0/1707039393 wait complete. 2024-09-13T18:26:28.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.916+0000 7f97839c2740 1 -- start start 2024-09-13T18:26:28.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.916+0000 7f97839c2740 1 Processor -- start 2024-09-13T18:26:28.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.917+0000 7f97839c2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568296d5970 0x5568296f8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:28.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.917+0000 7f97839c2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568296d6290 0x5568296f6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:28.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.917+0000 7f97839c2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568296e0b30 0x5568296f7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:28.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.917+0000 7f97839c2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568296f7b80 con 0x5568296e0b30 2024-09-13T18:26:28.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.917+0000 7f97839c2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568296f7cf0 con 0x5568296d5970 2024-09-13T18:26:28.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.917+0000 7f97839c2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556829768840 con 0x5568296d6290 2024-09-13T18:26:28.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.917+0000 7f97839c2740 10 client.0 _ll_register_callbacks cb 0x7f9774096ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:26:28.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.917+0000 7f977ad9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568296e0b30 0x5568296f7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:28.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.917+0000 7f977ad9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568296e0b30 0x5568296f7480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49398/0 (socket says 172.21.15.114:49398) 2024-09-13T18:26:28.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.917+0000 7f977ad9d640 1 -- 172.21.15.114:0/3373369698 learned_addr learned my addr 172.21.15.114:0/3373369698 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:28.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.918+0000 7f977ad9d640 1 -- 172.21.15.114:0/3373369698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568296d6290 msgr2=0x5568296f6f40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:26:28.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.918+0000 7f977ad9d640 1 --2- 172.21.15.114:0/3373369698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568296d6290 0x5568296f6f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.918+0000 7f977ad9d640 1 -- 172.21.15.114:0/3373369698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568296d5970 msgr2=0x5568296f8ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:26:28.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.918+0000 7f977ad9d640 1 --2- 172.21.15.114:0/3373369698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568296d5970 0x5568296f8ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.918+0000 7f977ad9d640 1 -- 172.21.15.114:0/3373369698 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9770004030 con 0x5568296e0b30 2024-09-13T18:26:28.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.919+0000 7f977ad9d640 1 --2- 172.21.15.114:0/3373369698 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568296e0b30 0x5568296f7480 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f977000b320 tx=0x7f977000b520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d8713c8682c13de server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:28.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.919+0000 7f9761ffb640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:26:28.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.919+0000 7f9761ffb640 1 -- 172.21.15.114:0/3373369698 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9770020cb0 con 0x5568296e0b30 2024-09-13T18:26:28.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.919+0000 7f97839c2740 1 -- 172.21.15.114:0/3373369698 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9774096ea0 con 0x5568296e0b30 2024-09-13T18:26:28.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.919+0000 7f9761ffb640 1 -- 172.21.15.114:0/3373369698 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f9770017070 con 0x5568296e0b30 2024-09-13T18:26:28.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.920+0000 7f9761ffb640 1 -- 172.21.15.114:0/3373369698 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9770020e10 con 0x5568296e0b30 2024-09-13T18:26:28.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.920+0000 7f97839c2740 10 client.16971 fetch_fsmap learned FSMap version 32 2024-09-13T18:26:28.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.920+0000 7f97839c2740 1 -- 172.21.15.114:0/3373369698 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5568295fee80 con 0x5568296e0b30 2024-09-13T18:26:28.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.920+0000 7f9761ffb640 1 -- 172.21.15.114:0/3373369698 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9774096ea0 con 0x5568296e0b30 2024-09-13T18:26:28.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.920+0000 7f9761ffb640 1 -- 172.21.15.114:0/3373369698 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f977001d040 con 0x5568296e0b30 2024-09-13T18:26:28.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.920+0000 7f97839c2740 10 client.16971 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:26:28.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.920+0000 7f97839c2740 10 client.16971 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-13T18:26:28.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.920+0000 7f97839c2740 20 client.16971 populate_metadata read hostname 'smithi114' 2024-09-13T18:26:28.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.921+0000 7f97839c2740 1 --2- 172.21.15.114:0/3373369698 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x556829769d70 0x55682976c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:28.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.921+0000 7f97839c2740 4 client.16971 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:26:28.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.921+0000 7f97839c2740 1 -- 172.21.15.114:0/3373369698 --> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55682932c4e0 con 0x556829769d70 2024-09-13T18:26:28.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.921+0000 7f977a59c640 1 --2- 172.21.15.114:0/3373369698 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x556829769d70 0x55682976c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:28.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.923+0000 7f977a59c640 1 --2- 172.21.15.114:0/3373369698 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x556829769d70 0x55682976c190 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=748129ee45ccea31 server_cookie=e328ec419abd8a94 in_seq=0 out_seq=0 2024-09-13T18:26:28.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.923+0000 7f9761ffb640 10 client.16971 ms_handle_connect on v2:172.21.15.134:6834/1150003275 2024-09-13T18:26:28.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.923+0000 7f9761ffb640 1 -- 172.21.15.114:0/3373369698 <== mds.3 v2:172.21.15.134:6834/1150003275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55682932c4e0 con 0x556829769d70 2024-09-13T18:26:28.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.923+0000 7f9761ffb640 10 client.16971 handle_command_reply: tid=0 2024-09-13T18:26:28.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.924+0000 7f97839c2740 1 client.16971 shutdown 2024-09-13T18:26:28.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.924+0000 7f97839c2740 1 -- 172.21.15.114:0/3373369698 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568296e0b30 msgr2=0x5568296f7480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:28.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.924+0000 7f97839c2740 1 --2- 172.21.15.114:0/3373369698 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568296e0b30 0x5568296f7480 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f977000b320 tx=0x7f977000b520 comp rx=0 tx=0).stop 2024-09-13T18:26:28.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.927+0000 7f97839c2740 1 -- 172.21.15.114:0/3373369698 shutdown_connections 2024-09-13T18:26:28.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.927+0000 7f97839c2740 1 --2- 172.21.15.114:0/3373369698 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x556829769d70 0x55682976c190 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.928+0000 7f97839c2740 1 --2- 172.21.15.114:0/3373369698 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568296e0b30 0x5568296f7480 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.928+0000 7f97839c2740 1 --2- 172.21.15.114:0/3373369698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568296d6290 0x5568296f6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.928+0000 7f97839c2740 1 --2- 172.21.15.114:0/3373369698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568296d5970 0x5568296f8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.928+0000 7f97839c2740 1 -- 172.21.15.114:0/3373369698 >> 172.21.15.114:0/3373369698 conn(0x5568296d1ee0 msgr2=0x5568296d84f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:28.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.928+0000 7f9761ffb640 0 client.16971 ms_handle_reset on v2:172.21.15.134:6834/1150003275 2024-09-13T18:26:28.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.930+0000 7f97839c2740 1 -- 172.21.15.114:0/3373369698 shutdown_connections 2024-09-13T18:26:28.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.931+0000 7f97839c2740 1 -- 172.21.15.114:0/3373369698 wait complete. 2024-09-13T18:26:28.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.931+0000 7f97839c2740 20 client.16971 trim_cache size 0 max 16384 2024-09-13T18:26:28.934 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:26:28.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.932+0000 7f9780bc7640 1 -- 172.21.15.114:0/3175924400 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f97640a1340 msgr2=0x7f97640a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:28.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.932+0000 7f9780bc7640 1 --2- 172.21.15.114:0/3175924400 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f97640a1340 0x7f97640a3800 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x5568296d3260 tx=0x7f977000bb00 comp rx=0 tx=0).stop 2024-09-13T18:26:28.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.932+0000 7f9780bc7640 1 -- 172.21.15.114:0/3175924400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f977c0fec90 msgr2=0x7f977c1a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:28.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.932+0000 7f9780bc7640 1 --2- 172.21.15.114:0/3175924400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f977c0fec90 0x7f977c1a2580 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f977400ebc0 tx=0x7f977400c6a0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.933+0000 7f9780bc7640 1 -- 172.21.15.114:0/3175924400 shutdown_connections 2024-09-13T18:26:28.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.933+0000 7f9780bc7640 1 --2- 172.21.15.114:0/3175924400 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f97640a1340 0x7f97640a3800 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.933+0000 7f9780bc7640 1 --2- 172.21.15.114:0/3175924400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f977c1040c0 0x7f977c19c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.933+0000 7f9780bc7640 1 --2- 172.21.15.114:0/3175924400 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f977c0ff5b0 0x7f977c1a2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.933+0000 7f9780bc7640 1 --2- 172.21.15.114:0/3175924400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f977c0fec90 0x7f977c1a2580 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:28.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.933+0000 7f9780bc7640 1 -- 172.21.15.114:0/3175924400 >> 172.21.15.114:0/3175924400 conn(0x7f977c075bf0 msgr2=0x7f977c10d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:28.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.936+0000 7f9780bc7640 1 -- 172.21.15.114:0/3175924400 shutdown_connections 2024-09-13T18:26:28.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:28.936+0000 7f9780bc7640 1 -- 172.21.15.114:0/3175924400 wait complete. 2024-09-13T18:26:28.963 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-09-13T18:26:29.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.219+0000 7f74205ff640 1 -- 172.21.15.114:0/3976250344 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7418102180 msgr2=0x7f7418102560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:29.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.219+0000 7f74205ff640 1 --2- 172.21.15.114:0/3976250344 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7418102180 0x7f7418102560 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f7414009890 tx=0x7f741402a380 comp rx=0 tx=0).stop 2024-09-13T18:26:29.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.219+0000 7f74205ff640 1 -- 172.21.15.114:0/3976250344 shutdown_connections 2024-09-13T18:26:29.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.219+0000 7f74205ff640 1 --2- 172.21.15.114:0/3976250344 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74181072d0 0x7f741810ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.219+0000 7f74205ff640 1 --2- 172.21.15.114:0/3976250344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7418102b30 0x7f7418106b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.219+0000 7f74205ff640 1 --2- 172.21.15.114:0/3976250344 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7418102180 0x7f7418102560 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.219+0000 7f74205ff640 1 -- 172.21.15.114:0/3976250344 >> 172.21.15.114:0/3976250344 conn(0x7f74180fdbd0 msgr2=0x7f74180ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:29.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.220+0000 7f74205ff640 1 -- 172.21.15.114:0/3976250344 shutdown_connections 2024-09-13T18:26:29.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.220+0000 7f74205ff640 1 -- 172.21.15.114:0/3976250344 wait complete. 2024-09-13T18:26:29.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.220+0000 7f74205ff640 1 Processor -- start 2024-09-13T18:26:29.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.221+0000 7f74205ff640 1 -- start start 2024-09-13T18:26:29.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.221+0000 7f74205ff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7418102180 0x7f74181947a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.221+0000 7f74205ff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74181072d0 0x7f7418194ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.221+0000 7f74205ff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7418198930 0x7f7418198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.221+0000 7f74205ff640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7418199320 con 0x7f74181072d0 2024-09-13T18:26:29.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.221+0000 7f74205ff640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7418199490 con 0x7f7418198930 2024-09-13T18:26:29.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.221+0000 7f74205ff640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7418199600 con 0x7f7418102180 2024-09-13T18:26:29.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.221+0000 7f741e39b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7418102180 0x7f74181947a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.221+0000 7f741eb9c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7418198930 0x7f7418198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.221+0000 7f741eb9c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7418198930 0x7f7418198de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:60642/0 (socket says 172.21.15.114:60642) 2024-09-13T18:26:29.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.221+0000 7f741eb9c640 1 -- 172.21.15.114:0/2708812531 learned_addr learned my addr 172.21.15.114:0/2708812531 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:29.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.222+0000 7f741e39b640 1 -- 172.21.15.114:0/2708812531 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7418198930 msgr2=0x7f7418198de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:29.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.222+0000 7f741e39b640 1 --2- 172.21.15.114:0/2708812531 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7418198930 0x7f7418198de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.222+0000 7f741e39b640 1 -- 172.21.15.114:0/2708812531 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74181072d0 msgr2=0x7f7418194ce0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:26:29.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.222+0000 7f741e39b640 1 --2- 172.21.15.114:0/2708812531 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74181072d0 0x7f7418194ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.222+0000 7f741e39b640 1 -- 172.21.15.114:0/2708812531 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7408011d70 con 0x7f7418102180 2024-09-13T18:26:29.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.222+0000 7f741e39b640 1 --2- 172.21.15.114:0/2708812531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7418102180 0x7f74181947a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f741402a8a0 tx=0x7f7414033d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:29.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.226+0000 7f740f7fe640 1 -- 172.21.15.114:0/2708812531 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74140308f0 con 0x7f7418102180 2024-09-13T18:26:29.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.226+0000 7f740f7fe640 1 -- 172.21.15.114:0/2708812531 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f74140044d0 con 0x7f7418102180 2024-09-13T18:26:29.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.226+0000 7f740f7fe640 1 -- 172.21.15.114:0/2708812531 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741403d440 con 0x7f7418102180 2024-09-13T18:26:29.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.226+0000 7f74205ff640 1 -- 172.21.15.114:0/2708812531 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7414009490 con 0x7f7418102180 2024-09-13T18:26:29.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.226+0000 7f74205ff640 1 -- 172.21.15.114:0/2708812531 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74181a0e00 con 0x7f7418102180 2024-09-13T18:26:29.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.231+0000 7f74233f9740 1 Processor -- start 2024-09-13T18:26:29.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.232+0000 7f74233f9740 1 -- start start 2024-09-13T18:26:29.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.232+0000 7f74233f9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c69b656790 0x55c69b656b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.232+0000 7f74233f9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c69b6570b0 0x55c69b661410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.232+0000 7f74233f9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c69b661950 0x55c69b663d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.232+0000 7f74233f9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c69b5c0cc0 con 0x55c69b661950 2024-09-13T18:26:29.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.232+0000 7f74233f9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c69b5be450 con 0x55c69b6570b0 2024-09-13T18:26:29.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.232+0000 7f74233f9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c69b5b9d40 con 0x55c69b656790 2024-09-13T18:26:29.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.232+0000 7f740f7fe640 1 -- 172.21.15.114:0/2708812531 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f7414039040 con 0x7f7418102180 2024-09-13T18:26:29.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.233+0000 7f741e39b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c69b6570b0 0x55c69b661410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.233+0000 7f741eb9c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c69b656790 0x55c69b656b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.233+0000 7f741eb9c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c69b656790 0x55c69b656b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:52676/0 (socket says 172.21.15.114:52676) 2024-09-13T18:26:29.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.233+0000 7f741eb9c640 1 -- 172.21.15.114:0/2734147479 learned_addr learned my addr 172.21.15.114:0/2734147479 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:29.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.234+0000 7f741db9a640 1 --2- 172.21.15.114:0/2734147479 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c69b661950 0x55c69b663d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.236+0000 7f740f7fe640 1 --2- 172.21.15.114:0/2708812531 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f74040a13d0 0x7f74040a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.236+0000 7f741eb9c640 1 -- 172.21.15.114:0/2734147479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c69b6570b0 msgr2=0x55c69b661410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:29.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.237+0000 7f741eb9c640 1 --2- 172.21.15.114:0/2734147479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c69b6570b0 0x55c69b661410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.237+0000 7f741eb9c640 1 -- 172.21.15.114:0/2734147479 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c69b661950 msgr2=0x55c69b663d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:29.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.237+0000 7f741eb9c640 1 --2- 172.21.15.114:0/2734147479 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c69b661950 0x55c69b663d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.237+0000 7f741eb9c640 1 -- 172.21.15.114:0/2734147479 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c69b581620 con 0x55c69b656790 2024-09-13T18:26:29.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.237+0000 7f741eb9c640 1 --2- 172.21.15.114:0/2708812531 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f74040a13d0 0x7f74040a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.238+0000 7f741eb9c640 1 --2- 172.21.15.114:0/2734147479 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c69b656790 0x55c69b656b70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7410007540 tx=0x7f7410007a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8447a0eae436d7b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:29.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.238+0000 7f741db9a640 1 --2- 172.21.15.114:0/2734147479 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c69b661950 0x55c69b663d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:26:29.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.238+0000 7f741e39b640 1 --2- 172.21.15.114:0/2734147479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c69b6570b0 0x55c69b661410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:26:29.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.238+0000 7f740f7fe640 1 -- 172.21.15.114:0/2708812531 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f74140ed8e0 con 0x7f7418102180 2024-09-13T18:26:29.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.243+0000 7f740cff9640 1 -- 172.21.15.114:0/2734147479 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7410020630 con 0x55c69b656790 2024-09-13T18:26:29.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.243+0000 7f740cff9640 1 -- 172.21.15.114:0/2734147479 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7410016580 con 0x55c69b656790 2024-09-13T18:26:29.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.243+0000 7f740cff9640 1 -- 172.21.15.114:0/2734147479 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741001f970 con 0x55c69b656790 2024-09-13T18:26:29.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.244+0000 7f74233f9740 1 -- 172.21.15.114:0/2734147479 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c69b656790 msgr2=0x55c69b656b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:29.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.244+0000 7f74233f9740 1 --2- 172.21.15.114:0/2734147479 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c69b656790 0x55c69b656b70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7410007540 tx=0x7f7410007a00 comp rx=0 tx=0).stop 2024-09-13T18:26:29.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.244+0000 7f741eb9c640 1 --2- 172.21.15.114:0/2708812531 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f74040a13d0 0x7f74040a3890 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f741001e000 tx=0x7f741001b000 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:29.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.244+0000 7f74233f9740 1 -- 172.21.15.114:0/2734147479 shutdown_connections 2024-09-13T18:26:29.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.244+0000 7f74233f9740 1 --2- 172.21.15.114:0/2734147479 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c69b661950 0x55c69b663d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.244+0000 7f74233f9740 1 --2- 172.21.15.114:0/2734147479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c69b6570b0 0x55c69b661410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.244+0000 7f74233f9740 1 --2- 172.21.15.114:0/2734147479 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c69b656790 0x55c69b656b70 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.244+0000 7f74233f9740 1 -- 172.21.15.114:0/2734147479 >> 172.21.15.114:0/2734147479 conn(0x55c69b652d00 msgr2=0x55c69b6530e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:29.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.245+0000 7f74233f9740 1 -- 172.21.15.114:0/2734147479 shutdown_connections 2024-09-13T18:26:29.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.248+0000 7f74233f9740 1 -- 172.21.15.114:0/2734147479 wait complete. 2024-09-13T18:26:29.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.249+0000 7f74233f9740 1 -- start start 2024-09-13T18:26:29.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.249+0000 7f74233f9740 1 Processor -- start 2024-09-13T18:26:29.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.255+0000 7f74233f9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c69b656790 0x55c69b660f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.255+0000 7f74233f9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c69b6570b0 0x55c69b65e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.255+0000 7f74233f9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c69b661950 0x55c69b65e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.255+0000 7f74233f9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c69b578180 con 0x55c69b6570b0 2024-09-13T18:26:29.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.255+0000 7f74233f9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c69b292430 con 0x55c69b661950 2024-09-13T18:26:29.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.255+0000 7f74233f9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c69b53d5e0 con 0x55c69b656790 2024-09-13T18:26:29.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.255+0000 7f74233f9740 10 client.0 _ll_register_callbacks cb 0x55c69b5a9aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:26:29.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.255+0000 7f741e39b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c69b656790 0x55c69b660f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.255+0000 7f741e39b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c69b656790 0x55c69b660f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:52682/0 (socket says 172.21.15.114:52682) 2024-09-13T18:26:29.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.255+0000 7f741e39b640 1 -- 172.21.15.114:0/617019728 learned_addr learned my addr 172.21.15.114:0/617019728 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:29.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.255+0000 7f741eb9c640 1 --2- 172.21.15.114:0/617019728 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c69b661950 0x55c69b65e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.255+0000 7f741db9a640 1 --2- 172.21.15.114:0/617019728 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c69b6570b0 0x55c69b65e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.256+0000 7f741e39b640 1 -- 172.21.15.114:0/617019728 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c69b661950 msgr2=0x55c69b65e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:29.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.256+0000 7f741e39b640 1 --2- 172.21.15.114:0/617019728 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c69b661950 0x55c69b65e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.256+0000 7f741e39b640 1 -- 172.21.15.114:0/617019728 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c69b6570b0 msgr2=0x55c69b65e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:29.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.256+0000 7f741e39b640 1 --2- 172.21.15.114:0/617019728 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c69b6570b0 0x55c69b65e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.256+0000 7f741e39b640 1 -- 172.21.15.114:0/617019728 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7410009c90 con 0x55c69b656790 2024-09-13T18:26:29.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.256+0000 7f741e39b640 1 --2- 172.21.15.114:0/617019728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c69b656790 0x55c69b660f50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7414009780 tx=0x7f74140b6bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=484f461ef7cd72bb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:29.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.256+0000 7f7401ffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:26:29.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.256+0000 7f7401ffb640 1 -- 172.21.15.114:0/617019728 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741400b0b0 con 0x55c69b656790 2024-09-13T18:26:29.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.257+0000 7f74233f9740 1 -- 172.21.15.114:0/617019728 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c69b5e3b80 con 0x55c69b656790 2024-09-13T18:26:29.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.257+0000 7f7401ffb640 1 -- 172.21.15.114:0/617019728 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f74140bae50 con 0x55c69b656790 2024-09-13T18:26:29.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.257+0000 7f7401ffb640 1 -- 172.21.15.114:0/617019728 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74140ba750 con 0x55c69b656790 2024-09-13T18:26:29.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.258+0000 7f74233f9740 10 client.26344 fetch_fsmap learned FSMap version 32 2024-09-13T18:26:29.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.258+0000 7f74233f9740 1 -- 172.21.15.114:0/617019728 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55c69b5b83c0 con 0x55c69b656790 2024-09-13T18:26:29.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.258+0000 7f7401ffb640 1 -- 172.21.15.114:0/617019728 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c69b5e3b80 con 0x55c69b656790 2024-09-13T18:26:29.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.258+0000 7f7401ffb640 1 -- 172.21.15.114:0/617019728 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f74140b6e60 con 0x55c69b656790 2024-09-13T18:26:29.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.260+0000 7f74233f9740 10 client.26344 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:26:29.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.260+0000 7f74233f9740 10 client.26344 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-13T18:26:29.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.260+0000 7f74233f9740 20 client.26344 populate_metadata read hostname 'smithi114' 2024-09-13T18:26:29.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.260+0000 7f74233f9740 1 --2- 172.21.15.114:0/617019728 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55c69b671010 0x55c69b673400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.260+0000 7f74233f9740 4 client.26344 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:26:29.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.260+0000 7f74233f9740 1 -- 172.21.15.114:0/617019728 --> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c69b5ac1f0 con 0x55c69b671010 2024-09-13T18:26:29.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.260+0000 7f741db9a640 1 --2- 172.21.15.114:0/617019728 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55c69b671010 0x55c69b673400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.261+0000 7f741db9a640 1 --2- 172.21.15.114:0/617019728 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55c69b671010 0x55c69b673400 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=75e32a8b52716ffe server_cookie=455c37a9bd25213d in_seq=0 out_seq=0 2024-09-13T18:26:29.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.261+0000 7f7401ffb640 10 client.26344 ms_handle_connect on v2:172.21.15.204:6838/344883912 2024-09-13T18:26:29.264 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.262+0000 7f7401ffb640 1 -- 172.21.15.114:0/617019728 <== mds.2 v2:172.21.15.204:6838/344883912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55c69b5ac1f0 con 0x55c69b671010 2024-09-13T18:26:29.264 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.262+0000 7f7401ffb640 10 client.26344 handle_command_reply: tid=0 2024-09-13T18:26:29.264 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.262+0000 7f74233f9740 1 client.26344 shutdown 2024-09-13T18:26:29.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.263+0000 7f74233f9740 1 -- 172.21.15.114:0/617019728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c69b656790 msgr2=0x55c69b660f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:29.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.263+0000 7f74233f9740 1 --2- 172.21.15.114:0/617019728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c69b656790 0x55c69b660f50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7414009780 tx=0x7f74140b6bc0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.263+0000 7f74233f9740 1 -- 172.21.15.114:0/617019728 shutdown_connections 2024-09-13T18:26:29.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.263+0000 7f74233f9740 1 --2- 172.21.15.114:0/617019728 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55c69b671010 0x55c69b673400 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.263+0000 7f74233f9740 1 --2- 172.21.15.114:0/617019728 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c69b661950 0x55c69b65e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.263+0000 7f74233f9740 1 --2- 172.21.15.114:0/617019728 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c69b6570b0 0x55c69b65e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.263+0000 7f7401ffb640 0 client.26344 ms_handle_reset on v2:172.21.15.204:6838/344883912 2024-09-13T18:26:29.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.263+0000 7f74233f9740 1 --2- 172.21.15.114:0/617019728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c69b656790 0x55c69b660f50 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.263+0000 7f74233f9740 1 -- 172.21.15.114:0/617019728 >> 172.21.15.114:0/617019728 conn(0x55c69b652d00 msgr2=0x55c69b651eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:29.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.263+0000 7f74233f9740 1 -- 172.21.15.114:0/617019728 shutdown_connections 2024-09-13T18:26:29.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.263+0000 7f74233f9740 1 -- 172.21.15.114:0/617019728 wait complete. 2024-09-13T18:26:29.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.264+0000 7f74233f9740 20 client.26344 trim_cache size 0 max 16384 2024-09-13T18:26:29.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.286+0000 7f74233f9740 1 Processor -- start 2024-09-13T18:26:29.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.286+0000 7f74233f9740 1 -- start start 2024-09-13T18:26:29.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.286+0000 7f74233f9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c69b656790 0x55c69b6609f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.287+0000 7f74233f9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c69b6570b0 0x55c69b660f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.287+0000 7f74233f9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c69b661950 0x55c69b65e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.287+0000 7f74233f9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c69b664400 con 0x55c69b6570b0 2024-09-13T18:26:29.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.287+0000 7f74233f9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c69b65c7c0 con 0x55c69b661950 2024-09-13T18:26:29.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.287+0000 7f741eb9c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c69b661950 0x55c69b65e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.287+0000 7f741eb9c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c69b661950 0x55c69b65e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:60664/0 (socket says 172.21.15.114:60664) 2024-09-13T18:26:29.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.287+0000 7f741eb9c640 1 -- 172.21.15.114:0/1825003776 learned_addr learned my addr 172.21.15.114:0/1825003776 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:29.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.287+0000 7f74233f9740 1 -- 172.21.15.114:0/1825003776 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c69b678520 con 0x55c69b656790 2024-09-13T18:26:29.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.287+0000 7f741e39b640 1 --2- 172.21.15.114:0/1825003776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c69b656790 0x55c69b6609f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.287+0000 7f741db9a640 1 --2- 172.21.15.114:0/1825003776 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c69b6570b0 0x55c69b660f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.288+0000 7f741e39b640 1 -- 172.21.15.114:0/1825003776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c69b661950 msgr2=0x55c69b65e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:29.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.288+0000 7f741e39b640 1 --2- 172.21.15.114:0/1825003776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c69b661950 0x55c69b65e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.288+0000 7f741e39b640 1 -- 172.21.15.114:0/1825003776 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c69b6570b0 msgr2=0x55c69b660f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:29.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.288+0000 7f741e39b640 1 --2- 172.21.15.114:0/1825003776 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c69b6570b0 0x55c69b660f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.288+0000 7f741e39b640 1 -- 172.21.15.114:0/1825003776 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c69b55a770 con 0x55c69b656790 2024-09-13T18:26:29.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.289+0000 7f741e39b640 1 --2- 172.21.15.114:0/1825003776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c69b656790 0x55c69b6609f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f74140b6c50 tx=0x7f741402a380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8aba64507b388a9d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:29.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.290+0000 7f740d7fa640 1 -- 172.21.15.114:0/1825003776 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74140be2f0 con 0x55c69b656790 2024-09-13T18:26:29.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.291+0000 7f740d7fa640 1 -- 172.21.15.114:0/1825003776 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f74140ba290 con 0x55c69b656790 2024-09-13T18:26:29.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.291+0000 7f740d7fa640 1 -- 172.21.15.114:0/1825003776 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74140c2c60 con 0x55c69b656790 2024-09-13T18:26:29.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.292+0000 7f74233f9740 1 -- 172.21.15.114:0/1825003776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c69b656790 msgr2=0x55c69b6609f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:29.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.292+0000 7f74233f9740 1 --2- 172.21.15.114:0/1825003776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c69b656790 0x55c69b6609f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f74140b6c50 tx=0x7f741402a380 comp rx=0 tx=0).stop 2024-09-13T18:26:29.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.295+0000 7f74233f9740 1 -- 172.21.15.114:0/1825003776 shutdown_connections 2024-09-13T18:26:29.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.295+0000 7f74233f9740 1 --2- 172.21.15.114:0/1825003776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c69b661950 0x55c69b65e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.295+0000 7f74233f9740 1 --2- 172.21.15.114:0/1825003776 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c69b6570b0 0x55c69b660f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.295+0000 7f74233f9740 1 --2- 172.21.15.114:0/1825003776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c69b656790 0x55c69b6609f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.295+0000 7f74233f9740 1 -- 172.21.15.114:0/1825003776 >> 172.21.15.114:0/1825003776 conn(0x55c69b652d00 msgr2=0x55c69b65d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:29.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.298+0000 7f74233f9740 1 -- 172.21.15.114:0/1825003776 shutdown_connections 2024-09-13T18:26:29.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.298+0000 7f74233f9740 1 -- 172.21.15.114:0/1825003776 wait complete. 2024-09-13T18:26:29.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.300+0000 7f74233f9740 1 -- start start 2024-09-13T18:26:29.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.302+0000 7f74233f9740 1 Processor -- start 2024-09-13T18:26:29.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.303+0000 7f74233f9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c69b656790 0x55c69b664c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.303+0000 7f74233f9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c69b6570b0 0x55c69b665170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.303+0000 7f74233f9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c69b661950 0x55c69b667f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.303+0000 7f74233f9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c69b668500 con 0x55c69b656790 2024-09-13T18:26:29.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.303+0000 7f74233f9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c69b668670 con 0x55c69b661950 2024-09-13T18:26:29.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.303+0000 7f74233f9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c69b668830 con 0x55c69b6570b0 2024-09-13T18:26:29.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.303+0000 7f74233f9740 10 client.0 _ll_register_callbacks cb 0x7f7410009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:26:29.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.303+0000 7f741db9a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c69b6570b0 0x55c69b665170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.303+0000 7f741db9a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c69b6570b0 0x55c69b665170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:52692/0 (socket says 172.21.15.114:52692) 2024-09-13T18:26:29.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.303+0000 7f741db9a640 1 -- 172.21.15.114:0/3421280199 learned_addr learned my addr 172.21.15.114:0/3421280199 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:29.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.305+0000 7f741eb9c640 1 --2- 172.21.15.114:0/3421280199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c69b661950 0x55c69b667f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.305+0000 7f741e39b640 1 --2- 172.21.15.114:0/3421280199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c69b656790 0x55c69b664c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.305+0000 7f741db9a640 1 -- 172.21.15.114:0/3421280199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c69b661950 msgr2=0x55c69b667f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:29.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.305+0000 7f741db9a640 1 --2- 172.21.15.114:0/3421280199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c69b661950 0x55c69b667f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.305+0000 7f741db9a640 1 -- 172.21.15.114:0/3421280199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c69b656790 msgr2=0x55c69b664c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:29.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.305+0000 7f741db9a640 1 --2- 172.21.15.114:0/3421280199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c69b656790 0x55c69b664c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.305+0000 7f741db9a640 1 -- 172.21.15.114:0/3421280199 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c69b5b83c0 con 0x55c69b6570b0 2024-09-13T18:26:29.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.309+0000 7f741db9a640 1 --2- 172.21.15.114:0/3421280199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c69b6570b0 0x55c69b665170 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7418067e70 tx=0x7f7408011a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64a3c6bc660b55d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:29.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.311+0000 7f7401ffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:26:29.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.311+0000 7f7401ffb640 1 -- 172.21.15.114:0/3421280199 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7408023030 con 0x55c69b6570b0 2024-09-13T18:26:29.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.311+0000 7f7401ffb640 1 -- 172.21.15.114:0/3421280199 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f7408024070 con 0x55c69b6570b0 2024-09-13T18:26:29.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.311+0000 7f74233f9740 1 -- 172.21.15.114:0/3421280199 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7410016580 con 0x55c69b6570b0 2024-09-13T18:26:29.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.314+0000 7f7401ffb640 1 -- 172.21.15.114:0/3421280199 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7408016040 con 0x55c69b6570b0 2024-09-13T18:26:29.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.314+0000 7f74233f9740 10 client.26350 fetch_fsmap learned FSMap version 32 2024-09-13T18:26:29.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.314+0000 7f74233f9740 1 -- 172.21.15.114:0/3421280199 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55c69b57fe80 con 0x55c69b6570b0 2024-09-13T18:26:29.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.314+0000 7f7401ffb640 1 -- 172.21.15.114:0/3421280199 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7410016580 con 0x55c69b6570b0 2024-09-13T18:26:29.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.316+0000 7f7401ffb640 1 -- 172.21.15.114:0/3421280199 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f740800b070 con 0x55c69b6570b0 2024-09-13T18:26:29.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.316+0000 7f74233f9740 10 client.26350 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:26:29.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.316+0000 7f74233f9740 10 client.26350 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-13T18:26:29.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.316+0000 7f74233f9740 20 client.26350 populate_metadata read hostname 'smithi114' 2024-09-13T18:26:29.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.316+0000 7f74233f9740 1 --2- 172.21.15.114:0/3421280199 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55c69b6d9720 0x55c69b6dbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.316+0000 7f74233f9740 4 client.26350 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:26:29.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.316+0000 7f74233f9740 1 -- 172.21.15.114:0/3421280199 --> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c69b2ad4e0 con 0x55c69b6d9720 2024-09-13T18:26:29.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.316+0000 7f741e39b640 1 --2- 172.21.15.114:0/3421280199 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55c69b6d9720 0x55c69b6dbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.319+0000 7f741e39b640 1 --2- 172.21.15.114:0/3421280199 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55c69b6d9720 0x55c69b6dbb40 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=344658439b80c6ee server_cookie=3f674be62fb3a877 in_seq=0 out_seq=0 2024-09-13T18:26:29.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.320+0000 7f7401ffb640 10 client.26350 ms_handle_connect on v2:172.21.15.204:6838/344883912 2024-09-13T18:26:29.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.320+0000 7f7401ffb640 1 -- 172.21.15.114:0/3421280199 <== mds.2 v2:172.21.15.204:6838/344883912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c69b2ad4e0 con 0x55c69b6d9720 2024-09-13T18:26:29.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.320+0000 7f7401ffb640 10 client.26350 handle_command_reply: tid=0 2024-09-13T18:26:29.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.320+0000 7f74233f9740 1 client.26350 shutdown 2024-09-13T18:26:29.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.321+0000 7f74233f9740 1 -- 172.21.15.114:0/3421280199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c69b6570b0 msgr2=0x55c69b665170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:29.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.321+0000 7f74233f9740 1 --2- 172.21.15.114:0/3421280199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c69b6570b0 0x55c69b665170 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7418067e70 tx=0x7f7408011a70 comp rx=0 tx=0).stop 2024-09-13T18:26:29.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.322+0000 7f74233f9740 1 -- 172.21.15.114:0/3421280199 shutdown_connections 2024-09-13T18:26:29.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.322+0000 7f74233f9740 1 --2- 172.21.15.114:0/3421280199 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55c69b6d9720 0x55c69b6dbb40 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.322+0000 7f74233f9740 1 --2- 172.21.15.114:0/3421280199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c69b661950 0x55c69b667f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.322+0000 7f74233f9740 1 --2- 172.21.15.114:0/3421280199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c69b6570b0 0x55c69b665170 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.322+0000 7f74233f9740 1 --2- 172.21.15.114:0/3421280199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c69b656790 0x55c69b664c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.322+0000 7f74233f9740 1 -- 172.21.15.114:0/3421280199 >> 172.21.15.114:0/3421280199 conn(0x55c69b652d00 msgr2=0x55c69b659360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:29.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.322+0000 7f7401ffb640 0 client.26350 ms_handle_reset on v2:172.21.15.204:6838/344883912 2024-09-13T18:26:29.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.323+0000 7f74233f9740 1 -- 172.21.15.114:0/3421280199 shutdown_connections 2024-09-13T18:26:29.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.324+0000 7f74233f9740 1 -- 172.21.15.114:0/3421280199 wait complete. 2024-09-13T18:26:29.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.327+0000 7f74233f9740 20 client.26350 trim_cache size 0 max 16384 2024-09-13T18:26:29.330 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:26:29.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.328+0000 7f74205ff640 1 -- 172.21.15.114:0/2708812531 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f74040a13d0 msgr2=0x7f74040a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:29.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.328+0000 7f74205ff640 1 --2- 172.21.15.114:0/2708812531 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f74040a13d0 0x7f74040a3890 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f741001e000 tx=0x7f741001b000 comp rx=0 tx=0).stop 2024-09-13T18:26:29.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.328+0000 7f74205ff640 1 -- 172.21.15.114:0/2708812531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7418102180 msgr2=0x7f74181947a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:29.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.328+0000 7f74205ff640 1 --2- 172.21.15.114:0/2708812531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7418102180 0x7f74181947a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f741402a8a0 tx=0x7f7414033d00 comp rx=0 tx=0).stop 2024-09-13T18:26:29.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.330+0000 7f74205ff640 1 -- 172.21.15.114:0/2708812531 shutdown_connections 2024-09-13T18:26:29.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.330+0000 7f74205ff640 1 --2- 172.21.15.114:0/2708812531 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f74040a13d0 0x7f74040a3890 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.330+0000 7f74205ff640 1 --2- 172.21.15.114:0/2708812531 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7418198930 0x7f7418198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.330+0000 7f74205ff640 1 --2- 172.21.15.114:0/2708812531 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74181072d0 0x7f7418194ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.330+0000 7f74205ff640 1 --2- 172.21.15.114:0/2708812531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7418102180 0x7f74181947a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.330+0000 7f74205ff640 1 -- 172.21.15.114:0/2708812531 >> 172.21.15.114:0/2708812531 conn(0x7f74180fdbd0 msgr2=0x7f7418105b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:29.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.331+0000 7f74205ff640 1 -- 172.21.15.114:0/2708812531 shutdown_connections 2024-09-13T18:26:29.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.331+0000 7f74205ff640 1 -- 172.21.15.114:0/2708812531 wait complete. 2024-09-13T18:26:29.365 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-09-13T18:26:29.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.599+0000 7f692d2a6640 1 -- 172.21.15.114:0/1253475467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6928119400 msgr2=0x7f692811b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:29.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.599+0000 7f692d2a6640 1 --2- 172.21.15.114:0/1253475467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6928119400 0x7f692811b7f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6920009be0 tx=0x7f692002a6c0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.599+0000 7f692d2a6640 1 -- 172.21.15.114:0/1253475467 shutdown_connections 2024-09-13T18:26:29.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.599+0000 7f692d2a6640 1 --2- 172.21.15.114:0/1253475467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6928119400 0x7f692811b7f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.599+0000 7f692d2a6640 1 --2- 172.21.15.114:0/1253475467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6928107890 0x7f6928107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.599+0000 7f692d2a6640 1 --2- 172.21.15.114:0/1253475467 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69281074b0 0x7f692810e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.599+0000 7f692d2a6640 1 -- 172.21.15.114:0/1253475467 >> 172.21.15.114:0/1253475467 conn(0x7f692806af40 msgr2=0x7f692806b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:29.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.600+0000 7f692d2a6640 1 -- 172.21.15.114:0/1253475467 shutdown_connections 2024-09-13T18:26:29.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.600+0000 7f692d2a6640 1 -- 172.21.15.114:0/1253475467 wait complete. 2024-09-13T18:26:29.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.601+0000 7f692d2a6640 1 Processor -- start 2024-09-13T18:26:29.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.601+0000 7f692d2a6640 1 -- start start 2024-09-13T18:26:29.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.601+0000 7f692d2a6640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69281074b0 0x7f69281a4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.601+0000 7f692d2a6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6928107890 0x7f69281a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.601+0000 7f692d2a6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6928119400 0x7f69281a9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.601+0000 7f692d2a6640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69281a9e40 con 0x7f69281074b0 2024-09-13T18:26:29.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.601+0000 7f692d2a6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69281a9fb0 con 0x7f6928107890 2024-09-13T18:26:29.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.601+0000 7f692d2a6640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69281aa170 con 0x7f6928119400 2024-09-13T18:26:29.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.601+0000 7f691ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6928107890 0x7f69281a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.601+0000 7f692caa5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6928119400 0x7f69281a9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.601+0000 7f692caa5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6928119400 0x7f69281a9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:52710/0 (socket says 172.21.15.114:52710) 2024-09-13T18:26:29.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.601+0000 7f692caa5640 1 -- 172.21.15.114:0/1133077298 learned_addr learned my addr 172.21.15.114:0/1133077298 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:29.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.602+0000 7f6927fff640 1 --2- 172.21.15.114:0/1133077298 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69281074b0 0x7f69281a4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.602+0000 7f691ffff640 1 -- 172.21.15.114:0/1133077298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6928119400 msgr2=0x7f69281a9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:29.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.602+0000 7f691ffff640 1 --2- 172.21.15.114:0/1133077298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6928119400 0x7f69281a9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.602+0000 7f691ffff640 1 -- 172.21.15.114:0/1133077298 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69281074b0 msgr2=0x7f69281a4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:29.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.602+0000 7f691ffff640 1 --2- 172.21.15.114:0/1133077298 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69281074b0 0x7f69281a4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.602+0000 7f691ffff640 1 -- 172.21.15.114:0/1133077298 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6918009490 con 0x7f6928107890 2024-09-13T18:26:29.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.602+0000 7f691ffff640 1 --2- 172.21.15.114:0/1133077298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6928107890 0x7f69281a5450 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f691000e790 tx=0x7f691000ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:29.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.603+0000 7f6925ffb640 1 -- 172.21.15.114:0/1133077298 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691000cc40 con 0x7f6928107890 2024-09-13T18:26:29.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.603+0000 7f692d2a6640 1 -- 172.21.15.114:0/1133077298 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6920009840 con 0x7f6928107890 2024-09-13T18:26:29.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.603+0000 7f692d2a6640 1 -- 172.21.15.114:0/1133077298 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69281b1af0 con 0x7f6928107890 2024-09-13T18:26:29.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.603+0000 7f6925ffb640 1 -- 172.21.15.114:0/1133077298 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f691000eea0 con 0x7f6928107890 2024-09-13T18:26:29.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.603+0000 7f6925ffb640 1 -- 172.21.15.114:0/1133077298 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6910018680 con 0x7f6928107890 2024-09-13T18:26:29.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.608+0000 7f6931227740 1 Processor -- start 2024-09-13T18:26:29.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.608+0000 7f6931227740 1 -- start start 2024-09-13T18:26:29.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.608+0000 7f6931227740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56312cbfc970 0x56312cbfcd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.608+0000 7f6931227740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56312cbfd290 0x56312cc075f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.608+0000 7f6931227740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56312cc07b30 0x56312cc09f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.608+0000 7f6931227740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56312cb66cc0 con 0x56312cbfc970 2024-09-13T18:26:29.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.608+0000 7f6931227740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56312cb64450 con 0x56312cbfd290 2024-09-13T18:26:29.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.608+0000 7f6931227740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56312cb5fd40 con 0x56312cc07b30 2024-09-13T18:26:29.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.608+0000 7f6925ffb640 1 -- 172.21.15.114:0/1133077298 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f6910010430 con 0x7f6928107890 2024-09-13T18:26:29.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.608+0000 7f6927fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56312cbfd290 0x56312cc075f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.608+0000 7f6927fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56312cbfd290 0x56312cc075f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:60698/0 (socket says 172.21.15.114:60698) 2024-09-13T18:26:29.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.608+0000 7f6927fff640 1 -- 172.21.15.114:0/2981700814 learned_addr learned my addr 172.21.15.114:0/2981700814 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:29.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.609+0000 7f691ffff640 1 --2- 172.21.15.114:0/2981700814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56312cc07b30 0x56312cc09f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.609+0000 7f691ffff640 1 -- 172.21.15.114:0/2981700814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56312cbfd290 msgr2=0x56312cc075f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:29.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.609+0000 7f691ffff640 1 --2- 172.21.15.114:0/2981700814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56312cbfd290 0x56312cc075f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.609+0000 7f691ffff640 1 -- 172.21.15.114:0/2981700814 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56312cbfc970 msgr2=0x56312cbfcd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:29.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.609+0000 7f691ffff640 1 --2- 172.21.15.114:0/2981700814 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56312cbfc970 0x56312cbfcd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.609+0000 7f691ffff640 1 -- 172.21.15.114:0/2981700814 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56312cb27620 con 0x56312cc07b30 2024-09-13T18:26:29.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.609+0000 7f691ffff640 1 --2- 172.21.15.114:0/2981700814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56312cc07b30 0x56312cc09f20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f69100ca430 tx=0x7f69100d54b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d870e1b25449394 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:29.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.610+0000 7f6925ffb640 1 --2- 172.21.15.114:0/1133077298 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f69140a1340 0x7f69140a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.611+0000 7f692caa5640 1 --2- 172.21.15.114:0/1133077298 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f69140a1340 0x7f69140a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.611+0000 7f692caa5640 1 --2- 172.21.15.114:0/1133077298 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f69140a1340 0x7f69140a3800 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6920031040 tx=0x7f6920005bd0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:29.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.611+0000 7f6925ffb640 1 -- 172.21.15.114:0/1133077298 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f6910014070 con 0x7f6928107890 2024-09-13T18:26:29.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.612+0000 7f691e7fc640 1 -- 172.21.15.114:0/2981700814 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69100db400 con 0x56312cc07b30 2024-09-13T18:26:29.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.613+0000 7f691e7fc640 1 -- 172.21.15.114:0/2981700814 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f69100db9f0 con 0x56312cc07b30 2024-09-13T18:26:29.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.614+0000 7f6931227740 1 -- 172.21.15.114:0/2981700814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56312cc07b30 msgr2=0x56312cc09f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:29.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.614+0000 7f6931227740 1 --2- 172.21.15.114:0/2981700814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56312cc07b30 0x56312cc09f20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f69100ca430 tx=0x7f69100d54b0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.616+0000 7f6931227740 1 -- 172.21.15.114:0/2981700814 shutdown_connections 2024-09-13T18:26:29.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.616+0000 7f6931227740 1 --2- 172.21.15.114:0/2981700814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56312cc07b30 0x56312cc09f20 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.616+0000 7f6931227740 1 --2- 172.21.15.114:0/2981700814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56312cbfd290 0x56312cc075f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.616+0000 7f6931227740 1 --2- 172.21.15.114:0/2981700814 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56312cbfc970 0x56312cbfcd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.616+0000 7f6931227740 1 -- 172.21.15.114:0/2981700814 >> 172.21.15.114:0/2981700814 conn(0x56312cbf8ee0 msgr2=0x56312cbf92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:29.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:26:29 smithi134 ceph-mon[34283]: pgmap v123: 129 pgs: 129 active+clean; 14 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 44 MiB/s wr, 710 op/s 2024-09-13T18:26:29.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.618+0000 7f6931227740 1 -- 172.21.15.114:0/2981700814 shutdown_connections 2024-09-13T18:26:29.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.625+0000 7f6931227740 1 -- 172.21.15.114:0/2981700814 wait complete. 2024-09-13T18:26:29.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.626+0000 7f6931227740 1 -- start start 2024-09-13T18:26:29.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.626+0000 7f6931227740 1 Processor -- start 2024-09-13T18:26:29.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.627+0000 7f6931227740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56312cbfc970 0x56312cc07090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.627+0000 7f6931227740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56312cbfd290 0x56312cc075d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.627+0000 7f6931227740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56312cc07b30 0x56312cc045e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.627+0000 7f6931227740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56312cb1e180 con 0x56312cbfd290 2024-09-13T18:26:29.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.627+0000 7f6931227740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56312c838430 con 0x56312cbfc970 2024-09-13T18:26:29.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.628+0000 7f6931227740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56312cae35e0 con 0x56312cc07b30 2024-09-13T18:26:29.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.628+0000 7f6931227740 10 client.0 _ll_register_callbacks cb 0x56312cb4faa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:26:29.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.628+0000 7f6927fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56312cbfc970 0x56312cc07090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.628+0000 7f6927fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56312cbfc970 0x56312cc07090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:60706/0 (socket says 172.21.15.114:60706) 2024-09-13T18:26:29.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.628+0000 7f6927fff640 1 -- 172.21.15.114:0/2018643864 learned_addr learned my addr 172.21.15.114:0/2018643864 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:29.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.628+0000 7f692caa5640 1 --2- 172.21.15.114:0/2018643864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56312cc07b30 0x56312cc045e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.630+0000 7f6927fff640 1 -- 172.21.15.114:0/2018643864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56312cc07b30 msgr2=0x56312cc045e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:29.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.630+0000 7f6927fff640 1 --2- 172.21.15.114:0/2018643864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56312cc07b30 0x56312cc045e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.630+0000 7f6927fff640 1 -- 172.21.15.114:0/2018643864 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56312cbfd290 msgr2=0x56312cc075d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:29.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.630+0000 7f6927fff640 1 --2- 172.21.15.114:0/2018643864 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56312cbfd290 0x56312cc075d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.630+0000 7f6927fff640 1 -- 172.21.15.114:0/2018643864 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69281b1af0 con 0x56312cbfc970 2024-09-13T18:26:29.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.631+0000 7f6927fff640 1 --2- 172.21.15.114:0/2018643864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56312cbfc970 0x56312cc07090 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6918012880 tx=0x7f6918012d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b566d81c106a247 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:29.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.633+0000 7f69077fe640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:26:29.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.633+0000 7f69077fe640 1 -- 172.21.15.114:0/2018643864 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69180122d0 con 0x56312cbfc970 2024-09-13T18:26:29.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.633+0000 7f6931227740 1 -- 172.21.15.114:0/2018643864 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56312cb89b80 con 0x56312cbfc970 2024-09-13T18:26:29.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.633+0000 7f69077fe640 1 -- 172.21.15.114:0/2018643864 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f691800f920 con 0x56312cbfc970 2024-09-13T18:26:29.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.633+0000 7f69077fe640 1 -- 172.21.15.114:0/2018643864 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f691800fae0 con 0x56312cbfc970 2024-09-13T18:26:29.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.633+0000 7f6931227740 10 client.26348 fetch_fsmap learned FSMap version 32 2024-09-13T18:26:29.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.633+0000 7f6931227740 1 -- 172.21.15.114:0/2018643864 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56312cb5e3c0 con 0x56312cbfc970 2024-09-13T18:26:29.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.637+0000 7f69077fe640 1 -- 172.21.15.114:0/2018643864 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56312cb89b80 con 0x56312cbfc970 2024-09-13T18:26:29.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.638+0000 7f69077fe640 1 -- 172.21.15.114:0/2018643864 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f6918010430 con 0x56312cbfc970 2024-09-13T18:26:29.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.638+0000 7f6931227740 10 client.26348 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:26:29.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.638+0000 7f6931227740 10 client.26348 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-13T18:26:29.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.638+0000 7f6931227740 20 client.26348 populate_metadata read hostname 'smithi114' 2024-09-13T18:26:29.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.638+0000 7f6931227740 1 --2- 172.21.15.114:0/2018643864 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x56312cc15170 0x56312cc17590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.638+0000 7f6931227740 4 client.26348 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:26:29.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.638+0000 7f6931227740 1 -- 172.21.15.114:0/2018643864 --> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56312cb521f0 con 0x56312cc15170 2024-09-13T18:26:29.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.639+0000 7f691ffff640 1 --2- 172.21.15.114:0/2018643864 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x56312cc15170 0x56312cc17590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.642+0000 7f691ffff640 1 --2- 172.21.15.114:0/2018643864 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x56312cc15170 0x56312cc17590 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dbcd47dc95204ed6 server_cookie=87bbb724cafcf4d2 in_seq=0 out_seq=0 2024-09-13T18:26:29.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.643+0000 7f69077fe640 10 client.26348 ms_handle_connect on v2:172.21.15.134:6836/891692210 2024-09-13T18:26:29.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.645+0000 7f69077fe640 1 -- 172.21.15.114:0/2018643864 <== mds.1 v2:172.21.15.134:6836/891692210 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x56312cb521f0 con 0x56312cc15170 2024-09-13T18:26:29.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.646+0000 7f69077fe640 10 client.26348 handle_command_reply: tid=0 2024-09-13T18:26:29.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.646+0000 7f6931227740 1 client.26348 shutdown 2024-09-13T18:26:29.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.648+0000 7f6931227740 1 -- 172.21.15.114:0/2018643864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56312cbfc970 msgr2=0x56312cc07090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:29.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.648+0000 7f6931227740 1 --2- 172.21.15.114:0/2018643864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56312cbfc970 0x56312cc07090 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6918012880 tx=0x7f6918012d40 comp rx=0 tx=0).stop 2024-09-13T18:26:29.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.650+0000 7f6931227740 1 -- 172.21.15.114:0/2018643864 shutdown_connections 2024-09-13T18:26:29.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.650+0000 7f6931227740 1 --2- 172.21.15.114:0/2018643864 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x56312cc15170 0x56312cc17590 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.650+0000 7f6931227740 1 --2- 172.21.15.114:0/2018643864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56312cc07b30 0x56312cc045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.650+0000 7f6931227740 1 --2- 172.21.15.114:0/2018643864 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56312cbfd290 0x56312cc075d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.650+0000 7f6931227740 1 --2- 172.21.15.114:0/2018643864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56312cbfc970 0x56312cc07090 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.650+0000 7f6931227740 1 -- 172.21.15.114:0/2018643864 >> 172.21.15.114:0/2018643864 conn(0x56312cbf8ee0 msgr2=0x56312cbf8230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:29.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.650+0000 7f69077fe640 0 client.26348 ms_handle_reset on v2:172.21.15.134:6836/891692210 2024-09-13T18:26:29.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.650+0000 7f6931227740 1 -- 172.21.15.114:0/2018643864 shutdown_connections 2024-09-13T18:26:29.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.650+0000 7f6931227740 1 -- 172.21.15.114:0/2018643864 wait complete. 2024-09-13T18:26:29.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.655+0000 7f6931227740 20 client.26348 trim_cache size 0 max 16384 2024-09-13T18:26:29.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.691+0000 7f6931227740 1 Processor -- start 2024-09-13T18:26:29.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.692+0000 7f6931227740 1 -- start start 2024-09-13T18:26:29.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:26:29 smithi204 ceph-mon[34777]: pgmap v123: 129 pgs: 129 active+clean; 14 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 44 MiB/s wr, 710 op/s 2024-09-13T18:26:29.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.692+0000 7f6931227740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56312cbfc970 0x56312cc24450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.692+0000 7f6931227740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56312cbfd290 0x56312cc1bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.692+0000 7f6931227740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56312cc07b30 0x56312cc0e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.692+0000 7f6931227740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56312cc04be0 con 0x56312cbfc970 2024-09-13T18:26:29.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.692+0000 7f6931227740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56312cbfd930 con 0x56312cc07b30 2024-09-13T18:26:29.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.692+0000 7f6931227740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56312cbfc6f0 con 0x56312cbfd290 2024-09-13T18:26:29.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.693+0000 7f6927fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56312cbfc970 0x56312cc24450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.693+0000 7f692caa5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56312cc07b30 0x56312cc0e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.693+0000 7f6927fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56312cbfc970 0x56312cc24450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49498/0 (socket says 172.21.15.114:49498) 2024-09-13T18:26:29.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.693+0000 7f6927fff640 1 -- 172.21.15.114:0/3647916768 learned_addr learned my addr 172.21.15.114:0/3647916768 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:29.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.693+0000 7f691ffff640 1 --2- 172.21.15.114:0/3647916768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56312cbfd290 0x56312cc1bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.693+0000 7f691ffff640 1 -- 172.21.15.114:0/3647916768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56312cc07b30 msgr2=0x56312cc0e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:29.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.693+0000 7f691ffff640 1 --2- 172.21.15.114:0/3647916768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56312cc07b30 0x56312cc0e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.693+0000 7f691ffff640 1 -- 172.21.15.114:0/3647916768 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56312cbfc970 msgr2=0x56312cc24450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:29.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.693+0000 7f691ffff640 1 --2- 172.21.15.114:0/3647916768 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56312cbfc970 0x56312cc24450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.693+0000 7f691ffff640 1 -- 172.21.15.114:0/3647916768 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56312cb00770 con 0x56312cbfd290 2024-09-13T18:26:29.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.693+0000 7f691ffff640 1 --2- 172.21.15.114:0/3647916768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56312cbfd290 0x56312cc1bf10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x56312cc03c20 tx=0x7f691000e710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f034995259d345e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:29.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.694+0000 7f691effd640 1 -- 172.21.15.114:0/3647916768 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69100d0840 con 0x56312cbfd290 2024-09-13T18:26:29.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.694+0000 7f691effd640 1 -- 172.21.15.114:0/3647916768 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f69100961e0 con 0x56312cbfd290 2024-09-13T18:26:29.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.694+0000 7f691effd640 1 -- 172.21.15.114:0/3647916768 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691000cda0 con 0x56312cbfd290 2024-09-13T18:26:29.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.695+0000 7f6931227740 1 -- 172.21.15.114:0/3647916768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56312cbfd290 msgr2=0x56312cc1bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:29.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.695+0000 7f6931227740 1 --2- 172.21.15.114:0/3647916768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56312cbfd290 0x56312cc1bf10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x56312cc03c20 tx=0x7f691000e710 comp rx=0 tx=0).stop 2024-09-13T18:26:29.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.695+0000 7f6931227740 1 -- 172.21.15.114:0/3647916768 shutdown_connections 2024-09-13T18:26:29.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.695+0000 7f6931227740 1 --2- 172.21.15.114:0/3647916768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56312cc07b30 0x56312cc0e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.695+0000 7f6931227740 1 --2- 172.21.15.114:0/3647916768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56312cbfd290 0x56312cc1bf10 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.695+0000 7f6931227740 1 --2- 172.21.15.114:0/3647916768 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56312cbfc970 0x56312cc24450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.695+0000 7f6931227740 1 -- 172.21.15.114:0/3647916768 >> 172.21.15.114:0/3647916768 conn(0x56312cbf8ee0 msgr2=0x56312cbf92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:29.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.695+0000 7f6931227740 1 -- 172.21.15.114:0/3647916768 shutdown_connections 2024-09-13T18:26:29.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.695+0000 7f6931227740 1 -- 172.21.15.114:0/3647916768 wait complete. 2024-09-13T18:26:29.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.696+0000 7f6931227740 1 -- start start 2024-09-13T18:26:29.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.696+0000 7f6931227740 1 Processor -- start 2024-09-13T18:26:29.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.696+0000 7f6931227740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56312cbfc970 0x56312cc0e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.697+0000 7f6931227740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56312cbfd290 0x56312cc0ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.697+0000 7f6927fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56312cbfc970 0x56312cc0e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.697+0000 7f6927fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56312cbfc970 0x56312cc0e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:52756/0 (socket says 172.21.15.114:52756) 2024-09-13T18:26:29.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.697+0000 7f6927fff640 1 -- 172.21.15.114:0/1627879393 learned_addr learned my addr 172.21.15.114:0/1627879393 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:29.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.697+0000 7f6931227740 1 --2- 172.21.15.114:0/1627879393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56312cc07b30 0x56312cc00600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.697+0000 7f6931227740 1 -- 172.21.15.114:0/1627879393 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56312cc00be0 con 0x56312cbfd290 2024-09-13T18:26:29.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.697+0000 7f691ffff640 1 --2- 172.21.15.114:0/1627879393 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56312cbfd290 0x56312cc0ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.697+0000 7f692caa5640 1 --2- 172.21.15.114:0/1627879393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56312cc07b30 0x56312cc00600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.697+0000 7f6931227740 1 -- 172.21.15.114:0/1627879393 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56312cc00d50 con 0x56312cc07b30 2024-09-13T18:26:29.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.697+0000 7f6931227740 1 -- 172.21.15.114:0/1627879393 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56312cc00f10 con 0x56312cbfc970 2024-09-13T18:26:29.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.698+0000 7f6927fff640 1 -- 172.21.15.114:0/1627879393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56312cc07b30 msgr2=0x56312cc00600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:29.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.698+0000 7f6927fff640 1 --2- 172.21.15.114:0/1627879393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56312cc07b30 0x56312cc00600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.698+0000 7f6927fff640 1 -- 172.21.15.114:0/1627879393 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56312cbfd290 msgr2=0x56312cc0ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:29.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.698+0000 7f6927fff640 1 --2- 172.21.15.114:0/1627879393 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56312cbfd290 0x56312cc0ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.698+0000 7f6927fff640 1 -- 172.21.15.114:0/1627879393 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56312cb27620 con 0x56312cbfc970 2024-09-13T18:26:29.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.698+0000 7f6931227740 10 client.0 _ll_register_callbacks cb 0x7f69100d0790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:26:29.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.698+0000 7f6927fff640 1 --2- 172.21.15.114:0/1627879393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56312cbfc970 0x56312cc0e930 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6918005bc0 tx=0x7f6918010660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22d96f1cd6b65ac8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:29.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.698+0000 7f69077fe640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:26:29.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.698+0000 7f6931227740 1 -- 172.21.15.114:0/1627879393 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f69100db9f0 con 0x56312cbfc970 2024-09-13T18:26:29.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.698+0000 7f69077fe640 1 -- 172.21.15.114:0/1627879393 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6918011f10 con 0x56312cbfc970 2024-09-13T18:26:29.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.699+0000 7f69077fe640 1 -- 172.21.15.114:0/1627879393 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f69180122d0 con 0x56312cbfc970 2024-09-13T18:26:29.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.699+0000 7f69077fe640 1 -- 172.21.15.114:0/1627879393 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6918012490 con 0x56312cbfc970 2024-09-13T18:26:29.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.699+0000 7f6931227740 10 client.26368 fetch_fsmap learned FSMap version 32 2024-09-13T18:26:29.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.699+0000 7f6931227740 1 -- 172.21.15.114:0/1627879393 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({fsmap=32}) v3 -- 0x56312cb25e80 con 0x56312cbfc970 2024-09-13T18:26:29.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.699+0000 7f69077fe640 1 -- 172.21.15.114:0/1627879393 <== mon.1 v2:172.21.15.204:3300/0 4 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f6918010bc0 con 0x56312cbfc970 2024-09-13T18:26:29.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.700+0000 7f6931227740 10 client.26368 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:26:29.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.700+0000 7f6931227740 10 client.26368 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-13T18:26:29.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.700+0000 7f6931227740 20 client.26368 populate_metadata read hostname 'smithi114' 2024-09-13T18:26:29.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.700+0000 7f6931227740 1 --2- 172.21.15.114:0/1627879393 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x56312cc6f060 0x56312cc71450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.700+0000 7f6931227740 4 client.26368 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:26:29.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.700+0000 7f6931227740 1 -- 172.21.15.114:0/1627879393 --> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f69100db9f0 con 0x56312cc6f060 2024-09-13T18:26:29.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.700+0000 7f691ffff640 1 --2- 172.21.15.114:0/1627879393 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x56312cc6f060 0x56312cc71450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.701+0000 7f691ffff640 1 --2- 172.21.15.114:0/1627879393 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x56312cc6f060 0x56312cc71450 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b3c1d34ca529ff30 server_cookie=94390c0d89aac7fc in_seq=0 out_seq=0 2024-09-13T18:26:29.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.701+0000 7f69077fe640 10 client.26368 ms_handle_connect on v2:172.21.15.134:6836/891692210 2024-09-13T18:26:29.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.701+0000 7f69077fe640 1 -- 172.21.15.114:0/1627879393 <== mds.1 v2:172.21.15.134:6836/891692210 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f69100db9f0 con 0x56312cc6f060 2024-09-13T18:26:29.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.701+0000 7f69077fe640 10 client.26368 handle_command_reply: tid=0 2024-09-13T18:26:29.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.701+0000 7f6931227740 1 client.26368 shutdown 2024-09-13T18:26:29.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.703+0000 7f6931227740 1 -- 172.21.15.114:0/1627879393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56312cbfc970 msgr2=0x56312cc0e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:29.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.703+0000 7f6931227740 1 --2- 172.21.15.114:0/1627879393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56312cbfc970 0x56312cc0e930 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6918005bc0 tx=0x7f6918010660 comp rx=0 tx=0).stop 2024-09-13T18:26:29.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.704+0000 7f6931227740 1 -- 172.21.15.114:0/1627879393 shutdown_connections 2024-09-13T18:26:29.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.704+0000 7f6931227740 1 --2- 172.21.15.114:0/1627879393 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x56312cc6f060 0x56312cc71450 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.704+0000 7f6931227740 1 --2- 172.21.15.114:0/1627879393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56312cc07b30 0x56312cc00600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.704+0000 7f6931227740 1 --2- 172.21.15.114:0/1627879393 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56312cbfd290 0x56312cc0ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.704+0000 7f6931227740 1 --2- 172.21.15.114:0/1627879393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56312cbfc970 0x56312cc0e930 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.704+0000 7f6931227740 1 -- 172.21.15.114:0/1627879393 >> 172.21.15.114:0/1627879393 conn(0x56312cbf8ee0 msgr2=0x56312cbf92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:29.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.704+0000 7f69077fe640 0 client.26368 ms_handle_reset on v2:172.21.15.134:6836/891692210 2024-09-13T18:26:29.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.704+0000 7f6931227740 1 -- 172.21.15.114:0/1627879393 shutdown_connections 2024-09-13T18:26:29.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.704+0000 7f6931227740 1 -- 172.21.15.114:0/1627879393 wait complete. 2024-09-13T18:26:29.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.704+0000 7f6931227740 20 client.26368 trim_cache size 0 max 16384 2024-09-13T18:26:29.708 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:26:29.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.705+0000 7f692d2a6640 1 -- 172.21.15.114:0/1133077298 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f69140a1340 msgr2=0x7f69140a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:29.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.705+0000 7f692d2a6640 1 --2- 172.21.15.114:0/1133077298 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f69140a1340 0x7f69140a3800 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6920031040 tx=0x7f6920005bd0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.706+0000 7f692d2a6640 1 -- 172.21.15.114:0/1133077298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6928107890 msgr2=0x7f69281a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:29.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.706+0000 7f692d2a6640 1 --2- 172.21.15.114:0/1133077298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6928107890 0x7f69281a5450 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f691000e790 tx=0x7f691000ec50 comp rx=0 tx=0).stop 2024-09-13T18:26:29.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.706+0000 7f692d2a6640 1 -- 172.21.15.114:0/1133077298 shutdown_connections 2024-09-13T18:26:29.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.706+0000 7f692d2a6640 1 --2- 172.21.15.114:0/1133077298 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f69140a1340 0x7f69140a3800 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.706+0000 7f692d2a6640 1 --2- 172.21.15.114:0/1133077298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6928119400 0x7f69281a9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.706+0000 7f692d2a6640 1 --2- 172.21.15.114:0/1133077298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6928107890 0x7f69281a5450 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.706+0000 7f692d2a6640 1 --2- 172.21.15.114:0/1133077298 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69281074b0 0x7f69281a4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.706+0000 7f692d2a6640 1 -- 172.21.15.114:0/1133077298 >> 172.21.15.114:0/1133077298 conn(0x7f692806af40 msgr2=0x7f6928119df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:29.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.708+0000 7f692d2a6640 1 -- 172.21.15.114:0/1133077298 shutdown_connections 2024-09-13T18:26:29.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.708+0000 7f692d2a6640 1 -- 172.21.15.114:0/1133077298 wait complete. 2024-09-13T18:26:29.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:26:29 smithi114 ceph-mon[30897]: pgmap v123: 129 pgs: 129 active+clean; 14 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 44 MiB/s wr, 710 op/s 2024-09-13T18:26:29.733 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-09-13T18:26:29.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.945+0000 7fe296b5e640 1 -- 172.21.15.114:0/2596475907 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe290107890 msgr2=0x7fe290107cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:29.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.945+0000 7fe296b5e640 1 --2- 172.21.15.114:0/2596475907 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe290107890 0x7fe290107cd0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fe284009900 tx=0x7fe28402a3e0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.945+0000 7fe296b5e640 1 -- 172.21.15.114:0/2596475907 shutdown_connections 2024-09-13T18:26:29.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.945+0000 7fe296b5e640 1 --2- 172.21.15.114:0/2596475907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe290119400 0x7fe29011b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.945+0000 7fe296b5e640 1 --2- 172.21.15.114:0/2596475907 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe290107890 0x7fe290107cd0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.945+0000 7fe296b5e640 1 --2- 172.21.15.114:0/2596475907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2901074b0 0x7fe29010e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.946+0000 7fe296b5e640 1 -- 172.21.15.114:0/2596475907 >> 172.21.15.114:0/2596475907 conn(0x7fe29006af70 msgr2=0x7fe29006b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:29.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.946+0000 7fe296b5e640 1 -- 172.21.15.114:0/2596475907 shutdown_connections 2024-09-13T18:26:29.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.946+0000 7fe296b5e640 1 -- 172.21.15.114:0/2596475907 wait complete. 2024-09-13T18:26:29.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.947+0000 7fe296b5e640 1 Processor -- start 2024-09-13T18:26:29.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.947+0000 7fe296b5e640 1 -- start start 2024-09-13T18:26:29.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.947+0000 7fe296b5e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2901074b0 0x7fe2901a4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.947+0000 7fe296b5e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe290107890 0x7fe2901a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.948+0000 7fe296b5e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe290119400 0x7fe2901a9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.948+0000 7fe296b5e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2901a9e30 con 0x7fe290119400 2024-09-13T18:26:29.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.948+0000 7fe296b5e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2901a9f70 con 0x7fe290107890 2024-09-13T18:26:29.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.948+0000 7fe296b5e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2901aa130 con 0x7fe2901074b0 2024-09-13T18:26:29.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.948+0000 7fe2948fa640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2901074b0 0x7fe2901a4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.948+0000 7fe2950fb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe290119400 0x7fe2901a9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.948+0000 7fe2948fa640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2901074b0 0x7fe2901a4f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:52768/0 (socket says 172.21.15.114:52768) 2024-09-13T18:26:29.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.948+0000 7fe2948fa640 1 -- 172.21.15.114:0/4275938174 learned_addr learned my addr 172.21.15.114:0/4275938174 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:29.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.949+0000 7fe2950fb640 1 -- 172.21.15.114:0/4275938174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2901074b0 msgr2=0x7fe2901a4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:29.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.949+0000 7fe2950fb640 1 --2- 172.21.15.114:0/4275938174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2901074b0 0x7fe2901a4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.949+0000 7fe2950fb640 1 -- 172.21.15.114:0/4275938174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe290107890 msgr2=0x7fe2901a5440 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:26:29.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.949+0000 7fe2950fb640 1 --2- 172.21.15.114:0/4275938174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe290107890 0x7fe2901a5440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.949+0000 7fe2950fb640 1 -- 172.21.15.114:0/4275938174 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe284009560 con 0x7fe290119400 2024-09-13T18:26:29.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.949+0000 7fe2950fb640 1 --2- 172.21.15.114:0/4275938174 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe290119400 0x7fe2901a9830 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fe28000d680 tx=0x7fe28000db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:29.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.949+0000 7fe2948fa640 1 --2- 172.21.15.114:0/4275938174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2901074b0 0x7fe2901a4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:26:29.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.949+0000 7fe28dffb640 1 -- 172.21.15.114:0/4275938174 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe280005220 con 0x7fe290119400 2024-09-13T18:26:29.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.950+0000 7fe296b5e640 1 -- 172.21.15.114:0/4275938174 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2901aa410 con 0x7fe290119400 2024-09-13T18:26:29.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.950+0000 7fe28dffb640 1 -- 172.21.15.114:0/4275938174 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe280009cb0 con 0x7fe290119400 2024-09-13T18:26:29.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.950+0000 7fe28dffb640 1 -- 172.21.15.114:0/4275938174 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe280004d60 con 0x7fe290119400 2024-09-13T18:26:29.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.950+0000 7fe296b5e640 1 -- 172.21.15.114:0/4275938174 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe29010edc0 con 0x7fe290119400 2024-09-13T18:26:29.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.962+0000 7fe299880740 1 Processor -- start 2024-09-13T18:26:29.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.969+0000 7fe299880740 1 -- start start 2024-09-13T18:26:29.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.969+0000 7fe299880740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564cceb0e970 0x564cceb0ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.969+0000 7fe299880740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cceb0f290 0x564cceb195f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.969+0000 7fe299880740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564cceb19b30 0x564cceb1bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.969+0000 7fe299880740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ccea78cc0 con 0x564cceb19b30 2024-09-13T18:26:29.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.969+0000 7fe299880740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ccea76450 con 0x564cceb0e970 2024-09-13T18:26:29.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.969+0000 7fe299880740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ccea71d40 con 0x564cceb0f290 2024-09-13T18:26:29.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.969+0000 7fe28dffb640 1 -- 172.21.15.114:0/4275938174 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fe280005380 con 0x7fe290119400 2024-09-13T18:26:29.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.969+0000 7fe28ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564cceb0e970 0x564cceb0ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.969+0000 7fe28ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564cceb0e970 0x564cceb0ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:60748/0 (socket says 172.21.15.114:60748) 2024-09-13T18:26:29.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.969+0000 7fe28ffff640 1 -- 172.21.15.114:0/4019343987 learned_addr learned my addr 172.21.15.114:0/4019343987 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:29.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.972+0000 7fe2948fa640 1 --2- 172.21.15.114:0/4019343987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cceb0f290 0x564cceb195f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.972+0000 7fe28ffff640 1 --2- 172.21.15.114:0/4019343987 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564cceb19b30 0x564cceb1bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.973+0000 7fe28ffff640 1 -- 172.21.15.114:0/4019343987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cceb0f290 msgr2=0x564cceb195f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:29.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.973+0000 7fe28ffff640 1 --2- 172.21.15.114:0/4019343987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cceb0f290 0x564cceb195f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.973+0000 7fe28ffff640 1 -- 172.21.15.114:0/4019343987 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564cceb19b30 msgr2=0x564cceb1bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:29.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.973+0000 7fe28ffff640 1 --2- 172.21.15.114:0/4019343987 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564cceb19b30 0x564cceb1bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.973+0000 7fe28ffff640 1 -- 172.21.15.114:0/4019343987 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ccea39620 con 0x564cceb0e970 2024-09-13T18:26:29.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.973+0000 7fe2948fa640 1 --2- 172.21.15.114:0/4019343987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cceb0f290 0x564cceb195f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:26:29.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.974+0000 7fe28dffb640 1 --2- 172.21.15.114:0/4275938174 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe2780a1410 0x7fe2780a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.974+0000 7fe28dffb640 1 -- 172.21.15.114:0/4275938174 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fe2800cc790 con 0x7fe290119400 2024-09-13T18:26:29.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.976+0000 7fe28ffff640 1 --2- 172.21.15.114:0/4019343987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564cceb0e970 0x564cceb0ed50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe284031460 tx=0x7fe28402a8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0f65b4db7f2d634 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:29.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.976+0000 7fe2948fa640 1 --2- 172.21.15.114:0/4275938174 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe2780a1410 0x7fe2780a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.977+0000 7fe2948fa640 1 --2- 172.21.15.114:0/4275938174 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe2780a1410 0x7fe2780a38d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x564cceb0bd40 tx=0x7fe288007700 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:29.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.978+0000 7fe276ffd640 1 -- 172.21.15.114:0/4019343987 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe284031de0 con 0x564cceb0e970 2024-09-13T18:26:29.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.978+0000 7fe276ffd640 1 -- 172.21.15.114:0/4019343987 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe2840044c0 con 0x564cceb0e970 2024-09-13T18:26:29.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.979+0000 7fe299880740 1 -- 172.21.15.114:0/4019343987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564cceb0e970 msgr2=0x564cceb0ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:29.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.979+0000 7fe299880740 1 --2- 172.21.15.114:0/4019343987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564cceb0e970 0x564cceb0ed50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe284031460 tx=0x7fe28402a8e0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.980+0000 7fe299880740 1 -- 172.21.15.114:0/4019343987 shutdown_connections 2024-09-13T18:26:29.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.980+0000 7fe299880740 1 --2- 172.21.15.114:0/4019343987 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564cceb19b30 0x564cceb1bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.980+0000 7fe299880740 1 --2- 172.21.15.114:0/4019343987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cceb0f290 0x564cceb195f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.980+0000 7fe299880740 1 --2- 172.21.15.114:0/4019343987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564cceb0e970 0x564cceb0ed50 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.980+0000 7fe299880740 1 -- 172.21.15.114:0/4019343987 >> 172.21.15.114:0/4019343987 conn(0x564cceb0aee0 msgr2=0x564cceb0b2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:29.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.981+0000 7fe299880740 1 -- 172.21.15.114:0/4019343987 shutdown_connections 2024-09-13T18:26:29.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.981+0000 7fe299880740 1 -- 172.21.15.114:0/4019343987 wait complete. 2024-09-13T18:26:29.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.981+0000 7fe299880740 1 -- start start 2024-09-13T18:26:29.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.981+0000 7fe299880740 1 Processor -- start 2024-09-13T18:26:29.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.982+0000 7fe299880740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564cceb0e970 0x564cceb1edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.982+0000 7fe299880740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564cceb0f290 0x564cceb1f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.982+0000 7fe299880740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cceb19b30 0x564cceb163a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.982+0000 7fe299880740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ccea30180 con 0x564cceb0f290 2024-09-13T18:26:29.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.982+0000 7fe299880740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cce74a430 con 0x564cceb0e970 2024-09-13T18:26:29.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.982+0000 7fe299880740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cce9f55e0 con 0x564cceb19b30 2024-09-13T18:26:29.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.982+0000 7fe299880740 10 client.0 _ll_register_callbacks cb 0x564ccea61aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:26:29.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.982+0000 7fe2950fb640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cceb19b30 0x564cceb163a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.982+0000 7fe2950fb640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cceb19b30 0x564cceb163a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:52784/0 (socket says 172.21.15.114:52784) 2024-09-13T18:26:29.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.982+0000 7fe2950fb640 1 -- 172.21.15.114:0/2095352646 learned_addr learned my addr 172.21.15.114:0/2095352646 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:29.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.982+0000 7fe2948fa640 1 --2- 172.21.15.114:0/2095352646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564cceb0e970 0x564cceb1edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.983+0000 7fe2948fa640 1 -- 172.21.15.114:0/2095352646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cceb19b30 msgr2=0x564cceb163a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:29.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.983+0000 7fe2948fa640 1 --2- 172.21.15.114:0/2095352646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cceb19b30 0x564cceb163a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.983+0000 7fe2948fa640 1 -- 172.21.15.114:0/2095352646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564cceb0f290 msgr2=0x564cceb1f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:29.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.983+0000 7fe2948fa640 1 --2- 172.21.15.114:0/2095352646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564cceb0f290 0x564cceb1f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.983+0000 7fe2948fa640 1 -- 172.21.15.114:0/2095352646 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe29011c1a0 con 0x564cceb0e970 2024-09-13T18:26:29.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.983+0000 7fe2948fa640 1 --2- 172.21.15.114:0/2095352646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564cceb0e970 0x564cceb1edd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe288010d40 tx=0x7fe288004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7876aa6c8b848cac server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:29.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.983+0000 7fe26ffff640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:26:29.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.983+0000 7fe26ffff640 1 -- 172.21.15.114:0/2095352646 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe288003d80 con 0x564cceb0e970 2024-09-13T18:26:29.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.984+0000 7fe26ffff640 1 -- 172.21.15.114:0/2095352646 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fe288017070 con 0x564cceb0e970 2024-09-13T18:26:29.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.984+0000 7fe299880740 1 -- 172.21.15.114:0/2095352646 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564ccea9bb80 con 0x564cceb0e970 2024-09-13T18:26:29.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.984+0000 7fe26ffff640 1 -- 172.21.15.114:0/2095352646 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe28800eed0 con 0x564cceb0e970 2024-09-13T18:26:29.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.985+0000 7fe299880740 10 client.26360 fetch_fsmap learned FSMap version 32 2024-09-13T18:26:29.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.985+0000 7fe299880740 1 -- 172.21.15.114:0/2095352646 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x564ccea703c0 con 0x564cceb0e970 2024-09-13T18:26:29.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.986+0000 7fe26ffff640 1 -- 172.21.15.114:0/2095352646 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564ccea9bb80 con 0x564cceb0e970 2024-09-13T18:26:29.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.986+0000 7fe26ffff640 1 -- 172.21.15.114:0/2095352646 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fe28801e790 con 0x564cceb0e970 2024-09-13T18:26:29.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.986+0000 7fe299880740 10 client.26360 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:26:29.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.986+0000 7fe299880740 10 client.26360 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:26:29.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.986+0000 7fe299880740 20 client.26360 populate_metadata read hostname 'smithi114' 2024-09-13T18:26:29.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.986+0000 7fe299880740 1 --2- 172.21.15.114:0/2095352646 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x564cceb29620 0x564cceb2ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:29.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.986+0000 7fe299880740 4 client.26360 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:26:29.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.986+0000 7fe299880740 1 -- 172.21.15.114:0/2095352646 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564ccea641f0 con 0x564cceb29620 2024-09-13T18:26:29.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.986+0000 7fe28ffff640 1 --2- 172.21.15.114:0/2095352646 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x564cceb29620 0x564cceb2ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:29.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.987+0000 7fe26ffff640 10 client.26360 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:26:29.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.988+0000 7fe28ffff640 1 --2- 172.21.15.114:0/2095352646 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x564cceb29620 0x564cceb2ba40 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6496b771b371cabb server_cookie=8a0de8d2862b5d80 in_seq=0 out_seq=0 2024-09-13T18:26:29.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.989+0000 7fe26ffff640 1 -- 172.21.15.114:0/2095352646 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x564ccea641f0 con 0x564cceb29620 2024-09-13T18:26:29.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.989+0000 7fe26ffff640 10 client.26360 handle_command_reply: tid=0 2024-09-13T18:26:29.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.989+0000 7fe299880740 1 client.26360 shutdown 2024-09-13T18:26:29.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.990+0000 7fe299880740 1 -- 172.21.15.114:0/2095352646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564cceb0e970 msgr2=0x564cceb1edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:29.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.990+0000 7fe299880740 1 --2- 172.21.15.114:0/2095352646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564cceb0e970 0x564cceb1edd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe288010d40 tx=0x7fe288004a40 comp rx=0 tx=0).stop 2024-09-13T18:26:29.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.990+0000 7fe299880740 1 -- 172.21.15.114:0/2095352646 shutdown_connections 2024-09-13T18:26:29.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.990+0000 7fe299880740 1 --2- 172.21.15.114:0/2095352646 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x564cceb29620 0x564cceb2ba40 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.990+0000 7fe299880740 1 --2- 172.21.15.114:0/2095352646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cceb19b30 0x564cceb163a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.990+0000 7fe299880740 1 --2- 172.21.15.114:0/2095352646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564cceb0f290 0x564cceb1f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:29.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.990+0000 7fe299880740 1 --2- 172.21.15.114:0/2095352646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564cceb0e970 0x564cceb1edd0 secure :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe288010d40 tx=0x7fe288004a40 comp rx=0 tx=0).stop 2024-09-13T18:26:29.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.990+0000 7fe299880740 1 -- 172.21.15.114:0/2095352646 >> 172.21.15.114:0/2095352646 conn(0x564cceb0aee0 msgr2=0x564cceb0a230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:29.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.990+0000 7fe26ffff640 0 client.26360 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:26:29.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.992+0000 7fe299880740 1 -- 172.21.15.114:0/2095352646 shutdown_connections 2024-09-13T18:26:29.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.993+0000 7fe299880740 1 -- 172.21.15.114:0/2095352646 wait complete. 2024-09-13T18:26:29.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:29.995+0000 7fe299880740 20 client.26360 trim_cache size 0 max 16384 2024-09-13T18:26:30.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.020+0000 7fe299880740 1 Processor -- start 2024-09-13T18:26:30.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.021+0000 7fe299880740 1 -- start start 2024-09-13T18:26:30.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.021+0000 7fe299880740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cceb0f290 0x564cceb16350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:30.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.021+0000 7fe299880740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564cceb19b30 0x564cceb1af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:30.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.021+0000 7fe2948fa640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cceb0f290 0x564cceb16350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:30.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.022+0000 7fe2948fa640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cceb0f290 0x564cceb16350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:52786/0 (socket says 172.21.15.114:52786) 2024-09-13T18:26:30.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.022+0000 7fe2948fa640 1 -- 172.21.15.114:0/3611712644 learned_addr learned my addr 172.21.15.114:0/3611712644 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:30.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.022+0000 7fe299880740 1 --2- 172.21.15.114:0/3611712644 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564cceb0aa20 0x564cceb1b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:30.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.022+0000 7fe28ffff640 1 --2- 172.21.15.114:0/3611712644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564cceb19b30 0x564cceb1af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:30.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.022+0000 7fe299880740 1 -- 172.21.15.114:0/3611712644 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cceb169a0 con 0x564cceb0aa20 2024-09-13T18:26:30.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.022+0000 7fe299880740 1 -- 172.21.15.114:0/3611712644 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cceb0f930 con 0x564cceb19b30 2024-09-13T18:26:30.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.022+0000 7fe299880740 1 -- 172.21.15.114:0/3611712644 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cceb0e7a0 con 0x564cceb0f290 2024-09-13T18:26:30.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.022+0000 7fe2950fb640 1 --2- 172.21.15.114:0/3611712644 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564cceb0aa20 0x564cceb1b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:30.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.023+0000 7fe28ffff640 1 -- 172.21.15.114:0/3611712644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cceb0f290 msgr2=0x564cceb16350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:30.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.023+0000 7fe28ffff640 1 --2- 172.21.15.114:0/3611712644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cceb0f290 0x564cceb16350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:30.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.023+0000 7fe28ffff640 1 -- 172.21.15.114:0/3611712644 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564cceb0aa20 msgr2=0x564cceb1b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:30.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.023+0000 7fe28ffff640 1 --2- 172.21.15.114:0/3611712644 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564cceb0aa20 0x564cceb1b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:30.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.023+0000 7fe28ffff640 1 -- 172.21.15.114:0/3611712644 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ccea12770 con 0x564cceb19b30 2024-09-13T18:26:30.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.023+0000 7fe28ffff640 1 --2- 172.21.15.114:0/3611712644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564cceb19b30 0x564cceb1af20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe284040040 tx=0x7fe28402a3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac8800e977ca7e29 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:30.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.024+0000 7fe2777fe640 1 -- 172.21.15.114:0/3611712644 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2840037f0 con 0x564cceb19b30 2024-09-13T18:26:30.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.024+0000 7fe2777fe640 1 -- 172.21.15.114:0/3611712644 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe284042040 con 0x564cceb19b30 2024-09-13T18:26:30.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.024+0000 7fe2777fe640 1 -- 172.21.15.114:0/3611712644 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28404f3c0 con 0x564cceb19b30 2024-09-13T18:26:30.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.025+0000 7fe299880740 1 -- 172.21.15.114:0/3611712644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564cceb19b30 msgr2=0x564cceb1af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:30.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.025+0000 7fe299880740 1 --2- 172.21.15.114:0/3611712644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564cceb19b30 0x564cceb1af20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe284040040 tx=0x7fe28402a3e0 comp rx=0 tx=0).stop 2024-09-13T18:26:30.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.025+0000 7fe299880740 1 -- 172.21.15.114:0/3611712644 shutdown_connections 2024-09-13T18:26:30.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.026+0000 7fe299880740 1 --2- 172.21.15.114:0/3611712644 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564cceb0aa20 0x564cceb1b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:30.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.026+0000 7fe299880740 1 --2- 172.21.15.114:0/3611712644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564cceb19b30 0x564cceb1af20 secure :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe284040040 tx=0x7fe28402a3e0 comp rx=0 tx=0).stop 2024-09-13T18:26:30.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.026+0000 7fe299880740 1 --2- 172.21.15.114:0/3611712644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cceb0f290 0x564cceb16350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:30.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.026+0000 7fe299880740 1 -- 172.21.15.114:0/3611712644 >> 172.21.15.114:0/3611712644 conn(0x564cceb0aee0 msgr2=0x564cceb15c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:30.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.026+0000 7fe299880740 1 -- 172.21.15.114:0/3611712644 shutdown_connections 2024-09-13T18:26:30.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.026+0000 7fe299880740 1 -- 172.21.15.114:0/3611712644 wait complete. 2024-09-13T18:26:30.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.027+0000 7fe299880740 1 -- start start 2024-09-13T18:26:30.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.027+0000 7fe299880740 1 Processor -- start 2024-09-13T18:26:30.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.028+0000 7fe299880740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564cceb0aa20 0x564cceb21f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:30.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.028+0000 7fe299880740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564cceb0f290 0x564cceb224a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:30.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.028+0000 7fe299880740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cceb229e0 0x564cceb7fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:30.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.028+0000 7fe299880740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cceb25e10 con 0x564cceb0f290 2024-09-13T18:26:30.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.028+0000 7fe299880740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cceb800b0 con 0x564cceb0aa20 2024-09-13T18:26:30.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.028+0000 7fe299880740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cceb80270 con 0x564cceb229e0 2024-09-13T18:26:30.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.028+0000 7fe299880740 10 client.0 _ll_register_callbacks cb 0x7fe284009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:26:30.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.028+0000 7fe2948fa640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564cceb0aa20 0x564cceb21f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:30.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.028+0000 7fe2948fa640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564cceb0aa20 0x564cceb21f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:60758/0 (socket says 172.21.15.114:60758) 2024-09-13T18:26:30.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.028+0000 7fe2948fa640 1 -- 172.21.15.114:0/2148885451 learned_addr learned my addr 172.21.15.114:0/2148885451 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:30.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.028+0000 7fe2950fb640 1 --2- 172.21.15.114:0/2148885451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cceb229e0 0x564cceb7fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:30.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.029+0000 7fe28ffff640 1 --2- 172.21.15.114:0/2148885451 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564cceb0f290 0x564cceb224a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:30.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.029+0000 7fe2948fa640 1 -- 172.21.15.114:0/2148885451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cceb229e0 msgr2=0x564cceb7fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:30.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.029+0000 7fe2948fa640 1 --2- 172.21.15.114:0/2148885451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cceb229e0 0x564cceb7fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:30.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.029+0000 7fe2948fa640 1 -- 172.21.15.114:0/2148885451 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564cceb0f290 msgr2=0x564cceb224a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:30.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.029+0000 7fe2948fa640 1 --2- 172.21.15.114:0/2148885451 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564cceb0f290 0x564cceb224a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:30.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.029+0000 7fe2948fa640 1 -- 172.21.15.114:0/2148885451 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe28404fe70 con 0x564cceb0aa20 2024-09-13T18:26:30.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.030+0000 7fe2948fa640 1 --2- 172.21.15.114:0/2148885451 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564cceb0aa20 0x564cceb21f60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe288010de0 tx=0x7fe28801e650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=338a9450cff50a95 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:30.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.030+0000 7fe26ffff640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:26:30.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.031+0000 7fe26ffff640 1 -- 172.21.15.114:0/2148885451 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe288037070 con 0x564cceb0aa20 2024-09-13T18:26:30.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.032+0000 7fe26ffff640 1 -- 172.21.15.114:0/2148885451 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fe288017040 con 0x564cceb0aa20 2024-09-13T18:26:30.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.032+0000 7fe299880740 1 -- 172.21.15.114:0/2148885451 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe2840044c0 con 0x564cceb0aa20 2024-09-13T18:26:30.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.032+0000 7fe26ffff640 1 -- 172.21.15.114:0/2148885451 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe28800eed0 con 0x564cceb0aa20 2024-09-13T18:26:30.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.032+0000 7fe299880740 10 client.26366 fetch_fsmap learned FSMap version 32 2024-09-13T18:26:30.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.032+0000 7fe299880740 1 -- 172.21.15.114:0/2148885451 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x564ccea37e80 con 0x564cceb0aa20 2024-09-13T18:26:30.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.033+0000 7fe26ffff640 1 -- 172.21.15.114:0/2148885451 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe288015070 con 0x564cceb0aa20 2024-09-13T18:26:30.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.033+0000 7fe26ffff640 1 -- 172.21.15.114:0/2148885451 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fe2880051d0 con 0x564cceb0aa20 2024-09-13T18:26:30.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.034+0000 7fe299880740 10 client.26366 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:26:30.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.035+0000 7fe299880740 10 client.26366 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:26:30.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.035+0000 7fe299880740 20 client.26366 populate_metadata read hostname 'smithi114' 2024-09-13T18:26:30.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.035+0000 7fe299880740 1 --2- 172.21.15.114:0/2148885451 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x564cceb81a00 0x564cceb83e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:30.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.035+0000 7fe299880740 4 client.26366 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:26:30.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.035+0000 7fe299880740 1 -- 172.21.15.114:0/2148885451 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fe2840044c0 con 0x564cceb81a00 2024-09-13T18:26:30.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.035+0000 7fe28ffff640 1 --2- 172.21.15.114:0/2148885451 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x564cceb81a00 0x564cceb83e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:30.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.036+0000 7fe28ffff640 1 --2- 172.21.15.114:0/2148885451 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x564cceb81a00 0x564cceb83e20 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc0c6f6a789f267d server_cookie=f784ae4f7190e182 in_seq=0 out_seq=0 2024-09-13T18:26:30.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.036+0000 7fe26ffff640 10 client.26366 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:26:30.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.037+0000 7fe26ffff640 1 -- 172.21.15.114:0/2148885451 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fe2840044c0 con 0x564cceb81a00 2024-09-13T18:26:30.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.037+0000 7fe26ffff640 10 client.26366 handle_command_reply: tid=0 2024-09-13T18:26:30.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.037+0000 7fe299880740 1 client.26366 shutdown 2024-09-13T18:26:30.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.037+0000 7fe299880740 1 -- 172.21.15.114:0/2148885451 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564cceb0aa20 msgr2=0x564cceb21f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:30.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.037+0000 7fe299880740 1 --2- 172.21.15.114:0/2148885451 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564cceb0aa20 0x564cceb21f60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe288010de0 tx=0x7fe28801e650 comp rx=0 tx=0).stop 2024-09-13T18:26:30.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.041+0000 7fe299880740 1 -- 172.21.15.114:0/2148885451 shutdown_connections 2024-09-13T18:26:30.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.041+0000 7fe299880740 1 --2- 172.21.15.114:0/2148885451 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x564cceb81a00 0x564cceb83e20 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:30.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.041+0000 7fe299880740 1 --2- 172.21.15.114:0/2148885451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cceb229e0 0x564cceb7fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:30.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.041+0000 7fe26ffff640 0 client.26366 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:26:30.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.041+0000 7fe299880740 1 --2- 172.21.15.114:0/2148885451 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564cceb0f290 0x564cceb224a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:30.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.041+0000 7fe299880740 1 --2- 172.21.15.114:0/2148885451 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564cceb0aa20 0x564cceb21f60 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:30.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.041+0000 7fe299880740 1 -- 172.21.15.114:0/2148885451 >> 172.21.15.114:0/2148885451 conn(0x564cceb0aee0 msgr2=0x564cceb1e910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:30.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.041+0000 7fe299880740 1 -- 172.21.15.114:0/2148885451 shutdown_connections 2024-09-13T18:26:30.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.041+0000 7fe299880740 1 -- 172.21.15.114:0/2148885451 wait complete. 2024-09-13T18:26:30.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.043+0000 7fe299880740 20 client.26366 trim_cache size 0 max 16384 2024-09-13T18:26:30.046 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:26:30.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.051+0000 7fe296b5e640 1 -- 172.21.15.114:0/4275938174 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe2780a1410 msgr2=0x7fe2780a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:30.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.051+0000 7fe296b5e640 1 --2- 172.21.15.114:0/4275938174 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe2780a1410 0x7fe2780a38d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x564cceb0bd40 tx=0x7fe288007700 comp rx=0 tx=0).stop 2024-09-13T18:26:30.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.051+0000 7fe296b5e640 1 -- 172.21.15.114:0/4275938174 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe290119400 msgr2=0x7fe2901a9830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:30.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.051+0000 7fe296b5e640 1 --2- 172.21.15.114:0/4275938174 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe290119400 0x7fe2901a9830 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fe28000d680 tx=0x7fe28000db40 comp rx=0 tx=0).stop 2024-09-13T18:26:30.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.051+0000 7fe296b5e640 1 -- 172.21.15.114:0/4275938174 shutdown_connections 2024-09-13T18:26:30.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.051+0000 7fe296b5e640 1 --2- 172.21.15.114:0/4275938174 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe2780a1410 0x7fe2780a38d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:30.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.051+0000 7fe296b5e640 1 --2- 172.21.15.114:0/4275938174 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe290119400 0x7fe2901a9830 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:30.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.051+0000 7fe296b5e640 1 --2- 172.21.15.114:0/4275938174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe290107890 0x7fe2901a5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:30.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.051+0000 7fe296b5e640 1 --2- 172.21.15.114:0/4275938174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2901074b0 0x7fe2901a4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:30.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.051+0000 7fe296b5e640 1 -- 172.21.15.114:0/4275938174 >> 172.21.15.114:0/4275938174 conn(0x7fe29006af70 msgr2=0x7fe290119a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:30.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.058+0000 7fe296b5e640 1 -- 172.21.15.114:0/4275938174 shutdown_connections 2024-09-13T18:26:30.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:30.061+0000 7fe296b5e640 1 -- 172.21.15.114:0/4275938174 wait complete. 2024-09-13T18:26:31.092 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-13T18:26:31.093 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-09-13T18:26:31.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.237+0000 7fc808d84640 1 -- 172.21.15.114:0/2137022629 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8040fee60 msgr2=0x7fc8040ff240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:31.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.237+0000 7fc808d84640 1 --2- 172.21.15.114:0/2137022629 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8040fee60 0x7fc8040ff240 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec009930 tx=0x7fc7ec02a420 comp rx=0 tx=0).stop 2024-09-13T18:26:31.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.238+0000 7fc808d84640 1 -- 172.21.15.114:0/2137022629 shutdown_connections 2024-09-13T18:26:31.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.238+0000 7fc808d84640 1 --2- 172.21.15.114:0/2137022629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc804103fb0 0x7fc80410bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.238+0000 7fc808d84640 1 --2- 172.21.15.114:0/2137022629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8040ff810 0x7fc804103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.238+0000 7fc808d84640 1 --2- 172.21.15.114:0/2137022629 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8040fee60 0x7fc8040ff240 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.238+0000 7fc808d84640 1 -- 172.21.15.114:0/2137022629 >> 172.21.15.114:0/2137022629 conn(0x7fc804075c60 msgr2=0x7fc8040fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:31.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.238+0000 7fc808d84640 1 -- 172.21.15.114:0/2137022629 shutdown_connections 2024-09-13T18:26:31.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.238+0000 7fc808d84640 1 -- 172.21.15.114:0/2137022629 wait complete. 2024-09-13T18:26:31.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.238+0000 7fc808d84640 1 Processor -- start 2024-09-13T18:26:31.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.238+0000 7fc808d84640 1 -- start start 2024-09-13T18:26:31.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.239+0000 7fc808d84640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8040fee60 0x7fc804196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:31.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.239+0000 7fc808d84640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc804103fb0 0x7fc8041967b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:31.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.239+0000 7fc808d84640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc80419aba0 0x7fc80419b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:31.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.239+0000 7fc808d84640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc80419b590 con 0x7fc804103fb0 2024-09-13T18:26:31.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.239+0000 7fc808d84640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc80419b700 con 0x7fc8040fee60 2024-09-13T18:26:31.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.239+0000 7fc808d84640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc80419b870 con 0x7fc80419aba0 2024-09-13T18:26:31.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.239+0000 7fc801d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc804103fb0 0x7fc8041967b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:31.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.239+0000 7fc801d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc804103fb0 0x7fc8041967b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49588/0 (socket says 172.21.15.114:49588) 2024-09-13T18:26:31.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.239+0000 7fc801d9b640 1 -- 172.21.15.114:0/1839976865 learned_addr learned my addr 172.21.15.114:0/1839976865 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:31.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.239+0000 7fc802d9d640 1 --2- 172.21.15.114:0/1839976865 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc80419aba0 0x7fc80419b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:31.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.239+0000 7fc80259c640 1 --2- 172.21.15.114:0/1839976865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8040fee60 0x7fc804196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:31.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.239+0000 7fc801d9b640 1 -- 172.21.15.114:0/1839976865 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc80419aba0 msgr2=0x7fc80419b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:31.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.239+0000 7fc801d9b640 1 --2- 172.21.15.114:0/1839976865 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc80419aba0 0x7fc80419b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.239+0000 7fc801d9b640 1 -- 172.21.15.114:0/1839976865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8040fee60 msgr2=0x7fc804196270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:31.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.239+0000 7fc801d9b640 1 --2- 172.21.15.114:0/1839976865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8040fee60 0x7fc804196270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.239+0000 7fc801d9b640 1 -- 172.21.15.114:0/1839976865 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7ec009560 con 0x7fc804103fb0 2024-09-13T18:26:31.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.239+0000 7fc801d9b640 1 --2- 172.21.15.114:0/1839976865 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc804103fb0 0x7fc8041967b0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fc7f400c970 tx=0x7fc7f400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:31.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.239+0000 7fc7fb7fe640 1 -- 172.21.15.114:0/1839976865 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f4007bb0 con 0x7fc804103fb0 2024-09-13T18:26:31.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.239+0000 7fc7fb7fe640 1 -- 172.21.15.114:0/1839976865 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc7f4007d10 con 0x7fc804103fb0 2024-09-13T18:26:31.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.239+0000 7fc808d84640 1 -- 172.21.15.114:0/1839976865 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8041a2e50 con 0x7fc804103fb0 2024-09-13T18:26:31.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.239+0000 7fc7fb7fe640 1 -- 172.21.15.114:0/1839976865 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f4004d50 con 0x7fc804103fb0 2024-09-13T18:26:31.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.239+0000 7fc808d84640 1 -- 172.21.15.114:0/1839976865 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8041a3270 con 0x7fc804103fb0 2024-09-13T18:26:31.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.242+0000 7fc7fb7fe640 1 -- 172.21.15.114:0/1839976865 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fc7f4004fb0 con 0x7fc804103fb0 2024-09-13T18:26:31.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.242+0000 7fc80baa4740 1 Processor -- start 2024-09-13T18:26:31.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.242+0000 7fc80baa4740 1 -- start start 2024-09-13T18:26:31.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.242+0000 7fc80baa4740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55de7bf217c0 0x55de7bf21ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:31.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.242+0000 7fc80baa4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de7bf220e0 0x55de7bf2c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:31.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.242+0000 7fc80baa4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de7bf2c980 0x55de7bf2ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:31.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.242+0000 7fc7fb7fe640 1 --2- 172.21.15.114:0/1839976865 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc7f00a1200 0x7fc7f00a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:31.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.242+0000 7fc80baa4740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de7be8cc60 con 0x55de7bf217c0 2024-09-13T18:26:31.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.242+0000 7fc80baa4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de7be8a3f0 con 0x55de7bf220e0 2024-09-13T18:26:31.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.242+0000 7fc80baa4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de7be85ce0 con 0x55de7bf2c980 2024-09-13T18:26:31.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.243+0000 7fc802d9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55de7bf217c0 0x55de7bf21ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:31.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.243+0000 7fc802d9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55de7bf217c0 0x55de7bf21ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49604/0 (socket says 172.21.15.114:49604) 2024-09-13T18:26:31.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.243+0000 7fc802d9d640 1 -- 172.21.15.114:0/602849040 learned_addr learned my addr 172.21.15.114:0/602849040 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:31.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.243+0000 7fc801d9b640 1 --2- 172.21.15.114:0/602849040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de7bf2c980 0x55de7bf2ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:31.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.243+0000 7fc802d9d640 1 --2- 172.21.15.114:0/1839976865 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc7f00a1200 0x7fc7f00a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:31.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.243+0000 7fc7fb7fe640 1 -- 172.21.15.114:0/1839976865 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fc7f40ccc80 con 0x7fc804103fb0 2024-09-13T18:26:31.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.243+0000 7fc80259c640 1 --2- 172.21.15.114:0/602849040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de7bf220e0 0x55de7bf2c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:31.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.243+0000 7fc802d9d640 1 -- 172.21.15.114:0/602849040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de7bf2c980 msgr2=0x55de7bf2ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:31.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.243+0000 7fc802d9d640 1 --2- 172.21.15.114:0/602849040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de7bf2c980 0x55de7bf2ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.243+0000 7fc802d9d640 1 -- 172.21.15.114:0/602849040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de7bf220e0 msgr2=0x55de7bf2c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:31.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.243+0000 7fc802d9d640 1 --2- 172.21.15.114:0/602849040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de7bf220e0 0x55de7bf2c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.243+0000 7fc802d9d640 1 -- 172.21.15.114:0/602849040 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de7be41970 con 0x55de7bf217c0 2024-09-13T18:26:31.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.243+0000 7fc801d9b640 1 --2- 172.21.15.114:0/602849040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de7bf2c980 0x55de7bf2ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:26:31.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.243+0000 7fc802d9d640 1 --2- 172.21.15.114:0/602849040 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55de7bf217c0 0x55de7bf21ba0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc012cf0 tx=0x7fc7fc010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b09f314886bad2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:31.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.243+0000 7fc802d9d640 1 --2- 172.21.15.114:0/1839976865 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc7f00a1200 0x7fc7f00a36c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc010c40 tx=0x7fc7fc00c000 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:31.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.243+0000 7fc7f8ff9640 1 -- 172.21.15.114:0/602849040 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc012de0 con 0x55de7bf217c0 2024-09-13T18:26:31.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.244+0000 7fc7f8ff9640 1 -- 172.21.15.114:0/602849040 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc7fc009d10 con 0x55de7bf217c0 2024-09-13T18:26:31.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.244+0000 7fc7f8ff9640 1 -- 172.21.15.114:0/602849040 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc0122b0 con 0x55de7bf217c0 2024-09-13T18:26:31.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.244+0000 7fc80baa4740 1 -- 172.21.15.114:0/602849040 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55de7bf217c0 msgr2=0x55de7bf21ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:31.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.244+0000 7fc80baa4740 1 --2- 172.21.15.114:0/602849040 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55de7bf217c0 0x55de7bf21ba0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc012cf0 tx=0x7fc7fc010740 comp rx=0 tx=0).stop 2024-09-13T18:26:31.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.244+0000 7fc80baa4740 1 -- 172.21.15.114:0/602849040 shutdown_connections 2024-09-13T18:26:31.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.244+0000 7fc80baa4740 1 --2- 172.21.15.114:0/602849040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de7bf2c980 0x55de7bf2ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.244+0000 7fc80baa4740 1 --2- 172.21.15.114:0/602849040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de7bf220e0 0x55de7bf2c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.244+0000 7fc80baa4740 1 --2- 172.21.15.114:0/602849040 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55de7bf217c0 0x55de7bf21ba0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.244+0000 7fc80baa4740 1 -- 172.21.15.114:0/602849040 >> 172.21.15.114:0/602849040 conn(0x55de7bf1dd50 msgr2=0x55de7bf1e130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:31.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.244+0000 7fc80baa4740 1 -- 172.21.15.114:0/602849040 shutdown_connections 2024-09-13T18:26:31.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.244+0000 7fc80baa4740 1 -- 172.21.15.114:0/602849040 wait complete. 2024-09-13T18:26:31.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.245+0000 7fc80baa4740 1 -- start start 2024-09-13T18:26:31.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.245+0000 7fc80baa4740 1 Processor -- start 2024-09-13T18:26:31.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.245+0000 7fc80baa4740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55de7bf217c0 0x55de7bf2c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:31.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.245+0000 7fc80baa4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de7bf220e0 0x55de7bf296d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:31.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.245+0000 7fc80baa4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de7bf2c980 0x55de7bf29e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:31.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.245+0000 7fc80baa4740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de7be43f80 con 0x55de7bf217c0 2024-09-13T18:26:31.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.245+0000 7fc80baa4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de7be43c20 con 0x55de7bf2c980 2024-09-13T18:26:31.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.245+0000 7fc80baa4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de7be10ab0 con 0x55de7bf220e0 2024-09-13T18:26:31.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.245+0000 7fc80baa4740 10 client.0 _ll_register_callbacks cb 0x55de7be7f350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:26:31.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.245+0000 7fc80259c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55de7bf217c0 0x55de7bf2c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:31.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.245+0000 7fc80259c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55de7bf217c0 0x55de7bf2c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49612/0 (socket says 172.21.15.114:49612) 2024-09-13T18:26:31.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.245+0000 7fc80259c640 1 -- 172.21.15.114:0/1701897833 learned_addr learned my addr 172.21.15.114:0/1701897833 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:31.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.245+0000 7fc802d9d640 1 --2- 172.21.15.114:0/1701897833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de7bf2c980 0x55de7bf29e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:31.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.245+0000 7fc801d9b640 1 --2- 172.21.15.114:0/1701897833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de7bf220e0 0x55de7bf296d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:31.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.245+0000 7fc80259c640 1 -- 172.21.15.114:0/1701897833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de7bf220e0 msgr2=0x55de7bf296d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:31.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.245+0000 7fc80259c640 1 --2- 172.21.15.114:0/1701897833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de7bf220e0 0x55de7bf296d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.245+0000 7fc80259c640 1 -- 172.21.15.114:0/1701897833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de7bf2c980 msgr2=0x55de7bf29e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:31.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.245+0000 7fc80259c640 1 --2- 172.21.15.114:0/1701897833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de7bf2c980 0x55de7bf29e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.245+0000 7fc80259c640 1 -- 172.21.15.114:0/1701897833 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7fc01a040 con 0x55de7bf217c0 2024-09-13T18:26:31.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.245+0000 7fc80259c640 1 --2- 172.21.15.114:0/1701897833 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55de7bf217c0 0x55de7bf2c400 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec02aa10 tx=0x7fc7ec0039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcc55abc0e7e8568 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:31.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.245+0000 7fc7e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:26:31.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.246+0000 7fc7e9ffb640 1 -- 172.21.15.114:0/1701897833 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec003d60 con 0x55de7bf217c0 2024-09-13T18:26:31.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.246+0000 7fc7e9ffb640 1 -- 172.21.15.114:0/1701897833 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fc7ec046470 con 0x55de7bf217c0 2024-09-13T18:26:31.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.246+0000 7fc80baa4740 1 -- 172.21.15.114:0/1701897833 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55de7bb79590 con 0x55de7bf217c0 2024-09-13T18:26:31.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.246+0000 7fc7e9ffb640 1 -- 172.21.15.114:0/1701897833 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7ec046630 con 0x55de7bf217c0 2024-09-13T18:26:31.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.246+0000 7fc80baa4740 10 client.17025 fetch_fsmap learned FSMap version 32 2024-09-13T18:26:31.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.246+0000 7fc80baa4740 1 -- 172.21.15.114:0/1701897833 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55de7be84360 con 0x55de7bf217c0 2024-09-13T18:26:31.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.246+0000 7fc7e9ffb640 1 -- 172.21.15.114:0/1701897833 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55de7bb79590 con 0x55de7bf217c0 2024-09-13T18:26:31.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.246+0000 7fc7e9ffb640 1 -- 172.21.15.114:0/1701897833 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fc7ec03d7b0 con 0x55de7bf217c0 2024-09-13T18:26:31.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.246+0000 7fc80baa4740 10 client.17025 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:26:31.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.246+0000 7fc80baa4740 10 client.17025 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:26:31.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.246+0000 7fc80baa4740 20 client.17025 populate_metadata read hostname 'smithi114' 2024-09-13T18:26:31.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.246+0000 7fc80baa4740 1 --2- 172.21.15.114:0/1701897833 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55de7bf3c830 0x55de7bf3ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:31.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.246+0000 7fc80baa4740 4 client.17025 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:26:31.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.246+0000 7fc80baa4740 1 -- 172.21.15.114:0/1701897833 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55de7bb8a860 con 0x55de7bf3c830 2024-09-13T18:26:31.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.247+0000 7fc801d9b640 1 --2- 172.21.15.114:0/1701897833 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55de7bf3c830 0x55de7bf3ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:31.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.247+0000 7fc7e9ffb640 10 client.17025 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:26:31.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.247+0000 7fc801d9b640 1 --2- 172.21.15.114:0/1701897833 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55de7bf3c830 0x55de7bf3ec50 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba295051f40df9d9 server_cookie=22ea6de80e0df1 in_seq=0 out_seq=0 2024-09-13T18:26:31.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.248+0000 7fc7e9ffb640 1 -- 172.21.15.114:0/1701897833 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55de7bb8a860 con 0x55de7bf3c830 2024-09-13T18:26:31.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.248+0000 7fc7e9ffb640 10 client.17025 handle_command_reply: tid=0 2024-09-13T18:26:31.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.248+0000 7fc80baa4740 1 client.17025 shutdown 2024-09-13T18:26:31.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.248+0000 7fc80baa4740 1 -- 172.21.15.114:0/1701897833 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55de7bf217c0 msgr2=0x55de7bf2c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:31.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.248+0000 7fc80baa4740 1 --2- 172.21.15.114:0/1701897833 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55de7bf217c0 0x55de7bf2c400 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec02aa10 tx=0x7fc7ec0039f0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.248+0000 7fc80baa4740 1 -- 172.21.15.114:0/1701897833 shutdown_connections 2024-09-13T18:26:31.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.248+0000 7fc80baa4740 1 --2- 172.21.15.114:0/1701897833 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55de7bf3c830 0x55de7bf3ec50 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.248+0000 7fc80baa4740 1 --2- 172.21.15.114:0/1701897833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de7bf2c980 0x55de7bf29e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.248+0000 7fc80baa4740 1 --2- 172.21.15.114:0/1701897833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de7bf220e0 0x55de7bf296d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.248+0000 7fc80baa4740 1 --2- 172.21.15.114:0/1701897833 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55de7bf217c0 0x55de7bf2c400 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.248+0000 7fc7e9ffb640 0 client.17025 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:26:31.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.248+0000 7fc80baa4740 1 -- 172.21.15.114:0/1701897833 >> 172.21.15.114:0/1701897833 conn(0x55de7bf1dd50 msgr2=0x55de7bf231a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:31.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.248+0000 7fc80baa4740 1 -- 172.21.15.114:0/1701897833 shutdown_connections 2024-09-13T18:26:31.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.248+0000 7fc80baa4740 1 -- 172.21.15.114:0/1701897833 wait complete. 2024-09-13T18:26:31.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.248+0000 7fc80baa4740 20 client.17025 trim_cache size 0 max 16384 2024-09-13T18:26:31.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.262+0000 7fc80baa4740 1 Processor -- start 2024-09-13T18:26:31.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.262+0000 7fc80baa4740 1 -- start start 2024-09-13T18:26:31.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.262+0000 7fc80baa4740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55de7bf217c0 0x55de7bf33650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:31.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.262+0000 7fc80baa4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de7bf220e0 0x55de7bf33b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:31.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.262+0000 7fc80baa4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de7bf2c980 0x55de7bf27f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:31.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.262+0000 7fc80baa4740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de7bf20fd0 con 0x55de7bf217c0 2024-09-13T18:26:31.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.262+0000 7fc80baa4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de7bf27c00 con 0x55de7bf2c980 2024-09-13T18:26:31.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.262+0000 7fc80baa4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de7bf226b0 con 0x55de7bf220e0 2024-09-13T18:26:31.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.262+0000 7fc80259c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55de7bf217c0 0x55de7bf33650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:31.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.262+0000 7fc80259c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55de7bf217c0 0x55de7bf33650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49622/0 (socket says 172.21.15.114:49622) 2024-09-13T18:26:31.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.262+0000 7fc80259c640 1 -- 172.21.15.114:0/4012614746 learned_addr learned my addr 172.21.15.114:0/4012614746 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:31.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.262+0000 7fc801d9b640 1 --2- 172.21.15.114:0/4012614746 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de7bf220e0 0x55de7bf33b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:31.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.263+0000 7fc80259c640 1 -- 172.21.15.114:0/4012614746 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de7bf220e0 msgr2=0x55de7bf33b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:31.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.263+0000 7fc80259c640 1 --2- 172.21.15.114:0/4012614746 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de7bf220e0 0x55de7bf33b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.263+0000 7fc80259c640 1 -- 172.21.15.114:0/4012614746 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de7bf2c980 msgr2=0x55de7bf27f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:31.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.263+0000 7fc802d9d640 1 --2- 172.21.15.114:0/4012614746 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de7bf2c980 0x55de7bf27f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:31.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.263+0000 7fc80259c640 1 --2- 172.21.15.114:0/4012614746 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de7bf2c980 0x55de7bf27f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.263+0000 7fc80259c640 1 -- 172.21.15.114:0/4012614746 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de7be261c0 con 0x55de7bf217c0 2024-09-13T18:26:31.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.263+0000 7fc80259c640 1 --2- 172.21.15.114:0/4012614746 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55de7bf217c0 0x55de7bf33650 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec005ec0 tx=0x7fc7ec005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=995cd0b153a190aa server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:31.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.263+0000 7fc7f97fa640 1 -- 172.21.15.114:0/4012614746 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec005890 con 0x55de7bf217c0 2024-09-13T18:26:31.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.263+0000 7fc7f97fa640 1 -- 172.21.15.114:0/4012614746 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc7ec041070 con 0x55de7bf217c0 2024-09-13T18:26:31.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.263+0000 7fc7f97fa640 1 -- 172.21.15.114:0/4012614746 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec0303a0 con 0x55de7bf217c0 2024-09-13T18:26:31.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.263+0000 7fc80baa4740 1 -- 172.21.15.114:0/4012614746 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55de7bf217c0 msgr2=0x55de7bf33650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:31.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.263+0000 7fc80baa4740 1 --2- 172.21.15.114:0/4012614746 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55de7bf217c0 0x55de7bf33650 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec005ec0 tx=0x7fc7ec005a20 comp rx=0 tx=0).stop 2024-09-13T18:26:31.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.263+0000 7fc80baa4740 1 -- 172.21.15.114:0/4012614746 shutdown_connections 2024-09-13T18:26:31.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.263+0000 7fc80baa4740 1 --2- 172.21.15.114:0/4012614746 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de7bf2c980 0x55de7bf27f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.263+0000 7fc80baa4740 1 --2- 172.21.15.114:0/4012614746 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de7bf220e0 0x55de7bf33b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.263+0000 7fc80baa4740 1 --2- 172.21.15.114:0/4012614746 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55de7bf217c0 0x55de7bf33650 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.263+0000 7fc80baa4740 1 -- 172.21.15.114:0/4012614746 >> 172.21.15.114:0/4012614746 conn(0x55de7bf1dd50 msgr2=0x55de7bf249e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:31.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.264+0000 7fc80baa4740 1 -- 172.21.15.114:0/4012614746 shutdown_connections 2024-09-13T18:26:31.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.264+0000 7fc80baa4740 1 -- 172.21.15.114:0/4012614746 wait complete. 2024-09-13T18:26:31.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.264+0000 7fc80baa4740 1 -- start start 2024-09-13T18:26:31.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.264+0000 7fc80baa4740 1 Processor -- start 2024-09-13T18:26:31.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.264+0000 7fc80baa4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de7bf217c0 0x55de7bf33950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:31.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.264+0000 7fc80baa4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de7bf220e0 0x55de7bf31d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:31.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.264+0000 7fc80baa4740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55de7bf2c980 0x55de7bf32280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:31.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.264+0000 7fc80baa4740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de7bf32960 con 0x55de7bf2c980 2024-09-13T18:26:31.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.264+0000 7fc80baa4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de7bf32ad0 con 0x55de7bf220e0 2024-09-13T18:26:31.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.264+0000 7fc80baa4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de7bf92d00 con 0x55de7bf217c0 2024-09-13T18:26:31.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.264+0000 7fc80baa4740 10 client.0 _ll_register_callbacks cb 0x7fc7ec005720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:26:31.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.264+0000 7fc802d9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55de7bf2c980 0x55de7bf32280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:31.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.264+0000 7fc802d9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55de7bf2c980 0x55de7bf32280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49634/0 (socket says 172.21.15.114:49634) 2024-09-13T18:26:31.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.264+0000 7fc802d9d640 1 -- 172.21.15.114:0/3568670264 learned_addr learned my addr 172.21.15.114:0/3568670264 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:31.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.264+0000 7fc80259c640 1 --2- 172.21.15.114:0/3568670264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de7bf217c0 0x55de7bf33950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:31.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.264+0000 7fc801d9b640 1 --2- 172.21.15.114:0/3568670264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de7bf220e0 0x55de7bf31d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:31.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.264+0000 7fc802d9d640 1 -- 172.21.15.114:0/3568670264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de7bf217c0 msgr2=0x55de7bf33950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:31.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.264+0000 7fc802d9d640 1 --2- 172.21.15.114:0/3568670264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de7bf217c0 0x55de7bf33950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.264+0000 7fc802d9d640 1 -- 172.21.15.114:0/3568670264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de7bf220e0 msgr2=0x55de7bf31d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:31.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.264+0000 7fc802d9d640 1 --2- 172.21.15.114:0/3568670264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de7bf220e0 0x55de7bf31d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.264+0000 7fc802d9d640 1 -- 172.21.15.114:0/3568670264 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55de7be84360 con 0x55de7bf2c980 2024-09-13T18:26:31.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.265+0000 7fc802d9d640 1 --2- 172.21.15.114:0/3568670264 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55de7bf2c980 0x55de7bf32280 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc012cf0 tx=0x7fc7fc009910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a268c24a0db6bb2c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:31.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.265+0000 7fc7e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:26:31.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.265+0000 7fc7e9ffb640 1 -- 172.21.15.114:0/3568670264 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc00cbc0 con 0x55de7bf2c980 2024-09-13T18:26:31.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.265+0000 7fc7e9ffb640 1 -- 172.21.15.114:0/3568670264 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fc7fc012050 con 0x55de7bf2c980 2024-09-13T18:26:31.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.265+0000 7fc80baa4740 1 -- 172.21.15.114:0/3568670264 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc7fc009d10 con 0x55de7bf2c980 2024-09-13T18:26:31.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.265+0000 7fc7e9ffb640 1 -- 172.21.15.114:0/3568670264 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7fc012210 con 0x55de7bf2c980 2024-09-13T18:26:31.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.265+0000 7fc80baa4740 10 client.17031 fetch_fsmap learned FSMap version 32 2024-09-13T18:26:31.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.265+0000 7fc80baa4740 1 -- 172.21.15.114:0/3568670264 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55de7be4b990 con 0x55de7bf2c980 2024-09-13T18:26:31.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.265+0000 7fc7e9ffb640 1 -- 172.21.15.114:0/3568670264 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc7fc009d10 con 0x55de7bf2c980 2024-09-13T18:26:31.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.266+0000 7fc7e9ffb640 1 -- 172.21.15.114:0/3568670264 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fc7fc005450 con 0x55de7bf2c980 2024-09-13T18:26:31.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.266+0000 7fc80baa4740 10 client.17031 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:26:31.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.266+0000 7fc80baa4740 10 client.17031 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:26:31.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.266+0000 7fc80baa4740 20 client.17031 populate_metadata read hostname 'smithi114' 2024-09-13T18:26:31.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.266+0000 7fc80baa4740 1 --2- 172.21.15.114:0/3568670264 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55de7bf94470 0x55de7bf96890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:31.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.266+0000 7fc80baa4740 4 client.17031 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-13T18:26:31.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.266+0000 7fc80baa4740 1 -- 172.21.15.114:0/3568670264 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55de7bbddeb0 con 0x55de7bf94470 2024-09-13T18:26:31.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.266+0000 7fc80259c640 1 --2- 172.21.15.114:0/3568670264 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55de7bf94470 0x55de7bf96890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:31.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.267+0000 7fc7e9ffb640 10 client.17031 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:26:31.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.267+0000 7fc80259c640 1 --2- 172.21.15.114:0/3568670264 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55de7bf94470 0x55de7bf96890 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c0fb388456a3466 server_cookie=a7d7012589bf3eb1 in_seq=0 out_seq=0 2024-09-13T18:26:31.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.267+0000 7fc7e9ffb640 1 -- 172.21.15.114:0/3568670264 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55de7bbddeb0 con 0x55de7bf94470 2024-09-13T18:26:31.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.267+0000 7fc7e9ffb640 10 client.17031 handle_command_reply: tid=0 2024-09-13T18:26:31.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.267+0000 7fc80baa4740 1 client.17031 shutdown 2024-09-13T18:26:31.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.268+0000 7fc80baa4740 1 -- 172.21.15.114:0/3568670264 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55de7bf2c980 msgr2=0x55de7bf32280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:31.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.268+0000 7fc80baa4740 1 --2- 172.21.15.114:0/3568670264 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55de7bf2c980 0x55de7bf32280 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc012cf0 tx=0x7fc7fc009910 comp rx=0 tx=0).stop 2024-09-13T18:26:31.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.268+0000 7fc80baa4740 1 -- 172.21.15.114:0/3568670264 shutdown_connections 2024-09-13T18:26:31.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.268+0000 7fc80baa4740 1 --2- 172.21.15.114:0/3568670264 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55de7bf94470 0x55de7bf96890 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.268+0000 7fc80baa4740 1 --2- 172.21.15.114:0/3568670264 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55de7bf2c980 0x55de7bf32280 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.268+0000 7fc80baa4740 1 --2- 172.21.15.114:0/3568670264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de7bf220e0 0x55de7bf31d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.268+0000 7fc80baa4740 1 --2- 172.21.15.114:0/3568670264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de7bf217c0 0x55de7bf33950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.268+0000 7fc80baa4740 1 -- 172.21.15.114:0/3568670264 >> 172.21.15.114:0/3568670264 conn(0x55de7bf1dd50 msgr2=0x55de7bf27f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:31.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.268+0000 7fc7e9ffb640 0 client.17031 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:26:31.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.268+0000 7fc80baa4740 1 -- 172.21.15.114:0/3568670264 shutdown_connections 2024-09-13T18:26:31.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.268+0000 7fc80baa4740 1 -- 172.21.15.114:0/3568670264 wait complete. 2024-09-13T18:26:31.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.268+0000 7fc80baa4740 20 client.17031 trim_cache size 0 max 16384 2024-09-13T18:26:31.277 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:26:31.277 INFO:teuthology.orchestra.run.smithi114.stdout: "return_code": 0, 2024-09-13T18:26:31.278 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_tag": "168d4137-0689-462b-b012-bc200051d8cc", 2024-09-13T18:26:31.278 INFO:teuthology.orchestra.run.smithi114.stdout: "mode": "asynchronous" 2024-09-13T18:26:31.278 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:26:31.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.269+0000 7fc808d84640 1 -- 172.21.15.114:0/1839976865 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc7f00a1200 msgr2=0x7fc7f00a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:31.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.269+0000 7fc808d84640 1 --2- 172.21.15.114:0/1839976865 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc7f00a1200 0x7fc7f00a36c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc010c40 tx=0x7fc7fc00c000 comp rx=0 tx=0).stop 2024-09-13T18:26:31.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.269+0000 7fc808d84640 1 -- 172.21.15.114:0/1839976865 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc804103fb0 msgr2=0x7fc8041967b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:31.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.269+0000 7fc808d84640 1 --2- 172.21.15.114:0/1839976865 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc804103fb0 0x7fc8041967b0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fc7f400c970 tx=0x7fc7f400ce30 comp rx=0 tx=0).stop 2024-09-13T18:26:31.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.269+0000 7fc808d84640 1 -- 172.21.15.114:0/1839976865 shutdown_connections 2024-09-13T18:26:31.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.269+0000 7fc808d84640 1 --2- 172.21.15.114:0/1839976865 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc7f00a1200 0x7fc7f00a36c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.269+0000 7fc808d84640 1 --2- 172.21.15.114:0/1839976865 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc80419aba0 0x7fc80419b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.269+0000 7fc808d84640 1 --2- 172.21.15.114:0/1839976865 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc804103fb0 0x7fc8041967b0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.269+0000 7fc808d84640 1 --2- 172.21.15.114:0/1839976865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8040fee60 0x7fc804196270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.269+0000 7fc808d84640 1 -- 172.21.15.114:0/1839976865 >> 172.21.15.114:0/1839976865 conn(0x7fc804075c60 msgr2=0x7fc804102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:31.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.269+0000 7fc808d84640 1 -- 172.21.15.114:0/1839976865 shutdown_connections 2024-09-13T18:26:31.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.269+0000 7fc808d84640 1 -- 172.21.15.114:0/1839976865 wait complete. 2024-09-13T18:26:31.283 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T18:26:31.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:26:31 smithi114 ceph-mon[30897]: pgmap v124: 129 pgs: 129 active+clean; 14 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 65 MiB/s wr, 1.05k op/s 2024-09-13T18:26:31.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.484+0000 7fc13359e640 1 -- 172.21.15.114:0/3430941843 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc12c004c40 msgr2=0x7fc12c005080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:31.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.484+0000 7fc13359e640 1 --2- 172.21.15.114:0/3430941843 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc12c004c40 0x7fc12c005080 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fc128009900 tx=0x7fc12802a440 comp rx=0 tx=0).stop 2024-09-13T18:26:31.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.484+0000 7fc13359e640 1 -- 172.21.15.114:0/3430941843 shutdown_connections 2024-09-13T18:26:31.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.484+0000 7fc13359e640 1 --2- 172.21.15.114:0/3430941843 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc12c0a1f30 0x7fc12c0a4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.485+0000 7fc13359e640 1 --2- 172.21.15.114:0/3430941843 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc12c004c40 0x7fc12c005080 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.485+0000 7fc13359e640 1 --2- 172.21.15.114:0/3430941843 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc12c0964e0 0x7fc12c004700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.485+0000 7fc13359e640 1 -- 172.21.15.114:0/3430941843 >> 172.21.15.114:0/3430941843 conn(0x7fc12c00a1a0 msgr2=0x7fc12c00a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:31.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.488+0000 7fc13359e640 1 -- 172.21.15.114:0/3430941843 shutdown_connections 2024-09-13T18:26:31.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.490+0000 7fc13359e640 1 -- 172.21.15.114:0/3430941843 wait complete. 2024-09-13T18:26:31.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.491+0000 7fc13359e640 1 Processor -- start 2024-09-13T18:26:31.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.491+0000 7fc13359e640 1 -- start start 2024-09-13T18:26:31.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.492+0000 7fc13359e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc12c004c40 0x7fc12c12dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:31.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.492+0000 7fc13359e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc12c0964e0 0x7fc12c12e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:31.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.492+0000 7fc13259c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc12c004c40 0x7fc12c12dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:31.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.493+0000 7fc13259c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc12c004c40 0x7fc12c12dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:52862/0 (socket says 172.21.15.114:52862) 2024-09-13T18:26:31.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.493+0000 7fc13259c640 1 -- 172.21.15.114:0/287680340 learned_addr learned my addr 172.21.15.114:0/287680340 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:31.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.493+0000 7fc13359e640 1 --2- 172.21.15.114:0/287680340 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc12c0a1f30 0x7fc12c132650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:31.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.493+0000 7fc13359e640 1 -- 172.21.15.114:0/287680340 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12c132c50 con 0x7fc12c0a1f30 2024-09-13T18:26:31.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.493+0000 7fc13359e640 1 -- 172.21.15.114:0/287680340 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12c132dc0 con 0x7fc12c0964e0 2024-09-13T18:26:31.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.493+0000 7fc13359e640 1 -- 172.21.15.114:0/287680340 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12c132f80 con 0x7fc12c004c40 2024-09-13T18:26:31.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.493+0000 7fc132d9d640 1 --2- 172.21.15.114:0/287680340 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc12c0a1f30 0x7fc12c132650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:31.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.494+0000 7fc131d9b640 1 --2- 172.21.15.114:0/287680340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc12c0964e0 0x7fc12c12e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:31.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.494+0000 7fc132d9d640 1 -- 172.21.15.114:0/287680340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc12c004c40 msgr2=0x7fc12c12dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:31.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.494+0000 7fc132d9d640 1 --2- 172.21.15.114:0/287680340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc12c004c40 0x7fc12c12dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.494+0000 7fc132d9d640 1 -- 172.21.15.114:0/287680340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc12c0964e0 msgr2=0x7fc12c12e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:31.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.494+0000 7fc132d9d640 1 --2- 172.21.15.114:0/287680340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc12c0964e0 0x7fc12c12e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.494+0000 7fc132d9d640 1 -- 172.21.15.114:0/287680340 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc128009560 con 0x7fc12c0a1f30 2024-09-13T18:26:31.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.497+0000 7fc132d9d640 1 --2- 172.21.15.114:0/287680340 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc12c0a1f30 0x7fc12c132650 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fc13404fb10 tx=0x7fc1340774b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:31.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.497+0000 7fc1237fe640 1 -- 172.21.15.114:0/287680340 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc134081070 con 0x7fc12c0a1f30 2024-09-13T18:26:31.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.497+0000 7fc1237fe640 1 -- 172.21.15.114:0/287680340 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc134077e60 con 0x7fc12c0a1f30 2024-09-13T18:26:31.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.497+0000 7fc1237fe640 1 -- 172.21.15.114:0/287680340 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13407c780 con 0x7fc12c0a1f30 2024-09-13T18:26:31.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.497+0000 7fc13359e640 1 -- 172.21.15.114:0/287680340 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc12c133260 con 0x7fc12c0a1f30 2024-09-13T18:26:31.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.498+0000 7fc13359e640 1 -- 172.21.15.114:0/287680340 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc12c13aae0 con 0x7fc12c0a1f30 2024-09-13T18:26:31.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.503+0000 7fc1237fe640 1 -- 172.21.15.114:0/287680340 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fc13407c8e0 con 0x7fc12c0a1f30 2024-09-13T18:26:31.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.504+0000 7fc1237fe640 1 --2- 172.21.15.114:0/287680340 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc11c0a13d0 0x7fc11c0a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:31.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.504+0000 7fc13bc2d740 1 Processor -- start 2024-09-13T18:26:31.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.504+0000 7fc1237fe640 1 -- 172.21.15.114:0/287680340 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fc134136b40 con 0x7fc12c0a1f30 2024-09-13T18:26:31.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.504+0000 7fc13259c640 1 --2- 172.21.15.114:0/287680340 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc11c0a13d0 0x7fc11c0a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:31.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.507+0000 7fc13bc2d740 1 -- start start 2024-09-13T18:26:31.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.508+0000 7fc13bc2d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558066a40840 0x558066a40c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:31.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.508+0000 7fc13bc2d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558066a41160 0x558066a4b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:31.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.508+0000 7fc13bc2d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558066a4ba00 0x558066a4ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:31.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.508+0000 7fc13bc2d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580669a2f20 con 0x558066a40840 2024-09-13T18:26:31.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.508+0000 7fc13bc2d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580669a06b0 con 0x558066a4ba00 2024-09-13T18:26:31.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.508+0000 7fc13bc2d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806699bfa0 con 0x558066a41160 2024-09-13T18:26:31.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.508+0000 7fc132d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558066a4ba00 0x558066a4ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:31.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.508+0000 7fc132d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558066a4ba00 0x558066a4ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:60806/0 (socket says 172.21.15.114:60806) 2024-09-13T18:26:31.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.508+0000 7fc132d9d640 1 -- 172.21.15.114:0/4283306170 learned_addr learned my addr 172.21.15.114:0/4283306170 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:31.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.509+0000 7fc132d9d640 1 -- 172.21.15.114:0/4283306170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558066a41160 msgr2=0x558066a4b4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:26:31.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.509+0000 7fc132d9d640 1 --2- 172.21.15.114:0/4283306170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558066a41160 0x558066a4b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.509+0000 7fc132d9d640 1 -- 172.21.15.114:0/4283306170 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558066a40840 msgr2=0x558066a40c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:31.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.509+0000 7fc132d9d640 1 --2- 172.21.15.114:0/4283306170 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558066a40840 0x558066a40c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.509+0000 7fc132d9d640 1 -- 172.21.15.114:0/4283306170 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558066956840 con 0x558066a4ba00 2024-09-13T18:26:31.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.511+0000 7fc132d9d640 1 --2- 172.21.15.114:0/4283306170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558066a4ba00 0x558066a4ddf0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc1341004b0 tx=0x7fc134104140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44cf2ba0e7370b12 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:31.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.512+0000 7fc120ff9640 1 -- 172.21.15.114:0/4283306170 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13410be30 con 0x558066a4ba00 2024-09-13T18:26:31.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.512+0000 7fc120ff9640 1 -- 172.21.15.114:0/4283306170 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc134101de0 con 0x558066a4ba00 2024-09-13T18:26:31.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.512+0000 7fc120ff9640 1 -- 172.21.15.114:0/4283306170 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13410b9a0 con 0x558066a4ba00 2024-09-13T18:26:31.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.512+0000 7fc13259c640 1 --2- 172.21.15.114:0/287680340 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc11c0a13d0 0x7fc11c0a3890 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc124009ba0 tx=0x7fc1240095b0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:31.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.513+0000 7fc13bc2d740 1 -- 172.21.15.114:0/4283306170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558066a4ba00 msgr2=0x558066a4ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:31.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.513+0000 7fc13bc2d740 1 --2- 172.21.15.114:0/4283306170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558066a4ba00 0x558066a4ddf0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc1341004b0 tx=0x7fc134104140 comp rx=0 tx=0).stop 2024-09-13T18:26:31.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.516+0000 7fc13bc2d740 1 -- 172.21.15.114:0/4283306170 shutdown_connections 2024-09-13T18:26:31.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.516+0000 7fc13bc2d740 1 --2- 172.21.15.114:0/4283306170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558066a4ba00 0x558066a4ddf0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.516+0000 7fc13bc2d740 1 --2- 172.21.15.114:0/4283306170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558066a41160 0x558066a4b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.516+0000 7fc13bc2d740 1 --2- 172.21.15.114:0/4283306170 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558066a40840 0x558066a40c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.516+0000 7fc13bc2d740 1 -- 172.21.15.114:0/4283306170 >> 172.21.15.114:0/4283306170 conn(0x558066a3cdb0 msgr2=0x558066a3d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:31.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.520+0000 7fc13bc2d740 1 -- 172.21.15.114:0/4283306170 shutdown_connections 2024-09-13T18:26:31.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.521+0000 7fc13bc2d740 1 -- 172.21.15.114:0/4283306170 wait complete. 2024-09-13T18:26:31.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.521+0000 7fc13bc2d740 1 -- start start 2024-09-13T18:26:31.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.521+0000 7fc13bc2d740 1 Processor -- start 2024-09-13T18:26:31.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.524+0000 7fc13bc2d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558066a40840 0x558066a3c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:31.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.524+0000 7fc13bc2d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558066a41160 0x558066a395e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:31.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.524+0000 7fc13bc2d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558066a4ba00 0x558066a39d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:31.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.524+0000 7fc13bc2d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806693d2a0 con 0x558066a41160 2024-09-13T18:26:31.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.524+0000 7fc13bc2d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558066674430 con 0x558066a4ba00 2024-09-13T18:26:31.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.524+0000 7fc13bc2d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806691f680 con 0x558066a40840 2024-09-13T18:26:31.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.524+0000 7fc13bc2d740 10 client.0 _ll_register_callbacks cb 0x558066995610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:26:31.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.525+0000 7fc132d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558066a4ba00 0x558066a39d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:31.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.526+0000 7fc132d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558066a4ba00 0x558066a39d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:60822/0 (socket says 172.21.15.114:60822) 2024-09-13T18:26:31.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.526+0000 7fc132d9d640 1 -- 172.21.15.114:0/2906804898 learned_addr learned my addr 172.21.15.114:0/2906804898 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:31.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.526+0000 7fc132d9d640 1 -- 172.21.15.114:0/2906804898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558066a40840 msgr2=0x558066a3c2f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:26:31.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.526+0000 7fc132d9d640 1 --2- 172.21.15.114:0/2906804898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558066a40840 0x558066a3c2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.526+0000 7fc132d9d640 1 -- 172.21.15.114:0/2906804898 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558066a41160 msgr2=0x558066a395e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:31.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.526+0000 7fc132d9d640 1 --2- 172.21.15.114:0/2906804898 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558066a41160 0x558066a395e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.526+0000 7fc132d9d640 1 -- 172.21.15.114:0/2906804898 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc12c13aae0 con 0x558066a4ba00 2024-09-13T18:26:31.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.526+0000 7fc132d9d640 1 --2- 172.21.15.114:0/2906804898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558066a4ba00 0x558066a39d70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc13402c9a0 tx=0x7fc134101f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7e278cbab4a6ad8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:31.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.527+0000 7fc119ffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:26:31.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.527+0000 7fc119ffb640 1 -- 172.21.15.114:0/2906804898 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc134069ea0 con 0x558066a4ba00 2024-09-13T18:26:31.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.527+0000 7fc119ffb640 1 -- 172.21.15.114:0/2906804898 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fc13410be30 con 0x558066a4ba00 2024-09-13T18:26:31.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.527+0000 7fc13bc2d740 1 -- 172.21.15.114:0/2906804898 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55806668f4e0 con 0x558066a4ba00 2024-09-13T18:26:31.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.529+0000 7fc119ffb640 1 -- 172.21.15.114:0/2906804898 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc134134360 con 0x558066a4ba00 2024-09-13T18:26:31.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.530+0000 7fc13bc2d740 10 client.26384 fetch_fsmap learned FSMap version 32 2024-09-13T18:26:31.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.530+0000 7fc13bc2d740 1 -- 172.21.15.114:0/2906804898 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55806699a620 con 0x558066a4ba00 2024-09-13T18:26:31.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.531+0000 7fc119ffb640 1 -- 172.21.15.114:0/2906804898 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55806668f4e0 con 0x558066a4ba00 2024-09-13T18:26:31.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.531+0000 7fc119ffb640 1 -- 172.21.15.114:0/2906804898 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fc1340ff830 con 0x558066a4ba00 2024-09-13T18:26:31.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.531+0000 7fc13bc2d740 10 client.26384 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:26:31.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.531+0000 7fc13bc2d740 10 client.26384 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:26:31.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.532+0000 7fc13bc2d740 20 client.26384 populate_metadata read hostname 'smithi114' 2024-09-13T18:26:31.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.532+0000 7fc13bc2d740 1 --2- 172.21.15.114:0/2906804898 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x558066a74b30 0x558066a76f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:31.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.532+0000 7fc13bc2d740 4 client.26384 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:26:31.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.532+0000 7fc13bc2d740 1 -- 172.21.15.114:0/2906804898 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5580666a07e0 con 0x558066a74b30 2024-09-13T18:26:31.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.535+0000 7fc13259c640 1 --2- 172.21.15.114:0/2906804898 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x558066a74b30 0x558066a76f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:31.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.536+0000 7fc13259c640 1 --2- 172.21.15.114:0/2906804898 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x558066a74b30 0x558066a76f20 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=988d5a8f6e360401 server_cookie=9d847711d13b0e52 in_seq=0 out_seq=0 2024-09-13T18:26:31.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.536+0000 7fc119ffb640 10 client.26384 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:26:31.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.538+0000 7fc119ffb640 1 -- 172.21.15.114:0/2906804898 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5580666a07e0 con 0x558066a74b30 2024-09-13T18:26:31.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.538+0000 7fc119ffb640 10 client.26384 handle_command_reply: tid=0 2024-09-13T18:26:31.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.538+0000 7fc13bc2d740 1 client.26384 shutdown 2024-09-13T18:26:31.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.539+0000 7fc13bc2d740 1 -- 172.21.15.114:0/2906804898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558066a4ba00 msgr2=0x558066a39d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:31.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.539+0000 7fc13bc2d740 1 --2- 172.21.15.114:0/2906804898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558066a4ba00 0x558066a39d70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc13402c9a0 tx=0x7fc134101f70 comp rx=0 tx=0).stop 2024-09-13T18:26:31.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.539+0000 7fc13bc2d740 1 -- 172.21.15.114:0/2906804898 shutdown_connections 2024-09-13T18:26:31.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.539+0000 7fc13bc2d740 1 --2- 172.21.15.114:0/2906804898 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x558066a74b30 0x558066a76f20 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.539+0000 7fc13bc2d740 1 --2- 172.21.15.114:0/2906804898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558066a4ba00 0x558066a39d70 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.539+0000 7fc13bc2d740 1 --2- 172.21.15.114:0/2906804898 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558066a41160 0x558066a395e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.539+0000 7fc13bc2d740 1 --2- 172.21.15.114:0/2906804898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558066a40840 0x558066a3c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.539+0000 7fc13bc2d740 1 -- 172.21.15.114:0/2906804898 >> 172.21.15.114:0/2906804898 conn(0x558066a3cdb0 msgr2=0x558066a42b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:31.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.540+0000 7fc119ffb640 0 client.26384 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:26:31.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.540+0000 7fc13bc2d740 1 -- 172.21.15.114:0/2906804898 shutdown_connections 2024-09-13T18:26:31.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.540+0000 7fc13bc2d740 1 -- 172.21.15.114:0/2906804898 wait complete. 2024-09-13T18:26:31.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.542+0000 7fc13bc2d740 20 client.26384 trim_cache size 0 max 16384 2024-09-13T18:26:31.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.579+0000 7fc13bc2d740 1 Processor -- start 2024-09-13T18:26:31.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.579+0000 7fc13bc2d740 1 -- start start 2024-09-13T18:26:31.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.579+0000 7fc13bc2d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558066a40840 0x558066a76b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:31.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.579+0000 7fc13bc2d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558066a41160 0x558066a77080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:31.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.580+0000 7fc13bc2d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558066a4ba00 0x558066a3bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:31.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.580+0000 7fc13bc2d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806668f510 con 0x558066a41160 2024-09-13T18:26:31.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.580+0000 7fc13bc2d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558066a42090 con 0x558066a4ba00 2024-09-13T18:26:31.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.580+0000 7fc13bc2d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558066a2fee0 con 0x558066a40840 2024-09-13T18:26:31.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.580+0000 7fc131d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558066a41160 0x558066a77080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:31.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.580+0000 7fc131d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558066a41160 0x558066a77080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49670/0 (socket says 172.21.15.114:49670) 2024-09-13T18:26:31.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.580+0000 7fc131d9b640 1 -- 172.21.15.114:0/2672703882 learned_addr learned my addr 172.21.15.114:0/2672703882 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:31.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.580+0000 7fc132d9d640 1 --2- 172.21.15.114:0/2672703882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558066a4ba00 0x558066a3bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:31.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.581+0000 7fc13259c640 1 --2- 172.21.15.114:0/2672703882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558066a40840 0x558066a76b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:31.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.581+0000 7fc131d9b640 1 -- 172.21.15.114:0/2672703882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558066a40840 msgr2=0x558066a76b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:31.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.581+0000 7fc131d9b640 1 --2- 172.21.15.114:0/2672703882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558066a40840 0x558066a76b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.581+0000 7fc131d9b640 1 -- 172.21.15.114:0/2672703882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558066a4ba00 msgr2=0x558066a3bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:31.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.581+0000 7fc131d9b640 1 --2- 172.21.15.114:0/2672703882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558066a4ba00 0x558066a3bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.581+0000 7fc131d9b640 1 -- 172.21.15.114:0/2672703882 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55806693c790 con 0x558066a41160 2024-09-13T18:26:31.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.581+0000 7fc131d9b640 1 --2- 172.21.15.114:0/2672703882 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558066a41160 0x558066a77080 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fc12802aa30 tx=0x7fc1280040f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6295dbad2ebcb460 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:31.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.582+0000 7fc1217fa640 1 -- 172.21.15.114:0/2672703882 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc128004460 con 0x558066a41160 2024-09-13T18:26:31.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.582+0000 7fc1217fa640 1 -- 172.21.15.114:0/2672703882 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc128005490 con 0x558066a41160 2024-09-13T18:26:31.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.583+0000 7fc13bc2d740 1 -- 172.21.15.114:0/2672703882 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558066a41160 msgr2=0x558066a77080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:31.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.583+0000 7fc13bc2d740 1 --2- 172.21.15.114:0/2672703882 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558066a41160 0x558066a77080 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fc12802aa30 tx=0x7fc1280040f0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.583+0000 7fc13bc2d740 1 -- 172.21.15.114:0/2672703882 shutdown_connections 2024-09-13T18:26:31.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.583+0000 7fc13bc2d740 1 --2- 172.21.15.114:0/2672703882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558066a4ba00 0x558066a3bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.583+0000 7fc13bc2d740 1 --2- 172.21.15.114:0/2672703882 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558066a41160 0x558066a77080 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.583+0000 7fc13bc2d740 1 --2- 172.21.15.114:0/2672703882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558066a40840 0x558066a76b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.583+0000 7fc13bc2d740 1 -- 172.21.15.114:0/2672703882 >> 172.21.15.114:0/2672703882 conn(0x558066a3cdb0 msgr2=0x558066a348b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:31.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.588+0000 7fc13bc2d740 1 -- 172.21.15.114:0/2672703882 shutdown_connections 2024-09-13T18:26:31.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.590+0000 7fc13bc2d740 1 -- 172.21.15.114:0/2672703882 wait complete. 2024-09-13T18:26:31.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.591+0000 7fc13bc2d740 1 -- start start 2024-09-13T18:26:31.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.591+0000 7fc13bc2d740 1 Processor -- start 2024-09-13T18:26:31.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.591+0000 7fc13bc2d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558066a40840 0x558066a467b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:31.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.591+0000 7fc13bc2d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558066a41160 0x558066a36ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:31.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.591+0000 7fc13bc2d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558066a4ba00 0x558066a35830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:31.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.591+0000 7fc13bc2d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x558066a375f0 con 0x558066a40840 2024-09-13T18:26:31.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.591+0000 7fc13bc2d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558066a35d70 con 0x558066a41160 2024-09-13T18:26:31.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.591+0000 7fc13bc2d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558066a35f30 con 0x558066a4ba00 2024-09-13T18:26:31.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.592+0000 7fc13bc2d740 10 client.0 _ll_register_callbacks cb 0x7fc13411a070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:26:31.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.592+0000 7fc131d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558066a41160 0x558066a36ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:31.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.592+0000 7fc131d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558066a41160 0x558066a36ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:60836/0 (socket says 172.21.15.114:60836) 2024-09-13T18:26:31.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.592+0000 7fc131d9b640 1 -- 172.21.15.114:0/1339407252 learned_addr learned my addr 172.21.15.114:0/1339407252 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:31.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.592+0000 7fc13259c640 1 --2- 172.21.15.114:0/1339407252 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558066a40840 0x558066a467b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:31.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.592+0000 7fc132d9d640 1 --2- 172.21.15.114:0/1339407252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558066a4ba00 0x558066a35830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:31.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.592+0000 7fc131d9b640 1 -- 172.21.15.114:0/1339407252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558066a4ba00 msgr2=0x558066a35830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:31.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.592+0000 7fc131d9b640 1 --2- 172.21.15.114:0/1339407252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558066a4ba00 0x558066a35830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.592+0000 7fc131d9b640 1 -- 172.21.15.114:0/1339407252 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558066a40840 msgr2=0x558066a467b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:31.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.592+0000 7fc131d9b640 1 --2- 172.21.15.114:0/1339407252 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558066a40840 0x558066a467b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.592+0000 7fc131d9b640 1 -- 172.21.15.114:0/1339407252 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc12c0a4ed0 con 0x558066a41160 2024-09-13T18:26:31.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.593+0000 7fc131d9b640 1 --2- 172.21.15.114:0/1339407252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558066a41160 0x558066a36ca0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc128005ec0 tx=0x7fc128005230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fda23afb55277bc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:31.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.593+0000 7fc119ffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:26:31.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.595+0000 7fc119ffb640 1 -- 172.21.15.114:0/1339407252 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc12804b070 con 0x558066a41160 2024-09-13T18:26:31.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.595+0000 7fc119ffb640 1 -- 172.21.15.114:0/1339407252 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fc1280052a0 con 0x558066a41160 2024-09-13T18:26:31.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.595+0000 7fc13bc2d740 1 -- 172.21.15.114:0/1339407252 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc134101de0 con 0x558066a41160 2024-09-13T18:26:31.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.595+0000 7fc119ffb640 1 -- 172.21.15.114:0/1339407252 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1280048d0 con 0x558066a41160 2024-09-13T18:26:31.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.595+0000 7fc13bc2d740 10 client.26390 fetch_fsmap learned FSMap version 32 2024-09-13T18:26:31.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.595+0000 7fc13bc2d740 1 -- 172.21.15.114:0/1339407252 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x558066961ea0 con 0x558066a41160 2024-09-13T18:26:31.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.597+0000 7fc119ffb640 1 -- 172.21.15.114:0/1339407252 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc134101de0 con 0x558066a41160 2024-09-13T18:26:31.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.597+0000 7fc119ffb640 1 -- 172.21.15.114:0/1339407252 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fc12803d790 con 0x558066a41160 2024-09-13T18:26:31.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.597+0000 7fc13bc2d740 10 client.26390 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:26:31.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.597+0000 7fc13bc2d740 10 client.26390 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:26:31.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.597+0000 7fc13bc2d740 20 client.26390 populate_metadata read hostname 'smithi114' 2024-09-13T18:26:31.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.597+0000 7fc13bc2d740 1 --2- 172.21.15.114:0/1339407252 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x558066acb8c0 0x558066acdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:31.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.597+0000 7fc13bc2d740 4 client.26390 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T18:26:31.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.597+0000 7fc13bc2d740 1 -- 172.21.15.114:0/1339407252 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5580666f3e30 con 0x558066acb8c0 2024-09-13T18:26:31.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.599+0000 7fc13259c640 1 --2- 172.21.15.114:0/1339407252 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x558066acb8c0 0x558066acdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:31.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.600+0000 7fc13259c640 1 --2- 172.21.15.114:0/1339407252 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x558066acb8c0 0x558066acdce0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a37dfa7d399b1406 server_cookie=2caf44a09ac8a907 in_seq=0 out_seq=0 2024-09-13T18:26:31.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.600+0000 7fc119ffb640 10 client.26390 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:26:31.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.601+0000 7fc119ffb640 1 -- 172.21.15.114:0/1339407252 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5580666f3e30 con 0x558066acb8c0 2024-09-13T18:26:31.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.601+0000 7fc119ffb640 10 client.26390 handle_command_reply: tid=0 2024-09-13T18:26:31.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.601+0000 7fc13bc2d740 1 client.26390 shutdown 2024-09-13T18:26:31.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.606+0000 7fc13bc2d740 1 -- 172.21.15.114:0/1339407252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558066a41160 msgr2=0x558066a36ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:31.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.606+0000 7fc13bc2d740 1 --2- 172.21.15.114:0/1339407252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558066a41160 0x558066a36ca0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc128005ec0 tx=0x7fc128005230 comp rx=0 tx=0).stop 2024-09-13T18:26:31.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.608+0000 7fc13bc2d740 1 -- 172.21.15.114:0/1339407252 shutdown_connections 2024-09-13T18:26:31.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.608+0000 7fc13bc2d740 1 --2- 172.21.15.114:0/1339407252 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x558066acb8c0 0x558066acdce0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.608+0000 7fc13bc2d740 1 --2- 172.21.15.114:0/1339407252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558066a4ba00 0x558066a35830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.608+0000 7fc13bc2d740 1 --2- 172.21.15.114:0/1339407252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558066a41160 0x558066a36ca0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.608+0000 7fc13bc2d740 1 --2- 172.21.15.114:0/1339407252 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558066a40840 0x558066a467b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.608+0000 7fc13bc2d740 1 -- 172.21.15.114:0/1339407252 >> 172.21.15.114:0/1339407252 conn(0x558066a3cdb0 msgr2=0x558066a348b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:31.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.608+0000 7fc119ffb640 0 client.26390 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:26:31.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.608+0000 7fc13bc2d740 1 -- 172.21.15.114:0/1339407252 shutdown_connections 2024-09-13T18:26:31.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.608+0000 7fc13bc2d740 1 -- 172.21.15.114:0/1339407252 wait complete. 2024-09-13T18:26:31.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.609+0000 7fc13bc2d740 20 client.26390 trim_cache size 0 max 16384 2024-09-13T18:26:31.612 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:26:31.612 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-13T18:26:31.612 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": { 2024-09-13T18:26:31.612 INFO:teuthology.orchestra.run.smithi114.stdout: "168d4137-0689-462b-b012-bc200051d8cc": { 2024-09-13T18:26:31.612 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-13T18:26:31.612 INFO:teuthology.orchestra.run.smithi114.stdout: "tag": "168d4137-0689-462b-b012-bc200051d8cc", 2024-09-13T18:26:31.612 INFO:teuthology.orchestra.run.smithi114.stdout: "options": "recursive,force" 2024-09-13T18:26:31.612 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:26:31.612 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:26:31.612 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:26:31.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:26:31 smithi134 ceph-mon[34283]: pgmap v124: 129 pgs: 129 active+clean; 14 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 65 MiB/s wr, 1.05k op/s 2024-09-13T18:26:31.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.617+0000 7fc13359e640 1 -- 172.21.15.114:0/287680340 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc11c0a13d0 msgr2=0x7fc11c0a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:31.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.617+0000 7fc13359e640 1 --2- 172.21.15.114:0/287680340 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc11c0a13d0 0x7fc11c0a3890 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc124009ba0 tx=0x7fc1240095b0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.617+0000 7fc13359e640 1 -- 172.21.15.114:0/287680340 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc12c0a1f30 msgr2=0x7fc12c132650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:31.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.617+0000 7fc13359e640 1 --2- 172.21.15.114:0/287680340 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc12c0a1f30 0x7fc12c132650 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fc13404fb10 tx=0x7fc1340774b0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.617+0000 7fc13359e640 1 -- 172.21.15.114:0/287680340 shutdown_connections 2024-09-13T18:26:31.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.617+0000 7fc13359e640 1 --2- 172.21.15.114:0/287680340 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc11c0a13d0 0x7fc11c0a3890 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.617+0000 7fc13359e640 1 --2- 172.21.15.114:0/287680340 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc12c0a1f30 0x7fc12c132650 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.617+0000 7fc13359e640 1 --2- 172.21.15.114:0/287680340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc12c0964e0 0x7fc12c12e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.617+0000 7fc13359e640 1 --2- 172.21.15.114:0/287680340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc12c004c40 0x7fc12c12dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:31.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.617+0000 7fc13359e640 1 -- 172.21.15.114:0/287680340 >> 172.21.15.114:0/287680340 conn(0x7fc12c00a1a0 msgr2=0x7fc12c0a3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:31.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.618+0000 7fc13359e640 1 -- 172.21.15.114:0/287680340 shutdown_connections 2024-09-13T18:26:31.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:26:31.618+0000 7fc13359e640 1 -- 172.21.15.114:0/287680340 wait complete. 2024-09-13T18:26:31.645 INFO:tasks.cephfs.filesystem:scrub status for tag:168d4137-0689-462b-b012-bc200051d8cc - {'path': '/', 'tag': '168d4137-0689-462b-b012-bc200051d8cc', 'options': 'recursive,force'} 2024-09-13T18:26:31.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:26:31 smithi204 ceph-mon[34777]: pgmap v124: 129 pgs: 129 active+clean; 14 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 65 MiB/s wr, 1.05k op/s 2024-09-13T18:26:32.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:26:32 smithi134 ceph-mon[34283]: scrub queued for path: / 2024-09-13T18:26:32.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:26:32 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:26:32.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:26:32 smithi134 ceph-mon[34283]: scrub summary: active paths [/] 2024-09-13T18:26:32.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:26:32 smithi134 ceph-mon[34283]: scrub summary: active paths [/] 2024-09-13T18:26:32.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:26:32 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:26:32.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:26:32 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:26:32.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:26:32 smithi204 ceph-mon[34777]: scrub queued for path: / 2024-09-13T18:26:32.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:26:32 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:26:32.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:26:32 smithi204 ceph-mon[34777]: scrub summary: active paths [/] 2024-09-13T18:26:32.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:26:32 smithi204 ceph-mon[34777]: scrub summary: active paths [/] 2024-09-13T18:26:32.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:26:32 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:26:32.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:26:32 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:26:32.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:26:32 smithi114 ceph-mon[30897]: scrub queued for path: / 2024-09-13T18:26:32.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:26:32 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:26:32.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:26:32 smithi114 ceph-mon[30897]: scrub summary: active paths [/] 2024-09-13T18:26:32.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:26:32 smithi114 ceph-mon[30897]: scrub summary: active paths [/] 2024-09-13T18:26:32.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:26:32 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:26:32.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:26:32 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:26:33.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:26:33 smithi134 ceph-mon[34283]: scrub summary: active paths [/] 2024-09-13T18:26:33.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:26:33 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:26:33.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:26:33 smithi134 ceph-mon[34283]: pgmap v125: 129 pgs: 129 active+clean; 14 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 43 MiB/s wr, 689 op/s 2024-09-13T18:26:33.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:26:33 smithi204 ceph-mon[34777]: scrub summary: active paths [/] 2024-09-13T18:26:33.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:26:33 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:26:33.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:26:33 smithi204 ceph-mon[34777]: pgmap v125: 129 pgs: 129 active+clean; 14 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 43 MiB/s wr, 689 op/s 2024-09-13T18:26:33.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:26:33 smithi114 ceph-mon[30897]: scrub summary: active paths [/] 2024-09-13T18:26:33.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:26:33 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:26:33.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:26:33 smithi114 ceph-mon[30897]: pgmap v125: 129 pgs: 129 active+clean; 14 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 43 MiB/s wr, 689 op/s 2024-09-13T18:26:35.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:26:35 smithi134 ceph-mon[34283]: pgmap v126: 129 pgs: 129 active+clean; 15 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 58 MiB/s wr, 945 op/s 2024-09-13T18:26:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.685+0000 7fafe72b6640 1 -- 172.21.15.114:0/510086470 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fafe0101630 msgr2=0x7fafe0109b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.685+0000 7fafe72b6640 1 --2- 172.21.15.114:0/510086470 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fafe0101630 0x7fafe0109b10 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fafd0009900 tx=0x7fafd002a480 comp rx=0 tx=0).stop 2024-09-13T18:26:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.685+0000 7fafe72b6640 1 -- 172.21.15.114:0/510086470 shutdown_connections 2024-09-13T18:26:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.685+0000 7fafe72b6640 1 --2- 172.21.15.114:0/510086470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafe010a0e0 0x7fafe010c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.685+0000 7fafe72b6640 1 --2- 172.21.15.114:0/510086470 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fafe0101630 0x7fafe0109b10 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.685+0000 7fafe72b6640 1 --2- 172.21.15.114:0/510086470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafe0100d10 0x7fafe01010f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.685+0000 7fafe72b6640 1 -- 172.21.15.114:0/510086470 >> 172.21.15.114:0/510086470 conn(0x7fafe00759b0 msgr2=0x7fafe00fcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.685+0000 7fafe72b6640 1 -- 172.21.15.114:0/510086470 shutdown_connections 2024-09-13T18:26:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.685+0000 7fafe72b6640 1 -- 172.21.15.114:0/510086470 wait complete. 2024-09-13T18:26:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.686+0000 7fafe72b6640 1 Processor -- start 2024-09-13T18:26:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.686+0000 7fafe72b6640 1 -- start start 2024-09-13T18:26:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.686+0000 7fafe72b6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafe0100d10 0x7fafe0195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.686+0000 7fafe72b6640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fafe0101630 0x7fafe01963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.686+0000 7fafe72b6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafe010a0e0 0x7fafe019a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.686+0000 7fafe72b6640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe019ad90 con 0x7fafe0101630 2024-09-13T18:26:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.686+0000 7fafe72b6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe019af00 con 0x7fafe0100d10 2024-09-13T18:26:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.686+0000 7fafe72b6640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe019b0c0 con 0x7fafe010a0e0 2024-09-13T18:26:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.686+0000 7fafe5ab3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fafe0101630 0x7fafe01963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.686+0000 7fafe5ab3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fafe0101630 0x7fafe01963a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37238/0 (socket says 172.21.15.114:37238) 2024-09-13T18:26:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.686+0000 7fafe5ab3640 1 -- 172.21.15.114:0/572566190 learned_addr learned my addr 172.21.15.114:0/572566190 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.686+0000 7fafe62b4640 1 --2- 172.21.15.114:0/572566190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafe0100d10 0x7fafe0195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.686+0000 7fafe6ab5640 1 --2- 172.21.15.114:0/572566190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafe010a0e0 0x7fafe019a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.686+0000 7fafe5ab3640 1 -- 172.21.15.114:0/572566190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafe010a0e0 msgr2=0x7fafe019a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.686+0000 7fafe5ab3640 1 --2- 172.21.15.114:0/572566190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafe010a0e0 0x7fafe019a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.686+0000 7fafe5ab3640 1 -- 172.21.15.114:0/572566190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafe0100d10 msgr2=0x7fafe0195e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.686+0000 7fafe5ab3640 1 --2- 172.21.15.114:0/572566190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafe0100d10 0x7fafe0195e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.686+0000 7fafe5ab3640 1 -- 172.21.15.114:0/572566190 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafd0009560 con 0x7fafe0101630 2024-09-13T18:26:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.687+0000 7fafe5ab3640 1 --2- 172.21.15.114:0/572566190 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fafe0101630 0x7fafe01963a0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fafd00057c0 tx=0x7fafd0032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.687+0000 7fafd77fe640 1 -- 172.21.15.114:0/572566190 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd0032e10 con 0x7fafe0101630 2024-09-13T18:26:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.687+0000 7fafe72b6640 1 -- 172.21.15.114:0/572566190 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafe019b340 con 0x7fafe0101630 2024-09-13T18:26:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.687+0000 7fafd77fe640 1 -- 172.21.15.114:0/572566190 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fafd00046c0 con 0x7fafe0101630 2024-09-13T18:26:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.687+0000 7fafd77fe640 1 -- 172.21.15.114:0/572566190 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd0003850 con 0x7fafe0101630 2024-09-13T18:26:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.687+0000 7fafe72b6640 1 -- 172.21.15.114:0/572566190 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafe019b610 con 0x7fafe0101630 2024-09-13T18:26:35.698 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:26:35 smithi204 ceph-mon[34777]: pgmap v126: 129 pgs: 129 active+clean; 15 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 58 MiB/s wr, 945 op/s 2024-09-13T18:26:35.698 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.689+0000 7fafd77fe640 1 -- 172.21.15.114:0/572566190 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fafd0005210 con 0x7fafe0101630 2024-09-13T18:26:35.699 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.689+0000 7fafeb238740 1 Processor -- start 2024-09-13T18:26:35.699 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.689+0000 7fafeb238740 1 -- start start 2024-09-13T18:26:35.699 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.690+0000 7fafeb238740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c685779500 0x55c6857798e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:35.699 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.690+0000 7fafd77fe640 1 --2- 172.21.15.114:0/572566190 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fafcc0a12a0 0x7fafcc0a3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:35.699 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.690+0000 7fafeb238740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c685779e20 0x55c685784180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:35.699 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.690+0000 7fafeb238740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c6857846c0 0x55c685786ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:35.699 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.690+0000 7fafeb238740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6856e4c30 con 0x55c6857846c0 2024-09-13T18:26:35.699 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.690+0000 7fafeb238740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6856e23c0 con 0x55c685779e20 2024-09-13T18:26:35.699 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.690+0000 7fafeb238740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6856ddcb0 con 0x55c685779500 2024-09-13T18:26:35.699 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.690+0000 7fafe62b4640 1 --2- 172.21.15.114:0/572566190 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fafcc0a12a0 0x7fafcc0a3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:35.699 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.690+0000 7fafe6ab5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c6857846c0 0x55c685786ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:35.699 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.690+0000 7fafe6ab5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c6857846c0 0x55c685786ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37252/0 (socket says 172.21.15.114:37252) 2024-09-13T18:26:35.700 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.690+0000 7fafe6ab5640 1 -- 172.21.15.114:0/1337424083 learned_addr learned my addr 172.21.15.114:0/1337424083 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:35.700 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.690+0000 7fafe5ab3640 1 --2- 172.21.15.114:0/1337424083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c685779e20 0x55c685784180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:35.700 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.690+0000 7fafe6ab5640 1 --2- 172.21.15.114:0/1337424083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c685779500 0x55c6857798e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:35.700 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.690+0000 7fafd77fe640 1 -- 172.21.15.114:0/572566190 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fafd00edf00 con 0x7fafe0101630 2024-09-13T18:26:35.700 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.690+0000 7fafe6ab5640 1 -- 172.21.15.114:0/1337424083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c685779500 msgr2=0x55c6857798e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:35.700 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.690+0000 7fafe6ab5640 1 --2- 172.21.15.114:0/1337424083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c685779500 0x55c6857798e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:35.700 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.690+0000 7fafe6ab5640 1 -- 172.21.15.114:0/1337424083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c685779e20 msgr2=0x55c685784180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:35.700 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.690+0000 7fafe6ab5640 1 --2- 172.21.15.114:0/1337424083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c685779e20 0x55c685784180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:35.700 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.690+0000 7fafe6ab5640 1 -- 172.21.15.114:0/1337424083 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c685684910 con 0x55c6857846c0 2024-09-13T18:26:35.700 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.690+0000 7fafe5ab3640 1 --2- 172.21.15.114:0/1337424083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c685779e20 0x55c685784180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:26:35.700 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.690+0000 7fafe62b4640 1 --2- 172.21.15.114:0/572566190 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fafcc0a12a0 0x7fafcc0a3760 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fafdc00bbe0 tx=0x7fafdc00b390 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:35.700 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.690+0000 7fafe6ab5640 1 --2- 172.21.15.114:0/1337424083 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c6857846c0 0x55c685786ab0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fafd8012a20 tx=0x7fafd80099f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b47ae0958c743208 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.691+0000 7fafd4ff9640 1 -- 172.21.15.114:0/1337424083 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd8003970 con 0x55c6857846c0 2024-09-13T18:26:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.691+0000 7fafd4ff9640 1 -- 172.21.15.114:0/1337424083 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fafd8010770 con 0x55c6857846c0 2024-09-13T18:26:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.691+0000 7fafd4ff9640 1 -- 172.21.15.114:0/1337424083 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd8004290 con 0x55c6857846c0 2024-09-13T18:26:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.691+0000 7fafeb238740 1 -- 172.21.15.114:0/1337424083 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c6857846c0 msgr2=0x55c685786ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.691+0000 7fafeb238740 1 --2- 172.21.15.114:0/1337424083 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c6857846c0 0x55c685786ab0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fafd8012a20 tx=0x7fafd80099f0 comp rx=0 tx=0).stop 2024-09-13T18:26:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.691+0000 7fafeb238740 1 -- 172.21.15.114:0/1337424083 shutdown_connections 2024-09-13T18:26:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.691+0000 7fafeb238740 1 --2- 172.21.15.114:0/1337424083 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c6857846c0 0x55c685786ab0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.691+0000 7fafeb238740 1 --2- 172.21.15.114:0/1337424083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c685779e20 0x55c685784180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.691+0000 7fafeb238740 1 --2- 172.21.15.114:0/1337424083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c685779500 0x55c6857798e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.691+0000 7fafeb238740 1 -- 172.21.15.114:0/1337424083 >> 172.21.15.114:0/1337424083 conn(0x55c685775a90 msgr2=0x55c685775e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.691+0000 7fafeb238740 1 -- 172.21.15.114:0/1337424083 shutdown_connections 2024-09-13T18:26:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.691+0000 7fafeb238740 1 -- 172.21.15.114:0/1337424083 wait complete. 2024-09-13T18:26:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.692+0000 7fafeb238740 1 -- start start 2024-09-13T18:26:35.702 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.692+0000 7fafeb238740 1 Processor -- start 2024-09-13T18:26:35.702 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.692+0000 7fafeb238740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c685779500 0x55c685784140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:35.702 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.692+0000 7fafeb238740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c685779e20 0x55c685781430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:35.702 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.692+0000 7fafeb238740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c6857846c0 0x55c685781b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:35.702 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.692+0000 7fafeb238740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c68569be80 con 0x55c6857846c0 2024-09-13T18:26:35.702 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.692+0000 7fafeb238740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c68569bb20 con 0x55c685779500 2024-09-13T18:26:35.702 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.692+0000 7fafeb238740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c685661260 con 0x55c685779e20 2024-09-13T18:26:35.702 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.692+0000 7fafeb238740 10 client.0 _ll_register_callbacks cb 0x55c685696df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:26:35.702 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.692+0000 7fafe6ab5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c6857846c0 0x55c685781b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:35.702 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.692+0000 7fafe6ab5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c6857846c0 0x55c685781b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37260/0 (socket says 172.21.15.114:37260) 2024-09-13T18:26:35.702 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.692+0000 7fafe6ab5640 1 -- 172.21.15.114:0/1364513737 learned_addr learned my addr 172.21.15.114:0/1364513737 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:35.702 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.692+0000 7fafe5ab3640 1 --2- 172.21.15.114:0/1364513737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c685779e20 0x55c685781430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:35.702 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.692+0000 7fafe62b4640 1 --2- 172.21.15.114:0/1364513737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c685779500 0x55c685784140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:35.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.692+0000 7fafe6ab5640 1 -- 172.21.15.114:0/1364513737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c685779e20 msgr2=0x55c685781430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:35.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.692+0000 7fafe6ab5640 1 --2- 172.21.15.114:0/1364513737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c685779e20 0x55c685781430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:35.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.692+0000 7fafe6ab5640 1 -- 172.21.15.114:0/1364513737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c685779500 msgr2=0x55c685784140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:35.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.692+0000 7fafe6ab5640 1 --2- 172.21.15.114:0/1364513737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c685779500 0x55c685784140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:35.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.692+0000 7fafe6ab5640 1 -- 172.21.15.114:0/1364513737 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafd801a040 con 0x55c6857846c0 2024-09-13T18:26:35.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.692+0000 7fafe62b4640 1 --2- 172.21.15.114:0/1364513737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c685779500 0x55c685784140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:26:35.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.693+0000 7fafe6ab5640 1 --2- 172.21.15.114:0/1364513737 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c6857846c0 0x55c685781b70 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fafd8002230 tx=0x7fafd8010b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=985fba470800955d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:35.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.693+0000 7fafc9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:26:35.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.693+0000 7fafc9ffb640 1 -- 172.21.15.114:0/1364513737 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd8003b80 con 0x55c6857846c0 2024-09-13T18:26:35.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.693+0000 7fafc9ffb640 1 -- 172.21.15.114:0/1364513737 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fafd8003ce0 con 0x55c6857846c0 2024-09-13T18:26:35.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.693+0000 7fafeb238740 1 -- 172.21.15.114:0/1364513737 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c685706680 con 0x55c6857846c0 2024-09-13T18:26:35.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.693+0000 7fafc9ffb640 1 -- 172.21.15.114:0/1364513737 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fafd8003ea0 con 0x55c6857846c0 2024-09-13T18:26:35.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.693+0000 7fafeb238740 10 client.17055 fetch_fsmap learned FSMap version 32 2024-09-13T18:26:35.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.693+0000 7fafeb238740 1 -- 172.21.15.114:0/1364513737 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55c6856dc330 con 0x55c6857846c0 2024-09-13T18:26:35.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.693+0000 7fafc9ffb640 1 -- 172.21.15.114:0/1364513737 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c685706680 con 0x55c6857846c0 2024-09-13T18:26:35.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.693+0000 7fafc9ffb640 1 -- 172.21.15.114:0/1364513737 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fafd801f570 con 0x55c6857846c0 2024-09-13T18:26:35.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.693+0000 7fafeb238740 10 client.17055 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:26:35.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.693+0000 7fafeb238740 10 client.17055 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:26:35.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.693+0000 7fafeb238740 20 client.17055 populate_metadata read hostname 'smithi114' 2024-09-13T18:26:35.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.693+0000 7fafeb238740 1 --2- 172.21.15.114:0/1364513737 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55c685794020 0x55c685796440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:35.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.693+0000 7fafeb238740 4 client.17055 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:26:35.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.693+0000 7fafeb238740 1 -- 172.21.15.114:0/1364513737 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c6856d15d0 con 0x55c685794020 2024-09-13T18:26:35.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.694+0000 7fafe62b4640 1 --2- 172.21.15.114:0/1364513737 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55c685794020 0x55c685796440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:35.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.694+0000 7fafc9ffb640 10 client.17055 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:26:35.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.694+0000 7fafe62b4640 1 --2- 172.21.15.114:0/1364513737 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55c685794020 0x55c685796440 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32ce4fc8c277a284 server_cookie=23e3c1496bbee236 in_seq=0 out_seq=0 2024-09-13T18:26:35.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.695+0000 7fafc9ffb640 1 -- 172.21.15.114:0/1364513737 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55c6856d15d0 con 0x55c685794020 2024-09-13T18:26:35.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.695+0000 7fafc9ffb640 10 client.17055 handle_command_reply: tid=0 2024-09-13T18:26:35.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.695+0000 7fafeb238740 1 client.17055 shutdown 2024-09-13T18:26:35.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.695+0000 7fafeb238740 1 -- 172.21.15.114:0/1364513737 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c6857846c0 msgr2=0x55c685781b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:35.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.695+0000 7fafeb238740 1 --2- 172.21.15.114:0/1364513737 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c6857846c0 0x55c685781b70 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fafd8002230 tx=0x7fafd8010b10 comp rx=0 tx=0).stop 2024-09-13T18:26:35.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.695+0000 7fafeb238740 1 -- 172.21.15.114:0/1364513737 shutdown_connections 2024-09-13T18:26:35.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.695+0000 7fafeb238740 1 --2- 172.21.15.114:0/1364513737 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55c685794020 0x55c685796440 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:35.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.695+0000 7fafeb238740 1 --2- 172.21.15.114:0/1364513737 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c6857846c0 0x55c685781b70 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:35.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.695+0000 7fafc9ffb640 0 client.17055 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:26:35.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.695+0000 7fafeb238740 1 --2- 172.21.15.114:0/1364513737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c685779e20 0x55c685781430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:35.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.695+0000 7fafeb238740 1 --2- 172.21.15.114:0/1364513737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c685779500 0x55c685784140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:35.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.695+0000 7fafeb238740 1 -- 172.21.15.114:0/1364513737 >> 172.21.15.114:0/1364513737 conn(0x55c685775a90 msgr2=0x55c685774d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:35.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.695+0000 7fafeb238740 1 -- 172.21.15.114:0/1364513737 shutdown_connections 2024-09-13T18:26:35.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.695+0000 7fafeb238740 1 -- 172.21.15.114:0/1364513737 wait complete. 2024-09-13T18:26:35.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.695+0000 7fafeb238740 20 client.17055 trim_cache size 0 max 16384 2024-09-13T18:26:35.714 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.708+0000 7fafeb238740 1 Processor -- start 2024-09-13T18:26:35.714 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.708+0000 7fafeb238740 1 -- start start 2024-09-13T18:26:35.714 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.708+0000 7fafeb238740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c685779500 0x55c685775550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:35.714 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.708+0000 7fafeb238740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c685779e20 0x55c685770c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:35.715 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.708+0000 7fafeb238740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6857846c0 0x55c6857711a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:35.715 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.708+0000 7fafeb238740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6856d15d0 con 0x55c685779500 2024-09-13T18:26:35.715 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.708+0000 7fafeb238740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c685706680 con 0x55c6857846c0 2024-09-13T18:26:35.715 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.708+0000 7fafeb238740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c685779c60 con 0x55c685779e20 2024-09-13T18:26:35.715 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.708+0000 7fafe62b4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c685779500 0x55c685775550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:35.715 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.708+0000 7fafe62b4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c685779500 0x55c685775550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37270/0 (socket says 172.21.15.114:37270) 2024-09-13T18:26:35.715 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.708+0000 7fafe62b4640 1 -- 172.21.15.114:0/1184023697 learned_addr learned my addr 172.21.15.114:0/1184023697 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:35.715 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.708+0000 7fafe6ab5640 1 --2- 172.21.15.114:0/1184023697 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6857846c0 0x55c6857711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:35.715 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.708+0000 7fafe62b4640 1 -- 172.21.15.114:0/1184023697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c685779e20 msgr2=0x55c685770c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:35.715 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.708+0000 7fafe5ab3640 1 --2- 172.21.15.114:0/1184023697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c685779e20 0x55c685770c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:35.715 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.708+0000 7fafe62b4640 1 --2- 172.21.15.114:0/1184023697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c685779e20 0x55c685770c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:35.715 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.708+0000 7fafe62b4640 1 -- 172.21.15.114:0/1184023697 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6857846c0 msgr2=0x55c6857711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:35.715 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.708+0000 7fafe62b4640 1 --2- 172.21.15.114:0/1184023697 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6857846c0 0x55c6857711a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.708+0000 7fafe62b4640 1 -- 172.21.15.114:0/1184023697 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c68567e300 con 0x55c685779500 2024-09-13T18:26:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.708+0000 7fafe6ab5640 1 --2- 172.21.15.114:0/1184023697 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6857846c0 0x55c6857711a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:26:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.708+0000 7fafe62b4640 1 --2- 172.21.15.114:0/1184023697 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c685779500 0x55c685775550 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fafdc010860 tx=0x7fafdc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62fc33c3ab65aeca server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.708+0000 7fafd57fa640 1 -- 172.21.15.114:0/1184023697 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafdc023040 con 0x55c685779500 2024-09-13T18:26:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.709+0000 7fafd57fa640 1 -- 172.21.15.114:0/1184023697 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fafdc015070 con 0x55c685779500 2024-09-13T18:26:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.709+0000 7fafd57fa640 1 -- 172.21.15.114:0/1184023697 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafdc01e680 con 0x55c685779500 2024-09-13T18:26:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.709+0000 7fafeb238740 1 -- 172.21.15.114:0/1184023697 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c685779500 msgr2=0x55c685775550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.709+0000 7fafeb238740 1 --2- 172.21.15.114:0/1184023697 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c685779500 0x55c685775550 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fafdc010860 tx=0x7fafdc013000 comp rx=0 tx=0).stop 2024-09-13T18:26:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.709+0000 7fafeb238740 1 -- 172.21.15.114:0/1184023697 shutdown_connections 2024-09-13T18:26:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.709+0000 7fafeb238740 1 --2- 172.21.15.114:0/1184023697 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6857846c0 0x55c6857711a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.709+0000 7fafeb238740 1 --2- 172.21.15.114:0/1184023697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c685779e20 0x55c685770c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.709+0000 7fafeb238740 1 --2- 172.21.15.114:0/1184023697 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c685779500 0x55c685775550 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.709+0000 7fafeb238740 1 -- 172.21.15.114:0/1184023697 >> 172.21.15.114:0/1184023697 conn(0x55c685775a90 msgr2=0x55c68577c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.709+0000 7fafeb238740 1 -- 172.21.15.114:0/1184023697 shutdown_connections 2024-09-13T18:26:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.709+0000 7fafeb238740 1 -- 172.21.15.114:0/1184023697 wait complete. 2024-09-13T18:26:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.710+0000 7fafeb238740 1 -- start start 2024-09-13T18:26:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.710+0000 7fafeb238740 1 Processor -- start 2024-09-13T18:26:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.710+0000 7fafeb238740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c685779500 0x55c68578b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.710+0000 7fafeb238740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c685779e20 0x55c68578b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.710+0000 7fafeb238740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6857846c0 0x55c685787b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.710+0000 7fafeb238740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c685788290 con 0x55c685779500 2024-09-13T18:26:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.710+0000 7fafeb238740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c685788400 con 0x55c685779e20 2024-09-13T18:26:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.710+0000 7fafe62b4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c685779500 0x55c68578b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.710+0000 7fafeb238740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6857885c0 con 0x55c6857846c0 2024-09-13T18:26:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.710+0000 7fafe62b4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c685779500 0x55c68578b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37284/0 (socket says 172.21.15.114:37284) 2024-09-13T18:26:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.710+0000 7fafe62b4640 1 -- 172.21.15.114:0/649901735 learned_addr learned my addr 172.21.15.114:0/649901735 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.710+0000 7fafeb238740 10 client.0 _ll_register_callbacks cb 0x7fafd8009d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:26:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.710+0000 7fafe5ab3640 1 --2- 172.21.15.114:0/649901735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c685779e20 0x55c68578b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.710+0000 7fafe6ab5640 1 --2- 172.21.15.114:0/649901735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6857846c0 0x55c685787b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.710+0000 7fafe62b4640 1 -- 172.21.15.114:0/649901735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6857846c0 msgr2=0x55c685787b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.710+0000 7fafe62b4640 1 --2- 172.21.15.114:0/649901735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6857846c0 0x55c685787b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.710+0000 7fafe62b4640 1 -- 172.21.15.114:0/649901735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c685779e20 msgr2=0x55c68578b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.710+0000 7fafe62b4640 1 --2- 172.21.15.114:0/649901735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c685779e20 0x55c68578b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.710+0000 7fafe62b4640 1 -- 172.21.15.114:0/649901735 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafdc010e20 con 0x55c685779500 2024-09-13T18:26:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.710+0000 7fafe6ab5640 1 --2- 172.21.15.114:0/649901735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6857846c0 0x55c685787b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:26:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.710+0000 7fafe5ab3640 1 --2- 172.21.15.114:0/649901735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c685779e20 0x55c68578b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:26:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.710+0000 7fafe62b4640 1 --2- 172.21.15.114:0/649901735 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c685779500 0x55c68578b330 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fafdc026420 tx=0x7fafdc026450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f870574552786 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.710+0000 7fafc9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:26:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.711+0000 7fafc9ffb640 1 -- 172.21.15.114:0/649901735 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafdc026be0 con 0x55c685779500 2024-09-13T18:26:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.711+0000 7fafc9ffb640 1 -- 172.21.15.114:0/649901735 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fafdc017070 con 0x55c685779500 2024-09-13T18:26:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.711+0000 7fafeb238740 1 -- 172.21.15.114:0/649901735 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fafd8010770 con 0x55c685779500 2024-09-13T18:26:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.711+0000 7fafc9ffb640 1 -- 172.21.15.114:0/649901735 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fafdc026d40 con 0x55c685779500 2024-09-13T18:26:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.711+0000 7fafeb238740 10 client.17061 fetch_fsmap learned FSMap version 32 2024-09-13T18:26:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.711+0000 7fafeb238740 1 -- 172.21.15.114:0/649901735 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55c6856a38e0 con 0x55c685779500 2024-09-13T18:26:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.711+0000 7fafc9ffb640 1 -- 172.21.15.114:0/649901735 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fafd8010770 con 0x55c685779500 2024-09-13T18:26:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.711+0000 7fafc9ffb640 1 -- 172.21.15.114:0/649901735 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fafdc02c020 con 0x55c685779500 2024-09-13T18:26:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.711+0000 7fafeb238740 10 client.17061 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:26:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.711+0000 7fafeb238740 10 client.17061 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:26:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.711+0000 7fafeb238740 20 client.17061 populate_metadata read hostname 'smithi114' 2024-09-13T18:26:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.711+0000 7fafeb238740 1 --2- 172.21.15.114:0/649901735 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55c685794020 0x55c6857ed8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:35.720 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.711+0000 7fafeb238740 4 client.17061 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T18:26:35.720 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.711+0000 7fafeb238740 1 -- 172.21.15.114:0/649901735 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c6853d1300 con 0x55c685794020 2024-09-13T18:26:35.720 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.711+0000 7fafe5ab3640 1 --2- 172.21.15.114:0/649901735 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55c685794020 0x55c6857ed8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:35.720 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.712+0000 7fafe5ab3640 1 --2- 172.21.15.114:0/649901735 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55c685794020 0x55c6857ed8c0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78f93beb81b698e8 server_cookie=b8f0e0c138a6c754 in_seq=0 out_seq=0 2024-09-13T18:26:35.720 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.712+0000 7fafc9ffb640 10 client.17061 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:26:35.721 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.715+0000 7fafc9ffb640 1 -- 172.21.15.114:0/649901735 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c6853d1300 con 0x55c685794020 2024-09-13T18:26:35.721 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.715+0000 7fafc9ffb640 10 client.17061 handle_command_reply: tid=0 2024-09-13T18:26:35.721 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.715+0000 7fafeb238740 1 client.17061 shutdown 2024-09-13T18:26:35.721 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.715+0000 7fafeb238740 1 -- 172.21.15.114:0/649901735 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c685779500 msgr2=0x55c68578b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:35.721 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.715+0000 7fafeb238740 1 --2- 172.21.15.114:0/649901735 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c685779500 0x55c68578b330 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fafdc026420 tx=0x7fafdc026450 comp rx=0 tx=0).stop 2024-09-13T18:26:35.722 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.715+0000 7fafeb238740 1 -- 172.21.15.114:0/649901735 shutdown_connections 2024-09-13T18:26:35.722 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.715+0000 7fafeb238740 1 --2- 172.21.15.114:0/649901735 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55c685794020 0x55c6857ed8c0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:35.722 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.715+0000 7fafeb238740 1 --2- 172.21.15.114:0/649901735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6857846c0 0x55c685787b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:35.722 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.715+0000 7fafc9ffb640 0 client.17061 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:26:35.722 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.715+0000 7fafeb238740 1 --2- 172.21.15.114:0/649901735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c685779e20 0x55c68578b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:35.722 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.715+0000 7fafeb238740 1 --2- 172.21.15.114:0/649901735 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c685779500 0x55c68578b330 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:35.722 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.715+0000 7fafeb238740 1 -- 172.21.15.114:0/649901735 >> 172.21.15.114:0/649901735 conn(0x55c685775a90 msgr2=0x55c68578fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:35.722 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.715+0000 7fafeb238740 1 -- 172.21.15.114:0/649901735 shutdown_connections 2024-09-13T18:26:35.722 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.715+0000 7fafeb238740 1 -- 172.21.15.114:0/649901735 wait complete. 2024-09-13T18:26:35.722 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.715+0000 7fafeb238740 20 client.17061 trim_cache size 0 max 16384 2024-09-13T18:26:35.723 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-13T18:26:35.723 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-13T18:26:35.723 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-13T18:26:35.723 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-13T18:26:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.715+0000 7fafe72b6640 1 -- 172.21.15.114:0/572566190 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fafcc0a12a0 msgr2=0x7fafcc0a3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.715+0000 7fafe72b6640 1 --2- 172.21.15.114:0/572566190 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fafcc0a12a0 0x7fafcc0a3760 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fafdc00bbe0 tx=0x7fafdc00b390 comp rx=0 tx=0).stop 2024-09-13T18:26:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.715+0000 7fafe72b6640 1 -- 172.21.15.114:0/572566190 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fafe0101630 msgr2=0x7fafe01963a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.715+0000 7fafe72b6640 1 --2- 172.21.15.114:0/572566190 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fafe0101630 0x7fafe01963a0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fafd00057c0 tx=0x7fafd0032c20 comp rx=0 tx=0).stop 2024-09-13T18:26:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.715+0000 7fafe72b6640 1 -- 172.21.15.114:0/572566190 shutdown_connections 2024-09-13T18:26:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.715+0000 7fafe72b6640 1 --2- 172.21.15.114:0/572566190 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fafcc0a12a0 0x7fafcc0a3760 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.715+0000 7fafe72b6640 1 --2- 172.21.15.114:0/572566190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafe010a0e0 0x7fafe019a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.715+0000 7fafe72b6640 1 --2- 172.21.15.114:0/572566190 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fafe0101630 0x7fafe01963a0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.715+0000 7fafe72b6640 1 --2- 172.21.15.114:0/572566190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafe0100d10 0x7fafe0195e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.715+0000 7fafe72b6640 1 -- 172.21.15.114:0/572566190 >> 172.21.15.114:0/572566190 conn(0x7fafe00759b0 msgr2=0x7fafe00fb310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.716+0000 7fafe72b6640 1 -- 172.21.15.114:0/572566190 shutdown_connections 2024-09-13T18:26:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:35.716+0000 7fafe72b6640 1 -- 172.21.15.114:0/572566190 wait complete. 2024-09-13T18:26:35.725 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:26:35 smithi114 ceph-mon[30897]: pgmap v126: 129 pgs: 129 active+clean; 15 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 58 MiB/s wr, 945 op/s 2024-09-13T18:26:37.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:26:37 smithi134 ceph-mon[34283]: pgmap v127: 129 pgs: 129 active+clean; 15 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 47 MiB/s wr, 757 op/s 2024-09-13T18:26:37.700 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:26:37 smithi204 ceph-mon[34777]: pgmap v127: 129 pgs: 129 active+clean; 15 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 47 MiB/s wr, 757 op/s 2024-09-13T18:26:37.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:26:37 smithi114 ceph-mon[30897]: pgmap v127: 129 pgs: 129 active+clean; 15 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 47 MiB/s wr, 757 op/s 2024-09-13T18:26:39.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:26:39 smithi134 ceph-mon[34283]: pgmap v128: 129 pgs: 129 active+clean; 15 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 42 MiB/s wr, 676 op/s 2024-09-13T18:26:39.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:26:39 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:26:39.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:26:39 smithi204 ceph-mon[34777]: pgmap v128: 129 pgs: 129 active+clean; 15 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 42 MiB/s wr, 676 op/s 2024-09-13T18:26:39.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:26:39 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:26:39.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:26:39 smithi114 ceph-mon[30897]: pgmap v128: 129 pgs: 129 active+clean; 15 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 42 MiB/s wr, 676 op/s 2024-09-13T18:26:39.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:26:39 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:26:40.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:26:40 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:26:40.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:26:40 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:26:40.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:26:40 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:26:40.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:26:40 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:26:40.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:26:40 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:26:40.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:26:40 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:26:40.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:26:40 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:26:40.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:26:40 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:26:40.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:26:40 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:26:42.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:26:42 smithi134 ceph-mon[34283]: pgmap v129: 129 pgs: 129 active+clean; 15 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 63 MiB/s wr, 1.02k op/s 2024-09-13T18:26:42.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:26:42 smithi134 ceph-mon[34283]: scrub summary: idle 2024-09-13T18:26:42.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:26:42 smithi134 ceph-mon[34283]: scrub summary: idle 2024-09-13T18:26:42.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:26:42 smithi204 ceph-mon[34777]: pgmap v129: 129 pgs: 129 active+clean; 15 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 63 MiB/s wr, 1.02k op/s 2024-09-13T18:26:42.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:26:42 smithi204 ceph-mon[34777]: scrub summary: idle 2024-09-13T18:26:42.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:26:42 smithi204 ceph-mon[34777]: scrub summary: idle 2024-09-13T18:26:42.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:26:42 smithi114 ceph-mon[30897]: pgmap v129: 129 pgs: 129 active+clean; 15 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 63 MiB/s wr, 1.02k op/s 2024-09-13T18:26:42.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:26:42 smithi114 ceph-mon[30897]: scrub summary: idle 2024-09-13T18:26:42.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:26:42 smithi114 ceph-mon[30897]: scrub summary: idle 2024-09-13T18:26:43.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:26:43 smithi134 ceph-mon[34283]: scrub summary: idle 2024-09-13T18:26:43.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:26:43 smithi204 ceph-mon[34777]: scrub summary: idle 2024-09-13T18:26:43.469 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:26:43 smithi114 ceph-mon[30897]: scrub summary: idle 2024-09-13T18:26:44.368 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:26:44 smithi134 ceph-mon[34283]: pgmap v130: 129 pgs: 129 active+clean; 15 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 42 MiB/s wr, 681 op/s 2024-09-13T18:26:44.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:26:44 smithi204 ceph-mon[34777]: pgmap v130: 129 pgs: 129 active+clean; 15 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 42 MiB/s wr, 681 op/s 2024-09-13T18:26:44.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:26:44 smithi114 ceph-mon[30897]: pgmap v130: 129 pgs: 129 active+clean; 15 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 42 MiB/s wr, 681 op/s 2024-09-13T18:26:46.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:26:46 smithi134 ceph-mon[34283]: pgmap v131: 129 pgs: 129 active+clean; 15 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 58 MiB/s wr, 944 op/s 2024-09-13T18:26:46.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:26:46 smithi204 ceph-mon[34777]: pgmap v131: 129 pgs: 129 active+clean; 15 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 58 MiB/s wr, 944 op/s 2024-09-13T18:26:46.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:26:46 smithi114 ceph-mon[30897]: pgmap v131: 129 pgs: 129 active+clean; 15 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 58 MiB/s wr, 944 op/s 2024-09-13T18:26:48.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:26:48 smithi134 ceph-mon[34283]: pgmap v132: 129 pgs: 129 active+clean; 15 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 47 MiB/s wr, 763 op/s 2024-09-13T18:26:48.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:26:48 smithi204 ceph-mon[34777]: pgmap v132: 129 pgs: 129 active+clean; 15 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 47 MiB/s wr, 763 op/s 2024-09-13T18:26:48.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:26:48 smithi114 ceph-mon[30897]: pgmap v132: 129 pgs: 129 active+clean; 15 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 47 MiB/s wr, 763 op/s 2024-09-13T18:26:50.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.287+0000 7f7577c3a640 1 -- 172.21.15.114:0/63848059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f75701074b0 msgr2=0x7f757010e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:50.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.287+0000 7f7577c3a640 1 --2- 172.21.15.114:0/63848059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f75701074b0 0x7f757010e510 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f756c009900 tx=0x7f756c02a3f0 comp rx=0 tx=0).stop 2024-09-13T18:26:50.291 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.288+0000 7f7577c3a640 1 -- 172.21.15.114:0/63848059 shutdown_connections 2024-09-13T18:26:50.291 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.288+0000 7f7577c3a640 1 --2- 172.21.15.114:0/63848059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7570119400 0x7f757011b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:50.291 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.288+0000 7f7577c3a640 1 --2- 172.21.15.114:0/63848059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7570107890 0x7f7570107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:50.291 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.288+0000 7f7577c3a640 1 --2- 172.21.15.114:0/63848059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f75701074b0 0x7f757010e510 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:50.291 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.288+0000 7f7577c3a640 1 -- 172.21.15.114:0/63848059 >> 172.21.15.114:0/63848059 conn(0x7f757006af50 msgr2=0x7f757006b360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:50.291 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.289+0000 7f7577c3a640 1 -- 172.21.15.114:0/63848059 shutdown_connections 2024-09-13T18:26:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.289+0000 7f7577c3a640 1 -- 172.21.15.114:0/63848059 wait complete. 2024-09-13T18:26:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.290+0000 7f7577c3a640 1 Processor -- start 2024-09-13T18:26:50.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.290+0000 7f7577c3a640 1 -- start start 2024-09-13T18:26:50.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.291+0000 7f7577c3a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75701074b0 0x7f7570114ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:50.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.291+0000 7f7577c3a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7570107890 0x7f75701151e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:50.295 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.291+0000 7f7577c3a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7570119400 0x7f75701a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:50.296 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.291+0000 7f7577c3a640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75701a1920 con 0x7f7570107890 2024-09-13T18:26:50.296 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.291+0000 7f7577c3a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75701a1a90 con 0x7f75701074b0 2024-09-13T18:26:50.296 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.291+0000 7f7577c3a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75701a1c50 con 0x7f7570119400 2024-09-13T18:26:50.296 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.291+0000 7f75761d7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7570119400 0x7f75701a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:50.296 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.291+0000 7f75761d7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7570119400 0x7f75701a1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:56380/0 (socket says 172.21.15.114:56380) 2024-09-13T18:26:50.296 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.291+0000 7f75761d7640 1 -- 172.21.15.114:0/841329538 learned_addr learned my addr 172.21.15.114:0/841329538 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:50.296 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.291+0000 7f75759d6640 1 --2- 172.21.15.114:0/841329538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75701074b0 0x7f7570114ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:50.296 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.291+0000 7f75761d7640 1 -- 172.21.15.114:0/841329538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75701074b0 msgr2=0x7f7570114ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:50.296 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.291+0000 7f75761d7640 1 --2- 172.21.15.114:0/841329538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75701074b0 0x7f7570114ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:50.296 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.291+0000 7f75761d7640 1 -- 172.21.15.114:0/841329538 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7570107890 msgr2=0x7f75701151e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:50.296 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.291+0000 7f75761d7640 1 --2- 172.21.15.114:0/841329538 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7570107890 0x7f75701151e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:50.296 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.291+0000 7f75761d7640 1 -- 172.21.15.114:0/841329538 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f756c009560 con 0x7f7570119400 2024-09-13T18:26:50.296 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.292+0000 7f75761d7640 1 --2- 172.21.15.114:0/841329538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7570119400 0x7f75701a1190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7560009530 tx=0x7f75600099f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.292+0000 7f7566ffd640 1 -- 172.21.15.114:0/841329538 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7560004230 con 0x7f7570119400 2024-09-13T18:26:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.293+0000 7f7577c3a640 1 -- 172.21.15.114:0/841329538 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75701a1f30 con 0x7f7570119400 2024-09-13T18:26:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.293+0000 7f7577c3a640 1 -- 172.21.15.114:0/841329538 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75701a95b0 con 0x7f7570119400 2024-09-13T18:26:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.293+0000 7f7566ffd640 1 -- 172.21.15.114:0/841329538 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f75600047a0 con 0x7f7570119400 2024-09-13T18:26:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.293+0000 7f7566ffd640 1 -- 172.21.15.114:0/841329538 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756001fb90 con 0x7f7570119400 2024-09-13T18:26:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.296+0000 7f757aa3b740 1 Processor -- start 2024-09-13T18:26:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.298+0000 7f7566ffd640 1 -- 172.21.15.114:0/841329538 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f756001fcf0 con 0x7f7570119400 2024-09-13T18:26:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.298+0000 7f757aa3b740 1 -- start start 2024-09-13T18:26:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.298+0000 7f757aa3b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609fd9da330 0x5609fd9da710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.298+0000 7f757aa3b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5609fd9dac50 0x5609fd9e4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.299+0000 7f757aa3b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609fd9e54f0 0x5609fd9e78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.299+0000 7f757aa3b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609fd945c30 con 0x5609fd9dac50 2024-09-13T18:26:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.299+0000 7f757aa3b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609fd9433c0 con 0x5609fd9e54f0 2024-09-13T18:26:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.299+0000 7f757aa3b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609fd93ecb0 con 0x5609fd9da330 2024-09-13T18:26:50.302 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.299+0000 7f7566ffd640 1 --2- 172.21.15.114:0/841329538 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f755c0a1410 0x7f755c0a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:50.302 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.300+0000 7f75761d7640 1 --2- 172.21.15.114:0/841329538 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f755c0a1410 0x7f755c0a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.300+0000 7f75759d6640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5609fd9dac50 0x5609fd9e4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.300+0000 7f75759d6640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5609fd9dac50 0x5609fd9e4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:53498/0 (socket says 172.21.15.114:53498) 2024-09-13T18:26:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.300+0000 7f75759d6640 1 -- 172.21.15.114:0/929372568 learned_addr learned my addr 172.21.15.114:0/929372568 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.300+0000 7f75751d5640 1 --2- 172.21.15.114:0/929372568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609fd9da330 0x5609fd9da710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.300+0000 7f75751d5640 1 --2- 172.21.15.114:0/929372568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609fd9e54f0 0x5609fd9e78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.300+0000 7f75759d6640 1 -- 172.21.15.114:0/929372568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609fd9da330 msgr2=0x5609fd9da710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.300+0000 7f75759d6640 1 --2- 172.21.15.114:0/929372568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609fd9da330 0x5609fd9da710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.300+0000 7f75759d6640 1 -- 172.21.15.114:0/929372568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609fd9e54f0 msgr2=0x5609fd9e78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.300+0000 7f75759d6640 1 --2- 172.21.15.114:0/929372568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609fd9e54f0 0x5609fd9e78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.300+0000 7f75759d6640 1 -- 172.21.15.114:0/929372568 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609fd8e5910 con 0x5609fd9dac50 2024-09-13T18:26:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.301+0000 7f7566ffd640 1 -- 172.21.15.114:0/841329538 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f7560025070 con 0x7f7570119400 2024-09-13T18:26:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.301+0000 7f75759d6640 1 --2- 172.21.15.114:0/929372568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5609fd9dac50 0x5609fd9e4fb0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f756c000c00 tx=0x7f756c0040a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3579f7f05f22772c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.301+0000 7f755bfff640 1 -- 172.21.15.114:0/929372568 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c0035f0 con 0x5609fd9dac50 2024-09-13T18:26:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.301+0000 7f755bfff640 1 -- 172.21.15.114:0/929372568 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f756c003be0 con 0x5609fd9dac50 2024-09-13T18:26:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.301+0000 7f755bfff640 1 -- 172.21.15.114:0/929372568 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c046b10 con 0x5609fd9dac50 2024-09-13T18:26:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.302+0000 7f757aa3b740 1 -- 172.21.15.114:0/929372568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5609fd9dac50 msgr2=0x5609fd9e4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.302+0000 7f757aa3b740 1 --2- 172.21.15.114:0/929372568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5609fd9dac50 0x5609fd9e4fb0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f756c000c00 tx=0x7f756c0040a0 comp rx=0 tx=0).stop 2024-09-13T18:26:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.303+0000 7f75761d7640 1 --2- 172.21.15.114:0/841329538 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f755c0a1410 0x7f755c0a38d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7560005680 tx=0x7f75600055b0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.303+0000 7f757aa3b740 1 -- 172.21.15.114:0/929372568 shutdown_connections 2024-09-13T18:26:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.303+0000 7f757aa3b740 1 --2- 172.21.15.114:0/929372568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609fd9e54f0 0x5609fd9e78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.303+0000 7f757aa3b740 1 --2- 172.21.15.114:0/929372568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5609fd9dac50 0x5609fd9e4fb0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.303+0000 7f757aa3b740 1 --2- 172.21.15.114:0/929372568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609fd9da330 0x5609fd9da710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.303+0000 7f757aa3b740 1 -- 172.21.15.114:0/929372568 >> 172.21.15.114:0/929372568 conn(0x5609fd9d68c0 msgr2=0x5609fd9d6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.304+0000 7f757aa3b740 1 -- 172.21.15.114:0/929372568 shutdown_connections 2024-09-13T18:26:50.307 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.304+0000 7f757aa3b740 1 -- 172.21.15.114:0/929372568 wait complete. 2024-09-13T18:26:50.307 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.305+0000 7f757aa3b740 1 -- start start 2024-09-13T18:26:50.307 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.305+0000 7f757aa3b740 1 Processor -- start 2024-09-13T18:26:50.308 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.305+0000 7f757aa3b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609fd9da330 0x5609fd9e4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:50.308 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.306+0000 7f757aa3b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5609fd9dac50 0x5609fd9e2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:50.308 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.306+0000 7f75751d5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609fd9da330 0x5609fd9e4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:50.308 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.306+0000 7f75751d5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609fd9da330 0x5609fd9e4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:56396/0 (socket says 172.21.15.114:56396) 2024-09-13T18:26:50.308 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.306+0000 7f75751d5640 1 -- 172.21.15.114:0/2419810795 learned_addr learned my addr 172.21.15.114:0/2419810795 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:50.309 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.306+0000 7f757aa3b740 1 --2- 172.21.15.114:0/2419810795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609fd9e54f0 0x5609fd9e26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:50.309 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.306+0000 7f757aa3b740 1 -- 172.21.15.114:0/2419810795 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609fd8fce80 con 0x5609fd9dac50 2024-09-13T18:26:50.309 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.306+0000 7f757aa3b740 1 -- 172.21.15.114:0/2419810795 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609fd8fcb20 con 0x5609fd9e54f0 2024-09-13T18:26:50.309 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.306+0000 7f757aa3b740 1 -- 172.21.15.114:0/2419810795 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609fd8c2260 con 0x5609fd9da330 2024-09-13T18:26:50.309 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.306+0000 7f75751d5640 1 --2- 172.21.15.114:0/2419810795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609fd9e54f0 0x5609fd9e26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:50.310 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.307+0000 7f75751d5640 1 -- 172.21.15.114:0/2419810795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609fd9e54f0 msgr2=0x5609fd9e26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:50.310 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.307+0000 7f75751d5640 1 --2- 172.21.15.114:0/2419810795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609fd9e54f0 0x5609fd9e26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:50.310 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.307+0000 7f75751d5640 1 -- 172.21.15.114:0/2419810795 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5609fd9dac50 msgr2=0x5609fd9e2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:50.310 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.307+0000 7f75751d5640 1 --2- 172.21.15.114:0/2419810795 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5609fd9dac50 0x5609fd9e2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:50.310 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.307+0000 7f75751d5640 1 -- 172.21.15.114:0/2419810795 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f757011c230 con 0x5609fd9da330 2024-09-13T18:26:50.310 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.307+0000 7f757aa3b740 10 client.0 _ll_register_callbacks cb 0x5609fd8f7df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:26:50.310 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.307+0000 7f75751d5640 1 --2- 172.21.15.114:0/2419810795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609fd9da330 0x5609fd9e4e80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7568005fd0 tx=0x7f7568004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e66b1313e5ba3c9f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:50.310 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.307+0000 7f757aa3b740 1 -- 172.21.15.114:0/2419810795 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5609fd967680 con 0x5609fd9da330 2024-09-13T18:26:50.311 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.309+0000 7f75597fa640 10 client.26437 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:26:50.312 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.309+0000 7f75597fa640 1 -- 172.21.15.114:0/2419810795 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75680140d0 con 0x5609fd9da330 2024-09-13T18:26:50.312 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.309+0000 7f75597fa640 1 -- 172.21.15.114:0/2419810795 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f7568014230 con 0x5609fd9da330 2024-09-13T18:26:50.312 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.309+0000 7f75597fa640 1 -- 172.21.15.114:0/2419810795 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75680143f0 con 0x5609fd9da330 2024-09-13T18:26:50.312 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.309+0000 7f757aa3b740 10 client.26437 fetch_fsmap learned FSMap version 32 2024-09-13T18:26:50.312 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.309+0000 7f757aa3b740 1 -- 172.21.15.114:0/2419810795 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({fsmap=32}) v3 -- 0x5609fd93d330 con 0x5609fd9da330 2024-09-13T18:26:50.312 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.310+0000 7f75597fa640 1 -- 172.21.15.114:0/2419810795 <== mon.1 v2:172.21.15.204:3300/0 4 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f756801e020 con 0x5609fd9da330 2024-09-13T18:26:50.313 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.310+0000 7f757aa3b740 10 client.26437 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:26:50.313 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.310+0000 7f757aa3b740 10 client.26437 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:26:50.313 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.310+0000 7f757aa3b740 20 client.26437 populate_metadata read hostname 'smithi114' 2024-09-13T18:26:50.313 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.310+0000 7f757aa3b740 1 --2- 172.21.15.114:0/2419810795 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5609fd9f4fd0 0x5609fd9f7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:50.313 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.310+0000 7f757aa3b740 4 client.26437 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:26:50.313 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.310+0000 7f757aa3b740 1 -- 172.21.15.114:0/2419810795 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5609fd9325d0 con 0x5609fd9f4fd0 2024-09-13T18:26:50.313 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.310+0000 7f75759d6640 1 --2- 172.21.15.114:0/2419810795 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5609fd9f4fd0 0x5609fd9f7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.314+0000 7f75759d6640 1 --2- 172.21.15.114:0/2419810795 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5609fd9f4fd0 0x5609fd9f7490 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd04243c41c03e10 server_cookie=241fc41c3289ac72 in_seq=0 out_seq=0 2024-09-13T18:26:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.314+0000 7f75597fa640 10 client.26437 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:26:50.319 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.316+0000 7f75597fa640 1 -- 172.21.15.114:0/2419810795 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5609fd9325d0 con 0x5609fd9f4fd0 2024-09-13T18:26:50.319 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.316+0000 7f75597fa640 10 client.26437 handle_command_reply: tid=0 2024-09-13T18:26:50.319 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.316+0000 7f757aa3b740 1 client.26437 shutdown 2024-09-13T18:26:50.319 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.316+0000 7f757aa3b740 1 -- 172.21.15.114:0/2419810795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609fd9da330 msgr2=0x5609fd9e4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:50.319 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.316+0000 7f757aa3b740 1 --2- 172.21.15.114:0/2419810795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609fd9da330 0x5609fd9e4e80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7568005fd0 tx=0x7f7568004a40 comp rx=0 tx=0).stop 2024-09-13T18:26:50.319 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.317+0000 7f757aa3b740 1 -- 172.21.15.114:0/2419810795 shutdown_connections 2024-09-13T18:26:50.319 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.317+0000 7f757aa3b740 1 --2- 172.21.15.114:0/2419810795 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5609fd9f4fd0 0x5609fd9f7490 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:50.319 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.317+0000 7f757aa3b740 1 --2- 172.21.15.114:0/2419810795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609fd9e54f0 0x5609fd9e26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:50.319 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.317+0000 7f757aa3b740 1 --2- 172.21.15.114:0/2419810795 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5609fd9dac50 0x5609fd9e2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:50.319 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.317+0000 7f757aa3b740 1 --2- 172.21.15.114:0/2419810795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609fd9da330 0x5609fd9e4e80 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:50.320 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.317+0000 7f757aa3b740 1 -- 172.21.15.114:0/2419810795 >> 172.21.15.114:0/2419810795 conn(0x5609fd9d68c0 msgr2=0x5609fd9d5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:50.320 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.317+0000 7f75597fa640 0 client.26437 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:26:50.321 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.319+0000 7f757aa3b740 1 -- 172.21.15.114:0/2419810795 shutdown_connections 2024-09-13T18:26:50.321 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.319+0000 7f757aa3b740 1 -- 172.21.15.114:0/2419810795 wait complete. 2024-09-13T18:26:50.322 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.320+0000 7f757aa3b740 20 client.26437 trim_cache size 0 max 16384 2024-09-13T18:26:50.353 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.350+0000 7f757aa3b740 1 Processor -- start 2024-09-13T18:26:50.353 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.350+0000 7f757aa3b740 1 -- start start 2024-09-13T18:26:50.354 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.351+0000 7f757aa3b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5609fd9da330 0x5609fd9e49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:50.354 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.351+0000 7f757aa3b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609fd9dac50 0x5609fd9ec250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:50.354 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.351+0000 7f757aa3b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609fd9e54f0 0x5609fd9ec790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:50.354 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.351+0000 7f757aa3b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609fd9325d0 con 0x5609fd9da330 2024-09-13T18:26:50.354 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.351+0000 7f757aa3b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609fd9f1c30 con 0x5609fd9dac50 2024-09-13T18:26:50.354 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.351+0000 7f757aa3b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609fd9db2a0 con 0x5609fd9e54f0 2024-09-13T18:26:50.354 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.351+0000 7f75751d5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5609fd9da330 0x5609fd9e49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:50.354 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.351+0000 7f75751d5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5609fd9da330 0x5609fd9e49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:53524/0 (socket says 172.21.15.114:53524) 2024-09-13T18:26:50.354 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.351+0000 7f75751d5640 1 -- 172.21.15.114:0/2352216065 learned_addr learned my addr 172.21.15.114:0/2352216065 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:50.354 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.351+0000 7f75759d6640 1 --2- 172.21.15.114:0/2352216065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609fd9dac50 0x5609fd9ec250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:50.355 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.351+0000 7f75751d5640 1 --2- 172.21.15.114:0/2352216065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609fd9e54f0 0x5609fd9ec790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:50.355 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.352+0000 7f75759d6640 1 -- 172.21.15.114:0/2352216065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609fd9e54f0 msgr2=0x5609fd9ec790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:50.355 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.352+0000 7f75759d6640 1 --2- 172.21.15.114:0/2352216065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609fd9e54f0 0x5609fd9ec790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:50.355 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.352+0000 7f75759d6640 1 -- 172.21.15.114:0/2352216065 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5609fd9da330 msgr2=0x5609fd9e49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:50.355 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.352+0000 7f75759d6640 1 --2- 172.21.15.114:0/2352216065 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5609fd9da330 0x5609fd9e49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:50.355 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.352+0000 7f75759d6640 1 -- 172.21.15.114:0/2352216065 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609fd8df300 con 0x5609fd9dac50 2024-09-13T18:26:50.357 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.352+0000 7f75759d6640 1 --2- 172.21.15.114:0/2352216065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609fd9dac50 0x5609fd9ec250 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x5609fd9e3720 tx=0x7f756c005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3014690057d6e24 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:50.357 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.352+0000 7f7564e05640 1 -- 172.21.15.114:0/2352216065 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c004350 con 0x5609fd9dac50 2024-09-13T18:26:50.357 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.352+0000 7f7564e05640 1 -- 172.21.15.114:0/2352216065 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f756c04b040 con 0x5609fd9dac50 2024-09-13T18:26:50.357 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.352+0000 7f7564e05640 1 -- 172.21.15.114:0/2352216065 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c0508c0 con 0x5609fd9dac50 2024-09-13T18:26:50.357 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.353+0000 7f757aa3b740 1 -- 172.21.15.114:0/2352216065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609fd9dac50 msgr2=0x5609fd9ec250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:50.357 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.353+0000 7f757aa3b740 1 --2- 172.21.15.114:0/2352216065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609fd9dac50 0x5609fd9ec250 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x5609fd9e3720 tx=0x7f756c005c00 comp rx=0 tx=0).stop 2024-09-13T18:26:50.357 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.353+0000 7f757aa3b740 1 -- 172.21.15.114:0/2352216065 shutdown_connections 2024-09-13T18:26:50.357 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.353+0000 7f757aa3b740 1 --2- 172.21.15.114:0/2352216065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609fd9e54f0 0x5609fd9ec790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:50.357 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.353+0000 7f757aa3b740 1 --2- 172.21.15.114:0/2352216065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609fd9dac50 0x5609fd9ec250 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:50.357 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.353+0000 7f757aa3b740 1 --2- 172.21.15.114:0/2352216065 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5609fd9da330 0x5609fd9e49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:50.357 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.353+0000 7f757aa3b740 1 -- 172.21.15.114:0/2352216065 >> 172.21.15.114:0/2352216065 conn(0x5609fd9d68c0 msgr2=0x5609fd9dd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:50.357 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.354+0000 7f757aa3b740 1 -- 172.21.15.114:0/2352216065 shutdown_connections 2024-09-13T18:26:50.358 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.354+0000 7f757aa3b740 1 -- 172.21.15.114:0/2352216065 wait complete. 2024-09-13T18:26:50.358 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.354+0000 7f757aa3b740 1 -- start start 2024-09-13T18:26:50.358 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.354+0000 7f757aa3b740 1 Processor -- start 2024-09-13T18:26:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.355+0000 7f757aa3b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609fd9da330 0x5609fd9ec450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.355+0000 7f757aa3b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609fd9dac50 0x5609fd9ea8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.355+0000 7f757aa3b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5609fd9e54f0 0x5609fd9eae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.356+0000 7f75751d5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609fd9da330 0x5609fd9ec450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.356+0000 7f75751d5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609fd9da330 0x5609fd9ec450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:56420/0 (socket says 172.21.15.114:56420) 2024-09-13T18:26:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.356+0000 7f75759d6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609fd9dac50 0x5609fd9ea8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.356+0000 7f75759d6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609fd9dac50 0x5609fd9ea8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:45772/0 (socket says 172.21.15.114:45772) 2024-09-13T18:26:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.356+0000 7f75751d5640 1 -- 172.21.15.114:0/714206653 learned_addr learned my addr 172.21.15.114:0/714206653 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:26:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.356+0000 7f757aa3b740 1 -- 172.21.15.114:0/714206653 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609fd9eb370 con 0x5609fd9e54f0 2024-09-13T18:26:50.361 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.356+0000 7f757aa3b740 1 -- 172.21.15.114:0/714206653 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609fd9eb4e0 con 0x5609fd9dac50 2024-09-13T18:26:50.361 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.356+0000 7f757aa3b740 1 -- 172.21.15.114:0/714206653 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609fd9eb6a0 con 0x5609fd9da330 2024-09-13T18:26:50.361 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.356+0000 7f75751d5640 1 --2- 172.21.15.114:0/714206653 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5609fd9e54f0 0x5609fd9eae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:50.361 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.356+0000 7f757aa3b740 10 client.0 _ll_register_callbacks cb 0x7f75680093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:26:50.361 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.356+0000 7f75751d5640 1 -- 172.21.15.114:0/714206653 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609fd9da330 msgr2=0x5609fd9ec450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:50.361 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.356+0000 7f75751d5640 1 --2- 172.21.15.114:0/714206653 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609fd9da330 0x5609fd9ec450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:50.361 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.356+0000 7f75751d5640 1 -- 172.21.15.114:0/714206653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609fd9dac50 msgr2=0x5609fd9ea8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:26:50.361 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.356+0000 7f75751d5640 1 --2- 172.21.15.114:0/714206653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609fd9dac50 0x5609fd9ea8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:50.361 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.356+0000 7f75751d5640 1 -- 172.21.15.114:0/714206653 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5609fd8e5910 con 0x5609fd9e54f0 2024-09-13T18:26:50.361 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.357+0000 7f757aa3b740 1 -- 172.21.15.114:0/714206653 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f756c003be0 con 0x5609fd9e54f0 2024-09-13T18:26:50.361 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.357+0000 7f75751d5640 1 --2- 172.21.15.114:0/714206653 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5609fd9e54f0 0x5609fd9eae00 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f7568009a50 tx=0x7f7568014f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e608dd1707cfcef server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:26:50.361 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.357+0000 7f75597fa640 10 client.17073 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:26:50.361 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.357+0000 7f75597fa640 1 -- 172.21.15.114:0/714206653 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7568014300 con 0x5609fd9e54f0 2024-09-13T18:26:50.362 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.357+0000 7f75597fa640 1 -- 172.21.15.114:0/714206653 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f756800fb30 con 0x5609fd9e54f0 2024-09-13T18:26:50.362 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.357+0000 7f75597fa640 1 -- 172.21.15.114:0/714206653 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f756800fcf0 con 0x5609fd9e54f0 2024-09-13T18:26:50.362 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.358+0000 7f757aa3b740 10 client.17073 fetch_fsmap learned FSMap version 32 2024-09-13T18:26:50.362 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.358+0000 7f757aa3b740 1 -- 172.21.15.114:0/714206653 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({fsmap=32}) v3 -- 0x5609fd9048e0 con 0x5609fd9e54f0 2024-09-13T18:26:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.358+0000 7f75597fa640 1 -- 172.21.15.114:0/714206653 <== mon.0 v2:172.21.15.114:3300/0 4 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f7568037030 con 0x5609fd9e54f0 2024-09-13T18:26:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.358+0000 7f757aa3b740 10 client.17073 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:26:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.358+0000 7f757aa3b740 10 client.17073 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:26:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.358+0000 7f757aa3b740 20 client.17073 populate_metadata read hostname 'smithi114' 2024-09-13T18:26:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.358+0000 7f757aa3b740 1 --2- 172.21.15.114:0/714206653 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5609fda4c810 0x5609fda4ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:26:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.358+0000 7f757aa3b740 4 client.17073 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T18:26:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.358+0000 7f757aa3b740 1 -- 172.21.15.114:0/714206653 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5609fd632300 con 0x5609fda4c810 2024-09-13T18:26:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.358+0000 7f75759d6640 1 --2- 172.21.15.114:0/714206653 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5609fda4c810 0x5609fda4ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:26:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.359+0000 7f75759d6640 1 --2- 172.21.15.114:0/714206653 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5609fda4c810 0x5609fda4ec30 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd4abf3097747896 server_cookie=5dd99d347c5ab58a in_seq=0 out_seq=0 2024-09-13T18:26:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.359+0000 7f75597fa640 10 client.17073 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:26:50.368 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.365+0000 7f75597fa640 1 -- 172.21.15.114:0/714206653 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5609fd632300 con 0x5609fda4c810 2024-09-13T18:26:50.368 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.365+0000 7f75597fa640 10 client.17073 handle_command_reply: tid=0 2024-09-13T18:26:50.368 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.365+0000 7f757aa3b740 1 client.17073 shutdown 2024-09-13T18:26:50.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:26:50 smithi134 ceph-mon[34283]: pgmap v133: 129 pgs: 129 active+clean; 15 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 42 MiB/s wr, 679 op/s 2024-09-13T18:26:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.366+0000 7f757aa3b740 1 -- 172.21.15.114:0/714206653 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5609fd9e54f0 msgr2=0x5609fd9eae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.366+0000 7f757aa3b740 1 --2- 172.21.15.114:0/714206653 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5609fd9e54f0 0x5609fd9eae00 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f7568009a50 tx=0x7f7568014f90 comp rx=0 tx=0).stop 2024-09-13T18:26:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.366+0000 7f757aa3b740 1 -- 172.21.15.114:0/714206653 shutdown_connections 2024-09-13T18:26:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.366+0000 7f757aa3b740 1 --2- 172.21.15.114:0/714206653 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5609fda4c810 0x5609fda4ec30 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.366+0000 7f75597fa640 0 client.17073 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:26:50.370 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.366+0000 7f757aa3b740 1 --2- 172.21.15.114:0/714206653 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5609fd9e54f0 0x5609fd9eae00 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:50.370 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.366+0000 7f757aa3b740 1 --2- 172.21.15.114:0/714206653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609fd9dac50 0x5609fd9ea8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:50.370 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.366+0000 7f757aa3b740 1 --2- 172.21.15.114:0/714206653 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609fd9da330 0x5609fd9ec450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:50.370 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.366+0000 7f757aa3b740 1 -- 172.21.15.114:0/714206653 >> 172.21.15.114:0/714206653 conn(0x5609fd9d68c0 msgr2=0x5609fd9d1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:50.370 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.366+0000 7f757aa3b740 1 -- 172.21.15.114:0/714206653 shutdown_connections 2024-09-13T18:26:50.370 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.366+0000 7f757aa3b740 1 -- 172.21.15.114:0/714206653 wait complete. 2024-09-13T18:26:50.371 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-13T18:26:50.371 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-13T18:26:50.371 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-13T18:26:50.371 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-13T18:26:50.371 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.369+0000 7f757aa3b740 20 client.17073 trim_cache size 0 max 16384 2024-09-13T18:26:50.378 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.374+0000 7f7577c3a640 1 -- 172.21.15.114:0/841329538 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f755c0a1410 msgr2=0x7f755c0a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:50.378 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.374+0000 7f7577c3a640 1 --2- 172.21.15.114:0/841329538 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f755c0a1410 0x7f755c0a38d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7560005680 tx=0x7f75600055b0 comp rx=0 tx=0).stop 2024-09-13T18:26:50.378 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.374+0000 7f7577c3a640 1 -- 172.21.15.114:0/841329538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7570119400 msgr2=0x7f75701a1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:26:50.378 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.374+0000 7f7577c3a640 1 --2- 172.21.15.114:0/841329538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7570119400 0x7f75701a1190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7560009530 tx=0x7f75600099f0 comp rx=0 tx=0).stop 2024-09-13T18:26:50.378 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.375+0000 7f7577c3a640 1 -- 172.21.15.114:0/841329538 shutdown_connections 2024-09-13T18:26:50.378 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.375+0000 7f7577c3a640 1 --2- 172.21.15.114:0/841329538 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f755c0a1410 0x7f755c0a38d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:50.378 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.375+0000 7f7577c3a640 1 --2- 172.21.15.114:0/841329538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7570119400 0x7f75701a1190 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:50.378 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.375+0000 7f7577c3a640 1 --2- 172.21.15.114:0/841329538 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7570107890 0x7f75701151e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:50.378 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.375+0000 7f7577c3a640 1 --2- 172.21.15.114:0/841329538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75701074b0 0x7f7570114ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:26:50.378 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.375+0000 7f7577c3a640 1 -- 172.21.15.114:0/841329538 >> 172.21.15.114:0/841329538 conn(0x7f757006af50 msgr2=0x7f7570070300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:26:50.378 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.375+0000 7f7577c3a640 1 -- 172.21.15.114:0/841329538 shutdown_connections 2024-09-13T18:26:50.379 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:26:50.375+0000 7f7577c3a640 1 -- 172.21.15.114:0/841329538 wait complete. 2024-09-13T18:26:50.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:26:50 smithi204 ceph-mon[34777]: pgmap v133: 129 pgs: 129 active+clean; 15 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 42 MiB/s wr, 679 op/s 2024-09-13T18:26:50.473 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:26:50 smithi114 ceph-mon[30897]: pgmap v133: 129 pgs: 129 active+clean; 15 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 42 MiB/s wr, 679 op/s 2024-09-13T18:26:52.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:26:52 smithi134 ceph-mon[34283]: pgmap v134: 129 pgs: 129 active+clean; 15 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 62 MiB/s wr, 1.01k op/s 2024-09-13T18:26:52.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:26:52 smithi204 ceph-mon[34777]: pgmap v134: 129 pgs: 129 active+clean; 15 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 62 MiB/s wr, 1.01k op/s 2024-09-13T18:26:52.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:26:52 smithi114 ceph-mon[30897]: pgmap v134: 129 pgs: 129 active+clean; 15 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 62 MiB/s wr, 1.01k op/s 2024-09-13T18:26:54.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:26:54 smithi134 ceph-mon[34283]: pgmap v135: 129 pgs: 129 active+clean; 15 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 41 MiB/s wr, 668 op/s 2024-09-13T18:26:54.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:26:54 smithi204 ceph-mon[34777]: pgmap v135: 129 pgs: 129 active+clean; 15 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 41 MiB/s wr, 668 op/s 2024-09-13T18:26:54.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:26:54 smithi114 ceph-mon[30897]: pgmap v135: 129 pgs: 129 active+clean; 15 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 41 MiB/s wr, 668 op/s 2024-09-13T18:26:56.368 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:26:56 smithi134 ceph-mon[34283]: pgmap v136: 129 pgs: 129 active+clean; 15 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 56 MiB/s wr, 903 op/s 2024-09-13T18:26:56.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:26:56 smithi204 ceph-mon[34777]: pgmap v136: 129 pgs: 129 active+clean; 15 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 56 MiB/s wr, 903 op/s 2024-09-13T18:26:56.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:26:56 smithi114 ceph-mon[30897]: pgmap v136: 129 pgs: 129 active+clean; 15 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 56 MiB/s wr, 903 op/s 2024-09-13T18:26:58.368 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:26:58 smithi134 ceph-mon[34283]: pgmap v137: 129 pgs: 129 active+clean; 15 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 46 MiB/s wr, 739 op/s 2024-09-13T18:26:58.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:26:58 smithi204 ceph-mon[34777]: pgmap v137: 129 pgs: 129 active+clean; 15 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 46 MiB/s wr, 739 op/s 2024-09-13T18:26:58.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:26:58 smithi114 ceph-mon[30897]: pgmap v137: 129 pgs: 129 active+clean; 15 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 46 MiB/s wr, 739 op/s 2024-09-13T18:27:00.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:00 smithi134 ceph-mon[34283]: pgmap v138: 129 pgs: 129 active+clean; 15 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 41 MiB/s wr, 664 op/s 2024-09-13T18:27:00.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:00 smithi204 ceph-mon[34777]: pgmap v138: 129 pgs: 129 active+clean; 15 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 41 MiB/s wr, 664 op/s 2024-09-13T18:27:00.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:00 smithi114 ceph-mon[30897]: pgmap v138: 129 pgs: 129 active+clean; 15 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 41 MiB/s wr, 664 op/s 2024-09-13T18:27:01.647 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T18:27:01.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.920+0000 7f313759e640 1 -- 172.21.15.114:0/2524916830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31381074b0 msgr2=0x7f3138107920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:01.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.920+0000 7f313759e640 1 --2- 172.21.15.114:0/2524916830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31381074b0 0x7f3138107920 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3130009ca0 tx=0x7f313002a8c0 comp rx=0 tx=0).stop 2024-09-13T18:27:01.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.923+0000 7f313759e640 1 -- 172.21.15.114:0/2524916830 shutdown_connections 2024-09-13T18:27:01.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.923+0000 7f313759e640 1 --2- 172.21.15.114:0/2524916830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31381074b0 0x7f3138107920 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:01.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.923+0000 7f313759e640 1 --2- 172.21.15.114:0/2524916830 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3138108ee0 0x7f3138111630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:01.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.924+0000 7f313759e640 1 --2- 172.21.15.114:0/2524916830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31381085c0 0x7f31381089a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:01.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.924+0000 7f313759e640 1 -- 172.21.15.114:0/2524916830 >> 172.21.15.114:0/2524916830 conn(0x7f313806b390 msgr2=0x7f313806b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:01.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.930+0000 7f313759e640 1 -- 172.21.15.114:0/2524916830 shutdown_connections 2024-09-13T18:27:01.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.931+0000 7f313759e640 1 -- 172.21.15.114:0/2524916830 wait complete. 2024-09-13T18:27:01.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.932+0000 7f313759e640 1 Processor -- start 2024-09-13T18:27:01.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.934+0000 7f313759e640 1 -- start start 2024-09-13T18:27:01.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.934+0000 7f313759e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31381074b0 0x7f3138111c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:01.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.934+0000 7f313759e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31381085c0 0x7f3138112180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:01.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.934+0000 7f313759e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3138108ee0 0x7f31381126c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:01.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.934+0000 7f313759e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3138113bf0 con 0x7f31381085c0 2024-09-13T18:27:01.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.934+0000 7f313759e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31381a29c0 con 0x7f31381074b0 2024-09-13T18:27:01.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.934+0000 7f313759e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31381a2b80 con 0x7f3138108ee0 2024-09-13T18:27:01.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.935+0000 7f313659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31381074b0 0x7f3138111c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:01.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.935+0000 7f313659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31381074b0 0x7f3138111c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:37026/0 (socket says 172.21.15.114:37026) 2024-09-13T18:27:01.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.935+0000 7f313659c640 1 -- 172.21.15.114:0/3347013646 learned_addr learned my addr 172.21.15.114:0/3347013646 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:01.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.936+0000 7f313659c640 1 -- 172.21.15.114:0/3347013646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3138108ee0 msgr2=0x7f31381126c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:27:01.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.936+0000 7f313659c640 1 --2- 172.21.15.114:0/3347013646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3138108ee0 0x7f31381126c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:01.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.936+0000 7f313659c640 1 -- 172.21.15.114:0/3347013646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31381085c0 msgr2=0x7f3138112180 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:27:01.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.936+0000 7f313659c640 1 --2- 172.21.15.114:0/3347013646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31381085c0 0x7f3138112180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:01.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.936+0000 7f313659c640 1 -- 172.21.15.114:0/3347013646 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3130009900 con 0x7f31381074b0 2024-09-13T18:27:01.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.937+0000 7f313659c640 1 --2- 172.21.15.114:0/3347013646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31381074b0 0x7f3138111c40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f312800e9f0 tx=0x7f312800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:01.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.937+0000 7f31277fe640 1 -- 172.21.15.114:0/3347013646 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312800cd80 con 0x7f31381074b0 2024-09-13T18:27:01.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.941+0000 7f31277fe640 1 -- 172.21.15.114:0/3347013646 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f31280054b0 con 0x7f31381074b0 2024-09-13T18:27:01.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.941+0000 7f31277fe640 1 -- 172.21.15.114:0/3347013646 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3128010430 con 0x7f31381074b0 2024-09-13T18:27:01.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.941+0000 7f313759e640 1 -- 172.21.15.114:0/3347013646 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31381a2e60 con 0x7f31381074b0 2024-09-13T18:27:01.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.941+0000 7f313759e640 1 -- 172.21.15.114:0/3347013646 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31381a3320 con 0x7f31381074b0 2024-09-13T18:27:01.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.946+0000 7f31277fe640 1 -- 172.21.15.114:0/3347013646 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f3128010590 con 0x7f31381074b0 2024-09-13T18:27:01.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.947+0000 7f31277fe640 1 --2- 172.21.15.114:0/3347013646 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f31200a1410 0x7f31200a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:01.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.947+0000 7f31277fe640 1 -- 172.21.15.114:0/3347013646 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f3128014070 con 0x7f31381074b0 2024-09-13T18:27:01.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.948+0000 7f3135d9b640 1 --2- 172.21.15.114:0/3347013646 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f31200a1410 0x7f31200a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:01.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.949+0000 7f3135d9b640 1 --2- 172.21.15.114:0/3347013646 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f31200a1410 0x7f31200a38d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f31381133c0 tx=0x7f312c005f70 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:01.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.949+0000 7f313f895740 1 Processor -- start 2024-09-13T18:27:01.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.949+0000 7f313f895740 1 -- start start 2024-09-13T18:27:01.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.950+0000 7f313f895740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565126041fe0 0x5651260423c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:01.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.950+0000 7f313f895740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565126042900 0x56512604cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:01.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.950+0000 7f313f895740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56512604d1a0 0x56512604f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:01.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.950+0000 7f313f895740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x565125facf20 con 0x56512604d1a0 2024-09-13T18:27:01.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.950+0000 7f313f895740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x565125faa6b0 con 0x565126041fe0 2024-09-13T18:27:01.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.950+0000 7f313f895740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x565125fa5fa0 con 0x565126042900 2024-09-13T18:27:01.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.950+0000 7f313659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565126041fe0 0x5651260423c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:01.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.950+0000 7f313659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565126041fe0 0x5651260423c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:37034/0 (socket says 172.21.15.114:37034) 2024-09-13T18:27:01.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.950+0000 7f313659c640 1 -- 172.21.15.114:0/3287874340 learned_addr learned my addr 172.21.15.114:0/3287874340 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:01.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.950+0000 7f3136d9d640 1 --2- 172.21.15.114:0/3287874340 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56512604d1a0 0x56512604f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:01.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.950+0000 7f313659c640 1 -- 172.21.15.114:0/3287874340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565126042900 msgr2=0x56512604cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:01.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.950+0000 7f313659c640 1 --2- 172.21.15.114:0/3287874340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565126042900 0x56512604cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:01.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.950+0000 7f313659c640 1 -- 172.21.15.114:0/3287874340 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56512604d1a0 msgr2=0x56512604f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:01.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.950+0000 7f313659c640 1 --2- 172.21.15.114:0/3287874340 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56512604d1a0 0x56512604f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:01.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.950+0000 7f313659c640 1 -- 172.21.15.114:0/3287874340 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565125f60840 con 0x565126041fe0 2024-09-13T18:27:01.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.951+0000 7f313659c640 1 --2- 172.21.15.114:0/3287874340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565126041fe0 0x5651260423c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3128004a40 tx=0x7f312809ae20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a847e1056cf2eb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:01.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.951+0000 7f3124ff9640 1 -- 172.21.15.114:0/3287874340 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312809b2e0 con 0x565126041fe0 2024-09-13T18:27:01.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.951+0000 7f3124ff9640 1 -- 172.21.15.114:0/3287874340 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f31280b3e90 con 0x565126041fe0 2024-09-13T18:27:01.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.952+0000 7f313f895740 1 -- 172.21.15.114:0/3287874340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565126041fe0 msgr2=0x5651260423c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:01.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.952+0000 7f313f895740 1 --2- 172.21.15.114:0/3287874340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565126041fe0 0x5651260423c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3128004a40 tx=0x7f312809ae20 comp rx=0 tx=0).stop 2024-09-13T18:27:01.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.953+0000 7f313f895740 1 -- 172.21.15.114:0/3287874340 shutdown_connections 2024-09-13T18:27:01.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.953+0000 7f313f895740 1 --2- 172.21.15.114:0/3287874340 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56512604d1a0 0x56512604f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:01.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.953+0000 7f313f895740 1 --2- 172.21.15.114:0/3287874340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565126042900 0x56512604cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:01.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.953+0000 7f313f895740 1 --2- 172.21.15.114:0/3287874340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565126041fe0 0x5651260423c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:01.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.953+0000 7f313f895740 1 -- 172.21.15.114:0/3287874340 >> 172.21.15.114:0/3287874340 conn(0x56512603e570 msgr2=0x56512603e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:01.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.956+0000 7f313f895740 1 -- 172.21.15.114:0/3287874340 shutdown_connections 2024-09-13T18:27:01.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.956+0000 7f313f895740 1 -- 172.21.15.114:0/3287874340 wait complete. 2024-09-13T18:27:01.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.957+0000 7f313f895740 1 -- start start 2024-09-13T18:27:01.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.957+0000 7f313f895740 1 Processor -- start 2024-09-13T18:27:01.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.957+0000 7f313f895740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565126042900 0x56512604c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:01.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.957+0000 7f313f895740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56512604d1a0 0x565126049b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:01.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.957+0000 7f313f895740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56512604a060 0x56512604a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:01.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.957+0000 7f313f895740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x565125f472a0 con 0x56512604d1a0 2024-09-13T18:27:01.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.957+0000 7f313f895740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x565125c7e430 con 0x565126042900 2024-09-13T18:27:01.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.957+0000 7f313f895740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x565125f29680 con 0x56512604a060 2024-09-13T18:27:01.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.957+0000 7f313f895740 10 client.0 _ll_register_callbacks cb 0x565125f9f610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:27:01.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.958+0000 7f3135d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56512604d1a0 0x565126049b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:01.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.958+0000 7f3136d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56512604a060 0x56512604a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:01.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.958+0000 7f3135d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56512604d1a0 0x565126049b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34834/0 (socket says 172.21.15.114:34834) 2024-09-13T18:27:01.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.958+0000 7f3135d9b640 1 -- 172.21.15.114:0/2900796266 learned_addr learned my addr 172.21.15.114:0/2900796266 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:01.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.958+0000 7f313659c640 1 --2- 172.21.15.114:0/2900796266 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565126042900 0x56512604c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:01.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.958+0000 7f3135d9b640 1 -- 172.21.15.114:0/2900796266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56512604a060 msgr2=0x56512604a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:01.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.958+0000 7f3135d9b640 1 --2- 172.21.15.114:0/2900796266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56512604a060 0x56512604a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:01.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.958+0000 7f3135d9b640 1 -- 172.21.15.114:0/2900796266 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565126042900 msgr2=0x56512604c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:01.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.958+0000 7f3135d9b640 1 --2- 172.21.15.114:0/2900796266 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565126042900 0x56512604c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:01.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.958+0000 7f3135d9b640 1 -- 172.21.15.114:0/2900796266 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31381a3320 con 0x56512604d1a0 2024-09-13T18:27:01.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.959+0000 7f3136d9d640 1 --2- 172.21.15.114:0/2900796266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56512604a060 0x56512604a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:27:01.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.959+0000 7f3135d9b640 1 --2- 172.21.15.114:0/2900796266 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56512604d1a0 0x565126049b20 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f312c00dd80 tx=0x7f312c010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b1ba02d38f55c2d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:01.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.961+0000 7f311dffb640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:27:01.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.961+0000 7f311dffb640 1 -- 172.21.15.114:0/2900796266 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312c00b5e0 con 0x56512604d1a0 2024-09-13T18:27:01.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.961+0000 7f313f895740 1 -- 172.21.15.114:0/2900796266 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565125caa7e0 con 0x56512604d1a0 2024-09-13T18:27:01.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.961+0000 7f311dffb640 1 -- 172.21.15.114:0/2900796266 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f312c01d440 con 0x56512604d1a0 2024-09-13T18:27:01.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.962+0000 7f311dffb640 1 -- 172.21.15.114:0/2900796266 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f312c01d640 con 0x56512604d1a0 2024-09-13T18:27:01.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.963+0000 7f313f895740 10 client.17082 fetch_fsmap learned FSMap version 32 2024-09-13T18:27:01.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.963+0000 7f313f895740 1 -- 172.21.15.114:0/2900796266 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x565125fa4620 con 0x56512604d1a0 2024-09-13T18:27:01.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.964+0000 7f311dffb640 1 -- 172.21.15.114:0/2900796266 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x565125caa7e0 con 0x56512604d1a0 2024-09-13T18:27:01.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.964+0000 7f311dffb640 1 -- 172.21.15.114:0/2900796266 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f312c00f8c0 con 0x56512604d1a0 2024-09-13T18:27:01.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.965+0000 7f313f895740 10 client.17082 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:27:01.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.965+0000 7f313f895740 10 client.17082 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:27:01.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.965+0000 7f313f895740 20 client.17082 populate_metadata read hostname 'smithi114' 2024-09-13T18:27:01.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.965+0000 7f313f895740 1 --2- 172.21.15.114:0/2900796266 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56512605d390 0x56512605f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:01.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.965+0000 7f313f895740 4 client.17082 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:27:01.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.965+0000 7f313f895740 1 -- 172.21.15.114:0/2900796266 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565125cfde30 con 0x56512605d390 2024-09-13T18:27:01.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.966+0000 7f313659c640 1 --2- 172.21.15.114:0/2900796266 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56512605d390 0x56512605f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:01.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.966+0000 7f313659c640 1 --2- 172.21.15.114:0/2900796266 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56512605d390 0x56512605f7b0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b51bbe908c8bce3b server_cookie=1bf642f5197359cd in_seq=0 out_seq=0 2024-09-13T18:27:01.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.967+0000 7f311dffb640 10 client.17082 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:27:01.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.968+0000 7f311dffb640 1 -- 172.21.15.114:0/2900796266 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x565125cfde30 con 0x56512605d390 2024-09-13T18:27:01.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.968+0000 7f311dffb640 10 client.17082 handle_command_reply: tid=0 2024-09-13T18:27:01.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.968+0000 7f313f895740 1 client.17082 shutdown 2024-09-13T18:27:01.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.969+0000 7f313f895740 1 -- 172.21.15.114:0/2900796266 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56512604d1a0 msgr2=0x565126049b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:01.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.969+0000 7f313f895740 1 --2- 172.21.15.114:0/2900796266 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56512604d1a0 0x565126049b20 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f312c00dd80 tx=0x7f312c010830 comp rx=0 tx=0).stop 2024-09-13T18:27:01.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.969+0000 7f313f895740 1 -- 172.21.15.114:0/2900796266 shutdown_connections 2024-09-13T18:27:01.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.969+0000 7f313f895740 1 --2- 172.21.15.114:0/2900796266 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56512605d390 0x56512605f7b0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:01.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.969+0000 7f313f895740 1 --2- 172.21.15.114:0/2900796266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56512604a060 0x56512604a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:01.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.969+0000 7f313f895740 1 --2- 172.21.15.114:0/2900796266 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56512604d1a0 0x565126049b20 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:01.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.969+0000 7f313f895740 1 --2- 172.21.15.114:0/2900796266 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565126042900 0x56512604c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:01.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.969+0000 7f313f895740 1 -- 172.21.15.114:0/2900796266 >> 172.21.15.114:0/2900796266 conn(0x56512603e570 msgr2=0x56512603cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:01.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.969+0000 7f311dffb640 0 client.17082 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:27:01.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.971+0000 7f313f895740 1 -- 172.21.15.114:0/2900796266 shutdown_connections 2024-09-13T18:27:01.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.971+0000 7f313f895740 1 -- 172.21.15.114:0/2900796266 wait complete. 2024-09-13T18:27:01.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:01.972+0000 7f313f895740 20 client.17082 trim_cache size 0 max 16384 2024-09-13T18:27:02.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.009+0000 7f313f895740 1 Processor -- start 2024-09-13T18:27:02.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.012+0000 7f313f895740 1 -- start start 2024-09-13T18:27:02.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.012+0000 7f313f895740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56512603e570 0x565126058ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:02.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.012+0000 7f313f895740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565126042900 0x565126044f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:02.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.012+0000 7f313f895740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56512604a060 0x565126041f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:02.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.013+0000 7f313659c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56512603e570 0x565126058ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:02.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.013+0000 7f313659c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56512603e570 0x565126058ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:39246/0 (socket says 172.21.15.114:39246) 2024-09-13T18:27:02.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.013+0000 7f3135d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565126042900 0x565126044f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:02.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.013+0000 7f3135d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565126042900 0x565126044f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:37056/0 (socket says 172.21.15.114:37056) 2024-09-13T18:27:02.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.013+0000 7f313659c640 1 -- 172.21.15.114:0/3999056816 learned_addr learned my addr 172.21.15.114:0/3999056816 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:02.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.013+0000 7f313f895740 1 -- 172.21.15.114:0/3999056816 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x565126039f30 con 0x56512604a060 2024-09-13T18:27:02.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.013+0000 7f313f895740 1 -- 172.21.15.114:0/3999056816 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56512604ab10 con 0x565126042900 2024-09-13T18:27:02.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.013+0000 7f313f895740 1 -- 172.21.15.114:0/3999056816 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x565126041e00 con 0x56512603e570 2024-09-13T18:27:02.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.016+0000 7f3136d9d640 1 --2- 172.21.15.114:0/3999056816 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56512604a060 0x565126041f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:02.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.018+0000 7f3135d9b640 1 -- 172.21.15.114:0/3999056816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56512603e570 msgr2=0x565126058ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:02.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.018+0000 7f3135d9b640 1 --2- 172.21.15.114:0/3999056816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56512603e570 0x565126058ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:02.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.018+0000 7f3135d9b640 1 -- 172.21.15.114:0/3999056816 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56512604a060 msgr2=0x565126041f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:02.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.018+0000 7f3135d9b640 1 --2- 172.21.15.114:0/3999056816 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56512604a060 0x565126041f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:02.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.018+0000 7f3135d9b640 1 -- 172.21.15.114:0/3999056816 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565125f46790 con 0x565126042900 2024-09-13T18:27:02.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.018+0000 7f3136d9d640 1 --2- 172.21.15.114:0/3999056816 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56512604a060 0x565126041f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:27:02.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.019+0000 7f3135d9b640 1 --2- 172.21.15.114:0/3999056816 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565126042900 0x565126044f00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f312c00b2d0 tx=0x7f312c010f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d012a83b317e37f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:02.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.020+0000 7f31257fa640 1 -- 172.21.15.114:0/3999056816 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312c00b5e0 con 0x565126042900 2024-09-13T18:27:02.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.020+0000 7f31257fa640 1 -- 172.21.15.114:0/3999056816 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f312c00d930 con 0x565126042900 2024-09-13T18:27:02.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.021+0000 7f313f895740 1 -- 172.21.15.114:0/3999056816 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565126042900 msgr2=0x565126044f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:02.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.021+0000 7f313f895740 1 --2- 172.21.15.114:0/3999056816 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565126042900 0x565126044f00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f312c00b2d0 tx=0x7f312c010f80 comp rx=0 tx=0).stop 2024-09-13T18:27:02.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.022+0000 7f313f895740 1 -- 172.21.15.114:0/3999056816 shutdown_connections 2024-09-13T18:27:02.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.022+0000 7f313f895740 1 --2- 172.21.15.114:0/3999056816 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56512604a060 0x565126041f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:02.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.022+0000 7f313f895740 1 --2- 172.21.15.114:0/3999056816 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565126042900 0x565126044f00 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:02.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.022+0000 7f313f895740 1 --2- 172.21.15.114:0/3999056816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56512603e570 0x565126058ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:02.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.022+0000 7f313f895740 1 -- 172.21.15.114:0/3999056816 >> 172.21.15.114:0/3999056816 conn(0x56512604ca00 msgr2=0x565126039330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:02.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.030+0000 7f313f895740 1 -- 172.21.15.114:0/3999056816 shutdown_connections 2024-09-13T18:27:02.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.030+0000 7f313f895740 1 -- 172.21.15.114:0/3999056816 wait complete. 2024-09-13T18:27:02.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.030+0000 7f313f895740 1 -- start start 2024-09-13T18:27:02.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.030+0000 7f313f895740 1 Processor -- start 2024-09-13T18:27:02.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.033+0000 7f313f895740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56512603e570 0x565126051be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:02.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.034+0000 7f313659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56512603e570 0x565126051be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:02.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.034+0000 7f313659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56512603e570 0x565126051be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:37064/0 (socket says 172.21.15.114:37064) 2024-09-13T18:27:02.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.036+0000 7f313f895740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x565126042900 0x565126052120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:02.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.036+0000 7f313f895740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56512604a060 0x565126052660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:02.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.036+0000 7f313f895740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651260b2920 con 0x565126042900 2024-09-13T18:27:02.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.036+0000 7f313f895740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651260b2a90 con 0x56512603e570 2024-09-13T18:27:02.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.036+0000 7f313f895740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651260b2c50 con 0x56512604a060 2024-09-13T18:27:02.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.036+0000 7f313f895740 10 client.0 _ll_register_callbacks cb 0x56512604d0f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:27:02.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.036+0000 7f313659c640 1 -- 172.21.15.114:0/3256280594 learned_addr learned my addr 172.21.15.114:0/3256280594 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:02.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.036+0000 7f3136d9d640 1 --2- 172.21.15.114:0/3256280594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56512604a060 0x565126052660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:02.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.036+0000 7f3135d9b640 1 --2- 172.21.15.114:0/3256280594 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x565126042900 0x565126052120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:02.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.037+0000 7f313659c640 1 -- 172.21.15.114:0/3256280594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56512604a060 msgr2=0x565126052660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:02.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.037+0000 7f313659c640 1 --2- 172.21.15.114:0/3256280594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56512604a060 0x565126052660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:02.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.037+0000 7f313659c640 1 -- 172.21.15.114:0/3256280594 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x565126042900 msgr2=0x565126052120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:02.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.037+0000 7f313659c640 1 --2- 172.21.15.114:0/3256280594 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x565126042900 0x565126052120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:02.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.037+0000 7f313659c640 1 -- 172.21.15.114:0/3256280594 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3138108410 con 0x56512603e570 2024-09-13T18:27:02.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.037+0000 7f313659c640 1 --2- 172.21.15.114:0/3256280594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56512603e570 0x565126051be0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f31280977e0 tx=0x7f3128005620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59e12bceb482f9a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:02.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.038+0000 7f311dffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:27:02.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.038+0000 7f311dffb640 1 -- 172.21.15.114:0/3256280594 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312809ac40 con 0x56512603e570 2024-09-13T18:27:02.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.038+0000 7f313f895740 1 -- 172.21.15.114:0/3256280594 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565125c994e0 con 0x56512603e570 2024-09-13T18:27:02.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.042+0000 7f311dffb640 1 -- 172.21.15.114:0/3256280594 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f312809f110 con 0x56512603e570 2024-09-13T18:27:02.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.042+0000 7f311dffb640 1 -- 172.21.15.114:0/3256280594 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f312809b2e0 con 0x56512603e570 2024-09-13T18:27:02.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.043+0000 7f313f895740 10 client.26438 fetch_fsmap learned FSMap version 32 2024-09-13T18:27:02.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.043+0000 7f313f895740 1 -- 172.21.15.114:0/3256280594 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x565125fa4620 con 0x56512603e570 2024-09-13T18:27:02.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.043+0000 7f311dffb640 1 -- 172.21.15.114:0/3256280594 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x565125c994e0 con 0x56512603e570 2024-09-13T18:27:02.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.044+0000 7f311dffb640 1 -- 172.21.15.114:0/3256280594 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f31280ab070 con 0x56512603e570 2024-09-13T18:27:02.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.044+0000 7f313f895740 10 client.26438 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:27:02.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.044+0000 7f313f895740 10 client.26438 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:27:02.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.044+0000 7f313f895740 20 client.26438 populate_metadata read hostname 'smithi114' 2024-09-13T18:27:02.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.044+0000 7f313f895740 1 --2- 172.21.15.114:0/3256280594 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56512604d1a0 0x5651260b6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:02.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.044+0000 7f313f895740 4 client.26438 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T18:27:02.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.044+0000 7f313f895740 1 -- 172.21.15.114:0/3256280594 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5651260b6ae0 con 0x56512604d1a0 2024-09-13T18:27:02.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.046+0000 7f3135d9b640 1 --2- 172.21.15.114:0/3256280594 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56512604d1a0 0x5651260b6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:02.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.049+0000 7f3135d9b640 1 --2- 172.21.15.114:0/3256280594 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56512604d1a0 0x5651260b6310 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90e92b4eb4921060 server_cookie=34e78ad946fc28f0 in_seq=0 out_seq=0 2024-09-13T18:27:02.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.051+0000 7f311dffb640 10 client.26438 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:27:02.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.051+0000 7f311dffb640 1 -- 172.21.15.114:0/3256280594 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5651260b6ae0 con 0x56512604d1a0 2024-09-13T18:27:02.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.051+0000 7f311dffb640 10 client.26438 handle_command_reply: tid=0 2024-09-13T18:27:02.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.051+0000 7f313f895740 1 client.26438 shutdown 2024-09-13T18:27:02.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.059+0000 7f313f895740 1 -- 172.21.15.114:0/3256280594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56512603e570 msgr2=0x565126051be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:02.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.059+0000 7f313f895740 1 --2- 172.21.15.114:0/3256280594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56512603e570 0x565126051be0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f31280977e0 tx=0x7f3128005620 comp rx=0 tx=0).stop 2024-09-13T18:27:02.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.060+0000 7f313f895740 1 -- 172.21.15.114:0/3256280594 shutdown_connections 2024-09-13T18:27:02.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.060+0000 7f313f895740 1 --2- 172.21.15.114:0/3256280594 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56512604d1a0 0x5651260b6310 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:02.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.060+0000 7f313f895740 1 --2- 172.21.15.114:0/3256280594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56512604a060 0x565126052660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:02.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.060+0000 7f313f895740 1 --2- 172.21.15.114:0/3256280594 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x565126042900 0x565126052120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:02.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.060+0000 7f313f895740 1 --2- 172.21.15.114:0/3256280594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56512603e570 0x565126051be0 secure :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f31280977e0 tx=0x7f3128005620 comp rx=0 tx=0).stop 2024-09-13T18:27:02.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.060+0000 7f313f895740 1 -- 172.21.15.114:0/3256280594 >> 172.21.15.114:0/3256280594 conn(0x56512604ca00 msgr2=0x565126044f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:02.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.061+0000 7f311dffb640 0 client.26438 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:27:02.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.065+0000 7f313f895740 1 -- 172.21.15.114:0/3256280594 shutdown_connections 2024-09-13T18:27:02.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.067+0000 7f313f895740 1 -- 172.21.15.114:0/3256280594 wait complete. 2024-09-13T18:27:02.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.075+0000 7f313f895740 20 client.26438 trim_cache size 0 max 16384 2024-09-13T18:27:02.081 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:27:02.081 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "no active scrubs running", 2024-09-13T18:27:02.081 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": {} 2024-09-13T18:27:02.081 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:27:02.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.085+0000 7f313759e640 1 -- 172.21.15.114:0/3347013646 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f31200a1410 msgr2=0x7f31200a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:02.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.085+0000 7f313759e640 1 --2- 172.21.15.114:0/3347013646 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f31200a1410 0x7f31200a38d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f31381133c0 tx=0x7f312c005f70 comp rx=0 tx=0).stop 2024-09-13T18:27:02.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.085+0000 7f313759e640 1 -- 172.21.15.114:0/3347013646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31381074b0 msgr2=0x7f3138111c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:02.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.085+0000 7f313759e640 1 --2- 172.21.15.114:0/3347013646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31381074b0 0x7f3138111c40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f312800e9f0 tx=0x7f312800edf0 comp rx=0 tx=0).stop 2024-09-13T18:27:02.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.091+0000 7f313759e640 1 -- 172.21.15.114:0/3347013646 shutdown_connections 2024-09-13T18:27:02.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.091+0000 7f313759e640 1 --2- 172.21.15.114:0/3347013646 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f31200a1410 0x7f31200a38d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:02.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.091+0000 7f313759e640 1 --2- 172.21.15.114:0/3347013646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3138108ee0 0x7f31381126c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:02.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.091+0000 7f313759e640 1 --2- 172.21.15.114:0/3347013646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31381085c0 0x7f3138112180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:02.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.091+0000 7f313759e640 1 --2- 172.21.15.114:0/3347013646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31381074b0 0x7f3138111c40 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:02.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.091+0000 7f313759e640 1 -- 172.21.15.114:0/3347013646 >> 172.21.15.114:0/3347013646 conn(0x7f313806b390 msgr2=0x7f3138070130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:02.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.095+0000 7f313759e640 1 -- 172.21.15.114:0/3347013646 shutdown_connections 2024-09-13T18:27:02.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.096+0000 7f313759e640 1 -- 172.21.15.114:0/3347013646 wait complete. 2024-09-13T18:27:02.133 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-13T18:27:02.133 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:27:02.368 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:02 smithi134 ceph-mon[34283]: pgmap v139: 129 pgs: 129 active+clean; 16 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 62 MiB/s wr, 999 op/s 2024-09-13T18:27:02.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.413+0000 7f8eae8b6640 1 -- 172.21.15.114:0/1222834684 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8ea8106550 msgr2=0x7f8ea8106930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:02.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.413+0000 7f8eae8b6640 1 --2- 172.21.15.114:0/1222834684 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8ea8106550 0x7f8ea8106930 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0009c10 tx=0x7f8ea002a6e0 comp rx=0 tx=0).stop 2024-09-13T18:27:02.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.414+0000 7f8eae8b6640 1 -- 172.21.15.114:0/1222834684 shutdown_connections 2024-09-13T18:27:02.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.414+0000 7f8eae8b6640 1 --2- 172.21.15.114:0/1222834684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ea810b420 0x7f8ea8112fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:02.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.414+0000 7f8eae8b6640 1 --2- 172.21.15.114:0/1222834684 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ea8106f00 0x7f8ea810acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:02.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.414+0000 7f8eae8b6640 1 --2- 172.21.15.114:0/1222834684 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8ea8106550 0x7f8ea8106930 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:02.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.414+0000 7f8eae8b6640 1 -- 172.21.15.114:0/1222834684 >> 172.21.15.114:0/1222834684 conn(0x7f8ea8075a10 msgr2=0x7f8ea8077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:02.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.417+0000 7f8eae8b6640 1 -- 172.21.15.114:0/1222834684 shutdown_connections 2024-09-13T18:27:02.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.419+0000 7f8eae8b6640 1 -- 172.21.15.114:0/1222834684 wait complete. 2024-09-13T18:27:02.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.420+0000 7f8eae8b6640 1 Processor -- start 2024-09-13T18:27:02.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.421+0000 7f8eae8b6640 1 -- start start 2024-09-13T18:27:02.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.421+0000 7f8eae8b6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ea8106550 0x7f8ea819c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:02.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.421+0000 7f8eae8b6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ea810b420 0x7f8ea819cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:02.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.421+0000 7f8eae8b6640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8ea81a0f40 0x7f8ea81a13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:02.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.421+0000 7f8eae8b6640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea81a1930 con 0x7f8ea81a0f40 2024-09-13T18:27:02.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.421+0000 7f8eae8b6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea81a1a70 con 0x7f8ea8106550 2024-09-13T18:27:02.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.421+0000 7f8eae8b6640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea81a1c30 con 0x7f8ea810b420 2024-09-13T18:27:02.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.421+0000 7f8eae0b5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8ea81a0f40 0x7f8ea81a13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:02.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.421+0000 7f8eae0b5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8ea81a0f40 0x7f8ea81a13f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34868/0 (socket says 172.21.15.114:34868) 2024-09-13T18:27:02.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.421+0000 7f8eae0b5640 1 -- 172.21.15.114:0/169201425 learned_addr learned my addr 172.21.15.114:0/169201425 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:02.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.421+0000 7f8ead0b3640 1 --2- 172.21.15.114:0/169201425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ea810b420 0x7f8ea819cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:02.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.422+0000 7f8eae0b5640 1 -- 172.21.15.114:0/169201425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ea810b420 msgr2=0x7f8ea819cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:02.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.422+0000 7f8eae0b5640 1 --2- 172.21.15.114:0/169201425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ea810b420 0x7f8ea819cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:02.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.422+0000 7f8eae0b5640 1 -- 172.21.15.114:0/169201425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ea8106550 msgr2=0x7f8ea819c610 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:27:02.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.422+0000 7f8eae0b5640 1 --2- 172.21.15.114:0/169201425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ea8106550 0x7f8ea819c610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:02.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.422+0000 7f8eae0b5640 1 -- 172.21.15.114:0/169201425 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea0009840 con 0x7f8ea81a0f40 2024-09-13T18:27:02.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.422+0000 7f8eae0b5640 1 --2- 172.21.15.114:0/169201425 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8ea81a0f40 0x7f8ea81a13f0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f8ea400e970 tx=0x7f8ea400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:02.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.423+0000 7f8e9effd640 1 -- 172.21.15.114:0/169201425 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea400ccc0 con 0x7f8ea81a0f40 2024-09-13T18:27:02.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.423+0000 7f8eae8b6640 1 -- 172.21.15.114:0/169201425 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ea8069ed0 con 0x7f8ea81a0f40 2024-09-13T18:27:02.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.423+0000 7f8e9effd640 1 -- 172.21.15.114:0/169201425 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8ea400ce20 con 0x7f8ea81a0f40 2024-09-13T18:27:02.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.423+0000 7f8e9effd640 1 -- 172.21.15.114:0/169201425 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea4004e40 con 0x7f8ea81a0f40 2024-09-13T18:27:02.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.423+0000 7f8eae8b6640 1 -- 172.21.15.114:0/169201425 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ea806a3e0 con 0x7f8ea81a0f40 2024-09-13T18:27:02.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.425+0000 7f8eae8b6640 1 -- 172.21.15.114:0/169201425 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ea81077d0 con 0x7f8ea81a0f40 2024-09-13T18:27:02.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:02 smithi204 ceph-mon[34777]: pgmap v139: 129 pgs: 129 active+clean; 16 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 62 MiB/s wr, 999 op/s 2024-09-13T18:27:02.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.442+0000 7f8e9effd640 1 -- 172.21.15.114:0/169201425 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f8ea4005020 con 0x7f8ea81a0f40 2024-09-13T18:27:02.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.443+0000 7f8e9effd640 1 --2- 172.21.15.114:0/169201425 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f8e940c3420 0x7f8e940c58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:02.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.443+0000 7f8e9effd640 1 -- 172.21.15.114:0/169201425 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f8ea4014070 con 0x7f8ea81a0f40 2024-09-13T18:27:02.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.444+0000 7f8e9effd640 1 -- 172.21.15.114:0/169201425 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f8ea40cdb60 con 0x7f8ea81a0f40 2024-09-13T18:27:02.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.451+0000 7f8ead8b4640 1 --2- 172.21.15.114:0/169201425 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f8e940c3420 0x7f8e940c58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:02.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:02.453+0000 7f8ead8b4640 1 --2- 172.21.15.114:0/169201425 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f8e940c3420 0x7f8e940c58e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0004af0 tx=0x7f8ea0005fd0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:02.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:02 smithi114 ceph-mon[30897]: pgmap v139: 129 pgs: 129 active+clean; 16 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 62 MiB/s wr, 999 op/s 2024-09-13T18:27:03.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.041+0000 7f8eae8b6640 1 -- 172.21.15.114:0/169201425 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ea810e5f0 con 0x7f8ea81a0f40 2024-09-13T18:27:03.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.043+0000 7f8e9effd640 1 -- 172.21.15.114:0/169201425 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+10937 (secure 0 0 0) 0x7f8ea40969c0 con 0x7f8ea81a0f40 2024-09-13T18:27:03.049 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:27:03.049 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4162817497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4162817497},{"type":"v1","addr":"172.21.15.114:6835","nonce":4162817497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/2435428478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":2435428478},{"type":"v1","addr":"172.21.15.114:6837","nonce":2435428478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T18:20:01.895117+0000","modified":"2024-09-13T18:24:01.324695+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24482,"mds_2":24475,"mds_3":24458,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/2046516451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":2046516451},{"type":"v1","addr":"172.21.15.114:6839","nonce":2046516451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":10,"addr":"172.21.15.114:6841/1956923446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":1956923446},{"type":"v1","addr":"172.21.15.114:6841","nonce":1956923446}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24391":{"gid":24391,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/507301496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":507301496},{"type":"v1","addr":"172.21.15.204:6833","nonce":507301496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4028657219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4028657219},{"type":"v1","addr":"172.21.15.134:6833","nonce":4028657219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/57029119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":57029119},{"type":"v1","addr":"172.21.15.204:6835","nonce":57029119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3068880939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3068880939},{"type":"v1","addr":"172.21.15.204:6837","nonce":3068880939}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"e","rank":3,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.134:6835/1150003275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1150003275},{"type":"v1","addr":"172.21.15.134:6835","nonce":1150003275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/344883912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":344883912},{"type":"v1","addr":"172.21.15.204:6839","nonce":344883912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6837/891692210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":891692210},{"type":"v1","addr":"172.21.15.134:6837","nonce":891692210}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6839/3466309724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3466309724},{"type":"v1","addr":"172.21.15.134:6839","nonce":3466309724}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:27:03.049 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 32 2024-09-13T18:27:03.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.054+0000 7f8e9cff9640 1 -- 172.21.15.114:0/169201425 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f8e940c3420 msgr2=0x7f8e940c58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.054+0000 7f8e9cff9640 1 --2- 172.21.15.114:0/169201425 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f8e940c3420 0x7f8e940c58e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0004af0 tx=0x7f8ea0005fd0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.054+0000 7f8e9cff9640 1 -- 172.21.15.114:0/169201425 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8ea81a0f40 msgr2=0x7f8ea81a13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.054+0000 7f8e9cff9640 1 --2- 172.21.15.114:0/169201425 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8ea81a0f40 0x7f8ea81a13f0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f8ea400e970 tx=0x7f8ea400ed70 comp rx=0 tx=0).stop 2024-09-13T18:27:03.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.054+0000 7f8e9cff9640 1 -- 172.21.15.114:0/169201425 shutdown_connections 2024-09-13T18:27:03.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.054+0000 7f8e9cff9640 1 --2- 172.21.15.114:0/169201425 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f8e940c3420 0x7f8e940c58e0 secure :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0004af0 tx=0x7f8ea0005fd0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.054+0000 7f8e9cff9640 1 --2- 172.21.15.114:0/169201425 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8ea81a0f40 0x7f8ea81a13f0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.054+0000 7f8e9cff9640 1 --2- 172.21.15.114:0/169201425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ea810b420 0x7f8ea819cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.054+0000 7f8e9cff9640 1 --2- 172.21.15.114:0/169201425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ea8106550 0x7f8ea819c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.054+0000 7f8e9cff9640 1 -- 172.21.15.114:0/169201425 >> 172.21.15.114:0/169201425 conn(0x7f8ea8075a10 msgr2=0x7f8ea8076fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:03.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.056+0000 7f8e9cff9640 1 -- 172.21.15.114:0/169201425 shutdown_connections 2024-09-13T18:27:03.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.059+0000 7f8e9cff9640 1 -- 172.21.15.114:0/169201425 wait complete. 2024-09-13T18:27:03.089 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-09-13T18:27:03.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.342+0000 7fba6d648640 1 -- 172.21.15.114:0/2090187477 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba68107540 msgr2=0x7fba6810e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.342+0000 7fba6d648640 1 --2- 172.21.15.114:0/2090187477 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba68107540 0x7fba6810e510 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fba58009900 tx=0x7fba5802a440 comp rx=0 tx=0).stop 2024-09-13T18:27:03.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.346+0000 7fba6d648640 1 -- 172.21.15.114:0/2090187477 shutdown_connections 2024-09-13T18:27:03.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.346+0000 7fba6d648640 1 --2- 172.21.15.114:0/2090187477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba68119400 0x7fba6811b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.346+0000 7fba6d648640 1 --2- 172.21.15.114:0/2090187477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba68107920 0x7fba68107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.346+0000 7fba6d648640 1 --2- 172.21.15.114:0/2090187477 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba68107540 0x7fba6810e510 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.346+0000 7fba6d648640 1 -- 172.21.15.114:0/2090187477 >> 172.21.15.114:0/2090187477 conn(0x7fba6806afe0 msgr2=0x7fba6806b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:03.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.352+0000 7fba6d648640 1 -- 172.21.15.114:0/2090187477 shutdown_connections 2024-09-13T18:27:03.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.357+0000 7fba6d648640 1 -- 172.21.15.114:0/2090187477 wait complete. 2024-09-13T18:27:03.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.358+0000 7fba6d648640 1 Processor -- start 2024-09-13T18:27:03.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.360+0000 7fba6d648640 1 -- start start 2024-09-13T18:27:03.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.360+0000 7fba6d648640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba68107540 0x7fba681a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.360+0000 7fba6d648640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba68107920 0x7fba681a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.361+0000 7fba6d648640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba68119400 0x7fba681a99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.361+0000 7fba6d648640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba681a9fb0 con 0x7fba68107920 2024-09-13T18:27:03.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.361+0000 7fba6d648640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba681aa0f0 con 0x7fba68107540 2024-09-13T18:27:03.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.361+0000 7fba6d648640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba681aa2b0 con 0x7fba68119400 2024-09-13T18:27:03.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.363+0000 7fba667fc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba68107920 0x7fba681a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.364+0000 7fba667fc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba68107920 0x7fba681a55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59754/0 (socket says 172.21.15.114:59754) 2024-09-13T18:27:03.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.364+0000 7fba667fc640 1 -- 172.21.15.114:0/1063859024 learned_addr learned my addr 172.21.15.114:0/1063859024 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:03.368 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:03 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/169201425' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:27:03.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.365+0000 7fba677fe640 1 --2- 172.21.15.114:0/1063859024 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba68119400 0x7fba681a99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.365+0000 7fba66ffd640 1 --2- 172.21.15.114:0/1063859024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba68107540 0x7fba681a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.366+0000 7fba667fc640 1 -- 172.21.15.114:0/1063859024 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba68119400 msgr2=0x7fba681a99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.366+0000 7fba667fc640 1 --2- 172.21.15.114:0/1063859024 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba68119400 0x7fba681a99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.366+0000 7fba667fc640 1 -- 172.21.15.114:0/1063859024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba68107540 msgr2=0x7fba681a5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.366+0000 7fba667fc640 1 --2- 172.21.15.114:0/1063859024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba68107540 0x7fba681a5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.366+0000 7fba667fc640 1 -- 172.21.15.114:0/1063859024 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba58009560 con 0x7fba68107920 2024-09-13T18:27:03.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.366+0000 7fba66ffd640 1 --2- 172.21.15.114:0/1063859024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba68107540 0x7fba681a5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:27:03.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.367+0000 7fba667fc640 1 --2- 172.21.15.114:0/1063859024 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba68107920 0x7fba681a55c0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fba6000cce0 tx=0x7fba6000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.369+0000 7fba4ffff640 1 -- 172.21.15.114:0/1063859024 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6000f650 con 0x7fba68107920 2024-09-13T18:27:03.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.369+0000 7fba4ffff640 1 -- 172.21.15.114:0/1063859024 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fba6000ce90 con 0x7fba68107920 2024-09-13T18:27:03.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.369+0000 7fba4ffff640 1 -- 172.21.15.114:0/1063859024 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba60018aa0 con 0x7fba68107920 2024-09-13T18:27:03.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.372+0000 7fba6d648640 1 -- 172.21.15.114:0/1063859024 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba681aa590 con 0x7fba68107920 2024-09-13T18:27:03.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.372+0000 7fba6d648640 1 -- 172.21.15.114:0/1063859024 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba6810edc0 con 0x7fba68107920 2024-09-13T18:27:03.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.375+0000 7fba70448740 1 Processor -- start 2024-09-13T18:27:03.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.376+0000 7fba70448740 1 -- start start 2024-09-13T18:27:03.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.377+0000 7fba70448740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5588d00f2790 0x5588d00f2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.377+0000 7fba70448740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5588d00f30b0 0x5588d00fd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.377+0000 7fba70448740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5588d00fd950 0x5588d00ffd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.377+0000 7fba70448740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588d005ccc0 con 0x5588d00f2790 2024-09-13T18:27:03.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.377+0000 7fba70448740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588d005a450 con 0x5588d00fd950 2024-09-13T18:27:03.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.377+0000 7fba70448740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588d0055d40 con 0x5588d00f30b0 2024-09-13T18:27:03.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.377+0000 7fba66ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5588d00f30b0 0x5588d00fd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.377+0000 7fba66ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5588d00f30b0 0x5588d00fd410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:59898/0 (socket says 172.21.15.114:59898) 2024-09-13T18:27:03.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.377+0000 7fba66ffd640 1 -- 172.21.15.114:0/663416470 learned_addr learned my addr 172.21.15.114:0/663416470 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:03.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.377+0000 7fba667fc640 1 --2- 172.21.15.114:0/663416470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5588d00fd950 0x5588d00ffd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.378+0000 7fba667fc640 1 -- 172.21.15.114:0/663416470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5588d00f30b0 msgr2=0x5588d00fd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:03.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.378+0000 7fba667fc640 1 --2- 172.21.15.114:0/663416470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5588d00f30b0 0x5588d00fd410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.378+0000 7fba667fc640 1 -- 172.21.15.114:0/663416470 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5588d00f2790 msgr2=0x5588d00f2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:03.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.378+0000 7fba667fc640 1 --2- 172.21.15.114:0/663416470 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5588d00f2790 0x5588d00f2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.378+0000 7fba667fc640 1 -- 172.21.15.114:0/663416470 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588d001d620 con 0x5588d00fd950 2024-09-13T18:27:03.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.378+0000 7fba667fc640 1 --2- 172.21.15.114:0/663416470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5588d00fd950 0x5588d00ffd40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fba600ca680 tx=0x7fba600d57a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2edc0688a2f27052 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.378+0000 7fba4ffff640 1 -- 172.21.15.114:0/1063859024 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fba60018c00 con 0x7fba68107920 2024-09-13T18:27:03.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.380+0000 7fba4d7fa640 1 -- 172.21.15.114:0/663416470 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba600cf4e0 con 0x5588d00fd950 2024-09-13T18:27:03.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.380+0000 7fba4d7fa640 1 -- 172.21.15.114:0/663416470 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fba600cdb60 con 0x5588d00fd950 2024-09-13T18:27:03.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.380+0000 7fba4d7fa640 1 -- 172.21.15.114:0/663416470 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba600dba70 con 0x5588d00fd950 2024-09-13T18:27:03.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.381+0000 7fba70448740 1 -- 172.21.15.114:0/663416470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5588d00fd950 msgr2=0x5588d00ffd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.381+0000 7fba70448740 1 --2- 172.21.15.114:0/663416470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5588d00fd950 0x5588d00ffd40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fba600ca680 tx=0x7fba600d57a0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.382+0000 7fba4ffff640 1 --2- 172.21.15.114:0/1063859024 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fba500a1410 0x7fba500a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.382+0000 7fba4ffff640 1 -- 172.21.15.114:0/1063859024 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fba600cd2e0 con 0x7fba68107920 2024-09-13T18:27:03.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.383+0000 7fba70448740 1 -- 172.21.15.114:0/663416470 shutdown_connections 2024-09-13T18:27:03.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.383+0000 7fba70448740 1 --2- 172.21.15.114:0/663416470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5588d00fd950 0x5588d00ffd40 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.383+0000 7fba70448740 1 --2- 172.21.15.114:0/663416470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5588d00f30b0 0x5588d00fd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.383+0000 7fba70448740 1 --2- 172.21.15.114:0/663416470 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5588d00f2790 0x5588d00f2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.383+0000 7fba70448740 1 -- 172.21.15.114:0/663416470 >> 172.21.15.114:0/663416470 conn(0x5588d00eed00 msgr2=0x5588d00ef0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:03.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.383+0000 7fba70448740 1 -- 172.21.15.114:0/663416470 shutdown_connections 2024-09-13T18:27:03.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.383+0000 7fba70448740 1 -- 172.21.15.114:0/663416470 wait complete. 2024-09-13T18:27:03.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.384+0000 7fba70448740 1 -- start start 2024-09-13T18:27:03.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.384+0000 7fba70448740 1 Processor -- start 2024-09-13T18:27:03.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.385+0000 7fba677fe640 1 --2- 172.21.15.114:0/1063859024 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fba500a1410 0x7fba500a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.385+0000 7fba70448740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5588d00f2790 0x5588d00fd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.386+0000 7fba70448740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5588d00f30b0 0x5588d00fa3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.386+0000 7fba66ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5588d00f2790 0x5588d00fd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.386+0000 7fba66ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5588d00f2790 0x5588d00fd020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:59902/0 (socket says 172.21.15.114:59902) 2024-09-13T18:27:03.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.386+0000 7fba66ffd640 1 -- 172.21.15.114:0/612504932 learned_addr learned my addr 172.21.15.114:0/612504932 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:03.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.386+0000 7fba70448740 1 --2- 172.21.15.114:0/612504932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5588d00fd950 0x5588d00fa900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.386+0000 7fba70448740 1 -- 172.21.15.114:0/612504932 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588d0014180 con 0x5588d00f30b0 2024-09-13T18:27:03.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.386+0000 7fba70448740 1 -- 172.21.15.114:0/612504932 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588cfd2e430 con 0x5588d00fd950 2024-09-13T18:27:03.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.386+0000 7fba70448740 1 -- 172.21.15.114:0/612504932 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588cffd95e0 con 0x5588d00f2790 2024-09-13T18:27:03.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.386+0000 7fba677fe640 1 --2- 172.21.15.114:0/612504932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5588d00fd950 0x5588d00fa900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.387+0000 7fba70448740 10 client.0 _ll_register_callbacks cb 0x5588d0045aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:27:03.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.387+0000 7fba667fc640 1 --2- 172.21.15.114:0/612504932 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5588d00f30b0 0x5588d00fa3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.387+0000 7fba66ffd640 1 -- 172.21.15.114:0/612504932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5588d00fd950 msgr2=0x5588d00fa900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:03.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.387+0000 7fba66ffd640 1 --2- 172.21.15.114:0/612504932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5588d00fd950 0x5588d00fa900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.387+0000 7fba66ffd640 1 -- 172.21.15.114:0/612504932 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5588d00f30b0 msgr2=0x5588d00fa3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:03.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.387+0000 7fba66ffd640 1 --2- 172.21.15.114:0/612504932 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5588d00f30b0 0x5588d00fa3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.387+0000 7fba66ffd640 1 -- 172.21.15.114:0/612504932 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba6810edc0 con 0x5588d00f2790 2024-09-13T18:27:03.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.388+0000 7fba66ffd640 1 --2- 172.21.15.114:0/612504932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5588d00f2790 0x5588d00fd020 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fba58033000 tx=0x7fba58005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58221ca16e844c2f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.388+0000 7fba70448740 1 -- 172.21.15.114:0/612504932 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5588d007fb80 con 0x5588d00f2790 2024-09-13T18:27:03.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.388+0000 7fba4aff5640 10 client.26470 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:27:03.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.389+0000 7fba677fe640 1 --2- 172.21.15.114:0/1063859024 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fba500a1410 0x7fba500a38d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fba5c004790 tx=0x7fba5c0069e0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.391+0000 7fba4aff5640 1 -- 172.21.15.114:0/612504932 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba580048c0 con 0x5588d00f2790 2024-09-13T18:27:03.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.391+0000 7fba4aff5640 1 -- 172.21.15.114:0/612504932 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fba58004a20 con 0x5588d00f2790 2024-09-13T18:27:03.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.392+0000 7fba4aff5640 1 -- 172.21.15.114:0/612504932 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba58004c20 con 0x5588d00f2790 2024-09-13T18:27:03.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.392+0000 7fba70448740 10 client.26470 fetch_fsmap learned FSMap version 32 2024-09-13T18:27:03.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.392+0000 7fba70448740 1 -- 172.21.15.114:0/612504932 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({fsmap=32}) v3 -- 0x5588d00543c0 con 0x5588d00f2790 2024-09-13T18:27:03.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.392+0000 7fba4aff5640 1 -- 172.21.15.114:0/612504932 <== mon.1 v2:172.21.15.204:3300/0 4 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fba580035f0 con 0x5588d00f2790 2024-09-13T18:27:03.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.392+0000 7fba70448740 10 client.26470 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:27:03.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.392+0000 7fba70448740 10 client.26470 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-13T18:27:03.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.392+0000 7fba70448740 20 client.26470 populate_metadata read hostname 'smithi114' 2024-09-13T18:27:03.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.393+0000 7fba70448740 1 --2- 172.21.15.114:0/612504932 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x5588d010d150 0x5588d010f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.393+0000 7fba70448740 4 client.26470 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:27:03.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.393+0000 7fba70448740 1 -- 172.21.15.114:0/612504932 --> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5588d007fb80 con 0x5588d010d150 2024-09-13T18:27:03.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.393+0000 7fba667fc640 1 --2- 172.21.15.114:0/612504932 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x5588d010d150 0x5588d010f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.395+0000 7fba667fc640 1 --2- 172.21.15.114:0/612504932 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x5588d010d150 0x5588d010f570 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2c98af4c05d74103 server_cookie=a6c73956a7f08ec4 in_seq=0 out_seq=0 2024-09-13T18:27:03.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.395+0000 7fba4aff5640 10 client.26470 ms_handle_connect on v2:172.21.15.114:6840/1956923446 2024-09-13T18:27:03.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.396+0000 7fba4aff5640 1 -- 172.21.15.114:0/612504932 <== mds.4 v2:172.21.15.114:6840/1956923446 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5588d007fb80 con 0x5588d010d150 2024-09-13T18:27:03.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.396+0000 7fba4aff5640 10 client.26470 handle_command_reply: tid=0 2024-09-13T18:27:03.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.396+0000 7fba70448740 1 client.26470 shutdown 2024-09-13T18:27:03.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.396+0000 7fba70448740 1 -- 172.21.15.114:0/612504932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5588d00f2790 msgr2=0x5588d00fd020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.396+0000 7fba70448740 1 --2- 172.21.15.114:0/612504932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5588d00f2790 0x5588d00fd020 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fba58033000 tx=0x7fba58005270 comp rx=0 tx=0).stop 2024-09-13T18:27:03.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.399+0000 7fba70448740 1 -- 172.21.15.114:0/612504932 shutdown_connections 2024-09-13T18:27:03.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.399+0000 7fba70448740 1 --2- 172.21.15.114:0/612504932 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x5588d010d150 0x5588d010f570 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.399+0000 7fba70448740 1 --2- 172.21.15.114:0/612504932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5588d00fd950 0x5588d00fa900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.399+0000 7fba70448740 1 --2- 172.21.15.114:0/612504932 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5588d00f30b0 0x5588d00fa3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.399+0000 7fba70448740 1 --2- 172.21.15.114:0/612504932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5588d00f2790 0x5588d00fd020 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.399+0000 7fba70448740 1 -- 172.21.15.114:0/612504932 >> 172.21.15.114:0/612504932 conn(0x5588d00eed00 msgr2=0x5588d00f4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:03.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.399+0000 7fba4aff5640 0 client.26470 ms_handle_reset on v2:172.21.15.114:6840/1956923446 2024-09-13T18:27:03.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.401+0000 7fba70448740 1 -- 172.21.15.114:0/612504932 shutdown_connections 2024-09-13T18:27:03.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.401+0000 7fba70448740 1 -- 172.21.15.114:0/612504932 wait complete. 2024-09-13T18:27:03.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.404+0000 7fba70448740 20 client.26470 trim_cache size 0 max 16384 2024-09-13T18:27:03.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.433+0000 7fba70448740 1 Processor -- start 2024-09-13T18:27:03.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.433+0000 7fba70448740 1 -- start start 2024-09-13T18:27:03.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.433+0000 7fba70448740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5588d00f2790 0x5588d0104550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.433+0000 7fba70448740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5588d00f30b0 0x5588d0104a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.433+0000 7fba70448740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5588d00fd950 0x5588d00f8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.433+0000 7fba70448740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588d00fd3c0 con 0x5588d00f30b0 2024-09-13T18:27:03.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.434+0000 7fba70448740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588d00f3750 con 0x5588d00f2790 2024-09-13T18:27:03.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.434+0000 7fba677fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5588d00fd950 0x5588d00f8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.434+0000 7fba677fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5588d00fd950 0x5588d00f8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:59908/0 (socket says 172.21.15.114:59908) 2024-09-13T18:27:03.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.434+0000 7fba677fe640 1 -- 172.21.15.114:0/4249217901 learned_addr learned my addr 172.21.15.114:0/4249217901 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:03.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.434+0000 7fba667fc640 1 --2- 172.21.15.114:0/4249217901 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5588d00f30b0 0x5588d0104a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.434+0000 7fba70448740 1 -- 172.21.15.114:0/4249217901 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588d00f3930 con 0x5588d00fd950 2024-09-13T18:27:03.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.434+0000 7fba677fe640 1 -- 172.21.15.114:0/4249217901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5588d00f2790 msgr2=0x5588d0104550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:03.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.434+0000 7fba677fe640 1 --2- 172.21.15.114:0/4249217901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5588d00f2790 0x5588d0104550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.434+0000 7fba677fe640 1 -- 172.21.15.114:0/4249217901 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5588d00f30b0 msgr2=0x5588d0104a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:03.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.434+0000 7fba677fe640 1 --2- 172.21.15.114:0/4249217901 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5588d00f30b0 0x5588d0104a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.434+0000 7fba677fe640 1 -- 172.21.15.114:0/4249217901 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588cfff6770 con 0x5588d00fd950 2024-09-13T18:27:03.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.435+0000 7fba677fe640 1 --2- 172.21.15.114:0/4249217901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5588d00fd950 0x5588d00f8c50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fba5c0156e0 tx=0x7fba5c012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5c13b37eb2c9468 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.435+0000 7fba4dffb640 1 -- 172.21.15.114:0/4249217901 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5c0223f0 con 0x5588d00fd950 2024-09-13T18:27:03.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.435+0000 7fba4dffb640 1 -- 172.21.15.114:0/4249217901 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fba5c0229e0 con 0x5588d00fd950 2024-09-13T18:27:03.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.435+0000 7fba4dffb640 1 -- 172.21.15.114:0/4249217901 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5c018920 con 0x5588d00fd950 2024-09-13T18:27:03.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.436+0000 7fba70448740 1 -- 172.21.15.114:0/4249217901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5588d00fd950 msgr2=0x5588d00f8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.436+0000 7fba70448740 1 --2- 172.21.15.114:0/4249217901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5588d00fd950 0x5588d00f8c50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fba5c0156e0 tx=0x7fba5c012000 comp rx=0 tx=0).stop 2024-09-13T18:27:03.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.437+0000 7fba70448740 1 -- 172.21.15.114:0/4249217901 shutdown_connections 2024-09-13T18:27:03.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.437+0000 7fba70448740 1 --2- 172.21.15.114:0/4249217901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5588d00fd950 0x5588d00f8c50 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.437+0000 7fba70448740 1 --2- 172.21.15.114:0/4249217901 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5588d00f30b0 0x5588d0104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.437+0000 7fba70448740 1 --2- 172.21.15.114:0/4249217901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5588d00f2790 0x5588d0104550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.437+0000 7fba70448740 1 -- 172.21.15.114:0/4249217901 >> 172.21.15.114:0/4249217901 conn(0x5588d00eed00 msgr2=0x5588d0109010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:03.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.440+0000 7fba70448740 1 -- 172.21.15.114:0/4249217901 shutdown_connections 2024-09-13T18:27:03.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.441+0000 7fba70448740 1 -- 172.21.15.114:0/4249217901 wait complete. 2024-09-13T18:27:03.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:03 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/169201425' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:27:03.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.441+0000 7fba70448740 1 -- start start 2024-09-13T18:27:03.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.441+0000 7fba70448740 1 Processor -- start 2024-09-13T18:27:03.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.442+0000 7fba70448740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5588d00f2790 0x5588d0104240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.442+0000 7fba70448740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5588d00f30b0 0x5588d0104780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.442+0000 7fba70448740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5588d00fd950 0x5588d0100710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.443+0000 7fba70448740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588d0100de0 con 0x5588d00f2790 2024-09-13T18:27:03.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.443+0000 7fba70448740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588d0100f50 con 0x5588d00f30b0 2024-09-13T18:27:03.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.443+0000 7fba70448740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588d0101110 con 0x5588d00fd950 2024-09-13T18:27:03.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.443+0000 7fba677fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5588d00fd950 0x5588d0100710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.443+0000 7fba677fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5588d00fd950 0x5588d0100710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:59910/0 (socket says 172.21.15.114:59910) 2024-09-13T18:27:03.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.443+0000 7fba677fe640 1 -- 172.21.15.114:0/2350944707 learned_addr learned my addr 172.21.15.114:0/2350944707 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:03.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.443+0000 7fba70448740 10 client.0 _ll_register_callbacks cb 0x7fba600d0c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:27:03.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.444+0000 7fba677fe640 1 -- 172.21.15.114:0/2350944707 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5588d00f30b0 msgr2=0x5588d0104780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:03.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.444+0000 7fba677fe640 1 --2- 172.21.15.114:0/2350944707 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5588d00f30b0 0x5588d0104780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.444+0000 7fba677fe640 1 -- 172.21.15.114:0/2350944707 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5588d00f2790 msgr2=0x5588d0104240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:03.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.444+0000 7fba677fe640 1 --2- 172.21.15.114:0/2350944707 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5588d00f2790 0x5588d0104240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.444+0000 7fba677fe640 1 -- 172.21.15.114:0/2350944707 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba5c015420 con 0x5588d00fd950 2024-09-13T18:27:03.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.444+0000 7fba677fe640 1 --2- 172.21.15.114:0/2350944707 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5588d00fd950 0x5588d0100710 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fba5c0156b0 tx=0x7fba5c022620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ea91518e2ae5284 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.444+0000 7fba4aff5640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:27:03.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.444+0000 7fba4aff5640 1 -- 172.21.15.114:0/2350944707 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5c004230 con 0x5588d00fd950 2024-09-13T18:27:03.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.446+0000 7fba70448740 1 -- 172.21.15.114:0/2350944707 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fba600cdb60 con 0x5588d00fd950 2024-09-13T18:27:03.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.446+0000 7fba4aff5640 1 -- 172.21.15.114:0/2350944707 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fba5c004390 con 0x5588d00fd950 2024-09-13T18:27:03.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.448+0000 7fba4aff5640 1 -- 172.21.15.114:0/2350944707 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba5c009b10 con 0x5588d00fd950 2024-09-13T18:27:03.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.451+0000 7fba70448740 10 client.26476 fetch_fsmap learned FSMap version 32 2024-09-13T18:27:03.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.451+0000 7fba70448740 1 -- 172.21.15.114:0/2350944707 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5588d001be80 con 0x5588d00fd950 2024-09-13T18:27:03.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.452+0000 7fba4aff5640 1 -- 172.21.15.114:0/2350944707 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fba600cdb60 con 0x5588d00fd950 2024-09-13T18:27:03.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.452+0000 7fba4aff5640 1 -- 172.21.15.114:0/2350944707 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fba5c0037a0 con 0x5588d00fd950 2024-09-13T18:27:03.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.453+0000 7fba70448740 10 client.26476 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:27:03.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.453+0000 7fba70448740 10 client.26476 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-13T18:27:03.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.453+0000 7fba70448740 20 client.26476 populate_metadata read hostname 'smithi114' 2024-09-13T18:27:03.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.453+0000 7fba70448740 1 --2- 172.21.15.114:0/2350944707 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x5588d01645e0 0x5588d01669f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.453+0000 7fba70448740 4 client.26476 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:27:03.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.453+0000 7fba70448740 1 -- 172.21.15.114:0/2350944707 --> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5588d00481f0 con 0x5588d01645e0 2024-09-13T18:27:03.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.453+0000 7fba66ffd640 1 --2- 172.21.15.114:0/2350944707 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x5588d01645e0 0x5588d01669f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.454+0000 7fba66ffd640 1 --2- 172.21.15.114:0/2350944707 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x5588d01645e0 0x5588d01669f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=eba0add355693d78 server_cookie=671ea8c27f3a01be in_seq=0 out_seq=0 2024-09-13T18:27:03.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.456+0000 7fba4aff5640 10 client.26476 ms_handle_connect on v2:172.21.15.114:6840/1956923446 2024-09-13T18:27:03.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.458+0000 7fba4aff5640 1 -- 172.21.15.114:0/2350944707 <== mds.4 v2:172.21.15.114:6840/1956923446 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5588d00481f0 con 0x5588d01645e0 2024-09-13T18:27:03.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.458+0000 7fba4aff5640 10 client.26476 handle_command_reply: tid=0 2024-09-13T18:27:03.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.458+0000 7fba70448740 1 client.26476 shutdown 2024-09-13T18:27:03.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.466+0000 7fba70448740 1 -- 172.21.15.114:0/2350944707 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5588d00fd950 msgr2=0x5588d0100710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.466+0000 7fba70448740 1 --2- 172.21.15.114:0/2350944707 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5588d00fd950 0x5588d0100710 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fba5c0156b0 tx=0x7fba5c022620 comp rx=0 tx=0).stop 2024-09-13T18:27:03.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:03 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/169201425' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:27:03.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.467+0000 7fba70448740 1 -- 172.21.15.114:0/2350944707 shutdown_connections 2024-09-13T18:27:03.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.467+0000 7fba70448740 1 --2- 172.21.15.114:0/2350944707 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x5588d01645e0 0x5588d01669f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.467+0000 7fba70448740 1 --2- 172.21.15.114:0/2350944707 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5588d00fd950 0x5588d0100710 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.467+0000 7fba70448740 1 --2- 172.21.15.114:0/2350944707 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5588d00f30b0 0x5588d0104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.467+0000 7fba70448740 1 --2- 172.21.15.114:0/2350944707 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5588d00f2790 0x5588d0104240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.467+0000 7fba70448740 1 -- 172.21.15.114:0/2350944707 >> 172.21.15.114:0/2350944707 conn(0x5588d00eed00 msgr2=0x5588d00f8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:03.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.467+0000 7fba4aff5640 0 client.26476 ms_handle_reset on v2:172.21.15.114:6840/1956923446 2024-09-13T18:27:03.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.467+0000 7fba70448740 1 -- 172.21.15.114:0/2350944707 shutdown_connections 2024-09-13T18:27:03.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.468+0000 7fba70448740 1 -- 172.21.15.114:0/2350944707 wait complete. 2024-09-13T18:27:03.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.468+0000 7fba70448740 20 client.26476 trim_cache size 0 max 16384 2024-09-13T18:27:03.472 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:27:03.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.477+0000 7fba6d648640 1 -- 172.21.15.114:0/1063859024 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fba500a1410 msgr2=0x7fba500a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.477+0000 7fba6d648640 1 --2- 172.21.15.114:0/1063859024 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fba500a1410 0x7fba500a38d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fba5c004790 tx=0x7fba5c0069e0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.477+0000 7fba6d648640 1 -- 172.21.15.114:0/1063859024 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba68107920 msgr2=0x7fba681a55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.477+0000 7fba6d648640 1 --2- 172.21.15.114:0/1063859024 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba68107920 0x7fba681a55c0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fba6000cce0 tx=0x7fba6000a620 comp rx=0 tx=0).stop 2024-09-13T18:27:03.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.478+0000 7fba6d648640 1 -- 172.21.15.114:0/1063859024 shutdown_connections 2024-09-13T18:27:03.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.478+0000 7fba6d648640 1 --2- 172.21.15.114:0/1063859024 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fba500a1410 0x7fba500a38d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.478+0000 7fba6d648640 1 --2- 172.21.15.114:0/1063859024 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba68119400 0x7fba681a99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.478+0000 7fba6d648640 1 --2- 172.21.15.114:0/1063859024 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fba68107920 0x7fba681a55c0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.478+0000 7fba6d648640 1 --2- 172.21.15.114:0/1063859024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba68107540 0x7fba681a5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.478+0000 7fba6d648640 1 -- 172.21.15.114:0/1063859024 >> 172.21.15.114:0/1063859024 conn(0x7fba6806afe0 msgr2=0x7fba68119b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:03.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.479+0000 7fba6d648640 1 -- 172.21.15.114:0/1063859024 shutdown_connections 2024-09-13T18:27:03.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.479+0000 7fba6d648640 1 -- 172.21.15.114:0/1063859024 wait complete. 2024-09-13T18:27:03.518 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-09-13T18:27:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.548+0000 7f9191fdf640 1 -- 172.21.15.114:0/1036587854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f918c119400 msgr2=0x7f918c11b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.548+0000 7f9191fdf640 1 --2- 172.21.15.114:0/1036587854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f918c119400 0x7f918c11b7f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f918401ace0 tx=0x7f918403b990 comp rx=0 tx=0).stop 2024-09-13T18:27:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.548+0000 7f9191fdf640 1 -- 172.21.15.114:0/1036587854 shutdown_connections 2024-09-13T18:27:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.548+0000 7f9191fdf640 1 --2- 172.21.15.114:0/1036587854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f918c119400 0x7f918c11b7f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.548+0000 7f9191fdf640 1 --2- 172.21.15.114:0/1036587854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f918c107890 0x7f918c107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.548+0000 7f9191fdf640 1 --2- 172.21.15.114:0/1036587854 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f918c1074b0 0x7f918c10e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.548+0000 7f9191fdf640 1 -- 172.21.15.114:0/1036587854 >> 172.21.15.114:0/1036587854 conn(0x7f918c06adb0 msgr2=0x7f918c06b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.550+0000 7f9191fdf640 1 -- 172.21.15.114:0/1036587854 shutdown_connections 2024-09-13T18:27:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.551+0000 7f9191fdf640 1 -- 172.21.15.114:0/1036587854 wait complete. 2024-09-13T18:27:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.554+0000 7f9191fdf640 1 Processor -- start 2024-09-13T18:27:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.555+0000 7f9191fdf640 1 -- start start 2024-09-13T18:27:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.555+0000 7f9191fdf640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f918c1074b0 0x7f918c1148f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.555+0000 7f9191fdf640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f918c107890 0x7f918c114e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.555+0000 7f9191fdf640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f918c119400 0x7f918c1a1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.555+0000 7f9191fdf640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f918c1a25e0 con 0x7f918c1074b0 2024-09-13T18:27:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.555+0000 7f9191fdf640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f918c1a2750 con 0x7f918c119400 2024-09-13T18:27:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.555+0000 7f9191fdf640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f918c1a2910 con 0x7f918c107890 2024-09-13T18:27:03.574 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.556+0000 7f91917de640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f918c119400 0x7f918c1a1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.575 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.556+0000 7f91917de640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f918c119400 0x7f918c1a1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:35938/0 (socket says 172.21.15.114:35938) 2024-09-13T18:27:03.575 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.556+0000 7f91917de640 1 -- 172.21.15.114:0/248873626 learned_addr learned my addr 172.21.15.114:0/248873626 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.556+0000 7f918bfff640 1 --2- 172.21.15.114:0/248873626 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f918c107890 0x7f918c114e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.558+0000 7f9190fdd640 1 --2- 172.21.15.114:0/248873626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f918c1074b0 0x7f918c1148f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.558+0000 7f91917de640 1 -- 172.21.15.114:0/248873626 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f918c107890 msgr2=0x7f918c114e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.558+0000 7f91917de640 1 --2- 172.21.15.114:0/248873626 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f918c107890 0x7f918c114e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.558+0000 7f91917de640 1 -- 172.21.15.114:0/248873626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f918c1074b0 msgr2=0x7f918c1148f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.559+0000 7f91917de640 1 --2- 172.21.15.114:0/248873626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f918c1074b0 0x7f918c1148f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.559+0000 7f91917de640 1 -- 172.21.15.114:0/248873626 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f918401a910 con 0x7f918c119400 2024-09-13T18:27:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.559+0000 7f91917de640 1 --2- 172.21.15.114:0/248873626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f918c119400 0x7f918c1a1fe0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f918403beb0 tx=0x7f9184003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.560+0000 7f9189ffb640 1 -- 172.21.15.114:0/248873626 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9184041440 con 0x7f918c119400 2024-09-13T18:27:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.560+0000 7f9191fdf640 1 -- 172.21.15.114:0/248873626 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f918c1a2b90 con 0x7f918c119400 2024-09-13T18:27:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.560+0000 7f9189ffb640 1 -- 172.21.15.114:0/248873626 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9184040da0 con 0x7f918c119400 2024-09-13T18:27:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.560+0000 7f9189ffb640 1 -- 172.21.15.114:0/248873626 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91840046c0 con 0x7f918c119400 2024-09-13T18:27:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.561+0000 7f9191fdf640 1 -- 172.21.15.114:0/248873626 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f918c1a2e60 con 0x7f918c119400 2024-09-13T18:27:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.572+0000 7f9196047740 1 Processor -- start 2024-09-13T18:27:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.572+0000 7f9196047740 1 -- start start 2024-09-13T18:27:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.573+0000 7f9196047740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556752bb6d90 0x556752bb55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.573+0000 7f9196047740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556752bb5b00 0x556752bb5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.573+0000 7f9196047740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556752be2460 0x556752be4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.573+0000 7f9196047740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x556752b20c30 con 0x556752be2460 2024-09-13T18:27:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.573+0000 7f9196047740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556752b1e3c0 con 0x556752bb5b00 2024-09-13T18:27:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.573+0000 7f9196047740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556752b19cb0 con 0x556752bb6d90 2024-09-13T18:27:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.573+0000 7f9190fdd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556752bb5b00 0x556752bb5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.573+0000 7f9190fdd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556752bb5b00 0x556752bb5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:35946/0 (socket says 172.21.15.114:35946) 2024-09-13T18:27:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.573+0000 7f9190fdd640 1 -- 172.21.15.114:0/3478462614 learned_addr learned my addr 172.21.15.114:0/3478462614 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.573+0000 7f9190fdd640 1 -- 172.21.15.114:0/3478462614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556752bb6d90 msgr2=0x556752bb55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.573+0000 7f9190fdd640 1 --2- 172.21.15.114:0/3478462614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556752bb6d90 0x556752bb55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.573+0000 7f9190fdd640 1 -- 172.21.15.114:0/3478462614 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556752be2460 msgr2=0x556752be4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.573+0000 7f9190fdd640 1 --2- 172.21.15.114:0/3478462614 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556752be2460 0x556752be4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.573+0000 7f9190fdd640 1 -- 172.21.15.114:0/3478462614 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556752ac0910 con 0x556752bb5b00 2024-09-13T18:27:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.574+0000 7f9190fdd640 1 --2- 172.21.15.114:0/3478462614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556752bb5b00 0x556752bb5f40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f918000bcb0 tx=0x7f918000f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4fe932a96e8d5a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.577+0000 7f9189ffb640 1 -- 172.21.15.114:0/248873626 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f918403f050 con 0x7f918c119400 2024-09-13T18:27:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.575+0000 7f9191fdf640 1 -- 172.21.15.114:0/3478462614 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918001a460 con 0x556752bb5b00 2024-09-13T18:27:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.578+0000 7f9191fdf640 1 -- 172.21.15.114:0/3478462614 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f918001aa50 con 0x556752bb5b00 2024-09-13T18:27:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.578+0000 7f9191fdf640 1 -- 172.21.15.114:0/3478462614 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9180011a00 con 0x556752bb5b00 2024-09-13T18:27:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.578+0000 7f9189ffb640 1 --2- 172.21.15.114:0/248873626 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f91740a1480 0x7f91740a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.579+0000 7f9189ffb640 1 -- 172.21.15.114:0/248873626 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f918410f860 con 0x7f918c119400 2024-09-13T18:27:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.579+0000 7f9196047740 1 -- 172.21.15.114:0/3478462614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556752bb5b00 msgr2=0x556752bb5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.579+0000 7f9196047740 1 --2- 172.21.15.114:0/3478462614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556752bb5b00 0x556752bb5f40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f918000bcb0 tx=0x7f918000f730 comp rx=0 tx=0).stop 2024-09-13T18:27:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.580+0000 7f918bfff640 1 --2- 172.21.15.114:0/248873626 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f91740a1480 0x7f91740a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.580+0000 7f9196047740 1 -- 172.21.15.114:0/3478462614 shutdown_connections 2024-09-13T18:27:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.580+0000 7f9196047740 1 --2- 172.21.15.114:0/3478462614 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556752be2460 0x556752be4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.580+0000 7f9196047740 1 --2- 172.21.15.114:0/3478462614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556752bb5b00 0x556752bb5f40 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.580+0000 7f9196047740 1 --2- 172.21.15.114:0/3478462614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556752bb6d90 0x556752bb55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.580+0000 7f9196047740 1 -- 172.21.15.114:0/3478462614 >> 172.21.15.114:0/3478462614 conn(0x556752bb1d60 msgr2=0x556752bb2140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.581+0000 7f9196047740 1 -- 172.21.15.114:0/3478462614 shutdown_connections 2024-09-13T18:27:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.581+0000 7f918bfff640 1 --2- 172.21.15.114:0/248873626 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f91740a1480 0x7f91740a3940 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x556752bb30e0 tx=0x7f917c00a7c0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.587 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.583+0000 7f9196047740 1 -- 172.21.15.114:0/3478462614 wait complete. 2024-09-13T18:27:03.587 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.583+0000 7f9196047740 1 -- start start 2024-09-13T18:27:03.587 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.583+0000 7f9196047740 1 Processor -- start 2024-09-13T18:27:03.588 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.585+0000 7f9196047740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556752bb5b00 0x556752bbbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.588 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.585+0000 7f9196047740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556752bb6d90 0x556752bb9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.586+0000 7f9196047740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556752be2460 0x556752bb97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.586+0000 7f9196047740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x556752ad7e80 con 0x556752bb6d90 2024-09-13T18:27:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.586+0000 7f9196047740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556752ad7b20 con 0x556752be2460 2024-09-13T18:27:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.586+0000 7f9196047740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556752a9d260 con 0x556752bb5b00 2024-09-13T18:27:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.586+0000 7f9196047740 10 client.0 _ll_register_callbacks cb 0x556752ad2df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:27:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.588+0000 7f9190fdd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556752bb5b00 0x556752bbbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.588+0000 7f9190fdd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556752bb5b00 0x556752bbbe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:59936/0 (socket says 172.21.15.114:59936) 2024-09-13T18:27:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.588+0000 7f9190fdd640 1 -- 172.21.15.114:0/948872383 learned_addr learned my addr 172.21.15.114:0/948872383 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.589+0000 7f9190fdd640 1 -- 172.21.15.114:0/948872383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556752be2460 msgr2=0x556752bb97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.589+0000 7f9190fdd640 1 --2- 172.21.15.114:0/948872383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556752be2460 0x556752bb97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.589+0000 7f9190fdd640 1 -- 172.21.15.114:0/948872383 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556752bb6d90 msgr2=0x556752bb9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.589+0000 7f9190fdd640 1 --2- 172.21.15.114:0/948872383 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556752bb6d90 0x556752bb9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.589+0000 7f9190fdd640 1 -- 172.21.15.114:0/948872383 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f918000b8e0 con 0x556752bb5b00 2024-09-13T18:27:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.590+0000 7f9190fdd640 1 --2- 172.21.15.114:0/948872383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556752bb5b00 0x556752bbbe30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9180011580 tx=0x7f918000bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70afb3ec7bddc3b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.590+0000 7f916bfff640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:27:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.594+0000 7f916bfff640 1 -- 172.21.15.114:0/948872383 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918001a660 con 0x556752bb5b00 2024-09-13T18:27:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.594+0000 7f916bfff640 1 -- 172.21.15.114:0/948872383 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f918001a7c0 con 0x556752bb5b00 2024-09-13T18:27:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.594+0000 7f9196047740 1 -- 172.21.15.114:0/948872383 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556752b42680 con 0x556752bb5b00 2024-09-13T18:27:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.599+0000 7f916bfff640 1 -- 172.21.15.114:0/948872383 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9180011780 con 0x556752bb5b00 2024-09-13T18:27:03.606 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.603+0000 7f9196047740 10 client.26482 fetch_fsmap learned FSMap version 32 2024-09-13T18:27:03.606 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.603+0000 7f9196047740 1 -- 172.21.15.114:0/948872383 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x556752b18330 con 0x556752bb5b00 2024-09-13T18:27:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.607+0000 7f916bfff640 1 -- 172.21.15.114:0/948872383 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x556752b42680 con 0x556752bb5b00 2024-09-13T18:27:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.607+0000 7f916bfff640 1 -- 172.21.15.114:0/948872383 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f9180022070 con 0x556752bb5b00 2024-09-13T18:27:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.608+0000 7f9196047740 10 client.26482 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:27:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.608+0000 7f9196047740 10 client.26482 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:27:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.608+0000 7f9196047740 20 client.26482 populate_metadata read hostname 'smithi114' 2024-09-13T18:27:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.608+0000 7f9196047740 1 --2- 172.21.15.114:0/948872383 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x556752bf2150 0x556752bf45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.608+0000 7f9196047740 4 client.26482 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:27:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.608+0000 7f9196047740 1 -- 172.21.15.114:0/948872383 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556752b0d5d0 con 0x556752bf2150 2024-09-13T18:27:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.609+0000 7f918bfff640 1 --2- 172.21.15.114:0/948872383 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x556752bf2150 0x556752bf45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.612+0000 7f918bfff640 1 --2- 172.21.15.114:0/948872383 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x556752bf2150 0x556752bf45d0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c7aea554172f316 server_cookie=3ba93d38d943b1d in_seq=0 out_seq=0 2024-09-13T18:27:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.612+0000 7f916bfff640 10 client.26482 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:27:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.612+0000 7f916bfff640 1 -- 172.21.15.114:0/948872383 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x556752b0d5d0 con 0x556752bf2150 2024-09-13T18:27:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.613+0000 7f916bfff640 10 client.26482 handle_command_reply: tid=0 2024-09-13T18:27:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.613+0000 7f9196047740 1 client.26482 shutdown 2024-09-13T18:27:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.613+0000 7f9196047740 1 -- 172.21.15.114:0/948872383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556752bb5b00 msgr2=0x556752bbbe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.613+0000 7f9196047740 1 --2- 172.21.15.114:0/948872383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556752bb5b00 0x556752bbbe30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9180011580 tx=0x7f918000bfd0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.613+0000 7f9196047740 1 -- 172.21.15.114:0/948872383 shutdown_connections 2024-09-13T18:27:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.613+0000 7f9196047740 1 --2- 172.21.15.114:0/948872383 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x556752bf2150 0x556752bf45d0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.613+0000 7f9196047740 1 --2- 172.21.15.114:0/948872383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556752be2460 0x556752bb97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.613+0000 7f9196047740 1 --2- 172.21.15.114:0/948872383 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556752bb6d90 0x556752bb9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.613+0000 7f9196047740 1 --2- 172.21.15.114:0/948872383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556752bb5b00 0x556752bbbe30 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.613+0000 7f9196047740 1 -- 172.21.15.114:0/948872383 >> 172.21.15.114:0/948872383 conn(0x556752bb1d60 msgr2=0x556752bb7a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.613+0000 7f916bfff640 0 client.26482 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:27:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.613+0000 7f9196047740 1 -- 172.21.15.114:0/948872383 shutdown_connections 2024-09-13T18:27:03.618 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.614+0000 7f9196047740 1 -- 172.21.15.114:0/948872383 wait complete. 2024-09-13T18:27:03.618 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.614+0000 7f9196047740 20 client.26482 trim_cache size 0 max 16384 2024-09-13T18:27:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.663+0000 7f9196047740 1 Processor -- start 2024-09-13T18:27:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.663+0000 7f9196047740 1 -- start start 2024-09-13T18:27:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.664+0000 7f9196047740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556752bb5b00 0x556752be3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.671 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.664+0000 7f9196047740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556752bb6d90 0x556752bbb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.671 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.664+0000 7f9196047740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556752be2460 0x556752bf8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.671 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.665+0000 7f9196047740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x556752bb6a10 con 0x556752bb5b00 2024-09-13T18:27:03.671 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.665+0000 7f9196047740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556752be4e20 con 0x556752bb6d90 2024-09-13T18:27:03.671 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.665+0000 7f9190fdd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556752bb5b00 0x556752be3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.672 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.665+0000 7f9190fdd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556752bb5b00 0x556752be3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59834/0 (socket says 172.21.15.114:59834) 2024-09-13T18:27:03.672 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.665+0000 7f9190fdd640 1 -- 172.21.15.114:0/1760806437 learned_addr learned my addr 172.21.15.114:0/1760806437 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:03.672 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.665+0000 7f91917de640 1 --2- 172.21.15.114:0/1760806437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556752be2460 0x556752bf8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.672 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.665+0000 7f9196047740 1 -- 172.21.15.114:0/1760806437 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556752bb8210 con 0x556752be2460 2024-09-13T18:27:03.672 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.665+0000 7f9190fdd640 1 -- 172.21.15.114:0/1760806437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556752be2460 msgr2=0x556752bf8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:03.672 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.665+0000 7f9190fdd640 1 --2- 172.21.15.114:0/1760806437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556752be2460 0x556752bf8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.672 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.665+0000 7f9190fdd640 1 -- 172.21.15.114:0/1760806437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556752bb6d90 msgr2=0x556752bbb990 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:27:03.673 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.665+0000 7f9190fdd640 1 --2- 172.21.15.114:0/1760806437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556752bb6d90 0x556752bbb990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.673 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.665+0000 7f9190fdd640 1 -- 172.21.15.114:0/1760806437 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556752aba300 con 0x556752bb5b00 2024-09-13T18:27:03.673 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.666+0000 7f9190fdd640 1 --2- 172.21.15.114:0/1760806437 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556752bb5b00 0x556752be3850 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f9180005850 tx=0x7f918000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6f29e24a911c09c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.678 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.674+0000 7f9172ffd640 1 -- 172.21.15.114:0/1760806437 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9180025600 con 0x556752bb5b00 2024-09-13T18:27:03.679 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.674+0000 7f9172ffd640 1 -- 172.21.15.114:0/1760806437 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f918001f040 con 0x556752bb5b00 2024-09-13T18:27:03.679 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.674+0000 7f9172ffd640 1 -- 172.21.15.114:0/1760806437 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91800119e0 con 0x556752bb5b00 2024-09-13T18:27:03.682 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.675+0000 7f9196047740 1 -- 172.21.15.114:0/1760806437 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556752bb5b00 msgr2=0x556752be3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.682 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.675+0000 7f9196047740 1 --2- 172.21.15.114:0/1760806437 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556752bb5b00 0x556752be3850 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f9180005850 tx=0x7f918000f730 comp rx=0 tx=0).stop 2024-09-13T18:27:03.682 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.678+0000 7f9196047740 1 -- 172.21.15.114:0/1760806437 shutdown_connections 2024-09-13T18:27:03.682 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.678+0000 7f9196047740 1 --2- 172.21.15.114:0/1760806437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556752be2460 0x556752bf8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.683 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.678+0000 7f9196047740 1 --2- 172.21.15.114:0/1760806437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556752bb6d90 0x556752bbb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.683 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.678+0000 7f9196047740 1 --2- 172.21.15.114:0/1760806437 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556752bb5b00 0x556752be3850 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.683 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.678+0000 7f9196047740 1 -- 172.21.15.114:0/1760806437 >> 172.21.15.114:0/1760806437 conn(0x556752bb1d60 msgr2=0x556752bb2140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:03.694 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.690+0000 7f9196047740 1 -- 172.21.15.114:0/1760806437 shutdown_connections 2024-09-13T18:27:03.694 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.690+0000 7f9196047740 1 -- 172.21.15.114:0/1760806437 wait complete. 2024-09-13T18:27:03.694 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.690+0000 7f9196047740 1 -- start start 2024-09-13T18:27:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.690+0000 7f9196047740 1 Processor -- start 2024-09-13T18:27:03.696 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.691+0000 7f9196047740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556752bb5b00 0x556752be5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.697 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.691+0000 7f9196047740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556752bb6d90 0x556752be5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.697 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.692+0000 7f9196047740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556752be2460 0x556752be5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.697 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.692+0000 7f918bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556752bb6d90 0x556752be5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.697 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.692+0000 7f918bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556752bb6d90 0x556752be5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:59954/0 (socket says 172.21.15.114:59954) 2024-09-13T18:27:03.697 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.692+0000 7f918bfff640 1 -- 172.21.15.114:0/1805636330 learned_addr learned my addr 172.21.15.114:0/1805636330 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:03.697 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.692+0000 7f9196047740 1 -- 172.21.15.114:0/1805636330 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x556752c476e0 con 0x556752bb5b00 2024-09-13T18:27:03.697 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.692+0000 7f91917de640 1 --2- 172.21.15.114:0/1805636330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556752be2460 0x556752be5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.697 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.692+0000 7f9196047740 1 -- 172.21.15.114:0/1805636330 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556752c47820 con 0x556752be2460 2024-09-13T18:27:03.697 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.692+0000 7f9196047740 1 -- 172.21.15.114:0/1805636330 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556752c47960 con 0x556752bb6d90 2024-09-13T18:27:03.698 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.693+0000 7f9196047740 10 client.0 _ll_register_callbacks cb 0x7f918000b440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:27:03.698 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.693+0000 7f918bfff640 1 -- 172.21.15.114:0/1805636330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556752be2460 msgr2=0x556752be5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:03.698 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.693+0000 7f918bfff640 1 --2- 172.21.15.114:0/1805636330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556752be2460 0x556752be5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.698 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.693+0000 7f918bfff640 1 -- 172.21.15.114:0/1805636330 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556752bb5b00 msgr2=0x556752be5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:03.698 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.693+0000 7f918bfff640 1 --2- 172.21.15.114:0/1805636330 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556752bb5b00 0x556752be5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.698 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.693+0000 7f918bfff640 1 -- 172.21.15.114:0/1805636330 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556752b18330 con 0x556752bb6d90 2024-09-13T18:27:03.698 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.693+0000 7f918bfff640 1 --2- 172.21.15.114:0/1805636330 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556752bb6d90 0x556752be5890 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f918c115850 tx=0x7f917c00a2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=136349cc69775027 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.698 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.693+0000 7f9196047740 1 -- 172.21.15.114:0/1805636330 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f918001aa50 con 0x556752bb6d90 2024-09-13T18:27:03.700 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.696+0000 7f916bfff640 10 client.26488 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:27:03.700 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.696+0000 7f916bfff640 1 -- 172.21.15.114:0/1805636330 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917c0247f0 con 0x556752bb6d90 2024-09-13T18:27:03.700 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.696+0000 7f916bfff640 1 -- 172.21.15.114:0/1805636330 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f917c021050 con 0x556752bb6d90 2024-09-13T18:27:03.700 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.696+0000 7f916bfff640 1 -- 172.21.15.114:0/1805636330 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f917c031560 con 0x556752bb6d90 2024-09-13T18:27:03.700 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.696+0000 7f9196047740 10 client.26488 fetch_fsmap learned FSMap version 32 2024-09-13T18:27:03.700 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.696+0000 7f9196047740 1 -- 172.21.15.114:0/1805636330 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({fsmap=32}) v3 -- 0x556752adf8e0 con 0x556752bb6d90 2024-09-13T18:27:03.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.701+0000 7f916bfff640 1 -- 172.21.15.114:0/1805636330 <== mon.1 v2:172.21.15.204:3300/0 4 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f917c018040 con 0x556752bb6d90 2024-09-13T18:27:03.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.701+0000 7f9196047740 10 client.26488 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:27:03.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.701+0000 7f9196047740 10 client.26488 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:27:03.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.701+0000 7f9196047740 20 client.26488 populate_metadata read hostname 'smithi114' 2024-09-13T18:27:03.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.701+0000 7f9196047740 1 --2- 172.21.15.114:0/1805636330 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x556752c49140 0x556752c4b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.701+0000 7f9196047740 4 client.26488 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T18:27:03.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.701+0000 7f9196047740 1 -- 172.21.15.114:0/1805636330 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7f918001aa50 con 0x556752c49140 2024-09-13T18:27:03.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.701+0000 7f9190fdd640 1 --2- 172.21.15.114:0/1805636330 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x556752c49140 0x556752c4b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.702+0000 7f916bfff640 10 client.26488 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:27:03.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.702+0000 7f9190fdd640 1 --2- 172.21.15.114:0/1805636330 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x556752c49140 0x556752c4b560 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbe9e54c41f5dfae server_cookie=b3d7075604ebe685 in_seq=0 out_seq=0 2024-09-13T18:27:03.731 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.728+0000 7f916bfff640 1 -- 172.21.15.114:0/1805636330 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f918001aa50 con 0x556752c49140 2024-09-13T18:27:03.732 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.728+0000 7f916bfff640 10 client.26488 handle_command_reply: tid=0 2024-09-13T18:27:03.732 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.728+0000 7f9196047740 1 client.26488 shutdown 2024-09-13T18:27:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.735+0000 7f9196047740 1 -- 172.21.15.114:0/1805636330 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556752bb6d90 msgr2=0x556752be5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.738 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.735+0000 7f9196047740 1 --2- 172.21.15.114:0/1805636330 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556752bb6d90 0x556752be5890 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f918c115850 tx=0x7f917c00a2d0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.738+0000 7f9196047740 1 -- 172.21.15.114:0/1805636330 shutdown_connections 2024-09-13T18:27:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.738+0000 7f9196047740 1 --2- 172.21.15.114:0/1805636330 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x556752c49140 0x556752c4b560 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.738+0000 7f9196047740 1 --2- 172.21.15.114:0/1805636330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556752be2460 0x556752be5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.738+0000 7f9196047740 1 --2- 172.21.15.114:0/1805636330 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556752bb6d90 0x556752be5890 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.738+0000 7f9196047740 1 --2- 172.21.15.114:0/1805636330 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556752bb5b00 0x556752be5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.741 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.738+0000 7f916bfff640 0 client.26488 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:27:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.738+0000 7f9196047740 1 -- 172.21.15.114:0/1805636330 >> 172.21.15.114:0/1805636330 conn(0x556752bb1d60 msgr2=0x556752bb2140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:03.743 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.740+0000 7f9196047740 1 -- 172.21.15.114:0/1805636330 shutdown_connections 2024-09-13T18:27:03.743 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.740+0000 7f9196047740 1 -- 172.21.15.114:0/1805636330 wait complete. 2024-09-13T18:27:03.746 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.743+0000 7f9196047740 20 client.26488 trim_cache size 0 max 16384 2024-09-13T18:27:03.747 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-13T18:27:03.747 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-13T18:27:03.747 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-13T18:27:03.747 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-13T18:27:03.748 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.745+0000 7f91737fe640 1 -- 172.21.15.114:0/248873626 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f91740a1480 msgr2=0x7f91740a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.748 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.745+0000 7f91737fe640 1 --2- 172.21.15.114:0/248873626 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f91740a1480 0x7f91740a3940 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x556752bb30e0 tx=0x7f917c00a7c0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.749 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.745+0000 7f91737fe640 1 -- 172.21.15.114:0/248873626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f918c119400 msgr2=0x7f918c1a1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.749 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.745+0000 7f91737fe640 1 --2- 172.21.15.114:0/248873626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f918c119400 0x7f918c1a1fe0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f918403beb0 tx=0x7f9184003ba0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.750 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.746+0000 7f91737fe640 1 -- 172.21.15.114:0/248873626 shutdown_connections 2024-09-13T18:27:03.750 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.746+0000 7f91737fe640 1 --2- 172.21.15.114:0/248873626 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f91740a1480 0x7f91740a3940 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.750 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.746+0000 7f91737fe640 1 --2- 172.21.15.114:0/248873626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f918c119400 0x7f918c1a1fe0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.750 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.746+0000 7f91737fe640 1 --2- 172.21.15.114:0/248873626 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f918c107890 0x7f918c114e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.750 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.746+0000 7f91737fe640 1 --2- 172.21.15.114:0/248873626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f918c1074b0 0x7f918c1148f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.750 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.746+0000 7f91737fe640 1 -- 172.21.15.114:0/248873626 >> 172.21.15.114:0/248873626 conn(0x7f918c06adb0 msgr2=0x7f918c06fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:03.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.752+0000 7f91737fe640 1 -- 172.21.15.114:0/248873626 shutdown_connections 2024-09-13T18:27:03.757 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:03.754+0000 7f91737fe640 1 -- 172.21.15.114:0/248873626 wait complete. 2024-09-13T18:27:03.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.847+0000 7f99008b8640 1 -- 172.21.15.114:0/114375004 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98fc107540 msgr2=0x7f98fc10e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.847+0000 7f99008b8640 1 --2- 172.21.15.114:0/114375004 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98fc107540 0x7f98fc10e510 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f98ec009900 tx=0x7f98ec02a420 comp rx=0 tx=0).stop 2024-09-13T18:27:03.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.847+0000 7f99008b8640 1 -- 172.21.15.114:0/114375004 shutdown_connections 2024-09-13T18:27:03.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.847+0000 7f99008b8640 1 --2- 172.21.15.114:0/114375004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98fc119400 0x7f98fc11b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.847+0000 7f99008b8640 1 --2- 172.21.15.114:0/114375004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98fc107920 0x7f98fc107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.847+0000 7f99008b8640 1 --2- 172.21.15.114:0/114375004 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98fc107540 0x7f98fc10e510 secure :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f98ec009900 tx=0x7f98ec02a420 comp rx=0 tx=0).stop 2024-09-13T18:27:03.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.847+0000 7f99008b8640 1 -- 172.21.15.114:0/114375004 >> 172.21.15.114:0/114375004 conn(0x7f98fc06afe0 msgr2=0x7f98fc06b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:03.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.856+0000 7f99008b8640 1 -- 172.21.15.114:0/114375004 shutdown_connections 2024-09-13T18:27:03.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.860+0000 7f99008b8640 1 -- 172.21.15.114:0/114375004 wait complete. 2024-09-13T18:27:03.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.861+0000 7f99008b8640 1 Processor -- start 2024-09-13T18:27:03.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.862+0000 7f99008b8640 1 -- start start 2024-09-13T18:27:03.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.862+0000 7f99008b8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98fc107920 0x7f98fc116b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.862+0000 7f99008b8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98fc119400 0x7f98fc10fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.862+0000 7f99008b8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98fc110100 0x7f98fc1105b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.862+0000 7f99008b8640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc1146d0 con 0x7f98fc110100 2024-09-13T18:27:03.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.862+0000 7f99008b8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc114840 con 0x7f98fc107920 2024-09-13T18:27:03.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.862+0000 7f99008b8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc114a00 con 0x7f98fc119400 2024-09-13T18:27:03.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.862+0000 7f98fad9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98fc110100 0x7f98fc1105b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.862+0000 7f98f9d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98fc119400 0x7f98fc10fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.862+0000 7f98fad9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98fc110100 0x7f98fc1105b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59864/0 (socket says 172.21.15.114:59864) 2024-09-13T18:27:03.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.862+0000 7f98fad9d640 1 -- 172.21.15.114:0/4279661958 learned_addr learned my addr 172.21.15.114:0/4279661958 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:03.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.863+0000 7f98fad9d640 1 -- 172.21.15.114:0/4279661958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98fc119400 msgr2=0x7f98fc10fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.863+0000 7f98fad9d640 1 --2- 172.21.15.114:0/4279661958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98fc119400 0x7f98fc10fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.863+0000 7f98fad9d640 1 -- 172.21.15.114:0/4279661958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98fc107920 msgr2=0x7f98fc116b40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:27:03.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.863+0000 7f98fad9d640 1 --2- 172.21.15.114:0/4279661958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98fc107920 0x7f98fc116b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.863+0000 7f98fad9d640 1 -- 172.21.15.114:0/4279661958 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98ec009560 con 0x7f98fc110100 2024-09-13T18:27:03.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.864+0000 7f98fad9d640 1 --2- 172.21.15.114:0/4279661958 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98fc110100 0x7f98fc1105b0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f98f400cad0 tx=0x7f98f400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.865+0000 7f98eb7fe640 1 -- 172.21.15.114:0/4279661958 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f400ae20 con 0x7f98fc110100 2024-09-13T18:27:03.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.865+0000 7f98eb7fe640 1 -- 172.21.15.114:0/4279661958 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f98f4005020 con 0x7f98fc110100 2024-09-13T18:27:03.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.865+0000 7f98eb7fe640 1 -- 172.21.15.114:0/4279661958 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f400f6b0 con 0x7f98fc110100 2024-09-13T18:27:03.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.865+0000 7f99008b8640 1 -- 172.21.15.114:0/4279661958 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98fc114c80 con 0x7f98fc110100 2024-09-13T18:27:03.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.865+0000 7f99008b8640 1 -- 172.21.15.114:0/4279661958 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98fc115140 con 0x7f98fc110100 2024-09-13T18:27:03.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.870+0000 7f98eb7fe640 1 -- 172.21.15.114:0/4279661958 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f98f4005190 con 0x7f98fc110100 2024-09-13T18:27:03.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.871+0000 7f98eb7fe640 1 --2- 172.21.15.114:0/4279661958 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f98e40a1410 0x7f98e40a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.872+0000 7f98eb7fe640 1 -- 172.21.15.114:0/4279661958 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f98f40ccf50 con 0x7f98fc110100 2024-09-13T18:27:03.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.873+0000 7f98fa59c640 1 --2- 172.21.15.114:0/4279661958 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f98e40a1410 0x7f98e40a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.875+0000 7f99035da740 1 Processor -- start 2024-09-13T18:27:03.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.875+0000 7f99035da740 1 -- start start 2024-09-13T18:27:03.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.875+0000 7f99035da740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e02de2790 0x561e02de2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.875+0000 7f99035da740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561e02de30b0 0x561e02ded410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.875+0000 7f99035da740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e02ded950 0x561e02defd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.875+0000 7f99035da740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e02d4ccc0 con 0x561e02de30b0 2024-09-13T18:27:03.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.875+0000 7f99035da740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e02d4a450 con 0x561e02de2790 2024-09-13T18:27:03.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.875+0000 7f99035da740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e02d45d40 con 0x561e02ded950 2024-09-13T18:27:03.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.875+0000 7f98fad9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e02ded950 0x561e02defd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.876+0000 7f98fad9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e02ded950 0x561e02defd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:59984/0 (socket says 172.21.15.114:59984) 2024-09-13T18:27:03.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.876+0000 7f98fad9d640 1 -- 172.21.15.114:0/960844645 learned_addr learned my addr 172.21.15.114:0/960844645 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:03.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.876+0000 7f98fa59c640 1 --2- 172.21.15.114:0/960844645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e02de2790 0x561e02de2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.876+0000 7f98fad9d640 1 -- 172.21.15.114:0/960844645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e02de2790 msgr2=0x561e02de2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:03.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.876+0000 7f98fad9d640 1 --2- 172.21.15.114:0/960844645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e02de2790 0x561e02de2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.876+0000 7f98fad9d640 1 -- 172.21.15.114:0/960844645 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561e02de30b0 msgr2=0x561e02ded410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:27:03.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.876+0000 7f98fad9d640 1 --2- 172.21.15.114:0/960844645 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561e02de30b0 0x561e02ded410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.876+0000 7f98fad9d640 1 -- 172.21.15.114:0/960844645 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e02d0d620 con 0x561e02ded950 2024-09-13T18:27:03.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.876+0000 7f98fad9d640 1 --2- 172.21.15.114:0/960844645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e02ded950 0x561e02defd40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f98f400a6e0 tx=0x7f98f4005490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9217c940e6ded130 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.877+0000 7f98fa59c640 1 --2- 172.21.15.114:0/4279661958 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f98e40a1410 0x7f98e40a38d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f98ec034040 tx=0x7f98ec005fd0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.879+0000 7f98e8ff9640 1 -- 172.21.15.114:0/960844645 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f409dae0 con 0x561e02ded950 2024-09-13T18:27:03.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.879+0000 7f98e8ff9640 1 -- 172.21.15.114:0/960844645 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f98f40a2130 con 0x561e02ded950 2024-09-13T18:27:03.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.880+0000 7f99035da740 1 -- 172.21.15.114:0/960844645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e02ded950 msgr2=0x561e02defd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.880+0000 7f99035da740 1 --2- 172.21.15.114:0/960844645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e02ded950 0x561e02defd40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f98f400a6e0 tx=0x7f98f4005490 comp rx=0 tx=0).stop 2024-09-13T18:27:03.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.880+0000 7f99035da740 1 -- 172.21.15.114:0/960844645 shutdown_connections 2024-09-13T18:27:03.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.880+0000 7f99035da740 1 --2- 172.21.15.114:0/960844645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e02ded950 0x561e02defd40 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.880+0000 7f99035da740 1 --2- 172.21.15.114:0/960844645 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561e02de30b0 0x561e02ded410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.880+0000 7f99035da740 1 --2- 172.21.15.114:0/960844645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e02de2790 0x561e02de2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.880+0000 7f99035da740 1 -- 172.21.15.114:0/960844645 >> 172.21.15.114:0/960844645 conn(0x561e02dded00 msgr2=0x561e02ddf0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:03.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.880+0000 7f99035da740 1 -- 172.21.15.114:0/960844645 shutdown_connections 2024-09-13T18:27:03.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.881+0000 7f99035da740 1 -- 172.21.15.114:0/960844645 wait complete. 2024-09-13T18:27:03.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.881+0000 7f99035da740 1 -- start start 2024-09-13T18:27:03.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.881+0000 7f99035da740 1 Processor -- start 2024-09-13T18:27:03.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.881+0000 7f99035da740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e02de2790 0x561e02decb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.881+0000 7f99035da740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561e02de30b0 0x561e02ded070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.881+0000 7f99035da740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e02def1e0 0x561e02def660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.881+0000 7f99035da740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e02d04180 con 0x561e02de30b0 2024-09-13T18:27:03.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.881+0000 7f99035da740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e02a1e430 con 0x561e02de2790 2024-09-13T18:27:03.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.881+0000 7f99035da740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e02cc95e0 con 0x561e02def1e0 2024-09-13T18:27:03.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.882+0000 7f99035da740 10 client.0 _ll_register_callbacks cb 0x561e02d35aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:27:03.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.882+0000 7f98f9d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561e02de30b0 0x561e02ded070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.882+0000 7f98f9d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561e02de30b0 0x561e02ded070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59880/0 (socket says 172.21.15.114:59880) 2024-09-13T18:27:03.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.882+0000 7f98f9d9b640 1 -- 172.21.15.114:0/2165831972 learned_addr learned my addr 172.21.15.114:0/2165831972 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:03.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.883+0000 7f98f9d9b640 1 -- 172.21.15.114:0/2165831972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e02def1e0 msgr2=0x561e02def660 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:27:03.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.883+0000 7f98f9d9b640 1 --2- 172.21.15.114:0/2165831972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e02def1e0 0x561e02def660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.883+0000 7f98f9d9b640 1 -- 172.21.15.114:0/2165831972 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e02de2790 msgr2=0x561e02decb30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:27:03.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.883+0000 7f98f9d9b640 1 --2- 172.21.15.114:0/2165831972 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e02de2790 0x561e02decb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.883+0000 7f98f9d9b640 1 -- 172.21.15.114:0/2165831972 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98fc115140 con 0x561e02de30b0 2024-09-13T18:27:03.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.883+0000 7f98f9d9b640 1 --2- 172.21.15.114:0/2165831972 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561e02de30b0 0x561e02ded070 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f98f000be40 tx=0x7f98f00107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=681587899a963d41 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.883+0000 7f98e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:27:03.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.884+0000 7f98e1ffb640 1 -- 172.21.15.114:0/2165831972 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f0004230 con 0x561e02de30b0 2024-09-13T18:27:03.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.884+0000 7f99035da740 1 -- 172.21.15.114:0/2165831972 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561e02d381f0 con 0x561e02de30b0 2024-09-13T18:27:03.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.884+0000 7f98e1ffb640 1 -- 172.21.15.114:0/2165831972 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f98f0017070 con 0x561e02de30b0 2024-09-13T18:27:03.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.884+0000 7f98e1ffb640 1 -- 172.21.15.114:0/2165831972 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98f0004390 con 0x561e02de30b0 2024-09-13T18:27:03.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.884+0000 7f99035da740 10 client.17124 fetch_fsmap learned FSMap version 32 2024-09-13T18:27:03.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.884+0000 7f99035da740 1 -- 172.21.15.114:0/2165831972 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x561e02d443c0 con 0x561e02de30b0 2024-09-13T18:27:03.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.885+0000 7f98e1ffb640 1 -- 172.21.15.114:0/2165831972 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561e02d381f0 con 0x561e02de30b0 2024-09-13T18:27:03.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.886+0000 7f98e1ffb640 1 -- 172.21.15.114:0/2165831972 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f98f0003770 con 0x561e02de30b0 2024-09-13T18:27:03.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.886+0000 7f99035da740 10 client.17124 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:27:03.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.886+0000 7f99035da740 10 client.17124 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-13T18:27:03.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.886+0000 7f99035da740 20 client.17124 populate_metadata read hostname 'smithi114' 2024-09-13T18:27:03.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.886+0000 7f99035da740 1 --2- 172.21.15.114:0/2165831972 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x561e02dfd750 0x561e02dffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.886+0000 7f99035da740 4 client.17124 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:27:03.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.886+0000 7f99035da740 1 -- 172.21.15.114:0/2165831972 --> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561e02a394e0 con 0x561e02dfd750 2024-09-13T18:27:03.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.891+0000 7f98fa59c640 1 --2- 172.21.15.114:0/2165831972 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x561e02dfd750 0x561e02dffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.895+0000 7f98fa59c640 1 --2- 172.21.15.114:0/2165831972 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x561e02dfd750 0x561e02dffbb0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8b49cd36875a8fac server_cookie=fed5a49ffd034dc8 in_seq=0 out_seq=0 2024-09-13T18:27:03.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.897+0000 7f98e1ffb640 10 client.17124 ms_handle_connect on v2:172.21.15.134:6834/1150003275 2024-09-13T18:27:03.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.898+0000 7f98e1ffb640 1 -- 172.21.15.114:0/2165831972 <== mds.3 v2:172.21.15.134:6834/1150003275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x561e02a394e0 con 0x561e02dfd750 2024-09-13T18:27:03.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.898+0000 7f98e1ffb640 10 client.17124 handle_command_reply: tid=0 2024-09-13T18:27:03.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.898+0000 7f99035da740 1 client.17124 shutdown 2024-09-13T18:27:03.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.900+0000 7f99035da740 1 -- 172.21.15.114:0/2165831972 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561e02de30b0 msgr2=0x561e02ded070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.900+0000 7f99035da740 1 --2- 172.21.15.114:0/2165831972 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561e02de30b0 0x561e02ded070 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f98f000be40 tx=0x7f98f00107a0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.902+0000 7f99035da740 1 -- 172.21.15.114:0/2165831972 shutdown_connections 2024-09-13T18:27:03.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.902+0000 7f99035da740 1 --2- 172.21.15.114:0/2165831972 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x561e02dfd750 0x561e02dffbb0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.902+0000 7f99035da740 1 --2- 172.21.15.114:0/2165831972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e02def1e0 0x561e02def660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.902+0000 7f99035da740 1 --2- 172.21.15.114:0/2165831972 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561e02de30b0 0x561e02ded070 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.902+0000 7f99035da740 1 --2- 172.21.15.114:0/2165831972 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e02de2790 0x561e02decb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.902+0000 7f99035da740 1 -- 172.21.15.114:0/2165831972 >> 172.21.15.114:0/2165831972 conn(0x561e02dded00 msgr2=0x561e02de4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:03.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.903+0000 7f98e1ffb640 0 client.17124 ms_handle_reset on v2:172.21.15.134:6834/1150003275 2024-09-13T18:27:03.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.908+0000 7f99035da740 1 -- 172.21.15.114:0/2165831972 shutdown_connections 2024-09-13T18:27:03.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.909+0000 7f99035da740 1 -- 172.21.15.114:0/2165831972 wait complete. 2024-09-13T18:27:03.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.911+0000 7f99035da740 20 client.17124 trim_cache size 0 max 16384 2024-09-13T18:27:03.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.949+0000 7f99035da740 1 Processor -- start 2024-09-13T18:27:03.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.949+0000 7f99035da740 1 -- start start 2024-09-13T18:27:03.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.950+0000 7f99035da740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561e02de2790 0x561e02df67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.950+0000 7f99035da740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e02de30b0 0x561e02df6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.950+0000 7f98fa59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561e02de2790 0x561e02df67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.950+0000 7f98fa59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561e02de2790 0x561e02df67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59884/0 (socket says 172.21.15.114:59884) 2024-09-13T18:27:03.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.950+0000 7f98fa59c640 1 -- 172.21.15.114:0/3151026300 learned_addr learned my addr 172.21.15.114:0/3151026300 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:03.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.950+0000 7f99035da740 1 --2- 172.21.15.114:0/3151026300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e02def1e0 0x561e02df9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.951+0000 7f99035da740 1 -- 172.21.15.114:0/3151026300 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e02d381f0 con 0x561e02de2790 2024-09-13T18:27:03.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.951+0000 7f99035da740 1 -- 172.21.15.114:0/3151026300 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e02dfbf10 con 0x561e02def1e0 2024-09-13T18:27:03.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.951+0000 7f99035da740 1 -- 172.21.15.114:0/3151026300 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e02ddf7c0 con 0x561e02de30b0 2024-09-13T18:27:03.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.951+0000 7f98fad9d640 1 --2- 172.21.15.114:0/3151026300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e02def1e0 0x561e02df9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.951+0000 7f98f9d9b640 1 --2- 172.21.15.114:0/3151026300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e02de30b0 0x561e02df6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.953+0000 7f98fa59c640 1 -- 172.21.15.114:0/3151026300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e02de30b0 msgr2=0x561e02df6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:03.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.953+0000 7f98fa59c640 1 --2- 172.21.15.114:0/3151026300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e02de30b0 0x561e02df6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.953+0000 7f98fa59c640 1 -- 172.21.15.114:0/3151026300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e02def1e0 msgr2=0x561e02df9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:03.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.953+0000 7f98fa59c640 1 --2- 172.21.15.114:0/3151026300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e02def1e0 0x561e02df9270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.953+0000 7f98fa59c640 1 -- 172.21.15.114:0/3151026300 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e02ce6770 con 0x561e02de2790 2024-09-13T18:27:03.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.953+0000 7f98fa59c640 1 --2- 172.21.15.114:0/3151026300 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561e02de2790 0x561e02df67c0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f98ec004380 tx=0x7f98ec03e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bae17c52e7cf35e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.954+0000 7f98e97fa640 1 -- 172.21.15.114:0/3151026300 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ec0044f0 con 0x561e02de2790 2024-09-13T18:27:03.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.954+0000 7f98e97fa640 1 -- 172.21.15.114:0/3151026300 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f98ec005590 con 0x561e02de2790 2024-09-13T18:27:03.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.954+0000 7f98e97fa640 1 -- 172.21.15.114:0/3151026300 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ec004c20 con 0x561e02de2790 2024-09-13T18:27:03.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.955+0000 7f99035da740 1 -- 172.21.15.114:0/3151026300 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561e02de2790 msgr2=0x561e02df67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.955+0000 7f99035da740 1 --2- 172.21.15.114:0/3151026300 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561e02de2790 0x561e02df67c0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f98ec004380 tx=0x7f98ec03e000 comp rx=0 tx=0).stop 2024-09-13T18:27:03.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.961+0000 7f99035da740 1 -- 172.21.15.114:0/3151026300 shutdown_connections 2024-09-13T18:27:03.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.961+0000 7f99035da740 1 --2- 172.21.15.114:0/3151026300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e02def1e0 0x561e02df9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.961+0000 7f99035da740 1 --2- 172.21.15.114:0/3151026300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e02de30b0 0x561e02df6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.961+0000 7f99035da740 1 --2- 172.21.15.114:0/3151026300 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561e02de2790 0x561e02df67c0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.961+0000 7f99035da740 1 -- 172.21.15.114:0/3151026300 >> 172.21.15.114:0/3151026300 conn(0x561e02dded00 msgr2=0x561e02decb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:03.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.967+0000 7f99035da740 1 -- 172.21.15.114:0/3151026300 shutdown_connections 2024-09-13T18:27:03.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.971+0000 7f99035da740 1 -- 172.21.15.114:0/3151026300 wait complete. 2024-09-13T18:27:03.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.971+0000 7f99035da740 1 -- start start 2024-09-13T18:27:03.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.971+0000 7f99035da740 1 Processor -- start 2024-09-13T18:27:03.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.974+0000 7f99035da740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e02de2790 0x561e02df2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.974+0000 7f99035da740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561e02de30b0 0x561e02df33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.974+0000 7f99035da740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e02def1e0 0x561e02e4bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.974+0000 7f99035da740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e02e4c290 con 0x561e02de30b0 2024-09-13T18:27:03.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.974+0000 7f99035da740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e02e4c400 con 0x561e02de2790 2024-09-13T18:27:03.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.974+0000 7f99035da740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e02e4c5c0 con 0x561e02def1e0 2024-09-13T18:27:03.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.974+0000 7f99035da740 10 client.0 _ll_register_callbacks cb 0x7f98f4005790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:27:03.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.976+0000 7f98f9d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561e02de30b0 0x561e02df33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.976+0000 7f98f9d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561e02de30b0 0x561e02df33a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59900/0 (socket says 172.21.15.114:59900) 2024-09-13T18:27:03.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.976+0000 7f98f9d9b640 1 -- 172.21.15.114:0/1433414696 learned_addr learned my addr 172.21.15.114:0/1433414696 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:03.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.977+0000 7f98fad9d640 1 --2- 172.21.15.114:0/1433414696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e02def1e0 0x561e02e4bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.977+0000 7f98fa59c640 1 --2- 172.21.15.114:0/1433414696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e02de2790 0x561e02df2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.978+0000 7f98f9d9b640 1 -- 172.21.15.114:0/1433414696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e02def1e0 msgr2=0x561e02e4bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:03.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.978+0000 7f98f9d9b640 1 --2- 172.21.15.114:0/1433414696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e02def1e0 0x561e02e4bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.978+0000 7f98f9d9b640 1 -- 172.21.15.114:0/1433414696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e02de2790 msgr2=0x561e02df2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:03.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.978+0000 7f98f9d9b640 1 --2- 172.21.15.114:0/1433414696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e02de2790 0x561e02df2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.978+0000 7f98f9d9b640 1 -- 172.21.15.114:0/1433414696 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98fc11c2e0 con 0x561e02de30b0 2024-09-13T18:27:03.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.978+0000 7f98f9d9b640 1 --2- 172.21.15.114:0/1433414696 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561e02de30b0 0x561e02df33a0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f98f0003b30 tx=0x7f98f0003cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef91582ea02710d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:03.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.979+0000 7f98e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:27:03.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.979+0000 7f98e1ffb640 1 -- 172.21.15.114:0/1433414696 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f0026a40 con 0x561e02de30b0 2024-09-13T18:27:03.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.979+0000 7f98e1ffb640 1 -- 172.21.15.114:0/1433414696 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f98f0000e40 con 0x561e02de30b0 2024-09-13T18:27:03.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.980+0000 7f99035da740 1 -- 172.21.15.114:0/1433414696 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561e02d6fb80 con 0x561e02de30b0 2024-09-13T18:27:03.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.983+0000 7f98e1ffb640 1 -- 172.21.15.114:0/1433414696 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98f0026ba0 con 0x561e02de30b0 2024-09-13T18:27:03.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.986+0000 7f99035da740 10 client.17130 fetch_fsmap learned FSMap version 32 2024-09-13T18:27:03.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.986+0000 7f99035da740 1 -- 172.21.15.114:0/1433414696 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x561e02d0be80 con 0x561e02de30b0 2024-09-13T18:27:03.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.989+0000 7f98e1ffb640 1 -- 172.21.15.114:0/1433414696 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561e02d6fb80 con 0x561e02de30b0 2024-09-13T18:27:03.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.990+0000 7f98e1ffb640 1 -- 172.21.15.114:0/1433414696 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f98f002e020 con 0x561e02de30b0 2024-09-13T18:27:03.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.990+0000 7f99035da740 10 client.17130 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:27:03.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.990+0000 7f99035da740 10 client.17130 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-13T18:27:03.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.990+0000 7f99035da740 20 client.17130 populate_metadata read hostname 'smithi114' 2024-09-13T18:27:03.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.990+0000 7f99035da740 1 --2- 172.21.15.114:0/1433414696 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x561e02e54f40 0x561e02e57360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:03.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.990+0000 7f99035da740 4 client.17130 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:27:03.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.990+0000 7f99035da740 1 -- 172.21.15.114:0/1433414696 --> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561e02a4a7e0 con 0x561e02e54f40 2024-09-13T18:27:03.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.990+0000 7f98fa59c640 1 --2- 172.21.15.114:0/1433414696 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x561e02e54f40 0x561e02e57360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:03.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.993+0000 7f98fa59c640 1 --2- 172.21.15.114:0/1433414696 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x561e02e54f40 0x561e02e57360 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e29ab573ab964876 server_cookie=d0942b511d025eda in_seq=0 out_seq=0 2024-09-13T18:27:03.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.993+0000 7f98e1ffb640 10 client.17130 ms_handle_connect on v2:172.21.15.134:6834/1150003275 2024-09-13T18:27:03.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.994+0000 7f98e1ffb640 1 -- 172.21.15.114:0/1433414696 <== mds.3 v2:172.21.15.134:6834/1150003275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561e02a4a7e0 con 0x561e02e54f40 2024-09-13T18:27:03.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.994+0000 7f98e1ffb640 10 client.17130 handle_command_reply: tid=0 2024-09-13T18:27:03.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.994+0000 7f99035da740 1 client.17130 shutdown 2024-09-13T18:27:03.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.994+0000 7f99035da740 1 -- 172.21.15.114:0/1433414696 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561e02de30b0 msgr2=0x561e02df33a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:03.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.994+0000 7f99035da740 1 --2- 172.21.15.114:0/1433414696 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561e02de30b0 0x561e02df33a0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f98f0003b30 tx=0x7f98f0003cb0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.995+0000 7f99035da740 1 -- 172.21.15.114:0/1433414696 shutdown_connections 2024-09-13T18:27:03.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.995+0000 7f99035da740 1 --2- 172.21.15.114:0/1433414696 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x561e02e54f40 0x561e02e57360 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.995+0000 7f99035da740 1 --2- 172.21.15.114:0/1433414696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e02def1e0 0x561e02e4bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.995+0000 7f99035da740 1 --2- 172.21.15.114:0/1433414696 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561e02de30b0 0x561e02df33a0 secure :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f98f0003b30 tx=0x7f98f0003cb0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.995+0000 7f99035da740 1 --2- 172.21.15.114:0/1433414696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e02de2790 0x561e02df2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:03.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.995+0000 7f99035da740 1 -- 172.21.15.114:0/1433414696 >> 172.21.15.114:0/1433414696 conn(0x561e02dded00 msgr2=0x561e02deb170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:03.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.995+0000 7f98e1ffb640 0 client.17130 ms_handle_reset on v2:172.21.15.134:6834/1150003275 2024-09-13T18:27:04.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:03.997+0000 7f99035da740 1 -- 172.21.15.114:0/1433414696 shutdown_connections 2024-09-13T18:27:04.010 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:27:04.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.003+0000 7f99035da740 1 -- 172.21.15.114:0/1433414696 wait complete. 2024-09-13T18:27:04.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.006+0000 7f99035da740 20 client.17130 trim_cache size 0 max 16384 2024-09-13T18:27:04.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.019+0000 7f99008b8640 1 -- 172.21.15.114:0/4279661958 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f98e40a1410 msgr2=0x7f98e40a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:04.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.019+0000 7f99008b8640 1 --2- 172.21.15.114:0/4279661958 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f98e40a1410 0x7f98e40a38d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f98ec034040 tx=0x7f98ec005fd0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.019+0000 7f99008b8640 1 -- 172.21.15.114:0/4279661958 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98fc110100 msgr2=0x7f98fc1105b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:04.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.019+0000 7f99008b8640 1 --2- 172.21.15.114:0/4279661958 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98fc110100 0x7f98fc1105b0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f98f400cad0 tx=0x7f98f400cf90 comp rx=0 tx=0).stop 2024-09-13T18:27:04.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.023+0000 7f99008b8640 1 -- 172.21.15.114:0/4279661958 shutdown_connections 2024-09-13T18:27:04.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.023+0000 7f99008b8640 1 --2- 172.21.15.114:0/4279661958 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f98e40a1410 0x7f98e40a38d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.023+0000 7f99008b8640 1 --2- 172.21.15.114:0/4279661958 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98fc110100 0x7f98fc1105b0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.023+0000 7f99008b8640 1 --2- 172.21.15.114:0/4279661958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98fc119400 0x7f98fc10fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.023+0000 7f99008b8640 1 --2- 172.21.15.114:0/4279661958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98fc107920 0x7f98fc116b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.023+0000 7f99008b8640 1 -- 172.21.15.114:0/4279661958 >> 172.21.15.114:0/4279661958 conn(0x7f98fc06afe0 msgr2=0x7f98fc119f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:04.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.026+0000 7f99008b8640 1 -- 172.21.15.114:0/4279661958 shutdown_connections 2024-09-13T18:27:04.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.026+0000 7f99008b8640 1 -- 172.21.15.114:0/4279661958 wait complete. 2024-09-13T18:27:04.061 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-09-13T18:27:04.368 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:04 smithi134 ceph-mon[34283]: pgmap v140: 129 pgs: 129 active+clean; 16 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 41 MiB/s wr, 668 op/s 2024-09-13T18:27:04.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.381+0000 7f87309f8640 1 -- 172.21.15.114:0/2695296316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f872c1074b0 msgr2=0x7f872c10e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:04.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.381+0000 7f87309f8640 1 --2- 172.21.15.114:0/2695296316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f872c1074b0 0x7f872c10e510 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8724009ca0 tx=0x7f872402a8f0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.383+0000 7f87309f8640 1 -- 172.21.15.114:0/2695296316 shutdown_connections 2024-09-13T18:27:04.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.383+0000 7f87309f8640 1 --2- 172.21.15.114:0/2695296316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f872c119400 0x7f872c11b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.383+0000 7f87309f8640 1 --2- 172.21.15.114:0/2695296316 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f872c107890 0x7f872c107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.383+0000 7f87309f8640 1 --2- 172.21.15.114:0/2695296316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f872c1074b0 0x7f872c10e510 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.383+0000 7f87309f8640 1 -- 172.21.15.114:0/2695296316 >> 172.21.15.114:0/2695296316 conn(0x7f872c06af40 msgr2=0x7f872c06b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:04.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.387+0000 7f87309f8640 1 -- 172.21.15.114:0/2695296316 shutdown_connections 2024-09-13T18:27:04.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.387+0000 7f87309f8640 1 -- 172.21.15.114:0/2695296316 wait complete. 2024-09-13T18:27:04.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.388+0000 7f87309f8640 1 Processor -- start 2024-09-13T18:27:04.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.389+0000 7f87309f8640 1 -- start start 2024-09-13T18:27:04.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.389+0000 7f87309f8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f872c1074b0 0x7f872c1168d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.389+0000 7f87309f8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f872c107890 0x7f872c116e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.389+0000 7f87309f8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f872c119400 0x7f872c10fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.390+0000 7f87309f8640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872c110360 con 0x7f872c119400 2024-09-13T18:27:04.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.390+0000 7f87309f8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872c1104d0 con 0x7f872c1074b0 2024-09-13T18:27:04.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.390+0000 7f87309f8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872c110690 con 0x7f872c107890 2024-09-13T18:27:04.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.391+0000 7f872affd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f872c107890 0x7f872c116e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.391+0000 7f872affd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f872c107890 0x7f872c116e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:60044/0 (socket says 172.21.15.114:60044) 2024-09-13T18:27:04.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.391+0000 7f872affd640 1 -- 172.21.15.114:0/4237047174 learned_addr learned my addr 172.21.15.114:0/4237047174 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:04.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.392+0000 7f872affd640 1 -- 172.21.15.114:0/4237047174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f872c1074b0 msgr2=0x7f872c1168d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:27:04.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.392+0000 7f872affd640 1 --2- 172.21.15.114:0/4237047174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f872c1074b0 0x7f872c1168d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.392+0000 7f872affd640 1 -- 172.21.15.114:0/4237047174 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f872c119400 msgr2=0x7f872c10fbd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:27:04.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.392+0000 7f872affd640 1 --2- 172.21.15.114:0/4237047174 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f872c119400 0x7f872c10fbd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.392+0000 7f872affd640 1 -- 172.21.15.114:0/4237047174 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8724009900 con 0x7f872c107890 2024-09-13T18:27:04.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.392+0000 7f872affd640 1 --2- 172.21.15.114:0/4237047174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f872c107890 0x7f872c116e30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f871c00b4a0 tx=0x7f871c00b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:04.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.395+0000 7f8728ff9640 1 -- 172.21.15.114:0/4237047174 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871c010610 con 0x7f872c107890 2024-09-13T18:27:04.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.395+0000 7f8728ff9640 1 -- 172.21.15.114:0/4237047174 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f871c010c00 con 0x7f872c107890 2024-09-13T18:27:04.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.395+0000 7f8728ff9640 1 -- 172.21.15.114:0/4237047174 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871c017c60 con 0x7f872c107890 2024-09-13T18:27:04.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.395+0000 7f87309f8640 1 -- 172.21.15.114:0/4237047174 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f872c114460 con 0x7f872c107890 2024-09-13T18:27:04.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.396+0000 7f87309f8640 1 -- 172.21.15.114:0/4237047174 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f872c114920 con 0x7f872c107890 2024-09-13T18:27:04.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.402+0000 7f8734a55740 1 Processor -- start 2024-09-13T18:27:04.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.403+0000 7f8734a55740 1 -- start start 2024-09-13T18:27:04.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.403+0000 7f8734a55740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8083a7970 0x55d8083a7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.403+0000 7f8734a55740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8083a8290 0x55d8083b25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.403+0000 7f8734a55740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8083b2b30 0x55d8083b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.403+0000 7f8734a55740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d808311cc0 con 0x55d8083b2b30 2024-09-13T18:27:04.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.403+0000 7f8734a55740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d80830f450 con 0x55d8083a8290 2024-09-13T18:27:04.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.403+0000 7f8734a55740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d80830ad40 con 0x55d8083a7970 2024-09-13T18:27:04.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.403+0000 7f872bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8083a7970 0x55d8083a7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.403+0000 7f872bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8083a7970 0x55d8083a7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:60054/0 (socket says 172.21.15.114:60054) 2024-09-13T18:27:04.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.403+0000 7f872bfff640 1 -- 172.21.15.114:0/1375983802 learned_addr learned my addr 172.21.15.114:0/1375983802 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:04.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.405+0000 7f872bfff640 1 -- 172.21.15.114:0/1375983802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8083a8290 msgr2=0x55d8083b25f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:27:04.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.405+0000 7f872bfff640 1 --2- 172.21.15.114:0/1375983802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8083a8290 0x55d8083b25f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.405+0000 7f872bfff640 1 -- 172.21.15.114:0/1375983802 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8083b2b30 msgr2=0x55d8083b4f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:27:04.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.405+0000 7f872bfff640 1 --2- 172.21.15.114:0/1375983802 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8083b2b30 0x55d8083b4f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.405+0000 7f872bfff640 1 -- 172.21.15.114:0/1375983802 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8082d2620 con 0x55d8083a7970 2024-09-13T18:27:04.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.405+0000 7f872bfff640 1 --2- 172.21.15.114:0/1375983802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8083a7970 0x55d8083a7d50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f87200096f0 tx=0x7f8720009bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5ac478d6c6d8311 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:04.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.406+0000 7f8728ff9640 1 -- 172.21.15.114:0/4237047174 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f871c010770 con 0x7f872c107890 2024-09-13T18:27:04.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.407+0000 7f8728ff9640 1 --2- 172.21.15.114:0/4237047174 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f87140a1410 0x7f87140a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.407+0000 7f8728ff9640 1 -- 172.21.15.114:0/4237047174 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f871c0cc4d0 con 0x7f872c107890 2024-09-13T18:27:04.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.407+0000 7f8711ffb640 1 -- 172.21.15.114:0/1375983802 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872000b8d0 con 0x55d8083a7970 2024-09-13T18:27:04.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.407+0000 7f8711ffb640 1 -- 172.21.15.114:0/1375983802 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8720011420 con 0x55d8083a7970 2024-09-13T18:27:04.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.408+0000 7f872b7fe640 1 --2- 172.21.15.114:0/4237047174 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f87140a1410 0x7f87140a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.408+0000 7f8711ffb640 1 -- 172.21.15.114:0/1375983802 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87200105e0 con 0x55d8083a7970 2024-09-13T18:27:04.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.408+0000 7f8734a55740 1 -- 172.21.15.114:0/1375983802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8083a7970 msgr2=0x55d8083a7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:04.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.408+0000 7f8734a55740 1 --2- 172.21.15.114:0/1375983802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8083a7970 0x55d8083a7d50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f87200096f0 tx=0x7f8720009bb0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.409+0000 7f8734a55740 1 -- 172.21.15.114:0/1375983802 shutdown_connections 2024-09-13T18:27:04.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.409+0000 7f8734a55740 1 --2- 172.21.15.114:0/1375983802 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8083b2b30 0x55d8083b4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.409+0000 7f8734a55740 1 --2- 172.21.15.114:0/1375983802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8083a8290 0x55d8083b25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.409+0000 7f8734a55740 1 --2- 172.21.15.114:0/1375983802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8083a7970 0x55d8083a7d50 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.409+0000 7f8734a55740 1 -- 172.21.15.114:0/1375983802 >> 172.21.15.114:0/1375983802 conn(0x55d8083a3ee0 msgr2=0x55d8083a42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:04.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.410+0000 7f872b7fe640 1 --2- 172.21.15.114:0/4237047174 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f87140a1410 0x7f87140a38d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f872402aee0 tx=0x7f8724000f40 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:04.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.413+0000 7f8734a55740 1 -- 172.21.15.114:0/1375983802 shutdown_connections 2024-09-13T18:27:04.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.413+0000 7f8734a55740 1 -- 172.21.15.114:0/1375983802 wait complete. 2024-09-13T18:27:04.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.413+0000 7f8734a55740 1 -- start start 2024-09-13T18:27:04.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.414+0000 7f8734a55740 1 Processor -- start 2024-09-13T18:27:04.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.414+0000 7f8734a55740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8083a7970 0x55d8083b6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.414+0000 7f8734a55740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8083a8290 0x55d8083b71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.414+0000 7f8734a55740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8083b2b30 0x55d8083af3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.414+0000 7f8734a55740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8082c9180 con 0x55d8083b2b30 2024-09-13T18:27:04.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.414+0000 7f8734a55740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d807fe3430 con 0x55d8083a7970 2024-09-13T18:27:04.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.414+0000 7f8734a55740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d80828e5e0 con 0x55d8083a8290 2024-09-13T18:27:04.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.414+0000 7f8734a55740 10 client.0 _ll_register_callbacks cb 0x55d8082faaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:27:04.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.415+0000 7f872b7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8083a7970 0x55d8083b6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.415+0000 7f872b7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8083a7970 0x55d8083b6cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:36060/0 (socket says 172.21.15.114:36060) 2024-09-13T18:27:04.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.415+0000 7f872b7fe640 1 -- 172.21.15.114:0/1050771449 learned_addr learned my addr 172.21.15.114:0/1050771449 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:04.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.415+0000 7f872affd640 1 --2- 172.21.15.114:0/1050771449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8083a8290 0x55d8083b71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.415+0000 7f872affd640 1 -- 172.21.15.114:0/1050771449 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8083a7970 msgr2=0x55d8083b6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:04.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.415+0000 7f872affd640 1 --2- 172.21.15.114:0/1050771449 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8083a7970 0x55d8083b6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.415+0000 7f872affd640 1 -- 172.21.15.114:0/1050771449 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8083b2b30 msgr2=0x55d8083af3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:04.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.415+0000 7f872affd640 1 --2- 172.21.15.114:0/1050771449 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8083b2b30 0x55d8083af3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.415+0000 7f872affd640 1 -- 172.21.15.114:0/1050771449 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8720009320 con 0x55d8083a8290 2024-09-13T18:27:04.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.416+0000 7f872affd640 1 --2- 172.21.15.114:0/1050771449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8083a8290 0x55d8083b71f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f871c0c9980 tx=0x7f871c010b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fd9e73fb654016b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:04.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.419+0000 7f870affd640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:27:04.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.419+0000 7f870affd640 1 -- 172.21.15.114:0/1050771449 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871c0a17c0 con 0x55d8083a8290 2024-09-13T18:27:04.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.419+0000 7f8734a55740 1 -- 172.21.15.114:0/1050771449 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d808334b80 con 0x55d8083a8290 2024-09-13T18:27:04.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.419+0000 7f870affd640 1 -- 172.21.15.114:0/1050771449 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f871c0cc170 con 0x55d8083a8290 2024-09-13T18:27:04.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.421+0000 7f870affd640 1 -- 172.21.15.114:0/1050771449 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f871c095a50 con 0x55d8083a8290 2024-09-13T18:27:04.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.421+0000 7f8734a55740 10 client.26512 fetch_fsmap learned FSMap version 32 2024-09-13T18:27:04.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.421+0000 7f8734a55740 1 -- 172.21.15.114:0/1050771449 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55d8083093c0 con 0x55d8083a8290 2024-09-13T18:27:04.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.422+0000 7f870affd640 1 -- 172.21.15.114:0/1050771449 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d808334b80 con 0x55d8083a8290 2024-09-13T18:27:04.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.422+0000 7f870affd640 1 -- 172.21.15.114:0/1050771449 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f871c0c9af0 con 0x55d8083a8290 2024-09-13T18:27:04.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.422+0000 7f8734a55740 10 client.26512 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:27:04.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.422+0000 7f8734a55740 10 client.26512 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-13T18:27:04.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.422+0000 7f8734a55740 20 client.26512 populate_metadata read hostname 'smithi114' 2024-09-13T18:27:04.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.422+0000 7f8734a55740 1 --2- 172.21.15.114:0/1050771449 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55d8083e46c0 0x55d8083e6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.422+0000 7f8734a55740 4 client.26512 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:27:04.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.422+0000 7f8734a55740 1 -- 172.21.15.114:0/1050771449 --> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d8082fd1f0 con 0x55d8083e46c0 2024-09-13T18:27:04.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.423+0000 7f872b7fe640 1 --2- 172.21.15.114:0/1050771449 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55d8083e46c0 0x55d8083e6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.428+0000 7f872b7fe640 1 --2- 172.21.15.114:0/1050771449 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55d8083e46c0 0x55d8083e6ae0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fd8d49449759532a server_cookie=27f3d57552b73f19 in_seq=0 out_seq=0 2024-09-13T18:27:04.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.428+0000 7f870affd640 10 client.26512 ms_handle_connect on v2:172.21.15.204:6838/344883912 2024-09-13T18:27:04.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.432+0000 7f870affd640 1 -- 172.21.15.114:0/1050771449 <== mds.2 v2:172.21.15.204:6838/344883912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55d8082fd1f0 con 0x55d8083e46c0 2024-09-13T18:27:04.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.432+0000 7f870affd640 10 client.26512 handle_command_reply: tid=0 2024-09-13T18:27:04.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.432+0000 7f8734a55740 1 client.26512 shutdown 2024-09-13T18:27:04.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.432+0000 7f8734a55740 1 -- 172.21.15.114:0/1050771449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8083a8290 msgr2=0x55d8083b71f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:04.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.432+0000 7f8734a55740 1 --2- 172.21.15.114:0/1050771449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8083a8290 0x55d8083b71f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f871c0c9980 tx=0x7f871c010b60 comp rx=0 tx=0).stop 2024-09-13T18:27:04.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.432+0000 7f8734a55740 1 -- 172.21.15.114:0/1050771449 shutdown_connections 2024-09-13T18:27:04.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.432+0000 7f8734a55740 1 --2- 172.21.15.114:0/1050771449 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55d8083e46c0 0x55d8083e6ae0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.432+0000 7f8734a55740 1 --2- 172.21.15.114:0/1050771449 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8083b2b30 0x55d8083af3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.432+0000 7f8734a55740 1 --2- 172.21.15.114:0/1050771449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8083a8290 0x55d8083b71f0 secure :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f871c0c9980 tx=0x7f871c010b60 comp rx=0 tx=0).stop 2024-09-13T18:27:04.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.432+0000 7f8734a55740 1 --2- 172.21.15.114:0/1050771449 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8083a7970 0x55d8083b6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.432+0000 7f8734a55740 1 -- 172.21.15.114:0/1050771449 >> 172.21.15.114:0/1050771449 conn(0x55d8083a3ee0 msgr2=0x55d8083a3230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:04.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.432+0000 7f870affd640 0 client.26512 ms_handle_reset on v2:172.21.15.204:6838/344883912 2024-09-13T18:27:04.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.432+0000 7f8734a55740 1 -- 172.21.15.114:0/1050771449 shutdown_connections 2024-09-13T18:27:04.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.432+0000 7f8734a55740 1 -- 172.21.15.114:0/1050771449 wait complete. 2024-09-13T18:27:04.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.433+0000 7f8734a55740 20 client.26512 trim_cache size 0 max 16384 2024-09-13T18:27:04.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:04 smithi204 ceph-mon[34777]: pgmap v140: 129 pgs: 129 active+clean; 16 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 41 MiB/s wr, 668 op/s 2024-09-13T18:27:04.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:04 smithi114 ceph-mon[30897]: pgmap v140: 129 pgs: 129 active+clean; 16 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 41 MiB/s wr, 668 op/s 2024-09-13T18:27:04.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.468+0000 7f8734a55740 1 Processor -- start 2024-09-13T18:27:04.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.468+0000 7f8734a55740 1 -- start start 2024-09-13T18:27:04.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.468+0000 7f8734a55740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8083a7970 0x55d8083bb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.469+0000 7f8734a55740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8083b2b30 0x55d8083bbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.469+0000 7f872b7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8083a7970 0x55d8083bb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.469+0000 7f8734a55740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8083adc60 0x55d8083ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.469+0000 7f872b7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8083a7970 0x55d8083bb9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59918/0 (socket says 172.21.15.114:59918) 2024-09-13T18:27:04.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.469+0000 7f872affd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8083b2b30 0x55d8083bbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.469+0000 7f872affd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8083b2b30 0x55d8083bbef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:60080/0 (socket says 172.21.15.114:60080) 2024-09-13T18:27:04.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.469+0000 7f872b7fe640 1 -- 172.21.15.114:0/797738377 learned_addr learned my addr 172.21.15.114:0/797738377 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:04.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.469+0000 7f8734a55740 1 -- 172.21.15.114:0/797738377 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8083b68e0 con 0x55d8083a7970 2024-09-13T18:27:04.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.469+0000 7f8734a55740 1 -- 172.21.15.114:0/797738377 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d80839e6f0 con 0x55d8083adc60 2024-09-13T18:27:04.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.469+0000 7f8734a55740 1 -- 172.21.15.114:0/797738377 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8083a5050 con 0x55d8083b2b30 2024-09-13T18:27:04.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.469+0000 7f872bfff640 1 --2- 172.21.15.114:0/797738377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8083adc60 0x55d8083ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.470+0000 7f872affd640 1 -- 172.21.15.114:0/797738377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8083adc60 msgr2=0x55d8083ae0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:04.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.470+0000 7f872affd640 1 --2- 172.21.15.114:0/797738377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8083adc60 0x55d8083ae0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.470+0000 7f872affd640 1 -- 172.21.15.114:0/797738377 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8083a7970 msgr2=0x55d8083bb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:04.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.470+0000 7f872affd640 1 --2- 172.21.15.114:0/797738377 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8083a7970 0x55d8083bb9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.470+0000 7f872affd640 1 -- 172.21.15.114:0/797738377 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8082ab770 con 0x55d8083b2b30 2024-09-13T18:27:04.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.470+0000 7f872affd640 1 --2- 172.21.15.114:0/797738377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8083b2b30 0x55d8083bbef0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x55d8083b04a0 tx=0x7f871c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9b3ac2aff0d8749 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:04.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.472+0000 7f87127fc640 1 -- 172.21.15.114:0/797738377 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871c0a13d0 con 0x55d8083b2b30 2024-09-13T18:27:04.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.472+0000 7f87127fc640 1 -- 172.21.15.114:0/797738377 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f871c0a15c0 con 0x55d8083b2b30 2024-09-13T18:27:04.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.472+0000 7f87127fc640 1 -- 172.21.15.114:0/797738377 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871c017360 con 0x55d8083b2b30 2024-09-13T18:27:04.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.475+0000 7f8734a55740 1 -- 172.21.15.114:0/797738377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8083b2b30 msgr2=0x55d8083bbef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:04.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.475+0000 7f8734a55740 1 --2- 172.21.15.114:0/797738377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8083b2b30 0x55d8083bbef0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x55d8083b04a0 tx=0x7f871c0046c0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.478+0000 7f8734a55740 1 -- 172.21.15.114:0/797738377 shutdown_connections 2024-09-13T18:27:04.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.478+0000 7f8734a55740 1 --2- 172.21.15.114:0/797738377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8083adc60 0x55d8083ae0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.478+0000 7f8734a55740 1 --2- 172.21.15.114:0/797738377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8083b2b30 0x55d8083bbef0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.478+0000 7f8734a55740 1 --2- 172.21.15.114:0/797738377 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8083a7970 0x55d8083bb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.478+0000 7f8734a55740 1 -- 172.21.15.114:0/797738377 >> 172.21.15.114:0/797738377 conn(0x55d8083a3ee0 msgr2=0x55d8083a42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:04.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.481+0000 7f8734a55740 1 -- 172.21.15.114:0/797738377 shutdown_connections 2024-09-13T18:27:04.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.483+0000 7f8734a55740 1 -- 172.21.15.114:0/797738377 wait complete. 2024-09-13T18:27:04.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.484+0000 7f8734a55740 1 -- start start 2024-09-13T18:27:04.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.484+0000 7f8734a55740 1 Processor -- start 2024-09-13T18:27:04.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.485+0000 7f8734a55740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8083a7970 0x55d8083ba500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.485+0000 7f8734a55740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8083adc60 0x55d8083baa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.485+0000 7f8734a55740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8083b2b30 0x55d8083baf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.485+0000 7f8734a55740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d80843abb0 con 0x55d8083adc60 2024-09-13T18:27:04.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.485+0000 7f872b7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8083a7970 0x55d8083ba500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.485+0000 7f8734a55740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d80843acf0 con 0x55d8083a7970 2024-09-13T18:27:04.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.485+0000 7f8734a55740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d80843ae30 con 0x55d8083b2b30 2024-09-13T18:27:04.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.485+0000 7f8734a55740 10 client.0 _ll_register_callbacks cb 0x7f8720006eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:27:04.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.485+0000 7f872bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8083b2b30 0x55d8083baf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.485+0000 7f872bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8083b2b30 0x55d8083baf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:60088/0 (socket says 172.21.15.114:60088) 2024-09-13T18:27:04.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.485+0000 7f872bfff640 1 -- 172.21.15.114:0/1770331116 learned_addr learned my addr 172.21.15.114:0/1770331116 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:04.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.486+0000 7f872affd640 1 --2- 172.21.15.114:0/1770331116 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8083adc60 0x55d8083baa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.486+0000 7f872bfff640 1 -- 172.21.15.114:0/1770331116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8083a7970 msgr2=0x55d8083ba500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:04.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.486+0000 7f872bfff640 1 --2- 172.21.15.114:0/1770331116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8083a7970 0x55d8083ba500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.486+0000 7f872bfff640 1 -- 172.21.15.114:0/1770331116 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8083adc60 msgr2=0x55d8083baa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:04.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.486+0000 7f872bfff640 1 --2- 172.21.15.114:0/1770331116 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8083adc60 0x55d8083baa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.486+0000 7f872bfff640 1 -- 172.21.15.114:0/1770331116 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d8083093c0 con 0x55d8083b2b30 2024-09-13T18:27:04.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.486+0000 7f872bfff640 1 --2- 172.21.15.114:0/1770331116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8083b2b30 0x55d8083baf80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f872000bc70 tx=0x7f872000bca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3101c784939d2f5e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:04.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.487+0000 7f870affd640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:27:04.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.487+0000 7f870affd640 1 -- 172.21.15.114:0/1770331116 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8720004450 con 0x55d8083b2b30 2024-09-13T18:27:04.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.487+0000 7f8734a55740 1 -- 172.21.15.114:0/1770331116 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8720011420 con 0x55d8083b2b30 2024-09-13T18:27:04.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.487+0000 7f870affd640 1 -- 172.21.15.114:0/1770331116 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f87200045b0 con 0x55d8083b2b30 2024-09-13T18:27:04.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.490+0000 7f870affd640 1 -- 172.21.15.114:0/1770331116 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8720003690 con 0x55d8083b2b30 2024-09-13T18:27:04.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.491+0000 7f8734a55740 10 client.26518 fetch_fsmap learned FSMap version 32 2024-09-13T18:27:04.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.491+0000 7f8734a55740 1 -- 172.21.15.114:0/1770331116 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55d8082d0e80 con 0x55d8083b2b30 2024-09-13T18:27:04.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.491+0000 7f870affd640 1 -- 172.21.15.114:0/1770331116 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8720011420 con 0x55d8083b2b30 2024-09-13T18:27:04.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.492+0000 7f870affd640 1 -- 172.21.15.114:0/1770331116 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f8720011a60 con 0x55d8083b2b30 2024-09-13T18:27:04.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.495+0000 7f8734a55740 10 client.26518 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:27:04.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.495+0000 7f8734a55740 10 client.26518 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-13T18:27:04.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.495+0000 7f8734a55740 20 client.26518 populate_metadata read hostname 'smithi114' 2024-09-13T18:27:04.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.495+0000 7f8734a55740 1 --2- 172.21.15.114:0/1770331116 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55d80843c4b0 0x55d80843e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.495+0000 7f8734a55740 4 client.26518 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:27:04.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.495+0000 7f8734a55740 1 -- 172.21.15.114:0/1770331116 --> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d807ffe4e0 con 0x55d80843c4b0 2024-09-13T18:27:04.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.498+0000 7f872b7fe640 1 --2- 172.21.15.114:0/1770331116 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55d80843c4b0 0x55d80843e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.507+0000 7f872b7fe640 1 --2- 172.21.15.114:0/1770331116 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55d80843c4b0 0x55d80843e8d0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2dd87b8107c7689d server_cookie=e6a56da652f162b4 in_seq=0 out_seq=0 2024-09-13T18:27:04.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.507+0000 7f870affd640 10 client.26518 ms_handle_connect on v2:172.21.15.204:6838/344883912 2024-09-13T18:27:04.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.512+0000 7f870affd640 1 -- 172.21.15.114:0/1770331116 <== mds.2 v2:172.21.15.204:6838/344883912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d807ffe4e0 con 0x55d80843c4b0 2024-09-13T18:27:04.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.512+0000 7f870affd640 10 client.26518 handle_command_reply: tid=0 2024-09-13T18:27:04.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.512+0000 7f8734a55740 1 client.26518 shutdown 2024-09-13T18:27:04.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.516+0000 7f8734a55740 1 -- 172.21.15.114:0/1770331116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8083b2b30 msgr2=0x55d8083baf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:04.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.516+0000 7f8734a55740 1 --2- 172.21.15.114:0/1770331116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8083b2b30 0x55d8083baf80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f872000bc70 tx=0x7f872000bca0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.518+0000 7f8734a55740 1 -- 172.21.15.114:0/1770331116 shutdown_connections 2024-09-13T18:27:04.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.518+0000 7f8734a55740 1 --2- 172.21.15.114:0/1770331116 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55d80843c4b0 0x55d80843e8d0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.519+0000 7f870affd640 0 client.26518 ms_handle_reset on v2:172.21.15.204:6838/344883912 2024-09-13T18:27:04.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.519+0000 7f8734a55740 1 --2- 172.21.15.114:0/1770331116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8083b2b30 0x55d8083baf80 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.519+0000 7f8734a55740 1 --2- 172.21.15.114:0/1770331116 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8083adc60 0x55d8083baa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.519+0000 7f8734a55740 1 --2- 172.21.15.114:0/1770331116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8083a7970 0x55d8083ba500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.519+0000 7f8734a55740 1 -- 172.21.15.114:0/1770331116 >> 172.21.15.114:0/1770331116 conn(0x55d8083a3ee0 msgr2=0x55d8083a42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:04.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.519+0000 7f8734a55740 1 -- 172.21.15.114:0/1770331116 shutdown_connections 2024-09-13T18:27:04.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.519+0000 7f8734a55740 1 -- 172.21.15.114:0/1770331116 wait complete. 2024-09-13T18:27:04.535 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:27:04.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.529+0000 7f8734a55740 20 client.26518 trim_cache size 0 max 16384 2024-09-13T18:27:04.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.539+0000 7f87309f8640 1 -- 172.21.15.114:0/4237047174 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f87140a1410 msgr2=0x7f87140a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:04.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.539+0000 7f87309f8640 1 --2- 172.21.15.114:0/4237047174 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f87140a1410 0x7f87140a38d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f872402aee0 tx=0x7f8724000f40 comp rx=0 tx=0).stop 2024-09-13T18:27:04.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.539+0000 7f87309f8640 1 -- 172.21.15.114:0/4237047174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f872c107890 msgr2=0x7f872c116e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:04.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.539+0000 7f87309f8640 1 --2- 172.21.15.114:0/4237047174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f872c107890 0x7f872c116e30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f871c00b4a0 tx=0x7f871c00b960 comp rx=0 tx=0).stop 2024-09-13T18:27:04.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.544+0000 7f87309f8640 1 -- 172.21.15.114:0/4237047174 shutdown_connections 2024-09-13T18:27:04.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.544+0000 7f87309f8640 1 --2- 172.21.15.114:0/4237047174 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f87140a1410 0x7f87140a38d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.544+0000 7f87309f8640 1 --2- 172.21.15.114:0/4237047174 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f872c119400 0x7f872c10fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.544+0000 7f87309f8640 1 --2- 172.21.15.114:0/4237047174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f872c107890 0x7f872c116e30 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.544+0000 7f87309f8640 1 --2- 172.21.15.114:0/4237047174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f872c1074b0 0x7f872c1168d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.544+0000 7f87309f8640 1 -- 172.21.15.114:0/4237047174 >> 172.21.15.114:0/4237047174 conn(0x7f872c06af40 msgr2=0x7f872c119d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:04.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.547+0000 7f87309f8640 1 -- 172.21.15.114:0/4237047174 shutdown_connections 2024-09-13T18:27:04.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.548+0000 7f87309f8640 1 -- 172.21.15.114:0/4237047174 wait complete. 2024-09-13T18:27:04.580 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-09-13T18:27:04.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.877+0000 7fbea5caf640 1 -- 172.21.15.114:0/2183587007 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbea0103f00 msgr2=0x7fbea010ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:04.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.877+0000 7fbea5caf640 1 --2- 172.21.15.114:0/2183587007 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbea0103f00 0x7fbea010ba50 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fbe90009cd0 tx=0x7fbe9002a930 comp rx=0 tx=0).stop 2024-09-13T18:27:04.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.880+0000 7fbea5caf640 1 -- 172.21.15.114:0/2183587007 shutdown_connections 2024-09-13T18:27:04.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.880+0000 7fbea5caf640 1 --2- 172.21.15.114:0/2183587007 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbea0103f00 0x7fbea010ba50 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.880+0000 7fbea5caf640 1 --2- 172.21.15.114:0/2183587007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbea00ff760 0x7fbea01037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.880+0000 7fbea5caf640 1 --2- 172.21.15.114:0/2183587007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbea00fedb0 0x7fbea00ff190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.880+0000 7fbea5caf640 1 -- 172.21.15.114:0/2183587007 >> 172.21.15.114:0/2183587007 conn(0x7fbea00fa840 msgr2=0x7fbea00fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:04.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.884+0000 7fbea5caf640 1 -- 172.21.15.114:0/2183587007 shutdown_connections 2024-09-13T18:27:04.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.894+0000 7fbea5caf640 1 -- 172.21.15.114:0/2183587007 wait complete. 2024-09-13T18:27:04.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.898+0000 7fbea5caf640 1 Processor -- start 2024-09-13T18:27:04.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.898+0000 7fbea5caf640 1 -- start start 2024-09-13T18:27:04.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.898+0000 7fbea5caf640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbea00fedb0 0x7fbea0071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.898+0000 7fbea5caf640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbea0103f00 0x7fbea0072160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.898+0000 7fbea5caf640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbea006ade0 0x7fbea006b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.898+0000 7fbea5caf640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea006ba20 con 0x7fbea006ade0 2024-09-13T18:27:04.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.898+0000 7fbea5caf640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea006f790 con 0x7fbea00fedb0 2024-09-13T18:27:04.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.898+0000 7fbea5caf640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea006f950 con 0x7fbea0103f00 2024-09-13T18:27:04.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.901+0000 7fbea4cad640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbea00fedb0 0x7fbea0071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.901+0000 7fbea4cad640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbea00fedb0 0x7fbea0071c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:36080/0 (socket says 172.21.15.114:36080) 2024-09-13T18:27:04.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.901+0000 7fbea4cad640 1 -- 172.21.15.114:0/650607111 learned_addr learned my addr 172.21.15.114:0/650607111 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:04.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.902+0000 7fbe9ffff640 1 --2- 172.21.15.114:0/650607111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbea0103f00 0x7fbea0072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.903+0000 7fbea54ae640 1 --2- 172.21.15.114:0/650607111 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbea006ade0 0x7fbea006b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.903+0000 7fbe9ffff640 1 -- 172.21.15.114:0/650607111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbea00fedb0 msgr2=0x7fbea0071c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:04.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.903+0000 7fbe9ffff640 1 --2- 172.21.15.114:0/650607111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbea00fedb0 0x7fbea0071c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.903+0000 7fbe9ffff640 1 -- 172.21.15.114:0/650607111 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbea006ade0 msgr2=0x7fbea006b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:04.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.903+0000 7fbe9ffff640 1 --2- 172.21.15.114:0/650607111 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbea006ade0 0x7fbea006b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.903+0000 7fbe9ffff640 1 -- 172.21.15.114:0/650607111 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe90009900 con 0x7fbea0103f00 2024-09-13T18:27:04.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.903+0000 7fbe9ffff640 1 --2- 172.21.15.114:0/650607111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbea0103f00 0x7fbea0072160 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbe98009fd0 tx=0x7fbe9800b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:04.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.904+0000 7fbe9dffb640 1 -- 172.21.15.114:0/650607111 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe98011830 con 0x7fbea0103f00 2024-09-13T18:27:04.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.904+0000 7fbe9dffb640 1 -- 172.21.15.114:0/650607111 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbe98011e20 con 0x7fbea0103f00 2024-09-13T18:27:04.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.904+0000 7fbea54ae640 1 --2- 172.21.15.114:0/650607111 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbea006ade0 0x7fbea006b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:27:04.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.904+0000 7fbea4cad640 1 --2- 172.21.15.114:0/650607111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbea00fedb0 0x7fbea0071c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:27:04.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.904+0000 7fbea5caf640 1 -- 172.21.15.114:0/650607111 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbea006fc30 con 0x7fbea0103f00 2024-09-13T18:27:04.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.904+0000 7fbe9dffb640 1 -- 172.21.15.114:0/650607111 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9800f2e0 con 0x7fbea0103f00 2024-09-13T18:27:04.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.904+0000 7fbea5caf640 1 -- 172.21.15.114:0/650607111 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbea0070140 con 0x7fbea0103f00 2024-09-13T18:27:04.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.912+0000 7fbe9dffb640 1 -- 172.21.15.114:0/650607111 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fbe98011990 con 0x7fbea0103f00 2024-09-13T18:27:04.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.912+0000 7fbea9c33740 1 Processor -- start 2024-09-13T18:27:04.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.912+0000 7fbea9c33740 1 -- start start 2024-09-13T18:27:04.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.913+0000 7fbea9c33740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3f8a10890 0x55e3f8a10c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.913+0000 7fbea9c33740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e3f8a111b0 0x55e3f8a23e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.913+0000 7fbea9c33740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e3f8a24340 0x55e3f8a26730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.913+0000 7fbea9c33740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3f897acc0 con 0x55e3f8a111b0 2024-09-13T18:27:04.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.913+0000 7fbea9c33740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3f8978450 con 0x55e3f8a10890 2024-09-13T18:27:04.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.913+0000 7fbea9c33740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3f8973d40 con 0x55e3f8a24340 2024-09-13T18:27:04.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.913+0000 7fbea54ae640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3f8a10890 0x55e3f8a10c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.913+0000 7fbea54ae640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3f8a10890 0x55e3f8a10c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:36084/0 (socket says 172.21.15.114:36084) 2024-09-13T18:27:04.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.913+0000 7fbea54ae640 1 -- 172.21.15.114:0/1832736671 learned_addr learned my addr 172.21.15.114:0/1832736671 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:04.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.913+0000 7fbe9ffff640 1 --2- 172.21.15.114:0/1832736671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e3f8a24340 0x55e3f8a26730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.914+0000 7fbea54ae640 1 -- 172.21.15.114:0/1832736671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e3f8a24340 msgr2=0x55e3f8a26730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:04.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.914+0000 7fbea54ae640 1 --2- 172.21.15.114:0/1832736671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e3f8a24340 0x55e3f8a26730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.914+0000 7fbea54ae640 1 -- 172.21.15.114:0/1832736671 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e3f8a111b0 msgr2=0x55e3f8a23e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:04.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.914+0000 7fbea54ae640 1 --2- 172.21.15.114:0/1832736671 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e3f8a111b0 0x55e3f8a23e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.914+0000 7fbea54ae640 1 -- 172.21.15.114:0/1832736671 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3f893b620 con 0x55e3f8a10890 2024-09-13T18:27:04.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.914+0000 7fbea54ae640 1 --2- 172.21.15.114:0/1832736671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3f8a10890 0x55e3f8a10c70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbe90009bf0 tx=0x7fbe9002f3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c64bb5300f9d4bc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:04.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.914+0000 7fbe9ffff640 1 --2- 172.21.15.114:0/1832736671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e3f8a24340 0x55e3f8a26730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:27:04.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.916+0000 7fbe9dffb640 1 --2- 172.21.15.114:0/650607111 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbe880a1420 0x7fbe880a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.916+0000 7fbe9dffb640 1 -- 172.21.15.114:0/650607111 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fbe980cca30 con 0x7fbea0103f00 2024-09-13T18:27:04.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.916+0000 7fbe86ffd640 1 -- 172.21.15.114:0/1832736671 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe90003970 con 0x55e3f8a10890 2024-09-13T18:27:04.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.916+0000 7fbe86ffd640 1 -- 172.21.15.114:0/1832736671 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbe90040070 con 0x55e3f8a10890 2024-09-13T18:27:04.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.916+0000 7fbe86ffd640 1 -- 172.21.15.114:0/1832736671 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe90030850 con 0x55e3f8a10890 2024-09-13T18:27:04.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.917+0000 7fbea9c33740 1 -- 172.21.15.114:0/1832736671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3f8a10890 msgr2=0x55e3f8a10c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:04.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.917+0000 7fbea9c33740 1 --2- 172.21.15.114:0/1832736671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3f8a10890 0x55e3f8a10c70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbe90009bf0 tx=0x7fbe9002f3f0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.917+0000 7fbea4cad640 1 --2- 172.21.15.114:0/650607111 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbe880a1420 0x7fbe880a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.918+0000 7fbea9c33740 1 -- 172.21.15.114:0/1832736671 shutdown_connections 2024-09-13T18:27:04.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.918+0000 7fbea9c33740 1 --2- 172.21.15.114:0/1832736671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e3f8a24340 0x55e3f8a26730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.918+0000 7fbea9c33740 1 --2- 172.21.15.114:0/1832736671 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e3f8a111b0 0x55e3f8a23e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.918+0000 7fbea9c33740 1 --2- 172.21.15.114:0/1832736671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3f8a10890 0x55e3f8a10c70 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.918+0000 7fbea9c33740 1 -- 172.21.15.114:0/1832736671 >> 172.21.15.114:0/1832736671 conn(0x55e3f8a0cee0 msgr2=0x55e3f8a0d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:04.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.919+0000 7fbea4cad640 1 --2- 172.21.15.114:0/650607111 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbe880a1420 0x7fbe880a38e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbe940093b0 tx=0x7fbe94009340 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:04.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.921+0000 7fbea9c33740 1 -- 172.21.15.114:0/1832736671 shutdown_connections 2024-09-13T18:27:04.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.922+0000 7fbea9c33740 1 -- 172.21.15.114:0/1832736671 wait complete. 2024-09-13T18:27:04.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.922+0000 7fbea9c33740 1 -- start start 2024-09-13T18:27:04.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.922+0000 7fbea9c33740 1 Processor -- start 2024-09-13T18:27:04.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.923+0000 7fbea9c33740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3f8a10890 0x55e3f8a1d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.923+0000 7fbea9c33740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e3f8a111b0 0x55e3f8a1daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.923+0000 7fbea9c33740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e3f8a24340 0x55e3f8a18930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.923+0000 7fbea9c33740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3f8932180 con 0x55e3f8a111b0 2024-09-13T18:27:04.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.923+0000 7fbea9c33740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3f864c430 con 0x55e3f8a10890 2024-09-13T18:27:04.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.923+0000 7fbea9c33740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3f88f75e0 con 0x55e3f8a24340 2024-09-13T18:27:04.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.923+0000 7fbea9c33740 10 client.0 _ll_register_callbacks cb 0x55e3f8963aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:27:04.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.923+0000 7fbea4cad640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3f8a10890 0x55e3f8a1d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.923+0000 7fbea4cad640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3f8a10890 0x55e3f8a1d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:36092/0 (socket says 172.21.15.114:36092) 2024-09-13T18:27:04.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.923+0000 7fbea4cad640 1 -- 172.21.15.114:0/3949144615 learned_addr learned my addr 172.21.15.114:0/3949144615 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:04.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.924+0000 7fbea54ae640 1 --2- 172.21.15.114:0/3949144615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e3f8a24340 0x55e3f8a18930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.924+0000 7fbe9ffff640 1 --2- 172.21.15.114:0/3949144615 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e3f8a111b0 0x55e3f8a1daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.924+0000 7fbea4cad640 1 -- 172.21.15.114:0/3949144615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e3f8a24340 msgr2=0x55e3f8a18930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:04.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.924+0000 7fbea4cad640 1 --2- 172.21.15.114:0/3949144615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e3f8a24340 0x55e3f8a18930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.924+0000 7fbea4cad640 1 -- 172.21.15.114:0/3949144615 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e3f8a111b0 msgr2=0x55e3f8a1daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:04.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.924+0000 7fbea4cad640 1 --2- 172.21.15.114:0/3949144615 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e3f8a111b0 0x55e3f8a1daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.924+0000 7fbea4cad640 1 -- 172.21.15.114:0/3949144615 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbea010c290 con 0x55e3f8a10890 2024-09-13T18:27:04.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.924+0000 7fbe9ffff640 1 --2- 172.21.15.114:0/3949144615 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e3f8a111b0 0x55e3f8a1daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:27:04.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.924+0000 7fbea4cad640 1 --2- 172.21.15.114:0/3949144615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3f8a10890 0x55e3f8a1d5b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbe94015ac0 tx=0x7fbe94004700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=beca16790fed7c84 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:04.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.924+0000 7fbe7ffff640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:27:04.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.924+0000 7fbe7ffff640 1 -- 172.21.15.114:0/3949144615 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe94013ad0 con 0x55e3f8a10890 2024-09-13T18:27:04.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.924+0000 7fbe7ffff640 1 -- 172.21.15.114:0/3949144615 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fbe94013c30 con 0x55e3f8a10890 2024-09-13T18:27:04.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.925+0000 7fbea9c33740 1 -- 172.21.15.114:0/3949144615 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e3f899db80 con 0x55e3f8a10890 2024-09-13T18:27:04.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.925+0000 7fbe7ffff640 1 -- 172.21.15.114:0/3949144615 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe94013df0 con 0x55e3f8a10890 2024-09-13T18:27:04.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.927+0000 7fbea9c33740 10 client.26486 fetch_fsmap learned FSMap version 32 2024-09-13T18:27:04.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.927+0000 7fbea9c33740 1 -- 172.21.15.114:0/3949144615 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55e3f89723c0 con 0x55e3f8a10890 2024-09-13T18:27:04.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.927+0000 7fbe7ffff640 1 -- 172.21.15.114:0/3949144615 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e3f899db80 con 0x55e3f8a10890 2024-09-13T18:27:04.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.928+0000 7fbe7ffff640 1 -- 172.21.15.114:0/3949144615 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fbe94011070 con 0x55e3f8a10890 2024-09-13T18:27:04.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.928+0000 7fbea9c33740 10 client.26486 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:27:04.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.928+0000 7fbea9c33740 10 client.26486 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-13T18:27:04.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.928+0000 7fbea9c33740 20 client.26486 populate_metadata read hostname 'smithi114' 2024-09-13T18:27:04.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.928+0000 7fbea9c33740 1 --2- 172.21.15.114:0/3949144615 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55e3f8a3c600 0x55e3f8a3e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.928+0000 7fbea9c33740 4 client.26486 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:27:04.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.928+0000 7fbea9c33740 1 -- 172.21.15.114:0/3949144615 --> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e3f89661f0 con 0x55e3f8a3c600 2024-09-13T18:27:04.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.929+0000 7fbe9ffff640 1 --2- 172.21.15.114:0/3949144615 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55e3f8a3c600 0x55e3f8a3e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.930+0000 7fbe9ffff640 1 --2- 172.21.15.114:0/3949144615 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55e3f8a3c600 0x55e3f8a3e9f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d6f9002f0f158795 server_cookie=e89d2126a566095e in_seq=0 out_seq=0 2024-09-13T18:27:04.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.930+0000 7fbe7ffff640 10 client.26486 ms_handle_connect on v2:172.21.15.134:6836/891692210 2024-09-13T18:27:04.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.931+0000 7fbe7ffff640 1 -- 172.21.15.114:0/3949144615 <== mds.1 v2:172.21.15.134:6836/891692210 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55e3f89661f0 con 0x55e3f8a3c600 2024-09-13T18:27:04.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.931+0000 7fbe7ffff640 10 client.26486 handle_command_reply: tid=0 2024-09-13T18:27:04.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.935+0000 7fbea9c33740 1 client.26486 shutdown 2024-09-13T18:27:04.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.937+0000 7fbea9c33740 1 -- 172.21.15.114:0/3949144615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3f8a10890 msgr2=0x55e3f8a1d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:04.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.937+0000 7fbea9c33740 1 --2- 172.21.15.114:0/3949144615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3f8a10890 0x55e3f8a1d5b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbe94015ac0 tx=0x7fbe94004700 comp rx=0 tx=0).stop 2024-09-13T18:27:04.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.937+0000 7fbea9c33740 1 -- 172.21.15.114:0/3949144615 shutdown_connections 2024-09-13T18:27:04.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.937+0000 7fbea9c33740 1 --2- 172.21.15.114:0/3949144615 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55e3f8a3c600 0x55e3f8a3e9f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.937+0000 7fbea9c33740 1 --2- 172.21.15.114:0/3949144615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e3f8a24340 0x55e3f8a18930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.937+0000 7fbea9c33740 1 --2- 172.21.15.114:0/3949144615 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e3f8a111b0 0x55e3f8a1daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.937+0000 7fbea9c33740 1 --2- 172.21.15.114:0/3949144615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3f8a10890 0x55e3f8a1d5b0 secure :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbe94015ac0 tx=0x7fbe94004700 comp rx=0 tx=0).stop 2024-09-13T18:27:04.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.937+0000 7fbea9c33740 1 -- 172.21.15.114:0/3949144615 >> 172.21.15.114:0/3949144615 conn(0x55e3f8a0cee0 msgr2=0x55e3f8a0c230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:04.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.937+0000 7fbe7ffff640 0 client.26486 ms_handle_reset on v2:172.21.15.134:6836/891692210 2024-09-13T18:27:04.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.943+0000 7fbea9c33740 1 -- 172.21.15.114:0/3949144615 shutdown_connections 2024-09-13T18:27:04.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.944+0000 7fbea9c33740 1 -- 172.21.15.114:0/3949144615 wait complete. 2024-09-13T18:27:04.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.949+0000 7fbea9c33740 20 client.26486 trim_cache size 0 max 16384 2024-09-13T18:27:04.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.986+0000 7fbea9c33740 1 Processor -- start 2024-09-13T18:27:04.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.987+0000 7fbea9c33740 1 -- start start 2024-09-13T18:27:04.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.987+0000 7fbea9c33740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3f8a111b0 0x55e3f8a14150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.987+0000 7fbea9c33740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e3f8a24340 0x55e3f8a0d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.987+0000 7fbea9c33740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e3f8a0ca20 0x55e3f8a12a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:04.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.987+0000 7fbea9c33740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3f8a08810 con 0x55e3f8a0ca20 2024-09-13T18:27:04.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.987+0000 7fbea9c33740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3f8a0d830 con 0x55e3f8a111b0 2024-09-13T18:27:04.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.987+0000 7fbea9c33740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3f8a11e10 con 0x55e3f8a24340 2024-09-13T18:27:04.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.987+0000 7fbe9ffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e3f8a24340 0x55e3f8a0d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.987+0000 7fbe9ffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e3f8a24340 0x55e3f8a0d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:60104/0 (socket says 172.21.15.114:60104) 2024-09-13T18:27:04.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.987+0000 7fbe9ffff640 1 -- 172.21.15.114:0/358657424 learned_addr learned my addr 172.21.15.114:0/358657424 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:04.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.988+0000 7fbea54ae640 1 --2- 172.21.15.114:0/358657424 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e3f8a0ca20 0x55e3f8a12a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.988+0000 7fbea4cad640 1 --2- 172.21.15.114:0/358657424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3f8a111b0 0x55e3f8a14150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:04.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.988+0000 7fbea54ae640 1 -- 172.21.15.114:0/358657424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e3f8a24340 msgr2=0x55e3f8a0d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:04.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.988+0000 7fbea54ae640 1 --2- 172.21.15.114:0/358657424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e3f8a24340 0x55e3f8a0d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.988+0000 7fbea54ae640 1 -- 172.21.15.114:0/358657424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3f8a111b0 msgr2=0x55e3f8a14150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:04.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.988+0000 7fbea54ae640 1 --2- 172.21.15.114:0/358657424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3f8a111b0 0x55e3f8a14150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.988+0000 7fbea54ae640 1 -- 172.21.15.114:0/358657424 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3f8914770 con 0x55e3f8a0ca20 2024-09-13T18:27:04.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.988+0000 7fbea4cad640 1 --2- 172.21.15.114:0/358657424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3f8a111b0 0x55e3f8a14150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:27:04.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.989+0000 7fbea54ae640 1 --2- 172.21.15.114:0/358657424 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e3f8a0ca20 0x55e3f8a12a50 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fbe90031040 tx=0x7fbe9002a930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34250c3e07f9fcb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:04.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.989+0000 7fbe877fe640 1 -- 172.21.15.114:0/358657424 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe90003b70 con 0x55e3f8a0ca20 2024-09-13T18:27:04.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.990+0000 7fbe877fe640 1 -- 172.21.15.114:0/358657424 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbe90031910 con 0x55e3f8a0ca20 2024-09-13T18:27:04.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.991+0000 7fbea9c33740 1 -- 172.21.15.114:0/358657424 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e3f8a0ca20 msgr2=0x55e3f8a12a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:04.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.991+0000 7fbea9c33740 1 --2- 172.21.15.114:0/358657424 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e3f8a0ca20 0x55e3f8a12a50 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fbe90031040 tx=0x7fbe9002a930 comp rx=0 tx=0).stop 2024-09-13T18:27:04.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.994+0000 7fbea9c33740 1 -- 172.21.15.114:0/358657424 shutdown_connections 2024-09-13T18:27:04.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.994+0000 7fbea9c33740 1 --2- 172.21.15.114:0/358657424 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e3f8a0ca20 0x55e3f8a12a50 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.994+0000 7fbea9c33740 1 --2- 172.21.15.114:0/358657424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e3f8a24340 0x55e3f8a0d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.994+0000 7fbea9c33740 1 --2- 172.21.15.114:0/358657424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3f8a111b0 0x55e3f8a14150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:04.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.994+0000 7fbea9c33740 1 -- 172.21.15.114:0/358657424 >> 172.21.15.114:0/358657424 conn(0x55e3f8a0cee0 msgr2=0x55e3f8a12340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:04.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.996+0000 7fbea9c33740 1 -- 172.21.15.114:0/358657424 shutdown_connections 2024-09-13T18:27:05.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.997+0000 7fbea9c33740 1 -- 172.21.15.114:0/358657424 wait complete. 2024-09-13T18:27:05.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.997+0000 7fbea9c33740 1 -- start start 2024-09-13T18:27:05.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.997+0000 7fbea9c33740 1 Processor -- start 2024-09-13T18:27:05.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.998+0000 7fbea9c33740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e3f8a0ca20 0x55e3f8a21dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:05.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.998+0000 7fbea9c33740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3f8a111b0 0x55e3f8a22300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:05.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.998+0000 7fbea9c33740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e3f8a24340 0x55e3f8a92ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:05.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.998+0000 7fbea9c33740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3f8a93010 con 0x55e3f8a0ca20 2024-09-13T18:27:05.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.998+0000 7fbea9c33740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3f8a93150 con 0x55e3f8a111b0 2024-09-13T18:27:05.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.998+0000 7fbea9c33740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3f8a932c0 con 0x55e3f8a24340 2024-09-13T18:27:05.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.998+0000 7fbea9c33740 10 client.0 _ll_register_callbacks cb 0x7fbe90009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:27:05.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.999+0000 7fbe9ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3f8a111b0 0x55e3f8a22300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:05.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.999+0000 7fbe9ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3f8a111b0 0x55e3f8a22300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:36122/0 (socket says 172.21.15.114:36122) 2024-09-13T18:27:05.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.999+0000 7fbe9ffff640 1 -- 172.21.15.114:0/3227890663 learned_addr learned my addr 172.21.15.114:0/3227890663 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:05.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.999+0000 7fbea54ae640 1 --2- 172.21.15.114:0/3227890663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e3f8a24340 0x55e3f8a92ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:05.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.999+0000 7fbea4cad640 1 --2- 172.21.15.114:0/3227890663 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e3f8a0ca20 0x55e3f8a21dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:05.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.999+0000 7fbe9ffff640 1 -- 172.21.15.114:0/3227890663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e3f8a24340 msgr2=0x55e3f8a92ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:05.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.999+0000 7fbe9ffff640 1 --2- 172.21.15.114:0/3227890663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e3f8a24340 0x55e3f8a92ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.999+0000 7fbe9ffff640 1 -- 172.21.15.114:0/3227890663 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e3f8a0ca20 msgr2=0x55e3f8a21dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:05.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.999+0000 7fbe9ffff640 1 --2- 172.21.15.114:0/3227890663 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e3f8a0ca20 0x55e3f8a21dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:04.999+0000 7fbe9ffff640 1 -- 172.21.15.114:0/3227890663 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e3f893b620 con 0x55e3f8a111b0 2024-09-13T18:27:05.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.000+0000 7fbe9ffff640 1 --2- 172.21.15.114:0/3227890663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3f8a111b0 0x55e3f8a22300 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbe980a17f0 tx=0x7fbe980cedc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86e044033fa387c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:05.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.001+0000 7fbe7ffff640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:27:05.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.001+0000 7fbe7ffff640 1 -- 172.21.15.114:0/3227890663 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe980ceea0 con 0x55e3f8a111b0 2024-09-13T18:27:05.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.001+0000 7fbea9c33740 1 -- 172.21.15.114:0/3227890663 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbe90040070 con 0x55e3f8a111b0 2024-09-13T18:27:05.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.001+0000 7fbe7ffff640 1 -- 172.21.15.114:0/3227890663 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fbe980998a0 con 0x55e3f8a111b0 2024-09-13T18:27:05.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.003+0000 7fbe7ffff640 1 -- 172.21.15.114:0/3227890663 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe980b2270 con 0x55e3f8a111b0 2024-09-13T18:27:05.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.003+0000 7fbea9c33740 10 client.26489 fetch_fsmap learned FSMap version 32 2024-09-13T18:27:05.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.003+0000 7fbea9c33740 1 -- 172.21.15.114:0/3227890663 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55e3f8939e80 con 0x55e3f8a111b0 2024-09-13T18:27:05.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.004+0000 7fbe7ffff640 1 -- 172.21.15.114:0/3227890663 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbe90040070 con 0x55e3f8a111b0 2024-09-13T18:27:05.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.004+0000 7fbe7ffff640 1 -- 172.21.15.114:0/3227890663 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fbe98095fb0 con 0x55e3f8a111b0 2024-09-13T18:27:05.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.004+0000 7fbea9c33740 10 client.26489 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:27:05.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.004+0000 7fbea9c33740 10 client.26489 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-13T18:27:05.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.004+0000 7fbea9c33740 20 client.26489 populate_metadata read hostname 'smithi114' 2024-09-13T18:27:05.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.004+0000 7fbea9c33740 1 --2- 172.21.15.114:0/3227890663 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55e3f8a94b30 0x55e3f8a96f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:05.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.004+0000 7fbea9c33740 4 client.26489 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:27:05.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.004+0000 7fbea9c33740 1 -- 172.21.15.114:0/3227890663 --> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e3f86674e0 con 0x55e3f8a94b30 2024-09-13T18:27:05.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.010+0000 7fbea4cad640 1 --2- 172.21.15.114:0/3227890663 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55e3f8a94b30 0x55e3f8a96f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:05.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.016+0000 7fbea4cad640 1 --2- 172.21.15.114:0/3227890663 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55e3f8a94b30 0x55e3f8a96f50 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b44939ca95c01a27 server_cookie=e5b251ea257a2963 in_seq=0 out_seq=0 2024-09-13T18:27:05.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.019+0000 7fbe7ffff640 10 client.26489 ms_handle_connect on v2:172.21.15.134:6836/891692210 2024-09-13T18:27:05.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.019+0000 7fbe7ffff640 1 -- 172.21.15.114:0/3227890663 <== mds.1 v2:172.21.15.134:6836/891692210 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e3f86674e0 con 0x55e3f8a94b30 2024-09-13T18:27:05.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.019+0000 7fbe7ffff640 10 client.26489 handle_command_reply: tid=0 2024-09-13T18:27:05.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.019+0000 7fbea9c33740 1 client.26489 shutdown 2024-09-13T18:27:05.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.020+0000 7fbea9c33740 1 -- 172.21.15.114:0/3227890663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3f8a111b0 msgr2=0x55e3f8a22300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:05.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.020+0000 7fbea9c33740 1 --2- 172.21.15.114:0/3227890663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3f8a111b0 0x55e3f8a22300 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbe980a17f0 tx=0x7fbe980cedc0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.021+0000 7fbea9c33740 1 -- 172.21.15.114:0/3227890663 shutdown_connections 2024-09-13T18:27:05.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.021+0000 7fbea9c33740 1 --2- 172.21.15.114:0/3227890663 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55e3f8a94b30 0x55e3f8a96f50 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.021+0000 7fbea9c33740 1 --2- 172.21.15.114:0/3227890663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e3f8a24340 0x55e3f8a92ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.021+0000 7fbea9c33740 1 --2- 172.21.15.114:0/3227890663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3f8a111b0 0x55e3f8a22300 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.021+0000 7fbea9c33740 1 --2- 172.21.15.114:0/3227890663 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e3f8a0ca20 0x55e3f8a21dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.021+0000 7fbea9c33740 1 -- 172.21.15.114:0/3227890663 >> 172.21.15.114:0/3227890663 conn(0x55e3f8a0cee0 msgr2=0x55e3f8a0c230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:05.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.025+0000 7fbe7ffff640 0 client.26489 ms_handle_reset on v2:172.21.15.134:6836/891692210 2024-09-13T18:27:05.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.025+0000 7fbea9c33740 1 -- 172.21.15.114:0/3227890663 shutdown_connections 2024-09-13T18:27:05.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.025+0000 7fbea9c33740 1 -- 172.21.15.114:0/3227890663 wait complete. 2024-09-13T18:27:05.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.026+0000 7fbea9c33740 20 client.26489 trim_cache size 0 max 16384 2024-09-13T18:27:05.029 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:27:05.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.030+0000 7fbea5caf640 1 -- 172.21.15.114:0/650607111 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbe880a1420 msgr2=0x7fbe880a38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:05.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.030+0000 7fbea5caf640 1 --2- 172.21.15.114:0/650607111 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbe880a1420 0x7fbe880a38e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbe940093b0 tx=0x7fbe94009340 comp rx=0 tx=0).stop 2024-09-13T18:27:05.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.030+0000 7fbea5caf640 1 -- 172.21.15.114:0/650607111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbea0103f00 msgr2=0x7fbea0072160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:05.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.030+0000 7fbea5caf640 1 --2- 172.21.15.114:0/650607111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbea0103f00 0x7fbea0072160 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbe98009fd0 tx=0x7fbe9800b870 comp rx=0 tx=0).stop 2024-09-13T18:27:05.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.035+0000 7fbea5caf640 1 -- 172.21.15.114:0/650607111 shutdown_connections 2024-09-13T18:27:05.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.035+0000 7fbea5caf640 1 --2- 172.21.15.114:0/650607111 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbe880a1420 0x7fbe880a38e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.035+0000 7fbea5caf640 1 --2- 172.21.15.114:0/650607111 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbea006ade0 0x7fbea006b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.035+0000 7fbea5caf640 1 --2- 172.21.15.114:0/650607111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbea0103f00 0x7fbea0072160 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.035+0000 7fbea5caf640 1 --2- 172.21.15.114:0/650607111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbea00fedb0 0x7fbea0071c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.035+0000 7fbea5caf640 1 -- 172.21.15.114:0/650607111 >> 172.21.15.114:0/650607111 conn(0x7fbea00fa840 msgr2=0x7fbea00fbd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:05.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.036+0000 7fbea5caf640 1 -- 172.21.15.114:0/650607111 shutdown_connections 2024-09-13T18:27:05.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.036+0000 7fbea5caf640 1 -- 172.21.15.114:0/650607111 wait complete. 2024-09-13T18:27:05.079 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-09-13T18:27:05.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.306+0000 7f6c91841640 1 -- 172.21.15.114:0/3647506960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c8c119400 msgr2=0x7f6c8c11b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:05.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.306+0000 7f6c91841640 1 --2- 172.21.15.114:0/3647506960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c8c119400 0x7f6c8c11b7f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6c84009ca0 tx=0x7f6c8402a8e0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.312+0000 7f6c91841640 1 -- 172.21.15.114:0/3647506960 shutdown_connections 2024-09-13T18:27:05.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.312+0000 7f6c91841640 1 --2- 172.21.15.114:0/3647506960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c8c119400 0x7f6c8c11b7f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.312+0000 7f6c91841640 1 --2- 172.21.15.114:0/3647506960 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c8c107920 0x7f6c8c107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.312+0000 7f6c91841640 1 --2- 172.21.15.114:0/3647506960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c8c107540 0x7f6c8c10e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.312+0000 7f6c91841640 1 -- 172.21.15.114:0/3647506960 >> 172.21.15.114:0/3647506960 conn(0x7f6c8c06afe0 msgr2=0x7f6c8c06b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:05.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.321+0000 7f6c91841640 1 -- 172.21.15.114:0/3647506960 shutdown_connections 2024-09-13T18:27:05.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.324+0000 7f6c91841640 1 -- 172.21.15.114:0/3647506960 wait complete. 2024-09-13T18:27:05.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.325+0000 7f6c91841640 1 Processor -- start 2024-09-13T18:27:05.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.325+0000 7f6c91841640 1 -- start start 2024-09-13T18:27:05.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.325+0000 7f6c91841640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c8c107540 0x7f6c8c1a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:05.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.325+0000 7f6c91841640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c8c107920 0x7f6c8c1a55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:05.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.325+0000 7f6c91841640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c8c119400 0x7f6c8c1a99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:05.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.325+0000 7f6c91841640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8c1a9fd0 con 0x7f6c8c107920 2024-09-13T18:27:05.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.325+0000 7f6c91841640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8c1aa140 con 0x7f6c8c119400 2024-09-13T18:27:05.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.325+0000 7f6c91841640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8c1aa300 con 0x7f6c8c107540 2024-09-13T18:27:05.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.326+0000 7f6c8affd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c8c107540 0x7f6c8c1a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:05.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.326+0000 7f6c8affd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c8c107540 0x7f6c8c1a50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:60122/0 (socket says 172.21.15.114:60122) 2024-09-13T18:27:05.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.326+0000 7f6c8affd640 1 -- 172.21.15.114:0/302691400 learned_addr learned my addr 172.21.15.114:0/302691400 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:05.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.326+0000 7f6c8b7fe640 1 --2- 172.21.15.114:0/302691400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c8c119400 0x7f6c8c1a99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:05.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.326+0000 7f6c8a7fc640 1 --2- 172.21.15.114:0/302691400 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c8c107920 0x7f6c8c1a55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:05.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.326+0000 7f6c8b7fe640 1 -- 172.21.15.114:0/302691400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c8c107540 msgr2=0x7f6c8c1a50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:05.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.326+0000 7f6c8b7fe640 1 --2- 172.21.15.114:0/302691400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c8c107540 0x7f6c8c1a50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.326+0000 7f6c8b7fe640 1 -- 172.21.15.114:0/302691400 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c8c107920 msgr2=0x7f6c8c1a55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:05.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.326+0000 7f6c8b7fe640 1 --2- 172.21.15.114:0/302691400 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c8c107920 0x7f6c8c1a55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.326+0000 7f6c8b7fe640 1 -- 172.21.15.114:0/302691400 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c84009900 con 0x7f6c8c119400 2024-09-13T18:27:05.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.327+0000 7f6c8b7fe640 1 --2- 172.21.15.114:0/302691400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c8c119400 0x7f6c8c1a99d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6c8402ae50 tx=0x7f6c84031a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:05.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.329+0000 7f6c9083f640 1 -- 172.21.15.114:0/302691400 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c840309f0 con 0x7f6c8c119400 2024-09-13T18:27:05.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.329+0000 7f6c9083f640 1 -- 172.21.15.114:0/302691400 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6c84031ba0 con 0x7f6c8c119400 2024-09-13T18:27:05.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.329+0000 7f6c9083f640 1 -- 172.21.15.114:0/302691400 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8402f850 con 0x7f6c8c119400 2024-09-13T18:27:05.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.331+0000 7f6c91841640 1 -- 172.21.15.114:0/302691400 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c8c1aa580 con 0x7f6c8c119400 2024-09-13T18:27:05.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.331+0000 7f6c91841640 1 -- 172.21.15.114:0/302691400 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c8c1b1e60 con 0x7f6c8c119400 2024-09-13T18:27:05.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.334+0000 7f6c94642740 1 Processor -- start 2024-09-13T18:27:05.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.334+0000 7f6c94642740 1 -- start start 2024-09-13T18:27:05.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.334+0000 7f6c94642740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5574d3457790 0x5574d3457b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:05.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.334+0000 7f6c94642740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574d34580b0 0x5574d3462410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:05.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.334+0000 7f6c94642740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574d3462950 0x5574d3464d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:05.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.334+0000 7f6c94642740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574d33c1cc0 con 0x5574d3457790 2024-09-13T18:27:05.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.334+0000 7f6c94642740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574d33bf450 con 0x5574d3462950 2024-09-13T18:27:05.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.334+0000 7f6c94642740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574d33bad40 con 0x5574d34580b0 2024-09-13T18:27:05.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.335+0000 7f6c8a7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574d3462950 0x5574d3464d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:05.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.335+0000 7f6c8a7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574d3462950 0x5574d3464d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:36144/0 (socket says 172.21.15.114:36144) 2024-09-13T18:27:05.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.335+0000 7f6c8a7fc640 1 -- 172.21.15.114:0/3931313443 learned_addr learned my addr 172.21.15.114:0/3931313443 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:05.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.335+0000 7f6c8affd640 1 --2- 172.21.15.114:0/3931313443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574d34580b0 0x5574d3462410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:05.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.335+0000 7f6c8a7fc640 1 -- 172.21.15.114:0/3931313443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574d34580b0 msgr2=0x5574d3462410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:05.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.335+0000 7f6c8a7fc640 1 --2- 172.21.15.114:0/3931313443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574d34580b0 0x5574d3462410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.335+0000 7f6c8a7fc640 1 -- 172.21.15.114:0/3931313443 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5574d3457790 msgr2=0x5574d3457b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:05.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.335+0000 7f6c8a7fc640 1 --2- 172.21.15.114:0/3931313443 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5574d3457790 0x5574d3457b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.335+0000 7f6c8a7fc640 1 -- 172.21.15.114:0/3931313443 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574d3382620 con 0x5574d3462950 2024-09-13T18:27:05.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.338+0000 7f6c8a7fc640 1 --2- 172.21.15.114:0/3931313443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574d3462950 0x5574d3464d40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6c80009aa0 tx=0x7f6c80009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae63637ec4a73e22 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:05.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.338+0000 7f6c71ffb640 1 -- 172.21.15.114:0/3931313443 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c800329f0 con 0x5574d3462950 2024-09-13T18:27:05.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.338+0000 7f6c71ffb640 1 -- 172.21.15.114:0/3931313443 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6c80036480 con 0x5574d3462950 2024-09-13T18:27:05.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.339+0000 7f6c94642740 1 -- 172.21.15.114:0/3931313443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574d3462950 msgr2=0x5574d3464d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:05.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.339+0000 7f6c94642740 1 --2- 172.21.15.114:0/3931313443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574d3462950 0x5574d3464d40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6c80009aa0 tx=0x7f6c80009f60 comp rx=0 tx=0).stop 2024-09-13T18:27:05.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.339+0000 7f6c9083f640 1 -- 172.21.15.114:0/302691400 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f6c84042070 con 0x7f6c8c119400 2024-09-13T18:27:05.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.341+0000 7f6c9083f640 1 --2- 172.21.15.114:0/302691400 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f6c740a1410 0x7f6c740a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:05.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.342+0000 7f6c8a7fc640 1 --2- 172.21.15.114:0/302691400 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f6c740a1410 0x7f6c740a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:05.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.342+0000 7f6c9083f640 1 -- 172.21.15.114:0/302691400 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f6c840ee8d0 con 0x7f6c8c119400 2024-09-13T18:27:05.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.346+0000 7f6c94642740 1 -- 172.21.15.114:0/3931313443 shutdown_connections 2024-09-13T18:27:05.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.346+0000 7f6c94642740 1 --2- 172.21.15.114:0/3931313443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574d3462950 0x5574d3464d40 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.346+0000 7f6c94642740 1 --2- 172.21.15.114:0/3931313443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574d34580b0 0x5574d3462410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.346+0000 7f6c94642740 1 --2- 172.21.15.114:0/3931313443 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5574d3457790 0x5574d3457b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.346+0000 7f6c94642740 1 -- 172.21.15.114:0/3931313443 >> 172.21.15.114:0/3931313443 conn(0x5574d3453d00 msgr2=0x5574d34540e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:05.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.347+0000 7f6c94642740 1 -- 172.21.15.114:0/3931313443 shutdown_connections 2024-09-13T18:27:05.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.347+0000 7f6c94642740 1 -- 172.21.15.114:0/3931313443 wait complete. 2024-09-13T18:27:05.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.348+0000 7f6c94642740 1 -- start start 2024-09-13T18:27:05.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.348+0000 7f6c94642740 1 Processor -- start 2024-09-13T18:27:05.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.348+0000 7f6c94642740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5574d3457790 0x5574d3461e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:05.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.349+0000 7f6c94642740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574d34580b0 0x5574d3462390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:05.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.349+0000 7f6c94642740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574d3462950 0x5574d345f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:05.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.349+0000 7f6c94642740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574d3379180 con 0x5574d3457790 2024-09-13T18:27:05.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.349+0000 7f6c94642740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574d3093430 con 0x5574d3462950 2024-09-13T18:27:05.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.349+0000 7f6c94642740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574d333e5e0 con 0x5574d34580b0 2024-09-13T18:27:05.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.349+0000 7f6c8affd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5574d3457790 0x5574d3461e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:05.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.349+0000 7f6c8affd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5574d3457790 0x5574d3461e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60030/0 (socket says 172.21.15.114:60030) 2024-09-13T18:27:05.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.349+0000 7f6c8affd640 1 -- 172.21.15.114:0/75648431 learned_addr learned my addr 172.21.15.114:0/75648431 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:05.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.349+0000 7f6c8a7fc640 1 --2- 172.21.15.114:0/75648431 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574d34580b0 0x5574d3462390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:05.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.349+0000 7f6c94642740 10 client.0 _ll_register_callbacks cb 0x5574d33aaaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:27:05.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.349+0000 7f6c8affd640 1 -- 172.21.15.114:0/75648431 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574d34580b0 msgr2=0x5574d3462390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:05.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.349+0000 7f6c8affd640 1 --2- 172.21.15.114:0/75648431 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574d34580b0 0x5574d3462390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.349+0000 7f6c8affd640 1 -- 172.21.15.114:0/75648431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574d3462950 msgr2=0x5574d345f3c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:27:05.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.349+0000 7f6c8affd640 1 --2- 172.21.15.114:0/75648431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574d3462950 0x5574d345f3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.349+0000 7f6c8affd640 1 -- 172.21.15.114:0/75648431 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c80009750 con 0x5574d3457790 2024-09-13T18:27:05.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.350+0000 7f6c8affd640 1 --2- 172.21.15.114:0/75648431 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5574d3457790 0x5574d3461e50 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c0128d0 tx=0x7f6c7c012d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdaddde74cd26b2c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:05.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.351+0000 7f6c6affd640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:27:05.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.351+0000 7f6c6affd640 1 -- 172.21.15.114:0/75648431 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c004250 con 0x5574d3457790 2024-09-13T18:27:05.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.351+0000 7f6c94642740 1 -- 172.21.15.114:0/75648431 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5574d33e4b80 con 0x5574d3457790 2024-09-13T18:27:05.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.351+0000 7f6c6affd640 1 -- 172.21.15.114:0/75648431 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f6c7c0043b0 con 0x5574d3457790 2024-09-13T18:27:05.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.352+0000 7f6c8a7fc640 1 --2- 172.21.15.114:0/302691400 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f6c740a1410 0x7f6c740a38d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6c80036fd0 tx=0x7f6c80009450 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:05.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.352+0000 7f6c6affd640 1 -- 172.21.15.114:0/75648431 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c7c00f920 con 0x5574d3457790 2024-09-13T18:27:05.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.352+0000 7f6c94642740 10 client.17151 fetch_fsmap learned FSMap version 32 2024-09-13T18:27:05.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.352+0000 7f6c94642740 1 -- 172.21.15.114:0/75648431 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5574d33b93c0 con 0x5574d3457790 2024-09-13T18:27:05.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.353+0000 7f6c6affd640 1 -- 172.21.15.114:0/75648431 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5574d33e4b80 con 0x5574d3457790 2024-09-13T18:27:05.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.353+0000 7f6c6affd640 1 -- 172.21.15.114:0/75648431 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f6c7c003c60 con 0x5574d3457790 2024-09-13T18:27:05.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.354+0000 7f6c94642740 10 client.17151 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:27:05.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.354+0000 7f6c94642740 10 client.17151 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:27:05.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.354+0000 7f6c94642740 20 client.17151 populate_metadata read hostname 'smithi114' 2024-09-13T18:27:05.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.354+0000 7f6c94642740 1 --2- 172.21.15.114:0/75648431 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5574d346ff00 0x5574d3472340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:05.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.354+0000 7f6c94642740 4 client.17151 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:27:05.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.354+0000 7f6c94642740 1 -- 172.21.15.114:0/75648431 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5574d33ad1f0 con 0x5574d346ff00 2024-09-13T18:27:05.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.354+0000 7f6c8a7fc640 1 --2- 172.21.15.114:0/75648431 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5574d346ff00 0x5574d3472340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:05.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.355+0000 7f6c8a7fc640 1 --2- 172.21.15.114:0/75648431 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5574d346ff00 0x5574d3472340 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1a0f36c151a5acb server_cookie=b36f090fafbe5239 in_seq=0 out_seq=0 2024-09-13T18:27:05.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.355+0000 7f6c6affd640 10 client.17151 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:27:05.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.356+0000 7f6c6affd640 1 -- 172.21.15.114:0/75648431 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5574d33ad1f0 con 0x5574d346ff00 2024-09-13T18:27:05.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.356+0000 7f6c6affd640 10 client.17151 handle_command_reply: tid=0 2024-09-13T18:27:05.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.357+0000 7f6c94642740 1 client.17151 shutdown 2024-09-13T18:27:05.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.358+0000 7f6c94642740 1 -- 172.21.15.114:0/75648431 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5574d3457790 msgr2=0x5574d3461e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:05.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.358+0000 7f6c94642740 1 --2- 172.21.15.114:0/75648431 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5574d3457790 0x5574d3461e50 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c0128d0 tx=0x7f6c7c012d90 comp rx=0 tx=0).stop 2024-09-13T18:27:05.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.358+0000 7f6c94642740 1 -- 172.21.15.114:0/75648431 shutdown_connections 2024-09-13T18:27:05.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.358+0000 7f6c94642740 1 --2- 172.21.15.114:0/75648431 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5574d346ff00 0x5574d3472340 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.358+0000 7f6c94642740 1 --2- 172.21.15.114:0/75648431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574d3462950 0x5574d345f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.358+0000 7f6c94642740 1 --2- 172.21.15.114:0/75648431 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574d34580b0 0x5574d3462390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.358+0000 7f6c94642740 1 --2- 172.21.15.114:0/75648431 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5574d3457790 0x5574d3461e50 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.358+0000 7f6c94642740 1 -- 172.21.15.114:0/75648431 >> 172.21.15.114:0/75648431 conn(0x5574d3453d00 msgr2=0x5574d3459a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:05.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.358+0000 7f6c6affd640 0 client.17151 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:27:05.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.359+0000 7f6c94642740 1 -- 172.21.15.114:0/75648431 shutdown_connections 2024-09-13T18:27:05.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.359+0000 7f6c94642740 1 -- 172.21.15.114:0/75648431 wait complete. 2024-09-13T18:27:05.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.361+0000 7f6c94642740 20 client.17151 trim_cache size 0 max 16384 2024-09-13T18:27:05.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.389+0000 7f6c94642740 1 Processor -- start 2024-09-13T18:27:05.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.390+0000 7f6c94642740 1 -- start start 2024-09-13T18:27:05.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.390+0000 7f6c94642740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574d3457790 0x5574d347f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:05.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.391+0000 7f6c94642740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5574d34580b0 0x5574d3469740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:05.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.391+0000 7f6c94642740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574d3462950 0x5574d3469c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:05.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.391+0000 7f6c94642740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574d3458750 con 0x5574d34580b0 2024-09-13T18:27:05.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.391+0000 7f6c8affd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574d3457790 0x5574d347f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:05.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.391+0000 7f6c8affd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574d3457790 0x5574d347f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:36148/0 (socket says 172.21.15.114:36148) 2024-09-13T18:27:05.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.391+0000 7f6c8affd640 1 -- 172.21.15.114:0/2698172658 learned_addr learned my addr 172.21.15.114:0/2698172658 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:05.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.391+0000 7f6c8b7fe640 1 --2- 172.21.15.114:0/2698172658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574d3462950 0x5574d3469c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:05.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.391+0000 7f6c8a7fc640 1 --2- 172.21.15.114:0/2698172658 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5574d34580b0 0x5574d3469740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:05.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.391+0000 7f6c94642740 1 -- 172.21.15.114:0/2698172658 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574d344f8e0 con 0x5574d3457790 2024-09-13T18:27:05.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.391+0000 7f6c94642740 1 -- 172.21.15.114:0/2698172658 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574d345d6a0 con 0x5574d3462950 2024-09-13T18:27:05.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.392+0000 7f6c8a7fc640 1 -- 172.21.15.114:0/2698172658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574d3462950 msgr2=0x5574d3469c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:05.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.392+0000 7f6c8a7fc640 1 --2- 172.21.15.114:0/2698172658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574d3462950 0x5574d3469c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.392+0000 7f6c8a7fc640 1 -- 172.21.15.114:0/2698172658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574d3457790 msgr2=0x5574d347f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:05.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.392+0000 7f6c8a7fc640 1 --2- 172.21.15.114:0/2698172658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574d3457790 0x5574d347f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.392+0000 7f6c8a7fc640 1 -- 172.21.15.114:0/2698172658 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574d335b770 con 0x5574d34580b0 2024-09-13T18:27:05.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.392+0000 7f6c8a7fc640 1 --2- 172.21.15.114:0/2698172658 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5574d34580b0 0x5574d3469740 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f6c800094c0 tx=0x7f6c80008070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=651d221c22e1e00e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:05.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.392+0000 7f6c727fc640 1 -- 172.21.15.114:0/2698172658 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c80045070 con 0x5574d34580b0 2024-09-13T18:27:05.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.392+0000 7f6c727fc640 1 -- 172.21.15.114:0/2698172658 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6c80035b00 con 0x5574d34580b0 2024-09-13T18:27:05.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.393+0000 7f6c727fc640 1 -- 172.21.15.114:0/2698172658 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8004d460 con 0x5574d34580b0 2024-09-13T18:27:05.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.394+0000 7f6c94642740 1 -- 172.21.15.114:0/2698172658 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5574d34580b0 msgr2=0x5574d3469740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:05.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.394+0000 7f6c94642740 1 --2- 172.21.15.114:0/2698172658 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5574d34580b0 0x5574d3469740 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f6c800094c0 tx=0x7f6c80008070 comp rx=0 tx=0).stop 2024-09-13T18:27:05.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.397+0000 7f6c94642740 1 -- 172.21.15.114:0/2698172658 shutdown_connections 2024-09-13T18:27:05.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.397+0000 7f6c94642740 1 --2- 172.21.15.114:0/2698172658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574d3462950 0x5574d3469c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.397+0000 7f6c94642740 1 --2- 172.21.15.114:0/2698172658 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5574d34580b0 0x5574d3469740 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.397+0000 7f6c94642740 1 --2- 172.21.15.114:0/2698172658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574d3457790 0x5574d347f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.397+0000 7f6c94642740 1 -- 172.21.15.114:0/2698172658 >> 172.21.15.114:0/2698172658 conn(0x5574d3453d00 msgr2=0x5574d345a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:05.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.400+0000 7f6c94642740 1 -- 172.21.15.114:0/2698172658 shutdown_connections 2024-09-13T18:27:05.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.400+0000 7f6c94642740 1 -- 172.21.15.114:0/2698172658 wait complete. 2024-09-13T18:27:05.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.400+0000 7f6c94642740 1 -- start start 2024-09-13T18:27:05.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.401+0000 7f6c94642740 1 Processor -- start 2024-09-13T18:27:05.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.401+0000 7f6c94642740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574d3457790 0x5574d3466b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:05.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.401+0000 7f6c94642740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5574d34580b0 0x5574d3467070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:05.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.402+0000 7f6c94642740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574d3462950 0x5574d34c1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:05.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.402+0000 7f6c94642740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574d34c2300 con 0x5574d34580b0 2024-09-13T18:27:05.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.402+0000 7f6c94642740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574d34c2470 con 0x5574d3457790 2024-09-13T18:27:05.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.402+0000 7f6c94642740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574d34c2630 con 0x5574d3462950 2024-09-13T18:27:05.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.402+0000 7f6c8a7fc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5574d34580b0 0x5574d3467070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:05.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.402+0000 7f6c8b7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574d3462950 0x5574d34c1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:05.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.402+0000 7f6c8a7fc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5574d34580b0 0x5574d3467070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60038/0 (socket says 172.21.15.114:60038) 2024-09-13T18:27:05.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.402+0000 7f6c8a7fc640 1 -- 172.21.15.114:0/2259769705 learned_addr learned my addr 172.21.15.114:0/2259769705 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:05.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.402+0000 7f6c94642740 10 client.0 _ll_register_callbacks cb 0x7f6c800094f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:27:05.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.403+0000 7f6c8a7fc640 1 -- 172.21.15.114:0/2259769705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574d3462950 msgr2=0x5574d34c1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:05.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.403+0000 7f6c8a7fc640 1 --2- 172.21.15.114:0/2259769705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574d3462950 0x5574d34c1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.403+0000 7f6c8a7fc640 1 -- 172.21.15.114:0/2259769705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574d3457790 msgr2=0x5574d3466b30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:27:05.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.403+0000 7f6c8a7fc640 1 --2- 172.21.15.114:0/2259769705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574d3457790 0x5574d3466b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.403+0000 7f6c8a7fc640 1 -- 172.21.15.114:0/2259769705 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5574d3382620 con 0x5574d34580b0 2024-09-13T18:27:05.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.404+0000 7f6c8a7fc640 1 --2- 172.21.15.114:0/2259769705 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5574d34580b0 0x5574d3467070 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f6c80009a40 tx=0x7f6c80035de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ab6a7ba50a42b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:05.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.404+0000 7f6c6affd640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:27:05.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.404+0000 7f6c6affd640 1 -- 172.21.15.114:0/2259769705 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c80045070 con 0x5574d34580b0 2024-09-13T18:27:05.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.405+0000 7f6c94642740 1 -- 172.21.15.114:0/2259769705 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6c80036480 con 0x5574d34580b0 2024-09-13T18:27:05.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.405+0000 7f6c6affd640 1 -- 172.21.15.114:0/2259769705 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f6c80053070 con 0x5574d34580b0 2024-09-13T18:27:05.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.408+0000 7f6c6affd640 1 -- 172.21.15.114:0/2259769705 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c80004040 con 0x5574d34580b0 2024-09-13T18:27:05.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.408+0000 7f6c94642740 10 client.17157 fetch_fsmap learned FSMap version 32 2024-09-13T18:27:05.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.408+0000 7f6c94642740 1 -- 172.21.15.114:0/2259769705 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5574d3380e80 con 0x5574d34580b0 2024-09-13T18:27:05.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.411+0000 7f6c6affd640 1 -- 172.21.15.114:0/2259769705 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6c80004630 con 0x5574d34580b0 2024-09-13T18:27:05.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.411+0000 7f6c6affd640 1 -- 172.21.15.114:0/2259769705 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f6c8004d460 con 0x5574d34580b0 2024-09-13T18:27:05.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.412+0000 7f6c94642740 10 client.17157 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:27:05.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.412+0000 7f6c94642740 10 client.17157 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:27:05.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.412+0000 7f6c94642740 20 client.17157 populate_metadata read hostname 'smithi114' 2024-09-13T18:27:05.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.412+0000 7f6c94642740 1 --2- 172.21.15.114:0/2259769705 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5574d34c9c30 0x5574d34cc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:05.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.412+0000 7f6c94642740 4 client.17157 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:27:05.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.412+0000 7f6c94642740 1 -- 172.21.15.114:0/2259769705 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f6c80036480 con 0x5574d34c9c30 2024-09-13T18:27:05.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.413+0000 7f6c8affd640 1 --2- 172.21.15.114:0/2259769705 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5574d34c9c30 0x5574d34cc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:05.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.417+0000 7f6c8affd640 1 --2- 172.21.15.114:0/2259769705 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5574d34c9c30 0x5574d34cc050 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ddd1be1c72f159d server_cookie=833bafa57a24454 in_seq=0 out_seq=0 2024-09-13T18:27:05.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.417+0000 7f6c6affd640 10 client.17157 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:27:05.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.417+0000 7f6c6affd640 1 -- 172.21.15.114:0/2259769705 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f6c80036480 con 0x5574d34c9c30 2024-09-13T18:27:05.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.417+0000 7f6c6affd640 10 client.17157 handle_command_reply: tid=0 2024-09-13T18:27:05.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.418+0000 7f6c94642740 1 client.17157 shutdown 2024-09-13T18:27:05.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.426+0000 7f6c94642740 1 -- 172.21.15.114:0/2259769705 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5574d34580b0 msgr2=0x5574d3467070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:05.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.426+0000 7f6c94642740 1 --2- 172.21.15.114:0/2259769705 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5574d34580b0 0x5574d3467070 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f6c80009a40 tx=0x7f6c80035de0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.431+0000 7f6c94642740 1 -- 172.21.15.114:0/2259769705 shutdown_connections 2024-09-13T18:27:05.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.431+0000 7f6c94642740 1 --2- 172.21.15.114:0/2259769705 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5574d34c9c30 0x5574d34cc050 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.431+0000 7f6c94642740 1 --2- 172.21.15.114:0/2259769705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574d3462950 0x5574d34c1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.431+0000 7f6c94642740 1 --2- 172.21.15.114:0/2259769705 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5574d34580b0 0x5574d3467070 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.431+0000 7f6c94642740 1 --2- 172.21.15.114:0/2259769705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574d3457790 0x5574d3466b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.431+0000 7f6c94642740 1 -- 172.21.15.114:0/2259769705 >> 172.21.15.114:0/2259769705 conn(0x5574d3453d00 msgr2=0x5574d345d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:05.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.432+0000 7f6c6affd640 0 client.17157 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:27:05.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.434+0000 7f6c94642740 1 -- 172.21.15.114:0/2259769705 shutdown_connections 2024-09-13T18:27:05.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.434+0000 7f6c94642740 1 -- 172.21.15.114:0/2259769705 wait complete. 2024-09-13T18:27:05.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.437+0000 7f6c94642740 20 client.17157 trim_cache size 0 max 16384 2024-09-13T18:27:05.441 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:27:05.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.441+0000 7f6c91841640 1 -- 172.21.15.114:0/302691400 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f6c740a1410 msgr2=0x7f6c740a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:05.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.441+0000 7f6c91841640 1 --2- 172.21.15.114:0/302691400 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f6c740a1410 0x7f6c740a38d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6c80036fd0 tx=0x7f6c80009450 comp rx=0 tx=0).stop 2024-09-13T18:27:05.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.441+0000 7f6c91841640 1 -- 172.21.15.114:0/302691400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c8c119400 msgr2=0x7f6c8c1a99d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:05.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.441+0000 7f6c91841640 1 --2- 172.21.15.114:0/302691400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c8c119400 0x7f6c8c1a99d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6c8402ae50 tx=0x7f6c84031a10 comp rx=0 tx=0).stop 2024-09-13T18:27:05.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.442+0000 7f6c91841640 1 -- 172.21.15.114:0/302691400 shutdown_connections 2024-09-13T18:27:05.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.442+0000 7f6c91841640 1 --2- 172.21.15.114:0/302691400 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f6c740a1410 0x7f6c740a38d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.442+0000 7f6c91841640 1 --2- 172.21.15.114:0/302691400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c8c119400 0x7f6c8c1a99d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.442+0000 7f6c91841640 1 --2- 172.21.15.114:0/302691400 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c8c107920 0x7f6c8c1a55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.442+0000 7f6c91841640 1 --2- 172.21.15.114:0/302691400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c8c107540 0x7f6c8c1a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:05.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.442+0000 7f6c91841640 1 -- 172.21.15.114:0/302691400 >> 172.21.15.114:0/302691400 conn(0x7f6c8c06afe0 msgr2=0x7f6c8c119ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:05.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.444+0000 7f6c91841640 1 -- 172.21.15.114:0/302691400 shutdown_connections 2024-09-13T18:27:05.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:05.445+0000 7f6c91841640 1 -- 172.21.15.114:0/302691400 wait complete. 2024-09-13T18:27:06.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:06 smithi134 ceph-mon[34283]: pgmap v141: 129 pgs: 129 active+clean; 16 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 55 MiB/s wr, 885 op/s 2024-09-13T18:27:06.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:06 smithi204 ceph-mon[34777]: pgmap v141: 129 pgs: 129 active+clean; 16 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 55 MiB/s wr, 885 op/s 2024-09-13T18:27:06.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:06 smithi114 ceph-mon[30897]: pgmap v141: 129 pgs: 129 active+clean; 16 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 55 MiB/s wr, 885 op/s 2024-09-13T18:27:06.480 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-13T18:27:06.481 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-09-13T18:27:06.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.728+0000 7f963359e640 1 -- 172.21.15.114:0/868153706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f962c099000 msgr2=0x7f962c0a0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.728+0000 7f963359e640 1 --2- 172.21.15.114:0/868153706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f962c099000 0x7f962c0a0b50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f962801a910 tx=0x7f962803b430 comp rx=0 tx=0).stop 2024-09-13T18:27:06.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.729+0000 7f963359e640 1 -- 172.21.15.114:0/868153706 shutdown_connections 2024-09-13T18:27:06.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.729+0000 7f963359e640 1 --2- 172.21.15.114:0/868153706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f962c099000 0x7f962c0a0b50 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.729+0000 7f963359e640 1 --2- 172.21.15.114:0/868153706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f962c094c70 0x7f962c0988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.729+0000 7f963359e640 1 --2- 172.21.15.114:0/868153706 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f962c0942c0 0x7f962c0946a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.729+0000 7f963359e640 1 -- 172.21.15.114:0/868153706 >> 172.21.15.114:0/868153706 conn(0x7f962c009990 msgr2=0x7f962c009da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:06.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.734+0000 7f963359e640 1 -- 172.21.15.114:0/868153706 shutdown_connections 2024-09-13T18:27:06.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.735+0000 7f963359e640 1 -- 172.21.15.114:0/868153706 wait complete. 2024-09-13T18:27:06.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.736+0000 7f963359e640 1 Processor -- start 2024-09-13T18:27:06.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.736+0000 7f963359e640 1 -- start start 2024-09-13T18:27:06.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.738+0000 7f963359e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f962c0942c0 0x7f962c09d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.738+0000 7f963359e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f962c099000 0x7f962c09d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.738+0000 7f963359e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f962c09da90 0x7f962c13de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.738+0000 7f963359e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962c13e4c0 con 0x7f962c09da90 2024-09-13T18:27:06.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.738+0000 7f963359e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962c13e630 con 0x7f962c099000 2024-09-13T18:27:06.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.738+0000 7f963359e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962c13e7f0 con 0x7f962c0942c0 2024-09-13T18:27:06.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.738+0000 7f963259c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f962c0942c0 0x7f962c09d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.738+0000 7f963259c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f962c0942c0 0x7f962c09d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:60182/0 (socket says 172.21.15.114:60182) 2024-09-13T18:27:06.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.738+0000 7f963259c640 1 -- 172.21.15.114:0/1962956742 learned_addr learned my addr 172.21.15.114:0/1962956742 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:06.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.738+0000 7f9631d9b640 1 --2- 172.21.15.114:0/1962956742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f962c099000 0x7f962c09d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.739+0000 7f9631d9b640 1 -- 172.21.15.114:0/1962956742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f962c0942c0 msgr2=0x7f962c09d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.739+0000 7f9631d9b640 1 --2- 172.21.15.114:0/1962956742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f962c0942c0 0x7f962c09d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.739+0000 7f9631d9b640 1 -- 172.21.15.114:0/1962956742 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f962c09da90 msgr2=0x7f962c13de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.739+0000 7f9631d9b640 1 --2- 172.21.15.114:0/1962956742 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f962c09da90 0x7f962c13de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.739+0000 7f9631d9b640 1 -- 172.21.15.114:0/1962956742 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f962801a570 con 0x7f962c099000 2024-09-13T18:27:06.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.739+0000 7f9631d9b640 1 --2- 172.21.15.114:0/1962956742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f962c099000 0x7f962c09d550 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f96340660a0 tx=0x7f9634071e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:06.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.740+0000 7f96237fe640 1 -- 172.21.15.114:0/1962956742 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963406fcf0 con 0x7f962c099000 2024-09-13T18:27:06.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.741+0000 7f963359e640 1 -- 172.21.15.114:0/1962956742 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f962c13ead0 con 0x7f962c099000 2024-09-13T18:27:06.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.741+0000 7f963359e640 1 -- 172.21.15.114:0/1962956742 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f962c13ef60 con 0x7f962c099000 2024-09-13T18:27:06.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.741+0000 7f96237fe640 1 -- 172.21.15.114:0/1962956742 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f963406fe50 con 0x7f962c099000 2024-09-13T18:27:06.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.741+0000 7f96237fe640 1 -- 172.21.15.114:0/1962956742 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9634072620 con 0x7f962c099000 2024-09-13T18:27:06.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.744+0000 7f963bbfc740 1 Processor -- start 2024-09-13T18:27:06.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.745+0000 7f963bbfc740 1 -- start start 2024-09-13T18:27:06.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.745+0000 7f963bbfc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5632d515e9a0 0x5632d515ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.745+0000 7f963bbfc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632d515f2c0 0x5632d5169620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.745+0000 7f963bbfc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5632d5169b60 0x5632d516bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.745+0000 7f963bbfc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632d50c9c60 con 0x5632d515e9a0 2024-09-13T18:27:06.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.745+0000 7f963bbfc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632d50c73f0 con 0x5632d515f2c0 2024-09-13T18:27:06.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.745+0000 7f963bbfc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632d50c2ce0 con 0x5632d5169b60 2024-09-13T18:27:06.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.746+0000 7f9632d9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5632d515e9a0 0x5632d515ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.746+0000 7f963259c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632d515f2c0 0x5632d5169620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.746+0000 7f9632d9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5632d515e9a0 0x5632d515ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60062/0 (socket says 172.21.15.114:60062) 2024-09-13T18:27:06.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.746+0000 7f9632d9d640 1 -- 172.21.15.114:0/3787302764 learned_addr learned my addr 172.21.15.114:0/3787302764 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:06.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.746+0000 7f9632d9d640 1 -- 172.21.15.114:0/3787302764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5632d5169b60 msgr2=0x5632d516bf50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:27:06.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.746+0000 7f9632d9d640 1 --2- 172.21.15.114:0/3787302764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5632d5169b60 0x5632d516bf50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.746+0000 7f9632d9d640 1 -- 172.21.15.114:0/3787302764 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632d515f2c0 msgr2=0x5632d5169620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:06.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.746+0000 7f9632d9d640 1 --2- 172.21.15.114:0/3787302764 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632d515f2c0 0x5632d5169620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.746+0000 7f9632d9d640 1 -- 172.21.15.114:0/3787302764 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632d507e970 con 0x5632d515e9a0 2024-09-13T18:27:06.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.747+0000 7f9632d9d640 1 --2- 172.21.15.114:0/3787302764 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5632d515e9a0 0x5632d515ed80 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f962803ba20 tx=0x7f9628000cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64ad6d0858ee4bd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:06.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.750+0000 7f96237fe640 1 -- 172.21.15.114:0/1962956742 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f9634072780 con 0x7f962c099000 2024-09-13T18:27:06.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.751+0000 7f96237fe640 1 --2- 172.21.15.114:0/1962956742 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f961c0a1410 0x7f961c0a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.751+0000 7f96237fe640 1 -- 172.21.15.114:0/1962956742 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f963412f550 con 0x7f962c099000 2024-09-13T18:27:06.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.752+0000 7f9620ff9640 1 -- 172.21.15.114:0/3787302764 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9628004990 con 0x5632d515e9a0 2024-09-13T18:27:06.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.752+0000 7f9620ff9640 1 -- 172.21.15.114:0/3787302764 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f96280044c0 con 0x5632d515e9a0 2024-09-13T18:27:06.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.753+0000 7f963bbfc740 1 -- 172.21.15.114:0/3787302764 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5632d515e9a0 msgr2=0x5632d515ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.753+0000 7f963bbfc740 1 --2- 172.21.15.114:0/3787302764 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5632d515e9a0 0x5632d515ed80 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f962803ba20 tx=0x7f9628000cc0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.753+0000 7f963259c640 1 --2- 172.21.15.114:0/1962956742 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f961c0a1410 0x7f961c0a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.754+0000 7f963259c640 1 --2- 172.21.15.114:0/1962956742 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f961c0a1410 0x7f961c0a38d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x5632d515aa30 tx=0x7f96240091c0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:06.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.755+0000 7f963bbfc740 1 -- 172.21.15.114:0/3787302764 shutdown_connections 2024-09-13T18:27:06.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.755+0000 7f963bbfc740 1 --2- 172.21.15.114:0/3787302764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5632d5169b60 0x5632d516bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.755+0000 7f963bbfc740 1 --2- 172.21.15.114:0/3787302764 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632d515f2c0 0x5632d5169620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.755+0000 7f963bbfc740 1 --2- 172.21.15.114:0/3787302764 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5632d515e9a0 0x5632d515ed80 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.755+0000 7f963bbfc740 1 -- 172.21.15.114:0/3787302764 >> 172.21.15.114:0/3787302764 conn(0x5632d515af30 msgr2=0x5632d515b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:06.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.756+0000 7f963bbfc740 1 -- 172.21.15.114:0/3787302764 shutdown_connections 2024-09-13T18:27:06.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.756+0000 7f963bbfc740 1 -- 172.21.15.114:0/3787302764 wait complete. 2024-09-13T18:27:06.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.756+0000 7f963bbfc740 1 -- start start 2024-09-13T18:27:06.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.756+0000 7f963bbfc740 1 Processor -- start 2024-09-13T18:27:06.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.761+0000 7f963bbfc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5632d515e9a0 0x5632d5169440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.761+0000 7f963bbfc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632d515f2c0 0x5632d5166780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.761+0000 7f963bbfc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5632d5169b60 0x5632d5166cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.761+0000 7f963bbfc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632d5080f80 con 0x5632d5169b60 2024-09-13T18:27:06.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.761+0000 7f963bbfc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632d5080c20 con 0x5632d515f2c0 2024-09-13T18:27:06.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.761+0000 7f963bbfc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632d504dab0 con 0x5632d515e9a0 2024-09-13T18:27:06.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.761+0000 7f963bbfc740 10 client.0 _ll_register_callbacks cb 0x5632d50bc350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:27:06.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.762+0000 7f963259c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5632d515e9a0 0x5632d5169440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.762+0000 7f963259c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5632d515e9a0 0x5632d5169440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:60192/0 (socket says 172.21.15.114:60192) 2024-09-13T18:27:06.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.762+0000 7f963259c640 1 -- 172.21.15.114:0/424165198 learned_addr learned my addr 172.21.15.114:0/424165198 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:06.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.762+0000 7f9631d9b640 1 --2- 172.21.15.114:0/424165198 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632d515f2c0 0x5632d5166780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.763+0000 7f9632d9d640 1 --2- 172.21.15.114:0/424165198 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5632d5169b60 0x5632d5166cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.763+0000 7f9632d9d640 1 -- 172.21.15.114:0/424165198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5632d515e9a0 msgr2=0x5632d5169440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:06.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.763+0000 7f9632d9d640 1 --2- 172.21.15.114:0/424165198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5632d515e9a0 0x5632d5169440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.763+0000 7f9632d9d640 1 -- 172.21.15.114:0/424165198 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632d515f2c0 msgr2=0x5632d5166780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:06.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.763+0000 7f9632d9d640 1 --2- 172.21.15.114:0/424165198 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632d515f2c0 0x5632d5166780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.763+0000 7f9632d9d640 1 -- 172.21.15.114:0/424165198 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f962c0a1390 con 0x5632d5169b60 2024-09-13T18:27:06.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.764+0000 7f9632d9d640 1 --2- 172.21.15.114:0/424165198 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5632d5169b60 0x5632d5166cc0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f962801a860 tx=0x7f9628004bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12a4ba4f2ef49c9c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:06.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.764+0000 7f9619ffb640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:27:06.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.764+0000 7f9619ffb640 1 -- 172.21.15.114:0/424165198 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9628059070 con 0x5632d5169b60 2024-09-13T18:27:06.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.764+0000 7f9619ffb640 1 -- 172.21.15.114:0/424165198 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f9628003fa0 con 0x5632d5169b60 2024-09-13T18:27:06.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.764+0000 7f963bbfc740 1 -- 172.21.15.114:0/424165198 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5632d4db6590 con 0x5632d5169b60 2024-09-13T18:27:06.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.764+0000 7f9619ffb640 1 -- 172.21.15.114:0/424165198 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9628004180 con 0x5632d5169b60 2024-09-13T18:27:06.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.767+0000 7f963bbfc740 10 client.17163 fetch_fsmap learned FSMap version 32 2024-09-13T18:27:06.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.767+0000 7f963bbfc740 1 -- 172.21.15.114:0/424165198 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5632d50c1360 con 0x5632d5169b60 2024-09-13T18:27:06.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.769+0000 7f9619ffb640 1 -- 172.21.15.114:0/424165198 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5632d4db6590 con 0x5632d5169b60 2024-09-13T18:27:06.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.769+0000 7f9619ffb640 1 -- 172.21.15.114:0/424165198 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f9628040510 con 0x5632d5169b60 2024-09-13T18:27:06.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.769+0000 7f963bbfc740 10 client.17163 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:27:06.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.769+0000 7f963bbfc740 10 client.17163 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:27:06.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.769+0000 7f963bbfc740 20 client.17163 populate_metadata read hostname 'smithi114' 2024-09-13T18:27:06.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.769+0000 7f963bbfc740 1 --2- 172.21.15.114:0/424165198 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5632d5179440 0x5632d517b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.769+0000 7f963bbfc740 4 client.17163 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:27:06.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.769+0000 7f963bbfc740 1 -- 172.21.15.114:0/424165198 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5632d4dc7860 con 0x5632d5179440 2024-09-13T18:27:06.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.769+0000 7f963259c640 1 --2- 172.21.15.114:0/424165198 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5632d5179440 0x5632d517b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.770+0000 7f9619ffb640 10 client.17163 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:27:06.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.771+0000 7f963259c640 1 --2- 172.21.15.114:0/424165198 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5632d5179440 0x5632d517b830 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12928f70ddc49e5e server_cookie=d4bddd94a7487522 in_seq=0 out_seq=0 2024-09-13T18:27:06.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.773+0000 7f9619ffb640 1 -- 172.21.15.114:0/424165198 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5632d4dc7860 con 0x5632d5179440 2024-09-13T18:27:06.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.773+0000 7f9619ffb640 10 client.17163 handle_command_reply: tid=0 2024-09-13T18:27:06.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.773+0000 7f963bbfc740 1 client.17163 shutdown 2024-09-13T18:27:06.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.776+0000 7f963bbfc740 1 -- 172.21.15.114:0/424165198 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5632d5169b60 msgr2=0x5632d5166cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.776+0000 7f963bbfc740 1 --2- 172.21.15.114:0/424165198 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5632d5169b60 0x5632d5166cc0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f962801a860 tx=0x7f9628004bc0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.776+0000 7f963bbfc740 1 -- 172.21.15.114:0/424165198 shutdown_connections 2024-09-13T18:27:06.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.776+0000 7f963bbfc740 1 --2- 172.21.15.114:0/424165198 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5632d5179440 0x5632d517b830 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.776+0000 7f963bbfc740 1 --2- 172.21.15.114:0/424165198 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5632d5169b60 0x5632d5166cc0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.776+0000 7f963bbfc740 1 --2- 172.21.15.114:0/424165198 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632d515f2c0 0x5632d5166780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.776+0000 7f963bbfc740 1 --2- 172.21.15.114:0/424165198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5632d515e9a0 0x5632d5169440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.776+0000 7f963bbfc740 1 -- 172.21.15.114:0/424165198 >> 172.21.15.114:0/424165198 conn(0x5632d515af30 msgr2=0x5632d515a160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:06.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.778+0000 7f9619ffb640 0 client.17163 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:27:06.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.780+0000 7f963bbfc740 1 -- 172.21.15.114:0/424165198 shutdown_connections 2024-09-13T18:27:06.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.784+0000 7f963bbfc740 1 -- 172.21.15.114:0/424165198 wait complete. 2024-09-13T18:27:06.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.790+0000 7f963bbfc740 20 client.17163 trim_cache size 0 max 16384 2024-09-13T18:27:06.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.842+0000 7f963bbfc740 1 Processor -- start 2024-09-13T18:27:06.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.843+0000 7f963bbfc740 1 -- start start 2024-09-13T18:27:06.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.843+0000 7f963bbfc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5632d515e9a0 0x5632d515a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.843+0000 7f963bbfc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632d515f2c0 0x5632d5168f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.843+0000 7f963bbfc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5632d5169b60 0x5632d516af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.843+0000 7f963bbfc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632d515fc80 con 0x5632d515e9a0 2024-09-13T18:27:06.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.843+0000 7f963bbfc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632d4db6590 con 0x5632d515f2c0 2024-09-13T18:27:06.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.843+0000 7f963bbfc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632d5176600 con 0x5632d5169b60 2024-09-13T18:27:06.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.844+0000 7f9632d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5632d5169b60 0x5632d516af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.844+0000 7f9632d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5632d5169b60 0x5632d516af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:60208/0 (socket says 172.21.15.114:60208) 2024-09-13T18:27:06.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.844+0000 7f9632d9d640 1 -- 172.21.15.114:0/314700962 learned_addr learned my addr 172.21.15.114:0/314700962 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:06.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.844+0000 7f9632d9d640 1 -- 172.21.15.114:0/314700962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632d515f2c0 msgr2=0x5632d5168f50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:27:06.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.844+0000 7f9632d9d640 1 --2- 172.21.15.114:0/314700962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632d515f2c0 0x5632d5168f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.844+0000 7f9632d9d640 1 -- 172.21.15.114:0/314700962 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5632d515e9a0 msgr2=0x5632d515a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:06.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.844+0000 7f9632d9d640 1 --2- 172.21.15.114:0/314700962 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5632d515e9a0 0x5632d515a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.844+0000 7f9632d9d640 1 -- 172.21.15.114:0/314700962 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632d50631c0 con 0x5632d5169b60 2024-09-13T18:27:06.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.845+0000 7f9632d9d640 1 --2- 172.21.15.114:0/314700962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5632d5169b60 0x5632d516af50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9628004650 tx=0x7f9628000cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b4b49728a59824e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:06.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.845+0000 7f96217fa640 1 -- 172.21.15.114:0/314700962 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9628059070 con 0x5632d5169b60 2024-09-13T18:27:06.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.845+0000 7f96217fa640 1 -- 172.21.15.114:0/314700962 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f96280038c0 con 0x5632d5169b60 2024-09-13T18:27:06.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.845+0000 7f96217fa640 1 -- 172.21.15.114:0/314700962 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9628040eb0 con 0x5632d5169b60 2024-09-13T18:27:06.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.846+0000 7f963bbfc740 1 -- 172.21.15.114:0/314700962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5632d5169b60 msgr2=0x5632d516af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.846+0000 7f963bbfc740 1 --2- 172.21.15.114:0/314700962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5632d5169b60 0x5632d516af50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9628004650 tx=0x7f9628000cc0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.847+0000 7f963bbfc740 1 -- 172.21.15.114:0/314700962 shutdown_connections 2024-09-13T18:27:06.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.847+0000 7f963bbfc740 1 --2- 172.21.15.114:0/314700962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5632d5169b60 0x5632d516af50 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.847+0000 7f963bbfc740 1 --2- 172.21.15.114:0/314700962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632d515f2c0 0x5632d5168f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.847+0000 7f963bbfc740 1 --2- 172.21.15.114:0/314700962 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5632d515e9a0 0x5632d515a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.847+0000 7f963bbfc740 1 -- 172.21.15.114:0/314700962 >> 172.21.15.114:0/314700962 conn(0x5632d515af30 msgr2=0x5632d5159940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:06.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.852+0000 7f963bbfc740 1 -- 172.21.15.114:0/314700962 shutdown_connections 2024-09-13T18:27:06.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.856+0000 7f963bbfc740 1 -- 172.21.15.114:0/314700962 wait complete. 2024-09-13T18:27:06.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.856+0000 7f963bbfc740 1 -- start start 2024-09-13T18:27:06.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.856+0000 7f963bbfc740 1 Processor -- start 2024-09-13T18:27:06.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.857+0000 7f963bbfc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5632d515f2c0 0x5632d5163b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.857+0000 7f963bbfc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5632d5169b60 0x5632d5164040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.857+0000 7f963bbfc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632d51cb8f0 0x5632d51cbd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.857+0000 7f963bbfc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632d51646e0 con 0x5632d515f2c0 2024-09-13T18:27:06.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.857+0000 7f963bbfc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632d51cc290 con 0x5632d51cb8f0 2024-09-13T18:27:06.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.857+0000 7f963bbfc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632d51cc450 con 0x5632d5169b60 2024-09-13T18:27:06.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.857+0000 7f963bbfc740 10 client.0 _ll_register_callbacks cb 0x7f9624009470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:27:06.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.859+0000 7f963259c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5632d515f2c0 0x5632d5163b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.859+0000 7f963259c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5632d515f2c0 0x5632d5163b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60104/0 (socket says 172.21.15.114:60104) 2024-09-13T18:27:06.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.859+0000 7f963259c640 1 -- 172.21.15.114:0/19478320 learned_addr learned my addr 172.21.15.114:0/19478320 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:06.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.859+0000 7f9631d9b640 1 --2- 172.21.15.114:0/19478320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5632d5169b60 0x5632d5164040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.859+0000 7f9632d9d640 1 --2- 172.21.15.114:0/19478320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632d51cb8f0 0x5632d51cbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.860+0000 7f963259c640 1 -- 172.21.15.114:0/19478320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5632d5169b60 msgr2=0x5632d5164040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:06.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.860+0000 7f963259c640 1 --2- 172.21.15.114:0/19478320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5632d5169b60 0x5632d5164040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.860+0000 7f963259c640 1 -- 172.21.15.114:0/19478320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632d51cb8f0 msgr2=0x5632d51cbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:06.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.860+0000 7f963259c640 1 --2- 172.21.15.114:0/19478320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632d51cb8f0 0x5632d51cbd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.860+0000 7f963259c640 1 -- 172.21.15.114:0/19478320 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5632d50c1360 con 0x5632d515f2c0 2024-09-13T18:27:06.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.860+0000 7f963259c640 1 --2- 172.21.15.114:0/19478320 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5632d515f2c0 0x5632d5163b00 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f9624011750 tx=0x7f9624014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d298841e6bb534d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:06.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.861+0000 7f9619ffb640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:27:06.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.861+0000 7f9619ffb640 1 -- 172.21.15.114:0/19478320 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96240248b0 con 0x5632d515f2c0 2024-09-13T18:27:06.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.861+0000 7f963bbfc740 1 -- 172.21.15.114:0/19478320 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f96280044c0 con 0x5632d515f2c0 2024-09-13T18:27:06.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.862+0000 7f9619ffb640 1 -- 172.21.15.114:0/19478320 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f962401c040 con 0x5632d515f2c0 2024-09-13T18:27:06.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.863+0000 7f9619ffb640 1 -- 172.21.15.114:0/19478320 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9624024a10 con 0x5632d515f2c0 2024-09-13T18:27:06.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.864+0000 7f963bbfc740 10 client.17166 fetch_fsmap learned FSMap version 32 2024-09-13T18:27:06.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.865+0000 7f963bbfc740 1 -- 172.21.15.114:0/19478320 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5632d5088990 con 0x5632d515f2c0 2024-09-13T18:27:06.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.866+0000 7f9619ffb640 1 -- 172.21.15.114:0/19478320 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f96280044c0 con 0x5632d515f2c0 2024-09-13T18:27:06.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.866+0000 7f9619ffb640 1 -- 172.21.15.114:0/19478320 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f9624029560 con 0x5632d515f2c0 2024-09-13T18:27:06.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.866+0000 7f963bbfc740 10 client.17166 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:27:06.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.866+0000 7f963bbfc740 10 client.17166 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:27:06.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.866+0000 7f963bbfc740 20 client.17166 populate_metadata read hostname 'smithi114' 2024-09-13T18:27:06.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.866+0000 7f963bbfc740 1 --2- 172.21.15.114:0/19478320 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5632d51e2d30 0x5632d51e5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:06.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.866+0000 7f963bbfc740 4 client.17166 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-13T18:27:06.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.866+0000 7f963bbfc740 1 -- 172.21.15.114:0/19478320 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5632d4e1aeb0 con 0x5632d51e2d30 2024-09-13T18:27:06.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.867+0000 7f9631d9b640 1 --2- 172.21.15.114:0/19478320 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5632d51e2d30 0x5632d51e5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:06.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.882+0000 7f9631d9b640 1 --2- 172.21.15.114:0/19478320 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5632d51e2d30 0x5632d51e5150 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90d0e05e3f3396e8 server_cookie=a38c41d68c3e8074 in_seq=0 out_seq=0 2024-09-13T18:27:06.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.883+0000 7f9619ffb640 10 client.17166 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:27:06.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.883+0000 7f9619ffb640 1 -- 172.21.15.114:0/19478320 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5632d4e1aeb0 con 0x5632d51e2d30 2024-09-13T18:27:06.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.883+0000 7f9619ffb640 10 client.17166 handle_command_reply: tid=0 2024-09-13T18:27:06.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.885+0000 7f963bbfc740 1 client.17166 shutdown 2024-09-13T18:27:06.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.885+0000 7f963bbfc740 1 -- 172.21.15.114:0/19478320 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5632d515f2c0 msgr2=0x5632d5163b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.885+0000 7f963bbfc740 1 --2- 172.21.15.114:0/19478320 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5632d515f2c0 0x5632d5163b00 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f9624011750 tx=0x7f9624014000 comp rx=0 tx=0).stop 2024-09-13T18:27:06.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.888+0000 7f963bbfc740 1 -- 172.21.15.114:0/19478320 shutdown_connections 2024-09-13T18:27:06.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.888+0000 7f963bbfc740 1 --2- 172.21.15.114:0/19478320 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5632d51e2d30 0x5632d51e5150 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.888+0000 7f963bbfc740 1 --2- 172.21.15.114:0/19478320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632d51cb8f0 0x5632d51cbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.888+0000 7f963bbfc740 1 --2- 172.21.15.114:0/19478320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5632d5169b60 0x5632d5164040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.888+0000 7f963bbfc740 1 --2- 172.21.15.114:0/19478320 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5632d515f2c0 0x5632d5163b00 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.888+0000 7f963bbfc740 1 -- 172.21.15.114:0/19478320 >> 172.21.15.114:0/19478320 conn(0x5632d515af30 msgr2=0x5632d5159940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:06.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.890+0000 7f9619ffb640 0 client.17166 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:27:06.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.896+0000 7f963bbfc740 1 -- 172.21.15.114:0/19478320 shutdown_connections 2024-09-13T18:27:06.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.897+0000 7f963bbfc740 1 -- 172.21.15.114:0/19478320 wait complete. 2024-09-13T18:27:06.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.901+0000 7f963bbfc740 20 client.17166 trim_cache size 0 max 16384 2024-09-13T18:27:06.909 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:27:06.909 INFO:teuthology.orchestra.run.smithi114.stdout: "return_code": 0, 2024-09-13T18:27:06.909 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_tag": "4944d28d-7f7f-46be-bf12-f95da35e05b4", 2024-09-13T18:27:06.909 INFO:teuthology.orchestra.run.smithi114.stdout: "mode": "asynchronous" 2024-09-13T18:27:06.909 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:27:06.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.910+0000 7f963359e640 1 -- 172.21.15.114:0/1962956742 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f961c0a1410 msgr2=0x7f961c0a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.910+0000 7f963359e640 1 --2- 172.21.15.114:0/1962956742 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f961c0a1410 0x7f961c0a38d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x5632d515aa30 tx=0x7f96240091c0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.910+0000 7f963359e640 1 -- 172.21.15.114:0/1962956742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f962c099000 msgr2=0x7f962c09d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:06.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.910+0000 7f963359e640 1 --2- 172.21.15.114:0/1962956742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f962c099000 0x7f962c09d550 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f96340660a0 tx=0x7f9634071e70 comp rx=0 tx=0).stop 2024-09-13T18:27:06.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.913+0000 7f963359e640 1 -- 172.21.15.114:0/1962956742 shutdown_connections 2024-09-13T18:27:06.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.913+0000 7f963359e640 1 --2- 172.21.15.114:0/1962956742 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f961c0a1410 0x7f961c0a38d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.913+0000 7f963359e640 1 --2- 172.21.15.114:0/1962956742 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f962c09da90 0x7f962c13de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.913+0000 7f963359e640 1 --2- 172.21.15.114:0/1962956742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f962c099000 0x7f962c09d550 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.913+0000 7f963359e640 1 --2- 172.21.15.114:0/1962956742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f962c0942c0 0x7f962c09d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:06.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.913+0000 7f963359e640 1 -- 172.21.15.114:0/1962956742 >> 172.21.15.114:0/1962956742 conn(0x7f962c009990 msgr2=0x7f962c091f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:06.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.917+0000 7f963359e640 1 -- 172.21.15.114:0/1962956742 shutdown_connections 2024-09-13T18:27:06.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:06.917+0000 7f963359e640 1 -- 172.21.15.114:0/1962956742 wait complete. 2024-09-13T18:27:06.955 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T18:27:07.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.194+0000 7fa83b24f640 1 -- 172.21.15.114:0/1987946703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8341074b0 msgr2=0x7fa83410e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:07.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.194+0000 7fa83b24f640 1 --2- 172.21.15.114:0/1987946703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8341074b0 0x7fa83410e510 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa824009860 tx=0x7fa82402a380 comp rx=0 tx=0).stop 2024-09-13T18:27:07.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.200+0000 7fa83b24f640 1 -- 172.21.15.114:0/1987946703 shutdown_connections 2024-09-13T18:27:07.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.200+0000 7fa83b24f640 1 --2- 172.21.15.114:0/1987946703 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa834119400 0x7fa83411b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.200+0000 7fa83b24f640 1 --2- 172.21.15.114:0/1987946703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa834107890 0x7fa834107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.200+0000 7fa83b24f640 1 --2- 172.21.15.114:0/1987946703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8341074b0 0x7fa83410e510 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.200+0000 7fa83b24f640 1 -- 172.21.15.114:0/1987946703 >> 172.21.15.114:0/1987946703 conn(0x7fa83406af40 msgr2=0x7fa83406b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:07.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.208+0000 7fa83b24f640 1 -- 172.21.15.114:0/1987946703 shutdown_connections 2024-09-13T18:27:07.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.211+0000 7fa83b24f640 1 -- 172.21.15.114:0/1987946703 wait complete. 2024-09-13T18:27:07.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.211+0000 7fa83b24f640 1 Processor -- start 2024-09-13T18:27:07.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.212+0000 7fa83b24f640 1 -- start start 2024-09-13T18:27:07.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.212+0000 7fa83b24f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa8341074b0 0x7fa8341187f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:07.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.212+0000 7fa83b24f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa834107890 0x7fa834118d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:07.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.212+0000 7fa83b24f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa834119400 0x7fa8341128c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:07.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.212+0000 7fa83b24f640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa834113050 con 0x7fa8341074b0 2024-09-13T18:27:07.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.212+0000 7fa83b24f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8341131c0 con 0x7fa834119400 2024-09-13T18:27:07.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.212+0000 7fa83b24f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa834113380 con 0x7fa834107890 2024-09-13T18:27:07.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.214+0000 7fa83a24d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa8341074b0 0x7fa8341187f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:07.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.214+0000 7fa83a24d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa8341074b0 0x7fa8341187f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60110/0 (socket says 172.21.15.114:60110) 2024-09-13T18:27:07.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.214+0000 7fa83a24d640 1 -- 172.21.15.114:0/3683386470 learned_addr learned my addr 172.21.15.114:0/3683386470 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:07.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.214+0000 7fa839a4c640 1 --2- 172.21.15.114:0/3683386470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa834107890 0x7fa834118d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:07.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.214+0000 7fa83aa4e640 1 --2- 172.21.15.114:0/3683386470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa834119400 0x7fa8341128c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:07.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.215+0000 7fa83a24d640 1 -- 172.21.15.114:0/3683386470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa834107890 msgr2=0x7fa834118d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:07.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.215+0000 7fa83a24d640 1 --2- 172.21.15.114:0/3683386470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa834107890 0x7fa834118d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.215+0000 7fa83a24d640 1 -- 172.21.15.114:0/3683386470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa834119400 msgr2=0x7fa8341128c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:07.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.215+0000 7fa83a24d640 1 --2- 172.21.15.114:0/3683386470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa834119400 0x7fa8341128c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.215+0000 7fa83a24d640 1 -- 172.21.15.114:0/3683386470 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa824009490 con 0x7fa8341074b0 2024-09-13T18:27:07.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.216+0000 7fa83a24d640 1 --2- 172.21.15.114:0/3683386470 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa8341074b0 0x7fa8341187f0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fa82402a8f0 tx=0x7fa824032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:07.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.217+0000 7fa82b7fe640 1 -- 172.21.15.114:0/3683386470 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa824030710 con 0x7fa8341074b0 2024-09-13T18:27:07.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.217+0000 7fa83b24f640 1 -- 172.21.15.114:0/3683386470 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa834113600 con 0x7fa8341074b0 2024-09-13T18:27:07.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.217+0000 7fa83b24f640 1 -- 172.21.15.114:0/3683386470 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8341ba3f0 con 0x7fa8341074b0 2024-09-13T18:27:07.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.217+0000 7fa82b7fe640 1 -- 172.21.15.114:0/3683386470 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa8240044d0 con 0x7fa8341074b0 2024-09-13T18:27:07.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.217+0000 7fa82b7fe640 1 -- 172.21.15.114:0/3683386470 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa824004920 con 0x7fa8341074b0 2024-09-13T18:27:07.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.220+0000 7fa83f1d7740 1 Processor -- start 2024-09-13T18:27:07.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.221+0000 7fa83f1d7740 1 -- start start 2024-09-13T18:27:07.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.221+0000 7fa83f1d7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564eee230fe0 0x564eee2313c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:07.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.221+0000 7fa83f1d7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564eee231900 0x564eee23bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:07.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.221+0000 7fa83f1d7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564eee23c1a0 0x564eee23e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:07.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.221+0000 7fa83f1d7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eee19bf20 con 0x564eee230fe0 2024-09-13T18:27:07.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.221+0000 7fa83f1d7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eee1996b0 con 0x564eee23c1a0 2024-09-13T18:27:07.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.221+0000 7fa83f1d7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eee194fa0 con 0x564eee231900 2024-09-13T18:27:07.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.222+0000 7fa839a4c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564eee23c1a0 0x564eee23e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:07.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.222+0000 7fa839a4c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564eee23c1a0 0x564eee23e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:36234/0 (socket says 172.21.15.114:36234) 2024-09-13T18:27:07.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.222+0000 7fa839a4c640 1 -- 172.21.15.114:0/403993412 learned_addr learned my addr 172.21.15.114:0/403993412 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:07.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.222+0000 7fa83aa4e640 1 --2- 172.21.15.114:0/403993412 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564eee230fe0 0x564eee2313c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:07.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.223+0000 7fa839a4c640 1 -- 172.21.15.114:0/403993412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564eee231900 msgr2=0x564eee23bc60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:27:07.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.223+0000 7fa839a4c640 1 --2- 172.21.15.114:0/403993412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564eee231900 0x564eee23bc60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.223+0000 7fa839a4c640 1 -- 172.21.15.114:0/403993412 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564eee230fe0 msgr2=0x564eee2313c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:07.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.223+0000 7fa839a4c640 1 --2- 172.21.15.114:0/403993412 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564eee230fe0 0x564eee2313c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.223+0000 7fa839a4c640 1 -- 172.21.15.114:0/403993412 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564eee14f840 con 0x564eee23c1a0 2024-09-13T18:27:07.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.223+0000 7fa839a4c640 1 --2- 172.21.15.114:0/403993412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564eee23c1a0 0x564eee23e590 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa82c003a20 tx=0x7fa82c009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50bb4fbb2ff31676 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:07.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.223+0000 7fa82b7fe640 1 -- 172.21.15.114:0/3683386470 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fa824005050 con 0x7fa8341074b0 2024-09-13T18:27:07.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.224+0000 7fa828ff9640 1 -- 172.21.15.114:0/403993412 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82c004b20 con 0x564eee23c1a0 2024-09-13T18:27:07.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.224+0000 7fa828ff9640 1 -- 172.21.15.114:0/403993412 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa82c004c80 con 0x564eee23c1a0 2024-09-13T18:27:07.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.224+0000 7fa828ff9640 1 -- 172.21.15.114:0/403993412 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82c0126b0 con 0x564eee23c1a0 2024-09-13T18:27:07.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.225+0000 7fa82b7fe640 1 --2- 172.21.15.114:0/3683386470 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa8200a1410 0x7fa8200a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:07.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.225+0000 7fa82b7fe640 1 -- 172.21.15.114:0/3683386470 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fa8240eda30 con 0x7fa8341074b0 2024-09-13T18:27:07.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.226+0000 7fa83f1d7740 1 -- 172.21.15.114:0/403993412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564eee23c1a0 msgr2=0x564eee23e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:07.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.226+0000 7fa83f1d7740 1 --2- 172.21.15.114:0/403993412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564eee23c1a0 0x564eee23e590 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa82c003a20 tx=0x7fa82c009f90 comp rx=0 tx=0).stop 2024-09-13T18:27:07.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.227+0000 7fa83f1d7740 1 -- 172.21.15.114:0/403993412 shutdown_connections 2024-09-13T18:27:07.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.227+0000 7fa83f1d7740 1 --2- 172.21.15.114:0/403993412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564eee23c1a0 0x564eee23e590 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.227+0000 7fa83f1d7740 1 --2- 172.21.15.114:0/403993412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564eee231900 0x564eee23bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.227+0000 7fa83f1d7740 1 --2- 172.21.15.114:0/403993412 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564eee230fe0 0x564eee2313c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.227+0000 7fa83f1d7740 1 -- 172.21.15.114:0/403993412 >> 172.21.15.114:0/403993412 conn(0x564eee22d570 msgr2=0x564eee22d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:07.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.229+0000 7fa83f1d7740 1 -- 172.21.15.114:0/403993412 shutdown_connections 2024-09-13T18:27:07.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.229+0000 7fa83f1d7740 1 -- 172.21.15.114:0/403993412 wait complete. 2024-09-13T18:27:07.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.229+0000 7fa83f1d7740 1 -- start start 2024-09-13T18:27:07.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.229+0000 7fa83f1d7740 1 Processor -- start 2024-09-13T18:27:07.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.230+0000 7fa83f1d7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564eee230fe0 0x564eee23bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:07.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.230+0000 7fa83f1d7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564eee231900 0x564eee238e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:07.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.230+0000 7fa83f1d7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564eee23c1a0 0x564eee239430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:07.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.230+0000 7fa83f1d7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eee1362a0 con 0x564eee23c1a0 2024-09-13T18:27:07.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.230+0000 7fa83f1d7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eede6d430 con 0x564eee231900 2024-09-13T18:27:07.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.230+0000 7fa83f1d7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eee118680 con 0x564eee230fe0 2024-09-13T18:27:07.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.230+0000 7fa83f1d7740 10 client.0 _ll_register_callbacks cb 0x564eee18e610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:27:07.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.230+0000 7fa83aa4e640 1 --2- 172.21.15.114:0/3683386470 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa8200a1410 0x7fa8200a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:07.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.231+0000 7fa83aa4e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564eee23c1a0 0x564eee239430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:07.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.232+0000 7fa83aa4e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564eee23c1a0 0x564eee239430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60126/0 (socket says 172.21.15.114:60126) 2024-09-13T18:27:07.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.232+0000 7fa83aa4e640 1 -- 172.21.15.114:0/2039451366 learned_addr learned my addr 172.21.15.114:0/2039451366 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:07.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.233+0000 7fa83a24d640 1 --2- 172.21.15.114:0/2039451366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564eee230fe0 0x564eee23bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:07.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.234+0000 7fa839a4c640 1 --2- 172.21.15.114:0/2039451366 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564eee231900 0x564eee238e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:07.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.235+0000 7fa83aa4e640 1 -- 172.21.15.114:0/2039451366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564eee230fe0 msgr2=0x564eee23bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:07.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.235+0000 7fa83aa4e640 1 --2- 172.21.15.114:0/2039451366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564eee230fe0 0x564eee23bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.235+0000 7fa83aa4e640 1 -- 172.21.15.114:0/2039451366 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564eee231900 msgr2=0x564eee238e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:07.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.235+0000 7fa83aa4e640 1 --2- 172.21.15.114:0/2039451366 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564eee231900 0x564eee238e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.235+0000 7fa83aa4e640 1 -- 172.21.15.114:0/2039451366 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa82c009c70 con 0x564eee23c1a0 2024-09-13T18:27:07.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.237+0000 7fa83aa4e640 1 --2- 172.21.15.114:0/2039451366 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564eee23c1a0 0x564eee239430 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fa830009670 tx=0x7fa830009a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42bba26d4e5c058e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:07.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.237+0000 7fa81dffb640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:27:07.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.237+0000 7fa81dffb640 1 -- 172.21.15.114:0/2039451366 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83000ed80 con 0x564eee23c1a0 2024-09-13T18:27:07.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.238+0000 7fa83f1d7740 1 -- 172.21.15.114:0/2039451366 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564eede884e0 con 0x564eee23c1a0 2024-09-13T18:27:07.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.238+0000 7fa81dffb640 1 -- 172.21.15.114:0/2039451366 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fa830023070 con 0x564eee23c1a0 2024-09-13T18:27:07.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.238+0000 7fa83aa4e640 1 --2- 172.21.15.114:0/3683386470 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa8200a1410 0x7fa8200a38d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa8341193d0 tx=0x7fa8300098c0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:07.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.238+0000 7fa81dffb640 1 -- 172.21.15.114:0/2039451366 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa83000eee0 con 0x564eee23c1a0 2024-09-13T18:27:07.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.240+0000 7fa83f1d7740 10 client.17178 fetch_fsmap learned FSMap version 32 2024-09-13T18:27:07.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.240+0000 7fa83f1d7740 1 -- 172.21.15.114:0/2039451366 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x564eee193620 con 0x564eee23c1a0 2024-09-13T18:27:07.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.241+0000 7fa81dffb640 1 -- 172.21.15.114:0/2039451366 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564eede884e0 con 0x564eee23c1a0 2024-09-13T18:27:07.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.241+0000 7fa81dffb640 1 -- 172.21.15.114:0/2039451366 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fa830004d10 con 0x564eee23c1a0 2024-09-13T18:27:07.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.241+0000 7fa83f1d7740 10 client.17178 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:27:07.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.241+0000 7fa83f1d7740 10 client.17178 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:27:07.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.241+0000 7fa83f1d7740 20 client.17178 populate_metadata read hostname 'smithi114' 2024-09-13T18:27:07.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.241+0000 7fa83f1d7740 1 --2- 172.21.15.114:0/2039451366 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x564eee24c240 0x564eee24e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:07.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.241+0000 7fa83f1d7740 4 client.17178 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:27:07.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.241+0000 7fa83f1d7740 1 -- 172.21.15.114:0/2039451366 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564eede997e0 con 0x564eee24c240 2024-09-13T18:27:07.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.242+0000 7fa83a24d640 1 --2- 172.21.15.114:0/2039451366 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x564eee24c240 0x564eee24e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:07.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.243+0000 7fa83a24d640 1 --2- 172.21.15.114:0/2039451366 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x564eee24c240 0x564eee24e660 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6331260952ad4797 server_cookie=1ce800fa1390d737 in_seq=0 out_seq=0 2024-09-13T18:27:07.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.243+0000 7fa81dffb640 10 client.17178 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:27:07.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.245+0000 7fa81dffb640 1 -- 172.21.15.114:0/2039451366 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x564eede997e0 con 0x564eee24c240 2024-09-13T18:27:07.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.245+0000 7fa81dffb640 10 client.17178 handle_command_reply: tid=0 2024-09-13T18:27:07.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.245+0000 7fa83f1d7740 1 client.17178 shutdown 2024-09-13T18:27:07.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.245+0000 7fa83f1d7740 1 -- 172.21.15.114:0/2039451366 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564eee23c1a0 msgr2=0x564eee239430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:07.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.245+0000 7fa83f1d7740 1 --2- 172.21.15.114:0/2039451366 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564eee23c1a0 0x564eee239430 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fa830009670 tx=0x7fa830009a70 comp rx=0 tx=0).stop 2024-09-13T18:27:07.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.248+0000 7fa83f1d7740 1 -- 172.21.15.114:0/2039451366 shutdown_connections 2024-09-13T18:27:07.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.248+0000 7fa83f1d7740 1 --2- 172.21.15.114:0/2039451366 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x564eee24c240 0x564eee24e660 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.248+0000 7fa83f1d7740 1 --2- 172.21.15.114:0/2039451366 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564eee23c1a0 0x564eee239430 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.248+0000 7fa83f1d7740 1 --2- 172.21.15.114:0/2039451366 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564eee231900 0x564eee238e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.248+0000 7fa83f1d7740 1 --2- 172.21.15.114:0/2039451366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564eee230fe0 0x564eee23bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.248+0000 7fa83f1d7740 1 -- 172.21.15.114:0/2039451366 >> 172.21.15.114:0/2039451366 conn(0x564eee22d570 msgr2=0x564eee22c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:07.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.248+0000 7fa81dffb640 0 client.17178 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:27:07.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.248+0000 7fa83f1d7740 1 -- 172.21.15.114:0/2039451366 shutdown_connections 2024-09-13T18:27:07.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.248+0000 7fa83f1d7740 1 -- 172.21.15.114:0/2039451366 wait complete. 2024-09-13T18:27:07.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.249+0000 7fa83f1d7740 20 client.17178 trim_cache size 0 max 16384 2024-09-13T18:27:07.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.283+0000 7fa83f1d7740 1 Processor -- start 2024-09-13T18:27:07.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.283+0000 7fa83f1d7740 1 -- start start 2024-09-13T18:27:07.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.284+0000 7fa83f1d7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564eee230fe0 0x564eee22cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:07.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.284+0000 7fa83f1d7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564eee231900 0x564eee242f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:07.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.284+0000 7fa83f1d7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564eee23c1a0 0x564eee2434d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:07.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.284+0000 7fa839a4c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564eee231900 0x564eee242f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:07.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.284+0000 7fa839a4c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564eee231900 0x564eee242f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:60254/0 (socket says 172.21.15.114:60254) 2024-09-13T18:27:07.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.284+0000 7fa839a4c640 1 -- 172.21.15.114:0/516930817 learned_addr learned my addr 172.21.15.114:0/516930817 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:07.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.285+0000 7fa83aa4e640 1 --2- 172.21.15.114:0/516930817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564eee23c1a0 0x564eee2434d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:07.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.285+0000 7fa83f1d7740 1 -- 172.21.15.114:0/516930817 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eee228ee0 con 0x564eee230fe0 2024-09-13T18:27:07.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.285+0000 7fa83f1d7740 1 -- 172.21.15.114:0/516930817 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eee230d80 con 0x564eee23c1a0 2024-09-13T18:27:07.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.285+0000 7fa83f1d7740 1 -- 172.21.15.114:0/516930817 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eee2278b0 con 0x564eee231900 2024-09-13T18:27:07.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.285+0000 7fa83a24d640 1 --2- 172.21.15.114:0/516930817 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564eee230fe0 0x564eee22cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:07.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.285+0000 7fa839a4c640 1 -- 172.21.15.114:0/516930817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564eee23c1a0 msgr2=0x564eee2434d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:07.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.285+0000 7fa839a4c640 1 --2- 172.21.15.114:0/516930817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564eee23c1a0 0x564eee2434d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.285+0000 7fa839a4c640 1 -- 172.21.15.114:0/516930817 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564eee230fe0 msgr2=0x564eee22cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:07.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.285+0000 7fa839a4c640 1 --2- 172.21.15.114:0/516930817 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564eee230fe0 0x564eee22cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.285+0000 7fa839a4c640 1 -- 172.21.15.114:0/516930817 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564eee135790 con 0x564eee231900 2024-09-13T18:27:07.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.286+0000 7fa839a4c640 1 --2- 172.21.15.114:0/516930817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564eee231900 0x564eee242f90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x564eee22e660 tx=0x7fa82c0046e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=305889dcc824a5c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:07.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.288+0000 7fa8297fa640 1 -- 172.21.15.114:0/516930817 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82c0049d0 con 0x564eee231900 2024-09-13T18:27:07.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.288+0000 7fa8297fa640 1 -- 172.21.15.114:0/516930817 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa82c0268e0 con 0x564eee231900 2024-09-13T18:27:07.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.289+0000 7fa83f1d7740 1 -- 172.21.15.114:0/516930817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564eee231900 msgr2=0x564eee242f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:07.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.289+0000 7fa83f1d7740 1 --2- 172.21.15.114:0/516930817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564eee231900 0x564eee242f90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x564eee22e660 tx=0x7fa82c0046e0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.291+0000 7fa83f1d7740 1 -- 172.21.15.114:0/516930817 shutdown_connections 2024-09-13T18:27:07.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.291+0000 7fa83f1d7740 1 --2- 172.21.15.114:0/516930817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564eee23c1a0 0x564eee2434d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.291+0000 7fa83f1d7740 1 --2- 172.21.15.114:0/516930817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564eee231900 0x564eee242f90 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.291+0000 7fa83f1d7740 1 --2- 172.21.15.114:0/516930817 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564eee230fe0 0x564eee22cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.291+0000 7fa83f1d7740 1 -- 172.21.15.114:0/516930817 >> 172.21.15.114:0/516930817 conn(0x564eee22d570 msgr2=0x564eee252fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:07.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.294+0000 7fa83f1d7740 1 -- 172.21.15.114:0/516930817 shutdown_connections 2024-09-13T18:27:07.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.294+0000 7fa83f1d7740 1 -- 172.21.15.114:0/516930817 wait complete. 2024-09-13T18:27:07.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.295+0000 7fa83f1d7740 1 -- start start 2024-09-13T18:27:07.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.295+0000 7fa83f1d7740 1 Processor -- start 2024-09-13T18:27:07.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.295+0000 7fa83f1d7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564eee230fe0 0x564eee243430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:07.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.295+0000 7fa83f1d7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564eee231900 0x564eee2361d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:07.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.296+0000 7fa83f1d7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564eee23c1a0 0x564eee236830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:07.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.296+0000 7fa83f1d7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eee236f30 con 0x564eee231900 2024-09-13T18:27:07.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.296+0000 7fa83f1d7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eee2350e0 con 0x564eee23c1a0 2024-09-13T18:27:07.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.296+0000 7fa83f1d7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eee2352a0 con 0x564eee230fe0 2024-09-13T18:27:07.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.296+0000 7fa83a24d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564eee230fe0 0x564eee243430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:07.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.296+0000 7fa83aa4e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564eee23c1a0 0x564eee236830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:07.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.296+0000 7fa83f1d7740 10 client.0 _ll_register_callbacks cb 0x7fa830009e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:27:07.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.296+0000 7fa83aa4e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564eee23c1a0 0x564eee236830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:36258/0 (socket says 172.21.15.114:36258) 2024-09-13T18:27:07.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.296+0000 7fa83aa4e640 1 -- 172.21.15.114:0/2143607077 learned_addr learned my addr 172.21.15.114:0/2143607077 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:07.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.297+0000 7fa839a4c640 1 --2- 172.21.15.114:0/2143607077 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564eee231900 0x564eee2361d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:07.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.298+0000 7fa839a4c640 1 -- 172.21.15.114:0/2143607077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564eee230fe0 msgr2=0x564eee243430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:07.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.298+0000 7fa839a4c640 1 --2- 172.21.15.114:0/2143607077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564eee230fe0 0x564eee243430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.298+0000 7fa839a4c640 1 -- 172.21.15.114:0/2143607077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564eee23c1a0 msgr2=0x564eee236830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:07.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.298+0000 7fa839a4c640 1 --2- 172.21.15.114:0/2143607077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564eee23c1a0 0x564eee236830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.298+0000 7fa839a4c640 1 -- 172.21.15.114:0/2143607077 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564eee14f840 con 0x564eee231900 2024-09-13T18:27:07.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.298+0000 7fa839a4c640 1 --2- 172.21.15.114:0/2143607077 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564eee231900 0x564eee2361d0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x564eede99850 tx=0x7fa82c004a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d19970f81b3ff13 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:07.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.298+0000 7fa81dffb640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:27:07.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.302+0000 7fa81dffb640 1 -- 172.21.15.114:0/2143607077 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82c026e60 con 0x564eee231900 2024-09-13T18:27:07.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.302+0000 7fa81dffb640 1 -- 172.21.15.114:0/2143607077 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fa82c012720 con 0x564eee231900 2024-09-13T18:27:07.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.302+0000 7fa83f1d7740 1 -- 172.21.15.114:0/2143607077 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa82c004c80 con 0x564eee231900 2024-09-13T18:27:07.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.308+0000 7fa81dffb640 1 -- 172.21.15.114:0/2143607077 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa82c0129a0 con 0x564eee231900 2024-09-13T18:27:07.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.308+0000 7fa83f1d7740 10 client.17184 fetch_fsmap learned FSMap version 32 2024-09-13T18:27:07.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.308+0000 7fa83f1d7740 1 -- 172.21.15.114:0/2143607077 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x564eee15aea0 con 0x564eee231900 2024-09-13T18:27:07.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.309+0000 7fa81dffb640 1 -- 172.21.15.114:0/2143607077 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa82c01a400 con 0x564eee231900 2024-09-13T18:27:07.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.309+0000 7fa81dffb640 1 -- 172.21.15.114:0/2143607077 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fa82c018030 con 0x564eee231900 2024-09-13T18:27:07.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.310+0000 7fa83f1d7740 10 client.17184 fetch_fsmap finished waiting for FSMap version 32 2024-09-13T18:27:07.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.310+0000 7fa83f1d7740 10 client.17184 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:27:07.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.310+0000 7fa83f1d7740 20 client.17184 populate_metadata read hostname 'smithi114' 2024-09-13T18:27:07.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.310+0000 7fa83f1d7740 1 --2- 172.21.15.114:0/2143607077 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x564eee2a2bb0 0x564eee2a4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:07.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.310+0000 7fa83f1d7740 4 client.17184 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T18:27:07.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.310+0000 7fa83f1d7740 1 -- 172.21.15.114:0/2143607077 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7fa82c004c80 con 0x564eee2a2bb0 2024-09-13T18:27:07.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.310+0000 7fa83a24d640 1 --2- 172.21.15.114:0/2143607077 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x564eee2a2bb0 0x564eee2a4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:07.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.313+0000 7fa81dffb640 10 client.17184 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:27:07.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.313+0000 7fa83a24d640 1 --2- 172.21.15.114:0/2143607077 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x564eee2a2bb0 0x564eee2a4fa0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6764ac7be75f0225 server_cookie=a95f4394b520ffc in_seq=0 out_seq=0 2024-09-13T18:27:07.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.316+0000 7fa81dffb640 1 -- 172.21.15.114:0/2143607077 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x7fa82c004c80 con 0x564eee2a2bb0 2024-09-13T18:27:07.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.317+0000 7fa81dffb640 10 client.17184 handle_command_reply: tid=0 2024-09-13T18:27:07.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.317+0000 7fa83f1d7740 1 client.17184 shutdown 2024-09-13T18:27:07.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.318+0000 7fa83f1d7740 1 -- 172.21.15.114:0/2143607077 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564eee231900 msgr2=0x564eee2361d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:07.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.318+0000 7fa83f1d7740 1 --2- 172.21.15.114:0/2143607077 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564eee231900 0x564eee2361d0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x564eede99850 tx=0x7fa82c004a00 comp rx=0 tx=0).stop 2024-09-13T18:27:07.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.323+0000 7fa83f1d7740 1 -- 172.21.15.114:0/2143607077 shutdown_connections 2024-09-13T18:27:07.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.323+0000 7fa83f1d7740 1 --2- 172.21.15.114:0/2143607077 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x564eee2a2bb0 0x564eee2a4fa0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.323+0000 7fa83f1d7740 1 --2- 172.21.15.114:0/2143607077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564eee23c1a0 0x564eee236830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.323+0000 7fa83f1d7740 1 --2- 172.21.15.114:0/2143607077 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564eee231900 0x564eee2361d0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.323+0000 7fa83f1d7740 1 --2- 172.21.15.114:0/2143607077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564eee230fe0 0x564eee243430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.323+0000 7fa83f1d7740 1 -- 172.21.15.114:0/2143607077 >> 172.21.15.114:0/2143607077 conn(0x564eee22d570 msgr2=0x564eee234160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:07.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.323+0000 7fa81dffb640 0 client.17184 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:27:07.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.324+0000 7fa83f1d7740 1 -- 172.21.15.114:0/2143607077 shutdown_connections 2024-09-13T18:27:07.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.324+0000 7fa83f1d7740 1 -- 172.21.15.114:0/2143607077 wait complete. 2024-09-13T18:27:07.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.327+0000 7fa83f1d7740 20 client.17184 trim_cache size 0 max 16384 2024-09-13T18:27:07.330 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:27:07.330 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-13T18:27:07.330 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": { 2024-09-13T18:27:07.331 INFO:teuthology.orchestra.run.smithi114.stdout: "4944d28d-7f7f-46be-bf12-f95da35e05b4": { 2024-09-13T18:27:07.331 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-13T18:27:07.331 INFO:teuthology.orchestra.run.smithi114.stdout: "tag": "4944d28d-7f7f-46be-bf12-f95da35e05b4", 2024-09-13T18:27:07.331 INFO:teuthology.orchestra.run.smithi114.stdout: "options": "recursive,force" 2024-09-13T18:27:07.331 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:27:07.331 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:27:07.331 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:27:07.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.332+0000 7fa83b24f640 1 -- 172.21.15.114:0/3683386470 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa8200a1410 msgr2=0x7fa8200a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:07.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.332+0000 7fa83b24f640 1 --2- 172.21.15.114:0/3683386470 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa8200a1410 0x7fa8200a38d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa8341193d0 tx=0x7fa8300098c0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.332+0000 7fa83b24f640 1 -- 172.21.15.114:0/3683386470 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa8341074b0 msgr2=0x7fa8341187f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:07.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.332+0000 7fa83b24f640 1 --2- 172.21.15.114:0/3683386470 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa8341074b0 0x7fa8341187f0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fa82402a8f0 tx=0x7fa824032670 comp rx=0 tx=0).stop 2024-09-13T18:27:07.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.333+0000 7fa83b24f640 1 -- 172.21.15.114:0/3683386470 shutdown_connections 2024-09-13T18:27:07.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.333+0000 7fa83b24f640 1 --2- 172.21.15.114:0/3683386470 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa8200a1410 0x7fa8200a38d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.333+0000 7fa83b24f640 1 --2- 172.21.15.114:0/3683386470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa834119400 0x7fa8341128c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.333+0000 7fa83b24f640 1 --2- 172.21.15.114:0/3683386470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa834107890 0x7fa834118d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:07.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.333+0000 7fa83b24f640 1 --2- 172.21.15.114:0/3683386470 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa8341074b0 0x7fa8341187f0 secure :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fa82402a8f0 tx=0x7fa824032670 comp rx=0 tx=0).stop 2024-09-13T18:27:07.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.333+0000 7fa83b24f640 1 -- 172.21.15.114:0/3683386470 >> 172.21.15.114:0/3683386470 conn(0x7fa83406af40 msgr2=0x7fa83406ecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:07.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.335+0000 7fa83b24f640 1 -- 172.21.15.114:0/3683386470 shutdown_connections 2024-09-13T18:27:07.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:07.335+0000 7fa83b24f640 1 -- 172.21.15.114:0/3683386470 wait complete. 2024-09-13T18:27:07.373 INFO:tasks.cephfs.filesystem:scrub status for tag:4944d28d-7f7f-46be-bf12-f95da35e05b4 - {'path': '/', 'tag': '4944d28d-7f7f-46be-bf12-f95da35e05b4', 'options': 'recursive,force'} 2024-09-13T18:27:08.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:08 smithi204 ceph-mon[34777]: pgmap v142: 129 pgs: 129 active+clean; 16 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 46 MiB/s wr, 755 op/s 2024-09-13T18:27:08.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:08 smithi204 ceph-mon[34777]: scrub queued for path: / 2024-09-13T18:27:08.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:08 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:27:08.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:08 smithi204 ceph-mon[34777]: scrub summary: active paths [/] 2024-09-13T18:27:08.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:08 smithi204 ceph-mon[34777]: scrub summary: active paths [/] 2024-09-13T18:27:08.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:08 smithi204 ceph-mon[34777]: scrub summary: active paths [/] 2024-09-13T18:27:08.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:08 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:27:08.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:08 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:27:08.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:08 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:27:08.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:08 smithi114 ceph-mon[30897]: pgmap v142: 129 pgs: 129 active+clean; 16 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 46 MiB/s wr, 755 op/s 2024-09-13T18:27:08.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:08 smithi114 ceph-mon[30897]: scrub queued for path: / 2024-09-13T18:27:08.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:08 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:27:08.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:08 smithi114 ceph-mon[30897]: scrub summary: active paths [/] 2024-09-13T18:27:08.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:08 smithi114 ceph-mon[30897]: scrub summary: active paths [/] 2024-09-13T18:27:08.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:08 smithi114 ceph-mon[30897]: scrub summary: active paths [/] 2024-09-13T18:27:08.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:08 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:27:08.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:08 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:27:08.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:08 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:27:08.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:08 smithi134 ceph-mon[34283]: pgmap v142: 129 pgs: 129 active+clean; 16 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 46 MiB/s wr, 755 op/s 2024-09-13T18:27:08.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:08 smithi134 ceph-mon[34283]: scrub queued for path: / 2024-09-13T18:27:08.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:08 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:27:08.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:08 smithi134 ceph-mon[34283]: scrub summary: active paths [/] 2024-09-13T18:27:08.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:08 smithi134 ceph-mon[34283]: scrub summary: active paths [/] 2024-09-13T18:27:08.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:08 smithi134 ceph-mon[34283]: scrub summary: active paths [/] 2024-09-13T18:27:08.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:08 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:27:08.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:08 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:27:08.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:08 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:27:10.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:10 smithi204 ceph-mon[34777]: pgmap v143: 129 pgs: 129 active+clean; 16 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 40 MiB/s wr, 656 op/s 2024-09-13T18:27:10.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:10 smithi114 ceph-mon[30897]: pgmap v143: 129 pgs: 129 active+clean; 16 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 40 MiB/s wr, 656 op/s 2024-09-13T18:27:10.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:10 smithi134 ceph-mon[34283]: pgmap v143: 129 pgs: 129 active+clean; 16 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 40 MiB/s wr, 656 op/s 2024-09-13T18:27:12.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:12 smithi204 ceph-mon[34777]: pgmap v144: 129 pgs: 129 active+clean; 16 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 59 MiB/s wr, 972 op/s 2024-09-13T18:27:12.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:12 smithi114 ceph-mon[30897]: pgmap v144: 129 pgs: 129 active+clean; 16 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 59 MiB/s wr, 972 op/s 2024-09-13T18:27:12.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:12 smithi134 ceph-mon[34283]: pgmap v144: 129 pgs: 129 active+clean; 16 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 59 MiB/s wr, 972 op/s 2024-09-13T18:27:13.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:13 smithi204 ceph-mon[34777]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-13T18:27:13.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:13 smithi114 ceph-mon[30897]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-13T18:27:13.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:13 smithi134 ceph-mon[34283]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-13T18:27:14.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:14 smithi204 ceph-mon[34777]: pgmap v145: 129 pgs: 129 active+clean; 16 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 38 MiB/s wr, 636 op/s 2024-09-13T18:27:14.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:14 smithi114 ceph-mon[30897]: pgmap v145: 129 pgs: 129 active+clean; 16 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 38 MiB/s wr, 636 op/s 2024-09-13T18:27:14.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:14 smithi134 ceph-mon[34283]: pgmap v145: 129 pgs: 129 active+clean; 16 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 38 MiB/s wr, 636 op/s 2024-09-13T18:27:16.420 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:16 smithi134 ceph-mon[34283]: pgmap v146: 129 pgs: 129 active+clean; 16 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 52 MiB/s wr, 865 op/s 2024-09-13T18:27:16.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:16 smithi204 ceph-mon[34777]: pgmap v146: 129 pgs: 129 active+clean; 16 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 52 MiB/s wr, 865 op/s 2024-09-13T18:27:16.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:16 smithi114 ceph-mon[30897]: pgmap v146: 129 pgs: 129 active+clean; 16 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 52 MiB/s wr, 865 op/s 2024-09-13T18:27:16.953 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.947+0000 7f824359e640 1 -- 172.21.15.114:0/2814178059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8244119400 msgr2=0x7f824411b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:16.953 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.947+0000 7f824359e640 1 --2- 172.21.15.114:0/2814178059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8244119400 0x7f824411b7f0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f823c009d00 tx=0x7f823c02a950 comp rx=0 tx=0).stop 2024-09-13T18:27:16.953 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.949+0000 7f824359e640 1 -- 172.21.15.114:0/2814178059 shutdown_connections 2024-09-13T18:27:16.953 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.949+0000 7f824359e640 1 --2- 172.21.15.114:0/2814178059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8244119400 0x7f824411b7f0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:16.953 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.949+0000 7f824359e640 1 --2- 172.21.15.114:0/2814178059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8244107890 0x7f8244107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:16.953 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.949+0000 7f824359e640 1 --2- 172.21.15.114:0/2814178059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82441074b0 0x7f824410e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:16.953 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.949+0000 7f824359e640 1 -- 172.21.15.114:0/2814178059 >> 172.21.15.114:0/2814178059 conn(0x7f824406adb0 msgr2=0x7f824406b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:16.954 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.952+0000 7f824359e640 1 -- 172.21.15.114:0/2814178059 shutdown_connections 2024-09-13T18:27:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.955+0000 7f824359e640 1 -- 172.21.15.114:0/2814178059 wait complete. 2024-09-13T18:27:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.956+0000 7f824359e640 1 Processor -- start 2024-09-13T18:27:16.959 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.956+0000 7f824359e640 1 -- start start 2024-09-13T18:27:16.960 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.957+0000 7f824359e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82441074b0 0x7f8244112400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:16.960 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.957+0000 7f824359e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8244107890 0x7f8244112940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:16.960 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.957+0000 7f824359e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8244119400 0x7f8244116ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:16.960 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.957+0000 7f824359e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8244113260 con 0x7f82441074b0 2024-09-13T18:27:16.960 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.957+0000 7f824359e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8244117400 con 0x7f8244107890 2024-09-13T18:27:16.960 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.957+0000 7f824359e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82441175c0 con 0x7f8244119400 2024-09-13T18:27:16.961 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.957+0000 7f8242d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8244119400 0x7f8244116ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:16.963 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.957+0000 7f8242d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8244119400 0x7f8244116ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:34196/0 (socket says 172.21.15.114:34196) 2024-09-13T18:27:16.963 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.957+0000 7f8242d9d640 1 -- 172.21.15.114:0/1493899087 learned_addr learned my addr 172.21.15.114:0/1493899087 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:16.964 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.957+0000 7f8241d9b640 1 --2- 172.21.15.114:0/1493899087 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8244107890 0x7f8244112940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:16.964 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.958+0000 7f8242d9d640 1 -- 172.21.15.114:0/1493899087 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8244107890 msgr2=0x7f8244112940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:16.964 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.958+0000 7f8242d9d640 1 --2- 172.21.15.114:0/1493899087 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8244107890 0x7f8244112940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:16.964 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.958+0000 7f8242d9d640 1 -- 172.21.15.114:0/1493899087 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82441074b0 msgr2=0x7f8244112400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:16.964 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.958+0000 7f8242d9d640 1 --2- 172.21.15.114:0/1493899087 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82441074b0 0x7f8244112400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:16.964 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.958+0000 7f8242d9d640 1 -- 172.21.15.114:0/1493899087 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8238009490 con 0x7f8244119400 2024-09-13T18:27:16.964 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.958+0000 7f8242d9d640 1 --2- 172.21.15.114:0/1493899087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8244119400 0x7f8244116ec0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f823c0098d0 tx=0x7f823c0308b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:16.964 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.958+0000 7f82337fe640 1 -- 172.21.15.114:0/1493899087 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823c039420 con 0x7f8244119400 2024-09-13T18:27:16.964 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.958+0000 7f82337fe640 1 -- 172.21.15.114:0/1493899087 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f823c039a10 con 0x7f8244119400 2024-09-13T18:27:16.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.958+0000 7f82337fe640 1 -- 172.21.15.114:0/1493899087 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823c0428a0 con 0x7f8244119400 2024-09-13T18:27:16.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.959+0000 7f824359e640 1 -- 172.21.15.114:0/1493899087 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f823c009900 con 0x7f8244119400 2024-09-13T18:27:16.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.959+0000 7f824359e640 1 -- 172.21.15.114:0/1493899087 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8244117b10 con 0x7f8244119400 2024-09-13T18:27:16.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.962+0000 7f82337fe640 1 -- 172.21.15.114:0/1493899087 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f823c00a040 con 0x7f8244119400 2024-09-13T18:27:16.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.963+0000 7f82337fe640 1 --2- 172.21.15.114:0/1493899087 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f822c0a1400 0x7f822c0a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:16.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.963+0000 7f82337fe640 1 -- 172.21.15.114:0/1493899087 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f823c0ede00 con 0x7f8244119400 2024-09-13T18:27:16.968 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.966+0000 7f824b6c1740 1 Processor -- start 2024-09-13T18:27:16.969 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.966+0000 7f824b6c1740 1 -- start start 2024-09-13T18:27:16.969 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.966+0000 7f824b6c1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a515bca500 0x55a515bca8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:16.969 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.966+0000 7f824b6c1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a515bcae20 0x55a515bd5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:16.969 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.966+0000 7f824b6c1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a515bd56c0 0x55a515bd7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:16.969 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.967+0000 7f824b6c1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a515b35c30 con 0x55a515bcae20 2024-09-13T18:27:16.969 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.967+0000 7f824b6c1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a515b333c0 con 0x55a515bca500 2024-09-13T18:27:16.969 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.967+0000 7f824b6c1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a515b2ecb0 con 0x55a515bd56c0 2024-09-13T18:27:16.970 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.967+0000 7f8241d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a515bcae20 0x55a515bd5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:16.970 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.967+0000 7f8241d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a515bcae20 0x55a515bd5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33270/0 (socket says 172.21.15.114:33270) 2024-09-13T18:27:16.970 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.967+0000 7f8241d9b640 1 -- 172.21.15.114:0/3807209282 learned_addr learned my addr 172.21.15.114:0/3807209282 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:16.970 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.967+0000 7f824259c640 1 --2- 172.21.15.114:0/1493899087 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f822c0a1400 0x7f822c0a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:16.970 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.968+0000 7f824259c640 1 --2- 172.21.15.114:0/3807209282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a515bca500 0x55a515bca8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:16.970 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.968+0000 7f8242d9d640 1 --2- 172.21.15.114:0/3807209282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a515bd56c0 0x55a515bd7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:16.971 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.969+0000 7f8241d9b640 1 -- 172.21.15.114:0/3807209282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a515bd56c0 msgr2=0x55a515bd7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:16.971 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.969+0000 7f8241d9b640 1 --2- 172.21.15.114:0/3807209282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a515bd56c0 0x55a515bd7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:16.971 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.969+0000 7f8241d9b640 1 -- 172.21.15.114:0/3807209282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a515bca500 msgr2=0x55a515bca8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:16.972 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.969+0000 7f8241d9b640 1 --2- 172.21.15.114:0/3807209282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a515bca500 0x55a515bca8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:16.972 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.969+0000 7f8241d9b640 1 -- 172.21.15.114:0/3807209282 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a515ad5910 con 0x55a515bcae20 2024-09-13T18:27:16.974 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.972+0000 7f8241d9b640 1 --2- 172.21.15.114:0/3807209282 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a515bcae20 0x55a515bd5180 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f823400ab70 tx=0x7f823400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=697880fd26d87490 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:16.976 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.974+0000 7f8230ff9640 1 -- 172.21.15.114:0/3807209282 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8234018640 con 0x55a515bcae20 2024-09-13T18:27:16.976 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.974+0000 7f8230ff9640 1 -- 172.21.15.114:0/3807209282 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f823400ae70 con 0x55a515bcae20 2024-09-13T18:27:16.976 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.974+0000 7f8230ff9640 1 -- 172.21.15.114:0/3807209282 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8234010a80 con 0x55a515bcae20 2024-09-13T18:27:16.977 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.975+0000 7f824b6c1740 1 -- 172.21.15.114:0/3807209282 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a515bcae20 msgr2=0x55a515bd5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:16.977 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.975+0000 7f824b6c1740 1 --2- 172.21.15.114:0/3807209282 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a515bcae20 0x55a515bd5180 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f823400ab70 tx=0x7f823400f790 comp rx=0 tx=0).stop 2024-09-13T18:27:16.978 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.976+0000 7f824b6c1740 1 -- 172.21.15.114:0/3807209282 shutdown_connections 2024-09-13T18:27:16.978 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.976+0000 7f824b6c1740 1 --2- 172.21.15.114:0/3807209282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a515bd56c0 0x55a515bd7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:16.978 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.976+0000 7f824b6c1740 1 --2- 172.21.15.114:0/3807209282 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a515bcae20 0x55a515bd5180 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:16.978 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.976+0000 7f824b6c1740 1 --2- 172.21.15.114:0/3807209282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a515bca500 0x55a515bca8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:16.978 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.976+0000 7f824b6c1740 1 -- 172.21.15.114:0/3807209282 >> 172.21.15.114:0/3807209282 conn(0x55a515bc6a90 msgr2=0x55a515bc6e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:16.979 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.977+0000 7f824b6c1740 1 -- 172.21.15.114:0/3807209282 shutdown_connections 2024-09-13T18:27:16.980 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.978+0000 7f824b6c1740 1 -- 172.21.15.114:0/3807209282 wait complete. 2024-09-13T18:27:16.980 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.978+0000 7f824b6c1740 1 -- start start 2024-09-13T18:27:16.980 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.978+0000 7f824b6c1740 1 Processor -- start 2024-09-13T18:27:16.981 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.979+0000 7f824b6c1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a515bca500 0x55a515bd5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:16.981 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.979+0000 7f824b6c1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a515bcae20 0x55a515bd2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:16.981 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.979+0000 7f824b6c1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a515bd56c0 0x55a515bd2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:16.981 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.979+0000 7f824b6c1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a515aece80 con 0x55a515bd56c0 2024-09-13T18:27:16.981 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.979+0000 7f824b6c1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a515aecb20 con 0x55a515bcae20 2024-09-13T18:27:16.981 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.979+0000 7f824b6c1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a515ab2260 con 0x55a515bca500 2024-09-13T18:27:16.981 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.979+0000 7f824b6c1740 10 client.0 _ll_register_callbacks cb 0x55a515ae7df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:27:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.980+0000 7f8242d9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a515bd56c0 0x55a515bd2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.980+0000 7f8242d9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a515bd56c0 0x55a515bd2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33272/0 (socket says 172.21.15.114:33272) 2024-09-13T18:27:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.980+0000 7f8242d9d640 1 -- 172.21.15.114:0/310716286 learned_addr learned my addr 172.21.15.114:0/310716286 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.980+0000 7f8241d9b640 1 --2- 172.21.15.114:0/310716286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a515bcae20 0x55a515bd2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.980+0000 7f8241d9b640 1 -- 172.21.15.114:0/310716286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a515bca500 msgr2=0x55a515bd5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.980+0000 7f824259c640 1 --2- 172.21.15.114:0/310716286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a515bca500 0x55a515bd5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.980+0000 7f8241d9b640 1 --2- 172.21.15.114:0/310716286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a515bca500 0x55a515bd5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.980+0000 7f8241d9b640 1 -- 172.21.15.114:0/310716286 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a515bd56c0 msgr2=0x55a515bd2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.980+0000 7f8241d9b640 1 --2- 172.21.15.114:0/310716286 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a515bd56c0 0x55a515bd2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.980+0000 7f8241d9b640 1 -- 172.21.15.114:0/310716286 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f823400a7d0 con 0x55a515bcae20 2024-09-13T18:27:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.981+0000 7f824259c640 1 --2- 172.21.15.114:0/310716286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a515bca500 0x55a515bd5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:27:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.983+0000 7f8241d9b640 1 --2- 172.21.15.114:0/310716286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a515bcae20 0x55a515bd2310 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f823400ab70 tx=0x7f8234010f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a340297a85c8618e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.985+0000 7f824259c640 1 --2- 172.21.15.114:0/1493899087 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f822c0a1400 0x7f822c0a38c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8238005d00 tx=0x7f82380097f0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:16.988 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.986+0000 7f8229ffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:27:16.988 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.986+0000 7f8229ffb640 1 -- 172.21.15.114:0/310716286 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8234005450 con 0x55a515bcae20 2024-09-13T18:27:16.988 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.986+0000 7f8229ffb640 1 -- 172.21.15.114:0/310716286 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f8234010430 con 0x55a515bcae20 2024-09-13T18:27:16.990 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.987+0000 7f824b6c1740 1 -- 172.21.15.114:0/310716286 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a515b57680 con 0x55a515bcae20 2024-09-13T18:27:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.991+0000 7f8229ffb640 1 -- 172.21.15.114:0/310716286 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82340055b0 con 0x55a515bcae20 2024-09-13T18:27:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.991+0000 7f824b6c1740 10 client.26543 fetch_fsmap learned FSMap version 33 2024-09-13T18:27:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.991+0000 7f824b6c1740 1 -- 172.21.15.114:0/310716286 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55a515b2d330 con 0x55a515bcae20 2024-09-13T18:27:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.995+0000 7f8229ffb640 1 -- 172.21.15.114:0/310716286 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a515b57680 con 0x55a515bcae20 2024-09-13T18:27:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.995+0000 7f8229ffb640 1 -- 172.21.15.114:0/310716286 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 33) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f8234018870 con 0x55a515bcae20 2024-09-13T18:27:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.995+0000 7f824b6c1740 10 client.26543 fetch_fsmap finished waiting for FSMap version 33 2024-09-13T18:27:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.995+0000 7f824b6c1740 10 client.26543 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:27:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.995+0000 7f824b6c1740 20 client.26543 populate_metadata read hostname 'smithi114' 2024-09-13T18:27:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.995+0000 7f824b6c1740 1 --2- 172.21.15.114:0/310716286 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55a515be5190 0x55a515be75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.995+0000 7f824b6c1740 4 client.26543 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:27:17.000 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.995+0000 7f824b6c1740 1 -- 172.21.15.114:0/310716286 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a515b225d0 con 0x55a515be5190 2024-09-13T18:27:17.000 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.996+0000 7f824259c640 1 --2- 172.21.15.114:0/310716286 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55a515be5190 0x55a515be75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.998+0000 7f824259c640 1 --2- 172.21.15.114:0/310716286 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55a515be5190 0x55a515be75b0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c49c9b94f8f90f9 server_cookie=e64cce905feb23fd in_seq=0 out_seq=0 2024-09-13T18:27:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.998+0000 7f8229ffb640 10 client.26543 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:27:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.999+0000 7f8229ffb640 1 -- 172.21.15.114:0/310716286 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55a515b225d0 con 0x55a515be5190 2024-09-13T18:27:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:16.999+0000 7f8229ffb640 10 client.26543 handle_command_reply: tid=0 2024-09-13T18:27:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.002+0000 7f824b6c1740 1 client.26543 shutdown 2024-09-13T18:27:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.003+0000 7f824b6c1740 1 -- 172.21.15.114:0/310716286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a515bcae20 msgr2=0x55a515bd2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.003+0000 7f824b6c1740 1 --2- 172.21.15.114:0/310716286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a515bcae20 0x55a515bd2310 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f823400ab70 tx=0x7f8234010f70 comp rx=0 tx=0).stop 2024-09-13T18:27:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.003+0000 7f824b6c1740 1 -- 172.21.15.114:0/310716286 shutdown_connections 2024-09-13T18:27:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.003+0000 7f824b6c1740 1 --2- 172.21.15.114:0/310716286 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55a515be5190 0x55a515be75b0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.003+0000 7f824b6c1740 1 --2- 172.21.15.114:0/310716286 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a515bd56c0 0x55a515bd2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.003+0000 7f824b6c1740 1 --2- 172.21.15.114:0/310716286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a515bcae20 0x55a515bd2310 secure :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f823400ab70 tx=0x7f8234010f70 comp rx=0 tx=0).stop 2024-09-13T18:27:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.003+0000 7f824b6c1740 1 --2- 172.21.15.114:0/310716286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a515bca500 0x55a515bd5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.003+0000 7f824b6c1740 1 -- 172.21.15.114:0/310716286 >> 172.21.15.114:0/310716286 conn(0x55a515bc6a90 msgr2=0x55a515bc5d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.003+0000 7f8229ffb640 0 client.26543 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:27:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.005+0000 7f824b6c1740 1 -- 172.21.15.114:0/310716286 shutdown_connections 2024-09-13T18:27:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.005+0000 7f824b6c1740 1 -- 172.21.15.114:0/310716286 wait complete. 2024-09-13T18:27:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.007+0000 7f824b6c1740 20 client.26543 trim_cache size 0 max 16384 2024-09-13T18:27:17.038 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.036+0000 7f824b6c1740 1 Processor -- start 2024-09-13T18:27:17.038 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.036+0000 7f824b6c1740 1 -- start start 2024-09-13T18:27:17.038 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.036+0000 7f824b6c1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a515bca500 0x55a515bc6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:17.040 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.037+0000 7f824b6c1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a515bd56c0 0x55a515bc73b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:17.040 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.037+0000 7f824259c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a515bca500 0x55a515bc6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:17.040 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.037+0000 7f824259c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a515bca500 0x55a515bc6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:34232/0 (socket says 172.21.15.114:34232) 2024-09-13T18:27:17.040 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.037+0000 7f824259c640 1 -- 172.21.15.114:0/1830618464 learned_addr learned my addr 172.21.15.114:0/1830618464 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:17.040 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.037+0000 7f8241d9b640 1 --2- 172.21.15.114:0/1830618464 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a515bd56c0 0x55a515bc73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:17.040 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.037+0000 7f824b6c1740 1 --2- 172.21.15.114:0/1830618464 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a515bd4a90 0x55a515bd4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:17.040 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.037+0000 7f824b6c1740 1 -- 172.21.15.114:0/1830618464 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a515bd5480 con 0x55a515bd4a90 2024-09-13T18:27:17.040 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.037+0000 7f8242d9d640 1 --2- 172.21.15.114:0/1830618464 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a515bd4a90 0x55a515bd4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:17.040 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.037+0000 7f824b6c1740 1 -- 172.21.15.114:0/1830618464 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a515bc9d60 con 0x55a515bd56c0 2024-09-13T18:27:17.040 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.037+0000 7f824b6c1740 1 -- 172.21.15.114:0/1830618464 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a515bd2c30 con 0x55a515bca500 2024-09-13T18:27:17.041 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.038+0000 7f824259c640 1 -- 172.21.15.114:0/1830618464 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a515bd56c0 msgr2=0x55a515bc73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:17.041 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.038+0000 7f824259c640 1 --2- 172.21.15.114:0/1830618464 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a515bd56c0 0x55a515bc73b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:17.041 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.038+0000 7f824259c640 1 -- 172.21.15.114:0/1830618464 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a515bd4a90 msgr2=0x55a515bd4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:17.041 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.038+0000 7f824259c640 1 --2- 172.21.15.114:0/1830618464 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a515bd4a90 0x55a515bd4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:17.041 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.038+0000 7f824259c640 1 -- 172.21.15.114:0/1830618464 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a515acf300 con 0x55a515bca500 2024-09-13T18:27:17.041 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.039+0000 7f824259c640 1 --2- 172.21.15.114:0/1830618464 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a515bca500 0x55a515bc6e70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8238009c80 tx=0x7f8238005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c887f73d8aefb84 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:17.051 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.049+0000 7f82317fa640 1 -- 172.21.15.114:0/1830618464 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8238004460 con 0x55a515bca500 2024-09-13T18:27:17.051 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.049+0000 7f82317fa640 1 -- 172.21.15.114:0/1830618464 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f82380054b0 con 0x55a515bca500 2024-09-13T18:27:17.053 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.050+0000 7f824b6c1740 1 -- 172.21.15.114:0/1830618464 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a515bca500 msgr2=0x55a515bc6e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:17.053 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.050+0000 7f824b6c1740 1 --2- 172.21.15.114:0/1830618464 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a515bca500 0x55a515bc6e70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8238009c80 tx=0x7f8238005f90 comp rx=0 tx=0).stop 2024-09-13T18:27:17.053 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.050+0000 7f824b6c1740 1 -- 172.21.15.114:0/1830618464 shutdown_connections 2024-09-13T18:27:17.053 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.050+0000 7f824b6c1740 1 --2- 172.21.15.114:0/1830618464 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a515bd4a90 0x55a515bd4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:17.053 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.050+0000 7f824b6c1740 1 --2- 172.21.15.114:0/1830618464 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a515bd56c0 0x55a515bc73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:17.053 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.050+0000 7f824b6c1740 1 --2- 172.21.15.114:0/1830618464 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a515bca500 0x55a515bc6e70 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:17.053 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.050+0000 7f824b6c1740 1 -- 172.21.15.114:0/1830618464 >> 172.21.15.114:0/1830618464 conn(0x55a515bc6a90 msgr2=0x55a515be0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:17.053 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.051+0000 7f824b6c1740 1 -- 172.21.15.114:0/1830618464 shutdown_connections 2024-09-13T18:27:17.054 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.051+0000 7f824b6c1740 1 -- 172.21.15.114:0/1830618464 wait complete. 2024-09-13T18:27:17.054 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.051+0000 7f824b6c1740 1 -- start start 2024-09-13T18:27:17.054 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.051+0000 7f824b6c1740 1 Processor -- start 2024-09-13T18:27:17.054 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.052+0000 7f824b6c1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a515bd4a90 0x55a515bdc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:17.054 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.052+0000 7f824b6c1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a515bd56c0 0x55a515bdc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:17.054 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.052+0000 7f824b6c1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a515bceb00 0x55a515bcef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:17.054 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.052+0000 7f824b6c1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a515bcf680 con 0x55a515bd4a90 2024-09-13T18:27:17.054 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.052+0000 7f824b6c1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a515bcf7f0 con 0x55a515bd56c0 2024-09-13T18:27:17.054 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.052+0000 7f824b6c1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a515c3b690 con 0x55a515bceb00 2024-09-13T18:27:17.055 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.052+0000 7f824b6c1740 10 client.0 _ll_register_callbacks cb 0x7f8244119350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:27:17.055 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.052+0000 7f8241d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a515bd56c0 0x55a515bdc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:17.055 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.052+0000 7f8241d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a515bd56c0 0x55a515bdc900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:39294/0 (socket says 172.21.15.114:39294) 2024-09-13T18:27:17.055 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.052+0000 7f8241d9b640 1 -- 172.21.15.114:0/37923668 learned_addr learned my addr 172.21.15.114:0/37923668 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:17.056 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.052+0000 7f8241d9b640 1 -- 172.21.15.114:0/37923668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a515bceb00 msgr2=0x55a515bcef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:17.056 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.052+0000 7f8241d9b640 1 --2- 172.21.15.114:0/37923668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a515bceb00 0x55a515bcef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:17.056 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.052+0000 7f8241d9b640 1 -- 172.21.15.114:0/37923668 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a515bd4a90 msgr2=0x55a515bdc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:17.056 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.053+0000 7f8241d9b640 1 --2- 172.21.15.114:0/37923668 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a515bd4a90 0x55a515bdc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:17.056 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.053+0000 7f8241d9b640 1 -- 172.21.15.114:0/37923668 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82380040c0 con 0x55a515bd56c0 2024-09-13T18:27:17.056 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.053+0000 7f8241d9b640 1 --2- 172.21.15.114:0/37923668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a515bd56c0 0x55a515bdc900 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f823400a7a0 tx=0x7f8234005480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1261d992629f305b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:17.056 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.053+0000 7f8229ffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:27:17.056 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.053+0000 7f8229ffb640 1 -- 172.21.15.114:0/37923668 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8234010b10 con 0x55a515bd56c0 2024-09-13T18:27:17.056 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.053+0000 7f8229ffb640 1 -- 172.21.15.114:0/37923668 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f8234000e40 con 0x55a515bd56c0 2024-09-13T18:27:17.056 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.054+0000 7f824b6c1740 1 -- 172.21.15.114:0/37923668 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a515822300 con 0x55a515bd56c0 2024-09-13T18:27:17.062 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.056+0000 7f8229ffb640 1 -- 172.21.15.114:0/37923668 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8234018650 con 0x55a515bd56c0 2024-09-13T18:27:17.062 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.056+0000 7f824b6c1740 10 client.26549 fetch_fsmap learned FSMap version 33 2024-09-13T18:27:17.062 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.056+0000 7f824b6c1740 1 -- 172.21.15.114:0/37923668 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55a515af48e0 con 0x55a515bd56c0 2024-09-13T18:27:17.071 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.068+0000 7f8229ffb640 1 -- 172.21.15.114:0/37923668 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a515822300 con 0x55a515bd56c0 2024-09-13T18:27:17.071 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.069+0000 7f8229ffb640 1 -- 172.21.15.114:0/37923668 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 33) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f8234004cb0 con 0x55a515bd56c0 2024-09-13T18:27:17.071 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.069+0000 7f824b6c1740 10 client.26549 fetch_fsmap finished waiting for FSMap version 33 2024-09-13T18:27:17.071 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.069+0000 7f824b6c1740 10 client.26549 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:27:17.072 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.069+0000 7f824b6c1740 20 client.26549 populate_metadata read hostname 'smithi114' 2024-09-13T18:27:17.072 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.069+0000 7f824b6c1740 1 --2- 172.21.15.114:0/37923668 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55a515c3cb30 0x55a515c3ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:17.072 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.069+0000 7f824b6c1740 4 client.26549 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T18:27:17.072 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.069+0000 7f824b6c1740 1 -- 172.21.15.114:0/37923668 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a515833600 con 0x55a515c3cb30 2024-09-13T18:27:17.072 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.070+0000 7f824259c640 1 --2- 172.21.15.114:0/37923668 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55a515c3cb30 0x55a515c3ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:17.073 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.071+0000 7f824259c640 1 --2- 172.21.15.114:0/37923668 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55a515c3cb30 0x55a515c3ef50 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1263c5545f0fab6 server_cookie=bcca227282115b70 in_seq=0 out_seq=0 2024-09-13T18:27:17.073 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.071+0000 7f8229ffb640 10 client.26549 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:27:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.107+0000 7f8229ffb640 1 -- 172.21.15.114:0/37923668 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a515833600 con 0x55a515c3cb30 2024-09-13T18:27:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.107+0000 7f8229ffb640 10 client.26549 handle_command_reply: tid=0 2024-09-13T18:27:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.107+0000 7f824b6c1740 1 client.26549 shutdown 2024-09-13T18:27:17.111 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.109+0000 7f824b6c1740 1 -- 172.21.15.114:0/37923668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a515bd56c0 msgr2=0x55a515bdc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:17.111 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.109+0000 7f824b6c1740 1 --2- 172.21.15.114:0/37923668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a515bd56c0 0x55a515bdc900 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f823400a7a0 tx=0x7f8234005480 comp rx=0 tx=0).stop 2024-09-13T18:27:17.112 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.109+0000 7f824b6c1740 1 -- 172.21.15.114:0/37923668 shutdown_connections 2024-09-13T18:27:17.112 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.109+0000 7f824b6c1740 1 --2- 172.21.15.114:0/37923668 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55a515c3cb30 0x55a515c3ef50 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:17.112 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.109+0000 7f824b6c1740 1 --2- 172.21.15.114:0/37923668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a515bceb00 0x55a515bcef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:17.112 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.109+0000 7f824b6c1740 1 --2- 172.21.15.114:0/37923668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a515bd56c0 0x55a515bdc900 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:17.112 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.109+0000 7f824b6c1740 1 --2- 172.21.15.114:0/37923668 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a515bd4a90 0x55a515bdc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:17.112 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.109+0000 7f824b6c1740 1 -- 172.21.15.114:0/37923668 >> 172.21.15.114:0/37923668 conn(0x55a515bc6a90 msgr2=0x55a515be0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:17.112 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.109+0000 7f8229ffb640 0 client.26549 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:27:17.112 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.109+0000 7f824b6c1740 1 -- 172.21.15.114:0/37923668 shutdown_connections 2024-09-13T18:27:17.112 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.109+0000 7f824b6c1740 1 -- 172.21.15.114:0/37923668 wait complete. 2024-09-13T18:27:17.113 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.110+0000 7f824b6c1740 20 client.26549 trim_cache size 0 max 16384 2024-09-13T18:27:17.113 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-13T18:27:17.113 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-13T18:27:17.113 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-13T18:27:17.113 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-13T18:27:17.114 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.111+0000 7f824359e640 1 -- 172.21.15.114:0/1493899087 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f822c0a1400 msgr2=0x7f822c0a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:17.114 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.111+0000 7f824359e640 1 --2- 172.21.15.114:0/1493899087 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f822c0a1400 0x7f822c0a38c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8238005d00 tx=0x7f82380097f0 comp rx=0 tx=0).stop 2024-09-13T18:27:17.114 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.111+0000 7f824359e640 1 -- 172.21.15.114:0/1493899087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8244119400 msgr2=0x7f8244116ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:17.114 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.111+0000 7f824359e640 1 --2- 172.21.15.114:0/1493899087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8244119400 0x7f8244116ec0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f823c0098d0 tx=0x7f823c0308b0 comp rx=0 tx=0).stop 2024-09-13T18:27:17.115 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.112+0000 7f824359e640 1 -- 172.21.15.114:0/1493899087 shutdown_connections 2024-09-13T18:27:17.115 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.112+0000 7f824359e640 1 --2- 172.21.15.114:0/1493899087 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f822c0a1400 0x7f822c0a38c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:17.115 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.112+0000 7f824359e640 1 --2- 172.21.15.114:0/1493899087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8244119400 0x7f8244116ec0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:17.115 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.112+0000 7f824359e640 1 --2- 172.21.15.114:0/1493899087 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8244107890 0x7f8244112940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:17.115 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.112+0000 7f824359e640 1 --2- 172.21.15.114:0/1493899087 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82441074b0 0x7f8244112400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:17.115 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.112+0000 7f824359e640 1 -- 172.21.15.114:0/1493899087 >> 172.21.15.114:0/1493899087 conn(0x7f824406adb0 msgr2=0x7f824406fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:17.115 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.112+0000 7f824359e640 1 -- 172.21.15.114:0/1493899087 shutdown_connections 2024-09-13T18:27:17.115 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:17.112+0000 7f824359e640 1 -- 172.21.15.114:0/1493899087 wait complete. 2024-09-13T18:27:18.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:18 smithi204 ceph-mon[34777]: pgmap v147: 129 pgs: 129 active+clean; 16 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 45 MiB/s wr, 743 op/s 2024-09-13T18:27:18.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:18 smithi114 ceph-mon[30897]: pgmap v147: 129 pgs: 129 active+clean; 16 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 45 MiB/s wr, 743 op/s 2024-09-13T18:27:18.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:18 smithi134 ceph-mon[34283]: pgmap v147: 129 pgs: 129 active+clean; 16 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 45 MiB/s wr, 743 op/s 2024-09-13T18:27:20.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:20 smithi204 ceph-mon[34777]: pgmap v148: 129 pgs: 129 active+clean; 16 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 MiB/s wr, 640 op/s 2024-09-13T18:27:20.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:20 smithi114 ceph-mon[30897]: pgmap v148: 129 pgs: 129 active+clean; 16 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 MiB/s wr, 640 op/s 2024-09-13T18:27:20.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:20 smithi134 ceph-mon[34283]: pgmap v148: 129 pgs: 129 active+clean; 16 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 MiB/s wr, 640 op/s 2024-09-13T18:27:21.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:21 smithi204 ceph-mon[34777]: pgmap v149: 129 pgs: 129 active+clean; 16 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 59 MiB/s wr, 959 op/s 2024-09-13T18:27:21.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:21 smithi114 ceph-mon[30897]: pgmap v149: 129 pgs: 129 active+clean; 16 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 59 MiB/s wr, 959 op/s 2024-09-13T18:27:21.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:21 smithi134 ceph-mon[34283]: pgmap v149: 129 pgs: 129 active+clean; 16 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 59 MiB/s wr, 959 op/s 2024-09-13T18:27:22.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:22 smithi204 ceph-mon[34777]: scrub summary: idle 2024-09-13T18:27:22.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:22 smithi204 ceph-mon[34777]: scrub summary: idle 2024-09-13T18:27:22.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:22 smithi114 ceph-mon[30897]: scrub summary: idle 2024-09-13T18:27:22.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:22 smithi114 ceph-mon[30897]: scrub summary: idle 2024-09-13T18:27:22.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:22 smithi134 ceph-mon[34283]: scrub summary: idle 2024-09-13T18:27:22.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:22 smithi134 ceph-mon[34283]: scrub summary: idle 2024-09-13T18:27:23.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:23 smithi204 ceph-mon[34777]: scrub summary: idle 2024-09-13T18:27:23.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:23 smithi204 ceph-mon[34777]: pgmap v150: 129 pgs: 129 active+clean; 16 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 39 MiB/s wr, 643 op/s 2024-09-13T18:27:23.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:23 smithi204 ceph-mon[34777]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-13T18:27:23.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:23 smithi114 ceph-mon[30897]: scrub summary: idle 2024-09-13T18:27:23.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:23 smithi114 ceph-mon[30897]: pgmap v150: 129 pgs: 129 active+clean; 16 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 39 MiB/s wr, 643 op/s 2024-09-13T18:27:23.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:23 smithi114 ceph-mon[30897]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-13T18:27:23.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:23 smithi134 ceph-mon[34283]: scrub summary: idle 2024-09-13T18:27:23.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:23 smithi134 ceph-mon[34283]: pgmap v150: 129 pgs: 129 active+clean; 16 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 39 MiB/s wr, 643 op/s 2024-09-13T18:27:23.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:23 smithi134 ceph-mon[34283]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-13T18:27:25.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:25 smithi114 ceph-mon[30897]: pgmap v151: 129 pgs: 129 active+clean; 17 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 52 MiB/s wr, 847 op/s 2024-09-13T18:27:25.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:25 smithi134 ceph-mon[34283]: pgmap v151: 129 pgs: 129 active+clean; 17 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 52 MiB/s wr, 847 op/s 2024-09-13T18:27:25.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:25 smithi204 ceph-mon[34777]: pgmap v151: 129 pgs: 129 active+clean; 17 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 52 MiB/s wr, 847 op/s 2024-09-13T18:27:27.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:27 smithi114 ceph-mon[30897]: pgmap v152: 129 pgs: 129 active+clean; 17 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 45 MiB/s wr, 729 op/s 2024-09-13T18:27:27.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:27 smithi134 ceph-mon[34283]: pgmap v152: 129 pgs: 129 active+clean; 17 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 45 MiB/s wr, 729 op/s 2024-09-13T18:27:27.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:27 smithi204 ceph-mon[34777]: pgmap v152: 129 pgs: 129 active+clean; 17 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 45 MiB/s wr, 729 op/s 2024-09-13T18:27:29.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:29 smithi114 ceph-mon[30897]: pgmap v153: 129 pgs: 129 active+clean; 17 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 39 MiB/s wr, 633 op/s 2024-09-13T18:27:29.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:29 smithi134 ceph-mon[34283]: pgmap v153: 129 pgs: 129 active+clean; 17 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 39 MiB/s wr, 633 op/s 2024-09-13T18:27:29.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:29 smithi204 ceph-mon[34777]: pgmap v153: 129 pgs: 129 active+clean; 17 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 39 MiB/s wr, 633 op/s 2024-09-13T18:27:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.321+0000 7f4053f9c640 1 -- 172.21.15.114:0/4013387736 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f404c1074b0 msgr2=0x7f404c10e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.321+0000 7f4053f9c640 1 --2- 172.21.15.114:0/4013387736 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f404c1074b0 0x7f404c10e510 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f4044009ca0 tx=0x7f404402a8d0 comp rx=0 tx=0).stop 2024-09-13T18:27:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.321+0000 7f4053f9c640 1 -- 172.21.15.114:0/4013387736 shutdown_connections 2024-09-13T18:27:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.321+0000 7f4053f9c640 1 --2- 172.21.15.114:0/4013387736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f404c119400 0x7f404c11b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.321+0000 7f4053f9c640 1 --2- 172.21.15.114:0/4013387736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f404c107890 0x7f404c107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.321+0000 7f4053f9c640 1 --2- 172.21.15.114:0/4013387736 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f404c1074b0 0x7f404c10e510 secure :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f4044009ca0 tx=0x7f404402a8d0 comp rx=0 tx=0).stop 2024-09-13T18:27:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.321+0000 7f4053f9c640 1 -- 172.21.15.114:0/4013387736 >> 172.21.15.114:0/4013387736 conn(0x7f404c06af50 msgr2=0x7f404c06b360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.325+0000 7f4053f9c640 1 -- 172.21.15.114:0/4013387736 shutdown_connections 2024-09-13T18:27:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.325+0000 7f4053f9c640 1 -- 172.21.15.114:0/4013387736 wait complete. 2024-09-13T18:27:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.326+0000 7f4053f9c640 1 Processor -- start 2024-09-13T18:27:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.326+0000 7f4053f9c640 1 -- start start 2024-09-13T18:27:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.326+0000 7f4053f9c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f404c107890 0x7f404c118a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.326+0000 7f4053f9c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f404c119400 0x7f404c112ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.326+0000 7f4053f9c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f404c113020 0x7f404c1134a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.326+0000 7f4053f9c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f404c119290 con 0x7f404c119400 2024-09-13T18:27:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.326+0000 7f4053f9c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f404c124520 con 0x7f404c113020 2024-09-13T18:27:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.326+0000 7f4053f9c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f404c1246e0 con 0x7f404c107890 2024-09-13T18:27:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.326+0000 7f4052539640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f404c113020 0x7f404c1134a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.326+0000 7f4052539640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f404c113020 0x7f404c1134a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:44530/0 (socket says 172.21.15.114:44530) 2024-09-13T18:27:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.326+0000 7f4052539640 1 -- 172.21.15.114:0/2521787082 learned_addr learned my addr 172.21.15.114:0/2521787082 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.327+0000 7f4051d38640 1 --2- 172.21.15.114:0/2521787082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f404c107890 0x7f404c118a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.327+0000 7f4052539640 1 -- 172.21.15.114:0/2521787082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f404c107890 msgr2=0x7f404c118a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.327+0000 7f4052539640 1 --2- 172.21.15.114:0/2521787082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f404c107890 0x7f404c118a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.327+0000 7f4052539640 1 -- 172.21.15.114:0/2521787082 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f404c119400 msgr2=0x7f404c112ae0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:27:30.332 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.328+0000 7f4051537640 1 --2- 172.21.15.114:0/2521787082 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f404c119400 0x7f404c112ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:30.332 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.328+0000 7f4052539640 1 --2- 172.21.15.114:0/2521787082 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f404c119400 0x7f404c112ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:30.333 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.329+0000 7f4052539640 1 -- 172.21.15.114:0/2521787082 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4044009900 con 0x7f404c113020 2024-09-13T18:27:30.333 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.329+0000 7f4051537640 1 --2- 172.21.15.114:0/2521787082 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f404c119400 0x7f404c112ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:27:30.333 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.329+0000 7f4052539640 1 --2- 172.21.15.114:0/2521787082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f404c113020 0x7f404c1134a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f404800c7a0 tx=0x7f404800cc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:30.333 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.330+0000 7f4042ffd640 1 -- 172.21.15.114:0/2521787082 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4048007bb0 con 0x7f404c113020 2024-09-13T18:27:30.333 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.330+0000 7f4042ffd640 1 -- 172.21.15.114:0/2521787082 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4048007d10 con 0x7f404c113020 2024-09-13T18:27:30.333 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.330+0000 7f4053f9c640 1 -- 172.21.15.114:0/2521787082 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f404c1249c0 con 0x7f404c113020 2024-09-13T18:27:30.333 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.330+0000 7f4053f9c640 1 -- 172.21.15.114:0/2521787082 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f404c124ed0 con 0x7f404c113020 2024-09-13T18:27:30.334 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.330+0000 7f4042ffd640 1 -- 172.21.15.114:0/2521787082 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4048004cc0 con 0x7f404c113020 2024-09-13T18:27:30.340 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.334+0000 7f4056cc4740 1 Processor -- start 2024-09-13T18:27:30.340 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.334+0000 7f4042ffd640 1 -- 172.21.15.114:0/2521787082 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f4048004f60 con 0x7f404c113020 2024-09-13T18:27:30.340 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.335+0000 7f4042ffd640 1 --2- 172.21.15.114:0/2521787082 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f40380a1390 0x7f40380a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:30.340 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.335+0000 7f4042ffd640 1 -- 172.21.15.114:0/2521787082 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f404809a150 con 0x7f404c113020 2024-09-13T18:27:30.340 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.336+0000 7f4056cc4740 1 -- start start 2024-09-13T18:27:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.336+0000 7f4056cc4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f612058330 0x55f612058710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.336+0000 7f4056cc4740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f612058c50 0x55f612062fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.336+0000 7f4056cc4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6120634f0 0x55f6120658e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.336+0000 7f4056cc4740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f611fc3c30 con 0x55f612058c50 2024-09-13T18:27:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.336+0000 7f4056cc4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f611fc13c0 con 0x55f612058330 2024-09-13T18:27:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.336+0000 7f4056cc4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f611fbccb0 con 0x55f6120634f0 2024-09-13T18:27:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.338+0000 7f4052539640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6120634f0 0x55f6120658e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.338+0000 7f4052539640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6120634f0 0x55f6120658e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:33848/0 (socket says 172.21.15.114:33848) 2024-09-13T18:27:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.338+0000 7f4052539640 1 -- 172.21.15.114:0/2852014908 learned_addr learned my addr 172.21.15.114:0/2852014908 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.338+0000 7f4052539640 1 -- 172.21.15.114:0/2852014908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f612058330 msgr2=0x55f612058710 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:27:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.338+0000 7f4052539640 1 --2- 172.21.15.114:0/2852014908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f612058330 0x55f612058710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.338+0000 7f4052539640 1 -- 172.21.15.114:0/2852014908 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f612058c50 msgr2=0x55f612062fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.338+0000 7f4052539640 1 --2- 172.21.15.114:0/2852014908 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f612058c50 0x55f612062fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.338+0000 7f4052539640 1 -- 172.21.15.114:0/2852014908 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f611f63910 con 0x55f6120634f0 2024-09-13T18:27:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.340+0000 7f4052539640 1 --2- 172.21.15.114:0/2852014908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6120634f0 0x55f6120658e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4048096d70 tx=0x7f404809e730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d4a0187962ed213 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.340+0000 7f4051537640 1 --2- 172.21.15.114:0/2521787082 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f40380a1390 0x7f40380a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.340+0000 7f4037fff640 1 -- 172.21.15.114:0/2852014908 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404800a3f0 con 0x55f6120634f0 2024-09-13T18:27:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.340+0000 7f4037fff640 1 -- 172.21.15.114:0/2852014908 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4048007e80 con 0x55f6120634f0 2024-09-13T18:27:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.341+0000 7f4037fff640 1 -- 172.21.15.114:0/2852014908 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40480a1ed0 con 0x55f6120634f0 2024-09-13T18:27:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.341+0000 7f4056cc4740 1 -- 172.21.15.114:0/2852014908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6120634f0 msgr2=0x55f6120658e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.341+0000 7f4056cc4740 1 --2- 172.21.15.114:0/2852014908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6120634f0 0x55f6120658e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4048096d70 tx=0x7f404809e730 comp rx=0 tx=0).stop 2024-09-13T18:27:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.341+0000 7f4051537640 1 --2- 172.21.15.114:0/2521787082 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f40380a1390 0x7f40380a3850 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f403c006fd0 tx=0x7f403c008040 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.341+0000 7f4056cc4740 1 -- 172.21.15.114:0/2852014908 shutdown_connections 2024-09-13T18:27:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.341+0000 7f4056cc4740 1 --2- 172.21.15.114:0/2852014908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6120634f0 0x55f6120658e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.341+0000 7f4056cc4740 1 --2- 172.21.15.114:0/2852014908 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f612058c50 0x55f612062fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.341+0000 7f4056cc4740 1 --2- 172.21.15.114:0/2852014908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f612058330 0x55f612058710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.342+0000 7f4056cc4740 1 -- 172.21.15.114:0/2852014908 >> 172.21.15.114:0/2852014908 conn(0x55f6120548c0 msgr2=0x55f612054ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.344+0000 7f4056cc4740 1 -- 172.21.15.114:0/2852014908 shutdown_connections 2024-09-13T18:27:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.344+0000 7f4056cc4740 1 -- 172.21.15.114:0/2852014908 wait complete. 2024-09-13T18:27:30.347 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.345+0000 7f4056cc4740 1 -- start start 2024-09-13T18:27:30.347 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.345+0000 7f4056cc4740 1 Processor -- start 2024-09-13T18:27:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.348+0000 7f4056cc4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f612058330 0x55f612062ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.348+0000 7f4056cc4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f612058c50 0x55f612060160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.348+0000 7f4056cc4740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f6120634f0 0x55f612060750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.348+0000 7f4056cc4740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f611f7ae80 con 0x55f6120634f0 2024-09-13T18:27:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.348+0000 7f4056cc4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f611f7ab20 con 0x55f612058330 2024-09-13T18:27:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.348+0000 7f4056cc4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f611f40260 con 0x55f612058c50 2024-09-13T18:27:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.348+0000 7f4056cc4740 10 client.0 _ll_register_callbacks cb 0x55f611f75df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:27:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.348+0000 7f4051d38640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f612058330 0x55f612062ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.348+0000 7f4051d38640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f612058330 0x55f612062ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:44548/0 (socket says 172.21.15.114:44548) 2024-09-13T18:27:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.348+0000 7f4051d38640 1 -- 172.21.15.114:0/424261352 learned_addr learned my addr 172.21.15.114:0/424261352 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.349+0000 7f4052539640 1 --2- 172.21.15.114:0/424261352 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f6120634f0 0x55f612060750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.349+0000 7f4051d38640 1 -- 172.21.15.114:0/424261352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f612058c50 msgr2=0x55f612060160 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:27:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.349+0000 7f4051d38640 1 --2- 172.21.15.114:0/424261352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f612058c50 0x55f612060160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.349+0000 7f4051d38640 1 -- 172.21.15.114:0/424261352 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f6120634f0 msgr2=0x55f612060750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.349+0000 7f4051d38640 1 --2- 172.21.15.114:0/424261352 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f6120634f0 0x55f612060750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.349+0000 7f4051d38640 1 -- 172.21.15.114:0/424261352 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f404c124ed0 con 0x55f612058330 2024-09-13T18:27:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.349+0000 7f4051d38640 1 --2- 172.21.15.114:0/424261352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f612058330 0x55f612062ec0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f404402aec0 tx=0x7f4044005fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf195052516ed445 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.350+0000 7f40357fa640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:27:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.351+0000 7f40357fa640 1 -- 172.21.15.114:0/424261352 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4044003b70 con 0x55f612058330 2024-09-13T18:27:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.351+0000 7f4056cc4740 1 -- 172.21.15.114:0/424261352 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f611fb05d0 con 0x55f612058330 2024-09-13T18:27:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.351+0000 7f40357fa640 1 -- 172.21.15.114:0/424261352 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f4044003cd0 con 0x55f612058330 2024-09-13T18:27:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.359+0000 7f40357fa640 1 -- 172.21.15.114:0/424261352 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4044003e90 con 0x55f612058330 2024-09-13T18:27:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.359+0000 7f4056cc4740 10 client.26555 fetch_fsmap learned FSMap version 34 2024-09-13T18:27:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.359+0000 7f4056cc4740 1 -- 172.21.15.114:0/424261352 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55f611fbb330 con 0x55f612058330 2024-09-13T18:27:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.361+0000 7f40357fa640 1 -- 172.21.15.114:0/424261352 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f611fb05d0 con 0x55f612058330 2024-09-13T18:27:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.361+0000 7f40357fa640 1 -- 172.21.15.114:0/424261352 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f404406d080 con 0x55f612058330 2024-09-13T18:27:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.361+0000 7f4056cc4740 10 client.26555 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:27:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.361+0000 7f4056cc4740 10 client.26555 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:27:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.361+0000 7f4056cc4740 20 client.26555 populate_metadata read hostname 'smithi114' 2024-09-13T18:27:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.361+0000 7f4056cc4740 1 --2- 172.21.15.114:0/424261352 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f612072e20 0x55f612075240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.361+0000 7f4056cc4740 4 client.26555 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:27:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.361+0000 7f4056cc4740 1 -- 172.21.15.114:0/424261352 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f611cb0300 con 0x55f612072e20 2024-09-13T18:27:30.367 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.365+0000 7f4051537640 1 --2- 172.21.15.114:0/424261352 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f612072e20 0x55f612075240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:30.375 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.373+0000 7f40357fa640 10 client.26555 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:27:30.375 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.373+0000 7f4051537640 1 --2- 172.21.15.114:0/424261352 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f612072e20 0x55f612075240 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34f891138217a3c5 server_cookie=77dad635a0e150cc in_seq=0 out_seq=0 2024-09-13T18:27:30.378 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.376+0000 7f40357fa640 1 -- 172.21.15.114:0/424261352 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55f611cb0300 con 0x55f612072e20 2024-09-13T18:27:30.378 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.376+0000 7f40357fa640 10 client.26555 handle_command_reply: tid=0 2024-09-13T18:27:30.379 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.376+0000 7f4056cc4740 1 client.26555 shutdown 2024-09-13T18:27:30.386 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.384+0000 7f4056cc4740 1 -- 172.21.15.114:0/424261352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f612058330 msgr2=0x55f612062ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:30.386 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.384+0000 7f4056cc4740 1 --2- 172.21.15.114:0/424261352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f612058330 0x55f612062ec0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f404402aec0 tx=0x7f4044005fb0 comp rx=0 tx=0).stop 2024-09-13T18:27:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.386+0000 7f4056cc4740 1 -- 172.21.15.114:0/424261352 shutdown_connections 2024-09-13T18:27:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.386+0000 7f4056cc4740 1 --2- 172.21.15.114:0/424261352 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f612072e20 0x55f612075240 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.386+0000 7f4056cc4740 1 --2- 172.21.15.114:0/424261352 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f6120634f0 0x55f612060750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.386+0000 7f4056cc4740 1 --2- 172.21.15.114:0/424261352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f612058c50 0x55f612060160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.386+0000 7f4056cc4740 1 --2- 172.21.15.114:0/424261352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f612058330 0x55f612062ec0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.386+0000 7f4056cc4740 1 -- 172.21.15.114:0/424261352 >> 172.21.15.114:0/424261352 conn(0x55f6120548c0 msgr2=0x55f612053b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.387+0000 7f40357fa640 0 client.26555 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:27:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.390+0000 7f4056cc4740 1 -- 172.21.15.114:0/424261352 shutdown_connections 2024-09-13T18:27:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.390+0000 7f4056cc4740 1 -- 172.21.15.114:0/424261352 wait complete. 2024-09-13T18:27:30.394 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.392+0000 7f4056cc4740 20 client.26555 trim_cache size 0 max 16384 2024-09-13T18:27:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.417+0000 7f4056cc4740 1 Processor -- start 2024-09-13T18:27:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.417+0000 7f4056cc4740 1 -- start start 2024-09-13T18:27:30.420 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.417+0000 7f4056cc4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f612058330 0x55f6120629d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:30.420 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.417+0000 7f4056cc4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f612058c50 0x55f61204fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:30.420 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.417+0000 7f4051d38640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f612058330 0x55f6120629d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:30.420 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.417+0000 7f4051d38640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f612058330 0x55f6120629d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:44556/0 (socket says 172.21.15.114:44556) 2024-09-13T18:27:30.420 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.418+0000 7f4051d38640 1 -- 172.21.15.114:0/4282911108 learned_addr learned my addr 172.21.15.114:0/4282911108 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:30.420 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.418+0000 7f4056cc4740 1 --2- 172.21.15.114:0/4282911108 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f6120634f0 0x55f61204ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:30.421 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.418+0000 7f4056cc4740 1 -- 172.21.15.114:0/4282911108 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f612061300 con 0x55f6120634f0 2024-09-13T18:27:30.421 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.418+0000 7f4056cc4740 1 -- 172.21.15.114:0/4282911108 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f612055550 con 0x55f612058330 2024-09-13T18:27:30.421 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.418+0000 7f4056cc4740 1 -- 172.21.15.114:0/4282911108 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f611fb0600 con 0x55f612058c50 2024-09-13T18:27:30.421 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.418+0000 7f4052539640 1 --2- 172.21.15.114:0/4282911108 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f6120634f0 0x55f61204ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:30.422 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.418+0000 7f4051d38640 1 -- 172.21.15.114:0/4282911108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f612058c50 msgr2=0x55f61204fa10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:27:30.422 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.418+0000 7f4051d38640 1 --2- 172.21.15.114:0/4282911108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f612058c50 0x55f61204fa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:30.422 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.418+0000 7f4051d38640 1 -- 172.21.15.114:0/4282911108 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f6120634f0 msgr2=0x55f61204ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:30.422 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.418+0000 7f4051d38640 1 --2- 172.21.15.114:0/4282911108 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f6120634f0 0x55f61204ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:30.422 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.418+0000 7f4051d38640 1 -- 172.21.15.114:0/4282911108 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f611f5d300 con 0x55f612058330 2024-09-13T18:27:30.422 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.419+0000 7f4052539640 1 --2- 172.21.15.114:0/4282911108 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f6120634f0 0x55f61204ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:27:30.422 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.419+0000 7f4051d38640 1 --2- 172.21.15.114:0/4282911108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f612058330 0x55f6120629d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4044033040 tx=0x7f4044009ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b96e81672646105 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:30.422 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.420+0000 7f4040ff9640 1 -- 172.21.15.114:0/4282911108 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4044004d40 con 0x55f612058330 2024-09-13T18:27:30.422 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.420+0000 7f4040ff9640 1 -- 172.21.15.114:0/4282911108 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4044070040 con 0x55f612058330 2024-09-13T18:27:30.422 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.420+0000 7f4040ff9640 1 -- 172.21.15.114:0/4282911108 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404403a700 con 0x55f612058330 2024-09-13T18:27:30.423 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.421+0000 7f4056cc4740 1 -- 172.21.15.114:0/4282911108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f612058330 msgr2=0x55f6120629d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:30.423 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.421+0000 7f4056cc4740 1 --2- 172.21.15.114:0/4282911108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f612058330 0x55f6120629d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4044033040 tx=0x7f4044009ba0 comp rx=0 tx=0).stop 2024-09-13T18:27:30.423 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.421+0000 7f4056cc4740 1 -- 172.21.15.114:0/4282911108 shutdown_connections 2024-09-13T18:27:30.424 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.421+0000 7f4056cc4740 1 --2- 172.21.15.114:0/4282911108 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f6120634f0 0x55f61204ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:30.424 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.421+0000 7f4056cc4740 1 --2- 172.21.15.114:0/4282911108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f612058c50 0x55f61204fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:30.424 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.421+0000 7f4056cc4740 1 --2- 172.21.15.114:0/4282911108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f612058330 0x55f6120629d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:30.424 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.421+0000 7f4056cc4740 1 -- 172.21.15.114:0/4282911108 >> 172.21.15.114:0/4282911108 conn(0x55f6120548c0 msgr2=0x55f61205ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:30.424 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.421+0000 7f4056cc4740 1 -- 172.21.15.114:0/4282911108 shutdown_connections 2024-09-13T18:27:30.424 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.421+0000 7f4056cc4740 1 -- 172.21.15.114:0/4282911108 wait complete. 2024-09-13T18:27:30.425 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.422+0000 7f4056cc4740 1 -- start start 2024-09-13T18:27:30.425 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.422+0000 7f4056cc4740 1 Processor -- start 2024-09-13T18:27:30.425 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.422+0000 7f4056cc4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f612058330 0x55f61206a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:30.425 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.423+0000 7f4056cc4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f612058c50 0x55f6120665a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:30.425 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.423+0000 7f4051d38640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f612058330 0x55f61206a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:30.425 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.423+0000 7f4051d38640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f612058330 0x55f61206a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:33878/0 (socket says 172.21.15.114:33878) 2024-09-13T18:27:30.425 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.423+0000 7f4051d38640 1 -- 172.21.15.114:0/770832651 learned_addr learned my addr 172.21.15.114:0/770832651 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.423+0000 7f4051537640 1 --2- 172.21.15.114:0/770832651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f612058c50 0x55f6120665a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.424+0000 7f4056cc4740 1 --2- 172.21.15.114:0/770832651 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f6120634f0 0x55f612066ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.424+0000 7f4052539640 1 --2- 172.21.15.114:0/770832651 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f6120634f0 0x55f612066ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.424+0000 7f4056cc4740 1 -- 172.21.15.114:0/770832651 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f612067020 con 0x55f6120634f0 2024-09-13T18:27:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.424+0000 7f4056cc4740 1 -- 172.21.15.114:0/770832651 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f612067190 con 0x55f612058c50 2024-09-13T18:27:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.424+0000 7f4056cc4740 1 -- 172.21.15.114:0/770832651 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f612067350 con 0x55f612058330 2024-09-13T18:27:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.425+0000 7f4056cc4740 10 client.0 _ll_register_callbacks cb 0x7f404801b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:27:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.425+0000 7f4051d38640 1 -- 172.21.15.114:0/770832651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f612058c50 msgr2=0x55f6120665a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.425+0000 7f4051d38640 1 --2- 172.21.15.114:0/770832651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f612058c50 0x55f6120665a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.425+0000 7f4051d38640 1 -- 172.21.15.114:0/770832651 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f6120634f0 msgr2=0x55f612066ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.425+0000 7f4051d38640 1 --2- 172.21.15.114:0/770832651 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f6120634f0 0x55f612066ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.425+0000 7f4051d38640 1 -- 172.21.15.114:0/770832651 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f611fbb330 con 0x55f612058330 2024-09-13T18:27:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.425+0000 7f4052539640 1 --2- 172.21.15.114:0/770832651 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f6120634f0 0x55f612066ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:27:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.426+0000 7f4051d38640 1 --2- 172.21.15.114:0/770832651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f612058330 0x55f61206a100 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4044004f00 tx=0x7f4044009360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8626b35c14fa41be server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.426+0000 7f40357fa640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:27:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.426+0000 7f40357fa640 1 -- 172.21.15.114:0/770832651 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4044004800 con 0x55f612058330 2024-09-13T18:27:30.430 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.426+0000 7f40357fa640 1 -- 172.21.15.114:0/770832651 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f4044030c00 con 0x55f612058330 2024-09-13T18:27:30.430 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.426+0000 7f4056cc4740 1 -- 172.21.15.114:0/770832651 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f611fe5680 con 0x55f612058330 2024-09-13T18:27:30.430 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.427+0000 7f40357fa640 1 -- 172.21.15.114:0/770832651 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f404403aa30 con 0x55f612058330 2024-09-13T18:27:30.431 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.427+0000 7f4056cc4740 10 client.26593 fetch_fsmap learned FSMap version 34 2024-09-13T18:27:30.431 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.427+0000 7f4056cc4740 1 -- 172.21.15.114:0/770832651 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55f611f828e0 con 0x55f612058330 2024-09-13T18:27:30.431 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.427+0000 7f40357fa640 1 -- 172.21.15.114:0/770832651 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4044031460 con 0x55f612058330 2024-09-13T18:27:30.431 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.428+0000 7f40357fa640 1 -- 172.21.15.114:0/770832651 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f404406d040 con 0x55f612058330 2024-09-13T18:27:30.431 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.428+0000 7f4056cc4740 10 client.26593 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:27:30.431 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.428+0000 7f4056cc4740 10 client.26593 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:27:30.431 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.428+0000 7f4056cc4740 20 client.26593 populate_metadata read hostname 'smithi114' 2024-09-13T18:27:30.431 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.428+0000 7f4056cc4740 1 --2- 172.21.15.114:0/770832651 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f6120ca330 0x55f6120cc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:30.431 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.428+0000 7f4056cc4740 4 client.26593 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T18:27:30.431 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.428+0000 7f4056cc4740 1 -- 172.21.15.114:0/770832651 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f611fe5680 con 0x55f6120ca330 2024-09-13T18:27:30.437 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.435+0000 7f4051537640 1 --2- 172.21.15.114:0/770832651 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f6120ca330 0x55f6120cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:30.455 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.450+0000 7f4051537640 1 --2- 172.21.15.114:0/770832651 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f6120ca330 0x55f6120cc720 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdd102e9d2ec6c21 server_cookie=4d61294496355145 in_seq=0 out_seq=0 2024-09-13T18:27:30.455 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.450+0000 7f40357fa640 10 client.26593 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:27:30.476 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.471+0000 7f40357fa640 1 -- 172.21.15.114:0/770832651 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f611fe5680 con 0x55f6120ca330 2024-09-13T18:27:30.476 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.471+0000 7f40357fa640 10 client.26593 handle_command_reply: tid=0 2024-09-13T18:27:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.478+0000 7f4056cc4740 1 client.26593 shutdown 2024-09-13T18:27:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.479+0000 7f4056cc4740 1 -- 172.21.15.114:0/770832651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f612058330 msgr2=0x55f61206a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.479+0000 7f4056cc4740 1 --2- 172.21.15.114:0/770832651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f612058330 0x55f61206a100 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4044004f00 tx=0x7f4044009360 comp rx=0 tx=0).stop 2024-09-13T18:27:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.481+0000 7f4056cc4740 1 -- 172.21.15.114:0/770832651 shutdown_connections 2024-09-13T18:27:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.481+0000 7f4056cc4740 1 --2- 172.21.15.114:0/770832651 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f6120ca330 0x55f6120cc720 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.481+0000 7f4056cc4740 1 --2- 172.21.15.114:0/770832651 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f6120634f0 0x55f612066ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.481+0000 7f40357fa640 0 client.26593 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:27:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.481+0000 7f4056cc4740 1 --2- 172.21.15.114:0/770832651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f612058c50 0x55f6120665a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.481+0000 7f4056cc4740 1 --2- 172.21.15.114:0/770832651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f612058330 0x55f61206a100 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.481+0000 7f4056cc4740 1 -- 172.21.15.114:0/770832651 >> 172.21.15.114:0/770832651 conn(0x55f6120548c0 msgr2=0x55f61205e970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.481+0000 7f4056cc4740 1 -- 172.21.15.114:0/770832651 shutdown_connections 2024-09-13T18:27:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.482+0000 7f4056cc4740 1 -- 172.21.15.114:0/770832651 wait complete. 2024-09-13T18:27:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.484+0000 7f4056cc4740 20 client.26593 trim_cache size 0 max 16384 2024-09-13T18:27:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-13T18:27:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-13T18:27:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-13T18:27:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-13T18:27:30.491 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.486+0000 7f4053f9c640 1 -- 172.21.15.114:0/2521787082 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f40380a1390 msgr2=0x7f40380a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:30.491 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.486+0000 7f4053f9c640 1 --2- 172.21.15.114:0/2521787082 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f40380a1390 0x7f40380a3850 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f403c006fd0 tx=0x7f403c008040 comp rx=0 tx=0).stop 2024-09-13T18:27:30.491 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.486+0000 7f4053f9c640 1 -- 172.21.15.114:0/2521787082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f404c113020 msgr2=0x7f404c1134a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:30.491 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.486+0000 7f4053f9c640 1 --2- 172.21.15.114:0/2521787082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f404c113020 0x7f404c1134a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f404800c7a0 tx=0x7f404800cc60 comp rx=0 tx=0).stop 2024-09-13T18:27:30.491 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.487+0000 7f4053f9c640 1 -- 172.21.15.114:0/2521787082 shutdown_connections 2024-09-13T18:27:30.491 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.487+0000 7f4053f9c640 1 --2- 172.21.15.114:0/2521787082 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f40380a1390 0x7f40380a3850 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:30.491 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.487+0000 7f4053f9c640 1 --2- 172.21.15.114:0/2521787082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f404c113020 0x7f404c1134a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:30.491 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.487+0000 7f4053f9c640 1 --2- 172.21.15.114:0/2521787082 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f404c119400 0x7f404c112ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:30.491 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.487+0000 7f4053f9c640 1 --2- 172.21.15.114:0/2521787082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f404c107890 0x7f404c118a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:30.491 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.487+0000 7f4053f9c640 1 -- 172.21.15.114:0/2521787082 >> 172.21.15.114:0/2521787082 conn(0x7f404c06af50 msgr2=0x7f404c06ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:30.495 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.493+0000 7f4053f9c640 1 -- 172.21.15.114:0/2521787082 shutdown_connections 2024-09-13T18:27:30.495 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:30.493+0000 7f4053f9c640 1 -- 172.21.15.114:0/2521787082 wait complete. 2024-09-13T18:27:31.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:31 smithi114 ceph-mon[30897]: pgmap v154: 129 pgs: 129 active+clean; 17 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 58 MiB/s wr, 958 op/s 2024-09-13T18:27:31.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:31 smithi134 ceph-mon[34283]: pgmap v154: 129 pgs: 129 active+clean; 17 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 58 MiB/s wr, 958 op/s 2024-09-13T18:27:31.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:31 smithi204 ceph-mon[34777]: pgmap v154: 129 pgs: 129 active+clean; 17 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 58 MiB/s wr, 958 op/s 2024-09-13T18:27:33.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:33 smithi114 ceph-mon[30897]: pgmap v155: 129 pgs: 129 active+clean; 17 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 39 MiB/s wr, 638 op/s 2024-09-13T18:27:33.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:33 smithi134 ceph-mon[34283]: pgmap v155: 129 pgs: 129 active+clean; 17 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 39 MiB/s wr, 638 op/s 2024-09-13T18:27:33.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:33 smithi204 ceph-mon[34777]: pgmap v155: 129 pgs: 129 active+clean; 17 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 39 MiB/s wr, 638 op/s 2024-09-13T18:27:35.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:35 smithi114 ceph-mon[30897]: pgmap v156: 129 pgs: 129 active+clean; 17 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 51 MiB/s wr, 844 op/s 2024-09-13T18:27:35.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:35 smithi134 ceph-mon[34283]: pgmap v156: 129 pgs: 129 active+clean; 17 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 51 MiB/s wr, 844 op/s 2024-09-13T18:27:35.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:35 smithi204 ceph-mon[34777]: pgmap v156: 129 pgs: 129 active+clean; 17 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 51 MiB/s wr, 844 op/s 2024-09-13T18:27:37.373 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T18:27:37.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.583+0000 7fa6bcf8a640 1 -- 172.21.15.114:0/3681467320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6b8119400 msgr2=0x7fa6b811b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:37.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.583+0000 7fa6bcf8a640 1 --2- 172.21.15.114:0/3681467320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6b8119400 0x7fa6b811b7f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac01a910 tx=0x7fa6ac03b470 comp rx=0 tx=0).stop 2024-09-13T18:27:37.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.584+0000 7fa6bcf8a640 1 -- 172.21.15.114:0/3681467320 shutdown_connections 2024-09-13T18:27:37.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.584+0000 7fa6bcf8a640 1 --2- 172.21.15.114:0/3681467320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6b8119400 0x7fa6b811b7f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:37.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.584+0000 7fa6bcf8a640 1 --2- 172.21.15.114:0/3681467320 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6b8107920 0x7fa6b8107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:37.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.584+0000 7fa6bcf8a640 1 --2- 172.21.15.114:0/3681467320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6b8107540 0x7fa6b810e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:37.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.584+0000 7fa6bcf8a640 1 -- 172.21.15.114:0/3681467320 >> 172.21.15.114:0/3681467320 conn(0x7fa6b806afe0 msgr2=0x7fa6b806b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:37.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.584+0000 7fa6bcf8a640 1 -- 172.21.15.114:0/3681467320 shutdown_connections 2024-09-13T18:27:37.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.585+0000 7fa6bcf8a640 1 -- 172.21.15.114:0/3681467320 wait complete. 2024-09-13T18:27:37.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.587+0000 7fa6bcf8a640 1 Processor -- start 2024-09-13T18:27:37.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.589+0000 7fa6bcf8a640 1 -- start start 2024-09-13T18:27:37.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.593+0000 7fa6bcf8a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6b8107540 0x7fa6b81188a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:37.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.593+0000 7fa6bcf8a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6b8107920 0x7fa6b8118de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:37.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.593+0000 7fa6bcf8a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6b8119400 0x7fa6b81149c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:37.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.593+0000 7fa6bcf8a640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b8115150 con 0x7fa6b8119400 2024-09-13T18:27:37.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.593+0000 7fa6bcf8a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b81152c0 con 0x7fa6b8107920 2024-09-13T18:27:37.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.593+0000 7fa6bcf8a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b8115480 con 0x7fa6b8107540 2024-09-13T18:27:37.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.594+0000 7fa6b5d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6b8107920 0x7fa6b8118de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:37.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.594+0000 7fa6b5d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6b8107920 0x7fa6b8118de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:36588/0 (socket says 172.21.15.114:36588) 2024-09-13T18:27:37.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.594+0000 7fa6b5d9b640 1 -- 172.21.15.114:0/2555179642 learned_addr learned my addr 172.21.15.114:0/2555179642 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:37.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.595+0000 7fa6b5d9b640 1 -- 172.21.15.114:0/2555179642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6b8107540 msgr2=0x7fa6b81188a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:37.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.595+0000 7fa6b5d9b640 1 --2- 172.21.15.114:0/2555179642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6b8107540 0x7fa6b81188a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:37.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.595+0000 7fa6b5d9b640 1 -- 172.21.15.114:0/2555179642 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6b8119400 msgr2=0x7fa6b81149c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:37.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.595+0000 7fa6b5d9b640 1 --2- 172.21.15.114:0/2555179642 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6b8119400 0x7fa6b81149c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:37.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.595+0000 7fa6b5d9b640 1 -- 172.21.15.114:0/2555179642 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ac01a570 con 0x7fa6b8107920 2024-09-13T18:27:37.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.595+0000 7fa6b5d9b640 1 --2- 172.21.15.114:0/2555179642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6b8107920 0x7fa6b8118de0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa6b000ba50 tx=0x7fa6b000bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:37.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.595+0000 7fa6a77fe640 1 -- 172.21.15.114:0/2555179642 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b000cbe0 con 0x7fa6b8107920 2024-09-13T18:27:37.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.596+0000 7fa6a77fe640 1 -- 172.21.15.114:0/2555179642 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa6b000cd40 con 0x7fa6b8107920 2024-09-13T18:27:37.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.597+0000 7fa6a77fe640 1 -- 172.21.15.114:0/2555179642 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b0007680 con 0x7fa6b8107920 2024-09-13T18:27:37.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.598+0000 7fa6bcf8a640 1 -- 172.21.15.114:0/2555179642 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6b8115760 con 0x7fa6b8107920 2024-09-13T18:27:37.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.598+0000 7fa6bcf8a640 1 -- 172.21.15.114:0/2555179642 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6b81a9590 con 0x7fa6b8107920 2024-09-13T18:27:37.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.602+0000 7fa6bfcaf740 1 Processor -- start 2024-09-13T18:27:37.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.603+0000 7fa6bfcaf740 1 -- start start 2024-09-13T18:27:37.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.603+0000 7fa6bfcaf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b95927e10 0x562b959281f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:37.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.603+0000 7fa6bfcaf740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b95928730 0x562b95932a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:37.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.603+0000 7fa6bfcaf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b95932fd0 0x562b959353c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:37.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.603+0000 7fa6bfcaf740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b95892f20 con 0x562b95928730 2024-09-13T18:27:37.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.603+0000 7fa6bfcaf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b958906b0 con 0x562b95932fd0 2024-09-13T18:27:37.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.603+0000 7fa6bfcaf740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b9588bfa0 con 0x562b95927e10 2024-09-13T18:27:37.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.604+0000 7fa6b659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b95928730 0x562b95932a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:37.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.604+0000 7fa6b659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b95928730 0x562b95932a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36076/0 (socket says 172.21.15.114:36076) 2024-09-13T18:27:37.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.605+0000 7fa6b659c640 1 -- 172.21.15.114:0/3494011309 learned_addr learned my addr 172.21.15.114:0/3494011309 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:37.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.606+0000 7fa6b6d9d640 1 --2- 172.21.15.114:0/3494011309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b95927e10 0x562b959281f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:37.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:37 smithi134 ceph-mon[34283]: pgmap v157: 129 pgs: 129 active+clean; 17 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 45 MiB/s wr, 759 op/s 2024-09-13T18:27:37.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.619+0000 7fa6b5d9b640 1 --2- 172.21.15.114:0/3494011309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b95932fd0 0x562b959353c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:37.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.620+0000 7fa6b659c640 1 -- 172.21.15.114:0/3494011309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b95927e10 msgr2=0x562b959281f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:37.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.620+0000 7fa6b659c640 1 --2- 172.21.15.114:0/3494011309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b95927e10 0x562b959281f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:37.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.620+0000 7fa6b659c640 1 -- 172.21.15.114:0/3494011309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b95932fd0 msgr2=0x562b959353c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:37.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.620+0000 7fa6b659c640 1 --2- 172.21.15.114:0/3494011309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b95932fd0 0x562b959353c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:37.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.620+0000 7fa6b659c640 1 -- 172.21.15.114:0/3494011309 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b95846840 con 0x562b95928730 2024-09-13T18:27:37.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.621+0000 7fa6a77fe640 1 -- 172.21.15.114:0/2555179642 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fa6b0014440 con 0x7fa6b8107920 2024-09-13T18:27:37.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.628+0000 7fa6a77fe640 1 --2- 172.21.15.114:0/2555179642 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa6a00a1390 0x7fa6a00a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:37.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.629+0000 7fa6b659c640 1 --2- 172.21.15.114:0/3494011309 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b95928730 0x562b95932a90 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fa6a80097c0 tx=0x7fa6a8009c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9ba81b1804a0d19 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:37.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.629+0000 7fa6a77fe640 1 -- 172.21.15.114:0/2555179642 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fa6b00cc550 con 0x7fa6b8107920 2024-09-13T18:27:37.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.631+0000 7fa6b6d9d640 1 --2- 172.21.15.114:0/2555179642 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa6a00a1390 0x7fa6a00a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:37.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.632+0000 7fa6a4ff9640 1 -- 172.21.15.114:0/3494011309 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a800bad0 con 0x562b95928730 2024-09-13T18:27:37.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.632+0000 7fa6a4ff9640 1 -- 172.21.15.114:0/3494011309 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa6a800bc30 con 0x562b95928730 2024-09-13T18:27:37.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.632+0000 7fa6a4ff9640 1 -- 172.21.15.114:0/3494011309 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a80105e0 con 0x562b95928730 2024-09-13T18:27:37.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.633+0000 7fa6bfcaf740 1 -- 172.21.15.114:0/3494011309 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b95928730 msgr2=0x562b95932a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:37.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.633+0000 7fa6bfcaf740 1 --2- 172.21.15.114:0/3494011309 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b95928730 0x562b95932a90 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fa6a80097c0 tx=0x7fa6a8009c80 comp rx=0 tx=0).stop 2024-09-13T18:27:37.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.633+0000 7fa6b6d9d640 1 --2- 172.21.15.114:0/2555179642 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa6a00a1390 0x7fa6a00a3850 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac005c20 tx=0x7fa6ac03b970 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:37.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.635+0000 7fa6bfcaf740 1 -- 172.21.15.114:0/3494011309 shutdown_connections 2024-09-13T18:27:37.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.635+0000 7fa6bfcaf740 1 --2- 172.21.15.114:0/3494011309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b95932fd0 0x562b959353c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:37.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.635+0000 7fa6bfcaf740 1 --2- 172.21.15.114:0/3494011309 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b95928730 0x562b95932a90 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:37.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.635+0000 7fa6bfcaf740 1 --2- 172.21.15.114:0/3494011309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b95927e10 0x562b959281f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:37.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.635+0000 7fa6bfcaf740 1 -- 172.21.15.114:0/3494011309 >> 172.21.15.114:0/3494011309 conn(0x562b95924380 msgr2=0x562b95924760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:37.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.635+0000 7fa6bfcaf740 1 -- 172.21.15.114:0/3494011309 shutdown_connections 2024-09-13T18:27:37.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.638+0000 7fa6bfcaf740 1 -- 172.21.15.114:0/3494011309 wait complete. 2024-09-13T18:27:37.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.638+0000 7fa6bfcaf740 1 -- start start 2024-09-13T18:27:37.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.638+0000 7fa6bfcaf740 1 Processor -- start 2024-09-13T18:27:37.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.639+0000 7fa6bfcaf740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b95927e10 0x562b95932a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:37.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.639+0000 7fa6bfcaf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b95928730 0x562b9592fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:37.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.639+0000 7fa6bfcaf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b95932fd0 0x562b95930470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:37.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.639+0000 7fa6bfcaf740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b9582d2a0 con 0x562b95927e10 2024-09-13T18:27:37.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.639+0000 7fa6bfcaf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b95564430 con 0x562b95928730 2024-09-13T18:27:37.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.639+0000 7fa6bfcaf740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b9580f680 con 0x562b95932fd0 2024-09-13T18:27:37.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.639+0000 7fa6bfcaf740 10 client.0 _ll_register_callbacks cb 0x562b95885610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:27:37.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.641+0000 7fa6b6d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b95932fd0 0x562b95930470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:37.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.641+0000 7fa6b6d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b95932fd0 0x562b95930470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:53740/0 (socket says 172.21.15.114:53740) 2024-09-13T18:27:37.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.641+0000 7fa6b6d9d640 1 -- 172.21.15.114:0/2852428311 learned_addr learned my addr 172.21.15.114:0/2852428311 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:37.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.641+0000 7fa6b5d9b640 1 --2- 172.21.15.114:0/2852428311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b95928730 0x562b9592fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:37.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.641+0000 7fa6b6d9d640 1 -- 172.21.15.114:0/2852428311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b95928730 msgr2=0x562b9592fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:37.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.641+0000 7fa6b6d9d640 1 --2- 172.21.15.114:0/2852428311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b95928730 0x562b9592fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:37.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.641+0000 7fa6b6d9d640 1 -- 172.21.15.114:0/2852428311 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b95927e10 msgr2=0x562b95932a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:37.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.641+0000 7fa6b6d9d640 1 --2- 172.21.15.114:0/2852428311 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b95927e10 0x562b95932a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:37.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.641+0000 7fa6b6d9d640 1 -- 172.21.15.114:0/2852428311 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6a8009420 con 0x562b95932fd0 2024-09-13T18:27:37.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.642+0000 7fa6b6d9d640 1 --2- 172.21.15.114:0/2852428311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b95932fd0 0x562b95930470 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac004500 tx=0x7fa6ac01a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2cf710c96d37bd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:37.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.642+0000 7fa69dffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:27:37.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.643+0000 7fa69dffb640 1 -- 172.21.15.114:0/2852428311 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac004d20 con 0x562b95932fd0 2024-09-13T18:27:37.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.643+0000 7fa69dffb640 1 -- 172.21.15.114:0/2852428311 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fa6ac053070 con 0x562b95932fd0 2024-09-13T18:27:37.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.644+0000 7fa6bfcaf740 1 -- 172.21.15.114:0/2852428311 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562b9557f4e0 con 0x562b95932fd0 2024-09-13T18:27:37.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.648+0000 7fa69dffb640 1 -- 172.21.15.114:0/2852428311 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6ac004e80 con 0x562b95932fd0 2024-09-13T18:27:37.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.648+0000 7fa6bfcaf740 10 client.26602 fetch_fsmap learned FSMap version 34 2024-09-13T18:27:37.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.649+0000 7fa6bfcaf740 1 -- 172.21.15.114:0/2852428311 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x562b9588a620 con 0x562b95932fd0 2024-09-13T18:27:37.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.654+0000 7fa69dffb640 1 -- 172.21.15.114:0/2852428311 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562b9557f4e0 con 0x562b95932fd0 2024-09-13T18:27:37.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.654+0000 7fa69dffb640 1 -- 172.21.15.114:0/2852428311 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa6ac0408f0 con 0x562b95932fd0 2024-09-13T18:27:37.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.654+0000 7fa6bfcaf740 10 client.26602 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:27:37.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.654+0000 7fa6bfcaf740 10 client.26602 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:27:37.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.654+0000 7fa6bfcaf740 20 client.26602 populate_metadata read hostname 'smithi114' 2024-09-13T18:27:37.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.654+0000 7fa6bfcaf740 1 --2- 172.21.15.114:0/2852428311 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x562b95942eb0 0x562b95945310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:37.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.654+0000 7fa6bfcaf740 4 client.26602 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:27:37.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.654+0000 7fa6bfcaf740 1 -- 172.21.15.114:0/2852428311 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562b955907e0 con 0x562b95942eb0 2024-09-13T18:27:37.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.657+0000 7fa6b659c640 1 --2- 172.21.15.114:0/2852428311 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x562b95942eb0 0x562b95945310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:37.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.658+0000 7fa6b659c640 1 --2- 172.21.15.114:0/2852428311 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x562b95942eb0 0x562b95945310 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b73bd3f514c46cf6 server_cookie=4af4c31bea990d94 in_seq=0 out_seq=0 2024-09-13T18:27:37.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.659+0000 7fa69dffb640 10 client.26602 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:27:37.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.659+0000 7fa69dffb640 1 -- 172.21.15.114:0/2852428311 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x562b955907e0 con 0x562b95942eb0 2024-09-13T18:27:37.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.660+0000 7fa69dffb640 10 client.26602 handle_command_reply: tid=0 2024-09-13T18:27:37.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.660+0000 7fa6bfcaf740 1 client.26602 shutdown 2024-09-13T18:27:37.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.664+0000 7fa6bfcaf740 1 -- 172.21.15.114:0/2852428311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b95932fd0 msgr2=0x562b95930470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:37.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.664+0000 7fa6bfcaf740 1 --2- 172.21.15.114:0/2852428311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b95932fd0 0x562b95930470 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac004500 tx=0x7fa6ac01a270 comp rx=0 tx=0).stop 2024-09-13T18:27:37.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.666+0000 7fa6bfcaf740 1 -- 172.21.15.114:0/2852428311 shutdown_connections 2024-09-13T18:27:37.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.666+0000 7fa6bfcaf740 1 --2- 172.21.15.114:0/2852428311 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x562b95942eb0 0x562b95945310 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:37.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.666+0000 7fa6bfcaf740 1 --2- 172.21.15.114:0/2852428311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b95932fd0 0x562b95930470 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:37.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.666+0000 7fa6bfcaf740 1 --2- 172.21.15.114:0/2852428311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b95928730 0x562b9592fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:37.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.666+0000 7fa6bfcaf740 1 --2- 172.21.15.114:0/2852428311 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b95927e10 0x562b95932a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:37.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.666+0000 7fa6bfcaf740 1 -- 172.21.15.114:0/2852428311 >> 172.21.15.114:0/2852428311 conn(0x562b95924380 msgr2=0x562b9592a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:37.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.666+0000 7fa69dffb640 0 client.26602 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:27:37.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.669+0000 7fa6bfcaf740 1 -- 172.21.15.114:0/2852428311 shutdown_connections 2024-09-13T18:27:37.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.672+0000 7fa6bfcaf740 1 -- 172.21.15.114:0/2852428311 wait complete. 2024-09-13T18:27:37.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.674+0000 7fa6bfcaf740 20 client.26602 trim_cache size 0 max 16384 2024-09-13T18:27:37.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:37 smithi204 ceph-mon[34777]: pgmap v157: 129 pgs: 129 active+clean; 17 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 45 MiB/s wr, 759 op/s 2024-09-13T18:27:37.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:37 smithi114 ceph-mon[30897]: pgmap v157: 129 pgs: 129 active+clean; 17 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 45 MiB/s wr, 759 op/s 2024-09-13T18:27:37.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.725+0000 7fa6bfcaf740 1 Processor -- start 2024-09-13T18:27:37.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.725+0000 7fa6bfcaf740 1 -- start start 2024-09-13T18:27:37.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.726+0000 7fa6bfcaf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b95924380 0x562b95949cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:37.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.726+0000 7fa6bfcaf740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b95927e10 0x562b9592b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:37.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.726+0000 7fa6bfcaf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b95928730 0x562b9592b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:37.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.726+0000 7fa6bfcaf740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b9557f4e0 con 0x562b95927e10 2024-09-13T18:27:37.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.726+0000 7fa6bfcaf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b9593ec80 con 0x562b95928730 2024-09-13T18:27:37.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.726+0000 7fa6bfcaf740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b95931060 con 0x562b95924380 2024-09-13T18:27:37.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.726+0000 7fa6b6d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b95928730 0x562b9592b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:37.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.726+0000 7fa6b6d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b95928730 0x562b9592b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:36600/0 (socket says 172.21.15.114:36600) 2024-09-13T18:27:37.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.726+0000 7fa6b6d9d640 1 -- 172.21.15.114:0/4089078544 learned_addr learned my addr 172.21.15.114:0/4089078544 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:37.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.726+0000 7fa6b659c640 1 --2- 172.21.15.114:0/4089078544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b95924380 0x562b95949cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:37.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.727+0000 7fa6b6d9d640 1 -- 172.21.15.114:0/4089078544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b95924380 msgr2=0x562b95949cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:37.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.727+0000 7fa6b6d9d640 1 --2- 172.21.15.114:0/4089078544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b95924380 0x562b95949cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:37.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.727+0000 7fa6b6d9d640 1 -- 172.21.15.114:0/4089078544 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b95927e10 msgr2=0x562b9592b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:37.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.727+0000 7fa6b6d9d640 1 --2- 172.21.15.114:0/4089078544 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b95927e10 0x562b9592b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:37.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.727+0000 7fa6b6d9d640 1 -- 172.21.15.114:0/4089078544 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b9582c790 con 0x562b95928730 2024-09-13T18:27:37.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.728+0000 7fa6b6d9d640 1 --2- 172.21.15.114:0/4089078544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b95928730 0x562b9592b570 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac03b9e0 tx=0x7fa6ac05a4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56eae77154575b10 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:37.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.729+0000 7fa6a57fa640 1 -- 172.21.15.114:0/4089078544 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac003d40 con 0x562b95928730 2024-09-13T18:27:37.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.729+0000 7fa6a57fa640 1 -- 172.21.15.114:0/4089078544 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa6ac051070 con 0x562b95928730 2024-09-13T18:27:37.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.729+0000 7fa6a57fa640 1 -- 172.21.15.114:0/4089078544 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac05aa30 con 0x562b95928730 2024-09-13T18:27:37.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.730+0000 7fa6bfcaf740 1 -- 172.21.15.114:0/4089078544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b95928730 msgr2=0x562b9592b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:37.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.730+0000 7fa6bfcaf740 1 --2- 172.21.15.114:0/4089078544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b95928730 0x562b9592b570 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac03b9e0 tx=0x7fa6ac05a4d0 comp rx=0 tx=0).stop 2024-09-13T18:27:37.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.730+0000 7fa6bfcaf740 1 -- 172.21.15.114:0/4089078544 shutdown_connections 2024-09-13T18:27:37.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.730+0000 7fa6bfcaf740 1 --2- 172.21.15.114:0/4089078544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b95928730 0x562b9592b570 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:37.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.730+0000 7fa6bfcaf740 1 --2- 172.21.15.114:0/4089078544 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b95927e10 0x562b9592b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:37.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.730+0000 7fa6bfcaf740 1 --2- 172.21.15.114:0/4089078544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b95924380 0x562b95949cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:37.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.730+0000 7fa6bfcaf740 1 -- 172.21.15.114:0/4089078544 >> 172.21.15.114:0/4089078544 conn(0x562b95932bc0 msgr2=0x562b95924760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:37.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.730+0000 7fa6bfcaf740 1 -- 172.21.15.114:0/4089078544 shutdown_connections 2024-09-13T18:27:37.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.730+0000 7fa6bfcaf740 1 -- 172.21.15.114:0/4089078544 wait complete. 2024-09-13T18:27:37.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.731+0000 7fa6bfcaf740 1 -- start start 2024-09-13T18:27:37.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.731+0000 7fa6bfcaf740 1 Processor -- start 2024-09-13T18:27:37.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.732+0000 7fa6bfcaf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b95924380 0x562b9593a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:37.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.732+0000 7fa6bfcaf740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b95927e10 0x562b9592d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:37.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.732+0000 7fa6bfcaf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b95928730 0x562b9592bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:37.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.732+0000 7fa6bfcaf740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b9592dca0 con 0x562b95927e10 2024-09-13T18:27:37.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.732+0000 7fa6bfcaf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b9592de10 con 0x562b95928730 2024-09-13T18:27:37.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.732+0000 7fa6bfcaf740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b9592c4c0 con 0x562b95924380 2024-09-13T18:27:37.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.732+0000 7fa6bfcaf740 10 client.0 _ll_register_callbacks cb 0x7fa6ac004c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:27:37.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.732+0000 7fa6b659c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b95924380 0x562b9593a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:37.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.732+0000 7fa6b659c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b95924380 0x562b9593a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:53768/0 (socket says 172.21.15.114:53768) 2024-09-13T18:27:37.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.732+0000 7fa6b659c640 1 -- 172.21.15.114:0/3024212776 learned_addr learned my addr 172.21.15.114:0/3024212776 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:37.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.732+0000 7fa6b6d9d640 1 --2- 172.21.15.114:0/3024212776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b95928730 0x562b9592bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:37.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.733+0000 7fa6b659c640 1 -- 172.21.15.114:0/3024212776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b95928730 msgr2=0x562b9592bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:37.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.733+0000 7fa6b659c640 1 --2- 172.21.15.114:0/3024212776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b95928730 0x562b9592bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:37.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.733+0000 7fa6b659c640 1 -- 172.21.15.114:0/3024212776 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b95927e10 msgr2=0x562b9592d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:37.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.733+0000 7fa6b659c640 1 --2- 172.21.15.114:0/3024212776 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b95927e10 0x562b9592d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:37.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.733+0000 7fa6b659c640 1 -- 172.21.15.114:0/3024212776 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562b9588a620 con 0x562b95924380 2024-09-13T18:27:37.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.733+0000 7fa6b659c640 1 --2- 172.21.15.114:0/3024212776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b95924380 0x562b9593a250 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8009710 tx=0x7fa6a8011cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c86682cce294144f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:37.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.733+0000 7fa69dffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:27:37.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.733+0000 7fa69dffb640 1 -- 172.21.15.114:0/3024212776 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a800bad0 con 0x562b95924380 2024-09-13T18:27:37.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.733+0000 7fa6bfcaf740 1 -- 172.21.15.114:0/3024212776 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa6a800bc30 con 0x562b95924380 2024-09-13T18:27:37.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.735+0000 7fa69dffb640 1 -- 172.21.15.114:0/3024212776 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fa6a801f040 con 0x562b95924380 2024-09-13T18:27:37.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.735+0000 7fa69dffb640 1 -- 172.21.15.114:0/3024212776 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6a8011b10 con 0x562b95924380 2024-09-13T18:27:37.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.735+0000 7fa6bfcaf740 10 client.26608 fetch_fsmap learned FSMap version 34 2024-09-13T18:27:37.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.735+0000 7fa6bfcaf740 1 -- 172.21.15.114:0/3024212776 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x562b95851ea0 con 0x562b95924380 2024-09-13T18:27:37.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.736+0000 7fa69dffb640 1 -- 172.21.15.114:0/3024212776 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa6a800bc30 con 0x562b95924380 2024-09-13T18:27:37.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.736+0000 7fa69dffb640 1 -- 172.21.15.114:0/3024212776 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa6a802c020 con 0x562b95924380 2024-09-13T18:27:37.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.736+0000 7fa6bfcaf740 10 client.26608 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:27:37.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.736+0000 7fa6bfcaf740 10 client.26608 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:27:37.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.736+0000 7fa6bfcaf740 20 client.26608 populate_metadata read hostname 'smithi114' 2024-09-13T18:27:37.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.736+0000 7fa6bfcaf740 1 --2- 172.21.15.114:0/3024212776 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x562b95932fd0 0x562b9599c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:37.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.736+0000 7fa6bfcaf740 4 client.26608 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T18:27:37.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.736+0000 7fa6bfcaf740 1 -- 172.21.15.114:0/3024212776 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562b955e3e30 con 0x562b95932fd0 2024-09-13T18:27:37.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.737+0000 7fa6b5d9b640 1 --2- 172.21.15.114:0/3024212776 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x562b95932fd0 0x562b9599c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:37.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.738+0000 7fa6b5d9b640 1 --2- 172.21.15.114:0/3024212776 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x562b95932fd0 0x562b9599c0b0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f554a6413b8fe49 server_cookie=2fa6adea912df976 in_seq=0 out_seq=0 2024-09-13T18:27:37.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.739+0000 7fa69dffb640 10 client.26608 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:27:37.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.739+0000 7fa69dffb640 1 -- 172.21.15.114:0/3024212776 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x562b955e3e30 con 0x562b95932fd0 2024-09-13T18:27:37.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.739+0000 7fa69dffb640 10 client.26608 handle_command_reply: tid=0 2024-09-13T18:27:37.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.739+0000 7fa6bfcaf740 1 client.26608 shutdown 2024-09-13T18:27:37.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.748+0000 7fa6bfcaf740 1 -- 172.21.15.114:0/3024212776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b95924380 msgr2=0x562b9593a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:37.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.748+0000 7fa6bfcaf740 1 --2- 172.21.15.114:0/3024212776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b95924380 0x562b9593a250 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8009710 tx=0x7fa6a8011cb0 comp rx=0 tx=0).stop 2024-09-13T18:27:37.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.752+0000 7fa6bfcaf740 1 -- 172.21.15.114:0/3024212776 shutdown_connections 2024-09-13T18:27:37.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.753+0000 7fa6bfcaf740 1 --2- 172.21.15.114:0/3024212776 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x562b95932fd0 0x562b9599c0b0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:37.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.753+0000 7fa6bfcaf740 1 --2- 172.21.15.114:0/3024212776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b95928730 0x562b9592bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:37.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.753+0000 7fa6bfcaf740 1 --2- 172.21.15.114:0/3024212776 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b95927e10 0x562b9592d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:37.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.753+0000 7fa6bfcaf740 1 --2- 172.21.15.114:0/3024212776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b95924380 0x562b9593a250 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:37.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.753+0000 7fa6bfcaf740 1 -- 172.21.15.114:0/3024212776 >> 172.21.15.114:0/3024212776 conn(0x562b95932bc0 msgr2=0x562b95924760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:37.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.755+0000 7fa69dffb640 0 client.26608 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:27:37.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.755+0000 7fa6bfcaf740 1 -- 172.21.15.114:0/3024212776 shutdown_connections 2024-09-13T18:27:37.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.755+0000 7fa6bfcaf740 1 -- 172.21.15.114:0/3024212776 wait complete. 2024-09-13T18:27:37.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.756+0000 7fa6bfcaf740 20 client.26608 trim_cache size 0 max 16384 2024-09-13T18:27:37.760 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:27:37.760 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "no active scrubs running", 2024-09-13T18:27:37.760 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": {} 2024-09-13T18:27:37.760 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:27:37.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.758+0000 7fa6bcf8a640 1 -- 172.21.15.114:0/2555179642 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa6a00a1390 msgr2=0x7fa6a00a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:37.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.758+0000 7fa6bcf8a640 1 --2- 172.21.15.114:0/2555179642 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa6a00a1390 0x7fa6a00a3850 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac005c20 tx=0x7fa6ac03b970 comp rx=0 tx=0).stop 2024-09-13T18:27:37.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.758+0000 7fa6bcf8a640 1 -- 172.21.15.114:0/2555179642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6b8107920 msgr2=0x7fa6b8118de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:37.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.758+0000 7fa6bcf8a640 1 --2- 172.21.15.114:0/2555179642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6b8107920 0x7fa6b8118de0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa6b000ba50 tx=0x7fa6b000bf10 comp rx=0 tx=0).stop 2024-09-13T18:27:37.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.759+0000 7fa6bcf8a640 1 -- 172.21.15.114:0/2555179642 shutdown_connections 2024-09-13T18:27:37.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.759+0000 7fa6bcf8a640 1 --2- 172.21.15.114:0/2555179642 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa6a00a1390 0x7fa6a00a3850 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:37.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.759+0000 7fa6bcf8a640 1 --2- 172.21.15.114:0/2555179642 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6b8119400 0x7fa6b81149c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:37.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.759+0000 7fa6bcf8a640 1 --2- 172.21.15.114:0/2555179642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6b8107920 0x7fa6b8118de0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:37.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.759+0000 7fa6bcf8a640 1 --2- 172.21.15.114:0/2555179642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6b8107540 0x7fa6b81188a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:37.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.759+0000 7fa6bcf8a640 1 -- 172.21.15.114:0/2555179642 >> 172.21.15.114:0/2555179642 conn(0x7fa6b806afe0 msgr2=0x7fa6b806ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:37.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.759+0000 7fa6bcf8a640 1 -- 172.21.15.114:0/2555179642 shutdown_connections 2024-09-13T18:27:37.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:37.759+0000 7fa6bcf8a640 1 -- 172.21.15.114:0/2555179642 wait complete. 2024-09-13T18:27:37.782 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-13T18:27:37.782 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:27:38.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.050+0000 7f3a01009640 1 -- 172.21.15.114:0/3442146726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39f40b7500 msgr2=0x7f39f40b98f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:38.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.050+0000 7f3a01009640 1 --2- 172.21.15.114:0/3442146726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39f40b7500 0x7f39f40b98f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f39f0009860 tx=0x7f39f002a340 comp rx=0 tx=0).stop 2024-09-13T18:27:38.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.051+0000 7f3a01009640 1 -- 172.21.15.114:0/3442146726 shutdown_connections 2024-09-13T18:27:38.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.051+0000 7f3a01009640 1 --2- 172.21.15.114:0/3442146726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39f40b7500 0x7f39f40b98f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:38.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.051+0000 7f3a01009640 1 --2- 172.21.15.114:0/3442146726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39f40a5100 0x7f39f40b6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:38.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.051+0000 7f3a01009640 1 --2- 172.21.15.114:0/3442146726 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39f40a47e0 0x7f39f40a4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:38.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.051+0000 7f3a01009640 1 -- 172.21.15.114:0/3442146726 >> 172.21.15.114:0/3442146726 conn(0x7f39f401a0c0 msgr2=0x7f39f401a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:38.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.051+0000 7f3a01009640 1 -- 172.21.15.114:0/3442146726 shutdown_connections 2024-09-13T18:27:38.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.052+0000 7f3a01009640 1 -- 172.21.15.114:0/3442146726 wait complete. 2024-09-13T18:27:38.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.052+0000 7f3a01009640 1 Processor -- start 2024-09-13T18:27:38.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.053+0000 7f3a01009640 1 -- start start 2024-09-13T18:27:38.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.053+0000 7f3a01009640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39f40a47e0 0x7f39f4142fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:38.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.053+0000 7f3a01009640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39f40a5100 0x7f39f4143520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:38.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.053+0000 7f3a01009640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39f40b7500 0x7f39f4147910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:38.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.053+0000 7f3a01009640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f4147f10 con 0x7f39f40a5100 2024-09-13T18:27:38.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.053+0000 7f3a01009640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f4148080 con 0x7f39f40b7500 2024-09-13T18:27:38.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.053+0000 7f3a01009640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f4148240 con 0x7f39f40a47e0 2024-09-13T18:27:38.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.053+0000 7f39fb7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39f40a5100 0x7f39f4143520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:38.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.054+0000 7f39fb7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39f40a5100 0x7f39f4143520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36108/0 (socket says 172.21.15.114:36108) 2024-09-13T18:27:38.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.054+0000 7f39fb7fe640 1 -- 172.21.15.114:0/1829921510 learned_addr learned my addr 172.21.15.114:0/1829921510 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:38.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.054+0000 7f39fbfff640 1 --2- 172.21.15.114:0/1829921510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39f40a47e0 0x7f39f4142fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:38.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.054+0000 7f3a00808640 1 --2- 172.21.15.114:0/1829921510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39f40b7500 0x7f39f4147910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:38.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.054+0000 7f39fbfff640 1 -- 172.21.15.114:0/1829921510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39f40b7500 msgr2=0x7f39f4147910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:38.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.054+0000 7f39fbfff640 1 --2- 172.21.15.114:0/1829921510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39f40b7500 0x7f39f4147910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:38.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.054+0000 7f39fbfff640 1 -- 172.21.15.114:0/1829921510 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39f40a5100 msgr2=0x7f39f4143520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:38.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.054+0000 7f39fbfff640 1 --2- 172.21.15.114:0/1829921510 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39f40a5100 0x7f39f4143520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:38.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.054+0000 7f39fbfff640 1 -- 172.21.15.114:0/1829921510 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f0009490 con 0x7f39f40a47e0 2024-09-13T18:27:38.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.055+0000 7f39fbfff640 1 --2- 172.21.15.114:0/1829921510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39f40a47e0 0x7f39f4142fe0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f39fc04e950 tx=0x7f39fc071e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:38.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.055+0000 7f39f97fa640 1 -- 172.21.15.114:0/1829921510 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39fc06fe10 con 0x7f39f40a47e0 2024-09-13T18:27:38.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.055+0000 7f3a01009640 1 -- 172.21.15.114:0/1829921510 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39f4148520 con 0x7f39f40a47e0 2024-09-13T18:27:38.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.055+0000 7f3a01009640 1 -- 172.21.15.114:0/1829921510 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39f414fda0 con 0x7f39f40a47e0 2024-09-13T18:27:38.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.056+0000 7f39f97fa640 1 -- 172.21.15.114:0/1829921510 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f39fc0738c0 con 0x7f39f40a47e0 2024-09-13T18:27:38.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.056+0000 7f39f97fa640 1 -- 172.21.15.114:0/1829921510 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39fc07b6f0 con 0x7f39f40a47e0 2024-09-13T18:27:38.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.059+0000 7f3a01009640 1 -- 172.21.15.114:0/1829921510 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39e0005250 con 0x7f39f40a47e0 2024-09-13T18:27:38.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.059+0000 7f39f97fa640 1 -- 172.21.15.114:0/1829921510 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f39fc07b850 con 0x7f39f40a47e0 2024-09-13T18:27:38.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.060+0000 7f39f97fa640 1 --2- 172.21.15.114:0/1829921510 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f39ec0a1340 0x7f39ec0a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:38.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.061+0000 7f39f97fa640 1 -- 172.21.15.114:0/1829921510 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f39fc12f360 con 0x7f39f40a47e0 2024-09-13T18:27:38.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.061+0000 7f39fb7fe640 1 --2- 172.21.15.114:0/1829921510 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f39ec0a1340 0x7f39ec0a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:38.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.062+0000 7f39fb7fe640 1 --2- 172.21.15.114:0/1829921510 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f39ec0a1340 0x7f39ec0a3800 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f39f4144580 tx=0x7f39e800a4b0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:38.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.067+0000 7f39f97fa640 1 -- 172.21.15.114:0/1829921510 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f39fc0fbc00 con 0x7f39f40a47e0 2024-09-13T18:27:38.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.580+0000 7f3a01009640 1 -- 172.21.15.114:0/1829921510 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f39e00050c0 con 0x7f39f40a47e0 2024-09-13T18:27:38.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.584+0000 7f39f97fa640 1 -- 172.21.15.114:0/1829921510 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+10930 (secure 0 0 0) 0x7f39fc0fba20 con 0x7f39f40a47e0 2024-09-13T18:27:38.590 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:27:38.590 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4162817497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4162817497},{"type":"v1","addr":"172.21.15.114:6835","nonce":4162817497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/2435428478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":2435428478},{"type":"v1","addr":"172.21.15.114:6837","nonce":2435428478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T18:20:01.895117+0000","modified":"2024-09-13T18:27:22.172031+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24482,"mds_2":24475,"mds_3":24458,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/2046516451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":2046516451},{"type":"v1","addr":"172.21.15.114:6839","nonce":2046516451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":10,"addr":"172.21.15.114:6841/1956923446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":1956923446},{"type":"v1","addr":"172.21.15.114:6841","nonce":1956923446}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24391":{"gid":24391,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/507301496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":507301496},{"type":"v1","addr":"172.21.15.204:6833","nonce":507301496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4028657219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4028657219},{"type":"v1","addr":"172.21.15.134:6833","nonce":4028657219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/57029119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":57029119},{"type":"v1","addr":"172.21.15.204:6835","nonce":57029119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3068880939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3068880939},{"type":"v1","addr":"172.21.15.204:6837","nonce":3068880939}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"e","rank":3,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.134:6835/1150003275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1150003275},{"type":"v1","addr":"172.21.15.134:6835","nonce":1150003275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/344883912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":344883912},{"type":"v1","addr":"172.21.15.204:6839","nonce":344883912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6837/891692210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":891692210},{"type":"v1","addr":"172.21.15.134:6837","nonce":891692210}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6839/3466309724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3466309724},{"type":"v1","addr":"172.21.15.134:6839","nonce":3466309724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:27:38.590 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 34 2024-09-13T18:27:38.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.597+0000 7f3a01009640 1 -- 172.21.15.114:0/1829921510 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f39ec0a1340 msgr2=0x7f39ec0a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:38.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.597+0000 7f3a01009640 1 --2- 172.21.15.114:0/1829921510 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f39ec0a1340 0x7f39ec0a3800 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f39f4144580 tx=0x7f39e800a4b0 comp rx=0 tx=0).stop 2024-09-13T18:27:38.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.597+0000 7f3a01009640 1 -- 172.21.15.114:0/1829921510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39f40a47e0 msgr2=0x7f39f4142fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:38.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.597+0000 7f3a01009640 1 --2- 172.21.15.114:0/1829921510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39f40a47e0 0x7f39f4142fe0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f39fc04e950 tx=0x7f39fc071e70 comp rx=0 tx=0).stop 2024-09-13T18:27:38.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.597+0000 7f3a01009640 1 -- 172.21.15.114:0/1829921510 shutdown_connections 2024-09-13T18:27:38.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.597+0000 7f3a01009640 1 --2- 172.21.15.114:0/1829921510 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f39ec0a1340 0x7f39ec0a3800 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:38.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.598+0000 7f3a01009640 1 --2- 172.21.15.114:0/1829921510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39f40b7500 0x7f39f4147910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:38.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.598+0000 7f3a01009640 1 --2- 172.21.15.114:0/1829921510 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39f40a5100 0x7f39f4143520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:38.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.598+0000 7f3a01009640 1 --2- 172.21.15.114:0/1829921510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39f40a47e0 0x7f39f4142fe0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:38.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.598+0000 7f3a01009640 1 -- 172.21.15.114:0/1829921510 >> 172.21.15.114:0/1829921510 conn(0x7f39f401a0c0 msgr2=0x7f39f40b4fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:38.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.598+0000 7f3a01009640 1 -- 172.21.15.114:0/1829921510 shutdown_connections 2024-09-13T18:27:38.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.598+0000 7f3a01009640 1 -- 172.21.15.114:0/1829921510 wait complete. 2024-09-13T18:27:38.637 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-09-13T18:27:38.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.900+0000 7f0797b4c640 1 -- 172.21.15.114:0/2750772729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0790107540 msgr2=0x7f079010e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:38.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.900+0000 7f0797b4c640 1 --2- 172.21.15.114:0/2750772729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0790107540 0x7f079010e510 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f078001a910 tx=0x7f078003b450 comp rx=0 tx=0).stop 2024-09-13T18:27:38.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.903+0000 7f0797b4c640 1 -- 172.21.15.114:0/2750772729 shutdown_connections 2024-09-13T18:27:38.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.903+0000 7f0797b4c640 1 --2- 172.21.15.114:0/2750772729 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0790119400 0x7f079011b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:38.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.903+0000 7f0797b4c640 1 --2- 172.21.15.114:0/2750772729 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0790107920 0x7f0790107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:38.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.903+0000 7f0797b4c640 1 --2- 172.21.15.114:0/2750772729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0790107540 0x7f079010e510 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:38.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.903+0000 7f0797b4c640 1 -- 172.21.15.114:0/2750772729 >> 172.21.15.114:0/2750772729 conn(0x7f079006afe0 msgr2=0x7f079006b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:38.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.909+0000 7f0797b4c640 1 -- 172.21.15.114:0/2750772729 shutdown_connections 2024-09-13T18:27:38.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.910+0000 7f0797b4c640 1 -- 172.21.15.114:0/2750772729 wait complete. 2024-09-13T18:27:38.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.911+0000 7f0797b4c640 1 Processor -- start 2024-09-13T18:27:38.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.911+0000 7f0797b4c640 1 -- start start 2024-09-13T18:27:38.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.911+0000 7f0797b4c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0790107540 0x7f0790114b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:38.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.912+0000 7f0797b4c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0790107920 0x7f0790115070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:38.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.912+0000 7f0797b4c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0790119400 0x7f07901a1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:38.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.912+0000 7f0797b4c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07901a1910 con 0x7f0790107920 2024-09-13T18:27:38.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.912+0000 7f0797b4c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07901a1a80 con 0x7f0790107540 2024-09-13T18:27:38.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.912+0000 7f0797b4c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07901a1c40 con 0x7f0790119400 2024-09-13T18:27:38.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.913+0000 7f07958e8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0790107540 0x7f0790114b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:38.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.913+0000 7f07958e8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0790107540 0x7f0790114b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:36652/0 (socket says 172.21.15.114:36652) 2024-09-13T18:27:38.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.913+0000 7f07958e8640 1 -- 172.21.15.114:0/3688569495 learned_addr learned my addr 172.21.15.114:0/3688569495 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:38.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.914+0000 7f07958e8640 1 -- 172.21.15.114:0/3688569495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0790119400 msgr2=0x7f07901a1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:38.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.914+0000 7f07958e8640 1 --2- 172.21.15.114:0/3688569495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0790119400 0x7f07901a1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:38.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.914+0000 7f07958e8640 1 -- 172.21.15.114:0/3688569495 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0790107920 msgr2=0x7f0790115070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:38.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.914+0000 7f07958e8640 1 --2- 172.21.15.114:0/3688569495 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0790107920 0x7f0790115070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:38.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.914+0000 7f07958e8640 1 -- 172.21.15.114:0/3688569495 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f078001a570 con 0x7f0790107540 2024-09-13T18:27:38.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.914+0000 7f07958e8640 1 --2- 172.21.15.114:0/3688569495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0790107540 0x7f0790114b30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0780006010 tx=0x7f0780005430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:38.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.916+0000 7f0786ffd640 1 -- 172.21.15.114:0/3688569495 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0780004050 con 0x7f0790107540 2024-09-13T18:27:38.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.916+0000 7f0786ffd640 1 -- 172.21.15.114:0/3688569495 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f07800035f0 con 0x7f0790107540 2024-09-13T18:27:38.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.916+0000 7f0786ffd640 1 -- 172.21.15.114:0/3688569495 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0780040770 con 0x7f0790107540 2024-09-13T18:27:38.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.916+0000 7f0797b4c640 1 -- 172.21.15.114:0/3688569495 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07901a1ec0 con 0x7f0790107540 2024-09-13T18:27:38.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.916+0000 7f0797b4c640 1 -- 172.21.15.114:0/3688569495 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07900788f0 con 0x7f0790107540 2024-09-13T18:27:38.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.928+0000 7f079a870740 1 Processor -- start 2024-09-13T18:27:38.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.929+0000 7f0786ffd640 1 -- 172.21.15.114:0/3688569495 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f0780003760 con 0x7f0790107540 2024-09-13T18:27:38.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.934+0000 7f0786ffd640 1 --2- 172.21.15.114:0/3688569495 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f077c0a1390 0x7f077c0a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:38.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.934+0000 7f0786ffd640 1 -- 172.21.15.114:0/3688569495 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f07800fea30 con 0x7f0790107540 2024-09-13T18:27:38.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.936+0000 7f079a870740 1 -- start start 2024-09-13T18:27:38.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.937+0000 7f079a870740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a87944d790 0x55a87944db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:38.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.937+0000 7f079a870740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a87944e0b0 0x55a879458410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:38.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.937+0000 7f079a870740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a879458950 0x55a87945ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:38.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.937+0000 7f079a870740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8793b7cc0 con 0x55a879458950 2024-09-13T18:27:38.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.937+0000 7f079a870740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8793b5450 con 0x55a87944e0b0 2024-09-13T18:27:38.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.937+0000 7f079a870740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8793b0d40 con 0x55a87944d790 2024-09-13T18:27:38.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.937+0000 7f07960e9640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a879458950 0x55a87945ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:38.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.937+0000 7f07960e9640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a879458950 0x55a87945ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36130/0 (socket says 172.21.15.114:36130) 2024-09-13T18:27:38.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.937+0000 7f07960e9640 1 -- 172.21.15.114:0/17370798 learned_addr learned my addr 172.21.15.114:0/17370798 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:38.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.938+0000 7f07960e9640 1 -- 172.21.15.114:0/17370798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a87944d790 msgr2=0x55a87944db70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:27:38.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.938+0000 7f07960e9640 1 --2- 172.21.15.114:0/17370798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a87944d790 0x55a87944db70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:38.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.938+0000 7f07960e9640 1 -- 172.21.15.114:0/17370798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a87944e0b0 msgr2=0x55a879458410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:27:38.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.938+0000 7f07960e9640 1 --2- 172.21.15.114:0/17370798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a87944e0b0 0x55a879458410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:38.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.938+0000 7f07960e9640 1 -- 172.21.15.114:0/17370798 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a879378620 con 0x55a879458950 2024-09-13T18:27:38.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.938+0000 7f07960e9640 1 --2- 172.21.15.114:0/3688569495 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f077c0a1390 0x7f077c0a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:38.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.938+0000 7f07960e9640 1 --2- 172.21.15.114:0/17370798 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a879458950 0x55a87945ad40 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f0788009e80 tx=0x7f078800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e48457cbc747471 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:38.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.939+0000 7f07960e9640 1 --2- 172.21.15.114:0/3688569495 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f077c0a1390 0x7f077c0a3850 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0788009fd0 tx=0x7f0788018000 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:38.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.939+0000 7f077bfff640 1 -- 172.21.15.114:0/17370798 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078800d8b0 con 0x55a879458950 2024-09-13T18:27:38.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.939+0000 7f077bfff640 1 -- 172.21.15.114:0/17370798 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f078801b420 con 0x55a879458950 2024-09-13T18:27:38.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.940+0000 7f079a870740 1 -- 172.21.15.114:0/17370798 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a879458950 msgr2=0x55a87945ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:38.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.940+0000 7f079a870740 1 --2- 172.21.15.114:0/17370798 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a879458950 0x55a87945ad40 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f0788009e80 tx=0x7f078800f730 comp rx=0 tx=0).stop 2024-09-13T18:27:38.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.940+0000 7f079a870740 1 -- 172.21.15.114:0/17370798 shutdown_connections 2024-09-13T18:27:38.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.940+0000 7f079a870740 1 --2- 172.21.15.114:0/17370798 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a879458950 0x55a87945ad40 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:38.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.940+0000 7f079a870740 1 --2- 172.21.15.114:0/17370798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a87944e0b0 0x55a879458410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:38.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.940+0000 7f079a870740 1 --2- 172.21.15.114:0/17370798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a87944d790 0x55a87944db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:38.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.940+0000 7f079a870740 1 -- 172.21.15.114:0/17370798 >> 172.21.15.114:0/17370798 conn(0x55a879449d00 msgr2=0x55a87944a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:38.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.945+0000 7f079a870740 1 -- 172.21.15.114:0/17370798 shutdown_connections 2024-09-13T18:27:38.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.946+0000 7f079a870740 1 -- 172.21.15.114:0/17370798 wait complete. 2024-09-13T18:27:38.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.946+0000 7f079a870740 1 -- start start 2024-09-13T18:27:38.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.946+0000 7f079a870740 1 Processor -- start 2024-09-13T18:27:38.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.949+0000 7f079a870740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a87944d790 0x55a879457c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:38.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.949+0000 7f079a870740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a87944e0b0 0x55a8794581d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:38.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.950+0000 7f079a870740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a87945c750 0x55a87945cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:38.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.950+0000 7f079a870740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a87936f180 con 0x55a87945c750 2024-09-13T18:27:38.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.950+0000 7f079a870740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a879089430 con 0x55a87944e0b0 2024-09-13T18:27:38.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.950+0000 7f079a870740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8793345e0 con 0x55a87944d790 2024-09-13T18:27:38.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.950+0000 7f079a870740 10 client.0 _ll_register_callbacks cb 0x55a8793a0aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:27:38.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.951+0000 7f07950e7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a87944e0b0 0x55a8794581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:38.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.951+0000 7f07950e7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a87944e0b0 0x55a8794581d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:36658/0 (socket says 172.21.15.114:36658) 2024-09-13T18:27:38.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.951+0000 7f07950e7640 1 -- 172.21.15.114:0/3428899276 learned_addr learned my addr 172.21.15.114:0/3428899276 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:38.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.951+0000 7f07958e8640 1 --2- 172.21.15.114:0/3428899276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a87944d790 0x55a879457c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:38.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.951+0000 7f07950e7640 1 -- 172.21.15.114:0/3428899276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a87944d790 msgr2=0x55a879457c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:38.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.951+0000 7f07950e7640 1 --2- 172.21.15.114:0/3428899276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a87944d790 0x55a879457c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:38.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.951+0000 7f07950e7640 1 -- 172.21.15.114:0/3428899276 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a87945c750 msgr2=0x55a87945cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:38.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.951+0000 7f07950e7640 1 --2- 172.21.15.114:0/3428899276 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a87945c750 0x55a87945cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:38.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.951+0000 7f07950e7640 1 -- 172.21.15.114:0/3428899276 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0788009ab0 con 0x55a87944e0b0 2024-09-13T18:27:38.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.952+0000 7f07950e7640 1 --2- 172.21.15.114:0/3428899276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a87944e0b0 0x55a8794581d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f078c00bc10 tx=0x7f078c00d6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3dc96efff00b6fd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:38.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.952+0000 7f07797fa640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:27:38.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.952+0000 7f07797fa640 1 -- 172.21.15.114:0/3428899276 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c010660 con 0x55a87944e0b0 2024-09-13T18:27:38.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.952+0000 7f07797fa640 1 -- 172.21.15.114:0/3428899276 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f078c0107c0 con 0x55a87944e0b0 2024-09-13T18:27:38.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.956+0000 7f079a870740 1 -- 172.21.15.114:0/3428899276 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a8793a31f0 con 0x55a87944e0b0 2024-09-13T18:27:38.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.956+0000 7f07797fa640 1 -- 172.21.15.114:0/3428899276 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f078c00b450 con 0x55a87944e0b0 2024-09-13T18:27:38.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.958+0000 7f079a870740 10 client.26591 fetch_fsmap learned FSMap version 34 2024-09-13T18:27:38.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.958+0000 7f079a870740 1 -- 172.21.15.114:0/3428899276 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55a8793af3c0 con 0x55a87944e0b0 2024-09-13T18:27:38.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.960+0000 7f07797fa640 1 -- 172.21.15.114:0/3428899276 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a8793a31f0 con 0x55a87944e0b0 2024-09-13T18:27:38.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.960+0000 7f07797fa640 1 -- 172.21.15.114:0/3428899276 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f078c01bb10 con 0x55a87944e0b0 2024-09-13T18:27:38.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.960+0000 7f079a870740 10 client.26591 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:27:38.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.960+0000 7f079a870740 10 client.26591 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-13T18:27:38.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.960+0000 7f079a870740 20 client.26591 populate_metadata read hostname 'smithi114' 2024-09-13T18:27:38.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.960+0000 7f079a870740 1 --2- 172.21.15.114:0/3428899276 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55a879468790 0x55a87946abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:38.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.960+0000 7f079a870740 4 client.26591 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:27:38.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.960+0000 7f079a870740 1 -- 172.21.15.114:0/3428899276 --> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a8790a44e0 con 0x55a879468790 2024-09-13T18:27:38.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.961+0000 7f07958e8640 1 --2- 172.21.15.114:0/3428899276 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55a879468790 0x55a87946abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:38.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.962+0000 7f07797fa640 10 client.26591 ms_handle_connect on v2:172.21.15.114:6840/1956923446 2024-09-13T18:27:38.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.963+0000 7f07958e8640 1 --2- 172.21.15.114:0/3428899276 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55a879468790 0x55a87946abb0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=310597c80849ed65 server_cookie=322e9666586ea3e7 in_seq=0 out_seq=0 2024-09-13T18:27:38.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.964+0000 7f07797fa640 1 -- 172.21.15.114:0/3428899276 <== mds.4 v2:172.21.15.114:6840/1956923446 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55a8790a44e0 con 0x55a879468790 2024-09-13T18:27:38.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.964+0000 7f07797fa640 10 client.26591 handle_command_reply: tid=0 2024-09-13T18:27:38.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.964+0000 7f079a870740 1 client.26591 shutdown 2024-09-13T18:27:38.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.967+0000 7f079a870740 1 -- 172.21.15.114:0/3428899276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a87944e0b0 msgr2=0x55a8794581d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:38.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.967+0000 7f079a870740 1 --2- 172.21.15.114:0/3428899276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a87944e0b0 0x55a8794581d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f078c00bc10 tx=0x7f078c00d6b0 comp rx=0 tx=0).stop 2024-09-13T18:27:38.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.967+0000 7f079a870740 1 -- 172.21.15.114:0/3428899276 shutdown_connections 2024-09-13T18:27:38.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.967+0000 7f079a870740 1 --2- 172.21.15.114:0/3428899276 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55a879468790 0x55a87946abb0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:38.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.967+0000 7f079a870740 1 --2- 172.21.15.114:0/3428899276 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a87945c750 0x55a87945cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:38.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.967+0000 7f079a870740 1 --2- 172.21.15.114:0/3428899276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a87944e0b0 0x55a8794581d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:38.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.967+0000 7f079a870740 1 --2- 172.21.15.114:0/3428899276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a87944d790 0x55a879457c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:38.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.967+0000 7f079a870740 1 -- 172.21.15.114:0/3428899276 >> 172.21.15.114:0/3428899276 conn(0x55a879449d00 msgr2=0x55a87944fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:38.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.967+0000 7f07797fa640 0 client.26591 ms_handle_reset on v2:172.21.15.114:6840/1956923446 2024-09-13T18:27:38.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.973+0000 7f079a870740 1 -- 172.21.15.114:0/3428899276 shutdown_connections 2024-09-13T18:27:38.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.973+0000 7f079a870740 1 -- 172.21.15.114:0/3428899276 wait complete. 2024-09-13T18:27:38.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:38.974+0000 7f079a870740 20 client.26591 trim_cache size 0 max 16384 2024-09-13T18:27:39.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.015+0000 7f079a870740 1 Processor -- start 2024-09-13T18:27:39.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.015+0000 7f079a870740 1 -- start start 2024-09-13T18:27:39.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.015+0000 7f079a870740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a87944d790 0x55a8794496a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:39.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.015+0000 7f079a870740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a87944e0b0 0x55a879457a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:39.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.015+0000 7f07958e8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a87944d790 0x55a8794496a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:39.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.015+0000 7f07958e8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a87944d790 0x55a8794496a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:36672/0 (socket says 172.21.15.114:36672) 2024-09-13T18:27:39.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.015+0000 7f079a870740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a87945c750 0x55a879457fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:39.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.015+0000 7f07958e8640 1 -- 172.21.15.114:0/87629029 learned_addr learned my addr 172.21.15.114:0/87629029 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:39.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.015+0000 7f079a870740 1 -- 172.21.15.114:0/87629029 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a879466fa0 con 0x55a87944e0b0 2024-09-13T18:27:39.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.015+0000 7f079a870740 1 -- 172.21.15.114:0/87629029 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a879458a30 con 0x55a87944d790 2024-09-13T18:27:39.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.015+0000 7f079a870740 1 -- 172.21.15.114:0/87629029 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a879452370 con 0x55a87945c750 2024-09-13T18:27:39.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.015+0000 7f07960e9640 1 --2- 172.21.15.114:0/87629029 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a87945c750 0x55a879457fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:39.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.016+0000 7f07950e7640 1 --2- 172.21.15.114:0/87629029 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a87944e0b0 0x55a879457a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:39.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.020+0000 7f07958e8640 1 -- 172.21.15.114:0/87629029 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a87945c750 msgr2=0x55a879457fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:39.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.020+0000 7f07958e8640 1 --2- 172.21.15.114:0/87629029 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a87945c750 0x55a879457fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.020+0000 7f07958e8640 1 -- 172.21.15.114:0/87629029 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a87944e0b0 msgr2=0x55a879457a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:39.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.020+0000 7f07958e8640 1 --2- 172.21.15.114:0/87629029 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a87944e0b0 0x55a879457a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.020+0000 7f07958e8640 1 -- 172.21.15.114:0/87629029 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a879351770 con 0x55a87944d790 2024-09-13T18:27:39.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.022+0000 7f07958e8640 1 --2- 172.21.15.114:0/87629029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a87944d790 0x55a8794496a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f07800c7c90 tx=0x7f07800eb040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e51198423803862e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:39.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.026+0000 7f0784e05640 1 -- 172.21.15.114:0/87629029 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07800d3c10 con 0x55a87944d790 2024-09-13T18:27:39.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.026+0000 7f0784e05640 1 -- 172.21.15.114:0/87629029 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f07800c9b30 con 0x55a87944d790 2024-09-13T18:27:39.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.026+0000 7f0784e05640 1 -- 172.21.15.114:0/87629029 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07800eb270 con 0x55a87944d790 2024-09-13T18:27:39.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.027+0000 7f079a870740 1 -- 172.21.15.114:0/87629029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a87944d790 msgr2=0x55a8794496a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:39.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.027+0000 7f079a870740 1 --2- 172.21.15.114:0/87629029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a87944d790 0x55a8794496a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f07800c7c90 tx=0x7f07800eb040 comp rx=0 tx=0).stop 2024-09-13T18:27:39.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.027+0000 7f079a870740 1 -- 172.21.15.114:0/87629029 shutdown_connections 2024-09-13T18:27:39.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.027+0000 7f079a870740 1 --2- 172.21.15.114:0/87629029 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a87945c750 0x55a879457fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.027+0000 7f079a870740 1 --2- 172.21.15.114:0/87629029 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a87944e0b0 0x55a879457a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.027+0000 7f079a870740 1 --2- 172.21.15.114:0/87629029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a87944d790 0x55a8794496a0 secure :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f07800c7c90 tx=0x7f07800eb040 comp rx=0 tx=0).stop 2024-09-13T18:27:39.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.027+0000 7f079a870740 1 -- 172.21.15.114:0/87629029 >> 172.21.15.114:0/87629029 conn(0x55a879449d00 msgr2=0x55a87944a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:39.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.030+0000 7f079a870740 1 -- 172.21.15.114:0/87629029 shutdown_connections 2024-09-13T18:27:39.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.032+0000 7f079a870740 1 -- 172.21.15.114:0/87629029 wait complete. 2024-09-13T18:27:39.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.033+0000 7f079a870740 1 -- start start 2024-09-13T18:27:39.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.033+0000 7f079a870740 1 Processor -- start 2024-09-13T18:27:39.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.034+0000 7f079a870740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a87944e0b0 0x55a879460230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:39.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.034+0000 7f079a870740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a87945c750 0x55a879460770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:39.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.034+0000 7f079a870740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a879460cb0 0x55a8794bdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:39.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.034+0000 7f079a870740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8794be1f0 con 0x55a879460cb0 2024-09-13T18:27:39.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.034+0000 7f079a870740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8794be360 con 0x55a87944e0b0 2024-09-13T18:27:39.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.034+0000 7f079a870740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8794be520 con 0x55a87945c750 2024-09-13T18:27:39.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.034+0000 7f079a870740 10 client.0 _ll_register_callbacks cb 0x7f0788009690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:27:39.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.035+0000 7f07958e8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a87944e0b0 0x55a879460230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:39.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.035+0000 7f07958e8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a87944e0b0 0x55a879460230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:36684/0 (socket says 172.21.15.114:36684) 2024-09-13T18:27:39.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.035+0000 7f07958e8640 1 -- 172.21.15.114:0/290188805 learned_addr learned my addr 172.21.15.114:0/290188805 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:39.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.035+0000 7f07960e9640 1 --2- 172.21.15.114:0/290188805 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a879460cb0 0x55a8794bdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:39.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.035+0000 7f07950e7640 1 --2- 172.21.15.114:0/290188805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a87945c750 0x55a879460770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:39.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.035+0000 7f07958e8640 1 -- 172.21.15.114:0/290188805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a87945c750 msgr2=0x55a879460770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:39.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.035+0000 7f07958e8640 1 --2- 172.21.15.114:0/290188805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a87945c750 0x55a879460770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.035+0000 7f07958e8640 1 -- 172.21.15.114:0/290188805 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a879460cb0 msgr2=0x55a8794bdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:39.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.035+0000 7f07958e8640 1 --2- 172.21.15.114:0/290188805 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a879460cb0 0x55a8794bdc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.035+0000 7f07958e8640 1 -- 172.21.15.114:0/290188805 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07900788f0 con 0x55a87944e0b0 2024-09-13T18:27:39.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.035+0000 7f07958e8640 1 --2- 172.21.15.114:0/290188805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a87944e0b0 0x55a879460230 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0780005ab0 tx=0x7f0780005be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=740f886d7f4f378 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:39.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.036+0000 7f07797fa640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:27:39.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.036+0000 7f07797fa640 1 -- 172.21.15.114:0/290188805 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07800eb270 con 0x55a87944e0b0 2024-09-13T18:27:39.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.036+0000 7f079a870740 1 -- 172.21.15.114:0/290188805 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a8793dab80 con 0x55a87944e0b0 2024-09-13T18:27:39.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.036+0000 7f07797fa640 1 -- 172.21.15.114:0/290188805 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f07800d3c10 con 0x55a87944e0b0 2024-09-13T18:27:39.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.036+0000 7f07797fa640 1 -- 172.21.15.114:0/290188805 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07800fc050 con 0x55a87944e0b0 2024-09-13T18:27:39.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.036+0000 7f079a870740 10 client.26597 fetch_fsmap learned FSMap version 34 2024-09-13T18:27:39.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.036+0000 7f079a870740 1 -- 172.21.15.114:0/290188805 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55a879376e80 con 0x55a87944e0b0 2024-09-13T18:27:39.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.037+0000 7f07797fa640 1 -- 172.21.15.114:0/290188805 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a8793dab80 con 0x55a87944e0b0 2024-09-13T18:27:39.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.037+0000 7f07797fa640 1 -- 172.21.15.114:0/290188805 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f078005c060 con 0x55a87944e0b0 2024-09-13T18:27:39.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.037+0000 7f079a870740 10 client.26597 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:27:39.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.037+0000 7f079a870740 10 client.26597 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-13T18:27:39.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.037+0000 7f079a870740 20 client.26597 populate_metadata read hostname 'smithi114' 2024-09-13T18:27:39.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.037+0000 7f079a870740 1 --2- 172.21.15.114:0/290188805 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55a8794bf800 0x55a8794c1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:39.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.037+0000 7f079a870740 4 client.26597 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:27:39.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.037+0000 7f079a870740 1 -- 172.21.15.114:0/290188805 --> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a8790b57e0 con 0x55a8794bf800 2024-09-13T18:27:39.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.037+0000 7f07950e7640 1 --2- 172.21.15.114:0/290188805 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55a8794bf800 0x55a8794c1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:39.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.038+0000 7f07950e7640 1 --2- 172.21.15.114:0/290188805 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55a8794bf800 0x55a8794c1bf0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=543c03a50f37d9b2 server_cookie=10727ae755fc4ff1 in_seq=0 out_seq=0 2024-09-13T18:27:39.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.041+0000 7f07797fa640 10 client.26597 ms_handle_connect on v2:172.21.15.114:6840/1956923446 2024-09-13T18:27:39.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.041+0000 7f07797fa640 1 -- 172.21.15.114:0/290188805 <== mds.4 v2:172.21.15.114:6840/1956923446 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a8790b57e0 con 0x55a8794bf800 2024-09-13T18:27:39.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.041+0000 7f07797fa640 10 client.26597 handle_command_reply: tid=0 2024-09-13T18:27:39.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.041+0000 7f079a870740 1 client.26597 shutdown 2024-09-13T18:27:39.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.041+0000 7f079a870740 1 -- 172.21.15.114:0/290188805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a87944e0b0 msgr2=0x55a879460230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:39.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.041+0000 7f079a870740 1 --2- 172.21.15.114:0/290188805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a87944e0b0 0x55a879460230 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0780005ab0 tx=0x7f0780005be0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.043+0000 7f079a870740 1 -- 172.21.15.114:0/290188805 shutdown_connections 2024-09-13T18:27:39.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.043+0000 7f079a870740 1 --2- 172.21.15.114:0/290188805 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55a8794bf800 0x55a8794c1bf0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.043+0000 7f079a870740 1 --2- 172.21.15.114:0/290188805 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a879460cb0 0x55a8794bdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.044+0000 7f079a870740 1 --2- 172.21.15.114:0/290188805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a87945c750 0x55a879460770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.044+0000 7f079a870740 1 --2- 172.21.15.114:0/290188805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a87944e0b0 0x55a879460230 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.044+0000 7f079a870740 1 -- 172.21.15.114:0/290188805 >> 172.21.15.114:0/290188805 conn(0x55a879449d00 msgr2=0x55a8794578a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:39.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.046+0000 7f07797fa640 0 client.26597 ms_handle_reset on v2:172.21.15.114:6840/1956923446 2024-09-13T18:27:39.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.053+0000 7f079a870740 1 -- 172.21.15.114:0/290188805 shutdown_connections 2024-09-13T18:27:39.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.054+0000 7f079a870740 1 -- 172.21.15.114:0/290188805 wait complete. 2024-09-13T18:27:39.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.064+0000 7f079a870740 20 client.26597 trim_cache size 0 max 16384 2024-09-13T18:27:39.068 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:27:39.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.072+0000 7f0797b4c640 1 -- 172.21.15.114:0/3688569495 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f077c0a1390 msgr2=0x7f077c0a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:39.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.072+0000 7f0797b4c640 1 --2- 172.21.15.114:0/3688569495 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f077c0a1390 0x7f077c0a3850 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0788009fd0 tx=0x7f0788018000 comp rx=0 tx=0).stop 2024-09-13T18:27:39.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.072+0000 7f0797b4c640 1 -- 172.21.15.114:0/3688569495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0790107540 msgr2=0x7f0790114b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:39.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.072+0000 7f0797b4c640 1 --2- 172.21.15.114:0/3688569495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0790107540 0x7f0790114b30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0780006010 tx=0x7f0780005430 comp rx=0 tx=0).stop 2024-09-13T18:27:39.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.073+0000 7f0797b4c640 1 -- 172.21.15.114:0/3688569495 shutdown_connections 2024-09-13T18:27:39.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.073+0000 7f0797b4c640 1 --2- 172.21.15.114:0/3688569495 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f077c0a1390 0x7f077c0a3850 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.073+0000 7f0797b4c640 1 --2- 172.21.15.114:0/3688569495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0790119400 0x7f07901a1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.073+0000 7f0797b4c640 1 --2- 172.21.15.114:0/3688569495 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0790107920 0x7f0790115070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.073+0000 7f0797b4c640 1 --2- 172.21.15.114:0/3688569495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0790107540 0x7f0790114b30 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.073+0000 7f0797b4c640 1 -- 172.21.15.114:0/3688569495 >> 172.21.15.114:0/3688569495 conn(0x7f079006afe0 msgr2=0x7f079006ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:39.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.082+0000 7f0797b4c640 1 -- 172.21.15.114:0/3688569495 shutdown_connections 2024-09-13T18:27:39.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.082+0000 7f0797b4c640 1 -- 172.21.15.114:0/3688569495 wait complete. 2024-09-13T18:27:39.116 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-09-13T18:27:39.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.386+0000 7f228d096640 1 -- 172.21.15.114:0/2359184682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22881109a0 msgr2=0x7f2288112d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:39.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.386+0000 7f228d096640 1 --2- 172.21.15.114:0/2359184682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22881109a0 0x7f2288112d90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2278009900 tx=0x7f227802a4d0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.387+0000 7f228d096640 1 -- 172.21.15.114:0/2359184682 shutdown_connections 2024-09-13T18:27:39.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.387+0000 7f228d096640 1 --2- 172.21.15.114:0/2359184682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22881109a0 0x7f2288112d90 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.387+0000 7f228d096640 1 --2- 172.21.15.114:0/2359184682 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22880fe630 0x7f2288110460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.387+0000 7f228d096640 1 --2- 172.21.15.114:0/2359184682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22880fdd10 0x7f22880fe0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.387+0000 7f228d096640 1 -- 172.21.15.114:0/2359184682 >> 172.21.15.114:0/2359184682 conn(0x7f22880f9bd0 msgr2=0x7f22880fbff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:39.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.392+0000 7f228d096640 1 -- 172.21.15.114:0/2359184682 shutdown_connections 2024-09-13T18:27:39.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.395+0000 7f228d096640 1 -- 172.21.15.114:0/2359184682 wait complete. 2024-09-13T18:27:39.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.396+0000 7f228d096640 1 Processor -- start 2024-09-13T18:27:39.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.397+0000 7f228d096640 1 -- start start 2024-09-13T18:27:39.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.397+0000 7f228d096640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22880fdd10 0x7f228806c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:39.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.397+0000 7f228d096640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22880fe630 0x7f228806cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:39.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.397+0000 7f228d096640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22881109a0 0x7f228810bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:39.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.397+0000 7f228d096640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228806d5f0 con 0x7f22881109a0 2024-09-13T18:27:39.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.397+0000 7f228d096640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228810c1e0 con 0x7f22880fdd10 2024-09-13T18:27:39.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.397+0000 7f228d096640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228810c3a0 con 0x7f22880fe630 2024-09-13T18:27:39.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.397+0000 7f228659c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22880fe630 0x7f228806cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:39.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.397+0000 7f228659c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22880fe630 0x7f228806cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:53852/0 (socket says 172.21.15.114:53852) 2024-09-13T18:27:39.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.397+0000 7f228659c640 1 -- 172.21.15.114:0/2095826431 learned_addr learned my addr 172.21.15.114:0/2095826431 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:39.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.397+0000 7f228759e640 1 --2- 172.21.15.114:0/2095826431 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22881109a0 0x7f228810bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:39.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.397+0000 7f2286d9d640 1 --2- 172.21.15.114:0/2095826431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22880fdd10 0x7f228806c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:39.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.399+0000 7f2286d9d640 1 -- 172.21.15.114:0/2095826431 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22880fe630 msgr2=0x7f228806cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:39.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.399+0000 7f2286d9d640 1 --2- 172.21.15.114:0/2095826431 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22880fe630 0x7f228806cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.399+0000 7f2286d9d640 1 -- 172.21.15.114:0/2095826431 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22881109a0 msgr2=0x7f228810bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:39.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.399+0000 7f2286d9d640 1 --2- 172.21.15.114:0/2095826431 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22881109a0 0x7f228810bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.399+0000 7f2286d9d640 1 -- 172.21.15.114:0/2095826431 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2278009560 con 0x7f22880fdd10 2024-09-13T18:27:39.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.399+0000 7f2286d9d640 1 --2- 172.21.15.114:0/2095826431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22880fdd10 0x7f228806c720 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f227c009fd0 tx=0x7f227c00b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:39.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.400+0000 7f226ffff640 1 -- 172.21.15.114:0/2095826431 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227c011830 con 0x7f22880fdd10 2024-09-13T18:27:39.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.400+0000 7f226ffff640 1 -- 172.21.15.114:0/2095826431 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f227c011e20 con 0x7f22880fdd10 2024-09-13T18:27:39.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.400+0000 7f226ffff640 1 -- 172.21.15.114:0/2095826431 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227c010e40 con 0x7f22880fdd10 2024-09-13T18:27:39.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.400+0000 7f228d096640 1 -- 172.21.15.114:0/2095826431 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f228810c680 con 0x7f22880fdd10 2024-09-13T18:27:39.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.400+0000 7f228d096640 1 -- 172.21.15.114:0/2095826431 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f228810cb40 con 0x7f22880fdd10 2024-09-13T18:27:39.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.405+0000 7f228fdbe740 1 Processor -- start 2024-09-13T18:27:39.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.405+0000 7f228fdbe740 1 -- start start 2024-09-13T18:27:39.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.405+0000 7f228fdbe740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e662fc6970 0x55e662fc6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:39.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.405+0000 7f228fdbe740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e662fc7290 0x55e662fd15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:39.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.405+0000 7f228fdbe740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e662fd1b30 0x55e662fd3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:39.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.405+0000 7f228fdbe740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e662f30cc0 con 0x55e662fc6970 2024-09-13T18:27:39.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.405+0000 7f228fdbe740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e662f2e450 con 0x55e662fc7290 2024-09-13T18:27:39.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.405+0000 7f228fdbe740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e662f29d40 con 0x55e662fd1b30 2024-09-13T18:27:39.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.407+0000 7f2286d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e662fc7290 0x55e662fd15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:39.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.407+0000 7f2286d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e662fc7290 0x55e662fd15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:36702/0 (socket says 172.21.15.114:36702) 2024-09-13T18:27:39.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.407+0000 7f2286d9d640 1 -- 172.21.15.114:0/2415853347 learned_addr learned my addr 172.21.15.114:0/2415853347 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:39.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.407+0000 7f226ffff640 1 -- 172.21.15.114:0/2095826431 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f227c011990 con 0x7f22880fdd10 2024-09-13T18:27:39.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.407+0000 7f2286d9d640 1 -- 172.21.15.114:0/2415853347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e662fd1b30 msgr2=0x55e662fd3f20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:27:39.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.407+0000 7f2286d9d640 1 --2- 172.21.15.114:0/2415853347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e662fd1b30 0x55e662fd3f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.407+0000 7f2286d9d640 1 -- 172.21.15.114:0/2415853347 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e662fc6970 msgr2=0x55e662fc6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:39.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.407+0000 7f2286d9d640 1 --2- 172.21.15.114:0/2415853347 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e662fc6970 0x55e662fc6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.407+0000 7f2286d9d640 1 -- 172.21.15.114:0/2415853347 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e662ef1620 con 0x55e662fc7290 2024-09-13T18:27:39.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.408+0000 7f2286d9d640 1 --2- 172.21.15.114:0/2415853347 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e662fc7290 0x55e662fd15f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f227c0c9dd0 tx=0x7f227c0d2e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7db2bff74de8ed39 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:39.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.409+0000 7f226ffff640 1 --2- 172.21.15.114:0/2095826431 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f22700a1410 0x7f22700a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:39.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.409+0000 7f226d7fa640 1 -- 172.21.15.114:0/2415853347 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227c0d4cd0 con 0x55e662fc7290 2024-09-13T18:27:39.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.409+0000 7f226d7fa640 1 -- 172.21.15.114:0/2415853347 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f227c0d4e30 con 0x55e662fc7290 2024-09-13T18:27:39.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.410+0000 7f228fdbe740 1 -- 172.21.15.114:0/2415853347 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e662fc7290 msgr2=0x55e662fd15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:39.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.410+0000 7f228fdbe740 1 --2- 172.21.15.114:0/2415853347 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e662fc7290 0x55e662fd15f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f227c0c9dd0 tx=0x7f227c0d2e30 comp rx=0 tx=0).stop 2024-09-13T18:27:39.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.411+0000 7f228fdbe740 1 -- 172.21.15.114:0/2415853347 shutdown_connections 2024-09-13T18:27:39.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.411+0000 7f228fdbe740 1 --2- 172.21.15.114:0/2415853347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e662fd1b30 0x55e662fd3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.411+0000 7f228fdbe740 1 --2- 172.21.15.114:0/2415853347 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e662fc7290 0x55e662fd15f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.411+0000 7f228fdbe740 1 --2- 172.21.15.114:0/2415853347 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e662fc6970 0x55e662fc6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.411+0000 7f228fdbe740 1 -- 172.21.15.114:0/2415853347 >> 172.21.15.114:0/2415853347 conn(0x55e662fc2ee0 msgr2=0x55e662fc32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:39.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.411+0000 7f228fdbe740 1 -- 172.21.15.114:0/2415853347 shutdown_connections 2024-09-13T18:27:39.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.411+0000 7f228fdbe740 1 -- 172.21.15.114:0/2415853347 wait complete. 2024-09-13T18:27:39.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.411+0000 7f228fdbe740 1 -- start start 2024-09-13T18:27:39.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.412+0000 7f228fdbe740 1 Processor -- start 2024-09-13T18:27:39.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.412+0000 7f228fdbe740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e662fc6970 0x55e662fd6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:39.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.412+0000 7f228fdbe740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e662fd1b30 0x55e662fd72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:39.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.412+0000 7f228fdbe740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e662fce2b0 0x55e662fce730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:39.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.412+0000 7f228fdbe740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e662ee8180 con 0x55e662fd1b30 2024-09-13T18:27:39.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.412+0000 7f228fdbe740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e662c02430 con 0x55e662fce2b0 2024-09-13T18:27:39.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.412+0000 7f228fdbe740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e662ead5e0 con 0x55e662fc6970 2024-09-13T18:27:39.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.412+0000 7f228fdbe740 10 client.0 _ll_register_callbacks cb 0x55e662f19aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:27:39.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.412+0000 7f2286d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e662fc6970 0x55e662fd6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:39.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.412+0000 7f2286d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e662fc6970 0x55e662fd6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:53872/0 (socket says 172.21.15.114:53872) 2024-09-13T18:27:39.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.412+0000 7f2286d9d640 1 -- 172.21.15.114:0/2270770475 learned_addr learned my addr 172.21.15.114:0/2270770475 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:39.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.413+0000 7f2286d9d640 1 -- 172.21.15.114:0/2270770475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e662fce2b0 msgr2=0x55e662fce730 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:27:39.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.413+0000 7f2286d9d640 1 --2- 172.21.15.114:0/2270770475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e662fce2b0 0x55e662fce730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.413+0000 7f2286d9d640 1 -- 172.21.15.114:0/2270770475 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e662fd1b30 msgr2=0x55e662fd72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:39.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.413+0000 7f2286d9d640 1 --2- 172.21.15.114:0/2270770475 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e662fd1b30 0x55e662fd72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.413+0000 7f2286d9d640 1 -- 172.21.15.114:0/2270770475 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f228810cb40 con 0x55e662fc6970 2024-09-13T18:27:39.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.413+0000 7f2286d9d640 1 --2- 172.21.15.114:0/2270770475 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e662fc6970 0x55e662fd6da0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f227c0d0fd0 tx=0x7f227c095cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e82b487b42a2843 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:39.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.413+0000 7f226aff5640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:27:39.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.414+0000 7f226aff5640 1 -- 172.21.15.114:0/2270770475 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227c099620 con 0x55e662fc6970 2024-09-13T18:27:39.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.414+0000 7f226aff5640 1 -- 172.21.15.114:0/2270770475 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f227c099f50 con 0x55e662fc6970 2024-09-13T18:27:39.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.414+0000 7f228fdbe740 1 -- 172.21.15.114:0/2270770475 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e662f1c1f0 con 0x55e662fc6970 2024-09-13T18:27:39.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.414+0000 7f226aff5640 1 -- 172.21.15.114:0/2270770475 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f227c099390 con 0x55e662fc6970 2024-09-13T18:27:39.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.414+0000 7f228fdbe740 10 client.26632 fetch_fsmap learned FSMap version 34 2024-09-13T18:27:39.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.414+0000 7f228fdbe740 1 -- 172.21.15.114:0/2270770475 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55e662f283c0 con 0x55e662fc6970 2024-09-13T18:27:39.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.414+0000 7f228759e640 1 --2- 172.21.15.114:0/2095826431 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f22700a1410 0x7f22700a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:39.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.414+0000 7f226aff5640 1 -- 172.21.15.114:0/2270770475 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e662f1c1f0 con 0x55e662fc6970 2024-09-13T18:27:39.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.414+0000 7f226aff5640 1 -- 172.21.15.114:0/2270770475 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f227c095f60 con 0x55e662fc6970 2024-09-13T18:27:39.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.415+0000 7f228fdbe740 10 client.26632 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:27:39.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.415+0000 7f228fdbe740 10 client.26632 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-13T18:27:39.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.416+0000 7f228fdbe740 20 client.26632 populate_metadata read hostname 'smithi114' 2024-09-13T18:27:39.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.416+0000 7f228fdbe740 1 --2- 172.21.15.114:0/2270770475 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55e662fe1380 0x55e662fe37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:39.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.416+0000 7f228fdbe740 4 client.26632 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:27:39.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.416+0000 7f228fdbe740 1 -- 172.21.15.114:0/2270770475 --> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e662c1d4e0 con 0x55e662fe1380 2024-09-13T18:27:39.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.416+0000 7f226ffff640 1 -- 172.21.15.114:0/2095826431 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f227c0cc9e0 con 0x7f22880fdd10 2024-09-13T18:27:39.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.418+0000 7f228659c640 1 --2- 172.21.15.114:0/2270770475 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55e662fe1380 0x55e662fe37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:39.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.420+0000 7f228759e640 1 --2- 172.21.15.114:0/2095826431 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f22700a1410 0x7f22700a38d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2278005c40 tx=0x7f2278005b50 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:39.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.425+0000 7f228659c640 1 --2- 172.21.15.114:0/2270770475 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55e662fe1380 0x55e662fe37b0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7875209f418ed069 server_cookie=b2caf1e10872d2a0 in_seq=0 out_seq=0 2024-09-13T18:27:39.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.425+0000 7f226aff5640 10 client.26632 ms_handle_connect on v2:172.21.15.134:6834/1150003275 2024-09-13T18:27:39.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.428+0000 7f226aff5640 1 -- 172.21.15.114:0/2270770475 <== mds.3 v2:172.21.15.134:6834/1150003275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55e662c1d4e0 con 0x55e662fe1380 2024-09-13T18:27:39.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.428+0000 7f226aff5640 10 client.26632 handle_command_reply: tid=0 2024-09-13T18:27:39.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.428+0000 7f228fdbe740 1 client.26632 shutdown 2024-09-13T18:27:39.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.429+0000 7f228fdbe740 1 -- 172.21.15.114:0/2270770475 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e662fc6970 msgr2=0x55e662fd6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:39.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.429+0000 7f228fdbe740 1 --2- 172.21.15.114:0/2270770475 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e662fc6970 0x55e662fd6da0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f227c0d0fd0 tx=0x7f227c095cb0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.429+0000 7f228fdbe740 1 -- 172.21.15.114:0/2270770475 shutdown_connections 2024-09-13T18:27:39.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.429+0000 7f228fdbe740 1 --2- 172.21.15.114:0/2270770475 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55e662fe1380 0x55e662fe37b0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.429+0000 7f228fdbe740 1 --2- 172.21.15.114:0/2270770475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e662fce2b0 0x55e662fce730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.429+0000 7f228fdbe740 1 --2- 172.21.15.114:0/2270770475 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e662fd1b30 0x55e662fd72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.429+0000 7f228fdbe740 1 --2- 172.21.15.114:0/2270770475 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e662fc6970 0x55e662fd6da0 secure :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f227c0d0fd0 tx=0x7f227c095cb0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.429+0000 7f228fdbe740 1 -- 172.21.15.114:0/2270770475 >> 172.21.15.114:0/2270770475 conn(0x55e662fc2ee0 msgr2=0x55e662fc1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:39.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.431+0000 7f226aff5640 0 client.26632 ms_handle_reset on v2:172.21.15.134:6834/1150003275 2024-09-13T18:27:39.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.434+0000 7f228fdbe740 1 -- 172.21.15.114:0/2270770475 shutdown_connections 2024-09-13T18:27:39.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.434+0000 7f228fdbe740 1 -- 172.21.15.114:0/2270770475 wait complete. 2024-09-13T18:27:39.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.436+0000 7f228fdbe740 20 client.26632 trim_cache size 0 max 16384 2024-09-13T18:27:39.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.461+0000 7f228fdbe740 1 Processor -- start 2024-09-13T18:27:39.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.461+0000 7f228fdbe740 1 -- start start 2024-09-13T18:27:39.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.462+0000 7f228fdbe740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e662fce2b0 0x55e662fccb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:39.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.462+0000 7f228fdbe740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e662fd1b30 0x55e662fcd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:39.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.462+0000 7f2286d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e662fce2b0 0x55e662fccb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:39.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.462+0000 7f2286d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e662fce2b0 0x55e662fccb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:53888/0 (socket says 172.21.15.114:53888) 2024-09-13T18:27:39.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.462+0000 7f2286d9d640 1 -- 172.21.15.114:0/175417425 learned_addr learned my addr 172.21.15.114:0/175417425 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:39.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.462+0000 7f228659c640 1 --2- 172.21.15.114:0/175417425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e662fd1b30 0x55e662fcd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:39.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.462+0000 7f228fdbe740 1 --2- 172.21.15.114:0/175417425 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e662fc9c50 0x55e662fc1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:39.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.462+0000 7f228fdbe740 1 -- 172.21.15.114:0/175417425 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e662c1d510 con 0x55e662fc9c50 2024-09-13T18:27:39.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.462+0000 7f228fdbe740 1 -- 172.21.15.114:0/175417425 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e662f1c220 con 0x55e662fd1b30 2024-09-13T18:27:39.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.463+0000 7f228fdbe740 1 -- 172.21.15.114:0/175417425 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e662fbeaf0 con 0x55e662fce2b0 2024-09-13T18:27:39.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.463+0000 7f228759e640 1 --2- 172.21.15.114:0/175417425 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e662fc9c50 0x55e662fc1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:39.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.463+0000 7f228659c640 1 -- 172.21.15.114:0/175417425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e662fce2b0 msgr2=0x55e662fccb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:39.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.463+0000 7f228659c640 1 --2- 172.21.15.114:0/175417425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e662fce2b0 0x55e662fccb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.463+0000 7f228659c640 1 -- 172.21.15.114:0/175417425 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e662fc9c50 msgr2=0x55e662fc1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:39.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.463+0000 7f228659c640 1 --2- 172.21.15.114:0/175417425 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e662fc9c50 0x55e662fc1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.463+0000 7f228659c640 1 -- 172.21.15.114:0/175417425 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e662fe3b60 con 0x55e662fd1b30 2024-09-13T18:27:39.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.464+0000 7f228659c640 1 --2- 172.21.15.114:0/175417425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e662fd1b30 0x55e662fcd0a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x55e662fbec80 tx=0x7f22800157b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef77850d61a6262c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:39.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.464+0000 7f226dffb640 1 -- 172.21.15.114:0/175417425 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2280018270 con 0x55e662fd1b30 2024-09-13T18:27:39.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.464+0000 7f226dffb640 1 -- 172.21.15.114:0/175417425 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f228000f040 con 0x55e662fd1b30 2024-09-13T18:27:39.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.465+0000 7f228fdbe740 1 -- 172.21.15.114:0/175417425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e662fd1b30 msgr2=0x55e662fcd0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:39.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.465+0000 7f228fdbe740 1 --2- 172.21.15.114:0/175417425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e662fd1b30 0x55e662fcd0a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x55e662fbec80 tx=0x7f22800157b0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.465+0000 7f228fdbe740 1 -- 172.21.15.114:0/175417425 shutdown_connections 2024-09-13T18:27:39.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.465+0000 7f228fdbe740 1 --2- 172.21.15.114:0/175417425 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e662fc9c50 0x55e662fc1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.465+0000 7f228fdbe740 1 --2- 172.21.15.114:0/175417425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e662fd1b30 0x55e662fcd0a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.465+0000 7f228fdbe740 1 --2- 172.21.15.114:0/175417425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e662fce2b0 0x55e662fccb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.465+0000 7f228fdbe740 1 -- 172.21.15.114:0/175417425 >> 172.21.15.114:0/175417425 conn(0x55e662fc2ee0 msgr2=0x55e662fc9540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:39.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.466+0000 7f228fdbe740 1 -- 172.21.15.114:0/175417425 shutdown_connections 2024-09-13T18:27:39.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.467+0000 7f228fdbe740 1 -- 172.21.15.114:0/175417425 wait complete. 2024-09-13T18:27:39.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.468+0000 7f228fdbe740 1 -- start start 2024-09-13T18:27:39.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.468+0000 7f228fdbe740 1 Processor -- start 2024-09-13T18:27:39.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.468+0000 7f228fdbe740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e662fc9c50 0x55e662fdc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:39.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.468+0000 7f228fdbe740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e662fce2b0 0x55e662fdcae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:39.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.468+0000 7f228fdbe740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e662fd1b30 0x55e6630378a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:39.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.468+0000 7f228fdbe740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e662fdd430 con 0x55e662fd1b30 2024-09-13T18:27:39.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.468+0000 7f228fdbe740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e663037de0 con 0x55e662fc9c50 2024-09-13T18:27:39.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.468+0000 7f228fdbe740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e663037f50 con 0x55e662fce2b0 2024-09-13T18:27:39.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.468+0000 7f228fdbe740 10 client.0 _ll_register_callbacks cb 0x7f227c0d2400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:27:39.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.469+0000 7f2286d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e662fc9c50 0x55e662fdc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:39.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.469+0000 7f2286d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e662fc9c50 0x55e662fdc5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:36740/0 (socket says 172.21.15.114:36740) 2024-09-13T18:27:39.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.469+0000 7f2286d9d640 1 -- 172.21.15.114:0/2438038578 learned_addr learned my addr 172.21.15.114:0/2438038578 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:39.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.469+0000 7f228659c640 1 --2- 172.21.15.114:0/2438038578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e662fce2b0 0x55e662fdcae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:39.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.469+0000 7f228759e640 1 --2- 172.21.15.114:0/2438038578 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e662fd1b30 0x55e6630378a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:39.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.469+0000 7f2286d9d640 1 -- 172.21.15.114:0/2438038578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e662fce2b0 msgr2=0x55e662fdcae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:39.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.469+0000 7f2286d9d640 1 --2- 172.21.15.114:0/2438038578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e662fce2b0 0x55e662fdcae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.469+0000 7f2286d9d640 1 -- 172.21.15.114:0/2438038578 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e662fd1b30 msgr2=0x55e6630378a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:39.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.469+0000 7f2286d9d640 1 --2- 172.21.15.114:0/2438038578 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e662fd1b30 0x55e6630378a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.469+0000 7f2286d9d640 1 -- 172.21.15.114:0/2438038578 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2280009cf0 con 0x55e662fc9c50 2024-09-13T18:27:39.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.470+0000 7f2286d9d640 1 --2- 172.21.15.114:0/2438038578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e662fc9c50 0x55e662fdc5a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f227c095d50 tx=0x7f227c096210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a597ebbc9bb718 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:39.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.470+0000 7f226aff5640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:27:39.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.470+0000 7f226aff5640 1 -- 172.21.15.114:0/2438038578 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227c00bea0 con 0x55e662fc9c50 2024-09-13T18:27:39.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.470+0000 7f228fdbe740 1 -- 172.21.15.114:0/2438038578 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e662f53b80 con 0x55e662fc9c50 2024-09-13T18:27:39.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.470+0000 7f226aff5640 1 -- 172.21.15.114:0/2438038578 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f227c0b7e40 con 0x55e662fc9c50 2024-09-13T18:27:39.477 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:39 smithi114 ceph-mon[30897]: pgmap v158: 129 pgs: 129 active+clean; 17 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 39 MiB/s wr, 649 op/s 2024-09-13T18:27:39.477 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:39 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/1829921510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:27:39.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.471+0000 7f226aff5640 1 -- 172.21.15.114:0/2438038578 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f227c00d040 con 0x55e662fc9c50 2024-09-13T18:27:39.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.471+0000 7f228fdbe740 10 client.26609 fetch_fsmap learned FSMap version 34 2024-09-13T18:27:39.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.471+0000 7f228fdbe740 1 -- 172.21.15.114:0/2438038578 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55e662eca770 con 0x55e662fc9c50 2024-09-13T18:27:39.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.471+0000 7f226aff5640 1 -- 172.21.15.114:0/2438038578 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e662f53b80 con 0x55e662fc9c50 2024-09-13T18:27:39.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.472+0000 7f226aff5640 1 -- 172.21.15.114:0/2438038578 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f227c098d40 con 0x55e662fc9c50 2024-09-13T18:27:39.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.472+0000 7f228fdbe740 10 client.26609 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:27:39.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.472+0000 7f228fdbe740 10 client.26609 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-13T18:27:39.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.472+0000 7f228fdbe740 20 client.26609 populate_metadata read hostname 'smithi114' 2024-09-13T18:27:39.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.472+0000 7f228fdbe740 1 --2- 172.21.15.114:0/2438038578 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55e663039220 0x55e66303b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:39.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.472+0000 7f228fdbe740 4 client.26609 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:27:39.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.472+0000 7f228fdbe740 1 -- 172.21.15.114:0/2438038578 --> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e662c2e7e0 con 0x55e663039220 2024-09-13T18:27:39.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.476+0000 7f228659c640 1 --2- 172.21.15.114:0/2438038578 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55e663039220 0x55e66303b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:39.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.479+0000 7f228659c640 1 --2- 172.21.15.114:0/2438038578 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55e663039220 0x55e66303b610 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7f3ed525ef3dd9c0 server_cookie=fa58ac17bdb5d9de in_seq=0 out_seq=0 2024-09-13T18:27:39.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.479+0000 7f226aff5640 10 client.26609 ms_handle_connect on v2:172.21.15.134:6834/1150003275 2024-09-13T18:27:39.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.479+0000 7f226aff5640 1 -- 172.21.15.114:0/2438038578 <== mds.3 v2:172.21.15.134:6834/1150003275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e662c2e7e0 con 0x55e663039220 2024-09-13T18:27:39.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.479+0000 7f226aff5640 10 client.26609 handle_command_reply: tid=0 2024-09-13T18:27:39.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.480+0000 7f228fdbe740 1 client.26609 shutdown 2024-09-13T18:27:39.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.483+0000 7f228fdbe740 1 -- 172.21.15.114:0/2438038578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e662fc9c50 msgr2=0x55e662fdc5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:39.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.483+0000 7f228fdbe740 1 --2- 172.21.15.114:0/2438038578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e662fc9c50 0x55e662fdc5a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f227c095d50 tx=0x7f227c096210 comp rx=0 tx=0).stop 2024-09-13T18:27:39.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.485+0000 7f228fdbe740 1 -- 172.21.15.114:0/2438038578 shutdown_connections 2024-09-13T18:27:39.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.485+0000 7f228fdbe740 1 --2- 172.21.15.114:0/2438038578 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55e663039220 0x55e66303b610 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.485+0000 7f228fdbe740 1 --2- 172.21.15.114:0/2438038578 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e662fd1b30 0x55e6630378a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.485+0000 7f228fdbe740 1 --2- 172.21.15.114:0/2438038578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e662fce2b0 0x55e662fdcae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.485+0000 7f228fdbe740 1 --2- 172.21.15.114:0/2438038578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e662fc9c50 0x55e662fdc5a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.485+0000 7f228fdbe740 1 -- 172.21.15.114:0/2438038578 >> 172.21.15.114:0/2438038578 conn(0x55e662fc2ee0 msgr2=0x55e662fcca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:39.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.486+0000 7f226aff5640 0 client.26609 ms_handle_reset on v2:172.21.15.134:6834/1150003275 2024-09-13T18:27:39.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.486+0000 7f228fdbe740 1 -- 172.21.15.114:0/2438038578 shutdown_connections 2024-09-13T18:27:39.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.486+0000 7f228fdbe740 1 -- 172.21.15.114:0/2438038578 wait complete. 2024-09-13T18:27:39.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.488+0000 7f228fdbe740 20 client.26609 trim_cache size 0 max 16384 2024-09-13T18:27:39.491 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:27:39.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.489+0000 7f228d096640 1 -- 172.21.15.114:0/2095826431 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f22700a1410 msgr2=0x7f22700a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:39.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.489+0000 7f228d096640 1 --2- 172.21.15.114:0/2095826431 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f22700a1410 0x7f22700a38d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2278005c40 tx=0x7f2278005b50 comp rx=0 tx=0).stop 2024-09-13T18:27:39.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.489+0000 7f228d096640 1 -- 172.21.15.114:0/2095826431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22880fdd10 msgr2=0x7f228806c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:39.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.489+0000 7f228d096640 1 --2- 172.21.15.114:0/2095826431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22880fdd10 0x7f228806c720 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f227c009fd0 tx=0x7f227c00b870 comp rx=0 tx=0).stop 2024-09-13T18:27:39.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.491+0000 7f228d096640 1 -- 172.21.15.114:0/2095826431 shutdown_connections 2024-09-13T18:27:39.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.491+0000 7f228d096640 1 --2- 172.21.15.114:0/2095826431 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f22700a1410 0x7f22700a38d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.492+0000 7f228d096640 1 --2- 172.21.15.114:0/2095826431 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22881109a0 0x7f228810bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.492+0000 7f228d096640 1 --2- 172.21.15.114:0/2095826431 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22880fe630 0x7f228806cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.492+0000 7f228d096640 1 --2- 172.21.15.114:0/2095826431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22880fdd10 0x7f228806c720 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.492+0000 7f228d096640 1 -- 172.21.15.114:0/2095826431 >> 172.21.15.114:0/2095826431 conn(0x7f22880f9bd0 msgr2=0x7f228810f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:39.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.495+0000 7f228d096640 1 -- 172.21.15.114:0/2095826431 shutdown_connections 2024-09-13T18:27:39.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.496+0000 7f228d096640 1 -- 172.21.15.114:0/2095826431 wait complete. 2024-09-13T18:27:39.531 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-09-13T18:27:39.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:39 smithi134 ceph-mon[34283]: pgmap v158: 129 pgs: 129 active+clean; 17 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 39 MiB/s wr, 649 op/s 2024-09-13T18:27:39.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:39 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/1829921510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:27:39.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:39 smithi204 ceph-mon[34777]: pgmap v158: 129 pgs: 129 active+clean; 17 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 39 MiB/s wr, 649 op/s 2024-09-13T18:27:39.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:39 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/1829921510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:27:39.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.778+0000 7f71fe7b5640 1 -- 172.21.15.114:0/2370215340 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71f8107dd0 msgr2=0x7f71f8114c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:39.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.778+0000 7f71fe7b5640 1 --2- 172.21.15.114:0/2370215340 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71f8107dd0 0x7f71f8114c30 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f71f0009ca0 tx=0x7f71f002a8b0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.779+0000 7f71fe7b5640 1 -- 172.21.15.114:0/2370215340 shutdown_connections 2024-09-13T18:27:39.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.779+0000 7f71fe7b5640 1 --2- 172.21.15.114:0/2370215340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71f8115170 0x7f71f8117560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.779+0000 7f71fe7b5640 1 --2- 172.21.15.114:0/2370215340 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71f8107dd0 0x7f71f8114c30 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.779+0000 7f71fe7b5640 1 --2- 172.21.15.114:0/2370215340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71f81074b0 0x7f71f8107890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.779+0000 7f71fe7b5640 1 -- 172.21.15.114:0/2370215340 >> 172.21.15.114:0/2370215340 conn(0x7f71f806af70 msgr2=0x7f71f806b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:39.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.783+0000 7f71fe7b5640 1 -- 172.21.15.114:0/2370215340 shutdown_connections 2024-09-13T18:27:39.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.783+0000 7f71fe7b5640 1 -- 172.21.15.114:0/2370215340 wait complete. 2024-09-13T18:27:39.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.783+0000 7f71fe7b5640 1 Processor -- start 2024-09-13T18:27:39.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.783+0000 7f71fe7b5640 1 -- start start 2024-09-13T18:27:39.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.784+0000 7f71fe7b5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71f81074b0 0x7f71f81a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:39.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.784+0000 7f71fe7b5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71f8107dd0 0x7f71f81a56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:39.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.784+0000 7f71fe7b5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71f8115170 0x7f71f81a98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:39.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.784+0000 7f71fe7b5640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f81a9ec0 con 0x7f71f81074b0 2024-09-13T18:27:39.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.784+0000 7f71fe7b5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f81aa030 con 0x7f71f8115170 2024-09-13T18:27:39.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.784+0000 7f71fe7b5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f81aa1f0 con 0x7f71f8107dd0 2024-09-13T18:27:39.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.784+0000 7f71fcd52640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71f8115170 0x7f71f81a98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:39.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.784+0000 7f71fcd52640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71f8115170 0x7f71f81a98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:36756/0 (socket says 172.21.15.114:36756) 2024-09-13T18:27:39.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.784+0000 7f71fcd52640 1 -- 172.21.15.114:0/827842626 learned_addr learned my addr 172.21.15.114:0/827842626 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:39.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.785+0000 7f71fcd52640 1 -- 172.21.15.114:0/827842626 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71f8107dd0 msgr2=0x7f71f81a56e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:27:39.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.785+0000 7f71fcd52640 1 --2- 172.21.15.114:0/827842626 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71f8107dd0 0x7f71f81a56e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.785+0000 7f71fcd52640 1 -- 172.21.15.114:0/827842626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71f81074b0 msgr2=0x7f71f81a51a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:27:39.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.785+0000 7f71fcd52640 1 --2- 172.21.15.114:0/827842626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71f81074b0 0x7f71f81a51a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.785+0000 7f71fcd52640 1 -- 172.21.15.114:0/827842626 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f0009900 con 0x7f71f8115170 2024-09-13T18:27:39.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.785+0000 7f71fcd52640 1 --2- 172.21.15.114:0/827842626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71f8115170 0x7f71f81a98c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f71e800e940 tx=0x7f71e800ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:39.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.786+0000 7f71f57fa640 1 -- 172.21.15.114:0/827842626 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e800ccc0 con 0x7f71f8115170 2024-09-13T18:27:39.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.786+0000 7f71f57fa640 1 -- 172.21.15.114:0/827842626 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f71e800ce20 con 0x7f71f8115170 2024-09-13T18:27:39.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.786+0000 7f71f57fa640 1 -- 172.21.15.114:0/827842626 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e8004e70 con 0x7f71f8115170 2024-09-13T18:27:39.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.788+0000 7f71fe7b5640 1 -- 172.21.15.114:0/827842626 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71f81aa4d0 con 0x7f71f8115170 2024-09-13T18:27:39.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.788+0000 7f71fe7b5640 1 -- 172.21.15.114:0/827842626 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71f80788f0 con 0x7f71f8115170 2024-09-13T18:27:39.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.795+0000 7f72014d6740 1 Processor -- start 2024-09-13T18:27:39.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.796+0000 7f72014d6740 1 -- start start 2024-09-13T18:27:39.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.796+0000 7f72014d6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e72ff2c970 0x55e72ff2cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:39.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.796+0000 7f72014d6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e72ff2d290 0x55e72ff375f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:39.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.796+0000 7f72014d6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e72ff37b30 0x55e72ff39f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:39.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.796+0000 7f72014d6740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e72fe96cc0 con 0x55e72ff2c970 2024-09-13T18:27:39.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.796+0000 7f72014d6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e72fe94450 con 0x55e72ff2d290 2024-09-13T18:27:39.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.796+0000 7f72014d6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e72fe8fd40 con 0x55e72ff37b30 2024-09-13T18:27:39.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.796+0000 7f71f57fa640 1 -- 172.21.15.114:0/827842626 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f71e8005020 con 0x7f71f8115170 2024-09-13T18:27:39.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.797+0000 7f71f57fa640 1 --2- 172.21.15.114:0/827842626 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f71e00a1410 0x7f71e00a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:39.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.798+0000 7f71f57fa640 1 -- 172.21.15.114:0/827842626 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f71e8014070 con 0x7f71f8115170 2024-09-13T18:27:39.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.799+0000 7f71fcd52640 1 --2- 172.21.15.114:0/827842626 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f71e00a1410 0x7f71e00a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:39.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.799+0000 7f71fcd52640 1 --2- 172.21.15.114:0/827842626 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f71e00a1410 0x7f71e00a38d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f71e80a20e0 tx=0x7f71e800ef30 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:39.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.800+0000 7f71f77fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e72ff2c970 0x55e72ff2cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:39.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.800+0000 7f71f77fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e72ff2c970 0x55e72ff2cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36230/0 (socket says 172.21.15.114:36230) 2024-09-13T18:27:39.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.800+0000 7f71f77fe640 1 -- 172.21.15.114:0/2107802562 learned_addr learned my addr 172.21.15.114:0/2107802562 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:39.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.800+0000 7f71f77fe640 1 --2- 172.21.15.114:0/2107802562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e72ff37b30 0x55e72ff39f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:39.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.801+0000 7f71f7fff640 1 --2- 172.21.15.114:0/2107802562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e72ff2d290 0x55e72ff375f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:39.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.802+0000 7f71f7fff640 1 -- 172.21.15.114:0/2107802562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e72ff37b30 msgr2=0x55e72ff39f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:39.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.802+0000 7f71f7fff640 1 --2- 172.21.15.114:0/2107802562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e72ff37b30 0x55e72ff39f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.802+0000 7f71f7fff640 1 -- 172.21.15.114:0/2107802562 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e72ff2c970 msgr2=0x55e72ff2cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:39.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.802+0000 7f71f7fff640 1 --2- 172.21.15.114:0/2107802562 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e72ff2c970 0x55e72ff2cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.802+0000 7f71f7fff640 1 -- 172.21.15.114:0/2107802562 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e72fe57620 con 0x55e72ff2d290 2024-09-13T18:27:39.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.802+0000 7f71f77fe640 1 --2- 172.21.15.114:0/2107802562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e72ff37b30 0x55e72ff39f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:27:39.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.802+0000 7f71f7fff640 1 --2- 172.21.15.114:0/2107802562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e72ff2d290 0x55e72ff375f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f71ec009ad0 tx=0x7f71ec009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bc2a7df622dac4c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:39.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.808+0000 7f71de7fc640 1 -- 172.21.15.114:0/2107802562 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ec00ed10 con 0x55e72ff2d290 2024-09-13T18:27:39.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.809+0000 7f71de7fc640 1 -- 172.21.15.114:0/2107802562 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f71ec0044c0 con 0x55e72ff2d290 2024-09-13T18:27:39.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.809+0000 7f71de7fc640 1 -- 172.21.15.114:0/2107802562 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ec004a00 con 0x55e72ff2d290 2024-09-13T18:27:39.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.812+0000 7f72014d6740 1 -- 172.21.15.114:0/2107802562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e72ff2d290 msgr2=0x55e72ff375f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:39.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.812+0000 7f72014d6740 1 --2- 172.21.15.114:0/2107802562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e72ff2d290 0x55e72ff375f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f71ec009ad0 tx=0x7f71ec009f90 comp rx=0 tx=0).stop 2024-09-13T18:27:39.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.813+0000 7f72014d6740 1 -- 172.21.15.114:0/2107802562 shutdown_connections 2024-09-13T18:27:39.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.813+0000 7f72014d6740 1 --2- 172.21.15.114:0/2107802562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e72ff37b30 0x55e72ff39f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.813+0000 7f72014d6740 1 --2- 172.21.15.114:0/2107802562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e72ff2d290 0x55e72ff375f0 secure :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f71ec009ad0 tx=0x7f71ec009f90 comp rx=0 tx=0).stop 2024-09-13T18:27:39.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.813+0000 7f72014d6740 1 --2- 172.21.15.114:0/2107802562 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e72ff2c970 0x55e72ff2cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.813+0000 7f72014d6740 1 -- 172.21.15.114:0/2107802562 >> 172.21.15.114:0/2107802562 conn(0x55e72ff28ee0 msgr2=0x55e72ff292c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:39.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.817+0000 7f72014d6740 1 -- 172.21.15.114:0/2107802562 shutdown_connections 2024-09-13T18:27:39.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.821+0000 7f72014d6740 1 -- 172.21.15.114:0/2107802562 wait complete. 2024-09-13T18:27:39.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.821+0000 7f72014d6740 1 -- start start 2024-09-13T18:27:39.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.821+0000 7f72014d6740 1 Processor -- start 2024-09-13T18:27:39.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.822+0000 7f72014d6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e72ff2c970 0x55e72ff37420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:39.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.822+0000 7f72014d6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e72ff37b30 0x55e72ff346c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:39.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.822+0000 7f72014d6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e72ff34c00 0x55e72ff35060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:39.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.822+0000 7f72014d6740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e72fe4e180 con 0x55e72ff2c970 2024-09-13T18:27:39.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.822+0000 7f72014d6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e72fb68430 con 0x55e72ff37b30 2024-09-13T18:27:39.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.822+0000 7f72014d6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e72fe135e0 con 0x55e72ff34c00 2024-09-13T18:27:39.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.822+0000 7f72014d6740 10 client.0 _ll_register_callbacks cb 0x55e72fe7faa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:27:39.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.822+0000 7f71f7fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e72ff37b30 0x55e72ff346c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:39.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.823+0000 7f71f7fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e72ff37b30 0x55e72ff346c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:36780/0 (socket says 172.21.15.114:36780) 2024-09-13T18:27:39.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.823+0000 7f71f7fff640 1 -- 172.21.15.114:0/3017326180 learned_addr learned my addr 172.21.15.114:0/3017326180 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:39.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.823+0000 7f71f7fff640 1 -- 172.21.15.114:0/3017326180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e72ff34c00 msgr2=0x55e72ff35060 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:27:39.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.823+0000 7f71f7fff640 1 --2- 172.21.15.114:0/3017326180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e72ff34c00 0x55e72ff35060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.823+0000 7f71f7fff640 1 -- 172.21.15.114:0/3017326180 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e72ff2c970 msgr2=0x55e72ff37420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:39.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.823+0000 7f71f7fff640 1 --2- 172.21.15.114:0/3017326180 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e72ff2c970 0x55e72ff37420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.823+0000 7f71f7fff640 1 -- 172.21.15.114:0/3017326180 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71ec009780 con 0x55e72ff37b30 2024-09-13T18:27:39.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.826+0000 7f71f7fff640 1 --2- 172.21.15.114:0/3017326180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e72ff37b30 0x55e72ff346c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f71ec005660 tx=0x7f71ec00ed40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f56ed1eb2777bcd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:39.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.828+0000 7f71d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:27:39.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.828+0000 7f71d77fe640 1 -- 172.21.15.114:0/3017326180 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ec01c050 con 0x55e72ff37b30 2024-09-13T18:27:39.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.829+0000 7f71d77fe640 1 -- 172.21.15.114:0/3017326180 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f71ec014070 con 0x55e72ff37b30 2024-09-13T18:27:39.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.829+0000 7f72014d6740 1 -- 172.21.15.114:0/3017326180 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e72feb9b80 con 0x55e72ff37b30 2024-09-13T18:27:39.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.831+0000 7f71d77fe640 1 -- 172.21.15.114:0/3017326180 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71ec0048c0 con 0x55e72ff37b30 2024-09-13T18:27:39.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.832+0000 7f72014d6740 10 client.26621 fetch_fsmap learned FSMap version 34 2024-09-13T18:27:39.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.832+0000 7f72014d6740 1 -- 172.21.15.114:0/3017326180 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55e72fe8e3c0 con 0x55e72ff37b30 2024-09-13T18:27:39.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.837+0000 7f71d77fe640 1 -- 172.21.15.114:0/3017326180 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e72feb9b80 con 0x55e72ff37b30 2024-09-13T18:27:39.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.838+0000 7f71d77fe640 1 -- 172.21.15.114:0/3017326180 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f71ec0189f0 con 0x55e72ff37b30 2024-09-13T18:27:39.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.838+0000 7f72014d6740 10 client.26621 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:27:39.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.838+0000 7f72014d6740 10 client.26621 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-13T18:27:39.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.838+0000 7f72014d6740 20 client.26621 populate_metadata read hostname 'smithi114' 2024-09-13T18:27:39.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.838+0000 7f72014d6740 1 --2- 172.21.15.114:0/3017326180 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55e72ff589e0 0x55e72ff5ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:39.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.838+0000 7f72014d6740 4 client.26621 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:27:39.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.838+0000 7f72014d6740 1 -- 172.21.15.114:0/3017326180 --> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e72fe821f0 con 0x55e72ff589e0 2024-09-13T18:27:39.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.841+0000 7f71f77fe640 1 --2- 172.21.15.114:0/3017326180 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55e72ff589e0 0x55e72ff5ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:39.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.844+0000 7f71f77fe640 1 --2- 172.21.15.114:0/3017326180 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55e72ff589e0 0x55e72ff5ae00 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7c06c27ccf7bf699 server_cookie=8bc8ba951f8288ff in_seq=0 out_seq=0 2024-09-13T18:27:39.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.846+0000 7f71d77fe640 10 client.26621 ms_handle_connect on v2:172.21.15.204:6838/344883912 2024-09-13T18:27:39.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.847+0000 7f71d77fe640 1 -- 172.21.15.114:0/3017326180 <== mds.2 v2:172.21.15.204:6838/344883912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55e72fe821f0 con 0x55e72ff589e0 2024-09-13T18:27:39.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.847+0000 7f71d77fe640 10 client.26621 handle_command_reply: tid=0 2024-09-13T18:27:39.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.847+0000 7f72014d6740 1 client.26621 shutdown 2024-09-13T18:27:39.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.847+0000 7f72014d6740 1 -- 172.21.15.114:0/3017326180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e72ff37b30 msgr2=0x55e72ff346c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:39.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.847+0000 7f72014d6740 1 --2- 172.21.15.114:0/3017326180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e72ff37b30 0x55e72ff346c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f71ec005660 tx=0x7f71ec00ed40 comp rx=0 tx=0).stop 2024-09-13T18:27:39.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.848+0000 7f72014d6740 1 -- 172.21.15.114:0/3017326180 shutdown_connections 2024-09-13T18:27:39.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.848+0000 7f72014d6740 1 --2- 172.21.15.114:0/3017326180 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55e72ff589e0 0x55e72ff5ae00 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.848+0000 7f72014d6740 1 --2- 172.21.15.114:0/3017326180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e72ff34c00 0x55e72ff35060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.848+0000 7f72014d6740 1 --2- 172.21.15.114:0/3017326180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e72ff37b30 0x55e72ff346c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.848+0000 7f72014d6740 1 --2- 172.21.15.114:0/3017326180 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e72ff2c970 0x55e72ff37420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.848+0000 7f72014d6740 1 -- 172.21.15.114:0/3017326180 >> 172.21.15.114:0/3017326180 conn(0x55e72ff28ee0 msgr2=0x55e72ff28230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:39.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.848+0000 7f71d77fe640 0 client.26621 ms_handle_reset on v2:172.21.15.204:6838/344883912 2024-09-13T18:27:39.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.852+0000 7f72014d6740 1 -- 172.21.15.114:0/3017326180 shutdown_connections 2024-09-13T18:27:39.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.853+0000 7f72014d6740 1 -- 172.21.15.114:0/3017326180 wait complete. 2024-09-13T18:27:39.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.853+0000 7f72014d6740 20 client.26621 trim_cache size 0 max 16384 2024-09-13T18:27:39.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.878+0000 7f72014d6740 1 Processor -- start 2024-09-13T18:27:39.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.880+0000 7f72014d6740 1 -- start start 2024-09-13T18:27:39.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.881+0000 7f72014d6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e72ff2c970 0x55e72ff36f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:39.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.881+0000 7f72014d6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e72ff34c00 0x55e72ff3e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:39.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.884+0000 7f71f77fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e72ff2c970 0x55e72ff36f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:39.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.884+0000 7f71f77fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e72ff2c970 0x55e72ff36f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:53952/0 (socket says 172.21.15.114:53952) 2024-09-13T18:27:39.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.885+0000 7f71f7fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e72ff34c00 0x55e72ff3e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:39.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.885+0000 7f71f77fe640 1 -- 172.21.15.114:0/1826124761 learned_addr learned my addr 172.21.15.114:0/1826124761 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:39.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.885+0000 7f72014d6740 1 --2- 172.21.15.114:0/1826124761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e72ff37b30 0x55e72ff3ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:39.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.885+0000 7f72014d6740 1 -- 172.21.15.114:0/1826124761 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e72ff35640 con 0x55e72ff34c00 2024-09-13T18:27:39.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.885+0000 7f72014d6740 1 -- 172.21.15.114:0/1826124761 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e72ff29e40 con 0x55e72ff37b30 2024-09-13T18:27:39.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.885+0000 7f72014d6740 1 -- 172.21.15.114:0/1826124761 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e72ff29bd0 con 0x55e72ff2c970 2024-09-13T18:27:39.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.886+0000 7f71f7fff640 1 -- 172.21.15.114:0/1826124761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e72ff2c970 msgr2=0x55e72ff36f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:39.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.886+0000 7f71f7fff640 1 --2- 172.21.15.114:0/1826124761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e72ff2c970 0x55e72ff36f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.886+0000 7f71f7fff640 1 -- 172.21.15.114:0/1826124761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e72ff37b30 msgr2=0x55e72ff3ed40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:27:39.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.886+0000 7f71f7fff640 1 --2- 172.21.15.114:0/1826124761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e72ff37b30 0x55e72ff3ed40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.886+0000 7f71f7fff640 1 -- 172.21.15.114:0/1826124761 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e72fe30770 con 0x55e72ff34c00 2024-09-13T18:27:39.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.887+0000 7f71f7fff640 1 --2- 172.21.15.114:0/1826124761 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e72ff34c00 0x55e72ff3e800 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x55e72ff35c70 tx=0x7f71ec005690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccbb337c7978f9fb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:39.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.888+0000 7f71deffd640 1 -- 172.21.15.114:0/1826124761 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ec01c070 con 0x55e72ff34c00 2024-09-13T18:27:39.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.888+0000 7f71deffd640 1 -- 172.21.15.114:0/1826124761 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f71ec01d040 con 0x55e72ff34c00 2024-09-13T18:27:39.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.888+0000 7f71deffd640 1 -- 172.21.15.114:0/1826124761 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ec0108d0 con 0x55e72ff34c00 2024-09-13T18:27:39.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.893+0000 7f72014d6740 1 -- 172.21.15.114:0/1826124761 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e72ff34c00 msgr2=0x55e72ff3e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:39.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.893+0000 7f72014d6740 1 --2- 172.21.15.114:0/1826124761 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e72ff34c00 0x55e72ff3e800 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x55e72ff35c70 tx=0x7f71ec005690 comp rx=0 tx=0).stop 2024-09-13T18:27:39.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.898+0000 7f72014d6740 1 -- 172.21.15.114:0/1826124761 shutdown_connections 2024-09-13T18:27:39.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.898+0000 7f72014d6740 1 --2- 172.21.15.114:0/1826124761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e72ff37b30 0x55e72ff3ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.898+0000 7f72014d6740 1 --2- 172.21.15.114:0/1826124761 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e72ff34c00 0x55e72ff3e800 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.898+0000 7f72014d6740 1 --2- 172.21.15.114:0/1826124761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e72ff2c970 0x55e72ff36f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.898+0000 7f72014d6740 1 -- 172.21.15.114:0/1826124761 >> 172.21.15.114:0/1826124761 conn(0x55e72ff28ee0 msgr2=0x55e72ff5f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:39.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.907+0000 7f72014d6740 1 -- 172.21.15.114:0/1826124761 shutdown_connections 2024-09-13T18:27:39.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.909+0000 7f72014d6740 1 -- 172.21.15.114:0/1826124761 wait complete. 2024-09-13T18:27:39.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.911+0000 7f72014d6740 1 -- start start 2024-09-13T18:27:39.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.912+0000 7f72014d6740 1 Processor -- start 2024-09-13T18:27:39.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.912+0000 7f72014d6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e72ff2c970 0x55e72ff3cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:39.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.912+0000 7f72014d6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e72ff34c00 0x55e72ff3d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:39.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.913+0000 7f72014d6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e72ff37b30 0x55e72ffa7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:39.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.913+0000 7f72014d6740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e72ffa8420 con 0x55e72ff37b30 2024-09-13T18:27:39.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.913+0000 7f72014d6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e72ffa8590 con 0x55e72ff34c00 2024-09-13T18:27:39.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.913+0000 7f72014d6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e72ffa8750 con 0x55e72ff2c970 2024-09-13T18:27:39.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.913+0000 7f72014d6740 10 client.0 _ll_register_callbacks cb 0x7f71ec009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:27:39.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.913+0000 7f71f77fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e72ff2c970 0x55e72ff3cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:39.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.913+0000 7f71f77fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e72ff2c970 0x55e72ff3cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:53960/0 (socket says 172.21.15.114:53960) 2024-09-13T18:27:39.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.913+0000 7f71f77fe640 1 -- 172.21.15.114:0/3941375439 learned_addr learned my addr 172.21.15.114:0/3941375439 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:39.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.913+0000 7f71f77fe640 1 --2- 172.21.15.114:0/3941375439 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e72ff37b30 0x55e72ffa7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:39.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.914+0000 7f71f7fff640 1 --2- 172.21.15.114:0/3941375439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e72ff34c00 0x55e72ff3d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:39.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.916+0000 7f71f77fe640 1 -- 172.21.15.114:0/3941375439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e72ff34c00 msgr2=0x55e72ff3d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:39.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.916+0000 7f71f77fe640 1 --2- 172.21.15.114:0/3941375439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e72ff34c00 0x55e72ff3d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.916+0000 7f71f77fe640 1 -- 172.21.15.114:0/3941375439 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e72ff37b30 msgr2=0x55e72ffa7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:39.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.916+0000 7f71f77fe640 1 --2- 172.21.15.114:0/3941375439 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e72ff37b30 0x55e72ffa7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.916+0000 7f71f77fe640 1 -- 172.21.15.114:0/3941375439 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e72fe8e3c0 con 0x55e72ff2c970 2024-09-13T18:27:39.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.917+0000 7f71f77fe640 1 --2- 172.21.15.114:0/3941375439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e72ff2c970 0x55e72ff3cbd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f71f0009620 tx=0x7f71f0040040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbc0fd260757735b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:39.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.918+0000 7f71d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:27:39.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.918+0000 7f71d77fe640 1 -- 172.21.15.114:0/3941375439 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f0045040 con 0x55e72ff2c970 2024-09-13T18:27:39.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.919+0000 7f71d77fe640 1 -- 172.21.15.114:0/3941375439 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f71f0030b80 con 0x55e72ff2c970 2024-09-13T18:27:39.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.919+0000 7f72014d6740 1 -- 172.21.15.114:0/3941375439 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f71ec0044c0 con 0x55e72ff2c970 2024-09-13T18:27:39.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.919+0000 7f71d77fe640 1 -- 172.21.15.114:0/3941375439 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71f0003970 con 0x55e72ff2c970 2024-09-13T18:27:39.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.920+0000 7f72014d6740 10 client.26650 fetch_fsmap learned FSMap version 34 2024-09-13T18:27:39.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.920+0000 7f72014d6740 1 -- 172.21.15.114:0/3941375439 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55e72fe55e80 con 0x55e72ff2c970 2024-09-13T18:27:39.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.924+0000 7f71d77fe640 1 -- 172.21.15.114:0/3941375439 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f71ec0044c0 con 0x55e72ff2c970 2024-09-13T18:27:39.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.924+0000 7f71d77fe640 1 -- 172.21.15.114:0/3941375439 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f71f0004030 con 0x55e72ff2c970 2024-09-13T18:27:39.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.924+0000 7f72014d6740 10 client.26650 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:27:39.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.924+0000 7f72014d6740 10 client.26650 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-13T18:27:39.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.924+0000 7f72014d6740 20 client.26650 populate_metadata read hostname 'smithi114' 2024-09-13T18:27:39.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.924+0000 7f72014d6740 1 --2- 172.21.15.114:0/3941375439 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55e72ffb0100 0x55e72ffb2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:39.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.924+0000 7f72014d6740 4 client.26650 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:27:39.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.924+0000 7f72014d6740 1 -- 172.21.15.114:0/3941375439 --> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e72fb834e0 con 0x55e72ffb0100 2024-09-13T18:27:39.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.926+0000 7f71f7fff640 1 --2- 172.21.15.114:0/3941375439 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55e72ffb0100 0x55e72ffb2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:39.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.927+0000 7f71d77fe640 10 client.26650 ms_handle_connect on v2:172.21.15.204:6838/344883912 2024-09-13T18:27:39.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.927+0000 7f71f7fff640 1 --2- 172.21.15.114:0/3941375439 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55e72ffb0100 0x55e72ffb2520 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cd8e597bc2f37389 server_cookie=d72742ee489cd1a0 in_seq=0 out_seq=0 2024-09-13T18:27:39.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.930+0000 7f71d77fe640 1 -- 172.21.15.114:0/3941375439 <== mds.2 v2:172.21.15.204:6838/344883912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e72fb834e0 con 0x55e72ffb0100 2024-09-13T18:27:39.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.930+0000 7f71d77fe640 10 client.26650 handle_command_reply: tid=0 2024-09-13T18:27:39.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.930+0000 7f72014d6740 1 client.26650 shutdown 2024-09-13T18:27:39.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.930+0000 7f72014d6740 1 -- 172.21.15.114:0/3941375439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e72ff2c970 msgr2=0x55e72ff3cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:39.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.930+0000 7f72014d6740 1 --2- 172.21.15.114:0/3941375439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e72ff2c970 0x55e72ff3cbd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f71f0009620 tx=0x7f71f0040040 comp rx=0 tx=0).stop 2024-09-13T18:27:39.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.930+0000 7f72014d6740 1 -- 172.21.15.114:0/3941375439 shutdown_connections 2024-09-13T18:27:39.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.930+0000 7f72014d6740 1 --2- 172.21.15.114:0/3941375439 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55e72ffb0100 0x55e72ffb2520 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.930+0000 7f72014d6740 1 --2- 172.21.15.114:0/3941375439 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e72ff37b30 0x55e72ffa7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.930+0000 7f72014d6740 1 --2- 172.21.15.114:0/3941375439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e72ff34c00 0x55e72ff3d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.930+0000 7f72014d6740 1 --2- 172.21.15.114:0/3941375439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e72ff2c970 0x55e72ff3cbd0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.930+0000 7f72014d6740 1 -- 172.21.15.114:0/3941375439 >> 172.21.15.114:0/3941375439 conn(0x55e72ff28ee0 msgr2=0x55e72ff33f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:39.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.931+0000 7f71d77fe640 0 client.26650 ms_handle_reset on v2:172.21.15.204:6838/344883912 2024-09-13T18:27:39.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.931+0000 7f72014d6740 1 -- 172.21.15.114:0/3941375439 shutdown_connections 2024-09-13T18:27:39.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.931+0000 7f72014d6740 1 -- 172.21.15.114:0/3941375439 wait complete. 2024-09-13T18:27:39.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.934+0000 7f72014d6740 20 client.26650 trim_cache size 0 max 16384 2024-09-13T18:27:39.938 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:27:39.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.936+0000 7f71fe7b5640 1 -- 172.21.15.114:0/827842626 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f71e00a1410 msgr2=0x7f71e00a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:39.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.936+0000 7f71fe7b5640 1 --2- 172.21.15.114:0/827842626 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f71e00a1410 0x7f71e00a38d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f71e80a20e0 tx=0x7f71e800ef30 comp rx=0 tx=0).stop 2024-09-13T18:27:39.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.936+0000 7f71fe7b5640 1 -- 172.21.15.114:0/827842626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71f8115170 msgr2=0x7f71f81a98c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:39.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.936+0000 7f71fe7b5640 1 --2- 172.21.15.114:0/827842626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71f8115170 0x7f71f81a98c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f71e800e940 tx=0x7f71e800ee00 comp rx=0 tx=0).stop 2024-09-13T18:27:39.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.940+0000 7f71fe7b5640 1 -- 172.21.15.114:0/827842626 shutdown_connections 2024-09-13T18:27:39.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.940+0000 7f71fe7b5640 1 --2- 172.21.15.114:0/827842626 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f71e00a1410 0x7f71e00a38d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.940+0000 7f71fe7b5640 1 --2- 172.21.15.114:0/827842626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71f8115170 0x7f71f81a98c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.940+0000 7f71fe7b5640 1 --2- 172.21.15.114:0/827842626 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71f8107dd0 0x7f71f81a56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.940+0000 7f71fe7b5640 1 --2- 172.21.15.114:0/827842626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71f81074b0 0x7f71f81a51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:39.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.940+0000 7f71fe7b5640 1 -- 172.21.15.114:0/827842626 >> 172.21.15.114:0/827842626 conn(0x7f71f806af70 msgr2=0x7f71f806ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:39.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.941+0000 7f71fe7b5640 1 -- 172.21.15.114:0/827842626 shutdown_connections 2024-09-13T18:27:39.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:39.941+0000 7f71fe7b5640 1 -- 172.21.15.114:0/827842626 wait complete. 2024-09-13T18:27:39.980 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-09-13T18:27:40.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.229+0000 7fa87569f640 1 -- 172.21.15.114:0/3590616749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8700ff770 msgr2=0x7fa8701037b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:40.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.229+0000 7fa87569f640 1 --2- 172.21.15.114:0/3590616749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8700ff770 0x7fa8701037b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa864009890 tx=0x7fa86402a380 comp rx=0 tx=0).stop 2024-09-13T18:27:40.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.231+0000 7fa87569f640 1 -- 172.21.15.114:0/3590616749 shutdown_connections 2024-09-13T18:27:40.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.231+0000 7fa87569f640 1 --2- 172.21.15.114:0/3590616749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa870103f10 0x7fa87010ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.231+0000 7fa87569f640 1 --2- 172.21.15.114:0/3590616749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8700ff770 0x7fa8701037b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.231+0000 7fa87569f640 1 --2- 172.21.15.114:0/3590616749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8700fedc0 0x7fa8700ff1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.231+0000 7fa87569f640 1 -- 172.21.15.114:0/3590616749 >> 172.21.15.114:0/3590616749 conn(0x7fa8700fa870 msgr2=0x7fa8700fcc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:40.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.237+0000 7fa87569f640 1 -- 172.21.15.114:0/3590616749 shutdown_connections 2024-09-13T18:27:40.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.237+0000 7fa87569f640 1 -- 172.21.15.114:0/3590616749 wait complete. 2024-09-13T18:27:40.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.238+0000 7fa87569f640 1 Processor -- start 2024-09-13T18:27:40.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.238+0000 7fa87569f640 1 -- start start 2024-09-13T18:27:40.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.238+0000 7fa87569f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8700fedc0 0x7fa870071cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:40.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.238+0000 7fa87569f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa870103f10 0x7fa87006ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:40.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.238+0000 7fa87569f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa870072230 0x7fa87006b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:40.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.238+0000 7fa87569f640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87006b930 con 0x7fa870072230 2024-09-13T18:27:40.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.238+0000 7fa87569f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87006baa0 con 0x7fa8700fedc0 2024-09-13T18:27:40.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.238+0000 7fa87569f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8701a50e0 con 0x7fa870103f10 2024-09-13T18:27:40.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.239+0000 7fa86f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa870103f10 0x7fa87006ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:40.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.239+0000 7fa86f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa870103f10 0x7fa87006ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:53982/0 (socket says 172.21.15.114:53982) 2024-09-13T18:27:40.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.239+0000 7fa86f7fe640 1 -- 172.21.15.114:0/371402825 learned_addr learned my addr 172.21.15.114:0/371402825 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:40.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.239+0000 7fa86f7fe640 1 -- 172.21.15.114:0/371402825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8700fedc0 msgr2=0x7fa870071cf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:27:40.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.239+0000 7fa86f7fe640 1 --2- 172.21.15.114:0/371402825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8700fedc0 0x7fa870071cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.239+0000 7fa86f7fe640 1 -- 172.21.15.114:0/371402825 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa870072230 msgr2=0x7fa87006b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:40.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.239+0000 7fa86f7fe640 1 --2- 172.21.15.114:0/371402825 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa870072230 0x7fa87006b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.239+0000 7fa86f7fe640 1 -- 172.21.15.114:0/371402825 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa864009490 con 0x7fa870103f10 2024-09-13T18:27:40.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.239+0000 7fa86f7fe640 1 --2- 172.21.15.114:0/371402825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa870103f10 0x7fa87006ac90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa864035040 tx=0x7fa864004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:40.247 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:40 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:27:40.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.255+0000 7fa86d7fa640 1 -- 172.21.15.114:0/371402825 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8640037f0 con 0x7fa870103f10 2024-09-13T18:27:40.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.255+0000 7fa86d7fa640 1 -- 172.21.15.114:0/371402825 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa864003de0 con 0x7fa870103f10 2024-09-13T18:27:40.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.255+0000 7fa86d7fa640 1 -- 172.21.15.114:0/371402825 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa864030db0 con 0x7fa870103f10 2024-09-13T18:27:40.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.255+0000 7fa87569f640 1 -- 172.21.15.114:0/371402825 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8701a5360 con 0x7fa870103f10 2024-09-13T18:27:40.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.255+0000 7fa87569f640 1 -- 172.21.15.114:0/371402825 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8701a57c0 con 0x7fa870103f10 2024-09-13T18:27:40.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.260+0000 7fa87961f740 1 Processor -- start 2024-09-13T18:27:40.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.262+0000 7fa87961f740 1 -- start start 2024-09-13T18:27:40.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.262+0000 7fa87961f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9fdc22970 0x55b9fdc22d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:40.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.262+0000 7fa87961f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9fdc23290 0x55b9fdc2d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:40.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.262+0000 7fa87961f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9fdc2db30 0x55b9fdc2ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:40.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.262+0000 7fa87961f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9fdb8ccc0 con 0x55b9fdc2db30 2024-09-13T18:27:40.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.262+0000 7fa87961f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9fdb8a450 con 0x55b9fdc23290 2024-09-13T18:27:40.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.262+0000 7fa87961f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9fdb85d40 con 0x55b9fdc22970 2024-09-13T18:27:40.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.263+0000 7fa86ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9fdc23290 0x55b9fdc2d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:40.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.263+0000 7fa86f7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9fdc2db30 0x55b9fdc2ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:40.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.263+0000 7fa86f7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9fdc2db30 0x55b9fdc2ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36268/0 (socket says 172.21.15.114:36268) 2024-09-13T18:27:40.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.263+0000 7fa86f7fe640 1 -- 172.21.15.114:0/2403940257 learned_addr learned my addr 172.21.15.114:0/2403940257 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:40.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.263+0000 7fa86d7fa640 1 -- 172.21.15.114:0/371402825 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fa864039040 con 0x7fa870103f10 2024-09-13T18:27:40.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.265+0000 7fa86d7fa640 1 --2- 172.21.15.114:0/371402825 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa8580a1410 0x7fa8580a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:40.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.265+0000 7fa86f7fe640 1 -- 172.21.15.114:0/2403940257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9fdc22970 msgr2=0x55b9fdc22d50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:27:40.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.265+0000 7fa86f7fe640 1 --2- 172.21.15.114:0/2403940257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9fdc22970 0x55b9fdc22d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.265+0000 7fa86f7fe640 1 -- 172.21.15.114:0/2403940257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9fdc23290 msgr2=0x55b9fdc2d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:40.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.265+0000 7fa86f7fe640 1 --2- 172.21.15.114:0/2403940257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9fdc23290 0x55b9fdc2d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.265+0000 7fa86f7fe640 1 -- 172.21.15.114:0/2403940257 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9fdb4d620 con 0x55b9fdc2db30 2024-09-13T18:27:40.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.265+0000 7fa86d7fa640 1 -- 172.21.15.114:0/371402825 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fa8640ed660 con 0x7fa870103f10 2024-09-13T18:27:40.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.266+0000 7fa86f7fe640 1 --2- 172.21.15.114:0/2403940257 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9fdc2db30 0x55b9fdc2ff20 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fa8640eaa80 tx=0x7fa8640f3a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abd17aa2dc51998d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:40.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.268+0000 7fa8567fc640 1 -- 172.21.15.114:0/2403940257 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8640b6810 con 0x55b9fdc2db30 2024-09-13T18:27:40.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.268+0000 7fa8567fc640 1 -- 172.21.15.114:0/2403940257 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa8640ba2a0 con 0x55b9fdc2db30 2024-09-13T18:27:40.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.269+0000 7fa87961f740 1 -- 172.21.15.114:0/2403940257 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9fdc2db30 msgr2=0x55b9fdc2ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:40.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.269+0000 7fa87961f740 1 --2- 172.21.15.114:0/2403940257 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9fdc2db30 0x55b9fdc2ff20 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fa8640eaa80 tx=0x7fa8640f3a40 comp rx=0 tx=0).stop 2024-09-13T18:27:40.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.270+0000 7fa874e9e640 1 --2- 172.21.15.114:0/371402825 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa8580a1410 0x7fa8580a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:40.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.271+0000 7fa87961f740 1 -- 172.21.15.114:0/2403940257 shutdown_connections 2024-09-13T18:27:40.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.271+0000 7fa87961f740 1 --2- 172.21.15.114:0/2403940257 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9fdc2db30 0x55b9fdc2ff20 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.271+0000 7fa87961f740 1 --2- 172.21.15.114:0/2403940257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9fdc23290 0x55b9fdc2d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.271+0000 7fa87961f740 1 --2- 172.21.15.114:0/2403940257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9fdc22970 0x55b9fdc22d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.271+0000 7fa87961f740 1 -- 172.21.15.114:0/2403940257 >> 172.21.15.114:0/2403940257 conn(0x55b9fdc1eee0 msgr2=0x55b9fdc1f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:40.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.272+0000 7fa874e9e640 1 --2- 172.21.15.114:0/371402825 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa8580a1410 0x7fa8580a38d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa868005c40 tx=0x7fa868009310 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:40.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.279+0000 7fa87961f740 1 -- 172.21.15.114:0/2403940257 shutdown_connections 2024-09-13T18:27:40.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.279+0000 7fa87961f740 1 -- 172.21.15.114:0/2403940257 wait complete. 2024-09-13T18:27:40.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.279+0000 7fa87961f740 1 -- start start 2024-09-13T18:27:40.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.280+0000 7fa87961f740 1 Processor -- start 2024-09-13T18:27:40.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.280+0000 7fa87961f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9fdc22970 0x55b9fdc2d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:40.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.280+0000 7fa87961f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9fdc23290 0x55b9fdc2a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:40.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.280+0000 7fa87961f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9fdc2db30 0x55b9fdc2aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:40.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.280+0000 7fa87961f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9fdb44180 con 0x55b9fdc22970 2024-09-13T18:27:40.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.280+0000 7fa87961f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9fd85e430 con 0x55b9fdc23290 2024-09-13T18:27:40.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.280+0000 7fa87961f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9fdb095e0 con 0x55b9fdc2db30 2024-09-13T18:27:40.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.280+0000 7fa87961f740 10 client.0 _ll_register_callbacks cb 0x55b9fdb75aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:27:40.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.280+0000 7fa86ffff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9fdc22970 0x55b9fdc2d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:40.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.280+0000 7fa86ffff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9fdc22970 0x55b9fdc2d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36280/0 (socket says 172.21.15.114:36280) 2024-09-13T18:27:40.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.280+0000 7fa86ffff640 1 -- 172.21.15.114:0/2445680498 learned_addr learned my addr 172.21.15.114:0/2445680498 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:40.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.281+0000 7fa86ffff640 1 -- 172.21.15.114:0/2445680498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9fdc2db30 msgr2=0x55b9fdc2aa40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:27:40.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.281+0000 7fa86ffff640 1 --2- 172.21.15.114:0/2445680498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9fdc2db30 0x55b9fdc2aa40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.281+0000 7fa86ffff640 1 -- 172.21.15.114:0/2445680498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9fdc23290 msgr2=0x55b9fdc2a500 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:27:40.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.281+0000 7fa86ffff640 1 --2- 172.21.15.114:0/2445680498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9fdc23290 0x55b9fdc2a500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.281+0000 7fa86ffff640 1 -- 172.21.15.114:0/2445680498 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8701a57c0 con 0x55b9fdc22970 2024-09-13T18:27:40.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.282+0000 7fa86ffff640 1 --2- 172.21.15.114:0/2445680498 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9fdc22970 0x55b9fdc2d230 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fa86000bdb0 tx=0x7fa86000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cd7a70470ce5682 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:40.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.282+0000 7fa84f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:27:40.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.282+0000 7fa84f7fe640 1 -- 172.21.15.114:0/2445680498 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa860011650 con 0x55b9fdc22970 2024-09-13T18:27:40.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.282+0000 7fa84f7fe640 1 -- 172.21.15.114:0/2445680498 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fa8600117b0 con 0x55b9fdc22970 2024-09-13T18:27:40.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.282+0000 7fa87961f740 1 -- 172.21.15.114:0/2445680498 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b9fdb781f0 con 0x55b9fdc22970 2024-09-13T18:27:40.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.285+0000 7fa84f7fe640 1 -- 172.21.15.114:0/2445680498 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa860011970 con 0x55b9fdc22970 2024-09-13T18:27:40.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.286+0000 7fa87961f740 10 client.17256 fetch_fsmap learned FSMap version 34 2024-09-13T18:27:40.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.286+0000 7fa87961f740 1 -- 172.21.15.114:0/2445680498 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55b9fdb843c0 con 0x55b9fdc22970 2024-09-13T18:27:40.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.287+0000 7fa84f7fe640 1 -- 172.21.15.114:0/2445680498 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b9fdb781f0 con 0x55b9fdc22970 2024-09-13T18:27:40.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.287+0000 7fa84f7fe640 1 -- 172.21.15.114:0/2445680498 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa86001ac50 con 0x55b9fdc22970 2024-09-13T18:27:40.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.287+0000 7fa87961f740 10 client.17256 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:27:40.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.287+0000 7fa87961f740 10 client.17256 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-13T18:27:40.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.287+0000 7fa87961f740 20 client.17256 populate_metadata read hostname 'smithi114' 2024-09-13T18:27:40.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.287+0000 7fa87961f740 1 --2- 172.21.15.114:0/2445680498 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55b9fdc3d450 0x55b9fdc3f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:40.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.287+0000 7fa87961f740 4 client.17256 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:27:40.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.287+0000 7fa87961f740 1 -- 172.21.15.114:0/2445680498 --> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b9fd8794e0 con 0x55b9fdc3d450 2024-09-13T18:27:40.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.291+0000 7fa86f7fe640 1 --2- 172.21.15.114:0/2445680498 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55b9fdc3d450 0x55b9fdc3f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:40.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.292+0000 7fa86f7fe640 1 --2- 172.21.15.114:0/2445680498 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55b9fdc3d450 0x55b9fdc3f870 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=322dd50df54676d1 server_cookie=a4574333a89f7d26 in_seq=0 out_seq=0 2024-09-13T18:27:40.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.293+0000 7fa84f7fe640 10 client.17256 ms_handle_connect on v2:172.21.15.134:6836/891692210 2024-09-13T18:27:40.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.295+0000 7fa84f7fe640 1 -- 172.21.15.114:0/2445680498 <== mds.1 v2:172.21.15.134:6836/891692210 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55b9fd8794e0 con 0x55b9fdc3d450 2024-09-13T18:27:40.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.295+0000 7fa84f7fe640 10 client.17256 handle_command_reply: tid=0 2024-09-13T18:27:40.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.295+0000 7fa87961f740 1 client.17256 shutdown 2024-09-13T18:27:40.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.300+0000 7fa87961f740 1 -- 172.21.15.114:0/2445680498 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9fdc22970 msgr2=0x55b9fdc2d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:40.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.300+0000 7fa87961f740 1 --2- 172.21.15.114:0/2445680498 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9fdc22970 0x55b9fdc2d230 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fa86000bdb0 tx=0x7fa86000f730 comp rx=0 tx=0).stop 2024-09-13T18:27:40.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.304+0000 7fa87961f740 1 -- 172.21.15.114:0/2445680498 shutdown_connections 2024-09-13T18:27:40.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.304+0000 7fa87961f740 1 --2- 172.21.15.114:0/2445680498 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55b9fdc3d450 0x55b9fdc3f870 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.304+0000 7fa87961f740 1 --2- 172.21.15.114:0/2445680498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9fdc2db30 0x55b9fdc2aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.304+0000 7fa87961f740 1 --2- 172.21.15.114:0/2445680498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9fdc23290 0x55b9fdc2a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.304+0000 7fa87961f740 1 --2- 172.21.15.114:0/2445680498 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9fdc22970 0x55b9fdc2d230 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.304+0000 7fa87961f740 1 -- 172.21.15.114:0/2445680498 >> 172.21.15.114:0/2445680498 conn(0x55b9fdc1eee0 msgr2=0x55b9fdc1e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:40.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.306+0000 7fa84f7fe640 0 client.17256 ms_handle_reset on v2:172.21.15.134:6836/891692210 2024-09-13T18:27:40.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.307+0000 7fa87961f740 1 -- 172.21.15.114:0/2445680498 shutdown_connections 2024-09-13T18:27:40.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.307+0000 7fa87961f740 1 -- 172.21.15.114:0/2445680498 wait complete. 2024-09-13T18:27:40.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.313+0000 7fa87961f740 20 client.17256 trim_cache size 0 max 16384 2024-09-13T18:27:40.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.348+0000 7fa87961f740 1 Processor -- start 2024-09-13T18:27:40.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.349+0000 7fa87961f740 1 -- start start 2024-09-13T18:27:40.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.349+0000 7fa87961f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9fdc22970 0x55b9fdc34570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:40.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.349+0000 7fa87961f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9fdc23290 0x55b9fdc34ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:40.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.349+0000 7fa87961f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9fdc2db30 0x55b9fdc2ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:40.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.349+0000 7fa87961f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9fdb781f0 con 0x55b9fdc23290 2024-09-13T18:27:40.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.349+0000 7fa87961f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9fdc3bc40 con 0x55b9fdc22970 2024-09-13T18:27:40.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.349+0000 7fa87961f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9fdc1a790 con 0x55b9fdc2db30 2024-09-13T18:27:40.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.351+0000 7fa86f7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9fdc23290 0x55b9fdc34ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:40.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.351+0000 7fa86ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9fdc22970 0x55b9fdc34570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:40.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.351+0000 7fa86f7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9fdc23290 0x55b9fdc34ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36294/0 (socket says 172.21.15.114:36294) 2024-09-13T18:27:40.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.351+0000 7fa86ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9fdc22970 0x55b9fdc34570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:36828/0 (socket says 172.21.15.114:36828) 2024-09-13T18:27:40.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.351+0000 7fa86f7fe640 1 -- 172.21.15.114:0/260588763 learned_addr learned my addr 172.21.15.114:0/260588763 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:40.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.351+0000 7fa874e9e640 1 --2- 172.21.15.114:0/260588763 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9fdc2db30 0x55b9fdc2ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:40.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.351+0000 7fa86f7fe640 1 -- 172.21.15.114:0/260588763 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9fdc2db30 msgr2=0x55b9fdc2ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:40.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.351+0000 7fa86f7fe640 1 --2- 172.21.15.114:0/260588763 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9fdc2db30 0x55b9fdc2ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.351+0000 7fa86f7fe640 1 -- 172.21.15.114:0/260588763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9fdc22970 msgr2=0x55b9fdc34570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:40.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.351+0000 7fa86f7fe640 1 --2- 172.21.15.114:0/260588763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9fdc22970 0x55b9fdc34570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.351+0000 7fa86f7fe640 1 -- 172.21.15.114:0/260588763 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9fdb26770 con 0x55b9fdc23290 2024-09-13T18:27:40.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.351+0000 7fa86f7fe640 1 --2- 172.21.15.114:0/260588763 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9fdc23290 0x55b9fdc34ab0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fa8640ba460 tx=0x7fa8640da040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a893f0386eb313f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:40.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.352+0000 7fa856ffd640 1 -- 172.21.15.114:0/260588763 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8640be110 con 0x55b9fdc23290 2024-09-13T18:27:40.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.352+0000 7fa856ffd640 1 -- 172.21.15.114:0/260588763 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa8640ba2a0 con 0x55b9fdc23290 2024-09-13T18:27:40.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.352+0000 7fa856ffd640 1 -- 172.21.15.114:0/260588763 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8640ba010 con 0x55b9fdc23290 2024-09-13T18:27:40.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.353+0000 7fa87961f740 1 -- 172.21.15.114:0/260588763 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9fdc23290 msgr2=0x55b9fdc34ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:40.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.353+0000 7fa87961f740 1 --2- 172.21.15.114:0/260588763 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9fdc23290 0x55b9fdc34ab0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fa8640ba460 tx=0x7fa8640da040 comp rx=0 tx=0).stop 2024-09-13T18:27:40.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.359+0000 7fa87961f740 1 -- 172.21.15.114:0/260588763 shutdown_connections 2024-09-13T18:27:40.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.359+0000 7fa87961f740 1 --2- 172.21.15.114:0/260588763 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9fdc2db30 0x55b9fdc2ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.359+0000 7fa87961f740 1 --2- 172.21.15.114:0/260588763 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9fdc23290 0x55b9fdc34ab0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.359+0000 7fa87961f740 1 --2- 172.21.15.114:0/260588763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9fdc22970 0x55b9fdc34570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.359+0000 7fa87961f740 1 -- 172.21.15.114:0/260588763 >> 172.21.15.114:0/260588763 conn(0x55b9fdc1eee0 msgr2=0x55b9fdc39030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:40.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.363+0000 7fa87961f740 1 -- 172.21.15.114:0/260588763 shutdown_connections 2024-09-13T18:27:40.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.365+0000 7fa87961f740 1 -- 172.21.15.114:0/260588763 wait complete. 2024-09-13T18:27:40.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.366+0000 7fa87961f740 1 -- start start 2024-09-13T18:27:40.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.367+0000 7fa87961f740 1 Processor -- start 2024-09-13T18:27:40.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.368+0000 7fa87961f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9fdc22970 0x55b9fdc34a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:40.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.368+0000 7fa87961f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9fdc23290 0x55b9fdc27c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:40.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.368+0000 7fa87961f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9fdc2db30 0x55b9fdc26770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:40.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.368+0000 7fa87961f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9fdc285a0 con 0x55b9fdc22970 2024-09-13T18:27:40.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.368+0000 7fa87961f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9fdc26cb0 con 0x55b9fdc23290 2024-09-13T18:27:40.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.368+0000 7fa86f7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9fdc23290 0x55b9fdc27c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:40.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.368+0000 7fa87961f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9fdc26e70 con 0x55b9fdc2db30 2024-09-13T18:27:40.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.369+0000 7fa87961f740 10 client.0 _ll_register_callbacks cb 0x7fa8640f3400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:27:40.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.369+0000 7fa874e9e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9fdc2db30 0x55b9fdc26770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:40.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.369+0000 7fa874e9e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9fdc2db30 0x55b9fdc26770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:54022/0 (socket says 172.21.15.114:54022) 2024-09-13T18:27:40.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.369+0000 7fa874e9e640 1 -- 172.21.15.114:0/3588374170 learned_addr learned my addr 172.21.15.114:0/3588374170 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:40.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.369+0000 7fa86ffff640 1 --2- 172.21.15.114:0/3588374170 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9fdc22970 0x55b9fdc34a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:40.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.370+0000 7fa874e9e640 1 -- 172.21.15.114:0/3588374170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9fdc23290 msgr2=0x55b9fdc27c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:40.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.370+0000 7fa874e9e640 1 --2- 172.21.15.114:0/3588374170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9fdc23290 0x55b9fdc27c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.370+0000 7fa874e9e640 1 -- 172.21.15.114:0/3588374170 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9fdc22970 msgr2=0x55b9fdc34a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:40.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.370+0000 7fa874e9e640 1 --2- 172.21.15.114:0/3588374170 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9fdc22970 0x55b9fdc34a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.370+0000 7fa874e9e640 1 -- 172.21.15.114:0/3588374170 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b9fdb4d620 con 0x55b9fdc2db30 2024-09-13T18:27:40.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.373+0000 7fa874e9e640 1 --2- 172.21.15.114:0/3588374170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9fdc2db30 0x55b9fdc26770 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa86800fdd0 tx=0x7fa868005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12d9fa37fe0d512d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:40.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.374+0000 7fa84f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:27:40.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.377+0000 7fa84f7fe640 1 -- 172.21.15.114:0/3588374170 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86800ecc0 con 0x55b9fdc2db30 2024-09-13T18:27:40.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.377+0000 7fa87961f740 1 -- 172.21.15.114:0/3588374170 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b9fdbafb80 con 0x55b9fdc2db30 2024-09-13T18:27:40.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.377+0000 7fa84f7fe640 1 -- 172.21.15.114:0/3588374170 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fa868015070 con 0x55b9fdc2db30 2024-09-13T18:27:40.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.379+0000 7fa84f7fe640 1 -- 172.21.15.114:0/3588374170 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa86800ee20 con 0x55b9fdc2db30 2024-09-13T18:27:40.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.381+0000 7fa87961f740 10 client.26662 fetch_fsmap learned FSMap version 34 2024-09-13T18:27:40.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.381+0000 7fa87961f740 1 -- 172.21.15.114:0/3588374170 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55b9fdb4be80 con 0x55b9fdc2db30 2024-09-13T18:27:40.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.381+0000 7fa84f7fe640 1 -- 172.21.15.114:0/3588374170 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b9fdbafb80 con 0x55b9fdc2db30 2024-09-13T18:27:40.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.381+0000 7fa84f7fe640 1 -- 172.21.15.114:0/3588374170 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa868030600 con 0x55b9fdc2db30 2024-09-13T18:27:40.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.382+0000 7fa87961f740 10 client.26662 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:27:40.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.382+0000 7fa87961f740 10 client.26662 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-13T18:27:40.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.382+0000 7fa87961f740 20 client.26662 populate_metadata read hostname 'smithi114' 2024-09-13T18:27:40.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.382+0000 7fa87961f740 1 --2- 172.21.15.114:0/3588374170 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55b9fdc94740 0x55b9fdc96b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:40.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.382+0000 7fa87961f740 4 client.26662 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:27:40.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.382+0000 7fa87961f740 1 -- 172.21.15.114:0/3588374170 --> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b9fd88a7e0 con 0x55b9fdc94740 2024-09-13T18:27:40.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.382+0000 7fa86ffff640 1 --2- 172.21.15.114:0/3588374170 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55b9fdc94740 0x55b9fdc96b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:40.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.385+0000 7fa86ffff640 1 --2- 172.21.15.114:0/3588374170 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55b9fdc94740 0x55b9fdc96b60 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=be2899e9be820be3 server_cookie=c88a7f62573b5685 in_seq=0 out_seq=0 2024-09-13T18:27:40.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.385+0000 7fa84f7fe640 10 client.26662 ms_handle_connect on v2:172.21.15.134:6836/891692210 2024-09-13T18:27:40.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.386+0000 7fa84f7fe640 1 -- 172.21.15.114:0/3588374170 <== mds.1 v2:172.21.15.134:6836/891692210 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b9fd88a7e0 con 0x55b9fdc94740 2024-09-13T18:27:40.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.386+0000 7fa84f7fe640 10 client.26662 handle_command_reply: tid=0 2024-09-13T18:27:40.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.386+0000 7fa87961f740 1 client.26662 shutdown 2024-09-13T18:27:40.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.391+0000 7fa87961f740 1 -- 172.21.15.114:0/3588374170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9fdc2db30 msgr2=0x55b9fdc26770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:40.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.391+0000 7fa87961f740 1 --2- 172.21.15.114:0/3588374170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9fdc2db30 0x55b9fdc26770 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa86800fdd0 tx=0x7fa868005bd0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.391+0000 7fa87961f740 1 -- 172.21.15.114:0/3588374170 shutdown_connections 2024-09-13T18:27:40.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.391+0000 7fa87961f740 1 --2- 172.21.15.114:0/3588374170 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55b9fdc94740 0x55b9fdc96b60 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.391+0000 7fa87961f740 1 --2- 172.21.15.114:0/3588374170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9fdc2db30 0x55b9fdc26770 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.391+0000 7fa87961f740 1 --2- 172.21.15.114:0/3588374170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9fdc23290 0x55b9fdc27c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.391+0000 7fa87961f740 1 --2- 172.21.15.114:0/3588374170 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9fdc22970 0x55b9fdc34a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.391+0000 7fa87961f740 1 -- 172.21.15.114:0/3588374170 >> 172.21.15.114:0/3588374170 conn(0x55b9fdc1eee0 msgr2=0x55b9fdc25540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:40.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.393+0000 7fa84f7fe640 0 client.26662 ms_handle_reset on v2:172.21.15.134:6836/891692210 2024-09-13T18:27:40.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.396+0000 7fa87961f740 1 -- 172.21.15.114:0/3588374170 shutdown_connections 2024-09-13T18:27:40.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.397+0000 7fa87961f740 1 -- 172.21.15.114:0/3588374170 wait complete. 2024-09-13T18:27:40.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.404+0000 7fa87961f740 20 client.26662 trim_cache size 0 max 16384 2024-09-13T18:27:40.407 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:27:40.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.413+0000 7fa87569f640 1 -- 172.21.15.114:0/371402825 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa8580a1410 msgr2=0x7fa8580a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:40.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.413+0000 7fa87569f640 1 --2- 172.21.15.114:0/371402825 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa8580a1410 0x7fa8580a38d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa868005c40 tx=0x7fa868009310 comp rx=0 tx=0).stop 2024-09-13T18:27:40.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.413+0000 7fa87569f640 1 -- 172.21.15.114:0/371402825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa870103f10 msgr2=0x7fa87006ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:40.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.413+0000 7fa87569f640 1 --2- 172.21.15.114:0/371402825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa870103f10 0x7fa87006ac90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa864035040 tx=0x7fa864004270 comp rx=0 tx=0).stop 2024-09-13T18:27:40.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.414+0000 7fa87569f640 1 -- 172.21.15.114:0/371402825 shutdown_connections 2024-09-13T18:27:40.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.415+0000 7fa87569f640 1 --2- 172.21.15.114:0/371402825 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa8580a1410 0x7fa8580a38d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.415+0000 7fa87569f640 1 --2- 172.21.15.114:0/371402825 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa870072230 0x7fa87006b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.415+0000 7fa87569f640 1 --2- 172.21.15.114:0/371402825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa870103f10 0x7fa87006ac90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.415+0000 7fa87569f640 1 --2- 172.21.15.114:0/371402825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8700fedc0 0x7fa870071cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.415+0000 7fa87569f640 1 -- 172.21.15.114:0/371402825 >> 172.21.15.114:0/371402825 conn(0x7fa8700fa870 msgr2=0x7fa8700fac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:40.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.418+0000 7fa87569f640 1 -- 172.21.15.114:0/371402825 shutdown_connections 2024-09-13T18:27:40.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.418+0000 7fa87569f640 1 -- 172.21.15.114:0/371402825 wait complete. 2024-09-13T18:27:40.456 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-09-13T18:27:40.595 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:40 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:27:40.618 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:40 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:27:40.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.720+0000 7f8902690640 1 -- 172.21.15.114:0/1742413680 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88fc110bc0 msgr2=0x7f88fc112fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:40.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.720+0000 7f8902690640 1 --2- 172.21.15.114:0/1742413680 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88fc110bc0 0x7f88fc112fb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f88f4009900 tx=0x7f88f402a420 comp rx=0 tx=0).stop 2024-09-13T18:27:40.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.725+0000 7f8902690640 1 -- 172.21.15.114:0/1742413680 shutdown_connections 2024-09-13T18:27:40.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.725+0000 7f8902690640 1 --2- 172.21.15.114:0/1742413680 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88fc110bc0 0x7f88fc112fb0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.725+0000 7f8902690640 1 --2- 172.21.15.114:0/1742413680 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88fc075300 0x7f88fc110680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.725+0000 7f8902690640 1 --2- 172.21.15.114:0/1742413680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88fc0749e0 0x7f88fc074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.725+0000 7f8902690640 1 -- 172.21.15.114:0/1742413680 >> 172.21.15.114:0/1742413680 conn(0x7f88fc0fdba0 msgr2=0x7f88fc0fffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:40.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.746+0000 7f8902690640 1 -- 172.21.15.114:0/1742413680 shutdown_connections 2024-09-13T18:27:40.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.748+0000 7f8902690640 1 -- 172.21.15.114:0/1742413680 wait complete. 2024-09-13T18:27:40.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.748+0000 7f8902690640 1 Processor -- start 2024-09-13T18:27:40.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.749+0000 7f8902690640 1 -- start start 2024-09-13T18:27:40.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.749+0000 7f8902690640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88fc0749e0 0x7f88fc071bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:40.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.749+0000 7f8902690640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88fc075300 0x7f88fc072130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:40.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.749+0000 7f8902690640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88fc110bc0 0x7f88fc06aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:40.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.749+0000 7f8902690640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88fc06b660 con 0x7f88fc075300 2024-09-13T18:27:40.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.749+0000 7f8902690640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88fc06b7a0 con 0x7f88fc110bc0 2024-09-13T18:27:40.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.749+0000 7f8902690640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88fc06b960 con 0x7f88fc0749e0 2024-09-13T18:27:40.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.749+0000 7f88fbfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88fc0749e0 0x7f88fc071bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:40.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.749+0000 7f88fbfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88fc0749e0 0x7f88fc071bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:54034/0 (socket says 172.21.15.114:54034) 2024-09-13T18:27:40.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.749+0000 7f88fbfff640 1 -- 172.21.15.114:0/3879681674 learned_addr learned my addr 172.21.15.114:0/3879681674 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:40.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.749+0000 7f8900c2d640 1 --2- 172.21.15.114:0/3879681674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88fc110bc0 0x7f88fc06aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:40.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.750+0000 7f88fbfff640 1 -- 172.21.15.114:0/3879681674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88fc110bc0 msgr2=0x7f88fc06aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:40.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.750+0000 7f88fbfff640 1 --2- 172.21.15.114:0/3879681674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88fc110bc0 0x7f88fc06aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.750+0000 7f88fbfff640 1 -- 172.21.15.114:0/3879681674 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88fc075300 msgr2=0x7f88fc072130 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:27:40.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.750+0000 7f88fbfff640 1 --2- 172.21.15.114:0/3879681674 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88fc075300 0x7f88fc072130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.750+0000 7f88fbfff640 1 -- 172.21.15.114:0/3879681674 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f4009560 con 0x7f88fc0749e0 2024-09-13T18:27:40.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.750+0000 7f88fbfff640 1 --2- 172.21.15.114:0/3879681674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88fc0749e0 0x7f88fc071bd0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f88e400e9f0 tx=0x7f88e400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:40.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.753+0000 7f88f9ffb640 1 -- 172.21.15.114:0/3879681674 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e400cd80 con 0x7f88fc0749e0 2024-09-13T18:27:40.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.754+0000 7f8902690640 1 -- 172.21.15.114:0/3879681674 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88fc06f760 con 0x7f88fc0749e0 2024-09-13T18:27:40.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.754+0000 7f8902690640 1 -- 172.21.15.114:0/3879681674 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88fc06fc70 con 0x7f88fc0749e0 2024-09-13T18:27:40.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.754+0000 7f88f9ffb640 1 -- 172.21.15.114:0/3879681674 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f88e40108c0 con 0x7f88fc0749e0 2024-09-13T18:27:40.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.754+0000 7f88f9ffb640 1 -- 172.21.15.114:0/3879681674 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e4018710 con 0x7f88fc0749e0 2024-09-13T18:27:40.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.761+0000 7f890548d740 1 Processor -- start 2024-09-13T18:27:40.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.763+0000 7f890548d740 1 -- start start 2024-09-13T18:27:40.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.764+0000 7f890548d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b458da0790 0x55b458da0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:40.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.764+0000 7f890548d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b458da10b0 0x55b458dab410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:40.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.764+0000 7f890548d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b458dab950 0x55b458dadd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:40.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.764+0000 7f890548d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b458d0acc0 con 0x55b458da10b0 2024-09-13T18:27:40.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.764+0000 7f890548d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b458d08450 con 0x55b458dab950 2024-09-13T18:27:40.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.764+0000 7f890548d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b458d03d40 con 0x55b458da0790 2024-09-13T18:27:40.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.764+0000 7f88f37fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b458dab950 0x55b458dadd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:40.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.764+0000 7f88f37fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b458dab950 0x55b458dadd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:36858/0 (socket says 172.21.15.114:36858) 2024-09-13T18:27:40.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.764+0000 7f88f37fe640 1 -- 172.21.15.114:0/3605084995 learned_addr learned my addr 172.21.15.114:0/3605084995 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:40.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.764+0000 7f8900c2d640 1 --2- 172.21.15.114:0/3605084995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b458da0790 0x55b458da0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:40.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.764+0000 7f88f37fe640 1 -- 172.21.15.114:0/3605084995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b458da0790 msgr2=0x55b458da0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:40.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.764+0000 7f88f37fe640 1 --2- 172.21.15.114:0/3605084995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b458da0790 0x55b458da0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.764+0000 7f88f37fe640 1 -- 172.21.15.114:0/3605084995 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b458da10b0 msgr2=0x55b458dab410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:40.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.764+0000 7f88f37fe640 1 --2- 172.21.15.114:0/3605084995 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b458da10b0 0x55b458dab410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.764+0000 7f88f37fe640 1 -- 172.21.15.114:0/3605084995 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b458ccb620 con 0x55b458dab950 2024-09-13T18:27:40.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.765+0000 7f88f37fe640 1 --2- 172.21.15.114:0/3605084995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b458dab950 0x55b458dadd40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f88ec009d80 tx=0x7f88ec00d6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f511712dc3f90ed0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:40.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.765+0000 7f88f27fc640 1 -- 172.21.15.114:0/3605084995 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ec003970 con 0x55b458dab950 2024-09-13T18:27:40.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.766+0000 7f88f27fc640 1 -- 172.21.15.114:0/3605084995 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f88ec004020 con 0x55b458dab950 2024-09-13T18:27:40.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.767+0000 7f890548d740 1 -- 172.21.15.114:0/3605084995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b458dab950 msgr2=0x55b458dadd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:40.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.767+0000 7f890548d740 1 --2- 172.21.15.114:0/3605084995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b458dab950 0x55b458dadd40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f88ec009d80 tx=0x7f88ec00d6b0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.769+0000 7f88f9ffb640 1 -- 172.21.15.114:0/3879681674 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f88e4010a30 con 0x7f88fc0749e0 2024-09-13T18:27:40.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.773+0000 7f890548d740 1 -- 172.21.15.114:0/3605084995 shutdown_connections 2024-09-13T18:27:40.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.773+0000 7f890548d740 1 --2- 172.21.15.114:0/3605084995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b458dab950 0x55b458dadd40 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.773+0000 7f890548d740 1 --2- 172.21.15.114:0/3605084995 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b458da10b0 0x55b458dab410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.773+0000 7f890548d740 1 --2- 172.21.15.114:0/3605084995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b458da0790 0x55b458da0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.773+0000 7f890548d740 1 -- 172.21.15.114:0/3605084995 >> 172.21.15.114:0/3605084995 conn(0x55b458d9cd00 msgr2=0x55b458d9d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:40.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.775+0000 7f88f9ffb640 1 --2- 172.21.15.114:0/3879681674 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f88e80a1410 0x7f88e80a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:40.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.776+0000 7f88f9ffb640 1 -- 172.21.15.114:0/3879681674 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f88e4014070 con 0x7f88fc0749e0 2024-09-13T18:27:40.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.777+0000 7f890548d740 1 -- 172.21.15.114:0/3605084995 shutdown_connections 2024-09-13T18:27:40.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.778+0000 7f88f37fe640 1 --2- 172.21.15.114:0/3879681674 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f88e80a1410 0x7f88e80a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:40.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.780+0000 7f890548d740 1 -- 172.21.15.114:0/3605084995 wait complete. 2024-09-13T18:27:40.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.780+0000 7f890548d740 1 -- start start 2024-09-13T18:27:40.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.781+0000 7f890548d740 1 Processor -- start 2024-09-13T18:27:40.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.781+0000 7f890548d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b458da0790 0x55b458da8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:40.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.781+0000 7f890548d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b458da10b0 0x55b458da8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:40.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.781+0000 7f890548d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b458dab950 0x55b458daa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:40.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.781+0000 7f890548d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b458cc2180 con 0x55b458da10b0 2024-09-13T18:27:40.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.781+0000 7f890548d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4589dc430 con 0x55b458dab950 2024-09-13T18:27:40.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.781+0000 7f890548d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b458c875e0 con 0x55b458da0790 2024-09-13T18:27:40.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.781+0000 7f890548d740 10 client.0 _ll_register_callbacks cb 0x55b458cf3aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:27:40.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.782+0000 7f88fbfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b458da0790 0x55b458da8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:40.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.782+0000 7f8900c2d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b458dab950 0x55b458daa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:40.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.782+0000 7f8900c2d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b458dab950 0x55b458daa8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:36870/0 (socket says 172.21.15.114:36870) 2024-09-13T18:27:40.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.782+0000 7f88fbfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b458da0790 0x55b458da8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:54054/0 (socket says 172.21.15.114:54054) 2024-09-13T18:27:40.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.782+0000 7f8900c2d640 1 -- 172.21.15.114:0/4124540396 learned_addr learned my addr 172.21.15.114:0/4124540396 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:40.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.782+0000 7f88fbfff640 1 -- 172.21.15.114:0/4124540396 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b458dab950 msgr2=0x55b458daa8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:40.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.782+0000 7f88fbfff640 1 --2- 172.21.15.114:0/4124540396 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b458dab950 0x55b458daa8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.782+0000 7f88fbfff640 1 -- 172.21.15.114:0/4124540396 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b458da10b0 msgr2=0x55b458da8890 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:27:40.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.782+0000 7f88fbfff640 1 --2- 172.21.15.114:0/4124540396 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b458da10b0 0x55b458da8890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.782+0000 7f88fbfff640 1 -- 172.21.15.114:0/4124540396 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88ec0099b0 con 0x55b458da0790 2024-09-13T18:27:40.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.783+0000 7f88fbfff640 1 --2- 172.21.15.114:0/4124540396 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b458da0790 0x55b458da8350 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f88e40cca40 tx=0x7f88e4010da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=140ce13ee2fcaf6e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:40.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.784+0000 7f88db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:27:40.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.787+0000 7f88f37fe640 1 --2- 172.21.15.114:0/3879681674 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f88e80a1410 0x7f88e80a38d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f88ec00dbb0 tx=0x7f88ec0048c0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:40.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.787+0000 7f88db7fe640 1 -- 172.21.15.114:0/4124540396 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e40ccec0 con 0x55b458da0790 2024-09-13T18:27:40.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.787+0000 7f88db7fe640 1 -- 172.21.15.114:0/4124540396 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f88e40ce040 con 0x55b458da0790 2024-09-13T18:27:40.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.788+0000 7f890548d740 1 -- 172.21.15.114:0/4124540396 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b458d2db80 con 0x55b458da0790 2024-09-13T18:27:40.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.793+0000 7f88db7fe640 1 -- 172.21.15.114:0/4124540396 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88e40cf4b0 con 0x55b458da0790 2024-09-13T18:27:40.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.793+0000 7f890548d740 10 client.26674 fetch_fsmap learned FSMap version 34 2024-09-13T18:27:40.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.793+0000 7f890548d740 1 -- 172.21.15.114:0/4124540396 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55b458d023c0 con 0x55b458da0790 2024-09-13T18:27:40.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.797+0000 7f88db7fe640 1 -- 172.21.15.114:0/4124540396 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b458d2db80 con 0x55b458da0790 2024-09-13T18:27:40.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.797+0000 7f88db7fe640 1 -- 172.21.15.114:0/4124540396 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f88e40a43f0 con 0x55b458da0790 2024-09-13T18:27:40.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.797+0000 7f890548d740 10 client.26674 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:27:40.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.797+0000 7f890548d740 10 client.26674 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:27:40.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.797+0000 7f890548d740 20 client.26674 populate_metadata read hostname 'smithi114' 2024-09-13T18:27:40.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.797+0000 7f890548d740 1 --2- 172.21.15.114:0/4124540396 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55b458dbb240 0x55b458dbd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:40.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.797+0000 7f890548d740 4 client.26674 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:27:40.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.797+0000 7f890548d740 1 -- 172.21.15.114:0/4124540396 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b458cf61f0 con 0x55b458dbb240 2024-09-13T18:27:40.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.799+0000 7f88f37fe640 1 --2- 172.21.15.114:0/4124540396 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55b458dbb240 0x55b458dbd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:40.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.802+0000 7f88f37fe640 1 --2- 172.21.15.114:0/4124540396 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55b458dbb240 0x55b458dbd660 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8d4cd6932a2d5b8 server_cookie=287d89b471c1d882 in_seq=0 out_seq=0 2024-09-13T18:27:40.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.804+0000 7f88db7fe640 10 client.26674 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:27:40.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.805+0000 7f88db7fe640 1 -- 172.21.15.114:0/4124540396 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55b458cf61f0 con 0x55b458dbb240 2024-09-13T18:27:40.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.805+0000 7f88db7fe640 10 client.26674 handle_command_reply: tid=0 2024-09-13T18:27:40.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.805+0000 7f890548d740 1 client.26674 shutdown 2024-09-13T18:27:40.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.811+0000 7f890548d740 1 -- 172.21.15.114:0/4124540396 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b458da0790 msgr2=0x55b458da8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:40.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.811+0000 7f890548d740 1 --2- 172.21.15.114:0/4124540396 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b458da0790 0x55b458da8350 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f88e40cca40 tx=0x7f88e4010da0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.815+0000 7f890548d740 1 -- 172.21.15.114:0/4124540396 shutdown_connections 2024-09-13T18:27:40.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.815+0000 7f890548d740 1 --2- 172.21.15.114:0/4124540396 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55b458dbb240 0x55b458dbd660 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.815+0000 7f890548d740 1 --2- 172.21.15.114:0/4124540396 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b458dab950 0x55b458daa8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.815+0000 7f890548d740 1 --2- 172.21.15.114:0/4124540396 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b458da10b0 0x55b458da8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.815+0000 7f890548d740 1 --2- 172.21.15.114:0/4124540396 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b458da0790 0x55b458da8350 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.815+0000 7f890548d740 1 -- 172.21.15.114:0/4124540396 >> 172.21.15.114:0/4124540396 conn(0x55b458d9cd00 msgr2=0x55b458da2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:40.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.818+0000 7f88db7fe640 0 client.26674 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:27:40.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.818+0000 7f890548d740 1 -- 172.21.15.114:0/4124540396 shutdown_connections 2024-09-13T18:27:40.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.818+0000 7f890548d740 1 -- 172.21.15.114:0/4124540396 wait complete. 2024-09-13T18:27:40.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.820+0000 7f890548d740 20 client.26674 trim_cache size 0 max 16384 2024-09-13T18:27:40.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.852+0000 7f890548d740 1 Processor -- start 2024-09-13T18:27:40.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.852+0000 7f890548d740 1 -- start start 2024-09-13T18:27:40.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.853+0000 7f890548d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b458da0790 0x55b458d9c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:40.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.853+0000 7f890548d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b458da10b0 0x55b458daa870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:40.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.853+0000 7f890548d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b458dab950 0x55b458dc1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:40.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.853+0000 7f890548d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b458d9dc90 con 0x55b458da10b0 2024-09-13T18:27:40.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.853+0000 7f890548d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b458da66b0 con 0x55b458da0790 2024-09-13T18:27:40.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.853+0000 7f890548d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b458db7300 con 0x55b458dab950 2024-09-13T18:27:40.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.853+0000 7f8900c2d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b458dab950 0x55b458dc1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:40.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.853+0000 7f8900c2d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b458dab950 0x55b458dc1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:54066/0 (socket says 172.21.15.114:54066) 2024-09-13T18:27:40.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.853+0000 7f8900c2d640 1 -- 172.21.15.114:0/3203895511 learned_addr learned my addr 172.21.15.114:0/3203895511 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:40.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.855+0000 7f88fbfff640 1 --2- 172.21.15.114:0/3203895511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b458da0790 0x55b458d9c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:40.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.856+0000 7f88fbfff640 1 -- 172.21.15.114:0/3203895511 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b458dab950 msgr2=0x55b458dc1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:40.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.856+0000 7f88fbfff640 1 --2- 172.21.15.114:0/3203895511 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b458dab950 0x55b458dc1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.856+0000 7f88fbfff640 1 -- 172.21.15.114:0/3203895511 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b458da10b0 msgr2=0x55b458daa870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:27:40.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.856+0000 7f88fbfff640 1 --2- 172.21.15.114:0/3203895511 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b458da10b0 0x55b458daa870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.856+0000 7f88fbfff640 1 -- 172.21.15.114:0/3203895511 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b458ca4770 con 0x55b458da0790 2024-09-13T18:27:40.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.857+0000 7f88fbfff640 1 --2- 172.21.15.114:0/3203895511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b458da0790 0x55b458d9c6a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f88e409bed0 tx=0x7f88e4098110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfc0f0bae8f8bd0f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:40.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.858+0000 7f88f2ffd640 1 -- 172.21.15.114:0/3203895511 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e40cf4b0 con 0x55b458da0790 2024-09-13T18:27:40.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.858+0000 7f88f2ffd640 1 -- 172.21.15.114:0/3203895511 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f88e40a0230 con 0x55b458da0790 2024-09-13T18:27:40.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.858+0000 7f88f2ffd640 1 -- 172.21.15.114:0/3203895511 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e40cccc0 con 0x55b458da0790 2024-09-13T18:27:40.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.859+0000 7f890548d740 1 -- 172.21.15.114:0/3203895511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b458da0790 msgr2=0x55b458d9c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:40.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.859+0000 7f890548d740 1 --2- 172.21.15.114:0/3203895511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b458da0790 0x55b458d9c6a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f88e409bed0 tx=0x7f88e4098110 comp rx=0 tx=0).stop 2024-09-13T18:27:40.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.859+0000 7f890548d740 1 -- 172.21.15.114:0/3203895511 shutdown_connections 2024-09-13T18:27:40.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.859+0000 7f890548d740 1 --2- 172.21.15.114:0/3203895511 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b458dab950 0x55b458dc1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.859+0000 7f890548d740 1 --2- 172.21.15.114:0/3203895511 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b458da10b0 0x55b458daa870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.859+0000 7f890548d740 1 --2- 172.21.15.114:0/3203895511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b458da0790 0x55b458d9c6a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.859+0000 7f890548d740 1 -- 172.21.15.114:0/3203895511 >> 172.21.15.114:0/3203895511 conn(0x55b458d9cd00 msgr2=0x55b458da1930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:40.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.861+0000 7f890548d740 1 -- 172.21.15.114:0/3203895511 shutdown_connections 2024-09-13T18:27:40.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.861+0000 7f890548d740 1 -- 172.21.15.114:0/3203895511 wait complete. 2024-09-13T18:27:40.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.862+0000 7f890548d740 1 -- start start 2024-09-13T18:27:40.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.862+0000 7f890548d740 1 Processor -- start 2024-09-13T18:27:40.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.862+0000 7f890548d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b458da0790 0x55b458db4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:40.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.862+0000 7f890548d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b458da10b0 0x55b458db2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:40.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.862+0000 7f890548d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b458dab950 0x55b458db3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:40.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.862+0000 7f890548d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b458db38a0 con 0x55b458da10b0 2024-09-13T18:27:40.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.862+0000 7f890548d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b458db3a10 con 0x55b458dab950 2024-09-13T18:27:40.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.862+0000 7f890548d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b458db3bd0 con 0x55b458da0790 2024-09-13T18:27:40.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.862+0000 7f890548d740 10 client.0 _ll_register_callbacks cb 0x7f88ec009520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:27:40.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.863+0000 7f88fbfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b458da0790 0x55b458db4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:40.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.863+0000 7f8900c2d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b458dab950 0x55b458db3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:40.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.863+0000 7f8900c2d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b458dab950 0x55b458db3360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:36878/0 (socket says 172.21.15.114:36878) 2024-09-13T18:27:40.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.863+0000 7f8900c2d640 1 -- 172.21.15.114:0/3867858055 learned_addr learned my addr 172.21.15.114:0/3867858055 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:40.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.864+0000 7f8900c2d640 1 -- 172.21.15.114:0/3867858055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b458da0790 msgr2=0x55b458db4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:40.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.864+0000 7f8900c2d640 1 --2- 172.21.15.114:0/3867858055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b458da0790 0x55b458db4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.864+0000 7f8900c2d640 1 -- 172.21.15.114:0/3867858055 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b458da10b0 msgr2=0x55b458db2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:40.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.864+0000 7f8900c2d640 1 --2- 172.21.15.114:0/3867858055 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b458da10b0 0x55b458db2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.864+0000 7f8900c2d640 1 -- 172.21.15.114:0/3867858055 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b458d023c0 con 0x55b458dab950 2024-09-13T18:27:40.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.864+0000 7f8900c2d640 1 --2- 172.21.15.114:0/3867858055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b458dab950 0x55b458db3360 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f88f4009900 tx=0x7f88f4004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebcff35813a62375 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:40.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.864+0000 7f88db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:27:40.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.865+0000 7f88db7fe640 1 -- 172.21.15.114:0/3867858055 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f4043070 con 0x55b458dab950 2024-09-13T18:27:40.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.865+0000 7f890548d740 1 -- 172.21.15.114:0/3867858055 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f88ec004020 con 0x55b458dab950 2024-09-13T18:27:40.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.865+0000 7f88db7fe640 1 -- 172.21.15.114:0/3867858055 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f88f4004fe0 con 0x55b458dab950 2024-09-13T18:27:40.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.866+0000 7f88db7fe640 1 -- 172.21.15.114:0/3867858055 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88f4005d50 con 0x55b458dab950 2024-09-13T18:27:40.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.868+0000 7f890548d740 10 client.26651 fetch_fsmap learned FSMap version 34 2024-09-13T18:27:40.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.868+0000 7f890548d740 1 -- 172.21.15.114:0/3867858055 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55b458cc9e80 con 0x55b458dab950 2024-09-13T18:27:40.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.868+0000 7f88db7fe640 1 -- 172.21.15.114:0/3867858055 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f88ec004020 con 0x55b458dab950 2024-09-13T18:27:40.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.868+0000 7f88db7fe640 1 -- 172.21.15.114:0/3867858055 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f88f4039040 con 0x55b458dab950 2024-09-13T18:27:40.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.869+0000 7f890548d740 10 client.26651 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:27:40.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.869+0000 7f890548d740 10 client.26651 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:27:40.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.869+0000 7f890548d740 20 client.26651 populate_metadata read hostname 'smithi114' 2024-09-13T18:27:40.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.869+0000 7f890548d740 1 --2- 172.21.15.114:0/3867858055 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55b458e12a60 0x55b458e14e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:40.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.869+0000 7f890548d740 4 client.26651 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:27:40.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.869+0000 7f890548d740 1 -- 172.21.15.114:0/3867858055 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b4589f74e0 con 0x55b458e12a60 2024-09-13T18:27:40.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.869+0000 7f88fbfff640 1 --2- 172.21.15.114:0/3867858055 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55b458e12a60 0x55b458e14e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:40.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.870+0000 7f88db7fe640 10 client.26651 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:27:40.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.871+0000 7f88fbfff640 1 --2- 172.21.15.114:0/3867858055 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55b458e12a60 0x55b458e14e80 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d8ba0e4bdb37a38 server_cookie=a466e4982cbc9f0 in_seq=0 out_seq=0 2024-09-13T18:27:40.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.871+0000 7f88db7fe640 1 -- 172.21.15.114:0/3867858055 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b4589f74e0 con 0x55b458e12a60 2024-09-13T18:27:40.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.871+0000 7f88db7fe640 10 client.26651 handle_command_reply: tid=0 2024-09-13T18:27:40.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.871+0000 7f890548d740 1 client.26651 shutdown 2024-09-13T18:27:40.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.877+0000 7f890548d740 1 -- 172.21.15.114:0/3867858055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b458dab950 msgr2=0x55b458db3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:40.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.877+0000 7f890548d740 1 --2- 172.21.15.114:0/3867858055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b458dab950 0x55b458db3360 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f88f4009900 tx=0x7f88f4004270 comp rx=0 tx=0).stop 2024-09-13T18:27:40.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.878+0000 7f890548d740 1 -- 172.21.15.114:0/3867858055 shutdown_connections 2024-09-13T18:27:40.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.878+0000 7f890548d740 1 --2- 172.21.15.114:0/3867858055 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55b458e12a60 0x55b458e14e80 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.878+0000 7f890548d740 1 --2- 172.21.15.114:0/3867858055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b458dab950 0x55b458db3360 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.879+0000 7f890548d740 1 --2- 172.21.15.114:0/3867858055 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b458da10b0 0x55b458db2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.879+0000 7f890548d740 1 --2- 172.21.15.114:0/3867858055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b458da0790 0x55b458db4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.879+0000 7f88db7fe640 0 client.26651 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:27:40.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.879+0000 7f890548d740 1 -- 172.21.15.114:0/3867858055 >> 172.21.15.114:0/3867858055 conn(0x55b458d9cd00 msgr2=0x55b458dc1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:40.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.881+0000 7f890548d740 1 -- 172.21.15.114:0/3867858055 shutdown_connections 2024-09-13T18:27:40.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.881+0000 7f890548d740 1 -- 172.21.15.114:0/3867858055 wait complete. 2024-09-13T18:27:40.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.884+0000 7f890548d740 20 client.26651 trim_cache size 0 max 16384 2024-09-13T18:27:40.888 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:27:40.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.887+0000 7f8902690640 1 -- 172.21.15.114:0/3879681674 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f88e80a1410 msgr2=0x7f88e80a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:40.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.887+0000 7f8902690640 1 --2- 172.21.15.114:0/3879681674 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f88e80a1410 0x7f88e80a38d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f88ec00dbb0 tx=0x7f88ec0048c0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.887+0000 7f8902690640 1 -- 172.21.15.114:0/3879681674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88fc0749e0 msgr2=0x7f88fc071bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:40.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.887+0000 7f8902690640 1 --2- 172.21.15.114:0/3879681674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88fc0749e0 0x7f88fc071bd0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f88e400e9f0 tx=0x7f88e400eeb0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.891+0000 7f8902690640 1 -- 172.21.15.114:0/3879681674 shutdown_connections 2024-09-13T18:27:40.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.891+0000 7f8902690640 1 --2- 172.21.15.114:0/3879681674 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f88e80a1410 0x7f88e80a38d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.891+0000 7f8902690640 1 --2- 172.21.15.114:0/3879681674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88fc110bc0 0x7f88fc06aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.891+0000 7f8902690640 1 --2- 172.21.15.114:0/3879681674 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88fc075300 0x7f88fc072130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.891+0000 7f8902690640 1 --2- 172.21.15.114:0/3879681674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88fc0749e0 0x7f88fc071bd0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:40.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.891+0000 7f8902690640 1 -- 172.21.15.114:0/3879681674 >> 172.21.15.114:0/3879681674 conn(0x7f88fc0fdba0 msgr2=0x7f88fc1112d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:40.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.891+0000 7f8902690640 1 -- 172.21.15.114:0/3879681674 shutdown_connections 2024-09-13T18:27:40.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:40.892+0000 7f8902690640 1 -- 172.21.15.114:0/3879681674 wait complete. 2024-09-13T18:27:41.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:41 smithi134 ceph-mon[34283]: pgmap v159: 129 pgs: 129 active+clean; 17 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 57 MiB/s wr, 969 op/s 2024-09-13T18:27:41.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:41 smithi204 ceph-mon[34777]: pgmap v159: 129 pgs: 129 active+clean; 17 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 57 MiB/s wr, 969 op/s 2024-09-13T18:27:41.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:41 smithi114 ceph-mon[30897]: pgmap v159: 129 pgs: 129 active+clean; 17 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 57 MiB/s wr, 969 op/s 2024-09-13T18:27:41.938 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-13T18:27:41.938 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-09-13T18:27:42.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.178+0000 7ff03f1be640 1 -- 172.21.15.114:0/3092800161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff038107890 msgr2=0x7ff038107d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:42.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.178+0000 7ff03f1be640 1 --2- 172.21.15.114:0/3092800161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff038107890 0x7ff038107d10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff03401a910 tx=0x7ff03403b450 comp rx=0 tx=0).stop 2024-09-13T18:27:42.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.178+0000 7ff03f1be640 1 -- 172.21.15.114:0/3092800161 shutdown_connections 2024-09-13T18:27:42.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.178+0000 7ff03f1be640 1 --2- 172.21.15.114:0/3092800161 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff038119400 0x7ff03811b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.178+0000 7ff03f1be640 1 --2- 172.21.15.114:0/3092800161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff038107890 0x7ff038107d10 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.178+0000 7ff03f1be640 1 --2- 172.21.15.114:0/3092800161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0381074b0 0x7ff03810e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.178+0000 7ff03f1be640 1 -- 172.21.15.114:0/3092800161 >> 172.21.15.114:0/3092800161 conn(0x7ff03806aec0 msgr2=0x7ff03806b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:42.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.178+0000 7ff03f1be640 1 -- 172.21.15.114:0/3092800161 shutdown_connections 2024-09-13T18:27:42.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.178+0000 7ff03f1be640 1 -- 172.21.15.114:0/3092800161 wait complete. 2024-09-13T18:27:42.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.179+0000 7ff03f1be640 1 Processor -- start 2024-09-13T18:27:42.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.179+0000 7ff03f1be640 1 -- start start 2024-09-13T18:27:42.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.179+0000 7ff03f1be640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0381074b0 0x7ff0381188e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:42.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.179+0000 7ff03f1be640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff038107890 0x7ff038118e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:42.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.179+0000 7ff03f1be640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff038119400 0x7ff038114a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:42.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.179+0000 7ff03f1be640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0381151e0 con 0x7ff0381074b0 2024-09-13T18:27:42.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.179+0000 7ff03f1be640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff038115350 con 0x7ff038107890 2024-09-13T18:27:42.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.179+0000 7ff03f1be640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff038115510 con 0x7ff038119400 2024-09-13T18:27:42.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.180+0000 7ff03d9bb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff038107890 0x7ff038118e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:42.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.180+0000 7ff03d9bb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff038107890 0x7ff038118e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:36902/0 (socket says 172.21.15.114:36902) 2024-09-13T18:27:42.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.180+0000 7ff03d9bb640 1 -- 172.21.15.114:0/1400961704 learned_addr learned my addr 172.21.15.114:0/1400961704 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:42.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.180+0000 7ff03e9bd640 1 --2- 172.21.15.114:0/1400961704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff038119400 0x7ff038114a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:42.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.180+0000 7ff03d9bb640 1 -- 172.21.15.114:0/1400961704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff038119400 msgr2=0x7ff038114a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:42.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.180+0000 7ff03d9bb640 1 --2- 172.21.15.114:0/1400961704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff038119400 0x7ff038114a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.180+0000 7ff03d9bb640 1 -- 172.21.15.114:0/1400961704 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0381074b0 msgr2=0x7ff0381188e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:42.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.180+0000 7ff03d9bb640 1 --2- 172.21.15.114:0/1400961704 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0381074b0 0x7ff0381188e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.180+0000 7ff03d9bb640 1 -- 172.21.15.114:0/1400961704 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff03401a570 con 0x7ff038107890 2024-09-13T18:27:42.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.180+0000 7ff03d9bb640 1 --2- 172.21.15.114:0/1400961704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff038107890 0x7ff038118e20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff034005ab0 tx=0x7ff034005510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:42.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.181+0000 7ff02f7fe640 1 -- 172.21.15.114:0/1400961704 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff034040720 con 0x7ff038107890 2024-09-13T18:27:42.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.181+0000 7ff02f7fe640 1 -- 172.21.15.114:0/1400961704 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff034040d10 con 0x7ff038107890 2024-09-13T18:27:42.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.182+0000 7ff02f7fe640 1 -- 172.21.15.114:0/1400961704 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03404cce0 con 0x7ff038107890 2024-09-13T18:27:42.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.182+0000 7ff03f1be640 1 -- 172.21.15.114:0/1400961704 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff038115790 con 0x7ff038107890 2024-09-13T18:27:42.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.182+0000 7ff03f1be640 1 -- 172.21.15.114:0/1400961704 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0381a9330 con 0x7ff038107890 2024-09-13T18:27:42.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.186+0000 7ff043222740 1 Processor -- start 2024-09-13T18:27:42.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.186+0000 7ff043222740 1 -- start start 2024-09-13T18:27:42.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.186+0000 7ff043222740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559a67349e0 0x5559a6734dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:42.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.186+0000 7ff043222740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a6735300 0x5559a673f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:42.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.186+0000 7ff043222740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5559a673fba0 0x5559a6741f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:42.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.186+0000 7ff043222740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559a669fc60 con 0x5559a673fba0 2024-09-13T18:27:42.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.186+0000 7ff043222740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559a669d3f0 con 0x5559a67349e0 2024-09-13T18:27:42.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.186+0000 7ff043222740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559a6698ce0 con 0x5559a6735300 2024-09-13T18:27:42.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.187+0000 7ff03e9bd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559a67349e0 0x5559a6734dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:42.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.187+0000 7ff03e9bd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559a67349e0 0x5559a6734dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:36914/0 (socket says 172.21.15.114:36914) 2024-09-13T18:27:42.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.187+0000 7ff03e9bd640 1 -- 172.21.15.114:0/3361698105 learned_addr learned my addr 172.21.15.114:0/3361698105 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:42.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.187+0000 7ff03e1bc640 1 --2- 172.21.15.114:0/3361698105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a6735300 0x5559a673f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:42.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.187+0000 7ff03e9bd640 1 -- 172.21.15.114:0/3361698105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a6735300 msgr2=0x5559a673f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:42.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.187+0000 7ff03e9bd640 1 --2- 172.21.15.114:0/3361698105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a6735300 0x5559a673f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.187+0000 7ff03e9bd640 1 -- 172.21.15.114:0/3361698105 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5559a673fba0 msgr2=0x5559a6741f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:42.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.187+0000 7ff03e9bd640 1 --2- 172.21.15.114:0/3361698105 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5559a673fba0 0x5559a6741f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.187+0000 7ff03e9bd640 1 -- 172.21.15.114:0/3361698105 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5559a6654970 con 0x5559a67349e0 2024-09-13T18:27:42.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.188+0000 7ff03e9bd640 1 --2- 172.21.15.114:0/3361698105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559a67349e0 0x5559a6734dc0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff030009fd0 tx=0x7ff030009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d15e1a672cb9be05 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:42.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.188+0000 7ff02f7fe640 1 -- 172.21.15.114:0/1400961704 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7ff03404c460 con 0x7ff038107890 2024-09-13T18:27:42.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.188+0000 7ff02ce05640 1 -- 172.21.15.114:0/3361698105 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff030010040 con 0x5559a67349e0 2024-09-13T18:27:42.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.189+0000 7ff02ce05640 1 -- 172.21.15.114:0/3361698105 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff030013c20 con 0x5559a67349e0 2024-09-13T18:27:42.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.189+0000 7ff02ce05640 1 -- 172.21.15.114:0/3361698105 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff030014bd0 con 0x5559a67349e0 2024-09-13T18:27:42.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.189+0000 7ff02f7fe640 1 --2- 172.21.15.114:0/1400961704 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff0240a13d0 0x7ff0240a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:42.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.189+0000 7ff02f7fe640 1 -- 172.21.15.114:0/1400961704 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ff03404ce40 con 0x7ff038107890 2024-09-13T18:27:42.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.190+0000 7ff043222740 1 -- 172.21.15.114:0/3361698105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559a67349e0 msgr2=0x5559a6734dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:42.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.190+0000 7ff043222740 1 --2- 172.21.15.114:0/3361698105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559a67349e0 0x5559a6734dc0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff030009fd0 tx=0x7ff030009650 comp rx=0 tx=0).stop 2024-09-13T18:27:42.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.190+0000 7ff043222740 1 -- 172.21.15.114:0/3361698105 shutdown_connections 2024-09-13T18:27:42.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.190+0000 7ff043222740 1 --2- 172.21.15.114:0/3361698105 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5559a673fba0 0x5559a6741f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.190+0000 7ff043222740 1 --2- 172.21.15.114:0/3361698105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a6735300 0x5559a673f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.190+0000 7ff043222740 1 --2- 172.21.15.114:0/3361698105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559a67349e0 0x5559a6734dc0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.190+0000 7ff043222740 1 -- 172.21.15.114:0/3361698105 >> 172.21.15.114:0/3361698105 conn(0x5559a6730f70 msgr2=0x5559a6731350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:42.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.190+0000 7ff03e1bc640 1 --2- 172.21.15.114:0/1400961704 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff0240a13d0 0x7ff0240a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:42.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.190+0000 7ff043222740 1 -- 172.21.15.114:0/3361698105 shutdown_connections 2024-09-13T18:27:42.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.190+0000 7ff043222740 1 -- 172.21.15.114:0/3361698105 wait complete. 2024-09-13T18:27:42.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.190+0000 7ff043222740 1 -- start start 2024-09-13T18:27:42.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.191+0000 7ff043222740 1 Processor -- start 2024-09-13T18:27:42.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.191+0000 7ff043222740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a67349e0 0x5559a673f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:42.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.191+0000 7ff043222740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5559a6735300 0x5559a673c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:42.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.191+0000 7ff043222740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559a673fba0 0x5559a673cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:42.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.191+0000 7ff043222740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559a6656f80 con 0x5559a6735300 2024-09-13T18:27:42.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.191+0000 7ff043222740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559a6656c20 con 0x5559a673fba0 2024-09-13T18:27:42.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.191+0000 7ff043222740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559a6623ab0 con 0x5559a67349e0 2024-09-13T18:27:42.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.191+0000 7ff043222740 10 client.0 _ll_register_callbacks cb 0x5559a6692350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:27:42.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.191+0000 7ff03e9bd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559a673fba0 0x5559a673cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:42.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.191+0000 7ff03e9bd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559a673fba0 0x5559a673cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:36930/0 (socket says 172.21.15.114:36930) 2024-09-13T18:27:42.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.191+0000 7ff03e9bd640 1 -- 172.21.15.114:0/1420071854 learned_addr learned my addr 172.21.15.114:0/1420071854 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:42.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.191+0000 7ff03e1bc640 1 --2- 172.21.15.114:0/1420071854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a67349e0 0x5559a673f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:42.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.192+0000 7ff03e1bc640 1 --2- 172.21.15.114:0/1400961704 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff0240a13d0 0x7ff0240a3890 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x5559a6730a70 tx=0x7ff028009ce0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:42.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.192+0000 7ff03e9bd640 1 -- 172.21.15.114:0/1420071854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a67349e0 msgr2=0x5559a673f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:42.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.192+0000 7ff03e9bd640 1 --2- 172.21.15.114:0/1420071854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a67349e0 0x5559a673f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.192+0000 7ff03e9bd640 1 -- 172.21.15.114:0/1420071854 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5559a6735300 msgr2=0x5559a673c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:42.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.192+0000 7ff03e9bd640 1 --2- 172.21.15.114:0/1420071854 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5559a6735300 0x5559a673c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.192+0000 7ff03e9bd640 1 -- 172.21.15.114:0/1420071854 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff030009c00 con 0x5559a673fba0 2024-09-13T18:27:42.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.192+0000 7ff03e9bd640 1 --2- 172.21.15.114:0/1420071854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559a673fba0 0x5559a673cf10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff030009900 tx=0x7ff030013fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1064686c1e75c5af server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:42.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.192+0000 7ff021ffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:27:42.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.195+0000 7ff021ffb640 1 -- 172.21.15.114:0/1420071854 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff030010040 con 0x5559a673fba0 2024-09-13T18:27:42.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.195+0000 7ff021ffb640 1 -- 172.21.15.114:0/1420071854 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ff030014910 con 0x5559a673fba0 2024-09-13T18:27:42.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.199+0000 7ff043222740 1 -- 172.21.15.114:0/1420071854 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5559a638c590 con 0x5559a673fba0 2024-09-13T18:27:42.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.201+0000 7ff021ffb640 1 -- 172.21.15.114:0/1420071854 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff030013aa0 con 0x5559a673fba0 2024-09-13T18:27:42.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.202+0000 7ff043222740 10 client.26663 fetch_fsmap learned FSMap version 34 2024-09-13T18:27:42.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.202+0000 7ff043222740 1 -- 172.21.15.114:0/1420071854 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5559a6697360 con 0x5559a673fba0 2024-09-13T18:27:42.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.206+0000 7ff021ffb640 1 -- 172.21.15.114:0/1420071854 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5559a638c590 con 0x5559a673fba0 2024-09-13T18:27:42.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.207+0000 7ff021ffb640 1 -- 172.21.15.114:0/1420071854 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff030008030 con 0x5559a673fba0 2024-09-13T18:27:42.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.207+0000 7ff043222740 10 client.26663 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:27:42.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.207+0000 7ff043222740 10 client.26663 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:27:42.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.207+0000 7ff043222740 20 client.26663 populate_metadata read hostname 'smithi114' 2024-09-13T18:27:42.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.207+0000 7ff043222740 1 --2- 172.21.15.114:0/1420071854 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5559a674f9a0 0x5559a6751d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:42.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.207+0000 7ff043222740 4 client.26663 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:27:42.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.207+0000 7ff043222740 1 -- 172.21.15.114:0/1420071854 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5559a639d860 con 0x5559a674f9a0 2024-09-13T18:27:42.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.210+0000 7ff03e1bc640 1 --2- 172.21.15.114:0/1420071854 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5559a674f9a0 0x5559a6751d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:42.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.213+0000 7ff03e1bc640 1 --2- 172.21.15.114:0/1420071854 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5559a674f9a0 0x5559a6751d90 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6ded513a375ff07 server_cookie=e087d9cc6bbc116a in_seq=0 out_seq=0 2024-09-13T18:27:42.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.215+0000 7ff021ffb640 10 client.26663 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:27:42.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.222+0000 7ff021ffb640 1 -- 172.21.15.114:0/1420071854 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5559a639d860 con 0x5559a674f9a0 2024-09-13T18:27:42.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.222+0000 7ff021ffb640 10 client.26663 handle_command_reply: tid=0 2024-09-13T18:27:42.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.222+0000 7ff043222740 1 client.26663 shutdown 2024-09-13T18:27:42.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.223+0000 7ff043222740 1 -- 172.21.15.114:0/1420071854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559a673fba0 msgr2=0x5559a673cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:42.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.223+0000 7ff043222740 1 --2- 172.21.15.114:0/1420071854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559a673fba0 0x5559a673cf10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff030009900 tx=0x7ff030013fc0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.230+0000 7ff043222740 1 -- 172.21.15.114:0/1420071854 shutdown_connections 2024-09-13T18:27:42.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.230+0000 7ff043222740 1 --2- 172.21.15.114:0/1420071854 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5559a674f9a0 0x5559a6751d90 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.230+0000 7ff043222740 1 --2- 172.21.15.114:0/1420071854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559a673fba0 0x5559a673cf10 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.230+0000 7ff043222740 1 --2- 172.21.15.114:0/1420071854 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5559a6735300 0x5559a673c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.230+0000 7ff043222740 1 --2- 172.21.15.114:0/1420071854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a67349e0 0x5559a673f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.230+0000 7ff043222740 1 -- 172.21.15.114:0/1420071854 >> 172.21.15.114:0/1420071854 conn(0x5559a6730f70 msgr2=0x5559a6730160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:42.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.230+0000 7ff021ffb640 0 client.26663 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:27:42.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.237+0000 7ff043222740 1 -- 172.21.15.114:0/1420071854 shutdown_connections 2024-09-13T18:27:42.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.244+0000 7ff043222740 1 -- 172.21.15.114:0/1420071854 wait complete. 2024-09-13T18:27:42.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.248+0000 7ff043222740 20 client.26663 trim_cache size 0 max 16384 2024-09-13T18:27:42.252 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:42 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:27:42.252 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:42 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:27:42.252 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:42 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:27:42.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.312+0000 7ff043222740 1 Processor -- start 2024-09-13T18:27:42.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.312+0000 7ff043222740 1 -- start start 2024-09-13T18:27:42.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.313+0000 7ff043222740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5559a6730f70 0x5559a6746860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:42.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.313+0000 7ff043222740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a67349e0 0x5559a6746da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:42.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.313+0000 7ff03e1bc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5559a6730f70 0x5559a6746860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:42.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.313+0000 7ff03e1bc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5559a6730f70 0x5559a6746860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36386/0 (socket says 172.21.15.114:36386) 2024-09-13T18:27:42.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.313+0000 7ff03e1bc640 1 -- 172.21.15.114:0/2680352191 learned_addr learned my addr 172.21.15.114:0/2680352191 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:42.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.313+0000 7ff043222740 1 --2- 172.21.15.114:0/2680352191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559a6735300 0x5559a673b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:42.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.313+0000 7ff043222740 1 -- 172.21.15.114:0/2680352191 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559a674c670 con 0x5559a6730f70 2024-09-13T18:27:42.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.313+0000 7ff043222740 1 -- 172.21.15.114:0/2680352191 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559a674b780 con 0x5559a6735300 2024-09-13T18:27:42.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.313+0000 7ff043222740 1 -- 172.21.15.114:0/2680352191 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559a6735f00 con 0x5559a67349e0 2024-09-13T18:27:42.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.314+0000 7ff03d9bb640 1 --2- 172.21.15.114:0/2680352191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a67349e0 0x5559a6746da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:42.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.314+0000 7ff03e1bc640 1 -- 172.21.15.114:0/2680352191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a67349e0 msgr2=0x5559a6746da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:42.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.314+0000 7ff03e1bc640 1 --2- 172.21.15.114:0/2680352191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a67349e0 0x5559a6746da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.314+0000 7ff03e1bc640 1 -- 172.21.15.114:0/2680352191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559a6735300 msgr2=0x5559a673b160 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:27:42.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.314+0000 7ff03e1bc640 1 --2- 172.21.15.114:0/2680352191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559a6735300 0x5559a673b160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.314+0000 7ff03e1bc640 1 -- 172.21.15.114:0/2680352191 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5559a66391c0 con 0x5559a6730f70 2024-09-13T18:27:42.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.315+0000 7ff03d9bb640 1 --2- 172.21.15.114:0/2680352191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a67349e0 0x5559a6746da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:27:42.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.316+0000 7ff03e1bc640 1 --2- 172.21.15.114:0/2680352191 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5559a6730f70 0x5559a6746860 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x5559a6752630 tx=0x7ff028019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb2c98716ce8f27e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:42.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.319+0000 7ff02d606640 1 -- 172.21.15.114:0/2680352191 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff028025070 con 0x5559a6730f70 2024-09-13T18:27:42.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.319+0000 7ff02d606640 1 -- 172.21.15.114:0/2680352191 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff02800ee70 con 0x5559a6730f70 2024-09-13T18:27:42.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.320+0000 7ff02d606640 1 -- 172.21.15.114:0/2680352191 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff028026700 con 0x5559a6730f70 2024-09-13T18:27:42.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.320+0000 7ff043222740 1 -- 172.21.15.114:0/2680352191 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5559a6730f70 msgr2=0x5559a6746860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:42.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.320+0000 7ff043222740 1 --2- 172.21.15.114:0/2680352191 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5559a6730f70 0x5559a6746860 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x5559a6752630 tx=0x7ff028019000 comp rx=0 tx=0).stop 2024-09-13T18:27:42.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.320+0000 7ff043222740 1 -- 172.21.15.114:0/2680352191 shutdown_connections 2024-09-13T18:27:42.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.320+0000 7ff043222740 1 --2- 172.21.15.114:0/2680352191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559a6735300 0x5559a673b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.320+0000 7ff043222740 1 --2- 172.21.15.114:0/2680352191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a67349e0 0x5559a6746da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.320+0000 7ff043222740 1 --2- 172.21.15.114:0/2680352191 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5559a6730f70 0x5559a6746860 secure :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0x5559a6752630 tx=0x7ff028019000 comp rx=0 tx=0).stop 2024-09-13T18:27:42.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.320+0000 7ff043222740 1 -- 172.21.15.114:0/2680352191 >> 172.21.15.114:0/2680352191 conn(0x5559a673f6f0 msgr2=0x5559a6737b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:42.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.320+0000 7ff043222740 1 -- 172.21.15.114:0/2680352191 shutdown_connections 2024-09-13T18:27:42.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.320+0000 7ff043222740 1 -- 172.21.15.114:0/2680352191 wait complete. 2024-09-13T18:27:42.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.321+0000 7ff043222740 1 -- start start 2024-09-13T18:27:42.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.321+0000 7ff043222740 1 Processor -- start 2024-09-13T18:27:42.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.322+0000 7ff043222740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5559a67349e0 0x5559a67468f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:42.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.322+0000 7ff043222740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a6735300 0x5559a6743cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:42.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.322+0000 7ff043222740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559a673fba0 0x5559a67441f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:42.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.322+0000 7ff043222740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559a6744730 con 0x5559a67349e0 2024-09-13T18:27:42.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.322+0000 7ff043222740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559a67448a0 con 0x5559a673fba0 2024-09-13T18:27:42.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.322+0000 7ff03e9bd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559a673fba0 0x5559a67441f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:42.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.322+0000 7ff03e9bd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559a673fba0 0x5559a67441f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:36952/0 (socket says 172.21.15.114:36952) 2024-09-13T18:27:42.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.322+0000 7ff03e9bd640 1 -- 172.21.15.114:0/784921594 learned_addr learned my addr 172.21.15.114:0/784921594 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:42.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.323+0000 7ff043222740 1 -- 172.21.15.114:0/784921594 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559a6744a60 con 0x5559a6735300 2024-09-13T18:27:42.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.323+0000 7ff043222740 10 client.0 _ll_register_callbacks cb 0x7ff028006ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:27:42.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.323+0000 7ff03e9bd640 1 -- 172.21.15.114:0/784921594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a6735300 msgr2=0x5559a6743cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:27:42.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.323+0000 7ff03e9bd640 1 --2- 172.21.15.114:0/784921594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a6735300 0x5559a6743cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.323+0000 7ff03e9bd640 1 -- 172.21.15.114:0/784921594 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5559a67349e0 msgr2=0x5559a67468f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:42.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.323+0000 7ff03e9bd640 1 --2- 172.21.15.114:0/784921594 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5559a67349e0 0x5559a67468f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.323+0000 7ff03e9bd640 1 -- 172.21.15.114:0/784921594 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff02800eba0 con 0x5559a673fba0 2024-09-13T18:27:42.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.324+0000 7ff03e9bd640 1 --2- 172.21.15.114:0/784921594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559a673fba0 0x5559a67441f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff030002230 tx=0x7ff030009eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e37fd14e2b11260a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:42.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.324+0000 7ff021ffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:27:42.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.324+0000 7ff021ffb640 1 -- 172.21.15.114:0/784921594 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03001eb60 con 0x5559a673fba0 2024-09-13T18:27:42.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.324+0000 7ff021ffb640 1 -- 172.21.15.114:0/784921594 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ff030009650 con 0x5559a673fba0 2024-09-13T18:27:42.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.324+0000 7ff043222740 1 -- 172.21.15.114:0/784921594 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff030013c20 con 0x5559a673fba0 2024-09-13T18:27:42.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.325+0000 7ff021ffb640 1 -- 172.21.15.114:0/784921594 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff030013ab0 con 0x5559a673fba0 2024-09-13T18:27:42.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.325+0000 7ff043222740 10 client.26666 fetch_fsmap learned FSMap version 34 2024-09-13T18:27:42.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.325+0000 7ff043222740 1 -- 172.21.15.114:0/784921594 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5559a665e990 con 0x5559a673fba0 2024-09-13T18:27:42.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.325+0000 7ff021ffb640 1 -- 172.21.15.114:0/784921594 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff03000e070 con 0x5559a673fba0 2024-09-13T18:27:42.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.325+0000 7ff021ffb640 1 -- 172.21.15.114:0/784921594 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff03003a070 con 0x5559a673fba0 2024-09-13T18:27:42.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.325+0000 7ff043222740 10 client.26666 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:27:42.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.326+0000 7ff043222740 10 client.26666 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:27:42.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.326+0000 7ff043222740 20 client.26666 populate_metadata read hostname 'smithi114' 2024-09-13T18:27:42.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.326+0000 7ff043222740 1 --2- 172.21.15.114:0/784921594 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5559a67a7e60 0x5559a67aa280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:42.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.326+0000 7ff043222740 4 client.26666 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-13T18:27:42.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.326+0000 7ff043222740 1 -- 172.21.15.114:0/784921594 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x7ff030013c20 con 0x5559a67a7e60 2024-09-13T18:27:42.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.327+0000 7ff03e1bc640 1 --2- 172.21.15.114:0/784921594 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5559a67a7e60 0x5559a67aa280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:42.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.331+0000 7ff03e1bc640 1 --2- 172.21.15.114:0/784921594 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5559a67a7e60 0x5559a67aa280 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8a8b66e362318a4 server_cookie=c7eaf95a07887f92 in_seq=0 out_seq=0 2024-09-13T18:27:42.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.331+0000 7ff021ffb640 10 client.26666 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:27:42.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.332+0000 7ff021ffb640 1 -- 172.21.15.114:0/784921594 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x7ff030013c20 con 0x5559a67a7e60 2024-09-13T18:27:42.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.332+0000 7ff021ffb640 10 client.26666 handle_command_reply: tid=0 2024-09-13T18:27:42.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.333+0000 7ff043222740 1 client.26666 shutdown 2024-09-13T18:27:42.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.334+0000 7ff043222740 1 -- 172.21.15.114:0/784921594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559a673fba0 msgr2=0x5559a67441f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:42.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.334+0000 7ff043222740 1 --2- 172.21.15.114:0/784921594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559a673fba0 0x5559a67441f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff030002230 tx=0x7ff030009eb0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.338+0000 7ff043222740 1 -- 172.21.15.114:0/784921594 shutdown_connections 2024-09-13T18:27:42.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.338+0000 7ff043222740 1 --2- 172.21.15.114:0/784921594 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5559a67a7e60 0x5559a67aa280 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.338+0000 7ff043222740 1 --2- 172.21.15.114:0/784921594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559a673fba0 0x5559a67441f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.338+0000 7ff043222740 1 --2- 172.21.15.114:0/784921594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a6735300 0x5559a6743cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.338+0000 7ff043222740 1 --2- 172.21.15.114:0/784921594 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5559a67349e0 0x5559a67468f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.338+0000 7ff043222740 1 -- 172.21.15.114:0/784921594 >> 172.21.15.114:0/784921594 conn(0x5559a673f6f0 msgr2=0x5559a673b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:42.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.341+0000 7ff021ffb640 0 client.26666 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:27:42.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.341+0000 7ff043222740 1 -- 172.21.15.114:0/784921594 shutdown_connections 2024-09-13T18:27:42.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.341+0000 7ff043222740 1 -- 172.21.15.114:0/784921594 wait complete. 2024-09-13T18:27:42.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.348+0000 7ff043222740 20 client.26666 trim_cache size 0 max 16384 2024-09-13T18:27:42.353 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:27:42.353 INFO:teuthology.orchestra.run.smithi114.stdout: "return_code": 0, 2024-09-13T18:27:42.353 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_tag": "23bd2e7e-f0d4-47ed-9104-84bf71750a03", 2024-09-13T18:27:42.353 INFO:teuthology.orchestra.run.smithi114.stdout: "mode": "asynchronous" 2024-09-13T18:27:42.354 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:27:42.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.354+0000 7ff03f1be640 1 -- 172.21.15.114:0/1400961704 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff0240a13d0 msgr2=0x7ff0240a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:42.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.354+0000 7ff03f1be640 1 --2- 172.21.15.114:0/1400961704 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff0240a13d0 0x7ff0240a3890 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x5559a6730a70 tx=0x7ff028009ce0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.354+0000 7ff03f1be640 1 -- 172.21.15.114:0/1400961704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff038107890 msgr2=0x7ff038118e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:42.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.354+0000 7ff03f1be640 1 --2- 172.21.15.114:0/1400961704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff038107890 0x7ff038118e20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff034005ab0 tx=0x7ff034005510 comp rx=0 tx=0).stop 2024-09-13T18:27:42.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.355+0000 7ff03f1be640 1 -- 172.21.15.114:0/1400961704 shutdown_connections 2024-09-13T18:27:42.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.355+0000 7ff03f1be640 1 --2- 172.21.15.114:0/1400961704 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff0240a13d0 0x7ff0240a3890 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.355+0000 7ff03f1be640 1 --2- 172.21.15.114:0/1400961704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff038119400 0x7ff038114a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.355+0000 7ff03f1be640 1 --2- 172.21.15.114:0/1400961704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff038107890 0x7ff038118e20 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.355+0000 7ff03f1be640 1 --2- 172.21.15.114:0/1400961704 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0381074b0 0x7ff0381188e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.355+0000 7ff03f1be640 1 -- 172.21.15.114:0/1400961704 >> 172.21.15.114:0/1400961704 conn(0x7ff03806aec0 msgr2=0x7ff038070040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:42.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.360+0000 7ff03f1be640 1 -- 172.21.15.114:0/1400961704 shutdown_connections 2024-09-13T18:27:42.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.360+0000 7ff03f1be640 1 -- 172.21.15.114:0/1400961704 wait complete. 2024-09-13T18:27:42.388 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T18:27:42.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:42 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:27:42.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:42 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:27:42.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:42 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:27:42.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.642+0000 7f5d99d87640 1 -- 172.21.15.114:0/2972078160 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d94104c00 msgr2=0x7f5d94104fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:42.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.642+0000 7f5d99d87640 1 --2- 172.21.15.114:0/2972078160 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d94104c00 0x7f5d94104fe0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c009be0 tx=0x7f5d8c02a710 comp rx=0 tx=0).stop 2024-09-13T18:27:42.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.642+0000 7f5d99d87640 1 -- 172.21.15.114:0/2972078160 shutdown_connections 2024-09-13T18:27:42.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.642+0000 7f5d99d87640 1 --2- 172.21.15.114:0/2972078160 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d94068060 0x7f5d94102750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.642+0000 7f5d99d87640 1 --2- 172.21.15.114:0/2972078160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d940675a0 0x7f5d940679e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.642+0000 7f5d99d87640 1 --2- 172.21.15.114:0/2972078160 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d94104c00 0x7f5d94104fe0 secure :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c009be0 tx=0x7f5d8c02a710 comp rx=0 tx=0).stop 2024-09-13T18:27:42.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.642+0000 7f5d99d87640 1 -- 172.21.15.114:0/2972078160 >> 172.21.15.114:0/2972078160 conn(0x7f5d94074980 msgr2=0x7f5d94074d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:42.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.647+0000 7f5d99d87640 1 -- 172.21.15.114:0/2972078160 shutdown_connections 2024-09-13T18:27:42.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.650+0000 7f5d99d87640 1 -- 172.21.15.114:0/2972078160 wait complete. 2024-09-13T18:27:42.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.651+0000 7f5d99d87640 1 Processor -- start 2024-09-13T18:27:42.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.657+0000 7f5d99d87640 1 -- start start 2024-09-13T18:27:42.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.658+0000 7f5d99d87640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d940675a0 0x7f5d94071140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:42.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.658+0000 7f5d99d87640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d94068060 0x7f5d94071680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:42.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.658+0000 7f5d99d87640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d941a13e0 0x7f5d941a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:42.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.658+0000 7f5d99d87640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d941a1e60 con 0x7f5d94068060 2024-09-13T18:27:42.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.658+0000 7f5d99d87640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d941a1fd0 con 0x7f5d941a13e0 2024-09-13T18:27:42.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.658+0000 7f5d99d87640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d941a2190 con 0x7f5d940675a0 2024-09-13T18:27:42.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.658+0000 7f5d93fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d94068060 0x7f5d94071680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:42.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.658+0000 7f5d93fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d94068060 0x7f5d94071680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36412/0 (socket says 172.21.15.114:36412) 2024-09-13T18:27:42.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.658+0000 7f5d93fff640 1 -- 172.21.15.114:0/1354587455 learned_addr learned my addr 172.21.15.114:0/1354587455 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:42.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.659+0000 7f5d93fff640 1 -- 172.21.15.114:0/1354587455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d940675a0 msgr2=0x7f5d94071140 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:27:42.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.659+0000 7f5d93fff640 1 --2- 172.21.15.114:0/1354587455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d940675a0 0x7f5d94071140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.659+0000 7f5d93fff640 1 -- 172.21.15.114:0/1354587455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d941a13e0 msgr2=0x7f5d941a1890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:27:42.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.659+0000 7f5d93fff640 1 --2- 172.21.15.114:0/1354587455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d941a13e0 0x7f5d941a1890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.659+0000 7f5d93fff640 1 -- 172.21.15.114:0/1354587455 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d8c009840 con 0x7f5d94068060 2024-09-13T18:27:42.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.659+0000 7f5d93fff640 1 --2- 172.21.15.114:0/1354587455 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d94068060 0x7f5d94071680 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f5d8400b450 tx=0x7f5d8400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:42.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.660+0000 7f5d91ffb640 1 -- 172.21.15.114:0/1354587455 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d84011890 con 0x7f5d94068060 2024-09-13T18:27:42.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.660+0000 7f5d91ffb640 1 -- 172.21.15.114:0/1354587455 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5d84011e80 con 0x7f5d94068060 2024-09-13T18:27:42.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.660+0000 7f5d91ffb640 1 -- 172.21.15.114:0/1354587455 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8400f340 con 0x7f5d94068060 2024-09-13T18:27:42.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.660+0000 7f5d99d87640 1 -- 172.21.15.114:0/1354587455 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d941a96f0 con 0x7f5d94068060 2024-09-13T18:27:42.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.660+0000 7f5d99d87640 1 -- 172.21.15.114:0/1354587455 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d941a9b80 con 0x7f5d94068060 2024-09-13T18:27:42.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.665+0000 7f5d91ffb640 1 -- 172.21.15.114:0/1354587455 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f5d8400f4a0 con 0x7f5d94068060 2024-09-13T18:27:42.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.665+0000 7f5d9dde7740 1 Processor -- start 2024-09-13T18:27:42.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.666+0000 7f5d9dde7740 1 -- start start 2024-09-13T18:27:42.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.666+0000 7f5d9dde7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c71de3ffe0 0x55c71de403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:42.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.667+0000 7f5d91ffb640 1 --2- 172.21.15.114:0/1354587455 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f5d7c0a1410 0x7f5d7c0a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:42.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.667+0000 7f5d9dde7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c71de40900 0x55c71de4ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:42.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.667+0000 7f5d9dde7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c71de4b1a0 0x55c71de4d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:42.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.667+0000 7f5d9dde7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c71ddaaf20 con 0x55c71de4b1a0 2024-09-13T18:27:42.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.667+0000 7f5d9dde7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c71dda86b0 con 0x55c71de3ffe0 2024-09-13T18:27:42.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.667+0000 7f5d9dde7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c71dda3fa0 con 0x55c71de40900 2024-09-13T18:27:42.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.667+0000 7f5d99586640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c71de3ffe0 0x55c71de403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:42.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.667+0000 7f5d99586640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c71de3ffe0 0x55c71de403c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:36968/0 (socket says 172.21.15.114:36968) 2024-09-13T18:27:42.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.667+0000 7f5d99586640 1 -- 172.21.15.114:0/2566065047 learned_addr learned my addr 172.21.15.114:0/2566065047 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:42.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.667+0000 7f5d91ffb640 1 -- 172.21.15.114:0/1354587455 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f5d840cdc40 con 0x7f5d94068060 2024-09-13T18:27:42.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.667+0000 7f5d99586640 1 --2- 172.21.15.114:0/2566065047 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c71de4b1a0 0x55c71de4d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:42.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.667+0000 7f5d99586640 1 -- 172.21.15.114:0/2566065047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c71de40900 msgr2=0x55c71de4ac60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:27:42.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.667+0000 7f5d93fff640 1 --2- 172.21.15.114:0/1354587455 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f5d7c0a1410 0x7f5d7c0a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:42.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.667+0000 7f5d99586640 1 --2- 172.21.15.114:0/2566065047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c71de40900 0x55c71de4ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.667+0000 7f5d99586640 1 -- 172.21.15.114:0/2566065047 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c71de4b1a0 msgr2=0x55c71de4d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:42.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.667+0000 7f5d99586640 1 --2- 172.21.15.114:0/2566065047 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c71de4b1a0 0x55c71de4d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.667+0000 7f5d99586640 1 -- 172.21.15.114:0/2566065047 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c71dd5e840 con 0x55c71de3ffe0 2024-09-13T18:27:42.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.668+0000 7f5d93fff640 1 --2- 172.21.15.114:0/1354587455 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f5d7c0a1410 0x7f5d7c0a38d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5d84097520 tx=0x7f5d8400e000 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:42.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.670+0000 7f5d99586640 1 --2- 172.21.15.114:0/2566065047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c71de3ffe0 0x55c71de403c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5d8803ef90 tx=0x7f5d8803e410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43b97485fa3d8ccd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:42.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.670+0000 7f5d7affd640 1 -- 172.21.15.114:0/2566065047 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d88042c70 con 0x55c71de3ffe0 2024-09-13T18:27:42.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.670+0000 7f5d7affd640 1 -- 172.21.15.114:0/2566065047 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5d8804b070 con 0x55c71de3ffe0 2024-09-13T18:27:42.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.670+0000 7f5d7affd640 1 -- 172.21.15.114:0/2566065047 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d88004b00 con 0x55c71de3ffe0 2024-09-13T18:27:42.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.672+0000 7f5d9dde7740 1 -- 172.21.15.114:0/2566065047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c71de3ffe0 msgr2=0x55c71de403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:42.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.672+0000 7f5d9dde7740 1 --2- 172.21.15.114:0/2566065047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c71de3ffe0 0x55c71de403c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5d8803ef90 tx=0x7f5d8803e410 comp rx=0 tx=0).stop 2024-09-13T18:27:42.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.672+0000 7f5d9dde7740 1 -- 172.21.15.114:0/2566065047 shutdown_connections 2024-09-13T18:27:42.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.672+0000 7f5d9dde7740 1 --2- 172.21.15.114:0/2566065047 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c71de4b1a0 0x55c71de4d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.672+0000 7f5d9dde7740 1 --2- 172.21.15.114:0/2566065047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c71de40900 0x55c71de4ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.672+0000 7f5d9dde7740 1 --2- 172.21.15.114:0/2566065047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c71de3ffe0 0x55c71de403c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.672+0000 7f5d9dde7740 1 -- 172.21.15.114:0/2566065047 >> 172.21.15.114:0/2566065047 conn(0x55c71de3c570 msgr2=0x55c71de3c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:42.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.673+0000 7f5d9dde7740 1 -- 172.21.15.114:0/2566065047 shutdown_connections 2024-09-13T18:27:42.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.673+0000 7f5d9dde7740 1 -- 172.21.15.114:0/2566065047 wait complete. 2024-09-13T18:27:42.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.673+0000 7f5d9dde7740 1 -- start start 2024-09-13T18:27:42.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.673+0000 7f5d9dde7740 1 Processor -- start 2024-09-13T18:27:42.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.674+0000 7f5d9dde7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c71de3ffe0 0x55c71de4aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:42.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.674+0000 7f5d9dde7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c71de40900 0x55c71de47d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:42.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.674+0000 7f5d9dde7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c71de4b1a0 0x55c71de48310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:42.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.674+0000 7f5d9dde7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c71dd452a0 con 0x55c71de40900 2024-09-13T18:27:42.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.674+0000 7f5d9dde7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c71da7c430 con 0x55c71de4b1a0 2024-09-13T18:27:42.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.674+0000 7f5d9dde7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c71dd27680 con 0x55c71de3ffe0 2024-09-13T18:27:42.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.674+0000 7f5d9dde7740 10 client.0 _ll_register_callbacks cb 0x55c71dd9d610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:27:42.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.674+0000 7f5d99586640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c71de3ffe0 0x55c71de4aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:42.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.674+0000 7f5d99586640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c71de3ffe0 0x55c71de4aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:54156/0 (socket says 172.21.15.114:54156) 2024-09-13T18:27:42.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.674+0000 7f5d99586640 1 -- 172.21.15.114:0/2792135779 learned_addr learned my addr 172.21.15.114:0/2792135779 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:42.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.674+0000 7f5d99586640 1 --2- 172.21.15.114:0/2792135779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c71de4b1a0 0x55c71de48310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:42.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.675+0000 7f5d99586640 1 -- 172.21.15.114:0/2792135779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c71de4b1a0 msgr2=0x55c71de48310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:42.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.675+0000 7f5d99586640 1 --2- 172.21.15.114:0/2792135779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c71de4b1a0 0x55c71de48310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.675+0000 7f5d99586640 1 -- 172.21.15.114:0/2792135779 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c71de40900 msgr2=0x55c71de47d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:42.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.675+0000 7f5d99586640 1 --2- 172.21.15.114:0/2792135779 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c71de40900 0x55c71de47d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.675+0000 7f5d99586640 1 -- 172.21.15.114:0/2792135779 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d8803eb40 con 0x55c71de3ffe0 2024-09-13T18:27:42.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.677+0000 7f5d99586640 1 --2- 172.21.15.114:0/2792135779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c71de3ffe0 0x55c71de4aaf0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5d88006010 tx=0x7f5d88049040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=714c1d6cb310c352 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:42.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.677+0000 7f5d73fff640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:27:42.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.677+0000 7f5d73fff640 1 -- 172.21.15.114:0/2792135779 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d88005180 con 0x55c71de3ffe0 2024-09-13T18:27:42.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.677+0000 7f5d73fff640 1 -- 172.21.15.114:0/2792135779 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f5d880052e0 con 0x55c71de3ffe0 2024-09-13T18:27:42.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.677+0000 7f5d9dde7740 1 -- 172.21.15.114:0/2792135779 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c71da974e0 con 0x55c71de3ffe0 2024-09-13T18:27:42.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.677+0000 7f5d73fff640 1 -- 172.21.15.114:0/2792135779 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d880054a0 con 0x55c71de3ffe0 2024-09-13T18:27:42.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.677+0000 7f5d9dde7740 10 client.26695 fetch_fsmap learned FSMap version 34 2024-09-13T18:27:42.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.677+0000 7f5d9dde7740 1 -- 172.21.15.114:0/2792135779 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55c71dda2620 con 0x55c71de3ffe0 2024-09-13T18:27:42.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.678+0000 7f5d73fff640 1 -- 172.21.15.114:0/2792135779 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c71da974e0 con 0x55c71de3ffe0 2024-09-13T18:27:42.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.678+0000 7f5d73fff640 1 -- 172.21.15.114:0/2792135779 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5d88004a20 con 0x55c71de3ffe0 2024-09-13T18:27:42.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.678+0000 7f5d9dde7740 10 client.26695 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:27:42.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.678+0000 7f5d9dde7740 10 client.26695 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:27:42.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.678+0000 7f5d9dde7740 20 client.26695 populate_metadata read hostname 'smithi114' 2024-09-13T18:27:42.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.678+0000 7f5d9dde7740 1 --2- 172.21.15.114:0/2792135779 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55c71de5ae50 0x55c71de5d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:42.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.678+0000 7f5d9dde7740 4 client.26695 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:27:42.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.678+0000 7f5d9dde7740 1 -- 172.21.15.114:0/2792135779 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c71daa87e0 con 0x55c71de5ae50 2024-09-13T18:27:42.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.679+0000 7f5d98d85640 1 --2- 172.21.15.114:0/2792135779 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55c71de5ae50 0x55c71de5d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:42.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.680+0000 7f5d98d85640 1 --2- 172.21.15.114:0/2792135779 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55c71de5ae50 0x55c71de5d270 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dc4a9ea1dce389a server_cookie=cfb1506fc56df5e8 in_seq=0 out_seq=0 2024-09-13T18:27:42.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.682+0000 7f5d73fff640 10 client.26695 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:27:42.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.682+0000 7f5d73fff640 1 -- 172.21.15.114:0/2792135779 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55c71daa87e0 con 0x55c71de5ae50 2024-09-13T18:27:42.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.682+0000 7f5d73fff640 10 client.26695 handle_command_reply: tid=0 2024-09-13T18:27:42.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.682+0000 7f5d9dde7740 1 client.26695 shutdown 2024-09-13T18:27:42.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.682+0000 7f5d9dde7740 1 -- 172.21.15.114:0/2792135779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c71de3ffe0 msgr2=0x55c71de4aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:42.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.682+0000 7f5d9dde7740 1 --2- 172.21.15.114:0/2792135779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c71de3ffe0 0x55c71de4aaf0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5d88006010 tx=0x7f5d88049040 comp rx=0 tx=0).stop 2024-09-13T18:27:42.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.682+0000 7f5d9dde7740 1 -- 172.21.15.114:0/2792135779 shutdown_connections 2024-09-13T18:27:42.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.682+0000 7f5d9dde7740 1 --2- 172.21.15.114:0/2792135779 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55c71de5ae50 0x55c71de5d270 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.682+0000 7f5d9dde7740 1 --2- 172.21.15.114:0/2792135779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c71de4b1a0 0x55c71de48310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.682+0000 7f5d9dde7740 1 --2- 172.21.15.114:0/2792135779 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c71de40900 0x55c71de47d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.682+0000 7f5d9dde7740 1 --2- 172.21.15.114:0/2792135779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c71de3ffe0 0x55c71de4aaf0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.682+0000 7f5d9dde7740 1 -- 172.21.15.114:0/2792135779 >> 172.21.15.114:0/2792135779 conn(0x55c71de3c570 msgr2=0x55c71de3b760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:42.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.684+0000 7f5d73fff640 0 client.26695 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:27:42.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.685+0000 7f5d9dde7740 1 -- 172.21.15.114:0/2792135779 shutdown_connections 2024-09-13T18:27:42.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.685+0000 7f5d9dde7740 1 -- 172.21.15.114:0/2792135779 wait complete. 2024-09-13T18:27:42.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.688+0000 7f5d9dde7740 20 client.26695 trim_cache size 0 max 16384 2024-09-13T18:27:42.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:42 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:27:42.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:42 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:27:42.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:42 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:27:42.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.726+0000 7f5d9dde7740 1 Processor -- start 2024-09-13T18:27:42.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.726+0000 7f5d9dde7740 1 -- start start 2024-09-13T18:27:42.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.726+0000 7f5d9dde7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c71de3ffe0 0x55c71de482f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:42.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.727+0000 7f5d9dde7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c71de40900 0x55c71de4a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:42.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.727+0000 7f5d9dde7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c71de4b1a0 0x55c71de51fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:42.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.727+0000 7f5d9dde7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c71de37ee0 con 0x55c71de40900 2024-09-13T18:27:42.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.727+0000 7f5d9dde7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c71de4dbc0 con 0x55c71de4b1a0 2024-09-13T18:27:42.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.727+0000 7f5d9dde7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c71de3d550 con 0x55c71de3ffe0 2024-09-13T18:27:42.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.728+0000 7f5d99586640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c71de3ffe0 0x55c71de482f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:42.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.728+0000 7f5d99586640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c71de3ffe0 0x55c71de482f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:54158/0 (socket says 172.21.15.114:54158) 2024-09-13T18:27:42.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.728+0000 7f5d99586640 1 -- 172.21.15.114:0/2893781190 learned_addr learned my addr 172.21.15.114:0/2893781190 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:42.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.728+0000 7f5d99586640 1 --2- 172.21.15.114:0/2893781190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c71de4b1a0 0x55c71de51fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:42.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.728+0000 7f5d99586640 1 -- 172.21.15.114:0/2893781190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c71de4b1a0 msgr2=0x55c71de51fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:42.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.728+0000 7f5d99586640 1 --2- 172.21.15.114:0/2893781190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c71de4b1a0 0x55c71de51fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.728+0000 7f5d99586640 1 -- 172.21.15.114:0/2893781190 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c71de40900 msgr2=0x55c71de4a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:42.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.728+0000 7f5d99586640 1 --2- 172.21.15.114:0/2893781190 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c71de40900 0x55c71de4a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.728+0000 7f5d99586640 1 -- 172.21.15.114:0/2893781190 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c71dd44790 con 0x55c71de3ffe0 2024-09-13T18:27:42.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.729+0000 7f5d99586640 1 --2- 172.21.15.114:0/2893781190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c71de3ffe0 0x55c71de482f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5d88004de0 tx=0x7f5d8803e4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f03c5ffe56d764 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:42.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.733+0000 7f5d7b7fe640 1 -- 172.21.15.114:0/2893781190 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d88003640 con 0x55c71de3ffe0 2024-09-13T18:27:42.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.733+0000 7f5d7b7fe640 1 -- 172.21.15.114:0/2893781190 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5d88003880 con 0x55c71de3ffe0 2024-09-13T18:27:42.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.733+0000 7f5d7b7fe640 1 -- 172.21.15.114:0/2893781190 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8804f4f0 con 0x55c71de3ffe0 2024-09-13T18:27:42.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.736+0000 7f5d9dde7740 1 -- 172.21.15.114:0/2893781190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c71de3ffe0 msgr2=0x55c71de482f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:42.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.736+0000 7f5d9dde7740 1 --2- 172.21.15.114:0/2893781190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c71de3ffe0 0x55c71de482f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5d88004de0 tx=0x7f5d8803e4b0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.744+0000 7f5d9dde7740 1 -- 172.21.15.114:0/2893781190 shutdown_connections 2024-09-13T18:27:42.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.744+0000 7f5d9dde7740 1 --2- 172.21.15.114:0/2893781190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c71de4b1a0 0x55c71de51fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.744+0000 7f5d9dde7740 1 --2- 172.21.15.114:0/2893781190 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c71de40900 0x55c71de4a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.744+0000 7f5d9dde7740 1 --2- 172.21.15.114:0/2893781190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c71de3ffe0 0x55c71de482f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.744+0000 7f5d9dde7740 1 -- 172.21.15.114:0/2893781190 >> 172.21.15.114:0/2893781190 conn(0x55c71de3c570 msgr2=0x55c71de47290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:42.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.752+0000 7f5d9dde7740 1 -- 172.21.15.114:0/2893781190 shutdown_connections 2024-09-13T18:27:42.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.755+0000 7f5d9dde7740 1 -- 172.21.15.114:0/2893781190 wait complete. 2024-09-13T18:27:42.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.755+0000 7f5d9dde7740 1 -- start start 2024-09-13T18:27:42.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.755+0000 7f5d9dde7740 1 Processor -- start 2024-09-13T18:27:42.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.756+0000 7f5d9dde7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c71de3ffe0 0x55c71de52060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:42.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.756+0000 7f5d9dde7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c71de40900 0x55c71de50400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:42.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.756+0000 7f5d9dde7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c71de4b1a0 0x55c71de50940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:42.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.757+0000 7f5d99586640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c71de3ffe0 0x55c71de52060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:42.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.757+0000 7f5d99586640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c71de3ffe0 0x55c71de52060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:54170/0 (socket says 172.21.15.114:54170) 2024-09-13T18:27:42.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.757+0000 7f5d98d85640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c71de40900 0x55c71de50400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:42.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.757+0000 7f5d98d85640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c71de40900 0x55c71de50400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:36988/0 (socket says 172.21.15.114:36988) 2024-09-13T18:27:42.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.757+0000 7f5d99586640 1 -- 172.21.15.114:0/2748974506 learned_addr learned my addr 172.21.15.114:0/2748974506 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:42.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.757+0000 7f5d9dde7740 1 -- 172.21.15.114:0/2748974506 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c71de50e80 con 0x55c71de4b1a0 2024-09-13T18:27:42.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.757+0000 7f5d9dde7740 1 -- 172.21.15.114:0/2748974506 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c71de50ff0 con 0x55c71de40900 2024-09-13T18:27:42.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.757+0000 7f5d9dde7740 1 -- 172.21.15.114:0/2748974506 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c71de511b0 con 0x55c71de3ffe0 2024-09-13T18:27:42.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.757+0000 7f5d9dde7740 10 client.0 _ll_register_callbacks cb 0x7f5d8803edf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:27:42.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.757+0000 7f5d99586640 1 -- 172.21.15.114:0/2748974506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c71de40900 msgr2=0x55c71de50400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:42.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.757+0000 7f5d99586640 1 --2- 172.21.15.114:0/2748974506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c71de40900 0x55c71de50400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.757+0000 7f5d99586640 1 -- 172.21.15.114:0/2748974506 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c71de4b1a0 msgr2=0x55c71de50940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:42.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.757+0000 7f5d99586640 1 --2- 172.21.15.114:0/2748974506 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c71de4b1a0 0x55c71de50940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.757+0000 7f5d99586640 1 -- 172.21.15.114:0/2748974506 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c71dda2620 con 0x55c71de3ffe0 2024-09-13T18:27:42.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.758+0000 7f5d99586640 1 --2- 172.21.15.114:0/2748974506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c71de3ffe0 0x55c71de52060 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5d8803eee0 tx=0x7f5d88003640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfc900256b30a0f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:42.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.760+0000 7f5d73fff640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:27:42.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.764+0000 7f5d73fff640 1 -- 172.21.15.114:0/2748974506 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8806e070 con 0x55c71de3ffe0 2024-09-13T18:27:42.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.764+0000 7f5d9dde7740 1 -- 172.21.15.114:0/2748974506 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5d8804b070 con 0x55c71de3ffe0 2024-09-13T18:27:42.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.764+0000 7f5d73fff640 1 -- 172.21.15.114:0/2748974506 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f5d88000e40 con 0x55c71de3ffe0 2024-09-13T18:27:42.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.767+0000 7f5d73fff640 1 -- 172.21.15.114:0/2748974506 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d88005460 con 0x55c71de3ffe0 2024-09-13T18:27:42.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.767+0000 7f5d9dde7740 10 client.26701 fetch_fsmap learned FSMap version 34 2024-09-13T18:27:42.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.767+0000 7f5d9dde7740 1 -- 172.21.15.114:0/2748974506 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55c71dd69ea0 con 0x55c71de3ffe0 2024-09-13T18:27:42.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.779+0000 7f5d73fff640 1 -- 172.21.15.114:0/2748974506 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5d8804b070 con 0x55c71de3ffe0 2024-09-13T18:27:42.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.779+0000 7f5d73fff640 1 -- 172.21.15.114:0/2748974506 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5d88068040 con 0x55c71de3ffe0 2024-09-13T18:27:42.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.780+0000 7f5d9dde7740 10 client.26701 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:27:42.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.780+0000 7f5d9dde7740 10 client.26701 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:27:42.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.780+0000 7f5d9dde7740 20 client.26701 populate_metadata read hostname 'smithi114' 2024-09-13T18:27:42.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.780+0000 7f5d9dde7740 1 --2- 172.21.15.114:0/2748974506 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55c71deb22b0 0x55c71deb46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:42.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.780+0000 7f5d9dde7740 4 client.26701 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T18:27:42.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.780+0000 7f5d9dde7740 1 -- 172.21.15.114:0/2748974506 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c71dafbe30 con 0x55c71deb22b0 2024-09-13T18:27:42.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.783+0000 7f5d98d85640 1 --2- 172.21.15.114:0/2748974506 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55c71deb22b0 0x55c71deb46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:42.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.788+0000 7f5d98d85640 1 --2- 172.21.15.114:0/2748974506 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55c71deb22b0 0x55c71deb46a0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdcb4234dfa51231 server_cookie=afedd3780df84a56 in_seq=0 out_seq=0 2024-09-13T18:27:42.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.788+0000 7f5d73fff640 10 client.26701 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:27:42.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.788+0000 7f5d73fff640 1 -- 172.21.15.114:0/2748974506 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55c71dafbe30 con 0x55c71deb22b0 2024-09-13T18:27:42.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.788+0000 7f5d73fff640 10 client.26701 handle_command_reply: tid=0 2024-09-13T18:27:42.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.788+0000 7f5d9dde7740 1 client.26701 shutdown 2024-09-13T18:27:42.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.789+0000 7f5d9dde7740 1 -- 172.21.15.114:0/2748974506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c71de3ffe0 msgr2=0x55c71de52060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:42.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.789+0000 7f5d9dde7740 1 --2- 172.21.15.114:0/2748974506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c71de3ffe0 0x55c71de52060 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5d8803eee0 tx=0x7f5d88003640 comp rx=0 tx=0).stop 2024-09-13T18:27:42.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.800+0000 7f5d9dde7740 1 -- 172.21.15.114:0/2748974506 shutdown_connections 2024-09-13T18:27:42.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.800+0000 7f5d9dde7740 1 --2- 172.21.15.114:0/2748974506 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55c71deb22b0 0x55c71deb46a0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.800+0000 7f5d9dde7740 1 --2- 172.21.15.114:0/2748974506 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c71de4b1a0 0x55c71de50940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.800+0000 7f5d9dde7740 1 --2- 172.21.15.114:0/2748974506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c71de40900 0x55c71de50400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.800+0000 7f5d9dde7740 1 --2- 172.21.15.114:0/2748974506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c71de3ffe0 0x55c71de52060 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.800+0000 7f5d9dde7740 1 -- 172.21.15.114:0/2748974506 >> 172.21.15.114:0/2748974506 conn(0x55c71de3c570 msgr2=0x55c71de43020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:42.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.800+0000 7f5d73fff640 0 client.26701 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:27:42.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.803+0000 7f5d9dde7740 1 -- 172.21.15.114:0/2748974506 shutdown_connections 2024-09-13T18:27:42.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.807+0000 7f5d9dde7740 1 -- 172.21.15.114:0/2748974506 wait complete. 2024-09-13T18:27:42.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.814+0000 7f5d9dde7740 20 client.26701 trim_cache size 0 max 16384 2024-09-13T18:27:42.817 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:27:42.817 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-13T18:27:42.817 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": { 2024-09-13T18:27:42.817 INFO:teuthology.orchestra.run.smithi114.stdout: "23bd2e7e-f0d4-47ed-9104-84bf71750a03": { 2024-09-13T18:27:42.818 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-13T18:27:42.818 INFO:teuthology.orchestra.run.smithi114.stdout: "tag": "23bd2e7e-f0d4-47ed-9104-84bf71750a03", 2024-09-13T18:27:42.818 INFO:teuthology.orchestra.run.smithi114.stdout: "options": "recursive,force" 2024-09-13T18:27:42.818 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:27:42.818 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:27:42.818 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:27:42.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.816+0000 7f5d99d87640 1 -- 172.21.15.114:0/1354587455 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f5d7c0a1410 msgr2=0x7f5d7c0a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:42.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.816+0000 7f5d99d87640 1 --2- 172.21.15.114:0/1354587455 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f5d7c0a1410 0x7f5d7c0a38d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5d84097520 tx=0x7f5d8400e000 comp rx=0 tx=0).stop 2024-09-13T18:27:42.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.816+0000 7f5d99d87640 1 -- 172.21.15.114:0/1354587455 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d94068060 msgr2=0x7f5d94071680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:42.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.816+0000 7f5d99d87640 1 --2- 172.21.15.114:0/1354587455 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d94068060 0x7f5d94071680 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f5d8400b450 tx=0x7f5d8400b910 comp rx=0 tx=0).stop 2024-09-13T18:27:42.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.817+0000 7f5d99d87640 1 -- 172.21.15.114:0/1354587455 shutdown_connections 2024-09-13T18:27:42.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.817+0000 7f5d99d87640 1 --2- 172.21.15.114:0/1354587455 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f5d7c0a1410 0x7f5d7c0a38d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.817+0000 7f5d99d87640 1 --2- 172.21.15.114:0/1354587455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d941a13e0 0x7f5d941a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.817+0000 7f5d99d87640 1 --2- 172.21.15.114:0/1354587455 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d94068060 0x7f5d94071680 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.817+0000 7f5d99d87640 1 --2- 172.21.15.114:0/1354587455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d940675a0 0x7f5d94071140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:42.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.817+0000 7f5d99d87640 1 -- 172.21.15.114:0/1354587455 >> 172.21.15.114:0/1354587455 conn(0x7f5d94074980 msgr2=0x7f5d940fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:42.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.820+0000 7f5d99d87640 1 -- 172.21.15.114:0/1354587455 shutdown_connections 2024-09-13T18:27:42.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:27:42.820+0000 7f5d99d87640 1 -- 172.21.15.114:0/1354587455 wait complete. 2024-09-13T18:27:42.855 INFO:tasks.cephfs.filesystem:scrub status for tag:23bd2e7e-f0d4-47ed-9104-84bf71750a03 - {'path': '/', 'tag': '23bd2e7e-f0d4-47ed-9104-84bf71750a03', 'options': 'recursive,force'} 2024-09-13T18:27:43.711 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.708+0000 7f85f1925640 1 -- 172.21.15.114:0/1356652451 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85ec107890 msgr2=0x7f85ec107cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:43.711 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.708+0000 7f85f1925640 1 --2- 172.21.15.114:0/1356652451 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85ec107890 0x7f85ec107cf0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f85e0009900 tx=0x7f85e002a410 comp rx=0 tx=0).stop 2024-09-13T18:27:43.713 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.709+0000 7f85f1925640 1 -- 172.21.15.114:0/1356652451 shutdown_connections 2024-09-13T18:27:43.713 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.709+0000 7f85f1925640 1 --2- 172.21.15.114:0/1356652451 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85ec119400 0x7f85ec11b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:43.713 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.709+0000 7f85f1925640 1 --2- 172.21.15.114:0/1356652451 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85ec107890 0x7f85ec107cf0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:43.714 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.709+0000 7f85f1925640 1 --2- 172.21.15.114:0/1356652451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85ec1074b0 0x7f85ec10e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:43.714 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.709+0000 7f85f1925640 1 -- 172.21.15.114:0/1356652451 >> 172.21.15.114:0/1356652451 conn(0x7f85ec071d80 msgr2=0x7f85ec06c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:43.718 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.715+0000 7f85f1925640 1 -- 172.21.15.114:0/1356652451 shutdown_connections 2024-09-13T18:27:43.718 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.715+0000 7f85f1925640 1 -- 172.21.15.114:0/1356652451 wait complete. 2024-09-13T18:27:43.719 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.716+0000 7f85f1925640 1 Processor -- start 2024-09-13T18:27:43.723 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.717+0000 7f85f1925640 1 -- start start 2024-09-13T18:27:43.723 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.718+0000 7f85f1925640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85ec1074b0 0x7f85ec1a4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:43.723 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.718+0000 7f85f1925640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85ec107890 0x7f85ec1a54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:43.723 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.718+0000 7f85f1925640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85ec119400 0x7f85ec1a98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:43.723 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.718+0000 7f85f1925640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ec1a9ea0 con 0x7f85ec119400 2024-09-13T18:27:43.723 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.718+0000 7f85f1925640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ec1a9fe0 con 0x7f85ec1074b0 2024-09-13T18:27:43.723 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.718+0000 7f85f1925640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ec1aa1a0 con 0x7f85ec107890 2024-09-13T18:27:43.723 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.718+0000 7f85eaffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85ec1074b0 0x7f85ec1a4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:43.723 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.719+0000 7f85eaffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85ec1074b0 0x7f85ec1a4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:36828/0 (socket says 172.21.15.114:36828) 2024-09-13T18:27:43.724 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.719+0000 7f85eaffd640 1 -- 172.21.15.114:0/1212092675 learned_addr learned my addr 172.21.15.114:0/1212092675 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:43.724 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.719+0000 7f85eb7fe640 1 --2- 172.21.15.114:0/1212092675 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85ec119400 0x7f85ec1a98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:43.724 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.719+0000 7f85eaffd640 1 -- 172.21.15.114:0/1212092675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85ec107890 msgr2=0x7f85ec1a54b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:27:43.724 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.719+0000 7f85eaffd640 1 --2- 172.21.15.114:0/1212092675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85ec107890 0x7f85ec1a54b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:43.724 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.719+0000 7f85eaffd640 1 -- 172.21.15.114:0/1212092675 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85ec119400 msgr2=0x7f85ec1a98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:43.724 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.719+0000 7f85eaffd640 1 --2- 172.21.15.114:0/1212092675 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85ec119400 0x7f85ec1a98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:43.724 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.719+0000 7f85eaffd640 1 -- 172.21.15.114:0/1212092675 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e0009560 con 0x7f85ec1074b0 2024-09-13T18:27:43.724 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.720+0000 7f85eb7fe640 1 --2- 172.21.15.114:0/1212092675 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85ec119400 0x7f85ec1a98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:27:43.724 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.720+0000 7f85eaffd640 1 --2- 172.21.15.114:0/1212092675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85ec1074b0 0x7f85ec1a4f70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f85dc00b480 tx=0x7f85dc00b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.724+0000 7f85f0923640 1 -- 172.21.15.114:0/1212092675 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85dc010870 con 0x7f85ec1074b0 2024-09-13T18:27:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.724+0000 7f85f1925640 1 -- 172.21.15.114:0/1212092675 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85ec1aa480 con 0x7f85ec1074b0 2024-09-13T18:27:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.724+0000 7f85f1925640 1 -- 172.21.15.114:0/1212092675 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85ec10edc0 con 0x7f85ec1074b0 2024-09-13T18:27:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.724+0000 7f85f0923640 1 -- 172.21.15.114:0/1212092675 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f85dc010e60 con 0x7f85ec1074b0 2024-09-13T18:27:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.724+0000 7f85f0923640 1 -- 172.21.15.114:0/1212092675 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85dc00d360 con 0x7f85ec1074b0 2024-09-13T18:27:43.730 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.727+0000 7f85f464b740 1 Processor -- start 2024-09-13T18:27:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.730+0000 7f85f464b740 1 -- start start 2024-09-13T18:27:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.730+0000 7f85f464b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8306f4330 0x55d8306f4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.730+0000 7f85f464b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8306f4c50 0x55d8306fefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.730+0000 7f85f464b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8306ff4f0 0x55d8307018e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.730+0000 7f85f464b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d83065fc30 con 0x55d8306f4c50 2024-09-13T18:27:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.730+0000 7f85f464b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d83065d3c0 con 0x55d8306ff4f0 2024-09-13T18:27:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.730+0000 7f85f464b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d830658cb0 con 0x55d8306f4330 2024-09-13T18:27:43.735 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.731+0000 7f85eb7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8306f4330 0x55d8306f4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:43.736 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.732+0000 7f85eb7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8306f4330 0x55d8306f4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:47958/0 (socket says 172.21.15.114:47958) 2024-09-13T18:27:43.736 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.732+0000 7f85eb7fe640 1 -- 172.21.15.114:0/823956284 learned_addr learned my addr 172.21.15.114:0/823956284 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:43.736 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.732+0000 7f85f0923640 1 -- 172.21.15.114:0/1212092675 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f85dc0109d0 con 0x7f85ec1074b0 2024-09-13T18:27:43.736 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.732+0000 7f85eb7fe640 1 -- 172.21.15.114:0/823956284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8306ff4f0 msgr2=0x55d8307018e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:27:43.736 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.732+0000 7f85eb7fe640 1 --2- 172.21.15.114:0/823956284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8306ff4f0 0x55d8307018e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:43.736 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.732+0000 7f85eb7fe640 1 -- 172.21.15.114:0/823956284 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8306f4c50 msgr2=0x55d8306fefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:43.736 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.732+0000 7f85eb7fe640 1 --2- 172.21.15.114:0/823956284 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8306f4c50 0x55d8306fefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:43.736 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.732+0000 7f85eb7fe640 1 -- 172.21.15.114:0/823956284 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8305ff910 con 0x55d8306f4330 2024-09-13T18:27:43.736 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.733+0000 7f85eb7fe640 1 --2- 172.21.15.114:0/823956284 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8306f4330 0x55d8306f4710 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f85e4009f60 tx=0x7f85e4009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=574d3afcc183e448 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:43.740 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.733+0000 7f85f0923640 1 --2- 172.21.15.114:0/1212092675 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f85d40a1410 0x7f85d40a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:43.740 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.733+0000 7f85d1ffb640 1 -- 172.21.15.114:0/823956284 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e4003b70 con 0x55d8306f4330 2024-09-13T18:27:43.740 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.733+0000 7f85d1ffb640 1 -- 172.21.15.114:0/823956284 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f85e4003cd0 con 0x55d8306f4330 2024-09-13T18:27:43.740 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.733+0000 7f85d1ffb640 1 -- 172.21.15.114:0/823956284 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e4011730 con 0x55d8306f4330 2024-09-13T18:27:43.741 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.734+0000 7f85f464b740 1 -- 172.21.15.114:0/823956284 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8306f4330 msgr2=0x55d8306f4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:43.741 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.734+0000 7f85f464b740 1 --2- 172.21.15.114:0/823956284 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8306f4330 0x55d8306f4710 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f85e4009f60 tx=0x7f85e4009f90 comp rx=0 tx=0).stop 2024-09-13T18:27:43.741 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.734+0000 7f85f464b740 1 -- 172.21.15.114:0/823956284 shutdown_connections 2024-09-13T18:27:43.741 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.734+0000 7f85f464b740 1 --2- 172.21.15.114:0/823956284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8306ff4f0 0x55d8307018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:43.741 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.734+0000 7f85f464b740 1 --2- 172.21.15.114:0/823956284 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8306f4c50 0x55d8306fefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:43.741 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.734+0000 7f85f464b740 1 --2- 172.21.15.114:0/823956284 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8306f4330 0x55d8306f4710 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:43.741 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.734+0000 7f85f464b740 1 -- 172.21.15.114:0/823956284 >> 172.21.15.114:0/823956284 conn(0x55d8306f08c0 msgr2=0x55d8306f0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:43.741 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.734+0000 7f85f0923640 1 -- 172.21.15.114:0/1212092675 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f85dc0cc7b0 con 0x7f85ec1074b0 2024-09-13T18:27:43.741 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.734+0000 7f85eaffd640 1 --2- 172.21.15.114:0/1212092675 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f85d40a1410 0x7f85d40a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:43.741 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.735+0000 7f85f464b740 1 -- 172.21.15.114:0/823956284 shutdown_connections 2024-09-13T18:27:43.741 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.735+0000 7f85f464b740 1 -- 172.21.15.114:0/823956284 wait complete. 2024-09-13T18:27:43.741 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.735+0000 7f85f464b740 1 -- start start 2024-09-13T18:27:43.742 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.735+0000 7f85f464b740 1 Processor -- start 2024-09-13T18:27:43.742 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.735+0000 7f85f464b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8306f4330 0x55d8306feef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:43.742 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.735+0000 7f85f464b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8306f4c50 0x55d8306fc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:43.742 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.736+0000 7f85f464b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8306ff4f0 0x55d8306fc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:43.742 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.736+0000 7f85f464b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d830616e80 con 0x55d8306f4c50 2024-09-13T18:27:43.742 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.736+0000 7f85f464b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d830616b20 con 0x55d8306ff4f0 2024-09-13T18:27:43.742 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.736+0000 7f85f464b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8305dc260 con 0x55d8306f4330 2024-09-13T18:27:43.742 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.736+0000 7f85f464b740 10 client.0 _ll_register_callbacks cb 0x55d830611df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:27:43.742 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.736+0000 7f85eb7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8306f4330 0x55d8306feef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:43.742 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.736+0000 7f85eb7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8306ff4f0 0x55d8306fc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:43.742 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.736+0000 7f85eb7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8306f4330 0x55d8306feef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:47964/0 (socket says 172.21.15.114:47964) 2024-09-13T18:27:43.742 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.736+0000 7f85eb7fe640 1 -- 172.21.15.114:0/2008394700 learned_addr learned my addr 172.21.15.114:0/2008394700 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:43.743 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.737+0000 7f85ea7fc640 1 --2- 172.21.15.114:0/2008394700 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8306f4c50 0x55d8306fc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:43.743 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.737+0000 7f85eb7fe640 1 -- 172.21.15.114:0/2008394700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8306ff4f0 msgr2=0x55d8306fc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:43.743 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.737+0000 7f85eb7fe640 1 --2- 172.21.15.114:0/2008394700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8306ff4f0 0x55d8306fc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:43.743 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.737+0000 7f85eb7fe640 1 -- 172.21.15.114:0/2008394700 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8306f4c50 msgr2=0x55d8306fc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:43.743 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.737+0000 7f85eb7fe640 1 --2- 172.21.15.114:0/2008394700 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8306f4c50 0x55d8306fc190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:43.743 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.737+0000 7f85eb7fe640 1 -- 172.21.15.114:0/2008394700 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85e4009c10 con 0x55d8306f4330 2024-09-13T18:27:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.737+0000 7f85eb7fe640 1 --2- 172.21.15.114:0/2008394700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8306f4330 0x55d8306feef0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f85e4004430 tx=0x7f85e4004920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3476951dd8c2cca server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.738+0000 7f85cf7f6640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:27:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.738+0000 7f85cf7f6640 1 -- 172.21.15.114:0/2008394700 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e40114d0 con 0x55d8306f4330 2024-09-13T18:27:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.738+0000 7f85cf7f6640 1 -- 172.21.15.114:0/2008394700 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f85e4011630 con 0x55d8306f4330 2024-09-13T18:27:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.738+0000 7f85f464b740 1 -- 172.21.15.114:0/2008394700 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d830681680 con 0x55d8306f4330 2024-09-13T18:27:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.738+0000 7f85eaffd640 1 --2- 172.21.15.114:0/1212092675 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f85d40a1410 0x7f85d40a38d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x55d8306f03c0 tx=0x7f85dc00ffe0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.738+0000 7f85cf7f6640 1 -- 172.21.15.114:0/2008394700 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85e40117f0 con 0x55d8306f4330 2024-09-13T18:27:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.739+0000 7f85f464b740 10 client.26707 fetch_fsmap learned FSMap version 34 2024-09-13T18:27:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.739+0000 7f85f464b740 1 -- 172.21.15.114:0/2008394700 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55d830657330 con 0x55d8306f4330 2024-09-13T18:27:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.739+0000 7f85cf7f6640 1 -- 172.21.15.114:0/2008394700 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d830681680 con 0x55d8306f4330 2024-09-13T18:27:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.739+0000 7f85cf7f6640 1 -- 172.21.15.114:0/2008394700 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f85e401b720 con 0x55d8306f4330 2024-09-13T18:27:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.739+0000 7f85f464b740 10 client.26707 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:27:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.739+0000 7f85f464b740 10 client.26707 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:27:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.739+0000 7f85f464b740 20 client.26707 populate_metadata read hostname 'smithi114' 2024-09-13T18:27:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.739+0000 7f85f464b740 1 --2- 172.21.15.114:0/2008394700 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55d83070ef10 0x55d830711330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.739+0000 7f85f464b740 4 client.26707 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:27:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.739+0000 7f85f464b740 1 -- 172.21.15.114:0/2008394700 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d83064c5d0 con 0x55d83070ef10 2024-09-13T18:27:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.743+0000 7f85ea7fc640 1 --2- 172.21.15.114:0/2008394700 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55d83070ef10 0x55d830711330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:43.749 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.746+0000 7f85ea7fc640 1 --2- 172.21.15.114:0/2008394700 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55d83070ef10 0x55d830711330 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dab487739982f45c server_cookie=9d42664b6f8ca12d in_seq=0 out_seq=0 2024-09-13T18:27:43.749 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.746+0000 7f85cf7f6640 10 client.26707 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:27:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.748+0000 7f85cf7f6640 1 -- 172.21.15.114:0/2008394700 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55d83064c5d0 con 0x55d83070ef10 2024-09-13T18:27:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.748+0000 7f85cf7f6640 10 client.26707 handle_command_reply: tid=0 2024-09-13T18:27:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.748+0000 7f85f464b740 1 client.26707 shutdown 2024-09-13T18:27:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.750+0000 7f85f464b740 1 -- 172.21.15.114:0/2008394700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8306f4330 msgr2=0x55d8306feef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.750+0000 7f85f464b740 1 --2- 172.21.15.114:0/2008394700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8306f4330 0x55d8306feef0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f85e4004430 tx=0x7f85e4004920 comp rx=0 tx=0).stop 2024-09-13T18:27:43.756 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.753+0000 7f85f464b740 1 -- 172.21.15.114:0/2008394700 shutdown_connections 2024-09-13T18:27:43.756 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.753+0000 7f85f464b740 1 --2- 172.21.15.114:0/2008394700 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55d83070ef10 0x55d830711330 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:43.756 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.753+0000 7f85f464b740 1 --2- 172.21.15.114:0/2008394700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8306ff4f0 0x55d8306fc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:43.756 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.753+0000 7f85f464b740 1 --2- 172.21.15.114:0/2008394700 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8306f4c50 0x55d8306fc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:43.756 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.753+0000 7f85f464b740 1 --2- 172.21.15.114:0/2008394700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8306f4330 0x55d8306feef0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:43.756 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.753+0000 7f85f464b740 1 -- 172.21.15.114:0/2008394700 >> 172.21.15.114:0/2008394700 conn(0x55d8306f08c0 msgr2=0x55d8306efb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:43.756 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.753+0000 7f85cf7f6640 0 client.26707 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:27:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.754+0000 7f85f464b740 1 -- 172.21.15.114:0/2008394700 shutdown_connections 2024-09-13T18:27:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.754+0000 7f85f464b740 1 -- 172.21.15.114:0/2008394700 wait complete. 2024-09-13T18:27:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.756+0000 7f85f464b740 20 client.26707 trim_cache size 0 max 16384 2024-09-13T18:27:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.793+0000 7f85f464b740 1 Processor -- start 2024-09-13T18:27:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.794+0000 7f85f464b740 1 -- start start 2024-09-13T18:27:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.794+0000 7f85f464b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8306f4330 0x55d8306fc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.794+0000 7f85f464b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8306f4c50 0x55d8306fea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.795+0000 7f85f464b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8306ff4f0 0x55d8306eba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.795+0000 7f85f464b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8306fa6c0 con 0x55d8306ff4f0 2024-09-13T18:27:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.795+0000 7f85f464b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d83070b520 con 0x55d8306f4c50 2024-09-13T18:27:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.795+0000 7f85f464b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8306fba00 con 0x55d8306f4330 2024-09-13T18:27:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.800+0000 7f85ea7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8306f4c50 0x55d8306fea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.800+0000 7f85eb7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8306f4330 0x55d8306fc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.800+0000 7f85ea7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8306f4c50 0x55d8306fea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:36854/0 (socket says 172.21.15.114:36854) 2024-09-13T18:27:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.800+0000 7f85ea7fc640 1 -- 172.21.15.114:0/2312063694 learned_addr learned my addr 172.21.15.114:0/2312063694 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.800+0000 7f85eb7fe640 1 --2- 172.21.15.114:0/2312063694 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8306ff4f0 0x55d8306eba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.800+0000 7f85eb7fe640 1 -- 172.21.15.114:0/2312063694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8306f4c50 msgr2=0x55d8306fea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.800+0000 7f85eb7fe640 1 --2- 172.21.15.114:0/2312063694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8306f4c50 0x55d8306fea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.800+0000 7f85eb7fe640 1 -- 172.21.15.114:0/2312063694 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8306ff4f0 msgr2=0x55d8306eba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.800+0000 7f85eb7fe640 1 --2- 172.21.15.114:0/2312063694 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8306ff4f0 0x55d8306eba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.800+0000 7f85eb7fe640 1 -- 172.21.15.114:0/2312063694 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8305f9300 con 0x55d8306f4330 2024-09-13T18:27:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.811+0000 7f85eb7fe640 1 --2- 172.21.15.114:0/2312063694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8306f4330 0x55d8306fc150 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f85e4003ca0 tx=0x7f85e4003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6ca244d27049d88 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.813+0000 7f85d27fc640 1 -- 172.21.15.114:0/2312063694 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e400fc80 con 0x55d8306f4330 2024-09-13T18:27:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.813+0000 7f85d27fc640 1 -- 172.21.15.114:0/2312063694 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f85e401f040 con 0x55d8306f4330 2024-09-13T18:27:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.813+0000 7f85d27fc640 1 -- 172.21.15.114:0/2312063694 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e40049c0 con 0x55d8306f4330 2024-09-13T18:27:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.814+0000 7f85f464b740 1 -- 172.21.15.114:0/2312063694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8306f4330 msgr2=0x55d8306fc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.814+0000 7f85f464b740 1 --2- 172.21.15.114:0/2312063694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8306f4330 0x55d8306fc150 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f85e4003ca0 tx=0x7f85e4003c10 comp rx=0 tx=0).stop 2024-09-13T18:27:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.814+0000 7f85f464b740 1 -- 172.21.15.114:0/2312063694 shutdown_connections 2024-09-13T18:27:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.814+0000 7f85f464b740 1 --2- 172.21.15.114:0/2312063694 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8306ff4f0 0x55d8306eba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.814+0000 7f85f464b740 1 --2- 172.21.15.114:0/2312063694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8306f4c50 0x55d8306fea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.814+0000 7f85f464b740 1 --2- 172.21.15.114:0/2312063694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8306f4330 0x55d8306fc150 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.814+0000 7f85f464b740 1 -- 172.21.15.114:0/2312063694 >> 172.21.15.114:0/2312063694 conn(0x55d8306f08c0 msgr2=0x55d8306f0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.815+0000 7f85f464b740 1 -- 172.21.15.114:0/2312063694 shutdown_connections 2024-09-13T18:27:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.815+0000 7f85f464b740 1 -- 172.21.15.114:0/2312063694 wait complete. 2024-09-13T18:27:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.816+0000 7f85f464b740 1 -- start start 2024-09-13T18:27:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.816+0000 7f85f464b740 1 Processor -- start 2024-09-13T18:27:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.817+0000 7f85f464b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8306f4330 0x55d8307065f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.817+0000 7f85f464b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8306f4c50 0x55d830702970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.817+0000 7f85f464b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8306ff4f0 0x55d830702eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.817+0000 7f85f464b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d830703580 con 0x55d8306f4c50 2024-09-13T18:27:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.817+0000 7f85f464b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8307036f0 con 0x55d8306ff4f0 2024-09-13T18:27:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.817+0000 7f85f464b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d830765410 con 0x55d8306f4330 2024-09-13T18:27:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.817+0000 7f85f464b740 10 client.0 _ll_register_callbacks cb 0x7f85ec06e0c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:27:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.817+0000 7f85eb7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8306f4330 0x55d8307065f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.817+0000 7f85eb7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8306f4330 0x55d8307065f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:47980/0 (socket says 172.21.15.114:47980) 2024-09-13T18:27:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.817+0000 7f85eb7fe640 1 -- 172.21.15.114:0/4097013328 learned_addr learned my addr 172.21.15.114:0/4097013328 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.817+0000 7f85ea7fc640 1 --2- 172.21.15.114:0/4097013328 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8306f4c50 0x55d830702970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.818+0000 7f85ea7fc640 1 -- 172.21.15.114:0/4097013328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8306f4330 msgr2=0x55d8307065f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.818+0000 7f85ea7fc640 1 --2- 172.21.15.114:0/4097013328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8306f4330 0x55d8307065f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.818+0000 7f85ea7fc640 1 -- 172.21.15.114:0/4097013328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8306ff4f0 msgr2=0x55d830702eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.818+0000 7f85ea7fc640 1 --2- 172.21.15.114:0/4097013328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8306ff4f0 0x55d830702eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.818+0000 7f85ea7fc640 1 -- 172.21.15.114:0/4097013328 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d830657330 con 0x55d8306f4c50 2024-09-13T18:27:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.818+0000 7f85ea7fc640 1 --2- 172.21.15.114:0/4097013328 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8306f4c50 0x55d830702970 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f85e0006010 tx=0x7f85e0004590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ece5ff6f2c356838 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.818+0000 7f85cf7f6640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:27:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.819+0000 7f85cf7f6640 1 -- 172.21.15.114:0/4097013328 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e0005200 con 0x55d8306f4c50 2024-09-13T18:27:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.819+0000 7f85cf7f6640 1 -- 172.21.15.114:0/4097013328 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f85e0005360 con 0x55d8306f4c50 2024-09-13T18:27:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.819+0000 7f85f464b740 1 -- 172.21.15.114:0/4097013328 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f85e4003cd0 con 0x55d8306f4c50 2024-09-13T18:27:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.819+0000 7f85cf7f6640 1 -- 172.21.15.114:0/4097013328 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85e0043e50 con 0x55d8306f4c50 2024-09-13T18:27:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.819+0000 7f85f464b740 10 client.17289 fetch_fsmap learned FSMap version 34 2024-09-13T18:27:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.819+0000 7f85f464b740 1 -- 172.21.15.114:0/4097013328 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55d83061e8e0 con 0x55d8306f4c50 2024-09-13T18:27:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.820+0000 7f85cf7f6640 1 -- 172.21.15.114:0/4097013328 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f85e4003cd0 con 0x55d8306f4c50 2024-09-13T18:27:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.820+0000 7f85cf7f6640 1 -- 172.21.15.114:0/4097013328 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f85e0043400 con 0x55d8306f4c50 2024-09-13T18:27:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.820+0000 7f85f464b740 10 client.17289 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:27:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.820+0000 7f85f464b740 10 client.17289 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:27:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.820+0000 7f85f464b740 20 client.17289 populate_metadata read hostname 'smithi114' 2024-09-13T18:27:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.820+0000 7f85f464b740 1 --2- 172.21.15.114:0/4097013328 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55d830766c80 0x55d8307690a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.820+0000 7f85f464b740 4 client.17289 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T18:27:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.820+0000 7f85f464b740 1 -- 172.21.15.114:0/4097013328 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d83034c300 con 0x55d830766c80 2024-09-13T18:27:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.821+0000 7f85eb7fe640 1 --2- 172.21.15.114:0/4097013328 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55d830766c80 0x55d8307690a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.834+0000 7f85eb7fe640 1 --2- 172.21.15.114:0/4097013328 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55d830766c80 0x55d8307690a0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45ad261ebe8fee62 server_cookie=a35242f84e0ef867 in_seq=0 out_seq=0 2024-09-13T18:27:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.834+0000 7f85cf7f6640 10 client.17289 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:27:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.841+0000 7f85cf7f6640 1 -- 172.21.15.114:0/4097013328 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d83034c300 con 0x55d830766c80 2024-09-13T18:27:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.841+0000 7f85cf7f6640 10 client.17289 handle_command_reply: tid=0 2024-09-13T18:27:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.841+0000 7f85f464b740 1 client.17289 shutdown 2024-09-13T18:27:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.841+0000 7f85f464b740 1 -- 172.21.15.114:0/4097013328 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8306f4c50 msgr2=0x55d830702970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.841+0000 7f85f464b740 1 --2- 172.21.15.114:0/4097013328 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8306f4c50 0x55d830702970 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f85e0006010 tx=0x7f85e0004590 comp rx=0 tx=0).stop 2024-09-13T18:27:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.841+0000 7f85f464b740 1 -- 172.21.15.114:0/4097013328 shutdown_connections 2024-09-13T18:27:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.841+0000 7f85f464b740 1 --2- 172.21.15.114:0/4097013328 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55d830766c80 0x55d8307690a0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.841+0000 7f85f464b740 1 --2- 172.21.15.114:0/4097013328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8306ff4f0 0x55d830702eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.841+0000 7f85f464b740 1 --2- 172.21.15.114:0/4097013328 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8306f4c50 0x55d830702970 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.841+0000 7f85f464b740 1 --2- 172.21.15.114:0/4097013328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8306f4330 0x55d8307065f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.841+0000 7f85f464b740 1 -- 172.21.15.114:0/4097013328 >> 172.21.15.114:0/4097013328 conn(0x55d8306f08c0 msgr2=0x55d8306eba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.841+0000 7f85cf7f6640 0 client.17289 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:27:43.847 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.841+0000 7f85f464b740 1 -- 172.21.15.114:0/4097013328 shutdown_connections 2024-09-13T18:27:43.853 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.847+0000 7f85f464b740 1 -- 172.21.15.114:0/4097013328 wait complete. 2024-09-13T18:27:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-13T18:27:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-13T18:27:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-13T18:27:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-13T18:27:43.860 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.856+0000 7f85f464b740 20 client.17289 trim_cache size 0 max 16384 2024-09-13T18:27:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.859+0000 7f85f1925640 1 -- 172.21.15.114:0/1212092675 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f85d40a1410 msgr2=0x7f85d40a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.859+0000 7f85f1925640 1 --2- 172.21.15.114:0/1212092675 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f85d40a1410 0x7f85d40a38d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x55d8306f03c0 tx=0x7f85dc00ffe0 comp rx=0 tx=0).stop 2024-09-13T18:27:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.859+0000 7f85f1925640 1 -- 172.21.15.114:0/1212092675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85ec1074b0 msgr2=0x7f85ec1a4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.859+0000 7f85f1925640 1 --2- 172.21.15.114:0/1212092675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85ec1074b0 0x7f85ec1a4f70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f85dc00b480 tx=0x7f85dc00b940 comp rx=0 tx=0).stop 2024-09-13T18:27:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.862+0000 7f85f1925640 1 -- 172.21.15.114:0/1212092675 shutdown_connections 2024-09-13T18:27:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.862+0000 7f85f1925640 1 --2- 172.21.15.114:0/1212092675 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f85d40a1410 0x7f85d40a38d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.862+0000 7f85f1925640 1 --2- 172.21.15.114:0/1212092675 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85ec119400 0x7f85ec1a98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.862+0000 7f85f1925640 1 --2- 172.21.15.114:0/1212092675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85ec107890 0x7f85ec1a54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.862+0000 7f85f1925640 1 --2- 172.21.15.114:0/1212092675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85ec1074b0 0x7f85ec1a4f70 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.862+0000 7f85f1925640 1 -- 172.21.15.114:0/1212092675 >> 172.21.15.114:0/1212092675 conn(0x7f85ec071d80 msgr2=0x7f85ec06c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:43.868 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:43 smithi134 ceph-mon[34283]: pgmap v160: 129 pgs: 129 active+clean; 17 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 38 MiB/s wr, 644 op/s 2024-09-13T18:27:43.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:43 smithi134 ceph-mon[34283]: scrub queued for path: / 2024-09-13T18:27:43.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:43 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:27:43.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:43 smithi134 ceph-mon[34283]: scrub summary: active paths [/] 2024-09-13T18:27:43.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:43 smithi134 ceph-mon[34283]: scrub summary: active paths [/] 2024-09-13T18:27:43.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:43 smithi134 ceph-mon[34283]: scrub summary: active paths [/] 2024-09-13T18:27:43.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:43 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:27:43.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:43 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:27:43.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:43 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:27:43.870 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.864+0000 7f85f1925640 1 -- 172.21.15.114:0/1212092675 shutdown_connections 2024-09-13T18:27:43.870 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:43.867+0000 7f85f1925640 1 -- 172.21.15.114:0/1212092675 wait complete. 2024-09-13T18:27:43.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:43 smithi204 ceph-mon[34777]: pgmap v160: 129 pgs: 129 active+clean; 17 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 38 MiB/s wr, 644 op/s 2024-09-13T18:27:43.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:43 smithi204 ceph-mon[34777]: scrub queued for path: / 2024-09-13T18:27:43.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:43 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:27:43.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:43 smithi204 ceph-mon[34777]: scrub summary: active paths [/] 2024-09-13T18:27:43.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:43 smithi204 ceph-mon[34777]: scrub summary: active paths [/] 2024-09-13T18:27:43.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:43 smithi204 ceph-mon[34777]: scrub summary: active paths [/] 2024-09-13T18:27:43.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:43 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:27:43.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:43 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:27:43.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:43 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:27:43.972 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:43 smithi114 ceph-mon[30897]: pgmap v160: 129 pgs: 129 active+clean; 17 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 38 MiB/s wr, 644 op/s 2024-09-13T18:27:43.972 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:43 smithi114 ceph-mon[30897]: scrub queued for path: / 2024-09-13T18:27:43.972 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:43 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:27:43.972 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:43 smithi114 ceph-mon[30897]: scrub summary: active paths [/] 2024-09-13T18:27:43.972 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:43 smithi114 ceph-mon[30897]: scrub summary: active paths [/] 2024-09-13T18:27:43.972 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:43 smithi114 ceph-mon[30897]: scrub summary: active paths [/] 2024-09-13T18:27:43.972 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:43 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:27:43.972 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:43 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:27:43.973 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:43 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:27:45.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:45 smithi134 ceph-mon[34283]: pgmap v161: 129 pgs: 129 active+clean; 17 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 49 MiB/s wr, 856 op/s 2024-09-13T18:27:45.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:45 smithi204 ceph-mon[34777]: pgmap v161: 129 pgs: 129 active+clean; 17 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 49 MiB/s wr, 856 op/s 2024-09-13T18:27:45.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:45 smithi114 ceph-mon[30897]: pgmap v161: 129 pgs: 129 active+clean; 17 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 49 MiB/s wr, 856 op/s 2024-09-13T18:27:47.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:47 smithi134 ceph-mon[34283]: pgmap v162: 129 pgs: 129 active+clean; 17 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 44 MiB/s wr, 768 op/s 2024-09-13T18:27:47.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:47 smithi204 ceph-mon[34777]: pgmap v162: 129 pgs: 129 active+clean; 17 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 44 MiB/s wr, 768 op/s 2024-09-13T18:27:47.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:47 smithi114 ceph-mon[30897]: pgmap v162: 129 pgs: 129 active+clean; 17 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 44 MiB/s wr, 768 op/s 2024-09-13T18:27:49.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:49 smithi134 ceph-mon[34283]: pgmap v163: 129 pgs: 129 active+clean; 17 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 37 MiB/s wr, 649 op/s 2024-09-13T18:27:49.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:49 smithi204 ceph-mon[34777]: pgmap v163: 129 pgs: 129 active+clean; 17 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 37 MiB/s wr, 649 op/s 2024-09-13T18:27:49.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:49 smithi114 ceph-mon[30897]: pgmap v163: 129 pgs: 129 active+clean; 17 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 37 MiB/s wr, 649 op/s 2024-09-13T18:27:51.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:51 smithi134 ceph-mon[34283]: pgmap v164: 129 pgs: 129 active+clean; 18 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 56 MiB/s wr, 987 op/s 2024-09-13T18:27:51.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:51 smithi204 ceph-mon[34777]: pgmap v164: 129 pgs: 129 active+clean; 18 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 56 MiB/s wr, 987 op/s 2024-09-13T18:27:51.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:51 smithi114 ceph-mon[30897]: pgmap v164: 129 pgs: 129 active+clean; 18 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 56 MiB/s wr, 987 op/s 2024-09-13T18:27:53.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:53 smithi134 ceph-mon[34283]: pgmap v165: 129 pgs: 129 active+clean; 18 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 37 MiB/s wr, 666 op/s 2024-09-13T18:27:53.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:53 smithi204 ceph-mon[34777]: pgmap v165: 129 pgs: 129 active+clean; 18 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 37 MiB/s wr, 666 op/s 2024-09-13T18:27:53.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:53 smithi114 ceph-mon[30897]: pgmap v165: 129 pgs: 129 active+clean; 18 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 37 MiB/s wr, 666 op/s 2024-09-13T18:27:55.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:55 smithi134 ceph-mon[34283]: pgmap v166: 129 pgs: 129 active+clean; 18 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 48 MiB/s wr, 871 op/s 2024-09-13T18:27:55.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:55 smithi204 ceph-mon[34777]: pgmap v166: 129 pgs: 129 active+clean; 18 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 48 MiB/s wr, 871 op/s 2024-09-13T18:27:55.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:55 smithi114 ceph-mon[30897]: pgmap v166: 129 pgs: 129 active+clean; 18 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 48 MiB/s wr, 871 op/s 2024-09-13T18:27:56.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:56 smithi134 ceph-mon[34283]: scrub summary: idle 2024-09-13T18:27:56.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:56 smithi204 ceph-mon[34777]: scrub summary: idle 2024-09-13T18:27:56.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:56 smithi114 ceph-mon[30897]: scrub summary: idle 2024-09-13T18:27:57.093 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.089+0000 7fecc1c9c640 1 -- 172.21.15.114:0/1856260466 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fecbc110be0 msgr2=0x7fecbc113050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:57.093 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.089+0000 7fecc1c9c640 1 --2- 172.21.15.114:0/1856260466 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fecbc110be0 0x7fecbc113050 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fecac009900 tx=0x7fecac02a490 comp rx=0 tx=0).stop 2024-09-13T18:27:57.093 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.090+0000 7fecc1c9c640 1 -- 172.21.15.114:0/1856260466 shutdown_connections 2024-09-13T18:27:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.090+0000 7fecc1c9c640 1 --2- 172.21.15.114:0/1856260466 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fecbc110be0 0x7fecbc113050 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.090+0000 7fecc1c9c640 1 --2- 172.21.15.114:0/1856260466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fecbc108040 0x7fecbc110520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.090+0000 7fecc1c9c640 1 --2- 172.21.15.114:0/1856260466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fecbc107720 0x7fecbc107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.090+0000 7fecc1c9c640 1 -- 172.21.15.114:0/1856260466 >> 172.21.15.114:0/1856260466 conn(0x7fecbc06b280 msgr2=0x7fecbc06b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.090+0000 7fecc1c9c640 1 -- 172.21.15.114:0/1856260466 shutdown_connections 2024-09-13T18:27:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.090+0000 7fecc1c9c640 1 -- 172.21.15.114:0/1856260466 wait complete. 2024-09-13T18:27:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.091+0000 7fecc1c9c640 1 Processor -- start 2024-09-13T18:27:57.097 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.092+0000 7fecc1c9c640 1 -- start start 2024-09-13T18:27:57.097 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.092+0000 7fecc1c9c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fecbc107720 0x7fecbc115250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:57.097 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.092+0000 7fecc1c9c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fecbc108040 0x7fecbc115790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.092+0000 7fecc1c9c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fecbc110be0 0x7fecbc115cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.092+0000 7fecc1c9c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecbc117210 con 0x7fecbc108040 2024-09-13T18:27:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.092+0000 7fecc1c9c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecbc1a2990 con 0x7fecbc107720 2024-09-13T18:27:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.092+0000 7fecc1c9c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecbc1a2b50 con 0x7fecbc110be0 2024-09-13T18:27:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.093+0000 7fecbb7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fecbc107720 0x7fecbc115250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.093+0000 7fecbb7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fecbc107720 0x7fecbc115250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:35428/0 (socket says 172.21.15.114:35428) 2024-09-13T18:27:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.093+0000 7fecbb7fe640 1 -- 172.21.15.114:0/4152759433 learned_addr learned my addr 172.21.15.114:0/4152759433 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.093+0000 7fecbbfff640 1 --2- 172.21.15.114:0/4152759433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fecbc110be0 0x7fecbc115cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.093+0000 7fecbb7fe640 1 -- 172.21.15.114:0/4152759433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fecbc110be0 msgr2=0x7fecbc115cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.093+0000 7fecbb7fe640 1 --2- 172.21.15.114:0/4152759433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fecbc110be0 0x7fecbc115cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.093+0000 7fecbb7fe640 1 -- 172.21.15.114:0/4152759433 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fecbc108040 msgr2=0x7fecbc115790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.093+0000 7fecbaffd640 1 --2- 172.21.15.114:0/4152759433 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fecbc108040 0x7fecbc115790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.093+0000 7fecbb7fe640 1 --2- 172.21.15.114:0/4152759433 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fecbc108040 0x7fecbc115790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.093+0000 7fecbb7fe640 1 -- 172.21.15.114:0/4152759433 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecac009560 con 0x7fecbc107720 2024-09-13T18:27:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.094+0000 7fecbaffd640 1 --2- 172.21.15.114:0/4152759433 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fecbc108040 0x7fecbc115790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:27:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.094+0000 7fecbb7fe640 1 --2- 172.21.15.114:0/4152759433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fecbc107720 0x7fecbc115250 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fecb400ebf0 tx=0x7fecb400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.095+0000 7fecb8ff9640 1 -- 172.21.15.114:0/4152759433 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb4010630 con 0x7fecbc107720 2024-09-13T18:27:57.102 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.096+0000 7fecc1c9c640 1 -- 172.21.15.114:0/4152759433 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecbc1a2e30 con 0x7fecbc107720 2024-09-13T18:27:57.103 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.096+0000 7fecc1c9c640 1 -- 172.21.15.114:0/4152759433 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecbc1a3340 con 0x7fecbc107720 2024-09-13T18:27:57.103 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.096+0000 7fecb8ff9640 1 -- 172.21.15.114:0/4152759433 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fecb400ced0 con 0x7fecbc107720 2024-09-13T18:27:57.103 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.096+0000 7fecb8ff9640 1 -- 172.21.15.114:0/4152759433 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb40189e0 con 0x7fecbc107720 2024-09-13T18:27:57.104 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.100+0000 7fecc49c2740 1 Processor -- start 2024-09-13T18:27:57.107 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.102+0000 7fecc49c2740 1 -- start start 2024-09-13T18:27:57.107 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.102+0000 7fecc49c2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f29ef500 0x55e0f29ef8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:57.107 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.102+0000 7fecc49c2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0f29efe20 0x55e0f29fa180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:57.107 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.102+0000 7fecc49c2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0f29fa6c0 0x55e0f29fcab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:57.107 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.102+0000 7fecc49c2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0f295ac30 con 0x55e0f29fa6c0 2024-09-13T18:27:57.107 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.102+0000 7fecc49c2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0f29583c0 con 0x55e0f29ef500 2024-09-13T18:27:57.107 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.102+0000 7fecc49c2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0f2953cb0 con 0x55e0f29efe20 2024-09-13T18:27:57.108 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.103+0000 7fecb8ff9640 1 -- 172.21.15.114:0/4152759433 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fecb4018b40 con 0x7fecbc107720 2024-09-13T18:27:57.108 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.103+0000 7fecbbfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f29ef500 0x55e0f29ef8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:57.108 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.103+0000 7fecbb7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0f29efe20 0x55e0f29fa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:57.108 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.103+0000 7fecbbfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f29ef500 0x55e0f29ef8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:35434/0 (socket says 172.21.15.114:35434) 2024-09-13T18:27:57.108 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.103+0000 7fecbbfff640 1 -- 172.21.15.114:0/504738173 learned_addr learned my addr 172.21.15.114:0/504738173 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:57.108 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.103+0000 7fecbbfff640 1 -- 172.21.15.114:0/504738173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0f29efe20 msgr2=0x55e0f29fa180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:57.108 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.103+0000 7fecbbfff640 1 --2- 172.21.15.114:0/504738173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0f29efe20 0x55e0f29fa180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:57.108 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.103+0000 7fecbbfff640 1 -- 172.21.15.114:0/504738173 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0f29fa6c0 msgr2=0x55e0f29fcab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:57.108 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.103+0000 7fecbbfff640 1 --2- 172.21.15.114:0/504738173 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0f29fa6c0 0x55e0f29fcab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:57.108 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.103+0000 7fecbbfff640 1 -- 172.21.15.114:0/504738173 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0f28fa910 con 0x55e0f29ef500 2024-09-13T18:27:57.108 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.103+0000 7fecbbfff640 1 --2- 172.21.15.114:0/504738173 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f29ef500 0x55e0f29ef8e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fecac009900 tx=0x7fecac004180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4cd7da0d2afb042d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:57.109 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.104+0000 7feca1ffb640 1 -- 172.21.15.114:0/504738173 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecac0035f0 con 0x55e0f29ef500 2024-09-13T18:27:57.109 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.105+0000 7feca1ffb640 1 -- 172.21.15.114:0/504738173 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fecac003be0 con 0x55e0f29ef500 2024-09-13T18:27:57.109 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.105+0000 7fecb8ff9640 1 --2- 172.21.15.114:0/4152759433 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7feca40a13d0 0x7feca40a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:57.109 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.105+0000 7fecbb7fe640 1 --2- 172.21.15.114:0/4152759433 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7feca40a13d0 0x7feca40a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:57.110 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.106+0000 7fecc49c2740 1 -- 172.21.15.114:0/504738173 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f29ef500 msgr2=0x55e0f29ef8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:57.110 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.106+0000 7fecc49c2740 1 --2- 172.21.15.114:0/504738173 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f29ef500 0x55e0f29ef8e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fecac009900 tx=0x7fecac004180 comp rx=0 tx=0).stop 2024-09-13T18:27:57.110 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.106+0000 7fecb8ff9640 1 -- 172.21.15.114:0/4152759433 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fecb4014070 con 0x7fecbc107720 2024-09-13T18:27:57.110 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.107+0000 7fecbb7fe640 1 --2- 172.21.15.114:0/4152759433 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7feca40a13d0 0x7feca40a3890 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x55e0f29eb590 tx=0x7fecb40d3190 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.114+0000 7fecc49c2740 1 -- 172.21.15.114:0/504738173 shutdown_connections 2024-09-13T18:27:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.114+0000 7fecc49c2740 1 --2- 172.21.15.114:0/504738173 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0f29fa6c0 0x55e0f29fcab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.114+0000 7fecc49c2740 1 --2- 172.21.15.114:0/504738173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0f29efe20 0x55e0f29fa180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.114+0000 7fecc49c2740 1 --2- 172.21.15.114:0/504738173 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f29ef500 0x55e0f29ef8e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.114+0000 7fecc49c2740 1 -- 172.21.15.114:0/504738173 >> 172.21.15.114:0/504738173 conn(0x55e0f29eba90 msgr2=0x55e0f29ebe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:57.119 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.115+0000 7fecc49c2740 1 -- 172.21.15.114:0/504738173 shutdown_connections 2024-09-13T18:27:57.120 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.116+0000 7fecc49c2740 1 -- 172.21.15.114:0/504738173 wait complete. 2024-09-13T18:27:57.120 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.116+0000 7fecc49c2740 1 -- start start 2024-09-13T18:27:57.120 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.116+0000 7fecc49c2740 1 Processor -- start 2024-09-13T18:27:57.121 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.116+0000 7fecc49c2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0f29ef500 0x55e0f29f9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:57.121 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.116+0000 7fecc49c2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f29efe20 0x55e0f29f7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:57.121 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.116+0000 7fecc49c2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0f29fa6c0 0x55e0f29f77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:57.121 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.116+0000 7fecc49c2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0f2911e80 con 0x55e0f29ef500 2024-09-13T18:27:57.121 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.116+0000 7fecc49c2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0f2911b20 con 0x55e0f29efe20 2024-09-13T18:27:57.121 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.116+0000 7fecc49c2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0f28d7260 con 0x55e0f29fa6c0 2024-09-13T18:27:57.121 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.116+0000 7fecc49c2740 10 client.0 _ll_register_callbacks cb 0x55e0f290cdf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:27:57.124 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.119+0000 7fecbaffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f29efe20 0x55e0f29f7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:57.124 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.120+0000 7fecbbfff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0f29ef500 0x55e0f29f9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:57.124 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.120+0000 7fecbbfff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0f29ef500 0x55e0f29f9fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50742/0 (socket says 172.21.15.114:50742) 2024-09-13T18:27:57.124 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.120+0000 7fecbbfff640 1 -- 172.21.15.114:0/167916255 learned_addr learned my addr 172.21.15.114:0/167916255 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:57.124 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.120+0000 7fecbbfff640 1 --2- 172.21.15.114:0/167916255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0f29fa6c0 0x55e0f29f77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.122+0000 7fecbbfff640 1 -- 172.21.15.114:0/167916255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f29efe20 msgr2=0x55e0f29f7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.122+0000 7fecbbfff640 1 --2- 172.21.15.114:0/167916255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f29efe20 0x55e0f29f7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.122+0000 7fecbbfff640 1 -- 172.21.15.114:0/167916255 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0f29ef500 msgr2=0x55e0f29f9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.122+0000 7fecbbfff640 1 --2- 172.21.15.114:0/167916255 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0f29ef500 0x55e0f29f9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.122+0000 7fecbbfff640 1 -- 172.21.15.114:0/167916255 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecbc1139a0 con 0x55e0f29fa6c0 2024-09-13T18:27:57.129 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.125+0000 7fecbbfff640 1 --2- 172.21.15.114:0/167916255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0f29fa6c0 0x55e0f29f77c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fecac0468f0 tx=0x7fecac005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3649310f2574d2e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:57.129 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.125+0000 7fec9f7f6640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:27:57.134 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.129+0000 7fec9f7f6640 1 -- 172.21.15.114:0/167916255 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecac04a030 con 0x55e0f29fa6c0 2024-09-13T18:27:57.134 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.129+0000 7fec9f7f6640 1 -- 172.21.15.114:0/167916255 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fecac04b070 con 0x55e0f29fa6c0 2024-09-13T18:27:57.134 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.130+0000 7fecc49c2740 1 -- 172.21.15.114:0/167916255 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e0f297c680 con 0x55e0f29fa6c0 2024-09-13T18:27:57.134 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.132+0000 7fec9f7f6640 1 -- 172.21.15.114:0/167916255 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fecac04d030 con 0x55e0f29fa6c0 2024-09-13T18:27:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.134+0000 7fecc49c2740 10 client.26722 fetch_fsmap learned FSMap version 34 2024-09-13T18:27:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.134+0000 7fecc49c2740 1 -- 172.21.15.114:0/167916255 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55e0f2952330 con 0x55e0f29fa6c0 2024-09-13T18:27:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.142+0000 7fec9f7f6640 1 -- 172.21.15.114:0/167916255 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e0f297c680 con 0x55e0f29fa6c0 2024-09-13T18:27:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.142+0000 7fec9f7f6640 1 -- 172.21.15.114:0/167916255 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fecac03d9a0 con 0x55e0f29fa6c0 2024-09-13T18:27:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.142+0000 7fecc49c2740 10 client.26722 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:27:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.142+0000 7fecc49c2740 10 client.26722 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:27:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.142+0000 7fecc49c2740 20 client.26722 populate_metadata read hostname 'smithi114' 2024-09-13T18:27:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.142+0000 7fecc49c2740 1 --2- 172.21.15.114:0/167916255 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55e0f2a09cd0 0x55e0f2a0c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.142+0000 7fecc49c2740 4 client.26722 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:27:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.142+0000 7fecc49c2740 1 -- 172.21.15.114:0/167916255 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e0f29475d0 con 0x55e0f2a09cd0 2024-09-13T18:27:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.143+0000 7fecbaffd640 1 --2- 172.21.15.114:0/167916255 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55e0f2a09cd0 0x55e0f2a0c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.143+0000 7fecbaffd640 1 --2- 172.21.15.114:0/167916255 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55e0f2a09cd0 0x55e0f2a0c0f0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de2a8a86fd329466 server_cookie=96e74b2cfa562857 in_seq=0 out_seq=0 2024-09-13T18:27:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.143+0000 7fec9f7f6640 10 client.26722 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:27:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.144+0000 7fec9f7f6640 1 -- 172.21.15.114:0/167916255 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55e0f29475d0 con 0x55e0f2a09cd0 2024-09-13T18:27:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.144+0000 7fec9f7f6640 10 client.26722 handle_command_reply: tid=0 2024-09-13T18:27:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.144+0000 7fecc49c2740 1 client.26722 shutdown 2024-09-13T18:27:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.144+0000 7fecc49c2740 1 -- 172.21.15.114:0/167916255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0f29fa6c0 msgr2=0x55e0f29f77c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.144+0000 7fecc49c2740 1 --2- 172.21.15.114:0/167916255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0f29fa6c0 0x55e0f29f77c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fecac0468f0 tx=0x7fecac005720 comp rx=0 tx=0).stop 2024-09-13T18:27:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.145+0000 7fecc49c2740 1 -- 172.21.15.114:0/167916255 shutdown_connections 2024-09-13T18:27:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.145+0000 7fecc49c2740 1 --2- 172.21.15.114:0/167916255 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55e0f2a09cd0 0x55e0f2a0c0f0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.145+0000 7fecc49c2740 1 --2- 172.21.15.114:0/167916255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0f29fa6c0 0x55e0f29f77c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.145+0000 7fecc49c2740 1 --2- 172.21.15.114:0/167916255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f29efe20 0x55e0f29f7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.145+0000 7fec9f7f6640 0 client.26722 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:27:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.145+0000 7fecc49c2740 1 --2- 172.21.15.114:0/167916255 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0f29ef500 0x55e0f29f9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.145+0000 7fecc49c2740 1 -- 172.21.15.114:0/167916255 >> 172.21.15.114:0/167916255 conn(0x55e0f29eba90 msgr2=0x55e0f29ead60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.145+0000 7fecc49c2740 1 -- 172.21.15.114:0/167916255 shutdown_connections 2024-09-13T18:27:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.145+0000 7fecc49c2740 1 -- 172.21.15.114:0/167916255 wait complete. 2024-09-13T18:27:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.146+0000 7fecc49c2740 20 client.26722 trim_cache size 0 max 16384 2024-09-13T18:27:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.176+0000 7fecc49c2740 1 Processor -- start 2024-09-13T18:27:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.176+0000 7fecc49c2740 1 -- start start 2024-09-13T18:27:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.176+0000 7fecc49c2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0f29ef500 0x55e0f29f77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.176+0000 7fecc49c2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f29efe20 0x55e0f29f5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.176+0000 7fecc49c2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0f29fa6c0 0x55e0f29f6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.176+0000 7fecc49c2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0f29475d0 con 0x55e0f29fa6c0 2024-09-13T18:27:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.176+0000 7fecc49c2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0f297c680 con 0x55e0f29efe20 2024-09-13T18:27:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.176+0000 7fecc49c2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0f2a06e10 con 0x55e0f29ef500 2024-09-13T18:27:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.176+0000 7fecbbfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0f29ef500 0x55e0f29f77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.177+0000 7fecbbfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0f29ef500 0x55e0f29f77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:60052/0 (socket says 172.21.15.114:60052) 2024-09-13T18:27:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.177+0000 7fecbbfff640 1 -- 172.21.15.114:0/3113524264 learned_addr learned my addr 172.21.15.114:0/3113524264 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.178+0000 7fecbbfff640 1 --2- 172.21.15.114:0/3113524264 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0f29fa6c0 0x55e0f29f6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.178+0000 7fecbbfff640 1 -- 172.21.15.114:0/3113524264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f29efe20 msgr2=0x55e0f29f5b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:27:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.178+0000 7fecbbfff640 1 --2- 172.21.15.114:0/3113524264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f29efe20 0x55e0f29f5b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.178+0000 7fecbbfff640 1 -- 172.21.15.114:0/3113524264 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0f29fa6c0 msgr2=0x55e0f29f6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:27:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.178+0000 7fecbbfff640 1 --2- 172.21.15.114:0/3113524264 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0f29fa6c0 0x55e0f29f6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:57.183 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.178+0000 7fecbbfff640 1 -- 172.21.15.114:0/3113524264 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0f28f4300 con 0x55e0f29ef500 2024-09-13T18:27:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.181+0000 7fecbbfff640 1 --2- 172.21.15.114:0/3113524264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0f29ef500 0x55e0f29f77b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fecac001eb0 tx=0x7fecac005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=831caea413aab970 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.182+0000 7feca27fc640 1 -- 172.21.15.114:0/3113524264 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecac04d070 con 0x55e0f29ef500 2024-09-13T18:27:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.182+0000 7feca27fc640 1 -- 172.21.15.114:0/3113524264 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fecac03bea0 con 0x55e0f29ef500 2024-09-13T18:27:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.182+0000 7feca27fc640 1 -- 172.21.15.114:0/3113524264 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecac04a040 con 0x55e0f29ef500 2024-09-13T18:27:57.189 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.185+0000 7fecc49c2740 1 -- 172.21.15.114:0/3113524264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0f29ef500 msgr2=0x55e0f29f77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:57.189 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.185+0000 7fecc49c2740 1 --2- 172.21.15.114:0/3113524264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0f29ef500 0x55e0f29f77b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fecac001eb0 tx=0x7fecac005fd0 comp rx=0 tx=0).stop 2024-09-13T18:27:57.189 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.185+0000 7fecc49c2740 1 -- 172.21.15.114:0/3113524264 shutdown_connections 2024-09-13T18:27:57.189 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.185+0000 7fecc49c2740 1 --2- 172.21.15.114:0/3113524264 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0f29fa6c0 0x55e0f29f6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:57.189 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.185+0000 7fecc49c2740 1 --2- 172.21.15.114:0/3113524264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f29efe20 0x55e0f29f5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:57.189 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.185+0000 7fecc49c2740 1 --2- 172.21.15.114:0/3113524264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0f29ef500 0x55e0f29f77b0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:57.189 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.185+0000 7fecc49c2740 1 -- 172.21.15.114:0/3113524264 >> 172.21.15.114:0/3113524264 conn(0x55e0f29eba90 msgr2=0x55e0f29f2500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:57.189 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.186+0000 7fecc49c2740 1 -- 172.21.15.114:0/3113524264 shutdown_connections 2024-09-13T18:27:57.189 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.186+0000 7fecc49c2740 1 -- 172.21.15.114:0/3113524264 wait complete. 2024-09-13T18:27:57.190 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.186+0000 7fecc49c2740 1 -- start start 2024-09-13T18:27:57.190 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.186+0000 7fecc49c2740 1 Processor -- start 2024-09-13T18:27:57.191 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.187+0000 7fecc49c2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0f29ef500 0x55e0f2a01560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:57.191 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.187+0000 7fecc49c2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0f29efe20 0x55e0f29ff9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:57.191 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.187+0000 7fecc49c2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f29fa6c0 0x55e0f29fff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:57.191 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.187+0000 7fecc49c2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0f2a00450 con 0x55e0f29efe20 2024-09-13T18:27:57.191 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.187+0000 7fecc49c2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0f2a005c0 con 0x55e0f29fa6c0 2024-09-13T18:27:57.191 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.187+0000 7fecc49c2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0f2a00780 con 0x55e0f29ef500 2024-09-13T18:27:57.191 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.187+0000 7fecc49c2740 10 client.0 _ll_register_callbacks cb 0x7fecac009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:27:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.188+0000 7fecbaffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0f29efe20 0x55e0f29ff9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.188+0000 7fecbaffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0f29efe20 0x55e0f29ff9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50754/0 (socket says 172.21.15.114:50754) 2024-09-13T18:27:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.188+0000 7fecbaffd640 1 -- 172.21.15.114:0/275182336 learned_addr learned my addr 172.21.15.114:0/275182336 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:27:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.188+0000 7fecbaffd640 1 -- 172.21.15.114:0/275182336 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0f29ef500 msgr2=0x55e0f2a01560 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:27:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.188+0000 7fecbaffd640 1 --2- 172.21.15.114:0/275182336 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0f29ef500 0x55e0f2a01560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.188+0000 7fecbaffd640 1 -- 172.21.15.114:0/275182336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f29fa6c0 msgr2=0x55e0f29fff10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:27:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.188+0000 7fecbaffd640 1 --2- 172.21.15.114:0/275182336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f29fa6c0 0x55e0f29fff10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.188+0000 7fecbaffd640 1 -- 172.21.15.114:0/275182336 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e0f2952330 con 0x55e0f29efe20 2024-09-13T18:27:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.189+0000 7fecbaffd640 1 --2- 172.21.15.114:0/275182336 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0f29efe20 0x55e0f29ff9d0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fecb000da80 tx=0x7fecb0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b22816f40373f48 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:27:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.189+0000 7fec9f7f6640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:27:57.194 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.190+0000 7fec9f7f6640 1 -- 172.21.15.114:0/275182336 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb0015040 con 0x55e0f29efe20 2024-09-13T18:27:57.194 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.190+0000 7fec9f7f6640 1 -- 172.21.15.114:0/275182336 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fecb0018530 con 0x55e0f29efe20 2024-09-13T18:27:57.194 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.190+0000 7fecc49c2740 1 -- 172.21.15.114:0/275182336 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fecac003be0 con 0x55e0f29efe20 2024-09-13T18:27:57.194 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.191+0000 7fec9f7f6640 1 -- 172.21.15.114:0/275182336 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fecb00186f0 con 0x55e0f29efe20 2024-09-13T18:27:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.193+0000 7fecc49c2740 10 client.17301 fetch_fsmap learned FSMap version 34 2024-09-13T18:27:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.193+0000 7fecc49c2740 1 -- 172.21.15.114:0/275182336 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55e0f29198e0 con 0x55e0f29efe20 2024-09-13T18:27:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.198+0000 7fec9f7f6640 1 -- 172.21.15.114:0/275182336 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fecb0029030 con 0x55e0f29efe20 2024-09-13T18:27:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.198+0000 7fec9f7f6640 1 -- 172.21.15.114:0/275182336 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fecb0018050 con 0x55e0f29efe20 2024-09-13T18:27:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.198+0000 7fecc49c2740 10 client.17301 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:27:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.198+0000 7fecc49c2740 10 client.17301 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:27:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.198+0000 7fecc49c2740 20 client.17301 populate_metadata read hostname 'smithi114' 2024-09-13T18:27:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.198+0000 7fecc49c2740 1 --2- 172.21.15.114:0/275182336 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55e0f2a61730 0x55e0f2a63b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:27:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.198+0000 7fecc49c2740 4 client.17301 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T18:27:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.198+0000 7fecc49c2740 1 -- 172.21.15.114:0/275182336 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e0f2647300 con 0x55e0f2a61730 2024-09-13T18:27:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.199+0000 7fecbbfff640 1 --2- 172.21.15.114:0/275182336 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55e0f2a61730 0x55e0f2a63b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:27:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.207+0000 7fecbbfff640 1 --2- 172.21.15.114:0/275182336 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55e0f2a61730 0x55e0f2a63b50 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=754174b263a65d9 server_cookie=f244d4ee40df6316 in_seq=0 out_seq=0 2024-09-13T18:27:57.211 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.207+0000 7fec9f7f6640 10 client.17301 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:27:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.217+0000 7fec9f7f6640 1 -- 172.21.15.114:0/275182336 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e0f2647300 con 0x55e0f2a61730 2024-09-13T18:27:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.217+0000 7fec9f7f6640 10 client.17301 handle_command_reply: tid=0 2024-09-13T18:27:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.218+0000 7fecc49c2740 1 client.17301 shutdown 2024-09-13T18:27:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.219+0000 7fecc49c2740 1 -- 172.21.15.114:0/275182336 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0f29efe20 msgr2=0x55e0f29ff9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.219+0000 7fecc49c2740 1 --2- 172.21.15.114:0/275182336 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0f29efe20 0x55e0f29ff9d0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fecb000da80 tx=0x7fecb0011810 comp rx=0 tx=0).stop 2024-09-13T18:27:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.219+0000 7fecc49c2740 1 -- 172.21.15.114:0/275182336 shutdown_connections 2024-09-13T18:27:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.219+0000 7fecc49c2740 1 --2- 172.21.15.114:0/275182336 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55e0f2a61730 0x55e0f2a63b50 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.219+0000 7fecc49c2740 1 --2- 172.21.15.114:0/275182336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0f29fa6c0 0x55e0f29fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.219+0000 7fecc49c2740 1 --2- 172.21.15.114:0/275182336 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0f29efe20 0x55e0f29ff9d0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.219+0000 7fecc49c2740 1 --2- 172.21.15.114:0/275182336 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0f29ef500 0x55e0f2a01560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.219+0000 7fecc49c2740 1 -- 172.21.15.114:0/275182336 >> 172.21.15.114:0/275182336 conn(0x55e0f29eba90 msgr2=0x55e0f29f5af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.219+0000 7fec9f7f6640 0 client.17301 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:27:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.219+0000 7fecc49c2740 1 -- 172.21.15.114:0/275182336 shutdown_connections 2024-09-13T18:27:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.219+0000 7fecc49c2740 1 -- 172.21.15.114:0/275182336 wait complete. 2024-09-13T18:27:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.220+0000 7fecc49c2740 20 client.17301 trim_cache size 0 max 16384 2024-09-13T18:27:57.224 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-13T18:27:57.224 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-13T18:27:57.224 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-13T18:27:57.224 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-13T18:27:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.222+0000 7fecc1c9c640 1 -- 172.21.15.114:0/4152759433 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7feca40a13d0 msgr2=0x7feca40a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.222+0000 7fecc1c9c640 1 --2- 172.21.15.114:0/4152759433 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7feca40a13d0 0x7feca40a3890 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x55e0f29eb590 tx=0x7fecb40d3190 comp rx=0 tx=0).stop 2024-09-13T18:27:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.222+0000 7fecc1c9c640 1 -- 172.21.15.114:0/4152759433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fecbc107720 msgr2=0x7fecbc115250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:27:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.222+0000 7fecc1c9c640 1 --2- 172.21.15.114:0/4152759433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fecbc107720 0x7fecbc115250 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fecb400ebf0 tx=0x7fecb400c6a0 comp rx=0 tx=0).stop 2024-09-13T18:27:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.222+0000 7fecc1c9c640 1 -- 172.21.15.114:0/4152759433 shutdown_connections 2024-09-13T18:27:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.222+0000 7fecc1c9c640 1 --2- 172.21.15.114:0/4152759433 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7feca40a13d0 0x7feca40a3890 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.222+0000 7fecc1c9c640 1 --2- 172.21.15.114:0/4152759433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fecbc110be0 0x7fecbc115cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.222+0000 7fecc1c9c640 1 --2- 172.21.15.114:0/4152759433 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fecbc108040 0x7fecbc115790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.222+0000 7fecc1c9c640 1 --2- 172.21.15.114:0/4152759433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fecbc107720 0x7fecbc115250 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:27:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.222+0000 7fecc1c9c640 1 -- 172.21.15.114:0/4152759433 >> 172.21.15.114:0/4152759433 conn(0x7fecbc06b280 msgr2=0x7fecbc06ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:27:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.223+0000 7fecc1c9c640 1 -- 172.21.15.114:0/4152759433 shutdown_connections 2024-09-13T18:27:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:27:57.223+0000 7fecc1c9c640 1 -- 172.21.15.114:0/4152759433 wait complete. 2024-09-13T18:27:57.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:57 smithi134 ceph-mon[34283]: pgmap v167: 129 pgs: 129 active+clean; 18 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 42 MiB/s wr, 769 op/s 2024-09-13T18:27:57.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:57 smithi134 ceph-mon[34283]: scrub summary: idle 2024-09-13T18:27:57.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:57 smithi134 ceph-mon[34283]: scrub summary: idle 2024-09-13T18:27:57.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:57 smithi204 ceph-mon[34777]: pgmap v167: 129 pgs: 129 active+clean; 18 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 42 MiB/s wr, 769 op/s 2024-09-13T18:27:57.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:57 smithi204 ceph-mon[34777]: scrub summary: idle 2024-09-13T18:27:57.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:57 smithi204 ceph-mon[34777]: scrub summary: idle 2024-09-13T18:27:57.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:57 smithi114 ceph-mon[30897]: pgmap v167: 129 pgs: 129 active+clean; 18 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 42 MiB/s wr, 769 op/s 2024-09-13T18:27:57.972 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:57 smithi114 ceph-mon[30897]: scrub summary: idle 2024-09-13T18:27:57.972 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:57 smithi114 ceph-mon[30897]: scrub summary: idle 2024-09-13T18:27:59.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:27:59 smithi134 ceph-mon[34283]: pgmap v168: 129 pgs: 129 active+clean; 18 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 36 MiB/s wr, 652 op/s 2024-09-13T18:27:59.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:27:59 smithi204 ceph-mon[34777]: pgmap v168: 129 pgs: 129 active+clean; 18 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 36 MiB/s wr, 652 op/s 2024-09-13T18:27:59.972 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:27:59 smithi114 ceph-mon[30897]: pgmap v168: 129 pgs: 129 active+clean; 18 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 36 MiB/s wr, 652 op/s 2024-09-13T18:28:01.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:01 smithi134 ceph-mon[34283]: pgmap v169: 129 pgs: 129 active+clean; 18 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 54 MiB/s wr, 984 op/s 2024-09-13T18:28:01.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:01 smithi204 ceph-mon[34777]: pgmap v169: 129 pgs: 129 active+clean; 18 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 54 MiB/s wr, 984 op/s 2024-09-13T18:28:01.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:01 smithi114 ceph-mon[30897]: pgmap v169: 129 pgs: 129 active+clean; 18 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 54 MiB/s wr, 984 op/s 2024-09-13T18:28:03.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:03 smithi134 ceph-mon[34283]: pgmap v170: 129 pgs: 129 active+clean; 18 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 35 MiB/s wr, 647 op/s 2024-09-13T18:28:03.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:03 smithi204 ceph-mon[34777]: pgmap v170: 129 pgs: 129 active+clean; 18 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 35 MiB/s wr, 647 op/s 2024-09-13T18:28:03.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:03 smithi114 ceph-mon[30897]: pgmap v170: 129 pgs: 129 active+clean; 18 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 35 MiB/s wr, 647 op/s 2024-09-13T18:28:05.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:05 smithi134 ceph-mon[34283]: pgmap v171: 129 pgs: 129 active+clean; 18 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 41 MiB/s wr, 748 op/s 2024-09-13T18:28:05.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:05 smithi204 ceph-mon[34777]: pgmap v171: 129 pgs: 129 active+clean; 18 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 41 MiB/s wr, 748 op/s 2024-09-13T18:28:05.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:05 smithi114 ceph-mon[30897]: pgmap v171: 129 pgs: 129 active+clean; 18 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 41 MiB/s wr, 748 op/s 2024-09-13T18:28:07.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:07 smithi134 ceph-mon[34283]: pgmap v172: 129 pgs: 129 active+clean; 18 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 41 MiB/s wr, 767 op/s 2024-09-13T18:28:07.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:07 smithi204 ceph-mon[34777]: pgmap v172: 129 pgs: 129 active+clean; 18 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 41 MiB/s wr, 767 op/s 2024-09-13T18:28:07.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:07 smithi114 ceph-mon[30897]: pgmap v172: 129 pgs: 129 active+clean; 18 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 41 MiB/s wr, 767 op/s 2024-09-13T18:28:09.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:09 smithi134 ceph-mon[34283]: pgmap v173: 129 pgs: 129 active+clean; 18 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 35 MiB/s wr, 657 op/s 2024-09-13T18:28:09.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:09 smithi204 ceph-mon[34777]: pgmap v173: 129 pgs: 129 active+clean; 18 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 35 MiB/s wr, 657 op/s 2024-09-13T18:28:09.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:09 smithi114 ceph-mon[30897]: pgmap v173: 129 pgs: 129 active+clean; 18 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 35 MiB/s wr, 657 op/s 2024-09-13T18:28:10.431 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.428+0000 7f9313380640 1 -- 172.21.15.114:0/2091830564 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f930c107890 msgr2=0x7f930c107cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:10.431 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.428+0000 7f9313380640 1 --2- 172.21.15.114:0/2091830564 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f930c107890 0x7f930c107cd0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f92fc009900 tx=0x7f92fc02a420 comp rx=0 tx=0).stop 2024-09-13T18:28:10.440 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.437+0000 7f9313380640 1 -- 172.21.15.114:0/2091830564 shutdown_connections 2024-09-13T18:28:10.440 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.437+0000 7f9313380640 1 --2- 172.21.15.114:0/2091830564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f930c119400 0x7f930c11b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:10.440 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.437+0000 7f9313380640 1 --2- 172.21.15.114:0/2091830564 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f930c107890 0x7f930c107cd0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:10.440 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.437+0000 7f9313380640 1 --2- 172.21.15.114:0/2091830564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f930c1074b0 0x7f930c10e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:10.440 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.437+0000 7f9313380640 1 -- 172.21.15.114:0/2091830564 >> 172.21.15.114:0/2091830564 conn(0x7f930c06adf0 msgr2=0x7f930c06b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:10.443 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.439+0000 7f9313380640 1 -- 172.21.15.114:0/2091830564 shutdown_connections 2024-09-13T18:28:10.445 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.443+0000 7f9313380640 1 -- 172.21.15.114:0/2091830564 wait complete. 2024-09-13T18:28:10.446 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.443+0000 7f9313380640 1 Processor -- start 2024-09-13T18:28:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.444+0000 7f9313380640 1 -- start start 2024-09-13T18:28:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.444+0000 7f9313380640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f930c1074b0 0x7f930c116930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.444+0000 7f9313380640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f930c107890 0x7f930c116e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.444+0000 7f9313380640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f930c119400 0x7f930c10fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.444+0000 7f9313380640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930c110300 con 0x7f930c119400 2024-09-13T18:28:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.444+0000 7f9313380640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930c110470 con 0x7f930c107890 2024-09-13T18:28:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.444+0000 7f9313380640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930c110630 con 0x7f930c1074b0 2024-09-13T18:28:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.444+0000 7f931237e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f930c1074b0 0x7f930c116930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.444+0000 7f931237e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f930c1074b0 0x7f930c116930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:36580/0 (socket says 172.21.15.114:36580) 2024-09-13T18:28:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.444+0000 7f931237e640 1 -- 172.21.15.114:0/2049898226 learned_addr learned my addr 172.21.15.114:0/2049898226 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:10.448 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.445+0000 7f9311b7d640 1 --2- 172.21.15.114:0/2049898226 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f930c107890 0x7f930c116e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:10.448 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.446+0000 7f9312b7f640 1 --2- 172.21.15.114:0/2049898226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f930c119400 0x7f930c10fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:10.448 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.446+0000 7f931237e640 1 -- 172.21.15.114:0/2049898226 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f930c107890 msgr2=0x7f930c116e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:10.449 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.446+0000 7f931237e640 1 --2- 172.21.15.114:0/2049898226 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f930c107890 0x7f930c116e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:10.449 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.446+0000 7f931237e640 1 -- 172.21.15.114:0/2049898226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f930c119400 msgr2=0x7f930c10fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:10.449 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.446+0000 7f931237e640 1 --2- 172.21.15.114:0/2049898226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f930c119400 0x7f930c10fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:10.449 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.446+0000 7f931237e640 1 -- 172.21.15.114:0/2049898226 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92fc009560 con 0x7f930c1074b0 2024-09-13T18:28:10.449 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.446+0000 7f9312b7f640 1 --2- 172.21.15.114:0/2049898226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f930c119400 0x7f930c10fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:28:10.450 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.447+0000 7f931237e640 1 --2- 172.21.15.114:0/2049898226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f930c1074b0 0x7f930c116930 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f930400ba50 tx=0x7f930400bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:10.450 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.447+0000 7f93037fe640 1 -- 172.21.15.114:0/2049898226 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930400cc10 con 0x7f930c1074b0 2024-09-13T18:28:10.450 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.447+0000 7f93037fe640 1 -- 172.21.15.114:0/2049898226 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f930400cd70 con 0x7f930c1074b0 2024-09-13T18:28:10.450 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.447+0000 7f93037fe640 1 -- 172.21.15.114:0/2049898226 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93040076a0 con 0x7f930c1074b0 2024-09-13T18:28:10.450 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.447+0000 7f9313380640 1 -- 172.21.15.114:0/2049898226 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f930c114450 con 0x7f930c1074b0 2024-09-13T18:28:10.450 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.447+0000 7f9313380640 1 -- 172.21.15.114:0/2049898226 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f930c1148e0 con 0x7f930c1074b0 2024-09-13T18:28:10.454 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.452+0000 7f93037fe640 1 -- 172.21.15.114:0/2049898226 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f9304014440 con 0x7f930c1074b0 2024-09-13T18:28:10.456 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.454+0000 7f93173da740 1 Processor -- start 2024-09-13T18:28:10.457 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.455+0000 7f93037fe640 1 --2- 172.21.15.114:0/2049898226 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f92f80a1340 0x7f92f80a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:10.457 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.455+0000 7f93037fe640 1 -- 172.21.15.114:0/2049898226 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f93040cc7d0 con 0x7f930c1074b0 2024-09-13T18:28:10.461 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.458+0000 7f93173da740 1 -- start start 2024-09-13T18:28:10.461 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.458+0000 7f93173da740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570f6e23560 0x5570f6e23940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:10.461 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.458+0000 7f93173da740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570f6e23e80 0x5570f6e2e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:10.461 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.458+0000 7f93173da740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570f6e2e720 0x5570f6e30b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:10.461 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.458+0000 7f93173da740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570f6d8ec30 con 0x5570f6e2e720 2024-09-13T18:28:10.461 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.458+0000 7f93173da740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570f6d8c3c0 con 0x5570f6e23e80 2024-09-13T18:28:10.461 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.458+0000 7f93173da740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570f6d87cb0 con 0x5570f6e23560 2024-09-13T18:28:10.461 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.459+0000 7f931237e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570f6e23560 0x5570f6e23940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:10.462 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.459+0000 7f931237e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570f6e23560 0x5570f6e23940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:36590/0 (socket says 172.21.15.114:36590) 2024-09-13T18:28:10.462 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.459+0000 7f931237e640 1 -- 172.21.15.114:0/1223478009 learned_addr learned my addr 172.21.15.114:0/1223478009 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:10.462 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.460+0000 7f931237e640 1 -- 172.21.15.114:0/1223478009 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570f6e23e80 msgr2=0x5570f6e2e1e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:28:10.462 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.460+0000 7f931237e640 1 --2- 172.21.15.114:0/1223478009 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570f6e23e80 0x5570f6e2e1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:10.462 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.460+0000 7f931237e640 1 -- 172.21.15.114:0/1223478009 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570f6e2e720 msgr2=0x5570f6e30b10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:28:10.462 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.460+0000 7f931237e640 1 --2- 172.21.15.114:0/1223478009 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570f6e2e720 0x5570f6e30b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:10.462 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.460+0000 7f931237e640 1 -- 172.21.15.114:0/1223478009 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5570f6d2e910 con 0x5570f6e23560 2024-09-13T18:28:10.463 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.460+0000 7f931237e640 1 --2- 172.21.15.114:0/1223478009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570f6e23560 0x5570f6e23940 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9304096160 tx=0x7f93040992f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=167195e89c8ba761 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:10.463 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.461+0000 7f9300ff9640 1 -- 172.21.15.114:0/1223478009 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93040a1ac0 con 0x5570f6e23560 2024-09-13T18:28:10.463 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.461+0000 7f9300ff9640 1 -- 172.21.15.114:0/1223478009 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9304097a90 con 0x5570f6e23560 2024-09-13T18:28:10.464 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.461+0000 7f9300ff9640 1 -- 172.21.15.114:0/1223478009 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93040c9df0 con 0x5570f6e23560 2024-09-13T18:28:10.464 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.462+0000 7f93173da740 1 -- 172.21.15.114:0/1223478009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570f6e23560 msgr2=0x5570f6e23940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:10.464 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.462+0000 7f93173da740 1 --2- 172.21.15.114:0/1223478009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570f6e23560 0x5570f6e23940 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9304096160 tx=0x7f93040992f0 comp rx=0 tx=0).stop 2024-09-13T18:28:10.465 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.462+0000 7f9312b7f640 1 --2- 172.21.15.114:0/2049898226 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f92f80a1340 0x7f92f80a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:10.465 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.462+0000 7f93173da740 1 -- 172.21.15.114:0/1223478009 shutdown_connections 2024-09-13T18:28:10.465 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.462+0000 7f93173da740 1 --2- 172.21.15.114:0/1223478009 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570f6e2e720 0x5570f6e30b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:10.465 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.463+0000 7f93173da740 1 --2- 172.21.15.114:0/1223478009 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570f6e23e80 0x5570f6e2e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:10.465 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.463+0000 7f93173da740 1 --2- 172.21.15.114:0/1223478009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570f6e23560 0x5570f6e23940 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:10.465 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.463+0000 7f93173da740 1 -- 172.21.15.114:0/1223478009 >> 172.21.15.114:0/1223478009 conn(0x5570f6e1faf0 msgr2=0x5570f6e1fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:10.465 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.463+0000 7f93173da740 1 -- 172.21.15.114:0/1223478009 shutdown_connections 2024-09-13T18:28:10.466 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.463+0000 7f9312b7f640 1 --2- 172.21.15.114:0/2049898226 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f92f80a1340 0x7f92f80a3800 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x5570f6e20e70 tx=0x7f9308008040 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:10.466 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.463+0000 7f93173da740 1 -- 172.21.15.114:0/1223478009 wait complete. 2024-09-13T18:28:10.466 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.464+0000 7f93173da740 1 -- start start 2024-09-13T18:28:10.466 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.464+0000 7f93173da740 1 Processor -- start 2024-09-13T18:28:10.467 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.464+0000 7f93173da740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570f6e23560 0x5570f6e2e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:10.467 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.464+0000 7f93173da740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570f6e23e80 0x5570f6e2b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:10.467 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.464+0000 7f93173da740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570f6e2e720 0x5570f6e2b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:10.467 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.464+0000 7f93173da740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570f6d45e80 con 0x5570f6e23560 2024-09-13T18:28:10.467 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.464+0000 7f93173da740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570f6d45b20 con 0x5570f6e23e80 2024-09-13T18:28:10.467 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.464+0000 7f93173da740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570f6d0b260 con 0x5570f6e2e720 2024-09-13T18:28:10.467 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.464+0000 7f93173da740 10 client.0 _ll_register_callbacks cb 0x5570f6d40df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:28:10.468 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.465+0000 7f9311b7d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570f6e23e80 0x5570f6e2b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:10.468 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.465+0000 7f9311b7d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570f6e23e80 0x5570f6e2b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:52846/0 (socket says 172.21.15.114:52846) 2024-09-13T18:28:10.468 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.465+0000 7f9311b7d640 1 -- 172.21.15.114:0/2798130323 learned_addr learned my addr 172.21.15.114:0/2798130323 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:10.468 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.465+0000 7f931237e640 1 --2- 172.21.15.114:0/2798130323 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570f6e23560 0x5570f6e2e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:10.468 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.465+0000 7f9311b7d640 1 -- 172.21.15.114:0/2798130323 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570f6e2e720 msgr2=0x5570f6e2b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:10.468 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.465+0000 7f9311b7d640 1 --2- 172.21.15.114:0/2798130323 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570f6e2e720 0x5570f6e2b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:10.468 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.465+0000 7f9311b7d640 1 -- 172.21.15.114:0/2798130323 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570f6e23560 msgr2=0x5570f6e2e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:10.468 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.465+0000 7f9311b7d640 1 --2- 172.21.15.114:0/2798130323 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570f6e23560 0x5570f6e2e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:10.468 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.465+0000 7f9311b7d640 1 -- 172.21.15.114:0/2798130323 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f930c1148e0 con 0x5570f6e23e80 2024-09-13T18:28:10.469 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.467+0000 7f9311b7d640 1 --2- 172.21.15.114:0/2798130323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570f6e23e80 0x5570f6e2b2a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f92fc009900 tx=0x7f92fc0053e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91ec8f58bbbe5bf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:10.471 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.468+0000 7f92f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:28:10.471 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.469+0000 7f92f5ffb640 1 -- 172.21.15.114:0/2798130323 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92fc004ad0 con 0x5570f6e23e80 2024-09-13T18:28:10.471 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.469+0000 7f93173da740 1 -- 172.21.15.114:0/2798130323 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5570f6db0680 con 0x5570f6e23e80 2024-09-13T18:28:10.472 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.469+0000 7f92f5ffb640 1 -- 172.21.15.114:0/2798130323 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f92fc004c30 con 0x5570f6e23e80 2024-09-13T18:28:10.472 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.470+0000 7f92f5ffb640 1 -- 172.21.15.114:0/2798130323 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92fc004df0 con 0x5570f6e23e80 2024-09-13T18:28:10.472 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.470+0000 7f93173da740 10 client.26708 fetch_fsmap learned FSMap version 34 2024-09-13T18:28:10.472 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.470+0000 7f93173da740 1 -- 172.21.15.114:0/2798130323 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5570f6d86330 con 0x5570f6e23e80 2024-09-13T18:28:10.473 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.471+0000 7f92f5ffb640 1 -- 172.21.15.114:0/2798130323 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5570f6db0680 con 0x5570f6e23e80 2024-09-13T18:28:10.473 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.471+0000 7f92f5ffb640 1 -- 172.21.15.114:0/2798130323 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f92fc03d5e0 con 0x5570f6e23e80 2024-09-13T18:28:10.473 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.471+0000 7f93173da740 10 client.26708 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:28:10.474 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.471+0000 7f93173da740 10 client.26708 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:28:10.474 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.471+0000 7f93173da740 20 client.26708 populate_metadata read hostname 'smithi114' 2024-09-13T18:28:10.474 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.471+0000 7f93173da740 1 --2- 172.21.15.114:0/2798130323 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5570f6e3e2b0 0x5570f6e406d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:10.474 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.471+0000 7f93173da740 4 client.26708 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:28:10.474 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.471+0000 7f93173da740 1 -- 172.21.15.114:0/2798130323 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5570f6d7b5d0 con 0x5570f6e3e2b0 2024-09-13T18:28:10.479 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.477+0000 7f931237e640 1 --2- 172.21.15.114:0/2798130323 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5570f6e3e2b0 0x5570f6e406d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:10.480 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.478+0000 7f931237e640 1 --2- 172.21.15.114:0/2798130323 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5570f6e3e2b0 0x5570f6e406d0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71a167d9f0ce4fd1 server_cookie=4ce7b8194e31995b in_seq=0 out_seq=0 2024-09-13T18:28:10.480 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.478+0000 7f92f5ffb640 10 client.26708 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:28:10.481 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.479+0000 7f92f5ffb640 1 -- 172.21.15.114:0/2798130323 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5570f6d7b5d0 con 0x5570f6e3e2b0 2024-09-13T18:28:10.481 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.479+0000 7f92f5ffb640 10 client.26708 handle_command_reply: tid=0 2024-09-13T18:28:10.481 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.479+0000 7f93173da740 1 client.26708 shutdown 2024-09-13T18:28:10.482 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.480+0000 7f93173da740 1 -- 172.21.15.114:0/2798130323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570f6e23e80 msgr2=0x5570f6e2b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:10.482 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.480+0000 7f93173da740 1 --2- 172.21.15.114:0/2798130323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570f6e23e80 0x5570f6e2b2a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f92fc009900 tx=0x7f92fc0053e0 comp rx=0 tx=0).stop 2024-09-13T18:28:10.484 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.482+0000 7f93173da740 1 -- 172.21.15.114:0/2798130323 shutdown_connections 2024-09-13T18:28:10.485 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.482+0000 7f93173da740 1 --2- 172.21.15.114:0/2798130323 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5570f6e3e2b0 0x5570f6e406d0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:10.485 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.482+0000 7f93173da740 1 --2- 172.21.15.114:0/2798130323 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570f6e2e720 0x5570f6e2b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:10.485 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.482+0000 7f93173da740 1 --2- 172.21.15.114:0/2798130323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570f6e23e80 0x5570f6e2b2a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:10.485 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.482+0000 7f93173da740 1 --2- 172.21.15.114:0/2798130323 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570f6e23560 0x5570f6e2e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:10.485 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.482+0000 7f93173da740 1 -- 172.21.15.114:0/2798130323 >> 172.21.15.114:0/2798130323 conn(0x5570f6e1faf0 msgr2=0x5570f6e1ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:10.485 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.482+0000 7f92f5ffb640 0 client.26708 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:28:10.488 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.485+0000 7f93173da740 1 -- 172.21.15.114:0/2798130323 shutdown_connections 2024-09-13T18:28:10.489 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.487+0000 7f93173da740 1 -- 172.21.15.114:0/2798130323 wait complete. 2024-09-13T18:28:10.489 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.487+0000 7f93173da740 20 client.26708 trim_cache size 0 max 16384 2024-09-13T18:28:10.518 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.515+0000 7f93173da740 1 Processor -- start 2024-09-13T18:28:10.518 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.516+0000 7f93173da740 1 -- start start 2024-09-13T18:28:10.518 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.516+0000 7f93173da740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570f6e23560 0x5570f6e29b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:10.518 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.516+0000 7f93173da740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570f6e23e80 0x5570f6e2a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:10.518 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.516+0000 7f93173da740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570f6e2e720 0x5570f6e284d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:10.518 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.516+0000 7f93173da740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570f6e20c60 con 0x5570f6e23e80 2024-09-13T18:28:10.518 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.516+0000 7f93173da740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570f6e3ae60 con 0x5570f6e23560 2024-09-13T18:28:10.519 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.516+0000 7f93173da740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570f6e2a5d0 con 0x5570f6e2e720 2024-09-13T18:28:10.521 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.519+0000 7f9312b7f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570f6e2e720 0x5570f6e284d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:10.521 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.519+0000 7f9312b7f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570f6e2e720 0x5570f6e284d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:36596/0 (socket says 172.21.15.114:36596) 2024-09-13T18:28:10.521 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.519+0000 7f9312b7f640 1 -- 172.21.15.114:0/3233313742 learned_addr learned my addr 172.21.15.114:0/3233313742 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:10.523 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.521+0000 7f9312b7f640 1 -- 172.21.15.114:0/3233313742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570f6e23560 msgr2=0x5570f6e29b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:10.523 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.521+0000 7f9312b7f640 1 --2- 172.21.15.114:0/3233313742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570f6e23560 0x5570f6e29b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:10.523 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.521+0000 7f9312b7f640 1 -- 172.21.15.114:0/3233313742 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570f6e23e80 msgr2=0x5570f6e2a090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:28:10.523 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.521+0000 7f9312b7f640 1 --2- 172.21.15.114:0/3233313742 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570f6e23e80 0x5570f6e2a090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:10.523 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.521+0000 7f9312b7f640 1 -- 172.21.15.114:0/3233313742 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5570f6d28300 con 0x5570f6e2e720 2024-09-13T18:28:10.526 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.522+0000 7f9312b7f640 1 --2- 172.21.15.114:0/3233313742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570f6e2e720 0x5570f6e284d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9308009630 tx=0x7f93080046e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f3bccf3012bf5b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:10.526 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.522+0000 7f93017fa640 1 -- 172.21.15.114:0/3233313742 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9308014840 con 0x5570f6e2e720 2024-09-13T18:28:10.526 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.522+0000 7f93017fa640 1 -- 172.21.15.114:0/3233313742 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9308014e30 con 0x5570f6e2e720 2024-09-13T18:28:10.526 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.523+0000 7f93173da740 1 -- 172.21.15.114:0/3233313742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570f6e2e720 msgr2=0x5570f6e284d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:10.526 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.523+0000 7f93173da740 1 --2- 172.21.15.114:0/3233313742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570f6e2e720 0x5570f6e284d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9308009630 tx=0x7f93080046e0 comp rx=0 tx=0).stop 2024-09-13T18:28:10.526 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.523+0000 7f93173da740 1 -- 172.21.15.114:0/3233313742 shutdown_connections 2024-09-13T18:28:10.526 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.523+0000 7f93173da740 1 --2- 172.21.15.114:0/3233313742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570f6e2e720 0x5570f6e284d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:10.527 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.523+0000 7f93173da740 1 --2- 172.21.15.114:0/3233313742 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570f6e23e80 0x5570f6e2a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:10.527 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.523+0000 7f93173da740 1 --2- 172.21.15.114:0/3233313742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570f6e23560 0x5570f6e29b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:10.527 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.523+0000 7f93173da740 1 -- 172.21.15.114:0/3233313742 >> 172.21.15.114:0/3233313742 conn(0x5570f6e1faf0 msgr2=0x5570f6e1fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:10.527 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.523+0000 7f93173da740 1 -- 172.21.15.114:0/3233313742 shutdown_connections 2024-09-13T18:28:10.527 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.523+0000 7f93173da740 1 -- 172.21.15.114:0/3233313742 wait complete. 2024-09-13T18:28:10.527 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.524+0000 7f93173da740 1 -- start start 2024-09-13T18:28:10.527 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.524+0000 7f93173da740 1 Processor -- start 2024-09-13T18:28:10.528 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.524+0000 7f93173da740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570f6e23560 0x5570f6e35970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:10.528 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.524+0000 7f93173da740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570f6e23e80 0x5570f6e33c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:10.528 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.524+0000 7f93173da740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570f6e34250 0x5570f6e346d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:10.528 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.524+0000 7f93173da740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570f6e94900 con 0x5570f6e34250 2024-09-13T18:28:10.528 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.524+0000 7f93173da740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570f6e94a70 con 0x5570f6e23560 2024-09-13T18:28:10.528 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.524+0000 7f93173da740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570f6e94c30 con 0x5570f6e23e80 2024-09-13T18:28:10.528 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.524+0000 7f93173da740 10 client.0 _ll_register_callbacks cb 0x7f9304014cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:28:10.528 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.525+0000 7f9311b7d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570f6e23e80 0x5570f6e33c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:10.528 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.525+0000 7f931237e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570f6e23560 0x5570f6e35970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:10.528 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.525+0000 7f931237e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570f6e23560 0x5570f6e35970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:52870/0 (socket says 172.21.15.114:52870) 2024-09-13T18:28:10.529 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.525+0000 7f931237e640 1 -- 172.21.15.114:0/2191796330 learned_addr learned my addr 172.21.15.114:0/2191796330 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:10.529 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.525+0000 7f9312b7f640 1 --2- 172.21.15.114:0/2191796330 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570f6e34250 0x5570f6e346d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:10.530 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.528+0000 7f931237e640 1 -- 172.21.15.114:0/2191796330 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570f6e23e80 msgr2=0x5570f6e33c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:10.530 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.528+0000 7f931237e640 1 --2- 172.21.15.114:0/2191796330 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570f6e23e80 0x5570f6e33c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:10.530 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.528+0000 7f931237e640 1 -- 172.21.15.114:0/2191796330 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570f6e34250 msgr2=0x5570f6e346d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:10.530 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.528+0000 7f931237e640 1 --2- 172.21.15.114:0/2191796330 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570f6e34250 0x5570f6e346d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:10.530 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.528+0000 7f931237e640 1 -- 172.21.15.114:0/2191796330 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f930800e040 con 0x5570f6e23560 2024-09-13T18:28:10.532 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.529+0000 7f931237e640 1 --2- 172.21.15.114:0/2191796330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570f6e23560 0x5570f6e35970 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f930400c7d0 tx=0x7f9304009710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b32e12acdee2b1fe server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:10.532 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.529+0000 7f92f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:28:10.533 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.531+0000 7f92f5ffb640 1 -- 172.21.15.114:0/2191796330 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93040cc400 con 0x5570f6e23560 2024-09-13T18:28:10.533 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.531+0000 7f92f5ffb640 1 -- 172.21.15.114:0/2191796330 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f93040cc560 con 0x5570f6e23560 2024-09-13T18:28:10.534 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.531+0000 7f93173da740 1 -- 172.21.15.114:0/2191796330 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5570f6a7b300 con 0x5570f6e23560 2024-09-13T18:28:10.534 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.532+0000 7f92f5ffb640 1 -- 172.21.15.114:0/2191796330 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93040c9df0 con 0x5570f6e23560 2024-09-13T18:28:10.534 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.532+0000 7f93173da740 10 client.26711 fetch_fsmap learned FSMap version 34 2024-09-13T18:28:10.534 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.532+0000 7f93173da740 1 -- 172.21.15.114:0/2191796330 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5570f6d4d8e0 con 0x5570f6e23560 2024-09-13T18:28:10.536 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.534+0000 7f92f5ffb640 1 -- 172.21.15.114:0/2191796330 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f93040a5020 con 0x5570f6e23560 2024-09-13T18:28:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.535+0000 7f92f5ffb640 1 -- 172.21.15.114:0/2191796330 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f93040a1630 con 0x5570f6e23560 2024-09-13T18:28:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.535+0000 7f93173da740 10 client.26711 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:28:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.535+0000 7f93173da740 10 client.26711 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:28:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.535+0000 7f93173da740 20 client.26711 populate_metadata read hostname 'smithi114' 2024-09-13T18:28:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.535+0000 7f93173da740 1 --2- 172.21.15.114:0/2191796330 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5570f6e961f0 0x5570f6e985e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.535+0000 7f93173da740 4 client.26711 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T18:28:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.535+0000 7f93173da740 1 -- 172.21.15.114:0/2191796330 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5570f6a8c600 con 0x5570f6e961f0 2024-09-13T18:28:10.543 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.541+0000 7f9311b7d640 1 --2- 172.21.15.114:0/2191796330 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5570f6e961f0 0x5570f6e985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:10.561 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.559+0000 7f9311b7d640 1 --2- 172.21.15.114:0/2191796330 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5570f6e961f0 0x5570f6e985e0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d03552177870f2f server_cookie=45e22e84f130cfba in_seq=0 out_seq=0 2024-09-13T18:28:10.561 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.560+0000 7f92f5ffb640 10 client.26711 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:28:10.605 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.603+0000 7f92f5ffb640 1 -- 172.21.15.114:0/2191796330 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5570f6a8c600 con 0x5570f6e961f0 2024-09-13T18:28:10.605 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.603+0000 7f92f5ffb640 10 client.26711 handle_command_reply: tid=0 2024-09-13T18:28:10.605 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.603+0000 7f93173da740 1 client.26711 shutdown 2024-09-13T18:28:10.611 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.609+0000 7f93173da740 1 -- 172.21.15.114:0/2191796330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570f6e23560 msgr2=0x5570f6e35970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:10.611 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.609+0000 7f93173da740 1 --2- 172.21.15.114:0/2191796330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570f6e23560 0x5570f6e35970 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f930400c7d0 tx=0x7f9304009710 comp rx=0 tx=0).stop 2024-09-13T18:28:10.612 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.610+0000 7f93173da740 1 -- 172.21.15.114:0/2191796330 shutdown_connections 2024-09-13T18:28:10.612 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.610+0000 7f93173da740 1 --2- 172.21.15.114:0/2191796330 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5570f6e961f0 0x5570f6e985e0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:10.612 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.610+0000 7f93173da740 1 --2- 172.21.15.114:0/2191796330 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570f6e34250 0x5570f6e346d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:10.612 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.610+0000 7f93173da740 1 --2- 172.21.15.114:0/2191796330 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570f6e23e80 0x5570f6e33c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:10.612 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.610+0000 7f93173da740 1 --2- 172.21.15.114:0/2191796330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570f6e23560 0x5570f6e35970 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:10.612 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.610+0000 7f93173da740 1 -- 172.21.15.114:0/2191796330 >> 172.21.15.114:0/2191796330 conn(0x5570f6e1faf0 msgr2=0x5570f6e29750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.614+0000 7f92f5ffb640 0 client.26711 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:28:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.614+0000 7f93173da740 1 -- 172.21.15.114:0/2191796330 shutdown_connections 2024-09-13T18:28:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.614+0000 7f93173da740 1 -- 172.21.15.114:0/2191796330 wait complete. 2024-09-13T18:28:10.619 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-13T18:28:10.620 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-13T18:28:10.620 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-13T18:28:10.620 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-13T18:28:10.620 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.616+0000 7f93173da740 20 client.26711 trim_cache size 0 max 16384 2024-09-13T18:28:10.622 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.620+0000 7f9313380640 1 -- 172.21.15.114:0/2049898226 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f92f80a1340 msgr2=0x7f92f80a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:10.622 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.620+0000 7f9313380640 1 --2- 172.21.15.114:0/2049898226 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f92f80a1340 0x7f92f80a3800 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x5570f6e20e70 tx=0x7f9308008040 comp rx=0 tx=0).stop 2024-09-13T18:28:10.622 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.620+0000 7f9313380640 1 -- 172.21.15.114:0/2049898226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f930c1074b0 msgr2=0x7f930c116930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:10.622 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.620+0000 7f9313380640 1 --2- 172.21.15.114:0/2049898226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f930c1074b0 0x7f930c116930 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f930400ba50 tx=0x7f930400bf10 comp rx=0 tx=0).stop 2024-09-13T18:28:10.623 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.621+0000 7f9313380640 1 -- 172.21.15.114:0/2049898226 shutdown_connections 2024-09-13T18:28:10.623 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.621+0000 7f9313380640 1 --2- 172.21.15.114:0/2049898226 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f92f80a1340 0x7f92f80a3800 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:10.624 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.621+0000 7f9313380640 1 --2- 172.21.15.114:0/2049898226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f930c119400 0x7f930c10fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:10.624 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.621+0000 7f9313380640 1 --2- 172.21.15.114:0/2049898226 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f930c107890 0x7f930c116e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:10.624 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.621+0000 7f9313380640 1 --2- 172.21.15.114:0/2049898226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f930c1074b0 0x7f930c116930 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:10.624 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.621+0000 7f9313380640 1 -- 172.21.15.114:0/2049898226 >> 172.21.15.114:0/2049898226 conn(0x7f930c06adf0 msgr2=0x7f930c119ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:10.626 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.623+0000 7f9313380640 1 -- 172.21.15.114:0/2049898226 shutdown_connections 2024-09-13T18:28:10.626 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:10.624+0000 7f9313380640 1 -- 172.21.15.114:0/2049898226 wait complete. 2024-09-13T18:28:11.868 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:11 smithi134 ceph-mon[34283]: pgmap v174: 129 pgs: 129 active+clean; 18 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 51 MiB/s wr, 974 op/s 2024-09-13T18:28:11.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:11 smithi204 ceph-mon[34777]: pgmap v174: 129 pgs: 129 active+clean; 18 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 51 MiB/s wr, 974 op/s 2024-09-13T18:28:11.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:11 smithi114 ceph-mon[30897]: pgmap v174: 129 pgs: 129 active+clean; 18 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 51 MiB/s wr, 974 op/s 2024-09-13T18:28:12.857 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T18:28:13.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.088+0000 7fbd08687640 1 -- 172.21.15.114:0/1403715093 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbd00107920 msgr2=0x7fbd00107d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:13.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.088+0000 7fbd08687640 1 --2- 172.21.15.114:0/1403715093 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbd00107920 0x7fbd00107d60 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0009900 tx=0x7fbcf002a440 comp rx=0 tx=0).stop 2024-09-13T18:28:13.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.091+0000 7fbd06c24640 1 --2- 172.21.15.114:0/1403715093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd00119400 0x7fbd0011b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:28:13.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.094+0000 7fbd08687640 1 -- 172.21.15.114:0/1403715093 shutdown_connections 2024-09-13T18:28:13.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.094+0000 7fbd08687640 1 --2- 172.21.15.114:0/1403715093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd00119400 0x7fbd0011b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:13.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.094+0000 7fbd08687640 1 --2- 172.21.15.114:0/1403715093 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbd00107920 0x7fbd00107d60 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:13.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.094+0000 7fbd08687640 1 --2- 172.21.15.114:0/1403715093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd00107540 0x7fbd0010e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:13.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.094+0000 7fbd08687640 1 -- 172.21.15.114:0/1403715093 >> 172.21.15.114:0/1403715093 conn(0x7fbd0006afe0 msgr2=0x7fbd0006b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:13.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.101+0000 7fbd08687640 1 -- 172.21.15.114:0/1403715093 shutdown_connections 2024-09-13T18:28:13.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.105+0000 7fbd08687640 1 -- 172.21.15.114:0/1403715093 wait complete. 2024-09-13T18:28:13.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.105+0000 7fbd08687640 1 Processor -- start 2024-09-13T18:28:13.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.106+0000 7fbd08687640 1 -- start start 2024-09-13T18:28:13.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.106+0000 7fbd08687640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd00107540 0x7fbd001a4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:13.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.106+0000 7fbd08687640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbd00107920 0x7fbd001a54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:13.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.106+0000 7fbd08687640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd00119400 0x7fbd001a9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:13.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.106+0000 7fbd08687640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd001a9e90 con 0x7fbd00107920 2024-09-13T18:28:13.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.106+0000 7fbd08687640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd001a9fd0 con 0x7fbd00119400 2024-09-13T18:28:13.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.106+0000 7fbd08687640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd001aa190 con 0x7fbd00107540 2024-09-13T18:28:13.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.106+0000 7fbd05c22640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbd00107920 0x7fbd001a54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:13.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.106+0000 7fbd05c22640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbd00107920 0x7fbd001a54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:58072/0 (socket says 172.21.15.114:58072) 2024-09-13T18:28:13.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.106+0000 7fbd05c22640 1 -- 172.21.15.114:0/3923578858 learned_addr learned my addr 172.21.15.114:0/3923578858 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:13.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.107+0000 7fbd05c22640 1 -- 172.21.15.114:0/3923578858 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd00107540 msgr2=0x7fbd001a4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:13.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.107+0000 7fbd05c22640 1 --2- 172.21.15.114:0/3923578858 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd00107540 0x7fbd001a4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:13.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.107+0000 7fbd05c22640 1 -- 172.21.15.114:0/3923578858 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd00119400 msgr2=0x7fbd001a9890 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:28:13.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.107+0000 7fbd05c22640 1 --2- 172.21.15.114:0/3923578858 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd00119400 0x7fbd001a9890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:13.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.107+0000 7fbd05c22640 1 -- 172.21.15.114:0/3923578858 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcf0009560 con 0x7fbd00107920 2024-09-13T18:28:13.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.107+0000 7fbd05c22640 1 --2- 172.21.15.114:0/3923578858 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbd00107920 0x7fbd001a54a0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fbcf002aa30 tx=0x7fbcf0003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:13.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.108+0000 7fbcf77fe640 1 -- 172.21.15.114:0/3923578858 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf0005440 con 0x7fbd00107920 2024-09-13T18:28:13.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.108+0000 7fbcf77fe640 1 -- 172.21.15.114:0/3923578858 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbcf00055a0 con 0x7fbd00107920 2024-09-13T18:28:13.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.108+0000 7fbcf77fe640 1 -- 172.21.15.114:0/3923578858 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf003b950 con 0x7fbd00107920 2024-09-13T18:28:13.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.110+0000 7fbd08687640 1 -- 172.21.15.114:0/3923578858 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd001aa410 con 0x7fbd00107920 2024-09-13T18:28:13.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.110+0000 7fbd08687640 1 -- 172.21.15.114:0/3923578858 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd0010ebe0 con 0x7fbd00107920 2024-09-13T18:28:13.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.128+0000 7fbd0b483740 1 Processor -- start 2024-09-13T18:28:13.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.132+0000 7fbd0b483740 1 -- start start 2024-09-13T18:28:13.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.132+0000 7fbd0b483740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562607ef5e10 0x562607ef61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:13.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.132+0000 7fbd0b483740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562607ef6730 0x562607f00a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:13.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.132+0000 7fbd0b483740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562607f00fd0 0x562607f033c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:13.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.132+0000 7fbd0b483740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x562607e60f20 con 0x562607ef6730 2024-09-13T18:28:13.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.132+0000 7fbd0b483740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562607e5e6b0 con 0x562607f00fd0 2024-09-13T18:28:13.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.132+0000 7fbd0b483740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562607e59fa0 con 0x562607ef5e10 2024-09-13T18:28:13.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.133+0000 7fbd06c24640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562607ef5e10 0x562607ef61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:13.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.133+0000 7fbd06c24640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562607ef5e10 0x562607ef61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:36620/0 (socket says 172.21.15.114:36620) 2024-09-13T18:28:13.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.133+0000 7fbd06c24640 1 -- 172.21.15.114:0/2399995736 learned_addr learned my addr 172.21.15.114:0/2399995736 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:13.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.133+0000 7fbcf77fe640 1 -- 172.21.15.114:0/3923578858 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fbcf00046c0 con 0x7fbd00107920 2024-09-13T18:28:13.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.133+0000 7fbd05c22640 1 --2- 172.21.15.114:0/2399995736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562607f00fd0 0x562607f033c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:13.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.134+0000 7fbd06c24640 1 -- 172.21.15.114:0/2399995736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562607f00fd0 msgr2=0x562607f033c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:13.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.134+0000 7fbd06c24640 1 --2- 172.21.15.114:0/2399995736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562607f00fd0 0x562607f033c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:13.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.134+0000 7fbd06c24640 1 -- 172.21.15.114:0/2399995736 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562607ef6730 msgr2=0x562607f00a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:13.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.134+0000 7fbd06c24640 1 --2- 172.21.15.114:0/2399995736 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562607ef6730 0x562607f00a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:13.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.134+0000 7fbd06c24640 1 -- 172.21.15.114:0/2399995736 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562607e14840 con 0x562607ef5e10 2024-09-13T18:28:13.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.134+0000 7fbd06c24640 1 --2- 172.21.15.114:0/2399995736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562607ef5e10 0x562607ef61f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8007c70 tx=0x7fbcf800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c6f1ab552ca60d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:13.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.134+0000 7fbcf4ff9640 1 -- 172.21.15.114:0/2399995736 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf800f650 con 0x562607ef5e10 2024-09-13T18:28:13.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.134+0000 7fbcf4ff9640 1 -- 172.21.15.114:0/2399995736 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbcf800fc40 con 0x562607ef5e10 2024-09-13T18:28:13.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.135+0000 7fbcf4ff9640 1 -- 172.21.15.114:0/2399995736 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf800eb20 con 0x562607ef5e10 2024-09-13T18:28:13.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.135+0000 7fbd0b483740 1 -- 172.21.15.114:0/2399995736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562607ef5e10 msgr2=0x562607ef61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:13.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.135+0000 7fbd0b483740 1 --2- 172.21.15.114:0/2399995736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562607ef5e10 0x562607ef61f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8007c70 tx=0x7fbcf800c6a0 comp rx=0 tx=0).stop 2024-09-13T18:28:13.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.136+0000 7fbd0b483740 1 -- 172.21.15.114:0/2399995736 shutdown_connections 2024-09-13T18:28:13.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.136+0000 7fbd0b483740 1 --2- 172.21.15.114:0/2399995736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562607f00fd0 0x562607f033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:13.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.136+0000 7fbd0b483740 1 --2- 172.21.15.114:0/2399995736 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562607ef6730 0x562607f00a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:13.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.136+0000 7fbd0b483740 1 --2- 172.21.15.114:0/2399995736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562607ef5e10 0x562607ef61f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:13.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.136+0000 7fbd0b483740 1 -- 172.21.15.114:0/2399995736 >> 172.21.15.114:0/2399995736 conn(0x562607ef2380 msgr2=0x562607ef2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:13.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.138+0000 7fbd0b483740 1 -- 172.21.15.114:0/2399995736 shutdown_connections 2024-09-13T18:28:13.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.138+0000 7fbd0b483740 1 -- 172.21.15.114:0/2399995736 wait complete. 2024-09-13T18:28:13.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.139+0000 7fbd0b483740 1 -- start start 2024-09-13T18:28:13.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.142+0000 7fbd0b483740 1 Processor -- start 2024-09-13T18:28:13.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.142+0000 7fbd0b483740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562607ef5e10 0x562607f008d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:13.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.142+0000 7fbd0b483740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562607ef6730 0x562607efdb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:13.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.142+0000 7fbd0b483740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562607f00fd0 0x562607efe040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:13.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.142+0000 7fbd0b483740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x562607dfb2a0 con 0x562607ef6730 2024-09-13T18:28:13.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.142+0000 7fbd0b483740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562607b32430 con 0x562607ef5e10 2024-09-13T18:28:13.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.142+0000 7fbd0b483740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562607ddd680 con 0x562607f00fd0 2024-09-13T18:28:13.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.142+0000 7fbd0b483740 10 client.0 _ll_register_callbacks cb 0x562607e53610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:28:13.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.143+0000 7fbcf77fe640 1 --2- 172.21.15.114:0/3923578858 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbcec0a12e0 0x7fbcec0a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:13.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.143+0000 7fbd05c22640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562607f00fd0 0x562607efe040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:13.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.143+0000 7fbd06c24640 1 --2- 172.21.15.114:0/3923578858 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbcec0a12e0 0x7fbcec0a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:13.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.143+0000 7fbd06c24640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562607ef5e10 0x562607f008d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:13.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.143+0000 7fbd06c24640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562607ef5e10 0x562607f008d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:52892/0 (socket says 172.21.15.114:52892) 2024-09-13T18:28:13.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.143+0000 7fbd06c24640 1 -- 172.21.15.114:0/2362596963 learned_addr learned my addr 172.21.15.114:0/2362596963 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:13.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.143+0000 7fbcf77fe640 1 -- 172.21.15.114:0/3923578858 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fbcf00edb70 con 0x7fbd00107920 2024-09-13T18:28:13.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.144+0000 7fbd06423640 1 --2- 172.21.15.114:0/2362596963 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562607ef6730 0x562607efdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:13.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.144+0000 7fbd06c24640 1 -- 172.21.15.114:0/2362596963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562607f00fd0 msgr2=0x562607efe040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:13.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.144+0000 7fbd06c24640 1 --2- 172.21.15.114:0/2362596963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562607f00fd0 0x562607efe040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:13.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.144+0000 7fbd06c24640 1 -- 172.21.15.114:0/2362596963 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562607ef6730 msgr2=0x562607efdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:13.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.144+0000 7fbd06c24640 1 --2- 172.21.15.114:0/2362596963 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562607ef6730 0x562607efdb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:13.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.144+0000 7fbd06c24640 1 -- 172.21.15.114:0/2362596963 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcf80078a0 con 0x562607ef5e10 2024-09-13T18:28:13.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.144+0000 7fbd06c24640 1 --2- 172.21.15.114:0/2362596963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562607ef5e10 0x562607f008d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fbcf800f4d0 tx=0x7fbcf800f500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ce90f245e1fd66c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:13.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.144+0000 7fbce9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:28:13.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.144+0000 7fbce9ffb640 1 -- 172.21.15.114:0/2362596963 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf800f930 con 0x562607ef5e10 2024-09-13T18:28:13.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.145+0000 7fbd0b483740 1 -- 172.21.15.114:0/2362596963 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562607b4d4e0 con 0x562607ef5e10 2024-09-13T18:28:13.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.145+0000 7fbce9ffb640 1 -- 172.21.15.114:0/2362596963 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fbcf8028480 con 0x562607ef5e10 2024-09-13T18:28:13.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.145+0000 7fbce9ffb640 1 -- 172.21.15.114:0/2362596963 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbcf800fa90 con 0x562607ef5e10 2024-09-13T18:28:13.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.145+0000 7fbd0b483740 10 client.26717 fetch_fsmap learned FSMap version 34 2024-09-13T18:28:13.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.145+0000 7fbd0b483740 1 -- 172.21.15.114:0/2362596963 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x562607e58620 con 0x562607ef5e10 2024-09-13T18:28:13.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.146+0000 7fbce9ffb640 1 -- 172.21.15.114:0/2362596963 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562607b4d4e0 con 0x562607ef5e10 2024-09-13T18:28:13.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.146+0000 7fbce9ffb640 1 -- 172.21.15.114:0/2362596963 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbcf8019790 con 0x562607ef5e10 2024-09-13T18:28:13.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.146+0000 7fbd0b483740 10 client.26717 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:28:13.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.146+0000 7fbd0b483740 10 client.26717 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:28:13.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.146+0000 7fbd0b483740 20 client.26717 populate_metadata read hostname 'smithi114' 2024-09-13T18:28:13.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.146+0000 7fbd0b483740 1 --2- 172.21.15.114:0/2362596963 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x562607f10f90 0x562607f133b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:13.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.146+0000 7fbd0b483740 4 client.26717 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:28:13.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.146+0000 7fbd0b483740 1 -- 172.21.15.114:0/2362596963 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562607b5e7e0 con 0x562607f10f90 2024-09-13T18:28:13.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.146+0000 7fbd06423640 1 --2- 172.21.15.114:0/2362596963 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x562607f10f90 0x562607f133b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:13.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.147+0000 7fbd06c24640 1 --2- 172.21.15.114:0/3923578858 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbcec0a12e0 0x7fbcec0a37a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8010020 tx=0x7fbcf8030720 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:13.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.147+0000 7fbd06423640 1 --2- 172.21.15.114:0/2362596963 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x562607f10f90 0x562607f133b0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f51eb07d8f46046 server_cookie=cb4b0f0b2c685b32 in_seq=0 out_seq=0 2024-09-13T18:28:13.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.147+0000 7fbce9ffb640 10 client.26717 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:28:13.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.148+0000 7fbce9ffb640 1 -- 172.21.15.114:0/2362596963 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x562607b5e7e0 con 0x562607f10f90 2024-09-13T18:28:13.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.148+0000 7fbce9ffb640 10 client.26717 handle_command_reply: tid=0 2024-09-13T18:28:13.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.148+0000 7fbd0b483740 1 client.26717 shutdown 2024-09-13T18:28:13.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.153+0000 7fbd0b483740 1 -- 172.21.15.114:0/2362596963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562607ef5e10 msgr2=0x562607f008d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:13.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.153+0000 7fbd0b483740 1 --2- 172.21.15.114:0/2362596963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562607ef5e10 0x562607f008d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fbcf800f4d0 tx=0x7fbcf800f500 comp rx=0 tx=0).stop 2024-09-13T18:28:13.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.154+0000 7fbd0b483740 1 -- 172.21.15.114:0/2362596963 shutdown_connections 2024-09-13T18:28:13.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.154+0000 7fbd0b483740 1 --2- 172.21.15.114:0/2362596963 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x562607f10f90 0x562607f133b0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:13.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.154+0000 7fbd0b483740 1 --2- 172.21.15.114:0/2362596963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562607f00fd0 0x562607efe040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:13.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.154+0000 7fbd0b483740 1 --2- 172.21.15.114:0/2362596963 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562607ef6730 0x562607efdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:13.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.154+0000 7fbd0b483740 1 --2- 172.21.15.114:0/2362596963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562607ef5e10 0x562607f008d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:13.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.154+0000 7fbd0b483740 1 -- 172.21.15.114:0/2362596963 >> 172.21.15.114:0/2362596963 conn(0x562607ef2380 msgr2=0x562607ef80e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:13.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.154+0000 7fbce9ffb640 0 client.26717 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:28:13.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.157+0000 7fbd0b483740 1 -- 172.21.15.114:0/2362596963 shutdown_connections 2024-09-13T18:28:13.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.158+0000 7fbd0b483740 1 -- 172.21.15.114:0/2362596963 wait complete. 2024-09-13T18:28:13.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.158+0000 7fbd0b483740 20 client.26717 trim_cache size 0 max 16384 2024-09-13T18:28:13.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.195+0000 7fbd0b483740 1 Processor -- start 2024-09-13T18:28:13.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.195+0000 7fbd0b483740 1 -- start start 2024-09-13T18:28:13.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.196+0000 7fbd0b483740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562607ef5e10 0x562607f003e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:13.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.196+0000 7fbd0b483740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562607ef6730 0x562607f07e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:13.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.196+0000 7fbd0b483740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562607f00fd0 0x562607f08380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:13.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.196+0000 7fbd06423640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562607ef5e10 0x562607f003e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:13.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.196+0000 7fbd06423640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562607ef5e10 0x562607f003e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:52902/0 (socket says 172.21.15.114:52902) 2024-09-13T18:28:13.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.196+0000 7fbd06423640 1 -- 172.21.15.114:0/4055670563 learned_addr learned my addr 172.21.15.114:0/4055670563 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:13.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.196+0000 7fbd0b483740 1 -- 172.21.15.114:0/4055670563 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x562607eed760 con 0x562607f00fd0 2024-09-13T18:28:13.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.196+0000 7fbd0b483740 1 -- 172.21.15.114:0/4055670563 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562607efc050 con 0x562607ef5e10 2024-09-13T18:28:13.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.196+0000 7fbd0b483740 1 -- 172.21.15.114:0/4055670563 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562607ef6520 con 0x562607ef6730 2024-09-13T18:28:13.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.196+0000 7fbd05c22640 1 --2- 172.21.15.114:0/4055670563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562607ef6730 0x562607f07e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:13.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.197+0000 7fbd06423640 1 -- 172.21.15.114:0/4055670563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562607ef6730 msgr2=0x562607f07e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:13.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.197+0000 7fbd06423640 1 --2- 172.21.15.114:0/4055670563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562607ef6730 0x562607f07e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:13.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.197+0000 7fbd06423640 1 -- 172.21.15.114:0/4055670563 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562607f00fd0 msgr2=0x562607f08380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:13.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.197+0000 7fbd06423640 1 --2- 172.21.15.114:0/4055670563 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562607f00fd0 0x562607f08380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:13.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.197+0000 7fbd06423640 1 -- 172.21.15.114:0/4055670563 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562607dfa790 con 0x562607ef5e10 2024-09-13T18:28:13.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.197+0000 7fbd06423640 1 --2- 172.21.15.114:0/4055670563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562607ef5e10 0x562607f003e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x562607efd370 tx=0x7fbcfc005890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5c04e9fd1d50fd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:13.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.197+0000 7fbcf57fa640 1 -- 172.21.15.114:0/4055670563 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc015040 con 0x562607ef5e10 2024-09-13T18:28:13.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.197+0000 7fbcf57fa640 1 -- 172.21.15.114:0/4055670563 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbcfc017e30 con 0x562607ef5e10 2024-09-13T18:28:13.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.198+0000 7fbd0b483740 1 -- 172.21.15.114:0/4055670563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562607ef5e10 msgr2=0x562607f003e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:13.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.198+0000 7fbd0b483740 1 --2- 172.21.15.114:0/4055670563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562607ef5e10 0x562607f003e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x562607efd370 tx=0x7fbcfc005890 comp rx=0 tx=0).stop 2024-09-13T18:28:13.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.198+0000 7fbd0b483740 1 -- 172.21.15.114:0/4055670563 shutdown_connections 2024-09-13T18:28:13.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.199+0000 7fbd0b483740 1 --2- 172.21.15.114:0/4055670563 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562607f00fd0 0x562607f08380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:13.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.199+0000 7fbd0b483740 1 --2- 172.21.15.114:0/4055670563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562607ef6730 0x562607f07e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:13.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.199+0000 7fbd0b483740 1 --2- 172.21.15.114:0/4055670563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562607ef5e10 0x562607f003e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:13.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.199+0000 7fbd0b483740 1 -- 172.21.15.114:0/4055670563 >> 172.21.15.114:0/4055670563 conn(0x562607ef2380 msgr2=0x562607ef2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:13.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.204+0000 7fbd0b483740 1 -- 172.21.15.114:0/4055670563 shutdown_connections 2024-09-13T18:28:13.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.206+0000 7fbd0b483740 1 -- 172.21.15.114:0/4055670563 wait complete. 2024-09-13T18:28:13.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.206+0000 7fbd0b483740 1 -- start start 2024-09-13T18:28:13.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.206+0000 7fbd0b483740 1 Processor -- start 2024-09-13T18:28:13.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.206+0000 7fbd0b483740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562607ef6730 0x562607f06070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:13.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.206+0000 7fbd0b483740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562607f00fd0 0x562607f065b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:13.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.206+0000 7fbd0b483740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562607f64480 0x562607f66870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:13.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.206+0000 7fbd0b483740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x562607f66db0 con 0x562607f64480 2024-09-13T18:28:13.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.206+0000 7fbd0b483740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562607f66ef0 con 0x562607f00fd0 2024-09-13T18:28:13.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.206+0000 7fbd0b483740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562607f67080 con 0x562607ef6730 2024-09-13T18:28:13.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.207+0000 7fbd0b483740 10 client.0 _ll_register_callbacks cb 0x7fbcf8007ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:28:13.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.207+0000 7fbd05c22640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562607f00fd0 0x562607f065b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:13.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.207+0000 7fbd05c22640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562607f00fd0 0x562607f065b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:52916/0 (socket says 172.21.15.114:52916) 2024-09-13T18:28:13.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.207+0000 7fbd05c22640 1 -- 172.21.15.114:0/966594244 learned_addr learned my addr 172.21.15.114:0/966594244 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:13.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.209+0000 7fbd05c22640 1 -- 172.21.15.114:0/966594244 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562607ef6730 msgr2=0x562607f06070 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:28:13.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.209+0000 7fbd05c22640 1 --2- 172.21.15.114:0/966594244 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562607ef6730 0x562607f06070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:13.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.209+0000 7fbd05c22640 1 -- 172.21.15.114:0/966594244 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562607f64480 msgr2=0x562607f66870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:13.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.209+0000 7fbd05c22640 1 --2- 172.21.15.114:0/966594244 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562607f64480 0x562607f66870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:13.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.209+0000 7fbd05c22640 1 -- 172.21.15.114:0/966594244 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcfc00c700 con 0x562607f00fd0 2024-09-13T18:28:13.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.210+0000 7fbd05c22640 1 --2- 172.21.15.114:0/966594244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562607f00fd0 0x562607f065b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbcf003b5e0 tx=0x7fbcf00ba760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52f9b469c381c05a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:13.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.210+0000 7fbce9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:28:13.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.211+0000 7fbce9ffb640 1 -- 172.21.15.114:0/966594244 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf00f5040 con 0x562607f00fd0 2024-09-13T18:28:13.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.211+0000 7fbd0b483740 1 -- 172.21.15.114:0/966594244 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562607bb1e30 con 0x562607f00fd0 2024-09-13T18:28:13.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.211+0000 7fbce9ffb640 1 -- 172.21.15.114:0/966594244 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fbcf00ba8d0 con 0x562607f00fd0 2024-09-13T18:28:13.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.211+0000 7fbce9ffb640 1 -- 172.21.15.114:0/966594244 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbcf00be700 con 0x562607f00fd0 2024-09-13T18:28:13.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.212+0000 7fbd0b483740 10 client.26723 fetch_fsmap learned FSMap version 34 2024-09-13T18:28:13.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.212+0000 7fbd0b483740 1 -- 172.21.15.114:0/966594244 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x562607e1fea0 con 0x562607f00fd0 2024-09-13T18:28:13.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.214+0000 7fbce9ffb640 1 -- 172.21.15.114:0/966594244 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562607bb1e30 con 0x562607f00fd0 2024-09-13T18:28:13.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.214+0000 7fbce9ffb640 1 -- 172.21.15.114:0/966594244 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbcf00b9dc0 con 0x562607f00fd0 2024-09-13T18:28:13.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.215+0000 7fbd0b483740 10 client.26723 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:28:13.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.215+0000 7fbd0b483740 10 client.26723 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:28:13.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.215+0000 7fbd0b483740 20 client.26723 populate_metadata read hostname 'smithi114' 2024-09-13T18:28:13.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.215+0000 7fbd0b483740 1 --2- 172.21.15.114:0/966594244 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x562607f8a700 0x562607f8cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:13.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.215+0000 7fbd0b483740 4 client.26723 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T18:28:13.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.215+0000 7fbd0b483740 1 -- 172.21.15.114:0/966594244 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562607f8d370 con 0x562607f8a700 2024-09-13T18:28:13.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.215+0000 7fbd06423640 1 --2- 172.21.15.114:0/966594244 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x562607f8a700 0x562607f8cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:13.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.216+0000 7fbd06423640 1 --2- 172.21.15.114:0/966594244 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x562607f8a700 0x562607f8cb20 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8f7165167daf2fa server_cookie=5eca87af09fb6f10 in_seq=0 out_seq=0 2024-09-13T18:28:13.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.216+0000 7fbce9ffb640 10 client.26723 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:28:13.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.216+0000 7fbce9ffb640 1 -- 172.21.15.114:0/966594244 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x562607f8d370 con 0x562607f8a700 2024-09-13T18:28:13.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.216+0000 7fbce9ffb640 10 client.26723 handle_command_reply: tid=0 2024-09-13T18:28:13.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.216+0000 7fbd0b483740 1 client.26723 shutdown 2024-09-13T18:28:13.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.220+0000 7fbd0b483740 1 -- 172.21.15.114:0/966594244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562607f00fd0 msgr2=0x562607f065b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:13.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.220+0000 7fbd0b483740 1 --2- 172.21.15.114:0/966594244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562607f00fd0 0x562607f065b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbcf003b5e0 tx=0x7fbcf00ba760 comp rx=0 tx=0).stop 2024-09-13T18:28:13.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.220+0000 7fbd0b483740 1 -- 172.21.15.114:0/966594244 shutdown_connections 2024-09-13T18:28:13.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.220+0000 7fbd0b483740 1 --2- 172.21.15.114:0/966594244 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x562607f8a700 0x562607f8cb20 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:13.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.220+0000 7fbd0b483740 1 --2- 172.21.15.114:0/966594244 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562607f64480 0x562607f66870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:13.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.220+0000 7fbd0b483740 1 --2- 172.21.15.114:0/966594244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562607f00fd0 0x562607f065b0 secure :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbcf003b5e0 tx=0x7fbcf00ba760 comp rx=0 tx=0).stop 2024-09-13T18:28:13.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.220+0000 7fbd0b483740 1 --2- 172.21.15.114:0/966594244 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562607ef6730 0x562607f06070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:13.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.220+0000 7fbd0b483740 1 -- 172.21.15.114:0/966594244 >> 172.21.15.114:0/966594244 conn(0x562607ef2380 msgr2=0x562607ef2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:13.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.220+0000 7fbce9ffb640 0 client.26723 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:28:13.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.224+0000 7fbd0b483740 1 -- 172.21.15.114:0/966594244 shutdown_connections 2024-09-13T18:28:13.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.229+0000 7fbd0b483740 1 -- 172.21.15.114:0/966594244 wait complete. 2024-09-13T18:28:13.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.229+0000 7fbd0b483740 20 client.26723 trim_cache size 0 max 16384 2024-09-13T18:28:13.231 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:28:13.232 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "no active scrubs running", 2024-09-13T18:28:13.232 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": {} 2024-09-13T18:28:13.232 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:28:13.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.230+0000 7fbd08687640 1 -- 172.21.15.114:0/3923578858 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbcec0a12e0 msgr2=0x7fbcec0a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:13.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.230+0000 7fbd08687640 1 --2- 172.21.15.114:0/3923578858 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbcec0a12e0 0x7fbcec0a37a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8010020 tx=0x7fbcf8030720 comp rx=0 tx=0).stop 2024-09-13T18:28:13.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.230+0000 7fbd08687640 1 -- 172.21.15.114:0/3923578858 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbd00107920 msgr2=0x7fbd001a54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:13.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.230+0000 7fbd08687640 1 --2- 172.21.15.114:0/3923578858 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbd00107920 0x7fbd001a54a0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fbcf002aa30 tx=0x7fbcf0003d50 comp rx=0 tx=0).stop 2024-09-13T18:28:13.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.239+0000 7fbd08687640 1 -- 172.21.15.114:0/3923578858 shutdown_connections 2024-09-13T18:28:13.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.239+0000 7fbd08687640 1 --2- 172.21.15.114:0/3923578858 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbcec0a12e0 0x7fbcec0a37a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:13.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.239+0000 7fbd08687640 1 --2- 172.21.15.114:0/3923578858 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd00119400 0x7fbd001a9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:13.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.239+0000 7fbd08687640 1 --2- 172.21.15.114:0/3923578858 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbd00107920 0x7fbd001a54a0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:13.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.239+0000 7fbd08687640 1 --2- 172.21.15.114:0/3923578858 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd00107540 0x7fbd001a4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:13.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.239+0000 7fbd08687640 1 -- 172.21.15.114:0/3923578858 >> 172.21.15.114:0/3923578858 conn(0x7fbd0006afe0 msgr2=0x7fbd00119980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:13.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.241+0000 7fbd08687640 1 -- 172.21.15.114:0/3923578858 shutdown_connections 2024-09-13T18:28:13.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.241+0000 7fbd08687640 1 -- 172.21.15.114:0/3923578858 wait complete. 2024-09-13T18:28:13.287 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-13T18:28:13.288 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:28:13.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.544+0000 7fcb664d0640 1 -- 172.21.15.114:0/1906587746 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb601074b0 msgr2=0x7fcb6010e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:13.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.544+0000 7fcb664d0640 1 --2- 172.21.15.114:0/1906587746 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb601074b0 0x7fcb6010e510 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcb58009be0 tx=0x7fcb5802a6c0 comp rx=0 tx=0).stop 2024-09-13T18:28:13.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.546+0000 7fcb664d0640 1 -- 172.21.15.114:0/1906587746 shutdown_connections 2024-09-13T18:28:13.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.546+0000 7fcb664d0640 1 --2- 172.21.15.114:0/1906587746 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb60119400 0x7fcb6011b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:13.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.546+0000 7fcb664d0640 1 --2- 172.21.15.114:0/1906587746 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb60107890 0x7fcb60107cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:13.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.546+0000 7fcb664d0640 1 --2- 172.21.15.114:0/1906587746 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb601074b0 0x7fcb6010e510 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:13.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.546+0000 7fcb664d0640 1 -- 172.21.15.114:0/1906587746 >> 172.21.15.114:0/1906587746 conn(0x7fcb6006aff0 msgr2=0x7fcb6006b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:13.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.548+0000 7fcb664d0640 1 -- 172.21.15.114:0/1906587746 shutdown_connections 2024-09-13T18:28:13.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.549+0000 7fcb664d0640 1 -- 172.21.15.114:0/1906587746 wait complete. 2024-09-13T18:28:13.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.549+0000 7fcb664d0640 1 Processor -- start 2024-09-13T18:28:13.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.549+0000 7fcb664d0640 1 -- start start 2024-09-13T18:28:13.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.550+0000 7fcb664d0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb601074b0 0x7fcb601a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:13.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.550+0000 7fcb664d0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb60107890 0x7fcb601a56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:13.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.550+0000 7fcb664d0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb60119400 0x7fcb601a9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:13.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.550+0000 7fcb664d0640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb601aa0a0 con 0x7fcb601074b0 2024-09-13T18:28:13.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.550+0000 7fcb664d0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb601aa1e0 con 0x7fcb60107890 2024-09-13T18:28:13.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.550+0000 7fcb664d0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb601aa3a0 con 0x7fcb60119400 2024-09-13T18:28:13.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.550+0000 7fcb64a6d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb60119400 0x7fcb601a9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:13.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.550+0000 7fcb64a6d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb60119400 0x7fcb601a9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:38722/0 (socket says 172.21.15.114:38722) 2024-09-13T18:28:13.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.550+0000 7fcb64a6d640 1 -- 172.21.15.114:0/663538685 learned_addr learned my addr 172.21.15.114:0/663538685 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:13.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.550+0000 7fcb5f7fe640 1 --2- 172.21.15.114:0/663538685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb60107890 0x7fcb601a56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:13.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.550+0000 7fcb5ffff640 1 --2- 172.21.15.114:0/663538685 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb601074b0 0x7fcb601a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:13.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.551+0000 7fcb5ffff640 1 -- 172.21.15.114:0/663538685 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb60119400 msgr2=0x7fcb601a9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:13.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.551+0000 7fcb5ffff640 1 --2- 172.21.15.114:0/663538685 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb60119400 0x7fcb601a9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:13.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.552+0000 7fcb5ffff640 1 -- 172.21.15.114:0/663538685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb60107890 msgr2=0x7fcb601a56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:13.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.552+0000 7fcb5ffff640 1 --2- 172.21.15.114:0/663538685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb60107890 0x7fcb601a56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:13.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.552+0000 7fcb5ffff640 1 -- 172.21.15.114:0/663538685 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb58009840 con 0x7fcb601074b0 2024-09-13T18:28:13.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.552+0000 7fcb5ffff640 1 --2- 172.21.15.114:0/663538685 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb601074b0 0x7fcb601a5170 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fcb5802acb0 tx=0x7fcb58003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:13.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.555+0000 7fcb5d7fa640 1 -- 172.21.15.114:0/663538685 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb58003ea0 con 0x7fcb601074b0 2024-09-13T18:28:13.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.555+0000 7fcb5d7fa640 1 -- 172.21.15.114:0/663538685 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcb580044d0 con 0x7fcb601074b0 2024-09-13T18:28:13.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.555+0000 7fcb5d7fa640 1 -- 172.21.15.114:0/663538685 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb58003ea0 con 0x7fcb601074b0 2024-09-13T18:28:13.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.555+0000 7fcb664d0640 1 -- 172.21.15.114:0/663538685 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb601aa620 con 0x7fcb601074b0 2024-09-13T18:28:13.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.556+0000 7fcb664d0640 1 -- 172.21.15.114:0/663538685 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb6010ebe0 con 0x7fcb601074b0 2024-09-13T18:28:13.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.557+0000 7fcb664d0640 1 -- 172.21.15.114:0/663538685 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb6006bdc0 con 0x7fcb601074b0 2024-09-13T18:28:13.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.564+0000 7fcb5d7fa640 1 -- 172.21.15.114:0/663538685 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fcb58004640 con 0x7fcb601074b0 2024-09-13T18:28:13.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.564+0000 7fcb5d7fa640 1 --2- 172.21.15.114:0/663538685 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fcb480a1390 0x7fcb480a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:13.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.565+0000 7fcb5d7fa640 1 -- 172.21.15.114:0/663538685 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fcb580edd80 con 0x7fcb601074b0 2024-09-13T18:28:13.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.565+0000 7fcb5f7fe640 1 --2- 172.21.15.114:0/663538685 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fcb480a1390 0x7fcb480a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:13.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.566+0000 7fcb5f7fe640 1 --2- 172.21.15.114:0/663538685 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fcb480a1390 0x7fcb480a3850 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcb601a6710 tx=0x7fcb50009260 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:13.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:13.573+0000 7fcb5d7fa640 1 -- 172.21.15.114:0/663538685 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fcb580b6e20 con 0x7fcb601074b0 2024-09-13T18:28:13.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:13 smithi134 ceph-mon[34283]: pgmap v175: 129 pgs: 129 active+clean; 18 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 33 MiB/s wr, 641 op/s 2024-09-13T18:28:13.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:13 smithi204 ceph-mon[34777]: pgmap v175: 129 pgs: 129 active+clean; 18 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 33 MiB/s wr, 641 op/s 2024-09-13T18:28:13.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:13 smithi114 ceph-mon[30897]: pgmap v175: 129 pgs: 129 active+clean; 18 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 33 MiB/s wr, 641 op/s 2024-09-13T18:28:14.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.052+0000 7fcb664d0640 1 -- 172.21.15.114:0/663538685 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb601a64c0 con 0x7fcb601074b0 2024-09-13T18:28:14.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.060+0000 7fcb5d7fa640 1 -- 172.21.15.114:0/663538685 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+10930 (secure 0 0 0) 0x7fcb58040090 con 0x7fcb601074b0 2024-09-13T18:28:14.063 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:28:14.063 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4162817497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4162817497},{"type":"v1","addr":"172.21.15.114:6835","nonce":4162817497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/2435428478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":2435428478},{"type":"v1","addr":"172.21.15.114:6837","nonce":2435428478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T18:20:01.895117+0000","modified":"2024-09-13T18:27:22.172031+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24482,"mds_2":24475,"mds_3":24458,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/2046516451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":2046516451},{"type":"v1","addr":"172.21.15.114:6839","nonce":2046516451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":10,"addr":"172.21.15.114:6841/1956923446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":1956923446},{"type":"v1","addr":"172.21.15.114:6841","nonce":1956923446}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24391":{"gid":24391,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/507301496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":507301496},{"type":"v1","addr":"172.21.15.204:6833","nonce":507301496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4028657219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4028657219},{"type":"v1","addr":"172.21.15.134:6833","nonce":4028657219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/57029119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":57029119},{"type":"v1","addr":"172.21.15.204:6835","nonce":57029119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3068880939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3068880939},{"type":"v1","addr":"172.21.15.204:6837","nonce":3068880939}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"e","rank":3,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.134:6835/1150003275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1150003275},{"type":"v1","addr":"172.21.15.134:6835","nonce":1150003275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/344883912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":344883912},{"type":"v1","addr":"172.21.15.204:6839","nonce":344883912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6837/891692210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":891692210},{"type":"v1","addr":"172.21.15.134:6837","nonce":891692210}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6839/3466309724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3466309724},{"type":"v1","addr":"172.21.15.134:6839","nonce":3466309724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:28:14.063 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 34 2024-09-13T18:28:14.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.065+0000 7fcb664d0640 1 -- 172.21.15.114:0/663538685 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fcb480a1390 msgr2=0x7fcb480a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:14.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.065+0000 7fcb664d0640 1 --2- 172.21.15.114:0/663538685 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fcb480a1390 0x7fcb480a3850 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcb601a6710 tx=0x7fcb50009260 comp rx=0 tx=0).stop 2024-09-13T18:28:14.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.065+0000 7fcb664d0640 1 -- 172.21.15.114:0/663538685 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb601074b0 msgr2=0x7fcb601a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:14.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.065+0000 7fcb664d0640 1 --2- 172.21.15.114:0/663538685 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb601074b0 0x7fcb601a5170 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fcb5802acb0 tx=0x7fcb58003ba0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.066+0000 7fcb664d0640 1 -- 172.21.15.114:0/663538685 shutdown_connections 2024-09-13T18:28:14.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.066+0000 7fcb664d0640 1 --2- 172.21.15.114:0/663538685 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fcb480a1390 0x7fcb480a3850 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.067+0000 7fcb664d0640 1 --2- 172.21.15.114:0/663538685 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb60119400 0x7fcb601a9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.067+0000 7fcb664d0640 1 --2- 172.21.15.114:0/663538685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb60107890 0x7fcb601a56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.067+0000 7fcb664d0640 1 --2- 172.21.15.114:0/663538685 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb601074b0 0x7fcb601a5170 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.067+0000 7fcb664d0640 1 -- 172.21.15.114:0/663538685 >> 172.21.15.114:0/663538685 conn(0x7fcb6006aff0 msgr2=0x7fcb60119fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:14.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.067+0000 7fcb664d0640 1 -- 172.21.15.114:0/663538685 shutdown_connections 2024-09-13T18:28:14.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.067+0000 7fcb664d0640 1 -- 172.21.15.114:0/663538685 wait complete. 2024-09-13T18:28:14.092 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-09-13T18:28:14.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.316+0000 7f2106a87640 1 -- 172.21.15.114:0/1935144323 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2100107540 msgr2=0x7f210010e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:14.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.316+0000 7f2106a87640 1 --2- 172.21.15.114:0/1935144323 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2100107540 0x7f210010e510 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f20f801acb0 tx=0x7f20f803b920 comp rx=0 tx=0).stop 2024-09-13T18:28:14.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.319+0000 7f2106a87640 1 -- 172.21.15.114:0/1935144323 shutdown_connections 2024-09-13T18:28:14.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.319+0000 7f2106a87640 1 --2- 172.21.15.114:0/1935144323 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2100119400 0x7f210011b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.319+0000 7f2106a87640 1 --2- 172.21.15.114:0/1935144323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2100107920 0x7f2100107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.319+0000 7f2106a87640 1 --2- 172.21.15.114:0/1935144323 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2100107540 0x7f210010e510 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.319+0000 7f2106a87640 1 -- 172.21.15.114:0/1935144323 >> 172.21.15.114:0/1935144323 conn(0x7f210006afe0 msgr2=0x7f210006b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:14.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.324+0000 7f2106a87640 1 -- 172.21.15.114:0/1935144323 shutdown_connections 2024-09-13T18:28:14.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.326+0000 7f2106a87640 1 -- 172.21.15.114:0/1935144323 wait complete. 2024-09-13T18:28:14.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.326+0000 7f2106a87640 1 Processor -- start 2024-09-13T18:28:14.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.326+0000 7f2106a87640 1 -- start start 2024-09-13T18:28:14.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.330+0000 7f2106a87640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2100107540 0x7f2100114b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:14.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.330+0000 7f2106a87640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2100107920 0x7f21001150c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:14.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.330+0000 7f2106a87640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2100119400 0x7f21001a1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:14.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.330+0000 7f2106a87640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21001a1840 con 0x7f2100119400 2024-09-13T18:28:14.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.330+0000 7f2106a87640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21001a19b0 con 0x7f2100107920 2024-09-13T18:28:14.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.330+0000 7f2106a87640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21001a1b70 con 0x7f2100107540 2024-09-13T18:28:14.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.330+0000 7f2105024640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2100119400 0x7f21001a1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:14.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.330+0000 7f2105024640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2100119400 0x7f21001a1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55406/0 (socket says 172.21.15.114:55406) 2024-09-13T18:28:14.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.330+0000 7f2105024640 1 -- 172.21.15.114:0/2401426228 learned_addr learned my addr 172.21.15.114:0/2401426228 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:14.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.330+0000 7f2104823640 1 --2- 172.21.15.114:0/2401426228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2100107540 0x7f2100114b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:14.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.332+0000 7f20fffff640 1 --2- 172.21.15.114:0/2401426228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2100107920 0x7f21001150c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:14.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.333+0000 7f20fffff640 1 -- 172.21.15.114:0/2401426228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2100107540 msgr2=0x7f2100114b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:14.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.333+0000 7f20fffff640 1 --2- 172.21.15.114:0/2401426228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2100107540 0x7f2100114b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.333+0000 7f20fffff640 1 -- 172.21.15.114:0/2401426228 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2100119400 msgr2=0x7f21001a1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:14.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.333+0000 7f20fffff640 1 --2- 172.21.15.114:0/2401426228 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2100119400 0x7f21001a1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.333+0000 7f20fffff640 1 -- 172.21.15.114:0/2401426228 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f801a910 con 0x7f2100107920 2024-09-13T18:28:14.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.333+0000 7f2105024640 1 --2- 172.21.15.114:0/2401426228 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2100119400 0x7f21001a1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:28:14.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.333+0000 7f2104823640 1 --2- 172.21.15.114:0/2401426228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2100107540 0x7f2100114b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:28:14.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.333+0000 7f20fffff640 1 --2- 172.21.15.114:0/2401426228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2100107920 0x7f21001150c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f20f000d6d0 tx=0x7f20f000db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:14.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.334+0000 7f20fdffb640 1 -- 172.21.15.114:0/2401426228 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f0005220 con 0x7f2100107920 2024-09-13T18:28:14.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.334+0000 7f20fdffb640 1 -- 172.21.15.114:0/2401426228 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f20f0009cb0 con 0x7f2100107920 2024-09-13T18:28:14.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.334+0000 7f20fdffb640 1 -- 172.21.15.114:0/2401426228 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f0004c60 con 0x7f2100107920 2024-09-13T18:28:14.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.334+0000 7f2106a87640 1 -- 172.21.15.114:0/2401426228 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21001a1e50 con 0x7f2100107920 2024-09-13T18:28:14.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.334+0000 7f2106a87640 1 -- 172.21.15.114:0/2401426228 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21001a9600 con 0x7f2100107920 2024-09-13T18:28:14.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.338+0000 7f2109884740 1 Processor -- start 2024-09-13T18:28:14.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.338+0000 7f2109884740 1 -- start start 2024-09-13T18:28:14.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.339+0000 7f2109884740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aef0705790 0x55aef0705b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:14.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.339+0000 7f2109884740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aef07060b0 0x55aef0710410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:14.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.339+0000 7f2109884740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef0710950 0x55aef0712d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:14.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.339+0000 7f2109884740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef066fcc0 con 0x55aef0705790 2024-09-13T18:28:14.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.339+0000 7f2109884740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef066d450 con 0x55aef07060b0 2024-09-13T18:28:14.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.339+0000 7f2109884740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef0668d40 con 0x55aef0710950 2024-09-13T18:28:14.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.339+0000 7f20fdffb640 1 -- 172.21.15.114:0/2401426228 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f20f0010460 con 0x7f2100107920 2024-09-13T18:28:14.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.340+0000 7f20fdffb640 1 --2- 172.21.15.114:0/2401426228 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f20e80a1350 0x7f20e80a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:14.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.340+0000 7f20fdffb640 1 -- 172.21.15.114:0/2401426228 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f20f0009e20 con 0x7f2100107920 2024-09-13T18:28:14.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.341+0000 7f2105024640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aef0705790 0x55aef0705b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:14.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.341+0000 7f20fffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef0710950 0x55aef0712d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:14.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.341+0000 7f20fffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef0710950 0x55aef0712d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:38758/0 (socket says 172.21.15.114:38758) 2024-09-13T18:28:14.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.341+0000 7f20fffff640 1 -- 172.21.15.114:0/2034729957 learned_addr learned my addr 172.21.15.114:0/2034729957 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:14.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.342+0000 7f2104823640 1 --2- 172.21.15.114:0/2034729957 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aef07060b0 0x55aef0710410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:14.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.342+0000 7f2104823640 1 -- 172.21.15.114:0/2034729957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef0710950 msgr2=0x55aef0712d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:14.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.342+0000 7f2104823640 1 --2- 172.21.15.114:0/2034729957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef0710950 0x55aef0712d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.342+0000 7f2104823640 1 -- 172.21.15.114:0/2034729957 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aef0705790 msgr2=0x55aef0705b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:14.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.342+0000 7f2104823640 1 --2- 172.21.15.114:0/2034729957 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aef0705790 0x55aef0705b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.342+0000 7f2104823640 1 -- 172.21.15.114:0/2034729957 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aef0630620 con 0x55aef07060b0 2024-09-13T18:28:14.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.343+0000 7f2104823640 1 --2- 172.21.15.114:0/2034729957 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aef07060b0 0x55aef0710410 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f20f8006010 tx=0x7f20f80407f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9c5bc272275d56 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:14.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.343+0000 7f20e6ffd640 1 -- 172.21.15.114:0/2034729957 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f8051040 con 0x55aef07060b0 2024-09-13T18:28:14.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.343+0000 7f2105024640 1 --2- 172.21.15.114:0/2401426228 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f20e80a1350 0x7f20e80a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:14.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.344+0000 7f20e6ffd640 1 -- 172.21.15.114:0/2034729957 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f20f8003e00 con 0x55aef07060b0 2024-09-13T18:28:14.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.344+0000 7f20e6ffd640 1 -- 172.21.15.114:0/2034729957 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f8041620 con 0x55aef07060b0 2024-09-13T18:28:14.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.345+0000 7f2109884740 1 -- 172.21.15.114:0/2034729957 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aef07060b0 msgr2=0x55aef0710410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:14.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.345+0000 7f2109884740 1 --2- 172.21.15.114:0/2034729957 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aef07060b0 0x55aef0710410 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f20f8006010 tx=0x7f20f80407f0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.345+0000 7f2105024640 1 --2- 172.21.15.114:0/2401426228 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f20e80a1350 0x7f20e80a3810 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f20f4009ec0 tx=0x7f20f4008040 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:14.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.345+0000 7f2109884740 1 -- 172.21.15.114:0/2034729957 shutdown_connections 2024-09-13T18:28:14.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.345+0000 7f2109884740 1 --2- 172.21.15.114:0/2034729957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef0710950 0x55aef0712d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.345+0000 7f2109884740 1 --2- 172.21.15.114:0/2034729957 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aef07060b0 0x55aef0710410 secure :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f20f8006010 tx=0x7f20f80407f0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.345+0000 7f2109884740 1 --2- 172.21.15.114:0/2034729957 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aef0705790 0x55aef0705b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.345+0000 7f2109884740 1 -- 172.21.15.114:0/2034729957 >> 172.21.15.114:0/2034729957 conn(0x55aef0701d00 msgr2=0x55aef07020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:14.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.348+0000 7f2109884740 1 -- 172.21.15.114:0/2034729957 shutdown_connections 2024-09-13T18:28:14.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.349+0000 7f2109884740 1 -- 172.21.15.114:0/2034729957 wait complete. 2024-09-13T18:28:14.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.350+0000 7f2109884740 1 -- start start 2024-09-13T18:28:14.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.350+0000 7f2109884740 1 Processor -- start 2024-09-13T18:28:14.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.351+0000 7f2109884740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aef0705790 0x55aef07100c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:14.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.351+0000 7f2109884740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef0710950 0x55aef070d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:14.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.351+0000 7f2109884740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aef070d880 0x55aef070dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:14.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.351+0000 7f2109884740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef0627180 con 0x55aef070d880 2024-09-13T18:28:14.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.351+0000 7f2109884740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef0341430 con 0x55aef0705790 2024-09-13T18:28:14.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.351+0000 7f2109884740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef05ec5e0 con 0x55aef0710950 2024-09-13T18:28:14.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.351+0000 7f2109884740 10 client.0 _ll_register_callbacks cb 0x55aef0658aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:28:14.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.351+0000 7f2104823640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aef0705790 0x55aef07100c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:14.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.351+0000 7f2104823640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aef0705790 0x55aef07100c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:36098/0 (socket says 172.21.15.114:36098) 2024-09-13T18:28:14.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.351+0000 7f2104823640 1 -- 172.21.15.114:0/2636380775 learned_addr learned my addr 172.21.15.114:0/2636380775 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:14.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.352+0000 7f2104823640 1 -- 172.21.15.114:0/2636380775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef0710950 msgr2=0x55aef070d340 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:28:14.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.352+0000 7f2104823640 1 --2- 172.21.15.114:0/2636380775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef0710950 0x55aef070d340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.352+0000 7f2104823640 1 -- 172.21.15.114:0/2636380775 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aef070d880 msgr2=0x55aef070dd00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:28:14.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.352+0000 7f2104823640 1 --2- 172.21.15.114:0/2636380775 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aef070d880 0x55aef070dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.352+0000 7f2104823640 1 -- 172.21.15.114:0/2636380775 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f210011c2e0 con 0x55aef0705790 2024-09-13T18:28:14.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.352+0000 7f2104823640 1 --2- 172.21.15.114:0/2636380775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aef0705790 0x55aef07100c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f20f8005e30 tx=0x7f20f8040f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45d8ad00112a503e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:14.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.353+0000 7f20dffff640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:28:14.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.353+0000 7f20dffff640 1 -- 172.21.15.114:0/2636380775 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f80416f0 con 0x55aef0705790 2024-09-13T18:28:14.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.353+0000 7f2109884740 1 -- 172.21.15.114:0/2636380775 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aef0692b80 con 0x55aef0705790 2024-09-13T18:28:14.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.353+0000 7f20dffff640 1 -- 172.21.15.114:0/2636380775 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f20f8003b80 con 0x55aef0705790 2024-09-13T18:28:14.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.353+0000 7f20dffff640 1 -- 172.21.15.114:0/2636380775 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20f8041a80 con 0x55aef0705790 2024-09-13T18:28:14.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.356+0000 7f2109884740 10 client.26741 fetch_fsmap learned FSMap version 34 2024-09-13T18:28:14.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.356+0000 7f2109884740 1 -- 172.21.15.114:0/2636380775 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55aef06673c0 con 0x55aef0705790 2024-09-13T18:28:14.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.361+0000 7f20dffff640 1 -- 172.21.15.114:0/2636380775 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55aef0692b80 con 0x55aef0705790 2024-09-13T18:28:14.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.361+0000 7f20dffff640 1 -- 172.21.15.114:0/2636380775 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f20f8004720 con 0x55aef0705790 2024-09-13T18:28:14.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.361+0000 7f2109884740 10 client.26741 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:28:14.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.361+0000 7f2109884740 10 client.26741 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-13T18:28:14.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.361+0000 7f2109884740 20 client.26741 populate_metadata read hostname 'smithi114' 2024-09-13T18:28:14.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.361+0000 7f2109884740 1 --2- 172.21.15.114:0/2636380775 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55aef0720540 0x55aef0722960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:14.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.361+0000 7f2109884740 4 client.26741 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:28:14.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.361+0000 7f2109884740 1 -- 172.21.15.114:0/2636380775 --> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aef065b1f0 con 0x55aef0720540 2024-09-13T18:28:14.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.364+0000 7f20fffff640 1 --2- 172.21.15.114:0/2636380775 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55aef0720540 0x55aef0722960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:14.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.366+0000 7f20fffff640 1 --2- 172.21.15.114:0/2636380775 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55aef0720540 0x55aef0722960 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=458a02d039568e0d server_cookie=c3e384b10cae4796 in_seq=0 out_seq=0 2024-09-13T18:28:14.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.366+0000 7f20dffff640 10 client.26741 ms_handle_connect on v2:172.21.15.114:6840/1956923446 2024-09-13T18:28:14.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.367+0000 7f20dffff640 1 -- 172.21.15.114:0/2636380775 <== mds.4 v2:172.21.15.114:6840/1956923446 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55aef065b1f0 con 0x55aef0720540 2024-09-13T18:28:14.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.368+0000 7f20dffff640 10 client.26741 handle_command_reply: tid=0 2024-09-13T18:28:14.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.368+0000 7f2109884740 1 client.26741 shutdown 2024-09-13T18:28:14.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.377+0000 7f2109884740 1 -- 172.21.15.114:0/2636380775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aef0705790 msgr2=0x55aef07100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:14.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.377+0000 7f2109884740 1 --2- 172.21.15.114:0/2636380775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aef0705790 0x55aef07100c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f20f8005e30 tx=0x7f20f8040f80 comp rx=0 tx=0).stop 2024-09-13T18:28:14.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.381+0000 7f2109884740 1 -- 172.21.15.114:0/2636380775 shutdown_connections 2024-09-13T18:28:14.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.381+0000 7f2109884740 1 --2- 172.21.15.114:0/2636380775 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55aef0720540 0x55aef0722960 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.381+0000 7f2109884740 1 --2- 172.21.15.114:0/2636380775 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aef070d880 0x55aef070dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.381+0000 7f2109884740 1 --2- 172.21.15.114:0/2636380775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef0710950 0x55aef070d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.381+0000 7f2109884740 1 --2- 172.21.15.114:0/2636380775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aef0705790 0x55aef07100c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.381+0000 7f2109884740 1 -- 172.21.15.114:0/2636380775 >> 172.21.15.114:0/2636380775 conn(0x55aef0701d00 msgr2=0x55aef0707a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:14.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.381+0000 7f20dffff640 0 client.26741 ms_handle_reset on v2:172.21.15.114:6840/1956923446 2024-09-13T18:28:14.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.382+0000 7f2109884740 1 -- 172.21.15.114:0/2636380775 shutdown_connections 2024-09-13T18:28:14.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.390+0000 7f2109884740 1 -- 172.21.15.114:0/2636380775 wait complete. 2024-09-13T18:28:14.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.392+0000 7f2109884740 20 client.26741 trim_cache size 0 max 16384 2024-09-13T18:28:14.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.432+0000 7f2109884740 1 Processor -- start 2024-09-13T18:28:14.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.432+0000 7f2109884740 1 -- start start 2024-09-13T18:28:14.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.433+0000 7f2109884740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef0705790 0x55aef0712c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:14.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.433+0000 7f2109884740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aef070d880 0x55aef07016a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:14.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.433+0000 7f2109884740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aef0710950 0x55aef0711d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:14.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.433+0000 7f2109884740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef0702e90 con 0x55aef0710950 2024-09-13T18:28:14.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.433+0000 7f2109884740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef07066d0 con 0x55aef070d880 2024-09-13T18:28:14.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.433+0000 7f2109884740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef0705510 con 0x55aef0705790 2024-09-13T18:28:14.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.433+0000 7f20fffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aef070d880 0x55aef07016a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:14.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.434+0000 7f20fffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aef070d880 0x55aef07016a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:36108/0 (socket says 172.21.15.114:36108) 2024-09-13T18:28:14.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.434+0000 7f20fffff640 1 -- 172.21.15.114:0/739146456 learned_addr learned my addr 172.21.15.114:0/739146456 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:14.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.434+0000 7f2104823640 1 --2- 172.21.15.114:0/739146456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef0705790 0x55aef0712c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:14.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.435+0000 7f2105024640 1 --2- 172.21.15.114:0/739146456 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aef0710950 0x55aef0711d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:14.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.436+0000 7f20fffff640 1 -- 172.21.15.114:0/739146456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef0705790 msgr2=0x55aef0712c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:14.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.436+0000 7f20fffff640 1 --2- 172.21.15.114:0/739146456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef0705790 0x55aef0712c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.436+0000 7f20fffff640 1 -- 172.21.15.114:0/739146456 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aef0710950 msgr2=0x55aef0711d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:14.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.436+0000 7f20fffff640 1 --2- 172.21.15.114:0/739146456 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aef0710950 0x55aef0711d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.436+0000 7f20fffff640 1 -- 172.21.15.114:0/739146456 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aef0609770 con 0x55aef070d880 2024-09-13T18:28:14.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.438+0000 7f20fffff640 1 --2- 172.21.15.114:0/739146456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aef070d880 0x55aef07016a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f20f00048b0 tx=0x7f20f009d660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f25ef1faa548eca8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:14.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.447+0000 7f20e77fe640 1 -- 172.21.15.114:0/739146456 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f00994c0 con 0x55aef070d880 2024-09-13T18:28:14.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.447+0000 7f20e77fe640 1 -- 172.21.15.114:0/739146456 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f20f00a1ac0 con 0x55aef070d880 2024-09-13T18:28:14.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.447+0000 7f20e77fe640 1 -- 172.21.15.114:0/739146456 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f00992b0 con 0x55aef070d880 2024-09-13T18:28:14.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.450+0000 7f2109884740 1 -- 172.21.15.114:0/739146456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aef070d880 msgr2=0x55aef07016a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:14.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.450+0000 7f2109884740 1 --2- 172.21.15.114:0/739146456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aef070d880 0x55aef07016a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f20f00048b0 tx=0x7f20f009d660 comp rx=0 tx=0).stop 2024-09-13T18:28:14.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.451+0000 7f2109884740 1 -- 172.21.15.114:0/739146456 shutdown_connections 2024-09-13T18:28:14.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.451+0000 7f2109884740 1 --2- 172.21.15.114:0/739146456 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aef0710950 0x55aef0711d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.451+0000 7f2109884740 1 --2- 172.21.15.114:0/739146456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aef070d880 0x55aef07016a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.451+0000 7f2109884740 1 --2- 172.21.15.114:0/739146456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef0705790 0x55aef0712c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.451+0000 7f2109884740 1 -- 172.21.15.114:0/739146456 >> 172.21.15.114:0/739146456 conn(0x55aef0701d00 msgr2=0x55aef07020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:14.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.453+0000 7f2109884740 1 -- 172.21.15.114:0/739146456 shutdown_connections 2024-09-13T18:28:14.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.457+0000 7f2109884740 1 -- 172.21.15.114:0/739146456 wait complete. 2024-09-13T18:28:14.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.457+0000 7f2109884740 1 -- start start 2024-09-13T18:28:14.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.457+0000 7f2109884740 1 Processor -- start 2024-09-13T18:28:14.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.458+0000 7f2109884740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef0705790 0x55aef07137e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:14.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.458+0000 7f2109884740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aef070d880 0x55aef0713d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:14.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.458+0000 7f2109884740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aef0710950 0x55aef0775a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:14.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.458+0000 7f2109884740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef0714490 con 0x55aef0710950 2024-09-13T18:28:14.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.458+0000 7f2109884740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef0775fb0 con 0x55aef070d880 2024-09-13T18:28:14.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.458+0000 7f2109884740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef0776170 con 0x55aef0705790 2024-09-13T18:28:14.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.458+0000 7f2109884740 10 client.0 _ll_register_callbacks cb 0x7f20f801a7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:28:14.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.458+0000 7f20fffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aef070d880 0x55aef0713d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:14.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.458+0000 7f20fffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aef070d880 0x55aef0713d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:36122/0 (socket says 172.21.15.114:36122) 2024-09-13T18:28:14.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.458+0000 7f20fffff640 1 -- 172.21.15.114:0/1875501106 learned_addr learned my addr 172.21.15.114:0/1875501106 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:14.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.460+0000 7f20fffff640 1 -- 172.21.15.114:0/1875501106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef0705790 msgr2=0x55aef07137e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:28:14.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.460+0000 7f20fffff640 1 --2- 172.21.15.114:0/1875501106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef0705790 0x55aef07137e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.461+0000 7f20fffff640 1 -- 172.21.15.114:0/1875501106 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aef0710950 msgr2=0x55aef0775a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:14.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.461+0000 7f20fffff640 1 --2- 172.21.15.114:0/1875501106 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aef0710950 0x55aef0775a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.461+0000 7f20fffff640 1 -- 172.21.15.114:0/1875501106 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21001a9600 con 0x55aef070d880 2024-09-13T18:28:14.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.464+0000 7f20fffff640 1 --2- 172.21.15.114:0/1875501106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aef070d880 0x55aef0713d20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f20f00966a0 tx=0x7f20f0099530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1552cfa78a331471 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:14.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.464+0000 7f20dffff640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:28:14.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.464+0000 7f20dffff640 1 -- 172.21.15.114:0/1875501106 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f0096000 con 0x55aef070d880 2024-09-13T18:28:14.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.464+0000 7f20dffff640 1 -- 172.21.15.114:0/1875501106 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f20f0009e20 con 0x55aef070d880 2024-09-13T18:28:14.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.465+0000 7f2109884740 1 -- 172.21.15.114:0/1875501106 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f20f8003e00 con 0x55aef070d880 2024-09-13T18:28:14.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.469+0000 7f20dffff640 1 -- 172.21.15.114:0/1875501106 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20f0099050 con 0x55aef070d880 2024-09-13T18:28:14.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.470+0000 7f2109884740 10 client.26747 fetch_fsmap learned FSMap version 34 2024-09-13T18:28:14.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.470+0000 7f2109884740 1 -- 172.21.15.114:0/1875501106 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55aef062ee80 con 0x55aef070d880 2024-09-13T18:28:14.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.470+0000 7f20dffff640 1 -- 172.21.15.114:0/1875501106 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f20f8003e00 con 0x55aef070d880 2024-09-13T18:28:14.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.470+0000 7f20dffff640 1 -- 172.21.15.114:0/1875501106 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f20f0005380 con 0x55aef070d880 2024-09-13T18:28:14.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.470+0000 7f2109884740 10 client.26747 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:28:14.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.470+0000 7f2109884740 10 client.26747 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-13T18:28:14.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.470+0000 7f2109884740 20 client.26747 populate_metadata read hostname 'smithi114' 2024-09-13T18:28:14.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.470+0000 7f2109884740 1 --2- 172.21.15.114:0/1875501106 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55aef0777400 0x55aef0779820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:14.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.470+0000 7f2109884740 4 client.26747 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:28:14.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.470+0000 7f2109884740 1 -- 172.21.15.114:0/1875501106 --> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55aef035c4e0 con 0x55aef0777400 2024-09-13T18:28:14.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.471+0000 7f2104823640 1 --2- 172.21.15.114:0/1875501106 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55aef0777400 0x55aef0779820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:14.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.472+0000 7f2104823640 1 --2- 172.21.15.114:0/1875501106 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55aef0777400 0x55aef0779820 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a9f200ed767088aa server_cookie=9b2381988396ea42 in_seq=0 out_seq=0 2024-09-13T18:28:14.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.472+0000 7f20dffff640 10 client.26747 ms_handle_connect on v2:172.21.15.114:6840/1956923446 2024-09-13T18:28:14.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.472+0000 7f20dffff640 1 -- 172.21.15.114:0/1875501106 <== mds.4 v2:172.21.15.114:6840/1956923446 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55aef035c4e0 con 0x55aef0777400 2024-09-13T18:28:14.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.472+0000 7f20dffff640 10 client.26747 handle_command_reply: tid=0 2024-09-13T18:28:14.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.475+0000 7f2109884740 1 client.26747 shutdown 2024-09-13T18:28:14.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.479+0000 7f2109884740 1 -- 172.21.15.114:0/1875501106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aef070d880 msgr2=0x55aef0713d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:14.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.479+0000 7f2109884740 1 --2- 172.21.15.114:0/1875501106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aef070d880 0x55aef0713d20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f20f00966a0 tx=0x7f20f0099530 comp rx=0 tx=0).stop 2024-09-13T18:28:14.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.479+0000 7f2109884740 1 -- 172.21.15.114:0/1875501106 shutdown_connections 2024-09-13T18:28:14.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.479+0000 7f2109884740 1 --2- 172.21.15.114:0/1875501106 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55aef0777400 0x55aef0779820 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.479+0000 7f2109884740 1 --2- 172.21.15.114:0/1875501106 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aef0710950 0x55aef0775a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.479+0000 7f2109884740 1 --2- 172.21.15.114:0/1875501106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aef070d880 0x55aef0713d20 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.479+0000 7f2109884740 1 --2- 172.21.15.114:0/1875501106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef0705790 0x55aef07137e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.479+0000 7f20dffff640 0 client.26747 ms_handle_reset on v2:172.21.15.114:6840/1956923446 2024-09-13T18:28:14.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.479+0000 7f2109884740 1 -- 172.21.15.114:0/1875501106 >> 172.21.15.114:0/1875501106 conn(0x55aef0701d00 msgr2=0x55aef070bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:14.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.480+0000 7f2109884740 1 -- 172.21.15.114:0/1875501106 shutdown_connections 2024-09-13T18:28:14.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.480+0000 7f2109884740 1 -- 172.21.15.114:0/1875501106 wait complete. 2024-09-13T18:28:14.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.483+0000 7f2109884740 20 client.26747 trim_cache size 0 max 16384 2024-09-13T18:28:14.486 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:28:14.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.488+0000 7f2106a87640 1 -- 172.21.15.114:0/2401426228 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f20e80a1350 msgr2=0x7f20e80a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:14.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.488+0000 7f2106a87640 1 --2- 172.21.15.114:0/2401426228 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f20e80a1350 0x7f20e80a3810 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f20f4009ec0 tx=0x7f20f4008040 comp rx=0 tx=0).stop 2024-09-13T18:28:14.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.488+0000 7f2106a87640 1 -- 172.21.15.114:0/2401426228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2100107920 msgr2=0x7f21001150c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:14.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.488+0000 7f2106a87640 1 --2- 172.21.15.114:0/2401426228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2100107920 0x7f21001150c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f20f000d6d0 tx=0x7f20f000db90 comp rx=0 tx=0).stop 2024-09-13T18:28:14.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.488+0000 7f2106a87640 1 -- 172.21.15.114:0/2401426228 shutdown_connections 2024-09-13T18:28:14.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.488+0000 7f2106a87640 1 --2- 172.21.15.114:0/2401426228 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f20e80a1350 0x7f20e80a3810 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.488+0000 7f2106a87640 1 --2- 172.21.15.114:0/2401426228 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2100119400 0x7f21001a1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.488+0000 7f2106a87640 1 --2- 172.21.15.114:0/2401426228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2100107920 0x7f21001150c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.488+0000 7f2106a87640 1 --2- 172.21.15.114:0/2401426228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2100107540 0x7f2100114b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.488+0000 7f2106a87640 1 -- 172.21.15.114:0/2401426228 >> 172.21.15.114:0/2401426228 conn(0x7f210006afe0 msgr2=0x7f210006ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:14.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.491+0000 7f2106a87640 1 -- 172.21.15.114:0/2401426228 shutdown_connections 2024-09-13T18:28:14.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.491+0000 7f2106a87640 1 -- 172.21.15.114:0/2401426228 wait complete. 2024-09-13T18:28:14.517 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-09-13T18:28:14.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.771+0000 7fa812ffd640 1 -- 172.21.15.114:0/2414974010 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa804040770 con 0x7fa814107920 2024-09-13T18:28:14.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.771+0000 7fa81a69c640 1 -- 172.21.15.114:0/2414974010 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa814107920 msgr2=0x7fa814107d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:14.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.771+0000 7fa81a69c640 1 --2- 172.21.15.114:0/2414974010 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa814107920 0x7fa814107d60 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fa80401a910 tx=0x7fa80403b450 comp rx=0 tx=0).stop 2024-09-13T18:28:14.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.772+0000 7fa81a69c640 1 -- 172.21.15.114:0/2414974010 shutdown_connections 2024-09-13T18:28:14.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.772+0000 7fa81a69c640 1 --2- 172.21.15.114:0/2414974010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa814119400 0x7fa81411b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.772+0000 7fa81a69c640 1 --2- 172.21.15.114:0/2414974010 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa814107920 0x7fa814107d60 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.772+0000 7fa81a69c640 1 --2- 172.21.15.114:0/2414974010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa814107540 0x7fa81410e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.772+0000 7fa81a69c640 1 -- 172.21.15.114:0/2414974010 >> 172.21.15.114:0/2414974010 conn(0x7fa81406afe0 msgr2=0x7fa81406b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:14.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.773+0000 7fa81a69c640 1 -- 172.21.15.114:0/2414974010 shutdown_connections 2024-09-13T18:28:14.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.773+0000 7fa81a69c640 1 -- 172.21.15.114:0/2414974010 wait complete. 2024-09-13T18:28:14.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.774+0000 7fa81a69c640 1 Processor -- start 2024-09-13T18:28:14.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.774+0000 7fa81a69c640 1 -- start start 2024-09-13T18:28:14.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.776+0000 7fa81a69c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa814107540 0x7fa814114c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:14.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.777+0000 7fa81a69c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa814107920 0x7fa814115170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:14.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.777+0000 7fa81a69c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa814119400 0x7fa8141a1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:14.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.777+0000 7fa81a69c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8141a18b0 con 0x7fa814107920 2024-09-13T18:28:14.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.777+0000 7fa81a69c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8141a1a20 con 0x7fa814107540 2024-09-13T18:28:14.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.777+0000 7fa81a69c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8141a1be0 con 0x7fa814119400 2024-09-13T18:28:14.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.777+0000 7fa813fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa814107540 0x7fa814114c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:14.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.777+0000 7fa813fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa814107540 0x7fa814114c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:36142/0 (socket says 172.21.15.114:36142) 2024-09-13T18:28:14.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.777+0000 7fa813fff640 1 -- 172.21.15.114:0/3975888877 learned_addr learned my addr 172.21.15.114:0/3975888877 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:14.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.777+0000 7fa818c39640 1 --2- 172.21.15.114:0/3975888877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa814119400 0x7fa8141a1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:14.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.778+0000 7fa818c39640 1 -- 172.21.15.114:0/3975888877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa814107540 msgr2=0x7fa814114c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:14.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.778+0000 7fa818c39640 1 --2- 172.21.15.114:0/3975888877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa814107540 0x7fa814114c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.778+0000 7fa818c39640 1 -- 172.21.15.114:0/3975888877 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa814107920 msgr2=0x7fa814115170 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:28:14.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.779+0000 7fa818c39640 1 --2- 172.21.15.114:0/3975888877 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa814107920 0x7fa814115170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.779+0000 7fa818c39640 1 -- 172.21.15.114:0/3975888877 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa80401a570 con 0x7fa814119400 2024-09-13T18:28:14.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.782+0000 7fa818c39640 1 --2- 172.21.15.114:0/3975888877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa814119400 0x7fa8141a1240 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa80800e970 tx=0x7fa80800ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:14.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.783+0000 7fa8117fa640 1 -- 172.21.15.114:0/3975888877 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa808009620 con 0x7fa814119400 2024-09-13T18:28:14.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.783+0000 7fa81a69c640 1 -- 172.21.15.114:0/3975888877 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8141a1ec0 con 0x7fa814119400 2024-09-13T18:28:14.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.783+0000 7fa81a69c640 1 -- 172.21.15.114:0/3975888877 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8141a9660 con 0x7fa814119400 2024-09-13T18:28:14.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.784+0000 7fa8117fa640 1 -- 172.21.15.114:0/3975888877 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa8080044b0 con 0x7fa814119400 2024-09-13T18:28:14.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.784+0000 7fa8117fa640 1 -- 172.21.15.114:0/3975888877 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8080047a0 con 0x7fa814119400 2024-09-13T18:28:14.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.788+0000 7fa81d3c6740 1 Processor -- start 2024-09-13T18:28:14.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.793+0000 7fa81d3c6740 1 -- start start 2024-09-13T18:28:14.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.793+0000 7fa81d3c6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd0e703790 0x55bd0e703b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:14.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.793+0000 7fa81d3c6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd0e7040b0 0x55bd0e70e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:14.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.793+0000 7fa81d3c6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd0e70e950 0x55bd0e710d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:14.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.793+0000 7fa81d3c6740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd0e66dcc0 con 0x55bd0e70e950 2024-09-13T18:28:14.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.793+0000 7fa81d3c6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd0e66b450 con 0x55bd0e7040b0 2024-09-13T18:28:14.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.793+0000 7fa81d3c6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd0e666d40 con 0x55bd0e703790 2024-09-13T18:28:14.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.793+0000 7fa8117fa640 1 -- 172.21.15.114:0/3975888877 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fa80800ccc0 con 0x7fa814119400 2024-09-13T18:28:14.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.794+0000 7fa8137fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd0e703790 0x55bd0e703b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:14.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.794+0000 7fa8137fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd0e703790 0x55bd0e703b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:38814/0 (socket says 172.21.15.114:38814) 2024-09-13T18:28:14.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.794+0000 7fa8137fe640 1 -- 172.21.15.114:0/3193375467 learned_addr learned my addr 172.21.15.114:0/3193375467 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:14.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.797+0000 7fa8117fa640 1 --2- 172.21.15.114:0/3975888877 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa7fc0a1410 0x7fa7fc0a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:14.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.797+0000 7fa8117fa640 1 -- 172.21.15.114:0/3975888877 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fa808014070 con 0x7fa814119400 2024-09-13T18:28:14.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.798+0000 7fa818c39640 1 --2- 172.21.15.114:0/3975888877 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa7fc0a1410 0x7fa7fc0a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:14.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.799+0000 7fa818c39640 1 --2- 172.21.15.114:0/3975888877 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa7fc0a1410 0x7fa7fc0a38d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa808096640 tx=0x7fa8080046c0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:14.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.799+0000 7fa8137fe640 1 --2- 172.21.15.114:0/3193375467 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd0e70e950 0x55bd0e710d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:14.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.800+0000 7fa813fff640 1 --2- 172.21.15.114:0/3193375467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd0e7040b0 0x55bd0e70e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:14.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.801+0000 7fa813fff640 1 -- 172.21.15.114:0/3193375467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd0e703790 msgr2=0x55bd0e703b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:14.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.801+0000 7fa813fff640 1 --2- 172.21.15.114:0/3193375467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd0e703790 0x55bd0e703b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.801+0000 7fa813fff640 1 -- 172.21.15.114:0/3193375467 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd0e70e950 msgr2=0x55bd0e710d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:14.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.801+0000 7fa813fff640 1 --2- 172.21.15.114:0/3193375467 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd0e70e950 0x55bd0e710d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.801+0000 7fa813fff640 1 -- 172.21.15.114:0/3193375467 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd0e62e620 con 0x55bd0e7040b0 2024-09-13T18:28:14.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.802+0000 7fa813fff640 1 --2- 172.21.15.114:0/3193375467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd0e7040b0 0x55bd0e70e410 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa80c00e1c0 tx=0x7fa80c009790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=134c17c10426f63c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:14.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.805+0000 7fa7fa7fc640 1 -- 172.21.15.114:0/3193375467 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80c014680 con 0x55bd0e7040b0 2024-09-13T18:28:14.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.805+0000 7fa7fa7fc640 1 -- 172.21.15.114:0/3193375467 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa80c00f040 con 0x55bd0e7040b0 2024-09-13T18:28:14.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.806+0000 7fa81d3c6740 1 -- 172.21.15.114:0/3193375467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd0e7040b0 msgr2=0x55bd0e70e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:14.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.806+0000 7fa81d3c6740 1 --2- 172.21.15.114:0/3193375467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd0e7040b0 0x55bd0e70e410 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa80c00e1c0 tx=0x7fa80c009790 comp rx=0 tx=0).stop 2024-09-13T18:28:14.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.807+0000 7fa81d3c6740 1 -- 172.21.15.114:0/3193375467 shutdown_connections 2024-09-13T18:28:14.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.807+0000 7fa81d3c6740 1 --2- 172.21.15.114:0/3193375467 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd0e70e950 0x55bd0e710d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.807+0000 7fa81d3c6740 1 --2- 172.21.15.114:0/3193375467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd0e7040b0 0x55bd0e70e410 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.807+0000 7fa81d3c6740 1 --2- 172.21.15.114:0/3193375467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd0e703790 0x55bd0e703b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.807+0000 7fa81d3c6740 1 -- 172.21.15.114:0/3193375467 >> 172.21.15.114:0/3193375467 conn(0x55bd0e6ffd00 msgr2=0x55bd0e7000e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:14.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.810+0000 7fa81d3c6740 1 -- 172.21.15.114:0/3193375467 shutdown_connections 2024-09-13T18:28:14.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.814+0000 7fa81d3c6740 1 -- 172.21.15.114:0/3193375467 wait complete. 2024-09-13T18:28:14.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.814+0000 7fa81d3c6740 1 -- start start 2024-09-13T18:28:14.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.814+0000 7fa81d3c6740 1 Processor -- start 2024-09-13T18:28:14.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.815+0000 7fa81d3c6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd0e703790 0x55bd0e70b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:14.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.815+0000 7fa81d3c6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd0e7040b0 0x55bd0e70b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:14.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.815+0000 7fa81d3c6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd0e70e950 0x55bd0e7127b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:14.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.815+0000 7fa81d3c6740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd0e625180 con 0x55bd0e70e950 2024-09-13T18:28:14.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.815+0000 7fa81d3c6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd0e33f430 con 0x55bd0e703790 2024-09-13T18:28:14.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.815+0000 7fa81d3c6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd0e5ea5e0 con 0x55bd0e7040b0 2024-09-13T18:28:14.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.815+0000 7fa81d3c6740 10 client.0 _ll_register_callbacks cb 0x55bd0e656aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:28:14.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.818+0000 7fa8137fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd0e703790 0x55bd0e70b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:14.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.818+0000 7fa8137fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd0e703790 0x55bd0e70b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:36150/0 (socket says 172.21.15.114:36150) 2024-09-13T18:28:14.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.818+0000 7fa8137fe640 1 -- 172.21.15.114:0/3360781486 learned_addr learned my addr 172.21.15.114:0/3360781486 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:14.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.818+0000 7fa8137fe640 1 --2- 172.21.15.114:0/3360781486 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd0e70e950 0x55bd0e7127b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:14.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.818+0000 7fa813fff640 1 --2- 172.21.15.114:0/3360781486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd0e7040b0 0x55bd0e70b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:14.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.819+0000 7fa8137fe640 1 -- 172.21.15.114:0/3360781486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd0e7040b0 msgr2=0x55bd0e70b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:14.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.819+0000 7fa8137fe640 1 --2- 172.21.15.114:0/3360781486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd0e7040b0 0x55bd0e70b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.819+0000 7fa8137fe640 1 -- 172.21.15.114:0/3360781486 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd0e70e950 msgr2=0x55bd0e7127b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:14.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.819+0000 7fa8137fe640 1 --2- 172.21.15.114:0/3360781486 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd0e70e950 0x55bd0e7127b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.819+0000 7fa8137fe640 1 -- 172.21.15.114:0/3360781486 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa80c009d20 con 0x55bd0e703790 2024-09-13T18:28:14.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.822+0000 7fa8137fe640 1 --2- 172.21.15.114:0/3360781486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd0e703790 0x55bd0e70b310 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa804005ab0 tx=0x7fa80401a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28de2b48005f0808 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:14.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.822+0000 7fa7f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:28:14.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.822+0000 7fa7f37fe640 1 -- 172.21.15.114:0/3360781486 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8040054c0 con 0x55bd0e703790 2024-09-13T18:28:14.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.822+0000 7fa7f37fe640 1 -- 172.21.15.114:0/3360781486 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fa804005620 con 0x55bd0e703790 2024-09-13T18:28:14.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.822+0000 7fa81d3c6740 1 -- 172.21.15.114:0/3360781486 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bd0e690b80 con 0x55bd0e703790 2024-09-13T18:28:14.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.823+0000 7fa7f37fe640 1 -- 172.21.15.114:0/3360781486 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa804004030 con 0x55bd0e703790 2024-09-13T18:28:14.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.823+0000 7fa81d3c6740 10 client.26759 fetch_fsmap learned FSMap version 34 2024-09-13T18:28:14.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.823+0000 7fa81d3c6740 1 -- 172.21.15.114:0/3360781486 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55bd0e6653c0 con 0x55bd0e703790 2024-09-13T18:28:14.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.823+0000 7fa7f37fe640 1 -- 172.21.15.114:0/3360781486 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55bd0e690b80 con 0x55bd0e703790 2024-09-13T18:28:14.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.824+0000 7fa7f37fe640 1 -- 172.21.15.114:0/3360781486 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa804003d20 con 0x55bd0e703790 2024-09-13T18:28:14.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.824+0000 7fa81d3c6740 10 client.26759 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:28:14.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.824+0000 7fa81d3c6740 10 client.26759 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-13T18:28:14.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.824+0000 7fa81d3c6740 20 client.26759 populate_metadata read hostname 'smithi114' 2024-09-13T18:28:14.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.824+0000 7fa81d3c6740 1 --2- 172.21.15.114:0/3360781486 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55bd0e71e290 0x55bd0e7206b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:14.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.824+0000 7fa81d3c6740 4 client.26759 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:28:14.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.824+0000 7fa81d3c6740 1 -- 172.21.15.114:0/3360781486 --> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bd0e6591f0 con 0x55bd0e71e290 2024-09-13T18:28:14.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.824+0000 7fa813fff640 1 --2- 172.21.15.114:0/3360781486 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55bd0e71e290 0x55bd0e7206b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:14.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.826+0000 7fa7f37fe640 10 client.26759 ms_handle_connect on v2:172.21.15.134:6834/1150003275 2024-09-13T18:28:14.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.826+0000 7fa813fff640 1 --2- 172.21.15.114:0/3360781486 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55bd0e71e290 0x55bd0e7206b0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=955d67aaf90edd2f server_cookie=9f0df4e99e5ef856 in_seq=0 out_seq=0 2024-09-13T18:28:14.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.827+0000 7fa7f37fe640 1 -- 172.21.15.114:0/3360781486 <== mds.3 v2:172.21.15.134:6834/1150003275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55bd0e6591f0 con 0x55bd0e71e290 2024-09-13T18:28:14.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.827+0000 7fa7f37fe640 10 client.26759 handle_command_reply: tid=0 2024-09-13T18:28:14.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.827+0000 7fa81d3c6740 1 client.26759 shutdown 2024-09-13T18:28:14.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.828+0000 7fa81d3c6740 1 -- 172.21.15.114:0/3360781486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd0e703790 msgr2=0x55bd0e70b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:14.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.828+0000 7fa81d3c6740 1 --2- 172.21.15.114:0/3360781486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd0e703790 0x55bd0e70b310 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa804005ab0 tx=0x7fa80401a270 comp rx=0 tx=0).stop 2024-09-13T18:28:14.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.829+0000 7fa81d3c6740 1 -- 172.21.15.114:0/3360781486 shutdown_connections 2024-09-13T18:28:14.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.829+0000 7fa81d3c6740 1 --2- 172.21.15.114:0/3360781486 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55bd0e71e290 0x55bd0e7206b0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.829+0000 7fa81d3c6740 1 --2- 172.21.15.114:0/3360781486 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd0e70e950 0x55bd0e7127b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.829+0000 7fa81d3c6740 1 --2- 172.21.15.114:0/3360781486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd0e7040b0 0x55bd0e70b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.829+0000 7fa81d3c6740 1 --2- 172.21.15.114:0/3360781486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd0e703790 0x55bd0e70b310 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.829+0000 7fa81d3c6740 1 -- 172.21.15.114:0/3360781486 >> 172.21.15.114:0/3360781486 conn(0x55bd0e6ffd00 msgr2=0x55bd0e6feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:14.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.829+0000 7fa7f37fe640 0 client.26759 ms_handle_reset on v2:172.21.15.134:6834/1150003275 2024-09-13T18:28:14.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.829+0000 7fa81d3c6740 1 -- 172.21.15.114:0/3360781486 shutdown_connections 2024-09-13T18:28:14.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.829+0000 7fa81d3c6740 1 -- 172.21.15.114:0/3360781486 wait complete. 2024-09-13T18:28:14.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.830+0000 7fa81d3c6740 20 client.26759 trim_cache size 0 max 16384 2024-09-13T18:28:14.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.842+0000 7fa81d3c6740 1 Processor -- start 2024-09-13T18:28:14.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.842+0000 7fa81d3c6740 1 -- start start 2024-09-13T18:28:14.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.842+0000 7fa81d3c6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd0e703790 0x55bd0e712730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:14.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.842+0000 7fa81d3c6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd0e7040b0 0x55bd0e706200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:14.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.842+0000 7fa81d3c6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd0e70e950 0x55bd0e706740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:14.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.842+0000 7fa81d3c6740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd0e690bb0 con 0x55bd0e70e950 2024-09-13T18:28:14.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.842+0000 7fa81d3c6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd0e700c90 con 0x55bd0e7040b0 2024-09-13T18:28:14.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.842+0000 7fa81d3c6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd0e702fa0 con 0x55bd0e703790 2024-09-13T18:28:14.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.842+0000 7fa8137fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd0e70e950 0x55bd0e706740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:14.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.842+0000 7fa8137fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd0e70e950 0x55bd0e706740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55462/0 (socket says 172.21.15.114:55462) 2024-09-13T18:28:14.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.842+0000 7fa8137fe640 1 -- 172.21.15.114:0/2720636933 learned_addr learned my addr 172.21.15.114:0/2720636933 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:14.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.842+0000 7fa813fff640 1 --2- 172.21.15.114:0/2720636933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd0e7040b0 0x55bd0e706200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:14.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.842+0000 7fa8137fe640 1 --2- 172.21.15.114:0/2720636933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd0e703790 0x55bd0e712730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:14.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.843+0000 7fa8137fe640 1 -- 172.21.15.114:0/2720636933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd0e703790 msgr2=0x55bd0e712730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:14.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.843+0000 7fa8137fe640 1 --2- 172.21.15.114:0/2720636933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd0e703790 0x55bd0e712730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.843+0000 7fa8137fe640 1 -- 172.21.15.114:0/2720636933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd0e7040b0 msgr2=0x55bd0e706200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:14.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.843+0000 7fa8137fe640 1 --2- 172.21.15.114:0/2720636933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd0e7040b0 0x55bd0e706200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.843+0000 7fa8137fe640 1 -- 172.21.15.114:0/2720636933 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd0e607770 con 0x55bd0e70e950 2024-09-13T18:28:14.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.843+0000 7fa8137fe640 1 --2- 172.21.15.114:0/2720636933 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd0e70e950 0x55bd0e706740 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fa804005fd0 tx=0x7fa80403b450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e8f4ece4ac22cd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:14.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.843+0000 7fa7faffd640 1 -- 172.21.15.114:0/2720636933 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8040054c0 con 0x55bd0e70e950 2024-09-13T18:28:14.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.843+0000 7fa7faffd640 1 -- 172.21.15.114:0/2720636933 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa80405aec0 con 0x55bd0e70e950 2024-09-13T18:28:14.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.843+0000 7fa7faffd640 1 -- 172.21.15.114:0/2720636933 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa804040c70 con 0x55bd0e70e950 2024-09-13T18:28:14.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.844+0000 7fa81d3c6740 1 -- 172.21.15.114:0/2720636933 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd0e70e950 msgr2=0x55bd0e706740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:14.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.844+0000 7fa81d3c6740 1 --2- 172.21.15.114:0/2720636933 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd0e70e950 0x55bd0e706740 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fa804005fd0 tx=0x7fa80403b450 comp rx=0 tx=0).stop 2024-09-13T18:28:14.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.844+0000 7fa81d3c6740 1 -- 172.21.15.114:0/2720636933 shutdown_connections 2024-09-13T18:28:14.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.844+0000 7fa81d3c6740 1 --2- 172.21.15.114:0/2720636933 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd0e70e950 0x55bd0e706740 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.844+0000 7fa81d3c6740 1 --2- 172.21.15.114:0/2720636933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd0e7040b0 0x55bd0e706200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.844+0000 7fa81d3c6740 1 --2- 172.21.15.114:0/2720636933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd0e703790 0x55bd0e712730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.844+0000 7fa81d3c6740 1 -- 172.21.15.114:0/2720636933 >> 172.21.15.114:0/2720636933 conn(0x55bd0e6ffd00 msgr2=0x55bd0e7099d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:14.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.844+0000 7fa81d3c6740 1 -- 172.21.15.114:0/2720636933 shutdown_connections 2024-09-13T18:28:14.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.844+0000 7fa81d3c6740 1 -- 172.21.15.114:0/2720636933 wait complete. 2024-09-13T18:28:14.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.844+0000 7fa81d3c6740 1 -- start start 2024-09-13T18:28:14.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.844+0000 7fa81d3c6740 1 Processor -- start 2024-09-13T18:28:14.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.845+0000 7fa81d3c6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd0e703790 0x55bd0e715ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:14.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.845+0000 7fa81d3c6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd0e7040b0 0x55bd0e7160e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:14.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.845+0000 7fa81d3c6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd0e70e950 0x55bd0e76d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:14.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.845+0000 7fa81d3c6740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd0e76dd50 con 0x55bd0e70e950 2024-09-13T18:28:14.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.845+0000 7fa81d3c6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd0e76dec0 con 0x55bd0e7040b0 2024-09-13T18:28:14.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.845+0000 7fa81d3c6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd0e76e030 con 0x55bd0e703790 2024-09-13T18:28:14.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.845+0000 7fa81d3c6740 10 client.0 _ll_register_callbacks cb 0x7fa80c009b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:28:14.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.845+0000 7fa8137fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd0e70e950 0x55bd0e76d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:14.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.845+0000 7fa8137fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd0e70e950 0x55bd0e76d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55470/0 (socket says 172.21.15.114:55470) 2024-09-13T18:28:14.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.845+0000 7fa8137fe640 1 -- 172.21.15.114:0/146793531 learned_addr learned my addr 172.21.15.114:0/146793531 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:14.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.845+0000 7fa8137fe640 1 --2- 172.21.15.114:0/146793531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd0e703790 0x55bd0e715ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:14.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.845+0000 7fa813fff640 1 --2- 172.21.15.114:0/146793531 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd0e7040b0 0x55bd0e7160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:14.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.845+0000 7fa8137fe640 1 -- 172.21.15.114:0/146793531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd0e703790 msgr2=0x55bd0e715ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:14.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.845+0000 7fa8137fe640 1 --2- 172.21.15.114:0/146793531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd0e703790 0x55bd0e715ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.845+0000 7fa8137fe640 1 -- 172.21.15.114:0/146793531 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd0e7040b0 msgr2=0x55bd0e7160e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:14.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.845+0000 7fa8137fe640 1 --2- 172.21.15.114:0/146793531 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd0e7040b0 0x55bd0e7160e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.845+0000 7fa8137fe640 1 -- 172.21.15.114:0/146793531 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bd0e6653c0 con 0x55bd0e70e950 2024-09-13T18:28:14.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.845+0000 7fa813fff640 1 --2- 172.21.15.114:0/146793531 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd0e7040b0 0x55bd0e7160e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:28:14.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.845+0000 7fa8137fe640 1 --2- 172.21.15.114:0/146793531 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd0e70e950 0x55bd0e76d810 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fa804005ab0 tx=0x7fa804040680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f62ed22f1f1a9d3e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:14.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.845+0000 7fa7f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:28:14.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.845+0000 7fa7f37fe640 1 -- 172.21.15.114:0/146793531 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80405a390 con 0x55bd0e70e950 2024-09-13T18:28:14.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.846+0000 7fa7f37fe640 1 -- 172.21.15.114:0/146793531 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fa80405a4f0 con 0x55bd0e70e950 2024-09-13T18:28:14.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.846+0000 7fa81d3c6740 1 -- 172.21.15.114:0/146793531 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa80c00f040 con 0x55bd0e70e950 2024-09-13T18:28:14.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.846+0000 7fa7f37fe640 1 -- 172.21.15.114:0/146793531 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa80405aa30 con 0x55bd0e70e950 2024-09-13T18:28:14.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.846+0000 7fa81d3c6740 10 client.17355 fetch_fsmap learned FSMap version 34 2024-09-13T18:28:14.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.846+0000 7fa81d3c6740 1 -- 172.21.15.114:0/146793531 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55bd0e62ce80 con 0x55bd0e70e950 2024-09-13T18:28:14.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.846+0000 7fa7f37fe640 1 -- 172.21.15.114:0/146793531 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa804079080 con 0x55bd0e70e950 2024-09-13T18:28:14.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.846+0000 7fa7f37fe640 1 -- 172.21.15.114:0/146793531 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa80405b600 con 0x55bd0e70e950 2024-09-13T18:28:14.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.846+0000 7fa81d3c6740 10 client.17355 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:28:14.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.846+0000 7fa81d3c6740 10 client.17355 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-13T18:28:14.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.846+0000 7fa81d3c6740 20 client.17355 populate_metadata read hostname 'smithi114' 2024-09-13T18:28:14.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.846+0000 7fa81d3c6740 1 --2- 172.21.15.114:0/146793531 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55bd0e775380 0x55bd0e777770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:14.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.846+0000 7fa81d3c6740 4 client.17355 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:28:14.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.846+0000 7fa81d3c6740 1 -- 172.21.15.114:0/146793531 --> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bd0e35a4e0 con 0x55bd0e775380 2024-09-13T18:28:14.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.847+0000 7fa813fff640 1 --2- 172.21.15.114:0/146793531 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55bd0e775380 0x55bd0e777770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:14.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.847+0000 7fa813fff640 1 --2- 172.21.15.114:0/146793531 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55bd0e775380 0x55bd0e777770 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a00f07aaf5472827 server_cookie=515963be41e2aa9c in_seq=0 out_seq=0 2024-09-13T18:28:14.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.847+0000 7fa7f37fe640 10 client.17355 ms_handle_connect on v2:172.21.15.134:6834/1150003275 2024-09-13T18:28:14.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.847+0000 7fa7f37fe640 1 -- 172.21.15.114:0/146793531 <== mds.3 v2:172.21.15.134:6834/1150003275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bd0e35a4e0 con 0x55bd0e775380 2024-09-13T18:28:14.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.847+0000 7fa7f37fe640 10 client.17355 handle_command_reply: tid=0 2024-09-13T18:28:14.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.847+0000 7fa81d3c6740 1 client.17355 shutdown 2024-09-13T18:28:14.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.847+0000 7fa81d3c6740 1 -- 172.21.15.114:0/146793531 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd0e70e950 msgr2=0x55bd0e76d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:14.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.847+0000 7fa81d3c6740 1 --2- 172.21.15.114:0/146793531 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd0e70e950 0x55bd0e76d810 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fa804005ab0 tx=0x7fa804040680 comp rx=0 tx=0).stop 2024-09-13T18:28:14.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.847+0000 7fa81d3c6740 1 -- 172.21.15.114:0/146793531 shutdown_connections 2024-09-13T18:28:14.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.847+0000 7fa81d3c6740 1 --2- 172.21.15.114:0/146793531 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55bd0e775380 0x55bd0e777770 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.847+0000 7fa81d3c6740 1 --2- 172.21.15.114:0/146793531 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd0e70e950 0x55bd0e76d810 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.847+0000 7fa81d3c6740 1 --2- 172.21.15.114:0/146793531 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd0e7040b0 0x55bd0e7160e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.847+0000 7fa81d3c6740 1 --2- 172.21.15.114:0/146793531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd0e703790 0x55bd0e715ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.847+0000 7fa81d3c6740 1 -- 172.21.15.114:0/146793531 >> 172.21.15.114:0/146793531 conn(0x55bd0e6ffd00 msgr2=0x55bd0e70a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:14.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.847+0000 7fa7f37fe640 0 client.17355 ms_handle_reset on v2:172.21.15.134:6834/1150003275 2024-09-13T18:28:14.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.847+0000 7fa81d3c6740 1 -- 172.21.15.114:0/146793531 shutdown_connections 2024-09-13T18:28:14.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.847+0000 7fa81d3c6740 1 -- 172.21.15.114:0/146793531 wait complete. 2024-09-13T18:28:14.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.848+0000 7fa81d3c6740 20 client.17355 trim_cache size 0 max 16384 2024-09-13T18:28:14.857 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:28:14.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.848+0000 7fa81a69c640 1 -- 172.21.15.114:0/3975888877 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa7fc0a1410 msgr2=0x7fa7fc0a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:14.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.848+0000 7fa81a69c640 1 --2- 172.21.15.114:0/3975888877 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa7fc0a1410 0x7fa7fc0a38d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa808096640 tx=0x7fa8080046c0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.848+0000 7fa81a69c640 1 -- 172.21.15.114:0/3975888877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa814119400 msgr2=0x7fa8141a1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:14.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.848+0000 7fa81a69c640 1 --2- 172.21.15.114:0/3975888877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa814119400 0x7fa8141a1240 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa80800e970 tx=0x7fa80800ee30 comp rx=0 tx=0).stop 2024-09-13T18:28:14.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.848+0000 7fa81a69c640 1 -- 172.21.15.114:0/3975888877 shutdown_connections 2024-09-13T18:28:14.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.848+0000 7fa81a69c640 1 --2- 172.21.15.114:0/3975888877 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa7fc0a1410 0x7fa7fc0a38d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.848+0000 7fa81a69c640 1 --2- 172.21.15.114:0/3975888877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa814119400 0x7fa8141a1240 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.848+0000 7fa81a69c640 1 --2- 172.21.15.114:0/3975888877 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa814107920 0x7fa814115170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.848+0000 7fa81a69c640 1 --2- 172.21.15.114:0/3975888877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa814107540 0x7fa814114c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:14.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.848+0000 7fa81a69c640 1 -- 172.21.15.114:0/3975888877 >> 172.21.15.114:0/3975888877 conn(0x7fa81406afe0 msgr2=0x7fa814070200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:14.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.848+0000 7fa81a69c640 1 -- 172.21.15.114:0/3975888877 shutdown_connections 2024-09-13T18:28:14.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:14.848+0000 7fa81a69c640 1 -- 172.21.15.114:0/3975888877 wait complete. 2024-09-13T18:28:14.859 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:14 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/663538685' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:28:14.861 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-09-13T18:28:14.868 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:14 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/663538685' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:28:14.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:14 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/663538685' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:28:15.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.026+0000 7fd0f706e640 1 -- 172.21.15.114:0/1367637755 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0f0119400 msgr2=0x7fd0f011b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:15.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.026+0000 7fd0f706e640 1 --2- 172.21.15.114:0/1367637755 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0f0119400 0x7fd0f011b7f0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4009900 tx=0x7fd0e402a420 comp rx=0 tx=0).stop 2024-09-13T18:28:15.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.028+0000 7fd0f706e640 1 -- 172.21.15.114:0/1367637755 shutdown_connections 2024-09-13T18:28:15.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.028+0000 7fd0f706e640 1 --2- 172.21.15.114:0/1367637755 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0f0119400 0x7fd0f011b7f0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.028+0000 7fd0f706e640 1 --2- 172.21.15.114:0/1367637755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0f0107890 0x7fd0f0107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.028+0000 7fd0f706e640 1 --2- 172.21.15.114:0/1367637755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0f01074b0 0x7fd0f010e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.028+0000 7fd0f706e640 1 -- 172.21.15.114:0/1367637755 >> 172.21.15.114:0/1367637755 conn(0x7fd0f006af40 msgr2=0x7fd0f006b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:15.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.032+0000 7fd0f706e640 1 -- 172.21.15.114:0/1367637755 shutdown_connections 2024-09-13T18:28:15.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.034+0000 7fd0f706e640 1 -- 172.21.15.114:0/1367637755 wait complete. 2024-09-13T18:28:15.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.034+0000 7fd0f706e640 1 Processor -- start 2024-09-13T18:28:15.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.035+0000 7fd0f706e640 1 -- start start 2024-09-13T18:28:15.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.035+0000 7fd0f706e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0f01074b0 0x7fd0f0112400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.035+0000 7fd0f706e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0f0107890 0x7fd0f0112940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.035+0000 7fd0f706e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0f0119400 0x7fd0f0116f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.035+0000 7fd0f706e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f0117450 con 0x7fd0f0119400 2024-09-13T18:28:15.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.035+0000 7fd0f706e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f01175c0 con 0x7fd0f0107890 2024-09-13T18:28:15.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.035+0000 7fd0f706e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f0117780 con 0x7fd0f01074b0 2024-09-13T18:28:15.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.036+0000 7fd0f686d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0f0119400 0x7fd0f0116f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:15.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.036+0000 7fd0f686d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0f0119400 0x7fd0f0116f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55494/0 (socket says 172.21.15.114:55494) 2024-09-13T18:28:15.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.036+0000 7fd0f686d640 1 -- 172.21.15.114:0/2579575335 learned_addr learned my addr 172.21.15.114:0/2579575335 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:15.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.036+0000 7fd0f586b640 1 --2- 172.21.15.114:0/2579575335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0f0107890 0x7fd0f0112940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:15.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.036+0000 7fd0f686d640 1 -- 172.21.15.114:0/2579575335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0f01074b0 msgr2=0x7fd0f0112400 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:28:15.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.036+0000 7fd0f686d640 1 --2- 172.21.15.114:0/2579575335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0f01074b0 0x7fd0f0112400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.036+0000 7fd0f686d640 1 -- 172.21.15.114:0/2579575335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0f0107890 msgr2=0x7fd0f0112940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:15.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.036+0000 7fd0f686d640 1 --2- 172.21.15.114:0/2579575335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0f0107890 0x7fd0f0112940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.037+0000 7fd0f686d640 1 -- 172.21.15.114:0/2579575335 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e4009560 con 0x7fd0f0119400 2024-09-13T18:28:15.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.037+0000 7fd0f586b640 1 --2- 172.21.15.114:0/2579575335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0f0107890 0x7fd0f0112940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:28:15.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.037+0000 7fd0f686d640 1 --2- 172.21.15.114:0/2579575335 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0f0119400 0x7fd0f0116f10 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4005640 tx=0x7fd0e4005670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:15.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.037+0000 7fd0e37fe640 1 -- 172.21.15.114:0/2579575335 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e4004b50 con 0x7fd0f0119400 2024-09-13T18:28:15.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.037+0000 7fd0e37fe640 1 -- 172.21.15.114:0/2579575335 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd0e4004cb0 con 0x7fd0f0119400 2024-09-13T18:28:15.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.037+0000 7fd0e37fe640 1 -- 172.21.15.114:0/2579575335 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e403c640 con 0x7fd0f0119400 2024-09-13T18:28:15.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.037+0000 7fd0f706e640 1 -- 172.21.15.114:0/2579575335 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0f01179a0 con 0x7fd0f0119400 2024-09-13T18:28:15.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.037+0000 7fd0f706e640 1 -- 172.21.15.114:0/2579575335 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0f01ba590 con 0x7fd0f0119400 2024-09-13T18:28:15.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.043+0000 7fd0e37fe640 1 -- 172.21.15.114:0/2579575335 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fd0e4039040 con 0x7fd0f0119400 2024-09-13T18:28:15.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.043+0000 7fd0e37fe640 1 --2- 172.21.15.114:0/2579575335 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fd0dc0a1340 0x7fd0dc0a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.044+0000 7fd0e37fe640 1 -- 172.21.15.114:0/2579575335 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fd0e40ef1a0 con 0x7fd0f0119400 2024-09-13T18:28:15.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.044+0000 7fd0fafef740 1 Processor -- start 2024-09-13T18:28:15.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.045+0000 7fd0fafef740 1 -- start start 2024-09-13T18:28:15.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.045+0000 7fd0fafef740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624419be970 0x5624419bed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.045+0000 7fd0fafef740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624419bf290 0x5624419c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.045+0000 7fd0fafef740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624419c9b30 0x5624419cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.045+0000 7fd0fafef740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x562441928cc0 con 0x5624419bf290 2024-09-13T18:28:15.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.045+0000 7fd0fafef740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562441926450 con 0x5624419c9b30 2024-09-13T18:28:15.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.045+0000 7fd0fafef740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562441921d40 con 0x5624419be970 2024-09-13T18:28:15.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.045+0000 7fd0f686d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624419c9b30 0x5624419cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:15.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.045+0000 7fd0f686d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624419c9b30 0x5624419cbf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:36202/0 (socket says 172.21.15.114:36202) 2024-09-13T18:28:15.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.045+0000 7fd0f686d640 1 -- 172.21.15.114:0/255381390 learned_addr learned my addr 172.21.15.114:0/255381390 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:15.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.047+0000 7fd0f686d640 1 -- 172.21.15.114:0/255381390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624419be970 msgr2=0x5624419bed50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:28:15.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.047+0000 7fd0f686d640 1 --2- 172.21.15.114:0/255381390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624419be970 0x5624419bed50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.047+0000 7fd0f686d640 1 -- 172.21.15.114:0/255381390 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624419bf290 msgr2=0x5624419c95f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:28:15.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.047+0000 7fd0f686d640 1 --2- 172.21.15.114:0/255381390 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624419bf290 0x5624419c95f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.047+0000 7fd0f686d640 1 -- 172.21.15.114:0/255381390 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624418e9620 con 0x5624419c9b30 2024-09-13T18:28:15.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.048+0000 7fd0f686d640 1 --2- 172.21.15.114:0/255381390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624419c9b30 0x5624419cbf20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd0e403dd50 tx=0x7fd0e40bba40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1df4e70afb4cd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:15.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.048+0000 7fd0f606c640 1 --2- 172.21.15.114:0/2579575335 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fd0dc0a1340 0x7fd0dc0a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:15.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.049+0000 7fd0e0e05640 1 -- 172.21.15.114:0/255381390 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e40316f0 con 0x5624419c9b30 2024-09-13T18:28:15.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.049+0000 7fd0e0e05640 1 -- 172.21.15.114:0/255381390 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd0e40bc600 con 0x5624419c9b30 2024-09-13T18:28:15.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.049+0000 7fd0e0e05640 1 -- 172.21.15.114:0/255381390 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e40eca00 con 0x5624419c9b30 2024-09-13T18:28:15.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.050+0000 7fd0fafef740 1 -- 172.21.15.114:0/255381390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624419c9b30 msgr2=0x5624419cbf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:15.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.050+0000 7fd0fafef740 1 --2- 172.21.15.114:0/255381390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624419c9b30 0x5624419cbf20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd0e403dd50 tx=0x7fd0e40bba40 comp rx=0 tx=0).stop 2024-09-13T18:28:15.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.050+0000 7fd0f606c640 1 --2- 172.21.15.114:0/2579575335 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fd0dc0a1340 0x7fd0dc0a3800 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd0e800abb0 tx=0x7fd0e8009230 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:15.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.051+0000 7fd0fafef740 1 -- 172.21.15.114:0/255381390 shutdown_connections 2024-09-13T18:28:15.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.051+0000 7fd0fafef740 1 --2- 172.21.15.114:0/255381390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624419c9b30 0x5624419cbf20 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.051+0000 7fd0fafef740 1 --2- 172.21.15.114:0/255381390 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624419bf290 0x5624419c95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.051+0000 7fd0fafef740 1 --2- 172.21.15.114:0/255381390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624419be970 0x5624419bed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.051+0000 7fd0fafef740 1 -- 172.21.15.114:0/255381390 >> 172.21.15.114:0/255381390 conn(0x5624419baee0 msgr2=0x5624419bb2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:15.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.062+0000 7fd0fafef740 1 -- 172.21.15.114:0/255381390 shutdown_connections 2024-09-13T18:28:15.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.062+0000 7fd0fafef740 1 -- 172.21.15.114:0/255381390 wait complete. 2024-09-13T18:28:15.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.063+0000 7fd0fafef740 1 -- start start 2024-09-13T18:28:15.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.063+0000 7fd0fafef740 1 Processor -- start 2024-09-13T18:28:15.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.067+0000 7fd0fafef740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624419be970 0x5624419c90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.067+0000 7fd0fafef740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624419bf290 0x5624419c63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.067+0000 7fd0fafef740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624419c9b30 0x5624419c6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.067+0000 7fd0fafef740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624418e0180 con 0x5624419be970 2024-09-13T18:28:15.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.067+0000 7fd0fafef740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624415fa430 con 0x5624419bf290 2024-09-13T18:28:15.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.067+0000 7fd0fafef740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624418a55e0 con 0x5624419c9b30 2024-09-13T18:28:15.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.067+0000 7fd0fafef740 10 client.0 _ll_register_callbacks cb 0x562441911aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:28:15.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.070+0000 7fd0f606c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624419be970 0x5624419c90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:15.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.070+0000 7fd0f586b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624419bf290 0x5624419c63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:15.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.070+0000 7fd0f686d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624419c9b30 0x5624419c6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:15.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.070+0000 7fd0f606c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624419be970 0x5624419c90f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55504/0 (socket says 172.21.15.114:55504) 2024-09-13T18:28:15.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.070+0000 7fd0f606c640 1 -- 172.21.15.114:0/2066641190 learned_addr learned my addr 172.21.15.114:0/2066641190 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:15.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.071+0000 7fd0f606c640 1 -- 172.21.15.114:0/2066641190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624419c9b30 msgr2=0x5624419c6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:15.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.071+0000 7fd0f606c640 1 --2- 172.21.15.114:0/2066641190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624419c9b30 0x5624419c6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.071+0000 7fd0f606c640 1 -- 172.21.15.114:0/2066641190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624419bf290 msgr2=0x5624419c63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:15.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.071+0000 7fd0f606c640 1 --2- 172.21.15.114:0/2066641190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624419bf290 0x5624419c63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.071+0000 7fd0f606c640 1 -- 172.21.15.114:0/2066641190 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0f01ba590 con 0x5624419be970 2024-09-13T18:28:15.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.071+0000 7fd0f586b640 1 --2- 172.21.15.114:0/2066641190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624419bf290 0x5624419c63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:28:15.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.072+0000 7fd0f606c640 1 --2- 172.21.15.114:0/2066641190 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624419be970 0x5624419c90f0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8009fa0 tx=0x7fd0e8004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a03618a27e40b65a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:15.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.073+0000 7fd0d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:28:15.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.073+0000 7fd0d9ffb640 1 -- 172.21.15.114:0/2066641190 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e8017ea0 con 0x5624419be970 2024-09-13T18:28:15.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.073+0000 7fd0d9ffb640 1 -- 172.21.15.114:0/2066641190 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fd0e801d070 con 0x5624419be970 2024-09-13T18:28:15.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.076+0000 7fd0fafef740 1 -- 172.21.15.114:0/2066641190 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56244194bb80 con 0x5624419be970 2024-09-13T18:28:15.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.078+0000 7fd0d9ffb640 1 -- 172.21.15.114:0/2066641190 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd0e8019830 con 0x5624419be970 2024-09-13T18:28:15.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.080+0000 7fd0fafef740 10 client.17364 fetch_fsmap learned FSMap version 34 2024-09-13T18:28:15.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.080+0000 7fd0fafef740 1 -- 172.21.15.114:0/2066641190 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5624419203c0 con 0x5624419be970 2024-09-13T18:28:15.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.084+0000 7fd0d9ffb640 1 -- 172.21.15.114:0/2066641190 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56244194bb80 con 0x5624419be970 2024-09-13T18:28:15.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.084+0000 7fd0d9ffb640 1 -- 172.21.15.114:0/2066641190 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd0e8022b70 con 0x5624419be970 2024-09-13T18:28:15.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.086+0000 7fd0fafef740 10 client.17364 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:28:15.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.086+0000 7fd0fafef740 10 client.17364 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-13T18:28:15.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.086+0000 7fd0fafef740 20 client.17364 populate_metadata read hostname 'smithi114' 2024-09-13T18:28:15.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.086+0000 7fd0fafef740 1 --2- 172.21.15.114:0/2066641190 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x5624419d90b0 0x5624419db4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.086+0000 7fd0fafef740 4 client.17364 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:28:15.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.086+0000 7fd0fafef740 1 -- 172.21.15.114:0/2066641190 --> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5624419141f0 con 0x5624419d90b0 2024-09-13T18:28:15.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.089+0000 7fd0f586b640 1 --2- 172.21.15.114:0/2066641190 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x5624419d90b0 0x5624419db4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:15.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.091+0000 7fd0d9ffb640 10 client.17364 ms_handle_connect on v2:172.21.15.204:6838/344883912 2024-09-13T18:28:15.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.091+0000 7fd0f586b640 1 --2- 172.21.15.114:0/2066641190 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x5624419d90b0 0x5624419db4d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ab53877644794709 server_cookie=407fd3540e1961b in_seq=0 out_seq=0 2024-09-13T18:28:15.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.091+0000 7fd0d9ffb640 1 -- 172.21.15.114:0/2066641190 <== mds.2 v2:172.21.15.204:6838/344883912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5624419141f0 con 0x5624419d90b0 2024-09-13T18:28:15.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.092+0000 7fd0d9ffb640 10 client.17364 handle_command_reply: tid=0 2024-09-13T18:28:15.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.094+0000 7fd0fafef740 1 client.17364 shutdown 2024-09-13T18:28:15.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.097+0000 7fd0fafef740 1 -- 172.21.15.114:0/2066641190 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624419be970 msgr2=0x5624419c90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:15.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.097+0000 7fd0fafef740 1 --2- 172.21.15.114:0/2066641190 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624419be970 0x5624419c90f0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8009fa0 tx=0x7fd0e8004af0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.102+0000 7fd0fafef740 1 -- 172.21.15.114:0/2066641190 shutdown_connections 2024-09-13T18:28:15.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.102+0000 7fd0fafef740 1 --2- 172.21.15.114:0/2066641190 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x5624419d90b0 0x5624419db4d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.102+0000 7fd0fafef740 1 --2- 172.21.15.114:0/2066641190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624419c9b30 0x5624419c6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.102+0000 7fd0fafef740 1 --2- 172.21.15.114:0/2066641190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624419bf290 0x5624419c63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.102+0000 7fd0fafef740 1 --2- 172.21.15.114:0/2066641190 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624419be970 0x5624419c90f0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.102+0000 7fd0fafef740 1 -- 172.21.15.114:0/2066641190 >> 172.21.15.114:0/2066641190 conn(0x5624419baee0 msgr2=0x5624419ba230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:15.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.104+0000 7fd0d9ffb640 0 client.17364 ms_handle_reset on v2:172.21.15.204:6838/344883912 2024-09-13T18:28:15.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.107+0000 7fd0fafef740 1 -- 172.21.15.114:0/2066641190 shutdown_connections 2024-09-13T18:28:15.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.110+0000 7fd0fafef740 1 -- 172.21.15.114:0/2066641190 wait complete. 2024-09-13T18:28:15.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.113+0000 7fd0fafef740 20 client.17364 trim_cache size 0 max 16384 2024-09-13T18:28:15.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.139+0000 7fd0fafef740 1 Processor -- start 2024-09-13T18:28:15.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.140+0000 7fd0fafef740 1 -- start start 2024-09-13T18:28:15.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.140+0000 7fd0fafef740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624419be970 0x5624419cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.140+0000 7fd0fafef740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624419bf290 0x5624419ce480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.140+0000 7fd0fafef740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624419c9b30 0x5624419c8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.141+0000 7fd0fafef740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624419c9940 con 0x5624419c9b30 2024-09-13T18:28:15.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.141+0000 7fd0fafef740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624419bbfd0 con 0x5624419be970 2024-09-13T18:28:15.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.141+0000 7fd0fafef740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624419c61c0 con 0x5624419bf290 2024-09-13T18:28:15.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.141+0000 7fd0f606c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624419be970 0x5624419cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:15.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.141+0000 7fd0f606c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624419be970 0x5624419cdf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:36218/0 (socket says 172.21.15.114:36218) 2024-09-13T18:28:15.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.141+0000 7fd0f606c640 1 -- 172.21.15.114:0/4122778497 learned_addr learned my addr 172.21.15.114:0/4122778497 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:15.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.141+0000 7fd0f686d640 1 --2- 172.21.15.114:0/4122778497 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624419c9b30 0x5624419c8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:15.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.142+0000 7fd0f606c640 1 -- 172.21.15.114:0/4122778497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624419bf290 msgr2=0x5624419ce480 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:28:15.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.142+0000 7fd0f606c640 1 --2- 172.21.15.114:0/4122778497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624419bf290 0x5624419ce480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.142+0000 7fd0f606c640 1 -- 172.21.15.114:0/4122778497 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624419c9b30 msgr2=0x5624419c8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:15.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.142+0000 7fd0f606c640 1 --2- 172.21.15.114:0/4122778497 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624419c9b30 0x5624419c8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.142+0000 7fd0f606c640 1 -- 172.21.15.114:0/4122778497 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624418c2770 con 0x5624419be970 2024-09-13T18:28:15.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.143+0000 7fd0f606c640 1 --2- 172.21.15.114:0/4122778497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624419be970 0x5624419cdf40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd0e800a020 tx=0x7fd0e8009bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da39076d434c9e7c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:15.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.144+0000 7fd0e1606640 1 -- 172.21.15.114:0/4122778497 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e8003c30 con 0x5624419be970 2024-09-13T18:28:15.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.144+0000 7fd0e1606640 1 -- 172.21.15.114:0/4122778497 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd0e8007000 con 0x5624419be970 2024-09-13T18:28:15.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.144+0000 7fd0e1606640 1 -- 172.21.15.114:0/4122778497 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e8019630 con 0x5624419be970 2024-09-13T18:28:15.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.145+0000 7fd0fafef740 1 -- 172.21.15.114:0/4122778497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624419be970 msgr2=0x5624419cdf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:15.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.145+0000 7fd0fafef740 1 --2- 172.21.15.114:0/4122778497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624419be970 0x5624419cdf40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd0e800a020 tx=0x7fd0e8009bd0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.151+0000 7fd0fafef740 1 -- 172.21.15.114:0/4122778497 shutdown_connections 2024-09-13T18:28:15.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.151+0000 7fd0fafef740 1 --2- 172.21.15.114:0/4122778497 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624419c9b30 0x5624419c8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.151+0000 7fd0fafef740 1 --2- 172.21.15.114:0/4122778497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624419bf290 0x5624419ce480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.151+0000 7fd0fafef740 1 --2- 172.21.15.114:0/4122778497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624419be970 0x5624419cdf40 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.151+0000 7fd0fafef740 1 -- 172.21.15.114:0/4122778497 >> 172.21.15.114:0/4122778497 conn(0x5624419baee0 msgr2=0x5624419c6380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:15.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.154+0000 7fd0fafef740 1 -- 172.21.15.114:0/4122778497 shutdown_connections 2024-09-13T18:28:15.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.155+0000 7fd0fafef740 1 -- 172.21.15.114:0/4122778497 wait complete. 2024-09-13T18:28:15.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.155+0000 7fd0fafef740 1 -- start start 2024-09-13T18:28:15.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.155+0000 7fd0fafef740 1 Processor -- start 2024-09-13T18:28:15.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.156+0000 7fd0fafef740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624419be970 0x5624419ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.156+0000 7fd0fafef740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624419bf290 0x5624419cc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.157+0000 7fd0fafef740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624419c9b30 0x5624419ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.157+0000 7fd0f586b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624419bf290 0x5624419cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:15.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.157+0000 7fd0f586b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624419bf290 0x5624419cc600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:36230/0 (socket says 172.21.15.114:36230) 2024-09-13T18:28:15.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.157+0000 7fd0f586b640 1 -- 172.21.15.114:0/2117344404 learned_addr learned my addr 172.21.15.114:0/2117344404 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:15.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.157+0000 7fd0fafef740 1 -- 172.21.15.114:0/2117344404 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624419cd210 con 0x5624419c9b30 2024-09-13T18:28:15.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.157+0000 7fd0fafef740 1 -- 172.21.15.114:0/2117344404 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624419cd380 con 0x5624419bf290 2024-09-13T18:28:15.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.157+0000 7fd0fafef740 1 -- 172.21.15.114:0/2117344404 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562441a2f5a0 con 0x5624419be970 2024-09-13T18:28:15.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.157+0000 7fd0fafef740 10 client.0 _ll_register_callbacks cb 0x7fd0e4005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:28:15.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.158+0000 7fd0f586b640 1 -- 172.21.15.114:0/2117344404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624419be970 msgr2=0x5624419ce1c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:28:15.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.158+0000 7fd0f586b640 1 --2- 172.21.15.114:0/2117344404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624419be970 0x5624419ce1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.158+0000 7fd0f586b640 1 -- 172.21.15.114:0/2117344404 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624419c9b30 msgr2=0x5624419ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:15.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.158+0000 7fd0f586b640 1 --2- 172.21.15.114:0/2117344404 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624419c9b30 0x5624419ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.158+0000 7fd0f586b640 1 -- 172.21.15.114:0/2117344404 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5624419203c0 con 0x5624419bf290 2024-09-13T18:28:15.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.158+0000 7fd0f586b640 1 --2- 172.21.15.114:0/2117344404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624419bf290 0x5624419cc600 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec004700 tx=0x7fd0ec016910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4ff2dac4625f4bf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:15.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.159+0000 7fd0d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:28:15.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.159+0000 7fd0d9ffb640 1 -- 172.21.15.114:0/2117344404 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ec014530 con 0x5624419bf290 2024-09-13T18:28:15.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.160+0000 7fd0d9ffb640 1 -- 172.21.15.114:0/2117344404 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fd0ec014690 con 0x5624419bf290 2024-09-13T18:28:15.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.160+0000 7fd0fafef740 1 -- 172.21.15.114:0/2117344404 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd0e40bc600 con 0x5624419bf290 2024-09-13T18:28:15.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.162+0000 7fd0d9ffb640 1 -- 172.21.15.114:0/2117344404 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd0ec014850 con 0x5624419bf290 2024-09-13T18:28:15.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.162+0000 7fd0fafef740 10 client.26777 fetch_fsmap learned FSMap version 34 2024-09-13T18:28:15.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.162+0000 7fd0fafef740 1 -- 172.21.15.114:0/2117344404 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5624418e7e80 con 0x5624419bf290 2024-09-13T18:28:15.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.165+0000 7fd0d9ffb640 1 -- 172.21.15.114:0/2117344404 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd0e40bc600 con 0x5624419bf290 2024-09-13T18:28:15.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.165+0000 7fd0d9ffb640 1 -- 172.21.15.114:0/2117344404 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd0ec02d9f0 con 0x5624419bf290 2024-09-13T18:28:15.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.166+0000 7fd0fafef740 10 client.26777 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:28:15.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.166+0000 7fd0fafef740 10 client.26777 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-13T18:28:15.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.166+0000 7fd0fafef740 20 client.26777 populate_metadata read hostname 'smithi114' 2024-09-13T18:28:15.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.166+0000 7fd0fafef740 1 --2- 172.21.15.114:0/2117344404 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x562441a30bc0 0x562441a32fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.166+0000 7fd0fafef740 4 client.26777 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:28:15.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.166+0000 7fd0fafef740 1 -- 172.21.15.114:0/2117344404 --> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5624416154e0 con 0x562441a30bc0 2024-09-13T18:28:15.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.166+0000 7fd0f606c640 1 --2- 172.21.15.114:0/2117344404 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x562441a30bc0 0x562441a32fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:15.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.167+0000 7fd0f606c640 1 --2- 172.21.15.114:0/2117344404 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x562441a30bc0 0x562441a32fe0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b315a7d759288fae server_cookie=9d2fdb1a6cf651f0 in_seq=0 out_seq=0 2024-09-13T18:28:15.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.169+0000 7fd0d9ffb640 10 client.26777 ms_handle_connect on v2:172.21.15.204:6838/344883912 2024-09-13T18:28:15.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.169+0000 7fd0d9ffb640 1 -- 172.21.15.114:0/2117344404 <== mds.2 v2:172.21.15.204:6838/344883912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5624416154e0 con 0x562441a30bc0 2024-09-13T18:28:15.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.169+0000 7fd0d9ffb640 10 client.26777 handle_command_reply: tid=0 2024-09-13T18:28:15.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.169+0000 7fd0fafef740 1 client.26777 shutdown 2024-09-13T18:28:15.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.175+0000 7fd0fafef740 1 -- 172.21.15.114:0/2117344404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624419bf290 msgr2=0x5624419cc600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:15.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.175+0000 7fd0fafef740 1 --2- 172.21.15.114:0/2117344404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624419bf290 0x5624419cc600 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec004700 tx=0x7fd0ec016910 comp rx=0 tx=0).stop 2024-09-13T18:28:15.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.175+0000 7fd0fafef740 1 -- 172.21.15.114:0/2117344404 shutdown_connections 2024-09-13T18:28:15.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.175+0000 7fd0fafef740 1 --2- 172.21.15.114:0/2117344404 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x562441a30bc0 0x562441a32fe0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.175+0000 7fd0fafef740 1 --2- 172.21.15.114:0/2117344404 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624419c9b30 0x5624419ccb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.175+0000 7fd0fafef740 1 --2- 172.21.15.114:0/2117344404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624419bf290 0x5624419cc600 secure :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec004700 tx=0x7fd0ec016910 comp rx=0 tx=0).stop 2024-09-13T18:28:15.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.175+0000 7fd0fafef740 1 --2- 172.21.15.114:0/2117344404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624419be970 0x5624419ce1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.175+0000 7fd0fafef740 1 -- 172.21.15.114:0/2117344404 >> 172.21.15.114:0/2117344404 conn(0x5624419baee0 msgr2=0x5624419c1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:15.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.177+0000 7fd0d9ffb640 0 client.26777 ms_handle_reset on v2:172.21.15.204:6838/344883912 2024-09-13T18:28:15.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.177+0000 7fd0fafef740 1 -- 172.21.15.114:0/2117344404 shutdown_connections 2024-09-13T18:28:15.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.181+0000 7fd0fafef740 1 -- 172.21.15.114:0/2117344404 wait complete. 2024-09-13T18:28:15.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.184+0000 7fd0fafef740 20 client.26777 trim_cache size 0 max 16384 2024-09-13T18:28:15.188 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:28:15.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.190+0000 7fd0f706e640 1 -- 172.21.15.114:0/2579575335 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fd0dc0a1340 msgr2=0x7fd0dc0a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:15.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.190+0000 7fd0f706e640 1 --2- 172.21.15.114:0/2579575335 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fd0dc0a1340 0x7fd0dc0a3800 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd0e800abb0 tx=0x7fd0e8009230 comp rx=0 tx=0).stop 2024-09-13T18:28:15.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.190+0000 7fd0f706e640 1 -- 172.21.15.114:0/2579575335 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0f0119400 msgr2=0x7fd0f0116f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:15.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.190+0000 7fd0f706e640 1 --2- 172.21.15.114:0/2579575335 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0f0119400 0x7fd0f0116f10 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4005640 tx=0x7fd0e4005670 comp rx=0 tx=0).stop 2024-09-13T18:28:15.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.192+0000 7fd0f706e640 1 -- 172.21.15.114:0/2579575335 shutdown_connections 2024-09-13T18:28:15.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.192+0000 7fd0f706e640 1 --2- 172.21.15.114:0/2579575335 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fd0dc0a1340 0x7fd0dc0a3800 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.192+0000 7fd0f706e640 1 --2- 172.21.15.114:0/2579575335 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0f0119400 0x7fd0f0116f10 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.192+0000 7fd0f706e640 1 --2- 172.21.15.114:0/2579575335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0f0107890 0x7fd0f0112940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.192+0000 7fd0f706e640 1 --2- 172.21.15.114:0/2579575335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0f01074b0 0x7fd0f0112400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.192+0000 7fd0f706e640 1 -- 172.21.15.114:0/2579575335 >> 172.21.15.114:0/2579575335 conn(0x7fd0f006af40 msgr2=0x7fd0f006eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:15.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.193+0000 7fd0f706e640 1 -- 172.21.15.114:0/2579575335 shutdown_connections 2024-09-13T18:28:15.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.193+0000 7fd0f706e640 1 -- 172.21.15.114:0/2579575335 wait complete. 2024-09-13T18:28:15.218 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-09-13T18:28:15.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.459+0000 7fa4dec87640 1 -- 172.21.15.114:0/1120844944 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4d8119400 msgr2=0x7fa4d811b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:15.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.459+0000 7fa4dec87640 1 --2- 172.21.15.114:0/1120844944 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4d8119400 0x7fa4d811b7f0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0009be0 tx=0x7fa4d002a710 comp rx=0 tx=0).stop 2024-09-13T18:28:15.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.463+0000 7fa4dec87640 1 -- 172.21.15.114:0/1120844944 shutdown_connections 2024-09-13T18:28:15.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.463+0000 7fa4dec87640 1 --2- 172.21.15.114:0/1120844944 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4d8119400 0x7fa4d811b7f0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.463+0000 7fa4dec87640 1 --2- 172.21.15.114:0/1120844944 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4d8107890 0x7fa4d8107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.463+0000 7fa4dec87640 1 --2- 172.21.15.114:0/1120844944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4d81074b0 0x7fa4d810e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.463+0000 7fa4dec87640 1 -- 172.21.15.114:0/1120844944 >> 172.21.15.114:0/1120844944 conn(0x7fa4d806af40 msgr2=0x7fa4d806b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:15.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.470+0000 7fa4dec87640 1 -- 172.21.15.114:0/1120844944 shutdown_connections 2024-09-13T18:28:15.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.470+0000 7fa4dec87640 1 -- 172.21.15.114:0/1120844944 wait complete. 2024-09-13T18:28:15.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.471+0000 7fa4dec87640 1 Processor -- start 2024-09-13T18:28:15.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.471+0000 7fa4dec87640 1 -- start start 2024-09-13T18:28:15.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.471+0000 7fa4dec87640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4d81074b0 0x7fa4d8116910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.471+0000 7fa4dec87640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4d8107890 0x7fa4d8116e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.472+0000 7fa4dec87640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4d8119400 0x7fa4d810fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.472+0000 7fa4dec87640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d81103a0 con 0x7fa4d81074b0 2024-09-13T18:28:15.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.472+0000 7fa4dec87640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d8110510 con 0x7fa4d8119400 2024-09-13T18:28:15.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.472+0000 7fa4dec87640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d81106d0 con 0x7fa4d8107890 2024-09-13T18:28:15.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.472+0000 7fa4dd484640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4d8107890 0x7fa4d8116e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:15.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.472+0000 7fa4dd484640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4d8107890 0x7fa4d8116e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:38892/0 (socket says 172.21.15.114:38892) 2024-09-13T18:28:15.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.472+0000 7fa4dd484640 1 -- 172.21.15.114:0/1139419027 learned_addr learned my addr 172.21.15.114:0/1139419027 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:15.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.472+0000 7fa4de486640 1 --2- 172.21.15.114:0/1139419027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4d8119400 0x7fa4d810fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:15.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.472+0000 7fa4dd484640 1 -- 172.21.15.114:0/1139419027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4d8119400 msgr2=0x7fa4d810fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:15.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.472+0000 7fa4dd484640 1 --2- 172.21.15.114:0/1139419027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4d8119400 0x7fa4d810fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.472+0000 7fa4dd484640 1 -- 172.21.15.114:0/1139419027 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4d81074b0 msgr2=0x7fa4d8116910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:15.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.472+0000 7fa4dd484640 1 --2- 172.21.15.114:0/1139419027 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4d81074b0 0x7fa4d8116910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.472+0000 7fa4dd484640 1 -- 172.21.15.114:0/1139419027 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d0009840 con 0x7fa4d8107890 2024-09-13T18:28:15.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.472+0000 7fa4de486640 1 --2- 172.21.15.114:0/1139419027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4d8119400 0x7fa4d810fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:28:15.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.473+0000 7fa4dd484640 1 --2- 172.21.15.114:0/1139419027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4d8107890 0x7fa4d8116e70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa4d400d950 tx=0x7fa4d400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:15.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.473+0000 7fa4ceffd640 1 -- 172.21.15.114:0/1139419027 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4009c30 con 0x7fa4d8107890 2024-09-13T18:28:15.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.473+0000 7fa4dec87640 1 -- 172.21.15.114:0/1139419027 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4d81144a0 con 0x7fa4d8107890 2024-09-13T18:28:15.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.473+0000 7fa4dec87640 1 -- 172.21.15.114:0/1139419027 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4d81149b0 con 0x7fa4d8107890 2024-09-13T18:28:15.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.474+0000 7fa4ceffd640 1 -- 172.21.15.114:0/1139419027 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa4d4009d90 con 0x7fa4d8107890 2024-09-13T18:28:15.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.474+0000 7fa4ceffd640 1 -- 172.21.15.114:0/1139419027 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d400f620 con 0x7fa4d8107890 2024-09-13T18:28:15.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.479+0000 7fa4ceffd640 1 -- 172.21.15.114:0/1139419027 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fa4d400f780 con 0x7fa4d8107890 2024-09-13T18:28:15.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.480+0000 7fa4e2c0c740 1 Processor -- start 2024-09-13T18:28:15.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.480+0000 7fa4e2c0c740 1 -- start start 2024-09-13T18:28:15.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.480+0000 7fa4e2c0c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a73055970 0x564a73055d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.480+0000 7fa4e2c0c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a73056290 0x564a730605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.480+0000 7fa4e2c0c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a73060b30 0x564a73062f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.480+0000 7fa4e2c0c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a72fbfcc0 con 0x564a73060b30 2024-09-13T18:28:15.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.480+0000 7fa4e2c0c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a72fbd450 con 0x564a73055970 2024-09-13T18:28:15.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.480+0000 7fa4e2c0c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a72fb8d40 con 0x564a73056290 2024-09-13T18:28:15.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.481+0000 7fa4ceffd640 1 --2- 172.21.15.114:0/1139419027 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa4c40a1410 0x7fa4c40a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.482+0000 7fa4ceffd640 1 -- 172.21.15.114:0/1139419027 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fa4d40cdad0 con 0x7fa4d8107890 2024-09-13T18:28:15.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.482+0000 7fa4de486640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a73055970 0x564a73055d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:15.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.482+0000 7fa4de486640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a73055970 0x564a73055d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:36248/0 (socket says 172.21.15.114:36248) 2024-09-13T18:28:15.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.482+0000 7fa4de486640 1 -- 172.21.15.114:0/1695900646 learned_addr learned my addr 172.21.15.114:0/1695900646 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:15.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.482+0000 7fa4de486640 1 --2- 172.21.15.114:0/1139419027 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa4c40a1410 0x7fa4c40a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:15.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.482+0000 7fa4ddc85640 1 --2- 172.21.15.114:0/1695900646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a73056290 0x564a730605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:15.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.482+0000 7fa4de486640 1 -- 172.21.15.114:0/1695900646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a73056290 msgr2=0x564a730605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:15.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.482+0000 7fa4de486640 1 --2- 172.21.15.114:0/1695900646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a73056290 0x564a730605f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.482+0000 7fa4de486640 1 -- 172.21.15.114:0/1695900646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a73060b30 msgr2=0x564a73062f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:28:15.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.482+0000 7fa4de486640 1 --2- 172.21.15.114:0/1695900646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a73060b30 0x564a73062f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.482+0000 7fa4de486640 1 -- 172.21.15.114:0/1695900646 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a72f80620 con 0x564a73055970 2024-09-13T18:28:15.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.483+0000 7fa4de486640 1 --2- 172.21.15.114:0/1695900646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a73055970 0x564a73055d50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0009be0 tx=0x7fa4d00309e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af8b07cde3316ccd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:15.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.483+0000 7fa4c3fff640 1 -- 172.21.15.114:0/1695900646 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d0004240 con 0x564a73055970 2024-09-13T18:28:15.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.483+0000 7fa4c3fff640 1 -- 172.21.15.114:0/1695900646 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa4d0004830 con 0x564a73055970 2024-09-13T18:28:15.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.484+0000 7fa4c3fff640 1 -- 172.21.15.114:0/1695900646 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d0031c10 con 0x564a73055970 2024-09-13T18:28:15.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.484+0000 7fa4e2c0c740 1 -- 172.21.15.114:0/1695900646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a73055970 msgr2=0x564a73055d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:15.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.485+0000 7fa4e2c0c740 1 --2- 172.21.15.114:0/1695900646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a73055970 0x564a73055d50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0009be0 tx=0x7fa4d00309e0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.485+0000 7fa4e2c0c740 1 -- 172.21.15.114:0/1695900646 shutdown_connections 2024-09-13T18:28:15.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.485+0000 7fa4e2c0c740 1 --2- 172.21.15.114:0/1695900646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a73060b30 0x564a73062f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.485+0000 7fa4e2c0c740 1 --2- 172.21.15.114:0/1695900646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a73056290 0x564a730605f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.485+0000 7fa4e2c0c740 1 --2- 172.21.15.114:0/1695900646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a73055970 0x564a73055d50 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.485+0000 7fa4e2c0c740 1 -- 172.21.15.114:0/1695900646 >> 172.21.15.114:0/1695900646 conn(0x564a73051ee0 msgr2=0x564a730522c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:15.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.485+0000 7fa4de486640 1 --2- 172.21.15.114:0/1139419027 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa4c40a1410 0x7fa4c40a38d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa4d004b000 tx=0x7fa4d0046000 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:15.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.485+0000 7fa4e2c0c740 1 -- 172.21.15.114:0/1695900646 shutdown_connections 2024-09-13T18:28:15.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.485+0000 7fa4e2c0c740 1 -- 172.21.15.114:0/1695900646 wait complete. 2024-09-13T18:28:15.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.486+0000 7fa4e2c0c740 1 -- start start 2024-09-13T18:28:15.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.486+0000 7fa4e2c0c740 1 Processor -- start 2024-09-13T18:28:15.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.486+0000 7fa4e2c0c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a73055970 0x564a7305fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.487+0000 7fa4e2c0c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a73056290 0x564a73060530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.487+0000 7fa4e2c0c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a73060b30 0x564a7305d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.487+0000 7fa4e2c0c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a72f77180 con 0x564a73060b30 2024-09-13T18:28:15.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.487+0000 7fa4e2c0c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a72c91430 con 0x564a73056290 2024-09-13T18:28:15.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.487+0000 7fa4e2c0c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a72f3c5e0 con 0x564a73055970 2024-09-13T18:28:15.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.487+0000 7fa4e2c0c740 10 client.0 _ll_register_callbacks cb 0x564a72fa8aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:28:15.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.488+0000 7fa4de486640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a73060b30 0x564a7305d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:15.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.488+0000 7fa4de486640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a73060b30 0x564a7305d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55574/0 (socket says 172.21.15.114:55574) 2024-09-13T18:28:15.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.488+0000 7fa4de486640 1 -- 172.21.15.114:0/1546051750 learned_addr learned my addr 172.21.15.114:0/1546051750 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:15.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.488+0000 7fa4de486640 1 -- 172.21.15.114:0/1546051750 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a73055970 msgr2=0x564a7305fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:15.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.488+0000 7fa4de486640 1 --2- 172.21.15.114:0/1546051750 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a73055970 0x564a7305fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.488+0000 7fa4de486640 1 -- 172.21.15.114:0/1546051750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a73056290 msgr2=0x564a73060530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:15.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.488+0000 7fa4de486640 1 --2- 172.21.15.114:0/1546051750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a73056290 0x564a73060530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.488+0000 7fa4de486640 1 -- 172.21.15.114:0/1546051750 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4d811c170 con 0x564a73060b30 2024-09-13T18:28:15.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.489+0000 7fa4de486640 1 --2- 172.21.15.114:0/1546051750 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a73060b30 0x564a7305d430 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0034040 tx=0x7fa4d0031930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6a2c04ed377707d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:15.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.489+0000 7fa4c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:28:15.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.492+0000 7fa4c17fa640 1 -- 172.21.15.114:0/1546051750 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d0057070 con 0x564a73060b30 2024-09-13T18:28:15.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.492+0000 7fa4c17fa640 1 -- 172.21.15.114:0/1546051750 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fa4d0004250 con 0x564a73060b30 2024-09-13T18:28:15.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.492+0000 7fa4e2c0c740 1 -- 172.21.15.114:0/1546051750 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564a72fe2b80 con 0x564a73060b30 2024-09-13T18:28:15.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.497+0000 7fa4c17fa640 1 -- 172.21.15.114:0/1546051750 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4d0004410 con 0x564a73060b30 2024-09-13T18:28:15.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.497+0000 7fa4e2c0c740 10 client.17373 fetch_fsmap learned FSMap version 34 2024-09-13T18:28:15.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.497+0000 7fa4e2c0c740 1 -- 172.21.15.114:0/1546051750 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x564a72fb73c0 con 0x564a73060b30 2024-09-13T18:28:15.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.500+0000 7fa4c17fa640 1 -- 172.21.15.114:0/1546051750 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa4d0004570 con 0x564a73060b30 2024-09-13T18:28:15.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.500+0000 7fa4c17fa640 1 -- 172.21.15.114:0/1546051750 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa4d003b040 con 0x564a73060b30 2024-09-13T18:28:15.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.501+0000 7fa4e2c0c740 10 client.17373 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:28:15.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.501+0000 7fa4e2c0c740 10 client.17373 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-13T18:28:15.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.501+0000 7fa4e2c0c740 20 client.17373 populate_metadata read hostname 'smithi114' 2024-09-13T18:28:15.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.501+0000 7fa4e2c0c740 1 --2- 172.21.15.114:0/1546051750 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x564a73070130 0x564a73072550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.501+0000 7fa4e2c0c740 4 client.17373 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:28:15.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.501+0000 7fa4e2c0c740 1 -- 172.21.15.114:0/1546051750 --> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564a72fe2b80 con 0x564a73070130 2024-09-13T18:28:15.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.504+0000 7fa4ddc85640 1 --2- 172.21.15.114:0/1546051750 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x564a73070130 0x564a73072550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:15.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.510+0000 7fa4ddc85640 1 --2- 172.21.15.114:0/1546051750 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x564a73070130 0x564a73072550 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b9137280f339e752 server_cookie=c0f5cb7d8e316132 in_seq=0 out_seq=0 2024-09-13T18:28:15.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.511+0000 7fa4c17fa640 10 client.17373 ms_handle_connect on v2:172.21.15.134:6836/891692210 2024-09-13T18:28:15.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.519+0000 7fa4c17fa640 1 -- 172.21.15.114:0/1546051750 <== mds.1 v2:172.21.15.134:6836/891692210 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x564a72fe2b80 con 0x564a73070130 2024-09-13T18:28:15.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.519+0000 7fa4c17fa640 10 client.17373 handle_command_reply: tid=0 2024-09-13T18:28:15.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.519+0000 7fa4e2c0c740 1 client.17373 shutdown 2024-09-13T18:28:15.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.520+0000 7fa4e2c0c740 1 -- 172.21.15.114:0/1546051750 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a73060b30 msgr2=0x564a7305d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:15.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.520+0000 7fa4e2c0c740 1 --2- 172.21.15.114:0/1546051750 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a73060b30 0x564a7305d430 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0034040 tx=0x7fa4d0031930 comp rx=0 tx=0).stop 2024-09-13T18:28:15.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.521+0000 7fa4e2c0c740 1 -- 172.21.15.114:0/1546051750 shutdown_connections 2024-09-13T18:28:15.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.522+0000 7fa4e2c0c740 1 --2- 172.21.15.114:0/1546051750 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x564a73070130 0x564a73072550 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.522+0000 7fa4c17fa640 0 client.17373 ms_handle_reset on v2:172.21.15.134:6836/891692210 2024-09-13T18:28:15.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.522+0000 7fa4e2c0c740 1 --2- 172.21.15.114:0/1546051750 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a73060b30 0x564a7305d430 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.522+0000 7fa4e2c0c740 1 --2- 172.21.15.114:0/1546051750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a73056290 0x564a73060530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.522+0000 7fa4e2c0c740 1 --2- 172.21.15.114:0/1546051750 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a73055970 0x564a7305fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.522+0000 7fa4e2c0c740 1 -- 172.21.15.114:0/1546051750 >> 172.21.15.114:0/1546051750 conn(0x564a73051ee0 msgr2=0x564a73051230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:15.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.522+0000 7fa4e2c0c740 1 -- 172.21.15.114:0/1546051750 shutdown_connections 2024-09-13T18:28:15.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.523+0000 7fa4e2c0c740 1 -- 172.21.15.114:0/1546051750 wait complete. 2024-09-13T18:28:15.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.524+0000 7fa4e2c0c740 20 client.17373 trim_cache size 0 max 16384 2024-09-13T18:28:15.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.563+0000 7fa4e2c0c740 1 Processor -- start 2024-09-13T18:28:15.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.563+0000 7fa4e2c0c740 1 -- start start 2024-09-13T18:28:15.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.564+0000 7fa4e2c0c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a73055970 0x564a73067900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.564+0000 7fa4e2c0c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a73056290 0x564a73067e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.564+0000 7fa4e2c0c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a73060b30 0x564a7305bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.564+0000 7fa4ddc85640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a73055970 0x564a73067900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:15.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.564+0000 7fa4ddc85640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a73055970 0x564a73067900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:38922/0 (socket says 172.21.15.114:38922) 2024-09-13T18:28:15.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.564+0000 7fa4ddc85640 1 -- 172.21.15.114:0/285382841 learned_addr learned my addr 172.21.15.114:0/285382841 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:15.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.564+0000 7fa4e2c0c740 1 -- 172.21.15.114:0/285382841 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a72fe2b80 con 0x564a73056290 2024-09-13T18:28:15.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.564+0000 7fa4de486640 1 --2- 172.21.15.114:0/285382841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a73060b30 0x564a7305bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:15.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.564+0000 7fa4e2c0c740 1 -- 172.21.15.114:0/285382841 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a73056930 con 0x564a73060b30 2024-09-13T18:28:15.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.564+0000 7fa4e2c0c740 1 -- 172.21.15.114:0/285382841 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a7304dd00 con 0x564a73055970 2024-09-13T18:28:15.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.565+0000 7fa4dd484640 1 --2- 172.21.15.114:0/285382841 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a73056290 0x564a73067e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:15.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.565+0000 7fa4ddc85640 1 -- 172.21.15.114:0/285382841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a73060b30 msgr2=0x564a7305bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:15.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.565+0000 7fa4ddc85640 1 --2- 172.21.15.114:0/285382841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a73060b30 0x564a7305bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.565+0000 7fa4ddc85640 1 -- 172.21.15.114:0/285382841 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a73056290 msgr2=0x564a73067e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:15.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.565+0000 7fa4ddc85640 1 --2- 172.21.15.114:0/285382841 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a73056290 0x564a73067e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.565+0000 7fa4ddc85640 1 -- 172.21.15.114:0/285382841 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a72f59770 con 0x564a73055970 2024-09-13T18:28:15.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.566+0000 7fa4ddc85640 1 --2- 172.21.15.114:0/285382841 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a73055970 0x564a73067900 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x564a73052d40 tx=0x7fa4c800bdc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5ae434810d20f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:15.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.568+0000 7fa4ccff9640 1 -- 172.21.15.114:0/285382841 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c80175b0 con 0x564a73055970 2024-09-13T18:28:15.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.568+0000 7fa4ccff9640 1 -- 172.21.15.114:0/285382841 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa4c8015d40 con 0x564a73055970 2024-09-13T18:28:15.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.568+0000 7fa4ccff9640 1 -- 172.21.15.114:0/285382841 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c8016880 con 0x564a73055970 2024-09-13T18:28:15.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.569+0000 7fa4e2c0c740 1 -- 172.21.15.114:0/285382841 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a73055970 msgr2=0x564a73067900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:15.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.569+0000 7fa4e2c0c740 1 --2- 172.21.15.114:0/285382841 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a73055970 0x564a73067900 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x564a73052d40 tx=0x7fa4c800bdc0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.575+0000 7fa4e2c0c740 1 -- 172.21.15.114:0/285382841 shutdown_connections 2024-09-13T18:28:15.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.575+0000 7fa4e2c0c740 1 --2- 172.21.15.114:0/285382841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a73060b30 0x564a7305bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.575+0000 7fa4e2c0c740 1 --2- 172.21.15.114:0/285382841 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a73056290 0x564a73067e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.575+0000 7fa4e2c0c740 1 --2- 172.21.15.114:0/285382841 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a73055970 0x564a73067900 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.575+0000 7fa4e2c0c740 1 -- 172.21.15.114:0/285382841 >> 172.21.15.114:0/285382841 conn(0x564a73051ee0 msgr2=0x564a730522c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:15.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.584+0000 7fa4e2c0c740 1 -- 172.21.15.114:0/285382841 shutdown_connections 2024-09-13T18:28:15.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.589+0000 7fa4e2c0c740 1 -- 172.21.15.114:0/285382841 wait complete. 2024-09-13T18:28:15.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.589+0000 7fa4e2c0c740 1 -- start start 2024-09-13T18:28:15.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.590+0000 7fa4e2c0c740 1 Processor -- start 2024-09-13T18:28:15.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.590+0000 7fa4e2c0c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a73055970 0x564a73066690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.590+0000 7fa4e2c0c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a73056290 0x564a73066bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.591+0000 7fa4e2c0c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a73060b30 0x564a73067130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.591+0000 7fa4e2c0c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a730c6820 con 0x564a73056290 2024-09-13T18:28:15.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.591+0000 7fa4e2c0c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a730c6990 con 0x564a73060b30 2024-09-13T18:28:15.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.591+0000 7fa4de486640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a73060b30 0x564a73067130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:15.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.591+0000 7fa4de486640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a73060b30 0x564a73067130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:36268/0 (socket says 172.21.15.114:36268) 2024-09-13T18:28:15.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.591+0000 7fa4de486640 1 -- 172.21.15.114:0/2172240472 learned_addr learned my addr 172.21.15.114:0/2172240472 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:15.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.591+0000 7fa4e2c0c740 1 -- 172.21.15.114:0/2172240472 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a730c6b50 con 0x564a73055970 2024-09-13T18:28:15.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.591+0000 7fa4ddc85640 1 --2- 172.21.15.114:0/2172240472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a73055970 0x564a73066690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:15.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.591+0000 7fa4e2c0c740 10 client.0 _ll_register_callbacks cb 0x7fa4d0009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:28:15.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.593+0000 7fa4de486640 1 -- 172.21.15.114:0/2172240472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a73055970 msgr2=0x564a73066690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:15.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.593+0000 7fa4de486640 1 --2- 172.21.15.114:0/2172240472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a73055970 0x564a73066690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.593+0000 7fa4de486640 1 -- 172.21.15.114:0/2172240472 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a73056290 msgr2=0x564a73066bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:15.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.593+0000 7fa4de486640 1 --2- 172.21.15.114:0/2172240472 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a73056290 0x564a73066bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.593+0000 7fa4de486640 1 -- 172.21.15.114:0/2172240472 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4c800e040 con 0x564a73060b30 2024-09-13T18:28:15.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.593+0000 7fa4de486640 1 --2- 172.21.15.114:0/2172240472 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a73060b30 0x564a73067130 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0009b30 tx=0x7fa4d0003f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7e4e22927e81ada server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:15.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.595+0000 7fa4c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:28:15.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.597+0000 7fa4c17fa640 1 -- 172.21.15.114:0/2172240472 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d0057070 con 0x564a73060b30 2024-09-13T18:28:15.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.597+0000 7fa4c17fa640 1 -- 172.21.15.114:0/2172240472 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fa4d0040480 con 0x564a73060b30 2024-09-13T18:28:15.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.598+0000 7fa4e2c0c740 1 -- 172.21.15.114:0/2172240472 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa4d0004830 con 0x564a73060b30 2024-09-13T18:28:15.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.600+0000 7fa4c17fa640 1 -- 172.21.15.114:0/2172240472 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4d0009480 con 0x564a73060b30 2024-09-13T18:28:15.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.601+0000 7fa4e2c0c740 10 client.26792 fetch_fsmap learned FSMap version 34 2024-09-13T18:28:15.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.601+0000 7fa4e2c0c740 1 -- 172.21.15.114:0/2172240472 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x564a72f7ee80 con 0x564a73060b30 2024-09-13T18:28:15.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.602+0000 7fa4c17fa640 1 -- 172.21.15.114:0/2172240472 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa4d0004830 con 0x564a73060b30 2024-09-13T18:28:15.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.603+0000 7fa4c17fa640 1 -- 172.21.15.114:0/2172240472 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa4d006c050 con 0x564a73060b30 2024-09-13T18:28:15.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.603+0000 7fa4e2c0c740 10 client.26792 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:28:15.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.603+0000 7fa4e2c0c740 10 client.26792 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-13T18:28:15.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.603+0000 7fa4e2c0c740 20 client.26792 populate_metadata read hostname 'smithi114' 2024-09-13T18:28:15.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.603+0000 7fa4e2c0c740 1 --2- 172.21.15.114:0/2172240472 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x564a73070130 0x564a730c9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.603+0000 7fa4e2c0c740 4 client.26792 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:28:15.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.603+0000 7fa4e2c0c740 1 -- 172.21.15.114:0/2172240472 --> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564a72fab1f0 con 0x564a73070130 2024-09-13T18:28:15.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.609+0000 7fa4ddc85640 1 --2- 172.21.15.114:0/2172240472 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x564a73070130 0x564a730c9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:15.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.610+0000 7fa4ddc85640 1 --2- 172.21.15.114:0/2172240472 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x564a73070130 0x564a730c9e50 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e21513fba355333d server_cookie=37e67f951beadcb5 in_seq=0 out_seq=0 2024-09-13T18:28:15.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.612+0000 7fa4c17fa640 10 client.26792 ms_handle_connect on v2:172.21.15.134:6836/891692210 2024-09-13T18:28:15.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.612+0000 7fa4c17fa640 1 -- 172.21.15.114:0/2172240472 <== mds.1 v2:172.21.15.134:6836/891692210 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564a72fab1f0 con 0x564a73070130 2024-09-13T18:28:15.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.612+0000 7fa4c17fa640 10 client.26792 handle_command_reply: tid=0 2024-09-13T18:28:15.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.612+0000 7fa4e2c0c740 1 client.26792 shutdown 2024-09-13T18:28:15.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.614+0000 7fa4e2c0c740 1 -- 172.21.15.114:0/2172240472 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a73060b30 msgr2=0x564a73067130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:15.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.614+0000 7fa4e2c0c740 1 --2- 172.21.15.114:0/2172240472 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a73060b30 0x564a73067130 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0009b30 tx=0x7fa4d0003f80 comp rx=0 tx=0).stop 2024-09-13T18:28:15.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.621+0000 7fa4e2c0c740 1 -- 172.21.15.114:0/2172240472 shutdown_connections 2024-09-13T18:28:15.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.621+0000 7fa4e2c0c740 1 --2- 172.21.15.114:0/2172240472 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x564a73070130 0x564a730c9e50 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.621+0000 7fa4e2c0c740 1 --2- 172.21.15.114:0/2172240472 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a73060b30 0x564a73067130 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.621+0000 7fa4e2c0c740 1 --2- 172.21.15.114:0/2172240472 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a73056290 0x564a73066bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.621+0000 7fa4e2c0c740 1 --2- 172.21.15.114:0/2172240472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a73055970 0x564a73066690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.621+0000 7fa4e2c0c740 1 -- 172.21.15.114:0/2172240472 >> 172.21.15.114:0/2172240472 conn(0x564a73051ee0 msgr2=0x564a7305bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:15.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.625+0000 7fa4c17fa640 0 client.26792 ms_handle_reset on v2:172.21.15.134:6836/891692210 2024-09-13T18:28:15.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.625+0000 7fa4e2c0c740 1 -- 172.21.15.114:0/2172240472 shutdown_connections 2024-09-13T18:28:15.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.625+0000 7fa4e2c0c740 1 -- 172.21.15.114:0/2172240472 wait complete. 2024-09-13T18:28:15.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.626+0000 7fa4e2c0c740 20 client.26792 trim_cache size 0 max 16384 2024-09-13T18:28:15.628 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:28:15.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.627+0000 7fa4dec87640 1 -- 172.21.15.114:0/1139419027 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa4c40a1410 msgr2=0x7fa4c40a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:15.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.627+0000 7fa4dec87640 1 --2- 172.21.15.114:0/1139419027 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa4c40a1410 0x7fa4c40a38d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa4d004b000 tx=0x7fa4d0046000 comp rx=0 tx=0).stop 2024-09-13T18:28:15.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.627+0000 7fa4dec87640 1 -- 172.21.15.114:0/1139419027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4d8107890 msgr2=0x7fa4d8116e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:15.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.627+0000 7fa4dec87640 1 --2- 172.21.15.114:0/1139419027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4d8107890 0x7fa4d8116e70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa4d400d950 tx=0x7fa4d400de10 comp rx=0 tx=0).stop 2024-09-13T18:28:15.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.627+0000 7fa4dec87640 1 -- 172.21.15.114:0/1139419027 shutdown_connections 2024-09-13T18:28:15.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.627+0000 7fa4dec87640 1 --2- 172.21.15.114:0/1139419027 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa4c40a1410 0x7fa4c40a38d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.627+0000 7fa4dec87640 1 --2- 172.21.15.114:0/1139419027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4d8119400 0x7fa4d810fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.627+0000 7fa4dec87640 1 --2- 172.21.15.114:0/1139419027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4d8107890 0x7fa4d8116e70 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.627+0000 7fa4dec87640 1 --2- 172.21.15.114:0/1139419027 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4d81074b0 0x7fa4d8116910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.627+0000 7fa4dec87640 1 -- 172.21.15.114:0/1139419027 >> 172.21.15.114:0/1139419027 conn(0x7fa4d806af40 msgr2=0x7fa4d8119dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:15.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.629+0000 7fa4dec87640 1 -- 172.21.15.114:0/1139419027 shutdown_connections 2024-09-13T18:28:15.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.629+0000 7fa4dec87640 1 -- 172.21.15.114:0/1139419027 wait complete. 2024-09-13T18:28:15.653 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-09-13T18:28:15.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:15 smithi134 ceph-mon[34283]: pgmap v176: 129 pgs: 129 active+clean; 18 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 39 MiB/s wr, 748 op/s 2024-09-13T18:28:15.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.904+0000 7f474b58b640 1 -- 172.21.15.114:0/238523465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f473c0b74e0 msgr2=0x7f473c0b98d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:15.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.904+0000 7f474b58b640 1 --2- 172.21.15.114:0/238523465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f473c0b74e0 0x7f473c0b98d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4734009860 tx=0x7f473402a340 comp rx=0 tx=0).stop 2024-09-13T18:28:15.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.904+0000 7f474b58b640 1 -- 172.21.15.114:0/238523465 shutdown_connections 2024-09-13T18:28:15.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.904+0000 7f474b58b640 1 --2- 172.21.15.114:0/238523465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f473c0b74e0 0x7f473c0b98d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.904+0000 7f474b58b640 1 --2- 172.21.15.114:0/238523465 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f473c0a50e0 0x7f473c0b6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.904+0000 7f474b58b640 1 --2- 172.21.15.114:0/238523465 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f473c0a47c0 0x7f473c0a4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.904+0000 7f474b58b640 1 -- 172.21.15.114:0/238523465 >> 172.21.15.114:0/238523465 conn(0x7f473c01a0a0 msgr2=0x7f473c01a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:15.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.906+0000 7f474b58b640 1 -- 172.21.15.114:0/238523465 shutdown_connections 2024-09-13T18:28:15.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.906+0000 7f474b58b640 1 -- 172.21.15.114:0/238523465 wait complete. 2024-09-13T18:28:15.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.907+0000 7f474b58b640 1 Processor -- start 2024-09-13T18:28:15.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.907+0000 7f474b58b640 1 -- start start 2024-09-13T18:28:15.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.907+0000 7f474b58b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f473c0a47c0 0x7f473c142e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.908+0000 7f474b58b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f473c0a50e0 0x7f473c143390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.908+0000 7f474b58b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f473c0b74e0 0x7f473c147780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.908+0000 7f474b58b640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473c147d80 con 0x7f473c0b74e0 2024-09-13T18:28:15.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.908+0000 7f474b58b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473c147ef0 con 0x7f473c0a47c0 2024-09-13T18:28:15.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.908+0000 7f474b58b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473c1480b0 con 0x7f473c0a50e0 2024-09-13T18:28:15.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.908+0000 7f474a589640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f473c0a47c0 0x7f473c142e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:15.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.908+0000 7f474a589640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f473c0a47c0 0x7f473c142e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:36286/0 (socket says 172.21.15.114:36286) 2024-09-13T18:28:15.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.908+0000 7f474a589640 1 -- 172.21.15.114:0/3864464576 learned_addr learned my addr 172.21.15.114:0/3864464576 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:15.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.908+0000 7f4749d88640 1 --2- 172.21.15.114:0/3864464576 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f473c0a50e0 0x7f473c143390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:15.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.909+0000 7f474a589640 1 -- 172.21.15.114:0/3864464576 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f473c0a50e0 msgr2=0x7f473c143390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:15.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.909+0000 7f474a589640 1 --2- 172.21.15.114:0/3864464576 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f473c0a50e0 0x7f473c143390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.909+0000 7f474a589640 1 -- 172.21.15.114:0/3864464576 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f473c0b74e0 msgr2=0x7f473c147780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:15.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.909+0000 7f474a589640 1 --2- 172.21.15.114:0/3864464576 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f473c0b74e0 0x7f473c147780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.909+0000 7f474a589640 1 -- 172.21.15.114:0/3864464576 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f474404eac0 con 0x7f473c0a47c0 2024-09-13T18:28:15.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.910+0000 7f474a589640 1 --2- 172.21.15.114:0/3864464576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f473c0a47c0 0x7f473c142e50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f47440639a0 tx=0x7f474406f6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:15.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.911+0000 7f473b7fe640 1 -- 172.21.15.114:0/3864464576 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4744066ac0 con 0x7f473c0a47c0 2024-09-13T18:28:15.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.911+0000 7f473b7fe640 1 -- 172.21.15.114:0/3864464576 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4744071e70 con 0x7f473c0a47c0 2024-09-13T18:28:15.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.912+0000 7f473b7fe640 1 -- 172.21.15.114:0/3864464576 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4744073690 con 0x7f473c0a47c0 2024-09-13T18:28:15.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.912+0000 7f474b58b640 1 -- 172.21.15.114:0/3864464576 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4734009490 con 0x7f473c0a47c0 2024-09-13T18:28:15.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.912+0000 7f474b58b640 1 -- 172.21.15.114:0/3864464576 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f473c14fa30 con 0x7f473c0a47c0 2024-09-13T18:28:15.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.916+0000 7f474f5ea740 1 Processor -- start 2024-09-13T18:28:15.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.920+0000 7f474f5ea740 1 -- start start 2024-09-13T18:28:15.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.920+0000 7f474f5ea740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b34746970 0x562b34746d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.920+0000 7f474f5ea740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b34747290 0x562b347515f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.920+0000 7f474f5ea740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b34751b30 0x562b34753f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.920+0000 7f474f5ea740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b346b0cc0 con 0x562b34751b30 2024-09-13T18:28:15.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.920+0000 7f474f5ea740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b346ae450 con 0x562b34746970 2024-09-13T18:28:15.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.920+0000 7f474f5ea740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b346a9d40 con 0x562b34747290 2024-09-13T18:28:15.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.921+0000 7f474ad8a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b34746970 0x562b34746d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:15.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.921+0000 7f474ad8a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b34746970 0x562b34746d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:36296/0 (socket says 172.21.15.114:36296) 2024-09-13T18:28:15.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.921+0000 7f474ad8a640 1 -- 172.21.15.114:0/3102418736 learned_addr learned my addr 172.21.15.114:0/3102418736 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:15.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.921+0000 7f474a589640 1 --2- 172.21.15.114:0/3102418736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b34747290 0x562b347515f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:15.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.921+0000 7f4749d88640 1 --2- 172.21.15.114:0/3102418736 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b34751b30 0x562b34753f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:15.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.921+0000 7f474ad8a640 1 -- 172.21.15.114:0/3102418736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b34747290 msgr2=0x562b347515f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:15.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.921+0000 7f474ad8a640 1 --2- 172.21.15.114:0/3102418736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b34747290 0x562b347515f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.921+0000 7f474ad8a640 1 -- 172.21.15.114:0/3102418736 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b34751b30 msgr2=0x562b34753f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:15.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.921+0000 7f474ad8a640 1 --2- 172.21.15.114:0/3102418736 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b34751b30 0x562b34753f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.921+0000 7f474ad8a640 1 -- 172.21.15.114:0/3102418736 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b34671620 con 0x562b34746970 2024-09-13T18:28:15.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.921+0000 7f474ad8a640 1 --2- 172.21.15.114:0/3102418736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b34746970 0x562b34746d50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f473402a860 tx=0x7f4734005d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=848e1ce089dd8de8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:15.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.922+0000 7f4738ff9640 1 -- 172.21.15.114:0/3102418736 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473400b0d0 con 0x562b34746970 2024-09-13T18:28:15.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.922+0000 7f473b7fe640 1 -- 172.21.15.114:0/3864464576 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f47440680d0 con 0x7f473c0a47c0 2024-09-13T18:28:15.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.922+0000 7f4738ff9640 1 -- 172.21.15.114:0/3102418736 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4734032a40 con 0x562b34746970 2024-09-13T18:28:15.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.923+0000 7f473b7fe640 1 --2- 172.21.15.114:0/3864464576 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f47300a1230 0x7f47300a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.923+0000 7f474f5ea740 1 -- 172.21.15.114:0/3102418736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b34746970 msgr2=0x562b34746d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:15.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.923+0000 7f474f5ea740 1 --2- 172.21.15.114:0/3102418736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b34746970 0x562b34746d50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f473402a860 tx=0x7f4734005d40 comp rx=0 tx=0).stop 2024-09-13T18:28:15.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.923+0000 7f474f5ea740 1 -- 172.21.15.114:0/3102418736 shutdown_connections 2024-09-13T18:28:15.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.923+0000 7f474f5ea740 1 --2- 172.21.15.114:0/3102418736 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b34751b30 0x562b34753f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.923+0000 7f474f5ea740 1 --2- 172.21.15.114:0/3102418736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b34747290 0x562b347515f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.923+0000 7f473b7fe640 1 -- 172.21.15.114:0/3864464576 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f474412f1a0 con 0x7f473c0a47c0 2024-09-13T18:28:15.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.923+0000 7f474f5ea740 1 --2- 172.21.15.114:0/3102418736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b34746970 0x562b34746d50 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.923+0000 7f474f5ea740 1 -- 172.21.15.114:0/3102418736 >> 172.21.15.114:0/3102418736 conn(0x562b34742ee0 msgr2=0x562b347432c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:15.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.926+0000 7f474f5ea740 1 -- 172.21.15.114:0/3102418736 shutdown_connections 2024-09-13T18:28:15.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.927+0000 7f474a589640 1 --2- 172.21.15.114:0/3864464576 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f47300a1230 0x7f47300a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:15.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.929+0000 7f474f5ea740 1 -- 172.21.15.114:0/3102418736 wait complete. 2024-09-13T18:28:15.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.930+0000 7f474f5ea740 1 -- start start 2024-09-13T18:28:15.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.930+0000 7f474f5ea740 1 Processor -- start 2024-09-13T18:28:15.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.930+0000 7f474f5ea740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b34746970 0x562b3474e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.930+0000 7f474f5ea740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b34747290 0x562b3474ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.930+0000 7f474f5ea740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b34751b30 0x562b34755990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.930+0000 7f474f5ea740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b34668180 con 0x562b34746970 2024-09-13T18:28:15.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.930+0000 7f474f5ea740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b34382430 con 0x562b34751b30 2024-09-13T18:28:15.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.930+0000 7f474f5ea740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b3462d5e0 con 0x562b34747290 2024-09-13T18:28:15.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.930+0000 7f474f5ea740 10 client.0 _ll_register_callbacks cb 0x562b34699aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:28:15.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.931+0000 7f474ad8a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b34746970 0x562b3474e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:15.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.931+0000 7f474ad8a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b34746970 0x562b3474e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55612/0 (socket says 172.21.15.114:55612) 2024-09-13T18:28:15.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.931+0000 7f474ad8a640 1 -- 172.21.15.114:0/2191964962 learned_addr learned my addr 172.21.15.114:0/2191964962 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:15.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.931+0000 7f474ad8a640 1 --2- 172.21.15.114:0/2191964962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b34751b30 0x562b34755990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:15.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.931+0000 7f474ad8a640 1 -- 172.21.15.114:0/2191964962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b34747290 msgr2=0x562b3474ea80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:28:15.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.931+0000 7f474ad8a640 1 --2- 172.21.15.114:0/2191964962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b34747290 0x562b3474ea80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.931+0000 7f474ad8a640 1 -- 172.21.15.114:0/2191964962 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b34746970 msgr2=0x562b3474e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:15.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.931+0000 7f474ad8a640 1 --2- 172.21.15.114:0/2191964962 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b34746970 0x562b3474e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.931+0000 7f474ad8a640 1 -- 172.21.15.114:0/2191964962 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f473c0ba300 con 0x562b34751b30 2024-09-13T18:28:15.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.932+0000 7f474ad8a640 1 --2- 172.21.15.114:0/2191964962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b34751b30 0x562b34755990 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f47340057c0 tx=0x7f4734030740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d12839f38b4bc1eb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:15.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.934+0000 7f472dffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:28:15.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.934+0000 7f472dffb640 1 -- 172.21.15.114:0/2191964962 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47340308e0 con 0x562b34751b30 2024-09-13T18:28:15.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.934+0000 7f472dffb640 1 -- 172.21.15.114:0/2191964962 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f4734030d70 con 0x562b34751b30 2024-09-13T18:28:15.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.934+0000 7f474f5ea740 1 -- 172.21.15.114:0/2191964962 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562b346d3b80 con 0x562b34751b30 2024-09-13T18:28:15.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.935+0000 7f472dffb640 1 -- 172.21.15.114:0/2191964962 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4734005020 con 0x562b34751b30 2024-09-13T18:28:15.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.935+0000 7f474f5ea740 10 client.26804 fetch_fsmap learned FSMap version 34 2024-09-13T18:28:15.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.935+0000 7f474f5ea740 1 -- 172.21.15.114:0/2191964962 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x562b346a83c0 con 0x562b34751b30 2024-09-13T18:28:15.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.939+0000 7f472dffb640 1 -- 172.21.15.114:0/2191964962 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4734051070 con 0x562b34751b30 2024-09-13T18:28:15.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.939+0000 7f472dffb640 1 -- 172.21.15.114:0/2191964962 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4734004bb0 con 0x562b34751b30 2024-09-13T18:28:15.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.940+0000 7f474f5ea740 10 client.26804 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:28:15.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.940+0000 7f474f5ea740 10 client.26804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:28:15.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.940+0000 7f474f5ea740 20 client.26804 populate_metadata read hostname 'smithi114' 2024-09-13T18:28:15.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.940+0000 7f474f5ea740 1 --2- 172.21.15.114:0/2191964962 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x562b34761130 0x562b34763550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.940+0000 7f474f5ea740 4 client.26804 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:28:15.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.940+0000 7f474f5ea740 1 -- 172.21.15.114:0/2191964962 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562b3469c1f0 con 0x562b34761130 2024-09-13T18:28:15.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.941+0000 7f4749d88640 1 --2- 172.21.15.114:0/2191964962 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x562b34761130 0x562b34763550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:15.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:15 smithi204 ceph-mon[34777]: pgmap v176: 129 pgs: 129 active+clean; 18 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 39 MiB/s wr, 748 op/s 2024-09-13T18:28:15.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.945+0000 7f4749d88640 1 --2- 172.21.15.114:0/2191964962 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x562b34761130 0x562b34763550 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f03aacbef75f74c3 server_cookie=c45ab2c3abb510e9 in_seq=0 out_seq=0 2024-09-13T18:28:15.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.945+0000 7f472dffb640 10 client.26804 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:28:15.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.947+0000 7f472dffb640 1 -- 172.21.15.114:0/2191964962 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x562b3469c1f0 con 0x562b34761130 2024-09-13T18:28:15.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.947+0000 7f472dffb640 10 client.26804 handle_command_reply: tid=0 2024-09-13T18:28:15.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.948+0000 7f474f5ea740 1 client.26804 shutdown 2024-09-13T18:28:15.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.948+0000 7f474f5ea740 1 -- 172.21.15.114:0/2191964962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b34751b30 msgr2=0x562b34755990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:15.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.948+0000 7f474f5ea740 1 --2- 172.21.15.114:0/2191964962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b34751b30 0x562b34755990 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f47340057c0 tx=0x7f4734030740 comp rx=0 tx=0).stop 2024-09-13T18:28:15.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.948+0000 7f474f5ea740 1 -- 172.21.15.114:0/2191964962 shutdown_connections 2024-09-13T18:28:15.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.948+0000 7f474f5ea740 1 --2- 172.21.15.114:0/2191964962 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x562b34761130 0x562b34763550 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.948+0000 7f474f5ea740 1 --2- 172.21.15.114:0/2191964962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b34751b30 0x562b34755990 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.948+0000 7f474f5ea740 1 --2- 172.21.15.114:0/2191964962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b34747290 0x562b3474ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.948+0000 7f474f5ea740 1 --2- 172.21.15.114:0/2191964962 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b34746970 0x562b3474e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.948+0000 7f474f5ea740 1 -- 172.21.15.114:0/2191964962 >> 172.21.15.114:0/2191964962 conn(0x562b34742ee0 msgr2=0x562b34742230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:15.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.949+0000 7f474a589640 1 --2- 172.21.15.114:0/3864464576 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f47300a1230 0x7f47300a36f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x562b34743d40 tx=0x7f4744132c50 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:15.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.949+0000 7f472dffb640 0 client.26804 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:28:15.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.950+0000 7f474f5ea740 1 -- 172.21.15.114:0/2191964962 shutdown_connections 2024-09-13T18:28:15.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.950+0000 7f474f5ea740 1 -- 172.21.15.114:0/2191964962 wait complete. 2024-09-13T18:28:15.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.955+0000 7f474f5ea740 20 client.26804 trim_cache size 0 max 16384 2024-09-13T18:28:15.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:15 smithi114 ceph-mon[30897]: pgmap v176: 129 pgs: 129 active+clean; 18 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 39 MiB/s wr, 748 op/s 2024-09-13T18:28:15.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.986+0000 7f474f5ea740 1 Processor -- start 2024-09-13T18:28:15.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.986+0000 7f474f5ea740 1 -- start start 2024-09-13T18:28:15.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.987+0000 7f474f5ea740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b34746970 0x562b34755910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.987+0000 7f474f5ea740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b34747290 0x562b34749540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.987+0000 7f474ad8a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b34746970 0x562b34755910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:15.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.987+0000 7f474ad8a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b34746970 0x562b34755910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55618/0 (socket says 172.21.15.114:55618) 2024-09-13T18:28:15.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.987+0000 7f474ad8a640 1 -- 172.21.15.114:0/2829313836 learned_addr learned my addr 172.21.15.114:0/2829313836 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:15.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.987+0000 7f474f5ea740 1 --2- 172.21.15.114:0/2829313836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b34751b30 0x562b34749a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:15.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.987+0000 7f4749d88640 1 --2- 172.21.15.114:0/2829313836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b34747290 0x562b34749540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:15.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.988+0000 7f474f5ea740 1 -- 172.21.15.114:0/2829313836 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b34749fc0 con 0x562b34746970 2024-09-13T18:28:15.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.988+0000 7f474f5ea740 1 -- 172.21.15.114:0/2829313836 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b347478e0 con 0x562b34747290 2024-09-13T18:28:15.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.988+0000 7f474f5ea740 1 -- 172.21.15.114:0/2829313836 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b3474c880 con 0x562b34751b30 2024-09-13T18:28:15.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.988+0000 7f474ad8a640 1 --2- 172.21.15.114:0/2829313836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b34751b30 0x562b34749a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:15.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.988+0000 7f4749d88640 1 -- 172.21.15.114:0/2829313836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b34751b30 msgr2=0x562b34749a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:15.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.988+0000 7f4749d88640 1 --2- 172.21.15.114:0/2829313836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b34751b30 0x562b34749a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.988+0000 7f4749d88640 1 -- 172.21.15.114:0/2829313836 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b34746970 msgr2=0x562b34755910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:15.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.988+0000 7f4749d88640 1 --2- 172.21.15.114:0/2829313836 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b34746970 0x562b34755910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.988+0000 7f4749d88640 1 -- 172.21.15.114:0/2829313836 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b3464a770 con 0x562b34747290 2024-09-13T18:28:15.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.989+0000 7f4749d88640 1 --2- 172.21.15.114:0/2829313836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b34747290 0x562b34749540 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x562b3473ebb0 tx=0x7f474000b390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e42b01a37648dff1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:15.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.990+0000 7f47397fa640 1 -- 172.21.15.114:0/2829313836 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4740017040 con 0x562b34747290 2024-09-13T18:28:15.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.990+0000 7f47397fa640 1 -- 172.21.15.114:0/2829313836 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f47400054b0 con 0x562b34747290 2024-09-13T18:28:15.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.992+0000 7f474f5ea740 1 -- 172.21.15.114:0/2829313836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b34747290 msgr2=0x562b34749540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:15.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.992+0000 7f474f5ea740 1 --2- 172.21.15.114:0/2829313836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b34747290 0x562b34749540 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x562b3473ebb0 tx=0x7f474000b390 comp rx=0 tx=0).stop 2024-09-13T18:28:15.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.996+0000 7f474f5ea740 1 -- 172.21.15.114:0/2829313836 shutdown_connections 2024-09-13T18:28:15.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.996+0000 7f474f5ea740 1 --2- 172.21.15.114:0/2829313836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b34751b30 0x562b34749a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.996+0000 7f474f5ea740 1 --2- 172.21.15.114:0/2829313836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b34747290 0x562b34749540 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.996+0000 7f474f5ea740 1 --2- 172.21.15.114:0/2829313836 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b34746970 0x562b34755910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:15.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:15.996+0000 7f474f5ea740 1 -- 172.21.15.114:0/2829313836 >> 172.21.15.114:0/2829313836 conn(0x562b34742ee0 msgr2=0x562b347432c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:16.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.001+0000 7f474f5ea740 1 -- 172.21.15.114:0/2829313836 shutdown_connections 2024-09-13T18:28:16.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.001+0000 7f474f5ea740 1 -- 172.21.15.114:0/2829313836 wait complete. 2024-09-13T18:28:16.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.002+0000 7f474f5ea740 1 -- start start 2024-09-13T18:28:16.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.002+0000 7f474f5ea740 1 Processor -- start 2024-09-13T18:28:16.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.002+0000 7f474f5ea740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b34746970 0x562b3475d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:16.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.002+0000 7f474f5ea740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b34747290 0x562b3475d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:16.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.002+0000 7f474f5ea740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b34751b30 0x562b34759e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:16.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.002+0000 7f474f5ea740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b3475a500 con 0x562b34746970 2024-09-13T18:28:16.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.002+0000 7f474f5ea740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b3475a670 con 0x562b34751b30 2024-09-13T18:28:16.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.002+0000 7f474f5ea740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b3475a830 con 0x562b34747290 2024-09-13T18:28:16.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.002+0000 7f474f5ea740 10 client.0 _ll_register_callbacks cb 0x7f4734035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:28:16.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.003+0000 7f4749d88640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b34747290 0x562b3475d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:16.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.003+0000 7f4749d88640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b34747290 0x562b3475d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:38998/0 (socket says 172.21.15.114:38998) 2024-09-13T18:28:16.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.003+0000 7f4749d88640 1 -- 172.21.15.114:0/280106518 learned_addr learned my addr 172.21.15.114:0/280106518 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:16.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.003+0000 7f474ad8a640 1 --2- 172.21.15.114:0/280106518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b34751b30 0x562b34759e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:16.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.003+0000 7f4749d88640 1 -- 172.21.15.114:0/280106518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b34751b30 msgr2=0x562b34759e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:16.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.003+0000 7f4749d88640 1 --2- 172.21.15.114:0/280106518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b34751b30 0x562b34759e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:16.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.003+0000 7f4749d88640 1 -- 172.21.15.114:0/280106518 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b34746970 msgr2=0x562b3475d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:16.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.003+0000 7f4749d88640 1 --2- 172.21.15.114:0/280106518 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b34746970 0x562b3475d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:16.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.003+0000 7f4749d88640 1 -- 172.21.15.114:0/280106518 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4740004340 con 0x562b34747290 2024-09-13T18:28:16.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.004+0000 7f4749d88640 1 --2- 172.21.15.114:0/280106518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b34747290 0x562b3475d980 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f474000b5d0 tx=0x7f4740004830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18eed6666cc0c85b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:16.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.006+0000 7f472dffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:28:16.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.006+0000 7f472dffb640 1 -- 172.21.15.114:0/280106518 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47400192b0 con 0x562b34747290 2024-09-13T18:28:16.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.006+0000 7f474f5ea740 1 -- 172.21.15.114:0/280106518 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4734032a40 con 0x562b34747290 2024-09-13T18:28:16.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.006+0000 7f472dffb640 1 -- 172.21.15.114:0/280106518 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f4740019410 con 0x562b34747290 2024-09-13T18:28:16.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.007+0000 7f472dffb640 1 -- 172.21.15.114:0/280106518 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47400195d0 con 0x562b34747290 2024-09-13T18:28:16.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.007+0000 7f474f5ea740 10 client.26797 fetch_fsmap learned FSMap version 34 2024-09-13T18:28:16.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.007+0000 7f474f5ea740 1 -- 172.21.15.114:0/280106518 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x562b3466fe80 con 0x562b34747290 2024-09-13T18:28:16.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.008+0000 7f472dffb640 1 -- 172.21.15.114:0/280106518 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4734032a40 con 0x562b34747290 2024-09-13T18:28:16.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.008+0000 7f472dffb640 1 -- 172.21.15.114:0/280106518 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f47400037f0 con 0x562b34747290 2024-09-13T18:28:16.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.008+0000 7f474f5ea740 10 client.26797 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:28:16.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.008+0000 7f474f5ea740 10 client.26797 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:28:16.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.008+0000 7f474f5ea740 20 client.26797 populate_metadata read hostname 'smithi114' 2024-09-13T18:28:16.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.008+0000 7f474f5ea740 1 --2- 172.21.15.114:0/280106518 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x562b347b88e0 0x562b347bad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:16.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.008+0000 7f474f5ea740 4 client.26797 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:28:16.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.008+0000 7f474f5ea740 1 -- 172.21.15.114:0/280106518 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562b3439d4e0 con 0x562b347b88e0 2024-09-13T18:28:16.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.009+0000 7f474ad8a640 1 --2- 172.21.15.114:0/280106518 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x562b347b88e0 0x562b347bad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:16.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.010+0000 7f472dffb640 10 client.26797 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:28:16.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.014+0000 7f474ad8a640 1 --2- 172.21.15.114:0/280106518 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x562b347b88e0 0x562b347bad00 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7021b212a7deac1c server_cookie=aed910809e33472f in_seq=0 out_seq=0 2024-09-13T18:28:16.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.014+0000 7f472dffb640 1 -- 172.21.15.114:0/280106518 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562b3439d4e0 con 0x562b347b88e0 2024-09-13T18:28:16.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.014+0000 7f472dffb640 10 client.26797 handle_command_reply: tid=0 2024-09-13T18:28:16.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.014+0000 7f474f5ea740 1 client.26797 shutdown 2024-09-13T18:28:16.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.014+0000 7f474f5ea740 1 -- 172.21.15.114:0/280106518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b34747290 msgr2=0x562b3475d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:16.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.014+0000 7f474f5ea740 1 --2- 172.21.15.114:0/280106518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b34747290 0x562b3475d980 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f474000b5d0 tx=0x7f4740004830 comp rx=0 tx=0).stop 2024-09-13T18:28:16.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.016+0000 7f474f5ea740 1 -- 172.21.15.114:0/280106518 shutdown_connections 2024-09-13T18:28:16.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.016+0000 7f474f5ea740 1 --2- 172.21.15.114:0/280106518 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x562b347b88e0 0x562b347bad00 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:16.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.016+0000 7f474f5ea740 1 --2- 172.21.15.114:0/280106518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b34751b30 0x562b34759e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:16.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.016+0000 7f474f5ea740 1 --2- 172.21.15.114:0/280106518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b34747290 0x562b3475d980 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:16.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.016+0000 7f474f5ea740 1 --2- 172.21.15.114:0/280106518 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b34746970 0x562b3475d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:16.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.016+0000 7f474f5ea740 1 -- 172.21.15.114:0/280106518 >> 172.21.15.114:0/280106518 conn(0x562b34742ee0 msgr2=0x562b34749540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:16.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.017+0000 7f472dffb640 0 client.26797 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:28:16.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.020+0000 7f474f5ea740 1 -- 172.21.15.114:0/280106518 shutdown_connections 2024-09-13T18:28:16.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.021+0000 7f474f5ea740 1 -- 172.21.15.114:0/280106518 wait complete. 2024-09-13T18:28:16.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.024+0000 7f474f5ea740 20 client.26797 trim_cache size 0 max 16384 2024-09-13T18:28:16.026 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:28:16.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.029+0000 7f474b58b640 1 -- 172.21.15.114:0/3864464576 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f47300a1230 msgr2=0x7f47300a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:16.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.029+0000 7f474b58b640 1 --2- 172.21.15.114:0/3864464576 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f47300a1230 0x7f47300a36f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x562b34743d40 tx=0x7f4744132c50 comp rx=0 tx=0).stop 2024-09-13T18:28:16.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.029+0000 7f474b58b640 1 -- 172.21.15.114:0/3864464576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f473c0a47c0 msgr2=0x7f473c142e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:16.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.029+0000 7f474b58b640 1 --2- 172.21.15.114:0/3864464576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f473c0a47c0 0x7f473c142e50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f47440639a0 tx=0x7f474406f6a0 comp rx=0 tx=0).stop 2024-09-13T18:28:16.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.032+0000 7f474b58b640 1 -- 172.21.15.114:0/3864464576 shutdown_connections 2024-09-13T18:28:16.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.033+0000 7f474b58b640 1 --2- 172.21.15.114:0/3864464576 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f47300a1230 0x7f47300a36f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:16.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.033+0000 7f474b58b640 1 --2- 172.21.15.114:0/3864464576 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f473c0b74e0 0x7f473c147780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:16.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.033+0000 7f474b58b640 1 --2- 172.21.15.114:0/3864464576 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f473c0a50e0 0x7f473c143390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:16.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.033+0000 7f474b58b640 1 --2- 172.21.15.114:0/3864464576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f473c0a47c0 0x7f473c142e50 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:16.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.033+0000 7f474b58b640 1 -- 172.21.15.114:0/3864464576 >> 172.21.15.114:0/3864464576 conn(0x7f473c01a0a0 msgr2=0x7f473c0b4f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:16.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.038+0000 7f474b58b640 1 -- 172.21.15.114:0/3864464576 shutdown_connections 2024-09-13T18:28:16.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:16.038+0000 7f474b58b640 1 -- 172.21.15.114:0/3864464576 wait complete. 2024-09-13T18:28:17.073 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-13T18:28:17.073 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-09-13T18:28:17.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.284+0000 7f640359e640 1 -- 172.21.15.114:0/1221284598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64041074b0 msgr2=0x7f64041078f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:17.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.284+0000 7f640359e640 1 --2- 172.21.15.114:0/1221284598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64041074b0 0x7f64041078f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f63fc00b4a0 tx=0x7f63fc02b9e0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.286+0000 7f640359e640 1 -- 172.21.15.114:0/1221284598 shutdown_connections 2024-09-13T18:28:17.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.286+0000 7f640359e640 1 --2- 172.21.15.114:0/1221284598 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6404107e30 0x7f64040791f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.286+0000 7f640359e640 1 --2- 172.21.15.114:0/1221284598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64041074b0 0x7f64041078f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.286+0000 7f640359e640 1 --2- 172.21.15.114:0/1221284598 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f640410b8d0 0x7f640410bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.286+0000 7f640359e640 1 -- 172.21.15.114:0/1221284598 >> 172.21.15.114:0/1221284598 conn(0x7f640406af40 msgr2=0x7f640406b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:17.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.293+0000 7f640359e640 1 -- 172.21.15.114:0/1221284598 shutdown_connections 2024-09-13T18:28:17.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.293+0000 7f640359e640 1 -- 172.21.15.114:0/1221284598 wait complete. 2024-09-13T18:28:17.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.293+0000 7f640359e640 1 Processor -- start 2024-09-13T18:28:17.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.294+0000 7f640359e640 1 -- start start 2024-09-13T18:28:17.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.294+0000 7f640359e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f64041074b0 0x7f6404082bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:17.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.294+0000 7f640359e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6404107e30 0x7f6404083100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:17.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.294+0000 7f640359e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f640410b8d0 0x7f640407fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:17.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.294+0000 7f640359e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64040804f0 con 0x7f64041074b0 2024-09-13T18:28:17.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.294+0000 7f640359e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6404080660 con 0x7f6404107e30 2024-09-13T18:28:17.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.294+0000 7f640359e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6404080820 con 0x7f640410b8d0 2024-09-13T18:28:17.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.294+0000 7f6401d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6404107e30 0x7f6404083100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:17.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.294+0000 7f6401d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6404107e30 0x7f6404083100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:36320/0 (socket says 172.21.15.114:36320) 2024-09-13T18:28:17.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.294+0000 7f6401d9b640 1 -- 172.21.15.114:0/1352892580 learned_addr learned my addr 172.21.15.114:0/1352892580 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:17.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.294+0000 7f6402d9d640 1 --2- 172.21.15.114:0/1352892580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f640410b8d0 0x7f640407fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:17.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.296+0000 7f640259c640 1 --2- 172.21.15.114:0/1352892580 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f64041074b0 0x7f6404082bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:17.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.296+0000 7f6401d9b640 1 -- 172.21.15.114:0/1352892580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f640410b8d0 msgr2=0x7f640407fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:17.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.296+0000 7f6401d9b640 1 --2- 172.21.15.114:0/1352892580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f640410b8d0 0x7f640407fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.296+0000 7f6401d9b640 1 -- 172.21.15.114:0/1352892580 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f64041074b0 msgr2=0x7f6404082bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:17.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.296+0000 7f6401d9b640 1 --2- 172.21.15.114:0/1352892580 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f64041074b0 0x7f6404082bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.296+0000 7f6401d9b640 1 -- 172.21.15.114:0/1352892580 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63fc009900 con 0x7f6404107e30 2024-09-13T18:28:17.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.297+0000 7f6401d9b640 1 --2- 172.21.15.114:0/1352892580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6404107e30 0x7f6404083100 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f63fc00b5c0 tx=0x7f63fc02f930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:17.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.299+0000 7f63f37fe640 1 -- 172.21.15.114:0/1352892580 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63fc03e070 con 0x7f6404107e30 2024-09-13T18:28:17.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.299+0000 7f63f37fe640 1 -- 172.21.15.114:0/1352892580 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f63fc02fed0 con 0x7f6404107e30 2024-09-13T18:28:17.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.299+0000 7f63f37fe640 1 -- 172.21.15.114:0/1352892580 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63fc004710 con 0x7f6404107e30 2024-09-13T18:28:17.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.302+0000 7f640359e640 1 -- 172.21.15.114:0/1352892580 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6404080aa0 con 0x7f6404107e30 2024-09-13T18:28:17.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.302+0000 7f640359e640 1 -- 172.21.15.114:0/1352892580 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6404073460 con 0x7f6404107e30 2024-09-13T18:28:17.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.307+0000 7f63f37fe640 1 -- 172.21.15.114:0/1352892580 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f63fc00a040 con 0x7f6404107e30 2024-09-13T18:28:17.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.308+0000 7f63f37fe640 1 --2- 172.21.15.114:0/1352892580 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f63ec0a1410 0x7f63ec0a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:17.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.308+0000 7f63f37fe640 1 -- 172.21.15.114:0/1352892580 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f63fc0ef0c0 con 0x7f6404107e30 2024-09-13T18:28:17.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.312+0000 7f640bc4e740 1 Processor -- start 2024-09-13T18:28:17.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.313+0000 7f640259c640 1 --2- 172.21.15.114:0/1352892580 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f63ec0a1410 0x7f63ec0a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:17.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.313+0000 7f640bc4e740 1 -- start start 2024-09-13T18:28:17.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.313+0000 7f640bc4e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559b14c9c9a0 0x559b14c9cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:17.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.313+0000 7f640bc4e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b14c9d2c0 0x559b14ca7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:17.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.313+0000 7f640bc4e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b14ca7b60 0x559b14ca9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:17.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.313+0000 7f640bc4e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b14c07c60 con 0x559b14c9c9a0 2024-09-13T18:28:17.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.313+0000 7f640bc4e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b14c053f0 con 0x559b14ca7b60 2024-09-13T18:28:17.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.313+0000 7f640bc4e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b14c00ce0 con 0x559b14c9d2c0 2024-09-13T18:28:17.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.315+0000 7f6402d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b14ca7b60 0x559b14ca9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:17.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.315+0000 7f6402d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b14ca7b60 0x559b14ca9f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:36328/0 (socket says 172.21.15.114:36328) 2024-09-13T18:28:17.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.315+0000 7f6402d9d640 1 -- 172.21.15.114:0/4172425025 learned_addr learned my addr 172.21.15.114:0/4172425025 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:17.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.315+0000 7f6402d9d640 1 -- 172.21.15.114:0/4172425025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b14c9d2c0 msgr2=0x559b14ca7620 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:28:17.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.315+0000 7f6402d9d640 1 --2- 172.21.15.114:0/4172425025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b14c9d2c0 0x559b14ca7620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.315+0000 7f6402d9d640 1 -- 172.21.15.114:0/4172425025 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559b14c9c9a0 msgr2=0x559b14c9cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:17.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.315+0000 7f6402d9d640 1 --2- 172.21.15.114:0/4172425025 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559b14c9c9a0 0x559b14c9cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.315+0000 7f6402d9d640 1 -- 172.21.15.114:0/4172425025 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b14bbc970 con 0x559b14ca7b60 2024-09-13T18:28:17.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.318+0000 7f6402d9d640 1 --2- 172.21.15.114:0/4172425025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b14ca7b60 0x559b14ca9f50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f63f800a9b0 tx=0x7f63f800ae70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec636611b1fa3a2b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:17.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.324+0000 7f640259c640 1 --2- 172.21.15.114:0/1352892580 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f63ec0a1410 0x7f63ec0a38d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f63f40163c0 tx=0x7f63f4011920 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:17.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.326+0000 7f63f0ff9640 1 -- 172.21.15.114:0/4172425025 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f800dc70 con 0x559b14ca7b60 2024-09-13T18:28:17.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.326+0000 7f63f0ff9640 1 -- 172.21.15.114:0/4172425025 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f63f800c050 con 0x559b14ca7b60 2024-09-13T18:28:17.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.327+0000 7f640bc4e740 1 -- 172.21.15.114:0/4172425025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b14ca7b60 msgr2=0x559b14ca9f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:17.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.327+0000 7f640bc4e740 1 --2- 172.21.15.114:0/4172425025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b14ca7b60 0x559b14ca9f50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f63f800a9b0 tx=0x7f63f800ae70 comp rx=0 tx=0).stop 2024-09-13T18:28:17.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.329+0000 7f640bc4e740 1 -- 172.21.15.114:0/4172425025 shutdown_connections 2024-09-13T18:28:17.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.329+0000 7f640bc4e740 1 --2- 172.21.15.114:0/4172425025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b14ca7b60 0x559b14ca9f50 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.329+0000 7f640bc4e740 1 --2- 172.21.15.114:0/4172425025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b14c9d2c0 0x559b14ca7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.329+0000 7f640bc4e740 1 --2- 172.21.15.114:0/4172425025 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559b14c9c9a0 0x559b14c9cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.329+0000 7f640bc4e740 1 -- 172.21.15.114:0/4172425025 >> 172.21.15.114:0/4172425025 conn(0x559b14c98f30 msgr2=0x559b14c99310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:17.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.332+0000 7f640bc4e740 1 -- 172.21.15.114:0/4172425025 shutdown_connections 2024-09-13T18:28:17.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.334+0000 7f640bc4e740 1 -- 172.21.15.114:0/4172425025 wait complete. 2024-09-13T18:28:17.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.334+0000 7f640bc4e740 1 -- start start 2024-09-13T18:28:17.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.335+0000 7f640bc4e740 1 Processor -- start 2024-09-13T18:28:17.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.335+0000 7f640bc4e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b14c9c9a0 0x559b14ca7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:17.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.335+0000 7f640bc4e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b14c9d2c0 0x559b14ca48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:17.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.335+0000 7f640bc4e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559b14ca7b60 0x559b14ca4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:17.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.335+0000 7f640bc4e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b14bbef80 con 0x559b14ca7b60 2024-09-13T18:28:17.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.335+0000 7f640bc4e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b14bbec20 con 0x559b14c9d2c0 2024-09-13T18:28:17.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.335+0000 7f640bc4e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b14b8bab0 con 0x559b14c9c9a0 2024-09-13T18:28:17.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.335+0000 7f640bc4e740 10 client.0 _ll_register_callbacks cb 0x559b14bfa350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:28:17.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.336+0000 7f640259c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b14c9c9a0 0x559b14ca7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:17.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.336+0000 7f640259c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b14c9c9a0 0x559b14ca7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:39042/0 (socket says 172.21.15.114:39042) 2024-09-13T18:28:17.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.336+0000 7f640259c640 1 -- 172.21.15.114:0/178087817 learned_addr learned my addr 172.21.15.114:0/178087817 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:17.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.336+0000 7f6401d9b640 1 --2- 172.21.15.114:0/178087817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b14c9d2c0 0x559b14ca48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:17.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.337+0000 7f6401d9b640 1 -- 172.21.15.114:0/178087817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b14c9c9a0 msgr2=0x559b14ca7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:17.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.339+0000 7f6402d9d640 1 --2- 172.21.15.114:0/178087817 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559b14ca7b60 0x559b14ca4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:17.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.339+0000 7f6401d9b640 1 --2- 172.21.15.114:0/178087817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b14c9c9a0 0x559b14ca7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.339+0000 7f6401d9b640 1 -- 172.21.15.114:0/178087817 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559b14ca7b60 msgr2=0x559b14ca4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:17.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.339+0000 7f6401d9b640 1 --2- 172.21.15.114:0/178087817 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559b14ca7b60 0x559b14ca4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.339+0000 7f6401d9b640 1 -- 172.21.15.114:0/178087817 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63f800a5e0 con 0x559b14c9d2c0 2024-09-13T18:28:17.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.340+0000 7f6402d9d640 1 --2- 172.21.15.114:0/178087817 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559b14ca7b60 0x559b14ca4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:28:17.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.340+0000 7f640259c640 1 --2- 172.21.15.114:0/178087817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b14c9c9a0 0x559b14ca7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:28:17.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.340+0000 7f6401d9b640 1 --2- 172.21.15.114:0/178087817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b14c9d2c0 0x559b14ca48d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f63fc0ec460 tx=0x7f63fc0bb960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=611c0c09ec460319 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:17.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.342+0000 7f63e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:28:17.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.342+0000 7f63e9ffb640 1 -- 172.21.15.114:0/178087817 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63fc0bbe20 con 0x559b14c9d2c0 2024-09-13T18:28:17.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.342+0000 7f63e9ffb640 1 -- 172.21.15.114:0/178087817 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f63fc0bfc50 con 0x559b14c9d2c0 2024-09-13T18:28:17.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.342+0000 7f640bc4e740 1 -- 172.21.15.114:0/178087817 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559b14905860 con 0x559b14c9d2c0 2024-09-13T18:28:17.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.347+0000 7f63e9ffb640 1 -- 172.21.15.114:0/178087817 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63fc0c42a0 con 0x559b14c9d2c0 2024-09-13T18:28:17.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.348+0000 7f640bc4e740 10 client.26819 fetch_fsmap learned FSMap version 34 2024-09-13T18:28:17.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.348+0000 7f640bc4e740 1 -- 172.21.15.114:0/178087817 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x559b14bff360 con 0x559b14c9d2c0 2024-09-13T18:28:17.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.355+0000 7f63e9ffb640 1 -- 172.21.15.114:0/178087817 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559b14905860 con 0x559b14c9d2c0 2024-09-13T18:28:17.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.355+0000 7f63e9ffb640 1 -- 172.21.15.114:0/178087817 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f63fc0c3f20 con 0x559b14c9d2c0 2024-09-13T18:28:17.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.355+0000 7f640bc4e740 10 client.26819 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:28:17.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.355+0000 7f640bc4e740 10 client.26819 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:28:17.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.355+0000 7f640bc4e740 20 client.26819 populate_metadata read hostname 'smithi114' 2024-09-13T18:28:17.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.355+0000 7f640bc4e740 1 --2- 172.21.15.114:0/178087817 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559b14cc8730 0x559b14ccab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:17.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.355+0000 7f640bc4e740 4 client.26819 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:28:17.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.355+0000 7f640bc4e740 1 -- 172.21.15.114:0/178087817 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559b14958eb0 con 0x559b14cc8730 2024-09-13T18:28:17.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.360+0000 7f640259c640 1 --2- 172.21.15.114:0/178087817 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559b14cc8730 0x559b14ccab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:17.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.368+0000 7f640259c640 1 --2- 172.21.15.114:0/178087817 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559b14cc8730 0x559b14ccab50 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95c82f1d3bfc1c36 server_cookie=ad8cd54f3bdfd50b in_seq=0 out_seq=0 2024-09-13T18:28:17.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.369+0000 7f63e9ffb640 10 client.26819 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:28:17.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.373+0000 7f63e9ffb640 1 -- 172.21.15.114:0/178087817 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x559b14958eb0 con 0x559b14cc8730 2024-09-13T18:28:17.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.373+0000 7f63e9ffb640 10 client.26819 handle_command_reply: tid=0 2024-09-13T18:28:17.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.373+0000 7f640bc4e740 1 client.26819 shutdown 2024-09-13T18:28:17.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.376+0000 7f640bc4e740 1 -- 172.21.15.114:0/178087817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b14c9d2c0 msgr2=0x559b14ca48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:17.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.376+0000 7f640bc4e740 1 --2- 172.21.15.114:0/178087817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b14c9d2c0 0x559b14ca48d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f63fc0ec460 tx=0x7f63fc0bb960 comp rx=0 tx=0).stop 2024-09-13T18:28:17.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.381+0000 7f640bc4e740 1 -- 172.21.15.114:0/178087817 shutdown_connections 2024-09-13T18:28:17.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.381+0000 7f640bc4e740 1 --2- 172.21.15.114:0/178087817 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559b14cc8730 0x559b14ccab50 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.381+0000 7f640bc4e740 1 --2- 172.21.15.114:0/178087817 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559b14ca7b60 0x559b14ca4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.381+0000 7f640bc4e740 1 --2- 172.21.15.114:0/178087817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b14c9d2c0 0x559b14ca48d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.381+0000 7f640bc4e740 1 --2- 172.21.15.114:0/178087817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b14c9c9a0 0x559b14ca7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.381+0000 7f640bc4e740 1 -- 172.21.15.114:0/178087817 >> 172.21.15.114:0/178087817 conn(0x559b14c98f30 msgr2=0x559b14c9ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:17.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.381+0000 7f63e9ffb640 0 client.26819 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:28:17.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.383+0000 7f640bc4e740 1 -- 172.21.15.114:0/178087817 shutdown_connections 2024-09-13T18:28:17.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.383+0000 7f640bc4e740 1 -- 172.21.15.114:0/178087817 wait complete. 2024-09-13T18:28:17.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.389+0000 7f640bc4e740 20 client.26819 trim_cache size 0 max 16384 2024-09-13T18:28:17.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.434+0000 7f640bc4e740 1 Processor -- start 2024-09-13T18:28:17.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.441+0000 7f640bc4e740 1 -- start start 2024-09-13T18:28:17.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.441+0000 7f640bc4e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b14c98f30 0x559b14c98950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:17.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.441+0000 7f640bc4e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559b14c9c9a0 0x559b14ca3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:17.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.441+0000 7f640bc4e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b14c9d2c0 0x559b14ca36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:17.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.441+0000 7f640bc4e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b14c9dec0 con 0x559b14c9c9a0 2024-09-13T18:28:17.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.441+0000 7f640bc4e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b14c9dc40 con 0x559b14c9d2c0 2024-09-13T18:28:17.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.441+0000 7f640bc4e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b14c935e0 con 0x559b14c98f30 2024-09-13T18:28:17.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.443+0000 7f6402d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b14c9d2c0 0x559b14ca36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:17.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.444+0000 7f6402d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b14c9d2c0 0x559b14ca36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:36336/0 (socket says 172.21.15.114:36336) 2024-09-13T18:28:17.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.444+0000 7f6402d9d640 1 -- 172.21.15.114:0/3934371450 learned_addr learned my addr 172.21.15.114:0/3934371450 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:17.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.445+0000 7f640259c640 1 --2- 172.21.15.114:0/3934371450 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b14c98f30 0x559b14c98950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:17.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.445+0000 7f6401d9b640 1 --2- 172.21.15.114:0/3934371450 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559b14c9c9a0 0x559b14ca3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:17.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.449+0000 7f640259c640 1 -- 172.21.15.114:0/3934371450 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b14c9d2c0 msgr2=0x559b14ca36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:17.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.449+0000 7f640259c640 1 --2- 172.21.15.114:0/3934371450 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b14c9d2c0 0x559b14ca36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.449+0000 7f640259c640 1 -- 172.21.15.114:0/3934371450 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559b14c9c9a0 msgr2=0x559b14ca3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:17.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.449+0000 7f640259c640 1 --2- 172.21.15.114:0/3934371450 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559b14c9c9a0 0x559b14ca3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.449+0000 7f640259c640 1 -- 172.21.15.114:0/3934371450 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b14ba11c0 con 0x559b14c98f30 2024-09-13T18:28:17.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.449+0000 7f640259c640 1 --2- 172.21.15.114:0/3934371450 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b14c98f30 0x559b14c98950 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x559b14cc8b10 tx=0x7f63f401f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1418bb7bf716fc1c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:17.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.453+0000 7f63f17fa640 1 -- 172.21.15.114:0/3934371450 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f4024070 con 0x559b14c98f30 2024-09-13T18:28:17.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.453+0000 7f63f17fa640 1 -- 172.21.15.114:0/3934371450 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f63f400c040 con 0x559b14c98f30 2024-09-13T18:28:17.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.454+0000 7f640bc4e740 1 -- 172.21.15.114:0/3934371450 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b14c98f30 msgr2=0x559b14c98950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:17.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.454+0000 7f640bc4e740 1 --2- 172.21.15.114:0/3934371450 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b14c98f30 0x559b14c98950 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x559b14cc8b10 tx=0x7f63f401f000 comp rx=0 tx=0).stop 2024-09-13T18:28:17.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.460+0000 7f640bc4e740 1 -- 172.21.15.114:0/3934371450 shutdown_connections 2024-09-13T18:28:17.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.460+0000 7f640bc4e740 1 --2- 172.21.15.114:0/3934371450 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b14c9d2c0 0x559b14ca36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.460+0000 7f640bc4e740 1 --2- 172.21.15.114:0/3934371450 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559b14c9c9a0 0x559b14ca3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.460+0000 7f640bc4e740 1 --2- 172.21.15.114:0/3934371450 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b14c98f30 0x559b14c98950 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.460+0000 7f640bc4e740 1 -- 172.21.15.114:0/3934371450 >> 172.21.15.114:0/3934371450 conn(0x559b14ca76e0 msgr2=0x559b14c9fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:17.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.462+0000 7f640bc4e740 1 -- 172.21.15.114:0/3934371450 shutdown_connections 2024-09-13T18:28:17.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.463+0000 7f640bc4e740 1 -- 172.21.15.114:0/3934371450 wait complete. 2024-09-13T18:28:17.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.463+0000 7f640bc4e740 1 -- start start 2024-09-13T18:28:17.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.463+0000 7f640bc4e740 1 Processor -- start 2024-09-13T18:28:17.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.464+0000 7f640bc4e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b14c98f30 0x559b14cae970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:17.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.464+0000 7f640bc4e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559b14c9c9a0 0x559b14caeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:17.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.464+0000 7f640bc4e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b14c9d2c0 0x559b14cabee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:17.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.464+0000 7f640bc4e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b14cac5e0 con 0x559b14c9c9a0 2024-09-13T18:28:17.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.464+0000 7f640bc4e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b14cac750 con 0x559b14c98f30 2024-09-13T18:28:17.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.464+0000 7f640bc4e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b14cac910 con 0x559b14c9d2c0 2024-09-13T18:28:17.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.464+0000 7f640bc4e740 10 client.0 _ll_register_callbacks cb 0x7f63fc02f9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:28:17.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.464+0000 7f640259c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b14c98f30 0x559b14cae970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:17.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.464+0000 7f640259c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b14c98f30 0x559b14cae970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:36352/0 (socket says 172.21.15.114:36352) 2024-09-13T18:28:17.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.464+0000 7f640259c640 1 -- 172.21.15.114:0/2292111269 learned_addr learned my addr 172.21.15.114:0/2292111269 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:17.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.465+0000 7f640259c640 1 -- 172.21.15.114:0/2292111269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b14c9d2c0 msgr2=0x559b14cabee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:17.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.465+0000 7f640259c640 1 --2- 172.21.15.114:0/2292111269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b14c9d2c0 0x559b14cabee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.465+0000 7f640259c640 1 -- 172.21.15.114:0/2292111269 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559b14c9c9a0 msgr2=0x559b14caeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:17.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.465+0000 7f640259c640 1 --2- 172.21.15.114:0/2292111269 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559b14c9c9a0 0x559b14caeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.465+0000 7f640259c640 1 -- 172.21.15.114:0/2292111269 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63f401d650 con 0x559b14c98f30 2024-09-13T18:28:17.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.465+0000 7f640259c640 1 --2- 172.21.15.114:0/2292111269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b14c98f30 0x559b14cae970 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f63f401d600 tx=0x7f63f401f950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a4bfac4db4ba559 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:17.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.465+0000 7f63e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:28:17.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.465+0000 7f63e9ffb640 1 -- 172.21.15.114:0/2292111269 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f4024040 con 0x559b14c98f30 2024-09-13T18:28:17.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.466+0000 7f640bc4e740 1 -- 172.21.15.114:0/2292111269 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559b148f4590 con 0x559b14c98f30 2024-09-13T18:28:17.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.466+0000 7f63e9ffb640 1 -- 172.21.15.114:0/2292111269 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f63f401fd80 con 0x559b14c98f30 2024-09-13T18:28:17.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.466+0000 7f63e9ffb640 1 -- 172.21.15.114:0/2292111269 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63f402d470 con 0x559b14c98f30 2024-09-13T18:28:17.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.469+0000 7f640bc4e740 10 client.26825 fetch_fsmap learned FSMap version 34 2024-09-13T18:28:17.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.469+0000 7f640bc4e740 1 -- 172.21.15.114:0/2292111269 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x559b14bc6990 con 0x559b14c98f30 2024-09-13T18:28:17.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.473+0000 7f63e9ffb640 1 -- 172.21.15.114:0/2292111269 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559b148f4590 con 0x559b14c98f30 2024-09-13T18:28:17.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.473+0000 7f63e9ffb640 1 -- 172.21.15.114:0/2292111269 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f63f4035810 con 0x559b14c98f30 2024-09-13T18:28:17.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.473+0000 7f640bc4e740 10 client.26825 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:28:17.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.473+0000 7f640bc4e740 10 client.26825 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:28:17.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.473+0000 7f640bc4e740 20 client.26825 populate_metadata read hostname 'smithi114' 2024-09-13T18:28:17.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.473+0000 7f640bc4e740 1 --2- 172.21.15.114:0/2292111269 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559b14ca7b60 0x559b14d22ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:17.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.473+0000 7f640bc4e740 4 client.26825 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-13T18:28:17.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.473+0000 7f640bc4e740 1 -- 172.21.15.114:0/2292111269 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x559b14d23740 con 0x559b14ca7b60 2024-09-13T18:28:17.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.475+0000 7f6401d9b640 1 --2- 172.21.15.114:0/2292111269 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559b14ca7b60 0x559b14d22ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:17.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.479+0000 7f6401d9b640 1 --2- 172.21.15.114:0/2292111269 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559b14ca7b60 0x559b14d22ec0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ed2a86c951692e8 server_cookie=9d2c4109941557a8 in_seq=0 out_seq=0 2024-09-13T18:28:17.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.481+0000 7f63e9ffb640 10 client.26825 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:28:17.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.481+0000 7f63e9ffb640 1 -- 172.21.15.114:0/2292111269 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x559b14d23740 con 0x559b14ca7b60 2024-09-13T18:28:17.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.481+0000 7f63e9ffb640 10 client.26825 handle_command_reply: tid=0 2024-09-13T18:28:17.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.484+0000 7f640bc4e740 1 client.26825 shutdown 2024-09-13T18:28:17.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.484+0000 7f640bc4e740 1 -- 172.21.15.114:0/2292111269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b14c98f30 msgr2=0x559b14cae970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:17.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.484+0000 7f640bc4e740 1 --2- 172.21.15.114:0/2292111269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b14c98f30 0x559b14cae970 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f63f401d600 tx=0x7f63f401f950 comp rx=0 tx=0).stop 2024-09-13T18:28:17.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.488+0000 7f640bc4e740 1 -- 172.21.15.114:0/2292111269 shutdown_connections 2024-09-13T18:28:17.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.488+0000 7f640bc4e740 1 --2- 172.21.15.114:0/2292111269 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559b14ca7b60 0x559b14d22ec0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.488+0000 7f640bc4e740 1 --2- 172.21.15.114:0/2292111269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b14c9d2c0 0x559b14cabee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.488+0000 7f640bc4e740 1 --2- 172.21.15.114:0/2292111269 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559b14c9c9a0 0x559b14caeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.488+0000 7f640bc4e740 1 --2- 172.21.15.114:0/2292111269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b14c98f30 0x559b14cae970 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.488+0000 7f640bc4e740 1 -- 172.21.15.114:0/2292111269 >> 172.21.15.114:0/2292111269 conn(0x559b14ca76e0 msgr2=0x559b14c9fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:17.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.491+0000 7f63e9ffb640 0 client.26825 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:28:17.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.492+0000 7f640bc4e740 1 -- 172.21.15.114:0/2292111269 shutdown_connections 2024-09-13T18:28:17.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.494+0000 7f640bc4e740 1 -- 172.21.15.114:0/2292111269 wait complete. 2024-09-13T18:28:17.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.497+0000 7f640bc4e740 20 client.26825 trim_cache size 0 max 16384 2024-09-13T18:28:17.501 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:28:17.501 INFO:teuthology.orchestra.run.smithi114.stdout: "return_code": 0, 2024-09-13T18:28:17.501 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_tag": "216b0a5b-3f0d-4ebf-b2a7-97c247b20f7a", 2024-09-13T18:28:17.501 INFO:teuthology.orchestra.run.smithi114.stdout: "mode": "asynchronous" 2024-09-13T18:28:17.501 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:28:17.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.500+0000 7f640359e640 1 -- 172.21.15.114:0/1352892580 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f63ec0a1410 msgr2=0x7f63ec0a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:17.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.500+0000 7f640359e640 1 --2- 172.21.15.114:0/1352892580 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f63ec0a1410 0x7f63ec0a38d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f63f40163c0 tx=0x7f63f4011920 comp rx=0 tx=0).stop 2024-09-13T18:28:17.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.501+0000 7f640359e640 1 -- 172.21.15.114:0/1352892580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6404107e30 msgr2=0x7f6404083100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:17.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.501+0000 7f640359e640 1 --2- 172.21.15.114:0/1352892580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6404107e30 0x7f6404083100 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f63fc00b5c0 tx=0x7f63fc02f930 comp rx=0 tx=0).stop 2024-09-13T18:28:17.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.501+0000 7f640359e640 1 -- 172.21.15.114:0/1352892580 shutdown_connections 2024-09-13T18:28:17.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.506+0000 7f640359e640 1 --2- 172.21.15.114:0/1352892580 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f63ec0a1410 0x7f63ec0a38d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.506+0000 7f640359e640 1 --2- 172.21.15.114:0/1352892580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f640410b8d0 0x7f640407fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.506+0000 7f640359e640 1 --2- 172.21.15.114:0/1352892580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6404107e30 0x7f6404083100 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.506+0000 7f640359e640 1 --2- 172.21.15.114:0/1352892580 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f64041074b0 0x7f6404082bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.507+0000 7f640359e640 1 -- 172.21.15.114:0/1352892580 >> 172.21.15.114:0/1352892580 conn(0x7f640406af40 msgr2=0x7f640406f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:17.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.509+0000 7f640359e640 1 -- 172.21.15.114:0/1352892580 shutdown_connections 2024-09-13T18:28:17.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.512+0000 7f640359e640 1 -- 172.21.15.114:0/1352892580 wait complete. 2024-09-13T18:28:17.546 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T18:28:17.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.821+0000 7f2c15daa640 1 -- 172.21.15.114:0/4128547837 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c10110a10 msgr2=0x7f2c10112e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:17.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.821+0000 7f2c15daa640 1 --2- 172.21.15.114:0/4128547837 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c10110a10 0x7f2c10112e00 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f2c04009900 tx=0x7f2c0402a4b0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.825+0000 7f2c15daa640 1 -- 172.21.15.114:0/4128547837 shutdown_connections 2024-09-13T18:28:17.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.825+0000 7f2c15daa640 1 --2- 172.21.15.114:0/4128547837 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c10110a10 0x7f2c10112e00 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.825+0000 7f2c15daa640 1 --2- 172.21.15.114:0/4128547837 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c100fe6a0 0x7f2c101104d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.825+0000 7f2c15daa640 1 --2- 172.21.15.114:0/4128547837 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c100fdd80 0x7f2c100fe160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.825+0000 7f2c15daa640 1 -- 172.21.15.114:0/4128547837 >> 172.21.15.114:0/4128547837 conn(0x7f2c100f9c40 msgr2=0x7f2c100fc060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:17.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.831+0000 7f2c15daa640 1 -- 172.21.15.114:0/4128547837 shutdown_connections 2024-09-13T18:28:17.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.833+0000 7f2c15daa640 1 -- 172.21.15.114:0/4128547837 wait complete. 2024-09-13T18:28:17.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.833+0000 7f2c15daa640 1 Processor -- start 2024-09-13T18:28:17.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.834+0000 7f2c15daa640 1 -- start start 2024-09-13T18:28:17.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.834+0000 7f2c15daa640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c100fdd80 0x7f2c1010fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:17.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.834+0000 7f2c15daa640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c100fe6a0 0x7f2c101103b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:17.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.834+0000 7f2c15daa640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c10110a10 0x7f2c10109fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:17.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.834+0000 7f2c15daa640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1010a740 con 0x7f2c10110a10 2024-09-13T18:28:17.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.834+0000 7f2c15daa640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1010a8b0 con 0x7f2c100fe6a0 2024-09-13T18:28:17.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.834+0000 7f2c15daa640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1010aa70 con 0x7f2c100fdd80 2024-09-13T18:28:17.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.834+0000 7f2c0f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c100fdd80 0x7f2c1010fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:17.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.834+0000 7f2c0f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c100fdd80 0x7f2c1010fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:39056/0 (socket says 172.21.15.114:39056) 2024-09-13T18:28:17.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.834+0000 7f2c0f7fe640 1 -- 172.21.15.114:0/2662692235 learned_addr learned my addr 172.21.15.114:0/2662692235 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:17.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.835+0000 7f2c0f7fe640 1 -- 172.21.15.114:0/2662692235 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c100fe6a0 msgr2=0x7f2c101103b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:28:17.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.835+0000 7f2c0f7fe640 1 --2- 172.21.15.114:0/2662692235 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c100fe6a0 0x7f2c101103b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.835+0000 7f2c0f7fe640 1 -- 172.21.15.114:0/2662692235 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c10110a10 msgr2=0x7f2c10109fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:17.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.835+0000 7f2c0f7fe640 1 --2- 172.21.15.114:0/2662692235 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c10110a10 0x7f2c10109fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.835+0000 7f2c0f7fe640 1 -- 172.21.15.114:0/2662692235 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c04009560 con 0x7f2c100fdd80 2024-09-13T18:28:17.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.835+0000 7f2c0f7fe640 1 --2- 172.21.15.114:0/2662692235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c100fdd80 0x7f2c1010fe70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2c0000c890 tx=0x7f2c0000cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:17.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.837+0000 7f2c0cff9640 1 -- 172.21.15.114:0/2662692235 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c00004220 con 0x7f2c100fdd80 2024-09-13T18:28:17.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.838+0000 7f2c15daa640 1 -- 172.21.15.114:0/2662692235 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c101b19a0 con 0x7f2c100fdd80 2024-09-13T18:28:17.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.838+0000 7f2c15daa640 1 -- 172.21.15.114:0/2662692235 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c101b1dc0 con 0x7f2c100fdd80 2024-09-13T18:28:17.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.841+0000 7f2c0cff9640 1 -- 172.21.15.114:0/2662692235 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2c00007e40 con 0x7f2c100fdd80 2024-09-13T18:28:17.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.842+0000 7f2c0cff9640 1 -- 172.21.15.114:0/2662692235 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c00004a00 con 0x7f2c100fdd80 2024-09-13T18:28:17.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.847+0000 7f2c0cff9640 1 -- 172.21.15.114:0/2662692235 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f2c00004380 con 0x7f2c100fdd80 2024-09-13T18:28:17.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.848+0000 7f2c18acc740 1 Processor -- start 2024-09-13T18:28:17.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.850+0000 7f2c0cff9640 1 --2- 172.21.15.114:0/2662692235 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2bf80a1340 0x7f2bf80a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:17.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.850+0000 7f2c0cff9640 1 -- 172.21.15.114:0/2662692235 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2c000cd100 con 0x7f2c100fdd80 2024-09-13T18:28:17.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.852+0000 7f2c18acc740 1 -- start start 2024-09-13T18:28:17.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.852+0000 7f2c18acc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5583624a8e10 0x5583624a91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:17.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.852+0000 7f2c18acc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5583624a9730 0x5583624b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:17.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.852+0000 7f2c18acc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583624b3fd0 0x5583624b63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:17.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.852+0000 7f2c0ffff640 1 --2- 172.21.15.114:0/2662692235 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2bf80a1340 0x7f2bf80a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:17.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.852+0000 7f2c18acc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x558362413f20 con 0x5583624a8e10 2024-09-13T18:28:17.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.852+0000 7f2c18acc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583624116b0 con 0x5583624b3fd0 2024-09-13T18:28:17.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.852+0000 7f2c18acc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55836240cfa0 con 0x5583624a9730 2024-09-13T18:28:17.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.852+0000 7f2c0ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583624b3fd0 0x5583624b63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:17.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.852+0000 7f2c0ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583624b3fd0 0x5583624b63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:36374/0 (socket says 172.21.15.114:36374) 2024-09-13T18:28:17.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.852+0000 7f2c0ffff640 1 -- 172.21.15.114:0/73228736 learned_addr learned my addr 172.21.15.114:0/73228736 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:17.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.853+0000 7f2c0ffff640 1 --2- 172.21.15.114:0/2662692235 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2bf80a1340 0x7f2bf80a3800 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2c04033000 tx=0x7f2c04003640 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:17.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.853+0000 7f2c0ffff640 1 -- 172.21.15.114:0/73228736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5583624a9730 msgr2=0x5583624b3a90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:28:17.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.853+0000 7f2c0ffff640 1 --2- 172.21.15.114:0/73228736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5583624a9730 0x5583624b3a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.853+0000 7f2c0ffff640 1 -- 172.21.15.114:0/73228736 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5583624a8e10 msgr2=0x5583624a91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:17.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.853+0000 7f2c0ffff640 1 --2- 172.21.15.114:0/73228736 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5583624a8e10 0x5583624a91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.853+0000 7f2c0ffff640 1 -- 172.21.15.114:0/73228736 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583623c7840 con 0x5583624b3fd0 2024-09-13T18:28:17.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.856+0000 7f2c0ffff640 1 --2- 172.21.15.114:0/73228736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583624b3fd0 0x5583624b63c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2c040036b0 tx=0x7f2c04046000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65e4eb351889cd15 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:17.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.859+0000 7f2bf5ffb640 1 -- 172.21.15.114:0/73228736 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c04049650 con 0x5583624b3fd0 2024-09-13T18:28:17.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.859+0000 7f2bf5ffb640 1 -- 172.21.15.114:0/73228736 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2c04049c40 con 0x5583624b3fd0 2024-09-13T18:28:17.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.860+0000 7f2c18acc740 1 -- 172.21.15.114:0/73228736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583624b3fd0 msgr2=0x5583624b63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:17.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.860+0000 7f2c18acc740 1 --2- 172.21.15.114:0/73228736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583624b3fd0 0x5583624b63c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2c040036b0 tx=0x7f2c04046000 comp rx=0 tx=0).stop 2024-09-13T18:28:17.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.864+0000 7f2c18acc740 1 -- 172.21.15.114:0/73228736 shutdown_connections 2024-09-13T18:28:17.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.864+0000 7f2c18acc740 1 --2- 172.21.15.114:0/73228736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583624b3fd0 0x5583624b63c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.864+0000 7f2c18acc740 1 --2- 172.21.15.114:0/73228736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5583624a9730 0x5583624b3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.864+0000 7f2c18acc740 1 --2- 172.21.15.114:0/73228736 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5583624a8e10 0x5583624a91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.864+0000 7f2c18acc740 1 -- 172.21.15.114:0/73228736 >> 172.21.15.114:0/73228736 conn(0x5583624a5380 msgr2=0x5583624a5760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:17.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.866+0000 7f2c18acc740 1 -- 172.21.15.114:0/73228736 shutdown_connections 2024-09-13T18:28:17.868 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:17 smithi134 ceph-mon[34283]: pgmap v177: 129 pgs: 129 active+clean; 19 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 45 MiB/s wr, 865 op/s 2024-09-13T18:28:17.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.866+0000 7f2c18acc740 1 -- 172.21.15.114:0/73228736 wait complete. 2024-09-13T18:28:17.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.867+0000 7f2c18acc740 1 -- start start 2024-09-13T18:28:17.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.867+0000 7f2c18acc740 1 Processor -- start 2024-09-13T18:28:17.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.867+0000 7f2c18acc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583624a8e10 0x5583624b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:17.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.867+0000 7f2c18acc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5583624a9730 0x5583624b0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:17.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.867+0000 7f2c18acc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5583624b3fd0 0x5583624b1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:17.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.867+0000 7f2c18acc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583623ae2a0 con 0x5583624b3fd0 2024-09-13T18:28:17.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.867+0000 7f2c18acc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583620e5430 con 0x5583624a8e10 2024-09-13T18:28:17.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.867+0000 7f2c18acc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558362390680 con 0x5583624a9730 2024-09-13T18:28:17.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.867+0000 7f2c18acc740 10 client.0 _ll_register_callbacks cb 0x558362406610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:28:17.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.868+0000 7f2c0effd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5583624a9730 0x5583624b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:17.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.868+0000 7f2c0effd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5583624a9730 0x5583624b0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:39062/0 (socket says 172.21.15.114:39062) 2024-09-13T18:28:17.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.868+0000 7f2c0effd640 1 -- 172.21.15.114:0/1796039022 learned_addr learned my addr 172.21.15.114:0/1796039022 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:17.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.868+0000 7f2c0f7fe640 1 --2- 172.21.15.114:0/1796039022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583624a8e10 0x5583624b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:17.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.868+0000 7f2c0effd640 1 -- 172.21.15.114:0/1796039022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583624a8e10 msgr2=0x5583624b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:17.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.868+0000 7f2c0effd640 1 --2- 172.21.15.114:0/1796039022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583624a8e10 0x5583624b39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.868+0000 7f2c0effd640 1 -- 172.21.15.114:0/1796039022 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5583624b3fd0 msgr2=0x5583624b1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:17.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.868+0000 7f2c0effd640 1 --2- 172.21.15.114:0/1796039022 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5583624b3fd0 0x5583624b1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.868+0000 7f2c0effd640 1 -- 172.21.15.114:0/1796039022 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c04045070 con 0x5583624a9730 2024-09-13T18:28:17.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.872+0000 7f2c0effd640 1 --2- 172.21.15.114:0/1796039022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5583624a9730 0x5583624b0ca0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2c0800b440 tx=0x7f2c0800b900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41c0d9c3b12f470e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:17.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.872+0000 7f2beeffd640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:28:17.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.875+0000 7f2beeffd640 1 -- 172.21.15.114:0/1796039022 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c08011800 con 0x5583624a9730 2024-09-13T18:28:17.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.875+0000 7f2beeffd640 1 -- 172.21.15.114:0/1796039022 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2c0800e070 con 0x5583624a9730 2024-09-13T18:28:17.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.876+0000 7f2c18acc740 1 -- 172.21.15.114:0/1796039022 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5583621004e0 con 0x5583624a9730 2024-09-13T18:28:17.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.877+0000 7f2beeffd640 1 -- 172.21.15.114:0/1796039022 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c08011960 con 0x5583624a9730 2024-09-13T18:28:17.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.877+0000 7f2c18acc740 10 client.26815 fetch_fsmap learned FSMap version 34 2024-09-13T18:28:17.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.877+0000 7f2c18acc740 1 -- 172.21.15.114:0/1796039022 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55836240b620 con 0x5583624a9730 2024-09-13T18:28:17.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.878+0000 7f2beeffd640 1 -- 172.21.15.114:0/1796039022 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5583621004e0 con 0x5583624a9730 2024-09-13T18:28:17.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.878+0000 7f2beeffd640 1 -- 172.21.15.114:0/1796039022 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2c08010070 con 0x5583624a9730 2024-09-13T18:28:17.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.879+0000 7f2c18acc740 10 client.26815 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:28:17.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.879+0000 7f2c18acc740 10 client.26815 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:28:17.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.879+0000 7f2c18acc740 20 client.26815 populate_metadata read hostname 'smithi114' 2024-09-13T18:28:17.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.879+0000 7f2c18acc740 1 --2- 172.21.15.114:0/1796039022 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5583624d4f50 0x5583624d7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:17.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.879+0000 7f2c18acc740 4 client.26815 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:28:17.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.879+0000 7f2c18acc740 1 -- 172.21.15.114:0/1796039022 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5583621117e0 con 0x5583624d4f50 2024-09-13T18:28:17.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.879+0000 7f2c0f7fe640 1 --2- 172.21.15.114:0/1796039022 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5583624d4f50 0x5583624d7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:17.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.880+0000 7f2c0f7fe640 1 --2- 172.21.15.114:0/1796039022 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5583624d4f50 0x5583624d7370 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf20d162aaa552ec server_cookie=3da170bded5c9370 in_seq=0 out_seq=0 2024-09-13T18:28:17.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.880+0000 7f2beeffd640 10 client.26815 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:28:17.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.881+0000 7f2beeffd640 1 -- 172.21.15.114:0/1796039022 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5583621117e0 con 0x5583624d4f50 2024-09-13T18:28:17.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.882+0000 7f2beeffd640 10 client.26815 handle_command_reply: tid=0 2024-09-13T18:28:17.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.883+0000 7f2c18acc740 1 client.26815 shutdown 2024-09-13T18:28:17.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.884+0000 7f2c18acc740 1 -- 172.21.15.114:0/1796039022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5583624a9730 msgr2=0x5583624b0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:17.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.884+0000 7f2c18acc740 1 --2- 172.21.15.114:0/1796039022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5583624a9730 0x5583624b0ca0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2c0800b440 tx=0x7f2c0800b900 comp rx=0 tx=0).stop 2024-09-13T18:28:17.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.884+0000 7f2c18acc740 1 -- 172.21.15.114:0/1796039022 shutdown_connections 2024-09-13T18:28:17.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.884+0000 7f2c18acc740 1 --2- 172.21.15.114:0/1796039022 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5583624d4f50 0x5583624d7370 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.884+0000 7f2c18acc740 1 --2- 172.21.15.114:0/1796039022 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5583624b3fd0 0x5583624b1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.884+0000 7f2c18acc740 1 --2- 172.21.15.114:0/1796039022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5583624a9730 0x5583624b0ca0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.884+0000 7f2c18acc740 1 --2- 172.21.15.114:0/1796039022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583624a8e10 0x5583624b39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.884+0000 7f2c18acc740 1 -- 172.21.15.114:0/1796039022 >> 172.21.15.114:0/1796039022 conn(0x5583624a5380 msgr2=0x5583624ab0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:17.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.884+0000 7f2beeffd640 0 client.26815 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:28:17.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.885+0000 7f2c18acc740 1 -- 172.21.15.114:0/1796039022 shutdown_connections 2024-09-13T18:28:17.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.885+0000 7f2c18acc740 1 -- 172.21.15.114:0/1796039022 wait complete. 2024-09-13T18:28:17.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.885+0000 7f2c18acc740 20 client.26815 trim_cache size 0 max 16384 2024-09-13T18:28:17.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.921+0000 7f2c18acc740 1 Processor -- start 2024-09-13T18:28:17.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.925+0000 7f2c18acc740 1 -- start start 2024-09-13T18:28:17.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.926+0000 7f2c18acc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5583624a8e10 0x5583624a4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:17.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.926+0000 7f2c18acc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5583624a9730 0x5583624b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:17.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.926+0000 7f2c18acc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583624b3fd0 0x5583624cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:17.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.926+0000 7f2c18acc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583621117e0 con 0x5583624a8e10 2024-09-13T18:28:17.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.926+0000 7f2c0f7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5583624a8e10 0x5583624a4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:17.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.926+0000 7f2c0f7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5583624a8e10 0x5583624a4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55716/0 (socket says 172.21.15.114:55716) 2024-09-13T18:28:17.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.926+0000 7f2c0f7fe640 1 -- 172.21.15.114:0/1296919324 learned_addr learned my addr 172.21.15.114:0/1296919324 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:17.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.926+0000 7f2c18acc740 1 -- 172.21.15.114:0/1296919324 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583621004e0 con 0x5583624b3fd0 2024-09-13T18:28:17.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.926+0000 7f2c18acc740 1 -- 172.21.15.114:0/1296919324 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583624c7a90 con 0x5583624a9730 2024-09-13T18:28:17.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.927+0000 7f2c0f7fe640 1 -- 172.21.15.114:0/1296919324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5583624a9730 msgr2=0x5583624b34c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:28:17.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.927+0000 7f2c0f7fe640 1 --2- 172.21.15.114:0/1296919324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5583624a9730 0x5583624b34c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.927+0000 7f2c0f7fe640 1 -- 172.21.15.114:0/1296919324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583624b3fd0 msgr2=0x5583624cbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:17.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.927+0000 7f2c0f7fe640 1 --2- 172.21.15.114:0/1296919324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583624b3fd0 0x5583624cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.927+0000 7f2c0f7fe640 1 -- 172.21.15.114:0/1296919324 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583623ad790 con 0x5583624a8e10 2024-09-13T18:28:17.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.928+0000 7f2c0f7fe640 1 --2- 172.21.15.114:0/1296919324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5583624a8e10 0x5583624a4d00 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f2c000968d0 tx=0x7f2c0009a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abbd0136dc443c40 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:17.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.930+0000 7f2bf67fc640 1 -- 172.21.15.114:0/1296919324 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c000b1070 con 0x5583624a8e10 2024-09-13T18:28:17.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.930+0000 7f2bf67fc640 1 -- 172.21.15.114:0/1296919324 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2c00097fa0 con 0x5583624a8e10 2024-09-13T18:28:17.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.931+0000 7f2c18acc740 1 -- 172.21.15.114:0/1296919324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5583624a8e10 msgr2=0x5583624a4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:17.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.931+0000 7f2c18acc740 1 --2- 172.21.15.114:0/1296919324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5583624a8e10 0x5583624a4d00 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f2c000968d0 tx=0x7f2c0009a630 comp rx=0 tx=0).stop 2024-09-13T18:28:17.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.934+0000 7f2c18acc740 1 -- 172.21.15.114:0/1296919324 shutdown_connections 2024-09-13T18:28:17.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.934+0000 7f2c18acc740 1 --2- 172.21.15.114:0/1296919324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583624b3fd0 0x5583624cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.934+0000 7f2c18acc740 1 --2- 172.21.15.114:0/1296919324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5583624a9730 0x5583624b34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.934+0000 7f2c18acc740 1 --2- 172.21.15.114:0/1296919324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5583624a8e10 0x5583624a4d00 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.934+0000 7f2c18acc740 1 -- 172.21.15.114:0/1296919324 >> 172.21.15.114:0/1296919324 conn(0x5583624a5380 msgr2=0x5583624abf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:17.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.936+0000 7f2c18acc740 1 -- 172.21.15.114:0/1296919324 shutdown_connections 2024-09-13T18:28:17.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.936+0000 7f2c18acc740 1 -- 172.21.15.114:0/1296919324 wait complete. 2024-09-13T18:28:17.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.936+0000 7f2c18acc740 1 -- start start 2024-09-13T18:28:17.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.937+0000 7f2c18acc740 1 Processor -- start 2024-09-13T18:28:17.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.937+0000 7f2c18acc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583624a8e10 0x5583624cbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:17.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.937+0000 7f2c18acc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5583624a9730 0x5583624c8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:17.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.938+0000 7f2c18acc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5583624b3fd0 0x5583624c89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:17.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.938+0000 7f2c18acc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583624c8ee0 con 0x5583624b3fd0 2024-09-13T18:28:17.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.938+0000 7f2c18acc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583624c9050 con 0x5583624a8e10 2024-09-13T18:28:17.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.938+0000 7f2c18acc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583624c91c0 con 0x5583624a9730 2024-09-13T18:28:17.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.938+0000 7f2c18acc740 10 client.0 _ll_register_callbacks cb 0x7f2c080094d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:28:17.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.940+0000 7f2c0f7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583624a8e10 0x5583624cbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:17.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.940+0000 7f2c0effd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5583624a9730 0x5583624c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:17.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.940+0000 7f2c0f7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583624a8e10 0x5583624cbf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:36384/0 (socket says 172.21.15.114:36384) 2024-09-13T18:28:17.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.940+0000 7f2c0f7fe640 1 -- 172.21.15.114:0/725645159 learned_addr learned my addr 172.21.15.114:0/725645159 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:17.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:17 smithi204 ceph-mon[34777]: pgmap v177: 129 pgs: 129 active+clean; 19 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 45 MiB/s wr, 865 op/s 2024-09-13T18:28:17.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.943+0000 7f2c0effd640 1 -- 172.21.15.114:0/725645159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583624a8e10 msgr2=0x5583624cbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:17.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.943+0000 7f2c0effd640 1 --2- 172.21.15.114:0/725645159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583624a8e10 0x5583624cbf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.943+0000 7f2c0effd640 1 -- 172.21.15.114:0/725645159 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5583624b3fd0 msgr2=0x5583624c89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:17.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.943+0000 7f2c0effd640 1 --2- 172.21.15.114:0/725645159 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5583624b3fd0 0x5583624c89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.943+0000 7f2c0effd640 1 -- 172.21.15.114:0/725645159 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c0009ae70 con 0x5583624a9730 2024-09-13T18:28:17.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.944+0000 7f2c0effd640 1 --2- 172.21.15.114:0/725645159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5583624a9730 0x5583624c8460 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x5583624a6390 tx=0x7f2c08019f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b23065bc4614815 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:17.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.944+0000 7f2beeffd640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:28:17.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.944+0000 7f2beeffd640 1 -- 172.21.15.114:0/725645159 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c080103a0 con 0x5583624a9730 2024-09-13T18:28:17.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.944+0000 7f2beeffd640 1 -- 172.21.15.114:0/725645159 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2c080108c0 con 0x5583624a9730 2024-09-13T18:28:17.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.945+0000 7f2c18acc740 1 -- 172.21.15.114:0/725645159 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2c04049c40 con 0x5583624a9730 2024-09-13T18:28:17.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.946+0000 7f2beeffd640 1 -- 172.21.15.114:0/725645159 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c08010ae0 con 0x5583624a9730 2024-09-13T18:28:17.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.946+0000 7f2c18acc740 10 client.26818 fetch_fsmap learned FSMap version 34 2024-09-13T18:28:17.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.946+0000 7f2c18acc740 1 -- 172.21.15.114:0/725645159 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5583623d2ea0 con 0x5583624a9730 2024-09-13T18:28:17.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.946+0000 7f2beeffd640 1 -- 172.21.15.114:0/725645159 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2c08032030 con 0x5583624a9730 2024-09-13T18:28:17.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.946+0000 7f2beeffd640 1 -- 172.21.15.114:0/725645159 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2c0802c060 con 0x5583624a9730 2024-09-13T18:28:17.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.946+0000 7f2c18acc740 10 client.26818 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:28:17.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.946+0000 7f2c18acc740 10 client.26818 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:28:17.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.947+0000 7f2c18acc740 20 client.26818 populate_metadata read hostname 'smithi114' 2024-09-13T18:28:17.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.947+0000 7f2c18acc740 1 --2- 172.21.15.114:0/725645159 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55836252c860 0x55836252ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:17.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.947+0000 7f2c18acc740 4 client.26818 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T18:28:17.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.947+0000 7f2c18acc740 1 -- 172.21.15.114:0/725645159 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558362164e30 con 0x55836252c860 2024-09-13T18:28:17.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.948+0000 7f2c0f7fe640 1 --2- 172.21.15.114:0/725645159 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55836252c860 0x55836252ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:17.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.949+0000 7f2beeffd640 10 client.26818 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:28:17.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.951+0000 7f2c0f7fe640 1 --2- 172.21.15.114:0/725645159 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55836252c860 0x55836252ec80 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fb1bebfddff2eba server_cookie=5cb36a7c1b5be32d in_seq=0 out_seq=0 2024-09-13T18:28:17.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.951+0000 7f2beeffd640 1 -- 172.21.15.114:0/725645159 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x558362164e30 con 0x55836252c860 2024-09-13T18:28:17.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.951+0000 7f2beeffd640 10 client.26818 handle_command_reply: tid=0 2024-09-13T18:28:17.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.951+0000 7f2c18acc740 1 client.26818 shutdown 2024-09-13T18:28:17.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.956+0000 7f2c18acc740 1 -- 172.21.15.114:0/725645159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5583624a9730 msgr2=0x5583624c8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:17.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.956+0000 7f2c18acc740 1 --2- 172.21.15.114:0/725645159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5583624a9730 0x5583624c8460 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x5583624a6390 tx=0x7f2c08019f10 comp rx=0 tx=0).stop 2024-09-13T18:28:17.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.961+0000 7f2c18acc740 1 -- 172.21.15.114:0/725645159 shutdown_connections 2024-09-13T18:28:17.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.961+0000 7f2c18acc740 1 --2- 172.21.15.114:0/725645159 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55836252c860 0x55836252ec80 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.961+0000 7f2c18acc740 1 --2- 172.21.15.114:0/725645159 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5583624b3fd0 0x5583624c89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.961+0000 7f2c18acc740 1 --2- 172.21.15.114:0/725645159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5583624a9730 0x5583624c8460 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.961+0000 7f2c18acc740 1 --2- 172.21.15.114:0/725645159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583624a8e10 0x5583624cbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.961+0000 7f2c18acc740 1 -- 172.21.15.114:0/725645159 >> 172.21.15.114:0/725645159 conn(0x5583624a5380 msgr2=0x5583624aa8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:17.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.961+0000 7f2beeffd640 0 client.26818 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:28:17.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.962+0000 7f2c18acc740 1 -- 172.21.15.114:0/725645159 shutdown_connections 2024-09-13T18:28:17.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.962+0000 7f2c18acc740 1 -- 172.21.15.114:0/725645159 wait complete. 2024-09-13T18:28:17.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.962+0000 7f2c18acc740 20 client.26818 trim_cache size 0 max 16384 2024-09-13T18:28:17.965 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:28:17.965 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-13T18:28:17.965 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": { 2024-09-13T18:28:17.965 INFO:teuthology.orchestra.run.smithi114.stdout: "216b0a5b-3f0d-4ebf-b2a7-97c247b20f7a": { 2024-09-13T18:28:17.965 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-13T18:28:17.965 INFO:teuthology.orchestra.run.smithi114.stdout: "tag": "216b0a5b-3f0d-4ebf-b2a7-97c247b20f7a", 2024-09-13T18:28:17.965 INFO:teuthology.orchestra.run.smithi114.stdout: "options": "recursive,force" 2024-09-13T18:28:17.966 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:28:17.966 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:28:17.966 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:28:17.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.964+0000 7f2c15daa640 1 -- 172.21.15.114:0/2662692235 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2bf80a1340 msgr2=0x7f2bf80a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:17.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.964+0000 7f2c15daa640 1 --2- 172.21.15.114:0/2662692235 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2bf80a1340 0x7f2bf80a3800 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2c04033000 tx=0x7f2c04003640 comp rx=0 tx=0).stop 2024-09-13T18:28:17.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.964+0000 7f2c15daa640 1 -- 172.21.15.114:0/2662692235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c100fdd80 msgr2=0x7f2c1010fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:17.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.964+0000 7f2c15daa640 1 --2- 172.21.15.114:0/2662692235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c100fdd80 0x7f2c1010fe70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2c0000c890 tx=0x7f2c0000cd50 comp rx=0 tx=0).stop 2024-09-13T18:28:17.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.964+0000 7f2c15daa640 1 -- 172.21.15.114:0/2662692235 shutdown_connections 2024-09-13T18:28:17.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.964+0000 7f2c15daa640 1 --2- 172.21.15.114:0/2662692235 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2bf80a1340 0x7f2bf80a3800 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.964+0000 7f2c15daa640 1 --2- 172.21.15.114:0/2662692235 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c10110a10 0x7f2c10109fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.964+0000 7f2c15daa640 1 --2- 172.21.15.114:0/2662692235 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c100fe6a0 0x7f2c101103b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.965+0000 7f2c15daa640 1 --2- 172.21.15.114:0/2662692235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c100fdd80 0x7f2c1010fe70 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:17.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.965+0000 7f2c15daa640 1 -- 172.21.15.114:0/2662692235 >> 172.21.15.114:0/2662692235 conn(0x7f2c100f9c40 msgr2=0x7f2c101110f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:17.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.968+0000 7f2c15daa640 1 -- 172.21.15.114:0/2662692235 shutdown_connections 2024-09-13T18:28:17.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:17 smithi114 ceph-mon[30897]: pgmap v177: 129 pgs: 129 active+clean; 19 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 45 MiB/s wr, 865 op/s 2024-09-13T18:28:17.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:17.969+0000 7f2c15daa640 1 -- 172.21.15.114:0/2662692235 wait complete. 2024-09-13T18:28:17.994 INFO:tasks.cephfs.filesystem:scrub status for tag:216b0a5b-3f0d-4ebf-b2a7-97c247b20f7a - {'path': '/', 'tag': '216b0a5b-3f0d-4ebf-b2a7-97c247b20f7a', 'options': 'recursive,force'} 2024-09-13T18:28:18.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:18 smithi134 ceph-mon[34283]: scrub queued for path: / 2024-09-13T18:28:18.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:18 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:28:18.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:18 smithi134 ceph-mon[34283]: scrub summary: active paths [/] 2024-09-13T18:28:18.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:18 smithi134 ceph-mon[34283]: scrub summary: active paths [/] 2024-09-13T18:28:18.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:18 smithi134 ceph-mon[34283]: scrub summary: active paths [/] 2024-09-13T18:28:18.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:18 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:28:18.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:18 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:28:18.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:18 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:28:18.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:18 smithi204 ceph-mon[34777]: scrub queued for path: / 2024-09-13T18:28:18.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:18 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:28:18.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:18 smithi204 ceph-mon[34777]: scrub summary: active paths [/] 2024-09-13T18:28:18.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:18 smithi204 ceph-mon[34777]: scrub summary: active paths [/] 2024-09-13T18:28:18.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:18 smithi204 ceph-mon[34777]: scrub summary: active paths [/] 2024-09-13T18:28:18.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:18 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:28:18.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:18 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:28:18.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:18 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:28:18.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:18 smithi114 ceph-mon[30897]: scrub queued for path: / 2024-09-13T18:28:18.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:18 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:28:18.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:18 smithi114 ceph-mon[30897]: scrub summary: active paths [/] 2024-09-13T18:28:18.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:18 smithi114 ceph-mon[30897]: scrub summary: active paths [/] 2024-09-13T18:28:18.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:18 smithi114 ceph-mon[30897]: scrub summary: active paths [/] 2024-09-13T18:28:18.972 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:18 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:28:18.972 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:18 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:28:18.972 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:18 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:28:19.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:19 smithi134 ceph-mon[34283]: pgmap v178: 129 pgs: 129 active+clean; 19 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 33 MiB/s wr, 642 op/s 2024-09-13T18:28:19.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:19 smithi204 ceph-mon[34777]: pgmap v178: 129 pgs: 129 active+clean; 19 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 33 MiB/s wr, 642 op/s 2024-09-13T18:28:19.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:19 smithi114 ceph-mon[30897]: pgmap v178: 129 pgs: 129 active+clean; 19 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 33 MiB/s wr, 642 op/s 2024-09-13T18:28:21.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:21 smithi204 ceph-mon[34777]: pgmap v179: 129 pgs: 129 active+clean; 19 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 50 MiB/s wr, 970 op/s 2024-09-13T18:28:21.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:21 smithi114 ceph-mon[30897]: pgmap v179: 129 pgs: 129 active+clean; 19 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 50 MiB/s wr, 970 op/s 2024-09-13T18:28:22.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:21 smithi134 ceph-mon[34283]: pgmap v179: 129 pgs: 129 active+clean; 19 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 50 MiB/s wr, 970 op/s 2024-09-13T18:28:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.838+0000 7fe16359e640 1 -- 172.21.15.114:0/3032158793 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe164106550 msgr2=0x7fe164106930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.838+0000 7fe16359e640 1 --2- 172.21.15.114:0/3032158793 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe164106550 0x7fe164106930 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fe15801a910 tx=0x7fe15803b430 comp rx=0 tx=0).stop 2024-09-13T18:28:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.846+0000 7fe16359e640 1 -- 172.21.15.114:0/3032158793 shutdown_connections 2024-09-13T18:28:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.846+0000 7fe16359e640 1 --2- 172.21.15.114:0/3032158793 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe16410b4c0 0x7fe164113060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.846+0000 7fe16359e640 1 --2- 172.21.15.114:0/3032158793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe164106f00 0x7fe16410adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.846+0000 7fe16359e640 1 --2- 172.21.15.114:0/3032158793 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe164106550 0x7fe164106930 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:23.849 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.846+0000 7fe16359e640 1 -- 172.21.15.114:0/3032158793 >> 172.21.15.114:0/3032158793 conn(0x7fe164075ae0 msgr2=0x7fe164077f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.851+0000 7fe16359e640 1 -- 172.21.15.114:0/3032158793 shutdown_connections 2024-09-13T18:28:23.861 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.857+0000 7fe16359e640 1 -- 172.21.15.114:0/3032158793 wait complete. 2024-09-13T18:28:23.861 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.857+0000 7fe16359e640 1 Processor -- start 2024-09-13T18:28:23.865 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.860+0000 7fe16359e640 1 -- start start 2024-09-13T18:28:23.865 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.861+0000 7fe16359e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe164106550 0x7fe164112c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:23.865 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.861+0000 7fe16359e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe16410b4c0 0x7fe16410ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:23.865 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.861+0000 7fe16359e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe16410f2a0 0x7fe16410f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:23.865 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.861+0000 7fe16359e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1641133e0 con 0x7fe16410b4c0 2024-09-13T18:28:23.865 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.861+0000 7fe16359e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe164080b40 con 0x7fe164106550 2024-09-13T18:28:23.865 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.861+0000 7fe16359e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe164080cb0 con 0x7fe16410f2a0 2024-09-13T18:28:23.865 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.861+0000 7fe162d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe16410f2a0 0x7fe16410f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:23.865 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.861+0000 7fe162d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe16410f2a0 0x7fe16410f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:38226/0 (socket says 172.21.15.114:38226) 2024-09-13T18:28:23.866 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.861+0000 7fe162d9d640 1 -- 172.21.15.114:0/2571904006 learned_addr learned my addr 172.21.15.114:0/2571904006 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:23.866 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.861+0000 7fe162d9d640 1 -- 172.21.15.114:0/2571904006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe164106550 msgr2=0x7fe164112c50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:28:23.866 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.861+0000 7fe162d9d640 1 --2- 172.21.15.114:0/2571904006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe164106550 0x7fe164112c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:23.866 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.861+0000 7fe162d9d640 1 -- 172.21.15.114:0/2571904006 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe16410b4c0 msgr2=0x7fe16410ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:23.866 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.861+0000 7fe162d9d640 1 --2- 172.21.15.114:0/2571904006 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe16410b4c0 0x7fe16410ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:23.866 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.861+0000 7fe162d9d640 1 -- 172.21.15.114:0/2571904006 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe15801a570 con 0x7fe16410f2a0 2024-09-13T18:28:23.866 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.862+0000 7fe162d9d640 1 --2- 172.21.15.114:0/2571904006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe16410f2a0 0x7fe16410f750 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe15400b3e0 tx=0x7fe154009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:23.867 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.863+0000 7fe1537fe640 1 -- 172.21.15.114:0/2571904006 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe154011830 con 0x7fe16410f2a0 2024-09-13T18:28:23.867 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.863+0000 7fe1537fe640 1 -- 172.21.15.114:0/2571904006 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe154011e20 con 0x7fe16410f2a0 2024-09-13T18:28:23.867 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.863+0000 7fe1537fe640 1 -- 172.21.15.114:0/2571904006 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe154010e50 con 0x7fe16410f2a0 2024-09-13T18:28:23.868 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.865+0000 7fe16359e640 1 -- 172.21.15.114:0/2571904006 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe164080f90 con 0x7fe16410f2a0 2024-09-13T18:28:23.869 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.865+0000 7fe16359e640 1 -- 172.21.15.114:0/2571904006 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1641a97c0 con 0x7fe16410f2a0 2024-09-13T18:28:23.876 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.872+0000 7fe1537fe640 1 -- 172.21.15.114:0/2571904006 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fe154011990 con 0x7fe16410f2a0 2024-09-13T18:28:23.882 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.874+0000 7fe1537fe640 1 --2- 172.21.15.114:0/2571904006 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe14c0a14d0 0x7fe14c0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:23.882 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.874+0000 7fe1537fe640 1 -- 172.21.15.114:0/2571904006 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fe1540cc910 con 0x7fe16410f2a0 2024-09-13T18:28:23.882 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.879+0000 7fe16b6aa740 1 Processor -- start 2024-09-13T18:28:23.884 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.879+0000 7fe16259c640 1 --2- 172.21.15.114:0/2571904006 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe14c0a14d0 0x7fe14c0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:23.885 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.882+0000 7fe16b6aa740 1 -- start start 2024-09-13T18:28:23.885 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.882+0000 7fe16b6aa740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561687496500 0x5616874968e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:23.885 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.882+0000 7fe16b6aa740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561687496e20 0x5616874a1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:23.885 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.882+0000 7fe16b6aa740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5616874a16c0 0x5616874a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:23.886 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.882+0000 7fe16b6aa740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x561687401c30 con 0x561687496500 2024-09-13T18:28:23.886 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.882+0000 7fe16b6aa740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616873ff3c0 con 0x5616874a16c0 2024-09-13T18:28:23.886 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.882+0000 7fe16b6aa740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616873facb0 con 0x561687496e20 2024-09-13T18:28:23.886 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.883+0000 7fe161d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561687496e20 0x5616874a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.883+0000 7fe161d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561687496e20 0x5616874a1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:38242/0 (socket says 172.21.15.114:38242) 2024-09-13T18:28:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.883+0000 7fe161d9b640 1 -- 172.21.15.114:0/1970423048 learned_addr learned my addr 172.21.15.114:0/1970423048 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.884+0000 7fe161d9b640 1 -- 172.21.15.114:0/1970423048 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5616874a16c0 msgr2=0x5616874a3ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:28:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.884+0000 7fe161d9b640 1 --2- 172.21.15.114:0/1970423048 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5616874a16c0 0x5616874a3ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.884+0000 7fe161d9b640 1 -- 172.21.15.114:0/1970423048 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561687496500 msgr2=0x5616874968e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.884+0000 7fe161d9b640 1 --2- 172.21.15.114:0/1970423048 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561687496500 0x5616874968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.884+0000 7fe161d9b640 1 -- 172.21.15.114:0/1970423048 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616873a1910 con 0x561687496e20 2024-09-13T18:28:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.884+0000 7fe16259c640 1 --2- 172.21.15.114:0/2571904006 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe14c0a14d0 0x7fe14c0a3990 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe158005ab0 tx=0x7fe158000bc0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.885+0000 7fe161d9b640 1 --2- 172.21.15.114:0/1970423048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561687496e20 0x5616874a1180 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe15c009da0 tx=0x7fe15c00b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=789cd9232075b2ba server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:23.890 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.887+0000 7fe150ff9640 1 -- 172.21.15.114:0/1970423048 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15c013410 con 0x561687496e20 2024-09-13T18:28:23.890 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.887+0000 7fe150ff9640 1 -- 172.21.15.114:0/1970423048 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe15c013a00 con 0x561687496e20 2024-09-13T18:28:23.891 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.888+0000 7fe16b6aa740 1 -- 172.21.15.114:0/1970423048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561687496e20 msgr2=0x5616874a1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:23.891 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.888+0000 7fe16b6aa740 1 --2- 172.21.15.114:0/1970423048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561687496e20 0x5616874a1180 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe15c009da0 tx=0x7fe15c00b820 comp rx=0 tx=0).stop 2024-09-13T18:28:23.901 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.896+0000 7fe16b6aa740 1 -- 172.21.15.114:0/1970423048 shutdown_connections 2024-09-13T18:28:23.901 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.896+0000 7fe16b6aa740 1 --2- 172.21.15.114:0/1970423048 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5616874a16c0 0x5616874a3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:23.901 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.896+0000 7fe16b6aa740 1 --2- 172.21.15.114:0/1970423048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561687496e20 0x5616874a1180 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:23.902 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.896+0000 7fe16b6aa740 1 --2- 172.21.15.114:0/1970423048 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561687496500 0x5616874968e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:23.902 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.896+0000 7fe16b6aa740 1 -- 172.21.15.114:0/1970423048 >> 172.21.15.114:0/1970423048 conn(0x561687492a90 msgr2=0x561687492e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:23.902 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.898+0000 7fe16b6aa740 1 -- 172.21.15.114:0/1970423048 shutdown_connections 2024-09-13T18:28:23.906 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.902+0000 7fe16b6aa740 1 -- 172.21.15.114:0/1970423048 wait complete. 2024-09-13T18:28:23.906 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.902+0000 7fe16b6aa740 1 -- start start 2024-09-13T18:28:23.906 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.903+0000 7fe16b6aa740 1 Processor -- start 2024-09-13T18:28:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.905+0000 7fe16b6aa740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561687496500 0x5616874a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.905+0000 7fe16b6aa740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561687496e20 0x56168749e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.905+0000 7fe16b6aa740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5616874a16c0 0x56168749e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.905+0000 7fe16b6aa740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616873b8e80 con 0x561687496e20 2024-09-13T18:28:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.905+0000 7fe16b6aa740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616873b8b20 con 0x5616874a16c0 2024-09-13T18:28:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.905+0000 7fe16b6aa740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56168737e260 con 0x561687496500 2024-09-13T18:28:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.905+0000 7fe16b6aa740 10 client.0 _ll_register_callbacks cb 0x5616873b3df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:28:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.905+0000 7fe161d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561687496e20 0x56168749e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.905+0000 7fe161d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561687496e20 0x56168749e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59468/0 (socket says 172.21.15.114:59468) 2024-09-13T18:28:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.905+0000 7fe161d9b640 1 -- 172.21.15.114:0/1771067447 learned_addr learned my addr 172.21.15.114:0/1771067447 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.905+0000 7fe162d9d640 1 --2- 172.21.15.114:0/1771067447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5616874a16c0 0x56168749e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.906+0000 7fe161d9b640 1 -- 172.21.15.114:0/1771067447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561687496500 msgr2=0x5616874a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.906+0000 7fe161d9b640 1 --2- 172.21.15.114:0/1771067447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561687496500 0x5616874a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.906+0000 7fe161d9b640 1 -- 172.21.15.114:0/1771067447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5616874a16c0 msgr2=0x56168749e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.906+0000 7fe161d9b640 1 --2- 172.21.15.114:0/1771067447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5616874a16c0 0x56168749e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.906+0000 7fe161d9b640 1 -- 172.21.15.114:0/1771067447 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe15c0099d0 con 0x561687496e20 2024-09-13T18:28:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.906+0000 7fe161d9b640 1 --2- 172.21.15.114:0/1771067447 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561687496e20 0x56168749e2a0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fe15c016040 tx=0x7fe15c012550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24c0c6b14566ab29 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.907+0000 7fe149ffb640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:28:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.907+0000 7fe149ffb640 1 -- 172.21.15.114:0/1771067447 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15c0123b0 con 0x561687496e20 2024-09-13T18:28:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.908+0000 7fe16b6aa740 1 -- 172.21.15.114:0/1771067447 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561687423680 con 0x561687496e20 2024-09-13T18:28:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.908+0000 7fe149ffb640 1 -- 172.21.15.114:0/1771067447 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fe15c012990 con 0x561687496e20 2024-09-13T18:28:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.910+0000 7fe149ffb640 1 -- 172.21.15.114:0/1771067447 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe15c007000 con 0x561687496e20 2024-09-13T18:28:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.910+0000 7fe16b6aa740 10 client.17406 fetch_fsmap learned FSMap version 34 2024-09-13T18:28:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.910+0000 7fe16b6aa740 1 -- 172.21.15.114:0/1771067447 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5616873f9330 con 0x561687496e20 2024-09-13T18:28:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.910+0000 7fe149ffb640 1 -- 172.21.15.114:0/1771067447 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561687423680 con 0x561687496e20 2024-09-13T18:28:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.910+0000 7fe149ffb640 1 -- 172.21.15.114:0/1771067447 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe15c03e430 con 0x561687496e20 2024-09-13T18:28:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.911+0000 7fe16b6aa740 10 client.17406 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:28:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.911+0000 7fe16b6aa740 10 client.17406 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:28:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.911+0000 7fe16b6aa740 20 client.17406 populate_metadata read hostname 'smithi114' 2024-09-13T18:28:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.911+0000 7fe16b6aa740 1 --2- 172.21.15.114:0/1771067447 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5616874b1060 0x5616874b3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.911+0000 7fe16b6aa740 4 client.17406 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:28:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.911+0000 7fe16b6aa740 1 -- 172.21.15.114:0/1771067447 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5616873ee5d0 con 0x5616874b1060 2024-09-13T18:28:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.911+0000 7fe16259c640 1 --2- 172.21.15.114:0/1771067447 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5616874b1060 0x5616874b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:23.916 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.912+0000 7fe16259c640 1 --2- 172.21.15.114:0/1771067447 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5616874b1060 0x5616874b3480 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45db90620c5b3095 server_cookie=b61b58c522573fee in_seq=0 out_seq=0 2024-09-13T18:28:23.916 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.912+0000 7fe149ffb640 10 client.17406 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:28:23.916 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.913+0000 7fe149ffb640 1 -- 172.21.15.114:0/1771067447 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5616873ee5d0 con 0x5616874b1060 2024-09-13T18:28:23.916 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.913+0000 7fe149ffb640 10 client.17406 handle_command_reply: tid=0 2024-09-13T18:28:23.916 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.913+0000 7fe16b6aa740 1 client.17406 shutdown 2024-09-13T18:28:23.918 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.915+0000 7fe16b6aa740 1 -- 172.21.15.114:0/1771067447 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561687496e20 msgr2=0x56168749e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:23.918 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.915+0000 7fe16b6aa740 1 --2- 172.21.15.114:0/1771067447 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561687496e20 0x56168749e2a0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fe15c016040 tx=0x7fe15c012550 comp rx=0 tx=0).stop 2024-09-13T18:28:23.920 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.917+0000 7fe16b6aa740 1 -- 172.21.15.114:0/1771067447 shutdown_connections 2024-09-13T18:28:23.920 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.917+0000 7fe16b6aa740 1 --2- 172.21.15.114:0/1771067447 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5616874b1060 0x5616874b3480 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:23.920 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.917+0000 7fe16b6aa740 1 --2- 172.21.15.114:0/1771067447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5616874a16c0 0x56168749e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:23.920 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.917+0000 7fe16b6aa740 1 --2- 172.21.15.114:0/1771067447 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561687496e20 0x56168749e2a0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:23.920 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.917+0000 7fe16b6aa740 1 --2- 172.21.15.114:0/1771067447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561687496500 0x5616874a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:23.920 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.917+0000 7fe16b6aa740 1 -- 172.21.15.114:0/1771067447 >> 172.21.15.114:0/1771067447 conn(0x561687492a90 msgr2=0x561687491d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:23.922 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.919+0000 7fe149ffb640 0 client.17406 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:28:23.922 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.919+0000 7fe16b6aa740 1 -- 172.21.15.114:0/1771067447 shutdown_connections 2024-09-13T18:28:23.923 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.919+0000 7fe16b6aa740 1 -- 172.21.15.114:0/1771067447 wait complete. 2024-09-13T18:28:23.923 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.920+0000 7fe16b6aa740 20 client.17406 trim_cache size 0 max 16384 2024-09-13T18:28:23.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:23 smithi204 ceph-mon[34777]: pgmap v180: 129 pgs: 129 active+clean; 19 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 33 MiB/s wr, 653 op/s 2024-09-13T18:28:23.957 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.952+0000 7fe16b6aa740 1 Processor -- start 2024-09-13T18:28:23.958 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.952+0000 7fe16b6aa740 1 -- start start 2024-09-13T18:28:23.958 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.953+0000 7fe16b6aa740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561687496500 0x5616874a8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:23.958 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.953+0000 7fe16b6aa740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561687496e20 0x5616874a89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:23.958 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.953+0000 7fe16b6aa740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5616874a16c0 0x561687492e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:23.958 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.953+0000 7fe16b6aa740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616874a4170 con 0x561687496500 2024-09-13T18:28:23.958 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.953+0000 7fe16b6aa740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56168749c7d0 con 0x561687496e20 2024-09-13T18:28:23.958 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.953+0000 7fe16b6aa740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56168749eb30 con 0x5616874a16c0 2024-09-13T18:28:23.958 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.953+0000 7fe162d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5616874a16c0 0x561687492e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:23.958 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.953+0000 7fe162d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5616874a16c0 0x561687492e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:38262/0 (socket says 172.21.15.114:38262) 2024-09-13T18:28:23.958 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.953+0000 7fe162d9d640 1 -- 172.21.15.114:0/3024428546 learned_addr learned my addr 172.21.15.114:0/3024428546 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:23.958 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.953+0000 7fe161d9b640 1 --2- 172.21.15.114:0/3024428546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561687496e20 0x5616874a89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:23.958 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.954+0000 7fe162d9d640 1 -- 172.21.15.114:0/3024428546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561687496e20 msgr2=0x5616874a89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:23.959 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.954+0000 7fe162d9d640 1 --2- 172.21.15.114:0/3024428546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561687496e20 0x5616874a89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:23.959 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.954+0000 7fe162d9d640 1 -- 172.21.15.114:0/3024428546 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561687496500 msgr2=0x5616874a8490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:28:23.959 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.954+0000 7fe162d9d640 1 --2- 172.21.15.114:0/3024428546 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561687496500 0x5616874a8490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:23.959 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.954+0000 7fe162d9d640 1 -- 172.21.15.114:0/3024428546 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56168739b300 con 0x5616874a16c0 2024-09-13T18:28:23.959 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.954+0000 7fe162d9d640 1 --2- 172.21.15.114:0/3024428546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5616874a16c0 0x561687492e70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe154006fd0 tx=0x7fe1540991a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ef3318104a30c57 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:23.959 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.954+0000 7fe1517fa640 1 -- 172.21.15.114:0/3024428546 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe154099780 con 0x5616874a16c0 2024-09-13T18:28:23.959 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.954+0000 7fe1517fa640 1 -- 172.21.15.114:0/3024428546 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe15409d5b0 con 0x5616874a16c0 2024-09-13T18:28:23.959 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.956+0000 7fe16b6aa740 1 -- 172.21.15.114:0/3024428546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5616874a16c0 msgr2=0x561687492e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:23.960 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.956+0000 7fe16b6aa740 1 --2- 172.21.15.114:0/3024428546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5616874a16c0 0x561687492e70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe154006fd0 tx=0x7fe1540991a0 comp rx=0 tx=0).stop 2024-09-13T18:28:23.960 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.957+0000 7fe16b6aa740 1 -- 172.21.15.114:0/3024428546 shutdown_connections 2024-09-13T18:28:23.961 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.957+0000 7fe16b6aa740 1 --2- 172.21.15.114:0/3024428546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5616874a16c0 0x561687492e70 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:23.961 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.957+0000 7fe16b6aa740 1 --2- 172.21.15.114:0/3024428546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561687496e20 0x5616874a89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:23.961 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.957+0000 7fe16b6aa740 1 --2- 172.21.15.114:0/3024428546 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561687496500 0x5616874a8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:23.961 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.957+0000 7fe16b6aa740 1 -- 172.21.15.114:0/3024428546 >> 172.21.15.114:0/3024428546 conn(0x561687492a90 msgr2=0x5616874b7de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:23.961 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.958+0000 7fe16b6aa740 1 -- 172.21.15.114:0/3024428546 shutdown_connections 2024-09-13T18:28:23.963 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.958+0000 7fe16b6aa740 1 -- 172.21.15.114:0/3024428546 wait complete. 2024-09-13T18:28:23.963 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.958+0000 7fe16b6aa740 1 -- start start 2024-09-13T18:28:23.963 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.958+0000 7fe16b6aa740 1 Processor -- start 2024-09-13T18:28:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.959+0000 7fe16b6aa740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561687496500 0x5616874a8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.959+0000 7fe16b6aa740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561687496e20 0x5616874a69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.959+0000 7fe16b6aa740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616874a16c0 0x5616874a6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.959+0000 7fe16b6aa740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616874a7450 con 0x5616874a16c0 2024-09-13T18:28:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.959+0000 7fe16b6aa740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616874a75c0 con 0x561687496e20 2024-09-13T18:28:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.959+0000 7fe16b6aa740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616874a7730 con 0x561687496500 2024-09-13T18:28:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.959+0000 7fe16b6aa740 10 client.0 _ll_register_callbacks cb 0x7fe15c009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:28:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.959+0000 7fe162d9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616874a16c0 0x5616874a6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.959+0000 7fe162d9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616874a16c0 0x5616874a6f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59474/0 (socket says 172.21.15.114:59474) 2024-09-13T18:28:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.959+0000 7fe162d9d640 1 -- 172.21.15.114:0/3627894257 learned_addr learned my addr 172.21.15.114:0/3627894257 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.959+0000 7fe16259c640 1 --2- 172.21.15.114:0/3627894257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561687496500 0x5616874a8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.960+0000 7fe162d9d640 1 -- 172.21.15.114:0/3627894257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561687496500 msgr2=0x5616874a8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.960+0000 7fe162d9d640 1 --2- 172.21.15.114:0/3627894257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561687496500 0x5616874a8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.960+0000 7fe162d9d640 1 -- 172.21.15.114:0/3627894257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561687496e20 msgr2=0x5616874a69d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:28:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.960+0000 7fe162d9d640 1 --2- 172.21.15.114:0/3627894257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561687496e20 0x5616874a69d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.960+0000 7fe162d9d640 1 -- 172.21.15.114:0/3627894257 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1641a97c0 con 0x5616874a16c0 2024-09-13T18:28:23.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.961+0000 7fe162d9d640 1 --2- 172.21.15.114:0/3627894257 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616874a16c0 0x5616874a6f10 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fe1540a16d0 tx=0x7fe154099210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64fc2cfabbeadfbd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:23.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.961+0000 7fe149ffb640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:28:23.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.961+0000 7fe149ffb640 1 -- 172.21.15.114:0/3627894257 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1540c9f30 con 0x5616874a16c0 2024-09-13T18:28:23.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.961+0000 7fe16b6aa740 1 -- 172.21.15.114:0/3627894257 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe15c013a00 con 0x5616874a16c0 2024-09-13T18:28:23.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.961+0000 7fe149ffb640 1 -- 172.21.15.114:0/3627894257 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fe1540ca110 con 0x5616874a16c0 2024-09-13T18:28:23.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.962+0000 7fe149ffb640 1 -- 172.21.15.114:0/3627894257 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe154099780 con 0x5616874a16c0 2024-09-13T18:28:23.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.963+0000 7fe16b6aa740 10 client.17409 fetch_fsmap learned FSMap version 34 2024-09-13T18:28:23.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.963+0000 7fe16b6aa740 1 -- 172.21.15.114:0/3627894257 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5616873c08e0 con 0x5616874a16c0 2024-09-13T18:28:23.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.964+0000 7fe149ffb640 1 -- 172.21.15.114:0/3627894257 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe15c013a00 con 0x5616874a16c0 2024-09-13T18:28:23.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.964+0000 7fe149ffb640 1 -- 172.21.15.114:0/3627894257 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe1540986e0 con 0x5616874a16c0 2024-09-13T18:28:23.968 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.965+0000 7fe16b6aa740 10 client.17409 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:28:23.968 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.965+0000 7fe16b6aa740 10 client.17409 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:28:23.968 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.965+0000 7fe16b6aa740 20 client.17409 populate_metadata read hostname 'smithi114' 2024-09-13T18:28:23.968 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.965+0000 7fe16b6aa740 1 --2- 172.21.15.114:0/3627894257 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5616875087a0 0x56168750abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:23.968 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.965+0000 7fe16b6aa740 4 client.17409 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T18:28:23.968 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.965+0000 7fe16b6aa740 1 -- 172.21.15.114:0/3627894257 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5616870ee300 con 0x5616875087a0 2024-09-13T18:28:23.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:23 smithi114 ceph-mon[30897]: pgmap v180: 129 pgs: 129 active+clean; 19 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 33 MiB/s wr, 653 op/s 2024-09-13T18:28:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.967+0000 7fe16259c640 1 --2- 172.21.15.114:0/3627894257 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5616875087a0 0x56168750abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:23.974 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.971+0000 7fe16259c640 1 --2- 172.21.15.114:0/3627894257 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5616875087a0 0x56168750abc0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=861295ad43f92013 server_cookie=1e1e21fa20196275 in_seq=0 out_seq=0 2024-09-13T18:28:23.974 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.971+0000 7fe149ffb640 10 client.17409 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:28:23.991 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.988+0000 7fe149ffb640 1 -- 172.21.15.114:0/3627894257 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5616870ee300 con 0x5616875087a0 2024-09-13T18:28:23.991 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.988+0000 7fe149ffb640 10 client.17409 handle_command_reply: tid=0 2024-09-13T18:28:23.992 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.989+0000 7fe16b6aa740 1 client.17409 shutdown 2024-09-13T18:28:23.999 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.996+0000 7fe16b6aa740 1 -- 172.21.15.114:0/3627894257 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616874a16c0 msgr2=0x5616874a6f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:23.999 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:23.996+0000 7fe16b6aa740 1 --2- 172.21.15.114:0/3627894257 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616874a16c0 0x5616874a6f10 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fe1540a16d0 tx=0x7fe154099210 comp rx=0 tx=0).stop 2024-09-13T18:28:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:24.001+0000 7fe16b6aa740 1 -- 172.21.15.114:0/3627894257 shutdown_connections 2024-09-13T18:28:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:24.001+0000 7fe16b6aa740 1 --2- 172.21.15.114:0/3627894257 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5616875087a0 0x56168750abc0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:24.001+0000 7fe16b6aa740 1 --2- 172.21.15.114:0/3627894257 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616874a16c0 0x5616874a6f10 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:24.001+0000 7fe16b6aa740 1 --2- 172.21.15.114:0/3627894257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561687496e20 0x5616874a69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:24.001+0000 7fe16b6aa740 1 --2- 172.21.15.114:0/3627894257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561687496500 0x5616874a8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:24.001+0000 7fe16b6aa740 1 -- 172.21.15.114:0/3627894257 >> 172.21.15.114:0/3627894257 conn(0x561687492a90 msgr2=0x561687492e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:24.004+0000 7fe149ffb640 0 client.17409 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:28:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:24.004+0000 7fe16b6aa740 1 -- 172.21.15.114:0/3627894257 shutdown_connections 2024-09-13T18:28:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:24.004+0000 7fe16b6aa740 1 -- 172.21.15.114:0/3627894257 wait complete. 2024-09-13T18:28:24.017 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-13T18:28:24.017 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-13T18:28:24.017 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-13T18:28:24.017 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-13T18:28:24.017 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:24.012+0000 7fe16b6aa740 20 client.17409 trim_cache size 0 max 16384 2024-09-13T18:28:24.019 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:24.016+0000 7fe16359e640 1 -- 172.21.15.114:0/2571904006 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe14c0a14d0 msgr2=0x7fe14c0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:24.019 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:24.016+0000 7fe16359e640 1 --2- 172.21.15.114:0/2571904006 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe14c0a14d0 0x7fe14c0a3990 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe158005ab0 tx=0x7fe158000bc0 comp rx=0 tx=0).stop 2024-09-13T18:28:24.019 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:24.016+0000 7fe16359e640 1 -- 172.21.15.114:0/2571904006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe16410f2a0 msgr2=0x7fe16410f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:24.019 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:24.016+0000 7fe16359e640 1 --2- 172.21.15.114:0/2571904006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe16410f2a0 0x7fe16410f750 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe15400b3e0 tx=0x7fe154009f90 comp rx=0 tx=0).stop 2024-09-13T18:28:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:24.018+0000 7fe16359e640 1 -- 172.21.15.114:0/2571904006 shutdown_connections 2024-09-13T18:28:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:24.018+0000 7fe16359e640 1 --2- 172.21.15.114:0/2571904006 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe14c0a14d0 0x7fe14c0a3990 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:24.018+0000 7fe16359e640 1 --2- 172.21.15.114:0/2571904006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe16410f2a0 0x7fe16410f750 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:24.022 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:24.018+0000 7fe16359e640 1 --2- 172.21.15.114:0/2571904006 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe16410b4c0 0x7fe16410ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:24.022 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:24.018+0000 7fe16359e640 1 --2- 172.21.15.114:0/2571904006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe164106550 0x7fe164112c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:24.022 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:24.018+0000 7fe16359e640 1 -- 172.21.15.114:0/2571904006 >> 172.21.15.114:0/2571904006 conn(0x7fe164075ae0 msgr2=0x7fe164076e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:24.022 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:24.018+0000 7fe16359e640 1 -- 172.21.15.114:0/2571904006 shutdown_connections 2024-09-13T18:28:24.022 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:24.019+0000 7fe16359e640 1 -- 172.21.15.114:0/2571904006 wait complete. 2024-09-13T18:28:24.118 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:23 smithi134 ceph-mon[34283]: pgmap v180: 129 pgs: 129 active+clean; 19 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 33 MiB/s wr, 653 op/s 2024-09-13T18:28:25.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:25 smithi204 ceph-mon[34777]: pgmap v181: 129 pgs: 129 active+clean; 19 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 39 MiB/s wr, 762 op/s 2024-09-13T18:28:25.974 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:25 smithi114 ceph-mon[30897]: pgmap v181: 129 pgs: 129 active+clean; 19 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 39 MiB/s wr, 762 op/s 2024-09-13T18:28:26.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:25 smithi134 ceph-mon[34283]: pgmap v181: 129 pgs: 129 active+clean; 19 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 39 MiB/s wr, 762 op/s 2024-09-13T18:28:27.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:27 smithi204 ceph-mon[34777]: pgmap v182: 129 pgs: 129 active+clean; 19 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 45 MiB/s wr, 884 op/s 2024-09-13T18:28:27.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:27 smithi114 ceph-mon[30897]: pgmap v182: 129 pgs: 129 active+clean; 19 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 45 MiB/s wr, 884 op/s 2024-09-13T18:28:28.118 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:27 smithi134 ceph-mon[34283]: pgmap v182: 129 pgs: 129 active+clean; 19 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 45 MiB/s wr, 884 op/s 2024-09-13T18:28:29.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:29 smithi204 ceph-mon[34777]: pgmap v183: 129 pgs: 129 active+clean; 19 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 33 MiB/s wr, 665 op/s 2024-09-13T18:28:29.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:29 smithi114 ceph-mon[30897]: pgmap v183: 129 pgs: 129 active+clean; 19 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 33 MiB/s wr, 665 op/s 2024-09-13T18:28:30.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:29 smithi134 ceph-mon[34283]: pgmap v183: 129 pgs: 129 active+clean; 19 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 33 MiB/s wr, 665 op/s 2024-09-13T18:28:31.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:31 smithi204 ceph-mon[34777]: pgmap v184: 129 pgs: 129 active+clean; 19 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 50 MiB/s wr, 1.01k op/s 2024-09-13T18:28:31.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:31 smithi204 ceph-mon[34777]: scrub summary: idle 2024-09-13T18:28:31.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:31 smithi114 ceph-mon[30897]: pgmap v184: 129 pgs: 129 active+clean; 19 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 50 MiB/s wr, 1.01k op/s 2024-09-13T18:28:31.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:31 smithi114 ceph-mon[30897]: scrub summary: idle 2024-09-13T18:28:32.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:31 smithi134 ceph-mon[34283]: pgmap v184: 129 pgs: 129 active+clean; 19 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 50 MiB/s wr, 1.01k op/s 2024-09-13T18:28:32.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:31 smithi134 ceph-mon[34283]: scrub summary: idle 2024-09-13T18:28:32.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:32 smithi204 ceph-mon[34777]: scrub summary: idle 2024-09-13T18:28:32.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:32 smithi204 ceph-mon[34777]: scrub summary: idle 2024-09-13T18:28:32.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:32 smithi114 ceph-mon[30897]: scrub summary: idle 2024-09-13T18:28:32.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:32 smithi114 ceph-mon[30897]: scrub summary: idle 2024-09-13T18:28:33.118 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:32 smithi134 ceph-mon[34283]: scrub summary: idle 2024-09-13T18:28:33.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:32 smithi134 ceph-mon[34283]: scrub summary: idle 2024-09-13T18:28:33.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:33 smithi204 ceph-mon[34777]: pgmap v185: 129 pgs: 129 active+clean; 19 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 34 MiB/s wr, 684 op/s 2024-09-13T18:28:33.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:33 smithi114 ceph-mon[30897]: pgmap v185: 129 pgs: 129 active+clean; 19 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 34 MiB/s wr, 684 op/s 2024-09-13T18:28:34.118 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:33 smithi134 ceph-mon[34283]: pgmap v185: 129 pgs: 129 active+clean; 19 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 34 MiB/s wr, 684 op/s 2024-09-13T18:28:35.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:35 smithi204 ceph-mon[34777]: pgmap v186: 129 pgs: 129 active+clean; 19 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 39 MiB/s wr, 792 op/s 2024-09-13T18:28:35.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:35 smithi114 ceph-mon[30897]: pgmap v186: 129 pgs: 129 active+clean; 19 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 39 MiB/s wr, 792 op/s 2024-09-13T18:28:36.118 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:35 smithi134 ceph-mon[34283]: pgmap v186: 129 pgs: 129 active+clean; 19 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 39 MiB/s wr, 792 op/s 2024-09-13T18:28:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.233+0000 7fe370b3e640 1 -- 172.21.15.114:0/3695442808 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe36c1074b0 msgr2=0x7fe36c10e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.233+0000 7fe370b3e640 1 --2- 172.21.15.114:0/3695442808 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe36c1074b0 0x7fe36c10e510 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe35c009cd0 tx=0x7fe35c02a930 comp rx=0 tx=0).stop 2024-09-13T18:28:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.243+0000 7fe370b3e640 1 -- 172.21.15.114:0/3695442808 shutdown_connections 2024-09-13T18:28:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.243+0000 7fe370b3e640 1 --2- 172.21.15.114:0/3695442808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe36c119400 0x7fe36c11b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.243+0000 7fe370b3e640 1 --2- 172.21.15.114:0/3695442808 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe36c107890 0x7fe36c107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.243+0000 7fe370b3e640 1 --2- 172.21.15.114:0/3695442808 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe36c1074b0 0x7fe36c10e510 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.243+0000 7fe370b3e640 1 -- 172.21.15.114:0/3695442808 >> 172.21.15.114:0/3695442808 conn(0x7fe36c06ae30 msgr2=0x7fe36c06b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:37.254 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.251+0000 7fe370b3e640 1 -- 172.21.15.114:0/3695442808 shutdown_connections 2024-09-13T18:28:37.258 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.254+0000 7fe370b3e640 1 -- 172.21.15.114:0/3695442808 wait complete. 2024-09-13T18:28:37.258 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.255+0000 7fe370b3e640 1 Processor -- start 2024-09-13T18:28:37.261 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.257+0000 7fe370b3e640 1 -- start start 2024-09-13T18:28:37.261 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.258+0000 7fe370b3e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe36c1074b0 0x7fe36c116780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:37.261 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.258+0000 7fe370b3e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe36c107890 0x7fe36c116ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:37.261 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.258+0000 7fe370b3e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe36c119400 0x7fe36c10f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:37.261 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.258+0000 7fe370b3e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c110040 con 0x7fe36c119400 2024-09-13T18:28:37.261 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.258+0000 7fe370b3e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c1101b0 con 0x7fe36c1074b0 2024-09-13T18:28:37.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.258+0000 7fe370b3e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c110370 con 0x7fe36c107890 2024-09-13T18:28:37.263 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.260+0000 7fe36a59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe36c1074b0 0x7fe36c116780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.260+0000 7fe36a59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe36c1074b0 0x7fe36c116780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:56586/0 (socket says 172.21.15.114:56586) 2024-09-13T18:28:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.260+0000 7fe36a59c640 1 -- 172.21.15.114:0/698667378 learned_addr learned my addr 172.21.15.114:0/698667378 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.264+0000 7fe369d9b640 1 --2- 172.21.15.114:0/698667378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe36c107890 0x7fe36c116ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:37.269 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.266+0000 7fe36a59c640 1 -- 172.21.15.114:0/698667378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe36c107890 msgr2=0x7fe36c116ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:37.269 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.266+0000 7fe36a59c640 1 --2- 172.21.15.114:0/698667378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe36c107890 0x7fe36c116ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:37.269 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.266+0000 7fe36a59c640 1 -- 172.21.15.114:0/698667378 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe36c119400 msgr2=0x7fe36c10f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:37.270 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.266+0000 7fe36a59c640 1 --2- 172.21.15.114:0/698667378 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe36c119400 0x7fe36c10f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:37.270 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.266+0000 7fe36a59c640 1 -- 172.21.15.114:0/698667378 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe35c009900 con 0x7fe36c1074b0 2024-09-13T18:28:37.275 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.272+0000 7fe36a59c640 1 --2- 172.21.15.114:0/698667378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe36c1074b0 0x7fe36c116780 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe35c009ca0 tx=0x7fe35c030b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:37.275 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.272+0000 7fe35b7fe640 1 -- 172.21.15.114:0/698667378 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35c02fa10 con 0x7fe36c1074b0 2024-09-13T18:28:37.279 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.272+0000 7fe370b3e640 1 -- 172.21.15.114:0/698667378 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe36c1105f0 con 0x7fe36c1074b0 2024-09-13T18:28:37.279 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.273+0000 7fe370b3e640 1 -- 172.21.15.114:0/698667378 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe36c1145b0 con 0x7fe36c1074b0 2024-09-13T18:28:37.279 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.276+0000 7fe37385e740 1 Processor -- start 2024-09-13T18:28:37.279 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.276+0000 7fe35b7fe640 1 -- 172.21.15.114:0/698667378 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe35c03f070 con 0x7fe36c1074b0 2024-09-13T18:28:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.276+0000 7fe35b7fe640 1 -- 172.21.15.114:0/698667378 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35c045ac0 con 0x7fe36c1074b0 2024-09-13T18:28:37.283 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.279+0000 7fe37385e740 1 -- start start 2024-09-13T18:28:37.283 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.279+0000 7fe37385e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559b3cea5500 0x559b3cea58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:37.283 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.279+0000 7fe37385e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b3cea5e20 0x559b3ceb0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:37.283 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.279+0000 7fe37385e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b3ceb06c0 0x559b3ceb2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:37.283 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.279+0000 7fe37385e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b3ce10c30 con 0x559b3cea5500 2024-09-13T18:28:37.283 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.279+0000 7fe37385e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b3ce0e3c0 con 0x559b3cea5e20 2024-09-13T18:28:37.283 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.279+0000 7fe37385e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b3ce09cb0 con 0x559b3ceb06c0 2024-09-13T18:28:37.284 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.280+0000 7fe36ad9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559b3cea5500 0x559b3cea58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:37.284 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.280+0000 7fe36ad9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559b3cea5500 0x559b3cea58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57716/0 (socket says 172.21.15.114:57716) 2024-09-13T18:28:37.284 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.280+0000 7fe36ad9d640 1 -- 172.21.15.114:0/992443386 learned_addr learned my addr 172.21.15.114:0/992443386 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:37.284 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.280+0000 7fe36ad9d640 1 -- 172.21.15.114:0/992443386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b3ceb06c0 msgr2=0x559b3ceb2ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:28:37.284 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.280+0000 7fe36ad9d640 1 --2- 172.21.15.114:0/992443386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b3ceb06c0 0x559b3ceb2ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:37.284 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.280+0000 7fe36ad9d640 1 -- 172.21.15.114:0/992443386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b3cea5e20 msgr2=0x559b3ceb0180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:28:37.284 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.280+0000 7fe36ad9d640 1 --2- 172.21.15.114:0/992443386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b3cea5e20 0x559b3ceb0180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:37.284 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.280+0000 7fe36ad9d640 1 -- 172.21.15.114:0/992443386 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b3cdb0910 con 0x559b3cea5500 2024-09-13T18:28:37.285 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.283+0000 7fe35b7fe640 1 -- 172.21.15.114:0/698667378 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fe35c00a040 con 0x7fe36c1074b0 2024-09-13T18:28:37.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.283+0000 7fe36ad9d640 1 --2- 172.21.15.114:0/992443386 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559b3cea5500 0x559b3cea58e0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fe364009710 tx=0x7fe364009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beb0e15d2276ce52 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:37.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.284+0000 7fe35b7fe640 1 --2- 172.21.15.114:0/698667378 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe3540a14d0 0x7fe3540a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:37.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.284+0000 7fe35b7fe640 1 -- 172.21.15.114:0/698667378 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fe35c041070 con 0x7fe36c1074b0 2024-09-13T18:28:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.285+0000 7fe36a59c640 1 --2- 172.21.15.114:0/698667378 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe3540a14d0 0x7fe3540a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:37.292 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.289+0000 7fe358ff9640 1 -- 172.21.15.114:0/992443386 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36400bad0 con 0x559b3cea5500 2024-09-13T18:28:37.292 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.289+0000 7fe358ff9640 1 -- 172.21.15.114:0/992443386 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe364009e20 con 0x559b3cea5500 2024-09-13T18:28:37.292 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.289+0000 7fe358ff9640 1 -- 172.21.15.114:0/992443386 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe364010380 con 0x559b3cea5500 2024-09-13T18:28:37.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.290+0000 7fe37385e740 1 -- 172.21.15.114:0/992443386 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559b3cea5500 msgr2=0x559b3cea58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:37.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.290+0000 7fe37385e740 1 --2- 172.21.15.114:0/992443386 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559b3cea5500 0x559b3cea58e0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fe364009710 tx=0x7fe364009bd0 comp rx=0 tx=0).stop 2024-09-13T18:28:37.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.290+0000 7fe36a59c640 1 --2- 172.21.15.114:0/698667378 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe3540a14d0 0x7fe3540a3990 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x559b3cea1590 tx=0x7fe35c0095c0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:37.297 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.294+0000 7fe37385e740 1 -- 172.21.15.114:0/992443386 shutdown_connections 2024-09-13T18:28:37.297 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.294+0000 7fe37385e740 1 --2- 172.21.15.114:0/992443386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b3ceb06c0 0x559b3ceb2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:37.297 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.294+0000 7fe37385e740 1 --2- 172.21.15.114:0/992443386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b3cea5e20 0x559b3ceb0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:37.297 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.294+0000 7fe37385e740 1 --2- 172.21.15.114:0/992443386 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559b3cea5500 0x559b3cea58e0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:37.297 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.294+0000 7fe37385e740 1 -- 172.21.15.114:0/992443386 >> 172.21.15.114:0/992443386 conn(0x559b3cea1a90 msgr2=0x559b3cea1e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:37.298 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.294+0000 7fe37385e740 1 -- 172.21.15.114:0/992443386 shutdown_connections 2024-09-13T18:28:37.298 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.295+0000 7fe37385e740 1 -- 172.21.15.114:0/992443386 wait complete. 2024-09-13T18:28:37.298 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.295+0000 7fe37385e740 1 -- start start 2024-09-13T18:28:37.298 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.295+0000 7fe37385e740 1 Processor -- start 2024-09-13T18:28:37.299 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.295+0000 7fe37385e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b3cea5500 0x559b3ceb00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:37.299 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.295+0000 7fe37385e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b3cea5e20 0x559b3cead3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:37.299 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.295+0000 7fe37385e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559b3ceb06c0 0x559b3cead990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:37.299 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.295+0000 7fe37385e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b3cdc7e80 con 0x559b3ceb06c0 2024-09-13T18:28:37.299 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.295+0000 7fe37385e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b3cdc7b20 con 0x559b3cea5500 2024-09-13T18:28:37.299 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.295+0000 7fe37385e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b3cd8d260 con 0x559b3cea5e20 2024-09-13T18:28:37.299 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.296+0000 7fe37385e740 10 client.0 _ll_register_callbacks cb 0x559b3cdc2df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:28:37.299 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.296+0000 7fe36ad9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b3cea5500 0x559b3ceb00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:37.300 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.296+0000 7fe36ad9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b3cea5500 0x559b3ceb00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:56592/0 (socket says 172.21.15.114:56592) 2024-09-13T18:28:37.300 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.296+0000 7fe36ad9d640 1 -- 172.21.15.114:0/225013113 learned_addr learned my addr 172.21.15.114:0/225013113 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:37.300 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.296+0000 7fe36ad9d640 1 -- 172.21.15.114:0/225013113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b3cea5e20 msgr2=0x559b3cead3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:37.300 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.296+0000 7fe36ad9d640 1 --2- 172.21.15.114:0/225013113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b3cea5e20 0x559b3cead3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:37.300 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.296+0000 7fe36ad9d640 1 -- 172.21.15.114:0/225013113 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559b3ceb06c0 msgr2=0x559b3cead990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:37.300 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.296+0000 7fe36ad9d640 1 --2- 172.21.15.114:0/225013113 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559b3ceb06c0 0x559b3cead990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:37.301 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.296+0000 7fe36ad9d640 1 -- 172.21.15.114:0/225013113 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe364009370 con 0x559b3cea5500 2024-09-13T18:28:37.301 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.297+0000 7fe36ad9d640 1 --2- 172.21.15.114:0/225013113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b3cea5500 0x559b3ceb00b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe364006e70 tx=0x7fe364010490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45ec1ae40803c03a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:37.301 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.297+0000 7fe351ffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:28:37.301 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.298+0000 7fe351ffb640 1 -- 172.21.15.114:0/225013113 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36400bad0 con 0x559b3cea5500 2024-09-13T18:28:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.298+0000 7fe351ffb640 1 -- 172.21.15.114:0/225013113 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fe36400bc30 con 0x559b3cea5500 2024-09-13T18:28:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.298+0000 7fe37385e740 1 -- 172.21.15.114:0/225013113 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559b3ce32680 con 0x559b3cea5500 2024-09-13T18:28:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.299+0000 7fe351ffb640 1 -- 172.21.15.114:0/225013113 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe36400bdf0 con 0x559b3cea5500 2024-09-13T18:28:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.299+0000 7fe37385e740 10 client.26846 fetch_fsmap learned FSMap version 34 2024-09-13T18:28:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.299+0000 7fe37385e740 1 -- 172.21.15.114:0/225013113 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x559b3ce08330 con 0x559b3cea5500 2024-09-13T18:28:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.300+0000 7fe351ffb640 1 -- 172.21.15.114:0/225013113 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559b3ce32680 con 0x559b3cea5500 2024-09-13T18:28:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.300+0000 7fe351ffb640 1 -- 172.21.15.114:0/225013113 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe36402f4b0 con 0x559b3cea5500 2024-09-13T18:28:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.300+0000 7fe37385e740 10 client.26846 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:28:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.300+0000 7fe37385e740 10 client.26846 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:28:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.300+0000 7fe37385e740 20 client.26846 populate_metadata read hostname 'smithi114' 2024-09-13T18:28:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.300+0000 7fe37385e740 1 --2- 172.21.15.114:0/225013113 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559b3cec00f0 0x559b3cec2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.300+0000 7fe37385e740 4 client.26846 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:28:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.300+0000 7fe37385e740 1 -- 172.21.15.114:0/225013113 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559b3cdfd5d0 con 0x559b3cec00f0 2024-09-13T18:28:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.300+0000 7fe369d9b640 1 --2- 172.21.15.114:0/225013113 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559b3cec00f0 0x559b3cec2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.301+0000 7fe369d9b640 1 --2- 172.21.15.114:0/225013113 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559b3cec00f0 0x559b3cec2510 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f605fffc2fbe02fc server_cookie=78e42fefb68241d in_seq=0 out_seq=0 2024-09-13T18:28:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.301+0000 7fe351ffb640 10 client.26846 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:28:37.305 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.302+0000 7fe351ffb640 1 -- 172.21.15.114:0/225013113 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x559b3cdfd5d0 con 0x559b3cec00f0 2024-09-13T18:28:37.305 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.302+0000 7fe351ffb640 10 client.26846 handle_command_reply: tid=0 2024-09-13T18:28:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.303+0000 7fe37385e740 1 client.26846 shutdown 2024-09-13T18:28:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.303+0000 7fe37385e740 1 -- 172.21.15.114:0/225013113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b3cea5500 msgr2=0x559b3ceb00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.303+0000 7fe37385e740 1 --2- 172.21.15.114:0/225013113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b3cea5500 0x559b3ceb00b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe364006e70 tx=0x7fe364010490 comp rx=0 tx=0).stop 2024-09-13T18:28:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.304+0000 7fe37385e740 1 -- 172.21.15.114:0/225013113 shutdown_connections 2024-09-13T18:28:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.304+0000 7fe37385e740 1 --2- 172.21.15.114:0/225013113 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559b3cec00f0 0x559b3cec2510 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.304+0000 7fe37385e740 1 --2- 172.21.15.114:0/225013113 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559b3ceb06c0 0x559b3cead990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.304+0000 7fe37385e740 1 --2- 172.21.15.114:0/225013113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b3cea5e20 0x559b3cead3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.304+0000 7fe37385e740 1 --2- 172.21.15.114:0/225013113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b3cea5500 0x559b3ceb00b0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.304+0000 7fe37385e740 1 -- 172.21.15.114:0/225013113 >> 172.21.15.114:0/225013113 conn(0x559b3cea1a90 msgr2=0x559b3cea0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.304+0000 7fe351ffb640 0 client.26846 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:28:37.308 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.304+0000 7fe37385e740 1 -- 172.21.15.114:0/225013113 shutdown_connections 2024-09-13T18:28:37.308 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.304+0000 7fe37385e740 1 -- 172.21.15.114:0/225013113 wait complete. 2024-09-13T18:28:37.309 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.306+0000 7fe37385e740 20 client.26846 trim_cache size 0 max 16384 2024-09-13T18:28:37.356 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.353+0000 7fe37385e740 1 Processor -- start 2024-09-13T18:28:37.358 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.354+0000 7fe37385e740 1 -- start start 2024-09-13T18:28:37.358 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.354+0000 7fe37385e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559b3cea5500 0x559b3ceafbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:37.358 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.354+0000 7fe37385e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b3cea5e20 0x559b3ce9cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:37.358 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.354+0000 7fe37385e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b3ceb06c0 0x559b3ce9d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:37.358 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.354+0000 7fe37385e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b3cebcea0 con 0x559b3cea5500 2024-09-13T18:28:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.354+0000 7fe37385e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b3ceae7e0 con 0x559b3cea5e20 2024-09-13T18:28:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.354+0000 7fe37385e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b3ceae490 con 0x559b3ceb06c0 2024-09-13T18:28:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.355+0000 7fe369d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b3cea5e20 0x559b3ce9cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.355+0000 7fe369d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b3cea5e20 0x559b3ce9cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:56596/0 (socket says 172.21.15.114:56596) 2024-09-13T18:28:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.355+0000 7fe369d9b640 1 -- 172.21.15.114:0/2127870902 learned_addr learned my addr 172.21.15.114:0/2127870902 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.355+0000 7fe369d9b640 1 -- 172.21.15.114:0/2127870902 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b3ceb06c0 msgr2=0x559b3ce9d1a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:28:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.355+0000 7fe369d9b640 1 --2- 172.21.15.114:0/2127870902 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b3ceb06c0 0x559b3ce9d1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.355+0000 7fe369d9b640 1 -- 172.21.15.114:0/2127870902 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559b3cea5500 msgr2=0x559b3ceafbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.355+0000 7fe369d9b640 1 --2- 172.21.15.114:0/2127870902 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559b3cea5500 0x559b3ceafbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.355+0000 7fe369d9b640 1 -- 172.21.15.114:0/2127870902 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b3cdaa300 con 0x559b3cea5e20 2024-09-13T18:28:37.363 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.360+0000 7fe369d9b640 1 --2- 172.21.15.114:0/2127870902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b3cea5e20 0x559b3ce9cc60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe360005fd0 tx=0x7fe360009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24c255ee122649cd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:37.363 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.360+0000 7fe3597fa640 1 -- 172.21.15.114:0/2127870902 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe360012980 con 0x559b3cea5e20 2024-09-13T18:28:37.363 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.360+0000 7fe3597fa640 1 -- 172.21.15.114:0/2127870902 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe360019320 con 0x559b3cea5e20 2024-09-13T18:28:37.363 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.360+0000 7fe3597fa640 1 -- 172.21.15.114:0/2127870902 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe360005280 con 0x559b3cea5e20 2024-09-13T18:28:37.364 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.361+0000 7fe37385e740 1 -- 172.21.15.114:0/2127870902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b3cea5e20 msgr2=0x559b3ce9cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:37.364 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.361+0000 7fe37385e740 1 --2- 172.21.15.114:0/2127870902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b3cea5e20 0x559b3ce9cc60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe360005fd0 tx=0x7fe360009840 comp rx=0 tx=0).stop 2024-09-13T18:28:37.366 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.362+0000 7fe37385e740 1 -- 172.21.15.114:0/2127870902 shutdown_connections 2024-09-13T18:28:37.366 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.362+0000 7fe37385e740 1 --2- 172.21.15.114:0/2127870902 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b3ceb06c0 0x559b3ce9d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:37.366 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.362+0000 7fe37385e740 1 --2- 172.21.15.114:0/2127870902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b3cea5e20 0x559b3ce9cc60 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:37.366 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.362+0000 7fe37385e740 1 --2- 172.21.15.114:0/2127870902 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559b3cea5500 0x559b3ceafbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:37.366 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.363+0000 7fe37385e740 1 -- 172.21.15.114:0/2127870902 >> 172.21.15.114:0/2127870902 conn(0x559b3cea1a90 msgr2=0x559b3cea1e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:37.370 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.367+0000 7fe37385e740 1 -- 172.21.15.114:0/2127870902 shutdown_connections 2024-09-13T18:28:37.371 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.367+0000 7fe37385e740 1 -- 172.21.15.114:0/2127870902 wait complete. 2024-09-13T18:28:37.371 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.368+0000 7fe37385e740 1 -- start start 2024-09-13T18:28:37.371 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.368+0000 7fe37385e740 1 Processor -- start 2024-09-13T18:28:37.375 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.372+0000 7fe37385e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b3cea5500 0x559b3ceb7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:37.375 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.372+0000 7fe37385e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559b3cea5e20 0x559b3ceb3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.372+0000 7fe36ad9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b3cea5500 0x559b3ceb7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.372+0000 7fe36ad9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b3cea5500 0x559b3ceb7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:40332/0 (socket says 172.21.15.114:40332) 2024-09-13T18:28:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.372+0000 7fe36ad9d640 1 -- 172.21.15.114:0/4116020637 learned_addr learned my addr 172.21.15.114:0/4116020637 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.372+0000 7fe37385e740 1 --2- 172.21.15.114:0/4116020637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b3ceb06c0 0x559b3ceb4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.372+0000 7fe37385e740 1 -- 172.21.15.114:0/4116020637 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b3ceb4810 con 0x559b3cea5e20 2024-09-13T18:28:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.372+0000 7fe37385e740 1 -- 172.21.15.114:0/4116020637 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b3ceb4980 con 0x559b3ceb06c0 2024-09-13T18:28:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.372+0000 7fe37385e740 1 -- 172.21.15.114:0/4116020637 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b3cf16610 con 0x559b3cea5500 2024-09-13T18:28:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.373+0000 7fe36ad9d640 1 --2- 172.21.15.114:0/4116020637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b3ceb06c0 0x559b3ceb4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.373+0000 7fe36ad9d640 1 -- 172.21.15.114:0/4116020637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b3ceb06c0 msgr2=0x559b3ceb4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.373+0000 7fe36ad9d640 1 --2- 172.21.15.114:0/4116020637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b3ceb06c0 0x559b3ceb4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.373+0000 7fe36ad9d640 1 -- 172.21.15.114:0/4116020637 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559b3cea5e20 msgr2=0x559b3ceb3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.373+0000 7fe36ad9d640 1 --2- 172.21.15.114:0/4116020637 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559b3cea5e20 0x559b3ceb3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.373+0000 7fe36ad9d640 1 -- 172.21.15.114:0/4116020637 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe360009d30 con 0x559b3cea5500 2024-09-13T18:28:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.374+0000 7fe36ad9d640 1 --2- 172.21.15.114:0/4116020637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b3cea5500 0x559b3ceb7780 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe36400b690 tx=0x7fe364009fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4bd42c6f7e3d8b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.374+0000 7fe351ffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:28:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.374+0000 7fe351ffb640 1 -- 172.21.15.114:0/4116020637 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe364011a60 con 0x559b3cea5500 2024-09-13T18:28:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.374+0000 7fe351ffb640 1 -- 172.21.15.114:0/4116020637 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fe364004020 con 0x559b3cea5500 2024-09-13T18:28:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.374+0000 7fe351ffb640 10 client.0 inactive, discarding osd_map(94..94 src has 1..94) v4 2024-09-13T18:28:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.375+0000 7fe37385e740 10 client.0 _ll_register_callbacks cb 0x7fe364006ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:28:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.375+0000 7fe37385e740 1 -- 172.21.15.114:0/4116020637 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe364009e20 con 0x559b3cea5500 2024-09-13T18:28:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.375+0000 7fe351ffb640 1 -- 172.21.15.114:0/4116020637 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe36402fb50 con 0x559b3cea5500 2024-09-13T18:28:37.382 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.379+0000 7fe37385e740 10 client.26839 fetch_fsmap learned FSMap version 34 2024-09-13T18:28:37.382 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.379+0000 7fe37385e740 1 -- 172.21.15.114:0/4116020637 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({fsmap=34}) v3 -- 0x559b3cdcf8e0 con 0x559b3cea5500 2024-09-13T18:28:37.383 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.380+0000 7fe351ffb640 1 -- 172.21.15.114:0/4116020637 <== mon.1 v2:172.21.15.204:3300/0 4 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe3640041e0 con 0x559b3cea5500 2024-09-13T18:28:37.383 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.380+0000 7fe37385e740 10 client.26839 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:28:37.383 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.380+0000 7fe37385e740 10 client.26839 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:28:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.380+0000 7fe37385e740 20 client.26839 populate_metadata read hostname 'smithi114' 2024-09-13T18:28:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.380+0000 7fe37385e740 1 --2- 172.21.15.114:0/4116020637 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559b3cf17650 0x559b3cf19b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.380+0000 7fe37385e740 4 client.26839 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T18:28:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.380+0000 7fe37385e740 1 -- 172.21.15.114:0/4116020637 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7fe364009e20 con 0x559b3cf17650 2024-09-13T18:28:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.380+0000 7fe369d9b640 1 --2- 172.21.15.114:0/4116020637 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559b3cf17650 0x559b3cf19b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:37.385 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.382+0000 7fe369d9b640 1 --2- 172.21.15.114:0/4116020637 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559b3cf17650 0x559b3cf19b10 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e416eb7da92506e0 server_cookie=76f47e677f5d42ae in_seq=0 out_seq=0 2024-09-13T18:28:37.385 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.382+0000 7fe351ffb640 10 client.26839 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:28:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.386+0000 7fe351ffb640 1 -- 172.21.15.114:0/4116020637 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7fe364009e20 con 0x559b3cf17650 2024-09-13T18:28:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.386+0000 7fe351ffb640 10 client.26839 handle_command_reply: tid=0 2024-09-13T18:28:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.386+0000 7fe37385e740 1 client.26839 shutdown 2024-09-13T18:28:37.393 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.389+0000 7fe37385e740 1 -- 172.21.15.114:0/4116020637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b3cea5500 msgr2=0x559b3ceb7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:37.393 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.389+0000 7fe37385e740 1 --2- 172.21.15.114:0/4116020637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b3cea5500 0x559b3ceb7780 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe36400b690 tx=0x7fe364009fd0 comp rx=0 tx=0).stop 2024-09-13T18:28:37.393 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.390+0000 7fe37385e740 1 -- 172.21.15.114:0/4116020637 shutdown_connections 2024-09-13T18:28:37.393 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.390+0000 7fe37385e740 1 --2- 172.21.15.114:0/4116020637 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559b3cf17650 0x559b3cf19b10 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:37.394 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.390+0000 7fe351ffb640 0 client.26839 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:28:37.394 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.390+0000 7fe37385e740 1 --2- 172.21.15.114:0/4116020637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b3ceb06c0 0x559b3ceb4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:37.394 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.390+0000 7fe37385e740 1 --2- 172.21.15.114:0/4116020637 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559b3cea5e20 0x559b3ceb3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:37.394 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.390+0000 7fe37385e740 1 --2- 172.21.15.114:0/4116020637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b3cea5500 0x559b3ceb7780 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:37.394 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.390+0000 7fe37385e740 1 -- 172.21.15.114:0/4116020637 >> 172.21.15.114:0/4116020637 conn(0x559b3cea1a90 msgr2=0x559b3ce9cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:37.397 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.394+0000 7fe37385e740 1 -- 172.21.15.114:0/4116020637 shutdown_connections 2024-09-13T18:28:37.398 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.395+0000 7fe37385e740 1 -- 172.21.15.114:0/4116020637 wait complete. 2024-09-13T18:28:37.403 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.400+0000 7fe37385e740 20 client.26839 trim_cache size 0 max 16384 2024-09-13T18:28:37.405 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-13T18:28:37.405 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-13T18:28:37.405 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-13T18:28:37.405 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-13T18:28:37.406 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.403+0000 7fe370b3e640 1 -- 172.21.15.114:0/698667378 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe3540a14d0 msgr2=0x7fe3540a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:37.406 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.403+0000 7fe370b3e640 1 --2- 172.21.15.114:0/698667378 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe3540a14d0 0x7fe3540a3990 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x559b3cea1590 tx=0x7fe35c0095c0 comp rx=0 tx=0).stop 2024-09-13T18:28:37.406 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.403+0000 7fe370b3e640 1 -- 172.21.15.114:0/698667378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe36c1074b0 msgr2=0x7fe36c116780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:37.406 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.403+0000 7fe370b3e640 1 --2- 172.21.15.114:0/698667378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe36c1074b0 0x7fe36c116780 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe35c009ca0 tx=0x7fe35c030b00 comp rx=0 tx=0).stop 2024-09-13T18:28:37.410 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.407+0000 7fe370b3e640 1 -- 172.21.15.114:0/698667378 shutdown_connections 2024-09-13T18:28:37.410 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.407+0000 7fe370b3e640 1 --2- 172.21.15.114:0/698667378 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe3540a14d0 0x7fe3540a3990 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:37.410 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.407+0000 7fe370b3e640 1 --2- 172.21.15.114:0/698667378 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe36c119400 0x7fe36c10f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:37.410 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.407+0000 7fe370b3e640 1 --2- 172.21.15.114:0/698667378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe36c107890 0x7fe36c116ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:37.410 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.407+0000 7fe370b3e640 1 --2- 172.21.15.114:0/698667378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe36c1074b0 0x7fe36c116780 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:37.410 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.407+0000 7fe370b3e640 1 -- 172.21.15.114:0/698667378 >> 172.21.15.114:0/698667378 conn(0x7fe36c06ae30 msgr2=0x7fe36c119980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:37.411 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.408+0000 7fe370b3e640 1 -- 172.21.15.114:0/698667378 shutdown_connections 2024-09-13T18:28:37.411 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:37.408+0000 7fe370b3e640 1 -- 172.21.15.114:0/698667378 wait complete. 2024-09-13T18:28:37.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:37 smithi204 ceph-mon[34777]: pgmap v187: 129 pgs: 129 active+clean; 19 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 45 MiB/s wr, 906 op/s 2024-09-13T18:28:37.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:37 smithi114 ceph-mon[30897]: pgmap v187: 129 pgs: 129 active+clean; 19 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 45 MiB/s wr, 906 op/s 2024-09-13T18:28:38.118 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:37 smithi134 ceph-mon[34283]: pgmap v187: 129 pgs: 129 active+clean; 19 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 45 MiB/s wr, 906 op/s 2024-09-13T18:28:39.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:39 smithi204 ceph-mon[34777]: pgmap v188: 129 pgs: 129 active+clean; 19 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 33 MiB/s wr, 678 op/s 2024-09-13T18:28:39.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:39 smithi114 ceph-mon[30897]: pgmap v188: 129 pgs: 129 active+clean; 19 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 33 MiB/s wr, 678 op/s 2024-09-13T18:28:40.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:39 smithi134 ceph-mon[34283]: pgmap v188: 129 pgs: 129 active+clean; 19 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 33 MiB/s wr, 678 op/s 2024-09-13T18:28:41.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:41 smithi204 ceph-mon[34777]: pgmap v189: 129 pgs: 129 active+clean; 19 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 50 MiB/s wr, 1.01k op/s 2024-09-13T18:28:41.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:41 smithi114 ceph-mon[30897]: pgmap v189: 129 pgs: 129 active+clean; 19 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 50 MiB/s wr, 1.01k op/s 2024-09-13T18:28:42.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:41 smithi134 ceph-mon[34283]: pgmap v189: 129 pgs: 129 active+clean; 19 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 50 MiB/s wr, 1.01k op/s 2024-09-13T18:28:42.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:42 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:28:42.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:42 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:28:42.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:42 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:28:42.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:42 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:28:43.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:42 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:28:43.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:42 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:28:43.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:42 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:28:43.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:42 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:28:43.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:42 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:28:43.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:42 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:28:43.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:42 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:28:43.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:42 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:28:43.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:43 smithi114 ceph-mon[30897]: pgmap v190: 129 pgs: 129 active+clean; 19 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 33 MiB/s wr, 667 op/s 2024-09-13T18:28:44.118 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:43 smithi134 ceph-mon[34283]: pgmap v190: 129 pgs: 129 active+clean; 19 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 33 MiB/s wr, 667 op/s 2024-09-13T18:28:44.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:43 smithi204 ceph-mon[34777]: pgmap v190: 129 pgs: 129 active+clean; 19 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 33 MiB/s wr, 667 op/s 2024-09-13T18:28:45.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:45 smithi114 ceph-mon[30897]: pgmap v191: 129 pgs: 129 active+clean; 19 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 38 MiB/s wr, 779 op/s 2024-09-13T18:28:46.118 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:45 smithi134 ceph-mon[34283]: pgmap v191: 129 pgs: 129 active+clean; 19 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 38 MiB/s wr, 779 op/s 2024-09-13T18:28:46.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:45 smithi204 ceph-mon[34777]: pgmap v191: 129 pgs: 129 active+clean; 19 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 38 MiB/s wr, 779 op/s 2024-09-13T18:28:47.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:47 smithi114 ceph-mon[30897]: pgmap v192: 129 pgs: 129 active+clean; 20 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 43 MiB/s wr, 888 op/s 2024-09-13T18:28:47.995 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T18:28:48.118 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:47 smithi134 ceph-mon[34283]: pgmap v192: 129 pgs: 129 active+clean; 20 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 43 MiB/s wr, 888 op/s 2024-09-13T18:28:48.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:47 smithi204 ceph-mon[34777]: pgmap v192: 129 pgs: 129 active+clean; 20 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 43 MiB/s wr, 888 op/s 2024-09-13T18:28:48.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.244+0000 7ffb10d80640 1 -- 172.21.15.114:0/2665679075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb0c111b70 msgr2=0x7ffb0c113f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:48.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.244+0000 7ffb10d80640 1 --2- 172.21.15.114:0/2665679075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb0c111b70 0x7ffb0c113f60 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ffafc009860 tx=0x7ffafc02a340 comp rx=0 tx=0).stop 2024-09-13T18:28:48.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.245+0000 7ffb10d80640 1 -- 172.21.15.114:0/2665679075 shutdown_connections 2024-09-13T18:28:48.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.245+0000 7ffb10d80640 1 --2- 172.21.15.114:0/2665679075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb0c111b70 0x7ffb0c113f60 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:48.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.245+0000 7ffb10d80640 1 --2- 172.21.15.114:0/2665679075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb0c108ee0 0x7ffb0c111630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:48.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.245+0000 7ffb10d80640 1 --2- 172.21.15.114:0/2665679075 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb0c1085c0 0x7ffb0c1089a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:48.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.245+0000 7ffb10d80640 1 -- 172.21.15.114:0/2665679075 >> 172.21.15.114:0/2665679075 conn(0x7ffb0c06aec0 msgr2=0x7ffb0c06b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:48.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.248+0000 7ffb10d80640 1 -- 172.21.15.114:0/2665679075 shutdown_connections 2024-09-13T18:28:48.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.249+0000 7ffb10d80640 1 -- 172.21.15.114:0/2665679075 wait complete. 2024-09-13T18:28:48.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.250+0000 7ffb10d80640 1 Processor -- start 2024-09-13T18:28:48.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.250+0000 7ffb10d80640 1 -- start start 2024-09-13T18:28:48.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.250+0000 7ffb10d80640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb0c1085c0 0x7ffb0c111490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:48.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.250+0000 7ffb10d80640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb0c108ee0 0x7ffb0c121d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:48.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.250+0000 7ffb10d80640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb0c111b70 0x7ffb0c10f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:48.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.251+0000 7ffb10d80640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c10fb20 con 0x7ffb0c1085c0 2024-09-13T18:28:48.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.251+0000 7ffb10d80640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c10fc90 con 0x7ffb0c108ee0 2024-09-13T18:28:48.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.251+0000 7ffb10d80640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c10fe50 con 0x7ffb0c111b70 2024-09-13T18:28:48.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.251+0000 7ffb0affd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb0c108ee0 0x7ffb0c121d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:48.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.251+0000 7ffb0affd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb0c108ee0 0x7ffb0c121d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:55750/0 (socket says 172.21.15.114:55750) 2024-09-13T18:28:48.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.251+0000 7ffb0affd640 1 -- 172.21.15.114:0/144133039 learned_addr learned my addr 172.21.15.114:0/144133039 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:48.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.251+0000 7ffb0affd640 1 -- 172.21.15.114:0/144133039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb0c111b70 msgr2=0x7ffb0c10f520 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:28:48.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.251+0000 7ffb0affd640 1 --2- 172.21.15.114:0/144133039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb0c111b70 0x7ffb0c10f520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:48.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.251+0000 7ffb0affd640 1 -- 172.21.15.114:0/144133039 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb0c1085c0 msgr2=0x7ffb0c111490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:48.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.251+0000 7ffb0affd640 1 --2- 172.21.15.114:0/144133039 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb0c1085c0 0x7ffb0c111490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:48.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.251+0000 7ffb0affd640 1 -- 172.21.15.114:0/144133039 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffafc009490 con 0x7ffb0c108ee0 2024-09-13T18:28:48.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.252+0000 7ffb0affd640 1 --2- 172.21.15.114:0/144133039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb0c108ee0 0x7ffb0c121d70 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ffb0000e9f0 tx=0x7ffb0000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:48.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.253+0000 7ffb08ff9640 1 -- 172.21.15.114:0/144133039 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0000cd80 con 0x7ffb0c108ee0 2024-09-13T18:28:48.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.253+0000 7ffb08ff9640 1 -- 172.21.15.114:0/144133039 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffb000108c0 con 0x7ffb0c108ee0 2024-09-13T18:28:48.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.253+0000 7ffb08ff9640 1 -- 172.21.15.114:0/144133039 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb00018710 con 0x7ffb0c108ee0 2024-09-13T18:28:48.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.253+0000 7ffb10d80640 1 -- 172.21.15.114:0/144133039 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb0c110130 con 0x7ffb0c108ee0 2024-09-13T18:28:48.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.253+0000 7ffb10d80640 1 -- 172.21.15.114:0/144133039 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb0c1ba430 con 0x7ffb0c108ee0 2024-09-13T18:28:48.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.257+0000 7ffb14dda740 1 Processor -- start 2024-09-13T18:28:48.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.257+0000 7ffb14dda740 1 -- start start 2024-09-13T18:28:48.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.258+0000 7ffb14dda740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a73f3f030 0x557a73f3f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:48.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.258+0000 7ffb14dda740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557a73f3f950 0x557a73f49cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:48.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.258+0000 7ffb14dda740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a73f4a1f0 0x557a73f4c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:48.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.258+0000 7ffb14dda740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a73ea9f20 con 0x557a73f3f950 2024-09-13T18:28:48.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.258+0000 7ffb14dda740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a73ea76b0 con 0x557a73f3f030 2024-09-13T18:28:48.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.258+0000 7ffb14dda740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a73ea2fa0 con 0x557a73f4a1f0 2024-09-13T18:28:48.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.259+0000 7ffb0affd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a73f4a1f0 0x557a73f4c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:48.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.259+0000 7ffb0affd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a73f4a1f0 0x557a73f4c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:55286/0 (socket says 172.21.15.114:55286) 2024-09-13T18:28:48.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.259+0000 7ffb0affd640 1 -- 172.21.15.114:0/2180854694 learned_addr learned my addr 172.21.15.114:0/2180854694 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:48.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.259+0000 7ffb0bfff640 1 --2- 172.21.15.114:0/2180854694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a73f3f030 0x557a73f3f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:48.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.261+0000 7ffb0affd640 1 -- 172.21.15.114:0/2180854694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a73f3f030 msgr2=0x557a73f3f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:48.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.261+0000 7ffb0affd640 1 --2- 172.21.15.114:0/2180854694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a73f3f030 0x557a73f3f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:48.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.261+0000 7ffb0affd640 1 -- 172.21.15.114:0/2180854694 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557a73f3f950 msgr2=0x557a73f49cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:48.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.261+0000 7ffb0affd640 1 --2- 172.21.15.114:0/2180854694 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557a73f3f950 0x557a73f49cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:48.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.261+0000 7ffb0affd640 1 -- 172.21.15.114:0/2180854694 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a73e5d840 con 0x557a73f4a1f0 2024-09-13T18:28:48.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.261+0000 7ffb08ff9640 1 -- 172.21.15.114:0/144133039 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7ffb00010a30 con 0x7ffb0c108ee0 2024-09-13T18:28:48.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.262+0000 7ffb08ff9640 1 --2- 172.21.15.114:0/144133039 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ffaf40a1400 0x7ffaf40a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:48.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.262+0000 7ffb08ff9640 1 -- 172.21.15.114:0/144133039 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ffb00014070 con 0x7ffb0c108ee0 2024-09-13T18:28:48.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.264+0000 7ffb0affd640 1 --2- 172.21.15.114:0/2180854694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a73f4a1f0 0x557a73f4c5e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ffb000d11d0 tx=0x7ffb000d46f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f83ee86af321c0b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:48.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.266+0000 7ffb0bfff640 1 --2- 172.21.15.114:0/144133039 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ffaf40a1400 0x7ffaf40a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:48.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.267+0000 7ffaf9ffb640 1 -- 172.21.15.114:0/2180854694 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb00017e90 con 0x557a73f4a1f0 2024-09-13T18:28:48.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.267+0000 7ffaf9ffb640 1 -- 172.21.15.114:0/2180854694 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffb0009ce40 con 0x557a73f4a1f0 2024-09-13T18:28:48.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.267+0000 7ffaf9ffb640 1 -- 172.21.15.114:0/2180854694 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb000a1290 con 0x557a73f4a1f0 2024-09-13T18:28:48.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.268+0000 7ffb14dda740 1 -- 172.21.15.114:0/2180854694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a73f4a1f0 msgr2=0x557a73f4c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:48.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.268+0000 7ffb14dda740 1 --2- 172.21.15.114:0/2180854694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a73f4a1f0 0x557a73f4c5e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ffb000d11d0 tx=0x7ffb000d46f0 comp rx=0 tx=0).stop 2024-09-13T18:28:48.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.269+0000 7ffb14dda740 1 -- 172.21.15.114:0/2180854694 shutdown_connections 2024-09-13T18:28:48.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.269+0000 7ffb14dda740 1 --2- 172.21.15.114:0/2180854694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a73f4a1f0 0x557a73f4c5e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:48.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.269+0000 7ffb0bfff640 1 --2- 172.21.15.114:0/144133039 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ffaf40a1400 0x7ffaf40a38c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ffafc009830 tx=0x7ffafc02a840 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:48.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.269+0000 7ffb14dda740 1 --2- 172.21.15.114:0/2180854694 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557a73f3f950 0x557a73f49cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:48.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.269+0000 7ffb14dda740 1 --2- 172.21.15.114:0/2180854694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a73f3f030 0x557a73f3f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:48.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.269+0000 7ffb14dda740 1 -- 172.21.15.114:0/2180854694 >> 172.21.15.114:0/2180854694 conn(0x557a73f3b5a0 msgr2=0x557a73f3b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:48.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.270+0000 7ffb14dda740 1 -- 172.21.15.114:0/2180854694 shutdown_connections 2024-09-13T18:28:48.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.270+0000 7ffb14dda740 1 -- 172.21.15.114:0/2180854694 wait complete. 2024-09-13T18:28:48.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.271+0000 7ffb14dda740 1 -- start start 2024-09-13T18:28:48.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.271+0000 7ffb14dda740 1 Processor -- start 2024-09-13T18:28:48.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.271+0000 7ffb14dda740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a73f3f030 0x557a73f49b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:48.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.271+0000 7ffb14dda740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557a73f3f950 0x557a73f46eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:48.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.271+0000 7ffb14dda740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a73f4a1f0 0x557a73f47430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:48.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.271+0000 7ffb14dda740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a73e442a0 con 0x557a73f3f950 2024-09-13T18:28:48.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.271+0000 7ffb14dda740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a73b7b430 con 0x557a73f4a1f0 2024-09-13T18:28:48.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.271+0000 7ffb14dda740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a73e26680 con 0x557a73f3f030 2024-09-13T18:28:48.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.271+0000 7ffb14dda740 10 client.0 _ll_register_callbacks cb 0x557a73e9c610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:28:48.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.272+0000 7ffb0affd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557a73f3f950 0x557a73f46eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:48.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.272+0000 7ffb0bfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a73f4a1f0 0x557a73f47430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:48.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.272+0000 7ffb0affd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557a73f3f950 0x557a73f46eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47244/0 (socket says 172.21.15.114:47244) 2024-09-13T18:28:48.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.272+0000 7ffb0affd640 1 -- 172.21.15.114:0/3501977698 learned_addr learned my addr 172.21.15.114:0/3501977698 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:48.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.273+0000 7ffb0bfff640 1 -- 172.21.15.114:0/3501977698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a73f3f030 msgr2=0x557a73f49b70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:28:48.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.273+0000 7ffb0bfff640 1 --2- 172.21.15.114:0/3501977698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a73f3f030 0x557a73f49b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:48.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.273+0000 7ffb0bfff640 1 -- 172.21.15.114:0/3501977698 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557a73f3f950 msgr2=0x557a73f46eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:48.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.273+0000 7ffb0bfff640 1 --2- 172.21.15.114:0/3501977698 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557a73f3f950 0x557a73f46eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:48.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.273+0000 7ffb0bfff640 1 -- 172.21.15.114:0/3501977698 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb0c1ba430 con 0x557a73f4a1f0 2024-09-13T18:28:48.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.273+0000 7ffb0bfff640 1 --2- 172.21.15.114:0/3501977698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a73f4a1f0 0x557a73f47430 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ffafc02a8b0 tx=0x7ffafc005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cde2d8629856c0e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:48.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.273+0000 7ffaeeffd640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:28:48.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.274+0000 7ffaeeffd640 1 -- 172.21.15.114:0/3501977698 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffafc0308f0 con 0x557a73f4a1f0 2024-09-13T18:28:48.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.274+0000 7ffaeeffd640 1 -- 172.21.15.114:0/3501977698 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ffafc004040 con 0x557a73f4a1f0 2024-09-13T18:28:48.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.274+0000 7ffb14dda740 1 -- 172.21.15.114:0/3501977698 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557a73b964e0 con 0x557a73f4a1f0 2024-09-13T18:28:48.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.276+0000 7ffaeeffd640 1 -- 172.21.15.114:0/3501977698 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffafc004200 con 0x557a73f4a1f0 2024-09-13T18:28:48.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.277+0000 7ffb14dda740 10 client.26864 fetch_fsmap learned FSMap version 34 2024-09-13T18:28:48.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.277+0000 7ffb14dda740 1 -- 172.21.15.114:0/3501977698 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x557a73ea1620 con 0x557a73f4a1f0 2024-09-13T18:28:48.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.277+0000 7ffaeeffd640 1 -- 172.21.15.114:0/3501977698 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557a73b964e0 con 0x557a73f4a1f0 2024-09-13T18:28:48.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.277+0000 7ffaeeffd640 1 -- 172.21.15.114:0/3501977698 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ffafc048480 con 0x557a73f4a1f0 2024-09-13T18:28:48.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.277+0000 7ffb14dda740 10 client.26864 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:28:48.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.277+0000 7ffb14dda740 10 client.26864 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:28:48.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.277+0000 7ffb14dda740 20 client.26864 populate_metadata read hostname 'smithi114' 2024-09-13T18:28:48.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.277+0000 7ffb14dda740 1 --2- 172.21.15.114:0/3501977698 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x557a73f59f20 0x557a73f5c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:48.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.277+0000 7ffb14dda740 4 client.26864 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:28:48.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.277+0000 7ffb14dda740 1 -- 172.21.15.114:0/3501977698 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557a73ba77e0 con 0x557a73f59f20 2024-09-13T18:28:48.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.278+0000 7ffb0b7fe640 1 --2- 172.21.15.114:0/3501977698 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x557a73f59f20 0x557a73f5c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:48.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.278+0000 7ffb0b7fe640 1 --2- 172.21.15.114:0/3501977698 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x557a73f59f20 0x557a73f5c360 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c15bd777e3dabf9 server_cookie=48300f1cf5a18607 in_seq=0 out_seq=0 2024-09-13T18:28:48.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.278+0000 7ffaeeffd640 10 client.26864 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:28:48.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.279+0000 7ffaeeffd640 1 -- 172.21.15.114:0/3501977698 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x557a73ba77e0 con 0x557a73f59f20 2024-09-13T18:28:48.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.279+0000 7ffaeeffd640 10 client.26864 handle_command_reply: tid=0 2024-09-13T18:28:48.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.279+0000 7ffb14dda740 1 client.26864 shutdown 2024-09-13T18:28:48.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.281+0000 7ffb14dda740 1 -- 172.21.15.114:0/3501977698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a73f4a1f0 msgr2=0x557a73f47430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:48.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.281+0000 7ffb14dda740 1 --2- 172.21.15.114:0/3501977698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a73f4a1f0 0x557a73f47430 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ffafc02a8b0 tx=0x7ffafc005f90 comp rx=0 tx=0).stop 2024-09-13T18:28:48.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.281+0000 7ffb14dda740 1 -- 172.21.15.114:0/3501977698 shutdown_connections 2024-09-13T18:28:48.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.281+0000 7ffb14dda740 1 --2- 172.21.15.114:0/3501977698 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x557a73f59f20 0x557a73f5c360 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:48.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.281+0000 7ffb14dda740 1 --2- 172.21.15.114:0/3501977698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a73f4a1f0 0x557a73f47430 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:48.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.281+0000 7ffb14dda740 1 --2- 172.21.15.114:0/3501977698 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557a73f3f950 0x557a73f46eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:48.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.281+0000 7ffb14dda740 1 --2- 172.21.15.114:0/3501977698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a73f3f030 0x557a73f49b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:48.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.281+0000 7ffb14dda740 1 -- 172.21.15.114:0/3501977698 >> 172.21.15.114:0/3501977698 conn(0x557a73f3b5a0 msgr2=0x557a73f41300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:48.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.281+0000 7ffaeeffd640 0 client.26864 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:28:48.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.283+0000 7ffb14dda740 1 -- 172.21.15.114:0/3501977698 shutdown_connections 2024-09-13T18:28:48.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.283+0000 7ffb14dda740 1 -- 172.21.15.114:0/3501977698 wait complete. 2024-09-13T18:28:48.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.289+0000 7ffb14dda740 20 client.26864 trim_cache size 0 max 16384 2024-09-13T18:28:48.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.328+0000 7ffb14dda740 1 Processor -- start 2024-09-13T18:28:48.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.332+0000 7ffb14dda740 1 -- start start 2024-09-13T18:28:48.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.332+0000 7ffb14dda740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557a73f3f030 0x557a73f55aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:48.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.332+0000 7ffb14dda740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a73f3f950 0x557a73f45760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:48.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.332+0000 7ffb14dda740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a73f4a1f0 0x557a73f45ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:48.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.333+0000 7ffb14dda740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a73f453e0 con 0x557a73f3f030 2024-09-13T18:28:48.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.333+0000 7ffb14dda740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a73f3f740 con 0x557a73f3f950 2024-09-13T18:28:48.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.333+0000 7ffb0bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a73f4a1f0 0x557a73f45ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:48.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.333+0000 7ffb0bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a73f4a1f0 0x557a73f45ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:55308/0 (socket says 172.21.15.114:55308) 2024-09-13T18:28:48.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.333+0000 7ffb0bfff640 1 -- 172.21.15.114:0/3245747370 learned_addr learned my addr 172.21.15.114:0/3245747370 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:48.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.333+0000 7ffb14dda740 1 -- 172.21.15.114:0/3245747370 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a73f47760 con 0x557a73f4a1f0 2024-09-13T18:28:48.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.333+0000 7ffb0b7fe640 1 --2- 172.21.15.114:0/3245747370 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557a73f3f030 0x557a73f55aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:48.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.334+0000 7ffb0affd640 1 --2- 172.21.15.114:0/3245747370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a73f3f950 0x557a73f45760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:48.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.337+0000 7ffb0b7fe640 1 -- 172.21.15.114:0/3245747370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a73f4a1f0 msgr2=0x557a73f45ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:48.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.337+0000 7ffb0b7fe640 1 --2- 172.21.15.114:0/3245747370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a73f4a1f0 0x557a73f45ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:48.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.337+0000 7ffb0b7fe640 1 -- 172.21.15.114:0/3245747370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a73f3f950 msgr2=0x557a73f45760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:48.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.337+0000 7ffb0b7fe640 1 --2- 172.21.15.114:0/3245747370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a73f3f950 0x557a73f45760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:48.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.337+0000 7ffb0b7fe640 1 -- 172.21.15.114:0/3245747370 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a73e43790 con 0x557a73f3f030 2024-09-13T18:28:48.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.338+0000 7ffb0b7fe640 1 --2- 172.21.15.114:0/3245747370 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557a73f3f030 0x557a73f55aa0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x557a73f5a300 tx=0x7ffb040099a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cfd8f1e2188dd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:48.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.339+0000 7ffafa7fc640 1 -- 172.21.15.114:0/3245747370 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb04016ea0 con 0x557a73f3f030 2024-09-13T18:28:48.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.339+0000 7ffafa7fc640 1 -- 172.21.15.114:0/3245747370 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffb04003e00 con 0x557a73f3f030 2024-09-13T18:28:48.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.339+0000 7ffafa7fc640 1 -- 172.21.15.114:0/3245747370 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb040042b0 con 0x557a73f3f030 2024-09-13T18:28:48.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.341+0000 7ffb14dda740 1 -- 172.21.15.114:0/3245747370 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557a73f3f030 msgr2=0x557a73f55aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:48.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.341+0000 7ffb14dda740 1 --2- 172.21.15.114:0/3245747370 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557a73f3f030 0x557a73f55aa0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x557a73f5a300 tx=0x7ffb040099a0 comp rx=0 tx=0).stop 2024-09-13T18:28:48.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.342+0000 7ffb14dda740 1 -- 172.21.15.114:0/3245747370 shutdown_connections 2024-09-13T18:28:48.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.342+0000 7ffb14dda740 1 --2- 172.21.15.114:0/3245747370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a73f4a1f0 0x557a73f45ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:48.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.342+0000 7ffb14dda740 1 --2- 172.21.15.114:0/3245747370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a73f3f950 0x557a73f45760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:48.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.342+0000 7ffb14dda740 1 --2- 172.21.15.114:0/3245747370 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557a73f3f030 0x557a73f55aa0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:48.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.342+0000 7ffb14dda740 1 -- 172.21.15.114:0/3245747370 >> 172.21.15.114:0/3245747370 conn(0x557a73f3b5a0 msgr2=0x557a73f42170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:48.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.345+0000 7ffb14dda740 1 -- 172.21.15.114:0/3245747370 shutdown_connections 2024-09-13T18:28:48.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.345+0000 7ffb14dda740 1 -- 172.21.15.114:0/3245747370 wait complete. 2024-09-13T18:28:48.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.346+0000 7ffb14dda740 1 -- start start 2024-09-13T18:28:48.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.346+0000 7ffb14dda740 1 Processor -- start 2024-09-13T18:28:48.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.347+0000 7ffb14dda740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a73f3f030 0x557a73f51220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:48.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.347+0000 7ffb14dda740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557a73f3f950 0x557a73f4f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:48.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.347+0000 7ffb0b7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a73f3f030 0x557a73f51220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:48.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.347+0000 7ffb0b7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a73f3f030 0x557a73f51220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:55792/0 (socket says 172.21.15.114:55792) 2024-09-13T18:28:48.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.347+0000 7ffb0b7fe640 1 -- 172.21.15.114:0/1538625872 learned_addr learned my addr 172.21.15.114:0/1538625872 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:48.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.347+0000 7ffb14dda740 1 --2- 172.21.15.114:0/1538625872 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a73f4a1f0 0x557a73f4fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:48.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.347+0000 7ffb14dda740 1 -- 172.21.15.114:0/1538625872 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a73f50190 con 0x557a73f3f950 2024-09-13T18:28:48.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.347+0000 7ffb14dda740 1 -- 172.21.15.114:0/1538625872 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a73f50300 con 0x557a73f3f030 2024-09-13T18:28:48.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.347+0000 7ffb14dda740 1 -- 172.21.15.114:0/1538625872 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a73f504c0 con 0x557a73f4a1f0 2024-09-13T18:28:48.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.347+0000 7ffb0bfff640 1 --2- 172.21.15.114:0/1538625872 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a73f4a1f0 0x557a73f4fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:48.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.347+0000 7ffb0affd640 1 --2- 172.21.15.114:0/1538625872 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557a73f3f950 0x557a73f4f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:48.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.348+0000 7ffb0b7fe640 1 -- 172.21.15.114:0/1538625872 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a73f4a1f0 msgr2=0x557a73f4fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:48.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.348+0000 7ffb0b7fe640 1 --2- 172.21.15.114:0/1538625872 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a73f4a1f0 0x557a73f4fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:48.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.348+0000 7ffb0b7fe640 1 -- 172.21.15.114:0/1538625872 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557a73f3f950 msgr2=0x557a73f4f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:48.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.348+0000 7ffb0b7fe640 1 --2- 172.21.15.114:0/1538625872 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557a73f3f950 0x557a73f4f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:48.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.348+0000 7ffb0b7fe640 1 -- 172.21.15.114:0/1538625872 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb04009b30 con 0x557a73f3f030 2024-09-13T18:28:48.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.348+0000 7ffb14dda740 10 client.0 _ll_register_callbacks cb 0x7ffb04009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:28:48.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.348+0000 7ffb14dda740 1 -- 172.21.15.114:0/1538625872 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffb0009ce40 con 0x557a73f3f030 2024-09-13T18:28:48.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.349+0000 7ffb0b7fe640 1 --2- 172.21.15.114:0/1538625872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a73f3f030 0x557a73f51220 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ffb040092b0 tx=0x7ffb04018ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=773121589845af96 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:48.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.350+0000 7ffaeeffd640 10 client.26870 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:28:48.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.353+0000 7ffaeeffd640 1 -- 172.21.15.114:0/1538625872 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb04024070 con 0x557a73f3f030 2024-09-13T18:28:48.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.353+0000 7ffaeeffd640 1 -- 172.21.15.114:0/1538625872 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ffb04013050 con 0x557a73f3f030 2024-09-13T18:28:48.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.354+0000 7ffaeeffd640 1 -- 172.21.15.114:0/1538625872 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb04003970 con 0x557a73f3f030 2024-09-13T18:28:48.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.357+0000 7ffb14dda740 10 client.26870 fetch_fsmap learned FSMap version 34 2024-09-13T18:28:48.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.357+0000 7ffb14dda740 1 -- 172.21.15.114:0/1538625872 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({fsmap=34}) v3 -- 0x557a73e68ea0 con 0x557a73f3f030 2024-09-13T18:28:48.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.359+0000 7ffaeeffd640 1 -- 172.21.15.114:0/1538625872 <== mon.2 v2:172.21.15.134:3300/0 4 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ffb04003b10 con 0x557a73f3f030 2024-09-13T18:28:48.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.359+0000 7ffb14dda740 10 client.26870 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:28:48.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.359+0000 7ffb14dda740 10 client.26870 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:28:48.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.359+0000 7ffb14dda740 20 client.26870 populate_metadata read hostname 'smithi114' 2024-09-13T18:28:48.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.359+0000 7ffb14dda740 1 --2- 172.21.15.114:0/1538625872 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x557a73fb16f0 0x557a73fb3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:48.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.359+0000 7ffb14dda740 4 client.26870 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T18:28:48.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.359+0000 7ffb14dda740 1 -- 172.21.15.114:0/1538625872 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557a73bfae30 con 0x557a73fb16f0 2024-09-13T18:28:48.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.363+0000 7ffb0affd640 1 --2- 172.21.15.114:0/1538625872 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x557a73fb16f0 0x557a73fb3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:48.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.368+0000 7ffb0affd640 1 --2- 172.21.15.114:0/1538625872 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x557a73fb16f0 0x557a73fb3bb0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa1372f73d77b4b2 server_cookie=ca9b68a4177b8bb8 in_seq=0 out_seq=0 2024-09-13T18:28:48.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.369+0000 7ffaeeffd640 10 client.26870 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:28:48.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.371+0000 7ffaeeffd640 1 -- 172.21.15.114:0/1538625872 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x557a73bfae30 con 0x557a73fb16f0 2024-09-13T18:28:48.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.371+0000 7ffaeeffd640 10 client.26870 handle_command_reply: tid=0 2024-09-13T18:28:48.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.371+0000 7ffb14dda740 1 client.26870 shutdown 2024-09-13T18:28:48.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.373+0000 7ffb14dda740 1 -- 172.21.15.114:0/1538625872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a73f3f030 msgr2=0x557a73f51220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:48.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.373+0000 7ffb14dda740 1 --2- 172.21.15.114:0/1538625872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a73f3f030 0x557a73f51220 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ffb040092b0 tx=0x7ffb04018ce0 comp rx=0 tx=0).stop 2024-09-13T18:28:48.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.376+0000 7ffb14dda740 1 -- 172.21.15.114:0/1538625872 shutdown_connections 2024-09-13T18:28:48.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.376+0000 7ffb14dda740 1 --2- 172.21.15.114:0/1538625872 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x557a73fb16f0 0x557a73fb3bb0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:48.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.376+0000 7ffb14dda740 1 --2- 172.21.15.114:0/1538625872 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a73f4a1f0 0x557a73f4fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:48.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.376+0000 7ffb14dda740 1 --2- 172.21.15.114:0/1538625872 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557a73f3f950 0x557a73f4f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:48.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.376+0000 7ffb14dda740 1 --2- 172.21.15.114:0/1538625872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a73f3f030 0x557a73f51220 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:48.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.376+0000 7ffb14dda740 1 -- 172.21.15.114:0/1538625872 >> 172.21.15.114:0/1538625872 conn(0x557a73f3b5a0 msgr2=0x557a73f456a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:48.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.380+0000 7ffaeeffd640 0 client.26870 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:28:48.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.382+0000 7ffb14dda740 1 -- 172.21.15.114:0/1538625872 shutdown_connections 2024-09-13T18:28:48.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.382+0000 7ffb14dda740 1 -- 172.21.15.114:0/1538625872 wait complete. 2024-09-13T18:28:48.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.387+0000 7ffb14dda740 20 client.26870 trim_cache size 0 max 16384 2024-09-13T18:28:48.391 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:28:48.391 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "no active scrubs running", 2024-09-13T18:28:48.391 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": {} 2024-09-13T18:28:48.391 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:28:48.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.393+0000 7ffb10d80640 1 -- 172.21.15.114:0/144133039 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ffaf40a1400 msgr2=0x7ffaf40a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:48.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.393+0000 7ffb10d80640 1 --2- 172.21.15.114:0/144133039 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ffaf40a1400 0x7ffaf40a38c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ffafc009830 tx=0x7ffafc02a840 comp rx=0 tx=0).stop 2024-09-13T18:28:48.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.393+0000 7ffb10d80640 1 -- 172.21.15.114:0/144133039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb0c108ee0 msgr2=0x7ffb0c121d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:48.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.393+0000 7ffb10d80640 1 --2- 172.21.15.114:0/144133039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb0c108ee0 0x7ffb0c121d70 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ffb0000e9f0 tx=0x7ffb0000eeb0 comp rx=0 tx=0).stop 2024-09-13T18:28:48.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.393+0000 7ffb10d80640 1 -- 172.21.15.114:0/144133039 shutdown_connections 2024-09-13T18:28:48.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.393+0000 7ffb10d80640 1 --2- 172.21.15.114:0/144133039 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ffaf40a1400 0x7ffaf40a38c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:48.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.394+0000 7ffb10d80640 1 --2- 172.21.15.114:0/144133039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb0c111b70 0x7ffb0c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:48.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.394+0000 7ffb10d80640 1 --2- 172.21.15.114:0/144133039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb0c108ee0 0x7ffb0c121d70 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:48.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.394+0000 7ffb10d80640 1 --2- 172.21.15.114:0/144133039 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb0c1085c0 0x7ffb0c111490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:48.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.394+0000 7ffb10d80640 1 -- 172.21.15.114:0/144133039 >> 172.21.15.114:0/144133039 conn(0x7ffb0c06aec0 msgr2=0x7ffb0c06e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:48.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.394+0000 7ffb10d80640 1 -- 172.21.15.114:0/144133039 shutdown_connections 2024-09-13T18:28:48.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.394+0000 7ffb10d80640 1 -- 172.21.15.114:0/144133039 wait complete. 2024-09-13T18:28:48.423 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-13T18:28:48.424 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:28:48.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.672+0000 7f2be6783640 1 -- 172.21.15.114:0/1270084868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2be0119400 msgr2=0x7f2be011b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:48.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.672+0000 7f2be6783640 1 --2- 172.21.15.114:0/1270084868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2be0119400 0x7f2be011b7f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0009be0 tx=0x7f2bd002a6f0 comp rx=0 tx=0).stop 2024-09-13T18:28:48.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.672+0000 7f2be6783640 1 -- 172.21.15.114:0/1270084868 shutdown_connections 2024-09-13T18:28:48.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.672+0000 7f2be6783640 1 --2- 172.21.15.114:0/1270084868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2be0119400 0x7f2be011b7f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:48.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.672+0000 7f2be6783640 1 --2- 172.21.15.114:0/1270084868 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2be0107890 0x7f2be0107cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:48.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.672+0000 7f2be6783640 1 --2- 172.21.15.114:0/1270084868 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2be01074b0 0x7f2be010e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:48.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.672+0000 7f2be6783640 1 -- 172.21.15.114:0/1270084868 >> 172.21.15.114:0/1270084868 conn(0x7f2be006aff0 msgr2=0x7f2be006b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:48.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.678+0000 7f2be6783640 1 -- 172.21.15.114:0/1270084868 shutdown_connections 2024-09-13T18:28:48.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.678+0000 7f2be6783640 1 -- 172.21.15.114:0/1270084868 wait complete. 2024-09-13T18:28:48.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.678+0000 7f2be6783640 1 Processor -- start 2024-09-13T18:28:48.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.679+0000 7f2be6783640 1 -- start start 2024-09-13T18:28:48.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.688+0000 7f2be6783640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2be01074b0 0x7f2be01a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:48.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.688+0000 7f2be6783640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2be0107890 0x7f2be01a5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:48.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.688+0000 7f2be6783640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2be0119400 0x7f2be01a99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:48.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.688+0000 7f2be6783640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be01a9ff0 con 0x7f2be01074b0 2024-09-13T18:28:48.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.688+0000 7f2be6783640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be01aa160 con 0x7f2be0119400 2024-09-13T18:28:48.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.688+0000 7f2be6783640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be01aa320 con 0x7f2be0107890 2024-09-13T18:28:48.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.689+0000 7f2bdf7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2be0107890 0x7f2be01a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:48.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.689+0000 7f2bdf7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2be0107890 0x7f2be01a5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:55336/0 (socket says 172.21.15.114:55336) 2024-09-13T18:28:48.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.689+0000 7f2bdf7fe640 1 -- 172.21.15.114:0/2313499306 learned_addr learned my addr 172.21.15.114:0/2313499306 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:48.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.689+0000 7f2bdffff640 1 --2- 172.21.15.114:0/2313499306 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2be01074b0 0x7f2be01a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:48.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.689+0000 7f2bdffff640 1 -- 172.21.15.114:0/2313499306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2be0107890 msgr2=0x7f2be01a5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:48.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.689+0000 7f2bdffff640 1 --2- 172.21.15.114:0/2313499306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2be0107890 0x7f2be01a5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:48.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.689+0000 7f2bdffff640 1 -- 172.21.15.114:0/2313499306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2be0119400 msgr2=0x7f2be01a99f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:28:48.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.689+0000 7f2bdffff640 1 --2- 172.21.15.114:0/2313499306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2be0119400 0x7f2be01a99f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:48.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.690+0000 7f2bdffff640 1 -- 172.21.15.114:0/2313499306 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd0009840 con 0x7f2be01074b0 2024-09-13T18:28:48.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.690+0000 7f2bdffff640 1 --2- 172.21.15.114:0/2313499306 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2be01074b0 0x7f2be01a50c0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4009fd0 tx=0x7f2bd400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:48.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.691+0000 7f2bdd7fa640 1 -- 172.21.15.114:0/2313499306 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd4011830 con 0x7f2be01074b0 2024-09-13T18:28:48.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.691+0000 7f2bdd7fa640 1 -- 172.21.15.114:0/2313499306 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2bd4011e20 con 0x7f2be01074b0 2024-09-13T18:28:48.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.691+0000 7f2bdd7fa640 1 -- 172.21.15.114:0/2313499306 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd4010e70 con 0x7f2be01074b0 2024-09-13T18:28:48.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.691+0000 7f2be6783640 1 -- 172.21.15.114:0/2313499306 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2be01aa600 con 0x7f2be01074b0 2024-09-13T18:28:48.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.691+0000 7f2be6783640 1 -- 172.21.15.114:0/2313499306 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2be01b1e80 con 0x7f2be01074b0 2024-09-13T18:28:48.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.693+0000 7f2be6783640 1 -- 172.21.15.114:0/2313499306 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bcc005250 con 0x7f2be01074b0 2024-09-13T18:28:48.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.704+0000 7f2bdd7fa640 1 -- 172.21.15.114:0/2313499306 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f2bd4011990 con 0x7f2be01074b0 2024-09-13T18:28:48.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.705+0000 7f2bdd7fa640 1 --2- 172.21.15.114:0/2313499306 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2bc80a1390 0x7f2bc80a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:48.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.705+0000 7f2bdd7fa640 1 -- 172.21.15.114:0/2313499306 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2bd40ccba0 con 0x7f2be01074b0 2024-09-13T18:28:48.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.706+0000 7f2bdf7fe640 1 --2- 172.21.15.114:0/2313499306 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2bc80a1390 0x7f2bc80a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:48.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.707+0000 7f2bdf7fe640 1 --2- 172.21.15.114:0/2313499306 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2bc80a1390 0x7f2bc80a3850 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2be01a6660 tx=0x7f2bd8009260 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:48.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:48.708+0000 7f2bdd7fa640 1 -- 172.21.15.114:0/2313499306 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f2bd40cce60 con 0x7f2be01074b0 2024-09-13T18:28:49.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.165+0000 7f2be6783640 1 -- 172.21.15.114:0/2313499306 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2bcc0050c0 con 0x7f2be01074b0 2024-09-13T18:28:49.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.166+0000 7f2bdd7fa640 1 -- 172.21.15.114:0/2313499306 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+10930 (secure 0 0 0) 0x7f2bd4015020 con 0x7f2be01074b0 2024-09-13T18:28:49.171 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:28:49.171 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4162817497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4162817497},{"type":"v1","addr":"172.21.15.114:6835","nonce":4162817497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/2435428478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":2435428478},{"type":"v1","addr":"172.21.15.114:6837","nonce":2435428478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T18:20:01.895117+0000","modified":"2024-09-13T18:27:22.172031+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24482,"mds_2":24475,"mds_3":24458,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/2046516451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":2046516451},{"type":"v1","addr":"172.21.15.114:6839","nonce":2046516451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":10,"addr":"172.21.15.114:6841/1956923446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":1956923446},{"type":"v1","addr":"172.21.15.114:6841","nonce":1956923446}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24391":{"gid":24391,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/507301496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":507301496},{"type":"v1","addr":"172.21.15.204:6833","nonce":507301496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4028657219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4028657219},{"type":"v1","addr":"172.21.15.134:6833","nonce":4028657219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/57029119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":57029119},{"type":"v1","addr":"172.21.15.204:6835","nonce":57029119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3068880939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3068880939},{"type":"v1","addr":"172.21.15.204:6837","nonce":3068880939}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"e","rank":3,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.134:6835/1150003275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1150003275},{"type":"v1","addr":"172.21.15.134:6835","nonce":1150003275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/344883912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":344883912},{"type":"v1","addr":"172.21.15.204:6839","nonce":344883912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6837/891692210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":891692210},{"type":"v1","addr":"172.21.15.134:6837","nonce":891692210}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6839/3466309724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3466309724},{"type":"v1","addr":"172.21.15.134:6839","nonce":3466309724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:28:49.171 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 34 2024-09-13T18:28:49.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.181+0000 7f2bc6ffd640 1 -- 172.21.15.114:0/2313499306 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2bc80a1390 msgr2=0x7f2bc80a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:49.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.181+0000 7f2bc6ffd640 1 --2- 172.21.15.114:0/2313499306 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2bc80a1390 0x7f2bc80a3850 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2be01a6660 tx=0x7f2bd8009260 comp rx=0 tx=0).stop 2024-09-13T18:28:49.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.181+0000 7f2bc6ffd640 1 -- 172.21.15.114:0/2313499306 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2be01074b0 msgr2=0x7f2be01a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:49.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.181+0000 7f2bc6ffd640 1 --2- 172.21.15.114:0/2313499306 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2be01074b0 0x7f2be01a50c0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4009fd0 tx=0x7f2bd400b870 comp rx=0 tx=0).stop 2024-09-13T18:28:49.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.181+0000 7f2bc6ffd640 1 -- 172.21.15.114:0/2313499306 shutdown_connections 2024-09-13T18:28:49.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.181+0000 7f2bc6ffd640 1 --2- 172.21.15.114:0/2313499306 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2bc80a1390 0x7f2bc80a3850 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.181+0000 7f2bc6ffd640 1 --2- 172.21.15.114:0/2313499306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2be0119400 0x7f2be01a99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.181+0000 7f2bc6ffd640 1 --2- 172.21.15.114:0/2313499306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2be0107890 0x7f2be01a5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.181+0000 7f2bc6ffd640 1 --2- 172.21.15.114:0/2313499306 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2be01074b0 0x7f2be01a50c0 secure :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4009fd0 tx=0x7f2bd400b870 comp rx=0 tx=0).stop 2024-09-13T18:28:49.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.181+0000 7f2bc6ffd640 1 -- 172.21.15.114:0/2313499306 >> 172.21.15.114:0/2313499306 conn(0x7f2be006aff0 msgr2=0x7f2be0119d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:49.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.188+0000 7f2bc6ffd640 1 -- 172.21.15.114:0/2313499306 shutdown_connections 2024-09-13T18:28:49.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.188+0000 7f2bc6ffd640 1 -- 172.21.15.114:0/2313499306 wait complete. 2024-09-13T18:28:49.215 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-09-13T18:28:49.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.422+0000 7f058934c640 1 -- 172.21.15.114:0/1386232966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0584106550 msgr2=0x7f0584106930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:49.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.422+0000 7f058934c640 1 --2- 172.21.15.114:0/1386232966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0584106550 0x7f0584106930 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f057c009860 tx=0x7f057c02a360 comp rx=0 tx=0).stop 2024-09-13T18:28:49.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.423+0000 7f058934c640 1 -- 172.21.15.114:0/1386232966 shutdown_connections 2024-09-13T18:28:49.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.423+0000 7f058934c640 1 --2- 172.21.15.114:0/1386232966 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f058410b650 0x7f05841131a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.423+0000 7f058934c640 1 --2- 172.21.15.114:0/1386232966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0584106f00 0x7f058410aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.423+0000 7f058934c640 1 --2- 172.21.15.114:0/1386232966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0584106550 0x7f0584106930 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.423+0000 7f058934c640 1 -- 172.21.15.114:0/1386232966 >> 172.21.15.114:0/1386232966 conn(0x7f0584075c80 msgr2=0x7f05840780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:49.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.426+0000 7f058934c640 1 -- 172.21.15.114:0/1386232966 shutdown_connections 2024-09-13T18:28:49.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.426+0000 7f058934c640 1 -- 172.21.15.114:0/1386232966 wait complete. 2024-09-13T18:28:49.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.427+0000 7f058934c640 1 Processor -- start 2024-09-13T18:28:49.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.427+0000 7f058934c640 1 -- start start 2024-09-13T18:28:49.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.427+0000 7f058934c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0584106550 0x7f05840719f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:49.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.427+0000 7f058934c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f058410b650 0x7f0584071f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:49.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.427+0000 7f058934c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f058406aac0 0x7f058406af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:49.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.427+0000 7f058934c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f058406b610 con 0x7f0584106550 2024-09-13T18:28:49.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.427+0000 7f058934c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f058406b780 con 0x7f058406aac0 2024-09-13T18:28:49.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.427+0000 7f058934c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f058406b940 con 0x7f058410b650 2024-09-13T18:28:49.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.428+0000 7f0583fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0584106550 0x7f05840719f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:49.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.428+0000 7f0583fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0584106550 0x7f05840719f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47306/0 (socket says 172.21.15.114:47306) 2024-09-13T18:28:49.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.428+0000 7f0583fff640 1 -- 172.21.15.114:0/89467004 learned_addr learned my addr 172.21.15.114:0/89467004 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:49.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.429+0000 7f0588b4b640 1 --2- 172.21.15.114:0/89467004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f058406aac0 0x7f058406af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:49.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.429+0000 7f05837fe640 1 --2- 172.21.15.114:0/89467004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f058410b650 0x7f0584071f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:49.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.430+0000 7f0588b4b640 1 -- 172.21.15.114:0/89467004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f058410b650 msgr2=0x7f0584071f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:49.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.430+0000 7f0588b4b640 1 --2- 172.21.15.114:0/89467004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f058410b650 0x7f0584071f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.430+0000 7f0588b4b640 1 -- 172.21.15.114:0/89467004 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0584106550 msgr2=0x7f05840719f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:49.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.430+0000 7f0588b4b640 1 --2- 172.21.15.114:0/89467004 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0584106550 0x7f05840719f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.430+0000 7f0588b4b640 1 -- 172.21.15.114:0/89467004 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f057c009490 con 0x7f058406aac0 2024-09-13T18:28:49.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.430+0000 7f0588b4b640 1 --2- 172.21.15.114:0/89467004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f058406aac0 0x7f058406af70 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f057400ee70 tx=0x7f057400c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:49.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.431+0000 7f05817fa640 1 -- 172.21.15.114:0/89467004 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05740099a0 con 0x7f058406aac0 2024-09-13T18:28:49.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.431+0000 7f05817fa640 1 -- 172.21.15.114:0/89467004 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0574003970 con 0x7f058406aac0 2024-09-13T18:28:49.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.431+0000 7f058934c640 1 -- 172.21.15.114:0/89467004 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f058406f6d0 con 0x7f058406aac0 2024-09-13T18:28:49.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.431+0000 7f058934c640 1 -- 172.21.15.114:0/89467004 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f058406fbe0 con 0x7f058406aac0 2024-09-13T18:28:49.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.431+0000 7f05817fa640 1 -- 172.21.15.114:0/89467004 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05740046e0 con 0x7f058406aac0 2024-09-13T18:28:49.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.437+0000 7f05817fa640 1 -- 172.21.15.114:0/89467004 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f057400a040 con 0x7f058406aac0 2024-09-13T18:28:49.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.439+0000 7f058d2d0740 1 Processor -- start 2024-09-13T18:28:49.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.442+0000 7f05817fa640 1 --2- 172.21.15.114:0/89467004 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f05700a1340 0x7f05700a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:49.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.442+0000 7f05817fa640 1 -- 172.21.15.114:0/89467004 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f05740ccd40 con 0x7f058406aac0 2024-09-13T18:28:49.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.443+0000 7f058d2d0740 1 -- start start 2024-09-13T18:28:49.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.443+0000 7f058d2d0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563471e84970 0x563471e84d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:49.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.443+0000 7f058d2d0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563471e85290 0x563471e8f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:49.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.443+0000 7f058d2d0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563471e8fb30 0x563471e91f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:49.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.443+0000 7f058d2d0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x563471deecc0 con 0x563471e8fb30 2024-09-13T18:28:49.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.443+0000 7f058d2d0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563471dec450 con 0x563471e84970 2024-09-13T18:28:49.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.443+0000 7f058d2d0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563471de7d40 con 0x563471e85290 2024-09-13T18:28:49.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.443+0000 7f05837fe640 1 --2- 172.21.15.114:0/89467004 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f05700a1340 0x7f05700a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:49.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.443+0000 7f0583fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563471e84970 0x563471e84d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:49.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.443+0000 7f0583fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563471e84970 0x563471e84d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:55850/0 (socket says 172.21.15.114:55850) 2024-09-13T18:28:49.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.443+0000 7f0583fff640 1 -- 172.21.15.114:0/4075081564 learned_addr learned my addr 172.21.15.114:0/4075081564 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:49.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.443+0000 7f0588b4b640 1 --2- 172.21.15.114:0/4075081564 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563471e8fb30 0x563471e91f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:49.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.443+0000 7f0583fff640 1 -- 172.21.15.114:0/4075081564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563471e85290 msgr2=0x563471e8f5f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:28:49.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.443+0000 7f0583fff640 1 --2- 172.21.15.114:0/4075081564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563471e85290 0x563471e8f5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.443+0000 7f0583fff640 1 -- 172.21.15.114:0/4075081564 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563471e8fb30 msgr2=0x563471e91f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:49.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.443+0000 7f0583fff640 1 --2- 172.21.15.114:0/4075081564 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563471e8fb30 0x563471e91f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.443+0000 7f0583fff640 1 -- 172.21.15.114:0/4075081564 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563471daf620 con 0x563471e84970 2024-09-13T18:28:49.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.444+0000 7f0583fff640 1 --2- 172.21.15.114:0/4075081564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563471e84970 0x563471e84d50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f057c035040 tx=0x7f057c005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a00d1840f734b581 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:49.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.444+0000 7f05837fe640 1 --2- 172.21.15.114:0/89467004 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f05700a1340 0x7f05700a3800 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x563471e81d40 tx=0x7f056c00b390 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:49.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.445+0000 7f057a7fc640 1 -- 172.21.15.114:0/4075081564 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057c030b50 con 0x563471e84970 2024-09-13T18:28:49.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.445+0000 7f057a7fc640 1 -- 172.21.15.114:0/4075081564 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f057c030cb0 con 0x563471e84970 2024-09-13T18:28:49.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.445+0000 7f057a7fc640 1 -- 172.21.15.114:0/4075081564 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057c004030 con 0x563471e84970 2024-09-13T18:28:49.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.446+0000 7f058d2d0740 1 -- 172.21.15.114:0/4075081564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563471e84970 msgr2=0x563471e84d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:49.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.446+0000 7f058d2d0740 1 --2- 172.21.15.114:0/4075081564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563471e84970 0x563471e84d50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f057c035040 tx=0x7f057c005c90 comp rx=0 tx=0).stop 2024-09-13T18:28:49.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.446+0000 7f058d2d0740 1 -- 172.21.15.114:0/4075081564 shutdown_connections 2024-09-13T18:28:49.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.446+0000 7f058d2d0740 1 --2- 172.21.15.114:0/4075081564 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563471e8fb30 0x563471e91f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.446+0000 7f058d2d0740 1 --2- 172.21.15.114:0/4075081564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563471e85290 0x563471e8f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.446+0000 7f058d2d0740 1 --2- 172.21.15.114:0/4075081564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563471e84970 0x563471e84d50 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.446+0000 7f058d2d0740 1 -- 172.21.15.114:0/4075081564 >> 172.21.15.114:0/4075081564 conn(0x563471e80ee0 msgr2=0x563471e812c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:49.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.447+0000 7f058d2d0740 1 -- 172.21.15.114:0/4075081564 shutdown_connections 2024-09-13T18:28:49.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.447+0000 7f058d2d0740 1 -- 172.21.15.114:0/4075081564 wait complete. 2024-09-13T18:28:49.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.447+0000 7f058d2d0740 1 -- start start 2024-09-13T18:28:49.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.447+0000 7f058d2d0740 1 Processor -- start 2024-09-13T18:28:49.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.448+0000 7f058d2d0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563471e84970 0x563471e8f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:49.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.448+0000 7f058d2d0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563471e85290 0x563471e8c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:49.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.448+0000 7f058d2d0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563471e8fb30 0x563471e8cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:49.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.448+0000 7f058d2d0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x563471da6180 con 0x563471e84970 2024-09-13T18:28:49.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.448+0000 7f058d2d0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563471ac0430 con 0x563471e8fb30 2024-09-13T18:28:49.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.448+0000 7f058d2d0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563471d6b5e0 con 0x563471e85290 2024-09-13T18:28:49.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.448+0000 7f058d2d0740 10 client.0 _ll_register_callbacks cb 0x563471dd7aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:28:49.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.448+0000 7f0583fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563471e84970 0x563471e8f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:49.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.448+0000 7f0588b4b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563471e8fb30 0x563471e8cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:49.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.448+0000 7f0583fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563471e84970 0x563471e8f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47326/0 (socket says 172.21.15.114:47326) 2024-09-13T18:28:49.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.448+0000 7f0583fff640 1 -- 172.21.15.114:0/3766406051 learned_addr learned my addr 172.21.15.114:0/3766406051 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:49.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.448+0000 7f0588b4b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563471e8fb30 0x563471e8cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:55866/0 (socket says 172.21.15.114:55866) 2024-09-13T18:28:49.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.448+0000 7f05837fe640 1 --2- 172.21.15.114:0/3766406051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563471e85290 0x563471e8c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:49.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.449+0000 7f0588b4b640 1 -- 172.21.15.114:0/3766406051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563471e85290 msgr2=0x563471e8c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:49.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.449+0000 7f0588b4b640 1 --2- 172.21.15.114:0/3766406051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563471e85290 0x563471e8c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.449+0000 7f0588b4b640 1 -- 172.21.15.114:0/3766406051 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563471e84970 msgr2=0x563471e8f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:49.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.449+0000 7f0588b4b640 1 --2- 172.21.15.114:0/3766406051 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563471e84970 0x563471e8f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.449+0000 7f0588b4b640 1 -- 172.21.15.114:0/3766406051 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05841139e0 con 0x563471e8fb30 2024-09-13T18:28:49.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.449+0000 7f0583fff640 1 --2- 172.21.15.114:0/3766406051 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563471e84970 0x563471e8f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:28:49.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.449+0000 7f0588b4b640 1 --2- 172.21.15.114:0/3766406051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563471e8fb30 0x563471e8cac0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f05740ca0e0 tx=0x7f0574099ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bd2d11f0bf506ce server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:49.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.449+0000 7f05637fe640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:28:49.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.450+0000 7f05637fe640 1 -- 172.21.15.114:0/3766406051 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05740990f0 con 0x563471e8fb30 2024-09-13T18:28:49.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.450+0000 7f058d2d0740 1 -- 172.21.15.114:0/3766406051 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563471e11b80 con 0x563471e8fb30 2024-09-13T18:28:49.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.450+0000 7f05637fe640 1 -- 172.21.15.114:0/3766406051 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f05740a19f0 con 0x563471e8fb30 2024-09-13T18:28:49.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.450+0000 7f05637fe640 1 -- 172.21.15.114:0/3766406051 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05740ca570 con 0x563471e8fb30 2024-09-13T18:28:49.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.450+0000 7f058d2d0740 10 client.26885 fetch_fsmap learned FSMap version 34 2024-09-13T18:28:49.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.450+0000 7f058d2d0740 1 -- 172.21.15.114:0/3766406051 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x563471de63c0 con 0x563471e8fb30 2024-09-13T18:28:49.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.451+0000 7f05637fe640 1 -- 172.21.15.114:0/3766406051 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563471e11b80 con 0x563471e8fb30 2024-09-13T18:28:49.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.451+0000 7f05637fe640 1 -- 172.21.15.114:0/3766406051 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0574098960 con 0x563471e8fb30 2024-09-13T18:28:49.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.451+0000 7f058d2d0740 10 client.26885 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:28:49.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.451+0000 7f058d2d0740 10 client.26885 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-13T18:28:49.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.451+0000 7f058d2d0740 20 client.26885 populate_metadata read hostname 'smithi114' 2024-09-13T18:28:49.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.451+0000 7f058d2d0740 1 --2- 172.21.15.114:0/3766406051 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x563471e9f5a0 0x563471ea19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:49.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.451+0000 7f058d2d0740 4 client.26885 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:28:49.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.451+0000 7f058d2d0740 1 -- 172.21.15.114:0/3766406051 --> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563471dda1f0 con 0x563471e9f5a0 2024-09-13T18:28:49.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.454+0000 7f0583fff640 1 --2- 172.21.15.114:0/3766406051 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x563471e9f5a0 0x563471ea19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:49.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.456+0000 7f0583fff640 1 --2- 172.21.15.114:0/3766406051 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x563471e9f5a0 0x563471ea19c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=394e951cc705af77 server_cookie=37eaecb261087fc1 in_seq=0 out_seq=0 2024-09-13T18:28:49.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.457+0000 7f05637fe640 10 client.26885 ms_handle_connect on v2:172.21.15.114:6840/1956923446 2024-09-13T18:28:49.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.459+0000 7f05637fe640 1 -- 172.21.15.114:0/3766406051 <== mds.4 v2:172.21.15.114:6840/1956923446 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x563471dda1f0 con 0x563471e9f5a0 2024-09-13T18:28:49.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.459+0000 7f05637fe640 10 client.26885 handle_command_reply: tid=0 2024-09-13T18:28:49.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.459+0000 7f058d2d0740 1 client.26885 shutdown 2024-09-13T18:28:49.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.460+0000 7f058d2d0740 1 -- 172.21.15.114:0/3766406051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563471e8fb30 msgr2=0x563471e8cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:49.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.460+0000 7f058d2d0740 1 --2- 172.21.15.114:0/3766406051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563471e8fb30 0x563471e8cac0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f05740ca0e0 tx=0x7f0574099ac0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.466+0000 7f058d2d0740 1 -- 172.21.15.114:0/3766406051 shutdown_connections 2024-09-13T18:28:49.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.466+0000 7f058d2d0740 1 --2- 172.21.15.114:0/3766406051 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x563471e9f5a0 0x563471ea19c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.466+0000 7f058d2d0740 1 --2- 172.21.15.114:0/3766406051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563471e8fb30 0x563471e8cac0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.466+0000 7f058d2d0740 1 --2- 172.21.15.114:0/3766406051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563471e85290 0x563471e8c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.466+0000 7f058d2d0740 1 --2- 172.21.15.114:0/3766406051 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563471e84970 0x563471e8f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.466+0000 7f058d2d0740 1 -- 172.21.15.114:0/3766406051 >> 172.21.15.114:0/3766406051 conn(0x563471e80ee0 msgr2=0x563471e80230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:49.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.466+0000 7f05637fe640 0 client.26885 ms_handle_reset on v2:172.21.15.114:6840/1956923446 2024-09-13T18:28:49.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.477+0000 7f058d2d0740 1 -- 172.21.15.114:0/3766406051 shutdown_connections 2024-09-13T18:28:49.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.480+0000 7f058d2d0740 1 -- 172.21.15.114:0/3766406051 wait complete. 2024-09-13T18:28:49.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.482+0000 7f058d2d0740 20 client.26885 trim_cache size 0 max 16384 2024-09-13T18:28:49.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.512+0000 7f058d2d0740 1 Processor -- start 2024-09-13T18:28:49.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.512+0000 7f058d2d0740 1 -- start start 2024-09-13T18:28:49.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.513+0000 7f058d2d0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563471e84970 0x563471e8ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:49.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.513+0000 7f058d2d0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563471e85290 0x563471e8b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:49.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.513+0000 7f058d2d0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563471e8fb30 0x563471e90f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:49.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.513+0000 7f058d2d0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x563471e8d930 con 0x563471e8fb30 2024-09-13T18:28:49.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.513+0000 7f058d2d0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563471e925e0 con 0x563471e84970 2024-09-13T18:28:49.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.513+0000 7f058d2d0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563471e8ed40 con 0x563471e85290 2024-09-13T18:28:49.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.513+0000 7f0583fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563471e84970 0x563471e8ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:49.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.513+0000 7f0583fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563471e84970 0x563471e8ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:55868/0 (socket says 172.21.15.114:55868) 2024-09-13T18:28:49.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.513+0000 7f0583fff640 1 -- 172.21.15.114:0/2945315066 learned_addr learned my addr 172.21.15.114:0/2945315066 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:49.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.513+0000 7f0588b4b640 1 --2- 172.21.15.114:0/2945315066 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563471e8fb30 0x563471e90f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:49.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.514+0000 7f05837fe640 1 --2- 172.21.15.114:0/2945315066 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563471e85290 0x563471e8b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:49.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.514+0000 7f0588b4b640 1 -- 172.21.15.114:0/2945315066 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563471e85290 msgr2=0x563471e8b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:49.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.514+0000 7f0588b4b640 1 --2- 172.21.15.114:0/2945315066 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563471e85290 0x563471e8b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.514+0000 7f0588b4b640 1 -- 172.21.15.114:0/2945315066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563471e84970 msgr2=0x563471e8ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:49.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.514+0000 7f0588b4b640 1 --2- 172.21.15.114:0/2945315066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563471e84970 0x563471e8ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.514+0000 7f0588b4b640 1 -- 172.21.15.114:0/2945315066 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563471d88770 con 0x563471e8fb30 2024-09-13T18:28:49.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.514+0000 7f0588b4b640 1 --2- 172.21.15.114:0/2945315066 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563471e8fb30 0x563471e90f20 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f0574096470 tx=0x7f05740a2000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7d6ea6e7e1a0f63 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:49.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.515+0000 7f057affd640 1 -- 172.21.15.114:0/2945315066 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0574004840 con 0x563471e8fb30 2024-09-13T18:28:49.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.515+0000 7f057affd640 1 -- 172.21.15.114:0/2945315066 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f05740ca570 con 0x563471e8fb30 2024-09-13T18:28:49.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.515+0000 7f057affd640 1 -- 172.21.15.114:0/2945315066 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057409e8c0 con 0x563471e8fb30 2024-09-13T18:28:49.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.516+0000 7f058d2d0740 1 -- 172.21.15.114:0/2945315066 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563471e8fb30 msgr2=0x563471e90f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:49.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.516+0000 7f058d2d0740 1 --2- 172.21.15.114:0/2945315066 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563471e8fb30 0x563471e90f20 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f0574096470 tx=0x7f05740a2000 comp rx=0 tx=0).stop 2024-09-13T18:28:49.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.517+0000 7f058d2d0740 1 -- 172.21.15.114:0/2945315066 shutdown_connections 2024-09-13T18:28:49.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.517+0000 7f058d2d0740 1 --2- 172.21.15.114:0/2945315066 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563471e8fb30 0x563471e90f20 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.517+0000 7f058d2d0740 1 --2- 172.21.15.114:0/2945315066 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563471e85290 0x563471e8b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.517+0000 7f058d2d0740 1 --2- 172.21.15.114:0/2945315066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563471e84970 0x563471e8ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.517+0000 7f058d2d0740 1 -- 172.21.15.114:0/2945315066 >> 172.21.15.114:0/2945315066 conn(0x563471e80ee0 msgr2=0x563471e812c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:49.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.518+0000 7f058d2d0740 1 -- 172.21.15.114:0/2945315066 shutdown_connections 2024-09-13T18:28:49.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.518+0000 7f058d2d0740 1 -- 172.21.15.114:0/2945315066 wait complete. 2024-09-13T18:28:49.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.518+0000 7f058d2d0740 1 -- start start 2024-09-13T18:28:49.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.519+0000 7f058d2d0740 1 Processor -- start 2024-09-13T18:28:49.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.520+0000 7f058d2d0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563471e84970 0x563471e96bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:49.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.520+0000 7f058d2d0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563471e85290 0x563471e89010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:49.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.520+0000 7f058d2d0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563471e8fb30 0x563471e895d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:49.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.520+0000 7f058d2d0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x563471e89cd0 con 0x563471e84970 2024-09-13T18:28:49.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.520+0000 7f058d2d0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563471e89e40 con 0x563471e8fb30 2024-09-13T18:28:49.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.520+0000 7f058d2d0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563471f06ab0 con 0x563471e85290 2024-09-13T18:28:49.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.520+0000 7f058d2d0740 10 client.0 _ll_register_callbacks cb 0x7f0574009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:28:49.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.520+0000 7f0583fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563471e84970 0x563471e96bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:49.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.521+0000 7f0583fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563471e84970 0x563471e96bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47348/0 (socket says 172.21.15.114:47348) 2024-09-13T18:28:49.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.521+0000 7f0583fff640 1 -- 172.21.15.114:0/4079370483 learned_addr learned my addr 172.21.15.114:0/4079370483 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:49.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.521+0000 7f0588b4b640 1 --2- 172.21.15.114:0/4079370483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563471e8fb30 0x563471e895d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:49.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.521+0000 7f0588b4b640 1 -- 172.21.15.114:0/4079370483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563471e85290 msgr2=0x563471e89010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:49.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.521+0000 7f0588b4b640 1 --2- 172.21.15.114:0/4079370483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563471e85290 0x563471e89010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.521+0000 7f0588b4b640 1 -- 172.21.15.114:0/4079370483 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563471e84970 msgr2=0x563471e96bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:49.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.521+0000 7f0588b4b640 1 --2- 172.21.15.114:0/4079370483 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563471e84970 0x563471e96bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.521+0000 7f0588b4b640 1 -- 172.21.15.114:0/4079370483 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563471de63c0 con 0x563471e8fb30 2024-09-13T18:28:49.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.521+0000 7f0583fff640 1 --2- 172.21.15.114:0/4079370483 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563471e84970 0x563471e96bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:28:49.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.522+0000 7f0588b4b640 1 --2- 172.21.15.114:0/4079370483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563471e8fb30 0x563471e895d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f057409b000 tx=0x7f057409eaf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5096040c0f71af6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:49.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.522+0000 7f05637fe640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:28:49.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.522+0000 7f05637fe640 1 -- 172.21.15.114:0/4079370483 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0574004840 con 0x563471e8fb30 2024-09-13T18:28:49.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.522+0000 7f058d2d0740 1 -- 172.21.15.114:0/4079370483 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f057c030cb0 con 0x563471e8fb30 2024-09-13T18:28:49.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.522+0000 7f05637fe640 1 -- 172.21.15.114:0/4079370483 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f057409d830 con 0x563471e8fb30 2024-09-13T18:28:49.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.522+0000 7f05637fe640 1 -- 172.21.15.114:0/4079370483 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0574095d40 con 0x563471e8fb30 2024-09-13T18:28:49.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.522+0000 7f058d2d0740 10 client.26891 fetch_fsmap learned FSMap version 34 2024-09-13T18:28:49.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.522+0000 7f058d2d0740 1 -- 172.21.15.114:0/4079370483 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x563471dade80 con 0x563471e8fb30 2024-09-13T18:28:49.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.523+0000 7f05637fe640 1 -- 172.21.15.114:0/4079370483 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f057c030cb0 con 0x563471e8fb30 2024-09-13T18:28:49.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.523+0000 7f05637fe640 1 -- 172.21.15.114:0/4079370483 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0574004a40 con 0x563471e8fb30 2024-09-13T18:28:49.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.523+0000 7f058d2d0740 10 client.26891 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:28:49.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.523+0000 7f058d2d0740 10 client.26891 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-13T18:28:49.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.523+0000 7f058d2d0740 20 client.26891 populate_metadata read hostname 'smithi114' 2024-09-13T18:28:49.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.523+0000 7f058d2d0740 1 --2- 172.21.15.114:0/4079370483 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x563471f07d80 0x563471f0a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:49.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.523+0000 7f058d2d0740 4 client.26891 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:28:49.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.523+0000 7f058d2d0740 1 -- 172.21.15.114:0/4079370483 --> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563471adb4e0 con 0x563471f07d80 2024-09-13T18:28:49.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.524+0000 7f0583fff640 1 --2- 172.21.15.114:0/4079370483 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x563471f07d80 0x563471f0a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:49.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.524+0000 7f05637fe640 10 client.26891 ms_handle_connect on v2:172.21.15.114:6840/1956923446 2024-09-13T18:28:49.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.524+0000 7f0583fff640 1 --2- 172.21.15.114:0/4079370483 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x563471f07d80 0x563471f0a170 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=93c08cb3bdab2b22 server_cookie=6ea6897cb4802d3b in_seq=0 out_seq=0 2024-09-13T18:28:49.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.525+0000 7f05637fe640 1 -- 172.21.15.114:0/4079370483 <== mds.4 v2:172.21.15.114:6840/1956923446 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563471adb4e0 con 0x563471f07d80 2024-09-13T18:28:49.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.525+0000 7f05637fe640 10 client.26891 handle_command_reply: tid=0 2024-09-13T18:28:49.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.525+0000 7f058d2d0740 1 client.26891 shutdown 2024-09-13T18:28:49.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.525+0000 7f058d2d0740 1 -- 172.21.15.114:0/4079370483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563471e8fb30 msgr2=0x563471e895d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:49.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.525+0000 7f058d2d0740 1 --2- 172.21.15.114:0/4079370483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563471e8fb30 0x563471e895d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f057409b000 tx=0x7f057409eaf0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.525+0000 7f058d2d0740 1 -- 172.21.15.114:0/4079370483 shutdown_connections 2024-09-13T18:28:49.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.525+0000 7f058d2d0740 1 --2- 172.21.15.114:0/4079370483 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x563471f07d80 0x563471f0a170 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.525+0000 7f058d2d0740 1 --2- 172.21.15.114:0/4079370483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563471e8fb30 0x563471e895d0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.525+0000 7f058d2d0740 1 --2- 172.21.15.114:0/4079370483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563471e85290 0x563471e89010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.525+0000 7f058d2d0740 1 --2- 172.21.15.114:0/4079370483 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563471e84970 0x563471e96bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.525+0000 7f058d2d0740 1 -- 172.21.15.114:0/4079370483 >> 172.21.15.114:0/4079370483 conn(0x563471e80ee0 msgr2=0x563471e8aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:49.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.525+0000 7f05637fe640 0 client.26891 ms_handle_reset on v2:172.21.15.114:6840/1956923446 2024-09-13T18:28:49.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.526+0000 7f058d2d0740 1 -- 172.21.15.114:0/4079370483 shutdown_connections 2024-09-13T18:28:49.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.526+0000 7f058d2d0740 1 -- 172.21.15.114:0/4079370483 wait complete. 2024-09-13T18:28:49.530 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:28:49.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.527+0000 7f058d2d0740 20 client.26891 trim_cache size 0 max 16384 2024-09-13T18:28:49.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.528+0000 7f058934c640 1 -- 172.21.15.114:0/89467004 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f05700a1340 msgr2=0x7f05700a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:49.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.528+0000 7f058934c640 1 --2- 172.21.15.114:0/89467004 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f05700a1340 0x7f05700a3800 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x563471e81d40 tx=0x7f056c00b390 comp rx=0 tx=0).stop 2024-09-13T18:28:49.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.528+0000 7f058934c640 1 -- 172.21.15.114:0/89467004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f058406aac0 msgr2=0x7f058406af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:49.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.528+0000 7f058934c640 1 --2- 172.21.15.114:0/89467004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f058406aac0 0x7f058406af70 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f057400ee70 tx=0x7f057400c740 comp rx=0 tx=0).stop 2024-09-13T18:28:49.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.529+0000 7f058934c640 1 -- 172.21.15.114:0/89467004 shutdown_connections 2024-09-13T18:28:49.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.529+0000 7f058934c640 1 --2- 172.21.15.114:0/89467004 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f05700a1340 0x7f05700a3800 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.529+0000 7f058934c640 1 --2- 172.21.15.114:0/89467004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f058406aac0 0x7f058406af70 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.529+0000 7f058934c640 1 --2- 172.21.15.114:0/89467004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f058410b650 0x7f0584071f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.529+0000 7f058934c640 1 --2- 172.21.15.114:0/89467004 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0584106550 0x7f05840719f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.529+0000 7f058934c640 1 -- 172.21.15.114:0/89467004 >> 172.21.15.114:0/89467004 conn(0x7f0584075c80 msgr2=0x7f0584076f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:49.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.532+0000 7f058934c640 1 -- 172.21.15.114:0/89467004 shutdown_connections 2024-09-13T18:28:49.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.533+0000 7f058934c640 1 -- 172.21.15.114:0/89467004 wait complete. 2024-09-13T18:28:49.564 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-09-13T18:28:49.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.822+0000 7f8a1e841640 1 -- 172.21.15.114:0/2178956297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a18107890 msgr2=0x7f8a18107cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:49.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.822+0000 7f8a1e841640 1 --2- 172.21.15.114:0/2178956297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a18107890 0x7f8a18107cd0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8a08009860 tx=0x7f8a0802a340 comp rx=0 tx=0).stop 2024-09-13T18:28:49.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.823+0000 7f8a1e841640 1 -- 172.21.15.114:0/2178956297 shutdown_connections 2024-09-13T18:28:49.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.823+0000 7f8a1e841640 1 --2- 172.21.15.114:0/2178956297 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a18119400 0x7f8a1811b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.823+0000 7f8a1e841640 1 --2- 172.21.15.114:0/2178956297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a18107890 0x7f8a18107cd0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.823+0000 7f8a1e841640 1 --2- 172.21.15.114:0/2178956297 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a181074b0 0x7f8a1810e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.823+0000 7f8a1e841640 1 -- 172.21.15.114:0/2178956297 >> 172.21.15.114:0/2178956297 conn(0x7f8a1806af40 msgr2=0x7f8a1806b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:49.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.827+0000 7f8a1e841640 1 -- 172.21.15.114:0/2178956297 shutdown_connections 2024-09-13T18:28:49.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.829+0000 7f8a1e841640 1 -- 172.21.15.114:0/2178956297 wait complete. 2024-09-13T18:28:49.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.829+0000 7f8a1e841640 1 Processor -- start 2024-09-13T18:28:49.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.832+0000 7f8a1e841640 1 -- start start 2024-09-13T18:28:49.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.832+0000 7f8a1e841640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a181074b0 0x7f8a181a4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:49.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.832+0000 7f8a1e841640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a18107890 0x7f8a181a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:49.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.832+0000 7f8a1e841640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a18119400 0x7f8a181a9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:49.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.832+0000 7f8a1e841640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a181a9e60 con 0x7f8a18107890 2024-09-13T18:28:49.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.832+0000 7f8a1e841640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a181a9fa0 con 0x7f8a18119400 2024-09-13T18:28:49.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.832+0000 7f8a1e841640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a181aa160 con 0x7f8a181074b0 2024-09-13T18:28:49.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.833+0000 7f8a1d83f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a181074b0 0x7f8a181a4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:49.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.833+0000 7f8a1d83f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a181074b0 0x7f8a181a4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:55412/0 (socket says 172.21.15.114:55412) 2024-09-13T18:28:49.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.833+0000 7f8a1d83f640 1 -- 172.21.15.114:0/3337928878 learned_addr learned my addr 172.21.15.114:0/3337928878 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:49.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.833+0000 7f8a1d83f640 1 -- 172.21.15.114:0/3337928878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a18119400 msgr2=0x7f8a181a9860 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:28:49.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.833+0000 7f8a1d83f640 1 --2- 172.21.15.114:0/3337928878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a18119400 0x7f8a181a9860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.833+0000 7f8a1d83f640 1 -- 172.21.15.114:0/3337928878 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a18107890 msgr2=0x7f8a181a5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:49.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.833+0000 7f8a1d83f640 1 --2- 172.21.15.114:0/3337928878 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a18107890 0x7f8a181a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.833+0000 7f8a1d83f640 1 -- 172.21.15.114:0/3337928878 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a08009490 con 0x7f8a181074b0 2024-09-13T18:28:49.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.834+0000 7f8a1d83f640 1 --2- 172.21.15.114:0/3337928878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a181074b0 0x7f8a181a4f30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8a1400d980 tx=0x7f8a1400de40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:49.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.834+0000 7f8a0effd640 1 -- 172.21.15.114:0/3337928878 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a14009c30 con 0x7f8a181074b0 2024-09-13T18:28:49.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.834+0000 7f8a0effd640 1 -- 172.21.15.114:0/3337928878 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8a1400b660 con 0x7f8a181074b0 2024-09-13T18:28:49.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.835+0000 7f8a0effd640 1 -- 172.21.15.114:0/3337928878 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1400f620 con 0x7f8a181074b0 2024-09-13T18:28:49.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.835+0000 7f8a1e841640 1 -- 172.21.15.114:0/3337928878 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a181aa440 con 0x7f8a181074b0 2024-09-13T18:28:49.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.835+0000 7f8a1e841640 1 -- 172.21.15.114:0/3337928878 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a1810edc0 con 0x7f8a181074b0 2024-09-13T18:28:49.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.838+0000 7f8a227cd740 1 Processor -- start 2024-09-13T18:28:49.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.839+0000 7f8a227cd740 1 -- start start 2024-09-13T18:28:49.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.839+0000 7f8a227cd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6ab855970 0x55e6ab855d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:49.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.839+0000 7f8a227cd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6ab856290 0x55e6ab8605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:49.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.839+0000 7f8a227cd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e6ab860b30 0x55e6ab862f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:49.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.839+0000 7f8a227cd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6ab7bfcc0 con 0x55e6ab860b30 2024-09-13T18:28:49.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.839+0000 7f8a227cd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6ab7bd450 con 0x55e6ab856290 2024-09-13T18:28:49.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.839+0000 7f8a227cd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6ab7b8d40 con 0x55e6ab855970 2024-09-13T18:28:49.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.839+0000 7f8a1d03e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e6ab860b30 0x55e6ab862f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:49.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.840+0000 7f8a1e040640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6ab855970 0x55e6ab855d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:49.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.840+0000 7f8a0effd640 1 -- 172.21.15.114:0/3337928878 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f8a14010460 con 0x7f8a181074b0 2024-09-13T18:28:49.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.841+0000 7f8a1d03e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e6ab860b30 0x55e6ab862f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47370/0 (socket says 172.21.15.114:47370) 2024-09-13T18:28:49.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.841+0000 7f8a1d03e640 1 -- 172.21.15.114:0/464533633 learned_addr learned my addr 172.21.15.114:0/464533633 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:49.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.841+0000 7f8a0effd640 1 --2- 172.21.15.114:0/3337928878 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f8a040a1400 0x7f8a040a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:49.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.842+0000 7f8a0effd640 1 -- 172.21.15.114:0/3337928878 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f8a140ccc70 con 0x7f8a181074b0 2024-09-13T18:28:49.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.842+0000 7f8a1d03e640 1 -- 172.21.15.114:0/464533633 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6ab855970 msgr2=0x55e6ab855d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:49.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.842+0000 7f8a1d03e640 1 --2- 172.21.15.114:0/464533633 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6ab855970 0x55e6ab855d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.842+0000 7f8a1d03e640 1 -- 172.21.15.114:0/464533633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6ab856290 msgr2=0x55e6ab8605f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:28:49.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.842+0000 7f8a1d03e640 1 --2- 172.21.15.114:0/464533633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6ab856290 0x55e6ab8605f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.842+0000 7f8a1d03e640 1 -- 172.21.15.114:0/464533633 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6ab780620 con 0x55e6ab860b30 2024-09-13T18:28:49.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.842+0000 7f8a1e040640 1 --2- 172.21.15.114:0/3337928878 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f8a040a1400 0x7f8a040a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:49.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.842+0000 7f8a1d03e640 1 --2- 172.21.15.114:0/464533633 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e6ab860b30 0x55e6ab862f20 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f8a08005b60 tx=0x7f8a08030410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e8846c0cf9b00d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:49.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.843+0000 7f8a1e040640 1 --2- 172.21.15.114:0/3337928878 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f8a040a1400 0x7f8a040a38c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8a10005fd0 tx=0x7f8a10004a40 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:49.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.843+0000 7f8a03fff640 1 -- 172.21.15.114:0/464533633 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a08032700 con 0x55e6ab860b30 2024-09-13T18:28:49.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.843+0000 7f8a03fff640 1 -- 172.21.15.114:0/464533633 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8a080304e0 con 0x55e6ab860b30 2024-09-13T18:28:49.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.843+0000 7f8a03fff640 1 -- 172.21.15.114:0/464533633 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a080042a0 con 0x55e6ab860b30 2024-09-13T18:28:49.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.844+0000 7f8a227cd740 1 -- 172.21.15.114:0/464533633 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e6ab860b30 msgr2=0x55e6ab862f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:49.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.844+0000 7f8a227cd740 1 --2- 172.21.15.114:0/464533633 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e6ab860b30 0x55e6ab862f20 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f8a08005b60 tx=0x7f8a08030410 comp rx=0 tx=0).stop 2024-09-13T18:28:49.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.844+0000 7f8a227cd740 1 -- 172.21.15.114:0/464533633 shutdown_connections 2024-09-13T18:28:49.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.844+0000 7f8a227cd740 1 --2- 172.21.15.114:0/464533633 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e6ab860b30 0x55e6ab862f20 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.844+0000 7f8a227cd740 1 --2- 172.21.15.114:0/464533633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6ab856290 0x55e6ab8605f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.844+0000 7f8a227cd740 1 --2- 172.21.15.114:0/464533633 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6ab855970 0x55e6ab855d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.844+0000 7f8a227cd740 1 -- 172.21.15.114:0/464533633 >> 172.21.15.114:0/464533633 conn(0x55e6ab851ee0 msgr2=0x55e6ab8522c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:49.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.845+0000 7f8a227cd740 1 -- 172.21.15.114:0/464533633 shutdown_connections 2024-09-13T18:28:49.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.845+0000 7f8a227cd740 1 -- 172.21.15.114:0/464533633 wait complete. 2024-09-13T18:28:49.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.845+0000 7f8a227cd740 1 -- start start 2024-09-13T18:28:49.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.845+0000 7f8a227cd740 1 Processor -- start 2024-09-13T18:28:49.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.846+0000 7f8a227cd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e6ab855970 0x55e6ab8602a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:49.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.846+0000 7f8a227cd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6ab860b30 0x55e6ab85d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:49.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.846+0000 7f8a227cd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6ab85dab0 0x55e6ab85df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:49.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.846+0000 7f8a227cd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6ab777180 con 0x55e6ab855970 2024-09-13T18:28:49.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.846+0000 7f8a227cd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6ab491430 con 0x55e6ab860b30 2024-09-13T18:28:49.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.846+0000 7f8a227cd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6ab73c5e0 con 0x55e6ab85dab0 2024-09-13T18:28:49.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.846+0000 7f8a1e040640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6ab85dab0 0x55e6ab85df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:49.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.846+0000 7f8a1e040640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6ab85dab0 0x55e6ab85df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:55440/0 (socket says 172.21.15.114:55440) 2024-09-13T18:28:49.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.846+0000 7f8a1e040640 1 -- 172.21.15.114:0/3458576389 learned_addr learned my addr 172.21.15.114:0/3458576389 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:49.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.846+0000 7f8a1d83f640 1 --2- 172.21.15.114:0/3458576389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e6ab855970 0x55e6ab8602a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:49.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.847+0000 7f8a227cd740 10 client.0 _ll_register_callbacks cb 0x55e6ab7a8aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:28:49.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.847+0000 7f8a1e040640 1 -- 172.21.15.114:0/3458576389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6ab860b30 msgr2=0x55e6ab85d570 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:28:49.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.847+0000 7f8a1e040640 1 --2- 172.21.15.114:0/3458576389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6ab860b30 0x55e6ab85d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.847+0000 7f8a1e040640 1 -- 172.21.15.114:0/3458576389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e6ab855970 msgr2=0x55e6ab8602a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:49.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.847+0000 7f8a1e040640 1 --2- 172.21.15.114:0/3458576389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e6ab855970 0x55e6ab8602a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.847+0000 7f8a1e040640 1 -- 172.21.15.114:0/3458576389 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a1811c170 con 0x55e6ab85dab0 2024-09-13T18:28:49.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.847+0000 7f8a1d83f640 1 --2- 172.21.15.114:0/3458576389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e6ab855970 0x55e6ab8602a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:28:49.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.847+0000 7f8a1e040640 1 --2- 172.21.15.114:0/3458576389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6ab85dab0 0x55e6ab85df30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8a100118e0 tx=0x7f8a10004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e77a8d5f1fc83a09 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:49.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.847+0000 7f8a017fa640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:28:49.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.847+0000 7f8a017fa640 1 -- 172.21.15.114:0/3458576389 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a10024070 con 0x55e6ab85dab0 2024-09-13T18:28:49.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.847+0000 7f8a227cd740 1 -- 172.21.15.114:0/3458576389 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e6ab7e2b80 con 0x55e6ab85dab0 2024-09-13T18:28:49.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.848+0000 7f8a017fa640 1 -- 172.21.15.114:0/3458576389 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f8a10019030 con 0x55e6ab85dab0 2024-09-13T18:28:49.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.848+0000 7f8a017fa640 1 -- 172.21.15.114:0/3458576389 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a1000fb40 con 0x55e6ab85dab0 2024-09-13T18:28:49.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.848+0000 7f8a227cd740 10 client.26878 fetch_fsmap learned FSMap version 34 2024-09-13T18:28:49.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.848+0000 7f8a227cd740 1 -- 172.21.15.114:0/3458576389 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55e6ab7b73c0 con 0x55e6ab85dab0 2024-09-13T18:28:49.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.850+0000 7f8a017fa640 1 -- 172.21.15.114:0/3458576389 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e6ab7e2b80 con 0x55e6ab85dab0 2024-09-13T18:28:49.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.852+0000 7f8a017fa640 1 -- 172.21.15.114:0/3458576389 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8a10027020 con 0x55e6ab85dab0 2024-09-13T18:28:49.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.853+0000 7f8a227cd740 10 client.26878 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:28:49.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.853+0000 7f8a227cd740 10 client.26878 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-13T18:28:49.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.853+0000 7f8a227cd740 20 client.26878 populate_metadata read hostname 'smithi114' 2024-09-13T18:28:49.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.853+0000 7f8a227cd740 1 --2- 172.21.15.114:0/3458576389 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55e6ab870a00 0x55e6ab872e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:49.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.853+0000 7f8a227cd740 4 client.26878 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:28:49.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.853+0000 7f8a227cd740 1 -- 172.21.15.114:0/3458576389 --> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e6ab7ab1f0 con 0x55e6ab870a00 2024-09-13T18:28:49.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.854+0000 7f8a1d83f640 1 --2- 172.21.15.114:0/3458576389 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55e6ab870a00 0x55e6ab872e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:49.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.866+0000 7f8a1d83f640 1 --2- 172.21.15.114:0/3458576389 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55e6ab870a00 0x55e6ab872e20 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=581fba2bb571b5b6 server_cookie=3fde2aa2260a3f49 in_seq=0 out_seq=0 2024-09-13T18:28:49.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.866+0000 7f8a017fa640 10 client.26878 ms_handle_connect on v2:172.21.15.134:6834/1150003275 2024-09-13T18:28:49.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.871+0000 7f8a017fa640 1 -- 172.21.15.114:0/3458576389 <== mds.3 v2:172.21.15.134:6834/1150003275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55e6ab7ab1f0 con 0x55e6ab870a00 2024-09-13T18:28:49.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.871+0000 7f8a017fa640 10 client.26878 handle_command_reply: tid=0 2024-09-13T18:28:49.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.871+0000 7f8a227cd740 1 client.26878 shutdown 2024-09-13T18:28:49.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.874+0000 7f8a227cd740 1 -- 172.21.15.114:0/3458576389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6ab85dab0 msgr2=0x55e6ab85df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:49.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.874+0000 7f8a227cd740 1 --2- 172.21.15.114:0/3458576389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6ab85dab0 0x55e6ab85df30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8a100118e0 tx=0x7f8a10004ab0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.876+0000 7f8a227cd740 1 -- 172.21.15.114:0/3458576389 shutdown_connections 2024-09-13T18:28:49.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.876+0000 7f8a227cd740 1 --2- 172.21.15.114:0/3458576389 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55e6ab870a00 0x55e6ab872e20 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.876+0000 7f8a227cd740 1 --2- 172.21.15.114:0/3458576389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6ab85dab0 0x55e6ab85df30 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.876+0000 7f8a227cd740 1 --2- 172.21.15.114:0/3458576389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6ab860b30 0x55e6ab85d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.876+0000 7f8a227cd740 1 --2- 172.21.15.114:0/3458576389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e6ab855970 0x55e6ab8602a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.876+0000 7f8a227cd740 1 -- 172.21.15.114:0/3458576389 >> 172.21.15.114:0/3458576389 conn(0x55e6ab851ee0 msgr2=0x55e6ab851230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:49.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.876+0000 7f8a017fa640 0 client.26878 ms_handle_reset on v2:172.21.15.134:6834/1150003275 2024-09-13T18:28:49.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.880+0000 7f8a227cd740 1 -- 172.21.15.114:0/3458576389 shutdown_connections 2024-09-13T18:28:49.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.880+0000 7f8a227cd740 1 -- 172.21.15.114:0/3458576389 wait complete. 2024-09-13T18:28:49.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.884+0000 7f8a227cd740 20 client.26878 trim_cache size 0 max 16384 2024-09-13T18:28:49.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.917+0000 7f8a227cd740 1 Processor -- start 2024-09-13T18:28:49.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.917+0000 7f8a227cd740 1 -- start start 2024-09-13T18:28:49.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.918+0000 7f8a227cd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e6ab855970 0x55e6ab858c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:49.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.918+0000 7f8a227cd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6ab85dab0 0x55e6ab85a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:49.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.918+0000 7f8a1d83f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e6ab855970 0x55e6ab858c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:49.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.918+0000 7f8a1d83f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e6ab855970 0x55e6ab858c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47388/0 (socket says 172.21.15.114:47388) 2024-09-13T18:28:49.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.918+0000 7f8a1d83f640 1 -- 172.21.15.114:0/3129404049 learned_addr learned my addr 172.21.15.114:0/3129404049 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:49.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.918+0000 7f8a227cd740 1 --2- 172.21.15.114:0/3129404049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6ab860b30 0x55e6ab85ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:49.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.918+0000 7f8a1d03e640 1 --2- 172.21.15.114:0/3129404049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6ab85dab0 0x55e6ab85a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:49.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.918+0000 7f8a227cd740 1 -- 172.21.15.114:0/3129404049 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6ab85e530 con 0x55e6ab855970 2024-09-13T18:28:49.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.918+0000 7f8a227cd740 1 -- 172.21.15.114:0/3129404049 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6ab85e990 con 0x55e6ab860b30 2024-09-13T18:28:49.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.918+0000 7f8a227cd740 1 -- 172.21.15.114:0/3129404049 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6ab7ab1f0 con 0x55e6ab85dab0 2024-09-13T18:28:49.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.918+0000 7f8a1e040640 1 --2- 172.21.15.114:0/3129404049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6ab860b30 0x55e6ab85ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:49.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.920+0000 7f8a1d03e640 1 -- 172.21.15.114:0/3129404049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6ab860b30 msgr2=0x55e6ab85ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:49.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.920+0000 7f8a1d03e640 1 --2- 172.21.15.114:0/3129404049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6ab860b30 0x55e6ab85ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.920+0000 7f8a1d03e640 1 -- 172.21.15.114:0/3129404049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e6ab855970 msgr2=0x55e6ab858c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:49.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.920+0000 7f8a1d03e640 1 --2- 172.21.15.114:0/3129404049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e6ab855970 0x55e6ab858c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.920+0000 7f8a1d03e640 1 -- 172.21.15.114:0/3129404049 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6ab759770 con 0x55e6ab85dab0 2024-09-13T18:28:49.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.921+0000 7f8a1d03e640 1 --2- 172.21.15.114:0/3129404049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6ab85dab0 0x55e6ab85a7f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x55e6ab85cde0 tx=0x7f8a08000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd0359d979f5a827 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:49.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.922+0000 7f8a0ce05640 1 -- 172.21.15.114:0/3129404049 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a08030850 con 0x55e6ab85dab0 2024-09-13T18:28:49.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.922+0000 7f8a0ce05640 1 -- 172.21.15.114:0/3129404049 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8a08032e70 con 0x55e6ab85dab0 2024-09-13T18:28:49.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.923+0000 7f8a0ce05640 1 -- 172.21.15.114:0/3129404049 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a08004290 con 0x55e6ab85dab0 2024-09-13T18:28:49.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.923+0000 7f8a227cd740 1 -- 172.21.15.114:0/3129404049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6ab85dab0 msgr2=0x55e6ab85a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:49.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.923+0000 7f8a227cd740 1 --2- 172.21.15.114:0/3129404049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6ab85dab0 0x55e6ab85a7f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x55e6ab85cde0 tx=0x7f8a08000bc0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.923+0000 7f8a227cd740 1 -- 172.21.15.114:0/3129404049 shutdown_connections 2024-09-13T18:28:49.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.923+0000 7f8a227cd740 1 --2- 172.21.15.114:0/3129404049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6ab860b30 0x55e6ab85ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.923+0000 7f8a227cd740 1 --2- 172.21.15.114:0/3129404049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6ab85dab0 0x55e6ab85a7f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.923+0000 7f8a227cd740 1 --2- 172.21.15.114:0/3129404049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e6ab855970 0x55e6ab858c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.923+0000 7f8a227cd740 1 -- 172.21.15.114:0/3129404049 >> 172.21.15.114:0/3129404049 conn(0x55e6ab851ee0 msgr2=0x55e6ab858540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:49.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.927+0000 7f8a227cd740 1 -- 172.21.15.114:0/3129404049 shutdown_connections 2024-09-13T18:28:49.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.931+0000 7f8a227cd740 1 -- 172.21.15.114:0/3129404049 wait complete. 2024-09-13T18:28:49.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.931+0000 7f8a227cd740 1 -- start start 2024-09-13T18:28:49.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.931+0000 7f8a227cd740 1 Processor -- start 2024-09-13T18:28:49.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.932+0000 7f8a227cd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6ab855970 0x55e6ab85a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:49.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.932+0000 7f8a227cd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6ab85dab0 0x55e6ab85ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:49.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.932+0000 7f8a227cd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e6ab860b30 0x55e6ab8c6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:49.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.932+0000 7f8a1d83f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6ab855970 0x55e6ab85a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:49.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.932+0000 7f8a1d83f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6ab855970 0x55e6ab85a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:55928/0 (socket says 172.21.15.114:55928) 2024-09-13T18:28:49.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.932+0000 7f8a1d83f640 1 -- 172.21.15.114:0/717012426 learned_addr learned my addr 172.21.15.114:0/717012426 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:49.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.932+0000 7f8a227cd740 1 -- 172.21.15.114:0/717012426 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6ab85b5b0 con 0x55e6ab860b30 2024-09-13T18:28:49.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.932+0000 7f8a1e040640 1 --2- 172.21.15.114:0/717012426 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e6ab860b30 0x55e6ab8c6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:49.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.933+0000 7f8a227cd740 1 -- 172.21.15.114:0/717012426 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6ab8c7460 con 0x55e6ab855970 2024-09-13T18:28:49.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.933+0000 7f8a227cd740 1 -- 172.21.15.114:0/717012426 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6ab8c75a0 con 0x55e6ab85dab0 2024-09-13T18:28:49.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.933+0000 7f8a1d03e640 1 --2- 172.21.15.114:0/717012426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6ab85dab0 0x55e6ab85ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:49.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.933+0000 7f8a227cd740 10 client.0 _ll_register_callbacks cb 0x7f8a08035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:28:49.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.933+0000 7f8a1e040640 1 -- 172.21.15.114:0/717012426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6ab85dab0 msgr2=0x55e6ab85ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:49.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.933+0000 7f8a1e040640 1 --2- 172.21.15.114:0/717012426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6ab85dab0 0x55e6ab85ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.933+0000 7f8a1e040640 1 -- 172.21.15.114:0/717012426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6ab855970 msgr2=0x55e6ab85a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:49.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.933+0000 7f8a1e040640 1 --2- 172.21.15.114:0/717012426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6ab855970 0x55e6ab85a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.933+0000 7f8a1e040640 1 -- 172.21.15.114:0/717012426 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e6ab780620 con 0x55e6ab860b30 2024-09-13T18:28:49.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.933+0000 7f8a227cd740 1 -- 172.21.15.114:0/717012426 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8a080304e0 con 0x55e6ab860b30 2024-09-13T18:28:49.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.934+0000 7f8a1e040640 1 --2- 172.21.15.114:0/717012426 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e6ab860b30 0x55e6ab8c6f20 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f8a10026030 tx=0x7f8a1000fdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bac13ebadf0fb43 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:49.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.934+0000 7f8a017fa640 10 client.17466 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:28:49.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.934+0000 7f8a017fa640 1 -- 172.21.15.114:0/717012426 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a10024070 con 0x55e6ab860b30 2024-09-13T18:28:49.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.934+0000 7f8a017fa640 1 -- 172.21.15.114:0/717012426 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f8a10019040 con 0x55e6ab860b30 2024-09-13T18:28:49.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.935+0000 7f8a017fa640 1 -- 172.21.15.114:0/717012426 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a10009440 con 0x55e6ab860b30 2024-09-13T18:28:49.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.937+0000 7f8a227cd740 10 client.17466 fetch_fsmap learned FSMap version 34 2024-09-13T18:28:49.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.937+0000 7f8a227cd740 1 -- 172.21.15.114:0/717012426 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({fsmap=34}) v3 -- 0x55e6ab77ee80 con 0x55e6ab860b30 2024-09-13T18:28:49.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.944+0000 7f8a017fa640 1 -- 172.21.15.114:0/717012426 <== mon.0 v2:172.21.15.114:3300/0 4 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8a10027040 con 0x55e6ab860b30 2024-09-13T18:28:49.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.946+0000 7f8a227cd740 10 client.17466 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:28:49.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.946+0000 7f8a227cd740 10 client.17466 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-13T18:28:49.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.946+0000 7f8a227cd740 20 client.17466 populate_metadata read hostname 'smithi114' 2024-09-13T18:28:49.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.946+0000 7f8a227cd740 1 --2- 172.21.15.114:0/717012426 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55e6ab8c8a90 0x55e6ab8caed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:49.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.946+0000 7f8a227cd740 4 client.17466 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:28:49.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.946+0000 7f8a227cd740 1 -- 172.21.15.114:0/717012426 --> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e6ab4ac4e0 con 0x55e6ab8c8a90 2024-09-13T18:28:49.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.947+0000 7f8a1d83f640 1 --2- 172.21.15.114:0/717012426 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55e6ab8c8a90 0x55e6ab8caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:49.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.962+0000 7f8a1d83f640 1 --2- 172.21.15.114:0/717012426 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55e6ab8c8a90 0x55e6ab8caed0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c7ff5858fa08725d server_cookie=a31ac46d58434300 in_seq=0 out_seq=0 2024-09-13T18:28:49.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.962+0000 7f8a017fa640 10 client.17466 ms_handle_connect on v2:172.21.15.134:6834/1150003275 2024-09-13T18:28:49.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.963+0000 7f8a017fa640 1 -- 172.21.15.114:0/717012426 <== mds.3 v2:172.21.15.134:6834/1150003275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e6ab4ac4e0 con 0x55e6ab8c8a90 2024-09-13T18:28:49.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.963+0000 7f8a017fa640 10 client.17466 handle_command_reply: tid=0 2024-09-13T18:28:49.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.966+0000 7f8a227cd740 1 client.17466 shutdown 2024-09-13T18:28:49.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:49 smithi114 ceph-mon[30897]: pgmap v193: 129 pgs: 129 active+clean; 20 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 32 MiB/s wr, 666 op/s 2024-09-13T18:28:49.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:49 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2313499306' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:28:49.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.969+0000 7f8a227cd740 1 -- 172.21.15.114:0/717012426 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e6ab860b30 msgr2=0x55e6ab8c6f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:49.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.969+0000 7f8a227cd740 1 --2- 172.21.15.114:0/717012426 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e6ab860b30 0x55e6ab8c6f20 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f8a10026030 tx=0x7f8a1000fdd0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.969+0000 7f8a227cd740 1 -- 172.21.15.114:0/717012426 shutdown_connections 2024-09-13T18:28:49.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.969+0000 7f8a227cd740 1 --2- 172.21.15.114:0/717012426 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55e6ab8c8a90 0x55e6ab8caed0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.969+0000 7f8a227cd740 1 --2- 172.21.15.114:0/717012426 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e6ab860b30 0x55e6ab8c6f20 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.969+0000 7f8a227cd740 1 --2- 172.21.15.114:0/717012426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6ab85dab0 0x55e6ab85ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.969+0000 7f8a227cd740 1 --2- 172.21.15.114:0/717012426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6ab855970 0x55e6ab85a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.969+0000 7f8a227cd740 1 -- 172.21.15.114:0/717012426 >> 172.21.15.114:0/717012426 conn(0x55e6ab851ee0 msgr2=0x55e6ab85be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:49.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.974+0000 7f8a017fa640 0 client.17466 ms_handle_reset on v2:172.21.15.134:6834/1150003275 2024-09-13T18:28:49.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.974+0000 7f8a227cd740 1 -- 172.21.15.114:0/717012426 shutdown_connections 2024-09-13T18:28:49.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.974+0000 7f8a227cd740 1 -- 172.21.15.114:0/717012426 wait complete. 2024-09-13T18:28:49.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.979+0000 7f8a227cd740 20 client.17466 trim_cache size 0 max 16384 2024-09-13T18:28:49.982 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:28:49.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.983+0000 7f8a1e841640 1 -- 172.21.15.114:0/3337928878 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f8a040a1400 msgr2=0x7f8a040a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:49.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.983+0000 7f8a1e841640 1 --2- 172.21.15.114:0/3337928878 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f8a040a1400 0x7f8a040a38c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8a10005fd0 tx=0x7f8a10004a40 comp rx=0 tx=0).stop 2024-09-13T18:28:49.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.983+0000 7f8a1e841640 1 -- 172.21.15.114:0/3337928878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a181074b0 msgr2=0x7f8a181a4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:49.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.983+0000 7f8a1e841640 1 --2- 172.21.15.114:0/3337928878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a181074b0 0x7f8a181a4f30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8a1400d980 tx=0x7f8a1400de40 comp rx=0 tx=0).stop 2024-09-13T18:28:49.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.989+0000 7f8a1e841640 1 -- 172.21.15.114:0/3337928878 shutdown_connections 2024-09-13T18:28:49.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.989+0000 7f8a1e841640 1 --2- 172.21.15.114:0/3337928878 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f8a040a1400 0x7f8a040a38c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.989+0000 7f8a1e841640 1 --2- 172.21.15.114:0/3337928878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a18119400 0x7f8a181a9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.989+0000 7f8a1e841640 1 --2- 172.21.15.114:0/3337928878 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a18107890 0x7f8a181a5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.989+0000 7f8a1e841640 1 --2- 172.21.15.114:0/3337928878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a181074b0 0x7f8a181a4f30 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:49.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.989+0000 7f8a1e841640 1 -- 172.21.15.114:0/3337928878 >> 172.21.15.114:0/3337928878 conn(0x7f8a1806af40 msgr2=0x7f8a18119b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:49.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.991+0000 7f8a1e841640 1 -- 172.21.15.114:0/3337928878 shutdown_connections 2024-09-13T18:28:49.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:49.991+0000 7f8a1e841640 1 -- 172.21.15.114:0/3337928878 wait complete. 2024-09-13T18:28:50.024 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-09-13T18:28:50.118 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:49 smithi134 ceph-mon[34283]: pgmap v193: 129 pgs: 129 active+clean; 20 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 32 MiB/s wr, 666 op/s 2024-09-13T18:28:50.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:49 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/2313499306' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:28:50.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:49 smithi204 ceph-mon[34777]: pgmap v193: 129 pgs: 129 active+clean; 20 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 32 MiB/s wr, 666 op/s 2024-09-13T18:28:50.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:49 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/2313499306' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:28:50.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.272+0000 7f5c00d56640 1 -- 172.21.15.114:0/3198722113 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf4031770 con 0x7f5bfc119400 2024-09-13T18:28:50.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.274+0000 7f5c02d5a640 1 -- 172.21.15.114:0/3198722113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bfc119400 msgr2=0x7f5bfc11b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:50.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.274+0000 7f5c02d5a640 1 --2- 172.21.15.114:0/3198722113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bfc119400 0x7f5bfc11b7f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4009900 tx=0x7f5bf402a440 comp rx=0 tx=0).stop 2024-09-13T18:28:50.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.275+0000 7f5c02d5a640 1 -- 172.21.15.114:0/3198722113 shutdown_connections 2024-09-13T18:28:50.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.275+0000 7f5c02d5a640 1 --2- 172.21.15.114:0/3198722113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bfc119400 0x7f5bfc11b7f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.275+0000 7f5c02d5a640 1 --2- 172.21.15.114:0/3198722113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5bfc107890 0x7f5bfc107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.275+0000 7f5c02d5a640 1 --2- 172.21.15.114:0/3198722113 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5bfc1074b0 0x7f5bfc10e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.275+0000 7f5c02d5a640 1 -- 172.21.15.114:0/3198722113 >> 172.21.15.114:0/3198722113 conn(0x7f5bfc06af40 msgr2=0x7f5bfc06b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:50.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.275+0000 7f5c02d5a640 1 -- 172.21.15.114:0/3198722113 shutdown_connections 2024-09-13T18:28:50.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.279+0000 7f5c02d5a640 1 -- 172.21.15.114:0/3198722113 wait complete. 2024-09-13T18:28:50.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.279+0000 7f5c02d5a640 1 Processor -- start 2024-09-13T18:28:50.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.284+0000 7f5c02d5a640 1 -- start start 2024-09-13T18:28:50.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.284+0000 7f5c02d5a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bfc1074b0 0x7f5bfc1a4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:50.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.284+0000 7f5c02d5a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5bfc107890 0x7f5bfc1a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:50.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.284+0000 7f5c02d5a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5bfc119400 0x7f5bfc1a98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:50.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.284+0000 7f5c02d5a640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bfc1a9ed0 con 0x7f5bfc107890 2024-09-13T18:28:50.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.284+0000 7f5c02d5a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bfc1aa040 con 0x7f5bfc119400 2024-09-13T18:28:50.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.284+0000 7f5c02d5a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bfc1aa200 con 0x7f5bfc1074b0 2024-09-13T18:28:50.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.285+0000 7f5c02559640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5bfc119400 0x7f5bfc1a98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:50.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.285+0000 7f5c01557640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5bfc107890 0x7f5bfc1a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:50.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.285+0000 7f5c01557640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5bfc107890 0x7f5bfc1a54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47398/0 (socket says 172.21.15.114:47398) 2024-09-13T18:28:50.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.285+0000 7f5c01557640 1 -- 172.21.15.114:0/3733354415 learned_addr learned my addr 172.21.15.114:0/3733354415 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:50.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.286+0000 7f5c02559640 1 -- 172.21.15.114:0/3733354415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bfc1074b0 msgr2=0x7f5bfc1a4fa0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:28:50.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.286+0000 7f5c02559640 1 --2- 172.21.15.114:0/3733354415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bfc1074b0 0x7f5bfc1a4fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.286+0000 7f5c02559640 1 -- 172.21.15.114:0/3733354415 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5bfc107890 msgr2=0x7f5bfc1a54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:50.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.287+0000 7f5c02559640 1 --2- 172.21.15.114:0/3733354415 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5bfc107890 0x7f5bfc1a54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.287+0000 7f5c02559640 1 -- 172.21.15.114:0/3733354415 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bf4009560 con 0x7f5bfc119400 2024-09-13T18:28:50.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.287+0000 7f5c02559640 1 --2- 172.21.15.114:0/3733354415 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5bfc119400 0x7f5bfc1a98d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4009850 tx=0x7f5bf4003cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:50.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.287+0000 7f5c01557640 1 --2- 172.21.15.114:0/3733354415 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5bfc107890 0x7f5bfc1a54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:28:50.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.288+0000 7f5bfaffd640 1 -- 172.21.15.114:0/3733354415 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf4003e60 con 0x7f5bfc119400 2024-09-13T18:28:50.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.288+0000 7f5bfaffd640 1 -- 172.21.15.114:0/3733354415 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5bf403bb00 con 0x7f5bfc119400 2024-09-13T18:28:50.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.288+0000 7f5c02d5a640 1 -- 172.21.15.114:0/3733354415 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bfc1aa480 con 0x7f5bfc119400 2024-09-13T18:28:50.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.289+0000 7f5c02d5a640 1 -- 172.21.15.114:0/3733354415 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bfc1aa750 con 0x7f5bfc119400 2024-09-13T18:28:50.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.290+0000 7f5bfaffd640 1 -- 172.21.15.114:0/3733354415 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf4031ae0 con 0x7f5bfc119400 2024-09-13T18:28:50.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.292+0000 7f5bfaffd640 1 -- 172.21.15.114:0/3733354415 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f5bf4031d00 con 0x7f5bfc119400 2024-09-13T18:28:50.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.295+0000 7f5c06dbd740 1 Processor -- start 2024-09-13T18:28:50.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.295+0000 7f5c06dbd740 1 -- start start 2024-09-13T18:28:50.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.296+0000 7f5c06dbd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d59ce1f970 0x55d59ce1fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:50.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.296+0000 7f5c06dbd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d59ce20290 0x55d59ce2a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:50.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.296+0000 7f5c06dbd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d59ce2ab30 0x55d59ce2cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:50.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.296+0000 7f5c06dbd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d59cd89cc0 con 0x55d59ce1f970 2024-09-13T18:28:50.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.296+0000 7f5c06dbd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d59cd87450 con 0x55d59ce20290 2024-09-13T18:28:50.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.296+0000 7f5c06dbd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d59cd82d40 con 0x55d59ce2ab30 2024-09-13T18:28:50.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.296+0000 7f5c01d58640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d59ce20290 0x55d59ce2a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:50.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.296+0000 7f5c01d58640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d59ce20290 0x55d59ce2a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:55962/0 (socket says 172.21.15.114:55962) 2024-09-13T18:28:50.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.296+0000 7f5c01d58640 1 -- 172.21.15.114:0/2377549359 learned_addr learned my addr 172.21.15.114:0/2377549359 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:50.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.296+0000 7f5c01557640 1 --2- 172.21.15.114:0/2377549359 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d59ce1f970 0x55d59ce1fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:50.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.296+0000 7f5c01557640 1 --2- 172.21.15.114:0/2377549359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d59ce2ab30 0x55d59ce2cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:50.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.297+0000 7f5c01d58640 1 -- 172.21.15.114:0/2377549359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d59ce2ab30 msgr2=0x55d59ce2cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:50.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.297+0000 7f5c01d58640 1 --2- 172.21.15.114:0/2377549359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d59ce2ab30 0x55d59ce2cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.297+0000 7f5c01d58640 1 -- 172.21.15.114:0/2377549359 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d59ce1f970 msgr2=0x55d59ce1fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:50.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.297+0000 7f5c01d58640 1 --2- 172.21.15.114:0/2377549359 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d59ce1f970 0x55d59ce1fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.297+0000 7f5c01d58640 1 -- 172.21.15.114:0/2377549359 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d59cd4a620 con 0x55d59ce20290 2024-09-13T18:28:50.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.297+0000 7f5c01557640 1 --2- 172.21.15.114:0/2377549359 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d59ce1f970 0x55d59ce1fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:28:50.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.297+0000 7f5bfaffd640 1 --2- 172.21.15.114:0/3733354415 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f5be40a1390 0x7f5be40a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:50.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.297+0000 7f5bfaffd640 1 -- 172.21.15.114:0/3733354415 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f5bf40ed050 con 0x7f5bfc119400 2024-09-13T18:28:50.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.297+0000 7f5c01d58640 1 --2- 172.21.15.114:0/2377549359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d59ce20290 0x55d59ce2a5f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0009780 tx=0x7f5bf0009c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=392a6aa43ad5befd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:50.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.298+0000 7f5be3fff640 1 -- 172.21.15.114:0/2377549359 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf000bad0 con 0x55d59ce20290 2024-09-13T18:28:50.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.298+0000 7f5c01557640 1 --2- 172.21.15.114:0/3733354415 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f5be40a1390 0x7f5be40a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:50.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.298+0000 7f5be3fff640 1 -- 172.21.15.114:0/2377549359 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5bf0009e90 con 0x55d59ce20290 2024-09-13T18:28:50.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.298+0000 7f5be3fff640 1 -- 172.21.15.114:0/2377549359 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf00103e0 con 0x55d59ce20290 2024-09-13T18:28:50.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.299+0000 7f5c06dbd740 1 -- 172.21.15.114:0/2377549359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d59ce20290 msgr2=0x55d59ce2a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:50.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.299+0000 7f5c06dbd740 1 --2- 172.21.15.114:0/2377549359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d59ce20290 0x55d59ce2a5f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0009780 tx=0x7f5bf0009c40 comp rx=0 tx=0).stop 2024-09-13T18:28:50.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.299+0000 7f5c06dbd740 1 -- 172.21.15.114:0/2377549359 shutdown_connections 2024-09-13T18:28:50.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.299+0000 7f5c06dbd740 1 --2- 172.21.15.114:0/2377549359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d59ce2ab30 0x55d59ce2cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.299+0000 7f5c06dbd740 1 --2- 172.21.15.114:0/2377549359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d59ce20290 0x55d59ce2a5f0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.299+0000 7f5c06dbd740 1 --2- 172.21.15.114:0/2377549359 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d59ce1f970 0x55d59ce1fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.299+0000 7f5c06dbd740 1 -- 172.21.15.114:0/2377549359 >> 172.21.15.114:0/2377549359 conn(0x55d59ce1bee0 msgr2=0x55d59ce1c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:50.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.303+0000 7f5c06dbd740 1 -- 172.21.15.114:0/2377549359 shutdown_connections 2024-09-13T18:28:50.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.306+0000 7f5c01557640 1 --2- 172.21.15.114:0/3733354415 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f5be40a1390 0x7f5be40a3850 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x55d59ce1d260 tx=0x7f5bec009790 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:50.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.307+0000 7f5c06dbd740 1 -- 172.21.15.114:0/2377549359 wait complete. 2024-09-13T18:28:50.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.307+0000 7f5c06dbd740 1 -- start start 2024-09-13T18:28:50.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.308+0000 7f5c06dbd740 1 Processor -- start 2024-09-13T18:28:50.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.308+0000 7f5c06dbd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d59ce1f970 0x55d59ce29fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:50.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.308+0000 7f5c06dbd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d59ce20290 0x55d59ce2a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:50.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.308+0000 7f5c06dbd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d59ce2ab30 0x55d59ce274c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:50.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.308+0000 7f5c06dbd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d59cd41180 con 0x55d59ce1f970 2024-09-13T18:28:50.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.308+0000 7f5c06dbd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d59ca5b430 con 0x55d59ce20290 2024-09-13T18:28:50.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.308+0000 7f5c06dbd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d59cd065e0 con 0x55d59ce2ab30 2024-09-13T18:28:50.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.308+0000 7f5c06dbd740 10 client.0 _ll_register_callbacks cb 0x55d59cd72aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:28:50.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.309+0000 7f5c02559640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d59ce2ab30 0x55d59ce274c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:50.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.309+0000 7f5c02559640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d59ce2ab30 0x55d59ce274c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:55492/0 (socket says 172.21.15.114:55492) 2024-09-13T18:28:50.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.309+0000 7f5c02559640 1 -- 172.21.15.114:0/2426938232 learned_addr learned my addr 172.21.15.114:0/2426938232 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:50.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.309+0000 7f5c02559640 1 -- 172.21.15.114:0/2426938232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d59ce20290 msgr2=0x55d59ce2a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:50.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.309+0000 7f5c02559640 1 --2- 172.21.15.114:0/2426938232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d59ce20290 0x55d59ce2a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.309+0000 7f5c02559640 1 -- 172.21.15.114:0/2426938232 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d59ce1f970 msgr2=0x55d59ce29fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:50.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.309+0000 7f5c02559640 1 --2- 172.21.15.114:0/2426938232 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d59ce1f970 0x55d59ce29fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.309+0000 7f5c02559640 1 -- 172.21.15.114:0/2426938232 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bf00093e0 con 0x55d59ce2ab30 2024-09-13T18:28:50.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.310+0000 7f5c02559640 1 --2- 172.21.15.114:0/2426938232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d59ce2ab30 0x55d59ce274c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5bf40c1d60 tx=0x7f5bf40b9e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72c87deec6677212 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:50.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.311+0000 7f5be17fa640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:28:50.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.312+0000 7f5be17fa640 1 -- 172.21.15.114:0/2426938232 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf40ea6d0 con 0x55d59ce2ab30 2024-09-13T18:28:50.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.312+0000 7f5c06dbd740 1 -- 172.21.15.114:0/2426938232 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d59cdacb80 con 0x55d59ce2ab30 2024-09-13T18:28:50.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.312+0000 7f5be17fa640 1 -- 172.21.15.114:0/2426938232 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f5bf40ea830 con 0x55d59ce2ab30 2024-09-13T18:28:50.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.317+0000 7f5be17fa640 1 -- 172.21.15.114:0/2426938232 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5bf40b6520 con 0x55d59ce2ab30 2024-09-13T18:28:50.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.319+0000 7f5c06dbd740 10 client.26890 fetch_fsmap learned FSMap version 34 2024-09-13T18:28:50.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.319+0000 7f5c06dbd740 1 -- 172.21.15.114:0/2426938232 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55d59cd813c0 con 0x55d59ce2ab30 2024-09-13T18:28:50.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.319+0000 7f5be17fa640 1 -- 172.21.15.114:0/2426938232 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d59cdacb80 con 0x55d59ce2ab30 2024-09-13T18:28:50.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.319+0000 7f5be17fa640 1 -- 172.21.15.114:0/2426938232 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5bf40c1e00 con 0x55d59ce2ab30 2024-09-13T18:28:50.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.320+0000 7f5c06dbd740 10 client.26890 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:28:50.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.320+0000 7f5c06dbd740 10 client.26890 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-13T18:28:50.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.320+0000 7f5c06dbd740 20 client.26890 populate_metadata read hostname 'smithi114' 2024-09-13T18:28:50.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.320+0000 7f5c06dbd740 1 --2- 172.21.15.114:0/2426938232 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55d59ce3a270 0x55d59ce3c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:50.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.320+0000 7f5c06dbd740 4 client.26890 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:28:50.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.320+0000 7f5c06dbd740 1 -- 172.21.15.114:0/2426938232 --> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d59cd751f0 con 0x55d59ce3a270 2024-09-13T18:28:50.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.325+0000 7f5c01d58640 1 --2- 172.21.15.114:0/2426938232 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55d59ce3a270 0x55d59ce3c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:50.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.331+0000 7f5c01d58640 1 --2- 172.21.15.114:0/2426938232 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55d59ce3a270 0x55d59ce3c6c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ec1bf0150bf1e965 server_cookie=fe00576b98d95fe6 in_seq=0 out_seq=0 2024-09-13T18:28:50.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.332+0000 7f5be17fa640 10 client.26890 ms_handle_connect on v2:172.21.15.204:6838/344883912 2024-09-13T18:28:50.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.335+0000 7f5be17fa640 1 -- 172.21.15.114:0/2426938232 <== mds.2 v2:172.21.15.204:6838/344883912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55d59cd751f0 con 0x55d59ce3a270 2024-09-13T18:28:50.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.335+0000 7f5be17fa640 10 client.26890 handle_command_reply: tid=0 2024-09-13T18:28:50.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.339+0000 7f5c06dbd740 1 client.26890 shutdown 2024-09-13T18:28:50.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.341+0000 7f5c06dbd740 1 -- 172.21.15.114:0/2426938232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d59ce2ab30 msgr2=0x55d59ce274c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:50.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.341+0000 7f5c06dbd740 1 --2- 172.21.15.114:0/2426938232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d59ce2ab30 0x55d59ce274c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5bf40c1d60 tx=0x7f5bf40b9e10 comp rx=0 tx=0).stop 2024-09-13T18:28:50.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.345+0000 7f5c06dbd740 1 -- 172.21.15.114:0/2426938232 shutdown_connections 2024-09-13T18:28:50.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.345+0000 7f5c06dbd740 1 --2- 172.21.15.114:0/2426938232 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55d59ce3a270 0x55d59ce3c6c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.345+0000 7f5c06dbd740 1 --2- 172.21.15.114:0/2426938232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d59ce2ab30 0x55d59ce274c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.345+0000 7f5c06dbd740 1 --2- 172.21.15.114:0/2426938232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d59ce20290 0x55d59ce2a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.345+0000 7f5c06dbd740 1 --2- 172.21.15.114:0/2426938232 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d59ce1f970 0x55d59ce29fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.345+0000 7f5c06dbd740 1 -- 172.21.15.114:0/2426938232 >> 172.21.15.114:0/2426938232 conn(0x55d59ce1bee0 msgr2=0x55d59ce1b230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:50.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.345+0000 7f5be17fa640 0 client.26890 ms_handle_reset on v2:172.21.15.204:6838/344883912 2024-09-13T18:28:50.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.349+0000 7f5c06dbd740 1 -- 172.21.15.114:0/2426938232 shutdown_connections 2024-09-13T18:28:50.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.351+0000 7f5c06dbd740 1 -- 172.21.15.114:0/2426938232 wait complete. 2024-09-13T18:28:50.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.354+0000 7f5c06dbd740 20 client.26890 trim_cache size 0 max 16384 2024-09-13T18:28:50.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.390+0000 7f5c06dbd740 1 Processor -- start 2024-09-13T18:28:50.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.393+0000 7f5c06dbd740 1 -- start start 2024-09-13T18:28:50.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.393+0000 7f5c06dbd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d59ce1f970 0x55d59ce41060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:50.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.393+0000 7f5c06dbd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d59ce20290 0x55d59ce318a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:50.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.393+0000 7f5c06dbd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d59ce2ab30 0x55d59ce31de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:50.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.394+0000 7f5c06dbd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d59ce17d00 con 0x55d59ce20290 2024-09-13T18:28:50.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.394+0000 7f5c01d58640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d59ce1f970 0x55d59ce41060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:50.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.394+0000 7f5c01d58640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d59ce1f970 0x55d59ce41060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:55986/0 (socket says 172.21.15.114:55986) 2024-09-13T18:28:50.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.394+0000 7f5c01d58640 1 -- 172.21.15.114:0/3635607621 learned_addr learned my addr 172.21.15.114:0/3635607621 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:50.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.394+0000 7f5c06dbd740 1 -- 172.21.15.114:0/3635607621 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d59ce166f0 con 0x55d59ce1f970 2024-09-13T18:28:50.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.394+0000 7f5c06dbd740 1 -- 172.21.15.114:0/3635607621 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d59ce28680 con 0x55d59ce2ab30 2024-09-13T18:28:50.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.394+0000 7f5c02559640 1 --2- 172.21.15.114:0/3635607621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d59ce2ab30 0x55d59ce31de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:50.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.394+0000 7f5c02559640 1 -- 172.21.15.114:0/3635607621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d59ce1f970 msgr2=0x55d59ce41060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:50.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.394+0000 7f5c02559640 1 --2- 172.21.15.114:0/3635607621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d59ce1f970 0x55d59ce41060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.394+0000 7f5c02559640 1 -- 172.21.15.114:0/3635607621 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d59ce20290 msgr2=0x55d59ce318a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:50.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.394+0000 7f5c02559640 1 --2- 172.21.15.114:0/3635607621 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d59ce20290 0x55d59ce318a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.395+0000 7f5c02559640 1 -- 172.21.15.114:0/3635607621 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d59cd23770 con 0x55d59ce2ab30 2024-09-13T18:28:50.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.395+0000 7f5c02559640 1 --2- 172.21.15.114:0/3635607621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d59ce2ab30 0x55d59ce31de0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5bf40b8160 tx=0x7f5bf40b83c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ead0ccaeb9401fc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:50.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.397+0000 7f5bf8ff9640 1 -- 172.21.15.114:0/3635607621 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf40eaba0 con 0x55d59ce2ab30 2024-09-13T18:28:50.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.398+0000 7f5bf8ff9640 1 -- 172.21.15.114:0/3635607621 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5bf40b7500 con 0x55d59ce2ab30 2024-09-13T18:28:50.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.398+0000 7f5bf8ff9640 1 -- 172.21.15.114:0/3635607621 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf40ea6d0 con 0x55d59ce2ab30 2024-09-13T18:28:50.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.398+0000 7f5c06dbd740 1 -- 172.21.15.114:0/3635607621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d59ce2ab30 msgr2=0x55d59ce31de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:50.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.398+0000 7f5c06dbd740 1 --2- 172.21.15.114:0/3635607621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d59ce2ab30 0x55d59ce31de0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5bf40b8160 tx=0x7f5bf40b83c0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.399+0000 7f5c06dbd740 1 -- 172.21.15.114:0/3635607621 shutdown_connections 2024-09-13T18:28:50.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.399+0000 7f5c06dbd740 1 --2- 172.21.15.114:0/3635607621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d59ce2ab30 0x55d59ce31de0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.399+0000 7f5c06dbd740 1 --2- 172.21.15.114:0/3635607621 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d59ce20290 0x55d59ce318a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.399+0000 7f5c06dbd740 1 --2- 172.21.15.114:0/3635607621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d59ce1f970 0x55d59ce41060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.399+0000 7f5c06dbd740 1 -- 172.21.15.114:0/3635607621 >> 172.21.15.114:0/3635607621 conn(0x55d59ce1bee0 msgr2=0x55d59ce1c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:50.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.408+0000 7f5c06dbd740 1 -- 172.21.15.114:0/3635607621 shutdown_connections 2024-09-13T18:28:50.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.408+0000 7f5c06dbd740 1 -- 172.21.15.114:0/3635607621 wait complete. 2024-09-13T18:28:50.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.408+0000 7f5c06dbd740 1 -- start start 2024-09-13T18:28:50.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.408+0000 7f5c06dbd740 1 Processor -- start 2024-09-13T18:28:50.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.409+0000 7f5c06dbd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d59ce1f970 0x55d59ce2f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:50.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.409+0000 7f5c06dbd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d59ce20290 0x55d59ce2fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:50.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.409+0000 7f5c06dbd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d59ce2ab30 0x55d59ce8f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:50.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.409+0000 7f5c06dbd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d59ce306c0 con 0x55d59ce20290 2024-09-13T18:28:50.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.409+0000 7f5c06dbd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d59ce8fd20 con 0x55d59ce1f970 2024-09-13T18:28:50.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.409+0000 7f5c06dbd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d59cea0ea0 con 0x55d59ce2ab30 2024-09-13T18:28:50.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.409+0000 7f5c06dbd740 10 client.0 _ll_register_callbacks cb 0x7f5bf0009060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:28:50.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.409+0000 7f5c01557640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d59ce20290 0x55d59ce2fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:50.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.409+0000 7f5c01557640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d59ce20290 0x55d59ce2fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47438/0 (socket says 172.21.15.114:47438) 2024-09-13T18:28:50.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.409+0000 7f5c01557640 1 -- 172.21.15.114:0/1708496496 learned_addr learned my addr 172.21.15.114:0/1708496496 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:50.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.411+0000 7f5c02559640 1 --2- 172.21.15.114:0/1708496496 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d59ce2ab30 0x55d59ce8f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:50.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.411+0000 7f5c01d58640 1 --2- 172.21.15.114:0/1708496496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d59ce1f970 0x55d59ce2f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:50.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.416+0000 7f5c01557640 1 -- 172.21.15.114:0/1708496496 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d59ce2ab30 msgr2=0x55d59ce8f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:50.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.416+0000 7f5c01557640 1 --2- 172.21.15.114:0/1708496496 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d59ce2ab30 0x55d59ce8f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.416+0000 7f5c01557640 1 -- 172.21.15.114:0/1708496496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d59ce1f970 msgr2=0x55d59ce2f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:50.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.416+0000 7f5c01557640 1 --2- 172.21.15.114:0/1708496496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d59ce1f970 0x55d59ce2f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.416+0000 7f5c01557640 1 -- 172.21.15.114:0/1708496496 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d59cd813c0 con 0x55d59ce20290 2024-09-13T18:28:50.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.416+0000 7f5c01557640 1 --2- 172.21.15.114:0/1708496496 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d59ce20290 0x55d59ce2fdc0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x55d59ce22510 tx=0x7f5bec017830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bc6fcc51adee235 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:50.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.417+0000 7f5be17fa640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:28:50.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.418+0000 7f5be17fa640 1 -- 172.21.15.114:0/1708496496 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bec018610 con 0x55d59ce20290 2024-09-13T18:28:50.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.419+0000 7f5c06dbd740 1 -- 172.21.15.114:0/1708496496 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5bf0009e90 con 0x55d59ce20290 2024-09-13T18:28:50.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.419+0000 7f5be17fa640 1 -- 172.21.15.114:0/1708496496 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f5bec018770 con 0x55d59ce20290 2024-09-13T18:28:50.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.420+0000 7f5be17fa640 1 -- 172.21.15.114:0/1708496496 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5bec007610 con 0x55d59ce20290 2024-09-13T18:28:50.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.424+0000 7f5c06dbd740 10 client.17475 fetch_fsmap learned FSMap version 34 2024-09-13T18:28:50.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.424+0000 7f5c06dbd740 1 -- 172.21.15.114:0/1708496496 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55d59cd48e80 con 0x55d59ce20290 2024-09-13T18:28:50.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.425+0000 7f5be17fa640 1 -- 172.21.15.114:0/1708496496 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5bf0009e90 con 0x55d59ce20290 2024-09-13T18:28:50.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.425+0000 7f5be17fa640 1 -- 172.21.15.114:0/1708496496 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5bec021070 con 0x55d59ce20290 2024-09-13T18:28:50.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.425+0000 7f5c06dbd740 10 client.17475 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:28:50.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.425+0000 7f5c06dbd740 10 client.17475 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-13T18:28:50.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.425+0000 7f5c06dbd740 20 client.17475 populate_metadata read hostname 'smithi114' 2024-09-13T18:28:50.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.425+0000 7f5c06dbd740 1 --2- 172.21.15.114:0/1708496496 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55d59cea2790 0x55d59cea4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:50.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.425+0000 7f5c06dbd740 4 client.17475 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:28:50.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.425+0000 7f5c06dbd740 1 -- 172.21.15.114:0/1708496496 --> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d59ca764e0 con 0x55d59cea2790 2024-09-13T18:28:50.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.428+0000 7f5c01d58640 1 --2- 172.21.15.114:0/1708496496 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55d59cea2790 0x55d59cea4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:50.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.429+0000 7f5be17fa640 10 client.17475 ms_handle_connect on v2:172.21.15.204:6838/344883912 2024-09-13T18:28:50.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.429+0000 7f5c01d58640 1 --2- 172.21.15.114:0/1708496496 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55d59cea2790 0x55d59cea4bb0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5cef8df772e615f8 server_cookie=1b199e477166b49e in_seq=0 out_seq=0 2024-09-13T18:28:50.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.432+0000 7f5be17fa640 1 -- 172.21.15.114:0/1708496496 <== mds.2 v2:172.21.15.204:6838/344883912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d59ca764e0 con 0x55d59cea2790 2024-09-13T18:28:50.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.432+0000 7f5be17fa640 10 client.17475 handle_command_reply: tid=0 2024-09-13T18:28:50.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.432+0000 7f5c06dbd740 1 client.17475 shutdown 2024-09-13T18:28:50.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.432+0000 7f5c06dbd740 1 -- 172.21.15.114:0/1708496496 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d59ce20290 msgr2=0x55d59ce2fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:50.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.432+0000 7f5c06dbd740 1 --2- 172.21.15.114:0/1708496496 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d59ce20290 0x55d59ce2fdc0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x55d59ce22510 tx=0x7f5bec017830 comp rx=0 tx=0).stop 2024-09-13T18:28:50.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.436+0000 7f5c06dbd740 1 -- 172.21.15.114:0/1708496496 shutdown_connections 2024-09-13T18:28:50.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.436+0000 7f5c06dbd740 1 --2- 172.21.15.114:0/1708496496 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55d59cea2790 0x55d59cea4bb0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.436+0000 7f5c06dbd740 1 --2- 172.21.15.114:0/1708496496 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d59ce2ab30 0x55d59ce8f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.436+0000 7f5c06dbd740 1 --2- 172.21.15.114:0/1708496496 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d59ce20290 0x55d59ce2fdc0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.436+0000 7f5c06dbd740 1 --2- 172.21.15.114:0/1708496496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d59ce1f970 0x55d59ce2f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.436+0000 7f5c06dbd740 1 -- 172.21.15.114:0/1708496496 >> 172.21.15.114:0/1708496496 conn(0x55d59ce1bee0 msgr2=0x55d59ce1c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:50.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.437+0000 7f5be17fa640 0 client.17475 ms_handle_reset on v2:172.21.15.204:6838/344883912 2024-09-13T18:28:50.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.439+0000 7f5c06dbd740 1 -- 172.21.15.114:0/1708496496 shutdown_connections 2024-09-13T18:28:50.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.439+0000 7f5c06dbd740 1 -- 172.21.15.114:0/1708496496 wait complete. 2024-09-13T18:28:50.442 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:28:50.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.440+0000 7f5c06dbd740 20 client.17475 trim_cache size 0 max 16384 2024-09-13T18:28:50.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.446+0000 7f5c02d5a640 1 -- 172.21.15.114:0/3733354415 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f5be40a1390 msgr2=0x7f5be40a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:50.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.446+0000 7f5c02d5a640 1 --2- 172.21.15.114:0/3733354415 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f5be40a1390 0x7f5be40a3850 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x55d59ce1d260 tx=0x7f5bec009790 comp rx=0 tx=0).stop 2024-09-13T18:28:50.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.446+0000 7f5c02d5a640 1 -- 172.21.15.114:0/3733354415 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5bfc119400 msgr2=0x7f5bfc1a98d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:50.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.446+0000 7f5c02d5a640 1 --2- 172.21.15.114:0/3733354415 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5bfc119400 0x7f5bfc1a98d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4009850 tx=0x7f5bf4003cf0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.446+0000 7f5c02d5a640 1 -- 172.21.15.114:0/3733354415 shutdown_connections 2024-09-13T18:28:50.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.446+0000 7f5c02d5a640 1 --2- 172.21.15.114:0/3733354415 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f5be40a1390 0x7f5be40a3850 secure :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0x55d59ce1d260 tx=0x7f5bec009790 comp rx=0 tx=0).stop 2024-09-13T18:28:50.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.446+0000 7f5c02d5a640 1 --2- 172.21.15.114:0/3733354415 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5bfc119400 0x7f5bfc1a98d0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.446+0000 7f5c02d5a640 1 --2- 172.21.15.114:0/3733354415 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5bfc107890 0x7f5bfc1a54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.446+0000 7f5c02d5a640 1 --2- 172.21.15.114:0/3733354415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bfc1074b0 0x7f5bfc1a4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.446+0000 7f5c02d5a640 1 -- 172.21.15.114:0/3733354415 >> 172.21.15.114:0/3733354415 conn(0x7f5bfc06af40 msgr2=0x7f5bfc119c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:50.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.451+0000 7f5c02d5a640 1 -- 172.21.15.114:0/3733354415 shutdown_connections 2024-09-13T18:28:50.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.452+0000 7f5c02d5a640 1 -- 172.21.15.114:0/3733354415 wait complete. 2024-09-13T18:28:50.483 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-09-13T18:28:50.640 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.635+0000 7f7e805ff640 1 -- 172.21.15.114:0/70197889 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7e78119400 msgr2=0x7f7e7811b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:50.640 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.635+0000 7f7e805ff640 1 --2- 172.21.15.114:0/70197889 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7e78119400 0x7f7e7811b7f0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f7e70009cd0 tx=0x7f7e7002a930 comp rx=0 tx=0).stop 2024-09-13T18:28:50.640 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.635+0000 7f7e805ff640 1 -- 172.21.15.114:0/70197889 shutdown_connections 2024-09-13T18:28:50.640 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.635+0000 7f7e805ff640 1 --2- 172.21.15.114:0/70197889 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7e78119400 0x7f7e7811b7f0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.640 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.635+0000 7f7e805ff640 1 --2- 172.21.15.114:0/70197889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e78107890 0x7f7e78107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.641 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.635+0000 7f7e805ff640 1 --2- 172.21.15.114:0/70197889 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e781074b0 0x7f7e7810e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.641 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.635+0000 7f7e805ff640 1 -- 172.21.15.114:0/70197889 >> 172.21.15.114:0/70197889 conn(0x7f7e7806ae30 msgr2=0x7f7e7806b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:50.641 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.636+0000 7f7e805ff640 1 -- 172.21.15.114:0/70197889 shutdown_connections 2024-09-13T18:28:50.641 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.636+0000 7f7e805ff640 1 -- 172.21.15.114:0/70197889 wait complete. 2024-09-13T18:28:50.641 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.636+0000 7f7e805ff640 1 Processor -- start 2024-09-13T18:28:50.641 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.637+0000 7f7e805ff640 1 -- start start 2024-09-13T18:28:50.641 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.637+0000 7f7e805ff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e781074b0 0x7f7e781a4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:50.641 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.637+0000 7f7e805ff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e78107890 0x7f7e781a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:50.641 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.637+0000 7f7e805ff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7e78119400 0x7f7e781a97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:50.641 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.637+0000 7f7e7e39b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e781074b0 0x7f7e781a4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:50.641 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.637+0000 7f7e7e39b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e781074b0 0x7f7e781a4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:56008/0 (socket says 172.21.15.114:56008) 2024-09-13T18:28:50.641 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.637+0000 7f7e7e39b640 1 -- 172.21.15.114:0/4250305988 learned_addr learned my addr 172.21.15.114:0/4250305988 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:50.641 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.637+0000 7f7e7db9a640 1 --2- 172.21.15.114:0/4250305988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e78107890 0x7f7e781a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:50.642 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.637+0000 7f7e805ff640 1 -- 172.21.15.114:0/4250305988 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e781a9db0 con 0x7f7e78119400 2024-09-13T18:28:50.642 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.638+0000 7f7e805ff640 1 -- 172.21.15.114:0/4250305988 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e781a9f20 con 0x7f7e781074b0 2024-09-13T18:28:50.642 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.638+0000 7f7e805ff640 1 -- 172.21.15.114:0/4250305988 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e781aa0e0 con 0x7f7e78107890 2024-09-13T18:28:50.643 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.638+0000 7f7e7db9a640 1 -- 172.21.15.114:0/4250305988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e781074b0 msgr2=0x7f7e781a4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:50.643 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.638+0000 7f7e7db9a640 1 --2- 172.21.15.114:0/4250305988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e781074b0 0x7f7e781a4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.643 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.638+0000 7f7e7db9a640 1 -- 172.21.15.114:0/4250305988 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7e78119400 msgr2=0x7f7e781a97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:50.643 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.638+0000 7f7e7db9a640 1 --2- 172.21.15.114:0/4250305988 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7e78119400 0x7f7e781a97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.643 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.638+0000 7f7e7db9a640 1 -- 172.21.15.114:0/4250305988 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e70009900 con 0x7f7e78107890 2024-09-13T18:28:50.643 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.639+0000 7f7e7db9a640 1 --2- 172.21.15.114:0/4250305988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e78107890 0x7f7e781a53c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7e6800e9f0 tx=0x7f7e6800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:50.644 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.639+0000 7f7e6f7fe640 1 -- 172.21.15.114:0/4250305988 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6800ccc0 con 0x7f7e78107890 2024-09-13T18:28:50.644 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.639+0000 7f7e6f7fe640 1 -- 172.21.15.114:0/4250305988 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7e6800ce20 con 0x7f7e78107890 2024-09-13T18:28:50.644 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.639+0000 7f7e6f7fe640 1 -- 172.21.15.114:0/4250305988 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e68018710 con 0x7f7e78107890 2024-09-13T18:28:50.644 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.639+0000 7f7e805ff640 1 -- 172.21.15.114:0/4250305988 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e781aa3c0 con 0x7f7e78107890 2024-09-13T18:28:50.644 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.639+0000 7f7e805ff640 1 -- 172.21.15.114:0/4250305988 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e781b1a60 con 0x7f7e78107890 2024-09-13T18:28:50.647 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.645+0000 7f7e833fd740 1 Processor -- start 2024-09-13T18:28:50.647 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.645+0000 7f7e833fd740 1 -- start start 2024-09-13T18:28:50.648 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.646+0000 7f7e833fd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad69344500 0x55ad693448e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:50.648 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.646+0000 7f7e833fd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ad69344e20 0x55ad6934f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:50.648 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.646+0000 7f7e833fd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad6934f6c0 0x55ad69351ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:50.648 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.646+0000 7f7e833fd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad692afc30 con 0x55ad69344e20 2024-09-13T18:28:50.649 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.646+0000 7f7e833fd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad692ad3c0 con 0x55ad6934f6c0 2024-09-13T18:28:50.649 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.646+0000 7f7e833fd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad692a8cb0 con 0x55ad69344500 2024-09-13T18:28:50.652 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.646+0000 7f7e7e39b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ad69344e20 0x55ad6934f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:50.652 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.646+0000 7f7e7e39b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ad69344e20 0x55ad6934f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47468/0 (socket says 172.21.15.114:47468) 2024-09-13T18:28:50.653 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.646+0000 7f7e7e39b640 1 -- 172.21.15.114:0/752802531 learned_addr learned my addr 172.21.15.114:0/752802531 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:50.653 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.647+0000 7f7e6f7fe640 1 -- 172.21.15.114:0/4250305988 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f7e68018870 con 0x7f7e78107890 2024-09-13T18:28:50.653 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.648+0000 7f7e6f7fe640 1 --2- 172.21.15.114:0/4250305988 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7e640a14d0 0x7f7e640a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:50.653 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.648+0000 7f7e6f7fe640 1 -- 172.21.15.114:0/4250305988 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f7e68014070 con 0x7f7e78107890 2024-09-13T18:28:50.653 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.648+0000 7f7e7eb9c640 1 --2- 172.21.15.114:0/752802531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad69344500 0x55ad693448e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:50.653 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.649+0000 7f7e7db9a640 1 --2- 172.21.15.114:0/752802531 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad6934f6c0 0x55ad69351ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:50.653 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.649+0000 7f7e7eb9c640 1 -- 172.21.15.114:0/752802531 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad6934f6c0 msgr2=0x55ad69351ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:50.653 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.649+0000 7f7e7eb9c640 1 --2- 172.21.15.114:0/752802531 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad6934f6c0 0x55ad69351ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.654 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.649+0000 7f7e7eb9c640 1 -- 172.21.15.114:0/752802531 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ad69344e20 msgr2=0x55ad6934f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:50.654 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.649+0000 7f7e7eb9c640 1 --2- 172.21.15.114:0/752802531 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ad69344e20 0x55ad6934f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.654 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.649+0000 7f7e7eb9c640 1 -- 172.21.15.114:0/752802531 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad6924f910 con 0x55ad69344500 2024-09-13T18:28:50.654 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.649+0000 7f7e7eb9c640 1 --2- 172.21.15.114:0/4250305988 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7e640a14d0 0x7f7e640a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:50.654 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.649+0000 7f7e7eb9c640 1 --2- 172.21.15.114:0/752802531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad69344500 0x55ad693448e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7e7002af20 tx=0x7f7e7002f340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0c7368b8a81e958 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:50.654 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.649+0000 7f7e6cff9640 1 -- 172.21.15.114:0/752802531 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e70004240 con 0x55ad69344500 2024-09-13T18:28:50.654 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.650+0000 7f7e6cff9640 1 -- 172.21.15.114:0/752802531 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7e70031910 con 0x55ad69344500 2024-09-13T18:28:50.654 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.650+0000 7f7e6cff9640 1 -- 172.21.15.114:0/752802531 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e7002fa40 con 0x55ad69344500 2024-09-13T18:28:50.654 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.650+0000 7f7e7eb9c640 1 --2- 172.21.15.114:0/4250305988 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7e640a14d0 0x7f7e640a3990 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7e70047000 tx=0x7f7e7004c000 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:50.655 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.651+0000 7f7e833fd740 1 -- 172.21.15.114:0/752802531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad69344500 msgr2=0x55ad693448e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:50.655 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.651+0000 7f7e833fd740 1 --2- 172.21.15.114:0/752802531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad69344500 0x55ad693448e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7e7002af20 tx=0x7f7e7002f340 comp rx=0 tx=0).stop 2024-09-13T18:28:50.655 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.651+0000 7f7e833fd740 1 -- 172.21.15.114:0/752802531 shutdown_connections 2024-09-13T18:28:50.655 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.651+0000 7f7e833fd740 1 --2- 172.21.15.114:0/752802531 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad6934f6c0 0x55ad69351ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.655 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.651+0000 7f7e833fd740 1 --2- 172.21.15.114:0/752802531 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ad69344e20 0x55ad6934f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.655 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.651+0000 7f7e833fd740 1 --2- 172.21.15.114:0/752802531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad69344500 0x55ad693448e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.655 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.651+0000 7f7e833fd740 1 -- 172.21.15.114:0/752802531 >> 172.21.15.114:0/752802531 conn(0x55ad69340a90 msgr2=0x55ad69340e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:50.655 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.652+0000 7f7e833fd740 1 -- 172.21.15.114:0/752802531 shutdown_connections 2024-09-13T18:28:50.657 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.654+0000 7f7e833fd740 1 -- 172.21.15.114:0/752802531 wait complete. 2024-09-13T18:28:50.657 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.655+0000 7f7e833fd740 1 -- start start 2024-09-13T18:28:50.657 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.655+0000 7f7e833fd740 1 Processor -- start 2024-09-13T18:28:50.661 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.656+0000 7f7e833fd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad69344500 0x55ad6934f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:50.661 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.656+0000 7f7e833fd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad69344e20 0x55ad6934c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:50.661 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.656+0000 7f7e833fd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ad6934f6c0 0x55ad6934c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:50.661 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.656+0000 7f7e833fd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad69266e80 con 0x55ad6934f6c0 2024-09-13T18:28:50.661 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.656+0000 7f7e833fd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad69266b20 con 0x55ad69344e20 2024-09-13T18:28:50.661 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.656+0000 7f7e833fd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad6922c260 con 0x55ad69344500 2024-09-13T18:28:50.662 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.656+0000 7f7e833fd740 10 client.0 _ll_register_callbacks cb 0x55ad69261df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:28:50.662 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.656+0000 7f7e7e39b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad69344500 0x55ad6934f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:50.662 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.656+0000 7f7e7e39b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad69344500 0x55ad6934f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:55540/0 (socket says 172.21.15.114:55540) 2024-09-13T18:28:50.662 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.656+0000 7f7e7e39b640 1 -- 172.21.15.114:0/922211662 learned_addr learned my addr 172.21.15.114:0/922211662 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:50.662 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.659+0000 7f7e7db9a640 1 --2- 172.21.15.114:0/922211662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad69344e20 0x55ad6934c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:50.663 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.660+0000 7f7e7e39b640 1 -- 172.21.15.114:0/922211662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad69344e20 msgr2=0x55ad6934c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:50.663 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.660+0000 7f7e7e39b640 1 --2- 172.21.15.114:0/922211662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad69344e20 0x55ad6934c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.663 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.660+0000 7f7e7e39b640 1 -- 172.21.15.114:0/922211662 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ad6934f6c0 msgr2=0x55ad6934c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:50.663 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.660+0000 7f7e7e39b640 1 --2- 172.21.15.114:0/922211662 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ad6934f6c0 0x55ad6934c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.663 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.660+0000 7f7e7e39b640 1 -- 172.21.15.114:0/922211662 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e7811c030 con 0x55ad69344500 2024-09-13T18:28:50.663 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.661+0000 7f7e7db9a640 1 --2- 172.21.15.114:0/922211662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad69344e20 0x55ad6934c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:28:50.670 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.668+0000 7f7e7e39b640 1 --2- 172.21.15.114:0/922211662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad69344500 0x55ad6934f040 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7e74012860 tx=0x7f7e74012c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abb176dbf121ace6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:50.670 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.668+0000 7f7e61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:28:50.676 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.674+0000 7f7e61ffb640 1 -- 172.21.15.114:0/922211662 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e74017070 con 0x55ad69344500 2024-09-13T18:28:50.676 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.674+0000 7f7e61ffb640 1 -- 172.21.15.114:0/922211662 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f7e7400f920 con 0x55ad69344500 2024-09-13T18:28:50.676 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.674+0000 7f7e833fd740 1 -- 172.21.15.114:0/922211662 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ad692d1680 con 0x55ad69344500 2024-09-13T18:28:50.684 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.682+0000 7f7e61ffb640 1 -- 172.21.15.114:0/922211662 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e740122e0 con 0x55ad69344500 2024-09-13T18:28:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.682+0000 7f7e833fd740 10 client.26908 fetch_fsmap learned FSMap version 34 2024-09-13T18:28:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.682+0000 7f7e833fd740 1 -- 172.21.15.114:0/922211662 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55ad692a7330 con 0x55ad69344500 2024-09-13T18:28:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.687+0000 7f7e61ffb640 1 -- 172.21.15.114:0/922211662 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ad692d1680 con 0x55ad69344500 2024-09-13T18:28:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.687+0000 7f7e61ffb640 1 -- 172.21.15.114:0/922211662 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7e74010c00 con 0x55ad69344500 2024-09-13T18:28:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.687+0000 7f7e833fd740 10 client.26908 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:28:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.687+0000 7f7e833fd740 10 client.26908 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:28:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.687+0000 7f7e833fd740 20 client.26908 populate_metadata read hostname 'smithi114' 2024-09-13T18:28:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.687+0000 7f7e833fd740 1 --2- 172.21.15.114:0/922211662 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55ad6935f1a0 0x55ad693615e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.687+0000 7f7e833fd740 4 client.26908 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:28:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.688+0000 7f7e833fd740 1 -- 172.21.15.114:0/922211662 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ad6929c5d0 con 0x55ad6935f1a0 2024-09-13T18:28:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.689+0000 7f7e7db9a640 1 --2- 172.21.15.114:0/922211662 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55ad6935f1a0 0x55ad693615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.693+0000 7f7e7db9a640 1 --2- 172.21.15.114:0/922211662 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55ad6935f1a0 0x55ad693615e0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=984e36090c44a15f server_cookie=b1f38d8391d3f63c in_seq=0 out_seq=0 2024-09-13T18:28:50.701 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.698+0000 7f7e61ffb640 10 client.26908 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:28:50.701 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.698+0000 7f7e61ffb640 1 -- 172.21.15.114:0/922211662 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55ad6929c5d0 con 0x55ad6935f1a0 2024-09-13T18:28:50.701 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.698+0000 7f7e61ffb640 10 client.26908 handle_command_reply: tid=0 2024-09-13T18:28:50.701 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.698+0000 7f7e833fd740 1 client.26908 shutdown 2024-09-13T18:28:50.701 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.699+0000 7f7e833fd740 1 -- 172.21.15.114:0/922211662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad69344500 msgr2=0x55ad6934f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:50.701 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.699+0000 7f7e833fd740 1 --2- 172.21.15.114:0/922211662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad69344500 0x55ad6934f040 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7e74012860 tx=0x7f7e74012c60 comp rx=0 tx=0).stop 2024-09-13T18:28:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.704+0000 7f7e833fd740 1 -- 172.21.15.114:0/922211662 shutdown_connections 2024-09-13T18:28:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.704+0000 7f7e833fd740 1 --2- 172.21.15.114:0/922211662 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55ad6935f1a0 0x55ad693615e0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.704+0000 7f7e833fd740 1 --2- 172.21.15.114:0/922211662 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ad6934f6c0 0x55ad6934c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.704+0000 7f7e833fd740 1 --2- 172.21.15.114:0/922211662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad69344e20 0x55ad6934c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.704+0000 7f7e833fd740 1 --2- 172.21.15.114:0/922211662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad69344500 0x55ad6934f040 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.704+0000 7f7e833fd740 1 -- 172.21.15.114:0/922211662 >> 172.21.15.114:0/922211662 conn(0x55ad69340a90 msgr2=0x55ad6933fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.706+0000 7f7e61ffb640 0 client.26908 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:28:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.706+0000 7f7e833fd740 1 -- 172.21.15.114:0/922211662 shutdown_connections 2024-09-13T18:28:50.710 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.708+0000 7f7e833fd740 1 -- 172.21.15.114:0/922211662 wait complete. 2024-09-13T18:28:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.716+0000 7f7e833fd740 20 client.26908 trim_cache size 0 max 16384 2024-09-13T18:28:50.750 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.747+0000 7f7e833fd740 1 Processor -- start 2024-09-13T18:28:50.750 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.747+0000 7f7e833fd740 1 -- start start 2024-09-13T18:28:50.750 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.747+0000 7f7e833fd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ad69344500 0x55ad6934eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:50.750 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.747+0000 7f7e833fd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad69344e20 0x55ad693563c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:50.750 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.747+0000 7f7e833fd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad6934f6c0 0x55ad69356900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:50.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.748+0000 7f7e833fd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad69352170 con 0x55ad69344500 2024-09-13T18:28:50.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.748+0000 7f7e833fd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad69261d10 con 0x55ad6934f6c0 2024-09-13T18:28:50.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.748+0000 7f7e833fd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad6934d640 con 0x55ad69344e20 2024-09-13T18:28:50.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.748+0000 7f7e7eb9c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad6934f6c0 0x55ad69356900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:50.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.748+0000 7f7e7eb9c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad6934f6c0 0x55ad69356900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:56018/0 (socket says 172.21.15.114:56018) 2024-09-13T18:28:50.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.748+0000 7f7e7eb9c640 1 -- 172.21.15.114:0/996207257 learned_addr learned my addr 172.21.15.114:0/996207257 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:50.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.748+0000 7f7e7e39b640 1 --2- 172.21.15.114:0/996207257 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ad69344500 0x55ad6934eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:50.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.748+0000 7f7e7e39b640 1 -- 172.21.15.114:0/996207257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad69344e20 msgr2=0x55ad693563c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:28:50.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.748+0000 7f7e7e39b640 1 --2- 172.21.15.114:0/996207257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad69344e20 0x55ad693563c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.748+0000 7f7e7e39b640 1 -- 172.21.15.114:0/996207257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad6934f6c0 msgr2=0x55ad69356900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:50.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.748+0000 7f7e7e39b640 1 --2- 172.21.15.114:0/996207257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad6934f6c0 0x55ad69356900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.748+0000 7f7e7e39b640 1 -- 172.21.15.114:0/996207257 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad69249300 con 0x55ad69344500 2024-09-13T18:28:50.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.749+0000 7f7e7e39b640 1 --2- 172.21.15.114:0/996207257 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ad69344500 0x55ad6934eb50 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f7e74000c00 tx=0x7f7e74005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0b7a2c20d621b21 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:50.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.751+0000 7f7e6d7fa640 1 -- 172.21.15.114:0/996207257 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e74017040 con 0x55ad69344500 2024-09-13T18:28:50.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.751+0000 7f7e6d7fa640 1 -- 172.21.15.114:0/996207257 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7e7401c6e0 con 0x55ad69344500 2024-09-13T18:28:50.756 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.752+0000 7f7e833fd740 1 -- 172.21.15.114:0/996207257 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ad69344500 msgr2=0x55ad6934eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:50.756 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.752+0000 7f7e833fd740 1 --2- 172.21.15.114:0/996207257 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ad69344500 0x55ad6934eb50 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f7e74000c00 tx=0x7f7e74005db0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.759 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.757+0000 7f7e833fd740 1 -- 172.21.15.114:0/996207257 shutdown_connections 2024-09-13T18:28:50.759 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.757+0000 7f7e833fd740 1 --2- 172.21.15.114:0/996207257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad6934f6c0 0x55ad69356900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.759 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.757+0000 7f7e833fd740 1 --2- 172.21.15.114:0/996207257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad69344e20 0x55ad693563c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.759 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.757+0000 7f7e833fd740 1 --2- 172.21.15.114:0/996207257 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ad69344500 0x55ad6934eb50 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.759 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.757+0000 7f7e833fd740 1 -- 172.21.15.114:0/996207257 >> 172.21.15.114:0/996207257 conn(0x55ad69340a90 msgr2=0x55ad69340e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:50.763 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.761+0000 7f7e833fd740 1 -- 172.21.15.114:0/996207257 shutdown_connections 2024-09-13T18:28:50.764 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.761+0000 7f7e833fd740 1 -- 172.21.15.114:0/996207257 wait complete. 2024-09-13T18:28:50.764 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.761+0000 7f7e833fd740 1 -- start start 2024-09-13T18:28:50.764 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.762+0000 7f7e833fd740 1 Processor -- start 2024-09-13T18:28:50.764 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.762+0000 7f7e833fd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad69344500 0x55ad693565a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:50.764 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.762+0000 7f7e833fd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad69344e20 0x55ad69354ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:50.764 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.762+0000 7f7e833fd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ad6934f6c0 0x55ad69354ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:50.765 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.762+0000 7f7e833fd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad69355530 con 0x55ad6934f6c0 2024-09-13T18:28:50.765 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.762+0000 7f7e833fd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad693556a0 con 0x55ad69344500 2024-09-13T18:28:50.765 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.762+0000 7f7e833fd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad69355860 con 0x55ad69344e20 2024-09-13T18:28:50.765 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.762+0000 7f7e833fd740 10 client.0 _ll_register_callbacks cb 0x7f7e68005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:28:50.765 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.762+0000 7f7e7db9a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad69344e20 0x55ad69354ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:50.765 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.762+0000 7f7e7eb9c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ad6934f6c0 0x55ad69354ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:50.765 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.762+0000 7f7e7db9a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad69344e20 0x55ad69354ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:55554/0 (socket says 172.21.15.114:55554) 2024-09-13T18:28:50.765 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.762+0000 7f7e7db9a640 1 -- 172.21.15.114:0/3908507776 learned_addr learned my addr 172.21.15.114:0/3908507776 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:50.766 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.763+0000 7f7e7eb9c640 1 -- 172.21.15.114:0/3908507776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad69344e20 msgr2=0x55ad69354ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:50.766 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.763+0000 7f7e7eb9c640 1 --2- 172.21.15.114:0/3908507776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad69344e20 0x55ad69354ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.766 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.763+0000 7f7e7eb9c640 1 -- 172.21.15.114:0/3908507776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad69344500 msgr2=0x55ad693565a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:50.766 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.763+0000 7f7e7eb9c640 1 --2- 172.21.15.114:0/3908507776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad69344500 0x55ad693565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.766 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.763+0000 7f7e7eb9c640 1 -- 172.21.15.114:0/3908507776 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ad692a7330 con 0x55ad6934f6c0 2024-09-13T18:28:50.767 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.764+0000 7f7e7eb9c640 1 --2- 172.21.15.114:0/3908507776 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ad6934f6c0 0x55ad69354ff0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f7e70005dc0 tx=0x7f7e7002f830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fecef0fd0d273949 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:50.767 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.765+0000 7f7e61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:28:50.768 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.765+0000 7f7e61ffb640 1 -- 172.21.15.114:0/3908507776 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e70005eb0 con 0x55ad6934f6c0 2024-09-13T18:28:50.768 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.765+0000 7f7e61ffb640 1 -- 172.21.15.114:0/3908507776 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f7e70030800 con 0x55ad6934f6c0 2024-09-13T18:28:50.768 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.766+0000 7f7e833fd740 1 -- 172.21.15.114:0/3908507776 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7e70031910 con 0x55ad6934f6c0 2024-09-13T18:28:50.768 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.766+0000 7f7e61ffb640 1 -- 172.21.15.114:0/3908507776 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e70009540 con 0x55ad6934f6c0 2024-09-13T18:28:50.772 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.770+0000 7f7e833fd740 10 client.17487 fetch_fsmap learned FSMap version 34 2024-09-13T18:28:50.772 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.770+0000 7f7e833fd740 1 -- 172.21.15.114:0/3908507776 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55ad6926e8e0 con 0x55ad6934f6c0 2024-09-13T18:28:50.773 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.771+0000 7f7e61ffb640 1 -- 172.21.15.114:0/3908507776 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7e70031910 con 0x55ad6934f6c0 2024-09-13T18:28:50.773 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.771+0000 7f7e61ffb640 1 -- 172.21.15.114:0/3908507776 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7e7005d070 con 0x55ad6934f6c0 2024-09-13T18:28:50.774 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.771+0000 7f7e833fd740 10 client.17487 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:28:50.774 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.771+0000 7f7e833fd740 10 client.17487 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:28:50.774 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.771+0000 7f7e833fd740 20 client.17487 populate_metadata read hostname 'smithi114' 2024-09-13T18:28:50.774 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.771+0000 7f7e833fd740 1 --2- 172.21.15.114:0/3908507776 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55ad693c7890 0x55ad693c9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:50.774 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.771+0000 7f7e833fd740 4 client.17487 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T18:28:50.774 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.771+0000 7f7e833fd740 1 -- 172.21.15.114:0/3908507776 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ad68f9c300 con 0x55ad693c7890 2024-09-13T18:28:50.775 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.773+0000 7f7e7e39b640 1 --2- 172.21.15.114:0/3908507776 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55ad693c7890 0x55ad693c9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:50.776 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.774+0000 7f7e7e39b640 1 --2- 172.21.15.114:0/3908507776 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55ad693c7890 0x55ad693c9c80 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f86aac76653d2ef server_cookie=7cdc867ff0942952 in_seq=0 out_seq=0 2024-09-13T18:28:50.776 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.774+0000 7f7e61ffb640 10 client.17487 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:28:50.780 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.778+0000 7f7e61ffb640 1 -- 172.21.15.114:0/3908507776 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ad68f9c300 con 0x55ad693c7890 2024-09-13T18:28:50.780 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.778+0000 7f7e61ffb640 10 client.17487 handle_command_reply: tid=0 2024-09-13T18:28:50.784 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.782+0000 7f7e833fd740 1 client.17487 shutdown 2024-09-13T18:28:50.786 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.784+0000 7f7e833fd740 1 -- 172.21.15.114:0/3908507776 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ad6934f6c0 msgr2=0x55ad69354ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.784+0000 7f7e833fd740 1 --2- 172.21.15.114:0/3908507776 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ad6934f6c0 0x55ad69354ff0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f7e70005dc0 tx=0x7f7e7002f830 comp rx=0 tx=0).stop 2024-09-13T18:28:50.791 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.788+0000 7f7e833fd740 1 -- 172.21.15.114:0/3908507776 shutdown_connections 2024-09-13T18:28:50.791 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.788+0000 7f7e833fd740 1 --2- 172.21.15.114:0/3908507776 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55ad693c7890 0x55ad693c9c80 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.791 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.788+0000 7f7e833fd740 1 --2- 172.21.15.114:0/3908507776 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ad6934f6c0 0x55ad69354ff0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.791 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.788+0000 7f7e833fd740 1 --2- 172.21.15.114:0/3908507776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ad69344e20 0x55ad69354ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.791 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.788+0000 7f7e833fd740 1 --2- 172.21.15.114:0/3908507776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad69344500 0x55ad693565a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.791 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.788+0000 7f7e833fd740 1 -- 172.21.15.114:0/3908507776 >> 172.21.15.114:0/3908507776 conn(0x55ad69340a90 msgr2=0x55ad69347500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:50.791 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.788+0000 7f7e61ffb640 0 client.17487 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:28:50.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.787+0000 7f4fc1353640 1 -- 172.21.15.114:0/3976383311 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4fbc107920 msgr2=0x7f4fbc107d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:50.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.787+0000 7f4fc1353640 1 --2- 172.21.15.114:0/3976383311 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4fbc107920 0x7f4fbc107d60 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4009be0 tx=0x7f4fb402a6f0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.787+0000 7f4fc1353640 1 -- 172.21.15.114:0/3976383311 shutdown_connections 2024-09-13T18:28:50.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.787+0000 7f4fc1353640 1 --2- 172.21.15.114:0/3976383311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fbc119400 0x7f4fbc11b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.787+0000 7f4fc1353640 1 --2- 172.21.15.114:0/3976383311 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4fbc107920 0x7f4fbc107d60 secure :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4009be0 tx=0x7f4fb402a6f0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.787+0000 7f4fc1353640 1 --2- 172.21.15.114:0/3976383311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fbc107540 0x7f4fbc10e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.787+0000 7f4fc1353640 1 -- 172.21.15.114:0/3976383311 >> 172.21.15.114:0/3976383311 conn(0x7f4fbc06afe0 msgr2=0x7f4fbc06b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:50.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.796+0000 7f4fc1353640 1 -- 172.21.15.114:0/3976383311 shutdown_connections 2024-09-13T18:28:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.797+0000 7f7e833fd740 1 -- 172.21.15.114:0/3908507776 shutdown_connections 2024-09-13T18:28:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.797+0000 7f7e833fd740 1 -- 172.21.15.114:0/3908507776 wait complete. 2024-09-13T18:28:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.798+0000 7f7e833fd740 20 client.17487 trim_cache size 0 max 16384 2024-09-13T18:28:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-13T18:28:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-13T18:28:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-13T18:28:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-13T18:28:50.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.802+0000 7f4fc1353640 1 -- 172.21.15.114:0/3976383311 wait complete. 2024-09-13T18:28:50.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.802+0000 7f4fc1353640 1 Processor -- start 2024-09-13T18:28:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.804+0000 7f7e805ff640 1 -- 172.21.15.114:0/4250305988 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7e640a14d0 msgr2=0x7f7e640a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.804+0000 7f7e805ff640 1 --2- 172.21.15.114:0/4250305988 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7e640a14d0 0x7f7e640a3990 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7e70047000 tx=0x7f7e7004c000 comp rx=0 tx=0).stop 2024-09-13T18:28:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.804+0000 7f7e805ff640 1 -- 172.21.15.114:0/4250305988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e78107890 msgr2=0x7f7e781a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.804+0000 7f7e805ff640 1 --2- 172.21.15.114:0/4250305988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e78107890 0x7f7e781a53c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7e6800e9f0 tx=0x7f7e6800eeb0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.807+0000 7f4fc1353640 1 -- start start 2024-09-13T18:28:50.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.807+0000 7f4fc1353640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fbc107540 0x7f4fbc1a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:50.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.807+0000 7f4fc1353640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fbc119400 0x7f4fbc1a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:50.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.807+0000 7f4fc1353640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4fbc1a9bb0 0x7f4fbc1aa060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:50.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.807+0000 7f4fc1353640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fbc1aa5a0 con 0x7f4fbc1a9bb0 2024-09-13T18:28:50.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.807+0000 7f4fc1353640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fbc1aa6e0 con 0x7f4fbc119400 2024-09-13T18:28:50.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.807+0000 7f4fc1353640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fbc1aa8a0 con 0x7f4fbc107540 2024-09-13T18:28:50.811 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.808+0000 7f7e805ff640 1 -- 172.21.15.114:0/4250305988 shutdown_connections 2024-09-13T18:28:50.811 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.808+0000 7f7e805ff640 1 --2- 172.21.15.114:0/4250305988 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7e640a14d0 0x7f7e640a3990 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.811 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.808+0000 7f7e805ff640 1 --2- 172.21.15.114:0/4250305988 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7e78119400 0x7f7e781a97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.811 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.808+0000 7f7e805ff640 1 --2- 172.21.15.114:0/4250305988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e78107890 0x7f7e781a53c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.811 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.808+0000 7f7e805ff640 1 --2- 172.21.15.114:0/4250305988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e781074b0 0x7f7e781a4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.811 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.808+0000 7f7e805ff640 1 -- 172.21.15.114:0/4250305988 >> 172.21.15.114:0/4250305988 conn(0x7f7e7806ae30 msgr2=0x7f7e78119c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:50.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.809+0000 7f4fbaffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fbc107540 0x7f4fbc1a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:50.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.809+0000 7f4fbaffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fbc107540 0x7f4fbc1a5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:55574/0 (socket says 172.21.15.114:55574) 2024-09-13T18:28:50.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.809+0000 7f4fbaffd640 1 -- 172.21.15.114:0/1314887161 learned_addr learned my addr 172.21.15.114:0/1314887161 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:50.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.810+0000 7f4fba7fc640 1 --2- 172.21.15.114:0/1314887161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fbc119400 0x7f4fbc1a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:50.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.810+0000 7f4fbaffd640 1 -- 172.21.15.114:0/1314887161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fbc119400 msgr2=0x7f4fbc1a57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:50.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.810+0000 7f4fbaffd640 1 --2- 172.21.15.114:0/1314887161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fbc119400 0x7f4fbc1a57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.810+0000 7f4fbaffd640 1 -- 172.21.15.114:0/1314887161 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4fbc1a9bb0 msgr2=0x7f4fbc1aa060 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:28:50.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.810+0000 7f4fbaffd640 1 --2- 172.21.15.114:0/1314887161 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4fbc1a9bb0 0x7f4fbc1aa060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.810+0000 7f4fbaffd640 1 -- 172.21.15.114:0/1314887161 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb4009840 con 0x7f4fbc107540 2024-09-13T18:28:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.813+0000 7f7e805ff640 1 -- 172.21.15.114:0/4250305988 shutdown_connections 2024-09-13T18:28:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:28:50.813+0000 7f7e805ff640 1 -- 172.21.15.114:0/4250305988 wait complete. 2024-09-13T18:28:50.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.813+0000 7f4fbaffd640 1 --2- 172.21.15.114:0/1314887161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fbc107540 0x7f4fbc1a5280 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4fac00c890 tx=0x7f4fac00cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:50.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.819+0000 7f4fa3fff640 1 -- 172.21.15.114:0/1314887161 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fac004220 con 0x7f4fbc107540 2024-09-13T18:28:50.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.819+0000 7f4fa3fff640 1 -- 172.21.15.114:0/1314887161 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4fac007e40 con 0x7f4fbc107540 2024-09-13T18:28:50.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.819+0000 7f4fa3fff640 1 -- 172.21.15.114:0/1314887161 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fac004a00 con 0x7f4fbc107540 2024-09-13T18:28:50.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.822+0000 7f4fc1353640 1 -- 172.21.15.114:0/1314887161 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fbc10ebe0 con 0x7f4fbc107540 2024-09-13T18:28:50.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.822+0000 7f4fc1353640 1 -- 172.21.15.114:0/1314887161 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fbc10f070 con 0x7f4fbc107540 2024-09-13T18:28:50.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.829+0000 7f4fa3fff640 1 -- 172.21.15.114:0/1314887161 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f4fac0079b0 con 0x7f4fbc107540 2024-09-13T18:28:50.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.834+0000 7f4fa3fff640 1 --2- 172.21.15.114:0/1314887161 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f4fa40a1410 0x7f4fa40a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:50.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.834+0000 7f4fa3fff640 1 -- 172.21.15.114:0/1314887161 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f4fac0cd050 con 0x7f4fbc107540 2024-09-13T18:28:50.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.836+0000 7f4fc4073740 1 Processor -- start 2024-09-13T18:28:50.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.836+0000 7f4fc4073740 1 -- start start 2024-09-13T18:28:50.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.837+0000 7f4fc4073740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dbd67ed790 0x55dbd67edb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:50.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.837+0000 7f4fc4073740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dbd67ee0b0 0x55dbd67f8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:50.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.837+0000 7f4fba7fc640 1 --2- 172.21.15.114:0/1314887161 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f4fa40a1410 0x7f4fa40a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:50.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.837+0000 7f4fbaffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dbd67ed790 0x55dbd67edb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:50.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.837+0000 7f4fbaffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dbd67ed790 0x55dbd67edb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:55590/0 (socket says 172.21.15.114:55590) 2024-09-13T18:28:50.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.837+0000 7f4fbaffd640 1 -- 172.21.15.114:0/2776555335 learned_addr learned my addr 172.21.15.114:0/2776555335 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:50.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.837+0000 7f4fba7fc640 1 --2- 172.21.15.114:0/2776555335 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dbd67ee0b0 0x55dbd67f8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:50.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.838+0000 7f4fc4073740 1 --2- 172.21.15.114:0/2776555335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbd67f8950 0x55dbd67fad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:50.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.838+0000 7f4fc4073740 1 -- 172.21.15.114:0/2776555335 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbd6757cc0 con 0x55dbd67ee0b0 2024-09-13T18:28:50.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.838+0000 7f4fc4073740 1 -- 172.21.15.114:0/2776555335 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbd6755450 con 0x55dbd67f8950 2024-09-13T18:28:50.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.838+0000 7f4fc4073740 1 -- 172.21.15.114:0/2776555335 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbd6750d40 con 0x55dbd67ed790 2024-09-13T18:28:50.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.838+0000 7f4fbb7fe640 1 --2- 172.21.15.114:0/2776555335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbd67f8950 0x55dbd67fad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:50.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.839+0000 7f4fbaffd640 1 -- 172.21.15.114:0/2776555335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbd67f8950 msgr2=0x55dbd67fad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:50.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.839+0000 7f4fbaffd640 1 --2- 172.21.15.114:0/2776555335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbd67f8950 0x55dbd67fad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.839+0000 7f4fbaffd640 1 -- 172.21.15.114:0/2776555335 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dbd67ee0b0 msgr2=0x55dbd67f8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:50.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.839+0000 7f4fbaffd640 1 --2- 172.21.15.114:0/2776555335 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dbd67ee0b0 0x55dbd67f8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.839+0000 7f4fbaffd640 1 -- 172.21.15.114:0/2776555335 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dbd6718620 con 0x55dbd67ed790 2024-09-13T18:28:50.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.839+0000 7f4fba7fc640 1 --2- 172.21.15.114:0/2776555335 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dbd67ee0b0 0x55dbd67f8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:28:50.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.839+0000 7f4fbaffd640 1 --2- 172.21.15.114:0/2776555335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dbd67ed790 0x55dbd67edb70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4fac0968a0 tx=0x7f4fac005160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fadb549ebfb9720 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:50.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.844+0000 7f4fa17fa640 1 -- 172.21.15.114:0/2776555335 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fac099e30 con 0x55dbd67ed790 2024-09-13T18:28:50.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.844+0000 7f4fa17fa640 1 -- 172.21.15.114:0/2776555335 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4fac09dc60 con 0x55dbd67ed790 2024-09-13T18:28:50.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.844+0000 7f4fba7fc640 1 --2- 172.21.15.114:0/1314887161 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f4fa40a1410 0x7f4fa40a38d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4fb402ace0 tx=0x7f4fb4000f40 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:50.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.844+0000 7f4fa17fa640 1 -- 172.21.15.114:0/2776555335 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fac0a1e20 con 0x55dbd67ed790 2024-09-13T18:28:50.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.846+0000 7f4fc4073740 1 -- 172.21.15.114:0/2776555335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dbd67ed790 msgr2=0x55dbd67edb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:50.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.846+0000 7f4fc4073740 1 --2- 172.21.15.114:0/2776555335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dbd67ed790 0x55dbd67edb70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4fac0968a0 tx=0x7f4fac005160 comp rx=0 tx=0).stop 2024-09-13T18:28:50.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.847+0000 7f4fc4073740 1 -- 172.21.15.114:0/2776555335 shutdown_connections 2024-09-13T18:28:50.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.847+0000 7f4fc4073740 1 --2- 172.21.15.114:0/2776555335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbd67f8950 0x55dbd67fad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.848+0000 7f4fc4073740 1 --2- 172.21.15.114:0/2776555335 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dbd67ee0b0 0x55dbd67f8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.848+0000 7f4fc4073740 1 --2- 172.21.15.114:0/2776555335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dbd67ed790 0x55dbd67edb70 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.848+0000 7f4fc4073740 1 -- 172.21.15.114:0/2776555335 >> 172.21.15.114:0/2776555335 conn(0x55dbd67e9d00 msgr2=0x55dbd67ea0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:50.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.853+0000 7f4fc4073740 1 -- 172.21.15.114:0/2776555335 shutdown_connections 2024-09-13T18:28:50.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.855+0000 7f4fc4073740 1 -- 172.21.15.114:0/2776555335 wait complete. 2024-09-13T18:28:50.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.855+0000 7f4fc4073740 1 -- start start 2024-09-13T18:28:50.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.855+0000 7f4fc4073740 1 Processor -- start 2024-09-13T18:28:50.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.859+0000 7f4fc4073740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dbd67ed790 0x55dbd67f7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:50.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.860+0000 7f4fc4073740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dbd67ee0b0 0x55dbd67f52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:50.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.860+0000 7f4fc4073740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbd67f8950 0x55dbd67f57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:50.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.860+0000 7f4fc4073740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbd670f180 con 0x55dbd67ee0b0 2024-09-13T18:28:50.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.860+0000 7f4fc4073740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbd6429430 con 0x55dbd67f8950 2024-09-13T18:28:50.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.860+0000 7f4fc4073740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbd66d45e0 con 0x55dbd67ed790 2024-09-13T18:28:50.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.860+0000 7f4fc4073740 10 client.0 _ll_register_callbacks cb 0x55dbd6740aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:28:50.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.861+0000 7f4fbaffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dbd67ed790 0x55dbd67f7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:50.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.861+0000 7f4fbaffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dbd67ed790 0x55dbd67f7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:55604/0 (socket says 172.21.15.114:55604) 2024-09-13T18:28:50.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.861+0000 7f4fbaffd640 1 -- 172.21.15.114:0/918585177 learned_addr learned my addr 172.21.15.114:0/918585177 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:50.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.861+0000 7f4fba7fc640 1 --2- 172.21.15.114:0/918585177 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dbd67ee0b0 0x55dbd67f52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:50.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.862+0000 7f4fbb7fe640 1 --2- 172.21.15.114:0/918585177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbd67f8950 0x55dbd67f57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:50.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.863+0000 7f4fba7fc640 1 -- 172.21.15.114:0/918585177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dbd67ed790 msgr2=0x55dbd67f7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:50.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.863+0000 7f4fba7fc640 1 --2- 172.21.15.114:0/918585177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dbd67ed790 0x55dbd67f7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.863+0000 7f4fba7fc640 1 -- 172.21.15.114:0/918585177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbd67f8950 msgr2=0x55dbd67f57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:50.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.863+0000 7f4fba7fc640 1 --2- 172.21.15.114:0/918585177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbd67f8950 0x55dbd67f57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.863+0000 7f4fba7fc640 1 -- 172.21.15.114:0/918585177 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fbc10f070 con 0x55dbd67ee0b0 2024-09-13T18:28:50.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.869+0000 7f4fba7fc640 1 --2- 172.21.15.114:0/918585177 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dbd67ee0b0 0x55dbd67f52b0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc06c4b0 tx=0x7f4fb402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4126735f826441a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:50.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.873+0000 7f4f9eff5640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:28:50.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.873+0000 7f4f9eff5640 1 -- 172.21.15.114:0/918585177 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4004710 con 0x55dbd67ee0b0 2024-09-13T18:28:50.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.873+0000 7f4fc4073740 1 -- 172.21.15.114:0/918585177 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dbd677ab80 con 0x55dbd67ee0b0 2024-09-13T18:28:50.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.873+0000 7f4f9eff5640 1 -- 172.21.15.114:0/918585177 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f4fb4031440 con 0x55dbd67ee0b0 2024-09-13T18:28:50.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.878+0000 7f4f9eff5640 1 -- 172.21.15.114:0/918585177 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4fb4003e70 con 0x55dbd67ee0b0 2024-09-13T18:28:50.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.878+0000 7f4fc4073740 10 client.17496 fetch_fsmap learned FSMap version 34 2024-09-13T18:28:50.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.878+0000 7f4fc4073740 1 -- 172.21.15.114:0/918585177 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55dbd674f3c0 con 0x55dbd67ee0b0 2024-09-13T18:28:50.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.881+0000 7f4f9eff5640 1 -- 172.21.15.114:0/918585177 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55dbd677ab80 con 0x55dbd67ee0b0 2024-09-13T18:28:50.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.881+0000 7f4f9eff5640 1 -- 172.21.15.114:0/918585177 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4fb404b070 con 0x55dbd67ee0b0 2024-09-13T18:28:50.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.881+0000 7f4fc4073740 10 client.17496 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:28:50.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.881+0000 7f4fc4073740 10 client.17496 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-13T18:28:50.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.881+0000 7f4fc4073740 20 client.17496 populate_metadata read hostname 'smithi114' 2024-09-13T18:28:50.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.881+0000 7f4fc4073740 1 --2- 172.21.15.114:0/918585177 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55dbd6808230 0x55dbd680a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:50.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.881+0000 7f4fc4073740 4 client.17496 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:28:50.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.881+0000 7f4fc4073740 1 -- 172.21.15.114:0/918585177 --> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dbd67431f0 con 0x55dbd6808230 2024-09-13T18:28:50.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.881+0000 7f4fbaffd640 1 --2- 172.21.15.114:0/918585177 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55dbd6808230 0x55dbd680a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:50.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.884+0000 7f4fbaffd640 1 --2- 172.21.15.114:0/918585177 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55dbd6808230 0x55dbd680a650 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d544e9c6698f9221 server_cookie=de9204d68eb2dff1 in_seq=0 out_seq=0 2024-09-13T18:28:50.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.884+0000 7f4f9eff5640 10 client.17496 ms_handle_connect on v2:172.21.15.134:6836/891692210 2024-09-13T18:28:50.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.887+0000 7f4f9eff5640 1 -- 172.21.15.114:0/918585177 <== mds.1 v2:172.21.15.134:6836/891692210 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55dbd67431f0 con 0x55dbd6808230 2024-09-13T18:28:50.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.887+0000 7f4f9eff5640 10 client.17496 handle_command_reply: tid=0 2024-09-13T18:28:50.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.887+0000 7f4fc4073740 1 client.17496 shutdown 2024-09-13T18:28:50.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.892+0000 7f4fc4073740 1 -- 172.21.15.114:0/918585177 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dbd67ee0b0 msgr2=0x55dbd67f52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:50.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.892+0000 7f4fc4073740 1 --2- 172.21.15.114:0/918585177 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dbd67ee0b0 0x55dbd67f52b0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc06c4b0 tx=0x7f4fb402a6f0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.892+0000 7f4fc4073740 1 -- 172.21.15.114:0/918585177 shutdown_connections 2024-09-13T18:28:50.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.892+0000 7f4fc4073740 1 --2- 172.21.15.114:0/918585177 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55dbd6808230 0x55dbd680a650 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.892+0000 7f4fc4073740 1 --2- 172.21.15.114:0/918585177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbd67f8950 0x55dbd67f57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.892+0000 7f4fc4073740 1 --2- 172.21.15.114:0/918585177 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dbd67ee0b0 0x55dbd67f52b0 secure :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc06c4b0 tx=0x7f4fb402a6f0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.892+0000 7f4fc4073740 1 --2- 172.21.15.114:0/918585177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dbd67ed790 0x55dbd67f7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.892+0000 7f4fc4073740 1 -- 172.21.15.114:0/918585177 >> 172.21.15.114:0/918585177 conn(0x55dbd67e9d00 msgr2=0x55dbd67efa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:50.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.893+0000 7f4f9eff5640 0 client.17496 ms_handle_reset on v2:172.21.15.134:6836/891692210 2024-09-13T18:28:50.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.894+0000 7f4fc4073740 1 -- 172.21.15.114:0/918585177 shutdown_connections 2024-09-13T18:28:50.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.896+0000 7f4fc4073740 1 -- 172.21.15.114:0/918585177 wait complete. 2024-09-13T18:28:50.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.904+0000 7f4fc4073740 20 client.17496 trim_cache size 0 max 16384 2024-09-13T18:28:50.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.940+0000 7f4fc4073740 1 Processor -- start 2024-09-13T18:28:50.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.940+0000 7f4fc4073740 1 -- start start 2024-09-13T18:28:50.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.940+0000 7f4fc4073740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbd67ed790 0x55dbd67ff4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:50.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.940+0000 7f4fc4073740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dbd67f8950 0x55dbd67ffa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:50.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.940+0000 7f4fc4073740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dbd67ea7f0 0x55dbd67f3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:50.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.941+0000 7f4fba7fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dbd67f8950 0x55dbd67ffa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:50.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.941+0000 7f4fba7fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dbd67f8950 0x55dbd67ffa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:55612/0 (socket says 172.21.15.114:55612) 2024-09-13T18:28:50.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.941+0000 7f4fba7fc640 1 -- 172.21.15.114:0/856576994 learned_addr learned my addr 172.21.15.114:0/856576994 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:50.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.941+0000 7f4fc4073740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbd67431f0 con 0x55dbd67ea7f0 2024-09-13T18:28:50.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.941+0000 7f4fc4073740 1 -- 172.21.15.114:0/856576994 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbd677ab80 con 0x55dbd67ed790 2024-09-13T18:28:50.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.941+0000 7f4fc4073740 1 -- 172.21.15.114:0/856576994 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbd67eae90 con 0x55dbd67f8950 2024-09-13T18:28:50.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.941+0000 7f4fbb7fe640 1 --2- 172.21.15.114:0/856576994 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dbd67ea7f0 0x55dbd67f3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:50.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.942+0000 7f4fbb7fe640 1 -- 172.21.15.114:0/856576994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dbd67f8950 msgr2=0x55dbd67ffa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:50.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.942+0000 7f4fbb7fe640 1 --2- 172.21.15.114:0/856576994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dbd67f8950 0x55dbd67ffa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.942+0000 7f4fbb7fe640 1 -- 172.21.15.114:0/856576994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbd67ed790 msgr2=0x55dbd67ff4f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:28:50.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.942+0000 7f4fbb7fe640 1 --2- 172.21.15.114:0/856576994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbd67ed790 0x55dbd67ff4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.942+0000 7f4fbb7fe640 1 -- 172.21.15.114:0/856576994 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dbd66f1770 con 0x55dbd67ea7f0 2024-09-13T18:28:50.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.943+0000 7f4fbb7fe640 1 --2- 172.21.15.114:0/856576994 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dbd67ea7f0 0x55dbd67f3bc0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x55dbd67f0330 tx=0x7f4fb000fef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f15fd671d0745e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:50.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.944+0000 7f4fa1ffb640 1 -- 172.21.15.114:0/856576994 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0011d40 con 0x55dbd67ea7f0 2024-09-13T18:28:50.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.944+0000 7f4fa1ffb640 1 -- 172.21.15.114:0/856576994 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4fb0016070 con 0x55dbd67ea7f0 2024-09-13T18:28:50.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.945+0000 7f4fa1ffb640 1 -- 172.21.15.114:0/856576994 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb001c720 con 0x55dbd67ea7f0 2024-09-13T18:28:50.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.946+0000 7f4fc4073740 1 -- 172.21.15.114:0/856576994 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dbd67ea7f0 msgr2=0x55dbd67f3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:50.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.946+0000 7f4fc4073740 1 --2- 172.21.15.114:0/856576994 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dbd67ea7f0 0x55dbd67f3bc0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x55dbd67f0330 tx=0x7f4fb000fef0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.951+0000 7f4fc4073740 1 -- 172.21.15.114:0/856576994 shutdown_connections 2024-09-13T18:28:50.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.951+0000 7f4fc4073740 1 --2- 172.21.15.114:0/856576994 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dbd67ea7f0 0x55dbd67f3bc0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.951+0000 7f4fc4073740 1 --2- 172.21.15.114:0/856576994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dbd67f8950 0x55dbd67ffa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.951+0000 7f4fc4073740 1 --2- 172.21.15.114:0/856576994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbd67ed790 0x55dbd67ff4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.951+0000 7f4fc4073740 1 -- 172.21.15.114:0/856576994 >> 172.21.15.114:0/856576994 conn(0x55dbd67e9d00 msgr2=0x55dbd67ea0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:50.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.954+0000 7f4fc4073740 1 -- 172.21.15.114:0/856576994 shutdown_connections 2024-09-13T18:28:50.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.955+0000 7f4fc4073740 1 -- 172.21.15.114:0/856576994 wait complete. 2024-09-13T18:28:50.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.956+0000 7f4fc4073740 1 -- start start 2024-09-13T18:28:50.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.956+0000 7f4fc4073740 1 Processor -- start 2024-09-13T18:28:50.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.956+0000 7f4fc4073740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dbd67ea7f0 0x55dbd67ff710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:50.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.956+0000 7f4fc4073740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbd67ed790 0x55dbd67f1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:50.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.956+0000 7f4fc4073740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dbd67f8950 0x55dbd67f21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:50.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.957+0000 7f4fbaffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dbd67ea7f0 0x55dbd67ff710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:50.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.957+0000 7f4fc4073740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbd67f2750 con 0x55dbd67ea7f0 2024-09-13T18:28:50.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.957+0000 7f4fbaffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dbd67ea7f0 0x55dbd67ff710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47544/0 (socket says 172.21.15.114:47544) 2024-09-13T18:28:50.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.957+0000 7f4fbaffd640 1 -- 172.21.15.114:0/2201197375 learned_addr learned my addr 172.21.15.114:0/2201197375 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:50.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.957+0000 7f4fc4073740 1 -- 172.21.15.114:0/2201197375 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbd67f28c0 con 0x55dbd67ed790 2024-09-13T18:28:50.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.957+0000 7f4fc4073740 1 -- 172.21.15.114:0/2201197375 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbd67f2a80 con 0x55dbd67f8950 2024-09-13T18:28:50.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.957+0000 7f4fc4073740 10 client.0 _ll_register_callbacks cb 0x7f4fac099c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:28:50.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.957+0000 7f4fbb7fe640 1 --2- 172.21.15.114:0/2201197375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dbd67f8950 0x55dbd67f21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:50.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.957+0000 7f4fba7fc640 1 --2- 172.21.15.114:0/2201197375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbd67ed790 0x55dbd67f1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:50.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.958+0000 7f4fbaffd640 1 -- 172.21.15.114:0/2201197375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dbd67f8950 msgr2=0x55dbd67f21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:50.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.958+0000 7f4fbaffd640 1 --2- 172.21.15.114:0/2201197375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dbd67f8950 0x55dbd67f21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.958+0000 7f4fbaffd640 1 -- 172.21.15.114:0/2201197375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbd67ed790 msgr2=0x55dbd67f1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:50.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.958+0000 7f4fbaffd640 1 --2- 172.21.15.114:0/2201197375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbd67ed790 0x55dbd67f1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.958+0000 7f4fbaffd640 1 -- 172.21.15.114:0/2201197375 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fb000f740 con 0x55dbd67ea7f0 2024-09-13T18:28:50.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.958+0000 7f4fba7fc640 1 --2- 172.21.15.114:0/2201197375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbd67ed790 0x55dbd67f1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:28:50.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.962+0000 7f4fbaffd640 1 --2- 172.21.15.114:0/2201197375 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dbd67ea7f0 0x55dbd67ff710 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f4fac007fd0 tx=0x7f4fac096770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=786570deb6168134 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:50.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.962+0000 7f4f9eff5640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:28:50.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.964+0000 7f4f9eff5640 1 -- 172.21.15.114:0/2201197375 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fac0a6070 con 0x55dbd67ea7f0 2024-09-13T18:28:50.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.964+0000 7f4f9eff5640 1 -- 172.21.15.114:0/2201197375 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f4fac09ae40 con 0x55dbd67ea7f0 2024-09-13T18:28:50.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.964+0000 7f4fc4073740 1 -- 172.21.15.114:0/2201197375 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4fac09dc60 con 0x55dbd67ea7f0 2024-09-13T18:28:50.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.964+0000 7f4f9eff5640 1 -- 172.21.15.114:0/2201197375 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4fac005680 con 0x55dbd67ea7f0 2024-09-13T18:28:50.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.964+0000 7f4fc4073740 10 client.17502 fetch_fsmap learned FSMap version 34 2024-09-13T18:28:50.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.965+0000 7f4fc4073740 1 -- 172.21.15.114:0/2201197375 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55dbd6716e80 con 0x55dbd67ea7f0 2024-09-13T18:28:50.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.968+0000 7f4f9eff5640 1 -- 172.21.15.114:0/2201197375 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4fac0ca880 con 0x55dbd67ea7f0 2024-09-13T18:28:50.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.968+0000 7f4f9eff5640 1 -- 172.21.15.114:0/2201197375 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4fac016030 con 0x55dbd67ea7f0 2024-09-13T18:28:50.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.968+0000 7f4fc4073740 10 client.17502 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:28:50.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.968+0000 7f4fc4073740 10 client.17502 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-13T18:28:50.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.968+0000 7f4fc4073740 20 client.17502 populate_metadata read hostname 'smithi114' 2024-09-13T18:28:50.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.968+0000 7f4fc4073740 1 --2- 172.21.15.114:0/2201197375 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55dbd685f7d0 0x55dbd6861bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:50.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.968+0000 7f4fc4073740 4 client.17502 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:28:50.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.968+0000 7f4fc4073740 1 -- 172.21.15.114:0/2201197375 --> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f4fac09dc60 con 0x55dbd685f7d0 2024-09-13T18:28:50.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.971+0000 7f4fba7fc640 1 --2- 172.21.15.114:0/2201197375 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55dbd685f7d0 0x55dbd6861bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:50.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.976+0000 7f4fba7fc640 1 --2- 172.21.15.114:0/2201197375 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55dbd685f7d0 0x55dbd6861bf0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fd1f6a133e6079ea server_cookie=267ad37179b4de14 in_seq=0 out_seq=0 2024-09-13T18:28:50.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.979+0000 7f4f9eff5640 10 client.17502 ms_handle_connect on v2:172.21.15.134:6836/891692210 2024-09-13T18:28:50.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.979+0000 7f4f9eff5640 1 -- 172.21.15.114:0/2201197375 <== mds.1 v2:172.21.15.134:6836/891692210 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f4fac09dc60 con 0x55dbd685f7d0 2024-09-13T18:28:50.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.979+0000 7f4f9eff5640 10 client.17502 handle_command_reply: tid=0 2024-09-13T18:28:50.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.979+0000 7f4fc4073740 1 client.17502 shutdown 2024-09-13T18:28:50.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.982+0000 7f4fc4073740 1 -- 172.21.15.114:0/2201197375 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dbd67ea7f0 msgr2=0x55dbd67ff710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:50.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.982+0000 7f4fc4073740 1 --2- 172.21.15.114:0/2201197375 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dbd67ea7f0 0x55dbd67ff710 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f4fac007fd0 tx=0x7f4fac096770 comp rx=0 tx=0).stop 2024-09-13T18:28:50.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.983+0000 7f4fc4073740 1 -- 172.21.15.114:0/2201197375 shutdown_connections 2024-09-13T18:28:50.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.983+0000 7f4fc4073740 1 --2- 172.21.15.114:0/2201197375 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55dbd685f7d0 0x55dbd6861bf0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.983+0000 7f4fc4073740 1 --2- 172.21.15.114:0/2201197375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dbd67f8950 0x55dbd67f21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.983+0000 7f4fc4073740 1 --2- 172.21.15.114:0/2201197375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbd67ed790 0x55dbd67f1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.983+0000 7f4fc4073740 1 --2- 172.21.15.114:0/2201197375 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dbd67ea7f0 0x55dbd67ff710 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:50.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.983+0000 7f4fc4073740 1 -- 172.21.15.114:0/2201197375 >> 172.21.15.114:0/2201197375 conn(0x55dbd67e9d00 msgr2=0x55dbd6803fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:50.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.983+0000 7f4f9eff5640 0 client.17502 ms_handle_reset on v2:172.21.15.134:6836/891692210 2024-09-13T18:28:50.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.984+0000 7f4fc4073740 1 -- 172.21.15.114:0/2201197375 shutdown_connections 2024-09-13T18:28:50.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.987+0000 7f4fc4073740 1 -- 172.21.15.114:0/2201197375 wait complete. 2024-09-13T18:28:50.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.994+0000 7f4fc4073740 20 client.17502 trim_cache size 0 max 16384 2024-09-13T18:28:50.996 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:28:51.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.999+0000 7f4fc1353640 1 -- 172.21.15.114:0/1314887161 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f4fa40a1410 msgr2=0x7f4fa40a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:51.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.999+0000 7f4fc1353640 1 --2- 172.21.15.114:0/1314887161 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f4fa40a1410 0x7f4fa40a38d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4fb402ace0 tx=0x7f4fb4000f40 comp rx=0 tx=0).stop 2024-09-13T18:28:51.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.999+0000 7f4fc1353640 1 -- 172.21.15.114:0/1314887161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fbc107540 msgr2=0x7f4fbc1a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:51.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:50.999+0000 7f4fc1353640 1 --2- 172.21.15.114:0/1314887161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fbc107540 0x7f4fbc1a5280 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4fac00c890 tx=0x7f4fac00cd50 comp rx=0 tx=0).stop 2024-09-13T18:28:51.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.002+0000 7f4fc1353640 1 -- 172.21.15.114:0/1314887161 shutdown_connections 2024-09-13T18:28:51.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.002+0000 7f4fc1353640 1 --2- 172.21.15.114:0/1314887161 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f4fa40a1410 0x7f4fa40a38d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:51.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.002+0000 7f4fc1353640 1 --2- 172.21.15.114:0/1314887161 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4fbc1a9bb0 0x7f4fbc1aa060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:51.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.002+0000 7f4fc1353640 1 --2- 172.21.15.114:0/1314887161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fbc119400 0x7f4fbc1a57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:51.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.002+0000 7f4fc1353640 1 --2- 172.21.15.114:0/1314887161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fbc107540 0x7f4fbc1a5280 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:51.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.002+0000 7f4fc1353640 1 -- 172.21.15.114:0/1314887161 >> 172.21.15.114:0/1314887161 conn(0x7f4fbc06afe0 msgr2=0x7f4fbc119e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:51.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.008+0000 7f4fc1353640 1 -- 172.21.15.114:0/1314887161 shutdown_connections 2024-09-13T18:28:51.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.008+0000 7f4fc1353640 1 -- 172.21.15.114:0/1314887161 wait complete. 2024-09-13T18:28:51.041 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-09-13T18:28:51.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.311+0000 7fcb115b5640 1 -- 172.21.15.114:0/3841098031 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb0c1080d0 msgr2=0x7fcb0c10c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:51.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.311+0000 7fcb115b5640 1 --2- 172.21.15.114:0/3841098031 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb0c1080d0 0x7fcb0c10c5e0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fcafc0085a0 tx=0x7fcafc02c450 comp rx=0 tx=0).stop 2024-09-13T18:28:51.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.313+0000 7fcb115b5640 1 -- 172.21.15.114:0/3841098031 shutdown_connections 2024-09-13T18:28:51.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.313+0000 7fcb115b5640 1 --2- 172.21.15.114:0/3841098031 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb0c10cd40 0x7fcb0c1136d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:51.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.313+0000 7fcb115b5640 1 --2- 172.21.15.114:0/3841098031 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb0c1080d0 0x7fcb0c10c5e0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:51.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.313+0000 7fcb115b5640 1 --2- 172.21.15.114:0/3841098031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb0c107720 0x7fcb0c107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:51.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.313+0000 7fcb115b5640 1 -- 172.21.15.114:0/3841098031 >> 172.21.15.114:0/3841098031 conn(0x7fcb0c06afe0 msgr2=0x7fcb0c06b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:51.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.315+0000 7fcb115b5640 1 -- 172.21.15.114:0/3841098031 shutdown_connections 2024-09-13T18:28:51.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.315+0000 7fcb115b5640 1 -- 172.21.15.114:0/3841098031 wait complete. 2024-09-13T18:28:51.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.316+0000 7fcb115b5640 1 Processor -- start 2024-09-13T18:28:51.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.316+0000 7fcb115b5640 1 -- start start 2024-09-13T18:28:51.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.316+0000 7fcb115b5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb0c107720 0x7fcb0c1a5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:51.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.316+0000 7fcb115b5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb0c10cd40 0x7fcb0c1a5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:51.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.316+0000 7fcb115b5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb0c1a9ae0 0x7fcb0c1a9f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:51.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.316+0000 7fcb115b5640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0c1aa4d0 con 0x7fcb0c107720 2024-09-13T18:28:51.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.316+0000 7fcb115b5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0c1aa640 con 0x7fcb0c1a9ae0 2024-09-13T18:28:51.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.316+0000 7fcb115b5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0c1aa7b0 con 0x7fcb0c10cd40 2024-09-13T18:28:51.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.317+0000 7fcb0affd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb0c107720 0x7fcb0c1a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:51.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.317+0000 7fcb0affd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb0c107720 0x7fcb0c1a5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47558/0 (socket says 172.21.15.114:47558) 2024-09-13T18:28:51.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.317+0000 7fcb0affd640 1 -- 172.21.15.114:0/2687833817 learned_addr learned my addr 172.21.15.114:0/2687833817 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:51.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.317+0000 7fcb0b7fe640 1 --2- 172.21.15.114:0/2687833817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb0c1a9ae0 0x7fcb0c1a9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:51.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.317+0000 7fcb0affd640 1 -- 172.21.15.114:0/2687833817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb0c10cd40 msgr2=0x7fcb0c1a5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:51.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.317+0000 7fcb0affd640 1 --2- 172.21.15.114:0/2687833817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb0c10cd40 0x7fcb0c1a5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:51.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.317+0000 7fcb0affd640 1 -- 172.21.15.114:0/2687833817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb0c1a9ae0 msgr2=0x7fcb0c1a9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:51.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.317+0000 7fcb0affd640 1 --2- 172.21.15.114:0/2687833817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb0c1a9ae0 0x7fcb0c1a9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:51.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.317+0000 7fcb0affd640 1 -- 172.21.15.114:0/2687833817 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcafc0081d0 con 0x7fcb0c107720 2024-09-13T18:28:51.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.318+0000 7fcb0affd640 1 --2- 172.21.15.114:0/2687833817 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb0c107720 0x7fcb0c1a5200 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fcb0000d950 tx=0x7fcb0000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:51.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.319+0000 7fcaf3fff640 1 -- 172.21.15.114:0/2687833817 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb00009c30 con 0x7fcb0c107720 2024-09-13T18:28:51.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.319+0000 7fcb115b5640 1 -- 172.21.15.114:0/2687833817 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb0c1b1d90 con 0x7fcb0c107720 2024-09-13T18:28:51.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.319+0000 7fcb115b5640 1 -- 172.21.15.114:0/2687833817 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb0c1b2190 con 0x7fcb0c107720 2024-09-13T18:28:51.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.319+0000 7fcaf3fff640 1 -- 172.21.15.114:0/2687833817 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcb00009d90 con 0x7fcb0c107720 2024-09-13T18:28:51.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.319+0000 7fcaf3fff640 1 -- 172.21.15.114:0/2687833817 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0000f620 con 0x7fcb0c107720 2024-09-13T18:28:51.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.322+0000 7fcb142d9740 1 Processor -- start 2024-09-13T18:28:51.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.323+0000 7fcb142d9740 1 -- start start 2024-09-13T18:28:51.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.323+0000 7fcb142d9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556666fe1790 0x556666fe1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:51.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.323+0000 7fcb142d9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556666fe20b0 0x556666fec410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:51.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.323+0000 7fcb142d9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556666fec950 0x556666feed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:51.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.323+0000 7fcb142d9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x556666f4bcc0 con 0x556666fe1790 2024-09-13T18:28:51.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.323+0000 7fcb142d9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556666f49450 con 0x556666fec950 2024-09-13T18:28:51.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.323+0000 7fcb142d9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556666f44d40 con 0x556666fe20b0 2024-09-13T18:28:51.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.324+0000 7fcb0b7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556666fe1790 0x556666fe1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:51.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.324+0000 7fcb0b7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556666fe1790 0x556666fe1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47566/0 (socket says 172.21.15.114:47566) 2024-09-13T18:28:51.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.324+0000 7fcb0b7fe640 1 -- 172.21.15.114:0/1181820890 learned_addr learned my addr 172.21.15.114:0/1181820890 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:51.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.324+0000 7fcb0b7fe640 1 -- 172.21.15.114:0/1181820890 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556666fe20b0 msgr2=0x556666fec410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:51.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.324+0000 7fcb0affd640 1 --2- 172.21.15.114:0/1181820890 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556666fe20b0 0x556666fec410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:51.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.324+0000 7fcb0b7fe640 1 --2- 172.21.15.114:0/1181820890 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556666fe20b0 0x556666fec410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:51.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.324+0000 7fcb0b7fe640 1 -- 172.21.15.114:0/1181820890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556666fec950 msgr2=0x556666feed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:51.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.324+0000 7fcb0b7fe640 1 --2- 172.21.15.114:0/1181820890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556666fec950 0x556666feed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:51.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.324+0000 7fcb0b7fe640 1 -- 172.21.15.114:0/1181820890 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556666f0c620 con 0x556666fe1790 2024-09-13T18:28:51.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.325+0000 7fcb0affd640 1 --2- 172.21.15.114:0/1181820890 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556666fe20b0 0x556666fec410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:28:51.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.325+0000 7fcaf3fff640 1 -- 172.21.15.114:0/2687833817 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fcb0000f780 con 0x7fcb0c107720 2024-09-13T18:28:51.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.326+0000 7fcb0b7fe640 1 --2- 172.21.15.114:0/1181820890 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556666fe1790 0x556666fe1b70 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fcb0400a4d0 tx=0x7fcb04009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f7903d045ef902d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:51.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.326+0000 7fcaf3fff640 1 --2- 172.21.15.114:0/2687833817 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fcaf40a1410 0x7fcaf40a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:51.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.326+0000 7fcaf3fff640 1 -- 172.21.15.114:0/2687833817 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fcb000cc970 con 0x7fcb0c107720 2024-09-13T18:28:51.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.327+0000 7fcb0affd640 1 --2- 172.21.15.114:0/2687833817 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fcaf40a1410 0x7fcaf40a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:51.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.328+0000 7fcb0affd640 1 --2- 172.21.15.114:0/2687833817 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fcaf40a1410 0x7fcaf40a38d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x556666fdeb60 tx=0x7fcb00000bc0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:51.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.329+0000 7fcaf17fa640 1 -- 172.21.15.114:0/1181820890 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb04010040 con 0x556666fe1790 2024-09-13T18:28:51.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.329+0000 7fcaf17fa640 1 -- 172.21.15.114:0/1181820890 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcb04014ca0 con 0x556666fe1790 2024-09-13T18:28:51.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.330+0000 7fcb142d9740 1 -- 172.21.15.114:0/1181820890 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556666fe1790 msgr2=0x556666fe1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:51.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.330+0000 7fcb142d9740 1 --2- 172.21.15.114:0/1181820890 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556666fe1790 0x556666fe1b70 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fcb0400a4d0 tx=0x7fcb04009650 comp rx=0 tx=0).stop 2024-09-13T18:28:51.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.330+0000 7fcb142d9740 1 -- 172.21.15.114:0/1181820890 shutdown_connections 2024-09-13T18:28:51.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.330+0000 7fcb142d9740 1 --2- 172.21.15.114:0/1181820890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556666fec950 0x556666feed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:51.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.330+0000 7fcb142d9740 1 --2- 172.21.15.114:0/1181820890 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556666fe20b0 0x556666fec410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:51.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.330+0000 7fcb142d9740 1 --2- 172.21.15.114:0/1181820890 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556666fe1790 0x556666fe1b70 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:51.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.330+0000 7fcb142d9740 1 -- 172.21.15.114:0/1181820890 >> 172.21.15.114:0/1181820890 conn(0x556666fddd00 msgr2=0x556666fde0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:51.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.334+0000 7fcb142d9740 1 -- 172.21.15.114:0/1181820890 shutdown_connections 2024-09-13T18:28:51.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.337+0000 7fcb142d9740 1 -- 172.21.15.114:0/1181820890 wait complete. 2024-09-13T18:28:51.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.337+0000 7fcb142d9740 1 -- start start 2024-09-13T18:28:51.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.337+0000 7fcb142d9740 1 Processor -- start 2024-09-13T18:28:51.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.339+0000 7fcb142d9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556666fe20b0 0x556666fe9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:51.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.339+0000 7fcb142d9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556666fec950 0x556666fe9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:51.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.339+0000 7fcb142d9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556666feb8f0 0x556666febd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:51.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.339+0000 7fcb142d9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x556666f03180 con 0x556666fe20b0 2024-09-13T18:28:51.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.339+0000 7fcb142d9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556666c1d430 con 0x556666feb8f0 2024-09-13T18:28:51.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.339+0000 7fcb142d9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556666ec85e0 con 0x556666fec950 2024-09-13T18:28:51.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.339+0000 7fcb142d9740 10 client.0 _ll_register_callbacks cb 0x556666f34aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:28:51.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.341+0000 7fcb0a7fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556666fec950 0x556666fe9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:51.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.341+0000 7fcb0a7fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556666fec950 0x556666fe9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:55668/0 (socket says 172.21.15.114:55668) 2024-09-13T18:28:51.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.341+0000 7fcb0a7fc640 1 -- 172.21.15.114:0/1164798867 learned_addr learned my addr 172.21.15.114:0/1164798867 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:51.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.341+0000 7fcb0a7fc640 1 -- 172.21.15.114:0/1164798867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556666feb8f0 msgr2=0x556666febd70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:28:51.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.341+0000 7fcb0a7fc640 1 --2- 172.21.15.114:0/1164798867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556666feb8f0 0x556666febd70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:51.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.341+0000 7fcb0a7fc640 1 -- 172.21.15.114:0/1164798867 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556666fe20b0 msgr2=0x556666fe9200 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:28:51.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.341+0000 7fcb0a7fc640 1 --2- 172.21.15.114:0/1164798867 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556666fe20b0 0x556666fe9200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:51.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.341+0000 7fcb0a7fc640 1 -- 172.21.15.114:0/1164798867 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb04009d00 con 0x556666fec950 2024-09-13T18:28:51.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.343+0000 7fcb0a7fc640 1 --2- 172.21.15.114:0/1164798867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556666fec950 0x556666fe9740 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fcafc008570 tx=0x7fcafc009fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b98e9ceb9d9e764 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:51.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.344+0000 7fcaeeff5640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:28:51.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.347+0000 7fcaeeff5640 1 -- 172.21.15.114:0/1164798867 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc00d080 con 0x556666fec950 2024-09-13T18:28:51.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.347+0000 7fcaeeff5640 1 -- 172.21.15.114:0/1164798867 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fcafc004250 con 0x556666fec950 2024-09-13T18:28:51.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.347+0000 7fcb142d9740 1 -- 172.21.15.114:0/1164798867 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556666f371f0 con 0x556666fec950 2024-09-13T18:28:51.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.348+0000 7fcaeeff5640 1 -- 172.21.15.114:0/1164798867 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcafc004430 con 0x556666fec950 2024-09-13T18:28:51.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.349+0000 7fcb142d9740 10 client.26929 fetch_fsmap learned FSMap version 34 2024-09-13T18:28:51.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.349+0000 7fcb142d9740 1 -- 172.21.15.114:0/1164798867 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x556666f433c0 con 0x556666fec950 2024-09-13T18:28:51.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.354+0000 7fcaeeff5640 1 -- 172.21.15.114:0/1164798867 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x556666f371f0 con 0x556666fec950 2024-09-13T18:28:51.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.354+0000 7fcaeeff5640 1 -- 172.21.15.114:0/1164798867 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcafc03b070 con 0x556666fec950 2024-09-13T18:28:51.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.354+0000 7fcb142d9740 10 client.26929 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:28:51.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.354+0000 7fcb142d9740 10 client.26929 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:28:51.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.354+0000 7fcb142d9740 20 client.26929 populate_metadata read hostname 'smithi114' 2024-09-13T18:28:51.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.354+0000 7fcb142d9740 1 --2- 172.21.15.114:0/1164798867 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x556666ffc700 0x556666ffeb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:51.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.354+0000 7fcb142d9740 4 client.26929 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:28:51.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.354+0000 7fcb142d9740 1 -- 172.21.15.114:0/1164798867 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556666c384e0 con 0x556666ffc700 2024-09-13T18:28:51.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.364+0000 7fcb0b7fe640 1 --2- 172.21.15.114:0/1164798867 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x556666ffc700 0x556666ffeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:51.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.372+0000 7fcb0b7fe640 1 --2- 172.21.15.114:0/1164798867 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x556666ffc700 0x556666ffeb20 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8019243964cff26a server_cookie=f6efc35e124b629a in_seq=0 out_seq=0 2024-09-13T18:28:51.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.373+0000 7fcaeeff5640 10 client.26929 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:28:51.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.374+0000 7fcaeeff5640 1 -- 172.21.15.114:0/1164798867 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x556666c384e0 con 0x556666ffc700 2024-09-13T18:28:51.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.375+0000 7fcaeeff5640 10 client.26929 handle_command_reply: tid=0 2024-09-13T18:28:51.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.375+0000 7fcb142d9740 1 client.26929 shutdown 2024-09-13T18:28:51.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.381+0000 7fcb142d9740 1 -- 172.21.15.114:0/1164798867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556666fec950 msgr2=0x556666fe9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:51.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.381+0000 7fcb142d9740 1 --2- 172.21.15.114:0/1164798867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556666fec950 0x556666fe9740 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fcafc008570 tx=0x7fcafc009fb0 comp rx=0 tx=0).stop 2024-09-13T18:28:51.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.383+0000 7fcb142d9740 1 -- 172.21.15.114:0/1164798867 shutdown_connections 2024-09-13T18:28:51.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.383+0000 7fcb142d9740 1 --2- 172.21.15.114:0/1164798867 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x556666ffc700 0x556666ffeb20 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:51.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.383+0000 7fcb142d9740 1 --2- 172.21.15.114:0/1164798867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556666feb8f0 0x556666febd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:51.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.383+0000 7fcb142d9740 1 --2- 172.21.15.114:0/1164798867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556666fec950 0x556666fe9740 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:51.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.383+0000 7fcb142d9740 1 --2- 172.21.15.114:0/1164798867 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556666fe20b0 0x556666fe9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:51.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.383+0000 7fcb142d9740 1 -- 172.21.15.114:0/1164798867 >> 172.21.15.114:0/1164798867 conn(0x556666fddd00 msgr2=0x556666fe3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:51.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.383+0000 7fcaeeff5640 0 client.26929 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:28:51.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.385+0000 7fcb142d9740 1 -- 172.21.15.114:0/1164798867 shutdown_connections 2024-09-13T18:28:51.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.389+0000 7fcb142d9740 1 -- 172.21.15.114:0/1164798867 wait complete. 2024-09-13T18:28:51.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.390+0000 7fcb142d9740 20 client.26929 trim_cache size 0 max 16384 2024-09-13T18:28:51.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.424+0000 7fcb142d9740 1 Processor -- start 2024-09-13T18:28:51.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.424+0000 7fcb142d9740 1 -- start start 2024-09-13T18:28:51.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.424+0000 7fcb142d9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556666fe20b0 0x556666ff3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:51.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.424+0000 7fcb142d9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556666feb8f0 0x556666ff3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:51.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.424+0000 7fcb142d9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556666fec950 0x556666fe4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:51.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.424+0000 7fcb142d9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x556666ff9390 con 0x556666feb8f0 2024-09-13T18:28:51.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.424+0000 7fcb142d9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556666fea180 con 0x556666fec950 2024-09-13T18:28:51.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.424+0000 7fcb142d9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556666fd9790 con 0x556666fe20b0 2024-09-13T18:28:51.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.425+0000 7fcb0b7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556666fe20b0 0x556666ff3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:51.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.426+0000 7fcb0b7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556666fe20b0 0x556666ff3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:55676/0 (socket says 172.21.15.114:55676) 2024-09-13T18:28:51.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.426+0000 7fcb0b7fe640 1 -- 172.21.15.114:0/3464900767 learned_addr learned my addr 172.21.15.114:0/3464900767 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:51.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.427+0000 7fcb0b7fe640 1 --2- 172.21.15.114:0/3464900767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556666fec950 0x556666fe4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:51.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.428+0000 7fcb0a7fc640 1 --2- 172.21.15.114:0/3464900767 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556666feb8f0 0x556666ff3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:51.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.429+0000 7fcb0b7fe640 1 -- 172.21.15.114:0/3464900767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556666fec950 msgr2=0x556666fe4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:51.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.429+0000 7fcb0b7fe640 1 --2- 172.21.15.114:0/3464900767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556666fec950 0x556666fe4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:51.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.429+0000 7fcb0b7fe640 1 -- 172.21.15.114:0/3464900767 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556666feb8f0 msgr2=0x556666ff3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:51.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.429+0000 7fcb0b7fe640 1 --2- 172.21.15.114:0/3464900767 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556666feb8f0 0x556666ff3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:51.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.429+0000 7fcb0b7fe640 1 -- 172.21.15.114:0/3464900767 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556666ee5770 con 0x556666fe20b0 2024-09-13T18:28:51.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.433+0000 7fcb0b7fe640 1 --2- 172.21.15.114:0/3464900767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556666fe20b0 0x556666ff3750 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x556666feb8c0 tx=0x7fcb0400d040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73ad5ab2a5874dff server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:51.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.436+0000 7fcaf1ffb640 1 -- 172.21.15.114:0/3464900767 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb04003c10 con 0x556666fe20b0 2024-09-13T18:28:51.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.436+0000 7fcaf1ffb640 1 -- 172.21.15.114:0/3464900767 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcb04023070 con 0x556666fe20b0 2024-09-13T18:28:51.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.438+0000 7fcaf1ffb640 1 -- 172.21.15.114:0/3464900767 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb04014610 con 0x556666fe20b0 2024-09-13T18:28:51.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.438+0000 7fcb142d9740 1 -- 172.21.15.114:0/3464900767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556666fe20b0 msgr2=0x556666ff3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:51.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.438+0000 7fcb142d9740 1 --2- 172.21.15.114:0/3464900767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556666fe20b0 0x556666ff3750 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x556666feb8c0 tx=0x7fcb0400d040 comp rx=0 tx=0).stop 2024-09-13T18:28:51.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.439+0000 7fcb142d9740 1 -- 172.21.15.114:0/3464900767 shutdown_connections 2024-09-13T18:28:51.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.439+0000 7fcb142d9740 1 --2- 172.21.15.114:0/3464900767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556666fec950 0x556666fe4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:51.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.439+0000 7fcb142d9740 1 --2- 172.21.15.114:0/3464900767 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556666feb8f0 0x556666ff3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:51.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.439+0000 7fcb142d9740 1 --2- 172.21.15.114:0/3464900767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556666fe20b0 0x556666ff3750 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:51.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.439+0000 7fcb142d9740 1 -- 172.21.15.114:0/3464900767 >> 172.21.15.114:0/3464900767 conn(0x556666fddd00 msgr2=0x556666fe7900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:51.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.441+0000 7fcb142d9740 1 -- 172.21.15.114:0/3464900767 shutdown_connections 2024-09-13T18:28:51.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.444+0000 7fcb142d9740 1 -- 172.21.15.114:0/3464900767 wait complete. 2024-09-13T18:28:51.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.444+0000 7fcb142d9740 1 -- start start 2024-09-13T18:28:51.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.444+0000 7fcb142d9740 1 Processor -- start 2024-09-13T18:28:51.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.445+0000 7fcb142d9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556666fe20b0 0x556666ff2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:51.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.445+0000 7fcb142d9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556666feb8f0 0x556666ff2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:51.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.445+0000 7fcb142d9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556666fec950 0x556666ff2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:51.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.445+0000 7fcb142d9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x556667052bd0 con 0x556666fec950 2024-09-13T18:28:51.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.445+0000 7fcb142d9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556667052d10 con 0x556666fe20b0 2024-09-13T18:28:51.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.445+0000 7fcb142d9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556667052e80 con 0x556666feb8f0 2024-09-13T18:28:51.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.445+0000 7fcb142d9740 10 client.0 _ll_register_callbacks cb 0x7fcb04009af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:28:51.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.445+0000 7fcb0b7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556666fe20b0 0x556666ff2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:51.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.445+0000 7fcb0b7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556666fe20b0 0x556666ff2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:56122/0 (socket says 172.21.15.114:56122) 2024-09-13T18:28:51.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.445+0000 7fcb0b7fe640 1 -- 172.21.15.114:0/190638394 learned_addr learned my addr 172.21.15.114:0/190638394 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:51.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.446+0000 7fcb0a7fc640 1 --2- 172.21.15.114:0/190638394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556666feb8f0 0x556666ff2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:51.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.446+0000 7fcb0b7fe640 1 --2- 172.21.15.114:0/190638394 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556666fec950 0x556666ff2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:51.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.446+0000 7fcb0b7fe640 1 -- 172.21.15.114:0/190638394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556666feb8f0 msgr2=0x556666ff2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:51.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.446+0000 7fcb0b7fe640 1 --2- 172.21.15.114:0/190638394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556666feb8f0 0x556666ff2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:51.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.446+0000 7fcb0b7fe640 1 -- 172.21.15.114:0/190638394 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556666fec950 msgr2=0x556666ff2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:51.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.446+0000 7fcb0b7fe640 1 --2- 172.21.15.114:0/190638394 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556666fec950 0x556666ff2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:51.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.446+0000 7fcb0b7fe640 1 -- 172.21.15.114:0/190638394 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556666f0c620 con 0x556666fe20b0 2024-09-13T18:28:51.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.447+0000 7fcb0b7fe640 1 --2- 172.21.15.114:0/190638394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556666fe20b0 0x556666ff2300 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcb04027040 tx=0x7fcb04016040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9069371d15a0250e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:51.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.447+0000 7fcb0a7fc640 1 --2- 172.21.15.114:0/190638394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556666feb8f0 0x556666ff2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:28:51.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.447+0000 7fcaeeff5640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:28:51.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.447+0000 7fcaeeff5640 1 -- 172.21.15.114:0/190638394 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb040258b0 con 0x556666fe20b0 2024-09-13T18:28:51.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.447+0000 7fcaeeff5640 1 -- 172.21.15.114:0/190638394 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fcb04003a10 con 0x556666fe20b0 2024-09-13T18:28:51.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.447+0000 7fcb142d9740 1 -- 172.21.15.114:0/190638394 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556666f6eb80 con 0x556666fe20b0 2024-09-13T18:28:51.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.447+0000 7fcaeeff5640 1 -- 172.21.15.114:0/190638394 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb04003bf0 con 0x556666fe20b0 2024-09-13T18:28:51.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.452+0000 7fcb142d9740 10 client.26945 fetch_fsmap learned FSMap version 34 2024-09-13T18:28:51.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.452+0000 7fcb142d9740 1 -- 172.21.15.114:0/190638394 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x556666f0ae80 con 0x556666fe20b0 2024-09-13T18:28:51.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.452+0000 7fcaeeff5640 1 -- 172.21.15.114:0/190638394 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x556666f6eb80 con 0x556666fe20b0 2024-09-13T18:28:51.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.452+0000 7fcaeeff5640 1 -- 172.21.15.114:0/190638394 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcb04008030 con 0x556666fe20b0 2024-09-13T18:28:51.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.452+0000 7fcb142d9740 10 client.26945 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:28:51.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.452+0000 7fcb142d9740 10 client.26945 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:28:51.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.452+0000 7fcb142d9740 20 client.26945 populate_metadata read hostname 'smithi114' 2024-09-13T18:28:51.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.452+0000 7fcb142d9740 1 --2- 172.21.15.114:0/190638394 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x556667054150 0x556667056540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:51.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.452+0000 7fcb142d9740 4 client.26945 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:28:51.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.452+0000 7fcb142d9740 1 -- 172.21.15.114:0/190638394 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556666c497e0 con 0x556667054150 2024-09-13T18:28:51.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.453+0000 7fcb0a7fc640 1 --2- 172.21.15.114:0/190638394 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x556667054150 0x556667056540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:51.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.460+0000 7fcaeeff5640 10 client.26945 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:28:51.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.460+0000 7fcb0a7fc640 1 --2- 172.21.15.114:0/190638394 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x556667054150 0x556667056540 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb53bdcde0590b12 server_cookie=a208229e00138f11 in_seq=0 out_seq=0 2024-09-13T18:28:51.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.461+0000 7fcaeeff5640 1 -- 172.21.15.114:0/190638394 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556666c497e0 con 0x556667054150 2024-09-13T18:28:51.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.461+0000 7fcaeeff5640 10 client.26945 handle_command_reply: tid=0 2024-09-13T18:28:51.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.461+0000 7fcb142d9740 1 client.26945 shutdown 2024-09-13T18:28:51.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.468+0000 7fcb142d9740 1 -- 172.21.15.114:0/190638394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556666fe20b0 msgr2=0x556666ff2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:51.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.468+0000 7fcb142d9740 1 --2- 172.21.15.114:0/190638394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556666fe20b0 0x556666ff2300 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcb04027040 tx=0x7fcb04016040 comp rx=0 tx=0).stop 2024-09-13T18:28:51.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.468+0000 7fcb142d9740 1 -- 172.21.15.114:0/190638394 shutdown_connections 2024-09-13T18:28:51.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.468+0000 7fcb142d9740 1 --2- 172.21.15.114:0/190638394 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x556667054150 0x556667056540 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:51.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.468+0000 7fcb142d9740 1 --2- 172.21.15.114:0/190638394 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556666fec950 0x556666ff2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:51.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.468+0000 7fcb142d9740 1 --2- 172.21.15.114:0/190638394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556666feb8f0 0x556666ff2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:51.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.468+0000 7fcb142d9740 1 --2- 172.21.15.114:0/190638394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556666fe20b0 0x556666ff2300 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:51.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.468+0000 7fcb142d9740 1 -- 172.21.15.114:0/190638394 >> 172.21.15.114:0/190638394 conn(0x556666fddd00 msgr2=0x556666ff8200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:51.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.468+0000 7fcaeeff5640 0 client.26945 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:28:51.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.468+0000 7fcb142d9740 1 -- 172.21.15.114:0/190638394 shutdown_connections 2024-09-13T18:28:51.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.468+0000 7fcb142d9740 1 -- 172.21.15.114:0/190638394 wait complete. 2024-09-13T18:28:51.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.475+0000 7fcb142d9740 20 client.26945 trim_cache size 0 max 16384 2024-09-13T18:28:51.477 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:28:51.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.475+0000 7fcb115b5640 1 -- 172.21.15.114:0/2687833817 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fcaf40a1410 msgr2=0x7fcaf40a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:51.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.475+0000 7fcb115b5640 1 --2- 172.21.15.114:0/2687833817 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fcaf40a1410 0x7fcaf40a38d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x556666fdeb60 tx=0x7fcb00000bc0 comp rx=0 tx=0).stop 2024-09-13T18:28:51.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.475+0000 7fcb115b5640 1 -- 172.21.15.114:0/2687833817 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb0c107720 msgr2=0x7fcb0c1a5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:51.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.475+0000 7fcb115b5640 1 --2- 172.21.15.114:0/2687833817 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb0c107720 0x7fcb0c1a5200 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fcb0000d950 tx=0x7fcb0000de10 comp rx=0 tx=0).stop 2024-09-13T18:28:51.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.476+0000 7fcb115b5640 1 -- 172.21.15.114:0/2687833817 shutdown_connections 2024-09-13T18:28:51.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.476+0000 7fcb115b5640 1 --2- 172.21.15.114:0/2687833817 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fcaf40a1410 0x7fcaf40a38d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:51.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.476+0000 7fcb115b5640 1 --2- 172.21.15.114:0/2687833817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb0c1a9ae0 0x7fcb0c1a9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:51.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.476+0000 7fcb115b5640 1 --2- 172.21.15.114:0/2687833817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb0c10cd40 0x7fcb0c1a5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:51.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.476+0000 7fcb115b5640 1 --2- 172.21.15.114:0/2687833817 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb0c107720 0x7fcb0c1a5200 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:51.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.476+0000 7fcb115b5640 1 -- 172.21.15.114:0/2687833817 >> 172.21.15.114:0/2687833817 conn(0x7fcb0c06afe0 msgr2=0x7fcb0c10a540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:51.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.477+0000 7fcb115b5640 1 -- 172.21.15.114:0/2687833817 shutdown_connections 2024-09-13T18:28:51.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:51.478+0000 7fcb115b5640 1 -- 172.21.15.114:0/2687833817 wait complete. 2024-09-13T18:28:51.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:51 smithi114 ceph-mon[30897]: pgmap v194: 129 pgs: 129 active+clean; 20 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 48 MiB/s wr, 998 op/s 2024-09-13T18:28:52.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:51 smithi134 ceph-mon[34283]: pgmap v194: 129 pgs: 129 active+clean; 20 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 48 MiB/s wr, 998 op/s 2024-09-13T18:28:52.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:51 smithi204 ceph-mon[34777]: pgmap v194: 129 pgs: 129 active+clean; 20 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 48 MiB/s wr, 998 op/s 2024-09-13T18:28:52.510 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-13T18:28:52.510 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-09-13T18:28:52.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.778+0000 7f56ebfff640 1 -- 172.21.15.114:0/525842215 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56ec119400 msgr2=0x7f56ec11b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:52.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.778+0000 7f56ebfff640 1 --2- 172.21.15.114:0/525842215 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56ec119400 0x7f56ec11b7f0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f56e4009ca0 tx=0x7f56e402a8f0 comp rx=0 tx=0).stop 2024-09-13T18:28:52.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.778+0000 7f56ebfff640 1 -- 172.21.15.114:0/525842215 shutdown_connections 2024-09-13T18:28:52.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.778+0000 7f56ebfff640 1 --2- 172.21.15.114:0/525842215 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56ec119400 0x7f56ec11b7f0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:52.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.778+0000 7f56ebfff640 1 --2- 172.21.15.114:0/525842215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56ec107890 0x7f56ec107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:52.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.778+0000 7f56ebfff640 1 --2- 172.21.15.114:0/525842215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56ec1074b0 0x7f56ec10e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:52.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.778+0000 7f56ebfff640 1 -- 172.21.15.114:0/525842215 >> 172.21.15.114:0/525842215 conn(0x7f56ec06af40 msgr2=0x7f56ec06b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:52.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.784+0000 7f56ebfff640 1 -- 172.21.15.114:0/525842215 shutdown_connections 2024-09-13T18:28:52.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.784+0000 7f56ebfff640 1 -- 172.21.15.114:0/525842215 wait complete. 2024-09-13T18:28:52.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.785+0000 7f56ebfff640 1 Processor -- start 2024-09-13T18:28:52.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.786+0000 7f56ebfff640 1 -- start start 2024-09-13T18:28:52.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.787+0000 7f56ebfff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56ec1074b0 0x7f56ec1a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:52.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.787+0000 7f56ebfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56ec107890 0x7f56ec1a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:52.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.787+0000 7f56ebfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56ec119400 0x7f56ec1a99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:52.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.787+0000 7f56ebfff640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec1a9fb0 con 0x7f56ec1074b0 2024-09-13T18:28:52.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.787+0000 7f56ebfff640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec1aa120 con 0x7f56ec107890 2024-09-13T18:28:52.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.787+0000 7f56ebfff640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec1aa2e0 con 0x7f56ec119400 2024-09-13T18:28:52.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.787+0000 7f56eb7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56ec119400 0x7f56ec1a99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:52.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.787+0000 7f56eb7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56ec119400 0x7f56ec1a99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:55702/0 (socket says 172.21.15.114:55702) 2024-09-13T18:28:52.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.787+0000 7f56eb7fe640 1 -- 172.21.15.114:0/3440149025 learned_addr learned my addr 172.21.15.114:0/3440149025 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:52.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.788+0000 7f56eaffd640 1 --2- 172.21.15.114:0/3440149025 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56ec1074b0 0x7f56ec1a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:52.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.789+0000 7f56eaffd640 1 -- 172.21.15.114:0/3440149025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56ec119400 msgr2=0x7f56ec1a99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:52.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.789+0000 7f56eaffd640 1 --2- 172.21.15.114:0/3440149025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56ec119400 0x7f56ec1a99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:52.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.789+0000 7f56eaffd640 1 -- 172.21.15.114:0/3440149025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56ec107890 msgr2=0x7f56ec1a55c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:28:52.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.789+0000 7f56eaffd640 1 --2- 172.21.15.114:0/3440149025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56ec107890 0x7f56ec1a55c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:52.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.789+0000 7f56eaffd640 1 -- 172.21.15.114:0/3440149025 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e4009900 con 0x7f56ec1074b0 2024-09-13T18:28:52.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.789+0000 7f56eaffd640 1 --2- 172.21.15.114:0/3440149025 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56ec1074b0 0x7f56ec1a5080 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f56dc00d6d0 tx=0x7f56dc00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:52.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.790+0000 7f56d3fff640 1 -- 172.21.15.114:0/3440149025 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56dc005220 con 0x7f56ec1074b0 2024-09-13T18:28:52.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.790+0000 7f56d3fff640 1 -- 172.21.15.114:0/3440149025 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f56dc009cb0 con 0x7f56ec1074b0 2024-09-13T18:28:52.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.791+0000 7f56d3fff640 1 -- 172.21.15.114:0/3440149025 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56dc004c60 con 0x7f56ec1074b0 2024-09-13T18:28:52.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.791+0000 7f56ebfff640 1 -- 172.21.15.114:0/3440149025 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56ec1aa5c0 con 0x7f56ec1074b0 2024-09-13T18:28:52.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.791+0000 7f56ebfff640 1 -- 172.21.15.114:0/3440149025 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56ec1b1e40 con 0x7f56ec1074b0 2024-09-13T18:28:52.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.795+0000 7f56f4227740 1 Processor -- start 2024-09-13T18:28:52.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.796+0000 7f56f4227740 1 -- start start 2024-09-13T18:28:52.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.796+0000 7f56f4227740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb8434e9a0 0x55cb8434ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:52.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.796+0000 7f56f4227740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb8434f2c0 0x55cb84359620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:52.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.796+0000 7f56f4227740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cb84359b60 0x55cb8435bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:52.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.796+0000 7f56f4227740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb842b9c60 con 0x55cb84359b60 2024-09-13T18:28:52.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.796+0000 7f56f4227740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb842b73f0 con 0x55cb8434f2c0 2024-09-13T18:28:52.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.796+0000 7f56f4227740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb842b2ce0 con 0x55cb8434e9a0 2024-09-13T18:28:52.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.797+0000 7f56eb7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb8434e9a0 0x55cb8434ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:52.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.797+0000 7f56eb7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb8434e9a0 0x55cb8434ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:55714/0 (socket says 172.21.15.114:55714) 2024-09-13T18:28:52.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.797+0000 7f56eb7fe640 1 -- 172.21.15.114:0/272213879 learned_addr learned my addr 172.21.15.114:0/272213879 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:52.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.797+0000 7f56eb7fe640 1 -- 172.21.15.114:0/272213879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb8434f2c0 msgr2=0x55cb84359620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:28:52.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.797+0000 7f56eb7fe640 1 --2- 172.21.15.114:0/272213879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb8434f2c0 0x55cb84359620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:52.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.797+0000 7f56eb7fe640 1 -- 172.21.15.114:0/272213879 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cb84359b60 msgr2=0x55cb8435bf50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:28:52.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.797+0000 7f56eb7fe640 1 --2- 172.21.15.114:0/272213879 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cb84359b60 0x55cb8435bf50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:52.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.797+0000 7f56eb7fe640 1 -- 172.21.15.114:0/272213879 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb8426e970 con 0x55cb8434e9a0 2024-09-13T18:28:52.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.797+0000 7f56eb7fe640 1 --2- 172.21.15.114:0/272213879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb8434e9a0 0x55cb8434ed80 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f56e40098c0 tx=0x7f56e4031ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b18cdfce6944f2b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:52.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.798+0000 7f56d17fa640 1 -- 172.21.15.114:0/272213879 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e4031480 con 0x55cb8434e9a0 2024-09-13T18:28:52.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.798+0000 7f56d17fa640 1 -- 172.21.15.114:0/272213879 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f56e4003970 con 0x55cb8434e9a0 2024-09-13T18:28:52.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.798+0000 7f56d17fa640 1 -- 172.21.15.114:0/272213879 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e40044d0 con 0x55cb8434e9a0 2024-09-13T18:28:52.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.799+0000 7f56d3fff640 1 -- 172.21.15.114:0/3440149025 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f56dc009820 con 0x7f56ec1074b0 2024-09-13T18:28:52.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.800+0000 7f56d3fff640 1 --2- 172.21.15.114:0/3440149025 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f56d40a1390 0x7f56d40a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:52.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.800+0000 7f56d3fff640 1 -- 172.21.15.114:0/3440149025 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f56dc009e20 con 0x7f56ec1074b0 2024-09-13T18:28:52.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.801+0000 7f56f4227740 1 -- 172.21.15.114:0/272213879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb8434e9a0 msgr2=0x55cb8434ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:52.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.801+0000 7f56f4227740 1 --2- 172.21.15.114:0/272213879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb8434e9a0 0x55cb8434ed80 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f56e40098c0 tx=0x7f56e4031ae0 comp rx=0 tx=0).stop 2024-09-13T18:28:52.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.801+0000 7f56eaffd640 1 --2- 172.21.15.114:0/3440149025 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f56d40a1390 0x7f56d40a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:52.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.801+0000 7f56f4227740 1 -- 172.21.15.114:0/272213879 shutdown_connections 2024-09-13T18:28:52.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.801+0000 7f56f4227740 1 --2- 172.21.15.114:0/272213879 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cb84359b60 0x55cb8435bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:52.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.801+0000 7f56f4227740 1 --2- 172.21.15.114:0/272213879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb8434f2c0 0x55cb84359620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:52.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.801+0000 7f56f4227740 1 --2- 172.21.15.114:0/272213879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb8434e9a0 0x55cb8434ed80 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:52.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.801+0000 7f56f4227740 1 -- 172.21.15.114:0/272213879 >> 172.21.15.114:0/272213879 conn(0x55cb8434af30 msgr2=0x55cb8434b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:52.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.801+0000 7f56eaffd640 1 --2- 172.21.15.114:0/3440149025 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f56d40a1390 0x7f56d40a3850 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x55cb8434aa30 tx=0x7f56dc004730 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:52.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.806+0000 7f56f4227740 1 -- 172.21.15.114:0/272213879 shutdown_connections 2024-09-13T18:28:52.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.808+0000 7f56f4227740 1 -- 172.21.15.114:0/272213879 wait complete. 2024-09-13T18:28:52.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.808+0000 7f56f4227740 1 -- start start 2024-09-13T18:28:52.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.808+0000 7f56f4227740 1 Processor -- start 2024-09-13T18:28:52.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.808+0000 7f56f4227740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb8434e9a0 0x55cb843594e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:52.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.808+0000 7f56f4227740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cb8434f2c0 0x55cb84356760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:52.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.808+0000 7f56f4227740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb84359b60 0x55cb84356d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:52.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.808+0000 7f56f4227740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb84270f80 con 0x55cb8434f2c0 2024-09-13T18:28:52.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.808+0000 7f56f4227740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb84270c20 con 0x55cb84359b60 2024-09-13T18:28:52.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.808+0000 7f56f4227740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb8423dab0 con 0x55cb8434e9a0 2024-09-13T18:28:52.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.809+0000 7f56f4227740 10 client.0 _ll_register_callbacks cb 0x55cb842ac350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:28:52.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.809+0000 7f56eb7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb8434e9a0 0x55cb843594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:52.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.809+0000 7f56eb7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb8434e9a0 0x55cb843594e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:55720/0 (socket says 172.21.15.114:55720) 2024-09-13T18:28:52.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.809+0000 7f56eb7fe640 1 -- 172.21.15.114:0/2607308995 learned_addr learned my addr 172.21.15.114:0/2607308995 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:52.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.809+0000 7f56eb7fe640 1 --2- 172.21.15.114:0/2607308995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb84359b60 0x55cb84356d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:52.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.809+0000 7f56eb7fe640 1 -- 172.21.15.114:0/2607308995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb84359b60 msgr2=0x55cb84356d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:52.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.809+0000 7f56eb7fe640 1 --2- 172.21.15.114:0/2607308995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb84359b60 0x55cb84356d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:52.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.809+0000 7f56eb7fe640 1 -- 172.21.15.114:0/2607308995 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cb8434f2c0 msgr2=0x55cb84356760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:28:52.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.809+0000 7f56eb7fe640 1 --2- 172.21.15.114:0/2607308995 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cb8434f2c0 0x55cb84356760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:52.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.809+0000 7f56eb7fe640 1 -- 172.21.15.114:0/2607308995 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56ec11c170 con 0x55cb8434e9a0 2024-09-13T18:28:52.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.810+0000 7f56eb7fe640 1 --2- 172.21.15.114:0/2607308995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb8434e9a0 0x55cb843594e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f56e4003fd0 tx=0x7f56e4004150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d79b36b6bff981d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:52.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.810+0000 7f56ceff5640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:28:52.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.812+0000 7f56ceff5640 1 -- 172.21.15.114:0/2607308995 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e4031480 con 0x55cb8434e9a0 2024-09-13T18:28:52.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.812+0000 7f56ceff5640 1 -- 172.21.15.114:0/2607308995 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f56e40315e0 con 0x55cb8434e9a0 2024-09-13T18:28:52.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.815+0000 7f56f4227740 1 -- 172.21.15.114:0/2607308995 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cb83fa6590 con 0x55cb8434e9a0 2024-09-13T18:28:52.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.820+0000 7f56ceff5640 1 -- 172.21.15.114:0/2607308995 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56e40317a0 con 0x55cb8434e9a0 2024-09-13T18:28:52.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.820+0000 7f56f4227740 10 client.26944 fetch_fsmap learned FSMap version 34 2024-09-13T18:28:52.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.820+0000 7f56f4227740 1 -- 172.21.15.114:0/2607308995 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55cb842b1360 con 0x55cb8434e9a0 2024-09-13T18:28:52.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.822+0000 7f56ceff5640 1 -- 172.21.15.114:0/2607308995 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55cb83fa6590 con 0x55cb8434e9a0 2024-09-13T18:28:52.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.822+0000 7f56ceff5640 1 -- 172.21.15.114:0/2607308995 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f56e4070070 con 0x55cb8434e9a0 2024-09-13T18:28:52.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.823+0000 7f56f4227740 10 client.26944 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:28:52.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.823+0000 7f56f4227740 10 client.26944 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:28:52.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.823+0000 7f56f4227740 20 client.26944 populate_metadata read hostname 'smithi114' 2024-09-13T18:28:52.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.823+0000 7f56f4227740 1 --2- 172.21.15.114:0/2607308995 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55cb84369450 0x55cb8436b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:52.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.823+0000 7f56f4227740 4 client.26944 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:28:52.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.823+0000 7f56f4227740 1 -- 172.21.15.114:0/2607308995 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cb83fb7860 con 0x55cb84369450 2024-09-13T18:28:52.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.824+0000 7f56ea7fc640 1 --2- 172.21.15.114:0/2607308995 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55cb84369450 0x55cb8436b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:52.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.832+0000 7f56ea7fc640 1 --2- 172.21.15.114:0/2607308995 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55cb84369450 0x55cb8436b840 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=582cd72d6f13dcb2 server_cookie=c66cfbb32050b6d in_seq=0 out_seq=0 2024-09-13T18:28:52.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.832+0000 7f56ceff5640 10 client.26944 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:28:52.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.832+0000 7f56ceff5640 1 -- 172.21.15.114:0/2607308995 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55cb83fb7860 con 0x55cb84369450 2024-09-13T18:28:52.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.833+0000 7f56ceff5640 10 client.26944 handle_command_reply: tid=0 2024-09-13T18:28:52.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.833+0000 7f56f4227740 1 client.26944 shutdown 2024-09-13T18:28:52.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.840+0000 7f56f4227740 1 -- 172.21.15.114:0/2607308995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb8434e9a0 msgr2=0x55cb843594e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:52.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.840+0000 7f56f4227740 1 --2- 172.21.15.114:0/2607308995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb8434e9a0 0x55cb843594e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f56e4003fd0 tx=0x7f56e4004150 comp rx=0 tx=0).stop 2024-09-13T18:28:52.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.841+0000 7f56f4227740 1 -- 172.21.15.114:0/2607308995 shutdown_connections 2024-09-13T18:28:52.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.841+0000 7f56f4227740 1 --2- 172.21.15.114:0/2607308995 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55cb84369450 0x55cb8436b840 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:52.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.841+0000 7f56f4227740 1 --2- 172.21.15.114:0/2607308995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb84359b60 0x55cb84356d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:52.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.841+0000 7f56f4227740 1 --2- 172.21.15.114:0/2607308995 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cb8434f2c0 0x55cb84356760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:52.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.841+0000 7f56f4227740 1 --2- 172.21.15.114:0/2607308995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb8434e9a0 0x55cb843594e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:52.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.841+0000 7f56f4227740 1 -- 172.21.15.114:0/2607308995 >> 172.21.15.114:0/2607308995 conn(0x55cb8434af30 msgr2=0x55cb8434a160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:52.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.842+0000 7f56ceff5640 0 client.26944 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:28:52.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.843+0000 7f56f4227740 1 -- 172.21.15.114:0/2607308995 shutdown_connections 2024-09-13T18:28:52.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.843+0000 7f56f4227740 1 -- 172.21.15.114:0/2607308995 wait complete. 2024-09-13T18:28:52.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.845+0000 7f56f4227740 20 client.26944 trim_cache size 0 max 16384 2024-09-13T18:28:52.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.887+0000 7f56f4227740 1 Processor -- start 2024-09-13T18:28:52.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.889+0000 7f56f4227740 1 -- start start 2024-09-13T18:28:52.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.890+0000 7f56f4227740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb8434e9a0 0x55cb84359040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:52.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.890+0000 7f56f4227740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cb8434f2c0 0x55cb84356000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:52.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.890+0000 7f56f4227740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb84359b60 0x55cb84356540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:52.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.890+0000 7f56f4227740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb84354cb0 con 0x55cb8434f2c0 2024-09-13T18:28:52.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.890+0000 7f56f4227740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb8434f0b0 con 0x55cb84359b60 2024-09-13T18:28:52.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.890+0000 7f56f4227740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb84365ea0 con 0x55cb8434e9a0 2024-09-13T18:28:52.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.890+0000 7f56eb7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb8434e9a0 0x55cb84359040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:52.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.890+0000 7f56eb7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb8434e9a0 0x55cb84359040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:55722/0 (socket says 172.21.15.114:55722) 2024-09-13T18:28:52.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.890+0000 7f56eb7fe640 1 -- 172.21.15.114:0/299140441 learned_addr learned my addr 172.21.15.114:0/299140441 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:52.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.890+0000 7f56eb7fe640 1 --2- 172.21.15.114:0/299140441 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb84359b60 0x55cb84356540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:52.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.890+0000 7f56ea7fc640 1 --2- 172.21.15.114:0/299140441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cb8434f2c0 0x55cb84356000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:52.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.893+0000 7f56ea7fc640 1 -- 172.21.15.114:0/299140441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb8434e9a0 msgr2=0x55cb84359040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:52.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.893+0000 7f56ea7fc640 1 --2- 172.21.15.114:0/299140441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb8434e9a0 0x55cb84359040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:52.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.893+0000 7f56ea7fc640 1 -- 172.21.15.114:0/299140441 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb84359b60 msgr2=0x55cb84356540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:52.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.893+0000 7f56ea7fc640 1 --2- 172.21.15.114:0/299140441 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb84359b60 0x55cb84356540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:52.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.893+0000 7f56ea7fc640 1 -- 172.21.15.114:0/299140441 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb842531c0 con 0x55cb8434f2c0 2024-09-13T18:28:52.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.894+0000 7f56ea7fc640 1 --2- 172.21.15.114:0/299140441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cb8434f2c0 0x55cb84356000 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x55cb84357d80 tx=0x7f56e0009dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62b217fba9c24eae server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:52.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.894+0000 7f56d1ffb640 1 -- 172.21.15.114:0/299140441 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e0015440 con 0x55cb8434f2c0 2024-09-13T18:28:52.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.894+0000 7f56d1ffb640 1 -- 172.21.15.114:0/299140441 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f56e00127f0 con 0x55cb8434f2c0 2024-09-13T18:28:52.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.895+0000 7f56f4227740 1 -- 172.21.15.114:0/299140441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cb8434f2c0 msgr2=0x55cb84356000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:52.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.895+0000 7f56f4227740 1 --2- 172.21.15.114:0/299140441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cb8434f2c0 0x55cb84356000 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x55cb84357d80 tx=0x7f56e0009dc0 comp rx=0 tx=0).stop 2024-09-13T18:28:52.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.898+0000 7f56f4227740 1 -- 172.21.15.114:0/299140441 shutdown_connections 2024-09-13T18:28:52.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.898+0000 7f56f4227740 1 --2- 172.21.15.114:0/299140441 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb84359b60 0x55cb84356540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:52.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.898+0000 7f56f4227740 1 --2- 172.21.15.114:0/299140441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cb8434f2c0 0x55cb84356000 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:52.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.898+0000 7f56f4227740 1 --2- 172.21.15.114:0/299140441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb8434e9a0 0x55cb84359040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:52.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.898+0000 7f56f4227740 1 -- 172.21.15.114:0/299140441 >> 172.21.15.114:0/299140441 conn(0x55cb8434af30 msgr2=0x55cb84355030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:52.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.903+0000 7f56f4227740 1 -- 172.21.15.114:0/299140441 shutdown_connections 2024-09-13T18:28:52.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.904+0000 7f56f4227740 1 -- 172.21.15.114:0/299140441 wait complete. 2024-09-13T18:28:52.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.904+0000 7f56f4227740 1 -- start start 2024-09-13T18:28:52.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.904+0000 7f56f4227740 1 Processor -- start 2024-09-13T18:28:52.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.904+0000 7f56f4227740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cb8434e9a0 0x55cb84360b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:52.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.905+0000 7f56f4227740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb8434f2c0 0x55cb8435e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:52.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.905+0000 7f56f4227740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb84359b60 0x55cb8435ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:52.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.905+0000 7f56f4227740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb8435f450 con 0x55cb8434e9a0 2024-09-13T18:28:52.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.905+0000 7f56f4227740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb8435f5c0 con 0x55cb84359b60 2024-09-13T18:28:52.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.905+0000 7f56f4227740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb843c0930 con 0x55cb8434f2c0 2024-09-13T18:28:52.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.905+0000 7f56f4227740 10 client.0 _ll_register_callbacks cb 0x7f56e4009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:28:52.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.905+0000 7f56ea7fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb8434f2c0 0x55cb8435e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:52.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.905+0000 7f56ea7fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb8434f2c0 0x55cb8435e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:55734/0 (socket says 172.21.15.114:55734) 2024-09-13T18:28:52.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.905+0000 7f56ea7fc640 1 -- 172.21.15.114:0/3323797893 learned_addr learned my addr 172.21.15.114:0/3323797893 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:52.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.906+0000 7f56ea7fc640 1 -- 172.21.15.114:0/3323797893 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb84359b60 msgr2=0x55cb8435ed50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:28:52.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.906+0000 7f56ea7fc640 1 --2- 172.21.15.114:0/3323797893 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb84359b60 0x55cb8435ed50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:52.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.906+0000 7f56ea7fc640 1 -- 172.21.15.114:0/3323797893 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cb8434e9a0 msgr2=0x55cb84360b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:52.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.906+0000 7f56ea7fc640 1 --2- 172.21.15.114:0/3323797893 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cb8434e9a0 0x55cb84360b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:52.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.906+0000 7f56ea7fc640 1 -- 172.21.15.114:0/3323797893 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56e0009660 con 0x55cb8434f2c0 2024-09-13T18:28:52.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.906+0000 7f56ea7fc640 1 --2- 172.21.15.114:0/3323797893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb8434f2c0 0x55cb8435e810 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f56e000a010 tx=0x7f56e0015670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84e9393043b06bc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:52.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.906+0000 7f56ceff5640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:28:52.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.906+0000 7f56ceff5640 1 -- 172.21.15.114:0/3323797893 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e0012b60 con 0x55cb8434f2c0 2024-09-13T18:28:52.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.907+0000 7f56ceff5640 1 -- 172.21.15.114:0/3323797893 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f56e0012cc0 con 0x55cb8434f2c0 2024-09-13T18:28:52.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.908+0000 7f56f4227740 1 -- 172.21.15.114:0/3323797893 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f56e4003970 con 0x55cb8434f2c0 2024-09-13T18:28:52.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.910+0000 7f56ceff5640 1 -- 172.21.15.114:0/3323797893 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56e0012ec0 con 0x55cb8434f2c0 2024-09-13T18:28:52.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.911+0000 7f56f4227740 10 client.26950 fetch_fsmap learned FSMap version 34 2024-09-13T18:28:52.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.911+0000 7f56f4227740 1 -- 172.21.15.114:0/3323797893 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55cb84278990 con 0x55cb8434f2c0 2024-09-13T18:28:52.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.911+0000 7f56ceff5640 1 -- 172.21.15.114:0/3323797893 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f56e4003970 con 0x55cb8434f2c0 2024-09-13T18:28:52.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.911+0000 7f56ceff5640 1 -- 172.21.15.114:0/3323797893 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f56e0027820 con 0x55cb8434f2c0 2024-09-13T18:28:52.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.912+0000 7f56f4227740 10 client.26950 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:28:52.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.912+0000 7f56f4227740 10 client.26950 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:28:52.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.912+0000 7f56f4227740 20 client.26950 populate_metadata read hostname 'smithi114' 2024-09-13T18:28:52.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.912+0000 7f56f4227740 1 --2- 172.21.15.114:0/3323797893 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55cb843c1f20 0x55cb843c4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:52.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.912+0000 7f56f4227740 4 client.26950 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-13T18:28:52.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.912+0000 7f56f4227740 1 -- 172.21.15.114:0/3323797893 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55cb8400aeb0 con 0x55cb843c1f20 2024-09-13T18:28:52.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.913+0000 7f56eb7fe640 1 --2- 172.21.15.114:0/3323797893 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55cb843c1f20 0x55cb843c4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:52.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.914+0000 7f56eb7fe640 1 --2- 172.21.15.114:0/3323797893 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55cb843c1f20 0x55cb843c4340 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e92e9ca641c91d3 server_cookie=6f1f7775393b210c in_seq=0 out_seq=0 2024-09-13T18:28:52.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.914+0000 7f56ceff5640 10 client.26950 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:28:52.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.919+0000 7f56ceff5640 1 -- 172.21.15.114:0/3323797893 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55cb8400aeb0 con 0x55cb843c1f20 2024-09-13T18:28:52.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.919+0000 7f56ceff5640 10 client.26950 handle_command_reply: tid=0 2024-09-13T18:28:52.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.925+0000 7f56f4227740 1 client.26950 shutdown 2024-09-13T18:28:52.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.927+0000 7f56f4227740 1 -- 172.21.15.114:0/3323797893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb8434f2c0 msgr2=0x55cb8435e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:52.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.927+0000 7f56f4227740 1 --2- 172.21.15.114:0/3323797893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb8434f2c0 0x55cb8435e810 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f56e000a010 tx=0x7f56e0015670 comp rx=0 tx=0).stop 2024-09-13T18:28:52.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.928+0000 7f56f4227740 1 -- 172.21.15.114:0/3323797893 shutdown_connections 2024-09-13T18:28:52.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.928+0000 7f56f4227740 1 --2- 172.21.15.114:0/3323797893 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55cb843c1f20 0x55cb843c4340 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:52.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.928+0000 7f56f4227740 1 --2- 172.21.15.114:0/3323797893 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb84359b60 0x55cb8435ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:52.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.928+0000 7f56f4227740 1 --2- 172.21.15.114:0/3323797893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb8434f2c0 0x55cb8435e810 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:52.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.928+0000 7f56f4227740 1 --2- 172.21.15.114:0/3323797893 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cb8434e9a0 0x55cb84360b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:52.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.928+0000 7f56f4227740 1 -- 172.21.15.114:0/3323797893 >> 172.21.15.114:0/3323797893 conn(0x55cb8434af30 msgr2=0x55cb84354f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:52.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.928+0000 7f56ceff5640 0 client.26950 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:28:52.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.930+0000 7f56f4227740 1 -- 172.21.15.114:0/3323797893 shutdown_connections 2024-09-13T18:28:52.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.931+0000 7f56f4227740 1 -- 172.21.15.114:0/3323797893 wait complete. 2024-09-13T18:28:52.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.932+0000 7f56f4227740 20 client.26950 trim_cache size 0 max 16384 2024-09-13T18:28:52.935 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:28:52.935 INFO:teuthology.orchestra.run.smithi114.stdout: "return_code": 0, 2024-09-13T18:28:52.935 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_tag": "d36fc824-3a7e-4a09-91aa-da7f1e6a5a30", 2024-09-13T18:28:52.935 INFO:teuthology.orchestra.run.smithi114.stdout: "mode": "asynchronous" 2024-09-13T18:28:52.936 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:28:52.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.933+0000 7f56ebfff640 1 -- 172.21.15.114:0/3440149025 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f56d40a1390 msgr2=0x7f56d40a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:52.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.933+0000 7f56ebfff640 1 --2- 172.21.15.114:0/3440149025 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f56d40a1390 0x7f56d40a3850 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x55cb8434aa30 tx=0x7f56dc004730 comp rx=0 tx=0).stop 2024-09-13T18:28:52.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.933+0000 7f56ebfff640 1 -- 172.21.15.114:0/3440149025 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56ec1074b0 msgr2=0x7f56ec1a5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:52.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.933+0000 7f56ebfff640 1 --2- 172.21.15.114:0/3440149025 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56ec1074b0 0x7f56ec1a5080 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f56dc00d6d0 tx=0x7f56dc00db90 comp rx=0 tx=0).stop 2024-09-13T18:28:52.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.942+0000 7f56ebfff640 1 -- 172.21.15.114:0/3440149025 shutdown_connections 2024-09-13T18:28:52.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.942+0000 7f56ebfff640 1 --2- 172.21.15.114:0/3440149025 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f56d40a1390 0x7f56d40a3850 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:52.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.942+0000 7f56ebfff640 1 --2- 172.21.15.114:0/3440149025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56ec119400 0x7f56ec1a99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:52.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.942+0000 7f56ebfff640 1 --2- 172.21.15.114:0/3440149025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56ec107890 0x7f56ec1a55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:52.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.942+0000 7f56ebfff640 1 --2- 172.21.15.114:0/3440149025 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56ec1074b0 0x7f56ec1a5080 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:52.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.942+0000 7f56ebfff640 1 -- 172.21.15.114:0/3440149025 >> 172.21.15.114:0/3440149025 conn(0x7f56ec06af40 msgr2=0x7f56ec119c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:52.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.950+0000 7f56ebfff640 1 -- 172.21.15.114:0/3440149025 shutdown_connections 2024-09-13T18:28:52.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:52.950+0000 7f56ebfff640 1 -- 172.21.15.114:0/3440149025 wait complete. 2024-09-13T18:28:52.981 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T18:28:53.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.234+0000 7f8b07b25640 1 -- 172.21.15.114:0/3131834109 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b001074b0 msgr2=0x7f8b0010e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:53.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.234+0000 7f8b07b25640 1 --2- 172.21.15.114:0/3131834109 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b001074b0 0x7f8b0010e510 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f8afc01a940 tx=0x7f8afc03b470 comp rx=0 tx=0).stop 2024-09-13T18:28:53.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.241+0000 7f8b07b25640 1 -- 172.21.15.114:0/3131834109 shutdown_connections 2024-09-13T18:28:53.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.241+0000 7f8b07b25640 1 --2- 172.21.15.114:0/3131834109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b00119400 0x7f8b0011b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:53.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.241+0000 7f8b07b25640 1 --2- 172.21.15.114:0/3131834109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b00107890 0x7f8b00107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:53.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.241+0000 7f8b07b25640 1 --2- 172.21.15.114:0/3131834109 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b001074b0 0x7f8b0010e510 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:53.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.241+0000 7f8b07b25640 1 -- 172.21.15.114:0/3131834109 >> 172.21.15.114:0/3131834109 conn(0x7f8b0006af20 msgr2=0x7f8b0006b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:53.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.243+0000 7f8b07b25640 1 -- 172.21.15.114:0/3131834109 shutdown_connections 2024-09-13T18:28:53.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.245+0000 7f8b07b25640 1 -- 172.21.15.114:0/3131834109 wait complete. 2024-09-13T18:28:53.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.247+0000 7f8b07b25640 1 Processor -- start 2024-09-13T18:28:53.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.247+0000 7f8b07b25640 1 -- start start 2024-09-13T18:28:53.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.248+0000 7f8b07b25640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b001074b0 0x7f8b00114b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:53.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.248+0000 7f8b07b25640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b00107890 0x7f8b001150c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:53.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.248+0000 7f8b07b25640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b00119400 0x7f8b001a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:53.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.248+0000 7f8b07b25640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b001a1730 con 0x7f8b001074b0 2024-09-13T18:28:53.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.248+0000 7f8b07b25640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b001a18a0 con 0x7f8b00119400 2024-09-13T18:28:53.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.248+0000 7f8b07b25640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b001a1a60 con 0x7f8b00107890 2024-09-13T18:28:53.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.248+0000 7f8b060c2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b00119400 0x7f8b001a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:53.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.248+0000 7f8b060c2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b00119400 0x7f8b001a1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:50374/0 (socket says 172.21.15.114:50374) 2024-09-13T18:28:53.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.248+0000 7f8b060c2640 1 -- 172.21.15.114:0/2337598311 learned_addr learned my addr 172.21.15.114:0/2337598311 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:53.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.249+0000 7f8b060c2640 1 -- 172.21.15.114:0/2337598311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b00107890 msgr2=0x7f8b001150c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:28:53.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.249+0000 7f8b060c2640 1 --2- 172.21.15.114:0/2337598311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b00107890 0x7f8b001150c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:53.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.249+0000 7f8b060c2640 1 -- 172.21.15.114:0/2337598311 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b001074b0 msgr2=0x7f8b00114b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:53.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.249+0000 7f8b060c2640 1 --2- 172.21.15.114:0/2337598311 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b001074b0 0x7f8b00114b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:53.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.249+0000 7f8b060c2640 1 -- 172.21.15.114:0/2337598311 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8afc01a570 con 0x7f8b00119400 2024-09-13T18:28:53.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.249+0000 7f8b060c2640 1 --2- 172.21.15.114:0/2337598311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b00119400 0x7f8b001a1040 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8af800cab0 tx=0x7f8af800cf70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:53.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.250+0000 7f8af6ffd640 1 -- 172.21.15.114:0/2337598311 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af800ae20 con 0x7f8b00119400 2024-09-13T18:28:53.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.250+0000 7f8af6ffd640 1 -- 172.21.15.114:0/2337598311 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8af800f8e0 con 0x7f8b00119400 2024-09-13T18:28:53.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.250+0000 7f8b07b25640 1 -- 172.21.15.114:0/2337598311 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b001a1d40 con 0x7f8b00119400 2024-09-13T18:28:53.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.250+0000 7f8b07b25640 1 -- 172.21.15.114:0/2337598311 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b001a9440 con 0x7f8b00119400 2024-09-13T18:28:53.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.251+0000 7f8af6ffd640 1 -- 172.21.15.114:0/2337598311 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af8018940 con 0x7f8b00119400 2024-09-13T18:28:53.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.253+0000 7f8af6ffd640 1 -- 172.21.15.114:0/2337598311 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f8af800f450 con 0x7f8b00119400 2024-09-13T18:28:53.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.254+0000 7f8af6ffd640 1 --2- 172.21.15.114:0/2337598311 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f8aec0a11a0 0x7f8aec0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:53.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.254+0000 7f8af6ffd640 1 -- 172.21.15.114:0/2337598311 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f8af80cd430 con 0x7f8b00119400 2024-09-13T18:28:53.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.257+0000 7f8b0a84f740 1 Processor -- start 2024-09-13T18:28:53.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.258+0000 7f8b0a84f740 1 -- start start 2024-09-13T18:28:53.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.258+0000 7f8b0a84f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644e2a0efe0 0x5644e2a0f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:53.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.258+0000 7f8b0a84f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5644e2a0f900 0x5644e2a19c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:53.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.258+0000 7f8b0a84f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644e2a1a1a0 0x5644e2a1c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:53.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.258+0000 7f8b0a84f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644e2979f20 con 0x5644e2a0f900 2024-09-13T18:28:53.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.258+0000 7f8b0a84f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644e29776b0 con 0x5644e2a1a1a0 2024-09-13T18:28:53.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.258+0000 7f8b0a84f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644e2972fa0 con 0x5644e2a0efe0 2024-09-13T18:28:53.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.258+0000 7f8b060c2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644e2a1a1a0 0x5644e2a1c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:53.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.258+0000 7f8b060c2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644e2a1a1a0 0x5644e2a1c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:50378/0 (socket says 172.21.15.114:50378) 2024-09-13T18:28:53.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.258+0000 7f8b060c2640 1 -- 172.21.15.114:0/2795024990 learned_addr learned my addr 172.21.15.114:0/2795024990 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:53.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.259+0000 7f8b060c2640 1 -- 172.21.15.114:0/2795024990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644e2a0efe0 msgr2=0x5644e2a0f3c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:28:53.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.259+0000 7f8b060c2640 1 --2- 172.21.15.114:0/2795024990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644e2a0efe0 0x5644e2a0f3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:53.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.259+0000 7f8b060c2640 1 -- 172.21.15.114:0/2795024990 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5644e2a0f900 msgr2=0x5644e2a19c60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:28:53.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.259+0000 7f8b060c2640 1 --2- 172.21.15.114:0/2795024990 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5644e2a0f900 0x5644e2a19c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:53.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.259+0000 7f8b060c2640 1 -- 172.21.15.114:0/2795024990 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5644e292d840 con 0x5644e2a1a1a0 2024-09-13T18:28:53.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.259+0000 7f8b058c1640 1 --2- 172.21.15.114:0/2337598311 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f8aec0a11a0 0x7f8aec0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:53.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.259+0000 7f8b060c2640 1 --2- 172.21.15.114:0/2795024990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644e2a1a1a0 0x5644e2a1c590 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8af80a20e0 tx=0x7f8af8099bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77abe5c20e447313 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:53.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.260+0000 7f8aebfff640 1 -- 172.21.15.114:0/2795024990 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af809a190 con 0x5644e2a1a1a0 2024-09-13T18:28:53.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.260+0000 7f8aebfff640 1 -- 172.21.15.114:0/2795024990 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8af809dfc0 con 0x5644e2a1a1a0 2024-09-13T18:28:53.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.261+0000 7f8aebfff640 1 -- 172.21.15.114:0/2795024990 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af80a2180 con 0x5644e2a1a1a0 2024-09-13T18:28:53.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.261+0000 7f8b0a84f740 1 -- 172.21.15.114:0/2795024990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644e2a1a1a0 msgr2=0x5644e2a1c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:53.264 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.261+0000 7f8b0a84f740 1 --2- 172.21.15.114:0/2795024990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644e2a1a1a0 0x5644e2a1c590 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8af80a20e0 tx=0x7f8af8099bc0 comp rx=0 tx=0).stop 2024-09-13T18:28:53.264 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.261+0000 7f8b058c1640 1 --2- 172.21.15.114:0/2337598311 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f8aec0a11a0 0x7f8aec0a3660 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8afc044040 tx=0x7f8afc000bc0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:53.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.262+0000 7f8b0a84f740 1 -- 172.21.15.114:0/2795024990 shutdown_connections 2024-09-13T18:28:53.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.262+0000 7f8b0a84f740 1 --2- 172.21.15.114:0/2795024990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644e2a1a1a0 0x5644e2a1c590 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:53.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.262+0000 7f8b0a84f740 1 --2- 172.21.15.114:0/2795024990 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5644e2a0f900 0x5644e2a19c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:53.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.263+0000 7f8b0a84f740 1 --2- 172.21.15.114:0/2795024990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644e2a0efe0 0x5644e2a0f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:53.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.263+0000 7f8b0a84f740 1 -- 172.21.15.114:0/2795024990 >> 172.21.15.114:0/2795024990 conn(0x5644e2a0b570 msgr2=0x5644e2a0b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:53.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.263+0000 7f8b0a84f740 1 -- 172.21.15.114:0/2795024990 shutdown_connections 2024-09-13T18:28:53.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.263+0000 7f8b0a84f740 1 -- 172.21.15.114:0/2795024990 wait complete. 2024-09-13T18:28:53.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.264+0000 7f8b0a84f740 1 -- start start 2024-09-13T18:28:53.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.264+0000 7f8b0a84f740 1 Processor -- start 2024-09-13T18:28:53.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.268+0000 7f8b0a84f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644e2a0efe0 0x5644e2a19aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:53.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.268+0000 7f8b0a84f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644e2a0f900 0x5644e2a16d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:53.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.268+0000 7f8b0a84f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5644e2a1a1a0 0x5644e2a172b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:53.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.268+0000 7f8b0a84f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644e29142a0 con 0x5644e2a1a1a0 2024-09-13T18:28:53.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.268+0000 7f8b0a84f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644e264b430 con 0x5644e2a0f900 2024-09-13T18:28:53.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.268+0000 7f8b0a84f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644e28f6680 con 0x5644e2a0efe0 2024-09-13T18:28:53.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.268+0000 7f8b0a84f740 10 client.0 _ll_register_callbacks cb 0x5644e296c610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:28:53.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.269+0000 7f8b060c2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5644e2a1a1a0 0x5644e2a172b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:53.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.269+0000 7f8b060c2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5644e2a1a1a0 0x5644e2a172b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48262/0 (socket says 172.21.15.114:48262) 2024-09-13T18:28:53.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.269+0000 7f8b060c2640 1 -- 172.21.15.114:0/192625495 learned_addr learned my addr 172.21.15.114:0/192625495 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:53.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.269+0000 7f8b058c1640 1 --2- 172.21.15.114:0/192625495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644e2a0efe0 0x5644e2a19aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:53.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.270+0000 7f8b060c2640 1 -- 172.21.15.114:0/192625495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644e2a0efe0 msgr2=0x5644e2a19aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:53.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.270+0000 7f8b060c2640 1 --2- 172.21.15.114:0/192625495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644e2a0efe0 0x5644e2a19aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:53.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.270+0000 7f8b060c2640 1 -- 172.21.15.114:0/192625495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644e2a0f900 msgr2=0x5644e2a16d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:53.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.270+0000 7f8b060c2640 1 --2- 172.21.15.114:0/192625495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644e2a0f900 0x5644e2a16d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:53.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.270+0000 7f8b060c2640 1 -- 172.21.15.114:0/192625495 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b001a9440 con 0x5644e2a1a1a0 2024-09-13T18:28:53.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.270+0000 7f8b060c2640 1 --2- 172.21.15.114:0/192625495 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5644e2a1a1a0 0x5644e2a172b0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f8af800a6e0 tx=0x7f8af80a2730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ac3d95ef4d9d629 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:53.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.270+0000 7f8ae97fa640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:28:53.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.271+0000 7f8ae97fa640 1 -- 172.21.15.114:0/192625495 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af80cac10 con 0x5644e2a1a1a0 2024-09-13T18:28:53.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.271+0000 7f8b0a84f740 1 -- 172.21.15.114:0/192625495 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5644e26664e0 con 0x5644e2a1a1a0 2024-09-13T18:28:53.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.272+0000 7f8ae97fa640 1 -- 172.21.15.114:0/192625495 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f8af8004b40 con 0x5644e2a1a1a0 2024-09-13T18:28:53.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.272+0000 7f8ae97fa640 1 -- 172.21.15.114:0/192625495 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8af8004d00 con 0x5644e2a1a1a0 2024-09-13T18:28:53.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.276+0000 7f8b0a84f740 10 client.17532 fetch_fsmap learned FSMap version 34 2024-09-13T18:28:53.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.276+0000 7f8b0a84f740 1 -- 172.21.15.114:0/192625495 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5644e2971620 con 0x5644e2a1a1a0 2024-09-13T18:28:53.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.279+0000 7f8ae97fa640 1 -- 172.21.15.114:0/192625495 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8af80a6080 con 0x5644e2a1a1a0 2024-09-13T18:28:53.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.279+0000 7f8ae97fa640 1 -- 172.21.15.114:0/192625495 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8af8016020 con 0x5644e2a1a1a0 2024-09-13T18:28:53.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.279+0000 7f8b0a84f740 10 client.17532 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:28:53.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.279+0000 7f8b0a84f740 10 client.17532 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:28:53.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.279+0000 7f8b0a84f740 20 client.17532 populate_metadata read hostname 'smithi114' 2024-09-13T18:28:53.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.279+0000 7f8b0a84f740 1 --2- 172.21.15.114:0/192625495 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5644e2a29b00 0x5644e2a2bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:53.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.279+0000 7f8b0a84f740 4 client.17532 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:28:53.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.279+0000 7f8b0a84f740 1 -- 172.21.15.114:0/192625495 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5644e26777e0 con 0x5644e2a29b00 2024-09-13T18:28:53.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.281+0000 7f8b058c1640 1 --2- 172.21.15.114:0/192625495 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5644e2a29b00 0x5644e2a2bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:53.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.290+0000 7f8b058c1640 1 --2- 172.21.15.114:0/192625495 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5644e2a29b00 0x5644e2a2bef0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33f0813195a26562 server_cookie=a0541cec631fd1dd in_seq=0 out_seq=0 2024-09-13T18:28:53.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.292+0000 7f8ae97fa640 10 client.17532 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:28:53.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.292+0000 7f8ae97fa640 1 -- 172.21.15.114:0/192625495 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5644e26777e0 con 0x5644e2a29b00 2024-09-13T18:28:53.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.292+0000 7f8ae97fa640 10 client.17532 handle_command_reply: tid=0 2024-09-13T18:28:53.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.292+0000 7f8b0a84f740 1 client.17532 shutdown 2024-09-13T18:28:53.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.306+0000 7f8b0a84f740 1 -- 172.21.15.114:0/192625495 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5644e2a1a1a0 msgr2=0x5644e2a172b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:53.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.306+0000 7f8b0a84f740 1 --2- 172.21.15.114:0/192625495 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5644e2a1a1a0 0x5644e2a172b0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f8af800a6e0 tx=0x7f8af80a2730 comp rx=0 tx=0).stop 2024-09-13T18:28:53.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.309+0000 7f8b0a84f740 1 -- 172.21.15.114:0/192625495 shutdown_connections 2024-09-13T18:28:53.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.309+0000 7f8b0a84f740 1 --2- 172.21.15.114:0/192625495 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5644e2a29b00 0x5644e2a2bef0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:53.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.309+0000 7f8b0a84f740 1 --2- 172.21.15.114:0/192625495 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5644e2a1a1a0 0x5644e2a172b0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:53.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.309+0000 7f8b0a84f740 1 --2- 172.21.15.114:0/192625495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644e2a0f900 0x5644e2a16d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:53.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.309+0000 7f8b0a84f740 1 --2- 172.21.15.114:0/192625495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644e2a0efe0 0x5644e2a19aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:53.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.309+0000 7f8b0a84f740 1 -- 172.21.15.114:0/192625495 >> 172.21.15.114:0/192625495 conn(0x5644e2a0b570 msgr2=0x5644e2a0a760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:53.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.309+0000 7f8ae97fa640 0 client.17532 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:28:53.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.313+0000 7f8b0a84f740 1 -- 172.21.15.114:0/192625495 shutdown_connections 2024-09-13T18:28:53.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.315+0000 7f8b0a84f740 1 -- 172.21.15.114:0/192625495 wait complete. 2024-09-13T18:28:53.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.318+0000 7f8b0a84f740 20 client.17532 trim_cache size 0 max 16384 2024-09-13T18:28:53.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.360+0000 7f8b0a84f740 1 Processor -- start 2024-09-13T18:28:53.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.360+0000 7f8b0a84f740 1 -- start start 2024-09-13T18:28:53.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.360+0000 7f8b0a84f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644e2a0efe0 0x5644e2a0b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:53.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.360+0000 7f8b0a84f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644e2a0f900 0x5644e2a0be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:53.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.361+0000 7f8b058c1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644e2a0efe0 0x5644e2a0b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:53.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.361+0000 7f8b058c1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644e2a0efe0 0x5644e2a0b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:50398/0 (socket says 172.21.15.114:50398) 2024-09-13T18:28:53.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.361+0000 7f8b0a84f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5644e2a1a1a0 0x5644e2a12020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:53.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.361+0000 7f8b050c0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644e2a0f900 0x5644e2a0be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:53.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.361+0000 7f8b050c0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644e2a0f900 0x5644e2a0be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:55646/0 (socket says 172.21.15.114:55646) 2024-09-13T18:28:53.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.361+0000 7f8b058c1640 1 -- 172.21.15.114:0/1505905982 learned_addr learned my addr 172.21.15.114:0/1505905982 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:53.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.361+0000 7f8b0a84f740 1 -- 172.21.15.114:0/1505905982 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644e2a0e810 con 0x5644e2a1a1a0 2024-09-13T18:28:53.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.361+0000 7f8b0a84f740 1 -- 172.21.15.114:0/1505905982 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644e2a25f50 con 0x5644e2a0efe0 2024-09-13T18:28:53.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.361+0000 7f8b0a84f740 1 -- 172.21.15.114:0/1505905982 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644e2a17620 con 0x5644e2a0f900 2024-09-13T18:28:53.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.361+0000 7f8b060c2640 1 --2- 172.21.15.114:0/1505905982 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5644e2a1a1a0 0x5644e2a12020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:53.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.362+0000 7f8b060c2640 1 -- 172.21.15.114:0/1505905982 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644e2a0f900 msgr2=0x5644e2a0be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:53.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.362+0000 7f8b060c2640 1 --2- 172.21.15.114:0/1505905982 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644e2a0f900 0x5644e2a0be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:53.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.362+0000 7f8b060c2640 1 -- 172.21.15.114:0/1505905982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644e2a0efe0 msgr2=0x5644e2a0b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:53.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.362+0000 7f8b060c2640 1 --2- 172.21.15.114:0/1505905982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644e2a0efe0 0x5644e2a0b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:53.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.362+0000 7f8b060c2640 1 -- 172.21.15.114:0/1505905982 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5644e2913790 con 0x5644e2a1a1a0 2024-09-13T18:28:53.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.363+0000 7f8b060c2640 1 --2- 172.21.15.114:0/1505905982 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5644e2a1a1a0 0x5644e2a12020 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f8af809a120 tx=0x7f8af8099bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f844219f20ea2767 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:53.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.364+0000 7f8af4e05640 1 -- 172.21.15.114:0/1505905982 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af8018aa0 con 0x5644e2a1a1a0 2024-09-13T18:28:53.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.364+0000 7f8af4e05640 1 -- 172.21.15.114:0/1505905982 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5644e26664e0 con 0x5644e2a1a1a0 2024-09-13T18:28:53.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.364+0000 7f8af4e05640 1 -- 172.21.15.114:0/1505905982 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af8018df0 con 0x5644e2a1a1a0 2024-09-13T18:28:53.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.365+0000 7f8b0a84f740 1 -- 172.21.15.114:0/1505905982 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5644e2a1a1a0 msgr2=0x5644e2a12020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:53.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.365+0000 7f8b0a84f740 1 --2- 172.21.15.114:0/1505905982 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5644e2a1a1a0 0x5644e2a12020 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f8af809a120 tx=0x7f8af8099bc0 comp rx=0 tx=0).stop 2024-09-13T18:28:53.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.366+0000 7f8b0a84f740 1 -- 172.21.15.114:0/1505905982 shutdown_connections 2024-09-13T18:28:53.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.366+0000 7f8b0a84f740 1 --2- 172.21.15.114:0/1505905982 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5644e2a1a1a0 0x5644e2a12020 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:53.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.366+0000 7f8b0a84f740 1 --2- 172.21.15.114:0/1505905982 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644e2a0f900 0x5644e2a0be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:53.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.366+0000 7f8b0a84f740 1 --2- 172.21.15.114:0/1505905982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644e2a0efe0 0x5644e2a0b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:53.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.366+0000 7f8b0a84f740 1 -- 172.21.15.114:0/1505905982 >> 172.21.15.114:0/1505905982 conn(0x5644e2a0b570 msgr2=0x5644e2a30850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:53.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.366+0000 7f8b0a84f740 1 -- 172.21.15.114:0/1505905982 shutdown_connections 2024-09-13T18:28:53.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.366+0000 7f8b0a84f740 1 -- 172.21.15.114:0/1505905982 wait complete. 2024-09-13T18:28:53.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.367+0000 7f8b0a84f740 1 -- start start 2024-09-13T18:28:53.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.367+0000 7f8b0a84f740 1 Processor -- start 2024-09-13T18:28:53.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.367+0000 7f8b0a84f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644e2a0efe0 0x5644e2a1e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:53.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.367+0000 7f8b0a84f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644e2a0f900 0x5644e2a1ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:53.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.368+0000 7f8b058c1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644e2a0efe0 0x5644e2a1e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:53.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.368+0000 7f8b058c1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644e2a0efe0 0x5644e2a1e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:50400/0 (socket says 172.21.15.114:50400) 2024-09-13T18:28:53.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.368+0000 7f8b0a84f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5644e2a1a1a0 0x5644e2a1f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:53.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.368+0000 7f8b050c0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644e2a0f900 0x5644e2a1ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:53.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.368+0000 7f8b050c0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644e2a0f900 0x5644e2a1ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:55656/0 (socket says 172.21.15.114:55656) 2024-09-13T18:28:53.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.368+0000 7f8b058c1640 1 -- 172.21.15.114:0/458116822 learned_addr learned my addr 172.21.15.114:0/458116822 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:28:53.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.368+0000 7f8b0a84f740 1 -- 172.21.15.114:0/458116822 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644e2a21960 con 0x5644e2a1a1a0 2024-09-13T18:28:53.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.368+0000 7f8b0a84f740 1 -- 172.21.15.114:0/458116822 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644e2a7efd0 con 0x5644e2a0efe0 2024-09-13T18:28:53.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.368+0000 7f8b0a84f740 1 -- 172.21.15.114:0/458116822 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644e2a7f110 con 0x5644e2a0f900 2024-09-13T18:28:53.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.368+0000 7f8b0a84f740 10 client.0 _ll_register_callbacks cb 0x7f8af809a190 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:28:53.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.368+0000 7f8b058c1640 1 -- 172.21.15.114:0/458116822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644e2a0f900 msgr2=0x5644e2a1ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:53.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.368+0000 7f8b058c1640 1 --2- 172.21.15.114:0/458116822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644e2a0f900 0x5644e2a1ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:53.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.368+0000 7f8b058c1640 1 -- 172.21.15.114:0/458116822 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5644e2a1a1a0 msgr2=0x5644e2a1f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:28:53.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.368+0000 7f8b058c1640 1 --2- 172.21.15.114:0/458116822 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5644e2a1a1a0 0x5644e2a1f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:53.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.368+0000 7f8b058c1640 1 -- 172.21.15.114:0/458116822 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5644e2971620 con 0x5644e2a0efe0 2024-09-13T18:28:53.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.369+0000 7f8b050c0640 1 --2- 172.21.15.114:0/458116822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644e2a0f900 0x5644e2a1ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:28:53.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.369+0000 7f8b058c1640 1 --2- 172.21.15.114:0/458116822 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644e2a0efe0 0x5644e2a1e8c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8afc060b50 tx=0x7f8afc04b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fce1da40abfc2f52 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:28:53.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.369+0000 7f8ae97fa640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:28:53.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.371+0000 7f8ae97fa640 1 -- 172.21.15.114:0/458116822 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8afc060040 con 0x5644e2a0efe0 2024-09-13T18:28:53.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.371+0000 7f8ae97fa640 1 -- 172.21.15.114:0/458116822 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f8afc053040 con 0x5644e2a0efe0 2024-09-13T18:28:53.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.371+0000 7f8b0a84f740 1 -- 172.21.15.114:0/458116822 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8af809dfc0 con 0x5644e2a0efe0 2024-09-13T18:28:53.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.373+0000 7f8ae97fa640 1 -- 172.21.15.114:0/458116822 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8afc060e40 con 0x5644e2a0efe0 2024-09-13T18:28:53.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.373+0000 7f8b0a84f740 10 client.26963 fetch_fsmap learned FSMap version 34 2024-09-13T18:28:53.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.373+0000 7f8b0a84f740 1 -- 172.21.15.114:0/458116822 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5644e2938ea0 con 0x5644e2a0efe0 2024-09-13T18:28:53.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.374+0000 7f8ae97fa640 1 -- 172.21.15.114:0/458116822 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8afc064070 con 0x5644e2a0efe0 2024-09-13T18:28:53.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.374+0000 7f8ae97fa640 1 -- 172.21.15.114:0/458116822 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8afc04a040 con 0x5644e2a0efe0 2024-09-13T18:28:53.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.374+0000 7f8b0a84f740 10 client.26963 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:28:53.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.374+0000 7f8b0a84f740 10 client.26963 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:28:53.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.374+0000 7f8b0a84f740 20 client.26963 populate_metadata read hostname 'smithi114' 2024-09-13T18:28:53.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.374+0000 7f8b0a84f740 1 --2- 172.21.15.114:0/458116822 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5644e2aa28f0 0x5644e2aa4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:28:53.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.374+0000 7f8b0a84f740 4 client.26963 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T18:28:53.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.374+0000 7f8b0a84f740 1 -- 172.21.15.114:0/458116822 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f8af809dfc0 con 0x5644e2aa28f0 2024-09-13T18:28:53.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.380+0000 7f8b050c0640 1 --2- 172.21.15.114:0/458116822 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5644e2aa28f0 0x5644e2aa4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:28:53.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.387+0000 7f8b050c0640 1 --2- 172.21.15.114:0/458116822 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5644e2aa28f0 0x5644e2aa4d10 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe3b82f2f13a0a28 server_cookie=1372ede66ef60dbb in_seq=0 out_seq=0 2024-09-13T18:28:53.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.388+0000 7f8ae97fa640 10 client.26963 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:28:53.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.388+0000 7f8ae97fa640 1 -- 172.21.15.114:0/458116822 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x7f8af809dfc0 con 0x5644e2aa28f0 2024-09-13T18:28:53.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.388+0000 7f8ae97fa640 10 client.26963 handle_command_reply: tid=0 2024-09-13T18:28:53.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.388+0000 7f8b0a84f740 1 client.26963 shutdown 2024-09-13T18:28:53.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.391+0000 7f8b0a84f740 1 -- 172.21.15.114:0/458116822 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644e2a0efe0 msgr2=0x5644e2a1e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:53.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.391+0000 7f8b0a84f740 1 --2- 172.21.15.114:0/458116822 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644e2a0efe0 0x5644e2a1e8c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8afc060b50 tx=0x7f8afc04b000 comp rx=0 tx=0).stop 2024-09-13T18:28:53.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.398+0000 7f8b0a84f740 1 -- 172.21.15.114:0/458116822 shutdown_connections 2024-09-13T18:28:53.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.398+0000 7f8b0a84f740 1 --2- 172.21.15.114:0/458116822 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5644e2aa28f0 0x5644e2aa4d10 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:53.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.398+0000 7f8b0a84f740 1 --2- 172.21.15.114:0/458116822 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5644e2a1a1a0 0x5644e2a1f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:53.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.398+0000 7f8ae97fa640 0 client.26963 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:28:53.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.398+0000 7f8b0a84f740 1 --2- 172.21.15.114:0/458116822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644e2a0f900 0x5644e2a1ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:53.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.398+0000 7f8b0a84f740 1 --2- 172.21.15.114:0/458116822 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644e2a0efe0 0x5644e2a1e8c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:53.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.399+0000 7f8b0a84f740 1 -- 172.21.15.114:0/458116822 >> 172.21.15.114:0/458116822 conn(0x5644e2a0b570 msgr2=0x5644e2a0b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:53.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.403+0000 7f8b0a84f740 1 -- 172.21.15.114:0/458116822 shutdown_connections 2024-09-13T18:28:53.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.403+0000 7f8b0a84f740 1 -- 172.21.15.114:0/458116822 wait complete. 2024-09-13T18:28:53.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.413+0000 7f8b0a84f740 20 client.26963 trim_cache size 0 max 16384 2024-09-13T18:28:53.418 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:28:53.418 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-13T18:28:53.418 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": { 2024-09-13T18:28:53.418 INFO:teuthology.orchestra.run.smithi114.stdout: "d36fc824-3a7e-4a09-91aa-da7f1e6a5a30": { 2024-09-13T18:28:53.419 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-13T18:28:53.419 INFO:teuthology.orchestra.run.smithi114.stdout: "tag": "d36fc824-3a7e-4a09-91aa-da7f1e6a5a30", 2024-09-13T18:28:53.419 INFO:teuthology.orchestra.run.smithi114.stdout: "options": "recursive,force" 2024-09-13T18:28:53.419 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:28:53.419 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:28:53.419 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:28:53.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.424+0000 7f8b07b25640 1 -- 172.21.15.114:0/2337598311 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f8aec0a11a0 msgr2=0x7f8aec0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:53.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.424+0000 7f8b07b25640 1 --2- 172.21.15.114:0/2337598311 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f8aec0a11a0 0x7f8aec0a3660 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8afc044040 tx=0x7f8afc000bc0 comp rx=0 tx=0).stop 2024-09-13T18:28:53.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.424+0000 7f8b07b25640 1 -- 172.21.15.114:0/2337598311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b00119400 msgr2=0x7f8b001a1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:28:53.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.424+0000 7f8b07b25640 1 --2- 172.21.15.114:0/2337598311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b00119400 0x7f8b001a1040 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8af800cab0 tx=0x7f8af800cf70 comp rx=0 tx=0).stop 2024-09-13T18:28:53.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.430+0000 7f8b07b25640 1 -- 172.21.15.114:0/2337598311 shutdown_connections 2024-09-13T18:28:53.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.430+0000 7f8b07b25640 1 --2- 172.21.15.114:0/2337598311 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f8aec0a11a0 0x7f8aec0a3660 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:53.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.430+0000 7f8b07b25640 1 --2- 172.21.15.114:0/2337598311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b00119400 0x7f8b001a1040 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:53.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.430+0000 7f8b07b25640 1 --2- 172.21.15.114:0/2337598311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b00107890 0x7f8b001150c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:53.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.430+0000 7f8b07b25640 1 --2- 172.21.15.114:0/2337598311 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b001074b0 0x7f8b00114b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:28:53.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.430+0000 7f8b07b25640 1 -- 172.21.15.114:0/2337598311 >> 172.21.15.114:0/2337598311 conn(0x7f8b0006af20 msgr2=0x7f8b0006ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:28:53.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.431+0000 7f8b07b25640 1 -- 172.21.15.114:0/2337598311 shutdown_connections 2024-09-13T18:28:53.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:28:53.432+0000 7f8b07b25640 1 -- 172.21.15.114:0/2337598311 wait complete. 2024-09-13T18:28:53.460 INFO:tasks.cephfs.filesystem:scrub status for tag:d36fc824-3a7e-4a09-91aa-da7f1e6a5a30 - {'path': '/', 'tag': 'd36fc824-3a7e-4a09-91aa-da7f1e6a5a30', 'options': 'recursive,force'} 2024-09-13T18:28:54.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:53 smithi134 ceph-mon[34283]: pgmap v195: 129 pgs: 129 active+clean; 20 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 32 MiB/s wr, 662 op/s 2024-09-13T18:28:54.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:53 smithi134 ceph-mon[34283]: scrub queued for path: / 2024-09-13T18:28:54.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:53 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:28:54.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:53 smithi134 ceph-mon[34283]: scrub summary: active paths [/] 2024-09-13T18:28:54.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:53 smithi134 ceph-mon[34283]: scrub summary: active paths [/] 2024-09-13T18:28:54.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:53 smithi134 ceph-mon[34283]: scrub summary: active paths [/] 2024-09-13T18:28:54.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:53 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:28:54.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:53 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:28:54.120 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:53 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:28:54.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:53 smithi204 ceph-mon[34777]: pgmap v195: 129 pgs: 129 active+clean; 20 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 32 MiB/s wr, 662 op/s 2024-09-13T18:28:54.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:53 smithi204 ceph-mon[34777]: scrub queued for path: / 2024-09-13T18:28:54.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:53 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:28:54.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:53 smithi204 ceph-mon[34777]: scrub summary: active paths [/] 2024-09-13T18:28:54.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:53 smithi204 ceph-mon[34777]: scrub summary: active paths [/] 2024-09-13T18:28:54.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:53 smithi204 ceph-mon[34777]: scrub summary: active paths [/] 2024-09-13T18:28:54.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:53 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:28:54.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:53 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:28:54.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:53 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:28:54.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:53 smithi114 ceph-mon[30897]: pgmap v195: 129 pgs: 129 active+clean; 20 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 32 MiB/s wr, 662 op/s 2024-09-13T18:28:54.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:53 smithi114 ceph-mon[30897]: scrub queued for path: / 2024-09-13T18:28:54.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:53 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:28:54.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:53 smithi114 ceph-mon[30897]: scrub summary: active paths [/] 2024-09-13T18:28:54.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:53 smithi114 ceph-mon[30897]: scrub summary: active paths [/] 2024-09-13T18:28:54.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:53 smithi114 ceph-mon[30897]: scrub summary: active paths [/] 2024-09-13T18:28:54.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:53 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:28:54.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:53 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:28:54.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:53 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:28:56.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:55 smithi134 ceph-mon[34283]: pgmap v196: 129 pgs: 129 active+clean; 20 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 37 MiB/s wr, 771 op/s 2024-09-13T18:28:56.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:55 smithi204 ceph-mon[34777]: pgmap v196: 129 pgs: 129 active+clean; 20 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 37 MiB/s wr, 771 op/s 2024-09-13T18:28:56.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:55 smithi114 ceph-mon[30897]: pgmap v196: 129 pgs: 129 active+clean; 20 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 37 MiB/s wr, 771 op/s 2024-09-13T18:28:58.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:57 smithi134 ceph-mon[34283]: pgmap v197: 129 pgs: 129 active+clean; 20 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 42 MiB/s wr, 881 op/s 2024-09-13T18:28:58.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:57 smithi204 ceph-mon[34777]: pgmap v197: 129 pgs: 129 active+clean; 20 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 42 MiB/s wr, 881 op/s 2024-09-13T18:28:58.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:57 smithi114 ceph-mon[30897]: pgmap v197: 129 pgs: 129 active+clean; 20 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 42 MiB/s wr, 881 op/s 2024-09-13T18:29:00.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:28:59 smithi134 ceph-mon[34283]: pgmap v198: 129 pgs: 129 active+clean; 20 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 31 MiB/s wr, 663 op/s 2024-09-13T18:29:00.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:28:59 smithi204 ceph-mon[34777]: pgmap v198: 129 pgs: 129 active+clean; 20 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 31 MiB/s wr, 663 op/s 2024-09-13T18:29:00.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:28:59 smithi114 ceph-mon[30897]: pgmap v198: 129 pgs: 129 active+clean; 20 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 31 MiB/s wr, 663 op/s 2024-09-13T18:29:02.022 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:29:01 smithi114 ceph-mon[30897]: pgmap v199: 129 pgs: 129 active+clean; 20 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 47 MiB/s wr, 999 op/s 2024-09-13T18:29:02.118 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:29:01 smithi134 ceph-mon[34283]: pgmap v199: 129 pgs: 129 active+clean; 20 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 47 MiB/s wr, 999 op/s 2024-09-13T18:29:02.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:29:01 smithi204 ceph-mon[34777]: pgmap v199: 129 pgs: 129 active+clean; 20 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 47 MiB/s wr, 999 op/s 2024-09-13T18:29:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.032+0000 7fa25de6b640 1 -- 172.21.15.114:0/645417932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2500a4a30 msgr2=0x7fa2500a4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.032+0000 7fa25de6b640 1 --2- 172.21.15.114:0/645417932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2500a4a30 0x7fa2500a4e10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa24c009930 tx=0x7fa24c02a440 comp rx=0 tx=0).stop 2024-09-13T18:29:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.036+0000 7fa25de6b640 1 -- 172.21.15.114:0/645417932 shutdown_connections 2024-09-13T18:29:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.036+0000 7fa25de6b640 1 --2- 172.21.15.114:0/645417932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2500acb80 0x7fa2500b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.036+0000 7fa25de6b640 1 --2- 172.21.15.114:0/645417932 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2500a53e0 0x7fa2500ac420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.036+0000 7fa25de6b640 1 --2- 172.21.15.114:0/645417932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2500a4a30 0x7fa2500a4e10 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.036+0000 7fa25de6b640 1 -- 172.21.15.114:0/645417932 >> 172.21.15.114:0/645417932 conn(0x7fa25001a0a0 msgr2=0x7fa25001a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.038+0000 7fa25de6b640 1 -- 172.21.15.114:0/645417932 shutdown_connections 2024-09-13T18:29:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.038+0000 7fa25de6b640 1 -- 172.21.15.114:0/645417932 wait complete. 2024-09-13T18:29:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.039+0000 7fa25de6b640 1 Processor -- start 2024-09-13T18:29:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.039+0000 7fa25de6b640 1 -- start start 2024-09-13T18:29:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.040+0000 7fa25de6b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2500a4a30 0x7fa2500b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.040+0000 7fa25de6b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2500acb80 0x7fa2500b4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.040+0000 7fa25de6b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2500b4860 0x7fa25014f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.040+0000 7fa25de6b640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2500b7fc0 con 0x7fa2500a4a30 2024-09-13T18:29:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.040+0000 7fa25de6b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25013ea70 con 0x7fa2500acb80 2024-09-13T18:29:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.040+0000 7fa25de6b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25013ec30 con 0x7fa2500b4860 2024-09-13T18:29:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.040+0000 7fa2577fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2500a4a30 0x7fa2500b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.040+0000 7fa2577fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2500a4a30 0x7fa2500b3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60340/0 (socket says 172.21.15.114:60340) 2024-09-13T18:29:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.040+0000 7fa2577fe640 1 -- 172.21.15.114:0/2464567135 learned_addr learned my addr 172.21.15.114:0/2464567135 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.041+0000 7fa2577fe640 1 -- 172.21.15.114:0/2464567135 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2500b4860 msgr2=0x7fa25014f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.041+0000 7fa2577fe640 1 --2- 172.21.15.114:0/2464567135 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2500b4860 0x7fa25014f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.041+0000 7fa2577fe640 1 -- 172.21.15.114:0/2464567135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2500acb80 msgr2=0x7fa2500b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.041+0000 7fa2577fe640 1 --2- 172.21.15.114:0/2464567135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2500acb80 0x7fa2500b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.041+0000 7fa2577fe640 1 -- 172.21.15.114:0/2464567135 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa24c009560 con 0x7fa2500a4a30 2024-09-13T18:29:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.042+0000 7fa2577fe640 1 --2- 172.21.15.114:0/2464567135 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2500a4a30 0x7fa2500b3de0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fa24c02a9b0 tx=0x7fa24c0055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:04.048 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.044+0000 7fa254ff9640 1 -- 172.21.15.114:0/2464567135 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24c004c90 con 0x7fa2500a4a30 2024-09-13T18:29:04.048 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.044+0000 7fa254ff9640 1 -- 172.21.15.114:0/2464567135 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa24c004df0 con 0x7fa2500a4a30 2024-09-13T18:29:04.048 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.044+0000 7fa254ff9640 1 -- 172.21.15.114:0/2464567135 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24c03c8d0 con 0x7fa2500a4a30 2024-09-13T18:29:04.048 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.046+0000 7fa25de6b640 1 -- 172.21.15.114:0/2464567135 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa25013eeb0 con 0x7fa2500a4a30 2024-09-13T18:29:04.048 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.046+0000 7fa25de6b640 1 -- 172.21.15.114:0/2464567135 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa25013f310 con 0x7fa2500a4a30 2024-09-13T18:29:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.058+0000 7fa260c6b740 1 Processor -- start 2024-09-13T18:29:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.058+0000 7fa254ff9640 1 -- 172.21.15.114:0/2464567135 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fa24c0035f0 con 0x7fa2500a4a30 2024-09-13T18:29:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.059+0000 7fa254ff9640 1 --2- 172.21.15.114:0/2464567135 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa2480a1410 0x7fa2480a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.060+0000 7fa254ff9640 1 -- 172.21.15.114:0/2464567135 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fa24c0ede20 con 0x7fa2500a4a30 2024-09-13T18:29:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.063+0000 7fa260c6b740 1 -- start start 2024-09-13T18:29:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.063+0000 7fa260c6b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56068c017500 0x56068c0178e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.063+0000 7fa260c6b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56068c017e20 0x56068c022180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.063+0000 7fa260c6b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56068c0226c0 0x56068c024ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.063+0000 7fa260c6b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56068bf82c30 con 0x56068c017e20 2024-09-13T18:29:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.063+0000 7fa260c6b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56068bf803c0 con 0x56068c0226c0 2024-09-13T18:29:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.063+0000 7fa260c6b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56068bf7bcb0 con 0x56068c017500 2024-09-13T18:29:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.064+0000 7fa256ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56068c017e20 0x56068c022180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.064+0000 7fa256ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56068c017e20 0x56068c022180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60354/0 (socket says 172.21.15.114:60354) 2024-09-13T18:29:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.064+0000 7fa256ffd640 1 -- 172.21.15.114:0/3094696973 learned_addr learned my addr 172.21.15.114:0/3094696973 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.064+0000 7fa256ffd640 1 -- 172.21.15.114:0/3094696973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56068c017500 msgr2=0x56068c0178e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:29:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.064+0000 7fa256ffd640 1 --2- 172.21.15.114:0/3094696973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56068c017500 0x56068c0178e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.064+0000 7fa256ffd640 1 -- 172.21.15.114:0/3094696973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56068c0226c0 msgr2=0x56068c024ab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:29:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.064+0000 7fa256ffd640 1 --2- 172.21.15.114:0/3094696973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56068c0226c0 0x56068c024ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.065+0000 7fa256ffd640 1 -- 172.21.15.114:0/3094696973 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56068bf22910 con 0x56068c017e20 2024-09-13T18:29:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.065+0000 7fa256ffd640 1 --2- 172.21.15.114:0/3094696973 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56068c017e20 0x56068c022180 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fa24400ba70 tx=0x7fa24400bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=551aae8ec10b003b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.065+0000 7fa257fff640 1 --2- 172.21.15.114:0/2464567135 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa2480a1410 0x7fa2480a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.066+0000 7fa241ffb640 1 -- 172.21.15.114:0/3094696973 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24400dc40 con 0x56068c017e20 2024-09-13T18:29:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.066+0000 7fa241ffb640 1 -- 172.21.15.114:0/3094696973 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa24400dda0 con 0x56068c017e20 2024-09-13T18:29:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.067+0000 7fa260c6b740 1 -- 172.21.15.114:0/3094696973 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56068c017e20 msgr2=0x56068c022180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.067+0000 7fa260c6b740 1 --2- 172.21.15.114:0/3094696973 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56068c017e20 0x56068c022180 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fa24400ba70 tx=0x7fa24400bf30 comp rx=0 tx=0).stop 2024-09-13T18:29:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.067+0000 7fa260c6b740 1 -- 172.21.15.114:0/3094696973 shutdown_connections 2024-09-13T18:29:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.067+0000 7fa260c6b740 1 --2- 172.21.15.114:0/3094696973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56068c0226c0 0x56068c024ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.067+0000 7fa260c6b740 1 --2- 172.21.15.114:0/3094696973 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56068c017e20 0x56068c022180 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.067+0000 7fa260c6b740 1 --2- 172.21.15.114:0/3094696973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56068c017500 0x56068c0178e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.067+0000 7fa260c6b740 1 -- 172.21.15.114:0/3094696973 >> 172.21.15.114:0/3094696973 conn(0x56068c013a90 msgr2=0x56068c013e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:04.073 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.071+0000 7fa260c6b740 1 -- 172.21.15.114:0/3094696973 shutdown_connections 2024-09-13T18:29:04.074 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.071+0000 7fa260c6b740 1 -- 172.21.15.114:0/3094696973 wait complete. 2024-09-13T18:29:04.074 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.071+0000 7fa260c6b740 1 -- start start 2024-09-13T18:29:04.074 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.072+0000 7fa260c6b740 1 Processor -- start 2024-09-13T18:29:04.075 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.072+0000 7fa260c6b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56068c017500 0x56068c021f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:04.075 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.072+0000 7fa260c6b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56068c017e20 0x56068c01f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:04.075 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.072+0000 7fa260c6b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56068c0226c0 0x56068c01f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:04.075 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.072+0000 7fa260c6b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56068bf39e80 con 0x56068c017500 2024-09-13T18:29:04.075 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.072+0000 7fa260c6b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56068bf39b20 con 0x56068c017e20 2024-09-13T18:29:04.075 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.072+0000 7fa260c6b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56068beff260 con 0x56068c0226c0 2024-09-13T18:29:04.076 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.072+0000 7fa260c6b740 10 client.0 _ll_register_callbacks cb 0x56068bf34df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:29:04.076 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.073+0000 7fa256ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56068c017e20 0x56068c01f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:04.076 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.073+0000 7fa256ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56068c017e20 0x56068c01f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:34728/0 (socket says 172.21.15.114:34728) 2024-09-13T18:29:04.076 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.073+0000 7fa256ffd640 1 -- 172.21.15.114:0/2914645894 learned_addr learned my addr 172.21.15.114:0/2914645894 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:04.077 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.075+0000 7fa257fff640 1 --2- 172.21.15.114:0/2914645894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56068c0226c0 0x56068c01f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:04.077 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.075+0000 7fa2577fe640 1 --2- 172.21.15.114:0/2914645894 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56068c017500 0x56068c021f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:04.077 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.075+0000 7fa257fff640 1 --2- 172.21.15.114:0/2464567135 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa2480a1410 0x7fa2480a38d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa25804f760 tx=0x7fa25804f270 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:04.081 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.078+0000 7fa256ffd640 1 -- 172.21.15.114:0/2914645894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56068c0226c0 msgr2=0x56068c01f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:04.081 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.078+0000 7fa256ffd640 1 --2- 172.21.15.114:0/2914645894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56068c0226c0 0x56068c01f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:04.081 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.078+0000 7fa256ffd640 1 -- 172.21.15.114:0/2914645894 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56068c017500 msgr2=0x56068c021f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:04.081 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.078+0000 7fa256ffd640 1 --2- 172.21.15.114:0/2914645894 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56068c017500 0x56068c021f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:04.081 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.078+0000 7fa256ffd640 1 -- 172.21.15.114:0/2914645894 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa24400b6d0 con 0x56068c017e20 2024-09-13T18:29:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.079+0000 7fa256ffd640 1 --2- 172.21.15.114:0/2914645894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56068c017e20 0x56068c01f1b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa244005940 tx=0x7fa24400dc70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb884028ee3a5467 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.080+0000 7fa236ffd640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:29:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.080+0000 7fa236ffd640 1 -- 172.21.15.114:0/2914645894 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24401f070 con 0x56068c017e20 2024-09-13T18:29:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.080+0000 7fa260c6b740 1 -- 172.21.15.114:0/2914645894 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56068bfa4680 con 0x56068c017e20 2024-09-13T18:29:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.080+0000 7fa236ffd640 1 -- 172.21.15.114:0/2914645894 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fa244004250 con 0x56068c017e20 2024-09-13T18:29:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.080+0000 7fa236ffd640 1 -- 172.21.15.114:0/2914645894 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa244004410 con 0x56068c017e20 2024-09-13T18:29:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.080+0000 7fa260c6b740 10 client.26969 fetch_fsmap learned FSMap version 34 2024-09-13T18:29:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.080+0000 7fa260c6b740 1 -- 172.21.15.114:0/2914645894 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x56068bf7a330 con 0x56068c017e20 2024-09-13T18:29:04.087 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.080+0000 7fa236ffd640 1 -- 172.21.15.114:0/2914645894 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56068bfa4680 con 0x56068c017e20 2024-09-13T18:29:04.087 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.081+0000 7fa236ffd640 1 -- 172.21.15.114:0/2914645894 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa2440108d0 con 0x56068c017e20 2024-09-13T18:29:04.087 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.081+0000 7fa260c6b740 10 client.26969 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:29:04.087 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.081+0000 7fa260c6b740 10 client.26969 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:29:04.087 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.081+0000 7fa260c6b740 20 client.26969 populate_metadata read hostname 'smithi114' 2024-09-13T18:29:04.087 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.081+0000 7fa260c6b740 1 --2- 172.21.15.114:0/2914645894 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56068c031fd0 0x56068c0343f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:04.087 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.081+0000 7fa260c6b740 4 client.26969 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:29:04.087 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.081+0000 7fa260c6b740 1 -- 172.21.15.114:0/2914645894 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56068bf6f5d0 con 0x56068c031fd0 2024-09-13T18:29:04.087 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.082+0000 7fa2577fe640 1 --2- 172.21.15.114:0/2914645894 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56068c031fd0 0x56068c0343f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:04.091 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.086+0000 7fa2577fe640 1 --2- 172.21.15.114:0/2914645894 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56068c031fd0 0x56068c0343f0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d74d6567d746f566 server_cookie=f315f7c8822904f0 in_seq=0 out_seq=0 2024-09-13T18:29:04.091 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.086+0000 7fa236ffd640 10 client.26969 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:29:04.096 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.090+0000 7fa236ffd640 1 -- 172.21.15.114:0/2914645894 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x56068bf6f5d0 con 0x56068c031fd0 2024-09-13T18:29:04.096 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.090+0000 7fa236ffd640 10 client.26969 handle_command_reply: tid=0 2024-09-13T18:29:04.096 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.090+0000 7fa260c6b740 1 client.26969 shutdown 2024-09-13T18:29:04.098 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.096+0000 7fa260c6b740 1 -- 172.21.15.114:0/2914645894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56068c017e20 msgr2=0x56068c01f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:04.098 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.096+0000 7fa260c6b740 1 --2- 172.21.15.114:0/2914645894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56068c017e20 0x56068c01f1b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa244005940 tx=0x7fa24400dc70 comp rx=0 tx=0).stop 2024-09-13T18:29:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.101+0000 7fa260c6b740 1 -- 172.21.15.114:0/2914645894 shutdown_connections 2024-09-13T18:29:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.101+0000 7fa260c6b740 1 --2- 172.21.15.114:0/2914645894 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56068c031fd0 0x56068c0343f0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.101+0000 7fa260c6b740 1 --2- 172.21.15.114:0/2914645894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56068c0226c0 0x56068c01f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.101+0000 7fa260c6b740 1 --2- 172.21.15.114:0/2914645894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56068c017e20 0x56068c01f1b0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.101+0000 7fa260c6b740 1 --2- 172.21.15.114:0/2914645894 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56068c017500 0x56068c021f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.101+0000 7fa260c6b740 1 -- 172.21.15.114:0/2914645894 >> 172.21.15.114:0/2914645894 conn(0x56068c013a90 msgr2=0x56068c012d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.101+0000 7fa236ffd640 0 client.26969 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:29:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.104+0000 7fa260c6b740 1 -- 172.21.15.114:0/2914645894 shutdown_connections 2024-09-13T18:29:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.109+0000 7fa260c6b740 1 -- 172.21.15.114:0/2914645894 wait complete. 2024-09-13T18:29:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.110+0000 7fa260c6b740 20 client.26969 trim_cache size 0 max 16384 2024-09-13T18:29:04.118 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:29:03 smithi134 ceph-mon[34283]: pgmap v200: 129 pgs: 129 active+clean; 20 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 31 MiB/s wr, 667 op/s 2024-09-13T18:29:04.148 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.146+0000 7fa260c6b740 1 Processor -- start 2024-09-13T18:29:04.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.146+0000 7fa260c6b740 1 -- start start 2024-09-13T18:29:04.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.147+0000 7fa260c6b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56068c017500 0x56068c021a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:04.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.147+0000 7fa260c6b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56068c017e20 0x56068c01a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:04.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.147+0000 7fa2577fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56068c017500 0x56068c021a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:04.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.147+0000 7fa2577fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56068c017500 0x56068c021a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:60640/0 (socket says 172.21.15.114:60640) 2024-09-13T18:29:04.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.147+0000 7fa2577fe640 1 -- 172.21.15.114:0/2714950719 learned_addr learned my addr 172.21.15.114:0/2714950719 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:04.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.147+0000 7fa260c6b740 1 --2- 172.21.15.114:0/2714950719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56068c0226c0 0x56068c01a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:04.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.148+0000 7fa256ffd640 1 --2- 172.21.15.114:0/2714950719 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56068c017e20 0x56068c01a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:04.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.148+0000 7fa260c6b740 1 -- 172.21.15.114:0/2714950719 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56068c0172a0 con 0x56068c017e20 2024-09-13T18:29:04.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.148+0000 7fa260c6b740 1 -- 172.21.15.114:0/2714950719 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56068c019f40 con 0x56068c0226c0 2024-09-13T18:29:04.151 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.148+0000 7fa260c6b740 1 -- 172.21.15.114:0/2714950719 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56068c00f480 con 0x56068c017500 2024-09-13T18:29:04.153 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.149+0000 7fa257fff640 1 --2- 172.21.15.114:0/2714950719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56068c0226c0 0x56068c01a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:04.153 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.150+0000 7fa2577fe640 1 -- 172.21.15.114:0/2714950719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56068c0226c0 msgr2=0x56068c01a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:04.153 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.150+0000 7fa2577fe640 1 --2- 172.21.15.114:0/2714950719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56068c0226c0 0x56068c01a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:04.153 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.150+0000 7fa2577fe640 1 -- 172.21.15.114:0/2714950719 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56068c017e20 msgr2=0x56068c01a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:04.153 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.150+0000 7fa2577fe640 1 --2- 172.21.15.114:0/2714950719 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56068c017e20 0x56068c01a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:04.153 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.150+0000 7fa2577fe640 1 -- 172.21.15.114:0/2714950719 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56068bf1c300 con 0x56068c017500 2024-09-13T18:29:04.153 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.151+0000 7fa2577fe640 1 --2- 172.21.15.114:0/2714950719 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56068c017500 0x56068c021a90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa24c006010 tx=0x7fa24c005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c205b88c031a53fa server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.153+0000 7fa2427fc640 1 -- 172.21.15.114:0/2714950719 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24c0b8e10 con 0x56068c017500 2024-09-13T18:29:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.153+0000 7fa2427fc640 1 -- 172.21.15.114:0/2714950719 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa24c0b7320 con 0x56068c017500 2024-09-13T18:29:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.154+0000 7fa260c6b740 1 -- 172.21.15.114:0/2714950719 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56068c017500 msgr2=0x56068c021a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.154+0000 7fa260c6b740 1 --2- 172.21.15.114:0/2714950719 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56068c017500 0x56068c021a90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa24c006010 tx=0x7fa24c005720 comp rx=0 tx=0).stop 2024-09-13T18:29:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.154+0000 7fa260c6b740 1 -- 172.21.15.114:0/2714950719 shutdown_connections 2024-09-13T18:29:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.154+0000 7fa260c6b740 1 --2- 172.21.15.114:0/2714950719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56068c0226c0 0x56068c01a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.154+0000 7fa260c6b740 1 --2- 172.21.15.114:0/2714950719 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56068c017e20 0x56068c01a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.154+0000 7fa260c6b740 1 --2- 172.21.15.114:0/2714950719 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56068c017500 0x56068c021a90 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.154+0000 7fa260c6b740 1 -- 172.21.15.114:0/2714950719 >> 172.21.15.114:0/2714950719 conn(0x56068c013a90 msgr2=0x56068c038d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.155+0000 7fa260c6b740 1 -- 172.21.15.114:0/2714950719 shutdown_connections 2024-09-13T18:29:04.158 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.155+0000 7fa260c6b740 1 -- 172.21.15.114:0/2714950719 wait complete. 2024-09-13T18:29:04.159 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.155+0000 7fa260c6b740 1 -- start start 2024-09-13T18:29:04.159 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.155+0000 7fa260c6b740 1 Processor -- start 2024-09-13T18:29:04.159 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.156+0000 7fa260c6b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56068c017e20 0x56068c027000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:04.159 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.156+0000 7fa260c6b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56068c0226c0 0x56068c027540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:04.159 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.156+0000 7fa260c6b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56068c027a80 0x56068c0874d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:04.159 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.156+0000 7fa260c6b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56068c087a90 con 0x56068c0226c0 2024-09-13T18:29:04.159 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.156+0000 7fa260c6b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56068c087c00 con 0x56068c017e20 2024-09-13T18:29:04.159 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.156+0000 7fa260c6b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56068c087dc0 con 0x56068c027a80 2024-09-13T18:29:04.159 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.156+0000 7fa260c6b740 10 client.0 _ll_register_callbacks cb 0x7fa24400b260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:29:04.159 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.156+0000 7fa257fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56068c027a80 0x56068c0874d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:04.159 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.156+0000 7fa257fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56068c027a80 0x56068c0874d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:60644/0 (socket says 172.21.15.114:60644) 2024-09-13T18:29:04.159 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.156+0000 7fa257fff640 1 -- 172.21.15.114:0/3893540762 learned_addr learned my addr 172.21.15.114:0/3893540762 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:04.159 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.156+0000 7fa2577fe640 1 --2- 172.21.15.114:0/3893540762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56068c017e20 0x56068c027000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:04.160 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.157+0000 7fa257fff640 1 -- 172.21.15.114:0/3893540762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56068c017e20 msgr2=0x56068c027000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:04.160 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.157+0000 7fa257fff640 1 --2- 172.21.15.114:0/3893540762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56068c017e20 0x56068c027000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:04.160 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.157+0000 7fa257fff640 1 -- 172.21.15.114:0/3893540762 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56068c0226c0 msgr2=0x56068c027540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:04.160 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.157+0000 7fa257fff640 1 --2- 172.21.15.114:0/3893540762 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56068c0226c0 0x56068c027540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:04.160 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.157+0000 7fa257fff640 1 -- 172.21.15.114:0/3893540762 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa25013f310 con 0x56068c027a80 2024-09-13T18:29:04.160 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.157+0000 7fa257fff640 1 --2- 172.21.15.114:0/3893540762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56068c027a80 0x56068c0874d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa258065740 tx=0x7fa258066570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f99714027512d66 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.159+0000 7fa236ffd640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:29:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.159+0000 7fa236ffd640 1 -- 172.21.15.114:0/3893540762 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa258085070 con 0x56068c027a80 2024-09-13T18:29:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.159+0000 7fa260c6b740 1 -- 172.21.15.114:0/3893540762 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56068bc6f300 con 0x56068c027a80 2024-09-13T18:29:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.159+0000 7fa236ffd640 1 -- 172.21.15.114:0/3893540762 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fa258073e40 con 0x56068c027a80 2024-09-13T18:29:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.159+0000 7fa236ffd640 1 -- 172.21.15.114:0/3893540762 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa258080b00 con 0x56068c027a80 2024-09-13T18:29:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.161+0000 7fa260c6b740 10 client.26974 fetch_fsmap learned FSMap version 34 2024-09-13T18:29:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.161+0000 7fa260c6b740 1 -- 172.21.15.114:0/3893540762 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x56068bf418e0 con 0x56068c027a80 2024-09-13T18:29:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.163+0000 7fa236ffd640 1 -- 172.21.15.114:0/3893540762 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56068bc6f300 con 0x56068c027a80 2024-09-13T18:29:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.163+0000 7fa236ffd640 1 -- 172.21.15.114:0/3893540762 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa25807c040 con 0x56068c027a80 2024-09-13T18:29:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.163+0000 7fa260c6b740 10 client.26974 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:29:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.163+0000 7fa260c6b740 10 client.26974 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:29:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.163+0000 7fa260c6b740 20 client.26974 populate_metadata read hostname 'smithi114' 2024-09-13T18:29:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.163+0000 7fa260c6b740 1 --2- 172.21.15.114:0/3893540762 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56068c0896f0 0x56068c08bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:04.166 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.163+0000 7fa260c6b740 4 client.26974 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T18:29:04.166 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.163+0000 7fa260c6b740 1 -- 172.21.15.114:0/3893540762 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56068bc80600 con 0x56068c0896f0 2024-09-13T18:29:04.168 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.165+0000 7fa2577fe640 1 --2- 172.21.15.114:0/3893540762 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56068c0896f0 0x56068c08bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:04.168 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.166+0000 7fa236ffd640 10 client.26974 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:29:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.166+0000 7fa2577fe640 1 --2- 172.21.15.114:0/3893540762 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56068c0896f0 0x56068c08bb10 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d854f89203e754f server_cookie=c072ef89f1cff023 in_seq=0 out_seq=0 2024-09-13T18:29:04.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:29:03 smithi204 ceph-mon[34777]: pgmap v200: 129 pgs: 129 active+clean; 20 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 31 MiB/s wr, 667 op/s 2024-09-13T18:29:04.204 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.202+0000 7fa236ffd640 1 -- 172.21.15.114:0/3893540762 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56068bc80600 con 0x56068c0896f0 2024-09-13T18:29:04.204 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.202+0000 7fa236ffd640 10 client.26974 handle_command_reply: tid=0 2024-09-13T18:29:04.204 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.202+0000 7fa260c6b740 1 client.26974 shutdown 2024-09-13T18:29:04.206 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.204+0000 7fa260c6b740 1 -- 172.21.15.114:0/3893540762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56068c027a80 msgr2=0x56068c0874d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:04.206 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.204+0000 7fa260c6b740 1 --2- 172.21.15.114:0/3893540762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56068c027a80 0x56068c0874d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa258065740 tx=0x7fa258066570 comp rx=0 tx=0).stop 2024-09-13T18:29:04.209 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.207+0000 7fa260c6b740 1 -- 172.21.15.114:0/3893540762 shutdown_connections 2024-09-13T18:29:04.209 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.207+0000 7fa260c6b740 1 --2- 172.21.15.114:0/3893540762 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56068c0896f0 0x56068c08bb10 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:04.209 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.207+0000 7fa260c6b740 1 --2- 172.21.15.114:0/3893540762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56068c027a80 0x56068c0874d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:04.209 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.207+0000 7fa260c6b740 1 --2- 172.21.15.114:0/3893540762 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56068c0226c0 0x56068c027540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:04.209 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.207+0000 7fa260c6b740 1 --2- 172.21.15.114:0/3893540762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56068c017e20 0x56068c027000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:04.209 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.207+0000 7fa260c6b740 1 -- 172.21.15.114:0/3893540762 >> 172.21.15.114:0/3893540762 conn(0x56068c013a90 msgr2=0x56068c01a460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:04.210 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.207+0000 7fa236ffd640 0 client.26974 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:29:04.213 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.211+0000 7fa260c6b740 1 -- 172.21.15.114:0/3893540762 shutdown_connections 2024-09-13T18:29:04.214 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.211+0000 7fa260c6b740 1 -- 172.21.15.114:0/3893540762 wait complete. 2024-09-13T18:29:04.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:29:03 smithi114 ceph-mon[30897]: pgmap v200: 129 pgs: 129 active+clean; 20 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 31 MiB/s wr, 667 op/s 2024-09-13T18:29:04.221 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-13T18:29:04.221 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-13T18:29:04.221 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-13T18:29:04.221 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-13T18:29:04.221 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.217+0000 7fa260c6b740 20 client.26974 trim_cache size 0 max 16384 2024-09-13T18:29:04.221 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.217+0000 7fa25de6b640 1 -- 172.21.15.114:0/2464567135 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa2480a1410 msgr2=0x7fa2480a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:04.221 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.217+0000 7fa25de6b640 1 --2- 172.21.15.114:0/2464567135 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa2480a1410 0x7fa2480a38d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa25804f760 tx=0x7fa25804f270 comp rx=0 tx=0).stop 2024-09-13T18:29:04.222 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.217+0000 7fa25de6b640 1 -- 172.21.15.114:0/2464567135 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2500a4a30 msgr2=0x7fa2500b3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:04.222 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.217+0000 7fa25de6b640 1 --2- 172.21.15.114:0/2464567135 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2500a4a30 0x7fa2500b3de0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fa24c02a9b0 tx=0x7fa24c0055f0 comp rx=0 tx=0).stop 2024-09-13T18:29:04.223 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.220+0000 7fa25de6b640 1 -- 172.21.15.114:0/2464567135 shutdown_connections 2024-09-13T18:29:04.223 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.220+0000 7fa25de6b640 1 --2- 172.21.15.114:0/2464567135 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa2480a1410 0x7fa2480a38d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:04.223 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.220+0000 7fa25de6b640 1 --2- 172.21.15.114:0/2464567135 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2500b4860 0x7fa25014f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:04.223 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.220+0000 7fa25de6b640 1 --2- 172.21.15.114:0/2464567135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2500acb80 0x7fa2500b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:04.223 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.220+0000 7fa25de6b640 1 --2- 172.21.15.114:0/2464567135 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2500a4a30 0x7fa2500b3de0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:04.223 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.220+0000 7fa25de6b640 1 -- 172.21.15.114:0/2464567135 >> 172.21.15.114:0/2464567135 conn(0x7fa25001a0a0 msgr2=0x7fa2500aa730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:04.223 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.220+0000 7fa25de6b640 1 -- 172.21.15.114:0/2464567135 shutdown_connections 2024-09-13T18:29:04.223 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:04.220+0000 7fa25de6b640 1 -- 172.21.15.114:0/2464567135 wait complete. 2024-09-13T18:29:06.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:29:05 smithi134 ceph-mon[34283]: pgmap v201: 129 pgs: 129 active+clean; 20 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 36 MiB/s wr, 770 op/s 2024-09-13T18:29:06.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:29:05 smithi204 ceph-mon[34777]: pgmap v201: 129 pgs: 129 active+clean; 20 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 36 MiB/s wr, 770 op/s 2024-09-13T18:29:06.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:29:05 smithi114 ceph-mon[30897]: pgmap v201: 129 pgs: 129 active+clean; 20 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 36 MiB/s wr, 770 op/s 2024-09-13T18:29:07.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:29:06 smithi134 ceph-mon[34283]: scrub summary: idle 2024-09-13T18:29:07.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:29:06 smithi134 ceph-mon[34283]: scrub summary: idle 2024-09-13T18:29:07.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:29:06 smithi204 ceph-mon[34777]: scrub summary: idle 2024-09-13T18:29:07.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:29:06 smithi204 ceph-mon[34777]: scrub summary: idle 2024-09-13T18:29:07.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:29:06 smithi114 ceph-mon[30897]: scrub summary: idle 2024-09-13T18:29:07.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:29:06 smithi114 ceph-mon[30897]: scrub summary: idle 2024-09-13T18:29:08.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:29:07 smithi134 ceph-mon[34283]: pgmap v202: 129 pgs: 129 active+clean; 20 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 40 MiB/s wr, 880 op/s 2024-09-13T18:29:08.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:29:07 smithi134 ceph-mon[34283]: scrub summary: idle 2024-09-13T18:29:08.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:29:07 smithi134 ceph-mon[34283]: osdmap e95: 12 total, 12 up, 12 in 2024-09-13T18:29:08.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:29:07 smithi204 ceph-mon[34777]: pgmap v202: 129 pgs: 129 active+clean; 20 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 40 MiB/s wr, 880 op/s 2024-09-13T18:29:08.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:29:07 smithi204 ceph-mon[34777]: scrub summary: idle 2024-09-13T18:29:08.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:29:07 smithi204 ceph-mon[34777]: osdmap e95: 12 total, 12 up, 12 in 2024-09-13T18:29:08.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:29:07 smithi114 ceph-mon[30897]: pgmap v202: 129 pgs: 129 active+clean; 20 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 40 MiB/s wr, 880 op/s 2024-09-13T18:29:08.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:29:07 smithi114 ceph-mon[30897]: scrub summary: idle 2024-09-13T18:29:08.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:29:07 smithi114 ceph-mon[30897]: osdmap e95: 12 total, 12 up, 12 in 2024-09-13T18:29:10.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:29:09 smithi134 ceph-mon[34283]: pgmap v204: 129 pgs: 129 active+clean; 20 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 36 MiB/s wr, 790 op/s 2024-09-13T18:29:10.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:29:09 smithi204 ceph-mon[34777]: pgmap v204: 129 pgs: 129 active+clean; 20 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 36 MiB/s wr, 790 op/s 2024-09-13T18:29:10.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:29:09 smithi114 ceph-mon[30897]: pgmap v204: 129 pgs: 129 active+clean; 20 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 36 MiB/s wr, 790 op/s 2024-09-13T18:29:12.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:29:11 smithi134 ceph-mon[34283]: pgmap v205: 129 pgs: 129 active+clean; 20 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 34 MiB/s wr, 767 op/s 2024-09-13T18:29:12.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:29:11 smithi204 ceph-mon[34777]: pgmap v205: 129 pgs: 129 active+clean; 20 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 34 MiB/s wr, 767 op/s 2024-09-13T18:29:12.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:29:11 smithi114 ceph-mon[30897]: pgmap v205: 129 pgs: 129 active+clean; 20 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 34 MiB/s wr, 767 op/s 2024-09-13T18:29:14.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:29:13 smithi134 ceph-mon[34283]: pgmap v206: 129 pgs: 129 active+clean; 20 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 34 MiB/s wr, 767 op/s 2024-09-13T18:29:14.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:29:13 smithi134 ceph-mon[34283]: osdmap e96: 12 total, 12 up, 12 in 2024-09-13T18:29:14.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:29:13 smithi204 ceph-mon[34777]: pgmap v206: 129 pgs: 129 active+clean; 20 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 34 MiB/s wr, 767 op/s 2024-09-13T18:29:14.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:29:13 smithi204 ceph-mon[34777]: osdmap e96: 12 total, 12 up, 12 in 2024-09-13T18:29:14.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:29:13 smithi114 ceph-mon[30897]: pgmap v206: 129 pgs: 129 active+clean; 20 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 34 MiB/s wr, 767 op/s 2024-09-13T18:29:14.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:29:13 smithi114 ceph-mon[30897]: osdmap e96: 12 total, 12 up, 12 in 2024-09-13T18:29:16.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:29:15 smithi134 ceph-mon[34283]: pgmap v208: 129 pgs: 129 active+clean; 20 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 29 MiB/s wr, 637 op/s 2024-09-13T18:29:16.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:29:15 smithi204 ceph-mon[34777]: pgmap v208: 129 pgs: 129 active+clean; 20 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 29 MiB/s wr, 637 op/s 2024-09-13T18:29:16.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:29:15 smithi114 ceph-mon[30897]: pgmap v208: 129 pgs: 129 active+clean; 20 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 29 MiB/s wr, 637 op/s 2024-09-13T18:29:17.409 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.402+0000 7f748488b640 1 -- 172.21.15.114:0/2757142677 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7480119400 msgr2=0x7f748011b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:17.409 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.402+0000 7f748488b640 1 --2- 172.21.15.114:0/2757142677 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7480119400 0x7f748011b7f0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f7478009ca0 tx=0x7f747802a8f0 comp rx=0 tx=0).stop 2024-09-13T18:29:17.409 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.403+0000 7f748488b640 1 -- 172.21.15.114:0/2757142677 shutdown_connections 2024-09-13T18:29:17.409 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.403+0000 7f748488b640 1 --2- 172.21.15.114:0/2757142677 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7480119400 0x7f748011b7f0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:17.409 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.403+0000 7f748488b640 1 --2- 172.21.15.114:0/2757142677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7480107890 0x7f7480107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:17.409 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.403+0000 7f748488b640 1 --2- 172.21.15.114:0/2757142677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74801074b0 0x7f748010e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.403+0000 7f748488b640 1 -- 172.21.15.114:0/2757142677 >> 172.21.15.114:0/2757142677 conn(0x7f748006af50 msgr2=0x7f748006b360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.403+0000 7f748488b640 1 -- 172.21.15.114:0/2757142677 shutdown_connections 2024-09-13T18:29:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.403+0000 7f748488b640 1 -- 172.21.15.114:0/2757142677 wait complete. 2024-09-13T18:29:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.404+0000 7f748488b640 1 Processor -- start 2024-09-13T18:29:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.404+0000 7f748488b640 1 -- start start 2024-09-13T18:29:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.404+0000 7f748488b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74801074b0 0x7f74801a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.404+0000 7f748488b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7480107890 0x7f74801a55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.404+0000 7f748488b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7480119400 0x7f74801a99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.404+0000 7f748488b640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74801a9fd0 con 0x7f7480107890 2024-09-13T18:29:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.405+0000 7f748488b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74801aa110 con 0x7f74801074b0 2024-09-13T18:29:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.405+0000 7f748488b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74801aa2d0 con 0x7f7480119400 2024-09-13T18:29:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.405+0000 7f747e59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74801074b0 0x7f74801a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.405+0000 7f747e59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74801074b0 0x7f74801a50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:58352/0 (socket says 172.21.15.114:58352) 2024-09-13T18:29:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.405+0000 7f747e59c640 1 -- 172.21.15.114:0/2751035611 learned_addr learned my addr 172.21.15.114:0/2751035611 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.406+0000 7f747dd9b640 1 --2- 172.21.15.114:0/2751035611 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7480107890 0x7f74801a55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.406+0000 7f747ed9d640 1 --2- 172.21.15.114:0/2751035611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7480119400 0x7f74801a99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.406+0000 7f747dd9b640 1 -- 172.21.15.114:0/2751035611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7480119400 msgr2=0x7f74801a99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.406+0000 7f747dd9b640 1 --2- 172.21.15.114:0/2751035611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7480119400 0x7f74801a99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.407+0000 7f747dd9b640 1 -- 172.21.15.114:0/2751035611 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74801074b0 msgr2=0x7f74801a50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.407+0000 7f747dd9b640 1 --2- 172.21.15.114:0/2751035611 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74801074b0 0x7f74801a50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.407+0000 7f747dd9b640 1 -- 172.21.15.114:0/2751035611 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7478009900 con 0x7f7480107890 2024-09-13T18:29:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.407+0000 7f747dd9b640 1 --2- 172.21.15.114:0/2751035611 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7480107890 0x7f74801a55e0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f747000b3f0 tx=0x7f747000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.409+0000 7f74777fe640 1 -- 172.21.15.114:0/2751035611 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7470011830 con 0x7f7480107890 2024-09-13T18:29:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.409+0000 7f74777fe640 1 -- 172.21.15.114:0/2751035611 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7470011e20 con 0x7f7480107890 2024-09-13T18:29:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.409+0000 7f74777fe640 1 -- 172.21.15.114:0/2751035611 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7470010e10 con 0x7f7480107890 2024-09-13T18:29:17.413 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.410+0000 7f748488b640 1 -- 172.21.15.114:0/2751035611 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74801aa5b0 con 0x7f7480107890 2024-09-13T18:29:17.413 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.410+0000 7f748488b640 1 -- 172.21.15.114:0/2751035611 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f748010edc0 con 0x7f7480107890 2024-09-13T18:29:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.413+0000 7f748768a740 1 Processor -- start 2024-09-13T18:29:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.413+0000 7f748768a740 1 -- start start 2024-09-13T18:29:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.414+0000 7f748768a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fdf8a3330 0x555fdf8a3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.414+0000 7f748768a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fdf8a3c50 0x555fdf8adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.414+0000 7f748768a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555fdf8ae4f0 0x555fdf8b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.414+0000 7f748768a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fdf80ec30 con 0x555fdf8ae4f0 2024-09-13T18:29:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.414+0000 7f748768a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fdf80c3c0 con 0x555fdf8a3330 2024-09-13T18:29:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.414+0000 7f748768a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fdf807cb0 con 0x555fdf8a3c50 2024-09-13T18:29:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.414+0000 7f747ed9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fdf8a3330 0x555fdf8a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.414+0000 7f747ed9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fdf8a3330 0x555fdf8a3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:58356/0 (socket says 172.21.15.114:58356) 2024-09-13T18:29:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.414+0000 7f747ed9d640 1 -- 172.21.15.114:0/2086696727 learned_addr learned my addr 172.21.15.114:0/2086696727 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.415+0000 7f747e59c640 1 --2- 172.21.15.114:0/2086696727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fdf8a3c50 0x555fdf8adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.416+0000 7f747dd9b640 1 --2- 172.21.15.114:0/2086696727 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555fdf8ae4f0 0x555fdf8b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.417+0000 7f747ed9d640 1 -- 172.21.15.114:0/2086696727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fdf8a3c50 msgr2=0x555fdf8adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.417+0000 7f747ed9d640 1 --2- 172.21.15.114:0/2086696727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fdf8a3c50 0x555fdf8adfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.417+0000 7f747ed9d640 1 -- 172.21.15.114:0/2086696727 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555fdf8ae4f0 msgr2=0x555fdf8b08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.417+0000 7f747ed9d640 1 --2- 172.21.15.114:0/2086696727 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555fdf8ae4f0 0x555fdf8b08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.417+0000 7f747ed9d640 1 -- 172.21.15.114:0/2086696727 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555fdf7ae910 con 0x555fdf8a3330 2024-09-13T18:29:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.417+0000 7f747ed9d640 1 --2- 172.21.15.114:0/2086696727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fdf8a3330 0x555fdf8a3710 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7478009ca0 tx=0x7f74780306a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28e0e2dde519db67 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.417+0000 7f74777fe640 1 -- 172.21.15.114:0/2751035611 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f7470011990 con 0x7f7480107890 2024-09-13T18:29:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.418+0000 7f74777fe640 1 --2- 172.21.15.114:0/2751035611 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f746c0a1410 0x7f746c0a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.418+0000 7f747e59c640 1 --2- 172.21.15.114:0/2086696727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fdf8a3c50 0x555fdf8adfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:29:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.419+0000 7f7474ff9640 1 -- 172.21.15.114:0/2086696727 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7478030bf0 con 0x555fdf8a3330 2024-09-13T18:29:17.423 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.419+0000 7f7474ff9640 1 -- 172.21.15.114:0/2086696727 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7478003970 con 0x555fdf8a3330 2024-09-13T18:29:17.423 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.419+0000 7f7474ff9640 1 -- 172.21.15.114:0/2086696727 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7478031970 con 0x555fdf8a3330 2024-09-13T18:29:17.423 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.419+0000 7f747e59c640 1 --2- 172.21.15.114:0/2751035611 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f746c0a1410 0x7f746c0a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:17.423 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.420+0000 7f748768a740 1 -- 172.21.15.114:0/2086696727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fdf8a3330 msgr2=0x555fdf8a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:17.423 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.420+0000 7f748768a740 1 --2- 172.21.15.114:0/2086696727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fdf8a3330 0x555fdf8a3710 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7478009ca0 tx=0x7f74780306a0 comp rx=0 tx=0).stop 2024-09-13T18:29:17.424 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.420+0000 7f748768a740 1 -- 172.21.15.114:0/2086696727 shutdown_connections 2024-09-13T18:29:17.424 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.420+0000 7f748768a740 1 --2- 172.21.15.114:0/2086696727 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555fdf8ae4f0 0x555fdf8b08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:17.424 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.420+0000 7f748768a740 1 --2- 172.21.15.114:0/2086696727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fdf8a3c50 0x555fdf8adfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:17.424 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.420+0000 7f748768a740 1 --2- 172.21.15.114:0/2086696727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fdf8a3330 0x555fdf8a3710 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:17.424 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.420+0000 7f748768a740 1 -- 172.21.15.114:0/2086696727 >> 172.21.15.114:0/2086696727 conn(0x555fdf89f8c0 msgr2=0x555fdf89fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:17.424 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.420+0000 7f74777fe640 1 -- 172.21.15.114:0/2751035611 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f74700cea30 con 0x7f7480107890 2024-09-13T18:29:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.421+0000 7f747e59c640 1 --2- 172.21.15.114:0/2751035611 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f746c0a1410 0x7f746c0a38d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x555fdf89f3c0 tx=0x7f7468008040 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.424+0000 7f748768a740 1 -- 172.21.15.114:0/2086696727 shutdown_connections 2024-09-13T18:29:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.424+0000 7f748768a740 1 -- 172.21.15.114:0/2086696727 wait complete. 2024-09-13T18:29:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.424+0000 7f748768a740 1 -- start start 2024-09-13T18:29:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.424+0000 7f748768a740 1 Processor -- start 2024-09-13T18:29:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.425+0000 7f748768a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555fdf8a3330 0x555fdf8ade30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.425+0000 7f748768a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fdf8a3c50 0x555fdf8ab170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.425+0000 7f748768a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fdf8ae4f0 0x555fdf8ab6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.425+0000 7f748768a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fdf7c5e80 con 0x555fdf8a3330 2024-09-13T18:29:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.425+0000 7f748768a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fdf7c5b20 con 0x555fdf8a3c50 2024-09-13T18:29:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.425+0000 7f748768a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fdf78b260 con 0x555fdf8ae4f0 2024-09-13T18:29:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.425+0000 7f748768a740 10 client.0 _ll_register_callbacks cb 0x555fdf7c0df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:29:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.425+0000 7f747ed9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fdf8ae4f0 0x555fdf8ab6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.425+0000 7f747ed9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fdf8ae4f0 0x555fdf8ab6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:41660/0 (socket says 172.21.15.114:41660) 2024-09-13T18:29:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.425+0000 7f747ed9d640 1 -- 172.21.15.114:0/219181071 learned_addr learned my addr 172.21.15.114:0/219181071 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.425+0000 7f747dd9b640 1 --2- 172.21.15.114:0/219181071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fdf8a3c50 0x555fdf8ab170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.426+0000 7f747ed9d640 1 -- 172.21.15.114:0/219181071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fdf8a3c50 msgr2=0x555fdf8ab170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.426+0000 7f747ed9d640 1 --2- 172.21.15.114:0/219181071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fdf8a3c50 0x555fdf8ab170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.426+0000 7f747ed9d640 1 -- 172.21.15.114:0/219181071 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555fdf8a3330 msgr2=0x555fdf8ade30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:29:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.426+0000 7f747ed9d640 1 --2- 172.21.15.114:0/219181071 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555fdf8a3330 0x555fdf8ade30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.426+0000 7f747ed9d640 1 -- 172.21.15.114:0/219181071 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f748011c230 con 0x555fdf8ae4f0 2024-09-13T18:29:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.426+0000 7f747ed9d640 1 --2- 172.21.15.114:0/219181071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fdf8ae4f0 0x555fdf8ab6b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7478009bf0 tx=0x7f7478031da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f1086410f7ac55d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.426+0000 7f7465ffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:29:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.426+0000 7f7465ffb640 1 -- 172.21.15.114:0/219181071 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7478003db0 con 0x555fdf8ae4f0 2024-09-13T18:29:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.428+0000 7f7465ffb640 1 -- 172.21.15.114:0/219181071 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f7478031670 con 0x555fdf8ae4f0 2024-09-13T18:29:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.429+0000 7f748768a740 1 -- 172.21.15.114:0/219181071 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555fdf830680 con 0x555fdf8ae4f0 2024-09-13T18:29:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.430+0000 7f7465ffb640 1 -- 172.21.15.114:0/219181071 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7478031830 con 0x555fdf8ae4f0 2024-09-13T18:29:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.432+0000 7f748768a740 10 client.26980 fetch_fsmap learned FSMap version 34 2024-09-13T18:29:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.432+0000 7f748768a740 1 -- 172.21.15.114:0/219181071 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x555fdf806330 con 0x555fdf8ae4f0 2024-09-13T18:29:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.432+0000 7f7465ffb640 1 -- 172.21.15.114:0/219181071 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x555fdf830680 con 0x555fdf8ae4f0 2024-09-13T18:29:17.437 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.434+0000 7f7465ffb640 1 -- 172.21.15.114:0/219181071 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f747807a070 con 0x555fdf8ae4f0 2024-09-13T18:29:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.434+0000 7f748768a740 10 client.26980 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:29:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.434+0000 7f748768a740 10 client.26980 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:29:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.434+0000 7f748768a740 20 client.26980 populate_metadata read hostname 'smithi114' 2024-09-13T18:29:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.434+0000 7f748768a740 1 --2- 172.21.15.114:0/219181071 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x555fdf8bd9c0 0x555fdf8bfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.434+0000 7f748768a740 4 client.26980 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:29:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.434+0000 7f748768a740 1 -- 172.21.15.114:0/219181071 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555fdf7fb5d0 con 0x555fdf8bd9c0 2024-09-13T18:29:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.437+0000 7f747e59c640 1 --2- 172.21.15.114:0/219181071 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x555fdf8bd9c0 0x555fdf8bfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.439+0000 7f747e59c640 1 --2- 172.21.15.114:0/219181071 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x555fdf8bd9c0 0x555fdf8bfdb0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25f3c680370b8b3a server_cookie=6fb169b51261e822 in_seq=0 out_seq=0 2024-09-13T18:29:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.439+0000 7f7465ffb640 10 client.26980 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:29:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.443+0000 7f7465ffb640 1 -- 172.21.15.114:0/219181071 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x555fdf7fb5d0 con 0x555fdf8bd9c0 2024-09-13T18:29:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.443+0000 7f7465ffb640 10 client.26980 handle_command_reply: tid=0 2024-09-13T18:29:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.443+0000 7f748768a740 1 client.26980 shutdown 2024-09-13T18:29:17.449 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.445+0000 7f748768a740 1 -- 172.21.15.114:0/219181071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fdf8ae4f0 msgr2=0x555fdf8ab6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:17.449 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.445+0000 7f748768a740 1 --2- 172.21.15.114:0/219181071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fdf8ae4f0 0x555fdf8ab6b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7478009bf0 tx=0x7f7478031da0 comp rx=0 tx=0).stop 2024-09-13T18:29:17.449 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.445+0000 7f748768a740 1 -- 172.21.15.114:0/219181071 shutdown_connections 2024-09-13T18:29:17.449 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.446+0000 7f748768a740 1 --2- 172.21.15.114:0/219181071 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x555fdf8bd9c0 0x555fdf8bfdb0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:17.449 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.446+0000 7f748768a740 1 --2- 172.21.15.114:0/219181071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fdf8ae4f0 0x555fdf8ab6b0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:17.449 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.446+0000 7f748768a740 1 --2- 172.21.15.114:0/219181071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fdf8a3c50 0x555fdf8ab170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:17.449 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.446+0000 7f748768a740 1 --2- 172.21.15.114:0/219181071 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555fdf8a3330 0x555fdf8ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:17.449 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.446+0000 7f748768a740 1 -- 172.21.15.114:0/219181071 >> 172.21.15.114:0/219181071 conn(0x555fdf89f8c0 msgr2=0x555fdf89eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:17.449 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.446+0000 7f7465ffb640 0 client.26980 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:29:17.451 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.448+0000 7f748768a740 1 -- 172.21.15.114:0/219181071 shutdown_connections 2024-09-13T18:29:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.450+0000 7f748768a740 1 -- 172.21.15.114:0/219181071 wait complete. 2024-09-13T18:29:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.456+0000 7f748768a740 20 client.26980 trim_cache size 0 max 16384 2024-09-13T18:29:17.506 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.503+0000 7f748768a740 1 Processor -- start 2024-09-13T18:29:17.508 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.505+0000 7f748768a740 1 -- start start 2024-09-13T18:29:17.509 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.506+0000 7f748768a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fdf8a3330 0x555fdf8b5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:17.509 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.506+0000 7f748768a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fdf8a3c50 0x555fdf8b57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:17.510 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.506+0000 7f748768a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555fdf8ae4f0 0x555fdf8ad830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:17.510 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.507+0000 7f748768a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fdf8bac10 con 0x555fdf8ae4f0 2024-09-13T18:29:17.510 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.507+0000 7f748768a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fdf8a4850 con 0x555fdf8a3c50 2024-09-13T18:29:17.510 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.507+0000 7f748768a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fdf8aba20 con 0x555fdf8a3330 2024-09-13T18:29:17.511 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.508+0000 7f747dd9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fdf8a3c50 0x555fdf8b57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:17.511 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.508+0000 7f747dd9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fdf8a3c50 0x555fdf8b57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:58372/0 (socket says 172.21.15.114:58372) 2024-09-13T18:29:17.511 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.508+0000 7f747dd9b640 1 -- 172.21.15.114:0/560056889 learned_addr learned my addr 172.21.15.114:0/560056889 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:17.511 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.508+0000 7f747e59c640 1 --2- 172.21.15.114:0/560056889 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fdf8a3330 0x555fdf8b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:17.514 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.511+0000 7f747dd9b640 1 -- 172.21.15.114:0/560056889 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fdf8a3330 msgr2=0x555fdf8b5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:17.514 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.511+0000 7f747dd9b640 1 --2- 172.21.15.114:0/560056889 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fdf8a3330 0x555fdf8b5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:17.514 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.511+0000 7f747dd9b640 1 -- 172.21.15.114:0/560056889 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555fdf8ae4f0 msgr2=0x555fdf8ad830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:29:17.514 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.511+0000 7f747dd9b640 1 --2- 172.21.15.114:0/560056889 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555fdf8ae4f0 0x555fdf8ad830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:17.514 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.511+0000 7f747dd9b640 1 -- 172.21.15.114:0/560056889 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555fdf7a8300 con 0x555fdf8a3c50 2024-09-13T18:29:17.515 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.512+0000 7f747dd9b640 1 --2- 172.21.15.114:0/560056889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fdf8a3c50 0x555fdf8b57c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7470097d50 tx=0x7f7470011f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7af7a2744408e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:17.516 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.513+0000 7f74757fa640 1 -- 172.21.15.114:0/560056889 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7470010510 con 0x555fdf8a3c50 2024-09-13T18:29:17.516 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.513+0000 7f74757fa640 1 -- 172.21.15.114:0/560056889 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7470011c40 con 0x555fdf8a3c50 2024-09-13T18:29:17.516 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.513+0000 7f74757fa640 1 -- 172.21.15.114:0/560056889 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747009f680 con 0x555fdf8a3c50 2024-09-13T18:29:17.519 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.516+0000 7f748768a740 1 -- 172.21.15.114:0/560056889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fdf8a3c50 msgr2=0x555fdf8b57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:17.520 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.516+0000 7f748768a740 1 --2- 172.21.15.114:0/560056889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fdf8a3c50 0x555fdf8b57c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7470097d50 tx=0x7f7470011f90 comp rx=0 tx=0).stop 2024-09-13T18:29:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.517+0000 7f748768a740 1 -- 172.21.15.114:0/560056889 shutdown_connections 2024-09-13T18:29:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.517+0000 7f748768a740 1 --2- 172.21.15.114:0/560056889 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555fdf8ae4f0 0x555fdf8ad830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.517+0000 7f748768a740 1 --2- 172.21.15.114:0/560056889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fdf8a3c50 0x555fdf8b57c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.517+0000 7f748768a740 1 --2- 172.21.15.114:0/560056889 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fdf8a3330 0x555fdf8b5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.517+0000 7f748768a740 1 -- 172.21.15.114:0/560056889 >> 172.21.15.114:0/560056889 conn(0x555fdf89f8c0 msgr2=0x555fdf8a9a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.517+0000 7f748768a740 1 -- 172.21.15.114:0/560056889 shutdown_connections 2024-09-13T18:29:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.518+0000 7f748768a740 1 -- 172.21.15.114:0/560056889 wait complete. 2024-09-13T18:29:17.522 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.518+0000 7f748768a740 1 -- start start 2024-09-13T18:29:17.522 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.518+0000 7f748768a740 1 Processor -- start 2024-09-13T18:29:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.519+0000 7f748768a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fdf8a3330 0x555fdf8b2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.519+0000 7f748768a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555fdf8a3c50 0x555fdf8b2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.519+0000 7f748768a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fdf8ae4f0 0x555fdf913e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.519+0000 7f747dd9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555fdf8a3c50 0x555fdf8b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.520+0000 7f748768a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fdf8b2e90 con 0x555fdf8a3c50 2024-09-13T18:29:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.520+0000 7f748768a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fdf914360 con 0x555fdf8ae4f0 2024-09-13T18:29:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.520+0000 7f748768a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fdf9144a0 con 0x555fdf8a3330 2024-09-13T18:29:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.520+0000 7f747dd9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555fdf8a3c50 0x555fdf8b2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59316/0 (socket says 172.21.15.114:59316) 2024-09-13T18:29:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.520+0000 7f748768a740 10 client.0 _ll_register_callbacks cb 0x7f74780097e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:29:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.520+0000 7f747dd9b640 1 -- 172.21.15.114:0/3278694346 learned_addr learned my addr 172.21.15.114:0/3278694346 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.521+0000 7f747dd9b640 1 -- 172.21.15.114:0/3278694346 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fdf8a3330 msgr2=0x555fdf8b2000 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:29:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.521+0000 7f747dd9b640 1 --2- 172.21.15.114:0/3278694346 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fdf8a3330 0x555fdf8b2000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:17.525 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.521+0000 7f747dd9b640 1 -- 172.21.15.114:0/3278694346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fdf8ae4f0 msgr2=0x555fdf913e20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:29:17.525 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.521+0000 7f747dd9b640 1 --2- 172.21.15.114:0/3278694346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fdf8ae4f0 0x555fdf913e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:17.525 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.521+0000 7f747dd9b640 1 -- 172.21.15.114:0/3278694346 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f748010edc0 con 0x555fdf8a3c50 2024-09-13T18:29:17.525 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.521+0000 7f747dd9b640 1 --2- 172.21.15.114:0/3278694346 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555fdf8a3c50 0x555fdf8b2540 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f74700a37a0 tx=0x7f7470000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb0fbb2abf563a90 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:17.526 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.523+0000 7f7465ffb640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:29:17.527 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.523+0000 7f7465ffb640 1 -- 172.21.15.114:0/3278694346 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7470010510 con 0x555fdf8a3c50 2024-09-13T18:29:17.527 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.524+0000 7f7465ffb640 1 -- 172.21.15.114:0/3278694346 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f74700b9e40 con 0x555fdf8a3c50 2024-09-13T18:29:17.527 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.524+0000 7f748768a740 1 -- 172.21.15.114:0/3278694346 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7478003970 con 0x555fdf8a3c50 2024-09-13T18:29:17.530 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.527+0000 7f7465ffb640 1 -- 172.21.15.114:0/3278694346 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f747009bf50 con 0x555fdf8a3c50 2024-09-13T18:29:17.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.528+0000 7f748768a740 10 client.17562 fetch_fsmap learned FSMap version 34 2024-09-13T18:29:17.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.528+0000 7f748768a740 1 -- 172.21.15.114:0/3278694346 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x555fdf7cd8e0 con 0x555fdf8a3c50 2024-09-13T18:29:17.534 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.531+0000 7f7465ffb640 1 -- 172.21.15.114:0/3278694346 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7478003970 con 0x555fdf8a3c50 2024-09-13T18:29:17.534 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.531+0000 7f7465ffb640 1 -- 172.21.15.114:0/3278694346 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f74700a3840 con 0x555fdf8a3c50 2024-09-13T18:29:17.537 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.532+0000 7f748768a740 10 client.17562 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:29:17.537 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.532+0000 7f748768a740 10 client.17562 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:29:17.537 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.532+0000 7f748768a740 20 client.17562 populate_metadata read hostname 'smithi114' 2024-09-13T18:29:17.537 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.532+0000 7f748768a740 1 --2- 172.21.15.114:0/3278694346 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x555fdf915770 0x555fdf917b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:17.537 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.532+0000 7f748768a740 4 client.17562 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T18:29:17.537 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.532+0000 7f748768a740 1 -- 172.21.15.114:0/3278694346 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555fdf4fb300 con 0x555fdf915770 2024-09-13T18:29:17.538 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.535+0000 7f747e59c640 1 --2- 172.21.15.114:0/3278694346 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x555fdf915770 0x555fdf917b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.537+0000 7f747e59c640 1 --2- 172.21.15.114:0/3278694346 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x555fdf915770 0x555fdf917b90 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb022f6eacea501a server_cookie=c04afdde759951e7 in_seq=0 out_seq=0 2024-09-13T18:29:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.538+0000 7f7465ffb640 10 client.17562 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:29:17.576 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.573+0000 7f7465ffb640 1 -- 172.21.15.114:0/3278694346 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555fdf4fb300 con 0x555fdf915770 2024-09-13T18:29:17.576 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.573+0000 7f7465ffb640 10 client.17562 handle_command_reply: tid=0 2024-09-13T18:29:17.576 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.573+0000 7f748768a740 1 client.17562 shutdown 2024-09-13T18:29:17.577 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.573+0000 7f748768a740 1 -- 172.21.15.114:0/3278694346 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555fdf8a3c50 msgr2=0x555fdf8b2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:17.577 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.573+0000 7f748768a740 1 --2- 172.21.15.114:0/3278694346 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555fdf8a3c50 0x555fdf8b2540 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f74700a37a0 tx=0x7f7470000bc0 comp rx=0 tx=0).stop 2024-09-13T18:29:17.581 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.578+0000 7f748768a740 1 -- 172.21.15.114:0/3278694346 shutdown_connections 2024-09-13T18:29:17.581 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.578+0000 7f748768a740 1 --2- 172.21.15.114:0/3278694346 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x555fdf915770 0x555fdf917b90 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.578+0000 7f748768a740 1 --2- 172.21.15.114:0/3278694346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fdf8ae4f0 0x555fdf913e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.578+0000 7f748768a740 1 --2- 172.21.15.114:0/3278694346 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555fdf8a3c50 0x555fdf8b2540 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.578+0000 7f748768a740 1 --2- 172.21.15.114:0/3278694346 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fdf8a3330 0x555fdf8b2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.578+0000 7f748768a740 1 -- 172.21.15.114:0/3278694346 >> 172.21.15.114:0/3278694346 conn(0x555fdf89f8c0 msgr2=0x555fdf8a9980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.578+0000 7f7465ffb640 0 client.17562 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:29:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.578+0000 7f748768a740 1 -- 172.21.15.114:0/3278694346 shutdown_connections 2024-09-13T18:29:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.578+0000 7f748768a740 1 -- 172.21.15.114:0/3278694346 wait complete. 2024-09-13T18:29:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.579+0000 7f748768a740 20 client.17562 trim_cache size 0 max 16384 2024-09-13T18:29:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-13T18:29:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-13T18:29:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-13T18:29:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-13T18:29:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.580+0000 7f748488b640 1 -- 172.21.15.114:0/2751035611 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f746c0a1410 msgr2=0x7f746c0a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.580+0000 7f748488b640 1 --2- 172.21.15.114:0/2751035611 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f746c0a1410 0x7f746c0a38d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x555fdf89f3c0 tx=0x7f7468008040 comp rx=0 tx=0).stop 2024-09-13T18:29:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.580+0000 7f748488b640 1 -- 172.21.15.114:0/2751035611 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7480107890 msgr2=0x7f74801a55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.580+0000 7f748488b640 1 --2- 172.21.15.114:0/2751035611 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7480107890 0x7f74801a55e0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f747000b3f0 tx=0x7f747000b8b0 comp rx=0 tx=0).stop 2024-09-13T18:29:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.580+0000 7f748488b640 1 -- 172.21.15.114:0/2751035611 shutdown_connections 2024-09-13T18:29:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.580+0000 7f748488b640 1 --2- 172.21.15.114:0/2751035611 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f746c0a1410 0x7f746c0a38d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.580+0000 7f748488b640 1 --2- 172.21.15.114:0/2751035611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7480119400 0x7f74801a99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.580+0000 7f748488b640 1 --2- 172.21.15.114:0/2751035611 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7480107890 0x7f74801a55e0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.580+0000 7f748488b640 1 --2- 172.21.15.114:0/2751035611 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74801074b0 0x7f74801a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.580+0000 7f748488b640 1 -- 172.21.15.114:0/2751035611 >> 172.21.15.114:0/2751035611 conn(0x7f748006af50 msgr2=0x7f7480119ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.581+0000 7f748488b640 1 -- 172.21.15.114:0/2751035611 shutdown_connections 2024-09-13T18:29:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:17.581+0000 7f748488b640 1 -- 172.21.15.114:0/2751035611 wait complete. 2024-09-13T18:29:18.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:29:17 smithi134 ceph-mon[34283]: pgmap v209: 129 pgs: 129 active+clean; 20 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 38 MiB/s wr, 819 op/s 2024-09-13T18:29:18.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:29:17 smithi204 ceph-mon[34777]: pgmap v209: 129 pgs: 129 active+clean; 20 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 38 MiB/s wr, 819 op/s 2024-09-13T18:29:18.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:29:17 smithi114 ceph-mon[30897]: pgmap v209: 129 pgs: 129 active+clean; 20 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 38 MiB/s wr, 819 op/s 2024-09-13T18:29:20.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:29:19 smithi134 ceph-mon[34283]: pgmap v210: 129 pgs: 129 active+clean; 20 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 35 MiB/s wr, 772 op/s 2024-09-13T18:29:20.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:29:19 smithi204 ceph-mon[34777]: pgmap v210: 129 pgs: 129 active+clean; 20 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 35 MiB/s wr, 772 op/s 2024-09-13T18:29:20.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:29:19 smithi114 ceph-mon[30897]: pgmap v210: 129 pgs: 129 active+clean; 20 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 35 MiB/s wr, 772 op/s 2024-09-13T18:29:22.118 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:29:21 smithi134 ceph-mon[34283]: pgmap v211: 129 pgs: 129 active+clean; 21 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 36 MiB/s wr, 795 op/s 2024-09-13T18:29:22.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:29:21 smithi204 ceph-mon[34777]: pgmap v211: 129 pgs: 129 active+clean; 21 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 36 MiB/s wr, 795 op/s 2024-09-13T18:29:22.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:29:21 smithi114 ceph-mon[30897]: pgmap v211: 129 pgs: 129 active+clean; 21 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 36 MiB/s wr, 795 op/s 2024-09-13T18:29:23.461 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T18:29:23.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.694+0000 7f8b1fbe2640 1 -- 172.21.15.114:0/1336134575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b18107540 msgr2=0x7f8b1810e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:23.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.694+0000 7f8b1fbe2640 1 --2- 172.21.15.114:0/1336134575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b18107540 0x7f8b1810e510 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8b10009900 tx=0x7f8b1002a440 comp rx=0 tx=0).stop 2024-09-13T18:29:23.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.700+0000 7f8b1fbe2640 1 -- 172.21.15.114:0/1336134575 shutdown_connections 2024-09-13T18:29:23.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.700+0000 7f8b1fbe2640 1 --2- 172.21.15.114:0/1336134575 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b18119400 0x7f8b1811b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:23.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.700+0000 7f8b1fbe2640 1 --2- 172.21.15.114:0/1336134575 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b18107920 0x7f8b18107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:23.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.700+0000 7f8b1fbe2640 1 --2- 172.21.15.114:0/1336134575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b18107540 0x7f8b1810e510 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:23.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.700+0000 7f8b1fbe2640 1 -- 172.21.15.114:0/1336134575 >> 172.21.15.114:0/1336134575 conn(0x7f8b1806afe0 msgr2=0x7f8b1806b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:23.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.707+0000 7f8b1fbe2640 1 -- 172.21.15.114:0/1336134575 shutdown_connections 2024-09-13T18:29:23.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.713+0000 7f8b1fbe2640 1 -- 172.21.15.114:0/1336134575 wait complete. 2024-09-13T18:29:23.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.714+0000 7f8b1fbe2640 1 Processor -- start 2024-09-13T18:29:23.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.714+0000 7f8b1fbe2640 1 -- start start 2024-09-13T18:29:23.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.715+0000 7f8b1fbe2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b18107540 0x7f8b181168d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:23.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.715+0000 7f8b1fbe2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b18107920 0x7f8b18116e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:23.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.715+0000 7f8b1fbe2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b18119400 0x7f8b1810fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:23.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.715+0000 7f8b1fbe2640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b181101a0 con 0x7f8b18107540 2024-09-13T18:29:23.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.715+0000 7f8b1fbe2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b18110310 con 0x7f8b18107920 2024-09-13T18:29:23.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.715+0000 7f8b1fbe2640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b181104d0 con 0x7f8b18119400 2024-09-13T18:29:23.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.715+0000 7f8b1d97e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b18107540 0x7f8b181168d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:23.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.715+0000 7f8b1d97e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b18107540 0x7f8b181168d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57376/0 (socket says 172.21.15.114:57376) 2024-09-13T18:29:23.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.715+0000 7f8b1d97e640 1 -- 172.21.15.114:0/988538868 learned_addr learned my addr 172.21.15.114:0/988538868 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:23.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.717+0000 7f8b1d97e640 1 -- 172.21.15.114:0/988538868 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b18119400 msgr2=0x7f8b1810fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:23.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.717+0000 7f8b1d97e640 1 --2- 172.21.15.114:0/988538868 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b18119400 0x7f8b1810fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:23.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.717+0000 7f8b1d97e640 1 -- 172.21.15.114:0/988538868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b18107920 msgr2=0x7f8b18116e10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:29:23.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.717+0000 7f8b1d97e640 1 --2- 172.21.15.114:0/988538868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b18107920 0x7f8b18116e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:23.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.717+0000 7f8b1d97e640 1 -- 172.21.15.114:0/988538868 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b10009560 con 0x7f8b18107540 2024-09-13T18:29:23.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.717+0000 7f8b1d97e640 1 --2- 172.21.15.114:0/988538868 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b18107540 0x7f8b181168d0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f8b10006010 tx=0x7f8b10031670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:23.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.718+0000 7f8b16ffd640 1 -- 172.21.15.114:0/988538868 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b10031c70 con 0x7f8b18107540 2024-09-13T18:29:23.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.718+0000 7f8b1fbe2640 1 -- 172.21.15.114:0/988538868 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b18110750 con 0x7f8b18107540 2024-09-13T18:29:23.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.718+0000 7f8b1fbe2640 1 -- 172.21.15.114:0/988538868 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b18114710 con 0x7f8b18107540 2024-09-13T18:29:23.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.718+0000 7f8b16ffd640 1 -- 172.21.15.114:0/988538868 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8b10005650 con 0x7f8b18107540 2024-09-13T18:29:23.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.719+0000 7f8b16ffd640 1 -- 172.21.15.114:0/988538868 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b1003c610 con 0x7f8b18107540 2024-09-13T18:29:23.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.725+0000 7f8b229dc740 1 Processor -- start 2024-09-13T18:29:23.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.726+0000 7f8b229dc740 1 -- start start 2024-09-13T18:29:23.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.726+0000 7f8b229dc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592db61be10 0x5592db61c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:23.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.726+0000 7f8b229dc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5592db61c730 0x5592db626a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:23.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.726+0000 7f8b229dc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5592db626fd0 0x5592db6293c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:23.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.726+0000 7f8b229dc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592db586f20 con 0x5592db61c730 2024-09-13T18:29:23.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.726+0000 7f8b229dc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592db5846b0 con 0x5592db61be10 2024-09-13T18:29:23.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.726+0000 7f8b229dc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592db57ffa0 con 0x5592db626fd0 2024-09-13T18:29:23.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.726+0000 7f8b16ffd640 1 -- 172.21.15.114:0/988538868 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f8b1003c770 con 0x7f8b18107540 2024-09-13T18:29:23.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.726+0000 7f8b1e17f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592db61be10 0x5592db61c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:23.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.726+0000 7f8b1e17f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592db61be10 0x5592db61c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:56094/0 (socket says 172.21.15.114:56094) 2024-09-13T18:29:23.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.727+0000 7f8b1e17f640 1 -- 172.21.15.114:0/748177406 learned_addr learned my addr 172.21.15.114:0/748177406 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:23.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.727+0000 7f8b1e17f640 1 -- 172.21.15.114:0/748177406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5592db626fd0 msgr2=0x5592db6293c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:29:23.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.727+0000 7f8b1e17f640 1 --2- 172.21.15.114:0/748177406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5592db626fd0 0x5592db6293c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:23.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.727+0000 7f8b1e17f640 1 -- 172.21.15.114:0/748177406 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5592db61c730 msgr2=0x5592db626a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:23.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.727+0000 7f8b1e17f640 1 --2- 172.21.15.114:0/748177406 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5592db61c730 0x5592db626a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:23.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.727+0000 7f8b1e17f640 1 -- 172.21.15.114:0/748177406 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592db53a840 con 0x5592db61be10 2024-09-13T18:29:23.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.728+0000 7f8b16ffd640 1 --2- 172.21.15.114:0/988538868 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f8b080a1410 0x7f8b080a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:23.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.728+0000 7f8b1e17f640 1 --2- 172.21.15.114:0/748177406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592db61be10 0x5592db61c1f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c009d10 tx=0x7f8b0c00e6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a430022886fe0ed0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:23.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.729+0000 7f8b16ffd640 1 -- 172.21.15.114:0/988538868 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f8b100ee310 con 0x7f8b18107540 2024-09-13T18:29:23.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.731+0000 7f8b03fff640 1 -- 172.21.15.114:0/748177406 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c004b20 con 0x5592db61be10 2024-09-13T18:29:23.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.731+0000 7f8b1d97e640 1 --2- 172.21.15.114:0/988538868 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f8b080a1410 0x7f8b080a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:23.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.731+0000 7f8b03fff640 1 -- 172.21.15.114:0/748177406 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8b0c004c80 con 0x5592db61be10 2024-09-13T18:29:23.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.731+0000 7f8b03fff640 1 -- 172.21.15.114:0/748177406 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c0116d0 con 0x5592db61be10 2024-09-13T18:29:23.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.734+0000 7f8b229dc740 1 -- 172.21.15.114:0/748177406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592db61be10 msgr2=0x5592db61c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:23.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.734+0000 7f8b229dc740 1 --2- 172.21.15.114:0/748177406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592db61be10 0x5592db61c1f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c009d10 tx=0x7f8b0c00e6c0 comp rx=0 tx=0).stop 2024-09-13T18:29:23.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.734+0000 7f8b229dc740 1 -- 172.21.15.114:0/748177406 shutdown_connections 2024-09-13T18:29:23.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.734+0000 7f8b229dc740 1 --2- 172.21.15.114:0/748177406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5592db626fd0 0x5592db6293c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:23.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.734+0000 7f8b229dc740 1 --2- 172.21.15.114:0/748177406 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5592db61c730 0x5592db626a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:23.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.734+0000 7f8b229dc740 1 --2- 172.21.15.114:0/748177406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592db61be10 0x5592db61c1f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:23.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.734+0000 7f8b229dc740 1 -- 172.21.15.114:0/748177406 >> 172.21.15.114:0/748177406 conn(0x5592db618380 msgr2=0x5592db618760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:23.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.734+0000 7f8b1d97e640 1 --2- 172.21.15.114:0/988538868 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f8b080a1410 0x7f8b080a38d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x5592db6191e0 tx=0x7f8b1003df60 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:23.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.735+0000 7f8b229dc740 1 -- 172.21.15.114:0/748177406 shutdown_connections 2024-09-13T18:29:23.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.739+0000 7f8b229dc740 1 -- 172.21.15.114:0/748177406 wait complete. 2024-09-13T18:29:23.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.739+0000 7f8b229dc740 1 -- start start 2024-09-13T18:29:23.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.739+0000 7f8b229dc740 1 Processor -- start 2024-09-13T18:29:23.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.741+0000 7f8b229dc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5592db61be10 0x5592db626960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:23.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.741+0000 7f8b229dc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592db61c730 0x5592db623c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:23.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.741+0000 7f8b229dc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5592db626fd0 0x5592db624180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:23.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.741+0000 7f8b229dc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592db5212a0 con 0x5592db626fd0 2024-09-13T18:29:23.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.741+0000 7f8b229dc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592db258430 con 0x5592db61c730 2024-09-13T18:29:23.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.741+0000 7f8b229dc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592db503680 con 0x5592db61be10 2024-09-13T18:29:23.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.741+0000 7f8b229dc740 10 client.0 _ll_register_callbacks cb 0x5592db579610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:29:23.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.741+0000 7f8b1e17f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5592db61be10 0x5592db626960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:23.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.741+0000 7f8b1e17f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5592db61be10 0x5592db626960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:49022/0 (socket says 172.21.15.114:49022) 2024-09-13T18:29:23.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.741+0000 7f8b1e17f640 1 -- 172.21.15.114:0/3024652437 learned_addr learned my addr 172.21.15.114:0/3024652437 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:23.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.741+0000 7f8b1e17f640 1 --2- 172.21.15.114:0/3024652437 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5592db626fd0 0x5592db624180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:23.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.742+0000 7f8b1d17d640 1 --2- 172.21.15.114:0/3024652437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592db61c730 0x5592db623c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:23.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.742+0000 7f8b1e17f640 1 -- 172.21.15.114:0/3024652437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592db61c730 msgr2=0x5592db623c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:23.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.742+0000 7f8b1e17f640 1 --2- 172.21.15.114:0/3024652437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592db61c730 0x5592db623c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:23.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.742+0000 7f8b1e17f640 1 -- 172.21.15.114:0/3024652437 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5592db626fd0 msgr2=0x5592db624180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:23.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.742+0000 7f8b1e17f640 1 --2- 172.21.15.114:0/3024652437 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5592db626fd0 0x5592db624180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:23.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.742+0000 7f8b1e17f640 1 -- 172.21.15.114:0/3024652437 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b0c009940 con 0x5592db61be10 2024-09-13T18:29:23.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.742+0000 7f8b1d17d640 1 --2- 172.21.15.114:0/3024652437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592db61c730 0x5592db623c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:29:23.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.742+0000 7f8b1e17f640 1 --2- 172.21.15.114:0/3024652437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5592db61be10 0x5592db626960 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c005cc0 tx=0x7f8b0c009240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3659e31eb5353f03 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:23.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.742+0000 7f8b017fa640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:29:23.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.743+0000 7f8b017fa640 1 -- 172.21.15.114:0/3024652437 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c0119e0 con 0x5592db61be10 2024-09-13T18:29:23.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.743+0000 7f8b229dc740 1 -- 172.21.15.114:0/3024652437 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5592db2734e0 con 0x5592db61be10 2024-09-13T18:29:23.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.743+0000 7f8b017fa640 1 -- 172.21.15.114:0/3024652437 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f8b0c011b40 con 0x5592db61be10 2024-09-13T18:29:23.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.743+0000 7f8b017fa640 1 -- 172.21.15.114:0/3024652437 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b0c011d00 con 0x5592db61be10 2024-09-13T18:29:23.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.743+0000 7f8b229dc740 10 client.26986 fetch_fsmap learned FSMap version 34 2024-09-13T18:29:23.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.743+0000 7f8b229dc740 1 -- 172.21.15.114:0/3024652437 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5592db57e620 con 0x5592db61be10 2024-09-13T18:29:23.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.744+0000 7f8b017fa640 1 -- 172.21.15.114:0/3024652437 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5592db2734e0 con 0x5592db61be10 2024-09-13T18:29:23.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.744+0000 7f8b017fa640 1 -- 172.21.15.114:0/3024652437 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8b0c026070 con 0x5592db61be10 2024-09-13T18:29:23.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.744+0000 7f8b229dc740 10 client.26986 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:29:23.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.744+0000 7f8b229dc740 10 client.26986 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:29:23.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.744+0000 7f8b229dc740 20 client.26986 populate_metadata read hostname 'smithi114' 2024-09-13T18:29:23.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.744+0000 7f8b229dc740 1 --2- 172.21.15.114:0/3024652437 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5592db636cf0 0x5592db639110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:23.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.744+0000 7f8b229dc740 4 client.26986 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:29:23.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.744+0000 7f8b229dc740 1 -- 172.21.15.114:0/3024652437 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5592db2847e0 con 0x5592db636cf0 2024-09-13T18:29:23.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.744+0000 7f8b1d17d640 1 --2- 172.21.15.114:0/3024652437 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5592db636cf0 0x5592db639110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:23.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.745+0000 7f8b017fa640 10 client.26986 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:29:23.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.745+0000 7f8b1d17d640 1 --2- 172.21.15.114:0/3024652437 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5592db636cf0 0x5592db639110 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d49c4375ebb44caf server_cookie=10b6e6b76ceeea57 in_seq=0 out_seq=0 2024-09-13T18:29:23.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.745+0000 7f8b017fa640 1 -- 172.21.15.114:0/3024652437 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5592db2847e0 con 0x5592db636cf0 2024-09-13T18:29:23.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.745+0000 7f8b017fa640 10 client.26986 handle_command_reply: tid=0 2024-09-13T18:29:23.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.745+0000 7f8b229dc740 1 client.26986 shutdown 2024-09-13T18:29:23.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.746+0000 7f8b229dc740 1 -- 172.21.15.114:0/3024652437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5592db61be10 msgr2=0x5592db626960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:23.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.746+0000 7f8b229dc740 1 --2- 172.21.15.114:0/3024652437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5592db61be10 0x5592db626960 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c005cc0 tx=0x7f8b0c009240 comp rx=0 tx=0).stop 2024-09-13T18:29:23.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.746+0000 7f8b229dc740 1 -- 172.21.15.114:0/3024652437 shutdown_connections 2024-09-13T18:29:23.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.746+0000 7f8b229dc740 1 --2- 172.21.15.114:0/3024652437 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5592db636cf0 0x5592db639110 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:23.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.746+0000 7f8b229dc740 1 --2- 172.21.15.114:0/3024652437 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5592db626fd0 0x5592db624180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:23.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.746+0000 7f8b229dc740 1 --2- 172.21.15.114:0/3024652437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592db61c730 0x5592db623c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:23.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.746+0000 7f8b229dc740 1 --2- 172.21.15.114:0/3024652437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5592db61be10 0x5592db626960 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:23.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.746+0000 7f8b229dc740 1 -- 172.21.15.114:0/3024652437 >> 172.21.15.114:0/3024652437 conn(0x5592db618380 msgr2=0x5592db61e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:23.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.746+0000 7f8b017fa640 0 client.26986 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:29:23.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.746+0000 7f8b229dc740 1 -- 172.21.15.114:0/3024652437 shutdown_connections 2024-09-13T18:29:23.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.746+0000 7f8b229dc740 1 -- 172.21.15.114:0/3024652437 wait complete. 2024-09-13T18:29:23.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.746+0000 7f8b229dc740 20 client.26986 trim_cache size 0 max 16384 2024-09-13T18:29:23.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.788+0000 7f8b229dc740 1 Processor -- start 2024-09-13T18:29:23.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.789+0000 7f8b229dc740 1 -- start start 2024-09-13T18:29:23.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.789+0000 7f8b229dc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592db61be10 0x5592db617d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:23.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.789+0000 7f8b229dc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5592db61c730 0x5592db638d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:23.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.789+0000 7f8b1e17f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592db61be10 0x5592db617d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:23.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.789+0000 7f8b1e17f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592db61be10 0x5592db617d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:56100/0 (socket says 172.21.15.114:56100) 2024-09-13T18:29:23.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.790+0000 7f8b1e17f640 1 -- 172.21.15.114:0/3455092744 learned_addr learned my addr 172.21.15.114:0/3455092744 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:23.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.790+0000 7f8b1d17d640 1 --2- 172.21.15.114:0/3455092744 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5592db61c730 0x5592db638d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:23.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.790+0000 7f8b229dc740 1 --2- 172.21.15.114:0/3455092744 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5592db626fd0 0x5592db63da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:23.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.790+0000 7f8b229dc740 1 -- 172.21.15.114:0/3455092744 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592db619060 con 0x5592db626fd0 2024-09-13T18:29:23.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.790+0000 7f8b229dc740 1 -- 172.21.15.114:0/3455092744 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592db61d630 con 0x5592db61be10 2024-09-13T18:29:23.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.790+0000 7f8b229dc740 1 -- 172.21.15.114:0/3455092744 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592db61eab0 con 0x5592db61c730 2024-09-13T18:29:23.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.791+0000 7f8b1e17f640 1 -- 172.21.15.114:0/3455092744 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5592db61c730 msgr2=0x5592db638d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:23.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.791+0000 7f8b1e17f640 1 --2- 172.21.15.114:0/3455092744 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5592db61c730 0x5592db638d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:23.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.791+0000 7f8b1e17f640 1 -- 172.21.15.114:0/3455092744 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5592db626fd0 msgr2=0x5592db63da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:23.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.791+0000 7f8b1e17f640 1 --2- 172.21.15.114:0/3455092744 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5592db626fd0 0x5592db63da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:23.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.791+0000 7f8b1e17f640 1 -- 172.21.15.114:0/3455092744 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592db520790 con 0x5592db61be10 2024-09-13T18:29:23.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.791+0000 7f8b1e17f640 1 --2- 172.21.15.114:0/3455092744 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592db61be10 0x5592db617d00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c009c30 tx=0x7f8b0c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e04943c4d2dedc7f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:23.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.795+0000 7f8b14e05640 1 -- 172.21.15.114:0/3455092744 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c005b00 con 0x5592db61be10 2024-09-13T18:29:23.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.795+0000 7f8b14e05640 1 -- 172.21.15.114:0/3455092744 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8b0c020070 con 0x5592db61be10 2024-09-13T18:29:23.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.795+0000 7f8b14e05640 1 -- 172.21.15.114:0/3455092744 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c005b00 con 0x5592db61be10 2024-09-13T18:29:23.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.796+0000 7f8b229dc740 1 -- 172.21.15.114:0/3455092744 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592db61be10 msgr2=0x5592db617d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:23.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.796+0000 7f8b229dc740 1 --2- 172.21.15.114:0/3455092744 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592db61be10 0x5592db617d00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c009c30 tx=0x7f8b0c005f90 comp rx=0 tx=0).stop 2024-09-13T18:29:23.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.797+0000 7f8b229dc740 1 -- 172.21.15.114:0/3455092744 shutdown_connections 2024-09-13T18:29:23.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.797+0000 7f8b229dc740 1 --2- 172.21.15.114:0/3455092744 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5592db626fd0 0x5592db63da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:23.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.797+0000 7f8b229dc740 1 --2- 172.21.15.114:0/3455092744 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5592db61c730 0x5592db638d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:23.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.797+0000 7f8b229dc740 1 --2- 172.21.15.114:0/3455092744 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592db61be10 0x5592db617d00 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:23.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.797+0000 7f8b229dc740 1 -- 172.21.15.114:0/3455092744 >> 172.21.15.114:0/3455092744 conn(0x5592db618380 msgr2=0x5592db626570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:23.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.806+0000 7f8b229dc740 1 -- 172.21.15.114:0/3455092744 shutdown_connections 2024-09-13T18:29:23.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.806+0000 7f8b229dc740 1 -- 172.21.15.114:0/3455092744 wait complete. 2024-09-13T18:29:23.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.806+0000 7f8b229dc740 1 -- start start 2024-09-13T18:29:23.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.807+0000 7f8b229dc740 1 Processor -- start 2024-09-13T18:29:23.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.807+0000 7f8b229dc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5592db61be10 0x5592db62dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:23.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.807+0000 7f8b229dc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5592db61c730 0x5592db629d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:23.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.808+0000 7f8b1e17f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5592db61be10 0x5592db62dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:23.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.808+0000 7f8b1e17f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5592db61be10 0x5592db62dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:49042/0 (socket says 172.21.15.114:49042) 2024-09-13T18:29:23.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.808+0000 7f8b1e17f640 1 -- 172.21.15.114:0/999367911 learned_addr learned my addr 172.21.15.114:0/999367911 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:23.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.808+0000 7f8b229dc740 1 --2- 172.21.15.114:0/999367911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592db626fd0 0x5592db62a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:23.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.808+0000 7f8b229dc740 1 -- 172.21.15.114:0/999367911 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592db62e6c0 con 0x5592db61c730 2024-09-13T18:29:23.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.808+0000 7f8b229dc740 1 -- 172.21.15.114:0/999367911 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592db62a7f0 con 0x5592db626fd0 2024-09-13T18:29:23.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.808+0000 7f8b229dc740 1 -- 172.21.15.114:0/999367911 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592db62a980 con 0x5592db61be10 2024-09-13T18:29:23.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.808+0000 7f8b1e17f640 1 --2- 172.21.15.114:0/999367911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592db626fd0 0x5592db62a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:23.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.809+0000 7f8b1e17f640 1 -- 172.21.15.114:0/999367911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592db626fd0 msgr2=0x5592db62a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:23.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.809+0000 7f8b1e17f640 1 --2- 172.21.15.114:0/999367911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592db626fd0 0x5592db62a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:23.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.809+0000 7f8b1e17f640 1 -- 172.21.15.114:0/999367911 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5592db61c730 msgr2=0x5592db629d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:23.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.809+0000 7f8b1e17f640 1 --2- 172.21.15.114:0/999367911 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5592db61c730 0x5592db629d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:23.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.809+0000 7f8b1e17f640 1 -- 172.21.15.114:0/999367911 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5592db57e620 con 0x5592db61be10 2024-09-13T18:29:23.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.809+0000 7f8b229dc740 10 client.0 _ll_register_callbacks cb 0x7f8b0c009b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:29:23.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.809+0000 7f8b1e17f640 1 --2- 172.21.15.114:0/999367911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5592db61be10 0x5592db62dfa0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c0092c0 tx=0x7f8b0c00e6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bccf03698298c7db server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:23.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.809+0000 7f8b017fa640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:29:23.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.810+0000 7f8b017fa640 1 -- 172.21.15.114:0/999367911 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c005b00 con 0x5592db61be10 2024-09-13T18:29:23.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.810+0000 7f8b229dc740 1 -- 172.21.15.114:0/999367911 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8b0c004c80 con 0x5592db61be10 2024-09-13T18:29:23.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.810+0000 7f8b017fa640 1 -- 172.21.15.114:0/999367911 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f8b0c00cc80 con 0x5592db61be10 2024-09-13T18:29:23.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.810+0000 7f8b017fa640 1 -- 172.21.15.114:0/999367911 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b0c00ce40 con 0x5592db61be10 2024-09-13T18:29:23.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.812+0000 7f8b229dc740 10 client.26992 fetch_fsmap learned FSMap version 34 2024-09-13T18:29:23.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.812+0000 7f8b229dc740 1 -- 172.21.15.114:0/999367911 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({fsmap=34}) v3 -- 0x5592db545ea0 con 0x5592db61be10 2024-09-13T18:29:23.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.818+0000 7f8b017fa640 1 -- 172.21.15.114:0/999367911 <== mon.1 v2:172.21.15.204:3300/0 4 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8b0c026040 con 0x5592db61be10 2024-09-13T18:29:23.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.819+0000 7f8b229dc740 10 client.26992 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:29:23.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.819+0000 7f8b229dc740 10 client.26992 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:29:23.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.819+0000 7f8b229dc740 20 client.26992 populate_metadata read hostname 'smithi114' 2024-09-13T18:29:23.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.819+0000 7f8b229dc740 1 --2- 172.21.15.114:0/999367911 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5592db68ce90 0x5592db68f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:23.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.819+0000 7f8b229dc740 4 client.26992 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T18:29:23.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.819+0000 7f8b229dc740 1 -- 172.21.15.114:0/999367911 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f8b0c004c80 con 0x5592db68ce90 2024-09-13T18:29:23.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.819+0000 7f8b1d17d640 1 --2- 172.21.15.114:0/999367911 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5592db68ce90 0x5592db68f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:23.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.827+0000 7f8b1d17d640 1 --2- 172.21.15.114:0/999367911 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5592db68ce90 0x5592db68f280 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=335f206207123a52 server_cookie=8365b11b05091a2 in_seq=0 out_seq=0 2024-09-13T18:29:23.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.829+0000 7f8b017fa640 10 client.26992 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:29:23.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.831+0000 7f8b017fa640 1 -- 172.21.15.114:0/999367911 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x7f8b0c004c80 con 0x5592db68ce90 2024-09-13T18:29:23.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.831+0000 7f8b017fa640 10 client.26992 handle_command_reply: tid=0 2024-09-13T18:29:23.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.831+0000 7f8b229dc740 1 client.26992 shutdown 2024-09-13T18:29:23.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.832+0000 7f8b229dc740 1 -- 172.21.15.114:0/999367911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5592db61be10 msgr2=0x5592db62dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:23.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.832+0000 7f8b229dc740 1 --2- 172.21.15.114:0/999367911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5592db61be10 0x5592db62dfa0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c0092c0 tx=0x7f8b0c00e6c0 comp rx=0 tx=0).stop 2024-09-13T18:29:23.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.833+0000 7f8b229dc740 1 -- 172.21.15.114:0/999367911 shutdown_connections 2024-09-13T18:29:23.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.833+0000 7f8b229dc740 1 --2- 172.21.15.114:0/999367911 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5592db68ce90 0x5592db68f280 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:23.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.833+0000 7f8b229dc740 1 --2- 172.21.15.114:0/999367911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592db626fd0 0x5592db62a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:23.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.833+0000 7f8b017fa640 0 client.26992 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:29:23.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.833+0000 7f8b229dc740 1 --2- 172.21.15.114:0/999367911 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5592db61c730 0x5592db629d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:23.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.833+0000 7f8b229dc740 1 --2- 172.21.15.114:0/999367911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5592db61be10 0x5592db62dfa0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:23.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.833+0000 7f8b229dc740 1 -- 172.21.15.114:0/999367911 >> 172.21.15.114:0/999367911 conn(0x5592db618380 msgr2=0x5592db638d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:23.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.833+0000 7f8b229dc740 1 -- 172.21.15.114:0/999367911 shutdown_connections 2024-09-13T18:29:23.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.833+0000 7f8b229dc740 1 -- 172.21.15.114:0/999367911 wait complete. 2024-09-13T18:29:23.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.834+0000 7f8b229dc740 20 client.26992 trim_cache size 0 max 16384 2024-09-13T18:29:23.837 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:29:23.839 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "no active scrubs running", 2024-09-13T18:29:23.840 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": {} 2024-09-13T18:29:23.840 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:29:23.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.835+0000 7f8b1fbe2640 1 -- 172.21.15.114:0/988538868 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f8b080a1410 msgr2=0x7f8b080a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:23.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.835+0000 7f8b1fbe2640 1 --2- 172.21.15.114:0/988538868 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f8b080a1410 0x7f8b080a38d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x5592db6191e0 tx=0x7f8b1003df60 comp rx=0 tx=0).stop 2024-09-13T18:29:23.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.835+0000 7f8b1fbe2640 1 -- 172.21.15.114:0/988538868 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b18107540 msgr2=0x7f8b181168d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:23.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.835+0000 7f8b1fbe2640 1 --2- 172.21.15.114:0/988538868 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b18107540 0x7f8b181168d0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f8b10006010 tx=0x7f8b10031670 comp rx=0 tx=0).stop 2024-09-13T18:29:23.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.835+0000 7f8b1fbe2640 1 -- 172.21.15.114:0/988538868 shutdown_connections 2024-09-13T18:29:23.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.835+0000 7f8b1fbe2640 1 --2- 172.21.15.114:0/988538868 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f8b080a1410 0x7f8b080a38d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:23.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.835+0000 7f8b1fbe2640 1 --2- 172.21.15.114:0/988538868 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b18119400 0x7f8b1810fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:23.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.835+0000 7f8b1fbe2640 1 --2- 172.21.15.114:0/988538868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b18107920 0x7f8b18116e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:23.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.835+0000 7f8b1fbe2640 1 --2- 172.21.15.114:0/988538868 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b18107540 0x7f8b181168d0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:23.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.835+0000 7f8b1fbe2640 1 -- 172.21.15.114:0/988538868 >> 172.21.15.114:0/988538868 conn(0x7f8b1806afe0 msgr2=0x7f8b18119be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:23.842 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:29:23 smithi114 ceph-mon[30897]: pgmap v212: 129 pgs: 129 active+clean; 21 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 36 MiB/s wr, 795 op/s 2024-09-13T18:29:23.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.838+0000 7f8b1fbe2640 1 -- 172.21.15.114:0/988538868 shutdown_connections 2024-09-13T18:29:23.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:23.838+0000 7f8b1fbe2640 1 -- 172.21.15.114:0/988538868 wait complete. 2024-09-13T18:29:23.869 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-13T18:29:23.869 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:29:24.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.099+0000 7fec7a4af640 1 -- 172.21.15.114:0/1073142892 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec6c09c800 msgr2=0x7fec6c0a07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:24.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.099+0000 7fec7a4af640 1 --2- 172.21.15.114:0/1073142892 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec6c09c800 0x7fec6c0a07f0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fec7402c970 tx=0x7fec740a59b0 comp rx=0 tx=0).stop 2024-09-13T18:29:24.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.100+0000 7fec7a4af640 1 -- 172.21.15.114:0/1073142892 shutdown_connections 2024-09-13T18:29:24.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.100+0000 7fec7a4af640 1 --2- 172.21.15.114:0/1073142892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec6c0a0f50 0x7fec6c0a8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:24.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.100+0000 7fec7a4af640 1 --2- 172.21.15.114:0/1073142892 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec6c09c800 0x7fec6c0a07f0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:24.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.100+0000 7fec7a4af640 1 --2- 172.21.15.114:0/1073142892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec6c09be50 0x7fec6c09c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:24.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.100+0000 7fec7a4af640 1 -- 172.21.15.114:0/1073142892 >> 172.21.15.114:0/1073142892 conn(0x7fec6c008fe0 msgr2=0x7fec6c09a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:24.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.105+0000 7fec7a4af640 1 -- 172.21.15.114:0/1073142892 shutdown_connections 2024-09-13T18:29:24.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.106+0000 7fec7a4af640 1 -- 172.21.15.114:0/1073142892 wait complete. 2024-09-13T18:29:24.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.107+0000 7fec7a4af640 1 Processor -- start 2024-09-13T18:29:24.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.108+0000 7fec7a4af640 1 -- start start 2024-09-13T18:29:24.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.109+0000 7fec7a4af640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec6c09be50 0x7fec6c0a85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:24.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.109+0000 7fec7a4af640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec6c0a0f50 0x7fec6c0a4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:24.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.109+0000 7fec7a4af640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec6c0a4b70 0x7fec6c0a4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:24.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.109+0000 7fec7a4af640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6c0a8e00 con 0x7fec6c0a0f50 2024-09-13T18:29:24.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.109+0000 7fec7a4af640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6c1692b0 con 0x7fec6c0a4b70 2024-09-13T18:29:24.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.109+0000 7fec7a4af640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6c169420 con 0x7fec6c09be50 2024-09-13T18:29:24.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.109+0000 7fec79cae640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec6c0a4b70 0x7fec6c0a4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:24.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.109+0000 7fec79cae640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec6c0a4b70 0x7fec6c0a4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:56106/0 (socket says 172.21.15.114:56106) 2024-09-13T18:29:24.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.109+0000 7fec79cae640 1 -- 172.21.15.114:0/1917580790 learned_addr learned my addr 172.21.15.114:0/1917580790 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:24.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.110+0000 7fec78cac640 1 --2- 172.21.15.114:0/1917580790 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec6c0a0f50 0x7fec6c0a4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:24.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.111+0000 7fec78cac640 1 -- 172.21.15.114:0/1917580790 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec6c09be50 msgr2=0x7fec6c0a85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:24.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.111+0000 7fec78cac640 1 --2- 172.21.15.114:0/1917580790 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec6c09be50 0x7fec6c0a85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:24.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.111+0000 7fec78cac640 1 -- 172.21.15.114:0/1917580790 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec6c0a4b70 msgr2=0x7fec6c0a4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:24.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.111+0000 7fec78cac640 1 --2- 172.21.15.114:0/1917580790 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec6c0a4b70 0x7fec6c0a4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:24.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.111+0000 7fec78cac640 1 -- 172.21.15.114:0/1917580790 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec7404f100 con 0x7fec6c0a0f50 2024-09-13T18:29:24.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.111+0000 7fec78cac640 1 --2- 172.21.15.114:0/1917580790 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec6c0a0f50 0x7fec6c0a4630 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fec6c0075b0 tx=0x7fec74086670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:24.118 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:29:23 smithi134 ceph-mon[34283]: pgmap v212: 129 pgs: 129 active+clean; 21 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 36 MiB/s wr, 795 op/s 2024-09-13T18:29:24.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.116+0000 7fec6a7fc640 1 -- 172.21.15.114:0/1917580790 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec74070070 con 0x7fec6c0a0f50 2024-09-13T18:29:24.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.116+0000 7fec7a4af640 1 -- 172.21.15.114:0/1917580790 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec6c1696a0 con 0x7fec6c0a0f50 2024-09-13T18:29:24.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.116+0000 7fec7a4af640 1 -- 172.21.15.114:0/1917580790 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec6c169b20 con 0x7fec6c0a0f50 2024-09-13T18:29:24.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.116+0000 7fec6a7fc640 1 -- 172.21.15.114:0/1917580790 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec740724e0 con 0x7fec6c0a0f50 2024-09-13T18:29:24.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.117+0000 7fec6a7fc640 1 -- 172.21.15.114:0/1917580790 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec740aa640 con 0x7fec6c0a0f50 2024-09-13T18:29:24.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.122+0000 7fec6a7fc640 1 -- 172.21.15.114:0/1917580790 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fec740680a0 con 0x7fec6c0a0f50 2024-09-13T18:29:24.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.123+0000 7fec6a7fc640 1 --2- 172.21.15.114:0/1917580790 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fec600a1410 0x7fec600a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:24.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.123+0000 7fec6a7fc640 1 -- 172.21.15.114:0/1917580790 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fec740a7040 con 0x7fec6c0a0f50 2024-09-13T18:29:24.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.123+0000 7fec794ad640 1 --2- 172.21.15.114:0/1917580790 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fec600a1410 0x7fec600a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:24.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.127+0000 7fec7a4af640 1 -- 172.21.15.114:0/1917580790 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec58005250 con 0x7fec6c0a0f50 2024-09-13T18:29:24.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.133+0000 7fec794ad640 1 --2- 172.21.15.114:0/1917580790 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fec600a1410 0x7fec600a38d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fec700096f0 tx=0x7fec70006c40 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:24.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.136+0000 7fec6a7fc640 1 -- 172.21.15.114:0/1917580790 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fec74116dd0 con 0x7fec6c0a0f50 2024-09-13T18:29:24.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:29:23 smithi204 ceph-mon[34777]: pgmap v212: 129 pgs: 129 active+clean; 21 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 36 MiB/s wr, 795 op/s 2024-09-13T18:29:24.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.634+0000 7fec7a4af640 1 -- 172.21.15.114:0/1917580790 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec580050c0 con 0x7fec6c0a0f50 2024-09-13T18:29:24.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.636+0000 7fec6a7fc640 1 -- 172.21.15.114:0/1917580790 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+10930 (secure 0 0 0) 0x7fec74118d00 con 0x7fec6c0a0f50 2024-09-13T18:29:24.640 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:29:24.640 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4162817497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4162817497},{"type":"v1","addr":"172.21.15.114:6835","nonce":4162817497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/2435428478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":2435428478},{"type":"v1","addr":"172.21.15.114:6837","nonce":2435428478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T18:20:01.895117+0000","modified":"2024-09-13T18:27:22.172031+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24482,"mds_2":24475,"mds_3":24458,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/2046516451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":2046516451},{"type":"v1","addr":"172.21.15.114:6839","nonce":2046516451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":10,"addr":"172.21.15.114:6841/1956923446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":1956923446},{"type":"v1","addr":"172.21.15.114:6841","nonce":1956923446}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24391":{"gid":24391,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/507301496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":507301496},{"type":"v1","addr":"172.21.15.204:6833","nonce":507301496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4028657219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4028657219},{"type":"v1","addr":"172.21.15.134:6833","nonce":4028657219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/57029119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":57029119},{"type":"v1","addr":"172.21.15.204:6835","nonce":57029119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3068880939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3068880939},{"type":"v1","addr":"172.21.15.204:6837","nonce":3068880939}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"e","rank":3,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.134:6835/1150003275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1150003275},{"type":"v1","addr":"172.21.15.134:6835","nonce":1150003275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/344883912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":344883912},{"type":"v1","addr":"172.21.15.204:6839","nonce":344883912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6837/891692210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":891692210},{"type":"v1","addr":"172.21.15.134:6837","nonce":891692210}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6839/3466309724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3466309724},{"type":"v1","addr":"172.21.15.134:6839","nonce":3466309724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:29:24.640 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 34 2024-09-13T18:29:24.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.644+0000 7fec7a4af640 1 -- 172.21.15.114:0/1917580790 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fec600a1410 msgr2=0x7fec600a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:24.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.644+0000 7fec7a4af640 1 --2- 172.21.15.114:0/1917580790 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fec600a1410 0x7fec600a38d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fec700096f0 tx=0x7fec70006c40 comp rx=0 tx=0).stop 2024-09-13T18:29:24.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.644+0000 7fec7a4af640 1 -- 172.21.15.114:0/1917580790 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec6c0a0f50 msgr2=0x7fec6c0a4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:24.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.644+0000 7fec7a4af640 1 --2- 172.21.15.114:0/1917580790 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec6c0a0f50 0x7fec6c0a4630 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fec6c0075b0 tx=0x7fec74086670 comp rx=0 tx=0).stop 2024-09-13T18:29:24.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.645+0000 7fec7a4af640 1 -- 172.21.15.114:0/1917580790 shutdown_connections 2024-09-13T18:29:24.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.645+0000 7fec7a4af640 1 --2- 172.21.15.114:0/1917580790 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fec600a1410 0x7fec600a38d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:24.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.645+0000 7fec7a4af640 1 --2- 172.21.15.114:0/1917580790 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec6c0a4b70 0x7fec6c0a4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:24.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.646+0000 7fec7a4af640 1 --2- 172.21.15.114:0/1917580790 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec6c0a0f50 0x7fec6c0a4630 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:24.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.646+0000 7fec7a4af640 1 --2- 172.21.15.114:0/1917580790 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec6c09be50 0x7fec6c0a85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:24.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.647+0000 7fec7a4af640 1 -- 172.21.15.114:0/1917580790 >> 172.21.15.114:0/1917580790 conn(0x7fec6c008fe0 msgr2=0x7fec6c009a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:24.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.651+0000 7fec7a4af640 1 -- 172.21.15.114:0/1917580790 shutdown_connections 2024-09-13T18:29:24.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.652+0000 7fec7a4af640 1 -- 172.21.15.114:0/1917580790 wait complete. 2024-09-13T18:29:24.685 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-09-13T18:29:24.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.882+0000 7f0c69ac3640 1 -- 172.21.15.114:0/3231492312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c641074b0 msgr2=0x7f0c6410e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:24.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.882+0000 7f0c69ac3640 1 --2- 172.21.15.114:0/3231492312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c641074b0 0x7f0c6410e510 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0c50009900 tx=0x7f0c5002a420 comp rx=0 tx=0).stop 2024-09-13T18:29:24.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.883+0000 7f0c69ac3640 1 -- 172.21.15.114:0/3231492312 shutdown_connections 2024-09-13T18:29:24.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.883+0000 7f0c69ac3640 1 --2- 172.21.15.114:0/3231492312 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c64119400 0x7f0c6411b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:24.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.883+0000 7f0c69ac3640 1 --2- 172.21.15.114:0/3231492312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c64107890 0x7f0c64107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:24.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.883+0000 7f0c69ac3640 1 --2- 172.21.15.114:0/3231492312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c641074b0 0x7f0c6410e510 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:24.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.883+0000 7f0c69ac3640 1 -- 172.21.15.114:0/3231492312 >> 172.21.15.114:0/3231492312 conn(0x7f0c6406af40 msgr2=0x7f0c6406b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:24.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.887+0000 7f0c69ac3640 1 -- 172.21.15.114:0/3231492312 shutdown_connections 2024-09-13T18:29:24.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.887+0000 7f0c69ac3640 1 -- 172.21.15.114:0/3231492312 wait complete. 2024-09-13T18:29:24.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.888+0000 7f0c69ac3640 1 Processor -- start 2024-09-13T18:29:24.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.888+0000 7f0c69ac3640 1 -- start start 2024-09-13T18:29:24.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.889+0000 7f0c69ac3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c641074b0 0x7f0c641a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:24.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.889+0000 7f0c69ac3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c64107890 0x7f0c641a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:24.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.889+0000 7f0c69ac3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c64119400 0x7f0c641a9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:24.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.889+0000 7f0c69ac3640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c641a9e00 con 0x7f0c641074b0 2024-09-13T18:29:24.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.889+0000 7f0c69ac3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c641a9f70 con 0x7f0c64107890 2024-09-13T18:29:24.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.889+0000 7f0c69ac3640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c641aa130 con 0x7f0c64119400 2024-09-13T18:29:24.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.892+0000 7f0c68ac1640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c641074b0 0x7f0c641a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:24.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.892+0000 7f0c68ac1640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c641074b0 0x7f0c641a4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57454/0 (socket says 172.21.15.114:57454) 2024-09-13T18:29:24.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.892+0000 7f0c68ac1640 1 -- 172.21.15.114:0/623405018 learned_addr learned my addr 172.21.15.114:0/623405018 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:24.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.893+0000 7f0c692c2640 1 --2- 172.21.15.114:0/623405018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c64119400 0x7f0c641a9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:24.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.893+0000 7f0c63fff640 1 --2- 172.21.15.114:0/623405018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c64107890 0x7f0c641a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:24.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.893+0000 7f0c68ac1640 1 -- 172.21.15.114:0/623405018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c64119400 msgr2=0x7f0c641a9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:24.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.893+0000 7f0c68ac1640 1 --2- 172.21.15.114:0/623405018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c64119400 0x7f0c641a9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:24.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.893+0000 7f0c68ac1640 1 -- 172.21.15.114:0/623405018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c64107890 msgr2=0x7f0c641a5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:24.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.893+0000 7f0c68ac1640 1 --2- 172.21.15.114:0/623405018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c64107890 0x7f0c641a5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:24.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.893+0000 7f0c68ac1640 1 -- 172.21.15.114:0/623405018 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c50009560 con 0x7f0c641074b0 2024-09-13T18:29:24.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.893+0000 7f0c63fff640 1 --2- 172.21.15.114:0/623405018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c64107890 0x7f0c641a5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:29:24.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.893+0000 7f0c68ac1640 1 --2- 172.21.15.114:0/623405018 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c641074b0 0x7f0c641a4ed0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f0c50006010 tx=0x7f0c500055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:24.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.894+0000 7f0c61ffb640 1 -- 172.21.15.114:0/623405018 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c50004ad0 con 0x7f0c641074b0 2024-09-13T18:29:24.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.894+0000 7f0c61ffb640 1 -- 172.21.15.114:0/623405018 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0c50004c30 con 0x7f0c641074b0 2024-09-13T18:29:24.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.894+0000 7f0c61ffb640 1 -- 172.21.15.114:0/623405018 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5003c400 con 0x7f0c641074b0 2024-09-13T18:29:24.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.896+0000 7f0c69ac3640 1 -- 172.21.15.114:0/623405018 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c641aa3b0 con 0x7f0c641074b0 2024-09-13T18:29:24.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.896+0000 7f0c69ac3640 1 -- 172.21.15.114:0/623405018 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c641b1ab0 con 0x7f0c641074b0 2024-09-13T18:29:24.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.900+0000 7f0c61ffb640 1 -- 172.21.15.114:0/623405018 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f0c50039040 con 0x7f0c641074b0 2024-09-13T18:29:24.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.902+0000 7f0c61ffb640 1 --2- 172.21.15.114:0/623405018 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0c540a1410 0x7f0c540a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:24.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.902+0000 7f0c61ffb640 1 -- 172.21.15.114:0/623405018 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f0c500ee050 con 0x7f0c641074b0 2024-09-13T18:29:24.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.904+0000 7f0c63fff640 1 --2- 172.21.15.114:0/623405018 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0c540a1410 0x7f0c540a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:24.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.904+0000 7f0c63fff640 1 --2- 172.21.15.114:0/623405018 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0c540a1410 0x7f0c540a38d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0c641a6470 tx=0x7f0c58005f90 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:24.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.905+0000 7f0c6da4a740 1 Processor -- start 2024-09-13T18:29:24.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.905+0000 7f0c6da4a740 1 -- start start 2024-09-13T18:29:24.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.905+0000 7f0c6da4a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bb79e9970 0x558bb79e9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:24.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.905+0000 7f0c6da4a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bb79ea290 0x558bb79f45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:24.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.905+0000 7f0c6da4a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558bb79f4b30 0x558bb79f6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:24.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.905+0000 7f0c6da4a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bb7953cc0 con 0x558bb79ea290 2024-09-13T18:29:24.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.905+0000 7f0c6da4a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bb7951450 con 0x558bb79e9970 2024-09-13T18:29:24.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.905+0000 7f0c6da4a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bb794cd40 con 0x558bb79f4b30 2024-09-13T18:29:24.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.906+0000 7f0c68ac1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bb79e9970 0x558bb79e9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:24.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.906+0000 7f0c68ac1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bb79e9970 0x558bb79e9d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:56134/0 (socket says 172.21.15.114:56134) 2024-09-13T18:29:24.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.906+0000 7f0c68ac1640 1 -- 172.21.15.114:0/3536354005 learned_addr learned my addr 172.21.15.114:0/3536354005 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:24.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.906+0000 7f0c692c2640 1 --2- 172.21.15.114:0/3536354005 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558bb79f4b30 0x558bb79f6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:24.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.906+0000 7f0c63fff640 1 --2- 172.21.15.114:0/3536354005 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bb79ea290 0x558bb79f45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:24.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.906+0000 7f0c63fff640 1 -- 172.21.15.114:0/3536354005 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558bb79f4b30 msgr2=0x558bb79f6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:24.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.906+0000 7f0c63fff640 1 --2- 172.21.15.114:0/3536354005 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558bb79f4b30 0x558bb79f6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:24.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.906+0000 7f0c63fff640 1 -- 172.21.15.114:0/3536354005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bb79e9970 msgr2=0x558bb79e9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:24.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.906+0000 7f0c63fff640 1 --2- 172.21.15.114:0/3536354005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bb79e9970 0x558bb79e9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:24.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.906+0000 7f0c63fff640 1 -- 172.21.15.114:0/3536354005 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558bb7914620 con 0x558bb79ea290 2024-09-13T18:29:24.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.907+0000 7f0c63fff640 1 --2- 172.21.15.114:0/3536354005 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bb79ea290 0x558bb79f45f0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f0c58010e30 tx=0x7f0c58013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b13722d870d098c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:24.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.908+0000 7f0c4effd640 1 -- 172.21.15.114:0/3536354005 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c58004230 con 0x558bb79ea290 2024-09-13T18:29:24.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.908+0000 7f0c4effd640 1 -- 172.21.15.114:0/3536354005 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0c58012e60 con 0x558bb79ea290 2024-09-13T18:29:24.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.908+0000 7f0c4effd640 1 -- 172.21.15.114:0/3536354005 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5801f6b0 con 0x558bb79ea290 2024-09-13T18:29:24.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.909+0000 7f0c6da4a740 1 -- 172.21.15.114:0/3536354005 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bb79ea290 msgr2=0x558bb79f45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:24.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.909+0000 7f0c6da4a740 1 --2- 172.21.15.114:0/3536354005 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bb79ea290 0x558bb79f45f0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f0c58010e30 tx=0x7f0c58013000 comp rx=0 tx=0).stop 2024-09-13T18:29:24.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.909+0000 7f0c6da4a740 1 -- 172.21.15.114:0/3536354005 shutdown_connections 2024-09-13T18:29:24.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.909+0000 7f0c6da4a740 1 --2- 172.21.15.114:0/3536354005 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558bb79f4b30 0x558bb79f6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:24.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.909+0000 7f0c6da4a740 1 --2- 172.21.15.114:0/3536354005 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bb79ea290 0x558bb79f45f0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:24.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.909+0000 7f0c6da4a740 1 --2- 172.21.15.114:0/3536354005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bb79e9970 0x558bb79e9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:24.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.909+0000 7f0c6da4a740 1 -- 172.21.15.114:0/3536354005 >> 172.21.15.114:0/3536354005 conn(0x558bb79e5ee0 msgr2=0x558bb79e62c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:24.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.909+0000 7f0c6da4a740 1 -- 172.21.15.114:0/3536354005 shutdown_connections 2024-09-13T18:29:24.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.909+0000 7f0c6da4a740 1 -- 172.21.15.114:0/3536354005 wait complete. 2024-09-13T18:29:24.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.909+0000 7f0c6da4a740 1 -- start start 2024-09-13T18:29:24.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.910+0000 7f0c6da4a740 1 Processor -- start 2024-09-13T18:29:24.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.910+0000 7f0c6da4a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bb79e9970 0x558bb79f4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:24.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.910+0000 7f0c6da4a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558bb79ea290 0x558bb79f14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:24.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.910+0000 7f0c6da4a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bb79f4b30 0x558bb79f1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:24.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.910+0000 7f0c6da4a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bb790b180 con 0x558bb79f4b30 2024-09-13T18:29:24.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.910+0000 7f0c6da4a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bb7625430 con 0x558bb79e9970 2024-09-13T18:29:24.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.910+0000 7f0c6da4a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bb78d05e0 con 0x558bb79ea290 2024-09-13T18:29:24.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.910+0000 7f0c6da4a740 10 client.0 _ll_register_callbacks cb 0x558bb793caa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:29:24.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.911+0000 7f0c68ac1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bb79e9970 0x558bb79f4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:24.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.911+0000 7f0c68ac1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bb79e9970 0x558bb79f4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:56146/0 (socket says 172.21.15.114:56146) 2024-09-13T18:29:24.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.911+0000 7f0c68ac1640 1 -- 172.21.15.114:0/2693021793 learned_addr learned my addr 172.21.15.114:0/2693021793 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:24.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.911+0000 7f0c63fff640 1 --2- 172.21.15.114:0/2693021793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558bb79ea290 0x558bb79f14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:24.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.911+0000 7f0c692c2640 1 --2- 172.21.15.114:0/2693021793 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bb79f4b30 0x558bb79f1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:24.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.911+0000 7f0c68ac1640 1 -- 172.21.15.114:0/2693021793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558bb79ea290 msgr2=0x558bb79f14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:24.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.911+0000 7f0c68ac1640 1 --2- 172.21.15.114:0/2693021793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558bb79ea290 0x558bb79f14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:24.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.911+0000 7f0c68ac1640 1 -- 172.21.15.114:0/2693021793 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bb79f4b30 msgr2=0x558bb79f1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:24.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.911+0000 7f0c68ac1640 1 --2- 172.21.15.114:0/2693021793 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bb79f4b30 0x558bb79f1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:24.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.911+0000 7f0c68ac1640 1 -- 172.21.15.114:0/2693021793 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c58010b10 con 0x558bb79e9970 2024-09-13T18:29:24.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.911+0000 7f0c692c2640 1 --2- 172.21.15.114:0/2693021793 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bb79f4b30 0x558bb79f1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:29:24.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.912+0000 7f0c68ac1640 1 --2- 172.21.15.114:0/2693021793 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bb79e9970 0x558bb79f4150 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0c500baa60 tx=0x7f0c50004ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=912c771137ca7e5e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:24.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.912+0000 7f0c63fff640 1 --2- 172.21.15.114:0/2693021793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558bb79ea290 0x558bb79f14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:29:24.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.912+0000 7f0c43fff640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:29:24.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.912+0000 7f0c43fff640 1 -- 172.21.15.114:0/2693021793 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c500eb830 con 0x558bb79e9970 2024-09-13T18:29:24.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.912+0000 7f0c6da4a740 1 -- 172.21.15.114:0/2693021793 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558bb7976b80 con 0x558bb79e9970 2024-09-13T18:29:24.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.912+0000 7f0c43fff640 1 -- 172.21.15.114:0/2693021793 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f0c500c31e0 con 0x558bb79e9970 2024-09-13T18:29:24.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.912+0000 7f0c43fff640 1 -- 172.21.15.114:0/2693021793 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c500bafc0 con 0x558bb79e9970 2024-09-13T18:29:24.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.913+0000 7f0c6da4a740 10 client.27011 fetch_fsmap learned FSMap version 34 2024-09-13T18:29:24.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.913+0000 7f0c6da4a740 1 -- 172.21.15.114:0/2693021793 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x558bb794b3c0 con 0x558bb79e9970 2024-09-13T18:29:24.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.913+0000 7f0c43fff640 1 -- 172.21.15.114:0/2693021793 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0c500bb120 con 0x558bb79e9970 2024-09-13T18:29:24.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.914+0000 7f0c43fff640 1 -- 172.21.15.114:0/2693021793 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0c500b6b50 con 0x558bb79e9970 2024-09-13T18:29:24.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.914+0000 7f0c6da4a740 10 client.27011 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:29:24.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.914+0000 7f0c6da4a740 10 client.27011 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-13T18:29:24.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.914+0000 7f0c6da4a740 20 client.27011 populate_metadata read hostname 'smithi114' 2024-09-13T18:29:24.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.914+0000 7f0c6da4a740 1 --2- 172.21.15.114:0/2693021793 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x558bb7a040d0 0x558bb7a064e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:24.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.915+0000 7f0c6da4a740 4 client.27011 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:29:24.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.915+0000 7f0c6da4a740 1 -- 172.21.15.114:0/2693021793 --> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558bb7976b80 con 0x558bb7a040d0 2024-09-13T18:29:24.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.916+0000 7f0c63fff640 1 --2- 172.21.15.114:0/2693021793 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x558bb7a040d0 0x558bb7a064e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:24.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.921+0000 7f0c63fff640 1 --2- 172.21.15.114:0/2693021793 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x558bb7a040d0 0x558bb7a064e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e013e29a72b8067f server_cookie=6cf08f4796dcb928 in_seq=0 out_seq=0 2024-09-13T18:29:24.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.921+0000 7f0c43fff640 10 client.27011 ms_handle_connect on v2:172.21.15.114:6840/1956923446 2024-09-13T18:29:24.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.926+0000 7f0c43fff640 1 -- 172.21.15.114:0/2693021793 <== mds.4 v2:172.21.15.114:6840/1956923446 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x558bb7976b80 con 0x558bb7a040d0 2024-09-13T18:29:24.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.926+0000 7f0c43fff640 10 client.27011 handle_command_reply: tid=0 2024-09-13T18:29:24.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.926+0000 7f0c6da4a740 1 client.27011 shutdown 2024-09-13T18:29:24.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.935+0000 7f0c6da4a740 1 -- 172.21.15.114:0/2693021793 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bb79e9970 msgr2=0x558bb79f4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:24.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.935+0000 7f0c6da4a740 1 --2- 172.21.15.114:0/2693021793 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bb79e9970 0x558bb79f4150 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0c500baa60 tx=0x7f0c50004ef0 comp rx=0 tx=0).stop 2024-09-13T18:29:24.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.935+0000 7f0c6da4a740 1 -- 172.21.15.114:0/2693021793 shutdown_connections 2024-09-13T18:29:24.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.935+0000 7f0c6da4a740 1 --2- 172.21.15.114:0/2693021793 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x558bb7a040d0 0x558bb7a064e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:24.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.935+0000 7f0c6da4a740 1 --2- 172.21.15.114:0/2693021793 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bb79f4b30 0x558bb79f1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:24.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.935+0000 7f0c6da4a740 1 --2- 172.21.15.114:0/2693021793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558bb79ea290 0x558bb79f14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:24.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.935+0000 7f0c6da4a740 1 --2- 172.21.15.114:0/2693021793 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bb79e9970 0x558bb79f4150 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:24.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.935+0000 7f0c6da4a740 1 -- 172.21.15.114:0/2693021793 >> 172.21.15.114:0/2693021793 conn(0x558bb79e5ee0 msgr2=0x558bb79e5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:24.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.935+0000 7f0c43fff640 0 client.27011 ms_handle_reset on v2:172.21.15.114:6840/1956923446 2024-09-13T18:29:24.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.943+0000 7f0c6da4a740 1 -- 172.21.15.114:0/2693021793 shutdown_connections 2024-09-13T18:29:24.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.948+0000 7f0c6da4a740 1 -- 172.21.15.114:0/2693021793 wait complete. 2024-09-13T18:29:24.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.954+0000 7f0c6da4a740 20 client.27011 trim_cache size 0 max 16384 2024-09-13T18:29:24.972 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:29:24 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/1917580790' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:29:24.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.984+0000 7f0c6da4a740 1 Processor -- start 2024-09-13T18:29:24.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.984+0000 7f0c6da4a740 1 -- start start 2024-09-13T18:29:24.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.984+0000 7f0c6da4a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bb79e9970 0x558bb79ecca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:24.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.985+0000 7f0c6da4a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bb79ea290 0x558bb79f5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:24.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.985+0000 7f0c6da4a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558bb79f4b30 0x558bb79f6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:24.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.985+0000 7f0c6da4a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bb79f4730 con 0x558bb79e9970 2024-09-13T18:29:24.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.985+0000 7f0c6da4a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bb7976bb0 con 0x558bb79ea290 2024-09-13T18:29:24.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.985+0000 7f0c692c2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558bb79f4b30 0x558bb79f6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:24.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.985+0000 7f0c692c2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558bb79f4b30 0x558bb79f6460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:49104/0 (socket says 172.21.15.114:49104) 2024-09-13T18:29:24.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.985+0000 7f0c692c2640 1 -- 172.21.15.114:0/2146488438 learned_addr learned my addr 172.21.15.114:0/2146488438 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:24.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.985+0000 7f0c6da4a740 1 -- 172.21.15.114:0/2146488438 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bb79f49a0 con 0x558bb79f4b30 2024-09-13T18:29:24.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.985+0000 7f0c63fff640 1 --2- 172.21.15.114:0/2146488438 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bb79ea290 0x558bb79f5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:24.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.986+0000 7f0c63fff640 1 -- 172.21.15.114:0/2146488438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558bb79f4b30 msgr2=0x558bb79f6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:24.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.986+0000 7f0c63fff640 1 --2- 172.21.15.114:0/2146488438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558bb79f4b30 0x558bb79f6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:24.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.986+0000 7f0c63fff640 1 -- 172.21.15.114:0/2146488438 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bb79e9970 msgr2=0x558bb79ecca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:24.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.986+0000 7f0c63fff640 1 --2- 172.21.15.114:0/2146488438 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bb79e9970 0x558bb79ecca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:24.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.986+0000 7f0c63fff640 1 -- 172.21.15.114:0/2146488438 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558bb78ed770 con 0x558bb79ea290 2024-09-13T18:29:24.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.986+0000 7f0c63fff640 1 --2- 172.21.15.114:0/2146488438 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bb79ea290 0x558bb79f5f20 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0c58009800 tx=0x7f0c58016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59a545aa7c6dc556 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:24.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.986+0000 7f0c4f7fe640 1 -- 172.21.15.114:0/2146488438 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c58003b30 con 0x558bb79ea290 2024-09-13T18:29:24.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.986+0000 7f0c4f7fe640 1 -- 172.21.15.114:0/2146488438 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0c5802e040 con 0x558bb79ea290 2024-09-13T18:29:24.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.987+0000 7f0c4f7fe640 1 -- 172.21.15.114:0/2146488438 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5801fa00 con 0x558bb79ea290 2024-09-13T18:29:24.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.988+0000 7f0c6da4a740 1 -- 172.21.15.114:0/2146488438 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bb79ea290 msgr2=0x558bb79f5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:24.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.988+0000 7f0c6da4a740 1 --2- 172.21.15.114:0/2146488438 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bb79ea290 0x558bb79f5f20 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0c58009800 tx=0x7f0c58016000 comp rx=0 tx=0).stop 2024-09-13T18:29:24.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.988+0000 7f0c6da4a740 1 -- 172.21.15.114:0/2146488438 shutdown_connections 2024-09-13T18:29:24.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.988+0000 7f0c6da4a740 1 --2- 172.21.15.114:0/2146488438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558bb79f4b30 0x558bb79f6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:24.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.988+0000 7f0c6da4a740 1 --2- 172.21.15.114:0/2146488438 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bb79ea290 0x558bb79f5f20 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:24.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.988+0000 7f0c6da4a740 1 --2- 172.21.15.114:0/2146488438 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bb79e9970 0x558bb79ecca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:24.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.988+0000 7f0c6da4a740 1 -- 172.21.15.114:0/2146488438 >> 172.21.15.114:0/2146488438 conn(0x558bb79e5ee0 msgr2=0x558bb79ec590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:24.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.992+0000 7f0c6da4a740 1 -- 172.21.15.114:0/2146488438 shutdown_connections 2024-09-13T18:29:24.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.993+0000 7f0c6da4a740 1 -- 172.21.15.114:0/2146488438 wait complete. 2024-09-13T18:29:24.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.994+0000 7f0c6da4a740 1 -- start start 2024-09-13T18:29:24.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.994+0000 7f0c6da4a740 1 Processor -- start 2024-09-13T18:29:24.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.995+0000 7f0c6da4a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558bb79e9970 0x558bb79fb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:24.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.995+0000 7f0c6da4a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bb79ea290 0x558bb79f9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:24.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.995+0000 7f0c6da4a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bb79f4b30 0x558bb79fa3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:24.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.996+0000 7f0c68ac1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558bb79e9970 0x558bb79fb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:24.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.996+0000 7f0c68ac1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558bb79e9970 0x558bb79fb970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:49114/0 (socket says 172.21.15.114:49114) 2024-09-13T18:29:25.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.996+0000 7f0c692c2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bb79f4b30 0x558bb79fa3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:25.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.996+0000 7f0c692c2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bb79f4b30 0x558bb79fa3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57486/0 (socket says 172.21.15.114:57486) 2024-09-13T18:29:25.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.996+0000 7f0c63fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bb79ea290 0x558bb79f9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:25.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.996+0000 7f0c63fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bb79ea290 0x558bb79f9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:56152/0 (socket says 172.21.15.114:56152) 2024-09-13T18:29:25.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.996+0000 7f0c68ac1640 1 -- 172.21.15.114:0/3266433475 learned_addr learned my addr 172.21.15.114:0/3266433475 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:25.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.996+0000 7f0c6da4a740 1 -- 172.21.15.114:0/3266433475 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bb79fa900 con 0x558bb79f4b30 2024-09-13T18:29:25.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.996+0000 7f0c6da4a740 1 -- 172.21.15.114:0/3266433475 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bb79faa70 con 0x558bb79ea290 2024-09-13T18:29:25.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.996+0000 7f0c6da4a740 1 -- 172.21.15.114:0/3266433475 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bb79fac30 con 0x558bb79e9970 2024-09-13T18:29:25.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.996+0000 7f0c6da4a740 10 client.0 _ll_register_callbacks cb 0x7f0c58016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:29:25.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.997+0000 7f0c692c2640 1 -- 172.21.15.114:0/3266433475 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558bb79e9970 msgr2=0x558bb79fb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:25.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.997+0000 7f0c692c2640 1 --2- 172.21.15.114:0/3266433475 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558bb79e9970 0x558bb79fb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.997+0000 7f0c692c2640 1 -- 172.21.15.114:0/3266433475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bb79ea290 msgr2=0x558bb79f9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:25.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.997+0000 7f0c692c2640 1 --2- 172.21.15.114:0/3266433475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bb79ea290 0x558bb79f9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.997+0000 7f0c692c2640 1 -- 172.21.15.114:0/3266433475 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558bb7914620 con 0x558bb79f4b30 2024-09-13T18:29:25.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.997+0000 7f0c692c2640 1 --2- 172.21.15.114:0/3266433475 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bb79f4b30 0x558bb79fa3c0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c009fd0 tx=0x7f0c5c007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ff658ebacbebd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:25.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:24.998+0000 7f0c43fff640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:29:25.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.001+0000 7f0c43fff640 1 -- 172.21.15.114:0/3266433475 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c018820 con 0x558bb79f4b30 2024-09-13T18:29:25.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.001+0000 7f0c43fff640 1 -- 172.21.15.114:0/3266433475 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f0c5c018980 con 0x558bb79f4b30 2024-09-13T18:29:25.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.001+0000 7f0c6da4a740 1 -- 172.21.15.114:0/3266433475 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0c58012e60 con 0x558bb79f4b30 2024-09-13T18:29:25.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.003+0000 7f0c43fff640 1 -- 172.21.15.114:0/3266433475 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c5c018b40 con 0x558bb79f4b30 2024-09-13T18:29:25.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.006+0000 7f0c6da4a740 10 client.17589 fetch_fsmap learned FSMap version 34 2024-09-13T18:29:25.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.006+0000 7f0c6da4a740 1 -- 172.21.15.114:0/3266433475 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x558bb7912e80 con 0x558bb79f4b30 2024-09-13T18:29:25.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.007+0000 7f0c43fff640 1 -- 172.21.15.114:0/3266433475 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0c5c012040 con 0x558bb79f4b30 2024-09-13T18:29:25.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.007+0000 7f0c43fff640 1 -- 172.21.15.114:0/3266433475 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0c5c008040 con 0x558bb79f4b30 2024-09-13T18:29:25.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.007+0000 7f0c6da4a740 10 client.17589 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:29:25.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.007+0000 7f0c6da4a740 10 client.17589 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-13T18:29:25.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.007+0000 7f0c6da4a740 20 client.17589 populate_metadata read hostname 'smithi114' 2024-09-13T18:29:25.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.007+0000 7f0c6da4a740 1 --2- 172.21.15.114:0/3266433475 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x558bb7a5ba30 0x558bb7a5de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:25.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.007+0000 7f0c6da4a740 4 client.17589 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:29:25.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.007+0000 7f0c6da4a740 1 -- 172.21.15.114:0/3266433475 --> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f0c58012e60 con 0x558bb7a5ba30 2024-09-13T18:29:25.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.009+0000 7f0c68ac1640 1 --2- 172.21.15.114:0/3266433475 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x558bb7a5ba30 0x558bb7a5de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:25.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.010+0000 7f0c68ac1640 1 --2- 172.21.15.114:0/3266433475 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x558bb7a5ba30 0x558bb7a5de50 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3068fd47b7f5821f server_cookie=3865ec3975587245 in_seq=0 out_seq=0 2024-09-13T18:29:25.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.010+0000 7f0c43fff640 10 client.17589 ms_handle_connect on v2:172.21.15.114:6840/1956923446 2024-09-13T18:29:25.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.010+0000 7f0c43fff640 1 -- 172.21.15.114:0/3266433475 <== mds.4 v2:172.21.15.114:6840/1956923446 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f0c58012e60 con 0x558bb7a5ba30 2024-09-13T18:29:25.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.010+0000 7f0c43fff640 10 client.17589 handle_command_reply: tid=0 2024-09-13T18:29:25.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.010+0000 7f0c6da4a740 1 client.17589 shutdown 2024-09-13T18:29:25.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.012+0000 7f0c6da4a740 1 -- 172.21.15.114:0/3266433475 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bb79f4b30 msgr2=0x558bb79fa3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:25.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.012+0000 7f0c6da4a740 1 --2- 172.21.15.114:0/3266433475 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bb79f4b30 0x558bb79fa3c0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c009fd0 tx=0x7f0c5c007680 comp rx=0 tx=0).stop 2024-09-13T18:29:25.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.012+0000 7f0c6da4a740 1 -- 172.21.15.114:0/3266433475 shutdown_connections 2024-09-13T18:29:25.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.012+0000 7f0c6da4a740 1 --2- 172.21.15.114:0/3266433475 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x558bb7a5ba30 0x558bb7a5de50 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.013+0000 7f0c6da4a740 1 --2- 172.21.15.114:0/3266433475 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bb79f4b30 0x558bb79fa3c0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.013+0000 7f0c6da4a740 1 --2- 172.21.15.114:0/3266433475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bb79ea290 0x558bb79f9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.013+0000 7f0c6da4a740 1 --2- 172.21.15.114:0/3266433475 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558bb79e9970 0x558bb79fb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.013+0000 7f0c6da4a740 1 -- 172.21.15.114:0/3266433475 >> 172.21.15.114:0/3266433475 conn(0x558bb79e5ee0 msgr2=0x558bb79f2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:25.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.013+0000 7f0c43fff640 0 client.17589 ms_handle_reset on v2:172.21.15.114:6840/1956923446 2024-09-13T18:29:25.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.013+0000 7f0c6da4a740 1 -- 172.21.15.114:0/3266433475 shutdown_connections 2024-09-13T18:29:25.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.013+0000 7f0c6da4a740 1 -- 172.21.15.114:0/3266433475 wait complete. 2024-09-13T18:29:25.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.013+0000 7f0c6da4a740 20 client.17589 trim_cache size 0 max 16384 2024-09-13T18:29:25.017 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:29:25.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.022+0000 7f0c69ac3640 1 -- 172.21.15.114:0/623405018 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0c540a1410 msgr2=0x7f0c540a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:25.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.022+0000 7f0c69ac3640 1 --2- 172.21.15.114:0/623405018 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0c540a1410 0x7f0c540a38d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0c641a6470 tx=0x7f0c58005f90 comp rx=0 tx=0).stop 2024-09-13T18:29:25.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.022+0000 7f0c69ac3640 1 -- 172.21.15.114:0/623405018 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c641074b0 msgr2=0x7f0c641a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:25.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.022+0000 7f0c69ac3640 1 --2- 172.21.15.114:0/623405018 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c641074b0 0x7f0c641a4ed0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f0c50006010 tx=0x7f0c500055a0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.023+0000 7f0c69ac3640 1 -- 172.21.15.114:0/623405018 shutdown_connections 2024-09-13T18:29:25.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.023+0000 7f0c69ac3640 1 --2- 172.21.15.114:0/623405018 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0c540a1410 0x7f0c540a38d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.023+0000 7f0c69ac3640 1 --2- 172.21.15.114:0/623405018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c64119400 0x7f0c641a9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.023+0000 7f0c69ac3640 1 --2- 172.21.15.114:0/623405018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c64107890 0x7f0c641a5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.023+0000 7f0c69ac3640 1 --2- 172.21.15.114:0/623405018 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c641074b0 0x7f0c641a4ed0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.023+0000 7f0c69ac3640 1 -- 172.21.15.114:0/623405018 >> 172.21.15.114:0/623405018 conn(0x7f0c6406af40 msgr2=0x7f0c64119aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:25.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.023+0000 7f0c69ac3640 1 -- 172.21.15.114:0/623405018 shutdown_connections 2024-09-13T18:29:25.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.023+0000 7f0c69ac3640 1 -- 172.21.15.114:0/623405018 wait complete. 2024-09-13T18:29:25.060 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-09-13T18:29:25.118 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:29:24 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/1917580790' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:29:25.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:29:24 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/1917580790' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:29:25.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.330+0000 7f7839e10640 1 -- 172.21.15.114:0/2453031425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78341074b0 msgr2=0x7f783410e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:25.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.330+0000 7f7839e10640 1 --2- 172.21.15.114:0/2453031425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78341074b0 0x7f783410e510 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7828009900 tx=0x7f782802a410 comp rx=0 tx=0).stop 2024-09-13T18:29:25.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.333+0000 7f7839e10640 1 -- 172.21.15.114:0/2453031425 shutdown_connections 2024-09-13T18:29:25.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.333+0000 7f7839e10640 1 --2- 172.21.15.114:0/2453031425 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7834119400 0x7f783411b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.333+0000 7f7839e10640 1 --2- 172.21.15.114:0/2453031425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7834107890 0x7f7834107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.333+0000 7f7839e10640 1 --2- 172.21.15.114:0/2453031425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78341074b0 0x7f783410e510 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.333+0000 7f7839e10640 1 -- 172.21.15.114:0/2453031425 >> 172.21.15.114:0/2453031425 conn(0x7f783406af40 msgr2=0x7f783406b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:25.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.345+0000 7f7839e10640 1 -- 172.21.15.114:0/2453031425 shutdown_connections 2024-09-13T18:29:25.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.346+0000 7f7839e10640 1 -- 172.21.15.114:0/2453031425 wait complete. 2024-09-13T18:29:25.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.347+0000 7f7839e10640 1 Processor -- start 2024-09-13T18:29:25.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.351+0000 7f7839e10640 1 -- start start 2024-09-13T18:29:25.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.351+0000 7f7839e10640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78341074b0 0x7f78341168d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:25.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.351+0000 7f7839e10640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7834107890 0x7f7834116e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:25.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.351+0000 7f7839e10640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7834119400 0x7f783410fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:25.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.351+0000 7f7839e10640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7834110310 con 0x7f7834119400 2024-09-13T18:29:25.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.351+0000 7f7839e10640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7834110480 con 0x7f78341074b0 2024-09-13T18:29:25.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.351+0000 7f7839e10640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7834110640 con 0x7f7834107890 2024-09-13T18:29:25.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.351+0000 7f783960f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7834119400 0x7f783410fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:25.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.351+0000 7f783960f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7834119400 0x7f783410fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57500/0 (socket says 172.21.15.114:57500) 2024-09-13T18:29:25.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.351+0000 7f783960f640 1 -- 172.21.15.114:0/2093475553 learned_addr learned my addr 172.21.15.114:0/2093475553 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:25.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.352+0000 7f7838e0e640 1 --2- 172.21.15.114:0/2093475553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78341074b0 0x7f78341168d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:25.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.354+0000 7f7833fff640 1 --2- 172.21.15.114:0/2093475553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7834107890 0x7f7834116e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:25.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.355+0000 7f7838e0e640 1 -- 172.21.15.114:0/2093475553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7834107890 msgr2=0x7f7834116e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:25.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.355+0000 7f7838e0e640 1 --2- 172.21.15.114:0/2093475553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7834107890 0x7f7834116e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.355+0000 7f7838e0e640 1 -- 172.21.15.114:0/2093475553 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7834119400 msgr2=0x7f783410fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:25.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.355+0000 7f7838e0e640 1 --2- 172.21.15.114:0/2093475553 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7834119400 0x7f783410fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.355+0000 7f7838e0e640 1 -- 172.21.15.114:0/2093475553 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7828009560 con 0x7f78341074b0 2024-09-13T18:29:25.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.356+0000 7f7838e0e640 1 --2- 172.21.15.114:0/2093475553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78341074b0 0x7f78341168d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f782802aa00 tx=0x7f78280048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:25.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.357+0000 7f7831ffb640 1 -- 172.21.15.114:0/2093475553 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78280035f0 con 0x7f78341074b0 2024-09-13T18:29:25.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.357+0000 7f7831ffb640 1 -- 172.21.15.114:0/2093475553 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7828003be0 con 0x7f78341074b0 2024-09-13T18:29:25.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.357+0000 7f7831ffb640 1 -- 172.21.15.114:0/2093475553 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782803ca80 con 0x7f78341074b0 2024-09-13T18:29:25.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.357+0000 7f7839e10640 1 -- 172.21.15.114:0/2093475553 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7834114460 con 0x7f78341074b0 2024-09-13T18:29:25.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.357+0000 7f7839e10640 1 -- 172.21.15.114:0/2093475553 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7834114890 con 0x7f78341074b0 2024-09-13T18:29:25.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.366+0000 7f783de75740 1 Processor -- start 2024-09-13T18:29:25.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.368+0000 7f7831ffb640 1 -- 172.21.15.114:0/2093475553 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f782803cbe0 con 0x7f78341074b0 2024-09-13T18:29:25.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.369+0000 7f7831ffb640 1 --2- 172.21.15.114:0/2093475553 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f781c0a1410 0x7f781c0a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:25.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.369+0000 7f7831ffb640 1 -- 172.21.15.114:0/2093475553 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f78280ee170 con 0x7f78341074b0 2024-09-13T18:29:25.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.370+0000 7f783de75740 1 -- start start 2024-09-13T18:29:25.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.370+0000 7f783de75740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff69ba0970 0x55ff69ba0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:25.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.370+0000 7f783de75740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff69ba1290 0x55ff69bab5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:25.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.370+0000 7f783de75740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff69babb30 0x55ff69badf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:25.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.370+0000 7f783de75740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff69b0acc0 con 0x55ff69ba1290 2024-09-13T18:29:25.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.370+0000 7f783de75740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff69b08450 con 0x55ff69ba0970 2024-09-13T18:29:25.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.370+0000 7f783de75740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff69b03d40 con 0x55ff69babb30 2024-09-13T18:29:25.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.372+0000 7f7833fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff69ba1290 0x55ff69bab5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:25.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.372+0000 7f7833fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff69ba1290 0x55ff69bab5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57504/0 (socket says 172.21.15.114:57504) 2024-09-13T18:29:25.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.372+0000 7f7833fff640 1 -- 172.21.15.114:0/468530830 learned_addr learned my addr 172.21.15.114:0/468530830 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:25.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.375+0000 7f7838e0e640 1 --2- 172.21.15.114:0/468530830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff69ba0970 0x55ff69ba0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:25.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.376+0000 7f783960f640 1 --2- 172.21.15.114:0/468530830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff69babb30 0x55ff69badf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:25.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.376+0000 7f783960f640 1 --2- 172.21.15.114:0/2093475553 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f781c0a1410 0x7f781c0a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:25.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.377+0000 7f7838e0e640 1 -- 172.21.15.114:0/468530830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff69babb30 msgr2=0x55ff69badf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:25.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.378+0000 7f7838e0e640 1 --2- 172.21.15.114:0/468530830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff69babb30 0x55ff69badf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.378+0000 7f7838e0e640 1 -- 172.21.15.114:0/468530830 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff69ba1290 msgr2=0x55ff69bab5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:25.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.378+0000 7f7838e0e640 1 --2- 172.21.15.114:0/468530830 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff69ba1290 0x55ff69bab5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.378+0000 7f7838e0e640 1 -- 172.21.15.114:0/468530830 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff69acb620 con 0x55ff69ba0970 2024-09-13T18:29:25.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.379+0000 7f7838e0e640 1 --2- 172.21.15.114:0/468530830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff69ba0970 0x55ff69ba0d50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f78280eb4c0 tx=0x7f78280ba9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c6f972a529fe47 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:25.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.386+0000 7f781affd640 1 -- 172.21.15.114:0/468530830 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78280bb580 con 0x55ff69ba0970 2024-09-13T18:29:25.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.386+0000 7f781affd640 1 -- 172.21.15.114:0/468530830 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f78280bbe90 con 0x55ff69ba0970 2024-09-13T18:29:25.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.387+0000 7f783de75740 1 -- 172.21.15.114:0/468530830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff69ba0970 msgr2=0x55ff69ba0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:25.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.387+0000 7f783de75740 1 --2- 172.21.15.114:0/468530830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff69ba0970 0x55ff69ba0d50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f78280eb4c0 tx=0x7f78280ba9c0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.387+0000 7f783960f640 1 --2- 172.21.15.114:0/2093475553 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f781c0a1410 0x7f781c0a38d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x55ff69b9e260 tx=0x7f782c009490 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:25.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.388+0000 7f783de75740 1 -- 172.21.15.114:0/468530830 shutdown_connections 2024-09-13T18:29:25.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.388+0000 7f783de75740 1 --2- 172.21.15.114:0/468530830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff69babb30 0x55ff69badf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.388+0000 7f783de75740 1 --2- 172.21.15.114:0/468530830 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff69ba1290 0x55ff69bab5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.388+0000 7f783de75740 1 --2- 172.21.15.114:0/468530830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff69ba0970 0x55ff69ba0d50 secure :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f78280eb4c0 tx=0x7f78280ba9c0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.388+0000 7f783de75740 1 -- 172.21.15.114:0/468530830 >> 172.21.15.114:0/468530830 conn(0x55ff69b9cee0 msgr2=0x55ff69b9d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:25.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.397+0000 7f783de75740 1 -- 172.21.15.114:0/468530830 shutdown_connections 2024-09-13T18:29:25.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.399+0000 7f783de75740 1 -- 172.21.15.114:0/468530830 wait complete. 2024-09-13T18:29:25.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.399+0000 7f783de75740 1 -- start start 2024-09-13T18:29:25.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.400+0000 7f783de75740 1 Processor -- start 2024-09-13T18:29:25.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.400+0000 7f783de75740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff69ba1290 0x55ff69bab220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:25.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.400+0000 7f783de75740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff69babb30 0x55ff69ba85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:25.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.400+0000 7f783de75740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff69ba8b20 0x55ff69ba8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:25.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.401+0000 7f783de75740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff69ac2180 con 0x55ff69ba1290 2024-09-13T18:29:25.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.401+0000 7f783de75740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff697dc430 con 0x55ff69ba8b20 2024-09-13T18:29:25.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.401+0000 7f783de75740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff69a875e0 con 0x55ff69babb30 2024-09-13T18:29:25.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.401+0000 7f783de75740 10 client.0 _ll_register_callbacks cb 0x55ff69af3aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:29:25.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.402+0000 7f7838e0e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff69ba1290 0x55ff69bab220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:25.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.402+0000 7f7838e0e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff69ba1290 0x55ff69bab220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57512/0 (socket says 172.21.15.114:57512) 2024-09-13T18:29:25.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.402+0000 7f7838e0e640 1 -- 172.21.15.114:0/596344434 learned_addr learned my addr 172.21.15.114:0/596344434 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:25.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.402+0000 7f7833fff640 1 --2- 172.21.15.114:0/596344434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff69babb30 0x55ff69ba85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:25.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.402+0000 7f783960f640 1 --2- 172.21.15.114:0/596344434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff69ba8b20 0x55ff69ba8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:25.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.403+0000 7f7838e0e640 1 -- 172.21.15.114:0/596344434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff69babb30 msgr2=0x55ff69ba85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:25.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.403+0000 7f7838e0e640 1 --2- 172.21.15.114:0/596344434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff69babb30 0x55ff69ba85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.403+0000 7f7838e0e640 1 -- 172.21.15.114:0/596344434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff69ba8b20 msgr2=0x55ff69ba8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:25.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.403+0000 7f7838e0e640 1 --2- 172.21.15.114:0/596344434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff69ba8b20 0x55ff69ba8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.403+0000 7f7838e0e640 1 -- 172.21.15.114:0/596344434 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7834114890 con 0x55ff69ba1290 2024-09-13T18:29:25.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.403+0000 7f7838e0e640 1 --2- 172.21.15.114:0/596344434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff69ba1290 0x55ff69bab220 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f78280eb490 tx=0x7f78280b7820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b4dc72d5169506 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:25.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.403+0000 7f7813fff640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:29:25.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.403+0000 7f7813fff640 1 -- 172.21.15.114:0/596344434 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78280bae80 con 0x55ff69ba1290 2024-09-13T18:29:25.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.403+0000 7f7813fff640 1 -- 172.21.15.114:0/596344434 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f78280becb0 con 0x55ff69ba1290 2024-09-13T18:29:25.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.404+0000 7f783de75740 1 -- 172.21.15.114:0/596344434 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ff69b2db80 con 0x55ff69ba1290 2024-09-13T18:29:25.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.406+0000 7f7813fff640 1 -- 172.21.15.114:0/596344434 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f78280c3300 con 0x55ff69ba1290 2024-09-13T18:29:25.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.407+0000 7f783de75740 10 client.17598 fetch_fsmap learned FSMap version 34 2024-09-13T18:29:25.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.407+0000 7f783de75740 1 -- 172.21.15.114:0/596344434 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55ff69b023c0 con 0x55ff69ba1290 2024-09-13T18:29:25.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.408+0000 7f7813fff640 1 -- 172.21.15.114:0/596344434 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f78280c9020 con 0x55ff69ba1290 2024-09-13T18:29:25.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.409+0000 7f7813fff640 1 -- 172.21.15.114:0/596344434 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f78280bb0e0 con 0x55ff69ba1290 2024-09-13T18:29:25.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.409+0000 7f783de75740 10 client.17598 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:29:25.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.409+0000 7f783de75740 10 client.17598 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-13T18:29:25.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.409+0000 7f783de75740 20 client.17598 populate_metadata read hostname 'smithi114' 2024-09-13T18:29:25.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.409+0000 7f783de75740 1 --2- 172.21.15.114:0/596344434 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55ff69bbb290 0x55ff69bbd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:25.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.409+0000 7f783de75740 4 client.17598 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:29:25.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.409+0000 7f783de75740 1 -- 172.21.15.114:0/596344434 --> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ff69af61f0 con 0x55ff69bbb290 2024-09-13T18:29:25.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.412+0000 7f7833fff640 1 --2- 172.21.15.114:0/596344434 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55ff69bbb290 0x55ff69bbd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:25.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.415+0000 7f7833fff640 1 --2- 172.21.15.114:0/596344434 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55ff69bbb290 0x55ff69bbd6b0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4bd790503b29fb84 server_cookie=2a8b1ad27af02749 in_seq=0 out_seq=0 2024-09-13T18:29:25.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.416+0000 7f7813fff640 10 client.17598 ms_handle_connect on v2:172.21.15.134:6834/1150003275 2024-09-13T18:29:25.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.418+0000 7f7813fff640 1 -- 172.21.15.114:0/596344434 <== mds.3 v2:172.21.15.134:6834/1150003275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55ff69af61f0 con 0x55ff69bbb290 2024-09-13T18:29:25.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.418+0000 7f7813fff640 10 client.17598 handle_command_reply: tid=0 2024-09-13T18:29:25.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.418+0000 7f783de75740 1 client.17598 shutdown 2024-09-13T18:29:25.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.427+0000 7f783de75740 1 -- 172.21.15.114:0/596344434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff69ba1290 msgr2=0x55ff69bab220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:25.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.427+0000 7f783de75740 1 --2- 172.21.15.114:0/596344434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff69ba1290 0x55ff69bab220 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f78280eb490 tx=0x7f78280b7820 comp rx=0 tx=0).stop 2024-09-13T18:29:25.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.431+0000 7f783de75740 1 -- 172.21.15.114:0/596344434 shutdown_connections 2024-09-13T18:29:25.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.431+0000 7f783de75740 1 --2- 172.21.15.114:0/596344434 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55ff69bbb290 0x55ff69bbd6b0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.431+0000 7f783de75740 1 --2- 172.21.15.114:0/596344434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff69ba8b20 0x55ff69ba8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.431+0000 7f783de75740 1 --2- 172.21.15.114:0/596344434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff69babb30 0x55ff69ba85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.431+0000 7f783de75740 1 --2- 172.21.15.114:0/596344434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff69ba1290 0x55ff69bab220 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.431+0000 7f783de75740 1 -- 172.21.15.114:0/596344434 >> 172.21.15.114:0/596344434 conn(0x55ff69b9cee0 msgr2=0x55ff69b9c230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:25.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.431+0000 7f7813fff640 0 client.17598 ms_handle_reset on v2:172.21.15.134:6834/1150003275 2024-09-13T18:29:25.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.432+0000 7f783de75740 1 -- 172.21.15.114:0/596344434 shutdown_connections 2024-09-13T18:29:25.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.433+0000 7f783de75740 1 -- 172.21.15.114:0/596344434 wait complete. 2024-09-13T18:29:25.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.434+0000 7f783de75740 20 client.17598 trim_cache size 0 max 16384 2024-09-13T18:29:25.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.460+0000 7f783de75740 1 Processor -- start 2024-09-13T18:29:25.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.460+0000 7f783de75740 1 -- start start 2024-09-13T18:29:25.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.460+0000 7f783de75740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff69ba1290 0x55ff69b9d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:25.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.460+0000 7f783de75740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff69ba8b20 0x55ff69b9d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:25.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.460+0000 7f783de75740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff69babb30 0x55ff69ba6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:25.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.461+0000 7f783de75740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff69ba9990 con 0x55ff69ba1290 2024-09-13T18:29:25.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.461+0000 7f783de75740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff69b8ebf0 con 0x55ff69ba8b20 2024-09-13T18:29:25.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.461+0000 7f783de75740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff69b98cd0 con 0x55ff69babb30 2024-09-13T18:29:25.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.461+0000 7f783960f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff69babb30 0x55ff69ba6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:25.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.461+0000 7f783960f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff69babb30 0x55ff69ba6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:49152/0 (socket says 172.21.15.114:49152) 2024-09-13T18:29:25.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.461+0000 7f783960f640 1 -- 172.21.15.114:0/1841907246 learned_addr learned my addr 172.21.15.114:0/1841907246 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:25.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.461+0000 7f7838e0e640 1 --2- 172.21.15.114:0/1841907246 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff69ba1290 0x55ff69b9d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:25.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.462+0000 7f7838e0e640 1 -- 172.21.15.114:0/1841907246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff69babb30 msgr2=0x55ff69ba6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:25.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.462+0000 7f7838e0e640 1 --2- 172.21.15.114:0/1841907246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff69babb30 0x55ff69ba6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.462+0000 7f7838e0e640 1 -- 172.21.15.114:0/1841907246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff69ba8b20 msgr2=0x55ff69b9d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:25.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.462+0000 7f7838e0e640 1 --2- 172.21.15.114:0/1841907246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff69ba8b20 0x55ff69b9d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.462+0000 7f7838e0e640 1 -- 172.21.15.114:0/1841907246 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff69aa4770 con 0x55ff69ba1290 2024-09-13T18:29:25.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.462+0000 7f7838e0e640 1 --2- 172.21.15.114:0/1841907246 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff69ba1290 0x55ff69b9d2c0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f78280b7380 tx=0x7f7828005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1e820fa95fc6b2d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:25.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.464+0000 7f781b7fe640 1 -- 172.21.15.114:0/1841907246 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78280bae80 con 0x55ff69ba1290 2024-09-13T18:29:25.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.464+0000 7f781b7fe640 1 -- 172.21.15.114:0/1841907246 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f78280c9070 con 0x55ff69ba1290 2024-09-13T18:29:25.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.464+0000 7f781b7fe640 1 -- 172.21.15.114:0/1841907246 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78280bb580 con 0x55ff69ba1290 2024-09-13T18:29:25.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.465+0000 7f783de75740 1 -- 172.21.15.114:0/1841907246 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff69ba1290 msgr2=0x55ff69b9d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:25.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.465+0000 7f783de75740 1 --2- 172.21.15.114:0/1841907246 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff69ba1290 0x55ff69b9d2c0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f78280b7380 tx=0x7f7828005c90 comp rx=0 tx=0).stop 2024-09-13T18:29:25.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.467+0000 7f783de75740 1 -- 172.21.15.114:0/1841907246 shutdown_connections 2024-09-13T18:29:25.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.467+0000 7f783de75740 1 --2- 172.21.15.114:0/1841907246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff69babb30 0x55ff69ba6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.467+0000 7f783de75740 1 --2- 172.21.15.114:0/1841907246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff69ba8b20 0x55ff69b9d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.467+0000 7f783de75740 1 --2- 172.21.15.114:0/1841907246 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff69ba1290 0x55ff69b9d2c0 secure :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f78280b7380 tx=0x7f7828005c90 comp rx=0 tx=0).stop 2024-09-13T18:29:25.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.467+0000 7f783de75740 1 -- 172.21.15.114:0/1841907246 >> 172.21.15.114:0/1841907246 conn(0x55ff69b9cee0 msgr2=0x55ff69bc2010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:25.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.473+0000 7f783de75740 1 -- 172.21.15.114:0/1841907246 shutdown_connections 2024-09-13T18:29:25.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.473+0000 7f783de75740 1 -- 172.21.15.114:0/1841907246 wait complete. 2024-09-13T18:29:25.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.473+0000 7f783de75740 1 -- start start 2024-09-13T18:29:25.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.473+0000 7f783de75740 1 Processor -- start 2024-09-13T18:29:25.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.474+0000 7f783de75740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff69ba8b20 0x55ff69bae740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:25.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.474+0000 7f783de75740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff69babb30 0x55ff69baec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:25.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.475+0000 7f7838e0e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff69ba8b20 0x55ff69bae740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:25.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.475+0000 7f7838e0e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff69ba8b20 0x55ff69bae740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57536/0 (socket says 172.21.15.114:57536) 2024-09-13T18:29:25.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.475+0000 7f783de75740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff69c0e770 0x55ff69c10b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:25.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.475+0000 7f783de75740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff69baf350 con 0x55ff69ba8b20 2024-09-13T18:29:25.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.475+0000 7f783de75740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff69c110a0 con 0x55ff69c0e770 2024-09-13T18:29:25.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.475+0000 7f783de75740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff69c111e0 con 0x55ff69babb30 2024-09-13T18:29:25.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.475+0000 7f783de75740 10 client.0 _ll_register_callbacks cb 0x7f782803c470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:29:25.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.475+0000 7f7833fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff69babb30 0x55ff69baec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:25.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.475+0000 7f7833fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff69babb30 0x55ff69baec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:49168/0 (socket says 172.21.15.114:49168) 2024-09-13T18:29:25.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.475+0000 7f7833fff640 1 -- 172.21.15.114:0/3769988683 learned_addr learned my addr 172.21.15.114:0/3769988683 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:25.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.475+0000 7f783960f640 1 --2- 172.21.15.114:0/3769988683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff69c0e770 0x55ff69c10b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:25.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.477+0000 7f7833fff640 1 -- 172.21.15.114:0/3769988683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff69c0e770 msgr2=0x55ff69c10b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:25.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.477+0000 7f7833fff640 1 --2- 172.21.15.114:0/3769988683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff69c0e770 0x55ff69c10b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.477+0000 7f7833fff640 1 -- 172.21.15.114:0/3769988683 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff69ba8b20 msgr2=0x55ff69bae740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:25.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.477+0000 7f7833fff640 1 --2- 172.21.15.114:0/3769988683 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff69ba8b20 0x55ff69bae740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.477+0000 7f7833fff640 1 -- 172.21.15.114:0/3769988683 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ff69b023c0 con 0x55ff69babb30 2024-09-13T18:29:25.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.477+0000 7f7833fff640 1 --2- 172.21.15.114:0/3769988683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff69babb30 0x55ff69baec80 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f782400a150 tx=0x7f782400aee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbd5cb56b2de209b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:25.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.479+0000 7f7813fff640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:29:25.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.479+0000 7f7813fff640 1 -- 172.21.15.114:0/3769988683 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782401a070 con 0x55ff69babb30 2024-09-13T18:29:25.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.479+0000 7f783de75740 1 -- 172.21.15.114:0/3769988683 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f78280bbe90 con 0x55ff69babb30 2024-09-13T18:29:25.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.479+0000 7f7813fff640 1 -- 172.21.15.114:0/3769988683 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f7824005e40 con 0x55ff69babb30 2024-09-13T18:29:25.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.480+0000 7f7813fff640 1 -- 172.21.15.114:0/3769988683 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f782400a7d0 con 0x55ff69babb30 2024-09-13T18:29:25.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.481+0000 7f783de75740 10 client.27022 fetch_fsmap learned FSMap version 34 2024-09-13T18:29:25.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.481+0000 7f783de75740 1 -- 172.21.15.114:0/3769988683 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55ff69ac9e80 con 0x55ff69babb30 2024-09-13T18:29:25.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.483+0000 7f7813fff640 1 -- 172.21.15.114:0/3769988683 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f78280bbe90 con 0x55ff69babb30 2024-09-13T18:29:25.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.484+0000 7f7813fff640 1 -- 172.21.15.114:0/3769988683 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7824006670 con 0x55ff69babb30 2024-09-13T18:29:25.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.484+0000 7f783de75740 10 client.27022 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:29:25.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.484+0000 7f783de75740 10 client.27022 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-13T18:29:25.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.484+0000 7f783de75740 20 client.27022 populate_metadata read hostname 'smithi114' 2024-09-13T18:29:25.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.484+0000 7f783de75740 1 --2- 172.21.15.114:0/3769988683 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55ff69c12aa0 0x55ff69c14ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:25.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.484+0000 7f783de75740 4 client.27022 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:29:25.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.484+0000 7f783de75740 1 -- 172.21.15.114:0/3769988683 --> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ff697f74e0 con 0x55ff69c12aa0 2024-09-13T18:29:25.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.488+0000 7f7838e0e640 1 --2- 172.21.15.114:0/3769988683 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55ff69c12aa0 0x55ff69c14ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:25.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.490+0000 7f7838e0e640 1 --2- 172.21.15.114:0/3769988683 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55ff69c12aa0 0x55ff69c14ee0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7432e2f8a0dc84c4 server_cookie=cfda3aa8c11ddc3 in_seq=0 out_seq=0 2024-09-13T18:29:25.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.491+0000 7f7813fff640 10 client.27022 ms_handle_connect on v2:172.21.15.134:6834/1150003275 2024-09-13T18:29:25.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.495+0000 7f7813fff640 1 -- 172.21.15.114:0/3769988683 <== mds.3 v2:172.21.15.134:6834/1150003275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ff697f74e0 con 0x55ff69c12aa0 2024-09-13T18:29:25.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.495+0000 7f7813fff640 10 client.27022 handle_command_reply: tid=0 2024-09-13T18:29:25.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.495+0000 7f783de75740 1 client.27022 shutdown 2024-09-13T18:29:25.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.499+0000 7f783de75740 1 -- 172.21.15.114:0/3769988683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff69babb30 msgr2=0x55ff69baec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:25.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.499+0000 7f783de75740 1 --2- 172.21.15.114:0/3769988683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff69babb30 0x55ff69baec80 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f782400a150 tx=0x7f782400aee0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.499+0000 7f783de75740 1 -- 172.21.15.114:0/3769988683 shutdown_connections 2024-09-13T18:29:25.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.499+0000 7f783de75740 1 --2- 172.21.15.114:0/3769988683 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55ff69c12aa0 0x55ff69c14ee0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.499+0000 7f783de75740 1 --2- 172.21.15.114:0/3769988683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff69c0e770 0x55ff69c10b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.499+0000 7f783de75740 1 --2- 172.21.15.114:0/3769988683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff69babb30 0x55ff69baec80 secure :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f782400a150 tx=0x7f782400aee0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.499+0000 7f783de75740 1 --2- 172.21.15.114:0/3769988683 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff69ba8b20 0x55ff69bae740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.499+0000 7f783de75740 1 -- 172.21.15.114:0/3769988683 >> 172.21.15.114:0/3769988683 conn(0x55ff69b9cee0 msgr2=0x55ff69b9d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:25.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.500+0000 7f7813fff640 0 client.27022 ms_handle_reset on v2:172.21.15.134:6834/1150003275 2024-09-13T18:29:25.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.504+0000 7f783de75740 1 -- 172.21.15.114:0/3769988683 shutdown_connections 2024-09-13T18:29:25.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.507+0000 7f783de75740 1 -- 172.21.15.114:0/3769988683 wait complete. 2024-09-13T18:29:25.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.513+0000 7f783de75740 20 client.27022 trim_cache size 0 max 16384 2024-09-13T18:29:25.518 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:29:25.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.518+0000 7f7839e10640 1 -- 172.21.15.114:0/2093475553 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f781c0a1410 msgr2=0x7f781c0a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:25.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.518+0000 7f7839e10640 1 --2- 172.21.15.114:0/2093475553 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f781c0a1410 0x7f781c0a38d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x55ff69b9e260 tx=0x7f782c009490 comp rx=0 tx=0).stop 2024-09-13T18:29:25.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.518+0000 7f7839e10640 1 -- 172.21.15.114:0/2093475553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78341074b0 msgr2=0x7f78341168d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:25.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.518+0000 7f7839e10640 1 --2- 172.21.15.114:0/2093475553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78341074b0 0x7f78341168d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f782802aa00 tx=0x7f78280048f0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.519+0000 7f7839e10640 1 -- 172.21.15.114:0/2093475553 shutdown_connections 2024-09-13T18:29:25.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.519+0000 7f7839e10640 1 --2- 172.21.15.114:0/2093475553 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f781c0a1410 0x7f781c0a38d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.519+0000 7f7839e10640 1 --2- 172.21.15.114:0/2093475553 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7834119400 0x7f783410fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.519+0000 7f7839e10640 1 --2- 172.21.15.114:0/2093475553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7834107890 0x7f7834116e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.519+0000 7f7839e10640 1 --2- 172.21.15.114:0/2093475553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78341074b0 0x7f78341168d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.519+0000 7f7839e10640 1 -- 172.21.15.114:0/2093475553 >> 172.21.15.114:0/2093475553 conn(0x7f783406af40 msgr2=0x7f7834119dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:25.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.521+0000 7f7839e10640 1 -- 172.21.15.114:0/2093475553 shutdown_connections 2024-09-13T18:29:25.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.521+0000 7f7839e10640 1 -- 172.21.15.114:0/2093475553 wait complete. 2024-09-13T18:29:25.551 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-09-13T18:29:25.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.766+0000 7fdaa9e9c640 1 -- 172.21.15.114:0/1327383411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdaa4107dd0 msgr2=0x7fdaa4114c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:25.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.766+0000 7fdaa9e9c640 1 --2- 172.21.15.114:0/1327383411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdaa4107dd0 0x7fdaa4114c20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fda98009930 tx=0x7fda9802a460 comp rx=0 tx=0).stop 2024-09-13T18:29:25.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.769+0000 7fdaa9e9c640 1 -- 172.21.15.114:0/1327383411 shutdown_connections 2024-09-13T18:29:25.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.769+0000 7fdaa9e9c640 1 --2- 172.21.15.114:0/1327383411 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdaa4115160 0x7fdaa4117550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.769+0000 7fdaa9e9c640 1 --2- 172.21.15.114:0/1327383411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdaa4107dd0 0x7fdaa4114c20 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.769+0000 7fdaa9e9c640 1 --2- 172.21.15.114:0/1327383411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdaa41074b0 0x7fdaa4107890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.769+0000 7fdaa9e9c640 1 -- 172.21.15.114:0/1327383411 >> 172.21.15.114:0/1327383411 conn(0x7fdaa406afe0 msgr2=0x7fdaa406b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:25.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.772+0000 7fdaa9e9c640 1 -- 172.21.15.114:0/1327383411 shutdown_connections 2024-09-13T18:29:25.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.777+0000 7fdaa9e9c640 1 -- 172.21.15.114:0/1327383411 wait complete. 2024-09-13T18:29:25.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.777+0000 7fdaa9e9c640 1 Processor -- start 2024-09-13T18:29:25.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.778+0000 7fdaa9e9c640 1 -- start start 2024-09-13T18:29:25.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.778+0000 7fdaa9e9c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdaa41074b0 0x7fdaa41a72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:25.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.778+0000 7fdaa9e9c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdaa4107dd0 0x7fdaa41a77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:25.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.778+0000 7fdaa9e9c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdaa4115160 0x7fdaa410de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:25.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.778+0000 7fdaa9e9c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa410e490 con 0x7fdaa4107dd0 2024-09-13T18:29:25.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.778+0000 7fdaa9e9c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa410e600 con 0x7fdaa4115160 2024-09-13T18:29:25.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.778+0000 7fdaa9e9c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa410e7c0 con 0x7fdaa41074b0 2024-09-13T18:29:25.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.778+0000 7fdaa2ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdaa4107dd0 0x7fdaa41a77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:25.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.778+0000 7fdaa2ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdaa4107dd0 0x7fdaa41a77e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57562/0 (socket says 172.21.15.114:57562) 2024-09-13T18:29:25.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.778+0000 7fdaa2ffd640 1 -- 172.21.15.114:0/2457900939 learned_addr learned my addr 172.21.15.114:0/2457900939 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:25.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.778+0000 7fdaa3fff640 1 --2- 172.21.15.114:0/2457900939 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdaa4115160 0x7fdaa410de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:25.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.779+0000 7fdaa37fe640 1 --2- 172.21.15.114:0/2457900939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdaa41074b0 0x7fdaa41a72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:25.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.779+0000 7fdaa2ffd640 1 -- 172.21.15.114:0/2457900939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdaa41074b0 msgr2=0x7fdaa41a72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:25.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.779+0000 7fdaa2ffd640 1 --2- 172.21.15.114:0/2457900939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdaa41074b0 0x7fdaa41a72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.779+0000 7fdaa2ffd640 1 -- 172.21.15.114:0/2457900939 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdaa4115160 msgr2=0x7fdaa410de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:25.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.779+0000 7fdaa2ffd640 1 --2- 172.21.15.114:0/2457900939 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdaa4115160 0x7fdaa410de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.779+0000 7fdaa2ffd640 1 -- 172.21.15.114:0/2457900939 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda98009560 con 0x7fdaa4107dd0 2024-09-13T18:29:25.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.779+0000 7fdaa2ffd640 1 --2- 172.21.15.114:0/2457900939 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdaa4107dd0 0x7fdaa41a77e0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fda9802a9d0 tx=0x7fda98004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:25.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.779+0000 7fdaa0ff9640 1 -- 172.21.15.114:0/2457900939 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda98004c50 con 0x7fdaa4107dd0 2024-09-13T18:29:25.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.780+0000 7fdaa9e9c640 1 -- 172.21.15.114:0/2457900939 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaa410ea40 con 0x7fdaa4107dd0 2024-09-13T18:29:25.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.780+0000 7fdaa9e9c640 1 -- 172.21.15.114:0/2457900939 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaa41142f0 con 0x7fdaa4107dd0 2024-09-13T18:29:25.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.780+0000 7fdaa0ff9640 1 -- 172.21.15.114:0/2457900939 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fda98004db0 con 0x7fdaa4107dd0 2024-09-13T18:29:25.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.781+0000 7fdaa0ff9640 1 -- 172.21.15.114:0/2457900939 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9803c9a0 con 0x7fdaa4107dd0 2024-09-13T18:29:25.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.783+0000 7fdaacbc2740 1 Processor -- start 2024-09-13T18:29:25.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.783+0000 7fdaacbc2740 1 -- start start 2024-09-13T18:29:25.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.783+0000 7fdaacbc2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595ec835790 0x5595ec835b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:25.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.783+0000 7fdaacbc2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595ec8360b0 0x5595ec840410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:25.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.783+0000 7fdaacbc2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ec840950 0x5595ec842d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:25.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.783+0000 7fdaacbc2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595ec79fcc0 con 0x5595ec8360b0 2024-09-13T18:29:25.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.783+0000 7fdaacbc2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595ec79d450 con 0x5595ec835790 2024-09-13T18:29:25.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.783+0000 7fdaacbc2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595ec798d40 con 0x5595ec840950 2024-09-13T18:29:25.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.785+0000 7fdaa0ff9640 1 -- 172.21.15.114:0/2457900939 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fda980035f0 con 0x7fdaa4107dd0 2024-09-13T18:29:25.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.785+0000 7fdaa3fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595ec835790 0x5595ec835b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:25.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.786+0000 7fdaa3fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595ec835790 0x5595ec835b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:56216/0 (socket says 172.21.15.114:56216) 2024-09-13T18:29:25.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.786+0000 7fdaa3fff640 1 -- 172.21.15.114:0/2782475386 learned_addr learned my addr 172.21.15.114:0/2782475386 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:25.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.786+0000 7fdaa2ffd640 1 --2- 172.21.15.114:0/2782475386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ec840950 0x5595ec842d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:25.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.786+0000 7fdaa37fe640 1 --2- 172.21.15.114:0/2782475386 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595ec8360b0 0x5595ec840410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:25.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.790+0000 7fdaa3fff640 1 -- 172.21.15.114:0/2782475386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ec840950 msgr2=0x5595ec842d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:25.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.795+0000 7fdaa0ff9640 1 --2- 172.21.15.114:0/2457900939 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fda8c0a12c0 0x7fda8c0a3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:25.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.795+0000 7fdaa0ff9640 1 -- 172.21.15.114:0/2457900939 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fda980ede90 con 0x7fdaa4107dd0 2024-09-13T18:29:25.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.795+0000 7fdaa3fff640 1 --2- 172.21.15.114:0/2782475386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ec840950 0x5595ec842d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.795+0000 7fdaa3fff640 1 -- 172.21.15.114:0/2782475386 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595ec8360b0 msgr2=0x5595ec840410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:25.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.795+0000 7fdaa3fff640 1 --2- 172.21.15.114:0/2782475386 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595ec8360b0 0x5595ec840410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.795+0000 7fdaa3fff640 1 -- 172.21.15.114:0/2782475386 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595ec760620 con 0x5595ec835790 2024-09-13T18:29:25.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.797+0000 7fdaa3fff640 1 --2- 172.21.15.114:0/2782475386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595ec835790 0x5595ec835b70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fda9400b600 tx=0x7fda9400bac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e20583c3a808dafd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:25.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.797+0000 7fdaa3fff640 1 --2- 172.21.15.114:0/2457900939 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fda8c0a12c0 0x7fda8c0a3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:25.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.797+0000 7fdaa37fe640 1 --2- 172.21.15.114:0/2782475386 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595ec8360b0 0x5595ec840410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:29:25.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.797+0000 7fda89ffb640 1 -- 172.21.15.114:0/2782475386 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda94011440 con 0x5595ec835790 2024-09-13T18:29:25.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.797+0000 7fda89ffb640 1 -- 172.21.15.114:0/2782475386 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fda94011ab0 con 0x5595ec835790 2024-09-13T18:29:25.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.797+0000 7fda89ffb640 1 -- 172.21.15.114:0/2782475386 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda94011440 con 0x5595ec835790 2024-09-13T18:29:25.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.798+0000 7fdaa3fff640 1 --2- 172.21.15.114:0/2457900939 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fda8c0a12c0 0x7fda8c0a3780 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fda9401f000 tx=0x7fda94010810 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:25.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.798+0000 7fdaa2ffd640 1 --2- 172.21.15.114:0/2782475386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ec840950 0x5595ec842d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:29:25.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.799+0000 7fdaacbc2740 1 -- 172.21.15.114:0/2782475386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595ec835790 msgr2=0x5595ec835b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:25.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.799+0000 7fdaacbc2740 1 --2- 172.21.15.114:0/2782475386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595ec835790 0x5595ec835b70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fda9400b600 tx=0x7fda9400bac0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.800+0000 7fdaacbc2740 1 -- 172.21.15.114:0/2782475386 shutdown_connections 2024-09-13T18:29:25.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.800+0000 7fdaacbc2740 1 --2- 172.21.15.114:0/2782475386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ec840950 0x5595ec842d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.800+0000 7fdaacbc2740 1 --2- 172.21.15.114:0/2782475386 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595ec8360b0 0x5595ec840410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.800+0000 7fdaacbc2740 1 --2- 172.21.15.114:0/2782475386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595ec835790 0x5595ec835b70 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.800+0000 7fdaacbc2740 1 -- 172.21.15.114:0/2782475386 >> 172.21.15.114:0/2782475386 conn(0x5595ec831d00 msgr2=0x5595ec8320e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:25.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.805+0000 7fdaacbc2740 1 -- 172.21.15.114:0/2782475386 shutdown_connections 2024-09-13T18:29:25.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.808+0000 7fdaacbc2740 1 -- 172.21.15.114:0/2782475386 wait complete. 2024-09-13T18:29:25.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.808+0000 7fdaacbc2740 1 -- start start 2024-09-13T18:29:25.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.808+0000 7fdaacbc2740 1 Processor -- start 2024-09-13T18:29:25.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.808+0000 7fdaacbc2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ec835790 0x5595ec83fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:25.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.809+0000 7fdaacbc2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595ec8360b0 0x5595ec8403d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:25.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.809+0000 7fdaacbc2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595ec840950 0x5595ec83d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:25.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.809+0000 7fdaacbc2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595ec757180 con 0x5595ec840950 2024-09-13T18:29:25.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.809+0000 7fdaacbc2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595ec471430 con 0x5595ec8360b0 2024-09-13T18:29:25.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.809+0000 7fdaacbc2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595ec71c5e0 con 0x5595ec835790 2024-09-13T18:29:25.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.809+0000 7fdaacbc2740 10 client.0 _ll_register_callbacks cb 0x5595ec788aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:29:25.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.809+0000 7fdaa37fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ec835790 0x5595ec83fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:25.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.809+0000 7fdaa37fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ec835790 0x5595ec83fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:49198/0 (socket says 172.21.15.114:49198) 2024-09-13T18:29:25.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.809+0000 7fdaa37fe640 1 -- 172.21.15.114:0/582963385 learned_addr learned my addr 172.21.15.114:0/582963385 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:25.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.809+0000 7fdaa3fff640 1 --2- 172.21.15.114:0/582963385 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595ec840950 0x5595ec83d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:25.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.810+0000 7fdaa3fff640 1 -- 172.21.15.114:0/582963385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ec835790 msgr2=0x5595ec83fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:25.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.810+0000 7fdaa3fff640 1 --2- 172.21.15.114:0/582963385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ec835790 0x5595ec83fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.810+0000 7fdaa3fff640 1 -- 172.21.15.114:0/582963385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595ec8360b0 msgr2=0x5595ec8403d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:25.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.810+0000 7fdaa3fff640 1 --2- 172.21.15.114:0/582963385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595ec8360b0 0x5595ec8403d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.810+0000 7fdaa3fff640 1 -- 172.21.15.114:0/582963385 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda94004290 con 0x5595ec840950 2024-09-13T18:29:25.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.810+0000 7fdaa3fff640 1 --2- 172.21.15.114:0/582963385 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595ec840950 0x5595ec83d470 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fda94020040 tx=0x7fda9401e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4abb6e6e83b1cab2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:25.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.810+0000 7fda82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:29:25.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.811+0000 7fda82ffd640 1 -- 172.21.15.114:0/582963385 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda94011440 con 0x5595ec840950 2024-09-13T18:29:25.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.811+0000 7fdaacbc2740 1 -- 172.21.15.114:0/582963385 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5595ec7c2b80 con 0x5595ec840950 2024-09-13T18:29:25.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.811+0000 7fda82ffd640 1 -- 172.21.15.114:0/582963385 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fda94011e30 con 0x5595ec840950 2024-09-13T18:29:25.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.812+0000 7fda82ffd640 1 -- 172.21.15.114:0/582963385 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda9401c920 con 0x5595ec840950 2024-09-13T18:29:25.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.812+0000 7fdaacbc2740 10 client.17616 fetch_fsmap learned FSMap version 34 2024-09-13T18:29:25.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.812+0000 7fdaacbc2740 1 -- 172.21.15.114:0/582963385 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5595ec7973c0 con 0x5595ec840950 2024-09-13T18:29:25.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.813+0000 7fda82ffd640 1 -- 172.21.15.114:0/582963385 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5595ec7c2b80 con 0x5595ec840950 2024-09-13T18:29:25.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.813+0000 7fda82ffd640 1 -- 172.21.15.114:0/582963385 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fda94024030 con 0x5595ec840950 2024-09-13T18:29:25.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.814+0000 7fdaacbc2740 10 client.17616 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:29:25.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.814+0000 7fdaacbc2740 10 client.17616 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-13T18:29:25.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.814+0000 7fdaacbc2740 20 client.17616 populate_metadata read hostname 'smithi114' 2024-09-13T18:29:25.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.814+0000 7fdaacbc2740 1 --2- 172.21.15.114:0/582963385 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x5595ec850240 0x5595ec852670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:25.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.814+0000 7fdaacbc2740 4 client.17616 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:29:25.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.814+0000 7fdaacbc2740 1 -- 172.21.15.114:0/582963385 --> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5595ec78b1f0 con 0x5595ec850240 2024-09-13T18:29:25.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.815+0000 7fdaa37fe640 1 --2- 172.21.15.114:0/582963385 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x5595ec850240 0x5595ec852670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:25.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.820+0000 7fdaa37fe640 1 --2- 172.21.15.114:0/582963385 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x5595ec850240 0x5595ec852670 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=abcb62213c03fdf2 server_cookie=a901a4f1b1e3fcdb in_seq=0 out_seq=0 2024-09-13T18:29:25.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.820+0000 7fda82ffd640 10 client.17616 ms_handle_connect on v2:172.21.15.204:6838/344883912 2024-09-13T18:29:25.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.820+0000 7fda82ffd640 1 -- 172.21.15.114:0/582963385 <== mds.2 v2:172.21.15.204:6838/344883912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5595ec78b1f0 con 0x5595ec850240 2024-09-13T18:29:25.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.820+0000 7fda82ffd640 10 client.17616 handle_command_reply: tid=0 2024-09-13T18:29:25.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.821+0000 7fdaacbc2740 1 client.17616 shutdown 2024-09-13T18:29:25.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.823+0000 7fdaacbc2740 1 -- 172.21.15.114:0/582963385 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595ec840950 msgr2=0x5595ec83d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:25.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.823+0000 7fdaacbc2740 1 --2- 172.21.15.114:0/582963385 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595ec840950 0x5595ec83d470 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fda94020040 tx=0x7fda9401e040 comp rx=0 tx=0).stop 2024-09-13T18:29:25.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.823+0000 7fdaacbc2740 1 -- 172.21.15.114:0/582963385 shutdown_connections 2024-09-13T18:29:25.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.823+0000 7fdaacbc2740 1 --2- 172.21.15.114:0/582963385 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x5595ec850240 0x5595ec852670 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.823+0000 7fdaacbc2740 1 --2- 172.21.15.114:0/582963385 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595ec840950 0x5595ec83d470 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.823+0000 7fdaacbc2740 1 --2- 172.21.15.114:0/582963385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595ec8360b0 0x5595ec8403d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.823+0000 7fdaacbc2740 1 --2- 172.21.15.114:0/582963385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ec835790 0x5595ec83fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.823+0000 7fdaacbc2740 1 -- 172.21.15.114:0/582963385 >> 172.21.15.114:0/582963385 conn(0x5595ec831d00 msgr2=0x5595ec837a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:25.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.824+0000 7fda82ffd640 0 client.17616 ms_handle_reset on v2:172.21.15.204:6838/344883912 2024-09-13T18:29:25.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.825+0000 7fdaacbc2740 1 -- 172.21.15.114:0/582963385 shutdown_connections 2024-09-13T18:29:25.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.826+0000 7fdaacbc2740 1 -- 172.21.15.114:0/582963385 wait complete. 2024-09-13T18:29:25.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.830+0000 7fdaacbc2740 20 client.17616 trim_cache size 0 max 16384 2024-09-13T18:29:25.858 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:29:25 smithi114 ceph-mon[30897]: pgmap v213: 129 pgs: 129 active+clean; 21 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 39 MiB/s wr, 861 op/s 2024-09-13T18:29:25.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.858+0000 7fdaacbc2740 1 Processor -- start 2024-09-13T18:29:25.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.858+0000 7fdaacbc2740 1 -- start start 2024-09-13T18:29:25.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.859+0000 7fdaacbc2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595ec835790 0x5595ec857010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:25.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.859+0000 7fdaacbc2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595ec8360b0 0x5595ec847650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:25.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.859+0000 7fdaacbc2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ec840950 0x5595ec847b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:25.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.859+0000 7fdaacbc2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595ec84c950 con 0x5595ec8360b0 2024-09-13T18:29:25.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.859+0000 7fdaacbc2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595ec835620 con 0x5595ec835790 2024-09-13T18:29:25.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.859+0000 7fdaacbc2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595ec8367a0 con 0x5595ec840950 2024-09-13T18:29:25.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.859+0000 7fdaa3fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ec840950 0x5595ec847b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:25.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.859+0000 7fdaa3fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ec840950 0x5595ec847b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:49212/0 (socket says 172.21.15.114:49212) 2024-09-13T18:29:25.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.859+0000 7fdaa3fff640 1 -- 172.21.15.114:0/2305028075 learned_addr learned my addr 172.21.15.114:0/2305028075 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:25.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.864+0000 7fdaa3fff640 1 -- 172.21.15.114:0/2305028075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595ec835790 msgr2=0x5595ec857010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:25.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.864+0000 7fdaa3fff640 1 --2- 172.21.15.114:0/2305028075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595ec835790 0x5595ec857010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.864+0000 7fdaa3fff640 1 -- 172.21.15.114:0/2305028075 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595ec8360b0 msgr2=0x5595ec847650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:25.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.864+0000 7fdaa3fff640 1 --2- 172.21.15.114:0/2305028075 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595ec8360b0 0x5595ec847650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.864+0000 7fdaa3fff640 1 -- 172.21.15.114:0/2305028075 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595ec739770 con 0x5595ec840950 2024-09-13T18:29:25.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.869+0000 7fdaa3fff640 1 --2- 172.21.15.114:0/2305028075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ec840950 0x5595ec847b90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fda94025040 tx=0x7fda940044b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4537af7980e53338 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:25.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.869+0000 7fda8a7fc640 1 -- 172.21.15.114:0/2305028075 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9400b3d0 con 0x5595ec840950 2024-09-13T18:29:25.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.871+0000 7fda8a7fc640 1 -- 172.21.15.114:0/2305028075 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fda94011e30 con 0x5595ec840950 2024-09-13T18:29:25.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.871+0000 7fda8a7fc640 1 -- 172.21.15.114:0/2305028075 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda94011440 con 0x5595ec840950 2024-09-13T18:29:25.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.872+0000 7fdaacbc2740 1 -- 172.21.15.114:0/2305028075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ec840950 msgr2=0x5595ec847b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:25.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.872+0000 7fdaacbc2740 1 --2- 172.21.15.114:0/2305028075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ec840950 0x5595ec847b90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fda94025040 tx=0x7fda940044b0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.872+0000 7fdaacbc2740 1 -- 172.21.15.114:0/2305028075 shutdown_connections 2024-09-13T18:29:25.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.873+0000 7fdaacbc2740 1 --2- 172.21.15.114:0/2305028075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ec840950 0x5595ec847b90 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.873+0000 7fdaacbc2740 1 --2- 172.21.15.114:0/2305028075 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595ec8360b0 0x5595ec847650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.873+0000 7fdaacbc2740 1 --2- 172.21.15.114:0/2305028075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595ec835790 0x5595ec857010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.873+0000 7fdaacbc2740 1 -- 172.21.15.114:0/2305028075 >> 172.21.15.114:0/2305028075 conn(0x5595ec831d00 msgr2=0x5595ec8320e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:25.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.876+0000 7fdaacbc2740 1 -- 172.21.15.114:0/2305028075 shutdown_connections 2024-09-13T18:29:25.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.877+0000 7fdaacbc2740 1 -- 172.21.15.114:0/2305028075 wait complete. 2024-09-13T18:29:25.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.877+0000 7fdaacbc2740 1 -- start start 2024-09-13T18:29:25.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.878+0000 7fdaacbc2740 1 Processor -- start 2024-09-13T18:29:25.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.878+0000 7fdaacbc2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ec835790 0x5595ec89a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:25.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.878+0000 7fdaacbc2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595ec8360b0 0x5595ec89ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:25.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.878+0000 7fdaa37fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ec835790 0x5595ec89a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:25.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.878+0000 7fdaa37fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ec835790 0x5595ec89a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:49224/0 (socket says 172.21.15.114:49224) 2024-09-13T18:29:25.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.878+0000 7fdaacbc2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595ec840950 0x5595ec89e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:25.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.879+0000 7fdaa37fe640 1 -- 172.21.15.114:0/2029140951 learned_addr learned my addr 172.21.15.114:0/2029140951 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:25.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.879+0000 7fdaacbc2740 1 -- 172.21.15.114:0/2029140951 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595ec89ee20 con 0x5595ec840950 2024-09-13T18:29:25.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.879+0000 7fdaacbc2740 1 -- 172.21.15.114:0/2029140951 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595ec89ef90 con 0x5595ec8360b0 2024-09-13T18:29:25.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.879+0000 7fdaacbc2740 1 -- 172.21.15.114:0/2029140951 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595ec89f150 con 0x5595ec835790 2024-09-13T18:29:25.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.879+0000 7fdaacbc2740 10 client.0 _ll_register_callbacks cb 0x7fda94004080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:29:25.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.881+0000 7fdaa3fff640 1 --2- 172.21.15.114:0/2029140951 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595ec840950 0x5595ec89e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:25.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.883+0000 7fdaa2ffd640 1 --2- 172.21.15.114:0/2029140951 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595ec8360b0 0x5595ec89ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:25.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.884+0000 7fdaa37fe640 1 -- 172.21.15.114:0/2029140951 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595ec8360b0 msgr2=0x5595ec89ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:25.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.884+0000 7fdaa37fe640 1 --2- 172.21.15.114:0/2029140951 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595ec8360b0 0x5595ec89ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.884+0000 7fdaa37fe640 1 -- 172.21.15.114:0/2029140951 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595ec840950 msgr2=0x5595ec89e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:25.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.884+0000 7fdaa37fe640 1 --2- 172.21.15.114:0/2029140951 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595ec840950 0x5595ec89e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.884+0000 7fdaa37fe640 1 -- 172.21.15.114:0/2029140951 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5595ec7973c0 con 0x5595ec835790 2024-09-13T18:29:25.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.885+0000 7fdaa37fe640 1 --2- 172.21.15.114:0/2029140951 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ec835790 0x5595ec89a910 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fda9c0095b0 tx=0x7fda9c004260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66f3df888de328ce server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:25.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.888+0000 7fda82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:29:25.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.888+0000 7fda82ffd640 1 -- 172.21.15.114:0/2029140951 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9c017040 con 0x5595ec835790 2024-09-13T18:29:25.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.888+0000 7fda82ffd640 1 -- 172.21.15.114:0/2029140951 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fda9c01d830 con 0x5595ec835790 2024-09-13T18:29:25.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.892+0000 7fdaacbc2740 1 -- 172.21.15.114:0/2029140951 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda94011ab0 con 0x5595ec835790 2024-09-13T18:29:25.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.894+0000 7fda82ffd640 1 -- 172.21.15.114:0/2029140951 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda9c012450 con 0x5595ec835790 2024-09-13T18:29:25.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.894+0000 7fdaacbc2740 10 client.27037 fetch_fsmap learned FSMap version 34 2024-09-13T18:29:25.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.894+0000 7fdaacbc2740 1 -- 172.21.15.114:0/2029140951 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5595ec75ee80 con 0x5595ec835790 2024-09-13T18:29:25.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.894+0000 7fda82ffd640 1 -- 172.21.15.114:0/2029140951 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fda94011ab0 con 0x5595ec835790 2024-09-13T18:29:25.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.895+0000 7fda82ffd640 1 -- 172.21.15.114:0/2029140951 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fda9c0114f0 con 0x5595ec835790 2024-09-13T18:29:25.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.896+0000 7fdaacbc2740 10 client.27037 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:29:25.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.896+0000 7fdaacbc2740 10 client.27037 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-13T18:29:25.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.896+0000 7fdaacbc2740 20 client.27037 populate_metadata read hostname 'smithi114' 2024-09-13T18:29:25.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.896+0000 7fdaacbc2740 1 --2- 172.21.15.114:0/2029140951 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x5595ec8a7bf0 0x5595ec8aa010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:25.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.896+0000 7fdaacbc2740 4 client.27037 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:29:25.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.896+0000 7fdaacbc2740 1 -- 172.21.15.114:0/2029140951 --> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5595ec48c4e0 con 0x5595ec8a7bf0 2024-09-13T18:29:25.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.897+0000 7fdaa2ffd640 1 --2- 172.21.15.114:0/2029140951 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x5595ec8a7bf0 0x5595ec8aa010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:25.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.906+0000 7fdaa2ffd640 1 --2- 172.21.15.114:0/2029140951 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x5595ec8a7bf0 0x5595ec8aa010 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7b8653ff71d1cd4e server_cookie=586c0662567fa145 in_seq=0 out_seq=0 2024-09-13T18:29:25.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.906+0000 7fda82ffd640 10 client.27037 ms_handle_connect on v2:172.21.15.204:6838/344883912 2024-09-13T18:29:25.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.907+0000 7fda82ffd640 1 -- 172.21.15.114:0/2029140951 <== mds.2 v2:172.21.15.204:6838/344883912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5595ec48c4e0 con 0x5595ec8a7bf0 2024-09-13T18:29:25.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.907+0000 7fda82ffd640 10 client.27037 handle_command_reply: tid=0 2024-09-13T18:29:25.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.907+0000 7fdaacbc2740 1 client.27037 shutdown 2024-09-13T18:29:25.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.910+0000 7fdaacbc2740 1 -- 172.21.15.114:0/2029140951 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ec835790 msgr2=0x5595ec89a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:25.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.910+0000 7fdaacbc2740 1 --2- 172.21.15.114:0/2029140951 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ec835790 0x5595ec89a910 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fda9c0095b0 tx=0x7fda9c004260 comp rx=0 tx=0).stop 2024-09-13T18:29:25.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.911+0000 7fdaacbc2740 1 -- 172.21.15.114:0/2029140951 shutdown_connections 2024-09-13T18:29:25.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.911+0000 7fdaacbc2740 1 --2- 172.21.15.114:0/2029140951 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x5595ec8a7bf0 0x5595ec8aa010 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.911+0000 7fdaacbc2740 1 --2- 172.21.15.114:0/2029140951 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5595ec840950 0x5595ec89e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.911+0000 7fdaacbc2740 1 --2- 172.21.15.114:0/2029140951 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595ec8360b0 0x5595ec89ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.911+0000 7fdaacbc2740 1 --2- 172.21.15.114:0/2029140951 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595ec835790 0x5595ec89a910 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.911+0000 7fdaacbc2740 1 -- 172.21.15.114:0/2029140951 >> 172.21.15.114:0/2029140951 conn(0x5595ec831d00 msgr2=0x5595ec8320e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:25.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.911+0000 7fda82ffd640 0 client.27037 ms_handle_reset on v2:172.21.15.204:6838/344883912 2024-09-13T18:29:25.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.914+0000 7fdaacbc2740 1 -- 172.21.15.114:0/2029140951 shutdown_connections 2024-09-13T18:29:25.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.920+0000 7fdaacbc2740 1 -- 172.21.15.114:0/2029140951 wait complete. 2024-09-13T18:29:25.927 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:29:25.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.923+0000 7fdaacbc2740 20 client.27037 trim_cache size 0 max 16384 2024-09-13T18:29:25.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.926+0000 7fdaa9e9c640 1 -- 172.21.15.114:0/2457900939 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fda8c0a12c0 msgr2=0x7fda8c0a3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:25.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.926+0000 7fdaa9e9c640 1 --2- 172.21.15.114:0/2457900939 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fda8c0a12c0 0x7fda8c0a3780 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fda9401f000 tx=0x7fda94010810 comp rx=0 tx=0).stop 2024-09-13T18:29:25.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.926+0000 7fdaa9e9c640 1 -- 172.21.15.114:0/2457900939 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdaa4107dd0 msgr2=0x7fdaa41a77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:25.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.926+0000 7fdaa9e9c640 1 --2- 172.21.15.114:0/2457900939 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdaa4107dd0 0x7fdaa41a77e0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fda9802a9d0 tx=0x7fda98004600 comp rx=0 tx=0).stop 2024-09-13T18:29:25.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.926+0000 7fdaa9e9c640 1 -- 172.21.15.114:0/2457900939 shutdown_connections 2024-09-13T18:29:25.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.926+0000 7fdaa9e9c640 1 --2- 172.21.15.114:0/2457900939 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fda8c0a12c0 0x7fda8c0a3780 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.926+0000 7fdaa9e9c640 1 --2- 172.21.15.114:0/2457900939 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdaa4115160 0x7fdaa410de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.926+0000 7fdaa9e9c640 1 --2- 172.21.15.114:0/2457900939 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdaa4107dd0 0x7fdaa41a77e0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.926+0000 7fdaa9e9c640 1 --2- 172.21.15.114:0/2457900939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdaa41074b0 0x7fdaa41a72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:25.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.926+0000 7fdaa9e9c640 1 -- 172.21.15.114:0/2457900939 >> 172.21.15.114:0/2457900939 conn(0x7fdaa406afe0 msgr2=0x7fdaa41158a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:25.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.928+0000 7fdaa9e9c640 1 -- 172.21.15.114:0/2457900939 shutdown_connections 2024-09-13T18:29:25.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:25.928+0000 7fdaa9e9c640 1 -- 172.21.15.114:0/2457900939 wait complete. 2024-09-13T18:29:25.966 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-09-13T18:29:26.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:29:25 smithi134 ceph-mon[34283]: pgmap v213: 129 pgs: 129 active+clean; 21 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 39 MiB/s wr, 861 op/s 2024-09-13T18:29:26.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.174+0000 7f8fdb59e640 1 -- 172.21.15.114:0/575435238 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8fdc107890 msgr2=0x7f8fdc107cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:26.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.174+0000 7f8fdb59e640 1 --2- 172.21.15.114:0/575435238 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8fdc107890 0x7f8fdc107cd0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f8fd401acb0 tx=0x7f8fd403b900 comp rx=0 tx=0).stop 2024-09-13T18:29:26.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.178+0000 7f8fdb59e640 1 -- 172.21.15.114:0/575435238 shutdown_connections 2024-09-13T18:29:26.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.178+0000 7f8fdb59e640 1 --2- 172.21.15.114:0/575435238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fdc119400 0x7f8fdc11b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.178+0000 7f8fdb59e640 1 --2- 172.21.15.114:0/575435238 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8fdc107890 0x7f8fdc107cd0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.178+0000 7f8fdb59e640 1 --2- 172.21.15.114:0/575435238 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8fdc1074b0 0x7f8fdc10e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.178+0000 7f8fdb59e640 1 -- 172.21.15.114:0/575435238 >> 172.21.15.114:0/575435238 conn(0x7f8fdc06af40 msgr2=0x7f8fdc06b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:26.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.183+0000 7f8fdb59e640 1 -- 172.21.15.114:0/575435238 shutdown_connections 2024-09-13T18:29:26.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.185+0000 7f8fdb59e640 1 -- 172.21.15.114:0/575435238 wait complete. 2024-09-13T18:29:26.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.186+0000 7f8fdb59e640 1 Processor -- start 2024-09-13T18:29:26.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.187+0000 7f8fdb59e640 1 -- start start 2024-09-13T18:29:26.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.188+0000 7f8fdb59e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8fdc1074b0 0x7f8fdc1186f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:26.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.188+0000 7f8fdb59e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fdc107890 0x7f8fdc118c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:26.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.188+0000 7f8fdb59e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8fdc119400 0x7f8fdc1148d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:26.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.188+0000 7f8fdb59e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fdc114f70 con 0x7f8fdc119400 2024-09-13T18:29:26.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.188+0000 7f8fdb59e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fdc1150e0 con 0x7f8fdc107890 2024-09-13T18:29:26.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.188+0000 7f8fdb59e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fdc1152a0 con 0x7f8fdc1074b0 2024-09-13T18:29:26.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.188+0000 7f8fda59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8fdc1074b0 0x7f8fdc1186f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:26.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.188+0000 7f8fdad9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8fdc119400 0x7f8fdc1148d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:26.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.188+0000 7f8fda59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8fdc1074b0 0x7f8fdc1186f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:49240/0 (socket says 172.21.15.114:49240) 2024-09-13T18:29:26.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.188+0000 7f8fdad9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8fdc119400 0x7f8fdc1148d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57620/0 (socket says 172.21.15.114:57620) 2024-09-13T18:29:26.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.188+0000 7f8fda59c640 1 -- 172.21.15.114:0/1884918761 learned_addr learned my addr 172.21.15.114:0/1884918761 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:26.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.189+0000 7f8fdad9d640 1 -- 172.21.15.114:0/1884918761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8fdc1074b0 msgr2=0x7f8fdc1186f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:26.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.189+0000 7f8fdad9d640 1 --2- 172.21.15.114:0/1884918761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8fdc1074b0 0x7f8fdc1186f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.189+0000 7f8fdad9d640 1 -- 172.21.15.114:0/1884918761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fdc107890 msgr2=0x7f8fdc118c30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:29:26.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.189+0000 7f8fdad9d640 1 --2- 172.21.15.114:0/1884918761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fdc107890 0x7f8fdc118c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.189+0000 7f8fdad9d640 1 -- 172.21.15.114:0/1884918761 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd401a910 con 0x7f8fdc119400 2024-09-13T18:29:26.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.189+0000 7f8fda59c640 1 --2- 172.21.15.114:0/1884918761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8fdc1074b0 0x7f8fdc1186f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:29:26.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.189+0000 7f8fdad9d640 1 --2- 172.21.15.114:0/1884918761 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8fdc119400 0x7f8fdc1148d0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc00b410 tx=0x7f8fcc00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:26.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.190+0000 7f8fcb7fe640 1 -- 172.21.15.114:0/1884918761 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc011830 con 0x7f8fdc119400 2024-09-13T18:29:26.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.190+0000 7f8fcb7fe640 1 -- 172.21.15.114:0/1884918761 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8fcc011e20 con 0x7f8fdc119400 2024-09-13T18:29:26.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.190+0000 7f8fcb7fe640 1 -- 172.21.15.114:0/1884918761 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc010e10 con 0x7f8fdc119400 2024-09-13T18:29:26.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.190+0000 7f8fdb59e640 1 -- 172.21.15.114:0/1884918761 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fdc115520 con 0x7f8fdc119400 2024-09-13T18:29:26.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.190+0000 7f8fdb59e640 1 -- 172.21.15.114:0/1884918761 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fdc1a9500 con 0x7f8fdc119400 2024-09-13T18:29:26.196 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:29:25 smithi204 ceph-mon[34777]: pgmap v213: 129 pgs: 129 active+clean; 21 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 39 MiB/s wr, 861 op/s 2024-09-13T18:29:26.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.193+0000 7f8fe3a09740 1 Processor -- start 2024-09-13T18:29:26.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.195+0000 7f8fe3a09740 1 -- start start 2024-09-13T18:29:26.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.195+0000 7f8fe3a09740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562ed1ace970 0x562ed1aced50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:26.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.195+0000 7f8fe3a09740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ed1acf290 0x562ed1ad95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:26.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.195+0000 7f8fe3a09740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ed1ad9b30 0x562ed1adbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:26.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.195+0000 7f8fe3a09740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ed1a38cc0 con 0x562ed1ace970 2024-09-13T18:29:26.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.195+0000 7f8fe3a09740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ed1a36450 con 0x562ed1ad9b30 2024-09-13T18:29:26.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.195+0000 7f8fe3a09740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ed1a31d40 con 0x562ed1acf290 2024-09-13T18:29:26.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.195+0000 7f8fcb7fe640 1 -- 172.21.15.114:0/1884918761 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f8fcc011990 con 0x7f8fdc119400 2024-09-13T18:29:26.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.195+0000 7f8fda59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ed1acf290 0x562ed1ad95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:26.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.196+0000 7f8fda59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ed1acf290 0x562ed1ad95f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:49256/0 (socket says 172.21.15.114:49256) 2024-09-13T18:29:26.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.196+0000 7f8fda59c640 1 -- 172.21.15.114:0/3389332202 learned_addr learned my addr 172.21.15.114:0/3389332202 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:26.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.196+0000 7f8fd9d9b640 1 --2- 172.21.15.114:0/3389332202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ed1ad9b30 0x562ed1adbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:26.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.196+0000 7f8fda59c640 1 -- 172.21.15.114:0/3389332202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ed1ad9b30 msgr2=0x562ed1adbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:26.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.196+0000 7f8fda59c640 1 --2- 172.21.15.114:0/3389332202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ed1ad9b30 0x562ed1adbf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.196+0000 7f8fda59c640 1 -- 172.21.15.114:0/3389332202 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562ed1ace970 msgr2=0x562ed1aced50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:26.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.196+0000 7f8fda59c640 1 --2- 172.21.15.114:0/3389332202 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562ed1ace970 0x562ed1aced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.196+0000 7f8fda59c640 1 -- 172.21.15.114:0/3389332202 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ed19f9620 con 0x562ed1acf290 2024-09-13T18:29:26.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.196+0000 7f8fda59c640 1 --2- 172.21.15.114:0/3389332202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ed1acf290 0x562ed1ad95f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0009ea0 tx=0x7f8fd0009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39df216d689b70d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:26.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.197+0000 7f8fc8ff9640 1 -- 172.21.15.114:0/3389332202 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd0013470 con 0x562ed1acf290 2024-09-13T18:29:26.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.197+0000 7f8fc8ff9640 1 -- 172.21.15.114:0/3389332202 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8fd0013a60 con 0x562ed1acf290 2024-09-13T18:29:26.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.198+0000 7f8fe3a09740 1 -- 172.21.15.114:0/3389332202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ed1acf290 msgr2=0x562ed1ad95f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:26.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.198+0000 7f8fe3a09740 1 --2- 172.21.15.114:0/3389332202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ed1acf290 0x562ed1ad95f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0009ea0 tx=0x7f8fd0009f90 comp rx=0 tx=0).stop 2024-09-13T18:29:26.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.198+0000 7f8fcb7fe640 1 --2- 172.21.15.114:0/1884918761 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f8fc40a13d0 0x7f8fc40a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:26.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.199+0000 7f8fe3a09740 1 -- 172.21.15.114:0/3389332202 shutdown_connections 2024-09-13T18:29:26.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.199+0000 7f8fe3a09740 1 --2- 172.21.15.114:0/3389332202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ed1ad9b30 0x562ed1adbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.199+0000 7f8fe3a09740 1 --2- 172.21.15.114:0/3389332202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ed1acf290 0x562ed1ad95f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.199+0000 7f8fe3a09740 1 --2- 172.21.15.114:0/3389332202 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562ed1ace970 0x562ed1aced50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.199+0000 7f8fe3a09740 1 -- 172.21.15.114:0/3389332202 >> 172.21.15.114:0/3389332202 conn(0x562ed1acaee0 msgr2=0x562ed1acb2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:26.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.199+0000 7f8fcb7fe640 1 -- 172.21.15.114:0/1884918761 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f8fcc0cca10 con 0x7f8fdc119400 2024-09-13T18:29:26.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.199+0000 7f8fe3a09740 1 -- 172.21.15.114:0/3389332202 shutdown_connections 2024-09-13T18:29:26.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.199+0000 7f8fe3a09740 1 -- 172.21.15.114:0/3389332202 wait complete. 2024-09-13T18:29:26.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.199+0000 7f8fe3a09740 1 -- start start 2024-09-13T18:29:26.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.200+0000 7f8fd9d9b640 1 --2- 172.21.15.114:0/1884918761 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f8fc40a13d0 0x7f8fc40a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:26.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.200+0000 7f8fe3a09740 1 Processor -- start 2024-09-13T18:29:26.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.211+0000 7f8fe3a09740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ed1ace970 0x562ed1ad9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:26.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.211+0000 7f8fe3a09740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562ed1acf290 0x562ed1ad9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:26.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.211+0000 7f8fe3a09740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ed1ad9b30 0x562ed1ad65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:26.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.211+0000 7f8fe3a09740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ed19f0180 con 0x562ed1acf290 2024-09-13T18:29:26.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.211+0000 7f8fe3a09740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ed170a430 con 0x562ed1ad9b30 2024-09-13T18:29:26.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.211+0000 7f8fe3a09740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ed19b55e0 con 0x562ed1ace970 2024-09-13T18:29:26.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.211+0000 7f8fe3a09740 10 client.0 _ll_register_callbacks cb 0x562ed1a21aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:29:26.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.213+0000 7f8fda59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ed1ace970 0x562ed1ad9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:26.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.213+0000 7f8fda59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ed1ace970 0x562ed1ad9020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:49270/0 (socket says 172.21.15.114:49270) 2024-09-13T18:29:26.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.213+0000 7f8fda59c640 1 -- 172.21.15.114:0/2863331381 learned_addr learned my addr 172.21.15.114:0/2863331381 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:26.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.215+0000 7f8fdad9d640 1 --2- 172.21.15.114:0/2863331381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ed1ad9b30 0x562ed1ad65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:26.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.219+0000 7f8fda59c640 1 -- 172.21.15.114:0/2863331381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ed1ad9b30 msgr2=0x562ed1ad65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:26.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.219+0000 7f8fda59c640 1 --2- 172.21.15.114:0/2863331381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ed1ad9b30 0x562ed1ad65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.219+0000 7f8fda59c640 1 -- 172.21.15.114:0/2863331381 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562ed1acf290 msgr2=0x562ed1ad9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:26.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.219+0000 7f8fda59c640 1 --2- 172.21.15.114:0/2863331381 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562ed1acf290 0x562ed1ad9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.219+0000 7f8fda59c640 1 -- 172.21.15.114:0/2863331381 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fd0009b00 con 0x562ed1ace970 2024-09-13T18:29:26.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.220+0000 7f8fda59c640 1 --2- 172.21.15.114:0/2863331381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ed1ace970 0x562ed1ad9020 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0012dd0 tx=0x7f8fd0012e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=240d2003232d7225 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:26.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.220+0000 7f8fc1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:29:26.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.220+0000 7f8fc1ffb640 1 -- 172.21.15.114:0/2863331381 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd0004040 con 0x562ed1ace970 2024-09-13T18:29:26.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.220+0000 7f8fe3a09740 1 -- 172.21.15.114:0/2863331381 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562ed1a5bb80 con 0x562ed1ace970 2024-09-13T18:29:26.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.220+0000 7f8fc1ffb640 1 -- 172.21.15.114:0/2863331381 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f8fd00041a0 con 0x562ed1ace970 2024-09-13T18:29:26.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.220+0000 7f8fc1ffb640 1 -- 172.21.15.114:0/2863331381 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fd0004360 con 0x562ed1ace970 2024-09-13T18:29:26.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.223+0000 7f8fe3a09740 10 client.27046 fetch_fsmap learned FSMap version 34 2024-09-13T18:29:26.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.223+0000 7f8fe3a09740 1 -- 172.21.15.114:0/2863331381 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x562ed1a303c0 con 0x562ed1ace970 2024-09-13T18:29:26.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.224+0000 7f8fd9d9b640 1 --2- 172.21.15.114:0/1884918761 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f8fc40a13d0 0x7f8fc40a3890 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x562ed1acc260 tx=0x7f8fd401a640 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:26.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.224+0000 7f8fc1ffb640 1 -- 172.21.15.114:0/2863331381 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562ed1a5bb80 con 0x562ed1ace970 2024-09-13T18:29:26.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.224+0000 7f8fc1ffb640 1 -- 172.21.15.114:0/2863331381 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8fd0011070 con 0x562ed1ace970 2024-09-13T18:29:26.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.224+0000 7f8fe3a09740 10 client.27046 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:29:26.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.224+0000 7f8fe3a09740 10 client.27046 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-13T18:29:26.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.224+0000 7f8fe3a09740 20 client.27046 populate_metadata read hostname 'smithi114' 2024-09-13T18:29:26.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.224+0000 7f8fe3a09740 1 --2- 172.21.15.114:0/2863331381 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x562ed1ae9410 0x562ed1aeb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:26.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.224+0000 7f8fe3a09740 4 client.27046 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:29:26.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.224+0000 7f8fe3a09740 1 -- 172.21.15.114:0/2863331381 --> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562ed1a241f0 con 0x562ed1ae9410 2024-09-13T18:29:26.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.226+0000 7f8fd9d9b640 1 --2- 172.21.15.114:0/2863331381 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x562ed1ae9410 0x562ed1aeb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:26.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.230+0000 7f8fd9d9b640 1 --2- 172.21.15.114:0/2863331381 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x562ed1ae9410 0x562ed1aeb830 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=358c712b5882b34d server_cookie=4ef0ec271a7d8fd8 in_seq=0 out_seq=0 2024-09-13T18:29:26.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.230+0000 7f8fc1ffb640 10 client.27046 ms_handle_connect on v2:172.21.15.134:6836/891692210 2024-09-13T18:29:26.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.231+0000 7f8fc1ffb640 1 -- 172.21.15.114:0/2863331381 <== mds.1 v2:172.21.15.134:6836/891692210 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x562ed1a241f0 con 0x562ed1ae9410 2024-09-13T18:29:26.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.231+0000 7f8fc1ffb640 10 client.27046 handle_command_reply: tid=0 2024-09-13T18:29:26.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.231+0000 7f8fe3a09740 1 client.27046 shutdown 2024-09-13T18:29:26.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.235+0000 7f8fe3a09740 1 -- 172.21.15.114:0/2863331381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ed1ace970 msgr2=0x562ed1ad9020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:26.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.235+0000 7f8fe3a09740 1 --2- 172.21.15.114:0/2863331381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ed1ace970 0x562ed1ad9020 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0012dd0 tx=0x7f8fd0012e00 comp rx=0 tx=0).stop 2024-09-13T18:29:26.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.236+0000 7f8fe3a09740 1 -- 172.21.15.114:0/2863331381 shutdown_connections 2024-09-13T18:29:26.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.236+0000 7f8fe3a09740 1 --2- 172.21.15.114:0/2863331381 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x562ed1ae9410 0x562ed1aeb830 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.236+0000 7f8fe3a09740 1 --2- 172.21.15.114:0/2863331381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ed1ad9b30 0x562ed1ad65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.236+0000 7f8fe3a09740 1 --2- 172.21.15.114:0/2863331381 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562ed1acf290 0x562ed1ad9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.236+0000 7f8fe3a09740 1 --2- 172.21.15.114:0/2863331381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ed1ace970 0x562ed1ad9020 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.236+0000 7f8fe3a09740 1 -- 172.21.15.114:0/2863331381 >> 172.21.15.114:0/2863331381 conn(0x562ed1acaee0 msgr2=0x562ed1aca230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:26.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.236+0000 7f8fc1ffb640 0 client.27046 ms_handle_reset on v2:172.21.15.134:6836/891692210 2024-09-13T18:29:26.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.237+0000 7f8fe3a09740 1 -- 172.21.15.114:0/2863331381 shutdown_connections 2024-09-13T18:29:26.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.237+0000 7f8fe3a09740 1 -- 172.21.15.114:0/2863331381 wait complete. 2024-09-13T18:29:26.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.237+0000 7f8fe3a09740 20 client.27046 trim_cache size 0 max 16384 2024-09-13T18:29:26.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.278+0000 7f8fe3a09740 1 Processor -- start 2024-09-13T18:29:26.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.278+0000 7f8fe3a09740 1 -- start start 2024-09-13T18:29:26.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.278+0000 7f8fe3a09740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562ed1ace970 0x562ed1ae0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:26.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.279+0000 7f8fe3a09740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ed1acf290 0x562ed1ae0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:26.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.279+0000 7f8fda59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562ed1ace970 0x562ed1ae0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:26.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.279+0000 7f8fda59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562ed1ace970 0x562ed1ae0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57650/0 (socket says 172.21.15.114:57650) 2024-09-13T18:29:26.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.279+0000 7f8fda59c640 1 -- 172.21.15.114:0/3302148290 learned_addr learned my addr 172.21.15.114:0/3302148290 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:26.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.279+0000 7f8fe3a09740 1 --2- 172.21.15.114:0/3302148290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ed1ad9b30 0x562ed1ad4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:26.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.279+0000 7f8fd9d9b640 1 --2- 172.21.15.114:0/3302148290 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ed1acf290 0x562ed1ae0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:26.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.279+0000 7f8fe3a09740 1 -- 172.21.15.114:0/3302148290 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ed1ac6d00 con 0x562ed1ace970 2024-09-13T18:29:26.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.279+0000 7f8fe3a09740 1 -- 172.21.15.114:0/3302148290 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ed1ac56f0 con 0x562ed1acf290 2024-09-13T18:29:26.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.280+0000 7f8fdad9d640 1 --2- 172.21.15.114:0/3302148290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ed1ad9b30 0x562ed1ad4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:26.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.280+0000 7f8fe3a09740 1 -- 172.21.15.114:0/3302148290 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ed1acf800 con 0x562ed1ad9b30 2024-09-13T18:29:26.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.284+0000 7f8fd9d9b640 1 -- 172.21.15.114:0/3302148290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ed1ad9b30 msgr2=0x562ed1ad4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:26.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.284+0000 7f8fd9d9b640 1 --2- 172.21.15.114:0/3302148290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ed1ad9b30 0x562ed1ad4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.284+0000 7f8fd9d9b640 1 -- 172.21.15.114:0/3302148290 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562ed1ace970 msgr2=0x562ed1ae0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:26.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.284+0000 7f8fd9d9b640 1 --2- 172.21.15.114:0/3302148290 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562ed1ace970 0x562ed1ae0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.284+0000 7f8fd9d9b640 1 -- 172.21.15.114:0/3302148290 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ed19d2770 con 0x562ed1acf290 2024-09-13T18:29:26.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.284+0000 7f8fd9d9b640 1 --2- 172.21.15.114:0/3302148290 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ed1acf290 0x562ed1ae0e60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc116100 tx=0x7f8fd4050000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fedce69a35df8fb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:26.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.285+0000 7f8fc97fa640 1 -- 172.21.15.114:0/3302148290 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd405e040 con 0x562ed1acf290 2024-09-13T18:29:26.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.285+0000 7f8fc97fa640 1 -- 172.21.15.114:0/3302148290 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8fd4040560 con 0x562ed1acf290 2024-09-13T18:29:26.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.286+0000 7f8fc97fa640 1 -- 172.21.15.114:0/3302148290 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd4050070 con 0x562ed1acf290 2024-09-13T18:29:26.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.287+0000 7f8fe3a09740 1 -- 172.21.15.114:0/3302148290 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ed1acf290 msgr2=0x562ed1ae0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:26.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.287+0000 7f8fe3a09740 1 --2- 172.21.15.114:0/3302148290 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ed1acf290 0x562ed1ae0e60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc116100 tx=0x7f8fd4050000 comp rx=0 tx=0).stop 2024-09-13T18:29:26.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.290+0000 7f8fe3a09740 1 -- 172.21.15.114:0/3302148290 shutdown_connections 2024-09-13T18:29:26.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.290+0000 7f8fe3a09740 1 --2- 172.21.15.114:0/3302148290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ed1ad9b30 0x562ed1ad4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.290+0000 7f8fe3a09740 1 --2- 172.21.15.114:0/3302148290 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ed1acf290 0x562ed1ae0e60 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.290+0000 7f8fe3a09740 1 --2- 172.21.15.114:0/3302148290 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562ed1ace970 0x562ed1ae0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.290+0000 7f8fe3a09740 1 -- 172.21.15.114:0/3302148290 >> 172.21.15.114:0/3302148290 conn(0x562ed1acaee0 msgr2=0x562ed1acb2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:26.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.291+0000 7f8fe3a09740 1 -- 172.21.15.114:0/3302148290 shutdown_connections 2024-09-13T18:29:26.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.291+0000 7f8fe3a09740 1 -- 172.21.15.114:0/3302148290 wait complete. 2024-09-13T18:29:26.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.291+0000 7f8fe3a09740 1 -- start start 2024-09-13T18:29:26.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.291+0000 7f8fe3a09740 1 Processor -- start 2024-09-13T18:29:26.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.292+0000 7f8fe3a09740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ed1ace970 0x562ed1b38b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:26.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.292+0000 7f8fe3a09740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ed1acf290 0x562ed1b39090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:26.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.292+0000 7f8fe3a09740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562ed1ad9b30 0x562ed1b378a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:26.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.292+0000 7f8fe3a09740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ed1b37e10 con 0x562ed1ad9b30 2024-09-13T18:29:26.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.292+0000 7f8fe3a09740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ed1b37f80 con 0x562ed1ace970 2024-09-13T18:29:26.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.292+0000 7f8fe3a09740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ed1b38140 con 0x562ed1acf290 2024-09-13T18:29:26.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.292+0000 7f8fe3a09740 10 client.0 _ll_register_callbacks cb 0x7f8fd00098f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:29:26.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.293+0000 7f8fdad9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562ed1ad9b30 0x562ed1b378a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:26.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.293+0000 7f8fdad9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562ed1ad9b30 0x562ed1b378a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57652/0 (socket says 172.21.15.114:57652) 2024-09-13T18:29:26.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.293+0000 7f8fdad9d640 1 -- 172.21.15.114:0/2292645136 learned_addr learned my addr 172.21.15.114:0/2292645136 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:26.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.293+0000 7f8fdad9d640 1 -- 172.21.15.114:0/2292645136 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ed1acf290 msgr2=0x562ed1b39090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:29:26.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.293+0000 7f8fdad9d640 1 --2- 172.21.15.114:0/2292645136 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ed1acf290 0x562ed1b39090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.293+0000 7f8fdad9d640 1 -- 172.21.15.114:0/2292645136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ed1ace970 msgr2=0x562ed1b38b50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:29:26.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.293+0000 7f8fdad9d640 1 --2- 172.21.15.114:0/2292645136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ed1ace970 0x562ed1b38b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.293+0000 7f8fdad9d640 1 -- 172.21.15.114:0/2292645136 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fdc11c170 con 0x562ed1ad9b30 2024-09-13T18:29:26.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.294+0000 7f8fdad9d640 1 --2- 172.21.15.114:0/2292645136 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562ed1ad9b30 0x562ed1b378a0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc0997c0 tx=0x7f8fcc099250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7875d4f39e09ecb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:26.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.294+0000 7f8fc1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:29:26.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.294+0000 7f8fc1ffb640 1 -- 172.21.15.114:0/2292645136 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc0b2220 con 0x562ed1ad9b30 2024-09-13T18:29:26.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.294+0000 7f8fc1ffb640 1 -- 172.21.15.114:0/2292645136 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f8fcc099f30 con 0x562ed1ad9b30 2024-09-13T18:29:26.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.295+0000 7f8fe3a09740 1 -- 172.21.15.114:0/2292645136 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8fd0013a60 con 0x562ed1ad9b30 2024-09-13T18:29:26.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.295+0000 7f8fc1ffb640 1 -- 172.21.15.114:0/2292645136 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fcc0ca1e0 con 0x562ed1ad9b30 2024-09-13T18:29:26.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.295+0000 7f8fe3a09740 10 client.17631 fetch_fsmap learned FSMap version 34 2024-09-13T18:29:26.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.295+0000 7f8fe3a09740 1 -- 172.21.15.114:0/2292645136 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x562ed19f7e80 con 0x562ed1ad9b30 2024-09-13T18:29:26.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.296+0000 7f8fc1ffb640 1 -- 172.21.15.114:0/2292645136 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8fcc099830 con 0x562ed1ad9b30 2024-09-13T18:29:26.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.296+0000 7f8fc1ffb640 1 -- 172.21.15.114:0/2292645136 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8fcc0a1820 con 0x562ed1ad9b30 2024-09-13T18:29:26.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.297+0000 7f8fe3a09740 10 client.17631 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:29:26.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.298+0000 7f8fe3a09740 10 client.17631 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-13T18:29:26.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.298+0000 7f8fe3a09740 20 client.17631 populate_metadata read hostname 'smithi114' 2024-09-13T18:29:26.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.298+0000 7f8fe3a09740 1 --2- 172.21.15.114:0/2292645136 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x562ed1b40bd0 0x562ed1b42fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:26.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.298+0000 7f8fe3a09740 4 client.17631 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:29:26.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.298+0000 7f8fe3a09740 1 -- 172.21.15.114:0/2292645136 --> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f8fd0013a60 con 0x562ed1b40bd0 2024-09-13T18:29:26.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.301+0000 7f8fda59c640 1 --2- 172.21.15.114:0/2292645136 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x562ed1b40bd0 0x562ed1b42fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:26.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.302+0000 7f8fda59c640 1 --2- 172.21.15.114:0/2292645136 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x562ed1b40bd0 0x562ed1b42fc0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cf333bc01c65ea86 server_cookie=f5d8b2e3f4323038 in_seq=0 out_seq=0 2024-09-13T18:29:26.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.302+0000 7f8fc1ffb640 10 client.17631 ms_handle_connect on v2:172.21.15.134:6836/891692210 2024-09-13T18:29:26.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.302+0000 7f8fc1ffb640 1 -- 172.21.15.114:0/2292645136 <== mds.1 v2:172.21.15.134:6836/891692210 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f8fd0013a60 con 0x562ed1b40bd0 2024-09-13T18:29:26.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.302+0000 7f8fc1ffb640 10 client.17631 handle_command_reply: tid=0 2024-09-13T18:29:26.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.302+0000 7f8fe3a09740 1 client.17631 shutdown 2024-09-13T18:29:26.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.305+0000 7f8fe3a09740 1 -- 172.21.15.114:0/2292645136 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562ed1ad9b30 msgr2=0x562ed1b378a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:26.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.305+0000 7f8fe3a09740 1 --2- 172.21.15.114:0/2292645136 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562ed1ad9b30 0x562ed1b378a0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc0997c0 tx=0x7f8fcc099250 comp rx=0 tx=0).stop 2024-09-13T18:29:26.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.307+0000 7f8fe3a09740 1 -- 172.21.15.114:0/2292645136 shutdown_connections 2024-09-13T18:29:26.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.307+0000 7f8fe3a09740 1 --2- 172.21.15.114:0/2292645136 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x562ed1b40bd0 0x562ed1b42fc0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.307+0000 7f8fe3a09740 1 --2- 172.21.15.114:0/2292645136 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562ed1ad9b30 0x562ed1b378a0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.307+0000 7f8fe3a09740 1 --2- 172.21.15.114:0/2292645136 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ed1acf290 0x562ed1b39090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.307+0000 7f8fe3a09740 1 --2- 172.21.15.114:0/2292645136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ed1ace970 0x562ed1b38b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.307+0000 7f8fe3a09740 1 -- 172.21.15.114:0/2292645136 >> 172.21.15.114:0/2292645136 conn(0x562ed1acaee0 msgr2=0x562ed1ad14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:26.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.307+0000 7f8fc1ffb640 0 client.17631 ms_handle_reset on v2:172.21.15.134:6836/891692210 2024-09-13T18:29:26.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.308+0000 7f8fe3a09740 1 -- 172.21.15.114:0/2292645136 shutdown_connections 2024-09-13T18:29:26.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.308+0000 7f8fe3a09740 1 -- 172.21.15.114:0/2292645136 wait complete. 2024-09-13T18:29:26.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.317+0000 7f8fe3a09740 20 client.17631 trim_cache size 0 max 16384 2024-09-13T18:29:26.322 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:29:26.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.324+0000 7f8fdb59e640 1 -- 172.21.15.114:0/1884918761 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f8fc40a13d0 msgr2=0x7f8fc40a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:26.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.324+0000 7f8fdb59e640 1 --2- 172.21.15.114:0/1884918761 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f8fc40a13d0 0x7f8fc40a3890 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x562ed1acc260 tx=0x7f8fd401a640 comp rx=0 tx=0).stop 2024-09-13T18:29:26.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.324+0000 7f8fdb59e640 1 -- 172.21.15.114:0/1884918761 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8fdc119400 msgr2=0x7f8fdc1148d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:26.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.324+0000 7f8fdb59e640 1 --2- 172.21.15.114:0/1884918761 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8fdc119400 0x7f8fdc1148d0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc00b410 tx=0x7f8fcc00b8d0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.325+0000 7f8fdb59e640 1 -- 172.21.15.114:0/1884918761 shutdown_connections 2024-09-13T18:29:26.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.325+0000 7f8fdb59e640 1 --2- 172.21.15.114:0/1884918761 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f8fc40a13d0 0x7f8fc40a3890 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.325+0000 7f8fdb59e640 1 --2- 172.21.15.114:0/1884918761 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8fdc119400 0x7f8fdc1148d0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.325+0000 7f8fdb59e640 1 --2- 172.21.15.114:0/1884918761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fdc107890 0x7f8fdc118c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.325+0000 7f8fdb59e640 1 --2- 172.21.15.114:0/1884918761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8fdc1074b0 0x7f8fdc1186f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.325+0000 7f8fdb59e640 1 -- 172.21.15.114:0/1884918761 >> 172.21.15.114:0/1884918761 conn(0x7f8fdc06af40 msgr2=0x7f8fdc06ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:26.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.328+0000 7f8fdb59e640 1 -- 172.21.15.114:0/1884918761 shutdown_connections 2024-09-13T18:29:26.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.328+0000 7f8fdb59e640 1 -- 172.21.15.114:0/1884918761 wait complete. 2024-09-13T18:29:26.352 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-09-13T18:29:26.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.569+0000 7fc8491ce640 1 -- 172.21.15.114:0/2471185201 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc844104160 msgr2=0x7fc8441045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:26.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.569+0000 7fc8491ce640 1 --2- 172.21.15.114:0/2471185201 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc844104160 0x7fc8441045f0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fc838009900 tx=0x7fc83802a4d0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.575+0000 7fc8491ce640 1 -- 172.21.15.114:0/2471185201 shutdown_connections 2024-09-13T18:29:26.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.575+0000 7fc8491ce640 1 --2- 172.21.15.114:0/2471185201 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc844104160 0x7fc8441045f0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.575+0000 7fc8491ce640 1 --2- 172.21.15.114:0/2471185201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8440ff650 0x7fc844103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.575+0000 7fc8491ce640 1 --2- 172.21.15.114:0/2471185201 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8440fed30 0x7fc8440ff110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.575+0000 7fc8491ce640 1 -- 172.21.15.114:0/2471185201 >> 172.21.15.114:0/2471185201 conn(0x7fc844075c60 msgr2=0x7fc8440fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:26.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.578+0000 7fc8491ce640 1 -- 172.21.15.114:0/2471185201 shutdown_connections 2024-09-13T18:29:26.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.580+0000 7fc8491ce640 1 -- 172.21.15.114:0/2471185201 wait complete. 2024-09-13T18:29:26.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.581+0000 7fc8491ce640 1 Processor -- start 2024-09-13T18:29:26.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.581+0000 7fc8491ce640 1 -- start start 2024-09-13T18:29:26.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.581+0000 7fc8491ce640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8440fed30 0x7fc8441a2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:26.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.581+0000 7fc8491ce640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8440ff650 0x7fc8441a2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:26.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.581+0000 7fc8491ce640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc844104160 0x7fc84419c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:26.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.581+0000 7fc8491ce640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84419cc90 con 0x7fc8440ff650 2024-09-13T18:29:26.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.581+0000 7fc8491ce640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84419ce00 con 0x7fc8440fed30 2024-09-13T18:29:26.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.581+0000 7fc8491ce640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84419cfc0 con 0x7fc844104160 2024-09-13T18:29:26.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.582+0000 7fc842d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8440fed30 0x7fc8441a2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:26.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.582+0000 7fc842d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8440fed30 0x7fc8441a2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:56298/0 (socket says 172.21.15.114:56298) 2024-09-13T18:29:26.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.582+0000 7fc842d9d640 1 -- 172.21.15.114:0/3588709575 learned_addr learned my addr 172.21.15.114:0/3588709575 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:26.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.582+0000 7fc84359e640 1 --2- 172.21.15.114:0/3588709575 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc844104160 0x7fc84419c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:26.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.583+0000 7fc842d9d640 1 -- 172.21.15.114:0/3588709575 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc844104160 msgr2=0x7fc84419c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:26.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.585+0000 7fc842d9d640 1 --2- 172.21.15.114:0/3588709575 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc844104160 0x7fc84419c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.585+0000 7fc842d9d640 1 -- 172.21.15.114:0/3588709575 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8440ff650 msgr2=0x7fc8441a2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:26.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.585+0000 7fc842d9d640 1 --2- 172.21.15.114:0/3588709575 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8440ff650 0x7fc8441a2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.585+0000 7fc842d9d640 1 -- 172.21.15.114:0/3588709575 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc838009560 con 0x7fc8440fed30 2024-09-13T18:29:26.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.587+0000 7fc842d9d640 1 --2- 172.21.15.114:0/3588709575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8440fed30 0x7fc8441a2590 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc83c00b4a0 tx=0x7fc83c00b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:26.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.589+0000 7fc82bfff640 1 -- 172.21.15.114:0/3588709575 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc83c010610 con 0x7fc8440fed30 2024-09-13T18:29:26.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.589+0000 7fc82bfff640 1 -- 172.21.15.114:0/3588709575 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc83c010c00 con 0x7fc8440fed30 2024-09-13T18:29:26.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.590+0000 7fc8491ce640 1 -- 172.21.15.114:0/3588709575 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc84419d2a0 con 0x7fc8440fed30 2024-09-13T18:29:26.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.590+0000 7fc8491ce640 1 -- 172.21.15.114:0/3588709575 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc84406a0b0 con 0x7fc8440fed30 2024-09-13T18:29:26.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.591+0000 7fc82bfff640 1 -- 172.21.15.114:0/3588709575 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc83c017cb0 con 0x7fc8440fed30 2024-09-13T18:29:26.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.595+0000 7fc84bfca740 1 Processor -- start 2024-09-13T18:29:26.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.596+0000 7fc84bfca740 1 -- start start 2024-09-13T18:29:26.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.599+0000 7fc84bfca740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5586db54e790 0x5586db54eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:26.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.599+0000 7fc84bfca740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586db54f0b0 0x5586db559410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:26.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.599+0000 7fc84bfca740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586db559950 0x5586db55bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:26.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.599+0000 7fc84bfca740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586db4b8cc0 con 0x5586db54e790 2024-09-13T18:29:26.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.599+0000 7fc84bfca740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586db4b6450 con 0x5586db559950 2024-09-13T18:29:26.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.599+0000 7fc84bfca740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586db4b1d40 con 0x5586db54f0b0 2024-09-13T18:29:26.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.599+0000 7fc82bfff640 1 -- 172.21.15.114:0/3588709575 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fc83c017360 con 0x7fc8440fed30 2024-09-13T18:29:26.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.599+0000 7fc842d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586db54f0b0 0x5586db559410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:26.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.600+0000 7fc842d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586db54f0b0 0x5586db559410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:49308/0 (socket says 172.21.15.114:49308) 2024-09-13T18:29:26.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.600+0000 7fc842d9d640 1 -- 172.21.15.114:0/165033479 learned_addr learned my addr 172.21.15.114:0/165033479 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:26.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.602+0000 7fc842d9d640 1 -- 172.21.15.114:0/165033479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586db559950 msgr2=0x5586db55bd40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:29:26.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.602+0000 7fc842d9d640 1 --2- 172.21.15.114:0/165033479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586db559950 0x5586db55bd40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.602+0000 7fc842d9d640 1 -- 172.21.15.114:0/165033479 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5586db54e790 msgr2=0x5586db54eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:26.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.602+0000 7fc842d9d640 1 --2- 172.21.15.114:0/165033479 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5586db54e790 0x5586db54eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.602+0000 7fc842d9d640 1 -- 172.21.15.114:0/165033479 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586db479620 con 0x5586db54f0b0 2024-09-13T18:29:26.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.603+0000 7fc842d9d640 1 --2- 172.21.15.114:0/165033479 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586db54f0b0 0x5586db559410 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc83c0c9c00 tx=0x7fc83c0d2ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e42f1fc792e6025d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:26.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.603+0000 7fc82bfff640 1 --2- 172.21.15.114:0/3588709575 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc82c0a12e0 0x7fc82c0a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:26.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.603+0000 7fc82bfff640 1 -- 172.21.15.114:0/3588709575 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fc83c0cc860 con 0x7fc8440fed30 2024-09-13T18:29:26.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.606+0000 7fc8297fa640 1 -- 172.21.15.114:0/165033479 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc83c0d4e10 con 0x5586db54f0b0 2024-09-13T18:29:26.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.606+0000 7fc8297fa640 1 -- 172.21.15.114:0/165033479 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc83c0959c0 con 0x5586db54f0b0 2024-09-13T18:29:26.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.607+0000 7fc84bfca740 1 -- 172.21.15.114:0/165033479 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586db54f0b0 msgr2=0x5586db559410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:26.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.607+0000 7fc84bfca740 1 --2- 172.21.15.114:0/165033479 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586db54f0b0 0x5586db559410 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc83c0c9c00 tx=0x7fc83c0d2ef0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.608+0000 7fc84359e640 1 --2- 172.21.15.114:0/3588709575 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc82c0a12e0 0x7fc82c0a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:26.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.609+0000 7fc84359e640 1 --2- 172.21.15.114:0/3588709575 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc82c0a12e0 0x7fc82c0a37a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc8380058c0 tx=0x7fc838005850 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:26.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.609+0000 7fc84bfca740 1 -- 172.21.15.114:0/165033479 shutdown_connections 2024-09-13T18:29:26.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.609+0000 7fc84bfca740 1 --2- 172.21.15.114:0/165033479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586db559950 0x5586db55bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.609+0000 7fc84bfca740 1 --2- 172.21.15.114:0/165033479 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586db54f0b0 0x5586db559410 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.609+0000 7fc84bfca740 1 --2- 172.21.15.114:0/165033479 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5586db54e790 0x5586db54eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.609+0000 7fc84bfca740 1 -- 172.21.15.114:0/165033479 >> 172.21.15.114:0/165033479 conn(0x5586db54ad00 msgr2=0x5586db54b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:26.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.618+0000 7fc84bfca740 1 -- 172.21.15.114:0/165033479 shutdown_connections 2024-09-13T18:29:26.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.618+0000 7fc84bfca740 1 -- 172.21.15.114:0/165033479 wait complete. 2024-09-13T18:29:26.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.619+0000 7fc84bfca740 1 -- start start 2024-09-13T18:29:26.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.619+0000 7fc84bfca740 1 Processor -- start 2024-09-13T18:29:26.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.620+0000 7fc84bfca740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5586db54e790 0x5586db5562f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:26.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.620+0000 7fc84bfca740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586db54f0b0 0x5586db556830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:26.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.620+0000 7fc84bfca740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586db559950 0x5586db55d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:26.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.620+0000 7fc84bfca740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586db470180 con 0x5586db54e790 2024-09-13T18:29:26.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.620+0000 7fc84bfca740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586db18a430 con 0x5586db54f0b0 2024-09-13T18:29:26.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.620+0000 7fc84bfca740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586db4355e0 con 0x5586db559950 2024-09-13T18:29:26.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.620+0000 7fc84bfca740 10 client.0 _ll_register_callbacks cb 0x5586db4a1aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:29:26.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.621+0000 7fc84259c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586db54f0b0 0x5586db556830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:26.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.621+0000 7fc84359e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586db559950 0x5586db55d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:26.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.621+0000 7fc84359e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586db559950 0x5586db55d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:49310/0 (socket says 172.21.15.114:49310) 2024-09-13T18:29:26.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.621+0000 7fc84259c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586db54f0b0 0x5586db556830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:56324/0 (socket says 172.21.15.114:56324) 2024-09-13T18:29:26.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.621+0000 7fc84259c640 1 -- 172.21.15.114:0/1061474850 learned_addr learned my addr 172.21.15.114:0/1061474850 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:26.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.624+0000 7fc842d9d640 1 --2- 172.21.15.114:0/1061474850 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5586db54e790 0x5586db5562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:26.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.624+0000 7fc84259c640 1 -- 172.21.15.114:0/1061474850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586db559950 msgr2=0x5586db55d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:26.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.624+0000 7fc84259c640 1 --2- 172.21.15.114:0/1061474850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586db559950 0x5586db55d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.624+0000 7fc84259c640 1 -- 172.21.15.114:0/1061474850 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5586db54e790 msgr2=0x5586db5562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:26.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.624+0000 7fc84259c640 1 --2- 172.21.15.114:0/1061474850 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5586db54e790 0x5586db5562f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.624+0000 7fc84259c640 1 -- 172.21.15.114:0/1061474850 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc84406a0b0 con 0x5586db54f0b0 2024-09-13T18:29:26.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.624+0000 7fc84359e640 1 --2- 172.21.15.114:0/1061474850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586db559950 0x5586db55d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:29:26.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.629+0000 7fc84259c640 1 --2- 172.21.15.114:0/1061474850 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586db54f0b0 0x5586db556830 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc834009fd0 tx=0x7fc83400eba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5158b641e91e1f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:26.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.630+0000 7fc826ff5640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:29:26.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.630+0000 7fc826ff5640 1 -- 172.21.15.114:0/1061474850 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc834004d20 con 0x5586db54f0b0 2024-09-13T18:29:26.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.630+0000 7fc826ff5640 1 -- 172.21.15.114:0/1061474850 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fc834014070 con 0x5586db54f0b0 2024-09-13T18:29:26.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.632+0000 7fc84bfca740 1 -- 172.21.15.114:0/1061474850 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5586db4dbb80 con 0x5586db54f0b0 2024-09-13T18:29:26.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.633+0000 7fc826ff5640 1 -- 172.21.15.114:0/1061474850 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc834004e80 con 0x5586db54f0b0 2024-09-13T18:29:26.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.640+0000 7fc84bfca740 10 client.27050 fetch_fsmap learned FSMap version 34 2024-09-13T18:29:26.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.640+0000 7fc84bfca740 1 -- 172.21.15.114:0/1061474850 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5586db4b03c0 con 0x5586db54f0b0 2024-09-13T18:29:26.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.642+0000 7fc826ff5640 1 -- 172.21.15.114:0/1061474850 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5586db4dbb80 con 0x5586db54f0b0 2024-09-13T18:29:26.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.642+0000 7fc826ff5640 1 -- 172.21.15.114:0/1061474850 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc834004030 con 0x5586db54f0b0 2024-09-13T18:29:26.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.642+0000 7fc84bfca740 10 client.27050 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:29:26.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.642+0000 7fc84bfca740 10 client.27050 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:29:26.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.642+0000 7fc84bfca740 20 client.27050 populate_metadata read hostname 'smithi114' 2024-09-13T18:29:26.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.642+0000 7fc84bfca740 1 --2- 172.21.15.114:0/1061474850 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5586db569270 0x5586db56b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:26.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.642+0000 7fc84bfca740 4 client.27050 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:29:26.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.642+0000 7fc84bfca740 1 -- 172.21.15.114:0/1061474850 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5586db4a41f0 con 0x5586db569270 2024-09-13T18:29:26.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.644+0000 7fc842d9d640 1 --2- 172.21.15.114:0/1061474850 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5586db569270 0x5586db56b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:26.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.645+0000 7fc842d9d640 1 --2- 172.21.15.114:0/1061474850 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5586db569270 0x5586db56b690 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d457821028dfc37 server_cookie=5b55d759d404b24d in_seq=0 out_seq=0 2024-09-13T18:29:26.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.645+0000 7fc826ff5640 10 client.27050 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:29:26.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.645+0000 7fc826ff5640 1 -- 172.21.15.114:0/1061474850 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5586db4a41f0 con 0x5586db569270 2024-09-13T18:29:26.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.645+0000 7fc826ff5640 10 client.27050 handle_command_reply: tid=0 2024-09-13T18:29:26.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.647+0000 7fc84bfca740 1 client.27050 shutdown 2024-09-13T18:29:26.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.652+0000 7fc84bfca740 1 -- 172.21.15.114:0/1061474850 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586db54f0b0 msgr2=0x5586db556830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:26.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.652+0000 7fc84bfca740 1 --2- 172.21.15.114:0/1061474850 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586db54f0b0 0x5586db556830 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc834009fd0 tx=0x7fc83400eba0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.653+0000 7fc84bfca740 1 -- 172.21.15.114:0/1061474850 shutdown_connections 2024-09-13T18:29:26.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.653+0000 7fc84bfca740 1 --2- 172.21.15.114:0/1061474850 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5586db569270 0x5586db56b690 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.653+0000 7fc84bfca740 1 --2- 172.21.15.114:0/1061474850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586db559950 0x5586db55d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.653+0000 7fc84bfca740 1 --2- 172.21.15.114:0/1061474850 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586db54f0b0 0x5586db556830 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.653+0000 7fc84bfca740 1 --2- 172.21.15.114:0/1061474850 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5586db54e790 0x5586db5562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.653+0000 7fc84bfca740 1 -- 172.21.15.114:0/1061474850 >> 172.21.15.114:0/1061474850 conn(0x5586db54ad00 msgr2=0x5586db550a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:26.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.653+0000 7fc826ff5640 0 client.27050 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:29:26.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.656+0000 7fc84bfca740 1 -- 172.21.15.114:0/1061474850 shutdown_connections 2024-09-13T18:29:26.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.656+0000 7fc84bfca740 1 -- 172.21.15.114:0/1061474850 wait complete. 2024-09-13T18:29:26.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.664+0000 7fc84bfca740 20 client.27050 trim_cache size 0 max 16384 2024-09-13T18:29:26.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.692+0000 7fc84bfca740 1 Processor -- start 2024-09-13T18:29:26.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.692+0000 7fc84bfca740 1 -- start start 2024-09-13T18:29:26.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.693+0000 7fc84bfca740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586db54e790 0x5586db55d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:26.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.693+0000 7fc84bfca740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586db54f0b0 0x5586db5549d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:26.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.693+0000 7fc84bfca740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5586db559950 0x5586db554f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:26.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.694+0000 7fc84359e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5586db559950 0x5586db554f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:26.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.694+0000 7fc84359e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5586db559950 0x5586db554f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57694/0 (socket says 172.21.15.114:57694) 2024-09-13T18:29:26.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.694+0000 7fc84359e640 1 -- 172.21.15.114:0/2605553463 learned_addr learned my addr 172.21.15.114:0/2605553463 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:26.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.694+0000 7fc84bfca740 1 -- 172.21.15.114:0/2605553463 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586db555450 con 0x5586db559950 2024-09-13T18:29:26.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.694+0000 7fc84bfca740 1 -- 172.21.15.114:0/2605553463 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586db54e510 con 0x5586db54f0b0 2024-09-13T18:29:26.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.694+0000 7fc84bfca740 1 -- 172.21.15.114:0/2605553463 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586db54fec0 con 0x5586db54e790 2024-09-13T18:29:26.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.695+0000 7fc84359e640 1 -- 172.21.15.114:0/2605553463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586db54e790 msgr2=0x5586db55d5e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:29:26.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.695+0000 7fc84359e640 1 --2- 172.21.15.114:0/2605553463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586db54e790 0x5586db55d5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.695+0000 7fc84359e640 1 -- 172.21.15.114:0/2605553463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586db54f0b0 msgr2=0x5586db5549d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:29:26.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.695+0000 7fc84359e640 1 --2- 172.21.15.114:0/2605553463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586db54f0b0 0x5586db5549d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.695+0000 7fc84359e640 1 -- 172.21.15.114:0/2605553463 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586db452770 con 0x5586db559950 2024-09-13T18:29:26.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.696+0000 7fc84359e640 1 --2- 172.21.15.114:0/2605553463 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5586db559950 0x5586db554f10 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fc838009230 tx=0x7fc83803d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7be56b0e3656203 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:26.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.696+0000 7fc829ffb640 1 -- 172.21.15.114:0/2605553463 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc838003970 con 0x5586db559950 2024-09-13T18:29:26.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.696+0000 7fc829ffb640 1 -- 172.21.15.114:0/2605553463 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc838053470 con 0x5586db559950 2024-09-13T18:29:26.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.697+0000 7fc84bfca740 1 -- 172.21.15.114:0/2605553463 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5586db559950 msgr2=0x5586db554f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:26.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.697+0000 7fc84bfca740 1 --2- 172.21.15.114:0/2605553463 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5586db559950 0x5586db554f10 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fc838009230 tx=0x7fc83803d000 comp rx=0 tx=0).stop 2024-09-13T18:29:26.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.698+0000 7fc84bfca740 1 -- 172.21.15.114:0/2605553463 shutdown_connections 2024-09-13T18:29:26.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.698+0000 7fc84bfca740 1 --2- 172.21.15.114:0/2605553463 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5586db559950 0x5586db554f10 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.698+0000 7fc84bfca740 1 --2- 172.21.15.114:0/2605553463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586db54f0b0 0x5586db5549d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.698+0000 7fc84bfca740 1 --2- 172.21.15.114:0/2605553463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586db54e790 0x5586db55d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.698+0000 7fc84bfca740 1 -- 172.21.15.114:0/2605553463 >> 172.21.15.114:0/2605553463 conn(0x5586db54ad00 msgr2=0x5586db54b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:26.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.701+0000 7fc84bfca740 1 -- 172.21.15.114:0/2605553463 shutdown_connections 2024-09-13T18:29:26.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.701+0000 7fc84bfca740 1 -- 172.21.15.114:0/2605553463 wait complete. 2024-09-13T18:29:26.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.702+0000 7fc84bfca740 1 -- start start 2024-09-13T18:29:26.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.702+0000 7fc84bfca740 1 Processor -- start 2024-09-13T18:29:26.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.703+0000 7fc84bfca740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586db559950 0x5586db560d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:26.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.703+0000 7fc84bfca740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5586db561280 0x5586db561700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:26.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.703+0000 7fc842d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586db559950 0x5586db560d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:26.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.703+0000 7fc842d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586db559950 0x5586db560d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:49312/0 (socket says 172.21.15.114:49312) 2024-09-13T18:29:26.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.703+0000 7fc842d9d640 1 -- 172.21.15.114:0/1325560777 learned_addr learned my addr 172.21.15.114:0/1325560777 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:26.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.703+0000 7fc84bfca740 1 --2- 172.21.15.114:0/1325560777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586db5bc750 0x5586db5beb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:26.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.703+0000 7fc84bfca740 1 -- 172.21.15.114:0/1325560777 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586db5bf1a0 con 0x5586db561280 2024-09-13T18:29:26.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.703+0000 7fc84bfca740 1 -- 172.21.15.114:0/1325560777 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586db5bf310 con 0x5586db5bc750 2024-09-13T18:29:26.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.703+0000 7fc84bfca740 1 -- 172.21.15.114:0/1325560777 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586db5bf4d0 con 0x5586db559950 2024-09-13T18:29:26.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.704+0000 7fc84359e640 1 --2- 172.21.15.114:0/1325560777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586db5bc750 0x5586db5beb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:26.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.704+0000 7fc84bfca740 10 client.0 _ll_register_callbacks cb 0x7fc83c0d2400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:29:26.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.704+0000 7fc84259c640 1 --2- 172.21.15.114:0/1325560777 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5586db561280 0x5586db561700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:26.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.706+0000 7fc842d9d640 1 -- 172.21.15.114:0/1325560777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586db5bc750 msgr2=0x5586db5beb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:26.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.706+0000 7fc842d9d640 1 --2- 172.21.15.114:0/1325560777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586db5bc750 0x5586db5beb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.706+0000 7fc842d9d640 1 -- 172.21.15.114:0/1325560777 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5586db561280 msgr2=0x5586db561700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:26.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.706+0000 7fc842d9d640 1 --2- 172.21.15.114:0/1325560777 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5586db561280 0x5586db561700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.706+0000 7fc842d9d640 1 -- 172.21.15.114:0/1325560777 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc844113a60 con 0x5586db559950 2024-09-13T18:29:26.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.707+0000 7fc842d9d640 1 --2- 172.21.15.114:0/1325560777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586db559950 0x5586db560d40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc83c095b30 tx=0x7fc83c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86fcec00f1e3114c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:26.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.707+0000 7fc826ff5640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:29:26.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.709+0000 7fc826ff5640 1 -- 172.21.15.114:0/1325560777 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc83c0cca70 con 0x5586db559950 2024-09-13T18:29:26.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.709+0000 7fc84bfca740 1 -- 172.21.15.114:0/1325560777 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc83c0959c0 con 0x5586db559950 2024-09-13T18:29:26.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.709+0000 7fc826ff5640 1 -- 172.21.15.114:0/1325560777 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fc83c099d80 con 0x5586db559950 2024-09-13T18:29:26.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.709+0000 7fc826ff5640 1 -- 172.21.15.114:0/1325560777 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc83c099450 con 0x5586db559950 2024-09-13T18:29:26.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.713+0000 7fc84bfca740 10 client.27064 fetch_fsmap learned FSMap version 34 2024-09-13T18:29:26.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.713+0000 7fc84bfca740 1 -- 172.21.15.114:0/1325560777 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5586db477e80 con 0x5586db559950 2024-09-13T18:29:26.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.715+0000 7fc826ff5640 1 -- 172.21.15.114:0/1325560777 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc83c0959c0 con 0x5586db559950 2024-09-13T18:29:26.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.715+0000 7fc826ff5640 1 -- 172.21.15.114:0/1325560777 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc83c020020 con 0x5586db559950 2024-09-13T18:29:26.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.715+0000 7fc84bfca740 10 client.27064 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:29:26.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.715+0000 7fc84bfca740 10 client.27064 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:29:26.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.715+0000 7fc84bfca740 20 client.27064 populate_metadata read hostname 'smithi114' 2024-09-13T18:29:26.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.715+0000 7fc84bfca740 1 --2- 172.21.15.114:0/1325560777 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5586db5c08b0 0x5586db5c2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:26.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.715+0000 7fc84bfca740 4 client.27064 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:29:26.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.715+0000 7fc84bfca740 1 -- 172.21.15.114:0/1325560777 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5586db1a54e0 con 0x5586db5c08b0 2024-09-13T18:29:26.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.717+0000 7fc84259c640 1 --2- 172.21.15.114:0/1325560777 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5586db5c08b0 0x5586db5c2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:26.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.718+0000 7fc84259c640 1 --2- 172.21.15.114:0/1325560777 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5586db5c08b0 0x5586db5c2ca0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1fea516621951d6 server_cookie=103cce90cf483b39 in_seq=0 out_seq=0 2024-09-13T18:29:26.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.718+0000 7fc826ff5640 10 client.27064 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:29:26.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.718+0000 7fc826ff5640 1 -- 172.21.15.114:0/1325560777 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5586db1a54e0 con 0x5586db5c08b0 2024-09-13T18:29:26.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.718+0000 7fc826ff5640 10 client.27064 handle_command_reply: tid=0 2024-09-13T18:29:26.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.718+0000 7fc84bfca740 1 client.27064 shutdown 2024-09-13T18:29:26.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.724+0000 7fc84bfca740 1 -- 172.21.15.114:0/1325560777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586db559950 msgr2=0x5586db560d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:26.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.724+0000 7fc84bfca740 1 --2- 172.21.15.114:0/1325560777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586db559950 0x5586db560d40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc83c095b30 tx=0x7fc83c0046c0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.725+0000 7fc84bfca740 1 -- 172.21.15.114:0/1325560777 shutdown_connections 2024-09-13T18:29:26.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.725+0000 7fc84bfca740 1 --2- 172.21.15.114:0/1325560777 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5586db5c08b0 0x5586db5c2ca0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.725+0000 7fc84bfca740 1 --2- 172.21.15.114:0/1325560777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586db5bc750 0x5586db5beb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.725+0000 7fc84bfca740 1 --2- 172.21.15.114:0/1325560777 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5586db561280 0x5586db561700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.725+0000 7fc84bfca740 1 --2- 172.21.15.114:0/1325560777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586db559950 0x5586db560d40 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.725+0000 7fc84bfca740 1 -- 172.21.15.114:0/1325560777 >> 172.21.15.114:0/1325560777 conn(0x5586db54ad00 msgr2=0x5586db554910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:26.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.725+0000 7fc826ff5640 0 client.27064 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:29:26.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.728+0000 7fc84bfca740 1 -- 172.21.15.114:0/1325560777 shutdown_connections 2024-09-13T18:29:26.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.729+0000 7fc84bfca740 1 -- 172.21.15.114:0/1325560777 wait complete. 2024-09-13T18:29:26.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.730+0000 7fc84bfca740 20 client.27064 trim_cache size 0 max 16384 2024-09-13T18:29:26.733 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:29:26.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.731+0000 7fc8491ce640 1 -- 172.21.15.114:0/3588709575 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc82c0a12e0 msgr2=0x7fc82c0a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:26.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.731+0000 7fc8491ce640 1 --2- 172.21.15.114:0/3588709575 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc82c0a12e0 0x7fc82c0a37a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc8380058c0 tx=0x7fc838005850 comp rx=0 tx=0).stop 2024-09-13T18:29:26.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.731+0000 7fc8491ce640 1 -- 172.21.15.114:0/3588709575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8440fed30 msgr2=0x7fc8441a2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:26.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.731+0000 7fc8491ce640 1 --2- 172.21.15.114:0/3588709575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8440fed30 0x7fc8441a2590 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc83c00b4a0 tx=0x7fc83c00b960 comp rx=0 tx=0).stop 2024-09-13T18:29:26.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.732+0000 7fc8491ce640 1 -- 172.21.15.114:0/3588709575 shutdown_connections 2024-09-13T18:29:26.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.732+0000 7fc8491ce640 1 --2- 172.21.15.114:0/3588709575 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc82c0a12e0 0x7fc82c0a37a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.732+0000 7fc8491ce640 1 --2- 172.21.15.114:0/3588709575 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc844104160 0x7fc84419c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.732+0000 7fc8491ce640 1 --2- 172.21.15.114:0/3588709575 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8440ff650 0x7fc8441a2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.732+0000 7fc8491ce640 1 --2- 172.21.15.114:0/3588709575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8440fed30 0x7fc8441a2590 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:26.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.732+0000 7fc8491ce640 1 -- 172.21.15.114:0/3588709575 >> 172.21.15.114:0/3588709575 conn(0x7fc844075c60 msgr2=0x7fc84410d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:26.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.732+0000 7fc8491ce640 1 -- 172.21.15.114:0/3588709575 shutdown_connections 2024-09-13T18:29:26.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:26.732+0000 7fc8491ce640 1 -- 172.21.15.114:0/3588709575 wait complete. 2024-09-13T18:29:27.774 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-13T18:29:27.775 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-09-13T18:29:28.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.010+0000 7f33db59e640 1 -- 172.21.15.114:0/553603034 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33dc1074b0 msgr2=0x7f33dc10e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:28.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.010+0000 7f33db59e640 1 --2- 172.21.15.114:0/553603034 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33dc1074b0 0x7f33dc10e510 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f33d4009ca0 tx=0x7f33d402a8e0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.014+0000 7f33db59e640 1 -- 172.21.15.114:0/553603034 shutdown_connections 2024-09-13T18:29:28.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.014+0000 7f33db59e640 1 --2- 172.21.15.114:0/553603034 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33dc119400 0x7f33dc11b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.014+0000 7f33db59e640 1 --2- 172.21.15.114:0/553603034 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f33dc107890 0x7f33dc107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.014+0000 7f33db59e640 1 --2- 172.21.15.114:0/553603034 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33dc1074b0 0x7f33dc10e510 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.014+0000 7f33db59e640 1 -- 172.21.15.114:0/553603034 >> 172.21.15.114:0/553603034 conn(0x7f33dc06af40 msgr2=0x7f33dc06b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:28.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.022+0000 7f33db59e640 1 -- 172.21.15.114:0/553603034 shutdown_connections 2024-09-13T18:29:28.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.023+0000 7f33db59e640 1 -- 172.21.15.114:0/553603034 wait complete. 2024-09-13T18:29:28.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.024+0000 7f33db59e640 1 Processor -- start 2024-09-13T18:29:28.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.028+0000 7f33db59e640 1 -- start start 2024-09-13T18:29:28.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.028+0000 7f33db59e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33dc1074b0 0x7f33dc116a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:28.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.028+0000 7f33db59e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f33dc107890 0x7f33dc10fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:28.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.028+0000 7f33db59e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33dc119400 0x7f33dc10fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:28.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.028+0000 7f33db59e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33dc1105f0 con 0x7f33dc107890 2024-09-13T18:29:28.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.028+0000 7f33db59e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33dc110760 con 0x7f33dc119400 2024-09-13T18:29:28.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.028+0000 7f33db59e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33dc110920 con 0x7f33dc1074b0 2024-09-13T18:29:28.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.029+0000 7f33dad9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33dc119400 0x7f33dc10fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:28.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.030+0000 7f33dad9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33dc119400 0x7f33dc10fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:56340/0 (socket says 172.21.15.114:56340) 2024-09-13T18:29:28.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.030+0000 7f33dad9d640 1 -- 172.21.15.114:0/2925962135 learned_addr learned my addr 172.21.15.114:0/2925962135 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:28.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.030+0000 7f33da59c640 1 --2- 172.21.15.114:0/2925962135 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33dc1074b0 0x7f33dc116a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:28.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.031+0000 7f33d9d9b640 1 --2- 172.21.15.114:0/2925962135 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f33dc107890 0x7f33dc10fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:28.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.032+0000 7f33d9d9b640 1 -- 172.21.15.114:0/2925962135 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33dc1074b0 msgr2=0x7f33dc116a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:28.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.032+0000 7f33d9d9b640 1 --2- 172.21.15.114:0/2925962135 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33dc1074b0 0x7f33dc116a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.032+0000 7f33d9d9b640 1 -- 172.21.15.114:0/2925962135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33dc119400 msgr2=0x7f33dc10fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:28.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.032+0000 7f33d9d9b640 1 --2- 172.21.15.114:0/2925962135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33dc119400 0x7f33dc10fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.032+0000 7f33d9d9b640 1 -- 172.21.15.114:0/2925962135 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d4009900 con 0x7f33dc107890 2024-09-13T18:29:28.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.033+0000 7f33d9d9b640 1 --2- 172.21.15.114:0/2925962135 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f33dc107890 0x7f33dc10fab0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f33d000b3f0 tx=0x7f33d000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:28.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.033+0000 7f33cb7fe640 1 -- 172.21.15.114:0/2925962135 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d0011830 con 0x7f33dc107890 2024-09-13T18:29:28.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.034+0000 7f33cb7fe640 1 -- 172.21.15.114:0/2925962135 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f33d0011e20 con 0x7f33dc107890 2024-09-13T18:29:28.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.034+0000 7f33cb7fe640 1 -- 172.21.15.114:0/2925962135 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d0010e30 con 0x7f33dc107890 2024-09-13T18:29:28.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.034+0000 7f33db59e640 1 -- 172.21.15.114:0/2925962135 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33dc114710 con 0x7f33dc107890 2024-09-13T18:29:28.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.034+0000 7f33db59e640 1 -- 172.21.15.114:0/2925962135 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33dc114bd0 con 0x7f33dc107890 2024-09-13T18:29:28.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.040+0000 7f33e35fc740 1 Processor -- start 2024-09-13T18:29:28.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.040+0000 7f33e35fc740 1 -- start start 2024-09-13T18:29:28.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.040+0000 7f33e35fc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1089079a0 0x55c108907d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:28.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.040+0000 7f33e35fc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c1089082c0 0x55c108912620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:28.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.040+0000 7f33e35fc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c108912b60 0x55c108914f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:28.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.040+0000 7f33e35fc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c108872c60 con 0x55c1089082c0 2024-09-13T18:29:28.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.040+0000 7f33e35fc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1088703f0 con 0x55c108912b60 2024-09-13T18:29:28.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.040+0000 7f33e35fc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c10886bce0 con 0x55c1089079a0 2024-09-13T18:29:28.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.042+0000 7f33cb7fe640 1 -- 172.21.15.114:0/2925962135 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f33d001a400 con 0x7f33dc107890 2024-09-13T18:29:28.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.042+0000 7f33dad9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1089079a0 0x55c108907d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:28.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.042+0000 7f33dad9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1089079a0 0x55c108907d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:49334/0 (socket says 172.21.15.114:49334) 2024-09-13T18:29:28.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.042+0000 7f33dad9d640 1 -- 172.21.15.114:0/1538195056 learned_addr learned my addr 172.21.15.114:0/1538195056 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:28.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.042+0000 7f33da59c640 1 --2- 172.21.15.114:0/1538195056 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c1089082c0 0x55c108912620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:28.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.046+0000 7f33dad9d640 1 -- 172.21.15.114:0/1538195056 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c108912b60 msgr2=0x55c108914f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:28.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.046+0000 7f33cb7fe640 1 --2- 172.21.15.114:0/2925962135 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f33c40a1340 0x7f33c40a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:28.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.046+0000 7f33d9d9b640 1 --2- 172.21.15.114:0/1538195056 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c108912b60 0x55c108914f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:28.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.046+0000 7f33dad9d640 1 --2- 172.21.15.114:0/1538195056 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c108912b60 0x55c108914f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.046+0000 7f33dad9d640 1 -- 172.21.15.114:0/1538195056 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c1089082c0 msgr2=0x55c108912620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:28.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.046+0000 7f33dad9d640 1 --2- 172.21.15.114:0/1538195056 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c1089082c0 0x55c108912620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.047+0000 7f33dad9d640 1 -- 172.21.15.114:0/1538195056 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c108827970 con 0x55c1089079a0 2024-09-13T18:29:28.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.047+0000 7f33dad9d640 1 --2- 172.21.15.114:0/2925962135 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f33c40a1340 0x7f33c40a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:28.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.047+0000 7f33dad9d640 1 --2- 172.21.15.114:0/1538195056 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1089079a0 0x55c108907d80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f33cc009a90 tx=0x7f33cc009f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9edecf337071ade4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:28.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.047+0000 7f33d9d9b640 1 --2- 172.21.15.114:0/1538195056 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c108912b60 0x55c108914f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:29:28.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.048+0000 7f33cb7fe640 1 -- 172.21.15.114:0/2925962135 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f33d00ccad0 con 0x7f33dc107890 2024-09-13T18:29:28.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.053+0000 7f33c8ff9640 1 -- 172.21.15.114:0/1538195056 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33cc01f490 con 0x55c1089079a0 2024-09-13T18:29:28.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.053+0000 7f33c8ff9640 1 -- 172.21.15.114:0/1538195056 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f33cc01fb00 con 0x55c1089079a0 2024-09-13T18:29:28.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.053+0000 7f33c8ff9640 1 -- 172.21.15.114:0/1538195056 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33cc013d00 con 0x55c1089079a0 2024-09-13T18:29:28.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.054+0000 7f33e35fc740 1 -- 172.21.15.114:0/1538195056 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1089079a0 msgr2=0x55c108907d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:28.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.054+0000 7f33e35fc740 1 --2- 172.21.15.114:0/1538195056 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1089079a0 0x55c108907d80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f33cc009a90 tx=0x7f33cc009f50 comp rx=0 tx=0).stop 2024-09-13T18:29:28.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.054+0000 7f33dad9d640 1 --2- 172.21.15.114:0/2925962135 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f33c40a1340 0x7f33c40a3800 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f33cc01e000 tx=0x7f33cc01e7a0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:28.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.054+0000 7f33e35fc740 1 -- 172.21.15.114:0/1538195056 shutdown_connections 2024-09-13T18:29:28.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.054+0000 7f33e35fc740 1 --2- 172.21.15.114:0/1538195056 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c108912b60 0x55c108914f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.054+0000 7f33e35fc740 1 --2- 172.21.15.114:0/1538195056 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c1089082c0 0x55c108912620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.054+0000 7f33e35fc740 1 --2- 172.21.15.114:0/1538195056 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1089079a0 0x55c108907d80 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.054+0000 7f33e35fc740 1 -- 172.21.15.114:0/1538195056 >> 172.21.15.114:0/1538195056 conn(0x55c108903f30 msgr2=0x55c108904310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:28.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.055+0000 7f33e35fc740 1 -- 172.21.15.114:0/1538195056 shutdown_connections 2024-09-13T18:29:28.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.055+0000 7f33e35fc740 1 -- 172.21.15.114:0/1538195056 wait complete. 2024-09-13T18:29:28.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.055+0000 7f33e35fc740 1 -- start start 2024-09-13T18:29:28.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.055+0000 7f33e35fc740 1 Processor -- start 2024-09-13T18:29:28.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.056+0000 7f33e35fc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1089079a0 0x55c1089124f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:28.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.056+0000 7f33e35fc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1089082c0 0x55c10890f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:28.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.056+0000 7f33e35fc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c108912b60 0x55c10890fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:28.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.056+0000 7f33e35fc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c108829f80 con 0x55c108912b60 2024-09-13T18:29:28.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.056+0000 7f33e35fc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c108829c20 con 0x55c1089079a0 2024-09-13T18:29:28.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.056+0000 7f33e35fc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1087f6ab0 con 0x55c1089082c0 2024-09-13T18:29:28.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.056+0000 7f33e35fc740 10 client.0 _ll_register_callbacks cb 0x55c108865350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:29:28.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.056+0000 7f33dad9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c108912b60 0x55c10890fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:28.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.056+0000 7f33dad9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c108912b60 0x55c10890fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57756/0 (socket says 172.21.15.114:57756) 2024-09-13T18:29:28.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.056+0000 7f33dad9d640 1 -- 172.21.15.114:0/3480794432 learned_addr learned my addr 172.21.15.114:0/3480794432 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:28.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.057+0000 7f33da59c640 1 --2- 172.21.15.114:0/3480794432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1089079a0 0x55c1089124f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:28.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.057+0000 7f33d9d9b640 1 --2- 172.21.15.114:0/3480794432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1089082c0 0x55c10890f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:28.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.060+0000 7f33da59c640 1 -- 172.21.15.114:0/3480794432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1089082c0 msgr2=0x55c10890f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:28.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.060+0000 7f33da59c640 1 --2- 172.21.15.114:0/3480794432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1089082c0 0x55c10890f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.060+0000 7f33da59c640 1 -- 172.21.15.114:0/3480794432 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c108912b60 msgr2=0x55c10890fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:28.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.060+0000 7f33da59c640 1 --2- 172.21.15.114:0/3480794432 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c108912b60 0x55c10890fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.060+0000 7f33da59c640 1 -- 172.21.15.114:0/3480794432 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33cc0096f0 con 0x55c1089079a0 2024-09-13T18:29:28.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.060+0000 7f33dad9d640 1 --2- 172.21.15.114:0/3480794432 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c108912b60 0x55c10890fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:29:28.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.061+0000 7f33da59c640 1 --2- 172.21.15.114:0/3480794432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1089079a0 0x55c1089124f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f33d402aed0 tx=0x7f33d4003d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f13faa1b509c78c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:28.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.063+0000 7f33c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:29:28.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.066+0000 7f33c1ffb640 1 -- 172.21.15.114:0/3480794432 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d4041070 con 0x55c1089079a0 2024-09-13T18:29:28.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.066+0000 7f33e35fc740 1 -- 172.21.15.114:0/3480794432 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c10855f590 con 0x55c1089079a0 2024-09-13T18:29:28.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.066+0000 7f33c1ffb640 1 -- 172.21.15.114:0/3480794432 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f33d4030bf0 con 0x55c1089079a0 2024-09-13T18:29:28.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.069+0000 7f33c1ffb640 1 -- 172.21.15.114:0/3480794432 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33d4030dd0 con 0x55c1089079a0 2024-09-13T18:29:28.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.078+0000 7f33e35fc740 10 client.27062 fetch_fsmap learned FSMap version 34 2024-09-13T18:29:28.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.078+0000 7f33e35fc740 1 -- 172.21.15.114:0/3480794432 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55c10886a360 con 0x55c1089079a0 2024-09-13T18:29:28.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.082+0000 7f33c1ffb640 1 -- 172.21.15.114:0/3480794432 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c10855f590 con 0x55c1089079a0 2024-09-13T18:29:28.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.082+0000 7f33c1ffb640 1 -- 172.21.15.114:0/3480794432 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f33d402f590 con 0x55c1089079a0 2024-09-13T18:29:28.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.083+0000 7f33e35fc740 10 client.27062 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:29:28.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.083+0000 7f33e35fc740 10 client.27062 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:29:28.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.083+0000 7f33e35fc740 20 client.27062 populate_metadata read hostname 'smithi114' 2024-09-13T18:29:28.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.083+0000 7f33e35fc740 1 --2- 172.21.15.114:0/3480794432 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55c1089448d0 0x55c108946cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:28.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.083+0000 7f33e35fc740 4 client.27062 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:29:28.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.083+0000 7f33e35fc740 1 -- 172.21.15.114:0/3480794432 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c108570860 con 0x55c1089448d0 2024-09-13T18:29:28.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.084+0000 7f33d9d9b640 1 --2- 172.21.15.114:0/3480794432 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55c1089448d0 0x55c108946cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:28.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.085+0000 7f33d9d9b640 1 --2- 172.21.15.114:0/3480794432 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55c1089448d0 0x55c108946cc0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f125bfc60633b4 server_cookie=649c431c0b84698a in_seq=0 out_seq=0 2024-09-13T18:29:28.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.088+0000 7f33c1ffb640 10 client.27062 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:29:28.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.088+0000 7f33c1ffb640 1 -- 172.21.15.114:0/3480794432 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55c108570860 con 0x55c1089448d0 2024-09-13T18:29:28.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.088+0000 7f33c1ffb640 10 client.27062 handle_command_reply: tid=0 2024-09-13T18:29:28.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.088+0000 7f33e35fc740 1 client.27062 shutdown 2024-09-13T18:29:28.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.094+0000 7f33e35fc740 1 -- 172.21.15.114:0/3480794432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1089079a0 msgr2=0x55c1089124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:28.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.094+0000 7f33e35fc740 1 --2- 172.21.15.114:0/3480794432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1089079a0 0x55c1089124f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f33d402aed0 tx=0x7f33d4003d70 comp rx=0 tx=0).stop 2024-09-13T18:29:28.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.094+0000 7f33e35fc740 1 -- 172.21.15.114:0/3480794432 shutdown_connections 2024-09-13T18:29:28.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.094+0000 7f33e35fc740 1 --2- 172.21.15.114:0/3480794432 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55c1089448d0 0x55c108946cc0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.094+0000 7f33e35fc740 1 --2- 172.21.15.114:0/3480794432 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c108912b60 0x55c10890fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.094+0000 7f33e35fc740 1 --2- 172.21.15.114:0/3480794432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1089082c0 0x55c10890f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.094+0000 7f33e35fc740 1 --2- 172.21.15.114:0/3480794432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1089079a0 0x55c1089124f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.094+0000 7f33c1ffb640 0 client.27062 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:29:28.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.094+0000 7f33e35fc740 1 -- 172.21.15.114:0/3480794432 >> 172.21.15.114:0/3480794432 conn(0x55c108903f30 msgr2=0x55c108903160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:28.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.096+0000 7f33e35fc740 1 -- 172.21.15.114:0/3480794432 shutdown_connections 2024-09-13T18:29:28.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.096+0000 7f33e35fc740 1 -- 172.21.15.114:0/3480794432 wait complete. 2024-09-13T18:29:28.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.097+0000 7f33e35fc740 20 client.27062 trim_cache size 0 max 16384 2024-09-13T18:29:28.118 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:29:27 smithi134 ceph-mon[34283]: pgmap v214: 129 pgs: 129 active+clean; 21 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 40 MiB/s wr, 891 op/s 2024-09-13T18:29:28.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.133+0000 7f33e35fc740 1 Processor -- start 2024-09-13T18:29:28.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.134+0000 7f33e35fc740 1 -- start start 2024-09-13T18:29:28.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.134+0000 7f33e35fc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1089079a0 0x55c108912000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:28.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.134+0000 7f33e35fc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c1089082c0 0x55c10890ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:28.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.134+0000 7f33e35fc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c108912b60 0x55c10890f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:28.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.134+0000 7f33e35fc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c10891e790 con 0x55c1089082c0 2024-09-13T18:29:28.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.134+0000 7f33e35fc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c108915610 con 0x55c1089079a0 2024-09-13T18:29:28.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.134+0000 7f33e35fc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c10890dc70 con 0x55c108912b60 2024-09-13T18:29:28.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.134+0000 7f33da59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1089079a0 0x55c108912000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:28.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.134+0000 7f33da59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1089079a0 0x55c108912000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:56372/0 (socket says 172.21.15.114:56372) 2024-09-13T18:29:28.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.134+0000 7f33da59c640 1 -- 172.21.15.114:0/469813191 learned_addr learned my addr 172.21.15.114:0/469813191 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:28.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.137+0000 7f33dad9d640 1 --2- 172.21.15.114:0/469813191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c108912b60 0x55c10890f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:28.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.137+0000 7f33d9d9b640 1 --2- 172.21.15.114:0/469813191 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c1089082c0 0x55c10890ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:28.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.137+0000 7f33dad9d640 1 -- 172.21.15.114:0/469813191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1089079a0 msgr2=0x55c108912000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:28.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.137+0000 7f33dad9d640 1 --2- 172.21.15.114:0/469813191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1089079a0 0x55c108912000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.137+0000 7f33dad9d640 1 -- 172.21.15.114:0/469813191 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c1089082c0 msgr2=0x55c10890ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:28.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.137+0000 7f33dad9d640 1 --2- 172.21.15.114:0/469813191 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c1089082c0 0x55c10890ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.137+0000 7f33dad9d640 1 -- 172.21.15.114:0/469813191 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c10880c1c0 con 0x55c108912b60 2024-09-13T18:29:28.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.140+0000 7f33dad9d640 1 --2- 172.21.15.114:0/469813191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c108912b60 0x55c10890f4d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f33cc006a50 tx=0x7f33cc025000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1fb07a4e77d6f0b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:28.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.142+0000 7f33c97fa640 1 -- 172.21.15.114:0/469813191 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33cc013670 con 0x55c108912b60 2024-09-13T18:29:28.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.143+0000 7f33c97fa640 1 -- 172.21.15.114:0/469813191 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f33cc01eee0 con 0x55c108912b60 2024-09-13T18:29:28.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.144+0000 7f33e35fc740 1 -- 172.21.15.114:0/469813191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c108912b60 msgr2=0x55c10890f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:28.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.144+0000 7f33e35fc740 1 --2- 172.21.15.114:0/469813191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c108912b60 0x55c10890f4d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f33cc006a50 tx=0x7f33cc025000 comp rx=0 tx=0).stop 2024-09-13T18:29:28.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.149+0000 7f33e35fc740 1 -- 172.21.15.114:0/469813191 shutdown_connections 2024-09-13T18:29:28.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.149+0000 7f33e35fc740 1 --2- 172.21.15.114:0/469813191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c108912b60 0x55c10890f4d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.149+0000 7f33e35fc740 1 --2- 172.21.15.114:0/469813191 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c1089082c0 0x55c10890ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.149+0000 7f33e35fc740 1 --2- 172.21.15.114:0/469813191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1089079a0 0x55c108912000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.149+0000 7f33e35fc740 1 -- 172.21.15.114:0/469813191 >> 172.21.15.114:0/469813191 conn(0x55c108903f30 msgr2=0x55c10890dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:28.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.155+0000 7f33e35fc740 1 -- 172.21.15.114:0/469813191 shutdown_connections 2024-09-13T18:29:28.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.159+0000 7f33e35fc740 1 -- 172.21.15.114:0/469813191 wait complete. 2024-09-13T18:29:28.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.159+0000 7f33e35fc740 1 -- start start 2024-09-13T18:29:28.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.159+0000 7f33e35fc740 1 Processor -- start 2024-09-13T18:29:28.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.160+0000 7f33e35fc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c1089079a0 0x55c10890c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:28.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.160+0000 7f33e35fc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1089082c0 0x55c10890ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:28.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.160+0000 7f33e35fc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c108912b60 0x55c10899ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:28.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.160+0000 7f33e35fc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c10890d5c0 con 0x55c1089079a0 2024-09-13T18:29:28.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.160+0000 7f33e35fc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c10899b280 con 0x55c1089082c0 2024-09-13T18:29:28.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.160+0000 7f33e35fc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c10899b3c0 con 0x55c108912b60 2024-09-13T18:29:28.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.160+0000 7f33e35fc740 10 client.0 _ll_register_callbacks cb 0x7f33d4009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:29:28.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.160+0000 7f33da59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c1089079a0 0x55c10890c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:28.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.160+0000 7f33da59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c1089079a0 0x55c10890c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57778/0 (socket says 172.21.15.114:57778) 2024-09-13T18:29:28.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.160+0000 7f33da59c640 1 -- 172.21.15.114:0/2101435536 learned_addr learned my addr 172.21.15.114:0/2101435536 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:28.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.160+0000 7f33dad9d640 1 --2- 172.21.15.114:0/2101435536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c108912b60 0x55c10899ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:28.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.161+0000 7f33d9d9b640 1 --2- 172.21.15.114:0/2101435536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1089082c0 0x55c10890ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:28.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.161+0000 7f33da59c640 1 -- 172.21.15.114:0/2101435536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c108912b60 msgr2=0x55c10899ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:28.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.161+0000 7f33da59c640 1 --2- 172.21.15.114:0/2101435536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c108912b60 0x55c10899ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.161+0000 7f33da59c640 1 -- 172.21.15.114:0/2101435536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1089082c0 msgr2=0x55c10890ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:28.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.161+0000 7f33da59c640 1 --2- 172.21.15.114:0/2101435536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1089082c0 0x55c10890ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.161+0000 7f33da59c640 1 -- 172.21.15.114:0/2101435536 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33cc023030 con 0x55c1089079a0 2024-09-13T18:29:28.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.162+0000 7f33da59c640 1 --2- 172.21.15.114:0/2101435536 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c1089079a0 0x55c10890c7b0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f33d4009ca0 tx=0x7f33d4009870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfdc6c9f414fd699 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:28.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.162+0000 7f33c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:29:28.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.162+0000 7f33c1ffb640 1 -- 172.21.15.114:0/2101435536 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d4041040 con 0x55c1089079a0 2024-09-13T18:29:28.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.162+0000 7f33e35fc740 1 -- 172.21.15.114:0/2101435536 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f33cc01fb00 con 0x55c1089079a0 2024-09-13T18:29:28.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.162+0000 7f33c1ffb640 1 -- 172.21.15.114:0/2101435536 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f33d402fc10 con 0x55c1089079a0 2024-09-13T18:29:28.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.163+0000 7f33c1ffb640 1 -- 172.21.15.114:0/2101435536 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33d40039e0 con 0x55c1089079a0 2024-09-13T18:29:28.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.166+0000 7f33e35fc740 10 client.17658 fetch_fsmap learned FSMap version 34 2024-09-13T18:29:28.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.166+0000 7f33e35fc740 1 -- 172.21.15.114:0/2101435536 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55c108831990 con 0x55c1089079a0 2024-09-13T18:29:28.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.167+0000 7f33c1ffb640 1 -- 172.21.15.114:0/2101435536 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f33cc01fb00 con 0x55c1089079a0 2024-09-13T18:29:28.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.168+0000 7f33c1ffb640 1 -- 172.21.15.114:0/2101435536 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f33d4004470 con 0x55c1089079a0 2024-09-13T18:29:28.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.169+0000 7f33e35fc740 10 client.17658 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:29:28.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.169+0000 7f33e35fc740 10 client.17658 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:29:28.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.169+0000 7f33e35fc740 20 client.17658 populate_metadata read hostname 'smithi114' 2024-09-13T18:29:28.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.169+0000 7f33e35fc740 1 --2- 172.21.15.114:0/2101435536 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55c10899c7a0 0x55c10899ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:28.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.169+0000 7f33e35fc740 4 client.17658 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-13T18:29:28.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.169+0000 7f33e35fc740 1 -- 172.21.15.114:0/2101435536 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55c1085c3eb0 con 0x55c10899c7a0 2024-09-13T18:29:28.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.173+0000 7f33d9d9b640 1 --2- 172.21.15.114:0/2101435536 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55c10899c7a0 0x55c10899ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:28.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.175+0000 7f33d9d9b640 1 --2- 172.21.15.114:0/2101435536 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55c10899c7a0 0x55c10899ebc0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7689203d6f248c1c server_cookie=fc60746ecf7a4a19 in_seq=0 out_seq=0 2024-09-13T18:29:28.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.175+0000 7f33c1ffb640 10 client.17658 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:29:28.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.176+0000 7f33c1ffb640 1 -- 172.21.15.114:0/2101435536 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c1085c3eb0 con 0x55c10899c7a0 2024-09-13T18:29:28.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.176+0000 7f33c1ffb640 10 client.17658 handle_command_reply: tid=0 2024-09-13T18:29:28.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.176+0000 7f33e35fc740 1 client.17658 shutdown 2024-09-13T18:29:28.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.179+0000 7f33e35fc740 1 -- 172.21.15.114:0/2101435536 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c1089079a0 msgr2=0x55c10890c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:28.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.179+0000 7f33e35fc740 1 --2- 172.21.15.114:0/2101435536 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c1089079a0 0x55c10890c7b0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f33d4009ca0 tx=0x7f33d4009870 comp rx=0 tx=0).stop 2024-09-13T18:29:28.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.179+0000 7f33e35fc740 1 -- 172.21.15.114:0/2101435536 shutdown_connections 2024-09-13T18:29:28.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.179+0000 7f33e35fc740 1 --2- 172.21.15.114:0/2101435536 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55c10899c7a0 0x55c10899ebc0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.179+0000 7f33e35fc740 1 --2- 172.21.15.114:0/2101435536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c108912b60 0x55c10899ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.179+0000 7f33e35fc740 1 --2- 172.21.15.114:0/2101435536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1089082c0 0x55c10890ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.179+0000 7f33e35fc740 1 --2- 172.21.15.114:0/2101435536 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c1089079a0 0x55c10890c7b0 secure :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f33d4009ca0 tx=0x7f33d4009870 comp rx=0 tx=0).stop 2024-09-13T18:29:28.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.179+0000 7f33e35fc740 1 -- 172.21.15.114:0/2101435536 >> 172.21.15.114:0/2101435536 conn(0x55c108903f30 msgr2=0x55c10890df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:28.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.179+0000 7f33c1ffb640 0 client.17658 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:29:28.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.180+0000 7f33e35fc740 1 -- 172.21.15.114:0/2101435536 shutdown_connections 2024-09-13T18:29:28.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.180+0000 7f33e35fc740 1 -- 172.21.15.114:0/2101435536 wait complete. 2024-09-13T18:29:28.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.181+0000 7f33e35fc740 20 client.17658 trim_cache size 0 max 16384 2024-09-13T18:29:28.184 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:29:28.184 INFO:teuthology.orchestra.run.smithi114.stdout: "return_code": 0, 2024-09-13T18:29:28.184 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_tag": "f3652753-e416-46cd-b581-b4cfb8add1bf", 2024-09-13T18:29:28.184 INFO:teuthology.orchestra.run.smithi114.stdout: "mode": "asynchronous" 2024-09-13T18:29:28.184 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:29:28.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.191+0000 7f33db59e640 1 -- 172.21.15.114:0/2925962135 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f33c40a1340 msgr2=0x7f33c40a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:28.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.192+0000 7f33db59e640 1 --2- 172.21.15.114:0/2925962135 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f33c40a1340 0x7f33c40a3800 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f33cc01e000 tx=0x7f33cc01e7a0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.192+0000 7f33db59e640 1 -- 172.21.15.114:0/2925962135 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f33dc107890 msgr2=0x7f33dc10fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:28.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.192+0000 7f33db59e640 1 --2- 172.21.15.114:0/2925962135 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f33dc107890 0x7f33dc10fab0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f33d000b3f0 tx=0x7f33d000b8b0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.192+0000 7f33db59e640 1 -- 172.21.15.114:0/2925962135 shutdown_connections 2024-09-13T18:29:28.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.192+0000 7f33db59e640 1 --2- 172.21.15.114:0/2925962135 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f33c40a1340 0x7f33c40a3800 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.192+0000 7f33db59e640 1 --2- 172.21.15.114:0/2925962135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33dc119400 0x7f33dc10fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.192+0000 7f33db59e640 1 --2- 172.21.15.114:0/2925962135 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f33dc107890 0x7f33dc10fab0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.192+0000 7f33db59e640 1 --2- 172.21.15.114:0/2925962135 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33dc1074b0 0x7f33dc116a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.192+0000 7f33db59e640 1 -- 172.21.15.114:0/2925962135 >> 172.21.15.114:0/2925962135 conn(0x7f33dc06af40 msgr2=0x7f33dc119e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:28.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:29:27 smithi204 ceph-mon[34777]: pgmap v214: 129 pgs: 129 active+clean; 21 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 40 MiB/s wr, 891 op/s 2024-09-13T18:29:28.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.193+0000 7f33db59e640 1 -- 172.21.15.114:0/2925962135 shutdown_connections 2024-09-13T18:29:28.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.193+0000 7f33db59e640 1 -- 172.21.15.114:0/2925962135 wait complete. 2024-09-13T18:29:28.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:29:27 smithi114 ceph-mon[30897]: pgmap v214: 129 pgs: 129 active+clean; 21 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 40 MiB/s wr, 891 op/s 2024-09-13T18:29:28.238 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T18:29:28.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.502+0000 7fa2b1bdb640 1 -- 172.21.15.114:0/4225606053 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2ac107890 msgr2=0x7fa2ac107cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:28.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.502+0000 7fa2b1bdb640 1 --2- 172.21.15.114:0/4225606053 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2ac107890 0x7fa2ac107cd0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa29c009900 tx=0x7fa29c02a410 comp rx=0 tx=0).stop 2024-09-13T18:29:28.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.503+0000 7fa2b1bdb640 1 -- 172.21.15.114:0/4225606053 shutdown_connections 2024-09-13T18:29:28.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.503+0000 7fa2b1bdb640 1 --2- 172.21.15.114:0/4225606053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2ac119400 0x7fa2ac11b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.503+0000 7fa2b1bdb640 1 --2- 172.21.15.114:0/4225606053 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2ac107890 0x7fa2ac107cd0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.503+0000 7fa2b1bdb640 1 --2- 172.21.15.114:0/4225606053 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2ac1074b0 0x7fa2ac10e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.503+0000 7fa2b1bdb640 1 -- 172.21.15.114:0/4225606053 >> 172.21.15.114:0/4225606053 conn(0x7fa2ac06af40 msgr2=0x7fa2ac06b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:28.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.504+0000 7fa2b1bdb640 1 -- 172.21.15.114:0/4225606053 shutdown_connections 2024-09-13T18:29:28.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.504+0000 7fa2b1bdb640 1 -- 172.21.15.114:0/4225606053 wait complete. 2024-09-13T18:29:28.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.504+0000 7fa2b1bdb640 1 Processor -- start 2024-09-13T18:29:28.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.505+0000 7fa2b1bdb640 1 -- start start 2024-09-13T18:29:28.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.505+0000 7fa2b1bdb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2ac1074b0 0x7fa2ac112750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:28.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.505+0000 7fa2b1bdb640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2ac107890 0x7fa2ac112c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:28.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.505+0000 7fa2b1bdb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2ac119400 0x7fa2ac117080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:28.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.505+0000 7fa2b1bdb640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac1175f0 con 0x7fa2ac1074b0 2024-09-13T18:29:28.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.505+0000 7fa2b1bdb640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac117760 con 0x7fa2ac119400 2024-09-13T18:29:28.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.505+0000 7fa2b1bdb640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac117920 con 0x7fa2ac107890 2024-09-13T18:29:28.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.505+0000 7fa2b0bd9640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2ac1074b0 0x7fa2ac112750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:28.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.506+0000 7fa2b0bd9640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2ac1074b0 0x7fa2ac112750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57800/0 (socket says 172.21.15.114:57800) 2024-09-13T18:29:28.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.506+0000 7fa2b0bd9640 1 -- 172.21.15.114:0/2413613499 learned_addr learned my addr 172.21.15.114:0/2413613499 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:28.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.506+0000 7fa2abfff640 1 --2- 172.21.15.114:0/2413613499 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2ac107890 0x7fa2ac112c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:28.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.506+0000 7fa2b13da640 1 --2- 172.21.15.114:0/2413613499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2ac119400 0x7fa2ac117080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:28.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.506+0000 7fa2b0bd9640 1 -- 172.21.15.114:0/2413613499 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2ac107890 msgr2=0x7fa2ac112c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:28.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.506+0000 7fa2b0bd9640 1 --2- 172.21.15.114:0/2413613499 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2ac107890 0x7fa2ac112c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.506+0000 7fa2b0bd9640 1 -- 172.21.15.114:0/2413613499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2ac119400 msgr2=0x7fa2ac117080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:28.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.506+0000 7fa2b0bd9640 1 --2- 172.21.15.114:0/2413613499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2ac119400 0x7fa2ac117080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.506+0000 7fa2b0bd9640 1 -- 172.21.15.114:0/2413613499 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa29c009560 con 0x7fa2ac1074b0 2024-09-13T18:29:28.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.507+0000 7fa2b0bd9640 1 --2- 172.21.15.114:0/2413613499 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2ac1074b0 0x7fa2ac112750 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fa29400c920 tx=0x7fa29400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:28.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.508+0000 7fa2a9ffb640 1 -- 172.21.15.114:0/2413613499 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa294007c70 con 0x7fa2ac1074b0 2024-09-13T18:29:28.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.508+0000 7fa2b1bdb640 1 -- 172.21.15.114:0/2413613499 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2ac117c00 con 0x7fa2ac1074b0 2024-09-13T18:29:28.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.508+0000 7fa2a9ffb640 1 -- 172.21.15.114:0/2413613499 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa294007dd0 con 0x7fa2ac1074b0 2024-09-13T18:29:28.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.508+0000 7fa2b1bdb640 1 -- 172.21.15.114:0/2413613499 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2ac1ba560 con 0x7fa2ac1074b0 2024-09-13T18:29:28.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.508+0000 7fa2a9ffb640 1 -- 172.21.15.114:0/2413613499 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa294007c70 con 0x7fa2ac1074b0 2024-09-13T18:29:28.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.513+0000 7fa2b5c35740 1 Processor -- start 2024-09-13T18:29:28.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.515+0000 7fa2b5c35740 1 -- start start 2024-09-13T18:29:28.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.515+0000 7fa2b5c35740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55973d030fe0 0x55973d0313c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:28.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.515+0000 7fa2b5c35740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55973d031900 0x55973d03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:28.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.515+0000 7fa2b5c35740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55973d03c1a0 0x55973d03e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:28.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.515+0000 7fa2b5c35740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55973cf9bf20 con 0x55973d03c1a0 2024-09-13T18:29:28.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.515+0000 7fa2b5c35740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55973cf996b0 con 0x55973d030fe0 2024-09-13T18:29:28.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.515+0000 7fa2b5c35740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55973cf94fa0 con 0x55973d031900 2024-09-13T18:29:28.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.515+0000 7fa2a9ffb640 1 -- 172.21.15.114:0/2413613499 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fa2940050c0 con 0x7fa2ac1074b0 2024-09-13T18:29:28.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.516+0000 7fa2b13da640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55973d030fe0 0x55973d0313c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:28.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.516+0000 7fa2b13da640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55973d030fe0 0x55973d0313c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:56386/0 (socket says 172.21.15.114:56386) 2024-09-13T18:29:28.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.516+0000 7fa2b13da640 1 -- 172.21.15.114:0/420207876 learned_addr learned my addr 172.21.15.114:0/420207876 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:28.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.517+0000 7fa2a9ffb640 1 --2- 172.21.15.114:0/2413613499 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa2980a1260 0x7fa2980a3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:28.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.517+0000 7fa2a9ffb640 1 -- 172.21.15.114:0/2413613499 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fa29409a5d0 con 0x7fa2ac1074b0 2024-09-13T18:29:28.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.517+0000 7fa2b13da640 1 -- 172.21.15.114:0/420207876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55973d031900 msgr2=0x55973d03bc60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:29:28.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.517+0000 7fa2b13da640 1 --2- 172.21.15.114:0/420207876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55973d031900 0x55973d03bc60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.517+0000 7fa2b13da640 1 -- 172.21.15.114:0/420207876 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55973d03c1a0 msgr2=0x55973d03e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:28.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.517+0000 7fa2b13da640 1 --2- 172.21.15.114:0/420207876 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55973d03c1a0 0x55973d03e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.517+0000 7fa2b13da640 1 -- 172.21.15.114:0/420207876 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55973cf4f840 con 0x55973d030fe0 2024-09-13T18:29:28.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.517+0000 7fa2b13da640 1 --2- 172.21.15.114:0/420207876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55973d030fe0 0x55973d0313c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400a4d0 tx=0x7fa2a4009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea307f1903124a95 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:28.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.518+0000 7fa2a2ffd640 1 -- 172.21.15.114:0/420207876 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a4010040 con 0x55973d030fe0 2024-09-13T18:29:28.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.518+0000 7fa2a2ffd640 1 -- 172.21.15.114:0/420207876 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa2a4012b60 con 0x55973d030fe0 2024-09-13T18:29:28.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.518+0000 7fa2a2ffd640 1 -- 172.21.15.114:0/420207876 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a4014bc0 con 0x55973d030fe0 2024-09-13T18:29:28.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.519+0000 7fa2b5c35740 1 -- 172.21.15.114:0/420207876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55973d030fe0 msgr2=0x55973d0313c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:28.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.519+0000 7fa2b5c35740 1 --2- 172.21.15.114:0/420207876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55973d030fe0 0x55973d0313c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400a4d0 tx=0x7fa2a4009650 comp rx=0 tx=0).stop 2024-09-13T18:29:28.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.519+0000 7fa2b13da640 1 --2- 172.21.15.114:0/2413613499 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa2980a1260 0x7fa2980a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:28.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.524+0000 7fa2b13da640 1 --2- 172.21.15.114:0/2413613499 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa2980a1260 0x7fa2980a3720 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400a4d0 tx=0x7fa2a4009650 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:28.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.524+0000 7fa2b5c35740 1 -- 172.21.15.114:0/420207876 shutdown_connections 2024-09-13T18:29:28.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.524+0000 7fa2b5c35740 1 --2- 172.21.15.114:0/420207876 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55973d03c1a0 0x55973d03e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.524+0000 7fa2b5c35740 1 --2- 172.21.15.114:0/420207876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55973d031900 0x55973d03bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.524+0000 7fa2b5c35740 1 --2- 172.21.15.114:0/420207876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55973d030fe0 0x55973d0313c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.524+0000 7fa2b5c35740 1 -- 172.21.15.114:0/420207876 >> 172.21.15.114:0/420207876 conn(0x55973d02d570 msgr2=0x55973d02d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:28.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.531+0000 7fa2b5c35740 1 -- 172.21.15.114:0/420207876 shutdown_connections 2024-09-13T18:29:28.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.533+0000 7fa2b5c35740 1 -- 172.21.15.114:0/420207876 wait complete. 2024-09-13T18:29:28.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.533+0000 7fa2b5c35740 1 -- start start 2024-09-13T18:29:28.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.534+0000 7fa2b5c35740 1 Processor -- start 2024-09-13T18:29:28.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.536+0000 7fa2b5c35740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55973d030fe0 0x55973d03bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:28.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.536+0000 7fa2b5c35740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55973d031900 0x55973d038de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:28.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.536+0000 7fa2b5c35740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55973d03c1a0 0x55973d039440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:28.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.536+0000 7fa2b5c35740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55973cf362a0 con 0x55973d03c1a0 2024-09-13T18:29:28.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.536+0000 7fa2b5c35740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55973cc6d430 con 0x55973d031900 2024-09-13T18:29:28.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.536+0000 7fa2b5c35740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55973cf18680 con 0x55973d030fe0 2024-09-13T18:29:28.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.536+0000 7fa2b5c35740 10 client.0 _ll_register_callbacks cb 0x55973cf8e610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:29:28.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.537+0000 7fa2b0bd9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55973d030fe0 0x55973d03bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:28.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.537+0000 7fa2b0bd9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55973d030fe0 0x55973d03bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:49388/0 (socket says 172.21.15.114:49388) 2024-09-13T18:29:28.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.537+0000 7fa2b0bd9640 1 -- 172.21.15.114:0/2099846775 learned_addr learned my addr 172.21.15.114:0/2099846775 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:28.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.538+0000 7fa2b0bd9640 1 -- 172.21.15.114:0/2099846775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55973d031900 msgr2=0x55973d038de0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:29:28.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.538+0000 7fa2b0bd9640 1 --2- 172.21.15.114:0/2099846775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55973d031900 0x55973d038de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.538+0000 7fa2b0bd9640 1 -- 172.21.15.114:0/2099846775 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55973d03c1a0 msgr2=0x55973d039440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:28.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.538+0000 7fa2b0bd9640 1 --2- 172.21.15.114:0/2099846775 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55973d03c1a0 0x55973d039440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.538+0000 7fa2b0bd9640 1 -- 172.21.15.114:0/2099846775 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2a4009d00 con 0x55973d030fe0 2024-09-13T18:29:28.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.538+0000 7fa2b0bd9640 1 --2- 172.21.15.114:0/2099846775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55973d030fe0 0x55973d03bb60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa2940973c0 tx=0x7fa2940a1e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ccdfd64dc37a60c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:28.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.539+0000 7fa293fff640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:29:28.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.539+0000 7fa293fff640 1 -- 172.21.15.114:0/2099846775 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2940a20c0 con 0x55973d030fe0 2024-09-13T18:29:28.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.539+0000 7fa2b5c35740 1 -- 172.21.15.114:0/2099846775 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55973cc884e0 con 0x55973d030fe0 2024-09-13T18:29:28.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.539+0000 7fa293fff640 1 -- 172.21.15.114:0/2099846775 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fa29409a5d0 con 0x55973d030fe0 2024-09-13T18:29:28.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.539+0000 7fa293fff640 1 -- 172.21.15.114:0/2099846775 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2940ca9c0 con 0x55973d030fe0 2024-09-13T18:29:28.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.544+0000 7fa2b5c35740 10 client.27088 fetch_fsmap learned FSMap version 34 2024-09-13T18:29:28.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.544+0000 7fa2b5c35740 1 -- 172.21.15.114:0/2099846775 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55973cf93620 con 0x55973d030fe0 2024-09-13T18:29:28.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.545+0000 7fa293fff640 1 -- 172.21.15.114:0/2099846775 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa2940b8030 con 0x55973d030fe0 2024-09-13T18:29:28.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.546+0000 7fa293fff640 1 -- 172.21.15.114:0/2099846775 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa294016020 con 0x55973d030fe0 2024-09-13T18:29:28.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.548+0000 7fa2b5c35740 10 client.27088 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:29:28.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.548+0000 7fa2b5c35740 10 client.27088 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:29:28.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.548+0000 7fa2b5c35740 20 client.27088 populate_metadata read hostname 'smithi114' 2024-09-13T18:29:28.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.548+0000 7fa2b5c35740 1 --2- 172.21.15.114:0/2099846775 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55973d04bdf0 0x55973d04e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:28.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.548+0000 7fa2b5c35740 4 client.27088 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:29:28.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.548+0000 7fa2b5c35740 1 -- 172.21.15.114:0/2099846775 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55973cc997e0 con 0x55973d04bdf0 2024-09-13T18:29:28.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.552+0000 7fa2abfff640 1 --2- 172.21.15.114:0/2099846775 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55973d04bdf0 0x55973d04e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:28.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.558+0000 7fa2abfff640 1 --2- 172.21.15.114:0/2099846775 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55973d04bdf0 0x55973d04e1e0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3acbb4803c40ae41 server_cookie=9efc2f420116ddb3 in_seq=0 out_seq=0 2024-09-13T18:29:28.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.560+0000 7fa293fff640 10 client.27088 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:29:28.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.565+0000 7fa293fff640 1 -- 172.21.15.114:0/2099846775 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55973cc997e0 con 0x55973d04bdf0 2024-09-13T18:29:28.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.565+0000 7fa293fff640 10 client.27088 handle_command_reply: tid=0 2024-09-13T18:29:28.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.565+0000 7fa2b5c35740 1 client.27088 shutdown 2024-09-13T18:29:28.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.571+0000 7fa2b5c35740 1 -- 172.21.15.114:0/2099846775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55973d030fe0 msgr2=0x55973d03bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:28.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.571+0000 7fa2b5c35740 1 --2- 172.21.15.114:0/2099846775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55973d030fe0 0x55973d03bb60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa2940973c0 tx=0x7fa2940a1e50 comp rx=0 tx=0).stop 2024-09-13T18:29:28.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.572+0000 7fa2b5c35740 1 -- 172.21.15.114:0/2099846775 shutdown_connections 2024-09-13T18:29:28.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.572+0000 7fa2b5c35740 1 --2- 172.21.15.114:0/2099846775 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55973d04bdf0 0x55973d04e1e0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.572+0000 7fa2b5c35740 1 --2- 172.21.15.114:0/2099846775 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55973d03c1a0 0x55973d039440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.572+0000 7fa2b5c35740 1 --2- 172.21.15.114:0/2099846775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55973d031900 0x55973d038de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.572+0000 7fa2b5c35740 1 --2- 172.21.15.114:0/2099846775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55973d030fe0 0x55973d03bb60 secure :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa2940973c0 tx=0x7fa2940a1e50 comp rx=0 tx=0).stop 2024-09-13T18:29:28.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.572+0000 7fa2b5c35740 1 -- 172.21.15.114:0/2099846775 >> 172.21.15.114:0/2099846775 conn(0x55973d02d570 msgr2=0x55973d0331e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:28.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.575+0000 7fa293fff640 0 client.27088 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:29:28.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.578+0000 7fa2b5c35740 1 -- 172.21.15.114:0/2099846775 shutdown_connections 2024-09-13T18:29:28.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.580+0000 7fa2b5c35740 1 -- 172.21.15.114:0/2099846775 wait complete. 2024-09-13T18:29:28.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.583+0000 7fa2b5c35740 20 client.27088 trim_cache size 0 max 16384 2024-09-13T18:29:28.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.626+0000 7fa2b5c35740 1 Processor -- start 2024-09-13T18:29:28.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.629+0000 7fa2b5c35740 1 -- start start 2024-09-13T18:29:28.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.630+0000 7fa2b5c35740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55973d031900 0x55973d02d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:28.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.630+0000 7fa2b5c35740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55973d03c1a0 0x55973d052ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:28.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.630+0000 7fa2b5c35740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55973d0339d0 0x55973d0340c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:28.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.630+0000 7fa2abfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55973d03c1a0 0x55973d052ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:28.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.630+0000 7fa2abfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55973d03c1a0 0x55973d052ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:49396/0 (socket says 172.21.15.114:49396) 2024-09-13T18:29:28.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.630+0000 7fa2abfff640 1 -- 172.21.15.114:0/4247573439 learned_addr learned my addr 172.21.15.114:0/4247573439 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:28.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.630+0000 7fa2b13da640 1 --2- 172.21.15.114:0/4247573439 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55973d0339d0 0x55973d0340c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:28.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.631+0000 7fa2b5c35740 1 -- 172.21.15.114:0/4247573439 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55973d0322c0 con 0x55973d0339d0 2024-09-13T18:29:28.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.631+0000 7fa2b5c35740 1 -- 172.21.15.114:0/4247573439 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55973d030d80 con 0x55973d031900 2024-09-13T18:29:28.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.631+0000 7fa2b5c35740 1 -- 172.21.15.114:0/4247573439 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55973d039200 con 0x55973d03c1a0 2024-09-13T18:29:28.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.632+0000 7fa2abfff640 1 -- 172.21.15.114:0/4247573439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55973d031900 msgr2=0x55973d02d950 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:29:28.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.632+0000 7fa2abfff640 1 --2- 172.21.15.114:0/4247573439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55973d031900 0x55973d02d950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.632+0000 7fa2abfff640 1 -- 172.21.15.114:0/4247573439 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55973d0339d0 msgr2=0x55973d0340c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:28.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.632+0000 7fa2abfff640 1 --2- 172.21.15.114:0/4247573439 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55973d0339d0 0x55973d0340c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.632+0000 7fa2abfff640 1 -- 172.21.15.114:0/4247573439 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55973cf35790 con 0x55973d03c1a0 2024-09-13T18:29:28.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.633+0000 7fa2abfff640 1 --2- 172.21.15.114:0/4247573439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55973d03c1a0 0x55973d052ae0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa29c0057c0 tx=0x7fa29c005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18c401b2f58771cc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:28.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.634+0000 7fa2a37fe640 1 -- 172.21.15.114:0/4247573439 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa29c043070 con 0x55973d03c1a0 2024-09-13T18:29:28.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.634+0000 7fa2a37fe640 1 -- 172.21.15.114:0/4247573439 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa29c004040 con 0x55973d03c1a0 2024-09-13T18:29:28.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.635+0000 7fa2b5c35740 1 -- 172.21.15.114:0/4247573439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55973d03c1a0 msgr2=0x55973d052ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:28.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.635+0000 7fa2b5c35740 1 --2- 172.21.15.114:0/4247573439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55973d03c1a0 0x55973d052ae0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa29c0057c0 tx=0x7fa29c005fd0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.639+0000 7fa2b5c35740 1 -- 172.21.15.114:0/4247573439 shutdown_connections 2024-09-13T18:29:28.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.639+0000 7fa2b5c35740 1 --2- 172.21.15.114:0/4247573439 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55973d0339d0 0x55973d0340c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.639+0000 7fa2b5c35740 1 --2- 172.21.15.114:0/4247573439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55973d03c1a0 0x55973d052ae0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.639+0000 7fa2b5c35740 1 --2- 172.21.15.114:0/4247573439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55973d031900 0x55973d02d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.639+0000 7fa2b5c35740 1 -- 172.21.15.114:0/4247573439 >> 172.21.15.114:0/4247573439 conn(0x55973d02d570 msgr2=0x55973d04de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:28.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.646+0000 7fa2b5c35740 1 -- 172.21.15.114:0/4247573439 shutdown_connections 2024-09-13T18:29:28.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.651+0000 7fa2b5c35740 1 -- 172.21.15.114:0/4247573439 wait complete. 2024-09-13T18:29:28.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.651+0000 7fa2b5c35740 1 -- start start 2024-09-13T18:29:28.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.651+0000 7fa2b5c35740 1 Processor -- start 2024-09-13T18:29:28.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.652+0000 7fa2b5c35740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55973d031900 0x55973d043400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:28.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.652+0000 7fa2b5c35740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55973d03c1a0 0x55973d03ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:28.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.652+0000 7fa2b5c35740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55973d040560 0x55973d0409c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:28.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.652+0000 7fa2b5c35740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55973d0a23a0 con 0x55973d031900 2024-09-13T18:29:28.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.652+0000 7fa2b5c35740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55973d0a2510 con 0x55973d040560 2024-09-13T18:29:28.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.652+0000 7fa2b5c35740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55973d0a26d0 con 0x55973d03c1a0 2024-09-13T18:29:28.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.652+0000 7fa2b5c35740 10 client.0 _ll_register_callbacks cb 0x7fa2a40094f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:29:28.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.652+0000 7fa2b13da640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55973d040560 0x55973d0409c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:28.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.652+0000 7fa2b13da640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55973d040560 0x55973d0409c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:56398/0 (socket says 172.21.15.114:56398) 2024-09-13T18:29:28.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.652+0000 7fa2b13da640 1 -- 172.21.15.114:0/1263966971 learned_addr learned my addr 172.21.15.114:0/1263966971 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:28.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.652+0000 7fa2abfff640 1 --2- 172.21.15.114:0/1263966971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55973d03c1a0 0x55973d03ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:28.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.653+0000 7fa2b13da640 1 -- 172.21.15.114:0/1263966971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55973d03c1a0 msgr2=0x55973d03ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:28.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.653+0000 7fa2b13da640 1 --2- 172.21.15.114:0/1263966971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55973d03c1a0 0x55973d03ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.653+0000 7fa2b13da640 1 -- 172.21.15.114:0/1263966971 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55973d031900 msgr2=0x55973d043400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:29:28.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.653+0000 7fa2b13da640 1 --2- 172.21.15.114:0/1263966971 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55973d031900 0x55973d043400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.653+0000 7fa2b13da640 1 -- 172.21.15.114:0/1263966971 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2ac11c170 con 0x55973d040560 2024-09-13T18:29:28.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.653+0000 7fa2b13da640 1 --2- 172.21.15.114:0/1263966971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55973d040560 0x55973d0409c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400ae20 tx=0x7fa2a4014e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e85ff9ae4899b5c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:28.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.653+0000 7fa293fff640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:29:28.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.653+0000 7fa293fff640 1 -- 172.21.15.114:0/1263966971 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a4023070 con 0x55973d040560 2024-09-13T18:29:28.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.653+0000 7fa2b5c35740 1 -- 172.21.15.114:0/1263966971 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa2a4012b60 con 0x55973d040560 2024-09-13T18:29:28.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.654+0000 7fa293fff640 1 -- 172.21.15.114:0/1263966971 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fa2a4003c20 con 0x55973d040560 2024-09-13T18:29:28.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.654+0000 7fa293fff640 1 -- 172.21.15.114:0/1263966971 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2a4003de0 con 0x55973d040560 2024-09-13T18:29:28.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.656+0000 7fa2b5c35740 10 client.27077 fetch_fsmap learned FSMap version 34 2024-09-13T18:29:28.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.656+0000 7fa2b5c35740 1 -- 172.21.15.114:0/1263966971 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55973cf5aea0 con 0x55973d040560 2024-09-13T18:29:28.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.660+0000 7fa293fff640 1 -- 172.21.15.114:0/1263966971 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa2a4012b60 con 0x55973d040560 2024-09-13T18:29:28.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.660+0000 7fa293fff640 1 -- 172.21.15.114:0/1263966971 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa2a4008030 con 0x55973d040560 2024-09-13T18:29:28.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.661+0000 7fa2b5c35740 10 client.27077 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:29:28.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.661+0000 7fa2b5c35740 10 client.27077 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:29:28.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.661+0000 7fa2b5c35740 20 client.27077 populate_metadata read hostname 'smithi114' 2024-09-13T18:29:28.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.661+0000 7fa2b5c35740 1 --2- 172.21.15.114:0/1263966971 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55973d0a3e20 0x55973d0a6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:28.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.661+0000 7fa2b5c35740 4 client.27077 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T18:29:28.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.661+0000 7fa2b5c35740 1 -- 172.21.15.114:0/1263966971 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55973ccece30 con 0x55973d0a3e20 2024-09-13T18:29:28.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.663+0000 7fa2b0bd9640 1 --2- 172.21.15.114:0/1263966971 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55973d0a3e20 0x55973d0a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:28.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.670+0000 7fa2b0bd9640 1 --2- 172.21.15.114:0/1263966971 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55973d0a3e20 0x55973d0a6240 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7986dc82a3d48bc5 server_cookie=98dbe95bb7fdf0ef in_seq=0 out_seq=0 2024-09-13T18:29:28.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.673+0000 7fa293fff640 10 client.27077 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:29:28.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.673+0000 7fa293fff640 1 -- 172.21.15.114:0/1263966971 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55973ccece30 con 0x55973d0a3e20 2024-09-13T18:29:28.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.673+0000 7fa293fff640 10 client.27077 handle_command_reply: tid=0 2024-09-13T18:29:28.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.673+0000 7fa2b5c35740 1 client.27077 shutdown 2024-09-13T18:29:28.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.686+0000 7fa2b5c35740 1 -- 172.21.15.114:0/1263966971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55973d040560 msgr2=0x55973d0409c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:28.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.686+0000 7fa2b5c35740 1 --2- 172.21.15.114:0/1263966971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55973d040560 0x55973d0409c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400ae20 tx=0x7fa2a4014e50 comp rx=0 tx=0).stop 2024-09-13T18:29:28.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.691+0000 7fa2b5c35740 1 -- 172.21.15.114:0/1263966971 shutdown_connections 2024-09-13T18:29:28.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.691+0000 7fa2b5c35740 1 --2- 172.21.15.114:0/1263966971 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55973d0a3e20 0x55973d0a6240 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.691+0000 7fa2b5c35740 1 --2- 172.21.15.114:0/1263966971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55973d040560 0x55973d0409c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.691+0000 7fa2b5c35740 1 --2- 172.21.15.114:0/1263966971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55973d03c1a0 0x55973d03ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.691+0000 7fa2b5c35740 1 --2- 172.21.15.114:0/1263966971 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55973d031900 0x55973d043400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.691+0000 7fa2b5c35740 1 -- 172.21.15.114:0/1263966971 >> 172.21.15.114:0/1263966971 conn(0x55973d02d570 msgr2=0x55973d02d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:28.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.693+0000 7fa293fff640 0 client.27077 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:29:28.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.698+0000 7fa2b5c35740 1 -- 172.21.15.114:0/1263966971 shutdown_connections 2024-09-13T18:29:28.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.699+0000 7fa2b5c35740 1 -- 172.21.15.114:0/1263966971 wait complete. 2024-09-13T18:29:28.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.702+0000 7fa2b5c35740 20 client.27077 trim_cache size 0 max 16384 2024-09-13T18:29:28.705 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:29:28.705 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-13T18:29:28.705 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": { 2024-09-13T18:29:28.706 INFO:teuthology.orchestra.run.smithi114.stdout: "f3652753-e416-46cd-b581-b4cfb8add1bf": { 2024-09-13T18:29:28.706 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-13T18:29:28.706 INFO:teuthology.orchestra.run.smithi114.stdout: "tag": "f3652753-e416-46cd-b581-b4cfb8add1bf", 2024-09-13T18:29:28.706 INFO:teuthology.orchestra.run.smithi114.stdout: "options": "recursive,force" 2024-09-13T18:29:28.706 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:29:28.706 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:29:28.706 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:29:28.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.704+0000 7fa2b1bdb640 1 -- 172.21.15.114:0/2413613499 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa2980a1260 msgr2=0x7fa2980a3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:28.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.704+0000 7fa2b1bdb640 1 --2- 172.21.15.114:0/2413613499 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa2980a1260 0x7fa2980a3720 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400a4d0 tx=0x7fa2a4009650 comp rx=0 tx=0).stop 2024-09-13T18:29:28.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.704+0000 7fa2b1bdb640 1 -- 172.21.15.114:0/2413613499 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2ac1074b0 msgr2=0x7fa2ac112750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:28.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.704+0000 7fa2b1bdb640 1 --2- 172.21.15.114:0/2413613499 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2ac1074b0 0x7fa2ac112750 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fa29400c920 tx=0x7fa29400cde0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.705+0000 7fa2b1bdb640 1 -- 172.21.15.114:0/2413613499 shutdown_connections 2024-09-13T18:29:28.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.705+0000 7fa2b1bdb640 1 --2- 172.21.15.114:0/2413613499 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa2980a1260 0x7fa2980a3720 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.705+0000 7fa2b1bdb640 1 --2- 172.21.15.114:0/2413613499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2ac119400 0x7fa2ac117080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.705+0000 7fa2b1bdb640 1 --2- 172.21.15.114:0/2413613499 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2ac107890 0x7fa2ac112c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.705+0000 7fa2b1bdb640 1 --2- 172.21.15.114:0/2413613499 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2ac1074b0 0x7fa2ac112750 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:28.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.705+0000 7fa2b1bdb640 1 -- 172.21.15.114:0/2413613499 >> 172.21.15.114:0/2413613499 conn(0x7fa2ac06af40 msgr2=0x7fa2ac070120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:28.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.705+0000 7fa2b1bdb640 1 -- 172.21.15.114:0/2413613499 shutdown_connections 2024-09-13T18:29:28.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:28.705+0000 7fa2b1bdb640 1 -- 172.21.15.114:0/2413613499 wait complete. 2024-09-13T18:29:28.744 INFO:tasks.cephfs.filesystem:scrub status for tag:f3652753-e416-46cd-b581-b4cfb8add1bf - {'path': '/', 'tag': 'f3652753-e416-46cd-b581-b4cfb8add1bf', 'options': 'recursive,force'} 2024-09-13T18:29:29.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:29:28 smithi204 ceph-mon[34777]: scrub queued for path: / 2024-09-13T18:29:29.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:29:28 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:29:29.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:29:28 smithi204 ceph-mon[34777]: scrub summary: active paths [/] 2024-09-13T18:29:29.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:29:28 smithi204 ceph-mon[34777]: scrub summary: active paths [/] 2024-09-13T18:29:29.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:29:28 smithi204 ceph-mon[34777]: scrub summary: active paths [/] 2024-09-13T18:29:29.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:29:28 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:29:29.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:29:28 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:29:29.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:29:28 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:29:29.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:29:28 smithi114 ceph-mon[30897]: scrub queued for path: / 2024-09-13T18:29:29.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:29:28 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:29:29.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:29:28 smithi114 ceph-mon[30897]: scrub summary: active paths [/] 2024-09-13T18:29:29.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:29:28 smithi114 ceph-mon[30897]: scrub summary: active paths [/] 2024-09-13T18:29:29.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:29:28 smithi114 ceph-mon[30897]: scrub summary: active paths [/] 2024-09-13T18:29:29.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:29:28 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:29:29.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:29:28 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:29:29.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:29:28 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:29:29.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:29:28 smithi134 ceph-mon[34283]: scrub queued for path: / 2024-09-13T18:29:29.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:29:28 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:29:29.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:29:28 smithi134 ceph-mon[34283]: scrub summary: active paths [/] 2024-09-13T18:29:29.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:29:28 smithi134 ceph-mon[34283]: scrub summary: active paths [/] 2024-09-13T18:29:29.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:29:28 smithi134 ceph-mon[34283]: scrub summary: active paths [/] 2024-09-13T18:29:29.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:29:28 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:29:29.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:29:28 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:29:29.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:29:28 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:29:30.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:29:29 smithi204 ceph-mon[34777]: pgmap v215: 129 pgs: 129 active+clean; 21 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 30 MiB/s wr, 672 op/s 2024-09-13T18:29:30.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:29:29 smithi114 ceph-mon[30897]: pgmap v215: 129 pgs: 129 active+clean; 21 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 30 MiB/s wr, 672 op/s 2024-09-13T18:29:30.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:29:29 smithi134 ceph-mon[34283]: pgmap v215: 129 pgs: 129 active+clean; 21 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 30 MiB/s wr, 672 op/s 2024-09-13T18:29:30.783 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.781+0000 7f7a4ffff640 1 -- 172.21.15.114:0/1604288189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a50119400 msgr2=0x7f7a5011b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:30.783 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.781+0000 7f7a4ffff640 1 --2- 172.21.15.114:0/1604288189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a50119400 0x7f7a5011b7f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7a40009930 tx=0x7f7a4002a440 comp rx=0 tx=0).stop 2024-09-13T18:29:30.784 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.781+0000 7f7a4ffff640 1 -- 172.21.15.114:0/1604288189 shutdown_connections 2024-09-13T18:29:30.784 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.781+0000 7f7a4ffff640 1 --2- 172.21.15.114:0/1604288189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a50119400 0x7f7a5011b7f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:30.784 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.781+0000 7f7a4ffff640 1 --2- 172.21.15.114:0/1604288189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a50107890 0x7f7a50107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:30.784 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.781+0000 7f7a4ffff640 1 --2- 172.21.15.114:0/1604288189 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a501074b0 0x7f7a5010e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:30.784 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.781+0000 7f7a4ffff640 1 -- 172.21.15.114:0/1604288189 >> 172.21.15.114:0/1604288189 conn(0x7f7a5006adb0 msgr2=0x7f7a5006b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:30.784 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.781+0000 7f7a4ffff640 1 -- 172.21.15.114:0/1604288189 shutdown_connections 2024-09-13T18:29:30.784 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.782+0000 7f7a4ffff640 1 -- 172.21.15.114:0/1604288189 wait complete. 2024-09-13T18:29:30.785 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.782+0000 7f7a4ffff640 1 Processor -- start 2024-09-13T18:29:30.785 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.783+0000 7f7a4ffff640 1 -- start start 2024-09-13T18:29:30.785 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.783+0000 7f7a4ffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a501074b0 0x7f7a50116850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:30.785 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.783+0000 7f7a4ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a50107890 0x7f7a50116db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:30.785 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.783+0000 7f7a4ffff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a50119400 0x7f7a5010fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:30.785 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.783+0000 7f7a4ffff640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a501101d0 con 0x7f7a50119400 2024-09-13T18:29:30.786 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.783+0000 7f7a4ffff640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a50110340 con 0x7f7a50107890 2024-09-13T18:29:30.786 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.783+0000 7f7a4ffff640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a50110500 con 0x7f7a501074b0 2024-09-13T18:29:30.786 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.783+0000 7f7a4e7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a50107890 0x7f7a50116db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:30.786 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.783+0000 7f7a4e7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a50107890 0x7f7a50116db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:56404/0 (socket says 172.21.15.114:56404) 2024-09-13T18:29:30.786 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.783+0000 7f7a4e7fc640 1 -- 172.21.15.114:0/3026662238 learned_addr learned my addr 172.21.15.114:0/3026662238 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:30.786 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.784+0000 7f7a4e7fc640 1 -- 172.21.15.114:0/3026662238 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a501074b0 msgr2=0x7f7a50116850 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:29:30.786 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.784+0000 7f7a4e7fc640 1 --2- 172.21.15.114:0/3026662238 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a501074b0 0x7f7a50116850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:30.786 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.784+0000 7f7a4e7fc640 1 -- 172.21.15.114:0/3026662238 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a50119400 msgr2=0x7f7a5010fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:30.787 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.784+0000 7f7a4e7fc640 1 --2- 172.21.15.114:0/3026662238 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a50119400 0x7f7a5010fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:30.787 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.784+0000 7f7a4e7fc640 1 -- 172.21.15.114:0/3026662238 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a40009560 con 0x7f7a50107890 2024-09-13T18:29:30.787 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.784+0000 7f7a4e7fc640 1 --2- 172.21.15.114:0/3026662238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a50107890 0x7f7a50116db0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7a4400d950 tx=0x7f7a4400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:30.787 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.785+0000 7f7a37fff640 1 -- 172.21.15.114:0/3026662238 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a44009c30 con 0x7f7a50107890 2024-09-13T18:29:30.787 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.785+0000 7f7a37fff640 1 -- 172.21.15.114:0/3026662238 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7a44009d90 con 0x7f7a50107890 2024-09-13T18:29:30.788 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.785+0000 7f7a37fff640 1 -- 172.21.15.114:0/3026662238 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4400f620 con 0x7f7a50107890 2024-09-13T18:29:30.788 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.785+0000 7f7a4ffff640 1 -- 172.21.15.114:0/3026662238 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a501143e0 con 0x7f7a50107890 2024-09-13T18:29:30.788 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.786+0000 7f7a4ffff640 1 -- 172.21.15.114:0/3026662238 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a501148a0 con 0x7f7a50107890 2024-09-13T18:29:30.792 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.790+0000 7f7a585d9740 1 Processor -- start 2024-09-13T18:29:30.795 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.791+0000 7f7a585d9740 1 -- start start 2024-09-13T18:29:30.795 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.791+0000 7f7a585d9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f026ed5500 0x55f026ed58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:30.795 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.791+0000 7f7a585d9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f026ed5e20 0x55f026ee0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:30.795 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.791+0000 7f7a585d9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f026ee06c0 0x55f026ee2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:30.795 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.791+0000 7f7a585d9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f026e40c30 con 0x55f026ed5500 2024-09-13T18:29:30.795 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.791+0000 7f7a585d9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f026e3e3c0 con 0x55f026ee06c0 2024-09-13T18:29:30.795 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.791+0000 7f7a585d9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f026e39cb0 con 0x55f026ed5e20 2024-09-13T18:29:30.795 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.791+0000 7f7a37fff640 1 -- 172.21.15.114:0/3026662238 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f7a4400f780 con 0x7f7a50107890 2024-09-13T18:29:30.796 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.792+0000 7f7a4f7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f026ed5500 0x55f026ed58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:30.796 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.792+0000 7f7a4f7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f026ed5500 0x55f026ed58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57858/0 (socket says 172.21.15.114:57858) 2024-09-13T18:29:30.796 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.792+0000 7f7a4f7fe640 1 -- 172.21.15.114:0/1773735615 learned_addr learned my addr 172.21.15.114:0/1773735615 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:30.796 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.792+0000 7f7a4e7fc640 1 --2- 172.21.15.114:0/1773735615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f026ee06c0 0x55f026ee2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:30.796 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.792+0000 7f7a37fff640 1 --2- 172.21.15.114:0/3026662238 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7a380a14d0 0x7f7a380a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:30.796 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.793+0000 7f7a37fff640 1 -- 172.21.15.114:0/3026662238 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f7a440ccba0 con 0x7f7a50107890 2024-09-13T18:29:30.796 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.793+0000 7f7a4e7fc640 1 -- 172.21.15.114:0/1773735615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f026ed5e20 msgr2=0x55f026ee0180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:29:30.796 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.793+0000 7f7a4e7fc640 1 --2- 172.21.15.114:0/1773735615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f026ed5e20 0x55f026ee0180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:30.797 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.793+0000 7f7a4e7fc640 1 -- 172.21.15.114:0/1773735615 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f026ed5500 msgr2=0x55f026ed58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:30.797 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.793+0000 7f7a4e7fc640 1 --2- 172.21.15.114:0/1773735615 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f026ed5500 0x55f026ed58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:30.797 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.793+0000 7f7a4e7fc640 1 -- 172.21.15.114:0/1773735615 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f026de0910 con 0x55f026ee06c0 2024-09-13T18:29:30.798 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.795+0000 7f7a4f7fe640 1 --2- 172.21.15.114:0/3026662238 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7a380a14d0 0x7f7a380a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.795+0000 7f7a4e7fc640 1 --2- 172.21.15.114:0/1773735615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f026ee06c0 0x55f026ee2ab0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7a440c9ef0 tx=0x7f7a44005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2977ea16d973dbd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.796+0000 7f7a357fa640 1 -- 172.21.15.114:0/1773735615 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a440b7ea0 con 0x55f026ee06c0 2024-09-13T18:29:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.796+0000 7f7a357fa640 1 -- 172.21.15.114:0/1773735615 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7a44099680 con 0x55f026ee06c0 2024-09-13T18:29:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.796+0000 7f7a357fa640 1 -- 172.21.15.114:0/1773735615 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4409d6e0 con 0x55f026ee06c0 2024-09-13T18:29:30.800 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.798+0000 7f7a585d9740 1 -- 172.21.15.114:0/1773735615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f026ee06c0 msgr2=0x55f026ee2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:30.800 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.798+0000 7f7a585d9740 1 --2- 172.21.15.114:0/1773735615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f026ee06c0 0x55f026ee2ab0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7a440c9ef0 tx=0x7f7a44005f90 comp rx=0 tx=0).stop 2024-09-13T18:29:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.799+0000 7f7a585d9740 1 -- 172.21.15.114:0/1773735615 shutdown_connections 2024-09-13T18:29:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.799+0000 7f7a585d9740 1 --2- 172.21.15.114:0/1773735615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f026ee06c0 0x55f026ee2ab0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.799+0000 7f7a585d9740 1 --2- 172.21.15.114:0/1773735615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f026ed5e20 0x55f026ee0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.799+0000 7f7a585d9740 1 --2- 172.21.15.114:0/1773735615 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f026ed5500 0x55f026ed58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.799+0000 7f7a585d9740 1 -- 172.21.15.114:0/1773735615 >> 172.21.15.114:0/1773735615 conn(0x55f026ed1a90 msgr2=0x55f026ed1e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:30.806 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.803+0000 7f7a585d9740 1 -- 172.21.15.114:0/1773735615 shutdown_connections 2024-09-13T18:29:30.806 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.803+0000 7f7a4f7fe640 1 --2- 172.21.15.114:0/3026662238 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7a380a14d0 0x7f7a380a3990 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7a40005ab0 tx=0x7f7a40005fd0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:30.806 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.803+0000 7f7a585d9740 1 -- 172.21.15.114:0/1773735615 wait complete. 2024-09-13T18:29:30.806 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.804+0000 7f7a585d9740 1 -- start start 2024-09-13T18:29:30.806 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.804+0000 7f7a585d9740 1 Processor -- start 2024-09-13T18:29:30.807 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.805+0000 7f7a585d9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f026ed5500 0x55f026ee0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:30.808 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.805+0000 7f7a585d9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f026ed5e20 0x55f026edd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:30.808 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.805+0000 7f7a585d9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f026ee06c0 0x55f026edd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:30.808 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.805+0000 7f7a585d9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f026df7e80 con 0x55f026ed5500 2024-09-13T18:29:30.808 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.805+0000 7f7a585d9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f026df7b20 con 0x55f026ee06c0 2024-09-13T18:29:30.808 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.805+0000 7f7a585d9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f026dbd260 con 0x55f026ed5e20 2024-09-13T18:29:30.808 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.805+0000 7f7a4e7fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f026ed5e20 0x55f026edd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:30.808 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.805+0000 7f7a4e7fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f026ed5e20 0x55f026edd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:49430/0 (socket says 172.21.15.114:49430) 2024-09-13T18:29:30.808 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.805+0000 7f7a4e7fc640 1 -- 172.21.15.114:0/2898393768 learned_addr learned my addr 172.21.15.114:0/2898393768 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:30.808 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.805+0000 7f7a585d9740 10 client.0 _ll_register_callbacks cb 0x55f026df2df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:29:30.808 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.805+0000 7f7a4f7fe640 1 --2- 172.21.15.114:0/2898393768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f026ee06c0 0x55f026edd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:30.808 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.806+0000 7f7a4effd640 1 --2- 172.21.15.114:0/2898393768 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f026ed5500 0x55f026ee0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:30.809 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.806+0000 7f7a4f7fe640 1 -- 172.21.15.114:0/2898393768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f026ed5e20 msgr2=0x55f026edd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:30.809 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.806+0000 7f7a4f7fe640 1 --2- 172.21.15.114:0/2898393768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f026ed5e20 0x55f026edd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:30.809 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.806+0000 7f7a4f7fe640 1 -- 172.21.15.114:0/2898393768 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f026ed5500 msgr2=0x55f026ee0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:30.809 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.806+0000 7f7a4f7fe640 1 --2- 172.21.15.114:0/2898393768 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f026ed5500 0x55f026ee0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:30.809 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.806+0000 7f7a4f7fe640 1 -- 172.21.15.114:0/2898393768 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a440d4040 con 0x55f026ee06c0 2024-09-13T18:29:30.812 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.807+0000 7f7a4f7fe640 1 --2- 172.21.15.114:0/2898393768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f026ee06c0 0x55f026edd8f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7a4002a9b0 tx=0x7f7a40009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31d99d6fc47888cf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:30.812 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.807+0000 7f7a32ff5640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:29:30.812 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.807+0000 7f7a32ff5640 1 -- 172.21.15.114:0/2898393768 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a400408d0 con 0x55f026ee06c0 2024-09-13T18:29:30.812 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.807+0000 7f7a32ff5640 1 -- 172.21.15.114:0/2898393768 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f7a40040a30 con 0x55f026ee06c0 2024-09-13T18:29:30.812 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.807+0000 7f7a585d9740 1 -- 172.21.15.114:0/2898393768 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f026e62680 con 0x55f026ee06c0 2024-09-13T18:29:30.813 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.811+0000 7f7a32ff5640 1 -- 172.21.15.114:0/2898393768 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a40040bf0 con 0x55f026ee06c0 2024-09-13T18:29:30.821 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.818+0000 7f7a585d9740 10 client.27089 fetch_fsmap learned FSMap version 34 2024-09-13T18:29:30.821 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.818+0000 7f7a585d9740 1 -- 172.21.15.114:0/2898393768 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55f026e38330 con 0x55f026ee06c0 2024-09-13T18:29:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.820+0000 7f7a32ff5640 1 -- 172.21.15.114:0/2898393768 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7a40040d50 con 0x55f026ee06c0 2024-09-13T18:29:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.820+0000 7f7a32ff5640 1 -- 172.21.15.114:0/2898393768 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7a40039030 con 0x55f026ee06c0 2024-09-13T18:29:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.820+0000 7f7a585d9740 10 client.27089 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:29:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.820+0000 7f7a585d9740 10 client.27089 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:29:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.821+0000 7f7a585d9740 20 client.27089 populate_metadata read hostname 'smithi114' 2024-09-13T18:29:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.821+0000 7f7a585d9740 1 --2- 172.21.15.114:0/2898393768 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f026eeffd0 0x55f026ef23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.821+0000 7f7a585d9740 4 client.27089 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:29:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.821+0000 7f7a585d9740 1 -- 172.21.15.114:0/2898393768 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f026e62680 con 0x55f026eeffd0 2024-09-13T18:29:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.825+0000 7f7a4effd640 1 --2- 172.21.15.114:0/2898393768 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f026eeffd0 0x55f026ef23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.829+0000 7f7a4effd640 1 --2- 172.21.15.114:0/2898393768 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f026eeffd0 0x55f026ef23f0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17e52fbeafaee460 server_cookie=a9f9c3c8f1652ea5 in_seq=0 out_seq=0 2024-09-13T18:29:30.832 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.829+0000 7f7a32ff5640 10 client.27089 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:29:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.832+0000 7f7a32ff5640 1 -- 172.21.15.114:0/2898393768 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55f026e62680 con 0x55f026eeffd0 2024-09-13T18:29:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.832+0000 7f7a32ff5640 10 client.27089 handle_command_reply: tid=0 2024-09-13T18:29:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.832+0000 7f7a585d9740 1 client.27089 shutdown 2024-09-13T18:29:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.833+0000 7f7a585d9740 1 -- 172.21.15.114:0/2898393768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f026ee06c0 msgr2=0x55f026edd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.833+0000 7f7a585d9740 1 --2- 172.21.15.114:0/2898393768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f026ee06c0 0x55f026edd8f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7a4002a9b0 tx=0x7f7a40009260 comp rx=0 tx=0).stop 2024-09-13T18:29:30.840 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.833+0000 7f7a585d9740 1 -- 172.21.15.114:0/2898393768 shutdown_connections 2024-09-13T18:29:30.840 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.833+0000 7f7a585d9740 1 --2- 172.21.15.114:0/2898393768 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f026eeffd0 0x55f026ef23f0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:30.840 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.833+0000 7f7a32ff5640 0 client.27089 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:29:30.840 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.833+0000 7f7a585d9740 1 --2- 172.21.15.114:0/2898393768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f026ee06c0 0x55f026edd8f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:30.840 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.833+0000 7f7a585d9740 1 --2- 172.21.15.114:0/2898393768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f026ed5e20 0x55f026edd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:30.840 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.833+0000 7f7a585d9740 1 --2- 172.21.15.114:0/2898393768 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f026ed5500 0x55f026ee0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:30.840 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.833+0000 7f7a585d9740 1 -- 172.21.15.114:0/2898393768 >> 172.21.15.114:0/2898393768 conn(0x55f026ed1a90 msgr2=0x55f026ed0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.843+0000 7f7a585d9740 1 -- 172.21.15.114:0/2898393768 shutdown_connections 2024-09-13T18:29:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.843+0000 7f7a585d9740 1 -- 172.21.15.114:0/2898393768 wait complete. 2024-09-13T18:29:30.853 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.849+0000 7f7a585d9740 20 client.27089 trim_cache size 0 max 16384 2024-09-13T18:29:30.888 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.885+0000 7f7a585d9740 1 Processor -- start 2024-09-13T18:29:30.888 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.886+0000 7f7a585d9740 1 -- start start 2024-09-13T18:29:30.888 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.886+0000 7f7a585d9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f026ed5500 0x55f026ed1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:30.889 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.886+0000 7f7a585d9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f026ed5e20 0x55f026edfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:30.889 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.886+0000 7f7a585d9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f026ee06c0 0x55f026ee73b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:30.889 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.886+0000 7f7a585d9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f026ef7570 con 0x55f026ed5e20 2024-09-13T18:29:30.889 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.886+0000 7f7a585d9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f026ed8d50 con 0x55f026ed5500 2024-09-13T18:29:30.889 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.886+0000 7f7a585d9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f026ed8f10 con 0x55f026ee06c0 2024-09-13T18:29:30.889 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.886+0000 7f7a4e7fc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f026ed5e20 0x55f026edfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:30.889 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.886+0000 7f7a4e7fc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f026ed5e20 0x55f026edfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57880/0 (socket says 172.21.15.114:57880) 2024-09-13T18:29:30.889 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.886+0000 7f7a4e7fc640 1 -- 172.21.15.114:0/3690448849 learned_addr learned my addr 172.21.15.114:0/3690448849 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:30.890 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.887+0000 7f7a4e7fc640 1 -- 172.21.15.114:0/3690448849 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f026ee06c0 msgr2=0x55f026ee73b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:29:30.890 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.887+0000 7f7a4e7fc640 1 --2- 172.21.15.114:0/3690448849 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f026ee06c0 0x55f026ee73b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:30.890 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.887+0000 7f7a4e7fc640 1 -- 172.21.15.114:0/3690448849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f026ed5500 msgr2=0x55f026ed1550 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:29:30.890 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.887+0000 7f7a4e7fc640 1 --2- 172.21.15.114:0/3690448849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f026ed5500 0x55f026ed1550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:30.890 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.887+0000 7f7a4e7fc640 1 -- 172.21.15.114:0/3690448849 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f026dda300 con 0x55f026ed5e20 2024-09-13T18:29:30.890 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.887+0000 7f7a4e7fc640 1 --2- 172.21.15.114:0/3690448849 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f026ed5e20 0x55f026edfb70 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f7a44099840 tx=0x7f7a440b7f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13d66ee3f6bb0fe8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:30.892 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.889+0000 7f7a35ffb640 1 -- 172.21.15.114:0/3690448849 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a44014020 con 0x55f026ed5e20 2024-09-13T18:29:30.892 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.889+0000 7f7a35ffb640 1 -- 172.21.15.114:0/3690448849 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7a440d2070 con 0x55f026ed5e20 2024-09-13T18:29:30.893 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.890+0000 7f7a35ffb640 1 -- 172.21.15.114:0/3690448849 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4409d6e0 con 0x55f026ed5e20 2024-09-13T18:29:30.893 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.890+0000 7f7a585d9740 1 -- 172.21.15.114:0/3690448849 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f026ed5e20 msgr2=0x55f026edfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:30.893 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.890+0000 7f7a585d9740 1 --2- 172.21.15.114:0/3690448849 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f026ed5e20 0x55f026edfb70 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f7a44099840 tx=0x7f7a440b7f50 comp rx=0 tx=0).stop 2024-09-13T18:29:30.894 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.891+0000 7f7a585d9740 1 -- 172.21.15.114:0/3690448849 shutdown_connections 2024-09-13T18:29:30.894 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.891+0000 7f7a585d9740 1 --2- 172.21.15.114:0/3690448849 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f026ee06c0 0x55f026ee73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:30.894 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.891+0000 7f7a585d9740 1 --2- 172.21.15.114:0/3690448849 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f026ed5e20 0x55f026edfb70 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:30.894 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.891+0000 7f7a585d9740 1 --2- 172.21.15.114:0/3690448849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f026ed5500 0x55f026ed1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:30.894 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.891+0000 7f7a585d9740 1 -- 172.21.15.114:0/3690448849 >> 172.21.15.114:0/3690448849 conn(0x55f026ed1a90 msgr2=0x55f026ed8640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:30.897 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.894+0000 7f7a585d9740 1 -- 172.21.15.114:0/3690448849 shutdown_connections 2024-09-13T18:29:30.898 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.895+0000 7f7a585d9740 1 -- 172.21.15.114:0/3690448849 wait complete. 2024-09-13T18:29:30.898 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.895+0000 7f7a585d9740 1 -- start start 2024-09-13T18:29:30.898 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.896+0000 7f7a585d9740 1 Processor -- start 2024-09-13T18:29:30.898 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.896+0000 7f7a585d9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f026ed5500 0x55f026ee3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.896+0000 7f7a585d9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f026ed5e20 0x55f026ee4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.896+0000 7f7a585d9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f026ee06c0 0x55f026ee49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.896+0000 7f7a585d9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f026f464d0 con 0x55f026ed5e20 2024-09-13T18:29:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.896+0000 7f7a585d9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f026f46610 con 0x55f026ee06c0 2024-09-13T18:29:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.896+0000 7f7a585d9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f026f46780 con 0x55f026ed5500 2024-09-13T18:29:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.896+0000 7f7a585d9740 10 client.0 _ll_register_callbacks cb 0x7f7a440d0550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:29:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.897+0000 7f7a4effd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f026ed5500 0x55f026ee3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.897+0000 7f7a4effd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f026ed5500 0x55f026ee3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:49446/0 (socket says 172.21.15.114:49446) 2024-09-13T18:29:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.897+0000 7f7a4effd640 1 -- 172.21.15.114:0/1080719802 learned_addr learned my addr 172.21.15.114:0/1080719802 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:30.900 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.897+0000 7f7a4f7fe640 1 --2- 172.21.15.114:0/1080719802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f026ee06c0 0x55f026ee49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:30.900 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.897+0000 7f7a4e7fc640 1 --2- 172.21.15.114:0/1080719802 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f026ed5e20 0x55f026ee4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:30.900 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.897+0000 7f7a4effd640 1 -- 172.21.15.114:0/1080719802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f026ee06c0 msgr2=0x55f026ee49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:30.901 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.897+0000 7f7a4effd640 1 --2- 172.21.15.114:0/1080719802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f026ee06c0 0x55f026ee49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:30.901 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.897+0000 7f7a4effd640 1 -- 172.21.15.114:0/1080719802 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f026ed5e20 msgr2=0x55f026ee4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:30.901 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.897+0000 7f7a4effd640 1 --2- 172.21.15.114:0/1080719802 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f026ed5e20 0x55f026ee4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:30.901 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.897+0000 7f7a4effd640 1 -- 172.21.15.114:0/1080719802 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f026de0910 con 0x55f026ed5500 2024-09-13T18:29:30.901 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.898+0000 7f7a4effd640 1 --2- 172.21.15.114:0/1080719802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f026ed5500 0x55f026ee3f40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7a48019df0 tx=0x7f7a48004ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b6ecaa2bfb86a52 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:30.903 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.901+0000 7f7a32ff5640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:29:30.903 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.901+0000 7f7a32ff5640 1 -- 172.21.15.114:0/1080719802 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4801d030 con 0x55f026ed5500 2024-09-13T18:29:30.903 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.901+0000 7f7a585d9740 1 -- 172.21.15.114:0/1080719802 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7a44099680 con 0x55f026ed5500 2024-09-13T18:29:30.903 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.901+0000 7f7a32ff5640 1 -- 172.21.15.114:0/1080719802 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f7a4801e070 con 0x55f026ed5500 2024-09-13T18:29:30.903 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.901+0000 7f7a32ff5640 1 -- 172.21.15.114:0/1080719802 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a48016040 con 0x55f026ed5500 2024-09-13T18:29:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.903+0000 7f7a585d9740 10 client.27103 fetch_fsmap learned FSMap version 34 2024-09-13T18:29:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.903+0000 7f7a585d9740 1 -- 172.21.15.114:0/1080719802 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55f026dff8e0 con 0x55f026ed5500 2024-09-13T18:29:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.907+0000 7f7a32ff5640 1 -- 172.21.15.114:0/1080719802 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7a44099680 con 0x55f026ed5500 2024-09-13T18:29:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.907+0000 7f7a32ff5640 1 -- 172.21.15.114:0/1080719802 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7a48019030 con 0x55f026ed5500 2024-09-13T18:29:30.910 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.908+0000 7f7a585d9740 10 client.27103 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:29:30.910 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.908+0000 7f7a585d9740 10 client.27103 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:29:30.910 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.908+0000 7f7a585d9740 20 client.27103 populate_metadata read hostname 'smithi114' 2024-09-13T18:29:30.910 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.908+0000 7f7a585d9740 1 --2- 172.21.15.114:0/1080719802 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f026eeffd0 0x55f026f4a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:30.910 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.908+0000 7f7a585d9740 4 client.27103 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T18:29:30.911 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.908+0000 7f7a585d9740 1 -- 172.21.15.114:0/1080719802 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f026e2d5d0 con 0x55f026eeffd0 2024-09-13T18:29:30.911 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.909+0000 7f7a4e7fc640 1 --2- 172.21.15.114:0/1080719802 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f026eeffd0 0x55f026f4a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:30.912 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.910+0000 7f7a4e7fc640 1 --2- 172.21.15.114:0/1080719802 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f026eeffd0 0x55f026f4a010 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1277c6f921714392 server_cookie=fc3ef0359f20e67d in_seq=0 out_seq=0 2024-09-13T18:29:30.912 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.910+0000 7f7a32ff5640 10 client.27103 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:29:30.917 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.913+0000 7f7a32ff5640 1 -- 172.21.15.114:0/1080719802 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f026e2d5d0 con 0x55f026eeffd0 2024-09-13T18:29:30.917 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.913+0000 7f7a32ff5640 10 client.27103 handle_command_reply: tid=0 2024-09-13T18:29:30.917 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.913+0000 7f7a585d9740 1 client.27103 shutdown 2024-09-13T18:29:30.917 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.913+0000 7f7a585d9740 1 -- 172.21.15.114:0/1080719802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f026ed5500 msgr2=0x55f026ee3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:30.917 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.913+0000 7f7a585d9740 1 --2- 172.21.15.114:0/1080719802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f026ed5500 0x55f026ee3f40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7a48019df0 tx=0x7f7a48004ef0 comp rx=0 tx=0).stop 2024-09-13T18:29:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.913+0000 7f7a585d9740 1 -- 172.21.15.114:0/1080719802 shutdown_connections 2024-09-13T18:29:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.913+0000 7f7a585d9740 1 --2- 172.21.15.114:0/1080719802 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f026eeffd0 0x55f026f4a010 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.913+0000 7f7a585d9740 1 --2- 172.21.15.114:0/1080719802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f026ee06c0 0x55f026ee49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.913+0000 7f7a585d9740 1 --2- 172.21.15.114:0/1080719802 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f026ed5e20 0x55f026ee4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.914+0000 7f7a585d9740 1 --2- 172.21.15.114:0/1080719802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f026ed5500 0x55f026ee3f40 secure :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7a48019df0 tx=0x7f7a48004ef0 comp rx=0 tx=0).stop 2024-09-13T18:29:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.914+0000 7f7a585d9740 1 -- 172.21.15.114:0/1080719802 >> 172.21.15.114:0/1080719802 conn(0x55f026ed1a90 msgr2=0x55f026edba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.914+0000 7f7a32ff5640 0 client.27103 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:29:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.914+0000 7f7a585d9740 1 -- 172.21.15.114:0/1080719802 shutdown_connections 2024-09-13T18:29:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.914+0000 7f7a585d9740 1 -- 172.21.15.114:0/1080719802 wait complete. 2024-09-13T18:29:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.914+0000 7f7a585d9740 20 client.27103 trim_cache size 0 max 16384 2024-09-13T18:29:30.920 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-13T18:29:30.920 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-13T18:29:30.920 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-13T18:29:30.920 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-13T18:29:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.915+0000 7f7a4ffff640 1 -- 172.21.15.114:0/3026662238 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7a380a14d0 msgr2=0x7f7a380a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.915+0000 7f7a4ffff640 1 --2- 172.21.15.114:0/3026662238 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7a380a14d0 0x7f7a380a3990 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7a40005ab0 tx=0x7f7a40005fd0 comp rx=0 tx=0).stop 2024-09-13T18:29:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.915+0000 7f7a4ffff640 1 -- 172.21.15.114:0/3026662238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a50107890 msgr2=0x7f7a50116db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.915+0000 7f7a4ffff640 1 --2- 172.21.15.114:0/3026662238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a50107890 0x7f7a50116db0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7a4400d950 tx=0x7f7a4400de10 comp rx=0 tx=0).stop 2024-09-13T18:29:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.915+0000 7f7a4ffff640 1 -- 172.21.15.114:0/3026662238 shutdown_connections 2024-09-13T18:29:30.924 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.916+0000 7f7a4ffff640 1 --2- 172.21.15.114:0/3026662238 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7a380a14d0 0x7f7a380a3990 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:30.924 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.916+0000 7f7a4ffff640 1 --2- 172.21.15.114:0/3026662238 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a50119400 0x7f7a5010fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:30.924 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.916+0000 7f7a4ffff640 1 --2- 172.21.15.114:0/3026662238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a50107890 0x7f7a50116db0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:30.924 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.916+0000 7f7a4ffff640 1 --2- 172.21.15.114:0/3026662238 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a501074b0 0x7f7a50116850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:30.924 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.916+0000 7f7a4ffff640 1 -- 172.21.15.114:0/3026662238 >> 172.21.15.114:0/3026662238 conn(0x7f7a5006adb0 msgr2=0x7f7a50119c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:30.924 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.916+0000 7f7a4ffff640 1 -- 172.21.15.114:0/3026662238 shutdown_connections 2024-09-13T18:29:30.925 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:30.916+0000 7f7a4ffff640 1 -- 172.21.15.114:0/3026662238 wait complete. 2024-09-13T18:29:32.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:29:31 smithi204 ceph-mon[34777]: pgmap v216: 129 pgs: 129 active+clean; 21 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 44 MiB/s wr, 997 op/s 2024-09-13T18:29:32.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:29:31 smithi114 ceph-mon[30897]: pgmap v216: 129 pgs: 129 active+clean; 21 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 44 MiB/s wr, 997 op/s 2024-09-13T18:29:32.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:29:31 smithi134 ceph-mon[34283]: pgmap v216: 129 pgs: 129 active+clean; 21 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 44 MiB/s wr, 997 op/s 2024-09-13T18:29:34.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:29:33 smithi204 ceph-mon[34777]: pgmap v217: 129 pgs: 129 active+clean; 21 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 29 MiB/s wr, 660 op/s 2024-09-13T18:29:34.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:29:33 smithi114 ceph-mon[30897]: pgmap v217: 129 pgs: 129 active+clean; 21 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 29 MiB/s wr, 660 op/s 2024-09-13T18:29:34.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:29:33 smithi134 ceph-mon[34283]: pgmap v217: 129 pgs: 129 active+clean; 21 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 29 MiB/s wr, 660 op/s 2024-09-13T18:29:36.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:29:35 smithi204 ceph-mon[34777]: pgmap v218: 129 pgs: 129 active+clean; 21 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 34 MiB/s wr, 770 op/s 2024-09-13T18:29:36.219 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:29:35 smithi114 ceph-mon[30897]: pgmap v218: 129 pgs: 129 active+clean; 21 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 34 MiB/s wr, 770 op/s 2024-09-13T18:29:36.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:29:35 smithi134 ceph-mon[34283]: pgmap v218: 129 pgs: 129 active+clean; 21 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 34 MiB/s wr, 770 op/s 2024-09-13T18:29:38.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:29:37 smithi204 ceph-mon[34777]: pgmap v219: 129 pgs: 129 active+clean; 21 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 38 MiB/s wr, 863 op/s 2024-09-13T18:29:38.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:29:37 smithi114 ceph-mon[30897]: pgmap v219: 129 pgs: 129 active+clean; 21 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 38 MiB/s wr, 863 op/s 2024-09-13T18:29:38.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:29:37 smithi134 ceph-mon[34283]: pgmap v219: 129 pgs: 129 active+clean; 21 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 38 MiB/s wr, 863 op/s 2024-09-13T18:29:40.196 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:29:39 smithi204 ceph-mon[34777]: pgmap v220: 129 pgs: 129 active+clean; 21 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 28 MiB/s wr, 642 op/s 2024-09-13T18:29:40.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:29:39 smithi114 ceph-mon[30897]: pgmap v220: 129 pgs: 129 active+clean; 21 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 28 MiB/s wr, 642 op/s 2024-09-13T18:29:40.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:29:39 smithi134 ceph-mon[34283]: pgmap v220: 129 pgs: 129 active+clean; 21 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 28 MiB/s wr, 642 op/s 2024-09-13T18:29:42.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:29:41 smithi204 ceph-mon[34777]: pgmap v221: 129 pgs: 129 active+clean; 21 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 42 MiB/s wr, 967 op/s 2024-09-13T18:29:42.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:29:41 smithi204 ceph-mon[34777]: scrub summary: idle 2024-09-13T18:29:42.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:29:41 smithi204 ceph-mon[34777]: scrub summary: idle 2024-09-13T18:29:42.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:29:41 smithi204 ceph-mon[34777]: scrub summary: idle 2024-09-13T18:29:42.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:29:41 smithi114 ceph-mon[30897]: pgmap v221: 129 pgs: 129 active+clean; 21 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 42 MiB/s wr, 967 op/s 2024-09-13T18:29:42.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:29:41 smithi114 ceph-mon[30897]: scrub summary: idle 2024-09-13T18:29:42.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:29:41 smithi114 ceph-mon[30897]: scrub summary: idle 2024-09-13T18:29:42.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:29:41 smithi114 ceph-mon[30897]: scrub summary: idle 2024-09-13T18:29:42.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:29:41 smithi134 ceph-mon[34283]: pgmap v221: 129 pgs: 129 active+clean; 21 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 42 MiB/s wr, 967 op/s 2024-09-13T18:29:42.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:29:41 smithi134 ceph-mon[34283]: scrub summary: idle 2024-09-13T18:29:42.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:29:41 smithi134 ceph-mon[34283]: scrub summary: idle 2024-09-13T18:29:42.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:29:41 smithi134 ceph-mon[34283]: scrub summary: idle 2024-09-13T18:29:43.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:29:42 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:29:43.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:29:42 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:29:43.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:29:42 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:29:44.148 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.144+0000 7f7ba9d72640 1 -- 172.21.15.114:0/2171494092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ba4107890 msgr2=0x7f7ba4107cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:44.148 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.144+0000 7f7ba9d72640 1 --2- 172.21.15.114:0/2171494092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ba4107890 0x7f7ba4107cd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7b98009890 tx=0x7f7b9802a3b0 comp rx=0 tx=0).stop 2024-09-13T18:29:44.148 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.144+0000 7f7ba9d72640 1 -- 172.21.15.114:0/2171494092 shutdown_connections 2024-09-13T18:29:44.148 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.144+0000 7f7ba9d72640 1 --2- 172.21.15.114:0/2171494092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ba4119400 0x7f7ba411b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:44.148 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.144+0000 7f7ba9d72640 1 --2- 172.21.15.114:0/2171494092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ba4107890 0x7f7ba4107cd0 secure :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7b98009890 tx=0x7f7b9802a3b0 comp rx=0 tx=0).stop 2024-09-13T18:29:44.148 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.144+0000 7f7ba9d72640 1 --2- 172.21.15.114:0/2171494092 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ba41074b0 0x7f7ba410e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:44.148 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.144+0000 7f7ba9d72640 1 -- 172.21.15.114:0/2171494092 >> 172.21.15.114:0/2171494092 conn(0x7f7ba406adb0 msgr2=0x7f7ba406b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:44.151 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.149+0000 7f7ba9d72640 1 -- 172.21.15.114:0/2171494092 shutdown_connections 2024-09-13T18:29:44.151 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.149+0000 7f7ba9d72640 1 -- 172.21.15.114:0/2171494092 wait complete. 2024-09-13T18:29:44.152 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.150+0000 7f7ba9d72640 1 Processor -- start 2024-09-13T18:29:44.152 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.150+0000 7f7ba9d72640 1 -- start start 2024-09-13T18:29:44.153 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.151+0000 7f7ba9d72640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ba41074b0 0x7f7ba41a4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:44.153 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.151+0000 7f7ba9d72640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ba4119400 0x7f7ba41a54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:44.153 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.151+0000 7f7ba9d72640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ba41a9890 0x7f7ba41a9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:44.153 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.151+0000 7f7ba9d72640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba41aa280 con 0x7f7ba41074b0 2024-09-13T18:29:44.154 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.151+0000 7f7ba9d72640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba41aa3f0 con 0x7f7ba4119400 2024-09-13T18:29:44.154 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.151+0000 7f7ba9d72640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba41aa560 con 0x7f7ba41a9890 2024-09-13T18:29:44.154 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.151+0000 7f7ba3fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ba4119400 0x7f7ba41a54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:44.155 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.151+0000 7f7ba8d70640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ba41074b0 0x7f7ba41a4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:44.156 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.151+0000 7f7ba3fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ba4119400 0x7f7ba41a54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:51230/0 (socket says 172.21.15.114:51230) 2024-09-13T18:29:44.156 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.151+0000 7f7ba3fff640 1 -- 172.21.15.114:0/1114673903 learned_addr learned my addr 172.21.15.114:0/1114673903 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:44.156 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.151+0000 7f7ba9571640 1 --2- 172.21.15.114:0/1114673903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ba41a9890 0x7f7ba41a9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:44.156 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.151+0000 7f7ba3fff640 1 -- 172.21.15.114:0/1114673903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ba41a9890 msgr2=0x7f7ba41a9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:44.156 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.151+0000 7f7ba3fff640 1 --2- 172.21.15.114:0/1114673903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ba41a9890 0x7f7ba41a9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:44.156 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.152+0000 7f7ba3fff640 1 -- 172.21.15.114:0/1114673903 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ba41074b0 msgr2=0x7f7ba41a4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:44.156 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.152+0000 7f7ba3fff640 1 --2- 172.21.15.114:0/1114673903 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ba41074b0 0x7f7ba41a4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:44.156 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.152+0000 7f7ba3fff640 1 -- 172.21.15.114:0/1114673903 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b98009490 con 0x7f7ba4119400 2024-09-13T18:29:44.156 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.152+0000 7f7ba3fff640 1 --2- 172.21.15.114:0/1114673903 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ba4119400 0x7f7ba41a54a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7b9802a920 tx=0x7f7b98004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.152+0000 7f7ba1ffb640 1 -- 172.21.15.114:0/1114673903 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b98005230 con 0x7f7ba4119400 2024-09-13T18:29:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.152+0000 7f7ba1ffb640 1 -- 172.21.15.114:0/1114673903 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7b98005390 con 0x7f7ba4119400 2024-09-13T18:29:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.152+0000 7f7ba1ffb640 1 -- 172.21.15.114:0/1114673903 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b98003850 con 0x7f7ba4119400 2024-09-13T18:29:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.152+0000 7f7ba9d72640 1 -- 172.21.15.114:0/1114673903 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ba41b1b40 con 0x7f7ba4119400 2024-09-13T18:29:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.153+0000 7f7ba9d72640 1 -- 172.21.15.114:0/1114673903 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ba41b1f10 con 0x7f7ba4119400 2024-09-13T18:29:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.156+0000 7f7ba1ffb640 1 -- 172.21.15.114:0/1114673903 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f7b980039b0 con 0x7f7ba4119400 2024-09-13T18:29:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.157+0000 7f7ba1ffb640 1 --2- 172.21.15.114:0/1114673903 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7b8c0a13f0 0x7f7b8c0a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:44.161 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.157+0000 7f7ba1ffb640 1 -- 172.21.15.114:0/1114673903 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f7b980ed660 con 0x7f7ba4119400 2024-09-13T18:29:44.161 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.159+0000 7f7ba8d70640 1 --2- 172.21.15.114:0/1114673903 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7b8c0a13f0 0x7f7b8c0a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:44.161 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.159+0000 7f7ba8d70640 1 --2- 172.21.15.114:0/1114673903 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7b8c0a13f0 0x7f7b8c0a38b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c005fd0 tx=0x7f7b9c010040 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.160+0000 7f7baddd3740 1 Processor -- start 2024-09-13T18:29:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.160+0000 7f7baddd3740 1 -- start start 2024-09-13T18:29:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.160+0000 7f7baddd3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f9a30b3c0 0x564f9a30b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.160+0000 7f7baddd3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f9a30bce0 0x564f9a327190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.160+0000 7f7baddd3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f9a3276d0 0x564f9a329ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.160+0000 7f7baddd3740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f9a276c30 con 0x564f9a30b3c0 2024-09-13T18:29:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.160+0000 7f7baddd3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f9a2743c0 con 0x564f9a3276d0 2024-09-13T18:29:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.160+0000 7f7baddd3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f9a26fcb0 con 0x564f9a30bce0 2024-09-13T18:29:44.165 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.161+0000 7f7ba9571640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f9a3276d0 0x564f9a329ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:44.165 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.161+0000 7f7ba9571640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f9a3276d0 0x564f9a329ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:51240/0 (socket says 172.21.15.114:51240) 2024-09-13T18:29:44.165 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.161+0000 7f7ba9571640 1 -- 172.21.15.114:0/2597886789 learned_addr learned my addr 172.21.15.114:0/2597886789 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:44.165 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.161+0000 7f7ba9571640 1 -- 172.21.15.114:0/2597886789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f9a30bce0 msgr2=0x564f9a327190 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:29:44.165 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.161+0000 7f7ba9571640 1 --2- 172.21.15.114:0/2597886789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f9a30bce0 0x564f9a327190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:44.165 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.161+0000 7f7ba9571640 1 -- 172.21.15.114:0/2597886789 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f9a30b3c0 msgr2=0x564f9a30b7a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:29:44.165 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.161+0000 7f7ba9571640 1 --2- 172.21.15.114:0/2597886789 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f9a30b3c0 0x564f9a30b7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:44.165 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.161+0000 7f7ba9571640 1 -- 172.21.15.114:0/2597886789 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f9a216910 con 0x564f9a3276d0 2024-09-13T18:29:44.165 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.161+0000 7f7ba9571640 1 --2- 172.21.15.114:0/2597886789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f9a3276d0 0x564f9a329ac0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7b9400bb70 tx=0x7f7b9400f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4cd86ce0f6a4c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:44.166 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.163+0000 7f7b8affd640 1 -- 172.21.15.114:0/2597886789 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9400be60 con 0x564f9a3276d0 2024-09-13T18:29:44.166 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.163+0000 7f7b8affd640 1 -- 172.21.15.114:0/2597886789 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7b940054b0 con 0x564f9a3276d0 2024-09-13T18:29:44.166 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.164+0000 7f7baddd3740 1 -- 172.21.15.114:0/2597886789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f9a3276d0 msgr2=0x564f9a329ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:44.166 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.164+0000 7f7baddd3740 1 --2- 172.21.15.114:0/2597886789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f9a3276d0 0x564f9a329ac0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7b9400bb70 tx=0x7f7b9400f730 comp rx=0 tx=0).stop 2024-09-13T18:29:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.164+0000 7f7baddd3740 1 -- 172.21.15.114:0/2597886789 shutdown_connections 2024-09-13T18:29:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.164+0000 7f7baddd3740 1 --2- 172.21.15.114:0/2597886789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f9a3276d0 0x564f9a329ac0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.164+0000 7f7baddd3740 1 --2- 172.21.15.114:0/2597886789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f9a30bce0 0x564f9a327190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.164+0000 7f7baddd3740 1 --2- 172.21.15.114:0/2597886789 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f9a30b3c0 0x564f9a30b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.164+0000 7f7baddd3740 1 -- 172.21.15.114:0/2597886789 >> 172.21.15.114:0/2597886789 conn(0x564f9a307a90 msgr2=0x564f9a307e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:44.168 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.165+0000 7f7baddd3740 1 -- 172.21.15.114:0/2597886789 shutdown_connections 2024-09-13T18:29:44.168 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.165+0000 7f7baddd3740 1 -- 172.21.15.114:0/2597886789 wait complete. 2024-09-13T18:29:44.169 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.165+0000 7f7baddd3740 1 -- start start 2024-09-13T18:29:44.169 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.165+0000 7f7baddd3740 1 Processor -- start 2024-09-13T18:29:44.169 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.166+0000 7f7baddd3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f9a30b3c0 0x564f9a3270f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:44.169 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.166+0000 7f7baddd3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f9a30bce0 0x564f9a3243e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:44.169 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.166+0000 7f7baddd3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f9a3276d0 0x564f9a3249f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:44.169 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.166+0000 7f7baddd3740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f9a22de80 con 0x564f9a3276d0 2024-09-13T18:29:44.169 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.166+0000 7f7baddd3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f9a22db20 con 0x564f9a30bce0 2024-09-13T18:29:44.169 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.166+0000 7f7baddd3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f9a1f3260 con 0x564f9a30b3c0 2024-09-13T18:29:44.169 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.166+0000 7f7baddd3740 10 client.0 _ll_register_callbacks cb 0x564f9a228df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:29:44.169 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.166+0000 7f7ba3fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f9a30bce0 0x564f9a3243e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:44.169 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.166+0000 7f7ba3fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f9a30bce0 0x564f9a3243e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:51252/0 (socket says 172.21.15.114:51252) 2024-09-13T18:29:44.170 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.166+0000 7f7ba3fff640 1 -- 172.21.15.114:0/4164675270 learned_addr learned my addr 172.21.15.114:0/4164675270 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:44.170 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.167+0000 7f7ba9571640 1 --2- 172.21.15.114:0/4164675270 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f9a3276d0 0x564f9a3249f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:44.171 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.167+0000 7f7ba3fff640 1 -- 172.21.15.114:0/4164675270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f9a30b3c0 msgr2=0x564f9a3270f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:29:44.171 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.167+0000 7f7ba3fff640 1 --2- 172.21.15.114:0/4164675270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f9a30b3c0 0x564f9a3270f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:44.171 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.167+0000 7f7ba3fff640 1 -- 172.21.15.114:0/4164675270 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f9a3276d0 msgr2=0x564f9a3249f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:44.171 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.167+0000 7f7ba3fff640 1 --2- 172.21.15.114:0/4164675270 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f9a3276d0 0x564f9a3249f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:44.171 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.167+0000 7f7ba3fff640 1 -- 172.21.15.114:0/4164675270 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b9400b7d0 con 0x564f9a30bce0 2024-09-13T18:29:44.171 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.167+0000 7f7ba3fff640 1 --2- 172.21.15.114:0/4164675270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f9a30bce0 0x564f9a3243e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7b980304b0 tx=0x7f7b980ba040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd1e59dcf2c5842f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:44.171 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.168+0000 7f7b83fff640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:29:44.171 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.168+0000 7f7b83fff640 1 -- 172.21.15.114:0/4164675270 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b98003e60 con 0x564f9a30bce0 2024-09-13T18:29:44.171 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.168+0000 7f7baddd3740 1 -- 172.21.15.114:0/4164675270 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564f9a298680 con 0x564f9a30bce0 2024-09-13T18:29:44.171 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.168+0000 7f7b83fff640 1 -- 172.21.15.114:0/4164675270 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f7b980ba480 con 0x564f9a30bce0 2024-09-13T18:29:44.172 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.169+0000 7f7b83fff640 1 -- 172.21.15.114:0/4164675270 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b980be2b0 con 0x564f9a30bce0 2024-09-13T18:29:44.177 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.175+0000 7f7baddd3740 10 client.27104 fetch_fsmap learned FSMap version 34 2024-09-13T18:29:44.177 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.175+0000 7f7baddd3740 1 -- 172.21.15.114:0/4164675270 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x564f9a26e330 con 0x564f9a30bce0 2024-09-13T18:29:44.178 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.176+0000 7f7b83fff640 1 -- 172.21.15.114:0/4164675270 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564f9a298680 con 0x564f9a30bce0 2024-09-13T18:29:44.178 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.176+0000 7f7b83fff640 1 -- 172.21.15.114:0/4164675270 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7b980ed200 con 0x564f9a30bce0 2024-09-13T18:29:44.179 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.177+0000 7f7baddd3740 10 client.27104 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:29:44.179 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.177+0000 7f7baddd3740 10 client.27104 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:29:44.179 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.177+0000 7f7baddd3740 20 client.27104 populate_metadata read hostname 'smithi114' 2024-09-13T18:29:44.179 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.177+0000 7f7baddd3740 1 --2- 172.21.15.114:0/4164675270 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x564f9a310e60 0x564f9a313280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:44.179 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.177+0000 7f7baddd3740 4 client.27104 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:29:44.179 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.177+0000 7f7baddd3740 1 -- 172.21.15.114:0/4164675270 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564f9a2635d0 con 0x564f9a310e60 2024-09-13T18:29:44.180 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.177+0000 7f7ba8d70640 1 --2- 172.21.15.114:0/4164675270 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x564f9a310e60 0x564f9a313280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:44.184 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.182+0000 7f7ba8d70640 1 --2- 172.21.15.114:0/4164675270 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x564f9a310e60 0x564f9a313280 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ccdd8aa29624d2c server_cookie=a3e347342d24e654 in_seq=0 out_seq=0 2024-09-13T18:29:44.184 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.182+0000 7f7b83fff640 10 client.27104 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:29:44.192 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.190+0000 7f7b83fff640 1 -- 172.21.15.114:0/4164675270 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x564f9a2635d0 con 0x564f9a310e60 2024-09-13T18:29:44.192 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.190+0000 7f7b83fff640 10 client.27104 handle_command_reply: tid=0 2024-09-13T18:29:44.192 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.190+0000 7f7baddd3740 1 client.27104 shutdown 2024-09-13T18:29:44.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:29:43 smithi204 ceph-mon[34777]: pgmap v222: 129 pgs: 129 active+clean; 21 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 28 MiB/s wr, 643 op/s 2024-09-13T18:29:44.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:29:43 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:29:44.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:29:43 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:29:44.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:29:43 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:29:44.200 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.198+0000 7f7baddd3740 1 -- 172.21.15.114:0/4164675270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f9a30bce0 msgr2=0x564f9a3243e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:44.200 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.198+0000 7f7baddd3740 1 --2- 172.21.15.114:0/4164675270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f9a30bce0 0x564f9a3243e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7b980304b0 tx=0x7f7b980ba040 comp rx=0 tx=0).stop 2024-09-13T18:29:44.201 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.198+0000 7f7baddd3740 1 -- 172.21.15.114:0/4164675270 shutdown_connections 2024-09-13T18:29:44.201 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.198+0000 7f7baddd3740 1 --2- 172.21.15.114:0/4164675270 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x564f9a310e60 0x564f9a313280 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:44.201 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.198+0000 7f7baddd3740 1 --2- 172.21.15.114:0/4164675270 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f9a3276d0 0x564f9a3249f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:44.201 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.198+0000 7f7baddd3740 1 --2- 172.21.15.114:0/4164675270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f9a30bce0 0x564f9a3243e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:44.201 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.198+0000 7f7baddd3740 1 --2- 172.21.15.114:0/4164675270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f9a30b3c0 0x564f9a3270f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:44.201 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.198+0000 7f7baddd3740 1 -- 172.21.15.114:0/4164675270 >> 172.21.15.114:0/4164675270 conn(0x564f9a307a90 msgr2=0x564f9a306d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:44.201 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.199+0000 7f7b83fff640 0 client.27104 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:29:44.204 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.201+0000 7f7baddd3740 1 -- 172.21.15.114:0/4164675270 shutdown_connections 2024-09-13T18:29:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.202+0000 7f7baddd3740 1 -- 172.21.15.114:0/4164675270 wait complete. 2024-09-13T18:29:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.215+0000 7f7baddd3740 20 client.27104 trim_cache size 0 max 16384 2024-09-13T18:29:44.222 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:29:43 smithi114 ceph-mon[30897]: pgmap v222: 129 pgs: 129 active+clean; 21 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 28 MiB/s wr, 643 op/s 2024-09-13T18:29:44.222 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:29:43 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:29:44.223 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:29:43 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:29:44.223 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:29:43 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:29:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.257+0000 7f7baddd3740 1 Processor -- start 2024-09-13T18:29:44.260 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.257+0000 7f7baddd3740 1 -- start start 2024-09-13T18:29:44.261 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.258+0000 7f7baddd3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f9a30b3c0 0x564f9a326c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:44.261 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.258+0000 7f7baddd3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f9a30bce0 0x564f9a322c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:44.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.259+0000 7f7ba3fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f9a30bce0 0x564f9a322c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:44.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.259+0000 7f7ba3fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f9a30bce0 0x564f9a322c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:51262/0 (socket says 172.21.15.114:51262) 2024-09-13T18:29:44.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.259+0000 7f7ba3fff640 1 -- 172.21.15.114:0/2059911485 learned_addr learned my addr 172.21.15.114:0/2059911485 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:44.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.259+0000 7f7baddd3740 1 --2- 172.21.15.114:0/2059911485 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f9a3276d0 0x564f9a3231d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:44.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.260+0000 7f7baddd3740 1 -- 172.21.15.114:0/2059911485 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f9a3253f0 con 0x564f9a3276d0 2024-09-13T18:29:44.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.260+0000 7f7baddd3740 1 -- 172.21.15.114:0/2059911485 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f9a323710 con 0x564f9a30bce0 2024-09-13T18:29:44.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.260+0000 7f7ba9571640 1 --2- 172.21.15.114:0/2059911485 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f9a3276d0 0x564f9a3231d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:44.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.260+0000 7f7baddd3740 1 -- 172.21.15.114:0/2059911485 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f9a30df60 con 0x564f9a30b3c0 2024-09-13T18:29:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.260+0000 7f7ba3fff640 1 -- 172.21.15.114:0/2059911485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f9a30b3c0 msgr2=0x564f9a326c00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:29:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.260+0000 7f7ba3fff640 1 --2- 172.21.15.114:0/2059911485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f9a30b3c0 0x564f9a326c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.260+0000 7f7ba3fff640 1 -- 172.21.15.114:0/2059911485 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f9a3276d0 msgr2=0x564f9a3231d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.260+0000 7f7ba3fff640 1 --2- 172.21.15.114:0/2059911485 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f9a3276d0 0x564f9a3231d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.260+0000 7f7ba3fff640 1 -- 172.21.15.114:0/2059911485 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f9a210300 con 0x564f9a30bce0 2024-09-13T18:29:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.260+0000 7f7ba3fff640 1 --2- 172.21.15.114:0/2059911485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f9a30bce0 0x564f9a322c90 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7b98009860 tx=0x7f7b980b8a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbf727908eb3d363 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:44.269 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.267+0000 7f7b8b7fe640 1 -- 172.21.15.114:0/2059911485 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b980ba9b0 con 0x564f9a30bce0 2024-09-13T18:29:44.269 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.267+0000 7f7b8b7fe640 1 -- 172.21.15.114:0/2059911485 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7b980ba7d0 con 0x564f9a30bce0 2024-09-13T18:29:44.270 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.268+0000 7f7baddd3740 1 -- 172.21.15.114:0/2059911485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f9a30bce0 msgr2=0x564f9a322c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:44.270 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.268+0000 7f7baddd3740 1 --2- 172.21.15.114:0/2059911485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f9a30bce0 0x564f9a322c90 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7b98009860 tx=0x7f7b980b8a30 comp rx=0 tx=0).stop 2024-09-13T18:29:44.271 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.269+0000 7f7baddd3740 1 -- 172.21.15.114:0/2059911485 shutdown_connections 2024-09-13T18:29:44.271 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.269+0000 7f7baddd3740 1 --2- 172.21.15.114:0/2059911485 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f9a3276d0 0x564f9a3231d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:44.271 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.269+0000 7f7baddd3740 1 --2- 172.21.15.114:0/2059911485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f9a30bce0 0x564f9a322c90 secure :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7b98009860 tx=0x7f7b980b8a30 comp rx=0 tx=0).stop 2024-09-13T18:29:44.271 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.269+0000 7f7baddd3740 1 --2- 172.21.15.114:0/2059911485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f9a30b3c0 0x564f9a326c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:44.271 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.269+0000 7f7baddd3740 1 -- 172.21.15.114:0/2059911485 >> 172.21.15.114:0/2059911485 conn(0x564f9a307a90 msgr2=0x564f9a31f690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:44.279 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.277+0000 7f7baddd3740 1 -- 172.21.15.114:0/2059911485 shutdown_connections 2024-09-13T18:29:44.279 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.277+0000 7f7baddd3740 1 -- 172.21.15.114:0/2059911485 wait complete. 2024-09-13T18:29:44.279 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.277+0000 7f7baddd3740 1 -- start start 2024-09-13T18:29:44.280 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.278+0000 7f7baddd3740 1 Processor -- start 2024-09-13T18:29:44.282 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.279+0000 7f7baddd3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f9a30b3c0 0x564f9a32e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:44.282 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.279+0000 7f7baddd3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f9a3276d0 0x564f9a32aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:44.282 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.279+0000 7f7baddd3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f9a32afb0 0x564f9a32b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:44.282 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.279+0000 7f7ba8d70640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f9a30b3c0 0x564f9a32e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:44.282 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.279+0000 7f7ba8d70640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f9a30b3c0 0x564f9a32e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:40410/0 (socket says 172.21.15.114:40410) 2024-09-13T18:29:44.282 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.279+0000 7f7ba3fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f9a3276d0 0x564f9a32aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:44.282 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.279+0000 7f7ba8d70640 1 -- 172.21.15.114:0/3829054449 learned_addr learned my addr 172.21.15.114:0/3829054449 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:44.283 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.280+0000 7f7baddd3740 1 -- 172.21.15.114:0/3829054449 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f9a32edc0 con 0x564f9a32afb0 2024-09-13T18:29:44.283 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.280+0000 7f7baddd3740 1 -- 172.21.15.114:0/3829054449 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f9a39e5a0 con 0x564f9a3276d0 2024-09-13T18:29:44.283 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.280+0000 7f7ba9571640 1 --2- 172.21.15.114:0/3829054449 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f9a32afb0 0x564f9a32b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:44.283 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.280+0000 7f7baddd3740 1 -- 172.21.15.114:0/3829054449 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f9a39e730 con 0x564f9a30b3c0 2024-09-13T18:29:44.283 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.280+0000 7f7baddd3740 10 client.0 _ll_register_callbacks cb 0x7f7b9400b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:29:44.283 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.280+0000 7f7ba8d70640 1 -- 172.21.15.114:0/3829054449 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f9a3276d0 msgr2=0x564f9a32aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:44.284 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.280+0000 7f7ba8d70640 1 --2- 172.21.15.114:0/3829054449 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f9a3276d0 0x564f9a32aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:44.284 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.280+0000 7f7ba8d70640 1 -- 172.21.15.114:0/3829054449 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f9a32afb0 msgr2=0x564f9a32b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:44.284 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.280+0000 7f7ba8d70640 1 --2- 172.21.15.114:0/3829054449 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f9a32afb0 0x564f9a32b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:44.284 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.280+0000 7f7ba8d70640 1 -- 172.21.15.114:0/3829054449 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564f9a26e330 con 0x564f9a30b3c0 2024-09-13T18:29:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.283+0000 7f7ba8d70640 1 --2- 172.21.15.114:0/3829054449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f9a30b3c0 0x564f9a32e6a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c003c70 tx=0x7f7b9c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9bee0ff7e501401 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:44.286 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.284+0000 7f7b83fff640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:29:44.286 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.284+0000 7f7b83fff640 1 -- 172.21.15.114:0/3829054449 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c013040 con 0x564f9a30b3c0 2024-09-13T18:29:44.286 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.284+0000 7f7b83fff640 1 -- 172.21.15.114:0/3829054449 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f7b9c026280 con 0x564f9a30b3c0 2024-09-13T18:29:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.285+0000 7f7baddd3740 1 -- 172.21.15.114:0/3829054449 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564f99f63300 con 0x564f9a30b3c0 2024-09-13T18:29:44.291 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.289+0000 7f7b83fff640 1 -- 172.21.15.114:0/3829054449 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b9c026440 con 0x564f9a30b3c0 2024-09-13T18:29:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.291+0000 7f7baddd3740 10 client.27112 fetch_fsmap learned FSMap version 34 2024-09-13T18:29:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.291+0000 7f7baddd3740 1 -- 172.21.15.114:0/3829054449 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x564f9a2358e0 con 0x564f9a30b3c0 2024-09-13T18:29:44.296 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.294+0000 7f7b83fff640 1 -- 172.21.15.114:0/3829054449 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564f99f63300 con 0x564f9a30b3c0 2024-09-13T18:29:44.296 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.294+0000 7f7b83fff640 1 -- 172.21.15.114:0/3829054449 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7b9c00dc40 con 0x564f9a30b3c0 2024-09-13T18:29:44.296 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.294+0000 7f7baddd3740 10 client.27112 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:29:44.297 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.294+0000 7f7baddd3740 10 client.27112 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:29:44.297 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.294+0000 7f7baddd3740 20 client.27112 populate_metadata read hostname 'smithi114' 2024-09-13T18:29:44.297 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.294+0000 7f7baddd3740 1 --2- 172.21.15.114:0/3829054449 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x564f9a39ffd0 0x564f9a3a23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:44.297 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.294+0000 7f7baddd3740 4 client.27112 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T18:29:44.297 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.294+0000 7f7baddd3740 1 -- 172.21.15.114:0/3829054449 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564f99f74600 con 0x564f9a39ffd0 2024-09-13T18:29:44.298 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.296+0000 7f7ba3fff640 1 --2- 172.21.15.114:0/3829054449 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x564f9a39ffd0 0x564f9a3a23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:44.304 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.301+0000 7f7ba3fff640 1 --2- 172.21.15.114:0/3829054449 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x564f9a39ffd0 0x564f9a3a23f0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=887713fee1a99c3c server_cookie=845ea01daf4bb459 in_seq=0 out_seq=0 2024-09-13T18:29:44.305 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.303+0000 7f7b83fff640 10 client.27112 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:29:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.312+0000 7f7b83fff640 1 -- 172.21.15.114:0/3829054449 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564f99f74600 con 0x564f9a39ffd0 2024-09-13T18:29:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.312+0000 7f7b83fff640 10 client.27112 handle_command_reply: tid=0 2024-09-13T18:29:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.312+0000 7f7baddd3740 1 client.27112 shutdown 2024-09-13T18:29:44.315 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.313+0000 7f7baddd3740 1 -- 172.21.15.114:0/3829054449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f9a30b3c0 msgr2=0x564f9a32e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:44.315 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.313+0000 7f7baddd3740 1 --2- 172.21.15.114:0/3829054449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f9a30b3c0 0x564f9a32e6a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c003c70 tx=0x7f7b9c004a40 comp rx=0 tx=0).stop 2024-09-13T18:29:44.315 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.313+0000 7f7baddd3740 1 -- 172.21.15.114:0/3829054449 shutdown_connections 2024-09-13T18:29:44.315 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.313+0000 7f7baddd3740 1 --2- 172.21.15.114:0/3829054449 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x564f9a39ffd0 0x564f9a3a23f0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:44.315 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.313+0000 7f7baddd3740 1 --2- 172.21.15.114:0/3829054449 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f9a32afb0 0x564f9a32b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:44.315 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.313+0000 7f7baddd3740 1 --2- 172.21.15.114:0/3829054449 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f9a3276d0 0x564f9a32aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:44.315 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.313+0000 7f7baddd3740 1 --2- 172.21.15.114:0/3829054449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f9a30b3c0 0x564f9a32e6a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:44.315 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.313+0000 7f7baddd3740 1 -- 172.21.15.114:0/3829054449 >> 172.21.15.114:0/3829054449 conn(0x564f9a307a90 msgr2=0x564f9a322c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:44.316 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.313+0000 7f7b83fff640 0 client.27112 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:29:44.317 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.315+0000 7f7baddd3740 1 -- 172.21.15.114:0/3829054449 shutdown_connections 2024-09-13T18:29:44.318 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.316+0000 7f7baddd3740 1 -- 172.21.15.114:0/3829054449 wait complete. 2024-09-13T18:29:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.317+0000 7f7baddd3740 20 client.27112 trim_cache size 0 max 16384 2024-09-13T18:29:44.324 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-13T18:29:44.324 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-13T18:29:44.324 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-13T18:29:44.324 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-13T18:29:44.339 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.337+0000 7f7ba9d72640 1 -- 172.21.15.114:0/1114673903 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7b8c0a13f0 msgr2=0x7f7b8c0a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:44.339 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.337+0000 7f7ba9d72640 1 --2- 172.21.15.114:0/1114673903 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7b8c0a13f0 0x7f7b8c0a38b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c005fd0 tx=0x7f7b9c010040 comp rx=0 tx=0).stop 2024-09-13T18:29:44.339 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.337+0000 7f7ba9d72640 1 -- 172.21.15.114:0/1114673903 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ba4119400 msgr2=0x7f7ba41a54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:44.339 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.337+0000 7f7ba9d72640 1 --2- 172.21.15.114:0/1114673903 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ba4119400 0x7f7ba41a54a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7b9802a920 tx=0x7f7b98004270 comp rx=0 tx=0).stop 2024-09-13T18:29:44.342 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.339+0000 7f7ba9d72640 1 -- 172.21.15.114:0/1114673903 shutdown_connections 2024-09-13T18:29:44.342 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.339+0000 7f7ba9d72640 1 --2- 172.21.15.114:0/1114673903 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7b8c0a13f0 0x7f7b8c0a38b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:44.342 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.339+0000 7f7ba9d72640 1 --2- 172.21.15.114:0/1114673903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ba41a9890 0x7f7ba41a9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:44.342 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.339+0000 7f7ba9d72640 1 --2- 172.21.15.114:0/1114673903 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ba4119400 0x7f7ba41a54a0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:44.342 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.339+0000 7f7ba9d72640 1 --2- 172.21.15.114:0/1114673903 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ba41074b0 0x7f7ba41a4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:44.342 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.339+0000 7f7ba9d72640 1 -- 172.21.15.114:0/1114673903 >> 172.21.15.114:0/1114673903 conn(0x7f7ba406adb0 msgr2=0x7f7ba4119d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:44.342 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.340+0000 7f7ba9d72640 1 -- 172.21.15.114:0/1114673903 shutdown_connections 2024-09-13T18:29:44.343 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:44.340+0000 7f7ba9d72640 1 -- 172.21.15.114:0/1114673903 wait complete. 2024-09-13T18:29:44.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:29:43 smithi134 ceph-mon[34283]: pgmap v222: 129 pgs: 129 active+clean; 21 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 28 MiB/s wr, 643 op/s 2024-09-13T18:29:44.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:29:43 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:29:44.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:29:43 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:29:44.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:29:43 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:29:46.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:29:45 smithi204 ceph-mon[34777]: pgmap v223: 129 pgs: 129 active+clean; 21 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 33 MiB/s wr, 749 op/s 2024-09-13T18:29:46.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:29:45 smithi114 ceph-mon[30897]: pgmap v223: 129 pgs: 129 active+clean; 21 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 33 MiB/s wr, 749 op/s 2024-09-13T18:29:46.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:29:45 smithi134 ceph-mon[34283]: pgmap v223: 129 pgs: 129 active+clean; 21 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 33 MiB/s wr, 749 op/s 2024-09-13T18:29:48.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:29:47 smithi204 ceph-mon[34777]: pgmap v224: 129 pgs: 129 active+clean; 21 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 38 MiB/s wr, 851 op/s 2024-09-13T18:29:48.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:29:47 smithi114 ceph-mon[30897]: pgmap v224: 129 pgs: 129 active+clean; 21 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 38 MiB/s wr, 851 op/s 2024-09-13T18:29:48.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:29:47 smithi134 ceph-mon[34283]: pgmap v224: 129 pgs: 129 active+clean; 21 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 38 MiB/s wr, 851 op/s 2024-09-13T18:29:50.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:29:49 smithi204 ceph-mon[34777]: pgmap v225: 129 pgs: 129 active+clean; 21 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 29 MiB/s wr, 643 op/s 2024-09-13T18:29:50.222 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:29:49 smithi114 ceph-mon[30897]: pgmap v225: 129 pgs: 129 active+clean; 21 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 29 MiB/s wr, 643 op/s 2024-09-13T18:29:50.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:29:49 smithi134 ceph-mon[34283]: pgmap v225: 129 pgs: 129 active+clean; 21 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 29 MiB/s wr, 643 op/s 2024-09-13T18:29:52.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:29:51 smithi114 ceph-mon[30897]: pgmap v226: 129 pgs: 129 active+clean; 22 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 43 MiB/s wr, 967 op/s 2024-09-13T18:29:52.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:29:51 smithi134 ceph-mon[34283]: pgmap v226: 129 pgs: 129 active+clean; 22 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 43 MiB/s wr, 967 op/s 2024-09-13T18:29:52.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:29:51 smithi204 ceph-mon[34777]: pgmap v226: 129 pgs: 129 active+clean; 22 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 43 MiB/s wr, 967 op/s 2024-09-13T18:29:54.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:29:53 smithi114 ceph-mon[30897]: pgmap v227: 129 pgs: 129 active+clean; 22 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 29 MiB/s wr, 641 op/s 2024-09-13T18:29:54.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:29:53 smithi134 ceph-mon[34283]: pgmap v227: 129 pgs: 129 active+clean; 22 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 29 MiB/s wr, 641 op/s 2024-09-13T18:29:54.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:29:53 smithi204 ceph-mon[34777]: pgmap v227: 129 pgs: 129 active+clean; 22 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 29 MiB/s wr, 641 op/s 2024-09-13T18:29:56.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:29:55 smithi114 ceph-mon[30897]: pgmap v228: 129 pgs: 129 active+clean; 22 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 34 MiB/s wr, 745 op/s 2024-09-13T18:29:56.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:29:55 smithi134 ceph-mon[34283]: pgmap v228: 129 pgs: 129 active+clean; 22 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 34 MiB/s wr, 745 op/s 2024-09-13T18:29:56.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:29:55 smithi204 ceph-mon[34777]: pgmap v228: 129 pgs: 129 active+clean; 22 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 34 MiB/s wr, 745 op/s 2024-09-13T18:29:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.532+0000 7f85e9b08640 1 -- 172.21.15.114:0/2490364082 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85e40717c0 msgr2=0x7f85e4107720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.532+0000 7f85e9b08640 1 --2- 172.21.15.114:0/2490364082 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85e40717c0 0x7f85e4107720 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f85dc009ca0 tx=0x7f85dc02a8f0 comp rx=0 tx=0).stop 2024-09-13T18:29:57.538 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.535+0000 7f85e9b08640 1 -- 172.21.15.114:0/2490364082 shutdown_connections 2024-09-13T18:29:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.535+0000 7f85e9b08640 1 --2- 172.21.15.114:0/2490364082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85e410b600 0x7f85e410ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.535+0000 7f85e9b08640 1 --2- 172.21.15.114:0/2490364082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85e4071c30 0x7f85e4107c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.535+0000 7f85e9b08640 1 --2- 172.21.15.114:0/2490364082 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85e40717c0 0x7f85e4107720 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.535+0000 7f85e9b08640 1 -- 172.21.15.114:0/2490364082 >> 172.21.15.114:0/2490364082 conn(0x7f85e406ae30 msgr2=0x7f85e406b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.542+0000 7f85e9b08640 1 -- 172.21.15.114:0/2490364082 shutdown_connections 2024-09-13T18:29:57.552 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.547+0000 7f85e9b08640 1 -- 172.21.15.114:0/2490364082 wait complete. 2024-09-13T18:29:57.552 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.547+0000 7f85e9b08640 1 Processor -- start 2024-09-13T18:29:57.560 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.556+0000 7f85e9b08640 1 -- start start 2024-09-13T18:29:57.563 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.560+0000 7f85e9b08640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85e40717c0 0x7f85e4116ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:57.563 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.560+0000 7f85e9b08640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85e4071c30 0x7f85e41171e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:57.563 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.560+0000 7f85e9b08640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85e410b600 0x7f85e4117720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:57.564 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.560+0000 7f85e9b08640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e41b1090 con 0x7f85e4071c30 2024-09-13T18:29:57.564 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.560+0000 7f85e9b08640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e41b1200 con 0x7f85e410b600 2024-09-13T18:29:57.564 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.560+0000 7f85e9b08640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e41b13c0 con 0x7f85e40717c0 2024-09-13T18:29:57.564 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.560+0000 7f85e3fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85e410b600 0x7f85e4117720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:57.564 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.560+0000 7f85e3fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85e410b600 0x7f85e4117720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:43376/0 (socket says 172.21.15.114:43376) 2024-09-13T18:29:57.564 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.560+0000 7f85e3fff640 1 -- 172.21.15.114:0/4260765693 learned_addr learned my addr 172.21.15.114:0/4260765693 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:57.564 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.560+0000 7f85e3fff640 1 -- 172.21.15.114:0/4260765693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85e40717c0 msgr2=0x7f85e4116ca0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:29:57.564 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.560+0000 7f85e3fff640 1 --2- 172.21.15.114:0/4260765693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85e40717c0 0x7f85e4116ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:57.564 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.560+0000 7f85e3fff640 1 -- 172.21.15.114:0/4260765693 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85e4071c30 msgr2=0x7f85e41171e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:57.564 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.560+0000 7f85e2ffd640 1 --2- 172.21.15.114:0/4260765693 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85e4071c30 0x7f85e41171e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:57.565 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.560+0000 7f85e3fff640 1 --2- 172.21.15.114:0/4260765693 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85e4071c30 0x7f85e41171e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:57.565 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.560+0000 7f85e3fff640 1 -- 172.21.15.114:0/4260765693 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85dc009900 con 0x7f85e410b600 2024-09-13T18:29:57.565 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.561+0000 7f85e3fff640 1 --2- 172.21.15.114:0/4260765693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85e410b600 0x7f85e4117720 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f85d800b3f0 tx=0x7f85d800b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:57.565 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.561+0000 7f85e0ff9640 1 -- 172.21.15.114:0/4260765693 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d8011830 con 0x7f85e410b600 2024-09-13T18:29:57.565 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.561+0000 7f85e9b08640 1 -- 172.21.15.114:0/4260765693 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85e41b16a0 con 0x7f85e410b600 2024-09-13T18:29:57.567 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.562+0000 7f85e9b08640 1 -- 172.21.15.114:0/4260765693 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85e41b1bb0 con 0x7f85e410b600 2024-09-13T18:29:57.568 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.565+0000 7f85ec824740 1 Processor -- start 2024-09-13T18:29:57.569 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.565+0000 7f85ec824740 1 -- start start 2024-09-13T18:29:57.569 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.566+0000 7f85ec824740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5590268627d0 0x559026862bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:57.569 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.566+0000 7f85ec824740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5590268630f0 0x55902686d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:57.569 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.566+0000 7f85ec824740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55902686d990 0x55902686fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:57.569 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.566+0000 7f85ec824740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590267cdc30 con 0x5590268630f0 2024-09-13T18:29:57.569 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.566+0000 7f85ec824740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590267cb3c0 con 0x55902686d990 2024-09-13T18:29:57.569 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.566+0000 7f85ec824740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590267c6cb0 con 0x5590268627d0 2024-09-13T18:29:57.573 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.566+0000 7f85e2ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5590268627d0 0x559026862bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:57.573 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.566+0000 7f85e2ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5590268627d0 0x559026862bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:57912/0 (socket says 172.21.15.114:57912) 2024-09-13T18:29:57.573 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.566+0000 7f85e2ffd640 1 -- 172.21.15.114:0/2185174158 learned_addr learned my addr 172.21.15.114:0/2185174158 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:57.574 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.566+0000 7f85e2ffd640 1 --2- 172.21.15.114:0/2185174158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55902686d990 0x55902686fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:57.574 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.568+0000 7f85e37fe640 1 --2- 172.21.15.114:0/2185174158 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5590268630f0 0x55902686d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:57.574 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.568+0000 7f85e0ff9640 1 -- 172.21.15.114:0/4260765693 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f85d8011e20 con 0x7f85e410b600 2024-09-13T18:29:57.574 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.569+0000 7f85e0ff9640 1 -- 172.21.15.114:0/4260765693 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d8010e90 con 0x7f85e410b600 2024-09-13T18:29:57.574 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.569+0000 7f85e0ff9640 1 -- 172.21.15.114:0/4260765693 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f85d801a440 con 0x7f85e410b600 2024-09-13T18:29:57.574 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.570+0000 7f85e37fe640 1 -- 172.21.15.114:0/2185174158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5590268627d0 msgr2=0x559026862bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:57.574 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.570+0000 7f85e37fe640 1 --2- 172.21.15.114:0/2185174158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5590268627d0 0x559026862bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:57.574 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.570+0000 7f85e37fe640 1 -- 172.21.15.114:0/2185174158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55902686d990 msgr2=0x55902686fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:57.574 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.570+0000 7f85e37fe640 1 --2- 172.21.15.114:0/2185174158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55902686d990 0x55902686fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:57.574 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.570+0000 7f85e37fe640 1 -- 172.21.15.114:0/2185174158 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55902676d910 con 0x5590268630f0 2024-09-13T18:29:57.574 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.570+0000 7f85e37fe640 1 --2- 172.21.15.114:0/2185174158 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5590268630f0 0x55902686d450 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f85dc000f80 tx=0x7f85dc005f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c17c43d284a3eb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:57.574 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.570+0000 7f85e0ff9640 1 --2- 172.21.15.114:0/4260765693 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f85cc0a1550 0x7f85cc0a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:57.575 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.571+0000 7f85e0ff9640 1 -- 172.21.15.114:0/4260765693 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f85d80cda70 con 0x7f85e410b600 2024-09-13T18:29:57.575 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.571+0000 7f85c97fa640 1 -- 172.21.15.114:0/2185174158 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85dc0309f0 con 0x5590268630f0 2024-09-13T18:29:57.575 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.571+0000 7f85c97fa640 1 -- 172.21.15.114:0/2185174158 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f85dc030b50 con 0x5590268630f0 2024-09-13T18:29:57.575 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.571+0000 7f85c97fa640 1 -- 172.21.15.114:0/2185174158 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85dc0316e0 con 0x5590268630f0 2024-09-13T18:29:57.582 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.574+0000 7f85e2ffd640 1 --2- 172.21.15.114:0/4260765693 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f85cc0a1550 0x7f85cc0a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:57.582 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.575+0000 7f85ec824740 1 -- 172.21.15.114:0/2185174158 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5590268630f0 msgr2=0x55902686d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:57.582 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.575+0000 7f85ec824740 1 --2- 172.21.15.114:0/2185174158 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5590268630f0 0x55902686d450 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f85dc000f80 tx=0x7f85dc005f60 comp rx=0 tx=0).stop 2024-09-13T18:29:57.582 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.578+0000 7f85ec824740 1 -- 172.21.15.114:0/2185174158 shutdown_connections 2024-09-13T18:29:57.582 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.578+0000 7f85ec824740 1 --2- 172.21.15.114:0/2185174158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55902686d990 0x55902686fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:57.582 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.578+0000 7f85ec824740 1 --2- 172.21.15.114:0/2185174158 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5590268630f0 0x55902686d450 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:57.582 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.578+0000 7f85ec824740 1 --2- 172.21.15.114:0/2185174158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5590268627d0 0x559026862bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:57.582 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.578+0000 7f85ec824740 1 -- 172.21.15.114:0/2185174158 >> 172.21.15.114:0/2185174158 conn(0x55902685ed60 msgr2=0x55902685f140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:57.583 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.580+0000 7f85ec824740 1 -- 172.21.15.114:0/2185174158 shutdown_connections 2024-09-13T18:29:57.586 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.583+0000 7f85ec824740 1 -- 172.21.15.114:0/2185174158 wait complete. 2024-09-13T18:29:57.587 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.584+0000 7f85ec824740 1 -- start start 2024-09-13T18:29:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.584+0000 7f85ec824740 1 Processor -- start 2024-09-13T18:29:57.590 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.586+0000 7f85ec824740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590268627d0 0x55902686d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:57.590 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.586+0000 7f85ec824740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5590268630f0 0x55902686a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.586+0000 7f85ec824740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55902686d990 0x55902686a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.586+0000 7f85ec824740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x559026784e80 con 0x5590268630f0 2024-09-13T18:29:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.586+0000 7f85ec824740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559026784b20 con 0x5590268627d0 2024-09-13T18:29:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.586+0000 7f85ec824740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55902674a260 con 0x55902686d990 2024-09-13T18:29:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.586+0000 7f85ec824740 10 client.0 _ll_register_callbacks cb 0x55902677fdf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:29:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.588+0000 7f85e3fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55902686d990 0x55902686a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:57.594 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.591+0000 7f85e2ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5590268630f0 0x55902686a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:57.594 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.591+0000 7f85e2ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5590268630f0 0x55902686a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43970/0 (socket says 172.21.15.114:43970) 2024-09-13T18:29:57.594 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.591+0000 7f85e2ffd640 1 -- 172.21.15.114:0/2521615065 learned_addr learned my addr 172.21.15.114:0/2521615065 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:57.594 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.591+0000 7f85e2ffd640 1 --2- 172.21.15.114:0/4260765693 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f85cc0a1550 0x7f85cc0a3a10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x5590268600e0 tx=0x7f85d4006ad0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.591+0000 7f85e2ffd640 1 -- 172.21.15.114:0/2521615065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55902686d990 msgr2=0x55902686a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.591+0000 7f85e2ffd640 1 --2- 172.21.15.114:0/2521615065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55902686d990 0x55902686a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.591+0000 7f85e2ffd640 1 -- 172.21.15.114:0/2521615065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590268627d0 msgr2=0x55902686d190 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:29:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.591+0000 7f85e2ffd640 1 --2- 172.21.15.114:0/2521615065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590268627d0 0x55902686d190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.591+0000 7f85e2ffd640 1 -- 172.21.15.114:0/2521615065 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85e410d880 con 0x5590268630f0 2024-09-13T18:29:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.592+0000 7f85e2ffd640 1 --2- 172.21.15.114:0/2521615065 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5590268630f0 0x55902686a430 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f85d40186f0 tx=0x7f85d4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31af2382fbcf106 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.592+0000 7f85c77f6640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:29:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.593+0000 7f85c77f6640 1 -- 172.21.15.114:0/2521615065 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d40225f0 con 0x5590268630f0 2024-09-13T18:29:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.593+0000 7f85ec824740 1 -- 172.21.15.114:0/2521615065 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5590267ef680 con 0x5590268630f0 2024-09-13T18:29:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.593+0000 7f85c77f6640 1 -- 172.21.15.114:0/2521615065 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f85d4022750 con 0x5590268630f0 2024-09-13T18:29:57.597 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.593+0000 7f85c77f6640 1 -- 172.21.15.114:0/2521615065 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85d4022950 con 0x5590268630f0 2024-09-13T18:29:57.597 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.594+0000 7f85ec824740 10 client.17703 fetch_fsmap learned FSMap version 34 2024-09-13T18:29:57.597 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.594+0000 7f85ec824740 1 -- 172.21.15.114:0/2521615065 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5590267c5330 con 0x5590268630f0 2024-09-13T18:29:57.600 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.597+0000 7f85c77f6640 1 -- 172.21.15.114:0/2521615065 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5590267ef680 con 0x5590268630f0 2024-09-13T18:29:57.600 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.597+0000 7f85c77f6640 1 -- 172.21.15.114:0/2521615065 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f85d4021070 con 0x5590268630f0 2024-09-13T18:29:57.601 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.598+0000 7f85ec824740 10 client.17703 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:29:57.601 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.598+0000 7f85ec824740 10 client.17703 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:29:57.601 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.598+0000 7f85ec824740 20 client.17703 populate_metadata read hostname 'smithi114' 2024-09-13T18:29:57.601 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.598+0000 7f85ec824740 1 --2- 172.21.15.114:0/2521615065 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55902687d2e0 0x55902687f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:57.601 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.598+0000 7f85ec824740 4 client.17703 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:29:57.602 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.598+0000 7f85ec824740 1 -- 172.21.15.114:0/2521615065 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5590267ba5d0 con 0x55902687d2e0 2024-09-13T18:29:57.604 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.601+0000 7f85e37fe640 1 --2- 172.21.15.114:0/2521615065 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55902687d2e0 0x55902687f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.602+0000 7f85e37fe640 1 --2- 172.21.15.114:0/2521615065 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55902687d2e0 0x55902687f700 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9185797b5df891d server_cookie=367d21a9c86904e in_seq=0 out_seq=0 2024-09-13T18:29:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.605+0000 7f85c77f6640 10 client.17703 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:29:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.605+0000 7f85c77f6640 1 -- 172.21.15.114:0/2521615065 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5590267ba5d0 con 0x55902687d2e0 2024-09-13T18:29:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.605+0000 7f85c77f6640 10 client.17703 handle_command_reply: tid=0 2024-09-13T18:29:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.605+0000 7f85ec824740 1 client.17703 shutdown 2024-09-13T18:29:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.613+0000 7f85ec824740 1 -- 172.21.15.114:0/2521615065 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5590268630f0 msgr2=0x55902686a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.613+0000 7f85ec824740 1 --2- 172.21.15.114:0/2521615065 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5590268630f0 0x55902686a430 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f85d40186f0 tx=0x7f85d4004770 comp rx=0 tx=0).stop 2024-09-13T18:29:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.613+0000 7f85ec824740 1 -- 172.21.15.114:0/2521615065 shutdown_connections 2024-09-13T18:29:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.613+0000 7f85ec824740 1 --2- 172.21.15.114:0/2521615065 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55902687d2e0 0x55902687f700 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.613+0000 7f85ec824740 1 --2- 172.21.15.114:0/2521615065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55902686d990 0x55902686a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.613+0000 7f85ec824740 1 --2- 172.21.15.114:0/2521615065 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5590268630f0 0x55902686a430 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.613+0000 7f85ec824740 1 --2- 172.21.15.114:0/2521615065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590268627d0 0x55902686d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:57.616 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.613+0000 7f85ec824740 1 -- 172.21.15.114:0/2521615065 >> 172.21.15.114:0/2521615065 conn(0x55902685ed60 msgr2=0x55902685df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.614+0000 7f85c77f6640 0 client.17703 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:29:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.616+0000 7f85ec824740 1 -- 172.21.15.114:0/2521615065 shutdown_connections 2024-09-13T18:29:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.622+0000 7f85ec824740 1 -- 172.21.15.114:0/2521615065 wait complete. 2024-09-13T18:29:57.630 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.627+0000 7f85ec824740 20 client.17703 trim_cache size 0 max 16384 2024-09-13T18:29:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.659+0000 7f85ec824740 1 Processor -- start 2024-09-13T18:29:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.659+0000 7f85ec824740 1 -- start start 2024-09-13T18:29:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.659+0000 7f85ec824740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590268627d0 0x5590268745f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.659+0000 7f85ec824740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55902686d990 0x559026874b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.659+0000 7f85ec824740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55902685a370 0x5590268658b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.660+0000 7f85ec824740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590267ba5d0 con 0x55902685a370 2024-09-13T18:29:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.660+0000 7f85ec824740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590267ef680 con 0x5590268627d0 2024-09-13T18:29:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.660+0000 7f85ec824740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559026863740 con 0x55902686d990 2024-09-13T18:29:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.660+0000 7f85e37fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590268627d0 0x5590268745f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.660+0000 7f85e3fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55902685a370 0x5590268658b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.660+0000 7f85e37fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590268627d0 0x5590268745f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:43408/0 (socket says 172.21.15.114:43408) 2024-09-13T18:29:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.660+0000 7f85e37fe640 1 -- 172.21.15.114:0/659481108 learned_addr learned my addr 172.21.15.114:0/659481108 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.660+0000 7f85e2ffd640 1 --2- 172.21.15.114:0/659481108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55902686d990 0x559026874b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.660+0000 7f85e37fe640 1 -- 172.21.15.114:0/659481108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55902686d990 msgr2=0x559026874b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.660+0000 7f85e37fe640 1 --2- 172.21.15.114:0/659481108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55902686d990 0x559026874b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.660+0000 7f85e37fe640 1 -- 172.21.15.114:0/659481108 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55902685a370 msgr2=0x5590268658b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.660+0000 7f85e37fe640 1 --2- 172.21.15.114:0/659481108 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55902685a370 0x5590268658b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.660+0000 7f85e37fe640 1 -- 172.21.15.114:0/659481108 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559026767300 con 0x5590268627d0 2024-09-13T18:29:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.661+0000 7f85e2ffd640 1 --2- 172.21.15.114:0/659481108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55902686d990 0x559026874b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:29:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.662+0000 7f85e37fe640 1 --2- 172.21.15.114:0/659481108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590268627d0 0x5590268745f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55902685e860 tx=0x7f85dc02a8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ed71803bb78c782 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.665+0000 7f85c9ffb640 1 -- 172.21.15.114:0/659481108 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85dc049040 con 0x5590268627d0 2024-09-13T18:29:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.665+0000 7f85c9ffb640 1 -- 172.21.15.114:0/659481108 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f85dc031480 con 0x5590268627d0 2024-09-13T18:29:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.666+0000 7f85ec824740 1 -- 172.21.15.114:0/659481108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590268627d0 msgr2=0x5590268745f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.666+0000 7f85ec824740 1 --2- 172.21.15.114:0/659481108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590268627d0 0x5590268745f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55902685e860 tx=0x7f85dc02a8f0 comp rx=0 tx=0).stop 2024-09-13T18:29:57.674 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.671+0000 7f85ec824740 1 -- 172.21.15.114:0/659481108 shutdown_connections 2024-09-13T18:29:57.674 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.671+0000 7f85ec824740 1 --2- 172.21.15.114:0/659481108 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55902685a370 0x5590268658b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:57.674 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.671+0000 7f85ec824740 1 --2- 172.21.15.114:0/659481108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55902686d990 0x559026874b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:57.674 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.671+0000 7f85ec824740 1 --2- 172.21.15.114:0/659481108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590268627d0 0x5590268745f0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:57.674 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.671+0000 7f85ec824740 1 -- 172.21.15.114:0/659481108 >> 172.21.15.114:0/659481108 conn(0x55902685ed60 msgr2=0x559026859c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:57.675 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.671+0000 7f85ec824740 1 -- 172.21.15.114:0/659481108 shutdown_connections 2024-09-13T18:29:57.675 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.672+0000 7f85ec824740 1 -- 172.21.15.114:0/659481108 wait complete. 2024-09-13T18:29:57.675 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.672+0000 7f85ec824740 1 -- start start 2024-09-13T18:29:57.675 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.672+0000 7f85ec824740 1 Processor -- start 2024-09-13T18:29:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.672+0000 7f85ec824740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55902685a370 0x5590268748d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.673+0000 7f85ec824740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5590268627d0 0x559026870c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.673+0000 7f85ec824740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55902686d990 0x559026871190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.673+0000 7f85ec824740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x559026871890 con 0x55902686d990 2024-09-13T18:29:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.673+0000 7f85ec824740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559026871a00 con 0x55902685a370 2024-09-13T18:29:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.673+0000 7f85ec824740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590268d37e0 con 0x5590268627d0 2024-09-13T18:29:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.673+0000 7f85ec824740 10 client.0 _ll_register_callbacks cb 0x7f85dc009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:29:57.677 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.673+0000 7f85e37fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55902685a370 0x5590268748d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:57.677 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.673+0000 7f85e37fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55902685a370 0x5590268748d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:43418/0 (socket says 172.21.15.114:43418) 2024-09-13T18:29:57.677 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.673+0000 7f85e37fe640 1 -- 172.21.15.114:0/3394456246 learned_addr learned my addr 172.21.15.114:0/3394456246 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:57.677 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.673+0000 7f85e2ffd640 1 --2- 172.21.15.114:0/3394456246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5590268627d0 0x559026870c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:57.677 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.674+0000 7f85e3fff640 1 --2- 172.21.15.114:0/3394456246 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55902686d990 0x559026871190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:57.677 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.674+0000 7f85e37fe640 1 -- 172.21.15.114:0/3394456246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5590268627d0 msgr2=0x559026870c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.674+0000 7f85e37fe640 1 --2- 172.21.15.114:0/3394456246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5590268627d0 0x559026870c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.674+0000 7f85e37fe640 1 -- 172.21.15.114:0/3394456246 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55902686d990 msgr2=0x559026871190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.674+0000 7f85e37fe640 1 --2- 172.21.15.114:0/3394456246 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55902686d990 0x559026871190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.674+0000 7f85e37fe640 1 -- 172.21.15.114:0/3394456246 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55902676d910 con 0x55902685a370 2024-09-13T18:29:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.674+0000 7f85e37fe640 1 --2- 172.21.15.114:0/3394456246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55902685a370 0x5590268748d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f85dc02ae30 tx=0x7f85dc051190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=246d2636621b5645 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.674+0000 7f85c6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:29:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.675+0000 7f85c6ff5640 1 -- 172.21.15.114:0/3394456246 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85dc049040 con 0x55902685a370 2024-09-13T18:29:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.675+0000 7f85ec824740 1 -- 172.21.15.114:0/3394456246 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f85dc030b50 con 0x55902685a370 2024-09-13T18:29:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.675+0000 7f85c6ff5640 1 -- 172.21.15.114:0/3394456246 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f85dc051490 con 0x55902685a370 2024-09-13T18:29:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.675+0000 7f85c6ff5640 1 -- 172.21.15.114:0/3394456246 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85dc051650 con 0x55902685a370 2024-09-13T18:29:57.680 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.677+0000 7f85ec824740 10 client.27122 fetch_fsmap learned FSMap version 34 2024-09-13T18:29:57.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.677+0000 7f85ec824740 1 -- 172.21.15.114:0/3394456246 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55902678c8e0 con 0x55902685a370 2024-09-13T18:29:57.682 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.679+0000 7f85c6ff5640 1 -- 172.21.15.114:0/3394456246 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f85dc030b50 con 0x55902685a370 2024-09-13T18:29:57.682 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.679+0000 7f85c6ff5640 1 -- 172.21.15.114:0/3394456246 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f85dc004240 con 0x55902685a370 2024-09-13T18:29:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.683+0000 7f85ec824740 10 client.27122 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:29:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.683+0000 7f85ec824740 10 client.27122 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:29:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.683+0000 7f85ec824740 20 client.27122 populate_metadata read hostname 'smithi114' 2024-09-13T18:29:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.683+0000 7f85ec824740 1 --2- 172.21.15.114:0/3394456246 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5590268d48e0 0x5590268d6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.683+0000 7f85ec824740 4 client.27122 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T18:29:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.683+0000 7f85ec824740 1 -- 172.21.15.114:0/3394456246 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5590264ba300 con 0x5590268d48e0 2024-09-13T18:29:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.683+0000 7f85e2ffd640 1 --2- 172.21.15.114:0/3394456246 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5590268d48e0 0x5590268d6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:57.690 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.687+0000 7f85e2ffd640 1 --2- 172.21.15.114:0/3394456246 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5590268d48e0 0x5590268d6cd0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0e679bb12eca3b3 server_cookie=780afd7b52e53b94 in_seq=0 out_seq=0 2024-09-13T18:29:57.690 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.687+0000 7f85c6ff5640 10 client.27122 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:29:57.712 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.708+0000 7f85c6ff5640 1 -- 172.21.15.114:0/3394456246 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5590264ba300 con 0x5590268d48e0 2024-09-13T18:29:57.712 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.709+0000 7f85c6ff5640 10 client.27122 handle_command_reply: tid=0 2024-09-13T18:29:57.712 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.709+0000 7f85ec824740 1 client.27122 shutdown 2024-09-13T18:29:57.718 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.713+0000 7f85ec824740 1 -- 172.21.15.114:0/3394456246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55902685a370 msgr2=0x5590268748d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:57.718 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.713+0000 7f85ec824740 1 --2- 172.21.15.114:0/3394456246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55902685a370 0x5590268748d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f85dc02ae30 tx=0x7f85dc051190 comp rx=0 tx=0).stop 2024-09-13T18:29:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.716+0000 7f85ec824740 1 -- 172.21.15.114:0/3394456246 shutdown_connections 2024-09-13T18:29:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.716+0000 7f85ec824740 1 --2- 172.21.15.114:0/3394456246 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5590268d48e0 0x5590268d6cd0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.716+0000 7f85ec824740 1 --2- 172.21.15.114:0/3394456246 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55902686d990 0x559026871190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.716+0000 7f85ec824740 1 --2- 172.21.15.114:0/3394456246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5590268627d0 0x559026870c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.716+0000 7f85ec824740 1 --2- 172.21.15.114:0/3394456246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55902685a370 0x5590268748d0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.716+0000 7f85ec824740 1 -- 172.21.15.114:0/3394456246 >> 172.21.15.114:0/3394456246 conn(0x55902685ed60 msgr2=0x5590268658b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:57.719 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.716+0000 7f85c6ff5640 0 client.27122 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:29:57.727 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.721+0000 7f85ec824740 1 -- 172.21.15.114:0/3394456246 shutdown_connections 2024-09-13T18:29:57.727 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.724+0000 7f85ec824740 1 -- 172.21.15.114:0/3394456246 wait complete. 2024-09-13T18:29:57.731 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.727+0000 7f85ec824740 20 client.27122 trim_cache size 0 max 16384 2024-09-13T18:29:57.733 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-13T18:29:57.733 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-13T18:29:57.734 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-13T18:29:57.734 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-13T18:29:57.738 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.735+0000 7f85ca7fc640 1 -- 172.21.15.114:0/4260765693 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f85cc0a1550 msgr2=0x7f85cc0a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:57.738 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.735+0000 7f85ca7fc640 1 --2- 172.21.15.114:0/4260765693 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f85cc0a1550 0x7f85cc0a3a10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x5590268600e0 tx=0x7f85d4006ad0 comp rx=0 tx=0).stop 2024-09-13T18:29:57.738 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.735+0000 7f85ca7fc640 1 -- 172.21.15.114:0/4260765693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85e410b600 msgr2=0x7f85e4117720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:57.738 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.735+0000 7f85ca7fc640 1 --2- 172.21.15.114:0/4260765693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85e410b600 0x7f85e4117720 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f85d800b3f0 tx=0x7f85d800b8b0 comp rx=0 tx=0).stop 2024-09-13T18:29:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.738+0000 7f85ca7fc640 1 -- 172.21.15.114:0/4260765693 shutdown_connections 2024-09-13T18:29:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.738+0000 7f85ca7fc640 1 --2- 172.21.15.114:0/4260765693 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f85cc0a1550 0x7f85cc0a3a10 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.738+0000 7f85ca7fc640 1 --2- 172.21.15.114:0/4260765693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85e410b600 0x7f85e4117720 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.738+0000 7f85ca7fc640 1 --2- 172.21.15.114:0/4260765693 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85e4071c30 0x7f85e41171e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.738+0000 7f85ca7fc640 1 --2- 172.21.15.114:0/4260765693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85e40717c0 0x7f85e4116ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.738+0000 7f85ca7fc640 1 -- 172.21.15.114:0/4260765693 >> 172.21.15.114:0/4260765693 conn(0x7f85e406ae30 msgr2=0x7f85e410a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.741+0000 7f85ca7fc640 1 -- 172.21.15.114:0/4260765693 shutdown_connections 2024-09-13T18:29:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:29:57.741+0000 7f85ca7fc640 1 -- 172.21.15.114:0/4260765693 wait complete. 2024-09-13T18:29:58.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:29:57 smithi134 ceph-mon[34283]: pgmap v229: 129 pgs: 129 active+clean; 22 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 39 MiB/s wr, 852 op/s 2024-09-13T18:29:58.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:29:57 smithi204 ceph-mon[34777]: pgmap v229: 129 pgs: 129 active+clean; 22 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 39 MiB/s wr, 852 op/s 2024-09-13T18:29:58.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:29:57 smithi114 ceph-mon[30897]: pgmap v229: 129 pgs: 129 active+clean; 22 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 39 MiB/s wr, 852 op/s 2024-09-13T18:29:58.745 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T18:29:58.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.974+0000 7fa3fbfff640 1 -- 172.21.15.114:0/839619136 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4040937b0 con 0x7fa3fc0b7510 2024-09-13T18:29:58.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.975+0000 7fa40a7a7640 1 -- 172.21.15.114:0/839619136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3fc0b7510 msgr2=0x7fa3fc0b9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:58.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.975+0000 7fa40a7a7640 1 --2- 172.21.15.114:0/839619136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3fc0b7510 0x7fa3fc0b9900 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa404063b50 tx=0x7fa40408c690 comp rx=0 tx=0).stop 2024-09-13T18:29:58.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.975+0000 7fa40a7a7640 1 -- 172.21.15.114:0/839619136 shutdown_connections 2024-09-13T18:29:58.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.975+0000 7fa40a7a7640 1 --2- 172.21.15.114:0/839619136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3fc0b7510 0x7fa3fc0b9900 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:58.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.975+0000 7fa40a7a7640 1 --2- 172.21.15.114:0/839619136 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3fc0a5110 0x7fa3fc0b6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:58.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.975+0000 7fa40a7a7640 1 --2- 172.21.15.114:0/839619136 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3fc0a47f0 0x7fa3fc0a4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:58.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.975+0000 7fa40a7a7640 1 -- 172.21.15.114:0/839619136 >> 172.21.15.114:0/839619136 conn(0x7fa3fc01a0e0 msgr2=0x7fa3fc01a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:58.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.976+0000 7fa40a7a7640 1 -- 172.21.15.114:0/839619136 shutdown_connections 2024-09-13T18:29:58.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.976+0000 7fa40a7a7640 1 -- 172.21.15.114:0/839619136 wait complete. 2024-09-13T18:29:58.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.977+0000 7fa40a7a7640 1 Processor -- start 2024-09-13T18:29:58.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.977+0000 7fa40a7a7640 1 -- start start 2024-09-13T18:29:58.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.977+0000 7fa40a7a7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3fc0a47f0 0x7fa3fc1430b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:58.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.977+0000 7fa40a7a7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3fc0a5110 0x7fa3fc1435f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:58.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.977+0000 7fa40a7a7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3fc0b7510 0x7fa3fc1479e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:58.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.977+0000 7fa40a7a7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3fc147fe0 con 0x7fa3fc0a5110 2024-09-13T18:29:58.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.977+0000 7fa40a7a7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3fc148150 con 0x7fa3fc0a47f0 2024-09-13T18:29:58.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.977+0000 7fa40a7a7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3fc148310 con 0x7fa3fc0b7510 2024-09-13T18:29:58.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.977+0000 7fa409fa6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3fc0b7510 0x7fa3fc1479e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:58.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.977+0000 7fa409fa6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3fc0b7510 0x7fa3fc1479e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:57946/0 (socket says 172.21.15.114:57946) 2024-09-13T18:29:58.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.977+0000 7fa409fa6640 1 -- 172.21.15.114:0/2673109246 learned_addr learned my addr 172.21.15.114:0/2673109246 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:58.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.978+0000 7fa409fa6640 1 -- 172.21.15.114:0/2673109246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3fc0a47f0 msgr2=0x7fa3fc1430b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:29:58.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.978+0000 7fa409fa6640 1 --2- 172.21.15.114:0/2673109246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3fc0a47f0 0x7fa3fc1430b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:58.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.978+0000 7fa409fa6640 1 -- 172.21.15.114:0/2673109246 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3fc0a5110 msgr2=0x7fa3fc1435f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:58.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.978+0000 7fa409fa6640 1 --2- 172.21.15.114:0/2673109246 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3fc0a5110 0x7fa3fc1435f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:58.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.978+0000 7fa409fa6640 1 -- 172.21.15.114:0/2673109246 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa40404eac0 con 0x7fa3fc0b7510 2024-09-13T18:29:58.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.978+0000 7fa409fa6640 1 --2- 172.21.15.114:0/2673109246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3fc0b7510 0x7fa3fc1479e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa404000b60 tx=0x7fa404068060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:58.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.979+0000 7fa3fa7fc640 1 -- 172.21.15.114:0/2673109246 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa404068720 con 0x7fa3fc0b7510 2024-09-13T18:29:58.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.979+0000 7fa3fa7fc640 1 -- 172.21.15.114:0/2673109246 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa4040948c0 con 0x7fa3fc0b7510 2024-09-13T18:29:58.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.979+0000 7fa3fa7fc640 1 -- 172.21.15.114:0/2673109246 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40409f910 con 0x7fa3fc0b7510 2024-09-13T18:29:58.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.980+0000 7fa40a7a7640 1 -- 172.21.15.114:0/2673109246 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3fc148590 con 0x7fa3fc0b7510 2024-09-13T18:29:58.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.980+0000 7fa40a7a7640 1 -- 172.21.15.114:0/2673109246 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3fc00b810 con 0x7fa3fc0b7510 2024-09-13T18:29:58.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.985+0000 7fa40e808740 1 Processor -- start 2024-09-13T18:29:58.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.985+0000 7fa3fa7fc640 1 -- 172.21.15.114:0/2673109246 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fa404091030 con 0x7fa3fc0b7510 2024-09-13T18:29:58.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.985+0000 7fa40e808740 1 -- start start 2024-09-13T18:29:58.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.986+0000 7fa40e808740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff4f4befe0 0x55ff4f4bf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:58.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.986+0000 7fa40e808740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff4f4bf900 0x55ff4f4c9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:58.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.986+0000 7fa40e808740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff4f4ca1a0 0x55ff4f4cc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:58.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.986+0000 7fa40e808740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff4f429f20 con 0x55ff4f4bf900 2024-09-13T18:29:58.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.986+0000 7fa40e808740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff4f4276b0 con 0x55ff4f4ca1a0 2024-09-13T18:29:58.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.986+0000 7fa40e808740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff4f422fa0 con 0x55ff4f4befe0 2024-09-13T18:29:58.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.986+0000 7fa408fa4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff4f4befe0 0x55ff4f4bf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:58.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.986+0000 7fa4097a5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff4f4bf900 0x55ff4f4c9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:58.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.986+0000 7fa408fa4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff4f4befe0 0x55ff4f4bf3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:57962/0 (socket says 172.21.15.114:57962) 2024-09-13T18:29:58.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.986+0000 7fa408fa4640 1 -- 172.21.15.114:0/1003357372 learned_addr learned my addr 172.21.15.114:0/1003357372 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:58.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.986+0000 7fa408fa4640 1 --2- 172.21.15.114:0/1003357372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff4f4ca1a0 0x55ff4f4cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:58.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.986+0000 7fa408fa4640 1 -- 172.21.15.114:0/1003357372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff4f4ca1a0 msgr2=0x55ff4f4cc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:58.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.987+0000 7fa408fa4640 1 --2- 172.21.15.114:0/1003357372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff4f4ca1a0 0x55ff4f4cc590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:58.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.987+0000 7fa408fa4640 1 -- 172.21.15.114:0/1003357372 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff4f4bf900 msgr2=0x55ff4f4c9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:58.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.987+0000 7fa408fa4640 1 --2- 172.21.15.114:0/1003357372 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff4f4bf900 0x55ff4f4c9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:58.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.987+0000 7fa408fa4640 1 -- 172.21.15.114:0/1003357372 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff4f3dd840 con 0x55ff4f4befe0 2024-09-13T18:29:58.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.987+0000 7fa3fa7fc640 1 --2- 172.21.15.114:0/2673109246 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa3f00a1350 0x7fa3f00a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:58.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.987+0000 7fa408fa4640 1 --2- 172.21.15.114:0/1003357372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff4f4befe0 0x55ff4f4bf3c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa3f4012790 tx=0x7fa3f400d3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7f74ab378b8e067 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:58.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.987+0000 7fa3fa7fc640 1 -- 172.21.15.114:0/2673109246 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fa40414f5c0 con 0x7fa3fc0b7510 2024-09-13T18:29:58.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.987+0000 7fa4097a5640 1 --2- 172.21.15.114:0/2673109246 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa3f00a1350 0x7fa3f00a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:58.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.987+0000 7fa3ef7fe640 1 -- 172.21.15.114:0/1003357372 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f4004040 con 0x55ff4f4befe0 2024-09-13T18:29:58.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.987+0000 7fa3ef7fe640 1 -- 172.21.15.114:0/1003357372 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa3f4018020 con 0x55ff4f4befe0 2024-09-13T18:29:58.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.987+0000 7fa3ef7fe640 1 -- 172.21.15.114:0/1003357372 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f40035f0 con 0x55ff4f4befe0 2024-09-13T18:29:58.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.988+0000 7fa40e808740 1 -- 172.21.15.114:0/1003357372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff4f4befe0 msgr2=0x55ff4f4bf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:58.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.988+0000 7fa40e808740 1 --2- 172.21.15.114:0/1003357372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff4f4befe0 0x55ff4f4bf3c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa3f4012790 tx=0x7fa3f400d3c0 comp rx=0 tx=0).stop 2024-09-13T18:29:58.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.989+0000 7fa40e808740 1 -- 172.21.15.114:0/1003357372 shutdown_connections 2024-09-13T18:29:58.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.989+0000 7fa40e808740 1 --2- 172.21.15.114:0/1003357372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff4f4ca1a0 0x55ff4f4cc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:58.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.989+0000 7fa40e808740 1 --2- 172.21.15.114:0/1003357372 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff4f4bf900 0x55ff4f4c9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:58.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.989+0000 7fa40e808740 1 --2- 172.21.15.114:0/1003357372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff4f4befe0 0x55ff4f4bf3c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:58.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.989+0000 7fa40e808740 1 -- 172.21.15.114:0/1003357372 >> 172.21.15.114:0/1003357372 conn(0x55ff4f4bb570 msgr2=0x55ff4f4bb950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:58.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.989+0000 7fa4097a5640 1 --2- 172.21.15.114:0/2673109246 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa3f00a1350 0x7fa3f00a3810 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x55ff4f4bb070 tx=0x7fa400009060 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:58.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.991+0000 7fa40e808740 1 -- 172.21.15.114:0/1003357372 shutdown_connections 2024-09-13T18:29:58.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.991+0000 7fa40e808740 1 -- 172.21.15.114:0/1003357372 wait complete. 2024-09-13T18:29:58.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.991+0000 7fa40e808740 1 -- start start 2024-09-13T18:29:58.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.991+0000 7fa40e808740 1 Processor -- start 2024-09-13T18:29:58.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.992+0000 7fa40e808740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff4f4befe0 0x55ff4f4c9b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:58.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.992+0000 7fa40e808740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff4f4bf900 0x55ff4f4c6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:58.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.992+0000 7fa40e808740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff4f4ca1a0 0x55ff4f4c74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:58.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.992+0000 7fa40e808740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff4f3c42a0 con 0x55ff4f4bf900 2024-09-13T18:29:58.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.992+0000 7fa40e808740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff4f0fb430 con 0x55ff4f4ca1a0 2024-09-13T18:29:58.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.992+0000 7fa40e808740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff4f3a6680 con 0x55ff4f4befe0 2024-09-13T18:29:58.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.992+0000 7fa40e808740 10 client.0 _ll_register_callbacks cb 0x55ff4f41c610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:29:58.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.992+0000 7fa408fa4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff4f4bf900 0x55ff4f4c6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:58.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.992+0000 7fa408fa4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff4f4bf900 0x55ff4f4c6f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44014/0 (socket says 172.21.15.114:44014) 2024-09-13T18:29:58.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.992+0000 7fa408fa4640 1 -- 172.21.15.114:0/171811247 learned_addr learned my addr 172.21.15.114:0/171811247 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:58.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.993+0000 7fa409fa6640 1 --2- 172.21.15.114:0/171811247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff4f4ca1a0 0x55ff4f4c74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:58.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.993+0000 7fa4097a5640 1 --2- 172.21.15.114:0/171811247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff4f4befe0 0x55ff4f4c9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:58.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.993+0000 7fa408fa4640 1 -- 172.21.15.114:0/171811247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff4f4befe0 msgr2=0x55ff4f4c9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:58.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.993+0000 7fa408fa4640 1 --2- 172.21.15.114:0/171811247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff4f4befe0 0x55ff4f4c9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:58.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.993+0000 7fa408fa4640 1 -- 172.21.15.114:0/171811247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff4f4ca1a0 msgr2=0x55ff4f4c74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:58.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.993+0000 7fa408fa4640 1 --2- 172.21.15.114:0/171811247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff4f4ca1a0 0x55ff4f4c74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:58.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.993+0000 7fa408fa4640 1 -- 172.21.15.114:0/171811247 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3f4012470 con 0x55ff4f4bf900 2024-09-13T18:29:58.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.993+0000 7fa408fa4640 1 --2- 172.21.15.114:0/171811247 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff4f4bf900 0x55ff4f4c6f10 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fa3f400dbc0 tx=0x7fa3f4005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe1aff0944536c08 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:58.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.993+0000 7fa3ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:29:58.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.994+0000 7fa3ecff9640 1 -- 172.21.15.114:0/171811247 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f4012e50 con 0x55ff4f4bf900 2024-09-13T18:29:58.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.994+0000 7fa40e808740 1 -- 172.21.15.114:0/171811247 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ff4f1164e0 con 0x55ff4f4bf900 2024-09-13T18:29:58.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.994+0000 7fa3ecff9640 1 -- 172.21.15.114:0/171811247 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fa3f4019030 con 0x55ff4f4bf900 2024-09-13T18:29:58.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.994+0000 7fa3ecff9640 1 -- 172.21.15.114:0/171811247 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3f400a8c0 con 0x55ff4f4bf900 2024-09-13T18:29:58.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.994+0000 7fa40e808740 10 client.17715 fetch_fsmap learned FSMap version 34 2024-09-13T18:29:58.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.994+0000 7fa40e808740 1 -- 172.21.15.114:0/171811247 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55ff4f421620 con 0x55ff4f4bf900 2024-09-13T18:29:58.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.995+0000 7fa3ecff9640 1 -- 172.21.15.114:0/171811247 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ff4f1164e0 con 0x55ff4f4bf900 2024-09-13T18:29:58.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.995+0000 7fa3ecff9640 1 -- 172.21.15.114:0/171811247 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa3f4003b80 con 0x55ff4f4bf900 2024-09-13T18:29:58.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.995+0000 7fa40e808740 10 client.17715 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:29:58.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.995+0000 7fa40e808740 10 client.17715 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:29:58.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.995+0000 7fa40e808740 20 client.17715 populate_metadata read hostname 'smithi114' 2024-09-13T18:29:59.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.995+0000 7fa40e808740 1 --2- 172.21.15.114:0/171811247 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55ff4f4cfd90 0x55ff4f4d21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:59.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.995+0000 7fa40e808740 4 client.17715 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:29:59.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.995+0000 7fa40e808740 1 -- 172.21.15.114:0/171811247 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ff4f1277e0 con 0x55ff4f4cfd90 2024-09-13T18:29:59.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.995+0000 7fa4097a5640 1 --2- 172.21.15.114:0/171811247 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55ff4f4cfd90 0x55ff4f4d21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:59.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.996+0000 7fa4097a5640 1 --2- 172.21.15.114:0/171811247 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55ff4f4cfd90 0x55ff4f4d21b0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e0918bbd649bace server_cookie=736807ba98086a08 in_seq=0 out_seq=0 2024-09-13T18:29:59.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.998+0000 7fa3ecff9640 10 client.17715 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:29:59.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.998+0000 7fa3ecff9640 1 -- 172.21.15.114:0/171811247 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55ff4f1277e0 con 0x55ff4f4cfd90 2024-09-13T18:29:59.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.998+0000 7fa3ecff9640 10 client.17715 handle_command_reply: tid=0 2024-09-13T18:29:59.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:58.998+0000 7fa40e808740 1 client.17715 shutdown 2024-09-13T18:29:59.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.003+0000 7fa40e808740 1 -- 172.21.15.114:0/171811247 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff4f4bf900 msgr2=0x55ff4f4c6f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:59.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.003+0000 7fa40e808740 1 --2- 172.21.15.114:0/171811247 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff4f4bf900 0x55ff4f4c6f10 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fa3f400dbc0 tx=0x7fa3f4005c50 comp rx=0 tx=0).stop 2024-09-13T18:29:59.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.007+0000 7fa40e808740 1 -- 172.21.15.114:0/171811247 shutdown_connections 2024-09-13T18:29:59.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.007+0000 7fa40e808740 1 --2- 172.21.15.114:0/171811247 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55ff4f4cfd90 0x55ff4f4d21b0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:59.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.007+0000 7fa40e808740 1 --2- 172.21.15.114:0/171811247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff4f4ca1a0 0x55ff4f4c74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:59.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.007+0000 7fa40e808740 1 --2- 172.21.15.114:0/171811247 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff4f4bf900 0x55ff4f4c6f10 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:59.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.007+0000 7fa40e808740 1 --2- 172.21.15.114:0/171811247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff4f4befe0 0x55ff4f4c9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:59.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.007+0000 7fa40e808740 1 -- 172.21.15.114:0/171811247 >> 172.21.15.114:0/171811247 conn(0x55ff4f4bb570 msgr2=0x55ff4f4ba760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:59.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.007+0000 7fa3ecff9640 0 client.17715 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:29:59.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.007+0000 7fa40e808740 1 -- 172.21.15.114:0/171811247 shutdown_connections 2024-09-13T18:29:59.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.007+0000 7fa40e808740 1 -- 172.21.15.114:0/171811247 wait complete. 2024-09-13T18:29:59.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.009+0000 7fa40e808740 20 client.17715 trim_cache size 0 max 16384 2024-09-13T18:29:59.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.047+0000 7fa40e808740 1 Processor -- start 2024-09-13T18:29:59.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.050+0000 7fa40e808740 1 -- start start 2024-09-13T18:29:59.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.050+0000 7fa40e808740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff4f4befe0 0x55ff4f4c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:59.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.050+0000 7fa40e808740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff4f4bf900 0x55ff4f4c2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:59.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.050+0000 7fa40e808740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff4f4ca1a0 0x55ff4f4c26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:59.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.050+0000 7fa40e808740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff4f4ccbc0 con 0x55ff4f4bf900 2024-09-13T18:29:59.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.050+0000 7fa40e808740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff4f4b58b0 con 0x55ff4f4befe0 2024-09-13T18:29:59.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.050+0000 7fa40e808740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff4f4e7620 con 0x55ff4f4ca1a0 2024-09-13T18:29:59.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.051+0000 7fa409fa6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff4f4ca1a0 0x55ff4f4c26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:59.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.051+0000 7fa409fa6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff4f4ca1a0 0x55ff4f4c26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:57970/0 (socket says 172.21.15.114:57970) 2024-09-13T18:29:59.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.051+0000 7fa409fa6640 1 -- 172.21.15.114:0/2628361985 learned_addr learned my addr 172.21.15.114:0/2628361985 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:59.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.051+0000 7fa408fa4640 1 --2- 172.21.15.114:0/2628361985 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff4f4bf900 0x55ff4f4c2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:59.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.051+0000 7fa408fa4640 1 -- 172.21.15.114:0/2628361985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff4f4ca1a0 msgr2=0x55ff4f4c26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:59.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.051+0000 7fa408fa4640 1 --2- 172.21.15.114:0/2628361985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff4f4ca1a0 0x55ff4f4c26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:59.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.051+0000 7fa408fa4640 1 -- 172.21.15.114:0/2628361985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff4f4befe0 msgr2=0x55ff4f4c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:59.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.051+0000 7fa408fa4640 1 --2- 172.21.15.114:0/2628361985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff4f4befe0 0x55ff4f4c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:59.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.051+0000 7fa408fa4640 1 -- 172.21.15.114:0/2628361985 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff4f3c3790 con 0x55ff4f4bf900 2024-09-13T18:29:59.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.052+0000 7fa408fa4640 1 --2- 172.21.15.114:0/2628361985 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff4f4bf900 0x55ff4f4c2160 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x55ff4f4bc5d0 tx=0x7fa3f4005cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8478d13a1e3721f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:59.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.052+0000 7fa3effff640 1 -- 172.21.15.114:0/2628361985 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f4024370 con 0x55ff4f4bf900 2024-09-13T18:29:59.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.052+0000 7fa3effff640 1 -- 172.21.15.114:0/2628361985 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa3f4016070 con 0x55ff4f4bf900 2024-09-13T18:29:59.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.053+0000 7fa40e808740 1 -- 172.21.15.114:0/2628361985 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff4f4bf900 msgr2=0x55ff4f4c2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:59.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.053+0000 7fa40e808740 1 --2- 172.21.15.114:0/2628361985 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff4f4bf900 0x55ff4f4c2160 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x55ff4f4bc5d0 tx=0x7fa3f4005cc0 comp rx=0 tx=0).stop 2024-09-13T18:29:59.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.054+0000 7fa40e808740 1 -- 172.21.15.114:0/2628361985 shutdown_connections 2024-09-13T18:29:59.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.054+0000 7fa40e808740 1 --2- 172.21.15.114:0/2628361985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff4f4ca1a0 0x55ff4f4c26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:59.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.054+0000 7fa40e808740 1 --2- 172.21.15.114:0/2628361985 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff4f4bf900 0x55ff4f4c2160 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:59.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.054+0000 7fa40e808740 1 --2- 172.21.15.114:0/2628361985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff4f4befe0 0x55ff4f4c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:59.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.054+0000 7fa40e808740 1 -- 172.21.15.114:0/2628361985 >> 172.21.15.114:0/2628361985 conn(0x55ff4f4bb570 msgr2=0x55ff4f4bb950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:59.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.058+0000 7fa40e808740 1 -- 172.21.15.114:0/2628361985 shutdown_connections 2024-09-13T18:29:59.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.058+0000 7fa40e808740 1 -- 172.21.15.114:0/2628361985 wait complete. 2024-09-13T18:29:59.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.058+0000 7fa40e808740 1 -- start start 2024-09-13T18:29:59.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.058+0000 7fa40e808740 1 Processor -- start 2024-09-13T18:29:59.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.059+0000 7fa40e808740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff4f4befe0 0x55ff4f4e0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:59.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.059+0000 7fa40e808740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff4f4bf900 0x55ff4f4dc030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:59.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.059+0000 7fa40e808740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff4f4ca1a0 0x55ff4f4dc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:59.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.059+0000 7fa40e808740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff4f4dccd0 con 0x55ff4f4befe0 2024-09-13T18:29:59.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.059+0000 7fa40e808740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff4f4dce40 con 0x55ff4f4bf900 2024-09-13T18:29:59.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.059+0000 7fa40e808740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff4f5512e0 con 0x55ff4f4ca1a0 2024-09-13T18:29:59.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.059+0000 7fa40e808740 10 client.0 _ll_register_callbacks cb 0x7fa400006e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:29:59.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.061+0000 7fa409fa6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff4f4ca1a0 0x55ff4f4dc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:59.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.061+0000 7fa409fa6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff4f4ca1a0 0x55ff4f4dc5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:57972/0 (socket says 172.21.15.114:57972) 2024-09-13T18:29:59.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.061+0000 7fa409fa6640 1 -- 172.21.15.114:0/3170243177 learned_addr learned my addr 172.21.15.114:0/3170243177 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:59.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.061+0000 7fa4097a5640 1 --2- 172.21.15.114:0/3170243177 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff4f4befe0 0x55ff4f4e0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:59.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.061+0000 7fa408fa4640 1 --2- 172.21.15.114:0/3170243177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff4f4bf900 0x55ff4f4dc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:59.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.064+0000 7fa409fa6640 1 -- 172.21.15.114:0/3170243177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff4f4bf900 msgr2=0x55ff4f4dc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:59.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.064+0000 7fa409fa6640 1 --2- 172.21.15.114:0/3170243177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff4f4bf900 0x55ff4f4dc030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:59.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.064+0000 7fa409fa6640 1 -- 172.21.15.114:0/3170243177 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff4f4befe0 msgr2=0x55ff4f4e0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:29:59.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.064+0000 7fa409fa6640 1 --2- 172.21.15.114:0/3170243177 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff4f4befe0 0x55ff4f4e0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:59.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.064+0000 7fa409fa6640 1 -- 172.21.15.114:0/3170243177 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ff4f421620 con 0x55ff4f4ca1a0 2024-09-13T18:29:59.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.066+0000 7fa409fa6640 1 --2- 172.21.15.114:0/3170243177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff4f4ca1a0 0x55ff4f4dc5d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa404065f10 tx=0x7fa40411c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70dfed6c373aa13e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:59.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.067+0000 7fa3ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:29:59.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.067+0000 7fa3ecff9640 1 -- 172.21.15.114:0/3170243177 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa404094a30 con 0x55ff4f4ca1a0 2024-09-13T18:29:59.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.067+0000 7fa40e808740 1 -- 172.21.15.114:0/3170243177 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa3f4018020 con 0x55ff4f4ca1a0 2024-09-13T18:29:59.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.067+0000 7fa3ecff9640 1 -- 172.21.15.114:0/3170243177 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fa4041189e0 con 0x55ff4f4ca1a0 2024-09-13T18:29:59.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.067+0000 7fa3ecff9640 1 -- 172.21.15.114:0/3170243177 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa404118ba0 con 0x55ff4f4ca1a0 2024-09-13T18:29:59.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.067+0000 7fa40e808740 10 client.27136 fetch_fsmap learned FSMap version 34 2024-09-13T18:29:59.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.067+0000 7fa40e808740 1 -- 172.21.15.114:0/3170243177 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55ff4f3e8ea0 con 0x55ff4f4ca1a0 2024-09-13T18:29:59.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.069+0000 7fa3ecff9640 1 -- 172.21.15.114:0/3170243177 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa3f4018020 con 0x55ff4f4ca1a0 2024-09-13T18:29:59.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.069+0000 7fa3ecff9640 1 -- 172.21.15.114:0/3170243177 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa40411b7c0 con 0x55ff4f4ca1a0 2024-09-13T18:29:59.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.069+0000 7fa40e808740 10 client.27136 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:29:59.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.069+0000 7fa40e808740 10 client.27136 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:29:59.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.069+0000 7fa40e808740 20 client.27136 populate_metadata read hostname 'smithi114' 2024-09-13T18:29:59.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.069+0000 7fa40e808740 1 --2- 172.21.15.114:0/3170243177 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55ff4f5527f0 0x55ff4f554c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:59.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.069+0000 7fa40e808740 4 client.27136 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T18:29:59.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.069+0000 7fa40e808740 1 -- 172.21.15.114:0/3170243177 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ff4f17ae30 con 0x55ff4f5527f0 2024-09-13T18:29:59.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.070+0000 7fa4097a5640 1 --2- 172.21.15.114:0/3170243177 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55ff4f5527f0 0x55ff4f554c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:59.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.074+0000 7fa4097a5640 1 --2- 172.21.15.114:0/3170243177 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55ff4f5527f0 0x55ff4f554c10 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af2a716b5a518c71 server_cookie=77b02b923f3a52dd in_seq=0 out_seq=0 2024-09-13T18:29:59.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.074+0000 7fa3ecff9640 10 client.27136 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:29:59.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.075+0000 7fa3ecff9640 1 -- 172.21.15.114:0/3170243177 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55ff4f17ae30 con 0x55ff4f5527f0 2024-09-13T18:29:59.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.075+0000 7fa3ecff9640 10 client.27136 handle_command_reply: tid=0 2024-09-13T18:29:59.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.075+0000 7fa40e808740 1 client.27136 shutdown 2024-09-13T18:29:59.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.075+0000 7fa40e808740 1 -- 172.21.15.114:0/3170243177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff4f4ca1a0 msgr2=0x55ff4f4dc5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:59.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.075+0000 7fa40e808740 1 --2- 172.21.15.114:0/3170243177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff4f4ca1a0 0x55ff4f4dc5d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa404065f10 tx=0x7fa40411c390 comp rx=0 tx=0).stop 2024-09-13T18:29:59.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.075+0000 7fa40e808740 1 -- 172.21.15.114:0/3170243177 shutdown_connections 2024-09-13T18:29:59.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.075+0000 7fa40e808740 1 --2- 172.21.15.114:0/3170243177 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55ff4f5527f0 0x55ff4f554c10 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:59.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.075+0000 7fa40e808740 1 --2- 172.21.15.114:0/3170243177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff4f4ca1a0 0x55ff4f4dc5d0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:59.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.075+0000 7fa40e808740 1 --2- 172.21.15.114:0/3170243177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff4f4bf900 0x55ff4f4dc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:59.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.075+0000 7fa40e808740 1 --2- 172.21.15.114:0/3170243177 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff4f4befe0 0x55ff4f4e0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:59.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.075+0000 7fa40e808740 1 -- 172.21.15.114:0/3170243177 >> 172.21.15.114:0/3170243177 conn(0x55ff4f4bb570 msgr2=0x55ff4f4c2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:59.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.076+0000 7fa3ecff9640 0 client.27136 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:29:59.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.076+0000 7fa40e808740 1 -- 172.21.15.114:0/3170243177 shutdown_connections 2024-09-13T18:29:59.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.076+0000 7fa40e808740 1 -- 172.21.15.114:0/3170243177 wait complete. 2024-09-13T18:29:59.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.078+0000 7fa40e808740 20 client.27136 trim_cache size 0 max 16384 2024-09-13T18:29:59.088 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:29:59.088 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "no active scrubs running", 2024-09-13T18:29:59.088 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": {} 2024-09-13T18:29:59.088 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:29:59.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.089+0000 7fa40a7a7640 1 -- 172.21.15.114:0/2673109246 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa3f00a1350 msgr2=0x7fa3f00a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:59.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.089+0000 7fa40a7a7640 1 --2- 172.21.15.114:0/2673109246 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa3f00a1350 0x7fa3f00a3810 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x55ff4f4bb070 tx=0x7fa400009060 comp rx=0 tx=0).stop 2024-09-13T18:29:59.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.089+0000 7fa40a7a7640 1 -- 172.21.15.114:0/2673109246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3fc0b7510 msgr2=0x7fa3fc1479e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:59.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.089+0000 7fa40a7a7640 1 --2- 172.21.15.114:0/2673109246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3fc0b7510 0x7fa3fc1479e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa404000b60 tx=0x7fa404068060 comp rx=0 tx=0).stop 2024-09-13T18:29:59.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.091+0000 7fa40a7a7640 1 -- 172.21.15.114:0/2673109246 shutdown_connections 2024-09-13T18:29:59.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.091+0000 7fa40a7a7640 1 --2- 172.21.15.114:0/2673109246 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa3f00a1350 0x7fa3f00a3810 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:59.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.091+0000 7fa40a7a7640 1 --2- 172.21.15.114:0/2673109246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3fc0b7510 0x7fa3fc1479e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:59.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.091+0000 7fa40a7a7640 1 --2- 172.21.15.114:0/2673109246 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3fc0a5110 0x7fa3fc1435f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:59.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.091+0000 7fa40a7a7640 1 --2- 172.21.15.114:0/2673109246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3fc0a47f0 0x7fa3fc1430b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:59.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.091+0000 7fa40a7a7640 1 -- 172.21.15.114:0/2673109246 >> 172.21.15.114:0/2673109246 conn(0x7fa3fc01a0e0 msgr2=0x7fa3fc0b4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:59.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.094+0000 7fa40a7a7640 1 -- 172.21.15.114:0/2673109246 shutdown_connections 2024-09-13T18:29:59.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.094+0000 7fa40a7a7640 1 -- 172.21.15.114:0/2673109246 wait complete. 2024-09-13T18:29:59.135 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-13T18:29:59.135 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:29:59.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.360+0000 7f1faefdb640 1 -- 172.21.15.114:0/3380898332 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1fa8102990 msgr2=0x7f1fa81069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:59.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.360+0000 7f1faefdb640 1 --2- 172.21.15.114:0/3380898332 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1fa8102990 0x7f1fa81069d0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f1f98009930 tx=0x7f1f9802a480 comp rx=0 tx=0).stop 2024-09-13T18:29:59.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.364+0000 7f1faefdb640 1 -- 172.21.15.114:0/3380898332 shutdown_connections 2024-09-13T18:29:59.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.364+0000 7f1faefdb640 1 --2- 172.21.15.114:0/3380898332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fa8107130 0x7f1fa810ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:59.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.364+0000 7f1faefdb640 1 --2- 172.21.15.114:0/3380898332 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1fa8102990 0x7f1fa81069d0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:59.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.364+0000 7f1faefdb640 1 --2- 172.21.15.114:0/3380898332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1fa8101fe0 0x7f1fa81023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:59.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.364+0000 7f1faefdb640 1 -- 172.21.15.114:0/3380898332 >> 172.21.15.114:0/3380898332 conn(0x7f1fa80fda10 msgr2=0x7f1fa80ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:59.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.370+0000 7f1faefdb640 1 -- 172.21.15.114:0/3380898332 shutdown_connections 2024-09-13T18:29:59.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.372+0000 7f1faefdb640 1 -- 172.21.15.114:0/3380898332 wait complete. 2024-09-13T18:29:59.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.373+0000 7f1faefdb640 1 Processor -- start 2024-09-13T18:29:59.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.373+0000 7f1faefdb640 1 -- start start 2024-09-13T18:29:59.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.373+0000 7f1faefdb640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fa8101fe0 0x7f1fa8113f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:59.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.373+0000 7f1faefdb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1fa8107130 0x7f1fa8114480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:59.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.373+0000 7f1faefdb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1fa810dd60 0x7f1fa810e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:59.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.373+0000 7f1faefdb640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa810e8e0 con 0x7f1fa8107130 2024-09-13T18:29:59.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.373+0000 7f1faefdb640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa810ea50 con 0x7f1fa810dd60 2024-09-13T18:29:59.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.373+0000 7f1faefdb640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa810ec10 con 0x7f1fa8101fe0 2024-09-13T18:29:59.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.374+0000 7f1facd77640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fa8101fe0 0x7f1fa8113f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:59.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.374+0000 7f1facd77640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fa8101fe0 0x7f1fa8113f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:57980/0 (socket says 172.21.15.114:57980) 2024-09-13T18:29:59.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.374+0000 7f1facd77640 1 -- 172.21.15.114:0/269360134 learned_addr learned my addr 172.21.15.114:0/269360134 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:29:59.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.374+0000 7f1facd77640 1 -- 172.21.15.114:0/269360134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1fa810dd60 msgr2=0x7f1fa810e210 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:29:59.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.374+0000 7f1facd77640 1 --2- 172.21.15.114:0/269360134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1fa810dd60 0x7f1fa810e210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:59.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.374+0000 7f1facd77640 1 -- 172.21.15.114:0/269360134 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1fa8107130 msgr2=0x7f1fa8114480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:29:59.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.374+0000 7f1facd77640 1 --2- 172.21.15.114:0/269360134 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1fa8107130 0x7f1fa8114480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:59.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.374+0000 7f1facd77640 1 -- 172.21.15.114:0/269360134 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f98009560 con 0x7f1fa8101fe0 2024-09-13T18:29:59.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.376+0000 7f1facd77640 1 --2- 172.21.15.114:0/269360134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fa8101fe0 0x7f1fa8113f40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1fa000ebc0 tx=0x7f1fa000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:59.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.376+0000 7f1fa5ffb640 1 -- 172.21.15.114:0/269360134 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa0010630 con 0x7f1fa8101fe0 2024-09-13T18:29:59.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.376+0000 7f1faefdb640 1 -- 172.21.15.114:0/269360134 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fa8117e50 con 0x7f1fa8101fe0 2024-09-13T18:29:59.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.377+0000 7f1faefdb640 1 -- 172.21.15.114:0/269360134 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fa8118360 con 0x7f1fa8101fe0 2024-09-13T18:29:59.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.378+0000 7f1fa5ffb640 1 -- 172.21.15.114:0/269360134 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1fa000eec0 con 0x7f1fa8101fe0 2024-09-13T18:29:59.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.378+0000 7f1faefdb640 1 -- 172.21.15.114:0/269360134 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fa81034b0 con 0x7f1fa8101fe0 2024-09-13T18:29:59.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.378+0000 7f1fa5ffb640 1 -- 172.21.15.114:0/269360134 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa0018a30 con 0x7f1fa8101fe0 2024-09-13T18:29:59.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.382+0000 7f1fa5ffb640 1 -- 172.21.15.114:0/269360134 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f1fa0018b90 con 0x7f1fa8101fe0 2024-09-13T18:29:59.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.383+0000 7f1fa5ffb640 1 --2- 172.21.15.114:0/269360134 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f1f900a13d0 0x7f1f900a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:29:59.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.383+0000 7f1fa5ffb640 1 -- 172.21.15.114:0/269360134 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f1fa0014070 con 0x7f1fa8101fe0 2024-09-13T18:29:59.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.385+0000 7f1fa7fff640 1 --2- 172.21.15.114:0/269360134 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f1f900a13d0 0x7f1f900a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:29:59.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.386+0000 7f1fa7fff640 1 --2- 172.21.15.114:0/269360134 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f1f900a13d0 0x7f1f900a3890 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1f98009230 tx=0x7f1f98005fd0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:29:59.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.389+0000 7f1fa5ffb640 1 -- 172.21.15.114:0/269360134 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f1fa0097ec0 con 0x7f1fa8101fe0 2024-09-13T18:29:59.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.914+0000 7f1faefdb640 1 -- 172.21.15.114:0/269360134 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1fa81023c0 con 0x7f1fa8101fe0 2024-09-13T18:29:59.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.917+0000 7f1fa5ffb640 1 -- 172.21.15.114:0/269360134 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+10930 (secure 0 0 0) 0x7f1fa0026070 con 0x7f1fa8101fe0 2024-09-13T18:29:59.922 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:29:59.922 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4162817497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4162817497},{"type":"v1","addr":"172.21.15.114:6835","nonce":4162817497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/2435428478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":2435428478},{"type":"v1","addr":"172.21.15.114:6837","nonce":2435428478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T18:20:01.895117+0000","modified":"2024-09-13T18:27:22.172031+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24482,"mds_2":24475,"mds_3":24458,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/2046516451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":2046516451},{"type":"v1","addr":"172.21.15.114:6839","nonce":2046516451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":10,"addr":"172.21.15.114:6841/1956923446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":1956923446},{"type":"v1","addr":"172.21.15.114:6841","nonce":1956923446}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24391":{"gid":24391,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/507301496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":507301496},{"type":"v1","addr":"172.21.15.204:6833","nonce":507301496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4028657219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4028657219},{"type":"v1","addr":"172.21.15.134:6833","nonce":4028657219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/57029119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":57029119},{"type":"v1","addr":"172.21.15.204:6835","nonce":57029119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3068880939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3068880939},{"type":"v1","addr":"172.21.15.204:6837","nonce":3068880939}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"e","rank":3,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.134:6835/1150003275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1150003275},{"type":"v1","addr":"172.21.15.134:6835","nonce":1150003275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/344883912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":344883912},{"type":"v1","addr":"172.21.15.204:6839","nonce":344883912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6837/891692210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":891692210},{"type":"v1","addr":"172.21.15.134:6837","nonce":891692210}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6839/3466309724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3466309724},{"type":"v1","addr":"172.21.15.134:6839","nonce":3466309724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:29:59.922 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 34 2024-09-13T18:29:59.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.926+0000 7f1faefdb640 1 -- 172.21.15.114:0/269360134 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f1f900a13d0 msgr2=0x7f1f900a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:59.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.926+0000 7f1faefdb640 1 --2- 172.21.15.114:0/269360134 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f1f900a13d0 0x7f1f900a3890 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1f98009230 tx=0x7f1f98005fd0 comp rx=0 tx=0).stop 2024-09-13T18:29:59.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.926+0000 7f1faefdb640 1 -- 172.21.15.114:0/269360134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fa8101fe0 msgr2=0x7f1fa8113f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:29:59.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.926+0000 7f1faefdb640 1 --2- 172.21.15.114:0/269360134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fa8101fe0 0x7f1fa8113f40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1fa000ebc0 tx=0x7f1fa000c6a0 comp rx=0 tx=0).stop 2024-09-13T18:29:59.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.928+0000 7f1faefdb640 1 -- 172.21.15.114:0/269360134 shutdown_connections 2024-09-13T18:29:59.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.928+0000 7f1faefdb640 1 --2- 172.21.15.114:0/269360134 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f1f900a13d0 0x7f1f900a3890 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:59.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.928+0000 7f1faefdb640 1 --2- 172.21.15.114:0/269360134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1fa810dd60 0x7f1fa810e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:59.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.928+0000 7f1faefdb640 1 --2- 172.21.15.114:0/269360134 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1fa8107130 0x7f1fa8114480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:59.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.928+0000 7f1faefdb640 1 --2- 172.21.15.114:0/269360134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fa8101fe0 0x7f1fa8113f40 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:29:59.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.928+0000 7f1faefdb640 1 -- 172.21.15.114:0/269360134 >> 172.21.15.114:0/269360134 conn(0x7f1fa80fda10 msgr2=0x7f1fa80ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:29:59.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.930+0000 7f1faefdb640 1 -- 172.21.15.114:0/269360134 shutdown_connections 2024-09-13T18:29:59.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:29:59.932+0000 7f1faefdb640 1 -- 172.21.15.114:0/269360134 wait complete. 2024-09-13T18:29:59.960 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-09-13T18:30:00.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.193+0000 7f236570f640 1 -- 172.21.15.114:0/565206824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2360107540 msgr2=0x7f236010e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:00.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.193+0000 7f236570f640 1 --2- 172.21.15.114:0/565206824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2360107540 0x7f236010e510 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2358009ca0 tx=0x7f235802a8f0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.195+0000 7f236570f640 1 -- 172.21.15.114:0/565206824 shutdown_connections 2024-09-13T18:30:00.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.195+0000 7f236570f640 1 --2- 172.21.15.114:0/565206824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2360119400 0x7f236011b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.195+0000 7f236570f640 1 --2- 172.21.15.114:0/565206824 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2360107920 0x7f2360107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.195+0000 7f236570f640 1 --2- 172.21.15.114:0/565206824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2360107540 0x7f236010e510 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.195+0000 7f236570f640 1 -- 172.21.15.114:0/565206824 >> 172.21.15.114:0/565206824 conn(0x7f236006afe0 msgr2=0x7f236006b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:00.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.204+0000 7f236570f640 1 -- 172.21.15.114:0/565206824 shutdown_connections 2024-09-13T18:30:00.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.207+0000 7f236570f640 1 -- 172.21.15.114:0/565206824 wait complete. 2024-09-13T18:30:00.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.208+0000 7f236570f640 1 Processor -- start 2024-09-13T18:30:00.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.208+0000 7f236570f640 1 -- start start 2024-09-13T18:30:00.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.208+0000 7f236570f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2360107540 0x7f23601a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:00.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.208+0000 7f236570f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2360107920 0x7f23601a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:00.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.208+0000 7f236570f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2360119400 0x7f23601a9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:00.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.208+0000 7f236570f640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23601a9f90 con 0x7f2360119400 2024-09-13T18:30:00.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.208+0000 7f236570f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23601aa100 con 0x7f2360107540 2024-09-13T18:30:00.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.208+0000 7f236570f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23601aa2c0 con 0x7f2360107920 2024-09-13T18:30:00.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.212+0000 7f235effd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2360107540 0x7f23601a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:00.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.212+0000 7f235effd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2360107540 0x7f23601a5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:43512/0 (socket says 172.21.15.114:43512) 2024-09-13T18:30:00.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.212+0000 7f235effd640 1 -- 172.21.15.114:0/3339129933 learned_addr learned my addr 172.21.15.114:0/3339129933 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:00.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.212+0000 7f235f7fe640 1 --2- 172.21.15.114:0/3339129933 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2360119400 0x7f23601a9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:00.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.212+0000 7f235effd640 1 -- 172.21.15.114:0/3339129933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2360107920 msgr2=0x7f23601a55a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:30:00.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.212+0000 7f235effd640 1 --2- 172.21.15.114:0/3339129933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2360107920 0x7f23601a55a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.212+0000 7f235effd640 1 -- 172.21.15.114:0/3339129933 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2360119400 msgr2=0x7f23601a9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:00.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.212+0000 7f235effd640 1 --2- 172.21.15.114:0/3339129933 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2360119400 0x7f23601a9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.212+0000 7f235effd640 1 -- 172.21.15.114:0/3339129933 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2358009900 con 0x7f2360107540 2024-09-13T18:30:00.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:29:59 smithi114 ceph-mon[30897]: pgmap v230: 129 pgs: 129 active+clean; 22 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 29 MiB/s wr, 640 op/s 2024-09-13T18:30:00.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:29:59 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/269360134' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:30:00.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.217+0000 7f235effd640 1 --2- 172.21.15.114:0/3339129933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2360107540 0x7f23601a5060 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2358006010 tx=0x7f2358003ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:00.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.221+0000 7f2347fff640 1 -- 172.21.15.114:0/3339129933 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2358031680 con 0x7f2360107540 2024-09-13T18:30:00.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.221+0000 7f2347fff640 1 -- 172.21.15.114:0/3339129933 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2358040070 con 0x7f2360107540 2024-09-13T18:30:00.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.222+0000 7f2347fff640 1 -- 172.21.15.114:0/3339129933 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23580043c0 con 0x7f2360107540 2024-09-13T18:30:00.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.222+0000 7f236570f640 1 -- 172.21.15.114:0/3339129933 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23601aa540 con 0x7f2360107540 2024-09-13T18:30:00.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.222+0000 7f236570f640 1 -- 172.21.15.114:0/3339129933 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2360110dd0 con 0x7f2360107540 2024-09-13T18:30:00.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.228+0000 7f236842f740 1 Processor -- start 2024-09-13T18:30:00.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.228+0000 7f236842f740 1 -- start start 2024-09-13T18:30:00.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.229+0000 7f236842f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5630a09a9790 0x5630a09a9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:00.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.229+0000 7f236842f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630a09aa0b0 0x5630a09b4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:00.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.229+0000 7f236842f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5630a09b4950 0x5630a09b6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:00.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.229+0000 7f236842f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630a0913cc0 con 0x5630a09b4950 2024-09-13T18:30:00.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.229+0000 7f236842f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630a0911450 con 0x5630a09aa0b0 2024-09-13T18:30:00.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.229+0000 7f236842f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630a090cd40 con 0x5630a09a9790 2024-09-13T18:30:00.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.229+0000 7f235effd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630a09aa0b0 0x5630a09b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:00.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.229+0000 7f235effd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630a09aa0b0 0x5630a09b4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:43522/0 (socket says 172.21.15.114:43522) 2024-09-13T18:30:00.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.229+0000 7f235effd640 1 -- 172.21.15.114:0/2289701750 learned_addr learned my addr 172.21.15.114:0/2289701750 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:00.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.229+0000 7f235f7fe640 1 --2- 172.21.15.114:0/2289701750 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5630a09a9790 0x5630a09a9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:00.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.230+0000 7f235f7fe640 1 -- 172.21.15.114:0/2289701750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630a09aa0b0 msgr2=0x5630a09b4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:00.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.230+0000 7f235f7fe640 1 --2- 172.21.15.114:0/2289701750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630a09aa0b0 0x5630a09b4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.230+0000 7f235f7fe640 1 -- 172.21.15.114:0/2289701750 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5630a09b4950 msgr2=0x5630a09b6d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:30:00.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.231+0000 7f235f7fe640 1 --2- 172.21.15.114:0/2289701750 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5630a09b4950 0x5630a09b6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.231+0000 7f235f7fe640 1 -- 172.21.15.114:0/2289701750 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5630a08d4620 con 0x5630a09a9790 2024-09-13T18:30:00.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.233+0000 7f2347fff640 1 -- 172.21.15.114:0/3339129933 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f2358042030 con 0x7f2360107540 2024-09-13T18:30:00.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.233+0000 7f235f7fe640 1 --2- 172.21.15.114:0/2289701750 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5630a09a9790 0x5630a09a9b70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f235400ca90 tx=0x7f235400cf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb984ec0aa575b00 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:00.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.234+0000 7f2347fff640 1 --2- 172.21.15.114:0/3339129933 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f23480a1260 0x7f23480a3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:00.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.234+0000 7f2347fff640 1 -- 172.21.15.114:0/3339129933 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f23580f51e0 con 0x7f2360107540 2024-09-13T18:30:00.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.235+0000 7f235effd640 1 --2- 172.21.15.114:0/3339129933 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f23480a1260 0x7f23480a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:00.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.238+0000 7f23457fa640 1 -- 172.21.15.114:0/2289701750 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2354004020 con 0x5630a09a9790 2024-09-13T18:30:00.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.238+0000 7f23457fa640 1 -- 172.21.15.114:0/2289701750 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f23540035f0 con 0x5630a09a9790 2024-09-13T18:30:00.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.238+0000 7f23457fa640 1 -- 172.21.15.114:0/2289701750 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2354010690 con 0x5630a09a9790 2024-09-13T18:30:00.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.239+0000 7f236842f740 1 -- 172.21.15.114:0/2289701750 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5630a09a9790 msgr2=0x5630a09a9b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:00.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.239+0000 7f236842f740 1 --2- 172.21.15.114:0/2289701750 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5630a09a9790 0x5630a09a9b70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f235400ca90 tx=0x7f235400cf50 comp rx=0 tx=0).stop 2024-09-13T18:30:00.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.239+0000 7f236842f740 1 -- 172.21.15.114:0/2289701750 shutdown_connections 2024-09-13T18:30:00.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.239+0000 7f236842f740 1 --2- 172.21.15.114:0/2289701750 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5630a09b4950 0x5630a09b6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.239+0000 7f236842f740 1 --2- 172.21.15.114:0/2289701750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630a09aa0b0 0x5630a09b4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.240+0000 7f236842f740 1 --2- 172.21.15.114:0/2289701750 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5630a09a9790 0x5630a09a9b70 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.240+0000 7f236842f740 1 -- 172.21.15.114:0/2289701750 >> 172.21.15.114:0/2289701750 conn(0x5630a09a5d00 msgr2=0x5630a09a60e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:00.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.240+0000 7f235effd640 1 --2- 172.21.15.114:0/3339129933 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f23480a1260 0x7f23480a3720 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x5630a09a6b60 tx=0x7f2358000f40 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:00.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.241+0000 7f236842f740 1 -- 172.21.15.114:0/2289701750 shutdown_connections 2024-09-13T18:30:00.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.242+0000 7f236842f740 1 -- 172.21.15.114:0/2289701750 wait complete. 2024-09-13T18:30:00.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.242+0000 7f236842f740 1 -- start start 2024-09-13T18:30:00.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.242+0000 7f236842f740 1 Processor -- start 2024-09-13T18:30:00.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.243+0000 7f236842f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5630a09a9790 0x5630a09b3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:00.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.243+0000 7f236842f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630a09aa0b0 0x5630a09b43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:00.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.243+0000 7f236842f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5630a09b4950 0x5630a09b1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:00.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.243+0000 7f236842f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630a08cb180 con 0x5630a09a9790 2024-09-13T18:30:00.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.243+0000 7f236842f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630a05e5430 con 0x5630a09aa0b0 2024-09-13T18:30:00.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.243+0000 7f236842f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630a08905e0 con 0x5630a09b4950 2024-09-13T18:30:00.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.243+0000 7f236842f740 10 client.0 _ll_register_callbacks cb 0x5630a08fcaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:30:00.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.243+0000 7f235e7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630a09aa0b0 0x5630a09b43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:00.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.243+0000 7f235e7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630a09aa0b0 0x5630a09b43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:43524/0 (socket says 172.21.15.114:43524) 2024-09-13T18:30:00.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.243+0000 7f235e7fc640 1 -- 172.21.15.114:0/2636300064 learned_addr learned my addr 172.21.15.114:0/2636300064 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:00.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.243+0000 7f235f7fe640 1 --2- 172.21.15.114:0/2636300064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5630a09b4950 0x5630a09b1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:00.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.243+0000 7f235f7fe640 1 --2- 172.21.15.114:0/2636300064 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5630a09a9790 0x5630a09b3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:00.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.244+0000 7f235f7fe640 1 -- 172.21.15.114:0/2636300064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630a09aa0b0 msgr2=0x5630a09b43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:00.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.244+0000 7f235f7fe640 1 --2- 172.21.15.114:0/2636300064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630a09aa0b0 0x5630a09b43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.244+0000 7f235f7fe640 1 -- 172.21.15.114:0/2636300064 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5630a09a9790 msgr2=0x5630a09b3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:00.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.244+0000 7f235f7fe640 1 --2- 172.21.15.114:0/2636300064 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5630a09a9790 0x5630a09b3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.244+0000 7f235f7fe640 1 -- 172.21.15.114:0/2636300064 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f235400c6c0 con 0x5630a09b4950 2024-09-13T18:30:00.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.244+0000 7f235f7fe640 1 --2- 172.21.15.114:0/2636300064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5630a09b4950 0x5630a09b1470 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f235400ca90 tx=0x7f2354003ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d402a822a4d1ed2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:00.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.244+0000 7f2342ff5640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:30:00.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.245+0000 7f2342ff5640 1 -- 172.21.15.114:0/2636300064 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2354003b90 con 0x5630a09b4950 2024-09-13T18:30:00.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.245+0000 7f2342ff5640 1 -- 172.21.15.114:0/2636300064 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f2354005020 con 0x5630a09b4950 2024-09-13T18:30:00.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.245+0000 7f236842f740 1 -- 172.21.15.114:0/2636300064 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5630a0936b80 con 0x5630a09b4950 2024-09-13T18:30:00.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.245+0000 7f2342ff5640 1 -- 172.21.15.114:0/2636300064 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23540051e0 con 0x5630a09b4950 2024-09-13T18:30:00.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.246+0000 7f236842f740 10 client.27145 fetch_fsmap learned FSMap version 34 2024-09-13T18:30:00.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.246+0000 7f236842f740 1 -- 172.21.15.114:0/2636300064 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5630a090b3c0 con 0x5630a09b4950 2024-09-13T18:30:00.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.247+0000 7f2342ff5640 1 -- 172.21.15.114:0/2636300064 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5630a0936b80 con 0x5630a09b4950 2024-09-13T18:30:00.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.247+0000 7f2342ff5640 1 -- 172.21.15.114:0/2636300064 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2354029460 con 0x5630a09b4950 2024-09-13T18:30:00.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.248+0000 7f236842f740 10 client.27145 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:30:00.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.248+0000 7f236842f740 10 client.27145 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-13T18:30:00.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.248+0000 7f236842f740 20 client.27145 populate_metadata read hostname 'smithi114' 2024-09-13T18:30:00.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.248+0000 7f236842f740 1 --2- 172.21.15.114:0/2636300064 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x5630a09c41d0 0x5630a09c65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:00.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.248+0000 7f236842f740 4 client.27145 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:30:00.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.248+0000 7f236842f740 1 -- 172.21.15.114:0/2636300064 --> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5630a08ff1f0 con 0x5630a09c41d0 2024-09-13T18:30:00.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.253+0000 7f235e7fc640 1 --2- 172.21.15.114:0/2636300064 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x5630a09c41d0 0x5630a09c65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:00.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.255+0000 7f2342ff5640 10 client.27145 ms_handle_connect on v2:172.21.15.114:6840/1956923446 2024-09-13T18:30:00.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.255+0000 7f235e7fc640 1 --2- 172.21.15.114:0/2636300064 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x5630a09c41d0 0x5630a09c65c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4be2c6f2fb5eb9a server_cookie=418b6aa0c9de3db9 in_seq=0 out_seq=0 2024-09-13T18:30:00.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.256+0000 7f2342ff5640 1 -- 172.21.15.114:0/2636300064 <== mds.4 v2:172.21.15.114:6840/1956923446 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5630a08ff1f0 con 0x5630a09c41d0 2024-09-13T18:30:00.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.256+0000 7f2342ff5640 10 client.27145 handle_command_reply: tid=0 2024-09-13T18:30:00.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.256+0000 7f236842f740 1 client.27145 shutdown 2024-09-13T18:30:00.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.259+0000 7f236842f740 1 -- 172.21.15.114:0/2636300064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5630a09b4950 msgr2=0x5630a09b1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:00.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.259+0000 7f236842f740 1 --2- 172.21.15.114:0/2636300064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5630a09b4950 0x5630a09b1470 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f235400ca90 tx=0x7f2354003ed0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.260+0000 7f236842f740 1 -- 172.21.15.114:0/2636300064 shutdown_connections 2024-09-13T18:30:00.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.260+0000 7f236842f740 1 --2- 172.21.15.114:0/2636300064 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x5630a09c41d0 0x5630a09c65c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.260+0000 7f236842f740 1 --2- 172.21.15.114:0/2636300064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5630a09b4950 0x5630a09b1470 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.260+0000 7f236842f740 1 --2- 172.21.15.114:0/2636300064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630a09aa0b0 0x5630a09b43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.260+0000 7f236842f740 1 --2- 172.21.15.114:0/2636300064 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5630a09a9790 0x5630a09b3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.260+0000 7f236842f740 1 -- 172.21.15.114:0/2636300064 >> 172.21.15.114:0/2636300064 conn(0x5630a09a5d00 msgr2=0x5630a09aba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:00.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.260+0000 7f2342ff5640 0 client.27145 ms_handle_reset on v2:172.21.15.114:6840/1956923446 2024-09-13T18:30:00.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.268+0000 7f236842f740 1 -- 172.21.15.114:0/2636300064 shutdown_connections 2024-09-13T18:30:00.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.269+0000 7f236842f740 1 -- 172.21.15.114:0/2636300064 wait complete. 2024-09-13T18:30:00.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.271+0000 7f236842f740 20 client.27145 trim_cache size 0 max 16384 2024-09-13T18:30:00.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.297+0000 7f236842f740 1 Processor -- start 2024-09-13T18:30:00.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.297+0000 7f236842f740 1 -- start start 2024-09-13T18:30:00.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.297+0000 7f236842f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630a09a9790 0x5630a09ac200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:00.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.297+0000 7f236842f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5630a09aa0b0 0x5630a09ac740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:00.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.298+0000 7f235f7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630a09a9790 0x5630a09ac200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:00.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.298+0000 7f235f7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630a09a9790 0x5630a09ac200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:43528/0 (socket says 172.21.15.114:43528) 2024-09-13T18:30:00.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.298+0000 7f235f7fe640 1 -- 172.21.15.114:0/1514949854 learned_addr learned my addr 172.21.15.114:0/1514949854 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:00.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.298+0000 7f236842f740 1 --2- 172.21.15.114:0/1514949854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5630a09b4950 0x5630a09b5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:00.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.298+0000 7f236842f740 1 -- 172.21.15.114:0/1514949854 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630a09c4000 con 0x5630a09aa0b0 2024-09-13T18:30:00.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.298+0000 7f236842f740 1 -- 172.21.15.114:0/1514949854 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630a09a18a0 con 0x5630a09a9790 2024-09-13T18:30:00.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.298+0000 7f236842f740 1 -- 172.21.15.114:0/1514949854 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630a09b2690 con 0x5630a09b4950 2024-09-13T18:30:00.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.298+0000 7f235e7fc640 1 --2- 172.21.15.114:0/1514949854 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5630a09aa0b0 0x5630a09ac740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:00.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.299+0000 7f235f7fe640 1 -- 172.21.15.114:0/1514949854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5630a09b4950 msgr2=0x5630a09b5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:00.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.299+0000 7f235f7fe640 1 --2- 172.21.15.114:0/1514949854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5630a09b4950 0x5630a09b5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.299+0000 7f235f7fe640 1 -- 172.21.15.114:0/1514949854 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5630a09aa0b0 msgr2=0x5630a09ac740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:00.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.299+0000 7f235f7fe640 1 --2- 172.21.15.114:0/1514949854 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5630a09aa0b0 0x5630a09ac740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.299+0000 7f235f7fe640 1 -- 172.21.15.114:0/1514949854 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5630a08ad770 con 0x5630a09a9790 2024-09-13T18:30:00.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.299+0000 7f235f7fe640 1 --2- 172.21.15.114:0/1514949854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630a09a9790 0x5630a09ac200 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f235400f710 tx=0x7f2354029940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cde416593b616468 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:00.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.300+0000 7f2345ffb640 1 -- 172.21.15.114:0/1514949854 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2354005dc0 con 0x5630a09a9790 2024-09-13T18:30:00.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.300+0000 7f2345ffb640 1 -- 172.21.15.114:0/1514949854 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f235400e7e0 con 0x5630a09a9790 2024-09-13T18:30:00.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.300+0000 7f2345ffb640 1 -- 172.21.15.114:0/1514949854 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23540296c0 con 0x5630a09a9790 2024-09-13T18:30:00.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.301+0000 7f236842f740 1 -- 172.21.15.114:0/1514949854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630a09a9790 msgr2=0x5630a09ac200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:00.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.301+0000 7f236842f740 1 --2- 172.21.15.114:0/1514949854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630a09a9790 0x5630a09ac200 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f235400f710 tx=0x7f2354029940 comp rx=0 tx=0).stop 2024-09-13T18:30:00.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.302+0000 7f236842f740 1 -- 172.21.15.114:0/1514949854 shutdown_connections 2024-09-13T18:30:00.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.302+0000 7f236842f740 1 --2- 172.21.15.114:0/1514949854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5630a09b4950 0x5630a09b5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.302+0000 7f236842f740 1 --2- 172.21.15.114:0/1514949854 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5630a09aa0b0 0x5630a09ac740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.303+0000 7f236842f740 1 --2- 172.21.15.114:0/1514949854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630a09a9790 0x5630a09ac200 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.303+0000 7f236842f740 1 -- 172.21.15.114:0/1514949854 >> 172.21.15.114:0/1514949854 conn(0x5630a09a5d00 msgr2=0x5630a09afad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:00.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.305+0000 7f236842f740 1 -- 172.21.15.114:0/1514949854 shutdown_connections 2024-09-13T18:30:00.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.312+0000 7f236842f740 1 -- 172.21.15.114:0/1514949854 wait complete. 2024-09-13T18:30:00.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.312+0000 7f236842f740 1 -- start start 2024-09-13T18:30:00.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.312+0000 7f236842f740 1 Processor -- start 2024-09-13T18:30:00.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.313+0000 7f236842f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5630a09a9790 0x5630a09bba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:00.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.313+0000 7f236842f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5630a09aa0b0 0x5630a09b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:00.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.313+0000 7f235f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5630a09a9790 0x5630a09bba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:00.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.313+0000 7f235f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5630a09a9790 0x5630a09bba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:58014/0 (socket says 172.21.15.114:58014) 2024-09-13T18:30:00.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.313+0000 7f235f7fe640 1 -- 172.21.15.114:0/1590298931 learned_addr learned my addr 172.21.15.114:0/1590298931 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:00.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.313+0000 7f236842f740 1 --2- 172.21.15.114:0/1590298931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630a09b4950 0x5630a09b82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:00.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.313+0000 7f236842f740 1 -- 172.21.15.114:0/1590298931 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630a09b89c0 con 0x5630a09aa0b0 2024-09-13T18:30:00.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.314+0000 7f236842f740 1 -- 172.21.15.114:0/1590298931 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630a09b8b30 con 0x5630a09b4950 2024-09-13T18:30:00.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.314+0000 7f236842f740 1 -- 172.21.15.114:0/1590298931 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630a0a1a6a0 con 0x5630a09a9790 2024-09-13T18:30:00.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.314+0000 7f236842f740 10 client.0 _ll_register_callbacks cb 0x7f2354014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:30:00.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.314+0000 7f235f7fe640 1 --2- 172.21.15.114:0/1590298931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630a09b4950 0x5630a09b82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:00.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.315+0000 7f235f7fe640 1 -- 172.21.15.114:0/1590298931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630a09b4950 msgr2=0x5630a09b82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:00.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.315+0000 7f235f7fe640 1 --2- 172.21.15.114:0/1590298931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630a09b4950 0x5630a09b82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.315+0000 7f235f7fe640 1 -- 172.21.15.114:0/1590298931 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5630a09aa0b0 msgr2=0x5630a09b7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:00.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.315+0000 7f235f7fe640 1 --2- 172.21.15.114:0/1590298931 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5630a09aa0b0 0x5630a09b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.315+0000 7f235f7fe640 1 -- 172.21.15.114:0/1590298931 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5630a090b3c0 con 0x5630a09a9790 2024-09-13T18:30:00.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.315+0000 7f235f7fe640 1 --2- 172.21.15.114:0/1590298931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5630a09a9790 0x5630a09bba30 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2354003f40 tx=0x7f2354005b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dde386a311ae09db server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:00.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.316+0000 7f2342ff5640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:30:00.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.319+0000 7f2342ff5640 1 -- 172.21.15.114:0/1590298931 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2354005dc0 con 0x5630a09a9790 2024-09-13T18:30:00.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.319+0000 7f2342ff5640 1 -- 172.21.15.114:0/1590298931 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f2354000e40 con 0x5630a09a9790 2024-09-13T18:30:00.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.322+0000 7f236842f740 1 -- 172.21.15.114:0/1590298931 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f23540035f0 con 0x5630a09a9790 2024-09-13T18:30:00.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.323+0000 7f2342ff5640 1 -- 172.21.15.114:0/1590298931 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2354029460 con 0x5630a09a9790 2024-09-13T18:30:00.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.332+0000 7f236842f740 10 client.27148 fetch_fsmap learned FSMap version 34 2024-09-13T18:30:00.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.332+0000 7f236842f740 1 -- 172.21.15.114:0/1590298931 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5630a08d2e80 con 0x5630a09a9790 2024-09-13T18:30:00.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.332+0000 7f2342ff5640 1 -- 172.21.15.114:0/1590298931 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f23540295c0 con 0x5630a09a9790 2024-09-13T18:30:00.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.332+0000 7f2342ff5640 1 -- 172.21.15.114:0/1590298931 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2354010430 con 0x5630a09a9790 2024-09-13T18:30:00.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.333+0000 7f236842f740 10 client.27148 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:30:00.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.333+0000 7f236842f740 10 client.27148 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-13T18:30:00.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.333+0000 7f236842f740 20 client.27148 populate_metadata read hostname 'smithi114' 2024-09-13T18:30:00.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.333+0000 7f236842f740 1 --2- 172.21.15.114:0/1590298931 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x5630a0a1b8a0 0x5630a0a1dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:00.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.333+0000 7f236842f740 4 client.27148 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:30:00.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.333+0000 7f236842f740 1 -- 172.21.15.114:0/1590298931 --> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f23540035f0 con 0x5630a0a1b8a0 2024-09-13T18:30:00.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.334+0000 7f235e7fc640 1 --2- 172.21.15.114:0/1590298931 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x5630a0a1b8a0 0x5630a0a1dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:00.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.353+0000 7f235e7fc640 1 --2- 172.21.15.114:0/1590298931 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x5630a0a1b8a0 0x5630a0a1dc90 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e258841b3933bb1d server_cookie=9682af88e19c3e88 in_seq=0 out_seq=0 2024-09-13T18:30:00.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.353+0000 7f2342ff5640 10 client.27148 ms_handle_connect on v2:172.21.15.114:6840/1956923446 2024-09-13T18:30:00.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.357+0000 7f2342ff5640 1 -- 172.21.15.114:0/1590298931 <== mds.4 v2:172.21.15.114:6840/1956923446 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f23540035f0 con 0x5630a0a1b8a0 2024-09-13T18:30:00.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.357+0000 7f2342ff5640 10 client.27148 handle_command_reply: tid=0 2024-09-13T18:30:00.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.357+0000 7f236842f740 1 client.27148 shutdown 2024-09-13T18:30:00.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.362+0000 7f236842f740 1 -- 172.21.15.114:0/1590298931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5630a09a9790 msgr2=0x5630a09bba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:00.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.362+0000 7f236842f740 1 --2- 172.21.15.114:0/1590298931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5630a09a9790 0x5630a09bba30 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2354003f40 tx=0x7f2354005b00 comp rx=0 tx=0).stop 2024-09-13T18:30:00.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.363+0000 7f236842f740 1 -- 172.21.15.114:0/1590298931 shutdown_connections 2024-09-13T18:30:00.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.363+0000 7f236842f740 1 --2- 172.21.15.114:0/1590298931 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x5630a0a1b8a0 0x5630a0a1dc90 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.363+0000 7f2342ff5640 0 client.27148 ms_handle_reset on v2:172.21.15.114:6840/1956923446 2024-09-13T18:30:00.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.363+0000 7f236842f740 1 --2- 172.21.15.114:0/1590298931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630a09b4950 0x5630a09b82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.363+0000 7f236842f740 1 --2- 172.21.15.114:0/1590298931 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5630a09aa0b0 0x5630a09b7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.363+0000 7f236842f740 1 --2- 172.21.15.114:0/1590298931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5630a09a9790 0x5630a09bba30 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.363+0000 7f236842f740 1 -- 172.21.15.114:0/1590298931 >> 172.21.15.114:0/1590298931 conn(0x5630a09a5d00 msgr2=0x5630a09b39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:00.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.363+0000 7f236842f740 1 -- 172.21.15.114:0/1590298931 shutdown_connections 2024-09-13T18:30:00.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.363+0000 7f236842f740 1 -- 172.21.15.114:0/1590298931 wait complete. 2024-09-13T18:30:00.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:29:59 smithi134 ceph-mon[34283]: pgmap v230: 129 pgs: 129 active+clean; 22 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 29 MiB/s wr, 640 op/s 2024-09-13T18:30:00.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:29:59 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/269360134' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:30:00.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.367+0000 7f236842f740 20 client.27148 trim_cache size 0 max 16384 2024-09-13T18:30:00.370 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:30:00.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.373+0000 7f236570f640 1 -- 172.21.15.114:0/3339129933 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f23480a1260 msgr2=0x7f23480a3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:00.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.373+0000 7f236570f640 1 --2- 172.21.15.114:0/3339129933 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f23480a1260 0x7f23480a3720 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x5630a09a6b60 tx=0x7f2358000f40 comp rx=0 tx=0).stop 2024-09-13T18:30:00.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.373+0000 7f236570f640 1 -- 172.21.15.114:0/3339129933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2360107540 msgr2=0x7f23601a5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:00.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.373+0000 7f236570f640 1 --2- 172.21.15.114:0/3339129933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2360107540 0x7f23601a5060 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2358006010 tx=0x7f2358003ce0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.382+0000 7f236570f640 1 -- 172.21.15.114:0/3339129933 shutdown_connections 2024-09-13T18:30:00.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.382+0000 7f236570f640 1 --2- 172.21.15.114:0/3339129933 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f23480a1260 0x7f23480a3720 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.382+0000 7f236570f640 1 --2- 172.21.15.114:0/3339129933 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2360119400 0x7f23601a9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.382+0000 7f236570f640 1 --2- 172.21.15.114:0/3339129933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2360107920 0x7f23601a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.382+0000 7f236570f640 1 --2- 172.21.15.114:0/3339129933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2360107540 0x7f23601a5060 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.382+0000 7f236570f640 1 -- 172.21.15.114:0/3339129933 >> 172.21.15.114:0/3339129933 conn(0x7f236006afe0 msgr2=0x7f2360119ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:00.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.382+0000 7f236570f640 1 -- 172.21.15.114:0/3339129933 shutdown_connections 2024-09-13T18:30:00.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.382+0000 7f236570f640 1 -- 172.21.15.114:0/3339129933 wait complete. 2024-09-13T18:30:00.406 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-09-13T18:30:00.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:29:59 smithi204 ceph-mon[34777]: pgmap v230: 129 pgs: 129 active+clean; 22 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 29 MiB/s wr, 640 op/s 2024-09-13T18:30:00.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:29:59 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/269360134' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:30:00.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.610+0000 7f10948e5640 1 -- 172.21.15.114:0/588423358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1090100900 msgr2=0x7f1090104940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:00.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.610+0000 7f10948e5640 1 --2- 172.21.15.114:0/588423358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1090100900 0x7f1090104940 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1084009900 tx=0x7f108402a3f0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.610+0000 7f10948e5640 1 -- 172.21.15.114:0/588423358 shutdown_connections 2024-09-13T18:30:00.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.610+0000 7f10948e5640 1 --2- 172.21.15.114:0/588423358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10901050a0 0x7f109010cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.610+0000 7f10948e5640 1 --2- 172.21.15.114:0/588423358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1090100900 0x7f1090104940 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.610+0000 7f10948e5640 1 --2- 172.21.15.114:0/588423358 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10900fff50 0x7f1090100330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.610+0000 7f10948e5640 1 -- 172.21.15.114:0/588423358 >> 172.21.15.114:0/588423358 conn(0x7f10900fb9e0 msgr2=0x7f10900fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:00.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.610+0000 7f10948e5640 1 -- 172.21.15.114:0/588423358 shutdown_connections 2024-09-13T18:30:00.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.611+0000 7f10948e5640 1 -- 172.21.15.114:0/588423358 wait complete. 2024-09-13T18:30:00.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.611+0000 7f10948e5640 1 Processor -- start 2024-09-13T18:30:00.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.611+0000 7f10948e5640 1 -- start start 2024-09-13T18:30:00.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.612+0000 7f10948e5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10900fff50 0x7f10901983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:00.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.612+0000 7f10948e5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10901050a0 0x7f10901988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:00.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.612+0000 7f10948e5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f109019ccd0 0x7f109019d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:00.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.612+0000 7f10948e5640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109019d6c0 con 0x7f109019ccd0 2024-09-13T18:30:00.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.612+0000 7f10948e5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109019d830 con 0x7f10901050a0 2024-09-13T18:30:00.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.612+0000 7f10948e5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109019d9a0 con 0x7f10900fff50 2024-09-13T18:30:00.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.612+0000 7f108e59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10900fff50 0x7f10901983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:00.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.612+0000 7f108e59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10900fff50 0x7f10901983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:58038/0 (socket says 172.21.15.114:58038) 2024-09-13T18:30:00.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.612+0000 7f108e59c640 1 -- 172.21.15.114:0/2642793820 learned_addr learned my addr 172.21.15.114:0/2642793820 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:00.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.612+0000 7f108dd9b640 1 --2- 172.21.15.114:0/2642793820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10901050a0 0x7f10901988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:00.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.612+0000 7f108ed9d640 1 --2- 172.21.15.114:0/2642793820 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f109019ccd0 0x7f109019d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:00.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.612+0000 7f108dd9b640 1 -- 172.21.15.114:0/2642793820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10900fff50 msgr2=0x7f10901983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:00.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.612+0000 7f108dd9b640 1 --2- 172.21.15.114:0/2642793820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10900fff50 0x7f10901983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.612+0000 7f108dd9b640 1 -- 172.21.15.114:0/2642793820 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f109019ccd0 msgr2=0x7f109019d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:00.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.612+0000 7f108dd9b640 1 --2- 172.21.15.114:0/2642793820 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f109019ccd0 0x7f109019d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.612+0000 7f108dd9b640 1 -- 172.21.15.114:0/2642793820 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1080009490 con 0x7f10901050a0 2024-09-13T18:30:00.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.613+0000 7f108dd9b640 1 --2- 172.21.15.114:0/2642793820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10901050a0 0x7f10901988e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1084009230 tx=0x7f1084005620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:00.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.613+0000 7f107f7fe640 1 -- 172.21.15.114:0/2642793820 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1084004bd0 con 0x7f10901050a0 2024-09-13T18:30:00.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.613+0000 7f107f7fe640 1 -- 172.21.15.114:0/2642793820 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1084004d30 con 0x7f10901050a0 2024-09-13T18:30:00.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.613+0000 7f10948e5640 1 -- 172.21.15.114:0/2642793820 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1088009840 con 0x7f10901050a0 2024-09-13T18:30:00.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.613+0000 7f107f7fe640 1 -- 172.21.15.114:0/2642793820 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108403c930 con 0x7f10901050a0 2024-09-13T18:30:00.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.614+0000 7f10948e5640 1 -- 172.21.15.114:0/2642793820 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1084009560 con 0x7f10901050a0 2024-09-13T18:30:00.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.617+0000 7f107f7fe640 1 -- 172.21.15.114:0/2642793820 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f1084003c80 con 0x7f10901050a0 2024-09-13T18:30:00.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.618+0000 7f107f7fe640 1 --2- 172.21.15.114:0/2642793820 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f10780a1350 0x7f10780a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:00.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.618+0000 7f107f7fe640 1 -- 172.21.15.114:0/2642793820 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f10840ee100 con 0x7f10901050a0 2024-09-13T18:30:00.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.618+0000 7f108e59c640 1 --2- 172.21.15.114:0/2642793820 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f10780a1350 0x7f10780a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:00.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.618+0000 7f1097601740 1 Processor -- start 2024-09-13T18:30:00.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.618+0000 7f1097601740 1 -- start start 2024-09-13T18:30:00.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.619+0000 7f1097601740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0ef6e8790 0x55e0ef6e8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:00.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.619+0000 7f1097601740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0ef6e90b0 0x55e0ef6f3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:00.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.619+0000 7f1097601740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0ef6f3950 0x55e0ef6f5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:00.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.619+0000 7f1097601740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0ef652cc0 con 0x55e0ef6f3950 2024-09-13T18:30:00.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.619+0000 7f1097601740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0ef650450 con 0x55e0ef6e8790 2024-09-13T18:30:00.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.619+0000 7f1097601740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0ef64bd40 con 0x55e0ef6e90b0 2024-09-13T18:30:00.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.619+0000 7f108dd9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0ef6e90b0 0x55e0ef6f3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:00.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.619+0000 7f108dd9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0ef6e90b0 0x55e0ef6f3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:58044/0 (socket says 172.21.15.114:58044) 2024-09-13T18:30:00.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.619+0000 7f108dd9b640 1 -- 172.21.15.114:0/775275331 learned_addr learned my addr 172.21.15.114:0/775275331 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:00.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.619+0000 7f108ed9d640 1 --2- 172.21.15.114:0/775275331 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0ef6f3950 0x55e0ef6f5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:00.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.620+0000 7f108dd9b640 1 -- 172.21.15.114:0/775275331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0ef6e8790 msgr2=0x55e0ef6e8b70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:30:00.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.620+0000 7f108dd9b640 1 --2- 172.21.15.114:0/775275331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0ef6e8790 0x55e0ef6e8b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.620+0000 7f108dd9b640 1 -- 172.21.15.114:0/775275331 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0ef6f3950 msgr2=0x55e0ef6f5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:00.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.620+0000 7f108dd9b640 1 --2- 172.21.15.114:0/775275331 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0ef6f3950 0x55e0ef6f5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.620+0000 7f108dd9b640 1 -- 172.21.15.114:0/775275331 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0ef613620 con 0x55e0ef6e90b0 2024-09-13T18:30:00.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.620+0000 7f108dd9b640 1 --2- 172.21.15.114:0/775275331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0ef6e90b0 0x55e0ef6f3410 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f10840b78e0 tx=0x7f10840ba7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=857749609f09ebd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:00.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.620+0000 7f108e59c640 1 --2- 172.21.15.114:0/2642793820 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f10780a1350 0x7f10780a3810 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1080009e60 tx=0x7f1080005f90 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:00.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.620+0000 7f107cff9640 1 -- 172.21.15.114:0/775275331 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10840beca0 con 0x55e0ef6e90b0 2024-09-13T18:30:00.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.620+0000 7f107cff9640 1 -- 172.21.15.114:0/775275331 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f10840c32f0 con 0x55e0ef6e90b0 2024-09-13T18:30:00.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.620+0000 7f107cff9640 1 -- 172.21.15.114:0/775275331 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10840bb330 con 0x55e0ef6e90b0 2024-09-13T18:30:00.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.622+0000 7f1097601740 1 -- 172.21.15.114:0/775275331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0ef6e90b0 msgr2=0x55e0ef6f3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:00.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.622+0000 7f1097601740 1 --2- 172.21.15.114:0/775275331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0ef6e90b0 0x55e0ef6f3410 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f10840b78e0 tx=0x7f10840ba7d0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.622+0000 7f1097601740 1 -- 172.21.15.114:0/775275331 shutdown_connections 2024-09-13T18:30:00.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.622+0000 7f1097601740 1 --2- 172.21.15.114:0/775275331 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0ef6f3950 0x55e0ef6f5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.622+0000 7f1097601740 1 --2- 172.21.15.114:0/775275331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0ef6e90b0 0x55e0ef6f3410 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.622+0000 7f1097601740 1 --2- 172.21.15.114:0/775275331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0ef6e8790 0x55e0ef6e8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.622+0000 7f1097601740 1 -- 172.21.15.114:0/775275331 >> 172.21.15.114:0/775275331 conn(0x55e0ef6e4d00 msgr2=0x55e0ef6e50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:00.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.622+0000 7f1097601740 1 -- 172.21.15.114:0/775275331 shutdown_connections 2024-09-13T18:30:00.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.622+0000 7f1097601740 1 -- 172.21.15.114:0/775275331 wait complete. 2024-09-13T18:30:00.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.623+0000 7f1097601740 1 -- start start 2024-09-13T18:30:00.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.623+0000 7f1097601740 1 Processor -- start 2024-09-13T18:30:00.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.623+0000 7f1097601740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0ef6e8790 0x55e0ef6f2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:00.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.623+0000 7f1097601740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0ef6e90b0 0x55e0ef6f01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:00.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.623+0000 7f1097601740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0ef6f3950 0x55e0ef6f0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:00.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.624+0000 7f1097601740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0ef60a180 con 0x55e0ef6e8790 2024-09-13T18:30:00.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.624+0000 7f1097601740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0ef324430 con 0x55e0ef6f3950 2024-09-13T18:30:00.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.624+0000 7f108ed9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0ef6f3950 0x55e0ef6f0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:00.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.624+0000 7f108ed9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0ef6f3950 0x55e0ef6f0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:43582/0 (socket says 172.21.15.114:43582) 2024-09-13T18:30:00.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.624+0000 7f108ed9d640 1 -- 172.21.15.114:0/1581587563 learned_addr learned my addr 172.21.15.114:0/1581587563 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:00.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.624+0000 7f1097601740 1 -- 172.21.15.114:0/1581587563 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0ef5cf5e0 con 0x55e0ef6e90b0 2024-09-13T18:30:00.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.624+0000 7f108dd9b640 1 --2- 172.21.15.114:0/1581587563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0ef6e90b0 0x55e0ef6f01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:00.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.624+0000 7f108e59c640 1 --2- 172.21.15.114:0/1581587563 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0ef6e8790 0x55e0ef6f2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:00.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.624+0000 7f108ed9d640 1 -- 172.21.15.114:0/1581587563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0ef6e90b0 msgr2=0x55e0ef6f01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:00.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.624+0000 7f108ed9d640 1 --2- 172.21.15.114:0/1581587563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0ef6e90b0 0x55e0ef6f01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.624+0000 7f108ed9d640 1 -- 172.21.15.114:0/1581587563 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0ef6e8790 msgr2=0x55e0ef6f2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:00.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.624+0000 7f108ed9d640 1 --2- 172.21.15.114:0/1581587563 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0ef6e8790 0x55e0ef6f2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.625+0000 7f108ed9d640 1 -- 172.21.15.114:0/1581587563 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1084009560 con 0x55e0ef6f3950 2024-09-13T18:30:00.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.624+0000 7f1097601740 10 client.0 _ll_register_callbacks cb 0x55e0ef63baa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:30:00.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.625+0000 7f1075ffb640 10 client.27161 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:30:00.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.625+0000 7f1097601740 1 -- 172.21.15.114:0/1581587563 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e0ef675b80 con 0x55e0ef6f3950 2024-09-13T18:30:00.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.625+0000 7f108ed9d640 1 --2- 172.21.15.114:0/1581587563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0ef6f3950 0x55e0ef6f0710 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1088011d10 tx=0x7f1088010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=309f4ea2134a5600 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:00.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.625+0000 7f1075ffb640 1 -- 172.21.15.114:0/1581587563 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1088016070 con 0x55e0ef6f3950 2024-09-13T18:30:00.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.625+0000 7f1075ffb640 1 -- 172.21.15.114:0/1581587563 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f1088003970 con 0x55e0ef6f3950 2024-09-13T18:30:00.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.625+0000 7f1075ffb640 1 -- 172.21.15.114:0/1581587563 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1088003bb0 con 0x55e0ef6f3950 2024-09-13T18:30:00.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.625+0000 7f1097601740 10 client.27161 fetch_fsmap learned FSMap version 34 2024-09-13T18:30:00.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.625+0000 7f1097601740 1 -- 172.21.15.114:0/1581587563 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({fsmap=34}) v3 -- 0x55e0ef64a3c0 con 0x55e0ef6f3950 2024-09-13T18:30:00.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.626+0000 7f1075ffb640 1 -- 172.21.15.114:0/1581587563 <== mon.2 v2:172.21.15.134:3300/0 4 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1088023410 con 0x55e0ef6f3950 2024-09-13T18:30:00.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.626+0000 7f1097601740 10 client.27161 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:30:00.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.626+0000 7f1097601740 10 client.27161 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-13T18:30:00.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.626+0000 7f1097601740 20 client.27161 populate_metadata read hostname 'smithi114' 2024-09-13T18:30:00.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.626+0000 7f1097601740 1 --2- 172.21.15.114:0/1581587563 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55e0ef702d70 0x55e0ef7051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:00.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.626+0000 7f1097601740 4 client.27161 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:30:00.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.626+0000 7f1097601740 1 -- 172.21.15.114:0/1581587563 --> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e0ef675b80 con 0x55e0ef702d70 2024-09-13T18:30:00.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.626+0000 7f108e59c640 1 --2- 172.21.15.114:0/1581587563 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55e0ef702d70 0x55e0ef7051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:00.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.627+0000 7f108e59c640 1 --2- 172.21.15.114:0/1581587563 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55e0ef702d70 0x55e0ef7051d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2a6db03164def1a1 server_cookie=2f73248022f6c0f8 in_seq=0 out_seq=0 2024-09-13T18:30:00.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.627+0000 7f1075ffb640 10 client.27161 ms_handle_connect on v2:172.21.15.134:6834/1150003275 2024-09-13T18:30:00.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.628+0000 7f1075ffb640 1 -- 172.21.15.114:0/1581587563 <== mds.3 v2:172.21.15.134:6834/1150003275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55e0ef675b80 con 0x55e0ef702d70 2024-09-13T18:30:00.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.628+0000 7f1075ffb640 10 client.27161 handle_command_reply: tid=0 2024-09-13T18:30:00.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.628+0000 7f1097601740 1 client.27161 shutdown 2024-09-13T18:30:00.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.628+0000 7f1097601740 1 -- 172.21.15.114:0/1581587563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0ef6f3950 msgr2=0x55e0ef6f0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:00.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.628+0000 7f1097601740 1 --2- 172.21.15.114:0/1581587563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0ef6f3950 0x55e0ef6f0710 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1088011d10 tx=0x7f1088010800 comp rx=0 tx=0).stop 2024-09-13T18:30:00.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.628+0000 7f1097601740 1 -- 172.21.15.114:0/1581587563 shutdown_connections 2024-09-13T18:30:00.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.628+0000 7f1097601740 1 --2- 172.21.15.114:0/1581587563 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55e0ef702d70 0x55e0ef7051d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.628+0000 7f1075ffb640 0 client.27161 ms_handle_reset on v2:172.21.15.134:6834/1150003275 2024-09-13T18:30:00.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.628+0000 7f1097601740 1 --2- 172.21.15.114:0/1581587563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0ef6f3950 0x55e0ef6f0710 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.628+0000 7f1097601740 1 --2- 172.21.15.114:0/1581587563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0ef6e90b0 0x55e0ef6f01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.628+0000 7f1097601740 1 --2- 172.21.15.114:0/1581587563 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0ef6e8790 0x55e0ef6f2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.628+0000 7f1097601740 1 -- 172.21.15.114:0/1581587563 >> 172.21.15.114:0/1581587563 conn(0x55e0ef6e4d00 msgr2=0x55e0ef6e3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:00.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.629+0000 7f1097601740 1 -- 172.21.15.114:0/1581587563 shutdown_connections 2024-09-13T18:30:00.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.629+0000 7f1097601740 1 -- 172.21.15.114:0/1581587563 wait complete. 2024-09-13T18:30:00.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.629+0000 7f1097601740 20 client.27161 trim_cache size 0 max 16384 2024-09-13T18:30:00.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.656+0000 7f1097601740 1 Processor -- start 2024-09-13T18:30:00.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.659+0000 7f1097601740 1 -- start start 2024-09-13T18:30:00.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.659+0000 7f1097601740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0ef6e8790 0x55e0ef6e46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:00.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.659+0000 7f1097601740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0ef6e90b0 0x55e0ef6fa390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:00.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.659+0000 7f1097601740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0ef6f3950 0x55e0ef6fa8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:00.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.659+0000 7f1097601740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0ef6ebe90 con 0x55e0ef6e8790 2024-09-13T18:30:00.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.659+0000 7f1097601740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0ef6e5c60 con 0x55e0ef6f3950 2024-09-13T18:30:00.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.659+0000 7f1097601740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0ef6e7fb0 con 0x55e0ef6e90b0 2024-09-13T18:30:00.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.661+0000 7f108ed9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0ef6f3950 0x55e0ef6fa8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:00.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.661+0000 7f108ed9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0ef6f3950 0x55e0ef6fa8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:43592/0 (socket says 172.21.15.114:43592) 2024-09-13T18:30:00.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.661+0000 7f108ed9d640 1 -- 172.21.15.114:0/2882450919 learned_addr learned my addr 172.21.15.114:0/2882450919 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:00.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.662+0000 7f108dd9b640 1 --2- 172.21.15.114:0/2882450919 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0ef6e90b0 0x55e0ef6fa390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:00.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.663+0000 7f108e59c640 1 --2- 172.21.15.114:0/2882450919 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0ef6e8790 0x55e0ef6e46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:00.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.663+0000 7f108e59c640 1 -- 172.21.15.114:0/2882450919 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0ef6e90b0 msgr2=0x55e0ef6fa390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:00.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.663+0000 7f108e59c640 1 --2- 172.21.15.114:0/2882450919 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0ef6e90b0 0x55e0ef6fa390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.663+0000 7f108e59c640 1 -- 172.21.15.114:0/2882450919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0ef6f3950 msgr2=0x55e0ef6fa8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:00.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.663+0000 7f108e59c640 1 --2- 172.21.15.114:0/2882450919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0ef6f3950 0x55e0ef6fa8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.663+0000 7f108e59c640 1 -- 172.21.15.114:0/2882450919 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0ef5ec770 con 0x55e0ef6e8790 2024-09-13T18:30:00.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.663+0000 7f108e59c640 1 --2- 172.21.15.114:0/2882450919 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0ef6e8790 0x55e0ef6e46a0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f1080013010 tx=0x7f1080014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe35eca093b0a06f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:00.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.664+0000 7f107d7fa640 1 -- 172.21.15.114:0/2882450919 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108002a070 con 0x55e0ef6e8790 2024-09-13T18:30:00.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.664+0000 7f107d7fa640 1 -- 172.21.15.114:0/2882450919 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1080025760 con 0x55e0ef6e8790 2024-09-13T18:30:00.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.665+0000 7f1097601740 1 -- 172.21.15.114:0/2882450919 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0ef6e8790 msgr2=0x55e0ef6e46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:00.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.665+0000 7f1097601740 1 --2- 172.21.15.114:0/2882450919 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0ef6e8790 0x55e0ef6e46a0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f1080013010 tx=0x7f1080014000 comp rx=0 tx=0).stop 2024-09-13T18:30:00.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.665+0000 7f1097601740 1 -- 172.21.15.114:0/2882450919 shutdown_connections 2024-09-13T18:30:00.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.665+0000 7f1097601740 1 --2- 172.21.15.114:0/2882450919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0ef6f3950 0x55e0ef6fa8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.665+0000 7f1097601740 1 --2- 172.21.15.114:0/2882450919 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0ef6e90b0 0x55e0ef6fa390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.665+0000 7f1097601740 1 --2- 172.21.15.114:0/2882450919 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0ef6e8790 0x55e0ef6e46a0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.665+0000 7f1097601740 1 -- 172.21.15.114:0/2882450919 >> 172.21.15.114:0/2882450919 conn(0x55e0ef6e4d00 msgr2=0x55e0ef6eb270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:00.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.665+0000 7f1097601740 1 -- 172.21.15.114:0/2882450919 shutdown_connections 2024-09-13T18:30:00.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.665+0000 7f1097601740 1 -- 172.21.15.114:0/2882450919 wait complete. 2024-09-13T18:30:00.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.666+0000 7f1097601740 1 -- start start 2024-09-13T18:30:00.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.666+0000 7f1097601740 1 Processor -- start 2024-09-13T18:30:00.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.666+0000 7f1097601740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0ef6e8790 0x55e0ef6f6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:00.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.666+0000 7f1097601740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0ef6e90b0 0x55e0ef6f6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:00.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.666+0000 7f1097601740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0ef6f3950 0x55e0ef6f9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:00.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.666+0000 7f1097601740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0ef6fa260 con 0x55e0ef6e90b0 2024-09-13T18:30:00.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.666+0000 7f1097601740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0ef6fa3d0 con 0x55e0ef6f3950 2024-09-13T18:30:00.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.666+0000 7f1097601740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0ef6fa590 con 0x55e0ef6e8790 2024-09-13T18:30:00.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.666+0000 7f1097601740 10 client.0 _ll_register_callbacks cb 0x7f1084005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:30:00.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.667+0000 7f108dd9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0ef6e90b0 0x55e0ef6f6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:00.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.667+0000 7f108dd9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0ef6e90b0 0x55e0ef6f6e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44122/0 (socket says 172.21.15.114:44122) 2024-09-13T18:30:00.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.667+0000 7f108dd9b640 1 -- 172.21.15.114:0/3788431704 learned_addr learned my addr 172.21.15.114:0/3788431704 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:00.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.668+0000 7f108dd9b640 1 -- 172.21.15.114:0/3788431704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0ef6e8790 msgr2=0x55e0ef6f6950 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:30:00.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.668+0000 7f108dd9b640 1 --2- 172.21.15.114:0/3788431704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0ef6e8790 0x55e0ef6f6950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.668+0000 7f108dd9b640 1 -- 172.21.15.114:0/3788431704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0ef6f3950 msgr2=0x55e0ef6f9cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:30:00.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.668+0000 7f108dd9b640 1 --2- 172.21.15.114:0/3788431704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0ef6f3950 0x55e0ef6f9cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.668+0000 7f108dd9b640 1 -- 172.21.15.114:0/3788431704 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1080017050 con 0x55e0ef6e90b0 2024-09-13T18:30:00.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.668+0000 7f108dd9b640 1 --2- 172.21.15.114:0/3788431704 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0ef6e90b0 0x55e0ef6f6e90 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f1084005ec0 tx=0x7f1084004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df41bbde2c269f50 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:00.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.668+0000 7f1075ffb640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:30:00.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.669+0000 7f1075ffb640 1 -- 172.21.15.114:0/3788431704 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10840beca0 con 0x55e0ef6e90b0 2024-09-13T18:30:00.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.669+0000 7f1075ffb640 1 -- 172.21.15.114:0/3788431704 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f10840ba360 con 0x55e0ef6e90b0 2024-09-13T18:30:00.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.672+0000 7f1097601740 1 -- 172.21.15.114:0/3788431704 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f10840c32f0 con 0x55e0ef6e90b0 2024-09-13T18:30:00.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.675+0000 7f1075ffb640 1 -- 172.21.15.114:0/3788431704 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10840ba520 con 0x55e0ef6e90b0 2024-09-13T18:30:00.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.682+0000 7f1097601740 10 client.17754 fetch_fsmap learned FSMap version 34 2024-09-13T18:30:00.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.682+0000 7f1097601740 1 -- 172.21.15.114:0/3788431704 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55e0ef611e80 con 0x55e0ef6e90b0 2024-09-13T18:30:00.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.683+0000 7f1075ffb640 1 -- 172.21.15.114:0/3788431704 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f10840c32f0 con 0x55e0ef6e90b0 2024-09-13T18:30:00.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.684+0000 7f1075ffb640 1 -- 172.21.15.114:0/3788431704 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f108404d070 con 0x55e0ef6e90b0 2024-09-13T18:30:00.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.686+0000 7f1097601740 10 client.17754 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:30:00.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.686+0000 7f1097601740 10 client.17754 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-13T18:30:00.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.686+0000 7f1097601740 20 client.17754 populate_metadata read hostname 'smithi114' 2024-09-13T18:30:00.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.686+0000 7f1097601740 1 --2- 172.21.15.114:0/3788431704 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55e0ef702d70 0x55e0ef76cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:00.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.686+0000 7f1097601740 4 client.17754 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:30:00.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.686+0000 7f1097601740 1 -- 172.21.15.114:0/3788431704 --> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e0ef63e1f0 con 0x55e0ef702d70 2024-09-13T18:30:00.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.693+0000 7f108e59c640 1 --2- 172.21.15.114:0/3788431704 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55e0ef702d70 0x55e0ef76cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:00.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.696+0000 7f108e59c640 1 --2- 172.21.15.114:0/3788431704 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55e0ef702d70 0x55e0ef76cfb0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=91d030fc2f417df4 server_cookie=b1320f65d65c686f in_seq=0 out_seq=0 2024-09-13T18:30:00.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.696+0000 7f1075ffb640 10 client.17754 ms_handle_connect on v2:172.21.15.134:6834/1150003275 2024-09-13T18:30:00.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.698+0000 7f1075ffb640 1 -- 172.21.15.114:0/3788431704 <== mds.3 v2:172.21.15.134:6834/1150003275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e0ef63e1f0 con 0x55e0ef702d70 2024-09-13T18:30:00.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.698+0000 7f1075ffb640 10 client.17754 handle_command_reply: tid=0 2024-09-13T18:30:00.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.698+0000 7f1097601740 1 client.17754 shutdown 2024-09-13T18:30:00.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.701+0000 7f1097601740 1 -- 172.21.15.114:0/3788431704 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0ef6e90b0 msgr2=0x55e0ef6f6e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:00.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.701+0000 7f1097601740 1 --2- 172.21.15.114:0/3788431704 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0ef6e90b0 0x55e0ef6f6e90 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f1084005ec0 tx=0x7f1084004f90 comp rx=0 tx=0).stop 2024-09-13T18:30:00.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.701+0000 7f1097601740 1 -- 172.21.15.114:0/3788431704 shutdown_connections 2024-09-13T18:30:00.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.701+0000 7f1097601740 1 --2- 172.21.15.114:0/3788431704 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55e0ef702d70 0x55e0ef76cfb0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.701+0000 7f1097601740 1 --2- 172.21.15.114:0/3788431704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0ef6f3950 0x55e0ef6f9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.701+0000 7f1097601740 1 --2- 172.21.15.114:0/3788431704 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0ef6e90b0 0x55e0ef6f6e90 secure :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f1084005ec0 tx=0x7f1084004f90 comp rx=0 tx=0).stop 2024-09-13T18:30:00.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.701+0000 7f1097601740 1 --2- 172.21.15.114:0/3788431704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e0ef6e8790 0x55e0ef6f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.701+0000 7f1097601740 1 -- 172.21.15.114:0/3788431704 >> 172.21.15.114:0/3788431704 conn(0x55e0ef6e4d00 msgr2=0x55e0ef6e3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:00.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.702+0000 7f1075ffb640 0 client.17754 ms_handle_reset on v2:172.21.15.134:6834/1150003275 2024-09-13T18:30:00.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.706+0000 7f1097601740 1 -- 172.21.15.114:0/3788431704 shutdown_connections 2024-09-13T18:30:00.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.708+0000 7f1097601740 1 -- 172.21.15.114:0/3788431704 wait complete. 2024-09-13T18:30:00.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.708+0000 7f1097601740 20 client.17754 trim_cache size 0 max 16384 2024-09-13T18:30:00.712 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:30:00.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.709+0000 7f10948e5640 1 -- 172.21.15.114:0/2642793820 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f10780a1350 msgr2=0x7f10780a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:00.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.709+0000 7f10948e5640 1 --2- 172.21.15.114:0/2642793820 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f10780a1350 0x7f10780a3810 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1080009e60 tx=0x7f1080005f90 comp rx=0 tx=0).stop 2024-09-13T18:30:00.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.709+0000 7f10948e5640 1 -- 172.21.15.114:0/2642793820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10901050a0 msgr2=0x7f10901988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:00.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.709+0000 7f10948e5640 1 --2- 172.21.15.114:0/2642793820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10901050a0 0x7f10901988e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1084009230 tx=0x7f1084005620 comp rx=0 tx=0).stop 2024-09-13T18:30:00.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.712+0000 7f10948e5640 1 -- 172.21.15.114:0/2642793820 shutdown_connections 2024-09-13T18:30:00.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.712+0000 7f10948e5640 1 --2- 172.21.15.114:0/2642793820 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f10780a1350 0x7f10780a3810 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.712+0000 7f10948e5640 1 --2- 172.21.15.114:0/2642793820 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f109019ccd0 0x7f109019d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.712+0000 7f10948e5640 1 --2- 172.21.15.114:0/2642793820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10901050a0 0x7f10901988e0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.712+0000 7f10948e5640 1 --2- 172.21.15.114:0/2642793820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10900fff50 0x7f10901983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:00.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.712+0000 7f10948e5640 1 -- 172.21.15.114:0/2642793820 >> 172.21.15.114:0/2642793820 conn(0x7f10900fb9e0 msgr2=0x7f10900fcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:00.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.716+0000 7f10948e5640 1 -- 172.21.15.114:0/2642793820 shutdown_connections 2024-09-13T18:30:00.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.716+0000 7f10948e5640 1 -- 172.21.15.114:0/2642793820 wait complete. 2024-09-13T18:30:00.759 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-09-13T18:30:00.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.996+0000 7f411bbaa640 1 -- 172.21.15.114:0/4117521702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41141074b0 msgr2=0x7f411410e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:00.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.996+0000 7f411bbaa640 1 --2- 172.21.15.114:0/4117521702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41141074b0 0x7f411410e510 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f410c009cd0 tx=0x7f410c02a930 comp rx=0 tx=0).stop 2024-09-13T18:30:01.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.998+0000 7f411bbaa640 1 -- 172.21.15.114:0/4117521702 shutdown_connections 2024-09-13T18:30:01.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.998+0000 7f411bbaa640 1 --2- 172.21.15.114:0/4117521702 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4114119400 0x7f411411b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.998+0000 7f411bbaa640 1 --2- 172.21.15.114:0/4117521702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4114107890 0x7f4114107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.998+0000 7f411bbaa640 1 --2- 172.21.15.114:0/4117521702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41141074b0 0x7f411410e510 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:00.998+0000 7f411bbaa640 1 -- 172.21.15.114:0/4117521702 >> 172.21.15.114:0/4117521702 conn(0x7f411406af70 msgr2=0x7f411406b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:01.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.007+0000 7f411bbaa640 1 -- 172.21.15.114:0/4117521702 shutdown_connections 2024-09-13T18:30:01.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.014+0000 7f411bbaa640 1 -- 172.21.15.114:0/4117521702 wait complete. 2024-09-13T18:30:01.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.014+0000 7f411bbaa640 1 Processor -- start 2024-09-13T18:30:01.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.015+0000 7f411bbaa640 1 -- start start 2024-09-13T18:30:01.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.015+0000 7f411bbaa640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41141074b0 0x7f4114116960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:01.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.015+0000 7f411bbaa640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4114107890 0x7f4114116ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:01.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.015+0000 7f411bbaa640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4114119400 0x7f411410fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:01.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.015+0000 7f411bbaa640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41141102e0 con 0x7f4114119400 2024-09-13T18:30:01.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.015+0000 7f411bbaa640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4114110450 con 0x7f4114107890 2024-09-13T18:30:01.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.015+0000 7f411bbaa640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4114110610 con 0x7f41141074b0 2024-09-13T18:30:01.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.015+0000 7f4119946640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41141074b0 0x7f4114116960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:01.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.015+0000 7f4119946640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41141074b0 0x7f4114116960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:58090/0 (socket says 172.21.15.114:58090) 2024-09-13T18:30:01.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.015+0000 7f4119946640 1 -- 172.21.15.114:0/3351988708 learned_addr learned my addr 172.21.15.114:0/3351988708 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:01.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.016+0000 7f411a147640 1 --2- 172.21.15.114:0/3351988708 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4114119400 0x7f411410fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:01.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.016+0000 7f4119946640 1 -- 172.21.15.114:0/3351988708 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4114107890 msgr2=0x7f4114116ea0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:30:01.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.016+0000 7f4119946640 1 --2- 172.21.15.114:0/3351988708 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4114107890 0x7f4114116ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.016+0000 7f4119946640 1 -- 172.21.15.114:0/3351988708 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4114119400 msgr2=0x7f411410fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:01.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.016+0000 7f4119946640 1 --2- 172.21.15.114:0/3351988708 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4114119400 0x7f411410fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.016+0000 7f4119946640 1 -- 172.21.15.114:0/3351988708 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f410c009900 con 0x7f41141074b0 2024-09-13T18:30:01.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.016+0000 7f411a147640 1 --2- 172.21.15.114:0/3351988708 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4114119400 0x7f411410fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:30:01.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.016+0000 7f4119946640 1 --2- 172.21.15.114:0/3351988708 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41141074b0 0x7f4114116960 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f410c0098d0 tx=0x7f410c030b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:01.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.017+0000 7f410affd640 1 -- 172.21.15.114:0/3351988708 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c02fa10 con 0x7f41141074b0 2024-09-13T18:30:01.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.017+0000 7f411bbaa640 1 -- 172.21.15.114:0/3351988708 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4114114480 con 0x7f41141074b0 2024-09-13T18:30:01.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.017+0000 7f411bbaa640 1 -- 172.21.15.114:0/3351988708 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41141149e0 con 0x7f41141074b0 2024-09-13T18:30:01.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.018+0000 7f410affd640 1 -- 172.21.15.114:0/3351988708 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f410c03f070 con 0x7f41141074b0 2024-09-13T18:30:01.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.019+0000 7f410affd640 1 -- 172.21.15.114:0/3351988708 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c03ba40 con 0x7f41141074b0 2024-09-13T18:30:01.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.021+0000 7f411e8cd740 1 Processor -- start 2024-09-13T18:30:01.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.023+0000 7f411e8cd740 1 -- start start 2024-09-13T18:30:01.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.023+0000 7f411e8cd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe9d1c6970 0x55fe9d1c6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:01.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.023+0000 7f411e8cd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe9d1c7290 0x55fe9d1d15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:01.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.023+0000 7f411e8cd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe9d1d1b30 0x55fe9d1d3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:01.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.023+0000 7f411e8cd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe9d130cc0 con 0x55fe9d1c7290 2024-09-13T18:30:01.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.023+0000 7f411e8cd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe9d12e450 con 0x55fe9d1c6970 2024-09-13T18:30:01.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.023+0000 7f411e8cd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe9d129d40 con 0x55fe9d1d1b30 2024-09-13T18:30:01.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.023+0000 7f4119946640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe9d1c7290 0x55fe9d1d15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:01.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.024+0000 7f4119946640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe9d1c7290 0x55fe9d1d15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44152/0 (socket says 172.21.15.114:44152) 2024-09-13T18:30:01.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.024+0000 7f4119946640 1 -- 172.21.15.114:0/454084356 learned_addr learned my addr 172.21.15.114:0/454084356 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:01.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.024+0000 7f411a147640 1 --2- 172.21.15.114:0/454084356 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe9d1c6970 0x55fe9d1c6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:01.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.024+0000 7f411a147640 1 -- 172.21.15.114:0/454084356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe9d1d1b30 msgr2=0x55fe9d1d3f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:30:01.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.024+0000 7f411a147640 1 --2- 172.21.15.114:0/454084356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe9d1d1b30 0x55fe9d1d3f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.024+0000 7f411a147640 1 -- 172.21.15.114:0/454084356 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe9d1c7290 msgr2=0x55fe9d1d15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:01.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.024+0000 7f411a147640 1 --2- 172.21.15.114:0/454084356 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe9d1c7290 0x55fe9d1d15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.024+0000 7f411a147640 1 -- 172.21.15.114:0/454084356 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe9d0f1620 con 0x55fe9d1c6970 2024-09-13T18:30:01.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.025+0000 7f411a147640 1 --2- 172.21.15.114:0/454084356 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe9d1c6970 0x55fe9d1c6d50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f410400a8c0 tx=0x7f410400ad80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29155728cbc598a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:01.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.026+0000 7f410affd640 1 -- 172.21.15.114:0/3351988708 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f410c00a040 con 0x7f41141074b0 2024-09-13T18:30:01.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.026+0000 7f4119946640 1 --2- 172.21.15.114:0/454084356 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe9d1c7290 0x55fe9d1d15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:30:01.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.027+0000 7f410affd640 1 --2- 172.21.15.114:0/3351988708 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f41000a1340 0x7f41000a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:01.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.028+0000 7f410affd640 1 -- 172.21.15.114:0/3351988708 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f410c041070 con 0x7f41141074b0 2024-09-13T18:30:01.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.028+0000 7f4119946640 1 --2- 172.21.15.114:0/3351988708 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f41000a1340 0x7f41000a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:01.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.028+0000 7f40fffff640 1 -- 172.21.15.114:0/454084356 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410400cb40 con 0x55fe9d1c6970 2024-09-13T18:30:01.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.029+0000 7f40fffff640 1 -- 172.21.15.114:0/454084356 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f410400cca0 con 0x55fe9d1c6970 2024-09-13T18:30:01.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.029+0000 7f40fffff640 1 -- 172.21.15.114:0/454084356 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41040185c0 con 0x55fe9d1c6970 2024-09-13T18:30:01.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.029+0000 7f4119946640 1 --2- 172.21.15.114:0/3351988708 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f41000a1340 0x7f41000a3800 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f410c0b7ff0 tx=0x7f410c0db040 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:01.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.030+0000 7f411e8cd740 1 -- 172.21.15.114:0/454084356 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe9d1c6970 msgr2=0x55fe9d1c6d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:01.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.030+0000 7f411e8cd740 1 --2- 172.21.15.114:0/454084356 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe9d1c6970 0x55fe9d1c6d50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f410400a8c0 tx=0x7f410400ad80 comp rx=0 tx=0).stop 2024-09-13T18:30:01.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.034+0000 7f411e8cd740 1 -- 172.21.15.114:0/454084356 shutdown_connections 2024-09-13T18:30:01.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.034+0000 7f411e8cd740 1 --2- 172.21.15.114:0/454084356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe9d1d1b30 0x55fe9d1d3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.034+0000 7f411e8cd740 1 --2- 172.21.15.114:0/454084356 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe9d1c7290 0x55fe9d1d15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.034+0000 7f411e8cd740 1 --2- 172.21.15.114:0/454084356 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe9d1c6970 0x55fe9d1c6d50 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.034+0000 7f411e8cd740 1 -- 172.21.15.114:0/454084356 >> 172.21.15.114:0/454084356 conn(0x55fe9d1c2ee0 msgr2=0x55fe9d1c32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:01.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.043+0000 7f411e8cd740 1 -- 172.21.15.114:0/454084356 shutdown_connections 2024-09-13T18:30:01.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.044+0000 7f411e8cd740 1 -- 172.21.15.114:0/454084356 wait complete. 2024-09-13T18:30:01.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.044+0000 7f411e8cd740 1 -- start start 2024-09-13T18:30:01.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.044+0000 7f411e8cd740 1 Processor -- start 2024-09-13T18:30:01.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.045+0000 7f411e8cd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe9d1c6970 0x55fe9d1d10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:01.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.045+0000 7f411e8cd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe9d1c7290 0x55fe9d1ce400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:01.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.045+0000 7f411e8cd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe9d1d1b30 0x55fe9d1ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:01.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.045+0000 7f411e8cd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe9d0e8180 con 0x55fe9d1c6970 2024-09-13T18:30:01.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.045+0000 7f411e8cd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe9ce02430 con 0x55fe9d1d1b30 2024-09-13T18:30:01.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.045+0000 7f411e8cd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe9d0ad5e0 con 0x55fe9d1c7290 2024-09-13T18:30:01.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.045+0000 7f411e8cd740 10 client.0 _ll_register_callbacks cb 0x55fe9d119aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:30:01.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.045+0000 7f411a147640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe9d1c6970 0x55fe9d1d10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:01.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.045+0000 7f411a147640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe9d1c6970 0x55fe9d1d10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44162/0 (socket says 172.21.15.114:44162) 2024-09-13T18:30:01.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.045+0000 7f411a147640 1 -- 172.21.15.114:0/1733976279 learned_addr learned my addr 172.21.15.114:0/1733976279 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:01.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.045+0000 7f411a147640 1 --2- 172.21.15.114:0/1733976279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe9d1d1b30 0x55fe9d1ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:01.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.046+0000 7f411a147640 1 -- 172.21.15.114:0/1733976279 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe9d1c7290 msgr2=0x55fe9d1ce400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:01.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.046+0000 7f4119145640 1 --2- 172.21.15.114:0/1733976279 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe9d1c7290 0x55fe9d1ce400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:01.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.046+0000 7f411a147640 1 --2- 172.21.15.114:0/1733976279 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe9d1c7290 0x55fe9d1ce400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.046+0000 7f411a147640 1 -- 172.21.15.114:0/1733976279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe9d1d1b30 msgr2=0x55fe9d1ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:01.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.046+0000 7f411a147640 1 --2- 172.21.15.114:0/1733976279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe9d1d1b30 0x55fe9d1ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.046+0000 7f411a147640 1 -- 172.21.15.114:0/1733976279 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f410400a4f0 con 0x55fe9d1c6970 2024-09-13T18:30:01.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.046+0000 7f4119145640 1 --2- 172.21.15.114:0/1733976279 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe9d1c7290 0x55fe9d1ce400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:30:01.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.046+0000 7f411a147640 1 --2- 172.21.15.114:0/1733976279 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe9d1c6970 0x55fe9d1d10c0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f410400d010 tx=0x7f4104011df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b303a1509a9b9795 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:01.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.047+0000 7f40fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:30:01.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.050+0000 7f40fd7fa640 1 -- 172.21.15.114:0/1733976279 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410401a650 con 0x55fe9d1c6970 2024-09-13T18:30:01.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.050+0000 7f40fd7fa640 1 -- 172.21.15.114:0/1733976279 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f410400ce50 con 0x55fe9d1c6970 2024-09-13T18:30:01.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.051+0000 7f411e8cd740 1 -- 172.21.15.114:0/1733976279 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fe9d153b80 con 0x55fe9d1c6970 2024-09-13T18:30:01.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.052+0000 7f40fd7fa640 1 -- 172.21.15.114:0/1733976279 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f410400cac0 con 0x55fe9d1c6970 2024-09-13T18:30:01.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.052+0000 7f411e8cd740 10 client.17760 fetch_fsmap learned FSMap version 34 2024-09-13T18:30:01.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.052+0000 7f411e8cd740 1 -- 172.21.15.114:0/1733976279 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55fe9d1283c0 con 0x55fe9d1c6970 2024-09-13T18:30:01.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.055+0000 7f40fd7fa640 1 -- 172.21.15.114:0/1733976279 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55fe9d153b80 con 0x55fe9d1c6970 2024-09-13T18:30:01.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.055+0000 7f40fd7fa640 1 -- 172.21.15.114:0/1733976279 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4104030070 con 0x55fe9d1c6970 2024-09-13T18:30:01.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.056+0000 7f411e8cd740 10 client.17760 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:30:01.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.056+0000 7f411e8cd740 10 client.17760 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-13T18:30:01.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.056+0000 7f411e8cd740 20 client.17760 populate_metadata read hostname 'smithi114' 2024-09-13T18:30:01.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.057+0000 7f411e8cd740 1 --2- 172.21.15.114:0/1733976279 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55fe9d1e14b0 0x55fe9d1e38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:01.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.057+0000 7f411e8cd740 4 client.17760 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:30:01.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.057+0000 7f411e8cd740 1 -- 172.21.15.114:0/1733976279 --> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fe9d11c1f0 con 0x55fe9d1e14b0 2024-09-13T18:30:01.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.057+0000 7f4119145640 1 --2- 172.21.15.114:0/1733976279 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55fe9d1e14b0 0x55fe9d1e38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:01.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.066+0000 7f4119145640 1 --2- 172.21.15.114:0/1733976279 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55fe9d1e14b0 0x55fe9d1e38d0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2f6acedadcba903e server_cookie=8c8457cfd7332dce in_seq=0 out_seq=0 2024-09-13T18:30:01.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.066+0000 7f40fd7fa640 10 client.17760 ms_handle_connect on v2:172.21.15.204:6838/344883912 2024-09-13T18:30:01.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.067+0000 7f40fd7fa640 1 -- 172.21.15.114:0/1733976279 <== mds.2 v2:172.21.15.204:6838/344883912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55fe9d11c1f0 con 0x55fe9d1e14b0 2024-09-13T18:30:01.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.068+0000 7f40fd7fa640 10 client.17760 handle_command_reply: tid=0 2024-09-13T18:30:01.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.070+0000 7f411e8cd740 1 client.17760 shutdown 2024-09-13T18:30:01.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.075+0000 7f411e8cd740 1 -- 172.21.15.114:0/1733976279 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe9d1c6970 msgr2=0x55fe9d1d10c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:01.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.075+0000 7f411e8cd740 1 --2- 172.21.15.114:0/1733976279 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe9d1c6970 0x55fe9d1d10c0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f410400d010 tx=0x7f4104011df0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.075+0000 7f411e8cd740 1 -- 172.21.15.114:0/1733976279 shutdown_connections 2024-09-13T18:30:01.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.075+0000 7f411e8cd740 1 --2- 172.21.15.114:0/1733976279 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55fe9d1e14b0 0x55fe9d1e38d0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.075+0000 7f411e8cd740 1 --2- 172.21.15.114:0/1733976279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe9d1d1b30 0x55fe9d1ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.075+0000 7f411e8cd740 1 --2- 172.21.15.114:0/1733976279 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe9d1c7290 0x55fe9d1ce400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.075+0000 7f411e8cd740 1 --2- 172.21.15.114:0/1733976279 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe9d1c6970 0x55fe9d1d10c0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.075+0000 7f411e8cd740 1 -- 172.21.15.114:0/1733976279 >> 172.21.15.114:0/1733976279 conn(0x55fe9d1c2ee0 msgr2=0x55fe9d1c2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:01.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.075+0000 7f40fd7fa640 0 client.17760 ms_handle_reset on v2:172.21.15.204:6838/344883912 2024-09-13T18:30:01.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.076+0000 7f411e8cd740 1 -- 172.21.15.114:0/1733976279 shutdown_connections 2024-09-13T18:30:01.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.076+0000 7f411e8cd740 1 -- 172.21.15.114:0/1733976279 wait complete. 2024-09-13T18:30:01.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.077+0000 7f411e8cd740 20 client.17760 trim_cache size 0 max 16384 2024-09-13T18:30:01.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.109+0000 7f411e8cd740 1 Processor -- start 2024-09-13T18:30:01.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.112+0000 7f411e8cd740 1 -- start start 2024-09-13T18:30:01.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.113+0000 7f411e8cd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe9d1c6970 0x55fe9d1d8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:01.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.113+0000 7f411e8cd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe9d1c7290 0x55fe9d1d8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:01.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.113+0000 7f411e8cd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe9d1d1b30 0x55fe9d1cccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:01.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.113+0000 7f411e8cd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe9d1cecb0 con 0x55fe9d1c6970 2024-09-13T18:30:01.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.113+0000 7f411e8cd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe9d1ddb60 con 0x55fe9d1d1b30 2024-09-13T18:30:01.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.113+0000 7f411e8cd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe9d1cf510 con 0x55fe9d1c7290 2024-09-13T18:30:01.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.116+0000 7f4119145640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe9d1c7290 0x55fe9d1d8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:01.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.116+0000 7f4119145640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe9d1c7290 0x55fe9d1d8dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:58108/0 (socket says 172.21.15.114:58108) 2024-09-13T18:30:01.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.116+0000 7f4119145640 1 -- 172.21.15.114:0/2797658083 learned_addr learned my addr 172.21.15.114:0/2797658083 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:01.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.119+0000 7f411a147640 1 --2- 172.21.15.114:0/2797658083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe9d1d1b30 0x55fe9d1cccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:01.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.119+0000 7f411a147640 1 --2- 172.21.15.114:0/2797658083 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe9d1c6970 0x55fe9d1d8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:01.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.119+0000 7f411a147640 1 -- 172.21.15.114:0/2797658083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe9d1c7290 msgr2=0x55fe9d1d8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:01.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.119+0000 7f411a147640 1 --2- 172.21.15.114:0/2797658083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe9d1c7290 0x55fe9d1d8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.119+0000 7f411a147640 1 -- 172.21.15.114:0/2797658083 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe9d1c6970 msgr2=0x55fe9d1d8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:01.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.119+0000 7f411a147640 1 --2- 172.21.15.114:0/2797658083 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe9d1c6970 0x55fe9d1d8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.119+0000 7f411a147640 1 -- 172.21.15.114:0/2797658083 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe9d0ca770 con 0x55fe9d1d1b30 2024-09-13T18:30:01.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.120+0000 7f411a147640 1 --2- 172.21.15.114:0/2797658083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe9d1d1b30 0x55fe9d1cccd0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4104000c00 tx=0x7f410400ad80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c461c7fa4d9c1631 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:01.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.120+0000 7f4108e05640 1 -- 172.21.15.114:0/2797658083 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4104028a70 con 0x55fe9d1d1b30 2024-09-13T18:30:01.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.121+0000 7f4108e05640 1 -- 172.21.15.114:0/2797658083 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f410401e040 con 0x55fe9d1d1b30 2024-09-13T18:30:01.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.122+0000 7f411e8cd740 1 -- 172.21.15.114:0/2797658083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe9d1d1b30 msgr2=0x55fe9d1cccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:01.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.122+0000 7f411e8cd740 1 --2- 172.21.15.114:0/2797658083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe9d1d1b30 0x55fe9d1cccd0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4104000c00 tx=0x7f410400ad80 comp rx=0 tx=0).stop 2024-09-13T18:30:01.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.123+0000 7f411e8cd740 1 -- 172.21.15.114:0/2797658083 shutdown_connections 2024-09-13T18:30:01.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.123+0000 7f411e8cd740 1 --2- 172.21.15.114:0/2797658083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe9d1d1b30 0x55fe9d1cccd0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.123+0000 7f411e8cd740 1 --2- 172.21.15.114:0/2797658083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe9d1c7290 0x55fe9d1d8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.123+0000 7f411e8cd740 1 --2- 172.21.15.114:0/2797658083 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe9d1c6970 0x55fe9d1d8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.123+0000 7f411e8cd740 1 -- 172.21.15.114:0/2797658083 >> 172.21.15.114:0/2797658083 conn(0x55fe9d1c2ee0 msgr2=0x55fe9d1e81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:01.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.124+0000 7f411e8cd740 1 -- 172.21.15.114:0/2797658083 shutdown_connections 2024-09-13T18:30:01.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.126+0000 7f411e8cd740 1 -- 172.21.15.114:0/2797658083 wait complete. 2024-09-13T18:30:01.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.126+0000 7f411e8cd740 1 -- start start 2024-09-13T18:30:01.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.126+0000 7f411e8cd740 1 Processor -- start 2024-09-13T18:30:01.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.130+0000 7f411e8cd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe9d1c6970 0x55fe9d1d8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:01.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.130+0000 7f411e8cd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe9d1c7290 0x55fe9d1cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:01.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.130+0000 7f411e8cd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe9d1d1b30 0x55fe9d1cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:01.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.130+0000 7f411e8cd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe9d1ca670 con 0x55fe9d1d1b30 2024-09-13T18:30:01.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.130+0000 7f411e8cd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe9d1ca7e0 con 0x55fe9d1c6970 2024-09-13T18:30:01.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.130+0000 7f411e8cd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe9d1ca9a0 con 0x55fe9d1c7290 2024-09-13T18:30:01.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.130+0000 7f411e8cd740 10 client.0 _ll_register_callbacks cb 0x7f410400a0d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:30:01.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.132+0000 7f411a147640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe9d1d1b30 0x55fe9d1cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:01.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.132+0000 7f411a147640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe9d1d1b30 0x55fe9d1cc040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44178/0 (socket says 172.21.15.114:44178) 2024-09-13T18:30:01.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.132+0000 7f411a147640 1 -- 172.21.15.114:0/1948050717 learned_addr learned my addr 172.21.15.114:0/1948050717 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:01.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.132+0000 7f411a147640 1 --2- 172.21.15.114:0/1948050717 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe9d1c6970 0x55fe9d1d8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:01.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.132+0000 7f4119145640 1 --2- 172.21.15.114:0/1948050717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe9d1c7290 0x55fe9d1cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:01.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.135+0000 7f4119145640 1 -- 172.21.15.114:0/1948050717 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe9d1c6970 msgr2=0x55fe9d1d8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:01.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.135+0000 7f4119145640 1 --2- 172.21.15.114:0/1948050717 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe9d1c6970 0x55fe9d1d8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.135+0000 7f4119145640 1 -- 172.21.15.114:0/1948050717 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe9d1d1b30 msgr2=0x55fe9d1cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:01.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.135+0000 7f4119145640 1 --2- 172.21.15.114:0/1948050717 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe9d1d1b30 0x55fe9d1cc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.135+0000 7f4119145640 1 -- 172.21.15.114:0/1948050717 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fe9d1283c0 con 0x55fe9d1c7290 2024-09-13T18:30:01.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.139+0000 7f4119145640 1 --2- 172.21.15.114:0/1948050717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe9d1c7290 0x55fe9d1cb8b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x55fe9d1e12d0 tx=0x7f4110016850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7f860d1a759ea8b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:01.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.140+0000 7f40fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:30:01.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.142+0000 7f40fd7fa640 1 -- 172.21.15.114:0/1948050717 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41100137a0 con 0x55fe9d1c7290 2024-09-13T18:30:01.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.142+0000 7f40fd7fa640 1 -- 172.21.15.114:0/1948050717 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f4110013900 con 0x55fe9d1c7290 2024-09-13T18:30:01.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.142+0000 7f411e8cd740 1 -- 172.21.15.114:0/1948050717 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f410400cca0 con 0x55fe9d1c7290 2024-09-13T18:30:01.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.148+0000 7f40fd7fa640 1 -- 172.21.15.114:0/1948050717 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4110013ac0 con 0x55fe9d1c7290 2024-09-13T18:30:01.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.148+0000 7f411e8cd740 10 client.27172 fetch_fsmap learned FSMap version 34 2024-09-13T18:30:01.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.148+0000 7f411e8cd740 1 -- 172.21.15.114:0/1948050717 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55fe9d0efe80 con 0x55fe9d1c7290 2024-09-13T18:30:01.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.152+0000 7f40fd7fa640 1 -- 172.21.15.114:0/1948050717 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f410400cca0 con 0x55fe9d1c7290 2024-09-13T18:30:01.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.152+0000 7f40fd7fa640 1 -- 172.21.15.114:0/1948050717 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4110012040 con 0x55fe9d1c7290 2024-09-13T18:30:01.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.152+0000 7f411e8cd740 10 client.27172 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:30:01.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.152+0000 7f411e8cd740 10 client.27172 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-13T18:30:01.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.152+0000 7f411e8cd740 20 client.27172 populate_metadata read hostname 'smithi114' 2024-09-13T18:30:01.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.152+0000 7f411e8cd740 1 --2- 172.21.15.114:0/1948050717 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55fe9d238580 0x55fe9d23a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:01.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.152+0000 7f411e8cd740 4 client.27172 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:30:01.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.152+0000 7f411e8cd740 1 -- 172.21.15.114:0/1948050717 --> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fe9ce1d4e0 con 0x55fe9d238580 2024-09-13T18:30:01.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.157+0000 7f411a147640 1 --2- 172.21.15.114:0/1948050717 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55fe9d238580 0x55fe9d23a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:01.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.160+0000 7f411a147640 1 --2- 172.21.15.114:0/1948050717 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55fe9d238580 0x55fe9d23a9a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=14edfe22d5d7b63d server_cookie=3e552cab9a7bb0f5 in_seq=0 out_seq=0 2024-09-13T18:30:01.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.163+0000 7f40fd7fa640 10 client.27172 ms_handle_connect on v2:172.21.15.204:6838/344883912 2024-09-13T18:30:01.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.167+0000 7f40fd7fa640 1 -- 172.21.15.114:0/1948050717 <== mds.2 v2:172.21.15.204:6838/344883912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fe9ce1d4e0 con 0x55fe9d238580 2024-09-13T18:30:01.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.167+0000 7f40fd7fa640 10 client.27172 handle_command_reply: tid=0 2024-09-13T18:30:01.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.167+0000 7f411e8cd740 1 client.27172 shutdown 2024-09-13T18:30:01.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.167+0000 7f411e8cd740 1 -- 172.21.15.114:0/1948050717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe9d1c7290 msgr2=0x55fe9d1cb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:01.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.167+0000 7f411e8cd740 1 --2- 172.21.15.114:0/1948050717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe9d1c7290 0x55fe9d1cb8b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x55fe9d1e12d0 tx=0x7f4110016850 comp rx=0 tx=0).stop 2024-09-13T18:30:01.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.167+0000 7f411e8cd740 1 -- 172.21.15.114:0/1948050717 shutdown_connections 2024-09-13T18:30:01.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.167+0000 7f411e8cd740 1 --2- 172.21.15.114:0/1948050717 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55fe9d238580 0x55fe9d23a9a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.167+0000 7f411e8cd740 1 --2- 172.21.15.114:0/1948050717 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe9d1d1b30 0x55fe9d1cc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.167+0000 7f411e8cd740 1 --2- 172.21.15.114:0/1948050717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fe9d1c7290 0x55fe9d1cb8b0 secure :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0x55fe9d1e12d0 tx=0x7f4110016850 comp rx=0 tx=0).stop 2024-09-13T18:30:01.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.167+0000 7f411e8cd740 1 --2- 172.21.15.114:0/1948050717 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe9d1c6970 0x55fe9d1d8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.167+0000 7f411e8cd740 1 -- 172.21.15.114:0/1948050717 >> 172.21.15.114:0/1948050717 conn(0x55fe9d1c2ee0 msgr2=0x55fe9d1c9590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:01.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.168+0000 7f40fd7fa640 0 client.27172 ms_handle_reset on v2:172.21.15.204:6838/344883912 2024-09-13T18:30:01.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.168+0000 7f411e8cd740 1 -- 172.21.15.114:0/1948050717 shutdown_connections 2024-09-13T18:30:01.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.172+0000 7f411e8cd740 1 -- 172.21.15.114:0/1948050717 wait complete. 2024-09-13T18:30:01.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.176+0000 7f411e8cd740 20 client.27172 trim_cache size 0 max 16384 2024-09-13T18:30:01.180 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:30:01.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.179+0000 7f411bbaa640 1 -- 172.21.15.114:0/3351988708 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f41000a1340 msgr2=0x7f41000a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:01.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.179+0000 7f411bbaa640 1 --2- 172.21.15.114:0/3351988708 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f41000a1340 0x7f41000a3800 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f410c0b7ff0 tx=0x7f410c0db040 comp rx=0 tx=0).stop 2024-09-13T18:30:01.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.179+0000 7f411bbaa640 1 -- 172.21.15.114:0/3351988708 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41141074b0 msgr2=0x7f4114116960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:01.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.179+0000 7f411bbaa640 1 --2- 172.21.15.114:0/3351988708 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41141074b0 0x7f4114116960 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f410c0098d0 tx=0x7f410c030b90 comp rx=0 tx=0).stop 2024-09-13T18:30:01.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.186+0000 7f411bbaa640 1 -- 172.21.15.114:0/3351988708 shutdown_connections 2024-09-13T18:30:01.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.186+0000 7f411bbaa640 1 --2- 172.21.15.114:0/3351988708 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f41000a1340 0x7f41000a3800 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.186+0000 7f411bbaa640 1 --2- 172.21.15.114:0/3351988708 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4114119400 0x7f411410fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.186+0000 7f411bbaa640 1 --2- 172.21.15.114:0/3351988708 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4114107890 0x7f4114116ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.186+0000 7f411bbaa640 1 --2- 172.21.15.114:0/3351988708 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41141074b0 0x7f4114116960 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.186+0000 7f411bbaa640 1 -- 172.21.15.114:0/3351988708 >> 172.21.15.114:0/3351988708 conn(0x7f411406af70 msgr2=0x7f4114119b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:01.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.186+0000 7f411bbaa640 1 -- 172.21.15.114:0/3351988708 shutdown_connections 2024-09-13T18:30:01.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.186+0000 7f411bbaa640 1 -- 172.21.15.114:0/3351988708 wait complete. 2024-09-13T18:30:01.214 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-09-13T18:30:01.216 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:00 smithi114 ceph-mon[30897]: overall HEALTH_OK 2024-09-13T18:30:01.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:00 smithi134 ceph-mon[34283]: overall HEALTH_OK 2024-09-13T18:30:01.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:00 smithi204 ceph-mon[34777]: overall HEALTH_OK 2024-09-13T18:30:01.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.451+0000 7f31d8661640 1 -- 172.21.15.114:0/753382158 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31d010c3e0 msgr2=0x7f31d010c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:01.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.451+0000 7f31d8661640 1 --2- 172.21.15.114:0/753382158 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31d010c3e0 0x7f31d010c7c0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f31c0009930 tx=0x7f31c002a510 comp rx=0 tx=0).stop 2024-09-13T18:30:01.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.455+0000 7f31d8661640 1 -- 172.21.15.114:0/753382158 shutdown_connections 2024-09-13T18:30:01.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.455+0000 7f31d8661640 1 --2- 172.21.15.114:0/753382158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31d0106ca0 0x7f31d0113180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.455+0000 7f31d8661640 1 --2- 172.21.15.114:0/753382158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31d0106320 0x7f31d0106760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.455+0000 7f31d8661640 1 --2- 172.21.15.114:0/753382158 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31d010c3e0 0x7f31d010c7c0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.455+0000 7f31d8661640 1 -- 172.21.15.114:0/753382158 >> 172.21.15.114:0/753382158 conn(0x7f31d0075c60 msgr2=0x7f31d0078080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:01.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.462+0000 7f31d8661640 1 -- 172.21.15.114:0/753382158 shutdown_connections 2024-09-13T18:30:01.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.465+0000 7f31d8661640 1 -- 172.21.15.114:0/753382158 wait complete. 2024-09-13T18:30:01.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.466+0000 7f31d8661640 1 Processor -- start 2024-09-13T18:30:01.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.467+0000 7f31d8661640 1 -- start start 2024-09-13T18:30:01.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.469+0000 7f31d8661640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31d0106320 0x7f31d01a2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:01.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.469+0000 7f31d8661640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31d0106ca0 0x7f31d01a2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:01.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.469+0000 7f31d8661640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31d010c3e0 0x7f31d019c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:01.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.469+0000 7f31d8661640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d019cc50 con 0x7f31d010c3e0 2024-09-13T18:30:01.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.469+0000 7f31d8661640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d019cdc0 con 0x7f31d0106320 2024-09-13T18:30:01.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.469+0000 7f31d8661640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d019cf80 con 0x7f31d0106ca0 2024-09-13T18:30:01.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.469+0000 7f31d6bfe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31d010c3e0 0x7f31d019c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:01.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.469+0000 7f31d6bfe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31d010c3e0 0x7f31d019c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44196/0 (socket says 172.21.15.114:44196) 2024-09-13T18:30:01.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.469+0000 7f31d6bfe640 1 -- 172.21.15.114:0/1450681494 learned_addr learned my addr 172.21.15.114:0/1450681494 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:01.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.469+0000 7f31d5bfc640 1 --2- 172.21.15.114:0/1450681494 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31d0106ca0 0x7f31d01a2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:01.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.471+0000 7f31d5bfc640 1 -- 172.21.15.114:0/1450681494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31d0106320 msgr2=0x7f31d01a2550 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:30:01.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.471+0000 7f31d5bfc640 1 --2- 172.21.15.114:0/1450681494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31d0106320 0x7f31d01a2550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.471+0000 7f31d5bfc640 1 -- 172.21.15.114:0/1450681494 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31d010c3e0 msgr2=0x7f31d019c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:01.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.471+0000 7f31d5bfc640 1 --2- 172.21.15.114:0/1450681494 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31d010c3e0 0x7f31d019c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.471+0000 7f31d5bfc640 1 -- 172.21.15.114:0/1450681494 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31c0009560 con 0x7f31d0106ca0 2024-09-13T18:30:01.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.472+0000 7f31d5bfc640 1 --2- 172.21.15.114:0/1450681494 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31d0106ca0 0x7f31d01a2a90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f31cc009fd0 tx=0x7f31cc00b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:01.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.473+0000 7f31c77fe640 1 -- 172.21.15.114:0/1450681494 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31cc011830 con 0x7f31d0106ca0 2024-09-13T18:30:01.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.473+0000 7f31d8661640 1 -- 172.21.15.114:0/1450681494 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31d019d260 con 0x7f31d0106ca0 2024-09-13T18:30:01.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.473+0000 7f31d8661640 1 -- 172.21.15.114:0/1450681494 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31d01a9540 con 0x7f31d0106ca0 2024-09-13T18:30:01.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.473+0000 7f31c77fe640 1 -- 172.21.15.114:0/1450681494 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f31cc011e20 con 0x7f31d0106ca0 2024-09-13T18:30:01.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.473+0000 7f31c77fe640 1 -- 172.21.15.114:0/1450681494 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31cc00f2e0 con 0x7f31d0106ca0 2024-09-13T18:30:01.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.482+0000 7f31db45e740 1 Processor -- start 2024-09-13T18:30:01.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.482+0000 7f31db45e740 1 -- start start 2024-09-13T18:30:01.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.482+0000 7f31db45e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608ec20d790 0x5608ec20db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:01.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.482+0000 7f31db45e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608ec20e0b0 0x5608ec218410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:01.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.482+0000 7f31db45e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5608ec218950 0x5608ec21ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:01.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.482+0000 7f31db45e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608ec177cc0 con 0x5608ec218950 2024-09-13T18:30:01.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.482+0000 7f31db45e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608ec175450 con 0x5608ec20d790 2024-09-13T18:30:01.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.482+0000 7f31db45e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608ec170d40 con 0x5608ec20e0b0 2024-09-13T18:30:01.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.483+0000 7f31d6bfe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608ec20d790 0x5608ec20db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:01.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.483+0000 7f31d6bfe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608ec20d790 0x5608ec20db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:43648/0 (socket says 172.21.15.114:43648) 2024-09-13T18:30:01.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.483+0000 7f31d6bfe640 1 -- 172.21.15.114:0/1118262307 learned_addr learned my addr 172.21.15.114:0/1118262307 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:01.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.483+0000 7f31d63fd640 1 --2- 172.21.15.114:0/1118262307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608ec20e0b0 0x5608ec218410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:01.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.483+0000 7f31d6bfe640 1 -- 172.21.15.114:0/1118262307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608ec20e0b0 msgr2=0x5608ec218410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:01.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.483+0000 7f31d6bfe640 1 --2- 172.21.15.114:0/1118262307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608ec20e0b0 0x5608ec218410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.483+0000 7f31d6bfe640 1 -- 172.21.15.114:0/1118262307 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5608ec218950 msgr2=0x5608ec21ad40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:30:01.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.483+0000 7f31d6bfe640 1 --2- 172.21.15.114:0/1118262307 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5608ec218950 0x5608ec21ad40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.483+0000 7f31d6bfe640 1 -- 172.21.15.114:0/1118262307 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608ec138620 con 0x5608ec20d790 2024-09-13T18:30:01.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.483+0000 7f31d6bfe640 1 --2- 172.21.15.114:0/1118262307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608ec20d790 0x5608ec20db70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f31c8009fb0 tx=0x7f31c8010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6f213eec230df87 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:01.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.486+0000 7f31c4ff9640 1 -- 172.21.15.114:0/1118262307 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c8012710 con 0x5608ec20d790 2024-09-13T18:30:01.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.486+0000 7f31c4ff9640 1 -- 172.21.15.114:0/1118262307 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f31c8012d00 con 0x5608ec20d790 2024-09-13T18:30:01.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.486+0000 7f31c4ff9640 1 -- 172.21.15.114:0/1118262307 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c8011be0 con 0x5608ec20d790 2024-09-13T18:30:01.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.486+0000 7f31c77fe640 1 -- 172.21.15.114:0/1450681494 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f31cc010380 con 0x7f31d0106ca0 2024-09-13T18:30:01.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.487+0000 7f31db45e740 1 -- 172.21.15.114:0/1118262307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608ec20d790 msgr2=0x5608ec20db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:01.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.487+0000 7f31db45e740 1 --2- 172.21.15.114:0/1118262307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608ec20d790 0x5608ec20db70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f31c8009fb0 tx=0x7f31c8010800 comp rx=0 tx=0).stop 2024-09-13T18:30:01.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.487+0000 7f31c77fe640 1 --2- 172.21.15.114:0/1450681494 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f31bc0a1340 0x7f31bc0a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:01.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.487+0000 7f31c77fe640 1 -- 172.21.15.114:0/1450681494 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f31cc0cca60 con 0x7f31d0106ca0 2024-09-13T18:30:01.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.490+0000 7f31db45e740 1 -- 172.21.15.114:0/1118262307 shutdown_connections 2024-09-13T18:30:01.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.490+0000 7f31db45e740 1 --2- 172.21.15.114:0/1118262307 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5608ec218950 0x5608ec21ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.490+0000 7f31db45e740 1 --2- 172.21.15.114:0/1118262307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608ec20e0b0 0x5608ec218410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.490+0000 7f31db45e740 1 --2- 172.21.15.114:0/1118262307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608ec20d790 0x5608ec20db70 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.490+0000 7f31d6bfe640 1 --2- 172.21.15.114:0/1450681494 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f31bc0a1340 0x7f31bc0a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:01.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.490+0000 7f31db45e740 1 -- 172.21.15.114:0/1118262307 >> 172.21.15.114:0/1118262307 conn(0x5608ec209d00 msgr2=0x5608ec20a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:01.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.491+0000 7f31db45e740 1 -- 172.21.15.114:0/1118262307 shutdown_connections 2024-09-13T18:30:01.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.491+0000 7f31db45e740 1 -- 172.21.15.114:0/1118262307 wait complete. 2024-09-13T18:30:01.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.492+0000 7f31d6bfe640 1 --2- 172.21.15.114:0/1450681494 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f31bc0a1340 0x7f31bc0a3800 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f31c8011760 tx=0x7f31c80117c0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:01.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.492+0000 7f31db45e740 1 -- start start 2024-09-13T18:30:01.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.492+0000 7f31db45e740 1 Processor -- start 2024-09-13T18:30:01.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.493+0000 7f31db45e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608ec20d790 0x5608ec217f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:01.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.493+0000 7f31db45e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608ec20e0b0 0x5608ec215230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:01.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.493+0000 7f31db45e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5608ec218950 0x5608ec215770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:01.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.493+0000 7f31db45e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608ec12f180 con 0x5608ec218950 2024-09-13T18:30:01.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.493+0000 7f31db45e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608ebe49430 con 0x5608ec20e0b0 2024-09-13T18:30:01.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.493+0000 7f31db45e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608ec0f45e0 con 0x5608ec20d790 2024-09-13T18:30:01.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.493+0000 7f31d5bfc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608ec20e0b0 0x5608ec215230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:01.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.493+0000 7f31db45e740 10 client.0 _ll_register_callbacks cb 0x5608ec160aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:30:01.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.493+0000 7f31d5bfc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608ec20e0b0 0x5608ec215230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:43658/0 (socket says 172.21.15.114:43658) 2024-09-13T18:30:01.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.493+0000 7f31d5bfc640 1 -- 172.21.15.114:0/3248026485 learned_addr learned my addr 172.21.15.114:0/3248026485 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:01.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.493+0000 7f31d63fd640 1 --2- 172.21.15.114:0/3248026485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608ec20d790 0x5608ec217f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:01.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.494+0000 7f31d63fd640 1 -- 172.21.15.114:0/3248026485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608ec20e0b0 msgr2=0x5608ec215230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:01.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.494+0000 7f31d63fd640 1 --2- 172.21.15.114:0/3248026485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608ec20e0b0 0x5608ec215230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.494+0000 7f31d63fd640 1 -- 172.21.15.114:0/3248026485 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5608ec218950 msgr2=0x5608ec215770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:01.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.494+0000 7f31d63fd640 1 --2- 172.21.15.114:0/3248026485 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5608ec218950 0x5608ec215770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.494+0000 7f31d63fd640 1 -- 172.21.15.114:0/3248026485 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31c8009be0 con 0x5608ec20d790 2024-09-13T18:30:01.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.494+0000 7f31d63fd640 1 --2- 172.21.15.114:0/3248026485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608ec20d790 0x5608ec217f10 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x5608ec20ab60 tx=0x7f31c00045c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8aad81d85f9babe0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:01.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.494+0000 7f31b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:30:01.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.495+0000 7f31b9ffb640 1 -- 172.21.15.114:0/3248026485 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c0005510 con 0x5608ec20d790 2024-09-13T18:30:01.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.495+0000 7f31db45e740 1 -- 172.21.15.114:0/3248026485 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5608ec19ab80 con 0x5608ec20d790 2024-09-13T18:30:01.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.495+0000 7f31b9ffb640 1 -- 172.21.15.114:0/3248026485 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f31c00046c0 con 0x5608ec20d790 2024-09-13T18:30:01.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.495+0000 7f31b9ffb640 1 -- 172.21.15.114:0/3248026485 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31c0004880 con 0x5608ec20d790 2024-09-13T18:30:01.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.496+0000 7f31db45e740 10 client.27181 fetch_fsmap learned FSMap version 34 2024-09-13T18:30:01.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.496+0000 7f31db45e740 1 -- 172.21.15.114:0/3248026485 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5608ec16f3c0 con 0x5608ec20d790 2024-09-13T18:30:01.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.496+0000 7f31b9ffb640 1 -- 172.21.15.114:0/3248026485 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5608ec19ab80 con 0x5608ec20d790 2024-09-13T18:30:01.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.497+0000 7f31b9ffb640 1 -- 172.21.15.114:0/3248026485 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f31c0043020 con 0x5608ec20d790 2024-09-13T18:30:01.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.498+0000 7f31db45e740 10 client.27181 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:30:01.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.498+0000 7f31db45e740 10 client.27181 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-13T18:30:01.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.498+0000 7f31db45e740 20 client.27181 populate_metadata read hostname 'smithi114' 2024-09-13T18:30:01.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.498+0000 7f31db45e740 1 --2- 172.21.15.114:0/3248026485 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x5608ec239360 0x5608ec23b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:01.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.498+0000 7f31db45e740 4 client.27181 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:30:01.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.498+0000 7f31db45e740 1 -- 172.21.15.114:0/3248026485 --> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5608ec1631f0 con 0x5608ec239360 2024-09-13T18:30:01.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.498+0000 7f31d5bfc640 1 --2- 172.21.15.114:0/3248026485 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x5608ec239360 0x5608ec23b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:01.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.500+0000 7f31d5bfc640 1 --2- 172.21.15.114:0/3248026485 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x5608ec239360 0x5608ec23b780 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d435c8f2dd63d968 server_cookie=6b9257e67b4a12b7 in_seq=0 out_seq=0 2024-09-13T18:30:01.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.500+0000 7f31b9ffb640 10 client.27181 ms_handle_connect on v2:172.21.15.134:6836/891692210 2024-09-13T18:30:01.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.503+0000 7f31b9ffb640 1 -- 172.21.15.114:0/3248026485 <== mds.1 v2:172.21.15.134:6836/891692210 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5608ec1631f0 con 0x5608ec239360 2024-09-13T18:30:01.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.503+0000 7f31b9ffb640 10 client.27181 handle_command_reply: tid=0 2024-09-13T18:30:01.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.503+0000 7f31db45e740 1 client.27181 shutdown 2024-09-13T18:30:01.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.504+0000 7f31db45e740 1 -- 172.21.15.114:0/3248026485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608ec20d790 msgr2=0x5608ec217f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:01.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.504+0000 7f31db45e740 1 --2- 172.21.15.114:0/3248026485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608ec20d790 0x5608ec217f10 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x5608ec20ab60 tx=0x7f31c00045c0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.504+0000 7f31db45e740 1 -- 172.21.15.114:0/3248026485 shutdown_connections 2024-09-13T18:30:01.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.504+0000 7f31db45e740 1 --2- 172.21.15.114:0/3248026485 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x5608ec239360 0x5608ec23b780 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.504+0000 7f31db45e740 1 --2- 172.21.15.114:0/3248026485 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5608ec218950 0x5608ec215770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.504+0000 7f31db45e740 1 --2- 172.21.15.114:0/3248026485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608ec20e0b0 0x5608ec215230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.504+0000 7f31db45e740 1 --2- 172.21.15.114:0/3248026485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608ec20d790 0x5608ec217f10 secure :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0x5608ec20ab60 tx=0x7f31c00045c0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.504+0000 7f31db45e740 1 -- 172.21.15.114:0/3248026485 >> 172.21.15.114:0/3248026485 conn(0x5608ec209d00 msgr2=0x5608ec208eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:01.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.504+0000 7f31b9ffb640 0 client.27181 ms_handle_reset on v2:172.21.15.134:6836/891692210 2024-09-13T18:30:01.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.505+0000 7f31db45e740 1 -- 172.21.15.114:0/3248026485 shutdown_connections 2024-09-13T18:30:01.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.505+0000 7f31db45e740 1 -- 172.21.15.114:0/3248026485 wait complete. 2024-09-13T18:30:01.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.505+0000 7f31db45e740 20 client.27181 trim_cache size 0 max 16384 2024-09-13T18:30:01.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.546+0000 7f31db45e740 1 Processor -- start 2024-09-13T18:30:01.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.546+0000 7f31db45e740 1 -- start start 2024-09-13T18:30:01.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.547+0000 7f31db45e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5608ec20e0b0 0x5608ec230680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:01.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.547+0000 7f31db45e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608ec218950 0x5608ec230bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:01.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.547+0000 7f31db45e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608ec217a00 0x5608ec217e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:01.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.547+0000 7f31db45e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608ec2183a0 con 0x5608ec20e0b0 2024-09-13T18:30:01.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.547+0000 7f31db45e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608ec2167b0 con 0x5608ec218950 2024-09-13T18:30:01.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.547+0000 7f31db45e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608ec215ae0 con 0x5608ec217a00 2024-09-13T18:30:01.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.547+0000 7f31d6bfe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608ec217a00 0x5608ec217e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:01.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.547+0000 7f31d6bfe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608ec217a00 0x5608ec217e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:58150/0 (socket says 172.21.15.114:58150) 2024-09-13T18:30:01.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.547+0000 7f31d6bfe640 1 -- 172.21.15.114:0/2983872736 learned_addr learned my addr 172.21.15.114:0/2983872736 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:01.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.547+0000 7f31d5bfc640 1 --2- 172.21.15.114:0/2983872736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608ec218950 0x5608ec230bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:01.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.547+0000 7f31d63fd640 1 --2- 172.21.15.114:0/2983872736 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5608ec20e0b0 0x5608ec230680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:01.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.548+0000 7f31d63fd640 1 -- 172.21.15.114:0/2983872736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608ec217a00 msgr2=0x5608ec217e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:01.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.548+0000 7f31d63fd640 1 --2- 172.21.15.114:0/2983872736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608ec217a00 0x5608ec217e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.548+0000 7f31d63fd640 1 -- 172.21.15.114:0/2983872736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608ec218950 msgr2=0x5608ec230bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:01.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.548+0000 7f31d63fd640 1 --2- 172.21.15.114:0/2983872736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608ec218950 0x5608ec230bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.548+0000 7f31d63fd640 1 -- 172.21.15.114:0/2983872736 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608ec111770 con 0x5608ec20e0b0 2024-09-13T18:30:01.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.554+0000 7f31d63fd640 1 --2- 172.21.15.114:0/2983872736 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5608ec20e0b0 0x5608ec230680 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f31c004af70 tx=0x7f31c004c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2c680d7a1533f8b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:01.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.556+0000 7f31c57fa640 1 -- 172.21.15.114:0/2983872736 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c0003ed0 con 0x5608ec20e0b0 2024-09-13T18:30:01.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.556+0000 7f31c57fa640 1 -- 172.21.15.114:0/2983872736 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f31c0003a80 con 0x5608ec20e0b0 2024-09-13T18:30:01.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.556+0000 7f31c57fa640 1 -- 172.21.15.114:0/2983872736 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c0004920 con 0x5608ec20e0b0 2024-09-13T18:30:01.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.557+0000 7f31db45e740 1 -- 172.21.15.114:0/2983872736 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5608ec20e0b0 msgr2=0x5608ec230680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:01.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.557+0000 7f31db45e740 1 --2- 172.21.15.114:0/2983872736 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5608ec20e0b0 0x5608ec230680 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f31c004af70 tx=0x7f31c004c000 comp rx=0 tx=0).stop 2024-09-13T18:30:01.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.561+0000 7f31db45e740 1 -- 172.21.15.114:0/2983872736 shutdown_connections 2024-09-13T18:30:01.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.561+0000 7f31db45e740 1 --2- 172.21.15.114:0/2983872736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608ec217a00 0x5608ec217e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.561+0000 7f31db45e740 1 --2- 172.21.15.114:0/2983872736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608ec218950 0x5608ec230bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.561+0000 7f31db45e740 1 --2- 172.21.15.114:0/2983872736 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5608ec20e0b0 0x5608ec230680 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.561+0000 7f31db45e740 1 -- 172.21.15.114:0/2983872736 >> 172.21.15.114:0/2983872736 conn(0x5608ec209d00 msgr2=0x5608ec240120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:01.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.567+0000 7f31db45e740 1 -- 172.21.15.114:0/2983872736 shutdown_connections 2024-09-13T18:30:01.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.570+0000 7f31db45e740 1 -- 172.21.15.114:0/2983872736 wait complete. 2024-09-13T18:30:01.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.570+0000 7f31db45e740 1 -- start start 2024-09-13T18:30:01.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.571+0000 7f31db45e740 1 Processor -- start 2024-09-13T18:30:01.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.574+0000 7f31db45e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5608ec20e0b0 0x5608ec230690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:01.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.574+0000 7f31db45e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608ec217a00 0x5608ec2114c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:01.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.574+0000 7f31db45e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608ec218950 0x5608ec211a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:01.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.574+0000 7f31db45e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608ec230bd0 con 0x5608ec20e0b0 2024-09-13T18:30:01.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.574+0000 7f31db45e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608ec230d40 con 0x5608ec218950 2024-09-13T18:30:01.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.574+0000 7f31db45e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608ec230f00 con 0x5608ec217a00 2024-09-13T18:30:01.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.574+0000 7f31db45e740 10 client.0 _ll_register_callbacks cb 0x7f31c8009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:30:01.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.576+0000 7f31d6bfe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608ec218950 0x5608ec211a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:01.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.576+0000 7f31d6bfe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608ec218950 0x5608ec211a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:43664/0 (socket says 172.21.15.114:43664) 2024-09-13T18:30:01.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.576+0000 7f31d6bfe640 1 -- 172.21.15.114:0/3785147771 learned_addr learned my addr 172.21.15.114:0/3785147771 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:01.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.576+0000 7f31d6bfe640 1 -- 172.21.15.114:0/3785147771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608ec217a00 msgr2=0x5608ec2114c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:30:01.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.576+0000 7f31d6bfe640 1 --2- 172.21.15.114:0/3785147771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608ec217a00 0x5608ec2114c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.576+0000 7f31d6bfe640 1 -- 172.21.15.114:0/3785147771 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5608ec20e0b0 msgr2=0x5608ec230690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:01.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.576+0000 7f31d6bfe640 1 --2- 172.21.15.114:0/3785147771 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5608ec20e0b0 0x5608ec230690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.576+0000 7f31d6bfe640 1 -- 172.21.15.114:0/3785147771 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5608ec16f3c0 con 0x5608ec218950 2024-09-13T18:30:01.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.576+0000 7f31d6bfe640 1 --2- 172.21.15.114:0/3785147771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608ec218950 0x5608ec211a00 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f31c8015040 tx=0x7f31c8011c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b3e6adbf0bb2dd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:01.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.576+0000 7f31b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:30:01.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.577+0000 7f31b9ffb640 1 -- 172.21.15.114:0/3785147771 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c8012b90 con 0x5608ec218950 2024-09-13T18:30:01.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.577+0000 7f31db45e740 1 -- 172.21.15.114:0/3785147771 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f31c8012d00 con 0x5608ec218950 2024-09-13T18:30:01.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.577+0000 7f31b9ffb640 1 -- 172.21.15.114:0/3785147771 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f31c8004fd0 con 0x5608ec218950 2024-09-13T18:30:01.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.578+0000 7f31b9ffb640 1 -- 172.21.15.114:0/3785147771 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31c8005190 con 0x5608ec218950 2024-09-13T18:30:01.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.581+0000 7f31db45e740 10 client.27194 fetch_fsmap learned FSMap version 34 2024-09-13T18:30:01.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.582+0000 7f31db45e740 1 -- 172.21.15.114:0/3785147771 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5608ec136e80 con 0x5608ec218950 2024-09-13T18:30:01.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.584+0000 7f31b9ffb640 1 -- 172.21.15.114:0/3785147771 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f31c8012d00 con 0x5608ec218950 2024-09-13T18:30:01.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.584+0000 7f31b9ffb640 1 -- 172.21.15.114:0/3785147771 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f31c8052070 con 0x5608ec218950 2024-09-13T18:30:01.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.586+0000 7f31db45e740 10 client.27194 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:30:01.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.586+0000 7f31db45e740 10 client.27194 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-13T18:30:01.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.586+0000 7f31db45e740 20 client.27194 populate_metadata read hostname 'smithi114' 2024-09-13T18:30:01.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.586+0000 7f31db45e740 1 --2- 172.21.15.114:0/3785147771 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x5608ec2906f0 0x5608ec292b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:01.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.586+0000 7f31db45e740 4 client.27194 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:30:01.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.586+0000 7f31db45e740 1 -- 172.21.15.114:0/3785147771 --> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5608ebe644e0 con 0x5608ec2906f0 2024-09-13T18:30:01.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.588+0000 7f31d63fd640 1 --2- 172.21.15.114:0/3785147771 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x5608ec2906f0 0x5608ec292b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:01.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.589+0000 7f31d63fd640 1 --2- 172.21.15.114:0/3785147771 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x5608ec2906f0 0x5608ec292b30 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=822d01ff290a61c8 server_cookie=af42eec9e98e2c0 in_seq=0 out_seq=0 2024-09-13T18:30:01.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.591+0000 7f31b9ffb640 10 client.27194 ms_handle_connect on v2:172.21.15.134:6836/891692210 2024-09-13T18:30:01.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.592+0000 7f31b9ffb640 1 -- 172.21.15.114:0/3785147771 <== mds.1 v2:172.21.15.134:6836/891692210 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5608ebe644e0 con 0x5608ec2906f0 2024-09-13T18:30:01.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.593+0000 7f31b9ffb640 10 client.27194 handle_command_reply: tid=0 2024-09-13T18:30:01.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.594+0000 7f31db45e740 1 client.27194 shutdown 2024-09-13T18:30:01.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.598+0000 7f31db45e740 1 -- 172.21.15.114:0/3785147771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608ec218950 msgr2=0x5608ec211a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:01.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.598+0000 7f31db45e740 1 --2- 172.21.15.114:0/3785147771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608ec218950 0x5608ec211a00 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f31c8015040 tx=0x7f31c8011c00 comp rx=0 tx=0).stop 2024-09-13T18:30:01.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.598+0000 7f31db45e740 1 -- 172.21.15.114:0/3785147771 shutdown_connections 2024-09-13T18:30:01.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.598+0000 7f31db45e740 1 --2- 172.21.15.114:0/3785147771 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x5608ec2906f0 0x5608ec292b30 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.598+0000 7f31db45e740 1 --2- 172.21.15.114:0/3785147771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608ec218950 0x5608ec211a00 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.598+0000 7f31db45e740 1 --2- 172.21.15.114:0/3785147771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5608ec217a00 0x5608ec2114c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.598+0000 7f31db45e740 1 --2- 172.21.15.114:0/3785147771 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5608ec20e0b0 0x5608ec230690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.598+0000 7f31db45e740 1 -- 172.21.15.114:0/3785147771 >> 172.21.15.114:0/3785147771 conn(0x5608ec209d00 msgr2=0x5608ec2103b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:01.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.598+0000 7f31b9ffb640 0 client.27194 ms_handle_reset on v2:172.21.15.134:6836/891692210 2024-09-13T18:30:01.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.602+0000 7f31db45e740 1 -- 172.21.15.114:0/3785147771 shutdown_connections 2024-09-13T18:30:01.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.603+0000 7f31db45e740 1 -- 172.21.15.114:0/3785147771 wait complete. 2024-09-13T18:30:01.612 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:30:01.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.606+0000 7f31db45e740 20 client.27194 trim_cache size 0 max 16384 2024-09-13T18:30:01.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.608+0000 7f31d8661640 1 -- 172.21.15.114:0/1450681494 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f31bc0a1340 msgr2=0x7f31bc0a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:01.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.608+0000 7f31d8661640 1 --2- 172.21.15.114:0/1450681494 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f31bc0a1340 0x7f31bc0a3800 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f31c8011760 tx=0x7f31c80117c0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.608+0000 7f31d8661640 1 -- 172.21.15.114:0/1450681494 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31d0106ca0 msgr2=0x7f31d01a2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:01.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.608+0000 7f31d8661640 1 --2- 172.21.15.114:0/1450681494 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31d0106ca0 0x7f31d01a2a90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f31cc009fd0 tx=0x7f31cc00b870 comp rx=0 tx=0).stop 2024-09-13T18:30:01.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.608+0000 7f31d8661640 1 -- 172.21.15.114:0/1450681494 shutdown_connections 2024-09-13T18:30:01.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.608+0000 7f31d8661640 1 --2- 172.21.15.114:0/1450681494 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f31bc0a1340 0x7f31bc0a3800 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.608+0000 7f31d8661640 1 --2- 172.21.15.114:0/1450681494 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31d010c3e0 0x7f31d019c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.609+0000 7f31d8661640 1 --2- 172.21.15.114:0/1450681494 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31d0106ca0 0x7f31d01a2a90 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.609+0000 7f31d8661640 1 --2- 172.21.15.114:0/1450681494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31d0106320 0x7f31d01a2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.609+0000 7f31d8661640 1 -- 172.21.15.114:0/1450681494 >> 172.21.15.114:0/1450681494 conn(0x7f31d0075c60 msgr2=0x7f31d0077720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:01.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.615+0000 7f31d8661640 1 -- 172.21.15.114:0/1450681494 shutdown_connections 2024-09-13T18:30:01.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.615+0000 7f31d8661640 1 -- 172.21.15.114:0/1450681494 wait complete. 2024-09-13T18:30:01.650 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-09-13T18:30:01.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.858+0000 7f5b5b9c7640 1 -- 172.21.15.114:0/633638103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b54119400 msgr2=0x7f5b5411b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:01.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.858+0000 7f5b5b9c7640 1 --2- 172.21.15.114:0/633638103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b54119400 0x7f5b5411b7f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c01acb0 tx=0x7f5b4c03b920 comp rx=0 tx=0).stop 2024-09-13T18:30:01.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.858+0000 7f5b5b9c7640 1 -- 172.21.15.114:0/633638103 shutdown_connections 2024-09-13T18:30:01.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.858+0000 7f5b5b9c7640 1 --2- 172.21.15.114:0/633638103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b54119400 0x7f5b5411b7f0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.858+0000 7f5b5b9c7640 1 --2- 172.21.15.114:0/633638103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b54107920 0x7f5b54107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.858+0000 7f5b5b9c7640 1 --2- 172.21.15.114:0/633638103 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b54107540 0x7f5b5410e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.858+0000 7f5b5b9c7640 1 -- 172.21.15.114:0/633638103 >> 172.21.15.114:0/633638103 conn(0x7f5b5406afe0 msgr2=0x7f5b5406b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:01.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.864+0000 7f5b5b9c7640 1 -- 172.21.15.114:0/633638103 shutdown_connections 2024-09-13T18:30:01.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.867+0000 7f5b5b9c7640 1 -- 172.21.15.114:0/633638103 wait complete. 2024-09-13T18:30:01.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.867+0000 7f5b5b9c7640 1 Processor -- start 2024-09-13T18:30:01.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.868+0000 7f5b5b9c7640 1 -- start start 2024-09-13T18:30:01.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.868+0000 7f5b5b9c7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b54107540 0x7f5b54114b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:01.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.868+0000 7f5b5b9c7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b54107920 0x7f5b541150b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:01.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.868+0000 7f5b5b9c7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b54119400 0x7f5b541a1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:01.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.868+0000 7f5b5b9c7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b541a19d0 con 0x7f5b54119400 2024-09-13T18:30:01.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.868+0000 7f5b5b9c7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b541a1b40 con 0x7f5b54107920 2024-09-13T18:30:01.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.868+0000 7f5b5b9c7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b541a1d00 con 0x7f5b54107540 2024-09-13T18:30:01.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.869+0000 7f5b59f64640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b54119400 0x7f5b541a1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:01.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.869+0000 7f5b59f64640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b54119400 0x7f5b541a1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44246/0 (socket says 172.21.15.114:44246) 2024-09-13T18:30:01.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.869+0000 7f5b59f64640 1 -- 172.21.15.114:0/369188424 learned_addr learned my addr 172.21.15.114:0/369188424 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:01.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.869+0000 7f5b59763640 1 --2- 172.21.15.114:0/369188424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b54107540 0x7f5b54114b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:01.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.869+0000 7f5b59f64640 1 -- 172.21.15.114:0/369188424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b54107540 msgr2=0x7f5b54114b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:01.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.869+0000 7f5b59f64640 1 --2- 172.21.15.114:0/369188424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b54107540 0x7f5b54114b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.869+0000 7f5b59f64640 1 -- 172.21.15.114:0/369188424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b54107920 msgr2=0x7f5b541150b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:01.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.869+0000 7f5b59f64640 1 --2- 172.21.15.114:0/369188424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b54107920 0x7f5b541150b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.869+0000 7f5b59f64640 1 -- 172.21.15.114:0/369188424 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b4c01a910 con 0x7f5b54119400 2024-09-13T18:30:01.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.870+0000 7f5b59f64640 1 --2- 172.21.15.114:0/369188424 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b54119400 0x7f5b541a1240 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c03be90 tx=0x7f5b4c040930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:01.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.872+0000 7f5b4a7fc640 1 -- 172.21.15.114:0/369188424 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c040c30 con 0x7f5b54119400 2024-09-13T18:30:01.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.872+0000 7f5b4a7fc640 1 -- 172.21.15.114:0/369188424 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5b4c040d90 con 0x7f5b54119400 2024-09-13T18:30:01.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.872+0000 7f5b4a7fc640 1 -- 172.21.15.114:0/369188424 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c041a10 con 0x7f5b54119400 2024-09-13T18:30:01.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.872+0000 7f5b5b9c7640 1 -- 172.21.15.114:0/369188424 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b541a1f80 con 0x7f5b54119400 2024-09-13T18:30:01.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.872+0000 7f5b5b9c7640 1 -- 172.21.15.114:0/369188424 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b541cb5f0 con 0x7f5b54119400 2024-09-13T18:30:01.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.880+0000 7f5b4a7fc640 1 -- 172.21.15.114:0/369188424 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f5b4c041b70 con 0x7f5b54119400 2024-09-13T18:30:01.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.892+0000 7f5b5e7c6740 1 Processor -- start 2024-09-13T18:30:01.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.901+0000 7f5b5e7c6740 1 -- start start 2024-09-13T18:30:01.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.901+0000 7f5b5e7c6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f91b032790 0x55f91b032b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:01.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.901+0000 7f5b5e7c6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f91b0330b0 0x55f91b03d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:01.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.901+0000 7f5b5e7c6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f91b03d950 0x55f91b03fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:01.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.901+0000 7f5b5e7c6740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91af9ccc0 con 0x55f91b032790 2024-09-13T18:30:01.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.901+0000 7f5b5e7c6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91af9a450 con 0x55f91b0330b0 2024-09-13T18:30:01.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.901+0000 7f5b5e7c6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91af95d40 con 0x55f91b03d950 2024-09-13T18:30:01.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.901+0000 7f5b4a7fc640 1 --2- 172.21.15.114:0/369188424 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f5b400a1410 0x7f5b400a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:01.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.901+0000 7f5b58f62640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f91b03d950 0x55f91b03fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:01.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.901+0000 7f5b58f62640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f91b03d950 0x55f91b03fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:58182/0 (socket says 172.21.15.114:58182) 2024-09-13T18:30:01.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.901+0000 7f5b58f62640 1 -- 172.21.15.114:0/2834089823 learned_addr learned my addr 172.21.15.114:0/2834089823 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:01.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.902+0000 7f5b4a7fc640 1 -- 172.21.15.114:0/369188424 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f5b4c0eecd0 con 0x7f5b54119400 2024-09-13T18:30:01.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.906+0000 7f5b59763640 1 --2- 172.21.15.114:0/2834089823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f91b0330b0 0x55f91b03d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:01.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.906+0000 7f5b59f64640 1 --2- 172.21.15.114:0/369188424 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f5b400a1410 0x7f5b400a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:01.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.908+0000 7f5b58f62640 1 -- 172.21.15.114:0/2834089823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f91b0330b0 msgr2=0x55f91b03d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:01.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.908+0000 7f5b58f62640 1 --2- 172.21.15.114:0/2834089823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f91b0330b0 0x55f91b03d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.908+0000 7f5b58f62640 1 -- 172.21.15.114:0/2834089823 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f91b032790 msgr2=0x55f91b032b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:01.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.908+0000 7f5b58f62640 1 --2- 172.21.15.114:0/2834089823 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f91b032790 0x55f91b032b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.908+0000 7f5b58f62640 1 -- 172.21.15.114:0/2834089823 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f91af5d620 con 0x55f91b03d950 2024-09-13T18:30:01.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.908+0000 7f5b58f62640 1 --2- 172.21.15.114:0/2834089823 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f91b03d950 0x55f91b03fd40 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5b4400bd50 tx=0x7f5b4400d420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fb6d29b789f381a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:01.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.910+0000 7f5b3f7fe640 1 -- 172.21.15.114:0/2834089823 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b44010070 con 0x55f91b03d950 2024-09-13T18:30:01.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.910+0000 7f5b3f7fe640 1 -- 172.21.15.114:0/2834089823 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5b44004d00 con 0x55f91b03d950 2024-09-13T18:30:01.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.911+0000 7f5b5e7c6740 1 -- 172.21.15.114:0/2834089823 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f91b03d950 msgr2=0x55f91b03fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:01.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.911+0000 7f5b5e7c6740 1 --2- 172.21.15.114:0/2834089823 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f91b03d950 0x55f91b03fd40 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5b4400bd50 tx=0x7f5b4400d420 comp rx=0 tx=0).stop 2024-09-13T18:30:01.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.912+0000 7f5b5e7c6740 1 -- 172.21.15.114:0/2834089823 shutdown_connections 2024-09-13T18:30:01.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.912+0000 7f5b5e7c6740 1 --2- 172.21.15.114:0/2834089823 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f91b03d950 0x55f91b03fd40 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.912+0000 7f5b5e7c6740 1 --2- 172.21.15.114:0/2834089823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f91b0330b0 0x55f91b03d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.912+0000 7f5b5e7c6740 1 --2- 172.21.15.114:0/2834089823 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f91b032790 0x55f91b032b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.912+0000 7f5b5e7c6740 1 -- 172.21.15.114:0/2834089823 >> 172.21.15.114:0/2834089823 conn(0x55f91b02ed00 msgr2=0x55f91b02f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:01.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.912+0000 7f5b59f64640 1 --2- 172.21.15.114:0/369188424 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f5b400a1410 0x7f5b400a38d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c005c40 tx=0x7f5b4c01a120 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:01.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.914+0000 7f5b5e7c6740 1 -- 172.21.15.114:0/2834089823 shutdown_connections 2024-09-13T18:30:01.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.918+0000 7f5b5e7c6740 1 -- 172.21.15.114:0/2834089823 wait complete. 2024-09-13T18:30:01.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.918+0000 7f5b5e7c6740 1 -- start start 2024-09-13T18:30:01.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.918+0000 7f5b5e7c6740 1 Processor -- start 2024-09-13T18:30:01.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.919+0000 7f5b5e7c6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f91b032790 0x55f91b03cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:01.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.919+0000 7f5b5e7c6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f91b0330b0 0x55f91b03a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:01.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.919+0000 7f5b5e7c6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f91b03d950 0x55f91b03a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:01.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.919+0000 7f5b5e7c6740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91af54180 con 0x55f91b0330b0 2024-09-13T18:30:01.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.919+0000 7f5b5e7c6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91ac6e430 con 0x55f91b032790 2024-09-13T18:30:01.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.919+0000 7f5b5e7c6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91af195e0 con 0x55f91b03d950 2024-09-13T18:30:01.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.919+0000 7f5b5e7c6740 10 client.0 _ll_register_callbacks cb 0x55f91af85aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:30:01.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.919+0000 7f5b58f62640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f91b032790 0x55f91b03cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:01.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.919+0000 7f5b58f62640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f91b032790 0x55f91b03cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:43682/0 (socket says 172.21.15.114:43682) 2024-09-13T18:30:01.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.919+0000 7f5b58f62640 1 -- 172.21.15.114:0/1081257712 learned_addr learned my addr 172.21.15.114:0/1081257712 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:01.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.919+0000 7f5b58f62640 1 --2- 172.21.15.114:0/1081257712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f91b03d950 0x55f91b03a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:01.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.921+0000 7f5b58f62640 1 -- 172.21.15.114:0/1081257712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f91b032790 msgr2=0x55f91b03cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:01.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.921+0000 7f5b58f62640 1 --2- 172.21.15.114:0/1081257712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f91b032790 0x55f91b03cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.921+0000 7f5b58f62640 1 -- 172.21.15.114:0/1081257712 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f91b0330b0 msgr2=0x55f91b03a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:01.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.921+0000 7f5b58f62640 1 --2- 172.21.15.114:0/1081257712 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f91b0330b0 0x55f91b03a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.921+0000 7f5b58f62640 1 -- 172.21.15.114:0/1081257712 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b4400ba00 con 0x55f91b03d950 2024-09-13T18:30:01.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.923+0000 7f5b58f62640 1 --2- 172.21.15.114:0/1081257712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f91b03d950 0x55f91b03a7b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5b4400d920 tx=0x7f5b44006800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26593c903fc9dc42 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:01.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.924+0000 7f5b3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:30:01.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.924+0000 7f5b3cff9640 1 -- 172.21.15.114:0/1081257712 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b44010070 con 0x55f91b03d950 2024-09-13T18:30:01.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.924+0000 7f5b5e7c6740 1 -- 172.21.15.114:0/1081257712 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f91af881f0 con 0x55f91b03d950 2024-09-13T18:30:01.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.924+0000 7f5b3cff9640 1 -- 172.21.15.114:0/1081257712 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f5b44004220 con 0x55f91b03d950 2024-09-13T18:30:01.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.924+0000 7f5b3cff9640 1 -- 172.21.15.114:0/1081257712 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b440043e0 con 0x55f91b03d950 2024-09-13T18:30:01.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.924+0000 7f5b5e7c6740 10 client.27193 fetch_fsmap learned FSMap version 34 2024-09-13T18:30:01.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.924+0000 7f5b5e7c6740 1 -- 172.21.15.114:0/1081257712 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55f91af943c0 con 0x55f91b03d950 2024-09-13T18:30:01.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.926+0000 7f5b3cff9640 1 -- 172.21.15.114:0/1081257712 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f91af881f0 con 0x55f91b03d950 2024-09-13T18:30:01.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.926+0000 7f5b3cff9640 1 -- 172.21.15.114:0/1081257712 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5b44029080 con 0x55f91b03d950 2024-09-13T18:30:01.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.926+0000 7f5b5e7c6740 10 client.27193 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:30:01.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.926+0000 7f5b5e7c6740 10 client.27193 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:30:01.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.926+0000 7f5b5e7c6740 20 client.27193 populate_metadata read hostname 'smithi114' 2024-09-13T18:30:01.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.926+0000 7f5b5e7c6740 1 --2- 172.21.15.114:0/1081257712 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f91b04d3e0 0x55f91b04f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:01.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.926+0000 7f5b5e7c6740 4 client.27193 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:30:01.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.926+0000 7f5b5e7c6740 1 -- 172.21.15.114:0/1081257712 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f91ac894e0 con 0x55f91b04d3e0 2024-09-13T18:30:01.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.926+0000 7f5b59763640 1 --2- 172.21.15.114:0/1081257712 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f91b04d3e0 0x55f91b04f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:01.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.928+0000 7f5b59763640 1 --2- 172.21.15.114:0/1081257712 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f91b04d3e0 0x55f91b04f800 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a4899a15f9bc273 server_cookie=b38fed760e20cb7c in_seq=0 out_seq=0 2024-09-13T18:30:01.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.928+0000 7f5b3cff9640 10 client.27193 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:30:01.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.932+0000 7f5b3cff9640 1 -- 172.21.15.114:0/1081257712 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55f91ac894e0 con 0x55f91b04d3e0 2024-09-13T18:30:01.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.932+0000 7f5b3cff9640 10 client.27193 handle_command_reply: tid=0 2024-09-13T18:30:01.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.932+0000 7f5b5e7c6740 1 client.27193 shutdown 2024-09-13T18:30:01.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.938+0000 7f5b5e7c6740 1 -- 172.21.15.114:0/1081257712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f91b03d950 msgr2=0x55f91b03a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:01.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.938+0000 7f5b5e7c6740 1 --2- 172.21.15.114:0/1081257712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f91b03d950 0x55f91b03a7b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5b4400d920 tx=0x7f5b44006800 comp rx=0 tx=0).stop 2024-09-13T18:30:01.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.939+0000 7f5b5e7c6740 1 -- 172.21.15.114:0/1081257712 shutdown_connections 2024-09-13T18:30:01.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.939+0000 7f5b5e7c6740 1 --2- 172.21.15.114:0/1081257712 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f91b04d3e0 0x55f91b04f800 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.939+0000 7f5b5e7c6740 1 --2- 172.21.15.114:0/1081257712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f91b03d950 0x55f91b03a7b0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.939+0000 7f5b5e7c6740 1 --2- 172.21.15.114:0/1081257712 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f91b0330b0 0x55f91b03a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.939+0000 7f5b5e7c6740 1 --2- 172.21.15.114:0/1081257712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f91b032790 0x55f91b03cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.939+0000 7f5b5e7c6740 1 -- 172.21.15.114:0/1081257712 >> 172.21.15.114:0/1081257712 conn(0x55f91b02ed00 msgr2=0x55f91b034a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:01.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.940+0000 7f5b3cff9640 0 client.27193 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:30:01.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.940+0000 7f5b5e7c6740 1 -- 172.21.15.114:0/1081257712 shutdown_connections 2024-09-13T18:30:01.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.940+0000 7f5b5e7c6740 1 -- 172.21.15.114:0/1081257712 wait complete. 2024-09-13T18:30:01.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.941+0000 7f5b5e7c6740 20 client.27193 trim_cache size 0 max 16384 2024-09-13T18:30:01.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.971+0000 7f5b5e7c6740 1 Processor -- start 2024-09-13T18:30:01.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.971+0000 7f5b5e7c6740 1 -- start start 2024-09-13T18:30:01.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.971+0000 7f5b5e7c6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f91b02ed00 0x55f91b02e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:01.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.971+0000 7f5b5e7c6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f91b032790 0x55f91b054160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:01.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.972+0000 7f5b5e7c6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f91b0330b0 0x55f91b044740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:01.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.972+0000 7f5b5e7c6740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91b03caf0 con 0x55f91b0330b0 2024-09-13T18:30:01.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.972+0000 7f5b59763640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f91b032790 0x55f91b054160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:01.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.972+0000 7f5b59763640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f91b032790 0x55f91b054160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:43698/0 (socket says 172.21.15.114:43698) 2024-09-13T18:30:01.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.972+0000 7f5b59763640 1 -- 172.21.15.114:0/2878516723 learned_addr learned my addr 172.21.15.114:0/2878516723 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:01.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.972+0000 7f5b5e7c6740 1 -- 172.21.15.114:0/2878516723 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91b0387a0 con 0x55f91b032790 2024-09-13T18:30:01.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.972+0000 7f5b5e7c6740 1 -- 172.21.15.114:0/2878516723 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91b049a20 con 0x55f91b02ed00 2024-09-13T18:30:01.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.973+0000 7f5b59763640 1 -- 172.21.15.114:0/2878516723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f91b02ed00 msgr2=0x55f91b02e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:01.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.973+0000 7f5b59763640 1 --2- 172.21.15.114:0/2878516723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f91b02ed00 0x55f91b02e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.973+0000 7f5b59763640 1 -- 172.21.15.114:0/2878516723 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f91b0330b0 msgr2=0x55f91b044740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:01.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.973+0000 7f5b59763640 1 --2- 172.21.15.114:0/2878516723 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f91b0330b0 0x55f91b044740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.973+0000 7f5b59763640 1 -- 172.21.15.114:0/2878516723 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f91af36770 con 0x55f91b032790 2024-09-13T18:30:01.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.974+0000 7f5b59763640 1 --2- 172.21.15.114:0/2878516723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f91b032790 0x55f91b054160 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5b5000b620 tx=0x7f5b5000b570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=514a2a1ef2ff63f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:01.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.975+0000 7f5b3ffff640 1 -- 172.21.15.114:0/2878516723 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b50017040 con 0x55f91b032790 2024-09-13T18:30:01.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.975+0000 7f5b3ffff640 1 -- 172.21.15.114:0/2878516723 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5b500035f0 con 0x55f91b032790 2024-09-13T18:30:01.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.975+0000 7f5b3ffff640 1 -- 172.21.15.114:0/2878516723 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b500116a0 con 0x55f91b032790 2024-09-13T18:30:01.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.976+0000 7f5b5e7c6740 1 -- 172.21.15.114:0/2878516723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f91b032790 msgr2=0x55f91b054160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:01.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.976+0000 7f5b5e7c6740 1 --2- 172.21.15.114:0/2878516723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f91b032790 0x55f91b054160 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5b5000b620 tx=0x7f5b5000b570 comp rx=0 tx=0).stop 2024-09-13T18:30:01.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.977+0000 7f5b5e7c6740 1 -- 172.21.15.114:0/2878516723 shutdown_connections 2024-09-13T18:30:01.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.977+0000 7f5b5e7c6740 1 --2- 172.21.15.114:0/2878516723 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f91b0330b0 0x55f91b044740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.977+0000 7f5b5e7c6740 1 --2- 172.21.15.114:0/2878516723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f91b032790 0x55f91b054160 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.977+0000 7f5b5e7c6740 1 --2- 172.21.15.114:0/2878516723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f91b02ed00 0x55f91b02e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.977+0000 7f5b5e7c6740 1 -- 172.21.15.114:0/2878516723 >> 172.21.15.114:0/2878516723 conn(0x55f91b03d060 msgr2=0x55f91b02f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:01.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.981+0000 7f5b5e7c6740 1 -- 172.21.15.114:0/2878516723 shutdown_connections 2024-09-13T18:30:01.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.981+0000 7f5b5e7c6740 1 -- 172.21.15.114:0/2878516723 wait complete. 2024-09-13T18:30:01.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.981+0000 7f5b5e7c6740 1 -- start start 2024-09-13T18:30:01.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.981+0000 7f5b5e7c6740 1 Processor -- start 2024-09-13T18:30:01.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.982+0000 7f5b5e7c6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f91b02ed00 0x55f91b040720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:01.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.982+0000 7f5b5e7c6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f91b032790 0x55f91b040c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:01.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.982+0000 7f5b5e7c6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f91b0330b0 0x55f91b0a28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:01.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.982+0000 7f5b5e7c6740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91b0415b0 con 0x55f91b032790 2024-09-13T18:30:01.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.982+0000 7f5b5e7c6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91b0a2e10 con 0x55f91b0330b0 2024-09-13T18:30:01.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.982+0000 7f5b5e7c6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91b0a2f80 con 0x55f91b02ed00 2024-09-13T18:30:01.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.982+0000 7f5b5e7c6740 10 client.0 _ll_register_callbacks cb 0x7f5b4400b580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:30:01.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.983+0000 7f5b59763640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f91b032790 0x55f91b040c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:01.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.983+0000 7f5b58f62640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f91b0330b0 0x55f91b0a28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:01.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.983+0000 7f5b58f62640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f91b0330b0 0x55f91b0a28d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:43704/0 (socket says 172.21.15.114:43704) 2024-09-13T18:30:01.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.983+0000 7f5b58f62640 1 -- 172.21.15.114:0/2616166739 learned_addr learned my addr 172.21.15.114:0/2616166739 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:01.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.983+0000 7f5b58f62640 1 --2- 172.21.15.114:0/2616166739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f91b02ed00 0x55f91b040720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:01.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.984+0000 7f5b58f62640 1 -- 172.21.15.114:0/2616166739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f91b02ed00 msgr2=0x55f91b040720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:01.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.984+0000 7f5b58f62640 1 --2- 172.21.15.114:0/2616166739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f91b02ed00 0x55f91b040720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.984+0000 7f5b58f62640 1 -- 172.21.15.114:0/2616166739 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f91b032790 msgr2=0x55f91b040c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:01.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.984+0000 7f5b58f62640 1 --2- 172.21.15.114:0/2616166739 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f91b032790 0x55f91b040c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:01.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.984+0000 7f5b58f62640 1 -- 172.21.15.114:0/2616166739 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b50004380 con 0x55f91b0330b0 2024-09-13T18:30:01.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.985+0000 7f5b58f62640 1 --2- 172.21.15.114:0/2616166739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f91b0330b0 0x55f91b0a28d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5b44006840 tx=0x7f5b4400d420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51cfc554f80b677 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:01.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.986+0000 7f5b3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:30:01.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.987+0000 7f5b3cff9640 1 -- 172.21.15.114:0/2616166739 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b44010070 con 0x55f91b0330b0 2024-09-13T18:30:01.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.989+0000 7f5b5e7c6740 1 -- 172.21.15.114:0/2616166739 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f91afbfb80 con 0x55f91b0330b0 2024-09-13T18:30:01.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.989+0000 7f5b3cff9640 1 -- 172.21.15.114:0/2616166739 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f5b440038b0 con 0x55f91b0330b0 2024-09-13T18:30:02.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:01.996+0000 7f5b3cff9640 1 -- 172.21.15.114:0/2616166739 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b44003a70 con 0x55f91b0330b0 2024-09-13T18:30:02.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:02.000+0000 7f5b5e7c6740 10 client.27209 fetch_fsmap learned FSMap version 34 2024-09-13T18:30:02.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:02.000+0000 7f5b5e7c6740 1 -- 172.21.15.114:0/2616166739 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55f91af5be80 con 0x55f91b0330b0 2024-09-13T18:30:02.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:02.000+0000 7f5b3cff9640 1 -- 172.21.15.114:0/2616166739 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f91afbfb80 con 0x55f91b0330b0 2024-09-13T18:30:02.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:02.000+0000 7f5b3cff9640 1 -- 172.21.15.114:0/2616166739 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5b4401e040 con 0x55f91b0330b0 2024-09-13T18:30:02.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:02.000+0000 7f5b5e7c6740 10 client.27209 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:30:02.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:02.000+0000 7f5b5e7c6740 10 client.27209 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:30:02.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:02.000+0000 7f5b5e7c6740 20 client.27209 populate_metadata read hostname 'smithi114' 2024-09-13T18:30:02.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:02.000+0000 7f5b5e7c6740 1 --2- 172.21.15.114:0/2616166739 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f91b03d950 0x55f91b0a6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:02.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:02.000+0000 7f5b5e7c6740 4 client.27209 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:30:02.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:02.000+0000 7f5b5e7c6740 1 -- 172.21.15.114:0/2616166739 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f91ac9a7e0 con 0x55f91b03d950 2024-09-13T18:30:02.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:02.001+0000 7f5b59763640 1 --2- 172.21.15.114:0/2616166739 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f91b03d950 0x55f91b0a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:02.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:02.008+0000 7f5b59763640 1 --2- 172.21.15.114:0/2616166739 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f91b03d950 0x55f91b0a6360 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f72f275894c778c4 server_cookie=b271e0c05af03d60 in_seq=0 out_seq=0 2024-09-13T18:30:02.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:02.009+0000 7f5b3cff9640 10 client.27209 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:30:02.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:02.010+0000 7f5b3cff9640 1 -- 172.21.15.114:0/2616166739 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f91ac9a7e0 con 0x55f91b03d950 2024-09-13T18:30:02.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:02.010+0000 7f5b3cff9640 10 client.27209 handle_command_reply: tid=0 2024-09-13T18:30:02.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:02.010+0000 7f5b5e7c6740 1 client.27209 shutdown 2024-09-13T18:30:02.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:02.019+0000 7f5b5e7c6740 1 -- 172.21.15.114:0/2616166739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f91b0330b0 msgr2=0x55f91b0a28d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:02.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:02.019+0000 7f5b5e7c6740 1 --2- 172.21.15.114:0/2616166739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f91b0330b0 0x55f91b0a28d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5b44006840 tx=0x7f5b4400d420 comp rx=0 tx=0).stop 2024-09-13T18:30:02.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:02.023+0000 7f5b5e7c6740 1 -- 172.21.15.114:0/2616166739 shutdown_connections 2024-09-13T18:30:02.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:02.023+0000 7f5b5e7c6740 1 --2- 172.21.15.114:0/2616166739 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f91b03d950 0x55f91b0a6360 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:02.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:02.023+0000 7f5b5e7c6740 1 --2- 172.21.15.114:0/2616166739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f91b0330b0 0x55f91b0a28d0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:02.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:02.023+0000 7f5b5e7c6740 1 --2- 172.21.15.114:0/2616166739 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f91b032790 0x55f91b040c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:02.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:02.023+0000 7f5b5e7c6740 1 --2- 172.21.15.114:0/2616166739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f91b02ed00 0x55f91b040720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:02.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:02.023+0000 7f5b5e7c6740 1 -- 172.21.15.114:0/2616166739 >> 172.21.15.114:0/2616166739 conn(0x55f91b03d060 msgr2=0x55f91b054160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:02.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:02.023+0000 7f5b3cff9640 0 client.27209 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:30:02.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:02.025+0000 7f5b5e7c6740 1 -- 172.21.15.114:0/2616166739 shutdown_connections 2024-09-13T18:30:02.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:02.027+0000 7f5b5e7c6740 1 -- 172.21.15.114:0/2616166739 wait complete. 2024-09-13T18:30:02.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:02.031+0000 7f5b5e7c6740 20 client.27209 trim_cache size 0 max 16384 2024-09-13T18:30:02.037 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:30:02.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:02.035+0000 7f5b5b9c7640 1 -- 172.21.15.114:0/369188424 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f5b400a1410 msgr2=0x7f5b400a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:02.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:02.035+0000 7f5b5b9c7640 1 --2- 172.21.15.114:0/369188424 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f5b400a1410 0x7f5b400a38d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c005c40 tx=0x7f5b4c01a120 comp rx=0 tx=0).stop 2024-09-13T18:30:02.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:02.035+0000 7f5b5b9c7640 1 -- 172.21.15.114:0/369188424 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b54119400 msgr2=0x7f5b541a1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:02.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:02.035+0000 7f5b5b9c7640 1 --2- 172.21.15.114:0/369188424 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b54119400 0x7f5b541a1240 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c03be90 tx=0x7f5b4c040930 comp rx=0 tx=0).stop 2024-09-13T18:30:02.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:02.038+0000 7f5b5b9c7640 1 -- 172.21.15.114:0/369188424 shutdown_connections 2024-09-13T18:30:02.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:02.038+0000 7f5b5b9c7640 1 --2- 172.21.15.114:0/369188424 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f5b400a1410 0x7f5b400a38d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:02.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:02.039+0000 7f5b5b9c7640 1 --2- 172.21.15.114:0/369188424 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b54119400 0x7f5b541a1240 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:02.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:02.039+0000 7f5b5b9c7640 1 --2- 172.21.15.114:0/369188424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b54107920 0x7f5b541150b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:02.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:02.039+0000 7f5b5b9c7640 1 --2- 172.21.15.114:0/369188424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b54107540 0x7f5b54114b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:02.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:02.039+0000 7f5b5b9c7640 1 -- 172.21.15.114:0/369188424 >> 172.21.15.114:0/369188424 conn(0x7f5b5406afe0 msgr2=0x7f5b5406ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:02.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:02.044+0000 7f5b5b9c7640 1 -- 172.21.15.114:0/369188424 shutdown_connections 2024-09-13T18:30:02.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:02.049+0000 7f5b5b9c7640 1 -- 172.21.15.114:0/369188424 wait complete. 2024-09-13T18:30:02.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:01 smithi134 ceph-mon[34283]: pgmap v231: 129 pgs: 129 active+clean; 22 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 44 MiB/s wr, 956 op/s 2024-09-13T18:30:02.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:01 smithi204 ceph-mon[34777]: pgmap v231: 129 pgs: 129 active+clean; 22 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 44 MiB/s wr, 956 op/s 2024-09-13T18:30:02.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:01 smithi114 ceph-mon[30897]: pgmap v231: 129 pgs: 129 active+clean; 22 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 44 MiB/s wr, 956 op/s 2024-09-13T18:30:03.080 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-13T18:30:03.080 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-09-13T18:30:03.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.303+0000 7fb0d4df8640 1 -- 172.21.15.114:0/3850917752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0d0119400 msgr2=0x7fb0d011b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:03.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.303+0000 7fb0d4df8640 1 --2- 172.21.15.114:0/3850917752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0d0119400 0x7fb0d011b7f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8009ca0 tx=0x7fb0c802a8d0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.309+0000 7fb0d4df8640 1 -- 172.21.15.114:0/3850917752 shutdown_connections 2024-09-13T18:30:03.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.309+0000 7fb0d4df8640 1 --2- 172.21.15.114:0/3850917752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0d0119400 0x7fb0d011b7f0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.309+0000 7fb0d4df8640 1 --2- 172.21.15.114:0/3850917752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0d0107890 0x7fb0d0107d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.309+0000 7fb0d4df8640 1 --2- 172.21.15.114:0/3850917752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0d01074b0 0x7fb0d010e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.309+0000 7fb0d4df8640 1 -- 172.21.15.114:0/3850917752 >> 172.21.15.114:0/3850917752 conn(0x7fb0d006aec0 msgr2=0x7fb0d006b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:03.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.316+0000 7fb0d4df8640 1 -- 172.21.15.114:0/3850917752 shutdown_connections 2024-09-13T18:30:03.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.317+0000 7fb0d4df8640 1 -- 172.21.15.114:0/3850917752 wait complete. 2024-09-13T18:30:03.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.317+0000 7fb0d4df8640 1 Processor -- start 2024-09-13T18:30:03.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.318+0000 7fb0d4df8640 1 -- start start 2024-09-13T18:30:03.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.318+0000 7fb0d4df8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0d01074b0 0x7fb0d0112650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:03.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.318+0000 7fb0d4df8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0d0107890 0x7fb0d0112b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:03.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.318+0000 7fb0d4df8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0d0119400 0x7fb0d0116f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:03.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.318+0000 7fb0d4df8640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d01174c0 con 0x7fb0d0119400 2024-09-13T18:30:03.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.318+0000 7fb0d4df8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d0117630 con 0x7fb0d01074b0 2024-09-13T18:30:03.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.318+0000 7fb0d4df8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d01177f0 con 0x7fb0d0107890 2024-09-13T18:30:03.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.321+0000 7fb0cffff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0d0119400 0x7fb0d0116f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:03.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.321+0000 7fb0cffff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0d0119400 0x7fb0d0116f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:56264/0 (socket says 172.21.15.114:56264) 2024-09-13T18:30:03.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.321+0000 7fb0cffff640 1 -- 172.21.15.114:0/982035274 learned_addr learned my addr 172.21.15.114:0/982035274 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:03.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.322+0000 7fb0cf7fe640 1 --2- 172.21.15.114:0/982035274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0d01074b0 0x7fb0d0112650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:03.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.322+0000 7fb0ceffd640 1 --2- 172.21.15.114:0/982035274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0d0107890 0x7fb0d0112b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:03.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.325+0000 7fb0cffff640 1 -- 172.21.15.114:0/982035274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0d0107890 msgr2=0x7fb0d0112b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:03.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.325+0000 7fb0cffff640 1 --2- 172.21.15.114:0/982035274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0d0107890 0x7fb0d0112b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.325+0000 7fb0cffff640 1 -- 172.21.15.114:0/982035274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0d01074b0 msgr2=0x7fb0d0112650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:03.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.325+0000 7fb0cffff640 1 --2- 172.21.15.114:0/982035274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0d01074b0 0x7fb0d0112650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.325+0000 7fb0cffff640 1 -- 172.21.15.114:0/982035274 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0c8009900 con 0x7fb0d0119400 2024-09-13T18:30:03.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.326+0000 7fb0cffff640 1 --2- 172.21.15.114:0/982035274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0d0119400 0x7fb0d0116f80 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8009bf0 tx=0x7fb0c8031a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:03.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.330+0000 7fb0ccff9640 1 -- 172.21.15.114:0/982035274 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c80309e0 con 0x7fb0d0119400 2024-09-13T18:30:03.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.330+0000 7fb0d4df8640 1 -- 172.21.15.114:0/982035274 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0d0117a70 con 0x7fb0d0119400 2024-09-13T18:30:03.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.331+0000 7fb0ccff9640 1 -- 172.21.15.114:0/982035274 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb0c8031b90 con 0x7fb0d0119400 2024-09-13T18:30:03.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.331+0000 7fb0ccff9640 1 -- 172.21.15.114:0/982035274 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c802f8d0 con 0x7fb0d0119400 2024-09-13T18:30:03.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.331+0000 7fb0d4df8640 1 -- 172.21.15.114:0/982035274 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0d01ba500 con 0x7fb0d0119400 2024-09-13T18:30:03.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.338+0000 7fb0ccff9640 1 -- 172.21.15.114:0/982035274 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fb0c8042070 con 0x7fb0d0119400 2024-09-13T18:30:03.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.339+0000 7fb0ccff9640 1 --2- 172.21.15.114:0/982035274 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fb0b80a1410 0x7fb0b80a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:03.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.339+0000 7fb0ccff9640 1 -- 172.21.15.114:0/982035274 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fb0c80ee890 con 0x7fb0d0119400 2024-09-13T18:30:03.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.341+0000 7fb0cf7fe640 1 --2- 172.21.15.114:0/982035274 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fb0b80a1410 0x7fb0b80a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:03.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.342+0000 7fb0d8e56740 1 Processor -- start 2024-09-13T18:30:03.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.342+0000 7fb0d8e56740 1 -- start start 2024-09-13T18:30:03.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.342+0000 7fb0d8e56740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561cb41299e0 0x561cb4129dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:03.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.342+0000 7fb0d8e56740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cb412a300 0x561cb4134660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:03.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.342+0000 7fb0d8e56740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cb4134ba0 0x561cb4136f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:03.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.342+0000 7fb0d8e56740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cb4094c60 con 0x561cb41299e0 2024-09-13T18:30:03.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.342+0000 7fb0d8e56740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cb40923f0 con 0x561cb412a300 2024-09-13T18:30:03.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.342+0000 7fb0d8e56740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cb408dce0 con 0x561cb4134ba0 2024-09-13T18:30:03.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.343+0000 7fb0ceffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cb412a300 0x561cb4134660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:03.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.343+0000 7fb0ceffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cb412a300 0x561cb4134660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:60742/0 (socket says 172.21.15.114:60742) 2024-09-13T18:30:03.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.343+0000 7fb0ceffd640 1 -- 172.21.15.114:0/413829677 learned_addr learned my addr 172.21.15.114:0/413829677 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:03.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.343+0000 7fb0ceffd640 1 -- 172.21.15.114:0/413829677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cb4134ba0 msgr2=0x561cb4136f90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:30:03.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.343+0000 7fb0ceffd640 1 --2- 172.21.15.114:0/413829677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cb4134ba0 0x561cb4136f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.343+0000 7fb0ceffd640 1 -- 172.21.15.114:0/413829677 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561cb41299e0 msgr2=0x561cb4129dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:03.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.343+0000 7fb0ceffd640 1 --2- 172.21.15.114:0/413829677 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561cb41299e0 0x561cb4129dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.343+0000 7fb0ceffd640 1 -- 172.21.15.114:0/413829677 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561cb4049970 con 0x561cb412a300 2024-09-13T18:30:03.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.348+0000 7fb0cf7fe640 1 --2- 172.21.15.114:0/982035274 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fb0b80a1410 0x7fb0b80a38d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0006fd0 tx=0x7fb0c00090e0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:03.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.348+0000 7fb0ceffd640 1 --2- 172.21.15.114:0/413829677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cb412a300 0x561cb4134660 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb0c400b7c0 tx=0x7fb0c400bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=231674c6a23ffc2b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:03.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.352+0000 7fb0b5ffb640 1 -- 172.21.15.114:0/413829677 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c400d950 con 0x561cb412a300 2024-09-13T18:30:03.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.352+0000 7fb0b5ffb640 1 -- 172.21.15.114:0/413829677 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb0c4013480 con 0x561cb412a300 2024-09-13T18:30:03.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.353+0000 7fb0d8e56740 1 -- 172.21.15.114:0/413829677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cb412a300 msgr2=0x561cb4134660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:03.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.353+0000 7fb0d8e56740 1 --2- 172.21.15.114:0/413829677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cb412a300 0x561cb4134660 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb0c400b7c0 tx=0x7fb0c400bc80 comp rx=0 tx=0).stop 2024-09-13T18:30:03.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.356+0000 7fb0d8e56740 1 -- 172.21.15.114:0/413829677 shutdown_connections 2024-09-13T18:30:03.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.356+0000 7fb0d8e56740 1 --2- 172.21.15.114:0/413829677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cb4134ba0 0x561cb4136f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.356+0000 7fb0d8e56740 1 --2- 172.21.15.114:0/413829677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cb412a300 0x561cb4134660 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.356+0000 7fb0d8e56740 1 --2- 172.21.15.114:0/413829677 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561cb41299e0 0x561cb4129dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.356+0000 7fb0d8e56740 1 -- 172.21.15.114:0/413829677 >> 172.21.15.114:0/413829677 conn(0x561cb4125f70 msgr2=0x561cb4126350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:03.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.368+0000 7fb0d8e56740 1 -- 172.21.15.114:0/413829677 shutdown_connections 2024-09-13T18:30:03.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.368+0000 7fb0d8e56740 1 -- 172.21.15.114:0/413829677 wait complete. 2024-09-13T18:30:03.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.368+0000 7fb0d8e56740 1 -- start start 2024-09-13T18:30:03.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.368+0000 7fb0d8e56740 1 Processor -- start 2024-09-13T18:30:03.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.369+0000 7fb0d8e56740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cb41299e0 0x561cb4134570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:03.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.369+0000 7fb0d8e56740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561cb412a300 0x561cb4131860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:03.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.369+0000 7fb0d8e56740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cb4134ba0 0x561cb4131e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:03.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.369+0000 7fb0d8e56740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cb404bf80 con 0x561cb412a300 2024-09-13T18:30:03.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.369+0000 7fb0d8e56740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cb404bc20 con 0x561cb41299e0 2024-09-13T18:30:03.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.369+0000 7fb0d8e56740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cb4018ab0 con 0x561cb4134ba0 2024-09-13T18:30:03.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.369+0000 7fb0d8e56740 10 client.0 _ll_register_callbacks cb 0x561cb4087350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:30:03.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.371+0000 7fb0ceffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561cb412a300 0x561cb4131860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:03.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.371+0000 7fb0ceffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561cb412a300 0x561cb4131860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:56274/0 (socket says 172.21.15.114:56274) 2024-09-13T18:30:03.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.371+0000 7fb0ceffd640 1 -- 172.21.15.114:0/1862723694 learned_addr learned my addr 172.21.15.114:0/1862723694 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:03.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.371+0000 7fb0cffff640 1 --2- 172.21.15.114:0/1862723694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cb4134ba0 0x561cb4131e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:03.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.374+0000 7fb0cf7fe640 1 --2- 172.21.15.114:0/1862723694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cb41299e0 0x561cb4134570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:03.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.377+0000 7fb0ceffd640 1 -- 172.21.15.114:0/1862723694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cb4134ba0 msgr2=0x561cb4131e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:03.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.377+0000 7fb0ceffd640 1 --2- 172.21.15.114:0/1862723694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cb4134ba0 0x561cb4131e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.377+0000 7fb0ceffd640 1 -- 172.21.15.114:0/1862723694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cb41299e0 msgr2=0x561cb4134570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:03.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.377+0000 7fb0ceffd640 1 --2- 172.21.15.114:0/1862723694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cb41299e0 0x561cb4134570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.377+0000 7fb0ceffd640 1 -- 172.21.15.114:0/1862723694 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0c400e040 con 0x561cb412a300 2024-09-13T18:30:03.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.382+0000 7fb0ceffd640 1 --2- 172.21.15.114:0/1862723694 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561cb412a300 0x561cb4131860 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4004700 tx=0x7fb0c4013820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2daa40cfd261de44 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:03.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.384+0000 7fb0aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:30:03.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.385+0000 7fb0aeffd640 1 -- 172.21.15.114:0/1862723694 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c4012640 con 0x561cb412a300 2024-09-13T18:30:03.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.386+0000 7fb0d8e56740 1 -- 172.21.15.114:0/1862723694 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561cb3d81590 con 0x561cb412a300 2024-09-13T18:30:03.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.386+0000 7fb0aeffd640 1 -- 172.21.15.114:0/1862723694 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fb0c400db60 con 0x561cb412a300 2024-09-13T18:30:03.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.392+0000 7fb0aeffd640 1 -- 172.21.15.114:0/1862723694 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0c4013bd0 con 0x561cb412a300 2024-09-13T18:30:03.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.395+0000 7fb0d8e56740 10 client.17790 fetch_fsmap learned FSMap version 34 2024-09-13T18:30:03.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.395+0000 7fb0d8e56740 1 -- 172.21.15.114:0/1862723694 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x561cb408c360 con 0x561cb412a300 2024-09-13T18:30:03.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.400+0000 7fb0aeffd640 1 -- 172.21.15.114:0/1862723694 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561cb3d81590 con 0x561cb412a300 2024-09-13T18:30:03.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.400+0000 7fb0aeffd640 1 -- 172.21.15.114:0/1862723694 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb0c40315c0 con 0x561cb412a300 2024-09-13T18:30:03.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.400+0000 7fb0d8e56740 10 client.17790 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:30:03.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.400+0000 7fb0d8e56740 10 client.17790 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:30:03.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.400+0000 7fb0d8e56740 20 client.17790 populate_metadata read hostname 'smithi114' 2024-09-13T18:30:03.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.400+0000 7fb0d8e56740 1 --2- 172.21.15.114:0/1862723694 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x561cb4144900 0x561cb4146d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:03.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.400+0000 7fb0d8e56740 4 client.17790 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:30:03.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.400+0000 7fb0d8e56740 1 -- 172.21.15.114:0/1862723694 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561cb3d92860 con 0x561cb4144900 2024-09-13T18:30:03.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.407+0000 7fb0cf7fe640 1 --2- 172.21.15.114:0/1862723694 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x561cb4144900 0x561cb4146d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:03.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.421+0000 7fb0cf7fe640 1 --2- 172.21.15.114:0/1862723694 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x561cb4144900 0x561cb4146d20 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fd17a32a4163081 server_cookie=68d884e89a9b187f in_seq=0 out_seq=0 2024-09-13T18:30:03.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.425+0000 7fb0aeffd640 10 client.17790 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:30:03.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.425+0000 7fb0aeffd640 1 -- 172.21.15.114:0/1862723694 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x561cb3d92860 con 0x561cb4144900 2024-09-13T18:30:03.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.425+0000 7fb0aeffd640 10 client.17790 handle_command_reply: tid=0 2024-09-13T18:30:03.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.425+0000 7fb0d8e56740 1 client.17790 shutdown 2024-09-13T18:30:03.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.429+0000 7fb0d8e56740 1 -- 172.21.15.114:0/1862723694 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561cb412a300 msgr2=0x561cb4131860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:03.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.429+0000 7fb0d8e56740 1 --2- 172.21.15.114:0/1862723694 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561cb412a300 0x561cb4131860 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4004700 tx=0x7fb0c4013820 comp rx=0 tx=0).stop 2024-09-13T18:30:03.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.429+0000 7fb0d8e56740 1 -- 172.21.15.114:0/1862723694 shutdown_connections 2024-09-13T18:30:03.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.429+0000 7fb0d8e56740 1 --2- 172.21.15.114:0/1862723694 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x561cb4144900 0x561cb4146d20 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.429+0000 7fb0d8e56740 1 --2- 172.21.15.114:0/1862723694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cb4134ba0 0x561cb4131e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.429+0000 7fb0d8e56740 1 --2- 172.21.15.114:0/1862723694 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561cb412a300 0x561cb4131860 secure :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4004700 tx=0x7fb0c4013820 comp rx=0 tx=0).stop 2024-09-13T18:30:03.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.429+0000 7fb0d8e56740 1 --2- 172.21.15.114:0/1862723694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cb41299e0 0x561cb4134570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.429+0000 7fb0d8e56740 1 -- 172.21.15.114:0/1862723694 >> 172.21.15.114:0/1862723694 conn(0x561cb4125f70 msgr2=0x561cb412bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:03.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.429+0000 7fb0aeffd640 0 client.17790 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:30:03.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.436+0000 7fb0d8e56740 1 -- 172.21.15.114:0/1862723694 shutdown_connections 2024-09-13T18:30:03.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.443+0000 7fb0d8e56740 1 -- 172.21.15.114:0/1862723694 wait complete. 2024-09-13T18:30:03.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.445+0000 7fb0d8e56740 20 client.17790 trim_cache size 0 max 16384 2024-09-13T18:30:03.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.481+0000 7fb0d8e56740 1 Processor -- start 2024-09-13T18:30:03.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.481+0000 7fb0d8e56740 1 -- start start 2024-09-13T18:30:03.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.481+0000 7fb0d8e56740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561cb41299e0 0x561cb4125950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:03.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.481+0000 7fb0d8e56740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cb4134ba0 0x561cb412ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:03.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.482+0000 7fb0d8e56740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cb4126a60 0x561cb412cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:03.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.482+0000 7fb0d8e56740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cb4120670 con 0x561cb41299e0 2024-09-13T18:30:03.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.482+0000 7fb0cf7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561cb41299e0 0x561cb4125950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:03.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.482+0000 7fb0cf7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561cb41299e0 0x561cb4125950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:56284/0 (socket says 172.21.15.114:56284) 2024-09-13T18:30:03.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.482+0000 7fb0cf7fe640 1 -- 172.21.15.114:0/353786297 learned_addr learned my addr 172.21.15.114:0/353786297 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:03.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.482+0000 7fb0d8e56740 1 -- 172.21.15.114:0/353786297 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cb4121780 con 0x561cb4126a60 2024-09-13T18:30:03.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.482+0000 7fb0d8e56740 1 -- 172.21.15.114:0/353786297 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cb414be60 con 0x561cb4134ba0 2024-09-13T18:30:03.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.483+0000 7fb0cffff640 1 --2- 172.21.15.114:0/353786297 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cb4126a60 0x561cb412cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:03.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.483+0000 7fb0cffff640 1 -- 172.21.15.114:0/353786297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cb4134ba0 msgr2=0x561cb412ca80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:30:03.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.483+0000 7fb0cffff640 1 --2- 172.21.15.114:0/353786297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cb4134ba0 0x561cb412ca80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.483+0000 7fb0cffff640 1 -- 172.21.15.114:0/353786297 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561cb41299e0 msgr2=0x561cb4125950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:03.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.483+0000 7fb0cffff640 1 --2- 172.21.15.114:0/353786297 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561cb41299e0 0x561cb4125950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.483+0000 7fb0cffff640 1 -- 172.21.15.114:0/353786297 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561cb402e1c0 con 0x561cb4126a60 2024-09-13T18:30:03.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.483+0000 7fb0cffff640 1 --2- 172.21.15.114:0/353786297 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cb4126a60 0x561cb412cfc0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb0c80ebcf0 tx=0x7fb0c8005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c31b33cf51654cb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:03.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.486+0000 7fb0b67fc640 1 -- 172.21.15.114:0/353786297 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c80bb6b0 con 0x561cb4126a60 2024-09-13T18:30:03.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.486+0000 7fb0b67fc640 1 -- 172.21.15.114:0/353786297 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb0c80bf4e0 con 0x561cb4126a60 2024-09-13T18:30:03.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.486+0000 7fb0b67fc640 1 -- 172.21.15.114:0/353786297 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c80bf160 con 0x561cb4126a60 2024-09-13T18:30:03.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.487+0000 7fb0cf7fe640 1 --2- 172.21.15.114:0/353786297 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561cb41299e0 0x561cb4125950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:30:03.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.488+0000 7fb0d8e56740 1 -- 172.21.15.114:0/353786297 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cb4126a60 msgr2=0x561cb412cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:03.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.488+0000 7fb0d8e56740 1 --2- 172.21.15.114:0/353786297 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cb4126a60 0x561cb412cfc0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb0c80ebcf0 tx=0x7fb0c8005fd0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.493+0000 7fb0d8e56740 1 -- 172.21.15.114:0/353786297 shutdown_connections 2024-09-13T18:30:03.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.493+0000 7fb0d8e56740 1 --2- 172.21.15.114:0/353786297 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cb4126a60 0x561cb412cfc0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.493+0000 7fb0d8e56740 1 --2- 172.21.15.114:0/353786297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cb4134ba0 0x561cb412ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.493+0000 7fb0d8e56740 1 --2- 172.21.15.114:0/353786297 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561cb41299e0 0x561cb4125950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.493+0000 7fb0d8e56740 1 -- 172.21.15.114:0/353786297 >> 172.21.15.114:0/353786297 conn(0x561cb4125f70 msgr2=0x561cb4126350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:03.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.500+0000 7fb0d8e56740 1 -- 172.21.15.114:0/353786297 shutdown_connections 2024-09-13T18:30:03.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.500+0000 7fb0d8e56740 1 -- 172.21.15.114:0/353786297 wait complete. 2024-09-13T18:30:03.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.501+0000 7fb0d8e56740 1 -- start start 2024-09-13T18:30:03.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.501+0000 7fb0d8e56740 1 Processor -- start 2024-09-13T18:30:03.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.503+0000 7fb0d8e56740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cb4126a60 0x561cb41384f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:03.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.504+0000 7fb0d8e56740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cb41299e0 0x561cb4138a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:03.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.504+0000 7fb0d8e56740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561cb4134ba0 0x561cb4138f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:03.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.504+0000 7fb0ceffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cb41299e0 0x561cb4138a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:03.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.504+0000 7fb0ceffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cb41299e0 0x561cb4138a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:60760/0 (socket says 172.21.15.114:60760) 2024-09-13T18:30:03.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.504+0000 7fb0ceffd640 1 -- 172.21.15.114:0/3052996059 learned_addr learned my addr 172.21.15.114:0/3052996059 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:03.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.504+0000 7fb0d8e56740 1 -- 172.21.15.114:0/3052996059 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cb419ae60 con 0x561cb4134ba0 2024-09-13T18:30:03.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.504+0000 7fb0d8e56740 1 -- 172.21.15.114:0/3052996059 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cb419afa0 con 0x561cb41299e0 2024-09-13T18:30:03.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.504+0000 7fb0d8e56740 1 -- 172.21.15.114:0/3052996059 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cb419b130 con 0x561cb4126a60 2024-09-13T18:30:03.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.504+0000 7fb0cf7fe640 1 --2- 172.21.15.114:0/3052996059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cb4126a60 0x561cb41384f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:03.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.505+0000 7fb0d8e56740 10 client.0 _ll_register_callbacks cb 0x7fb0c0009f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:30:03.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.508+0000 7fb0ceffd640 1 -- 172.21.15.114:0/3052996059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cb4126a60 msgr2=0x561cb41384f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:03.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.508+0000 7fb0ceffd640 1 --2- 172.21.15.114:0/3052996059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cb4126a60 0x561cb41384f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.508+0000 7fb0ceffd640 1 -- 172.21.15.114:0/3052996059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561cb4134ba0 msgr2=0x561cb4138f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:03.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.508+0000 7fb0ceffd640 1 --2- 172.21.15.114:0/3052996059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561cb4134ba0 0x561cb4138f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.508+0000 7fb0ceffd640 1 -- 172.21.15.114:0/3052996059 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0d01ba500 con 0x561cb41299e0 2024-09-13T18:30:03.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.509+0000 7fb0ceffd640 1 --2- 172.21.15.114:0/3052996059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cb41299e0 0x561cb4138a30 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x561cb41310d0 tx=0x7fb0c4006dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d31a4d777b41f412 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:03.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.512+0000 7fb0aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:30:03.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.512+0000 7fb0aeffd640 1 -- 172.21.15.114:0/3052996059 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c4013910 con 0x561cb41299e0 2024-09-13T18:30:03.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.512+0000 7fb0aeffd640 1 -- 172.21.15.114:0/3052996059 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fb0c4000e40 con 0x561cb41299e0 2024-09-13T18:30:03.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.512+0000 7fb0d8e56740 1 -- 172.21.15.114:0/3052996059 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb0c4013480 con 0x561cb41299e0 2024-09-13T18:30:03.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.513+0000 7fb0aeffd640 1 -- 172.21.15.114:0/3052996059 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0c4006bc0 con 0x561cb41299e0 2024-09-13T18:30:03.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.515+0000 7fb0d8e56740 10 client.27227 fetch_fsmap learned FSMap version 34 2024-09-13T18:30:03.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.515+0000 7fb0d8e56740 1 -- 172.21.15.114:0/3052996059 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x561cb4053990 con 0x561cb41299e0 2024-09-13T18:30:03.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.515+0000 7fb0aeffd640 1 -- 172.21.15.114:0/3052996059 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb0c4013480 con 0x561cb41299e0 2024-09-13T18:30:03.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.515+0000 7fb0aeffd640 1 -- 172.21.15.114:0/3052996059 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb0c40303a0 con 0x561cb41299e0 2024-09-13T18:30:03.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.515+0000 7fb0d8e56740 10 client.27227 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:30:03.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.515+0000 7fb0d8e56740 10 client.27227 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:30:03.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.515+0000 7fb0d8e56740 20 client.27227 populate_metadata read hostname 'smithi114' 2024-09-13T18:30:03.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.515+0000 7fb0d8e56740 1 --2- 172.21.15.114:0/3052996059 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x561cb419c5b0 0x561cb419e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:03.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.515+0000 7fb0d8e56740 4 client.27227 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-13T18:30:03.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.515+0000 7fb0d8e56740 1 -- 172.21.15.114:0/3052996059 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x561cb3de5eb0 con 0x561cb419c5b0 2024-09-13T18:30:03.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.516+0000 7fb0cf7fe640 1 --2- 172.21.15.114:0/3052996059 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x561cb419c5b0 0x561cb419e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:03.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.518+0000 7fb0aeffd640 10 client.27227 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:30:03.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.518+0000 7fb0cf7fe640 1 --2- 172.21.15.114:0/3052996059 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x561cb419c5b0 0x561cb419e9d0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7caf4cff3f14c87f server_cookie=8c0c6601d3caddc7 in_seq=0 out_seq=0 2024-09-13T18:30:03.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.523+0000 7fb0aeffd640 1 -- 172.21.15.114:0/3052996059 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x561cb3de5eb0 con 0x561cb419c5b0 2024-09-13T18:30:03.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.523+0000 7fb0aeffd640 10 client.27227 handle_command_reply: tid=0 2024-09-13T18:30:03.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.523+0000 7fb0d8e56740 1 client.27227 shutdown 2024-09-13T18:30:03.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.525+0000 7fb0d8e56740 1 -- 172.21.15.114:0/3052996059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cb41299e0 msgr2=0x561cb4138a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:03.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.525+0000 7fb0d8e56740 1 --2- 172.21.15.114:0/3052996059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cb41299e0 0x561cb4138a30 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x561cb41310d0 tx=0x7fb0c4006dd0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.525+0000 7fb0d8e56740 1 -- 172.21.15.114:0/3052996059 shutdown_connections 2024-09-13T18:30:03.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.525+0000 7fb0d8e56740 1 --2- 172.21.15.114:0/3052996059 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x561cb419c5b0 0x561cb419e9d0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.525+0000 7fb0d8e56740 1 --2- 172.21.15.114:0/3052996059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561cb4134ba0 0x561cb4138f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.525+0000 7fb0d8e56740 1 --2- 172.21.15.114:0/3052996059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cb41299e0 0x561cb4138a30 secure :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0x561cb41310d0 tx=0x7fb0c4006dd0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.525+0000 7fb0d8e56740 1 --2- 172.21.15.114:0/3052996059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cb4126a60 0x561cb41384f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.525+0000 7fb0d8e56740 1 -- 172.21.15.114:0/3052996059 >> 172.21.15.114:0/3052996059 conn(0x561cb4125f70 msgr2=0x561cb412ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:03.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.525+0000 7fb0aeffd640 0 client.27227 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:30:03.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.528+0000 7fb0d8e56740 1 -- 172.21.15.114:0/3052996059 shutdown_connections 2024-09-13T18:30:03.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.528+0000 7fb0d8e56740 1 -- 172.21.15.114:0/3052996059 wait complete. 2024-09-13T18:30:03.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.529+0000 7fb0d8e56740 20 client.27227 trim_cache size 0 max 16384 2024-09-13T18:30:03.534 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:30:03.534 INFO:teuthology.orchestra.run.smithi114.stdout: "return_code": 0, 2024-09-13T18:30:03.534 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_tag": "c7f4dc4e-54e8-4665-b2be-4c51314156bf", 2024-09-13T18:30:03.534 INFO:teuthology.orchestra.run.smithi114.stdout: "mode": "asynchronous" 2024-09-13T18:30:03.534 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:30:03.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.531+0000 7fb0d4df8640 1 -- 172.21.15.114:0/982035274 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fb0b80a1410 msgr2=0x7fb0b80a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:03.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.531+0000 7fb0d4df8640 1 --2- 172.21.15.114:0/982035274 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fb0b80a1410 0x7fb0b80a38d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0006fd0 tx=0x7fb0c00090e0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.531+0000 7fb0d4df8640 1 -- 172.21.15.114:0/982035274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0d0119400 msgr2=0x7fb0d0116f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:03.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.531+0000 7fb0d4df8640 1 --2- 172.21.15.114:0/982035274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0d0119400 0x7fb0d0116f80 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8009bf0 tx=0x7fb0c8031a00 comp rx=0 tx=0).stop 2024-09-13T18:30:03.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.533+0000 7fb0d4df8640 1 -- 172.21.15.114:0/982035274 shutdown_connections 2024-09-13T18:30:03.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.534+0000 7fb0d4df8640 1 --2- 172.21.15.114:0/982035274 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fb0b80a1410 0x7fb0b80a38d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.534+0000 7fb0d4df8640 1 --2- 172.21.15.114:0/982035274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0d0119400 0x7fb0d0116f80 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.534+0000 7fb0d4df8640 1 --2- 172.21.15.114:0/982035274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0d0107890 0x7fb0d0112b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.534+0000 7fb0d4df8640 1 --2- 172.21.15.114:0/982035274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0d01074b0 0x7fb0d0112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.534+0000 7fb0d4df8640 1 -- 172.21.15.114:0/982035274 >> 172.21.15.114:0/982035274 conn(0x7fb0d006aec0 msgr2=0x7fb0d006ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:03.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.535+0000 7fb0d4df8640 1 -- 172.21.15.114:0/982035274 shutdown_connections 2024-09-13T18:30:03.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.536+0000 7fb0d4df8640 1 -- 172.21.15.114:0/982035274 wait complete. 2024-09-13T18:30:03.571 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T18:30:03.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.854+0000 7fed7c609640 1 -- 172.21.15.114:0/2737385524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed74119400 msgr2=0x7fed7411b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:03.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.854+0000 7fed7c609640 1 --2- 172.21.15.114:0/2737385524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed74119400 0x7fed7411b7f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fed70009900 tx=0x7fed7002a440 comp rx=0 tx=0).stop 2024-09-13T18:30:03.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.855+0000 7fed7c609640 1 -- 172.21.15.114:0/2737385524 shutdown_connections 2024-09-13T18:30:03.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.855+0000 7fed7c609640 1 --2- 172.21.15.114:0/2737385524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed74119400 0x7fed7411b7f0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.855+0000 7fed7c609640 1 --2- 172.21.15.114:0/2737385524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed74107920 0x7fed74107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.855+0000 7fed7c609640 1 --2- 172.21.15.114:0/2737385524 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed74107540 0x7fed7410e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.855+0000 7fed7c609640 1 -- 172.21.15.114:0/2737385524 >> 172.21.15.114:0/2737385524 conn(0x7fed7406afe0 msgr2=0x7fed7406b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:03.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.860+0000 7fed7c609640 1 -- 172.21.15.114:0/2737385524 shutdown_connections 2024-09-13T18:30:03.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.866+0000 7fed7c609640 1 -- 172.21.15.114:0/2737385524 wait complete. 2024-09-13T18:30:03.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.866+0000 7fed7c609640 1 Processor -- start 2024-09-13T18:30:03.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.868+0000 7fed7c609640 1 -- start start 2024-09-13T18:30:03.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.868+0000 7fed7c609640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed74107540 0x7fed741a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:03.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.868+0000 7fed7c609640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed74107920 0x7fed741a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:03.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.868+0000 7fed7c609640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed74119400 0x7fed741a99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:03.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.868+0000 7fed7c609640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed741a9fa0 con 0x7fed74107540 2024-09-13T18:30:03.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.868+0000 7fed7c609640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed741aa0e0 con 0x7fed74119400 2024-09-13T18:30:03.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.868+0000 7fed7c609640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed741aa2a0 con 0x7fed74107920 2024-09-13T18:30:03.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.870+0000 7fed79ba4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed74107920 0x7fed741a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:03.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.870+0000 7fed79ba4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed74107920 0x7fed741a55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:53080/0 (socket says 172.21.15.114:53080) 2024-09-13T18:30:03.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.870+0000 7fed79ba4640 1 -- 172.21.15.114:0/777463579 learned_addr learned my addr 172.21.15.114:0/777463579 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:03.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.871+0000 7fed79ba4640 1 -- 172.21.15.114:0/777463579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed74119400 msgr2=0x7fed741a99a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:30:03.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.871+0000 7fed79ba4640 1 --2- 172.21.15.114:0/777463579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed74119400 0x7fed741a99a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.871+0000 7fed79ba4640 1 -- 172.21.15.114:0/777463579 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed74107540 msgr2=0x7fed741a5070 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:30:03.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.871+0000 7fed79ba4640 1 --2- 172.21.15.114:0/777463579 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed74107540 0x7fed741a5070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.871+0000 7fed79ba4640 1 -- 172.21.15.114:0/777463579 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed70009560 con 0x7fed74107920 2024-09-13T18:30:03.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.872+0000 7fed79ba4640 1 --2- 172.21.15.114:0/777463579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed74107920 0x7fed741a55b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fed6c00ba80 tx=0x7fed6c00bf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:03.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.872+0000 7fed6b7fe640 1 -- 172.21.15.114:0/777463579 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6c00cc00 con 0x7fed74107920 2024-09-13T18:30:03.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.872+0000 7fed6b7fe640 1 -- 172.21.15.114:0/777463579 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fed6c00cd60 con 0x7fed74107920 2024-09-13T18:30:03.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.872+0000 7fed6b7fe640 1 -- 172.21.15.114:0/777463579 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6c007750 con 0x7fed74107920 2024-09-13T18:30:03.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.873+0000 7fed7c609640 1 -- 172.21.15.114:0/777463579 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed741aa580 con 0x7fed74107920 2024-09-13T18:30:03.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.873+0000 7fed7c609640 1 -- 172.21.15.114:0/777463579 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed7410edc0 con 0x7fed74107920 2024-09-13T18:30:03.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.876+0000 7fed7f408740 1 Processor -- start 2024-09-13T18:30:03.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.876+0000 7fed7f408740 1 -- start start 2024-09-13T18:30:03.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.877+0000 7fed7f408740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604c103ce10 0x5604c103d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:03.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.877+0000 7fed7f408740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604c103d730 0x5604c1047a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:03.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.877+0000 7fed7f408740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604c1047fd0 0x5604c104a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:03.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.877+0000 7fed7f408740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604c0fa7f20 con 0x5604c103ce10 2024-09-13T18:30:03.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.877+0000 7fed7f408740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604c0fa56b0 con 0x5604c103d730 2024-09-13T18:30:03.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.877+0000 7fed7f408740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604c0fa0fa0 con 0x5604c1047fd0 2024-09-13T18:30:03.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.877+0000 7fed79ba4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604c1047fd0 0x5604c104a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:03.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.877+0000 7fed79ba4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604c1047fd0 0x5604c104a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:53086/0 (socket says 172.21.15.114:53086) 2024-09-13T18:30:03.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.877+0000 7fed79ba4640 1 -- 172.21.15.114:0/784198810 learned_addr learned my addr 172.21.15.114:0/784198810 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:03.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.878+0000 7fed7aba6640 1 --2- 172.21.15.114:0/784198810 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604c103ce10 0x5604c103d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:03.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.879+0000 7fed7a3a5640 1 --2- 172.21.15.114:0/784198810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604c103d730 0x5604c1047a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:03.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.879+0000 7fed6b7fe640 1 -- 172.21.15.114:0/777463579 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fed6c0078b0 con 0x7fed74107920 2024-09-13T18:30:03.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.883+0000 7fed7a3a5640 1 -- 172.21.15.114:0/784198810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604c1047fd0 msgr2=0x5604c104a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:03.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.883+0000 7fed7a3a5640 1 --2- 172.21.15.114:0/784198810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604c1047fd0 0x5604c104a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.883+0000 7fed7a3a5640 1 -- 172.21.15.114:0/784198810 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604c103ce10 msgr2=0x5604c103d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:03.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.883+0000 7fed7a3a5640 1 --2- 172.21.15.114:0/784198810 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604c103ce10 0x5604c103d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.883+0000 7fed7a3a5640 1 -- 172.21.15.114:0/784198810 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604c0f5b840 con 0x5604c103d730 2024-09-13T18:30:03.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.884+0000 7fed7a3a5640 1 --2- 172.21.15.114:0/784198810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604c103d730 0x5604c1047a90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fed6400b9d0 tx=0x7fed6400be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa778a9172793e9b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:03.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.885+0000 7fed79ba4640 1 --2- 172.21.15.114:0/784198810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604c1047fd0 0x5604c104a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:30:03.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.885+0000 7fed6b7fe640 1 --2- 172.21.15.114:0/777463579 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fed600a1340 0x7fed600a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:03.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.885+0000 7fed6b7fe640 1 -- 172.21.15.114:0/777463579 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fed6c0d2150 con 0x7fed74107920 2024-09-13T18:30:03.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.886+0000 7fed7aba6640 1 --2- 172.21.15.114:0/784198810 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604c103ce10 0x5604c103d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:30:03.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.887+0000 7fed68ff9640 1 -- 172.21.15.114:0/784198810 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6400dbe0 con 0x5604c103d730 2024-09-13T18:30:03.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.887+0000 7fed68ff9640 1 -- 172.21.15.114:0/784198810 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fed6400dd40 con 0x5604c103d730 2024-09-13T18:30:03.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.887+0000 7fed68ff9640 1 -- 172.21.15.114:0/784198810 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed64010460 con 0x5604c103d730 2024-09-13T18:30:03.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.888+0000 7fed7aba6640 1 --2- 172.21.15.114:0/777463579 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fed600a1340 0x7fed600a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:03.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.890+0000 7fed7f408740 1 -- 172.21.15.114:0/784198810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604c103d730 msgr2=0x5604c1047a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:03.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.890+0000 7fed7f408740 1 --2- 172.21.15.114:0/784198810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604c103d730 0x5604c1047a90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fed6400b9d0 tx=0x7fed6400be90 comp rx=0 tx=0).stop 2024-09-13T18:30:03.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.891+0000 7fed7f408740 1 -- 172.21.15.114:0/784198810 shutdown_connections 2024-09-13T18:30:03.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.891+0000 7fed7f408740 1 --2- 172.21.15.114:0/784198810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604c1047fd0 0x5604c104a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.891+0000 7fed7f408740 1 --2- 172.21.15.114:0/784198810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604c103d730 0x5604c1047a90 secure :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fed6400b9d0 tx=0x7fed6400be90 comp rx=0 tx=0).stop 2024-09-13T18:30:03.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.891+0000 7fed7f408740 1 --2- 172.21.15.114:0/784198810 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604c103ce10 0x5604c103d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.891+0000 7fed7f408740 1 -- 172.21.15.114:0/784198810 >> 172.21.15.114:0/784198810 conn(0x5604c1039380 msgr2=0x5604c1039760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:03.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.892+0000 7fed7aba6640 1 --2- 172.21.15.114:0/777463579 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fed600a1340 0x7fed600a3800 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fed7002a9b0 tx=0x7fed7002a940 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:03.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.895+0000 7fed7f408740 1 -- 172.21.15.114:0/784198810 shutdown_connections 2024-09-13T18:30:03.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.895+0000 7fed7f408740 1 -- 172.21.15.114:0/784198810 wait complete. 2024-09-13T18:30:03.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.895+0000 7fed7f408740 1 -- start start 2024-09-13T18:30:03.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.895+0000 7fed7f408740 1 Processor -- start 2024-09-13T18:30:03.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.896+0000 7fed7f408740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604c103ce10 0x5604c1044f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:03.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.896+0000 7fed7f408740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604c1047fd0 0x5604c1045460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:03.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.896+0000 7fed7f408740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604c10459a0 0x5604c1053e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:03.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.896+0000 7fed7f408740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604c0f422a0 con 0x5604c103ce10 2024-09-13T18:30:03.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.896+0000 7fed7f408740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604c0c79430 con 0x5604c10459a0 2024-09-13T18:30:03.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.896+0000 7fed7f408740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604c0f24680 con 0x5604c1047fd0 2024-09-13T18:30:03.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.896+0000 7fed7f408740 10 client.0 _ll_register_callbacks cb 0x5604c0f9a610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:30:03.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.896+0000 7fed79ba4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604c1047fd0 0x5604c1045460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:03.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.896+0000 7fed79ba4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604c1047fd0 0x5604c1045460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:53094/0 (socket says 172.21.15.114:53094) 2024-09-13T18:30:03.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.896+0000 7fed79ba4640 1 -- 172.21.15.114:0/2493082096 learned_addr learned my addr 172.21.15.114:0/2493082096 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:03.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.897+0000 7fed79ba4640 1 -- 172.21.15.114:0/2493082096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604c10459a0 msgr2=0x5604c1053e50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:30:03.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.897+0000 7fed79ba4640 1 --2- 172.21.15.114:0/2493082096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604c10459a0 0x5604c1053e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.897+0000 7fed79ba4640 1 -- 172.21.15.114:0/2493082096 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604c103ce10 msgr2=0x5604c1044f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:30:03.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.897+0000 7fed79ba4640 1 --2- 172.21.15.114:0/2493082096 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604c103ce10 0x5604c1044f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.897+0000 7fed79ba4640 1 -- 172.21.15.114:0/2493082096 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed6400b600 con 0x5604c1047fd0 2024-09-13T18:30:03.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.905+0000 7fed79ba4640 1 --2- 172.21.15.114:0/2493082096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604c1047fd0 0x5604c1045460 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fed6c09ef00 tx=0x7fed6c09eb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc52dcc056ee261a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:03.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.905+0000 7fed5dffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:30:03.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.905+0000 7fed5dffb640 1 -- 172.21.15.114:0/2493082096 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6c09e660 con 0x5604c1047fd0 2024-09-13T18:30:03.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.906+0000 7fed7f408740 1 -- 172.21.15.114:0/2493082096 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5604c0c944e0 con 0x5604c1047fd0 2024-09-13T18:30:03.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.906+0000 7fed5dffb640 1 -- 172.21.15.114:0/2493082096 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fed6c09f670 con 0x5604c1047fd0 2024-09-13T18:30:03.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.911+0000 7fed5dffb640 1 -- 172.21.15.114:0/2493082096 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed6c0cf930 con 0x5604c1047fd0 2024-09-13T18:30:03.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.911+0000 7fed7f408740 10 client.27214 fetch_fsmap learned FSMap version 34 2024-09-13T18:30:03.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.911+0000 7fed7f408740 1 -- 172.21.15.114:0/2493082096 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5604c0f9f620 con 0x5604c1047fd0 2024-09-13T18:30:03.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.913+0000 7fed5dffb640 1 -- 172.21.15.114:0/2493082096 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5604c0c944e0 con 0x5604c1047fd0 2024-09-13T18:30:03.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.913+0000 7fed5dffb640 1 -- 172.21.15.114:0/2493082096 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fed6c0a7080 con 0x5604c1047fd0 2024-09-13T18:30:03.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.913+0000 7fed7f408740 10 client.27214 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:30:03.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.913+0000 7fed7f408740 10 client.27214 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:30:03.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.913+0000 7fed7f408740 20 client.27214 populate_metadata read hostname 'smithi114' 2024-09-13T18:30:03.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.913+0000 7fed7f408740 1 --2- 172.21.15.114:0/2493082096 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5604c1058550 0x5604c105a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:03.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.913+0000 7fed7f408740 4 client.27214 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:30:03.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.913+0000 7fed7f408740 1 -- 172.21.15.114:0/2493082096 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5604c0ca57e0 con 0x5604c1058550 2024-09-13T18:30:03.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.915+0000 7fed7a3a5640 1 --2- 172.21.15.114:0/2493082096 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5604c1058550 0x5604c105a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:03.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.916+0000 7fed7a3a5640 1 --2- 172.21.15.114:0/2493082096 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5604c1058550 0x5604c105a970 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=721dafc12d864e16 server_cookie=b2cef00114a4a7e4 in_seq=0 out_seq=0 2024-09-13T18:30:03.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.916+0000 7fed5dffb640 10 client.27214 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:30:03.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.920+0000 7fed5dffb640 1 -- 172.21.15.114:0/2493082096 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5604c0ca57e0 con 0x5604c1058550 2024-09-13T18:30:03.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.920+0000 7fed5dffb640 10 client.27214 handle_command_reply: tid=0 2024-09-13T18:30:03.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.921+0000 7fed7f408740 1 client.27214 shutdown 2024-09-13T18:30:03.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.930+0000 7fed7f408740 1 -- 172.21.15.114:0/2493082096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604c1047fd0 msgr2=0x5604c1045460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:03.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.930+0000 7fed7f408740 1 --2- 172.21.15.114:0/2493082096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604c1047fd0 0x5604c1045460 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fed6c09ef00 tx=0x7fed6c09eb70 comp rx=0 tx=0).stop 2024-09-13T18:30:03.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.934+0000 7fed7f408740 1 -- 172.21.15.114:0/2493082096 shutdown_connections 2024-09-13T18:30:03.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.934+0000 7fed7f408740 1 --2- 172.21.15.114:0/2493082096 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5604c1058550 0x5604c105a970 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.934+0000 7fed7f408740 1 --2- 172.21.15.114:0/2493082096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604c10459a0 0x5604c1053e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.934+0000 7fed7f408740 1 --2- 172.21.15.114:0/2493082096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604c1047fd0 0x5604c1045460 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.934+0000 7fed7f408740 1 --2- 172.21.15.114:0/2493082096 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604c103ce10 0x5604c1044f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.934+0000 7fed7f408740 1 -- 172.21.15.114:0/2493082096 >> 172.21.15.114:0/2493082096 conn(0x5604c1039380 msgr2=0x5604c103f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:03.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.938+0000 7fed5dffb640 0 client.27214 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:30:03.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.944+0000 7fed7f408740 1 -- 172.21.15.114:0/2493082096 shutdown_connections 2024-09-13T18:30:03.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.944+0000 7fed7f408740 1 -- 172.21.15.114:0/2493082096 wait complete. 2024-09-13T18:30:03.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.950+0000 7fed7f408740 20 client.27214 trim_cache size 0 max 16384 2024-09-13T18:30:03.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.977+0000 7fed7f408740 1 Processor -- start 2024-09-13T18:30:03.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.978+0000 7fed7f408740 1 -- start start 2024-09-13T18:30:03.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.978+0000 7fed7f408740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604c103ce10 0x5604c103f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:03.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.978+0000 7fed7f408740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604c10459a0 0x5604c1053850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:03.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.978+0000 7fed7f408740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604c1047fd0 0x5604c1053d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:03.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.978+0000 7fed7f408740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604c1047ca0 con 0x5604c1047fd0 2024-09-13T18:30:03.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.978+0000 7fed7aba6640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604c1047fd0 0x5604c1053d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:03.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.978+0000 7fed7aba6640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604c1047fd0 0x5604c1053d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:56330/0 (socket says 172.21.15.114:56330) 2024-09-13T18:30:03.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.978+0000 7fed7aba6640 1 -- 172.21.15.114:0/2303538980 learned_addr learned my addr 172.21.15.114:0/2303538980 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:03.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.978+0000 7fed7f408740 1 -- 172.21.15.114:0/2303538980 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604c1033710 con 0x5604c103ce10 2024-09-13T18:30:03.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.978+0000 7fed7f408740 1 -- 172.21.15.114:0/2303538980 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604c103c610 con 0x5604c10459a0 2024-09-13T18:30:03.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.979+0000 7fed7aba6640 1 -- 172.21.15.114:0/2303538980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604c10459a0 msgr2=0x5604c1053850 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:30:03.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.979+0000 7fed7aba6640 1 --2- 172.21.15.114:0/2303538980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604c10459a0 0x5604c1053850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.979+0000 7fed7aba6640 1 -- 172.21.15.114:0/2303538980 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604c103ce10 msgr2=0x5604c103f8d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:30:03.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.979+0000 7fed7aba6640 1 --2- 172.21.15.114:0/2303538980 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604c103ce10 0x5604c103f8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.979+0000 7fed7aba6640 1 -- 172.21.15.114:0/2303538980 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604c0f41790 con 0x5604c1047fd0 2024-09-13T18:30:03.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.979+0000 7fed7aba6640 1 --2- 172.21.15.114:0/2303538980 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604c1047fd0 0x5604c1053d90 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fed70033000 tx=0x7fed7003d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1674580706ad5fb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:03.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.982+0000 7fed697fa640 1 -- 172.21.15.114:0/2303538980 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed700048c0 con 0x5604c1047fd0 2024-09-13T18:30:03.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.982+0000 7fed697fa640 1 -- 172.21.15.114:0/2303538980 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fed70005600 con 0x5604c1047fd0 2024-09-13T18:30:03.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.983+0000 7fed7f408740 1 -- 172.21.15.114:0/2303538980 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604c1047fd0 msgr2=0x5604c1053d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:03.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.983+0000 7fed7f408740 1 --2- 172.21.15.114:0/2303538980 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604c1047fd0 0x5604c1053d90 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fed70033000 tx=0x7fed7003d000 comp rx=0 tx=0).stop 2024-09-13T18:30:03.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.987+0000 7fed7f408740 1 -- 172.21.15.114:0/2303538980 shutdown_connections 2024-09-13T18:30:03.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.987+0000 7fed7f408740 1 --2- 172.21.15.114:0/2303538980 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604c1047fd0 0x5604c1053d90 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.987+0000 7fed7f408740 1 --2- 172.21.15.114:0/2303538980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604c10459a0 0x5604c1053850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.987+0000 7fed7f408740 1 --2- 172.21.15.114:0/2303538980 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604c103ce10 0x5604c103f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.987+0000 7fed7f408740 1 -- 172.21.15.114:0/2303538980 >> 172.21.15.114:0/2303538980 conn(0x5604c1039380 msgr2=0x5604c10401d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:03.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.991+0000 7fed7f408740 1 -- 172.21.15.114:0/2303538980 shutdown_connections 2024-09-13T18:30:03.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.992+0000 7fed7f408740 1 -- 172.21.15.114:0/2303538980 wait complete. 2024-09-13T18:30:03.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.992+0000 7fed7f408740 1 -- start start 2024-09-13T18:30:03.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.992+0000 7fed7f408740 1 Processor -- start 2024-09-13T18:30:03.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.993+0000 7fed7f408740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604c103ce10 0x5604c104cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:03.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.993+0000 7fed7f408740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604c10459a0 0x5604c104d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:03.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.993+0000 7fed7f408740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604c1047fd0 0x5604c10ada50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:03.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.993+0000 7fed7f408740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604c104dbe0 con 0x5604c10459a0 2024-09-13T18:30:03.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.993+0000 7fed7f408740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604c10adf90 con 0x5604c103ce10 2024-09-13T18:30:03.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.993+0000 7fed7f408740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604c10ae0d0 con 0x5604c1047fd0 2024-09-13T18:30:03.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.993+0000 7fed7aba6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604c1047fd0 0x5604c10ada50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:03.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.993+0000 7fed7aba6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604c1047fd0 0x5604c10ada50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:53120/0 (socket says 172.21.15.114:53120) 2024-09-13T18:30:03.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.993+0000 7fed7aba6640 1 -- 172.21.15.114:0/4090746405 learned_addr learned my addr 172.21.15.114:0/4090746405 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:03.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.993+0000 7fed7f408740 10 client.0 _ll_register_callbacks cb 0x7fed6c09ed40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:30:03.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.994+0000 7fed7a3a5640 1 --2- 172.21.15.114:0/4090746405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604c103ce10 0x5604c104cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:03.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.994+0000 7fed7aba6640 1 -- 172.21.15.114:0/4090746405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604c103ce10 msgr2=0x5604c104cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:03.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.994+0000 7fed7aba6640 1 --2- 172.21.15.114:0/4090746405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604c103ce10 0x5604c104cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.994+0000 7fed7aba6640 1 -- 172.21.15.114:0/4090746405 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604c10459a0 msgr2=0x5604c104d2c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:30:03.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.994+0000 7fed7aba6640 1 --2- 172.21.15.114:0/4090746405 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604c10459a0 0x5604c104d2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:03.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.994+0000 7fed7aba6640 1 -- 172.21.15.114:0/4090746405 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed7411c2e0 con 0x5604c1047fd0 2024-09-13T18:30:03.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.994+0000 7fed7aba6640 1 --2- 172.21.15.114:0/4090746405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604c1047fd0 0x5604c10ada50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fed700042c0 tx=0x7fed700045c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b17b73f6fa09ada4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:03.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.994+0000 7fed5dffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:30:03.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.995+0000 7fed7a3a5640 1 --2- 172.21.15.114:0/4090746405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604c103ce10 0x5604c104cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:30:03.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.995+0000 7fed5dffb640 1 -- 172.21.15.114:0/4090746405 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed70003ea0 con 0x5604c1047fd0 2024-09-13T18:30:04.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.995+0000 7fed7f408740 1 -- 172.21.15.114:0/4090746405 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fed6400dd40 con 0x5604c1047fd0 2024-09-13T18:30:04.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.995+0000 7fed5dffb640 1 -- 172.21.15.114:0/4090746405 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fed700046c0 con 0x5604c1047fd0 2024-09-13T18:30:04.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.996+0000 7fed5dffb640 1 -- 172.21.15.114:0/4090746405 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed70004880 con 0x5604c1047fd0 2024-09-13T18:30:04.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.996+0000 7fed7f408740 10 client.27217 fetch_fsmap learned FSMap version 34 2024-09-13T18:30:04.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.996+0000 7fed7f408740 1 -- 172.21.15.114:0/4090746405 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5604c0f66ea0 con 0x5604c1047fd0 2024-09-13T18:30:04.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.997+0000 7fed5dffb640 1 -- 172.21.15.114:0/4090746405 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fed6400dd40 con 0x5604c1047fd0 2024-09-13T18:30:04.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.997+0000 7fed5dffb640 1 -- 172.21.15.114:0/4090746405 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fed70049c60 con 0x5604c1047fd0 2024-09-13T18:30:04.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.997+0000 7fed7f408740 10 client.27217 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:30:04.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.997+0000 7fed7f408740 10 client.27217 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:30:04.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.997+0000 7fed7f408740 20 client.27217 populate_metadata read hostname 'smithi114' 2024-09-13T18:30:04.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.997+0000 7fed7f408740 1 --2- 172.21.15.114:0/4090746405 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5604c10af680 0x5604c10b1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:04.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.997+0000 7fed7f408740 4 client.27217 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T18:30:04.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.997+0000 7fed7f408740 1 -- 172.21.15.114:0/4090746405 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5604c0cf8e30 con 0x5604c10af680 2024-09-13T18:30:04.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:03.999+0000 7fed7a3a5640 1 --2- 172.21.15.114:0/4090746405 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5604c10af680 0x5604c10b1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:04.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:04.008+0000 7fed7a3a5640 1 --2- 172.21.15.114:0/4090746405 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5604c10af680 0x5604c10b1aa0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b83bdb0fe2694ccb server_cookie=155b99918c10301e in_seq=0 out_seq=0 2024-09-13T18:30:04.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:04.008+0000 7fed5dffb640 10 client.27217 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:30:04.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:04.009+0000 7fed5dffb640 1 -- 172.21.15.114:0/4090746405 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5604c0cf8e30 con 0x5604c10af680 2024-09-13T18:30:04.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:04.009+0000 7fed5dffb640 10 client.27217 handle_command_reply: tid=0 2024-09-13T18:30:04.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:04.009+0000 7fed7f408740 1 client.27217 shutdown 2024-09-13T18:30:04.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:04.012+0000 7fed7f408740 1 -- 172.21.15.114:0/4090746405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604c1047fd0 msgr2=0x5604c10ada50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:04.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:04.012+0000 7fed7f408740 1 --2- 172.21.15.114:0/4090746405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604c1047fd0 0x5604c10ada50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fed700042c0 tx=0x7fed700045c0 comp rx=0 tx=0).stop 2024-09-13T18:30:04.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:04.016+0000 7fed7f408740 1 -- 172.21.15.114:0/4090746405 shutdown_connections 2024-09-13T18:30:04.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:04.016+0000 7fed7f408740 1 --2- 172.21.15.114:0/4090746405 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5604c10af680 0x5604c10b1aa0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:04.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:04.016+0000 7fed7f408740 1 --2- 172.21.15.114:0/4090746405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604c1047fd0 0x5604c10ada50 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:04.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:04.016+0000 7fed7f408740 1 --2- 172.21.15.114:0/4090746405 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604c10459a0 0x5604c104d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:04.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:04.016+0000 7fed7f408740 1 --2- 172.21.15.114:0/4090746405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604c103ce10 0x5604c104cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:04.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:04.016+0000 7fed7f408740 1 -- 172.21.15.114:0/4090746405 >> 172.21.15.114:0/4090746405 conn(0x5604c1039380 msgr2=0x5604c10401d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:04.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:04.018+0000 7fed5dffb640 0 client.27217 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:30:04.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:04.020+0000 7fed7f408740 1 -- 172.21.15.114:0/4090746405 shutdown_connections 2024-09-13T18:30:04.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:04.021+0000 7fed7f408740 1 -- 172.21.15.114:0/4090746405 wait complete. 2024-09-13T18:30:04.032 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:30:04.032 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-13T18:30:04.032 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": { 2024-09-13T18:30:04.032 INFO:teuthology.orchestra.run.smithi114.stdout: "c7f4dc4e-54e8-4665-b2be-4c51314156bf": { 2024-09-13T18:30:04.032 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-13T18:30:04.032 INFO:teuthology.orchestra.run.smithi114.stdout: "tag": "c7f4dc4e-54e8-4665-b2be-4c51314156bf", 2024-09-13T18:30:04.033 INFO:teuthology.orchestra.run.smithi114.stdout: "options": "recursive,force" 2024-09-13T18:30:04.033 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:30:04.033 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:30:04.033 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:30:04.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:04.028+0000 7fed7f408740 20 client.27217 trim_cache size 0 max 16384 2024-09-13T18:30:04.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:04.034+0000 7fed7c609640 1 -- 172.21.15.114:0/777463579 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fed600a1340 msgr2=0x7fed600a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:04.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:04.034+0000 7fed7c609640 1 --2- 172.21.15.114:0/777463579 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fed600a1340 0x7fed600a3800 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fed7002a9b0 tx=0x7fed7002a940 comp rx=0 tx=0).stop 2024-09-13T18:30:04.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:04.034+0000 7fed7c609640 1 -- 172.21.15.114:0/777463579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed74107920 msgr2=0x7fed741a55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:04.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:04.034+0000 7fed7c609640 1 --2- 172.21.15.114:0/777463579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed74107920 0x7fed741a55b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fed6c00ba80 tx=0x7fed6c00bf40 comp rx=0 tx=0).stop 2024-09-13T18:30:04.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:04.039+0000 7fed7c609640 1 -- 172.21.15.114:0/777463579 shutdown_connections 2024-09-13T18:30:04.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:04.039+0000 7fed7c609640 1 --2- 172.21.15.114:0/777463579 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fed600a1340 0x7fed600a3800 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:04.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:04.039+0000 7fed7c609640 1 --2- 172.21.15.114:0/777463579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed74119400 0x7fed741a99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:04.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:04.039+0000 7fed7c609640 1 --2- 172.21.15.114:0/777463579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed74107920 0x7fed741a55b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:04.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:04.039+0000 7fed7c609640 1 --2- 172.21.15.114:0/777463579 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed74107540 0x7fed741a5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:04.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:04.039+0000 7fed7c609640 1 -- 172.21.15.114:0/777463579 >> 172.21.15.114:0/777463579 conn(0x7fed7406afe0 msgr2=0x7fed74119c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:04.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:04.043+0000 7fed7c609640 1 -- 172.21.15.114:0/777463579 shutdown_connections 2024-09-13T18:30:04.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:04.043+0000 7fed7c609640 1 -- 172.21.15.114:0/777463579 wait complete. 2024-09-13T18:30:04.066 INFO:tasks.cephfs.filesystem:scrub status for tag:c7f4dc4e-54e8-4665-b2be-4c51314156bf - {'path': '/', 'tag': 'c7f4dc4e-54e8-4665-b2be-4c51314156bf', 'options': 'recursive,force'} 2024-09-13T18:30:04.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:03 smithi134 ceph-mon[34283]: pgmap v232: 129 pgs: 129 active+clean; 22 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 29 MiB/s wr, 633 op/s 2024-09-13T18:30:04.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:03 smithi134 ceph-mon[34283]: scrub summary: active paths [/] 2024-09-13T18:30:04.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:03 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:30:04.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:03 smithi204 ceph-mon[34777]: pgmap v232: 129 pgs: 129 active+clean; 22 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 29 MiB/s wr, 633 op/s 2024-09-13T18:30:04.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:03 smithi204 ceph-mon[34777]: scrub summary: active paths [/] 2024-09-13T18:30:04.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:03 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:30:04.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:03 smithi114 ceph-mon[30897]: pgmap v232: 129 pgs: 129 active+clean; 22 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 29 MiB/s wr, 633 op/s 2024-09-13T18:30:04.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:03 smithi114 ceph-mon[30897]: scrub summary: active paths [/] 2024-09-13T18:30:04.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:03 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:30:05.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:05 smithi134 ceph-mon[34283]: scrub queued for path: / 2024-09-13T18:30:05.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:05 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:30:05.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:05 smithi134 ceph-mon[34283]: scrub summary: active paths [/] 2024-09-13T18:30:05.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:05 smithi134 ceph-mon[34283]: scrub summary: active paths [/] 2024-09-13T18:30:05.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:05 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:30:05.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:05 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:30:05.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:05 smithi204 ceph-mon[34777]: scrub queued for path: / 2024-09-13T18:30:05.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:05 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:30:05.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:05 smithi204 ceph-mon[34777]: scrub summary: active paths [/] 2024-09-13T18:30:05.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:05 smithi204 ceph-mon[34777]: scrub summary: active paths [/] 2024-09-13T18:30:05.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:05 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:30:05.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:05 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:30:05.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:05 smithi114 ceph-mon[30897]: scrub queued for path: / 2024-09-13T18:30:05.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:05 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:30:05.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:05 smithi114 ceph-mon[30897]: scrub summary: active paths [/] 2024-09-13T18:30:05.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:05 smithi114 ceph-mon[30897]: scrub summary: active paths [/] 2024-09-13T18:30:05.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:05 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:30:05.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:05 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:30:06.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:06 smithi134 ceph-mon[34283]: pgmap v233: 129 pgs: 129 active+clean; 22 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 33 MiB/s wr, 730 op/s 2024-09-13T18:30:06.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:06 smithi204 ceph-mon[34777]: pgmap v233: 129 pgs: 129 active+clean; 22 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 33 MiB/s wr, 730 op/s 2024-09-13T18:30:06.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:06 smithi114 ceph-mon[30897]: pgmap v233: 129 pgs: 129 active+clean; 22 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 33 MiB/s wr, 730 op/s 2024-09-13T18:30:08.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:08 smithi134 ceph-mon[34283]: pgmap v234: 129 pgs: 129 active+clean; 22 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 37 MiB/s wr, 829 op/s 2024-09-13T18:30:08.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:08 smithi134 ceph-mon[34283]: osdmap e97: 12 total, 12 up, 12 in 2024-09-13T18:30:08.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:08 smithi204 ceph-mon[34777]: pgmap v234: 129 pgs: 129 active+clean; 22 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 37 MiB/s wr, 829 op/s 2024-09-13T18:30:08.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:08 smithi204 ceph-mon[34777]: osdmap e97: 12 total, 12 up, 12 in 2024-09-13T18:30:08.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:08 smithi114 ceph-mon[30897]: pgmap v234: 129 pgs: 129 active+clean; 22 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 37 MiB/s wr, 829 op/s 2024-09-13T18:30:08.711 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:08 smithi114 ceph-mon[30897]: osdmap e97: 12 total, 12 up, 12 in 2024-09-13T18:30:10.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:10 smithi134 ceph-mon[34283]: pgmap v236: 129 pgs: 129 active+clean; 22 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 33 MiB/s wr, 738 op/s 2024-09-13T18:30:10.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:10 smithi204 ceph-mon[34777]: pgmap v236: 129 pgs: 129 active+clean; 22 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 33 MiB/s wr, 738 op/s 2024-09-13T18:30:10.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:10 smithi114 ceph-mon[30897]: pgmap v236: 129 pgs: 129 active+clean; 22 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 33 MiB/s wr, 738 op/s 2024-09-13T18:30:10.937 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.932+0000 7ffba5328640 1 -- 172.21.15.114:0/1813480969 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffba0119400 msgr2=0x7ffba011b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:10.937 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.932+0000 7ffba5328640 1 --2- 172.21.15.114:0/1813480969 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffba0119400 0x7ffba011b7f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ffb88009ca0 tx=0x7ffb8802a8d0 comp rx=0 tx=0).stop 2024-09-13T18:30:10.941 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.938+0000 7ffba5328640 1 -- 172.21.15.114:0/1813480969 shutdown_connections 2024-09-13T18:30:10.942 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.938+0000 7ffba5328640 1 --2- 172.21.15.114:0/1813480969 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffba0119400 0x7ffba011b7f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:10.942 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.938+0000 7ffba5328640 1 --2- 172.21.15.114:0/1813480969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffba0107890 0x7ffba0107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:10.942 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.938+0000 7ffba5328640 1 --2- 172.21.15.114:0/1813480969 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffba01074b0 0x7ffba010e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:10.942 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.938+0000 7ffba5328640 1 -- 172.21.15.114:0/1813480969 >> 172.21.15.114:0/1813480969 conn(0x7ffba006adf0 msgr2=0x7ffba006b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.940+0000 7ffba5328640 1 -- 172.21.15.114:0/1813480969 shutdown_connections 2024-09-13T18:30:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.940+0000 7ffba5328640 1 -- 172.21.15.114:0/1813480969 wait complete. 2024-09-13T18:30:10.944 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.941+0000 7ffba5328640 1 Processor -- start 2024-09-13T18:30:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.941+0000 7ffba5328640 1 -- start start 2024-09-13T18:30:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.942+0000 7ffba5328640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffba01074b0 0x7ffba0118910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.942+0000 7ffba5328640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffba0107890 0x7ffba0118e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.942+0000 7ffba5328640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffba0119400 0x7ffba0112b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.942+0000 7ffba5328640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba01132f0 con 0x7ffba0107890 2024-09-13T18:30:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.942+0000 7ffba5328640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba0113460 con 0x7ffba01074b0 2024-09-13T18:30:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.942+0000 7ffba5328640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba0113620 con 0x7ffba0119400 2024-09-13T18:30:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.942+0000 7ffb9f7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffba0107890 0x7ffba0118e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.942+0000 7ffb9f7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffba0107890 0x7ffba0118e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:56342/0 (socket says 172.21.15.114:56342) 2024-09-13T18:30:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.942+0000 7ffb9f7fe640 1 -- 172.21.15.114:0/686664013 learned_addr learned my addr 172.21.15.114:0/686664013 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.943+0000 7ffb9ffff640 1 --2- 172.21.15.114:0/686664013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffba01074b0 0x7ffba0118910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.943+0000 7ffb9f7fe640 1 -- 172.21.15.114:0/686664013 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffba0119400 msgr2=0x7ffba0112b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.943+0000 7ffba4b27640 1 --2- 172.21.15.114:0/686664013 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffba0119400 0x7ffba0112b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:10.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.943+0000 7ffb9f7fe640 1 --2- 172.21.15.114:0/686664013 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffba0119400 0x7ffba0112b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:10.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.943+0000 7ffb9f7fe640 1 -- 172.21.15.114:0/686664013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffba01074b0 msgr2=0x7ffba0118910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:10.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.943+0000 7ffb9f7fe640 1 --2- 172.21.15.114:0/686664013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffba01074b0 0x7ffba0118910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:10.949 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.943+0000 7ffb9f7fe640 1 -- 172.21.15.114:0/686664013 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb88009900 con 0x7ffba0107890 2024-09-13T18:30:10.949 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.944+0000 7ffb9ffff640 1 --2- 172.21.15.114:0/686664013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffba01074b0 0x7ffba0118910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:30:10.949 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.945+0000 7ffb9f7fe640 1 --2- 172.21.15.114:0/686664013 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffba0107890 0x7ffba0118e50 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7ffb9000d950 tx=0x7ffb9000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:10.950 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.946+0000 7ffb9d7fa640 1 -- 172.21.15.114:0/686664013 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb90009c30 con 0x7ffba0107890 2024-09-13T18:30:10.950 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.946+0000 7ffba5328640 1 -- 172.21.15.114:0/686664013 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffba01ba000 con 0x7ffba0107890 2024-09-13T18:30:10.950 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.946+0000 7ffba5328640 1 -- 172.21.15.114:0/686664013 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffba01ba450 con 0x7ffba0107890 2024-09-13T18:30:10.950 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.946+0000 7ffb9d7fa640 1 -- 172.21.15.114:0/686664013 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffb90009d90 con 0x7ffba0107890 2024-09-13T18:30:10.950 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.946+0000 7ffb9d7fa640 1 -- 172.21.15.114:0/686664013 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9000f620 con 0x7ffba0107890 2024-09-13T18:30:10.952 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.949+0000 7ffba92ae740 1 Processor -- start 2024-09-13T18:30:10.953 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.949+0000 7ffba92ae740 1 -- start start 2024-09-13T18:30:10.957 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.950+0000 7ffba92ae740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ec70ebdf0 0x561ec70ea620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:10.957 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.950+0000 7ffba92ae740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ec70eab60 0x561ec70eafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:10.957 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.950+0000 7ffba92ae740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ec70f54b0 0x561ec70f78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:10.957 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.950+0000 7ffba92ae740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ec7055c30 con 0x561ec70ebdf0 2024-09-13T18:30:10.957 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.950+0000 7ffba92ae740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ec70533c0 con 0x561ec70f54b0 2024-09-13T18:30:10.957 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.950+0000 7ffba92ae740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ec704ecb0 con 0x561ec70eab60 2024-09-13T18:30:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.950+0000 7ffb9f7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ec70f54b0 0x561ec70f78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.950+0000 7ffb9f7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ec70f54b0 0x561ec70f78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:60820/0 (socket says 172.21.15.114:60820) 2024-09-13T18:30:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.950+0000 7ffb9f7fe640 1 -- 172.21.15.114:0/3423138419 learned_addr learned my addr 172.21.15.114:0/3423138419 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.950+0000 7ffb9ffff640 1 --2- 172.21.15.114:0/3423138419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ec70eab60 0x561ec70eafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.951+0000 7ffba4b27640 1 --2- 172.21.15.114:0/3423138419 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ec70ebdf0 0x561ec70ea620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.951+0000 7ffb9ffff640 1 -- 172.21.15.114:0/3423138419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ec70f54b0 msgr2=0x561ec70f78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.951+0000 7ffb9ffff640 1 --2- 172.21.15.114:0/3423138419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ec70f54b0 0x561ec70f78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.951+0000 7ffb9ffff640 1 -- 172.21.15.114:0/3423138419 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ec70ebdf0 msgr2=0x561ec70ea620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.952+0000 7ffb9d7fa640 1 -- 172.21.15.114:0/686664013 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7ffb900109f0 con 0x7ffba0107890 2024-09-13T18:30:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.954+0000 7ffb9ffff640 1 --2- 172.21.15.114:0/3423138419 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ec70ebdf0 0x561ec70ea620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.954+0000 7ffb9ffff640 1 -- 172.21.15.114:0/3423138419 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ec6ff5910 con 0x561ec70eab60 2024-09-13T18:30:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.954+0000 7ffb9ffff640 1 --2- 172.21.15.114:0/3423138419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ec70eab60 0x561ec70eafa0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ffb9800bc80 tx=0x7ffb9800f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d568352d0358d166 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.954+0000 7ffba5328640 1 -- 172.21.15.114:0/3423138419 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9801a460 con 0x561ec70eab60 2024-09-13T18:30:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.954+0000 7ffba5328640 1 -- 172.21.15.114:0/3423138419 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffb980118d0 con 0x561ec70eab60 2024-09-13T18:30:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.954+0000 7ffba5328640 1 -- 172.21.15.114:0/3423138419 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb98010750 con 0x561ec70eab60 2024-09-13T18:30:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.955+0000 7ffb9d7fa640 1 --2- 172.21.15.114:0/686664013 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ffb8c0a1340 0x7ffb8c0a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.955+0000 7ffb9d7fa640 1 -- 172.21.15.114:0/686664013 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9541+0+0 (secure 0 0 0) 0x7ffb900d2400 con 0x7ffba0107890 2024-09-13T18:30:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.955+0000 7ffba92ae740 1 -- 172.21.15.114:0/3423138419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ec70eab60 msgr2=0x561ec70eafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.955+0000 7ffba92ae740 1 --2- 172.21.15.114:0/3423138419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ec70eab60 0x561ec70eafa0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ffb9800bc80 tx=0x7ffb9800f730 comp rx=0 tx=0).stop 2024-09-13T18:30:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.957+0000 7ffba4b27640 1 --2- 172.21.15.114:0/686664013 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ffb8c0a1340 0x7ffb8c0a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.957+0000 7ffba92ae740 1 -- 172.21.15.114:0/3423138419 shutdown_connections 2024-09-13T18:30:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.957+0000 7ffba92ae740 1 --2- 172.21.15.114:0/3423138419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ec70f54b0 0x561ec70f78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.957+0000 7ffba92ae740 1 --2- 172.21.15.114:0/3423138419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ec70eab60 0x561ec70eafa0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.957+0000 7ffba92ae740 1 --2- 172.21.15.114:0/3423138419 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ec70ebdf0 0x561ec70ea620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.957+0000 7ffba92ae740 1 -- 172.21.15.114:0/3423138419 >> 172.21.15.114:0/3423138419 conn(0x561ec70e6dc0 msgr2=0x561ec70e71a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.957+0000 7ffba92ae740 1 -- 172.21.15.114:0/3423138419 shutdown_connections 2024-09-13T18:30:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.957+0000 7ffba92ae740 1 -- 172.21.15.114:0/3423138419 wait complete. 2024-09-13T18:30:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.958+0000 7ffba92ae740 1 -- start start 2024-09-13T18:30:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.958+0000 7ffba92ae740 1 Processor -- start 2024-09-13T18:30:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.959+0000 7ffba92ae740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ec70eab60 0x561ec70f4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:10.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.959+0000 7ffba92ae740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ec70ebdf0 0x561ec70f2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:10.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.959+0000 7ffba92ae740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ec70f54b0 0x561ec70f26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:10.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.959+0000 7ffba92ae740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ec700ce80 con 0x561ec70ebdf0 2024-09-13T18:30:10.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.959+0000 7ffba92ae740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ec700cb20 con 0x561ec70eab60 2024-09-13T18:30:10.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.959+0000 7ffba92ae740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ec6fd2260 con 0x561ec70f54b0 2024-09-13T18:30:10.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.959+0000 7ffba92ae740 10 client.0 _ll_register_callbacks cb 0x561ec7007df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:30:10.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.959+0000 7ffb9ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ec70eab60 0x561ec70f4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:10.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.959+0000 7ffb9ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ec70eab60 0x561ec70f4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:60822/0 (socket says 172.21.15.114:60822) 2024-09-13T18:30:10.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.959+0000 7ffb9ffff640 1 -- 172.21.15.114:0/3591906535 learned_addr learned my addr 172.21.15.114:0/3591906535 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:10.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.959+0000 7ffb9ffff640 1 -- 172.21.15.114:0/3591906535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ec70f54b0 msgr2=0x561ec70f26d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:30:10.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.959+0000 7ffb9ffff640 1 --2- 172.21.15.114:0/3591906535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ec70f54b0 0x561ec70f26d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.960+0000 7ffb9ffff640 1 -- 172.21.15.114:0/3591906535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ec70ebdf0 msgr2=0x561ec70f2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.960+0000 7ffb9ffff640 1 --2- 172.21.15.114:0/3591906535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ec70ebdf0 0x561ec70f2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.960+0000 7ffb9ffff640 1 -- 172.21.15.114:0/3591906535 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb9800b8e0 con 0x561ec70eab60 2024-09-13T18:30:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.960+0000 7ffb9ffff640 1 --2- 172.21.15.114:0/3591906535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ec70eab60 0x561ec70f4e10 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ffb98011c70 tx=0x7ffb98011ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6aad1e2c6c7d3408 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.961+0000 7ffba4b27640 1 --2- 172.21.15.114:0/686664013 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ffb8c0a1340 0x7ffb8c0a3800 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x561ec70f3600 tx=0x7ffb8802f3c0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:10.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.962+0000 7ffb7f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:30:10.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.962+0000 7ffb7f7fe640 1 -- 172.21.15.114:0/3591906535 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9801e070 con 0x561ec70eab60 2024-09-13T18:30:10.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.962+0000 7ffb7f7fe640 1 -- 172.21.15.114:0/3591906535 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9541+0+0 (secure 0 0 0) 0x7ffb98005040 con 0x561ec70eab60 2024-09-13T18:30:10.968 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.962+0000 7ffba92ae740 1 -- 172.21.15.114:0/3591906535 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561ec7077680 con 0x561ec70eab60 2024-09-13T18:30:10.968 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.964+0000 7ffb7f7fe640 1 -- 172.21.15.114:0/3591906535 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb98005200 con 0x561ec70eab60 2024-09-13T18:30:10.968 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.965+0000 7ffba92ae740 10 client.27245 fetch_fsmap learned FSMap version 34 2024-09-13T18:30:10.968 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.965+0000 7ffba92ae740 1 -- 172.21.15.114:0/3591906535 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x561ec704d330 con 0x561ec70eab60 2024-09-13T18:30:10.972 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.967+0000 7ffb7f7fe640 1 -- 172.21.15.114:0/3591906535 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561ec7077680 con 0x561ec70eab60 2024-09-13T18:30:10.973 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.967+0000 7ffb7f7fe640 1 -- 172.21.15.114:0/3591906535 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ffb980104d0 con 0x561ec70eab60 2024-09-13T18:30:10.973 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.968+0000 7ffba92ae740 10 client.27245 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:30:10.973 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.968+0000 7ffba92ae740 10 client.27245 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:30:10.973 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.968+0000 7ffba92ae740 20 client.27245 populate_metadata read hostname 'smithi114' 2024-09-13T18:30:10.973 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.968+0000 7ffba92ae740 1 --2- 172.21.15.114:0/3591906535 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x561ec7104f40 0x561ec7107350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:10.973 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.968+0000 7ffba92ae740 4 client.27245 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:30:10.973 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.968+0000 7ffba92ae740 1 -- 172.21.15.114:0/3591906535 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561ec70425d0 con 0x561ec7104f40 2024-09-13T18:30:10.973 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.969+0000 7ffb9f7fe640 1 --2- 172.21.15.114:0/3591906535 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x561ec7104f40 0x561ec7107350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:10.974 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.970+0000 7ffb9f7fe640 1 --2- 172.21.15.114:0/3591906535 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x561ec7104f40 0x561ec7107350 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b1672c2b1380757 server_cookie=bd549db7072d0b4b in_seq=0 out_seq=0 2024-09-13T18:30:10.974 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.970+0000 7ffb7f7fe640 10 client.27245 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:30:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.971+0000 7ffb7f7fe640 1 -- 172.21.15.114:0/3591906535 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x561ec70425d0 con 0x561ec7104f40 2024-09-13T18:30:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.971+0000 7ffb7f7fe640 10 client.27245 handle_command_reply: tid=0 2024-09-13T18:30:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.971+0000 7ffba92ae740 1 client.27245 shutdown 2024-09-13T18:30:10.978 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.975+0000 7ffba92ae740 1 -- 172.21.15.114:0/3591906535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ec70eab60 msgr2=0x561ec70f4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:10.978 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.975+0000 7ffba92ae740 1 --2- 172.21.15.114:0/3591906535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ec70eab60 0x561ec70f4e10 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ffb98011c70 tx=0x7ffb98011ca0 comp rx=0 tx=0).stop 2024-09-13T18:30:10.979 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.976+0000 7ffba92ae740 1 -- 172.21.15.114:0/3591906535 shutdown_connections 2024-09-13T18:30:10.980 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.976+0000 7ffba92ae740 1 --2- 172.21.15.114:0/3591906535 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x561ec7104f40 0x561ec7107350 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:10.980 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.976+0000 7ffba92ae740 1 --2- 172.21.15.114:0/3591906535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ec70f54b0 0x561ec70f26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:10.980 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.976+0000 7ffba92ae740 1 --2- 172.21.15.114:0/3591906535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ec70ebdf0 0x561ec70f2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:10.980 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.976+0000 7ffba92ae740 1 --2- 172.21.15.114:0/3591906535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ec70eab60 0x561ec70f4e10 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:10.980 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.976+0000 7ffba92ae740 1 -- 172.21.15.114:0/3591906535 >> 172.21.15.114:0/3591906535 conn(0x561ec70e6dc0 msgr2=0x561ec70e5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:10.980 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.976+0000 7ffb7f7fe640 0 client.27245 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:30:10.981 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.976+0000 7ffba92ae740 1 -- 172.21.15.114:0/3591906535 shutdown_connections 2024-09-13T18:30:10.981 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.976+0000 7ffba92ae740 1 -- 172.21.15.114:0/3591906535 wait complete. 2024-09-13T18:30:10.981 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:10.977+0000 7ffba92ae740 20 client.27245 trim_cache size 0 max 16384 2024-09-13T18:30:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.005+0000 7ffba92ae740 1 Processor -- start 2024-09-13T18:30:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.007+0000 7ffba92ae740 1 -- start start 2024-09-13T18:30:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.007+0000 7ffba92ae740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ec70eab60 0x561ec70f68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.007+0000 7ffba92ae740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ec70ebdf0 0x561ec70f4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.007+0000 7ffba92ae740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ec70f54b0 0x561ec70ed450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.007+0000 7ffba92ae740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ec70eb570 con 0x561ec70eab60 2024-09-13T18:30:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.007+0000 7ffba92ae740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ec7101db0 con 0x561ec70ebdf0 2024-09-13T18:30:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.007+0000 7ffba92ae740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ec70f3200 con 0x561ec70f54b0 2024-09-13T18:30:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.007+0000 7ffb9ffff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ec70eab60 0x561ec70f68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.007+0000 7ffb9ffff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ec70eab60 0x561ec70f68a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:56370/0 (socket says 172.21.15.114:56370) 2024-09-13T18:30:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.007+0000 7ffb9ffff640 1 -- 172.21.15.114:0/928849426 learned_addr learned my addr 172.21.15.114:0/928849426 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.008+0000 7ffba4b27640 1 --2- 172.21.15.114:0/928849426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ec70f54b0 0x561ec70ed450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.009+0000 7ffb9ffff640 1 -- 172.21.15.114:0/928849426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ec70f54b0 msgr2=0x561ec70ed450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.009+0000 7ffb9ffff640 1 --2- 172.21.15.114:0/928849426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ec70f54b0 0x561ec70ed450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.009+0000 7ffb9ffff640 1 -- 172.21.15.114:0/928849426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ec70ebdf0 msgr2=0x561ec70f4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:11.015 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.009+0000 7ffb9ffff640 1 --2- 172.21.15.114:0/928849426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ec70ebdf0 0x561ec70f4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:11.015 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.009+0000 7ffb9ffff640 1 -- 172.21.15.114:0/928849426 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ec6fef300 con 0x561ec70eab60 2024-09-13T18:30:11.015 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.010+0000 7ffb9ffff640 1 --2- 172.21.15.114:0/928849426 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ec70eab60 0x561ec70f68a0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7ffb980109b0 tx=0x7ffb980109e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3692fbe2fd9d2bc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:11.015 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.011+0000 7ffb967fc640 1 -- 172.21.15.114:0/928849426 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9801e070 con 0x561ec70eab60 2024-09-13T18:30:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.011+0000 7ffb967fc640 1 -- 172.21.15.114:0/928849426 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffb9801f040 con 0x561ec70eab60 2024-09-13T18:30:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.012+0000 7ffba92ae740 1 -- 172.21.15.114:0/928849426 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ec70eab60 msgr2=0x561ec70f68a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.012+0000 7ffba92ae740 1 --2- 172.21.15.114:0/928849426 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ec70eab60 0x561ec70f68a0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7ffb980109b0 tx=0x7ffb980109e0 comp rx=0 tx=0).stop 2024-09-13T18:30:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.013+0000 7ffba92ae740 1 -- 172.21.15.114:0/928849426 shutdown_connections 2024-09-13T18:30:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.013+0000 7ffba92ae740 1 --2- 172.21.15.114:0/928849426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ec70f54b0 0x561ec70ed450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.013+0000 7ffba92ae740 1 --2- 172.21.15.114:0/928849426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ec70ebdf0 0x561ec70f4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.013+0000 7ffba92ae740 1 --2- 172.21.15.114:0/928849426 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ec70eab60 0x561ec70f68a0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.013+0000 7ffba92ae740 1 -- 172.21.15.114:0/928849426 >> 172.21.15.114:0/928849426 conn(0x561ec70e6dc0 msgr2=0x561ec70f1650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.015+0000 7ffba92ae740 1 -- 172.21.15.114:0/928849426 shutdown_connections 2024-09-13T18:30:11.022 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.015+0000 7ffba92ae740 1 -- 172.21.15.114:0/928849426 wait complete. 2024-09-13T18:30:11.022 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.016+0000 7ffba92ae740 1 -- start start 2024-09-13T18:30:11.022 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.016+0000 7ffba92ae740 1 Processor -- start 2024-09-13T18:30:11.022 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.016+0000 7ffba92ae740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ec70eab60 0x561ec70f8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:11.022 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.016+0000 7ffba92ae740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ec70ebdf0 0x561ec70f9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:11.022 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.016+0000 7ffba92ae740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ec70f54b0 0x561ec71533f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:11.022 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.016+0000 7ffba92ae740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ec7153af0 con 0x561ec70eab60 2024-09-13T18:30:11.022 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.016+0000 7ffba92ae740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ec7153c60 con 0x561ec70ebdf0 2024-09-13T18:30:11.023 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.016+0000 7ffba92ae740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ec7153e20 con 0x561ec70f54b0 2024-09-13T18:30:11.023 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.016+0000 7ffba92ae740 10 client.0 _ll_register_callbacks cb 0x7ffb9800b440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:30:11.023 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.017+0000 7ffb9f7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ec70ebdf0 0x561ec70f9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:11.023 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.017+0000 7ffb9f7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ec70ebdf0 0x561ec70f9140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:60844/0 (socket says 172.21.15.114:60844) 2024-09-13T18:30:11.023 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.017+0000 7ffb9f7fe640 1 -- 172.21.15.114:0/3587136386 learned_addr learned my addr 172.21.15.114:0/3587136386 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:11.023 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.017+0000 7ffba4b27640 1 --2- 172.21.15.114:0/3587136386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ec70f54b0 0x561ec71533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:11.023 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.018+0000 7ffb9f7fe640 1 -- 172.21.15.114:0/3587136386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ec70f54b0 msgr2=0x561ec71533f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:11.023 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.018+0000 7ffb9f7fe640 1 --2- 172.21.15.114:0/3587136386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ec70f54b0 0x561ec71533f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:11.023 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.018+0000 7ffb9f7fe640 1 -- 172.21.15.114:0/3587136386 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ec70eab60 msgr2=0x561ec70f8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:11.023 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.018+0000 7ffb9f7fe640 1 --2- 172.21.15.114:0/3587136386 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ec70eab60 0x561ec70f8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:11.023 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.018+0000 7ffb9f7fe640 1 -- 172.21.15.114:0/3587136386 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561ec704d330 con 0x561ec70ebdf0 2024-09-13T18:30:11.023 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.018+0000 7ffb9f7fe640 1 --2- 172.21.15.114:0/3587136386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ec70ebdf0 0x561ec70f9140 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ffb9009ba90 tx=0x7ffb9009b480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea0eb89230b7d26c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:11.023 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.018+0000 7ffb7f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:30:11.025 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.021+0000 7ffb7f7fe640 1 -- 172.21.15.114:0/3587136386 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9009f7b0 con 0x561ec70ebdf0 2024-09-13T18:30:11.025 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.021+0000 7ffb7f7fe640 1 -- 172.21.15.114:0/3587136386 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9541+0+0 (secure 0 0 0) 0x7ffb900cfe40 con 0x561ec70ebdf0 2024-09-13T18:30:11.025 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.022+0000 7ffba92ae740 1 -- 172.21.15.114:0/3587136386 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffb980118d0 con 0x561ec70ebdf0 2024-09-13T18:30:11.026 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.023+0000 7ffb7f7fe640 1 -- 172.21.15.114:0/3587136386 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb9009f0b0 con 0x561ec70ebdf0 2024-09-13T18:30:11.029 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.026+0000 7ffba92ae740 10 client.27248 fetch_fsmap learned FSMap version 34 2024-09-13T18:30:11.029 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.026+0000 7ffba92ae740 1 -- 172.21.15.114:0/3587136386 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x561ec70148e0 con 0x561ec70ebdf0 2024-09-13T18:30:11.033 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.030+0000 7ffb7f7fe640 1 -- 172.21.15.114:0/3587136386 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffb980118d0 con 0x561ec70ebdf0 2024-09-13T18:30:11.033 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.030+0000 7ffb7f7fe640 1 -- 172.21.15.114:0/3587136386 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ffb900cf970 con 0x561ec70ebdf0 2024-09-13T18:30:11.038 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.032+0000 7ffba92ae740 10 client.27248 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:30:11.038 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.032+0000 7ffba92ae740 10 client.27248 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:30:11.038 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.032+0000 7ffba92ae740 20 client.27248 populate_metadata read hostname 'smithi114' 2024-09-13T18:30:11.038 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.032+0000 7ffba92ae740 1 --2- 172.21.15.114:0/3587136386 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x561ec717e6a0 0x561ec7180ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:11.038 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.032+0000 7ffba92ae740 4 client.27248 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T18:30:11.038 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.032+0000 7ffba92ae740 1 -- 172.21.15.114:0/3587136386 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561ec6d42300 con 0x561ec717e6a0 2024-09-13T18:30:11.038 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.033+0000 7ffb9ffff640 1 --2- 172.21.15.114:0/3587136386 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x561ec717e6a0 0x561ec7180ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:11.047 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.043+0000 7ffb9ffff640 1 --2- 172.21.15.114:0/3587136386 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x561ec717e6a0 0x561ec7180ac0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ced839e8702d8b3 server_cookie=66f7c73e8e99dc9 in_seq=0 out_seq=0 2024-09-13T18:30:11.048 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.044+0000 7ffb7f7fe640 10 client.27248 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:30:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.075+0000 7ffb7f7fe640 1 -- 172.21.15.114:0/3587136386 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561ec6d42300 con 0x561ec717e6a0 2024-09-13T18:30:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.075+0000 7ffb7f7fe640 10 client.27248 handle_command_reply: tid=0 2024-09-13T18:30:11.079 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.076+0000 7ffba92ae740 1 client.27248 shutdown 2024-09-13T18:30:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.079+0000 7ffba92ae740 1 -- 172.21.15.114:0/3587136386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ec70ebdf0 msgr2=0x561ec70f9140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.079+0000 7ffba92ae740 1 --2- 172.21.15.114:0/3587136386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ec70ebdf0 0x561ec70f9140 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ffb9009ba90 tx=0x7ffb9009b480 comp rx=0 tx=0).stop 2024-09-13T18:30:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.080+0000 7ffba92ae740 1 -- 172.21.15.114:0/3587136386 shutdown_connections 2024-09-13T18:30:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.080+0000 7ffba92ae740 1 --2- 172.21.15.114:0/3587136386 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x561ec717e6a0 0x561ec7180ac0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.080+0000 7ffba92ae740 1 --2- 172.21.15.114:0/3587136386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ec70f54b0 0x561ec71533f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.080+0000 7ffba92ae740 1 --2- 172.21.15.114:0/3587136386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ec70ebdf0 0x561ec70f9140 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.080+0000 7ffba92ae740 1 --2- 172.21.15.114:0/3587136386 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ec70eab60 0x561ec70f8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.080+0000 7ffba92ae740 1 -- 172.21.15.114:0/3587136386 >> 172.21.15.114:0/3587136386 conn(0x561ec70e6dc0 msgr2=0x561ec70ed450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.080+0000 7ffb7f7fe640 0 client.27248 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:30:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.080+0000 7ffba92ae740 1 -- 172.21.15.114:0/3587136386 shutdown_connections 2024-09-13T18:30:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.080+0000 7ffba92ae740 1 -- 172.21.15.114:0/3587136386 wait complete. 2024-09-13T18:30:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.081+0000 7ffba92ae740 20 client.27248 trim_cache size 0 max 16384 2024-09-13T18:30:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-13T18:30:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-13T18:30:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-13T18:30:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-13T18:30:11.086 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.082+0000 7ffb96ffd640 1 -- 172.21.15.114:0/686664013 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ffb8c0a1340 msgr2=0x7ffb8c0a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:11.087 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.082+0000 7ffb96ffd640 1 --2- 172.21.15.114:0/686664013 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ffb8c0a1340 0x7ffb8c0a3800 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x561ec70f3600 tx=0x7ffb8802f3c0 comp rx=0 tx=0).stop 2024-09-13T18:30:11.087 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.082+0000 7ffb96ffd640 1 -- 172.21.15.114:0/686664013 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffba0107890 msgr2=0x7ffba0118e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:11.087 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.082+0000 7ffb96ffd640 1 --2- 172.21.15.114:0/686664013 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffba0107890 0x7ffba0118e50 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7ffb9000d950 tx=0x7ffb9000de10 comp rx=0 tx=0).stop 2024-09-13T18:30:11.087 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.082+0000 7ffb96ffd640 1 -- 172.21.15.114:0/686664013 shutdown_connections 2024-09-13T18:30:11.087 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.082+0000 7ffb96ffd640 1 --2- 172.21.15.114:0/686664013 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ffb8c0a1340 0x7ffb8c0a3800 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:11.087 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.082+0000 7ffb96ffd640 1 --2- 172.21.15.114:0/686664013 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffba0119400 0x7ffba0112b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:11.087 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.082+0000 7ffb96ffd640 1 --2- 172.21.15.114:0/686664013 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffba0107890 0x7ffba0118e50 secure :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7ffb9000d950 tx=0x7ffb9000de10 comp rx=0 tx=0).stop 2024-09-13T18:30:11.087 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.082+0000 7ffb96ffd640 1 --2- 172.21.15.114:0/686664013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffba01074b0 0x7ffba0118910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:11.087 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.082+0000 7ffb96ffd640 1 -- 172.21.15.114:0/686664013 >> 172.21.15.114:0/686664013 conn(0x7ffba006adf0 msgr2=0x7ffba006eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:11.087 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.083+0000 7ffb96ffd640 1 -- 172.21.15.114:0/686664013 shutdown_connections 2024-09-13T18:30:11.087 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:11.083+0000 7ffb96ffd640 1 -- 172.21.15.114:0/686664013 wait complete. 2024-09-13T18:30:12.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:12 smithi134 ceph-mon[34283]: pgmap v237: 129 pgs: 6 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 109 active+clean; 22 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 30 MiB/s wr, 666 op/s 2024-09-13T18:30:12.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:12 smithi204 ceph-mon[34777]: pgmap v237: 129 pgs: 6 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 109 active+clean; 22 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 30 MiB/s wr, 666 op/s 2024-09-13T18:30:12.474 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:12 smithi114 ceph-mon[30897]: pgmap v237: 129 pgs: 6 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 109 active+clean; 22 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 30 MiB/s wr, 666 op/s 2024-09-13T18:30:14.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:14 smithi134 ceph-mon[34283]: pgmap v238: 129 pgs: 6 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 109 active+clean; 22 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 30 MiB/s wr, 666 op/s 2024-09-13T18:30:14.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:14 smithi204 ceph-mon[34777]: pgmap v238: 129 pgs: 6 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 109 active+clean; 22 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 30 MiB/s wr, 666 op/s 2024-09-13T18:30:14.472 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:14 smithi114 ceph-mon[30897]: pgmap v238: 129 pgs: 6 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 109 active+clean; 22 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 30 MiB/s wr, 666 op/s 2024-09-13T18:30:16.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:16 smithi134 ceph-mon[34283]: pgmap v239: 129 pgs: 6 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 109 active+clean; 22 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 30 MiB/s wr, 667 op/s 2024-09-13T18:30:16.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:16 smithi204 ceph-mon[34777]: pgmap v239: 129 pgs: 6 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 109 active+clean; 22 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 30 MiB/s wr, 667 op/s 2024-09-13T18:30:16.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:16 smithi114 ceph-mon[30897]: pgmap v239: 129 pgs: 6 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 109 active+clean; 22 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 30 MiB/s wr, 667 op/s 2024-09-13T18:30:17.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:17 smithi134 ceph-mon[34283]: scrub summary: idle 2024-09-13T18:30:17.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:17 smithi134 ceph-mon[34283]: scrub summary: idle 2024-09-13T18:30:17.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:17 smithi134 ceph-mon[34283]: scrub summary: idle 2024-09-13T18:30:17.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:17 smithi204 ceph-mon[34777]: scrub summary: idle 2024-09-13T18:30:17.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:17 smithi204 ceph-mon[34777]: scrub summary: idle 2024-09-13T18:30:17.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:17 smithi204 ceph-mon[34777]: scrub summary: idle 2024-09-13T18:30:17.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:17 smithi114 ceph-mon[30897]: scrub summary: idle 2024-09-13T18:30:17.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:17 smithi114 ceph-mon[30897]: scrub summary: idle 2024-09-13T18:30:17.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:17 smithi114 ceph-mon[30897]: scrub summary: idle 2024-09-13T18:30:18.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:18 smithi134 ceph-mon[34283]: pgmap v240: 129 pgs: 129 active+clean; 22 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 30 MiB/s wr, 657 op/s 2024-09-13T18:30:18.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:18 smithi204 ceph-mon[34777]: pgmap v240: 129 pgs: 129 active+clean; 22 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 30 MiB/s wr, 657 op/s 2024-09-13T18:30:18.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:18 smithi114 ceph-mon[30897]: pgmap v240: 129 pgs: 129 active+clean; 22 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 30 MiB/s wr, 657 op/s 2024-09-13T18:30:19.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:19 smithi204 ceph-mon[34777]: pgmap v241: 129 pgs: 129 active+clean; 22 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 27 MiB/s wr, 585 op/s 2024-09-13T18:30:19.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:19 smithi204 ceph-mon[34777]: osdmap e98: 12 total, 12 up, 12 in 2024-09-13T18:30:19.725 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:19 smithi114 ceph-mon[30897]: pgmap v241: 129 pgs: 129 active+clean; 22 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 27 MiB/s wr, 585 op/s 2024-09-13T18:30:19.725 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:19 smithi114 ceph-mon[30897]: osdmap e98: 12 total, 12 up, 12 in 2024-09-13T18:30:19.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:19 smithi134 ceph-mon[34283]: pgmap v241: 129 pgs: 129 active+clean; 22 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 27 MiB/s wr, 585 op/s 2024-09-13T18:30:19.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:19 smithi134 ceph-mon[34283]: osdmap e98: 12 total, 12 up, 12 in 2024-09-13T18:30:21.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:21 smithi204 ceph-mon[34777]: pgmap v243: 129 pgs: 129 active+clean; 22 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 33 MiB/s wr, 702 op/s 2024-09-13T18:30:21.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:21 smithi114 ceph-mon[30897]: pgmap v243: 129 pgs: 129 active+clean; 22 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 33 MiB/s wr, 702 op/s 2024-09-13T18:30:21.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:21 smithi134 ceph-mon[34283]: pgmap v243: 129 pgs: 129 active+clean; 22 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 33 MiB/s wr, 702 op/s 2024-09-13T18:30:23.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:23 smithi204 ceph-mon[34777]: pgmap v244: 129 pgs: 129 active+clean; 22 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 33 MiB/s wr, 702 op/s 2024-09-13T18:30:23.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:23 smithi114 ceph-mon[30897]: pgmap v244: 129 pgs: 129 active+clean; 22 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 33 MiB/s wr, 702 op/s 2024-09-13T18:30:23.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:23 smithi134 ceph-mon[34283]: pgmap v244: 129 pgs: 129 active+clean; 22 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 33 MiB/s wr, 702 op/s 2024-09-13T18:30:24.263 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.260+0000 7ff9db7fe640 1 -- 172.21.15.114:0/429833282 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc031730 con 0x7ff9dc103880 2024-09-13T18:30:24.263 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.260+0000 7ff9e3108640 1 -- 172.21.15.114:0/429833282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9dc103880 msgr2=0x7ff9dc103c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:24.263 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.260+0000 7ff9e3108640 1 --2- 172.21.15.114:0/429833282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9dc103880 0x7ff9dc103c60 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc009900 tx=0x7ff9cc02a4b0 comp rx=0 tx=0).stop 2024-09-13T18:30:24.264 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.261+0000 7ff9e3108640 1 -- 172.21.15.114:0/429833282 shutdown_connections 2024-09-13T18:30:24.264 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.261+0000 7ff9e3108640 1 --2- 172.21.15.114:0/429833282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9dc110c30 0x7ff9dc113020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:24.264 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.261+0000 7ff9e3108640 1 --2- 172.21.15.114:0/429833282 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9dc1041a0 0x7ff9dc1106f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:24.264 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.261+0000 7ff9e3108640 1 --2- 172.21.15.114:0/429833282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9dc103880 0x7ff9dc103c60 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:24.264 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.261+0000 7ff9e3108640 1 -- 172.21.15.114:0/429833282 >> 172.21.15.114:0/429833282 conn(0x7ff9dc075380 msgr2=0x7ff9dc075790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:24.264 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.261+0000 7ff9e3108640 1 -- 172.21.15.114:0/429833282 shutdown_connections 2024-09-13T18:30:24.264 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.262+0000 7ff9e3108640 1 -- 172.21.15.114:0/429833282 wait complete. 2024-09-13T18:30:24.265 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.263+0000 7ff9e3108640 1 Processor -- start 2024-09-13T18:30:24.267 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.264+0000 7ff9e3108640 1 -- start start 2024-09-13T18:30:24.267 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.264+0000 7ff9e3108640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9dc103880 0x7ff9dc10b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:24.267 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.264+0000 7ff9e3108640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9dc1041a0 0x7ff9dc10bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:24.267 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.264+0000 7ff9e3108640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9dc110c30 0x7ff9dc106980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:24.267 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.264+0000 7ff9e3108640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9dc106ff0 con 0x7ff9dc110c30 2024-09-13T18:30:24.267 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.264+0000 7ff9e3108640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9dc107160 con 0x7ff9dc103880 2024-09-13T18:30:24.267 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.264+0000 7ff9e3108640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9dc107320 con 0x7ff9dc1041a0 2024-09-13T18:30:24.268 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.264+0000 7ff9e16a5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9dc110c30 0x7ff9dc106980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:24.268 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.264+0000 7ff9e16a5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9dc110c30 0x7ff9dc106980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50258/0 (socket says 172.21.15.114:50258) 2024-09-13T18:30:24.268 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.264+0000 7ff9e16a5640 1 -- 172.21.15.114:0/4203235539 learned_addr learned my addr 172.21.15.114:0/4203235539 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:24.269 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.266+0000 7ff9e16a5640 1 -- 172.21.15.114:0/4203235539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9dc1041a0 msgr2=0x7ff9dc10bcd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:30:24.269 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.266+0000 7ff9e16a5640 1 --2- 172.21.15.114:0/4203235539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9dc1041a0 0x7ff9dc10bcd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:24.269 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.266+0000 7ff9e16a5640 1 -- 172.21.15.114:0/4203235539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9dc103880 msgr2=0x7ff9dc10b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:24.269 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.266+0000 7ff9e16a5640 1 --2- 172.21.15.114:0/4203235539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9dc103880 0x7ff9dc10b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:24.269 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.266+0000 7ff9e16a5640 1 -- 172.21.15.114:0/4203235539 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9cc009560 con 0x7ff9dc110c30 2024-09-13T18:30:24.269 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.266+0000 7ff9e16a5640 1 --2- 172.21.15.114:0/4203235539 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9dc110c30 0x7ff9dc106980 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7ff9d000b410 tx=0x7ff9d000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:24.271 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.269+0000 7ff9d9ffb640 1 -- 172.21.15.114:0/4203235539 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d0011830 con 0x7ff9dc110c30 2024-09-13T18:30:24.271 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.269+0000 7ff9e3108640 1 -- 172.21.15.114:0/4203235539 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9dc107600 con 0x7ff9dc110c30 2024-09-13T18:30:24.272 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.269+0000 7ff9e3108640 1 -- 172.21.15.114:0/4203235539 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9dc10a6d0 con 0x7ff9dc110c30 2024-09-13T18:30:24.272 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.270+0000 7ff9d9ffb640 1 -- 172.21.15.114:0/4203235539 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff9d0011e20 con 0x7ff9dc110c30 2024-09-13T18:30:24.273 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.271+0000 7ff9d9ffb640 1 -- 172.21.15.114:0/4203235539 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d001ae60 con 0x7ff9dc110c30 2024-09-13T18:30:24.279 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.277+0000 7ff9d9ffb640 1 -- 172.21.15.114:0/4203235539 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7ff9d001a400 con 0x7ff9dc110c30 2024-09-13T18:30:24.281 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.279+0000 7ff9d9ffb640 1 --2- 172.21.15.114:0/4203235539 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff9c40a11e0 0x7ff9c40a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:24.282 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.280+0000 7ff9d9ffb640 1 -- 172.21.15.114:0/4203235539 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ff9d00ccaa0 con 0x7ff9dc110c30 2024-09-13T18:30:24.283 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.281+0000 7ff9e5e2b740 1 Processor -- start 2024-09-13T18:30:24.288 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.285+0000 7ff9e5e2b740 1 -- start start 2024-09-13T18:30:24.289 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.285+0000 7ff9e5e2b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3063ec330 0x55f3063ec710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:24.289 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.285+0000 7ff9e5e2b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3063ecc50 0x55f3063f6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:24.289 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.285+0000 7ff9e5e2b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f3063f74f0 0x55f3063f98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:24.289 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.285+0000 7ff9e5e2b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f306357c30 con 0x55f3063f74f0 2024-09-13T18:30:24.289 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.285+0000 7ff9e5e2b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3063553c0 con 0x55f3063ec330 2024-09-13T18:30:24.289 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.285+0000 7ff9e5e2b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f306350cb0 con 0x55f3063ecc50 2024-09-13T18:30:24.289 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.286+0000 7ff9e16a5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f3063f74f0 0x55f3063f98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:24.289 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.286+0000 7ff9e16a5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f3063f74f0 0x55f3063f98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50260/0 (socket says 172.21.15.114:50260) 2024-09-13T18:30:24.289 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.286+0000 7ff9e16a5640 1 -- 172.21.15.114:0/3531203053 learned_addr learned my addr 172.21.15.114:0/3531203053 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:24.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.287+0000 7ff9e0ea4640 1 --2- 172.21.15.114:0/4203235539 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff9c40a11e0 0x7ff9c40a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:24.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.288+0000 7ff9e16a5640 1 -- 172.21.15.114:0/3531203053 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3063ecc50 msgr2=0x55f3063f6fb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:30:24.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.288+0000 7ff9e16a5640 1 --2- 172.21.15.114:0/3531203053 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3063ecc50 0x55f3063f6fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:24.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.288+0000 7ff9e16a5640 1 -- 172.21.15.114:0/3531203053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3063ec330 msgr2=0x55f3063ec710 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:30:24.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.288+0000 7ff9e16a5640 1 --2- 172.21.15.114:0/3531203053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3063ec330 0x55f3063ec710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:24.291 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.288+0000 7ff9e16a5640 1 -- 172.21.15.114:0/3531203053 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3062f7910 con 0x55f3063f74f0 2024-09-13T18:30:24.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.290+0000 7ff9e16a5640 1 --2- 172.21.15.114:0/3531203053 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f3063f74f0 0x55f3063f98e0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0096330 tx=0x7ff9d0099220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40396f9074b23819 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:24.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.291+0000 7ff9c2ffd640 1 -- 172.21.15.114:0/3531203053 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d009d6f0 con 0x55f3063f74f0 2024-09-13T18:30:24.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.291+0000 7ff9c2ffd640 1 -- 172.21.15.114:0/3531203053 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff9d00a1d40 con 0x55f3063f74f0 2024-09-13T18:30:24.294 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.292+0000 7ff9e5e2b740 1 -- 172.21.15.114:0/3531203053 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f3063f74f0 msgr2=0x55f3063f98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:24.294 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.292+0000 7ff9e5e2b740 1 --2- 172.21.15.114:0/3531203053 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f3063f74f0 0x55f3063f98e0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0096330 tx=0x7ff9d0099220 comp rx=0 tx=0).stop 2024-09-13T18:30:24.298 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.296+0000 7ff9e5e2b740 1 -- 172.21.15.114:0/3531203053 shutdown_connections 2024-09-13T18:30:24.298 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.296+0000 7ff9e5e2b740 1 --2- 172.21.15.114:0/3531203053 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f3063f74f0 0x55f3063f98e0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:24.298 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.296+0000 7ff9e5e2b740 1 --2- 172.21.15.114:0/3531203053 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3063ecc50 0x55f3063f6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:24.298 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.296+0000 7ff9e5e2b740 1 --2- 172.21.15.114:0/3531203053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3063ec330 0x55f3063ec710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:24.298 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.296+0000 7ff9e5e2b740 1 -- 172.21.15.114:0/3531203053 >> 172.21.15.114:0/3531203053 conn(0x55f3063e88c0 msgr2=0x55f3063e8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:24.305 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.302+0000 7ff9e0ea4640 1 --2- 172.21.15.114:0/4203235539 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff9c40a11e0 0x7ff9c40a36a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc00b230 tx=0x7ff9cc005de0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:24.306 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.303+0000 7ff9e5e2b740 1 -- 172.21.15.114:0/3531203053 shutdown_connections 2024-09-13T18:30:24.310 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.305+0000 7ff9e5e2b740 1 -- 172.21.15.114:0/3531203053 wait complete. 2024-09-13T18:30:24.310 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.306+0000 7ff9e5e2b740 1 -- start start 2024-09-13T18:30:24.310 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.307+0000 7ff9e5e2b740 1 Processor -- start 2024-09-13T18:30:24.310 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.307+0000 7ff9e5e2b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3063ec330 0x55f3063f6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:24.311 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.307+0000 7ff9e5e2b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f3063ecc50 0x55f3063f40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:24.311 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.307+0000 7ff9e5e2b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3063f74f0 0x55f3063f45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:24.311 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.307+0000 7ff9e5e2b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f30630ee80 con 0x55f3063ecc50 2024-09-13T18:30:24.311 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.307+0000 7ff9e5e2b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f30630eb20 con 0x55f3063f74f0 2024-09-13T18:30:24.311 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.307+0000 7ff9e5e2b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3062d4260 con 0x55f3063ec330 2024-09-13T18:30:24.311 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.307+0000 7ff9e5e2b740 10 client.0 _ll_register_callbacks cb 0x55f306309df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:30:24.311 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.308+0000 7ff9e16a5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3063f74f0 0x55f3063f45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:24.311 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.308+0000 7ff9e16a5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3063f74f0 0x55f3063f45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:41690/0 (socket says 172.21.15.114:41690) 2024-09-13T18:30:24.311 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.308+0000 7ff9e16a5640 1 -- 172.21.15.114:0/3629088587 learned_addr learned my addr 172.21.15.114:0/3629088587 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:24.311 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.308+0000 7ff9e16a5640 1 -- 172.21.15.114:0/3629088587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3063ec330 msgr2=0x55f3063f6e00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:30:24.312 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.308+0000 7ff9e16a5640 1 --2- 172.21.15.114:0/3629088587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3063ec330 0x55f3063f6e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:24.312 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.308+0000 7ff9e16a5640 1 -- 172.21.15.114:0/3629088587 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f3063ecc50 msgr2=0x55f3063f40a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:30:24.312 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.308+0000 7ff9e16a5640 1 --2- 172.21.15.114:0/3629088587 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f3063ecc50 0x55f3063f40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:24.312 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.308+0000 7ff9e16a5640 1 -- 172.21.15.114:0/3629088587 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9dc10a6d0 con 0x55f3063f74f0 2024-09-13T18:30:24.312 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.309+0000 7ff9e16a5640 1 --2- 172.21.15.114:0/3629088587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3063f74f0 0x55f3063f45e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff9d000b3e0 tx=0x7ff9d001a7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f95d7ee7ecfb38fc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:24.312 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.309+0000 7ff9bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:30:24.312 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.310+0000 7ff9bbfff640 1 -- 172.21.15.114:0/3629088587 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d00b6070 con 0x55f3063f74f0 2024-09-13T18:30:24.312 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.310+0000 7ff9bbfff640 1 -- 172.21.15.114:0/3629088587 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ff9d00b90b0 con 0x55f3063f74f0 2024-09-13T18:30:24.315 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.313+0000 7ff9e5e2b740 1 -- 172.21.15.114:0/3629088587 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f306379680 con 0x55f3063f74f0 2024-09-13T18:30:24.317 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.315+0000 7ff9bbfff640 1 -- 172.21.15.114:0/3629088587 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9d00b22a0 con 0x55f3063f74f0 2024-09-13T18:30:24.317 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.315+0000 7ff9e5e2b740 10 client.27251 fetch_fsmap learned FSMap version 34 2024-09-13T18:30:24.317 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.315+0000 7ff9e5e2b740 1 -- 172.21.15.114:0/3629088587 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55f30634f330 con 0x55f3063f74f0 2024-09-13T18:30:24.318 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.316+0000 7ff9bbfff640 1 -- 172.21.15.114:0/3629088587 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f306379680 con 0x55f3063f74f0 2024-09-13T18:30:24.318 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.316+0000 7ff9bbfff640 1 -- 172.21.15.114:0/3629088587 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff9d0098820 con 0x55f3063f74f0 2024-09-13T18:30:24.319 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.316+0000 7ff9e5e2b740 10 client.27251 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:30:24.319 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.316+0000 7ff9e5e2b740 10 client.27251 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:30:24.319 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.316+0000 7ff9e5e2b740 20 client.27251 populate_metadata read hostname 'smithi114' 2024-09-13T18:30:24.319 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.316+0000 7ff9e5e2b740 1 --2- 172.21.15.114:0/3629088587 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f306428da0 0x55f30642b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:24.319 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.316+0000 7ff9e5e2b740 4 client.27251 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:30:24.319 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.316+0000 7ff9e5e2b740 1 -- 172.21.15.114:0/3629088587 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f3063445d0 con 0x55f306428da0 2024-09-13T18:30:24.319 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.316+0000 7ff9e0ea4640 1 --2- 172.21.15.114:0/3629088587 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f306428da0 0x55f30642b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:24.319 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.317+0000 7ff9e0ea4640 1 --2- 172.21.15.114:0/3629088587 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f306428da0 0x55f30642b1c0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9100f35366cc70eb server_cookie=88431e283bd786e0 in_seq=0 out_seq=0 2024-09-13T18:30:24.320 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.317+0000 7ff9bbfff640 10 client.27251 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:30:24.322 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.320+0000 7ff9bbfff640 1 -- 172.21.15.114:0/3629088587 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55f3063445d0 con 0x55f306428da0 2024-09-13T18:30:24.323 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.320+0000 7ff9bbfff640 10 client.27251 handle_command_reply: tid=0 2024-09-13T18:30:24.323 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.320+0000 7ff9e5e2b740 1 client.27251 shutdown 2024-09-13T18:30:24.323 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.321+0000 7ff9e5e2b740 1 -- 172.21.15.114:0/3629088587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3063f74f0 msgr2=0x55f3063f45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:24.324 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.321+0000 7ff9e5e2b740 1 --2- 172.21.15.114:0/3629088587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3063f74f0 0x55f3063f45e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff9d000b3e0 tx=0x7ff9d001a7b0 comp rx=0 tx=0).stop 2024-09-13T18:30:24.324 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.321+0000 7ff9e5e2b740 1 -- 172.21.15.114:0/3629088587 shutdown_connections 2024-09-13T18:30:24.324 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.321+0000 7ff9e5e2b740 1 --2- 172.21.15.114:0/3629088587 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f306428da0 0x55f30642b1c0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:24.324 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.321+0000 7ff9e5e2b740 1 --2- 172.21.15.114:0/3629088587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3063f74f0 0x55f3063f45e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:24.324 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.321+0000 7ff9e5e2b740 1 --2- 172.21.15.114:0/3629088587 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f3063ecc50 0x55f3063f40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:24.324 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.321+0000 7ff9e5e2b740 1 --2- 172.21.15.114:0/3629088587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3063ec330 0x55f3063f6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:24.324 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.321+0000 7ff9e5e2b740 1 -- 172.21.15.114:0/3629088587 >> 172.21.15.114:0/3629088587 conn(0x55f3063e88c0 msgr2=0x55f3063e7b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:24.324 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.321+0000 7ff9bbfff640 0 client.27251 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:30:24.324 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.322+0000 7ff9e5e2b740 1 -- 172.21.15.114:0/3629088587 shutdown_connections 2024-09-13T18:30:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.322+0000 7ff9e5e2b740 1 -- 172.21.15.114:0/3629088587 wait complete. 2024-09-13T18:30:24.329 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.323+0000 7ff9e5e2b740 20 client.27251 trim_cache size 0 max 16384 2024-09-13T18:30:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.369+0000 7ff9e5e2b740 1 Processor -- start 2024-09-13T18:30:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.369+0000 7ff9e5e2b740 1 -- start start 2024-09-13T18:30:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.369+0000 7ff9e5e2b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f3063ec330 0x55f3063f6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.370+0000 7ff9e5e2b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3063ecc50 0x55f3063f2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.370+0000 7ff9e5e2b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3063f74f0 0x55f3063f2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.370+0000 7ff9e5e2b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3063f33d0 con 0x55f3063ec330 2024-09-13T18:30:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.370+0000 7ff9e5e2b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3063f5470 con 0x55f3063f74f0 2024-09-13T18:30:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.370+0000 7ff9e5e2b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3063f25d0 con 0x55f3063ecc50 2024-09-13T18:30:24.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.370+0000 7ff9dbfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3063ecc50 0x55f3063f2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:24.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.371+0000 7ff9e16a5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3063f74f0 0x55f3063f2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:24.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.371+0000 7ff9e16a5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3063f74f0 0x55f3063f2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:41692/0 (socket says 172.21.15.114:41692) 2024-09-13T18:30:24.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.371+0000 7ff9dbfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3063ecc50 0x55f3063f2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:33184/0 (socket says 172.21.15.114:33184) 2024-09-13T18:30:24.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.371+0000 7ff9e16a5640 1 -- 172.21.15.114:0/3742622332 learned_addr learned my addr 172.21.15.114:0/3742622332 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:24.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.373+0000 7ff9dbfff640 1 -- 172.21.15.114:0/3742622332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3063f74f0 msgr2=0x55f3063f2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:24.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.373+0000 7ff9dbfff640 1 --2- 172.21.15.114:0/3742622332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3063f74f0 0x55f3063f2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:24.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.373+0000 7ff9dbfff640 1 -- 172.21.15.114:0/3742622332 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f3063ec330 msgr2=0x55f3063f6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:24.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.373+0000 7ff9dbfff640 1 --2- 172.21.15.114:0/3742622332 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f3063ec330 0x55f3063f6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:24.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.373+0000 7ff9dbfff640 1 -- 172.21.15.114:0/3742622332 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3062f1300 con 0x55f3063ecc50 2024-09-13T18:30:24.378 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.374+0000 7ff9dbfff640 1 --2- 172.21.15.114:0/3742622332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3063ecc50 0x55f3063f2950 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x55f3063f3910 tx=0x7ff9d400e6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f908d2de00785fc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:24.378 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.376+0000 7ff9c37fe640 1 -- 172.21.15.114:0/3742622332 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d4009c60 con 0x55f3063ecc50 2024-09-13T18:30:24.378 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.376+0000 7ff9c37fe640 1 -- 172.21.15.114:0/3742622332 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff9d4009dc0 con 0x55f3063ecc50 2024-09-13T18:30:24.379 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.377+0000 7ff9e5e2b740 1 -- 172.21.15.114:0/3742622332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3063ecc50 msgr2=0x55f3063f2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:24.379 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.377+0000 7ff9e5e2b740 1 --2- 172.21.15.114:0/3742622332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3063ecc50 0x55f3063f2950 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x55f3063f3910 tx=0x7ff9d400e6c0 comp rx=0 tx=0).stop 2024-09-13T18:30:24.380 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.378+0000 7ff9e5e2b740 1 -- 172.21.15.114:0/3742622332 shutdown_connections 2024-09-13T18:30:24.380 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.378+0000 7ff9e5e2b740 1 --2- 172.21.15.114:0/3742622332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3063f74f0 0x55f3063f2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:24.380 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.378+0000 7ff9e5e2b740 1 --2- 172.21.15.114:0/3742622332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3063ecc50 0x55f3063f2950 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:24.380 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.378+0000 7ff9e5e2b740 1 --2- 172.21.15.114:0/3742622332 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f3063ec330 0x55f3063f6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:24.380 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.378+0000 7ff9e5e2b740 1 -- 172.21.15.114:0/3742622332 >> 172.21.15.114:0/3742622332 conn(0x55f3063e88c0 msgr2=0x55f3063ef370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:24.388 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.383+0000 7ff9e5e2b740 1 -- 172.21.15.114:0/3742622332 shutdown_connections 2024-09-13T18:30:24.390 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.387+0000 7ff9e5e2b740 1 -- 172.21.15.114:0/3742622332 wait complete. 2024-09-13T18:30:24.390 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.387+0000 7ff9e5e2b740 1 -- start start 2024-09-13T18:30:24.390 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.388+0000 7ff9e5e2b740 1 Processor -- start 2024-09-13T18:30:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.388+0000 7ff9e5e2b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f3063ec330 0x55f3063fe490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.388+0000 7ff9e5e2b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3063ecc50 0x55f3063fc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.388+0000 7ff9e5e2b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3063f74f0 0x55f3063fce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.388+0000 7ff9e5e2b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3063fd380 con 0x55f3063ec330 2024-09-13T18:30:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.388+0000 7ff9e5e2b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3063fd4f0 con 0x55f3063f74f0 2024-09-13T18:30:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.388+0000 7ff9e5e2b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3063fd6b0 con 0x55f3063ecc50 2024-09-13T18:30:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.388+0000 7ff9e5e2b740 10 client.0 _ll_register_callbacks cb 0x7ff9d0095c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:30:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.390+0000 7ff9dbfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3063ecc50 0x55f3063fc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.390+0000 7ff9e16a5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3063f74f0 0x55f3063fce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.390+0000 7ff9dbfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3063ecc50 0x55f3063fc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:33196/0 (socket says 172.21.15.114:33196) 2024-09-13T18:30:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.390+0000 7ff9dbfff640 1 -- 172.21.15.114:0/3679773522 learned_addr learned my addr 172.21.15.114:0/3679773522 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.390+0000 7ff9e0ea4640 1 --2- 172.21.15.114:0/3679773522 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f3063ec330 0x55f3063fe490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.392+0000 7ff9e16a5640 1 -- 172.21.15.114:0/3679773522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3063ecc50 msgr2=0x55f3063fc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.392+0000 7ff9e16a5640 1 --2- 172.21.15.114:0/3679773522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3063ecc50 0x55f3063fc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.392+0000 7ff9e16a5640 1 -- 172.21.15.114:0/3679773522 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f3063ec330 msgr2=0x55f3063fe490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.392+0000 7ff9e16a5640 1 --2- 172.21.15.114:0/3679773522 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f3063ec330 0x55f3063fe490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.392+0000 7ff9e16a5640 1 -- 172.21.15.114:0/3679773522 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9d4009880 con 0x55f3063f74f0 2024-09-13T18:30:24.401 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.399+0000 7ff9e16a5640 1 --2- 172.21.15.114:0/3679773522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3063f74f0 0x55f3063fce10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0095dc0 tx=0x7ff9d001a6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db9ae77c3f3f113d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:24.406 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.401+0000 7ff9bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:30:24.410 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.406+0000 7ff9bbfff640 1 -- 172.21.15.114:0/3679773522 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d00b6060 con 0x55f3063f74f0 2024-09-13T18:30:24.410 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.406+0000 7ff9bbfff640 1 -- 172.21.15.114:0/3679773522 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ff9d009d120 con 0x55f3063f74f0 2024-09-13T18:30:24.410 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.408+0000 7ff9e5e2b740 1 -- 172.21.15.114:0/3679773522 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff9d00a1d40 con 0x55f3063f74f0 2024-09-13T18:30:24.416 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.414+0000 7ff9bbfff640 1 -- 172.21.15.114:0/3679773522 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9d0098a30 con 0x55f3063f74f0 2024-09-13T18:30:24.417 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.415+0000 7ff9e5e2b740 10 client.27257 fetch_fsmap learned FSMap version 34 2024-09-13T18:30:24.417 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.415+0000 7ff9e5e2b740 1 -- 172.21.15.114:0/3679773522 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55f3063168e0 con 0x55f3063f74f0 2024-09-13T18:30:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.420+0000 7ff9bbfff640 1 -- 172.21.15.114:0/3679773522 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff9d00a1d40 con 0x55f3063f74f0 2024-09-13T18:30:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.420+0000 7ff9bbfff640 1 -- 172.21.15.114:0/3679773522 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff9d00b91a0 con 0x55f3063f74f0 2024-09-13T18:30:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.421+0000 7ff9e5e2b740 10 client.27257 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:30:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.421+0000 7ff9e5e2b740 10 client.27257 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:30:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.421+0000 7ff9e5e2b740 20 client.27257 populate_metadata read hostname 'smithi114' 2024-09-13T18:30:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.421+0000 7ff9e5e2b740 1 --2- 172.21.15.114:0/3679773522 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f306480390 0x55f306482780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.421+0000 7ff9e5e2b740 4 client.27257 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T18:30:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.421+0000 7ff9e5e2b740 1 -- 172.21.15.114:0/3679773522 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f306044300 con 0x55f306480390 2024-09-13T18:30:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.423+0000 7ff9e0ea4640 1 --2- 172.21.15.114:0/3679773522 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f306480390 0x55f306482780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.434+0000 7ff9e0ea4640 1 --2- 172.21.15.114:0/3679773522 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f306480390 0x55f306482780 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fc02b13a3992902 server_cookie=c9927fa43bad30a8 in_seq=0 out_seq=0 2024-09-13T18:30:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.434+0000 7ff9bbfff640 10 client.27257 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:30:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.453+0000 7ff9bbfff640 1 -- 172.21.15.114:0/3679773522 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f306044300 con 0x55f306480390 2024-09-13T18:30:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.453+0000 7ff9bbfff640 10 client.27257 handle_command_reply: tid=0 2024-09-13T18:30:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.453+0000 7ff9e5e2b740 1 client.27257 shutdown 2024-09-13T18:30:24.461 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.459+0000 7ff9e5e2b740 1 -- 172.21.15.114:0/3679773522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3063f74f0 msgr2=0x55f3063fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:24.461 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.459+0000 7ff9e5e2b740 1 --2- 172.21.15.114:0/3679773522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3063f74f0 0x55f3063fce10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0095dc0 tx=0x7ff9d001a6b0 comp rx=0 tx=0).stop 2024-09-13T18:30:24.465 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.463+0000 7ff9e5e2b740 1 -- 172.21.15.114:0/3679773522 shutdown_connections 2024-09-13T18:30:24.466 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.463+0000 7ff9e5e2b740 1 --2- 172.21.15.114:0/3679773522 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f306480390 0x55f306482780 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:24.466 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.463+0000 7ff9e5e2b740 1 --2- 172.21.15.114:0/3679773522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3063f74f0 0x55f3063fce10 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:24.466 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.463+0000 7ff9e5e2b740 1 --2- 172.21.15.114:0/3679773522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3063ecc50 0x55f3063fc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:24.466 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.463+0000 7ff9e5e2b740 1 --2- 172.21.15.114:0/3679773522 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f3063ec330 0x55f3063fe490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:24.466 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.463+0000 7ff9e5e2b740 1 -- 172.21.15.114:0/3679773522 >> 172.21.15.114:0/3679773522 conn(0x55f3063e88c0 msgr2=0x55f3063f27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:24.466 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.463+0000 7ff9bbfff640 0 client.27257 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:30:24.469 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.467+0000 7ff9e5e2b740 1 -- 172.21.15.114:0/3679773522 shutdown_connections 2024-09-13T18:30:24.471 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.469+0000 7ff9e5e2b740 1 -- 172.21.15.114:0/3679773522 wait complete. 2024-09-13T18:30:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.479+0000 7ff9e5e2b740 20 client.27257 trim_cache size 0 max 16384 2024-09-13T18:30:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-13T18:30:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-13T18:30:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-13T18:30:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-13T18:30:24.485 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.480+0000 7ff9e3108640 1 -- 172.21.15.114:0/4203235539 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff9c40a11e0 msgr2=0x7ff9c40a36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:24.485 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.480+0000 7ff9e3108640 1 --2- 172.21.15.114:0/4203235539 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff9c40a11e0 0x7ff9c40a36a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc00b230 tx=0x7ff9cc005de0 comp rx=0 tx=0).stop 2024-09-13T18:30:24.485 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.480+0000 7ff9e3108640 1 -- 172.21.15.114:0/4203235539 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9dc110c30 msgr2=0x7ff9dc106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:24.485 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.480+0000 7ff9e3108640 1 --2- 172.21.15.114:0/4203235539 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9dc110c30 0x7ff9dc106980 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7ff9d000b410 tx=0x7ff9d000b8d0 comp rx=0 tx=0).stop 2024-09-13T18:30:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.483+0000 7ff9e3108640 1 -- 172.21.15.114:0/4203235539 shutdown_connections 2024-09-13T18:30:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.483+0000 7ff9e3108640 1 --2- 172.21.15.114:0/4203235539 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff9c40a11e0 0x7ff9c40a36a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.483+0000 7ff9e3108640 1 --2- 172.21.15.114:0/4203235539 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9dc110c30 0x7ff9dc106980 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.483+0000 7ff9e3108640 1 --2- 172.21.15.114:0/4203235539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9dc1041a0 0x7ff9dc10bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.483+0000 7ff9e3108640 1 --2- 172.21.15.114:0/4203235539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9dc103880 0x7ff9dc10b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.483+0000 7ff9e3108640 1 -- 172.21.15.114:0/4203235539 >> 172.21.15.114:0/4203235539 conn(0x7ff9dc075380 msgr2=0x7ff9dc111760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.488+0000 7ff9e3108640 1 -- 172.21.15.114:0/4203235539 shutdown_connections 2024-09-13T18:30:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:24.488+0000 7ff9e3108640 1 -- 172.21.15.114:0/4203235539 wait complete. 2024-09-13T18:30:25.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:25 smithi204 ceph-mon[34777]: pgmap v245: 129 pgs: 129 active+clean; 22 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 33 MiB/s wr, 704 op/s 2024-09-13T18:30:25.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:25 smithi114 ceph-mon[30897]: pgmap v245: 129 pgs: 129 active+clean; 22 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 33 MiB/s wr, 704 op/s 2024-09-13T18:30:25.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:25 smithi134 ceph-mon[34283]: pgmap v245: 129 pgs: 129 active+clean; 22 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 33 MiB/s wr, 704 op/s 2024-09-13T18:30:27.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:27 smithi204 ceph-mon[34777]: pgmap v246: 129 pgs: 129 active+clean; 22 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 34 MiB/s wr, 710 op/s 2024-09-13T18:30:27.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:27 smithi114 ceph-mon[30897]: pgmap v246: 129 pgs: 129 active+clean; 22 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 34 MiB/s wr, 710 op/s 2024-09-13T18:30:27.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:27 smithi134 ceph-mon[34283]: pgmap v246: 129 pgs: 129 active+clean; 22 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 34 MiB/s wr, 710 op/s 2024-09-13T18:30:29.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:29 smithi114 ceph-mon[30897]: pgmap v247: 129 pgs: 129 active+clean; 22 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 34 MiB/s wr, 710 op/s 2024-09-13T18:30:29.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:29 smithi134 ceph-mon[34283]: pgmap v247: 129 pgs: 129 active+clean; 22 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 34 MiB/s wr, 710 op/s 2024-09-13T18:30:29.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:29 smithi204 ceph-mon[34777]: pgmap v247: 129 pgs: 129 active+clean; 22 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 34 MiB/s wr, 710 op/s 2024-09-13T18:30:31.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:31 smithi114 ceph-mon[30897]: pgmap v248: 129 pgs: 129 active+clean; 22 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 43 MiB/s wr, 910 op/s 2024-09-13T18:30:31.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:31 smithi134 ceph-mon[34283]: pgmap v248: 129 pgs: 129 active+clean; 22 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 43 MiB/s wr, 910 op/s 2024-09-13T18:30:31.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:31 smithi204 ceph-mon[34777]: pgmap v248: 129 pgs: 129 active+clean; 22 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 43 MiB/s wr, 910 op/s 2024-09-13T18:30:33.723 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:33 smithi114 ceph-mon[30897]: pgmap v249: 129 pgs: 129 active+clean; 22 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 29 MiB/s wr, 604 op/s 2024-09-13T18:30:33.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:33 smithi134 ceph-mon[34283]: pgmap v249: 129 pgs: 129 active+clean; 22 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 29 MiB/s wr, 604 op/s 2024-09-13T18:30:33.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:33 smithi204 ceph-mon[34777]: pgmap v249: 129 pgs: 129 active+clean; 22 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 29 MiB/s wr, 604 op/s 2024-09-13T18:30:34.067 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T18:30:34.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.293+0000 7ff265faa640 1 -- 172.21.15.114:0/570257922 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff2600fed30 msgr2=0x7ff2600ff110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:34.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.293+0000 7ff265faa640 1 --2- 172.21.15.114:0/570257922 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff2600fed30 0x7ff2600ff110 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7ff250009900 tx=0x7ff25002a480 comp rx=0 tx=0).stop 2024-09-13T18:30:34.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.293+0000 7ff265faa640 1 -- 172.21.15.114:0/570257922 shutdown_connections 2024-09-13T18:30:34.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.293+0000 7ff265faa640 1 --2- 172.21.15.114:0/570257922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff260104160 0x7ff2601045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:34.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.293+0000 7ff265faa640 1 --2- 172.21.15.114:0/570257922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2600ff650 0x7ff260103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:34.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.293+0000 7ff265faa640 1 --2- 172.21.15.114:0/570257922 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff2600fed30 0x7ff2600ff110 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:34.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.293+0000 7ff265faa640 1 -- 172.21.15.114:0/570257922 >> 172.21.15.114:0/570257922 conn(0x7ff260075c60 msgr2=0x7ff2600fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:34.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.293+0000 7ff265faa640 1 -- 172.21.15.114:0/570257922 shutdown_connections 2024-09-13T18:30:34.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.293+0000 7ff265faa640 1 -- 172.21.15.114:0/570257922 wait complete. 2024-09-13T18:30:34.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.294+0000 7ff265faa640 1 Processor -- start 2024-09-13T18:30:34.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.294+0000 7ff265faa640 1 -- start start 2024-09-13T18:30:34.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.294+0000 7ff265faa640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2600fed30 0x7ff26019c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:34.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.294+0000 7ff265faa640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2600ff650 0x7ff26019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:34.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.294+0000 7ff265faa640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff260104160 0x7ff26019ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:34.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.294+0000 7ff265faa640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2601a05d0 con 0x7ff260104160 2024-09-13T18:30:34.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.294+0000 7ff265faa640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2601a0740 con 0x7ff2600ff650 2024-09-13T18:30:34.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.294+0000 7ff265faa640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2601a0900 con 0x7ff2600fed30 2024-09-13T18:30:34.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.294+0000 7ff25ffff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff260104160 0x7ff26019ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:34.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.294+0000 7ff25ffff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff260104160 0x7ff26019ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57728/0 (socket says 172.21.15.114:57728) 2024-09-13T18:30:34.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.294+0000 7ff25ffff640 1 -- 172.21.15.114:0/2858358520 learned_addr learned my addr 172.21.15.114:0/2858358520 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:34.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.294+0000 7ff25effd640 1 --2- 172.21.15.114:0/2858358520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2600ff650 0x7ff26019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:34.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.294+0000 7ff25f7fe640 1 --2- 172.21.15.114:0/2858358520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2600fed30 0x7ff26019c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:34.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.294+0000 7ff25ffff640 1 -- 172.21.15.114:0/2858358520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2600fed30 msgr2=0x7ff26019c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:34.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.294+0000 7ff25ffff640 1 --2- 172.21.15.114:0/2858358520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2600fed30 0x7ff26019c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:34.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.294+0000 7ff25ffff640 1 -- 172.21.15.114:0/2858358520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2600ff650 msgr2=0x7ff26019cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:34.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.294+0000 7ff25ffff640 1 --2- 172.21.15.114:0/2858358520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2600ff650 0x7ff26019cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:34.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.294+0000 7ff25ffff640 1 -- 172.21.15.114:0/2858358520 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff250009560 con 0x7ff260104160 2024-09-13T18:30:34.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.294+0000 7ff25effd640 1 --2- 172.21.15.114:0/2858358520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2600ff650 0x7ff26019cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:30:34.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.295+0000 7ff25ffff640 1 --2- 172.21.15.114:0/2858358520 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff260104160 0x7ff26019ffd0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7ff25800ea60 tx=0x7ff25800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:34.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.295+0000 7ff25cff9640 1 -- 172.21.15.114:0/2858358520 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25800ce40 con 0x7ff260104160 2024-09-13T18:30:34.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.295+0000 7ff25cff9640 1 -- 172.21.15.114:0/2858358520 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff258005020 con 0x7ff260104160 2024-09-13T18:30:34.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.295+0000 7ff265faa640 1 -- 172.21.15.114:0/2858358520 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2601a0be0 con 0x7ff260104160 2024-09-13T18:30:34.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.295+0000 7ff25cff9640 1 -- 172.21.15.114:0/2858358520 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff258010690 con 0x7ff260104160 2024-09-13T18:30:34.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.295+0000 7ff265faa640 1 -- 172.21.15.114:0/2858358520 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2601a9540 con 0x7ff260104160 2024-09-13T18:30:34.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.297+0000 7ff25cff9640 1 -- 172.21.15.114:0/2858358520 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7ff258005190 con 0x7ff260104160 2024-09-13T18:30:34.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.297+0000 7ff268ccb740 1 Processor -- start 2024-09-13T18:30:34.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.297+0000 7ff268ccb740 1 -- start start 2024-09-13T18:30:34.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.298+0000 7ff268ccb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d04375e10 0x564d043761f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:34.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.298+0000 7ff268ccb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d04376730 0x564d04380a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:34.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.298+0000 7ff268ccb740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564d04380fd0 0x564d043833c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:34.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.298+0000 7ff268ccb740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d042e0f20 con 0x564d04380fd0 2024-09-13T18:30:34.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.298+0000 7ff268ccb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d042de6b0 con 0x564d04375e10 2024-09-13T18:30:34.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.298+0000 7ff268ccb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d042d9fa0 con 0x564d04376730 2024-09-13T18:30:34.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.298+0000 7ff25cff9640 1 --2- 172.21.15.114:0/2858358520 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff24c0a1280 0x7ff24c0a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:34.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.298+0000 7ff25effd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564d04380fd0 0x564d043833c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:34.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.298+0000 7ff25f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d04376730 0x564d04380a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:34.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.298+0000 7ff25effd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564d04380fd0 0x564d043833c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57740/0 (socket says 172.21.15.114:57740) 2024-09-13T18:30:34.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.298+0000 7ff25f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d04376730 0x564d04380a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:58700/0 (socket says 172.21.15.114:58700) 2024-09-13T18:30:34.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.298+0000 7ff25effd640 1 -- 172.21.15.114:0/3357579384 learned_addr learned my addr 172.21.15.114:0/3357579384 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:34.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.298+0000 7ff25effd640 1 --2- 172.21.15.114:0/3357579384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d04375e10 0x564d043761f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:34.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.298+0000 7ff25cff9640 1 -- 172.21.15.114:0/2858358520 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ff258014070 con 0x7ff260104160 2024-09-13T18:30:34.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.298+0000 7ff25ffff640 1 --2- 172.21.15.114:0/2858358520 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff24c0a1280 0x7ff24c0a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:34.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.298+0000 7ff25effd640 1 -- 172.21.15.114:0/3357579384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d04376730 msgr2=0x564d04380a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:34.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.298+0000 7ff25effd640 1 --2- 172.21.15.114:0/3357579384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d04376730 0x564d04380a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:34.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.298+0000 7ff25effd640 1 -- 172.21.15.114:0/3357579384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d04375e10 msgr2=0x564d043761f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:34.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.298+0000 7ff25effd640 1 --2- 172.21.15.114:0/3357579384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d04375e10 0x564d043761f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:34.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.298+0000 7ff25effd640 1 -- 172.21.15.114:0/3357579384 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d04294840 con 0x564d04380fd0 2024-09-13T18:30:34.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.298+0000 7ff25effd640 1 --2- 172.21.15.114:0/3357579384 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564d04380fd0 0x564d043833c0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7ff2480149a0 tx=0x7ff24800b4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fad674c0bfc05639 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:34.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.299+0000 7ff255ffb640 1 -- 172.21.15.114:0/3357579384 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2480142d0 con 0x564d04380fd0 2024-09-13T18:30:34.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.299+0000 7ff25ffff640 1 --2- 172.21.15.114:0/2858358520 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff24c0a1280 0x7ff24c0a3740 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff2580a2350 tx=0x7ff2580099a0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:34.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.299+0000 7ff255ffb640 1 -- 172.21.15.114:0/3357579384 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff24801a030 con 0x564d04380fd0 2024-09-13T18:30:34.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.299+0000 7ff255ffb640 1 -- 172.21.15.114:0/3357579384 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff248004a00 con 0x564d04380fd0 2024-09-13T18:30:34.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.299+0000 7ff268ccb740 1 -- 172.21.15.114:0/3357579384 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564d04380fd0 msgr2=0x564d043833c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:34.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.299+0000 7ff268ccb740 1 --2- 172.21.15.114:0/3357579384 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564d04380fd0 0x564d043833c0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7ff2480149a0 tx=0x7ff24800b4e0 comp rx=0 tx=0).stop 2024-09-13T18:30:34.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.299+0000 7ff268ccb740 1 -- 172.21.15.114:0/3357579384 shutdown_connections 2024-09-13T18:30:34.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.299+0000 7ff268ccb740 1 --2- 172.21.15.114:0/3357579384 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564d04380fd0 0x564d043833c0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:34.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.299+0000 7ff268ccb740 1 --2- 172.21.15.114:0/3357579384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d04376730 0x564d04380a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:34.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.299+0000 7ff268ccb740 1 --2- 172.21.15.114:0/3357579384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d04375e10 0x564d043761f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:34.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.299+0000 7ff268ccb740 1 -- 172.21.15.114:0/3357579384 >> 172.21.15.114:0/3357579384 conn(0x564d04372380 msgr2=0x564d04372760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:34.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.299+0000 7ff268ccb740 1 -- 172.21.15.114:0/3357579384 shutdown_connections 2024-09-13T18:30:34.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.299+0000 7ff268ccb740 1 -- 172.21.15.114:0/3357579384 wait complete. 2024-09-13T18:30:34.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.299+0000 7ff268ccb740 1 -- start start 2024-09-13T18:30:34.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.300+0000 7ff268ccb740 1 Processor -- start 2024-09-13T18:30:34.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.300+0000 7ff268ccb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d04375e10 0x564d043809c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:34.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.300+0000 7ff268ccb740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564d04376730 0x564d0437dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:34.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.300+0000 7ff268ccb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d04380fd0 0x564d0437e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:34.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.300+0000 7ff268ccb740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d0427b2a0 con 0x564d04376730 2024-09-13T18:30:34.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.300+0000 7ff268ccb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d03fb2430 con 0x564d04375e10 2024-09-13T18:30:34.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.300+0000 7ff268ccb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d0425d680 con 0x564d04380fd0 2024-09-13T18:30:34.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.300+0000 7ff268ccb740 10 client.0 _ll_register_callbacks cb 0x564d042d3610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:30:34.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.300+0000 7ff25f7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564d04376730 0x564d0437dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:34.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.300+0000 7ff25f7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564d04376730 0x564d0437dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57746/0 (socket says 172.21.15.114:57746) 2024-09-13T18:30:34.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.300+0000 7ff25f7fe640 1 -- 172.21.15.114:0/4146925461 learned_addr learned my addr 172.21.15.114:0/4146925461 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:34.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.300+0000 7ff25effd640 1 --2- 172.21.15.114:0/4146925461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d04375e10 0x564d043809c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:34.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.300+0000 7ff25effd640 1 --2- 172.21.15.114:0/4146925461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d04380fd0 0x564d0437e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:34.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.300+0000 7ff25f7fe640 1 -- 172.21.15.114:0/4146925461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d04380fd0 msgr2=0x564d0437e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:34.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.300+0000 7ff25f7fe640 1 --2- 172.21.15.114:0/4146925461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d04380fd0 0x564d0437e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:34.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.300+0000 7ff25f7fe640 1 -- 172.21.15.114:0/4146925461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d04375e10 msgr2=0x564d043809c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:34.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.300+0000 7ff25f7fe640 1 --2- 172.21.15.114:0/4146925461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d04375e10 0x564d043809c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:34.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.300+0000 7ff25f7fe640 1 -- 172.21.15.114:0/4146925461 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff248014680 con 0x564d04376730 2024-09-13T18:30:34.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.300+0000 7ff25effd640 1 --2- 172.21.15.114:0/4146925461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d04380fd0 0x564d0437e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:30:34.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.301+0000 7ff25f7fe640 1 --2- 172.21.15.114:0/4146925461 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564d04376730 0x564d0437dc90 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7ff250005940 tx=0x7ff250004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=314708cc1809a4d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:34.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.301+0000 7ff23effd640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:30:34.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.301+0000 7ff23effd640 1 -- 172.21.15.114:0/4146925461 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff250031960 con 0x564d04376730 2024-09-13T18:30:34.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.301+0000 7ff23effd640 1 -- 172.21.15.114:0/4146925461 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ff250031ac0 con 0x564d04376730 2024-09-13T18:30:34.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.301+0000 7ff268ccb740 1 -- 172.21.15.114:0/4146925461 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564d03fcd4e0 con 0x564d04376730 2024-09-13T18:30:34.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.301+0000 7ff23effd640 1 -- 172.21.15.114:0/4146925461 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff250031c80 con 0x564d04376730 2024-09-13T18:30:34.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.301+0000 7ff268ccb740 10 client.17835 fetch_fsmap learned FSMap version 34 2024-09-13T18:30:34.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.301+0000 7ff268ccb740 1 -- 172.21.15.114:0/4146925461 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x564d042d8620 con 0x564d04376730 2024-09-13T18:30:34.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.301+0000 7ff23effd640 1 -- 172.21.15.114:0/4146925461 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564d03fcd4e0 con 0x564d04376730 2024-09-13T18:30:34.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.301+0000 7ff23effd640 1 -- 172.21.15.114:0/4146925461 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff250046470 con 0x564d04376730 2024-09-13T18:30:34.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.301+0000 7ff268ccb740 10 client.17835 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:30:34.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.301+0000 7ff268ccb740 10 client.17835 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:30:34.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.301+0000 7ff268ccb740 20 client.17835 populate_metadata read hostname 'smithi114' 2024-09-13T18:30:34.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.301+0000 7ff268ccb740 1 --2- 172.21.15.114:0/4146925461 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x564d04390dc0 0x564d043931e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:34.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.301+0000 7ff268ccb740 4 client.17835 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:30:34.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.301+0000 7ff268ccb740 1 -- 172.21.15.114:0/4146925461 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564d03fde7e0 con 0x564d04390dc0 2024-09-13T18:30:34.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.301+0000 7ff25effd640 1 --2- 172.21.15.114:0/4146925461 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x564d04390dc0 0x564d043931e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:34.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.302+0000 7ff23effd640 10 client.17835 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:30:34.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.302+0000 7ff25effd640 1 --2- 172.21.15.114:0/4146925461 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x564d04390dc0 0x564d043931e0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9daca3286b67507c server_cookie=4511e1a39da33135 in_seq=0 out_seq=0 2024-09-13T18:30:34.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.302+0000 7ff23effd640 1 -- 172.21.15.114:0/4146925461 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x564d03fde7e0 con 0x564d04390dc0 2024-09-13T18:30:34.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.302+0000 7ff23effd640 10 client.17835 handle_command_reply: tid=0 2024-09-13T18:30:34.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.302+0000 7ff268ccb740 1 client.17835 shutdown 2024-09-13T18:30:34.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.302+0000 7ff268ccb740 1 -- 172.21.15.114:0/4146925461 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564d04376730 msgr2=0x564d0437dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:34.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.302+0000 7ff268ccb740 1 --2- 172.21.15.114:0/4146925461 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564d04376730 0x564d0437dc90 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7ff250005940 tx=0x7ff250004270 comp rx=0 tx=0).stop 2024-09-13T18:30:34.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.303+0000 7ff268ccb740 1 -- 172.21.15.114:0/4146925461 shutdown_connections 2024-09-13T18:30:34.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.303+0000 7ff268ccb740 1 --2- 172.21.15.114:0/4146925461 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x564d04390dc0 0x564d043931e0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:34.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.303+0000 7ff268ccb740 1 --2- 172.21.15.114:0/4146925461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d04380fd0 0x564d0437e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:34.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.303+0000 7ff268ccb740 1 --2- 172.21.15.114:0/4146925461 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564d04376730 0x564d0437dc90 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:34.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.303+0000 7ff23effd640 0 client.17835 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:30:34.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.303+0000 7ff268ccb740 1 --2- 172.21.15.114:0/4146925461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d04375e10 0x564d043809c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:34.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.303+0000 7ff268ccb740 1 -- 172.21.15.114:0/4146925461 >> 172.21.15.114:0/4146925461 conn(0x564d04372380 msgr2=0x564d04378090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:34.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.303+0000 7ff268ccb740 1 -- 172.21.15.114:0/4146925461 shutdown_connections 2024-09-13T18:30:34.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.303+0000 7ff268ccb740 1 -- 172.21.15.114:0/4146925461 wait complete. 2024-09-13T18:30:34.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.303+0000 7ff268ccb740 20 client.17835 trim_cache size 0 max 16384 2024-09-13T18:30:34.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.334+0000 7ff268ccb740 1 Processor -- start 2024-09-13T18:30:34.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.335+0000 7ff268ccb740 1 -- start start 2024-09-13T18:30:34.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.335+0000 7ff268ccb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d04372380 0x564d0438c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:34.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.335+0000 7ff268ccb740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564d04375e10 0x564d0437c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:34.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.335+0000 7ff268ccb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d04376730 0x564d0437ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:34.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.335+0000 7ff268ccb740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d04373010 con 0x564d04375e10 2024-09-13T18:30:34.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.336+0000 7ff268ccb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d0437cfc0 con 0x564d04376730 2024-09-13T18:30:34.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.336+0000 7ff268ccb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d043732e0 con 0x564d04372380 2024-09-13T18:30:34.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.337+0000 7ff25effd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d04376730 0x564d0437ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:34.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.337+0000 7ff25effd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d04372380 0x564d0438c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:34.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.337+0000 7ff25effd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d04372380 0x564d0438c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:58730/0 (socket says 172.21.15.114:58730) 2024-09-13T18:30:34.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.337+0000 7ff25effd640 1 -- 172.21.15.114:0/3315843101 learned_addr learned my addr 172.21.15.114:0/3315843101 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:34.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.338+0000 7ff25f7fe640 1 --2- 172.21.15.114:0/3315843101 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564d04375e10 0x564d0437c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:34.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.338+0000 7ff25effd640 1 -- 172.21.15.114:0/3315843101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d04372380 msgr2=0x564d0438c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:34.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.338+0000 7ff25effd640 1 --2- 172.21.15.114:0/3315843101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d04372380 0x564d0438c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:34.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.338+0000 7ff25effd640 1 -- 172.21.15.114:0/3315843101 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564d04375e10 msgr2=0x564d0437c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:34.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.338+0000 7ff25effd640 1 --2- 172.21.15.114:0/3315843101 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564d04375e10 0x564d0437c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:34.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.338+0000 7ff25effd640 1 -- 172.21.15.114:0/3315843101 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d0427a790 con 0x564d04376730 2024-09-13T18:30:34.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.339+0000 7ff25f7fe640 1 --2- 172.21.15.114:0/3315843101 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564d04375e10 0x564d0437c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:30:34.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.339+0000 7ff25effd640 1 --2- 172.21.15.114:0/3315843101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d04376730 0x564d0437ca80 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff248014970 tx=0x7ff248005450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cda2ee34d1015dbe server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:34.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.345+0000 7ff2567fc640 1 -- 172.21.15.114:0/3315843101 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff248026040 con 0x564d04376730 2024-09-13T18:30:34.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.345+0000 7ff2567fc640 1 -- 172.21.15.114:0/3315843101 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff248005670 con 0x564d04376730 2024-09-13T18:30:34.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.346+0000 7ff268ccb740 1 -- 172.21.15.114:0/3315843101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d04376730 msgr2=0x564d0437ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:34.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.346+0000 7ff268ccb740 1 --2- 172.21.15.114:0/3315843101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d04376730 0x564d0437ca80 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff248014970 tx=0x7ff248005450 comp rx=0 tx=0).stop 2024-09-13T18:30:34.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.350+0000 7ff268ccb740 1 -- 172.21.15.114:0/3315843101 shutdown_connections 2024-09-13T18:30:34.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.350+0000 7ff268ccb740 1 --2- 172.21.15.114:0/3315843101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d04376730 0x564d0437ca80 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:34.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.350+0000 7ff268ccb740 1 --2- 172.21.15.114:0/3315843101 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564d04375e10 0x564d0437c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:34.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.350+0000 7ff268ccb740 1 --2- 172.21.15.114:0/3315843101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d04372380 0x564d0438c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:34.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.350+0000 7ff268ccb740 1 -- 172.21.15.114:0/3315843101 >> 172.21.15.114:0/3315843101 conn(0x564d04380b20 msgr2=0x564d04378f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:34.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.355+0000 7ff268ccb740 1 -- 172.21.15.114:0/3315843101 shutdown_connections 2024-09-13T18:30:34.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.355+0000 7ff268ccb740 1 -- 172.21.15.114:0/3315843101 wait complete. 2024-09-13T18:30:34.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.355+0000 7ff268ccb740 1 -- start start 2024-09-13T18:30:34.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.356+0000 7ff268ccb740 1 Processor -- start 2024-09-13T18:30:34.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.358+0000 7ff268ccb740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564d04372380 0x564d043847d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:34.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.358+0000 7ff268ccb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d04375e10 0x564d04384d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:34.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.358+0000 7ff268ccb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d04376730 0x564d04385250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:34.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.358+0000 7ff268ccb740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d043e72c0 con 0x564d04372380 2024-09-13T18:30:34.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.358+0000 7ff268ccb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d043e7400 con 0x564d04375e10 2024-09-13T18:30:34.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.358+0000 7ff268ccb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d043f85a0 con 0x564d04376730 2024-09-13T18:30:34.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.358+0000 7ff268ccb740 10 client.0 _ll_register_callbacks cb 0x7ff250009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:30:34.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.358+0000 7ff25effd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564d04372380 0x564d043847d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:34.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.359+0000 7ff25effd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564d04372380 0x564d043847d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57764/0 (socket says 172.21.15.114:57764) 2024-09-13T18:30:34.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.359+0000 7ff25effd640 1 -- 172.21.15.114:0/1895368527 learned_addr learned my addr 172.21.15.114:0/1895368527 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:34.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.359+0000 7ff25effd640 1 --2- 172.21.15.114:0/1895368527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d04376730 0x564d04385250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:34.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.359+0000 7ff25effd640 1 -- 172.21.15.114:0/1895368527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d04376730 msgr2=0x564d04385250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:34.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.359+0000 7ff25effd640 1 --2- 172.21.15.114:0/1895368527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d04376730 0x564d04385250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:34.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.359+0000 7ff25effd640 1 -- 172.21.15.114:0/1895368527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d04375e10 msgr2=0x564d04384d10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:30:34.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.359+0000 7ff25effd640 1 --2- 172.21.15.114:0/1895368527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d04375e10 0x564d04384d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:34.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.359+0000 7ff25effd640 1 -- 172.21.15.114:0/1895368527 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff248032040 con 0x564d04372380 2024-09-13T18:30:34.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.368+0000 7ff25effd640 1 --2- 172.21.15.114:0/1895368527 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564d04372380 0x564d043847d0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7ff248005bc0 tx=0x7ff248005540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a41f2676e1b2774a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:34.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.369+0000 7ff23effd640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:30:34.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.373+0000 7ff23effd640 1 -- 172.21.15.114:0/1895368527 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff248026070 con 0x564d04372380 2024-09-13T18:30:34.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.373+0000 7ff23effd640 1 -- 172.21.15.114:0/1895368527 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ff248000e40 con 0x564d04372380 2024-09-13T18:30:34.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.377+0000 7ff268ccb740 1 -- 172.21.15.114:0/1895368527 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564d04031e30 con 0x564d04372380 2024-09-13T18:30:34.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.390+0000 7ff23effd640 1 -- 172.21.15.114:0/1895368527 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff24800fc60 con 0x564d04372380 2024-09-13T18:30:34.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.392+0000 7ff268ccb740 10 client.17838 fetch_fsmap learned FSMap version 34 2024-09-13T18:30:34.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.392+0000 7ff268ccb740 1 -- 172.21.15.114:0/1895368527 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x564d0429fea0 con 0x564d04372380 2024-09-13T18:30:34.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.403+0000 7ff23effd640 1 -- 172.21.15.114:0/1895368527 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564d04031e30 con 0x564d04372380 2024-09-13T18:30:34.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.403+0000 7ff23effd640 1 -- 172.21.15.114:0/1895368527 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff2480035f0 con 0x564d04372380 2024-09-13T18:30:34.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.403+0000 7ff268ccb740 10 client.17838 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:30:34.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.403+0000 7ff268ccb740 10 client.17838 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:30:34.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.403+0000 7ff268ccb740 20 client.17838 populate_metadata read hostname 'smithi114' 2024-09-13T18:30:34.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.403+0000 7ff268ccb740 1 --2- 172.21.15.114:0/1895368527 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x564d04380fd0 0x564d043fb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:34.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.403+0000 7ff268ccb740 4 client.17838 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T18:30:34.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.403+0000 7ff268ccb740 1 -- 172.21.15.114:0/1895368527 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564d043fc0d0 con 0x564d04380fd0 2024-09-13T18:30:34.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.414+0000 7ff25f7fe640 1 --2- 172.21.15.114:0/1895368527 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x564d04380fd0 0x564d043fb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:34.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.433+0000 7ff25f7fe640 1 --2- 172.21.15.114:0/1895368527 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x564d04380fd0 0x564d043fb880 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cee72e721b9b27ac server_cookie=61427d58a35e3117 in_seq=0 out_seq=0 2024-09-13T18:30:34.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.433+0000 7ff23effd640 10 client.17838 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:30:34.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.435+0000 7ff23effd640 1 -- 172.21.15.114:0/1895368527 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x564d043fc0d0 con 0x564d04380fd0 2024-09-13T18:30:34.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.435+0000 7ff23effd640 10 client.17838 handle_command_reply: tid=0 2024-09-13T18:30:34.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.435+0000 7ff268ccb740 1 client.17838 shutdown 2024-09-13T18:30:34.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.437+0000 7ff268ccb740 1 -- 172.21.15.114:0/1895368527 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564d04372380 msgr2=0x564d043847d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:34.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.437+0000 7ff268ccb740 1 --2- 172.21.15.114:0/1895368527 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564d04372380 0x564d043847d0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7ff248005bc0 tx=0x7ff248005540 comp rx=0 tx=0).stop 2024-09-13T18:30:34.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.438+0000 7ff268ccb740 1 -- 172.21.15.114:0/1895368527 shutdown_connections 2024-09-13T18:30:34.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.438+0000 7ff268ccb740 1 --2- 172.21.15.114:0/1895368527 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x564d04380fd0 0x564d043fb880 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:34.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.438+0000 7ff268ccb740 1 --2- 172.21.15.114:0/1895368527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d04376730 0x564d04385250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:34.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.438+0000 7ff268ccb740 1 --2- 172.21.15.114:0/1895368527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d04375e10 0x564d04384d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:34.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.438+0000 7ff268ccb740 1 --2- 172.21.15.114:0/1895368527 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564d04372380 0x564d043847d0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:34.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.438+0000 7ff268ccb740 1 -- 172.21.15.114:0/1895368527 >> 172.21.15.114:0/1895368527 conn(0x564d04380b20 msgr2=0x564d04378f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:34.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.438+0000 7ff23effd640 0 client.17838 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:30:34.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.438+0000 7ff268ccb740 1 -- 172.21.15.114:0/1895368527 shutdown_connections 2024-09-13T18:30:34.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.438+0000 7ff268ccb740 1 -- 172.21.15.114:0/1895368527 wait complete. 2024-09-13T18:30:34.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.439+0000 7ff268ccb740 20 client.17838 trim_cache size 0 max 16384 2024-09-13T18:30:34.442 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:30:34.442 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "no active scrubs running", 2024-09-13T18:30:34.442 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": {} 2024-09-13T18:30:34.442 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:30:34.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.447+0000 7ff265faa640 1 -- 172.21.15.114:0/2858358520 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff24c0a1280 msgr2=0x7ff24c0a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:34.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.447+0000 7ff265faa640 1 --2- 172.21.15.114:0/2858358520 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff24c0a1280 0x7ff24c0a3740 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff2580a2350 tx=0x7ff2580099a0 comp rx=0 tx=0).stop 2024-09-13T18:30:34.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.447+0000 7ff265faa640 1 -- 172.21.15.114:0/2858358520 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff260104160 msgr2=0x7ff26019ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:34.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.447+0000 7ff265faa640 1 --2- 172.21.15.114:0/2858358520 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff260104160 0x7ff26019ffd0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7ff25800ea60 tx=0x7ff25800ef20 comp rx=0 tx=0).stop 2024-09-13T18:30:34.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.447+0000 7ff265faa640 1 -- 172.21.15.114:0/2858358520 shutdown_connections 2024-09-13T18:30:34.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.447+0000 7ff265faa640 1 --2- 172.21.15.114:0/2858358520 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff24c0a1280 0x7ff24c0a3740 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:34.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.447+0000 7ff265faa640 1 --2- 172.21.15.114:0/2858358520 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff260104160 0x7ff26019ffd0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:34.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.447+0000 7ff265faa640 1 --2- 172.21.15.114:0/2858358520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2600ff650 0x7ff26019cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:34.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.447+0000 7ff265faa640 1 --2- 172.21.15.114:0/2858358520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2600fed30 0x7ff26019c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:34.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.447+0000 7ff265faa640 1 -- 172.21.15.114:0/2858358520 >> 172.21.15.114:0/2858358520 conn(0x7ff260075c60 msgr2=0x7ff26010d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:34.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.452+0000 7ff265faa640 1 -- 172.21.15.114:0/2858358520 shutdown_connections 2024-09-13T18:30:34.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.454+0000 7ff265faa640 1 -- 172.21.15.114:0/2858358520 wait complete. 2024-09-13T18:30:34.487 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-13T18:30:34.487 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:30:34.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.743+0000 7f1efea41640 1 -- 172.21.15.114:0/338820824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ef8107e60 msgr2=0x7f1ef80766b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:34.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.743+0000 7f1efea41640 1 --2- 172.21.15.114:0/338820824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ef8107e60 0x7f1ef80766b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0009900 tx=0x7f1ee002a4a0 comp rx=0 tx=0).stop 2024-09-13T18:30:34.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.747+0000 7f1efea41640 1 -- 172.21.15.114:0/338820824 shutdown_connections 2024-09-13T18:30:34.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.747+0000 7f1efea41640 1 --2- 172.21.15.114:0/338820824 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ef8076bf0 0x7f1ef80790b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:34.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.747+0000 7f1efea41640 1 --2- 172.21.15.114:0/338820824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ef8107e60 0x7f1ef80766b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:34.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.747+0000 7f1efea41640 1 --2- 172.21.15.114:0/338820824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ef8107540 0x7f1ef8107920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:34.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.747+0000 7f1efea41640 1 -- 172.21.15.114:0/338820824 >> 172.21.15.114:0/338820824 conn(0x7f1ef806aff0 msgr2=0x7f1ef806b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:34.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.749+0000 7f1efea41640 1 -- 172.21.15.114:0/338820824 shutdown_connections 2024-09-13T18:30:34.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.752+0000 7f1efea41640 1 -- 172.21.15.114:0/338820824 wait complete. 2024-09-13T18:30:34.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.753+0000 7f1efea41640 1 Processor -- start 2024-09-13T18:30:34.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.753+0000 7f1efea41640 1 -- start start 2024-09-13T18:30:34.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.753+0000 7f1efea41640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ef8076bf0 0x7f1ef8082c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:34.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.753+0000 7f1efea41640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ef8107540 0x7f1ef8083190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:34.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.753+0000 7f1efea41640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ef8107e60 0x7f1ef807ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:34.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.753+0000 7f1efea41640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef807f5c0 con 0x7f1ef8076bf0 2024-09-13T18:30:34.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.753+0000 7f1efea41640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef807f730 con 0x7f1ef8107540 2024-09-13T18:30:34.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.753+0000 7f1efea41640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef807f8f0 con 0x7f1ef8107e60 2024-09-13T18:30:34.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.754+0000 7f1ef7fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ef8076bf0 0x7f1ef8082c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:34.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.754+0000 7f1ef7fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ef8076bf0 0x7f1ef8082c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57786/0 (socket says 172.21.15.114:57786) 2024-09-13T18:30:34.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.754+0000 7f1ef7fff640 1 -- 172.21.15.114:0/133224428 learned_addr learned my addr 172.21.15.114:0/133224428 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:34.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.754+0000 7f1efcfde640 1 --2- 172.21.15.114:0/133224428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ef8107e60 0x7f1ef807ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:34.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.754+0000 7f1ef7fff640 1 -- 172.21.15.114:0/133224428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ef8107e60 msgr2=0x7f1ef807ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:34.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.754+0000 7f1ef7fff640 1 --2- 172.21.15.114:0/133224428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ef8107e60 0x7f1ef807ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:34.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.754+0000 7f1ef7fff640 1 -- 172.21.15.114:0/133224428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ef8107540 msgr2=0x7f1ef8083190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:34.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.754+0000 7f1ef7fff640 1 --2- 172.21.15.114:0/133224428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ef8107540 0x7f1ef8083190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:34.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.754+0000 7f1ef7fff640 1 -- 172.21.15.114:0/133224428 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee0009560 con 0x7f1ef8076bf0 2024-09-13T18:30:34.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.755+0000 7f1ef7fff640 1 --2- 172.21.15.114:0/133224428 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ef8076bf0 0x7f1ef8082c50 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8009f50 tx=0x7f1ee800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:34.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.756+0000 7f1ef5ffb640 1 -- 172.21.15.114:0/133224428 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee800c9b0 con 0x7f1ef8076bf0 2024-09-13T18:30:34.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.756+0000 7f1ef5ffb640 1 -- 172.21.15.114:0/133224428 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1ee801f440 con 0x7f1ef8076bf0 2024-09-13T18:30:34.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.757+0000 7f1ef5ffb640 1 -- 172.21.15.114:0/133224428 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee801d5a0 con 0x7f1ef8076bf0 2024-09-13T18:30:34.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.758+0000 7f1efea41640 1 -- 172.21.15.114:0/133224428 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ef807d610 con 0x7f1ef8076bf0 2024-09-13T18:30:34.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.758+0000 7f1efea41640 1 -- 172.21.15.114:0/133224428 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ef807daa0 con 0x7f1ef8076bf0 2024-09-13T18:30:34.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.760+0000 7f1efea41640 1 -- 172.21.15.114:0/133224428 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ef806bdc0 con 0x7f1ef8076bf0 2024-09-13T18:30:34.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.763+0000 7f1ef5ffb640 1 -- 172.21.15.114:0/133224428 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f1ee800cb10 con 0x7f1ef8076bf0 2024-09-13T18:30:34.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.764+0000 7f1ef5ffb640 1 --2- 172.21.15.114:0/133224428 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f1ee40a12b0 0x7f1ee40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:34.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.765+0000 7f1ef5ffb640 1 -- 172.21.15.114:0/133224428 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f1ee80d5bd0 con 0x7f1ef8076bf0 2024-09-13T18:30:34.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.765+0000 7f1eeffff640 1 --2- 172.21.15.114:0/133224428 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f1ee40a12b0 0x7f1ee40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:34.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.766+0000 7f1eeffff640 1 --2- 172.21.15.114:0/133224428 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f1ee40a12b0 0x7f1ee40a3770 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0005c20 tx=0x7f1ee0000bc0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:34.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:34.771+0000 7f1ef5ffb640 1 -- 172.21.15.114:0/133224428 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f1ee809edb0 con 0x7f1ef8076bf0 2024-09-13T18:30:35.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.277+0000 7f1efea41640 1 -- 172.21.15.114:0/133224428 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1ef8107920 con 0x7f1ef8076bf0 2024-09-13T18:30:35.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.287+0000 7f1ef5ffb640 1 -- 172.21.15.114:0/133224428 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+10930 (secure 0 0 0) 0x7f1ee80a23d0 con 0x7f1ef8076bf0 2024-09-13T18:30:35.296 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:30:35.296 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4162817497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4162817497},{"type":"v1","addr":"172.21.15.114:6835","nonce":4162817497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/2435428478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":2435428478},{"type":"v1","addr":"172.21.15.114:6837","nonce":2435428478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T18:20:01.895117+0000","modified":"2024-09-13T18:27:22.172031+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24482,"mds_2":24475,"mds_3":24458,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/2046516451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":2046516451},{"type":"v1","addr":"172.21.15.114:6839","nonce":2046516451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":10,"addr":"172.21.15.114:6841/1956923446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":1956923446},{"type":"v1","addr":"172.21.15.114:6841","nonce":1956923446}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24391":{"gid":24391,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/507301496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":507301496},{"type":"v1","addr":"172.21.15.204:6833","nonce":507301496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4028657219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4028657219},{"type":"v1","addr":"172.21.15.134:6833","nonce":4028657219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/57029119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":57029119},{"type":"v1","addr":"172.21.15.204:6835","nonce":57029119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3068880939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3068880939},{"type":"v1","addr":"172.21.15.204:6837","nonce":3068880939}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"e","rank":3,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.134:6835/1150003275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1150003275},{"type":"v1","addr":"172.21.15.134:6835","nonce":1150003275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/344883912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":344883912},{"type":"v1","addr":"172.21.15.204:6839","nonce":344883912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6837/891692210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":891692210},{"type":"v1","addr":"172.21.15.134:6837","nonce":891692210}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6839/3466309724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3466309724},{"type":"v1","addr":"172.21.15.134:6839","nonce":3466309724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:30:35.296 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 34 2024-09-13T18:30:35.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.296+0000 7f1efea41640 1 -- 172.21.15.114:0/133224428 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f1ee40a12b0 msgr2=0x7f1ee40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:35.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.296+0000 7f1efea41640 1 --2- 172.21.15.114:0/133224428 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f1ee40a12b0 0x7f1ee40a3770 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0005c20 tx=0x7f1ee0000bc0 comp rx=0 tx=0).stop 2024-09-13T18:30:35.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.296+0000 7f1efea41640 1 -- 172.21.15.114:0/133224428 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ef8076bf0 msgr2=0x7f1ef8082c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:35.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.296+0000 7f1efea41640 1 --2- 172.21.15.114:0/133224428 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ef8076bf0 0x7f1ef8082c50 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8009f50 tx=0x7f1ee800ee50 comp rx=0 tx=0).stop 2024-09-13T18:30:35.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.297+0000 7f1efea41640 1 -- 172.21.15.114:0/133224428 shutdown_connections 2024-09-13T18:30:35.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.297+0000 7f1efea41640 1 --2- 172.21.15.114:0/133224428 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f1ee40a12b0 0x7f1ee40a3770 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:35.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.297+0000 7f1efea41640 1 --2- 172.21.15.114:0/133224428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ef8107e60 0x7f1ef807ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:35.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.297+0000 7f1efea41640 1 --2- 172.21.15.114:0/133224428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ef8107540 0x7f1ef8083190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:35.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.297+0000 7f1efea41640 1 --2- 172.21.15.114:0/133224428 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ef8076bf0 0x7f1ef8082c50 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:35.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.297+0000 7f1efea41640 1 -- 172.21.15.114:0/133224428 >> 172.21.15.114:0/133224428 conn(0x7f1ef806aff0 msgr2=0x7f1ef806ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:35.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.297+0000 7f1efea41640 1 -- 172.21.15.114:0/133224428 shutdown_connections 2024-09-13T18:30:35.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.297+0000 7f1efea41640 1 -- 172.21.15.114:0/133224428 wait complete. 2024-09-13T18:30:35.332 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-09-13T18:30:35.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.581+0000 7fd4f5051640 1 -- 172.21.15.114:0/4172365918 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e0031a90 con 0x7fd4f0072b10 2024-09-13T18:30:35.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.582+0000 7fd4f82b7640 1 -- 172.21.15.114:0/4172365918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4f0072b10 msgr2=0x7fd4f00794d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:35.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.582+0000 7fd4f82b7640 1 --2- 172.21.15.114:0/4172365918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4f0072b10 0x7fd4f00794d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0009d10 tx=0x7fd4e002b590 comp rx=0 tx=0).stop 2024-09-13T18:30:35.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.583+0000 7fd4f82b7640 1 -- 172.21.15.114:0/4172365918 shutdown_connections 2024-09-13T18:30:35.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.583+0000 7fd4f82b7640 1 --2- 172.21.15.114:0/4172365918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4f0072b10 0x7fd4f00794d0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:35.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.583+0000 7fd4f82b7640 1 --2- 172.21.15.114:0/4172365918 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4f0108040 0x7fd4f00723e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:35.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.583+0000 7fd4f82b7640 1 --2- 172.21.15.114:0/4172365918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4f0107720 0x7fd4f0107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:35.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.583+0000 7fd4f82b7640 1 -- 172.21.15.114:0/4172365918 >> 172.21.15.114:0/4172365918 conn(0x7fd4f006afe0 msgr2=0x7fd4f006b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:35.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.587+0000 7fd4f82b7640 1 -- 172.21.15.114:0/4172365918 shutdown_connections 2024-09-13T18:30:35.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.587+0000 7fd4f82b7640 1 -- 172.21.15.114:0/4172365918 wait complete. 2024-09-13T18:30:35.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.588+0000 7fd4f82b7640 1 Processor -- start 2024-09-13T18:30:35.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.593+0000 7fd4f82b7640 1 -- start start 2024-09-13T18:30:35.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.593+0000 7fd4f82b7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4f0072b10 0x7fd4f00829b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:35.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.593+0000 7fd4f82b7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4f0107720 0x7fd4f0082ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:35.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.593+0000 7fd4f82b7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4f0108040 0x7fd4f007daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:35.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.593+0000 7fd4f82b7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f007e210 con 0x7fd4f0108040 2024-09-13T18:30:35.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.593+0000 7fd4f82b7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f007e380 con 0x7fd4f0072b10 2024-09-13T18:30:35.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.593+0000 7fd4f82b7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f007e540 con 0x7fd4f0107720 2024-09-13T18:30:35.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.594+0000 7fd4f5852640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4f0107720 0x7fd4f0082ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:35.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.594+0000 7fd4f6053640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4f0072b10 0x7fd4f00829b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:35.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.594+0000 7fd4f6053640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4f0072b10 0x7fd4f00829b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:40272/0 (socket says 172.21.15.114:40272) 2024-09-13T18:30:35.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.594+0000 7fd4f6053640 1 -- 172.21.15.114:0/2322017332 learned_addr learned my addr 172.21.15.114:0/2322017332 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:35.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.594+0000 7fd4f6053640 1 -- 172.21.15.114:0/2322017332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4f0107720 msgr2=0x7fd4f0082ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:35.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.594+0000 7fd4f6053640 1 --2- 172.21.15.114:0/2322017332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4f0107720 0x7fd4f0082ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:35.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.594+0000 7fd4f6053640 1 -- 172.21.15.114:0/2322017332 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4f0108040 msgr2=0x7fd4f007daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:35.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.594+0000 7fd4f6053640 1 --2- 172.21.15.114:0/2322017332 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4f0108040 0x7fd4f007daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:35.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.594+0000 7fd4f6053640 1 -- 172.21.15.114:0/2322017332 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e0009490 con 0x7fd4f0072b10 2024-09-13T18:30:35.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.595+0000 7fd4f6053640 1 --2- 172.21.15.114:0/2322017332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4f0072b10 0x7fd4f00829b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec00b450 tx=0x7fd4ec00b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:35.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.599+0000 7fd4e77fe640 1 -- 172.21.15.114:0/2322017332 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec011890 con 0x7fd4f0072b10 2024-09-13T18:30:35.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.599+0000 7fd4f82b7640 1 -- 172.21.15.114:0/2322017332 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4f007e820 con 0x7fd4f0072b10 2024-09-13T18:30:35.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.599+0000 7fd4f82b7640 1 -- 172.21.15.114:0/2322017332 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4f01ba4f0 con 0x7fd4f0072b10 2024-09-13T18:30:35.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.599+0000 7fd4e77fe640 1 -- 172.21.15.114:0/2322017332 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd4ec011e80 con 0x7fd4f0072b10 2024-09-13T18:30:35.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.599+0000 7fd4e77fe640 1 -- 172.21.15.114:0/2322017332 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec00f340 con 0x7fd4f0072b10 2024-09-13T18:30:35.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.606+0000 7fd4e77fe640 1 -- 172.21.15.114:0/2322017332 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fd4ec0103e0 con 0x7fd4f0072b10 2024-09-13T18:30:35.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.607+0000 7fd4e77fe640 1 --2- 172.21.15.114:0/2322017332 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fd4dc0a1410 0x7fd4dc0a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:35.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.607+0000 7fd4e77fe640 1 -- 172.21.15.114:0/2322017332 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fd4ec0ccc60 con 0x7fd4f0072b10 2024-09-13T18:30:35.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.609+0000 7fd4f5852640 1 --2- 172.21.15.114:0/2322017332 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fd4dc0a1410 0x7fd4dc0a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:35.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.610+0000 7fd4f5852640 1 --2- 172.21.15.114:0/2322017332 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fd4dc0a1410 0x7fd4dc0a38d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd4f006fe80 tx=0x7fd4e8009680 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:35.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.611+0000 7fd4fafdd740 1 Processor -- start 2024-09-13T18:30:35.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.613+0000 7fd4fafdd740 1 -- start start 2024-09-13T18:30:35.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.613+0000 7fd4fafdd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a1ef7d4790 0x55a1ef7d4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:35.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.614+0000 7fd4fafdd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1ef7d50b0 0x55a1ef7df410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:35.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.614+0000 7fd4fafdd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1ef7df950 0x55a1ef7e1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:35.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.614+0000 7fd4fafdd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1ef73ecc0 con 0x55a1ef7d4790 2024-09-13T18:30:35.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.614+0000 7fd4fafdd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1ef73c450 con 0x55a1ef7df950 2024-09-13T18:30:35.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.614+0000 7fd4fafdd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1ef737d40 con 0x55a1ef7d50b0 2024-09-13T18:30:35.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.614+0000 7fd4f5852640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1ef7d50b0 0x55a1ef7df410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:35.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.614+0000 7fd4f5852640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1ef7d50b0 0x55a1ef7df410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:58786/0 (socket says 172.21.15.114:58786) 2024-09-13T18:30:35.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.614+0000 7fd4f5852640 1 -- 172.21.15.114:0/873108080 learned_addr learned my addr 172.21.15.114:0/873108080 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:35.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.614+0000 7fd4f6854640 1 --2- 172.21.15.114:0/873108080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1ef7df950 0x55a1ef7e1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:35.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.614+0000 7fd4f5852640 1 -- 172.21.15.114:0/873108080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1ef7df950 msgr2=0x55a1ef7e1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:35.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.614+0000 7fd4f5852640 1 --2- 172.21.15.114:0/873108080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1ef7df950 0x55a1ef7e1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:35.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.615+0000 7fd4f5852640 1 -- 172.21.15.114:0/873108080 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a1ef7d4790 msgr2=0x55a1ef7d4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:35.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.615+0000 7fd4f5852640 1 --2- 172.21.15.114:0/873108080 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a1ef7d4790 0x55a1ef7d4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:35.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.615+0000 7fd4f5852640 1 -- 172.21.15.114:0/873108080 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1ef6ff620 con 0x55a1ef7d50b0 2024-09-13T18:30:35.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.615+0000 7fd4f6854640 1 --2- 172.21.15.114:0/873108080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1ef7df950 0x55a1ef7e1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:30:35.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.622+0000 7fd4f5852640 1 --2- 172.21.15.114:0/873108080 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1ef7d50b0 0x55a1ef7df410 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd4e801d750 tx=0x7fd4e8014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81c789c49f2fb0f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:35.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.634+0000 7fd4e4e05640 1 -- 172.21.15.114:0/873108080 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e8014070 con 0x55a1ef7d50b0 2024-09-13T18:30:35.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.634+0000 7fd4e4e05640 1 -- 172.21.15.114:0/873108080 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd4e801ba00 con 0x55a1ef7d50b0 2024-09-13T18:30:35.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.634+0000 7fd4e4e05640 1 -- 172.21.15.114:0/873108080 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e80257a0 con 0x55a1ef7d50b0 2024-09-13T18:30:35.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.635+0000 7fd4fafdd740 1 -- 172.21.15.114:0/873108080 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1ef7d50b0 msgr2=0x55a1ef7df410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:35.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.635+0000 7fd4fafdd740 1 --2- 172.21.15.114:0/873108080 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1ef7d50b0 0x55a1ef7df410 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd4e801d750 tx=0x7fd4e8014000 comp rx=0 tx=0).stop 2024-09-13T18:30:35.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.635+0000 7fd4fafdd740 1 -- 172.21.15.114:0/873108080 shutdown_connections 2024-09-13T18:30:35.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.635+0000 7fd4fafdd740 1 --2- 172.21.15.114:0/873108080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1ef7df950 0x55a1ef7e1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:35.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.635+0000 7fd4fafdd740 1 --2- 172.21.15.114:0/873108080 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1ef7d50b0 0x55a1ef7df410 secure :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd4e801d750 tx=0x7fd4e8014000 comp rx=0 tx=0).stop 2024-09-13T18:30:35.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.635+0000 7fd4fafdd740 1 --2- 172.21.15.114:0/873108080 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a1ef7d4790 0x55a1ef7d4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:35.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.635+0000 7fd4fafdd740 1 -- 172.21.15.114:0/873108080 >> 172.21.15.114:0/873108080 conn(0x55a1ef7d0d00 msgr2=0x55a1ef7d10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:35.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.643+0000 7fd4fafdd740 1 -- 172.21.15.114:0/873108080 shutdown_connections 2024-09-13T18:30:35.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.649+0000 7fd4fafdd740 1 -- 172.21.15.114:0/873108080 wait complete. 2024-09-13T18:30:35.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.650+0000 7fd4fafdd740 1 -- start start 2024-09-13T18:30:35.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.650+0000 7fd4fafdd740 1 Processor -- start 2024-09-13T18:30:35.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.653+0000 7fd4fafdd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1ef7d4790 0x55a1ef7df100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:35.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.653+0000 7fd4fafdd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a1ef7df950 0x55a1ef7dc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:35.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.653+0000 7fd4fafdd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1ef7dc910 0x55a1ef7dcd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:35.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.653+0000 7fd4fafdd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1ef6f6180 con 0x55a1ef7df950 2024-09-13T18:30:35.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.653+0000 7fd4fafdd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1ef410430 con 0x55a1ef7d4790 2024-09-13T18:30:35.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.653+0000 7fd4fafdd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1ef6bb5e0 con 0x55a1ef7dc910 2024-09-13T18:30:35.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.653+0000 7fd4fafdd740 10 client.0 _ll_register_callbacks cb 0x55a1ef727aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:30:35.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.654+0000 7fd4f6053640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1ef7d4790 0x55a1ef7df100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:35.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.654+0000 7fd4f5852640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a1ef7df950 0x55a1ef7dc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:35.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.654+0000 7fd4f6053640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1ef7d4790 0x55a1ef7df100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:40292/0 (socket says 172.21.15.114:40292) 2024-09-13T18:30:35.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.654+0000 7fd4f6053640 1 -- 172.21.15.114:0/1167014831 learned_addr learned my addr 172.21.15.114:0/1167014831 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:35.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.654+0000 7fd4f6053640 1 -- 172.21.15.114:0/1167014831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1ef7dc910 msgr2=0x55a1ef7dcd90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:30:35.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.654+0000 7fd4f6053640 1 --2- 172.21.15.114:0/1167014831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1ef7dc910 0x55a1ef7dcd90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:35.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.654+0000 7fd4f6053640 1 -- 172.21.15.114:0/1167014831 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a1ef7df950 msgr2=0x55a1ef7dc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:35.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.654+0000 7fd4f6053640 1 --2- 172.21.15.114:0/1167014831 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a1ef7df950 0x55a1ef7dc3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:35.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.654+0000 7fd4f6053640 1 -- 172.21.15.114:0/1167014831 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4e801d430 con 0x55a1ef7d4790 2024-09-13T18:30:35.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.655+0000 7fd4f6053640 1 --2- 172.21.15.114:0/1167014831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1ef7d4790 0x55a1ef7df100 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec0ca120 tx=0x7fd4ec0993e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f9127c3674eeb0b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:35.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.655+0000 7fd4d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:30:35.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.655+0000 7fd4d9ffb640 1 -- 172.21.15.114:0/1167014831 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec0b2410 con 0x55a1ef7d4790 2024-09-13T18:30:35.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.656+0000 7fd4fafdd740 1 -- 172.21.15.114:0/1167014831 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a1ef761b80 con 0x55a1ef7d4790 2024-09-13T18:30:35.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.656+0000 7fd4d9ffb640 1 -- 172.21.15.114:0/1167014831 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fd4ec09d8b0 con 0x55a1ef7d4790 2024-09-13T18:30:35.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.656+0000 7fd4d9ffb640 1 -- 172.21.15.114:0/1167014831 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4ec0ca440 con 0x55a1ef7d4790 2024-09-13T18:30:35.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.660+0000 7fd4fafdd740 10 client.27281 fetch_fsmap learned FSMap version 34 2024-09-13T18:30:35.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.660+0000 7fd4fafdd740 1 -- 172.21.15.114:0/1167014831 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55a1ef7363c0 con 0x55a1ef7d4790 2024-09-13T18:30:35.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.663+0000 7fd4d9ffb640 1 -- 172.21.15.114:0/1167014831 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a1ef761b80 con 0x55a1ef7d4790 2024-09-13T18:30:35.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.663+0000 7fd4d9ffb640 1 -- 172.21.15.114:0/1167014831 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd4ec095870 con 0x55a1ef7d4790 2024-09-13T18:30:35.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.663+0000 7fd4fafdd740 10 client.27281 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:30:35.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.663+0000 7fd4fafdd740 10 client.27281 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-13T18:30:35.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.663+0000 7fd4fafdd740 20 client.27281 populate_metadata read hostname 'smithi114' 2024-09-13T18:30:35.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.663+0000 7fd4fafdd740 1 --2- 172.21.15.114:0/1167014831 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55a1ef7ef920 0x55a1ef7f1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:35.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.663+0000 7fd4fafdd740 4 client.27281 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:30:35.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.663+0000 7fd4fafdd740 1 -- 172.21.15.114:0/1167014831 --> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a1ef72a1f0 con 0x55a1ef7ef920 2024-09-13T18:30:35.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.664+0000 7fd4f5852640 1 --2- 172.21.15.114:0/1167014831 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55a1ef7ef920 0x55a1ef7f1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:35.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.665+0000 7fd4d9ffb640 10 client.27281 ms_handle_connect on v2:172.21.15.114:6840/1956923446 2024-09-13T18:30:35.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.665+0000 7fd4f5852640 1 --2- 172.21.15.114:0/1167014831 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55a1ef7ef920 0x55a1ef7f1d40 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=46528d8e924e256c server_cookie=c1e2b1d20c893412 in_seq=0 out_seq=0 2024-09-13T18:30:35.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.672+0000 7fd4d9ffb640 1 -- 172.21.15.114:0/1167014831 <== mds.4 v2:172.21.15.114:6840/1956923446 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55a1ef72a1f0 con 0x55a1ef7ef920 2024-09-13T18:30:35.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.672+0000 7fd4d9ffb640 10 client.27281 handle_command_reply: tid=0 2024-09-13T18:30:35.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.672+0000 7fd4fafdd740 1 client.27281 shutdown 2024-09-13T18:30:35.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.672+0000 7fd4fafdd740 1 -- 172.21.15.114:0/1167014831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1ef7d4790 msgr2=0x55a1ef7df100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:35.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.672+0000 7fd4fafdd740 1 --2- 172.21.15.114:0/1167014831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1ef7d4790 0x55a1ef7df100 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec0ca120 tx=0x7fd4ec0993e0 comp rx=0 tx=0).stop 2024-09-13T18:30:35.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.672+0000 7fd4fafdd740 1 -- 172.21.15.114:0/1167014831 shutdown_connections 2024-09-13T18:30:35.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.672+0000 7fd4fafdd740 1 --2- 172.21.15.114:0/1167014831 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55a1ef7ef920 0x55a1ef7f1d40 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:35.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.672+0000 7fd4fafdd740 1 --2- 172.21.15.114:0/1167014831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1ef7dc910 0x55a1ef7dcd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:35.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.672+0000 7fd4fafdd740 1 --2- 172.21.15.114:0/1167014831 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a1ef7df950 0x55a1ef7dc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:35.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.672+0000 7fd4fafdd740 1 --2- 172.21.15.114:0/1167014831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1ef7d4790 0x55a1ef7df100 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:35.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.672+0000 7fd4fafdd740 1 -- 172.21.15.114:0/1167014831 >> 172.21.15.114:0/1167014831 conn(0x55a1ef7d0d00 msgr2=0x55a1ef7d6a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:35.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.672+0000 7fd4d9ffb640 0 client.27281 ms_handle_reset on v2:172.21.15.114:6840/1956923446 2024-09-13T18:30:35.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.672+0000 7fd4fafdd740 1 -- 172.21.15.114:0/1167014831 shutdown_connections 2024-09-13T18:30:35.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.673+0000 7fd4fafdd740 1 -- 172.21.15.114:0/1167014831 wait complete. 2024-09-13T18:30:35.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.678+0000 7fd4fafdd740 20 client.27281 trim_cache size 0 max 16384 2024-09-13T18:30:35.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:35 smithi114 ceph-mon[30897]: pgmap v250: 129 pgs: 129 active+clean; 22 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 33 MiB/s wr, 704 op/s 2024-09-13T18:30:35.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:35 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/133224428' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:30:35.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.726+0000 7fd4fafdd740 1 Processor -- start 2024-09-13T18:30:35.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.726+0000 7fd4fafdd740 1 -- start start 2024-09-13T18:30:35.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.727+0000 7fd4fafdd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1ef7d4790 0x55a1ef7dec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:35.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.727+0000 7fd4fafdd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1ef7dc910 0x55a1ef7dbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:35.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.727+0000 7fd4fafdd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a1ef7df950 0x55a1ef7dc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:35.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.727+0000 7fd4f6053640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1ef7d4790 0x55a1ef7dec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:35.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.727+0000 7fd4f5852640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1ef7dc910 0x55a1ef7dbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:35.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.727+0000 7fd4f5852640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1ef7dc910 0x55a1ef7dbc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:40306/0 (socket says 172.21.15.114:40306) 2024-09-13T18:30:35.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.727+0000 7fd4f6053640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1ef7d4790 0x55a1ef7dec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:58798/0 (socket says 172.21.15.114:58798) 2024-09-13T18:30:35.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.732+0000 7fd4f6854640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a1ef7df950 0x55a1ef7dc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:35.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.732+0000 7fd4f5852640 1 -- 172.21.15.114:0/3400946994 learned_addr learned my addr 172.21.15.114:0/3400946994 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:35.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.732+0000 7fd4fafdd740 1 -- 172.21.15.114:0/3400946994 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1ef7dd370 con 0x55a1ef7df950 2024-09-13T18:30:35.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.732+0000 7fd4fafdd740 1 -- 172.21.15.114:0/3400946994 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1ef7ec640 con 0x55a1ef7dc910 2024-09-13T18:30:35.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.732+0000 7fd4fafdd740 1 -- 172.21.15.114:0/3400946994 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1ef7e2400 con 0x55a1ef7d4790 2024-09-13T18:30:35.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.732+0000 7fd4f6053640 1 -- 172.21.15.114:0/3400946994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1ef7dc910 msgr2=0x55a1ef7dbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:35.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.732+0000 7fd4f6053640 1 --2- 172.21.15.114:0/3400946994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1ef7dc910 0x55a1ef7dbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:35.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.732+0000 7fd4f6053640 1 -- 172.21.15.114:0/3400946994 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a1ef7df950 msgr2=0x55a1ef7dc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:35.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.732+0000 7fd4f6053640 1 --2- 172.21.15.114:0/3400946994 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a1ef7df950 0x55a1ef7dc1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:35.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.732+0000 7fd4f6053640 1 -- 172.21.15.114:0/3400946994 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1ef6d8770 con 0x55a1ef7d4790 2024-09-13T18:30:35.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.733+0000 7fd4f6053640 1 --2- 172.21.15.114:0/3400946994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1ef7d4790 0x55a1ef7dec60 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec00be10 tx=0x7fd4ec00bf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ac56a93e29b39f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:35.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.739+0000 7fd4e5606640 1 -- 172.21.15.114:0/3400946994 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec098a80 con 0x55a1ef7d4790 2024-09-13T18:30:35.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.739+0000 7fd4e5606640 1 -- 172.21.15.114:0/3400946994 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd4ec0ca440 con 0x55a1ef7d4790 2024-09-13T18:30:35.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.740+0000 7fd4fafdd740 1 -- 172.21.15.114:0/3400946994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1ef7d4790 msgr2=0x55a1ef7dec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:35.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.740+0000 7fd4fafdd740 1 --2- 172.21.15.114:0/3400946994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1ef7d4790 0x55a1ef7dec60 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec00be10 tx=0x7fd4ec00bf80 comp rx=0 tx=0).stop 2024-09-13T18:30:35.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.744+0000 7fd4fafdd740 1 -- 172.21.15.114:0/3400946994 shutdown_connections 2024-09-13T18:30:35.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.744+0000 7fd4fafdd740 1 --2- 172.21.15.114:0/3400946994 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a1ef7df950 0x55a1ef7dc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:35.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.744+0000 7fd4fafdd740 1 --2- 172.21.15.114:0/3400946994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1ef7dc910 0x55a1ef7dbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:35.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.744+0000 7fd4fafdd740 1 --2- 172.21.15.114:0/3400946994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1ef7d4790 0x55a1ef7dec60 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:35.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.744+0000 7fd4fafdd740 1 -- 172.21.15.114:0/3400946994 >> 172.21.15.114:0/3400946994 conn(0x55a1ef7d0d00 msgr2=0x55a1ef7f6660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:35.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.747+0000 7fd4fafdd740 1 -- 172.21.15.114:0/3400946994 shutdown_connections 2024-09-13T18:30:35.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.747+0000 7fd4fafdd740 1 -- 172.21.15.114:0/3400946994 wait complete. 2024-09-13T18:30:35.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.748+0000 7fd4fafdd740 1 -- start start 2024-09-13T18:30:35.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.748+0000 7fd4fafdd740 1 Processor -- start 2024-09-13T18:30:35.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.748+0000 7fd4fafdd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a1ef7d4790 0x55a1ef7e29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:35.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.748+0000 7fd4fafdd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1ef7dc910 0x55a1ef7e2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:35.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.748+0000 7fd4fafdd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1ef7df950 0x55a1ef844dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:35.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.749+0000 7fd4fafdd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1ef845340 con 0x55a1ef7d4790 2024-09-13T18:30:35.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.749+0000 7fd4fafdd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1ef8454b0 con 0x55a1ef7dc910 2024-09-13T18:30:35.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.749+0000 7fd4f5852640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1ef7dc910 0x55a1ef7e2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:35.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.749+0000 7fd4fafdd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1ef845670 con 0x55a1ef7df950 2024-09-13T18:30:35.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.749+0000 7fd4f5852640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1ef7dc910 0x55a1ef7e2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:40312/0 (socket says 172.21.15.114:40312) 2024-09-13T18:30:35.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.749+0000 7fd4f5852640 1 -- 172.21.15.114:0/1842414690 learned_addr learned my addr 172.21.15.114:0/1842414690 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:35.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.749+0000 7fd4f6854640 1 --2- 172.21.15.114:0/1842414690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1ef7df950 0x55a1ef844dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:35.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.749+0000 7fd4fafdd740 10 client.0 _ll_register_callbacks cb 0x7fd4ec0107d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:30:35.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.751+0000 7fd4f6053640 1 --2- 172.21.15.114:0/1842414690 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a1ef7d4790 0x55a1ef7e29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:35.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.751+0000 7fd4f5852640 1 -- 172.21.15.114:0/1842414690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1ef7df950 msgr2=0x55a1ef844dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:35.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.752+0000 7fd4f5852640 1 --2- 172.21.15.114:0/1842414690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1ef7df950 0x55a1ef844dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:35.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.752+0000 7fd4f5852640 1 -- 172.21.15.114:0/1842414690 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a1ef7d4790 msgr2=0x55a1ef7e29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:35.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.752+0000 7fd4f5852640 1 --2- 172.21.15.114:0/1842414690 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a1ef7d4790 0x55a1ef7e29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:35.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.752+0000 7fd4f5852640 1 -- 172.21.15.114:0/1842414690 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a1ef7363c0 con 0x55a1ef7dc910 2024-09-13T18:30:35.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.756+0000 7fd4f5852640 1 --2- 172.21.15.114:0/1842414690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1ef7dc910 0x55a1ef7e2ef0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd4e800fcd0 tx=0x7fd4e8009730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bd478e9654c26d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:35.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.757+0000 7fd4d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:30:35.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.758+0000 7fd4d9ffb640 1 -- 172.21.15.114:0/1842414690 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e8034ea0 con 0x55a1ef7dc910 2024-09-13T18:30:35.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.758+0000 7fd4fafdd740 1 -- 172.21.15.114:0/1842414690 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd4e801ba00 con 0x55a1ef7dc910 2024-09-13T18:30:35.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.758+0000 7fd4d9ffb640 1 -- 172.21.15.114:0/1842414690 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fd4e801be80 con 0x55a1ef7dc910 2024-09-13T18:30:35.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.758+0000 7fd4d9ffb640 1 -- 172.21.15.114:0/1842414690 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4e8004a00 con 0x55a1ef7dc910 2024-09-13T18:30:35.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.760+0000 7fd4fafdd740 10 client.27287 fetch_fsmap learned FSMap version 34 2024-09-13T18:30:35.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.760+0000 7fd4fafdd740 1 -- 172.21.15.114:0/1842414690 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55a1ef6fde80 con 0x55a1ef7dc910 2024-09-13T18:30:35.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.760+0000 7fd4d9ffb640 1 -- 172.21.15.114:0/1842414690 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd4e801ba00 con 0x55a1ef7dc910 2024-09-13T18:30:35.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.760+0000 7fd4d9ffb640 1 -- 172.21.15.114:0/1842414690 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd4e8005040 con 0x55a1ef7dc910 2024-09-13T18:30:35.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.760+0000 7fd4fafdd740 10 client.27287 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:30:35.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.760+0000 7fd4fafdd740 10 client.27287 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-13T18:30:35.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.760+0000 7fd4fafdd740 20 client.27287 populate_metadata read hostname 'smithi114' 2024-09-13T18:30:35.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.760+0000 7fd4fafdd740 1 --2- 172.21.15.114:0/1842414690 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55a1ef846e50 0x55a1ef849270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:35.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.760+0000 7fd4fafdd740 4 client.27287 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:30:35.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.760+0000 7fd4fafdd740 1 -- 172.21.15.114:0/1842414690 --> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a1ef42b4e0 con 0x55a1ef846e50 2024-09-13T18:30:35.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.768+0000 7fd4f6053640 1 --2- 172.21.15.114:0/1842414690 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55a1ef846e50 0x55a1ef849270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:35.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.778+0000 7fd4f6053640 1 --2- 172.21.15.114:0/1842414690 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55a1ef846e50 0x55a1ef849270 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d5c7e43695934707 server_cookie=7bd670898794ab84 in_seq=0 out_seq=0 2024-09-13T18:30:35.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.778+0000 7fd4d9ffb640 10 client.27287 ms_handle_connect on v2:172.21.15.114:6840/1956923446 2024-09-13T18:30:35.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.779+0000 7fd4d9ffb640 1 -- 172.21.15.114:0/1842414690 <== mds.4 v2:172.21.15.114:6840/1956923446 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a1ef42b4e0 con 0x55a1ef846e50 2024-09-13T18:30:35.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.779+0000 7fd4d9ffb640 10 client.27287 handle_command_reply: tid=0 2024-09-13T18:30:35.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.780+0000 7fd4fafdd740 1 client.27287 shutdown 2024-09-13T18:30:35.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.782+0000 7fd4fafdd740 1 -- 172.21.15.114:0/1842414690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1ef7dc910 msgr2=0x55a1ef7e2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:35.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.782+0000 7fd4fafdd740 1 --2- 172.21.15.114:0/1842414690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1ef7dc910 0x55a1ef7e2ef0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd4e800fcd0 tx=0x7fd4e8009730 comp rx=0 tx=0).stop 2024-09-13T18:30:35.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.786+0000 7fd4fafdd740 1 -- 172.21.15.114:0/1842414690 shutdown_connections 2024-09-13T18:30:35.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.786+0000 7fd4fafdd740 1 --2- 172.21.15.114:0/1842414690 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55a1ef846e50 0x55a1ef849270 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:35.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.786+0000 7fd4fafdd740 1 --2- 172.21.15.114:0/1842414690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1ef7df950 0x55a1ef844dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:35.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.786+0000 7fd4fafdd740 1 --2- 172.21.15.114:0/1842414690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1ef7dc910 0x55a1ef7e2ef0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:35.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.786+0000 7fd4fafdd740 1 --2- 172.21.15.114:0/1842414690 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a1ef7d4790 0x55a1ef7e29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:35.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.786+0000 7fd4fafdd740 1 -- 172.21.15.114:0/1842414690 >> 172.21.15.114:0/1842414690 conn(0x55a1ef7d0d00 msgr2=0x55a1ef7dbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:35.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.787+0000 7fd4d9ffb640 0 client.27287 ms_handle_reset on v2:172.21.15.114:6840/1956923446 2024-09-13T18:30:35.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.792+0000 7fd4fafdd740 1 -- 172.21.15.114:0/1842414690 shutdown_connections 2024-09-13T18:30:35.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.797+0000 7fd4fafdd740 1 -- 172.21.15.114:0/1842414690 wait complete. 2024-09-13T18:30:35.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.801+0000 7fd4fafdd740 20 client.27287 trim_cache size 0 max 16384 2024-09-13T18:30:35.808 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:30:35.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.809+0000 7fd4f82b7640 1 -- 172.21.15.114:0/2322017332 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fd4dc0a1410 msgr2=0x7fd4dc0a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:35.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.810+0000 7fd4f82b7640 1 --2- 172.21.15.114:0/2322017332 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fd4dc0a1410 0x7fd4dc0a38d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd4f006fe80 tx=0x7fd4e8009680 comp rx=0 tx=0).stop 2024-09-13T18:30:35.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.810+0000 7fd4f82b7640 1 -- 172.21.15.114:0/2322017332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4f0072b10 msgr2=0x7fd4f00829b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:35.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.810+0000 7fd4f82b7640 1 --2- 172.21.15.114:0/2322017332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4f0072b10 0x7fd4f00829b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec00b450 tx=0x7fd4ec00b910 comp rx=0 tx=0).stop 2024-09-13T18:30:35.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.814+0000 7fd4f82b7640 1 -- 172.21.15.114:0/2322017332 shutdown_connections 2024-09-13T18:30:35.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.814+0000 7fd4f82b7640 1 --2- 172.21.15.114:0/2322017332 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fd4dc0a1410 0x7fd4dc0a38d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:35.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.814+0000 7fd4f82b7640 1 --2- 172.21.15.114:0/2322017332 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4f0108040 0x7fd4f007daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:35.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.814+0000 7fd4f82b7640 1 --2- 172.21.15.114:0/2322017332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4f0107720 0x7fd4f0082ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:35.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.814+0000 7fd4f82b7640 1 --2- 172.21.15.114:0/2322017332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4f0072b10 0x7fd4f00829b0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:35.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.814+0000 7fd4f82b7640 1 -- 172.21.15.114:0/2322017332 >> 172.21.15.114:0/2322017332 conn(0x7fd4f006afe0 msgr2=0x7fd4f0070780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:35.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.817+0000 7fd4f82b7640 1 -- 172.21.15.114:0/2322017332 shutdown_connections 2024-09-13T18:30:35.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:35.818+0000 7fd4f82b7640 1 -- 172.21.15.114:0/2322017332 wait complete. 2024-09-13T18:30:35.859 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-09-13T18:30:35.868 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:35 smithi134 ceph-mon[34283]: pgmap v250: 129 pgs: 129 active+clean; 22 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 33 MiB/s wr, 704 op/s 2024-09-13T18:30:35.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:35 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/133224428' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:30:35.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:35 smithi204 ceph-mon[34777]: pgmap v250: 129 pgs: 129 active+clean; 22 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 33 MiB/s wr, 704 op/s 2024-09-13T18:30:35.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:35 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/133224428' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:30:36.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.076+0000 7fbb1df76640 1 -- 172.21.15.114:0/3540441432 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb100b7540 msgr2=0x7fbb100b9930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:36.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.076+0000 7fbb1df76640 1 --2- 172.21.15.114:0/3540441432 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb100b7540 0x7fbb100b9930 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fbb18063b50 tx=0x7fbb1809d6c0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.079+0000 7fbb1df76640 1 -- 172.21.15.114:0/3540441432 shutdown_connections 2024-09-13T18:30:36.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.079+0000 7fbb1df76640 1 --2- 172.21.15.114:0/3540441432 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb100b7540 0x7fbb100b9930 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.079+0000 7fbb1df76640 1 --2- 172.21.15.114:0/3540441432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb100a5140 0x7fbb100b7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.079+0000 7fbb1df76640 1 --2- 172.21.15.114:0/3540441432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb100a4820 0x7fbb100a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.079+0000 7fbb1df76640 1 -- 172.21.15.114:0/3540441432 >> 172.21.15.114:0/3540441432 conn(0x7fbb1001a110 msgr2=0x7fbb1001a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:36.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.083+0000 7fbb1df76640 1 -- 172.21.15.114:0/3540441432 shutdown_connections 2024-09-13T18:30:36.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.083+0000 7fbb1df76640 1 -- 172.21.15.114:0/3540441432 wait complete. 2024-09-13T18:30:36.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.084+0000 7fbb1df76640 1 Processor -- start 2024-09-13T18:30:36.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.085+0000 7fbb1df76640 1 -- start start 2024-09-13T18:30:36.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.085+0000 7fbb1df76640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb100a4820 0x7fbb100b6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:36.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.085+0000 7fbb1df76640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb100a5140 0x7fbb100b6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:36.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.085+0000 7fbb1df76640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb100b7540 0x7fbb100b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:36.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.085+0000 7fbb1df76640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb100b3480 con 0x7fbb100b7540 2024-09-13T18:30:36.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.085+0000 7fbb1df76640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb100b35f0 con 0x7fbb100a5140 2024-09-13T18:30:36.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.085+0000 7fbb1df76640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb100b37b0 con 0x7fbb100a4820 2024-09-13T18:30:36.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.085+0000 7fbb17fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb100b7540 0x7fbb100b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:36.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.085+0000 7fbb17fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb100b7540 0x7fbb100b2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57858/0 (socket says 172.21.15.114:57858) 2024-09-13T18:30:36.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.085+0000 7fbb17fff640 1 -- 172.21.15.114:0/1443514581 learned_addr learned my addr 172.21.15.114:0/1443514581 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:36.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.086+0000 7fbb17fff640 1 -- 172.21.15.114:0/1443514581 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb100a4820 msgr2=0x7fbb100b6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:36.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.086+0000 7fbb17fff640 1 --2- 172.21.15.114:0/1443514581 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb100a4820 0x7fbb100b6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.086+0000 7fbb17fff640 1 -- 172.21.15.114:0/1443514581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb100a5140 msgr2=0x7fbb100b6f90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:30:36.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.086+0000 7fbb17fff640 1 --2- 172.21.15.114:0/1443514581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb100a5140 0x7fbb100b6f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.086+0000 7fbb17fff640 1 -- 172.21.15.114:0/1443514581 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb1804eac0 con 0x7fbb100b7540 2024-09-13T18:30:36.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.086+0000 7fbb17fff640 1 --2- 172.21.15.114:0/1443514581 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb100b7540 0x7fbb100b2cf0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fbb18000b60 tx=0x7fbb18066f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:36.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.087+0000 7fbb14ff9640 1 -- 172.21.15.114:0/1443514581 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb18068430 con 0x7fbb100b7540 2024-09-13T18:30:36.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.087+0000 7fbb1df76640 1 -- 172.21.15.114:0/1443514581 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb10147290 con 0x7fbb100b7540 2024-09-13T18:30:36.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.087+0000 7fbb1df76640 1 -- 172.21.15.114:0/1443514581 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb10147600 con 0x7fbb100b7540 2024-09-13T18:30:36.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.087+0000 7fbb14ff9640 1 -- 172.21.15.114:0/1443514581 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbb18068590 con 0x7fbb100b7540 2024-09-13T18:30:36.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.087+0000 7fbb14ff9640 1 -- 172.21.15.114:0/1443514581 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb180a5e50 con 0x7fbb100b7540 2024-09-13T18:30:36.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.094+0000 7fbb14ff9640 1 -- 172.21.15.114:0/1443514581 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fbb180a2030 con 0x7fbb100b7540 2024-09-13T18:30:36.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.095+0000 7fbb14ff9640 1 --2- 172.21.15.114:0/1443514581 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbb080a1410 0x7fbb080a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:36.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.096+0000 7fbb14ff9640 1 -- 172.21.15.114:0/1443514581 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fbb1815fe10 con 0x7fbb100b7540 2024-09-13T18:30:36.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.098+0000 7fbb20c96740 1 Processor -- start 2024-09-13T18:30:36.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.096+0000 7fbb177fe640 1 --2- 172.21.15.114:0/1443514581 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbb080a1410 0x7fbb080a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:36.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.099+0000 7fbb20c96740 1 -- start start 2024-09-13T18:30:36.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.099+0000 7fbb20c96740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff2f73e970 0x55ff2f73ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:36.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.099+0000 7fbb20c96740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff2f73f290 0x55ff2f7495f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:36.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.099+0000 7fbb20c96740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff2f749b30 0x55ff2f74bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:36.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.099+0000 7fbb20c96740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff2f6a8cc0 con 0x55ff2f73f290 2024-09-13T18:30:36.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.099+0000 7fbb20c96740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff2f6a6450 con 0x55ff2f749b30 2024-09-13T18:30:36.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.099+0000 7fbb20c96740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff2f6a1d40 con 0x55ff2f73e970 2024-09-13T18:30:36.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.100+0000 7fbb17fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff2f749b30 0x55ff2f74bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:36.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.100+0000 7fbb177fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff2f73e970 0x55ff2f73ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:36.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.100+0000 7fbb177fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff2f73e970 0x55ff2f73ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:58812/0 (socket says 172.21.15.114:58812) 2024-09-13T18:30:36.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.100+0000 7fbb177fe640 1 -- 172.21.15.114:0/3675120536 learned_addr learned my addr 172.21.15.114:0/3675120536 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:36.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.101+0000 7fbb16ffd640 1 --2- 172.21.15.114:0/3675120536 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff2f73f290 0x55ff2f7495f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:36.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.101+0000 7fbb16ffd640 1 -- 172.21.15.114:0/3675120536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff2f73e970 msgr2=0x55ff2f73ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:36.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.101+0000 7fbb16ffd640 1 --2- 172.21.15.114:0/3675120536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff2f73e970 0x55ff2f73ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.101+0000 7fbb16ffd640 1 -- 172.21.15.114:0/3675120536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff2f749b30 msgr2=0x55ff2f74bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:36.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.101+0000 7fbb16ffd640 1 --2- 172.21.15.114:0/3675120536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff2f749b30 0x55ff2f74bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.101+0000 7fbb16ffd640 1 -- 172.21.15.114:0/3675120536 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff2f669620 con 0x55ff2f73f290 2024-09-13T18:30:36.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.102+0000 7fbb16ffd640 1 --2- 172.21.15.114:0/3675120536 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff2f73f290 0x55ff2f7495f0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fbb100b42a0 tx=0x7fbb0c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17b68e0145733ec server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:36.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.102+0000 7fbb01ffb640 1 -- 172.21.15.114:0/3675120536 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c00b890 con 0x55ff2f73f290 2024-09-13T18:30:36.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.103+0000 7fbb01ffb640 1 -- 172.21.15.114:0/3675120536 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbb0c00be80 con 0x55ff2f73f290 2024-09-13T18:30:36.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.103+0000 7fbb17fff640 1 --2- 172.21.15.114:0/3675120536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff2f749b30 0x55ff2f74bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:30:36.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.104+0000 7fbb20c96740 1 -- 172.21.15.114:0/3675120536 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff2f73f290 msgr2=0x55ff2f7495f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:36.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.104+0000 7fbb20c96740 1 --2- 172.21.15.114:0/3675120536 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff2f73f290 0x55ff2f7495f0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fbb100b42a0 tx=0x7fbb0c009b20 comp rx=0 tx=0).stop 2024-09-13T18:30:36.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.104+0000 7fbb20c96740 1 -- 172.21.15.114:0/3675120536 shutdown_connections 2024-09-13T18:30:36.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.104+0000 7fbb20c96740 1 --2- 172.21.15.114:0/3675120536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff2f749b30 0x55ff2f74bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.104+0000 7fbb20c96740 1 --2- 172.21.15.114:0/3675120536 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff2f73f290 0x55ff2f7495f0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.104+0000 7fbb20c96740 1 --2- 172.21.15.114:0/3675120536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff2f73e970 0x55ff2f73ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.104+0000 7fbb20c96740 1 -- 172.21.15.114:0/3675120536 >> 172.21.15.114:0/3675120536 conn(0x55ff2f73aee0 msgr2=0x55ff2f73b2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:36.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.105+0000 7fbb177fe640 1 --2- 172.21.15.114:0/1443514581 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbb080a1410 0x7fbb080a38d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbb04009780 tx=0x7fbb04009b80 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:36.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.105+0000 7fbb20c96740 1 -- 172.21.15.114:0/3675120536 shutdown_connections 2024-09-13T18:30:36.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.105+0000 7fbb20c96740 1 -- 172.21.15.114:0/3675120536 wait complete. 2024-09-13T18:30:36.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.105+0000 7fbb20c96740 1 -- start start 2024-09-13T18:30:36.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.106+0000 7fbb20c96740 1 Processor -- start 2024-09-13T18:30:36.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.106+0000 7fbb20c96740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff2f73e970 0x55ff2f746480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:36.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.106+0000 7fbb20c96740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff2f73f290 0x55ff2f7469c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:36.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.106+0000 7fbb20c96740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff2f749b30 0x55ff2f748af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:36.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.106+0000 7fbb20c96740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff2f660180 con 0x55ff2f749b30 2024-09-13T18:30:36.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.106+0000 7fbb20c96740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff2f37a430 con 0x55ff2f73f290 2024-09-13T18:30:36.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.106+0000 7fbb20c96740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff2f6255e0 con 0x55ff2f73e970 2024-09-13T18:30:36.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.106+0000 7fbb20c96740 10 client.0 _ll_register_callbacks cb 0x55ff2f691aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:30:36.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.107+0000 7fbb177fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff2f73e970 0x55ff2f746480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:36.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.107+0000 7fbb16ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff2f73f290 0x55ff2f7469c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:36.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.107+0000 7fbb16ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff2f73f290 0x55ff2f7469c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:40338/0 (socket says 172.21.15.114:40338) 2024-09-13T18:30:36.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.107+0000 7fbb16ffd640 1 -- 172.21.15.114:0/806988668 learned_addr learned my addr 172.21.15.114:0/806988668 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:36.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.108+0000 7fbb17fff640 1 --2- 172.21.15.114:0/806988668 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff2f749b30 0x55ff2f748af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:36.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.108+0000 7fbb16ffd640 1 -- 172.21.15.114:0/806988668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff2f73e970 msgr2=0x55ff2f746480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:36.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.108+0000 7fbb16ffd640 1 --2- 172.21.15.114:0/806988668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff2f73e970 0x55ff2f746480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.108+0000 7fbb16ffd640 1 -- 172.21.15.114:0/806988668 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff2f749b30 msgr2=0x55ff2f748af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:36.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.108+0000 7fbb16ffd640 1 --2- 172.21.15.114:0/806988668 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff2f749b30 0x55ff2f748af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.108+0000 7fbb16ffd640 1 -- 172.21.15.114:0/806988668 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb0c0092f0 con 0x55ff2f73f290 2024-09-13T18:30:36.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.109+0000 7fbb16ffd640 1 --2- 172.21.15.114:0/806988668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff2f73f290 0x55ff2f7469c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c011af0 tx=0x7fbb0c011650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c34f2f9ffa45fd14 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:36.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.109+0000 7fbaf6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:30:36.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.115+0000 7fbaf6ffd640 1 -- 172.21.15.114:0/806988668 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c004440 con 0x55ff2f73f290 2024-09-13T18:30:36.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.115+0000 7fbaf6ffd640 1 -- 172.21.15.114:0/806988668 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fbb0c00b630 con 0x55ff2f73f290 2024-09-13T18:30:36.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.119+0000 7fbb20c96740 1 -- 172.21.15.114:0/806988668 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ff2f6941f0 con 0x55ff2f73f290 2024-09-13T18:30:36.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.121+0000 7fbaf6ffd640 1 -- 172.21.15.114:0/806988668 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb0c0045a0 con 0x55ff2f73f290 2024-09-13T18:30:36.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.121+0000 7fbb20c96740 10 client.27293 fetch_fsmap learned FSMap version 34 2024-09-13T18:30:36.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.121+0000 7fbb20c96740 1 -- 172.21.15.114:0/806988668 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55ff2f6a03c0 con 0x55ff2f73f290 2024-09-13T18:30:36.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.125+0000 7fbaf6ffd640 1 -- 172.21.15.114:0/806988668 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ff2f6941f0 con 0x55ff2f73f290 2024-09-13T18:30:36.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.125+0000 7fbaf6ffd640 1 -- 172.21.15.114:0/806988668 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbb0c003c60 con 0x55ff2f73f290 2024-09-13T18:30:36.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.125+0000 7fbb20c96740 10 client.27293 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:30:36.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.125+0000 7fbb20c96740 10 client.27293 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-13T18:30:36.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.125+0000 7fbb20c96740 20 client.27293 populate_metadata read hostname 'smithi114' 2024-09-13T18:30:36.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.125+0000 7fbb20c96740 1 --2- 172.21.15.114:0/806988668 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55ff2f77b4c0 0x55ff2f77d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:36.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.125+0000 7fbb20c96740 4 client.27293 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:30:36.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.125+0000 7fbb20c96740 1 -- 172.21.15.114:0/806988668 --> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ff2f3954e0 con 0x55ff2f77b4c0 2024-09-13T18:30:36.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.128+0000 7fbb177fe640 1 --2- 172.21.15.114:0/806988668 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55ff2f77b4c0 0x55ff2f77d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:36.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.134+0000 7fbb177fe640 1 --2- 172.21.15.114:0/806988668 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55ff2f77b4c0 0x55ff2f77d8e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b698e6fc06688155 server_cookie=207419434a7d3af4 in_seq=0 out_seq=0 2024-09-13T18:30:36.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.134+0000 7fbaf6ffd640 10 client.27293 ms_handle_connect on v2:172.21.15.134:6834/1150003275 2024-09-13T18:30:36.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.135+0000 7fbaf6ffd640 1 -- 172.21.15.114:0/806988668 <== mds.3 v2:172.21.15.134:6834/1150003275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55ff2f3954e0 con 0x55ff2f77b4c0 2024-09-13T18:30:36.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.135+0000 7fbaf6ffd640 10 client.27293 handle_command_reply: tid=0 2024-09-13T18:30:36.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.137+0000 7fbb20c96740 1 client.27293 shutdown 2024-09-13T18:30:36.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.139+0000 7fbb20c96740 1 -- 172.21.15.114:0/806988668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff2f73f290 msgr2=0x55ff2f7469c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:36.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.139+0000 7fbb20c96740 1 --2- 172.21.15.114:0/806988668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff2f73f290 0x55ff2f7469c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c011af0 tx=0x7fbb0c011650 comp rx=0 tx=0).stop 2024-09-13T18:30:36.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.141+0000 7fbb20c96740 1 -- 172.21.15.114:0/806988668 shutdown_connections 2024-09-13T18:30:36.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.141+0000 7fbb20c96740 1 --2- 172.21.15.114:0/806988668 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55ff2f77b4c0 0x55ff2f77d8e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.141+0000 7fbb20c96740 1 --2- 172.21.15.114:0/806988668 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff2f749b30 0x55ff2f748af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.141+0000 7fbb20c96740 1 --2- 172.21.15.114:0/806988668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff2f73f290 0x55ff2f7469c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.141+0000 7fbb20c96740 1 --2- 172.21.15.114:0/806988668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff2f73e970 0x55ff2f746480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.141+0000 7fbb20c96740 1 -- 172.21.15.114:0/806988668 >> 172.21.15.114:0/806988668 conn(0x55ff2f73aee0 msgr2=0x55ff2f73a230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:36.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.141+0000 7fbaf6ffd640 0 client.27293 ms_handle_reset on v2:172.21.15.134:6834/1150003275 2024-09-13T18:30:36.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.143+0000 7fbb20c96740 1 -- 172.21.15.114:0/806988668 shutdown_connections 2024-09-13T18:30:36.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.144+0000 7fbb20c96740 1 -- 172.21.15.114:0/806988668 wait complete. 2024-09-13T18:30:36.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.148+0000 7fbb20c96740 20 client.27293 trim_cache size 0 max 16384 2024-09-13T18:30:36.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.172+0000 7fbb20c96740 1 Processor -- start 2024-09-13T18:30:36.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.173+0000 7fbb20c96740 1 -- start start 2024-09-13T18:30:36.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.173+0000 7fbb20c96740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff2f73e970 0x55ff2f748880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:36.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.173+0000 7fbb20c96740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff2f73f290 0x55ff2f744b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:36.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.173+0000 7fbb20c96740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff2f749b30 0x55ff2f745080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:36.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.173+0000 7fbb20c96740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff2f741350 con 0x55ff2f749b30 2024-09-13T18:30:36.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.173+0000 7fbb20c96740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff2f73f800 con 0x55ff2f73e970 2024-09-13T18:30:36.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.173+0000 7fbb20c96740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff2f7455c0 con 0x55ff2f73f290 2024-09-13T18:30:36.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.173+0000 7fbb16ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff2f73f290 0x55ff2f744b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:36.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.174+0000 7fbb177fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff2f73e970 0x55ff2f748880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:36.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.174+0000 7fbb16ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff2f73f290 0x55ff2f744b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:58822/0 (socket says 172.21.15.114:58822) 2024-09-13T18:30:36.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.174+0000 7fbb16ffd640 1 -- 172.21.15.114:0/2030690256 learned_addr learned my addr 172.21.15.114:0/2030690256 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:36.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.174+0000 7fbb16ffd640 1 -- 172.21.15.114:0/2030690256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff2f73e970 msgr2=0x55ff2f748880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:36.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.174+0000 7fbb16ffd640 1 --2- 172.21.15.114:0/2030690256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff2f73e970 0x55ff2f748880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.174+0000 7fbb16ffd640 1 -- 172.21.15.114:0/2030690256 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff2f749b30 msgr2=0x55ff2f745080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:36.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.174+0000 7fbb16ffd640 1 --2- 172.21.15.114:0/2030690256 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff2f749b30 0x55ff2f745080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.174+0000 7fbb16ffd640 1 -- 172.21.15.114:0/2030690256 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff2f642770 con 0x55ff2f73f290 2024-09-13T18:30:36.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.175+0000 7fbb16ffd640 1 --2- 172.21.15.114:0/2030690256 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff2f73f290 0x55ff2f744b40 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c001f70 tx=0x7fbb0c010710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=991182d9b507c992 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:36.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.175+0000 7fbb027fc640 1 -- 172.21.15.114:0/2030690256 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c006a00 con 0x55ff2f73f290 2024-09-13T18:30:36.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.176+0000 7fbb027fc640 1 -- 172.21.15.114:0/2030690256 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbb0c011770 con 0x55ff2f73f290 2024-09-13T18:30:36.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.177+0000 7fbb027fc640 1 -- 172.21.15.114:0/2030690256 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c010ed0 con 0x55ff2f73f290 2024-09-13T18:30:36.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.177+0000 7fbb20c96740 1 -- 172.21.15.114:0/2030690256 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff2f73f290 msgr2=0x55ff2f744b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:36.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.177+0000 7fbb20c96740 1 --2- 172.21.15.114:0/2030690256 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff2f73f290 0x55ff2f744b40 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c001f70 tx=0x7fbb0c010710 comp rx=0 tx=0).stop 2024-09-13T18:30:36.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.181+0000 7fbb20c96740 1 -- 172.21.15.114:0/2030690256 shutdown_connections 2024-09-13T18:30:36.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.181+0000 7fbb20c96740 1 --2- 172.21.15.114:0/2030690256 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff2f749b30 0x55ff2f745080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.181+0000 7fbb20c96740 1 --2- 172.21.15.114:0/2030690256 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff2f73f290 0x55ff2f744b40 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.181+0000 7fbb20c96740 1 --2- 172.21.15.114:0/2030690256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff2f73e970 0x55ff2f748880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.181+0000 7fbb20c96740 1 -- 172.21.15.114:0/2030690256 >> 172.21.15.114:0/2030690256 conn(0x55ff2f73aee0 msgr2=0x55ff2f741540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:36.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.184+0000 7fbb20c96740 1 -- 172.21.15.114:0/2030690256 shutdown_connections 2024-09-13T18:30:36.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.185+0000 7fbb20c96740 1 -- 172.21.15.114:0/2030690256 wait complete. 2024-09-13T18:30:36.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.185+0000 7fbb20c96740 1 -- start start 2024-09-13T18:30:36.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.185+0000 7fbb20c96740 1 Processor -- start 2024-09-13T18:30:36.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.186+0000 7fbb20c96740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff2f73e970 0x55ff2f74e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:36.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.186+0000 7fbb20c96740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff2f73f290 0x55ff2f74ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:36.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.186+0000 7fbb20c96740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff2f749b30 0x55ff2f7d09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:36.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.186+0000 7fbb20c96740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff2f74f7e0 con 0x55ff2f73f290 2024-09-13T18:30:36.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.186+0000 7fbb20c96740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff2f7d0f00 con 0x55ff2f73e970 2024-09-13T18:30:36.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.186+0000 7fbb20c96740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff2f7d1040 con 0x55ff2f749b30 2024-09-13T18:30:36.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.187+0000 7fbb20c96740 10 client.0 _ll_register_callbacks cb 0x7fbb0c006e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:30:36.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.187+0000 7fbb177fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff2f73e970 0x55ff2f74e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:36.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.187+0000 7fbb17fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff2f749b30 0x55ff2f7d09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:36.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.187+0000 7fbb17fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff2f749b30 0x55ff2f7d09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:58828/0 (socket says 172.21.15.114:58828) 2024-09-13T18:30:36.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.187+0000 7fbb17fff640 1 -- 172.21.15.114:0/3927993861 learned_addr learned my addr 172.21.15.114:0/3927993861 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:36.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.191+0000 7fbb16ffd640 1 --2- 172.21.15.114:0/3927993861 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff2f73f290 0x55ff2f74ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:36.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.194+0000 7fbb17fff640 1 -- 172.21.15.114:0/3927993861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff2f73e970 msgr2=0x55ff2f74e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:36.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.194+0000 7fbb17fff640 1 --2- 172.21.15.114:0/3927993861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff2f73e970 0x55ff2f74e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.194+0000 7fbb17fff640 1 -- 172.21.15.114:0/3927993861 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff2f73f290 msgr2=0x55ff2f74ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:36.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.194+0000 7fbb17fff640 1 --2- 172.21.15.114:0/3927993861 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff2f73f290 0x55ff2f74ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.194+0000 7fbb17fff640 1 -- 172.21.15.114:0/3927993861 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ff2f6a03c0 con 0x55ff2f749b30 2024-09-13T18:30:36.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.199+0000 7fbb17fff640 1 --2- 172.21.15.114:0/3927993861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff2f749b30 0x55ff2f7d09c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbb1804f480 tx=0x7fbb18066740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=accf910eab3d1824 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:36.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.200+0000 7fbaf6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:30:36.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.200+0000 7fbaf6ffd640 1 -- 172.21.15.114:0/3927993861 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb18134fa0 con 0x55ff2f749b30 2024-09-13T18:30:36.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.200+0000 7fbb20c96740 1 -- 172.21.15.114:0/3927993861 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ff2f6cbb80 con 0x55ff2f749b30 2024-09-13T18:30:36.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.200+0000 7fbaf6ffd640 1 -- 172.21.15.114:0/3927993861 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fbb1815d3e0 con 0x55ff2f749b30 2024-09-13T18:30:36.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.202+0000 7fbaf6ffd640 1 -- 172.21.15.114:0/3927993861 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb1815d5a0 con 0x55ff2f749b30 2024-09-13T18:30:36.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.202+0000 7fbb20c96740 10 client.27283 fetch_fsmap learned FSMap version 34 2024-09-13T18:30:36.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.202+0000 7fbb20c96740 1 -- 172.21.15.114:0/3927993861 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55ff2f667e80 con 0x55ff2f749b30 2024-09-13T18:30:36.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.204+0000 7fbaf6ffd640 1 -- 172.21.15.114:0/3927993861 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ff2f6cbb80 con 0x55ff2f749b30 2024-09-13T18:30:36.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.204+0000 7fbaf6ffd640 1 -- 172.21.15.114:0/3927993861 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbb180be020 con 0x55ff2f749b30 2024-09-13T18:30:36.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.204+0000 7fbb20c96740 10 client.27283 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:30:36.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.204+0000 7fbb20c96740 10 client.27283 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-13T18:30:36.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.204+0000 7fbb20c96740 20 client.27283 populate_metadata read hostname 'smithi114' 2024-09-13T18:30:36.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.204+0000 7fbb20c96740 1 --2- 172.21.15.114:0/3927993861 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55ff2f7d2510 0x55ff2f7d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:36.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.204+0000 7fbb20c96740 4 client.27283 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:30:36.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.204+0000 7fbb20c96740 1 -- 172.21.15.114:0/3927993861 --> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ff2f3a67e0 con 0x55ff2f7d2510 2024-09-13T18:30:36.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.204+0000 7fbb177fe640 1 --2- 172.21.15.114:0/3927993861 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55ff2f7d2510 0x55ff2f7d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:36.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.205+0000 7fbb177fe640 1 --2- 172.21.15.114:0/3927993861 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55ff2f7d2510 0x55ff2f7d4900 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=85b890f6198a1dd1 server_cookie=7052b8e94c8aab75 in_seq=0 out_seq=0 2024-09-13T18:30:36.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.206+0000 7fbaf6ffd640 10 client.27283 ms_handle_connect on v2:172.21.15.134:6834/1150003275 2024-09-13T18:30:36.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.210+0000 7fbaf6ffd640 1 -- 172.21.15.114:0/3927993861 <== mds.3 v2:172.21.15.134:6834/1150003275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ff2f3a67e0 con 0x55ff2f7d2510 2024-09-13T18:30:36.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.210+0000 7fbaf6ffd640 10 client.27283 handle_command_reply: tid=0 2024-09-13T18:30:36.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.210+0000 7fbb20c96740 1 client.27283 shutdown 2024-09-13T18:30:36.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.216+0000 7fbb20c96740 1 -- 172.21.15.114:0/3927993861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff2f749b30 msgr2=0x55ff2f7d09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:36.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.216+0000 7fbb20c96740 1 --2- 172.21.15.114:0/3927993861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff2f749b30 0x55ff2f7d09c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbb1804f480 tx=0x7fbb18066740 comp rx=0 tx=0).stop 2024-09-13T18:30:36.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.216+0000 7fbb20c96740 1 -- 172.21.15.114:0/3927993861 shutdown_connections 2024-09-13T18:30:36.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.216+0000 7fbb20c96740 1 --2- 172.21.15.114:0/3927993861 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55ff2f7d2510 0x55ff2f7d4900 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.216+0000 7fbb20c96740 1 --2- 172.21.15.114:0/3927993861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff2f749b30 0x55ff2f7d09c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.216+0000 7fbb20c96740 1 --2- 172.21.15.114:0/3927993861 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ff2f73f290 0x55ff2f74ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.216+0000 7fbb20c96740 1 --2- 172.21.15.114:0/3927993861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff2f73e970 0x55ff2f74e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.216+0000 7fbb20c96740 1 -- 172.21.15.114:0/3927993861 >> 172.21.15.114:0/3927993861 conn(0x55ff2f73aee0 msgr2=0x55ff2f744a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:36.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.217+0000 7fbaf6ffd640 0 client.27283 ms_handle_reset on v2:172.21.15.134:6834/1150003275 2024-09-13T18:30:36.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.218+0000 7fbb20c96740 1 -- 172.21.15.114:0/3927993861 shutdown_connections 2024-09-13T18:30:36.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.223+0000 7fbb20c96740 1 -- 172.21.15.114:0/3927993861 wait complete. 2024-09-13T18:30:36.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.225+0000 7fbb20c96740 20 client.27283 trim_cache size 0 max 16384 2024-09-13T18:30:36.229 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:30:36.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.228+0000 7fbb1df76640 1 -- 172.21.15.114:0/1443514581 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbb080a1410 msgr2=0x7fbb080a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:36.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.229+0000 7fbb1df76640 1 --2- 172.21.15.114:0/1443514581 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbb080a1410 0x7fbb080a38d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbb04009780 tx=0x7fbb04009b80 comp rx=0 tx=0).stop 2024-09-13T18:30:36.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.229+0000 7fbb1df76640 1 -- 172.21.15.114:0/1443514581 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb100b7540 msgr2=0x7fbb100b2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:36.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.229+0000 7fbb1df76640 1 --2- 172.21.15.114:0/1443514581 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb100b7540 0x7fbb100b2cf0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fbb18000b60 tx=0x7fbb18066f60 comp rx=0 tx=0).stop 2024-09-13T18:30:36.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.229+0000 7fbb1df76640 1 -- 172.21.15.114:0/1443514581 shutdown_connections 2024-09-13T18:30:36.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.229+0000 7fbb1df76640 1 --2- 172.21.15.114:0/1443514581 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbb080a1410 0x7fbb080a38d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.229+0000 7fbb1df76640 1 --2- 172.21.15.114:0/1443514581 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb100b7540 0x7fbb100b2cf0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.229+0000 7fbb1df76640 1 --2- 172.21.15.114:0/1443514581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb100a5140 0x7fbb100b6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.229+0000 7fbb1df76640 1 --2- 172.21.15.114:0/1443514581 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb100a4820 0x7fbb100b6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.229+0000 7fbb1df76640 1 -- 172.21.15.114:0/1443514581 >> 172.21.15.114:0/1443514581 conn(0x7fbb1001a110 msgr2=0x7fbb100b7d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:36.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.230+0000 7fbb1df76640 1 -- 172.21.15.114:0/1443514581 shutdown_connections 2024-09-13T18:30:36.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.230+0000 7fbb1df76640 1 -- 172.21.15.114:0/1443514581 wait complete. 2024-09-13T18:30:36.263 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-09-13T18:30:36.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.505+0000 7f0d8759e640 1 -- 172.21.15.114:0/49477489 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d88107890 msgr2=0x7f0d88107cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:36.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.505+0000 7f0d8759e640 1 --2- 172.21.15.114:0/49477489 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d88107890 0x7f0d88107cd0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c009900 tx=0x7f0d7c02a440 comp rx=0 tx=0).stop 2024-09-13T18:30:36.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.524+0000 7f0d8759e640 1 -- 172.21.15.114:0/49477489 shutdown_connections 2024-09-13T18:30:36.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.524+0000 7f0d8759e640 1 --2- 172.21.15.114:0/49477489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d88119400 0x7f0d8811b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.524+0000 7f0d8759e640 1 --2- 172.21.15.114:0/49477489 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d88107890 0x7f0d88107cd0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.524+0000 7f0d8759e640 1 --2- 172.21.15.114:0/49477489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d881074b0 0x7f0d8810e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.524+0000 7f0d8759e640 1 -- 172.21.15.114:0/49477489 >> 172.21.15.114:0/49477489 conn(0x7f0d8806af40 msgr2=0x7f0d8806b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:36.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.528+0000 7f0d8759e640 1 -- 172.21.15.114:0/49477489 shutdown_connections 2024-09-13T18:30:36.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.532+0000 7f0d8759e640 1 -- 172.21.15.114:0/49477489 wait complete. 2024-09-13T18:30:36.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.533+0000 7f0d8759e640 1 Processor -- start 2024-09-13T18:30:36.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.534+0000 7f0d8759e640 1 -- start start 2024-09-13T18:30:36.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.535+0000 7f0d8759e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d881074b0 0x7f0d881a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:36.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.535+0000 7f0d8759e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d88107890 0x7f0d881a5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:36.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.535+0000 7f0d8759e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d88119400 0x7f0d881a9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:36.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.535+0000 7f0d8759e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d881a9f80 con 0x7f0d881074b0 2024-09-13T18:30:36.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.535+0000 7f0d8759e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d881aa0c0 con 0x7f0d88119400 2024-09-13T18:30:36.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.535+0000 7f0d8759e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d881aa280 con 0x7f0d88107890 2024-09-13T18:30:36.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.537+0000 7f0d86d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d88119400 0x7f0d881a9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:36.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.537+0000 7f0d86d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d88119400 0x7f0d881a9980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:40368/0 (socket says 172.21.15.114:40368) 2024-09-13T18:30:36.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.537+0000 7f0d86d9d640 1 -- 172.21.15.114:0/1038553759 learned_addr learned my addr 172.21.15.114:0/1038553759 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:36.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.538+0000 7f0d86d9d640 1 -- 172.21.15.114:0/1038553759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d88107890 msgr2=0x7f0d881a5590 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:30:36.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.538+0000 7f0d86d9d640 1 --2- 172.21.15.114:0/1038553759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d88107890 0x7f0d881a5590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.538+0000 7f0d86d9d640 1 -- 172.21.15.114:0/1038553759 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d881074b0 msgr2=0x7f0d881a5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:36.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.538+0000 7f0d86d9d640 1 --2- 172.21.15.114:0/1038553759 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d881074b0 0x7f0d881a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.538+0000 7f0d86d9d640 1 -- 172.21.15.114:0/1038553759 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d7c009560 con 0x7f0d88119400 2024-09-13T18:30:36.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.542+0000 7f0d86d9d640 1 --2- 172.21.15.114:0/1038553759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d88119400 0x7f0d881a9980 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f0d8000b9a0 tx=0x7f0d8000be60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:36.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.546+0000 7f0d777fe640 1 -- 172.21.15.114:0/1038553759 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8000cba0 con 0x7f0d88119400 2024-09-13T18:30:36.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.546+0000 7f0d8759e640 1 -- 172.21.15.114:0/1038553759 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d881aa560 con 0x7f0d88119400 2024-09-13T18:30:36.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.546+0000 7f0d8759e640 1 -- 172.21.15.114:0/1038553759 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d8810edc0 con 0x7f0d88119400 2024-09-13T18:30:36.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.546+0000 7f0d777fe640 1 -- 172.21.15.114:0/1038553759 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0d8000cd00 con 0x7f0d88119400 2024-09-13T18:30:36.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.546+0000 7f0d777fe640 1 -- 172.21.15.114:0/1038553759 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d800075d0 con 0x7f0d88119400 2024-09-13T18:30:36.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.550+0000 7f0d8f664740 1 Processor -- start 2024-09-13T18:30:36.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.550+0000 7f0d8f664740 1 -- start start 2024-09-13T18:30:36.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.551+0000 7f0d8f664740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ed6ccd9970 0x55ed6ccd9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:36.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.551+0000 7f0d8f664740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed6ccda290 0x55ed6cce45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:36.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.551+0000 7f0d8f664740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed6cce4b30 0x55ed6cce6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:36.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.551+0000 7f0d8f664740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed6cc43cc0 con 0x55ed6ccd9970 2024-09-13T18:30:36.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.551+0000 7f0d8f664740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed6cc41450 con 0x55ed6cce4b30 2024-09-13T18:30:36.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.551+0000 7f0d8f664740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed6cc3cd40 con 0x55ed6ccda290 2024-09-13T18:30:36.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.552+0000 7f0d85d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed6cce4b30 0x55ed6cce6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:36.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.552+0000 7f0d85d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ed6ccd9970 0x55ed6ccd9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:36.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.552+0000 7f0d85d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ed6ccd9970 0x55ed6ccd9d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57924/0 (socket says 172.21.15.114:57924) 2024-09-13T18:30:36.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.552+0000 7f0d85d9b640 1 -- 172.21.15.114:0/2619658062 learned_addr learned my addr 172.21.15.114:0/2619658062 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:36.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.553+0000 7f0d8659c640 1 --2- 172.21.15.114:0/2619658062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed6ccda290 0x55ed6cce45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:36.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.554+0000 7f0d777fe640 1 -- 172.21.15.114:0/1038553759 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f0d80014440 con 0x7f0d88119400 2024-09-13T18:30:36.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.555+0000 7f0d85d9b640 1 -- 172.21.15.114:0/2619658062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed6ccda290 msgr2=0x55ed6cce45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:36.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.555+0000 7f0d85d9b640 1 --2- 172.21.15.114:0/2619658062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed6ccda290 0x55ed6cce45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.555+0000 7f0d85d9b640 1 -- 172.21.15.114:0/2619658062 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ed6ccd9970 msgr2=0x55ed6ccd9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:36.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.555+0000 7f0d85d9b640 1 --2- 172.21.15.114:0/2619658062 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ed6ccd9970 0x55ed6ccd9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.555+0000 7f0d85d9b640 1 -- 172.21.15.114:0/2619658062 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed6cc04620 con 0x55ed6cce4b30 2024-09-13T18:30:36.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.555+0000 7f0d85d9b640 1 --2- 172.21.15.114:0/2619658062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed6cce4b30 0x55ed6cce6f20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c033000 tx=0x7f0d7c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=897942f04a5081ee server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:36.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.559+0000 7f0d74ff9640 1 -- 172.21.15.114:0/2619658062 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c004230 con 0x55ed6cce4b30 2024-09-13T18:30:36.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.564+0000 7f0d74ff9640 1 -- 172.21.15.114:0/2619658062 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0d7c044030 con 0x55ed6cce4b30 2024-09-13T18:30:36.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.564+0000 7f0d74ff9640 1 -- 172.21.15.114:0/2619658062 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c003850 con 0x55ed6cce4b30 2024-09-13T18:30:36.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.564+0000 7f0d777fe640 1 --2- 172.21.15.114:0/1038553759 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0d700a1410 0x7f0d700a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:36.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.564+0000 7f0d777fe640 1 -- 172.21.15.114:0/1038553759 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f0d800cc770 con 0x7f0d88119400 2024-09-13T18:30:36.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.569+0000 7f0d8659c640 1 --2- 172.21.15.114:0/1038553759 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0d700a1410 0x7f0d700a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:36.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.571+0000 7f0d8f664740 1 -- 172.21.15.114:0/2619658062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed6cce4b30 msgr2=0x55ed6cce6f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:36.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.571+0000 7f0d8f664740 1 --2- 172.21.15.114:0/2619658062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed6cce4b30 0x55ed6cce6f20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c033000 tx=0x7f0d7c000bc0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.577+0000 7f0d8659c640 1 --2- 172.21.15.114:0/1038553759 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0d700a1410 0x7f0d700a38d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x55ed6ccd6d40 tx=0x7f0d78008040 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:36.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.578+0000 7f0d8f664740 1 -- 172.21.15.114:0/2619658062 shutdown_connections 2024-09-13T18:30:36.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.578+0000 7f0d8f664740 1 --2- 172.21.15.114:0/2619658062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed6cce4b30 0x55ed6cce6f20 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.578+0000 7f0d8f664740 1 --2- 172.21.15.114:0/2619658062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed6ccda290 0x55ed6cce45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.578+0000 7f0d8f664740 1 --2- 172.21.15.114:0/2619658062 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ed6ccd9970 0x55ed6ccd9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.578+0000 7f0d8f664740 1 -- 172.21.15.114:0/2619658062 >> 172.21.15.114:0/2619658062 conn(0x55ed6ccd5ee0 msgr2=0x55ed6ccd62c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:36.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.585+0000 7f0d8f664740 1 -- 172.21.15.114:0/2619658062 shutdown_connections 2024-09-13T18:30:36.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.593+0000 7f0d8f664740 1 -- 172.21.15.114:0/2619658062 wait complete. 2024-09-13T18:30:36.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.593+0000 7f0d8f664740 1 -- start start 2024-09-13T18:30:36.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.593+0000 7f0d8f664740 1 Processor -- start 2024-09-13T18:30:36.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.593+0000 7f0d8f664740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ed6ccd9970 0x55ed6cce4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:36.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.593+0000 7f0d8f664740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed6ccda290 0x55ed6cce1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:36.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.593+0000 7f0d8f664740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed6cce4b30 0x55ed6cce19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:36.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.593+0000 7f0d8f664740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed6cbfb180 con 0x55ed6ccd9970 2024-09-13T18:30:36.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.593+0000 7f0d8f664740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed6c915430 con 0x55ed6ccda290 2024-09-13T18:30:36.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.593+0000 7f0d8f664740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed6cbc05e0 con 0x55ed6cce4b30 2024-09-13T18:30:36.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.594+0000 7f0d8f664740 10 client.0 _ll_register_callbacks cb 0x55ed6cc2caa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:30:36.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.594+0000 7f0d86d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed6cce4b30 0x55ed6cce19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:36.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.595+0000 7f0d86d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed6cce4b30 0x55ed6cce19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:58846/0 (socket says 172.21.15.114:58846) 2024-09-13T18:30:36.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.595+0000 7f0d86d9d640 1 -- 172.21.15.114:0/3579141819 learned_addr learned my addr 172.21.15.114:0/3579141819 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:36.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.595+0000 7f0d8659c640 1 --2- 172.21.15.114:0/3579141819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ed6ccd9970 0x55ed6cce4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:36.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.596+0000 7f0d85d9b640 1 --2- 172.21.15.114:0/3579141819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed6ccda290 0x55ed6cce1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:36.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.596+0000 7f0d86d9d640 1 -- 172.21.15.114:0/3579141819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed6ccda290 msgr2=0x55ed6cce1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:36.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.596+0000 7f0d86d9d640 1 --2- 172.21.15.114:0/3579141819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed6ccda290 0x55ed6cce1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.596+0000 7f0d86d9d640 1 -- 172.21.15.114:0/3579141819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ed6ccd9970 msgr2=0x55ed6cce4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:36.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.596+0000 7f0d86d9d640 1 --2- 172.21.15.114:0/3579141819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ed6ccd9970 0x55ed6cce4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.596+0000 7f0d86d9d640 1 -- 172.21.15.114:0/3579141819 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d8811c170 con 0x55ed6cce4b30 2024-09-13T18:30:36.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.596+0000 7f0d86d9d640 1 --2- 172.21.15.114:0/3579141819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed6cce4b30 0x55ed6cce19c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0d80095a90 tx=0x7f0d80099d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a328df6c9427cf6a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:36.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.598+0000 7f0d6dffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:30:36.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.598+0000 7f0d6dffb640 1 -- 172.21.15.114:0/3579141819 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d800c9f50 con 0x55ed6cce4b30 2024-09-13T18:30:36.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.598+0000 7f0d6dffb640 1 -- 172.21.15.114:0/3579141819 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f0d800a1a10 con 0x55ed6cce4b30 2024-09-13T18:30:36.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.600+0000 7f0d8f664740 1 -- 172.21.15.114:0/3579141819 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ed6cc66b80 con 0x55ed6cce4b30 2024-09-13T18:30:36.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.603+0000 7f0d6dffb640 1 -- 172.21.15.114:0/3579141819 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d800cc330 con 0x55ed6cce4b30 2024-09-13T18:30:36.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.605+0000 7f0d8f664740 10 client.27289 fetch_fsmap learned FSMap version 34 2024-09-13T18:30:36.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.605+0000 7f0d8f664740 1 -- 172.21.15.114:0/3579141819 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55ed6cc3b3c0 con 0x55ed6cce4b30 2024-09-13T18:30:36.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.607+0000 7f0d6dffb640 1 -- 172.21.15.114:0/3579141819 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ed6cc66b80 con 0x55ed6cce4b30 2024-09-13T18:30:36.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.607+0000 7f0d6dffb640 1 -- 172.21.15.114:0/3579141819 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0d800cc050 con 0x55ed6cce4b30 2024-09-13T18:30:36.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.610+0000 7f0d8f664740 10 client.27289 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:30:36.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.610+0000 7f0d8f664740 10 client.27289 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-13T18:30:36.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.610+0000 7f0d8f664740 20 client.27289 populate_metadata read hostname 'smithi114' 2024-09-13T18:30:36.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.610+0000 7f0d8f664740 1 --2- 172.21.15.114:0/3579141819 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55ed6ccf45a0 0x55ed6ccf69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:36.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.610+0000 7f0d8f664740 4 client.27289 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:30:36.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.610+0000 7f0d8f664740 1 -- 172.21.15.114:0/3579141819 --> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ed6cc2f1f0 con 0x55ed6ccf45a0 2024-09-13T18:30:36.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.614+0000 7f0d8659c640 1 --2- 172.21.15.114:0/3579141819 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55ed6ccf45a0 0x55ed6ccf69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:36.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.618+0000 7f0d8659c640 1 --2- 172.21.15.114:0/3579141819 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55ed6ccf45a0 0x55ed6ccf69c0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7058d20c6d2ad899 server_cookie=cefb6e127e04ada6 in_seq=0 out_seq=0 2024-09-13T18:30:36.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.621+0000 7f0d6dffb640 10 client.27289 ms_handle_connect on v2:172.21.15.204:6838/344883912 2024-09-13T18:30:36.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.621+0000 7f0d6dffb640 1 -- 172.21.15.114:0/3579141819 <== mds.2 v2:172.21.15.204:6838/344883912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55ed6cc2f1f0 con 0x55ed6ccf45a0 2024-09-13T18:30:36.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.621+0000 7f0d6dffb640 10 client.27289 handle_command_reply: tid=0 2024-09-13T18:30:36.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.621+0000 7f0d8f664740 1 client.27289 shutdown 2024-09-13T18:30:36.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.623+0000 7f0d8f664740 1 -- 172.21.15.114:0/3579141819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed6cce4b30 msgr2=0x55ed6cce19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:36.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.623+0000 7f0d8f664740 1 --2- 172.21.15.114:0/3579141819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed6cce4b30 0x55ed6cce19c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0d80095a90 tx=0x7f0d80099d50 comp rx=0 tx=0).stop 2024-09-13T18:30:36.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.627+0000 7f0d8f664740 1 -- 172.21.15.114:0/3579141819 shutdown_connections 2024-09-13T18:30:36.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.627+0000 7f0d8f664740 1 --2- 172.21.15.114:0/3579141819 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55ed6ccf45a0 0x55ed6ccf69c0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.627+0000 7f0d8f664740 1 --2- 172.21.15.114:0/3579141819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed6cce4b30 0x55ed6cce19c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.627+0000 7f0d8f664740 1 --2- 172.21.15.114:0/3579141819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed6ccda290 0x55ed6cce1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.627+0000 7f0d8f664740 1 --2- 172.21.15.114:0/3579141819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ed6ccd9970 0x55ed6cce4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.627+0000 7f0d8f664740 1 -- 172.21.15.114:0/3579141819 >> 172.21.15.114:0/3579141819 conn(0x55ed6ccd5ee0 msgr2=0x55ed6ccd5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:36.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.629+0000 7f0d6dffb640 0 client.27289 ms_handle_reset on v2:172.21.15.204:6838/344883912 2024-09-13T18:30:36.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.634+0000 7f0d8f664740 1 -- 172.21.15.114:0/3579141819 shutdown_connections 2024-09-13T18:30:36.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.637+0000 7f0d8f664740 1 -- 172.21.15.114:0/3579141819 wait complete. 2024-09-13T18:30:36.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.644+0000 7f0d8f664740 20 client.27289 trim_cache size 0 max 16384 2024-09-13T18:30:36.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.670+0000 7f0d8f664740 1 Processor -- start 2024-09-13T18:30:36.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.680+0000 7f0d8f664740 1 -- start start 2024-09-13T18:30:36.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.680+0000 7f0d8f664740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed6ccd9970 0x55ed6cce3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:36.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.680+0000 7f0d8f664740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ed6ccda290 0x55ed6ccdfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:36.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.680+0000 7f0d8f664740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed6cce4b30 0x55ed6cce0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:36.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.680+0000 7f0d8f664740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed6cce4740 con 0x55ed6ccda290 2024-09-13T18:30:36.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.680+0000 7f0d8f664740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed6ccda930 con 0x55ed6ccd9970 2024-09-13T18:30:36.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.680+0000 7f0d8f664740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed6cce07b0 con 0x55ed6cce4b30 2024-09-13T18:30:36.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.680+0000 7f0d8659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed6ccd9970 0x55ed6cce3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:36.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.680+0000 7f0d86d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed6cce4b30 0x55ed6cce0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:36.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.680+0000 7f0d86d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed6cce4b30 0x55ed6cce0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:58858/0 (socket says 172.21.15.114:58858) 2024-09-13T18:30:36.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.680+0000 7f0d86d9d640 1 -- 172.21.15.114:0/2802017365 learned_addr learned my addr 172.21.15.114:0/2802017365 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:36.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.680+0000 7f0d85d9b640 1 --2- 172.21.15.114:0/2802017365 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ed6ccda290 0x55ed6ccdfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:36.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.681+0000 7f0d8659c640 1 -- 172.21.15.114:0/2802017365 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed6cce4b30 msgr2=0x55ed6cce0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:36.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.681+0000 7f0d8659c640 1 --2- 172.21.15.114:0/2802017365 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed6cce4b30 0x55ed6cce0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.681+0000 7f0d8659c640 1 -- 172.21.15.114:0/2802017365 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ed6ccda290 msgr2=0x55ed6ccdfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:36.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.681+0000 7f0d8659c640 1 --2- 172.21.15.114:0/2802017365 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ed6ccda290 0x55ed6ccdfd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.681+0000 7f0d8659c640 1 -- 172.21.15.114:0/2802017365 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed6cbdd770 con 0x55ed6ccd9970 2024-09-13T18:30:36.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.683+0000 7f0d8659c640 1 --2- 172.21.15.114:0/2802017365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed6ccd9970 0x55ed6cce3c50 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x55ed6ccf4980 tx=0x7f0d78013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e19df35a0b331f94 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:36.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.691+0000 7f0d757fa640 1 -- 172.21.15.114:0/2802017365 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d780240e0 con 0x55ed6ccd9970 2024-09-13T18:30:36.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.692+0000 7f0d757fa640 1 -- 172.21.15.114:0/2802017365 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0d780246d0 con 0x55ed6ccd9970 2024-09-13T18:30:36.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.692+0000 7f0d757fa640 1 -- 172.21.15.114:0/2802017365 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d78014a20 con 0x55ed6ccd9970 2024-09-13T18:30:36.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.693+0000 7f0d8f664740 1 -- 172.21.15.114:0/2802017365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed6ccd9970 msgr2=0x55ed6cce3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:36.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.693+0000 7f0d8f664740 1 --2- 172.21.15.114:0/2802017365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed6ccd9970 0x55ed6cce3c50 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x55ed6ccf4980 tx=0x7f0d78013000 comp rx=0 tx=0).stop 2024-09-13T18:30:36.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.695+0000 7f0d8f664740 1 -- 172.21.15.114:0/2802017365 shutdown_connections 2024-09-13T18:30:36.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.695+0000 7f0d8f664740 1 --2- 172.21.15.114:0/2802017365 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed6cce4b30 0x55ed6cce0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.695+0000 7f0d8f664740 1 --2- 172.21.15.114:0/2802017365 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ed6ccda290 0x55ed6ccdfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.695+0000 7f0d8f664740 1 --2- 172.21.15.114:0/2802017365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed6ccd9970 0x55ed6cce3c50 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.695+0000 7f0d8f664740 1 -- 172.21.15.114:0/2802017365 >> 172.21.15.114:0/2802017365 conn(0x55ed6ccd5ee0 msgr2=0x55ed6cce1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:36.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.699+0000 7f0d8f664740 1 -- 172.21.15.114:0/2802017365 shutdown_connections 2024-09-13T18:30:36.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.699+0000 7f0d8f664740 1 -- 172.21.15.114:0/2802017365 wait complete. 2024-09-13T18:30:36.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.699+0000 7f0d8f664740 1 -- start start 2024-09-13T18:30:36.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.699+0000 7f0d8f664740 1 Processor -- start 2024-09-13T18:30:36.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.700+0000 7f0d8f664740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed6ccd9970 0x55ed6ccebc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:36.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.700+0000 7f0d8f664740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ed6ccda290 0x55ed6cce8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:36.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.700+0000 7f0d8f664740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed6cce4b30 0x55ed6cce85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:36.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.700+0000 7f0d85d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ed6ccda290 0x55ed6cce8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:36.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.700+0000 7f0d85d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ed6ccda290 0x55ed6cce8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57952/0 (socket says 172.21.15.114:57952) 2024-09-13T18:30:36.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.700+0000 7f0d85d9b640 1 -- 172.21.15.114:0/1213853281 learned_addr learned my addr 172.21.15.114:0/1213853281 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:36.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.700+0000 7f0d8f664740 1 -- 172.21.15.114:0/1213853281 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed6cce8cd0 con 0x55ed6ccda290 2024-09-13T18:30:36.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.700+0000 7f0d8f664740 1 -- 172.21.15.114:0/1213853281 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed6cce8e40 con 0x55ed6ccd9970 2024-09-13T18:30:36.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.700+0000 7f0d8f664740 1 -- 172.21.15.114:0/1213853281 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed6cd4aac0 con 0x55ed6cce4b30 2024-09-13T18:30:36.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.700+0000 7f0d8659c640 1 --2- 172.21.15.114:0/1213853281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed6ccd9970 0x55ed6ccebc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:36.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.701+0000 7f0d8f664740 10 client.0 _ll_register_callbacks cb 0x7f0d7c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:30:36.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.701+0000 7f0d86d9d640 1 --2- 172.21.15.114:0/1213853281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed6cce4b30 0x55ed6cce85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:36.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.705+0000 7f0d8659c640 1 -- 172.21.15.114:0/1213853281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed6cce4b30 msgr2=0x55ed6cce85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:36.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.705+0000 7f0d8659c640 1 --2- 172.21.15.114:0/1213853281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed6cce4b30 0x55ed6cce85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.705+0000 7f0d8659c640 1 -- 172.21.15.114:0/1213853281 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ed6ccda290 msgr2=0x55ed6cce8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:36.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.705+0000 7f0d8659c640 1 --2- 172.21.15.114:0/1213853281 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ed6ccda290 0x55ed6cce8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.705+0000 7f0d8659c640 1 -- 172.21.15.114:0/1213853281 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d7801c040 con 0x55ed6ccd9970 2024-09-13T18:30:36.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.705+0000 7f0d85d9b640 1 --2- 172.21.15.114:0/1213853281 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ed6ccda290 0x55ed6cce8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:30:36.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.709+0000 7f0d8659c640 1 --2- 172.21.15.114:0/1213853281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed6ccd9970 0x55ed6ccebc20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x55ed6ccf4980 tx=0x7f0d78024040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=add29f85068567b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:36.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.710+0000 7f0d6dffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:30:36.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.713+0000 7f0d6dffb640 1 -- 172.21.15.114:0/1213853281 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d780244c0 con 0x55ed6ccd9970 2024-09-13T18:30:36.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.713+0000 7f0d8f664740 1 -- 172.21.15.114:0/1213853281 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0d7c044030 con 0x55ed6ccd9970 2024-09-13T18:30:36.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.713+0000 7f0d6dffb640 1 -- 172.21.15.114:0/1213853281 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f0d78014880 con 0x55ed6ccd9970 2024-09-13T18:30:36.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.721+0000 7f0d6dffb640 1 -- 172.21.15.114:0/1213853281 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d78014a80 con 0x55ed6ccd9970 2024-09-13T18:30:36.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.732+0000 7f0d8f664740 10 client.27314 fetch_fsmap learned FSMap version 34 2024-09-13T18:30:36.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.732+0000 7f0d8f664740 1 -- 172.21.15.114:0/1213853281 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55ed6cc02e80 con 0x55ed6ccd9970 2024-09-13T18:30:36.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.735+0000 7f0d6dffb640 1 -- 172.21.15.114:0/1213853281 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0d7c044030 con 0x55ed6ccd9970 2024-09-13T18:30:36.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.735+0000 7f0d6dffb640 1 -- 172.21.15.114:0/1213853281 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0d7801f040 con 0x55ed6ccd9970 2024-09-13T18:30:36.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.736+0000 7f0d8f664740 10 client.27314 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:30:36.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.736+0000 7f0d8f664740 10 client.27314 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-13T18:30:36.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.736+0000 7f0d8f664740 20 client.27314 populate_metadata read hostname 'smithi114' 2024-09-13T18:30:36.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.736+0000 7f0d8f664740 1 --2- 172.21.15.114:0/1213853281 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55ed6cd4c0b0 0x55ed6cd4e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:36.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.736+0000 7f0d8f664740 4 client.27314 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:30:36.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.736+0000 7f0d8f664740 1 -- 172.21.15.114:0/1213853281 --> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ed6c9304e0 con 0x55ed6cd4c0b0 2024-09-13T18:30:36.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.741+0000 7f0d85d9b640 1 --2- 172.21.15.114:0/1213853281 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55ed6cd4c0b0 0x55ed6cd4e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:36.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.746+0000 7f0d85d9b640 1 --2- 172.21.15.114:0/1213853281 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55ed6cd4c0b0 0x55ed6cd4e4d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3b8e8c88291f3959 server_cookie=c04a31700cc1f01b in_seq=0 out_seq=0 2024-09-13T18:30:36.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.746+0000 7f0d6dffb640 10 client.27314 ms_handle_connect on v2:172.21.15.204:6838/344883912 2024-09-13T18:30:36.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.746+0000 7f0d6dffb640 1 -- 172.21.15.114:0/1213853281 <== mds.2 v2:172.21.15.204:6838/344883912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ed6c9304e0 con 0x55ed6cd4c0b0 2024-09-13T18:30:36.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.746+0000 7f0d6dffb640 10 client.27314 handle_command_reply: tid=0 2024-09-13T18:30:36.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.746+0000 7f0d8f664740 1 client.27314 shutdown 2024-09-13T18:30:36.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.750+0000 7f0d8f664740 1 -- 172.21.15.114:0/1213853281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed6ccd9970 msgr2=0x55ed6ccebc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:36.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.750+0000 7f0d8f664740 1 --2- 172.21.15.114:0/1213853281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed6ccd9970 0x55ed6ccebc20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x55ed6ccf4980 tx=0x7f0d78024040 comp rx=0 tx=0).stop 2024-09-13T18:30:36.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.750+0000 7f0d8f664740 1 -- 172.21.15.114:0/1213853281 shutdown_connections 2024-09-13T18:30:36.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.750+0000 7f0d8f664740 1 --2- 172.21.15.114:0/1213853281 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55ed6cd4c0b0 0x55ed6cd4e4d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.750+0000 7f0d8f664740 1 --2- 172.21.15.114:0/1213853281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed6cce4b30 0x55ed6cce85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.750+0000 7f0d8f664740 1 --2- 172.21.15.114:0/1213853281 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ed6ccda290 0x55ed6cce8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.750+0000 7f0d8f664740 1 --2- 172.21.15.114:0/1213853281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed6ccd9970 0x55ed6ccebc20 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.750+0000 7f0d8f664740 1 -- 172.21.15.114:0/1213853281 >> 172.21.15.114:0/1213853281 conn(0x55ed6ccd5ee0 msgr2=0x55ed6ccf0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:36.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.751+0000 7f0d6dffb640 0 client.27314 ms_handle_reset on v2:172.21.15.204:6838/344883912 2024-09-13T18:30:36.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.753+0000 7f0d8f664740 1 -- 172.21.15.114:0/1213853281 shutdown_connections 2024-09-13T18:30:36.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.754+0000 7f0d8f664740 1 -- 172.21.15.114:0/1213853281 wait complete. 2024-09-13T18:30:36.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.754+0000 7f0d8f664740 20 client.27314 trim_cache size 0 max 16384 2024-09-13T18:30:36.758 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:30:36.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.755+0000 7f0d8759e640 1 -- 172.21.15.114:0/1038553759 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0d700a1410 msgr2=0x7f0d700a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:36.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.755+0000 7f0d8759e640 1 --2- 172.21.15.114:0/1038553759 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0d700a1410 0x7f0d700a38d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x55ed6ccd6d40 tx=0x7f0d78008040 comp rx=0 tx=0).stop 2024-09-13T18:30:36.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.755+0000 7f0d8759e640 1 -- 172.21.15.114:0/1038553759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d88119400 msgr2=0x7f0d881a9980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:36.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.755+0000 7f0d8759e640 1 --2- 172.21.15.114:0/1038553759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d88119400 0x7f0d881a9980 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f0d8000b9a0 tx=0x7f0d8000be60 comp rx=0 tx=0).stop 2024-09-13T18:30:36.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.755+0000 7f0d8759e640 1 -- 172.21.15.114:0/1038553759 shutdown_connections 2024-09-13T18:30:36.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.755+0000 7f0d8759e640 1 --2- 172.21.15.114:0/1038553759 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0d700a1410 0x7f0d700a38d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.755+0000 7f0d8759e640 1 --2- 172.21.15.114:0/1038553759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d88119400 0x7f0d881a9980 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.755+0000 7f0d8759e640 1 --2- 172.21.15.114:0/1038553759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d88107890 0x7f0d881a5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.755+0000 7f0d8759e640 1 --2- 172.21.15.114:0/1038553759 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d881074b0 0x7f0d881a5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:36.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.755+0000 7f0d8759e640 1 -- 172.21.15.114:0/1038553759 >> 172.21.15.114:0/1038553759 conn(0x7f0d8806af40 msgr2=0x7f0d88119d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:36.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.756+0000 7f0d8759e640 1 -- 172.21.15.114:0/1038553759 shutdown_connections 2024-09-13T18:30:36.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:36.756+0000 7f0d8759e640 1 -- 172.21.15.114:0/1038553759 wait complete. 2024-09-13T18:30:36.780 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-09-13T18:30:37.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.052+0000 7efcaffff640 1 -- 172.21.15.114:0/3615268704 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcb0119400 msgr2=0x7efcb011b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:37.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.052+0000 7efcaffff640 1 --2- 172.21.15.114:0/3615268704 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcb0119400 0x7efcb011b7f0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7efca0009900 tx=0x7efca002a420 comp rx=0 tx=0).stop 2024-09-13T18:30:37.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.057+0000 7efcaffff640 1 -- 172.21.15.114:0/3615268704 shutdown_connections 2024-09-13T18:30:37.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.057+0000 7efcaffff640 1 --2- 172.21.15.114:0/3615268704 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcb0119400 0x7efcb011b7f0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.057+0000 7efcaffff640 1 --2- 172.21.15.114:0/3615268704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcb0107890 0x7efcb0107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.057+0000 7efcaffff640 1 --2- 172.21.15.114:0/3615268704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efcb01074b0 0x7efcb010e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.057+0000 7efcaffff640 1 -- 172.21.15.114:0/3615268704 >> 172.21.15.114:0/3615268704 conn(0x7efcb006af40 msgr2=0x7efcb006b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:37.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.064+0000 7efcaffff640 1 -- 172.21.15.114:0/3615268704 shutdown_connections 2024-09-13T18:30:37.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.070+0000 7efcaffff640 1 -- 172.21.15.114:0/3615268704 wait complete. 2024-09-13T18:30:37.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.070+0000 7efcaffff640 1 Processor -- start 2024-09-13T18:30:37.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.077+0000 7efcaffff640 1 -- start start 2024-09-13T18:30:37.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.077+0000 7efcaffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efcb01074b0 0x7efcb0118810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:37.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.077+0000 7efcaffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcb0107890 0x7efcb0118d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:37.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.077+0000 7efcaffff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcb0119400 0x7efcb0112900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:37.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.077+0000 7efcaffff640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb01130e0 con 0x7efcb0119400 2024-09-13T18:30:37.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.077+0000 7efcaffff640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb0113250 con 0x7efcb0107890 2024-09-13T18:30:37.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.077+0000 7efcaffff640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb0113410 con 0x7efcb01074b0 2024-09-13T18:30:37.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.082+0000 7efcae7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcb0107890 0x7efcb0118d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:37.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.082+0000 7efcae7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcb0107890 0x7efcb0118d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:40422/0 (socket says 172.21.15.114:40422) 2024-09-13T18:30:37.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.082+0000 7efcae7fc640 1 -- 172.21.15.114:0/1900778425 learned_addr learned my addr 172.21.15.114:0/1900778425 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:37.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.083+0000 7efcae7fc640 1 -- 172.21.15.114:0/1900778425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efcb01074b0 msgr2=0x7efcb0118810 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:30:37.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.083+0000 7efcae7fc640 1 --2- 172.21.15.114:0/1900778425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efcb01074b0 0x7efcb0118810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.083+0000 7efcae7fc640 1 -- 172.21.15.114:0/1900778425 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcb0119400 msgr2=0x7efcb0112900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:37.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.083+0000 7efcae7fc640 1 --2- 172.21.15.114:0/1900778425 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcb0119400 0x7efcb0112900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.083+0000 7efcae7fc640 1 -- 172.21.15.114:0/1900778425 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca0009560 con 0x7efcb0107890 2024-09-13T18:30:37.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.084+0000 7efcae7fc640 1 --2- 172.21.15.114:0/1900778425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcb0107890 0x7efcb0118d50 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7efca800cce0 tx=0x7efca800a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:37.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.084+0000 7efc97fff640 1 -- 172.21.15.114:0/1900778425 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca8004b20 con 0x7efcb0107890 2024-09-13T18:30:37.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.084+0000 7efcaffff640 1 -- 172.21.15.114:0/1900778425 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcb0113690 con 0x7efcb0107890 2024-09-13T18:30:37.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.084+0000 7efcaffff640 1 -- 172.21.15.114:0/1900778425 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcb01ba490 con 0x7efcb0107890 2024-09-13T18:30:37.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.085+0000 7efc97fff640 1 -- 172.21.15.114:0/1900778425 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efca8004c80 con 0x7efcb0107890 2024-09-13T18:30:37.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.085+0000 7efc97fff640 1 -- 172.21.15.114:0/1900778425 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca8004b20 con 0x7efcb0107890 2024-09-13T18:30:37.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.090+0000 7efcb8257740 1 Processor -- start 2024-09-13T18:30:37.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.092+0000 7efcb8257740 1 -- start start 2024-09-13T18:30:37.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.093+0000 7efcb8257740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596ed7b1970 0x5596ed7b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:37.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.093+0000 7efcb8257740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596ed7b2290 0x5596ed7bc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:37.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.093+0000 7efcb8257740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596ed7bcb30 0x5596ed7bef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:37.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.093+0000 7efcb8257740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596ed71bcc0 con 0x5596ed7b1970 2024-09-13T18:30:37.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.093+0000 7efcb8257740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596ed719450 con 0x5596ed7b2290 2024-09-13T18:30:37.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.093+0000 7efcb8257740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596ed714d40 con 0x5596ed7bcb30 2024-09-13T18:30:37.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.095+0000 7efc97fff640 1 -- 172.21.15.114:0/1900778425 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7efca8005020 con 0x7efcb0107890 2024-09-13T18:30:37.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.099+0000 7efcaf7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596ed7b1970 0x5596ed7b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:37.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.099+0000 7efcaf7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596ed7b1970 0x5596ed7b1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57974/0 (socket says 172.21.15.114:57974) 2024-09-13T18:30:37.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.099+0000 7efcaf7fe640 1 -- 172.21.15.114:0/468295133 learned_addr learned my addr 172.21.15.114:0/468295133 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:37.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.106+0000 7efc97fff640 1 --2- 172.21.15.114:0/1900778425 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7efc980a1300 0x7efc980a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:37.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.106+0000 7efcaeffd640 1 --2- 172.21.15.114:0/468295133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596ed7b2290 0x5596ed7bc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:37.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.109+0000 7efcaf7fe640 1 --2- 172.21.15.114:0/1900778425 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7efc980a1300 0x7efc980a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:37.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.111+0000 7efc97fff640 1 -- 172.21.15.114:0/1900778425 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9453+0+0 (secure 0 0 0) 0x7efca80cccf0 con 0x7efcb0107890 2024-09-13T18:30:37.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.112+0000 7efcae7fc640 1 --2- 172.21.15.114:0/468295133 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596ed7bcb30 0x5596ed7bef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:37.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.113+0000 7efcaf7fe640 1 -- 172.21.15.114:0/468295133 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596ed7bcb30 msgr2=0x5596ed7bef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:37.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.113+0000 7efcaf7fe640 1 --2- 172.21.15.114:0/468295133 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596ed7bcb30 0x5596ed7bef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.113+0000 7efcaf7fe640 1 -- 172.21.15.114:0/468295133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596ed7b2290 msgr2=0x5596ed7bc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:37.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.113+0000 7efcaf7fe640 1 --2- 172.21.15.114:0/468295133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596ed7b2290 0x5596ed7bc5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.113+0000 7efcaf7fe640 1 -- 172.21.15.114:0/468295133 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596ed6dc620 con 0x5596ed7b1970 2024-09-13T18:30:37.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.121+0000 7efcaf7fe640 1 --2- 172.21.15.114:0/468295133 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596ed7b1970 0x5596ed7b1d50 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7efca0009230 tx=0x7efca0003740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ff28388d56b9036 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:37.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.128+0000 7efcaf7fe640 1 --2- 172.21.15.114:0/1900778425 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7efc980a1300 0x7efc980a37c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7efca0044000 tx=0x7efca0046000 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:37.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.128+0000 7efc957fa640 1 -- 172.21.15.114:0/468295133 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca0048670 con 0x5596ed7b1970 2024-09-13T18:30:37.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.128+0000 7efc957fa640 1 -- 172.21.15.114:0/468295133 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efca0048c60 con 0x5596ed7b1970 2024-09-13T18:30:37.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.128+0000 7efc957fa640 1 -- 172.21.15.114:0/468295133 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca003ebb0 con 0x5596ed7b1970 2024-09-13T18:30:37.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.130+0000 7efcb8257740 1 -- 172.21.15.114:0/468295133 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596ed7b1970 msgr2=0x5596ed7b1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:37.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.130+0000 7efcb8257740 1 --2- 172.21.15.114:0/468295133 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596ed7b1970 0x5596ed7b1d50 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7efca0009230 tx=0x7efca0003740 comp rx=0 tx=0).stop 2024-09-13T18:30:37.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.133+0000 7efcb8257740 1 -- 172.21.15.114:0/468295133 shutdown_connections 2024-09-13T18:30:37.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.133+0000 7efcb8257740 1 --2- 172.21.15.114:0/468295133 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596ed7bcb30 0x5596ed7bef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.133+0000 7efcb8257740 1 --2- 172.21.15.114:0/468295133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596ed7b2290 0x5596ed7bc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.133+0000 7efcb8257740 1 --2- 172.21.15.114:0/468295133 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596ed7b1970 0x5596ed7b1d50 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.133+0000 7efcb8257740 1 -- 172.21.15.114:0/468295133 >> 172.21.15.114:0/468295133 conn(0x5596ed7adee0 msgr2=0x5596ed7ae2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:37.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.141+0000 7efcb8257740 1 -- 172.21.15.114:0/468295133 shutdown_connections 2024-09-13T18:30:37.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.143+0000 7efcb8257740 1 -- 172.21.15.114:0/468295133 wait complete. 2024-09-13T18:30:37.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.144+0000 7efcb8257740 1 -- start start 2024-09-13T18:30:37.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.144+0000 7efcb8257740 1 Processor -- start 2024-09-13T18:30:37.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.145+0000 7efcb8257740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596ed7b1970 0x5596ed7bc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:37.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.145+0000 7efcb8257740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596ed7b2290 0x5596ed7b9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:37.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.145+0000 7efcb8257740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596ed7bcb30 0x5596ed7b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:37.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.145+0000 7efcb8257740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596ed6d3180 con 0x5596ed7bcb30 2024-09-13T18:30:37.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.145+0000 7efcb8257740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596ed3ed430 con 0x5596ed7b2290 2024-09-13T18:30:37.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.145+0000 7efcb8257740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596ed6985e0 con 0x5596ed7b1970 2024-09-13T18:30:37.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.145+0000 7efcb8257740 10 client.0 _ll_register_callbacks cb 0x5596ed704aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:30:37.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.146+0000 7efcaf7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596ed7bcb30 0x5596ed7b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:37.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.146+0000 7efcaf7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596ed7bcb30 0x5596ed7b9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57980/0 (socket says 172.21.15.114:57980) 2024-09-13T18:30:37.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.146+0000 7efcaf7fe640 1 -- 172.21.15.114:0/112491718 learned_addr learned my addr 172.21.15.114:0/112491718 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:37.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.147+0000 7efcaf7fe640 1 -- 172.21.15.114:0/112491718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596ed7b1970 msgr2=0x5596ed7bc170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:30:37.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.147+0000 7efcaf7fe640 1 --2- 172.21.15.114:0/112491718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596ed7b1970 0x5596ed7bc170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.147+0000 7efcaf7fe640 1 -- 172.21.15.114:0/112491718 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596ed7b2290 msgr2=0x5596ed7b9560 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:30:37.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.147+0000 7efcaf7fe640 1 --2- 172.21.15.114:0/112491718 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596ed7b2290 0x5596ed7b9560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.147+0000 7efcaf7fe640 1 -- 172.21.15.114:0/112491718 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcb011c170 con 0x5596ed7bcb30 2024-09-13T18:30:37.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.148+0000 7efcaf7fe640 1 --2- 172.21.15.114:0/112491718 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596ed7bcb30 0x5596ed7b9aa0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7efca0009850 tx=0x7efca002a920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfcd08457c835a3b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:37.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.148+0000 7efc92ff5640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:30:37.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.148+0000 7efc92ff5640 1 -- 172.21.15.114:0/112491718 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca0058070 con 0x5596ed7bcb30 2024-09-13T18:30:37.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.148+0000 7efcb8257740 1 -- 172.21.15.114:0/112491718 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5596ed73eb80 con 0x5596ed7bcb30 2024-09-13T18:30:37.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.149+0000 7efc92ff5640 1 -- 172.21.15.114:0/112491718 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9453+0+0 (secure 0 0 0) 0x7efca0048470 con 0x5596ed7bcb30 2024-09-13T18:30:37.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.149+0000 7efc92ff5640 1 -- 172.21.15.114:0/112491718 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7efca0048630 con 0x5596ed7bcb30 2024-09-13T18:30:37.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.152+0000 7efcb8257740 10 client.17889 fetch_fsmap learned FSMap version 34 2024-09-13T18:30:37.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.152+0000 7efcb8257740 1 -- 172.21.15.114:0/112491718 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5596ed7133c0 con 0x5596ed7bcb30 2024-09-13T18:30:37.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.155+0000 7efc92ff5640 1 -- 172.21.15.114:0/112491718 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5596ed73eb80 con 0x5596ed7bcb30 2024-09-13T18:30:37.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.155+0000 7efc92ff5640 1 -- 172.21.15.114:0/112491718 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7efca005b020 con 0x5596ed7bcb30 2024-09-13T18:30:37.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.156+0000 7efcb8257740 10 client.17889 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:30:37.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.156+0000 7efcb8257740 10 client.17889 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-13T18:30:37.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.156+0000 7efcb8257740 20 client.17889 populate_metadata read hostname 'smithi114' 2024-09-13T18:30:37.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.156+0000 7efcb8257740 1 --2- 172.21.15.114:0/112491718 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x5596ed7cc140 0x5596ed7ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:37.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.156+0000 7efcb8257740 4 client.17889 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:30:37.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.156+0000 7efcb8257740 1 -- 172.21.15.114:0/112491718 --> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5596ed7071f0 con 0x5596ed7cc140 2024-09-13T18:30:37.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.157+0000 7efcaeffd640 1 --2- 172.21.15.114:0/112491718 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x5596ed7cc140 0x5596ed7ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:37.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.176+0000 7efcaeffd640 1 --2- 172.21.15.114:0/112491718 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x5596ed7cc140 0x5596ed7ce560 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d8489cbc2d353255 server_cookie=cc34ddf614e06d10 in_seq=0 out_seq=0 2024-09-13T18:30:37.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.178+0000 7efc92ff5640 10 client.17889 ms_handle_connect on v2:172.21.15.134:6836/891692210 2024-09-13T18:30:37.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.179+0000 7efc92ff5640 1 -- 172.21.15.114:0/112491718 <== mds.1 v2:172.21.15.134:6836/891692210 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5596ed7071f0 con 0x5596ed7cc140 2024-09-13T18:30:37.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.179+0000 7efc92ff5640 10 client.17889 handle_command_reply: tid=0 2024-09-13T18:30:37.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.179+0000 7efcb8257740 1 client.17889 shutdown 2024-09-13T18:30:37.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.182+0000 7efcb8257740 1 -- 172.21.15.114:0/112491718 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596ed7bcb30 msgr2=0x5596ed7b9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:37.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.182+0000 7efcb8257740 1 --2- 172.21.15.114:0/112491718 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596ed7bcb30 0x5596ed7b9aa0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7efca0009850 tx=0x7efca002a920 comp rx=0 tx=0).stop 2024-09-13T18:30:37.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.184+0000 7efcb8257740 1 -- 172.21.15.114:0/112491718 shutdown_connections 2024-09-13T18:30:37.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.184+0000 7efcb8257740 1 --2- 172.21.15.114:0/112491718 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x5596ed7cc140 0x5596ed7ce560 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.184+0000 7efcb8257740 1 --2- 172.21.15.114:0/112491718 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596ed7bcb30 0x5596ed7b9aa0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.184+0000 7efcb8257740 1 --2- 172.21.15.114:0/112491718 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596ed7b2290 0x5596ed7b9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.184+0000 7efcb8257740 1 --2- 172.21.15.114:0/112491718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596ed7b1970 0x5596ed7bc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.184+0000 7efcb8257740 1 -- 172.21.15.114:0/112491718 >> 172.21.15.114:0/112491718 conn(0x5596ed7adee0 msgr2=0x5596ed7ad230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:37.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.187+0000 7efc92ff5640 0 client.17889 ms_handle_reset on v2:172.21.15.134:6836/891692210 2024-09-13T18:30:37.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.191+0000 7efcb8257740 1 -- 172.21.15.114:0/112491718 shutdown_connections 2024-09-13T18:30:37.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.192+0000 7efcb8257740 1 -- 172.21.15.114:0/112491718 wait complete. 2024-09-13T18:30:37.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.196+0000 7efcb8257740 20 client.17889 trim_cache size 0 max 16384 2024-09-13T18:30:37.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.227+0000 7efcb8257740 1 Processor -- start 2024-09-13T18:30:37.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.229+0000 7efcb8257740 1 -- start start 2024-09-13T18:30:37.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.229+0000 7efcb8257740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596ed7b1970 0x5596ed7c82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:37.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.229+0000 7efcb8257740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596ed7b2290 0x5596ed7b7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:37.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.229+0000 7efcb8257740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596ed7bcb30 0x5596ed7b8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:37.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.229+0000 7efcb8257740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596ed7bf3a0 con 0x5596ed7b2290 2024-09-13T18:30:37.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.229+0000 7efcb8257740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596ed7b8890 con 0x5596ed7bcb30 2024-09-13T18:30:37.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.229+0000 7efcb8257740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596ed7b7a90 con 0x5596ed7b1970 2024-09-13T18:30:37.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.232+0000 7efcaf7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596ed7bcb30 0x5596ed7b8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:37.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.232+0000 7efcaeffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596ed7b1970 0x5596ed7c82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:37.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.232+0000 7efcaf7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596ed7bcb30 0x5596ed7b8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:40436/0 (socket says 172.21.15.114:40436) 2024-09-13T18:30:37.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.232+0000 7efcaf7fe640 1 -- 172.21.15.114:0/1642984595 learned_addr learned my addr 172.21.15.114:0/1642984595 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:37.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.233+0000 7efcae7fc640 1 --2- 172.21.15.114:0/1642984595 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596ed7b2290 0x5596ed7b7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:37.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.233+0000 7efcaeffd640 1 -- 172.21.15.114:0/1642984595 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596ed7bcb30 msgr2=0x5596ed7b8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:37.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.233+0000 7efcaeffd640 1 --2- 172.21.15.114:0/1642984595 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596ed7bcb30 0x5596ed7b8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.233+0000 7efcaeffd640 1 -- 172.21.15.114:0/1642984595 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596ed7b2290 msgr2=0x5596ed7b7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:37.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.233+0000 7efcaeffd640 1 --2- 172.21.15.114:0/1642984595 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596ed7b2290 0x5596ed7b7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.233+0000 7efcaeffd640 1 -- 172.21.15.114:0/1642984595 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596ed6b5770 con 0x5596ed7b1970 2024-09-13T18:30:37.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.234+0000 7efcae7fc640 1 --2- 172.21.15.114:0/1642984595 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596ed7b2290 0x5596ed7b7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:30:37.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.234+0000 7efcaeffd640 1 --2- 172.21.15.114:0/1642984595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596ed7b1970 0x5596ed7c82e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x5596ed7cee30 tx=0x7efca4009dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=541bb588fde8f179 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:37.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.235+0000 7efc95ffb640 1 -- 172.21.15.114:0/1642984595 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca4005a50 con 0x5596ed7b1970 2024-09-13T18:30:37.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.236+0000 7efc95ffb640 1 -- 172.21.15.114:0/1642984595 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efca4015dd0 con 0x5596ed7b1970 2024-09-13T18:30:37.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.236+0000 7efc95ffb640 1 -- 172.21.15.114:0/1642984595 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca4018600 con 0x5596ed7b1970 2024-09-13T18:30:37.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.237+0000 7efcb8257740 1 -- 172.21.15.114:0/1642984595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596ed7b1970 msgr2=0x5596ed7c82e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:37.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.237+0000 7efcb8257740 1 --2- 172.21.15.114:0/1642984595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596ed7b1970 0x5596ed7c82e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x5596ed7cee30 tx=0x7efca4009dd0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.240+0000 7efcb8257740 1 -- 172.21.15.114:0/1642984595 shutdown_connections 2024-09-13T18:30:37.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.240+0000 7efcb8257740 1 --2- 172.21.15.114:0/1642984595 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596ed7bcb30 0x5596ed7b8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.240+0000 7efcb8257740 1 --2- 172.21.15.114:0/1642984595 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596ed7b2290 0x5596ed7b7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.240+0000 7efcb8257740 1 --2- 172.21.15.114:0/1642984595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596ed7b1970 0x5596ed7c82e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.240+0000 7efcb8257740 1 -- 172.21.15.114:0/1642984595 >> 172.21.15.114:0/1642984595 conn(0x5596ed7adee0 msgr2=0x5596ed7d2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:37.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.242+0000 7efcb8257740 1 -- 172.21.15.114:0/1642984595 shutdown_connections 2024-09-13T18:30:37.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.243+0000 7efcb8257740 1 -- 172.21.15.114:0/1642984595 wait complete. 2024-09-13T18:30:37.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.243+0000 7efcb8257740 1 -- start start 2024-09-13T18:30:37.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.243+0000 7efcb8257740 1 Processor -- start 2024-09-13T18:30:37.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.244+0000 7efcb8257740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596ed7b1970 0x5596ed7c3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:37.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.244+0000 7efcb8257740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596ed7b2290 0x5596ed7b5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:37.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.244+0000 7efcb8257740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596ed7bcb30 0x5596ed7b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:37.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.244+0000 7efcb8257740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596ed7b6a70 con 0x5596ed7b1970 2024-09-13T18:30:37.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.244+0000 7efcb8257740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596ed7b6be0 con 0x5596ed7b2290 2024-09-13T18:30:37.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.244+0000 7efcb8257740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596ed822660 con 0x5596ed7bcb30 2024-09-13T18:30:37.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.244+0000 7efcb8257740 10 client.0 _ll_register_callbacks cb 0x7efca0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:30:37.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.244+0000 7efcaf7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596ed7bcb30 0x5596ed7b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:37.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.244+0000 7efcaf7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596ed7bcb30 0x5596ed7b6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:58898/0 (socket says 172.21.15.114:58898) 2024-09-13T18:30:37.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.244+0000 7efcaf7fe640 1 -- 172.21.15.114:0/2002252219 learned_addr learned my addr 172.21.15.114:0/2002252219 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:37.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.244+0000 7efcaeffd640 1 --2- 172.21.15.114:0/2002252219 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596ed7b1970 0x5596ed7c3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:37.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.245+0000 7efcaf7fe640 1 -- 172.21.15.114:0/2002252219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596ed7b2290 msgr2=0x5596ed7b5e50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:30:37.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.245+0000 7efcaf7fe640 1 --2- 172.21.15.114:0/2002252219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596ed7b2290 0x5596ed7b5e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.245+0000 7efcaf7fe640 1 -- 172.21.15.114:0/2002252219 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596ed7b1970 msgr2=0x5596ed7c3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:37.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.245+0000 7efcaf7fe640 1 --2- 172.21.15.114:0/2002252219 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596ed7b1970 0x5596ed7c3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.245+0000 7efcaf7fe640 1 -- 172.21.15.114:0/2002252219 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efca40094b0 con 0x5596ed7bcb30 2024-09-13T18:30:37.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.245+0000 7efcaf7fe640 1 --2- 172.21.15.114:0/2002252219 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596ed7bcb30 0x5596ed7b6390 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7efca0052b10 tx=0x7efca0052cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b67ce5c929b35120 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:37.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.246+0000 7efc92ff5640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:30:37.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.246+0000 7efc92ff5640 1 -- 172.21.15.114:0/2002252219 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca0058020 con 0x5596ed7bcb30 2024-09-13T18:30:37.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.246+0000 7efcb8257740 1 -- 172.21.15.114:0/2002252219 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efca0048c60 con 0x5596ed7bcb30 2024-09-13T18:30:37.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.246+0000 7efc92ff5640 1 -- 172.21.15.114:0/2002252219 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9453+0+0 (secure 0 0 0) 0x7efca0000e40 con 0x5596ed7bcb30 2024-09-13T18:30:37.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.247+0000 7efc92ff5640 1 -- 172.21.15.114:0/2002252219 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7efca0052d80 con 0x5596ed7bcb30 2024-09-13T18:30:37.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.253+0000 7efcb8257740 10 client.27304 fetch_fsmap learned FSMap version 34 2024-09-13T18:30:37.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.253+0000 7efcb8257740 1 -- 172.21.15.114:0/2002252219 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5596ed6dae80 con 0x5596ed7bcb30 2024-09-13T18:30:37.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.253+0000 7efc92ff5640 1 -- 172.21.15.114:0/2002252219 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efca0048c60 con 0x5596ed7bcb30 2024-09-13T18:30:37.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.254+0000 7efc92ff5640 1 -- 172.21.15.114:0/2002252219 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7efca005b070 con 0x5596ed7bcb30 2024-09-13T18:30:37.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.255+0000 7efcb8257740 10 client.27304 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:30:37.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.255+0000 7efcb8257740 10 client.27304 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-13T18:30:37.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.255+0000 7efcb8257740 20 client.27304 populate_metadata read hostname 'smithi114' 2024-09-13T18:30:37.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.255+0000 7efcb8257740 1 --2- 172.21.15.114:0/2002252219 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x5596ed823950 0x5596ed825d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:37.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.255+0000 7efcb8257740 4 client.27304 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:30:37.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.255+0000 7efcb8257740 1 -- 172.21.15.114:0/2002252219 --> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5596ed4084e0 con 0x5596ed823950 2024-09-13T18:30:37.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.255+0000 7efcaeffd640 1 --2- 172.21.15.114:0/2002252219 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x5596ed823950 0x5596ed825d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:37.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.259+0000 7efcaeffd640 1 --2- 172.21.15.114:0/2002252219 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x5596ed823950 0x5596ed825d40 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=95eac358f7d5680c server_cookie=46ae7b94b7ae0586 in_seq=0 out_seq=0 2024-09-13T18:30:37.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.260+0000 7efc92ff5640 10 client.27304 ms_handle_connect on v2:172.21.15.134:6836/891692210 2024-09-13T18:30:37.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.260+0000 7efc92ff5640 1 -- 172.21.15.114:0/2002252219 <== mds.1 v2:172.21.15.134:6836/891692210 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5596ed4084e0 con 0x5596ed823950 2024-09-13T18:30:37.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.260+0000 7efc92ff5640 10 client.27304 handle_command_reply: tid=0 2024-09-13T18:30:37.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.260+0000 7efcb8257740 1 client.27304 shutdown 2024-09-13T18:30:37.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.262+0000 7efcb8257740 1 -- 172.21.15.114:0/2002252219 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596ed7bcb30 msgr2=0x5596ed7b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:37.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.262+0000 7efcb8257740 1 --2- 172.21.15.114:0/2002252219 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596ed7bcb30 0x5596ed7b6390 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7efca0052b10 tx=0x7efca0052cc0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.267+0000 7efcb8257740 1 -- 172.21.15.114:0/2002252219 shutdown_connections 2024-09-13T18:30:37.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.267+0000 7efcb8257740 1 --2- 172.21.15.114:0/2002252219 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x5596ed823950 0x5596ed825d40 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.267+0000 7efcb8257740 1 --2- 172.21.15.114:0/2002252219 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596ed7bcb30 0x5596ed7b6390 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.267+0000 7efcb8257740 1 --2- 172.21.15.114:0/2002252219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596ed7b2290 0x5596ed7b5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.267+0000 7efcb8257740 1 --2- 172.21.15.114:0/2002252219 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596ed7b1970 0x5596ed7c3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.267+0000 7efcb8257740 1 -- 172.21.15.114:0/2002252219 >> 172.21.15.114:0/2002252219 conn(0x5596ed7adee0 msgr2=0x5596ed7b7d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:37.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.267+0000 7efc92ff5640 0 client.27304 ms_handle_reset on v2:172.21.15.134:6836/891692210 2024-09-13T18:30:37.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.271+0000 7efcb8257740 1 -- 172.21.15.114:0/2002252219 shutdown_connections 2024-09-13T18:30:37.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.273+0000 7efcb8257740 1 -- 172.21.15.114:0/2002252219 wait complete. 2024-09-13T18:30:37.284 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:30:37.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.280+0000 7efcb8257740 20 client.27304 trim_cache size 0 max 16384 2024-09-13T18:30:37.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.282+0000 7efcaffff640 1 -- 172.21.15.114:0/1900778425 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7efc980a1300 msgr2=0x7efc980a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:37.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.282+0000 7efcaffff640 1 --2- 172.21.15.114:0/1900778425 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7efc980a1300 0x7efc980a37c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7efca0044000 tx=0x7efca0046000 comp rx=0 tx=0).stop 2024-09-13T18:30:37.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.282+0000 7efcaffff640 1 -- 172.21.15.114:0/1900778425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcb0107890 msgr2=0x7efcb0118d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:37.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.282+0000 7efcaffff640 1 --2- 172.21.15.114:0/1900778425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcb0107890 0x7efcb0118d50 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7efca800cce0 tx=0x7efca800a620 comp rx=0 tx=0).stop 2024-09-13T18:30:37.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.283+0000 7efcaffff640 1 -- 172.21.15.114:0/1900778425 shutdown_connections 2024-09-13T18:30:37.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.283+0000 7efcaffff640 1 --2- 172.21.15.114:0/1900778425 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7efc980a1300 0x7efc980a37c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.283+0000 7efcaffff640 1 --2- 172.21.15.114:0/1900778425 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efcb0119400 0x7efcb0112900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.283+0000 7efcaffff640 1 --2- 172.21.15.114:0/1900778425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcb0107890 0x7efcb0118d50 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.283+0000 7efcaffff640 1 --2- 172.21.15.114:0/1900778425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efcb01074b0 0x7efcb0118810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.283+0000 7efcaffff640 1 -- 172.21.15.114:0/1900778425 >> 172.21.15.114:0/1900778425 conn(0x7efcb006af40 msgr2=0x7efcb006ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:37.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.287+0000 7efcaffff640 1 -- 172.21.15.114:0/1900778425 shutdown_connections 2024-09-13T18:30:37.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.288+0000 7efcaffff640 1 -- 172.21.15.114:0/1900778425 wait complete. 2024-09-13T18:30:37.326 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-09-13T18:30:37.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.584+0000 7f942d467640 1 -- 172.21.15.114:0/2083846130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9428107e30 msgr2=0x7f94280791f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:37.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.584+0000 7f942d467640 1 --2- 172.21.15.114:0/2083846130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9428107e30 0x7f94280791f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9418009980 tx=0x7f941802c670 comp rx=0 tx=0).stop 2024-09-13T18:30:37.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.586+0000 7f942d467640 1 -- 172.21.15.114:0/2083846130 shutdown_connections 2024-09-13T18:30:37.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.586+0000 7f942d467640 1 --2- 172.21.15.114:0/2083846130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9428107e30 0x7f94280791f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.586+0000 7f942d467640 1 --2- 172.21.15.114:0/2083846130 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f94281074b0 0x7f94281078f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.586+0000 7f942d467640 1 --2- 172.21.15.114:0/2083846130 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f942810b8d0 0x7f942810bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.586+0000 7f942d467640 1 -- 172.21.15.114:0/2083846130 >> 172.21.15.114:0/2083846130 conn(0x7f942806af40 msgr2=0x7f942806b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:37.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.596+0000 7f942d467640 1 -- 172.21.15.114:0/2083846130 shutdown_connections 2024-09-13T18:30:37.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.597+0000 7f942d467640 1 -- 172.21.15.114:0/2083846130 wait complete. 2024-09-13T18:30:37.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.598+0000 7f942d467640 1 Processor -- start 2024-09-13T18:30:37.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.598+0000 7f942d467640 1 -- start start 2024-09-13T18:30:37.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.598+0000 7f942d467640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94281074b0 0x7f94281b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:37.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.598+0000 7f942d467640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9428107e30 0x7f94281b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:37.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.598+0000 7f942d467640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f942810b8d0 0x7f9428073ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:37.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.598+0000 7f942d467640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9428074250 con 0x7f9428107e30 2024-09-13T18:30:37.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.598+0000 7f942d467640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94280743c0 con 0x7f942810b8d0 2024-09-13T18:30:37.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.598+0000 7f942d467640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9428074580 con 0x7f94281074b0 2024-09-13T18:30:37.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.599+0000 7f942cc66640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f942810b8d0 0x7f9428073ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:37.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.600+0000 7f94277fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9428107e30 0x7f94281b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:37.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.600+0000 7f94277fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9428107e30 0x7f94281b3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:58006/0 (socket says 172.21.15.114:58006) 2024-09-13T18:30:37.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.600+0000 7f94277fe640 1 -- 172.21.15.114:0/1948776724 learned_addr learned my addr 172.21.15.114:0/1948776724 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:37.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.601+0000 7f94277fe640 1 -- 172.21.15.114:0/1948776724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94281074b0 msgr2=0x7f94281b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:37.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.601+0000 7f94277fe640 1 --2- 172.21.15.114:0/1948776724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94281074b0 0x7f94281b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.601+0000 7f94277fe640 1 -- 172.21.15.114:0/1948776724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f942810b8d0 msgr2=0x7f9428073ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:37.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.601+0000 7f94277fe640 1 --2- 172.21.15.114:0/1948776724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f942810b8d0 0x7f9428073ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.601+0000 7f94277fe640 1 -- 172.21.15.114:0/1948776724 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9418009560 con 0x7f9428107e30 2024-09-13T18:30:37.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.601+0000 7f94277fe640 1 --2- 172.21.15.114:0/1948776724 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9428107e30 0x7f94281b3ad0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f9420007ef0 tx=0x7f942000ae40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:37.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.603+0000 7f94257fa640 1 -- 172.21.15.114:0/1948776724 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942000d960 con 0x7f9428107e30 2024-09-13T18:30:37.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.603+0000 7f94257fa640 1 -- 172.21.15.114:0/1948776724 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9420012540 con 0x7f9428107e30 2024-09-13T18:30:37.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.603+0000 7f94257fa640 1 -- 172.21.15.114:0/1948776724 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9420008670 con 0x7f9428107e30 2024-09-13T18:30:37.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.604+0000 7f942d467640 1 -- 172.21.15.114:0/1948776724 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94280747a0 con 0x7f9428107e30 2024-09-13T18:30:37.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.604+0000 7f942d467640 1 -- 172.21.15.114:0/1948776724 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94281ba4d0 con 0x7f9428107e30 2024-09-13T18:30:37.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.607+0000 7f94313e3740 1 Processor -- start 2024-09-13T18:30:37.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.607+0000 7f94313e3740 1 -- start start 2024-09-13T18:30:37.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.607+0000 7f94313e3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641a023a040 0x5641a023a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:37.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.607+0000 7f94313e3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5641a023a960 0x5641a0235910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:37.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.607+0000 7f94313e3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5641a024ddb0 0x5641a02501a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:37.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.607+0000 7f94313e3740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641a019bcc0 con 0x5641a023a960 2024-09-13T18:30:37.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.607+0000 7f94313e3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641a0199450 con 0x5641a024ddb0 2024-09-13T18:30:37.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.607+0000 7f94313e3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641a0194d40 con 0x5641a023a040 2024-09-13T18:30:37.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.608+0000 7f9427fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5641a023a960 0x5641a0235910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:37.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.608+0000 7f9427fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5641a023a960 0x5641a0235910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:58018/0 (socket says 172.21.15.114:58018) 2024-09-13T18:30:37.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.608+0000 7f9427fff640 1 -- 172.21.15.114:0/2893326425 learned_addr learned my addr 172.21.15.114:0/2893326425 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:37.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.608+0000 7f942cc66640 1 --2- 172.21.15.114:0/2893326425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641a023a040 0x5641a023a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:37.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.608+0000 7f94277fe640 1 --2- 172.21.15.114:0/2893326425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5641a024ddb0 0x5641a02501a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:37.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.610+0000 7f9427fff640 1 -- 172.21.15.114:0/2893326425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641a023a040 msgr2=0x5641a023a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:37.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.610+0000 7f9427fff640 1 --2- 172.21.15.114:0/2893326425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641a023a040 0x5641a023a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.611+0000 7f9427fff640 1 -- 172.21.15.114:0/2893326425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5641a024ddb0 msgr2=0x5641a02501a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:37.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.611+0000 7f9427fff640 1 --2- 172.21.15.114:0/2893326425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5641a024ddb0 0x5641a02501a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.611+0000 7f9427fff640 1 -- 172.21.15.114:0/2893326425 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641a015c620 con 0x5641a023a960 2024-09-13T18:30:37.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.615+0000 7f94257fa640 1 -- 172.21.15.114:0/1948776724 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f94200126b0 con 0x7f9428107e30 2024-09-13T18:30:37.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.615+0000 7f94257fa640 1 --2- 172.21.15.114:0/1948776724 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f94100a1410 0x7f94100a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:37.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.616+0000 7f94257fa640 1 -- 172.21.15.114:0/1948776724 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f94200d1cf0 con 0x7f9428107e30 2024-09-13T18:30:37.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.616+0000 7f942cc66640 1 --2- 172.21.15.114:0/1948776724 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f94100a1410 0x7f94100a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:37.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.617+0000 7f942cc66640 1 --2- 172.21.15.114:0/1948776724 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f94100a1410 0x7f94100a38d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9418006010 tx=0x7f9418004040 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:37.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.619+0000 7f9427fff640 1 --2- 172.21.15.114:0/2893326425 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5641a023a960 0x5641a0235910 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f941c02b860 tx=0x7f941c02bd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b127133c344bfa4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:37.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.619+0000 7f940e7fc640 1 -- 172.21.15.114:0/2893326425 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941c02d8e0 con 0x5641a023a960 2024-09-13T18:30:37.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.619+0000 7f940e7fc640 1 -- 172.21.15.114:0/2893326425 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f941c033430 con 0x5641a023a960 2024-09-13T18:30:37.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.620+0000 7f94313e3740 1 -- 172.21.15.114:0/2893326425 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5641a023a960 msgr2=0x5641a0235910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:37.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.620+0000 7f94313e3740 1 --2- 172.21.15.114:0/2893326425 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5641a023a960 0x5641a0235910 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f941c02b860 tx=0x7f941c02bd20 comp rx=0 tx=0).stop 2024-09-13T18:30:37.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.620+0000 7f94313e3740 1 -- 172.21.15.114:0/2893326425 shutdown_connections 2024-09-13T18:30:37.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.620+0000 7f94313e3740 1 --2- 172.21.15.114:0/2893326425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5641a024ddb0 0x5641a02501a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.620+0000 7f94313e3740 1 --2- 172.21.15.114:0/2893326425 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5641a023a960 0x5641a0235910 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.620+0000 7f94313e3740 1 --2- 172.21.15.114:0/2893326425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641a023a040 0x5641a023a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.620+0000 7f94313e3740 1 -- 172.21.15.114:0/2893326425 >> 172.21.15.114:0/2893326425 conn(0x5641a02366f0 msgr2=0x5641a0236ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:37.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.621+0000 7f94313e3740 1 -- 172.21.15.114:0/2893326425 shutdown_connections 2024-09-13T18:30:37.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.621+0000 7f94313e3740 1 -- 172.21.15.114:0/2893326425 wait complete. 2024-09-13T18:30:37.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.621+0000 7f94313e3740 1 -- start start 2024-09-13T18:30:37.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.622+0000 7f94313e3740 1 Processor -- start 2024-09-13T18:30:37.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.622+0000 7f94313e3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5641a023a040 0x5641a0234de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:37.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.622+0000 7f94313e3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641a023a960 0x5641a0235320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:37.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.622+0000 7f94313e3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5641a024ddb0 0x5641a0240610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:37.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.622+0000 7f94313e3740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641a0153180 con 0x5641a023a040 2024-09-13T18:30:37.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.622+0000 7f94313e3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56419fe6d430 con 0x5641a024ddb0 2024-09-13T18:30:37.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.622+0000 7f94313e3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641a01185e0 con 0x5641a023a960 2024-09-13T18:30:37.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.622+0000 7f94313e3740 10 client.0 _ll_register_callbacks cb 0x5641a0184aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:30:37.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.623+0000 7f9427fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5641a023a040 0x5641a0234de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:37.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.623+0000 7f9427fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5641a023a040 0x5641a0234de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:58026/0 (socket says 172.21.15.114:58026) 2024-09-13T18:30:37.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.623+0000 7f9427fff640 1 -- 172.21.15.114:0/2679649163 learned_addr learned my addr 172.21.15.114:0/2679649163 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:37.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.623+0000 7f942cc66640 1 --2- 172.21.15.114:0/2679649163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5641a024ddb0 0x5641a0240610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:37.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.623+0000 7f9427fff640 1 -- 172.21.15.114:0/2679649163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641a023a960 msgr2=0x5641a0235320 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:30:37.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.623+0000 7f9427fff640 1 --2- 172.21.15.114:0/2679649163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641a023a960 0x5641a0235320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.623+0000 7f9427fff640 1 -- 172.21.15.114:0/2679649163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5641a024ddb0 msgr2=0x5641a0240610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:37.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.623+0000 7f9427fff640 1 --2- 172.21.15.114:0/2679649163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5641a024ddb0 0x5641a0240610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.623+0000 7f9427fff640 1 -- 172.21.15.114:0/2679649163 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f941c02b4c0 con 0x5641a023a040 2024-09-13T18:30:37.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.624+0000 7f9427fff640 1 --2- 172.21.15.114:0/2679649163 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5641a023a040 0x5641a0234de0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f941c0325c0 tx=0x7f941c02daf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6130d03e12e9db4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:37.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.624+0000 7f94077fe640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:30:37.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.624+0000 7f94077fe640 1 -- 172.21.15.114:0/2679649163 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941c033c20 con 0x5641a023a040 2024-09-13T18:30:37.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.624+0000 7f94313e3740 1 -- 172.21.15.114:0/2679649163 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5641a01beb80 con 0x5641a023a040 2024-09-13T18:30:37.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.624+0000 7f94077fe640 1 -- 172.21.15.114:0/2679649163 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f941c02dc10 con 0x5641a023a040 2024-09-13T18:30:37.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.628+0000 7f94077fe640 1 -- 172.21.15.114:0/2679649163 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f941c02de10 con 0x5641a023a040 2024-09-13T18:30:37.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.631+0000 7f94313e3740 10 client.17904 fetch_fsmap learned FSMap version 34 2024-09-13T18:30:37.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.631+0000 7f94313e3740 1 -- 172.21.15.114:0/2679649163 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5641a01933c0 con 0x5641a023a040 2024-09-13T18:30:37.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.634+0000 7f94077fe640 1 -- 172.21.15.114:0/2679649163 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5641a01beb80 con 0x5641a023a040 2024-09-13T18:30:37.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.638+0000 7f94077fe640 1 -- 172.21.15.114:0/2679649163 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f941c046790 con 0x5641a023a040 2024-09-13T18:30:37.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.638+0000 7f94313e3740 10 client.17904 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:30:37.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.638+0000 7f94313e3740 10 client.17904 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:30:37.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.638+0000 7f94313e3740 20 client.17904 populate_metadata read hostname 'smithi114' 2024-09-13T18:30:37.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.638+0000 7f94313e3740 1 --2- 172.21.15.114:0/2679649163 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5641a024c0c0 0x5641a024c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:37.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.638+0000 7f94313e3740 4 client.17904 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:30:37.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.638+0000 7f94313e3740 1 -- 172.21.15.114:0/2679649163 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5641a01871f0 con 0x5641a024c0c0 2024-09-13T18:30:37.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.639+0000 7f94277fe640 1 --2- 172.21.15.114:0/2679649163 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5641a024c0c0 0x5641a024c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:37.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.640+0000 7f94277fe640 1 --2- 172.21.15.114:0/2679649163 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5641a024c0c0 0x5641a024c4a0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=601beafd11530ef6 server_cookie=e31fd48670f7bbd7 in_seq=0 out_seq=0 2024-09-13T18:30:37.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.643+0000 7f94077fe640 10 client.17904 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:30:37.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.643+0000 7f94077fe640 1 -- 172.21.15.114:0/2679649163 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5641a01871f0 con 0x5641a024c0c0 2024-09-13T18:30:37.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.643+0000 7f94077fe640 10 client.17904 handle_command_reply: tid=0 2024-09-13T18:30:37.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.643+0000 7f94313e3740 1 client.17904 shutdown 2024-09-13T18:30:37.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.643+0000 7f94313e3740 1 -- 172.21.15.114:0/2679649163 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5641a023a040 msgr2=0x5641a0234de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:37.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.643+0000 7f94313e3740 1 --2- 172.21.15.114:0/2679649163 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5641a023a040 0x5641a0234de0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f941c0325c0 tx=0x7f941c02daf0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.648+0000 7f94313e3740 1 -- 172.21.15.114:0/2679649163 shutdown_connections 2024-09-13T18:30:37.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.648+0000 7f94313e3740 1 --2- 172.21.15.114:0/2679649163 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5641a024c0c0 0x5641a024c4a0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.648+0000 7f94313e3740 1 --2- 172.21.15.114:0/2679649163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5641a024ddb0 0x5641a0240610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.648+0000 7f94313e3740 1 --2- 172.21.15.114:0/2679649163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641a023a960 0x5641a0235320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.648+0000 7f94313e3740 1 --2- 172.21.15.114:0/2679649163 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5641a023a040 0x5641a0234de0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.648+0000 7f94313e3740 1 -- 172.21.15.114:0/2679649163 >> 172.21.15.114:0/2679649163 conn(0x5641a02366f0 msgr2=0x5641a023c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:37.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.651+0000 7f94077fe640 0 client.17904 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:30:37.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.655+0000 7f94313e3740 1 -- 172.21.15.114:0/2679649163 shutdown_connections 2024-09-13T18:30:37.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.663+0000 7f94313e3740 1 -- 172.21.15.114:0/2679649163 wait complete. 2024-09-13T18:30:37.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.669+0000 7f94313e3740 20 client.17904 trim_cache size 0 max 16384 2024-09-13T18:30:37.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.694+0000 7f94313e3740 1 Processor -- start 2024-09-13T18:30:37.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.697+0000 7f94313e3740 1 -- start start 2024-09-13T18:30:37.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.697+0000 7f94313e3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5641a023a040 0x5641a0240590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:37.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.697+0000 7f94313e3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641a023a960 0x5641a02541c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:37.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.697+0000 7f94313e3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5641a024ddb0 0x5641a0231e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:37.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.697+0000 7f94313e3740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641a023af10 con 0x5641a023a040 2024-09-13T18:30:37.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.697+0000 7f94313e3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641a0248e90 con 0x5641a024ddb0 2024-09-13T18:30:37.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.697+0000 7f94313e3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641a0236530 con 0x5641a023a960 2024-09-13T18:30:37.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.699+0000 7f942cc66640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5641a024ddb0 0x5641a0231e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:37.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.699+0000 7f942cc66640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5641a024ddb0 0x5641a0231e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:40486/0 (socket says 172.21.15.114:40486) 2024-09-13T18:30:37.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.699+0000 7f942cc66640 1 -- 172.21.15.114:0/3435526308 learned_addr learned my addr 172.21.15.114:0/3435526308 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:37.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.703+0000 7f94277fe640 1 --2- 172.21.15.114:0/3435526308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641a023a960 0x5641a02541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:37.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.704+0000 7f9427fff640 1 --2- 172.21.15.114:0/3435526308 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5641a023a040 0x5641a0240590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:37.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.704+0000 7f94277fe640 1 -- 172.21.15.114:0/3435526308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5641a024ddb0 msgr2=0x5641a0231e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:37.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.704+0000 7f94277fe640 1 --2- 172.21.15.114:0/3435526308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5641a024ddb0 0x5641a0231e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.704+0000 7f94277fe640 1 -- 172.21.15.114:0/3435526308 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5641a023a040 msgr2=0x5641a0240590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:37.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.704+0000 7f94277fe640 1 --2- 172.21.15.114:0/3435526308 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5641a023a040 0x5641a0240590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.704+0000 7f94277fe640 1 -- 172.21.15.114:0/3435526308 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641a0135770 con 0x5641a023a960 2024-09-13T18:30:37.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.704+0000 7f94277fe640 1 --2- 172.21.15.114:0/3435526308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641a023a960 0x5641a02541c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f94200cf150 tx=0x7f9420012ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64f56b95c5004984 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:37.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.704+0000 7f940effd640 1 -- 172.21.15.114:0/3435526308 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942000f040 con 0x5641a023a960 2024-09-13T18:30:37.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.704+0000 7f940effd640 1 -- 172.21.15.114:0/3435526308 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9420010040 con 0x5641a023a960 2024-09-13T18:30:37.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.705+0000 7f94313e3740 1 -- 172.21.15.114:0/3435526308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641a023a960 msgr2=0x5641a02541c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:37.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.705+0000 7f94313e3740 1 --2- 172.21.15.114:0/3435526308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641a023a960 0x5641a02541c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f94200cf150 tx=0x7f9420012ba0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.708+0000 7f94313e3740 1 -- 172.21.15.114:0/3435526308 shutdown_connections 2024-09-13T18:30:37.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.708+0000 7f94313e3740 1 --2- 172.21.15.114:0/3435526308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5641a024ddb0 0x5641a0231e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.708+0000 7f94313e3740 1 --2- 172.21.15.114:0/3435526308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641a023a960 0x5641a02541c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.708+0000 7f94313e3740 1 --2- 172.21.15.114:0/3435526308 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5641a023a040 0x5641a0240590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.708+0000 7f94313e3740 1 -- 172.21.15.114:0/3435526308 >> 172.21.15.114:0/3435526308 conn(0x5641a02366f0 msgr2=0x5641a024d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:37.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.711+0000 7f94313e3740 1 -- 172.21.15.114:0/3435526308 shutdown_connections 2024-09-13T18:30:37.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.711+0000 7f94313e3740 1 -- 172.21.15.114:0/3435526308 wait complete. 2024-09-13T18:30:37.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.711+0000 7f94313e3740 1 -- start start 2024-09-13T18:30:37.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.711+0000 7f94313e3740 1 Processor -- start 2024-09-13T18:30:37.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.712+0000 7f94313e3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5641a023a040 0x5641a02541f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:37.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.712+0000 7f94313e3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641a023a960 0x5641a024f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:37.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.712+0000 7f94313e3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5641a024ddb0 0x5641a024fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:37.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.712+0000 7f94313e3740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641a024ff70 con 0x5641a024ddb0 2024-09-13T18:30:37.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.712+0000 7f94313e3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641a02500e0 con 0x5641a023a040 2024-09-13T18:30:37.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.712+0000 7f94313e3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641a02502a0 con 0x5641a023a960 2024-09-13T18:30:37.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.712+0000 7f94313e3740 10 client.0 _ll_register_callbacks cb 0x7f941c02b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:30:37.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.712+0000 7f9427fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5641a023a040 0x5641a02541f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:37.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.712+0000 7f9427fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5641a023a040 0x5641a02541f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:40512/0 (socket says 172.21.15.114:40512) 2024-09-13T18:30:37.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.712+0000 7f9427fff640 1 -- 172.21.15.114:0/665289878 learned_addr learned my addr 172.21.15.114:0/665289878 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:37.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.712+0000 7f94277fe640 1 --2- 172.21.15.114:0/665289878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641a023a960 0x5641a024f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:37.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.713+0000 7f94277fe640 1 -- 172.21.15.114:0/665289878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5641a023a040 msgr2=0x5641a02541f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:37.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.713+0000 7f94277fe640 1 --2- 172.21.15.114:0/665289878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5641a023a040 0x5641a02541f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.713+0000 7f94277fe640 1 -- 172.21.15.114:0/665289878 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5641a024ddb0 msgr2=0x5641a024fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:37.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.713+0000 7f94277fe640 1 --2- 172.21.15.114:0/665289878 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5641a024ddb0 0x5641a024fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.713+0000 7f94277fe640 1 -- 172.21.15.114:0/665289878 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94281ba4d0 con 0x5641a023a960 2024-09-13T18:30:37.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.713+0000 7f94277fe640 1 --2- 172.21.15.114:0/665289878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641a023a960 0x5641a024f4f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f942009b4f0 tx=0x7f942000d5c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=990696559ad79db9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:37.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.714+0000 7f94077fe640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:30:37.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.715+0000 7f94077fe640 1 -- 172.21.15.114:0/665289878 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942000f050 con 0x5641a023a960 2024-09-13T18:30:37.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.715+0000 7f94077fe640 1 -- 172.21.15.114:0/665289878 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f94200129b0 con 0x5641a023a960 2024-09-13T18:30:37.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.715+0000 7f94313e3740 1 -- 172.21.15.114:0/665289878 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f941c033430 con 0x5641a023a960 2024-09-13T18:30:37.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:37 smithi114 ceph-mon[30897]: pgmap v251: 129 pgs: 129 active+clean; 23 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 38 MiB/s wr, 802 op/s 2024-09-13T18:30:37.721 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.717+0000 7f0f4759e640 1 -- 172.21.15.114:0/2954444479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f48119400 msgr2=0x7f0f4811b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:37.721 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.717+0000 7f0f4759e640 1 --2- 172.21.15.114:0/2954444479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f48119400 0x7f0f4811b7f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0f40009930 tx=0x7f0f4002a480 comp rx=0 tx=0).stop 2024-09-13T18:30:37.722 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.717+0000 7f0f4759e640 1 -- 172.21.15.114:0/2954444479 shutdown_connections 2024-09-13T18:30:37.722 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.717+0000 7f0f4759e640 1 --2- 172.21.15.114:0/2954444479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f48119400 0x7f0f4811b7f0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.722 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.717+0000 7f0f4759e640 1 --2- 172.21.15.114:0/2954444479 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f48107890 0x7f0f48107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.722 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.717+0000 7f0f4759e640 1 --2- 172.21.15.114:0/2954444479 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f481074b0 0x7f0f4810e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.722 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.717+0000 7f0f4759e640 1 -- 172.21.15.114:0/2954444479 >> 172.21.15.114:0/2954444479 conn(0x7f0f4806adb0 msgr2=0x7f0f4806b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:37.722 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.717+0000 7f0f4759e640 1 -- 172.21.15.114:0/2954444479 shutdown_connections 2024-09-13T18:30:37.722 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.717+0000 7f0f4759e640 1 -- 172.21.15.114:0/2954444479 wait complete. 2024-09-13T18:30:37.722 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.718+0000 7f0f4759e640 1 Processor -- start 2024-09-13T18:30:37.727 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.722+0000 7f0f4759e640 1 -- start start 2024-09-13T18:30:37.727 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.722+0000 7f0f4759e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f481074b0 0x7f0f48114b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:37.727 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.722+0000 7f0f4759e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f48107890 0x7f0f481150d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:37.727 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.722+0000 7f0f4759e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f48115610 0x7f0f481196a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:37.727 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.722+0000 7f0f4759e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f48119dc0 con 0x7f0f48107890 2024-09-13T18:30:37.727 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.722+0000 7f0f4759e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f48119f30 con 0x7f0f48115610 2024-09-13T18:30:37.727 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.722+0000 7f0f4759e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4811a0f0 con 0x7f0f481074b0 2024-09-13T18:30:37.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.724+0000 7f94077fe640 1 -- 172.21.15.114:0/665289878 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94200cf4c0 con 0x5641a023a960 2024-09-13T18:30:37.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.724+0000 7f94313e3740 10 client.27319 fetch_fsmap learned FSMap version 34 2024-09-13T18:30:37.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.724+0000 7f94313e3740 1 -- 172.21.15.114:0/665289878 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5641a015ae80 con 0x5641a023a960 2024-09-13T18:30:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.724+0000 7f0f4659c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f481074b0 0x7f0f48114b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.724+0000 7f0f45d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f48107890 0x7f0f481150d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.724+0000 7f0f45d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f48107890 0x7f0f481150d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:58064/0 (socket says 172.21.15.114:58064) 2024-09-13T18:30:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.724+0000 7f0f45d9b640 1 -- 172.21.15.114:0/944271734 learned_addr learned my addr 172.21.15.114:0/944271734 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.724+0000 7f0f46d9d640 1 --2- 172.21.15.114:0/944271734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f48115610 0x7f0f481196a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.726+0000 7f0f4659c640 1 -- 172.21.15.114:0/944271734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f48115610 msgr2=0x7f0f481196a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.726+0000 7f0f4659c640 1 --2- 172.21.15.114:0/944271734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f48115610 0x7f0f481196a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.726+0000 7f0f4659c640 1 -- 172.21.15.114:0/944271734 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f48107890 msgr2=0x7f0f481150d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.726+0000 7f0f4659c640 1 --2- 172.21.15.114:0/944271734 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f48107890 0x7f0f481150d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.726+0000 7f0f4659c640 1 -- 172.21.15.114:0/944271734 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f40009560 con 0x7f0f481074b0 2024-09-13T18:30:37.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.728+0000 7f94077fe640 1 -- 172.21.15.114:0/665289878 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f941c033430 con 0x5641a023a960 2024-09-13T18:30:37.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.728+0000 7f94077fe640 1 -- 172.21.15.114:0/665289878 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9420021060 con 0x5641a023a960 2024-09-13T18:30:37.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.728+0000 7f94313e3740 10 client.27319 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:30:37.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.728+0000 7f94313e3740 10 client.27319 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:30:37.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.728+0000 7f94313e3740 20 client.27319 populate_metadata read hostname 'smithi114' 2024-09-13T18:30:37.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.728+0000 7f94313e3740 1 --2- 172.21.15.114:0/665289878 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5641a02a3a00 0x5641a02a5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:37.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.728+0000 7f94313e3740 4 client.27319 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:30:37.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.728+0000 7f94313e3740 1 -- 172.21.15.114:0/665289878 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56419fe884e0 con 0x5641a02a3a00 2024-09-13T18:30:37.735 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.730+0000 7f0f4659c640 1 --2- 172.21.15.114:0/944271734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f481074b0 0x7f0f48114b90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0f3800eb00 tx=0x7f0f3800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:37.735 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.731+0000 7f0f3f7fe640 1 -- 172.21.15.114:0/944271734 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f38010430 con 0x7f0f481074b0 2024-09-13T18:30:37.735 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.731+0000 7f0f4759e640 1 -- 172.21.15.114:0/944271734 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f4811a370 con 0x7f0f481074b0 2024-09-13T18:30:37.735 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.731+0000 7f0f4759e640 1 -- 172.21.15.114:0/944271734 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f48124840 con 0x7f0f481074b0 2024-09-13T18:30:37.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.728+0000 7f9427fff640 1 --2- 172.21.15.114:0/665289878 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5641a02a3a00 0x5641a02a5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:37.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.735+0000 7f9427fff640 1 --2- 172.21.15.114:0/665289878 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5641a02a3a00 0x5641a02a5e20 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86169ebfd5c836f0 server_cookie=141f2dde457c741c in_seq=0 out_seq=0 2024-09-13T18:30:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.735+0000 7f0f4f9d2740 1 Processor -- start 2024-09-13T18:30:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.735+0000 7f0f4f9d2740 1 -- start start 2024-09-13T18:30:37.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.736+0000 7f94077fe640 10 client.27319 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:30:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.735+0000 7f0f3f7fe640 1 -- 172.21.15.114:0/944271734 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0f38010a20 con 0x7f0f481074b0 2024-09-13T18:30:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.735+0000 7f0f3f7fe640 1 -- 172.21.15.114:0/944271734 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f380179c0 con 0x7f0f481074b0 2024-09-13T18:30:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.735+0000 7f0f4f9d2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5646a309b500 0x5646a309b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.735+0000 7f0f4f9d2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646a309be20 0x5646a30a6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.736+0000 7f0f4f9d2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5646a30a66c0 0x5646a30a8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.736+0000 7f0f4f9d2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646a3006c30 con 0x5646a309be20 2024-09-13T18:30:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.736+0000 7f0f4f9d2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646a30043c0 con 0x5646a309b500 2024-09-13T18:30:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.736+0000 7f0f4f9d2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646a2fffcb0 con 0x5646a30a66c0 2024-09-13T18:30:37.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.737+0000 7f94077fe640 1 -- 172.21.15.114:0/665289878 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56419fe884e0 con 0x5641a02a3a00 2024-09-13T18:30:37.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.737+0000 7f94077fe640 10 client.27319 handle_command_reply: tid=0 2024-09-13T18:30:37.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.737+0000 7f94313e3740 1 client.27319 shutdown 2024-09-13T18:30:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.737+0000 7f0f4659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646a309be20 0x5646a30a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.737+0000 7f0f4659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646a309be20 0x5646a30a6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:58072/0 (socket says 172.21.15.114:58072) 2024-09-13T18:30:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.737+0000 7f0f4659c640 1 -- 172.21.15.114:0/3896135117 learned_addr learned my addr 172.21.15.114:0/3896135117 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.737+0000 7f0f45d9b640 1 --2- 172.21.15.114:0/3896135117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5646a30a66c0 0x5646a30a8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.739+0000 7f0f46d9d640 1 --2- 172.21.15.114:0/3896135117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5646a309b500 0x5646a309b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:37.742 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.739+0000 7f0f3f7fe640 1 -- 172.21.15.114:0/944271734 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f0f38017be0 con 0x7f0f481074b0 2024-09-13T18:30:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.740+0000 7f0f3f7fe640 1 --2- 172.21.15.114:0/944271734 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0f340a14d0 0x7f0f340a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:37.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.740+0000 7f94313e3740 1 -- 172.21.15.114:0/665289878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641a023a960 msgr2=0x5641a024f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:37.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.740+0000 7f94313e3740 1 --2- 172.21.15.114:0/665289878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641a023a960 0x5641a024f4f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f942009b4f0 tx=0x7f942000d5c0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.740+0000 7f94313e3740 1 -- 172.21.15.114:0/665289878 shutdown_connections 2024-09-13T18:30:37.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.740+0000 7f94313e3740 1 --2- 172.21.15.114:0/665289878 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5641a02a3a00 0x5641a02a5e20 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.740+0000 7f94313e3740 1 --2- 172.21.15.114:0/665289878 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5641a024ddb0 0x5641a024fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.740+0000 7f94313e3740 1 --2- 172.21.15.114:0/665289878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641a023a960 0x5641a024f4f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.740+0000 7f94313e3740 1 --2- 172.21.15.114:0/665289878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5641a023a040 0x5641a02541f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.740+0000 7f94313e3740 1 -- 172.21.15.114:0/665289878 >> 172.21.15.114:0/665289878 conn(0x5641a02366f0 msgr2=0x5641a0234490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.741+0000 7f0f3f7fe640 1 -- 172.21.15.114:0/944271734 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f0f38014070 con 0x7f0f481074b0 2024-09-13T18:30:37.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.741+0000 7f94077fe640 0 client.27319 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:30:37.746 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.742+0000 7f0f46d9d640 1 --2- 172.21.15.114:0/944271734 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0f340a14d0 0x7f0f340a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:37.746 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.742+0000 7f0f46d9d640 1 -- 172.21.15.114:0/3896135117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5646a30a66c0 msgr2=0x5646a30a8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:37.746 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.742+0000 7f0f46d9d640 1 --2- 172.21.15.114:0/3896135117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5646a30a66c0 0x5646a30a8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.746 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.742+0000 7f0f46d9d640 1 -- 172.21.15.114:0/3896135117 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646a309be20 msgr2=0x5646a30a6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:37.746 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.742+0000 7f0f46d9d640 1 --2- 172.21.15.114:0/3896135117 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646a309be20 0x5646a30a6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.746 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.742+0000 7f0f46d9d640 1 -- 172.21.15.114:0/3896135117 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646a2fa6910 con 0x5646a309b500 2024-09-13T18:30:37.746 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.743+0000 7f0f46d9d640 1 --2- 172.21.15.114:0/3896135117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5646a309b500 0x5646a309b8e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0f40009850 tx=0x7f0f400048c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99383e659232628c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:37.747 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.743+0000 7f0f46d9d640 1 --2- 172.21.15.114:0/944271734 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0f340a14d0 0x7f0f340a3990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0f40009230 tx=0x7f0f40045000 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:37.747 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.744+0000 7f0f3cff9640 1 -- 172.21.15.114:0/3896135117 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f40009350 con 0x5646a309b500 2024-09-13T18:30:37.747 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.744+0000 7f0f3cff9640 1 -- 172.21.15.114:0/3896135117 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0f40005390 con 0x5646a309b500 2024-09-13T18:30:37.748 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.745+0000 7f0f4f9d2740 1 -- 172.21.15.114:0/3896135117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5646a309b500 msgr2=0x5646a309b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:37.748 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.745+0000 7f0f4f9d2740 1 --2- 172.21.15.114:0/3896135117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5646a309b500 0x5646a309b8e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0f40009850 tx=0x7f0f400048c0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.748 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.745+0000 7f0f4f9d2740 1 -- 172.21.15.114:0/3896135117 shutdown_connections 2024-09-13T18:30:37.748 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.745+0000 7f0f4f9d2740 1 --2- 172.21.15.114:0/3896135117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5646a30a66c0 0x5646a30a8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.748 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.745+0000 7f0f4f9d2740 1 --2- 172.21.15.114:0/3896135117 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646a309be20 0x5646a30a6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.748 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.745+0000 7f0f4f9d2740 1 --2- 172.21.15.114:0/3896135117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5646a309b500 0x5646a309b8e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.749 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.745+0000 7f0f4f9d2740 1 -- 172.21.15.114:0/3896135117 >> 172.21.15.114:0/3896135117 conn(0x5646a3097a90 msgr2=0x5646a3097e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:37.749 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.745+0000 7f0f4f9d2740 1 -- 172.21.15.114:0/3896135117 shutdown_connections 2024-09-13T18:30:37.749 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.746+0000 7f0f4f9d2740 1 -- 172.21.15.114:0/3896135117 wait complete. 2024-09-13T18:30:37.749 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.746+0000 7f0f4f9d2740 1 -- start start 2024-09-13T18:30:37.749 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.746+0000 7f0f4f9d2740 1 Processor -- start 2024-09-13T18:30:37.750 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.747+0000 7f0f4f9d2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5646a309b500 0x5646a30a6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:37.750 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.747+0000 7f0f4f9d2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5646a309be20 0x5646a30a32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:37.750 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.747+0000 7f0f4f9d2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646a30a66c0 0x5646a30a3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:37.750 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.747+0000 7f0f4f9d2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646a2fbde80 con 0x5646a30a66c0 2024-09-13T18:30:37.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.747+0000 7f0f4f9d2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646a2fbdb20 con 0x5646a309b500 2024-09-13T18:30:37.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.747+0000 7f0f4f9d2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646a2f83260 con 0x5646a309be20 2024-09-13T18:30:37.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.747+0000 7f0f4f9d2740 10 client.0 _ll_register_callbacks cb 0x5646a2fb8df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:30:37.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.748+0000 7f0f46d9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646a30a66c0 0x5646a30a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:37.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.748+0000 7f0f46d9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646a30a66c0 0x5646a30a3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:58088/0 (socket says 172.21.15.114:58088) 2024-09-13T18:30:37.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.748+0000 7f0f46d9d640 1 -- 172.21.15.114:0/4047916134 learned_addr learned my addr 172.21.15.114:0/4047916134 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:37.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.748+0000 7f0f45d9b640 1 --2- 172.21.15.114:0/4047916134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5646a309be20 0x5646a30a32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:37.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.748+0000 7f0f4659c640 1 --2- 172.21.15.114:0/4047916134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5646a309b500 0x5646a30a6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:37.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.748+0000 7f0f45d9b640 1 -- 172.21.15.114:0/4047916134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5646a309b500 msgr2=0x5646a30a6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:37.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.748+0000 7f0f45d9b640 1 --2- 172.21.15.114:0/4047916134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5646a309b500 0x5646a30a6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.748+0000 7f0f45d9b640 1 -- 172.21.15.114:0/4047916134 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646a30a66c0 msgr2=0x5646a30a3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:37.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.748+0000 7f0f45d9b640 1 --2- 172.21.15.114:0/4047916134 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646a30a66c0 0x5646a30a3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.748+0000 7f0f45d9b640 1 -- 172.21.15.114:0/4047916134 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f4811c000 con 0x5646a309be20 2024-09-13T18:30:37.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.749+0000 7f0f45d9b640 1 --2- 172.21.15.114:0/4047916134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5646a309be20 0x5646a30a32d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0f30009620 tx=0x7f0f30012bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5beb7ce492fcae15 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:37.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.749+0000 7f0f2dffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:30:37.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.749+0000 7f94313e3740 1 -- 172.21.15.114:0/665289878 shutdown_connections 2024-09-13T18:30:37.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.749+0000 7f94313e3740 1 -- 172.21.15.114:0/665289878 wait complete. 2024-09-13T18:30:37.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.752+0000 7f0f2dffb640 1 -- 172.21.15.114:0/4047916134 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f30017070 con 0x5646a309be20 2024-09-13T18:30:37.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.752+0000 7f0f4f9d2740 1 -- 172.21.15.114:0/4047916134 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5646a3028680 con 0x5646a309be20 2024-09-13T18:30:37.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.752+0000 7f94313e3740 20 client.27319 trim_cache size 0 max 16384 2024-09-13T18:30:37.756 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.752+0000 7f0f2dffb640 1 -- 172.21.15.114:0/4047916134 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f0f3000f890 con 0x5646a309be20 2024-09-13T18:30:37.756 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.752+0000 7f0f2dffb640 1 -- 172.21.15.114:0/4047916134 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f300122e0 con 0x5646a309be20 2024-09-13T18:30:37.756 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:30:37.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.756+0000 7f942d467640 1 -- 172.21.15.114:0/1948776724 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f94100a1410 msgr2=0x7f94100a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:37.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.756+0000 7f942d467640 1 --2- 172.21.15.114:0/1948776724 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f94100a1410 0x7f94100a38d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9418006010 tx=0x7f9418004040 comp rx=0 tx=0).stop 2024-09-13T18:30:37.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.756+0000 7f942d467640 1 -- 172.21.15.114:0/1948776724 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9428107e30 msgr2=0x7f94281b3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:37.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.756+0000 7f942d467640 1 --2- 172.21.15.114:0/1948776724 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9428107e30 0x7f94281b3ad0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f9420007ef0 tx=0x7f942000ae40 comp rx=0 tx=0).stop 2024-09-13T18:30:37.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.757+0000 7f942d467640 1 -- 172.21.15.114:0/1948776724 shutdown_connections 2024-09-13T18:30:37.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.757+0000 7f942d467640 1 --2- 172.21.15.114:0/1948776724 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f94100a1410 0x7f94100a38d0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.757+0000 7f942d467640 1 --2- 172.21.15.114:0/1948776724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f942810b8d0 0x7f9428073ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.757+0000 7f942d467640 1 --2- 172.21.15.114:0/1948776724 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9428107e30 0x7f94281b3ad0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.757+0000 7f942d467640 1 --2- 172.21.15.114:0/1948776724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94281074b0 0x7f94281b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.757+0000 7f942d467640 1 -- 172.21.15.114:0/1948776724 >> 172.21.15.114:0/1948776724 conn(0x7f942806af40 msgr2=0x7f9428070970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:37.769 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.762+0000 7f0f4f9d2740 10 client.27328 fetch_fsmap learned FSMap version 34 2024-09-13T18:30:37.769 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.762+0000 7f0f4f9d2740 1 -- 172.21.15.114:0/4047916134 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5646a2ffe330 con 0x5646a309be20 2024-09-13T18:30:37.769 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.764+0000 7f0f2dffb640 1 -- 172.21.15.114:0/4047916134 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5646a3028680 con 0x5646a309be20 2024-09-13T18:30:37.769 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.764+0000 7f0f2dffb640 1 -- 172.21.15.114:0/4047916134 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0f3001cc80 con 0x5646a309be20 2024-09-13T18:30:37.769 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.764+0000 7f0f4f9d2740 10 client.27328 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:30:37.769 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.764+0000 7f0f4f9d2740 10 client.27328 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:30:37.770 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.764+0000 7f0f4f9d2740 20 client.27328 populate_metadata read hostname 'smithi114' 2024-09-13T18:30:37.770 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.764+0000 7f0f4f9d2740 1 --2- 172.21.15.114:0/4047916134 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5646a30b6200 0x5646a30b8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:37.770 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.764+0000 7f0f4f9d2740 4 client.27328 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:30:37.770 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.764+0000 7f0f4f9d2740 1 -- 172.21.15.114:0/4047916134 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5646a2ff35d0 con 0x5646a30b6200 2024-09-13T18:30:37.770 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.766+0000 7f0f4659c640 1 --2- 172.21.15.114:0/4047916134 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5646a30b6200 0x5646a30b8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:37.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.766+0000 7f942d467640 1 -- 172.21.15.114:0/1948776724 shutdown_connections 2024-09-13T18:30:37.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:37.766+0000 7f942d467640 1 -- 172.21.15.114:0/1948776724 wait complete. 2024-09-13T18:30:37.774 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.770+0000 7f0f4659c640 1 --2- 172.21.15.114:0/4047916134 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5646a30b6200 0x5646a30b8620 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c17d53d010210116 server_cookie=7adcf82a16df420f in_seq=0 out_seq=0 2024-09-13T18:30:37.774 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.771+0000 7f0f2dffb640 10 client.27328 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:30:37.774 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.771+0000 7f0f2dffb640 1 -- 172.21.15.114:0/4047916134 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5646a2ff35d0 con 0x5646a30b6200 2024-09-13T18:30:37.774 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.771+0000 7f0f2dffb640 10 client.27328 handle_command_reply: tid=0 2024-09-13T18:30:37.774 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.771+0000 7f0f4f9d2740 1 client.27328 shutdown 2024-09-13T18:30:37.776 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.772+0000 7f0f4f9d2740 1 -- 172.21.15.114:0/4047916134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5646a309be20 msgr2=0x5646a30a32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:37.776 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.772+0000 7f0f4f9d2740 1 --2- 172.21.15.114:0/4047916134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5646a309be20 0x5646a30a32d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0f30009620 tx=0x7f0f30012bd0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.776 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.773+0000 7f0f4f9d2740 1 -- 172.21.15.114:0/4047916134 shutdown_connections 2024-09-13T18:30:37.776 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.773+0000 7f0f4f9d2740 1 --2- 172.21.15.114:0/4047916134 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5646a30b6200 0x5646a30b8620 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.776 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.773+0000 7f0f4f9d2740 1 --2- 172.21.15.114:0/4047916134 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646a30a66c0 0x5646a30a3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.776 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.773+0000 7f0f4f9d2740 1 --2- 172.21.15.114:0/4047916134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5646a309be20 0x5646a30a32d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.776 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.773+0000 7f0f4f9d2740 1 --2- 172.21.15.114:0/4047916134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5646a309b500 0x5646a30a6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.777 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.773+0000 7f0f4f9d2740 1 -- 172.21.15.114:0/4047916134 >> 172.21.15.114:0/4047916134 conn(0x5646a3097a90 msgr2=0x5646a3096d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:37.777 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.773+0000 7f0f2dffb640 0 client.27328 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:30:37.781 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.778+0000 7f0f4f9d2740 1 -- 172.21.15.114:0/4047916134 shutdown_connections 2024-09-13T18:30:37.785 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.782+0000 7f0f4f9d2740 1 -- 172.21.15.114:0/4047916134 wait complete. 2024-09-13T18:30:37.798 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.795+0000 7f0f4f9d2740 20 client.27328 trim_cache size 0 max 16384 2024-09-13T18:30:37.828 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.825+0000 7f0f4f9d2740 1 Processor -- start 2024-09-13T18:30:37.829 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.825+0000 7f0f4f9d2740 1 -- start start 2024-09-13T18:30:37.829 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.825+0000 7f0f4f9d2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5646a309b500 0x5646a30a5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:37.829 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.825+0000 7f0f4f9d2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5646a309be20 0x5646a30a1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:37.830 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.826+0000 7f0f4659c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5646a309b500 0x5646a30a5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:37.830 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.826+0000 7f0f4659c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5646a309b500 0x5646a30a5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:58994/0 (socket says 172.21.15.114:58994) 2024-09-13T18:30:37.830 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.826+0000 7f0f45d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5646a309be20 0x5646a30a1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:37.830 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.826+0000 7f0f45d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5646a309be20 0x5646a30a1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:40554/0 (socket says 172.21.15.114:40554) 2024-09-13T18:30:37.830 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.826+0000 7f0f4f9d2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646a30a66c0 0x5646a30a20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:37.830 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.826+0000 7f0f4f9d2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646a30b2da0 con 0x5646a30a66c0 2024-09-13T18:30:37.830 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.826+0000 7f0f4f9d2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646a30b4950 con 0x5646a309be20 2024-09-13T18:30:37.830 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.826+0000 7f0f4f9d2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646a30a2630 con 0x5646a309b500 2024-09-13T18:30:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.826+0000 7f0f4659c640 1 -- 172.21.15.114:0/3704817960 learned_addr learned my addr 172.21.15.114:0/3704817960 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.826+0000 7f0f46d9d640 1 --2- 172.21.15.114:0/3704817960 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646a30a66c0 0x5646a30a20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.826+0000 7f0f4659c640 1 -- 172.21.15.114:0/3704817960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5646a309be20 msgr2=0x5646a30a1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.826+0000 7f0f4659c640 1 --2- 172.21.15.114:0/3704817960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5646a309be20 0x5646a30a1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.826+0000 7f0f4659c640 1 -- 172.21.15.114:0/3704817960 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646a30a66c0 msgr2=0x5646a30a20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.826+0000 7f0f4659c640 1 --2- 172.21.15.114:0/3704817960 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646a30a66c0 0x5646a30a20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.826+0000 7f0f4659c640 1 -- 172.21.15.114:0/3704817960 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646a2fa0300 con 0x5646a309b500 2024-09-13T18:30:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.827+0000 7f0f4659c640 1 --2- 172.21.15.114:0/3704817960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5646a309b500 0x5646a30a5b40 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0f3809fad0 tx=0x7f0f380a2040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90a8a7521e61b3a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:37.834 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.828+0000 7f0f3d7fa640 1 -- 172.21.15.114:0/3704817960 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f380a04d0 con 0x5646a309b500 2024-09-13T18:30:37.834 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.828+0000 7f0f3d7fa640 1 -- 172.21.15.114:0/3704817960 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0f3809fdd0 con 0x5646a309b500 2024-09-13T18:30:37.834 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.828+0000 7f0f3d7fa640 1 -- 172.21.15.114:0/3704817960 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3809c5f0 con 0x5646a309b500 2024-09-13T18:30:37.834 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.829+0000 7f0f4f9d2740 1 -- 172.21.15.114:0/3704817960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5646a309b500 msgr2=0x5646a30a5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:37.834 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.829+0000 7f0f4f9d2740 1 --2- 172.21.15.114:0/3704817960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5646a309b500 0x5646a30a5b40 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0f3809fad0 tx=0x7f0f380a2040 comp rx=0 tx=0).stop 2024-09-13T18:30:37.834 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.830+0000 7f0f4f9d2740 1 -- 172.21.15.114:0/3704817960 shutdown_connections 2024-09-13T18:30:37.834 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.830+0000 7f0f4f9d2740 1 --2- 172.21.15.114:0/3704817960 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646a30a66c0 0x5646a30a20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.834 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.830+0000 7f0f4f9d2740 1 --2- 172.21.15.114:0/3704817960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5646a309be20 0x5646a30a1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.834 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.830+0000 7f0f4f9d2740 1 --2- 172.21.15.114:0/3704817960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5646a309b500 0x5646a30a5b40 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.834 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.830+0000 7f0f4f9d2740 1 -- 172.21.15.114:0/3704817960 >> 172.21.15.114:0/3704817960 conn(0x5646a3097a90 msgr2=0x5646a309e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.834+0000 7f0f4f9d2740 1 -- 172.21.15.114:0/3704817960 shutdown_connections 2024-09-13T18:30:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.836+0000 7f0f4f9d2740 1 -- 172.21.15.114:0/3704817960 wait complete. 2024-09-13T18:30:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.836+0000 7f0f4f9d2740 1 -- start start 2024-09-13T18:30:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.837+0000 7f0f4f9d2740 1 Processor -- start 2024-09-13T18:30:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.837+0000 7f0f4f9d2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646a309b500 0x5646a30ad5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.837+0000 7f0f4f9d2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5646a309be20 0x5646a309fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.837+0000 7f0f4f9d2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5646a30a66c0 0x5646a30a0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.837+0000 7f0f4f9d2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646a30a0580 con 0x5646a309b500 2024-09-13T18:30:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.837+0000 7f0f4f9d2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646a30a06f0 con 0x5646a309be20 2024-09-13T18:30:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.837+0000 7f0f4f9d2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646a30a08b0 con 0x5646a30a66c0 2024-09-13T18:30:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.837+0000 7f0f4f9d2740 10 client.0 _ll_register_callbacks cb 0x7f0f30005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:30:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.838+0000 7f0f4659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646a309b500 0x5646a30ad5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.838+0000 7f0f4659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646a309b500 0x5646a30ad5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:58098/0 (socket says 172.21.15.114:58098) 2024-09-13T18:30:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.838+0000 7f0f4659c640 1 -- 172.21.15.114:0/3830450481 learned_addr learned my addr 172.21.15.114:0/3830450481 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.838+0000 7f0f46d9d640 1 --2- 172.21.15.114:0/3830450481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5646a30a66c0 0x5646a30a0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.838+0000 7f0f45d9b640 1 --2- 172.21.15.114:0/3830450481 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5646a309be20 0x5646a309fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.839+0000 7f0f45d9b640 1 -- 172.21.15.114:0/3830450481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5646a30a66c0 msgr2=0x5646a30a0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.839+0000 7f0f45d9b640 1 --2- 172.21.15.114:0/3830450481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5646a30a66c0 0x5646a30a0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.839+0000 7f0f45d9b640 1 -- 172.21.15.114:0/3830450481 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646a309b500 msgr2=0x5646a30ad5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.839+0000 7f0f45d9b640 1 --2- 172.21.15.114:0/3830450481 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646a309b500 0x5646a30ad5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.839+0000 7f0f45d9b640 1 -- 172.21.15.114:0/3830450481 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f48124840 con 0x5646a309be20 2024-09-13T18:30:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.839+0000 7f0f45d9b640 1 --2- 172.21.15.114:0/3830450481 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5646a309be20 0x5646a309fb00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0f30010500 tx=0x7f0f30009f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c15b10377dbbd09 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.839+0000 7f0f2dffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:30:37.844 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.840+0000 7f0f2dffb640 1 -- 172.21.15.114:0/3830450481 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f30017040 con 0x5646a309be20 2024-09-13T18:30:37.844 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.840+0000 7f0f4f9d2740 1 -- 172.21.15.114:0/3830450481 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0f40005390 con 0x5646a309be20 2024-09-13T18:30:37.844 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.840+0000 7f0f2dffb640 1 -- 172.21.15.114:0/3830450481 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f0f30010770 con 0x5646a309be20 2024-09-13T18:30:37.844 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.841+0000 7f0f2dffb640 1 -- 172.21.15.114:0/3830450481 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f300052d0 con 0x5646a309be20 2024-09-13T18:30:37.845 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.841+0000 7f0f4f9d2740 10 client.27356 fetch_fsmap learned FSMap version 34 2024-09-13T18:30:37.845 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.841+0000 7f0f4f9d2740 1 -- 172.21.15.114:0/3830450481 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5646a2fc58e0 con 0x5646a309be20 2024-09-13T18:30:37.845 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.842+0000 7f0f2dffb640 1 -- 172.21.15.114:0/3830450481 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0f30033080 con 0x5646a309be20 2024-09-13T18:30:37.845 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.842+0000 7f0f2dffb640 1 -- 172.21.15.114:0/3830450481 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0f30010d20 con 0x5646a309be20 2024-09-13T18:30:37.846 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.843+0000 7f0f4f9d2740 10 client.27356 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:30:37.846 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.843+0000 7f0f4f9d2740 10 client.27356 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:30:37.846 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.843+0000 7f0f4f9d2740 20 client.27356 populate_metadata read hostname 'smithi114' 2024-09-13T18:30:37.846 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.843+0000 7f0f4f9d2740 1 --2- 172.21.15.114:0/3830450481 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5646a310db40 0x5646a310ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:37.846 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.843+0000 7f0f4f9d2740 4 client.27356 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T18:30:37.846 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.843+0000 7f0f4f9d2740 1 -- 172.21.15.114:0/3830450481 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5646a2cf3300 con 0x5646a310db40 2024-09-13T18:30:37.847 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.843+0000 7f0f4659c640 1 --2- 172.21.15.114:0/3830450481 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5646a310db40 0x5646a310ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.847+0000 7f0f2dffb640 10 client.27356 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:30:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.848+0000 7f0f4659c640 1 --2- 172.21.15.114:0/3830450481 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5646a310db40 0x5646a310ff60 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47ef2d8daf73e3c9 server_cookie=410da21947e4ef8d in_seq=0 out_seq=0 2024-09-13T18:30:37.868 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:37 smithi134 ceph-mon[34283]: pgmap v251: 129 pgs: 129 active+clean; 23 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 38 MiB/s wr, 802 op/s 2024-09-13T18:30:37.870 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.867+0000 7f0f2dffb640 1 -- 172.21.15.114:0/3830450481 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5646a2cf3300 con 0x5646a310db40 2024-09-13T18:30:37.870 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.867+0000 7f0f2dffb640 10 client.27356 handle_command_reply: tid=0 2024-09-13T18:30:37.870 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.867+0000 7f0f4f9d2740 1 client.27356 shutdown 2024-09-13T18:30:37.870 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.867+0000 7f0f4f9d2740 1 -- 172.21.15.114:0/3830450481 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5646a309be20 msgr2=0x5646a309fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.867+0000 7f0f4f9d2740 1 --2- 172.21.15.114:0/3830450481 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5646a309be20 0x5646a309fb00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0f30010500 tx=0x7f0f30009f10 comp rx=0 tx=0).stop 2024-09-13T18:30:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.867+0000 7f0f4f9d2740 1 -- 172.21.15.114:0/3830450481 shutdown_connections 2024-09-13T18:30:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.867+0000 7f0f4f9d2740 1 --2- 172.21.15.114:0/3830450481 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5646a310db40 0x5646a310ff60 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.867+0000 7f0f4f9d2740 1 --2- 172.21.15.114:0/3830450481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5646a30a66c0 0x5646a30a0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.867+0000 7f0f4f9d2740 1 --2- 172.21.15.114:0/3830450481 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5646a309be20 0x5646a309fb00 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.867+0000 7f0f4f9d2740 1 --2- 172.21.15.114:0/3830450481 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646a309b500 0x5646a30ad5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.867+0000 7f0f4f9d2740 1 -- 172.21.15.114:0/3830450481 >> 172.21.15.114:0/3830450481 conn(0x5646a3097a90 msgr2=0x5646a309e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.872+0000 7f0f2dffb640 0 client.27356 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:30:37.879 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.875+0000 7f0f4f9d2740 1 -- 172.21.15.114:0/3830450481 shutdown_connections 2024-09-13T18:30:37.879 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.875+0000 7f0f4f9d2740 1 -- 172.21.15.114:0/3830450481 wait complete. 2024-09-13T18:30:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.877+0000 7f0f4f9d2740 20 client.27356 trim_cache size 0 max 16384 2024-09-13T18:30:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-13T18:30:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-13T18:30:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-13T18:30:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-13T18:30:37.884 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.881+0000 7f0f4759e640 1 -- 172.21.15.114:0/944271734 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0f340a14d0 msgr2=0x7f0f340a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:37.884 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.881+0000 7f0f4759e640 1 --2- 172.21.15.114:0/944271734 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0f340a14d0 0x7f0f340a3990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0f40009230 tx=0x7f0f40045000 comp rx=0 tx=0).stop 2024-09-13T18:30:37.884 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.881+0000 7f0f4759e640 1 -- 172.21.15.114:0/944271734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f481074b0 msgr2=0x7f0f48114b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:37.885 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.881+0000 7f0f4759e640 1 --2- 172.21.15.114:0/944271734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f481074b0 0x7f0f48114b90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0f3800eb00 tx=0x7f0f3800c2d0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.883+0000 7f0f4759e640 1 -- 172.21.15.114:0/944271734 shutdown_connections 2024-09-13T18:30:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.883+0000 7f0f4759e640 1 --2- 172.21.15.114:0/944271734 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0f340a14d0 0x7f0f340a3990 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.883+0000 7f0f4759e640 1 --2- 172.21.15.114:0/944271734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f48115610 0x7f0f481196a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.883+0000 7f0f4759e640 1 --2- 172.21.15.114:0/944271734 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f48107890 0x7f0f481150d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.883+0000 7f0f4759e640 1 --2- 172.21.15.114:0/944271734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f481074b0 0x7f0f48114b90 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.884+0000 7f0f4759e640 1 -- 172.21.15.114:0/944271734 >> 172.21.15.114:0/944271734 conn(0x7f0f4806adb0 msgr2=0x7f0f48070750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.888+0000 7f0f4759e640 1 -- 172.21.15.114:0/944271734 shutdown_connections 2024-09-13T18:30:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:37.888+0000 7f0f4759e640 1 -- 172.21.15.114:0/944271734 wait complete. 2024-09-13T18:30:37.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:37 smithi204 ceph-mon[34777]: pgmap v251: 129 pgs: 129 active+clean; 23 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 38 MiB/s wr, 802 op/s 2024-09-13T18:30:38.820 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-13T18:30:38.821 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-09-13T18:30:39.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.085+0000 7f7bc759e640 1 -- 172.21.15.114:0/696570349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bc81062e0 msgr2=0x7f7bc8106700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:39.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.085+0000 7f7bc759e640 1 --2- 172.21.15.114:0/696570349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bc81062e0 0x7f7bc8106700 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8009930 tx=0x7f7bb802a480 comp rx=0 tx=0).stop 2024-09-13T18:30:39.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.086+0000 7f7bc759e640 1 -- 172.21.15.114:0/696570349 shutdown_connections 2024-09-13T18:30:39.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.086+0000 7f7bc759e640 1 --2- 172.21.15.114:0/696570349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bc8106c40 0x7f7bc81130d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.086+0000 7f7bc759e640 1 --2- 172.21.15.114:0/696570349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bc81062e0 0x7f7bc8106700 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.086+0000 7f7bc759e640 1 --2- 172.21.15.114:0/696570349 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7bc810c2a0 0x7f7bc810c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.086+0000 7f7bc759e640 1 -- 172.21.15.114:0/696570349 >> 172.21.15.114:0/696570349 conn(0x7f7bc8075b80 msgr2=0x7f7bc8077fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:39.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.091+0000 7f7bc759e640 1 -- 172.21.15.114:0/696570349 shutdown_connections 2024-09-13T18:30:39.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.094+0000 7f7bc759e640 1 -- 172.21.15.114:0/696570349 wait complete. 2024-09-13T18:30:39.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.095+0000 7f7bc759e640 1 Processor -- start 2024-09-13T18:30:39.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.098+0000 7f7bc759e640 1 -- start start 2024-09-13T18:30:39.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.098+0000 7f7bc759e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7bc81062e0 0x7f7bc8080a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:39.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.098+0000 7f7bc759e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bc8106c40 0x7f7bc8080f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:39.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.098+0000 7f7bc759e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bc810c2a0 0x7f7bc8079c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:39.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.098+0000 7f7bc759e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc807a420 con 0x7f7bc81062e0 2024-09-13T18:30:39.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.098+0000 7f7bc759e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc807a560 con 0x7f7bc8106c40 2024-09-13T18:30:39.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.098+0000 7f7bc759e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc807a720 con 0x7f7bc810c2a0 2024-09-13T18:30:39.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.098+0000 7f7bc5d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bc8106c40 0x7f7bc8080f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:39.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.099+0000 7f7bc5d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bc8106c40 0x7f7bc8080f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:40584/0 (socket says 172.21.15.114:40584) 2024-09-13T18:30:39.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.099+0000 7f7bc5d9b640 1 -- 172.21.15.114:0/878701420 learned_addr learned my addr 172.21.15.114:0/878701420 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:39.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.099+0000 7f7bc6d9d640 1 --2- 172.21.15.114:0/878701420 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bc810c2a0 0x7f7bc8079c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:39.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.100+0000 7f7bc5d9b640 1 -- 172.21.15.114:0/878701420 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bc810c2a0 msgr2=0x7f7bc8079c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:39.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.100+0000 7f7bc5d9b640 1 --2- 172.21.15.114:0/878701420 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bc810c2a0 0x7f7bc8079c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.100+0000 7f7bc5d9b640 1 -- 172.21.15.114:0/878701420 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7bc81062e0 msgr2=0x7f7bc8080a50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:30:39.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.100+0000 7f7bc5d9b640 1 --2- 172.21.15.114:0/878701420 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7bc81062e0 0x7f7bc8080a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.100+0000 7f7bc5d9b640 1 -- 172.21.15.114:0/878701420 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb8009560 con 0x7f7bc8106c40 2024-09-13T18:30:39.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.100+0000 7f7bc5d9b640 1 --2- 172.21.15.114:0/878701420 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bc8106c40 0x7f7bc8080f90 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8006010 tx=0x7f7bb80048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:39.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.104+0000 7f7bb77fe640 1 -- 172.21.15.114:0/878701420 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb80035f0 con 0x7f7bc8106c40 2024-09-13T18:30:39.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.104+0000 7f7bb77fe640 1 -- 172.21.15.114:0/878701420 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7bb8003be0 con 0x7f7bc8106c40 2024-09-13T18:30:39.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.104+0000 7f7bb77fe640 1 -- 172.21.15.114:0/878701420 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb803cb80 con 0x7f7bc8106c40 2024-09-13T18:30:39.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.105+0000 7f7bc759e640 1 -- 172.21.15.114:0/878701420 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bc807e570 con 0x7f7bc8106c40 2024-09-13T18:30:39.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.105+0000 7f7bc759e640 1 -- 172.21.15.114:0/878701420 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bc807e9d0 con 0x7f7bc8106c40 2024-09-13T18:30:39.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.111+0000 7f7bcf866740 1 Processor -- start 2024-09-13T18:30:39.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.112+0000 7f7bcf866740 1 -- start start 2024-09-13T18:30:39.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.112+0000 7f7bcf866740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ca4cf39e0 0x556ca4cf3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:39.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.112+0000 7f7bcf866740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556ca4cf4300 0x556ca4cfe660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:39.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.112+0000 7f7bcf866740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ca4cfeba0 0x556ca4d00f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:39.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.112+0000 7f7bcf866740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ca4c5ec60 con 0x556ca4cf4300 2024-09-13T18:30:39.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.112+0000 7f7bcf866740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ca4c5c3f0 con 0x556ca4cf39e0 2024-09-13T18:30:39.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.112+0000 7f7bcf866740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ca4c57ce0 con 0x556ca4cfeba0 2024-09-13T18:30:39.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.113+0000 7f7bc659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556ca4cf4300 0x556ca4cfe660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:39.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.113+0000 7f7bc659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556ca4cf4300 0x556ca4cfe660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:58118/0 (socket says 172.21.15.114:58118) 2024-09-13T18:30:39.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.113+0000 7f7bc659c640 1 -- 172.21.15.114:0/4253546674 learned_addr learned my addr 172.21.15.114:0/4253546674 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:39.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.113+0000 7f7bc6d9d640 1 --2- 172.21.15.114:0/4253546674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ca4cf39e0 0x556ca4cf3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:39.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.113+0000 7f7bc659c640 1 -- 172.21.15.114:0/4253546674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ca4cfeba0 msgr2=0x556ca4d00f90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:30:39.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.113+0000 7f7bc659c640 1 --2- 172.21.15.114:0/4253546674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ca4cfeba0 0x556ca4d00f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.113+0000 7f7bc659c640 1 -- 172.21.15.114:0/4253546674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ca4cf39e0 msgr2=0x556ca4cf3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:39.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.113+0000 7f7bc659c640 1 --2- 172.21.15.114:0/4253546674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ca4cf39e0 0x556ca4cf3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.114+0000 7f7bc659c640 1 -- 172.21.15.114:0/4253546674 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ca4c13970 con 0x556ca4cf4300 2024-09-13T18:30:39.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.114+0000 7f7bc659c640 1 --2- 172.21.15.114:0/4253546674 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556ca4cf4300 0x556ca4cfe660 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0009d80 tx=0x7f7bc000b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebfea79687623936 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:39.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.118+0000 7f7bb77fe640 1 -- 172.21.15.114:0/878701420 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f7bb8003750 con 0x7f7bc8106c40 2024-09-13T18:30:39.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.118+0000 7f7bb4ff9640 1 -- 172.21.15.114:0/4253546674 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc000e070 con 0x556ca4cf4300 2024-09-13T18:30:39.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.118+0000 7f7bb4ff9640 1 -- 172.21.15.114:0/4253546674 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7bc000bd40 con 0x556ca4cf4300 2024-09-13T18:30:39.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.118+0000 7f7bb4ff9640 1 -- 172.21.15.114:0/4253546674 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc000e070 con 0x556ca4cf4300 2024-09-13T18:30:39.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.119+0000 7f7bb77fe640 1 --2- 172.21.15.114:0/878701420 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7bb00a1410 0x7f7bb00a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:39.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.119+0000 7f7bb77fe640 1 -- 172.21.15.114:0/878701420 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f7bb80ee270 con 0x7f7bc8106c40 2024-09-13T18:30:39.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.121+0000 7f7bc6d9d640 1 --2- 172.21.15.114:0/878701420 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7bb00a1410 0x7f7bb00a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:39.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.121+0000 7f7bcf866740 1 -- 172.21.15.114:0/4253546674 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556ca4cf4300 msgr2=0x556ca4cfe660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:39.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.121+0000 7f7bcf866740 1 --2- 172.21.15.114:0/4253546674 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556ca4cf4300 0x556ca4cfe660 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0009d80 tx=0x7f7bc000b420 comp rx=0 tx=0).stop 2024-09-13T18:30:39.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.124+0000 7f7bcf866740 1 -- 172.21.15.114:0/4253546674 shutdown_connections 2024-09-13T18:30:39.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.124+0000 7f7bcf866740 1 --2- 172.21.15.114:0/4253546674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ca4cfeba0 0x556ca4d00f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.124+0000 7f7bcf866740 1 --2- 172.21.15.114:0/4253546674 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556ca4cf4300 0x556ca4cfe660 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.124+0000 7f7bcf866740 1 --2- 172.21.15.114:0/4253546674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ca4cf39e0 0x556ca4cf3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.124+0000 7f7bcf866740 1 -- 172.21.15.114:0/4253546674 >> 172.21.15.114:0/4253546674 conn(0x556ca4ceff70 msgr2=0x556ca4cf0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:39.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.124+0000 7f7bc6d9d640 1 --2- 172.21.15.114:0/878701420 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7bb00a1410 0x7f7bb00a38d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc006fd0 tx=0x7f7bbc008040 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:39.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.125+0000 7f7bcf866740 1 -- 172.21.15.114:0/4253546674 shutdown_connections 2024-09-13T18:30:39.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.129+0000 7f7bcf866740 1 -- 172.21.15.114:0/4253546674 wait complete. 2024-09-13T18:30:39.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.129+0000 7f7bcf866740 1 -- start start 2024-09-13T18:30:39.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.130+0000 7f7bcf866740 1 Processor -- start 2024-09-13T18:30:39.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.132+0000 7f7bcf866740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556ca4cf39e0 0x556ca4cfe4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:39.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.132+0000 7f7bcf866740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ca4cf4300 0x556ca4cfb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:39.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.132+0000 7f7bcf866740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ca4cfeba0 0x556ca4cfbd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:39.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.132+0000 7f7bcf866740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ca4c15f80 con 0x556ca4cf39e0 2024-09-13T18:30:39.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.132+0000 7f7bcf866740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ca4c15c20 con 0x556ca4cfeba0 2024-09-13T18:30:39.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.132+0000 7f7bcf866740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ca4be2ab0 con 0x556ca4cf4300 2024-09-13T18:30:39.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.133+0000 7f7bcf866740 10 client.0 _ll_register_callbacks cb 0x556ca4c51350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:30:39.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.134+0000 7f7bc5d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ca4cf4300 0x556ca4cfb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:39.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.134+0000 7f7bc5d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ca4cf4300 0x556ca4cfb770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:59002/0 (socket says 172.21.15.114:59002) 2024-09-13T18:30:39.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.134+0000 7f7bc5d9b640 1 -- 172.21.15.114:0/1696822848 learned_addr learned my addr 172.21.15.114:0/1696822848 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:39.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.134+0000 7f7bc659c640 1 --2- 172.21.15.114:0/1696822848 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556ca4cf39e0 0x556ca4cfe4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:39.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.135+0000 7f7bc6d9d640 1 --2- 172.21.15.114:0/1696822848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ca4cfeba0 0x556ca4cfbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:39.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.135+0000 7f7bc5d9b640 1 -- 172.21.15.114:0/1696822848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ca4cfeba0 msgr2=0x556ca4cfbd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:39.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.135+0000 7f7bc5d9b640 1 --2- 172.21.15.114:0/1696822848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ca4cfeba0 0x556ca4cfbd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.135+0000 7f7bc5d9b640 1 -- 172.21.15.114:0/1696822848 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556ca4cf39e0 msgr2=0x556ca4cfe4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:39.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.135+0000 7f7bc5d9b640 1 --2- 172.21.15.114:0/1696822848 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556ca4cf39e0 0x556ca4cfe4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.135+0000 7f7bc5d9b640 1 -- 172.21.15.114:0/1696822848 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bc0009a30 con 0x556ca4cf4300 2024-09-13T18:30:39.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.135+0000 7f7bc5d9b640 1 --2- 172.21.15.114:0/1696822848 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ca4cf4300 0x556ca4cfb770 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8004030 tx=0x7f7bb8003a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99037a2f0289c715 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:39.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.135+0000 7f7badffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:30:39.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.136+0000 7f7badffb640 1 -- 172.21.15.114:0/1696822848 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb80bedb0 con 0x556ca4cf4300 2024-09-13T18:30:39.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.136+0000 7f7bcf866740 1 -- 172.21.15.114:0/1696822848 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556ca494b590 con 0x556ca4cf4300 2024-09-13T18:30:39.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.136+0000 7f7badffb640 1 -- 172.21.15.114:0/1696822848 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f7bb80c3400 con 0x556ca4cf4300 2024-09-13T18:30:39.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.136+0000 7f7badffb640 1 -- 172.21.15.114:0/1696822848 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7bb80b7690 con 0x556ca4cf4300 2024-09-13T18:30:39.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.136+0000 7f7bcf866740 10 client.27340 fetch_fsmap learned FSMap version 34 2024-09-13T18:30:39.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.136+0000 7f7bcf866740 1 -- 172.21.15.114:0/1696822848 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x556ca4c56360 con 0x556ca4cf4300 2024-09-13T18:30:39.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.139+0000 7f7badffb640 1 -- 172.21.15.114:0/1696822848 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x556ca494b590 con 0x556ca4cf4300 2024-09-13T18:30:39.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.139+0000 7f7badffb640 1 -- 172.21.15.114:0/1696822848 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7bb80ba470 con 0x556ca4cf4300 2024-09-13T18:30:39.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.139+0000 7f7bcf866740 10 client.27340 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:30:39.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.139+0000 7f7bcf866740 10 client.27340 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:30:39.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.139+0000 7f7bcf866740 20 client.27340 populate_metadata read hostname 'smithi114' 2024-09-13T18:30:39.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.139+0000 7f7bcf866740 1 --2- 172.21.15.114:0/1696822848 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x556ca4d0e660 0x556ca4d10a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:39.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.139+0000 7f7bcf866740 4 client.27340 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:30:39.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.139+0000 7f7bcf866740 1 -- 172.21.15.114:0/1696822848 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556ca495c860 con 0x556ca4d0e660 2024-09-13T18:30:39.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.140+0000 7f7bc659c640 1 --2- 172.21.15.114:0/1696822848 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x556ca4d0e660 0x556ca4d10a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:39.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.144+0000 7f7bc659c640 1 --2- 172.21.15.114:0/1696822848 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x556ca4d0e660 0x556ca4d10a80 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab0221d385efbe3d server_cookie=7fed1fca51d19b80 in_seq=0 out_seq=0 2024-09-13T18:30:39.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.144+0000 7f7badffb640 10 client.27340 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:30:39.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.148+0000 7f7badffb640 1 -- 172.21.15.114:0/1696822848 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x556ca495c860 con 0x556ca4d0e660 2024-09-13T18:30:39.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.148+0000 7f7badffb640 10 client.27340 handle_command_reply: tid=0 2024-09-13T18:30:39.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.151+0000 7f7bcf866740 1 client.27340 shutdown 2024-09-13T18:30:39.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.161+0000 7f7bcf866740 1 -- 172.21.15.114:0/1696822848 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ca4cf4300 msgr2=0x556ca4cfb770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:39.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.161+0000 7f7bcf866740 1 --2- 172.21.15.114:0/1696822848 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ca4cf4300 0x556ca4cfb770 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8004030 tx=0x7f7bb8003a00 comp rx=0 tx=0).stop 2024-09-13T18:30:39.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.165+0000 7f7bcf866740 1 -- 172.21.15.114:0/1696822848 shutdown_connections 2024-09-13T18:30:39.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.165+0000 7f7bcf866740 1 --2- 172.21.15.114:0/1696822848 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x556ca4d0e660 0x556ca4d10a80 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.165+0000 7f7bcf866740 1 --2- 172.21.15.114:0/1696822848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ca4cfeba0 0x556ca4cfbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.165+0000 7f7bcf866740 1 --2- 172.21.15.114:0/1696822848 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ca4cf4300 0x556ca4cfb770 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.165+0000 7f7bcf866740 1 --2- 172.21.15.114:0/1696822848 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556ca4cf39e0 0x556ca4cfe4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.165+0000 7f7bcf866740 1 -- 172.21.15.114:0/1696822848 >> 172.21.15.114:0/1696822848 conn(0x556ca4ceff70 msgr2=0x556ca4cef160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:39.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.166+0000 7f7badffb640 0 client.27340 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:30:39.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.166+0000 7f7bcf866740 1 -- 172.21.15.114:0/1696822848 shutdown_connections 2024-09-13T18:30:39.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.166+0000 7f7bcf866740 1 -- 172.21.15.114:0/1696822848 wait complete. 2024-09-13T18:30:39.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.171+0000 7f7bcf866740 20 client.27340 trim_cache size 0 max 16384 2024-09-13T18:30:39.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.208+0000 7f7bcf866740 1 Processor -- start 2024-09-13T18:30:39.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.208+0000 7f7bcf866740 1 -- start start 2024-09-13T18:30:39.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.208+0000 7f7bcf866740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556ca4ceff70 0x556ca4d153a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:39.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.208+0000 7f7bcf866740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ca4cf39e0 0x556ca4d058c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:39.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.208+0000 7f7bcf866740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ca4cf4300 0x556ca4d05e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:39.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.208+0000 7f7bcf866740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ca4d158e0 con 0x556ca4ceff70 2024-09-13T18:30:39.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.208+0000 7f7bcf866740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ca4cf1100 con 0x556ca4cf39e0 2024-09-13T18:30:39.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.208+0000 7f7bcf866740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ca4d01650 con 0x556ca4cf4300 2024-09-13T18:30:39.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.209+0000 7f7bc6d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ca4cf4300 0x556ca4d05e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:39.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.209+0000 7f7bc6d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ca4cf4300 0x556ca4d05e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:59016/0 (socket says 172.21.15.114:59016) 2024-09-13T18:30:39.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.209+0000 7f7bc6d9d640 1 -- 172.21.15.114:0/2608936383 learned_addr learned my addr 172.21.15.114:0/2608936383 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:39.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.209+0000 7f7bc659c640 1 --2- 172.21.15.114:0/2608936383 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556ca4ceff70 0x556ca4d153a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:39.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.209+0000 7f7bc5d9b640 1 --2- 172.21.15.114:0/2608936383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ca4cf39e0 0x556ca4d058c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:39.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.209+0000 7f7bc6d9d640 1 -- 172.21.15.114:0/2608936383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ca4cf39e0 msgr2=0x556ca4d058c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:39.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.209+0000 7f7bc6d9d640 1 --2- 172.21.15.114:0/2608936383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ca4cf39e0 0x556ca4d058c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.209+0000 7f7bc6d9d640 1 -- 172.21.15.114:0/2608936383 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556ca4ceff70 msgr2=0x556ca4d153a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:39.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.209+0000 7f7bc6d9d640 1 --2- 172.21.15.114:0/2608936383 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556ca4ceff70 0x556ca4d153a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.209+0000 7f7bc6d9d640 1 -- 172.21.15.114:0/2608936383 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ca4bf81c0 con 0x556ca4cf4300 2024-09-13T18:30:39.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.209+0000 7f7bc6d9d640 1 --2- 172.21.15.114:0/2608936383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ca4cf4300 0x556ca4d05e00 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc00f960 tx=0x7f7bbc012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb7642cf1463007f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:39.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.211+0000 7f7bb57fa640 1 -- 172.21.15.114:0/2608936383 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc014be0 con 0x556ca4cf4300 2024-09-13T18:30:39.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.211+0000 7f7bb57fa640 1 -- 172.21.15.114:0/2608936383 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7bbc014d40 con 0x556ca4cf4300 2024-09-13T18:30:39.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.211+0000 7f7bb57fa640 1 -- 172.21.15.114:0/2608936383 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc015410 con 0x556ca4cf4300 2024-09-13T18:30:39.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.213+0000 7f7bcf866740 1 -- 172.21.15.114:0/2608936383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ca4cf4300 msgr2=0x556ca4d05e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:39.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.213+0000 7f7bcf866740 1 --2- 172.21.15.114:0/2608936383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ca4cf4300 0x556ca4d05e00 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc00f960 tx=0x7f7bbc012000 comp rx=0 tx=0).stop 2024-09-13T18:30:39.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.213+0000 7f7bcf866740 1 -- 172.21.15.114:0/2608936383 shutdown_connections 2024-09-13T18:30:39.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.213+0000 7f7bcf866740 1 --2- 172.21.15.114:0/2608936383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ca4cf4300 0x556ca4d05e00 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.213+0000 7f7bcf866740 1 --2- 172.21.15.114:0/2608936383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ca4cf39e0 0x556ca4d058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.213+0000 7f7bcf866740 1 --2- 172.21.15.114:0/2608936383 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556ca4ceff70 0x556ca4d153a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.213+0000 7f7bcf866740 1 -- 172.21.15.114:0/2608936383 >> 172.21.15.114:0/2608936383 conn(0x556ca4cfe650 msgr2=0x556ca4cf0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:39.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.213+0000 7f7bcf866740 1 -- 172.21.15.114:0/2608936383 shutdown_connections 2024-09-13T18:30:39.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.213+0000 7f7bcf866740 1 -- 172.21.15.114:0/2608936383 wait complete. 2024-09-13T18:30:39.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.214+0000 7f7bcf866740 1 -- start start 2024-09-13T18:30:39.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.214+0000 7f7bcf866740 1 Processor -- start 2024-09-13T18:30:39.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.214+0000 7f7bcf866740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556ca4ceff70 0x556ca4d02420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:39.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.214+0000 7f7bcf866740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ca4cf39e0 0x556ca4d02960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:39.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.214+0000 7f7bcf866740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ca4cf4300 0x556ca4d02ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:39.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.214+0000 7f7bcf866740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ca4d64b20 con 0x556ca4ceff70 2024-09-13T18:30:39.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.214+0000 7f7bcf866740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ca4d64c60 con 0x556ca4cf39e0 2024-09-13T18:30:39.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.214+0000 7f7bcf866740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ca4d64df0 con 0x556ca4cf4300 2024-09-13T18:30:39.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.215+0000 7f7bcf866740 10 client.0 _ll_register_callbacks cb 0x7f7bb80baac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:30:39.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.215+0000 7f7bc6d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ca4cf4300 0x556ca4d02ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:39.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.215+0000 7f7bc6d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ca4cf4300 0x556ca4d02ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:59018/0 (socket says 172.21.15.114:59018) 2024-09-13T18:30:39.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.215+0000 7f7bc6d9d640 1 -- 172.21.15.114:0/4190601886 learned_addr learned my addr 172.21.15.114:0/4190601886 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:39.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.215+0000 7f7bc5d9b640 1 --2- 172.21.15.114:0/4190601886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ca4cf39e0 0x556ca4d02960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:39.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.215+0000 7f7bc659c640 1 --2- 172.21.15.114:0/4190601886 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556ca4ceff70 0x556ca4d02420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:39.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.215+0000 7f7bc5d9b640 1 -- 172.21.15.114:0/4190601886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ca4cf4300 msgr2=0x556ca4d02ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:39.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.215+0000 7f7bc5d9b640 1 --2- 172.21.15.114:0/4190601886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ca4cf4300 0x556ca4d02ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.215+0000 7f7bc5d9b640 1 -- 172.21.15.114:0/4190601886 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556ca4ceff70 msgr2=0x556ca4d02420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:39.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.215+0000 7f7bc5d9b640 1 --2- 172.21.15.114:0/4190601886 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556ca4ceff70 0x556ca4d02420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.215+0000 7f7bc5d9b640 1 -- 172.21.15.114:0/4190601886 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bbc00e040 con 0x556ca4cf39e0 2024-09-13T18:30:39.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.216+0000 7f7bc5d9b640 1 --2- 172.21.15.114:0/4190601886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ca4cf39e0 0x556ca4d02960 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x556ca4cfc490 tx=0x7f7bb80b9530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ecba6db1d42fc61 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:39.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.216+0000 7f7badffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:30:39.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.217+0000 7f7badffb640 1 -- 172.21.15.114:0/4190601886 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb80eba40 con 0x556ca4cf39e0 2024-09-13T18:30:39.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.217+0000 7f7bcf866740 1 -- 172.21.15.114:0/4190601886 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7bc000bd40 con 0x556ca4cf39e0 2024-09-13T18:30:39.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.218+0000 7f7badffb640 1 -- 172.21.15.114:0/4190601886 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f7bb80c31d0 con 0x556ca4cf39e0 2024-09-13T18:30:39.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.218+0000 7f7badffb640 1 -- 172.21.15.114:0/4190601886 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7bb80c3010 con 0x556ca4cf39e0 2024-09-13T18:30:39.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.221+0000 7f7bcf866740 10 client.27368 fetch_fsmap learned FSMap version 34 2024-09-13T18:30:39.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.221+0000 7f7bcf866740 1 -- 172.21.15.114:0/4190601886 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x556ca4c1d990 con 0x556ca4cf39e0 2024-09-13T18:30:39.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.222+0000 7f7badffb640 1 -- 172.21.15.114:0/4190601886 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7bc000bd40 con 0x556ca4cf39e0 2024-09-13T18:30:39.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.222+0000 7f7badffb640 1 -- 172.21.15.114:0/4190601886 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7bb80b5b90 con 0x556ca4cf39e0 2024-09-13T18:30:39.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.224+0000 7f7bcf866740 10 client.27368 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:30:39.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.224+0000 7f7bcf866740 10 client.27368 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:30:39.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.224+0000 7f7bcf866740 20 client.27368 populate_metadata read hostname 'smithi114' 2024-09-13T18:30:39.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.224+0000 7f7bcf866740 1 --2- 172.21.15.114:0/4190601886 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x556ca4cfeba0 0x556ca4d8a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:39.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.224+0000 7f7bcf866740 4 client.27368 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-13T18:30:39.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.224+0000 7f7bcf866740 1 -- 172.21.15.114:0/4190601886 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x556ca49afeb0 con 0x556ca4cfeba0 2024-09-13T18:30:39.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.227+0000 7f7bc659c640 1 --2- 172.21.15.114:0/4190601886 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x556ca4cfeba0 0x556ca4d8a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:39.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.231+0000 7f7bc659c640 1 --2- 172.21.15.114:0/4190601886 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x556ca4cfeba0 0x556ca4d8a120 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecfa9320b9bbee1c server_cookie=bd921a5c8fbc0587 in_seq=0 out_seq=0 2024-09-13T18:30:39.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.231+0000 7f7badffb640 10 client.27368 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:30:39.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.236+0000 7f7badffb640 1 -- 172.21.15.114:0/4190601886 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x556ca49afeb0 con 0x556ca4cfeba0 2024-09-13T18:30:39.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.236+0000 7f7badffb640 10 client.27368 handle_command_reply: tid=0 2024-09-13T18:30:39.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.237+0000 7f7bcf866740 1 client.27368 shutdown 2024-09-13T18:30:39.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.247+0000 7f7bcf866740 1 -- 172.21.15.114:0/4190601886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ca4cf39e0 msgr2=0x556ca4d02960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:39.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.247+0000 7f7bcf866740 1 --2- 172.21.15.114:0/4190601886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ca4cf39e0 0x556ca4d02960 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x556ca4cfc490 tx=0x7f7bb80b9530 comp rx=0 tx=0).stop 2024-09-13T18:30:39.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.249+0000 7f7bcf866740 1 -- 172.21.15.114:0/4190601886 shutdown_connections 2024-09-13T18:30:39.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.249+0000 7f7bcf866740 1 --2- 172.21.15.114:0/4190601886 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x556ca4cfeba0 0x556ca4d8a120 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.249+0000 7f7bcf866740 1 --2- 172.21.15.114:0/4190601886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ca4cf4300 0x556ca4d02ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.249+0000 7f7bcf866740 1 --2- 172.21.15.114:0/4190601886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556ca4cf39e0 0x556ca4d02960 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.249+0000 7f7bcf866740 1 --2- 172.21.15.114:0/4190601886 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556ca4ceff70 0x556ca4d02420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.249+0000 7f7bcf866740 1 -- 172.21.15.114:0/4190601886 >> 172.21.15.114:0/4190601886 conn(0x556ca4cfe650 msgr2=0x556ca4cf9fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:39.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.249+0000 7f7badffb640 0 client.27368 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:30:39.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.258+0000 7f7bcf866740 1 -- 172.21.15.114:0/4190601886 shutdown_connections 2024-09-13T18:30:39.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.262+0000 7f7bcf866740 1 -- 172.21.15.114:0/4190601886 wait complete. 2024-09-13T18:30:39.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.263+0000 7f7bcf866740 20 client.27368 trim_cache size 0 max 16384 2024-09-13T18:30:39.269 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:30:39.269 INFO:teuthology.orchestra.run.smithi114.stdout: "return_code": 0, 2024-09-13T18:30:39.269 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_tag": "3e348cce-b1ad-4b6f-8e82-203f96a72986", 2024-09-13T18:30:39.269 INFO:teuthology.orchestra.run.smithi114.stdout: "mode": "asynchronous" 2024-09-13T18:30:39.269 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:30:39.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.266+0000 7f7bc759e640 1 -- 172.21.15.114:0/878701420 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7bb00a1410 msgr2=0x7f7bb00a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:39.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.266+0000 7f7bc759e640 1 --2- 172.21.15.114:0/878701420 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7bb00a1410 0x7f7bb00a38d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc006fd0 tx=0x7f7bbc008040 comp rx=0 tx=0).stop 2024-09-13T18:30:39.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.266+0000 7f7bc759e640 1 -- 172.21.15.114:0/878701420 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bc8106c40 msgr2=0x7f7bc8080f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:39.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.267+0000 7f7bc759e640 1 --2- 172.21.15.114:0/878701420 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bc8106c40 0x7f7bc8080f90 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8006010 tx=0x7f7bb80048f0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.267+0000 7f7bc759e640 1 -- 172.21.15.114:0/878701420 shutdown_connections 2024-09-13T18:30:39.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.267+0000 7f7bc759e640 1 --2- 172.21.15.114:0/878701420 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7bb00a1410 0x7f7bb00a38d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.267+0000 7f7bc759e640 1 --2- 172.21.15.114:0/878701420 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bc810c2a0 0x7f7bc8079c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.267+0000 7f7bc759e640 1 --2- 172.21.15.114:0/878701420 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bc8106c40 0x7f7bc8080f90 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.267+0000 7f7bc759e640 1 --2- 172.21.15.114:0/878701420 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7bc81062e0 0x7f7bc8080a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.267+0000 7f7bc759e640 1 -- 172.21.15.114:0/878701420 >> 172.21.15.114:0/878701420 conn(0x7f7bc8075b80 msgr2=0x7f7bc8077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:39.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.269+0000 7f7bc759e640 1 -- 172.21.15.114:0/878701420 shutdown_connections 2024-09-13T18:30:39.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.270+0000 7f7bc759e640 1 -- 172.21.15.114:0/878701420 wait complete. 2024-09-13T18:30:39.293 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T18:30:39.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.536+0000 7fc1acd45640 1 -- 172.21.15.114:0/2090796404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1a8107540 msgr2=0x7fc1a810e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:39.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.536+0000 7fc1acd45640 1 --2- 172.21.15.114:0/2090796404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1a8107540 0x7fc1a810e510 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc198009900 tx=0x7fc19802a440 comp rx=0 tx=0).stop 2024-09-13T18:30:39.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.537+0000 7fc1acd45640 1 -- 172.21.15.114:0/2090796404 shutdown_connections 2024-09-13T18:30:39.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.537+0000 7fc1acd45640 1 --2- 172.21.15.114:0/2090796404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1a8119400 0x7fc1a811b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.537+0000 7fc1acd45640 1 --2- 172.21.15.114:0/2090796404 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc1a8107920 0x7fc1a8107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.537+0000 7fc1acd45640 1 --2- 172.21.15.114:0/2090796404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1a8107540 0x7fc1a810e510 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.537+0000 7fc1acd45640 1 -- 172.21.15.114:0/2090796404 >> 172.21.15.114:0/2090796404 conn(0x7fc1a806afe0 msgr2=0x7fc1a806b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:39.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.539+0000 7fc1acd45640 1 -- 172.21.15.114:0/2090796404 shutdown_connections 2024-09-13T18:30:39.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.543+0000 7fc1acd45640 1 -- 172.21.15.114:0/2090796404 wait complete. 2024-09-13T18:30:39.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.544+0000 7fc1acd45640 1 Processor -- start 2024-09-13T18:30:39.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.544+0000 7fc1acd45640 1 -- start start 2024-09-13T18:30:39.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.548+0000 7fc1acd45640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1a8107540 0x7fc1a81a50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:39.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.548+0000 7fc1acd45640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc1a8107920 0x7fc1a81a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:39.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.548+0000 7fc1acd45640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1a8119400 0x7fc1a81a99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:39.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.548+0000 7fc1acd45640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a81a9fe0 con 0x7fc1a8107920 2024-09-13T18:30:39.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.548+0000 7fc1acd45640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a81aa150 con 0x7fc1a8107540 2024-09-13T18:30:39.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.548+0000 7fc1acd45640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a81aa310 con 0x7fc1a8119400 2024-09-13T18:30:39.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.549+0000 7fc1a659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1a8107540 0x7fc1a81a50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:39.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.549+0000 7fc1a659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1a8107540 0x7fc1a81a50b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:40642/0 (socket says 172.21.15.114:40642) 2024-09-13T18:30:39.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.549+0000 7fc1a659c640 1 -- 172.21.15.114:0/2575691044 learned_addr learned my addr 172.21.15.114:0/2575691044 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:39.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.550+0000 7fc1a659c640 1 -- 172.21.15.114:0/2575691044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1a8119400 msgr2=0x7fc1a81a99e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:30:39.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.550+0000 7fc1a659c640 1 --2- 172.21.15.114:0/2575691044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1a8119400 0x7fc1a81a99e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.550+0000 7fc1a659c640 1 -- 172.21.15.114:0/2575691044 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc1a8107920 msgr2=0x7fc1a81a55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:39.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.550+0000 7fc1a659c640 1 --2- 172.21.15.114:0/2575691044 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc1a8107920 0x7fc1a81a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.550+0000 7fc1a659c640 1 -- 172.21.15.114:0/2575691044 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc198009560 con 0x7fc1a8107540 2024-09-13T18:30:39.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.550+0000 7fc1a659c640 1 --2- 172.21.15.114:0/2575691044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1a8107540 0x7fc1a81a50b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc198005940 tx=0x7fc198032f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:39.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.555+0000 7fc1977fe640 1 -- 172.21.15.114:0/2575691044 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc198031730 con 0x7fc1a8107540 2024-09-13T18:30:39.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.555+0000 7fc1977fe640 1 -- 172.21.15.114:0/2575691044 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc198005450 con 0x7fc1a8107540 2024-09-13T18:30:39.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.555+0000 7fc1977fe640 1 -- 172.21.15.114:0/2575691044 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1980038f0 con 0x7fc1a8107540 2024-09-13T18:30:39.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.557+0000 7fc1acd45640 1 -- 172.21.15.114:0/2575691044 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1a81aa590 con 0x7fc1a8107540 2024-09-13T18:30:39.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.557+0000 7fc1acd45640 1 -- 172.21.15.114:0/2575691044 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1a81b1c90 con 0x7fc1a8107540 2024-09-13T18:30:39.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.561+0000 7fc1afa63740 1 Processor -- start 2024-09-13T18:30:39.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.565+0000 7fc1afa63740 1 -- start start 2024-09-13T18:30:39.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.565+0000 7fc1afa63740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b148310cd0 0x55b1483110b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:39.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.565+0000 7fc1afa63740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1483115f0 0x55b14832caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:39.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.565+0000 7fc1afa63740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b14832cfe0 0x55b14832f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:39.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.566+0000 7fc1afa63740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b14827bf20 con 0x55b14832cfe0 2024-09-13T18:30:39.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.566+0000 7fc1afa63740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1482796b0 con 0x55b1483115f0 2024-09-13T18:30:39.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.566+0000 7fc1afa63740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b148274fa0 con 0x55b148310cd0 2024-09-13T18:30:39.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.569+0000 7fc1a6d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b148310cd0 0x55b1483110b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:39.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.569+0000 7fc1a6d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b148310cd0 0x55b1483110b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:59054/0 (socket says 172.21.15.114:59054) 2024-09-13T18:30:39.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.569+0000 7fc1a6d9d640 1 -- 172.21.15.114:0/917532571 learned_addr learned my addr 172.21.15.114:0/917532571 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:39.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.569+0000 7fc1a5d9b640 1 --2- 172.21.15.114:0/917532571 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b14832cfe0 0x55b14832f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:39.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.574+0000 7fc1977fe640 1 -- 172.21.15.114:0/2575691044 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fc198003a50 con 0x7fc1a8107540 2024-09-13T18:30:39.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.574+0000 7fc1a659c640 1 --2- 172.21.15.114:0/917532571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1483115f0 0x55b14832caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:39.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.574+0000 7fc1a6d9d640 1 -- 172.21.15.114:0/917532571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1483115f0 msgr2=0x55b14832caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:39.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.575+0000 7fc1a6d9d640 1 --2- 172.21.15.114:0/917532571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1483115f0 0x55b14832caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.575+0000 7fc1a6d9d640 1 -- 172.21.15.114:0/917532571 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b14832cfe0 msgr2=0x55b14832f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:39.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.575+0000 7fc1a6d9d640 1 --2- 172.21.15.114:0/917532571 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b14832cfe0 0x55b14832f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.575+0000 7fc1a6d9d640 1 -- 172.21.15.114:0/917532571 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b14822f840 con 0x55b148310cd0 2024-09-13T18:30:39.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.575+0000 7fc1a6d9d640 1 --2- 172.21.15.114:0/917532571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b148310cd0 0x55b1483110b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0007bd0 tx=0x7fc1a000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e10374344f79ebf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:39.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.576+0000 7fc1977fe640 1 --2- 172.21.15.114:0/2575691044 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc1900a13d0 0x7fc1900a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:39.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.576+0000 7fc1977fe640 1 -- 172.21.15.114:0/2575691044 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fc1980ee130 con 0x7fc1a8107540 2024-09-13T18:30:39.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.577+0000 7fc194ff9640 1 -- 172.21.15.114:0/917532571 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a000f650 con 0x55b148310cd0 2024-09-13T18:30:39.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.577+0000 7fc1a659c640 1 --2- 172.21.15.114:0/917532571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1483115f0 0x55b14832caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:30:39.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.577+0000 7fc194ff9640 1 -- 172.21.15.114:0/917532571 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc1a000fc40 con 0x55b148310cd0 2024-09-13T18:30:39.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.578+0000 7fc1afa63740 1 -- 172.21.15.114:0/917532571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b148310cd0 msgr2=0x55b1483110b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:39.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.578+0000 7fc1afa63740 1 --2- 172.21.15.114:0/917532571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b148310cd0 0x55b1483110b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0007bd0 tx=0x7fc1a000c6a0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.578+0000 7fc1a659c640 1 --2- 172.21.15.114:0/2575691044 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc1900a13d0 0x7fc1900a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:39.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.582+0000 7fc1afa63740 1 -- 172.21.15.114:0/917532571 shutdown_connections 2024-09-13T18:30:39.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.582+0000 7fc1afa63740 1 --2- 172.21.15.114:0/917532571 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b14832cfe0 0x55b14832f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.582+0000 7fc1afa63740 1 --2- 172.21.15.114:0/917532571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1483115f0 0x55b14832caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.582+0000 7fc1afa63740 1 --2- 172.21.15.114:0/917532571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b148310cd0 0x55b1483110b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.582+0000 7fc1afa63740 1 -- 172.21.15.114:0/917532571 >> 172.21.15.114:0/917532571 conn(0x55b14830d380 msgr2=0x55b14830d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:39.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.584+0000 7fc1a659c640 1 --2- 172.21.15.114:0/2575691044 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc1900a13d0 0x7fc1900a3890 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc1980badd0 tx=0x7fc198005fd0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:39.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.583+0000 7fc1afa63740 1 -- 172.21.15.114:0/917532571 shutdown_connections 2024-09-13T18:30:39.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.591+0000 7fc1afa63740 1 -- 172.21.15.114:0/917532571 wait complete. 2024-09-13T18:30:39.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.591+0000 7fc1afa63740 1 -- start start 2024-09-13T18:30:39.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.591+0000 7fc1afa63740 1 Processor -- start 2024-09-13T18:30:39.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.594+0000 7fc1afa63740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b148310cd0 0x55b14832c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:39.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.594+0000 7fc1afa63740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1483115f0 0x55b148329bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:39.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.594+0000 7fc1afa63740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b14832cfe0 0x55b14832a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:39.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.594+0000 7fc1a6d9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b148310cd0 0x55b14832c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:39.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.594+0000 7fc1afa63740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1482162a0 con 0x55b148310cd0 2024-09-13T18:30:39.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.594+0000 7fc1afa63740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b147f4d430 con 0x55b1483115f0 2024-09-13T18:30:39.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.594+0000 7fc1afa63740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1481f8680 con 0x55b14832cfe0 2024-09-13T18:30:39.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.594+0000 7fc1a6d9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b148310cd0 0x55b14832c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:58156/0 (socket says 172.21.15.114:58156) 2024-09-13T18:30:39.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.594+0000 7fc1a6d9d640 1 -- 172.21.15.114:0/1338384091 learned_addr learned my addr 172.21.15.114:0/1338384091 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:39.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.594+0000 7fc1a6d9d640 1 --2- 172.21.15.114:0/1338384091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b14832cfe0 0x55b14832a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:39.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.594+0000 7fc1afa63740 10 client.0 _ll_register_callbacks cb 0x55b14826e610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:30:39.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.595+0000 7fc1a6d9d640 1 -- 172.21.15.114:0/1338384091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b14832cfe0 msgr2=0x55b14832a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:39.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.595+0000 7fc1a6d9d640 1 --2- 172.21.15.114:0/1338384091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b14832cfe0 0x55b14832a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.595+0000 7fc1a6d9d640 1 -- 172.21.15.114:0/1338384091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1483115f0 msgr2=0x55b148329bc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:30:39.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.595+0000 7fc1a6d9d640 1 --2- 172.21.15.114:0/1338384091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1483115f0 0x55b148329bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.595+0000 7fc1a6d9d640 1 -- 172.21.15.114:0/1338384091 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1a0007800 con 0x55b148310cd0 2024-09-13T18:30:39.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.595+0000 7fc1a6d9d640 1 --2- 172.21.15.114:0/1338384091 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b148310cd0 0x55b14832c990 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fc1a000d010 tx=0x7fc1a000e700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eed93cd63503976 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:39.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.595+0000 7fc18dffb640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:30:39.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.597+0000 7fc18dffb640 1 -- 172.21.15.114:0/1338384091 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a000f650 con 0x55b148310cd0 2024-09-13T18:30:39.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.597+0000 7fc18dffb640 1 -- 172.21.15.114:0/1338384091 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fc1a000f7b0 con 0x55b148310cd0 2024-09-13T18:30:39.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.597+0000 7fc1afa63740 1 -- 172.21.15.114:0/1338384091 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b147f684e0 con 0x55b148310cd0 2024-09-13T18:30:39.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.598+0000 7fc18dffb640 1 -- 172.21.15.114:0/1338384091 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1a000f970 con 0x55b148310cd0 2024-09-13T18:30:39.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.598+0000 7fc1afa63740 10 client.17943 fetch_fsmap learned FSMap version 34 2024-09-13T18:30:39.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.598+0000 7fc1afa63740 1 -- 172.21.15.114:0/1338384091 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55b148273620 con 0x55b148310cd0 2024-09-13T18:30:39.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.602+0000 7fc18dffb640 1 -- 172.21.15.114:0/1338384091 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b147f684e0 con 0x55b148310cd0 2024-09-13T18:30:39.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.603+0000 7fc18dffb640 1 -- 172.21.15.114:0/1338384091 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc1a0019450 con 0x55b148310cd0 2024-09-13T18:30:39.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.603+0000 7fc1afa63740 10 client.17943 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:30:39.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.603+0000 7fc1afa63740 10 client.17943 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:30:39.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.603+0000 7fc1afa63740 20 client.17943 populate_metadata read hostname 'smithi114' 2024-09-13T18:30:39.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.603+0000 7fc1afa63740 1 --2- 172.21.15.114:0/1338384091 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55b14833cdc0 0x55b14833f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:39.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.603+0000 7fc1afa63740 4 client.17943 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:30:39.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.603+0000 7fc1afa63740 1 -- 172.21.15.114:0/1338384091 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b147f797e0 con 0x55b14833cdc0 2024-09-13T18:30:39.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.604+0000 7fc1a5d9b640 1 --2- 172.21.15.114:0/1338384091 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55b14833cdc0 0x55b14833f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:39.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.616+0000 7fc1a5d9b640 1 --2- 172.21.15.114:0/1338384091 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55b14833cdc0 0x55b14833f1e0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e3c109eb9deee19 server_cookie=252622e140dfe171 in_seq=0 out_seq=0 2024-09-13T18:30:39.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.619+0000 7fc18dffb640 10 client.17943 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:30:39.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.619+0000 7fc18dffb640 1 -- 172.21.15.114:0/1338384091 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55b147f797e0 con 0x55b14833cdc0 2024-09-13T18:30:39.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.619+0000 7fc18dffb640 10 client.17943 handle_command_reply: tid=0 2024-09-13T18:30:39.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.619+0000 7fc1afa63740 1 client.17943 shutdown 2024-09-13T18:30:39.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.622+0000 7fc1afa63740 1 -- 172.21.15.114:0/1338384091 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b148310cd0 msgr2=0x55b14832c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:39.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.622+0000 7fc1afa63740 1 --2- 172.21.15.114:0/1338384091 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b148310cd0 0x55b14832c990 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fc1a000d010 tx=0x7fc1a000e700 comp rx=0 tx=0).stop 2024-09-13T18:30:39.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.625+0000 7fc1afa63740 1 -- 172.21.15.114:0/1338384091 shutdown_connections 2024-09-13T18:30:39.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.625+0000 7fc1afa63740 1 --2- 172.21.15.114:0/1338384091 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55b14833cdc0 0x55b14833f1e0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.625+0000 7fc1afa63740 1 --2- 172.21.15.114:0/1338384091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b14832cfe0 0x55b14832a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.625+0000 7fc1afa63740 1 --2- 172.21.15.114:0/1338384091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1483115f0 0x55b148329bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.625+0000 7fc1afa63740 1 --2- 172.21.15.114:0/1338384091 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b148310cd0 0x55b14832c990 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.625+0000 7fc1afa63740 1 -- 172.21.15.114:0/1338384091 >> 172.21.15.114:0/1338384091 conn(0x55b14830d380 msgr2=0x55b14830c510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:39.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.626+0000 7fc18dffb640 0 client.17943 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:30:39.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.629+0000 7fc1afa63740 1 -- 172.21.15.114:0/1338384091 shutdown_connections 2024-09-13T18:30:39.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.631+0000 7fc1afa63740 1 -- 172.21.15.114:0/1338384091 wait complete. 2024-09-13T18:30:39.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.637+0000 7fc1afa63740 20 client.17943 trim_cache size 0 max 16384 2024-09-13T18:30:39.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.671+0000 7fc1afa63740 1 Processor -- start 2024-09-13T18:30:39.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.672+0000 7fc1afa63740 1 -- start start 2024-09-13T18:30:39.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.673+0000 7fc1afa63740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b148310cd0 0x55b14832c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:39.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.673+0000 7fc1afa63740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b1483115f0 0x55b148329460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:39.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.673+0000 7fc1afa63740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b14832cfe0 0x55b1483299a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:39.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.673+0000 7fc1afa63740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b14830e2e0 con 0x55b1483115f0 2024-09-13T18:30:39.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.673+0000 7fc1afa63740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b148307710 con 0x55b148310cd0 2024-09-13T18:30:39.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.673+0000 7fc1afa63740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b148312050 con 0x55b14832cfe0 2024-09-13T18:30:39.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.674+0000 7fc1a5d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b1483115f0 0x55b148329460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:39.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.674+0000 7fc1a5d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b1483115f0 0x55b148329460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:58166/0 (socket says 172.21.15.114:58166) 2024-09-13T18:30:39.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.674+0000 7fc1a5d9b640 1 -- 172.21.15.114:0/3354077127 learned_addr learned my addr 172.21.15.114:0/3354077127 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:39.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.675+0000 7fc1a6d9d640 1 --2- 172.21.15.114:0/3354077127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b148310cd0 0x55b14832c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:39.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.675+0000 7fc1a6d9d640 1 --2- 172.21.15.114:0/3354077127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b14832cfe0 0x55b1483299a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:39.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.675+0000 7fc1a6d9d640 1 -- 172.21.15.114:0/3354077127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b14832cfe0 msgr2=0x55b1483299a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:39.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.675+0000 7fc1a6d9d640 1 --2- 172.21.15.114:0/3354077127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b14832cfe0 0x55b1483299a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.675+0000 7fc1a6d9d640 1 -- 172.21.15.114:0/3354077127 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b1483115f0 msgr2=0x55b148329460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:39.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.675+0000 7fc1a6d9d640 1 --2- 172.21.15.114:0/3354077127 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b1483115f0 0x55b148329460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.675+0000 7fc1a6d9d640 1 -- 172.21.15.114:0/3354077127 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b148215790 con 0x55b148310cd0 2024-09-13T18:30:39.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.679+0000 7fc1a6d9d640 1 --2- 172.21.15.114:0/3354077127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b148310cd0 0x55b14832c4a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc1a000cba0 tx=0x7fc1a000e4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b66cd8b16e5c401f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:39.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.681+0000 7fc1957fa640 1 -- 172.21.15.114:0/3354077127 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0019a80 con 0x55b148310cd0 2024-09-13T18:30:39.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.681+0000 7fc1957fa640 1 -- 172.21.15.114:0/3354077127 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc1a001d070 con 0x55b148310cd0 2024-09-13T18:30:39.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.682+0000 7fc1afa63740 1 -- 172.21.15.114:0/3354077127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b148310cd0 msgr2=0x55b14832c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:39.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.682+0000 7fc1afa63740 1 --2- 172.21.15.114:0/3354077127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b148310cd0 0x55b14832c4a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc1a000cba0 tx=0x7fc1a000e4c0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.687+0000 7fc1afa63740 1 -- 172.21.15.114:0/3354077127 shutdown_connections 2024-09-13T18:30:39.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.687+0000 7fc1afa63740 1 --2- 172.21.15.114:0/3354077127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b14832cfe0 0x55b1483299a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.687+0000 7fc1afa63740 1 --2- 172.21.15.114:0/3354077127 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b1483115f0 0x55b148329460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.687+0000 7fc1afa63740 1 --2- 172.21.15.114:0/3354077127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b148310cd0 0x55b14832c4a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.687+0000 7fc1afa63740 1 -- 172.21.15.114:0/3354077127 >> 172.21.15.114:0/3354077127 conn(0x55b14830d380 msgr2=0x55b148328470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:39.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.687+0000 7fc1afa63740 1 -- 172.21.15.114:0/3354077127 shutdown_connections 2024-09-13T18:30:39.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.691+0000 7fc1afa63740 1 -- 172.21.15.114:0/3354077127 wait complete. 2024-09-13T18:30:39.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.692+0000 7fc1afa63740 1 -- start start 2024-09-13T18:30:39.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.692+0000 7fc1afa63740 1 Processor -- start 2024-09-13T18:30:39.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.693+0000 7fc1afa63740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b148310cd0 0x55b148333df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:39.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.693+0000 7fc1afa63740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1483115f0 0x55b148334330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:39.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.693+0000 7fc1afa63740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b14832cfe0 0x55b148325e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:39.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.693+0000 7fc1afa63740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b148326540 con 0x55b148310cd0 2024-09-13T18:30:39.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.693+0000 7fc1afa63740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1483266b0 con 0x55b1483115f0 2024-09-13T18:30:39.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.693+0000 7fc1afa63740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b148326870 con 0x55b14832cfe0 2024-09-13T18:30:39.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.693+0000 7fc1afa63740 10 client.0 _ll_register_callbacks cb 0x7fc1a0007a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:30:39.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.695+0000 7fc1a6d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b14832cfe0 0x55b148325e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:39.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.695+0000 7fc1a6d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b14832cfe0 0x55b148325e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:59066/0 (socket says 172.21.15.114:59066) 2024-09-13T18:30:39.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.695+0000 7fc1a6d9d640 1 -- 172.21.15.114:0/4133023685 learned_addr learned my addr 172.21.15.114:0/4133023685 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:39.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.696+0000 7fc1a5d9b640 1 --2- 172.21.15.114:0/4133023685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1483115f0 0x55b148334330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:39.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.696+0000 7fc1a6d9d640 1 -- 172.21.15.114:0/4133023685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1483115f0 msgr2=0x55b148334330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:39.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.696+0000 7fc1a6d9d640 1 --2- 172.21.15.114:0/4133023685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1483115f0 0x55b148334330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.696+0000 7fc1a6d9d640 1 -- 172.21.15.114:0/4133023685 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b148310cd0 msgr2=0x55b148333df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:39.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.696+0000 7fc1a6d9d640 1 --2- 172.21.15.114:0/4133023685 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b148310cd0 0x55b148333df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.696+0000 7fc1a6d9d640 1 -- 172.21.15.114:0/4133023685 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b148273620 con 0x55b14832cfe0 2024-09-13T18:30:39.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.696+0000 7fc1a5d9b640 1 --2- 172.21.15.114:0/4133023685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1483115f0 0x55b148334330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:30:39.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.696+0000 7fc1a6d9d640 1 --2- 172.21.15.114:0/4133023685 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b14832cfe0 0x55b148325e70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc1a000e450 tx=0x7fc1a0004d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62d074263bb2b296 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:39.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.699+0000 7fc18dffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:30:39.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.699+0000 7fc18dffb640 1 -- 172.21.15.114:0/4133023685 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0019450 con 0x55b14832cfe0 2024-09-13T18:30:39.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.699+0000 7fc1afa63740 1 -- 172.21.15.114:0/4133023685 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc1a000fc40 con 0x55b14832cfe0 2024-09-13T18:30:39.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.699+0000 7fc18dffb640 1 -- 172.21.15.114:0/4133023685 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fc1a003c070 con 0x55b14832cfe0 2024-09-13T18:30:39.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.708+0000 7fc18dffb640 1 -- 172.21.15.114:0/4133023685 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1a000e600 con 0x55b14832cfe0 2024-09-13T18:30:39.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.714+0000 7fc1afa63740 10 client.27358 fetch_fsmap learned FSMap version 34 2024-09-13T18:30:39.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.714+0000 7fc1afa63740 1 -- 172.21.15.114:0/4133023685 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55b14823aea0 con 0x55b14832cfe0 2024-09-13T18:30:39.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:39 smithi114 ceph-mon[30897]: pgmap v252: 129 pgs: 129 active+clean; 23 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 28 MiB/s wr, 603 op/s 2024-09-13T18:30:39.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:39 smithi114 ceph-mon[30897]: scrub queued for path: / 2024-09-13T18:30:39.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:39 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:30:39.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:39 smithi114 ceph-mon[30897]: scrub summary: active paths [/] 2024-09-13T18:30:39.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:39 smithi114 ceph-mon[30897]: scrub summary: active paths [/] 2024-09-13T18:30:39.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:39 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:30:39.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:39 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:30:39.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.719+0000 7fc18dffb640 1 -- 172.21.15.114:0/4133023685 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc1a000fc40 con 0x55b14832cfe0 2024-09-13T18:30:39.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.719+0000 7fc18dffb640 1 -- 172.21.15.114:0/4133023685 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc1a0029690 con 0x55b14832cfe0 2024-09-13T18:30:39.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.724+0000 7fc1afa63740 10 client.27358 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:30:39.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.724+0000 7fc1afa63740 10 client.27358 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:30:39.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.724+0000 7fc1afa63740 20 client.27358 populate_metadata read hostname 'smithi114' 2024-09-13T18:30:39.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.724+0000 7fc1afa63740 1 --2- 172.21.15.114:0/4133023685 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55b148393ec0 0x55b1483962b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:39.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.724+0000 7fc1afa63740 4 client.27358 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T18:30:39.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.724+0000 7fc1afa63740 1 -- 172.21.15.114:0/4133023685 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b147fcce30 con 0x55b148393ec0 2024-09-13T18:30:39.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.728+0000 7fc1a5d9b640 1 --2- 172.21.15.114:0/4133023685 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55b148393ec0 0x55b1483962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:39.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.729+0000 7fc1a5d9b640 1 --2- 172.21.15.114:0/4133023685 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55b148393ec0 0x55b1483962b0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad7bf1fd12ab2946 server_cookie=e86352647fe2ff0f in_seq=0 out_seq=0 2024-09-13T18:30:39.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.730+0000 7fc18dffb640 10 client.27358 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:30:39.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.730+0000 7fc18dffb640 1 -- 172.21.15.114:0/4133023685 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55b147fcce30 con 0x55b148393ec0 2024-09-13T18:30:39.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.730+0000 7fc18dffb640 10 client.27358 handle_command_reply: tid=0 2024-09-13T18:30:39.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.730+0000 7fc1afa63740 1 client.27358 shutdown 2024-09-13T18:30:39.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.736+0000 7fc1afa63740 1 -- 172.21.15.114:0/4133023685 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b14832cfe0 msgr2=0x55b148325e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:39.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.736+0000 7fc1afa63740 1 --2- 172.21.15.114:0/4133023685 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b14832cfe0 0x55b148325e70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc1a000e450 tx=0x7fc1a0004d00 comp rx=0 tx=0).stop 2024-09-13T18:30:39.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.753+0000 7fc1afa63740 1 -- 172.21.15.114:0/4133023685 shutdown_connections 2024-09-13T18:30:39.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.753+0000 7fc1afa63740 1 --2- 172.21.15.114:0/4133023685 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55b148393ec0 0x55b1483962b0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.753+0000 7fc1afa63740 1 --2- 172.21.15.114:0/4133023685 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b14832cfe0 0x55b148325e70 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.753+0000 7fc1afa63740 1 --2- 172.21.15.114:0/4133023685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1483115f0 0x55b148334330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.753+0000 7fc1afa63740 1 --2- 172.21.15.114:0/4133023685 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b148310cd0 0x55b148333df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.753+0000 7fc1afa63740 1 -- 172.21.15.114:0/4133023685 >> 172.21.15.114:0/4133023685 conn(0x55b14830d380 msgr2=0x55b148329460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:39.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.753+0000 7fc18dffb640 0 client.27358 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:30:39.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.753+0000 7fc1afa63740 1 -- 172.21.15.114:0/4133023685 shutdown_connections 2024-09-13T18:30:39.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.760+0000 7fc1afa63740 1 -- 172.21.15.114:0/4133023685 wait complete. 2024-09-13T18:30:39.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.764+0000 7fc1afa63740 20 client.27358 trim_cache size 0 max 16384 2024-09-13T18:30:39.771 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:30:39.771 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-13T18:30:39.771 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": { 2024-09-13T18:30:39.771 INFO:teuthology.orchestra.run.smithi114.stdout: "3e348cce-b1ad-4b6f-8e82-203f96a72986": { 2024-09-13T18:30:39.771 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-13T18:30:39.771 INFO:teuthology.orchestra.run.smithi114.stdout: "tag": "3e348cce-b1ad-4b6f-8e82-203f96a72986", 2024-09-13T18:30:39.772 INFO:teuthology.orchestra.run.smithi114.stdout: "options": "recursive,force" 2024-09-13T18:30:39.772 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:30:39.772 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:30:39.772 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:30:39.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.775+0000 7fc1acd45640 1 -- 172.21.15.114:0/2575691044 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc1900a13d0 msgr2=0x7fc1900a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:39.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.775+0000 7fc1acd45640 1 --2- 172.21.15.114:0/2575691044 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc1900a13d0 0x7fc1900a3890 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc1980badd0 tx=0x7fc198005fd0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.775+0000 7fc1acd45640 1 -- 172.21.15.114:0/2575691044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1a8107540 msgr2=0x7fc1a81a50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:39.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.775+0000 7fc1acd45640 1 --2- 172.21.15.114:0/2575691044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1a8107540 0x7fc1a81a50b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc198005940 tx=0x7fc198032f90 comp rx=0 tx=0).stop 2024-09-13T18:30:39.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.781+0000 7fc1acd45640 1 -- 172.21.15.114:0/2575691044 shutdown_connections 2024-09-13T18:30:39.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.781+0000 7fc1acd45640 1 --2- 172.21.15.114:0/2575691044 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc1900a13d0 0x7fc1900a3890 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.781+0000 7fc1acd45640 1 --2- 172.21.15.114:0/2575691044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1a8119400 0x7fc1a81a99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.781+0000 7fc1acd45640 1 --2- 172.21.15.114:0/2575691044 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc1a8107920 0x7fc1a81a55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.781+0000 7fc1acd45640 1 --2- 172.21.15.114:0/2575691044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1a8107540 0x7fc1a81a50b0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:39.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.781+0000 7fc1acd45640 1 -- 172.21.15.114:0/2575691044 >> 172.21.15.114:0/2575691044 conn(0x7fc1a806afe0 msgr2=0x7fc1a8119ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:39.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.785+0000 7fc1acd45640 1 -- 172.21.15.114:0/2575691044 shutdown_connections 2024-09-13T18:30:39.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:30:39.788+0000 7fc1acd45640 1 -- 172.21.15.114:0/2575691044 wait complete. 2024-09-13T18:30:39.815 INFO:tasks.cephfs.filesystem:scrub status for tag:3e348cce-b1ad-4b6f-8e82-203f96a72986 - {'path': '/', 'tag': '3e348cce-b1ad-4b6f-8e82-203f96a72986', 'options': 'recursive,force'} 2024-09-13T18:30:39.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:39 smithi134 ceph-mon[34283]: pgmap v252: 129 pgs: 129 active+clean; 23 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 28 MiB/s wr, 603 op/s 2024-09-13T18:30:39.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:39 smithi134 ceph-mon[34283]: scrub queued for path: / 2024-09-13T18:30:39.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:39 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:30:39.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:39 smithi134 ceph-mon[34283]: scrub summary: active paths [/] 2024-09-13T18:30:39.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:39 smithi134 ceph-mon[34283]: scrub summary: active paths [/] 2024-09-13T18:30:39.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:39 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:30:39.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:39 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:30:39.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:39 smithi204 ceph-mon[34777]: pgmap v252: 129 pgs: 129 active+clean; 23 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 28 MiB/s wr, 603 op/s 2024-09-13T18:30:39.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:39 smithi204 ceph-mon[34777]: scrub queued for path: / 2024-09-13T18:30:39.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:39 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:30:39.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:39 smithi204 ceph-mon[34777]: scrub summary: active paths [/] 2024-09-13T18:30:39.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:39 smithi204 ceph-mon[34777]: scrub summary: active paths [/] 2024-09-13T18:30:39.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:39 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:30:39.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:39 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:30:40.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:40 smithi134 ceph-mon[34283]: scrub summary: active paths [/] 2024-09-13T18:30:40.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:40 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:30:40.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:40 smithi204 ceph-mon[34777]: scrub summary: active paths [/] 2024-09-13T18:30:40.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:40 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:30:40.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:40 smithi114 ceph-mon[30897]: scrub summary: active paths [/] 2024-09-13T18:30:40.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:40 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:30:41.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:41 smithi134 ceph-mon[34283]: pgmap v253: 129 pgs: 129 active+clean; 23 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 42 MiB/s wr, 895 op/s 2024-09-13T18:30:41.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:41 smithi204 ceph-mon[34777]: pgmap v253: 129 pgs: 129 active+clean; 23 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 42 MiB/s wr, 895 op/s 2024-09-13T18:30:41.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:41 smithi114 ceph-mon[30897]: pgmap v253: 129 pgs: 129 active+clean; 23 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 42 MiB/s wr, 895 op/s 2024-09-13T18:30:43.747 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:43 smithi204 ceph-mon[34777]: pgmap v254: 129 pgs: 129 active+clean; 23 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 28 MiB/s wr, 588 op/s 2024-09-13T18:30:43.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:43 smithi134 ceph-mon[34283]: pgmap v254: 129 pgs: 129 active+clean; 23 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 28 MiB/s wr, 588 op/s 2024-09-13T18:30:43.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:43 smithi114 ceph-mon[30897]: pgmap v254: 129 pgs: 129 active+clean; 23 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 28 MiB/s wr, 588 op/s 2024-09-13T18:30:44.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:44 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:30:44.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:44 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:30:44.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:44 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:30:45.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:45 smithi134 ceph-mon[34283]: pgmap v255: 129 pgs: 129 active+clean; 23 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 32 MiB/s wr, 689 op/s 2024-09-13T18:30:45.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:45 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:30:45.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:45 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:30:45.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:45 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:30:45.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:45 smithi204 ceph-mon[34777]: pgmap v255: 129 pgs: 129 active+clean; 23 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 32 MiB/s wr, 689 op/s 2024-09-13T18:30:45.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:45 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:30:45.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:45 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:30:45.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:45 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:30:45.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:45 smithi114 ceph-mon[30897]: pgmap v255: 129 pgs: 129 active+clean; 23 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 32 MiB/s wr, 689 op/s 2024-09-13T18:30:45.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:45 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:30:45.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:45 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:30:45.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:45 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:30:47.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:47 smithi134 ceph-mon[34283]: pgmap v256: 129 pgs: 129 active+clean; 23 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 37 MiB/s wr, 786 op/s 2024-09-13T18:30:47.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:47 smithi204 ceph-mon[34777]: pgmap v256: 129 pgs: 129 active+clean; 23 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 37 MiB/s wr, 786 op/s 2024-09-13T18:30:47.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:47 smithi114 ceph-mon[30897]: pgmap v256: 129 pgs: 129 active+clean; 23 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 37 MiB/s wr, 786 op/s 2024-09-13T18:30:49.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:49 smithi134 ceph-mon[34283]: pgmap v257: 129 pgs: 129 active+clean; 23 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 28 MiB/s wr, 590 op/s 2024-09-13T18:30:49.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:49 smithi204 ceph-mon[34777]: pgmap v257: 129 pgs: 129 active+clean; 23 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 28 MiB/s wr, 590 op/s 2024-09-13T18:30:49.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:49 smithi114 ceph-mon[30897]: pgmap v257: 129 pgs: 129 active+clean; 23 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 28 MiB/s wr, 590 op/s 2024-09-13T18:30:51.093 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.090+0000 7f06c2987640 1 -- 172.21.15.114:0/1162528806 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06bc107890 msgr2=0x7f06bc107cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:51.093 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.090+0000 7f06c2987640 1 --2- 172.21.15.114:0/1162528806 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06bc107890 0x7f06bc107cd0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f06b0009860 tx=0x7f06b002a380 comp rx=0 tx=0).stop 2024-09-13T18:30:51.097 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.094+0000 7f06c2987640 1 -- 172.21.15.114:0/1162528806 shutdown_connections 2024-09-13T18:30:51.097 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.094+0000 7f06c2987640 1 --2- 172.21.15.114:0/1162528806 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06bc119400 0x7f06bc11b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:51.097 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.094+0000 7f06c2987640 1 --2- 172.21.15.114:0/1162528806 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06bc107890 0x7f06bc107cd0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:51.097 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.094+0000 7f06c2987640 1 --2- 172.21.15.114:0/1162528806 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06bc1074b0 0x7f06bc10e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:51.097 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.094+0000 7f06c2987640 1 -- 172.21.15.114:0/1162528806 >> 172.21.15.114:0/1162528806 conn(0x7f06bc06adf0 msgr2=0x7f06bc06b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:51.104 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.101+0000 7f06c2987640 1 -- 172.21.15.114:0/1162528806 shutdown_connections 2024-09-13T18:30:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.103+0000 7f06c2987640 1 -- 172.21.15.114:0/1162528806 wait complete. 2024-09-13T18:30:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.104+0000 7f06c2987640 1 Processor -- start 2024-09-13T18:30:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.105+0000 7f06c2987640 1 -- start start 2024-09-13T18:30:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.105+0000 7f06c2987640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06bc1074b0 0x7f06bc1187a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.105+0000 7f06c2987640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06bc107890 0x7f06bc118ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.105+0000 7f06c2987640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06bc119400 0x7f06bc112810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:51.110 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.105+0000 7f06c2987640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06bc112fa0 con 0x7f06bc119400 2024-09-13T18:30:51.110 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.105+0000 7f06c2987640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06bc113110 con 0x7f06bc1074b0 2024-09-13T18:30:51.110 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.105+0000 7f06c2987640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06bc1132d0 con 0x7f06bc107890 2024-09-13T18:30:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.107+0000 7f06c2186640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06bc119400 0x7f06bc112810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.107+0000 7f06c2186640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06bc119400 0x7f06bc112810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40528/0 (socket says 172.21.15.114:40528) 2024-09-13T18:30:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.107+0000 7f06c2186640 1 -- 172.21.15.114:0/2832859800 learned_addr learned my addr 172.21.15.114:0/2832859800 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.107+0000 7f06c1985640 1 --2- 172.21.15.114:0/2832859800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06bc1074b0 0x7f06bc1187a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.107+0000 7f06c1184640 1 --2- 172.21.15.114:0/2832859800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06bc107890 0x7f06bc118ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.108+0000 7f06c1985640 1 -- 172.21.15.114:0/2832859800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06bc107890 msgr2=0x7f06bc118ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.108+0000 7f06c1985640 1 --2- 172.21.15.114:0/2832859800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06bc107890 0x7f06bc118ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.108+0000 7f06c1985640 1 -- 172.21.15.114:0/2832859800 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06bc119400 msgr2=0x7f06bc112810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.108+0000 7f06c1985640 1 --2- 172.21.15.114:0/2832859800 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06bc119400 0x7f06bc112810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.108+0000 7f06c1985640 1 -- 172.21.15.114:0/2832859800 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06b0009490 con 0x7f06bc1074b0 2024-09-13T18:30:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.109+0000 7f06c1985640 1 --2- 172.21.15.114:0/2832859800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06bc1074b0 0x7f06bc1187a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f06ac00eb00 tx=0x7f06ac00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:51.116 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.113+0000 7f06baffd640 1 -- 172.21.15.114:0/2832859800 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06ac010430 con 0x7f06bc1074b0 2024-09-13T18:30:51.116 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.113+0000 7f06baffd640 1 -- 172.21.15.114:0/2832859800 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f06ac010a20 con 0x7f06bc1074b0 2024-09-13T18:30:51.119 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.113+0000 7f06baffd640 1 -- 172.21.15.114:0/2832859800 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06ac0179c0 con 0x7f06bc1074b0 2024-09-13T18:30:51.119 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.113+0000 7f06c2987640 1 -- 172.21.15.114:0/2832859800 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06bc1135b0 con 0x7f06bc1074b0 2024-09-13T18:30:51.119 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.113+0000 7f06c2987640 1 -- 172.21.15.114:0/2832859800 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06bc1ba280 con 0x7f06bc1074b0 2024-09-13T18:30:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.118+0000 7f06c69e4740 1 Processor -- start 2024-09-13T18:30:51.122 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.120+0000 7f06baffd640 1 -- 172.21.15.114:0/2832859800 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f06ac017b20 con 0x7f06bc1074b0 2024-09-13T18:30:51.124 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.120+0000 7f06baffd640 1 --2- 172.21.15.114:0/2832859800 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f06a40a1410 0x7f06a40a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:51.124 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.121+0000 7f06baffd640 1 -- 172.21.15.114:0/2832859800 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f06ac014070 con 0x7f06bc1074b0 2024-09-13T18:30:51.128 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.124+0000 7f06c69e4740 1 -- start start 2024-09-13T18:30:51.128 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.125+0000 7f06c69e4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56448bd09560 0x56448bd09940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:51.128 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.125+0000 7f06c69e4740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56448bd09e80 0x56448bd141e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:51.128 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.125+0000 7f06c69e4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56448bd14720 0x56448bd16b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:51.128 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.125+0000 7f06c69e4740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56448bc74c30 con 0x56448bd09e80 2024-09-13T18:30:51.128 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.125+0000 7f06c69e4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56448bc723c0 con 0x56448bd09560 2024-09-13T18:30:51.128 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.125+0000 7f06c69e4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56448bc6dcb0 con 0x56448bd14720 2024-09-13T18:30:51.130 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.126+0000 7f06c1184640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56448bd09e80 0x56448bd141e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:51.130 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.126+0000 7f06c1184640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56448bd09e80 0x56448bd141e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40532/0 (socket says 172.21.15.114:40532) 2024-09-13T18:30:51.130 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.126+0000 7f06c1184640 1 -- 172.21.15.114:0/3640275143 learned_addr learned my addr 172.21.15.114:0/3640275143 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:51.130 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.126+0000 7f06c1184640 1 -- 172.21.15.114:0/3640275143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56448bd14720 msgr2=0x56448bd16b10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:30:51.130 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.126+0000 7f06c1184640 1 --2- 172.21.15.114:0/3640275143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56448bd14720 0x56448bd16b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:51.130 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.126+0000 7f06c1184640 1 -- 172.21.15.114:0/3640275143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56448bd09560 msgr2=0x56448bd09940 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:30:51.130 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.126+0000 7f06c1184640 1 --2- 172.21.15.114:0/3640275143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56448bd09560 0x56448bd09940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:51.131 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.126+0000 7f06c1184640 1 -- 172.21.15.114:0/3640275143 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56448bc14910 con 0x56448bd09e80 2024-09-13T18:30:51.131 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.126+0000 7f06c1184640 1 --2- 172.21.15.114:0/3640275143 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56448bd09e80 0x56448bd141e0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f06b002a8f0 tx=0x7f06b00324d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a843260efffe88b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:51.131 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.127+0000 7f06a3fff640 1 -- 172.21.15.114:0/3640275143 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b0032840 con 0x56448bd09e80 2024-09-13T18:30:51.131 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.127+0000 7f06c2186640 1 --2- 172.21.15.114:0/2832859800 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f06a40a1410 0x7f06a40a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:51.131 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.127+0000 7f06a3fff640 1 -- 172.21.15.114:0/3640275143 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f06b00044c0 con 0x56448bd09e80 2024-09-13T18:30:51.131 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.127+0000 7f06a3fff640 1 -- 172.21.15.114:0/3640275143 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b00055b0 con 0x56448bd09e80 2024-09-13T18:30:51.131 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.128+0000 7f06c69e4740 1 -- 172.21.15.114:0/3640275143 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56448bd09e80 msgr2=0x56448bd141e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:51.131 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.128+0000 7f06c69e4740 1 --2- 172.21.15.114:0/3640275143 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56448bd09e80 0x56448bd141e0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f06b002a8f0 tx=0x7f06b00324d0 comp rx=0 tx=0).stop 2024-09-13T18:30:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.132+0000 7f06c69e4740 1 -- 172.21.15.114:0/3640275143 shutdown_connections 2024-09-13T18:30:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.132+0000 7f06c69e4740 1 --2- 172.21.15.114:0/3640275143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56448bd14720 0x56448bd16b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.132+0000 7f06c69e4740 1 --2- 172.21.15.114:0/3640275143 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56448bd09e80 0x56448bd141e0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.132+0000 7f06c69e4740 1 --2- 172.21.15.114:0/3640275143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56448bd09560 0x56448bd09940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.132+0000 7f06c69e4740 1 -- 172.21.15.114:0/3640275143 >> 172.21.15.114:0/3640275143 conn(0x56448bd05af0 msgr2=0x56448bd05ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.132+0000 7f06c2186640 1 --2- 172.21.15.114:0/2832859800 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f06a40a1410 0x7f06a40a38d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x56448bd06e70 tx=0x7f06b4009260 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:51.140 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.136+0000 7f06c69e4740 1 -- 172.21.15.114:0/3640275143 shutdown_connections 2024-09-13T18:30:51.143 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.139+0000 7f06c69e4740 1 -- 172.21.15.114:0/3640275143 wait complete. 2024-09-13T18:30:51.143 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.140+0000 7f06c69e4740 1 -- start start 2024-09-13T18:30:51.143 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.140+0000 7f06c69e4740 1 Processor -- start 2024-09-13T18:30:51.148 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.144+0000 7f06c69e4740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56448bd09560 0x56448bd13f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:51.148 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.144+0000 7f06c69e4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56448bd09e80 0x56448bd11250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:51.148 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.144+0000 7f06c69e4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56448bd14720 0x56448bd11790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:51.148 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.144+0000 7f06c69e4740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56448bc2be80 con 0x56448bd09560 2024-09-13T18:30:51.148 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.144+0000 7f06c69e4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56448bc2bb20 con 0x56448bd14720 2024-09-13T18:30:51.148 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.144+0000 7f06c69e4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56448bbf1260 con 0x56448bd09e80 2024-09-13T18:30:51.148 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.144+0000 7f06c69e4740 10 client.0 _ll_register_callbacks cb 0x56448bc26df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:30:51.151 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.148+0000 7f06c1184640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56448bd09e80 0x56448bd11250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:51.151 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.148+0000 7f06c1184640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56448bd09e80 0x56448bd11250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:38938/0 (socket says 172.21.15.114:38938) 2024-09-13T18:30:51.151 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.148+0000 7f06c1184640 1 -- 172.21.15.114:0/2754500761 learned_addr learned my addr 172.21.15.114:0/2754500761 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:51.151 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.148+0000 7f06c1985640 1 --2- 172.21.15.114:0/2754500761 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56448bd09560 0x56448bd13f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.152+0000 7f06c2186640 1 --2- 172.21.15.114:0/2754500761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56448bd14720 0x56448bd11790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.152+0000 7f06c1184640 1 -- 172.21.15.114:0/2754500761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56448bd14720 msgr2=0x56448bd11790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.152+0000 7f06c1184640 1 --2- 172.21.15.114:0/2754500761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56448bd14720 0x56448bd11790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.152+0000 7f06c1184640 1 -- 172.21.15.114:0/2754500761 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56448bd09560 msgr2=0x56448bd13f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.152+0000 7f06c1184640 1 --2- 172.21.15.114:0/2754500761 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56448bd09560 0x56448bd13f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.152+0000 7f06c1184640 1 -- 172.21.15.114:0/2754500761 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06bc11c060 con 0x56448bd09e80 2024-09-13T18:30:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.152+0000 7f06c2186640 1 --2- 172.21.15.114:0/2754500761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56448bd14720 0x56448bd11790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:30:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.155+0000 7f06c1184640 1 --2- 172.21.15.114:0/2754500761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56448bd09e80 0x56448bd11250 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f06b0004650 tx=0x7f06b0005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1d7b7fe4b852eae server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.155+0000 7f06a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:30:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.158+0000 7f06a17fa640 1 -- 172.21.15.114:0/2754500761 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b0005500 con 0x56448bd09e80 2024-09-13T18:30:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.158+0000 7f06a17fa640 1 -- 172.21.15.114:0/2754500761 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f06b00046c0 con 0x56448bd09e80 2024-09-13T18:30:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.159+0000 7f06c69e4740 1 -- 172.21.15.114:0/2754500761 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56448bc96680 con 0x56448bd09e80 2024-09-13T18:30:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.164+0000 7f06a17fa640 1 -- 172.21.15.114:0/2754500761 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06b0005660 con 0x56448bd09e80 2024-09-13T18:30:51.171 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.166+0000 7f06c69e4740 10 client.27364 fetch_fsmap learned FSMap version 34 2024-09-13T18:30:51.171 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.166+0000 7f06c69e4740 1 -- 172.21.15.114:0/2754500761 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x56448bc6c330 con 0x56448bd09e80 2024-09-13T18:30:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.175+0000 7f06a17fa640 1 -- 172.21.15.114:0/2754500761 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56448bc96680 con 0x56448bd09e80 2024-09-13T18:30:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.175+0000 7f06a17fa640 1 -- 172.21.15.114:0/2754500761 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f06b0030930 con 0x56448bd09e80 2024-09-13T18:30:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.175+0000 7f06c69e4740 10 client.27364 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:30:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.175+0000 7f06c69e4740 10 client.27364 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:30:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.175+0000 7f06c69e4740 20 client.27364 populate_metadata read hostname 'smithi114' 2024-09-13T18:30:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.175+0000 7f06c69e4740 1 --2- 172.21.15.114:0/2754500761 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56448bd23cf0 0x56448bd26110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.175+0000 7f06c69e4740 4 client.27364 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:30:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.175+0000 7f06c69e4740 1 -- 172.21.15.114:0/2754500761 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56448bc615d0 con 0x56448bd23cf0 2024-09-13T18:30:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.182+0000 7f06c1985640 1 --2- 172.21.15.114:0/2754500761 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56448bd23cf0 0x56448bd26110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.198+0000 7f06c1985640 1 --2- 172.21.15.114:0/2754500761 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56448bd23cf0 0x56448bd26110 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2636b9a510222 server_cookie=d0ee0973b3a83caf in_seq=0 out_seq=0 2024-09-13T18:30:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.198+0000 7f06a17fa640 10 client.27364 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:30:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.202+0000 7f06a17fa640 1 -- 172.21.15.114:0/2754500761 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x56448bc615d0 con 0x56448bd23cf0 2024-09-13T18:30:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.202+0000 7f06a17fa640 10 client.27364 handle_command_reply: tid=0 2024-09-13T18:30:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.202+0000 7f06c69e4740 1 client.27364 shutdown 2024-09-13T18:30:51.218 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.211+0000 7f06c69e4740 1 -- 172.21.15.114:0/2754500761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56448bd09e80 msgr2=0x56448bd11250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:51.218 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.211+0000 7f06c69e4740 1 --2- 172.21.15.114:0/2754500761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56448bd09e80 0x56448bd11250 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f06b0004650 tx=0x7f06b0005fd0 comp rx=0 tx=0).stop 2024-09-13T18:30:51.218 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.212+0000 7f06c69e4740 1 -- 172.21.15.114:0/2754500761 shutdown_connections 2024-09-13T18:30:51.218 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.212+0000 7f06c69e4740 1 --2- 172.21.15.114:0/2754500761 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56448bd23cf0 0x56448bd26110 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:51.218 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.212+0000 7f06c69e4740 1 --2- 172.21.15.114:0/2754500761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56448bd14720 0x56448bd11790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:51.218 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.212+0000 7f06c69e4740 1 --2- 172.21.15.114:0/2754500761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56448bd09e80 0x56448bd11250 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:51.218 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.212+0000 7f06c69e4740 1 --2- 172.21.15.114:0/2754500761 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56448bd09560 0x56448bd13f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:51.218 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.212+0000 7f06c69e4740 1 -- 172.21.15.114:0/2754500761 >> 172.21.15.114:0/2754500761 conn(0x56448bd05af0 msgr2=0x56448bd04d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:51.227 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.219+0000 7f06a17fa640 0 client.27364 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:30:51.228 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.219+0000 7f06c69e4740 1 -- 172.21.15.114:0/2754500761 shutdown_connections 2024-09-13T18:30:51.228 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.220+0000 7f06c69e4740 1 -- 172.21.15.114:0/2754500761 wait complete. 2024-09-13T18:30:51.228 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.220+0000 7f06c69e4740 20 client.27364 trim_cache size 0 max 16384 2024-09-13T18:30:51.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.259+0000 7f06c69e4740 1 Processor -- start 2024-09-13T18:30:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.262+0000 7f06c69e4740 1 -- start start 2024-09-13T18:30:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.262+0000 7f06c69e4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56448bd09560 0x56448bd0c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.262+0000 7f06c69e4740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56448bd09e80 0x56448bd0ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.262+0000 7f06c69e4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56448bd14720 0x56448bd0fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.262+0000 7f06c69e4740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56448bc615d0 con 0x56448bd09e80 2024-09-13T18:30:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.262+0000 7f06c69e4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56448bc96680 con 0x56448bd09560 2024-09-13T18:30:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.262+0000 7f06c69e4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56448bd20d90 con 0x56448bd14720 2024-09-13T18:30:51.266 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.262+0000 7f06c1985640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56448bd09560 0x56448bd0c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:51.266 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.262+0000 7f06c1985640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56448bd09560 0x56448bd0c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:43720/0 (socket says 172.21.15.114:43720) 2024-09-13T18:30:51.266 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.262+0000 7f06c1985640 1 -- 172.21.15.114:0/957382561 learned_addr learned my addr 172.21.15.114:0/957382561 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:51.266 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.263+0000 7f06c2186640 1 --2- 172.21.15.114:0/957382561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56448bd14720 0x56448bd0fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:51.268 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.264+0000 7f06c2186640 1 -- 172.21.15.114:0/957382561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56448bd09560 msgr2=0x56448bd0c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:51.268 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.264+0000 7f06c2186640 1 --2- 172.21.15.114:0/957382561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56448bd09560 0x56448bd0c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:51.268 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.264+0000 7f06c2186640 1 -- 172.21.15.114:0/957382561 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56448bd09e80 msgr2=0x56448bd0ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:51.268 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.264+0000 7f06c2186640 1 --2- 172.21.15.114:0/957382561 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56448bd09e80 0x56448bd0ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:51.268 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.264+0000 7f06c2186640 1 -- 172.21.15.114:0/957382561 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56448bc0e300 con 0x56448bd14720 2024-09-13T18:30:51.268 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.265+0000 7f06c2186640 1 --2- 172.21.15.114:0/957382561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56448bd14720 0x56448bd0fb00 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x56448bd06be0 tx=0x7f06b4014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f73b0d28aed19896 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:51.270 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.267+0000 7f06b8e05640 1 -- 172.21.15.114:0/957382561 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b401dd90 con 0x56448bd14720 2024-09-13T18:30:51.270 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.267+0000 7f06b8e05640 1 -- 172.21.15.114:0/957382561 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f06b401e8b0 con 0x56448bd14720 2024-09-13T18:30:51.271 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.268+0000 7f06c69e4740 1 -- 172.21.15.114:0/957382561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56448bd14720 msgr2=0x56448bd0fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:51.271 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.268+0000 7f06c69e4740 1 --2- 172.21.15.114:0/957382561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56448bd14720 0x56448bd0fb00 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x56448bd06be0 tx=0x7f06b4014000 comp rx=0 tx=0).stop 2024-09-13T18:30:51.274 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.271+0000 7f06c69e4740 1 -- 172.21.15.114:0/957382561 shutdown_connections 2024-09-13T18:30:51.274 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.271+0000 7f06c69e4740 1 --2- 172.21.15.114:0/957382561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56448bd14720 0x56448bd0fb00 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:51.274 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.271+0000 7f06c69e4740 1 --2- 172.21.15.114:0/957382561 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56448bd09e80 0x56448bd0ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:51.274 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.271+0000 7f06c69e4740 1 --2- 172.21.15.114:0/957382561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56448bd09560 0x56448bd0c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:51.274 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.271+0000 7f06c69e4740 1 -- 172.21.15.114:0/957382561 >> 172.21.15.114:0/957382561 conn(0x56448bd05af0 msgr2=0x56448bd2aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.277+0000 7f06c69e4740 1 -- 172.21.15.114:0/957382561 shutdown_connections 2024-09-13T18:30:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.277+0000 7f06c69e4740 1 -- 172.21.15.114:0/957382561 wait complete. 2024-09-13T18:30:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.278+0000 7f06c69e4740 1 -- start start 2024-09-13T18:30:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.278+0000 7f06c69e4740 1 Processor -- start 2024-09-13T18:30:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.278+0000 7f06c69e4740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56448bd09560 0x56448bd196a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.278+0000 7f06c69e4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56448bd09e80 0x56448bd19be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.278+0000 7f06c69e4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56448bd14720 0x56448bd791b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.278+0000 7f06c69e4740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56448bd1a340 con 0x56448bd09560 2024-09-13T18:30:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.278+0000 7f06c69e4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56448bd796f0 con 0x56448bd09e80 2024-09-13T18:30:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.278+0000 7f06c69e4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56448bd798b0 con 0x56448bd14720 2024-09-13T18:30:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.278+0000 7f06c69e4740 10 client.0 _ll_register_callbacks cb 0x7f06b0035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:30:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.279+0000 7f06c2186640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56448bd14720 0x56448bd791b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.279+0000 7f06c2186640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56448bd14720 0x56448bd791b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:38944/0 (socket says 172.21.15.114:38944) 2024-09-13T18:30:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.279+0000 7f06c2186640 1 -- 172.21.15.114:0/2532041704 learned_addr learned my addr 172.21.15.114:0/2532041704 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:30:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.280+0000 7f06c1184640 1 --2- 172.21.15.114:0/2532041704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56448bd09e80 0x56448bd19be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:51.284 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.281+0000 7f06c1184640 1 -- 172.21.15.114:0/2532041704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56448bd14720 msgr2=0x56448bd791b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.281+0000 7f06c1184640 1 --2- 172.21.15.114:0/2532041704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56448bd14720 0x56448bd791b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.281+0000 7f06c1184640 1 -- 172.21.15.114:0/2532041704 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56448bd09560 msgr2=0x56448bd196a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:30:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.281+0000 7f06c1184640 1 --2- 172.21.15.114:0/2532041704 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56448bd09560 0x56448bd196a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.281+0000 7f06c1184640 1 -- 172.21.15.114:0/2532041704 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06b4017070 con 0x56448bd09e80 2024-09-13T18:30:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.282+0000 7f06c1184640 1 --2- 172.21.15.114:0/2532041704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56448bd09e80 0x56448bd19be0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x56448bd12820 tx=0x7f06b0009740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20399949ca179019 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:30:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.283+0000 7f06a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:30:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.285+0000 7f06a17fa640 1 -- 172.21.15.114:0/2532041704 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b003cd70 con 0x56448bd09e80 2024-09-13T18:30:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.285+0000 7f06c69e4740 1 -- 172.21.15.114:0/2532041704 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f06b00044c0 con 0x56448bd09e80 2024-09-13T18:30:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.286+0000 7f06a17fa640 1 -- 172.21.15.114:0/2532041704 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f06b0000e40 con 0x56448bd09e80 2024-09-13T18:30:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.287+0000 7f06a17fa640 1 -- 172.21.15.114:0/2532041704 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06b0030eb0 con 0x56448bd09e80 2024-09-13T18:30:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.288+0000 7f06c69e4740 10 client.27389 fetch_fsmap learned FSMap version 34 2024-09-13T18:30:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.288+0000 7f06c69e4740 1 -- 172.21.15.114:0/2532041704 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x56448bc338e0 con 0x56448bd09e80 2024-09-13T18:30:51.299 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.293+0000 7f06a17fa640 1 -- 172.21.15.114:0/2532041704 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f06b00044c0 con 0x56448bd09e80 2024-09-13T18:30:51.299 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.293+0000 7f06a17fa640 1 -- 172.21.15.114:0/2532041704 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f06b00046c0 con 0x56448bd09e80 2024-09-13T18:30:51.299 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.295+0000 7f06c69e4740 10 client.27389 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:30:51.299 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.295+0000 7f06c69e4740 10 client.27389 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:30:51.299 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.295+0000 7f06c69e4740 20 client.27389 populate_metadata read hostname 'smithi114' 2024-09-13T18:30:51.299 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.295+0000 7f06c69e4740 1 --2- 172.21.15.114:0/2532041704 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56448bd7ac50 0x56448bd7d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:30:51.299 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.296+0000 7f06c69e4740 4 client.27389 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T18:30:51.299 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.296+0000 7f06c69e4740 1 -- 172.21.15.114:0/2532041704 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56448b961300 con 0x56448bd7ac50 2024-09-13T18:30:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.296+0000 7f06c1985640 1 --2- 172.21.15.114:0/2532041704 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56448bd7ac50 0x56448bd7d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:30:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.297+0000 7f06a17fa640 10 client.27389 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:30:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.297+0000 7f06c1985640 1 --2- 172.21.15.114:0/2532041704 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56448bd7ac50 0x56448bd7d070 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d4d97bbcd3620 server_cookie=db24d2acb076de49 in_seq=0 out_seq=0 2024-09-13T18:30:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.301+0000 7f06a17fa640 1 -- 172.21.15.114:0/2532041704 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56448b961300 con 0x56448bd7ac50 2024-09-13T18:30:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.301+0000 7f06a17fa640 10 client.27389 handle_command_reply: tid=0 2024-09-13T18:30:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.302+0000 7f06c69e4740 1 client.27389 shutdown 2024-09-13T18:30:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.303+0000 7f06c69e4740 1 -- 172.21.15.114:0/2532041704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56448bd09e80 msgr2=0x56448bd19be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:51.307 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.303+0000 7f06c69e4740 1 --2- 172.21.15.114:0/2532041704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56448bd09e80 0x56448bd19be0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x56448bd12820 tx=0x7f06b0009740 comp rx=0 tx=0).stop 2024-09-13T18:30:51.316 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.312+0000 7f06c69e4740 1 -- 172.21.15.114:0/2532041704 shutdown_connections 2024-09-13T18:30:51.316 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.312+0000 7f06c69e4740 1 --2- 172.21.15.114:0/2532041704 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56448bd7ac50 0x56448bd7d070 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:51.316 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.312+0000 7f06c69e4740 1 --2- 172.21.15.114:0/2532041704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56448bd14720 0x56448bd791b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:51.316 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.312+0000 7f06c69e4740 1 --2- 172.21.15.114:0/2532041704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56448bd09e80 0x56448bd19be0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:51.316 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.312+0000 7f06c69e4740 1 --2- 172.21.15.114:0/2532041704 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56448bd09560 0x56448bd196a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:51.316 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.312+0000 7f06c69e4740 1 -- 172.21.15.114:0/2532041704 >> 172.21.15.114:0/2532041704 conn(0x56448bd05af0 msgr2=0x56448bd0c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:51.316 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.312+0000 7f06a17fa640 0 client.27389 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:30:51.320 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.317+0000 7f06c69e4740 1 -- 172.21.15.114:0/2532041704 shutdown_connections 2024-09-13T18:30:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.322+0000 7f06c69e4740 1 -- 172.21.15.114:0/2532041704 wait complete. 2024-09-13T18:30:51.328 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.325+0000 7f06c69e4740 20 client.27389 trim_cache size 0 max 16384 2024-09-13T18:30:51.330 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-13T18:30:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-13T18:30:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-13T18:30:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-13T18:30:51.332 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.328+0000 7f06c2987640 1 -- 172.21.15.114:0/2832859800 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f06a40a1410 msgr2=0x7f06a40a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:51.332 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.328+0000 7f06c2987640 1 --2- 172.21.15.114:0/2832859800 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f06a40a1410 0x7f06a40a38d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x56448bd06e70 tx=0x7f06b4009260 comp rx=0 tx=0).stop 2024-09-13T18:30:51.332 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.328+0000 7f06c2987640 1 -- 172.21.15.114:0/2832859800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06bc1074b0 msgr2=0x7f06bc1187a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:30:51.332 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.328+0000 7f06c2987640 1 --2- 172.21.15.114:0/2832859800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06bc1074b0 0x7f06bc1187a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f06ac00eb00 tx=0x7f06ac00c2d0 comp rx=0 tx=0).stop 2024-09-13T18:30:51.332 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.329+0000 7f06c2987640 1 -- 172.21.15.114:0/2832859800 shutdown_connections 2024-09-13T18:30:51.332 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.329+0000 7f06c2987640 1 --2- 172.21.15.114:0/2832859800 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f06a40a1410 0x7f06a40a38d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:51.332 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.329+0000 7f06c2987640 1 --2- 172.21.15.114:0/2832859800 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06bc119400 0x7f06bc112810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:51.333 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.329+0000 7f06c2987640 1 --2- 172.21.15.114:0/2832859800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06bc107890 0x7f06bc118ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:51.333 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.329+0000 7f06c2987640 1 --2- 172.21.15.114:0/2832859800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06bc1074b0 0x7f06bc1187a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:30:51.333 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.329+0000 7f06c2987640 1 -- 172.21.15.114:0/2832859800 >> 172.21.15.114:0/2832859800 conn(0x7f06bc06adf0 msgr2=0x7f06bc0700e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:30:51.335 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.332+0000 7f06c2987640 1 -- 172.21.15.114:0/2832859800 shutdown_connections 2024-09-13T18:30:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:30:51.335+0000 7f06c2987640 1 -- 172.21.15.114:0/2832859800 wait complete. 2024-09-13T18:30:51.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:51 smithi134 ceph-mon[34283]: pgmap v258: 129 pgs: 129 active+clean; 23 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 40 MiB/s wr, 845 op/s 2024-09-13T18:30:51.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:51 smithi134 ceph-mon[34283]: scrub summary: idle 2024-09-13T18:30:51.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:51 smithi134 ceph-mon[34283]: scrub summary: idle 2024-09-13T18:30:51.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:51 smithi204 ceph-mon[34777]: pgmap v258: 129 pgs: 129 active+clean; 23 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 40 MiB/s wr, 845 op/s 2024-09-13T18:30:51.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:51 smithi204 ceph-mon[34777]: scrub summary: idle 2024-09-13T18:30:51.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:51 smithi204 ceph-mon[34777]: scrub summary: idle 2024-09-13T18:30:51.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:51 smithi114 ceph-mon[30897]: pgmap v258: 129 pgs: 129 active+clean; 23 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 40 MiB/s wr, 845 op/s 2024-09-13T18:30:51.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:51 smithi114 ceph-mon[30897]: scrub summary: idle 2024-09-13T18:30:51.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:51 smithi114 ceph-mon[30897]: scrub summary: idle 2024-09-13T18:30:52.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:52 smithi134 ceph-mon[34283]: scrub summary: idle 2024-09-13T18:30:52.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:52 smithi204 ceph-mon[34777]: scrub summary: idle 2024-09-13T18:30:52.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:52 smithi114 ceph-mon[30897]: scrub summary: idle 2024-09-13T18:30:53.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:53 smithi134 ceph-mon[34283]: pgmap v259: 129 pgs: 129 active+clean; 23 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 28 MiB/s wr, 592 op/s 2024-09-13T18:30:53.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:53 smithi204 ceph-mon[34777]: pgmap v259: 129 pgs: 129 active+clean; 23 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 28 MiB/s wr, 592 op/s 2024-09-13T18:30:53.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:53 smithi114 ceph-mon[30897]: pgmap v259: 129 pgs: 129 active+clean; 23 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 28 MiB/s wr, 592 op/s 2024-09-13T18:30:55.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:55 smithi134 ceph-mon[34283]: pgmap v260: 129 pgs: 129 active+clean; 23 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 32 MiB/s wr, 668 op/s 2024-09-13T18:30:55.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:55 smithi204 ceph-mon[34777]: pgmap v260: 129 pgs: 129 active+clean; 23 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 32 MiB/s wr, 668 op/s 2024-09-13T18:30:55.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:55 smithi114 ceph-mon[30897]: pgmap v260: 129 pgs: 129 active+clean; 23 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 32 MiB/s wr, 668 op/s 2024-09-13T18:30:57.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:57 smithi134 ceph-mon[34283]: pgmap v261: 129 pgs: 129 active+clean; 23 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 37 MiB/s wr, 786 op/s 2024-09-13T18:30:57.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:57 smithi204 ceph-mon[34777]: pgmap v261: 129 pgs: 129 active+clean; 23 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 37 MiB/s wr, 786 op/s 2024-09-13T18:30:57.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:57 smithi114 ceph-mon[30897]: pgmap v261: 129 pgs: 129 active+clean; 23 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 37 MiB/s wr, 786 op/s 2024-09-13T18:30:59.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:30:59 smithi134 ceph-mon[34283]: pgmap v262: 129 pgs: 129 active+clean; 23 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 28 MiB/s wr, 589 op/s 2024-09-13T18:30:59.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:30:59 smithi204 ceph-mon[34777]: pgmap v262: 129 pgs: 129 active+clean; 23 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 28 MiB/s wr, 589 op/s 2024-09-13T18:30:59.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:30:59 smithi114 ceph-mon[30897]: pgmap v262: 129 pgs: 129 active+clean; 23 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 28 MiB/s wr, 589 op/s 2024-09-13T18:31:01.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:01 smithi134 ceph-mon[34283]: pgmap v263: 129 pgs: 129 active+clean; 23 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 40 MiB/s wr, 841 op/s 2024-09-13T18:31:01.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:01 smithi204 ceph-mon[34777]: pgmap v263: 129 pgs: 129 active+clean; 23 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 40 MiB/s wr, 841 op/s 2024-09-13T18:31:01.973 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:01 smithi114 ceph-mon[30897]: pgmap v263: 129 pgs: 129 active+clean; 23 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 40 MiB/s wr, 841 op/s 2024-09-13T18:31:03.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:03 smithi134 ceph-mon[34283]: pgmap v264: 129 pgs: 129 active+clean; 23 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 29 MiB/s wr, 623 op/s 2024-09-13T18:31:03.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:03 smithi204 ceph-mon[34777]: pgmap v264: 129 pgs: 129 active+clean; 23 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 29 MiB/s wr, 623 op/s 2024-09-13T18:31:03.972 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:03 smithi114 ceph-mon[30897]: pgmap v264: 129 pgs: 129 active+clean; 23 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 29 MiB/s wr, 623 op/s 2024-09-13T18:31:04.538 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.533+0000 7fccb759e640 1 -- 172.21.15.114:0/3083725866 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fccb8107890 msgr2=0x7fccb8107cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:04.538 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.533+0000 7fccb759e640 1 --2- 172.21.15.114:0/3083725866 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fccb8107890 0x7fccb8107cd0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fcca8009960 tx=0x7fcca802a4a0 comp rx=0 tx=0).stop 2024-09-13T18:31:04.538 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.534+0000 7fccb759e640 1 -- 172.21.15.114:0/3083725866 shutdown_connections 2024-09-13T18:31:04.538 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.534+0000 7fccb759e640 1 --2- 172.21.15.114:0/3083725866 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fccb8119400 0x7fccb811b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:04.539 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.534+0000 7fccb759e640 1 --2- 172.21.15.114:0/3083725866 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fccb8107890 0x7fccb8107cd0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:04.539 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.534+0000 7fccb759e640 1 --2- 172.21.15.114:0/3083725866 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccb81074b0 0x7fccb810e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:04.539 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.534+0000 7fccb759e640 1 -- 172.21.15.114:0/3083725866 >> 172.21.15.114:0/3083725866 conn(0x7fccb806adb0 msgr2=0x7fccb806b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:04.546 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.543+0000 7fccb759e640 1 -- 172.21.15.114:0/3083725866 shutdown_connections 2024-09-13T18:31:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.547+0000 7fccb759e640 1 -- 172.21.15.114:0/3083725866 wait complete. 2024-09-13T18:31:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.547+0000 7fccb759e640 1 Processor -- start 2024-09-13T18:31:04.554 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.551+0000 7fccb759e640 1 -- start start 2024-09-13T18:31:04.554 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.551+0000 7fccb759e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fccb81074b0 0x7fccb81168f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:04.554 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.551+0000 7fccb759e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccb8107890 0x7fccb8116e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:04.555 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.551+0000 7fccb759e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fccb8119400 0x7fccb810fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:04.555 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.551+0000 7fccb759e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb81102c0 con 0x7fccb8119400 2024-09-13T18:31:04.555 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.551+0000 7fccb759e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb8110430 con 0x7fccb81074b0 2024-09-13T18:31:04.555 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.551+0000 7fccb759e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb81105f0 con 0x7fccb8107890 2024-09-13T18:31:04.555 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.551+0000 7fccb5d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccb8107890 0x7fccb8116e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:04.556 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.551+0000 7fccb5d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccb8107890 0x7fccb8116e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:53192/0 (socket says 172.21.15.114:53192) 2024-09-13T18:31:04.556 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.551+0000 7fccb5d9b640 1 -- 172.21.15.114:0/2734330583 learned_addr learned my addr 172.21.15.114:0/2734330583 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:04.556 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.552+0000 7fccb6d9d640 1 --2- 172.21.15.114:0/2734330583 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fccb8119400 0x7fccb810fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:04.557 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.555+0000 7fccb5d9b640 1 -- 172.21.15.114:0/2734330583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fccb81074b0 msgr2=0x7fccb81168f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:04.557 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.555+0000 7fccb5d9b640 1 --2- 172.21.15.114:0/2734330583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fccb81074b0 0x7fccb81168f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:04.558 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.555+0000 7fccb5d9b640 1 -- 172.21.15.114:0/2734330583 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fccb8119400 msgr2=0x7fccb810fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:04.558 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.555+0000 7fccb5d9b640 1 --2- 172.21.15.114:0/2734330583 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fccb8119400 0x7fccb810fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:04.558 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.555+0000 7fccb5d9b640 1 -- 172.21.15.114:0/2734330583 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcca8009560 con 0x7fccb8107890 2024-09-13T18:31:04.559 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.556+0000 7fccb5d9b640 1 --2- 172.21.15.114:0/2734330583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccb8107890 0x7fccb8116e30 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fcca80316d0 tx=0x7fcca80035f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:04.563 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.560+0000 7fcca77fe640 1 -- 172.21.15.114:0/2734330583 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca8003ba0 con 0x7fccb8107890 2024-09-13T18:31:04.564 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.561+0000 7fccb759e640 1 -- 172.21.15.114:0/2734330583 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccb8114410 con 0x7fccb8107890 2024-09-13T18:31:04.564 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.561+0000 7fccb759e640 1 -- 172.21.15.114:0/2734330583 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccb81148c0 con 0x7fccb8107890 2024-09-13T18:31:04.564 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.561+0000 7fcca77fe640 1 -- 172.21.15.114:0/2734330583 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcca8003d00 con 0x7fccb8107890 2024-09-13T18:31:04.564 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.561+0000 7fcca77fe640 1 -- 172.21.15.114:0/2734330583 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca803c640 con 0x7fccb8107890 2024-09-13T18:31:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.566+0000 7fcca77fe640 1 -- 172.21.15.114:0/2734330583 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fcca8039040 con 0x7fccb8107890 2024-09-13T18:31:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.568+0000 7fccbf6b9740 1 Processor -- start 2024-09-13T18:31:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.568+0000 7fccbf6b9740 1 -- start start 2024-09-13T18:31:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.568+0000 7fccbf6b9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5576cb8a9500 0x5576cb8a98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.568+0000 7fccbf6b9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576cb8a9e20 0x5576cb8b4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.569+0000 7fccb659c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576cb8a9e20 0x5576cb8b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.569+0000 7fccb659c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576cb8a9e20 0x5576cb8b4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:53200/0 (socket says 172.21.15.114:53200) 2024-09-13T18:31:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.569+0000 7fccb659c640 1 -- 172.21.15.114:0/4287820079 learned_addr learned my addr 172.21.15.114:0/4287820079 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.569+0000 7fccbf6b9740 1 --2- 172.21.15.114:0/4287820079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576cb8b46c0 0x5576cb8b6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.569+0000 7fccbf6b9740 1 -- 172.21.15.114:0/4287820079 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576cb814c30 con 0x5576cb8a9500 2024-09-13T18:31:04.575 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.569+0000 7fccbf6b9740 1 -- 172.21.15.114:0/4287820079 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576cb8123c0 con 0x5576cb8b46c0 2024-09-13T18:31:04.575 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.569+0000 7fccb5d9b640 1 --2- 172.21.15.114:0/4287820079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576cb8b46c0 0x5576cb8b6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:04.575 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.569+0000 7fccbf6b9740 1 -- 172.21.15.114:0/4287820079 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576cb80dcb0 con 0x5576cb8a9e20 2024-09-13T18:31:04.575 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.570+0000 7fccb5d9b640 1 -- 172.21.15.114:0/4287820079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576cb8a9e20 msgr2=0x5576cb8b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:04.575 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.570+0000 7fccb5d9b640 1 --2- 172.21.15.114:0/4287820079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576cb8a9e20 0x5576cb8b4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.570+0000 7fccb5d9b640 1 -- 172.21.15.114:0/4287820079 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5576cb8a9500 msgr2=0x5576cb8a98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.570+0000 7fccb5d9b640 1 --2- 172.21.15.114:0/4287820079 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5576cb8a9500 0x5576cb8a98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.570+0000 7fccb5d9b640 1 -- 172.21.15.114:0/4287820079 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5576cb7b4910 con 0x5576cb8b46c0 2024-09-13T18:31:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.570+0000 7fccb5d9b640 1 --2- 172.21.15.114:0/4287820079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576cb8b46c0 0x5576cb8b6ab0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fcca80ec690 tx=0x7fcca803d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4b52efcd36e8738 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.571+0000 7fcca77fe640 1 --2- 172.21.15.114:0/2734330583 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fcca00a14d0 0x7fcca00a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.571+0000 7fcca77fe640 1 -- 172.21.15.114:0/2734330583 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fcca80ef340 con 0x7fccb8107890 2024-09-13T18:31:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.574+0000 7fcca4ff9640 1 -- 172.21.15.114:0/4287820079 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca80f22e0 con 0x5576cb8b46c0 2024-09-13T18:31:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.574+0000 7fcca4ff9640 1 -- 172.21.15.114:0/4287820079 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcca80f2440 con 0x5576cb8b46c0 2024-09-13T18:31:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.574+0000 7fcca4ff9640 1 -- 172.21.15.114:0/4287820079 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca80fe900 con 0x5576cb8b46c0 2024-09-13T18:31:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.575+0000 7fccbf6b9740 1 -- 172.21.15.114:0/4287820079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576cb8b46c0 msgr2=0x5576cb8b6ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.575+0000 7fccbf6b9740 1 --2- 172.21.15.114:0/4287820079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576cb8b46c0 0x5576cb8b6ab0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fcca80ec690 tx=0x7fcca803d000 comp rx=0 tx=0).stop 2024-09-13T18:31:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.574+0000 7fccb6d9d640 1 --2- 172.21.15.114:0/2734330583 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fcca00a14d0 0x7fcca00a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.577+0000 7fccbf6b9740 1 -- 172.21.15.114:0/4287820079 shutdown_connections 2024-09-13T18:31:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.577+0000 7fccbf6b9740 1 --2- 172.21.15.114:0/4287820079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576cb8b46c0 0x5576cb8b6ab0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.577+0000 7fccbf6b9740 1 --2- 172.21.15.114:0/4287820079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576cb8a9e20 0x5576cb8b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.577+0000 7fccbf6b9740 1 --2- 172.21.15.114:0/4287820079 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5576cb8a9500 0x5576cb8a98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.577+0000 7fccbf6b9740 1 -- 172.21.15.114:0/4287820079 >> 172.21.15.114:0/4287820079 conn(0x5576cb8a5a90 msgr2=0x5576cb8a5e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.577+0000 7fccb6d9d640 1 --2- 172.21.15.114:0/2734330583 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fcca00a14d0 0x7fcca00a3990 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fccac006fd0 tx=0x7fccac008040 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.580+0000 7fccbf6b9740 1 -- 172.21.15.114:0/4287820079 shutdown_connections 2024-09-13T18:31:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.580+0000 7fccbf6b9740 1 -- 172.21.15.114:0/4287820079 wait complete. 2024-09-13T18:31:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.580+0000 7fccbf6b9740 1 -- start start 2024-09-13T18:31:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.580+0000 7fccbf6b9740 1 Processor -- start 2024-09-13T18:31:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.581+0000 7fccbf6b9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576cb8a9500 0x5576cb8b4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.581+0000 7fccbf6b9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576cb8a9e20 0x5576cb8b1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.581+0000 7fccbf6b9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5576cb8b46c0 0x5576cb8b1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.581+0000 7fccbf6b9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576cb7cbe80 con 0x5576cb8b46c0 2024-09-13T18:31:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.581+0000 7fccbf6b9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576cb7cbb20 con 0x5576cb8a9e20 2024-09-13T18:31:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.581+0000 7fccbf6b9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576cb791260 con 0x5576cb8a9500 2024-09-13T18:31:04.587 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.581+0000 7fccb659c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576cb8a9500 0x5576cb8b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:04.587 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.581+0000 7fccb659c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576cb8a9500 0x5576cb8b4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:53216/0 (socket says 172.21.15.114:53216) 2024-09-13T18:31:04.587 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.581+0000 7fccb659c640 1 -- 172.21.15.114:0/2089604655 learned_addr learned my addr 172.21.15.114:0/2089604655 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:04.587 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.581+0000 7fccb5d9b640 1 --2- 172.21.15.114:0/2089604655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576cb8a9e20 0x5576cb8b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:04.587 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.581+0000 7fccbf6b9740 10 client.0 _ll_register_callbacks cb 0x5576cb7c6df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:31:04.588 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.582+0000 7fccb659c640 1 -- 172.21.15.114:0/2089604655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576cb8a9e20 msgr2=0x5576cb8b1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:04.588 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.582+0000 7fccb659c640 1 --2- 172.21.15.114:0/2089604655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576cb8a9e20 0x5576cb8b1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:04.588 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.582+0000 7fccb659c640 1 -- 172.21.15.114:0/2089604655 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5576cb8b46c0 msgr2=0x5576cb8b1be0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:31:04.590 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.583+0000 7fccb659c640 1 --2- 172.21.15.114:0/2089604655 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5576cb8b46c0 0x5576cb8b1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:04.590 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.583+0000 7fccb659c640 1 -- 172.21.15.114:0/2089604655 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccb81148c0 con 0x5576cb8a9500 2024-09-13T18:31:04.590 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.583+0000 7fccb659c640 1 --2- 172.21.15.114:0/2089604655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576cb8a9500 0x5576cb8b4110 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fccb00307f0 tx=0x7fccb0030cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e15f61c1f49bffbe server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:04.590 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.584+0000 7fccbf6b9740 1 -- 172.21.15.114:0/2089604655 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5576cb836680 con 0x5576cb8a9500 2024-09-13T18:31:04.591 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.583+0000 7fcc9dffb640 10 client.27379 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:31:04.591 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.584+0000 7fcc9dffb640 1 -- 172.21.15.114:0/2089604655 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb0009b00 con 0x5576cb8a9500 2024-09-13T18:31:04.591 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.584+0000 7fcc9dffb640 1 -- 172.21.15.114:0/2089604655 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fccb002ea40 con 0x5576cb8a9500 2024-09-13T18:31:04.591 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.584+0000 7fcc9dffb640 1 -- 172.21.15.114:0/2089604655 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fccb002eca0 con 0x5576cb8a9500 2024-09-13T18:31:04.591 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.584+0000 7fccbf6b9740 10 client.27379 fetch_fsmap learned FSMap version 34 2024-09-13T18:31:04.592 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.584+0000 7fccbf6b9740 1 -- 172.21.15.114:0/2089604655 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({fsmap=34}) v3 -- 0x5576cb80c330 con 0x5576cb8a9500 2024-09-13T18:31:04.592 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.587+0000 7fcc9dffb640 1 -- 172.21.15.114:0/2089604655 <== mon.1 v2:172.21.15.204:3300/0 4 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fccb003e440 con 0x5576cb8a9500 2024-09-13T18:31:04.592 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.587+0000 7fccbf6b9740 10 client.27379 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:31:04.592 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.587+0000 7fccbf6b9740 10 client.27379 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:31:04.592 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.588+0000 7fccbf6b9740 20 client.27379 populate_metadata read hostname 'smithi114' 2024-09-13T18:31:04.594 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.588+0000 7fccbf6b9740 1 --2- 172.21.15.114:0/2089604655 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5576cb8c3d70 0x5576cb8c61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:04.594 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.588+0000 7fccbf6b9740 4 client.27379 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:31:04.594 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.588+0000 7fccbf6b9740 1 -- 172.21.15.114:0/2089604655 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5576cb836680 con 0x5576cb8c3d70 2024-09-13T18:31:04.594 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.589+0000 7fccb5d9b640 1 --2- 172.21.15.114:0/2089604655 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5576cb8c3d70 0x5576cb8c61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:04.595 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.592+0000 7fccb5d9b640 1 --2- 172.21.15.114:0/2089604655 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5576cb8c3d70 0x5576cb8c61f0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbb44f3986ad2771 server_cookie=29a457ab226093b in_seq=0 out_seq=0 2024-09-13T18:31:04.595 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.592+0000 7fcc9dffb640 10 client.27379 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:31:04.595 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.593+0000 7fcc9dffb640 1 -- 172.21.15.114:0/2089604655 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5576cb836680 con 0x5576cb8c3d70 2024-09-13T18:31:04.595 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.593+0000 7fcc9dffb640 10 client.27379 handle_command_reply: tid=0 2024-09-13T18:31:04.596 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.594+0000 7fccbf6b9740 1 client.27379 shutdown 2024-09-13T18:31:04.606 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.603+0000 7fccbf6b9740 1 -- 172.21.15.114:0/2089604655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576cb8a9500 msgr2=0x5576cb8b4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:04.606 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.603+0000 7fccbf6b9740 1 --2- 172.21.15.114:0/2089604655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576cb8a9500 0x5576cb8b4110 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fccb00307f0 tx=0x7fccb0030cb0 comp rx=0 tx=0).stop 2024-09-13T18:31:04.614 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.611+0000 7fccbf6b9740 1 -- 172.21.15.114:0/2089604655 shutdown_connections 2024-09-13T18:31:04.614 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.611+0000 7fccbf6b9740 1 --2- 172.21.15.114:0/2089604655 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5576cb8c3d70 0x5576cb8c61f0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:04.614 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.612+0000 7fccbf6b9740 1 --2- 172.21.15.114:0/2089604655 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5576cb8b46c0 0x5576cb8b1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:04.614 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.612+0000 7fccbf6b9740 1 --2- 172.21.15.114:0/2089604655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576cb8a9e20 0x5576cb8b1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:04.614 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.612+0000 7fccbf6b9740 1 --2- 172.21.15.114:0/2089604655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576cb8a9500 0x5576cb8b4110 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:04.614 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.612+0000 7fccbf6b9740 1 -- 172.21.15.114:0/2089604655 >> 172.21.15.114:0/2089604655 conn(0x5576cb8a5a90 msgr2=0x5576cb8a4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:04.617 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.613+0000 7fcc9dffb640 0 client.27379 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:31:04.619 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.617+0000 7fccbf6b9740 1 -- 172.21.15.114:0/2089604655 shutdown_connections 2024-09-13T18:31:04.620 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.618+0000 7fccbf6b9740 1 -- 172.21.15.114:0/2089604655 wait complete. 2024-09-13T18:31:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.629+0000 7fccbf6b9740 20 client.27379 trim_cache size 0 max 16384 2024-09-13T18:31:04.660 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.658+0000 7fccbf6b9740 1 Processor -- start 2024-09-13T18:31:04.661 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.658+0000 7fccbf6b9740 1 -- start start 2024-09-13T18:31:04.661 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.659+0000 7fccbf6b9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576cb8a9500 0x5576cb8b3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.659+0000 7fccbf6b9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576cb8a9e20 0x5576cb8ac720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.659+0000 7fccb659c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576cb8a9500 0x5576cb8b3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.659+0000 7fccb659c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576cb8a9500 0x5576cb8b3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:53220/0 (socket says 172.21.15.114:53220) 2024-09-13T18:31:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.659+0000 7fccbf6b9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5576cb8b46c0 0x5576cb8acc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.659+0000 7fccb5d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576cb8a9e20 0x5576cb8ac720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.659+0000 7fccb5d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576cb8a9e20 0x5576cb8ac720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:45692/0 (socket says 172.21.15.114:45692) 2024-09-13T18:31:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.660+0000 7fccb6d9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5576cb8b46c0 0x5576cb8acc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.660+0000 7fccb659c640 1 -- 172.21.15.114:0/2866247033 learned_addr learned my addr 172.21.15.114:0/2866247033 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.660+0000 7fccbf6b9740 1 -- 172.21.15.114:0/2866247033 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576cb8a9c60 con 0x5576cb8b46c0 2024-09-13T18:31:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.660+0000 7fccbf6b9740 1 -- 172.21.15.114:0/2866247033 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576cb8a6690 con 0x5576cb8a9e20 2024-09-13T18:31:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.660+0000 7fccbf6b9740 1 -- 172.21.15.114:0/2866247033 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576cb8cb2a0 con 0x5576cb8a9500 2024-09-13T18:31:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.661+0000 7fccb6d9d640 1 -- 172.21.15.114:0/2866247033 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576cb8a9500 msgr2=0x5576cb8b3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.661+0000 7fccb6d9d640 1 --2- 172.21.15.114:0/2866247033 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576cb8a9500 0x5576cb8b3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.661+0000 7fccb6d9d640 1 -- 172.21.15.114:0/2866247033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576cb8a9e20 msgr2=0x5576cb8ac720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.661+0000 7fccb6d9d640 1 --2- 172.21.15.114:0/2866247033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576cb8a9e20 0x5576cb8ac720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.661+0000 7fccb6d9d640 1 -- 172.21.15.114:0/2866247033 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5576cb7ae300 con 0x5576cb8b46c0 2024-09-13T18:31:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.661+0000 7fccb6d9d640 1 --2- 172.21.15.114:0/2866247033 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5576cb8b46c0 0x5576cb8acc60 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fccb80690c0 tx=0x7fccac004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77858d17e3cfd3c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.662+0000 7fcca57fa640 1 -- 172.21.15.114:0/2866247033 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccac00de00 con 0x5576cb8b46c0 2024-09-13T18:31:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.662+0000 7fcca57fa640 1 -- 172.21.15.114:0/2866247033 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fccac014da0 con 0x5576cb8b46c0 2024-09-13T18:31:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.662+0000 7fcca57fa640 1 -- 172.21.15.114:0/2866247033 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccac01ed10 con 0x5576cb8b46c0 2024-09-13T18:31:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.662+0000 7fccb659c640 1 --2- 172.21.15.114:0/2866247033 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576cb8a9500 0x5576cb8b3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:31:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.662+0000 7fccb5d9b640 1 --2- 172.21.15.114:0/2866247033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576cb8a9e20 0x5576cb8ac720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:31:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.663+0000 7fccbf6b9740 1 -- 172.21.15.114:0/2866247033 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5576cb8b46c0 msgr2=0x5576cb8acc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.663+0000 7fccbf6b9740 1 --2- 172.21.15.114:0/2866247033 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5576cb8b46c0 0x5576cb8acc60 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fccb80690c0 tx=0x7fccac004700 comp rx=0 tx=0).stop 2024-09-13T18:31:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.664+0000 7fccbf6b9740 1 -- 172.21.15.114:0/2866247033 shutdown_connections 2024-09-13T18:31:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.664+0000 7fccbf6b9740 1 --2- 172.21.15.114:0/2866247033 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5576cb8b46c0 0x5576cb8acc60 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.664+0000 7fccbf6b9740 1 --2- 172.21.15.114:0/2866247033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576cb8a9e20 0x5576cb8ac720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.664+0000 7fccbf6b9740 1 --2- 172.21.15.114:0/2866247033 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576cb8a9500 0x5576cb8b3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.664+0000 7fccbf6b9740 1 -- 172.21.15.114:0/2866247033 >> 172.21.15.114:0/2866247033 conn(0x5576cb8a5a90 msgr2=0x5576cb8cab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.665+0000 7fccbf6b9740 1 -- 172.21.15.114:0/2866247033 shutdown_connections 2024-09-13T18:31:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.665+0000 7fccbf6b9740 1 -- 172.21.15.114:0/2866247033 wait complete. 2024-09-13T18:31:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.666+0000 7fccbf6b9740 1 -- start start 2024-09-13T18:31:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.666+0000 7fccbf6b9740 1 Processor -- start 2024-09-13T18:31:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.666+0000 7fccbf6b9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5576cb8a9500 0x5576cb8ae180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.667+0000 7fccbf6b9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576cb8a9e20 0x5576cb8ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.667+0000 7fccbf6b9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576cb8b46c0 0x5576cb8aec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.667+0000 7fccbf6b9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576cb91a310 con 0x5576cb8a9500 2024-09-13T18:31:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.667+0000 7fccbf6b9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576cb91a450 con 0x5576cb8a9e20 2024-09-13T18:31:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.667+0000 7fccbf6b9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576cb91a590 con 0x5576cb8b46c0 2024-09-13T18:31:04.671 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.667+0000 7fccbf6b9740 10 client.0 _ll_register_callbacks cb 0x7fcca803d660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:31:04.671 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.667+0000 7fccb659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5576cb8a9500 0x5576cb8ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:04.671 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.668+0000 7fccb659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5576cb8a9500 0x5576cb8ae180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51710/0 (socket says 172.21.15.114:51710) 2024-09-13T18:31:04.671 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.668+0000 7fccb659c640 1 -- 172.21.15.114:0/750142652 learned_addr learned my addr 172.21.15.114:0/750142652 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:04.671 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.668+0000 7fccb5d9b640 1 --2- 172.21.15.114:0/750142652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576cb8a9e20 0x5576cb8ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:04.671 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.668+0000 7fccb6d9d640 1 --2- 172.21.15.114:0/750142652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576cb8b46c0 0x5576cb8aec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:04.671 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.668+0000 7fccb659c640 1 -- 172.21.15.114:0/750142652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576cb8b46c0 msgr2=0x5576cb8aec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:04.671 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.668+0000 7fccb659c640 1 --2- 172.21.15.114:0/750142652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576cb8b46c0 0x5576cb8aec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.668+0000 7fccb659c640 1 -- 172.21.15.114:0/750142652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576cb8a9e20 msgr2=0x5576cb8ae6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.668+0000 7fccb659c640 1 --2- 172.21.15.114:0/750142652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576cb8a9e20 0x5576cb8ae6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.668+0000 7fccb659c640 1 -- 172.21.15.114:0/750142652 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccac00e040 con 0x5576cb8a9500 2024-09-13T18:31:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.668+0000 7fccb659c640 1 --2- 172.21.15.114:0/750142652 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5576cb8a9500 0x5576cb8ae180 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fccb00070b0 tx=0x7fccb0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbcac884ad3d2554 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.669+0000 7fcc9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:31:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.669+0000 7fcc9dffb640 1 -- 172.21.15.114:0/750142652 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb0030410 con 0x5576cb8a9500 2024-09-13T18:31:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.669+0000 7fcc9dffb640 1 -- 172.21.15.114:0/750142652 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fccb003e900 con 0x5576cb8a9500 2024-09-13T18:31:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.669+0000 7fccbf6b9740 1 -- 172.21.15.114:0/750142652 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcca80f2440 con 0x5576cb8a9500 2024-09-13T18:31:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.669+0000 7fcc9dffb640 1 -- 172.21.15.114:0/750142652 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fccb003c360 con 0x5576cb8a9500 2024-09-13T18:31:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.669+0000 7fccbf6b9740 10 client.17970 fetch_fsmap learned FSMap version 34 2024-09-13T18:31:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.669+0000 7fccbf6b9740 1 -- 172.21.15.114:0/750142652 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5576cb7d38e0 con 0x5576cb8a9500 2024-09-13T18:31:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.674+0000 7fcc9dffb640 1 -- 172.21.15.114:0/750142652 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcca80f2440 con 0x5576cb8a9500 2024-09-13T18:31:04.677 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.675+0000 7fcc9dffb640 1 -- 172.21.15.114:0/750142652 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fccb003ca20 con 0x5576cb8a9500 2024-09-13T18:31:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.677+0000 7fccbf6b9740 10 client.17970 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:31:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.677+0000 7fccbf6b9740 10 client.17970 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:31:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.677+0000 7fccbf6b9740 20 client.17970 populate_metadata read hostname 'smithi114' 2024-09-13T18:31:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.677+0000 7fccbf6b9740 1 --2- 172.21.15.114:0/750142652 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5576cb91bb50 0x5576cb91df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.677+0000 7fccbf6b9740 4 client.17970 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T18:31:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.677+0000 7fccbf6b9740 1 -- 172.21.15.114:0/750142652 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5576cb8015d0 con 0x5576cb91bb50 2024-09-13T18:31:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.681+0000 7fccb5d9b640 1 --2- 172.21.15.114:0/750142652 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5576cb91bb50 0x5576cb91df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.694+0000 7fccb5d9b640 1 --2- 172.21.15.114:0/750142652 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5576cb91bb50 0x5576cb91df70 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=854eeb816b55ac38 server_cookie=e857f96b2e139a64 in_seq=0 out_seq=0 2024-09-13T18:31:04.700 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.695+0000 7fcc9dffb640 10 client.17970 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:31:04.732 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.730+0000 7fcc9dffb640 1 -- 172.21.15.114:0/750142652 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5576cb8015d0 con 0x5576cb91bb50 2024-09-13T18:31:04.732 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.730+0000 7fcc9dffb640 10 client.17970 handle_command_reply: tid=0 2024-09-13T18:31:04.732 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.730+0000 7fccbf6b9740 1 client.17970 shutdown 2024-09-13T18:31:04.739 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.736+0000 7fccbf6b9740 1 -- 172.21.15.114:0/750142652 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5576cb8a9500 msgr2=0x5576cb8ae180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:04.739 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.736+0000 7fccbf6b9740 1 --2- 172.21.15.114:0/750142652 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5576cb8a9500 0x5576cb8ae180 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fccb00070b0 tx=0x7fccb0004ad0 comp rx=0 tx=0).stop 2024-09-13T18:31:04.742 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.737+0000 7fccbf6b9740 1 -- 172.21.15.114:0/750142652 shutdown_connections 2024-09-13T18:31:04.742 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.737+0000 7fccbf6b9740 1 --2- 172.21.15.114:0/750142652 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5576cb91bb50 0x5576cb91df70 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:04.742 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.737+0000 7fccbf6b9740 1 --2- 172.21.15.114:0/750142652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576cb8b46c0 0x5576cb8aec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:04.742 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.737+0000 7fccbf6b9740 1 --2- 172.21.15.114:0/750142652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576cb8a9e20 0x5576cb8ae6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:04.742 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.737+0000 7fccbf6b9740 1 --2- 172.21.15.114:0/750142652 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5576cb8a9500 0x5576cb8ae180 secure :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fccb00070b0 tx=0x7fccb0004ad0 comp rx=0 tx=0).stop 2024-09-13T18:31:04.742 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.737+0000 7fcc9dffb640 0 client.17970 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:31:04.742 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.737+0000 7fccbf6b9740 1 -- 172.21.15.114:0/750142652 >> 172.21.15.114:0/750142652 conn(0x5576cb8a5a90 msgr2=0x5576cb8ac720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.740+0000 7fccbf6b9740 1 -- 172.21.15.114:0/750142652 shutdown_connections 2024-09-13T18:31:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.745+0000 7fccbf6b9740 1 -- 172.21.15.114:0/750142652 wait complete. 2024-09-13T18:31:04.754 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.750+0000 7fccbf6b9740 20 client.17970 trim_cache size 0 max 16384 2024-09-13T18:31:04.754 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-13T18:31:04.754 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-13T18:31:04.754 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-13T18:31:04.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-13T18:31:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.759+0000 7fccb759e640 1 -- 172.21.15.114:0/2734330583 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fcca00a14d0 msgr2=0x7fcca00a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.759+0000 7fccb759e640 1 --2- 172.21.15.114:0/2734330583 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fcca00a14d0 0x7fcca00a3990 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fccac006fd0 tx=0x7fccac008040 comp rx=0 tx=0).stop 2024-09-13T18:31:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.759+0000 7fccb759e640 1 -- 172.21.15.114:0/2734330583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccb8107890 msgr2=0x7fccb8116e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.759+0000 7fccb759e640 1 --2- 172.21.15.114:0/2734330583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccb8107890 0x7fccb8116e30 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fcca80316d0 tx=0x7fcca80035f0 comp rx=0 tx=0).stop 2024-09-13T18:31:04.764 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.759+0000 7fccb759e640 1 -- 172.21.15.114:0/2734330583 shutdown_connections 2024-09-13T18:31:04.764 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.759+0000 7fccb759e640 1 --2- 172.21.15.114:0/2734330583 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fcca00a14d0 0x7fcca00a3990 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:04.764 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.759+0000 7fccb759e640 1 --2- 172.21.15.114:0/2734330583 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fccb8119400 0x7fccb810fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:04.764 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.759+0000 7fccb759e640 1 --2- 172.21.15.114:0/2734330583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccb8107890 0x7fccb8116e30 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:04.764 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.759+0000 7fccb759e640 1 --2- 172.21.15.114:0/2734330583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fccb81074b0 0x7fccb81168f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:04.765 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.759+0000 7fccb759e640 1 -- 172.21.15.114:0/2734330583 >> 172.21.15.114:0/2734330583 conn(0x7fccb806adb0 msgr2=0x7fccb8119a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:04.767 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.763+0000 7fccb759e640 1 -- 172.21.15.114:0/2734330583 shutdown_connections 2024-09-13T18:31:04.771 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:04.765+0000 7fccb759e640 1 -- 172.21.15.114:0/2734330583 wait complete. 2024-09-13T18:31:05.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:05 smithi134 ceph-mon[34283]: pgmap v265: 129 pgs: 129 active+clean; 23 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 30 MiB/s wr, 652 op/s 2024-09-13T18:31:05.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:05 smithi204 ceph-mon[34777]: pgmap v265: 129 pgs: 129 active+clean; 23 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 30 MiB/s wr, 652 op/s 2024-09-13T18:31:05.973 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:05 smithi114 ceph-mon[30897]: pgmap v265: 129 pgs: 129 active+clean; 23 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 30 MiB/s wr, 652 op/s 2024-09-13T18:31:07.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:07 smithi204 ceph-mon[34777]: pgmap v266: 129 pgs: 129 active+clean; 23 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 37 MiB/s wr, 800 op/s 2024-09-13T18:31:07.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:07 smithi204 ceph-mon[34777]: osdmap e99: 12 total, 12 up, 12 in 2024-09-13T18:31:07.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:07 smithi114 ceph-mon[30897]: pgmap v266: 129 pgs: 129 active+clean; 23 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 37 MiB/s wr, 800 op/s 2024-09-13T18:31:07.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:07 smithi114 ceph-mon[30897]: osdmap e99: 12 total, 12 up, 12 in 2024-09-13T18:31:08.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:07 smithi134 ceph-mon[34283]: pgmap v266: 129 pgs: 129 active+clean; 23 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 37 MiB/s wr, 800 op/s 2024-09-13T18:31:08.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:07 smithi134 ceph-mon[34283]: osdmap e99: 12 total, 12 up, 12 in 2024-09-13T18:31:09.816 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T18:31:09.876 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:09 smithi114 ceph-mon[30897]: pgmap v268: 129 pgs: 129 active+clean; 23 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 32 MiB/s wr, 697 op/s 2024-09-13T18:31:09.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:09 smithi204 ceph-mon[34777]: pgmap v268: 129 pgs: 129 active+clean; 23 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 32 MiB/s wr, 697 op/s 2024-09-13T18:31:10.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.024+0000 7fe66720d640 1 -- 172.21.15.114:0/3370851566 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe660107540 msgr2=0x7fe66010e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:10.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.024+0000 7fe66720d640 1 --2- 172.21.15.114:0/3370851566 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe660107540 0x7fe66010e510 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fe650009900 tx=0x7fe65002a410 comp rx=0 tx=0).stop 2024-09-13T18:31:10.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.033+0000 7fe66720d640 1 -- 172.21.15.114:0/3370851566 shutdown_connections 2024-09-13T18:31:10.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.033+0000 7fe66720d640 1 --2- 172.21.15.114:0/3370851566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe660119400 0x7fe66011b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:10.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.033+0000 7fe66720d640 1 --2- 172.21.15.114:0/3370851566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe660107920 0x7fe660107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:10.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.033+0000 7fe66720d640 1 --2- 172.21.15.114:0/3370851566 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe660107540 0x7fe66010e510 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:10.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.033+0000 7fe66720d640 1 -- 172.21.15.114:0/3370851566 >> 172.21.15.114:0/3370851566 conn(0x7fe66006afe0 msgr2=0x7fe66006b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:10.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.042+0000 7fe66720d640 1 -- 172.21.15.114:0/3370851566 shutdown_connections 2024-09-13T18:31:10.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.046+0000 7fe66720d640 1 -- 172.21.15.114:0/3370851566 wait complete. 2024-09-13T18:31:10.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.046+0000 7fe66720d640 1 Processor -- start 2024-09-13T18:31:10.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.047+0000 7fe66720d640 1 -- start start 2024-09-13T18:31:10.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.047+0000 7fe66720d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe660107540 0x7fe660118820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:10.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.047+0000 7fe66720d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe660107920 0x7fe660118d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:10.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.047+0000 7fe66720d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe660119400 0x7fe660112910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:10.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.047+0000 7fe66720d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6601130a0 con 0x7fe660107540 2024-09-13T18:31:10.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.047+0000 7fe66720d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe660113210 con 0x7fe660107920 2024-09-13T18:31:10.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.047+0000 7fe66720d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6601133d0 con 0x7fe660119400 2024-09-13T18:31:10.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.051+0000 7fe6657aa640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe660119400 0x7fe660112910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:10.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.051+0000 7fe6657aa640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe660119400 0x7fe660112910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:53232/0 (socket says 172.21.15.114:53232) 2024-09-13T18:31:10.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.051+0000 7fe6657aa640 1 -- 172.21.15.114:0/2764678548 learned_addr learned my addr 172.21.15.114:0/2764678548 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:10.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.052+0000 7fe65ffff640 1 --2- 172.21.15.114:0/2764678548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe660107920 0x7fe660118d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:10.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.052+0000 7fe664fa9640 1 --2- 172.21.15.114:0/2764678548 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe660107540 0x7fe660118820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:10.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.054+0000 7fe65ffff640 1 -- 172.21.15.114:0/2764678548 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe660119400 msgr2=0x7fe660112910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:10.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.054+0000 7fe65ffff640 1 --2- 172.21.15.114:0/2764678548 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe660119400 0x7fe660112910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:10.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.054+0000 7fe65ffff640 1 -- 172.21.15.114:0/2764678548 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe660107540 msgr2=0x7fe660118820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:10.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.054+0000 7fe65ffff640 1 --2- 172.21.15.114:0/2764678548 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe660107540 0x7fe660118820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:10.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.054+0000 7fe65ffff640 1 -- 172.21.15.114:0/2764678548 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe650009560 con 0x7fe660107920 2024-09-13T18:31:10.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.055+0000 7fe65ffff640 1 --2- 172.21.15.114:0/2764678548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe660107920 0x7fe660118d60 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe65800bd70 tx=0x7fe65800be60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:10.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.056+0000 7fe65dffb640 1 -- 172.21.15.114:0/2764678548 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe658014440 con 0x7fe660107920 2024-09-13T18:31:10.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.056+0000 7fe66720d640 1 -- 172.21.15.114:0/2764678548 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6601136b0 con 0x7fe660107920 2024-09-13T18:31:10.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.056+0000 7fe66720d640 1 -- 172.21.15.114:0/2764678548 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6601ba6a0 con 0x7fe660107920 2024-09-13T18:31:10.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.056+0000 7fe65dffb640 1 -- 172.21.15.114:0/2764678548 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe658014a30 con 0x7fe660107920 2024-09-13T18:31:10.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.056+0000 7fe65dffb640 1 -- 172.21.15.114:0/2764678548 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe658012900 con 0x7fe660107920 2024-09-13T18:31:10.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.069+0000 7fe65dffb640 1 -- 172.21.15.114:0/2764678548 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fe658012b80 con 0x7fe660107920 2024-09-13T18:31:10.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.069+0000 7fe66a00c740 1 Processor -- start 2024-09-13T18:31:10.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.073+0000 7fe66a00c740 1 -- start start 2024-09-13T18:31:10.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.074+0000 7fe66a00c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea22d5fe10 0x55ea22d601f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:10.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.074+0000 7fe66a00c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ea22d60730 0x55ea22d6aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:10.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.074+0000 7fe66a00c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea22d6afd0 0x55ea22d6d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:10.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.074+0000 7fe66a00c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea22ccaf20 con 0x55ea22d60730 2024-09-13T18:31:10.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.074+0000 7fe66a00c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea22cc86b0 con 0x55ea22d6afd0 2024-09-13T18:31:10.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.074+0000 7fe66a00c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea22cc3fa0 con 0x55ea22d5fe10 2024-09-13T18:31:10.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.074+0000 7fe65ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea22d6afd0 0x55ea22d6d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:10.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.074+0000 7fe65dffb640 1 --2- 172.21.15.114:0/2764678548 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe6480a1410 0x7fe6480a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:10.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.075+0000 7fe6657aa640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea22d5fe10 0x55ea22d601f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:10.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.075+0000 7fe65dffb640 1 -- 172.21.15.114:0/2764678548 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fe6580ccad0 con 0x7fe660107920 2024-09-13T18:31:10.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.075+0000 7fe6657aa640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea22d5fe10 0x55ea22d601f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:53246/0 (socket says 172.21.15.114:53246) 2024-09-13T18:31:10.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.075+0000 7fe6657aa640 1 -- 172.21.15.114:0/1286964372 learned_addr learned my addr 172.21.15.114:0/1286964372 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:10.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.076+0000 7fe6657aa640 1 -- 172.21.15.114:0/1286964372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea22d6afd0 msgr2=0x55ea22d6d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:10.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.076+0000 7fe6657aa640 1 --2- 172.21.15.114:0/1286964372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea22d6afd0 0x55ea22d6d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:10.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.076+0000 7fe6657aa640 1 -- 172.21.15.114:0/1286964372 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ea22d60730 msgr2=0x55ea22d6aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:10.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.076+0000 7fe6657aa640 1 --2- 172.21.15.114:0/1286964372 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ea22d60730 0x55ea22d6aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:10.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.076+0000 7fe6657aa640 1 -- 172.21.15.114:0/1286964372 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea22c7e840 con 0x55ea22d5fe10 2024-09-13T18:31:10.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.076+0000 7fe6657aa640 1 --2- 172.21.15.114:0/2764678548 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe6480a1410 0x7fe6480a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:10.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.076+0000 7fe6657aa640 1 --2- 172.21.15.114:0/1286964372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea22d5fe10 0x55ea22d601f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe654009bb0 tx=0x7fe6540101f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7fb6465d30e0165 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:10.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.077+0000 7fe65ffff640 1 --2- 172.21.15.114:0/1286964372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea22d6afd0 0x55ea22d6d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:31:10.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.083+0000 7fe646ffd640 1 -- 172.21.15.114:0/1286964372 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe654010e40 con 0x55ea22d5fe10 2024-09-13T18:31:10.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.083+0000 7fe646ffd640 1 -- 172.21.15.114:0/1286964372 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe654018070 con 0x55ea22d5fe10 2024-09-13T18:31:10.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.083+0000 7fe646ffd640 1 -- 172.21.15.114:0/1286964372 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65401d800 con 0x55ea22d5fe10 2024-09-13T18:31:10.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.084+0000 7fe66a00c740 1 -- 172.21.15.114:0/1286964372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea22d5fe10 msgr2=0x55ea22d601f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:10.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.084+0000 7fe66a00c740 1 --2- 172.21.15.114:0/1286964372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea22d5fe10 0x55ea22d601f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe654009bb0 tx=0x7fe6540101f0 comp rx=0 tx=0).stop 2024-09-13T18:31:10.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.087+0000 7fe6657aa640 1 --2- 172.21.15.114:0/2764678548 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe6480a1410 0x7fe6480a38d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe65401e000 tx=0x7fe65401da40 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:10.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.087+0000 7fe66a00c740 1 -- 172.21.15.114:0/1286964372 shutdown_connections 2024-09-13T18:31:10.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.087+0000 7fe66a00c740 1 --2- 172.21.15.114:0/1286964372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea22d6afd0 0x55ea22d6d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:10.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.087+0000 7fe66a00c740 1 --2- 172.21.15.114:0/1286964372 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ea22d60730 0x55ea22d6aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:10.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.087+0000 7fe66a00c740 1 --2- 172.21.15.114:0/1286964372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea22d5fe10 0x55ea22d601f0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:10.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.087+0000 7fe66a00c740 1 -- 172.21.15.114:0/1286964372 >> 172.21.15.114:0/1286964372 conn(0x55ea22d5c380 msgr2=0x55ea22d5c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:10.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.091+0000 7fe66a00c740 1 -- 172.21.15.114:0/1286964372 shutdown_connections 2024-09-13T18:31:10.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.092+0000 7fe66a00c740 1 -- 172.21.15.114:0/1286964372 wait complete. 2024-09-13T18:31:10.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.092+0000 7fe66a00c740 1 -- start start 2024-09-13T18:31:10.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.093+0000 7fe66a00c740 1 Processor -- start 2024-09-13T18:31:10.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.098+0000 7fe66a00c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea22d5fe10 0x55ea22d6a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:10.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.098+0000 7fe66a00c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea22d60730 0x55ea22d67c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:10.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.098+0000 7fe66a00c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ea22d6afd0 0x55ea22d68260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:10.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.098+0000 7fe66a00c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea22c652a0 con 0x55ea22d6afd0 2024-09-13T18:31:10.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.098+0000 7fe66a00c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea2299c430 con 0x55ea22d60730 2024-09-13T18:31:10.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.098+0000 7fe66a00c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea22c47680 con 0x55ea22d5fe10 2024-09-13T18:31:10.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.098+0000 7fe66a00c740 10 client.0 _ll_register_callbacks cb 0x55ea22cbd610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:31:10.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.101+0000 7fe65ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea22d60730 0x55ea22d67c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:10.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.101+0000 7fe65ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea22d60730 0x55ea22d67c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:45746/0 (socket says 172.21.15.114:45746) 2024-09-13T18:31:10.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.101+0000 7fe65ffff640 1 -- 172.21.15.114:0/2438938863 learned_addr learned my addr 172.21.15.114:0/2438938863 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:10.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.101+0000 7fe664fa9640 1 --2- 172.21.15.114:0/2438938863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea22d5fe10 0x55ea22d6a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:10.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.103+0000 7fe65ffff640 1 -- 172.21.15.114:0/2438938863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea22d5fe10 msgr2=0x55ea22d6a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:10.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.103+0000 7fe65ffff640 1 --2- 172.21.15.114:0/2438938863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea22d5fe10 0x55ea22d6a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:10.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.103+0000 7fe65ffff640 1 -- 172.21.15.114:0/2438938863 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ea22d6afd0 msgr2=0x55ea22d68260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:10.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.103+0000 7fe65ffff640 1 --2- 172.21.15.114:0/2438938863 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ea22d6afd0 0x55ea22d68260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:10.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.103+0000 7fe65ffff640 1 -- 172.21.15.114:0/2438938863 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe654009810 con 0x55ea22d60730 2024-09-13T18:31:10.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.104+0000 7fe65ffff640 1 --2- 172.21.15.114:0/2438938863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea22d60730 0x55ea22d67c70 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe658097290 tx=0x7fe658004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21e8119c0e18912c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:10.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.104+0000 7fe63ffff640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:31:10.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.105+0000 7fe63ffff640 1 -- 172.21.15.114:0/2438938863 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65809e5a0 con 0x55ea22d60730 2024-09-13T18:31:10.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.105+0000 7fe66a00c740 1 -- 172.21.15.114:0/2438938863 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ea229b74e0 con 0x55ea22d60730 2024-09-13T18:31:10.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.105+0000 7fe63ffff640 1 -- 172.21.15.114:0/2438938863 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fe6580cce40 con 0x55ea22d60730 2024-09-13T18:31:10.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.105+0000 7fe63ffff640 1 -- 172.21.15.114:0/2438938863 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6580b3140 con 0x55ea22d60730 2024-09-13T18:31:10.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.105+0000 7fe66a00c740 10 client.27407 fetch_fsmap learned FSMap version 34 2024-09-13T18:31:10.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.105+0000 7fe66a00c740 1 -- 172.21.15.114:0/2438938863 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55ea22cc2620 con 0x55ea22d60730 2024-09-13T18:31:10.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.111+0000 7fe63ffff640 1 -- 172.21.15.114:0/2438938863 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ea229b74e0 con 0x55ea22d60730 2024-09-13T18:31:10.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.111+0000 7fe63ffff640 1 -- 172.21.15.114:0/2438938863 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe658008030 con 0x55ea22d60730 2024-09-13T18:31:10.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:09 smithi134 ceph-mon[34283]: pgmap v268: 129 pgs: 129 active+clean; 23 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 32 MiB/s wr, 697 op/s 2024-09-13T18:31:10.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.117+0000 7fe66a00c740 10 client.27407 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:31:10.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.117+0000 7fe66a00c740 10 client.27407 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:31:10.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.117+0000 7fe66a00c740 20 client.27407 populate_metadata read hostname 'smithi114' 2024-09-13T18:31:10.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.117+0000 7fe66a00c740 1 --2- 172.21.15.114:0/2438938863 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55ea22d7aeb0 0x55ea22d7d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:10.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.117+0000 7fe66a00c740 4 client.27407 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:31:10.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.117+0000 7fe66a00c740 1 -- 172.21.15.114:0/2438938863 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ea229c87e0 con 0x55ea22d7aeb0 2024-09-13T18:31:10.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.121+0000 7fe664fa9640 1 --2- 172.21.15.114:0/2438938863 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55ea22d7aeb0 0x55ea22d7d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:10.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.131+0000 7fe664fa9640 1 --2- 172.21.15.114:0/2438938863 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55ea22d7aeb0 0x55ea22d7d2d0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5510ac8e1f5c6df server_cookie=7d24e8f50fe2c014 in_seq=0 out_seq=0 2024-09-13T18:31:10.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.131+0000 7fe63ffff640 10 client.27407 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:31:10.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.134+0000 7fe63ffff640 1 -- 172.21.15.114:0/2438938863 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55ea229c87e0 con 0x55ea22d7aeb0 2024-09-13T18:31:10.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.134+0000 7fe63ffff640 10 client.27407 handle_command_reply: tid=0 2024-09-13T18:31:10.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.138+0000 7fe66a00c740 1 client.27407 shutdown 2024-09-13T18:31:10.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.140+0000 7fe66a00c740 1 -- 172.21.15.114:0/2438938863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea22d60730 msgr2=0x55ea22d67c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:10.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.140+0000 7fe66a00c740 1 --2- 172.21.15.114:0/2438938863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea22d60730 0x55ea22d67c70 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe658097290 tx=0x7fe658004a40 comp rx=0 tx=0).stop 2024-09-13T18:31:10.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.144+0000 7fe66a00c740 1 -- 172.21.15.114:0/2438938863 shutdown_connections 2024-09-13T18:31:10.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.144+0000 7fe66a00c740 1 --2- 172.21.15.114:0/2438938863 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55ea22d7aeb0 0x55ea22d7d2d0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:10.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.144+0000 7fe66a00c740 1 --2- 172.21.15.114:0/2438938863 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ea22d6afd0 0x55ea22d68260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:10.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.144+0000 7fe66a00c740 1 --2- 172.21.15.114:0/2438938863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea22d60730 0x55ea22d67c70 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:10.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.144+0000 7fe66a00c740 1 --2- 172.21.15.114:0/2438938863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea22d5fe10 0x55ea22d6a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:10.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.144+0000 7fe66a00c740 1 -- 172.21.15.114:0/2438938863 >> 172.21.15.114:0/2438938863 conn(0x55ea22d5c380 msgr2=0x55ea22d620e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:10.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.146+0000 7fe63ffff640 0 client.27407 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:31:10.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.146+0000 7fe66a00c740 1 -- 172.21.15.114:0/2438938863 shutdown_connections 2024-09-13T18:31:10.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.146+0000 7fe66a00c740 1 -- 172.21.15.114:0/2438938863 wait complete. 2024-09-13T18:31:10.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.154+0000 7fe66a00c740 20 client.27407 trim_cache size 0 max 16384 2024-09-13T18:31:10.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.185+0000 7fe66a00c740 1 Processor -- start 2024-09-13T18:31:10.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.185+0000 7fe66a00c740 1 -- start start 2024-09-13T18:31:10.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.186+0000 7fe66a00c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea22d5fe10 0x55ea22d81c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:10.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.186+0000 7fe66a00c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ea22d60730 0x55ea22d71d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:10.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.186+0000 7fe66a00c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea22d6afd0 0x55ea22d72270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:10.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.186+0000 7fe664fa9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea22d5fe10 0x55ea22d81c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:10.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.186+0000 7fe664fa9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea22d5fe10 0x55ea22d81c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:53268/0 (socket says 172.21.15.114:53268) 2024-09-13T18:31:10.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.186+0000 7fe664fa9640 1 -- 172.21.15.114:0/4037038110 learned_addr learned my addr 172.21.15.114:0/4037038110 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:10.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.187+0000 7fe66a00c740 1 -- 172.21.15.114:0/4037038110 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea22d57760 con 0x55ea22d60730 2024-09-13T18:31:10.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.187+0000 7fe6657aa640 1 --2- 172.21.15.114:0/4037038110 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea22d6afd0 0x55ea22d72270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:10.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.187+0000 7fe66a00c740 1 -- 172.21.15.114:0/4037038110 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea22d661a0 con 0x55ea22d6afd0 2024-09-13T18:31:10.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.187+0000 7fe66a00c740 1 -- 172.21.15.114:0/4037038110 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea22d68520 con 0x55ea22d5fe10 2024-09-13T18:31:10.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.187+0000 7fe65ffff640 1 --2- 172.21.15.114:0/4037038110 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ea22d60730 0x55ea22d71d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:10.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.187+0000 7fe664fa9640 1 -- 172.21.15.114:0/4037038110 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea22d6afd0 msgr2=0x55ea22d72270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:10.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.187+0000 7fe664fa9640 1 --2- 172.21.15.114:0/4037038110 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea22d6afd0 0x55ea22d72270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:10.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.187+0000 7fe664fa9640 1 -- 172.21.15.114:0/4037038110 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ea22d60730 msgr2=0x55ea22d71d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:10.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.187+0000 7fe664fa9640 1 --2- 172.21.15.114:0/4037038110 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ea22d60730 0x55ea22d71d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:10.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.187+0000 7fe664fa9640 1 -- 172.21.15.114:0/4037038110 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea22c64790 con 0x55ea22d5fe10 2024-09-13T18:31:10.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.188+0000 7fe664fa9640 1 --2- 172.21.15.114:0/4037038110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea22d5fe10 0x55ea22d81c30 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe650000c00 tx=0x7fe650004170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f71227c3b59c9a3a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:10.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.194+0000 7fe6477fe640 1 -- 172.21.15.114:0/4037038110 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe650042070 con 0x55ea22d5fe10 2024-09-13T18:31:10.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.194+0000 7fe6477fe640 1 -- 172.21.15.114:0/4037038110 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe650005000 con 0x55ea22d5fe10 2024-09-13T18:31:10.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.194+0000 7fe6477fe640 1 -- 172.21.15.114:0/4037038110 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe650044730 con 0x55ea22d5fe10 2024-09-13T18:31:10.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.195+0000 7fe66a00c740 1 -- 172.21.15.114:0/4037038110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea22d5fe10 msgr2=0x55ea22d81c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:10.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.195+0000 7fe66a00c740 1 --2- 172.21.15.114:0/4037038110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea22d5fe10 0x55ea22d81c30 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe650000c00 tx=0x7fe650004170 comp rx=0 tx=0).stop 2024-09-13T18:31:10.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.195+0000 7fe66a00c740 1 -- 172.21.15.114:0/4037038110 shutdown_connections 2024-09-13T18:31:10.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.195+0000 7fe66a00c740 1 --2- 172.21.15.114:0/4037038110 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea22d6afd0 0x55ea22d72270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:10.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.195+0000 7fe66a00c740 1 --2- 172.21.15.114:0/4037038110 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ea22d60730 0x55ea22d71d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:10.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.195+0000 7fe66a00c740 1 --2- 172.21.15.114:0/4037038110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea22d5fe10 0x55ea22d81c30 secure :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe650000c00 tx=0x7fe650004170 comp rx=0 tx=0).stop 2024-09-13T18:31:10.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.195+0000 7fe66a00c740 1 -- 172.21.15.114:0/4037038110 >> 172.21.15.114:0/4037038110 conn(0x55ea22d5c380 msgr2=0x55ea22d6a570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:10.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.200+0000 7fe66a00c740 1 -- 172.21.15.114:0/4037038110 shutdown_connections 2024-09-13T18:31:10.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.200+0000 7fe66a00c740 1 -- 172.21.15.114:0/4037038110 wait complete. 2024-09-13T18:31:10.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.200+0000 7fe66a00c740 1 -- start start 2024-09-13T18:31:10.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.200+0000 7fe66a00c740 1 Processor -- start 2024-09-13T18:31:10.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.201+0000 7fe66a00c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea22d60730 0x55ea22d71e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:10.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.201+0000 7fe66a00c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea22d6afd0 0x55ea22d6e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:10.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.201+0000 7fe66a00c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ea22d6e810 0x55ea22d6ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:10.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.201+0000 7fe66a00c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea22d72550 con 0x55ea22d6e810 2024-09-13T18:31:10.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.201+0000 7fe66a00c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea22dd13b0 con 0x55ea22d6afd0 2024-09-13T18:31:10.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.201+0000 7fe66a00c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea22dd14f0 con 0x55ea22d60730 2024-09-13T18:31:10.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.201+0000 7fe66a00c740 10 client.0 _ll_register_callbacks cb 0x7fe65800cf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:31:10.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.204+0000 7fe6657aa640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ea22d6e810 0x55ea22d6ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:10.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.204+0000 7fe6657aa640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ea22d6e810 0x55ea22d6ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51752/0 (socket says 172.21.15.114:51752) 2024-09-13T18:31:10.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.204+0000 7fe6657aa640 1 -- 172.21.15.114:0/1092037206 learned_addr learned my addr 172.21.15.114:0/1092037206 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:10.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.204+0000 7fe6657aa640 1 -- 172.21.15.114:0/1092037206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea22d60730 msgr2=0x55ea22d71e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:10.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.204+0000 7fe6657aa640 1 --2- 172.21.15.114:0/1092037206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea22d60730 0x55ea22d71e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:10.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.205+0000 7fe6657aa640 1 -- 172.21.15.114:0/1092037206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea22d6afd0 msgr2=0x55ea22d6e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:10.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.205+0000 7fe65ffff640 1 --2- 172.21.15.114:0/1092037206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea22d6afd0 0x55ea22d6e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:10.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.205+0000 7fe6657aa640 1 --2- 172.21.15.114:0/1092037206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea22d6afd0 0x55ea22d6e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:10.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.205+0000 7fe6657aa640 1 -- 172.21.15.114:0/1092037206 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe66011c2e0 con 0x55ea22d6e810 2024-09-13T18:31:10.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.205+0000 7fe65ffff640 1 --2- 172.21.15.114:0/1092037206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea22d6afd0 0x55ea22d6e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:31:10.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.209+0000 7fe6657aa640 1 --2- 172.21.15.114:0/1092037206 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ea22d6e810 0x55ea22d6ec90 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fe654009bb0 tx=0x7fe654015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77d33061b4cfe57c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:10.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.212+0000 7fe63ffff640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:31:10.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.212+0000 7fe63ffff640 1 -- 172.21.15.114:0/1092037206 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65402c040 con 0x55ea22d6e810 2024-09-13T18:31:10.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.212+0000 7fe63ffff640 1 -- 172.21.15.114:0/1092037206 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fe654010de0 con 0x55ea22d6e810 2024-09-13T18:31:10.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.214+0000 7fe66a00c740 1 -- 172.21.15.114:0/1092037206 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe654018070 con 0x55ea22d6e810 2024-09-13T18:31:10.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.215+0000 7fe63ffff640 1 -- 172.21.15.114:0/1092037206 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe65401d3e0 con 0x55ea22d6e810 2024-09-13T18:31:10.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.216+0000 7fe66a00c740 10 client.17982 fetch_fsmap learned FSMap version 34 2024-09-13T18:31:10.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.216+0000 7fe66a00c740 1 -- 172.21.15.114:0/1092037206 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55ea22c89ea0 con 0x55ea22d6e810 2024-09-13T18:31:10.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.216+0000 7fe63ffff640 1 -- 172.21.15.114:0/1092037206 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe654018070 con 0x55ea22d6e810 2024-09-13T18:31:10.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.217+0000 7fe63ffff640 1 -- 172.21.15.114:0/1092037206 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe65403ac20 con 0x55ea22d6e810 2024-09-13T18:31:10.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.220+0000 7fe66a00c740 10 client.17982 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:31:10.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.220+0000 7fe66a00c740 10 client.17982 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:31:10.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.220+0000 7fe66a00c740 20 client.17982 populate_metadata read hostname 'smithi114' 2024-09-13T18:31:10.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.220+0000 7fe66a00c740 1 --2- 172.21.15.114:0/1092037206 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55ea22dd2b60 0x55ea22dd4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:10.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.220+0000 7fe66a00c740 4 client.17982 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T18:31:10.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.220+0000 7fe66a00c740 1 -- 172.21.15.114:0/1092037206 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ea22a1be30 con 0x55ea22dd2b60 2024-09-13T18:31:10.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.223+0000 7fe664fa9640 1 --2- 172.21.15.114:0/1092037206 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55ea22dd2b60 0x55ea22dd4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:10.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.224+0000 7fe664fa9640 1 --2- 172.21.15.114:0/1092037206 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55ea22dd2b60 0x55ea22dd4fa0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcd9c33745db4792 server_cookie=a96bcd3a2330bbbd in_seq=0 out_seq=0 2024-09-13T18:31:10.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.224+0000 7fe63ffff640 10 client.17982 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:31:10.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.224+0000 7fe63ffff640 1 -- 172.21.15.114:0/1092037206 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55ea22a1be30 con 0x55ea22dd2b60 2024-09-13T18:31:10.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.224+0000 7fe63ffff640 10 client.17982 handle_command_reply: tid=0 2024-09-13T18:31:10.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.225+0000 7fe66a00c740 1 client.17982 shutdown 2024-09-13T18:31:10.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.233+0000 7fe66a00c740 1 -- 172.21.15.114:0/1092037206 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ea22d6e810 msgr2=0x55ea22d6ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:10.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.233+0000 7fe66a00c740 1 --2- 172.21.15.114:0/1092037206 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ea22d6e810 0x55ea22d6ec90 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fe654009bb0 tx=0x7fe654015000 comp rx=0 tx=0).stop 2024-09-13T18:31:10.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.234+0000 7fe66a00c740 1 -- 172.21.15.114:0/1092037206 shutdown_connections 2024-09-13T18:31:10.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.234+0000 7fe66a00c740 1 --2- 172.21.15.114:0/1092037206 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55ea22dd2b60 0x55ea22dd4fa0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:10.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.234+0000 7fe66a00c740 1 --2- 172.21.15.114:0/1092037206 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ea22d6e810 0x55ea22d6ec90 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:10.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.234+0000 7fe66a00c740 1 --2- 172.21.15.114:0/1092037206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea22d6afd0 0x55ea22d6e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:10.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.234+0000 7fe66a00c740 1 --2- 172.21.15.114:0/1092037206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea22d60730 0x55ea22d71e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:10.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.234+0000 7fe66a00c740 1 -- 172.21.15.114:0/1092037206 >> 172.21.15.114:0/1092037206 conn(0x55ea22d5c380 msgr2=0x55ea22d81c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:10.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.235+0000 7fe63ffff640 0 client.17982 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:31:10.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.241+0000 7fe66a00c740 1 -- 172.21.15.114:0/1092037206 shutdown_connections 2024-09-13T18:31:10.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.241+0000 7fe66a00c740 1 -- 172.21.15.114:0/1092037206 wait complete. 2024-09-13T18:31:10.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.242+0000 7fe66a00c740 20 client.17982 trim_cache size 0 max 16384 2024-09-13T18:31:10.246 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:31:10.246 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "no active scrubs running", 2024-09-13T18:31:10.246 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": {} 2024-09-13T18:31:10.246 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:31:10.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.246+0000 7fe66720d640 1 -- 172.21.15.114:0/2764678548 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe6480a1410 msgr2=0x7fe6480a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:10.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.247+0000 7fe66720d640 1 --2- 172.21.15.114:0/2764678548 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe6480a1410 0x7fe6480a38d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe65401e000 tx=0x7fe65401da40 comp rx=0 tx=0).stop 2024-09-13T18:31:10.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.247+0000 7fe66720d640 1 -- 172.21.15.114:0/2764678548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe660107920 msgr2=0x7fe660118d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:10.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.247+0000 7fe66720d640 1 --2- 172.21.15.114:0/2764678548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe660107920 0x7fe660118d60 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe65800bd70 tx=0x7fe65800be60 comp rx=0 tx=0).stop 2024-09-13T18:31:10.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.252+0000 7fe66720d640 1 -- 172.21.15.114:0/2764678548 shutdown_connections 2024-09-13T18:31:10.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.252+0000 7fe66720d640 1 --2- 172.21.15.114:0/2764678548 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe6480a1410 0x7fe6480a38d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:10.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.252+0000 7fe66720d640 1 --2- 172.21.15.114:0/2764678548 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe660119400 0x7fe660112910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:10.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.252+0000 7fe66720d640 1 --2- 172.21.15.114:0/2764678548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe660107920 0x7fe660118d60 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:10.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.252+0000 7fe66720d640 1 --2- 172.21.15.114:0/2764678548 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe660107540 0x7fe660118820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:10.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.252+0000 7fe66720d640 1 -- 172.21.15.114:0/2764678548 >> 172.21.15.114:0/2764678548 conn(0x7fe66006afe0 msgr2=0x7fe6600702b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:10.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.253+0000 7fe66720d640 1 -- 172.21.15.114:0/2764678548 shutdown_connections 2024-09-13T18:31:10.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.253+0000 7fe66720d640 1 -- 172.21.15.114:0/2764678548 wait complete. 2024-09-13T18:31:10.294 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-13T18:31:10.294 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:31:10.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.507+0000 7f3e5cbfb640 1 -- 172.21.15.114:0/429010219 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e58104180 msgr2=0x7f3e581045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:10.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.507+0000 7f3e5cbfb640 1 --2- 172.21.15.114:0/429010219 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e58104180 0x7f3e581045f0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f3e50009900 tx=0x7f3e5002a4d0 comp rx=0 tx=0).stop 2024-09-13T18:31:10.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.508+0000 7f3e5cbfb640 1 -- 172.21.15.114:0/429010219 shutdown_connections 2024-09-13T18:31:10.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.508+0000 7f3e5cbfb640 1 --2- 172.21.15.114:0/429010219 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e58104180 0x7f3e581045f0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:10.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.508+0000 7f3e5cbfb640 1 --2- 172.21.15.114:0/429010219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e580ff670 0x7f3e58103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:10.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.508+0000 7f3e5cbfb640 1 --2- 172.21.15.114:0/429010219 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e580fed50 0x7f3e580ff130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:10.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.508+0000 7f3e5cbfb640 1 -- 172.21.15.114:0/429010219 >> 172.21.15.114:0/429010219 conn(0x7f3e580fa990 msgr2=0x7f3e580fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:10.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.510+0000 7f3e5659c640 1 --2- 172.21.15.114:0/429010219 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e580fed50 0x7f3e580ff130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:31:10.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.510+0000 7f3e5cbfb640 1 -- 172.21.15.114:0/429010219 shutdown_connections 2024-09-13T18:31:10.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.517+0000 7f3e5cbfb640 1 -- 172.21.15.114:0/429010219 wait complete. 2024-09-13T18:31:10.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.518+0000 7f3e5cbfb640 1 Processor -- start 2024-09-13T18:31:10.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.518+0000 7f3e5cbfb640 1 -- start start 2024-09-13T18:31:10.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.518+0000 7f3e5cbfb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e580ff670 0x7f3e581a2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:10.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.518+0000 7f3e5cbfb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e58104180 0x7f3e581a2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:10.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.518+0000 7f3e5cbfb640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e5819c800 0x7f3e5819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:10.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.518+0000 7f3e5cbfb640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5819d2b0 con 0x7f3e58104180 2024-09-13T18:31:10.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.518+0000 7f3e5cbfb640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5819d420 con 0x7f3e580ff670 2024-09-13T18:31:10.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.518+0000 7f3e5cbfb640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5819d5e0 con 0x7f3e5819c800 2024-09-13T18:31:10.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.522+0000 7f3e5659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e580ff670 0x7f3e581a2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:10.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.523+0000 7f3e56d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e5819c800 0x7f3e5819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:10.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.523+0000 7f3e56d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e5819c800 0x7f3e5819ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:53296/0 (socket says 172.21.15.114:53296) 2024-09-13T18:31:10.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.523+0000 7f3e56d9d640 1 -- 172.21.15.114:0/469922759 learned_addr learned my addr 172.21.15.114:0/469922759 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:10.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.527+0000 7f3e55d9b640 1 --2- 172.21.15.114:0/469922759 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e58104180 0x7f3e581a2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:10.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.527+0000 7f3e56d9d640 1 -- 172.21.15.114:0/469922759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e580ff670 msgr2=0x7f3e581a2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:10.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.527+0000 7f3e56d9d640 1 --2- 172.21.15.114:0/469922759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e580ff670 0x7f3e581a2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:10.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.527+0000 7f3e56d9d640 1 -- 172.21.15.114:0/469922759 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e58104180 msgr2=0x7f3e581a2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:10.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.527+0000 7f3e56d9d640 1 --2- 172.21.15.114:0/469922759 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e58104180 0x7f3e581a2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:10.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.527+0000 7f3e56d9d640 1 -- 172.21.15.114:0/469922759 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e50009560 con 0x7f3e5819c800 2024-09-13T18:31:10.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.532+0000 7f3e56d9d640 1 --2- 172.21.15.114:0/469922759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e5819c800 0x7f3e5819ccb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3e5002aac0 tx=0x7f3e50032c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:10.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.539+0000 7f3e477fe640 1 -- 172.21.15.114:0/469922759 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e500044d0 con 0x7f3e5819c800 2024-09-13T18:31:10.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.539+0000 7f3e477fe640 1 -- 172.21.15.114:0/469922759 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3e50032e10 con 0x7f3e5819c800 2024-09-13T18:31:10.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.539+0000 7f3e477fe640 1 -- 172.21.15.114:0/469922759 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e50003850 con 0x7f3e5819c800 2024-09-13T18:31:10.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.540+0000 7f3e5cbfb640 1 -- 172.21.15.114:0/469922759 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e581a9590 con 0x7f3e5819c800 2024-09-13T18:31:10.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.540+0000 7f3e5cbfb640 1 -- 172.21.15.114:0/469922759 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e581a9a50 con 0x7f3e5819c800 2024-09-13T18:31:10.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.541+0000 7f3e5cbfb640 1 -- 172.21.15.114:0/469922759 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e58068f10 con 0x7f3e5819c800 2024-09-13T18:31:10.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.546+0000 7f3e477fe640 1 -- 172.21.15.114:0/469922759 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f3e50005210 con 0x7f3e5819c800 2024-09-13T18:31:10.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.547+0000 7f3e477fe640 1 --2- 172.21.15.114:0/469922759 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f3e400a13d0 0x7f3e400a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:10.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.547+0000 7f3e477fe640 1 -- 172.21.15.114:0/469922759 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f3e500ee250 con 0x7f3e5819c800 2024-09-13T18:31:10.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.554+0000 7f3e5659c640 1 --2- 172.21.15.114:0/469922759 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f3e400a13d0 0x7f3e400a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:10.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.554+0000 7f3e477fe640 1 -- 172.21.15.114:0/469922759 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f3e500b73e0 con 0x7f3e5819c800 2024-09-13T18:31:10.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.559+0000 7f3e5659c640 1 --2- 172.21.15.114:0/469922759 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f3e400a13d0 0x7f3e400a3890 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3e580688c0 tx=0x7f3e48009500 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:10.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.972+0000 7f3e5cbfb640 1 -- 172.21.15.114:0/469922759 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3e58103b30 con 0x7f3e5819c800 2024-09-13T18:31:10.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.975+0000 7f3e477fe640 1 -- 172.21.15.114:0/469922759 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+10930 (secure 0 0 0) 0x7f3e50040020 con 0x7f3e5819c800 2024-09-13T18:31:10.979 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:31:10.979 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4162817497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4162817497},{"type":"v1","addr":"172.21.15.114:6835","nonce":4162817497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/2435428478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":2435428478},{"type":"v1","addr":"172.21.15.114:6837","nonce":2435428478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T18:20:01.895117+0000","modified":"2024-09-13T18:27:22.172031+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24482,"mds_2":24475,"mds_3":24458,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/2046516451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":2046516451},{"type":"v1","addr":"172.21.15.114:6839","nonce":2046516451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":10,"addr":"172.21.15.114:6841/1956923446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":1956923446},{"type":"v1","addr":"172.21.15.114:6841","nonce":1956923446}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24391":{"gid":24391,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/507301496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":507301496},{"type":"v1","addr":"172.21.15.204:6833","nonce":507301496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4028657219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4028657219},{"type":"v1","addr":"172.21.15.134:6833","nonce":4028657219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/57029119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":57029119},{"type":"v1","addr":"172.21.15.204:6835","nonce":57029119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3068880939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3068880939},{"type":"v1","addr":"172.21.15.204:6837","nonce":3068880939}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"e","rank":3,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.134:6835/1150003275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1150003275},{"type":"v1","addr":"172.21.15.134:6835","nonce":1150003275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/344883912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":344883912},{"type":"v1","addr":"172.21.15.204:6839","nonce":344883912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6837/891692210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":891692210},{"type":"v1","addr":"172.21.15.134:6837","nonce":891692210}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6839/3466309724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3466309724},{"type":"v1","addr":"172.21.15.134:6839","nonce":3466309724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:31:10.979 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 34 2024-09-13T18:31:10.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.982+0000 7f3e5cbfb640 1 -- 172.21.15.114:0/469922759 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f3e400a13d0 msgr2=0x7f3e400a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:10.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.982+0000 7f3e5cbfb640 1 --2- 172.21.15.114:0/469922759 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f3e400a13d0 0x7f3e400a3890 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3e580688c0 tx=0x7f3e48009500 comp rx=0 tx=0).stop 2024-09-13T18:31:10.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.982+0000 7f3e5cbfb640 1 -- 172.21.15.114:0/469922759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e5819c800 msgr2=0x7f3e5819ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:10.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.982+0000 7f3e5cbfb640 1 --2- 172.21.15.114:0/469922759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e5819c800 0x7f3e5819ccb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3e5002aac0 tx=0x7f3e50032c20 comp rx=0 tx=0).stop 2024-09-13T18:31:10.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.990+0000 7f3e5cbfb640 1 -- 172.21.15.114:0/469922759 shutdown_connections 2024-09-13T18:31:10.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.990+0000 7f3e5cbfb640 1 --2- 172.21.15.114:0/469922759 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f3e400a13d0 0x7f3e400a3890 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:10.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.990+0000 7f3e5cbfb640 1 --2- 172.21.15.114:0/469922759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e5819c800 0x7f3e5819ccb0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:10.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.990+0000 7f3e5cbfb640 1 --2- 172.21.15.114:0/469922759 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e58104180 0x7f3e581a2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:10.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.990+0000 7f3e5cbfb640 1 --2- 172.21.15.114:0/469922759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e580ff670 0x7f3e581a2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:10.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.990+0000 7f3e5cbfb640 1 -- 172.21.15.114:0/469922759 >> 172.21.15.114:0/469922759 conn(0x7f3e580fa990 msgr2=0x7f3e580fbaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:10.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.996+0000 7f3e5cbfb640 1 -- 172.21.15.114:0/469922759 shutdown_connections 2024-09-13T18:31:11.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:10.997+0000 7f3e5cbfb640 1 -- 172.21.15.114:0/469922759 wait complete. 2024-09-13T18:31:11.025 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-09-13T18:31:11.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.255+0000 7f39fc8e8640 1 -- 172.21.15.114:0/3995716564 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39f8119400 msgr2=0x7f39f811b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:11.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.255+0000 7f39fc8e8640 1 --2- 172.21.15.114:0/3995716564 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39f8119400 0x7f39f811b7f0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f39e8009900 tx=0x7f39e802a420 comp rx=0 tx=0).stop 2024-09-13T18:31:11.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.259+0000 7f39fc8e8640 1 -- 172.21.15.114:0/3995716564 shutdown_connections 2024-09-13T18:31:11.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.259+0000 7f39fc8e8640 1 --2- 172.21.15.114:0/3995716564 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39f8119400 0x7f39f811b7f0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.259+0000 7f39fc8e8640 1 --2- 172.21.15.114:0/3995716564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39f8107890 0x7f39f8107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.259+0000 7f39fc8e8640 1 --2- 172.21.15.114:0/3995716564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39f81074b0 0x7f39f810e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.259+0000 7f39fc8e8640 1 -- 172.21.15.114:0/3995716564 >> 172.21.15.114:0/3995716564 conn(0x7f39f806af40 msgr2=0x7f39f806b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:11.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.269+0000 7f39fc8e8640 1 -- 172.21.15.114:0/3995716564 shutdown_connections 2024-09-13T18:31:11.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.273+0000 7f39fc8e8640 1 -- 172.21.15.114:0/3995716564 wait complete. 2024-09-13T18:31:11.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.273+0000 7f39fc8e8640 1 Processor -- start 2024-09-13T18:31:11.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.277+0000 7f39fc8e8640 1 -- start start 2024-09-13T18:31:11.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.277+0000 7f39fc8e8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39f81074b0 0x7f39f81a4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:11.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.277+0000 7f39fc8e8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39f8107890 0x7f39f81a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:11.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.277+0000 7f39fc8e8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39f8119400 0x7f39f81a9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:11.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.277+0000 7f39fc8e8640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f81a9e50 con 0x7f39f8107890 2024-09-13T18:31:11.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.277+0000 7f39fc8e8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f81a9fc0 con 0x7f39f8119400 2024-09-13T18:31:11.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.277+0000 7f39fc8e8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f81aa180 con 0x7f39f81074b0 2024-09-13T18:31:11.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.280+0000 7f39f659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39f8107890 0x7f39f81a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:11.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.280+0000 7f39f659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39f8107890 0x7f39f81a5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51778/0 (socket says 172.21.15.114:51778) 2024-09-13T18:31:11.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.280+0000 7f39f659c640 1 -- 172.21.15.114:0/874393657 learned_addr learned my addr 172.21.15.114:0/874393657 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:11.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.282+0000 7f39f659c640 1 -- 172.21.15.114:0/874393657 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39f81074b0 msgr2=0x7f39f81a4f20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:31:11.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.282+0000 7f39f659c640 1 --2- 172.21.15.114:0/874393657 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39f81074b0 0x7f39f81a4f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.282+0000 7f39f659c640 1 -- 172.21.15.114:0/874393657 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39f8119400 msgr2=0x7f39f81a9850 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:31:11.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.282+0000 7f39f659c640 1 --2- 172.21.15.114:0/874393657 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39f8119400 0x7f39f81a9850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.282+0000 7f39f659c640 1 -- 172.21.15.114:0/874393657 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e8009560 con 0x7f39f8107890 2024-09-13T18:31:11.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.282+0000 7f39f659c640 1 --2- 172.21.15.114:0/874393657 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39f8107890 0x7f39f81a5460 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f39ec00b450 tx=0x7f39ec00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:11.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.286+0000 7f39dffff640 1 -- 172.21.15.114:0/874393657 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ec011890 con 0x7f39f8107890 2024-09-13T18:31:11.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.286+0000 7f39dffff640 1 -- 172.21.15.114:0/874393657 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f39ec011e80 con 0x7f39f8107890 2024-09-13T18:31:11.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.286+0000 7f39dffff640 1 -- 172.21.15.114:0/874393657 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ec00f340 con 0x7f39f8107890 2024-09-13T18:31:11.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.287+0000 7f39fc8e8640 1 -- 172.21.15.114:0/874393657 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39f81aa460 con 0x7f39f8107890 2024-09-13T18:31:11.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.287+0000 7f39fc8e8640 1 -- 172.21.15.114:0/874393657 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39f81b1ce0 con 0x7f39f8107890 2024-09-13T18:31:11.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.291+0000 7f39dffff640 1 -- 172.21.15.114:0/874393657 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f39ec0103e0 con 0x7f39f8107890 2024-09-13T18:31:11.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.292+0000 7f39dffff640 1 --2- 172.21.15.114:0/874393657 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f39e00a1410 0x7f39e00a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:11.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.292+0000 7f39dffff640 1 -- 172.21.15.114:0/874393657 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f39ec0ccd10 con 0x7f39f8107890 2024-09-13T18:31:11.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.295+0000 7f39ffe09740 1 Processor -- start 2024-09-13T18:31:11.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.295+0000 7f39ffe09740 1 -- start start 2024-09-13T18:31:11.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.295+0000 7f39ffe09740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556263c15970 0x556263c15d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:11.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.295+0000 7f39ffe09740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556263c16290 0x556263c205f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:11.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.295+0000 7f39ffe09740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556263c20b30 0x556263c22f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:11.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.295+0000 7f39ffe09740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x556263b7fcc0 con 0x556263c20b30 2024-09-13T18:31:11.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.295+0000 7f39ffe09740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556263b7d450 con 0x556263c15970 2024-09-13T18:31:11.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.295+0000 7f39ffe09740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556263b78d40 con 0x556263c16290 2024-09-13T18:31:11.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.295+0000 7f39f759e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556263c20b30 0x556263c22f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:11.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.296+0000 7f39f759e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556263c20b30 0x556263c22f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51784/0 (socket says 172.21.15.114:51784) 2024-09-13T18:31:11.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.296+0000 7f39f6d9d640 1 --2- 172.21.15.114:0/874393657 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f39e00a1410 0x7f39e00a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:11.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.296+0000 7f39f759e640 1 -- 172.21.15.114:0/3344694693 learned_addr learned my addr 172.21.15.114:0/3344694693 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:11.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.296+0000 7f39f759e640 1 -- 172.21.15.114:0/3344694693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556263c16290 msgr2=0x556263c205f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:31:11.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.296+0000 7f39f759e640 1 --2- 172.21.15.114:0/3344694693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556263c16290 0x556263c205f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.296+0000 7f39f759e640 1 -- 172.21.15.114:0/3344694693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556263c15970 msgr2=0x556263c15d50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:31:11.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.296+0000 7f39f759e640 1 --2- 172.21.15.114:0/3344694693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556263c15970 0x556263c15d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.296+0000 7f39f759e640 1 -- 172.21.15.114:0/3344694693 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556263b40620 con 0x556263c20b30 2024-09-13T18:31:11.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.296+0000 7f39f759e640 1 --2- 172.21.15.114:0/3344694693 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556263c20b30 0x556263c22f20 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f39e802aa10 tx=0x7f39e8004180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6b28fa4af22c3e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:11.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.297+0000 7f39dd7fa640 1 -- 172.21.15.114:0/3344694693 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e80042f0 con 0x556263c20b30 2024-09-13T18:31:11.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.297+0000 7f39dd7fa640 1 -- 172.21.15.114:0/3344694693 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f39e8004450 con 0x556263c20b30 2024-09-13T18:31:11.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.298+0000 7f39ffe09740 1 -- 172.21.15.114:0/3344694693 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556263c20b30 msgr2=0x556263c22f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:11.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.298+0000 7f39ffe09740 1 --2- 172.21.15.114:0/3344694693 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556263c20b30 0x556263c22f20 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f39e802aa10 tx=0x7f39e8004180 comp rx=0 tx=0).stop 2024-09-13T18:31:11.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.298+0000 7f39ffe09740 1 -- 172.21.15.114:0/3344694693 shutdown_connections 2024-09-13T18:31:11.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.298+0000 7f39ffe09740 1 --2- 172.21.15.114:0/3344694693 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556263c20b30 0x556263c22f20 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.298+0000 7f39ffe09740 1 --2- 172.21.15.114:0/3344694693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556263c16290 0x556263c205f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.298+0000 7f39ffe09740 1 --2- 172.21.15.114:0/3344694693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556263c15970 0x556263c15d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.298+0000 7f39ffe09740 1 -- 172.21.15.114:0/3344694693 >> 172.21.15.114:0/3344694693 conn(0x556263c11ee0 msgr2=0x556263c122c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:11.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.300+0000 7f39ffe09740 1 -- 172.21.15.114:0/3344694693 shutdown_connections 2024-09-13T18:31:11.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.300+0000 7f39ffe09740 1 -- 172.21.15.114:0/3344694693 wait complete. 2024-09-13T18:31:11.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.301+0000 7f39ffe09740 1 -- start start 2024-09-13T18:31:11.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.301+0000 7f39ffe09740 1 Processor -- start 2024-09-13T18:31:11.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.301+0000 7f39ffe09740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556263c16290 0x556263c201e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:11.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.301+0000 7f39ffe09740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556263c20b30 0x556263c1d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:11.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.302+0000 7f39ffe09740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556263c1d9f0 0x556263c1de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:11.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.302+0000 7f39ffe09740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x556263b37180 con 0x556263c16290 2024-09-13T18:31:11.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.302+0000 7f39ffe09740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556263851430 con 0x556263c1d9f0 2024-09-13T18:31:11.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.302+0000 7f39ffe09740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556263afc5e0 con 0x556263c20b30 2024-09-13T18:31:11.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.302+0000 7f39ffe09740 10 client.0 _ll_register_callbacks cb 0x556263b68aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:31:11.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.302+0000 7f39f759e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556263c1d9f0 0x556263c1de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:11.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.302+0000 7f39f759e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556263c1d9f0 0x556263c1de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:45808/0 (socket says 172.21.15.114:45808) 2024-09-13T18:31:11.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.302+0000 7f39f759e640 1 -- 172.21.15.114:0/125837369 learned_addr learned my addr 172.21.15.114:0/125837369 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:11.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.302+0000 7f39f659c640 1 --2- 172.21.15.114:0/125837369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556263c20b30 0x556263c1d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:11.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.302+0000 7f39f6d9d640 1 --2- 172.21.15.114:0/125837369 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556263c16290 0x556263c201e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:11.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.302+0000 7f39f759e640 1 -- 172.21.15.114:0/125837369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556263c20b30 msgr2=0x556263c1d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:11.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.302+0000 7f39f759e640 1 --2- 172.21.15.114:0/125837369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556263c20b30 0x556263c1d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.302+0000 7f39f759e640 1 -- 172.21.15.114:0/125837369 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556263c16290 msgr2=0x556263c201e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:11.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.302+0000 7f39f759e640 1 --2- 172.21.15.114:0/125837369 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556263c16290 0x556263c201e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.302+0000 7f39f759e640 1 -- 172.21.15.114:0/125837369 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39f811c170 con 0x556263c1d9f0 2024-09-13T18:31:11.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.303+0000 7f39f759e640 1 --2- 172.21.15.114:0/125837369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556263c1d9f0 0x556263c1de70 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f39e8009900 tx=0x7f39e803ce20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e3570e8543a5fd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:11.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.304+0000 7f39f6d9d640 1 --2- 172.21.15.114:0/874393657 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f39e00a1410 0x7f39e00a38d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x556263c13180 tx=0x7f39f0007b00 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:11.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.306+0000 7f39daff5640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:31:11.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.306+0000 7f39daff5640 1 -- 172.21.15.114:0/125837369 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e80038c0 con 0x556263c1d9f0 2024-09-13T18:31:11.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.306+0000 7f39daff5640 1 -- 172.21.15.114:0/125837369 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f39e8003a20 con 0x556263c1d9f0 2024-09-13T18:31:11.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.306+0000 7f39ffe09740 1 -- 172.21.15.114:0/125837369 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556263ba2b80 con 0x556263c1d9f0 2024-09-13T18:31:11.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.307+0000 7f39daff5640 1 -- 172.21.15.114:0/125837369 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39e8003be0 con 0x556263c1d9f0 2024-09-13T18:31:11.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.307+0000 7f39ffe09740 10 client.27416 fetch_fsmap learned FSMap version 34 2024-09-13T18:31:11.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.307+0000 7f39ffe09740 1 -- 172.21.15.114:0/125837369 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x556263b773c0 con 0x556263c1d9f0 2024-09-13T18:31:11.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.307+0000 7f39daff5640 1 -- 172.21.15.114:0/125837369 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f39e8056080 con 0x556263c1d9f0 2024-09-13T18:31:11.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.307+0000 7f39daff5640 1 -- 172.21.15.114:0/125837369 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f39e8044410 con 0x556263c1d9f0 2024-09-13T18:31:11.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.307+0000 7f39ffe09740 10 client.27416 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:31:11.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.307+0000 7f39ffe09740 10 client.27416 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-13T18:31:11.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.307+0000 7f39ffe09740 20 client.27416 populate_metadata read hostname 'smithi114' 2024-09-13T18:31:11.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.307+0000 7f39ffe09740 1 --2- 172.21.15.114:0/125837369 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x556263c416d0 0x556263c43af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:11.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.307+0000 7f39ffe09740 4 client.27416 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:31:11.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.307+0000 7f39ffe09740 1 -- 172.21.15.114:0/125837369 --> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556263b6b1f0 con 0x556263c416d0 2024-09-13T18:31:11.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.310+0000 7f39f6d9d640 1 --2- 172.21.15.114:0/125837369 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x556263c416d0 0x556263c43af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:11.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.316+0000 7f39f6d9d640 1 --2- 172.21.15.114:0/125837369 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x556263c416d0 0x556263c43af0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=afd5bad632664569 server_cookie=5e09fee1cc52ba63 in_seq=0 out_seq=0 2024-09-13T18:31:11.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.316+0000 7f39daff5640 10 client.27416 ms_handle_connect on v2:172.21.15.114:6840/1956923446 2024-09-13T18:31:11.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.318+0000 7f39daff5640 1 -- 172.21.15.114:0/125837369 <== mds.4 v2:172.21.15.114:6840/1956923446 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x556263b6b1f0 con 0x556263c416d0 2024-09-13T18:31:11.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.318+0000 7f39daff5640 10 client.27416 handle_command_reply: tid=0 2024-09-13T18:31:11.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.318+0000 7f39ffe09740 1 client.27416 shutdown 2024-09-13T18:31:11.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.324+0000 7f39ffe09740 1 -- 172.21.15.114:0/125837369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556263c1d9f0 msgr2=0x556263c1de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:11.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.324+0000 7f39ffe09740 1 --2- 172.21.15.114:0/125837369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556263c1d9f0 0x556263c1de70 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f39e8009900 tx=0x7f39e803ce20 comp rx=0 tx=0).stop 2024-09-13T18:31:11.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.327+0000 7f39ffe09740 1 -- 172.21.15.114:0/125837369 shutdown_connections 2024-09-13T18:31:11.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.327+0000 7f39ffe09740 1 --2- 172.21.15.114:0/125837369 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x556263c416d0 0x556263c43af0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.327+0000 7f39ffe09740 1 --2- 172.21.15.114:0/125837369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556263c1d9f0 0x556263c1de70 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.327+0000 7f39ffe09740 1 --2- 172.21.15.114:0/125837369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556263c20b30 0x556263c1d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.327+0000 7f39ffe09740 1 --2- 172.21.15.114:0/125837369 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556263c16290 0x556263c201e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.327+0000 7f39ffe09740 1 -- 172.21.15.114:0/125837369 >> 172.21.15.114:0/125837369 conn(0x556263c11ee0 msgr2=0x556263c11230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:11.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.329+0000 7f39daff5640 0 client.27416 ms_handle_reset on v2:172.21.15.114:6840/1956923446 2024-09-13T18:31:11.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.332+0000 7f39ffe09740 1 -- 172.21.15.114:0/125837369 shutdown_connections 2024-09-13T18:31:11.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.334+0000 7f39ffe09740 1 -- 172.21.15.114:0/125837369 wait complete. 2024-09-13T18:31:11.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.343+0000 7f39ffe09740 20 client.27416 trim_cache size 0 max 16384 2024-09-13T18:31:11.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.367+0000 7f39ffe09740 1 Processor -- start 2024-09-13T18:31:11.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.368+0000 7f39ffe09740 1 -- start start 2024-09-13T18:31:11.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.368+0000 7f39ffe09740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556263c16290 0x556263c48450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:11.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.368+0000 7f39ffe09740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556263c1d9f0 0x556263c1cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:11.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.368+0000 7f39ffe09740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556263c20b30 0x556263c1d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:11.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.368+0000 7f39ffe09740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x556263c16b10 con 0x556263c16290 2024-09-13T18:31:11.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.368+0000 7f39ffe09740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556263c1e8d0 con 0x556263c20b30 2024-09-13T18:31:11.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.368+0000 7f39ffe09740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556263c1e450 con 0x556263c1d9f0 2024-09-13T18:31:11.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.368+0000 7f39f759e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556263c20b30 0x556263c1d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:11.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.368+0000 7f39f659c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556263c1d9f0 0x556263c1cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:11.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.368+0000 7f39f659c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556263c1d9f0 0x556263c1cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:53348/0 (socket says 172.21.15.114:53348) 2024-09-13T18:31:11.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.368+0000 7f39f659c640 1 -- 172.21.15.114:0/2129906426 learned_addr learned my addr 172.21.15.114:0/2129906426 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:11.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.375+0000 7f39f659c640 1 -- 172.21.15.114:0/2129906426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556263c20b30 msgr2=0x556263c1d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:11.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.375+0000 7f39f659c640 1 --2- 172.21.15.114:0/2129906426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556263c20b30 0x556263c1d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.375+0000 7f39f659c640 1 -- 172.21.15.114:0/2129906426 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556263c16290 msgr2=0x556263c48450 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:31:11.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.375+0000 7f39f659c640 1 --2- 172.21.15.114:0/2129906426 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556263c16290 0x556263c48450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.375+0000 7f39f659c640 1 -- 172.21.15.114:0/2129906426 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556263b19770 con 0x556263c1d9f0 2024-09-13T18:31:11.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.376+0000 7f39f659c640 1 --2- 172.21.15.114:0/2129906426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556263c1d9f0 0x556263c1cd50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f39ec095fd0 tx=0x7f39ec011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ca70ef9a4f65c70 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:11.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.377+0000 7f39ddffb640 1 -- 172.21.15.114:0/2129906426 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ec0a1f50 con 0x556263c1d9f0 2024-09-13T18:31:11.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.377+0000 7f39ddffb640 1 -- 172.21.15.114:0/2129906426 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f39ec0961e0 con 0x556263c1d9f0 2024-09-13T18:31:11.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.377+0000 7f39ddffb640 1 -- 172.21.15.114:0/2129906426 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ec0b2470 con 0x556263c1d9f0 2024-09-13T18:31:11.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.378+0000 7f39ffe09740 1 -- 172.21.15.114:0/2129906426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556263c1d9f0 msgr2=0x556263c1cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:11.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.378+0000 7f39ffe09740 1 --2- 172.21.15.114:0/2129906426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556263c1d9f0 0x556263c1cd50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f39ec095fd0 tx=0x7f39ec011000 comp rx=0 tx=0).stop 2024-09-13T18:31:11.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.382+0000 7f39ffe09740 1 -- 172.21.15.114:0/2129906426 shutdown_connections 2024-09-13T18:31:11.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.382+0000 7f39ffe09740 1 --2- 172.21.15.114:0/2129906426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556263c20b30 0x556263c1d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.382+0000 7f39ffe09740 1 --2- 172.21.15.114:0/2129906426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556263c1d9f0 0x556263c1cd50 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.382+0000 7f39ffe09740 1 --2- 172.21.15.114:0/2129906426 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556263c16290 0x556263c48450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.382+0000 7f39ffe09740 1 -- 172.21.15.114:0/2129906426 >> 172.21.15.114:0/2129906426 conn(0x556263c11ee0 msgr2=0x556263c436e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:11.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.387+0000 7f39ffe09740 1 -- 172.21.15.114:0/2129906426 shutdown_connections 2024-09-13T18:31:11.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.387+0000 7f39ffe09740 1 -- 172.21.15.114:0/2129906426 wait complete. 2024-09-13T18:31:11.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.387+0000 7f39ffe09740 1 -- start start 2024-09-13T18:31:11.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.387+0000 7f39ffe09740 1 Processor -- start 2024-09-13T18:31:11.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.388+0000 7f39ffe09740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556263c16290 0x556263c38a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:11.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.388+0000 7f39ffe09740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556263c1d9f0 0x556263c34f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:11.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.388+0000 7f39ffe09740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556263c20b30 0x556263c35470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:11.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.388+0000 7f39f659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556263c1d9f0 0x556263c34f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:11.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.388+0000 7f39f659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556263c1d9f0 0x556263c34f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:45820/0 (socket says 172.21.15.114:45820) 2024-09-13T18:31:11.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.389+0000 7f39f659c640 1 -- 172.21.15.114:0/3665643941 learned_addr learned my addr 172.21.15.114:0/3665643941 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:11.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.389+0000 7f39ffe09740 1 -- 172.21.15.114:0/3665643941 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x556263c359b0 con 0x556263c20b30 2024-09-13T18:31:11.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.389+0000 7f39ffe09740 1 -- 172.21.15.114:0/3665643941 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556263c35b20 con 0x556263c1d9f0 2024-09-13T18:31:11.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.389+0000 7f39ffe09740 1 -- 172.21.15.114:0/3665643941 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556263c35ce0 con 0x556263c16290 2024-09-13T18:31:11.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.389+0000 7f39ffe09740 10 client.0 _ll_register_callbacks cb 0x7f39e8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:31:11.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.390+0000 7f39f659c640 1 -- 172.21.15.114:0/3665643941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556263c16290 msgr2=0x556263c38a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:11.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.390+0000 7f39f659c640 1 --2- 172.21.15.114:0/3665643941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556263c16290 0x556263c38a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.390+0000 7f39f659c640 1 -- 172.21.15.114:0/3665643941 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556263c20b30 msgr2=0x556263c35470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:11.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.390+0000 7f39f659c640 1 --2- 172.21.15.114:0/3665643941 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556263c20b30 0x556263c35470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.390+0000 7f39f659c640 1 -- 172.21.15.114:0/3665643941 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39f81b1ce0 con 0x556263c1d9f0 2024-09-13T18:31:11.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.394+0000 7f39f659c640 1 --2- 172.21.15.114:0/3665643941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556263c1d9f0 0x556263c34f30 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f39ec000c00 tx=0x7f39ec098080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7314aff0da04ecf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:11.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.395+0000 7f39daff5640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:31:11.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.395+0000 7f39daff5640 1 -- 172.21.15.114:0/3665643941 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ec0a1cc0 con 0x556263c1d9f0 2024-09-13T18:31:11.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.395+0000 7f39daff5640 1 -- 172.21.15.114:0/3665643941 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f39ec0a1f50 con 0x556263c1d9f0 2024-09-13T18:31:11.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.395+0000 7f39ffe09740 1 -- 172.21.15.114:0/3665643941 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f39e8004450 con 0x556263c1d9f0 2024-09-13T18:31:11.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.401+0000 7f39daff5640 1 -- 172.21.15.114:0/3665643941 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39ec0119f0 con 0x556263c1d9f0 2024-09-13T18:31:11.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.402+0000 7f39ffe09740 10 client.27422 fetch_fsmap learned FSMap version 34 2024-09-13T18:31:11.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.402+0000 7f39ffe09740 1 -- 172.21.15.114:0/3665643941 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x556263b3ee80 con 0x556263c1d9f0 2024-09-13T18:31:11.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.409+0000 7f39daff5640 1 -- 172.21.15.114:0/3665643941 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f39e8004450 con 0x556263c1d9f0 2024-09-13T18:31:11.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.409+0000 7f39daff5640 1 -- 172.21.15.114:0/3665643941 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f39ec09aae0 con 0x556263c1d9f0 2024-09-13T18:31:11.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.409+0000 7f39ffe09740 10 client.27422 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:31:11.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.409+0000 7f39ffe09740 10 client.27422 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-13T18:31:11.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.409+0000 7f39ffe09740 20 client.27422 populate_metadata read hostname 'smithi114' 2024-09-13T18:31:11.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.409+0000 7f39ffe09740 1 --2- 172.21.15.114:0/3665643941 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x556263c98ca0 0x556263c9b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:11.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.409+0000 7f39ffe09740 4 client.27422 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:31:11.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.409+0000 7f39ffe09740 1 -- 172.21.15.114:0/3665643941 --> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55626386c4e0 con 0x556263c98ca0 2024-09-13T18:31:11.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.413+0000 7f39f6d9d640 1 --2- 172.21.15.114:0/3665643941 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x556263c98ca0 0x556263c9b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:11.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.414+0000 7f39daff5640 10 client.27422 ms_handle_connect on v2:172.21.15.114:6840/1956923446 2024-09-13T18:31:11.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.418+0000 7f39f6d9d640 1 --2- 172.21.15.114:0/3665643941 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x556263c98ca0 0x556263c9b0c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bc6877eb6b92fb06 server_cookie=e9c4afafb8c55927 in_seq=0 out_seq=0 2024-09-13T18:31:11.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.422+0000 7f39daff5640 1 -- 172.21.15.114:0/3665643941 <== mds.4 v2:172.21.15.114:6840/1956923446 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55626386c4e0 con 0x556263c98ca0 2024-09-13T18:31:11.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.422+0000 7f39daff5640 10 client.27422 handle_command_reply: tid=0 2024-09-13T18:31:11.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.422+0000 7f39ffe09740 1 client.27422 shutdown 2024-09-13T18:31:11.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.434+0000 7f39ffe09740 1 -- 172.21.15.114:0/3665643941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556263c1d9f0 msgr2=0x556263c34f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:11.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.434+0000 7f39ffe09740 1 --2- 172.21.15.114:0/3665643941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556263c1d9f0 0x556263c34f30 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f39ec000c00 tx=0x7f39ec098080 comp rx=0 tx=0).stop 2024-09-13T18:31:11.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.434+0000 7f39ffe09740 1 -- 172.21.15.114:0/3665643941 shutdown_connections 2024-09-13T18:31:11.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.434+0000 7f39ffe09740 1 --2- 172.21.15.114:0/3665643941 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x556263c98ca0 0x556263c9b0c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.434+0000 7f39ffe09740 1 --2- 172.21.15.114:0/3665643941 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556263c20b30 0x556263c35470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.434+0000 7f39ffe09740 1 --2- 172.21.15.114:0/3665643941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556263c1d9f0 0x556263c34f30 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.434+0000 7f39ffe09740 1 --2- 172.21.15.114:0/3665643941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556263c16290 0x556263c38a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.434+0000 7f39ffe09740 1 -- 172.21.15.114:0/3665643941 >> 172.21.15.114:0/3665643941 conn(0x556263c11ee0 msgr2=0x556263c48450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:11.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.434+0000 7f39daff5640 0 client.27422 ms_handle_reset on v2:172.21.15.114:6840/1956923446 2024-09-13T18:31:11.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.437+0000 7f39ffe09740 1 -- 172.21.15.114:0/3665643941 shutdown_connections 2024-09-13T18:31:11.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.437+0000 7f39ffe09740 1 -- 172.21.15.114:0/3665643941 wait complete. 2024-09-13T18:31:11.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.440+0000 7f39ffe09740 20 client.27422 trim_cache size 0 max 16384 2024-09-13T18:31:11.442 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:31:11.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.448+0000 7f39fc8e8640 1 -- 172.21.15.114:0/874393657 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f39e00a1410 msgr2=0x7f39e00a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:11.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.448+0000 7f39fc8e8640 1 --2- 172.21.15.114:0/874393657 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f39e00a1410 0x7f39e00a38d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x556263c13180 tx=0x7f39f0007b00 comp rx=0 tx=0).stop 2024-09-13T18:31:11.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.448+0000 7f39fc8e8640 1 -- 172.21.15.114:0/874393657 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39f8107890 msgr2=0x7f39f81a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:11.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.448+0000 7f39fc8e8640 1 --2- 172.21.15.114:0/874393657 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39f8107890 0x7f39f81a5460 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f39ec00b450 tx=0x7f39ec00b910 comp rx=0 tx=0).stop 2024-09-13T18:31:11.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.448+0000 7f39fc8e8640 1 -- 172.21.15.114:0/874393657 shutdown_connections 2024-09-13T18:31:11.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.448+0000 7f39fc8e8640 1 --2- 172.21.15.114:0/874393657 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f39e00a1410 0x7f39e00a38d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.448+0000 7f39fc8e8640 1 --2- 172.21.15.114:0/874393657 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39f8119400 0x7f39f81a9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.448+0000 7f39fc8e8640 1 --2- 172.21.15.114:0/874393657 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39f8107890 0x7f39f81a5460 secure :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f39ec00b450 tx=0x7f39ec00b910 comp rx=0 tx=0).stop 2024-09-13T18:31:11.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.448+0000 7f39fc8e8640 1 --2- 172.21.15.114:0/874393657 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39f81074b0 0x7f39f81a4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.448+0000 7f39fc8e8640 1 -- 172.21.15.114:0/874393657 >> 172.21.15.114:0/874393657 conn(0x7f39f806af40 msgr2=0x7f39f8119d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:11.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.456+0000 7f39fc8e8640 1 -- 172.21.15.114:0/874393657 shutdown_connections 2024-09-13T18:31:11.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.463+0000 7f39fc8e8640 1 -- 172.21.15.114:0/874393657 wait complete. 2024-09-13T18:31:11.512 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-09-13T18:31:11.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.757+0000 7f4c7ffff640 1 -- 172.21.15.114:0/2155689521 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4c78106340 msgr2=0x7f4c78106780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:11.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.757+0000 7f4c7ffff640 1 --2- 172.21.15.114:0/2155689521 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4c78106340 0x7f4c78106780 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c009cd0 tx=0x7f4c6c02a9c0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.760+0000 7f4c7ffff640 1 -- 172.21.15.114:0/2155689521 shutdown_connections 2024-09-13T18:31:11.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.760+0000 7f4c7ffff640 1 --2- 172.21.15.114:0/2155689521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c78106cc0 0x7f4c781131a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.760+0000 7f4c7ffff640 1 --2- 172.21.15.114:0/2155689521 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4c78106340 0x7f4c78106780 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.760+0000 7f4c7ffff640 1 --2- 172.21.15.114:0/2155689521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c7810c400 0x7f4c7810c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.760+0000 7f4c7ffff640 1 -- 172.21.15.114:0/2155689521 >> 172.21.15.114:0/2155689521 conn(0x7f4c78075c80 msgr2=0x7f4c780780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:11.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.768+0000 7f4c7ffff640 1 -- 172.21.15.114:0/2155689521 shutdown_connections 2024-09-13T18:31:11.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.768+0000 7f4c7ffff640 1 -- 172.21.15.114:0/2155689521 wait complete. 2024-09-13T18:31:11.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.769+0000 7f4c7ffff640 1 Processor -- start 2024-09-13T18:31:11.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.770+0000 7f4c7ffff640 1 -- start start 2024-09-13T18:31:11.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.770+0000 7f4c7ffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c78106340 0x7f4c7819c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:11.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.770+0000 7f4c7ffff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4c78106cc0 0x7f4c7819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:11.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.770+0000 7f4c7ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c7810c400 0x7f4c7819ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:11.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.770+0000 7f4c7ffff640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c781a0590 con 0x7f4c78106cc0 2024-09-13T18:31:11.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.770+0000 7f4c7ffff640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c781a0700 con 0x7f4c7810c400 2024-09-13T18:31:11.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.770+0000 7f4c7ffff640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c781a08c0 con 0x7f4c78106340 2024-09-13T18:31:11.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.770+0000 7f4c7e59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c7810c400 0x7f4c7819ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:11.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.770+0000 7f4c7e59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c7810c400 0x7f4c7819ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:45838/0 (socket says 172.21.15.114:45838) 2024-09-13T18:31:11.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.770+0000 7f4c7e59c640 1 -- 172.21.15.114:0/3819137730 learned_addr learned my addr 172.21.15.114:0/3819137730 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:11.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.773+0000 7f4c7dd9b640 1 --2- 172.21.15.114:0/3819137730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c78106340 0x7f4c7819c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:11.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.774+0000 7f4c7e59c640 1 -- 172.21.15.114:0/3819137730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c78106340 msgr2=0x7f4c7819c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:11.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.774+0000 7f4c7e59c640 1 --2- 172.21.15.114:0/3819137730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c78106340 0x7f4c7819c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.774+0000 7f4c7e59c640 1 -- 172.21.15.114:0/3819137730 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4c78106cc0 msgr2=0x7f4c7819cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:11.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.774+0000 7f4c7e59c640 1 --2- 172.21.15.114:0/3819137730 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4c78106cc0 0x7f4c7819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.774+0000 7f4c7e59c640 1 -- 172.21.15.114:0/3819137730 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c6c009900 con 0x7f4c7810c400 2024-09-13T18:31:11.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.785+0000 7f4c7e59c640 1 --2- 172.21.15.114:0/3819137730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c7810c400 0x7f4c7819ff90 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4c6400b3f0 tx=0x7f4c6400b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:11.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.786+0000 7f4c76ffd640 1 -- 172.21.15.114:0/3819137730 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c64011830 con 0x7f4c7810c400 2024-09-13T18:31:11.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.786+0000 7f4c76ffd640 1 -- 172.21.15.114:0/3819137730 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4c64011e20 con 0x7f4c7810c400 2024-09-13T18:31:11.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.786+0000 7f4c76ffd640 1 -- 172.21.15.114:0/3819137730 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c64010e10 con 0x7f4c7810c400 2024-09-13T18:31:11.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.787+0000 7f4c7ffff640 1 -- 172.21.15.114:0/3819137730 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c781a0b40 con 0x7f4c7810c400 2024-09-13T18:31:11.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.787+0000 7f4c7ffff640 1 -- 172.21.15.114:0/3819137730 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c7806c0c0 con 0x7f4c7810c400 2024-09-13T18:31:11.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.794+0000 7f4c82dfc740 1 Processor -- start 2024-09-13T18:31:11.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.794+0000 7f4c82dfc740 1 -- start start 2024-09-13T18:31:11.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.794+0000 7f4c82dfc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1778b8790 0x55c1778b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:11.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.794+0000 7f4c82dfc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c1778b90b0 0x55c1778c3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:11.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.794+0000 7f4c82dfc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1778c3950 0x55c1778c5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:11.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.794+0000 7f4c82dfc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c177822cc0 con 0x55c1778b90b0 2024-09-13T18:31:11.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.794+0000 7f4c82dfc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c177820450 con 0x55c1778b8790 2024-09-13T18:31:11.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.794+0000 7f4c82dfc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c17781bd40 con 0x55c1778c3950 2024-09-13T18:31:11.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.795+0000 7f4c7d59a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1778b8790 0x55c1778b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:11.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.795+0000 7f4c7d59a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1778c3950 0x55c1778c5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:11.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.795+0000 7f4c7dd9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c1778b90b0 0x55c1778c3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:11.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.795+0000 7f4c7dd9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c1778b90b0 0x55c1778c3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51826/0 (socket says 172.21.15.114:51826) 2024-09-13T18:31:11.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.795+0000 7f4c7dd9b640 1 -- 172.21.15.114:0/1362998663 learned_addr learned my addr 172.21.15.114:0/1362998663 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:11.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.796+0000 7f4c76ffd640 1 -- 172.21.15.114:0/3819137730 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f4c64010380 con 0x7f4c7810c400 2024-09-13T18:31:11.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.796+0000 7f4c7d59a640 1 -- 172.21.15.114:0/1362998663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1778c3950 msgr2=0x55c1778c5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:11.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.796+0000 7f4c7d59a640 1 --2- 172.21.15.114:0/1362998663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1778c3950 0x55c1778c5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.796+0000 7f4c7d59a640 1 -- 172.21.15.114:0/1362998663 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c1778b90b0 msgr2=0x55c1778c3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:11.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.796+0000 7f4c7d59a640 1 --2- 172.21.15.114:0/1362998663 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c1778b90b0 0x55c1778c3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.796+0000 7f4c7d59a640 1 -- 172.21.15.114:0/1362998663 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1777e3620 con 0x55c1778b8790 2024-09-13T18:31:11.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.797+0000 7f4c76ffd640 1 --2- 172.21.15.114:0/3819137730 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f4c680a1410 0x7f4c680a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:11.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.797+0000 7f4c7d59a640 1 --2- 172.21.15.114:0/1362998663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1778b8790 0x55c1778b8b70 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c02af30 tx=0x7f4c6c02aec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6ad3d4fd4761fa3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:11.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.798+0000 7f4c76ffd640 1 -- 172.21.15.114:0/3819137730 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f4c640cc9a0 con 0x7f4c7810c400 2024-09-13T18:31:11.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.798+0000 7f4c63fff640 1 -- 172.21.15.114:0/1362998663 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c02f5d0 con 0x55c1778b8790 2024-09-13T18:31:11.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.801+0000 7f4c63fff640 1 -- 172.21.15.114:0/1362998663 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4c6c030b10 con 0x55c1778b8790 2024-09-13T18:31:11.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.801+0000 7f4c63fff640 1 -- 172.21.15.114:0/1362998663 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c02e840 con 0x55c1778b8790 2024-09-13T18:31:11.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.801+0000 7f4c7dd9b640 1 --2- 172.21.15.114:0/3819137730 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f4c680a1410 0x7f4c680a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:11.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.802+0000 7f4c82dfc740 1 -- 172.21.15.114:0/1362998663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1778b8790 msgr2=0x55c1778b8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:11.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.802+0000 7f4c82dfc740 1 --2- 172.21.15.114:0/1362998663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1778b8790 0x55c1778b8b70 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c02af30 tx=0x7f4c6c02aec0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.813+0000 7f4c7dd9b640 1 --2- 172.21.15.114:0/3819137730 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f4c680a1410 0x7f4c680a38d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4c700058c0 tx=0x7f4c70005850 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:11.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.815+0000 7f4c82dfc740 1 -- 172.21.15.114:0/1362998663 shutdown_connections 2024-09-13T18:31:11.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.815+0000 7f4c82dfc740 1 --2- 172.21.15.114:0/1362998663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1778c3950 0x55c1778c5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.815+0000 7f4c82dfc740 1 --2- 172.21.15.114:0/1362998663 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c1778b90b0 0x55c1778c3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.816+0000 7f4c82dfc740 1 --2- 172.21.15.114:0/1362998663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1778b8790 0x55c1778b8b70 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.816+0000 7f4c82dfc740 1 -- 172.21.15.114:0/1362998663 >> 172.21.15.114:0/1362998663 conn(0x55c1778b4d00 msgr2=0x55c1778b50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:11.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.820+0000 7f4c82dfc740 1 -- 172.21.15.114:0/1362998663 shutdown_connections 2024-09-13T18:31:11.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.824+0000 7f4c82dfc740 1 -- 172.21.15.114:0/1362998663 wait complete. 2024-09-13T18:31:11.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.824+0000 7f4c82dfc740 1 -- start start 2024-09-13T18:31:11.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.824+0000 7f4c82dfc740 1 Processor -- start 2024-09-13T18:31:11.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.829+0000 7f4c82dfc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c1778b8790 0x55c1778c2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:11.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.829+0000 7f4c82dfc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1778b90b0 0x55c1778c02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:11.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.829+0000 7f4c82dfc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1778c3950 0x55c1778c07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:11.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.829+0000 7f4c82dfc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1777da180 con 0x55c1778b8790 2024-09-13T18:31:11.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.829+0000 7f4c82dfc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1774f4430 con 0x55c1778c3950 2024-09-13T18:31:11.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.829+0000 7f4c82dfc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c17779f5e0 con 0x55c1778b90b0 2024-09-13T18:31:11.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.829+0000 7f4c82dfc740 10 client.0 _ll_register_callbacks cb 0x55c17780baa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:31:11.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.833+0000 7f4c7dd9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c1778b8790 0x55c1778c2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:11.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.833+0000 7f4c7d59a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1778b90b0 0x55c1778c02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:11.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.833+0000 7f4c7d59a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1778b90b0 0x55c1778c02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:53390/0 (socket says 172.21.15.114:53390) 2024-09-13T18:31:11.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.833+0000 7f4c7d59a640 1 -- 172.21.15.114:0/1231441610 learned_addr learned my addr 172.21.15.114:0/1231441610 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:11.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.835+0000 7f4c7d59a640 1 -- 172.21.15.114:0/1231441610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1778c3950 msgr2=0x55c1778c07e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:31:11.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.835+0000 7f4c7d59a640 1 --2- 172.21.15.114:0/1231441610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1778c3950 0x55c1778c07e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.835+0000 7f4c7d59a640 1 -- 172.21.15.114:0/1231441610 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c1778b8790 msgr2=0x55c1778c2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:11.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.835+0000 7f4c7d59a640 1 --2- 172.21.15.114:0/1231441610 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c1778b8790 0x55c1778c2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.835+0000 7f4c7d59a640 1 -- 172.21.15.114:0/1231441610 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c6c045040 con 0x55c1778b90b0 2024-09-13T18:31:11.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.838+0000 7f4c7d59a640 1 --2- 172.21.15.114:0/1231441610 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1778b90b0 0x55c1778c02a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c03e040 tx=0x7f4c6c0097e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0e2ec7a009e3005 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:11.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.839+0000 7f4c617fa640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:31:11.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.839+0000 7f4c617fa640 1 -- 172.21.15.114:0/1231441610 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c02f5d0 con 0x55c1778b90b0 2024-09-13T18:31:11.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.839+0000 7f4c617fa640 1 -- 172.21.15.114:0/1231441610 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f4c6c02f730 con 0x55c1778b90b0 2024-09-13T18:31:11.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.840+0000 7f4c82dfc740 1 -- 172.21.15.114:0/1231441610 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c177845b80 con 0x55c1778b90b0 2024-09-13T18:31:11.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.842+0000 7f4c617fa640 1 -- 172.21.15.114:0/1231441610 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c6c02f8f0 con 0x55c1778b90b0 2024-09-13T18:31:11.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.845+0000 7f4c82dfc740 10 client.27412 fetch_fsmap learned FSMap version 34 2024-09-13T18:31:11.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.845+0000 7f4c82dfc740 1 -- 172.21.15.114:0/1231441610 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55c17781a3c0 con 0x55c1778b90b0 2024-09-13T18:31:11.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.846+0000 7f4c617fa640 1 -- 172.21.15.114:0/1231441610 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c177845b80 con 0x55c1778b90b0 2024-09-13T18:31:11.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.846+0000 7f4c617fa640 1 -- 172.21.15.114:0/1231441610 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4c6c02e050 con 0x55c1778b90b0 2024-09-13T18:31:11.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.847+0000 7f4c82dfc740 10 client.27412 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:31:11.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.847+0000 7f4c82dfc740 10 client.27412 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-13T18:31:11.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.847+0000 7f4c82dfc740 20 client.27412 populate_metadata read hostname 'smithi114' 2024-09-13T18:31:11.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.848+0000 7f4c82dfc740 1 --2- 172.21.15.114:0/1231441610 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55c1778d2f20 0x55c1778d5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:11.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.848+0000 7f4c82dfc740 4 client.27412 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:31:11.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.848+0000 7f4c82dfc740 1 -- 172.21.15.114:0/1231441610 --> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c17780e1f0 con 0x55c1778d2f20 2024-09-13T18:31:11.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.853+0000 7f4c7dd9b640 1 --2- 172.21.15.114:0/1231441610 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55c1778d2f20 0x55c1778d5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:11.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.874+0000 7f4c7dd9b640 1 --2- 172.21.15.114:0/1231441610 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55c1778d2f20 0x55c1778d5340 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c65bdcc55889fbe6 server_cookie=8ff088ab3179619e in_seq=0 out_seq=0 2024-09-13T18:31:11.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.874+0000 7f4c617fa640 10 client.27412 ms_handle_connect on v2:172.21.15.134:6834/1150003275 2024-09-13T18:31:11.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.879+0000 7f4c617fa640 1 -- 172.21.15.114:0/1231441610 <== mds.3 v2:172.21.15.134:6834/1150003275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55c17780e1f0 con 0x55c1778d2f20 2024-09-13T18:31:11.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.879+0000 7f4c617fa640 10 client.27412 handle_command_reply: tid=0 2024-09-13T18:31:11.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.879+0000 7f4c82dfc740 1 client.27412 shutdown 2024-09-13T18:31:11.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.880+0000 7f4c82dfc740 1 -- 172.21.15.114:0/1231441610 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1778b90b0 msgr2=0x55c1778c02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:11.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.880+0000 7f4c82dfc740 1 --2- 172.21.15.114:0/1231441610 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1778b90b0 0x55c1778c02a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c03e040 tx=0x7f4c6c0097e0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.882+0000 7f4c82dfc740 1 -- 172.21.15.114:0/1231441610 shutdown_connections 2024-09-13T18:31:11.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.882+0000 7f4c82dfc740 1 --2- 172.21.15.114:0/1231441610 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55c1778d2f20 0x55c1778d5340 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.882+0000 7f4c82dfc740 1 --2- 172.21.15.114:0/1231441610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1778c3950 0x55c1778c07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.882+0000 7f4c82dfc740 1 --2- 172.21.15.114:0/1231441610 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1778b90b0 0x55c1778c02a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.882+0000 7f4c82dfc740 1 --2- 172.21.15.114:0/1231441610 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c1778b8790 0x55c1778c2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.882+0000 7f4c82dfc740 1 -- 172.21.15.114:0/1231441610 >> 172.21.15.114:0/1231441610 conn(0x55c1778b4d00 msgr2=0x55c1778b3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:11.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.886+0000 7f4c617fa640 0 client.27412 ms_handle_reset on v2:172.21.15.134:6834/1150003275 2024-09-13T18:31:11.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.886+0000 7f4c82dfc740 1 -- 172.21.15.114:0/1231441610 shutdown_connections 2024-09-13T18:31:11.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.887+0000 7f4c82dfc740 1 -- 172.21.15.114:0/1231441610 wait complete. 2024-09-13T18:31:11.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.888+0000 7f4c82dfc740 20 client.27412 trim_cache size 0 max 16384 2024-09-13T18:31:11.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.923+0000 7f4c82dfc740 1 Processor -- start 2024-09-13T18:31:11.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.926+0000 7f4c82dfc740 1 -- start start 2024-09-13T18:31:11.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.934+0000 7f4c82dfc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1778b8790 0x55c1778d6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:11.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.935+0000 7f4c82dfc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c1778b90b0 0x55c1778ca610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:11.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.935+0000 7f4c82dfc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1778c3950 0x55c1778cab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:11.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.935+0000 7f4c82dfc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1778c37b0 con 0x55c1778b90b0 2024-09-13T18:31:11.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.935+0000 7f4c7dd9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1778b8790 0x55c1778d6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:11.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.935+0000 7f4c7dd9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1778b8790 0x55c1778d6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:53396/0 (socket says 172.21.15.114:53396) 2024-09-13T18:31:11.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.935+0000 7f4c7dd9b640 1 -- 172.21.15.114:0/2719619529 learned_addr learned my addr 172.21.15.114:0/2719619529 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:11.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.935+0000 7f4c82dfc740 1 -- 172.21.15.114:0/2719619529 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1778af3d0 con 0x55c1778c3950 2024-09-13T18:31:11.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.935+0000 7f4c82dfc740 1 -- 172.21.15.114:0/2719619529 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1778b9620 con 0x55c1778b8790 2024-09-13T18:31:11.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.937+0000 7f4c7e59c640 1 --2- 172.21.15.114:0/2719619529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1778c3950 0x55c1778cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:11.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.937+0000 7f4c7d59a640 1 --2- 172.21.15.114:0/2719619529 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c1778b90b0 0x55c1778ca610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:11.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.938+0000 7f4c7e59c640 1 -- 172.21.15.114:0/2719619529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1778b8790 msgr2=0x55c1778d6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:11.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.938+0000 7f4c7e59c640 1 --2- 172.21.15.114:0/2719619529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1778b8790 0x55c1778d6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.938+0000 7f4c7e59c640 1 -- 172.21.15.114:0/2719619529 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c1778b90b0 msgr2=0x55c1778ca610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:11.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.938+0000 7f4c7e59c640 1 --2- 172.21.15.114:0/2719619529 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c1778b90b0 0x55c1778ca610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.938+0000 7f4c7e59c640 1 -- 172.21.15.114:0/2719619529 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1777bc770 con 0x55c1778c3950 2024-09-13T18:31:11.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.939+0000 7f4c7e59c640 1 --2- 172.21.15.114:0/2719619529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1778c3950 0x55c1778cab50 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f4c640c9da0 tx=0x7f4c640992a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62b81f7e415841ed server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:11.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.940+0000 7f4c74ff9640 1 -- 172.21.15.114:0/2719619529 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c64099760 con 0x55c1778c3950 2024-09-13T18:31:11.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:11 smithi204 ceph-mon[34777]: pgmap v269: 129 pgs: 15 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 74 active+clean; 22 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 26 MiB/s wr, 584 op/s 2024-09-13T18:31:11.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:11 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/469922759' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:31:11.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.944+0000 7f4c74ff9640 1 -- 172.21.15.114:0/2719619529 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4c6409d590 con 0x55c1778c3950 2024-09-13T18:31:11.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.944+0000 7f4c74ff9640 1 -- 172.21.15.114:0/2719619529 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c640c9f10 con 0x55c1778c3950 2024-09-13T18:31:11.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.945+0000 7f4c82dfc740 1 -- 172.21.15.114:0/2719619529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1778c3950 msgr2=0x55c1778cab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:11.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.945+0000 7f4c82dfc740 1 --2- 172.21.15.114:0/2719619529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1778c3950 0x55c1778cab50 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f4c640c9da0 tx=0x7f4c640992a0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.946+0000 7f4c82dfc740 1 -- 172.21.15.114:0/2719619529 shutdown_connections 2024-09-13T18:31:11.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.946+0000 7f4c82dfc740 1 --2- 172.21.15.114:0/2719619529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1778c3950 0x55c1778cab50 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.947+0000 7f4c82dfc740 1 --2- 172.21.15.114:0/2719619529 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c1778b90b0 0x55c1778ca610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.947+0000 7f4c82dfc740 1 --2- 172.21.15.114:0/2719619529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1778b8790 0x55c1778d6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.947+0000 7f4c82dfc740 1 -- 172.21.15.114:0/2719619529 >> 172.21.15.114:0/2719619529 conn(0x55c1778b4d00 msgr2=0x55c1778cf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:11.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.953+0000 7f4c82dfc740 1 -- 172.21.15.114:0/2719619529 shutdown_connections 2024-09-13T18:31:11.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.953+0000 7f4c82dfc740 1 -- 172.21.15.114:0/2719619529 wait complete. 2024-09-13T18:31:11.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.953+0000 7f4c82dfc740 1 -- start start 2024-09-13T18:31:11.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.954+0000 7f4c82dfc740 1 Processor -- start 2024-09-13T18:31:11.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.960+0000 7f4c82dfc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1778b8790 0x55c1778bad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:11.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.960+0000 7f4c82dfc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1778b90b0 0x55c1778bb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:11.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.960+0000 7f4c82dfc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c1778c3950 0x55c1778bb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:11.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.960+0000 7f4c82dfc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c177928b40 con 0x55c1778c3950 2024-09-13T18:31:11.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.960+0000 7f4c82dfc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c177928cb0 con 0x55c1778b90b0 2024-09-13T18:31:11.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.960+0000 7f4c82dfc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c177928e70 con 0x55c1778b8790 2024-09-13T18:31:11.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.960+0000 7f4c82dfc740 10 client.0 _ll_register_callbacks cb 0x7f4c6c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:31:11.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.962+0000 7f4c7d59a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1778b90b0 0x55c1778bb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:11.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.962+0000 7f4c7d59a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1778b90b0 0x55c1778bb280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:45864/0 (socket says 172.21.15.114:45864) 2024-09-13T18:31:11.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.962+0000 7f4c7d59a640 1 -- 172.21.15.114:0/266008119 learned_addr learned my addr 172.21.15.114:0/266008119 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:11.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.963+0000 7f4c7dd9b640 1 --2- 172.21.15.114:0/266008119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1778b8790 0x55c1778bad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:11.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.963+0000 7f4c7e59c640 1 --2- 172.21.15.114:0/266008119 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c1778c3950 0x55c1778bb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:11.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.967+0000 7f4c7d59a640 1 -- 172.21.15.114:0/266008119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1778b8790 msgr2=0x55c1778bad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:11.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.967+0000 7f4c7d59a640 1 --2- 172.21.15.114:0/266008119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1778b8790 0x55c1778bad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.967+0000 7f4c7d59a640 1 -- 172.21.15.114:0/266008119 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c1778c3950 msgr2=0x55c1778bb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:11.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.967+0000 7f4c7d59a640 1 --2- 172.21.15.114:0/266008119 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c1778c3950 0x55c1778bb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:11.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.967+0000 7f4c7d59a640 1 -- 172.21.15.114:0/266008119 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c7806c0c0 con 0x55c1778b90b0 2024-09-13T18:31:11.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.968+0000 7f4c7d59a640 1 --2- 172.21.15.114:0/266008119 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1778b90b0 0x55c1778bb280 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x55c1778bfb10 tx=0x7f4c6c06c620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f52d91efb0ddcd01 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:11.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:11 smithi114 ceph-mon[30897]: pgmap v269: 129 pgs: 15 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 74 active+clean; 22 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 26 MiB/s wr, 584 op/s 2024-09-13T18:31:11.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:11 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/469922759' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:31:11.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.969+0000 7f4c617fa640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:31:11.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.973+0000 7f4c617fa640 1 -- 172.21.15.114:0/266008119 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c005bf0 con 0x55c1778b90b0 2024-09-13T18:31:11.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.973+0000 7f4c82dfc740 1 -- 172.21.15.114:0/266008119 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4c6c030b10 con 0x55c1778b90b0 2024-09-13T18:31:11.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.973+0000 7f4c617fa640 1 -- 172.21.15.114:0/266008119 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f4c6c06c770 con 0x55c1778b90b0 2024-09-13T18:31:11.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.977+0000 7f4c617fa640 1 -- 172.21.15.114:0/266008119 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c6c030c80 con 0x55c1778b90b0 2024-09-13T18:31:11.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.979+0000 7f4c82dfc740 10 client.27434 fetch_fsmap learned FSMap version 34 2024-09-13T18:31:11.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.979+0000 7f4c82dfc740 1 -- 172.21.15.114:0/266008119 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55c1777e1e80 con 0x55c1778b90b0 2024-09-13T18:31:11.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.982+0000 7f4c617fa640 1 -- 172.21.15.114:0/266008119 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4c6c030b10 con 0x55c1778b90b0 2024-09-13T18:31:11.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.982+0000 7f4c617fa640 1 -- 172.21.15.114:0/266008119 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4c6c06cd20 con 0x55c1778b90b0 2024-09-13T18:31:11.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.982+0000 7f4c82dfc740 10 client.27434 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:31:11.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.982+0000 7f4c82dfc740 10 client.27434 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-13T18:31:11.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.982+0000 7f4c82dfc740 20 client.27434 populate_metadata read hostname 'smithi114' 2024-09-13T18:31:11.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.982+0000 7f4c82dfc740 1 --2- 172.21.15.114:0/266008119 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55c17792a120 0x55c17792c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:11.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.982+0000 7f4c82dfc740 4 client.27434 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:31:11.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.982+0000 7f4c82dfc740 1 -- 172.21.15.114:0/266008119 --> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c17750f4e0 con 0x55c17792a120 2024-09-13T18:31:11.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.988+0000 7f4c7dd9b640 1 --2- 172.21.15.114:0/266008119 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55c17792a120 0x55c17792c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:11.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.991+0000 7f4c7dd9b640 1 --2- 172.21.15.114:0/266008119 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55c17792a120 0x55c17792c540 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c11136b21dfa3bfb server_cookie=59f05727fc140c6a in_seq=0 out_seq=0 2024-09-13T18:31:11.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.991+0000 7f4c617fa640 10 client.27434 ms_handle_connect on v2:172.21.15.134:6834/1150003275 2024-09-13T18:31:11.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.994+0000 7f4c617fa640 1 -- 172.21.15.114:0/266008119 <== mds.3 v2:172.21.15.134:6834/1150003275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c17750f4e0 con 0x55c17792a120 2024-09-13T18:31:11.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.994+0000 7f4c617fa640 10 client.27434 handle_command_reply: tid=0 2024-09-13T18:31:11.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.994+0000 7f4c82dfc740 1 client.27434 shutdown 2024-09-13T18:31:12.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.999+0000 7f4c82dfc740 1 -- 172.21.15.114:0/266008119 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1778b90b0 msgr2=0x55c1778bb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:12.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:11.999+0000 7f4c82dfc740 1 --2- 172.21.15.114:0/266008119 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1778b90b0 0x55c1778bb280 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x55c1778bfb10 tx=0x7f4c6c06c620 comp rx=0 tx=0).stop 2024-09-13T18:31:12.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.004+0000 7f4c82dfc740 1 -- 172.21.15.114:0/266008119 shutdown_connections 2024-09-13T18:31:12.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.004+0000 7f4c82dfc740 1 --2- 172.21.15.114:0/266008119 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55c17792a120 0x55c17792c540 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.004+0000 7f4c82dfc740 1 --2- 172.21.15.114:0/266008119 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c1778c3950 0x55c1778bb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.004+0000 7f4c82dfc740 1 --2- 172.21.15.114:0/266008119 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1778b90b0 0x55c1778bb280 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.004+0000 7f4c82dfc740 1 --2- 172.21.15.114:0/266008119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c1778b8790 0x55c1778bad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.004+0000 7f4c82dfc740 1 -- 172.21.15.114:0/266008119 >> 172.21.15.114:0/266008119 conn(0x55c1778b4d00 msgr2=0x55c1778b50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:12.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.004+0000 7f4c617fa640 0 client.27434 ms_handle_reset on v2:172.21.15.134:6834/1150003275 2024-09-13T18:31:12.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.004+0000 7f4c82dfc740 1 -- 172.21.15.114:0/266008119 shutdown_connections 2024-09-13T18:31:12.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.004+0000 7f4c82dfc740 1 -- 172.21.15.114:0/266008119 wait complete. 2024-09-13T18:31:12.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.006+0000 7f4c82dfc740 20 client.27434 trim_cache size 0 max 16384 2024-09-13T18:31:12.008 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:31:12.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.009+0000 7f4c7ffff640 1 -- 172.21.15.114:0/3819137730 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f4c680a1410 msgr2=0x7f4c680a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:12.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.009+0000 7f4c7ffff640 1 --2- 172.21.15.114:0/3819137730 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f4c680a1410 0x7f4c680a38d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4c700058c0 tx=0x7f4c70005850 comp rx=0 tx=0).stop 2024-09-13T18:31:12.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.009+0000 7f4c7ffff640 1 -- 172.21.15.114:0/3819137730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c7810c400 msgr2=0x7f4c7819ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:12.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.009+0000 7f4c7ffff640 1 --2- 172.21.15.114:0/3819137730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c7810c400 0x7f4c7819ff90 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4c6400b3f0 tx=0x7f4c6400b8b0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.012+0000 7f4c7ffff640 1 -- 172.21.15.114:0/3819137730 shutdown_connections 2024-09-13T18:31:12.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.012+0000 7f4c7ffff640 1 --2- 172.21.15.114:0/3819137730 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f4c680a1410 0x7f4c680a38d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.012+0000 7f4c7ffff640 1 --2- 172.21.15.114:0/3819137730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c7810c400 0x7f4c7819ff90 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.012+0000 7f4c7ffff640 1 --2- 172.21.15.114:0/3819137730 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4c78106cc0 0x7f4c7819cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.012+0000 7f4c7ffff640 1 --2- 172.21.15.114:0/3819137730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c78106340 0x7f4c7819c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.012+0000 7f4c7ffff640 1 -- 172.21.15.114:0/3819137730 >> 172.21.15.114:0/3819137730 conn(0x7f4c78075c80 msgr2=0x7f4c78077520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:12.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.013+0000 7f4c7ffff640 1 -- 172.21.15.114:0/3819137730 shutdown_connections 2024-09-13T18:31:12.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.013+0000 7f4c7ffff640 1 -- 172.21.15.114:0/3819137730 wait complete. 2024-09-13T18:31:12.049 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-09-13T18:31:12.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:11 smithi134 ceph-mon[34283]: pgmap v269: 129 pgs: 15 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 74 active+clean; 22 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 26 MiB/s wr, 584 op/s 2024-09-13T18:31:12.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:11 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/469922759' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:31:12.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.323+0000 7f74bd83a640 1 -- 172.21.15.114:0/303827765 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74b8119400 msgr2=0x7f74b811b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:12.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.323+0000 7f74bd83a640 1 --2- 172.21.15.114:0/303827765 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74b8119400 0x7f74b811b7f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f74b0009900 tx=0x7f74b002a440 comp rx=0 tx=0).stop 2024-09-13T18:31:12.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.325+0000 7f74bd83a640 1 -- 172.21.15.114:0/303827765 shutdown_connections 2024-09-13T18:31:12.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.325+0000 7f74bd83a640 1 --2- 172.21.15.114:0/303827765 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74b8119400 0x7f74b811b7f0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.325+0000 7f74bd83a640 1 --2- 172.21.15.114:0/303827765 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74b8107890 0x7f74b8107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.325+0000 7f74bd83a640 1 --2- 172.21.15.114:0/303827765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74b81074b0 0x7f74b810e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.325+0000 7f74bd83a640 1 -- 172.21.15.114:0/303827765 >> 172.21.15.114:0/303827765 conn(0x7f74b806af40 msgr2=0x7f74b806b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:12.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.333+0000 7f74bd83a640 1 -- 172.21.15.114:0/303827765 shutdown_connections 2024-09-13T18:31:12.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.337+0000 7f74bd83a640 1 -- 172.21.15.114:0/303827765 wait complete. 2024-09-13T18:31:12.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.337+0000 7f74bd83a640 1 Processor -- start 2024-09-13T18:31:12.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.342+0000 7f74bd83a640 1 -- start start 2024-09-13T18:31:12.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.347+0000 7f74bd83a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74b81074b0 0x7f74b8118830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:12.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.347+0000 7f74bd83a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74b8107890 0x7f74b8118d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:12.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.348+0000 7f74bd83a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74b8119400 0x7f74b8112970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:12.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.348+0000 7f74bd83a640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b8113100 con 0x7f74b8119400 2024-09-13T18:31:12.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.348+0000 7f74bd83a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b8113270 con 0x7f74b81074b0 2024-09-13T18:31:12.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.348+0000 7f74bd83a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b8113430 con 0x7f74b8107890 2024-09-13T18:31:12.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.348+0000 7f74bc838640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74b81074b0 0x7f74b8118830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:12.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.349+0000 7f74bc838640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74b81074b0 0x7f74b8118830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:45872/0 (socket says 172.21.15.114:45872) 2024-09-13T18:31:12.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.349+0000 7f74bc838640 1 -- 172.21.15.114:0/1569788885 learned_addr learned my addr 172.21.15.114:0/1569788885 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:12.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.349+0000 7f74bc838640 1 -- 172.21.15.114:0/1569788885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74b8107890 msgr2=0x7f74b8118d70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:31:12.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.349+0000 7f74bc838640 1 --2- 172.21.15.114:0/1569788885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74b8107890 0x7f74b8118d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.349+0000 7f74bc838640 1 -- 172.21.15.114:0/1569788885 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74b8119400 msgr2=0x7f74b8112970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:12.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.349+0000 7f74bc838640 1 --2- 172.21.15.114:0/1569788885 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74b8119400 0x7f74b8112970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.349+0000 7f74bc838640 1 -- 172.21.15.114:0/1569788885 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b0009560 con 0x7f74b81074b0 2024-09-13T18:31:12.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.350+0000 7f74bc838640 1 --2- 172.21.15.114:0/1569788885 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74b81074b0 0x7f74b8118830 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f74a800ba50 tx=0x7f74a800bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:12.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.353+0000 7f74b5ffb640 1 -- 172.21.15.114:0/1569788885 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a800cbe0 con 0x7f74b81074b0 2024-09-13T18:31:12.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.353+0000 7f74b5ffb640 1 -- 172.21.15.114:0/1569788885 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f74a800cd40 con 0x7f74b81074b0 2024-09-13T18:31:12.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.355+0000 7f74b5ffb640 1 -- 172.21.15.114:0/1569788885 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a8007650 con 0x7f74b81074b0 2024-09-13T18:31:12.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.356+0000 7f74bd83a640 1 -- 172.21.15.114:0/1569788885 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74b81ba190 con 0x7f74b81074b0 2024-09-13T18:31:12.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.356+0000 7f74bd83a640 1 -- 172.21.15.114:0/1569788885 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74b81ba500 con 0x7f74b81074b0 2024-09-13T18:31:12.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.360+0000 7f74c17c4740 1 Processor -- start 2024-09-13T18:31:12.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.360+0000 7f74c17c4740 1 -- start start 2024-09-13T18:31:12.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.360+0000 7f74c17c4740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a917867970 0x55a917867d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:12.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.360+0000 7f74c17c4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a917868290 0x55a9178725f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:12.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.360+0000 7f74c17c4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a917872b30 0x55a917874f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:12.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.360+0000 7f74c17c4740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9177d1cc0 con 0x55a917867970 2024-09-13T18:31:12.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.360+0000 7f74c17c4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9177cf450 con 0x55a917872b30 2024-09-13T18:31:12.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.360+0000 7f74c17c4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9177cad40 con 0x55a917868290 2024-09-13T18:31:12.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.361+0000 7f74b7fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a917872b30 0x55a917874f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:12.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.361+0000 7f74bd039640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a917867970 0x55a917867d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:12.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.361+0000 7f74bd039640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a917867970 0x55a917867d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51874/0 (socket says 172.21.15.114:51874) 2024-09-13T18:31:12.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.361+0000 7f74bd039640 1 -- 172.21.15.114:0/3660130946 learned_addr learned my addr 172.21.15.114:0/3660130946 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:12.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.361+0000 7f74b7fff640 1 -- 172.21.15.114:0/3660130946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a917868290 msgr2=0x55a9178725f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:12.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.361+0000 7f74b7fff640 1 --2- 172.21.15.114:0/3660130946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a917868290 0x55a9178725f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.361+0000 7f74b7fff640 1 -- 172.21.15.114:0/3660130946 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a917867970 msgr2=0x55a917867d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:12.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.361+0000 7f74b7fff640 1 --2- 172.21.15.114:0/3660130946 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a917867970 0x55a917867d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.361+0000 7f74b7fff640 1 -- 172.21.15.114:0/3660130946 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a917792620 con 0x55a917872b30 2024-09-13T18:31:12.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.363+0000 7f74b5ffb640 1 -- 172.21.15.114:0/1569788885 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f74a8014440 con 0x7f74b81074b0 2024-09-13T18:31:12.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.364+0000 7f74b5ffb640 1 --2- 172.21.15.114:0/1569788885 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f74a00a1260 0x7f74a00a3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:12.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.364+0000 7f74b5ffb640 1 -- 172.21.15.114:0/1569788885 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f74a80ce900 con 0x7f74b81074b0 2024-09-13T18:31:12.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.370+0000 7f74b7fff640 1 --2- 172.21.15.114:0/3660130946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a917872b30 0x55a917874f20 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f74b811b1e0 tx=0x7f74ac009b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb5783fbefd9eecb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:12.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.370+0000 7f74bd039640 1 --2- 172.21.15.114:0/1569788885 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f74a00a1260 0x7f74a00a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:12.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.373+0000 7f749effd640 1 -- 172.21.15.114:0/3660130946 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ac00b8c0 con 0x55a917872b30 2024-09-13T18:31:12.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.373+0000 7f749effd640 1 -- 172.21.15.114:0/3660130946 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f74ac011420 con 0x55a917872b30 2024-09-13T18:31:12.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.374+0000 7f74c17c4740 1 -- 172.21.15.114:0/3660130946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a917872b30 msgr2=0x55a917874f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:12.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.374+0000 7f74c17c4740 1 --2- 172.21.15.114:0/3660130946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a917872b30 0x55a917874f20 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f74b811b1e0 tx=0x7f74ac009b20 comp rx=0 tx=0).stop 2024-09-13T18:31:12.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.376+0000 7f74bd039640 1 --2- 172.21.15.114:0/1569788885 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f74a00a1260 0x7f74a00a3720 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f74b002aa30 tx=0x7f74b0005760 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:12.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.380+0000 7f74c17c4740 1 -- 172.21.15.114:0/3660130946 shutdown_connections 2024-09-13T18:31:12.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.380+0000 7f74c17c4740 1 --2- 172.21.15.114:0/3660130946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a917872b30 0x55a917874f20 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.380+0000 7f74c17c4740 1 --2- 172.21.15.114:0/3660130946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a917868290 0x55a9178725f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.380+0000 7f74c17c4740 1 --2- 172.21.15.114:0/3660130946 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a917867970 0x55a917867d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.380+0000 7f74c17c4740 1 -- 172.21.15.114:0/3660130946 >> 172.21.15.114:0/3660130946 conn(0x55a917863ee0 msgr2=0x55a9178642c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:12.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.387+0000 7f74c17c4740 1 -- 172.21.15.114:0/3660130946 shutdown_connections 2024-09-13T18:31:12.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.389+0000 7f74c17c4740 1 -- 172.21.15.114:0/3660130946 wait complete. 2024-09-13T18:31:12.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.389+0000 7f74c17c4740 1 -- start start 2024-09-13T18:31:12.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.389+0000 7f74c17c4740 1 Processor -- start 2024-09-13T18:31:12.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.390+0000 7f74c17c4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a917867970 0x55a91786f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:12.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.390+0000 7f74c17c4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a917868290 0x55a91786fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:12.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.390+0000 7f74c17c4740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a917872b30 0x55a917876960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:12.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.390+0000 7f74c17c4740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a917789180 con 0x55a917872b30 2024-09-13T18:31:12.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.390+0000 7f74c17c4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9174a3430 con 0x55a917868290 2024-09-13T18:31:12.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.390+0000 7f74c17c4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a91774e5e0 con 0x55a917867970 2024-09-13T18:31:12.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.390+0000 7f74c17c4740 10 client.0 _ll_register_callbacks cb 0x55a9177baaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:31:12.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.390+0000 7f74bc838640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a917867970 0x55a91786f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:12.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.390+0000 7f74bc838640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a917867970 0x55a91786f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:53434/0 (socket says 172.21.15.114:53434) 2024-09-13T18:31:12.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.390+0000 7f74bc838640 1 -- 172.21.15.114:0/2517470473 learned_addr learned my addr 172.21.15.114:0/2517470473 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:12.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.391+0000 7f74bc838640 1 -- 172.21.15.114:0/2517470473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a917868290 msgr2=0x55a91786fa10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:31:12.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.391+0000 7f74bc838640 1 --2- 172.21.15.114:0/2517470473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a917868290 0x55a91786fa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.391+0000 7f74bc838640 1 -- 172.21.15.114:0/2517470473 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a917872b30 msgr2=0x55a917876960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:12.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.391+0000 7f74bc838640 1 --2- 172.21.15.114:0/2517470473 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a917872b30 0x55a917876960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.391+0000 7f74bc838640 1 -- 172.21.15.114:0/2517470473 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74ac0092f0 con 0x55a917867970 2024-09-13T18:31:12.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.391+0000 7f74bc838640 1 --2- 172.21.15.114:0/2517470473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a917867970 0x55a91786f4d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f74a8097bc0 tx=0x7f74a809be80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e708c78446a5326 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:12.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.392+0000 7f7497fff640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:31:12.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.398+0000 7f7497fff640 1 -- 172.21.15.114:0/2517470473 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a80a3b40 con 0x55a917867970 2024-09-13T18:31:12.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.398+0000 7f74c17c4740 1 -- 172.21.15.114:0/2517470473 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a9177f4b80 con 0x55a917867970 2024-09-13T18:31:12.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.398+0000 7f7497fff640 1 -- 172.21.15.114:0/2517470473 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f74a80cbe70 con 0x55a917867970 2024-09-13T18:31:12.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.403+0000 7f7497fff640 1 -- 172.21.15.114:0/2517470473 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74a80cc050 con 0x55a917867970 2024-09-13T18:31:12.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.407+0000 7f74c17c4740 10 client.27421 fetch_fsmap learned FSMap version 34 2024-09-13T18:31:12.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.407+0000 7f74c17c4740 1 -- 172.21.15.114:0/2517470473 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55a9177c93c0 con 0x55a917867970 2024-09-13T18:31:12.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.415+0000 7f7497fff640 1 -- 172.21.15.114:0/2517470473 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a9177f4b80 con 0x55a917867970 2024-09-13T18:31:12.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.415+0000 7f7497fff640 1 -- 172.21.15.114:0/2517470473 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f74a809b920 con 0x55a917867970 2024-09-13T18:31:12.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.415+0000 7f74c17c4740 10 client.27421 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:31:12.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.415+0000 7f74c17c4740 10 client.27421 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-13T18:31:12.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.415+0000 7f74c17c4740 20 client.27421 populate_metadata read hostname 'smithi114' 2024-09-13T18:31:12.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.415+0000 7f74c17c4740 1 --2- 172.21.15.114:0/2517470473 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55a917882540 0x55a917884960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:12.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.415+0000 7f74c17c4740 4 client.27421 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:31:12.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.415+0000 7f74c17c4740 1 -- 172.21.15.114:0/2517470473 --> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a9177bd1f0 con 0x55a917882540 2024-09-13T18:31:12.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.421+0000 7f74b7fff640 1 --2- 172.21.15.114:0/2517470473 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55a917882540 0x55a917884960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:12.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.438+0000 7f74b7fff640 1 --2- 172.21.15.114:0/2517470473 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55a917882540 0x55a917884960 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4dcb4ce5851f42f server_cookie=640d1aff209bf995 in_seq=0 out_seq=0 2024-09-13T18:31:12.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.440+0000 7f7497fff640 10 client.27421 ms_handle_connect on v2:172.21.15.204:6838/344883912 2024-09-13T18:31:12.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.440+0000 7f7497fff640 1 -- 172.21.15.114:0/2517470473 <== mds.2 v2:172.21.15.204:6838/344883912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55a9177bd1f0 con 0x55a917882540 2024-09-13T18:31:12.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.440+0000 7f7497fff640 10 client.27421 handle_command_reply: tid=0 2024-09-13T18:31:12.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.440+0000 7f74c17c4740 1 client.27421 shutdown 2024-09-13T18:31:12.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.453+0000 7f74c17c4740 1 -- 172.21.15.114:0/2517470473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a917867970 msgr2=0x55a91786f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:12.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.453+0000 7f74c17c4740 1 --2- 172.21.15.114:0/2517470473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a917867970 0x55a91786f4d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f74a8097bc0 tx=0x7f74a809be80 comp rx=0 tx=0).stop 2024-09-13T18:31:12.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.459+0000 7f74c17c4740 1 -- 172.21.15.114:0/2517470473 shutdown_connections 2024-09-13T18:31:12.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.459+0000 7f74c17c4740 1 --2- 172.21.15.114:0/2517470473 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55a917882540 0x55a917884960 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.459+0000 7f74c17c4740 1 --2- 172.21.15.114:0/2517470473 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a917872b30 0x55a917876960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.459+0000 7f74c17c4740 1 --2- 172.21.15.114:0/2517470473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a917868290 0x55a91786fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.459+0000 7f74c17c4740 1 --2- 172.21.15.114:0/2517470473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a917867970 0x55a91786f4d0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.459+0000 7f74c17c4740 1 -- 172.21.15.114:0/2517470473 >> 172.21.15.114:0/2517470473 conn(0x55a917863ee0 msgr2=0x55a917863230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:12.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.459+0000 7f7497fff640 0 client.27421 ms_handle_reset on v2:172.21.15.204:6838/344883912 2024-09-13T18:31:12.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.466+0000 7f74c17c4740 1 -- 172.21.15.114:0/2517470473 shutdown_connections 2024-09-13T18:31:12.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.469+0000 7f74c17c4740 1 -- 172.21.15.114:0/2517470473 wait complete. 2024-09-13T18:31:12.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.475+0000 7f74c17c4740 20 client.27421 trim_cache size 0 max 16384 2024-09-13T18:31:12.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.498+0000 7f74c17c4740 1 Processor -- start 2024-09-13T18:31:12.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.499+0000 7f74c17c4740 1 -- start start 2024-09-13T18:31:12.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.499+0000 7f74c17c4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a917867970 0x55a9178768e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:12.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.499+0000 7f74c17c4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a917868290 0x55a91786a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:12.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.499+0000 7f74c17c4740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a917872b30 0x55a91786aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:12.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.499+0000 7f74c17c4740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9177bd1f0 con 0x55a917872b30 2024-09-13T18:31:12.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.499+0000 7f74c17c4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a917880d60 con 0x55a917867970 2024-09-13T18:31:12.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.499+0000 7f74c17c4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a91787f1b0 con 0x55a917868290 2024-09-13T18:31:12.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.500+0000 7f74b7fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a917868290 0x55a91786a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:12.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.502+0000 7f74b7fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a917868290 0x55a91786a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:53438/0 (socket says 172.21.15.114:53438) 2024-09-13T18:31:12.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.502+0000 7f74b7fff640 1 -- 172.21.15.114:0/3368575561 learned_addr learned my addr 172.21.15.114:0/3368575561 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:12.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.503+0000 7f74b7fff640 1 -- 172.21.15.114:0/3368575561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a917867970 msgr2=0x55a9178768e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:31:12.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.503+0000 7f74b7fff640 1 --2- 172.21.15.114:0/3368575561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a917867970 0x55a9178768e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.503+0000 7f74b7fff640 1 -- 172.21.15.114:0/3368575561 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a917872b30 msgr2=0x55a91786aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:12.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.503+0000 7f74b7fff640 1 --2- 172.21.15.114:0/3368575561 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a917872b30 0x55a91786aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.503+0000 7f74b7fff640 1 -- 172.21.15.114:0/3368575561 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a91776b770 con 0x55a917868290 2024-09-13T18:31:12.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.504+0000 7f74b7fff640 1 --2- 172.21.15.114:0/3368575561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a917868290 0x55a91786a540 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f74ac006fd0 tx=0x7f74ac011d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b45c2414c91bf344 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:12.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.509+0000 7f749f7fe640 1 -- 172.21.15.114:0/3368575561 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ac020040 con 0x55a917868290 2024-09-13T18:31:12.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.509+0000 7f749f7fe640 1 -- 172.21.15.114:0/3368575561 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f74ac027070 con 0x55a917868290 2024-09-13T18:31:12.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.509+0000 7f749f7fe640 1 -- 172.21.15.114:0/3368575561 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ac004040 con 0x55a917868290 2024-09-13T18:31:12.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.511+0000 7f74c17c4740 1 -- 172.21.15.114:0/3368575561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a917868290 msgr2=0x55a91786a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:12.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.511+0000 7f74c17c4740 1 --2- 172.21.15.114:0/3368575561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a917868290 0x55a91786a540 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f74ac006fd0 tx=0x7f74ac011d50 comp rx=0 tx=0).stop 2024-09-13T18:31:12.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.519+0000 7f74c17c4740 1 -- 172.21.15.114:0/3368575561 shutdown_connections 2024-09-13T18:31:12.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.519+0000 7f74c17c4740 1 --2- 172.21.15.114:0/3368575561 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a917872b30 0x55a91786aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.519+0000 7f74c17c4740 1 --2- 172.21.15.114:0/3368575561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a917868290 0x55a91786a540 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.519+0000 7f74c17c4740 1 --2- 172.21.15.114:0/3368575561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a917867970 0x55a9178768e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.519+0000 7f74c17c4740 1 -- 172.21.15.114:0/3368575561 >> 172.21.15.114:0/3368575561 conn(0x55a917863ee0 msgr2=0x55a91786db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:12.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.524+0000 7f74c17c4740 1 -- 172.21.15.114:0/3368575561 shutdown_connections 2024-09-13T18:31:12.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.527+0000 7f74c17c4740 1 -- 172.21.15.114:0/3368575561 wait complete. 2024-09-13T18:31:12.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.528+0000 7f74c17c4740 1 -- start start 2024-09-13T18:31:12.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.528+0000 7f74c17c4740 1 Processor -- start 2024-09-13T18:31:12.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.533+0000 7f74c17c4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a917867970 0x55a91787a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:12.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.533+0000 7f74c17c4740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a917868290 0x55a91787aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:12.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.533+0000 7f74c17c4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a917872b30 0x55a91787af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:12.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.533+0000 7f74c17c4740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9178d8a90 con 0x55a917868290 2024-09-13T18:31:12.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.533+0000 7f74c17c4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9178d8c00 con 0x55a917872b30 2024-09-13T18:31:12.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.533+0000 7f74c17c4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9178d8dc0 con 0x55a917867970 2024-09-13T18:31:12.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.533+0000 7f74c17c4740 10 client.0 _ll_register_callbacks cb 0x7f74ac006e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:31:12.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.534+0000 7f74bc838640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a917867970 0x55a91787a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:12.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.534+0000 7f74bc838640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a917867970 0x55a91787a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:53454/0 (socket says 172.21.15.114:53454) 2024-09-13T18:31:12.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.534+0000 7f74bc838640 1 -- 172.21.15.114:0/1805827243 learned_addr learned my addr 172.21.15.114:0/1805827243 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:12.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.535+0000 7f74bc838640 1 -- 172.21.15.114:0/1805827243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a917872b30 msgr2=0x55a91787af90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:31:12.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.535+0000 7f74bc838640 1 --2- 172.21.15.114:0/1805827243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a917872b30 0x55a91787af90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.535+0000 7f74bc838640 1 -- 172.21.15.114:0/1805827243 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a917868290 msgr2=0x55a91787aa50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:31:12.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.535+0000 7f74bc838640 1 --2- 172.21.15.114:0/1805827243 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a917868290 0x55a91787aa50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.535+0000 7f74bc838640 1 -- 172.21.15.114:0/1805827243 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a917792620 con 0x55a917867970 2024-09-13T18:31:12.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.536+0000 7f74bc838640 1 --2- 172.21.15.114:0/1805827243 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a917867970 0x55a91787a510 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f74a8098130 tx=0x7f74a8009a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f53390e444109d34 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:12.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.537+0000 7f7497fff640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:31:12.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.538+0000 7f7497fff640 1 -- 172.21.15.114:0/1805827243 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a80ae070 con 0x55a917867970 2024-09-13T18:31:12.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.538+0000 7f74c17c4740 1 -- 172.21.15.114:0/1805827243 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f74ac011420 con 0x55a917867970 2024-09-13T18:31:12.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.538+0000 7f7497fff640 1 -- 172.21.15.114:0/1805827243 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f74a809ac50 con 0x55a917867970 2024-09-13T18:31:12.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.539+0000 7f7497fff640 1 -- 172.21.15.114:0/1805827243 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74a809b200 con 0x55a917867970 2024-09-13T18:31:12.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.539+0000 7f74c17c4740 10 client.27427 fetch_fsmap learned FSMap version 34 2024-09-13T18:31:12.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.539+0000 7f74c17c4740 1 -- 172.21.15.114:0/1805827243 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55a917790e80 con 0x55a917867970 2024-09-13T18:31:12.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.541+0000 7f7497fff640 1 -- 172.21.15.114:0/1805827243 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f74ac011420 con 0x55a917867970 2024-09-13T18:31:12.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.541+0000 7f7497fff640 1 -- 172.21.15.114:0/1805827243 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f74a80cbe70 con 0x55a917867970 2024-09-13T18:31:12.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.541+0000 7f74c17c4740 10 client.27427 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:31:12.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.541+0000 7f74c17c4740 10 client.27427 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-13T18:31:12.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.541+0000 7f74c17c4740 20 client.27427 populate_metadata read hostname 'smithi114' 2024-09-13T18:31:12.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.542+0000 7f74c17c4740 1 --2- 172.21.15.114:0/1805827243 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55a9178da0b0 0x55a9178dc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:12.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.542+0000 7f74c17c4740 4 client.27427 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:31:12.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.542+0000 7f74c17c4740 1 -- 172.21.15.114:0/1805827243 --> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a9174be4e0 con 0x55a9178da0b0 2024-09-13T18:31:12.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.545+0000 7f74b7fff640 1 --2- 172.21.15.114:0/1805827243 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55a9178da0b0 0x55a9178dc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:12.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.546+0000 7f74b7fff640 1 --2- 172.21.15.114:0/1805827243 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55a9178da0b0 0x55a9178dc4a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e17834b1afb9c938 server_cookie=86eccc5aad07b79a in_seq=0 out_seq=0 2024-09-13T18:31:12.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.548+0000 7f7497fff640 10 client.27427 ms_handle_connect on v2:172.21.15.204:6838/344883912 2024-09-13T18:31:12.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.548+0000 7f7497fff640 1 -- 172.21.15.114:0/1805827243 <== mds.2 v2:172.21.15.204:6838/344883912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a9174be4e0 con 0x55a9178da0b0 2024-09-13T18:31:12.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.548+0000 7f7497fff640 10 client.27427 handle_command_reply: tid=0 2024-09-13T18:31:12.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.548+0000 7f74c17c4740 1 client.27427 shutdown 2024-09-13T18:31:12.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.551+0000 7f74c17c4740 1 -- 172.21.15.114:0/1805827243 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a917867970 msgr2=0x55a91787a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:12.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.551+0000 7f74c17c4740 1 --2- 172.21.15.114:0/1805827243 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a917867970 0x55a91787a510 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f74a8098130 tx=0x7f74a8009a10 comp rx=0 tx=0).stop 2024-09-13T18:31:12.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.555+0000 7f74c17c4740 1 -- 172.21.15.114:0/1805827243 shutdown_connections 2024-09-13T18:31:12.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.555+0000 7f74c17c4740 1 --2- 172.21.15.114:0/1805827243 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55a9178da0b0 0x55a9178dc4a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.555+0000 7f74c17c4740 1 --2- 172.21.15.114:0/1805827243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a917872b30 0x55a91787af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.555+0000 7f74c17c4740 1 --2- 172.21.15.114:0/1805827243 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a917868290 0x55a91787aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.555+0000 7f74c17c4740 1 --2- 172.21.15.114:0/1805827243 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a917867970 0x55a91787a510 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.555+0000 7f74c17c4740 1 -- 172.21.15.114:0/1805827243 >> 172.21.15.114:0/1805827243 conn(0x55a917863ee0 msgr2=0x55a91786a540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:12.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.556+0000 7f7497fff640 0 client.27427 ms_handle_reset on v2:172.21.15.204:6838/344883912 2024-09-13T18:31:12.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.561+0000 7f74c17c4740 1 -- 172.21.15.114:0/1805827243 shutdown_connections 2024-09-13T18:31:12.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.561+0000 7f74c17c4740 1 -- 172.21.15.114:0/1805827243 wait complete. 2024-09-13T18:31:12.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.564+0000 7f74c17c4740 20 client.27427 trim_cache size 0 max 16384 2024-09-13T18:31:12.569 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:31:12.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.571+0000 7f74bd83a640 1 -- 172.21.15.114:0/1569788885 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f74a00a1260 msgr2=0x7f74a00a3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:12.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.571+0000 7f74bd83a640 1 --2- 172.21.15.114:0/1569788885 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f74a00a1260 0x7f74a00a3720 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f74b002aa30 tx=0x7f74b0005760 comp rx=0 tx=0).stop 2024-09-13T18:31:12.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.571+0000 7f74bd83a640 1 -- 172.21.15.114:0/1569788885 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74b81074b0 msgr2=0x7f74b8118830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:12.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.571+0000 7f74bd83a640 1 --2- 172.21.15.114:0/1569788885 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74b81074b0 0x7f74b8118830 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f74a800ba50 tx=0x7f74a800bf10 comp rx=0 tx=0).stop 2024-09-13T18:31:12.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.576+0000 7f74bd83a640 1 -- 172.21.15.114:0/1569788885 shutdown_connections 2024-09-13T18:31:12.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.576+0000 7f74bd83a640 1 --2- 172.21.15.114:0/1569788885 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f74a00a1260 0x7f74a00a3720 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.576+0000 7f74bd83a640 1 --2- 172.21.15.114:0/1569788885 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74b8119400 0x7f74b8112970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.576+0000 7f74bd83a640 1 --2- 172.21.15.114:0/1569788885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74b8107890 0x7f74b8118d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.576+0000 7f74bd83a640 1 --2- 172.21.15.114:0/1569788885 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74b81074b0 0x7f74b8118830 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.576+0000 7f74bd83a640 1 -- 172.21.15.114:0/1569788885 >> 172.21.15.114:0/1569788885 conn(0x7f74b806af40 msgr2=0x7f74b8070180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:12.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.579+0000 7f74bd83a640 1 -- 172.21.15.114:0/1569788885 shutdown_connections 2024-09-13T18:31:12.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.579+0000 7f74bd83a640 1 -- 172.21.15.114:0/1569788885 wait complete. 2024-09-13T18:31:12.615 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-09-13T18:31:12.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.856+0000 7f3172b9c640 1 -- 172.21.15.114:0/181208067 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f316c119400 msgr2=0x7f316c11b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:12.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.856+0000 7f3172b9c640 1 --2- 172.21.15.114:0/181208067 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f316c119400 0x7f316c11b7f0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f315c009be0 tx=0x7f315c02a6f0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.860+0000 7f3172b9c640 1 -- 172.21.15.114:0/181208067 shutdown_connections 2024-09-13T18:31:12.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.860+0000 7f3172b9c640 1 --2- 172.21.15.114:0/181208067 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f316c119400 0x7f316c11b7f0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.860+0000 7f3172b9c640 1 --2- 172.21.15.114:0/181208067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f316c107920 0x7f316c107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.860+0000 7f3172b9c640 1 --2- 172.21.15.114:0/181208067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f316c107540 0x7f316c10e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.860+0000 7f3172b9c640 1 -- 172.21.15.114:0/181208067 >> 172.21.15.114:0/181208067 conn(0x7f316c06afe0 msgr2=0x7f316c06b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:12.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.868+0000 7f3172b9c640 1 -- 172.21.15.114:0/181208067 shutdown_connections 2024-09-13T18:31:12.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.869+0000 7f3172b9c640 1 -- 172.21.15.114:0/181208067 wait complete. 2024-09-13T18:31:12.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.869+0000 7f3172b9c640 1 Processor -- start 2024-09-13T18:31:12.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.869+0000 7f3172b9c640 1 -- start start 2024-09-13T18:31:12.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.870+0000 7f3172b9c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f316c107540 0x7f316c1123c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:12.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.870+0000 7f3172b9c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f316c107920 0x7f316c112900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:12.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.870+0000 7f3172b9c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f316c119400 0x7f316c116ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:12.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.870+0000 7f3172b9c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316c113290 con 0x7f316c107920 2024-09-13T18:31:12.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.870+0000 7f3172b9c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316c117410 con 0x7f316c119400 2024-09-13T18:31:12.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.870+0000 7f3172b9c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316c1175d0 con 0x7f316c107540 2024-09-13T18:31:12.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.872+0000 7f3171139640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f316c119400 0x7f316c116ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:12.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.873+0000 7f316bfff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f316c107920 0x7f316c112900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:12.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.873+0000 7f316bfff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f316c107920 0x7f316c112900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51928/0 (socket says 172.21.15.114:51928) 2024-09-13T18:31:12.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.873+0000 7f316bfff640 1 -- 172.21.15.114:0/2826608686 learned_addr learned my addr 172.21.15.114:0/2826608686 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:12.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.874+0000 7f316bfff640 1 -- 172.21.15.114:0/2826608686 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f316c107540 msgr2=0x7f316c1123c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:31:12.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.874+0000 7f316bfff640 1 --2- 172.21.15.114:0/2826608686 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f316c107540 0x7f316c1123c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.874+0000 7f316bfff640 1 -- 172.21.15.114:0/2826608686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f316c119400 msgr2=0x7f316c116ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:12.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.874+0000 7f316bfff640 1 --2- 172.21.15.114:0/2826608686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f316c119400 0x7f316c116ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.874+0000 7f316bfff640 1 -- 172.21.15.114:0/2826608686 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f315c009840 con 0x7f316c107920 2024-09-13T18:31:12.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.874+0000 7f316bfff640 1 --2- 172.21.15.114:0/2826608686 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f316c107920 0x7f316c112900 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f315400e9f0 tx=0x7f315400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:12.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.875+0000 7f3169ffb640 1 -- 172.21.15.114:0/2826608686 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315400cd80 con 0x7f316c107920 2024-09-13T18:31:12.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.876+0000 7f3169ffb640 1 -- 172.21.15.114:0/2826608686 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f31540108c0 con 0x7f316c107920 2024-09-13T18:31:12.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.876+0000 7f3169ffb640 1 -- 172.21.15.114:0/2826608686 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31540186f0 con 0x7f316c107920 2024-09-13T18:31:12.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.877+0000 7f3172b9c640 1 -- 172.21.15.114:0/2826608686 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f316c1178b0 con 0x7f316c107920 2024-09-13T18:31:12.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.877+0000 7f3172b9c640 1 -- 172.21.15.114:0/2826608686 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f316c124980 con 0x7f316c107920 2024-09-13T18:31:12.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.880+0000 7f31758c4740 1 Processor -- start 2024-09-13T18:31:12.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.880+0000 7f31758c4740 1 -- start start 2024-09-13T18:31:12.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.881+0000 7f31758c4740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606cb66e790 0x5606cb66eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:12.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.881+0000 7f31758c4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606cb66f0b0 0x5606cb679410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:12.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.881+0000 7f31758c4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606cb679950 0x5606cb67bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:12.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.881+0000 7f31758c4740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606cb5d8cc0 con 0x5606cb66e790 2024-09-13T18:31:12.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.881+0000 7f31758c4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606cb5d6450 con 0x5606cb66f0b0 2024-09-13T18:31:12.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.881+0000 7f31758c4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606cb5d1d40 con 0x5606cb679950 2024-09-13T18:31:12.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.883+0000 7f3171139640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606cb66e790 0x5606cb66eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:12.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.883+0000 7f3171139640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606cb66e790 0x5606cb66eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51936/0 (socket says 172.21.15.114:51936) 2024-09-13T18:31:12.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.883+0000 7f3171139640 1 -- 172.21.15.114:0/263392544 learned_addr learned my addr 172.21.15.114:0/263392544 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:12.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.887+0000 7f3170938640 1 --2- 172.21.15.114:0/263392544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606cb66f0b0 0x5606cb679410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:12.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.887+0000 7f316bfff640 1 --2- 172.21.15.114:0/263392544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606cb679950 0x5606cb67bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:12.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.887+0000 7f3170938640 1 -- 172.21.15.114:0/263392544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606cb679950 msgr2=0x5606cb67bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:12.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.887+0000 7f3170938640 1 --2- 172.21.15.114:0/263392544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606cb679950 0x5606cb67bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.887+0000 7f3170938640 1 -- 172.21.15.114:0/263392544 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606cb66e790 msgr2=0x5606cb66eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:12.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.887+0000 7f3170938640 1 --2- 172.21.15.114:0/263392544 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606cb66e790 0x5606cb66eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.887+0000 7f3170938640 1 -- 172.21.15.114:0/263392544 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606cb599620 con 0x5606cb66f0b0 2024-09-13T18:31:12.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.887+0000 7f3170938640 1 --2- 172.21.15.114:0/263392544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606cb66f0b0 0x5606cb679410 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f316400b9d0 tx=0x7f316400be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2c91eb778de824e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:12.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.889+0000 7f3162ffd640 1 -- 172.21.15.114:0/263392544 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3164004220 con 0x5606cb66f0b0 2024-09-13T18:31:12.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.889+0000 7f3162ffd640 1 -- 172.21.15.114:0/263392544 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f316400de70 con 0x5606cb66f0b0 2024-09-13T18:31:12.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.889+0000 7f3162ffd640 1 -- 172.21.15.114:0/263392544 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31640047a0 con 0x5606cb66f0b0 2024-09-13T18:31:12.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.890+0000 7f31758c4740 1 -- 172.21.15.114:0/263392544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606cb66f0b0 msgr2=0x5606cb679410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:12.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.890+0000 7f31758c4740 1 --2- 172.21.15.114:0/263392544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606cb66f0b0 0x5606cb679410 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f316400b9d0 tx=0x7f316400be90 comp rx=0 tx=0).stop 2024-09-13T18:31:12.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.890+0000 7f3169ffb640 1 -- 172.21.15.114:0/2826608686 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f3154010430 con 0x7f316c107920 2024-09-13T18:31:12.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.891+0000 7f3169ffb640 1 --2- 172.21.15.114:0/2826608686 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f31580a1410 0x7f31580a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:12.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.891+0000 7f3169ffb640 1 -- 172.21.15.114:0/2826608686 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f3154014070 con 0x7f316c107920 2024-09-13T18:31:12.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.892+0000 7f3171139640 1 --2- 172.21.15.114:0/2826608686 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f31580a1410 0x7f31580a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:12.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.893+0000 7f3171139640 1 --2- 172.21.15.114:0/2826608686 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f31580a1410 0x7f31580a38d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f315c031040 tx=0x7f315c0303a0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:12.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.896+0000 7f31758c4740 1 -- 172.21.15.114:0/263392544 shutdown_connections 2024-09-13T18:31:12.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.896+0000 7f31758c4740 1 --2- 172.21.15.114:0/263392544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606cb679950 0x5606cb67bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.896+0000 7f31758c4740 1 --2- 172.21.15.114:0/263392544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606cb66f0b0 0x5606cb679410 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.896+0000 7f31758c4740 1 --2- 172.21.15.114:0/263392544 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606cb66e790 0x5606cb66eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.896+0000 7f31758c4740 1 -- 172.21.15.114:0/263392544 >> 172.21.15.114:0/263392544 conn(0x5606cb66ad00 msgr2=0x5606cb66b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:12.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.896+0000 7f31758c4740 1 -- 172.21.15.114:0/263392544 shutdown_connections 2024-09-13T18:31:12.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.896+0000 7f31758c4740 1 -- 172.21.15.114:0/263392544 wait complete. 2024-09-13T18:31:12.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.896+0000 7f31758c4740 1 -- start start 2024-09-13T18:31:12.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.896+0000 7f31758c4740 1 Processor -- start 2024-09-13T18:31:12.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.897+0000 7f31758c4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606cb66e790 0x5606cb678ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:12.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.897+0000 7f31758c4740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606cb66f0b0 0x5606cb676280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:12.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.897+0000 7f31758c4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606cb679950 0x5606cb6767c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:12.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.897+0000 7f31758c4740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606cb590180 con 0x5606cb66f0b0 2024-09-13T18:31:12.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.897+0000 7f31758c4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606cb2aa430 con 0x5606cb66e790 2024-09-13T18:31:12.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.897+0000 7f31758c4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606cb5555e0 con 0x5606cb679950 2024-09-13T18:31:12.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.897+0000 7f31758c4740 10 client.0 _ll_register_callbacks cb 0x5606cb5c1aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:31:12.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.898+0000 7f3170938640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606cb66e790 0x5606cb678ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:12.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.898+0000 7f3171139640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606cb679950 0x5606cb6767c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:12.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.898+0000 7f3170938640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606cb66e790 0x5606cb678ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:45918/0 (socket says 172.21.15.114:45918) 2024-09-13T18:31:12.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.898+0000 7f3170938640 1 -- 172.21.15.114:0/1061748032 learned_addr learned my addr 172.21.15.114:0/1061748032 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:12.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.898+0000 7f316bfff640 1 --2- 172.21.15.114:0/1061748032 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606cb66f0b0 0x5606cb676280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:12.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.899+0000 7f316bfff640 1 -- 172.21.15.114:0/1061748032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606cb679950 msgr2=0x5606cb6767c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:12.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.899+0000 7f316bfff640 1 --2- 172.21.15.114:0/1061748032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606cb679950 0x5606cb6767c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.899+0000 7f316bfff640 1 -- 172.21.15.114:0/1061748032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606cb66e790 msgr2=0x5606cb678ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:12.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.899+0000 7f316bfff640 1 --2- 172.21.15.114:0/1061748032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606cb66e790 0x5606cb678ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.899+0000 7f316bfff640 1 -- 172.21.15.114:0/1061748032 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f316400b600 con 0x5606cb66f0b0 2024-09-13T18:31:12.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.899+0000 7f316bfff640 1 --2- 172.21.15.114:0/1061748032 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606cb66f0b0 0x5606cb676280 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f31540cd950 tx=0x7f31540106e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efd64acd7c3c900d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:12.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.899+0000 7f314bfff640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:31:12.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.899+0000 7f314bfff640 1 -- 172.21.15.114:0/1061748032 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315409ca00 con 0x5606cb66f0b0 2024-09-13T18:31:12.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.899+0000 7f314bfff640 1 -- 172.21.15.114:0/1061748032 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f315409d310 con 0x5606cb66f0b0 2024-09-13T18:31:12.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.900+0000 7f31758c4740 1 -- 172.21.15.114:0/1061748032 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5606cb5fbb80 con 0x5606cb66f0b0 2024-09-13T18:31:12.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.900+0000 7f314bfff640 1 -- 172.21.15.114:0/1061748032 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31540cdde0 con 0x5606cb66f0b0 2024-09-13T18:31:12.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.900+0000 7f31758c4740 10 client.18030 fetch_fsmap learned FSMap version 34 2024-09-13T18:31:12.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.900+0000 7f31758c4740 1 -- 172.21.15.114:0/1061748032 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5606cb5d03c0 con 0x5606cb66f0b0 2024-09-13T18:31:12.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.901+0000 7f314bfff640 1 -- 172.21.15.114:0/1061748032 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5606cb5fbb80 con 0x5606cb66f0b0 2024-09-13T18:31:12.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.901+0000 7f314bfff640 1 -- 172.21.15.114:0/1061748032 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3154086c20 con 0x5606cb66f0b0 2024-09-13T18:31:12.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.901+0000 7f31758c4740 10 client.18030 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:31:12.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.901+0000 7f31758c4740 10 client.18030 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-13T18:31:12.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.901+0000 7f31758c4740 20 client.18030 populate_metadata read hostname 'smithi114' 2024-09-13T18:31:12.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.901+0000 7f31758c4740 1 --2- 172.21.15.114:0/1061748032 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x5606cb6891f0 0x5606cb68b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:12.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.901+0000 7f31758c4740 4 client.18030 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:31:12.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.901+0000 7f31758c4740 1 -- 172.21.15.114:0/1061748032 --> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5606cb5c41f0 con 0x5606cb6891f0 2024-09-13T18:31:12.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.905+0000 7f3170938640 1 --2- 172.21.15.114:0/1061748032 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x5606cb6891f0 0x5606cb68b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:12.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.915+0000 7f3170938640 1 --2- 172.21.15.114:0/1061748032 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x5606cb6891f0 0x5606cb68b630 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e1994f05a9c461a2 server_cookie=62edaa27b6c713a3 in_seq=0 out_seq=0 2024-09-13T18:31:12.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.915+0000 7f314bfff640 10 client.18030 ms_handle_connect on v2:172.21.15.134:6836/891692210 2024-09-13T18:31:12.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.919+0000 7f314bfff640 1 -- 172.21.15.114:0/1061748032 <== mds.1 v2:172.21.15.134:6836/891692210 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5606cb5c41f0 con 0x5606cb6891f0 2024-09-13T18:31:12.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.919+0000 7f314bfff640 10 client.18030 handle_command_reply: tid=0 2024-09-13T18:31:12.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.919+0000 7f31758c4740 1 client.18030 shutdown 2024-09-13T18:31:12.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.921+0000 7f31758c4740 1 -- 172.21.15.114:0/1061748032 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606cb66f0b0 msgr2=0x5606cb676280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:12.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.921+0000 7f31758c4740 1 --2- 172.21.15.114:0/1061748032 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606cb66f0b0 0x5606cb676280 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f31540cd950 tx=0x7f31540106e0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.922+0000 7f31758c4740 1 -- 172.21.15.114:0/1061748032 shutdown_connections 2024-09-13T18:31:12.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.922+0000 7f31758c4740 1 --2- 172.21.15.114:0/1061748032 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x5606cb6891f0 0x5606cb68b630 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.922+0000 7f31758c4740 1 --2- 172.21.15.114:0/1061748032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606cb679950 0x5606cb6767c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.922+0000 7f31758c4740 1 --2- 172.21.15.114:0/1061748032 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606cb66f0b0 0x5606cb676280 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.922+0000 7f31758c4740 1 --2- 172.21.15.114:0/1061748032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606cb66e790 0x5606cb678ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.923+0000 7f31758c4740 1 -- 172.21.15.114:0/1061748032 >> 172.21.15.114:0/1061748032 conn(0x5606cb66ad00 msgr2=0x5606cb670a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:12.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.923+0000 7f314bfff640 0 client.18030 ms_handle_reset on v2:172.21.15.134:6836/891692210 2024-09-13T18:31:12.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.923+0000 7f31758c4740 1 -- 172.21.15.114:0/1061748032 shutdown_connections 2024-09-13T18:31:12.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.923+0000 7f31758c4740 1 -- 172.21.15.114:0/1061748032 wait complete. 2024-09-13T18:31:12.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.924+0000 7f31758c4740 20 client.18030 trim_cache size 0 max 16384 2024-09-13T18:31:12.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.951+0000 7f31758c4740 1 Processor -- start 2024-09-13T18:31:12.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.951+0000 7f31758c4740 1 -- start start 2024-09-13T18:31:12.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.952+0000 7f31758c4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606cb66e790 0x5606cb676240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:12.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.952+0000 7f31758c4740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606cb66f0b0 0x5606cb676780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:12.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.953+0000 7f316bfff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606cb66f0b0 0x5606cb676780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:12.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.953+0000 7f31758c4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606cb679950 0x5606cb66b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:12.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.953+0000 7f31758c4740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606cb6666b0 con 0x5606cb66f0b0 2024-09-13T18:31:12.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.953+0000 7f31758c4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606cb5c41f0 con 0x5606cb679950 2024-09-13T18:31:12.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.953+0000 7f31758c4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606cb67c400 con 0x5606cb66e790 2024-09-13T18:31:12.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.953+0000 7f3171139640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606cb679950 0x5606cb66b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:12.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.953+0000 7f3171139640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606cb679950 0x5606cb66b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:45922/0 (socket says 172.21.15.114:45922) 2024-09-13T18:31:12.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.953+0000 7f3171139640 1 -- 172.21.15.114:0/115313711 learned_addr learned my addr 172.21.15.114:0/115313711 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:12.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.953+0000 7f3171139640 1 -- 172.21.15.114:0/115313711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606cb66e790 msgr2=0x5606cb676240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:31:12.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.954+0000 7f3171139640 1 --2- 172.21.15.114:0/115313711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606cb66e790 0x5606cb676240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.954+0000 7f3171139640 1 -- 172.21.15.114:0/115313711 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606cb66f0b0 msgr2=0x5606cb676780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:12.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.954+0000 7f3171139640 1 --2- 172.21.15.114:0/115313711 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606cb66f0b0 0x5606cb676780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.954+0000 7f3171139640 1 -- 172.21.15.114:0/115313711 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606cb572770 con 0x5606cb679950 2024-09-13T18:31:12.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.954+0000 7f3171139640 1 --2- 172.21.15.114:0/115313711 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606cb679950 0x5606cb66b0e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f315c009be0 tx=0x7f315c03f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fbe32aee0ec9293 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:12.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.954+0000 7f31637fe640 1 -- 172.21.15.114:0/115313711 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315c030c10 con 0x5606cb679950 2024-09-13T18:31:12.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.955+0000 7f31637fe640 1 -- 172.21.15.114:0/115313711 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f315c041070 con 0x5606cb679950 2024-09-13T18:31:12.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.955+0000 7f31637fe640 1 -- 172.21.15.114:0/115313711 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315c031820 con 0x5606cb679950 2024-09-13T18:31:12.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.956+0000 7f31758c4740 1 -- 172.21.15.114:0/115313711 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606cb679950 msgr2=0x5606cb66b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:12.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.956+0000 7f31758c4740 1 --2- 172.21.15.114:0/115313711 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606cb679950 0x5606cb66b0e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f315c009be0 tx=0x7f315c03f000 comp rx=0 tx=0).stop 2024-09-13T18:31:12.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.959+0000 7f31758c4740 1 -- 172.21.15.114:0/115313711 shutdown_connections 2024-09-13T18:31:12.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.960+0000 7f31758c4740 1 --2- 172.21.15.114:0/115313711 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606cb679950 0x5606cb66b0e0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.960+0000 7f31758c4740 1 --2- 172.21.15.114:0/115313711 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606cb66f0b0 0x5606cb676780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.960+0000 7f31758c4740 1 --2- 172.21.15.114:0/115313711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606cb66e790 0x5606cb676240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.960+0000 7f31758c4740 1 -- 172.21.15.114:0/115313711 >> 172.21.15.114:0/115313711 conn(0x5606cb66ad00 msgr2=0x5606cb68ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:12.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.967+0000 7f31758c4740 1 -- 172.21.15.114:0/115313711 shutdown_connections 2024-09-13T18:31:12.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.968+0000 7f31758c4740 1 -- 172.21.15.114:0/115313711 wait complete. 2024-09-13T18:31:12.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.968+0000 7f31758c4740 1 -- start start 2024-09-13T18:31:12.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.968+0000 7f31758c4740 1 Processor -- start 2024-09-13T18:31:12.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.969+0000 7f31758c4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606cb66e790 0x5606cb6803d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:12.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.969+0000 7f31758c4740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606cb66f0b0 0x5606cb680910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:12.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.970+0000 7f31758c4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606cb679950 0x5606cb67ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:12.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.970+0000 7f31758c4740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606cb67d0e0 con 0x5606cb66f0b0 2024-09-13T18:31:12.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.970+0000 7f31758c4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606cb67d250 con 0x5606cb679950 2024-09-13T18:31:12.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.970+0000 7f31758c4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606cb67d410 con 0x5606cb66e790 2024-09-13T18:31:12.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.970+0000 7f3171139640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606cb679950 0x5606cb67ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:12.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.970+0000 7f31758c4740 10 client.0 _ll_register_callbacks cb 0x7f3164005f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:31:12.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.970+0000 7f3171139640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606cb679950 0x5606cb67ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:45938/0 (socket says 172.21.15.114:45938) 2024-09-13T18:31:12.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.970+0000 7f3171139640 1 -- 172.21.15.114:0/2047007287 learned_addr learned my addr 172.21.15.114:0/2047007287 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:12.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.970+0000 7f3170938640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606cb66e790 0x5606cb6803d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:12.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.975+0000 7f3171139640 1 -- 172.21.15.114:0/2047007287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606cb66e790 msgr2=0x5606cb6803d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:12.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.975+0000 7f3171139640 1 --2- 172.21.15.114:0/2047007287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606cb66e790 0x5606cb6803d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.975+0000 7f3171139640 1 -- 172.21.15.114:0/2047007287 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606cb66f0b0 msgr2=0x5606cb680910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:12.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.975+0000 7f3171139640 1 --2- 172.21.15.114:0/2047007287 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606cb66f0b0 0x5606cb680910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:12.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.975+0000 7f3171139640 1 -- 172.21.15.114:0/2047007287 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f316c11c2e0 con 0x5606cb679950 2024-09-13T18:31:12.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.976+0000 7f3171139640 1 --2- 172.21.15.114:0/2047007287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606cb679950 0x5606cb67ca10 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f315c009b30 tx=0x7f315c02a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=996bea2192033b19 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:12.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.978+0000 7f314bfff640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:31:12.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.979+0000 7f314bfff640 1 -- 172.21.15.114:0/2047007287 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315c056070 con 0x5606cb679950 2024-09-13T18:31:12.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.979+0000 7f314bfff640 1 -- 172.21.15.114:0/2047007287 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f315c043070 con 0x5606cb679950 2024-09-13T18:31:12.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.980+0000 7f31758c4740 1 -- 172.21.15.114:0/2047007287 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f316400de70 con 0x5606cb679950 2024-09-13T18:31:12.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.980+0000 7f314bfff640 1 -- 172.21.15.114:0/2047007287 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f315c05c660 con 0x5606cb679950 2024-09-13T18:31:12.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.982+0000 7f31758c4740 10 client.27461 fetch_fsmap learned FSMap version 34 2024-09-13T18:31:12.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.982+0000 7f31758c4740 1 -- 172.21.15.114:0/2047007287 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5606cb597e80 con 0x5606cb679950 2024-09-13T18:31:12.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.983+0000 7f314bfff640 1 -- 172.21.15.114:0/2047007287 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f316400de70 con 0x5606cb679950 2024-09-13T18:31:12.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.983+0000 7f314bfff640 1 -- 172.21.15.114:0/2047007287 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f315c064070 con 0x5606cb679950 2024-09-13T18:31:12.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.983+0000 7f31758c4740 10 client.27461 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:31:12.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.983+0000 7f31758c4740 10 client.27461 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-13T18:31:12.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.983+0000 7f31758c4740 20 client.27461 populate_metadata read hostname 'smithi114' 2024-09-13T18:31:12.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.983+0000 7f31758c4740 1 --2- 172.21.15.114:0/2047007287 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x5606cb6e0390 0x5606cb6e27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:12.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.983+0000 7f31758c4740 4 client.27461 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:31:12.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.983+0000 7f31758c4740 1 -- 172.21.15.114:0/2047007287 --> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5606cb2c54e0 con 0x5606cb6e0390 2024-09-13T18:31:12.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.986+0000 7f3170938640 1 --2- 172.21.15.114:0/2047007287 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x5606cb6e0390 0x5606cb6e27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:12.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.997+0000 7f3170938640 1 --2- 172.21.15.114:0/2047007287 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x5606cb6e0390 0x5606cb6e27b0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=13428dce1fcdbcf4 server_cookie=f88faa92ddd45897 in_seq=0 out_seq=0 2024-09-13T18:31:13.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.999+0000 7f314bfff640 10 client.27461 ms_handle_connect on v2:172.21.15.134:6836/891692210 2024-09-13T18:31:13.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.999+0000 7f314bfff640 1 -- 172.21.15.114:0/2047007287 <== mds.1 v2:172.21.15.134:6836/891692210 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5606cb2c54e0 con 0x5606cb6e0390 2024-09-13T18:31:13.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.999+0000 7f314bfff640 10 client.27461 handle_command_reply: tid=0 2024-09-13T18:31:13.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:12.999+0000 7f31758c4740 1 client.27461 shutdown 2024-09-13T18:31:13.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.004+0000 7f31758c4740 1 -- 172.21.15.114:0/2047007287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606cb679950 msgr2=0x5606cb67ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:13.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.004+0000 7f31758c4740 1 --2- 172.21.15.114:0/2047007287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606cb679950 0x5606cb67ca10 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f315c009b30 tx=0x7f315c02a6f0 comp rx=0 tx=0).stop 2024-09-13T18:31:13.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.004+0000 7f31758c4740 1 -- 172.21.15.114:0/2047007287 shutdown_connections 2024-09-13T18:31:13.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.004+0000 7f31758c4740 1 --2- 172.21.15.114:0/2047007287 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x5606cb6e0390 0x5606cb6e27b0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:13.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.004+0000 7f31758c4740 1 --2- 172.21.15.114:0/2047007287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606cb679950 0x5606cb67ca10 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:13.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.004+0000 7f31758c4740 1 --2- 172.21.15.114:0/2047007287 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606cb66f0b0 0x5606cb680910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:13.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.004+0000 7f31758c4740 1 --2- 172.21.15.114:0/2047007287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606cb66e790 0x5606cb6803d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:13.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.004+0000 7f31758c4740 1 -- 172.21.15.114:0/2047007287 >> 172.21.15.114:0/2047007287 conn(0x5606cb66ad00 msgr2=0x5606cb66b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:13.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.011+0000 7f314bfff640 0 client.27461 ms_handle_reset on v2:172.21.15.134:6836/891692210 2024-09-13T18:31:13.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.014+0000 7f31758c4740 1 -- 172.21.15.114:0/2047007287 shutdown_connections 2024-09-13T18:31:13.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.017+0000 7f31758c4740 1 -- 172.21.15.114:0/2047007287 wait complete. 2024-09-13T18:31:13.024 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:31:13.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.018+0000 7f31758c4740 20 client.27461 trim_cache size 0 max 16384 2024-09-13T18:31:13.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.020+0000 7f3172b9c640 1 -- 172.21.15.114:0/2826608686 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f31580a1410 msgr2=0x7f31580a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:13.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.020+0000 7f3172b9c640 1 --2- 172.21.15.114:0/2826608686 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f31580a1410 0x7f31580a38d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f315c031040 tx=0x7f315c0303a0 comp rx=0 tx=0).stop 2024-09-13T18:31:13.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.021+0000 7f3172b9c640 1 -- 172.21.15.114:0/2826608686 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f316c107920 msgr2=0x7f316c112900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:13.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.021+0000 7f3172b9c640 1 --2- 172.21.15.114:0/2826608686 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f316c107920 0x7f316c112900 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f315400e9f0 tx=0x7f315400eeb0 comp rx=0 tx=0).stop 2024-09-13T18:31:13.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.022+0000 7f3172b9c640 1 -- 172.21.15.114:0/2826608686 shutdown_connections 2024-09-13T18:31:13.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.022+0000 7f3172b9c640 1 --2- 172.21.15.114:0/2826608686 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f31580a1410 0x7f31580a38d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:13.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.022+0000 7f3172b9c640 1 --2- 172.21.15.114:0/2826608686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f316c119400 0x7f316c116ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:13.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.022+0000 7f3172b9c640 1 --2- 172.21.15.114:0/2826608686 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f316c107920 0x7f316c112900 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:13.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.022+0000 7f3172b9c640 1 --2- 172.21.15.114:0/2826608686 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f316c107540 0x7f316c1123c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:13.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.022+0000 7f3172b9c640 1 -- 172.21.15.114:0/2826608686 >> 172.21.15.114:0/2826608686 conn(0x7f316c06afe0 msgr2=0x7f316c06eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:13.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.025+0000 7f3172b9c640 1 -- 172.21.15.114:0/2826608686 shutdown_connections 2024-09-13T18:31:13.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.028+0000 7f3172b9c640 1 -- 172.21.15.114:0/2826608686 wait complete. 2024-09-13T18:31:13.070 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-09-13T18:31:13.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.321+0000 7f76d1153640 1 -- 172.21.15.114:0/3750493961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76cc119400 msgr2=0x7f76cc11b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:13.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.321+0000 7f76d1153640 1 --2- 172.21.15.114:0/3750493961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76cc119400 0x7f76cc11b7f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f76c4009be0 tx=0x7f76c402a710 comp rx=0 tx=0).stop 2024-09-13T18:31:13.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.323+0000 7f76d1153640 1 -- 172.21.15.114:0/3750493961 shutdown_connections 2024-09-13T18:31:13.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.323+0000 7f76d1153640 1 --2- 172.21.15.114:0/3750493961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76cc119400 0x7f76cc11b7f0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:13.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.323+0000 7f76d1153640 1 --2- 172.21.15.114:0/3750493961 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76cc107920 0x7f76cc107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:13.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.323+0000 7f76d1153640 1 --2- 172.21.15.114:0/3750493961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76cc107540 0x7f76cc10e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:13.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.323+0000 7f76d1153640 1 -- 172.21.15.114:0/3750493961 >> 172.21.15.114:0/3750493961 conn(0x7f76cc06afe0 msgr2=0x7f76cc06b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:13.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.330+0000 7f76d1153640 1 -- 172.21.15.114:0/3750493961 shutdown_connections 2024-09-13T18:31:13.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.332+0000 7f76d1153640 1 -- 172.21.15.114:0/3750493961 wait complete. 2024-09-13T18:31:13.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.332+0000 7f76d1153640 1 Processor -- start 2024-09-13T18:31:13.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.333+0000 7f76d1153640 1 -- start start 2024-09-13T18:31:13.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.333+0000 7f76d1153640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76cc107540 0x7f76cc1149d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:13.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.333+0000 7f76d1153640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76cc107920 0x7f76cc114f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:13.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.333+0000 7f76d1153640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76cc119400 0x7f76cc118360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:13.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.333+0000 7f76d1153640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76cc118960 con 0x7f76cc107540 2024-09-13T18:31:13.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.333+0000 7f76d1153640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76cc118ad0 con 0x7f76cc107920 2024-09-13T18:31:13.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.333+0000 7f76d1153640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76cc118c90 con 0x7f76cc119400 2024-09-13T18:31:13.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.335+0000 7f76cad9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76cc107540 0x7f76cc1149d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:13.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.335+0000 7f76cad9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76cc107540 0x7f76cc1149d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57206/0 (socket says 172.21.15.114:57206) 2024-09-13T18:31:13.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.335+0000 7f76cad9d640 1 -- 172.21.15.114:0/2605878796 learned_addr learned my addr 172.21.15.114:0/2605878796 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:13.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.335+0000 7f76cb59e640 1 --2- 172.21.15.114:0/2605878796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76cc119400 0x7f76cc118360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:13.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.335+0000 7f76ca59c640 1 --2- 172.21.15.114:0/2605878796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76cc107920 0x7f76cc114f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:13.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.336+0000 7f76ca59c640 1 -- 172.21.15.114:0/2605878796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76cc119400 msgr2=0x7f76cc118360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:13.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.336+0000 7f76ca59c640 1 --2- 172.21.15.114:0/2605878796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76cc119400 0x7f76cc118360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:13.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.336+0000 7f76ca59c640 1 -- 172.21.15.114:0/2605878796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76cc107540 msgr2=0x7f76cc1149d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:13.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.336+0000 7f76ca59c640 1 --2- 172.21.15.114:0/2605878796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76cc107540 0x7f76cc1149d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:13.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.336+0000 7f76ca59c640 1 -- 172.21.15.114:0/2605878796 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c4009840 con 0x7f76cc107920 2024-09-13T18:31:13.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.336+0000 7f76ca59c640 1 --2- 172.21.15.114:0/2605878796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76cc107920 0x7f76cc114f10 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f76bc01e990 tx=0x7f76bc01ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:13.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.338+0000 7f76c3fff640 1 -- 172.21.15.114:0/2605878796 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76bc01aa80 con 0x7f76cc107920 2024-09-13T18:31:13.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.338+0000 7f76d1153640 1 -- 172.21.15.114:0/2605878796 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76cc118f70 con 0x7f76cc107920 2024-09-13T18:31:13.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.338+0000 7f76d1153640 1 -- 172.21.15.114:0/2605878796 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76cc1a9600 con 0x7f76cc107920 2024-09-13T18:31:13.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.339+0000 7f76c3fff640 1 -- 172.21.15.114:0/2605878796 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f76bc01abe0 con 0x7f76cc107920 2024-09-13T18:31:13.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.339+0000 7f76c3fff640 1 -- 172.21.15.114:0/2605878796 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76bc020630 con 0x7f76cc107920 2024-09-13T18:31:13.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.343+0000 7f76d3e73740 1 Processor -- start 2024-09-13T18:31:13.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.343+0000 7f76d3e73740 1 -- start start 2024-09-13T18:31:13.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.344+0000 7f76d3e73740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605e3694790 0x5605e3694b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:13.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.344+0000 7f76d3e73740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605e36950b0 0x5605e369f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:13.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.344+0000 7f76d3e73740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605e369f950 0x5605e36a1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:13.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.344+0000 7f76d3e73740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605e35fecc0 con 0x5605e369f950 2024-09-13T18:31:13.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.344+0000 7f76d3e73740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605e35fc450 con 0x5605e36950b0 2024-09-13T18:31:13.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.344+0000 7f76d3e73740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605e35f7d40 con 0x5605e3694790 2024-09-13T18:31:13.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.344+0000 7f76cad9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605e36950b0 0x5605e369f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:13.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.344+0000 7f76cad9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605e36950b0 0x5605e369f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:41360/0 (socket says 172.21.15.114:41360) 2024-09-13T18:31:13.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.344+0000 7f76cad9d640 1 -- 172.21.15.114:0/3093920660 learned_addr learned my addr 172.21.15.114:0/3093920660 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:13.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.344+0000 7f76cb59e640 1 --2- 172.21.15.114:0/3093920660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605e3694790 0x5605e3694b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:13.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.345+0000 7f76c3fff640 1 -- 172.21.15.114:0/2605878796 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f76bc021470 con 0x7f76cc107920 2024-09-13T18:31:13.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.346+0000 7f76c3fff640 1 --2- 172.21.15.114:0/2605878796 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f76b80a1410 0x7f76b80a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:13.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.346+0000 7f76c3fff640 1 -- 172.21.15.114:0/2605878796 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f76bc0ddae0 con 0x7f76cc107920 2024-09-13T18:31:13.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.347+0000 7f76ca59c640 1 --2- 172.21.15.114:0/3093920660 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605e369f950 0x5605e36a1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:13.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.347+0000 7f76ca59c640 1 -- 172.21.15.114:0/3093920660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605e3694790 msgr2=0x5605e3694b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:13.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.347+0000 7f76ca59c640 1 --2- 172.21.15.114:0/3093920660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605e3694790 0x5605e3694b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:13.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.347+0000 7f76ca59c640 1 -- 172.21.15.114:0/3093920660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605e36950b0 msgr2=0x5605e369f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:13.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.347+0000 7f76ca59c640 1 --2- 172.21.15.114:0/3093920660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605e36950b0 0x5605e369f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:13.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.347+0000 7f76ca59c640 1 -- 172.21.15.114:0/3093920660 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605e35bf620 con 0x5605e369f950 2024-09-13T18:31:13.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.348+0000 7f76ca59c640 1 --2- 172.21.15.114:0/3093920660 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605e369f950 0x5605e36a1d40 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f76bc0dadd0 tx=0x7f76bc0aa730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=838d99a0fc8e1e06 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:13.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.348+0000 7f76cb59e640 1 --2- 172.21.15.114:0/2605878796 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f76b80a1410 0x7f76b80a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:13.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.349+0000 7f76cb59e640 1 --2- 172.21.15.114:0/2605878796 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f76b80a1410 0x7f76b80a38d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f76c4004af0 tx=0x7f76c4009500 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:13.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.351+0000 7f76c17fa640 1 -- 172.21.15.114:0/3093920660 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76bc0aa840 con 0x5605e369f950 2024-09-13T18:31:13.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.351+0000 7f76c17fa640 1 -- 172.21.15.114:0/3093920660 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f76bc0ae5c0 con 0x5605e369f950 2024-09-13T18:31:13.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.352+0000 7f76d3e73740 1 -- 172.21.15.114:0/3093920660 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605e369f950 msgr2=0x5605e36a1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:13.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.352+0000 7f76d3e73740 1 --2- 172.21.15.114:0/3093920660 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605e369f950 0x5605e36a1d40 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f76bc0dadd0 tx=0x7f76bc0aa730 comp rx=0 tx=0).stop 2024-09-13T18:31:13.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.357+0000 7f76d3e73740 1 -- 172.21.15.114:0/3093920660 shutdown_connections 2024-09-13T18:31:13.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.357+0000 7f76d3e73740 1 --2- 172.21.15.114:0/3093920660 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605e369f950 0x5605e36a1d40 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:13.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.357+0000 7f76d3e73740 1 --2- 172.21.15.114:0/3093920660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605e36950b0 0x5605e369f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:13.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.357+0000 7f76d3e73740 1 --2- 172.21.15.114:0/3093920660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605e3694790 0x5605e3694b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:13.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.357+0000 7f76d3e73740 1 -- 172.21.15.114:0/3093920660 >> 172.21.15.114:0/3093920660 conn(0x5605e3690d00 msgr2=0x5605e36910e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:13.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.361+0000 7f76d3e73740 1 -- 172.21.15.114:0/3093920660 shutdown_connections 2024-09-13T18:31:13.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.365+0000 7f76d3e73740 1 -- 172.21.15.114:0/3093920660 wait complete. 2024-09-13T18:31:13.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.365+0000 7f76d3e73740 1 -- start start 2024-09-13T18:31:13.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.365+0000 7f76d3e73740 1 Processor -- start 2024-09-13T18:31:13.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.366+0000 7f76d3e73740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605e3694790 0x5605e369ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:13.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.366+0000 7f76d3e73740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605e36950b0 0x5605e369f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:13.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.366+0000 7f76d3e73740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605e369f950 0x5605e369c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:13.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.367+0000 7f76d3e73740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605e35b6180 con 0x5605e36950b0 2024-09-13T18:31:13.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.367+0000 7f76d3e73740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605e32d0430 con 0x5605e3694790 2024-09-13T18:31:13.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.367+0000 7f76cad9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605e3694790 0x5605e369ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:13.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.367+0000 7f76cad9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605e3694790 0x5605e369ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:41366/0 (socket says 172.21.15.114:41366) 2024-09-13T18:31:13.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.367+0000 7f76cad9d640 1 -- 172.21.15.114:0/1838101254 learned_addr learned my addr 172.21.15.114:0/1838101254 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:13.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.367+0000 7f76d3e73740 1 -- 172.21.15.114:0/1838101254 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605e357b5e0 con 0x5605e369f950 2024-09-13T18:31:13.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.367+0000 7f76d3e73740 10 client.0 _ll_register_callbacks cb 0x5605e35e7aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:31:13.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.369+0000 7f76cad9d640 1 -- 172.21.15.114:0/1838101254 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605e369f950 msgr2=0x5605e369c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:13.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.369+0000 7f76cad9d640 1 --2- 172.21.15.114:0/1838101254 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605e369f950 0x5605e369c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:13.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.369+0000 7f76cad9d640 1 -- 172.21.15.114:0/1838101254 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605e36950b0 msgr2=0x5605e369f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:13.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.369+0000 7f76cad9d640 1 --2- 172.21.15.114:0/1838101254 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605e36950b0 0x5605e369f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:13.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.369+0000 7f76cad9d640 1 -- 172.21.15.114:0/1838101254 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76cc1a9600 con 0x5605e3694790 2024-09-13T18:31:13.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.371+0000 7f76cad9d640 1 --2- 172.21.15.114:0/1838101254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605e3694790 0x5605e369ee10 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f76b4010a50 tx=0x7f76b4010f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b150bbb811682e42 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:13.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.371+0000 7f76b27fc640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:31:13.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.375+0000 7f76b27fc640 1 -- 172.21.15.114:0/1838101254 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b4015070 con 0x5605e3694790 2024-09-13T18:31:13.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.375+0000 7f76b27fc640 1 -- 172.21.15.114:0/1838101254 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f76b400fbd0 con 0x5605e3694790 2024-09-13T18:31:13.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.377+0000 7f76d3e73740 1 -- 172.21.15.114:0/1838101254 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f76bc0ca270 con 0x5605e3694790 2024-09-13T18:31:13.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.378+0000 7f76b27fc640 1 -- 172.21.15.114:0/1838101254 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76b400fdb0 con 0x5605e3694790 2024-09-13T18:31:13.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.378+0000 7f76d3e73740 10 client.27473 fetch_fsmap learned FSMap version 34 2024-09-13T18:31:13.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.378+0000 7f76d3e73740 1 -- 172.21.15.114:0/1838101254 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5605e35f63c0 con 0x5605e3694790 2024-09-13T18:31:13.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.380+0000 7f76b27fc640 1 -- 172.21.15.114:0/1838101254 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f76bc0ca270 con 0x5605e3694790 2024-09-13T18:31:13.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.381+0000 7f76b27fc640 1 -- 172.21.15.114:0/1838101254 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f76b4004bc0 con 0x5605e3694790 2024-09-13T18:31:13.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.382+0000 7f76d3e73740 10 client.27473 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:31:13.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.382+0000 7f76d3e73740 10 client.27473 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:31:13.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.382+0000 7f76d3e73740 20 client.27473 populate_metadata read hostname 'smithi114' 2024-09-13T18:31:13.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.382+0000 7f76d3e73740 1 --2- 172.21.15.114:0/1838101254 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5605e36af390 0x5605e36b17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:13.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.382+0000 7f76d3e73740 4 client.27473 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:31:13.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.382+0000 7f76d3e73740 1 -- 172.21.15.114:0/1838101254 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5605e3621b80 con 0x5605e36af390 2024-09-13T18:31:13.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.385+0000 7f76ca59c640 1 --2- 172.21.15.114:0/1838101254 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5605e36af390 0x5605e36b17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:13.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.391+0000 7f76ca59c640 1 --2- 172.21.15.114:0/1838101254 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5605e36af390 0x5605e36b17f0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a74f6f7283008768 server_cookie=c19c985e993fbde7 in_seq=0 out_seq=0 2024-09-13T18:31:13.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.391+0000 7f76b27fc640 10 client.27473 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:31:13.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.392+0000 7f76b27fc640 1 -- 172.21.15.114:0/1838101254 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5605e3621b80 con 0x5605e36af390 2024-09-13T18:31:13.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.392+0000 7f76b27fc640 10 client.27473 handle_command_reply: tid=0 2024-09-13T18:31:13.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.392+0000 7f76d3e73740 1 client.27473 shutdown 2024-09-13T18:31:13.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.392+0000 7f76d3e73740 1 -- 172.21.15.114:0/1838101254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605e3694790 msgr2=0x5605e369ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:13.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.392+0000 7f76d3e73740 1 --2- 172.21.15.114:0/1838101254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605e3694790 0x5605e369ee10 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f76b4010a50 tx=0x7f76b4010f10 comp rx=0 tx=0).stop 2024-09-13T18:31:13.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.397+0000 7f76d3e73740 1 -- 172.21.15.114:0/1838101254 shutdown_connections 2024-09-13T18:31:13.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.397+0000 7f76d3e73740 1 --2- 172.21.15.114:0/1838101254 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5605e36af390 0x5605e36b17f0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:13.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.397+0000 7f76d3e73740 1 --2- 172.21.15.114:0/1838101254 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605e369f950 0x5605e369c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:13.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.397+0000 7f76d3e73740 1 --2- 172.21.15.114:0/1838101254 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605e36950b0 0x5605e369f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:13.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.397+0000 7f76d3e73740 1 --2- 172.21.15.114:0/1838101254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605e3694790 0x5605e369ee10 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:13.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.397+0000 7f76d3e73740 1 -- 172.21.15.114:0/1838101254 >> 172.21.15.114:0/1838101254 conn(0x5605e3690d00 msgr2=0x5605e368feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:13.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.398+0000 7f76b27fc640 0 client.27473 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:31:13.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.403+0000 7f76d3e73740 1 -- 172.21.15.114:0/1838101254 shutdown_connections 2024-09-13T18:31:13.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.409+0000 7f76d3e73740 1 -- 172.21.15.114:0/1838101254 wait complete. 2024-09-13T18:31:13.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.412+0000 7f76d3e73740 20 client.27473 trim_cache size 0 max 16384 2024-09-13T18:31:13.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.443+0000 7f76d3e73740 1 Processor -- start 2024-09-13T18:31:13.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.447+0000 7f76d3e73740 1 -- start start 2024-09-13T18:31:13.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.447+0000 7f76d3e73740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605e3694790 0x5605e36a6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:13.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.447+0000 7f76d3e73740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605e36950b0 0x5605e36a6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:13.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.447+0000 7f76d3e73740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605e369f950 0x5605e369a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:13.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.447+0000 7f76d3e73740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605e369d0e0 con 0x5605e3694790 2024-09-13T18:31:13.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.447+0000 7f76d3e73740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605e368c8e0 con 0x5605e369f950 2024-09-13T18:31:13.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.447+0000 7f76d3e73740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605e3694580 con 0x5605e36950b0 2024-09-13T18:31:13.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.448+0000 7f76cad9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605e3694790 0x5605e36a6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:13.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.448+0000 7f76cad9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605e3694790 0x5605e36a6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57226/0 (socket says 172.21.15.114:57226) 2024-09-13T18:31:13.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.448+0000 7f76cad9d640 1 -- 172.21.15.114:0/1598917470 learned_addr learned my addr 172.21.15.114:0/1598917470 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:13.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.449+0000 7f76cb59e640 1 --2- 172.21.15.114:0/1598917470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605e369f950 0x5605e369a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:13.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.450+0000 7f76cad9d640 1 -- 172.21.15.114:0/1598917470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605e36950b0 msgr2=0x5605e36a6c60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:31:13.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.450+0000 7f76cad9d640 1 --2- 172.21.15.114:0/1598917470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605e36950b0 0x5605e36a6c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:13.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.451+0000 7f76cad9d640 1 -- 172.21.15.114:0/1598917470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605e369f950 msgr2=0x5605e369a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:13.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.451+0000 7f76cad9d640 1 --2- 172.21.15.114:0/1598917470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605e369f950 0x5605e369a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:13.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.451+0000 7f76cad9d640 1 -- 172.21.15.114:0/1598917470 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605e3598770 con 0x5605e3694790 2024-09-13T18:31:13.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.459+0000 7f76cad9d640 1 --2- 172.21.15.114:0/1598917470 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605e3694790 0x5605e36a6720 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f76b4000c00 tx=0x7f76b400a4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6663ab78998feb9a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:13.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.460+0000 7f76c1ffb640 1 -- 172.21.15.114:0/1598917470 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b4015070 con 0x5605e3694790 2024-09-13T18:31:13.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.460+0000 7f76c1ffb640 1 -- 172.21.15.114:0/1598917470 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f76b4016040 con 0x5605e3694790 2024-09-13T18:31:13.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.461+0000 7f76d3e73740 1 -- 172.21.15.114:0/1598917470 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605e3694790 msgr2=0x5605e36a6720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:13.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.461+0000 7f76d3e73740 1 --2- 172.21.15.114:0/1598917470 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605e3694790 0x5605e36a6720 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f76b4000c00 tx=0x7f76b400a4f0 comp rx=0 tx=0).stop 2024-09-13T18:31:13.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.464+0000 7f76d3e73740 1 -- 172.21.15.114:0/1598917470 shutdown_connections 2024-09-13T18:31:13.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.464+0000 7f76d3e73740 1 --2- 172.21.15.114:0/1598917470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605e369f950 0x5605e369a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:13.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.464+0000 7f76d3e73740 1 --2- 172.21.15.114:0/1598917470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605e36950b0 0x5605e36a6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:13.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.464+0000 7f76d3e73740 1 --2- 172.21.15.114:0/1598917470 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605e3694790 0x5605e36a6720 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:13.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.464+0000 7f76d3e73740 1 -- 172.21.15.114:0/1598917470 >> 172.21.15.114:0/1598917470 conn(0x5605e3690d00 msgr2=0x5605e36910e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:13.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.472+0000 7f76d3e73740 1 -- 172.21.15.114:0/1598917470 shutdown_connections 2024-09-13T18:31:13.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.473+0000 7f76d3e73740 1 -- 172.21.15.114:0/1598917470 wait complete. 2024-09-13T18:31:13.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.473+0000 7f76d3e73740 1 -- start start 2024-09-13T18:31:13.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.473+0000 7f76d3e73740 1 Processor -- start 2024-09-13T18:31:13.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.474+0000 7f76d3e73740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605e3694790 0x5605e36a6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:13.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.474+0000 7f76d3e73740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605e36950b0 0x5605e36a4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:13.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.474+0000 7f76d3e73740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605e369f950 0x5605e36a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:13.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.474+0000 7f76d3e73740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605e36a7070 con 0x5605e369f950 2024-09-13T18:31:13.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.474+0000 7f76cad9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605e3694790 0x5605e36a6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:13.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.474+0000 7f76cad9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605e3694790 0x5605e36a6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:41390/0 (socket says 172.21.15.114:41390) 2024-09-13T18:31:13.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.474+0000 7f76cad9d640 1 -- 172.21.15.114:0/3278271568 learned_addr learned my addr 172.21.15.114:0/3278271568 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:13.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.474+0000 7f76ca59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605e36950b0 0x5605e36a4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:13.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.474+0000 7f76d3e73740 1 -- 172.21.15.114:0/3278271568 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605e36a5610 con 0x5605e3694790 2024-09-13T18:31:13.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.474+0000 7f76d3e73740 1 -- 172.21.15.114:0/3278271568 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605e36a5780 con 0x5605e36950b0 2024-09-13T18:31:13.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.475+0000 7f76d3e73740 10 client.0 _ll_register_callbacks cb 0x7f76bc01c560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:31:13.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.475+0000 7f76cb59e640 1 --2- 172.21.15.114:0/3278271568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605e369f950 0x5605e36a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:13.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.475+0000 7f76ca59c640 1 -- 172.21.15.114:0/3278271568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605e3694790 msgr2=0x5605e36a6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:13.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.475+0000 7f76ca59c640 1 --2- 172.21.15.114:0/3278271568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605e3694790 0x5605e36a6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:13.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.475+0000 7f76ca59c640 1 -- 172.21.15.114:0/3278271568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605e369f950 msgr2=0x5605e36a50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:13.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.475+0000 7f76ca59c640 1 --2- 172.21.15.114:0/3278271568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605e369f950 0x5605e36a50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:13.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.475+0000 7f76ca59c640 1 -- 172.21.15.114:0/3278271568 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5605e35f63c0 con 0x5605e36950b0 2024-09-13T18:31:13.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.475+0000 7f76ca59c640 1 --2- 172.21.15.114:0/3278271568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605e36950b0 0x5605e36a4b90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x5605e3695650 tx=0x7f76bc01c610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d26353a1a485397a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:13.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.475+0000 7f76b27fc640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:31:13.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.477+0000 7f76b27fc640 1 -- 172.21.15.114:0/3278271568 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76bc025070 con 0x5605e36950b0 2024-09-13T18:31:13.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.478+0000 7f76d3e73740 1 -- 172.21.15.114:0/3278271568 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f76bc0ae5c0 con 0x5605e36950b0 2024-09-13T18:31:13.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.478+0000 7f76b27fc640 1 -- 172.21.15.114:0/3278271568 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f76bc021720 con 0x5605e36950b0 2024-09-13T18:31:13.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.478+0000 7f76b27fc640 1 -- 172.21.15.114:0/3278271568 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76bc0aa8c0 con 0x5605e36950b0 2024-09-13T18:31:13.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.481+0000 7f76d3e73740 10 client.27445 fetch_fsmap learned FSMap version 34 2024-09-13T18:31:13.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.481+0000 7f76d3e73740 1 -- 172.21.15.114:0/3278271568 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5605e35bde80 con 0x5605e36950b0 2024-09-13T18:31:13.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.483+0000 7f76b27fc640 1 -- 172.21.15.114:0/3278271568 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f76bc0ae5c0 con 0x5605e36950b0 2024-09-13T18:31:13.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.483+0000 7f76b27fc640 1 -- 172.21.15.114:0/3278271568 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f76bc0db050 con 0x5605e36950b0 2024-09-13T18:31:13.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.483+0000 7f76d3e73740 10 client.27445 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:31:13.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.483+0000 7f76d3e73740 10 client.27445 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:31:13.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.483+0000 7f76d3e73740 20 client.27445 populate_metadata read hostname 'smithi114' 2024-09-13T18:31:13.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.483+0000 7f76d3e73740 1 --2- 172.21.15.114:0/3278271568 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5605e3706990 0x5605e3708db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:13.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.483+0000 7f76d3e73740 4 client.27445 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:31:13.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.483+0000 7f76d3e73740 1 -- 172.21.15.114:0/3278271568 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5605e35ea1f0 con 0x5605e3706990 2024-09-13T18:31:13.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.484+0000 7f76cad9d640 1 --2- 172.21.15.114:0/3278271568 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5605e3706990 0x5605e3708db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:13.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.484+0000 7f76cad9d640 1 --2- 172.21.15.114:0/3278271568 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5605e3706990 0x5605e3708db0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d474a48999ed56a9 server_cookie=82edb3affbab52b8 in_seq=0 out_seq=0 2024-09-13T18:31:13.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.484+0000 7f76b27fc640 10 client.27445 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:31:13.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.485+0000 7f76b27fc640 1 -- 172.21.15.114:0/3278271568 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5605e35ea1f0 con 0x5605e3706990 2024-09-13T18:31:13.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.485+0000 7f76b27fc640 10 client.27445 handle_command_reply: tid=0 2024-09-13T18:31:13.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.485+0000 7f76d3e73740 1 client.27445 shutdown 2024-09-13T18:31:13.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.490+0000 7f76d3e73740 1 -- 172.21.15.114:0/3278271568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605e36950b0 msgr2=0x5605e36a4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:13.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.490+0000 7f76d3e73740 1 --2- 172.21.15.114:0/3278271568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605e36950b0 0x5605e36a4b90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x5605e3695650 tx=0x7f76bc01c610 comp rx=0 tx=0).stop 2024-09-13T18:31:13.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.490+0000 7f76d3e73740 1 -- 172.21.15.114:0/3278271568 shutdown_connections 2024-09-13T18:31:13.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.490+0000 7f76d3e73740 1 --2- 172.21.15.114:0/3278271568 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5605e3706990 0x5605e3708db0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:13.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.490+0000 7f76d3e73740 1 --2- 172.21.15.114:0/3278271568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605e369f950 0x5605e36a50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:13.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.490+0000 7f76d3e73740 1 --2- 172.21.15.114:0/3278271568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5605e36950b0 0x5605e36a4b90 secure :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0x5605e3695650 tx=0x7f76bc01c610 comp rx=0 tx=0).stop 2024-09-13T18:31:13.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.490+0000 7f76d3e73740 1 --2- 172.21.15.114:0/3278271568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605e3694790 0x5605e36a6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:13.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.490+0000 7f76d3e73740 1 -- 172.21.15.114:0/3278271568 >> 172.21.15.114:0/3278271568 conn(0x5605e3690d00 msgr2=0x5605e369a920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:13.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.491+0000 7f76b27fc640 0 client.27445 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:31:13.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.491+0000 7f76d3e73740 1 -- 172.21.15.114:0/3278271568 shutdown_connections 2024-09-13T18:31:13.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.495+0000 7f76d3e73740 1 -- 172.21.15.114:0/3278271568 wait complete. 2024-09-13T18:31:13.498 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:31:13.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.496+0000 7f76d3e73740 20 client.27445 trim_cache size 0 max 16384 2024-09-13T18:31:13.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.496+0000 7f76d1153640 1 -- 172.21.15.114:0/2605878796 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f76b80a1410 msgr2=0x7f76b80a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:13.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.496+0000 7f76d1153640 1 --2- 172.21.15.114:0/2605878796 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f76b80a1410 0x7f76b80a38d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f76c4004af0 tx=0x7f76c4009500 comp rx=0 tx=0).stop 2024-09-13T18:31:13.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.497+0000 7f76d1153640 1 -- 172.21.15.114:0/2605878796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76cc107920 msgr2=0x7f76cc114f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:13.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.497+0000 7f76d1153640 1 --2- 172.21.15.114:0/2605878796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76cc107920 0x7f76cc114f10 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f76bc01e990 tx=0x7f76bc01ee50 comp rx=0 tx=0).stop 2024-09-13T18:31:13.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.497+0000 7f76d1153640 1 -- 172.21.15.114:0/2605878796 shutdown_connections 2024-09-13T18:31:13.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.497+0000 7f76d1153640 1 --2- 172.21.15.114:0/2605878796 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f76b80a1410 0x7f76b80a38d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:13.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.497+0000 7f76d1153640 1 --2- 172.21.15.114:0/2605878796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76cc119400 0x7f76cc118360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:13.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.497+0000 7f76d1153640 1 --2- 172.21.15.114:0/2605878796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76cc107920 0x7f76cc114f10 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:13.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.497+0000 7f76d1153640 1 --2- 172.21.15.114:0/2605878796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f76cc107540 0x7f76cc1149d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:13.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.497+0000 7f76d1153640 1 -- 172.21.15.114:0/2605878796 >> 172.21.15.114:0/2605878796 conn(0x7f76cc06afe0 msgr2=0x7f76cc06ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:13.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.500+0000 7f76d1153640 1 -- 172.21.15.114:0/2605878796 shutdown_connections 2024-09-13T18:31:13.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:13.500+0000 7f76d1153640 1 -- 172.21.15.114:0/2605878796 wait complete. 2024-09-13T18:31:13.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:13 smithi204 ceph-mon[34777]: pgmap v270: 129 pgs: 19 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 59 active+clean; 22 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 27 MiB/s wr, 589 op/s 2024-09-13T18:31:13.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:13 smithi114 ceph-mon[30897]: pgmap v270: 129 pgs: 19 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 59 active+clean; 22 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 27 MiB/s wr, 589 op/s 2024-09-13T18:31:14.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:13 smithi134 ceph-mon[34283]: pgmap v270: 129 pgs: 19 active+clean+snaptrim, 51 active+clean+snaptrim_wait, 59 active+clean; 22 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 27 MiB/s wr, 589 op/s 2024-09-13T18:31:14.533 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-13T18:31:14.534 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-09-13T18:31:14.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.750+0000 7f2674ce9640 1 -- 172.21.15.114:0/485647805 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2670103fd0 msgr2=0x7f267010bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:14.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.750+0000 7f2674ce9640 1 --2- 172.21.15.114:0/485647805 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2670103fd0 0x7f267010bb20 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f2668009cd0 tx=0x7f266802a8e0 comp rx=0 tx=0).stop 2024-09-13T18:31:14.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.750+0000 7f2674ce9640 1 -- 172.21.15.114:0/485647805 shutdown_connections 2024-09-13T18:31:14.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.750+0000 7f2674ce9640 1 --2- 172.21.15.114:0/485647805 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2670103fd0 0x7f267010bb20 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:14.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.750+0000 7f2674ce9640 1 --2- 172.21.15.114:0/485647805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26700ff830 0x7f2670103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:14.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.750+0000 7f2674ce9640 1 --2- 172.21.15.114:0/485647805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26700fee80 0x7f26700ff260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:14.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.750+0000 7f2674ce9640 1 -- 172.21.15.114:0/485647805 >> 172.21.15.114:0/485647805 conn(0x7f26700fa8d0 msgr2=0x7f26700fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:14.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.750+0000 7f2674ce9640 1 -- 172.21.15.114:0/485647805 shutdown_connections 2024-09-13T18:31:14.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.750+0000 7f2674ce9640 1 -- 172.21.15.114:0/485647805 wait complete. 2024-09-13T18:31:14.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.751+0000 7f2674ce9640 1 Processor -- start 2024-09-13T18:31:14.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.751+0000 7f2674ce9640 1 -- start start 2024-09-13T18:31:14.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.751+0000 7f2674ce9640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26700fee80 0x7f2670197030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:14.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.751+0000 7f2674ce9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2670103fd0 0x7f2670197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:14.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.751+0000 7f2674ce9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2670197ab0 0x7f26701a20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:14.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.751+0000 7f2674ce9640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26701a2750 con 0x7f26700fee80 2024-09-13T18:31:14.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.751+0000 7f2674ce9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26701a28c0 con 0x7f2670103fd0 2024-09-13T18:31:14.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.751+0000 7f2674ce9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26701a2a80 con 0x7f2670197ab0 2024-09-13T18:31:14.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.751+0000 7f266e59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26700fee80 0x7f2670197030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:14.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.751+0000 7f266e59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26700fee80 0x7f2670197030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57238/0 (socket says 172.21.15.114:57238) 2024-09-13T18:31:14.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.751+0000 7f266e59c640 1 -- 172.21.15.114:0/1271012172 learned_addr learned my addr 172.21.15.114:0/1271012172 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:14.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.751+0000 7f266dd9b640 1 --2- 172.21.15.114:0/1271012172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2670103fd0 0x7f2670197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:14.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.752+0000 7f266ed9d640 1 --2- 172.21.15.114:0/1271012172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2670197ab0 0x7f26701a20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:14.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.752+0000 7f266e59c640 1 -- 172.21.15.114:0/1271012172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2670197ab0 msgr2=0x7f26701a20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:14.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.752+0000 7f266e59c640 1 --2- 172.21.15.114:0/1271012172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2670197ab0 0x7f26701a20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:14.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.752+0000 7f266e59c640 1 -- 172.21.15.114:0/1271012172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2670103fd0 msgr2=0x7f2670197570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:14.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.752+0000 7f266e59c640 1 --2- 172.21.15.114:0/1271012172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2670103fd0 0x7f2670197570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:14.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.752+0000 7f266e59c640 1 -- 172.21.15.114:0/1271012172 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2668009900 con 0x7f26700fee80 2024-09-13T18:31:14.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.752+0000 7f266e59c640 1 --2- 172.21.15.114:0/1271012172 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26700fee80 0x7f2670197030 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f266000d950 tx=0x7f266000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:14.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.752+0000 7f265f7fe640 1 -- 172.21.15.114:0/1271012172 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2660009c30 con 0x7f26700fee80 2024-09-13T18:31:14.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.752+0000 7f265f7fe640 1 -- 172.21.15.114:0/1271012172 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2660009d90 con 0x7f26700fee80 2024-09-13T18:31:14.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.752+0000 7f2674ce9640 1 -- 172.21.15.114:0/1271012172 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26701a2d60 con 0x7f26700fee80 2024-09-13T18:31:14.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.752+0000 7f265f7fe640 1 -- 172.21.15.114:0/1271012172 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266000f620 con 0x7f26700fee80 2024-09-13T18:31:14.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.752+0000 7f2674ce9640 1 -- 172.21.15.114:0/1271012172 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26701a3210 con 0x7f26700fee80 2024-09-13T18:31:14.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.754+0000 7f265f7fe640 1 -- 172.21.15.114:0/1271012172 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f26600109f0 con 0x7f26700fee80 2024-09-13T18:31:14.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.755+0000 7f2677a0f740 1 Processor -- start 2024-09-13T18:31:14.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.755+0000 7f2677a0f740 1 -- start start 2024-09-13T18:31:14.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.755+0000 7f2677a0f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c2b5497c0 0x561c2b549ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:14.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.755+0000 7f265f7fe640 1 --2- 172.21.15.114:0/1271012172 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f26580a1200 0x7f26580a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:14.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.755+0000 7f2677a0f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561c2b54a0e0 0x561c2b554440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:14.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.755+0000 7f2677a0f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c2b554980 0x561c2b556d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:14.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.755+0000 7f2677a0f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c2b4b4c60 con 0x561c2b54a0e0 2024-09-13T18:31:14.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.755+0000 7f2677a0f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c2b4b23f0 con 0x561c2b5497c0 2024-09-13T18:31:14.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.755+0000 7f2677a0f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c2b4adce0 con 0x561c2b554980 2024-09-13T18:31:14.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.756+0000 7f266e59c640 1 --2- 172.21.15.114:0/1271012172 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f26580a1200 0x7f26580a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:14.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.756+0000 7f266dd9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561c2b54a0e0 0x561c2b554440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:14.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.756+0000 7f266dd9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561c2b54a0e0 0x561c2b554440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57248/0 (socket says 172.21.15.114:57248) 2024-09-13T18:31:14.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.756+0000 7f266dd9b640 1 -- 172.21.15.114:0/1375695512 learned_addr learned my addr 172.21.15.114:0/1375695512 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:14.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.756+0000 7f266ed9d640 1 --2- 172.21.15.114:0/1375695512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c2b5497c0 0x561c2b549ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:14.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.756+0000 7f265f7fe640 1 -- 172.21.15.114:0/1271012172 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f26600ccd40 con 0x7f26700fee80 2024-09-13T18:31:14.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.756+0000 7f266ed9d640 1 --2- 172.21.15.114:0/1375695512 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c2b554980 0x561c2b556d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:14.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.756+0000 7f266ed9d640 1 -- 172.21.15.114:0/1375695512 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c2b554980 msgr2=0x561c2b556d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:14.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.756+0000 7f266ed9d640 1 --2- 172.21.15.114:0/1375695512 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c2b554980 0x561c2b556d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:14.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.756+0000 7f266ed9d640 1 -- 172.21.15.114:0/1375695512 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561c2b54a0e0 msgr2=0x561c2b554440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:14.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.756+0000 7f266ed9d640 1 --2- 172.21.15.114:0/1375695512 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561c2b54a0e0 0x561c2b554440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:14.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.756+0000 7f266ed9d640 1 -- 172.21.15.114:0/1375695512 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c2b469970 con 0x561c2b5497c0 2024-09-13T18:31:14.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.756+0000 7f266e59c640 1 --2- 172.21.15.114:0/1271012172 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f26580a1200 0x7f26580a36c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2660099980 tx=0x7f266001e000 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:14.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.756+0000 7f266dd9b640 1 --2- 172.21.15.114:0/1375695512 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561c2b54a0e0 0x561c2b554440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:31:14.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.756+0000 7f266ed9d640 1 --2- 172.21.15.114:0/1375695512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c2b5497c0 0x561c2b549ba0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f26680098d0 tx=0x7f2668004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=913589aadeeeacec server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:14.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.756+0000 7f265cff9640 1 -- 172.21.15.114:0/1375695512 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2668030850 con 0x561c2b5497c0 2024-09-13T18:31:14.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.756+0000 7f265cff9640 1 -- 172.21.15.114:0/1375695512 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2668045030 con 0x561c2b5497c0 2024-09-13T18:31:14.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.756+0000 7f265cff9640 1 -- 172.21.15.114:0/1375695512 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2668003cd0 con 0x561c2b5497c0 2024-09-13T18:31:14.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.757+0000 7f2677a0f740 1 -- 172.21.15.114:0/1375695512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c2b5497c0 msgr2=0x561c2b549ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:14.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.757+0000 7f2677a0f740 1 --2- 172.21.15.114:0/1375695512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c2b5497c0 0x561c2b549ba0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f26680098d0 tx=0x7f2668004a40 comp rx=0 tx=0).stop 2024-09-13T18:31:14.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.757+0000 7f2677a0f740 1 -- 172.21.15.114:0/1375695512 shutdown_connections 2024-09-13T18:31:14.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.757+0000 7f2677a0f740 1 --2- 172.21.15.114:0/1375695512 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c2b554980 0x561c2b556d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:14.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.757+0000 7f2677a0f740 1 --2- 172.21.15.114:0/1375695512 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561c2b54a0e0 0x561c2b554440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:14.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.757+0000 7f2677a0f740 1 --2- 172.21.15.114:0/1375695512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c2b5497c0 0x561c2b549ba0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:14.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.757+0000 7f2677a0f740 1 -- 172.21.15.114:0/1375695512 >> 172.21.15.114:0/1375695512 conn(0x561c2b545d50 msgr2=0x561c2b546130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:14.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.757+0000 7f2677a0f740 1 -- 172.21.15.114:0/1375695512 shutdown_connections 2024-09-13T18:31:14.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.757+0000 7f2677a0f740 1 -- 172.21.15.114:0/1375695512 wait complete. 2024-09-13T18:31:14.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.758+0000 7f2677a0f740 1 -- start start 2024-09-13T18:31:14.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.758+0000 7f2677a0f740 1 Processor -- start 2024-09-13T18:31:14.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.758+0000 7f2677a0f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c2b5497c0 0x561c2b554330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:14.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.758+0000 7f2677a0f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561c2b54a0e0 0x561c2b551680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:14.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.758+0000 7f2677a0f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c2b554980 0x561c2b551c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:14.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.758+0000 7f2677a0f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c2b46bf80 con 0x561c2b54a0e0 2024-09-13T18:31:14.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.758+0000 7f2677a0f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c2b46bc20 con 0x561c2b5497c0 2024-09-13T18:31:14.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.758+0000 7f2677a0f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c2b438ab0 con 0x561c2b554980 2024-09-13T18:31:14.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.758+0000 7f2677a0f740 10 client.0 _ll_register_callbacks cb 0x561c2b4a7350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:31:14.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.758+0000 7f266dd9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561c2b54a0e0 0x561c2b551680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:14.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.758+0000 7f266dd9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561c2b54a0e0 0x561c2b551680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57260/0 (socket says 172.21.15.114:57260) 2024-09-13T18:31:14.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.758+0000 7f266dd9b640 1 -- 172.21.15.114:0/2262136238 learned_addr learned my addr 172.21.15.114:0/2262136238 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:14.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.758+0000 7f266ed9d640 1 --2- 172.21.15.114:0/2262136238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c2b5497c0 0x561c2b554330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:14.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.758+0000 7f266ed9d640 1 --2- 172.21.15.114:0/2262136238 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c2b554980 0x561c2b551c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:14.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.758+0000 7f266dd9b640 1 -- 172.21.15.114:0/2262136238 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c2b554980 msgr2=0x561c2b551c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:14.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.758+0000 7f266dd9b640 1 --2- 172.21.15.114:0/2262136238 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c2b554980 0x561c2b551c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:14.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.758+0000 7f266dd9b640 1 -- 172.21.15.114:0/2262136238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c2b5497c0 msgr2=0x561c2b554330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:14.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.758+0000 7f266dd9b640 1 --2- 172.21.15.114:0/2262136238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c2b5497c0 0x561c2b554330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:14.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.758+0000 7f266dd9b640 1 -- 172.21.15.114:0/2262136238 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f267010c360 con 0x561c2b54a0e0 2024-09-13T18:31:14.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.758+0000 7f266ed9d640 1 --2- 172.21.15.114:0/2262136238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c2b5497c0 0x561c2b554330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:31:14.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.758+0000 7f266dd9b640 1 --2- 172.21.15.114:0/2262136238 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561c2b54a0e0 0x561c2b551680 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f266400b9f0 tx=0x7f266400beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f28daa129afe6bc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:14.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.758+0000 7f2655ffb640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:31:14.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.759+0000 7f2655ffb640 1 -- 172.21.15.114:0/2262136238 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2664016070 con 0x561c2b54a0e0 2024-09-13T18:31:14.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.759+0000 7f2677a0f740 1 -- 172.21.15.114:0/2262136238 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561c2b1a1590 con 0x561c2b54a0e0 2024-09-13T18:31:14.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.759+0000 7f2655ffb640 1 -- 172.21.15.114:0/2262136238 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f2664003e20 con 0x561c2b54a0e0 2024-09-13T18:31:14.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.759+0000 7f2655ffb640 1 -- 172.21.15.114:0/2262136238 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2664011440 con 0x561c2b54a0e0 2024-09-13T18:31:14.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.759+0000 7f2677a0f740 10 client.18060 fetch_fsmap learned FSMap version 34 2024-09-13T18:31:14.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.759+0000 7f2677a0f740 1 -- 172.21.15.114:0/2262136238 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x561c2b4ac360 con 0x561c2b54a0e0 2024-09-13T18:31:14.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.759+0000 7f2655ffb640 1 -- 172.21.15.114:0/2262136238 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561c2b1a1590 con 0x561c2b54a0e0 2024-09-13T18:31:14.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.759+0000 7f2655ffb640 1 -- 172.21.15.114:0/2262136238 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f266401b650 con 0x561c2b54a0e0 2024-09-13T18:31:14.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.759+0000 7f2677a0f740 10 client.18060 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:31:14.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.759+0000 7f2677a0f740 10 client.18060 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:31:14.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.759+0000 7f2677a0f740 20 client.18060 populate_metadata read hostname 'smithi114' 2024-09-13T18:31:14.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.759+0000 7f2677a0f740 1 --2- 172.21.15.114:0/2262136238 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x561c2b564730 0x561c2b566b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:14.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.759+0000 7f2677a0f740 4 client.18060 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:31:14.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.759+0000 7f2677a0f740 1 -- 172.21.15.114:0/2262136238 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561c2b1b2860 con 0x561c2b564730 2024-09-13T18:31:14.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.759+0000 7f266ed9d640 1 --2- 172.21.15.114:0/2262136238 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x561c2b564730 0x561c2b566b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:14.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.760+0000 7f2655ffb640 10 client.18060 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:31:14.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.760+0000 7f266ed9d640 1 --2- 172.21.15.114:0/2262136238 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x561c2b564730 0x561c2b566b50 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d858f8c89109f1dd server_cookie=9878948e46e244a2 in_seq=0 out_seq=0 2024-09-13T18:31:14.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.760+0000 7f2655ffb640 1 -- 172.21.15.114:0/2262136238 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x561c2b1b2860 con 0x561c2b564730 2024-09-13T18:31:14.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.760+0000 7f2655ffb640 10 client.18060 handle_command_reply: tid=0 2024-09-13T18:31:14.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.760+0000 7f2677a0f740 1 client.18060 shutdown 2024-09-13T18:31:14.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.760+0000 7f2677a0f740 1 -- 172.21.15.114:0/2262136238 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561c2b54a0e0 msgr2=0x561c2b551680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:14.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.760+0000 7f2677a0f740 1 --2- 172.21.15.114:0/2262136238 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561c2b54a0e0 0x561c2b551680 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f266400b9f0 tx=0x7f266400beb0 comp rx=0 tx=0).stop 2024-09-13T18:31:14.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.761+0000 7f2677a0f740 1 -- 172.21.15.114:0/2262136238 shutdown_connections 2024-09-13T18:31:14.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.761+0000 7f2677a0f740 1 --2- 172.21.15.114:0/2262136238 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x561c2b564730 0x561c2b566b50 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:14.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.761+0000 7f2677a0f740 1 --2- 172.21.15.114:0/2262136238 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c2b554980 0x561c2b551c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:14.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.761+0000 7f2677a0f740 1 --2- 172.21.15.114:0/2262136238 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561c2b54a0e0 0x561c2b551680 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:14.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.761+0000 7f2677a0f740 1 --2- 172.21.15.114:0/2262136238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c2b5497c0 0x561c2b554330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:14.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.761+0000 7f2677a0f740 1 -- 172.21.15.114:0/2262136238 >> 172.21.15.114:0/2262136238 conn(0x561c2b545d50 msgr2=0x561c2b54b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:14.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.761+0000 7f2655ffb640 0 client.18060 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:31:14.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.761+0000 7f2677a0f740 1 -- 172.21.15.114:0/2262136238 shutdown_connections 2024-09-13T18:31:14.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.761+0000 7f2677a0f740 1 -- 172.21.15.114:0/2262136238 wait complete. 2024-09-13T18:31:14.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.761+0000 7f2677a0f740 20 client.18060 trim_cache size 0 max 16384 2024-09-13T18:31:14.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.780+0000 7f2677a0f740 1 Processor -- start 2024-09-13T18:31:14.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.784+0000 7f2677a0f740 1 -- start start 2024-09-13T18:31:14.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.784+0000 7f2677a0f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c2b545d50 0x561c2b55b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:14.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.784+0000 7f2677a0f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c2b5497c0 0x561c2b55bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:14.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.784+0000 7f2677a0f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561c2b54a0e0 0x561c2b54c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:14.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.784+0000 7f2677a0f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c2b1b2860 con 0x561c2b54a0e0 2024-09-13T18:31:14.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.784+0000 7f2677a0f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c2b1a1590 con 0x561c2b545d50 2024-09-13T18:31:14.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.784+0000 7f2677a0f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c2b562f70 con 0x561c2b5497c0 2024-09-13T18:31:14.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.785+0000 7f266dd9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c2b5497c0 0x561c2b55bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:14.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.785+0000 7f266dd9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c2b5497c0 0x561c2b55bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:46352/0 (socket says 172.21.15.114:46352) 2024-09-13T18:31:14.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.785+0000 7f266dd9b640 1 -- 172.21.15.114:0/1350703945 learned_addr learned my addr 172.21.15.114:0/1350703945 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:14.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.791+0000 7f266ed9d640 1 --2- 172.21.15.114:0/1350703945 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561c2b54a0e0 0x561c2b54c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:14.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.791+0000 7f266ed9d640 1 --2- 172.21.15.114:0/1350703945 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c2b545d50 0x561c2b55b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:14.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.791+0000 7f266dd9b640 1 -- 172.21.15.114:0/1350703945 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c2b545d50 msgr2=0x561c2b55b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:14.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.791+0000 7f266dd9b640 1 --2- 172.21.15.114:0/1350703945 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c2b545d50 0x561c2b55b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:14.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.791+0000 7f266dd9b640 1 -- 172.21.15.114:0/1350703945 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561c2b54a0e0 msgr2=0x561c2b54c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:14.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.791+0000 7f266dd9b640 1 --2- 172.21.15.114:0/1350703945 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561c2b54a0e0 0x561c2b54c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:14.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.791+0000 7f266dd9b640 1 -- 172.21.15.114:0/1350703945 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c2b44e1c0 con 0x561c2b5497c0 2024-09-13T18:31:14.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.795+0000 7f266dd9b640 1 --2- 172.21.15.114:0/1350703945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c2b5497c0 0x561c2b55bb90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f266401bb00 tx=0x7f266400b400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffaace734f08a046 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:14.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.805+0000 7f265d7fa640 1 -- 172.21.15.114:0/1350703945 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2664016040 con 0x561c2b5497c0 2024-09-13T18:31:14.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.805+0000 7f265d7fa640 1 -- 172.21.15.114:0/1350703945 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2664029480 con 0x561c2b5497c0 2024-09-13T18:31:14.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.805+0000 7f265d7fa640 1 -- 172.21.15.114:0/1350703945 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26640116a0 con 0x561c2b5497c0 2024-09-13T18:31:14.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.808+0000 7f2677a0f740 1 -- 172.21.15.114:0/1350703945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c2b5497c0 msgr2=0x561c2b55bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:14.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.808+0000 7f2677a0f740 1 --2- 172.21.15.114:0/1350703945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c2b5497c0 0x561c2b55bb90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f266401bb00 tx=0x7f266400b400 comp rx=0 tx=0).stop 2024-09-13T18:31:14.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.811+0000 7f2677a0f740 1 -- 172.21.15.114:0/1350703945 shutdown_connections 2024-09-13T18:31:14.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.811+0000 7f2677a0f740 1 --2- 172.21.15.114:0/1350703945 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561c2b54a0e0 0x561c2b54c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:14.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.811+0000 7f2677a0f740 1 --2- 172.21.15.114:0/1350703945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c2b5497c0 0x561c2b55bb90 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:14.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.811+0000 7f2677a0f740 1 --2- 172.21.15.114:0/1350703945 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c2b545d50 0x561c2b55b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:14.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.811+0000 7f2677a0f740 1 -- 172.21.15.114:0/1350703945 >> 172.21.15.114:0/1350703945 conn(0x561c2b554490 msgr2=0x561c2b546130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:14.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.814+0000 7f2677a0f740 1 -- 172.21.15.114:0/1350703945 shutdown_connections 2024-09-13T18:31:14.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.814+0000 7f2677a0f740 1 -- 172.21.15.114:0/1350703945 wait complete. 2024-09-13T18:31:14.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.815+0000 7f2677a0f740 1 -- start start 2024-09-13T18:31:14.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.815+0000 7f2677a0f740 1 Processor -- start 2024-09-13T18:31:14.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.815+0000 7f2677a0f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561c2b545d50 0x561c2b5bee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:14.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.816+0000 7f2677a0f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c2b5497c0 0x561c2b55af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:14.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.816+0000 7f2677a0f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c2b54a0e0 0x561c2b55b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:14.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.816+0000 7f266dd9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c2b5497c0 0x561c2b55af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:14.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.816+0000 7f266dd9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c2b5497c0 0x561c2b55af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:46362/0 (socket says 172.21.15.114:46362) 2024-09-13T18:31:14.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.816+0000 7f266dd9b640 1 -- 172.21.15.114:0/2767066679 learned_addr learned my addr 172.21.15.114:0/2767066679 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:14.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.816+0000 7f266ed9d640 1 --2- 172.21.15.114:0/2767066679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c2b54a0e0 0x561c2b55b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:14.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.816+0000 7f2677a0f740 1 -- 172.21.15.114:0/2767066679 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c2b55bdc0 con 0x561c2b545d50 2024-09-13T18:31:14.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.816+0000 7f2677a0f740 1 -- 172.21.15.114:0/2767066679 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c2b5bc910 con 0x561c2b54a0e0 2024-09-13T18:31:14.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.816+0000 7f2677a0f740 1 -- 172.21.15.114:0/2767066679 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c2b5bcad0 con 0x561c2b5497c0 2024-09-13T18:31:14.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.817+0000 7f2677a0f740 10 client.0 _ll_register_callbacks cb 0x7f2664004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:31:14.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.817+0000 7f266dd9b640 1 -- 172.21.15.114:0/2767066679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c2b54a0e0 msgr2=0x561c2b55b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:14.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.817+0000 7f266dd9b640 1 --2- 172.21.15.114:0/2767066679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c2b54a0e0 0x561c2b55b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:14.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.817+0000 7f266dd9b640 1 -- 172.21.15.114:0/2767066679 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561c2b545d50 msgr2=0x561c2b5bee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:14.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.817+0000 7f266dd9b640 1 --2- 172.21.15.114:0/2767066679 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561c2b545d50 0x561c2b5bee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:14.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.817+0000 7f266dd9b640 1 -- 172.21.15.114:0/2767066679 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561c2b4ac360 con 0x561c2b5497c0 2024-09-13T18:31:14.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.821+0000 7f266dd9b640 1 --2- 172.21.15.114:0/2767066679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c2b5497c0 0x561c2b55af30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f266401bb00 tx=0x7f266401b850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc1566ad51bd44c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:14.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.822+0000 7f2655ffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:31:14.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.823+0000 7f2655ffb640 1 -- 172.21.15.114:0/2767066679 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2664016040 con 0x561c2b5497c0 2024-09-13T18:31:14.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.823+0000 7f2655ffb640 1 -- 172.21.15.114:0/2767066679 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f26640297f0 con 0x561c2b5497c0 2024-09-13T18:31:14.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.825+0000 7f2677a0f740 1 -- 172.21.15.114:0/2767066679 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2668045030 con 0x561c2b5497c0 2024-09-13T18:31:14.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.827+0000 7f2655ffb640 1 -- 172.21.15.114:0/2767066679 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26640299b0 con 0x561c2b5497c0 2024-09-13T18:31:14.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.830+0000 7f2677a0f740 10 client.27460 fetch_fsmap learned FSMap version 34 2024-09-13T18:31:14.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.830+0000 7f2677a0f740 1 -- 172.21.15.114:0/2767066679 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x561c2b473990 con 0x561c2b5497c0 2024-09-13T18:31:14.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.833+0000 7f2655ffb640 1 -- 172.21.15.114:0/2767066679 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2664029b10 con 0x561c2b5497c0 2024-09-13T18:31:14.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.833+0000 7f2655ffb640 1 -- 172.21.15.114:0/2767066679 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2664045410 con 0x561c2b5497c0 2024-09-13T18:31:14.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.833+0000 7f2677a0f740 10 client.27460 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:31:14.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.833+0000 7f2677a0f740 10 client.27460 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:31:14.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.833+0000 7f2677a0f740 20 client.27460 populate_metadata read hostname 'smithi114' 2024-09-13T18:31:14.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.833+0000 7f2677a0f740 1 --2- 172.21.15.114:0/2767066679 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x561c2b554980 0x561c2b5d8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:14.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.833+0000 7f2677a0f740 4 client.27460 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-13T18:31:14.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.833+0000 7f2677a0f740 1 -- 172.21.15.114:0/2767066679 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x7f2668045030 con 0x561c2b554980 2024-09-13T18:31:14.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.835+0000 7f266ed9d640 1 --2- 172.21.15.114:0/2767066679 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x561c2b554980 0x561c2b5d8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:14.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.835+0000 7f266ed9d640 1 --2- 172.21.15.114:0/2767066679 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x561c2b554980 0x561c2b5d8920 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2aefe71f2cd4b8d server_cookie=299b4177f96202aa in_seq=0 out_seq=0 2024-09-13T18:31:14.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.835+0000 7f2655ffb640 10 client.27460 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:31:14.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.836+0000 7f2655ffb640 1 -- 172.21.15.114:0/2767066679 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x7f2668045030 con 0x561c2b554980 2024-09-13T18:31:14.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.836+0000 7f2655ffb640 10 client.27460 handle_command_reply: tid=0 2024-09-13T18:31:14.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.836+0000 7f2677a0f740 1 client.27460 shutdown 2024-09-13T18:31:14.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.836+0000 7f2677a0f740 1 -- 172.21.15.114:0/2767066679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c2b5497c0 msgr2=0x561c2b55af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:14.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.836+0000 7f2677a0f740 1 --2- 172.21.15.114:0/2767066679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c2b5497c0 0x561c2b55af30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f266401bb00 tx=0x7f266401b850 comp rx=0 tx=0).stop 2024-09-13T18:31:14.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.836+0000 7f2677a0f740 1 -- 172.21.15.114:0/2767066679 shutdown_connections 2024-09-13T18:31:14.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.836+0000 7f2677a0f740 1 --2- 172.21.15.114:0/2767066679 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x561c2b554980 0x561c2b5d8920 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:14.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.836+0000 7f2677a0f740 1 --2- 172.21.15.114:0/2767066679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c2b54a0e0 0x561c2b55b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:14.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.836+0000 7f2677a0f740 1 --2- 172.21.15.114:0/2767066679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c2b5497c0 0x561c2b55af30 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:14.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.836+0000 7f2677a0f740 1 --2- 172.21.15.114:0/2767066679 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561c2b545d50 0x561c2b5bee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:14.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.836+0000 7f2677a0f740 1 -- 172.21.15.114:0/2767066679 >> 172.21.15.114:0/2767066679 conn(0x561c2b554490 msgr2=0x561c2b54c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:14.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.837+0000 7f2655ffb640 0 client.27460 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:31:14.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.838+0000 7f2677a0f740 1 -- 172.21.15.114:0/2767066679 shutdown_connections 2024-09-13T18:31:14.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.838+0000 7f2677a0f740 1 -- 172.21.15.114:0/2767066679 wait complete. 2024-09-13T18:31:14.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.843+0000 7f2677a0f740 20 client.27460 trim_cache size 0 max 16384 2024-09-13T18:31:14.849 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:31:14.850 INFO:teuthology.orchestra.run.smithi114.stdout: "return_code": 0, 2024-09-13T18:31:14.850 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_tag": "f9f86a4d-82cd-4894-baf9-fec03e06c2ab", 2024-09-13T18:31:14.850 INFO:teuthology.orchestra.run.smithi114.stdout: "mode": "asynchronous" 2024-09-13T18:31:14.850 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:31:14.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.851+0000 7f2674ce9640 1 -- 172.21.15.114:0/1271012172 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f26580a1200 msgr2=0x7f26580a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:14.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.851+0000 7f2674ce9640 1 --2- 172.21.15.114:0/1271012172 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f26580a1200 0x7f26580a36c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2660099980 tx=0x7f266001e000 comp rx=0 tx=0).stop 2024-09-13T18:31:14.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.851+0000 7f2674ce9640 1 -- 172.21.15.114:0/1271012172 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26700fee80 msgr2=0x7f2670197030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:14.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.851+0000 7f2674ce9640 1 --2- 172.21.15.114:0/1271012172 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26700fee80 0x7f2670197030 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f266000d950 tx=0x7f266000de10 comp rx=0 tx=0).stop 2024-09-13T18:31:14.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.851+0000 7f2674ce9640 1 -- 172.21.15.114:0/1271012172 shutdown_connections 2024-09-13T18:31:14.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.851+0000 7f2674ce9640 1 --2- 172.21.15.114:0/1271012172 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f26580a1200 0x7f26580a36c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:14.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.851+0000 7f2674ce9640 1 --2- 172.21.15.114:0/1271012172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2670197ab0 0x7f26701a20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:14.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.851+0000 7f2674ce9640 1 --2- 172.21.15.114:0/1271012172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2670103fd0 0x7f2670197570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:14.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.851+0000 7f2674ce9640 1 --2- 172.21.15.114:0/1271012172 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26700fee80 0x7f2670197030 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:14.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.851+0000 7f2674ce9640 1 -- 172.21.15.114:0/1271012172 >> 172.21.15.114:0/1271012172 conn(0x7f26700fa8d0 msgr2=0x7f26700fbd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:14.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.852+0000 7f2674ce9640 1 -- 172.21.15.114:0/1271012172 shutdown_connections 2024-09-13T18:31:14.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:14.855+0000 7f2674ce9640 1 -- 172.21.15.114:0/1271012172 wait complete. 2024-09-13T18:31:14.881 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T18:31:15.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.090+0000 7f4b6592f640 1 -- 172.21.15.114:0/14115007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b60119400 msgr2=0x7f4b6011b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:15.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.090+0000 7f4b6592f640 1 --2- 172.21.15.114:0/14115007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b60119400 0x7f4b6011b7f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f4b5801acb0 tx=0x7f4b5803b8f0 comp rx=0 tx=0).stop 2024-09-13T18:31:15.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.097+0000 7f4b6592f640 1 -- 172.21.15.114:0/14115007 shutdown_connections 2024-09-13T18:31:15.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.097+0000 7f4b6592f640 1 --2- 172.21.15.114:0/14115007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b60119400 0x7f4b6011b7f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:15.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.097+0000 7f4b6592f640 1 --2- 172.21.15.114:0/14115007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b60107890 0x7f4b60107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:15.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.097+0000 7f4b6592f640 1 --2- 172.21.15.114:0/14115007 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b601074b0 0x7f4b6010e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:15.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.098+0000 7f4b6592f640 1 -- 172.21.15.114:0/14115007 >> 172.21.15.114:0/14115007 conn(0x7f4b6006af40 msgr2=0x7f4b6006b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:15.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.105+0000 7f4b6592f640 1 -- 172.21.15.114:0/14115007 shutdown_connections 2024-09-13T18:31:15.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.107+0000 7f4b6592f640 1 -- 172.21.15.114:0/14115007 wait complete. 2024-09-13T18:31:15.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.108+0000 7f4b6592f640 1 Processor -- start 2024-09-13T18:31:15.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.114+0000 7f4b6592f640 1 -- start start 2024-09-13T18:31:15.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.114+0000 7f4b6592f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b601074b0 0x7f4b60114b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:15.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.114+0000 7f4b6592f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b60107890 0x7f4b601150c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:15.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.114+0000 7f4b6592f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b60119400 0x7f4b601a10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:15.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.114+0000 7f4b6592f640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b601a16d0 con 0x7f4b601074b0 2024-09-13T18:31:15.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.114+0000 7f4b6592f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b601a1840 con 0x7f4b60107890 2024-09-13T18:31:15.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.114+0000 7f4b6592f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b601a1a00 con 0x7f4b60119400 2024-09-13T18:31:15.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.118+0000 7f4b6512e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b60119400 0x7f4b601a10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:15.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.118+0000 7f4b5ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b60107890 0x7f4b601150c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:15.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.118+0000 7f4b5ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b60107890 0x7f4b601150c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:41458/0 (socket says 172.21.15.114:41458) 2024-09-13T18:31:15.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.118+0000 7f4b5ffff640 1 -- 172.21.15.114:0/1265192061 learned_addr learned my addr 172.21.15.114:0/1265192061 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:15.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.119+0000 7f4b5ffff640 1 -- 172.21.15.114:0/1265192061 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b60119400 msgr2=0x7f4b601a10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:15.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.119+0000 7f4b5ffff640 1 --2- 172.21.15.114:0/1265192061 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b60119400 0x7f4b601a10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:15.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.119+0000 7f4b5ffff640 1 -- 172.21.15.114:0/1265192061 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b601074b0 msgr2=0x7f4b60114b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:15.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.119+0000 7f4b5ffff640 1 --2- 172.21.15.114:0/1265192061 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b601074b0 0x7f4b60114b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:15.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.119+0000 7f4b5ffff640 1 -- 172.21.15.114:0/1265192061 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b5801a910 con 0x7f4b60107890 2024-09-13T18:31:15.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.126+0000 7f4b5ffff640 1 --2- 172.21.15.114:0/1265192061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b60107890 0x7f4b601150c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c00d770 tx=0x7f4b4c00dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:15.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.136+0000 7f4b5dffb640 1 -- 172.21.15.114:0/1265192061 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c009c30 con 0x7f4b60107890 2024-09-13T18:31:15.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.136+0000 7f4b5dffb640 1 -- 172.21.15.114:0/1265192061 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4b4c00de80 con 0x7f4b60107890 2024-09-13T18:31:15.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.136+0000 7f4b5dffb640 1 -- 172.21.15.114:0/1265192061 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c004d40 con 0x7f4b60107890 2024-09-13T18:31:15.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.137+0000 7f4b6592f640 1 -- 172.21.15.114:0/1265192061 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b601a1ce0 con 0x7f4b60107890 2024-09-13T18:31:15.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.137+0000 7f4b6592f640 1 -- 172.21.15.114:0/1265192061 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b601a9490 con 0x7f4b60107890 2024-09-13T18:31:15.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.142+0000 7f4b698b1740 1 Processor -- start 2024-09-13T18:31:15.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.144+0000 7f4b698b1740 1 -- start start 2024-09-13T18:31:15.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.144+0000 7f4b698b1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559ccba18fe0 0x559ccba193c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:15.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.144+0000 7f4b698b1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ccba19900 0x559ccba23c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:15.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.144+0000 7f4b698b1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ccba241a0 0x559ccba26590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:15.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.144+0000 7f4b698b1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ccb983f20 con 0x559ccba18fe0 2024-09-13T18:31:15.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.144+0000 7f4b698b1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ccb9816b0 con 0x559ccba19900 2024-09-13T18:31:15.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.144+0000 7f4b698b1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ccb97cfa0 con 0x559ccba241a0 2024-09-13T18:31:15.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.147+0000 7f4b5dffb640 1 -- 172.21.15.114:0/1265192061 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f4b4c004ea0 con 0x7f4b60107890 2024-09-13T18:31:15.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.147+0000 7f4b6512e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559ccba18fe0 0x559ccba193c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:15.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.148+0000 7f4b6512e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559ccba18fe0 0x559ccba193c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57308/0 (socket says 172.21.15.114:57308) 2024-09-13T18:31:15.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.148+0000 7f4b6512e640 1 -- 172.21.15.114:0/955214810 learned_addr learned my addr 172.21.15.114:0/955214810 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:15.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.149+0000 7f4b5ffff640 1 --2- 172.21.15.114:0/955214810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ccba241a0 0x559ccba26590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:15.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.150+0000 7f4b5dffb640 1 --2- 172.21.15.114:0/1265192061 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f4b500a1410 0x7f4b500a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:15.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.149+0000 7f4b6492d640 1 --2- 172.21.15.114:0/955214810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ccba19900 0x559ccba23c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:15.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.152+0000 7f4b6512e640 1 -- 172.21.15.114:0/955214810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ccba241a0 msgr2=0x559ccba26590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:15.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.152+0000 7f4b6512e640 1 --2- 172.21.15.114:0/955214810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ccba241a0 0x559ccba26590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:15.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.152+0000 7f4b6512e640 1 -- 172.21.15.114:0/955214810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ccba19900 msgr2=0x559ccba23c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:15.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.152+0000 7f4b6512e640 1 --2- 172.21.15.114:0/955214810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ccba19900 0x559ccba23c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:15.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.152+0000 7f4b6512e640 1 -- 172.21.15.114:0/955214810 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ccb937840 con 0x559ccba18fe0 2024-09-13T18:31:15.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.152+0000 7f4b5ffff640 1 --2- 172.21.15.114:0/955214810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ccba241a0 0x559ccba26590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:31:15.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.154+0000 7f4b6512e640 1 --2- 172.21.15.114:0/1265192061 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f4b500a1410 0x7f4b500a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:15.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.154+0000 7f4b6512e640 1 --2- 172.21.15.114:0/955214810 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559ccba18fe0 0x559ccba193c0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f4b5803be60 tx=0x7f4b58041e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=282bf8be9d000f12 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:15.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.155+0000 7f4b5dffb640 1 -- 172.21.15.114:0/1265192061 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f4b4c0ccd30 con 0x7f4b60107890 2024-09-13T18:31:15.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.159+0000 7f4b4affd640 1 -- 172.21.15.114:0/955214810 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5805a030 con 0x559ccba18fe0 2024-09-13T18:31:15.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.159+0000 7f4b4affd640 1 -- 172.21.15.114:0/955214810 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4b58042970 con 0x559ccba18fe0 2024-09-13T18:31:15.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.160+0000 7f4b698b1740 1 -- 172.21.15.114:0/955214810 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559ccba18fe0 msgr2=0x559ccba193c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:15.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.160+0000 7f4b698b1740 1 --2- 172.21.15.114:0/955214810 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559ccba18fe0 0x559ccba193c0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f4b5803be60 tx=0x7f4b58041e20 comp rx=0 tx=0).stop 2024-09-13T18:31:15.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.163+0000 7f4b6512e640 1 --2- 172.21.15.114:0/1265192061 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f4b500a1410 0x7f4b500a38d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4b58041bb0 tx=0x7f4b58004410 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:15.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.163+0000 7f4b698b1740 1 -- 172.21.15.114:0/955214810 shutdown_connections 2024-09-13T18:31:15.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.163+0000 7f4b698b1740 1 --2- 172.21.15.114:0/955214810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ccba241a0 0x559ccba26590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:15.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.163+0000 7f4b698b1740 1 --2- 172.21.15.114:0/955214810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ccba19900 0x559ccba23c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:15.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.163+0000 7f4b698b1740 1 --2- 172.21.15.114:0/955214810 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559ccba18fe0 0x559ccba193c0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:15.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.163+0000 7f4b698b1740 1 -- 172.21.15.114:0/955214810 >> 172.21.15.114:0/955214810 conn(0x559ccba15570 msgr2=0x559ccba15950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:15.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.166+0000 7f4b698b1740 1 -- 172.21.15.114:0/955214810 shutdown_connections 2024-09-13T18:31:15.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.172+0000 7f4b698b1740 1 -- 172.21.15.114:0/955214810 wait complete. 2024-09-13T18:31:15.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.172+0000 7f4b698b1740 1 -- start start 2024-09-13T18:31:15.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.172+0000 7f4b698b1740 1 Processor -- start 2024-09-13T18:31:15.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.173+0000 7f4b698b1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559ccba18fe0 0x559ccba238d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:15.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.173+0000 7f4b698b1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ccba19900 0x559ccba20c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:15.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.173+0000 7f4b698b1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ccba241a0 0x559ccba211d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:15.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.173+0000 7f4b698b1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ccb91e2a0 con 0x559ccba18fe0 2024-09-13T18:31:15.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.173+0000 7f4b698b1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ccb655430 con 0x559ccba19900 2024-09-13T18:31:15.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.173+0000 7f4b698b1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ccb900680 con 0x559ccba241a0 2024-09-13T18:31:15.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.173+0000 7f4b698b1740 10 client.0 _ll_register_callbacks cb 0x559ccb976610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:31:15.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.173+0000 7f4b6492d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559ccba18fe0 0x559ccba238d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:15.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.173+0000 7f4b6492d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559ccba18fe0 0x559ccba238d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57318/0 (socket says 172.21.15.114:57318) 2024-09-13T18:31:15.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.173+0000 7f4b6492d640 1 -- 172.21.15.114:0/1641336371 learned_addr learned my addr 172.21.15.114:0/1641336371 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:15.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.174+0000 7f4b6492d640 1 -- 172.21.15.114:0/1641336371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ccba241a0 msgr2=0x559ccba211d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:31:15.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.174+0000 7f4b6492d640 1 --2- 172.21.15.114:0/1641336371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ccba241a0 0x559ccba211d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:15.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.174+0000 7f4b6492d640 1 -- 172.21.15.114:0/1641336371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ccba19900 msgr2=0x559ccba20c90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:31:15.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.174+0000 7f4b6492d640 1 --2- 172.21.15.114:0/1641336371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ccba19900 0x559ccba20c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:15.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.174+0000 7f4b6492d640 1 -- 172.21.15.114:0/1641336371 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b6011c170 con 0x559ccba18fe0 2024-09-13T18:31:15.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.174+0000 7f4b6492d640 1 --2- 172.21.15.114:0/1641336371 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559ccba18fe0 0x559ccba238d0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f4b5400d210 tx=0x7f4b5400d6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e49bfcf6678cbdbd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:15.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.174+0000 7f4b3ffff640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:31:15.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.176+0000 7f4b3ffff640 1 -- 172.21.15.114:0/1641336371 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b540125f0 con 0x559ccba18fe0 2024-09-13T18:31:15.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.177+0000 7f4b698b1740 1 -- 172.21.15.114:0/1641336371 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559ccb6704e0 con 0x559ccba18fe0 2024-09-13T18:31:15.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.177+0000 7f4b3ffff640 1 -- 172.21.15.114:0/1641336371 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f4b54012750 con 0x559ccba18fe0 2024-09-13T18:31:15.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.185+0000 7f4b3ffff640 1 -- 172.21.15.114:0/1641336371 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b5400b380 con 0x559ccba18fe0 2024-09-13T18:31:15.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.188+0000 7f4b698b1740 10 client.18069 fetch_fsmap learned FSMap version 34 2024-09-13T18:31:15.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.188+0000 7f4b698b1740 1 -- 172.21.15.114:0/1641336371 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x559ccb97b620 con 0x559ccba18fe0 2024-09-13T18:31:15.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.193+0000 7f4b3ffff640 1 -- 172.21.15.114:0/1641336371 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559ccb6704e0 con 0x559ccba18fe0 2024-09-13T18:31:15.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.193+0000 7f4b3ffff640 1 -- 172.21.15.114:0/1641336371 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4b54013bc0 con 0x559ccba18fe0 2024-09-13T18:31:15.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.193+0000 7f4b698b1740 10 client.18069 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:31:15.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.193+0000 7f4b698b1740 10 client.18069 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:31:15.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.193+0000 7f4b698b1740 20 client.18069 populate_metadata read hostname 'smithi114' 2024-09-13T18:31:15.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.193+0000 7f4b698b1740 1 --2- 172.21.15.114:0/1641336371 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559ccba33f20 0x559ccba36340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:15.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.193+0000 7f4b698b1740 4 client.18069 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:31:15.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.193+0000 7f4b698b1740 1 -- 172.21.15.114:0/1641336371 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559ccb6817e0 con 0x559ccba33f20 2024-09-13T18:31:15.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.194+0000 7f4b5ffff640 1 --2- 172.21.15.114:0/1641336371 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559ccba33f20 0x559ccba36340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:15.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.194+0000 7f4b5ffff640 1 --2- 172.21.15.114:0/1641336371 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559ccba33f20 0x559ccba36340 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2602adac5ba5e201 server_cookie=e186f0cbbe3d76cb in_seq=0 out_seq=0 2024-09-13T18:31:15.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.195+0000 7f4b3ffff640 10 client.18069 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:31:15.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.195+0000 7f4b3ffff640 1 -- 172.21.15.114:0/1641336371 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x559ccb6817e0 con 0x559ccba33f20 2024-09-13T18:31:15.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.195+0000 7f4b3ffff640 10 client.18069 handle_command_reply: tid=0 2024-09-13T18:31:15.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.195+0000 7f4b698b1740 1 client.18069 shutdown 2024-09-13T18:31:15.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.205+0000 7f4b698b1740 1 -- 172.21.15.114:0/1641336371 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559ccba18fe0 msgr2=0x559ccba238d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:15.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.205+0000 7f4b698b1740 1 --2- 172.21.15.114:0/1641336371 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559ccba18fe0 0x559ccba238d0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f4b5400d210 tx=0x7f4b5400d6d0 comp rx=0 tx=0).stop 2024-09-13T18:31:15.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.208+0000 7f4b698b1740 1 -- 172.21.15.114:0/1641336371 shutdown_connections 2024-09-13T18:31:15.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.208+0000 7f4b698b1740 1 --2- 172.21.15.114:0/1641336371 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559ccba33f20 0x559ccba36340 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:15.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.208+0000 7f4b698b1740 1 --2- 172.21.15.114:0/1641336371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ccba241a0 0x559ccba211d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:15.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.208+0000 7f4b698b1740 1 --2- 172.21.15.114:0/1641336371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ccba19900 0x559ccba20c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:15.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.208+0000 7f4b698b1740 1 --2- 172.21.15.114:0/1641336371 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559ccba18fe0 0x559ccba238d0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:15.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.208+0000 7f4b698b1740 1 -- 172.21.15.114:0/1641336371 >> 172.21.15.114:0/1641336371 conn(0x559ccba15570 msgr2=0x559ccba1b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:15.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.208+0000 7f4b3ffff640 0 client.18069 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:31:15.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.212+0000 7f4b698b1740 1 -- 172.21.15.114:0/1641336371 shutdown_connections 2024-09-13T18:31:15.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.213+0000 7f4b698b1740 1 -- 172.21.15.114:0/1641336371 wait complete. 2024-09-13T18:31:15.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.219+0000 7f4b698b1740 20 client.18069 trim_cache size 0 max 16384 2024-09-13T18:31:15.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.281+0000 7f4b698b1740 1 Processor -- start 2024-09-13T18:31:15.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.283+0000 7f4b698b1740 1 -- start start 2024-09-13T18:31:15.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.283+0000 7f4b698b1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559ccba18fe0 0x559ccba23430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:15.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.284+0000 7f4b698b1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ccba19900 0x559ccba2fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:15.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.284+0000 7f4b698b1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ccba241a0 0x559ccba2ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:15.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.284+0000 7f4b698b1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ccba21ca0 con 0x559ccba18fe0 2024-09-13T18:31:15.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.284+0000 7f4b698b1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ccba187b0 con 0x559ccba19900 2024-09-13T18:31:15.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.284+0000 7f4b698b1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ccba1f1c0 con 0x559ccba241a0 2024-09-13T18:31:15.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.284+0000 7f4b5ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ccba19900 0x559ccba2fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:15.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.284+0000 7f4b5ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ccba19900 0x559ccba2fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:41488/0 (socket says 172.21.15.114:41488) 2024-09-13T18:31:15.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.284+0000 7f4b5ffff640 1 -- 172.21.15.114:0/389436820 learned_addr learned my addr 172.21.15.114:0/389436820 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:15.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.284+0000 7f4b6512e640 1 --2- 172.21.15.114:0/389436820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ccba241a0 0x559ccba2ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:15.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.286+0000 7f4b5ffff640 1 -- 172.21.15.114:0/389436820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ccba241a0 msgr2=0x559ccba2ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:15.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.286+0000 7f4b5ffff640 1 --2- 172.21.15.114:0/389436820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ccba241a0 0x559ccba2ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:15.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.286+0000 7f4b5ffff640 1 -- 172.21.15.114:0/389436820 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559ccba18fe0 msgr2=0x559ccba23430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:15.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.286+0000 7f4b5ffff640 1 --2- 172.21.15.114:0/389436820 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559ccba18fe0 0x559ccba23430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:15.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.286+0000 7f4b5ffff640 1 -- 172.21.15.114:0/389436820 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ccb91d790 con 0x559ccba19900 2024-09-13T18:31:15.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.287+0000 7f4b5ffff640 1 --2- 172.21.15.114:0/389436820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ccba19900 0x559ccba2fa90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c0999f0 tx=0x7f4b4c0cc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=225237177df599cc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:15.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.288+0000 7f4b4b7fe640 1 -- 172.21.15.114:0/389436820 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c014020 con 0x559ccba19900 2024-09-13T18:31:15.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.288+0000 7f4b4b7fe640 1 -- 172.21.15.114:0/389436820 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4b4c0a1ee0 con 0x559ccba19900 2024-09-13T18:31:15.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.291+0000 7f4b698b1740 1 -- 172.21.15.114:0/389436820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ccba19900 msgr2=0x559ccba2fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:15.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.291+0000 7f4b698b1740 1 --2- 172.21.15.114:0/389436820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ccba19900 0x559ccba2fa90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c0999f0 tx=0x7f4b4c0cc040 comp rx=0 tx=0).stop 2024-09-13T18:31:15.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.291+0000 7f4b698b1740 1 -- 172.21.15.114:0/389436820 shutdown_connections 2024-09-13T18:31:15.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.291+0000 7f4b698b1740 1 --2- 172.21.15.114:0/389436820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ccba241a0 0x559ccba2ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:15.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.291+0000 7f4b698b1740 1 --2- 172.21.15.114:0/389436820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ccba19900 0x559ccba2fa90 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:15.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.291+0000 7f4b698b1740 1 --2- 172.21.15.114:0/389436820 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559ccba18fe0 0x559ccba23430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:15.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.291+0000 7f4b698b1740 1 -- 172.21.15.114:0/389436820 >> 172.21.15.114:0/389436820 conn(0x559ccba15570 msgr2=0x559ccba1bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:15.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.298+0000 7f4b698b1740 1 -- 172.21.15.114:0/389436820 shutdown_connections 2024-09-13T18:31:15.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.301+0000 7f4b698b1740 1 -- 172.21.15.114:0/389436820 wait complete. 2024-09-13T18:31:15.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.301+0000 7f4b698b1740 1 -- start start 2024-09-13T18:31:15.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.301+0000 7f4b698b1740 1 Processor -- start 2024-09-13T18:31:15.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.302+0000 7f4b698b1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559ccba18fe0 0x559ccba28a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:15.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.302+0000 7f4b698b1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ccba19900 0x559ccba28fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:15.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.302+0000 7f4b698b1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ccba241a0 0x559ccba29500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:15.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.302+0000 7f4b698b1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ccba89440 con 0x559ccba18fe0 2024-09-13T18:31:15.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.302+0000 7f4b698b1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ccba89580 con 0x559ccba241a0 2024-09-13T18:31:15.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.302+0000 7f4b698b1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ccba896c0 con 0x559ccba19900 2024-09-13T18:31:15.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.302+0000 7f4b698b1740 10 client.0 _ll_register_callbacks cb 0x7f4b5400b160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:31:15.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.305+0000 7f4b5ffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ccba19900 0x559ccba28fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:15.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.305+0000 7f4b5ffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ccba19900 0x559ccba28fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:46412/0 (socket says 172.21.15.114:46412) 2024-09-13T18:31:15.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.305+0000 7f4b5ffff640 1 -- 172.21.15.114:0/680918160 learned_addr learned my addr 172.21.15.114:0/680918160 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:15.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.306+0000 7f4b6512e640 1 --2- 172.21.15.114:0/680918160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ccba241a0 0x559ccba29500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:15.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.306+0000 7f4b6492d640 1 --2- 172.21.15.114:0/680918160 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559ccba18fe0 0x559ccba28a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:15.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.306+0000 7f4b5ffff640 1 -- 172.21.15.114:0/680918160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ccba241a0 msgr2=0x559ccba29500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:15.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.306+0000 7f4b5ffff640 1 --2- 172.21.15.114:0/680918160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ccba241a0 0x559ccba29500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:15.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.306+0000 7f4b5ffff640 1 -- 172.21.15.114:0/680918160 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559ccba18fe0 msgr2=0x559ccba28a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:15.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.307+0000 7f4b5ffff640 1 --2- 172.21.15.114:0/680918160 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559ccba18fe0 0x559ccba28a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:15.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.307+0000 7f4b5ffff640 1 -- 172.21.15.114:0/680918160 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b601a9490 con 0x559ccba19900 2024-09-13T18:31:15.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.308+0000 7f4b5ffff640 1 --2- 172.21.15.114:0/680918160 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ccba19900 0x559ccba28fc0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c095f60 tx=0x7f4b4c095e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=533cbab32318ecb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:15.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.310+0000 7f4b3ffff640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:31:15.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.310+0000 7f4b3ffff640 1 -- 172.21.15.114:0/680918160 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c0a3070 con 0x559ccba19900 2024-09-13T18:31:15.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.310+0000 7f4b3ffff640 1 -- 172.21.15.114:0/680918160 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f4b4c0cf040 con 0x559ccba19900 2024-09-13T18:31:15.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.311+0000 7f4b698b1740 1 -- 172.21.15.114:0/680918160 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4b58042970 con 0x559ccba19900 2024-09-13T18:31:15.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.317+0000 7f4b3ffff640 1 -- 172.21.15.114:0/680918160 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b4c014040 con 0x559ccba19900 2024-09-13T18:31:15.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.318+0000 7f4b698b1740 10 client.27469 fetch_fsmap learned FSMap version 34 2024-09-13T18:31:15.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.318+0000 7f4b698b1740 1 -- 172.21.15.114:0/680918160 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x559ccb942ea0 con 0x559ccba19900 2024-09-13T18:31:15.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.322+0000 7f4b3ffff640 1 -- 172.21.15.114:0/680918160 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4b58042970 con 0x559ccba19900 2024-09-13T18:31:15.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.323+0000 7f4b3ffff640 1 -- 172.21.15.114:0/680918160 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4b4c0a6070 con 0x559ccba19900 2024-09-13T18:31:15.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.323+0000 7f4b698b1740 10 client.27469 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:31:15.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.323+0000 7f4b698b1740 10 client.27469 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:31:15.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.323+0000 7f4b698b1740 20 client.27469 populate_metadata read hostname 'smithi114' 2024-09-13T18:31:15.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.323+0000 7f4b698b1740 1 --2- 172.21.15.114:0/680918160 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559ccba8a960 0x559ccba8cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:15.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.323+0000 7f4b698b1740 4 client.27469 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T18:31:15.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.323+0000 7f4b698b1740 1 -- 172.21.15.114:0/680918160 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559ccb6d4e30 con 0x559ccba8a960 2024-09-13T18:31:15.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.324+0000 7f4b6492d640 1 --2- 172.21.15.114:0/680918160 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559ccba8a960 0x559ccba8cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:15.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.331+0000 7f4b6492d640 1 --2- 172.21.15.114:0/680918160 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559ccba8a960 0x559ccba8cd80 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aab7f4a0fb5dd5d8 server_cookie=84fe613a9d4b5e5f in_seq=0 out_seq=0 2024-09-13T18:31:15.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.332+0000 7f4b3ffff640 10 client.27469 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:31:15.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.341+0000 7f4b3ffff640 1 -- 172.21.15.114:0/680918160 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x559ccb6d4e30 con 0x559ccba8a960 2024-09-13T18:31:15.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.341+0000 7f4b3ffff640 10 client.27469 handle_command_reply: tid=0 2024-09-13T18:31:15.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.343+0000 7f4b698b1740 1 client.27469 shutdown 2024-09-13T18:31:15.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.350+0000 7f4b698b1740 1 -- 172.21.15.114:0/680918160 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ccba19900 msgr2=0x559ccba28fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:15.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.350+0000 7f4b698b1740 1 --2- 172.21.15.114:0/680918160 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ccba19900 0x559ccba28fc0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c095f60 tx=0x7f4b4c095e10 comp rx=0 tx=0).stop 2024-09-13T18:31:15.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.355+0000 7f4b698b1740 1 -- 172.21.15.114:0/680918160 shutdown_connections 2024-09-13T18:31:15.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.355+0000 7f4b698b1740 1 --2- 172.21.15.114:0/680918160 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559ccba8a960 0x559ccba8cd80 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:15.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.355+0000 7f4b698b1740 1 --2- 172.21.15.114:0/680918160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ccba241a0 0x559ccba29500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:15.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.355+0000 7f4b698b1740 1 --2- 172.21.15.114:0/680918160 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ccba19900 0x559ccba28fc0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:15.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.355+0000 7f4b698b1740 1 --2- 172.21.15.114:0/680918160 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559ccba18fe0 0x559ccba28a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:15.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.355+0000 7f4b698b1740 1 -- 172.21.15.114:0/680918160 >> 172.21.15.114:0/680918160 conn(0x559ccba15570 msgr2=0x559ccba1bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:15.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.357+0000 7f4b3ffff640 0 client.27469 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:31:15.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.360+0000 7f4b698b1740 1 -- 172.21.15.114:0/680918160 shutdown_connections 2024-09-13T18:31:15.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.360+0000 7f4b698b1740 1 -- 172.21.15.114:0/680918160 wait complete. 2024-09-13T18:31:15.366 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:31:15.366 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-13T18:31:15.366 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": { 2024-09-13T18:31:15.366 INFO:teuthology.orchestra.run.smithi114.stdout: "f9f86a4d-82cd-4894-baf9-fec03e06c2ab": { 2024-09-13T18:31:15.367 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-13T18:31:15.367 INFO:teuthology.orchestra.run.smithi114.stdout: "tag": "f9f86a4d-82cd-4894-baf9-fec03e06c2ab", 2024-09-13T18:31:15.367 INFO:teuthology.orchestra.run.smithi114.stdout: "options": "recursive,force" 2024-09-13T18:31:15.367 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:31:15.367 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:31:15.367 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:31:15.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.364+0000 7f4b698b1740 20 client.27469 trim_cache size 0 max 16384 2024-09-13T18:31:15.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.378+0000 7f4b6592f640 1 -- 172.21.15.114:0/1265192061 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f4b500a1410 msgr2=0x7f4b500a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:15.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.378+0000 7f4b6592f640 1 --2- 172.21.15.114:0/1265192061 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f4b500a1410 0x7f4b500a38d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4b58041bb0 tx=0x7f4b58004410 comp rx=0 tx=0).stop 2024-09-13T18:31:15.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.378+0000 7f4b6592f640 1 -- 172.21.15.114:0/1265192061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b60107890 msgr2=0x7f4b601150c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:15.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.378+0000 7f4b6592f640 1 --2- 172.21.15.114:0/1265192061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b60107890 0x7f4b601150c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c00d770 tx=0x7f4b4c00dc30 comp rx=0 tx=0).stop 2024-09-13T18:31:15.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.384+0000 7f4b6592f640 1 -- 172.21.15.114:0/1265192061 shutdown_connections 2024-09-13T18:31:15.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.384+0000 7f4b6592f640 1 --2- 172.21.15.114:0/1265192061 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f4b500a1410 0x7f4b500a38d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:15.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.384+0000 7f4b6592f640 1 --2- 172.21.15.114:0/1265192061 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b60119400 0x7f4b601a10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:15.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.384+0000 7f4b6592f640 1 --2- 172.21.15.114:0/1265192061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b60107890 0x7f4b601150c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:15.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.384+0000 7f4b6592f640 1 --2- 172.21.15.114:0/1265192061 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b601074b0 0x7f4b60114b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:15.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.384+0000 7f4b6592f640 1 -- 172.21.15.114:0/1265192061 >> 172.21.15.114:0/1265192061 conn(0x7f4b6006af40 msgr2=0x7f4b6006ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:15.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.388+0000 7f4b6592f640 1 -- 172.21.15.114:0/1265192061 shutdown_connections 2024-09-13T18:31:15.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:15.388+0000 7f4b6592f640 1 -- 172.21.15.114:0/1265192061 wait complete. 2024-09-13T18:31:15.417 INFO:tasks.cephfs.filesystem:scrub status for tag:f9f86a4d-82cd-4894-baf9-fec03e06c2ab - {'path': '/', 'tag': 'f9f86a4d-82cd-4894-baf9-fec03e06c2ab', 'options': 'recursive,force'} 2024-09-13T18:31:15.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:15 smithi114 ceph-mon[30897]: pgmap v271: 129 pgs: 14 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 74 active+clean; 22 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 25 MiB/s wr, 556 op/s 2024-09-13T18:31:15.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:15 smithi114 ceph-mon[30897]: scrub queued for path: / 2024-09-13T18:31:15.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:15 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:31:15.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:15 smithi114 ceph-mon[30897]: scrub summary: active paths [/] 2024-09-13T18:31:15.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:15 smithi114 ceph-mon[30897]: scrub summary: active paths [/] 2024-09-13T18:31:15.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:15 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:31:15.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:15 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:31:15.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:15 smithi204 ceph-mon[34777]: pgmap v271: 129 pgs: 14 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 74 active+clean; 22 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 25 MiB/s wr, 556 op/s 2024-09-13T18:31:15.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:15 smithi204 ceph-mon[34777]: scrub queued for path: / 2024-09-13T18:31:15.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:15 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:31:15.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:15 smithi204 ceph-mon[34777]: scrub summary: active paths [/] 2024-09-13T18:31:15.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:15 smithi204 ceph-mon[34777]: scrub summary: active paths [/] 2024-09-13T18:31:15.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:15 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:31:15.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:15 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:31:16.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:15 smithi134 ceph-mon[34283]: pgmap v271: 129 pgs: 14 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 74 active+clean; 22 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 25 MiB/s wr, 556 op/s 2024-09-13T18:31:16.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:15 smithi134 ceph-mon[34283]: scrub queued for path: / 2024-09-13T18:31:16.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:15 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:31:16.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:15 smithi134 ceph-mon[34283]: scrub summary: active paths [/] 2024-09-13T18:31:16.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:15 smithi134 ceph-mon[34283]: scrub summary: active paths [/] 2024-09-13T18:31:16.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:15 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:31:16.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:15 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:31:16.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:16 smithi204 ceph-mon[34777]: scrub summary: active paths [/] 2024-09-13T18:31:16.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:16 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:31:16.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:16 smithi114 ceph-mon[30897]: scrub summary: active paths [/] 2024-09-13T18:31:16.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:16 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:31:17.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:16 smithi134 ceph-mon[34283]: scrub summary: active paths [/] 2024-09-13T18:31:17.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:16 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:31:17.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:17 smithi204 ceph-mon[34777]: pgmap v272: 129 pgs: 5 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 112 active+clean; 21 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 22 MiB/s wr, 481 op/s 2024-09-13T18:31:17.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:17 smithi114 ceph-mon[30897]: pgmap v272: 129 pgs: 5 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 112 active+clean; 21 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 22 MiB/s wr, 481 op/s 2024-09-13T18:31:17.978 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:17.974+0000 7efe9f244640 1 -- 172.21.15.114:0/2514069335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe9810b610 msgr2=0x7efe981131e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:17.978 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:17.974+0000 7efe9f244640 1 --2- 172.21.15.114:0/2514069335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe9810b610 0x7efe981131e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7efe8c009900 tx=0x7efe8c02a440 comp rx=0 tx=0).stop 2024-09-13T18:31:17.978 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:17.976+0000 7efe9f244640 1 -- 172.21.15.114:0/2514069335 shutdown_connections 2024-09-13T18:31:17.979 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:17.976+0000 7efe9f244640 1 --2- 172.21.15.114:0/2514069335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe9810b610 0x7efe981131e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:17.979 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:17.976+0000 7efe9f244640 1 --2- 172.21.15.114:0/2514069335 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efe98106f00 0x7efe9810af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:17.979 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:17.976+0000 7efe9f244640 1 --2- 172.21.15.114:0/2514069335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe98106550 0x7efe98106930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:17.979 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:17.976+0000 7efe9f244640 1 -- 172.21.15.114:0/2514069335 >> 172.21.15.114:0/2514069335 conn(0x7efe98075c30 msgr2=0x7efe98078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:17.985 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:17.980+0000 7efe9f244640 1 -- 172.21.15.114:0/2514069335 shutdown_connections 2024-09-13T18:31:17.990 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:17.984+0000 7efe9f244640 1 -- 172.21.15.114:0/2514069335 wait complete. 2024-09-13T18:31:17.990 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:17.984+0000 7efe9f244640 1 Processor -- start 2024-09-13T18:31:17.990 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:17.984+0000 7efe9f244640 1 -- start start 2024-09-13T18:31:17.990 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:17.985+0000 7efe9f244640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efe98106550 0x7efe9806bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:17.990 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:17.985+0000 7efe9f244640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe9810b610 0x7efe9806c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:17.990 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:17.985+0000 7efe9f244640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe980716c0 0x7efe98071b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:17.991 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:17.985+0000 7efe9f244640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe980721c0 con 0x7efe98106550 2024-09-13T18:31:17.991 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:17.985+0000 7efe9f244640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe98072330 con 0x7efe980716c0 2024-09-13T18:31:17.991 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:17.985+0000 7efe9f244640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe980724f0 con 0x7efe9810b610 2024-09-13T18:31:17.991 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:17.987+0000 7efe9d7e1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe980716c0 0x7efe98071b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:17.991 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:17.987+0000 7efe9d7e1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe980716c0 0x7efe98071b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:41524/0 (socket says 172.21.15.114:41524) 2024-09-13T18:31:17.991 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:17.987+0000 7efe9d7e1640 1 -- 172.21.15.114:0/2968074275 learned_addr learned my addr 172.21.15.114:0/2968074275 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:17.991 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:17.987+0000 7efe9cfe0640 1 --2- 172.21.15.114:0/2968074275 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efe98106550 0x7efe9806bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:17.993 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:17.988+0000 7efe9d7e1640 1 -- 172.21.15.114:0/2968074275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe9810b610 msgr2=0x7efe9806c330 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:31:17.993 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:17.988+0000 7efe9d7e1640 1 --2- 172.21.15.114:0/2968074275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe9810b610 0x7efe9806c330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:17.993 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:17.988+0000 7efe9d7e1640 1 -- 172.21.15.114:0/2968074275 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efe98106550 msgr2=0x7efe9806bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:17.993 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:17.988+0000 7efe9d7e1640 1 --2- 172.21.15.114:0/2968074275 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efe98106550 0x7efe9806bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:17.993 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:17.988+0000 7efe9d7e1640 1 -- 172.21.15.114:0/2968074275 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe8c009560 con 0x7efe980716c0 2024-09-13T18:31:17.993 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:17.989+0000 7efe9d7e1640 1 --2- 172.21.15.114:0/2968074275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe980716c0 0x7efe98071b70 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7efe8c0057c0 tx=0x7efe8c0055c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:17.993 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:17.989+0000 7efe95ffb640 1 -- 172.21.15.114:0/2968074275 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8c004ad0 con 0x7efe980716c0 2024-09-13T18:31:17.993 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:17.989+0000 7efe9f244640 1 -- 172.21.15.114:0/2968074275 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe9811bd50 con 0x7efe980716c0 2024-09-13T18:31:17.993 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:17.990+0000 7efe9f244640 1 -- 172.21.15.114:0/2968074275 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe9811c1d0 con 0x7efe980716c0 2024-09-13T18:31:17.993 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:17.990+0000 7efe95ffb640 1 -- 172.21.15.114:0/2968074275 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efe8c004c30 con 0x7efe980716c0 2024-09-13T18:31:17.993 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:17.990+0000 7efe95ffb640 1 -- 172.21.15.114:0/2968074275 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8c03c760 con 0x7efe980716c0 2024-09-13T18:31:18.004 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:17.998+0000 7efea2038740 1 Processor -- start 2024-09-13T18:31:18.004 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.000+0000 7efea2038740 1 -- start start 2024-09-13T18:31:18.004 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.000+0000 7efea2038740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f141a7e330 0x55f141a7e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:18.004 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.000+0000 7efea2038740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f141a7ec50 0x55f141a88fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:18.004 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.000+0000 7efea2038740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f141a894f0 0x55f141a8b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:18.004 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.000+0000 7efea2038740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1419e9c30 con 0x55f141a894f0 2024-09-13T18:31:18.004 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.000+0000 7efea2038740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1419e73c0 con 0x55f141a7e330 2024-09-13T18:31:18.005 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.000+0000 7efea2038740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1419e2cb0 con 0x55f141a7ec50 2024-09-13T18:31:18.005 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.000+0000 7efe9cfe0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f141a7ec50 0x55f141a88fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:18.005 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.000+0000 7efe9cfe0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f141a7ec50 0x55f141a88fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:46450/0 (socket says 172.21.15.114:46450) 2024-09-13T18:31:18.005 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.000+0000 7efe9cfe0640 1 -- 172.21.15.114:0/2331396800 learned_addr learned my addr 172.21.15.114:0/2331396800 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:18.005 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.001+0000 7efe9cfe0640 1 -- 172.21.15.114:0/2331396800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f141a7e330 msgr2=0x55f141a7e710 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:31:18.005 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.001+0000 7efe9cfe0640 1 --2- 172.21.15.114:0/2331396800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f141a7e330 0x55f141a7e710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:18.005 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.001+0000 7efe9cfe0640 1 -- 172.21.15.114:0/2331396800 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f141a894f0 msgr2=0x55f141a8b8e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:31:18.005 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.001+0000 7efe9cfe0640 1 --2- 172.21.15.114:0/2331396800 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f141a894f0 0x55f141a8b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:18.005 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.001+0000 7efe9cfe0640 1 -- 172.21.15.114:0/2331396800 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f141989910 con 0x55f141a7ec50 2024-09-13T18:31:18.005 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.001+0000 7efe9cfe0640 1 --2- 172.21.15.114:0/2331396800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f141a7ec50 0x55f141a88fb0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7efe9000ae90 tx=0x7efe9000f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7cd8b85f7cfff269 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:18.005 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.001+0000 7efe95ffb640 1 -- 172.21.15.114:0/2968074275 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7efe8c0035f0 con 0x7efe980716c0 2024-09-13T18:31:18.006 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.002+0000 7efe95ffb640 1 --2- 172.21.15.114:0/2968074275 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7efe800a1410 0x7efe800a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:18.006 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.003+0000 7efe95ffb640 1 -- 172.21.15.114:0/2968074275 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9541+0+0 (secure 0 0 0) 0x7efe8c0ede40 con 0x7efe980716c0 2024-09-13T18:31:18.006 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.004+0000 7efe7effd640 1 -- 172.21.15.114:0/2331396800 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9000fee0 con 0x55f141a7ec50 2024-09-13T18:31:18.007 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.005+0000 7efe97fff640 1 --2- 172.21.15.114:0/2968074275 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7efe800a1410 0x7efe800a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.006+0000 7efe97fff640 1 --2- 172.21.15.114:0/2968074275 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7efe800a1410 0x7efe800a38d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x55f141a7bc40 tx=0x7efe88005f90 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:18.012 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.008+0000 7efe7effd640 1 -- 172.21.15.114:0/2331396800 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efe90011ce0 con 0x55f141a7ec50 2024-09-13T18:31:18.012 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.009+0000 7efea2038740 1 -- 172.21.15.114:0/2331396800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f141a7ec50 msgr2=0x55f141a88fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:18.013 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.009+0000 7efea2038740 1 --2- 172.21.15.114:0/2331396800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f141a7ec50 0x55f141a88fb0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7efe9000ae90 tx=0x7efe9000f730 comp rx=0 tx=0).stop 2024-09-13T18:31:18.015 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.012+0000 7efea2038740 1 -- 172.21.15.114:0/2331396800 shutdown_connections 2024-09-13T18:31:18.015 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.012+0000 7efea2038740 1 --2- 172.21.15.114:0/2331396800 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f141a894f0 0x55f141a8b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:18.015 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.012+0000 7efea2038740 1 --2- 172.21.15.114:0/2331396800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f141a7ec50 0x55f141a88fb0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:18.015 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.012+0000 7efea2038740 1 --2- 172.21.15.114:0/2331396800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f141a7e330 0x55f141a7e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:18.015 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.012+0000 7efea2038740 1 -- 172.21.15.114:0/2331396800 >> 172.21.15.114:0/2331396800 conn(0x55f141a7a8c0 msgr2=0x55f141a7aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:18.017 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.015+0000 7efea2038740 1 -- 172.21.15.114:0/2331396800 shutdown_connections 2024-09-13T18:31:18.020 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.018+0000 7efea2038740 1 -- 172.21.15.114:0/2331396800 wait complete. 2024-09-13T18:31:18.021 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.018+0000 7efea2038740 1 -- start start 2024-09-13T18:31:18.021 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.018+0000 7efea2038740 1 Processor -- start 2024-09-13T18:31:18.024 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.021+0000 7efea2038740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f141a7e330 0x55f141a88dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:18.024 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.021+0000 7efea2038740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f141a7ec50 0x55f141a86110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:18.024 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.021+0000 7efea2038740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f141a894f0 0x55f141a86650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:18.024 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.021+0000 7efea2038740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1419a0e80 con 0x55f141a7ec50 2024-09-13T18:31:18.024 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.021+0000 7efea2038740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1419a0b20 con 0x55f141a7e330 2024-09-13T18:31:18.024 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.021+0000 7efea2038740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f141966260 con 0x55f141a894f0 2024-09-13T18:31:18.024 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.021+0000 7efea2038740 10 client.0 _ll_register_callbacks cb 0x55f14199bdf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:31:18.026 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.023+0000 7efe9cfe0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f141a7e330 0x55f141a88dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:18.026 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.023+0000 7efe9d7e1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f141a894f0 0x55f141a86650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:18.026 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.023+0000 7efe9d7e1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f141a894f0 0x55f141a86650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:46458/0 (socket says 172.21.15.114:46458) 2024-09-13T18:31:18.026 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.023+0000 7efe9d7e1640 1 -- 172.21.15.114:0/3408339999 learned_addr learned my addr 172.21.15.114:0/3408339999 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:18.026 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.024+0000 7efe97fff640 1 --2- 172.21.15.114:0/3408339999 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f141a7ec50 0x55f141a86110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.024+0000 7efe97fff640 1 -- 172.21.15.114:0/3408339999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f141a894f0 msgr2=0x55f141a86650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.024+0000 7efe97fff640 1 --2- 172.21.15.114:0/3408339999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f141a894f0 0x55f141a86650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.024+0000 7efe97fff640 1 -- 172.21.15.114:0/3408339999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f141a7e330 msgr2=0x55f141a88dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.024+0000 7efe97fff640 1 --2- 172.21.15.114:0/3408339999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f141a7e330 0x55f141a88dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.024+0000 7efe97fff640 1 -- 172.21.15.114:0/3408339999 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe9000aaf0 con 0x55f141a7ec50 2024-09-13T18:31:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.025+0000 7efe97fff640 1 --2- 172.21.15.114:0/3408339999 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f141a7ec50 0x55f141a86110 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x55f141a7b9b0 tx=0x7efe88004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee9d58affde179c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.025+0000 7efe77fff640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:31:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.025+0000 7efe77fff640 1 -- 172.21.15.114:0/3408339999 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8801c830 con 0x55f141a7ec50 2024-09-13T18:31:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.026+0000 7efea2038740 1 -- 172.21.15.114:0/3408339999 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f141a0b680 con 0x55f141a7ec50 2024-09-13T18:31:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.026+0000 7efe77fff640 1 -- 172.21.15.114:0/3408339999 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9541+0+0 (secure 0 0 0) 0x7efe88010e10 con 0x55f141a7ec50 2024-09-13T18:31:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.026+0000 7efe77fff640 1 -- 172.21.15.114:0/3408339999 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe8801c990 con 0x55f141a7ec50 2024-09-13T18:31:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.026+0000 7efea2038740 10 client.18081 fetch_fsmap learned FSMap version 34 2024-09-13T18:31:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.026+0000 7efea2038740 1 -- 172.21.15.114:0/3408339999 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55f1419e1330 con 0x55f141a7ec50 2024-09-13T18:31:18.030 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.028+0000 7efe77fff640 1 -- 172.21.15.114:0/3408339999 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f141a0b680 con 0x55f141a7ec50 2024-09-13T18:31:18.030 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.028+0000 7efe77fff640 1 -- 172.21.15.114:0/3408339999 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7efe88018070 con 0x55f141a7ec50 2024-09-13T18:31:18.030 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.028+0000 7efea2038740 10 client.18081 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:31:18.030 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.028+0000 7efea2038740 10 client.18081 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:31:18.030 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.028+0000 7efea2038740 20 client.18081 populate_metadata read hostname 'smithi114' 2024-09-13T18:31:18.031 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.028+0000 7efea2038740 1 --2- 172.21.15.114:0/3408339999 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f141aa9f70 0x55f141aac390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:18.031 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.028+0000 7efea2038740 4 client.18081 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:31:18.031 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.028+0000 7efea2038740 1 -- 172.21.15.114:0/3408339999 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f1419d65d0 con 0x55f141aa9f70 2024-09-13T18:31:18.031 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.028+0000 7efe9cfe0640 1 --2- 172.21.15.114:0/3408339999 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f141aa9f70 0x55f141aac390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:18.032 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.030+0000 7efe9cfe0640 1 --2- 172.21.15.114:0/3408339999 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f141aa9f70 0x55f141aac390 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7214fa4db65f244 server_cookie=4cc5b5c0dd29a223 in_seq=0 out_seq=0 2024-09-13T18:31:18.033 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.031+0000 7efe77fff640 10 client.18081 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:31:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.034+0000 7efe77fff640 1 -- 172.21.15.114:0/3408339999 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55f1419d65d0 con 0x55f141aa9f70 2024-09-13T18:31:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.034+0000 7efe77fff640 10 client.18081 handle_command_reply: tid=0 2024-09-13T18:31:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.034+0000 7efea2038740 1 client.18081 shutdown 2024-09-13T18:31:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.054+0000 7efea2038740 1 -- 172.21.15.114:0/3408339999 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f141a7ec50 msgr2=0x55f141a86110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.054+0000 7efea2038740 1 --2- 172.21.15.114:0/3408339999 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f141a7ec50 0x55f141a86110 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x55f141a7b9b0 tx=0x7efe88004730 comp rx=0 tx=0).stop 2024-09-13T18:31:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.058+0000 7efea2038740 1 -- 172.21.15.114:0/3408339999 shutdown_connections 2024-09-13T18:31:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.058+0000 7efea2038740 1 --2- 172.21.15.114:0/3408339999 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f141aa9f70 0x55f141aac390 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.058+0000 7efea2038740 1 --2- 172.21.15.114:0/3408339999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f141a894f0 0x55f141a86650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.058+0000 7efea2038740 1 --2- 172.21.15.114:0/3408339999 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f141a7ec50 0x55f141a86110 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.058+0000 7efea2038740 1 --2- 172.21.15.114:0/3408339999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f141a7e330 0x55f141a88dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.058+0000 7efea2038740 1 -- 172.21.15.114:0/3408339999 >> 172.21.15.114:0/3408339999 conn(0x55f141a7a8c0 msgr2=0x55f141a79b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.058+0000 7efe77fff640 0 client.18081 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:31:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.067+0000 7efea2038740 1 -- 172.21.15.114:0/3408339999 shutdown_connections 2024-09-13T18:31:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.071+0000 7efea2038740 1 -- 172.21.15.114:0/3408339999 wait complete. 2024-09-13T18:31:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.079+0000 7efea2038740 20 client.18081 trim_cache size 0 max 16384 2024-09-13T18:31:18.110 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.108+0000 7efea2038740 1 Processor -- start 2024-09-13T18:31:18.110 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.109+0000 7efea2038740 1 -- start start 2024-09-13T18:31:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.109+0000 7efea2038740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f141a7e330 0x55f141a888e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.109+0000 7efea2038740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f141a7ec50 0x55f141a902e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.109+0000 7efea2038740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f141a894f0 0x55f141a90820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.109+0000 7efea2038740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f141aa6c40 con 0x55f141a7ec50 2024-09-13T18:31:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.109+0000 7efea2038740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f141a873f0 con 0x55f141a894f0 2024-09-13T18:31:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.109+0000 7efea2038740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f141a7e180 con 0x55f141a7e330 2024-09-13T18:31:18.112 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.109+0000 7efe9d7e1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f141a894f0 0x55f141a90820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:18.112 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.109+0000 7efe9d7e1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f141a894f0 0x55f141a90820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:41540/0 (socket says 172.21.15.114:41540) 2024-09-13T18:31:18.112 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.109+0000 7efe9d7e1640 1 -- 172.21.15.114:0/155033169 learned_addr learned my addr 172.21.15.114:0/155033169 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:18.113 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.110+0000 7efe97fff640 1 --2- 172.21.15.114:0/155033169 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f141a7ec50 0x55f141a902e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:18.113 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.110+0000 7efe9d7e1640 1 -- 172.21.15.114:0/155033169 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f141a7e330 msgr2=0x55f141a888e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:18.113 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.110+0000 7efe9d7e1640 1 --2- 172.21.15.114:0/155033169 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f141a7e330 0x55f141a888e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:18.113 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.110+0000 7efe9d7e1640 1 -- 172.21.15.114:0/155033169 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f141a7ec50 msgr2=0x55f141a902e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:18.113 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.110+0000 7efe9d7e1640 1 --2- 172.21.15.114:0/155033169 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f141a7ec50 0x55f141a902e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:18.113 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.110+0000 7efe9d7e1640 1 -- 172.21.15.114:0/155033169 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f141983300 con 0x55f141a894f0 2024-09-13T18:31:18.113 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.110+0000 7efe97fff640 1 --2- 172.21.15.114:0/155033169 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f141a7ec50 0x55f141a902e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:31:18.113 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.110+0000 7efe9d7e1640 1 --2- 172.21.15.114:0/155033169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f141a894f0 0x55f141a90820 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7efe8c0baa80 tx=0x7efe8c0bb1f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c00f1bd93fa794d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.110+0000 7efe7f7fe640 1 -- 172.21.15.114:0/155033169 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8c0be920 con 0x55f141a894f0 2024-09-13T18:31:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.111+0000 7efe7f7fe640 1 -- 172.21.15.114:0/155033169 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efe8c0c2f70 con 0x55f141a894f0 2024-09-13T18:31:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.111+0000 7efe7f7fe640 1 -- 172.21.15.114:0/155033169 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8c0b9fe0 con 0x55f141a894f0 2024-09-13T18:31:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.112+0000 7efea2038740 1 -- 172.21.15.114:0/155033169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f141a894f0 msgr2=0x55f141a90820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.112+0000 7efea2038740 1 --2- 172.21.15.114:0/155033169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f141a894f0 0x55f141a90820 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7efe8c0baa80 tx=0x7efe8c0bb1f0 comp rx=0 tx=0).stop 2024-09-13T18:31:18.114 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.112+0000 7efea2038740 1 -- 172.21.15.114:0/155033169 shutdown_connections 2024-09-13T18:31:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.112+0000 7efea2038740 1 --2- 172.21.15.114:0/155033169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f141a894f0 0x55f141a90820 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.112+0000 7efea2038740 1 --2- 172.21.15.114:0/155033169 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f141a7ec50 0x55f141a902e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.112+0000 7efea2038740 1 --2- 172.21.15.114:0/155033169 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f141a7e330 0x55f141a888e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.112+0000 7efea2038740 1 -- 172.21.15.114:0/155033169 >> 172.21.15.114:0/155033169 conn(0x55f141a7a8c0 msgr2=0x55f141a849c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.112+0000 7efea2038740 1 -- 172.21.15.114:0/155033169 shutdown_connections 2024-09-13T18:31:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.112+0000 7efea2038740 1 -- 172.21.15.114:0/155033169 wait complete. 2024-09-13T18:31:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.112+0000 7efea2038740 1 -- start start 2024-09-13T18:31:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.112+0000 7efea2038740 1 Processor -- start 2024-09-13T18:31:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.113+0000 7efea2038740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f141a7e330 0x55f141a902f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.113+0000 7efea2038740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f141a7ec50 0x55f141a82730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.113+0000 7efea2038740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f141a894f0 0x55f141a82c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.113+0000 7efea2038740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f141a90c10 con 0x55f141a7e330 2024-09-13T18:31:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.113+0000 7efea2038740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f141a831b0 con 0x55f141a7ec50 2024-09-13T18:31:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.113+0000 7efea2038740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f141a83370 con 0x55f141a894f0 2024-09-13T18:31:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.113+0000 7efea2038740 10 client.0 _ll_register_callbacks cb 0x7efe9000a630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:31:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.113+0000 7efe9cfe0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f141a7e330 0x55f141a902f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.113+0000 7efe9d7e1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f141a894f0 0x55f141a82c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.113+0000 7efe9cfe0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f141a7e330 0x55f141a902f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57382/0 (socket says 172.21.15.114:57382) 2024-09-13T18:31:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.113+0000 7efe9cfe0640 1 -- 172.21.15.114:0/74490910 learned_addr learned my addr 172.21.15.114:0/74490910 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.114+0000 7efe9d7e1640 1 -- 172.21.15.114:0/74490910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f141a7ec50 msgr2=0x55f141a82730 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:31:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.114+0000 7efe9d7e1640 1 --2- 172.21.15.114:0/74490910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f141a7ec50 0x55f141a82730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.114+0000 7efe9d7e1640 1 -- 172.21.15.114:0/74490910 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f141a7e330 msgr2=0x55f141a902f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.114+0000 7efe9d7e1640 1 --2- 172.21.15.114:0/74490910 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f141a7e330 0x55f141a902f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.114+0000 7efe9d7e1640 1 -- 172.21.15.114:0/74490910 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe9811c1d0 con 0x55f141a894f0 2024-09-13T18:31:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.114+0000 7efe9d7e1640 1 --2- 172.21.15.114:0/74490910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f141a894f0 0x55f141a82c70 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7efe8c0b7560 tx=0x7efe8c0b6f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7531ca9adb13a243 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:18.118 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:17 smithi134 ceph-mon[34283]: pgmap v272: 129 pgs: 5 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 112 active+clean; 21 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 22 MiB/s wr, 481 op/s 2024-09-13T18:31:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.119+0000 7efe77fff640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:31:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.119+0000 7efe77fff640 1 -- 172.21.15.114:0/74490910 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8c0d3480 con 0x55f141a894f0 2024-09-13T18:31:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.119+0000 7efe77fff640 1 -- 172.21.15.114:0/74490910 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9541+0+0 (secure 0 0 0) 0x7efe8c0b9fe0 con 0x55f141a894f0 2024-09-13T18:31:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.119+0000 7efea2038740 1 -- 172.21.15.114:0/74490910 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efe90011ce0 con 0x55f141a894f0 2024-09-13T18:31:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.119+0000 7efe77fff640 1 -- 172.21.15.114:0/74490910 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe8c0eb5c0 con 0x55f141a894f0 2024-09-13T18:31:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.120+0000 7efea2038740 10 client.27481 fetch_fsmap learned FSMap version 34 2024-09-13T18:31:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.120+0000 7efea2038740 1 -- 172.21.15.114:0/74490910 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55f1419a88e0 con 0x55f141a894f0 2024-09-13T18:31:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.123+0000 7efe77fff640 1 -- 172.21.15.114:0/74490910 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efe8c0cc030 con 0x55f141a894f0 2024-09-13T18:31:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.123+0000 7efe77fff640 1 -- 172.21.15.114:0/74490910 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7efe8c0c6060 con 0x55f141a894f0 2024-09-13T18:31:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.123+0000 7efea2038740 10 client.27481 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:31:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.123+0000 7efea2038740 10 client.27481 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:31:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.123+0000 7efea2038740 20 client.27481 populate_metadata read hostname 'smithi114' 2024-09-13T18:31:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.123+0000 7efea2038740 1 --2- 172.21.15.114:0/74490910 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f141b01410 0x55f141b03800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.123+0000 7efea2038740 4 client.27481 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T18:31:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.123+0000 7efea2038740 1 -- 172.21.15.114:0/74490910 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f1416d6300 con 0x55f141b01410 2024-09-13T18:31:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.128+0000 7efe9cfe0640 1 --2- 172.21.15.114:0/74490910 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f141b01410 0x55f141b03800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.131+0000 7efe9cfe0640 1 --2- 172.21.15.114:0/74490910 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f141b01410 0x55f141b03800 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f134c09eb7d492de server_cookie=d08e8c4fbb04a3e5 in_seq=0 out_seq=0 2024-09-13T18:31:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.131+0000 7efe77fff640 10 client.27481 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:31:18.209 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.207+0000 7efe77fff640 1 -- 172.21.15.114:0/74490910 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f1416d6300 con 0x55f141b01410 2024-09-13T18:31:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.207+0000 7efe77fff640 10 client.27481 handle_command_reply: tid=0 2024-09-13T18:31:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.207+0000 7efea2038740 1 client.27481 shutdown 2024-09-13T18:31:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.207+0000 7efea2038740 1 -- 172.21.15.114:0/74490910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f141a894f0 msgr2=0x55f141a82c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:18.210 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.207+0000 7efea2038740 1 --2- 172.21.15.114:0/74490910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f141a894f0 0x55f141a82c70 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7efe8c0b7560 tx=0x7efe8c0b6f50 comp rx=0 tx=0).stop 2024-09-13T18:31:18.216 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.213+0000 7efea2038740 1 -- 172.21.15.114:0/74490910 shutdown_connections 2024-09-13T18:31:18.216 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.213+0000 7efea2038740 1 --2- 172.21.15.114:0/74490910 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f141b01410 0x55f141b03800 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:18.216 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.213+0000 7efea2038740 1 --2- 172.21.15.114:0/74490910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f141a894f0 0x55f141a82c70 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:18.216 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.213+0000 7efea2038740 1 --2- 172.21.15.114:0/74490910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f141a7ec50 0x55f141a82730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:18.216 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.213+0000 7efea2038740 1 --2- 172.21.15.114:0/74490910 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f141a7e330 0x55f141a902f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:18.216 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.214+0000 7efea2038740 1 -- 172.21.15.114:0/74490910 >> 172.21.15.114:0/74490910 conn(0x55f141a7a8c0 msgr2=0x55f141a7fd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:18.216 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.214+0000 7efe77fff640 0 client.27481 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:31:18.220 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.218+0000 7efea2038740 1 -- 172.21.15.114:0/74490910 shutdown_connections 2024-09-13T18:31:18.220 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.218+0000 7efea2038740 1 -- 172.21.15.114:0/74490910 wait complete. 2024-09-13T18:31:18.227 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-13T18:31:18.227 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-13T18:31:18.227 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-13T18:31:18.227 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-13T18:31:18.227 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.225+0000 7efea2038740 20 client.27481 trim_cache size 0 max 16384 2024-09-13T18:31:18.230 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.227+0000 7efe9f244640 1 -- 172.21.15.114:0/2968074275 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7efe800a1410 msgr2=0x7efe800a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:18.230 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.227+0000 7efe9f244640 1 --2- 172.21.15.114:0/2968074275 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7efe800a1410 0x7efe800a38d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x55f141a7bc40 tx=0x7efe88005f90 comp rx=0 tx=0).stop 2024-09-13T18:31:18.230 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.228+0000 7efe9f244640 1 -- 172.21.15.114:0/2968074275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe980716c0 msgr2=0x7efe98071b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:18.230 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.228+0000 7efe9f244640 1 --2- 172.21.15.114:0/2968074275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe980716c0 0x7efe98071b70 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7efe8c0057c0 tx=0x7efe8c0055c0 comp rx=0 tx=0).stop 2024-09-13T18:31:18.232 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.230+0000 7efe9f244640 1 -- 172.21.15.114:0/2968074275 shutdown_connections 2024-09-13T18:31:18.232 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.230+0000 7efe9f244640 1 --2- 172.21.15.114:0/2968074275 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7efe800a1410 0x7efe800a38d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.230+0000 7efe9f244640 1 --2- 172.21.15.114:0/2968074275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe980716c0 0x7efe98071b70 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.230+0000 7efe9f244640 1 --2- 172.21.15.114:0/2968074275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe9810b610 0x7efe9806c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.230+0000 7efe9f244640 1 --2- 172.21.15.114:0/2968074275 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efe98106550 0x7efe9806bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.230+0000 7efe9f244640 1 -- 172.21.15.114:0/2968074275 >> 172.21.15.114:0/2968074275 conn(0x7efe98075c30 msgr2=0x7efe98076b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.232+0000 7efe9f244640 1 -- 172.21.15.114:0/2968074275 shutdown_connections 2024-09-13T18:31:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:18.232+0000 7efe9f244640 1 -- 172.21.15.114:0/2968074275 wait complete. 2024-09-13T18:31:19.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:19 smithi204 ceph-mon[34777]: pgmap v273: 129 pgs: 5 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 112 active+clean; 21 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 19 MiB/s wr, 413 op/s 2024-09-13T18:31:19.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:19 smithi114 ceph-mon[30897]: pgmap v273: 129 pgs: 5 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 112 active+clean; 21 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 19 MiB/s wr, 413 op/s 2024-09-13T18:31:20.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:19 smithi134 ceph-mon[34283]: pgmap v273: 129 pgs: 5 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 112 active+clean; 21 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 19 MiB/s wr, 413 op/s 2024-09-13T18:31:21.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:21 smithi204 ceph-mon[34777]: pgmap v274: 129 pgs: 3 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 121 active+clean; 21 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 27 MiB/s wr, 607 op/s 2024-09-13T18:31:21.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:21 smithi114 ceph-mon[30897]: pgmap v274: 129 pgs: 3 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 121 active+clean; 21 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 27 MiB/s wr, 607 op/s 2024-09-13T18:31:22.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:21 smithi134 ceph-mon[34283]: pgmap v274: 129 pgs: 3 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 121 active+clean; 21 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 27 MiB/s wr, 607 op/s 2024-09-13T18:31:23.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:23 smithi204 ceph-mon[34777]: pgmap v275: 129 pgs: 129 active+clean; 21 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 23 MiB/s wr, 515 op/s 2024-09-13T18:31:23.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:23 smithi204 ceph-mon[34777]: osdmap e100: 12 total, 12 up, 12 in 2024-09-13T18:31:23.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:23 smithi114 ceph-mon[30897]: pgmap v275: 129 pgs: 129 active+clean; 21 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 23 MiB/s wr, 515 op/s 2024-09-13T18:31:23.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:23 smithi114 ceph-mon[30897]: osdmap e100: 12 total, 12 up, 12 in 2024-09-13T18:31:24.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:23 smithi134 ceph-mon[34283]: pgmap v275: 129 pgs: 129 active+clean; 21 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 23 MiB/s wr, 515 op/s 2024-09-13T18:31:24.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:23 smithi134 ceph-mon[34283]: osdmap e100: 12 total, 12 up, 12 in 2024-09-13T18:31:25.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:25 smithi204 ceph-mon[34777]: pgmap v277: 129 pgs: 129 active+clean; 21 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 27 MiB/s wr, 595 op/s 2024-09-13T18:31:25.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:25 smithi114 ceph-mon[30897]: pgmap v277: 129 pgs: 129 active+clean; 21 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 27 MiB/s wr, 595 op/s 2024-09-13T18:31:26.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:25 smithi134 ceph-mon[34283]: pgmap v277: 129 pgs: 129 active+clean; 21 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 27 MiB/s wr, 595 op/s 2024-09-13T18:31:26.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:26 smithi114 ceph-mon[30897]: scrub summary: idle 2024-09-13T18:31:26.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:26 smithi114 ceph-mon[30897]: scrub summary: idle 2024-09-13T18:31:26.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:26 smithi114 ceph-mon[30897]: scrub summary: idle 2024-09-13T18:31:27.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:26 smithi134 ceph-mon[34283]: scrub summary: idle 2024-09-13T18:31:27.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:26 smithi134 ceph-mon[34283]: scrub summary: idle 2024-09-13T18:31:27.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:26 smithi134 ceph-mon[34283]: scrub summary: idle 2024-09-13T18:31:27.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:26 smithi204 ceph-mon[34777]: scrub summary: idle 2024-09-13T18:31:27.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:26 smithi204 ceph-mon[34777]: scrub summary: idle 2024-09-13T18:31:27.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:26 smithi204 ceph-mon[34777]: scrub summary: idle 2024-09-13T18:31:27.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:27 smithi114 ceph-mon[30897]: pgmap v278: 129 pgs: 129 active+clean; 21 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 31 MiB/s wr, 677 op/s 2024-09-13T18:31:28.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:27 smithi134 ceph-mon[34283]: pgmap v278: 129 pgs: 129 active+clean; 21 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 31 MiB/s wr, 677 op/s 2024-09-13T18:31:28.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:27 smithi204 ceph-mon[34777]: pgmap v278: 129 pgs: 129 active+clean; 21 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 31 MiB/s wr, 677 op/s 2024-09-13T18:31:29.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:29 smithi114 ceph-mon[30897]: pgmap v279: 129 pgs: 129 active+clean; 21 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 31 MiB/s wr, 677 op/s 2024-09-13T18:31:30.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:29 smithi134 ceph-mon[34283]: pgmap v279: 129 pgs: 129 active+clean; 21 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 31 MiB/s wr, 677 op/s 2024-09-13T18:31:30.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:29 smithi204 ceph-mon[34777]: pgmap v279: 129 pgs: 129 active+clean; 21 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 31 MiB/s wr, 677 op/s 2024-09-13T18:31:31.418 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.413+0000 7f173f59e640 1 -- 172.21.15.114:0/1049359618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1740119400 msgr2=0x7f174011b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:31.418 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.413+0000 7f173f59e640 1 --2- 172.21.15.114:0/1049359618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1740119400 0x7f174011b7f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1734009890 tx=0x7f173402a3b0 comp rx=0 tx=0).stop 2024-09-13T18:31:31.419 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.415+0000 7f173f59e640 1 -- 172.21.15.114:0/1049359618 shutdown_connections 2024-09-13T18:31:31.419 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.415+0000 7f173f59e640 1 --2- 172.21.15.114:0/1049359618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1740119400 0x7f174011b7f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:31.419 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.415+0000 7f173f59e640 1 --2- 172.21.15.114:0/1049359618 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1740107890 0x7f1740107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:31.419 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.415+0000 7f173f59e640 1 --2- 172.21.15.114:0/1049359618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17401074b0 0x7f174010e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:31.419 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.415+0000 7f173f59e640 1 -- 172.21.15.114:0/1049359618 >> 172.21.15.114:0/1049359618 conn(0x7f174006adb0 msgr2=0x7f174006b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:31.419 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.415+0000 7f173f59e640 1 -- 172.21.15.114:0/1049359618 shutdown_connections 2024-09-13T18:31:31.419 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.415+0000 7f173f59e640 1 -- 172.21.15.114:0/1049359618 wait complete. 2024-09-13T18:31:31.419 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.416+0000 7f173f59e640 1 Processor -- start 2024-09-13T18:31:31.420 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.417+0000 7f173f59e640 1 -- start start 2024-09-13T18:31:31.420 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.417+0000 7f173f59e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17401074b0 0x7f17401aaa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:31.420 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.417+0000 7f173f59e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1740107890 0x7f17401aaf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.417+0000 7f173f59e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1740119400 0x7f17401a4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.417+0000 7f173f59e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17401a5100 con 0x7f1740107890 2024-09-13T18:31:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.417+0000 7f173f59e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17401a5270 con 0x7f1740119400 2024-09-13T18:31:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.417+0000 7f173f59e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17401a5430 con 0x7f17401074b0 2024-09-13T18:31:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.417+0000 7f173e59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17401074b0 0x7f17401aaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.417+0000 7f173dd9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1740107890 0x7f17401aaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.417+0000 7f173e59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17401074b0 0x7f17401aaa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:49222/0 (socket says 172.21.15.114:49222) 2024-09-13T18:31:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.417+0000 7f173e59c640 1 -- 172.21.15.114:0/2438354918 learned_addr learned my addr 172.21.15.114:0/2438354918 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.418+0000 7f173dd9b640 1 -- 172.21.15.114:0/2438354918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17401074b0 msgr2=0x7f17401aaa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.418+0000 7f173dd9b640 1 --2- 172.21.15.114:0/2438354918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17401074b0 0x7f17401aaa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.418+0000 7f173dd9b640 1 -- 172.21.15.114:0/2438354918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1740119400 msgr2=0x7f17401a4ae0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:31:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.418+0000 7f173dd9b640 1 --2- 172.21.15.114:0/2438354918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1740119400 0x7f17401a4ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.418+0000 7f173dd9b640 1 -- 172.21.15.114:0/2438354918 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1734009490 con 0x7f1740107890 2024-09-13T18:31:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.423+0000 7f173dd9b640 1 --2- 172.21.15.114:0/2438354918 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1740107890 0x7f17401aaf40 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f173800e9f0 tx=0x7f173800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:31.430 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.423+0000 7f172f7fe640 1 -- 172.21.15.114:0/2438354918 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173800cd80 con 0x7f1740107890 2024-09-13T18:31:31.430 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.423+0000 7f172f7fe640 1 -- 172.21.15.114:0/2438354918 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1738011900 con 0x7f1740107890 2024-09-13T18:31:31.430 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.423+0000 7f172f7fe640 1 -- 172.21.15.114:0/2438354918 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17380107d0 con 0x7f1740107890 2024-09-13T18:31:31.430 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.423+0000 7f173f59e640 1 -- 172.21.15.114:0/2438354918 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17401a5710 con 0x7f1740107890 2024-09-13T18:31:31.430 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.423+0000 7f173f59e640 1 -- 172.21.15.114:0/2438354918 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17401b19f0 con 0x7f1740107890 2024-09-13T18:31:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.427+0000 7f174766d740 1 Processor -- start 2024-09-13T18:31:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.429+0000 7f172f7fe640 1 -- 172.21.15.114:0/2438354918 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f1738011a70 con 0x7f1740107890 2024-09-13T18:31:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.431+0000 7f172f7fe640 1 --2- 172.21.15.114:0/2438354918 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f17280a14d0 0x7f17280a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.431+0000 7f172f7fe640 1 -- 172.21.15.114:0/2438354918 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f17380cd500 con 0x7f1740107890 2024-09-13T18:31:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.433+0000 7f174766d740 1 -- start start 2024-09-13T18:31:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.433+0000 7f174766d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e9967c7500 0x55e9967c78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.433+0000 7f174766d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9967c7e20 0x55e9967d2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.433+0000 7f174766d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9967d26c0 0x55e9967d4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.433+0000 7f174766d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e996732c30 con 0x55e9967c7500 2024-09-13T18:31:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.433+0000 7f174766d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9967303c0 con 0x55e9967c7e20 2024-09-13T18:31:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.433+0000 7f174766d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e99672bcb0 con 0x55e9967d26c0 2024-09-13T18:31:31.438 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.435+0000 7f173e59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e9967c7500 0x55e9967c78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:31.438 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.435+0000 7f173e59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e9967c7500 0x55e9967c78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43688/0 (socket says 172.21.15.114:43688) 2024-09-13T18:31:31.438 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.435+0000 7f173e59c640 1 -- 172.21.15.114:0/1271001511 learned_addr learned my addr 172.21.15.114:0/1271001511 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.439+0000 7f173dd9b640 1 --2- 172.21.15.114:0/1271001511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9967c7e20 0x55e9967d2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:31.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.441+0000 7f173ed9d640 1 --2- 172.21.15.114:0/2438354918 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f17280a14d0 0x7f17280a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:31.452 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.442+0000 7f173e59c640 1 -- 172.21.15.114:0/1271001511 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9967d26c0 msgr2=0x55e9967d4ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:31:31.452 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.442+0000 7f173e59c640 1 --2- 172.21.15.114:0/1271001511 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9967d26c0 0x55e9967d4ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:31.452 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.442+0000 7f173e59c640 1 -- 172.21.15.114:0/1271001511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9967c7e20 msgr2=0x55e9967d2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:31.452 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.442+0000 7f173e59c640 1 --2- 172.21.15.114:0/1271001511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9967c7e20 0x55e9967d2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:31.452 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.442+0000 7f173e59c640 1 -- 172.21.15.114:0/1271001511 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9966d2910 con 0x55e9967c7500 2024-09-13T18:31:31.452 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.449+0000 7f173e59c640 1 --2- 172.21.15.114:0/1271001511 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e9967c7500 0x55e9967c78e0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f173000b7e0 tx=0x7f173000bca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=188b44085c125ec9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.453+0000 7f173ed9d640 1 --2- 172.21.15.114:0/2438354918 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f17280a14d0 0x7f17280a3990 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x55e9967c4e10 tx=0x7f17340056d0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:31.465 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.458+0000 7f172cff9640 1 -- 172.21.15.114:0/1271001511 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1730003dd0 con 0x55e9967c7500 2024-09-13T18:31:31.465 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.458+0000 7f172cff9640 1 -- 172.21.15.114:0/1271001511 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f17300118d0 con 0x55e9967c7500 2024-09-13T18:31:31.465 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.459+0000 7f172cff9640 1 -- 172.21.15.114:0/1271001511 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1730019790 con 0x55e9967c7500 2024-09-13T18:31:31.465 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.460+0000 7f174766d740 1 -- 172.21.15.114:0/1271001511 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e9967c7500 msgr2=0x55e9967c78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:31.465 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.460+0000 7f174766d740 1 --2- 172.21.15.114:0/1271001511 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e9967c7500 0x55e9967c78e0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f173000b7e0 tx=0x7f173000bca0 comp rx=0 tx=0).stop 2024-09-13T18:31:31.475 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.463+0000 7f174766d740 1 -- 172.21.15.114:0/1271001511 shutdown_connections 2024-09-13T18:31:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.463+0000 7f174766d740 1 --2- 172.21.15.114:0/1271001511 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9967d26c0 0x55e9967d4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.463+0000 7f174766d740 1 --2- 172.21.15.114:0/1271001511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9967c7e20 0x55e9967d2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.463+0000 7f174766d740 1 --2- 172.21.15.114:0/1271001511 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e9967c7500 0x55e9967c78e0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.463+0000 7f174766d740 1 -- 172.21.15.114:0/1271001511 >> 172.21.15.114:0/1271001511 conn(0x55e9967c3a90 msgr2=0x55e9967c3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.466+0000 7f174766d740 1 -- 172.21.15.114:0/1271001511 shutdown_connections 2024-09-13T18:31:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.468+0000 7f174766d740 1 -- 172.21.15.114:0/1271001511 wait complete. 2024-09-13T18:31:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.468+0000 7f174766d740 1 -- start start 2024-09-13T18:31:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.468+0000 7f174766d740 1 Processor -- start 2024-09-13T18:31:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.470+0000 7f174766d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9967c7500 0x55e9967d2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.470+0000 7f174766d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9967c7e20 0x55e9967cf310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.470+0000 7f174766d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e9967d26c0 0x55e9967cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.470+0000 7f174766d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9966e9e80 con 0x55e9967d26c0 2024-09-13T18:31:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.470+0000 7f174766d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9966e9b20 con 0x55e9967c7e20 2024-09-13T18:31:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.470+0000 7f174766d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9966af260 con 0x55e9967c7500 2024-09-13T18:31:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.470+0000 7f174766d740 10 client.0 _ll_register_callbacks cb 0x55e9966e4df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:31:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.471+0000 7f173e59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9967c7500 0x55e9967d2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.471+0000 7f173e59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9967c7500 0x55e9967d2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:49250/0 (socket says 172.21.15.114:49250) 2024-09-13T18:31:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.471+0000 7f173e59c640 1 -- 172.21.15.114:0/3068928455 learned_addr learned my addr 172.21.15.114:0/3068928455 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.471+0000 7f173e59c640 1 -- 172.21.15.114:0/3068928455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9967c7e20 msgr2=0x55e9967cf310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.471+0000 7f173e59c640 1 --2- 172.21.15.114:0/3068928455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9967c7e20 0x55e9967cf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.471+0000 7f173e59c640 1 -- 172.21.15.114:0/3068928455 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e9967d26c0 msgr2=0x55e9967cf970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:31:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.471+0000 7f173e59c640 1 --2- 172.21.15.114:0/3068928455 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e9967d26c0 0x55e9967cf970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.471+0000 7f173e59c640 1 -- 172.21.15.114:0/3068928455 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1730004290 con 0x55e9967c7500 2024-09-13T18:31:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.472+0000 7f173e59c640 1 --2- 172.21.15.114:0/3068928455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9967c7500 0x55e9967d2090 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1730003ea0 tx=0x7f1730003ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32484eb5092e1632 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.472+0000 7f1725ffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:31:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.472+0000 7f1725ffb640 1 -- 172.21.15.114:0/3068928455 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1730011750 con 0x55e9967c7500 2024-09-13T18:31:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.472+0000 7f174766d740 1 -- 172.21.15.114:0/3068928455 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e996754680 con 0x55e9967c7500 2024-09-13T18:31:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.473+0000 7f1725ffb640 1 -- 172.21.15.114:0/3068928455 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f1730011a40 con 0x55e9967c7500 2024-09-13T18:31:31.483 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.480+0000 7f1725ffb640 1 -- 172.21.15.114:0/3068928455 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1730011c40 con 0x55e9967c7500 2024-09-13T18:31:31.486 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.481+0000 7f174766d740 10 client.27490 fetch_fsmap learned FSMap version 34 2024-09-13T18:31:31.486 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.481+0000 7f174766d740 1 -- 172.21.15.114:0/3068928455 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55e99672a330 con 0x55e9967c7500 2024-09-13T18:31:31.486 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.483+0000 7f1725ffb640 1 -- 172.21.15.114:0/3068928455 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e996754680 con 0x55e9967c7500 2024-09-13T18:31:31.486 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.483+0000 7f1725ffb640 1 -- 172.21.15.114:0/3068928455 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1730010510 con 0x55e9967c7500 2024-09-13T18:31:31.491 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.486+0000 7f174766d740 10 client.27490 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:31:31.491 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.486+0000 7f174766d740 10 client.27490 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:31:31.491 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.486+0000 7f174766d740 20 client.27490 populate_metadata read hostname 'smithi114' 2024-09-13T18:31:31.491 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.486+0000 7f174766d740 1 --2- 172.21.15.114:0/3068928455 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55e996803ff0 0x55e9968063e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:31.491 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.486+0000 7f174766d740 4 client.27490 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:31:31.491 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.486+0000 7f174766d740 1 -- 172.21.15.114:0/3068928455 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e99671f5d0 con 0x55e996803ff0 2024-09-13T18:31:31.491 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.486+0000 7f173dd9b640 1 --2- 172.21.15.114:0/3068928455 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55e996803ff0 0x55e9968063e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.489+0000 7f173dd9b640 1 --2- 172.21.15.114:0/3068928455 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55e996803ff0 0x55e9968063e0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a2e7a5b9a0d3193 server_cookie=6d1b684b904c807c in_seq=0 out_seq=0 2024-09-13T18:31:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.489+0000 7f1725ffb640 10 client.27490 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:31:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.489+0000 7f1725ffb640 1 -- 172.21.15.114:0/3068928455 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55e99671f5d0 con 0x55e996803ff0 2024-09-13T18:31:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.489+0000 7f1725ffb640 10 client.27490 handle_command_reply: tid=0 2024-09-13T18:31:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.490+0000 7f174766d740 1 client.27490 shutdown 2024-09-13T18:31:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.490+0000 7f174766d740 1 -- 172.21.15.114:0/3068928455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9967c7500 msgr2=0x55e9967d2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.490+0000 7f174766d740 1 --2- 172.21.15.114:0/3068928455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9967c7500 0x55e9967d2090 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1730003ea0 tx=0x7f1730003ed0 comp rx=0 tx=0).stop 2024-09-13T18:31:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.493+0000 7f174766d740 1 -- 172.21.15.114:0/3068928455 shutdown_connections 2024-09-13T18:31:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.493+0000 7f174766d740 1 --2- 172.21.15.114:0/3068928455 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55e996803ff0 0x55e9968063e0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.493+0000 7f174766d740 1 --2- 172.21.15.114:0/3068928455 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e9967d26c0 0x55e9967cf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.493+0000 7f174766d740 1 --2- 172.21.15.114:0/3068928455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9967c7e20 0x55e9967cf310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.493+0000 7f174766d740 1 --2- 172.21.15.114:0/3068928455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9967c7500 0x55e9967d2090 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.493+0000 7f174766d740 1 -- 172.21.15.114:0/3068928455 >> 172.21.15.114:0/3068928455 conn(0x55e9967c3a90 msgr2=0x55e9967c2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.494+0000 7f1725ffb640 0 client.27490 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:31:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.494+0000 7f174766d740 1 -- 172.21.15.114:0/3068928455 shutdown_connections 2024-09-13T18:31:31.499 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.496+0000 7f174766d740 1 -- 172.21.15.114:0/3068928455 wait complete. 2024-09-13T18:31:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.497+0000 7f174766d740 20 client.27490 trim_cache size 0 max 16384 2024-09-13T18:31:31.540 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.536+0000 7f174766d740 1 Processor -- start 2024-09-13T18:31:31.543 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.539+0000 7f174766d740 1 -- start start 2024-09-13T18:31:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.540+0000 7f174766d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e9967c7500 0x55e9967c3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.540+0000 7f174766d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9967c7e20 0x55e9967d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.540+0000 7f174766d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9967d26c0 0x55e9967d9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.540+0000 7f174766d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9967cff70 con 0x55e9967c7500 2024-09-13T18:31:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.540+0000 7f174766d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9967c7c10 con 0x55e9967d26c0 2024-09-13T18:31:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.540+0000 7f173ed9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9967d26c0 0x55e9967d9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.541+0000 7f173ed9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9967d26c0 0x55e9967d9380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:36498/0 (socket says 172.21.15.114:36498) 2024-09-13T18:31:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.541+0000 7f173ed9d640 1 -- 172.21.15.114:0/1502595817 learned_addr learned my addr 172.21.15.114:0/1502595817 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.541+0000 7f174766d740 1 -- 172.21.15.114:0/1502595817 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9967c4690 con 0x55e9967c7e20 2024-09-13T18:31:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.541+0000 7f173dd9b640 1 --2- 172.21.15.114:0/1502595817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9967c7e20 0x55e9967d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.541+0000 7f173dd9b640 1 -- 172.21.15.114:0/1502595817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9967d26c0 msgr2=0x55e9967d9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.541+0000 7f173dd9b640 1 --2- 172.21.15.114:0/1502595817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9967d26c0 0x55e9967d9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.541+0000 7f173dd9b640 1 -- 172.21.15.114:0/1502595817 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e9967c7500 msgr2=0x55e9967c3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.541+0000 7f173dd9b640 1 --2- 172.21.15.114:0/1502595817 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e9967c7500 0x55e9967c3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.541+0000 7f173dd9b640 1 -- 172.21.15.114:0/1502595817 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9966cc300 con 0x55e9967c7e20 2024-09-13T18:31:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.542+0000 7f173dd9b640 1 --2- 172.21.15.114:0/1502595817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9967c7e20 0x55e9967d1bf0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f17380ca850 tx=0x7f1738011f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f6e130b6990c5b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.546+0000 7f172d7fa640 1 -- 172.21.15.114:0/1502595817 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17380a2690 con 0x55e9967c7e20 2024-09-13T18:31:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.546+0000 7f172d7fa640 1 -- 172.21.15.114:0/1502595817 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f17380985b0 con 0x55e9967c7e20 2024-09-13T18:31:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.546+0000 7f172d7fa640 1 -- 172.21.15.114:0/1502595817 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17380caad0 con 0x55e9967c7e20 2024-09-13T18:31:31.551 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.548+0000 7f174766d740 1 -- 172.21.15.114:0/1502595817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9967c7e20 msgr2=0x55e9967d1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:31.551 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.548+0000 7f174766d740 1 --2- 172.21.15.114:0/1502595817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9967c7e20 0x55e9967d1bf0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f17380ca850 tx=0x7f1738011f60 comp rx=0 tx=0).stop 2024-09-13T18:31:31.551 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.548+0000 7f174766d740 1 -- 172.21.15.114:0/1502595817 shutdown_connections 2024-09-13T18:31:31.551 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.548+0000 7f174766d740 1 --2- 172.21.15.114:0/1502595817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9967d26c0 0x55e9967d9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.548+0000 7f174766d740 1 --2- 172.21.15.114:0/1502595817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9967c7e20 0x55e9967d1bf0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.548+0000 7f174766d740 1 --2- 172.21.15.114:0/1502595817 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e9967c7500 0x55e9967c3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.548+0000 7f174766d740 1 -- 172.21.15.114:0/1502595817 >> 172.21.15.114:0/1502595817 conn(0x55e9967c3a90 msgr2=0x55e9967bed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.548+0000 7f174766d740 1 -- 172.21.15.114:0/1502595817 shutdown_connections 2024-09-13T18:31:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.548+0000 7f174766d740 1 -- 172.21.15.114:0/1502595817 wait complete. 2024-09-13T18:31:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.549+0000 7f174766d740 1 -- start start 2024-09-13T18:31:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.549+0000 7f174766d740 1 Processor -- start 2024-09-13T18:31:31.553 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.549+0000 7f174766d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9967c7500 0x55e9967d9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:31.553 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.549+0000 7f174766d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9967c7e20 0x55e9967cbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:31.553 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.549+0000 7f174766d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e9967d26c0 0x55e9967cc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:31.553 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.549+0000 7f174766d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9967ccad0 con 0x55e9967d26c0 2024-09-13T18:31:31.553 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.549+0000 7f174766d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9967ccc40 con 0x55e9967c7500 2024-09-13T18:31:31.553 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.549+0000 7f174766d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e99686b4e0 con 0x55e9967c7e20 2024-09-13T18:31:31.553 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.549+0000 7f174766d740 10 client.0 _ll_register_callbacks cb 0x7f1730004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:31:31.553 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.550+0000 7f173e59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9967c7500 0x55e9967d9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:31.554 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.550+0000 7f173e59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9967c7500 0x55e9967d9820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:36512/0 (socket says 172.21.15.114:36512) 2024-09-13T18:31:31.554 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.550+0000 7f173e59c640 1 -- 172.21.15.114:0/1094640643 learned_addr learned my addr 172.21.15.114:0/1094640643 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:31.554 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.550+0000 7f173ed9d640 1 --2- 172.21.15.114:0/1094640643 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e9967d26c0 0x55e9967cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:31.555 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.550+0000 7f173dd9b640 1 --2- 172.21.15.114:0/1094640643 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9967c7e20 0x55e9967cbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:31.555 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.550+0000 7f173e59c640 1 -- 172.21.15.114:0/1094640643 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9967c7e20 msgr2=0x55e9967cbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:31.555 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.550+0000 7f173e59c640 1 --2- 172.21.15.114:0/1094640643 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9967c7e20 0x55e9967cbda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:31.555 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.550+0000 7f173e59c640 1 -- 172.21.15.114:0/1094640643 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e9967d26c0 msgr2=0x55e9967cc400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:31.555 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.550+0000 7f173e59c640 1 --2- 172.21.15.114:0/1094640643 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e9967d26c0 0x55e9967cc400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:31.555 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.550+0000 7f173e59c640 1 -- 172.21.15.114:0/1094640643 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17401b19f0 con 0x55e9967c7500 2024-09-13T18:31:31.555 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.551+0000 7f173e59c640 1 --2- 172.21.15.114:0/1094640643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9967c7500 0x55e9967d9820 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1730010920 tx=0x7f1730005f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d730cbd62c80dbb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:31.555 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.551+0000 7f1725ffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:31:31.555 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.551+0000 7f1725ffb640 1 -- 172.21.15.114:0/1094640643 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1730011750 con 0x55e9967c7500 2024-09-13T18:31:31.555 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.551+0000 7f174766d740 1 -- 172.21.15.114:0/1094640643 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f17300118d0 con 0x55e9967c7500 2024-09-13T18:31:31.556 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.551+0000 7f1725ffb640 1 -- 172.21.15.114:0/1094640643 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f1730000e40 con 0x55e9967c7500 2024-09-13T18:31:31.556 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.551+0000 7f1725ffb640 1 -- 172.21.15.114:0/1094640643 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1730010b90 con 0x55e9967c7500 2024-09-13T18:31:31.556 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.552+0000 7f174766d740 10 client.27530 fetch_fsmap learned FSMap version 34 2024-09-13T18:31:31.556 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.552+0000 7f174766d740 1 -- 172.21.15.114:0/1094640643 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55e9966f18e0 con 0x55e9967c7500 2024-09-13T18:31:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.556+0000 7f1725ffb640 1 -- 172.21.15.114:0/1094640643 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f17300118d0 con 0x55e9967c7500 2024-09-13T18:31:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.557+0000 7f1725ffb640 1 -- 172.21.15.114:0/1094640643 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f17300044b0 con 0x55e9967c7500 2024-09-13T18:31:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.557+0000 7f174766d740 10 client.27530 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:31:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.557+0000 7f174766d740 10 client.27530 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:31:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.557+0000 7f174766d740 20 client.27530 populate_metadata read hostname 'smithi114' 2024-09-13T18:31:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.557+0000 7f174766d740 1 --2- 172.21.15.114:0/1094640643 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55e99686c980 0x55e99686eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.557+0000 7f174766d740 4 client.27530 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T18:31:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.557+0000 7f174766d740 1 -- 172.21.15.114:0/1094640643 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e99641f300 con 0x55e99686c980 2024-09-13T18:31:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.557+0000 7f173dd9b640 1 --2- 172.21.15.114:0/1094640643 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55e99686c980 0x55e99686eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.558+0000 7f173dd9b640 1 --2- 172.21.15.114:0/1094640643 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55e99686c980 0x55e99686eda0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b8def667889fb3a server_cookie=882f0dd771b42e9 in_seq=0 out_seq=0 2024-09-13T18:31:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.558+0000 7f1725ffb640 10 client.27530 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:31:31.570 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.567+0000 7f1725ffb640 1 -- 172.21.15.114:0/1094640643 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e99641f300 con 0x55e99686c980 2024-09-13T18:31:31.570 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.567+0000 7f1725ffb640 10 client.27530 handle_command_reply: tid=0 2024-09-13T18:31:31.570 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.567+0000 7f174766d740 1 client.27530 shutdown 2024-09-13T18:31:31.573 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.570+0000 7f174766d740 1 -- 172.21.15.114:0/1094640643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9967c7500 msgr2=0x55e9967d9820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:31.573 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.570+0000 7f174766d740 1 --2- 172.21.15.114:0/1094640643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9967c7500 0x55e9967d9820 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1730010920 tx=0x7f1730005f80 comp rx=0 tx=0).stop 2024-09-13T18:31:31.576 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.573+0000 7f174766d740 1 -- 172.21.15.114:0/1094640643 shutdown_connections 2024-09-13T18:31:31.576 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.573+0000 7f174766d740 1 --2- 172.21.15.114:0/1094640643 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55e99686c980 0x55e99686eda0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:31.577 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.573+0000 7f174766d740 1 --2- 172.21.15.114:0/1094640643 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e9967d26c0 0x55e9967cc400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:31.577 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.573+0000 7f174766d740 1 --2- 172.21.15.114:0/1094640643 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9967c7e20 0x55e9967cbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:31.577 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.573+0000 7f174766d740 1 --2- 172.21.15.114:0/1094640643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9967c7500 0x55e9967d9820 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:31.577 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.573+0000 7f174766d740 1 -- 172.21.15.114:0/1094640643 >> 172.21.15.114:0/1094640643 conn(0x55e9967c3a90 msgr2=0x55e9967cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:31.581 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.577+0000 7f1725ffb640 0 client.27530 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:31:31.581 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.577+0000 7f174766d740 1 -- 172.21.15.114:0/1094640643 shutdown_connections 2024-09-13T18:31:31.584 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-13T18:31:31.584 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-13T18:31:31.584 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-13T18:31:31.584 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-13T18:31:31.584 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.578+0000 7f174766d740 1 -- 172.21.15.114:0/1094640643 wait complete. 2024-09-13T18:31:31.584 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.578+0000 7f174766d740 20 client.27530 trim_cache size 0 max 16384 2024-09-13T18:31:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.581+0000 7f173f59e640 1 -- 172.21.15.114:0/2438354918 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f17280a14d0 msgr2=0x7f17280a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.581+0000 7f173f59e640 1 --2- 172.21.15.114:0/2438354918 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f17280a14d0 0x7f17280a3990 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x55e9967c4e10 tx=0x7f17340056d0 comp rx=0 tx=0).stop 2024-09-13T18:31:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.581+0000 7f173f59e640 1 -- 172.21.15.114:0/2438354918 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1740107890 msgr2=0x7f17401aaf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.581+0000 7f173f59e640 1 --2- 172.21.15.114:0/2438354918 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1740107890 0x7f17401aaf40 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f173800e9f0 tx=0x7f173800eeb0 comp rx=0 tx=0).stop 2024-09-13T18:31:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.583+0000 7f173f59e640 1 -- 172.21.15.114:0/2438354918 shutdown_connections 2024-09-13T18:31:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.583+0000 7f173f59e640 1 --2- 172.21.15.114:0/2438354918 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f17280a14d0 0x7f17280a3990 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.583+0000 7f173f59e640 1 --2- 172.21.15.114:0/2438354918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1740119400 0x7f17401a4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.583+0000 7f173f59e640 1 --2- 172.21.15.114:0/2438354918 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1740107890 0x7f17401aaf40 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.583+0000 7f173f59e640 1 --2- 172.21.15.114:0/2438354918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17401074b0 0x7f17401aaa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.583+0000 7f173f59e640 1 -- 172.21.15.114:0/2438354918 >> 172.21.15.114:0/2438354918 conn(0x7f174006adb0 msgr2=0x7f1740119bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:31.591 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.588+0000 7f173f59e640 1 -- 172.21.15.114:0/2438354918 shutdown_connections 2024-09-13T18:31:31.592 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:31.589+0000 7f173f59e640 1 -- 172.21.15.114:0/2438354918 wait complete. 2024-09-13T18:31:31.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:31 smithi114 ceph-mon[30897]: pgmap v280: 129 pgs: 129 active+clean; 21 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 32 MiB/s wr, 703 op/s 2024-09-13T18:31:32.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:31 smithi134 ceph-mon[34283]: pgmap v280: 129 pgs: 129 active+clean; 21 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 32 MiB/s wr, 703 op/s 2024-09-13T18:31:32.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:31 smithi204 ceph-mon[34777]: pgmap v280: 129 pgs: 129 active+clean; 21 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 32 MiB/s wr, 703 op/s 2024-09-13T18:31:33.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:33 smithi114 ceph-mon[30897]: pgmap v281: 129 pgs: 129 active+clean; 21 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 32 MiB/s wr, 706 op/s 2024-09-13T18:31:34.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:33 smithi134 ceph-mon[34283]: pgmap v281: 129 pgs: 129 active+clean; 21 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 32 MiB/s wr, 706 op/s 2024-09-13T18:31:34.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:33 smithi204 ceph-mon[34777]: pgmap v281: 129 pgs: 129 active+clean; 21 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 32 MiB/s wr, 706 op/s 2024-09-13T18:31:36.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:35 smithi134 ceph-mon[34283]: pgmap v282: 129 pgs: 129 active+clean; 21 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 33 MiB/s wr, 723 op/s 2024-09-13T18:31:36.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:35 smithi204 ceph-mon[34777]: pgmap v282: 129 pgs: 129 active+clean; 21 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 33 MiB/s wr, 723 op/s 2024-09-13T18:31:36.222 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:35 smithi114 ceph-mon[30897]: pgmap v282: 129 pgs: 129 active+clean; 21 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 33 MiB/s wr, 723 op/s 2024-09-13T18:31:38.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:37 smithi134 ceph-mon[34283]: pgmap v283: 129 pgs: 129 active+clean; 21 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 37 MiB/s wr, 825 op/s 2024-09-13T18:31:38.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:37 smithi204 ceph-mon[34777]: pgmap v283: 129 pgs: 129 active+clean; 21 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 37 MiB/s wr, 825 op/s 2024-09-13T18:31:38.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:37 smithi114 ceph-mon[30897]: pgmap v283: 129 pgs: 129 active+clean; 21 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 37 MiB/s wr, 825 op/s 2024-09-13T18:31:40.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:39 smithi134 ceph-mon[34283]: pgmap v284: 129 pgs: 129 active+clean; 21 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 27 MiB/s wr, 595 op/s 2024-09-13T18:31:40.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:39 smithi204 ceph-mon[34777]: pgmap v284: 129 pgs: 129 active+clean; 21 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 27 MiB/s wr, 595 op/s 2024-09-13T18:31:40.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:39 smithi114 ceph-mon[30897]: pgmap v284: 129 pgs: 129 active+clean; 21 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 27 MiB/s wr, 595 op/s 2024-09-13T18:31:42.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:41 smithi134 ceph-mon[34283]: pgmap v285: 129 pgs: 129 active+clean; 22 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 37 MiB/s wr, 830 op/s 2024-09-13T18:31:42.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:41 smithi204 ceph-mon[34777]: pgmap v285: 129 pgs: 129 active+clean; 22 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 37 MiB/s wr, 830 op/s 2024-09-13T18:31:42.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:41 smithi114 ceph-mon[30897]: pgmap v285: 129 pgs: 129 active+clean; 22 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 37 MiB/s wr, 830 op/s 2024-09-13T18:31:44.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:43 smithi134 ceph-mon[34283]: pgmap v286: 129 pgs: 129 active+clean; 22 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 30 MiB/s wr, 669 op/s 2024-09-13T18:31:44.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:43 smithi204 ceph-mon[34777]: pgmap v286: 129 pgs: 129 active+clean; 22 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 30 MiB/s wr, 669 op/s 2024-09-13T18:31:44.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:43 smithi114 ceph-mon[30897]: pgmap v286: 129 pgs: 129 active+clean; 22 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 30 MiB/s wr, 669 op/s 2024-09-13T18:31:44.785 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.777+0000 7f8576420640 1 -- 172.21.15.114:0/1875965560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8570119400 msgr2=0x7f857011b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:44.785 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.777+0000 7f8576420640 1 --2- 172.21.15.114:0/1875965560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8570119400 0x7f857011b7f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f856801acb0 tx=0x7f856803b920 comp rx=0 tx=0).stop 2024-09-13T18:31:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.795+0000 7f8576420640 1 -- 172.21.15.114:0/1875965560 shutdown_connections 2024-09-13T18:31:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.795+0000 7f8576420640 1 --2- 172.21.15.114:0/1875965560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8570119400 0x7f857011b7f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.795+0000 7f8576420640 1 --2- 172.21.15.114:0/1875965560 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8570107890 0x7f8570107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.795+0000 7f8576420640 1 --2- 172.21.15.114:0/1875965560 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85701074b0 0x7f857010e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.795+0000 7f8576420640 1 -- 172.21.15.114:0/1875965560 >> 172.21.15.114:0/1875965560 conn(0x7f857006af50 msgr2=0x7f857006b360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:44.802 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.800+0000 7f8576420640 1 -- 172.21.15.114:0/1875965560 shutdown_connections 2024-09-13T18:31:44.807 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.803+0000 7f8576420640 1 -- 172.21.15.114:0/1875965560 wait complete. 2024-09-13T18:31:44.807 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.804+0000 7f8576420640 1 Processor -- start 2024-09-13T18:31:44.807 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.804+0000 7f8576420640 1 -- start start 2024-09-13T18:31:44.807 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.805+0000 7f8576420640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85701074b0 0x7f8570118930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:44.807 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.805+0000 7f8576420640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8570107890 0x7f8570118e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:44.807 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.805+0000 7f8576420640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8570119400 0x7f8570114ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:44.807 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.805+0000 7f8576420640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8570115270 con 0x7f8570107890 2024-09-13T18:31:44.807 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.805+0000 7f8576420640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85701153e0 con 0x7f85701074b0 2024-09-13T18:31:44.808 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.805+0000 7f8576420640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85701155a0 con 0x7f8570119400 2024-09-13T18:31:44.808 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.806+0000 7f856ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85701074b0 0x7f8570118930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:44.808 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.806+0000 7f856ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85701074b0 0x7f8570118930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:45962/0 (socket says 172.21.15.114:45962) 2024-09-13T18:31:44.808 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.806+0000 7f856ffff640 1 -- 172.21.15.114:0/169495299 learned_addr learned my addr 172.21.15.114:0/169495299 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:44.808 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.806+0000 7f85749bd640 1 --2- 172.21.15.114:0/169495299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8570119400 0x7f8570114ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:44.809 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.807+0000 7f85749bd640 1 -- 172.21.15.114:0/169495299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85701074b0 msgr2=0x7f8570118930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:44.809 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.807+0000 7f85749bd640 1 --2- 172.21.15.114:0/169495299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85701074b0 0x7f8570118930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:44.809 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.807+0000 7f85749bd640 1 -- 172.21.15.114:0/169495299 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8570107890 msgr2=0x7f8570118e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:44.809 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.807+0000 7f85749bd640 1 --2- 172.21.15.114:0/169495299 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8570107890 0x7f8570118e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:44.809 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.807+0000 7f85749bd640 1 -- 172.21.15.114:0/169495299 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f856801a910 con 0x7f8570119400 2024-09-13T18:31:44.809 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.807+0000 7f85749bd640 1 --2- 172.21.15.114:0/169495299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8570119400 0x7f8570114ae0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f856803bf10 tx=0x7f8568040cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:44.810 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.808+0000 7f856d7fa640 1 -- 172.21.15.114:0/169495299 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8568004040 con 0x7f8570119400 2024-09-13T18:31:44.810 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.808+0000 7f856d7fa640 1 -- 172.21.15.114:0/169495299 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8568004630 con 0x7f8570119400 2024-09-13T18:31:44.810 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.808+0000 7f856d7fa640 1 -- 172.21.15.114:0/169495299 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8568041bd0 con 0x7f8570119400 2024-09-13T18:31:44.811 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.809+0000 7f8576420640 1 -- 172.21.15.114:0/169495299 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8570115820 con 0x7f8570119400 2024-09-13T18:31:44.811 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.809+0000 7f8576420640 1 -- 172.21.15.114:0/169495299 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85701a94b0 con 0x7f8570119400 2024-09-13T18:31:44.816 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.814+0000 7f856d7fa640 1 -- 172.21.15.114:0/169495299 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f8568041d30 con 0x7f8570119400 2024-09-13T18:31:44.817 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.815+0000 7f856d7fa640 1 --2- 172.21.15.114:0/169495299 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f85600a1410 0x7f85600a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:44.817 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.815+0000 7f856d7fa640 1 -- 172.21.15.114:0/169495299 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f85680ffb80 con 0x7f8570119400 2024-09-13T18:31:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.816+0000 7f856ffff640 1 --2- 172.21.15.114:0/169495299 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f85600a1410 0x7f85600a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.817+0000 7f857921f740 1 Processor -- start 2024-09-13T18:31:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.817+0000 7f857921f740 1 -- start start 2024-09-13T18:31:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.817+0000 7f857921f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5b07cc330 0x55c5b07cc710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.817+0000 7f857921f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c5b07ccc50 0x55c5b07d6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.817+0000 7f857921f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5b07d74f0 0x55c5b07d98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.817+0000 7f857921f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5b0737c30 con 0x55c5b07ccc50 2024-09-13T18:31:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.817+0000 7f857921f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5b07353c0 con 0x55c5b07d74f0 2024-09-13T18:31:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.817+0000 7f857921f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5b0730cb0 con 0x55c5b07cc330 2024-09-13T18:31:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.818+0000 7f856f7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c5b07ccc50 0x55c5b07d6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.818+0000 7f856f7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c5b07ccc50 0x55c5b07d6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:58380/0 (socket says 172.21.15.114:58380) 2024-09-13T18:31:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.818+0000 7f856f7fe640 1 -- 172.21.15.114:0/4267050449 learned_addr learned my addr 172.21.15.114:0/4267050449 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.818+0000 7f856ffff640 1 --2- 172.21.15.114:0/4267050449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5b07cc330 0x55c5b07cc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.818+0000 7f85749bd640 1 --2- 172.21.15.114:0/4267050449 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5b07d74f0 0x55c5b07d98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.819+0000 7f856ffff640 1 --2- 172.21.15.114:0/169495299 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f85600a1410 0x7f85600a38d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8564009a30 tx=0x7f8564009190 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.819+0000 7f85749bd640 1 -- 172.21.15.114:0/4267050449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5b07cc330 msgr2=0x55c5b07cc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.819+0000 7f85749bd640 1 --2- 172.21.15.114:0/4267050449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5b07cc330 0x55c5b07cc710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:44.822 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.819+0000 7f85749bd640 1 -- 172.21.15.114:0/4267050449 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c5b07ccc50 msgr2=0x55c5b07d6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:44.822 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.819+0000 7f85749bd640 1 --2- 172.21.15.114:0/4267050449 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c5b07ccc50 0x55c5b07d6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:44.822 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.819+0000 7f85749bd640 1 -- 172.21.15.114:0/4267050449 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5b06d7910 con 0x55c5b07d74f0 2024-09-13T18:31:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.820+0000 7f85749bd640 1 --2- 172.21.15.114:0/4267050449 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5b07d74f0 0x55c5b07d98e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f85680fced0 tx=0x7f85680cc490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50bd82e4d0c45e9d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:44.826 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.823+0000 7f855a7fc640 1 -- 172.21.15.114:0/4267050449 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85680cc890 con 0x55c5b07d74f0 2024-09-13T18:31:44.826 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.823+0000 7f855a7fc640 1 -- 172.21.15.114:0/4267050449 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f85680d06c0 con 0x55c5b07d74f0 2024-09-13T18:31:44.826 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.824+0000 7f855a7fc640 1 -- 172.21.15.114:0/4267050449 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85680c9130 con 0x55c5b07d74f0 2024-09-13T18:31:44.826 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.824+0000 7f857921f740 1 -- 172.21.15.114:0/4267050449 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5b07d74f0 msgr2=0x55c5b07d98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:44.826 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.824+0000 7f857921f740 1 --2- 172.21.15.114:0/4267050449 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5b07d74f0 0x55c5b07d98e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f85680fced0 tx=0x7f85680cc490 comp rx=0 tx=0).stop 2024-09-13T18:31:44.827 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.824+0000 7f857921f740 1 -- 172.21.15.114:0/4267050449 shutdown_connections 2024-09-13T18:31:44.827 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.824+0000 7f857921f740 1 --2- 172.21.15.114:0/4267050449 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5b07d74f0 0x55c5b07d98e0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:44.827 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.824+0000 7f857921f740 1 --2- 172.21.15.114:0/4267050449 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c5b07ccc50 0x55c5b07d6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:44.827 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.824+0000 7f857921f740 1 --2- 172.21.15.114:0/4267050449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5b07cc330 0x55c5b07cc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:44.828 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.824+0000 7f857921f740 1 -- 172.21.15.114:0/4267050449 >> 172.21.15.114:0/4267050449 conn(0x55c5b07c88c0 msgr2=0x55c5b07c8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.828+0000 7f857921f740 1 -- 172.21.15.114:0/4267050449 shutdown_connections 2024-09-13T18:31:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.828+0000 7f857921f740 1 -- 172.21.15.114:0/4267050449 wait complete. 2024-09-13T18:31:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.829+0000 7f857921f740 1 -- start start 2024-09-13T18:31:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.829+0000 7f857921f740 1 Processor -- start 2024-09-13T18:31:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.829+0000 7f857921f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c5b07cc330 0x55c5b07d6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.829+0000 7f857921f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5b07ccc50 0x55c5b07d4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.829+0000 7f857921f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5b07d74f0 0x55c5b07d4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.829+0000 7f857921f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5b06eee80 con 0x55c5b07cc330 2024-09-13T18:31:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.829+0000 7f857921f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5b06eeb20 con 0x55c5b07d74f0 2024-09-13T18:31:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.829+0000 7f857921f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5b06b4260 con 0x55c5b07ccc50 2024-09-13T18:31:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.829+0000 7f857921f740 10 client.0 _ll_register_callbacks cb 0x55c5b06e9df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:31:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.830+0000 7f856ffff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c5b07cc330 0x55c5b07d6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.831+0000 7f856ffff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c5b07cc330 0x55c5b07d6e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:58392/0 (socket says 172.21.15.114:58392) 2024-09-13T18:31:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.831+0000 7f856ffff640 1 -- 172.21.15.114:0/155242102 learned_addr learned my addr 172.21.15.114:0/155242102 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.831+0000 7f856f7fe640 1 --2- 172.21.15.114:0/155242102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5b07ccc50 0x55c5b07d4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.831+0000 7f856ffff640 1 -- 172.21.15.114:0/155242102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5b07ccc50 msgr2=0x55c5b07d4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.831+0000 7f856ffff640 1 --2- 172.21.15.114:0/155242102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5b07ccc50 0x55c5b07d4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.831+0000 7f856ffff640 1 -- 172.21.15.114:0/155242102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5b07d74f0 msgr2=0x55c5b07d4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.831+0000 7f856ffff640 1 --2- 172.21.15.114:0/155242102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5b07d74f0 0x55c5b07d4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.831+0000 7f856ffff640 1 -- 172.21.15.114:0/155242102 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85701a94b0 con 0x55c5b07cc330 2024-09-13T18:31:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.832+0000 7f856ffff640 1 --2- 172.21.15.114:0/155242102 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c5b07cc330 0x55c5b07d6e60 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f8564015a30 tx=0x7f8564004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f097eb5ec95f6f63 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.832+0000 7f854f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:31:44.841 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.839+0000 7f854f7fe640 1 -- 172.21.15.114:0/155242102 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8564013bb0 con 0x55c5b07cc330 2024-09-13T18:31:44.841 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.839+0000 7f857921f740 1 -- 172.21.15.114:0/155242102 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c5b0759680 con 0x55c5b07cc330 2024-09-13T18:31:44.841 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.839+0000 7f854f7fe640 1 -- 172.21.15.114:0/155242102 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f856401a040 con 0x55c5b07cc330 2024-09-13T18:31:44.844 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.842+0000 7f854f7fe640 1 -- 172.21.15.114:0/155242102 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8564013d10 con 0x55c5b07cc330 2024-09-13T18:31:44.846 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.844+0000 7f857921f740 10 client.18102 fetch_fsmap learned FSMap version 34 2024-09-13T18:31:44.846 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.844+0000 7f857921f740 1 -- 172.21.15.114:0/155242102 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55c5b072f330 con 0x55c5b07cc330 2024-09-13T18:31:44.859 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.856+0000 7f854f7fe640 1 -- 172.21.15.114:0/155242102 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c5b0759680 con 0x55c5b07cc330 2024-09-13T18:31:44.859 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.856+0000 7f854f7fe640 1 -- 172.21.15.114:0/155242102 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8564011070 con 0x55c5b07cc330 2024-09-13T18:31:44.859 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.857+0000 7f857921f740 10 client.18102 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:31:44.859 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.857+0000 7f857921f740 10 client.18102 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:31:44.859 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.857+0000 7f857921f740 20 client.18102 populate_metadata read hostname 'smithi114' 2024-09-13T18:31:44.859 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.857+0000 7f857921f740 1 --2- 172.21.15.114:0/155242102 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55c5b07e6de0 0x55c5b07e9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:44.859 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.857+0000 7f857921f740 4 client.18102 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:31:44.860 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.857+0000 7f857921f740 1 -- 172.21.15.114:0/155242102 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c5b07245d0 con 0x55c5b07e6de0 2024-09-13T18:31:44.860 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.857+0000 7f856f7fe640 1 --2- 172.21.15.114:0/155242102 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55c5b07e6de0 0x55c5b07e9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:44.867 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.865+0000 7f856f7fe640 1 --2- 172.21.15.114:0/155242102 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55c5b07e6de0 0x55c5b07e9200 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68ae1edbdc20f191 server_cookie=e5f3bf8033779902 in_seq=0 out_seq=0 2024-09-13T18:31:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.866+0000 7f854f7fe640 10 client.18102 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:31:44.873 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.871+0000 7f854f7fe640 1 -- 172.21.15.114:0/155242102 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55c5b07245d0 con 0x55c5b07e6de0 2024-09-13T18:31:44.873 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.871+0000 7f854f7fe640 10 client.18102 handle_command_reply: tid=0 2024-09-13T18:31:44.873 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.871+0000 7f857921f740 1 client.18102 shutdown 2024-09-13T18:31:44.874 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.872+0000 7f857921f740 1 -- 172.21.15.114:0/155242102 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c5b07cc330 msgr2=0x55c5b07d6e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:44.874 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.872+0000 7f857921f740 1 --2- 172.21.15.114:0/155242102 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c5b07cc330 0x55c5b07d6e60 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f8564015a30 tx=0x7f8564004750 comp rx=0 tx=0).stop 2024-09-13T18:31:44.881 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.879+0000 7f857921f740 1 -- 172.21.15.114:0/155242102 shutdown_connections 2024-09-13T18:31:44.881 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.879+0000 7f857921f740 1 --2- 172.21.15.114:0/155242102 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55c5b07e6de0 0x55c5b07e9200 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:44.881 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.879+0000 7f857921f740 1 --2- 172.21.15.114:0/155242102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5b07d74f0 0x55c5b07d4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:44.881 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.879+0000 7f857921f740 1 --2- 172.21.15.114:0/155242102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5b07ccc50 0x55c5b07d4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:44.882 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.879+0000 7f857921f740 1 --2- 172.21.15.114:0/155242102 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c5b07cc330 0x55c5b07d6e60 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:44.882 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.879+0000 7f857921f740 1 -- 172.21.15.114:0/155242102 >> 172.21.15.114:0/155242102 conn(0x55c5b07c88c0 msgr2=0x55c5b07c7b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:44.886 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.880+0000 7f854f7fe640 0 client.18102 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:31:44.889 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.884+0000 7f857921f740 1 -- 172.21.15.114:0/155242102 shutdown_connections 2024-09-13T18:31:44.889 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.884+0000 7f857921f740 1 -- 172.21.15.114:0/155242102 wait complete. 2024-09-13T18:31:44.890 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.888+0000 7f857921f740 20 client.18102 trim_cache size 0 max 16384 2024-09-13T18:31:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.928+0000 7f857921f740 1 Processor -- start 2024-09-13T18:31:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.930+0000 7f857921f740 1 -- start start 2024-09-13T18:31:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.930+0000 7f857921f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5b07cc330 0x55c5b07d6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.930+0000 7f857921f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5b07ccc50 0x55c5b07de060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.930+0000 7f857921f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c5b07d74f0 0x55c5b07de5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.930+0000 7f857921f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5b07e3a30 con 0x55c5b07d74f0 2024-09-13T18:31:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.930+0000 7f857921f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5b07cd2d0 con 0x55c5b07ccc50 2024-09-13T18:31:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.930+0000 7f857921f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5b07c94c0 con 0x55c5b07cc330 2024-09-13T18:31:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.932+0000 7f856f7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5b07ccc50 0x55c5b07de060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.932+0000 7f856f7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5b07ccc50 0x55c5b07de060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:45994/0 (socket says 172.21.15.114:45994) 2024-09-13T18:31:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.932+0000 7f856f7fe640 1 -- 172.21.15.114:0/1454007547 learned_addr learned my addr 172.21.15.114:0/1454007547 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.933+0000 7f85749bd640 1 --2- 172.21.15.114:0/1454007547 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c5b07d74f0 0x55c5b07de5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.933+0000 7f856ffff640 1 --2- 172.21.15.114:0/1454007547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5b07cc330 0x55c5b07d6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.935+0000 7f85749bd640 1 -- 172.21.15.114:0/1454007547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5b07cc330 msgr2=0x55c5b07d6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.935+0000 7f85749bd640 1 --2- 172.21.15.114:0/1454007547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5b07cc330 0x55c5b07d6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.935+0000 7f85749bd640 1 -- 172.21.15.114:0/1454007547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5b07ccc50 msgr2=0x55c5b07de060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.935+0000 7f85749bd640 1 --2- 172.21.15.114:0/1454007547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5b07ccc50 0x55c5b07de060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.935+0000 7f85749bd640 1 -- 172.21.15.114:0/1454007547 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5b06d1300 con 0x55c5b07d74f0 2024-09-13T18:31:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.937+0000 7f85749bd640 1 --2- 172.21.15.114:0/1454007547 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c5b07d74f0 0x55c5b07de5a0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f8568005fa0 tx=0x7f8568040fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c9ec1fb0328e928 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:44.942 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.940+0000 7f855affd640 1 -- 172.21.15.114:0/1454007547 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85680e5260 con 0x55c5b07d74f0 2024-09-13T18:31:44.942 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.940+0000 7f855affd640 1 -- 172.21.15.114:0/1454007547 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f85680cc890 con 0x55c5b07d74f0 2024-09-13T18:31:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.940+0000 7f855affd640 1 -- 172.21.15.114:0/1454007547 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85680c9170 con 0x55c5b07d74f0 2024-09-13T18:31:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.941+0000 7f857921f740 1 -- 172.21.15.114:0/1454007547 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c5b07d74f0 msgr2=0x55c5b07de5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.941+0000 7f857921f740 1 --2- 172.21.15.114:0/1454007547 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c5b07d74f0 0x55c5b07de5a0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f8568005fa0 tx=0x7f8568040fc0 comp rx=0 tx=0).stop 2024-09-13T18:31:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.942+0000 7f857921f740 1 -- 172.21.15.114:0/1454007547 shutdown_connections 2024-09-13T18:31:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.942+0000 7f857921f740 1 --2- 172.21.15.114:0/1454007547 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c5b07d74f0 0x55c5b07de5a0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.942+0000 7f857921f740 1 --2- 172.21.15.114:0/1454007547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5b07ccc50 0x55c5b07de060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.942+0000 7f857921f740 1 --2- 172.21.15.114:0/1454007547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5b07cc330 0x55c5b07d6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.942+0000 7f857921f740 1 -- 172.21.15.114:0/1454007547 >> 172.21.15.114:0/1454007547 conn(0x55c5b07c88c0 msgr2=0x55c5b07edb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.948+0000 7f857921f740 1 -- 172.21.15.114:0/1454007547 shutdown_connections 2024-09-13T18:31:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.949+0000 7f857921f740 1 -- 172.21.15.114:0/1454007547 wait complete. 2024-09-13T18:31:44.952 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.949+0000 7f857921f740 1 -- start start 2024-09-13T18:31:44.952 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.949+0000 7f857921f740 1 Processor -- start 2024-09-13T18:31:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.950+0000 7f857921f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c5b07cc330 0x55c5b07dbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.950+0000 7f857921f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5b07ccc50 0x55c5b07dc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.950+0000 7f857921f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5b07d74f0 0x55c5b07dc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.950+0000 7f857921f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5b083c4c0 con 0x55c5b07cc330 2024-09-13T18:31:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.950+0000 7f857921f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5b083c630 con 0x55c5b07ccc50 2024-09-13T18:31:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.950+0000 7f857921f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5b083c7f0 con 0x55c5b07d74f0 2024-09-13T18:31:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.950+0000 7f857921f740 10 client.0 _ll_register_callbacks cb 0x7f8568059f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:31:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.950+0000 7f856f7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5b07ccc50 0x55c5b07dc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.950+0000 7f856f7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5b07ccc50 0x55c5b07dc1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:46006/0 (socket says 172.21.15.114:46006) 2024-09-13T18:31:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.950+0000 7f856f7fe640 1 -- 172.21.15.114:0/3854800521 learned_addr learned my addr 172.21.15.114:0/3854800521 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.950+0000 7f856f7fe640 1 -- 172.21.15.114:0/3854800521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5b07d74f0 msgr2=0x55c5b07dc6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:31:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.950+0000 7f856f7fe640 1 --2- 172.21.15.114:0/3854800521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5b07d74f0 0x55c5b07dc6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.950+0000 7f856f7fe640 1 -- 172.21.15.114:0/3854800521 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c5b07cc330 msgr2=0x55c5b07dbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.950+0000 7f856f7fe640 1 --2- 172.21.15.114:0/3854800521 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c5b07cc330 0x55c5b07dbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.951+0000 7f856f7fe640 1 -- 172.21.15.114:0/3854800521 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c5b06d7910 con 0x55c5b07ccc50 2024-09-13T18:31:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.951+0000 7f856f7fe640 1 --2- 172.21.15.114:0/3854800521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5b07ccc50 0x55c5b07dc1b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f855c006c40 tx=0x7f855c009f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=895a1912cee20f47 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.953+0000 7f854f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:31:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.955+0000 7f854f7fe640 1 -- 172.21.15.114:0/3854800521 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f855c01b040 con 0x55c5b07ccc50 2024-09-13T18:31:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.955+0000 7f854f7fe640 1 -- 172.21.15.114:0/3854800521 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f855c023420 con 0x55c5b07ccc50 2024-09-13T18:31:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.956+0000 7f857921f740 1 -- 172.21.15.114:0/3854800521 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f85680d06c0 con 0x55c5b07ccc50 2024-09-13T18:31:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.960+0000 7f854f7fe640 1 -- 172.21.15.114:0/3854800521 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f855c0235e0 con 0x55c5b07ccc50 2024-09-13T18:31:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.961+0000 7f857921f740 10 client.27542 fetch_fsmap learned FSMap version 34 2024-09-13T18:31:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.961+0000 7f857921f740 1 -- 172.21.15.114:0/3854800521 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55c5b06f68e0 con 0x55c5b07ccc50 2024-09-13T18:31:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.966+0000 7f854f7fe640 1 -- 172.21.15.114:0/3854800521 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f85680d06c0 con 0x55c5b07ccc50 2024-09-13T18:31:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.966+0000 7f854f7fe640 1 -- 172.21.15.114:0/3854800521 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f855c011040 con 0x55c5b07ccc50 2024-09-13T18:31:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.966+0000 7f857921f740 10 client.27542 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:31:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.966+0000 7f857921f740 10 client.27542 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:31:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.966+0000 7f857921f740 20 client.27542 populate_metadata read hostname 'smithi114' 2024-09-13T18:31:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.966+0000 7f857921f740 1 --2- 172.21.15.114:0/3854800521 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55c5b083e0b0 0x55c5b08404d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.966+0000 7f857921f740 4 client.27542 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T18:31:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.966+0000 7f857921f740 1 -- 172.21.15.114:0/3854800521 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c5b0424300 con 0x55c5b083e0b0 2024-09-13T18:31:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.968+0000 7f856ffff640 1 --2- 172.21.15.114:0/3854800521 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55c5b083e0b0 0x55c5b08404d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.968+0000 7f856ffff640 1 --2- 172.21.15.114:0/3854800521 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55c5b083e0b0 0x55c5b08404d0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2c995b575e7517 server_cookie=167a52cf97bc7730 in_seq=0 out_seq=0 2024-09-13T18:31:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.969+0000 7f854f7fe640 10 client.27542 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:31:44.999 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.997+0000 7f854f7fe640 1 -- 172.21.15.114:0/3854800521 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c5b0424300 con 0x55c5b083e0b0 2024-09-13T18:31:44.999 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.997+0000 7f854f7fe640 10 client.27542 handle_command_reply: tid=0 2024-09-13T18:31:44.999 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.997+0000 7f857921f740 1 client.27542 shutdown 2024-09-13T18:31:44.999 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.997+0000 7f857921f740 1 -- 172.21.15.114:0/3854800521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5b07ccc50 msgr2=0x55c5b07dc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:45.000 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:44.997+0000 7f857921f740 1 --2- 172.21.15.114:0/3854800521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5b07ccc50 0x55c5b07dc1b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f855c006c40 tx=0x7f855c009f20 comp rx=0 tx=0).stop 2024-09-13T18:31:45.002 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:45.000+0000 7f857921f740 1 -- 172.21.15.114:0/3854800521 shutdown_connections 2024-09-13T18:31:45.003 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:45.000+0000 7f857921f740 1 --2- 172.21.15.114:0/3854800521 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55c5b083e0b0 0x55c5b08404d0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:45.003 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:45.000+0000 7f857921f740 1 --2- 172.21.15.114:0/3854800521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5b07d74f0 0x55c5b07dc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:45.003 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:45.000+0000 7f857921f740 1 --2- 172.21.15.114:0/3854800521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5b07ccc50 0x55c5b07dc1b0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:45.003 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:45.000+0000 7f857921f740 1 --2- 172.21.15.114:0/3854800521 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c5b07cc330 0x55c5b07dbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:45.003 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:45.000+0000 7f857921f740 1 -- 172.21.15.114:0/3854800521 >> 172.21.15.114:0/3854800521 conn(0x55c5b07c88c0 msgr2=0x55c5b07d4040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:45.003 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:45.000+0000 7f854f7fe640 0 client.27542 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:31:45.007 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:45.005+0000 7f857921f740 1 -- 172.21.15.114:0/3854800521 shutdown_connections 2024-09-13T18:31:45.007 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:45.005+0000 7f857921f740 1 -- 172.21.15.114:0/3854800521 wait complete. 2024-09-13T18:31:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:45.010+0000 7f857921f740 20 client.27542 trim_cache size 0 max 16384 2024-09-13T18:31:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-13T18:31:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-13T18:31:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-13T18:31:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-13T18:31:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:45.011+0000 7f8576420640 1 -- 172.21.15.114:0/169495299 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f85600a1410 msgr2=0x7f85600a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:45.011+0000 7f8576420640 1 --2- 172.21.15.114:0/169495299 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f85600a1410 0x7f85600a38d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8564009a30 tx=0x7f8564009190 comp rx=0 tx=0).stop 2024-09-13T18:31:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:45.011+0000 7f8576420640 1 -- 172.21.15.114:0/169495299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8570119400 msgr2=0x7f8570114ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:45.011+0000 7f8576420640 1 --2- 172.21.15.114:0/169495299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8570119400 0x7f8570114ae0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f856803bf10 tx=0x7f8568040cc0 comp rx=0 tx=0).stop 2024-09-13T18:31:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:45.011+0000 7f8576420640 1 -- 172.21.15.114:0/169495299 shutdown_connections 2024-09-13T18:31:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:45.011+0000 7f8576420640 1 --2- 172.21.15.114:0/169495299 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f85600a1410 0x7f85600a38d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:45.011+0000 7f8576420640 1 --2- 172.21.15.114:0/169495299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8570119400 0x7f8570114ae0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:45.011+0000 7f8576420640 1 --2- 172.21.15.114:0/169495299 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8570107890 0x7f8570118e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:45.015 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:45.011+0000 7f8576420640 1 --2- 172.21.15.114:0/169495299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85701074b0 0x7f8570118930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:45.015 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:45.011+0000 7f8576420640 1 -- 172.21.15.114:0/169495299 >> 172.21.15.114:0/169495299 conn(0x7f857006af50 msgr2=0x7f857006ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:45.019 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:45.017+0000 7f8576420640 1 -- 172.21.15.114:0/169495299 shutdown_connections 2024-09-13T18:31:45.019 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:45.017+0000 7f8576420640 1 -- 172.21.15.114:0/169495299 wait complete. 2024-09-13T18:31:45.418 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T18:31:45.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.647+0000 7f181bb83640 1 -- 172.21.15.114:0/332982862 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1814119400 msgr2=0x7f181411b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:45.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.647+0000 7f181bb83640 1 --2- 172.21.15.114:0/332982862 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1814119400 0x7f181411b7f0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f180c01acb0 tx=0x7f180c03b920 comp rx=0 tx=0).stop 2024-09-13T18:31:45.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.649+0000 7f181bb83640 1 -- 172.21.15.114:0/332982862 shutdown_connections 2024-09-13T18:31:45.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.649+0000 7f181bb83640 1 --2- 172.21.15.114:0/332982862 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1814119400 0x7f181411b7f0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:45.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.649+0000 7f181bb83640 1 --2- 172.21.15.114:0/332982862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1814107920 0x7f1814107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:45.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.649+0000 7f181bb83640 1 --2- 172.21.15.114:0/332982862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1814107540 0x7f181410e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:45.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.649+0000 7f181bb83640 1 -- 172.21.15.114:0/332982862 >> 172.21.15.114:0/332982862 conn(0x7f181406afe0 msgr2=0x7f181406b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:45.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.653+0000 7f181bb83640 1 -- 172.21.15.114:0/332982862 shutdown_connections 2024-09-13T18:31:45.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.658+0000 7f181bb83640 1 -- 172.21.15.114:0/332982862 wait complete. 2024-09-13T18:31:45.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.658+0000 7f181bb83640 1 Processor -- start 2024-09-13T18:31:45.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.659+0000 7f181bb83640 1 -- start start 2024-09-13T18:31:45.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.659+0000 7f181bb83640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1814107540 0x7f1814114c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:45.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.659+0000 7f181bb83640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1814107920 0x7f18141151c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:45.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.659+0000 7f181bb83640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1814119400 0x7f18141a1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:45.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.659+0000 7f181bb83640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18141a19d0 con 0x7f1814107540 2024-09-13T18:31:45.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.659+0000 7f181bb83640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18141a1b40 con 0x7f1814107920 2024-09-13T18:31:45.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.659+0000 7f181bb83640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18141a1d00 con 0x7f1814119400 2024-09-13T18:31:45.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.661+0000 7f181a120640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1814119400 0x7f18141a1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:45.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.661+0000 7f181a120640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1814119400 0x7f18141a1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:41328/0 (socket says 172.21.15.114:41328) 2024-09-13T18:31:45.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.661+0000 7f181a120640 1 -- 172.21.15.114:0/415113644 learned_addr learned my addr 172.21.15.114:0/415113644 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:45.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.661+0000 7f181a120640 1 -- 172.21.15.114:0/415113644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1814107920 msgr2=0x7f18141151c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:31:45.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.661+0000 7f181a120640 1 --2- 172.21.15.114:0/415113644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1814107920 0x7f18141151c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:45.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.662+0000 7f181a120640 1 -- 172.21.15.114:0/415113644 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1814107540 msgr2=0x7f1814114c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:45.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.662+0000 7f181a120640 1 --2- 172.21.15.114:0/415113644 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1814107540 0x7f1814114c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:45.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.662+0000 7f181a120640 1 -- 172.21.15.114:0/415113644 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f180c01a910 con 0x7f1814119400 2024-09-13T18:31:45.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.662+0000 7f181a120640 1 --2- 172.21.15.114:0/415113644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1814119400 0x7f18141a1240 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f180c040580 tx=0x7f180c0405b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:45.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.662+0000 7f180affd640 1 -- 172.21.15.114:0/415113644 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180c004390 con 0x7f1814119400 2024-09-13T18:31:45.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.663+0000 7f180affd640 1 -- 172.21.15.114:0/415113644 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f180c041620 con 0x7f1814119400 2024-09-13T18:31:45.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.663+0000 7f180affd640 1 -- 172.21.15.114:0/415113644 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180c04a680 con 0x7f1814119400 2024-09-13T18:31:45.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.667+0000 7f181bb83640 1 -- 172.21.15.114:0/415113644 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18141a1f80 con 0x7f1814119400 2024-09-13T18:31:45.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.667+0000 7f181bb83640 1 -- 172.21.15.114:0/415113644 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18141a94e0 con 0x7f1814119400 2024-09-13T18:31:45.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.672+0000 7f181e8a5740 1 Processor -- start 2024-09-13T18:31:45.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.677+0000 7f180affd640 1 -- 172.21.15.114:0/415113644 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f180c041790 con 0x7f1814119400 2024-09-13T18:31:45.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.678+0000 7f180affd640 1 --2- 172.21.15.114:0/415113644 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f18000a1340 0x7f18000a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:45.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.678+0000 7f180affd640 1 -- 172.21.15.114:0/415113644 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f180c0ff2c0 con 0x7f1814119400 2024-09-13T18:31:45.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.680+0000 7f181e8a5740 1 -- start start 2024-09-13T18:31:45.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.680+0000 7f181e8a5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56318db31e10 0x56318db321f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:45.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.680+0000 7f181e8a5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56318db32730 0x56318db3ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:45.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.680+0000 7f181e8a5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56318db3cfd0 0x56318db3f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:45.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.680+0000 7f181e8a5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56318da9cf20 con 0x56318db31e10 2024-09-13T18:31:45.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.680+0000 7f181e8a5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56318da9a6b0 con 0x56318db32730 2024-09-13T18:31:45.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.680+0000 7f181e8a5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56318da95fa0 con 0x56318db3cfd0 2024-09-13T18:31:45.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.680+0000 7f181911e640 1 --2- 172.21.15.114:0/415113644 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f18000a1340 0x7f18000a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:45.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.681+0000 7f181911e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56318db32730 0x56318db3ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:45.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.685+0000 7f181911e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56318db32730 0x56318db3ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:46020/0 (socket says 172.21.15.114:46020) 2024-09-13T18:31:45.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.685+0000 7f181911e640 1 -- 172.21.15.114:0/2648383246 learned_addr learned my addr 172.21.15.114:0/2648383246 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:45.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.685+0000 7f181911e640 1 -- 172.21.15.114:0/2648383246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56318db3cfd0 msgr2=0x56318db3f3c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:31:45.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.685+0000 7f181911e640 1 --2- 172.21.15.114:0/2648383246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56318db3cfd0 0x56318db3f3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:45.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.685+0000 7f181911e640 1 -- 172.21.15.114:0/2648383246 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56318db31e10 msgr2=0x56318db321f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:45.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.685+0000 7f181911e640 1 --2- 172.21.15.114:0/2648383246 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56318db31e10 0x56318db321f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:45.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.685+0000 7f181911e640 1 -- 172.21.15.114:0/2648383246 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56318da50840 con 0x56318db32730 2024-09-13T18:31:45.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.689+0000 7f181911e640 1 --2- 172.21.15.114:0/2648383246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56318db32730 0x56318db3ca90 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1804013cd0 tx=0x7f1804016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4dca6c1311fff25 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:45.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.689+0000 7f181911e640 1 --2- 172.21.15.114:0/415113644 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f18000a1340 0x7f18000a3800 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f180400add0 tx=0x7f18040137d0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:45.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.694+0000 7f17fffff640 1 -- 172.21.15.114:0/2648383246 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1804005020 con 0x56318db32730 2024-09-13T18:31:45.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.694+0000 7f17fffff640 1 -- 172.21.15.114:0/2648383246 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1804005610 con 0x56318db32730 2024-09-13T18:31:45.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.694+0000 7f17fffff640 1 -- 172.21.15.114:0/2648383246 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180401d460 con 0x56318db32730 2024-09-13T18:31:45.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.695+0000 7f181e8a5740 1 -- 172.21.15.114:0/2648383246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56318db32730 msgr2=0x56318db3ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:45.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.695+0000 7f181e8a5740 1 --2- 172.21.15.114:0/2648383246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56318db32730 0x56318db3ca90 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1804013cd0 tx=0x7f1804016000 comp rx=0 tx=0).stop 2024-09-13T18:31:45.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.697+0000 7f181e8a5740 1 -- 172.21.15.114:0/2648383246 shutdown_connections 2024-09-13T18:31:45.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.697+0000 7f181e8a5740 1 --2- 172.21.15.114:0/2648383246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56318db3cfd0 0x56318db3f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:45.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.697+0000 7f181e8a5740 1 --2- 172.21.15.114:0/2648383246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56318db32730 0x56318db3ca90 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:45.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.697+0000 7f181e8a5740 1 --2- 172.21.15.114:0/2648383246 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56318db31e10 0x56318db321f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:45.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.697+0000 7f181e8a5740 1 -- 172.21.15.114:0/2648383246 >> 172.21.15.114:0/2648383246 conn(0x56318db2e380 msgr2=0x56318db2e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:45.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.699+0000 7f181e8a5740 1 -- 172.21.15.114:0/2648383246 shutdown_connections 2024-09-13T18:31:45.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.700+0000 7f181e8a5740 1 -- 172.21.15.114:0/2648383246 wait complete. 2024-09-13T18:31:45.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.700+0000 7f181e8a5740 1 -- start start 2024-09-13T18:31:45.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.701+0000 7f181e8a5740 1 Processor -- start 2024-09-13T18:31:45.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.701+0000 7f181e8a5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56318db31e10 0x56318db3ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:45.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.701+0000 7f181e8a5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56318db32730 0x56318db39ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:45.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.701+0000 7f181e8a5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56318db3cfd0 0x56318db3a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:45.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.701+0000 7f181e8a5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56318da372a0 con 0x56318db32730 2024-09-13T18:31:45.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.701+0000 7f181e8a5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56318d76e430 con 0x56318db3cfd0 2024-09-13T18:31:45.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.701+0000 7f181e8a5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56318da19680 con 0x56318db31e10 2024-09-13T18:31:45.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.701+0000 7f181e8a5740 10 client.0 _ll_register_callbacks cb 0x56318da8f610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:31:45.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.702+0000 7f181991f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56318db31e10 0x56318db3ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:45.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.702+0000 7f181991f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56318db31e10 0x56318db3ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:41340/0 (socket says 172.21.15.114:41340) 2024-09-13T18:31:45.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.702+0000 7f181991f640 1 -- 172.21.15.114:0/1193309662 learned_addr learned my addr 172.21.15.114:0/1193309662 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:45.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.702+0000 7f181a120640 1 --2- 172.21.15.114:0/1193309662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56318db3cfd0 0x56318db3a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:45.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.702+0000 7f181991f640 1 -- 172.21.15.114:0/1193309662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56318db3cfd0 msgr2=0x56318db3a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:45.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.702+0000 7f181991f640 1 --2- 172.21.15.114:0/1193309662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56318db3cfd0 0x56318db3a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:45.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.702+0000 7f181991f640 1 -- 172.21.15.114:0/1193309662 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56318db32730 msgr2=0x56318db39ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:45.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.702+0000 7f181991f640 1 --2- 172.21.15.114:0/1193309662 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56318db32730 0x56318db39ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:45.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.702+0000 7f181991f640 1 -- 172.21.15.114:0/1193309662 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1804019050 con 0x56318db31e10 2024-09-13T18:31:45.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.703+0000 7f181991f640 1 --2- 172.21.15.114:0/1193309662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56318db31e10 0x56318db3ca60 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1810009e70 tx=0x7f181000b4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2c701a563609cda server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:45.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.703+0000 7f17fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:31:45.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.707+0000 7f17fd7fa640 1 -- 172.21.15.114:0/1193309662 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1810011620 con 0x56318db31e10 2024-09-13T18:31:45.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.707+0000 7f181e8a5740 1 -- 172.21.15.114:0/1193309662 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56318d7894e0 con 0x56318db31e10 2024-09-13T18:31:45.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.707+0000 7f17fd7fa640 1 -- 172.21.15.114:0/1193309662 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f1810011780 con 0x56318db31e10 2024-09-13T18:31:45.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.707+0000 7f17fd7fa640 1 -- 172.21.15.114:0/1193309662 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f181000be90 con 0x56318db31e10 2024-09-13T18:31:45.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.713+0000 7f181e8a5740 10 client.27514 fetch_fsmap learned FSMap version 34 2024-09-13T18:31:45.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.713+0000 7f181e8a5740 1 -- 172.21.15.114:0/1193309662 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x56318da94620 con 0x56318db31e10 2024-09-13T18:31:45.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.714+0000 7f17fd7fa640 1 -- 172.21.15.114:0/1193309662 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56318d7894e0 con 0x56318db31e10 2024-09-13T18:31:45.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.714+0000 7f17fd7fa640 1 -- 172.21.15.114:0/1193309662 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1810019b00 con 0x56318db31e10 2024-09-13T18:31:45.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.714+0000 7f181e8a5740 10 client.27514 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:31:45.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.715+0000 7f181e8a5740 10 client.27514 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:31:45.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.715+0000 7f181e8a5740 20 client.27514 populate_metadata read hostname 'smithi114' 2024-09-13T18:31:45.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.715+0000 7f181e8a5740 1 --2- 172.21.15.114:0/1193309662 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56318db4d000 0x56318db4f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:45.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.715+0000 7f181e8a5740 4 client.27514 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:31:45.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.715+0000 7f181e8a5740 1 -- 172.21.15.114:0/1193309662 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56318d79a7e0 con 0x56318db4d000 2024-09-13T18:31:45.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.716+0000 7f181911e640 1 --2- 172.21.15.114:0/1193309662 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56318db4d000 0x56318db4f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:45.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.723+0000 7f181911e640 1 --2- 172.21.15.114:0/1193309662 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56318db4d000 0x56318db4f420 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9eed942b838185a3 server_cookie=80fff1a4c2a556ea in_seq=0 out_seq=0 2024-09-13T18:31:45.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.724+0000 7f17fd7fa640 10 client.27514 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:31:45.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.728+0000 7f17fd7fa640 1 -- 172.21.15.114:0/1193309662 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x56318d79a7e0 con 0x56318db4d000 2024-09-13T18:31:45.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.728+0000 7f17fd7fa640 10 client.27514 handle_command_reply: tid=0 2024-09-13T18:31:45.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.728+0000 7f181e8a5740 1 client.27514 shutdown 2024-09-13T18:31:45.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.732+0000 7f181e8a5740 1 -- 172.21.15.114:0/1193309662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56318db31e10 msgr2=0x56318db3ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:45.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.732+0000 7f181e8a5740 1 --2- 172.21.15.114:0/1193309662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56318db31e10 0x56318db3ca60 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1810009e70 tx=0x7f181000b4b0 comp rx=0 tx=0).stop 2024-09-13T18:31:45.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.741+0000 7f181e8a5740 1 -- 172.21.15.114:0/1193309662 shutdown_connections 2024-09-13T18:31:45.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.741+0000 7f181e8a5740 1 --2- 172.21.15.114:0/1193309662 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56318db4d000 0x56318db4f420 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:45.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.741+0000 7f181e8a5740 1 --2- 172.21.15.114:0/1193309662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56318db3cfd0 0x56318db3a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:45.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.741+0000 7f181e8a5740 1 --2- 172.21.15.114:0/1193309662 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56318db32730 0x56318db39ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:45.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.741+0000 7f181e8a5740 1 --2- 172.21.15.114:0/1193309662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56318db31e10 0x56318db3ca60 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:45.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.741+0000 7f181e8a5740 1 -- 172.21.15.114:0/1193309662 >> 172.21.15.114:0/1193309662 conn(0x56318db2e380 msgr2=0x56318db340e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:45.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.742+0000 7f17fd7fa640 0 client.27514 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:31:45.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.744+0000 7f181e8a5740 1 -- 172.21.15.114:0/1193309662 shutdown_connections 2024-09-13T18:31:45.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.746+0000 7f181e8a5740 1 -- 172.21.15.114:0/1193309662 wait complete. 2024-09-13T18:31:45.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.755+0000 7f181e8a5740 20 client.27514 trim_cache size 0 max 16384 2024-09-13T18:31:45.761 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:45 smithi114 ceph-mon[30897]: pgmap v287: 129 pgs: 129 active+clean; 22 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 30 MiB/s wr, 661 op/s 2024-09-13T18:31:45.761 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:45 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:31:45.761 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:45 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:31:45.761 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:45 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:31:45.761 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:45 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:31:45.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.791+0000 7f181e8a5740 1 Processor -- start 2024-09-13T18:31:45.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.791+0000 7f181e8a5740 1 -- start start 2024-09-13T18:31:45.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.791+0000 7f181e8a5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56318db31e10 0x56318db3c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:45.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.792+0000 7f181991f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56318db31e10 0x56318db3c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:45.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.792+0000 7f181991f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56318db31e10 0x56318db3c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:41352/0 (socket says 172.21.15.114:41352) 2024-09-13T18:31:45.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.792+0000 7f181991f640 1 -- 172.21.15.114:0/583296819 learned_addr learned my addr 172.21.15.114:0/583296819 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:45.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.792+0000 7f181e8a5740 1 --2- 172.21.15.114:0/583296819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56318db32730 0x56318db39580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:45.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.792+0000 7f181e8a5740 1 --2- 172.21.15.114:0/583296819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56318db3cfd0 0x56318db39ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:45.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.792+0000 7f181911e640 1 --2- 172.21.15.114:0/583296819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56318db32730 0x56318db39580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:45.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.792+0000 7f181e8a5740 1 -- 172.21.15.114:0/583296819 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56318db2f510 con 0x56318db3cfd0 2024-09-13T18:31:45.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.792+0000 7f181e8a5740 1 -- 172.21.15.114:0/583296819 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56318db31b90 con 0x56318db32730 2024-09-13T18:31:45.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.792+0000 7f181e8a5740 1 -- 172.21.15.114:0/583296819 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56318db28660 con 0x56318db31e10 2024-09-13T18:31:45.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.794+0000 7f181a120640 1 --2- 172.21.15.114:0/583296819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56318db3cfd0 0x56318db39ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:45.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.794+0000 7f181991f640 1 -- 172.21.15.114:0/583296819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56318db32730 msgr2=0x56318db39580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:45.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.794+0000 7f181991f640 1 --2- 172.21.15.114:0/583296819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56318db32730 0x56318db39580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:45.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.794+0000 7f181991f640 1 -- 172.21.15.114:0/583296819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56318db3cfd0 msgr2=0x56318db39ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:45.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.794+0000 7f181991f640 1 --2- 172.21.15.114:0/583296819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56318db3cfd0 0x56318db39ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:45.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.794+0000 7f181991f640 1 -- 172.21.15.114:0/583296819 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56318da36790 con 0x56318db31e10 2024-09-13T18:31:45.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.795+0000 7f181991f640 1 --2- 172.21.15.114:0/583296819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56318db31e10 0x56318db3c570 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f18100091e0 tx=0x7f18100099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=313090891200985b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:45.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.796+0000 7f1808e05640 1 -- 172.21.15.114:0/583296819 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181000be90 con 0x56318db31e10 2024-09-13T18:31:45.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.796+0000 7f1808e05640 1 -- 172.21.15.114:0/583296819 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1810010520 con 0x56318db31e10 2024-09-13T18:31:45.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.796+0000 7f1808e05640 1 -- 172.21.15.114:0/583296819 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1810019760 con 0x56318db31e10 2024-09-13T18:31:45.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.802+0000 7f181e8a5740 1 -- 172.21.15.114:0/583296819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56318db31e10 msgr2=0x56318db3c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:45.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.802+0000 7f181e8a5740 1 --2- 172.21.15.114:0/583296819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56318db31e10 0x56318db3c570 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f18100091e0 tx=0x7f18100099e0 comp rx=0 tx=0).stop 2024-09-13T18:31:45.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.802+0000 7f181e8a5740 1 -- 172.21.15.114:0/583296819 shutdown_connections 2024-09-13T18:31:45.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.802+0000 7f181e8a5740 1 --2- 172.21.15.114:0/583296819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56318db3cfd0 0x56318db39ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:45.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.802+0000 7f181e8a5740 1 --2- 172.21.15.114:0/583296819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56318db32730 0x56318db39580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:45.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.802+0000 7f181e8a5740 1 --2- 172.21.15.114:0/583296819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56318db31e10 0x56318db3c570 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:45.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.802+0000 7f181e8a5740 1 -- 172.21.15.114:0/583296819 >> 172.21.15.114:0/583296819 conn(0x56318db2e380 msgr2=0x56318db34fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:45.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.805+0000 7f181e8a5740 1 -- 172.21.15.114:0/583296819 shutdown_connections 2024-09-13T18:31:45.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.806+0000 7f181e8a5740 1 -- 172.21.15.114:0/583296819 wait complete. 2024-09-13T18:31:45.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.806+0000 7f181e8a5740 1 -- start start 2024-09-13T18:31:45.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.806+0000 7f181e8a5740 1 Processor -- start 2024-09-13T18:31:45.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.807+0000 7f181e8a5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56318db31e10 0x56318db43d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:45.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.807+0000 7f181e8a5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56318db32730 0x56318db442b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:45.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.807+0000 7f181e8a5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56318db3cfd0 0x56318db35fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:45.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.807+0000 7f181e8a5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56318db364f0 con 0x56318db31e10 2024-09-13T18:31:45.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.807+0000 7f181e8a5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56318db36660 con 0x56318db3cfd0 2024-09-13T18:31:45.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.807+0000 7f181e8a5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56318db36820 con 0x56318db32730 2024-09-13T18:31:45.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.807+0000 7f181911e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56318db32730 0x56318db442b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:45.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.807+0000 7f181911e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56318db32730 0x56318db442b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:41366/0 (socket says 172.21.15.114:41366) 2024-09-13T18:31:45.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.807+0000 7f181911e640 1 -- 172.21.15.114:0/2314571564 learned_addr learned my addr 172.21.15.114:0/2314571564 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:45.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.807+0000 7f181a120640 1 --2- 172.21.15.114:0/2314571564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56318db3cfd0 0x56318db35fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:45.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.807+0000 7f181e8a5740 10 client.0 _ll_register_callbacks cb 0x7f1810006f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:31:45.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.810+0000 7f181911e640 1 -- 172.21.15.114:0/2314571564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56318db3cfd0 msgr2=0x56318db35fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:45.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.810+0000 7f181911e640 1 --2- 172.21.15.114:0/2314571564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56318db3cfd0 0x56318db35fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:45.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.810+0000 7f181911e640 1 -- 172.21.15.114:0/2314571564 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56318db31e10 msgr2=0x56318db43d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:45.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.810+0000 7f181911e640 1 --2- 172.21.15.114:0/2314571564 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56318db31e10 0x56318db43d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:45.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.810+0000 7f181911e640 1 -- 172.21.15.114:0/2314571564 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56318da94620 con 0x56318db32730 2024-09-13T18:31:45.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.812+0000 7f181911e640 1 --2- 172.21.15.114:0/2314571564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56318db32730 0x56318db442b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1804010bd0 tx=0x7f180400a500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a508e4e9c1b85e78 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:45.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.813+0000 7f17fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:31:45.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.813+0000 7f17fd7fa640 1 -- 172.21.15.114:0/2314571564 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1804004680 con 0x56318db32730 2024-09-13T18:31:45.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.813+0000 7f181e8a5740 1 -- 172.21.15.114:0/2314571564 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1804005610 con 0x56318db32730 2024-09-13T18:31:45.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.814+0000 7f17fd7fa640 1 -- 172.21.15.114:0/2314571564 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f180402a040 con 0x56318db32730 2024-09-13T18:31:45.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.815+0000 7f17fd7fa640 1 -- 172.21.15.114:0/2314571564 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f18040047e0 con 0x56318db32730 2024-09-13T18:31:45.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.815+0000 7f181e8a5740 10 client.27520 fetch_fsmap learned FSMap version 34 2024-09-13T18:31:45.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.815+0000 7f181e8a5740 1 -- 172.21.15.114:0/2314571564 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x56318da5bea0 con 0x56318db32730 2024-09-13T18:31:45.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.815+0000 7f17fd7fa640 1 -- 172.21.15.114:0/2314571564 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1804005490 con 0x56318db32730 2024-09-13T18:31:45.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.815+0000 7f17fd7fa640 1 -- 172.21.15.114:0/2314571564 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f180402e380 con 0x56318db32730 2024-09-13T18:31:45.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.815+0000 7f181e8a5740 10 client.27520 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:31:45.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.815+0000 7f181e8a5740 10 client.27520 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:31:45.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.815+0000 7f181e8a5740 20 client.27520 populate_metadata read hostname 'smithi114' 2024-09-13T18:31:45.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.816+0000 7f181e8a5740 1 --2- 172.21.15.114:0/2314571564 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56318dba4480 0x56318dba68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:45.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.816+0000 7f181e8a5740 4 client.27520 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T18:31:45.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.816+0000 7f181e8a5740 1 -- 172.21.15.114:0/2314571564 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f1804005610 con 0x56318dba4480 2024-09-13T18:31:45.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.817+0000 7f181991f640 1 --2- 172.21.15.114:0/2314571564 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56318dba4480 0x56318dba68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:45.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.817+0000 7f17fd7fa640 10 client.27520 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:31:45.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.824+0000 7f181991f640 1 --2- 172.21.15.114:0/2314571564 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56318dba4480 0x56318dba68a0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b183f3c9d5e8a05 server_cookie=2fe1c8c36100c76d in_seq=0 out_seq=0 2024-09-13T18:31:45.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.825+0000 7f17fd7fa640 1 -- 172.21.15.114:0/2314571564 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x7f1804005610 con 0x56318dba4480 2024-09-13T18:31:45.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.825+0000 7f17fd7fa640 10 client.27520 handle_command_reply: tid=0 2024-09-13T18:31:45.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.825+0000 7f181e8a5740 1 client.27520 shutdown 2024-09-13T18:31:45.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.829+0000 7f181e8a5740 1 -- 172.21.15.114:0/2314571564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56318db32730 msgr2=0x56318db442b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:45.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.829+0000 7f181e8a5740 1 --2- 172.21.15.114:0/2314571564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56318db32730 0x56318db442b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1804010bd0 tx=0x7f180400a500 comp rx=0 tx=0).stop 2024-09-13T18:31:45.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.829+0000 7f181e8a5740 1 -- 172.21.15.114:0/2314571564 shutdown_connections 2024-09-13T18:31:45.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.829+0000 7f181e8a5740 1 --2- 172.21.15.114:0/2314571564 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x56318dba4480 0x56318dba68a0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:45.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.829+0000 7f181e8a5740 1 --2- 172.21.15.114:0/2314571564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56318db3cfd0 0x56318db35fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:45.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.829+0000 7f181e8a5740 1 --2- 172.21.15.114:0/2314571564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56318db32730 0x56318db442b0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:45.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.829+0000 7f181e8a5740 1 --2- 172.21.15.114:0/2314571564 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56318db31e10 0x56318db43d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:45.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.829+0000 7f181e8a5740 1 -- 172.21.15.114:0/2314571564 >> 172.21.15.114:0/2314571564 conn(0x56318db2e380 msgr2=0x56318db39580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:45.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.829+0000 7f17fd7fa640 0 client.27520 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:31:45.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.829+0000 7f181e8a5740 1 -- 172.21.15.114:0/2314571564 shutdown_connections 2024-09-13T18:31:45.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.829+0000 7f181e8a5740 1 -- 172.21.15.114:0/2314571564 wait complete. 2024-09-13T18:31:45.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.830+0000 7f181e8a5740 20 client.27520 trim_cache size 0 max 16384 2024-09-13T18:31:45.833 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:31:45.833 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "no active scrubs running", 2024-09-13T18:31:45.833 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": {} 2024-09-13T18:31:45.833 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:31:45.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.832+0000 7f181bb83640 1 -- 172.21.15.114:0/415113644 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f18000a1340 msgr2=0x7f18000a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:45.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.832+0000 7f181bb83640 1 --2- 172.21.15.114:0/415113644 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f18000a1340 0x7f18000a3800 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f180400add0 tx=0x7f18040137d0 comp rx=0 tx=0).stop 2024-09-13T18:31:45.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.832+0000 7f181bb83640 1 -- 172.21.15.114:0/415113644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1814119400 msgr2=0x7f18141a1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:45.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.832+0000 7f181bb83640 1 --2- 172.21.15.114:0/415113644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1814119400 0x7f18141a1240 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f180c040580 tx=0x7f180c0405b0 comp rx=0 tx=0).stop 2024-09-13T18:31:45.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.832+0000 7f181bb83640 1 -- 172.21.15.114:0/415113644 shutdown_connections 2024-09-13T18:31:45.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.832+0000 7f181bb83640 1 --2- 172.21.15.114:0/415113644 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f18000a1340 0x7f18000a3800 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:45.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.832+0000 7f181bb83640 1 --2- 172.21.15.114:0/415113644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1814119400 0x7f18141a1240 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:45.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.832+0000 7f181bb83640 1 --2- 172.21.15.114:0/415113644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1814107920 0x7f18141151c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:45.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.832+0000 7f181bb83640 1 --2- 172.21.15.114:0/415113644 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1814107540 0x7f1814114c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:45.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.832+0000 7f181bb83640 1 -- 172.21.15.114:0/415113644 >> 172.21.15.114:0/415113644 conn(0x7f181406afe0 msgr2=0x7f181406ebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:45.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.832+0000 7f181bb83640 1 -- 172.21.15.114:0/415113644 shutdown_connections 2024-09-13T18:31:45.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:45.833+0000 7f181bb83640 1 -- 172.21.15.114:0/415113644 wait complete. 2024-09-13T18:31:45.873 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-13T18:31:45.874 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:31:46.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:45 smithi134 ceph-mon[34283]: pgmap v287: 129 pgs: 129 active+clean; 22 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 30 MiB/s wr, 661 op/s 2024-09-13T18:31:46.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:45 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:31:46.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:45 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:31:46.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:45 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:31:46.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:45 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:31:46.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.155+0000 7f7915508640 1 -- 172.21.15.114:0/2984464441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79100fed30 msgr2=0x7f79100ff110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:46.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.155+0000 7f7915508640 1 --2- 172.21.15.114:0/2984464441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79100fed30 0x7f79100ff110 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f790401a970 tx=0x7f790403b530 comp rx=0 tx=0).stop 2024-09-13T18:31:46.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.159+0000 7f7915508640 1 -- 172.21.15.114:0/2984464441 shutdown_connections 2024-09-13T18:31:46.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.159+0000 7f7915508640 1 --2- 172.21.15.114:0/2984464441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7910104160 0x7f7910104610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:46.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.159+0000 7f7915508640 1 --2- 172.21.15.114:0/2984464441 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79100ff650 0x7f7910103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:46.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.159+0000 7f7915508640 1 --2- 172.21.15.114:0/2984464441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79100fed30 0x7f79100ff110 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:46.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.159+0000 7f7915508640 1 -- 172.21.15.114:0/2984464441 >> 172.21.15.114:0/2984464441 conn(0x7f7910075c60 msgr2=0x7f79100fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:46.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.165+0000 7f7915508640 1 -- 172.21.15.114:0/2984464441 shutdown_connections 2024-09-13T18:31:46.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.169+0000 7f7915508640 1 -- 172.21.15.114:0/2984464441 wait complete. 2024-09-13T18:31:46.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.169+0000 7f7915508640 1 Processor -- start 2024-09-13T18:31:46.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.170+0000 7f7915508640 1 -- start start 2024-09-13T18:31:46.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.170+0000 7f7915508640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79100fed30 0x7f7910071a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:46.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.170+0000 7f7915508640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79100ff650 0x7f7910071f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:46.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.170+0000 7f7915508640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7910104160 0x7f7910113250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:46.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.170+0000 7f7915508640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f791006fc90 con 0x7f79100ff650 2024-09-13T18:31:46.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.170+0000 7f7915508640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f791006fe00 con 0x7f79100fed30 2024-09-13T18:31:46.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.170+0000 7f7915508640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f791006ffc0 con 0x7f7910104160 2024-09-13T18:31:46.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.171+0000 7f790e7fc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79100ff650 0x7f7910071f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:46.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.171+0000 7f790e7fc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79100ff650 0x7f7910071f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:58474/0 (socket says 172.21.15.114:58474) 2024-09-13T18:31:46.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.171+0000 7f790e7fc640 1 -- 172.21.15.114:0/546768164 learned_addr learned my addr 172.21.15.114:0/546768164 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:46.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.172+0000 7f790effd640 1 --2- 172.21.15.114:0/546768164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79100fed30 0x7f7910071a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:46.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.172+0000 7f790f7fe640 1 --2- 172.21.15.114:0/546768164 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7910104160 0x7f7910113250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:46.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.172+0000 7f790effd640 1 -- 172.21.15.114:0/546768164 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7910104160 msgr2=0x7f7910113250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:46.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.172+0000 7f790effd640 1 --2- 172.21.15.114:0/546768164 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7910104160 0x7f7910113250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:46.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.172+0000 7f790effd640 1 -- 172.21.15.114:0/546768164 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79100ff650 msgr2=0x7f7910071f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:46.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.172+0000 7f790effd640 1 --2- 172.21.15.114:0/546768164 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79100ff650 0x7f7910071f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:46.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.172+0000 7f790effd640 1 -- 172.21.15.114:0/546768164 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f790401a570 con 0x7f79100fed30 2024-09-13T18:31:46.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.173+0000 7f790effd640 1 --2- 172.21.15.114:0/546768164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79100fed30 0x7f7910071a20 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7904000c00 tx=0x7f7904005430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:46.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.175+0000 7f78f7fff640 1 -- 172.21.15.114:0/546768164 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7904004460 con 0x7f79100fed30 2024-09-13T18:31:46.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.175+0000 7f78f7fff640 1 -- 172.21.15.114:0/546768164 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7904005660 con 0x7f79100fed30 2024-09-13T18:31:46.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.175+0000 7f78f7fff640 1 -- 172.21.15.114:0/546768164 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79040409a0 con 0x7f79100fed30 2024-09-13T18:31:46.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.175+0000 7f7915508640 1 -- 172.21.15.114:0/546768164 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7910070240 con 0x7f79100fed30 2024-09-13T18:31:46.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.175+0000 7f7915508640 1 -- 172.21.15.114:0/546768164 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79100706a0 con 0x7f79100fed30 2024-09-13T18:31:46.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.177+0000 7f7915508640 1 -- 172.21.15.114:0/546768164 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78fc005250 con 0x7f79100fed30 2024-09-13T18:31:46.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.183+0000 7f78f7fff640 1 -- 172.21.15.114:0/546768164 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f7904040b00 con 0x7f79100fed30 2024-09-13T18:31:46.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.185+0000 7f78f7fff640 1 --2- 172.21.15.114:0/546768164 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f78f80a1410 0x7f78f80a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:46.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.185+0000 7f78f7fff640 1 -- 172.21.15.114:0/546768164 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f79040edf30 con 0x7f79100fed30 2024-09-13T18:31:46.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.185+0000 7f790e7fc640 1 --2- 172.21.15.114:0/546768164 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f78f80a1410 0x7f78f80a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:46.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.191+0000 7f78f7fff640 1 -- 172.21.15.114:0/546768164 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f79040b7000 con 0x7f79100fed30 2024-09-13T18:31:46.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:45 smithi204 ceph-mon[34777]: pgmap v287: 129 pgs: 129 active+clean; 22 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 30 MiB/s wr, 661 op/s 2024-09-13T18:31:46.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:45 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:31:46.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:45 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:31:46.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:45 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:31:46.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:45 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:31:46.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.206+0000 7f790e7fc640 1 --2- 172.21.15.114:0/546768164 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f78f80a1410 0x7f78f80a38d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f79100712c0 tx=0x7f7900007620 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:46.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.708+0000 7f7915508640 1 -- 172.21.15.114:0/546768164 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f78fc0050c0 con 0x7f79100fed30 2024-09-13T18:31:46.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.710+0000 7f78f7fff640 1 -- 172.21.15.114:0/546768164 <== mon.2 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+10930 (secure 0 0 0) 0x7f7904046030 con 0x7f79100fed30 2024-09-13T18:31:46.715 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:31:46.715 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4162817497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4162817497},{"type":"v1","addr":"172.21.15.114:6835","nonce":4162817497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/2435428478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":2435428478},{"type":"v1","addr":"172.21.15.114:6837","nonce":2435428478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T18:20:01.895117+0000","modified":"2024-09-13T18:27:22.172031+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24482,"mds_2":24475,"mds_3":24458,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/2046516451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":2046516451},{"type":"v1","addr":"172.21.15.114:6839","nonce":2046516451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":10,"addr":"172.21.15.114:6841/1956923446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":1956923446},{"type":"v1","addr":"172.21.15.114:6841","nonce":1956923446}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24391":{"gid":24391,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/507301496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":507301496},{"type":"v1","addr":"172.21.15.204:6833","nonce":507301496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4028657219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4028657219},{"type":"v1","addr":"172.21.15.134:6833","nonce":4028657219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/57029119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":57029119},{"type":"v1","addr":"172.21.15.204:6835","nonce":57029119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3068880939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3068880939},{"type":"v1","addr":"172.21.15.204:6837","nonce":3068880939}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"e","rank":3,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.134:6835/1150003275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1150003275},{"type":"v1","addr":"172.21.15.134:6835","nonce":1150003275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/344883912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":344883912},{"type":"v1","addr":"172.21.15.204:6839","nonce":344883912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6837/891692210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":891692210},{"type":"v1","addr":"172.21.15.134:6837","nonce":891692210}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6839/3466309724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3466309724},{"type":"v1","addr":"172.21.15.134:6839","nonce":3466309724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:31:46.715 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 34 2024-09-13T18:31:46.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.716+0000 7f7915508640 1 -- 172.21.15.114:0/546768164 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f78f80a1410 msgr2=0x7f78f80a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:46.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.716+0000 7f7915508640 1 --2- 172.21.15.114:0/546768164 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f78f80a1410 0x7f78f80a38d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f79100712c0 tx=0x7f7900007620 comp rx=0 tx=0).stop 2024-09-13T18:31:46.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.716+0000 7f7915508640 1 -- 172.21.15.114:0/546768164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79100fed30 msgr2=0x7f7910071a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:46.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.716+0000 7f7915508640 1 --2- 172.21.15.114:0/546768164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79100fed30 0x7f7910071a20 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7904000c00 tx=0x7f7904005430 comp rx=0 tx=0).stop 2024-09-13T18:31:46.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.717+0000 7f7915508640 1 -- 172.21.15.114:0/546768164 shutdown_connections 2024-09-13T18:31:46.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.717+0000 7f7915508640 1 --2- 172.21.15.114:0/546768164 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f78f80a1410 0x7f78f80a38d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:46.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.717+0000 7f7915508640 1 --2- 172.21.15.114:0/546768164 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7910104160 0x7f7910113250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:46.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.717+0000 7f7915508640 1 --2- 172.21.15.114:0/546768164 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79100ff650 0x7f7910071f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:46.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.717+0000 7f7915508640 1 --2- 172.21.15.114:0/546768164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79100fed30 0x7f7910071a20 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:46.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.717+0000 7f7915508640 1 -- 172.21.15.114:0/546768164 >> 172.21.15.114:0/546768164 conn(0x7f7910075c60 msgr2=0x7f7910108040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:46.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.718+0000 7f7915508640 1 -- 172.21.15.114:0/546768164 shutdown_connections 2024-09-13T18:31:46.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.718+0000 7f7915508640 1 -- 172.21.15.114:0/546768164 wait complete. 2024-09-13T18:31:46.741 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-09-13T18:31:46.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.842+0000 7ff0826d7640 1 -- 172.21.15.114:0/3363021102 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff07c10c580 msgr2=0x7ff07c10e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:46.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.842+0000 7ff0826d7640 1 --2- 172.21.15.114:0/3363021102 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff07c10c580 0x7ff07c10e9b0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7ff074009ca0 tx=0x7ff07402a950 comp rx=0 tx=0).stop 2024-09-13T18:31:46.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.842+0000 7ff0826d7640 1 -- 172.21.15.114:0/3363021102 shutdown_connections 2024-09-13T18:31:46.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.842+0000 7ff0826d7640 1 --2- 172.21.15.114:0/3363021102 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff07c10c580 0x7ff07c10e9b0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:46.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.842+0000 7ff0826d7640 1 --2- 172.21.15.114:0/3363021102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff07c0731c0 0x7ff07c10c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:46.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.842+0000 7ff0826d7640 1 --2- 172.21.15.114:0/3363021102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff07c0728a0 0x7ff07c072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:46.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.842+0000 7ff0826d7640 1 -- 172.21.15.114:0/3363021102 >> 172.21.15.114:0/3363021102 conn(0x7ff07c0fb9c0 msgr2=0x7ff07c0fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:46.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.842+0000 7ff0826d7640 1 -- 172.21.15.114:0/3363021102 shutdown_connections 2024-09-13T18:31:46.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.842+0000 7ff0826d7640 1 -- 172.21.15.114:0/3363021102 wait complete. 2024-09-13T18:31:46.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.843+0000 7ff0826d7640 1 Processor -- start 2024-09-13T18:31:46.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.843+0000 7ff0826d7640 1 -- start start 2024-09-13T18:31:46.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.843+0000 7ff0826d7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff07c0728a0 0x7ff07c1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:46.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.843+0000 7ff0826d7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff07c0731c0 0x7ff07c198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:46.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.843+0000 7ff0826d7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff07c10c580 0x7ff07c19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:46.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.843+0000 7ff0826d7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff07c19d320 con 0x7ff07c0731c0 2024-09-13T18:31:46.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.843+0000 7ff0826d7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff07c19d490 con 0x7ff07c10c580 2024-09-13T18:31:46.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.843+0000 7ff0826d7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff07c19d650 con 0x7ff07c0728a0 2024-09-13T18:31:46.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.843+0000 7ff073fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff07c0731c0 0x7ff07c198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:46.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.843+0000 7ff073fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff07c0731c0 0x7ff07c198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:58502/0 (socket says 172.21.15.114:58502) 2024-09-13T18:31:46.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.843+0000 7ff073fff640 1 -- 172.21.15.114:0/2701839025 learned_addr learned my addr 172.21.15.114:0/2701839025 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:46.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.843+0000 7ff080c74640 1 --2- 172.21.15.114:0/2701839025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff07c10c580 0x7ff07c19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:46.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.844+0000 7ff073fff640 1 -- 172.21.15.114:0/2701839025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff07c0728a0 msgr2=0x7ff07c1983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:46.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.844+0000 7ff073fff640 1 --2- 172.21.15.114:0/2701839025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff07c0728a0 0x7ff07c1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:46.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.844+0000 7ff073fff640 1 -- 172.21.15.114:0/2701839025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff07c10c580 msgr2=0x7ff07c19cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:46.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.844+0000 7ff073fff640 1 --2- 172.21.15.114:0/2701839025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff07c10c580 0x7ff07c19cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:46.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.844+0000 7ff073fff640 1 -- 172.21.15.114:0/2701839025 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff064009490 con 0x7ff07c0731c0 2024-09-13T18:31:46.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.844+0000 7ff073fff640 1 --2- 172.21.15.114:0/2701839025 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff07c0731c0 0x7ff07c198930 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7ff06400ea60 tx=0x7ff06400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:46.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.844+0000 7ff079ffb640 1 -- 172.21.15.114:0/2701839025 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06400cde0 con 0x7ff07c0731c0 2024-09-13T18:31:46.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.844+0000 7ff0826d7640 1 -- 172.21.15.114:0/2701839025 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff074009900 con 0x7ff07c0731c0 2024-09-13T18:31:46.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.844+0000 7ff079ffb640 1 -- 172.21.15.114:0/2701839025 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff0640044b0 con 0x7ff07c0731c0 2024-09-13T18:31:46.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.844+0000 7ff079ffb640 1 -- 172.21.15.114:0/2701839025 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff064010430 con 0x7ff07c0731c0 2024-09-13T18:31:46.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.844+0000 7ff0826d7640 1 -- 172.21.15.114:0/2701839025 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff07c1a4fd0 con 0x7ff07c0731c0 2024-09-13T18:31:46.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.848+0000 7ff0853fb740 1 Processor -- start 2024-09-13T18:31:46.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.848+0000 7ff079ffb640 1 -- 172.21.15.114:0/2701839025 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7ff064004000 con 0x7ff07c0731c0 2024-09-13T18:31:46.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.849+0000 7ff079ffb640 1 --2- 172.21.15.114:0/2701839025 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff0680a1280 0x7ff0680a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:46.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.849+0000 7ff0853fb740 1 -- start start 2024-09-13T18:31:46.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.849+0000 7ff079ffb640 1 -- 172.21.15.114:0/2701839025 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ff064014070 con 0x7ff07c0731c0 2024-09-13T18:31:46.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.849+0000 7ff0853fb740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604ea13c670 0x5604ea13ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:46.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.849+0000 7ff0853fb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604ea13cf90 0x5604ea14fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:46.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.849+0000 7ff0853fb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604ea150160 0x5604ea152550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:46.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.849+0000 7ff0853fb740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604ea0a6cc0 con 0x5604ea13c670 2024-09-13T18:31:46.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.850+0000 7ff0853fb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604ea0a4450 con 0x5604ea150160 2024-09-13T18:31:46.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.850+0000 7ff0853fb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604ea09fd40 con 0x5604ea13cf90 2024-09-13T18:31:46.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.850+0000 7ff073fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604ea13cf90 0x5604ea14fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:46.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.850+0000 7ff073fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604ea13cf90 0x5604ea14fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:41410/0 (socket says 172.21.15.114:41410) 2024-09-13T18:31:46.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.850+0000 7ff073fff640 1 -- 172.21.15.114:0/1168385365 learned_addr learned my addr 172.21.15.114:0/1168385365 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:46.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.850+0000 7ff07bfff640 1 --2- 172.21.15.114:0/1168385365 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604ea13c670 0x5604ea13ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:46.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.850+0000 7ff07bfff640 1 -- 172.21.15.114:0/1168385365 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604ea13cf90 msgr2=0x5604ea14fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:46.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.850+0000 7ff07bfff640 1 --2- 172.21.15.114:0/1168385365 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604ea13cf90 0x5604ea14fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:46.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.850+0000 7ff07bfff640 1 -- 172.21.15.114:0/1168385365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604ea150160 msgr2=0x5604ea152550 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:31:46.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.850+0000 7ff07bfff640 1 --2- 172.21.15.114:0/1168385365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604ea150160 0x5604ea152550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:46.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.850+0000 7ff07bfff640 1 -- 172.21.15.114:0/1168385365 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604ea067620 con 0x5604ea13c670 2024-09-13T18:31:46.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.851+0000 7ff07bfff640 1 --2- 172.21.15.114:0/1168385365 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604ea13c670 0x5604ea13ca50 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7ff06c009780 tx=0x7ff06c009c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c8b8e680a38bdc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:46.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.852+0000 7ff080c74640 1 --2- 172.21.15.114:0/2701839025 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff0680a1280 0x7ff0680a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:46.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.853+0000 7ff0727fc640 1 -- 172.21.15.114:0/1168385365 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06c009e90 con 0x5604ea13c670 2024-09-13T18:31:46.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.853+0000 7ff0727fc640 1 -- 172.21.15.114:0/1168385365 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff06c00bd60 con 0x5604ea13c670 2024-09-13T18:31:46.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.853+0000 7ff0727fc640 1 -- 172.21.15.114:0/1168385365 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06c022e80 con 0x5604ea13c670 2024-09-13T18:31:46.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.854+0000 7ff0853fb740 1 -- 172.21.15.114:0/1168385365 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604ea13c670 msgr2=0x5604ea13ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:46.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.854+0000 7ff0853fb740 1 --2- 172.21.15.114:0/1168385365 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604ea13c670 0x5604ea13ca50 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7ff06c009780 tx=0x7ff06c009c40 comp rx=0 tx=0).stop 2024-09-13T18:31:46.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.854+0000 7ff080c74640 1 --2- 172.21.15.114:0/2701839025 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff0680a1280 0x7ff0680a3740 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff074006010 tx=0x7ff07403d040 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:46.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.854+0000 7ff0853fb740 1 -- 172.21.15.114:0/1168385365 shutdown_connections 2024-09-13T18:31:46.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.854+0000 7ff0853fb740 1 --2- 172.21.15.114:0/1168385365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604ea150160 0x5604ea152550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:46.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.854+0000 7ff0853fb740 1 --2- 172.21.15.114:0/1168385365 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604ea13cf90 0x5604ea14fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:46.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.854+0000 7ff0853fb740 1 --2- 172.21.15.114:0/1168385365 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604ea13c670 0x5604ea13ca50 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:46.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.854+0000 7ff0853fb740 1 -- 172.21.15.114:0/1168385365 >> 172.21.15.114:0/1168385365 conn(0x5604ea138d00 msgr2=0x5604ea1390e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:46.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.855+0000 7ff0853fb740 1 -- 172.21.15.114:0/1168385365 shutdown_connections 2024-09-13T18:31:46.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.855+0000 7ff0853fb740 1 -- 172.21.15.114:0/1168385365 wait complete. 2024-09-13T18:31:46.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.855+0000 7ff0853fb740 1 -- start start 2024-09-13T18:31:46.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.855+0000 7ff0853fb740 1 Processor -- start 2024-09-13T18:31:46.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.856+0000 7ff0853fb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604ea13c670 0x5604ea14f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:46.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.856+0000 7ff0853fb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604ea13cf90 0x5604ea14cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:46.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.856+0000 7ff0853fb740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604ea150160 0x5604ea14d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:46.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.856+0000 7ff0853fb740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604ea05e180 con 0x5604ea150160 2024-09-13T18:31:46.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.856+0000 7ff0853fb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604e9d78430 con 0x5604ea13cf90 2024-09-13T18:31:46.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.856+0000 7ff0853fb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604ea0235e0 con 0x5604ea13c670 2024-09-13T18:31:46.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.856+0000 7ff0853fb740 10 client.0 _ll_register_callbacks cb 0x5604ea08faa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:31:46.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.856+0000 7ff07bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604ea13c670 0x5604ea14f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:46.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.856+0000 7ff07bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604ea13c670 0x5604ea14f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:41424/0 (socket says 172.21.15.114:41424) 2024-09-13T18:31:46.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.856+0000 7ff07bfff640 1 -- 172.21.15.114:0/742851767 learned_addr learned my addr 172.21.15.114:0/742851767 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:46.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.856+0000 7ff073fff640 1 --2- 172.21.15.114:0/742851767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604ea13cf90 0x5604ea14cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:46.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.857+0000 7ff07bfff640 1 -- 172.21.15.114:0/742851767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604ea13cf90 msgr2=0x5604ea14cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:46.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.857+0000 7ff07bfff640 1 --2- 172.21.15.114:0/742851767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604ea13cf90 0x5604ea14cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:46.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.857+0000 7ff07bfff640 1 -- 172.21.15.114:0/742851767 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604ea150160 msgr2=0x5604ea14d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:46.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.857+0000 7ff07bfff640 1 --2- 172.21.15.114:0/742851767 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604ea150160 0x5604ea14d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:46.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.857+0000 7ff07bfff640 1 -- 172.21.15.114:0/742851767 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff06c009380 con 0x5604ea13c670 2024-09-13T18:31:46.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.857+0000 7ff07bfff640 1 --2- 172.21.15.114:0/742851767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604ea13c670 0x5604ea14f870 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff06c00bb30 tx=0x7ff06c00bb60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d0bd67bb42f3c5b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:46.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.857+0000 7ff0637fe640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:31:46.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.865+0000 7ff0637fe640 1 -- 172.21.15.114:0/742851767 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06c0228c0 con 0x5604ea13c670 2024-09-13T18:31:46.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.865+0000 7ff0853fb740 1 -- 172.21.15.114:0/742851767 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5604ea0c9b80 con 0x5604ea13c670 2024-09-13T18:31:46.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.865+0000 7ff0637fe640 1 -- 172.21.15.114:0/742851767 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ff06c02b630 con 0x5604ea13c670 2024-09-13T18:31:46.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.866+0000 7ff0637fe640 1 -- 172.21.15.114:0/742851767 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff06c009e90 con 0x5604ea13c670 2024-09-13T18:31:46.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.867+0000 7ff0853fb740 10 client.27529 fetch_fsmap learned FSMap version 34 2024-09-13T18:31:46.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.867+0000 7ff0853fb740 1 -- 172.21.15.114:0/742851767 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5604ea09e3c0 con 0x5604ea13c670 2024-09-13T18:31:46.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.867+0000 7ff0637fe640 1 -- 172.21.15.114:0/742851767 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5604ea0c9b80 con 0x5604ea13c670 2024-09-13T18:31:46.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.867+0000 7ff0637fe640 1 -- 172.21.15.114:0/742851767 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff06c034890 con 0x5604ea13c670 2024-09-13T18:31:46.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.867+0000 7ff0853fb740 10 client.27529 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:31:46.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.867+0000 7ff0853fb740 10 client.27529 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-13T18:31:46.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.867+0000 7ff0853fb740 20 client.27529 populate_metadata read hostname 'smithi114' 2024-09-13T18:31:46.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.867+0000 7ff0853fb740 1 --2- 172.21.15.114:0/742851767 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x5604ea15f9f0 0x5604ea161de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:46.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.867+0000 7ff0853fb740 4 client.27529 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:31:46.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.867+0000 7ff0853fb740 1 -- 172.21.15.114:0/742851767 --> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5604ea0921f0 con 0x5604ea15f9f0 2024-09-13T18:31:46.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.868+0000 7ff073fff640 1 --2- 172.21.15.114:0/742851767 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x5604ea15f9f0 0x5604ea161de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:46.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.869+0000 7ff0637fe640 10 client.27529 ms_handle_connect on v2:172.21.15.114:6840/1956923446 2024-09-13T18:31:46.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.870+0000 7ff073fff640 1 --2- 172.21.15.114:0/742851767 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x5604ea15f9f0 0x5604ea161de0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=99fa2b6b068ab76a server_cookie=f8cc830e92321f08 in_seq=0 out_seq=0 2024-09-13T18:31:46.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.877+0000 7ff0637fe640 1 -- 172.21.15.114:0/742851767 <== mds.4 v2:172.21.15.114:6840/1956923446 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5604ea0921f0 con 0x5604ea15f9f0 2024-09-13T18:31:46.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.877+0000 7ff0637fe640 10 client.27529 handle_command_reply: tid=0 2024-09-13T18:31:46.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.877+0000 7ff0853fb740 1 client.27529 shutdown 2024-09-13T18:31:46.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.877+0000 7ff0853fb740 1 -- 172.21.15.114:0/742851767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604ea13c670 msgr2=0x5604ea14f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:46.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.877+0000 7ff0853fb740 1 --2- 172.21.15.114:0/742851767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604ea13c670 0x5604ea14f870 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff06c00bb30 tx=0x7ff06c00bb60 comp rx=0 tx=0).stop 2024-09-13T18:31:46.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.878+0000 7ff0853fb740 1 -- 172.21.15.114:0/742851767 shutdown_connections 2024-09-13T18:31:46.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.878+0000 7ff0853fb740 1 --2- 172.21.15.114:0/742851767 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x5604ea15f9f0 0x5604ea161de0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:46.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.878+0000 7ff0853fb740 1 --2- 172.21.15.114:0/742851767 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604ea150160 0x5604ea14d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:46.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.878+0000 7ff0853fb740 1 --2- 172.21.15.114:0/742851767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604ea13cf90 0x5604ea14cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:46.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.878+0000 7ff0853fb740 1 --2- 172.21.15.114:0/742851767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604ea13c670 0x5604ea14f870 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:46.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.878+0000 7ff0853fb740 1 -- 172.21.15.114:0/742851767 >> 172.21.15.114:0/742851767 conn(0x5604ea138d00 msgr2=0x5604ea137eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:46.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.878+0000 7ff0637fe640 0 client.27529 ms_handle_reset on v2:172.21.15.114:6840/1956923446 2024-09-13T18:31:46.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.878+0000 7ff0853fb740 1 -- 172.21.15.114:0/742851767 shutdown_connections 2024-09-13T18:31:46.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.878+0000 7ff0853fb740 1 -- 172.21.15.114:0/742851767 wait complete. 2024-09-13T18:31:46.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.878+0000 7ff0853fb740 20 client.27529 trim_cache size 0 max 16384 2024-09-13T18:31:46.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.906+0000 7ff0853fb740 1 Processor -- start 2024-09-13T18:31:46.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.911+0000 7ff0853fb740 1 -- start start 2024-09-13T18:31:46.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.911+0000 7ff0853fb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604ea13c670 0x5604ea1386a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:46.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.911+0000 7ff0853fb740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604ea13cf90 0x5604ea156e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:46.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.912+0000 7ff0853fb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604ea150160 0x5604ea157380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:46.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.912+0000 7ff0853fb740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604ea0921f0 con 0x5604ea13cf90 2024-09-13T18:31:46.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.912+0000 7ff0853fb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604ea0c9b80 con 0x5604ea150160 2024-09-13T18:31:46.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.912+0000 7ff0853fb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604ea14db90 con 0x5604ea13c670 2024-09-13T18:31:46.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.913+0000 7ff080c74640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604ea150160 0x5604ea157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:46.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.913+0000 7ff080c74640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604ea150160 0x5604ea157380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:46074/0 (socket says 172.21.15.114:46074) 2024-09-13T18:31:46.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.913+0000 7ff080c74640 1 -- 172.21.15.114:0/4276884264 learned_addr learned my addr 172.21.15.114:0/4276884264 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:46.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.914+0000 7ff080c74640 1 -- 172.21.15.114:0/4276884264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604ea13c670 msgr2=0x5604ea1386a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:31:46.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.914+0000 7ff080c74640 1 --2- 172.21.15.114:0/4276884264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604ea13c670 0x5604ea1386a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:46.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.914+0000 7ff080c74640 1 -- 172.21.15.114:0/4276884264 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604ea13cf90 msgr2=0x5604ea156e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:46.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.914+0000 7ff080c74640 1 --2- 172.21.15.114:0/4276884264 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604ea13cf90 0x5604ea156e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:46.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.914+0000 7ff080c74640 1 -- 172.21.15.114:0/4276884264 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604ea040770 con 0x5604ea150160 2024-09-13T18:31:46.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.915+0000 7ff080c74640 1 --2- 172.21.15.114:0/4276884264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604ea150160 0x5604ea157380 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff07402f690 tx=0x7ff07402a950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1aad05073f96a7af server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:46.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.919+0000 7ff072ffd640 1 -- 172.21.15.114:0/4276884264 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff074040040 con 0x5604ea150160 2024-09-13T18:31:46.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.919+0000 7ff072ffd640 1 -- 172.21.15.114:0/4276884264 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff074003e00 con 0x5604ea150160 2024-09-13T18:31:46.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.919+0000 7ff072ffd640 1 -- 172.21.15.114:0/4276884264 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff074004040 con 0x5604ea150160 2024-09-13T18:31:46.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.923+0000 7ff0853fb740 1 -- 172.21.15.114:0/4276884264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604ea150160 msgr2=0x5604ea157380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:46.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.923+0000 7ff0853fb740 1 --2- 172.21.15.114:0/4276884264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604ea150160 0x5604ea157380 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff07402f690 tx=0x7ff07402a950 comp rx=0 tx=0).stop 2024-09-13T18:31:46.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.928+0000 7ff0853fb740 1 -- 172.21.15.114:0/4276884264 shutdown_connections 2024-09-13T18:31:46.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.928+0000 7ff0853fb740 1 --2- 172.21.15.114:0/4276884264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604ea150160 0x5604ea157380 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:46.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.928+0000 7ff0853fb740 1 --2- 172.21.15.114:0/4276884264 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604ea13cf90 0x5604ea156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:46.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.928+0000 7ff0853fb740 1 --2- 172.21.15.114:0/4276884264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604ea13c670 0x5604ea1386a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:46.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.928+0000 7ff0853fb740 1 -- 172.21.15.114:0/4276884264 >> 172.21.15.114:0/4276884264 conn(0x5604ea138d00 msgr2=0x5604ea14b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:46.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.939+0000 7ff0853fb740 1 -- 172.21.15.114:0/4276884264 shutdown_connections 2024-09-13T18:31:46.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.939+0000 7ff0853fb740 1 -- 172.21.15.114:0/4276884264 wait complete. 2024-09-13T18:31:46.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.940+0000 7ff0853fb740 1 -- start start 2024-09-13T18:31:46.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.940+0000 7ff0853fb740 1 Processor -- start 2024-09-13T18:31:46.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.946+0000 7ff0853fb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604ea13c670 0x5604ea157090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:46.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.946+0000 7ff0853fb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604ea13cf90 0x5604ea1533c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:46.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.946+0000 7ff0853fb740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604ea150160 0x5604ea153900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:46.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.947+0000 7ff0853fb740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604ea153ec0 con 0x5604ea150160 2024-09-13T18:31:46.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.947+0000 7ff07bfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604ea13c670 0x5604ea157090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:46.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.947+0000 7ff07bfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604ea13c670 0x5604ea157090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:46086/0 (socket says 172.21.15.114:46086) 2024-09-13T18:31:46.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.947+0000 7ff07bfff640 1 -- 172.21.15.114:0/2675216960 learned_addr learned my addr 172.21.15.114:0/2675216960 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:46.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.947+0000 7ff0853fb740 1 -- 172.21.15.114:0/2675216960 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604ea154030 con 0x5604ea13c670 2024-09-13T18:31:46.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.947+0000 7ff0853fb740 1 -- 172.21.15.114:0/2675216960 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604ea1541f0 con 0x5604ea13cf90 2024-09-13T18:31:46.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.947+0000 7ff0853fb740 10 client.0 _ll_register_callbacks cb 0x7ff06c006f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:31:46.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.947+0000 7ff073fff640 1 --2- 172.21.15.114:0/2675216960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604ea13cf90 0x5604ea1533c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:46.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.948+0000 7ff080c74640 1 --2- 172.21.15.114:0/2675216960 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604ea150160 0x5604ea153900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:46.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.950+0000 7ff073fff640 1 -- 172.21.15.114:0/2675216960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604ea13c670 msgr2=0x5604ea157090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:46.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.950+0000 7ff073fff640 1 --2- 172.21.15.114:0/2675216960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604ea13c670 0x5604ea157090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:46.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.950+0000 7ff073fff640 1 -- 172.21.15.114:0/2675216960 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604ea150160 msgr2=0x5604ea153900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:46.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.950+0000 7ff073fff640 1 --2- 172.21.15.114:0/2675216960 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604ea150160 0x5604ea153900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:46.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.950+0000 7ff073fff640 1 -- 172.21.15.114:0/2675216960 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff07c10f630 con 0x5604ea13cf90 2024-09-13T18:31:46.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.950+0000 7ff073fff640 1 --2- 172.21.15.114:0/2675216960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604ea13cf90 0x5604ea1533c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff0640a2260 tx=0x7ff064099e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10bbffbc8a35b4ce server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:46.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.951+0000 7ff0637fe640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:31:46.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.951+0000 7ff0637fe640 1 -- 172.21.15.114:0/2675216960 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0640a2790 con 0x5604ea13cf90 2024-09-13T18:31:46.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.951+0000 7ff0853fb740 1 -- 172.21.15.114:0/2675216960 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff06c00bd60 con 0x5604ea13cf90 2024-09-13T18:31:46.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.951+0000 7ff0637fe640 1 -- 172.21.15.114:0/2675216960 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ff0640042b0 con 0x5604ea13cf90 2024-09-13T18:31:46.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.951+0000 7ff0637fe640 1 -- 172.21.15.114:0/2675216960 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff064098450 con 0x5604ea13cf90 2024-09-13T18:31:46.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.951+0000 7ff0853fb740 10 client.27532 fetch_fsmap learned FSMap version 34 2024-09-13T18:31:46.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.951+0000 7ff0853fb740 1 -- 172.21.15.114:0/2675216960 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5604ea065e80 con 0x5604ea13cf90 2024-09-13T18:31:46.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.952+0000 7ff0637fe640 1 -- 172.21.15.114:0/2675216960 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff0640986b0 con 0x5604ea13cf90 2024-09-13T18:31:46.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.952+0000 7ff0637fe640 1 -- 172.21.15.114:0/2675216960 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff0640cabd0 con 0x5604ea13cf90 2024-09-13T18:31:46.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.952+0000 7ff0853fb740 10 client.27532 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:31:46.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.952+0000 7ff0853fb740 10 client.27532 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-13T18:31:46.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.952+0000 7ff0853fb740 20 client.27532 populate_metadata read hostname 'smithi114' 2024-09-13T18:31:46.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.952+0000 7ff0853fb740 1 --2- 172.21.15.114:0/2675216960 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x5604ea1b7470 0x5604ea1b9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:46.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.952+0000 7ff0853fb740 4 client.27532 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:31:46.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.952+0000 7ff0853fb740 1 -- 172.21.15.114:0/2675216960 --> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7ff06c00bd60 con 0x5604ea1b7470 2024-09-13T18:31:46.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.962+0000 7ff07bfff640 1 --2- 172.21.15.114:0/2675216960 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x5604ea1b7470 0x5604ea1b9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:46.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.965+0000 7ff07bfff640 1 --2- 172.21.15.114:0/2675216960 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x5604ea1b7470 0x5604ea1b9890 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=be3f950387a4ef7f server_cookie=3cb6f29eab0a408a in_seq=0 out_seq=0 2024-09-13T18:31:46.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.965+0000 7ff0637fe640 10 client.27532 ms_handle_connect on v2:172.21.15.114:6840/1956923446 2024-09-13T18:31:46.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.968+0000 7ff0637fe640 1 -- 172.21.15.114:0/2675216960 <== mds.4 v2:172.21.15.114:6840/1956923446 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7ff06c00bd60 con 0x5604ea1b7470 2024-09-13T18:31:46.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.968+0000 7ff0637fe640 10 client.27532 handle_command_reply: tid=0 2024-09-13T18:31:46.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.970+0000 7ff0853fb740 1 client.27532 shutdown 2024-09-13T18:31:46.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.975+0000 7ff0853fb740 1 -- 172.21.15.114:0/2675216960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604ea13cf90 msgr2=0x5604ea1533c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:46.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.975+0000 7ff0853fb740 1 --2- 172.21.15.114:0/2675216960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604ea13cf90 0x5604ea1533c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff0640a2260 tx=0x7ff064099e50 comp rx=0 tx=0).stop 2024-09-13T18:31:46.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.976+0000 7ff0853fb740 1 -- 172.21.15.114:0/2675216960 shutdown_connections 2024-09-13T18:31:46.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.976+0000 7ff0853fb740 1 --2- 172.21.15.114:0/2675216960 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x5604ea1b7470 0x5604ea1b9890 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:46.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.976+0000 7ff0853fb740 1 --2- 172.21.15.114:0/2675216960 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604ea150160 0x5604ea153900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:46.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.976+0000 7ff0853fb740 1 --2- 172.21.15.114:0/2675216960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604ea13cf90 0x5604ea1533c0 secure :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff0640a2260 tx=0x7ff064099e50 comp rx=0 tx=0).stop 2024-09-13T18:31:46.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.976+0000 7ff0853fb740 1 --2- 172.21.15.114:0/2675216960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604ea13c670 0x5604ea157090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:46.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.976+0000 7ff0853fb740 1 -- 172.21.15.114:0/2675216960 >> 172.21.15.114:0/2675216960 conn(0x5604ea138d00 msgr2=0x5604ea14b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:46.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.976+0000 7ff0637fe640 0 client.27532 ms_handle_reset on v2:172.21.15.114:6840/1956923446 2024-09-13T18:31:46.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.980+0000 7ff0853fb740 1 -- 172.21.15.114:0/2675216960 shutdown_connections 2024-09-13T18:31:46.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.982+0000 7ff0853fb740 1 -- 172.21.15.114:0/2675216960 wait complete. 2024-09-13T18:31:46.985 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:31:46.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.983+0000 7ff0853fb740 20 client.27532 trim_cache size 0 max 16384 2024-09-13T18:31:46.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.988+0000 7ff0826d7640 1 -- 172.21.15.114:0/2701839025 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff0680a1280 msgr2=0x7ff0680a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:46.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.988+0000 7ff0826d7640 1 --2- 172.21.15.114:0/2701839025 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff0680a1280 0x7ff0680a3740 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff074006010 tx=0x7ff07403d040 comp rx=0 tx=0).stop 2024-09-13T18:31:46.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.988+0000 7ff0826d7640 1 -- 172.21.15.114:0/2701839025 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff07c0731c0 msgr2=0x7ff07c198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:46.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.988+0000 7ff0826d7640 1 --2- 172.21.15.114:0/2701839025 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff07c0731c0 0x7ff07c198930 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7ff06400ea60 tx=0x7ff06400ef20 comp rx=0 tx=0).stop 2024-09-13T18:31:46.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.990+0000 7ff0826d7640 1 -- 172.21.15.114:0/2701839025 shutdown_connections 2024-09-13T18:31:46.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.990+0000 7ff0826d7640 1 --2- 172.21.15.114:0/2701839025 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff0680a1280 0x7ff0680a3740 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:46.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.990+0000 7ff0826d7640 1 --2- 172.21.15.114:0/2701839025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff07c10c580 0x7ff07c19cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:46.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.990+0000 7ff0826d7640 1 --2- 172.21.15.114:0/2701839025 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff07c0731c0 0x7ff07c198930 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:46.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.990+0000 7ff0826d7640 1 --2- 172.21.15.114:0/2701839025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff07c0728a0 0x7ff07c1983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:46.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.990+0000 7ff0826d7640 1 -- 172.21.15.114:0/2701839025 >> 172.21.15.114:0/2701839025 conn(0x7ff07c0fb9c0 msgr2=0x7ff07c0fd260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:46.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.992+0000 7ff0826d7640 1 -- 172.21.15.114:0/2701839025 shutdown_connections 2024-09-13T18:31:46.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:46.992+0000 7ff0826d7640 1 -- 172.21.15.114:0/2701839025 wait complete. 2024-09-13T18:31:47.020 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:46 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/546768164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:31:47.022 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-09-13T18:31:47.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:46 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/546768164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:31:47.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:46 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/546768164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:31:47.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.282+0000 7fa870bf8640 1 -- 172.21.15.114:0/3195803586 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa86c106f00 msgr2=0x7fa86c10aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:47.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.282+0000 7fa870bf8640 1 --2- 172.21.15.114:0/3195803586 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa86c106f00 0x7fa86c10aef0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fa85c01a940 tx=0x7fa85c03b490 comp rx=0 tx=0).stop 2024-09-13T18:31:47.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.284+0000 7fa870bf8640 1 -- 172.21.15.114:0/3195803586 shutdown_connections 2024-09-13T18:31:47.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.284+0000 7fa870bf8640 1 --2- 172.21.15.114:0/3195803586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa86c10b650 0x7fa86c1131a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.284+0000 7fa870bf8640 1 --2- 172.21.15.114:0/3195803586 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa86c106f00 0x7fa86c10aef0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.284+0000 7fa870bf8640 1 --2- 172.21.15.114:0/3195803586 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa86c106550 0x7fa86c106930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.284+0000 7fa870bf8640 1 -- 172.21.15.114:0/3195803586 >> 172.21.15.114:0/3195803586 conn(0x7fa86c075c80 msgr2=0x7fa86c0780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:47.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.285+0000 7fa870bf8640 1 -- 172.21.15.114:0/3195803586 shutdown_connections 2024-09-13T18:31:47.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.289+0000 7fa870bf8640 1 -- 172.21.15.114:0/3195803586 wait complete. 2024-09-13T18:31:47.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.289+0000 7fa870bf8640 1 Processor -- start 2024-09-13T18:31:47.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.292+0000 7fa870bf8640 1 -- start start 2024-09-13T18:31:47.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.292+0000 7fa870bf8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa86c106550 0x7fa86c10ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:47.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.292+0000 7fa870bf8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa86c10b650 0x7fa86c10f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:47.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.292+0000 7fa870bf8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa86c112670 0x7fa86c112b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:47.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.292+0000 7fa870bf8640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c113060 con 0x7fa86c10b650 2024-09-13T18:31:47.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.292+0000 7fa870bf8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c1131a0 con 0x7fa86c112670 2024-09-13T18:31:47.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.292+0000 7fa870bf8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c113360 con 0x7fa86c106550 2024-09-13T18:31:47.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.293+0000 7fa86bfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa86c112670 0x7fa86c112b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:47.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.293+0000 7fa86bfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa86c112670 0x7fa86c112b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:46102/0 (socket says 172.21.15.114:46102) 2024-09-13T18:31:47.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.293+0000 7fa86bfff640 1 -- 172.21.15.114:0/2682531979 learned_addr learned my addr 172.21.15.114:0/2682531979 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:47.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.294+0000 7fa86b7fe640 1 --2- 172.21.15.114:0/2682531979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa86c106550 0x7fa86c10ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:47.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.294+0000 7fa86affd640 1 --2- 172.21.15.114:0/2682531979 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa86c10b650 0x7fa86c10f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:47.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.295+0000 7fa86affd640 1 -- 172.21.15.114:0/2682531979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa86c106550 msgr2=0x7fa86c10ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:47.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.295+0000 7fa86affd640 1 --2- 172.21.15.114:0/2682531979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa86c106550 0x7fa86c10ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.295+0000 7fa86affd640 1 -- 172.21.15.114:0/2682531979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa86c112670 msgr2=0x7fa86c112b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:47.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.295+0000 7fa86affd640 1 --2- 172.21.15.114:0/2682531979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa86c112670 0x7fa86c112b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.295+0000 7fa86affd640 1 -- 172.21.15.114:0/2682531979 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa85c01a570 con 0x7fa86c10b650 2024-09-13T18:31:47.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.296+0000 7fa86affd640 1 --2- 172.21.15.114:0/2682531979 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa86c10b650 0x7fa86c10f3a0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fa85c03ba80 tx=0x7fa85c004960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:47.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.298+0000 7fa868ff9640 1 -- 172.21.15.114:0/2682531979 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85c004e60 con 0x7fa86c10b650 2024-09-13T18:31:47.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.298+0000 7fa868ff9640 1 -- 172.21.15.114:0/2682531979 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa85c005450 con 0x7fa86c10b650 2024-09-13T18:31:47.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.298+0000 7fa868ff9640 1 -- 172.21.15.114:0/2682531979 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85c040510 con 0x7fa86c10b650 2024-09-13T18:31:47.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.299+0000 7fa870bf8640 1 -- 172.21.15.114:0/2682531979 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa86c078d00 con 0x7fa86c10b650 2024-09-13T18:31:47.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.299+0000 7fa870bf8640 1 -- 172.21.15.114:0/2682531979 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa86c079190 con 0x7fa86c10b650 2024-09-13T18:31:47.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.302+0000 7fa874c59740 1 Processor -- start 2024-09-13T18:31:47.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.304+0000 7fa874c59740 1 -- start start 2024-09-13T18:31:47.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.304+0000 7fa874c59740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b87f47d970 0x55b87f47dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:47.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.304+0000 7fa874c59740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b87f47e290 0x55b87f4885f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:47.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.304+0000 7fa874c59740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b87f488b30 0x55b87f48af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:47.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.304+0000 7fa874c59740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b87f3e7cc0 con 0x55b87f488b30 2024-09-13T18:31:47.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.304+0000 7fa874c59740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b87f3e5450 con 0x55b87f47e290 2024-09-13T18:31:47.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.304+0000 7fa874c59740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b87f3e0d40 con 0x55b87f47d970 2024-09-13T18:31:47.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.304+0000 7fa86b7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b87f47e290 0x55b87f4885f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:47.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.304+0000 7fa86b7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b87f47e290 0x55b87f4885f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:46106/0 (socket says 172.21.15.114:46106) 2024-09-13T18:31:47.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.304+0000 7fa86b7fe640 1 -- 172.21.15.114:0/1889166912 learned_addr learned my addr 172.21.15.114:0/1889166912 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:47.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.304+0000 7fa86bfff640 1 --2- 172.21.15.114:0/1889166912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b87f47d970 0x55b87f47dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:47.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.305+0000 7fa86bfff640 1 -- 172.21.15.114:0/1889166912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b87f47e290 msgr2=0x55b87f4885f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:47.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.305+0000 7fa86bfff640 1 --2- 172.21.15.114:0/1889166912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b87f47e290 0x55b87f4885f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.305+0000 7fa86bfff640 1 -- 172.21.15.114:0/1889166912 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b87f488b30 msgr2=0x55b87f48af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:47.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.305+0000 7fa86bfff640 1 --2- 172.21.15.114:0/1889166912 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b87f488b30 0x55b87f48af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.305+0000 7fa86bfff640 1 -- 172.21.15.114:0/1889166912 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b87f3a8620 con 0x55b87f47d970 2024-09-13T18:31:47.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.305+0000 7fa868ff9640 1 -- 172.21.15.114:0/2682531979 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fa85c003680 con 0x7fa86c10b650 2024-09-13T18:31:47.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.306+0000 7fa86bfff640 1 --2- 172.21.15.114:0/1889166912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b87f47d970 0x55b87f47dd50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa864007d20 tx=0x7fa86400e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37a02ed08c42f620 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:47.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.306+0000 7fa851ffb640 1 -- 172.21.15.114:0/1889166912 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa864003b70 con 0x55b87f47d970 2024-09-13T18:31:47.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.307+0000 7fa851ffb640 1 -- 172.21.15.114:0/1889166912 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa864007e10 con 0x55b87f47d970 2024-09-13T18:31:47.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.308+0000 7fa874c59740 1 -- 172.21.15.114:0/1889166912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b87f47d970 msgr2=0x55b87f47dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:47.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.308+0000 7fa874c59740 1 --2- 172.21.15.114:0/1889166912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b87f47d970 0x55b87f47dd50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa864007d20 tx=0x7fa86400e720 comp rx=0 tx=0).stop 2024-09-13T18:31:47.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.308+0000 7fa868ff9640 1 --2- 172.21.15.114:0/2682531979 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa8540a1410 0x7fa8540a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:47.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.309+0000 7fa86bfff640 1 --2- 172.21.15.114:0/2682531979 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa8540a1410 0x7fa8540a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:47.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.309+0000 7fa874c59740 1 -- 172.21.15.114:0/1889166912 shutdown_connections 2024-09-13T18:31:47.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.309+0000 7fa874c59740 1 --2- 172.21.15.114:0/1889166912 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b87f488b30 0x55b87f48af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.309+0000 7fa874c59740 1 --2- 172.21.15.114:0/1889166912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b87f47e290 0x55b87f4885f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.309+0000 7fa874c59740 1 --2- 172.21.15.114:0/1889166912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b87f47d970 0x55b87f47dd50 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.309+0000 7fa874c59740 1 -- 172.21.15.114:0/1889166912 >> 172.21.15.114:0/1889166912 conn(0x55b87f479ee0 msgr2=0x55b87f47a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:47.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.309+0000 7fa868ff9640 1 -- 172.21.15.114:0/2682531979 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fa85c0fea90 con 0x7fa86c10b650 2024-09-13T18:31:47.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.310+0000 7fa86bfff640 1 --2- 172.21.15.114:0/2682531979 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa8540a1410 0x7fa8540a38d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa864000f80 tx=0x7fa8640048f0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:47.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.310+0000 7fa874c59740 1 -- 172.21.15.114:0/1889166912 shutdown_connections 2024-09-13T18:31:47.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.310+0000 7fa874c59740 1 -- 172.21.15.114:0/1889166912 wait complete. 2024-09-13T18:31:47.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.310+0000 7fa874c59740 1 -- start start 2024-09-13T18:31:47.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.311+0000 7fa874c59740 1 Processor -- start 2024-09-13T18:31:47.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.311+0000 7fa874c59740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b87f47d970 0x55b87f488080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:47.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.311+0000 7fa874c59740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b87f47e290 0x55b87f4885c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:47.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.311+0000 7fa86b7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b87f47d970 0x55b87f488080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:47.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.312+0000 7fa86b7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b87f47d970 0x55b87f488080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:41454/0 (socket says 172.21.15.114:41454) 2024-09-13T18:31:47.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.312+0000 7fa86b7fe640 1 -- 172.21.15.114:0/3673797644 learned_addr learned my addr 172.21.15.114:0/3673797644 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:47.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.312+0000 7fa874c59740 1 --2- 172.21.15.114:0/3673797644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b87f488b30 0x55b87f485610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:47.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.312+0000 7fa874c59740 1 -- 172.21.15.114:0/3673797644 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b87f39f180 con 0x55b87f47e290 2024-09-13T18:31:47.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.312+0000 7fa874c59740 1 -- 172.21.15.114:0/3673797644 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b87f0b9430 con 0x55b87f488b30 2024-09-13T18:31:47.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.312+0000 7fa874c59740 1 -- 172.21.15.114:0/3673797644 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b87f3645e0 con 0x55b87f47d970 2024-09-13T18:31:47.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.312+0000 7fa86bfff640 1 --2- 172.21.15.114:0/3673797644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b87f488b30 0x55b87f485610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:47.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.312+0000 7fa874c59740 10 client.0 _ll_register_callbacks cb 0x55b87f3d0aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:31:47.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.312+0000 7fa86affd640 1 --2- 172.21.15.114:0/3673797644 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b87f47e290 0x55b87f4885c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:47.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.313+0000 7fa86bfff640 1 -- 172.21.15.114:0/3673797644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b87f47d970 msgr2=0x55b87f488080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:47.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.313+0000 7fa86bfff640 1 --2- 172.21.15.114:0/3673797644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b87f47d970 0x55b87f488080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.313+0000 7fa86bfff640 1 -- 172.21.15.114:0/3673797644 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b87f47e290 msgr2=0x55b87f4885c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:47.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.313+0000 7fa86bfff640 1 --2- 172.21.15.114:0/3673797644 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b87f47e290 0x55b87f4885c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.313+0000 7fa86bfff640 1 -- 172.21.15.114:0/3673797644 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa864007950 con 0x55b87f488b30 2024-09-13T18:31:47.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.313+0000 7fa86bfff640 1 --2- 172.21.15.114:0/3673797644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b87f488b30 0x55b87f485610 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa86400f010 tx=0x7fa8640044b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=babe79b7f35b23ee server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:47.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.313+0000 7fa84affd640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:31:47.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.313+0000 7fa84affd640 1 -- 172.21.15.114:0/3673797644 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86401c030 con 0x55b87f488b30 2024-09-13T18:31:47.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.313+0000 7fa874c59740 1 -- 172.21.15.114:0/3673797644 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b87f40ab80 con 0x55b87f488b30 2024-09-13T18:31:47.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.313+0000 7fa84affd640 1 -- 172.21.15.114:0/3673797644 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fa864010e40 con 0x55b87f488b30 2024-09-13T18:31:47.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.314+0000 7fa84affd640 1 -- 172.21.15.114:0/3673797644 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa864004550 con 0x55b87f488b30 2024-09-13T18:31:47.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.314+0000 7fa874c59740 10 client.27581 fetch_fsmap learned FSMap version 34 2024-09-13T18:31:47.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.314+0000 7fa874c59740 1 -- 172.21.15.114:0/3673797644 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55b87f3df3c0 con 0x55b87f488b30 2024-09-13T18:31:47.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.322+0000 7fa84affd640 1 -- 172.21.15.114:0/3673797644 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa86401d040 con 0x55b87f488b30 2024-09-13T18:31:47.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.323+0000 7fa84affd640 1 -- 172.21.15.114:0/3673797644 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa86402f400 con 0x55b87f488b30 2024-09-13T18:31:47.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.323+0000 7fa874c59740 10 client.27581 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:31:47.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.323+0000 7fa874c59740 10 client.27581 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-13T18:31:47.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.323+0000 7fa874c59740 20 client.27581 populate_metadata read hostname 'smithi114' 2024-09-13T18:31:47.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.323+0000 7fa874c59740 1 --2- 172.21.15.114:0/3673797644 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55b87f4984b0 0x55b87f49a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:47.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.323+0000 7fa874c59740 4 client.27581 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:31:47.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.323+0000 7fa874c59740 1 -- 172.21.15.114:0/3673797644 --> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b87f40ab80 con 0x55b87f4984b0 2024-09-13T18:31:47.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.323+0000 7fa86b7fe640 1 --2- 172.21.15.114:0/3673797644 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55b87f4984b0 0x55b87f49a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:47.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.329+0000 7fa86b7fe640 1 --2- 172.21.15.114:0/3673797644 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55b87f4984b0 0x55b87f49a8a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8fab803c4a3465e7 server_cookie=ca456b037dd7197b in_seq=0 out_seq=0 2024-09-13T18:31:47.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.329+0000 7fa84affd640 10 client.27581 ms_handle_connect on v2:172.21.15.134:6834/1150003275 2024-09-13T18:31:47.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.333+0000 7fa84affd640 1 -- 172.21.15.114:0/3673797644 <== mds.3 v2:172.21.15.134:6834/1150003275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55b87f40ab80 con 0x55b87f4984b0 2024-09-13T18:31:47.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.333+0000 7fa84affd640 10 client.27581 handle_command_reply: tid=0 2024-09-13T18:31:47.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.333+0000 7fa874c59740 1 client.27581 shutdown 2024-09-13T18:31:47.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.338+0000 7fa874c59740 1 -- 172.21.15.114:0/3673797644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b87f488b30 msgr2=0x55b87f485610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:47.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.338+0000 7fa874c59740 1 --2- 172.21.15.114:0/3673797644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b87f488b30 0x55b87f485610 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa86400f010 tx=0x7fa8640044b0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.343+0000 7fa874c59740 1 -- 172.21.15.114:0/3673797644 shutdown_connections 2024-09-13T18:31:47.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.343+0000 7fa874c59740 1 --2- 172.21.15.114:0/3673797644 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55b87f4984b0 0x55b87f49a8a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.343+0000 7fa874c59740 1 --2- 172.21.15.114:0/3673797644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b87f488b30 0x55b87f485610 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.343+0000 7fa874c59740 1 --2- 172.21.15.114:0/3673797644 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b87f47e290 0x55b87f4885c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.343+0000 7fa874c59740 1 --2- 172.21.15.114:0/3673797644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b87f47d970 0x55b87f488080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.343+0000 7fa874c59740 1 -- 172.21.15.114:0/3673797644 >> 172.21.15.114:0/3673797644 conn(0x55b87f479ee0 msgr2=0x55b87f479230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:47.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.344+0000 7fa84affd640 0 client.27581 ms_handle_reset on v2:172.21.15.134:6834/1150003275 2024-09-13T18:31:47.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.347+0000 7fa874c59740 1 -- 172.21.15.114:0/3673797644 shutdown_connections 2024-09-13T18:31:47.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.348+0000 7fa874c59740 1 -- 172.21.15.114:0/3673797644 wait complete. 2024-09-13T18:31:47.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.355+0000 7fa874c59740 20 client.27581 trim_cache size 0 max 16384 2024-09-13T18:31:47.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.383+0000 7fa874c59740 1 Processor -- start 2024-09-13T18:31:47.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.383+0000 7fa874c59740 1 -- start start 2024-09-13T18:31:47.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.383+0000 7fa874c59740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b87f47d970 0x55b87f49a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:47.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.383+0000 7fa874c59740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b87f47e290 0x55b87f48f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:47.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.383+0000 7fa874c59740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b87f488b30 0x55b87f48fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:47.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.383+0000 7fa874c59740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b87f47e8b0 con 0x55b87f47d970 2024-09-13T18:31:47.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.383+0000 7fa874c59740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b87f47ae40 con 0x55b87f47e290 2024-09-13T18:31:47.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.383+0000 7fa874c59740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b87f47ac40 con 0x55b87f488b30 2024-09-13T18:31:47.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.385+0000 7fa86bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b87f488b30 0x55b87f48fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:47.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.385+0000 7fa86bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b87f488b30 0x55b87f48fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:41456/0 (socket says 172.21.15.114:41456) 2024-09-13T18:31:47.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.385+0000 7fa86bfff640 1 -- 172.21.15.114:0/969702030 learned_addr learned my addr 172.21.15.114:0/969702030 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:47.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.386+0000 7fa86bfff640 1 -- 172.21.15.114:0/969702030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b87f47e290 msgr2=0x55b87f48f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:47.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.386+0000 7fa86bfff640 1 --2- 172.21.15.114:0/969702030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b87f47e290 0x55b87f48f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.386+0000 7fa86bfff640 1 -- 172.21.15.114:0/969702030 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b87f47d970 msgr2=0x55b87f49a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:47.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.386+0000 7fa86bfff640 1 --2- 172.21.15.114:0/969702030 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b87f47d970 0x55b87f49a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.386+0000 7fa86bfff640 1 -- 172.21.15.114:0/969702030 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b87f381770 con 0x55b87f488b30 2024-09-13T18:31:47.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.386+0000 7fa86bfff640 1 --2- 172.21.15.114:0/969702030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b87f488b30 0x55b87f48fe30 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa864005020 tx=0x7fa864005f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abc4dd99e45099e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:47.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.387+0000 7fa8527fc640 1 -- 172.21.15.114:0/969702030 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86401d040 con 0x55b87f488b30 2024-09-13T18:31:47.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.387+0000 7fa8527fc640 1 -- 172.21.15.114:0/969702030 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa864028040 con 0x55b87f488b30 2024-09-13T18:31:47.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.387+0000 7fa8527fc640 1 -- 172.21.15.114:0/969702030 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86402fcc0 con 0x55b87f488b30 2024-09-13T18:31:47.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.390+0000 7fa874c59740 1 -- 172.21.15.114:0/969702030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b87f488b30 msgr2=0x55b87f48fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:47.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.390+0000 7fa874c59740 1 --2- 172.21.15.114:0/969702030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b87f488b30 0x55b87f48fe30 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa864005020 tx=0x7fa864005f40 comp rx=0 tx=0).stop 2024-09-13T18:31:47.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.392+0000 7fa874c59740 1 -- 172.21.15.114:0/969702030 shutdown_connections 2024-09-13T18:31:47.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.392+0000 7fa874c59740 1 --2- 172.21.15.114:0/969702030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b87f488b30 0x55b87f48fe30 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.392+0000 7fa874c59740 1 --2- 172.21.15.114:0/969702030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b87f47e290 0x55b87f48f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.392+0000 7fa874c59740 1 --2- 172.21.15.114:0/969702030 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b87f47d970 0x55b87f49a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.392+0000 7fa874c59740 1 -- 172.21.15.114:0/969702030 >> 172.21.15.114:0/969702030 conn(0x55b87f479ee0 msgr2=0x55b87f49f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:47.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.396+0000 7fa874c59740 1 -- 172.21.15.114:0/969702030 shutdown_connections 2024-09-13T18:31:47.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.398+0000 7fa874c59740 1 -- 172.21.15.114:0/969702030 wait complete. 2024-09-13T18:31:47.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.399+0000 7fa874c59740 1 -- start start 2024-09-13T18:31:47.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.399+0000 7fa874c59740 1 Processor -- start 2024-09-13T18:31:47.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.404+0000 7fa874c59740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b87f47d970 0x55b87f4e2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:47.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.404+0000 7fa874c59740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b87f47e290 0x55b87f4e3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:47.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.404+0000 7fa874c59740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b87f488b30 0x55b87f4ed970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:47.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.404+0000 7fa874c59740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b87f4edeb0 con 0x55b87f47e290 2024-09-13T18:31:47.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.404+0000 7fa874c59740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b87f4edff0 con 0x55b87f47d970 2024-09-13T18:31:47.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.404+0000 7fa874c59740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b87f4ee1b0 con 0x55b87f488b30 2024-09-13T18:31:47.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.404+0000 7fa874c59740 10 client.0 _ll_register_callbacks cb 0x7fa86c067eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:31:47.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.404+0000 7fa86b7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b87f47d970 0x55b87f4e2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:47.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.404+0000 7fa86b7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b87f47d970 0x55b87f4e2ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:46150/0 (socket says 172.21.15.114:46150) 2024-09-13T18:31:47.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.405+0000 7fa86b7fe640 1 -- 172.21.15.114:0/4270182312 learned_addr learned my addr 172.21.15.114:0/4270182312 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:47.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.405+0000 7fa86affd640 1 --2- 172.21.15.114:0/4270182312 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b87f47e290 0x55b87f4e3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:47.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.405+0000 7fa86b7fe640 1 -- 172.21.15.114:0/4270182312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b87f488b30 msgr2=0x55b87f4ed970 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:31:47.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.405+0000 7fa86b7fe640 1 --2- 172.21.15.114:0/4270182312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b87f488b30 0x55b87f4ed970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.405+0000 7fa86b7fe640 1 -- 172.21.15.114:0/4270182312 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b87f47e290 msgr2=0x55b87f4e3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:47.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.405+0000 7fa86b7fe640 1 --2- 172.21.15.114:0/4270182312 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b87f47e290 0x55b87f4e3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.405+0000 7fa86b7fe640 1 -- 172.21.15.114:0/4270182312 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b87f3df3c0 con 0x55b87f47d970 2024-09-13T18:31:47.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.406+0000 7fa86b7fe640 1 --2- 172.21.15.114:0/4270182312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b87f47d970 0x55b87f4e2ae0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa860009890 tx=0x7fa86001af40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea27e7a19b145b81 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:47.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.406+0000 7fa84affd640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:31:47.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.407+0000 7fa84affd640 1 -- 172.21.15.114:0/4270182312 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa860018040 con 0x55b87f47d970 2024-09-13T18:31:47.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.407+0000 7fa84affd640 1 -- 172.21.15.114:0/4270182312 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fa860014b50 con 0x55b87f47d970 2024-09-13T18:31:47.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.407+0000 7fa874c59740 1 -- 172.21.15.114:0/4270182312 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa864007e10 con 0x55b87f47d970 2024-09-13T18:31:47.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.408+0000 7fa84affd640 1 -- 172.21.15.114:0/4270182312 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa860014d10 con 0x55b87f47d970 2024-09-13T18:31:47.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.410+0000 7fa874c59740 10 client.27584 fetch_fsmap learned FSMap version 34 2024-09-13T18:31:47.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.410+0000 7fa874c59740 1 -- 172.21.15.114:0/4270182312 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55b87f3a6e80 con 0x55b87f47d970 2024-09-13T18:31:47.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.410+0000 7fa84affd640 1 -- 172.21.15.114:0/4270182312 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa864007e10 con 0x55b87f47d970 2024-09-13T18:31:47.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.411+0000 7fa84affd640 1 -- 172.21.15.114:0/4270182312 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa860012730 con 0x55b87f47d970 2024-09-13T18:31:47.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.411+0000 7fa874c59740 10 client.27584 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:31:47.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.411+0000 7fa874c59740 10 client.27584 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-13T18:31:47.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.411+0000 7fa874c59740 20 client.27584 populate_metadata read hostname 'smithi114' 2024-09-13T18:31:47.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.411+0000 7fa874c59740 1 --2- 172.21.15.114:0/4270182312 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55b87f4ef9b0 0x55b87f4f1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:47.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.411+0000 7fa874c59740 4 client.27584 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:31:47.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.411+0000 7fa874c59740 1 -- 172.21.15.114:0/4270182312 --> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b87f3d31f0 con 0x55b87f4ef9b0 2024-09-13T18:31:47.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.411+0000 7fa86affd640 1 --2- 172.21.15.114:0/4270182312 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55b87f4ef9b0 0x55b87f4f1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:47.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.412+0000 7fa86affd640 1 --2- 172.21.15.114:0/4270182312 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55b87f4ef9b0 0x55b87f4f1dd0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=30bc11c95377f609 server_cookie=24ab40de3bb30315 in_seq=0 out_seq=0 2024-09-13T18:31:47.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.412+0000 7fa84affd640 10 client.27584 ms_handle_connect on v2:172.21.15.134:6834/1150003275 2024-09-13T18:31:47.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.417+0000 7fa84affd640 1 -- 172.21.15.114:0/4270182312 <== mds.3 v2:172.21.15.134:6834/1150003275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b87f3d31f0 con 0x55b87f4ef9b0 2024-09-13T18:31:47.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.417+0000 7fa84affd640 10 client.27584 handle_command_reply: tid=0 2024-09-13T18:31:47.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.417+0000 7fa874c59740 1 client.27584 shutdown 2024-09-13T18:31:47.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.419+0000 7fa874c59740 1 -- 172.21.15.114:0/4270182312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b87f47d970 msgr2=0x55b87f4e2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:47.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.419+0000 7fa874c59740 1 --2- 172.21.15.114:0/4270182312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b87f47d970 0x55b87f4e2ae0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa860009890 tx=0x7fa86001af40 comp rx=0 tx=0).stop 2024-09-13T18:31:47.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.420+0000 7fa874c59740 1 -- 172.21.15.114:0/4270182312 shutdown_connections 2024-09-13T18:31:47.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.420+0000 7fa874c59740 1 --2- 172.21.15.114:0/4270182312 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x55b87f4ef9b0 0x55b87f4f1dd0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.420+0000 7fa874c59740 1 --2- 172.21.15.114:0/4270182312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b87f488b30 0x55b87f4ed970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.420+0000 7fa874c59740 1 --2- 172.21.15.114:0/4270182312 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b87f47e290 0x55b87f4e3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.420+0000 7fa874c59740 1 --2- 172.21.15.114:0/4270182312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b87f47d970 0x55b87f4e2ae0 secure :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa860009890 tx=0x7fa86001af40 comp rx=0 tx=0).stop 2024-09-13T18:31:47.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.420+0000 7fa874c59740 1 -- 172.21.15.114:0/4270182312 >> 172.21.15.114:0/4270182312 conn(0x55b87f479ee0 msgr2=0x55b87f483c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:47.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.420+0000 7fa84affd640 0 client.27584 ms_handle_reset on v2:172.21.15.134:6834/1150003275 2024-09-13T18:31:47.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.420+0000 7fa874c59740 1 -- 172.21.15.114:0/4270182312 shutdown_connections 2024-09-13T18:31:47.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.420+0000 7fa874c59740 1 -- 172.21.15.114:0/4270182312 wait complete. 2024-09-13T18:31:47.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.421+0000 7fa874c59740 20 client.27584 trim_cache size 0 max 16384 2024-09-13T18:31:47.423 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:31:47.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.422+0000 7fa870bf8640 1 -- 172.21.15.114:0/2682531979 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa8540a1410 msgr2=0x7fa8540a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:47.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.422+0000 7fa870bf8640 1 --2- 172.21.15.114:0/2682531979 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa8540a1410 0x7fa8540a38d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa864000f80 tx=0x7fa8640048f0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.422+0000 7fa870bf8640 1 -- 172.21.15.114:0/2682531979 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa86c10b650 msgr2=0x7fa86c10f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:47.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.422+0000 7fa870bf8640 1 --2- 172.21.15.114:0/2682531979 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa86c10b650 0x7fa86c10f3a0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fa85c03ba80 tx=0x7fa85c004960 comp rx=0 tx=0).stop 2024-09-13T18:31:47.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.422+0000 7fa870bf8640 1 -- 172.21.15.114:0/2682531979 shutdown_connections 2024-09-13T18:31:47.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.422+0000 7fa870bf8640 1 --2- 172.21.15.114:0/2682531979 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa8540a1410 0x7fa8540a38d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.422+0000 7fa870bf8640 1 --2- 172.21.15.114:0/2682531979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa86c112670 0x7fa86c112b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.422+0000 7fa870bf8640 1 --2- 172.21.15.114:0/2682531979 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa86c10b650 0x7fa86c10f3a0 secure :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fa85c03ba80 tx=0x7fa85c004960 comp rx=0 tx=0).stop 2024-09-13T18:31:47.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.422+0000 7fa870bf8640 1 --2- 172.21.15.114:0/2682531979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa86c106550 0x7fa86c10ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.423+0000 7fa870bf8640 1 -- 172.21.15.114:0/2682531979 >> 172.21.15.114:0/2682531979 conn(0x7fa86c075c80 msgr2=0x7fa86c0771c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:47.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.427+0000 7fa870bf8640 1 -- 172.21.15.114:0/2682531979 shutdown_connections 2024-09-13T18:31:47.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.427+0000 7fa870bf8640 1 -- 172.21.15.114:0/2682531979 wait complete. 2024-09-13T18:31:47.456 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-09-13T18:31:47.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.698+0000 7fbef5efc640 1 -- 172.21.15.114:0/4273317159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbee80b7510 msgr2=0x7fbee80b9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:47.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.698+0000 7fbef5efc640 1 --2- 172.21.15.114:0/4273317159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbee80b7510 0x7fbee80b9900 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fbedc009860 tx=0x7fbedc02a3f0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.699+0000 7fbef5efc640 1 -- 172.21.15.114:0/4273317159 shutdown_connections 2024-09-13T18:31:47.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.699+0000 7fbef5efc640 1 --2- 172.21.15.114:0/4273317159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbee80b7510 0x7fbee80b9900 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.699+0000 7fbef5efc640 1 --2- 172.21.15.114:0/4273317159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbee80a5110 0x7fbee80b6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.699+0000 7fbef5efc640 1 --2- 172.21.15.114:0/4273317159 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbee80a47f0 0x7fbee80a4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.699+0000 7fbef5efc640 1 -- 172.21.15.114:0/4273317159 >> 172.21.15.114:0/4273317159 conn(0x7fbee801a0e0 msgr2=0x7fbee801a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:47.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.703+0000 7fbef5efc640 1 -- 172.21.15.114:0/4273317159 shutdown_connections 2024-09-13T18:31:47.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.703+0000 7fbef5efc640 1 -- 172.21.15.114:0/4273317159 wait complete. 2024-09-13T18:31:47.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.704+0000 7fbef5efc640 1 Processor -- start 2024-09-13T18:31:47.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.704+0000 7fbef5efc640 1 -- start start 2024-09-13T18:31:47.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.704+0000 7fbef5efc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbee80a47f0 0x7fbee81430c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:47.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.704+0000 7fbef5efc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbee80a5110 0x7fbee8143600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:47.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.704+0000 7fbef5efc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbee80b7510 0x7fbee81479f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:47.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.704+0000 7fbef5efc640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee8147ff0 con 0x7fbee80a5110 2024-09-13T18:31:47.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.704+0000 7fbef5efc640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee8148160 con 0x7fbee80a47f0 2024-09-13T18:31:47.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.704+0000 7fbef5efc640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee8148320 con 0x7fbee80b7510 2024-09-13T18:31:47.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.705+0000 7fbef56fb640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbee80b7510 0x7fbee81479f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:47.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.705+0000 7fbef56fb640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbee80b7510 0x7fbee81479f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:41480/0 (socket says 172.21.15.114:41480) 2024-09-13T18:31:47.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.705+0000 7fbef56fb640 1 -- 172.21.15.114:0/3603636226 learned_addr learned my addr 172.21.15.114:0/3603636226 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:47.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.705+0000 7fbef56fb640 1 -- 172.21.15.114:0/3603636226 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbee80a47f0 msgr2=0x7fbee81430c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:31:47.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.705+0000 7fbef56fb640 1 --2- 172.21.15.114:0/3603636226 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbee80a47f0 0x7fbee81430c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.705+0000 7fbef56fb640 1 -- 172.21.15.114:0/3603636226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbee80a5110 msgr2=0x7fbee8143600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:47.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.705+0000 7fbef56fb640 1 --2- 172.21.15.114:0/3603636226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbee80a5110 0x7fbee8143600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.705+0000 7fbef56fb640 1 -- 172.21.15.114:0/3603636226 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef004eac0 con 0x7fbee80b7510 2024-09-13T18:31:47.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.708+0000 7fbef56fb640 1 --2- 172.21.15.114:0/3603636226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbee80b7510 0x7fbee81479f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fbedc0309c0 tx=0x7fbedc0309f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:47.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.709+0000 7fbeedffb640 1 -- 172.21.15.114:0/3603636226 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbedc0046c0 con 0x7fbee80b7510 2024-09-13T18:31:47.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.709+0000 7fbeedffb640 1 -- 172.21.15.114:0/3603636226 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbedc004cb0 con 0x7fbee80b7510 2024-09-13T18:31:47.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.709+0000 7fbeedffb640 1 -- 172.21.15.114:0/3603636226 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbedc03cc20 con 0x7fbee80b7510 2024-09-13T18:31:47.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.711+0000 7fbef5efc640 1 -- 172.21.15.114:0/3603636226 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbedc009490 con 0x7fbee80b7510 2024-09-13T18:31:47.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.711+0000 7fbef5efc640 1 -- 172.21.15.114:0/3603636226 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbee8148840 con 0x7fbee80b7510 2024-09-13T18:31:47.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.715+0000 7fbef9e86740 1 Processor -- start 2024-09-13T18:31:47.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.717+0000 7fbef9e86740 1 -- start start 2024-09-13T18:31:47.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.717+0000 7fbef9e86740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560ff2e64830 0x560ff2e64c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:47.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.717+0000 7fbef9e86740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ff2e65150 0x560ff2e80600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:47.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.717+0000 7fbef9e86740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560ff2e80b40 0x560ff2e82f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:47.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.717+0000 7fbef9e86740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ff2dcecc0 con 0x560ff2e64830 2024-09-13T18:31:47.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.717+0000 7fbef9e86740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ff2dcc450 con 0x560ff2e65150 2024-09-13T18:31:47.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.717+0000 7fbef9e86740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ff2dc7d40 con 0x560ff2e80b40 2024-09-13T18:31:47.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.717+0000 7fbeedffb640 1 -- 172.21.15.114:0/3603636226 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fbedc004820 con 0x7fbee80b7510 2024-09-13T18:31:47.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.719+0000 7fbeedffb640 1 --2- 172.21.15.114:0/3603636226 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbee00a1410 0x7fbee00a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:47.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.719+0000 7fbeedffb640 1 -- 172.21.15.114:0/3603636226 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fbedc0ee7e0 con 0x7fbee80b7510 2024-09-13T18:31:47.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.719+0000 7fbef4efa640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ff2e65150 0x560ff2e80600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:47.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.720+0000 7fbef4efa640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ff2e65150 0x560ff2e80600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:46166/0 (socket says 172.21.15.114:46166) 2024-09-13T18:31:47.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.720+0000 7fbef4efa640 1 -- 172.21.15.114:0/2861080635 learned_addr learned my addr 172.21.15.114:0/2861080635 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:47.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.720+0000 7fbeeffff640 1 --2- 172.21.15.114:0/2861080635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560ff2e80b40 0x560ff2e82f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:47.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.720+0000 7fbeeffff640 1 --2- 172.21.15.114:0/2861080635 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560ff2e64830 0x560ff2e64c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:47.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.720+0000 7fbef4efa640 1 -- 172.21.15.114:0/2861080635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560ff2e80b40 msgr2=0x560ff2e82f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:47.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.720+0000 7fbef4efa640 1 --2- 172.21.15.114:0/2861080635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560ff2e80b40 0x560ff2e82f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.720+0000 7fbef4efa640 1 -- 172.21.15.114:0/2861080635 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560ff2e64830 msgr2=0x560ff2e64c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:47.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.720+0000 7fbef4efa640 1 --2- 172.21.15.114:0/2861080635 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560ff2e64830 0x560ff2e64c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.720+0000 7fbef4efa640 1 -- 172.21.15.114:0/2861080635 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ff2d8f620 con 0x560ff2e65150 2024-09-13T18:31:47.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.720+0000 7fbeeffff640 1 --2- 172.21.15.114:0/2861080635 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560ff2e64830 0x560ff2e64c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:31:47.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.721+0000 7fbef4efa640 1 --2- 172.21.15.114:0/2861080635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ff2e65150 0x560ff2e80600 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fbee4009780 tx=0x7fbee4009c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd0d02bf6e2e2cad server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:47.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.722+0000 7fbedaffd640 1 -- 172.21.15.114:0/2861080635 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee400bad0 con 0x560ff2e65150 2024-09-13T18:31:47.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.723+0000 7fbef56fb640 1 --2- 172.21.15.114:0/3603636226 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbee00a1410 0x7fbee00a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:47.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.723+0000 7fbedaffd640 1 -- 172.21.15.114:0/2861080635 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbee4009e90 con 0x560ff2e65150 2024-09-13T18:31:47.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.723+0000 7fbedaffd640 1 -- 172.21.15.114:0/2861080635 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee40105e0 con 0x560ff2e65150 2024-09-13T18:31:47.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.724+0000 7fbef9e86740 1 -- 172.21.15.114:0/2861080635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ff2e65150 msgr2=0x560ff2e80600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:47.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.724+0000 7fbef9e86740 1 --2- 172.21.15.114:0/2861080635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ff2e65150 0x560ff2e80600 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fbee4009780 tx=0x7fbee4009c40 comp rx=0 tx=0).stop 2024-09-13T18:31:47.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.724+0000 7fbef56fb640 1 --2- 172.21.15.114:0/3603636226 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbee00a1410 0x7fbee00a38d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbedc03e040 tx=0x7fbedc0db040 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:47.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.724+0000 7fbef9e86740 1 -- 172.21.15.114:0/2861080635 shutdown_connections 2024-09-13T18:31:47.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.724+0000 7fbef9e86740 1 --2- 172.21.15.114:0/2861080635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560ff2e80b40 0x560ff2e82f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.724+0000 7fbef9e86740 1 --2- 172.21.15.114:0/2861080635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ff2e65150 0x560ff2e80600 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.724+0000 7fbef9e86740 1 --2- 172.21.15.114:0/2861080635 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560ff2e64830 0x560ff2e64c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.724+0000 7fbef9e86740 1 -- 172.21.15.114:0/2861080635 >> 172.21.15.114:0/2861080635 conn(0x560ff2e60ee0 msgr2=0x560ff2e612c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:47.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.726+0000 7fbef9e86740 1 -- 172.21.15.114:0/2861080635 shutdown_connections 2024-09-13T18:31:47.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.727+0000 7fbef9e86740 1 -- 172.21.15.114:0/2861080635 wait complete. 2024-09-13T18:31:47.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.727+0000 7fbef9e86740 1 -- start start 2024-09-13T18:31:47.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.727+0000 7fbef9e86740 1 Processor -- start 2024-09-13T18:31:47.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.728+0000 7fbef9e86740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ff2e64830 0x560ff2e80180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:47.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.728+0000 7fbef9e86740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560ff2e65150 0x560ff2e7d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:47.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.728+0000 7fbef9e86740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560ff2e80b40 0x560ff2e7da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:47.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.728+0000 7fbef9e86740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ff2d86180 con 0x560ff2e65150 2024-09-13T18:31:47.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.728+0000 7fbef9e86740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ff2aa0430 con 0x560ff2e64830 2024-09-13T18:31:47.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.728+0000 7fbef9e86740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ff2d4b5e0 con 0x560ff2e80b40 2024-09-13T18:31:47.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.728+0000 7fbef9e86740 10 client.0 _ll_register_callbacks cb 0x560ff2db7aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:31:47.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.730+0000 7fbeeffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ff2e64830 0x560ff2e80180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:47.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.730+0000 7fbeeffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560ff2e80b40 0x560ff2e7da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:47.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.730+0000 7fbeeffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ff2e64830 0x560ff2e80180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:46180/0 (socket says 172.21.15.114:46180) 2024-09-13T18:31:47.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.730+0000 7fbeeffff640 1 -- 172.21.15.114:0/4177843766 learned_addr learned my addr 172.21.15.114:0/4177843766 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:47.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.731+0000 7fbeeffff640 1 -- 172.21.15.114:0/4177843766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560ff2e80b40 msgr2=0x560ff2e7da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:47.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.731+0000 7fbeeffff640 1 --2- 172.21.15.114:0/4177843766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560ff2e80b40 0x560ff2e7da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.731+0000 7fbeeffff640 1 -- 172.21.15.114:0/4177843766 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560ff2e65150 msgr2=0x560ff2e7d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:47.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.731+0000 7fbef4efa640 1 --2- 172.21.15.114:0/4177843766 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560ff2e65150 0x560ff2e7d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:47.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.737+0000 7fbeeffff640 1 --2- 172.21.15.114:0/4177843766 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560ff2e65150 0x560ff2e7d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.737+0000 7fbeeffff640 1 -- 172.21.15.114:0/4177843766 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbee40093e0 con 0x560ff2e64830 2024-09-13T18:31:47.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.737+0000 7fbeeffff640 1 --2- 172.21.15.114:0/4177843766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ff2e64830 0x560ff2e80180 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fbef004efc0 tx=0x7fbef0066740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ae8e3b9f3dc6c35 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:47.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.738+0000 7fbef4efa640 1 --2- 172.21.15.114:0/4177843766 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560ff2e65150 0x560ff2e7d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:31:47.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.740+0000 7fbecffff640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:31:47.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.740+0000 7fbecffff640 1 -- 172.21.15.114:0/4177843766 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef0071dc0 con 0x560ff2e64830 2024-09-13T18:31:47.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.740+0000 7fbef9e86740 1 -- 172.21.15.114:0/4177843766 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560ff2df1b80 con 0x560ff2e64830 2024-09-13T18:31:47.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.741+0000 7fbecffff640 1 -- 172.21.15.114:0/4177843766 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fbef00668c0 con 0x560ff2e64830 2024-09-13T18:31:47.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.741+0000 7fbecffff640 1 -- 172.21.15.114:0/4177843766 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbef0066a80 con 0x560ff2e64830 2024-09-13T18:31:47.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.741+0000 7fbef9e86740 10 client.27593 fetch_fsmap learned FSMap version 34 2024-09-13T18:31:47.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.741+0000 7fbef9e86740 1 -- 172.21.15.114:0/4177843766 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x560ff2dc63c0 con 0x560ff2e64830 2024-09-13T18:31:47.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.745+0000 7fbecffff640 1 -- 172.21.15.114:0/4177843766 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560ff2df1b80 con 0x560ff2e64830 2024-09-13T18:31:47.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.745+0000 7fbecffff640 1 -- 172.21.15.114:0/4177843766 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbef0081020 con 0x560ff2e64830 2024-09-13T18:31:47.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.747+0000 7fbef9e86740 10 client.27593 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:31:47.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.747+0000 7fbef9e86740 10 client.27593 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-13T18:31:47.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.747+0000 7fbef9e86740 20 client.27593 populate_metadata read hostname 'smithi114' 2024-09-13T18:31:47.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.747+0000 7fbef9e86740 1 --2- 172.21.15.114:0/4177843766 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x560ff2e904a0 0x560ff2e928c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:47.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.747+0000 7fbef9e86740 4 client.27593 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:31:47.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.747+0000 7fbef9e86740 1 -- 172.21.15.114:0/4177843766 --> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560ff2dba1f0 con 0x560ff2e904a0 2024-09-13T18:31:47.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.748+0000 7fbef4efa640 1 --2- 172.21.15.114:0/4177843766 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x560ff2e904a0 0x560ff2e928c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:47.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.751+0000 7fbef4efa640 1 --2- 172.21.15.114:0/4177843766 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x560ff2e904a0 0x560ff2e928c0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d34ade1d6350b640 server_cookie=28372e635cb0be2a in_seq=0 out_seq=0 2024-09-13T18:31:47.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.751+0000 7fbecffff640 10 client.27593 ms_handle_connect on v2:172.21.15.204:6838/344883912 2024-09-13T18:31:47.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.754+0000 7fbecffff640 1 -- 172.21.15.114:0/4177843766 <== mds.2 v2:172.21.15.204:6838/344883912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x560ff2dba1f0 con 0x560ff2e904a0 2024-09-13T18:31:47.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.755+0000 7fbecffff640 10 client.27593 handle_command_reply: tid=0 2024-09-13T18:31:47.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.756+0000 7fbef9e86740 1 client.27593 shutdown 2024-09-13T18:31:47.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.759+0000 7fbef9e86740 1 -- 172.21.15.114:0/4177843766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ff2e64830 msgr2=0x560ff2e80180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:47.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.759+0000 7fbef9e86740 1 --2- 172.21.15.114:0/4177843766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ff2e64830 0x560ff2e80180 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fbef004efc0 tx=0x7fbef0066740 comp rx=0 tx=0).stop 2024-09-13T18:31:47.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.765+0000 7fbef9e86740 1 -- 172.21.15.114:0/4177843766 shutdown_connections 2024-09-13T18:31:47.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.765+0000 7fbef9e86740 1 --2- 172.21.15.114:0/4177843766 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x560ff2e904a0 0x560ff2e928c0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.765+0000 7fbef9e86740 1 --2- 172.21.15.114:0/4177843766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560ff2e80b40 0x560ff2e7da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.765+0000 7fbef9e86740 1 --2- 172.21.15.114:0/4177843766 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560ff2e65150 0x560ff2e7d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.765+0000 7fbef9e86740 1 --2- 172.21.15.114:0/4177843766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ff2e64830 0x560ff2e80180 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.765+0000 7fbef9e86740 1 -- 172.21.15.114:0/4177843766 >> 172.21.15.114:0/4177843766 conn(0x560ff2e60ee0 msgr2=0x560ff2e60230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:47.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.766+0000 7fbecffff640 0 client.27593 ms_handle_reset on v2:172.21.15.204:6838/344883912 2024-09-13T18:31:47.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.768+0000 7fbef9e86740 1 -- 172.21.15.114:0/4177843766 shutdown_connections 2024-09-13T18:31:47.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.772+0000 7fbef9e86740 1 -- 172.21.15.114:0/4177843766 wait complete. 2024-09-13T18:31:47.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.773+0000 7fbef9e86740 20 client.27593 trim_cache size 0 max 16384 2024-09-13T18:31:47.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.804+0000 7fbef9e86740 1 Processor -- start 2024-09-13T18:31:47.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.804+0000 7fbef9e86740 1 -- start start 2024-09-13T18:31:47.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.805+0000 7fbef9e86740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ff2e64830 0x560ff2e87860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:47.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.805+0000 7fbef9e86740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560ff2e65150 0x560ff2e87da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:47.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.805+0000 7fbef9e86740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560ff2e80b40 0x560ff2e7fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:47.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.805+0000 7fbef9e86740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ff2e80a00 con 0x560ff2e80b40 2024-09-13T18:31:47.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.805+0000 7fbef9e86740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ff2e5cd30 con 0x560ff2e64830 2024-09-13T18:31:47.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.805+0000 7fbef9e86740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ff2e835f0 con 0x560ff2e65150 2024-09-13T18:31:47.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.806+0000 7fbef4efa640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560ff2e65150 0x560ff2e87da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:47.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.806+0000 7fbef4efa640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560ff2e65150 0x560ff2e87da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:41492/0 (socket says 172.21.15.114:41492) 2024-09-13T18:31:47.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.806+0000 7fbef4efa640 1 -- 172.21.15.114:0/763020427 learned_addr learned my addr 172.21.15.114:0/763020427 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:47.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.806+0000 7fbef4efa640 1 -- 172.21.15.114:0/763020427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ff2e64830 msgr2=0x560ff2e87860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:47.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.806+0000 7fbef4efa640 1 --2- 172.21.15.114:0/763020427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ff2e64830 0x560ff2e87860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.806+0000 7fbef4efa640 1 -- 172.21.15.114:0/763020427 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560ff2e80b40 msgr2=0x560ff2e7fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:47.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.806+0000 7fbef4efa640 1 --2- 172.21.15.114:0/763020427 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560ff2e80b40 0x560ff2e7fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.806+0000 7fbef4efa640 1 -- 172.21.15.114:0/763020427 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ff2d68770 con 0x560ff2e65150 2024-09-13T18:31:47.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.806+0000 7fbef4efa640 1 --2- 172.21.15.114:0/763020427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560ff2e65150 0x560ff2e87da0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbee400b4b0 tx=0x7fbee4009190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcb99d2c891909bb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:47.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.807+0000 7fbedb7fe640 1 -- 172.21.15.114:0/763020427 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee4020040 con 0x560ff2e65150 2024-09-13T18:31:47.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.807+0000 7fbedb7fe640 1 -- 172.21.15.114:0/763020427 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbee4024850 con 0x560ff2e65150 2024-09-13T18:31:47.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.808+0000 7fbef9e86740 1 -- 172.21.15.114:0/763020427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560ff2e65150 msgr2=0x560ff2e87da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:47.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.808+0000 7fbef9e86740 1 --2- 172.21.15.114:0/763020427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560ff2e65150 0x560ff2e87da0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbee400b4b0 tx=0x7fbee4009190 comp rx=0 tx=0).stop 2024-09-13T18:31:47.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.810+0000 7fbef9e86740 1 -- 172.21.15.114:0/763020427 shutdown_connections 2024-09-13T18:31:47.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.810+0000 7fbef9e86740 1 --2- 172.21.15.114:0/763020427 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560ff2e80b40 0x560ff2e7fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.810+0000 7fbef9e86740 1 --2- 172.21.15.114:0/763020427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560ff2e65150 0x560ff2e87da0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.810+0000 7fbef9e86740 1 --2- 172.21.15.114:0/763020427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ff2e64830 0x560ff2e87860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.810+0000 7fbef9e86740 1 -- 172.21.15.114:0/763020427 >> 172.21.15.114:0/763020427 conn(0x560ff2e60ee0 msgr2=0x560ff2e8c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:47.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.814+0000 7fbef9e86740 1 -- 172.21.15.114:0/763020427 shutdown_connections 2024-09-13T18:31:47.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.814+0000 7fbef9e86740 1 -- 172.21.15.114:0/763020427 wait complete. 2024-09-13T18:31:47.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.814+0000 7fbef9e86740 1 -- start start 2024-09-13T18:31:47.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.814+0000 7fbef9e86740 1 Processor -- start 2024-09-13T18:31:47.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.815+0000 7fbef9e86740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560ff2e64830 0x560ff2e83e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:47.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.815+0000 7fbef9e86740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560ff2e65150 0x560ff2e84360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:47.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.815+0000 7fbef9e86740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ff2e80b40 0x560ff2e87190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:47.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.815+0000 7fbef9e86740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ff2e87700 con 0x560ff2e65150 2024-09-13T18:31:47.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.815+0000 7fbef9e86740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ff2e87870 con 0x560ff2e80b40 2024-09-13T18:31:47.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.815+0000 7fbef9e86740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ff2e87a30 con 0x560ff2e64830 2024-09-13T18:31:47.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.815+0000 7fbef9e86740 10 client.0 _ll_register_callbacks cb 0x7fbee4009060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:31:47.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.816+0000 7fbeeffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560ff2e64830 0x560ff2e83e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:47.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.816+0000 7fbeeffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560ff2e64830 0x560ff2e83e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:41506/0 (socket says 172.21.15.114:41506) 2024-09-13T18:31:47.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.816+0000 7fbeeffff640 1 -- 172.21.15.114:0/2782042685 learned_addr learned my addr 172.21.15.114:0/2782042685 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:47.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.816+0000 7fbeeffff640 1 --2- 172.21.15.114:0/2782042685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ff2e80b40 0x560ff2e87190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:47.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.816+0000 7fbef4efa640 1 --2- 172.21.15.114:0/2782042685 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560ff2e65150 0x560ff2e84360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:47.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.817+0000 7fbeeffff640 1 -- 172.21.15.114:0/2782042685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ff2e80b40 msgr2=0x560ff2e87190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:47.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.817+0000 7fbeeffff640 1 --2- 172.21.15.114:0/2782042685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ff2e80b40 0x560ff2e87190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.817+0000 7fbeeffff640 1 -- 172.21.15.114:0/2782042685 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560ff2e65150 msgr2=0x560ff2e84360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:47.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.817+0000 7fbeeffff640 1 --2- 172.21.15.114:0/2782042685 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560ff2e65150 0x560ff2e84360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.817+0000 7fbeeffff640 1 -- 172.21.15.114:0/2782042685 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560ff2d8f620 con 0x560ff2e64830 2024-09-13T18:31:47.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.818+0000 7fbeeffff640 1 --2- 172.21.15.114:0/2782042685 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560ff2e64830 0x560ff2e83e20 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fbef0065eb0 tx=0x7fbef007b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbdead68e11e3cf6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:47.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.818+0000 7fbecffff640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:31:47.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.829+0000 7fbecffff640 1 -- 172.21.15.114:0/2782042685 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef00769d0 con 0x560ff2e64830 2024-09-13T18:31:47.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.829+0000 7fbecffff640 1 -- 172.21.15.114:0/2782042685 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fbef00744b0 con 0x560ff2e64830 2024-09-13T18:31:47.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.833+0000 7fbef9e86740 1 -- 172.21.15.114:0/2782042685 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbee4009e90 con 0x560ff2e64830 2024-09-13T18:31:47.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.833+0000 7fbecffff640 1 -- 172.21.15.114:0/2782042685 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbef00680d0 con 0x560ff2e64830 2024-09-13T18:31:47.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.840+0000 7fbef9e86740 10 client.27562 fetch_fsmap learned FSMap version 34 2024-09-13T18:31:47.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.840+0000 7fbef9e86740 1 -- 172.21.15.114:0/2782042685 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x560ff2d8de80 con 0x560ff2e64830 2024-09-13T18:31:47.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.842+0000 7fbecffff640 1 -- 172.21.15.114:0/2782042685 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbee4009e90 con 0x560ff2e64830 2024-09-13T18:31:47.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.842+0000 7fbecffff640 1 -- 172.21.15.114:0/2782042685 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbef0081070 con 0x560ff2e64830 2024-09-13T18:31:47.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.848+0000 7fbef9e86740 10 client.27562 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:31:47.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.848+0000 7fbef9e86740 10 client.27562 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-13T18:31:47.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.848+0000 7fbef9e86740 20 client.27562 populate_metadata read hostname 'smithi114' 2024-09-13T18:31:47.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.848+0000 7fbef9e86740 1 --2- 172.21.15.114:0/2782042685 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x560ff2ee79c0 0x560ff2ee9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:47.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.848+0000 7fbef9e86740 4 client.27562 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:31:47.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.848+0000 7fbef9e86740 1 -- 172.21.15.114:0/2782042685 --> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560ff2abb4e0 con 0x560ff2ee79c0 2024-09-13T18:31:47.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.849+0000 7fbef4efa640 1 --2- 172.21.15.114:0/2782042685 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x560ff2ee79c0 0x560ff2ee9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:47.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.850+0000 7fbef4efa640 1 --2- 172.21.15.114:0/2782042685 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x560ff2ee79c0 0x560ff2ee9db0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=df618616f9d1fd45 server_cookie=f1d09abaa57df396 in_seq=0 out_seq=0 2024-09-13T18:31:47.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.850+0000 7fbecffff640 10 client.27562 ms_handle_connect on v2:172.21.15.204:6838/344883912 2024-09-13T18:31:47.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.850+0000 7fbecffff640 1 -- 172.21.15.114:0/2782042685 <== mds.2 v2:172.21.15.204:6838/344883912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560ff2abb4e0 con 0x560ff2ee79c0 2024-09-13T18:31:47.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.850+0000 7fbecffff640 10 client.27562 handle_command_reply: tid=0 2024-09-13T18:31:47.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.851+0000 7fbef9e86740 1 client.27562 shutdown 2024-09-13T18:31:47.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.858+0000 7fbef9e86740 1 -- 172.21.15.114:0/2782042685 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560ff2e64830 msgr2=0x560ff2e83e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:47.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.858+0000 7fbef9e86740 1 --2- 172.21.15.114:0/2782042685 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560ff2e64830 0x560ff2e83e20 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fbef0065eb0 tx=0x7fbef007b040 comp rx=0 tx=0).stop 2024-09-13T18:31:47.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.858+0000 7fbef9e86740 1 -- 172.21.15.114:0/2782042685 shutdown_connections 2024-09-13T18:31:47.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.858+0000 7fbef9e86740 1 --2- 172.21.15.114:0/2782042685 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x560ff2ee79c0 0x560ff2ee9db0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.858+0000 7fbef9e86740 1 --2- 172.21.15.114:0/2782042685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ff2e80b40 0x560ff2e87190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.858+0000 7fbef9e86740 1 --2- 172.21.15.114:0/2782042685 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560ff2e65150 0x560ff2e84360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.858+0000 7fbef9e86740 1 --2- 172.21.15.114:0/2782042685 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560ff2e64830 0x560ff2e83e20 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.858+0000 7fbef9e86740 1 -- 172.21.15.114:0/2782042685 >> 172.21.15.114:0/2782042685 conn(0x560ff2e60ee0 msgr2=0x560ff2e785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:47.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.858+0000 7fbecffff640 0 client.27562 ms_handle_reset on v2:172.21.15.204:6838/344883912 2024-09-13T18:31:47.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.858+0000 7fbef9e86740 1 -- 172.21.15.114:0/2782042685 shutdown_connections 2024-09-13T18:31:47.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.858+0000 7fbef9e86740 1 -- 172.21.15.114:0/2782042685 wait complete. 2024-09-13T18:31:47.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.859+0000 7fbef9e86740 20 client.27562 trim_cache size 0 max 16384 2024-09-13T18:31:47.862 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:31:47.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.860+0000 7fbef5efc640 1 -- 172.21.15.114:0/3603636226 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbee00a1410 msgr2=0x7fbee00a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:47.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.860+0000 7fbef5efc640 1 --2- 172.21.15.114:0/3603636226 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbee00a1410 0x7fbee00a38d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbedc03e040 tx=0x7fbedc0db040 comp rx=0 tx=0).stop 2024-09-13T18:31:47.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.860+0000 7fbef5efc640 1 -- 172.21.15.114:0/3603636226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbee80b7510 msgr2=0x7fbee81479f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:47.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.860+0000 7fbef5efc640 1 --2- 172.21.15.114:0/3603636226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbee80b7510 0x7fbee81479f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fbedc0309c0 tx=0x7fbedc0309f0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.860+0000 7fbef5efc640 1 -- 172.21.15.114:0/3603636226 shutdown_connections 2024-09-13T18:31:47.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.860+0000 7fbef5efc640 1 --2- 172.21.15.114:0/3603636226 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbee00a1410 0x7fbee00a38d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.860+0000 7fbef5efc640 1 --2- 172.21.15.114:0/3603636226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbee80b7510 0x7fbee81479f0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.860+0000 7fbef5efc640 1 --2- 172.21.15.114:0/3603636226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbee80a5110 0x7fbee8143600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.860+0000 7fbef5efc640 1 --2- 172.21.15.114:0/3603636226 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbee80a47f0 0x7fbee81430c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:47.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.860+0000 7fbef5efc640 1 -- 172.21.15.114:0/3603636226 >> 172.21.15.114:0/3603636226 conn(0x7fbee801a0e0 msgr2=0x7fbee80b5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:47.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.865+0000 7fbef5efc640 1 -- 172.21.15.114:0/3603636226 shutdown_connections 2024-09-13T18:31:47.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:47.867+0000 7fbef5efc640 1 -- 172.21.15.114:0/3603636226 wait complete. 2024-09-13T18:31:47.900 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-09-13T18:31:48.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.111+0000 7ff065080640 1 -- 172.21.15.114:0/940862771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff060119400 msgr2=0x7ff06011b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:48.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.111+0000 7ff065080640 1 --2- 172.21.15.114:0/940862771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff060119400 0x7ff06011b7f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff050009930 tx=0x7ff05002a440 comp rx=0 tx=0).stop 2024-09-13T18:31:48.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.112+0000 7ff065080640 1 -- 172.21.15.114:0/940862771 shutdown_connections 2024-09-13T18:31:48.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.112+0000 7ff065080640 1 --2- 172.21.15.114:0/940862771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff060119400 0x7ff06011b7f0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.112+0000 7ff065080640 1 --2- 172.21.15.114:0/940862771 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff060107890 0x7ff060107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.112+0000 7ff065080640 1 --2- 172.21.15.114:0/940862771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0601074b0 0x7ff06010e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.112+0000 7ff065080640 1 -- 172.21.15.114:0/940862771 >> 172.21.15.114:0/940862771 conn(0x7ff06006af70 msgr2=0x7ff06006b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:48.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.115+0000 7ff065080640 1 -- 172.21.15.114:0/940862771 shutdown_connections 2024-09-13T18:31:48.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.116+0000 7ff065080640 1 -- 172.21.15.114:0/940862771 wait complete. 2024-09-13T18:31:48.120 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:47 smithi134 ceph-mon[34283]: pgmap v288: 129 pgs: 129 active+clean; 22 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 37 MiB/s wr, 824 op/s 2024-09-13T18:31:48.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.118+0000 7ff065080640 1 Processor -- start 2024-09-13T18:31:48.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.119+0000 7ff065080640 1 -- start start 2024-09-13T18:31:48.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.119+0000 7ff065080640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0601074b0 0x7ff060114b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:48.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.119+0000 7ff065080640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff060107890 0x7ff0601150b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:48.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.119+0000 7ff065080640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff060119400 0x7ff0601a1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:48.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.119+0000 7ff065080640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0601a17f0 con 0x7ff060119400 2024-09-13T18:31:48.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.119+0000 7ff065080640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0601a1960 con 0x7ff0601074b0 2024-09-13T18:31:48.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.119+0000 7ff065080640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0601a1b20 con 0x7ff060107890 2024-09-13T18:31:48.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.119+0000 7ff05ed9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0601074b0 0x7ff060114b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:48.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.120+0000 7ff05f59e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff060119400 0x7ff0601a1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:48.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.120+0000 7ff05f59e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff060119400 0x7ff0601a1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:58640/0 (socket says 172.21.15.114:58640) 2024-09-13T18:31:48.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.120+0000 7ff05f59e640 1 -- 172.21.15.114:0/3006257167 learned_addr learned my addr 172.21.15.114:0/3006257167 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:48.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.120+0000 7ff05e59c640 1 --2- 172.21.15.114:0/3006257167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff060107890 0x7ff0601150b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:48.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.121+0000 7ff05e59c640 1 -- 172.21.15.114:0/3006257167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0601074b0 msgr2=0x7ff060114b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:48.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.121+0000 7ff05e59c640 1 --2- 172.21.15.114:0/3006257167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0601074b0 0x7ff060114b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.121+0000 7ff05e59c640 1 -- 172.21.15.114:0/3006257167 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff060119400 msgr2=0x7ff0601a1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:48.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.121+0000 7ff05e59c640 1 --2- 172.21.15.114:0/3006257167 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff060119400 0x7ff0601a1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.121+0000 7ff05e59c640 1 -- 172.21.15.114:0/3006257167 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff050009560 con 0x7ff060107890 2024-09-13T18:31:48.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.121+0000 7ff05e59c640 1 --2- 172.21.15.114:0/3006257167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff060107890 0x7ff0601150b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff05401fa90 tx=0x7ff05401ff50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:48.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.122+0000 7ff047fff640 1 -- 172.21.15.114:0/3006257167 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff05401dcb0 con 0x7ff060107890 2024-09-13T18:31:48.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.122+0000 7ff047fff640 1 -- 172.21.15.114:0/3006257167 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff05401de10 con 0x7ff060107890 2024-09-13T18:31:48.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.122+0000 7ff047fff640 1 -- 172.21.15.114:0/3006257167 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff054029780 con 0x7ff060107890 2024-09-13T18:31:48.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.124+0000 7ff065080640 1 -- 172.21.15.114:0/3006257167 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0601a1e00 con 0x7ff060107890 2024-09-13T18:31:48.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.124+0000 7ff065080640 1 -- 172.21.15.114:0/3006257167 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0601a93a0 con 0x7ff060107890 2024-09-13T18:31:48.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.131+0000 7ff047fff640 1 -- 172.21.15.114:0/3006257167 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7ff0540298e0 con 0x7ff060107890 2024-09-13T18:31:48.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.132+0000 7ff067e7d740 1 Processor -- start 2024-09-13T18:31:48.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.132+0000 7ff067e7d740 1 -- start start 2024-09-13T18:31:48.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.132+0000 7ff067e7d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560471d0c970 0x560471d0cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:48.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.132+0000 7ff067e7d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560471d0d290 0x560471d175f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:48.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.132+0000 7ff067e7d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560471d17b30 0x560471d19f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:48.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.132+0000 7ff067e7d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x560471c76cc0 con 0x560471d0c970 2024-09-13T18:31:48.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.132+0000 7ff067e7d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560471c74450 con 0x560471d17b30 2024-09-13T18:31:48.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.132+0000 7ff067e7d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560471c6fd40 con 0x560471d0d290 2024-09-13T18:31:48.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.133+0000 7ff05ed9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560471d0d290 0x560471d175f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:48.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.133+0000 7ff05ed9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560471d0d290 0x560471d175f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:41530/0 (socket says 172.21.15.114:41530) 2024-09-13T18:31:48.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.133+0000 7ff05ed9d640 1 -- 172.21.15.114:0/3653311435 learned_addr learned my addr 172.21.15.114:0/3653311435 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:48.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.133+0000 7ff05ed9d640 1 -- 172.21.15.114:0/3653311435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560471d17b30 msgr2=0x560471d19f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:31:48.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.133+0000 7ff05ed9d640 1 --2- 172.21.15.114:0/3653311435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560471d17b30 0x560471d19f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.133+0000 7ff05ed9d640 1 -- 172.21.15.114:0/3653311435 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560471d0c970 msgr2=0x560471d0cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:48.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.133+0000 7ff05ed9d640 1 --2- 172.21.15.114:0/3653311435 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560471d0c970 0x560471d0cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.134+0000 7ff05ed9d640 1 -- 172.21.15.114:0/3653311435 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560471c37620 con 0x560471d0d290 2024-09-13T18:31:48.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.134+0000 7ff05ed9d640 1 --2- 172.21.15.114:0/3653311435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560471d0d290 0x560471d175f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff05800af40 tx=0x7ff05800f850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb8437c0dcc3b4eb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:48.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.135+0000 7ff047fff640 1 --2- 172.21.15.114:0/3006257167 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff0480a13d0 0x7ff0480a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:48.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.135+0000 7ff047fff640 1 -- 172.21.15.114:0/3006257167 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ff054025070 con 0x7ff060107890 2024-09-13T18:31:48.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.137+0000 7ff05f59e640 1 --2- 172.21.15.114:0/3006257167 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff0480a13d0 0x7ff0480a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:48.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.137+0000 7ff0457fa640 1 -- 172.21.15.114:0/3653311435 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff058010900 con 0x560471d0d290 2024-09-13T18:31:48.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.138+0000 7ff0457fa640 1 -- 172.21.15.114:0/3653311435 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff058018440 con 0x560471d0d290 2024-09-13T18:31:48.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.138+0000 7ff0457fa640 1 -- 172.21.15.114:0/3653311435 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0580165a0 con 0x560471d0d290 2024-09-13T18:31:48.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.139+0000 7ff067e7d740 1 -- 172.21.15.114:0/3653311435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560471d0d290 msgr2=0x560471d175f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:48.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.139+0000 7ff067e7d740 1 --2- 172.21.15.114:0/3653311435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560471d0d290 0x560471d175f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff05800af40 tx=0x7ff05800f850 comp rx=0 tx=0).stop 2024-09-13T18:31:48.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.142+0000 7ff067e7d740 1 -- 172.21.15.114:0/3653311435 shutdown_connections 2024-09-13T18:31:48.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.142+0000 7ff067e7d740 1 --2- 172.21.15.114:0/3653311435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560471d17b30 0x560471d19f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.142+0000 7ff067e7d740 1 --2- 172.21.15.114:0/3653311435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560471d0d290 0x560471d175f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.142+0000 7ff067e7d740 1 --2- 172.21.15.114:0/3653311435 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560471d0c970 0x560471d0cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.142+0000 7ff067e7d740 1 -- 172.21.15.114:0/3653311435 >> 172.21.15.114:0/3653311435 conn(0x560471d08ee0 msgr2=0x560471d092c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:48.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.143+0000 7ff067e7d740 1 -- 172.21.15.114:0/3653311435 shutdown_connections 2024-09-13T18:31:48.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.147+0000 7ff067e7d740 1 -- 172.21.15.114:0/3653311435 wait complete. 2024-09-13T18:31:48.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.147+0000 7ff067e7d740 1 -- start start 2024-09-13T18:31:48.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.147+0000 7ff067e7d740 1 Processor -- start 2024-09-13T18:31:48.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.151+0000 7ff05f59e640 1 --2- 172.21.15.114:0/3006257167 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff0480a13d0 0x7ff0480a3890 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff050005ec0 tx=0x7ff050005e50 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:48.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.151+0000 7ff067e7d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560471d0c970 0x560471d17150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:48.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.151+0000 7ff067e7d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560471d0d290 0x560471d144e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:48.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.151+0000 7ff067e7d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560471d17b30 0x560471d14a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:48.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.151+0000 7ff067e7d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x560471c2e180 con 0x560471d17b30 2024-09-13T18:31:48.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.151+0000 7ff067e7d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560471948430 con 0x560471d0d290 2024-09-13T18:31:48.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.151+0000 7ff067e7d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560471bf35e0 con 0x560471d0c970 2024-09-13T18:31:48.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.151+0000 7ff067e7d740 10 client.0 _ll_register_callbacks cb 0x560471c5faa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:31:48.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.153+0000 7ff05e59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560471d0d290 0x560471d144e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:48.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.153+0000 7ff05e59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560471d0d290 0x560471d144e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:46232/0 (socket says 172.21.15.114:46232) 2024-09-13T18:31:48.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.153+0000 7ff05e59c640 1 -- 172.21.15.114:0/3458126529 learned_addr learned my addr 172.21.15.114:0/3458126529 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:48.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.153+0000 7ff05f59e640 1 --2- 172.21.15.114:0/3458126529 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560471d17b30 0x560471d14a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:48.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.153+0000 7ff05e59c640 1 -- 172.21.15.114:0/3458126529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560471d0c970 msgr2=0x560471d17150 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:31:48.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.153+0000 7ff05e59c640 1 --2- 172.21.15.114:0/3458126529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560471d0c970 0x560471d17150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.153+0000 7ff05e59c640 1 -- 172.21.15.114:0/3458126529 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560471d17b30 msgr2=0x560471d14a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:48.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.153+0000 7ff05e59c640 1 --2- 172.21.15.114:0/3458126529 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560471d17b30 0x560471d14a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.153+0000 7ff05e59c640 1 -- 172.21.15.114:0/3458126529 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff05800aba0 con 0x560471d0d290 2024-09-13T18:31:48.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.154+0000 7ff05e59c640 1 --2- 172.21.15.114:0/3458126529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560471d0d290 0x560471d144e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff054000c00 tx=0x7ff0540afd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a6027c834282ab3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:48.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.157+0000 7ff042ff5640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:31:48.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.157+0000 7ff042ff5640 1 -- 172.21.15.114:0/3458126529 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0540acea0 con 0x560471d0d290 2024-09-13T18:31:48.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.157+0000 7ff067e7d740 1 -- 172.21.15.114:0/3458126529 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560471c99b80 con 0x560471d0d290 2024-09-13T18:31:48.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.157+0000 7ff042ff5640 1 -- 172.21.15.114:0/3458126529 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ff0540b3e80 con 0x560471d0d290 2024-09-13T18:31:48.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.158+0000 7ff042ff5640 1 -- 172.21.15.114:0/3458126529 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0540dc950 con 0x560471d0d290 2024-09-13T18:31:48.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.158+0000 7ff067e7d740 10 client.27605 fetch_fsmap learned FSMap version 34 2024-09-13T18:31:48.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.158+0000 7ff067e7d740 1 -- 172.21.15.114:0/3458126529 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x560471c6e3c0 con 0x560471d0d290 2024-09-13T18:31:48.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.159+0000 7ff042ff5640 1 -- 172.21.15.114:0/3458126529 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff0540b8080 con 0x560471d0d290 2024-09-13T18:31:48.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.159+0000 7ff042ff5640 1 -- 172.21.15.114:0/3458126529 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff0540a7d30 con 0x560471d0d290 2024-09-13T18:31:48.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.160+0000 7ff067e7d740 10 client.27605 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:31:48.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.160+0000 7ff067e7d740 10 client.27605 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-13T18:31:48.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.160+0000 7ff067e7d740 20 client.27605 populate_metadata read hostname 'smithi114' 2024-09-13T18:31:48.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.160+0000 7ff067e7d740 1 --2- 172.21.15.114:0/3458126529 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x560471d27450 0x560471d29870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:48.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.160+0000 7ff067e7d740 4 client.27605 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:31:48.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.160+0000 7ff067e7d740 1 -- 172.21.15.114:0/3458126529 --> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560471c621f0 con 0x560471d27450 2024-09-13T18:31:48.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.161+0000 7ff05ed9d640 1 --2- 172.21.15.114:0/3458126529 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x560471d27450 0x560471d29870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:48.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.164+0000 7ff05ed9d640 1 --2- 172.21.15.114:0/3458126529 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x560471d27450 0x560471d29870 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=957e39bfce0c5663 server_cookie=5739fe41e3e0a526 in_seq=0 out_seq=0 2024-09-13T18:31:48.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.165+0000 7ff042ff5640 10 client.27605 ms_handle_connect on v2:172.21.15.134:6836/891692210 2024-09-13T18:31:48.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.165+0000 7ff042ff5640 1 -- 172.21.15.114:0/3458126529 <== mds.1 v2:172.21.15.134:6836/891692210 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x560471c621f0 con 0x560471d27450 2024-09-13T18:31:48.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.165+0000 7ff042ff5640 10 client.27605 handle_command_reply: tid=0 2024-09-13T18:31:48.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.166+0000 7ff067e7d740 1 client.27605 shutdown 2024-09-13T18:31:48.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.171+0000 7ff067e7d740 1 -- 172.21.15.114:0/3458126529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560471d0d290 msgr2=0x560471d144e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:48.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.171+0000 7ff067e7d740 1 --2- 172.21.15.114:0/3458126529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560471d0d290 0x560471d144e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff054000c00 tx=0x7ff0540afd80 comp rx=0 tx=0).stop 2024-09-13T18:31:48.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.172+0000 7ff067e7d740 1 -- 172.21.15.114:0/3458126529 shutdown_connections 2024-09-13T18:31:48.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.172+0000 7ff067e7d740 1 --2- 172.21.15.114:0/3458126529 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x560471d27450 0x560471d29870 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.172+0000 7ff067e7d740 1 --2- 172.21.15.114:0/3458126529 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560471d17b30 0x560471d14a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.172+0000 7ff067e7d740 1 --2- 172.21.15.114:0/3458126529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560471d0d290 0x560471d144e0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.172+0000 7ff067e7d740 1 --2- 172.21.15.114:0/3458126529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560471d0c970 0x560471d17150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.172+0000 7ff042ff5640 0 client.27605 ms_handle_reset on v2:172.21.15.134:6836/891692210 2024-09-13T18:31:48.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.172+0000 7ff067e7d740 1 -- 172.21.15.114:0/3458126529 >> 172.21.15.114:0/3458126529 conn(0x560471d08ee0 msgr2=0x560471d08230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:48.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.183+0000 7ff067e7d740 1 -- 172.21.15.114:0/3458126529 shutdown_connections 2024-09-13T18:31:48.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.185+0000 7ff067e7d740 1 -- 172.21.15.114:0/3458126529 wait complete. 2024-09-13T18:31:48.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:47 smithi204 ceph-mon[34777]: pgmap v288: 129 pgs: 129 active+clean; 22 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 37 MiB/s wr, 824 op/s 2024-09-13T18:31:48.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.193+0000 7ff067e7d740 20 client.27605 trim_cache size 0 max 16384 2024-09-13T18:31:48.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:47 smithi114 ceph-mon[30897]: pgmap v288: 129 pgs: 129 active+clean; 22 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 37 MiB/s wr, 824 op/s 2024-09-13T18:31:48.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.219+0000 7ff067e7d740 1 Processor -- start 2024-09-13T18:31:48.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.219+0000 7ff067e7d740 1 -- start start 2024-09-13T18:31:48.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.220+0000 7ff067e7d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560471d0c970 0x560471d1e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:48.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.220+0000 7ff067e7d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560471d0d290 0x560471d1ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:48.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.220+0000 7ff067e7d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560471d17b30 0x560471d18f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:48.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.220+0000 7ff05ed9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560471d0c970 0x560471d1e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:48.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.220+0000 7ff05ed9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560471d0c970 0x560471d1e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:41548/0 (socket says 172.21.15.114:41548) 2024-09-13T18:31:48.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.221+0000 7ff05f59e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560471d17b30 0x560471d18f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:48.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.221+0000 7ff05f59e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560471d17b30 0x560471d18f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:58660/0 (socket says 172.21.15.114:58660) 2024-09-13T18:31:48.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.221+0000 7ff05ed9d640 1 -- 172.21.15.114:0/1722619227 learned_addr learned my addr 172.21.15.114:0/1722619227 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:48.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.221+0000 7ff067e7d740 1 -- 172.21.15.114:0/1722619227 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x560471d2e870 con 0x560471d17b30 2024-09-13T18:31:48.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.221+0000 7ff067e7d740 1 -- 172.21.15.114:0/1722619227 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560471c621f0 con 0x560471d0d290 2024-09-13T18:31:48.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.221+0000 7ff067e7d740 1 -- 172.21.15.114:0/1722619227 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560471d15640 con 0x560471d0c970 2024-09-13T18:31:48.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.222+0000 7ff05f59e640 1 -- 172.21.15.114:0/1722619227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560471d0c970 msgr2=0x560471d1e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:48.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.222+0000 7ff05f59e640 1 --2- 172.21.15.114:0/1722619227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560471d0c970 0x560471d1e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.222+0000 7ff05f59e640 1 -- 172.21.15.114:0/1722619227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560471d0d290 msgr2=0x560471d1ed90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:31:48.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.222+0000 7ff05f59e640 1 --2- 172.21.15.114:0/1722619227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560471d0d290 0x560471d1ed90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.222+0000 7ff05f59e640 1 -- 172.21.15.114:0/1722619227 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560471c10770 con 0x560471d17b30 2024-09-13T18:31:48.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.222+0000 7ff05f59e640 1 --2- 172.21.15.114:0/1722619227 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560471d17b30 0x560471d18f20 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7ff050006010 tx=0x7ff05003e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11626baae38bf3ea server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:48.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.223+0000 7ff045ffb640 1 -- 172.21.15.114:0/1722619227 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff050003d40 con 0x560471d17b30 2024-09-13T18:31:48.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.223+0000 7ff045ffb640 1 -- 172.21.15.114:0/1722619227 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff050031970 con 0x560471d17b30 2024-09-13T18:31:48.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.224+0000 7ff045ffb640 1 -- 172.21.15.114:0/1722619227 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff05004a7c0 con 0x560471d17b30 2024-09-13T18:31:48.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.225+0000 7ff067e7d740 1 -- 172.21.15.114:0/1722619227 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560471d17b30 msgr2=0x560471d18f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:48.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.225+0000 7ff067e7d740 1 --2- 172.21.15.114:0/1722619227 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560471d17b30 0x560471d18f20 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7ff050006010 tx=0x7ff05003e000 comp rx=0 tx=0).stop 2024-09-13T18:31:48.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.225+0000 7ff067e7d740 1 -- 172.21.15.114:0/1722619227 shutdown_connections 2024-09-13T18:31:48.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.225+0000 7ff067e7d740 1 --2- 172.21.15.114:0/1722619227 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560471d17b30 0x560471d18f20 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.225+0000 7ff067e7d740 1 --2- 172.21.15.114:0/1722619227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560471d0d290 0x560471d1ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.225+0000 7ff067e7d740 1 --2- 172.21.15.114:0/1722619227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560471d0c970 0x560471d1e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.225+0000 7ff067e7d740 1 -- 172.21.15.114:0/1722619227 >> 172.21.15.114:0/1722619227 conn(0x560471d08ee0 msgr2=0x560471d12db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:48.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.226+0000 7ff067e7d740 1 -- 172.21.15.114:0/1722619227 shutdown_connections 2024-09-13T18:31:48.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.232+0000 7ff067e7d740 1 -- 172.21.15.114:0/1722619227 wait complete. 2024-09-13T18:31:48.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.232+0000 7ff067e7d740 1 -- start start 2024-09-13T18:31:48.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.232+0000 7ff067e7d740 1 Processor -- start 2024-09-13T18:31:48.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.234+0000 7ff067e7d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560471d0c970 0x560471d77c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:48.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.235+0000 7ff067e7d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560471d0d290 0x560471d781b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:48.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.235+0000 7ff067e7d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560471d17b30 0x560471d1ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:48.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.235+0000 7ff067e7d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x560471d1c140 con 0x560471d0d290 2024-09-13T18:31:48.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.235+0000 7ff067e7d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560471d1c2b0 con 0x560471d17b30 2024-09-13T18:31:48.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.235+0000 7ff067e7d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560471d1c470 con 0x560471d0c970 2024-09-13T18:31:48.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.235+0000 7ff067e7d740 10 client.0 _ll_register_callbacks cb 0x7ff05800a990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:31:48.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.235+0000 7ff05e59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560471d0d290 0x560471d781b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:48.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.235+0000 7ff05e59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560471d0d290 0x560471d781b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:58662/0 (socket says 172.21.15.114:58662) 2024-09-13T18:31:48.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.235+0000 7ff05e59c640 1 -- 172.21.15.114:0/1557001972 learned_addr learned my addr 172.21.15.114:0/1557001972 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:48.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.235+0000 7ff05ed9d640 1 --2- 172.21.15.114:0/1557001972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560471d0c970 0x560471d77c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:48.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.236+0000 7ff05ed9d640 1 -- 172.21.15.114:0/1557001972 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560471d17b30 msgr2=0x560471d1ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:31:48.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.236+0000 7ff05ed9d640 1 --2- 172.21.15.114:0/1557001972 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560471d17b30 0x560471d1ba40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.236+0000 7ff05ed9d640 1 -- 172.21.15.114:0/1557001972 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560471d0d290 msgr2=0x560471d781b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:48.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.236+0000 7ff05ed9d640 1 --2- 172.21.15.114:0/1557001972 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560471d0d290 0x560471d781b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.236+0000 7ff05ed9d640 1 -- 172.21.15.114:0/1557001972 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff06011c1a0 con 0x560471d0c970 2024-09-13T18:31:48.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.236+0000 7ff05ed9d640 1 --2- 172.21.15.114:0/1557001972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560471d0c970 0x560471d77c70 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff058000f80 tx=0x7ff058004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33c3564d6ba1534c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:48.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.237+0000 7ff042ff5640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:31:48.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.238+0000 7ff042ff5640 1 -- 172.21.15.114:0/1557001972 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff05802c070 con 0x560471d0c970 2024-09-13T18:31:48.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.238+0000 7ff042ff5640 1 -- 172.21.15.114:0/1557001972 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ff058003970 con 0x560471d0c970 2024-09-13T18:31:48.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.240+0000 7ff067e7d740 1 -- 172.21.15.114:0/1557001972 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff058018440 con 0x560471d0c970 2024-09-13T18:31:48.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.240+0000 7ff042ff5640 1 -- 172.21.15.114:0/1557001972 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff058027ea0 con 0x560471d0c970 2024-09-13T18:31:48.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.247+0000 7ff067e7d740 10 client.27574 fetch_fsmap learned FSMap version 34 2024-09-13T18:31:48.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.247+0000 7ff067e7d740 1 -- 172.21.15.114:0/1557001972 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x560471c35e80 con 0x560471d0c970 2024-09-13T18:31:48.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.249+0000 7ff042ff5640 1 -- 172.21.15.114:0/1557001972 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff058018440 con 0x560471d0c970 2024-09-13T18:31:48.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.249+0000 7ff042ff5640 1 -- 172.21.15.114:0/1557001972 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff05802f020 con 0x560471d0c970 2024-09-13T18:31:48.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.249+0000 7ff067e7d740 10 client.27574 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:31:48.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.249+0000 7ff067e7d740 10 client.27574 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-13T18:31:48.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.249+0000 7ff067e7d740 20 client.27574 populate_metadata read hostname 'smithi114' 2024-09-13T18:31:48.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.249+0000 7ff067e7d740 1 --2- 172.21.15.114:0/1557001972 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x560471d7fa60 0x560471d81e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:48.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.249+0000 7ff067e7d740 4 client.27574 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:31:48.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.249+0000 7ff067e7d740 1 -- 172.21.15.114:0/1557001972 --> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5604719634e0 con 0x560471d7fa60 2024-09-13T18:31:48.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.250+0000 7ff05e59c640 1 --2- 172.21.15.114:0/1557001972 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x560471d7fa60 0x560471d81e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:48.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.261+0000 7ff05e59c640 1 --2- 172.21.15.114:0/1557001972 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x560471d7fa60 0x560471d81e80 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c877d26cc353b6fb server_cookie=6b82e1078cfd9931 in_seq=0 out_seq=0 2024-09-13T18:31:48.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.263+0000 7ff042ff5640 10 client.27574 ms_handle_connect on v2:172.21.15.134:6836/891692210 2024-09-13T18:31:48.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.264+0000 7ff042ff5640 1 -- 172.21.15.114:0/1557001972 <== mds.1 v2:172.21.15.134:6836/891692210 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5604719634e0 con 0x560471d7fa60 2024-09-13T18:31:48.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.264+0000 7ff042ff5640 10 client.27574 handle_command_reply: tid=0 2024-09-13T18:31:48.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.264+0000 7ff067e7d740 1 client.27574 shutdown 2024-09-13T18:31:48.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.268+0000 7ff067e7d740 1 -- 172.21.15.114:0/1557001972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560471d0c970 msgr2=0x560471d77c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:48.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.268+0000 7ff067e7d740 1 --2- 172.21.15.114:0/1557001972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560471d0c970 0x560471d77c70 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff058000f80 tx=0x7ff058004b40 comp rx=0 tx=0).stop 2024-09-13T18:31:48.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.269+0000 7ff067e7d740 1 -- 172.21.15.114:0/1557001972 shutdown_connections 2024-09-13T18:31:48.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.269+0000 7ff067e7d740 1 --2- 172.21.15.114:0/1557001972 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x560471d7fa60 0x560471d81e80 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.269+0000 7ff067e7d740 1 --2- 172.21.15.114:0/1557001972 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560471d17b30 0x560471d1ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.269+0000 7ff067e7d740 1 --2- 172.21.15.114:0/1557001972 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560471d0d290 0x560471d781b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.269+0000 7ff067e7d740 1 --2- 172.21.15.114:0/1557001972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560471d0c970 0x560471d77c70 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.269+0000 7ff067e7d740 1 -- 172.21.15.114:0/1557001972 >> 172.21.15.114:0/1557001972 conn(0x560471d08ee0 msgr2=0x560471d07a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:48.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.269+0000 7ff042ff5640 0 client.27574 ms_handle_reset on v2:172.21.15.134:6836/891692210 2024-09-13T18:31:48.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.272+0000 7ff067e7d740 1 -- 172.21.15.114:0/1557001972 shutdown_connections 2024-09-13T18:31:48.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.272+0000 7ff067e7d740 1 -- 172.21.15.114:0/1557001972 wait complete. 2024-09-13T18:31:48.275 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:31:48.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.272+0000 7ff067e7d740 20 client.27574 trim_cache size 0 max 16384 2024-09-13T18:31:48.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.277+0000 7ff065080640 1 -- 172.21.15.114:0/3006257167 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff0480a13d0 msgr2=0x7ff0480a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:48.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.277+0000 7ff065080640 1 --2- 172.21.15.114:0/3006257167 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff0480a13d0 0x7ff0480a3890 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff050005ec0 tx=0x7ff050005e50 comp rx=0 tx=0).stop 2024-09-13T18:31:48.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.277+0000 7ff065080640 1 -- 172.21.15.114:0/3006257167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff060107890 msgr2=0x7ff0601150b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:48.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.277+0000 7ff065080640 1 --2- 172.21.15.114:0/3006257167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff060107890 0x7ff0601150b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff05401fa90 tx=0x7ff05401ff50 comp rx=0 tx=0).stop 2024-09-13T18:31:48.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.282+0000 7ff065080640 1 -- 172.21.15.114:0/3006257167 shutdown_connections 2024-09-13T18:31:48.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.282+0000 7ff065080640 1 --2- 172.21.15.114:0/3006257167 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff0480a13d0 0x7ff0480a3890 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.282+0000 7ff065080640 1 --2- 172.21.15.114:0/3006257167 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff060119400 0x7ff0601a1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.282+0000 7ff065080640 1 --2- 172.21.15.114:0/3006257167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff060107890 0x7ff0601150b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.282+0000 7ff065080640 1 --2- 172.21.15.114:0/3006257167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0601074b0 0x7ff060114b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.282+0000 7ff065080640 1 -- 172.21.15.114:0/3006257167 >> 172.21.15.114:0/3006257167 conn(0x7ff06006af70 msgr2=0x7ff06006ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:48.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.288+0000 7ff065080640 1 -- 172.21.15.114:0/3006257167 shutdown_connections 2024-09-13T18:31:48.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.288+0000 7ff065080640 1 -- 172.21.15.114:0/3006257167 wait complete. 2024-09-13T18:31:48.319 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-09-13T18:31:48.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.523+0000 7fcb0a1b5640 1 -- 172.21.15.114:0/3153810004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcafc0a0f90 msgr2=0x7fcafc0a8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:48.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.523+0000 7fcb0a1b5640 1 --2- 172.21.15.114:0/3153810004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcafc0a0f90 0x7fcafc0a8ae0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8009860 tx=0x7fcaf802a380 comp rx=0 tx=0).stop 2024-09-13T18:31:48.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.526+0000 7fcb0a1b5640 1 -- 172.21.15.114:0/3153810004 shutdown_connections 2024-09-13T18:31:48.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.526+0000 7fcb0a1b5640 1 --2- 172.21.15.114:0/3153810004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcafc0a0f90 0x7fcafc0a8ae0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.526+0000 7fcb0a1b5640 1 --2- 172.21.15.114:0/3153810004 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcafc09c7f0 0x7fcafc0a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.526+0000 7fcb0a1b5640 1 --2- 172.21.15.114:0/3153810004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcafc09be40 0x7fcafc09c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.526+0000 7fcb0a1b5640 1 -- 172.21.15.114:0/3153810004 >> 172.21.15.114:0/3153810004 conn(0x7fcafc00c250 msgr2=0x7fcafc09a180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:48.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.526+0000 7fcb0a1b5640 1 -- 172.21.15.114:0/3153810004 shutdown_connections 2024-09-13T18:31:48.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.530+0000 7fcb0a1b5640 1 -- 172.21.15.114:0/3153810004 wait complete. 2024-09-13T18:31:48.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.531+0000 7fcb0a1b5640 1 Processor -- start 2024-09-13T18:31:48.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.531+0000 7fcb0a1b5640 1 -- start start 2024-09-13T18:31:48.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.531+0000 7fcb0a1b5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcafc09be40 0x7fcafc0158e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:48.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.532+0000 7fcb0a1b5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcafc0a0f90 0x7fcafc015e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:48.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.532+0000 7fcb0a1b5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcafc00eac0 0x7fcafc00ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:48.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.532+0000 7fcb0a1b5640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcafc00f700 con 0x7fcafc09be40 2024-09-13T18:31:48.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.532+0000 7fcb0a1b5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcafc013470 con 0x7fcafc0a0f90 2024-09-13T18:31:48.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.532+0000 7fcb0a1b5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcafc013630 con 0x7fcafc00eac0 2024-09-13T18:31:48.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.532+0000 7fcb037fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcafc09be40 0x7fcafc0158e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:48.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.532+0000 7fcb037fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcafc09be40 0x7fcafc0158e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:58676/0 (socket says 172.21.15.114:58676) 2024-09-13T18:31:48.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.532+0000 7fcb037fe640 1 -- 172.21.15.114:0/2034790297 learned_addr learned my addr 172.21.15.114:0/2034790297 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:48.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.532+0000 7fcb02ffd640 1 --2- 172.21.15.114:0/2034790297 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcafc0a0f90 0x7fcafc015e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:48.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.532+0000 7fcb03fff640 1 --2- 172.21.15.114:0/2034790297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcafc00eac0 0x7fcafc00ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:48.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.533+0000 7fcb037fe640 1 -- 172.21.15.114:0/2034790297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcafc00eac0 msgr2=0x7fcafc00ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:48.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.533+0000 7fcb037fe640 1 --2- 172.21.15.114:0/2034790297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcafc00eac0 0x7fcafc00ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.533+0000 7fcb037fe640 1 -- 172.21.15.114:0/2034790297 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcafc0a0f90 msgr2=0x7fcafc015e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:48.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.533+0000 7fcb037fe640 1 --2- 172.21.15.114:0/2034790297 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcafc0a0f90 0x7fcafc015e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.533+0000 7fcb037fe640 1 -- 172.21.15.114:0/2034790297 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf8009490 con 0x7fcafc09be40 2024-09-13T18:31:48.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.533+0000 7fcb037fe640 1 --2- 172.21.15.114:0/2034790297 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcafc09be40 0x7fcafc0158e0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fcaf000b4a0 tx=0x7fcaf000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:48.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.539+0000 7fcb00ff9640 1 -- 172.21.15.114:0/2034790297 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf0010610 con 0x7fcafc09be40 2024-09-13T18:31:48.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.539+0000 7fcb00ff9640 1 -- 172.21.15.114:0/2034790297 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcaf0010c00 con 0x7fcafc09be40 2024-09-13T18:31:48.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.539+0000 7fcb00ff9640 1 -- 172.21.15.114:0/2034790297 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf0017ca0 con 0x7fcafc09be40 2024-09-13T18:31:48.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.539+0000 7fcb0a1b5640 1 -- 172.21.15.114:0/2034790297 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcafc013910 con 0x7fcafc09be40 2024-09-13T18:31:48.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.552+0000 7fcb0a1b5640 1 -- 172.21.15.114:0/2034790297 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcafc013da0 con 0x7fcafc09be40 2024-09-13T18:31:48.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.574+0000 7fcb0ced4740 1 Processor -- start 2024-09-13T18:31:48.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.575+0000 7fcb00ff9640 1 -- 172.21.15.114:0/2034790297 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fcaf0010770 con 0x7fcafc09be40 2024-09-13T18:31:48.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.576+0000 7fcb00ff9640 1 --2- 172.21.15.114:0/2034790297 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fcaf40a1410 0x7fcaf40a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:48.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.576+0000 7fcb00ff9640 1 -- 172.21.15.114:0/2034790297 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fcaf00cc810 con 0x7fcafc09be40 2024-09-13T18:31:48.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.577+0000 7fcb0ced4740 1 -- start start 2024-09-13T18:31:48.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.577+0000 7fcb0ced4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55932e792970 0x55932e792d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:48.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.577+0000 7fcb0ced4740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55932e793290 0x55932e79d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:48.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.577+0000 7fcb0ced4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55932e79db30 0x55932e79ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:48.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.577+0000 7fcb0ced4740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55932e6fccc0 con 0x55932e793290 2024-09-13T18:31:48.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.577+0000 7fcb0ced4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55932e6fa450 con 0x55932e792970 2024-09-13T18:31:48.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.577+0000 7fcb0ced4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55932e6f5d40 con 0x55932e79db30 2024-09-13T18:31:48.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.577+0000 7fcb037fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55932e792970 0x55932e792d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:48.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.577+0000 7fcb037fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55932e792970 0x55932e792d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:46272/0 (socket says 172.21.15.114:46272) 2024-09-13T18:31:48.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.577+0000 7fcb037fe640 1 -- 172.21.15.114:0/1887375397 learned_addr learned my addr 172.21.15.114:0/1887375397 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:48.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.578+0000 7fcb03fff640 1 --2- 172.21.15.114:0/2034790297 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fcaf40a1410 0x7fcaf40a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:48.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.581+0000 7fcb02ffd640 1 --2- 172.21.15.114:0/1887375397 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55932e793290 0x55932e79d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:48.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.582+0000 7fcb03fff640 1 --2- 172.21.15.114:0/1887375397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55932e79db30 0x55932e79ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:48.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.582+0000 7fcb037fe640 1 -- 172.21.15.114:0/1887375397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55932e79db30 msgr2=0x55932e79ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:48.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.583+0000 7fcb037fe640 1 --2- 172.21.15.114:0/1887375397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55932e79db30 0x55932e79ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.583+0000 7fcb037fe640 1 -- 172.21.15.114:0/1887375397 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55932e793290 msgr2=0x55932e79d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:48.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.583+0000 7fcb037fe640 1 --2- 172.21.15.114:0/1887375397 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55932e793290 0x55932e79d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.583+0000 7fcb037fe640 1 -- 172.21.15.114:0/1887375397 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55932e6bd620 con 0x55932e792970 2024-09-13T18:31:48.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.583+0000 7fcb03fff640 1 --2- 172.21.15.114:0/1887375397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55932e79db30 0x55932e79ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:31:48.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.583+0000 7fcb037fe640 1 --2- 172.21.15.114:0/1887375397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55932e792970 0x55932e792d50 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fcaf00a1580 tx=0x7fcaf00047d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a177fb8812680021 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:48.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.585+0000 7fcaedffb640 1 -- 172.21.15.114:0/1887375397 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf0010a20 con 0x55932e792970 2024-09-13T18:31:48.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.585+0000 7fcaedffb640 1 -- 172.21.15.114:0/1887375397 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcaf00996e0 con 0x55932e792970 2024-09-13T18:31:48.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.585+0000 7fcaedffb640 1 -- 172.21.15.114:0/1887375397 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf009d120 con 0x55932e792970 2024-09-13T18:31:48.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.586+0000 7fcb0ced4740 1 -- 172.21.15.114:0/1887375397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55932e792970 msgr2=0x55932e792d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:48.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.586+0000 7fcb0ced4740 1 --2- 172.21.15.114:0/1887375397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55932e792970 0x55932e792d50 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fcaf00a1580 tx=0x7fcaf00047d0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.587+0000 7fcb03fff640 1 --2- 172.21.15.114:0/2034790297 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fcaf40a1410 0x7fcaf40a38d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8009460 tx=0x7fcaf8032d80 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:48.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.587+0000 7fcb0ced4740 1 -- 172.21.15.114:0/1887375397 shutdown_connections 2024-09-13T18:31:48.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.587+0000 7fcb0ced4740 1 --2- 172.21.15.114:0/1887375397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55932e79db30 0x55932e79ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.587+0000 7fcb0ced4740 1 --2- 172.21.15.114:0/1887375397 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55932e793290 0x55932e79d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.587+0000 7fcb0ced4740 1 --2- 172.21.15.114:0/1887375397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55932e792970 0x55932e792d50 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.587+0000 7fcb0ced4740 1 -- 172.21.15.114:0/1887375397 >> 172.21.15.114:0/1887375397 conn(0x55932e78eee0 msgr2=0x55932e78f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:48.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.593+0000 7fcb0ced4740 1 -- 172.21.15.114:0/1887375397 shutdown_connections 2024-09-13T18:31:48.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.593+0000 7fcb0ced4740 1 -- 172.21.15.114:0/1887375397 wait complete. 2024-09-13T18:31:48.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.593+0000 7fcb0ced4740 1 -- start start 2024-09-13T18:31:48.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.593+0000 7fcb0ced4740 1 Processor -- start 2024-09-13T18:31:48.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.597+0000 7fcb0ced4740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55932e792970 0x55932e79d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:48.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.597+0000 7fcb0ced4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55932e793290 0x55932e79a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:48.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.597+0000 7fcb0ced4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55932e79db30 0x55932e79aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:48.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.597+0000 7fcb0ced4740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55932e6b4180 con 0x55932e792970 2024-09-13T18:31:48.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.597+0000 7fcb0ced4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55932e3ce430 con 0x55932e793290 2024-09-13T18:31:48.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.597+0000 7fcb0ced4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55932e6795e0 con 0x55932e79db30 2024-09-13T18:31:48.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.597+0000 7fcb0ced4740 10 client.0 _ll_register_callbacks cb 0x55932e6e5aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:31:48.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.598+0000 7fcb03fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55932e79db30 0x55932e79aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:48.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.598+0000 7fcb03fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55932e79db30 0x55932e79aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:41586/0 (socket says 172.21.15.114:41586) 2024-09-13T18:31:48.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.598+0000 7fcb03fff640 1 -- 172.21.15.114:0/3525105585 learned_addr learned my addr 172.21.15.114:0/3525105585 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:48.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.598+0000 7fcb037fe640 1 --2- 172.21.15.114:0/3525105585 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55932e792970 0x55932e79d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:48.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.598+0000 7fcb02ffd640 1 --2- 172.21.15.114:0/3525105585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55932e793290 0x55932e79a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:48.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.598+0000 7fcb03fff640 1 -- 172.21.15.114:0/3525105585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55932e793290 msgr2=0x55932e79a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:48.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.598+0000 7fcb03fff640 1 --2- 172.21.15.114:0/3525105585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55932e793290 0x55932e79a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.598+0000 7fcb03fff640 1 -- 172.21.15.114:0/3525105585 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55932e792970 msgr2=0x55932e79d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:48.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.598+0000 7fcb03fff640 1 --2- 172.21.15.114:0/3525105585 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55932e792970 0x55932e79d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.598+0000 7fcb03fff640 1 -- 172.21.15.114:0/3525105585 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcafc013da0 con 0x55932e79db30 2024-09-13T18:31:48.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.599+0000 7fcb03fff640 1 --2- 172.21.15.114:0/3525105585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55932e79db30 0x55932e79aa30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8009860 tx=0x7fcaf8005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f962657ef06525a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:48.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.599+0000 7fcae2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:31:48.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.599+0000 7fcae2ffd640 1 -- 172.21.15.114:0/3525105585 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf8004450 con 0x55932e79db30 2024-09-13T18:31:48.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.599+0000 7fcae2ffd640 1 -- 172.21.15.114:0/3525105585 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fcaf8047070 con 0x55932e79db30 2024-09-13T18:31:48.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.599+0000 7fcb0ced4740 1 -- 172.21.15.114:0/3525105585 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55932e71fb80 con 0x55932e79db30 2024-09-13T18:31:48.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.602+0000 7fcae2ffd640 1 -- 172.21.15.114:0/3525105585 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcaf80308e0 con 0x55932e79db30 2024-09-13T18:31:48.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.603+0000 7fcb0ced4740 10 client.27580 fetch_fsmap learned FSMap version 34 2024-09-13T18:31:48.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.603+0000 7fcb0ced4740 1 -- 172.21.15.114:0/3525105585 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55932e6f43c0 con 0x55932e79db30 2024-09-13T18:31:48.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.609+0000 7fcae2ffd640 1 -- 172.21.15.114:0/3525105585 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55932e71fb80 con 0x55932e79db30 2024-09-13T18:31:48.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.610+0000 7fcae2ffd640 1 -- 172.21.15.114:0/3525105585 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcaf8004b40 con 0x55932e79db30 2024-09-13T18:31:48.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.610+0000 7fcb0ced4740 10 client.27580 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:31:48.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.610+0000 7fcb0ced4740 10 client.27580 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:31:48.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.610+0000 7fcb0ced4740 20 client.27580 populate_metadata read hostname 'smithi114' 2024-09-13T18:31:48.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.610+0000 7fcb0ced4740 1 --2- 172.21.15.114:0/3525105585 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55932e7ad5b0 0x55932e7af9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:48.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.610+0000 7fcb0ced4740 4 client.27580 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:31:48.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.610+0000 7fcb0ced4740 1 -- 172.21.15.114:0/3525105585 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55932e6e81f0 con 0x55932e7ad5b0 2024-09-13T18:31:48.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.613+0000 7fcb037fe640 1 --2- 172.21.15.114:0/3525105585 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55932e7ad5b0 0x55932e7af9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:48.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.618+0000 7fcb037fe640 1 --2- 172.21.15.114:0/3525105585 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55932e7ad5b0 0x55932e7af9d0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea94a1fbddde899a server_cookie=a1ecc094f4fe5bb5 in_seq=0 out_seq=0 2024-09-13T18:31:48.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.618+0000 7fcae2ffd640 10 client.27580 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:31:48.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.621+0000 7fcae2ffd640 1 -- 172.21.15.114:0/3525105585 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55932e6e81f0 con 0x55932e7ad5b0 2024-09-13T18:31:48.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.621+0000 7fcae2ffd640 10 client.27580 handle_command_reply: tid=0 2024-09-13T18:31:48.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.621+0000 7fcb0ced4740 1 client.27580 shutdown 2024-09-13T18:31:48.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.621+0000 7fcb0ced4740 1 -- 172.21.15.114:0/3525105585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55932e79db30 msgr2=0x55932e79aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:48.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.621+0000 7fcb0ced4740 1 --2- 172.21.15.114:0/3525105585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55932e79db30 0x55932e79aa30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8009860 tx=0x7fcaf8005f90 comp rx=0 tx=0).stop 2024-09-13T18:31:48.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.622+0000 7fcb0ced4740 1 -- 172.21.15.114:0/3525105585 shutdown_connections 2024-09-13T18:31:48.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.622+0000 7fcb0ced4740 1 --2- 172.21.15.114:0/3525105585 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55932e7ad5b0 0x55932e7af9d0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.622+0000 7fcb0ced4740 1 --2- 172.21.15.114:0/3525105585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55932e79db30 0x55932e79aa30 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.622+0000 7fcb0ced4740 1 --2- 172.21.15.114:0/3525105585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55932e793290 0x55932e79a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.622+0000 7fcb0ced4740 1 --2- 172.21.15.114:0/3525105585 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55932e792970 0x55932e79d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.622+0000 7fcb0ced4740 1 -- 172.21.15.114:0/3525105585 >> 172.21.15.114:0/3525105585 conn(0x55932e78eee0 msgr2=0x55932e78e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:48.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.625+0000 7fcae2ffd640 0 client.27580 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:31:48.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.630+0000 7fcb0ced4740 1 -- 172.21.15.114:0/3525105585 shutdown_connections 2024-09-13T18:31:48.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.635+0000 7fcb0ced4740 1 -- 172.21.15.114:0/3525105585 wait complete. 2024-09-13T18:31:48.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.636+0000 7fcb0ced4740 20 client.27580 trim_cache size 0 max 16384 2024-09-13T18:31:48.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.669+0000 7fcb0ced4740 1 Processor -- start 2024-09-13T18:31:48.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.674+0000 7fcb0ced4740 1 -- start start 2024-09-13T18:31:48.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.675+0000 7fcb0ced4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55932e78eee0 0x55932e7a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:48.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.675+0000 7fcb0ced4740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55932e792970 0x55932e7a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:48.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.675+0000 7fcb0ced4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55932e793290 0x55932e79ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:48.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.675+0000 7fcb0ced4740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55932e79d760 con 0x55932e792970 2024-09-13T18:31:48.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.675+0000 7fcb0ced4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55932e79b450 con 0x55932e793290 2024-09-13T18:31:48.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.675+0000 7fcb0ced4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55932e79cd70 con 0x55932e78eee0 2024-09-13T18:31:48.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.675+0000 7fcb037fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55932e78eee0 0x55932e7a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:48.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.675+0000 7fcb02ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55932e792970 0x55932e7a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:48.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.675+0000 7fcb037fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55932e78eee0 0x55932e7a47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:41598/0 (socket says 172.21.15.114:41598) 2024-09-13T18:31:48.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.675+0000 7fcb037fe640 1 -- 172.21.15.114:0/1748535831 learned_addr learned my addr 172.21.15.114:0/1748535831 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:48.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.675+0000 7fcb03fff640 1 --2- 172.21.15.114:0/1748535831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55932e793290 0x55932e79ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:48.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.675+0000 7fcb037fe640 1 -- 172.21.15.114:0/1748535831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55932e793290 msgr2=0x55932e79ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:48.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.675+0000 7fcb037fe640 1 --2- 172.21.15.114:0/1748535831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55932e793290 0x55932e79ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.675+0000 7fcb037fe640 1 -- 172.21.15.114:0/1748535831 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55932e792970 msgr2=0x55932e7a4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:48.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.675+0000 7fcb037fe640 1 --2- 172.21.15.114:0/1748535831 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55932e792970 0x55932e7a4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.675+0000 7fcb037fe640 1 -- 172.21.15.114:0/1748535831 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55932e696770 con 0x55932e78eee0 2024-09-13T18:31:48.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.675+0000 7fcb02ffd640 1 --2- 172.21.15.114:0/1748535831 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55932e792970 0x55932e7a4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:31:48.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.676+0000 7fcb037fe640 1 --2- 172.21.15.114:0/1748535831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55932e78eee0 0x55932e7a47f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0009420 tx=0x7fcaf00995c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f79eb863cc4acca server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:48.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.678+0000 7fcaee7fc640 1 -- 172.21.15.114:0/1748535831 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf0097a80 con 0x55932e78eee0 2024-09-13T18:31:48.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.678+0000 7fcaee7fc640 1 -- 172.21.15.114:0/1748535831 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcaf009d120 con 0x55932e78eee0 2024-09-13T18:31:48.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.679+0000 7fcb0ced4740 1 -- 172.21.15.114:0/1748535831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55932e78eee0 msgr2=0x55932e7a47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:48.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.679+0000 7fcb0ced4740 1 --2- 172.21.15.114:0/1748535831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55932e78eee0 0x55932e7a47f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0009420 tx=0x7fcaf00995c0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.679+0000 7fcb0ced4740 1 -- 172.21.15.114:0/1748535831 shutdown_connections 2024-09-13T18:31:48.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.679+0000 7fcb0ced4740 1 --2- 172.21.15.114:0/1748535831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55932e793290 0x55932e79ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.679+0000 7fcb0ced4740 1 --2- 172.21.15.114:0/1748535831 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55932e792970 0x55932e7a4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.679+0000 7fcb0ced4740 1 --2- 172.21.15.114:0/1748535831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55932e78eee0 0x55932e7a47f0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.679+0000 7fcb0ced4740 1 -- 172.21.15.114:0/1748535831 >> 172.21.15.114:0/1748535831 conn(0x55932e79d2e0 msgr2=0x55932e78f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:48.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.682+0000 7fcb0ced4740 1 -- 172.21.15.114:0/1748535831 shutdown_connections 2024-09-13T18:31:48.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.682+0000 7fcb0ced4740 1 -- 172.21.15.114:0/1748535831 wait complete. 2024-09-13T18:31:48.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.682+0000 7fcb0ced4740 1 -- start start 2024-09-13T18:31:48.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.683+0000 7fcb0ced4740 1 Processor -- start 2024-09-13T18:31:48.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.683+0000 7fcb0ced4740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55932e78eee0 0x55932e7a4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:48.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.683+0000 7fcb0ced4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55932e792970 0x55932e796e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:48.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.684+0000 7fcb0ced4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55932e793290 0x55932e7973c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:48.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.684+0000 7fcb02ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55932e792970 0x55932e796e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:48.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.684+0000 7fcb02ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55932e792970 0x55932e796e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:46304/0 (socket says 172.21.15.114:46304) 2024-09-13T18:31:48.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.684+0000 7fcb03fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55932e793290 0x55932e7973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:48.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.684+0000 7fcb03fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55932e793290 0x55932e7973c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:41606/0 (socket says 172.21.15.114:41606) 2024-09-13T18:31:48.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.684+0000 7fcb02ffd640 1 -- 172.21.15.114:0/2521765480 learned_addr learned my addr 172.21.15.114:0/2521765480 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:48.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.684+0000 7fcb0ced4740 1 -- 172.21.15.114:0/2521765480 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55932e7979d0 con 0x55932e78eee0 2024-09-13T18:31:48.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.684+0000 7fcb0ced4740 1 -- 172.21.15.114:0/2521765480 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55932e797b40 con 0x55932e792970 2024-09-13T18:31:48.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.684+0000 7fcb0ced4740 1 -- 172.21.15.114:0/2521765480 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55932e797d00 con 0x55932e793290 2024-09-13T18:31:48.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.684+0000 7fcb0ced4740 10 client.0 _ll_register_callbacks cb 0x7fcaf0095a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:31:48.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.688+0000 7fcb03fff640 1 -- 172.21.15.114:0/2521765480 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55932e792970 msgr2=0x55932e796e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:48.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.688+0000 7fcb03fff640 1 --2- 172.21.15.114:0/2521765480 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55932e792970 0x55932e796e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.688+0000 7fcb03fff640 1 -- 172.21.15.114:0/2521765480 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55932e78eee0 msgr2=0x55932e7a4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:48.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.688+0000 7fcb03fff640 1 --2- 172.21.15.114:0/2521765480 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55932e78eee0 0x55932e7a4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.688+0000 7fcb03fff640 1 -- 172.21.15.114:0/2521765480 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55932e6bd620 con 0x55932e793290 2024-09-13T18:31:48.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.688+0000 7fcb03fff640 1 --2- 172.21.15.114:0/2521765480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55932e793290 0x55932e7973c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8030480 tx=0x7fcaf8004480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5080591ef2e1916 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:48.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.689+0000 7fcae2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:31:48.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.689+0000 7fcae2ffd640 1 -- 172.21.15.114:0/2521765480 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf8003d60 con 0x55932e793290 2024-09-13T18:31:48.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.690+0000 7fcb0ced4740 1 -- 172.21.15.114:0/2521765480 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55932e3e94e0 con 0x55932e793290 2024-09-13T18:31:48.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.690+0000 7fcae2ffd640 1 -- 172.21.15.114:0/2521765480 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fcaf8047040 con 0x55932e793290 2024-09-13T18:31:48.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.690+0000 7fcae2ffd640 1 -- 172.21.15.114:0/2521765480 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcaf8003760 con 0x55932e793290 2024-09-13T18:31:48.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.691+0000 7fcb0ced4740 10 client.27586 fetch_fsmap learned FSMap version 34 2024-09-13T18:31:48.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.691+0000 7fcb0ced4740 1 -- 172.21.15.114:0/2521765480 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55932e6bbe80 con 0x55932e793290 2024-09-13T18:31:48.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.696+0000 7fcae2ffd640 1 -- 172.21.15.114:0/2521765480 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55932e3e94e0 con 0x55932e793290 2024-09-13T18:31:48.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.698+0000 7fcae2ffd640 1 -- 172.21.15.114:0/2521765480 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcaf8004fa0 con 0x55932e793290 2024-09-13T18:31:48.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.701+0000 7fcb0ced4740 10 client.27586 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:31:48.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.701+0000 7fcb0ced4740 10 client.27586 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:31:48.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.701+0000 7fcb0ced4740 20 client.27586 populate_metadata read hostname 'smithi114' 2024-09-13T18:31:48.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.701+0000 7fcb0ced4740 1 --2- 172.21.15.114:0/2521765480 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55932e79db30 0x55932e806d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:48.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.701+0000 7fcb0ced4740 4 client.27586 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:31:48.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.701+0000 7fcb0ced4740 1 -- 172.21.15.114:0/2521765480 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55932e3fa7e0 con 0x55932e79db30 2024-09-13T18:31:48.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.703+0000 7fcb037fe640 1 --2- 172.21.15.114:0/2521765480 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55932e79db30 0x55932e806d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:48.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.706+0000 7fcb037fe640 1 --2- 172.21.15.114:0/2521765480 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55932e79db30 0x55932e806d20 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=166ab687164030ab server_cookie=ea7322d4b01e918c in_seq=0 out_seq=0 2024-09-13T18:31:48.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.706+0000 7fcae2ffd640 10 client.27586 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:31:48.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.709+0000 7fcae2ffd640 1 -- 172.21.15.114:0/2521765480 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55932e3fa7e0 con 0x55932e79db30 2024-09-13T18:31:48.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.709+0000 7fcae2ffd640 10 client.27586 handle_command_reply: tid=0 2024-09-13T18:31:48.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.709+0000 7fcb0ced4740 1 client.27586 shutdown 2024-09-13T18:31:48.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.725+0000 7fcb0ced4740 1 -- 172.21.15.114:0/2521765480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55932e793290 msgr2=0x55932e7973c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:48.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.725+0000 7fcb0ced4740 1 --2- 172.21.15.114:0/2521765480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55932e793290 0x55932e7973c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8030480 tx=0x7fcaf8004480 comp rx=0 tx=0).stop 2024-09-13T18:31:48.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.729+0000 7fcb0ced4740 1 -- 172.21.15.114:0/2521765480 shutdown_connections 2024-09-13T18:31:48.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.729+0000 7fcb0ced4740 1 --2- 172.21.15.114:0/2521765480 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55932e79db30 0x55932e806d20 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.729+0000 7fcb0ced4740 1 --2- 172.21.15.114:0/2521765480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55932e793290 0x55932e7973c0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.729+0000 7fcb0ced4740 1 --2- 172.21.15.114:0/2521765480 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55932e792970 0x55932e796e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.729+0000 7fcb0ced4740 1 --2- 172.21.15.114:0/2521765480 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55932e78eee0 0x55932e7a4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.729+0000 7fcb0ced4740 1 -- 172.21.15.114:0/2521765480 >> 172.21.15.114:0/2521765480 conn(0x55932e79d2e0 msgr2=0x55932e79bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:48.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.730+0000 7fcae2ffd640 0 client.27586 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:31:48.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.730+0000 7fcb0ced4740 1 -- 172.21.15.114:0/2521765480 shutdown_connections 2024-09-13T18:31:48.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.730+0000 7fcb0ced4740 1 -- 172.21.15.114:0/2521765480 wait complete. 2024-09-13T18:31:48.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.732+0000 7fcb0ced4740 20 client.27586 trim_cache size 0 max 16384 2024-09-13T18:31:48.735 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:31:48.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.734+0000 7fcb0a1b5640 1 -- 172.21.15.114:0/2034790297 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fcaf40a1410 msgr2=0x7fcaf40a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:48.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.734+0000 7fcb0a1b5640 1 --2- 172.21.15.114:0/2034790297 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fcaf40a1410 0x7fcaf40a38d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8009460 tx=0x7fcaf8032d80 comp rx=0 tx=0).stop 2024-09-13T18:31:48.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.734+0000 7fcb0a1b5640 1 -- 172.21.15.114:0/2034790297 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcafc09be40 msgr2=0x7fcafc0158e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:48.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.734+0000 7fcb0a1b5640 1 --2- 172.21.15.114:0/2034790297 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcafc09be40 0x7fcafc0158e0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fcaf000b4a0 tx=0x7fcaf000b960 comp rx=0 tx=0).stop 2024-09-13T18:31:48.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.734+0000 7fcb0a1b5640 1 -- 172.21.15.114:0/2034790297 shutdown_connections 2024-09-13T18:31:48.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.734+0000 7fcb0a1b5640 1 --2- 172.21.15.114:0/2034790297 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fcaf40a1410 0x7fcaf40a38d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.734+0000 7fcb0a1b5640 1 --2- 172.21.15.114:0/2034790297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcafc00eac0 0x7fcafc00ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.734+0000 7fcb0a1b5640 1 --2- 172.21.15.114:0/2034790297 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcafc0a0f90 0x7fcafc015e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.734+0000 7fcb0a1b5640 1 --2- 172.21.15.114:0/2034790297 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcafc09be40 0x7fcafc0158e0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:48.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.734+0000 7fcb0a1b5640 1 -- 172.21.15.114:0/2034790297 >> 172.21.15.114:0/2034790297 conn(0x7fcafc00c250 msgr2=0x7fcafc008cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:48.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.734+0000 7fcb0a1b5640 1 -- 172.21.15.114:0/2034790297 shutdown_connections 2024-09-13T18:31:48.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:48.734+0000 7fcb0a1b5640 1 -- 172.21.15.114:0/2034790297 wait complete. 2024-09-13T18:31:49.766 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-13T18:31:49.766 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-09-13T18:31:50.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:49.998+0000 7f46b9202640 1 -- 172.21.15.114:0/343619220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46b4119400 msgr2=0x7f46b411b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:50.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:49.998+0000 7f46b9202640 1 --2- 172.21.15.114:0/343619220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46b4119400 0x7f46b411b7f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f46a8009900 tx=0x7f46a802a420 comp rx=0 tx=0).stop 2024-09-13T18:31:50.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:49.998+0000 7f46b9202640 1 -- 172.21.15.114:0/343619220 shutdown_connections 2024-09-13T18:31:50.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:49.998+0000 7f46b9202640 1 --2- 172.21.15.114:0/343619220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46b4119400 0x7f46b411b7f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:49.998+0000 7f46b9202640 1 --2- 172.21.15.114:0/343619220 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46b4107920 0x7f46b4107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:49.998+0000 7f46b9202640 1 --2- 172.21.15.114:0/343619220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46b4107540 0x7f46b410e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:49.999+0000 7f46b9202640 1 -- 172.21.15.114:0/343619220 >> 172.21.15.114:0/343619220 conn(0x7f46b406afe0 msgr2=0x7f46b406b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:50.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.005+0000 7f46b9202640 1 -- 172.21.15.114:0/343619220 shutdown_connections 2024-09-13T18:31:50.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.007+0000 7f46b9202640 1 -- 172.21.15.114:0/343619220 wait complete. 2024-09-13T18:31:50.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.008+0000 7f46b9202640 1 Processor -- start 2024-09-13T18:31:50.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.008+0000 7f46b9202640 1 -- start start 2024-09-13T18:31:50.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.009+0000 7f46b9202640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46b4107540 0x7f46b41a5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:50.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.009+0000 7f46b9202640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46b4107920 0x7f46b41a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:50.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.009+0000 7f46b9202640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46b4119400 0x7f46b41a99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:50.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.009+0000 7f46b9202640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b41a9fc0 con 0x7f46b4119400 2024-09-13T18:31:50.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.009+0000 7f46b359e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46b4119400 0x7f46b41a99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:50.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.009+0000 7f46b359e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46b4119400 0x7f46b41a99c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:58732/0 (socket says 172.21.15.114:58732) 2024-09-13T18:31:50.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.009+0000 7f46b359e640 1 -- 172.21.15.114:0/1064271286 learned_addr learned my addr 172.21.15.114:0/1064271286 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:50.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.009+0000 7f46b9202640 1 -- 172.21.15.114:0/1064271286 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b41aa130 con 0x7f46b4107540 2024-09-13T18:31:50.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.010+0000 7f46b9202640 1 -- 172.21.15.114:0/1064271286 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b41aa2f0 con 0x7f46b4107920 2024-09-13T18:31:50.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.012+0000 7f46b359e640 1 -- 172.21.15.114:0/1064271286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46b4107920 msgr2=0x7f46b41a55d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:31:50.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.012+0000 7f46b359e640 1 --2- 172.21.15.114:0/1064271286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46b4107920 0x7f46b41a55d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.012+0000 7f46b359e640 1 -- 172.21.15.114:0/1064271286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46b4107540 msgr2=0x7f46b41a5090 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:31:50.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.012+0000 7f46b359e640 1 --2- 172.21.15.114:0/1064271286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46b4107540 0x7f46b41a5090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.012+0000 7f46b359e640 1 -- 172.21.15.114:0/1064271286 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a8009560 con 0x7f46b4119400 2024-09-13T18:31:50.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.013+0000 7f46b359e640 1 --2- 172.21.15.114:0/1064271286 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46b4119400 0x7f46b41a99c0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f46a80057c0 tx=0x7f46a8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:50.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.014+0000 7f469bfff640 1 -- 172.21.15.114:0/1064271286 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a8004c90 con 0x7f46b4119400 2024-09-13T18:31:50.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.014+0000 7f469bfff640 1 -- 172.21.15.114:0/1064271286 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f46a8004df0 con 0x7f46b4119400 2024-09-13T18:31:50.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.014+0000 7f46b9202640 1 -- 172.21.15.114:0/1064271286 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46b41aa570 con 0x7f46b4119400 2024-09-13T18:31:50.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.014+0000 7f46b9202640 1 -- 172.21.15.114:0/1064271286 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46b41aa840 con 0x7f46b4119400 2024-09-13T18:31:50.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.017+0000 7f46bc003740 1 Processor -- start 2024-09-13T18:31:50.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.018+0000 7f469bfff640 1 -- 172.21.15.114:0/1064271286 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a803c900 con 0x7f46b4119400 2024-09-13T18:31:50.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.018+0000 7f469bfff640 1 -- 172.21.15.114:0/1064271286 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f46a803caa0 con 0x7f46b4119400 2024-09-13T18:31:50.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.021+0000 7f469bfff640 1 --2- 172.21.15.114:0/1064271286 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f469c0a11e0 0x7f469c0a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:50.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.022+0000 7f469bfff640 1 -- 172.21.15.114:0/1064271286 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f46a80036d0 con 0x7f46b4119400 2024-09-13T18:31:50.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.024+0000 7f46bc003740 1 -- start start 2024-09-13T18:31:50.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.024+0000 7f46bc003740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e016f1d7c0 0x55e016f1dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:50.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.024+0000 7f46bc003740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e016f1e0e0 0x55e016f28440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:50.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.024+0000 7f46bc003740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e016f28980 0x55e016f2ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:50.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.024+0000 7f46bc003740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e016e88c60 con 0x55e016f1e0e0 2024-09-13T18:31:50.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.024+0000 7f46bc003740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e016e863f0 con 0x55e016f1d7c0 2024-09-13T18:31:50.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.024+0000 7f46bc003740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e016e81ce0 con 0x55e016f28980 2024-09-13T18:31:50.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.025+0000 7f46b359e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e016f28980 0x55e016f2ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:50.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.026+0000 7f46b359e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e016f28980 0x55e016f2ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:41632/0 (socket says 172.21.15.114:41632) 2024-09-13T18:31:50.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.026+0000 7f46b2d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e016f1d7c0 0x55e016f1dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:50.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.026+0000 7f46b359e640 1 -- 172.21.15.114:0/2188545187 learned_addr learned my addr 172.21.15.114:0/2188545187 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:50.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.026+0000 7f46b2d9d640 1 -- 172.21.15.114:0/2188545187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e016f28980 msgr2=0x55e016f2ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:50.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.026+0000 7f46b2d9d640 1 --2- 172.21.15.114:0/2188545187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e016f28980 0x55e016f2ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.026+0000 7f46b2d9d640 1 -- 172.21.15.114:0/2188545187 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e016f1e0e0 msgr2=0x55e016f28440 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:31:50.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.026+0000 7f46b2d9d640 1 --2- 172.21.15.114:0/2188545187 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e016f1e0e0 0x55e016f28440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.026+0000 7f46b2d9d640 1 -- 172.21.15.114:0/2188545187 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e016e3d970 con 0x55e016f1d7c0 2024-09-13T18:31:50.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.027+0000 7f46b2d9d640 1 --2- 172.21.15.114:0/2188545187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e016f1d7c0 0x55e016f1dba0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f46ac009cc0 tx=0x7f46ac00e6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de80987c9f377fdb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:50.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.027+0000 7f46997fa640 1 -- 172.21.15.114:0/2188545187 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ac004b20 con 0x55e016f1d7c0 2024-09-13T18:31:50.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.027+0000 7f46997fa640 1 -- 172.21.15.114:0/2188545187 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f46ac003e00 con 0x55e016f1d7c0 2024-09-13T18:31:50.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.028+0000 7f46bc003740 1 -- 172.21.15.114:0/2188545187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e016f1d7c0 msgr2=0x55e016f1dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:50.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.028+0000 7f46bc003740 1 --2- 172.21.15.114:0/2188545187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e016f1d7c0 0x55e016f1dba0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f46ac009cc0 tx=0x7f46ac00e6c0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.029+0000 7f46bc003740 1 -- 172.21.15.114:0/2188545187 shutdown_connections 2024-09-13T18:31:50.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.029+0000 7f46bc003740 1 --2- 172.21.15.114:0/2188545187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e016f28980 0x55e016f2ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.029+0000 7f46bc003740 1 --2- 172.21.15.114:0/2188545187 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e016f1e0e0 0x55e016f28440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.029+0000 7f46bc003740 1 --2- 172.21.15.114:0/2188545187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e016f1d7c0 0x55e016f1dba0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.029+0000 7f46bc003740 1 -- 172.21.15.114:0/2188545187 >> 172.21.15.114:0/2188545187 conn(0x55e016f19d50 msgr2=0x55e016f1a130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:50.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.032+0000 7f46b259c640 1 --2- 172.21.15.114:0/1064271286 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f469c0a11e0 0x7f469c0a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:50.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.033+0000 7f46bc003740 1 -- 172.21.15.114:0/2188545187 shutdown_connections 2024-09-13T18:31:50.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.034+0000 7f46bc003740 1 -- 172.21.15.114:0/2188545187 wait complete. 2024-09-13T18:31:50.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.035+0000 7f46bc003740 1 -- start start 2024-09-13T18:31:50.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.035+0000 7f46bc003740 1 Processor -- start 2024-09-13T18:31:50.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.036+0000 7f46bc003740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e016f1d7c0 0x55e016f282a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:50.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.036+0000 7f46bc003740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e016f1e0e0 0x55e016f25540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:50.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.036+0000 7f46b2d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e016f1d7c0 0x55e016f282a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:50.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.036+0000 7f46b2d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e016f1d7c0 0x55e016f282a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:46340/0 (socket says 172.21.15.114:46340) 2024-09-13T18:31:50.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.036+0000 7f46b2d9d640 1 -- 172.21.15.114:0/1825072704 learned_addr learned my addr 172.21.15.114:0/1825072704 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:50.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.037+0000 7f46bc003740 1 --2- 172.21.15.114:0/1825072704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e016f28980 0x55e016f25a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:50.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.037+0000 7f46bc003740 1 -- 172.21.15.114:0/1825072704 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e016e3ff80 con 0x55e016f1e0e0 2024-09-13T18:31:50.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.037+0000 7f46b359e640 1 --2- 172.21.15.114:0/1825072704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e016f28980 0x55e016f25a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:50.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.037+0000 7f46bc003740 1 -- 172.21.15.114:0/1825072704 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e016e3fc20 con 0x55e016f1d7c0 2024-09-13T18:31:50.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.037+0000 7f46bc003740 1 -- 172.21.15.114:0/1825072704 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e016e0cab0 con 0x55e016f28980 2024-09-13T18:31:50.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.037+0000 7f46bc003740 10 client.0 _ll_register_callbacks cb 0x55e016e7b350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:31:50.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.037+0000 7f46b2d9d640 1 -- 172.21.15.114:0/1825072704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e016f28980 msgr2=0x55e016f25a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:50.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.037+0000 7f46b2d9d640 1 --2- 172.21.15.114:0/1825072704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e016f28980 0x55e016f25a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.037+0000 7f46b2d9d640 1 -- 172.21.15.114:0/1825072704 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e016f1e0e0 msgr2=0x55e016f25540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:50.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.037+0000 7f46b259c640 1 --2- 172.21.15.114:0/1064271286 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f469c0a11e0 0x7f469c0a36a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x55e016f19850 tx=0x7f46a4006b70 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:50.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.037+0000 7f46b2d9d640 1 --2- 172.21.15.114:0/1825072704 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e016f1e0e0 0x55e016f25540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.038+0000 7f46b2d9d640 1 -- 172.21.15.114:0/1825072704 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46ac009920 con 0x55e016f1d7c0 2024-09-13T18:31:50.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.038+0000 7f46b359e640 1 --2- 172.21.15.114:0/1825072704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e016f28980 0x55e016f25a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:31:50.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.038+0000 7f46b2d9d640 1 --2- 172.21.15.114:0/1825072704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e016f1d7c0 0x55e016f282a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f46ac0042c0 tx=0x7f46ac003f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9219e53dfbcd5f54 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:50.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.038+0000 7f46bc003740 1 -- 172.21.15.114:0/1825072704 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e016b75590 con 0x55e016f1d7c0 2024-09-13T18:31:50.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.039+0000 7f4696ff5640 10 client.27632 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:31:50.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.039+0000 7f4696ff5640 1 -- 172.21.15.114:0/1825072704 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ac004c50 con 0x55e016f1d7c0 2024-09-13T18:31:50.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.039+0000 7f4696ff5640 1 -- 172.21.15.114:0/1825072704 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f46ac004db0 con 0x55e016f1d7c0 2024-09-13T18:31:50.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.039+0000 7f4696ff5640 1 -- 172.21.15.114:0/1825072704 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46ac0285b0 con 0x55e016f1d7c0 2024-09-13T18:31:50.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.039+0000 7f46bc003740 10 client.27632 fetch_fsmap learned FSMap version 34 2024-09-13T18:31:50.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.039+0000 7f46bc003740 1 -- 172.21.15.114:0/1825072704 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({fsmap=34}) v3 -- 0x55e016e80360 con 0x55e016f1d7c0 2024-09-13T18:31:50.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.043+0000 7f4696ff5640 1 -- 172.21.15.114:0/1825072704 <== mon.2 v2:172.21.15.134:3300/0 4 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f46ac00c070 con 0x55e016f1d7c0 2024-09-13T18:31:50.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.043+0000 7f46bc003740 10 client.27632 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:31:50.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.043+0000 7f46bc003740 10 client.27632 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:31:50.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.043+0000 7f46bc003740 20 client.27632 populate_metadata read hostname 'smithi114' 2024-09-13T18:31:50.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.043+0000 7f46bc003740 1 --2- 172.21.15.114:0/1825072704 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55e016f49600 0x55e016f4ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:50.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.043+0000 7f46bc003740 4 client.27632 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:31:50.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.043+0000 7f46bc003740 1 -- 172.21.15.114:0/1825072704 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e016b75590 con 0x55e016f49600 2024-09-13T18:31:50.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.043+0000 7f46b259c640 1 --2- 172.21.15.114:0/1825072704 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55e016f49600 0x55e016f4ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:50.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.044+0000 7f46b259c640 1 --2- 172.21.15.114:0/1825072704 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55e016f49600 0x55e016f4ba40 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c7fc3795df77592 server_cookie=f1ffca1b586048d1 in_seq=0 out_seq=0 2024-09-13T18:31:50.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.045+0000 7f4696ff5640 10 client.27632 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:31:50.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.045+0000 7f4696ff5640 1 -- 172.21.15.114:0/1825072704 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55e016b75590 con 0x55e016f49600 2024-09-13T18:31:50.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.045+0000 7f4696ff5640 10 client.27632 handle_command_reply: tid=0 2024-09-13T18:31:50.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.045+0000 7f46bc003740 1 client.27632 shutdown 2024-09-13T18:31:50.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.050+0000 7f46bc003740 1 -- 172.21.15.114:0/1825072704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e016f1d7c0 msgr2=0x55e016f282a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:50.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.050+0000 7f46bc003740 1 --2- 172.21.15.114:0/1825072704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e016f1d7c0 0x55e016f282a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f46ac0042c0 tx=0x7f46ac003f70 comp rx=0 tx=0).stop 2024-09-13T18:31:50.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.051+0000 7f46bc003740 1 -- 172.21.15.114:0/1825072704 shutdown_connections 2024-09-13T18:31:50.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.051+0000 7f46bc003740 1 --2- 172.21.15.114:0/1825072704 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55e016f49600 0x55e016f4ba40 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.051+0000 7f46bc003740 1 --2- 172.21.15.114:0/1825072704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e016f28980 0x55e016f25a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.051+0000 7f46bc003740 1 --2- 172.21.15.114:0/1825072704 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e016f1e0e0 0x55e016f25540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.051+0000 7f46bc003740 1 --2- 172.21.15.114:0/1825072704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e016f1d7c0 0x55e016f282a0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.051+0000 7f46bc003740 1 -- 172.21.15.114:0/1825072704 >> 172.21.15.114:0/1825072704 conn(0x55e016f19d50 msgr2=0x55e016f1f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:50.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.051+0000 7f4696ff5640 0 client.27632 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:31:50.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.055+0000 7f46bc003740 1 -- 172.21.15.114:0/1825072704 shutdown_connections 2024-09-13T18:31:50.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.056+0000 7f46bc003740 1 -- 172.21.15.114:0/1825072704 wait complete. 2024-09-13T18:31:50.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.063+0000 7f46bc003740 20 client.27632 trim_cache size 0 max 16384 2024-09-13T18:31:50.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.101+0000 7f46bc003740 1 Processor -- start 2024-09-13T18:31:50.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.109+0000 7f46bc003740 1 -- start start 2024-09-13T18:31:50.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.109+0000 7f46bc003740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e016f1d7c0 0x55e016f40730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:50.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.110+0000 7f46bc003740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e016f1e0e0 0x55e016f40c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:50.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.110+0000 7f46bc003740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e016f28980 0x55e016f23e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:50.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.110+0000 7f46b259c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e016f1e0e0 0x55e016f40c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:50.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.110+0000 7f46bc003740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e016f26080 con 0x55e016f28980 2024-09-13T18:31:50.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.110+0000 7f46bc003740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e016f23ab0 con 0x55e016f1d7c0 2024-09-13T18:31:50.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.110+0000 7f46bc003740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e016f50c00 con 0x55e016f1e0e0 2024-09-13T18:31:50.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.110+0000 7f46b259c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e016f1e0e0 0x55e016f40c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:41636/0 (socket says 172.21.15.114:41636) 2024-09-13T18:31:50.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.110+0000 7f46b259c640 1 -- 172.21.15.114:0/4250358241 learned_addr learned my addr 172.21.15.114:0/4250358241 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:50.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.110+0000 7f46b359e640 1 --2- 172.21.15.114:0/4250358241 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e016f28980 0x55e016f23e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:50.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.110+0000 7f46b259c640 1 -- 172.21.15.114:0/4250358241 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e016f1d7c0 msgr2=0x55e016f40730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:50.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.110+0000 7f46b259c640 1 --2- 172.21.15.114:0/4250358241 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e016f1d7c0 0x55e016f40730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.110+0000 7f46b259c640 1 -- 172.21.15.114:0/4250358241 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e016f28980 msgr2=0x55e016f23e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:50.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.110+0000 7f46b259c640 1 --2- 172.21.15.114:0/4250358241 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e016f28980 0x55e016f23e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.110+0000 7f46b259c640 1 -- 172.21.15.114:0/4250358241 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e016e221c0 con 0x55e016f1e0e0 2024-09-13T18:31:50.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.111+0000 7f46b259c640 1 --2- 172.21.15.114:0/4250358241 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e016f1e0e0 0x55e016f40c70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x55e016f26b80 tx=0x7f46a400e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d072ee66113a81ca server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:50.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.112+0000 7f4699ffb640 1 -- 172.21.15.114:0/4250358241 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a40128e0 con 0x55e016f1e0e0 2024-09-13T18:31:50.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.112+0000 7f4699ffb640 1 -- 172.21.15.114:0/4250358241 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f46a4024070 con 0x55e016f1e0e0 2024-09-13T18:31:50.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.112+0000 7f4699ffb640 1 -- 172.21.15.114:0/4250358241 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a4029430 con 0x55e016f1e0e0 2024-09-13T18:31:50.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:49 smithi134 ceph-mon[34283]: pgmap v289: 129 pgs: 129 active+clean; 22 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 26 MiB/s wr, 590 op/s 2024-09-13T18:31:50.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.116+0000 7f46bc003740 1 -- 172.21.15.114:0/4250358241 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e016f1e0e0 msgr2=0x55e016f40c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:50.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.117+0000 7f46bc003740 1 --2- 172.21.15.114:0/4250358241 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e016f1e0e0 0x55e016f40c70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x55e016f26b80 tx=0x7f46a400e000 comp rx=0 tx=0).stop 2024-09-13T18:31:50.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.121+0000 7f46bc003740 1 -- 172.21.15.114:0/4250358241 shutdown_connections 2024-09-13T18:31:50.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.122+0000 7f46bc003740 1 --2- 172.21.15.114:0/4250358241 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e016f28980 0x55e016f23e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.122+0000 7f46bc003740 1 --2- 172.21.15.114:0/4250358241 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e016f1e0e0 0x55e016f40c70 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.122+0000 7f46bc003740 1 --2- 172.21.15.114:0/4250358241 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e016f1d7c0 0x55e016f40730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.122+0000 7f46bc003740 1 -- 172.21.15.114:0/4250358241 >> 172.21.15.114:0/4250358241 conn(0x55e016f19d50 msgr2=0x55e016f20840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:50.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.124+0000 7f46bc003740 1 -- 172.21.15.114:0/4250358241 shutdown_connections 2024-09-13T18:31:50.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.124+0000 7f46bc003740 1 -- 172.21.15.114:0/4250358241 wait complete. 2024-09-13T18:31:50.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.124+0000 7f46bc003740 1 -- start start 2024-09-13T18:31:50.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.125+0000 7f46bc003740 1 Processor -- start 2024-09-13T18:31:50.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.125+0000 7f46bc003740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e016f1d7c0 0x55e016f3ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:50.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.125+0000 7f46bc003740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e016f1e0e0 0x55e016f404a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:50.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.126+0000 7f46bc003740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e016f28980 0x55e016f98e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:50.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.126+0000 7f46bc003740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e016f99370 con 0x55e016f1d7c0 2024-09-13T18:31:50.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.126+0000 7f46bc003740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e016f994e0 con 0x55e016f28980 2024-09-13T18:31:50.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.126+0000 7f46b359e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e016f28980 0x55e016f98e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:50.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.126+0000 7f46b359e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e016f28980 0x55e016f98e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:46368/0 (socket says 172.21.15.114:46368) 2024-09-13T18:31:50.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.126+0000 7f46b359e640 1 -- 172.21.15.114:0/3600063304 learned_addr learned my addr 172.21.15.114:0/3600063304 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:50.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.126+0000 7f46bc003740 1 -- 172.21.15.114:0/3600063304 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e016f996a0 con 0x55e016f1e0e0 2024-09-13T18:31:50.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.126+0000 7f46bc003740 10 client.0 _ll_register_callbacks cb 0x7f46a80bbab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:31:50.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.126+0000 7f46b359e640 1 -- 172.21.15.114:0/3600063304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e016f1e0e0 msgr2=0x55e016f404a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:31:50.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.126+0000 7f46b359e640 1 --2- 172.21.15.114:0/3600063304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e016f1e0e0 0x55e016f404a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.126+0000 7f46b359e640 1 -- 172.21.15.114:0/3600063304 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e016f1d7c0 msgr2=0x55e016f3ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:50.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.126+0000 7f46b359e640 1 --2- 172.21.15.114:0/3600063304 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e016f1d7c0 0x55e016f3ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.126+0000 7f46b359e640 1 -- 172.21.15.114:0/3600063304 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46a4013510 con 0x55e016f28980 2024-09-13T18:31:50.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.127+0000 7f46b359e640 1 --2- 172.21.15.114:0/3600063304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e016f28980 0x55e016f98e00 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f46a80bb990 tx=0x7f46a80bfc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43ce4451a696c035 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:50.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.127+0000 7f4696ff5640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:31:50.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.127+0000 7f4696ff5640 1 -- 172.21.15.114:0/3600063304 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a80c3cd0 con 0x55e016f28980 2024-09-13T18:31:50.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.127+0000 7f4696ff5640 1 -- 172.21.15.114:0/3600063304 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f46a80c3e30 con 0x55e016f28980 2024-09-13T18:31:50.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.128+0000 7f46bc003740 1 -- 172.21.15.114:0/3600063304 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f46ac003e00 con 0x55e016f28980 2024-09-13T18:31:50.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.136+0000 7f4696ff5640 1 -- 172.21.15.114:0/3600063304 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46a80cfbb0 con 0x55e016f28980 2024-09-13T18:31:50.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.136+0000 7f46bc003740 10 client.27635 fetch_fsmap learned FSMap version 34 2024-09-13T18:31:50.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.136+0000 7f46bc003740 1 -- 172.21.15.114:0/3600063304 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55e016e47990 con 0x55e016f28980 2024-09-13T18:31:50.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.136+0000 7f4696ff5640 1 -- 172.21.15.114:0/3600063304 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f46a80cfd10 con 0x55e016f28980 2024-09-13T18:31:50.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.136+0000 7f4696ff5640 1 -- 172.21.15.114:0/3600063304 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f46a80c0a30 con 0x55e016f28980 2024-09-13T18:31:50.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.140+0000 7f46bc003740 10 client.27635 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:31:50.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.140+0000 7f46bc003740 10 client.27635 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:31:50.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.140+0000 7f46bc003740 20 client.27635 populate_metadata read hostname 'smithi114' 2024-09-13T18:31:50.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.140+0000 7f46bc003740 1 --2- 172.21.15.114:0/3600063304 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55e016fa1e40 0x55e016fa4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:50.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.140+0000 7f46bc003740 4 client.27635 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-13T18:31:50.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.140+0000 7f46bc003740 1 -- 172.21.15.114:0/3600063304 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x7f46ac003e00 con 0x55e016fa1e40 2024-09-13T18:31:50.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.140+0000 7f46b2d9d640 1 --2- 172.21.15.114:0/3600063304 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55e016fa1e40 0x55e016fa4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:50.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.147+0000 7f46b2d9d640 1 --2- 172.21.15.114:0/3600063304 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55e016fa1e40 0x55e016fa4260 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89683ee91055c2d7 server_cookie=d2129c093f3d5f38 in_seq=0 out_seq=0 2024-09-13T18:31:50.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.147+0000 7f4696ff5640 10 client.27635 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:31:50.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.149+0000 7f4696ff5640 1 -- 172.21.15.114:0/3600063304 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x7f46ac003e00 con 0x55e016fa1e40 2024-09-13T18:31:50.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.149+0000 7f4696ff5640 10 client.27635 handle_command_reply: tid=0 2024-09-13T18:31:50.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.150+0000 7f46bc003740 1 client.27635 shutdown 2024-09-13T18:31:50.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.151+0000 7f46bc003740 1 -- 172.21.15.114:0/3600063304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e016f28980 msgr2=0x55e016f98e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:50.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.151+0000 7f46bc003740 1 --2- 172.21.15.114:0/3600063304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e016f28980 0x55e016f98e00 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f46a80bb990 tx=0x7f46a80bfc50 comp rx=0 tx=0).stop 2024-09-13T18:31:50.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.152+0000 7f46bc003740 1 -- 172.21.15.114:0/3600063304 shutdown_connections 2024-09-13T18:31:50.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.152+0000 7f46bc003740 1 --2- 172.21.15.114:0/3600063304 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55e016fa1e40 0x55e016fa4260 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.152+0000 7f4696ff5640 0 client.27635 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:31:50.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.152+0000 7f46bc003740 1 --2- 172.21.15.114:0/3600063304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e016f28980 0x55e016f98e00 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.152+0000 7f46bc003740 1 --2- 172.21.15.114:0/3600063304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e016f1e0e0 0x55e016f404a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.152+0000 7f46bc003740 1 --2- 172.21.15.114:0/3600063304 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e016f1d7c0 0x55e016f3ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.152+0000 7f46bc003740 1 -- 172.21.15.114:0/3600063304 >> 172.21.15.114:0/3600063304 conn(0x55e016f19d50 msgr2=0x55e016f1f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:50.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.154+0000 7f46bc003740 1 -- 172.21.15.114:0/3600063304 shutdown_connections 2024-09-13T18:31:50.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.154+0000 7f46bc003740 1 -- 172.21.15.114:0/3600063304 wait complete. 2024-09-13T18:31:50.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.157+0000 7f46bc003740 20 client.27635 trim_cache size 0 max 16384 2024-09-13T18:31:50.160 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:31:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: "return_code": 0, 2024-09-13T18:31:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_tag": "88499e83-28c8-4baf-a007-8e91c914cbe7", 2024-09-13T18:31:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: "mode": "asynchronous" 2024-09-13T18:31:50.160 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:31:50.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.161+0000 7f46b9202640 1 -- 172.21.15.114:0/1064271286 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f469c0a11e0 msgr2=0x7f469c0a36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:50.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.162+0000 7f46b9202640 1 --2- 172.21.15.114:0/1064271286 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f469c0a11e0 0x7f469c0a36a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x55e016f19850 tx=0x7f46a4006b70 comp rx=0 tx=0).stop 2024-09-13T18:31:50.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.162+0000 7f46b9202640 1 -- 172.21.15.114:0/1064271286 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46b4119400 msgr2=0x7f46b41a99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:50.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.162+0000 7f46b9202640 1 --2- 172.21.15.114:0/1064271286 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46b4119400 0x7f46b41a99c0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f46a80057c0 tx=0x7f46a8004600 comp rx=0 tx=0).stop 2024-09-13T18:31:50.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.163+0000 7f46b9202640 1 -- 172.21.15.114:0/1064271286 shutdown_connections 2024-09-13T18:31:50.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.163+0000 7f46b9202640 1 --2- 172.21.15.114:0/1064271286 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f469c0a11e0 0x7f469c0a36a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.163+0000 7f46b9202640 1 --2- 172.21.15.114:0/1064271286 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46b4119400 0x7f46b41a99c0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.163+0000 7f46b9202640 1 --2- 172.21.15.114:0/1064271286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46b4107920 0x7f46b41a55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.163+0000 7f46b9202640 1 --2- 172.21.15.114:0/1064271286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46b4107540 0x7f46b41a5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.163+0000 7f46b9202640 1 -- 172.21.15.114:0/1064271286 >> 172.21.15.114:0/1064271286 conn(0x7f46b406afe0 msgr2=0x7f46b4119d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:50.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.165+0000 7f46b9202640 1 -- 172.21.15.114:0/1064271286 shutdown_connections 2024-09-13T18:31:50.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.165+0000 7f46b9202640 1 -- 172.21.15.114:0/1064271286 wait complete. 2024-09-13T18:31:50.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:49 smithi204 ceph-mon[34777]: pgmap v289: 129 pgs: 129 active+clean; 22 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 26 MiB/s wr, 590 op/s 2024-09-13T18:31:50.198 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T18:31:50.201 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:49 smithi114 ceph-mon[30897]: pgmap v289: 129 pgs: 129 active+clean; 22 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 26 MiB/s wr, 590 op/s 2024-09-13T18:31:50.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.489+0000 7fd2f759e640 1 -- 172.21.15.114:0/1828548071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2f8119400 msgr2=0x7fd2f811b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:50.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.489+0000 7fd2f759e640 1 --2- 172.21.15.114:0/1828548071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2f8119400 0x7fd2f811b7f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec009860 tx=0x7fd2ec02a370 comp rx=0 tx=0).stop 2024-09-13T18:31:50.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.490+0000 7fd2f759e640 1 -- 172.21.15.114:0/1828548071 shutdown_connections 2024-09-13T18:31:50.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.490+0000 7fd2f759e640 1 --2- 172.21.15.114:0/1828548071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2f8119400 0x7fd2f811b7f0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.490+0000 7fd2f759e640 1 --2- 172.21.15.114:0/1828548071 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2f8107890 0x7fd2f8107d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.490+0000 7fd2f759e640 1 --2- 172.21.15.114:0/1828548071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2f81074b0 0x7fd2f810e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.490+0000 7fd2f759e640 1 -- 172.21.15.114:0/1828548071 >> 172.21.15.114:0/1828548071 conn(0x7fd2f806aec0 msgr2=0x7fd2f806b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:50.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.497+0000 7fd2f759e640 1 -- 172.21.15.114:0/1828548071 shutdown_connections 2024-09-13T18:31:50.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.497+0000 7fd2f759e640 1 -- 172.21.15.114:0/1828548071 wait complete. 2024-09-13T18:31:50.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.498+0000 7fd2f759e640 1 Processor -- start 2024-09-13T18:31:50.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.498+0000 7fd2f759e640 1 -- start start 2024-09-13T18:31:50.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.498+0000 7fd2f759e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2f81074b0 0x7fd2f8116ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:50.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.498+0000 7fd2f759e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2f8107890 0x7fd2f8117420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:50.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.498+0000 7fd2f759e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2f8119400 0x7fd2f81124e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:50.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.498+0000 7fd2f759e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f8112a20 con 0x7fd2f8119400 2024-09-13T18:31:50.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.498+0000 7fd2f759e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f8112b90 con 0x7fd2f81074b0 2024-09-13T18:31:50.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.498+0000 7fd2f759e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f8112d50 con 0x7fd2f8107890 2024-09-13T18:31:50.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.503+0000 7fd2f6d9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2f8119400 0x7fd2f81124e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:50.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.503+0000 7fd2f6d9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2f8119400 0x7fd2f81124e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:58774/0 (socket says 172.21.15.114:58774) 2024-09-13T18:31:50.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.503+0000 7fd2f6d9d640 1 -- 172.21.15.114:0/140483304 learned_addr learned my addr 172.21.15.114:0/140483304 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:50.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.503+0000 7fd2f659c640 1 --2- 172.21.15.114:0/140483304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2f81074b0 0x7fd2f8116ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:50.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.503+0000 7fd2f5d9b640 1 --2- 172.21.15.114:0/140483304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2f8107890 0x7fd2f8117420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:50.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.505+0000 7fd2f6d9d640 1 -- 172.21.15.114:0/140483304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2f8107890 msgr2=0x7fd2f8117420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:50.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.505+0000 7fd2f6d9d640 1 --2- 172.21.15.114:0/140483304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2f8107890 0x7fd2f8117420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.505+0000 7fd2f6d9d640 1 -- 172.21.15.114:0/140483304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2f81074b0 msgr2=0x7fd2f8116ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:50.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.505+0000 7fd2f6d9d640 1 --2- 172.21.15.114:0/140483304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2f81074b0 0x7fd2f8116ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.505+0000 7fd2f6d9d640 1 -- 172.21.15.114:0/140483304 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2ec009490 con 0x7fd2f8119400 2024-09-13T18:31:50.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.509+0000 7fd2f6d9d640 1 --2- 172.21.15.114:0/140483304 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2f8119400 0x7fd2f81124e0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec02a960 tx=0x7fd2ec032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:50.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.513+0000 7fd2e77fe640 1 -- 172.21.15.114:0/140483304 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec004250 con 0x7fd2f8119400 2024-09-13T18:31:50.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.513+0000 7fd2f759e640 1 -- 172.21.15.114:0/140483304 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2f8112fd0 con 0x7fd2f8119400 2024-09-13T18:31:50.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.513+0000 7fd2f759e640 1 -- 172.21.15.114:0/140483304 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2f81ba440 con 0x7fd2f8119400 2024-09-13T18:31:50.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.513+0000 7fd2e77fe640 1 -- 172.21.15.114:0/140483304 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd2ec0043b0 con 0x7fd2f8119400 2024-09-13T18:31:50.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.513+0000 7fd2e77fe640 1 -- 172.21.15.114:0/140483304 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec004bd0 con 0x7fd2f8119400 2024-09-13T18:31:50.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.517+0000 7fd2ffa45740 1 Processor -- start 2024-09-13T18:31:50.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.518+0000 7fd2e77fe640 1 -- 172.21.15.114:0/140483304 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fd2ec004d30 con 0x7fd2f8119400 2024-09-13T18:31:50.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.519+0000 7fd2e77fe640 1 --2- 172.21.15.114:0/140483304 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fd2e00a1410 0x7fd2e00a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:50.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.520+0000 7fd2e77fe640 1 -- 172.21.15.114:0/140483304 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fd2ec0ed900 con 0x7fd2f8119400 2024-09-13T18:31:50.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.521+0000 7fd2ffa45740 1 -- start start 2024-09-13T18:31:50.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.521+0000 7fd2ffa45740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568453aa030 0x5568453aa410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:50.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.521+0000 7fd2ffa45740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568453aa950 0x5568453b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:50.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.521+0000 7fd2ffa45740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568453b51f0 0x5568453b75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:50.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.522+0000 7fd2ffa45740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x556845314f20 con 0x5568453aa950 2024-09-13T18:31:50.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.522+0000 7fd2ffa45740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568453126b0 con 0x5568453b51f0 2024-09-13T18:31:50.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.522+0000 7fd2ffa45740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55684530dfa0 con 0x5568453aa030 2024-09-13T18:31:50.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.523+0000 7fd2f659c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568453aa030 0x5568453aa410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:50.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.524+0000 7fd2f5d9b640 1 --2- 172.21.15.114:0/140483304 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fd2e00a1410 0x7fd2e00a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:50.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.524+0000 7fd2f6d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568453b51f0 0x5568453b75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:50.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.524+0000 7fd2f6d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568453b51f0 0x5568453b75e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:46400/0 (socket says 172.21.15.114:46400) 2024-09-13T18:31:50.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.524+0000 7fd2f6d9d640 1 -- 172.21.15.114:0/842753094 learned_addr learned my addr 172.21.15.114:0/842753094 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:50.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.526+0000 7fd2f6d9d640 1 -- 172.21.15.114:0/842753094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568453aa030 msgr2=0x5568453aa410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:50.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.526+0000 7fd2f6d9d640 1 --2- 172.21.15.114:0/842753094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568453aa030 0x5568453aa410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.526+0000 7fd2f6d9d640 1 -- 172.21.15.114:0/842753094 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568453aa950 msgr2=0x5568453b4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:50.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.526+0000 7fd2f6d9d640 1 --2- 172.21.15.114:0/842753094 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568453aa950 0x5568453b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.526+0000 7fd2f6d9d640 1 -- 172.21.15.114:0/842753094 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568452c8840 con 0x5568453b51f0 2024-09-13T18:31:50.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.532+0000 7fd2f6d9d640 1 --2- 172.21.15.114:0/842753094 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568453b51f0 0x5568453b75e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec0b8080 tx=0x7fd2ec031f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65bbb7eff764abf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:50.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.533+0000 7fd2f5d9b640 1 --2- 172.21.15.114:0/140483304 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fd2e00a1410 0x7fd2e00a38d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x5568453a7400 tx=0x7fd2e8012760 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:50.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.543+0000 7fd2e4ff9640 1 -- 172.21.15.114:0/842753094 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec0b9750 con 0x5568453b51f0 2024-09-13T18:31:50.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.543+0000 7fd2e4ff9640 1 -- 172.21.15.114:0/842753094 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd2ec0b99b0 con 0x5568453b51f0 2024-09-13T18:31:50.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.543+0000 7fd2e4ff9640 1 -- 172.21.15.114:0/842753094 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec0bad00 con 0x5568453b51f0 2024-09-13T18:31:50.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.544+0000 7fd2ffa45740 1 -- 172.21.15.114:0/842753094 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568453b51f0 msgr2=0x5568453b75e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:50.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.544+0000 7fd2ffa45740 1 --2- 172.21.15.114:0/842753094 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568453b51f0 0x5568453b75e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec0b8080 tx=0x7fd2ec031f40 comp rx=0 tx=0).stop 2024-09-13T18:31:50.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.549+0000 7fd2ffa45740 1 -- 172.21.15.114:0/842753094 shutdown_connections 2024-09-13T18:31:50.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.549+0000 7fd2ffa45740 1 --2- 172.21.15.114:0/842753094 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568453b51f0 0x5568453b75e0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.549+0000 7fd2ffa45740 1 --2- 172.21.15.114:0/842753094 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568453aa950 0x5568453b4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.549+0000 7fd2ffa45740 1 --2- 172.21.15.114:0/842753094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568453aa030 0x5568453aa410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.549+0000 7fd2ffa45740 1 -- 172.21.15.114:0/842753094 >> 172.21.15.114:0/842753094 conn(0x5568453a65a0 msgr2=0x5568453a6980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:50.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.552+0000 7fd2ffa45740 1 -- 172.21.15.114:0/842753094 shutdown_connections 2024-09-13T18:31:50.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.552+0000 7fd2ffa45740 1 -- 172.21.15.114:0/842753094 wait complete. 2024-09-13T18:31:50.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.553+0000 7fd2ffa45740 1 -- start start 2024-09-13T18:31:50.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.553+0000 7fd2ffa45740 1 Processor -- start 2024-09-13T18:31:50.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.553+0000 7fd2ffa45740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568453aa030 0x5568453b4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:50.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.553+0000 7fd2ffa45740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568453aa950 0x5568453b1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:50.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.553+0000 7fd2ffa45740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568453b51f0 0x5568453b2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:50.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.553+0000 7fd2ffa45740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568452af2a0 con 0x5568453aa950 2024-09-13T18:31:50.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.553+0000 7fd2ffa45740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556844fe6430 con 0x5568453b51f0 2024-09-13T18:31:50.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.553+0000 7fd2ffa45740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556845291680 con 0x5568453aa030 2024-09-13T18:31:50.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.553+0000 7fd2ffa45740 10 client.0 _ll_register_callbacks cb 0x556845307610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:31:50.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.554+0000 7fd2f6d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568453b51f0 0x5568453b2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:50.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.554+0000 7fd2f6d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568453b51f0 0x5568453b2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:46410/0 (socket says 172.21.15.114:46410) 2024-09-13T18:31:50.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.554+0000 7fd2f6d9d640 1 -- 172.21.15.114:0/1313230239 learned_addr learned my addr 172.21.15.114:0/1313230239 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:50.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.554+0000 7fd2f6d9d640 1 -- 172.21.15.114:0/1313230239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568453aa030 msgr2=0x5568453b4af0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:31:50.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.555+0000 7fd2f6d9d640 1 --2- 172.21.15.114:0/1313230239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568453aa030 0x5568453b4af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.555+0000 7fd2f6d9d640 1 -- 172.21.15.114:0/1313230239 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568453aa950 msgr2=0x5568453b1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:50.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.555+0000 7fd2f6d9d640 1 --2- 172.21.15.114:0/1313230239 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568453aa950 0x5568453b1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.555+0000 7fd2f6d9d640 1 -- 172.21.15.114:0/1313230239 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2f81ba440 con 0x5568453b51f0 2024-09-13T18:31:50.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.555+0000 7fd2f6d9d640 1 --2- 172.21.15.114:0/1313230239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568453b51f0 0x5568453b2260 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec035040 tx=0x7fd2ec0b7290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c141b959fedbf8bd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:50.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.556+0000 7fd2ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:31:50.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.556+0000 7fd2ddffb640 1 -- 172.21.15.114:0/1313230239 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec0da070 con 0x5568453b51f0 2024-09-13T18:31:50.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.557+0000 7fd2ffa45740 1 -- 172.21.15.114:0/1313230239 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5568450014e0 con 0x5568453b51f0 2024-09-13T18:31:50.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.557+0000 7fd2ddffb640 1 -- 172.21.15.114:0/1313230239 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fd2ec0da1d0 con 0x5568453b51f0 2024-09-13T18:31:50.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.557+0000 7fd2ddffb640 1 -- 172.21.15.114:0/1313230239 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2ec0eb100 con 0x5568453b51f0 2024-09-13T18:31:50.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.557+0000 7fd2ffa45740 10 client.27644 fetch_fsmap learned FSMap version 34 2024-09-13T18:31:50.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.557+0000 7fd2ffa45740 1 -- 172.21.15.114:0/1313230239 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55684530c620 con 0x5568453b51f0 2024-09-13T18:31:50.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.558+0000 7fd2ddffb640 1 -- 172.21.15.114:0/1313230239 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5568450014e0 con 0x5568453b51f0 2024-09-13T18:31:50.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.558+0000 7fd2ddffb640 1 -- 172.21.15.114:0/1313230239 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd2ec0c4d90 con 0x5568453b51f0 2024-09-13T18:31:50.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.559+0000 7fd2ffa45740 10 client.27644 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:31:50.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.559+0000 7fd2ffa45740 10 client.27644 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:31:50.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.559+0000 7fd2ffa45740 20 client.27644 populate_metadata read hostname 'smithi114' 2024-09-13T18:31:50.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.559+0000 7fd2ffa45740 1 --2- 172.21.15.114:0/1313230239 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5568453c4890 0x5568453c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:50.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.559+0000 7fd2ffa45740 4 client.27644 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:31:50.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.559+0000 7fd2ffa45740 1 -- 172.21.15.114:0/1313230239 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5568450127e0 con 0x5568453c4890 2024-09-13T18:31:50.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.560+0000 7fd2f659c640 1 --2- 172.21.15.114:0/1313230239 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5568453c4890 0x5568453c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:50.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.560+0000 7fd2f659c640 1 --2- 172.21.15.114:0/1313230239 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5568453c4890 0x5568453c6cd0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64ccb668e485e358 server_cookie=efb8ed976d0cfc53 in_seq=0 out_seq=0 2024-09-13T18:31:50.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.560+0000 7fd2ddffb640 10 client.27644 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:31:50.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.561+0000 7fd2ddffb640 1 -- 172.21.15.114:0/1313230239 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5568450127e0 con 0x5568453c4890 2024-09-13T18:31:50.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.561+0000 7fd2ddffb640 10 client.27644 handle_command_reply: tid=0 2024-09-13T18:31:50.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.561+0000 7fd2ffa45740 1 client.27644 shutdown 2024-09-13T18:31:50.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.563+0000 7fd2ffa45740 1 -- 172.21.15.114:0/1313230239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568453b51f0 msgr2=0x5568453b2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:50.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.563+0000 7fd2ffa45740 1 --2- 172.21.15.114:0/1313230239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568453b51f0 0x5568453b2260 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec035040 tx=0x7fd2ec0b7290 comp rx=0 tx=0).stop 2024-09-13T18:31:50.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.563+0000 7fd2ffa45740 1 -- 172.21.15.114:0/1313230239 shutdown_connections 2024-09-13T18:31:50.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.563+0000 7fd2ffa45740 1 --2- 172.21.15.114:0/1313230239 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5568453c4890 0x5568453c6cd0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.563+0000 7fd2ffa45740 1 --2- 172.21.15.114:0/1313230239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568453b51f0 0x5568453b2260 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.563+0000 7fd2ffa45740 1 --2- 172.21.15.114:0/1313230239 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568453aa950 0x5568453b1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.563+0000 7fd2ffa45740 1 --2- 172.21.15.114:0/1313230239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568453aa030 0x5568453b4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.563+0000 7fd2ffa45740 1 -- 172.21.15.114:0/1313230239 >> 172.21.15.114:0/1313230239 conn(0x5568453a65a0 msgr2=0x5568453ac300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:50.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.563+0000 7fd2ddffb640 0 client.27644 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:31:50.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.563+0000 7fd2ffa45740 1 -- 172.21.15.114:0/1313230239 shutdown_connections 2024-09-13T18:31:50.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.563+0000 7fd2ffa45740 1 -- 172.21.15.114:0/1313230239 wait complete. 2024-09-13T18:31:50.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.564+0000 7fd2ffa45740 20 client.27644 trim_cache size 0 max 16384 2024-09-13T18:31:50.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.602+0000 7fd2ffa45740 1 Processor -- start 2024-09-13T18:31:50.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.602+0000 7fd2ffa45740 1 -- start start 2024-09-13T18:31:50.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.602+0000 7fd2ffa45740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568453aa030 0x5568453a6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:50.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.602+0000 7fd2ffa45740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568453aa950 0x5568453bbd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:50.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.602+0000 7fd2ffa45740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568453b51f0 0x5568453bc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:50.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.603+0000 7fd2ffa45740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568450014e0 con 0x5568453aa030 2024-09-13T18:31:50.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.603+0000 7fd2ffa45740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568453a7500 con 0x5568453aa950 2024-09-13T18:31:50.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.603+0000 7fd2ffa45740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568453c1a50 con 0x5568453b51f0 2024-09-13T18:31:50.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.603+0000 7fd2f5d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568453aa950 0x5568453bbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:50.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.603+0000 7fd2f5d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568453aa950 0x5568453bbd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:46426/0 (socket says 172.21.15.114:46426) 2024-09-13T18:31:50.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.603+0000 7fd2f5d9b640 1 -- 172.21.15.114:0/2791304125 learned_addr learned my addr 172.21.15.114:0/2791304125 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:50.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.605+0000 7fd2f6d9d640 1 --2- 172.21.15.114:0/2791304125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568453b51f0 0x5568453bc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:50.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.605+0000 7fd2f659c640 1 --2- 172.21.15.114:0/2791304125 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568453aa030 0x5568453a6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:50.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.610+0000 7fd2f5d9b640 1 -- 172.21.15.114:0/2791304125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568453b51f0 msgr2=0x5568453bc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:50.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.610+0000 7fd2f5d9b640 1 --2- 172.21.15.114:0/2791304125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568453b51f0 0x5568453bc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.610+0000 7fd2f5d9b640 1 -- 172.21.15.114:0/2791304125 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568453aa030 msgr2=0x5568453a6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:50.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.610+0000 7fd2f5d9b640 1 --2- 172.21.15.114:0/2791304125 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568453aa030 0x5568453a6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.610+0000 7fd2f5d9b640 1 -- 172.21.15.114:0/2791304125 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568452ae790 con 0x5568453aa950 2024-09-13T18:31:50.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.613+0000 7fd2f5d9b640 1 --2- 172.21.15.114:0/2791304125 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568453aa950 0x5568453bbd80 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8009c50 tx=0x7fd2e8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc9975bacb9e2bd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:50.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.621+0000 7fd2e57fa640 1 -- 172.21.15.114:0/2791304125 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8009c80 con 0x5568453aa950 2024-09-13T18:31:50.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.621+0000 7fd2e57fa640 1 -- 172.21.15.114:0/2791304125 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd2e8009de0 con 0x5568453aa950 2024-09-13T18:31:50.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.621+0000 7fd2e57fa640 1 -- 172.21.15.114:0/2791304125 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e801f6c0 con 0x5568453aa950 2024-09-13T18:31:50.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.623+0000 7fd2ffa45740 1 -- 172.21.15.114:0/2791304125 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568453aa950 msgr2=0x5568453bbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:50.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.623+0000 7fd2ffa45740 1 --2- 172.21.15.114:0/2791304125 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568453aa950 0x5568453bbd80 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8009c50 tx=0x7fd2e8004730 comp rx=0 tx=0).stop 2024-09-13T18:31:50.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.623+0000 7fd2ffa45740 1 -- 172.21.15.114:0/2791304125 shutdown_connections 2024-09-13T18:31:50.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.623+0000 7fd2ffa45740 1 --2- 172.21.15.114:0/2791304125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568453b51f0 0x5568453bc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.623+0000 7fd2ffa45740 1 --2- 172.21.15.114:0/2791304125 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568453aa950 0x5568453bbd80 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.623+0000 7fd2ffa45740 1 --2- 172.21.15.114:0/2791304125 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568453aa030 0x5568453a6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.623+0000 7fd2ffa45740 1 -- 172.21.15.114:0/2791304125 >> 172.21.15.114:0/2791304125 conn(0x5568453a65a0 msgr2=0x5568453cb670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:50.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.624+0000 7fd2ffa45740 1 -- 172.21.15.114:0/2791304125 shutdown_connections 2024-09-13T18:31:50.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.624+0000 7fd2ffa45740 1 -- 172.21.15.114:0/2791304125 wait complete. 2024-09-13T18:31:50.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.624+0000 7fd2ffa45740 1 -- start start 2024-09-13T18:31:50.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.624+0000 7fd2ffa45740 1 Processor -- start 2024-09-13T18:31:50.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.624+0000 7fd2ffa45740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568453aa030 0x5568453bbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:50.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.625+0000 7fd2ffa45740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568453aa950 0x5568453bc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:50.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.625+0000 7fd2f659c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568453aa030 0x5568453bbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:50.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.625+0000 7fd2f659c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568453aa030 0x5568453bbcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:41722/0 (socket says 172.21.15.114:41722) 2024-09-13T18:31:50.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.625+0000 7fd2f659c640 1 -- 172.21.15.114:0/3013546104 learned_addr learned my addr 172.21.15.114:0/3013546104 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:50.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.625+0000 7fd2ffa45740 1 --2- 172.21.15.114:0/3013546104 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568453b51f0 0x5568453ba480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:50.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.625+0000 7fd2ffa45740 1 -- 172.21.15.114:0/3013546104 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568453bab80 con 0x5568453aa950 2024-09-13T18:31:50.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.625+0000 7fd2ffa45740 1 -- 172.21.15.114:0/3013546104 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568453bacf0 con 0x5568453b51f0 2024-09-13T18:31:50.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.625+0000 7fd2ffa45740 1 -- 172.21.15.114:0/3013546104 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568453baeb0 con 0x5568453aa030 2024-09-13T18:31:50.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.625+0000 7fd2ffa45740 10 client.0 _ll_register_callbacks cb 0x7fd2ec0b79b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:31:50.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.625+0000 7fd2f6d9d640 1 --2- 172.21.15.114:0/3013546104 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568453b51f0 0x5568453ba480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:50.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.625+0000 7fd2f5d9b640 1 --2- 172.21.15.114:0/3013546104 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568453aa950 0x5568453bc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:50.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.626+0000 7fd2f6d9d640 1 -- 172.21.15.114:0/3013546104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568453aa030 msgr2=0x5568453bbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:50.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.626+0000 7fd2f6d9d640 1 --2- 172.21.15.114:0/3013546104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568453aa030 0x5568453bbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.626+0000 7fd2f6d9d640 1 -- 172.21.15.114:0/3013546104 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568453aa950 msgr2=0x5568453bc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:50.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.626+0000 7fd2f6d9d640 1 --2- 172.21.15.114:0/3013546104 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568453aa950 0x5568453bc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.626+0000 7fd2f6d9d640 1 -- 172.21.15.114:0/3013546104 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5568452d3ea0 con 0x5568453b51f0 2024-09-13T18:31:50.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.626+0000 7fd2f659c640 1 --2- 172.21.15.114:0/3013546104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568453aa030 0x5568453bbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:31:50.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.626+0000 7fd2f6d9d640 1 --2- 172.21.15.114:0/3013546104 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568453b51f0 0x5568453ba480 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec0c22e0 tx=0x7fd2ec0b7a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c5dbf0d4bdd9190 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:50.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.626+0000 7fd2ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:31:50.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.627+0000 7fd2ddffb640 1 -- 172.21.15.114:0/3013546104 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec0da040 con 0x5568453b51f0 2024-09-13T18:31:50.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.627+0000 7fd2ddffb640 1 -- 172.21.15.114:0/3013546104 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fd2ec0b7d90 con 0x5568453b51f0 2024-09-13T18:31:50.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.627+0000 7fd2ffa45740 1 -- 172.21.15.114:0/3013546104 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd2ec0b99b0 con 0x5568453b51f0 2024-09-13T18:31:50.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.627+0000 7fd2ddffb640 1 -- 172.21.15.114:0/3013546104 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2ec0c2c90 con 0x5568453b51f0 2024-09-13T18:31:50.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.627+0000 7fd2ffa45740 10 client.27650 fetch_fsmap learned FSMap version 34 2024-09-13T18:31:50.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.627+0000 7fd2ffa45740 1 -- 172.21.15.114:0/3013546104 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5568452ba8c0 con 0x5568453b51f0 2024-09-13T18:31:50.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.631+0000 7fd2ddffb640 1 -- 172.21.15.114:0/3013546104 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd2ec0b99b0 con 0x5568453b51f0 2024-09-13T18:31:50.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.631+0000 7fd2ddffb640 1 -- 172.21.15.114:0/3013546104 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd2ec0ba780 con 0x5568453b51f0 2024-09-13T18:31:50.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.631+0000 7fd2ffa45740 10 client.27650 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:31:50.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.631+0000 7fd2ffa45740 10 client.27650 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:31:50.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.631+0000 7fd2ffa45740 20 client.27650 populate_metadata read hostname 'smithi114' 2024-09-13T18:31:50.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.631+0000 7fd2ffa45740 1 --2- 172.21.15.114:0/3013546104 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55684541bc60 0x55684541e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:50.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.631+0000 7fd2ffa45740 4 client.27650 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T18:31:50.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.631+0000 7fd2ffa45740 1 -- 172.21.15.114:0/3013546104 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556845065e30 con 0x55684541bc60 2024-09-13T18:31:50.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.632+0000 7fd2f659c640 1 --2- 172.21.15.114:0/3013546104 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55684541bc60 0x55684541e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:50.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.641+0000 7fd2f659c640 1 --2- 172.21.15.114:0/3013546104 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55684541bc60 0x55684541e050 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63d9cf5f52153d20 server_cookie=a3c49661efca11db in_seq=0 out_seq=0 2024-09-13T18:31:50.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.641+0000 7fd2ddffb640 10 client.27650 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:31:50.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.646+0000 7fd2ddffb640 1 -- 172.21.15.114:0/3013546104 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x556845065e30 con 0x55684541bc60 2024-09-13T18:31:50.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.646+0000 7fd2ddffb640 10 client.27650 handle_command_reply: tid=0 2024-09-13T18:31:50.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.648+0000 7fd2ffa45740 1 client.27650 shutdown 2024-09-13T18:31:50.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.652+0000 7fd2ffa45740 1 -- 172.21.15.114:0/3013546104 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568453b51f0 msgr2=0x5568453ba480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:50.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.652+0000 7fd2ffa45740 1 --2- 172.21.15.114:0/3013546104 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568453b51f0 0x5568453ba480 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec0c22e0 tx=0x7fd2ec0b7a60 comp rx=0 tx=0).stop 2024-09-13T18:31:50.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.652+0000 7fd2ffa45740 1 -- 172.21.15.114:0/3013546104 shutdown_connections 2024-09-13T18:31:50.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.652+0000 7fd2ffa45740 1 --2- 172.21.15.114:0/3013546104 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55684541bc60 0x55684541e050 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.652+0000 7fd2ffa45740 1 --2- 172.21.15.114:0/3013546104 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568453b51f0 0x5568453ba480 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.652+0000 7fd2ffa45740 1 --2- 172.21.15.114:0/3013546104 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568453aa950 0x5568453bc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.652+0000 7fd2ffa45740 1 --2- 172.21.15.114:0/3013546104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568453aa030 0x5568453bbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.652+0000 7fd2ffa45740 1 -- 172.21.15.114:0/3013546104 >> 172.21.15.114:0/3013546104 conn(0x5568453a65a0 msgr2=0x5568453b1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:50.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.656+0000 7fd2ddffb640 0 client.27650 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:31:50.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.657+0000 7fd2ffa45740 1 -- 172.21.15.114:0/3013546104 shutdown_connections 2024-09-13T18:31:50.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.660+0000 7fd2ffa45740 1 -- 172.21.15.114:0/3013546104 wait complete. 2024-09-13T18:31:50.668 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:31:50.668 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-13T18:31:50.668 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": { 2024-09-13T18:31:50.668 INFO:teuthology.orchestra.run.smithi114.stdout: "88499e83-28c8-4baf-a007-8e91c914cbe7": { 2024-09-13T18:31:50.668 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-13T18:31:50.668 INFO:teuthology.orchestra.run.smithi114.stdout: "tag": "88499e83-28c8-4baf-a007-8e91c914cbe7", 2024-09-13T18:31:50.668 INFO:teuthology.orchestra.run.smithi114.stdout: "options": "recursive,force" 2024-09-13T18:31:50.668 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:31:50.668 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:31:50.668 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:31:50.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.664+0000 7fd2ffa45740 20 client.27650 trim_cache size 0 max 16384 2024-09-13T18:31:50.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.671+0000 7fd2f759e640 1 -- 172.21.15.114:0/140483304 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fd2e00a1410 msgr2=0x7fd2e00a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:50.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.671+0000 7fd2f759e640 1 --2- 172.21.15.114:0/140483304 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fd2e00a1410 0x7fd2e00a38d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x5568453a7400 tx=0x7fd2e8012760 comp rx=0 tx=0).stop 2024-09-13T18:31:50.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.671+0000 7fd2f759e640 1 -- 172.21.15.114:0/140483304 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2f8119400 msgr2=0x7fd2f81124e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:50.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.671+0000 7fd2f759e640 1 --2- 172.21.15.114:0/140483304 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2f8119400 0x7fd2f81124e0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec02a960 tx=0x7fd2ec032670 comp rx=0 tx=0).stop 2024-09-13T18:31:50.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.672+0000 7fd2f759e640 1 -- 172.21.15.114:0/140483304 shutdown_connections 2024-09-13T18:31:50.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.672+0000 7fd2f759e640 1 --2- 172.21.15.114:0/140483304 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fd2e00a1410 0x7fd2e00a38d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.672+0000 7fd2f759e640 1 --2- 172.21.15.114:0/140483304 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2f8119400 0x7fd2f81124e0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.672+0000 7fd2f759e640 1 --2- 172.21.15.114:0/140483304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2f8107890 0x7fd2f8117420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.672+0000 7fd2f759e640 1 --2- 172.21.15.114:0/140483304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2f81074b0 0x7fd2f8116ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:50.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.672+0000 7fd2f759e640 1 -- 172.21.15.114:0/140483304 >> 172.21.15.114:0/140483304 conn(0x7fd2f806aec0 msgr2=0x7fd2f806ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:50.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.675+0000 7fd2f759e640 1 -- 172.21.15.114:0/140483304 shutdown_connections 2024-09-13T18:31:50.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:31:50.675+0000 7fd2f759e640 1 -- 172.21.15.114:0/140483304 wait complete. 2024-09-13T18:31:50.699 INFO:tasks.cephfs.filesystem:scrub status for tag:88499e83-28c8-4baf-a007-8e91c914cbe7 - {'path': '/', 'tag': '88499e83-28c8-4baf-a007-8e91c914cbe7', 'options': 'recursive,force'} 2024-09-13T18:31:50.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:50 smithi114 ceph-mon[30897]: scrub queued for path: / 2024-09-13T18:31:50.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:50 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:31:50.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:50 smithi114 ceph-mon[30897]: scrub summary: active paths [/] 2024-09-13T18:31:50.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:50 smithi114 ceph-mon[30897]: scrub summary: active paths [/] 2024-09-13T18:31:50.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:50 smithi114 ceph-mon[30897]: scrub summary: active paths [/] 2024-09-13T18:31:50.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:50 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:31:50.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:50 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:31:50.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:50 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:31:51.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:50 smithi134 ceph-mon[34283]: scrub queued for path: / 2024-09-13T18:31:51.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:50 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:31:51.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:50 smithi134 ceph-mon[34283]: scrub summary: active paths [/] 2024-09-13T18:31:51.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:50 smithi134 ceph-mon[34283]: scrub summary: active paths [/] 2024-09-13T18:31:51.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:50 smithi134 ceph-mon[34283]: scrub summary: active paths [/] 2024-09-13T18:31:51.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:50 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:31:51.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:50 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:31:51.120 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:50 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:31:51.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:50 smithi204 ceph-mon[34777]: scrub queued for path: / 2024-09-13T18:31:51.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:50 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:31:51.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:50 smithi204 ceph-mon[34777]: scrub summary: active paths [/] 2024-09-13T18:31:51.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:50 smithi204 ceph-mon[34777]: scrub summary: active paths [/] 2024-09-13T18:31:51.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:50 smithi204 ceph-mon[34777]: scrub summary: active paths [/] 2024-09-13T18:31:51.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:50 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:31:51.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:50 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:31:51.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:50 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:31:52.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:51 smithi134 ceph-mon[34283]: pgmap v290: 129 pgs: 129 active+clean; 22 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 36 MiB/s wr, 797 op/s 2024-09-13T18:31:52.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:51 smithi204 ceph-mon[34777]: pgmap v290: 129 pgs: 129 active+clean; 22 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 36 MiB/s wr, 797 op/s 2024-09-13T18:31:52.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:51 smithi114 ceph-mon[30897]: pgmap v290: 129 pgs: 129 active+clean; 22 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 36 MiB/s wr, 797 op/s 2024-09-13T18:31:54.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:53 smithi134 ceph-mon[34283]: pgmap v291: 129 pgs: 129 active+clean; 22 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 28 MiB/s wr, 625 op/s 2024-09-13T18:31:54.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:53 smithi204 ceph-mon[34777]: pgmap v291: 129 pgs: 129 active+clean; 22 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 28 MiB/s wr, 625 op/s 2024-09-13T18:31:54.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:53 smithi114 ceph-mon[30897]: pgmap v291: 129 pgs: 129 active+clean; 22 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 28 MiB/s wr, 625 op/s 2024-09-13T18:31:56.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:55 smithi134 ceph-mon[34283]: pgmap v292: 129 pgs: 129 active+clean; 22 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 28 MiB/s wr, 620 op/s 2024-09-13T18:31:56.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:55 smithi204 ceph-mon[34777]: pgmap v292: 129 pgs: 129 active+clean; 22 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 28 MiB/s wr, 620 op/s 2024-09-13T18:31:56.222 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:55 smithi114 ceph-mon[30897]: pgmap v292: 129 pgs: 129 active+clean; 22 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 28 MiB/s wr, 620 op/s 2024-09-13T18:31:58.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:57 smithi134 ceph-mon[34283]: pgmap v293: 129 pgs: 129 active+clean; 22 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 35 MiB/s wr, 787 op/s 2024-09-13T18:31:58.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:57 smithi204 ceph-mon[34777]: pgmap v293: 129 pgs: 129 active+clean; 22 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 35 MiB/s wr, 787 op/s 2024-09-13T18:31:58.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:57 smithi114 ceph-mon[30897]: pgmap v293: 129 pgs: 129 active+clean; 22 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 35 MiB/s wr, 787 op/s 2024-09-13T18:31:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.239+0000 7f6c150fc640 1 -- 172.21.15.114:0/3492967059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c10119400 msgr2=0x7f6c1011b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.239+0000 7f6c150fc640 1 --2- 172.21.15.114:0/3492967059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c10119400 0x7f6c1011b7f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6c00009900 tx=0x7f6c0002a420 comp rx=0 tx=0).stop 2024-09-13T18:31:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.240+0000 7f6c150fc640 1 -- 172.21.15.114:0/3492967059 shutdown_connections 2024-09-13T18:31:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.240+0000 7f6c150fc640 1 --2- 172.21.15.114:0/3492967059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c10119400 0x7f6c1011b7f0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.240+0000 7f6c150fc640 1 --2- 172.21.15.114:0/3492967059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c10107890 0x7f6c10107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.240+0000 7f6c150fc640 1 --2- 172.21.15.114:0/3492967059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c101074b0 0x7f6c1010e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.240+0000 7f6c150fc640 1 -- 172.21.15.114:0/3492967059 >> 172.21.15.114:0/3492967059 conn(0x7f6c1006adf0 msgr2=0x7f6c1006b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:58.251 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.248+0000 7f6c150fc640 1 -- 172.21.15.114:0/3492967059 shutdown_connections 2024-09-13T18:31:58.251 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.248+0000 7f6c150fc640 1 -- 172.21.15.114:0/3492967059 wait complete. 2024-09-13T18:31:58.251 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.249+0000 7f6c150fc640 1 Processor -- start 2024-09-13T18:31:58.252 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.250+0000 7f6c150fc640 1 -- start start 2024-09-13T18:31:58.253 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.250+0000 7f6c150fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c101074b0 0x7f6c101a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:58.253 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.251+0000 7f6c150fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c10107890 0x7f6c101a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:58.253 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.251+0000 7f6c150fc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c10119400 0x7f6c101a97c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.251+0000 7f6c150fc640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c101a9dc0 con 0x7f6c10119400 2024-09-13T18:31:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.251+0000 7f6c150fc640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c101a9f30 con 0x7f6c101074b0 2024-09-13T18:31:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.251+0000 7f6c150fc640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c101aa0f0 con 0x7f6c10107890 2024-09-13T18:31:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.251+0000 7f6c0ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c101074b0 0x7f6c101a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.251+0000 7f6c0ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c101074b0 0x7f6c101a4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:38174/0 (socket says 172.21.15.114:38174) 2024-09-13T18:31:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.251+0000 7f6c0ffff640 1 -- 172.21.15.114:0/3266584614 learned_addr learned my addr 172.21.15.114:0/3266584614 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.252+0000 7f6c0ffff640 1 -- 172.21.15.114:0/3266584614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c10107890 msgr2=0x7f6c101a53d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:31:58.255 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.252+0000 7f6c0ffff640 1 --2- 172.21.15.114:0/3266584614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c10107890 0x7f6c101a53d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:58.255 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.252+0000 7f6c0ffff640 1 -- 172.21.15.114:0/3266584614 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c10119400 msgr2=0x7f6c101a97c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:31:58.255 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.252+0000 7f6c0ffff640 1 --2- 172.21.15.114:0/3266584614 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c10119400 0x7f6c101a97c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:58.255 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.252+0000 7f6c0ffff640 1 -- 172.21.15.114:0/3266584614 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c00009560 con 0x7f6c101074b0 2024-09-13T18:31:58.255 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.252+0000 7f6c0ffff640 1 --2- 172.21.15.114:0/3266584614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c101074b0 0x7f6c101a4e90 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6c0800ba50 tx=0x7f6c0800bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:58.255 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.253+0000 7f6c0d7fa640 1 -- 172.21.15.114:0/3266584614 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0800cbe0 con 0x7f6c101074b0 2024-09-13T18:31:58.256 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.253+0000 7f6c0d7fa640 1 -- 172.21.15.114:0/3266584614 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6c0800cd40 con 0x7f6c101074b0 2024-09-13T18:31:58.256 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.253+0000 7f6c0d7fa640 1 -- 172.21.15.114:0/3266584614 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c080076c0 con 0x7f6c101074b0 2024-09-13T18:31:58.256 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.253+0000 7f6c150fc640 1 -- 172.21.15.114:0/3266584614 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c101aa3d0 con 0x7f6c101074b0 2024-09-13T18:31:58.256 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.253+0000 7f6c150fc640 1 -- 172.21.15.114:0/3266584614 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c101b1c50 con 0x7f6c101074b0 2024-09-13T18:31:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.256+0000 7f6c19085740 1 Processor -- start 2024-09-13T18:31:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.257+0000 7f6c19085740 1 -- start start 2024-09-13T18:31:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.258+0000 7f6c19085740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3e9c28560 0x55d3e9c28940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.258+0000 7f6c19085740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3e9c28e80 0x55d3e9c331e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.258+0000 7f6c19085740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d3e9c33720 0x55d3e9c35b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.258+0000 7f6c19085740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e9b93c30 con 0x55d3e9c33720 2024-09-13T18:31:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.258+0000 7f6c19085740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e9b913c0 con 0x55d3e9c28560 2024-09-13T18:31:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.258+0000 7f6c19085740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e9b8ccb0 con 0x55d3e9c28e80 2024-09-13T18:31:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.258+0000 7f6c148fb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3e9c28560 0x55d3e9c28940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.258+0000 7f6c148fb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3e9c28560 0x55d3e9c28940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:38190/0 (socket says 172.21.15.114:38190) 2024-09-13T18:31:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.258+0000 7f6c148fb640 1 -- 172.21.15.114:0/933938412 learned_addr learned my addr 172.21.15.114:0/933938412 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.261+0000 7f6c0f7fe640 1 --2- 172.21.15.114:0/933938412 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d3e9c33720 0x55d3e9c35b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:58.264 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.262+0000 7f6c148fb640 1 -- 172.21.15.114:0/933938412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3e9c28e80 msgr2=0x55d3e9c331e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:31:58.264 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.262+0000 7f6c0d7fa640 1 -- 172.21.15.114:0/3266584614 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f6c08007820 con 0x7f6c101074b0 2024-09-13T18:31:58.264 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.262+0000 7f6c148fb640 1 --2- 172.21.15.114:0/933938412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3e9c28e80 0x55d3e9c331e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:58.264 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.262+0000 7f6c148fb640 1 -- 172.21.15.114:0/933938412 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d3e9c33720 msgr2=0x55d3e9c35b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:58.264 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.262+0000 7f6c148fb640 1 --2- 172.21.15.114:0/933938412 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d3e9c33720 0x55d3e9c35b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:58.265 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.262+0000 7f6c148fb640 1 -- 172.21.15.114:0/933938412 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3e9b33910 con 0x55d3e9c28560 2024-09-13T18:31:58.265 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.262+0000 7f6c0f7fe640 1 --2- 172.21.15.114:0/933938412 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d3e9c33720 0x55d3e9c35b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:31:58.265 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.263+0000 7f6c148fb640 1 --2- 172.21.15.114:0/933938412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3e9c28560 0x55d3e9c28940 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f6c00009230 tx=0x7f6c00004180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81f9fc9efe8ab3d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:58.266 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.263+0000 7f6c0d7fa640 1 --2- 172.21.15.114:0/3266584614 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f6bf40a1410 0x7f6bf40a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:58.266 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.263+0000 7f6c0d7fa640 1 -- 172.21.15.114:0/3266584614 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f6c080cc6f0 con 0x7f6c101074b0 2024-09-13T18:31:58.266 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.264+0000 7f6c0ffff640 1 --2- 172.21.15.114:0/3266584614 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f6bf40a1410 0x7f6bf40a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:58.266 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.264+0000 7f6c067fc640 1 -- 172.21.15.114:0/933938412 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c000044f0 con 0x55d3e9c28560 2024-09-13T18:31:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.264+0000 7f6c067fc640 1 -- 172.21.15.114:0/933938412 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6c00005490 con 0x55d3e9c28560 2024-09-13T18:31:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.265+0000 7f6c19085740 1 -- 172.21.15.114:0/933938412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3e9c28560 msgr2=0x55d3e9c28940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.265+0000 7f6c19085740 1 --2- 172.21.15.114:0/933938412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3e9c28560 0x55d3e9c28940 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f6c00009230 tx=0x7f6c00004180 comp rx=0 tx=0).stop 2024-09-13T18:31:58.270 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.266+0000 7f6c0ffff640 1 --2- 172.21.15.114:0/3266584614 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f6bf40a1410 0x7f6bf40a38d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x55d3e9c245f0 tx=0x7f6c08009800 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:58.273 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.270+0000 7f6c19085740 1 -- 172.21.15.114:0/933938412 shutdown_connections 2024-09-13T18:31:58.273 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.270+0000 7f6c19085740 1 --2- 172.21.15.114:0/933938412 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d3e9c33720 0x55d3e9c35b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:58.273 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.270+0000 7f6c19085740 1 --2- 172.21.15.114:0/933938412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3e9c28e80 0x55d3e9c331e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:58.273 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.270+0000 7f6c19085740 1 --2- 172.21.15.114:0/933938412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3e9c28560 0x55d3e9c28940 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:58.273 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.270+0000 7f6c19085740 1 -- 172.21.15.114:0/933938412 >> 172.21.15.114:0/933938412 conn(0x55d3e9c24af0 msgr2=0x55d3e9c24ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:58.274 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.271+0000 7f6c19085740 1 -- 172.21.15.114:0/933938412 shutdown_connections 2024-09-13T18:31:58.274 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.271+0000 7f6c19085740 1 -- 172.21.15.114:0/933938412 wait complete. 2024-09-13T18:31:58.274 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.271+0000 7f6c19085740 1 -- start start 2024-09-13T18:31:58.274 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.272+0000 7f6c19085740 1 Processor -- start 2024-09-13T18:31:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.272+0000 7f6c19085740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3e9c28560 0x55d3e9c32f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.272+0000 7f6c19085740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3e9c28e80 0x55d3e9c30230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.272+0000 7f6c19085740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d3e9c33720 0x55d3e9c30770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.272+0000 7f6c19085740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e9b4ae80 con 0x55d3e9c33720 2024-09-13T18:31:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.272+0000 7f6c19085740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e9b4ab20 con 0x55d3e9c28560 2024-09-13T18:31:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.272+0000 7f6c19085740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e9b10260 con 0x55d3e9c28e80 2024-09-13T18:31:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.272+0000 7f6c19085740 10 client.0 _ll_register_callbacks cb 0x55d3e9b45df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:31:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.273+0000 7f6c148fb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3e9c28560 0x55d3e9c32f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.273+0000 7f6c148fb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3e9c28560 0x55d3e9c32f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:38206/0 (socket says 172.21.15.114:38206) 2024-09-13T18:31:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.273+0000 7f6c148fb640 1 -- 172.21.15.114:0/806375509 learned_addr learned my addr 172.21.15.114:0/806375509 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:58.276 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.273+0000 7f6c0f7fe640 1 --2- 172.21.15.114:0/806375509 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3e9c28e80 0x55d3e9c30230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:58.276 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.274+0000 7f6c0f7fe640 1 -- 172.21.15.114:0/806375509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3e9c28560 msgr2=0x55d3e9c32f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:58.276 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.274+0000 7f6c0f7fe640 1 --2- 172.21.15.114:0/806375509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3e9c28560 0x55d3e9c32f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:58.276 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.274+0000 7f6c0f7fe640 1 -- 172.21.15.114:0/806375509 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d3e9c33720 msgr2=0x55d3e9c30770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:58.276 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.274+0000 7f6c0f7fe640 1 --2- 172.21.15.114:0/806375509 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d3e9c33720 0x55d3e9c30770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:58.276 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.274+0000 7f6c0f7fe640 1 -- 172.21.15.114:0/806375509 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c1011c060 con 0x55d3e9c28e80 2024-09-13T18:31:58.277 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.275+0000 7f6c0f7fe640 1 --2- 172.21.15.114:0/806375509 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3e9c28e80 0x55d3e9c30230 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x55d3e9c25be0 tx=0x7f6bfc00e600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=485412246d973cd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:58.279 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.277+0000 7f6bf37fe640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:31:58.279 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.277+0000 7f6bf37fe640 1 -- 172.21.15.114:0/806375509 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc00c070 con 0x55d3e9c28e80 2024-09-13T18:31:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.277+0000 7f6c19085740 1 -- 172.21.15.114:0/806375509 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d3e9bb5680 con 0x55d3e9c28e80 2024-09-13T18:31:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.277+0000 7f6bf37fe640 1 -- 172.21.15.114:0/806375509 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f6bfc013410 con 0x55d3e9c28e80 2024-09-13T18:31:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.277+0000 7f6bf37fe640 1 -- 172.21.15.114:0/806375509 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6bfc0135d0 con 0x55d3e9c28e80 2024-09-13T18:31:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.278+0000 7f6c19085740 10 client.27604 fetch_fsmap learned FSMap version 34 2024-09-13T18:31:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.278+0000 7f6c19085740 1 -- 172.21.15.114:0/806375509 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55d3e9b8b330 con 0x55d3e9c28e80 2024-09-13T18:31:58.281 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.278+0000 7f6bf37fe640 1 -- 172.21.15.114:0/806375509 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d3e9bb5680 con 0x55d3e9c28e80 2024-09-13T18:31:58.281 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.279+0000 7f6bf37fe640 1 -- 172.21.15.114:0/806375509 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6bfc01b890 con 0x55d3e9c28e80 2024-09-13T18:31:58.282 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.280+0000 7f6c19085740 10 client.27604 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:31:58.282 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.280+0000 7f6c19085740 10 client.27604 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:31:58.282 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.280+0000 7f6c19085740 20 client.27604 populate_metadata read hostname 'smithi114' 2024-09-13T18:31:58.282 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.280+0000 7f6c19085740 1 --2- 172.21.15.114:0/806375509 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55d3e9c541a0 0x55d3e9c565c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:58.283 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.280+0000 7f6c19085740 4 client.27604 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:31:58.283 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.280+0000 7f6c19085740 1 -- 172.21.15.114:0/806375509 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d3e9b805d0 con 0x55d3e9c541a0 2024-09-13T18:31:58.298 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.296+0000 7f6c148fb640 1 --2- 172.21.15.114:0/806375509 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55d3e9c541a0 0x55d3e9c565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:58.315 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.313+0000 7f6c148fb640 1 --2- 172.21.15.114:0/806375509 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55d3e9c541a0 0x55d3e9c565c0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd03d41e38aa0d74 server_cookie=b4fc9f061668d092 in_seq=0 out_seq=0 2024-09-13T18:31:58.317 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.315+0000 7f6bf37fe640 10 client.27604 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:31:58.317 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.315+0000 7f6bf37fe640 1 -- 172.21.15.114:0/806375509 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55d3e9b805d0 con 0x55d3e9c541a0 2024-09-13T18:31:58.317 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.315+0000 7f6bf37fe640 10 client.27604 handle_command_reply: tid=0 2024-09-13T18:31:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.318+0000 7f6c19085740 1 client.27604 shutdown 2024-09-13T18:31:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.325+0000 7f6c19085740 1 -- 172.21.15.114:0/806375509 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3e9c28e80 msgr2=0x55d3e9c30230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.325+0000 7f6c19085740 1 --2- 172.21.15.114:0/806375509 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3e9c28e80 0x55d3e9c30230 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x55d3e9c25be0 tx=0x7f6bfc00e600 comp rx=0 tx=0).stop 2024-09-13T18:31:58.329 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.327+0000 7f6c19085740 1 -- 172.21.15.114:0/806375509 shutdown_connections 2024-09-13T18:31:58.330 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.327+0000 7f6c19085740 1 --2- 172.21.15.114:0/806375509 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55d3e9c541a0 0x55d3e9c565c0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:58.330 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.327+0000 7f6c19085740 1 --2- 172.21.15.114:0/806375509 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d3e9c33720 0x55d3e9c30770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:58.330 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.327+0000 7f6c19085740 1 --2- 172.21.15.114:0/806375509 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3e9c28e80 0x55d3e9c30230 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:58.330 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.327+0000 7f6c19085740 1 --2- 172.21.15.114:0/806375509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3e9c28560 0x55d3e9c32f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:58.330 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.327+0000 7f6c19085740 1 -- 172.21.15.114:0/806375509 >> 172.21.15.114:0/806375509 conn(0x55d3e9c24af0 msgr2=0x55d3e9c23d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:58.330 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.327+0000 7f6bf37fe640 0 client.27604 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:31:58.330 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.327+0000 7f6c19085740 1 -- 172.21.15.114:0/806375509 shutdown_connections 2024-09-13T18:31:58.334 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.331+0000 7f6c19085740 1 -- 172.21.15.114:0/806375509 wait complete. 2024-09-13T18:31:58.334 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.332+0000 7f6c19085740 20 client.27604 trim_cache size 0 max 16384 2024-09-13T18:31:58.365 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.362+0000 7f6c19085740 1 Processor -- start 2024-09-13T18:31:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.365+0000 7f6c19085740 1 -- start start 2024-09-13T18:31:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.365+0000 7f6c19085740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3e9c28560 0x55d3e9c32aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.365+0000 7f6c19085740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d3e9c28e80 0x55d3e9c2eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.365+0000 7f6c19085740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3e9c33720 0x55d3e9c2f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.365+0000 7f6c19085740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e9c25660 con 0x55d3e9c28e80 2024-09-13T18:31:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.365+0000 7f6c19085740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e9b80600 con 0x55d3e9c33720 2024-09-13T18:31:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.365+0000 7f6c19085740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e9c294d0 con 0x55d3e9c28560 2024-09-13T18:31:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.367+0000 7f6c148fb640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3e9c28560 0x55d3e9c32aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.367+0000 7f6c148fb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3e9c33720 0x55d3e9c2f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.367+0000 7f6c148fb640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3e9c28560 0x55d3e9c32aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:38594/0 (socket says 172.21.15.114:38594) 2024-09-13T18:31:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.367+0000 7f6c148fb640 1 -- 172.21.15.114:0/970354794 learned_addr learned my addr 172.21.15.114:0/970354794 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:58.375 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.373+0000 7f6c148fb640 1 -- 172.21.15.114:0/970354794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3e9c33720 msgr2=0x55d3e9c2f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:58.375 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.373+0000 7f6c148fb640 1 --2- 172.21.15.114:0/970354794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3e9c33720 0x55d3e9c2f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:58.375 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.373+0000 7f6c148fb640 1 -- 172.21.15.114:0/970354794 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d3e9c28e80 msgr2=0x55d3e9c2eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:58.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.373+0000 7f6c148fb640 1 --2- 172.21.15.114:0/970354794 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d3e9c28e80 0x55d3e9c2eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:58.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.373+0000 7f6c148fb640 1 -- 172.21.15.114:0/970354794 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3e9b2d300 con 0x55d3e9c28560 2024-09-13T18:31:58.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.373+0000 7f6c148fb640 1 --2- 172.21.15.114:0/970354794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3e9c28560 0x55d3e9c32aa0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6c00005780 tx=0x7f6c00005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcdc6508df6b9e84 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:58.377 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.375+0000 7f6c06ffd640 1 -- 172.21.15.114:0/970354794 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c00058050 con 0x55d3e9c28560 2024-09-13T18:31:58.377 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.375+0000 7f6c06ffd640 1 -- 172.21.15.114:0/970354794 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6c00059040 con 0x55d3e9c28560 2024-09-13T18:31:58.377 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.375+0000 7f6c06ffd640 1 -- 172.21.15.114:0/970354794 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c00003910 con 0x55d3e9c28560 2024-09-13T18:31:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.377+0000 7f6c19085740 1 -- 172.21.15.114:0/970354794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3e9c28560 msgr2=0x55d3e9c32aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.377+0000 7f6c19085740 1 --2- 172.21.15.114:0/970354794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3e9c28560 0x55d3e9c32aa0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6c00005780 tx=0x7f6c00005d50 comp rx=0 tx=0).stop 2024-09-13T18:31:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.379+0000 7f6c19085740 1 -- 172.21.15.114:0/970354794 shutdown_connections 2024-09-13T18:31:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.379+0000 7f6c19085740 1 --2- 172.21.15.114:0/970354794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3e9c33720 0x55d3e9c2f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.379+0000 7f6c19085740 1 --2- 172.21.15.114:0/970354794 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d3e9c28e80 0x55d3e9c2eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.379+0000 7f6c19085740 1 --2- 172.21.15.114:0/970354794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3e9c28560 0x55d3e9c32aa0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.379+0000 7f6c19085740 1 -- 172.21.15.114:0/970354794 >> 172.21.15.114:0/970354794 conn(0x55d3e9c24af0 msgr2=0x55d3e9c2b560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.379+0000 7f6c19085740 1 -- 172.21.15.114:0/970354794 shutdown_connections 2024-09-13T18:31:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.379+0000 7f6c19085740 1 -- 172.21.15.114:0/970354794 wait complete. 2024-09-13T18:31:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.380+0000 7f6c19085740 1 -- start start 2024-09-13T18:31:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.380+0000 7f6c19085740 1 Processor -- start 2024-09-13T18:31:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.380+0000 7f6c19085740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3e9c28560 0x55d3e9c49270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.380+0000 7f6c19085740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3e9c28e80 0x55d3e9c497b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.380+0000 7f6c19085740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d3e9c33720 0x55d3e9ca96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.380+0000 7f6c19085740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e9c49f40 con 0x55d3e9c33720 2024-09-13T18:31:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.380+0000 7f6c19085740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e9c4a0b0 con 0x55d3e9c28560 2024-09-13T18:31:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.380+0000 7f6c19085740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e9ca9be0 con 0x55d3e9c28e80 2024-09-13T18:31:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.380+0000 7f6c19085740 10 client.0 _ll_register_callbacks cb 0x7f6c00009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:31:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.381+0000 7f6c0f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3e9c28e80 0x55d3e9c497b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.381+0000 7f6c148fb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3e9c28560 0x55d3e9c49270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.381+0000 7f6c148fb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3e9c28560 0x55d3e9c49270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:38224/0 (socket says 172.21.15.114:38224) 2024-09-13T18:31:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.381+0000 7f6c148fb640 1 -- 172.21.15.114:0/1067404884 learned_addr learned my addr 172.21.15.114:0/1067404884 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:31:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.381+0000 7f6c148fb640 1 --2- 172.21.15.114:0/1067404884 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d3e9c33720 0x55d3e9ca96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.381+0000 7f6c148fb640 1 -- 172.21.15.114:0/1067404884 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3e9c28e80 msgr2=0x55d3e9c497b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.381+0000 7f6c148fb640 1 --2- 172.21.15.114:0/1067404884 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3e9c28e80 0x55d3e9c497b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.381+0000 7f6c148fb640 1 -- 172.21.15.114:0/1067404884 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d3e9c33720 msgr2=0x55d3e9ca96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:31:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.381+0000 7f6c148fb640 1 --2- 172.21.15.114:0/1067404884 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d3e9c33720 0x55d3e9ca96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.381+0000 7f6c148fb640 1 -- 172.21.15.114:0/1067404884 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d3e9b33910 con 0x55d3e9c28560 2024-09-13T18:31:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.382+0000 7f6c148fb640 1 --2- 172.21.15.114:0/1067404884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3e9c28560 0x55d3e9c49270 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6c00005750 tx=0x7f6c00005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adddb96b37fc1976 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:31:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.382+0000 7f6bf37fe640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:31:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.382+0000 7f6bf37fe640 1 -- 172.21.15.114:0/1067404884 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c00058070 con 0x55d3e9c28560 2024-09-13T18:31:58.388 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.382+0000 7f6c19085740 1 -- 172.21.15.114:0/1067404884 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6c00005490 con 0x55d3e9c28560 2024-09-13T18:31:58.388 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.383+0000 7f6bf37fe640 1 -- 172.21.15.114:0/1067404884 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f6c00000e40 con 0x55d3e9c28560 2024-09-13T18:31:58.388 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.383+0000 7f6bf37fe640 1 -- 172.21.15.114:0/1067404884 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c00005210 con 0x55d3e9c28560 2024-09-13T18:31:58.388 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.383+0000 7f6c19085740 10 client.27668 fetch_fsmap learned FSMap version 34 2024-09-13T18:31:58.388 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.383+0000 7f6c19085740 1 -- 172.21.15.114:0/1067404884 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55d3e9b528e0 con 0x55d3e9c28560 2024-09-13T18:31:58.388 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.384+0000 7f6bf37fe640 1 -- 172.21.15.114:0/1067404884 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6c00005490 con 0x55d3e9c28560 2024-09-13T18:31:58.388 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.384+0000 7f6bf37fe640 1 -- 172.21.15.114:0/1067404884 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6c00003c90 con 0x55d3e9c28560 2024-09-13T18:31:58.388 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.384+0000 7f6c19085740 10 client.27668 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:31:58.388 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.384+0000 7f6c19085740 10 client.27668 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:31:58.388 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.384+0000 7f6c19085740 20 client.27668 populate_metadata read hostname 'smithi114' 2024-09-13T18:31:58.388 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.384+0000 7f6c19085740 1 --2- 172.21.15.114:0/1067404884 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55d3e9cccef0 0x55d3e9ccf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:31:58.388 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.384+0000 7f6c19085740 4 client.27668 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T18:31:58.389 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.384+0000 7f6c19085740 1 -- 172.21.15.114:0/1067404884 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d3e9880300 con 0x55d3e9cccef0 2024-09-13T18:31:58.389 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.384+0000 7f6c0f7fe640 1 --2- 172.21.15.114:0/1067404884 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55d3e9cccef0 0x55d3e9ccf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:31:58.389 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.385+0000 7f6bf37fe640 10 client.27668 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:31:58.397 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.394+0000 7f6c0f7fe640 1 --2- 172.21.15.114:0/1067404884 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55d3e9cccef0 0x55d3e9ccf2e0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbe03a972ccf9643 server_cookie=1de4b4532c633af4 in_seq=0 out_seq=0 2024-09-13T18:31:58.520 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.518+0000 7f6bf37fe640 1 -- 172.21.15.114:0/1067404884 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d3e9880300 con 0x55d3e9cccef0 2024-09-13T18:31:58.520 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.518+0000 7f6bf37fe640 10 client.27668 handle_command_reply: tid=0 2024-09-13T18:31:58.520 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.518+0000 7f6c19085740 1 client.27668 shutdown 2024-09-13T18:31:58.523 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.521+0000 7f6c19085740 1 -- 172.21.15.114:0/1067404884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3e9c28560 msgr2=0x55d3e9c49270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:58.523 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.521+0000 7f6c19085740 1 --2- 172.21.15.114:0/1067404884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3e9c28560 0x55d3e9c49270 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6c00005750 tx=0x7f6c00005fd0 comp rx=0 tx=0).stop 2024-09-13T18:31:58.523 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.521+0000 7f6c19085740 1 -- 172.21.15.114:0/1067404884 shutdown_connections 2024-09-13T18:31:58.523 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.521+0000 7f6c19085740 1 --2- 172.21.15.114:0/1067404884 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55d3e9cccef0 0x55d3e9ccf2e0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:58.523 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.521+0000 7f6c19085740 1 --2- 172.21.15.114:0/1067404884 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d3e9c33720 0x55d3e9ca96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:58.523 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.521+0000 7f6c19085740 1 --2- 172.21.15.114:0/1067404884 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3e9c28e80 0x55d3e9c497b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:58.523 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.521+0000 7f6c19085740 1 --2- 172.21.15.114:0/1067404884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3e9c28560 0x55d3e9c49270 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:58.523 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.521+0000 7f6c19085740 1 -- 172.21.15.114:0/1067404884 >> 172.21.15.114:0/1067404884 conn(0x55d3e9c24af0 msgr2=0x55d3e9c2e970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.521+0000 7f6bf37fe640 0 client.27668 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:31:58.528 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.526+0000 7f6c19085740 1 -- 172.21.15.114:0/1067404884 shutdown_connections 2024-09-13T18:31:58.532 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.530+0000 7f6c19085740 1 -- 172.21.15.114:0/1067404884 wait complete. 2024-09-13T18:31:58.536 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.534+0000 7f6c19085740 20 client.27668 trim_cache size 0 max 16384 2024-09-13T18:31:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-13T18:31:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-13T18:31:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-13T18:31:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-13T18:31:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.536+0000 7f6c150fc640 1 -- 172.21.15.114:0/3266584614 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f6bf40a1410 msgr2=0x7f6bf40a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.536+0000 7f6c150fc640 1 --2- 172.21.15.114:0/3266584614 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f6bf40a1410 0x7f6bf40a38d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x55d3e9c245f0 tx=0x7f6c08009800 comp rx=0 tx=0).stop 2024-09-13T18:31:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.536+0000 7f6c150fc640 1 -- 172.21.15.114:0/3266584614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c101074b0 msgr2=0x7f6c101a4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:31:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.537+0000 7f6c150fc640 1 --2- 172.21.15.114:0/3266584614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c101074b0 0x7f6c101a4e90 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6c0800ba50 tx=0x7f6c0800bf10 comp rx=0 tx=0).stop 2024-09-13T18:31:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.539+0000 7f6c150fc640 1 -- 172.21.15.114:0/3266584614 shutdown_connections 2024-09-13T18:31:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.539+0000 7f6c150fc640 1 --2- 172.21.15.114:0/3266584614 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f6bf40a1410 0x7f6bf40a38d0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.539+0000 7f6c150fc640 1 --2- 172.21.15.114:0/3266584614 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c10119400 0x7f6c101a97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.539+0000 7f6c150fc640 1 --2- 172.21.15.114:0/3266584614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c10107890 0x7f6c101a53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.539+0000 7f6c150fc640 1 --2- 172.21.15.114:0/3266584614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c101074b0 0x7f6c101a4e90 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:31:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.539+0000 7f6c150fc640 1 -- 172.21.15.114:0/3266584614 >> 172.21.15.114:0/3266584614 conn(0x7f6c1006adf0 msgr2=0x7f6c10119c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:31:58.542 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.539+0000 7f6c150fc640 1 -- 172.21.15.114:0/3266584614 shutdown_connections 2024-09-13T18:31:58.542 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:31:58.540+0000 7f6c150fc640 1 -- 172.21.15.114:0/3266584614 wait complete. 2024-09-13T18:32:00.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:31:59 smithi134 ceph-mon[34283]: pgmap v294: 129 pgs: 129 active+clean; 22 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 25 MiB/s wr, 559 op/s 2024-09-13T18:32:00.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:31:59 smithi204 ceph-mon[34777]: pgmap v294: 129 pgs: 129 active+clean; 22 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 25 MiB/s wr, 559 op/s 2024-09-13T18:32:00.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:31:59 smithi114 ceph-mon[30897]: pgmap v294: 129 pgs: 129 active+clean; 22 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 25 MiB/s wr, 559 op/s 2024-09-13T18:32:02.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:01 smithi134 ceph-mon[34283]: pgmap v295: 129 pgs: 129 active+clean; 22 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 35 MiB/s wr, 787 op/s 2024-09-13T18:32:02.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:01 smithi134 ceph-mon[34283]: scrub summary: idle 2024-09-13T18:32:02.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:01 smithi134 ceph-mon[34283]: scrub summary: idle 2024-09-13T18:32:02.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:01 smithi134 ceph-mon[34283]: scrub summary: idle 2024-09-13T18:32:02.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:01 smithi204 ceph-mon[34777]: pgmap v295: 129 pgs: 129 active+clean; 22 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 35 MiB/s wr, 787 op/s 2024-09-13T18:32:02.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:01 smithi204 ceph-mon[34777]: scrub summary: idle 2024-09-13T18:32:02.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:01 smithi204 ceph-mon[34777]: scrub summary: idle 2024-09-13T18:32:02.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:01 smithi204 ceph-mon[34777]: scrub summary: idle 2024-09-13T18:32:02.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:01 smithi114 ceph-mon[30897]: pgmap v295: 129 pgs: 129 active+clean; 22 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 35 MiB/s wr, 787 op/s 2024-09-13T18:32:02.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:01 smithi114 ceph-mon[30897]: scrub summary: idle 2024-09-13T18:32:02.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:01 smithi114 ceph-mon[30897]: scrub summary: idle 2024-09-13T18:32:02.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:01 smithi114 ceph-mon[30897]: scrub summary: idle 2024-09-13T18:32:04.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:03 smithi134 ceph-mon[34283]: pgmap v296: 129 pgs: 129 active+clean; 22 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 29 MiB/s wr, 650 op/s 2024-09-13T18:32:04.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:03 smithi204 ceph-mon[34777]: pgmap v296: 129 pgs: 129 active+clean; 22 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 29 MiB/s wr, 650 op/s 2024-09-13T18:32:04.222 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:03 smithi114 ceph-mon[30897]: pgmap v296: 129 pgs: 129 active+clean; 22 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 29 MiB/s wr, 650 op/s 2024-09-13T18:32:06.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:05 smithi134 ceph-mon[34283]: pgmap v297: 129 pgs: 129 active+clean; 22 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 29 MiB/s wr, 643 op/s 2024-09-13T18:32:06.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:05 smithi204 ceph-mon[34777]: pgmap v297: 129 pgs: 129 active+clean; 22 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 29 MiB/s wr, 643 op/s 2024-09-13T18:32:06.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:05 smithi114 ceph-mon[30897]: pgmap v297: 129 pgs: 129 active+clean; 22 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 29 MiB/s wr, 643 op/s 2024-09-13T18:32:08.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:07 smithi134 ceph-mon[34283]: pgmap v298: 129 pgs: 129 active+clean; 22 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 35 MiB/s wr, 808 op/s 2024-09-13T18:32:08.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:07 smithi134 ceph-mon[34283]: osdmap e101: 12 total, 12 up, 12 in 2024-09-13T18:32:08.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:07 smithi204 ceph-mon[34777]: pgmap v298: 129 pgs: 129 active+clean; 22 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 35 MiB/s wr, 808 op/s 2024-09-13T18:32:08.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:07 smithi204 ceph-mon[34777]: osdmap e101: 12 total, 12 up, 12 in 2024-09-13T18:32:08.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:07 smithi114 ceph-mon[30897]: pgmap v298: 129 pgs: 129 active+clean; 22 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 35 MiB/s wr, 808 op/s 2024-09-13T18:32:08.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:07 smithi114 ceph-mon[30897]: osdmap e101: 12 total, 12 up, 12 in 2024-09-13T18:32:10.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:09 smithi134 ceph-mon[34283]: pgmap v300: 129 pgs: 129 active+clean; 22 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 30 MiB/s wr, 697 op/s 2024-09-13T18:32:10.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:09 smithi204 ceph-mon[34777]: pgmap v300: 129 pgs: 129 active+clean; 22 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 30 MiB/s wr, 697 op/s 2024-09-13T18:32:10.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:09 smithi114 ceph-mon[30897]: pgmap v300: 129 pgs: 129 active+clean; 22 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 30 MiB/s wr, 697 op/s 2024-09-13T18:32:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.698+0000 7fed470b6640 1 -- 172.21.15.114:0/1905518828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed40119400 msgr2=0x7fed4011b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.698+0000 7fed470b6640 1 --2- 172.21.15.114:0/1905518828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed40119400 0x7fed4011b7f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fed3c009860 tx=0x7fed3c02a380 comp rx=0 tx=0).stop 2024-09-13T18:32:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.701+0000 7fed470b6640 1 -- 172.21.15.114:0/1905518828 shutdown_connections 2024-09-13T18:32:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.701+0000 7fed470b6640 1 --2- 172.21.15.114:0/1905518828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed40119400 0x7fed4011b7f0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.701+0000 7fed470b6640 1 --2- 172.21.15.114:0/1905518828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed40107890 0x7fed40107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.701+0000 7fed470b6640 1 --2- 172.21.15.114:0/1905518828 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed401074b0 0x7fed4010e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.701+0000 7fed470b6640 1 -- 172.21.15.114:0/1905518828 >> 172.21.15.114:0/1905518828 conn(0x7fed4006adb0 msgr2=0x7fed4006b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.706+0000 7fed470b6640 1 -- 172.21.15.114:0/1905518828 shutdown_connections 2024-09-13T18:32:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.707+0000 7fed470b6640 1 -- 172.21.15.114:0/1905518828 wait complete. 2024-09-13T18:32:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.708+0000 7fed470b6640 1 Processor -- start 2024-09-13T18:32:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.708+0000 7fed470b6640 1 -- start start 2024-09-13T18:32:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.709+0000 7fed470b6640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed401074b0 0x7fed40116850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.709+0000 7fed470b6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed40107890 0x7fed40116db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.709+0000 7fed470b6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed40119400 0x7fed4010fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.709+0000 7fed470b6640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed401101d0 con 0x7fed401074b0 2024-09-13T18:32:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.709+0000 7fed470b6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed40110340 con 0x7fed40107890 2024-09-13T18:32:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.709+0000 7fed470b6640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed40110500 con 0x7fed40119400 2024-09-13T18:32:11.714 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.710+0000 7fed460b4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed401074b0 0x7fed40116850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:11.714 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.710+0000 7fed460b4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed401074b0 0x7fed40116850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46864/0 (socket says 172.21.15.114:46864) 2024-09-13T18:32:11.714 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.710+0000 7fed460b4640 1 -- 172.21.15.114:0/2628598973 learned_addr learned my addr 172.21.15.114:0/2628598973 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:11.714 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.711+0000 7fed458b3640 1 --2- 172.21.15.114:0/2628598973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed40107890 0x7fed40116db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.713+0000 7fed460b4640 1 -- 172.21.15.114:0/2628598973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed40119400 msgr2=0x7fed4010fa40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:32:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.713+0000 7fed460b4640 1 --2- 172.21.15.114:0/2628598973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed40119400 0x7fed4010fa40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.713+0000 7fed460b4640 1 -- 172.21.15.114:0/2628598973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed40107890 msgr2=0x7fed40116db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.713+0000 7fed460b4640 1 --2- 172.21.15.114:0/2628598973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed40107890 0x7fed40116db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.713+0000 7fed460b4640 1 -- 172.21.15.114:0/2628598973 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed3c009490 con 0x7fed401074b0 2024-09-13T18:32:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.713+0000 7fed458b3640 1 --2- 172.21.15.114:0/2628598973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed40107890 0x7fed40116db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:32:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.717+0000 7fed460b4640 1 --2- 172.21.15.114:0/2628598973 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed401074b0 0x7fed40116850 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fed3800efb0 tx=0x7fed3800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.723+0000 7fed377fe640 1 -- 172.21.15.114:0/2628598973 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3800b570 con 0x7fed401074b0 2024-09-13T18:32:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.723+0000 7fed377fe640 1 -- 172.21.15.114:0/2628598973 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fed3800bb60 con 0x7fed401074b0 2024-09-13T18:32:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.723+0000 7fed377fe640 1 -- 172.21.15.114:0/2628598973 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed38016b30 con 0x7fed401074b0 2024-09-13T18:32:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.724+0000 7fed470b6640 1 -- 172.21.15.114:0/2628598973 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed401143e0 con 0x7fed401074b0 2024-09-13T18:32:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.724+0000 7fed470b6640 1 -- 172.21.15.114:0/2628598973 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed401148f0 con 0x7fed401074b0 2024-09-13T18:32:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.727+0000 7fed4b036740 1 Processor -- start 2024-09-13T18:32:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.727+0000 7fed4b036740 1 -- start start 2024-09-13T18:32:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.727+0000 7fed4b036740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a132e29500 0x55a132e298e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.727+0000 7fed4b036740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a132e29e20 0x55a132e34180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.727+0000 7fed4b036740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a132e346c0 0x55a132e36ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.728+0000 7fed4b036740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a132d94c30 con 0x55a132e29e20 2024-09-13T18:32:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.728+0000 7fed4b036740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a132d923c0 con 0x55a132e29500 2024-09-13T18:32:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.728+0000 7fed4b036740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a132d8dcb0 con 0x55a132e346c0 2024-09-13T18:32:11.732 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.728+0000 7fed468b5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a132e29500 0x55a132e298e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:11.732 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.728+0000 7fed468b5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a132e29500 0x55a132e298e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:46972/0 (socket says 172.21.15.114:46972) 2024-09-13T18:32:11.732 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.728+0000 7fed468b5640 1 -- 172.21.15.114:0/2782695486 learned_addr learned my addr 172.21.15.114:0/2782695486 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.730+0000 7fed460b4640 1 --2- 172.21.15.114:0/2782695486 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a132e29e20 0x55a132e34180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.732+0000 7fed458b3640 1 --2- 172.21.15.114:0/2782695486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a132e346c0 0x55a132e36ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.732+0000 7fed468b5640 1 -- 172.21.15.114:0/2782695486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a132e346c0 msgr2=0x55a132e36ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.732+0000 7fed468b5640 1 --2- 172.21.15.114:0/2782695486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a132e346c0 0x55a132e36ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.732+0000 7fed468b5640 1 -- 172.21.15.114:0/2782695486 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a132e29e20 msgr2=0x55a132e34180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.732+0000 7fed468b5640 1 --2- 172.21.15.114:0/2782695486 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a132e29e20 0x55a132e34180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.732+0000 7fed468b5640 1 -- 172.21.15.114:0/2782695486 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a132d34910 con 0x55a132e29500 2024-09-13T18:32:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.732+0000 7fed468b5640 1 --2- 172.21.15.114:0/2782695486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a132e29500 0x55a132e298e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fed3c02a8f0 tx=0x7fed3c005760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=178e7281b3c5f965 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.733+0000 7fed377fe640 1 -- 172.21.15.114:0/2628598973 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fed3800a040 con 0x7fed401074b0 2024-09-13T18:32:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.734+0000 7fed377fe640 1 --2- 172.21.15.114:0/2628598973 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fed2c0a14d0 0x7fed2c0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.734+0000 7fed377fe640 1 -- 172.21.15.114:0/2628598973 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fed380d2f20 con 0x7fed401074b0 2024-09-13T18:32:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.736+0000 7fed460b4640 1 --2- 172.21.15.114:0/2628598973 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fed2c0a14d0 0x7fed2c0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.737+0000 7fed34e05640 1 -- 172.21.15.114:0/2782695486 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3c032da0 con 0x55a132e29500 2024-09-13T18:32:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.737+0000 7fed34e05640 1 -- 172.21.15.114:0/2782695486 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fed3c032440 con 0x55a132e29500 2024-09-13T18:32:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.737+0000 7fed34e05640 1 -- 172.21.15.114:0/2782695486 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3c030970 con 0x55a132e29500 2024-09-13T18:32:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.737+0000 7fed460b4640 1 --2- 172.21.15.114:0/2628598973 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fed2c0a14d0 0x7fed2c0a3990 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x55a132e25590 tx=0x7fed3809be80 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.738+0000 7fed4b036740 1 -- 172.21.15.114:0/2782695486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a132e29500 msgr2=0x55a132e298e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.738+0000 7fed4b036740 1 --2- 172.21.15.114:0/2782695486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a132e29500 0x55a132e298e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fed3c02a8f0 tx=0x7fed3c005760 comp rx=0 tx=0).stop 2024-09-13T18:32:11.744 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.741+0000 7fed4b036740 1 -- 172.21.15.114:0/2782695486 shutdown_connections 2024-09-13T18:32:11.744 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.741+0000 7fed4b036740 1 --2- 172.21.15.114:0/2782695486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a132e346c0 0x55a132e36ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:11.744 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.741+0000 7fed4b036740 1 --2- 172.21.15.114:0/2782695486 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a132e29e20 0x55a132e34180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:11.744 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.741+0000 7fed4b036740 1 --2- 172.21.15.114:0/2782695486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a132e29500 0x55a132e298e0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:11.744 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.741+0000 7fed4b036740 1 -- 172.21.15.114:0/2782695486 >> 172.21.15.114:0/2782695486 conn(0x55a132e25a90 msgr2=0x55a132e25e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:11.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.747+0000 7fed4b036740 1 -- 172.21.15.114:0/2782695486 shutdown_connections 2024-09-13T18:32:11.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.749+0000 7fed4b036740 1 -- 172.21.15.114:0/2782695486 wait complete. 2024-09-13T18:32:11.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.749+0000 7fed4b036740 1 -- start start 2024-09-13T18:32:11.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.749+0000 7fed4b036740 1 Processor -- start 2024-09-13T18:32:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.750+0000 7fed4b036740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a132e29500 0x55a132e340d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.750+0000 7fed4b036740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a132e29e20 0x55a132e31300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.750+0000 7fed4b036740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a132e346c0 0x55a132e31910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.750+0000 7fed4b036740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a132d4be80 con 0x55a132e346c0 2024-09-13T18:32:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.750+0000 7fed4b036740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a132d4bb20 con 0x55a132e29e20 2024-09-13T18:32:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.750+0000 7fed4b036740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a132d11260 con 0x55a132e29500 2024-09-13T18:32:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.750+0000 7fed4b036740 10 client.0 _ll_register_callbacks cb 0x55a132d46df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:32:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.750+0000 7fed468b5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a132e29500 0x55a132e340d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.750+0000 7fed468b5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a132e29500 0x55a132e340d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:55488/0 (socket says 172.21.15.114:55488) 2024-09-13T18:32:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.750+0000 7fed468b5640 1 -- 172.21.15.114:0/1628831847 learned_addr learned my addr 172.21.15.114:0/1628831847 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.754+0000 7fed458b3640 1 --2- 172.21.15.114:0/1628831847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a132e29e20 0x55a132e31300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:11.762 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.755+0000 7fed468b5640 1 --2- 172.21.15.114:0/1628831847 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a132e346c0 0x55a132e31910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:11.762 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.755+0000 7fed468b5640 1 -- 172.21.15.114:0/1628831847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a132e29e20 msgr2=0x55a132e31300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:11.762 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.755+0000 7fed468b5640 1 --2- 172.21.15.114:0/1628831847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a132e29e20 0x55a132e31300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:11.762 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.755+0000 7fed468b5640 1 -- 172.21.15.114:0/1628831847 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a132e346c0 msgr2=0x55a132e31910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:11.762 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.755+0000 7fed468b5640 1 --2- 172.21.15.114:0/1628831847 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a132e346c0 0x55a132e31910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:11.762 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.755+0000 7fed468b5640 1 -- 172.21.15.114:0/1628831847 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed4011bfb0 con 0x55a132e29500 2024-09-13T18:32:11.765 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.760+0000 7fed468b5640 1 --2- 172.21.15.114:0/1628831847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a132e29500 0x55a132e340d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fed3c009860 tx=0x7fed3c030670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb5129db8de52f76 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:11.765 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.761+0000 7fed29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:32:11.771 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.765+0000 7fed29ffb640 1 -- 172.21.15.114:0/1628831847 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3c030d90 con 0x55a132e29500 2024-09-13T18:32:11.771 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.765+0000 7fed29ffb640 1 -- 172.21.15.114:0/1628831847 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fed3c004030 con 0x55a132e29500 2024-09-13T18:32:11.773 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.770+0000 7fed4b036740 1 -- 172.21.15.114:0/1628831847 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a132db6680 con 0x55a132e29500 2024-09-13T18:32:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.775+0000 7fed29ffb640 1 -- 172.21.15.114:0/1628831847 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed3c0041f0 con 0x55a132e29500 2024-09-13T18:32:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.776+0000 7fed4b036740 10 client.27616 fetch_fsmap learned FSMap version 34 2024-09-13T18:32:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.776+0000 7fed4b036740 1 -- 172.21.15.114:0/1628831847 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55a132d8c330 con 0x55a132e29500 2024-09-13T18:32:11.781 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.778+0000 7fed29ffb640 1 -- 172.21.15.114:0/1628831847 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a132db6680 con 0x55a132e29500 2024-09-13T18:32:11.782 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.778+0000 7fed29ffb640 1 -- 172.21.15.114:0/1628831847 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fed3c052030 con 0x55a132e29500 2024-09-13T18:32:11.782 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.778+0000 7fed4b036740 10 client.27616 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:32:11.782 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.778+0000 7fed4b036740 10 client.27616 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:32:11.782 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.778+0000 7fed4b036740 20 client.27616 populate_metadata read hostname 'smithi114' 2024-09-13T18:32:11.782 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.778+0000 7fed4b036740 1 --2- 172.21.15.114:0/1628831847 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55a132e43ef0 0x55a132e46310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:11.782 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.778+0000 7fed4b036740 4 client.27616 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:32:11.782 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.778+0000 7fed4b036740 1 -- 172.21.15.114:0/1628831847 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a132d815d0 con 0x55a132e43ef0 2024-09-13T18:32:11.783 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.780+0000 7fed458b3640 1 --2- 172.21.15.114:0/1628831847 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55a132e43ef0 0x55a132e46310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:11.787 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.784+0000 7fed458b3640 1 --2- 172.21.15.114:0/1628831847 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55a132e43ef0 0x55a132e46310 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afde2a5985e8144 server_cookie=140ac39e4d36fd87 in_seq=0 out_seq=0 2024-09-13T18:32:11.788 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.784+0000 7fed29ffb640 10 client.27616 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:32:11.792 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.789+0000 7fed29ffb640 1 -- 172.21.15.114:0/1628831847 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55a132d815d0 con 0x55a132e43ef0 2024-09-13T18:32:11.792 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.789+0000 7fed29ffb640 10 client.27616 handle_command_reply: tid=0 2024-09-13T18:32:11.792 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.789+0000 7fed4b036740 1 client.27616 shutdown 2024-09-13T18:32:11.804 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.800+0000 7fed4b036740 1 -- 172.21.15.114:0/1628831847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a132e29500 msgr2=0x55a132e340d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:11.804 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.800+0000 7fed4b036740 1 --2- 172.21.15.114:0/1628831847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a132e29500 0x55a132e340d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fed3c009860 tx=0x7fed3c030670 comp rx=0 tx=0).stop 2024-09-13T18:32:11.804 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.801+0000 7fed4b036740 1 -- 172.21.15.114:0/1628831847 shutdown_connections 2024-09-13T18:32:11.804 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.801+0000 7fed4b036740 1 --2- 172.21.15.114:0/1628831847 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55a132e43ef0 0x55a132e46310 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:11.804 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.801+0000 7fed4b036740 1 --2- 172.21.15.114:0/1628831847 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a132e346c0 0x55a132e31910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:11.804 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.801+0000 7fed4b036740 1 --2- 172.21.15.114:0/1628831847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a132e29e20 0x55a132e31300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:11.804 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.801+0000 7fed4b036740 1 --2- 172.21.15.114:0/1628831847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a132e29500 0x55a132e340d0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:11.804 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.801+0000 7fed4b036740 1 -- 172.21.15.114:0/1628831847 >> 172.21.15.114:0/1628831847 conn(0x55a132e25a90 msgr2=0x55a132e24d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:11.805 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.801+0000 7fed29ffb640 0 client.27616 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:32:11.805 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.802+0000 7fed4b036740 1 -- 172.21.15.114:0/1628831847 shutdown_connections 2024-09-13T18:32:11.809 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.806+0000 7fed4b036740 1 -- 172.21.15.114:0/1628831847 wait complete. 2024-09-13T18:32:11.811 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.808+0000 7fed4b036740 20 client.27616 trim_cache size 0 max 16384 2024-09-13T18:32:11.847 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.844+0000 7fed4b036740 1 Processor -- start 2024-09-13T18:32:11.852 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.847+0000 7fed4b036740 1 -- start start 2024-09-13T18:32:11.852 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.847+0000 7fed4b036740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a132e29500 0x55a132e33be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:11.853 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.847+0000 7fed4b036740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a132e29e20 0x55a132e2c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:11.853 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.847+0000 7fed4b036740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a132e346c0 0x55a132e2cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:11.853 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.847+0000 7fed4b036740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a132e31f10 con 0x55a132e29500 2024-09-13T18:32:11.853 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.847+0000 7fed4b036740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a132e31100 con 0x55a132e346c0 2024-09-13T18:32:11.853 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.847+0000 7fed4b036740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a132e258d0 con 0x55a132e29e20 2024-09-13T18:32:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.847+0000 7fed468b5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a132e29500 0x55a132e33be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.847+0000 7fed468b5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a132e29500 0x55a132e33be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46880/0 (socket says 172.21.15.114:46880) 2024-09-13T18:32:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.847+0000 7fed468b5640 1 -- 172.21.15.114:0/2094852165 learned_addr learned my addr 172.21.15.114:0/2094852165 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.849+0000 7fed468b5640 1 --2- 172.21.15.114:0/2094852165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a132e346c0 0x55a132e2cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.850+0000 7fed468b5640 1 -- 172.21.15.114:0/2094852165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a132e29e20 msgr2=0x55a132e2c5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:32:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.850+0000 7fed468b5640 1 --2- 172.21.15.114:0/2094852165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a132e29e20 0x55a132e2c5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.850+0000 7fed468b5640 1 -- 172.21.15.114:0/2094852165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a132e346c0 msgr2=0x55a132e2cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.850+0000 7fed468b5640 1 --2- 172.21.15.114:0/2094852165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a132e346c0 0x55a132e2cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.850+0000 7fed468b5640 1 -- 172.21.15.114:0/2094852165 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a132e26640 con 0x55a132e29500 2024-09-13T18:32:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.851+0000 7fed468b5640 1 --2- 172.21.15.114:0/2094852165 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a132e29500 0x55a132e33be0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fed3c003fb0 tx=0x7fed3c003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=335adffe58e4b30b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.856+0000 7fed35606640 1 -- 172.21.15.114:0/2094852165 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3c003d50 con 0x55a132e29500 2024-09-13T18:32:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.857+0000 7fed35606640 1 -- 172.21.15.114:0/2094852165 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fed3c0325b0 con 0x55a132e29500 2024-09-13T18:32:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.857+0000 7fed4b036740 1 -- 172.21.15.114:0/2094852165 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a132e29500 msgr2=0x55a132e33be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.857+0000 7fed4b036740 1 --2- 172.21.15.114:0/2094852165 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a132e29500 0x55a132e33be0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fed3c003fb0 tx=0x7fed3c003f40 comp rx=0 tx=0).stop 2024-09-13T18:32:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.859+0000 7fed4b036740 1 -- 172.21.15.114:0/2094852165 shutdown_connections 2024-09-13T18:32:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.859+0000 7fed4b036740 1 --2- 172.21.15.114:0/2094852165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a132e346c0 0x55a132e2cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.859+0000 7fed4b036740 1 --2- 172.21.15.114:0/2094852165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a132e29e20 0x55a132e2c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.859+0000 7fed4b036740 1 --2- 172.21.15.114:0/2094852165 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a132e29500 0x55a132e33be0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.859+0000 7fed4b036740 1 -- 172.21.15.114:0/2094852165 >> 172.21.15.114:0/2094852165 conn(0x55a132e25a90 msgr2=0x55a132e3fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.859+0000 7fed4b036740 1 -- 172.21.15.114:0/2094852165 shutdown_connections 2024-09-13T18:32:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.859+0000 7fed4b036740 1 -- 172.21.15.114:0/2094852165 wait complete. 2024-09-13T18:32:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.860+0000 7fed4b036740 1 -- start start 2024-09-13T18:32:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.860+0000 7fed4b036740 1 Processor -- start 2024-09-13T18:32:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.860+0000 7fed4b036740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a132e29500 0x55a132e3b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.860+0000 7fed4b036740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a132e29e20 0x55a132e3b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.860+0000 7fed4b036740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a132e346c0 0x55a132e37b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.860+0000 7fed4b036740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a132e38290 con 0x55a132e29e20 2024-09-13T18:32:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.860+0000 7fed4b036740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a132e38400 con 0x55a132e346c0 2024-09-13T18:32:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.860+0000 7fed4b036740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a132e385c0 con 0x55a132e29500 2024-09-13T18:32:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.860+0000 7fed4b036740 10 client.0 _ll_register_callbacks cb 0x7fed3c035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:32:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.861+0000 7fed468b5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a132e346c0 0x55a132e37b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.861+0000 7fed468b5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a132e346c0 0x55a132e37b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:47008/0 (socket says 172.21.15.114:47008) 2024-09-13T18:32:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.861+0000 7fed468b5640 1 -- 172.21.15.114:0/291306109 learned_addr learned my addr 172.21.15.114:0/291306109 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.861+0000 7fed468b5640 1 --2- 172.21.15.114:0/291306109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a132e29500 0x55a132e3b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.861+0000 7fed468b5640 1 -- 172.21.15.114:0/291306109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a132e29500 msgr2=0x55a132e3b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.861+0000 7fed468b5640 1 --2- 172.21.15.114:0/291306109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a132e29500 0x55a132e3b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.861+0000 7fed468b5640 1 -- 172.21.15.114:0/291306109 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a132e29e20 msgr2=0x55a132e3b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.861+0000 7fed468b5640 1 --2- 172.21.15.114:0/291306109 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a132e29e20 0x55a132e3b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.861+0000 7fed468b5640 1 -- 172.21.15.114:0/291306109 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a132d8c330 con 0x55a132e346c0 2024-09-13T18:32:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.862+0000 7fed468b5640 1 --2- 172.21.15.114:0/291306109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a132e346c0 0x55a132e37b90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fed3c006010 tx=0x7fed3c005970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e1b8e47edc7b200 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.862+0000 7fed29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:32:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.862+0000 7fed29ffb640 1 -- 172.21.15.114:0/291306109 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3c003d50 con 0x55a132e346c0 2024-09-13T18:32:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.862+0000 7fed29ffb640 1 -- 172.21.15.114:0/291306109 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fed3c000e40 con 0x55a132e346c0 2024-09-13T18:32:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.862+0000 7fed4b036740 1 -- 172.21.15.114:0/291306109 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fed3c032440 con 0x55a132e346c0 2024-09-13T18:32:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.862+0000 7fed29ffb640 1 -- 172.21.15.114:0/291306109 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed3c003790 con 0x55a132e346c0 2024-09-13T18:32:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.864+0000 7fed4b036740 10 client.27683 fetch_fsmap learned FSMap version 34 2024-09-13T18:32:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.864+0000 7fed4b036740 1 -- 172.21.15.114:0/291306109 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55a132d2e300 con 0x55a132e346c0 2024-09-13T18:32:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.864+0000 7fed29ffb640 1 -- 172.21.15.114:0/291306109 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fed3c032440 con 0x55a132e346c0 2024-09-13T18:32:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.864+0000 7fed29ffb640 1 -- 172.21.15.114:0/291306109 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fed3c05c040 con 0x55a132e346c0 2024-09-13T18:32:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.865+0000 7fed4b036740 10 client.27683 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:32:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.865+0000 7fed4b036740 10 client.27683 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:32:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.865+0000 7fed4b036740 20 client.27683 populate_metadata read hostname 'smithi114' 2024-09-13T18:32:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.865+0000 7fed4b036740 1 --2- 172.21.15.114:0/291306109 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55a132ebd240 0x55a132ebf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.865+0000 7fed4b036740 4 client.27683 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T18:32:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.865+0000 7fed4b036740 1 -- 172.21.15.114:0/291306109 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a132a81300 con 0x55a132ebd240 2024-09-13T18:32:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.871+0000 7fed458b3640 1 --2- 172.21.15.114:0/291306109 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55a132ebd240 0x55a132ebf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.885+0000 7fed458b3640 1 --2- 172.21.15.114:0/291306109 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55a132ebd240 0x55a132ebf630 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5192e7a441017d53 server_cookie=3e5c7af0d9a59fee in_seq=0 out_seq=0 2024-09-13T18:32:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.886+0000 7fed29ffb640 10 client.27683 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:32:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.902+0000 7fed29ffb640 1 -- 172.21.15.114:0/291306109 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a132a81300 con 0x55a132ebd240 2024-09-13T18:32:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.902+0000 7fed29ffb640 10 client.27683 handle_command_reply: tid=0 2024-09-13T18:32:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.902+0000 7fed4b036740 1 client.27683 shutdown 2024-09-13T18:32:11.913 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.906+0000 7fed4b036740 1 -- 172.21.15.114:0/291306109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a132e346c0 msgr2=0x55a132e37b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:11.913 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.906+0000 7fed4b036740 1 --2- 172.21.15.114:0/291306109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a132e346c0 0x55a132e37b90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fed3c006010 tx=0x7fed3c005970 comp rx=0 tx=0).stop 2024-09-13T18:32:11.918 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.914+0000 7fed4b036740 1 -- 172.21.15.114:0/291306109 shutdown_connections 2024-09-13T18:32:11.918 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.914+0000 7fed4b036740 1 --2- 172.21.15.114:0/291306109 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55a132ebd240 0x55a132ebf630 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:11.918 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.914+0000 7fed4b036740 1 --2- 172.21.15.114:0/291306109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a132e346c0 0x55a132e37b90 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:11.918 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.914+0000 7fed4b036740 1 --2- 172.21.15.114:0/291306109 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a132e29e20 0x55a132e3b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:11.918 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.914+0000 7fed4b036740 1 --2- 172.21.15.114:0/291306109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a132e29500 0x55a132e3b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:11.918 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.914+0000 7fed4b036740 1 -- 172.21.15.114:0/291306109 >> 172.21.15.114:0/291306109 conn(0x55a132e25a90 msgr2=0x55a132e3fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:11.918 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.914+0000 7fed29ffb640 0 client.27683 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:32:11.928 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.924+0000 7fed4b036740 1 -- 172.21.15.114:0/291306109 shutdown_connections 2024-09-13T18:32:11.930 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.925+0000 7fed4b036740 1 -- 172.21.15.114:0/291306109 wait complete. 2024-09-13T18:32:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.933+0000 7fed4b036740 20 client.27683 trim_cache size 0 max 16384 2024-09-13T18:32:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-13T18:32:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-13T18:32:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-13T18:32:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-13T18:32:11.945 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.939+0000 7fed470b6640 1 -- 172.21.15.114:0/2628598973 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fed2c0a14d0 msgr2=0x7fed2c0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:11.945 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.939+0000 7fed470b6640 1 --2- 172.21.15.114:0/2628598973 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fed2c0a14d0 0x7fed2c0a3990 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x55a132e25590 tx=0x7fed3809be80 comp rx=0 tx=0).stop 2024-09-13T18:32:11.945 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.939+0000 7fed470b6640 1 -- 172.21.15.114:0/2628598973 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed401074b0 msgr2=0x7fed40116850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:11.945 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.939+0000 7fed470b6640 1 --2- 172.21.15.114:0/2628598973 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed401074b0 0x7fed40116850 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fed3800efb0 tx=0x7fed3800c800 comp rx=0 tx=0).stop 2024-09-13T18:32:11.946 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.939+0000 7fed470b6640 1 -- 172.21.15.114:0/2628598973 shutdown_connections 2024-09-13T18:32:11.946 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.939+0000 7fed470b6640 1 --2- 172.21.15.114:0/2628598973 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fed2c0a14d0 0x7fed2c0a3990 secure :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0x55a132e25590 tx=0x7fed3809be80 comp rx=0 tx=0).stop 2024-09-13T18:32:11.946 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.939+0000 7fed470b6640 1 --2- 172.21.15.114:0/2628598973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed40119400 0x7fed4010fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:11.946 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.939+0000 7fed470b6640 1 --2- 172.21.15.114:0/2628598973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed40107890 0x7fed40116db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:11.946 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.939+0000 7fed470b6640 1 --2- 172.21.15.114:0/2628598973 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed401074b0 0x7fed40116850 secure :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fed3800efb0 tx=0x7fed3800c800 comp rx=0 tx=0).stop 2024-09-13T18:32:11.946 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.939+0000 7fed470b6640 1 -- 172.21.15.114:0/2628598973 >> 172.21.15.114:0/2628598973 conn(0x7fed4006adb0 msgr2=0x7fed40119a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:11.949 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.946+0000 7fed470b6640 1 -- 172.21.15.114:0/2628598973 shutdown_connections 2024-09-13T18:32:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:11.948+0000 7fed470b6640 1 -- 172.21.15.114:0/2628598973 wait complete. 2024-09-13T18:32:12.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:11 smithi134 ceph-mon[34283]: pgmap v301: 129 pgs: 14 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 79 active+clean; 22 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 26 MiB/s wr, 596 op/s 2024-09-13T18:32:12.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:11 smithi204 ceph-mon[34777]: pgmap v301: 129 pgs: 14 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 79 active+clean; 22 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 26 MiB/s wr, 596 op/s 2024-09-13T18:32:12.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:11 smithi114 ceph-mon[30897]: pgmap v301: 129 pgs: 14 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 79 active+clean; 22 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 26 MiB/s wr, 596 op/s 2024-09-13T18:32:14.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:13 smithi134 ceph-mon[34283]: pgmap v302: 129 pgs: 20 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 56 active+clean; 21 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 25 MiB/s wr, 580 op/s 2024-09-13T18:32:14.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:13 smithi204 ceph-mon[34777]: pgmap v302: 129 pgs: 20 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 56 active+clean; 21 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 25 MiB/s wr, 580 op/s 2024-09-13T18:32:14.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:13 smithi114 ceph-mon[30897]: pgmap v302: 129 pgs: 20 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 56 active+clean; 21 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 25 MiB/s wr, 580 op/s 2024-09-13T18:32:16.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:15 smithi134 ceph-mon[34283]: pgmap v303: 129 pgs: 15 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 69 active+clean; 21 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 24 MiB/s wr, 557 op/s 2024-09-13T18:32:16.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:15 smithi204 ceph-mon[34777]: pgmap v303: 129 pgs: 15 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 69 active+clean; 21 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 24 MiB/s wr, 557 op/s 2024-09-13T18:32:16.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:15 smithi114 ceph-mon[30897]: pgmap v303: 129 pgs: 15 active+clean+snaptrim, 45 active+clean+snaptrim_wait, 69 active+clean; 21 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 24 MiB/s wr, 557 op/s 2024-09-13T18:32:18.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:17 smithi204 ceph-mon[34777]: pgmap v304: 129 pgs: 5 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 112 active+clean; 20 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 20 MiB/s wr, 443 op/s 2024-09-13T18:32:18.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:17 smithi114 ceph-mon[30897]: pgmap v304: 129 pgs: 5 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 112 active+clean; 20 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 20 MiB/s wr, 443 op/s 2024-09-13T18:32:18.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:17 smithi134 ceph-mon[34283]: pgmap v304: 129 pgs: 5 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 112 active+clean; 20 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 20 MiB/s wr, 443 op/s 2024-09-13T18:32:20.171 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:19 smithi134 ceph-mon[34283]: pgmap v305: 129 pgs: 5 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 113 active+clean; 20 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 18 MiB/s wr, 418 op/s 2024-09-13T18:32:20.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:19 smithi204 ceph-mon[34777]: pgmap v305: 129 pgs: 5 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 113 active+clean; 20 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 18 MiB/s wr, 418 op/s 2024-09-13T18:32:20.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:19 smithi114 ceph-mon[30897]: pgmap v305: 129 pgs: 5 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 113 active+clean; 20 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 18 MiB/s wr, 418 op/s 2024-09-13T18:32:20.699 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T18:32:20.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.957+0000 7fd9f95b0640 1 -- 172.21.15.114:0/2235146948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9f410c2a0 msgr2=0x7fd9f410c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:20.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.957+0000 7fd9f95b0640 1 --2- 172.21.15.114:0/2235146948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9f410c2a0 0x7fd9f410c680 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec009cd0 tx=0x7fd9ec02a930 comp rx=0 tx=0).stop 2024-09-13T18:32:20.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.963+0000 7fd9f95b0640 1 -- 172.21.15.114:0/2235146948 shutdown_connections 2024-09-13T18:32:20.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.963+0000 7fd9f95b0640 1 --2- 172.21.15.114:0/2235146948 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9f4106c40 0x7fd9f41130d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:20.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.963+0000 7fd9f95b0640 1 --2- 172.21.15.114:0/2235146948 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9f41062e0 0x7fd9f4106700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:20.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.963+0000 7fd9f95b0640 1 --2- 172.21.15.114:0/2235146948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9f410c2a0 0x7fd9f410c680 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:20.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.963+0000 7fd9f95b0640 1 -- 172.21.15.114:0/2235146948 >> 172.21.15.114:0/2235146948 conn(0x7fd9f4075b80 msgr2=0x7fd9f4077fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:20.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.970+0000 7fd9f95b0640 1 -- 172.21.15.114:0/2235146948 shutdown_connections 2024-09-13T18:32:20.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.970+0000 7fd9f95b0640 1 -- 172.21.15.114:0/2235146948 wait complete. 2024-09-13T18:32:20.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.971+0000 7fd9f95b0640 1 Processor -- start 2024-09-13T18:32:20.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.972+0000 7fd9f95b0640 1 -- start start 2024-09-13T18:32:20.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.972+0000 7fd9f95b0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9f41062e0 0x7fd9f41a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:20.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.972+0000 7fd9f95b0640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9f4106c40 0x7fd9f41a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:20.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.972+0000 7fd9f95b0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9f410c2a0 0x7fd9f419c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:20.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.972+0000 7fd9f95b0640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f419cc20 con 0x7fd9f4106c40 2024-09-13T18:32:20.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.972+0000 7fd9f95b0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f419cd90 con 0x7fd9f41062e0 2024-09-13T18:32:20.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.972+0000 7fd9f95b0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f419cf50 con 0x7fd9f410c2a0 2024-09-13T18:32:20.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.974+0000 7fd9f3fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9f41062e0 0x7fd9f41a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:20.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.974+0000 7fd9f3fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9f41062e0 0x7fd9f41a2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:42310/0 (socket says 172.21.15.114:42310) 2024-09-13T18:32:20.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.974+0000 7fd9f3fff640 1 -- 172.21.15.114:0/516967946 learned_addr learned my addr 172.21.15.114:0/516967946 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:20.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.974+0000 7fd9f37fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9f4106c40 0x7fd9f41a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:20.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.974+0000 7fd9f8daf640 1 --2- 172.21.15.114:0/516967946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9f410c2a0 0x7fd9f419c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:20.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.975+0000 7fd9f3fff640 1 -- 172.21.15.114:0/516967946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9f410c2a0 msgr2=0x7fd9f419c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:20.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.975+0000 7fd9f3fff640 1 --2- 172.21.15.114:0/516967946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9f410c2a0 0x7fd9f419c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:20.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.975+0000 7fd9f3fff640 1 -- 172.21.15.114:0/516967946 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9f4106c40 msgr2=0x7fd9f41a2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:20.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.975+0000 7fd9f3fff640 1 --2- 172.21.15.114:0/516967946 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9f4106c40 0x7fd9f41a2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:20.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.975+0000 7fd9f3fff640 1 -- 172.21.15.114:0/516967946 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9ec009900 con 0x7fd9f41062e0 2024-09-13T18:32:20.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.976+0000 7fd9f3fff640 1 --2- 172.21.15.114:0/516967946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9f41062e0 0x7fd9f41a2520 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec031a20 tx=0x7fd9ec02f410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:20.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.977+0000 7fd9f17fa640 1 -- 172.21.15.114:0/516967946 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec030bc0 con 0x7fd9f41062e0 2024-09-13T18:32:20.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.977+0000 7fd9f17fa640 1 -- 172.21.15.114:0/516967946 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd9ec03f070 con 0x7fd9f41062e0 2024-09-13T18:32:20.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.977+0000 7fd9f17fa640 1 -- 172.21.15.114:0/516967946 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec004850 con 0x7fd9f41062e0 2024-09-13T18:32:20.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.980+0000 7fd9f95b0640 1 -- 172.21.15.114:0/516967946 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9f419d1d0 con 0x7fd9f41062e0 2024-09-13T18:32:20.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.980+0000 7fd9f95b0640 1 -- 172.21.15.114:0/516967946 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9f4078ee0 con 0x7fd9f41062e0 2024-09-13T18:32:20.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.984+0000 7fd9fd613740 1 Processor -- start 2024-09-13T18:32:20.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.989+0000 7fd9fd613740 1 -- start start 2024-09-13T18:32:20.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.990+0000 7fd9fd613740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55665ccd4030 0x55665ccd4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:20.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.990+0000 7fd9fd613740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55665ccd4950 0x55665ccdecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:20.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.990+0000 7fd9fd613740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55665ccdf1f0 0x55665cce15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:20.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.990+0000 7fd9fd613740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55665cc3ef20 con 0x55665ccdf1f0 2024-09-13T18:32:20.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.990+0000 7fd9fd613740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55665cc3c6b0 con 0x55665ccd4950 2024-09-13T18:32:20.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.990+0000 7fd9fd613740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55665cc37fa0 con 0x55665ccd4030 2024-09-13T18:32:20.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.991+0000 7fd9f3fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55665ccd4950 0x55665ccdecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:20.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.991+0000 7fd9f3fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55665ccd4950 0x55665ccdecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:42314/0 (socket says 172.21.15.114:42314) 2024-09-13T18:32:20.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.991+0000 7fd9f3fff640 1 -- 172.21.15.114:0/1493364543 learned_addr learned my addr 172.21.15.114:0/1493364543 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:20.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.992+0000 7fd9f17fa640 1 -- 172.21.15.114:0/516967946 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fd9ec00a040 con 0x7fd9f41062e0 2024-09-13T18:32:20.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.993+0000 7fd9f37fe640 1 --2- 172.21.15.114:0/1493364543 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55665ccdf1f0 0x55665cce15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:20.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.994+0000 7fd9f3fff640 1 -- 172.21.15.114:0/1493364543 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55665ccd4030 msgr2=0x55665ccd4410 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:32:20.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.994+0000 7fd9f3fff640 1 --2- 172.21.15.114:0/1493364543 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55665ccd4030 0x55665ccd4410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:20.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.994+0000 7fd9f3fff640 1 -- 172.21.15.114:0/1493364543 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55665ccdf1f0 msgr2=0x55665cce15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:20.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.994+0000 7fd9f3fff640 1 --2- 172.21.15.114:0/1493364543 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55665ccdf1f0 0x55665cce15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:20.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.994+0000 7fd9f3fff640 1 -- 172.21.15.114:0/1493364543 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55665cbf2840 con 0x55665ccd4950 2024-09-13T18:32:21.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.995+0000 7fd9f17fa640 1 --2- 172.21.15.114:0/516967946 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fd9d80a1340 0x7fd9d80a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:21.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.995+0000 7fd9f17fa640 1 -- 172.21.15.114:0/516967946 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fd9ec041070 con 0x7fd9f41062e0 2024-09-13T18:32:21.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.995+0000 7fd9f3fff640 1 --2- 172.21.15.114:0/1493364543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55665ccd4950 0x55665ccdecb0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec0ebb90 tx=0x7fd9ec0f4e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58ac9c0975bf58cd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:21.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.995+0000 7fd9ea7fc640 1 -- 172.21.15.114:0/1493364543 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec0db0d0 con 0x55665ccd4950 2024-09-13T18:32:21.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.996+0000 7fd9ea7fc640 1 -- 172.21.15.114:0/1493364543 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd9ec0b7950 con 0x55665ccd4950 2024-09-13T18:32:21.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.996+0000 7fd9f8daf640 1 --2- 172.21.15.114:0/516967946 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fd9d80a1340 0x7fd9d80a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:21.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.997+0000 7fd9fd613740 1 -- 172.21.15.114:0/1493364543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55665ccd4950 msgr2=0x55665ccdecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:21.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.997+0000 7fd9fd613740 1 --2- 172.21.15.114:0/1493364543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55665ccd4950 0x55665ccdecb0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec0ebb90 tx=0x7fd9ec0f4e30 comp rx=0 tx=0).stop 2024-09-13T18:32:21.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:20.997+0000 7fd9f8daf640 1 --2- 172.21.15.114:0/516967946 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fd9d80a1340 0x7fd9d80a3800 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd9e000a700 tx=0x7fd9e0008040 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:21.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.001+0000 7fd9fd613740 1 -- 172.21.15.114:0/1493364543 shutdown_connections 2024-09-13T18:32:21.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.001+0000 7fd9fd613740 1 --2- 172.21.15.114:0/1493364543 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55665ccdf1f0 0x55665cce15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:21.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.001+0000 7fd9fd613740 1 --2- 172.21.15.114:0/1493364543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55665ccd4950 0x55665ccdecb0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:21.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.001+0000 7fd9fd613740 1 --2- 172.21.15.114:0/1493364543 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55665ccd4030 0x55665ccd4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:21.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.001+0000 7fd9fd613740 1 -- 172.21.15.114:0/1493364543 >> 172.21.15.114:0/1493364543 conn(0x55665ccd05a0 msgr2=0x55665ccd0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:21.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.003+0000 7fd9fd613740 1 -- 172.21.15.114:0/1493364543 shutdown_connections 2024-09-13T18:32:21.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.003+0000 7fd9fd613740 1 -- 172.21.15.114:0/1493364543 wait complete. 2024-09-13T18:32:21.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.004+0000 7fd9fd613740 1 -- start start 2024-09-13T18:32:21.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.004+0000 7fd9fd613740 1 Processor -- start 2024-09-13T18:32:21.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.006+0000 7fd9fd613740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55665ccd4030 0x55665ccde9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:21.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.006+0000 7fd9fd613740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55665ccd4950 0x55665ccdbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:21.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.006+0000 7fd9fd613740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55665ccdf1f0 0x55665ccdc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:21.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.006+0000 7fd9fd613740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55665cbd92a0 con 0x55665ccd4950 2024-09-13T18:32:21.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.006+0000 7fd9fd613740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55665c910430 con 0x55665ccd4030 2024-09-13T18:32:21.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.006+0000 7fd9fd613740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55665cbbb680 con 0x55665ccdf1f0 2024-09-13T18:32:21.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.006+0000 7fd9fd613740 10 client.0 _ll_register_callbacks cb 0x55665cc31610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:32:21.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.006+0000 7fd9f8daf640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55665ccdf1f0 0x55665ccdc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:21.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.006+0000 7fd9f8daf640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55665ccdf1f0 0x55665ccdc200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:49202/0 (socket says 172.21.15.114:49202) 2024-09-13T18:32:21.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.006+0000 7fd9f8daf640 1 -- 172.21.15.114:0/337032338 learned_addr learned my addr 172.21.15.114:0/337032338 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:21.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.007+0000 7fd9f3fff640 1 --2- 172.21.15.114:0/337032338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55665ccd4030 0x55665ccde9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:21.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.008+0000 7fd9f3fff640 1 -- 172.21.15.114:0/337032338 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55665ccdf1f0 msgr2=0x55665ccdc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:21.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.008+0000 7fd9f3fff640 1 --2- 172.21.15.114:0/337032338 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55665ccdf1f0 0x55665ccdc200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:21.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.008+0000 7fd9f3fff640 1 -- 172.21.15.114:0/337032338 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55665ccd4950 msgr2=0x55665ccdbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:21.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.008+0000 7fd9f3fff640 1 --2- 172.21.15.114:0/337032338 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55665ccd4950 0x55665ccdbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:21.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.008+0000 7fd9f3fff640 1 -- 172.21.15.114:0/337032338 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9f4078ee0 con 0x55665ccd4030 2024-09-13T18:32:21.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.009+0000 7fd9f3fff640 1 --2- 172.21.15.114:0/337032338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55665ccd4030 0x55665ccde9a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x55665ccd1400 tx=0x7fd9ec0a62e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16b35c645822057 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:21.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.011+0000 7fd9d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:32:21.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.011+0000 7fd9d77fe640 1 -- 172.21.15.114:0/337032338 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec0bb150 con 0x55665ccd4030 2024-09-13T18:32:21.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.011+0000 7fd9d77fe640 1 -- 172.21.15.114:0/337032338 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fd9ec0bbd10 con 0x55665ccd4030 2024-09-13T18:32:21.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.011+0000 7fd9fd613740 1 -- 172.21.15.114:0/337032338 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55665c92b4e0 con 0x55665ccd4030 2024-09-13T18:32:21.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.012+0000 7fd9d77fe640 1 -- 172.21.15.114:0/337032338 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9ec0db0d0 con 0x55665ccd4030 2024-09-13T18:32:21.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.012+0000 7fd9fd613740 10 client.27695 fetch_fsmap learned FSMap version 34 2024-09-13T18:32:21.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.012+0000 7fd9fd613740 1 -- 172.21.15.114:0/337032338 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55665cc36620 con 0x55665ccd4030 2024-09-13T18:32:21.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.016+0000 7fd9d77fe640 1 -- 172.21.15.114:0/337032338 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd9ec0d3070 con 0x55665ccd4030 2024-09-13T18:32:21.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.016+0000 7fd9d77fe640 1 -- 172.21.15.114:0/337032338 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd9ec0bab00 con 0x55665ccd4030 2024-09-13T18:32:21.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.017+0000 7fd9fd613740 10 client.27695 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:32:21.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.017+0000 7fd9fd613740 10 client.27695 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:32:21.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.018+0000 7fd9fd613740 20 client.27695 populate_metadata read hostname 'smithi114' 2024-09-13T18:32:21.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.018+0000 7fd9fd613740 1 --2- 172.21.15.114:0/337032338 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55665ccff710 0x55665cd01b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:21.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.018+0000 7fd9fd613740 4 client.27695 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:32:21.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.018+0000 7fd9fd613740 1 -- 172.21.15.114:0/337032338 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55665c93c7e0 con 0x55665ccff710 2024-09-13T18:32:21.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.019+0000 7fd9f37fe640 1 --2- 172.21.15.114:0/337032338 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55665ccff710 0x55665cd01b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:21.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.019+0000 7fd9f37fe640 1 --2- 172.21.15.114:0/337032338 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55665ccff710 0x55665cd01b00 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dee5b501fa09cdb4 server_cookie=511ba7c318d8a129 in_seq=0 out_seq=0 2024-09-13T18:32:21.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.019+0000 7fd9d77fe640 10 client.27695 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:32:21.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.022+0000 7fd9d77fe640 1 -- 172.21.15.114:0/337032338 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55665c93c7e0 con 0x55665ccff710 2024-09-13T18:32:21.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.022+0000 7fd9d77fe640 10 client.27695 handle_command_reply: tid=0 2024-09-13T18:32:21.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.022+0000 7fd9fd613740 1 client.27695 shutdown 2024-09-13T18:32:21.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.023+0000 7fd9fd613740 1 -- 172.21.15.114:0/337032338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55665ccd4030 msgr2=0x55665ccde9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:21.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.023+0000 7fd9fd613740 1 --2- 172.21.15.114:0/337032338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55665ccd4030 0x55665ccde9a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x55665ccd1400 tx=0x7fd9ec0a62e0 comp rx=0 tx=0).stop 2024-09-13T18:32:21.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.030+0000 7fd9fd613740 1 -- 172.21.15.114:0/337032338 shutdown_connections 2024-09-13T18:32:21.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.030+0000 7fd9fd613740 1 --2- 172.21.15.114:0/337032338 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55665ccff710 0x55665cd01b00 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:21.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.030+0000 7fd9fd613740 1 --2- 172.21.15.114:0/337032338 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55665ccdf1f0 0x55665ccdc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:21.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.030+0000 7fd9fd613740 1 --2- 172.21.15.114:0/337032338 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55665ccd4950 0x55665ccdbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:21.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.030+0000 7fd9fd613740 1 --2- 172.21.15.114:0/337032338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55665ccd4030 0x55665ccde9a0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:21.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.030+0000 7fd9fd613740 1 -- 172.21.15.114:0/337032338 >> 172.21.15.114:0/337032338 conn(0x55665ccd05a0 msgr2=0x55665ccd62b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:21.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.031+0000 7fd9d77fe640 0 client.27695 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:32:21.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.033+0000 7fd9fd613740 1 -- 172.21.15.114:0/337032338 shutdown_connections 2024-09-13T18:32:21.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.037+0000 7fd9fd613740 1 -- 172.21.15.114:0/337032338 wait complete. 2024-09-13T18:32:21.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.041+0000 7fd9fd613740 20 client.27695 trim_cache size 0 max 16384 2024-09-13T18:32:21.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.084+0000 7fd9fd613740 1 Processor -- start 2024-09-13T18:32:21.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.084+0000 7fd9fd613740 1 -- start start 2024-09-13T18:32:21.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.085+0000 7fd9fd613740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55665ccd4030 0x55665cd06430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:21.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.085+0000 7fd9fd613740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55665ccd4950 0x55665ccf6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:21.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.085+0000 7fd9fd613740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55665ccdf1f0 0x55665ccf72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:21.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.085+0000 7fd9f3fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55665ccd4030 0x55665cd06430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:21.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.085+0000 7fd9f3fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55665ccd4030 0x55665cd06430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:42328/0 (socket says 172.21.15.114:42328) 2024-09-13T18:32:21.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.086+0000 7fd9fd613740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55665cce1910 con 0x55665ccdf1f0 2024-09-13T18:32:21.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.086+0000 7fd9fd613740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55665ccdcbc0 con 0x55665ccd4030 2024-09-13T18:32:21.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.086+0000 7fd9fd613740 1 -- 172.21.15.114:0/2807087736 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55665ccd0380 con 0x55665ccd4950 2024-09-13T18:32:21.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.086+0000 7fd9f3fff640 1 -- 172.21.15.114:0/2807087736 learned_addr learned my addr 172.21.15.114:0/2807087736 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:21.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.086+0000 7fd9f37fe640 1 --2- 172.21.15.114:0/2807087736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55665ccd4950 0x55665ccf6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:21.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.086+0000 7fd9f3fff640 1 -- 172.21.15.114:0/2807087736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55665ccd4950 msgr2=0x55665ccf6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:21.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.086+0000 7fd9f3fff640 1 --2- 172.21.15.114:0/2807087736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55665ccd4950 0x55665ccf6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:21.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.086+0000 7fd9f3fff640 1 -- 172.21.15.114:0/2807087736 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55665ccdf1f0 msgr2=0x55665ccf72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:21.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.086+0000 7fd9f3fff640 1 --2- 172.21.15.114:0/2807087736 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55665ccdf1f0 0x55665ccf72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:21.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.086+0000 7fd9f3fff640 1 -- 172.21.15.114:0/2807087736 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55665cbd8790 con 0x55665ccd4030 2024-09-13T18:32:21.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.088+0000 7fd9f3fff640 1 --2- 172.21.15.114:0/2807087736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55665ccd4030 0x55665cd06430 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec009ca0 tx=0x7fd9ec0043c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7ca67793377df56 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:21.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.090+0000 7fd9f37fe640 1 --2- 172.21.15.114:0/2807087736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55665ccd4950 0x55665ccf6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:32:21.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.094+0000 7fd9eaffd640 1 -- 172.21.15.114:0/2807087736 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec0bb150 con 0x55665ccd4030 2024-09-13T18:32:21.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.094+0000 7fd9eaffd640 1 -- 172.21.15.114:0/2807087736 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd9ec0d3020 con 0x55665ccd4030 2024-09-13T18:32:21.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.095+0000 7fd9fd613740 1 -- 172.21.15.114:0/2807087736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55665ccd4030 msgr2=0x55665cd06430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:21.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.095+0000 7fd9fd613740 1 --2- 172.21.15.114:0/2807087736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55665ccd4030 0x55665cd06430 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec009ca0 tx=0x7fd9ec0043c0 comp rx=0 tx=0).stop 2024-09-13T18:32:21.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.101+0000 7fd9fd613740 1 -- 172.21.15.114:0/2807087736 shutdown_connections 2024-09-13T18:32:21.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.101+0000 7fd9fd613740 1 --2- 172.21.15.114:0/2807087736 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55665ccdf1f0 0x55665ccf72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:21.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.101+0000 7fd9fd613740 1 --2- 172.21.15.114:0/2807087736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55665ccd4950 0x55665ccf6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:21.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.101+0000 7fd9fd613740 1 --2- 172.21.15.114:0/2807087736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55665ccd4030 0x55665cd06430 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:21.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.101+0000 7fd9fd613740 1 -- 172.21.15.114:0/2807087736 >> 172.21.15.114:0/2807087736 conn(0x55665ccd05a0 msgr2=0x55665ccde600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:21.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.107+0000 7fd9fd613740 1 -- 172.21.15.114:0/2807087736 shutdown_connections 2024-09-13T18:32:21.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.116+0000 7fd9fd613740 1 -- 172.21.15.114:0/2807087736 wait complete. 2024-09-13T18:32:21.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.116+0000 7fd9fd613740 1 -- start start 2024-09-13T18:32:21.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.116+0000 7fd9fd613740 1 Processor -- start 2024-09-13T18:32:21.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.117+0000 7fd9fd613740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55665ccd4030 0x55665ccf4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:21.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.117+0000 7fd9fd613740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55665ccd4950 0x55665ccf5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:21.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.117+0000 7fd9fd613740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55665ccdf1f0 0x55665cd54bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:21.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.117+0000 7fd9fd613740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55665ccf5b90 con 0x55665ccd4030 2024-09-13T18:32:21.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.117+0000 7fd9fd613740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55665cd550f0 con 0x55665ccd4950 2024-09-13T18:32:21.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.117+0000 7fd9fd613740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55665cd55230 con 0x55665ccdf1f0 2024-09-13T18:32:21.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.117+0000 7fd9fd613740 10 client.0 _ll_register_callbacks cb 0x7fd9e40093d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:32:21.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.117+0000 7fd9f8daf640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55665ccdf1f0 0x55665cd54bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:21.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.117+0000 7fd9f8daf640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55665ccdf1f0 0x55665cd54bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:49210/0 (socket says 172.21.15.114:49210) 2024-09-13T18:32:21.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.117+0000 7fd9f8daf640 1 -- 172.21.15.114:0/1982536313 learned_addr learned my addr 172.21.15.114:0/1982536313 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:21.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.117+0000 7fd9f37fe640 1 --2- 172.21.15.114:0/1982536313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55665ccd4950 0x55665ccf5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:21.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.118+0000 7fd9f8daf640 1 -- 172.21.15.114:0/1982536313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55665ccd4950 msgr2=0x55665ccf5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:21.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.118+0000 7fd9f8daf640 1 --2- 172.21.15.114:0/1982536313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55665ccd4950 0x55665ccf5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:21.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.118+0000 7fd9f8daf640 1 -- 172.21.15.114:0/1982536313 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55665ccd4030 msgr2=0x55665ccf4d50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:32:21.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.118+0000 7fd9f8daf640 1 --2- 172.21.15.114:0/1982536313 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55665ccd4030 0x55665ccf4d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:21.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.118+0000 7fd9f8daf640 1 -- 172.21.15.114:0/1982536313 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55665cc36620 con 0x55665ccdf1f0 2024-09-13T18:32:21.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.118+0000 7fd9f8daf640 1 --2- 172.21.15.114:0/1982536313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55665ccdf1f0 0x55665cd54bb0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd9e000cd10 tx=0x7fd9e00137d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac7e319b6fdfa793 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:21.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.119+0000 7fd9d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:32:21.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.119+0000 7fd9d77fe640 1 -- 172.21.15.114:0/1982536313 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e001f470 con 0x55665ccdf1f0 2024-09-13T18:32:21.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.119+0000 7fd9d77fe640 1 -- 172.21.15.114:0/1982536313 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fd9e001f5d0 con 0x55665ccdf1f0 2024-09-13T18:32:21.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.119+0000 7fd9fd613740 1 -- 172.21.15.114:0/1982536313 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd9ec0b7950 con 0x55665ccdf1f0 2024-09-13T18:32:21.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.123+0000 7fd9d77fe640 1 -- 172.21.15.114:0/1982536313 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9e001f790 con 0x55665ccdf1f0 2024-09-13T18:32:21.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.123+0000 7fd9fd613740 10 client.27628 fetch_fsmap learned FSMap version 34 2024-09-13T18:32:21.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.123+0000 7fd9fd613740 1 -- 172.21.15.114:0/1982536313 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55665cbfdea0 con 0x55665ccdf1f0 2024-09-13T18:32:21.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.127+0000 7fd9d77fe640 1 -- 172.21.15.114:0/1982536313 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd9ec0b7950 con 0x55665ccdf1f0 2024-09-13T18:32:21.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.128+0000 7fd9d77fe640 1 -- 172.21.15.114:0/1982536313 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd9e0028930 con 0x55665ccdf1f0 2024-09-13T18:32:21.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.129+0000 7fd9fd613740 10 client.27628 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:32:21.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.129+0000 7fd9fd613740 10 client.27628 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:32:21.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.129+0000 7fd9fd613740 20 client.27628 populate_metadata read hostname 'smithi114' 2024-09-13T18:32:21.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.129+0000 7fd9fd613740 1 --2- 172.21.15.114:0/1982536313 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55665cd569c0 0x55665cd58de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:21.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.129+0000 7fd9fd613740 4 client.27628 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T18:32:21.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.129+0000 7fd9fd613740 1 -- 172.21.15.114:0/1982536313 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55665c98fe30 con 0x55665cd569c0 2024-09-13T18:32:21.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.134+0000 7fd9f3fff640 1 --2- 172.21.15.114:0/1982536313 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55665cd569c0 0x55665cd58de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:21.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.147+0000 7fd9f3fff640 1 --2- 172.21.15.114:0/1982536313 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55665cd569c0 0x55665cd58de0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5db0148d2f6bc01a server_cookie=f321c0b38d6159ea in_seq=0 out_seq=0 2024-09-13T18:32:21.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.150+0000 7fd9d77fe640 10 client.27628 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:32:21.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.150+0000 7fd9d77fe640 1 -- 172.21.15.114:0/1982536313 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55665c98fe30 con 0x55665cd569c0 2024-09-13T18:32:21.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.150+0000 7fd9d77fe640 10 client.27628 handle_command_reply: tid=0 2024-09-13T18:32:21.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.150+0000 7fd9fd613740 1 client.27628 shutdown 2024-09-13T18:32:21.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.151+0000 7fd9fd613740 1 -- 172.21.15.114:0/1982536313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55665ccdf1f0 msgr2=0x55665cd54bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:21.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.151+0000 7fd9fd613740 1 --2- 172.21.15.114:0/1982536313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55665ccdf1f0 0x55665cd54bb0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd9e000cd10 tx=0x7fd9e00137d0 comp rx=0 tx=0).stop 2024-09-13T18:32:21.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.152+0000 7fd9fd613740 1 -- 172.21.15.114:0/1982536313 shutdown_connections 2024-09-13T18:32:21.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.152+0000 7fd9fd613740 1 --2- 172.21.15.114:0/1982536313 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55665cd569c0 0x55665cd58de0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:21.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.152+0000 7fd9fd613740 1 --2- 172.21.15.114:0/1982536313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55665ccdf1f0 0x55665cd54bb0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:21.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.152+0000 7fd9fd613740 1 --2- 172.21.15.114:0/1982536313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55665ccd4950 0x55665ccf5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:21.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.152+0000 7fd9fd613740 1 --2- 172.21.15.114:0/1982536313 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55665ccd4030 0x55665ccf4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:21.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.152+0000 7fd9fd613740 1 -- 172.21.15.114:0/1982536313 >> 172.21.15.114:0/1982536313 conn(0x55665ccd05a0 msgr2=0x55665ccde570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:21.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.152+0000 7fd9d77fe640 0 client.27628 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:32:21.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.153+0000 7fd9fd613740 1 -- 172.21.15.114:0/1982536313 shutdown_connections 2024-09-13T18:32:21.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.157+0000 7fd9fd613740 1 -- 172.21.15.114:0/1982536313 wait complete. 2024-09-13T18:32:21.164 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:32:21.164 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "no active scrubs running", 2024-09-13T18:32:21.164 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": {} 2024-09-13T18:32:21.164 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:32:21.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.160+0000 7fd9fd613740 20 client.27628 trim_cache size 0 max 16384 2024-09-13T18:32:21.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.162+0000 7fd9f95b0640 1 -- 172.21.15.114:0/516967946 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fd9d80a1340 msgr2=0x7fd9d80a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:21.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.162+0000 7fd9f95b0640 1 --2- 172.21.15.114:0/516967946 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fd9d80a1340 0x7fd9d80a3800 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd9e000a700 tx=0x7fd9e0008040 comp rx=0 tx=0).stop 2024-09-13T18:32:21.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.162+0000 7fd9f95b0640 1 -- 172.21.15.114:0/516967946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9f41062e0 msgr2=0x7fd9f41a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:21.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.162+0000 7fd9f95b0640 1 --2- 172.21.15.114:0/516967946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9f41062e0 0x7fd9f41a2520 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec031a20 tx=0x7fd9ec02f410 comp rx=0 tx=0).stop 2024-09-13T18:32:21.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.163+0000 7fd9f95b0640 1 -- 172.21.15.114:0/516967946 shutdown_connections 2024-09-13T18:32:21.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.163+0000 7fd9f95b0640 1 --2- 172.21.15.114:0/516967946 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fd9d80a1340 0x7fd9d80a3800 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:21.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.163+0000 7fd9f95b0640 1 --2- 172.21.15.114:0/516967946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9f410c2a0 0x7fd9f419c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:21.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.163+0000 7fd9f95b0640 1 --2- 172.21.15.114:0/516967946 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9f4106c40 0x7fd9f41a2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:21.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.163+0000 7fd9f95b0640 1 --2- 172.21.15.114:0/516967946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9f41062e0 0x7fd9f41a2520 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:21.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.163+0000 7fd9f95b0640 1 -- 172.21.15.114:0/516967946 >> 172.21.15.114:0/516967946 conn(0x7fd9f4075b80 msgr2=0x7fd9f4077660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:21.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.169+0000 7fd9f95b0640 1 -- 172.21.15.114:0/516967946 shutdown_connections 2024-09-13T18:32:21.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.170+0000 7fd9f95b0640 1 -- 172.21.15.114:0/516967946 wait complete. 2024-09-13T18:32:21.195 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-13T18:32:21.195 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:32:21.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.465+0000 7ff841b5e640 1 -- 172.21.15.114:0/3145031694 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff83c107890 msgr2=0x7ff83c107cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:21.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.465+0000 7ff841b5e640 1 --2- 172.21.15.114:0/3145031694 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff83c107890 0x7ff83c107cf0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7ff82c009900 tx=0x7ff82c02a420 comp rx=0 tx=0).stop 2024-09-13T18:32:21.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.467+0000 7ff841b5e640 1 -- 172.21.15.114:0/3145031694 shutdown_connections 2024-09-13T18:32:21.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.467+0000 7ff841b5e640 1 --2- 172.21.15.114:0/3145031694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff83c119400 0x7ff83c11b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:21.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.467+0000 7ff841b5e640 1 --2- 172.21.15.114:0/3145031694 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff83c107890 0x7ff83c107cf0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:21.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.467+0000 7ff841b5e640 1 --2- 172.21.15.114:0/3145031694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff83c1074b0 0x7ff83c10e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:21.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.467+0000 7ff841b5e640 1 -- 172.21.15.114:0/3145031694 >> 172.21.15.114:0/3145031694 conn(0x7ff83c06aff0 msgr2=0x7ff83c06b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:21.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.471+0000 7ff841b5e640 1 -- 172.21.15.114:0/3145031694 shutdown_connections 2024-09-13T18:32:21.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.471+0000 7ff841b5e640 1 -- 172.21.15.114:0/3145031694 wait complete. 2024-09-13T18:32:21.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.472+0000 7ff841b5e640 1 Processor -- start 2024-09-13T18:32:21.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.472+0000 7ff841b5e640 1 -- start start 2024-09-13T18:32:21.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.473+0000 7ff841b5e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff83c1074b0 0x7ff83c1a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:21.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.473+0000 7ff841b5e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff83c107890 0x7ff83c1a5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:21.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.473+0000 7ff841b5e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff83c119400 0x7ff83c1a9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:21.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.473+0000 7ff841b5e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff83c1aa000 con 0x7ff83c119400 2024-09-13T18:32:21.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.473+0000 7ff841b5e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff83c1aa170 con 0x7ff83c107890 2024-09-13T18:32:21.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.473+0000 7ff841b5e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff83c1aa330 con 0x7ff83c1074b0 2024-09-13T18:32:21.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.473+0000 7ff83bfff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff83c119400 0x7ff83c1a9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:21.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.473+0000 7ff83bfff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff83c119400 0x7ff83c1a9a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57650/0 (socket says 172.21.15.114:57650) 2024-09-13T18:32:21.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.473+0000 7ff83bfff640 1 -- 172.21.15.114:0/4133539276 learned_addr learned my addr 172.21.15.114:0/4133539276 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:21.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.474+0000 7ff83bfff640 1 -- 172.21.15.114:0/4133539276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff83c1074b0 msgr2=0x7ff83c1a50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:21.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.474+0000 7ff83bfff640 1 --2- 172.21.15.114:0/4133539276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff83c1074b0 0x7ff83c1a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:21.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.474+0000 7ff83bfff640 1 -- 172.21.15.114:0/4133539276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff83c107890 msgr2=0x7ff83c1a5610 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:32:21.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.474+0000 7ff83bfff640 1 --2- 172.21.15.114:0/4133539276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff83c107890 0x7ff83c1a5610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:21.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.474+0000 7ff83bfff640 1 -- 172.21.15.114:0/4133539276 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff82c009560 con 0x7ff83c119400 2024-09-13T18:32:21.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.474+0000 7ff83bfff640 1 --2- 172.21.15.114:0/4133539276 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff83c119400 0x7ff83c1a9a00 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7ff83000b3f0 tx=0x7ff83000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:21.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.475+0000 7ff838ff9640 1 -- 172.21.15.114:0/4133539276 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff830011830 con 0x7ff83c119400 2024-09-13T18:32:21.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.475+0000 7ff838ff9640 1 -- 172.21.15.114:0/4133539276 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff830011e20 con 0x7ff83c119400 2024-09-13T18:32:21.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.475+0000 7ff838ff9640 1 -- 172.21.15.114:0/4133539276 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff830010e50 con 0x7ff83c119400 2024-09-13T18:32:21.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.475+0000 7ff841b5e640 1 -- 172.21.15.114:0/4133539276 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff83c1aa610 con 0x7ff83c119400 2024-09-13T18:32:21.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.475+0000 7ff841b5e640 1 -- 172.21.15.114:0/4133539276 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff83c1b1e90 con 0x7ff83c119400 2024-09-13T18:32:21.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.477+0000 7ff841b5e640 1 -- 172.21.15.114:0/4133539276 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff83c06bdc0 con 0x7ff83c119400 2024-09-13T18:32:21.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.479+0000 7ff838ff9640 1 -- 172.21.15.114:0/4133539276 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7ff830011990 con 0x7ff83c119400 2024-09-13T18:32:21.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.480+0000 7ff838ff9640 1 --2- 172.21.15.114:0/4133539276 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff8240a1410 0x7ff8240a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:21.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.480+0000 7ff838ff9640 1 -- 172.21.15.114:0/4133539276 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9541+0+0 (secure 0 0 0) 0x7ff8300cc920 con 0x7ff83c119400 2024-09-13T18:32:21.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.489+0000 7ff838ff9640 1 -- 172.21.15.114:0/4133539276 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7ff830095a20 con 0x7ff83c119400 2024-09-13T18:32:21.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.496+0000 7ff83b7fe640 1 --2- 172.21.15.114:0/4133539276 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff8240a1410 0x7ff8240a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:21.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.512+0000 7ff83b7fe640 1 --2- 172.21.15.114:0/4133539276 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff8240a1410 0x7ff8240a38d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff834005fd0 tx=0x7ff834009710 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:21.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.973+0000 7ff841b5e640 1 -- 172.21.15.114:0/4133539276 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff83c1a64d0 con 0x7ff83c119400 2024-09-13T18:32:21.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.977+0000 7ff838ff9640 1 -- 172.21.15.114:0/4133539276 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+10930 (secure 0 0 0) 0x7ff83c1a64d0 con 0x7ff83c119400 2024-09-13T18:32:21.982 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:32:21.982 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4162817497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4162817497},{"type":"v1","addr":"172.21.15.114:6835","nonce":4162817497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/2435428478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":2435428478},{"type":"v1","addr":"172.21.15.114:6837","nonce":2435428478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T18:20:01.895117+0000","modified":"2024-09-13T18:27:22.172031+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24482,"mds_2":24475,"mds_3":24458,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/2046516451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":2046516451},{"type":"v1","addr":"172.21.15.114:6839","nonce":2046516451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":10,"addr":"172.21.15.114:6841/1956923446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":1956923446},{"type":"v1","addr":"172.21.15.114:6841","nonce":1956923446}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24391":{"gid":24391,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/507301496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":507301496},{"type":"v1","addr":"172.21.15.204:6833","nonce":507301496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4028657219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4028657219},{"type":"v1","addr":"172.21.15.134:6833","nonce":4028657219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/57029119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":57029119},{"type":"v1","addr":"172.21.15.204:6835","nonce":57029119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3068880939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3068880939},{"type":"v1","addr":"172.21.15.204:6837","nonce":3068880939}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"e","rank":3,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.134:6835/1150003275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1150003275},{"type":"v1","addr":"172.21.15.134:6835","nonce":1150003275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/344883912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":344883912},{"type":"v1","addr":"172.21.15.204:6839","nonce":344883912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6837/891692210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":891692210},{"type":"v1","addr":"172.21.15.134:6837","nonce":891692210}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6839/3466309724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3466309724},{"type":"v1","addr":"172.21.15.134:6839","nonce":3466309724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:32:21.982 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 34 2024-09-13T18:32:21.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.990+0000 7ff841b5e640 1 -- 172.21.15.114:0/4133539276 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff8240a1410 msgr2=0x7ff8240a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:21.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.990+0000 7ff841b5e640 1 --2- 172.21.15.114:0/4133539276 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff8240a1410 0x7ff8240a38d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff834005fd0 tx=0x7ff834009710 comp rx=0 tx=0).stop 2024-09-13T18:32:21.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.990+0000 7ff841b5e640 1 -- 172.21.15.114:0/4133539276 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff83c119400 msgr2=0x7ff83c1a9a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:21.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.990+0000 7ff841b5e640 1 --2- 172.21.15.114:0/4133539276 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff83c119400 0x7ff83c1a9a00 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7ff83000b3f0 tx=0x7ff83000b8b0 comp rx=0 tx=0).stop 2024-09-13T18:32:21.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.991+0000 7ff841b5e640 1 -- 172.21.15.114:0/4133539276 shutdown_connections 2024-09-13T18:32:21.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.991+0000 7ff841b5e640 1 --2- 172.21.15.114:0/4133539276 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff8240a1410 0x7ff8240a38d0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:21.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.991+0000 7ff841b5e640 1 --2- 172.21.15.114:0/4133539276 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff83c119400 0x7ff83c1a9a00 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:21.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.991+0000 7ff841b5e640 1 --2- 172.21.15.114:0/4133539276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff83c107890 0x7ff83c1a5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:21.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.991+0000 7ff841b5e640 1 --2- 172.21.15.114:0/4133539276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff83c1074b0 0x7ff83c1a50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:21.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.991+0000 7ff841b5e640 1 -- 172.21.15.114:0/4133539276 >> 172.21.15.114:0/4133539276 conn(0x7ff83c06aff0 msgr2=0x7ff83c119cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:21.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.995+0000 7ff841b5e640 1 -- 172.21.15.114:0/4133539276 shutdown_connections 2024-09-13T18:32:21.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:21.995+0000 7ff841b5e640 1 -- 172.21.15.114:0/4133539276 wait complete. 2024-09-13T18:32:22.028 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-09-13T18:32:22.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:21 smithi204 ceph-mon[34777]: pgmap v306: 129 pgs: 3 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 122 active+clean; 20 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 25 MiB/s wr, 576 op/s 2024-09-13T18:32:22.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:21 smithi114 ceph-mon[30897]: pgmap v306: 129 pgs: 3 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 122 active+clean; 20 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 25 MiB/s wr, 576 op/s 2024-09-13T18:32:22.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.267+0000 7f0e22cb1640 1 -- 172.21.15.114:0/3704826338 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e1c119400 msgr2=0x7f0e1c11b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:22.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.267+0000 7f0e22cb1640 1 --2- 172.21.15.114:0/3704826338 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e1c119400 0x7f0e1c11b7f0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f0e10009900 tx=0x7f0e1002a440 comp rx=0 tx=0).stop 2024-09-13T18:32:22.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.272+0000 7f0e22cb1640 1 -- 172.21.15.114:0/3704826338 shutdown_connections 2024-09-13T18:32:22.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.272+0000 7f0e22cb1640 1 --2- 172.21.15.114:0/3704826338 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e1c119400 0x7f0e1c11b7f0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.272+0000 7f0e22cb1640 1 --2- 172.21.15.114:0/3704826338 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e1c107920 0x7f0e1c107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.272+0000 7f0e22cb1640 1 --2- 172.21.15.114:0/3704826338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e1c107540 0x7f0e1c10e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.272+0000 7f0e22cb1640 1 -- 172.21.15.114:0/3704826338 >> 172.21.15.114:0/3704826338 conn(0x7f0e1c06afe0 msgr2=0x7f0e1c06b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:22.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.274+0000 7f0e22cb1640 1 -- 172.21.15.114:0/3704826338 shutdown_connections 2024-09-13T18:32:22.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.274+0000 7f0e22cb1640 1 -- 172.21.15.114:0/3704826338 wait complete. 2024-09-13T18:32:22.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.275+0000 7f0e22cb1640 1 Processor -- start 2024-09-13T18:32:22.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.275+0000 7f0e22cb1640 1 -- start start 2024-09-13T18:32:22.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.275+0000 7f0e22cb1640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e1c107540 0x7f0e1c1168b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:22.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.275+0000 7f0e22cb1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e1c107920 0x7f0e1c116e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:22.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.275+0000 7f0e22cb1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e1c119400 0x7f0e1c10fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:22.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.275+0000 7f0e22cb1640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1c1102f0 con 0x7f0e1c107540 2024-09-13T18:32:22.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.275+0000 7f0e22cb1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1c110460 con 0x7f0e1c119400 2024-09-13T18:32:22.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.275+0000 7f0e22cb1640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1c110620 con 0x7f0e1c107920 2024-09-13T18:32:22.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.277+0000 7f0e20a4d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e1c107540 0x7f0e1c1168b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:22.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.277+0000 7f0e20a4d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e1c107540 0x7f0e1c1168b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57664/0 (socket says 172.21.15.114:57664) 2024-09-13T18:32:22.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.277+0000 7f0e20a4d640 1 -- 172.21.15.114:0/3132604900 learned_addr learned my addr 172.21.15.114:0/3132604900 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:22.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.277+0000 7f0e2124e640 1 --2- 172.21.15.114:0/3132604900 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e1c119400 0x7f0e1c10fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:22.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.277+0000 7f0e1bfff640 1 --2- 172.21.15.114:0/3132604900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e1c107920 0x7f0e1c116e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:22.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.279+0000 7f0e2124e640 1 -- 172.21.15.114:0/3132604900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e1c107920 msgr2=0x7f0e1c116e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:22.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.279+0000 7f0e2124e640 1 --2- 172.21.15.114:0/3132604900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e1c107920 0x7f0e1c116e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.279+0000 7f0e2124e640 1 -- 172.21.15.114:0/3132604900 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e1c107540 msgr2=0x7f0e1c1168b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:22.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.279+0000 7f0e2124e640 1 --2- 172.21.15.114:0/3132604900 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e1c107540 0x7f0e1c1168b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.279+0000 7f0e2124e640 1 -- 172.21.15.114:0/3132604900 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e10009560 con 0x7f0e1c119400 2024-09-13T18:32:22.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.279+0000 7f0e1bfff640 1 --2- 172.21.15.114:0/3132604900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e1c107920 0x7f0e1c116e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:32:22.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.280+0000 7f0e2124e640 1 --2- 172.21.15.114:0/3132604900 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e1c119400 0x7f0e1c10fb60 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f0e10009850 tx=0x7f0e100055a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:22.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.281+0000 7f0e19ffb640 1 -- 172.21.15.114:0/3132604900 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e10004ad0 con 0x7f0e1c119400 2024-09-13T18:32:22.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.281+0000 7f0e19ffb640 1 -- 172.21.15.114:0/3132604900 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0e10004c30 con 0x7f0e1c119400 2024-09-13T18:32:22.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.282+0000 7f0e19ffb640 1 -- 172.21.15.114:0/3132604900 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1003c6d0 con 0x7f0e1c119400 2024-09-13T18:32:22.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.284+0000 7f0e22cb1640 1 -- 172.21.15.114:0/3132604900 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e1c114440 con 0x7f0e1c119400 2024-09-13T18:32:22.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.284+0000 7f0e22cb1640 1 -- 172.21.15.114:0/3132604900 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e1c114870 con 0x7f0e1c119400 2024-09-13T18:32:22.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.288+0000 7f0e259d5740 1 Processor -- start 2024-09-13T18:32:22.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.288+0000 7f0e259d5740 1 -- start start 2024-09-13T18:32:22.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.288+0000 7f0e259d5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558b8b477a80 0x558b8b477e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:22.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.288+0000 7f0e259d5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b8b4783a0 0x558b8b482700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:22.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.288+0000 7f0e259d5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b8b482c40 0x558b8b485030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:22.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.288+0000 7f0e259d5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b8b3e1cc0 con 0x558b8b477a80 2024-09-13T18:32:22.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.288+0000 7f0e259d5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b8b3df450 con 0x558b8b482c40 2024-09-13T18:32:22.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.288+0000 7f0e259d5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b8b3dad40 con 0x558b8b4783a0 2024-09-13T18:32:22.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.289+0000 7f0e20a4d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b8b4783a0 0x558b8b482700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:22.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.289+0000 7f0e20a4d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b8b4783a0 0x558b8b482700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:49258/0 (socket says 172.21.15.114:49258) 2024-09-13T18:32:22.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.290+0000 7f0e20a4d640 1 -- 172.21.15.114:0/2243767055 learned_addr learned my addr 172.21.15.114:0/2243767055 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:22.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.291+0000 7f0e20a4d640 1 -- 172.21.15.114:0/2243767055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b8b482c40 msgr2=0x558b8b485030 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:32:22.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.291+0000 7f0e20a4d640 1 --2- 172.21.15.114:0/2243767055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b8b482c40 0x558b8b485030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.291+0000 7f0e20a4d640 1 -- 172.21.15.114:0/2243767055 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558b8b477a80 msgr2=0x558b8b477e60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:32:22.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.291+0000 7f0e20a4d640 1 --2- 172.21.15.114:0/2243767055 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558b8b477a80 0x558b8b477e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.291+0000 7f0e20a4d640 1 -- 172.21.15.114:0/2243767055 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b8b3a2620 con 0x558b8b4783a0 2024-09-13T18:32:22.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.295+0000 7f0e20a4d640 1 --2- 172.21.15.114:0/2243767055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b8b4783a0 0x558b8b482700 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0e140115c0 tx=0x7f0e14011a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5411cf4aea3aa983 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:22.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.296+0000 7f0e19ffb640 1 -- 172.21.15.114:0/3132604900 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f0e100035f0 con 0x7f0e1c119400 2024-09-13T18:32:22.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.296+0000 7f0e19ffb640 1 --2- 172.21.15.114:0/3132604900 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0e040a11e0 0x7f0e040a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:22.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.297+0000 7f0e19ffb640 1 -- 172.21.15.114:0/3132604900 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f0e100ee050 con 0x7f0e1c119400 2024-09-13T18:32:22.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.297+0000 7f0e027fc640 1 -- 172.21.15.114:0/2243767055 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1400f6c0 con 0x558b8b4783a0 2024-09-13T18:32:22.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.297+0000 7f0e1bfff640 1 --2- 172.21.15.114:0/3132604900 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0e040a11e0 0x7f0e040a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:22.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.297+0000 7f0e027fc640 1 -- 172.21.15.114:0/2243767055 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0e1400fcb0 con 0x558b8b4783a0 2024-09-13T18:32:22.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.298+0000 7f0e259d5740 1 -- 172.21.15.114:0/2243767055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b8b4783a0 msgr2=0x558b8b482700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:22.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.298+0000 7f0e259d5740 1 --2- 172.21.15.114:0/2243767055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b8b4783a0 0x558b8b482700 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0e140115c0 tx=0x7f0e14011a80 comp rx=0 tx=0).stop 2024-09-13T18:32:22.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.298+0000 7f0e1bfff640 1 --2- 172.21.15.114:0/3132604900 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0e040a11e0 0x7f0e040a36a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x558b8b475370 tx=0x7f0e0c005f90 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:22.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.299+0000 7f0e259d5740 1 -- 172.21.15.114:0/2243767055 shutdown_connections 2024-09-13T18:32:22.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.299+0000 7f0e259d5740 1 --2- 172.21.15.114:0/2243767055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b8b482c40 0x558b8b485030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.299+0000 7f0e259d5740 1 --2- 172.21.15.114:0/2243767055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b8b4783a0 0x558b8b482700 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.299+0000 7f0e259d5740 1 --2- 172.21.15.114:0/2243767055 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558b8b477a80 0x558b8b477e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.299+0000 7f0e259d5740 1 -- 172.21.15.114:0/2243767055 >> 172.21.15.114:0/2243767055 conn(0x558b8b473ff0 msgr2=0x558b8b4743d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:22.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.302+0000 7f0e259d5740 1 -- 172.21.15.114:0/2243767055 shutdown_connections 2024-09-13T18:32:22.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.303+0000 7f0e259d5740 1 -- 172.21.15.114:0/2243767055 wait complete. 2024-09-13T18:32:22.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.303+0000 7f0e259d5740 1 -- start start 2024-09-13T18:32:22.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.303+0000 7f0e259d5740 1 Processor -- start 2024-09-13T18:32:22.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.306+0000 7f0e259d5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b8b477a80 0x558b8b47f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:22.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.306+0000 7f0e259d5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558b8b4783a0 0x558b8b47fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:22.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.306+0000 7f0e259d5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b8b482c40 0x558b8b486930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:22.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.306+0000 7f0e259d5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b8b399180 con 0x558b8b4783a0 2024-09-13T18:32:22.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.306+0000 7f0e259d5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b8b0b3430 con 0x558b8b477a80 2024-09-13T18:32:22.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.306+0000 7f0e259d5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b8b35e5e0 con 0x558b8b482c40 2024-09-13T18:32:22.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.306+0000 7f0e259d5740 10 client.0 _ll_register_callbacks cb 0x558b8b3caaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:32:22.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.306+0000 7f0e20a4d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b8b477a80 0x558b8b47f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:22.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.306+0000 7f0e20a4d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b8b477a80 0x558b8b47f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:42374/0 (socket says 172.21.15.114:42374) 2024-09-13T18:32:22.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.307+0000 7f0e20a4d640 1 -- 172.21.15.114:0/2699515524 learned_addr learned my addr 172.21.15.114:0/2699515524 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:22.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.307+0000 7f0e2124e640 1 --2- 172.21.15.114:0/2699515524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b8b482c40 0x558b8b486930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:22.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.307+0000 7f0e20a4d640 1 -- 172.21.15.114:0/2699515524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b8b482c40 msgr2=0x558b8b486930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:22.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.307+0000 7f0e20a4d640 1 --2- 172.21.15.114:0/2699515524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b8b482c40 0x558b8b486930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.307+0000 7f0e20a4d640 1 -- 172.21.15.114:0/2699515524 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558b8b4783a0 msgr2=0x558b8b47fae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:32:22.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.307+0000 7f0e20a4d640 1 --2- 172.21.15.114:0/2699515524 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558b8b4783a0 0x558b8b47fae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.307+0000 7f0e20a4d640 1 -- 172.21.15.114:0/2699515524 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e14009da0 con 0x558b8b477a80 2024-09-13T18:32:22.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.307+0000 7f0e20a4d640 1 --2- 172.21.15.114:0/2699515524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b8b477a80 0x558b8b47f5a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0e14009270 tx=0x7f0e14019640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0e8c91482330434 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:22.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.308+0000 7f0dfbfff640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:32:22.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.308+0000 7f0dfbfff640 1 -- 172.21.15.114:0/2699515524 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e14003e90 con 0x558b8b477a80 2024-09-13T18:32:22.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.308+0000 7f0e259d5740 1 -- 172.21.15.114:0/2699515524 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558b8b3cd1f0 con 0x558b8b477a80 2024-09-13T18:32:22.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.308+0000 7f0dfbfff640 1 -- 172.21.15.114:0/2699515524 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f0e1400f6c0 con 0x558b8b477a80 2024-09-13T18:32:22.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.309+0000 7f0dfbfff640 1 -- 172.21.15.114:0/2699515524 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e1400f8c0 con 0x558b8b477a80 2024-09-13T18:32:22.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.313+0000 7f0e259d5740 10 client.27710 fetch_fsmap learned FSMap version 34 2024-09-13T18:32:22.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.313+0000 7f0e259d5740 1 -- 172.21.15.114:0/2699515524 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x558b8b3d93c0 con 0x558b8b477a80 2024-09-13T18:32:22.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.314+0000 7f0dfbfff640 1 -- 172.21.15.114:0/2699515524 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558b8b3cd1f0 con 0x558b8b477a80 2024-09-13T18:32:22.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.314+0000 7f0dfbfff640 1 -- 172.21.15.114:0/2699515524 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0e14005650 con 0x558b8b477a80 2024-09-13T18:32:22.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.315+0000 7f0e259d5740 10 client.27710 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:32:22.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.315+0000 7f0e259d5740 10 client.27710 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-13T18:32:22.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.315+0000 7f0e259d5740 20 client.27710 populate_metadata read hostname 'smithi114' 2024-09-13T18:32:22.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.315+0000 7f0e259d5740 1 --2- 172.21.15.114:0/2699515524 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x558b8b4b42e0 0x558b8b4b6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:22.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.315+0000 7f0e259d5740 4 client.27710 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:32:22.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.315+0000 7f0e259d5740 1 -- 172.21.15.114:0/2699515524 --> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558b8b0ce4e0 con 0x558b8b4b42e0 2024-09-13T18:32:22.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.317+0000 7f0e1bfff640 1 --2- 172.21.15.114:0/2699515524 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x558b8b4b42e0 0x558b8b4b6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:22.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.320+0000 7f0e1bfff640 1 --2- 172.21.15.114:0/2699515524 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x558b8b4b42e0 0x558b8b4b6700 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2c020d2145505efa server_cookie=ed505b3237e8b803 in_seq=0 out_seq=0 2024-09-13T18:32:22.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.320+0000 7f0dfbfff640 10 client.27710 ms_handle_connect on v2:172.21.15.114:6840/1956923446 2024-09-13T18:32:22.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.323+0000 7f0dfbfff640 1 -- 172.21.15.114:0/2699515524 <== mds.4 v2:172.21.15.114:6840/1956923446 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x558b8b0ce4e0 con 0x558b8b4b42e0 2024-09-13T18:32:22.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.323+0000 7f0dfbfff640 10 client.27710 handle_command_reply: tid=0 2024-09-13T18:32:22.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.323+0000 7f0e259d5740 1 client.27710 shutdown 2024-09-13T18:32:22.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.326+0000 7f0e259d5740 1 -- 172.21.15.114:0/2699515524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b8b477a80 msgr2=0x558b8b47f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:22.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.326+0000 7f0e259d5740 1 --2- 172.21.15.114:0/2699515524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b8b477a80 0x558b8b47f5a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0e14009270 tx=0x7f0e14019640 comp rx=0 tx=0).stop 2024-09-13T18:32:22.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.329+0000 7f0e259d5740 1 -- 172.21.15.114:0/2699515524 shutdown_connections 2024-09-13T18:32:22.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.329+0000 7f0e259d5740 1 --2- 172.21.15.114:0/2699515524 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x558b8b4b42e0 0x558b8b4b6700 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.329+0000 7f0e259d5740 1 --2- 172.21.15.114:0/2699515524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b8b482c40 0x558b8b486930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.329+0000 7f0e259d5740 1 --2- 172.21.15.114:0/2699515524 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558b8b4783a0 0x558b8b47fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.329+0000 7f0e259d5740 1 --2- 172.21.15.114:0/2699515524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b8b477a80 0x558b8b47f5a0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.329+0000 7f0e259d5740 1 -- 172.21.15.114:0/2699515524 >> 172.21.15.114:0/2699515524 conn(0x558b8b473ff0 msgr2=0x558b8b479d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:22.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.329+0000 7f0dfbfff640 0 client.27710 ms_handle_reset on v2:172.21.15.114:6840/1956923446 2024-09-13T18:32:22.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.329+0000 7f0e259d5740 1 -- 172.21.15.114:0/2699515524 shutdown_connections 2024-09-13T18:32:22.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.337+0000 7f0e259d5740 1 -- 172.21.15.114:0/2699515524 wait complete. 2024-09-13T18:32:22.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.342+0000 7f0e259d5740 20 client.27710 trim_cache size 0 max 16384 2024-09-13T18:32:22.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:21 smithi134 ceph-mon[34283]: pgmap v306: 129 pgs: 3 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 122 active+clean; 20 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 25 MiB/s wr, 576 op/s 2024-09-13T18:32:22.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.370+0000 7f0e259d5740 1 Processor -- start 2024-09-13T18:32:22.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.370+0000 7f0e259d5740 1 -- start start 2024-09-13T18:32:22.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.371+0000 7f0e259d5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558b8b477a80 0x558b8b4868b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:22.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.371+0000 7f0e259d5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b8b4783a0 0x558b8b47a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:22.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.371+0000 7f0e259d5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b8b482c40 0x558b8b47ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:22.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.371+0000 7f0e259d5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b8b485660 con 0x558b8b477a80 2024-09-13T18:32:22.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.371+0000 7f0e259d5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b8b48ea60 con 0x558b8b482c40 2024-09-13T18:32:22.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.371+0000 7f0e259d5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b8b477800 con 0x558b8b4783a0 2024-09-13T18:32:22.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.372+0000 7f0e20a4d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558b8b477a80 0x558b8b4868b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:22.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.372+0000 7f0e20a4d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558b8b477a80 0x558b8b4868b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57674/0 (socket says 172.21.15.114:57674) 2024-09-13T18:32:22.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.372+0000 7f0e20a4d640 1 -- 172.21.15.114:0/1018857914 learned_addr learned my addr 172.21.15.114:0/1018857914 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:22.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.372+0000 7f0e2124e640 1 --2- 172.21.15.114:0/1018857914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b8b482c40 0x558b8b47ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:22.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.373+0000 7f0e2124e640 1 -- 172.21.15.114:0/1018857914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b8b4783a0 msgr2=0x558b8b47a650 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:32:22.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.373+0000 7f0e2124e640 1 --2- 172.21.15.114:0/1018857914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b8b4783a0 0x558b8b47a650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.373+0000 7f0e2124e640 1 -- 172.21.15.114:0/1018857914 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558b8b477a80 msgr2=0x558b8b4868b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:22.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.373+0000 7f0e2124e640 1 --2- 172.21.15.114:0/1018857914 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558b8b477a80 0x558b8b4868b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.373+0000 7f0e2124e640 1 -- 172.21.15.114:0/1018857914 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b8b37b770 con 0x558b8b482c40 2024-09-13T18:32:22.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.373+0000 7f0e2124e640 1 --2- 172.21.15.114:0/1018857914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b8b482c40 0x558b8b47ab90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f0e100eb340 tx=0x7f0e100ba720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3661569807738b04 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:22.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.376+0000 7f0e02ffd640 1 -- 172.21.15.114:0/1018857914 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e100bad00 con 0x558b8b482c40 2024-09-13T18:32:22.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.377+0000 7f0e02ffd640 1 -- 172.21.15.114:0/1018857914 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0e100beb30 con 0x558b8b482c40 2024-09-13T18:32:22.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.377+0000 7f0e02ffd640 1 -- 172.21.15.114:0/1018857914 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e100baf60 con 0x558b8b482c40 2024-09-13T18:32:22.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.378+0000 7f0e259d5740 1 -- 172.21.15.114:0/1018857914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b8b482c40 msgr2=0x558b8b47ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:22.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.378+0000 7f0e259d5740 1 --2- 172.21.15.114:0/1018857914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b8b482c40 0x558b8b47ab90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f0e100eb340 tx=0x7f0e100ba720 comp rx=0 tx=0).stop 2024-09-13T18:32:22.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.381+0000 7f0e259d5740 1 -- 172.21.15.114:0/1018857914 shutdown_connections 2024-09-13T18:32:22.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.381+0000 7f0e259d5740 1 --2- 172.21.15.114:0/1018857914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b8b482c40 0x558b8b47ab90 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.381+0000 7f0e259d5740 1 --2- 172.21.15.114:0/1018857914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b8b4783a0 0x558b8b47a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.381+0000 7f0e259d5740 1 --2- 172.21.15.114:0/1018857914 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558b8b477a80 0x558b8b4868b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.381+0000 7f0e259d5740 1 -- 172.21.15.114:0/1018857914 >> 172.21.15.114:0/1018857914 conn(0x558b8b473ff0 msgr2=0x558b8b47dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:22.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.381+0000 7f0e259d5740 1 -- 172.21.15.114:0/1018857914 shutdown_connections 2024-09-13T18:32:22.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.381+0000 7f0e259d5740 1 -- 172.21.15.114:0/1018857914 wait complete. 2024-09-13T18:32:22.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.382+0000 7f0e259d5740 1 -- start start 2024-09-13T18:32:22.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.382+0000 7f0e259d5740 1 Processor -- start 2024-09-13T18:32:22.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.382+0000 7f0e259d5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b8b477a80 0x558b8b48dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:22.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.382+0000 7f0e259d5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b8b4783a0 0x558b8b48e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:22.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.382+0000 7f0e259d5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558b8b482c40 0x558b8b488a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:22.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.382+0000 7f0e259d5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b8b488f70 con 0x558b8b482c40 2024-09-13T18:32:22.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.382+0000 7f0e259d5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b8b4890e0 con 0x558b8b477a80 2024-09-13T18:32:22.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.382+0000 7f0e259d5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b8b4892a0 con 0x558b8b4783a0 2024-09-13T18:32:22.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.382+0000 7f0e259d5740 10 client.0 _ll_register_callbacks cb 0x7f0e14009bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:32:22.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.384+0000 7f0e20a4d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b8b477a80 0x558b8b48dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:22.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.384+0000 7f0e20a4d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b8b477a80 0x558b8b48dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:42404/0 (socket says 172.21.15.114:42404) 2024-09-13T18:32:22.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.384+0000 7f0e20a4d640 1 -- 172.21.15.114:0/1815262608 learned_addr learned my addr 172.21.15.114:0/1815262608 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:22.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.384+0000 7f0e2124e640 1 --2- 172.21.15.114:0/1815262608 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558b8b482c40 0x558b8b488a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:22.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.385+0000 7f0e1bfff640 1 --2- 172.21.15.114:0/1815262608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b8b4783a0 0x558b8b48e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:22.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.385+0000 7f0e2124e640 1 -- 172.21.15.114:0/1815262608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b8b4783a0 msgr2=0x558b8b48e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:22.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.385+0000 7f0e2124e640 1 --2- 172.21.15.114:0/1815262608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b8b4783a0 0x558b8b48e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.385+0000 7f0e2124e640 1 -- 172.21.15.114:0/1815262608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b8b477a80 msgr2=0x558b8b48dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:22.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.385+0000 7f0e2124e640 1 --2- 172.21.15.114:0/1815262608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b8b477a80 0x558b8b48dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.385+0000 7f0e2124e640 1 -- 172.21.15.114:0/1815262608 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e1c114870 con 0x558b8b482c40 2024-09-13T18:32:22.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.386+0000 7f0e2124e640 1 --2- 172.21.15.114:0/1815262608 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558b8b482c40 0x558b8b488a30 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x558b8b48ebf0 tx=0x7f0e1002a940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c799d0f1687cad9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:22.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.386+0000 7f0dfb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:32:22.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.386+0000 7f0dfb7fe640 1 -- 172.21.15.114:0/1815262608 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e100bad00 con 0x558b8b482c40 2024-09-13T18:32:22.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.387+0000 7f0e259d5740 1 -- 172.21.15.114:0/1815262608 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558b8b404b80 con 0x558b8b482c40 2024-09-13T18:32:22.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.387+0000 7f0dfb7fe640 1 -- 172.21.15.114:0/1815262608 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f0e100ba1f0 con 0x558b8b482c40 2024-09-13T18:32:22.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.387+0000 7f0dfb7fe640 1 -- 172.21.15.114:0/1815262608 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e100ba3b0 con 0x558b8b482c40 2024-09-13T18:32:22.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.390+0000 7f0e259d5740 10 client.18234 fetch_fsmap learned FSMap version 34 2024-09-13T18:32:22.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.390+0000 7f0e259d5740 1 -- 172.21.15.114:0/1815262608 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x558b8b3a0e80 con 0x558b8b482c40 2024-09-13T18:32:22.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.397+0000 7f0dfb7fe640 1 -- 172.21.15.114:0/1815262608 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558b8b404b80 con 0x558b8b482c40 2024-09-13T18:32:22.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.398+0000 7f0dfb7fe640 1 -- 172.21.15.114:0/1815262608 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0e100cebd0 con 0x558b8b482c40 2024-09-13T18:32:22.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.398+0000 7f0e259d5740 10 client.18234 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:32:22.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.398+0000 7f0e259d5740 10 client.18234 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-13T18:32:22.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.398+0000 7f0e259d5740 20 client.18234 populate_metadata read hostname 'smithi114' 2024-09-13T18:32:22.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.398+0000 7f0e259d5740 1 --2- 172.21.15.114:0/1815262608 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x558b8b50b370 0x558b8b50d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:22.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.398+0000 7f0e259d5740 4 client.18234 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:32:22.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.398+0000 7f0e259d5740 1 -- 172.21.15.114:0/1815262608 --> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558b8b0df7e0 con 0x558b8b50b370 2024-09-13T18:32:22.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.402+0000 7f0e20a4d640 1 --2- 172.21.15.114:0/1815262608 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x558b8b50b370 0x558b8b50d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:22.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.405+0000 7f0e20a4d640 1 --2- 172.21.15.114:0/1815262608 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x558b8b50b370 0x558b8b50d760 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=684df2e089d2c9a1 server_cookie=e6115c1b632a2eee in_seq=0 out_seq=0 2024-09-13T18:32:22.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.405+0000 7f0dfb7fe640 10 client.18234 ms_handle_connect on v2:172.21.15.114:6840/1956923446 2024-09-13T18:32:22.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.406+0000 7f0dfb7fe640 1 -- 172.21.15.114:0/1815262608 <== mds.4 v2:172.21.15.114:6840/1956923446 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558b8b0df7e0 con 0x558b8b50b370 2024-09-13T18:32:22.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.406+0000 7f0dfb7fe640 10 client.18234 handle_command_reply: tid=0 2024-09-13T18:32:22.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.408+0000 7f0e259d5740 1 client.18234 shutdown 2024-09-13T18:32:22.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.411+0000 7f0e259d5740 1 -- 172.21.15.114:0/1815262608 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558b8b482c40 msgr2=0x558b8b488a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:22.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.411+0000 7f0e259d5740 1 --2- 172.21.15.114:0/1815262608 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558b8b482c40 0x558b8b488a30 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x558b8b48ebf0 tx=0x7f0e1002a940 comp rx=0 tx=0).stop 2024-09-13T18:32:22.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.417+0000 7f0e259d5740 1 -- 172.21.15.114:0/1815262608 shutdown_connections 2024-09-13T18:32:22.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.417+0000 7f0e259d5740 1 --2- 172.21.15.114:0/1815262608 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x558b8b50b370 0x558b8b50d760 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.417+0000 7f0e259d5740 1 --2- 172.21.15.114:0/1815262608 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558b8b482c40 0x558b8b488a30 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.417+0000 7f0e259d5740 1 --2- 172.21.15.114:0/1815262608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b8b4783a0 0x558b8b48e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.417+0000 7f0e259d5740 1 --2- 172.21.15.114:0/1815262608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b8b477a80 0x558b8b48dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.417+0000 7f0e259d5740 1 -- 172.21.15.114:0/1815262608 >> 172.21.15.114:0/1815262608 conn(0x558b8b473ff0 msgr2=0x558b8b47a650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:22.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.418+0000 7f0dfb7fe640 0 client.18234 ms_handle_reset on v2:172.21.15.114:6840/1956923446 2024-09-13T18:32:22.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.424+0000 7f0e259d5740 1 -- 172.21.15.114:0/1815262608 shutdown_connections 2024-09-13T18:32:22.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.428+0000 7f0e259d5740 1 -- 172.21.15.114:0/1815262608 wait complete. 2024-09-13T18:32:22.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.435+0000 7f0e259d5740 20 client.18234 trim_cache size 0 max 16384 2024-09-13T18:32:22.443 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:32:22.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.444+0000 7f0e037fe640 1 -- 172.21.15.114:0/3132604900 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0e040a11e0 msgr2=0x7f0e040a36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:22.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.445+0000 7f0e037fe640 1 --2- 172.21.15.114:0/3132604900 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0e040a11e0 0x7f0e040a36a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x558b8b475370 tx=0x7f0e0c005f90 comp rx=0 tx=0).stop 2024-09-13T18:32:22.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.445+0000 7f0e037fe640 1 -- 172.21.15.114:0/3132604900 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e1c119400 msgr2=0x7f0e1c10fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:22.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.445+0000 7f0e037fe640 1 --2- 172.21.15.114:0/3132604900 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e1c119400 0x7f0e1c10fb60 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f0e10009850 tx=0x7f0e100055a0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.449+0000 7f0e037fe640 1 -- 172.21.15.114:0/3132604900 shutdown_connections 2024-09-13T18:32:22.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.449+0000 7f0e037fe640 1 --2- 172.21.15.114:0/3132604900 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0e040a11e0 0x7f0e040a36a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.449+0000 7f0e037fe640 1 --2- 172.21.15.114:0/3132604900 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e1c119400 0x7f0e1c10fb60 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.449+0000 7f0e037fe640 1 --2- 172.21.15.114:0/3132604900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e1c107920 0x7f0e1c116e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.449+0000 7f0e037fe640 1 --2- 172.21.15.114:0/3132604900 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e1c107540 0x7f0e1c1168b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.449+0000 7f0e037fe640 1 -- 172.21.15.114:0/3132604900 >> 172.21.15.114:0/3132604900 conn(0x7f0e1c06afe0 msgr2=0x7f0e1c119d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:22.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.454+0000 7f0e037fe640 1 -- 172.21.15.114:0/3132604900 shutdown_connections 2024-09-13T18:32:22.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.455+0000 7f0e037fe640 1 -- 172.21.15.114:0/3132604900 wait complete. 2024-09-13T18:32:22.490 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-09-13T18:32:22.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.724+0000 7f31aa62b640 1 -- 172.21.15.114:0/2545316867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31a4107540 msgr2=0x7f31a410e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:22.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.724+0000 7f31aa62b640 1 --2- 172.21.15.114:0/2545316867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31a4107540 0x7f31a410e510 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f319c009be0 tx=0x7f319c02a6f0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.734+0000 7f31aa62b640 1 -- 172.21.15.114:0/2545316867 shutdown_connections 2024-09-13T18:32:22.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.734+0000 7f31aa62b640 1 --2- 172.21.15.114:0/2545316867 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31a4119400 0x7f31a411b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.734+0000 7f31aa62b640 1 --2- 172.21.15.114:0/2545316867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31a4107920 0x7f31a4107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.734+0000 7f31aa62b640 1 --2- 172.21.15.114:0/2545316867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31a4107540 0x7f31a410e510 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.734+0000 7f31aa62b640 1 -- 172.21.15.114:0/2545316867 >> 172.21.15.114:0/2545316867 conn(0x7f31a406afe0 msgr2=0x7f31a406b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:22.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.740+0000 7f31aa62b640 1 -- 172.21.15.114:0/2545316867 shutdown_connections 2024-09-13T18:32:22.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.741+0000 7f31aa62b640 1 -- 172.21.15.114:0/2545316867 wait complete. 2024-09-13T18:32:22.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.741+0000 7f31aa62b640 1 Processor -- start 2024-09-13T18:32:22.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.741+0000 7f31aa62b640 1 -- start start 2024-09-13T18:32:22.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.742+0000 7f31aa62b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31a4107540 0x7f31a4116910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:22.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.742+0000 7f31aa62b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31a4107920 0x7f31a4116e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:22.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.742+0000 7f31aa62b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31a4119400 0x7f31a410fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:22.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.742+0000 7f31aa62b640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a4110350 con 0x7f31a4119400 2024-09-13T18:32:22.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.742+0000 7f31aa62b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a41104c0 con 0x7f31a4107540 2024-09-13T18:32:22.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.742+0000 7f31aa62b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a4110680 con 0x7f31a4107920 2024-09-13T18:32:22.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.742+0000 7f31a8bc8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31a4119400 0x7f31a410fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:22.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.742+0000 7f31a8bc8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31a4119400 0x7f31a410fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57706/0 (socket says 172.21.15.114:57706) 2024-09-13T18:32:22.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.742+0000 7f31a8bc8640 1 -- 172.21.15.114:0/2289924077 learned_addr learned my addr 172.21.15.114:0/2289924077 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:22.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.743+0000 7f31a3fff640 1 --2- 172.21.15.114:0/2289924077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31a4107540 0x7f31a4116910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:22.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.743+0000 7f31a8bc8640 1 -- 172.21.15.114:0/2289924077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31a4107920 msgr2=0x7f31a4116e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:22.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.743+0000 7f31a37fe640 1 --2- 172.21.15.114:0/2289924077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31a4107920 0x7f31a4116e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:22.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.743+0000 7f31a8bc8640 1 --2- 172.21.15.114:0/2289924077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31a4107920 0x7f31a4116e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.743+0000 7f31a8bc8640 1 -- 172.21.15.114:0/2289924077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31a4107540 msgr2=0x7f31a4116910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:22.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.743+0000 7f31a8bc8640 1 --2- 172.21.15.114:0/2289924077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31a4107540 0x7f31a4116910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.743+0000 7f31a8bc8640 1 -- 172.21.15.114:0/2289924077 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f319c009840 con 0x7f31a4119400 2024-09-13T18:32:22.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.744+0000 7f31a8bc8640 1 --2- 172.21.15.114:0/2289924077 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31a4119400 0x7f31a410fbc0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f319800e970 tx=0x7f319800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:22.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.744+0000 7f31a3fff640 1 --2- 172.21.15.114:0/2289924077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31a4107540 0x7f31a4116910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:32:22.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.744+0000 7f31a37fe640 1 --2- 172.21.15.114:0/2289924077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31a4107920 0x7f31a4116e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:32:22.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.747+0000 7f31a17fa640 1 -- 172.21.15.114:0/2289924077 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319800ccc0 con 0x7f31a4119400 2024-09-13T18:32:22.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.748+0000 7f31a17fa640 1 -- 172.21.15.114:0/2289924077 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f319800ce20 con 0x7f31a4119400 2024-09-13T18:32:22.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.748+0000 7f31a17fa640 1 -- 172.21.15.114:0/2289924077 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3198004e00 con 0x7f31a4119400 2024-09-13T18:32:22.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.750+0000 7f31aa62b640 1 -- 172.21.15.114:0/2289924077 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31a41144a0 con 0x7f31a4119400 2024-09-13T18:32:22.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.750+0000 7f31aa62b640 1 -- 172.21.15.114:0/2289924077 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31a4114960 con 0x7f31a4119400 2024-09-13T18:32:22.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.755+0000 7f31ad42a740 1 Processor -- start 2024-09-13T18:32:22.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.755+0000 7f31ad42a740 1 -- start start 2024-09-13T18:32:22.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.755+0000 7f31ad42a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580afd01790 0x5580afd01b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:22.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.755+0000 7f31ad42a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580afd020b0 0x5580afd0c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:22.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.755+0000 7f31ad42a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580afd0c950 0x5580afd0ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:22.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.755+0000 7f31ad42a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580afc6bcc0 con 0x5580afd01790 2024-09-13T18:32:22.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.755+0000 7f31ad42a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580afc69450 con 0x5580afd0c950 2024-09-13T18:32:22.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.755+0000 7f31ad42a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580afc64d40 con 0x5580afd020b0 2024-09-13T18:32:22.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.757+0000 7f31a37fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580afd0c950 0x5580afd0ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:22.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.757+0000 7f31a37fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580afd0c950 0x5580afd0ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:42440/0 (socket says 172.21.15.114:42440) 2024-09-13T18:32:22.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.757+0000 7f31a37fe640 1 -- 172.21.15.114:0/1194865544 learned_addr learned my addr 172.21.15.114:0/1194865544 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:22.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.758+0000 7f31a17fa640 1 -- 172.21.15.114:0/2289924077 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f3198005020 con 0x7f31a4119400 2024-09-13T18:32:22.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.759+0000 7f31a37fe640 1 -- 172.21.15.114:0/1194865544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580afd020b0 msgr2=0x5580afd0c410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:32:22.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.760+0000 7f31a17fa640 1 --2- 172.21.15.114:0/2289924077 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f318c0a1410 0x7f318c0a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:22.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.761+0000 7f31a17fa640 1 -- 172.21.15.114:0/2289924077 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f3198014070 con 0x7f31a4119400 2024-09-13T18:32:22.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.763+0000 7f31a8bc8640 1 --2- 172.21.15.114:0/2289924077 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f318c0a1410 0x7f318c0a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:22.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.770+0000 7f31a37fe640 1 --2- 172.21.15.114:0/1194865544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580afd020b0 0x5580afd0c410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.771+0000 7f31a37fe640 1 -- 172.21.15.114:0/1194865544 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580afd01790 msgr2=0x5580afd01b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:22.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.771+0000 7f31a37fe640 1 --2- 172.21.15.114:0/1194865544 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580afd01790 0x5580afd01b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.771+0000 7f31a37fe640 1 -- 172.21.15.114:0/1194865544 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580afc2c620 con 0x5580afd0c950 2024-09-13T18:32:22.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.778+0000 7f31a37fe640 1 --2- 172.21.15.114:0/1194865544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580afd0c950 0x5580afd0ed40 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f319400ca40 tx=0x7f319400cf00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c22975775733cf5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:22.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.779+0000 7f318a7fc640 1 -- 172.21.15.114:0/1194865544 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3194017070 con 0x5580afd0c950 2024-09-13T18:32:22.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.779+0000 7f318a7fc640 1 -- 172.21.15.114:0/1194865544 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f31940044b0 con 0x5580afd0c950 2024-09-13T18:32:22.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.779+0000 7f318a7fc640 1 -- 172.21.15.114:0/1194865544 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3194003a30 con 0x5580afd0c950 2024-09-13T18:32:22.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.783+0000 7f31ad42a740 1 -- 172.21.15.114:0/1194865544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580afd0c950 msgr2=0x5580afd0ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:22.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.783+0000 7f31ad42a740 1 --2- 172.21.15.114:0/1194865544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580afd0c950 0x5580afd0ed40 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f319400ca40 tx=0x7f319400cf00 comp rx=0 tx=0).stop 2024-09-13T18:32:22.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.785+0000 7f31a8bc8640 1 --2- 172.21.15.114:0/2289924077 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f318c0a1410 0x7f318c0a38d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f319800c6a0 tx=0x7f3198009ef0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:22.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.792+0000 7f31ad42a740 1 -- 172.21.15.114:0/1194865544 shutdown_connections 2024-09-13T18:32:22.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.792+0000 7f31ad42a740 1 --2- 172.21.15.114:0/1194865544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580afd0c950 0x5580afd0ed40 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.792+0000 7f31ad42a740 1 --2- 172.21.15.114:0/1194865544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580afd020b0 0x5580afd0c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.792+0000 7f31ad42a740 1 --2- 172.21.15.114:0/1194865544 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580afd01790 0x5580afd01b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.792+0000 7f31ad42a740 1 -- 172.21.15.114:0/1194865544 >> 172.21.15.114:0/1194865544 conn(0x5580afcfdd00 msgr2=0x5580afcfe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:22.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.800+0000 7f31ad42a740 1 -- 172.21.15.114:0/1194865544 shutdown_connections 2024-09-13T18:32:22.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.803+0000 7f31ad42a740 1 -- 172.21.15.114:0/1194865544 wait complete. 2024-09-13T18:32:22.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.803+0000 7f31ad42a740 1 -- start start 2024-09-13T18:32:22.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.803+0000 7f31ad42a740 1 Processor -- start 2024-09-13T18:32:22.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.807+0000 7f31ad42a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580afd01790 0x5580afd0c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:22.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.807+0000 7f31ad42a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580afd020b0 0x5580afd092e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:22.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.807+0000 7f31ad42a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580afd0c950 0x5580afd09820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:22.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.807+0000 7f31ad42a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580afc23180 con 0x5580afd020b0 2024-09-13T18:32:22.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.807+0000 7f31ad42a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580af93d430 con 0x5580afd01790 2024-09-13T18:32:22.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.807+0000 7f31ad42a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580afbe85e0 con 0x5580afd0c950 2024-09-13T18:32:22.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.807+0000 7f31ad42a740 10 client.0 _ll_register_callbacks cb 0x5580afc54aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:32:22.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.807+0000 7f31a37fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580afd01790 0x5580afd0c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:22.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.807+0000 7f31a37fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580afd01790 0x5580afd0c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:42452/0 (socket says 172.21.15.114:42452) 2024-09-13T18:32:22.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.807+0000 7f31a37fe640 1 -- 172.21.15.114:0/3957067210 learned_addr learned my addr 172.21.15.114:0/3957067210 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:22.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.808+0000 7f31a37fe640 1 --2- 172.21.15.114:0/3957067210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580afd0c950 0x5580afd09820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:22.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.808+0000 7f31a37fe640 1 -- 172.21.15.114:0/3957067210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580afd0c950 msgr2=0x5580afd09820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:22.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.808+0000 7f31a37fe640 1 --2- 172.21.15.114:0/3957067210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580afd0c950 0x5580afd09820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.808+0000 7f31a37fe640 1 -- 172.21.15.114:0/3957067210 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580afd020b0 msgr2=0x5580afd092e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:22.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.808+0000 7f31a37fe640 1 --2- 172.21.15.114:0/3957067210 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580afd020b0 0x5580afd092e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.808+0000 7f31a37fe640 1 -- 172.21.15.114:0/3957067210 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f319400c6f0 con 0x5580afd01790 2024-09-13T18:32:22.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.809+0000 7f31a37fe640 1 --2- 172.21.15.114:0/3957067210 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580afd01790 0x5580afd0c010 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f3194004680 tx=0x7f3194008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=239ba23b6f01bde0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:22.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.809+0000 7f31837fe640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:32:22.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.813+0000 7f31837fe640 1 -- 172.21.15.114:0/3957067210 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3194024070 con 0x5580afd01790 2024-09-13T18:32:22.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.813+0000 7f31837fe640 1 -- 172.21.15.114:0/3957067210 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f3194003a90 con 0x5580afd01790 2024-09-13T18:32:22.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.814+0000 7f31ad42a740 1 -- 172.21.15.114:0/3957067210 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5580afc8eb80 con 0x5580afd01790 2024-09-13T18:32:22.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.816+0000 7f31837fe640 1 -- 172.21.15.114:0/3957067210 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3194017070 con 0x5580afd01790 2024-09-13T18:32:22.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.816+0000 7f31ad42a740 10 client.27725 fetch_fsmap learned FSMap version 34 2024-09-13T18:32:22.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.816+0000 7f31ad42a740 1 -- 172.21.15.114:0/3957067210 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5580afc633c0 con 0x5580afd01790 2024-09-13T18:32:22.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.816+0000 7f31837fe640 1 -- 172.21.15.114:0/3957067210 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5580afc8eb80 con 0x5580afd01790 2024-09-13T18:32:22.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.816+0000 7f31837fe640 1 -- 172.21.15.114:0/3957067210 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3194027030 con 0x5580afd01790 2024-09-13T18:32:22.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.817+0000 7f31ad42a740 10 client.27725 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:32:22.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.817+0000 7f31ad42a740 10 client.27725 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-13T18:32:22.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.817+0000 7f31ad42a740 20 client.27725 populate_metadata read hostname 'smithi114' 2024-09-13T18:32:22.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.818+0000 7f31ad42a740 1 --2- 172.21.15.114:0/3957067210 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x5580afd1c330 0x5580afd1e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:22.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.818+0000 7f31ad42a740 4 client.27725 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:32:22.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.818+0000 7f31ad42a740 1 -- 172.21.15.114:0/3957067210 --> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5580afc571f0 con 0x5580afd1c330 2024-09-13T18:32:22.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.818+0000 7f31a3fff640 1 --2- 172.21.15.114:0/3957067210 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x5580afd1c330 0x5580afd1e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:22.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.818+0000 7f31a3fff640 1 --2- 172.21.15.114:0/3957067210 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x5580afd1c330 0x5580afd1e720 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=478d9ed42737b546 server_cookie=3fdde007ffc79f1f in_seq=0 out_seq=0 2024-09-13T18:32:22.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.819+0000 7f31837fe640 10 client.27725 ms_handle_connect on v2:172.21.15.134:6834/1150003275 2024-09-13T18:32:22.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.820+0000 7f31837fe640 1 -- 172.21.15.114:0/3957067210 <== mds.3 v2:172.21.15.134:6834/1150003275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5580afc571f0 con 0x5580afd1c330 2024-09-13T18:32:22.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.820+0000 7f31837fe640 10 client.27725 handle_command_reply: tid=0 2024-09-13T18:32:22.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.820+0000 7f31ad42a740 1 client.27725 shutdown 2024-09-13T18:32:22.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.820+0000 7f31ad42a740 1 -- 172.21.15.114:0/3957067210 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580afd01790 msgr2=0x5580afd0c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:22.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.820+0000 7f31ad42a740 1 --2- 172.21.15.114:0/3957067210 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580afd01790 0x5580afd0c010 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f3194004680 tx=0x7f3194008040 comp rx=0 tx=0).stop 2024-09-13T18:32:22.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.821+0000 7f31ad42a740 1 -- 172.21.15.114:0/3957067210 shutdown_connections 2024-09-13T18:32:22.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.821+0000 7f31ad42a740 1 --2- 172.21.15.114:0/3957067210 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x5580afd1c330 0x5580afd1e720 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.821+0000 7f31ad42a740 1 --2- 172.21.15.114:0/3957067210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580afd0c950 0x5580afd09820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.821+0000 7f31ad42a740 1 --2- 172.21.15.114:0/3957067210 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580afd020b0 0x5580afd092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.821+0000 7f31ad42a740 1 --2- 172.21.15.114:0/3957067210 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580afd01790 0x5580afd0c010 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.821+0000 7f31ad42a740 1 -- 172.21.15.114:0/3957067210 >> 172.21.15.114:0/3957067210 conn(0x5580afcfdd00 msgr2=0x5580afd03a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:22.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.821+0000 7f31837fe640 0 client.27725 ms_handle_reset on v2:172.21.15.134:6834/1150003275 2024-09-13T18:32:22.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.821+0000 7f31ad42a740 1 -- 172.21.15.114:0/3957067210 shutdown_connections 2024-09-13T18:32:22.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.821+0000 7f31ad42a740 1 -- 172.21.15.114:0/3957067210 wait complete. 2024-09-13T18:32:22.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.822+0000 7f31ad42a740 20 client.27725 trim_cache size 0 max 16384 2024-09-13T18:32:22.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.853+0000 7f31ad42a740 1 Processor -- start 2024-09-13T18:32:22.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.856+0000 7f31ad42a740 1 -- start start 2024-09-13T18:32:22.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.856+0000 7f31ad42a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580afd01790 0x5580afd1e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:22.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.857+0000 7f31ad42a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580afd020b0 0x5580afd13670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:22.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.857+0000 7f31ad42a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580afd0c950 0x5580afd13bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:22.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.857+0000 7f31ad42a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580afcfe9f0 con 0x5580afd020b0 2024-09-13T18:32:22.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.857+0000 7f31ad42a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580afd0a270 con 0x5580afd01790 2024-09-13T18:32:22.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.857+0000 7f31ad42a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580afd238a0 con 0x5580afd0c950 2024-09-13T18:32:22.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.858+0000 7f31a37fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580afd01790 0x5580afd1e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:22.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.858+0000 7f31a37fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580afd01790 0x5580afd1e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:42460/0 (socket says 172.21.15.114:42460) 2024-09-13T18:32:22.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.858+0000 7f31a37fe640 1 -- 172.21.15.114:0/1820620884 learned_addr learned my addr 172.21.15.114:0/1820620884 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:22.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.858+0000 7f31a37fe640 1 --2- 172.21.15.114:0/1820620884 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580afd0c950 0x5580afd13bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:22.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.858+0000 7f31a37fe640 1 -- 172.21.15.114:0/1820620884 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580afd0c950 msgr2=0x5580afd13bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:22.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.858+0000 7f31a37fe640 1 --2- 172.21.15.114:0/1820620884 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580afd0c950 0x5580afd13bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.858+0000 7f31a37fe640 1 -- 172.21.15.114:0/1820620884 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580afd020b0 msgr2=0x5580afd13670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:32:22.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.858+0000 7f31a37fe640 1 --2- 172.21.15.114:0/1820620884 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580afd020b0 0x5580afd13670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.858+0000 7f31a37fe640 1 -- 172.21.15.114:0/1820620884 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580afc05770 con 0x5580afd01790 2024-09-13T18:32:22.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.859+0000 7f31a37fe640 1 --2- 172.21.15.114:0/1820620884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580afd01790 0x5580afd1e340 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f31940043b0 tx=0x7f3194003c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3a8b149217f55c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:22.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.862+0000 7f318affd640 1 -- 172.21.15.114:0/1820620884 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3194024040 con 0x5580afd01790 2024-09-13T18:32:22.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.862+0000 7f318affd640 1 -- 172.21.15.114:0/1820620884 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f319400c350 con 0x5580afd01790 2024-09-13T18:32:22.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.862+0000 7f318affd640 1 -- 172.21.15.114:0/1820620884 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3194017040 con 0x5580afd01790 2024-09-13T18:32:22.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.863+0000 7f31ad42a740 1 -- 172.21.15.114:0/1820620884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580afd01790 msgr2=0x5580afd1e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:22.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.864+0000 7f31ad42a740 1 --2- 172.21.15.114:0/1820620884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580afd01790 0x5580afd1e340 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f31940043b0 tx=0x7f3194003c50 comp rx=0 tx=0).stop 2024-09-13T18:32:22.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.866+0000 7f31ad42a740 1 -- 172.21.15.114:0/1820620884 shutdown_connections 2024-09-13T18:32:22.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.866+0000 7f31ad42a740 1 --2- 172.21.15.114:0/1820620884 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580afd0c950 0x5580afd13bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.866+0000 7f31ad42a740 1 --2- 172.21.15.114:0/1820620884 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580afd020b0 0x5580afd13670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.866+0000 7f31ad42a740 1 --2- 172.21.15.114:0/1820620884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580afd01790 0x5580afd1e340 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.866+0000 7f31ad42a740 1 -- 172.21.15.114:0/1820620884 >> 172.21.15.114:0/1820620884 conn(0x5580afcfdd00 msgr2=0x5580afd18130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:22.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.870+0000 7f31ad42a740 1 -- 172.21.15.114:0/1820620884 shutdown_connections 2024-09-13T18:32:22.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.870+0000 7f31ad42a740 1 -- 172.21.15.114:0/1820620884 wait complete. 2024-09-13T18:32:22.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.870+0000 7f31ad42a740 1 -- start start 2024-09-13T18:32:22.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.870+0000 7f31ad42a740 1 Processor -- start 2024-09-13T18:32:22.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.870+0000 7f31ad42a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580afd01790 0x5580afd0f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:22.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.871+0000 7f31ad42a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580afd020b0 0x5580afd0fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:22.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.871+0000 7f31ad42a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580afd0c950 0x5580afd717f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:22.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.871+0000 7f31ad42a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580afd104a0 con 0x5580afd020b0 2024-09-13T18:32:22.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.871+0000 7f31ad42a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580afd71d30 con 0x5580afd01790 2024-09-13T18:32:22.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.871+0000 7f31ad42a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580afd71ea0 con 0x5580afd0c950 2024-09-13T18:32:22.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.871+0000 7f31ad42a740 10 client.0 _ll_register_callbacks cb 0x7f319400c270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:32:22.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.871+0000 7f31a3fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580afd020b0 0x5580afd0fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:22.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.871+0000 7f31a37fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580afd01790 0x5580afd0f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:22.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.871+0000 7f31a37fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580afd01790 0x5580afd0f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:42468/0 (socket says 172.21.15.114:42468) 2024-09-13T18:32:22.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.871+0000 7f31a37fe640 1 -- 172.21.15.114:0/297674683 learned_addr learned my addr 172.21.15.114:0/297674683 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:22.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.875+0000 7f31a37fe640 1 --2- 172.21.15.114:0/297674683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580afd0c950 0x5580afd717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:22.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.875+0000 7f31a3fff640 1 -- 172.21.15.114:0/297674683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580afd0c950 msgr2=0x5580afd717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:22.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.875+0000 7f31a3fff640 1 --2- 172.21.15.114:0/297674683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580afd0c950 0x5580afd717f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.875+0000 7f31a3fff640 1 -- 172.21.15.114:0/297674683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580afd01790 msgr2=0x5580afd0f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:22.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.875+0000 7f31a3fff640 1 --2- 172.21.15.114:0/297674683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580afd01790 0x5580afd0f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.875+0000 7f31a3fff640 1 -- 172.21.15.114:0/297674683 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5580afc633c0 con 0x5580afd020b0 2024-09-13T18:32:22.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.876+0000 7f31a3fff640 1 --2- 172.21.15.114:0/297674683 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580afd020b0 0x5580afd0fd80 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f319c02ace0 tx=0x7f319c031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=631a4dfd1b84b008 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:22.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.876+0000 7f31837fe640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:32:22.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.880+0000 7f31837fe640 1 -- 172.21.15.114:0/297674683 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319c042070 con 0x5580afd020b0 2024-09-13T18:32:22.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.880+0000 7f31ad42a740 1 -- 172.21.15.114:0/297674683 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f31940044b0 con 0x5580afd020b0 2024-09-13T18:32:22.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.880+0000 7f31837fe640 1 -- 172.21.15.114:0/297674683 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f319c030820 con 0x5580afd020b0 2024-09-13T18:32:22.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.887+0000 7f31837fe640 1 -- 172.21.15.114:0/297674683 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f319c030a00 con 0x5580afd020b0 2024-09-13T18:32:22.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.889+0000 7f31ad42a740 10 client.18243 fetch_fsmap learned FSMap version 34 2024-09-13T18:32:22.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.889+0000 7f31ad42a740 1 -- 172.21.15.114:0/297674683 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5580afc2ae80 con 0x5580afd020b0 2024-09-13T18:32:22.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.894+0000 7f31837fe640 1 -- 172.21.15.114:0/297674683 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f31940044b0 con 0x5580afd020b0 2024-09-13T18:32:22.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.894+0000 7f31837fe640 1 -- 172.21.15.114:0/297674683 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f319c0423e0 con 0x5580afd020b0 2024-09-13T18:32:22.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.894+0000 7f31ad42a740 10 client.18243 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:32:22.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.894+0000 7f31ad42a740 10 client.18243 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-13T18:32:22.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.895+0000 7f31ad42a740 20 client.18243 populate_metadata read hostname 'smithi114' 2024-09-13T18:32:22.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.895+0000 7f31ad42a740 1 --2- 172.21.15.114:0/297674683 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x5580afd73670 0x5580afd75a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:22.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.895+0000 7f31ad42a740 4 client.18243 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:32:22.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.895+0000 7f31ad42a740 1 -- 172.21.15.114:0/297674683 --> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5580af9584e0 con 0x5580afd73670 2024-09-13T18:32:22.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.895+0000 7f31a37fe640 1 --2- 172.21.15.114:0/297674683 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x5580afd73670 0x5580afd75a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:22.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.895+0000 7f31a37fe640 1 --2- 172.21.15.114:0/297674683 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x5580afd73670 0x5580afd75a90 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2460e3169836d09e server_cookie=2c4a011babf6152d in_seq=0 out_seq=0 2024-09-13T18:32:22.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.896+0000 7f31837fe640 10 client.18243 ms_handle_connect on v2:172.21.15.134:6834/1150003275 2024-09-13T18:32:22.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.896+0000 7f31837fe640 1 -- 172.21.15.114:0/297674683 <== mds.3 v2:172.21.15.134:6834/1150003275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5580af9584e0 con 0x5580afd73670 2024-09-13T18:32:22.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.896+0000 7f31837fe640 10 client.18243 handle_command_reply: tid=0 2024-09-13T18:32:22.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.896+0000 7f31ad42a740 1 client.18243 shutdown 2024-09-13T18:32:22.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.898+0000 7f31ad42a740 1 -- 172.21.15.114:0/297674683 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580afd020b0 msgr2=0x5580afd0fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:22.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.898+0000 7f31ad42a740 1 --2- 172.21.15.114:0/297674683 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580afd020b0 0x5580afd0fd80 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f319c02ace0 tx=0x7f319c031000 comp rx=0 tx=0).stop 2024-09-13T18:32:22.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.899+0000 7f31ad42a740 1 -- 172.21.15.114:0/297674683 shutdown_connections 2024-09-13T18:32:22.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.899+0000 7f31ad42a740 1 --2- 172.21.15.114:0/297674683 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x5580afd73670 0x5580afd75a90 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.899+0000 7f31ad42a740 1 --2- 172.21.15.114:0/297674683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580afd0c950 0x5580afd717f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.899+0000 7f31ad42a740 1 --2- 172.21.15.114:0/297674683 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580afd020b0 0x5580afd0fd80 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.899+0000 7f31ad42a740 1 --2- 172.21.15.114:0/297674683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580afd01790 0x5580afd0f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.899+0000 7f31ad42a740 1 -- 172.21.15.114:0/297674683 >> 172.21.15.114:0/297674683 conn(0x5580afcfdd00 msgr2=0x5580afd08b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:22.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.901+0000 7f31837fe640 0 client.18243 ms_handle_reset on v2:172.21.15.134:6834/1150003275 2024-09-13T18:32:22.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.901+0000 7f31ad42a740 1 -- 172.21.15.114:0/297674683 shutdown_connections 2024-09-13T18:32:22.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.904+0000 7f31ad42a740 1 -- 172.21.15.114:0/297674683 wait complete. 2024-09-13T18:32:22.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.905+0000 7f31ad42a740 20 client.18243 trim_cache size 0 max 16384 2024-09-13T18:32:22.909 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:32:22.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.907+0000 7f31aa62b640 1 -- 172.21.15.114:0/2289924077 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f318c0a1410 msgr2=0x7f318c0a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:22.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.907+0000 7f31aa62b640 1 --2- 172.21.15.114:0/2289924077 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f318c0a1410 0x7f318c0a38d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f319800c6a0 tx=0x7f3198009ef0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.907+0000 7f31aa62b640 1 -- 172.21.15.114:0/2289924077 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31a4119400 msgr2=0x7f31a410fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:22.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.907+0000 7f31aa62b640 1 --2- 172.21.15.114:0/2289924077 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31a4119400 0x7f31a410fbc0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f319800e970 tx=0x7f319800ee30 comp rx=0 tx=0).stop 2024-09-13T18:32:22.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.907+0000 7f31aa62b640 1 -- 172.21.15.114:0/2289924077 shutdown_connections 2024-09-13T18:32:22.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.907+0000 7f31aa62b640 1 --2- 172.21.15.114:0/2289924077 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f318c0a1410 0x7f318c0a38d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.907+0000 7f31aa62b640 1 --2- 172.21.15.114:0/2289924077 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31a4119400 0x7f31a410fbc0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.907+0000 7f31aa62b640 1 --2- 172.21.15.114:0/2289924077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31a4107920 0x7f31a4116e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.907+0000 7f31aa62b640 1 --2- 172.21.15.114:0/2289924077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31a4107540 0x7f31a4116910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:22.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.907+0000 7f31aa62b640 1 -- 172.21.15.114:0/2289924077 >> 172.21.15.114:0/2289924077 conn(0x7f31a406afe0 msgr2=0x7f31a4119d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:22.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.908+0000 7f31aa62b640 1 -- 172.21.15.114:0/2289924077 shutdown_connections 2024-09-13T18:32:22.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:22.908+0000 7f31aa62b640 1 -- 172.21.15.114:0/2289924077 wait complete. 2024-09-13T18:32:22.931 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-09-13T18:32:23.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:22 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/4133539276' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:32:23.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.189+0000 7f7afd195640 1 -- 172.21.15.114:0/1735015620 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7af8107e30 msgr2=0x7f7af806f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:23.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.189+0000 7f7afd195640 1 --2- 172.21.15.114:0/1735015620 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7af8107e30 0x7f7af806f250 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f7aec009900 tx=0x7f7aec02a440 comp rx=0 tx=0).stop 2024-09-13T18:32:23.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.189+0000 7f7afd195640 1 -- 172.21.15.114:0/1735015620 shutdown_connections 2024-09-13T18:32:23.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.189+0000 7f7afd195640 1 --2- 172.21.15.114:0/1735015620 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7af8107e30 0x7f7af806f250 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.189+0000 7f7afd195640 1 --2- 172.21.15.114:0/1735015620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7af81074b0 0x7f7af81078f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.189+0000 7f7afd195640 1 --2- 172.21.15.114:0/1735015620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7af810d5b0 0x7f7af810d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.189+0000 7f7afd195640 1 -- 172.21.15.114:0/1735015620 >> 172.21.15.114:0/1735015620 conn(0x7f7af806a030 msgr2=0x7f7af806c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:23.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.194+0000 7f7afd195640 1 -- 172.21.15.114:0/1735015620 shutdown_connections 2024-09-13T18:32:23.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.199+0000 7f7afd195640 1 -- 172.21.15.114:0/1735015620 wait complete. 2024-09-13T18:32:23.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.199+0000 7f7afd195640 1 Processor -- start 2024-09-13T18:32:23.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.200+0000 7f7afd195640 1 -- start start 2024-09-13T18:32:23.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.200+0000 7f7afd195640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7af81074b0 0x7f7af81139b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:23.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.200+0000 7f7afd195640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7af8107e30 0x7f7af8113ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:23.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.200+0000 7f7afd195640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7af810d5b0 0x7f7af81a6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:23.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.200+0000 7f7afd195640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af81a7380 con 0x7f7af810d5b0 2024-09-13T18:32:23.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.200+0000 7f7afd195640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af81a74f0 con 0x7f7af8107e30 2024-09-13T18:32:23.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.200+0000 7f7afd195640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af81a76b0 con 0x7f7af81074b0 2024-09-13T18:32:23.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.200+0000 7f7af759e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7af810d5b0 0x7f7af81a6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:23.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.200+0000 7f7af759e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7af810d5b0 0x7f7af81a6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57780/0 (socket says 172.21.15.114:57780) 2024-09-13T18:32:23.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.200+0000 7f7af759e640 1 -- 172.21.15.114:0/2739374504 learned_addr learned my addr 172.21.15.114:0/2739374504 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:23.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.200+0000 7f7af659c640 1 --2- 172.21.15.114:0/2739374504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7af8107e30 0x7f7af8113ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:23.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.201+0000 7f7af759e640 1 -- 172.21.15.114:0/2739374504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7af81074b0 msgr2=0x7f7af81139b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:32:23.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.201+0000 7f7af759e640 1 --2- 172.21.15.114:0/2739374504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7af81074b0 0x7f7af81139b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.201+0000 7f7af759e640 1 -- 172.21.15.114:0/2739374504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7af8107e30 msgr2=0x7f7af8113ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:23.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.201+0000 7f7af759e640 1 --2- 172.21.15.114:0/2739374504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7af8107e30 0x7f7af8113ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.201+0000 7f7af759e640 1 -- 172.21.15.114:0/2739374504 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aec009560 con 0x7f7af810d5b0 2024-09-13T18:32:23.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.201+0000 7f7af759e640 1 --2- 172.21.15.114:0/2739374504 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7af810d5b0 0x7f7af81a6bf0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f7aec02aa30 tx=0x7f7aec004c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:23.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.202+0000 7f7adffff640 1 -- 172.21.15.114:0/2739374504 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec004e10 con 0x7f7af810d5b0 2024-09-13T18:32:23.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.202+0000 7f7afd195640 1 -- 172.21.15.114:0/2739374504 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7af81a7930 con 0x7f7af810d5b0 2024-09-13T18:32:23.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.202+0000 7f7afd195640 1 -- 172.21.15.114:0/2739374504 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7af8119910 con 0x7f7af810d5b0 2024-09-13T18:32:23.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.202+0000 7f7adffff640 1 -- 172.21.15.114:0/2739374504 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7aec03bb00 con 0x7f7af810d5b0 2024-09-13T18:32:23.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.202+0000 7f7adffff640 1 -- 172.21.15.114:0/2739374504 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec031a20 con 0x7f7af810d5b0 2024-09-13T18:32:23.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.207+0000 7f7adffff640 1 -- 172.21.15.114:0/2739374504 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f7aec03bc70 con 0x7f7af810d5b0 2024-09-13T18:32:23.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.208+0000 7f7affebb740 1 Processor -- start 2024-09-13T18:32:23.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.208+0000 7f7affebb740 1 -- start start 2024-09-13T18:32:23.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.208+0000 7f7affebb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55796b25f790 0x55796b25fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:23.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.208+0000 7f7affebb740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55796b2600b0 0x55796b26a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:23.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.208+0000 7f7affebb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55796b26a950 0x55796b26cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:23.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.208+0000 7f7affebb740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55796b1c9cc0 con 0x55796b2600b0 2024-09-13T18:32:23.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.208+0000 7f7affebb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55796b1c7450 con 0x55796b25f790 2024-09-13T18:32:23.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.208+0000 7f7affebb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55796b1c2d40 con 0x55796b26a950 2024-09-13T18:32:23.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.209+0000 7f7af659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55796b25f790 0x55796b25fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:23.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.209+0000 7f7af659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55796b25f790 0x55796b25fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:42506/0 (socket says 172.21.15.114:42506) 2024-09-13T18:32:23.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.209+0000 7f7af659c640 1 -- 172.21.15.114:0/3382590418 learned_addr learned my addr 172.21.15.114:0/3382590418 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:23.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.209+0000 7f7af659c640 1 --2- 172.21.15.114:0/3382590418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55796b26a950 0x55796b26cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:23.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.209+0000 7f7adffff640 1 --2- 172.21.15.114:0/2739374504 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7ae00a1410 0x7f7ae00a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:23.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.210+0000 7f7af659c640 1 -- 172.21.15.114:0/3382590418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55796b26a950 msgr2=0x55796b26cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:23.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.210+0000 7f7af659c640 1 --2- 172.21.15.114:0/3382590418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55796b26a950 0x55796b26cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.210+0000 7f7af659c640 1 -- 172.21.15.114:0/3382590418 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55796b2600b0 msgr2=0x55796b26a410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:32:23.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.210+0000 7f7af759e640 1 --2- 172.21.15.114:0/2739374504 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7ae00a1410 0x7f7ae00a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:23.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.210+0000 7f7adffff640 1 -- 172.21.15.114:0/2739374504 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f7aec0ed8b0 con 0x7f7af810d5b0 2024-09-13T18:32:23.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.210+0000 7f7af6d9d640 1 --2- 172.21.15.114:0/3382590418 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55796b2600b0 0x55796b26a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:23.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.211+0000 7f7af759e640 1 --2- 172.21.15.114:0/2739374504 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7ae00a1410 0x7f7ae00a38d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7aec0ba260 tx=0x7f7aec085a70 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:23.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.216+0000 7f7af659c640 1 --2- 172.21.15.114:0/3382590418 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55796b2600b0 0x55796b26a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.216+0000 7f7af659c640 1 -- 172.21.15.114:0/3382590418 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55796b18a620 con 0x55796b25f790 2024-09-13T18:32:23.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.218+0000 7f7af6d9d640 1 --2- 172.21.15.114:0/3382590418 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55796b2600b0 0x55796b26a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:32:23.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.218+0000 7f7af659c640 1 --2- 172.21.15.114:0/3382590418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55796b25f790 0x55796b25fb70 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7af0014c90 tx=0x7f7af0009790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=676e09f6b4e9b717 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:23.223 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:22 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/4133539276' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:32:23.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.220+0000 7f7add7fa640 1 -- 172.21.15.114:0/3382590418 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af0014310 con 0x55796b25f790 2024-09-13T18:32:23.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.220+0000 7f7add7fa640 1 -- 172.21.15.114:0/3382590418 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7af001a030 con 0x55796b25f790 2024-09-13T18:32:23.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.221+0000 7f7affebb740 1 -- 172.21.15.114:0/3382590418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55796b25f790 msgr2=0x55796b25fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:23.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.221+0000 7f7affebb740 1 --2- 172.21.15.114:0/3382590418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55796b25f790 0x55796b25fb70 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7af0014c90 tx=0x7f7af0009790 comp rx=0 tx=0).stop 2024-09-13T18:32:23.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.221+0000 7f7affebb740 1 -- 172.21.15.114:0/3382590418 shutdown_connections 2024-09-13T18:32:23.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.221+0000 7f7affebb740 1 --2- 172.21.15.114:0/3382590418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55796b26a950 0x55796b26cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.221+0000 7f7affebb740 1 --2- 172.21.15.114:0/3382590418 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55796b2600b0 0x55796b26a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.221+0000 7f7affebb740 1 --2- 172.21.15.114:0/3382590418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55796b25f790 0x55796b25fb70 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.221+0000 7f7affebb740 1 -- 172.21.15.114:0/3382590418 >> 172.21.15.114:0/3382590418 conn(0x55796b25bd00 msgr2=0x55796b25c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:23.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.225+0000 7f7affebb740 1 -- 172.21.15.114:0/3382590418 shutdown_connections 2024-09-13T18:32:23.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.226+0000 7f7affebb740 1 -- 172.21.15.114:0/3382590418 wait complete. 2024-09-13T18:32:23.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.227+0000 7f7affebb740 1 -- start start 2024-09-13T18:32:23.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.227+0000 7f7affebb740 1 Processor -- start 2024-09-13T18:32:23.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.229+0000 7f7affebb740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55796b25f790 0x55796b269ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:23.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.229+0000 7f7affebb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55796b2600b0 0x55796b26a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:23.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.229+0000 7f7affebb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55796b26a950 0x55796b2674f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:23.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.229+0000 7f7affebb740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55796b181180 con 0x55796b25f790 2024-09-13T18:32:23.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.229+0000 7f7affebb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55796ae9b430 con 0x55796b2600b0 2024-09-13T18:32:23.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.229+0000 7f7affebb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55796b1465e0 con 0x55796b26a950 2024-09-13T18:32:23.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.229+0000 7f7affebb740 10 client.0 _ll_register_callbacks cb 0x55796b1b2aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:32:23.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.229+0000 7f7af6d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55796b2600b0 0x55796b26a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:23.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.229+0000 7f7af6d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55796b2600b0 0x55796b26a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:42512/0 (socket says 172.21.15.114:42512) 2024-09-13T18:32:23.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.229+0000 7f7af6d9d640 1 -- 172.21.15.114:0/3488159486 learned_addr learned my addr 172.21.15.114:0/3488159486 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:23.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.229+0000 7f7af659c640 1 --2- 172.21.15.114:0/3488159486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55796b26a950 0x55796b2674f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:23.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.229+0000 7f7af659c640 1 --2- 172.21.15.114:0/3488159486 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55796b25f790 0x55796b269ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:23.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.230+0000 7f7af6d9d640 1 -- 172.21.15.114:0/3488159486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55796b26a950 msgr2=0x55796b2674f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:23.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.230+0000 7f7af6d9d640 1 --2- 172.21.15.114:0/3488159486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55796b26a950 0x55796b2674f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.230+0000 7f7af6d9d640 1 -- 172.21.15.114:0/3488159486 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55796b25f790 msgr2=0x55796b269ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:23.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.230+0000 7f7af6d9d640 1 --2- 172.21.15.114:0/3488159486 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55796b25f790 0x55796b269ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.230+0000 7f7af6d9d640 1 -- 172.21.15.114:0/3488159486 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7af0014940 con 0x55796b2600b0 2024-09-13T18:32:23.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.230+0000 7f7af6d9d640 1 --2- 172.21.15.114:0/3488159486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55796b2600b0 0x55796b26a400 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f7ae800e2f0 tx=0x7f7ae800e7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b61b19c582084b93 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:23.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.231+0000 7f7adaff5640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:32:23.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.231+0000 7f7adaff5640 1 -- 172.21.15.114:0/3488159486 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae8005220 con 0x55796b2600b0 2024-09-13T18:32:23.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.231+0000 7f7affebb740 1 -- 172.21.15.114:0/3488159486 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55796b1ecb80 con 0x55796b2600b0 2024-09-13T18:32:23.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.231+0000 7f7adaff5640 1 -- 172.21.15.114:0/3488159486 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f7ae8005380 con 0x55796b2600b0 2024-09-13T18:32:23.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.232+0000 7f7adaff5640 1 -- 172.21.15.114:0/3488159486 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ae8005540 con 0x55796b2600b0 2024-09-13T18:32:23.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.232+0000 7f7affebb740 10 client.27743 fetch_fsmap learned FSMap version 34 2024-09-13T18:32:23.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.232+0000 7f7affebb740 1 -- 172.21.15.114:0/3488159486 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55796b1c13c0 con 0x55796b2600b0 2024-09-13T18:32:23.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.233+0000 7f7adaff5640 1 -- 172.21.15.114:0/3488159486 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55796b1ecb80 con 0x55796b2600b0 2024-09-13T18:32:23.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.234+0000 7f7adaff5640 1 -- 172.21.15.114:0/3488159486 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7ae800c070 con 0x55796b2600b0 2024-09-13T18:32:23.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.234+0000 7f7affebb740 10 client.27743 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:32:23.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.234+0000 7f7affebb740 10 client.27743 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-13T18:32:23.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.234+0000 7f7affebb740 20 client.27743 populate_metadata read hostname 'smithi114' 2024-09-13T18:32:23.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.234+0000 7f7affebb740 1 --2- 172.21.15.114:0/3488159486 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55796b27a3f0 0x55796b27c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:23.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.234+0000 7f7affebb740 4 client.27743 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:32:23.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.234+0000 7f7affebb740 1 -- 172.21.15.114:0/3488159486 --> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55796b1b51f0 con 0x55796b27a3f0 2024-09-13T18:32:23.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.235+0000 7f7af659c640 1 --2- 172.21.15.114:0/3488159486 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55796b27a3f0 0x55796b27c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:23.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.236+0000 7f7af659c640 1 --2- 172.21.15.114:0/3488159486 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55796b27a3f0 0x55796b27c810 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d0f170a1727099ff server_cookie=b7c39f93393764d3 in_seq=0 out_seq=0 2024-09-13T18:32:23.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.236+0000 7f7adaff5640 10 client.27743 ms_handle_connect on v2:172.21.15.204:6838/344883912 2024-09-13T18:32:23.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.237+0000 7f7adaff5640 1 -- 172.21.15.114:0/3488159486 <== mds.2 v2:172.21.15.204:6838/344883912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55796b1b51f0 con 0x55796b27a3f0 2024-09-13T18:32:23.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.237+0000 7f7adaff5640 10 client.27743 handle_command_reply: tid=0 2024-09-13T18:32:23.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.237+0000 7f7affebb740 1 client.27743 shutdown 2024-09-13T18:32:23.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.238+0000 7f7affebb740 1 -- 172.21.15.114:0/3488159486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55796b2600b0 msgr2=0x55796b26a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:23.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.238+0000 7f7affebb740 1 --2- 172.21.15.114:0/3488159486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55796b2600b0 0x55796b26a400 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f7ae800e2f0 tx=0x7f7ae800e7b0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.238+0000 7f7affebb740 1 -- 172.21.15.114:0/3488159486 shutdown_connections 2024-09-13T18:32:23.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.238+0000 7f7affebb740 1 --2- 172.21.15.114:0/3488159486 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55796b27a3f0 0x55796b27c810 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.238+0000 7f7affebb740 1 --2- 172.21.15.114:0/3488159486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55796b26a950 0x55796b2674f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.238+0000 7f7affebb740 1 --2- 172.21.15.114:0/3488159486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55796b2600b0 0x55796b26a400 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.238+0000 7f7affebb740 1 --2- 172.21.15.114:0/3488159486 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55796b25f790 0x55796b269ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.238+0000 7f7affebb740 1 -- 172.21.15.114:0/3488159486 >> 172.21.15.114:0/3488159486 conn(0x55796b25bd00 msgr2=0x55796b261a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:23.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.238+0000 7f7adaff5640 0 client.27743 ms_handle_reset on v2:172.21.15.204:6838/344883912 2024-09-13T18:32:23.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.238+0000 7f7affebb740 1 -- 172.21.15.114:0/3488159486 shutdown_connections 2024-09-13T18:32:23.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.238+0000 7f7affebb740 1 -- 172.21.15.114:0/3488159486 wait complete. 2024-09-13T18:32:23.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.240+0000 7f7affebb740 20 client.27743 trim_cache size 0 max 16384 2024-09-13T18:32:23.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.268+0000 7f7affebb740 1 Processor -- start 2024-09-13T18:32:23.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.269+0000 7f7affebb740 1 -- start start 2024-09-13T18:32:23.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.269+0000 7f7affebb740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55796b25f790 0x55796b25b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:23.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.269+0000 7f7affebb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55796b2600b0 0x55796b265b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:23.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.269+0000 7f7affebb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55796b26a950 0x55796b266090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:23.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.269+0000 7f7affebb740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55796b2563b0 con 0x55796b25f790 2024-09-13T18:32:23.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.269+0000 7f7affebb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55796b278bd0 con 0x55796b2600b0 2024-09-13T18:32:23.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.269+0000 7f7affebb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55796b260750 con 0x55796b26a950 2024-09-13T18:32:23.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.274+0000 7f7af6d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55796b2600b0 0x55796b265b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:23.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.274+0000 7f7af659c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55796b26a950 0x55796b266090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:23.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.274+0000 7f7af6d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55796b2600b0 0x55796b265b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:54512/0 (socket says 172.21.15.114:54512) 2024-09-13T18:32:23.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.274+0000 7f7af6d9d640 1 -- 172.21.15.114:0/1569033749 learned_addr learned my addr 172.21.15.114:0/1569033749 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:23.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.275+0000 7f7af6d9d640 1 -- 172.21.15.114:0/1569033749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55796b26a950 msgr2=0x55796b266090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:23.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.275+0000 7f7af6d9d640 1 --2- 172.21.15.114:0/1569033749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55796b26a950 0x55796b266090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.275+0000 7f7af6d9d640 1 -- 172.21.15.114:0/1569033749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55796b25f790 msgr2=0x55796b25b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:23.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.275+0000 7f7af6d9d640 1 --2- 172.21.15.114:0/1569033749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55796b25f790 0x55796b25b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.275+0000 7f7af6d9d640 1 -- 172.21.15.114:0/1569033749 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55796b163770 con 0x55796b2600b0 2024-09-13T18:32:23.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.276+0000 7f7af6d9d640 1 --2- 172.21.15.114:0/1569033749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55796b2600b0 0x55796b265b50 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8000c00 tx=0x7f7ae8011e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5ba6a914c6ebeb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:23.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.278+0000 7f7addffb640 1 -- 172.21.15.114:0/1569033749 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae8005c50 con 0x55796b2600b0 2024-09-13T18:32:23.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.279+0000 7f7addffb640 1 -- 172.21.15.114:0/1569033749 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7ae80054b0 con 0x55796b2600b0 2024-09-13T18:32:23.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.279+0000 7f7addffb640 1 -- 172.21.15.114:0/1569033749 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae8010740 con 0x55796b2600b0 2024-09-13T18:32:23.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.280+0000 7f7affebb740 1 -- 172.21.15.114:0/1569033749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55796b2600b0 msgr2=0x55796b265b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:23.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.280+0000 7f7affebb740 1 --2- 172.21.15.114:0/1569033749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55796b2600b0 0x55796b265b50 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8000c00 tx=0x7f7ae8011e60 comp rx=0 tx=0).stop 2024-09-13T18:32:23.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.281+0000 7f7affebb740 1 -- 172.21.15.114:0/1569033749 shutdown_connections 2024-09-13T18:32:23.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.281+0000 7f7affebb740 1 --2- 172.21.15.114:0/1569033749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55796b26a950 0x55796b266090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.281+0000 7f7affebb740 1 --2- 172.21.15.114:0/1569033749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55796b2600b0 0x55796b265b50 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.281+0000 7f7affebb740 1 --2- 172.21.15.114:0/1569033749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55796b25f790 0x55796b25b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.281+0000 7f7affebb740 1 -- 172.21.15.114:0/1569033749 >> 172.21.15.114:0/1569033749 conn(0x55796b25bd00 msgr2=0x55796b281170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:23.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.284+0000 7f7affebb740 1 -- 172.21.15.114:0/1569033749 shutdown_connections 2024-09-13T18:32:23.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.285+0000 7f7affebb740 1 -- 172.21.15.114:0/1569033749 wait complete. 2024-09-13T18:32:23.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.285+0000 7f7affebb740 1 -- start start 2024-09-13T18:32:23.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.285+0000 7f7affebb740 1 Processor -- start 2024-09-13T18:32:23.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.286+0000 7f7affebb740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55796b25f790 0x55796b271ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:23.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.286+0000 7f7affebb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55796b2600b0 0x55796b26de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:23.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.286+0000 7f7affebb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55796b26a950 0x55796b26e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:23.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.286+0000 7f7affebb740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55796b26ea30 con 0x55796b25f790 2024-09-13T18:32:23.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.286+0000 7f7affebb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55796b26eba0 con 0x55796b26a950 2024-09-13T18:32:23.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.286+0000 7f7affebb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55796b2d08f0 con 0x55796b2600b0 2024-09-13T18:32:23.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.286+0000 7f7affebb740 10 client.0 _ll_register_callbacks cb 0x7f7af0009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:32:23.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.291+0000 7f7af659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55796b26a950 0x55796b26e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:23.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.291+0000 7f7af659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55796b26a950 0x55796b26e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:54520/0 (socket says 172.21.15.114:54520) 2024-09-13T18:32:23.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.291+0000 7f7af659c640 1 -- 172.21.15.114:0/3538146811 learned_addr learned my addr 172.21.15.114:0/3538146811 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:23.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.291+0000 7f7af659c640 1 --2- 172.21.15.114:0/3538146811 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55796b25f790 0x55796b271ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:23.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.291+0000 7f7af6d9d640 1 --2- 172.21.15.114:0/3538146811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55796b2600b0 0x55796b26de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:23.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.292+0000 7f7af659c640 1 -- 172.21.15.114:0/3538146811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55796b2600b0 msgr2=0x55796b26de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:23.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.292+0000 7f7af659c640 1 --2- 172.21.15.114:0/3538146811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55796b2600b0 0x55796b26de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.292+0000 7f7af659c640 1 -- 172.21.15.114:0/3538146811 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55796b25f790 msgr2=0x55796b271ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:23.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.292+0000 7f7af659c640 1 --2- 172.21.15.114:0/3538146811 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55796b25f790 0x55796b271ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.292+0000 7f7af659c640 1 -- 172.21.15.114:0/3538146811 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55796b1c13c0 con 0x55796b26a950 2024-09-13T18:32:23.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.292+0000 7f7af659c640 1 --2- 172.21.15.114:0/3538146811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55796b26a950 0x55796b26e380 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7af0014c30 tx=0x7f7af0009eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6dd9bc432c486d4a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:23.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.293+0000 7f7adaff5640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:32:23.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.293+0000 7f7adaff5640 1 -- 172.21.15.114:0/3538146811 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af00049e0 con 0x55796b26a950 2024-09-13T18:32:23.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.293+0000 7f7affebb740 1 -- 172.21.15.114:0/3538146811 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7af001a030 con 0x55796b26a950 2024-09-13T18:32:23.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.293+0000 7f7adaff5640 1 -- 172.21.15.114:0/3538146811 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f7af0004b40 con 0x55796b26a950 2024-09-13T18:32:23.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.293+0000 7f7adaff5640 1 -- 172.21.15.114:0/3538146811 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7af0004d00 con 0x55796b26a950 2024-09-13T18:32:23.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.294+0000 7f7affebb740 10 client.27749 fetch_fsmap learned FSMap version 34 2024-09-13T18:32:23.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.294+0000 7f7affebb740 1 -- 172.21.15.114:0/3538146811 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55796b188e80 con 0x55796b26a950 2024-09-13T18:32:23.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.295+0000 7f7adaff5640 1 -- 172.21.15.114:0/3538146811 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7af001a030 con 0x55796b26a950 2024-09-13T18:32:23.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.295+0000 7f7adaff5640 1 -- 172.21.15.114:0/3538146811 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7af00053e0 con 0x55796b26a950 2024-09-13T18:32:23.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.295+0000 7f7affebb740 10 client.27749 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:32:23.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.295+0000 7f7affebb740 10 client.27749 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-13T18:32:23.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.295+0000 7f7affebb740 20 client.27749 populate_metadata read hostname 'smithi114' 2024-09-13T18:32:23.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.295+0000 7f7affebb740 1 --2- 172.21.15.114:0/3538146811 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55796b2d1eb0 0x55796b2d42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:23.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.295+0000 7f7affebb740 4 client.27749 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:32:23.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.295+0000 7f7affebb740 1 -- 172.21.15.114:0/3538146811 --> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55796aeb64e0 con 0x55796b2d1eb0 2024-09-13T18:32:23.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.296+0000 7f7af6d9d640 1 --2- 172.21.15.114:0/3538146811 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55796b2d1eb0 0x55796b2d42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:23.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.297+0000 7f7af6d9d640 1 --2- 172.21.15.114:0/3538146811 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55796b2d1eb0 0x55796b2d42d0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=89cc6e3c71832be1 server_cookie=2432faf18d26c6bd in_seq=0 out_seq=0 2024-09-13T18:32:23.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.299+0000 7f7adaff5640 10 client.27749 ms_handle_connect on v2:172.21.15.204:6838/344883912 2024-09-13T18:32:23.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.299+0000 7f7adaff5640 1 -- 172.21.15.114:0/3538146811 <== mds.2 v2:172.21.15.204:6838/344883912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55796aeb64e0 con 0x55796b2d1eb0 2024-09-13T18:32:23.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.299+0000 7f7adaff5640 10 client.27749 handle_command_reply: tid=0 2024-09-13T18:32:23.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.299+0000 7f7affebb740 1 client.27749 shutdown 2024-09-13T18:32:23.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.299+0000 7f7affebb740 1 -- 172.21.15.114:0/3538146811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55796b26a950 msgr2=0x55796b26e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:23.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.299+0000 7f7affebb740 1 --2- 172.21.15.114:0/3538146811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55796b26a950 0x55796b26e380 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7af0014c30 tx=0x7f7af0009eb0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.299+0000 7f7affebb740 1 -- 172.21.15.114:0/3538146811 shutdown_connections 2024-09-13T18:32:23.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.299+0000 7f7affebb740 1 --2- 172.21.15.114:0/3538146811 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55796b2d1eb0 0x55796b2d42d0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.299+0000 7f7affebb740 1 --2- 172.21.15.114:0/3538146811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55796b26a950 0x55796b26e380 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.299+0000 7f7affebb740 1 --2- 172.21.15.114:0/3538146811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55796b2600b0 0x55796b26de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.299+0000 7f7affebb740 1 --2- 172.21.15.114:0/3538146811 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55796b25f790 0x55796b271ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.299+0000 7f7affebb740 1 -- 172.21.15.114:0/3538146811 >> 172.21.15.114:0/3538146811 conn(0x55796b25bd00 msgr2=0x55796b265a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:23.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.300+0000 7f7adaff5640 0 client.27749 ms_handle_reset on v2:172.21.15.204:6838/344883912 2024-09-13T18:32:23.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.300+0000 7f7affebb740 1 -- 172.21.15.114:0/3538146811 shutdown_connections 2024-09-13T18:32:23.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.300+0000 7f7affebb740 1 -- 172.21.15.114:0/3538146811 wait complete. 2024-09-13T18:32:23.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.300+0000 7f7affebb740 20 client.27749 trim_cache size 0 max 16384 2024-09-13T18:32:23.304 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:32:23.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.302+0000 7f7afd195640 1 -- 172.21.15.114:0/2739374504 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7ae00a1410 msgr2=0x7f7ae00a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:23.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.302+0000 7f7afd195640 1 --2- 172.21.15.114:0/2739374504 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7ae00a1410 0x7f7ae00a38d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7aec0ba260 tx=0x7f7aec085a70 comp rx=0 tx=0).stop 2024-09-13T18:32:23.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.302+0000 7f7afd195640 1 -- 172.21.15.114:0/2739374504 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7af810d5b0 msgr2=0x7f7af81a6bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:23.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.302+0000 7f7afd195640 1 --2- 172.21.15.114:0/2739374504 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7af810d5b0 0x7f7af81a6bf0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f7aec02aa30 tx=0x7f7aec004c60 comp rx=0 tx=0).stop 2024-09-13T18:32:23.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.309+0000 7f7afd195640 1 -- 172.21.15.114:0/2739374504 shutdown_connections 2024-09-13T18:32:23.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.309+0000 7f7afd195640 1 --2- 172.21.15.114:0/2739374504 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7ae00a1410 0x7f7ae00a38d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.309+0000 7f7afd195640 1 --2- 172.21.15.114:0/2739374504 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7af810d5b0 0x7f7af81a6bf0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.309+0000 7f7afd195640 1 --2- 172.21.15.114:0/2739374504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7af8107e30 0x7f7af8113ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.309+0000 7f7afd195640 1 --2- 172.21.15.114:0/2739374504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7af81074b0 0x7f7af81139b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.309+0000 7f7afd195640 1 -- 172.21.15.114:0/2739374504 >> 172.21.15.114:0/2739374504 conn(0x7f7af806a030 msgr2=0x7f7af8109df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:23.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.312+0000 7f7afd195640 1 -- 172.21.15.114:0/2739374504 shutdown_connections 2024-09-13T18:32:23.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.312+0000 7f7afd195640 1 -- 172.21.15.114:0/2739374504 wait complete. 2024-09-13T18:32:23.341 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-09-13T18:32:23.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:22 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/4133539276' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:32:23.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.519+0000 7ff680b25640 1 -- 172.21.15.114:0/3873301866 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff67c10a310 msgr2=0x7ff67c10c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:23.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.519+0000 7ff680b25640 1 --2- 172.21.15.114:0/3873301866 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff67c10a310 0x7ff67c10c790 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7ff674009ca0 tx=0x7ff67402a950 comp rx=0 tx=0).stop 2024-09-13T18:32:23.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.519+0000 7ff680b25640 1 -- 172.21.15.114:0/3873301866 shutdown_connections 2024-09-13T18:32:23.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.519+0000 7ff680b25640 1 --2- 172.21.15.114:0/3873301866 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff67c10a310 0x7ff67c10c790 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.519+0000 7ff680b25640 1 --2- 172.21.15.114:0/3873301866 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff67c101860 0x7ff67c109d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.519+0000 7ff680b25640 1 --2- 172.21.15.114:0/3873301866 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff67c100f40 0x7ff67c101320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.519+0000 7ff680b25640 1 -- 172.21.15.114:0/3873301866 >> 172.21.15.114:0/3873301866 conn(0x7ff67c075c60 msgr2=0x7ff67c0fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:23.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.519+0000 7ff680b25640 1 -- 172.21.15.114:0/3873301866 shutdown_connections 2024-09-13T18:32:23.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.519+0000 7ff680b25640 1 -- 172.21.15.114:0/3873301866 wait complete. 2024-09-13T18:32:23.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.520+0000 7ff680b25640 1 Processor -- start 2024-09-13T18:32:23.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.520+0000 7ff680b25640 1 -- start start 2024-09-13T18:32:23.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.520+0000 7ff680b25640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff67c100f40 0x7ff67c1960f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:23.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.520+0000 7ff680b25640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff67c101860 0x7ff67c196630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:23.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.520+0000 7ff680b25640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff67c10a310 0x7ff67c19aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:23.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.520+0000 7ff680b25640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c19b020 con 0x7ff67c101860 2024-09-13T18:32:23.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.520+0000 7ff680b25640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c19b190 con 0x7ff67c10a310 2024-09-13T18:32:23.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.520+0000 7ff680b25640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c19b350 con 0x7ff67c100f40 2024-09-13T18:32:23.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.520+0000 7ff679d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff67c101860 0x7ff67c196630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:23.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.520+0000 7ff679d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff67c101860 0x7ff67c196630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34880/0 (socket says 172.21.15.114:34880) 2024-09-13T18:32:23.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.520+0000 7ff679d9b640 1 -- 172.21.15.114:0/275556856 learned_addr learned my addr 172.21.15.114:0/275556856 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:23.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.520+0000 7ff67a59c640 1 --2- 172.21.15.114:0/275556856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff67c100f40 0x7ff67c1960f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:23.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.520+0000 7ff679d9b640 1 -- 172.21.15.114:0/275556856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff67c100f40 msgr2=0x7ff67c1960f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:23.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.520+0000 7ff679d9b640 1 --2- 172.21.15.114:0/275556856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff67c100f40 0x7ff67c1960f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.520+0000 7ff679d9b640 1 -- 172.21.15.114:0/275556856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff67c10a310 msgr2=0x7ff67c19aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:23.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.520+0000 7ff679d9b640 1 --2- 172.21.15.114:0/275556856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff67c10a310 0x7ff67c19aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.520+0000 7ff679d9b640 1 -- 172.21.15.114:0/275556856 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff674009900 con 0x7ff67c101860 2024-09-13T18:32:23.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.521+0000 7ff679d9b640 1 --2- 172.21.15.114:0/275556856 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff67c101860 0x7ff67c196630 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7ff67000e9f0 tx=0x7ff67000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:23.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.521+0000 7ff66b7fe640 1 -- 172.21.15.114:0/275556856 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff670009720 con 0x7ff67c101860 2024-09-13T18:32:23.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.521+0000 7ff66b7fe640 1 -- 172.21.15.114:0/275556856 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff6700054b0 con 0x7ff67c101860 2024-09-13T18:32:23.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.521+0000 7ff66b7fe640 1 -- 172.21.15.114:0/275556856 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff670004e80 con 0x7ff67c101860 2024-09-13T18:32:23.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.521+0000 7ff680b25640 1 -- 172.21.15.114:0/275556856 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff67c19b630 con 0x7ff67c101860 2024-09-13T18:32:23.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.521+0000 7ff680b25640 1 -- 172.21.15.114:0/275556856 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff67c1a2eb0 con 0x7ff67c101860 2024-09-13T18:32:23.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.524+0000 7ff66b7fe640 1 -- 172.21.15.114:0/275556856 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7ff670005020 con 0x7ff67c101860 2024-09-13T18:32:23.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.524+0000 7ff68384b740 1 Processor -- start 2024-09-13T18:32:23.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.524+0000 7ff68384b740 1 -- start start 2024-09-13T18:32:23.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.524+0000 7ff68384b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56316b2d2790 0x56316b2d2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:23.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.524+0000 7ff68384b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56316b2d30b0 0x56316b2dd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:23.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.524+0000 7ff68384b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56316b2dd950 0x56316b2dfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:23.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.524+0000 7ff68384b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56316b23ccc0 con 0x56316b2d30b0 2024-09-13T18:32:23.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.524+0000 7ff68384b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56316b23a450 con 0x56316b2d2790 2024-09-13T18:32:23.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.524+0000 7ff68384b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56316b235d40 con 0x56316b2dd950 2024-09-13T18:32:23.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.524+0000 7ff67a59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56316b2d30b0 0x56316b2dd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:23.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.524+0000 7ff67a59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56316b2d30b0 0x56316b2dd410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34884/0 (socket says 172.21.15.114:34884) 2024-09-13T18:32:23.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.524+0000 7ff67a59c640 1 -- 172.21.15.114:0/2852979060 learned_addr learned my addr 172.21.15.114:0/2852979060 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:23.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.524+0000 7ff67ad9d640 1 --2- 172.21.15.114:0/2852979060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56316b2d2790 0x56316b2d2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:23.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.524+0000 7ff679d9b640 1 --2- 172.21.15.114:0/2852979060 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56316b2dd950 0x56316b2dfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:23.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.524+0000 7ff66b7fe640 1 --2- 172.21.15.114:0/275556856 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff6640a1280 0x7ff6640a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:23.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.525+0000 7ff67ad9d640 1 --2- 172.21.15.114:0/275556856 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff6640a1280 0x7ff6640a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:23.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.525+0000 7ff67a59c640 1 -- 172.21.15.114:0/2852979060 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56316b2dd950 msgr2=0x56316b2dfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:23.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.525+0000 7ff67a59c640 1 --2- 172.21.15.114:0/2852979060 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56316b2dd950 0x56316b2dfd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.525+0000 7ff67a59c640 1 -- 172.21.15.114:0/2852979060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56316b2d2790 msgr2=0x56316b2d2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:23.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.525+0000 7ff67a59c640 1 --2- 172.21.15.114:0/2852979060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56316b2d2790 0x56316b2d2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.525+0000 7ff67a59c640 1 -- 172.21.15.114:0/2852979060 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56316b1fd620 con 0x56316b2d30b0 2024-09-13T18:32:23.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.525+0000 7ff66b7fe640 1 -- 172.21.15.114:0/275556856 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ff670014070 con 0x7ff67c101860 2024-09-13T18:32:23.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.525+0000 7ff67a59c640 1 --2- 172.21.15.114:0/2852979060 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56316b2d30b0 0x56316b2dd410 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7ff66c00ad00 tx=0x7ff66c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17c4a4027ec50a41 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:23.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.525+0000 7ff67ad9d640 1 --2- 172.21.15.114:0/275556856 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff6640a1280 0x7ff6640a3740 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff674032000 tx=0x7ff6740095c0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:23.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.525+0000 7ff668ff9640 1 -- 172.21.15.114:0/2852979060 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66c00a800 con 0x56316b2d30b0 2024-09-13T18:32:23.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.525+0000 7ff668ff9640 1 -- 172.21.15.114:0/2852979060 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff66c01c440 con 0x56316b2d30b0 2024-09-13T18:32:23.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.525+0000 7ff668ff9640 1 -- 172.21.15.114:0/2852979060 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66c01a5a0 con 0x56316b2d30b0 2024-09-13T18:32:23.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.527+0000 7ff68384b740 1 -- 172.21.15.114:0/2852979060 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56316b2d30b0 msgr2=0x56316b2dd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:23.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.527+0000 7ff68384b740 1 --2- 172.21.15.114:0/2852979060 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56316b2d30b0 0x56316b2dd410 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7ff66c00ad00 tx=0x7ff66c00f790 comp rx=0 tx=0).stop 2024-09-13T18:32:23.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.542+0000 7ff68384b740 1 -- 172.21.15.114:0/2852979060 shutdown_connections 2024-09-13T18:32:23.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.542+0000 7ff68384b740 1 --2- 172.21.15.114:0/2852979060 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56316b2dd950 0x56316b2dfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.542+0000 7ff68384b740 1 --2- 172.21.15.114:0/2852979060 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56316b2d30b0 0x56316b2dd410 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.542+0000 7ff68384b740 1 --2- 172.21.15.114:0/2852979060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56316b2d2790 0x56316b2d2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.542+0000 7ff68384b740 1 -- 172.21.15.114:0/2852979060 >> 172.21.15.114:0/2852979060 conn(0x56316b2ced00 msgr2=0x56316b2cf0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:23.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.550+0000 7ff68384b740 1 -- 172.21.15.114:0/2852979060 shutdown_connections 2024-09-13T18:32:23.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.553+0000 7ff68384b740 1 -- 172.21.15.114:0/2852979060 wait complete. 2024-09-13T18:32:23.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.554+0000 7ff68384b740 1 -- start start 2024-09-13T18:32:23.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.554+0000 7ff68384b740 1 Processor -- start 2024-09-13T18:32:23.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.554+0000 7ff68384b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56316b2d2790 0x56316b2dcec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:23.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.554+0000 7ff68384b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56316b2d30b0 0x56316b2dd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:23.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.554+0000 7ff68384b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56316b2dd950 0x56316b2da450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:23.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.554+0000 7ff68384b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56316b1f4180 con 0x56316b2d2790 2024-09-13T18:32:23.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.554+0000 7ff68384b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56316af0e430 con 0x56316b2dd950 2024-09-13T18:32:23.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.554+0000 7ff68384b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56316b1b95e0 con 0x56316b2d30b0 2024-09-13T18:32:23.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.555+0000 7ff68384b740 10 client.0 _ll_register_callbacks cb 0x56316b225aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:32:23.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.556+0000 7ff67ad9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56316b2dd950 0x56316b2da450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:23.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.556+0000 7ff67ad9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56316b2dd950 0x56316b2da450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:54566/0 (socket says 172.21.15.114:54566) 2024-09-13T18:32:23.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.556+0000 7ff67ad9d640 1 -- 172.21.15.114:0/2724101324 learned_addr learned my addr 172.21.15.114:0/2724101324 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:23.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.556+0000 7ff679d9b640 1 --2- 172.21.15.114:0/2724101324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56316b2d30b0 0x56316b2dd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:23.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.557+0000 7ff67ad9d640 1 -- 172.21.15.114:0/2724101324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56316b2d30b0 msgr2=0x56316b2dd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:23.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.557+0000 7ff67ad9d640 1 --2- 172.21.15.114:0/2724101324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56316b2d30b0 0x56316b2dd400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.557+0000 7ff67ad9d640 1 -- 172.21.15.114:0/2724101324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56316b2d2790 msgr2=0x56316b2dcec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:23.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.557+0000 7ff67ad9d640 1 --2- 172.21.15.114:0/2724101324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56316b2d2790 0x56316b2dcec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.557+0000 7ff67ad9d640 1 -- 172.21.15.114:0/2724101324 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff66c00a960 con 0x56316b2dd950 2024-09-13T18:32:23.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.561+0000 7ff67ad9d640 1 --2- 172.21.15.114:0/2724101324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56316b2dd950 0x56316b2da450 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff674035000 tx=0x7ff674004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d318d41839b9419b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:23.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.562+0000 7ff661ffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:32:23.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.562+0000 7ff661ffb640 1 -- 172.21.15.114:0/2724101324 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff674030850 con 0x56316b2dd950 2024-09-13T18:32:23.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.562+0000 7ff68384b740 1 -- 172.21.15.114:0/2724101324 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56316b25fb80 con 0x56316b2dd950 2024-09-13T18:32:23.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.562+0000 7ff661ffb640 1 -- 172.21.15.114:0/2724101324 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ff67402f750 con 0x56316b2dd950 2024-09-13T18:32:23.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.568+0000 7ff661ffb640 1 -- 172.21.15.114:0/2724101324 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6740309b0 con 0x56316b2dd950 2024-09-13T18:32:23.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.568+0000 7ff68384b740 10 client.27755 fetch_fsmap learned FSMap version 34 2024-09-13T18:32:23.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.568+0000 7ff68384b740 1 -- 172.21.15.114:0/2724101324 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x56316b2343c0 con 0x56316b2dd950 2024-09-13T18:32:23.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.575+0000 7ff661ffb640 1 -- 172.21.15.114:0/2724101324 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff674059030 con 0x56316b2dd950 2024-09-13T18:32:23.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.575+0000 7ff661ffb640 1 -- 172.21.15.114:0/2724101324 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff67404c6c0 con 0x56316b2dd950 2024-09-13T18:32:23.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.575+0000 7ff68384b740 10 client.27755 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:32:23.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.575+0000 7ff68384b740 10 client.27755 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-13T18:32:23.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.575+0000 7ff68384b740 20 client.27755 populate_metadata read hostname 'smithi114' 2024-09-13T18:32:23.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.575+0000 7ff68384b740 1 --2- 172.21.15.114:0/2724101324 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x56316b2fe250 0x56316b3006b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:23.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.575+0000 7ff68384b740 4 client.27755 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:32:23.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.575+0000 7ff68384b740 1 -- 172.21.15.114:0/2724101324 --> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56316b2281f0 con 0x56316b2fe250 2024-09-13T18:32:23.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.575+0000 7ff67a59c640 1 --2- 172.21.15.114:0/2724101324 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x56316b2fe250 0x56316b3006b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:23.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.576+0000 7ff661ffb640 10 client.27755 ms_handle_connect on v2:172.21.15.134:6836/891692210 2024-09-13T18:32:23.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.577+0000 7ff67a59c640 1 --2- 172.21.15.114:0/2724101324 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x56316b2fe250 0x56316b3006b0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=605620feebc97020 server_cookie=c085e887acb5e3a3 in_seq=0 out_seq=0 2024-09-13T18:32:23.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.577+0000 7ff661ffb640 1 -- 172.21.15.114:0/2724101324 <== mds.1 v2:172.21.15.134:6836/891692210 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x56316b2281f0 con 0x56316b2fe250 2024-09-13T18:32:23.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.577+0000 7ff661ffb640 10 client.27755 handle_command_reply: tid=0 2024-09-13T18:32:23.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.577+0000 7ff68384b740 1 client.27755 shutdown 2024-09-13T18:32:23.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.590+0000 7ff68384b740 1 -- 172.21.15.114:0/2724101324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56316b2dd950 msgr2=0x56316b2da450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:23.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.590+0000 7ff68384b740 1 --2- 172.21.15.114:0/2724101324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56316b2dd950 0x56316b2da450 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff674035000 tx=0x7ff674004a40 comp rx=0 tx=0).stop 2024-09-13T18:32:23.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.591+0000 7ff68384b740 1 -- 172.21.15.114:0/2724101324 shutdown_connections 2024-09-13T18:32:23.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.591+0000 7ff68384b740 1 --2- 172.21.15.114:0/2724101324 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x56316b2fe250 0x56316b3006b0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.591+0000 7ff68384b740 1 --2- 172.21.15.114:0/2724101324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56316b2dd950 0x56316b2da450 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.591+0000 7ff68384b740 1 --2- 172.21.15.114:0/2724101324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56316b2d30b0 0x56316b2dd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.591+0000 7ff68384b740 1 --2- 172.21.15.114:0/2724101324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56316b2d2790 0x56316b2dcec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.591+0000 7ff68384b740 1 -- 172.21.15.114:0/2724101324 >> 172.21.15.114:0/2724101324 conn(0x56316b2ced00 msgr2=0x56316b2d4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:23.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.595+0000 7ff661ffb640 0 client.27755 ms_handle_reset on v2:172.21.15.134:6836/891692210 2024-09-13T18:32:23.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.598+0000 7ff68384b740 1 -- 172.21.15.114:0/2724101324 shutdown_connections 2024-09-13T18:32:23.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.599+0000 7ff68384b740 1 -- 172.21.15.114:0/2724101324 wait complete. 2024-09-13T18:32:23.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.602+0000 7ff68384b740 20 client.27755 trim_cache size 0 max 16384 2024-09-13T18:32:23.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.626+0000 7ff68384b740 1 Processor -- start 2024-09-13T18:32:23.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.631+0000 7ff68384b740 1 -- start start 2024-09-13T18:32:23.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.632+0000 7ff68384b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56316b2d2790 0x56316b300260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:23.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.632+0000 7ff68384b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56316b2d30b0 0x56316b305050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:23.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.632+0000 7ff68384b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56316b2dd950 0x56316b2e4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:23.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.632+0000 7ff68384b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56316b2db600 con 0x56316b2d30b0 2024-09-13T18:32:23.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.632+0000 7ff68384b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56316b2d2ea0 con 0x56316b2d2790 2024-09-13T18:32:23.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.632+0000 7ff68384b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56316b2ca440 con 0x56316b2dd950 2024-09-13T18:32:23.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.633+0000 7ff67ad9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56316b2dd950 0x56316b2e4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:23.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.633+0000 7ff67ad9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56316b2dd950 0x56316b2e4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:33518/0 (socket says 172.21.15.114:33518) 2024-09-13T18:32:23.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.633+0000 7ff67ad9d640 1 -- 172.21.15.114:0/3359179101 learned_addr learned my addr 172.21.15.114:0/3359179101 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:23.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.633+0000 7ff67ad9d640 1 -- 172.21.15.114:0/3359179101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56316b2d2790 msgr2=0x56316b300260 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:32:23.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.633+0000 7ff67ad9d640 1 --2- 172.21.15.114:0/3359179101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56316b2d2790 0x56316b300260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.633+0000 7ff67ad9d640 1 -- 172.21.15.114:0/3359179101 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56316b2d30b0 msgr2=0x56316b305050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:23.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.633+0000 7ff67ad9d640 1 --2- 172.21.15.114:0/3359179101 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56316b2d30b0 0x56316b305050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.633+0000 7ff67ad9d640 1 -- 172.21.15.114:0/3359179101 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56316b1d6770 con 0x56316b2dd950 2024-09-13T18:32:23.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.634+0000 7ff67ad9d640 1 --2- 172.21.15.114:0/3359179101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56316b2dd950 0x56316b2e4620 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff67404ca70 tx=0x7ff674005fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70f0926940780b17 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:23.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.635+0000 7ff6697fa640 1 -- 172.21.15.114:0/3359179101 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff674009b20 con 0x56316b2dd950 2024-09-13T18:32:23.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.636+0000 7ff6697fa640 1 -- 172.21.15.114:0/3359179101 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff674059070 con 0x56316b2dd950 2024-09-13T18:32:23.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.637+0000 7ff68384b740 1 -- 172.21.15.114:0/3359179101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56316b2dd950 msgr2=0x56316b2e4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:23.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.637+0000 7ff68384b740 1 --2- 172.21.15.114:0/3359179101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56316b2dd950 0x56316b2e4620 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff67404ca70 tx=0x7ff674005fb0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.637+0000 7ff68384b740 1 -- 172.21.15.114:0/3359179101 shutdown_connections 2024-09-13T18:32:23.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.637+0000 7ff68384b740 1 --2- 172.21.15.114:0/3359179101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56316b2dd950 0x56316b2e4620 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.637+0000 7ff68384b740 1 --2- 172.21.15.114:0/3359179101 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56316b2d30b0 0x56316b305050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.637+0000 7ff68384b740 1 --2- 172.21.15.114:0/3359179101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56316b2d2790 0x56316b300260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.637+0000 7ff68384b740 1 -- 172.21.15.114:0/3359179101 >> 172.21.15.114:0/3359179101 conn(0x56316b2ced00 msgr2=0x56316b2d3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:23.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.643+0000 7ff68384b740 1 -- 172.21.15.114:0/3359179101 shutdown_connections 2024-09-13T18:32:23.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.651+0000 7ff68384b740 1 -- 172.21.15.114:0/3359179101 wait complete. 2024-09-13T18:32:23.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.651+0000 7ff68384b740 1 -- start start 2024-09-13T18:32:23.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.651+0000 7ff68384b740 1 Processor -- start 2024-09-13T18:32:23.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.651+0000 7ff68384b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56316b2d2790 0x56316b2e4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:23.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.652+0000 7ff68384b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56316b2d30b0 0x56316b2e2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:23.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.652+0000 7ff68384b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56316b2dd950 0x56316b2e3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:23.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.652+0000 7ff68384b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56316b2e36b0 con 0x56316b2dd950 2024-09-13T18:32:23.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.652+0000 7ff68384b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56316b2e3820 con 0x56316b2d2790 2024-09-13T18:32:23.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.652+0000 7ff68384b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56316b2e39e0 con 0x56316b2d30b0 2024-09-13T18:32:23.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.652+0000 7ff68384b740 10 client.0 _ll_register_callbacks cb 0x7ff66c00a730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:32:23.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.652+0000 7ff67a59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56316b2d2790 0x56316b2e4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:23.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.652+0000 7ff67a59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56316b2d2790 0x56316b2e4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:54584/0 (socket says 172.21.15.114:54584) 2024-09-13T18:32:23.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.652+0000 7ff67a59c640 1 -- 172.21.15.114:0/882006805 learned_addr learned my addr 172.21.15.114:0/882006805 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:23.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.652+0000 7ff67ad9d640 1 --2- 172.21.15.114:0/882006805 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56316b2dd950 0x56316b2e3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:23.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.653+0000 7ff67a59c640 1 -- 172.21.15.114:0/882006805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56316b2d30b0 msgr2=0x56316b2e2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:23.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.653+0000 7ff67a59c640 1 --2- 172.21.15.114:0/882006805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56316b2d30b0 0x56316b2e2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.653+0000 7ff67a59c640 1 -- 172.21.15.114:0/882006805 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56316b2dd950 msgr2=0x56316b2e3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:23.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.653+0000 7ff67a59c640 1 --2- 172.21.15.114:0/882006805 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56316b2dd950 0x56316b2e3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.653+0000 7ff67a59c640 1 -- 172.21.15.114:0/882006805 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56316b2343c0 con 0x56316b2d2790 2024-09-13T18:32:23.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.655+0000 7ff67a59c640 1 --2- 172.21.15.114:0/882006805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56316b2d2790 0x56316b2e4790 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ff66c01f040 tx=0x7ff66c00a860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1282a2b2b304b3bc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:23.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.655+0000 7ff661ffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:32:23.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.657+0000 7ff661ffb640 1 -- 172.21.15.114:0/882006805 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66c02bea0 con 0x56316b2d2790 2024-09-13T18:32:23.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.657+0000 7ff661ffb640 1 -- 172.21.15.114:0/882006805 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ff66c01ca80 con 0x56316b2d2790 2024-09-13T18:32:23.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.659+0000 7ff68384b740 1 -- 172.21.15.114:0/882006805 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff66c01c440 con 0x56316b2d2790 2024-09-13T18:32:23.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.659+0000 7ff661ffb640 1 -- 172.21.15.114:0/882006805 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff66c005050 con 0x56316b2d2790 2024-09-13T18:32:23.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.659+0000 7ff68384b740 10 client.27758 fetch_fsmap learned FSMap version 34 2024-09-13T18:32:23.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.659+0000 7ff68384b740 1 -- 172.21.15.114:0/882006805 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x56316b1fbe80 con 0x56316b2d2790 2024-09-13T18:32:23.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.661+0000 7ff661ffb640 1 -- 172.21.15.114:0/882006805 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff66c01c440 con 0x56316b2d2790 2024-09-13T18:32:23.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.661+0000 7ff661ffb640 1 -- 172.21.15.114:0/882006805 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff66c02b750 con 0x56316b2d2790 2024-09-13T18:32:23.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.661+0000 7ff68384b740 10 client.27758 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:32:23.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.661+0000 7ff68384b740 10 client.27758 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-13T18:32:23.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.661+0000 7ff68384b740 20 client.27758 populate_metadata read hostname 'smithi114' 2024-09-13T18:32:23.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.661+0000 7ff68384b740 1 --2- 172.21.15.114:0/882006805 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x56316b35e4f0 0x56316b360910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:23.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.661+0000 7ff68384b740 4 client.27758 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:32:23.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.661+0000 7ff68384b740 1 -- 172.21.15.114:0/882006805 --> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56316af294e0 con 0x56316b35e4f0 2024-09-13T18:32:23.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.662+0000 7ff679d9b640 1 --2- 172.21.15.114:0/882006805 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x56316b35e4f0 0x56316b360910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:23.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.668+0000 7ff679d9b640 1 --2- 172.21.15.114:0/882006805 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x56316b35e4f0 0x56316b360910 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7b283fadb23a38df server_cookie=a7a8b6eacc619873 in_seq=0 out_seq=0 2024-09-13T18:32:23.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.668+0000 7ff661ffb640 10 client.27758 ms_handle_connect on v2:172.21.15.134:6836/891692210 2024-09-13T18:32:23.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.669+0000 7ff661ffb640 1 -- 172.21.15.114:0/882006805 <== mds.1 v2:172.21.15.134:6836/891692210 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56316af294e0 con 0x56316b35e4f0 2024-09-13T18:32:23.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.669+0000 7ff661ffb640 10 client.27758 handle_command_reply: tid=0 2024-09-13T18:32:23.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.669+0000 7ff68384b740 1 client.27758 shutdown 2024-09-13T18:32:23.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.672+0000 7ff68384b740 1 -- 172.21.15.114:0/882006805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56316b2d2790 msgr2=0x56316b2e4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:23.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.672+0000 7ff68384b740 1 --2- 172.21.15.114:0/882006805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56316b2d2790 0x56316b2e4790 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ff66c01f040 tx=0x7ff66c00a860 comp rx=0 tx=0).stop 2024-09-13T18:32:23.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.679+0000 7ff68384b740 1 -- 172.21.15.114:0/882006805 shutdown_connections 2024-09-13T18:32:23.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.679+0000 7ff68384b740 1 --2- 172.21.15.114:0/882006805 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x56316b35e4f0 0x56316b360910 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.679+0000 7ff68384b740 1 --2- 172.21.15.114:0/882006805 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56316b2dd950 0x56316b2e3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.679+0000 7ff68384b740 1 --2- 172.21.15.114:0/882006805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56316b2d30b0 0x56316b2e2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.679+0000 7ff68384b740 1 --2- 172.21.15.114:0/882006805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56316b2d2790 0x56316b2e4790 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.679+0000 7ff68384b740 1 -- 172.21.15.114:0/882006805 >> 172.21.15.114:0/882006805 conn(0x56316b2ced00 msgr2=0x56316b300260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:23.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.679+0000 7ff661ffb640 0 client.27758 ms_handle_reset on v2:172.21.15.134:6836/891692210 2024-09-13T18:32:23.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.679+0000 7ff68384b740 1 -- 172.21.15.114:0/882006805 shutdown_connections 2024-09-13T18:32:23.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.679+0000 7ff68384b740 1 -- 172.21.15.114:0/882006805 wait complete. 2024-09-13T18:32:23.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.684+0000 7ff68384b740 20 client.27758 trim_cache size 0 max 16384 2024-09-13T18:32:23.689 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:32:23.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.687+0000 7ff680b25640 1 -- 172.21.15.114:0/275556856 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff6640a1280 msgr2=0x7ff6640a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:23.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.687+0000 7ff680b25640 1 --2- 172.21.15.114:0/275556856 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff6640a1280 0x7ff6640a3740 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff674032000 tx=0x7ff6740095c0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.687+0000 7ff680b25640 1 -- 172.21.15.114:0/275556856 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff67c101860 msgr2=0x7ff67c196630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:23.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.687+0000 7ff680b25640 1 --2- 172.21.15.114:0/275556856 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff67c101860 0x7ff67c196630 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7ff67000e9f0 tx=0x7ff67000edf0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.695+0000 7ff680b25640 1 -- 172.21.15.114:0/275556856 shutdown_connections 2024-09-13T18:32:23.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.695+0000 7ff680b25640 1 --2- 172.21.15.114:0/275556856 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff6640a1280 0x7ff6640a3740 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.695+0000 7ff680b25640 1 --2- 172.21.15.114:0/275556856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff67c10a310 0x7ff67c19aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.695+0000 7ff680b25640 1 --2- 172.21.15.114:0/275556856 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff67c101860 0x7ff67c196630 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.695+0000 7ff680b25640 1 --2- 172.21.15.114:0/275556856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff67c100f40 0x7ff67c1960f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:23.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.695+0000 7ff680b25640 1 -- 172.21.15.114:0/275556856 >> 172.21.15.114:0/275556856 conn(0x7ff67c075c60 msgr2=0x7ff67c0fb3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:23.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.697+0000 7ff680b25640 1 -- 172.21.15.114:0/275556856 shutdown_connections 2024-09-13T18:32:23.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.697+0000 7ff680b25640 1 -- 172.21.15.114:0/275556856 wait complete. 2024-09-13T18:32:23.744 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-09-13T18:32:24.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.999+0000 7f934c9a9640 1 -- 172.21.15.114:0/841639894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9348107890 msgr2=0x7f9348107cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:24.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:23.999+0000 7f934c9a9640 1 --2- 172.21.15.114:0/841639894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9348107890 0x7f9348107cd0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f933c009900 tx=0x7f933c02a400 comp rx=0 tx=0).stop 2024-09-13T18:32:24.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.007+0000 7f934c9a9640 1 -- 172.21.15.114:0/841639894 shutdown_connections 2024-09-13T18:32:24.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.007+0000 7f934c9a9640 1 --2- 172.21.15.114:0/841639894 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9348119400 0x7f934811b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:24.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.007+0000 7f934c9a9640 1 --2- 172.21.15.114:0/841639894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9348107890 0x7f9348107cd0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:24.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.007+0000 7f934c9a9640 1 --2- 172.21.15.114:0/841639894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93481074b0 0x7f934810e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:24.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.007+0000 7f934c9a9640 1 -- 172.21.15.114:0/841639894 >> 172.21.15.114:0/841639894 conn(0x7f934806af40 msgr2=0x7f934806b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:24.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.007+0000 7f934c9a9640 1 -- 172.21.15.114:0/841639894 shutdown_connections 2024-09-13T18:32:24.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.011+0000 7f934c9a9640 1 -- 172.21.15.114:0/841639894 wait complete. 2024-09-13T18:32:24.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.011+0000 7f934c9a9640 1 Processor -- start 2024-09-13T18:32:24.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.012+0000 7f934c9a9640 1 -- start start 2024-09-13T18:32:24.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.012+0000 7f934c9a9640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93481074b0 0x7f9348116880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:24.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.012+0000 7f934c9a9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9348107890 0x7f9348116dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:24.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.012+0000 7f934c9a9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9348119400 0x7f934810f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:24.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.012+0000 7f934c9a9640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9348110110 con 0x7f93481074b0 2024-09-13T18:32:24.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.012+0000 7f934c9a9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9348110280 con 0x7f9348119400 2024-09-13T18:32:24.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.012+0000 7f934c9a9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9348110440 con 0x7f9348107890 2024-09-13T18:32:24.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.013+0000 7f934659c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9348107890 0x7f9348116dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:24.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.013+0000 7f934659c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9348107890 0x7f9348116dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:33548/0 (socket says 172.21.15.114:33548) 2024-09-13T18:32:24.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.013+0000 7f934659c640 1 -- 172.21.15.114:0/3649839811 learned_addr learned my addr 172.21.15.114:0/3649839811 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:24.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.014+0000 7f934659c640 1 -- 172.21.15.114:0/3649839811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9348119400 msgr2=0x7f934810f980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:32:24.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.014+0000 7f934659c640 1 --2- 172.21.15.114:0/3649839811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9348119400 0x7f934810f980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:24.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.014+0000 7f934659c640 1 -- 172.21.15.114:0/3649839811 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93481074b0 msgr2=0x7f9348116880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:24.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.014+0000 7f934659c640 1 --2- 172.21.15.114:0/3649839811 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93481074b0 0x7f9348116880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:24.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.014+0000 7f934659c640 1 -- 172.21.15.114:0/3649839811 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9338009490 con 0x7f9348107890 2024-09-13T18:32:24.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.014+0000 7f934659c640 1 --2- 172.21.15.114:0/3649839811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9348107890 0x7f9348116dc0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f933c02a9f0 tx=0x7f933c0055a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:24.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.015+0000 7f932ffff640 1 -- 172.21.15.114:0/3649839811 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933c004ad0 con 0x7f9348107890 2024-09-13T18:32:24.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.015+0000 7f934c9a9640 1 -- 172.21.15.114:0/3649839811 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f933c009560 con 0x7f9348107890 2024-09-13T18:32:24.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.015+0000 7f934c9a9640 1 -- 172.21.15.114:0/3649839811 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9348114580 con 0x7f9348107890 2024-09-13T18:32:24.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.015+0000 7f932ffff640 1 -- 172.21.15.114:0/3649839811 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f933c004c30 con 0x7f9348107890 2024-09-13T18:32:24.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.015+0000 7f932ffff640 1 -- 172.21.15.114:0/3649839811 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933c03c640 con 0x7f9348107890 2024-09-13T18:32:24.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.021+0000 7f932ffff640 1 -- 172.21.15.114:0/3649839811 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f933c039040 con 0x7f9348107890 2024-09-13T18:32:24.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.022+0000 7f932ffff640 1 --2- 172.21.15.114:0/3649839811 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f93300a1410 0x7f93300a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:24.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.022+0000 7f932ffff640 1 -- 172.21.15.114:0/3649839811 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f933c0ef060 con 0x7f9348107890 2024-09-13T18:32:24.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.035+0000 7f9346d9d640 1 --2- 172.21.15.114:0/3649839811 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f93300a1410 0x7f93300a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:24.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.044+0000 7f934fecc740 1 Processor -- start 2024-09-13T18:32:24.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.045+0000 7f934fecc740 1 -- start start 2024-09-13T18:32:24.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.045+0000 7f934fecc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dea556b970 0x55dea556bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:24.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.045+0000 7f934fecc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dea556c290 0x55dea55765f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:24.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.045+0000 7f934fecc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dea5576b30 0x55dea5578f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:24.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.045+0000 7f934fecc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dea54d5cc0 con 0x55dea556c290 2024-09-13T18:32:24.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.045+0000 7f934fecc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dea54d3450 con 0x55dea556b970 2024-09-13T18:32:24.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.045+0000 7f934fecc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dea54ced40 con 0x55dea5576b30 2024-09-13T18:32:24.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.045+0000 7f934659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dea556c290 0x55dea55765f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:24.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.046+0000 7f934659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dea556c290 0x55dea55765f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34942/0 (socket says 172.21.15.114:34942) 2024-09-13T18:32:24.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.046+0000 7f934659c640 1 -- 172.21.15.114:0/3115721571 learned_addr learned my addr 172.21.15.114:0/3115721571 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:24.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.046+0000 7f934759e640 1 --2- 172.21.15.114:0/3115721571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dea5576b30 0x55dea5578f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:24.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.046+0000 7f934659c640 1 -- 172.21.15.114:0/3115721571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dea5576b30 msgr2=0x55dea5578f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:24.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.046+0000 7f934659c640 1 --2- 172.21.15.114:0/3115721571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dea5576b30 0x55dea5578f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:24.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.046+0000 7f934659c640 1 -- 172.21.15.114:0/3115721571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dea556b970 msgr2=0x55dea556bd50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:32:24.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.046+0000 7f934659c640 1 --2- 172.21.15.114:0/3115721571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dea556b970 0x55dea556bd50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:24.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.046+0000 7f934659c640 1 -- 172.21.15.114:0/3115721571 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dea5496620 con 0x55dea556c290 2024-09-13T18:32:24.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.046+0000 7f934659c640 1 --2- 172.21.15.114:0/3115721571 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dea556c290 0x55dea55765f0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f933c03dd90 tx=0x7f933c0bb6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5e6bc0ee6f1753c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:24.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.047+0000 7f932d7fa640 1 -- 172.21.15.114:0/3115721571 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933c004e60 con 0x55dea556c290 2024-09-13T18:32:24.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.047+0000 7f932d7fa640 1 -- 172.21.15.114:0/3115721571 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f933c0bbd70 con 0x55dea556c290 2024-09-13T18:32:24.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.048+0000 7f934fecc740 1 -- 172.21.15.114:0/3115721571 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dea556c290 msgr2=0x55dea55765f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:24.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.048+0000 7f934fecc740 1 --2- 172.21.15.114:0/3115721571 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dea556c290 0x55dea55765f0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f933c03dd90 tx=0x7f933c0bb6d0 comp rx=0 tx=0).stop 2024-09-13T18:32:24.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.048+0000 7f934fecc740 1 -- 172.21.15.114:0/3115721571 shutdown_connections 2024-09-13T18:32:24.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.048+0000 7f934fecc740 1 --2- 172.21.15.114:0/3115721571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dea5576b30 0x55dea5578f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:24.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.048+0000 7f934fecc740 1 --2- 172.21.15.114:0/3115721571 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dea556c290 0x55dea55765f0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:24.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.048+0000 7f934fecc740 1 --2- 172.21.15.114:0/3115721571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dea556b970 0x55dea556bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:24.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.048+0000 7f934fecc740 1 -- 172.21.15.114:0/3115721571 >> 172.21.15.114:0/3115721571 conn(0x55dea5567ee0 msgr2=0x55dea55682c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:24.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.051+0000 7f9346d9d640 1 --2- 172.21.15.114:0/3649839811 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f93300a1410 0x7f93300a38d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9340005fd0 tx=0x7f93400071c0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:24.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.052+0000 7f934fecc740 1 -- 172.21.15.114:0/3115721571 shutdown_connections 2024-09-13T18:32:24.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.054+0000 7f934fecc740 1 -- 172.21.15.114:0/3115721571 wait complete. 2024-09-13T18:32:24.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.055+0000 7f934fecc740 1 -- start start 2024-09-13T18:32:24.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.055+0000 7f934fecc740 1 Processor -- start 2024-09-13T18:32:24.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.055+0000 7f934fecc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dea556b970 0x55dea557b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:24.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.055+0000 7f934fecc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dea556c290 0x55dea557bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:24.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.055+0000 7f934fecc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dea5576b30 0x55dea5573300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:24.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.055+0000 7f934fecc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dea548d180 con 0x55dea556b970 2024-09-13T18:32:24.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.055+0000 7f934fecc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dea51a7430 con 0x55dea5576b30 2024-09-13T18:32:24.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.055+0000 7f934fecc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dea54525e0 con 0x55dea556c290 2024-09-13T18:32:24.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.055+0000 7f934fecc740 10 client.0 _ll_register_callbacks cb 0x55dea54beaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:32:24.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.056+0000 7f934759e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dea5576b30 0x55dea5573300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:24.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.056+0000 7f934759e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dea5576b30 0x55dea5573300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:54602/0 (socket says 172.21.15.114:54602) 2024-09-13T18:32:24.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.056+0000 7f934759e640 1 -- 172.21.15.114:0/3131474036 learned_addr learned my addr 172.21.15.114:0/3131474036 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:24.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.057+0000 7f934759e640 1 -- 172.21.15.114:0/3131474036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dea556c290 msgr2=0x55dea557bee0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:32:24.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.057+0000 7f934759e640 1 --2- 172.21.15.114:0/3131474036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dea556c290 0x55dea557bee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:24.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.057+0000 7f934759e640 1 -- 172.21.15.114:0/3131474036 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dea556b970 msgr2=0x55dea557b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:24.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.057+0000 7f934759e640 1 --2- 172.21.15.114:0/3131474036 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dea556b970 0x55dea557b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:24.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.057+0000 7f934759e640 1 -- 172.21.15.114:0/3131474036 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9348114580 con 0x55dea5576b30 2024-09-13T18:32:24.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.058+0000 7f934759e640 1 --2- 172.21.15.114:0/3131474036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dea5576b30 0x55dea5573300 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f9338011660 tx=0x7f9338011b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32dfa646a6bcdb5c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:24.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.058+0000 7f932aff5640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:32:24.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.066+0000 7f932aff5640 1 -- 172.21.15.114:0/3131474036 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9338004250 con 0x55dea5576b30 2024-09-13T18:32:24.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.066+0000 7f934fecc740 1 -- 172.21.15.114:0/3131474036 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dea54f8b80 con 0x55dea5576b30 2024-09-13T18:32:24.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.066+0000 7f932aff5640 1 -- 172.21.15.114:0/3131474036 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f93380043b0 con 0x55dea5576b30 2024-09-13T18:32:24.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.068+0000 7f932aff5640 1 -- 172.21.15.114:0/3131474036 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f933800f8c0 con 0x55dea5576b30 2024-09-13T18:32:24.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.069+0000 7f934fecc740 10 client.27761 fetch_fsmap learned FSMap version 34 2024-09-13T18:32:24.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.069+0000 7f934fecc740 1 -- 172.21.15.114:0/3131474036 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55dea54cd3c0 con 0x55dea5576b30 2024-09-13T18:32:24.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.069+0000 7f932aff5640 1 -- 172.21.15.114:0/3131474036 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55dea54f8b80 con 0x55dea5576b30 2024-09-13T18:32:24.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.070+0000 7f932aff5640 1 -- 172.21.15.114:0/3131474036 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9338003aa0 con 0x55dea5576b30 2024-09-13T18:32:24.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.070+0000 7f934fecc740 10 client.27761 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:32:24.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.070+0000 7f934fecc740 10 client.27761 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:32:24.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.070+0000 7f934fecc740 20 client.27761 populate_metadata read hostname 'smithi114' 2024-09-13T18:32:24.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.070+0000 7f934fecc740 1 --2- 172.21.15.114:0/3131474036 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55dea55861e0 0x55dea5588600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:24.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.070+0000 7f934fecc740 4 client.27761 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:32:24.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.070+0000 7f934fecc740 1 -- 172.21.15.114:0/3131474036 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dea54c11f0 con 0x55dea55861e0 2024-09-13T18:32:24.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.072+0000 7f9346d9d640 1 --2- 172.21.15.114:0/3131474036 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55dea55861e0 0x55dea5588600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:24.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.075+0000 7f9346d9d640 1 --2- 172.21.15.114:0/3131474036 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55dea55861e0 0x55dea5588600 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28686b291d05a199 server_cookie=610ddbed258d854e in_seq=0 out_seq=0 2024-09-13T18:32:24.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.076+0000 7f932aff5640 10 client.27761 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:32:24.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.079+0000 7f932aff5640 1 -- 172.21.15.114:0/3131474036 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55dea54c11f0 con 0x55dea55861e0 2024-09-13T18:32:24.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.079+0000 7f932aff5640 10 client.27761 handle_command_reply: tid=0 2024-09-13T18:32:24.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.079+0000 7f934fecc740 1 client.27761 shutdown 2024-09-13T18:32:24.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.093+0000 7f934fecc740 1 -- 172.21.15.114:0/3131474036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dea5576b30 msgr2=0x55dea5573300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:24.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.093+0000 7f934fecc740 1 --2- 172.21.15.114:0/3131474036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dea5576b30 0x55dea5573300 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f9338011660 tx=0x7f9338011b20 comp rx=0 tx=0).stop 2024-09-13T18:32:24.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.097+0000 7f934fecc740 1 -- 172.21.15.114:0/3131474036 shutdown_connections 2024-09-13T18:32:24.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.097+0000 7f934fecc740 1 --2- 172.21.15.114:0/3131474036 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55dea55861e0 0x55dea5588600 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:24.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.097+0000 7f934fecc740 1 --2- 172.21.15.114:0/3131474036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dea5576b30 0x55dea5573300 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:24.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.097+0000 7f934fecc740 1 --2- 172.21.15.114:0/3131474036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dea556c290 0x55dea557bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:24.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.097+0000 7f934fecc740 1 --2- 172.21.15.114:0/3131474036 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dea556b970 0x55dea557b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:24.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.097+0000 7f934fecc740 1 -- 172.21.15.114:0/3131474036 >> 172.21.15.114:0/3131474036 conn(0x55dea5567ee0 msgr2=0x55dea5567230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:24.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.097+0000 7f932aff5640 0 client.27761 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:32:24.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.099+0000 7f934fecc740 1 -- 172.21.15.114:0/3131474036 shutdown_connections 2024-09-13T18:32:24.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.100+0000 7f934fecc740 1 -- 172.21.15.114:0/3131474036 wait complete. 2024-09-13T18:32:24.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.100+0000 7f934fecc740 20 client.27761 trim_cache size 0 max 16384 2024-09-13T18:32:24.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.138+0000 7f934fecc740 1 Processor -- start 2024-09-13T18:32:24.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.138+0000 7f934fecc740 1 -- start start 2024-09-13T18:32:24.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.139+0000 7f934fecc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dea556b970 0x55dea55732b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:24.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.139+0000 7f934fecc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dea556c290 0x55dea55682c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:24.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.139+0000 7f934fecc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dea5576b30 0x55dea556e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:24.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.139+0000 7f934fecc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dea5568800 con 0x55dea556c290 2024-09-13T18:32:24.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.139+0000 7f934fecc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dea5559bf0 con 0x55dea5576b30 2024-09-13T18:32:24.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.139+0000 7f934fecc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dea558d850 con 0x55dea556b970 2024-09-13T18:32:24.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.140+0000 7f934759e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dea5576b30 0x55dea556e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:24.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.140+0000 7f934659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dea556c290 0x55dea55682c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:24.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.140+0000 7f934659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dea556c290 0x55dea55682c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34960/0 (socket says 172.21.15.114:34960) 2024-09-13T18:32:24.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.140+0000 7f934659c640 1 -- 172.21.15.114:0/823622572 learned_addr learned my addr 172.21.15.114:0/823622572 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:24.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.141+0000 7f9346d9d640 1 --2- 172.21.15.114:0/823622572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dea556b970 0x55dea55732b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:24.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.143+0000 7f934759e640 1 -- 172.21.15.114:0/823622572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dea556b970 msgr2=0x55dea55732b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:24.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.143+0000 7f934759e640 1 --2- 172.21.15.114:0/823622572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dea556b970 0x55dea55732b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:24.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.143+0000 7f934759e640 1 -- 172.21.15.114:0/823622572 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dea556c290 msgr2=0x55dea55682c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:24.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.143+0000 7f934759e640 1 --2- 172.21.15.114:0/823622572 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dea556c290 0x55dea55682c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:24.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.143+0000 7f934759e640 1 -- 172.21.15.114:0/823622572 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dea546f770 con 0x55dea5576b30 2024-09-13T18:32:24.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.148+0000 7f934759e640 1 --2- 172.21.15.114:0/823622572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dea5576b30 0x55dea556e430 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f9338001eb0 tx=0x7f9338011fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b52370a67ee68a9f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:24.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.151+0000 7f932dffb640 1 -- 172.21.15.114:0/823622572 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93380125e0 con 0x55dea5576b30 2024-09-13T18:32:24.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.152+0000 7f932dffb640 1 -- 172.21.15.114:0/823622572 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f933800fd50 con 0x55dea5576b30 2024-09-13T18:32:24.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.152+0000 7f932dffb640 1 -- 172.21.15.114:0/823622572 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9338012a60 con 0x55dea5576b30 2024-09-13T18:32:24.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.154+0000 7f934fecc740 1 -- 172.21.15.114:0/823622572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dea5576b30 msgr2=0x55dea556e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:24.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.154+0000 7f934fecc740 1 --2- 172.21.15.114:0/823622572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dea5576b30 0x55dea556e430 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f9338001eb0 tx=0x7f9338011fd0 comp rx=0 tx=0).stop 2024-09-13T18:32:24.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.158+0000 7f934fecc740 1 -- 172.21.15.114:0/823622572 shutdown_connections 2024-09-13T18:32:24.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.158+0000 7f934fecc740 1 --2- 172.21.15.114:0/823622572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dea5576b30 0x55dea556e430 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:24.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.158+0000 7f934fecc740 1 --2- 172.21.15.114:0/823622572 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dea556c290 0x55dea55682c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:24.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.158+0000 7f934fecc740 1 --2- 172.21.15.114:0/823622572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dea556b970 0x55dea55732b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:24.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.158+0000 7f934fecc740 1 -- 172.21.15.114:0/823622572 >> 172.21.15.114:0/823622572 conn(0x55dea5567ee0 msgr2=0x55dea5572ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:24.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.164+0000 7f934fecc740 1 -- 172.21.15.114:0/823622572 shutdown_connections 2024-09-13T18:32:24.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.164+0000 7f934fecc740 1 -- 172.21.15.114:0/823622572 wait complete. 2024-09-13T18:32:24.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.164+0000 7f934fecc740 1 -- start start 2024-09-13T18:32:24.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.164+0000 7f934fecc740 1 Processor -- start 2024-09-13T18:32:24.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.165+0000 7f934fecc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dea556b970 0x55dea5581190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:24.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.165+0000 7f934fecc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dea556c290 0x55dea55816d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:24.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.165+0000 7f934fecc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dea5576b30 0x55dea55d67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:24.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.166+0000 7f934fecc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dea55d6d20 con 0x55dea556c290 2024-09-13T18:32:24.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.166+0000 7f9346d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dea556b970 0x55dea5581190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:24.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.166+0000 7f934fecc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dea55d6e90 con 0x55dea5576b30 2024-09-13T18:32:24.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.166+0000 7f934fecc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dea55d7050 con 0x55dea556b970 2024-09-13T18:32:24.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.166+0000 7f934fecc740 10 client.0 _ll_register_callbacks cb 0x7f93380096c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:32:24.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.166+0000 7f9346d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dea556b970 0x55dea5581190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:33574/0 (socket says 172.21.15.114:33574) 2024-09-13T18:32:24.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.166+0000 7f9346d9d640 1 -- 172.21.15.114:0/3471076574 learned_addr learned my addr 172.21.15.114:0/3471076574 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:24.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.166+0000 7f934759e640 1 --2- 172.21.15.114:0/3471076574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dea5576b30 0x55dea55d67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:24.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.166+0000 7f9346d9d640 1 -- 172.21.15.114:0/3471076574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dea5576b30 msgr2=0x55dea55d67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:24.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.166+0000 7f9346d9d640 1 --2- 172.21.15.114:0/3471076574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dea5576b30 0x55dea55d67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:24.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.166+0000 7f9346d9d640 1 -- 172.21.15.114:0/3471076574 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dea556c290 msgr2=0x55dea55816d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:24.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.166+0000 7f9346d9d640 1 --2- 172.21.15.114:0/3471076574 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dea556c290 0x55dea55816d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:24.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.166+0000 7f9346d9d640 1 -- 172.21.15.114:0/3471076574 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dea54cd3c0 con 0x55dea556b970 2024-09-13T18:32:24.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.169+0000 7f9346d9d640 1 --2- 172.21.15.114:0/3471076574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dea556b970 0x55dea5581190 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9340007de0 tx=0x7f934001da50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0aa7e302fdec276 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:24.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.169+0000 7f932aff5640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:32:24.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.169+0000 7f932aff5640 1 -- 172.21.15.114:0/3471076574 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9340021ea0 con 0x55dea556b970 2024-09-13T18:32:24.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.169+0000 7f932aff5640 1 -- 172.21.15.114:0/3471076574 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f9340016040 con 0x55dea556b970 2024-09-13T18:32:24.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.171+0000 7f934fecc740 1 -- 172.21.15.114:0/3471076574 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f933c0bbd70 con 0x55dea556b970 2024-09-13T18:32:24.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.174+0000 7f932aff5640 1 -- 172.21.15.114:0/3471076574 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9340004d20 con 0x55dea556b970 2024-09-13T18:32:24.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.176+0000 7f934fecc740 10 client.27700 fetch_fsmap learned FSMap version 34 2024-09-13T18:32:24.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.176+0000 7f934fecc740 1 -- 172.21.15.114:0/3471076574 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55dea5494e80 con 0x55dea556b970 2024-09-13T18:32:24.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.178+0000 7f932aff5640 1 -- 172.21.15.114:0/3471076574 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f933c0bbd70 con 0x55dea556b970 2024-09-13T18:32:24.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.178+0000 7f932aff5640 1 -- 172.21.15.114:0/3471076574 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f93400054c0 con 0x55dea556b970 2024-09-13T18:32:24.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.178+0000 7f934fecc740 10 client.27700 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:32:24.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.178+0000 7f934fecc740 10 client.27700 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:32:24.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.178+0000 7f934fecc740 20 client.27700 populate_metadata read hostname 'smithi114' 2024-09-13T18:32:24.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.178+0000 7f934fecc740 1 --2- 172.21.15.114:0/3471076574 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55dea55de830 0x55dea55e0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:24.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.178+0000 7f934fecc740 4 client.27700 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:32:24.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.178+0000 7f934fecc740 1 -- 172.21.15.114:0/3471076574 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dea51c24e0 con 0x55dea55de830 2024-09-13T18:32:24.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.180+0000 7f934659c640 1 --2- 172.21.15.114:0/3471076574 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55dea55de830 0x55dea55e0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:24.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.182+0000 7f934659c640 1 --2- 172.21.15.114:0/3471076574 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55dea55de830 0x55dea55e0c50 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ea515b0d5d44670 server_cookie=a60e81d1eca2b43a in_seq=0 out_seq=0 2024-09-13T18:32:24.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.183+0000 7f932aff5640 10 client.27700 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:32:24.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.183+0000 7f932aff5640 1 -- 172.21.15.114:0/3471076574 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dea51c24e0 con 0x55dea55de830 2024-09-13T18:32:24.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.183+0000 7f932aff5640 10 client.27700 handle_command_reply: tid=0 2024-09-13T18:32:24.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.184+0000 7f934fecc740 1 client.27700 shutdown 2024-09-13T18:32:24.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.188+0000 7f934fecc740 1 -- 172.21.15.114:0/3471076574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dea556b970 msgr2=0x55dea5581190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:24.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.188+0000 7f934fecc740 1 --2- 172.21.15.114:0/3471076574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dea556b970 0x55dea5581190 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9340007de0 tx=0x7f934001da50 comp rx=0 tx=0).stop 2024-09-13T18:32:24.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.189+0000 7f934fecc740 1 -- 172.21.15.114:0/3471076574 shutdown_connections 2024-09-13T18:32:24.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.189+0000 7f934fecc740 1 --2- 172.21.15.114:0/3471076574 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55dea55de830 0x55dea55e0c50 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:24.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.189+0000 7f934fecc740 1 --2- 172.21.15.114:0/3471076574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dea5576b30 0x55dea55d67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:24.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.189+0000 7f934fecc740 1 --2- 172.21.15.114:0/3471076574 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dea556c290 0x55dea55816d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:24.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.189+0000 7f934fecc740 1 --2- 172.21.15.114:0/3471076574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dea556b970 0x55dea5581190 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:24.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.189+0000 7f934fecc740 1 -- 172.21.15.114:0/3471076574 >> 172.21.15.114:0/3471076574 conn(0x55dea5567ee0 msgr2=0x55dea5566a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:24.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:23 smithi204 ceph-mon[34777]: pgmap v307: 129 pgs: 129 active+clean; 20 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 22 MiB/s wr, 510 op/s 2024-09-13T18:32:24.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:23 smithi204 ceph-mon[34777]: osdmap e102: 12 total, 12 up, 12 in 2024-09-13T18:32:24.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.193+0000 7f932aff5640 0 client.27700 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:32:24.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.194+0000 7f934fecc740 1 -- 172.21.15.114:0/3471076574 shutdown_connections 2024-09-13T18:32:24.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.194+0000 7f934fecc740 1 -- 172.21.15.114:0/3471076574 wait complete. 2024-09-13T18:32:24.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.194+0000 7f934fecc740 20 client.27700 trim_cache size 0 max 16384 2024-09-13T18:32:24.197 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:32:24.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.197+0000 7f934c9a9640 1 -- 172.21.15.114:0/3649839811 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f93300a1410 msgr2=0x7f93300a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:24.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.197+0000 7f934c9a9640 1 --2- 172.21.15.114:0/3649839811 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f93300a1410 0x7f93300a38d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9340005fd0 tx=0x7f93400071c0 comp rx=0 tx=0).stop 2024-09-13T18:32:24.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.197+0000 7f934c9a9640 1 -- 172.21.15.114:0/3649839811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9348107890 msgr2=0x7f9348116dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:24.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.197+0000 7f934c9a9640 1 --2- 172.21.15.114:0/3649839811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9348107890 0x7f9348116dc0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f933c02a9f0 tx=0x7f933c0055a0 comp rx=0 tx=0).stop 2024-09-13T18:32:24.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.200+0000 7f934c9a9640 1 -- 172.21.15.114:0/3649839811 shutdown_connections 2024-09-13T18:32:24.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.200+0000 7f934c9a9640 1 --2- 172.21.15.114:0/3649839811 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f93300a1410 0x7f93300a38d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:24.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.200+0000 7f934c9a9640 1 --2- 172.21.15.114:0/3649839811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9348119400 0x7f934810f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:24.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.200+0000 7f934c9a9640 1 --2- 172.21.15.114:0/3649839811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9348107890 0x7f9348116dc0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:24.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.200+0000 7f934c9a9640 1 --2- 172.21.15.114:0/3649839811 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93481074b0 0x7f9348116880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:24.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.200+0000 7f934c9a9640 1 -- 172.21.15.114:0/3649839811 >> 172.21.15.114:0/3649839811 conn(0x7f934806af40 msgr2=0x7f9348119a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:24.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.200+0000 7f934c9a9640 1 -- 172.21.15.114:0/3649839811 shutdown_connections 2024-09-13T18:32:24.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:24.200+0000 7f934c9a9640 1 -- 172.21.15.114:0/3649839811 wait complete. 2024-09-13T18:32:24.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:23 smithi114 ceph-mon[30897]: pgmap v307: 129 pgs: 129 active+clean; 20 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 22 MiB/s wr, 510 op/s 2024-09-13T18:32:24.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:23 smithi114 ceph-mon[30897]: osdmap e102: 12 total, 12 up, 12 in 2024-09-13T18:32:24.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:23 smithi134 ceph-mon[34283]: pgmap v307: 129 pgs: 129 active+clean; 20 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 22 MiB/s wr, 510 op/s 2024-09-13T18:32:24.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:23 smithi134 ceph-mon[34283]: osdmap e102: 12 total, 12 up, 12 in 2024-09-13T18:32:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.133+0000 7fd46b412640 1 -- 172.21.15.114:0/2777712301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4641074b0 msgr2=0x7fd46410e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.133+0000 7fd46b412640 1 --2- 172.21.15.114:0/2777712301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4641074b0 0x7fd46410e510 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd454009900 tx=0x7fd45402a440 comp rx=0 tx=0).stop 2024-09-13T18:32:25.146 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.141+0000 7fd46b412640 1 -- 172.21.15.114:0/2777712301 shutdown_connections 2024-09-13T18:32:25.146 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.141+0000 7fd46b412640 1 --2- 172.21.15.114:0/2777712301 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd464119400 0x7fd46411b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.146 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.141+0000 7fd46b412640 1 --2- 172.21.15.114:0/2777712301 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd464107890 0x7fd464107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.146 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.141+0000 7fd46b412640 1 --2- 172.21.15.114:0/2777712301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4641074b0 0x7fd46410e510 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.146 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.141+0000 7fd46b412640 1 -- 172.21.15.114:0/2777712301 >> 172.21.15.114:0/2777712301 conn(0x7fd46406af50 msgr2=0x7fd46406b360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:25.152 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.148+0000 7fd46b412640 1 -- 172.21.15.114:0/2777712301 shutdown_connections 2024-09-13T18:32:25.152 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.149+0000 7fd46b412640 1 -- 172.21.15.114:0/2777712301 wait complete. 2024-09-13T18:32:25.154 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.149+0000 7fd46b412640 1 Processor -- start 2024-09-13T18:32:25.154 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.149+0000 7fd46b412640 1 -- start start 2024-09-13T18:32:25.154 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.150+0000 7fd46b412640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4641074b0 0x7fd4641a5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:25.155 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.150+0000 7fd46b412640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd464107890 0x7fd4641a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:25.155 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.150+0000 7fd46b412640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd464119400 0x7fd4641a99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:25.155 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.150+0000 7fd46b412640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4641a9fc0 con 0x7fd464119400 2024-09-13T18:32:25.155 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.150+0000 7fd46b412640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4641aa130 con 0x7fd4641074b0 2024-09-13T18:32:25.155 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.150+0000 7fd46b412640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4641aa2f0 con 0x7fd464107890 2024-09-13T18:32:25.155 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.150+0000 7fd4691ae640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4641074b0 0x7fd4641a5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:25.155 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.150+0000 7fd4691ae640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4641074b0 0x7fd4641a5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:54646/0 (socket says 172.21.15.114:54646) 2024-09-13T18:32:25.155 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.150+0000 7fd4691ae640 1 -- 172.21.15.114:0/1253566969 learned_addr learned my addr 172.21.15.114:0/1253566969 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:25.155 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.150+0000 7fd4691ae640 1 -- 172.21.15.114:0/1253566969 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd464107890 msgr2=0x7fd4641a55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:25.155 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.150+0000 7fd4691ae640 1 --2- 172.21.15.114:0/1253566969 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd464107890 0x7fd4641a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.155 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.150+0000 7fd4691ae640 1 -- 172.21.15.114:0/1253566969 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd464119400 msgr2=0x7fd4641a99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:25.155 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.150+0000 7fd4691ae640 1 --2- 172.21.15.114:0/1253566969 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd464119400 0x7fd4641a99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.155 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.151+0000 7fd4691ae640 1 -- 172.21.15.114:0/1253566969 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd454009560 con 0x7fd4641074b0 2024-09-13T18:32:25.155 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.151+0000 7fd4691ae640 1 --2- 172.21.15.114:0/1253566969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4641074b0 0x7fd4641a5090 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd45402aa30 tx=0x7fd454003d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:25.161 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.154+0000 7fd4627fc640 1 -- 172.21.15.114:0/1253566969 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd454005440 con 0x7fd4641074b0 2024-09-13T18:32:25.161 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.154+0000 7fd4627fc640 1 -- 172.21.15.114:0/1253566969 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd4540055a0 con 0x7fd4641074b0 2024-09-13T18:32:25.161 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.154+0000 7fd4627fc640 1 -- 172.21.15.114:0/1253566969 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45403b9b0 con 0x7fd4641074b0 2024-09-13T18:32:25.162 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.156+0000 7fd46b412640 1 -- 172.21.15.114:0/1253566969 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4641aa570 con 0x7fd4641074b0 2024-09-13T18:32:25.170 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.161+0000 7fd46b412640 1 -- 172.21.15.114:0/1253566969 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd464078710 con 0x7fd4641074b0 2024-09-13T18:32:25.175 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.170+0000 7fd46e212740 1 Processor -- start 2024-09-13T18:32:25.178 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.170+0000 7fd46e212740 1 -- start start 2024-09-13T18:32:25.178 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.171+0000 7fd46e212740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5635e970b330 0x5635e970b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:25.178 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.171+0000 7fd46e212740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635e970bc50 0x5635e9715fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:25.178 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.171+0000 7fd46e212740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5635e97164f0 0x5635e97188e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:25.179 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.171+0000 7fd46e212740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e9676c30 con 0x5635e97164f0 2024-09-13T18:32:25.179 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.171+0000 7fd46e212740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e96743c0 con 0x5635e970bc50 2024-09-13T18:32:25.179 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.171+0000 7fd46e212740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e966fcb0 con 0x5635e970b330 2024-09-13T18:32:25.179 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.171+0000 7fd4699af640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5635e970b330 0x5635e970b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:25.179 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.171+0000 7fd4699af640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5635e970b330 0x5635e970b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:33602/0 (socket says 172.21.15.114:33602) 2024-09-13T18:32:25.179 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.171+0000 7fd4699af640 1 -- 172.21.15.114:0/3697804173 learned_addr learned my addr 172.21.15.114:0/3697804173 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:25.179 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.172+0000 7fd4699af640 1 -- 172.21.15.114:0/3697804173 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635e970bc50 msgr2=0x5635e9715fb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:32:25.179 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.172+0000 7fd4699af640 1 --2- 172.21.15.114:0/3697804173 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635e970bc50 0x5635e9715fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.180 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.172+0000 7fd4699af640 1 -- 172.21.15.114:0/3697804173 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5635e97164f0 msgr2=0x5635e97188e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:25.180 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.172+0000 7fd4699af640 1 --2- 172.21.15.114:0/3697804173 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5635e97164f0 0x5635e97188e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.180 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.172+0000 7fd4699af640 1 -- 172.21.15.114:0/3697804173 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635e9616910 con 0x5635e970b330 2024-09-13T18:32:25.180 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.173+0000 7fd4699af640 1 --2- 172.21.15.114:0/3697804173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5635e970b330 0x5635e970b710 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd45800a930 tx=0x7fd45800adf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=169c14486782e67 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:25.181 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.174+0000 7fd4627fc640 1 -- 172.21.15.114:0/1253566969 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fd45403bb10 con 0x7fd4641074b0 2024-09-13T18:32:25.181 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.175+0000 7fd4627fc640 1 --2- 172.21.15.114:0/1253566969 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fd44c0a13d0 0x7fd44c0a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:25.182 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.176+0000 7fd4627fc640 1 -- 172.21.15.114:0/1253566969 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fd4540ed3e0 con 0x7fd4641074b0 2024-09-13T18:32:25.182 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.178+0000 7fd44b7fe640 1 -- 172.21.15.114:0/3697804173 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45800cd80 con 0x5635e970b330 2024-09-13T18:32:25.182 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.178+0000 7fd44b7fe640 1 -- 172.21.15.114:0/3697804173 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd45800f8e0 con 0x5635e970b330 2024-09-13T18:32:25.186 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.179+0000 7fd46e212740 1 -- 172.21.15.114:0/3697804173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5635e970b330 msgr2=0x5635e970b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:25.186 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.179+0000 7fd46e212740 1 --2- 172.21.15.114:0/3697804173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5635e970b330 0x5635e970b710 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd45800a930 tx=0x7fd45800adf0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.186 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.179+0000 7fd4691ae640 1 --2- 172.21.15.114:0/1253566969 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fd44c0a13d0 0x7fd44c0a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:25.187 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.183+0000 7fd4691ae640 1 --2- 172.21.15.114:0/1253566969 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fd44c0a13d0 0x7fd44c0a3890 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd4640788a0 tx=0x7fd45403b460 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:25.192 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.188+0000 7fd46e212740 1 -- 172.21.15.114:0/3697804173 shutdown_connections 2024-09-13T18:32:25.192 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.188+0000 7fd46e212740 1 --2- 172.21.15.114:0/3697804173 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5635e97164f0 0x5635e97188e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.192 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.188+0000 7fd46e212740 1 --2- 172.21.15.114:0/3697804173 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635e970bc50 0x5635e9715fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.192 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.188+0000 7fd46e212740 1 --2- 172.21.15.114:0/3697804173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5635e970b330 0x5635e970b710 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.192 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.188+0000 7fd46e212740 1 -- 172.21.15.114:0/3697804173 >> 172.21.15.114:0/3697804173 conn(0x5635e97078c0 msgr2=0x5635e9707ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.202+0000 7fd46e212740 1 -- 172.21.15.114:0/3697804173 shutdown_connections 2024-09-13T18:32:25.214 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.208+0000 7fd46e212740 1 -- 172.21.15.114:0/3697804173 wait complete. 2024-09-13T18:32:25.214 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.208+0000 7fd46e212740 1 -- start start 2024-09-13T18:32:25.214 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.208+0000 7fd46e212740 1 Processor -- start 2024-09-13T18:32:25.214 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.209+0000 7fd46e212740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5635e970b330 0x5635e9715ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:25.214 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.209+0000 7fd46e212740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635e970bc50 0x5635e9713180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:25.214 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.209+0000 7fd46e212740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5635e97164f0 0x5635e9713770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:25.214 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.209+0000 7fd46e212740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e962de80 con 0x5635e97164f0 2024-09-13T18:32:25.214 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.209+0000 7fd46e212740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e962db20 con 0x5635e970bc50 2024-09-13T18:32:25.215 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.209+0000 7fd46e212740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e95f3260 con 0x5635e970b330 2024-09-13T18:32:25.215 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.209+0000 7fd46e212740 10 client.0 _ll_register_callbacks cb 0x5635e9628df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:32:25.215 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.209+0000 7fd4699af640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5635e97164f0 0x5635e9713770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:25.215 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.209+0000 7fd4699af640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5635e97164f0 0x5635e9713770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34988/0 (socket says 172.21.15.114:34988) 2024-09-13T18:32:25.215 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.209+0000 7fd4699af640 1 -- 172.21.15.114:0/3516761060 learned_addr learned my addr 172.21.15.114:0/3516761060 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:25.215 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.209+0000 7fd4699af640 1 --2- 172.21.15.114:0/3516761060 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5635e970b330 0x5635e9715ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:25.215 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.210+0000 7fd4699af640 1 -- 172.21.15.114:0/3516761060 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5635e970b330 msgr2=0x5635e9715ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:25.215 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.210+0000 7fd4699af640 1 --2- 172.21.15.114:0/3516761060 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5635e970b330 0x5635e9715ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.215 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.210+0000 7fd4699af640 1 -- 172.21.15.114:0/3516761060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635e970bc50 msgr2=0x5635e9713180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:32:25.215 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.210+0000 7fd4699af640 1 --2- 172.21.15.114:0/3516761060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635e970bc50 0x5635e9713180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.215 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.210+0000 7fd4699af640 1 -- 172.21.15.114:0/3516761060 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd45800a570 con 0x5635e97164f0 2024-09-13T18:32:25.215 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.210+0000 7fd4699af640 1 --2- 172.21.15.114:0/3516761060 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5635e97164f0 0x5635e9713770 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fd458005e30 tx=0x7fd45800a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec9dfd2cbe3cb935 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:25.228 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.224+0000 7fd448ff9640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:32:25.228 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.224+0000 7fd448ff9640 1 -- 172.21.15.114:0/3516761060 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45800fcd0 con 0x5635e97164f0 2024-09-13T18:32:25.228 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.224+0000 7fd448ff9640 1 -- 172.21.15.114:0/3516761060 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fd458018b10 con 0x5635e97164f0 2024-09-13T18:32:25.228 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.224+0000 7fd46e212740 1 -- 172.21.15.114:0/3516761060 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5635e9698680 con 0x5635e97164f0 2024-09-13T18:32:25.228 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.225+0000 7fd448ff9640 1 -- 172.21.15.114:0/3516761060 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd458018cd0 con 0x5635e97164f0 2024-09-13T18:32:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.225+0000 7fd46e212740 10 client.18279 fetch_fsmap learned FSMap version 34 2024-09-13T18:32:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.225+0000 7fd46e212740 1 -- 172.21.15.114:0/3516761060 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5635e966e330 con 0x5635e97164f0 2024-09-13T18:32:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.226+0000 7fd448ff9640 1 -- 172.21.15.114:0/3516761060 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd45802e070 con 0x5635e97164f0 2024-09-13T18:32:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.226+0000 7fd448ff9640 1 -- 172.21.15.114:0/3516761060 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd458016020 con 0x5635e97164f0 2024-09-13T18:32:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.226+0000 7fd46e212740 10 client.18279 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:32:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.226+0000 7fd46e212740 10 client.18279 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:32:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.226+0000 7fd46e212740 20 client.18279 populate_metadata read hostname 'smithi114' 2024-09-13T18:32:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.226+0000 7fd46e212740 1 --2- 172.21.15.114:0/3516761060 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5635e9725fd0 0x5635e97283f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:25.230 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.226+0000 7fd46e212740 4 client.18279 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:32:25.230 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.226+0000 7fd46e212740 1 -- 172.21.15.114:0/3516761060 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5635e96635d0 con 0x5635e9725fd0 2024-09-13T18:32:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.229+0000 7fd4689ad640 1 --2- 172.21.15.114:0/3516761060 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5635e9725fd0 0x5635e97283f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:25.234 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.231+0000 7fd4689ad640 1 --2- 172.21.15.114:0/3516761060 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5635e9725fd0 0x5635e97283f0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3169e966fb0de2c2 server_cookie=8efb00bd45f18d7c in_seq=0 out_seq=0 2024-09-13T18:32:25.234 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.231+0000 7fd448ff9640 10 client.18279 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:32:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.233+0000 7fd448ff9640 1 -- 172.21.15.114:0/3516761060 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5635e96635d0 con 0x5635e9725fd0 2024-09-13T18:32:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.233+0000 7fd448ff9640 10 client.18279 handle_command_reply: tid=0 2024-09-13T18:32:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.234+0000 7fd46e212740 1 client.18279 shutdown 2024-09-13T18:32:25.242 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-13T18:32:25.243 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-09-13T18:32:25.246 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.243+0000 7fd46e212740 1 -- 172.21.15.114:0/3516761060 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5635e97164f0 msgr2=0x5635e9713770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:25.246 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.243+0000 7fd46e212740 1 --2- 172.21.15.114:0/3516761060 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5635e97164f0 0x5635e9713770 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fd458005e30 tx=0x7fd45800a2c0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.251 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.247+0000 7fd46e212740 1 -- 172.21.15.114:0/3516761060 shutdown_connections 2024-09-13T18:32:25.251 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.247+0000 7fd46e212740 1 --2- 172.21.15.114:0/3516761060 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5635e9725fd0 0x5635e97283f0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.251 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.247+0000 7fd46e212740 1 --2- 172.21.15.114:0/3516761060 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5635e97164f0 0x5635e9713770 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.251 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.247+0000 7fd46e212740 1 --2- 172.21.15.114:0/3516761060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635e970bc50 0x5635e9713180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.251 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.247+0000 7fd46e212740 1 --2- 172.21.15.114:0/3516761060 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5635e970b330 0x5635e9715ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.251 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.247+0000 7fd46e212740 1 -- 172.21.15.114:0/3516761060 >> 172.21.15.114:0/3516761060 conn(0x5635e97078c0 msgr2=0x5635e9706b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.250+0000 7fd448ff9640 0 client.18279 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:32:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.250+0000 7fd46e212740 1 -- 172.21.15.114:0/3516761060 shutdown_connections 2024-09-13T18:32:25.257 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.254+0000 7fd46e212740 1 -- 172.21.15.114:0/3516761060 wait complete. 2024-09-13T18:32:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.256+0000 7fd46e212740 20 client.18279 trim_cache size 0 max 16384 2024-09-13T18:32:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.304+0000 7fd46e212740 1 Processor -- start 2024-09-13T18:32:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.308+0000 7fd46e212740 1 -- start start 2024-09-13T18:32:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.308+0000 7fd46e212740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5635e970b330 0x5635e9707300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.308+0000 7fd46e212740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5635e970bc50 0x5635e9711a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.308+0000 7fd46e212740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635e97164f0 0x5635e9711f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.308+0000 7fd46e212740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e9718fd0 con 0x5635e970bc50 2024-09-13T18:32:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.308+0000 7fd46e212740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e97124b0 con 0x5635e97164f0 2024-09-13T18:32:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.308+0000 7fd46e212740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e9722560 con 0x5635e970b330 2024-09-13T18:32:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.310+0000 7fd4699af640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5635e970b330 0x5635e9707300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.310+0000 7fd4699af640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635e97164f0 0x5635e9711f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.310+0000 7fd4699af640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5635e970b330 0x5635e9707300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:33614/0 (socket says 172.21.15.114:33614) 2024-09-13T18:32:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.310+0000 7fd4699af640 1 -- 172.21.15.114:0/3869390349 learned_addr learned my addr 172.21.15.114:0/3869390349 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.310+0000 7fd4689ad640 1 --2- 172.21.15.114:0/3869390349 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5635e970bc50 0x5635e9711a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.312+0000 7fd4689ad640 1 -- 172.21.15.114:0/3869390349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5635e970b330 msgr2=0x5635e9707300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.312+0000 7fd4689ad640 1 --2- 172.21.15.114:0/3869390349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5635e970b330 0x5635e9707300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.312+0000 7fd4689ad640 1 -- 172.21.15.114:0/3869390349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635e97164f0 msgr2=0x5635e9711f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.312+0000 7fd4689ad640 1 --2- 172.21.15.114:0/3869390349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635e97164f0 0x5635e9711f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.312+0000 7fd4689ad640 1 -- 172.21.15.114:0/3869390349 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635e9610300 con 0x5635e970bc50 2024-09-13T18:32:25.319 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.316+0000 7fd4689ad640 1 --2- 172.21.15.114:0/3869390349 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5635e970bc50 0x5635e9711a30 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x5635e97129f0 tx=0x7fd45c009230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbddd152903f2b5d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:25.326 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.323+0000 7fd44bfff640 1 -- 172.21.15.114:0/3869390349 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45c00a6b0 con 0x5635e970bc50 2024-09-13T18:32:25.326 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.323+0000 7fd44bfff640 1 -- 172.21.15.114:0/3869390349 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd45c016830 con 0x5635e970bc50 2024-09-13T18:32:25.326 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.323+0000 7fd44bfff640 1 -- 172.21.15.114:0/3869390349 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45c00c850 con 0x5635e970bc50 2024-09-13T18:32:25.330 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.326+0000 7fd46e212740 1 -- 172.21.15.114:0/3869390349 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5635e970bc50 msgr2=0x5635e9711a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:25.330 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.326+0000 7fd46e212740 1 --2- 172.21.15.114:0/3869390349 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5635e970bc50 0x5635e9711a30 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x5635e97129f0 tx=0x7fd45c009230 comp rx=0 tx=0).stop 2024-09-13T18:32:25.330 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.327+0000 7fd46e212740 1 -- 172.21.15.114:0/3869390349 shutdown_connections 2024-09-13T18:32:25.330 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.327+0000 7fd46e212740 1 --2- 172.21.15.114:0/3869390349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635e97164f0 0x5635e9711f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.330 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.327+0000 7fd46e212740 1 --2- 172.21.15.114:0/3869390349 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5635e970bc50 0x5635e9711a30 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.330 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.327+0000 7fd46e212740 1 --2- 172.21.15.114:0/3869390349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5635e970b330 0x5635e9707300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.330 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.327+0000 7fd46e212740 1 -- 172.21.15.114:0/3869390349 >> 172.21.15.114:0/3869390349 conn(0x5635e97078c0 msgr2=0x5635e970e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:25.337 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.334+0000 7fd46e212740 1 -- 172.21.15.114:0/3869390349 shutdown_connections 2024-09-13T18:32:25.338 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.334+0000 7fd46e212740 1 -- 172.21.15.114:0/3869390349 wait complete. 2024-09-13T18:32:25.338 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.335+0000 7fd46e212740 1 -- start start 2024-09-13T18:32:25.338 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.335+0000 7fd46e212740 1 Processor -- start 2024-09-13T18:32:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.335+0000 7fd46e212740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5635e970b330 0x5635e971a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.335+0000 7fd46e212740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635e970bc50 0x5635e971a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.335+0000 7fd46e212740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5635e97164f0 0x5635e977c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.335+0000 7fd46e212740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e971aed0 con 0x5635e97164f0 2024-09-13T18:32:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.335+0000 7fd46e212740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e977ca10 con 0x5635e970bc50 2024-09-13T18:32:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.335+0000 7fd46e212740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e978db60 con 0x5635e970b330 2024-09-13T18:32:25.340 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.336+0000 7fd46e212740 10 client.0 _ll_register_callbacks cb 0x7fd458005e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:32:25.340 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.336+0000 7fd4689ad640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635e970bc50 0x5635e971a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:25.340 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.336+0000 7fd4699af640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5635e970b330 0x5635e971a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:25.340 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.336+0000 7fd4699af640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5635e970b330 0x5635e971a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:33624/0 (socket says 172.21.15.114:33624) 2024-09-13T18:32:25.340 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.336+0000 7fd4699af640 1 -- 172.21.15.114:0/713127308 learned_addr learned my addr 172.21.15.114:0/713127308 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:25.343 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.340+0000 7fd4699af640 1 --2- 172.21.15.114:0/713127308 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5635e97164f0 0x5635e977c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:25.343 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.340+0000 7fd4689ad640 1 -- 172.21.15.114:0/713127308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5635e970b330 msgr2=0x5635e971a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:25.343 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.340+0000 7fd4689ad640 1 --2- 172.21.15.114:0/713127308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5635e970b330 0x5635e971a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.343 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.340+0000 7fd4689ad640 1 -- 172.21.15.114:0/713127308 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5635e97164f0 msgr2=0x5635e977c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:25.343 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.340+0000 7fd4689ad640 1 --2- 172.21.15.114:0/713127308 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5635e97164f0 0x5635e977c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.343 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.340+0000 7fd4689ad640 1 -- 172.21.15.114:0/713127308 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd45c00ab20 con 0x5635e970bc50 2024-09-13T18:32:25.343 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.340+0000 7fd4699af640 1 --2- 172.21.15.114:0/713127308 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5635e97164f0 0x5635e977c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:32:25.344 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.340+0000 7fd4689ad640 1 --2- 172.21.15.114:0/713127308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635e970bc50 0x5635e971a580 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x5635e9708920 tx=0x7fd45c00a7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=654cca8113d43c71 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:25.349 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.342+0000 7fd448ff9640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:32:25.349 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.342+0000 7fd448ff9640 1 -- 172.21.15.114:0/713127308 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45c0165a0 con 0x5635e970bc50 2024-09-13T18:32:25.349 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.342+0000 7fd46e212740 1 -- 172.21.15.114:0/713127308 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd45800f8e0 con 0x5635e970bc50 2024-09-13T18:32:25.349 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.342+0000 7fd448ff9640 1 -- 172.21.15.114:0/713127308 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fd45c012070 con 0x5635e970bc50 2024-09-13T18:32:25.353 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.349+0000 7fd448ff9640 1 -- 172.21.15.114:0/713127308 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd45c016e10 con 0x5635e970bc50 2024-09-13T18:32:25.361 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.355+0000 7fd46e212740 10 client.27779 fetch_fsmap learned FSMap version 34 2024-09-13T18:32:25.361 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.355+0000 7fd46e212740 1 -- 172.21.15.114:0/713127308 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5635e96358e0 con 0x5635e970bc50 2024-09-13T18:32:25.361 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.357+0000 7fd448ff9640 1 -- 172.21.15.114:0/713127308 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd45800f8e0 con 0x5635e970bc50 2024-09-13T18:32:25.361 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.357+0000 7fd448ff9640 1 -- 172.21.15.114:0/713127308 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd45c032640 con 0x5635e970bc50 2024-09-13T18:32:25.362 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.358+0000 7fd46e212740 10 client.27779 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:32:25.362 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.358+0000 7fd46e212740 10 client.27779 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:32:25.362 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.358+0000 7fd46e212740 20 client.27779 populate_metadata read hostname 'smithi114' 2024-09-13T18:32:25.362 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.358+0000 7fd46e212740 1 --2- 172.21.15.114:0/713127308 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5635e978f190 0x5635e97915b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:25.362 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.359+0000 7fd46e212740 4 client.27779 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T18:32:25.362 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.359+0000 7fd46e212740 1 -- 172.21.15.114:0/713127308 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5635e9363300 con 0x5635e978f190 2024-09-13T18:32:25.364 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.361+0000 7fd4699af640 1 --2- 172.21.15.114:0/713127308 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5635e978f190 0x5635e97915b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:25.369 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.364+0000 7fd4699af640 1 --2- 172.21.15.114:0/713127308 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5635e978f190 0x5635e97915b0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5970498a50414a0 server_cookie=eeb2aa376cc36186 in_seq=0 out_seq=0 2024-09-13T18:32:25.369 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.364+0000 7fd448ff9640 10 client.27779 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:32:25.396 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.393+0000 7fd448ff9640 1 -- 172.21.15.114:0/713127308 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5635e9363300 con 0x5635e978f190 2024-09-13T18:32:25.396 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.393+0000 7fd448ff9640 10 client.27779 handle_command_reply: tid=0 2024-09-13T18:32:25.396 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.393+0000 7fd46e212740 1 client.27779 shutdown 2024-09-13T18:32:25.402 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.399+0000 7fd46e212740 1 -- 172.21.15.114:0/713127308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635e970bc50 msgr2=0x5635e971a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:25.402 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.399+0000 7fd46e212740 1 --2- 172.21.15.114:0/713127308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635e970bc50 0x5635e971a580 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x5635e9708920 tx=0x7fd45c00a7f0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.402 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.399+0000 7fd46e212740 1 -- 172.21.15.114:0/713127308 shutdown_connections 2024-09-13T18:32:25.402 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.399+0000 7fd46e212740 1 --2- 172.21.15.114:0/713127308 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5635e978f190 0x5635e97915b0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.402 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.399+0000 7fd46e212740 1 --2- 172.21.15.114:0/713127308 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5635e97164f0 0x5635e977c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.402 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.399+0000 7fd46e212740 1 --2- 172.21.15.114:0/713127308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635e970bc50 0x5635e971a580 secure :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0x5635e9708920 tx=0x7fd45c00a7f0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.403 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.399+0000 7fd46e212740 1 --2- 172.21.15.114:0/713127308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5635e970b330 0x5635e971a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.403 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.399+0000 7fd46e212740 1 -- 172.21.15.114:0/713127308 >> 172.21.15.114:0/713127308 conn(0x5635e97078c0 msgr2=0x5635e97119f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:25.403 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.399+0000 7fd448ff9640 0 client.27779 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:32:25.405 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.402+0000 7fd46e212740 1 -- 172.21.15.114:0/713127308 shutdown_connections 2024-09-13T18:32:25.405 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.402+0000 7fd46e212740 1 -- 172.21.15.114:0/713127308 wait complete. 2024-09-13T18:32:25.412 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.409+0000 7fd46e212740 20 client.27779 trim_cache size 0 max 16384 2024-09-13T18:32:25.416 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-13T18:32:25.416 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-13T18:32:25.416 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-13T18:32:25.416 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-13T18:32:25.423 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.419+0000 7fd46b412640 1 -- 172.21.15.114:0/1253566969 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fd44c0a13d0 msgr2=0x7fd44c0a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:25.423 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.420+0000 7fd46b412640 1 --2- 172.21.15.114:0/1253566969 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fd44c0a13d0 0x7fd44c0a3890 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd4640788a0 tx=0x7fd45403b460 comp rx=0 tx=0).stop 2024-09-13T18:32:25.423 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.420+0000 7fd46b412640 1 -- 172.21.15.114:0/1253566969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4641074b0 msgr2=0x7fd4641a5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:25.423 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.420+0000 7fd46b412640 1 --2- 172.21.15.114:0/1253566969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4641074b0 0x7fd4641a5090 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd45402aa30 tx=0x7fd454003d50 comp rx=0 tx=0).stop 2024-09-13T18:32:25.423 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.420+0000 7fd46b412640 1 -- 172.21.15.114:0/1253566969 shutdown_connections 2024-09-13T18:32:25.423 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.420+0000 7fd46b412640 1 --2- 172.21.15.114:0/1253566969 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fd44c0a13d0 0x7fd44c0a3890 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.424 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.420+0000 7fd46b412640 1 --2- 172.21.15.114:0/1253566969 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd464119400 0x7fd4641a99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.430 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.420+0000 7fd46b412640 1 --2- 172.21.15.114:0/1253566969 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd464107890 0x7fd4641a55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.430 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.424+0000 7fd46b412640 1 --2- 172.21.15.114:0/1253566969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4641074b0 0x7fd4641a5090 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.430 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.424+0000 7fd46b412640 1 -- 172.21.15.114:0/1253566969 >> 172.21.15.114:0/1253566969 conn(0x7fd46406af50 msgr2=0x7fd464119f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:25.430 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.425+0000 7fd46b412640 1 -- 172.21.15.114:0/1253566969 shutdown_connections 2024-09-13T18:32:25.430 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:25.425+0000 7fd46b412640 1 -- 172.21.15.114:0/1253566969 wait complete. 2024-09-13T18:32:25.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.526+0000 7f91f4ea6640 1 -- 172.21.15.114:0/2867225343 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f91f0119400 msgr2=0x7f91f011b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:25.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.526+0000 7f91f4ea6640 1 --2- 172.21.15.114:0/2867225343 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f91f0119400 0x7f91f011b7f0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f91e001a910 tx=0x7f91e003b450 comp rx=0 tx=0).stop 2024-09-13T18:32:25.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.526+0000 7f91f4ea6640 1 -- 172.21.15.114:0/2867225343 shutdown_connections 2024-09-13T18:32:25.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.526+0000 7f91f4ea6640 1 --2- 172.21.15.114:0/2867225343 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f91f0119400 0x7f91f011b7f0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.526+0000 7f91f4ea6640 1 --2- 172.21.15.114:0/2867225343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91f0107890 0x7f91f0107d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.526+0000 7f91f4ea6640 1 --2- 172.21.15.114:0/2867225343 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91f01074b0 0x7f91f010e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.526+0000 7f91f4ea6640 1 -- 172.21.15.114:0/2867225343 >> 172.21.15.114:0/2867225343 conn(0x7f91f006aec0 msgr2=0x7f91f006b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:25.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.527+0000 7f91f4ea6640 1 -- 172.21.15.114:0/2867225343 shutdown_connections 2024-09-13T18:32:25.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.527+0000 7f91f4ea6640 1 -- 172.21.15.114:0/2867225343 wait complete. 2024-09-13T18:32:25.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.527+0000 7f91f4ea6640 1 Processor -- start 2024-09-13T18:32:25.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.528+0000 7f91f4ea6640 1 -- start start 2024-09-13T18:32:25.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.528+0000 7f91f4ea6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91f01074b0 0x7f91f01188c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:25.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.528+0000 7f91f4ea6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91f0107890 0x7f91f0118e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:25.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.528+0000 7f91f4ea6640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f91f0119400 0x7f91f0114a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:25.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.528+0000 7f91f4ea6640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f0115190 con 0x7f91f0119400 2024-09-13T18:32:25.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.528+0000 7f91f4ea6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f0115300 con 0x7f91f0107890 2024-09-13T18:32:25.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.528+0000 7f91f4ea6640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f01154c0 con 0x7f91f01074b0 2024-09-13T18:32:25.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.528+0000 7f91effff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f91f0119400 0x7f91f0114a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:25.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.528+0000 7f91effff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f91f0119400 0x7f91f0114a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35030/0 (socket says 172.21.15.114:35030) 2024-09-13T18:32:25.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.528+0000 7f91effff640 1 -- 172.21.15.114:0/29193977 learned_addr learned my addr 172.21.15.114:0/29193977 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:25.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.529+0000 7f91eeffd640 1 --2- 172.21.15.114:0/29193977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91f0107890 0x7f91f0118e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:25.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.529+0000 7f91effff640 1 -- 172.21.15.114:0/29193977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91f01074b0 msgr2=0x7f91f01188c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:32:25.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.529+0000 7f91effff640 1 --2- 172.21.15.114:0/29193977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91f01074b0 0x7f91f01188c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.529+0000 7f91effff640 1 -- 172.21.15.114:0/29193977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91f0107890 msgr2=0x7f91f0118e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:25.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.529+0000 7f91effff640 1 --2- 172.21.15.114:0/29193977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91f0107890 0x7f91f0118e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.529+0000 7f91effff640 1 -- 172.21.15.114:0/29193977 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e001a570 con 0x7f91f0119400 2024-09-13T18:32:25.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.529+0000 7f91effff640 1 --2- 172.21.15.114:0/29193977 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f91f0119400 0x7f91f0114a00 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f91e001a860 tx=0x7f91e00053c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:25.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.533+0000 7f91ecff9640 1 -- 172.21.15.114:0/29193977 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e0040710 con 0x7f91f0119400 2024-09-13T18:32:25.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.533+0000 7f91f4ea6640 1 -- 172.21.15.114:0/29193977 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91f0115740 con 0x7f91f0119400 2024-09-13T18:32:25.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.533+0000 7f91f4ea6640 1 -- 172.21.15.114:0/29193977 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91f01cb3f0 con 0x7f91f0119400 2024-09-13T18:32:25.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.533+0000 7f91ecff9640 1 -- 172.21.15.114:0/29193977 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f91e0040d00 con 0x7f91f0119400 2024-09-13T18:32:25.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.533+0000 7f91ecff9640 1 -- 172.21.15.114:0/29193977 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e000fca0 con 0x7f91f0119400 2024-09-13T18:32:25.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.539+0000 7f91f8e24740 1 Processor -- start 2024-09-13T18:32:25.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.539+0000 7f91f8e24740 1 -- start start 2024-09-13T18:32:25.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.539+0000 7f91f8e24740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55930303c9e0 0x55930303cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:25.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.539+0000 7f91f8e24740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55930303d300 0x559303047660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:25.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.539+0000 7f91f8e24740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559303047ba0 0x559303049f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:25.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.539+0000 7f91f8e24740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x559302fa7c60 con 0x559303047ba0 2024-09-13T18:32:25.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.539+0000 7f91f8e24740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559302fa53f0 con 0x55930303c9e0 2024-09-13T18:32:25.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.539+0000 7f91f8e24740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559302fa0ce0 con 0x55930303d300 2024-09-13T18:32:25.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.540+0000 7f91ef7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55930303d300 0x559303047660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:25.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.540+0000 7f91ef7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55930303d300 0x559303047660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:33630/0 (socket says 172.21.15.114:33630) 2024-09-13T18:32:25.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.540+0000 7f91ef7fe640 1 -- 172.21.15.114:0/330567143 learned_addr learned my addr 172.21.15.114:0/330567143 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:25.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.541+0000 7f91ef7fe640 1 -- 172.21.15.114:0/330567143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55930303c9e0 msgr2=0x55930303cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:25.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.541+0000 7f91ef7fe640 1 --2- 172.21.15.114:0/330567143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55930303c9e0 0x55930303cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.541+0000 7f91ef7fe640 1 -- 172.21.15.114:0/330567143 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559303047ba0 msgr2=0x559303049f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:25.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.541+0000 7f91ef7fe640 1 --2- 172.21.15.114:0/330567143 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559303047ba0 0x559303049f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.541+0000 7f91ef7fe640 1 -- 172.21.15.114:0/330567143 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559302f5c970 con 0x55930303d300 2024-09-13T18:32:25.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.542+0000 7f91ef7fe640 1 --2- 172.21.15.114:0/330567143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55930303d300 0x559303047660 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f91e4009770 tx=0x7f91e4009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb0e1d3d4b0ebfef server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:25.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.543+0000 7f91ecff9640 1 -- 172.21.15.114:0/29193977 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f91e0040870 con 0x7f91f0119400 2024-09-13T18:32:25.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.543+0000 7f91d5ffb640 1 -- 172.21.15.114:0/330567143 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e400bad0 con 0x55930303d300 2024-09-13T18:32:25.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.543+0000 7f91d5ffb640 1 -- 172.21.15.114:0/330567143 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f91e4009e80 con 0x55930303d300 2024-09-13T18:32:25.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.543+0000 7f91d5ffb640 1 -- 172.21.15.114:0/330567143 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e40103e0 con 0x55930303d300 2024-09-13T18:32:25.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.544+0000 7f91ecff9640 1 --2- 172.21.15.114:0/29193977 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f91d80a1410 0x7f91d80a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:25.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.544+0000 7f91ecff9640 1 -- 172.21.15.114:0/29193977 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f91e00ee130 con 0x7f91f0119400 2024-09-13T18:32:25.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.544+0000 7f91f8e24740 1 -- 172.21.15.114:0/330567143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55930303d300 msgr2=0x559303047660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:25.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.544+0000 7f91f8e24740 1 --2- 172.21.15.114:0/330567143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55930303d300 0x559303047660 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f91e4009770 tx=0x7f91e4009c30 comp rx=0 tx=0).stop 2024-09-13T18:32:25.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.545+0000 7f91f8e24740 1 -- 172.21.15.114:0/330567143 shutdown_connections 2024-09-13T18:32:25.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.545+0000 7f91f8e24740 1 --2- 172.21.15.114:0/330567143 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559303047ba0 0x559303049f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.545+0000 7f91f8e24740 1 --2- 172.21.15.114:0/330567143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55930303d300 0x559303047660 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.545+0000 7f91f8e24740 1 --2- 172.21.15.114:0/330567143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55930303c9e0 0x55930303cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.545+0000 7f91f8e24740 1 -- 172.21.15.114:0/330567143 >> 172.21.15.114:0/330567143 conn(0x559303038f70 msgr2=0x559303039350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:25.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.545+0000 7f91eeffd640 1 --2- 172.21.15.114:0/29193977 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f91d80a1410 0x7f91d80a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:25.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.545+0000 7f91f8e24740 1 -- 172.21.15.114:0/330567143 shutdown_connections 2024-09-13T18:32:25.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.545+0000 7f91f8e24740 1 -- 172.21.15.114:0/330567143 wait complete. 2024-09-13T18:32:25.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.546+0000 7f91eeffd640 1 --2- 172.21.15.114:0/29193977 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f91d80a1410 0x7f91d80a38d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f91e8009440 tx=0x7f91e8004ad0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:25.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.546+0000 7f91f8e24740 1 -- start start 2024-09-13T18:32:25.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.546+0000 7f91f8e24740 1 Processor -- start 2024-09-13T18:32:25.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.546+0000 7f91f8e24740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55930303c9e0 0x559303047520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:25.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.546+0000 7f91f8e24740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55930303d300 0x5593030447a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:25.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.546+0000 7f91f8e24740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559303047ba0 0x559303044db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:25.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.546+0000 7f91f8e24740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x559302f5ef80 con 0x55930303c9e0 2024-09-13T18:32:25.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.546+0000 7f91f8e24740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559302f5ec20 con 0x559303047ba0 2024-09-13T18:32:25.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.546+0000 7f91f8e24740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559302f2bab0 con 0x55930303d300 2024-09-13T18:32:25.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.546+0000 7f91f8e24740 10 client.0 _ll_register_callbacks cb 0x559302f9a350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:32:25.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.547+0000 7f91eeffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55930303d300 0x5593030447a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:25.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.547+0000 7f91ef7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55930303c9e0 0x559303047520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:25.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.547+0000 7f91ef7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55930303c9e0 0x559303047520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35058/0 (socket says 172.21.15.114:35058) 2024-09-13T18:32:25.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.547+0000 7f91ef7fe640 1 -- 172.21.15.114:0/3499044170 learned_addr learned my addr 172.21.15.114:0/3499044170 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:25.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.547+0000 7f91effff640 1 --2- 172.21.15.114:0/3499044170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559303047ba0 0x559303044db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:25.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.547+0000 7f91ef7fe640 1 -- 172.21.15.114:0/3499044170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55930303d300 msgr2=0x5593030447a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:25.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.547+0000 7f91ef7fe640 1 --2- 172.21.15.114:0/3499044170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55930303d300 0x5593030447a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.547+0000 7f91ef7fe640 1 -- 172.21.15.114:0/3499044170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559303047ba0 msgr2=0x559303044db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:25.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.547+0000 7f91ef7fe640 1 --2- 172.21.15.114:0/3499044170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559303047ba0 0x559303044db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.547+0000 7f91ef7fe640 1 -- 172.21.15.114:0/3499044170 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91e40093a0 con 0x55930303c9e0 2024-09-13T18:32:25.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.548+0000 7f91ef7fe640 1 --2- 172.21.15.114:0/3499044170 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55930303c9e0 0x559303047520 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f91e4011bd0 tx=0x7f91e4011650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=706619fabcd06d72 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:25.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.548+0000 7f91ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:32:25.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.548+0000 7f91ceffd640 1 -- 172.21.15.114:0/3499044170 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e4011e70 con 0x55930303c9e0 2024-09-13T18:32:25.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.548+0000 7f91ceffd640 1 -- 172.21.15.114:0/3499044170 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f91e400bce0 con 0x55930303c9e0 2024-09-13T18:32:25.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.548+0000 7f91f8e24740 1 -- 172.21.15.114:0/3499044170 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559302ca5860 con 0x55930303c9e0 2024-09-13T18:32:25.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.548+0000 7f91ceffd640 1 -- 172.21.15.114:0/3499044170 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91e4004040 con 0x55930303c9e0 2024-09-13T18:32:25.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.548+0000 7f91f8e24740 10 client.18291 fetch_fsmap learned FSMap version 34 2024-09-13T18:32:25.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.548+0000 7f91f8e24740 1 -- 172.21.15.114:0/3499044170 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x559302f9f360 con 0x55930303c9e0 2024-09-13T18:32:25.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.549+0000 7f91ceffd640 1 -- 172.21.15.114:0/3499044170 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559302ca5860 con 0x55930303c9e0 2024-09-13T18:32:25.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.549+0000 7f91ceffd640 1 -- 172.21.15.114:0/3499044170 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f91e4003c80 con 0x55930303c9e0 2024-09-13T18:32:25.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.550+0000 7f91f8e24740 10 client.18291 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:32:25.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.550+0000 7f91f8e24740 10 client.18291 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:32:25.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.550+0000 7f91f8e24740 20 client.18291 populate_metadata read hostname 'smithi114' 2024-09-13T18:32:25.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.550+0000 7f91f8e24740 1 --2- 172.21.15.114:0/3499044170 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559303057920 0x559303059d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:25.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.550+0000 7f91f8e24740 4 client.18291 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:32:25.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.550+0000 7f91f8e24740 1 -- 172.21.15.114:0/3499044170 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559302cf8eb0 con 0x559303057920 2024-09-13T18:32:25.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.550+0000 7f91eeffd640 1 --2- 172.21.15.114:0/3499044170 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559303057920 0x559303059d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:25.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.551+0000 7f91eeffd640 1 --2- 172.21.15.114:0/3499044170 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559303057920 0x559303059d40 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=982d4725288128fb server_cookie=e5cacf2134a18062 in_seq=0 out_seq=0 2024-09-13T18:32:25.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.551+0000 7f91ceffd640 10 client.18291 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:32:25.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.552+0000 7f91ceffd640 1 -- 172.21.15.114:0/3499044170 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x559302cf8eb0 con 0x559303057920 2024-09-13T18:32:25.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.552+0000 7f91ceffd640 10 client.18291 handle_command_reply: tid=0 2024-09-13T18:32:25.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.552+0000 7f91f8e24740 1 client.18291 shutdown 2024-09-13T18:32:25.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.553+0000 7f91f8e24740 1 -- 172.21.15.114:0/3499044170 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55930303c9e0 msgr2=0x559303047520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:25.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.553+0000 7f91f8e24740 1 --2- 172.21.15.114:0/3499044170 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55930303c9e0 0x559303047520 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f91e4011bd0 tx=0x7f91e4011650 comp rx=0 tx=0).stop 2024-09-13T18:32:25.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.554+0000 7f91f8e24740 1 -- 172.21.15.114:0/3499044170 shutdown_connections 2024-09-13T18:32:25.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.554+0000 7f91f8e24740 1 --2- 172.21.15.114:0/3499044170 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559303057920 0x559303059d40 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.554+0000 7f91f8e24740 1 --2- 172.21.15.114:0/3499044170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559303047ba0 0x559303044db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.554+0000 7f91f8e24740 1 --2- 172.21.15.114:0/3499044170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55930303d300 0x5593030447a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.554+0000 7f91f8e24740 1 --2- 172.21.15.114:0/3499044170 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55930303c9e0 0x559303047520 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.554+0000 7f91f8e24740 1 -- 172.21.15.114:0/3499044170 >> 172.21.15.114:0/3499044170 conn(0x559303038f70 msgr2=0x559303038160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:25.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.554+0000 7f91ceffd640 0 client.18291 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:32:25.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.554+0000 7f91f8e24740 1 -- 172.21.15.114:0/3499044170 shutdown_connections 2024-09-13T18:32:25.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.554+0000 7f91f8e24740 1 -- 172.21.15.114:0/3499044170 wait complete. 2024-09-13T18:32:25.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.554+0000 7f91f8e24740 20 client.18291 trim_cache size 0 max 16384 2024-09-13T18:32:25.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.617+0000 7f91f8e24740 1 Processor -- start 2024-09-13T18:32:25.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.620+0000 7f91f8e24740 1 -- start start 2024-09-13T18:32:25.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.620+0000 7f91f8e24740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55930303c9e0 0x559303038950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:25.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.620+0000 7f91f8e24740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55930303d300 0x559303047080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:25.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.620+0000 7f91f8e24740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559303047ba0 0x55930304e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:25.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.621+0000 7f91ef7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55930303c9e0 0x559303038950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:25.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.621+0000 7f91ef7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55930303c9e0 0x559303038950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:33654/0 (socket says 172.21.15.114:33654) 2024-09-13T18:32:25.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.621+0000 7f91eeffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55930303d300 0x559303047080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:25.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.621+0000 7f91eeffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55930303d300 0x559303047080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:54694/0 (socket says 172.21.15.114:54694) 2024-09-13T18:32:25.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.621+0000 7f91ef7fe640 1 -- 172.21.15.114:0/2238854864 learned_addr learned my addr 172.21.15.114:0/2238854864 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:25.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.621+0000 7f91f8e24740 1 -- 172.21.15.114:0/2238854864 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593030453b0 con 0x559303047ba0 2024-09-13T18:32:25.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.621+0000 7f91f8e24740 1 -- 172.21.15.114:0/2238854864 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55930303c780 con 0x55930303d300 2024-09-13T18:32:25.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.621+0000 7f91f8e24740 1 -- 172.21.15.114:0/2238854864 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559303045d10 con 0x55930303c9e0 2024-09-13T18:32:25.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.621+0000 7f91ef7fe640 1 -- 172.21.15.114:0/2238854864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55930303d300 msgr2=0x559303047080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:25.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.621+0000 7f91ef7fe640 1 --2- 172.21.15.114:0/2238854864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55930303d300 0x559303047080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.622+0000 7f91ef7fe640 1 -- 172.21.15.114:0/2238854864 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559303047ba0 msgr2=0x55930304e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:25.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.622+0000 7f91ef7fe640 1 --2- 172.21.15.114:0/2238854864 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559303047ba0 0x55930304e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.622+0000 7f91ef7fe640 1 -- 172.21.15.114:0/2238854864 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559302f411c0 con 0x55930303c9e0 2024-09-13T18:32:25.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.622+0000 7f91ef7fe640 1 --2- 172.21.15.114:0/2238854864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55930303c9e0 0x559303038950 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f91e40118b0 tx=0x7f91e401a610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5d666e626a52e60 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:25.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.623+0000 7f91d67fc640 1 -- 172.21.15.114:0/2238854864 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e4011ca0 con 0x55930303c9e0 2024-09-13T18:32:25.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.623+0000 7f91d67fc640 1 -- 172.21.15.114:0/2238854864 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f91e401a760 con 0x55930303c9e0 2024-09-13T18:32:25.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.623+0000 7f91d67fc640 1 -- 172.21.15.114:0/2238854864 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e401ac00 con 0x55930303c9e0 2024-09-13T18:32:25.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.626+0000 7f91f8e24740 1 -- 172.21.15.114:0/2238854864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55930303c9e0 msgr2=0x559303038950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:25.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.626+0000 7f91f8e24740 1 --2- 172.21.15.114:0/2238854864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55930303c9e0 0x559303038950 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f91e40118b0 tx=0x7f91e401a610 comp rx=0 tx=0).stop 2024-09-13T18:32:25.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.627+0000 7f91f8e24740 1 -- 172.21.15.114:0/2238854864 shutdown_connections 2024-09-13T18:32:25.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.627+0000 7f91f8e24740 1 --2- 172.21.15.114:0/2238854864 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559303047ba0 0x55930304e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.627+0000 7f91f8e24740 1 --2- 172.21.15.114:0/2238854864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55930303d300 0x559303047080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.627+0000 7f91f8e24740 1 --2- 172.21.15.114:0/2238854864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55930303c9e0 0x559303038950 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.627+0000 7f91f8e24740 1 -- 172.21.15.114:0/2238854864 >> 172.21.15.114:0/2238854864 conn(0x559303038f70 msgr2=0x559303043050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:25.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.628+0000 7f91f8e24740 1 -- 172.21.15.114:0/2238854864 shutdown_connections 2024-09-13T18:32:25.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.628+0000 7f91f8e24740 1 -- 172.21.15.114:0/2238854864 wait complete. 2024-09-13T18:32:25.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.629+0000 7f91f8e24740 1 -- start start 2024-09-13T18:32:25.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.629+0000 7f91f8e24740 1 Processor -- start 2024-09-13T18:32:25.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.630+0000 7f91f8e24740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55930303c9e0 0x55930304ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:25.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.630+0000 7f91f8e24740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55930303d300 0x55930304b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:25.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.630+0000 7f91ef7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55930303c9e0 0x55930304ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:25.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.630+0000 7f91ef7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55930303c9e0 0x55930304ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:33670/0 (socket says 172.21.15.114:33670) 2024-09-13T18:32:25.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.630+0000 7f91eeffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55930303d300 0x55930304b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:25.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.630+0000 7f91ef7fe640 1 -- 172.21.15.114:0/1275221607 learned_addr learned my addr 172.21.15.114:0/1275221607 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:25.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.630+0000 7f91f8e24740 1 --2- 172.21.15.114:0/1275221607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559303047ba0 0x55930304b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:25.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.631+0000 7f91f8e24740 1 -- 172.21.15.114:0/1275221607 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55930304c050 con 0x55930303d300 2024-09-13T18:32:25.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.631+0000 7f91effff640 1 --2- 172.21.15.114:0/1275221607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559303047ba0 0x55930304b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:25.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.631+0000 7f91f8e24740 1 -- 172.21.15.114:0/1275221607 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593030ade20 con 0x559303047ba0 2024-09-13T18:32:25.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.631+0000 7f91f8e24740 1 -- 172.21.15.114:0/1275221607 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593030adf60 con 0x55930303c9e0 2024-09-13T18:32:25.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.631+0000 7f91f8e24740 10 client.0 _ll_register_callbacks cb 0x7f91e40095c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:32:25.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.631+0000 7f91ef7fe640 1 -- 172.21.15.114:0/1275221607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559303047ba0 msgr2=0x55930304b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:25.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.631+0000 7f91ef7fe640 1 --2- 172.21.15.114:0/1275221607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559303047ba0 0x55930304b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.631+0000 7f91ef7fe640 1 -- 172.21.15.114:0/1275221607 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55930303d300 msgr2=0x55930304b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:25.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.631+0000 7f91ef7fe640 1 --2- 172.21.15.114:0/1275221607 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55930303d300 0x55930304b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.631+0000 7f91ef7fe640 1 -- 172.21.15.114:0/1275221607 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559302f9f360 con 0x55930303c9e0 2024-09-13T18:32:25.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.632+0000 7f91effff640 1 --2- 172.21.15.114:0/1275221607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559303047ba0 0x55930304b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:32:25.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.633+0000 7f91ef7fe640 1 --2- 172.21.15.114:0/1275221607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55930303c9e0 0x55930304ee40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f91e4006bb0 tx=0x7f91e40096e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de0935cee535ef14 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:25.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.635+0000 7f91ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:32:25.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.635+0000 7f91ceffd640 1 -- 172.21.15.114:0/1275221607 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e400bae0 con 0x55930303c9e0 2024-09-13T18:32:25.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.635+0000 7f91ceffd640 1 -- 172.21.15.114:0/1275221607 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f91e400bc40 con 0x55930303c9e0 2024-09-13T18:32:25.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.636+0000 7f91f8e24740 1 -- 172.21.15.114:0/1275221607 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559302c94590 con 0x55930303c9e0 2024-09-13T18:32:25.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.636+0000 7f91ceffd640 1 -- 172.21.15.114:0/1275221607 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91e400be00 con 0x55930303c9e0 2024-09-13T18:32:25.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.646+0000 7f91f8e24740 10 client.27724 fetch_fsmap learned FSMap version 34 2024-09-13T18:32:25.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.646+0000 7f91f8e24740 1 -- 172.21.15.114:0/1275221607 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x559302f66990 con 0x55930303c9e0 2024-09-13T18:32:25.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.651+0000 7f91ceffd640 1 -- 172.21.15.114:0/1275221607 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559302c94590 con 0x55930303c9e0 2024-09-13T18:32:25.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.651+0000 7f91ceffd640 1 -- 172.21.15.114:0/1275221607 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f91e402f780 con 0x55930303c9e0 2024-09-13T18:32:25.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.652+0000 7f91f8e24740 10 client.27724 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:32:25.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.652+0000 7f91f8e24740 10 client.27724 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:32:25.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.652+0000 7f91f8e24740 20 client.27724 populate_metadata read hostname 'smithi114' 2024-09-13T18:32:25.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.652+0000 7f91f8e24740 1 --2- 172.21.15.114:0/1275221607 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5593030af220 0x5593030b1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:25.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.652+0000 7f91f8e24740 4 client.27724 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-13T18:32:25.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.652+0000 7f91f8e24740 1 -- 172.21.15.114:0/1275221607 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5593030b1ce0 con 0x5593030af220 2024-09-13T18:32:25.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.656+0000 7f91eeffd640 1 --2- 172.21.15.114:0/1275221607 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5593030af220 0x5593030b1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:25.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.662+0000 7f91eeffd640 1 --2- 172.21.15.114:0/1275221607 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5593030af220 0x5593030b1610 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a968c45b94c64fe9 server_cookie=45dd362ce4abbc9f in_seq=0 out_seq=0 2024-09-13T18:32:25.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.663+0000 7f91ceffd640 10 client.27724 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:32:25.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.665+0000 7f91ceffd640 1 -- 172.21.15.114:0/1275221607 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5593030b1ce0 con 0x5593030af220 2024-09-13T18:32:25.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.665+0000 7f91ceffd640 10 client.27724 handle_command_reply: tid=0 2024-09-13T18:32:25.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.665+0000 7f91f8e24740 1 client.27724 shutdown 2024-09-13T18:32:25.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.675+0000 7f91f8e24740 1 -- 172.21.15.114:0/1275221607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55930303c9e0 msgr2=0x55930304ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:25.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.675+0000 7f91f8e24740 1 --2- 172.21.15.114:0/1275221607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55930303c9e0 0x55930304ee40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f91e4006bb0 tx=0x7f91e40096e0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.681+0000 7f91f8e24740 1 -- 172.21.15.114:0/1275221607 shutdown_connections 2024-09-13T18:32:25.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.681+0000 7f91f8e24740 1 --2- 172.21.15.114:0/1275221607 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5593030af220 0x5593030b1610 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.681+0000 7f91f8e24740 1 --2- 172.21.15.114:0/1275221607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559303047ba0 0x55930304b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.681+0000 7f91f8e24740 1 --2- 172.21.15.114:0/1275221607 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55930303d300 0x55930304b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.681+0000 7f91f8e24740 1 --2- 172.21.15.114:0/1275221607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55930303c9e0 0x55930304ee40 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.682+0000 7f91f8e24740 1 -- 172.21.15.114:0/1275221607 >> 172.21.15.114:0/1275221607 conn(0x559303038f70 msgr2=0x55930303e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:25.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.684+0000 7f91ceffd640 0 client.27724 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:32:25.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.684+0000 7f91f8e24740 1 -- 172.21.15.114:0/1275221607 shutdown_connections 2024-09-13T18:32:25.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.684+0000 7f91f8e24740 1 -- 172.21.15.114:0/1275221607 wait complete. 2024-09-13T18:32:25.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.686+0000 7f91f8e24740 20 client.27724 trim_cache size 0 max 16384 2024-09-13T18:32:25.690 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:32:25.690 INFO:teuthology.orchestra.run.smithi114.stdout: "return_code": 0, 2024-09-13T18:32:25.690 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_tag": "e830f421-058c-4095-8120-9e85f5d3dc15", 2024-09-13T18:32:25.690 INFO:teuthology.orchestra.run.smithi114.stdout: "mode": "asynchronous" 2024-09-13T18:32:25.690 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:32:25.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.695+0000 7f91f4ea6640 1 -- 172.21.15.114:0/29193977 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f91d80a1410 msgr2=0x7f91d80a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:25.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.695+0000 7f91f4ea6640 1 --2- 172.21.15.114:0/29193977 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f91d80a1410 0x7f91d80a38d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f91e8009440 tx=0x7f91e8004ad0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.695+0000 7f91f4ea6640 1 -- 172.21.15.114:0/29193977 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f91f0119400 msgr2=0x7f91f0114a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:25.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.695+0000 7f91f4ea6640 1 --2- 172.21.15.114:0/29193977 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f91f0119400 0x7f91f0114a00 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f91e001a860 tx=0x7f91e00053c0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.697+0000 7f91f4ea6640 1 -- 172.21.15.114:0/29193977 shutdown_connections 2024-09-13T18:32:25.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.697+0000 7f91f4ea6640 1 --2- 172.21.15.114:0/29193977 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f91d80a1410 0x7f91d80a38d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.697+0000 7f91f4ea6640 1 --2- 172.21.15.114:0/29193977 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f91f0119400 0x7f91f0114a00 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.697+0000 7f91f4ea6640 1 --2- 172.21.15.114:0/29193977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91f0107890 0x7f91f0118e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.697+0000 7f91f4ea6640 1 --2- 172.21.15.114:0/29193977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91f01074b0 0x7f91f01188c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:25.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.697+0000 7f91f4ea6640 1 -- 172.21.15.114:0/29193977 >> 172.21.15.114:0/29193977 conn(0x7f91f006aec0 msgr2=0x7f91f006ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:25.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.698+0000 7f91f4ea6640 1 -- 172.21.15.114:0/29193977 shutdown_connections 2024-09-13T18:32:25.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:25.703+0000 7f91f4ea6640 1 -- 172.21.15.114:0/29193977 wait complete. 2024-09-13T18:32:25.737 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T18:32:26.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.037+0000 7f300cb9d640 1 -- 172.21.15.114:0/3819609059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3008119400 msgr2=0x7f300811b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:26.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.037+0000 7f300cb9d640 1 --2- 172.21.15.114:0/3819609059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3008119400 0x7f300811b7f0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f3000009ca0 tx=0x7f300002a8f0 comp rx=0 tx=0).stop 2024-09-13T18:32:26.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.041+0000 7f300cb9d640 1 -- 172.21.15.114:0/3819609059 shutdown_connections 2024-09-13T18:32:26.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.041+0000 7f300cb9d640 1 --2- 172.21.15.114:0/3819609059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3008119400 0x7f300811b7f0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:26.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.041+0000 7f300cb9d640 1 --2- 172.21.15.114:0/3819609059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3008107890 0x7f3008107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:26.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.041+0000 7f300cb9d640 1 --2- 172.21.15.114:0/3819609059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30081074b0 0x7f300810e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:26.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.041+0000 7f300cb9d640 1 -- 172.21.15.114:0/3819609059 >> 172.21.15.114:0/3819609059 conn(0x7f300806af40 msgr2=0x7f300806b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:26.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.044+0000 7f300cb9d640 1 -- 172.21.15.114:0/3819609059 shutdown_connections 2024-09-13T18:32:26.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.048+0000 7f300cb9d640 1 -- 172.21.15.114:0/3819609059 wait complete. 2024-09-13T18:32:26.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.048+0000 7f300cb9d640 1 Processor -- start 2024-09-13T18:32:26.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.048+0000 7f300cb9d640 1 -- start start 2024-09-13T18:32:26.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.049+0000 7f300cb9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30081074b0 0x7f3008112400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:26.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.049+0000 7f300cb9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3008107890 0x7f3008112940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:26.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.049+0000 7f300cb9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3008119400 0x7f3008116f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:26.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.049+0000 7f300cb9d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3008117450 con 0x7f3008119400 2024-09-13T18:32:26.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.049+0000 7f300cb9d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30081175c0 con 0x7f3008107890 2024-09-13T18:32:26.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.049+0000 7f300cb9d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3008117780 con 0x7f30081074b0 2024-09-13T18:32:26.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.050+0000 7f3007fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3008119400 0x7f3008116f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:26.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.050+0000 7f3007fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3008119400 0x7f3008116f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35102/0 (socket says 172.21.15.114:35102) 2024-09-13T18:32:26.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.050+0000 7f3007fff640 1 -- 172.21.15.114:0/4219200687 learned_addr learned my addr 172.21.15.114:0/4219200687 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:26.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.050+0000 7f30077fe640 1 --2- 172.21.15.114:0/4219200687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30081074b0 0x7f3008112400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:26.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.051+0000 7f3007fff640 1 -- 172.21.15.114:0/4219200687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30081074b0 msgr2=0x7f3008112400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:26.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.053+0000 7f3006ffd640 1 --2- 172.21.15.114:0/4219200687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3008107890 0x7f3008112940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:26.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.054+0000 7f3007fff640 1 --2- 172.21.15.114:0/4219200687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30081074b0 0x7f3008112400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:26.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.054+0000 7f3007fff640 1 -- 172.21.15.114:0/4219200687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3008107890 msgr2=0x7f3008112940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:26.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.054+0000 7f3007fff640 1 --2- 172.21.15.114:0/4219200687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3008107890 0x7f3008112940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:26.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.054+0000 7f3007fff640 1 -- 172.21.15.114:0/4219200687 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3000009900 con 0x7f3008119400 2024-09-13T18:32:26.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.054+0000 7f30077fe640 1 --2- 172.21.15.114:0/4219200687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30081074b0 0x7f3008112400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:32:26.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.054+0000 7f3006ffd640 1 --2- 172.21.15.114:0/4219200687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3008107890 0x7f3008112940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:32:26.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.054+0000 7f3007fff640 1 --2- 172.21.15.114:0/4219200687 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3008119400 0x7f3008116f10 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f30000098d0 tx=0x7f3000030ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:26.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.055+0000 7f3004ff9640 1 -- 172.21.15.114:0/4219200687 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300002fa10 con 0x7f3008119400 2024-09-13T18:32:26.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.055+0000 7f300cb9d640 1 -- 172.21.15.114:0/4219200687 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30081179a0 con 0x7f3008119400 2024-09-13T18:32:26.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.055+0000 7f300cb9d640 1 -- 172.21.15.114:0/4219200687 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30081ba510 con 0x7f3008119400 2024-09-13T18:32:26.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.055+0000 7f3004ff9640 1 -- 172.21.15.114:0/4219200687 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f300003f070 con 0x7f3008119400 2024-09-13T18:32:26.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.055+0000 7f3004ff9640 1 -- 172.21.15.114:0/4219200687 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300003bae0 con 0x7f3008119400 2024-09-13T18:32:26.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.058+0000 7f3010bfd740 1 Processor -- start 2024-09-13T18:32:26.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.059+0000 7f3004ff9640 1 -- 172.21.15.114:0/4219200687 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f300000a040 con 0x7f3008119400 2024-09-13T18:32:26.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.059+0000 7f3010bfd740 1 -- start start 2024-09-13T18:32:26.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.060+0000 7f3010bfd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559fb31aafe0 0x559fb31ab3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:26.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.060+0000 7f3010bfd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fb31ab900 0x559fb31b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:26.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.060+0000 7f3010bfd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559fb31b61a0 0x559fb31b8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:26.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.060+0000 7f3010bfd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fb3115f20 con 0x559fb31aafe0 2024-09-13T18:32:26.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.060+0000 7f3010bfd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fb31136b0 con 0x559fb31ab900 2024-09-13T18:32:26.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.060+0000 7f3010bfd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fb310efa0 con 0x559fb31b61a0 2024-09-13T18:32:26.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.060+0000 7f3006ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559fb31aafe0 0x559fb31ab3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:26.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.060+0000 7f30077fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fb31ab900 0x559fb31b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:26.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.060+0000 7f3006ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559fb31aafe0 0x559fb31ab3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35108/0 (socket says 172.21.15.114:35108) 2024-09-13T18:32:26.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.060+0000 7f3006ffd640 1 -- 172.21.15.114:0/1162317987 learned_addr learned my addr 172.21.15.114:0/1162317987 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:26.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.060+0000 7f3006ffd640 1 --2- 172.21.15.114:0/1162317987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559fb31b61a0 0x559fb31b8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:26.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.061+0000 7f3004ff9640 1 --2- 172.21.15.114:0/4219200687 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2ff00a1390 0x7f2ff00a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:26.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.061+0000 7f30077fe640 1 -- 172.21.15.114:0/1162317987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559fb31b61a0 msgr2=0x559fb31b8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:26.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.061+0000 7f30077fe640 1 --2- 172.21.15.114:0/1162317987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559fb31b61a0 0x559fb31b8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:26.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.061+0000 7f30077fe640 1 -- 172.21.15.114:0/1162317987 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559fb31aafe0 msgr2=0x559fb31ab3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:26.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.061+0000 7f30077fe640 1 --2- 172.21.15.114:0/1162317987 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559fb31aafe0 0x559fb31ab3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:26.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.061+0000 7f3007fff640 1 --2- 172.21.15.114:0/4219200687 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2ff00a1390 0x7f2ff00a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:26.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.061+0000 7f30077fe640 1 -- 172.21.15.114:0/1162317987 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559fb30c9840 con 0x559fb31ab900 2024-09-13T18:32:26.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.061+0000 7f3006ffd640 1 --2- 172.21.15.114:0/1162317987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559fb31b61a0 0x559fb31b8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:32:26.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.061+0000 7f3004ff9640 1 -- 172.21.15.114:0/4219200687 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f3000041070 con 0x7f3008119400 2024-09-13T18:32:26.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.061+0000 7f30077fe640 1 --2- 172.21.15.114:0/1162317987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fb31ab900 0x559fb31b5c60 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc00a8f0 tx=0x7f2ffc00adb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fff57b069b11a2e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:26.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.061+0000 7f3007fff640 1 --2- 172.21.15.114:0/4219200687 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2ff00a1390 0x7f2ff00a3850 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f300008df20 tx=0x7f30000e5280 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:26.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.061+0000 7f2fedffb640 1 -- 172.21.15.114:0/1162317987 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc00cb40 con 0x559fb31ab900 2024-09-13T18:32:26.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.061+0000 7f2fedffb640 1 -- 172.21.15.114:0/1162317987 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2ffc00cca0 con 0x559fb31ab900 2024-09-13T18:32:26.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.061+0000 7f2fedffb640 1 -- 172.21.15.114:0/1162317987 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc00cb40 con 0x559fb31ab900 2024-09-13T18:32:26.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.062+0000 7f3010bfd740 1 -- 172.21.15.114:0/1162317987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fb31ab900 msgr2=0x559fb31b5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:26.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.062+0000 7f3010bfd740 1 --2- 172.21.15.114:0/1162317987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fb31ab900 0x559fb31b5c60 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc00a8f0 tx=0x7f2ffc00adb0 comp rx=0 tx=0).stop 2024-09-13T18:32:26.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.062+0000 7f3010bfd740 1 -- 172.21.15.114:0/1162317987 shutdown_connections 2024-09-13T18:32:26.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.062+0000 7f3010bfd740 1 --2- 172.21.15.114:0/1162317987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559fb31b61a0 0x559fb31b8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:26.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.062+0000 7f3010bfd740 1 --2- 172.21.15.114:0/1162317987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fb31ab900 0x559fb31b5c60 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:26.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.062+0000 7f3010bfd740 1 --2- 172.21.15.114:0/1162317987 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559fb31aafe0 0x559fb31ab3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:26.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.062+0000 7f3010bfd740 1 -- 172.21.15.114:0/1162317987 >> 172.21.15.114:0/1162317987 conn(0x559fb31a7570 msgr2=0x559fb31a7950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:26.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.062+0000 7f3010bfd740 1 -- 172.21.15.114:0/1162317987 shutdown_connections 2024-09-13T18:32:26.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.062+0000 7f3010bfd740 1 -- 172.21.15.114:0/1162317987 wait complete. 2024-09-13T18:32:26.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.063+0000 7f3010bfd740 1 -- start start 2024-09-13T18:32:26.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.063+0000 7f3010bfd740 1 Processor -- start 2024-09-13T18:32:26.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.063+0000 7f3010bfd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559fb31aafe0 0x559fb31b59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:26.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.063+0000 7f3010bfd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559fb31ab900 0x559fb31b2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:26.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.063+0000 7f3010bfd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fb31b61a0 0x559fb31b31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:26.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.063+0000 7f3010bfd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fb30b02a0 con 0x559fb31ab900 2024-09-13T18:32:26.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.063+0000 7f3010bfd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fb2de7430 con 0x559fb31b61a0 2024-09-13T18:32:26.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.063+0000 7f3010bfd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fb3092680 con 0x559fb31aafe0 2024-09-13T18:32:26.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.063+0000 7f3010bfd740 10 client.0 _ll_register_callbacks cb 0x559fb3108610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:32:26.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.063+0000 7f30077fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559fb31ab900 0x559fb31b2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:26.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.063+0000 7f3006ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559fb31aafe0 0x559fb31b59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:26.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.063+0000 7f30077fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559fb31ab900 0x559fb31b2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35114/0 (socket says 172.21.15.114:35114) 2024-09-13T18:32:26.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.063+0000 7f30077fe640 1 -- 172.21.15.114:0/3437673034 learned_addr learned my addr 172.21.15.114:0/3437673034 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:26.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.063+0000 7f3006ffd640 1 --2- 172.21.15.114:0/3437673034 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fb31b61a0 0x559fb31b31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:26.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.063+0000 7f30077fe640 1 -- 172.21.15.114:0/3437673034 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559fb31aafe0 msgr2=0x559fb31b59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:26.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.063+0000 7f30077fe640 1 --2- 172.21.15.114:0/3437673034 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559fb31aafe0 0x559fb31b59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:26.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.063+0000 7f30077fe640 1 -- 172.21.15.114:0/3437673034 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fb31b61a0 msgr2=0x559fb31b31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:26.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.063+0000 7f30077fe640 1 --2- 172.21.15.114:0/3437673034 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fb31b61a0 0x559fb31b31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:26.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.064+0000 7f30077fe640 1 -- 172.21.15.114:0/3437673034 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ffc00a520 con 0x559fb31ab900 2024-09-13T18:32:26.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.064+0000 7f2fe6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:32:26.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.064+0000 7f30077fe640 1 --2- 172.21.15.114:0/3437673034 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559fb31ab900 0x559fb31b2c70 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc01a6b0 tx=0x7f2ffc01a6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=521032d382b24669 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:26.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.064+0000 7f2fe6ffd640 1 -- 172.21.15.114:0/3437673034 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc01f070 con 0x559fb31ab900 2024-09-13T18:32:26.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.064+0000 7f3010bfd740 1 -- 172.21.15.114:0/3437673034 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559fb2e024e0 con 0x559fb31ab900 2024-09-13T18:32:26.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.064+0000 7f2fe6ffd640 1 -- 172.21.15.114:0/3437673034 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f2ffc0118d0 con 0x559fb31ab900 2024-09-13T18:32:26.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.065+0000 7f2fe6ffd640 1 -- 172.21.15.114:0/3437673034 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ffc01f070 con 0x559fb31ab900 2024-09-13T18:32:26.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.065+0000 7f3010bfd740 10 client.18303 fetch_fsmap learned FSMap version 34 2024-09-13T18:32:26.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.065+0000 7f3010bfd740 1 -- 172.21.15.114:0/3437673034 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x559fb310d620 con 0x559fb31ab900 2024-09-13T18:32:26.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.065+0000 7f2fe6ffd640 1 -- 172.21.15.114:0/3437673034 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559fb2e024e0 con 0x559fb31ab900 2024-09-13T18:32:26.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.065+0000 7f2fe6ffd640 1 -- 172.21.15.114:0/3437673034 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2ffc02c070 con 0x559fb31ab900 2024-09-13T18:32:26.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.065+0000 7f3010bfd740 10 client.18303 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:32:26.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.065+0000 7f3010bfd740 10 client.18303 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:32:26.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.065+0000 7f3010bfd740 20 client.18303 populate_metadata read hostname 'smithi114' 2024-09-13T18:32:26.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.066+0000 7f3010bfd740 1 --2- 172.21.15.114:0/3437673034 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559fb31c5d90 0x559fb31c81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:26.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.066+0000 7f3010bfd740 4 client.18303 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:32:26.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.066+0000 7f3010bfd740 1 -- 172.21.15.114:0/3437673034 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559fb2e137e0 con 0x559fb31c5d90 2024-09-13T18:32:26.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.066+0000 7f3006ffd640 1 --2- 172.21.15.114:0/3437673034 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559fb31c5d90 0x559fb31c81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:26.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.067+0000 7f2fe6ffd640 10 client.18303 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:32:26.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.067+0000 7f3006ffd640 1 --2- 172.21.15.114:0/3437673034 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559fb31c5d90 0x559fb31c81b0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddf14c0b21ad51f9 server_cookie=b8f1d4a335b9eca9 in_seq=0 out_seq=0 2024-09-13T18:32:26.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.067+0000 7f2fe6ffd640 1 -- 172.21.15.114:0/3437673034 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x559fb2e137e0 con 0x559fb31c5d90 2024-09-13T18:32:26.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.067+0000 7f2fe6ffd640 10 client.18303 handle_command_reply: tid=0 2024-09-13T18:32:26.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.067+0000 7f3010bfd740 1 client.18303 shutdown 2024-09-13T18:32:26.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.067+0000 7f3010bfd740 1 -- 172.21.15.114:0/3437673034 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559fb31ab900 msgr2=0x559fb31b2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:26.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.067+0000 7f3010bfd740 1 --2- 172.21.15.114:0/3437673034 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559fb31ab900 0x559fb31b2c70 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc01a6b0 tx=0x7f2ffc01a6e0 comp rx=0 tx=0).stop 2024-09-13T18:32:26.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.067+0000 7f3010bfd740 1 -- 172.21.15.114:0/3437673034 shutdown_connections 2024-09-13T18:32:26.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.067+0000 7f3010bfd740 1 --2- 172.21.15.114:0/3437673034 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559fb31c5d90 0x559fb31c81b0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:26.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.067+0000 7f3010bfd740 1 --2- 172.21.15.114:0/3437673034 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fb31b61a0 0x559fb31b31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:26.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.067+0000 7f3010bfd740 1 --2- 172.21.15.114:0/3437673034 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559fb31ab900 0x559fb31b2c70 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:26.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.067+0000 7f3010bfd740 1 --2- 172.21.15.114:0/3437673034 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559fb31aafe0 0x559fb31b59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:26.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.067+0000 7f3010bfd740 1 -- 172.21.15.114:0/3437673034 >> 172.21.15.114:0/3437673034 conn(0x559fb31a7570 msgr2=0x559fb31a6760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:26.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.067+0000 7f2fe6ffd640 0 client.18303 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:32:26.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.067+0000 7f3010bfd740 1 -- 172.21.15.114:0/3437673034 shutdown_connections 2024-09-13T18:32:26.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.067+0000 7f3010bfd740 1 -- 172.21.15.114:0/3437673034 wait complete. 2024-09-13T18:32:26.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.068+0000 7f3010bfd740 20 client.18303 trim_cache size 0 max 16384 2024-09-13T18:32:26.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.091+0000 7f3010bfd740 1 Processor -- start 2024-09-13T18:32:26.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.092+0000 7f3010bfd740 1 -- start start 2024-09-13T18:32:26.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.092+0000 7f3010bfd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fb31a7570 0x559fb31b1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:26.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.092+0000 7f3010bfd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559fb31aafe0 0x559fb31b1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:26.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.092+0000 7f3010bfd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559fb31ab900 0x559fb31afeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:26.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.092+0000 7f3010bfd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fb2e024e0 con 0x559fb31aafe0 2024-09-13T18:32:26.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.092+0000 7f3010bfd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fb31a82d0 con 0x559fb31a7570 2024-09-13T18:32:26.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.092+0000 7f3010bfd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fb31b4040 con 0x559fb31ab900 2024-09-13T18:32:26.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.096+0000 7f30077fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559fb31aafe0 0x559fb31b1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:26.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.096+0000 7f30077fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559fb31aafe0 0x559fb31b1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35120/0 (socket says 172.21.15.114:35120) 2024-09-13T18:32:26.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.096+0000 7f30077fe640 1 -- 172.21.15.114:0/3871277914 learned_addr learned my addr 172.21.15.114:0/3871277914 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:26.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.097+0000 7f30077fe640 1 -- 172.21.15.114:0/3871277914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559fb31ab900 msgr2=0x559fb31afeb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:32:26.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.097+0000 7f30077fe640 1 --2- 172.21.15.114:0/3871277914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559fb31ab900 0x559fb31afeb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:26.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.097+0000 7f30077fe640 1 -- 172.21.15.114:0/3871277914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fb31a7570 msgr2=0x559fb31b1520 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:32:26.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.097+0000 7f30077fe640 1 --2- 172.21.15.114:0/3871277914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fb31a7570 0x559fb31b1520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:26.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.097+0000 7f30077fe640 1 -- 172.21.15.114:0/3871277914 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559fb30af790 con 0x559fb31aafe0 2024-09-13T18:32:26.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.098+0000 7f30077fe640 1 --2- 172.21.15.114:0/3871277914 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559fb31aafe0 0x559fb31b1a60 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc01a680 tx=0x7f2ffc00a740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84ea75ece6206fbe server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:26.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.098+0000 7f2fee7fc640 1 -- 172.21.15.114:0/3871277914 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc01f070 con 0x559fb31aafe0 2024-09-13T18:32:26.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.098+0000 7f2fee7fc640 1 -- 172.21.15.114:0/3871277914 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2ffc00ce10 con 0x559fb31aafe0 2024-09-13T18:32:26.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.099+0000 7f3010bfd740 1 -- 172.21.15.114:0/3871277914 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559fb31aafe0 msgr2=0x559fb31b1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:26.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.099+0000 7f3010bfd740 1 --2- 172.21.15.114:0/3871277914 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559fb31aafe0 0x559fb31b1a60 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc01a680 tx=0x7f2ffc00a740 comp rx=0 tx=0).stop 2024-09-13T18:32:26.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.100+0000 7f3010bfd740 1 -- 172.21.15.114:0/3871277914 shutdown_connections 2024-09-13T18:32:26.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.100+0000 7f3010bfd740 1 --2- 172.21.15.114:0/3871277914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559fb31ab900 0x559fb31afeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:26.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.100+0000 7f3010bfd740 1 --2- 172.21.15.114:0/3871277914 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559fb31aafe0 0x559fb31b1a60 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:26.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.100+0000 7f3010bfd740 1 --2- 172.21.15.114:0/3871277914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fb31a7570 0x559fb31b1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:26.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.100+0000 7f3010bfd740 1 -- 172.21.15.114:0/3871277914 >> 172.21.15.114:0/3871277914 conn(0x559fb31b5b00 msgr2=0x559fb31adf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:26.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.106+0000 7f3010bfd740 1 -- 172.21.15.114:0/3871277914 shutdown_connections 2024-09-13T18:32:26.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.111+0000 7f3010bfd740 1 -- 172.21.15.114:0/3871277914 wait complete. 2024-09-13T18:32:26.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.111+0000 7f3010bfd740 1 -- start start 2024-09-13T18:32:26.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.111+0000 7f3010bfd740 1 Processor -- start 2024-09-13T18:32:26.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.111+0000 7f3010bfd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fb31aafe0 0x559fb31bb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:26.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.111+0000 7f3010bfd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559fb31b61a0 0x559fb31bb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:26.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.111+0000 7f3010bfd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559fb3219240 0x559fb321b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:26.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.112+0000 7f3010bfd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fb321bb70 con 0x559fb3219240 2024-09-13T18:32:26.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.112+0000 7f3010bfd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fb321bcb0 con 0x559fb31aafe0 2024-09-13T18:32:26.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.112+0000 7f3010bfd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fb321bdf0 con 0x559fb31b61a0 2024-09-13T18:32:26.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.112+0000 7f3010bfd740 10 client.0 _ll_register_callbacks cb 0x7f2ff8009a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:32:26.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.112+0000 7f3006ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559fb3219240 0x559fb321b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:26.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.112+0000 7f3006ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559fb3219240 0x559fb321b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35122/0 (socket says 172.21.15.114:35122) 2024-09-13T18:32:26.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.112+0000 7f3006ffd640 1 -- 172.21.15.114:0/3204184859 learned_addr learned my addr 172.21.15.114:0/3204184859 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:26.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.114+0000 7f3006ffd640 1 --2- 172.21.15.114:0/3204184859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fb31aafe0 0x559fb31bb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:26.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.118+0000 7f3006ffd640 1 -- 172.21.15.114:0/3204184859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559fb31b61a0 msgr2=0x559fb31bb590 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:32:26.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.118+0000 7f3006ffd640 1 --2- 172.21.15.114:0/3204184859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559fb31b61a0 0x559fb31bb590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:26.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.118+0000 7f3006ffd640 1 -- 172.21.15.114:0/3204184859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fb31aafe0 msgr2=0x559fb31bb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:26.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.118+0000 7f3006ffd640 1 --2- 172.21.15.114:0/3204184859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fb31aafe0 0x559fb31bb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:26.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.119+0000 7f3006ffd640 1 -- 172.21.15.114:0/3204184859 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559fb310d620 con 0x559fb3219240 2024-09-13T18:32:26.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.127+0000 7f3006ffd640 1 --2- 172.21.15.114:0/3204184859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559fb3219240 0x559fb321b630 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x559fb31a88f0 tx=0x7f2ff8009410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6645252df204fa72 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:26.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.129+0000 7f2fe6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:32:26.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.132+0000 7f2fe6ffd640 1 -- 172.21.15.114:0/3204184859 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8010af0 con 0x559fb3219240 2024-09-13T18:32:26.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.132+0000 7f3010bfd740 1 -- 172.21.15.114:0/3204184859 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559fb2e66e30 con 0x559fb3219240 2024-09-13T18:32:26.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.133+0000 7f2fe6ffd640 1 -- 172.21.15.114:0/3204184859 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f2ff8010c50 con 0x559fb3219240 2024-09-13T18:32:26.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.139+0000 7f2fe6ffd640 1 -- 172.21.15.114:0/3204184859 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ff8010e10 con 0x559fb3219240 2024-09-13T18:32:26.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.148+0000 7f3010bfd740 10 client.18309 fetch_fsmap learned FSMap version 34 2024-09-13T18:32:26.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.148+0000 7f3010bfd740 1 -- 172.21.15.114:0/3204184859 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x559fb30d4ea0 con 0x559fb3219240 2024-09-13T18:32:26.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.149+0000 7f2fe6ffd640 1 -- 172.21.15.114:0/3204184859 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559fb2e66e30 con 0x559fb3219240 2024-09-13T18:32:26.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.149+0000 7f2fe6ffd640 1 -- 172.21.15.114:0/3204184859 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2ff8004ab0 con 0x559fb3219240 2024-09-13T18:32:26.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.149+0000 7f3010bfd740 10 client.18309 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:32:26.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.149+0000 7f3010bfd740 10 client.18309 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:32:26.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.149+0000 7f3010bfd740 20 client.18309 populate_metadata read hostname 'smithi114' 2024-09-13T18:32:26.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.149+0000 7f3010bfd740 1 --2- 172.21.15.114:0/3204184859 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559fb321d680 0x559fb321faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:26.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.149+0000 7f3010bfd740 4 client.18309 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T18:32:26.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.149+0000 7f3010bfd740 1 -- 172.21.15.114:0/3204184859 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559fb32202f0 con 0x559fb321d680 2024-09-13T18:32:26.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.149+0000 7f30077fe640 1 --2- 172.21.15.114:0/3204184859 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559fb321d680 0x559fb321faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:26.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.150+0000 7f2fe6ffd640 10 client.18309 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:32:26.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.150+0000 7f30077fe640 1 --2- 172.21.15.114:0/3204184859 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559fb321d680 0x559fb321faa0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4f9cb8e83627fb3 server_cookie=5e69883ecb3b06a7 in_seq=0 out_seq=0 2024-09-13T18:32:26.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.151+0000 7f2fe6ffd640 1 -- 172.21.15.114:0/3204184859 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x559fb32202f0 con 0x559fb321d680 2024-09-13T18:32:26.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.151+0000 7f2fe6ffd640 10 client.18309 handle_command_reply: tid=0 2024-09-13T18:32:26.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.151+0000 7f3010bfd740 1 client.18309 shutdown 2024-09-13T18:32:26.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.152+0000 7f3010bfd740 1 -- 172.21.15.114:0/3204184859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559fb3219240 msgr2=0x559fb321b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:26.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.152+0000 7f3010bfd740 1 --2- 172.21.15.114:0/3204184859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559fb3219240 0x559fb321b630 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x559fb31a88f0 tx=0x7f2ff8009410 comp rx=0 tx=0).stop 2024-09-13T18:32:26.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.153+0000 7f3010bfd740 1 -- 172.21.15.114:0/3204184859 shutdown_connections 2024-09-13T18:32:26.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.153+0000 7f3010bfd740 1 --2- 172.21.15.114:0/3204184859 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559fb321d680 0x559fb321faa0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:26.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.153+0000 7f3010bfd740 1 --2- 172.21.15.114:0/3204184859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559fb3219240 0x559fb321b630 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:26.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.153+0000 7f3010bfd740 1 --2- 172.21.15.114:0/3204184859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559fb31b61a0 0x559fb31bb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:26.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.153+0000 7f3010bfd740 1 --2- 172.21.15.114:0/3204184859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fb31aafe0 0x559fb31bb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:26.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.153+0000 7f3010bfd740 1 -- 172.21.15.114:0/3204184859 >> 172.21.15.114:0/3204184859 conn(0x559fb31b5b00 msgr2=0x559fb31b1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:26.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.153+0000 7f2fe6ffd640 0 client.18309 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:32:26.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.153+0000 7f3010bfd740 1 -- 172.21.15.114:0/3204184859 shutdown_connections 2024-09-13T18:32:26.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.153+0000 7f3010bfd740 1 -- 172.21.15.114:0/3204184859 wait complete. 2024-09-13T18:32:26.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.153+0000 7f3010bfd740 20 client.18309 trim_cache size 0 max 16384 2024-09-13T18:32:26.158 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:32:26.158 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-13T18:32:26.158 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": { 2024-09-13T18:32:26.158 INFO:teuthology.orchestra.run.smithi114.stdout: "e830f421-058c-4095-8120-9e85f5d3dc15": { 2024-09-13T18:32:26.159 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-13T18:32:26.159 INFO:teuthology.orchestra.run.smithi114.stdout: "tag": "e830f421-058c-4095-8120-9e85f5d3dc15", 2024-09-13T18:32:26.159 INFO:teuthology.orchestra.run.smithi114.stdout: "options": "recursive,force" 2024-09-13T18:32:26.159 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:32:26.159 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:32:26.159 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:32:26.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.155+0000 7f300cb9d640 1 -- 172.21.15.114:0/4219200687 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2ff00a1390 msgr2=0x7f2ff00a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:26.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.155+0000 7f300cb9d640 1 --2- 172.21.15.114:0/4219200687 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2ff00a1390 0x7f2ff00a3850 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f300008df20 tx=0x7f30000e5280 comp rx=0 tx=0).stop 2024-09-13T18:32:26.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.155+0000 7f300cb9d640 1 -- 172.21.15.114:0/4219200687 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3008119400 msgr2=0x7f3008116f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:26.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.155+0000 7f300cb9d640 1 --2- 172.21.15.114:0/4219200687 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3008119400 0x7f3008116f10 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f30000098d0 tx=0x7f3000030ae0 comp rx=0 tx=0).stop 2024-09-13T18:32:26.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.157+0000 7f300cb9d640 1 -- 172.21.15.114:0/4219200687 shutdown_connections 2024-09-13T18:32:26.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.157+0000 7f300cb9d640 1 --2- 172.21.15.114:0/4219200687 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2ff00a1390 0x7f2ff00a3850 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:26.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.157+0000 7f300cb9d640 1 --2- 172.21.15.114:0/4219200687 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3008119400 0x7f3008116f10 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:26.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.157+0000 7f300cb9d640 1 --2- 172.21.15.114:0/4219200687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3008107890 0x7f3008112940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:26.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.157+0000 7f300cb9d640 1 --2- 172.21.15.114:0/4219200687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30081074b0 0x7f3008112400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:26.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.157+0000 7f300cb9d640 1 -- 172.21.15.114:0/4219200687 >> 172.21.15.114:0/4219200687 conn(0x7f300806af40 msgr2=0x7f300806eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:26.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.158+0000 7f300cb9d640 1 -- 172.21.15.114:0/4219200687 shutdown_connections 2024-09-13T18:32:26.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:26.159+0000 7f300cb9d640 1 -- 172.21.15.114:0/4219200687 wait complete. 2024-09-13T18:32:26.185 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:25 smithi114 ceph-mon[30897]: pgmap v309: 129 pgs: 129 active+clean; 20 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 25 MiB/s wr, 572 op/s 2024-09-13T18:32:26.185 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:25 smithi114 ceph-mon[30897]: scrub summary: active paths [/] 2024-09-13T18:32:26.189 INFO:tasks.cephfs.filesystem:scrub status for tag:e830f421-058c-4095-8120-9e85f5d3dc15 - {'path': '/', 'tag': 'e830f421-058c-4095-8120-9e85f5d3dc15', 'options': 'recursive,force'} 2024-09-13T18:32:26.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:25 smithi204 ceph-mon[34777]: pgmap v309: 129 pgs: 129 active+clean; 20 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 25 MiB/s wr, 572 op/s 2024-09-13T18:32:26.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:25 smithi204 ceph-mon[34777]: scrub summary: active paths [/] 2024-09-13T18:32:26.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:25 smithi134 ceph-mon[34283]: pgmap v309: 129 pgs: 129 active+clean; 20 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 25 MiB/s wr, 572 op/s 2024-09-13T18:32:26.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:25 smithi134 ceph-mon[34283]: scrub summary: active paths [/] 2024-09-13T18:32:27.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:26 smithi204 ceph-mon[34777]: scrub queued for path: / 2024-09-13T18:32:27.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:26 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:32:27.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:26 smithi204 ceph-mon[34777]: scrub summary: active paths [/] 2024-09-13T18:32:27.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:26 smithi204 ceph-mon[34777]: scrub summary: active paths [/] 2024-09-13T18:32:27.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:26 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:32:27.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:26 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:32:27.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:26 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:32:27.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:26 smithi114 ceph-mon[30897]: scrub queued for path: / 2024-09-13T18:32:27.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:26 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:32:27.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:26 smithi114 ceph-mon[30897]: scrub summary: active paths [/] 2024-09-13T18:32:27.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:26 smithi114 ceph-mon[30897]: scrub summary: active paths [/] 2024-09-13T18:32:27.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:26 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:32:27.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:26 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:32:27.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:26 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:32:27.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:26 smithi134 ceph-mon[34283]: scrub queued for path: / 2024-09-13T18:32:27.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:26 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:32:27.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:26 smithi134 ceph-mon[34283]: scrub summary: active paths [/] 2024-09-13T18:32:27.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:26 smithi134 ceph-mon[34283]: scrub summary: active paths [/] 2024-09-13T18:32:27.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:26 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:32:27.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:26 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:32:27.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:26 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:32:28.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:27 smithi204 ceph-mon[34777]: pgmap v310: 129 pgs: 129 active+clean; 20 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 27 MiB/s wr, 629 op/s 2024-09-13T18:32:28.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:27 smithi114 ceph-mon[30897]: pgmap v310: 129 pgs: 129 active+clean; 20 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 27 MiB/s wr, 629 op/s 2024-09-13T18:32:28.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:27 smithi134 ceph-mon[34283]: pgmap v310: 129 pgs: 129 active+clean; 20 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 27 MiB/s wr, 629 op/s 2024-09-13T18:32:30.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:29 smithi204 ceph-mon[34777]: pgmap v311: 129 pgs: 129 active+clean; 20 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 28 MiB/s wr, 638 op/s 2024-09-13T18:32:30.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:29 smithi114 ceph-mon[30897]: pgmap v311: 129 pgs: 129 active+clean; 20 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 28 MiB/s wr, 638 op/s 2024-09-13T18:32:30.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:29 smithi134 ceph-mon[34283]: pgmap v311: 129 pgs: 129 active+clean; 20 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 28 MiB/s wr, 638 op/s 2024-09-13T18:32:32.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:31 smithi204 ceph-mon[34777]: pgmap v312: 129 pgs: 129 active+clean; 20 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 30 MiB/s wr, 679 op/s 2024-09-13T18:32:32.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:31 smithi114 ceph-mon[30897]: pgmap v312: 129 pgs: 129 active+clean; 20 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 30 MiB/s wr, 679 op/s 2024-09-13T18:32:32.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:31 smithi134 ceph-mon[34283]: pgmap v312: 129 pgs: 129 active+clean; 20 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 30 MiB/s wr, 679 op/s 2024-09-13T18:32:34.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:33 smithi204 ceph-mon[34777]: pgmap v313: 129 pgs: 129 active+clean; 20 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 30 MiB/s wr, 690 op/s 2024-09-13T18:32:34.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:33 smithi114 ceph-mon[30897]: pgmap v313: 129 pgs: 129 active+clean; 20 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 30 MiB/s wr, 690 op/s 2024-09-13T18:32:34.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:33 smithi134 ceph-mon[34283]: pgmap v313: 129 pgs: 129 active+clean; 20 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 30 MiB/s wr, 690 op/s 2024-09-13T18:32:36.194 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:35 smithi204 ceph-mon[34777]: pgmap v314: 129 pgs: 129 active+clean; 20 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 31 MiB/s wr, 705 op/s 2024-09-13T18:32:36.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:35 smithi114 ceph-mon[30897]: pgmap v314: 129 pgs: 129 active+clean; 20 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 31 MiB/s wr, 705 op/s 2024-09-13T18:32:36.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:35 smithi134 ceph-mon[34283]: pgmap v314: 129 pgs: 129 active+clean; 20 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 31 MiB/s wr, 705 op/s 2024-09-13T18:32:37.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:36 smithi114 ceph-mon[30897]: scrub summary: idle 2024-09-13T18:32:37.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:36 smithi114 ceph-mon[30897]: scrub summary: idle 2024-09-13T18:32:37.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:36 smithi114 ceph-mon[30897]: scrub summary: idle 2024-09-13T18:32:37.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:36 smithi134 ceph-mon[34283]: scrub summary: idle 2024-09-13T18:32:37.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:36 smithi134 ceph-mon[34283]: scrub summary: idle 2024-09-13T18:32:37.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:36 smithi134 ceph-mon[34283]: scrub summary: idle 2024-09-13T18:32:37.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:36 smithi204 ceph-mon[34777]: scrub summary: idle 2024-09-13T18:32:37.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:36 smithi204 ceph-mon[34777]: scrub summary: idle 2024-09-13T18:32:37.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:36 smithi204 ceph-mon[34777]: scrub summary: idle 2024-09-13T18:32:38.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:37 smithi114 ceph-mon[30897]: pgmap v315: 129 pgs: 129 active+clean; 20 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 34 MiB/s wr, 774 op/s 2024-09-13T18:32:38.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:37 smithi134 ceph-mon[34283]: pgmap v315: 129 pgs: 129 active+clean; 20 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 34 MiB/s wr, 774 op/s 2024-09-13T18:32:38.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:37 smithi204 ceph-mon[34777]: pgmap v315: 129 pgs: 129 active+clean; 20 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 34 MiB/s wr, 774 op/s 2024-09-13T18:32:38.639 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.636+0000 7f39f7ee9640 1 -- 172.21.15.114:0/1837048541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39f0107890 msgr2=0x7f39f0107cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:38.639 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.636+0000 7f39f7ee9640 1 --2- 172.21.15.114:0/1837048541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39f0107890 0x7f39f0107cd0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f39ec009890 tx=0x7f39ec02a370 comp rx=0 tx=0).stop 2024-09-13T18:32:38.643 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.641+0000 7f39f7ee9640 1 -- 172.21.15.114:0/1837048541 shutdown_connections 2024-09-13T18:32:38.643 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.641+0000 7f39f7ee9640 1 --2- 172.21.15.114:0/1837048541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39f0119400 0x7f39f011b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:38.643 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.641+0000 7f39f7ee9640 1 --2- 172.21.15.114:0/1837048541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39f0107890 0x7f39f0107cd0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:38.643 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.641+0000 7f39f7ee9640 1 --2- 172.21.15.114:0/1837048541 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39f01074b0 0x7f39f010e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:38.643 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.641+0000 7f39f7ee9640 1 -- 172.21.15.114:0/1837048541 >> 172.21.15.114:0/1837048541 conn(0x7f39f006ae30 msgr2=0x7f39f006b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:38.645 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.643+0000 7f39f7ee9640 1 -- 172.21.15.114:0/1837048541 shutdown_connections 2024-09-13T18:32:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.644+0000 7f39f7ee9640 1 -- 172.21.15.114:0/1837048541 wait complete. 2024-09-13T18:32:38.647 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.645+0000 7f39f7ee9640 1 Processor -- start 2024-09-13T18:32:38.649 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.647+0000 7f39f7ee9640 1 -- start start 2024-09-13T18:32:38.651 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.647+0000 7f39f7ee9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39f01074b0 0x7f39f0116850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:38.651 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.647+0000 7f39f7ee9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39f0107890 0x7f39f0116db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:38.651 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.647+0000 7f39f7ee9640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39f0119400 0x7f39f010fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:38.651 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.647+0000 7f39f7ee9640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f0110220 con 0x7f39f0119400 2024-09-13T18:32:38.651 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.647+0000 7f39f7ee9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f0110390 con 0x7f39f0107890 2024-09-13T18:32:38.651 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.647+0000 7f39f7ee9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f0110550 con 0x7f39f01074b0 2024-09-13T18:32:38.652 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.649+0000 7f39f6486640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39f0119400 0x7f39f010fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:38.652 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.649+0000 7f39f6486640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39f0119400 0x7f39f010fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43716/0 (socket says 172.21.15.114:43716) 2024-09-13T18:32:38.652 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.649+0000 7f39f6486640 1 -- 172.21.15.114:0/2050020932 learned_addr learned my addr 172.21.15.114:0/2050020932 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:38.652 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.649+0000 7f39f5c85640 1 --2- 172.21.15.114:0/2050020932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39f01074b0 0x7f39f0116850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:38.652 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.650+0000 7f39f5484640 1 --2- 172.21.15.114:0/2050020932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39f0107890 0x7f39f0116db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.651+0000 7f39f5484640 1 -- 172.21.15.114:0/2050020932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39f01074b0 msgr2=0x7f39f0116850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.651+0000 7f39f5484640 1 --2- 172.21.15.114:0/2050020932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39f01074b0 0x7f39f0116850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.651+0000 7f39f5484640 1 -- 172.21.15.114:0/2050020932 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39f0119400 msgr2=0x7f39f010fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.651+0000 7f39f5484640 1 --2- 172.21.15.114:0/2050020932 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39f0119400 0x7f39f010fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.651+0000 7f39f5484640 1 -- 172.21.15.114:0/2050020932 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39ec009490 con 0x7f39f0107890 2024-09-13T18:32:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.652+0000 7f39f5484640 1 --2- 172.21.15.114:0/2050020932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39f0107890 0x7f39f0116db0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f39ec000c00 tx=0x7f39ec033c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.654+0000 7f39e6ffd640 1 -- 172.21.15.114:0/2050020932 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ec004250 con 0x7f39f0107890 2024-09-13T18:32:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.654+0000 7f39e6ffd640 1 -- 172.21.15.114:0/2050020932 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f39ec0043b0 con 0x7f39f0107890 2024-09-13T18:32:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.654+0000 7f39e6ffd640 1 -- 172.21.15.114:0/2050020932 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ec003c80 con 0x7f39f0107890 2024-09-13T18:32:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.656+0000 7f39f7ee9640 1 -- 172.21.15.114:0/2050020932 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39f01143e0 con 0x7f39f0107890 2024-09-13T18:32:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.656+0000 7f39f7ee9640 1 -- 172.21.15.114:0/2050020932 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39f0114840 con 0x7f39f0107890 2024-09-13T18:32:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.669+0000 7f39fac0a740 1 Processor -- start 2024-09-13T18:32:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.669+0000 7f39fac0a740 1 -- start start 2024-09-13T18:32:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.669+0000 7f39fac0a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e00edfe500 0x55e00edfe8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.669+0000 7f39fac0a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e00edfee20 0x55e00ee09180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.669+0000 7f39fac0a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e00ee096c0 0x55e00ee0bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.669+0000 7f39fac0a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e00ed69c30 con 0x55e00edfee20 2024-09-13T18:32:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.669+0000 7f39fac0a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e00ed673c0 con 0x55e00ee096c0 2024-09-13T18:32:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.669+0000 7f39fac0a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e00ed62cb0 con 0x55e00edfe500 2024-09-13T18:32:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.670+0000 7f39f6486640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e00edfe500 0x55e00edfe8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:38.678 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.675+0000 7f39f6486640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e00edfe500 0x55e00edfe8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:58208/0 (socket says 172.21.15.114:58208) 2024-09-13T18:32:38.678 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.675+0000 7f39f6486640 1 -- 172.21.15.114:0/2812815836 learned_addr learned my addr 172.21.15.114:0/2812815836 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:38.678 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.676+0000 7f39e6ffd640 1 -- 172.21.15.114:0/2050020932 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f39ec039040 con 0x7f39f0107890 2024-09-13T18:32:38.678 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.676+0000 7f39f5c85640 1 --2- 172.21.15.114:0/2812815836 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e00edfee20 0x55e00ee09180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:38.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.676+0000 7f39e6ffd640 1 --2- 172.21.15.114:0/2050020932 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f39dc0a14d0 0x7f39dc0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:38.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.677+0000 7f39e6ffd640 1 -- 172.21.15.114:0/2050020932 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f39ec0ed2e0 con 0x7f39f0107890 2024-09-13T18:32:38.684 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.681+0000 7f39f6486640 1 -- 172.21.15.114:0/2812815836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e00ee096c0 msgr2=0x55e00ee0bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:38.684 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.681+0000 7f39f6486640 1 --2- 172.21.15.114:0/2812815836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e00ee096c0 0x55e00ee0bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:38.685 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.681+0000 7f39f6486640 1 -- 172.21.15.114:0/2812815836 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e00edfee20 msgr2=0x55e00ee09180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:38.685 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.681+0000 7f39f6486640 1 --2- 172.21.15.114:0/2812815836 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e00edfee20 0x55e00ee09180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:38.685 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.681+0000 7f39f6486640 1 -- 172.21.15.114:0/2812815836 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e00ed09910 con 0x55e00edfe500 2024-09-13T18:32:38.685 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.681+0000 7f39f6486640 1 --2- 172.21.15.114:0/2050020932 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f39dc0a14d0 0x7f39dc0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:38.685 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.681+0000 7f39f6486640 1 --2- 172.21.15.114:0/2812815836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e00edfe500 0x55e00edfe8e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f39e8011470 tx=0x7f39e8011930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87f8bb2695ad23a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.688+0000 7f39dbfff640 1 -- 172.21.15.114:0/2812815836 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e800fa80 con 0x55e00edfe500 2024-09-13T18:32:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.688+0000 7f39f6486640 1 --2- 172.21.15.114:0/2050020932 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f39dc0a14d0 0x7f39dc0a3990 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f39e8020000 tx=0x7f39e8018c80 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.688+0000 7f39dbfff640 1 -- 172.21.15.114:0/2812815836 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f39e800fbe0 con 0x55e00edfe500 2024-09-13T18:32:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.689+0000 7f39fac0a740 1 -- 172.21.15.114:0/2812815836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e00edfe500 msgr2=0x55e00edfe8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.689+0000 7f39fac0a740 1 --2- 172.21.15.114:0/2812815836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e00edfe500 0x55e00edfe8e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f39e8011470 tx=0x7f39e8011930 comp rx=0 tx=0).stop 2024-09-13T18:32:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.691+0000 7f39fac0a740 1 -- 172.21.15.114:0/2812815836 shutdown_connections 2024-09-13T18:32:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.691+0000 7f39fac0a740 1 --2- 172.21.15.114:0/2812815836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e00ee096c0 0x55e00ee0bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.691+0000 7f39fac0a740 1 --2- 172.21.15.114:0/2812815836 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e00edfee20 0x55e00ee09180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.691+0000 7f39fac0a740 1 --2- 172.21.15.114:0/2812815836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e00edfe500 0x55e00edfe8e0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.691+0000 7f39fac0a740 1 -- 172.21.15.114:0/2812815836 >> 172.21.15.114:0/2812815836 conn(0x55e00edfaa90 msgr2=0x55e00edfae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:38.695 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.692+0000 7f39fac0a740 1 -- 172.21.15.114:0/2812815836 shutdown_connections 2024-09-13T18:32:38.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.700+0000 7f39fac0a740 1 -- 172.21.15.114:0/2812815836 wait complete. 2024-09-13T18:32:38.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.700+0000 7f39fac0a740 1 -- start start 2024-09-13T18:32:38.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.700+0000 7f39fac0a740 1 Processor -- start 2024-09-13T18:32:38.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.703+0000 7f39fac0a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e00edfe500 0x55e00ee08f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:38.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.703+0000 7f39fac0a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e00edfee20 0x55e00ee06210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:38.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.703+0000 7f39fac0a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e00ee096c0 0x55e00ee06750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:38.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.703+0000 7f39fac0a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e00ed20e80 con 0x55e00ee096c0 2024-09-13T18:32:38.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.703+0000 7f39fac0a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e00ed20b20 con 0x55e00edfe500 2024-09-13T18:32:38.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.703+0000 7f39fac0a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e00ece6260 con 0x55e00edfee20 2024-09-13T18:32:38.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.704+0000 7f39fac0a740 10 client.0 _ll_register_callbacks cb 0x55e00ed1bdf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:32:38.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.704+0000 7f39f5484640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e00edfee20 0x55e00ee06210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:38.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.704+0000 7f39f5484640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e00edfee20 0x55e00ee06210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:58220/0 (socket says 172.21.15.114:58220) 2024-09-13T18:32:38.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.704+0000 7f39f5484640 1 -- 172.21.15.114:0/3274614277 learned_addr learned my addr 172.21.15.114:0/3274614277 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:38.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.706+0000 7f39f5484640 1 -- 172.21.15.114:0/3274614277 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e00edfe500 msgr2=0x55e00ee08f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:38.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.706+0000 7f39f5484640 1 --2- 172.21.15.114:0/3274614277 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e00edfe500 0x55e00ee08f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:38.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.706+0000 7f39f5484640 1 -- 172.21.15.114:0/3274614277 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e00ee096c0 msgr2=0x55e00ee06750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:38.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.706+0000 7f39f5484640 1 --2- 172.21.15.114:0/3274614277 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e00ee096c0 0x55e00ee06750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:38.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.706+0000 7f39f5484640 1 -- 172.21.15.114:0/3274614277 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39e8009cc0 con 0x55e00edfee20 2024-09-13T18:32:38.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.706+0000 7f39f5484640 1 --2- 172.21.15.114:0/3274614277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e00edfee20 0x55e00ee06210 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f39ec0ba090 tx=0x7f39ec0b9c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36df65952d8f994d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:38.712 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.709+0000 7f39d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:32:38.712 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.709+0000 7f39d97fa640 1 -- 172.21.15.114:0/3274614277 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ec0ba100 con 0x55e00edfee20 2024-09-13T18:32:38.713 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.709+0000 7f39fac0a740 1 -- 172.21.15.114:0/3274614277 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e00ed8b680 con 0x55e00edfee20 2024-09-13T18:32:38.713 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.709+0000 7f39d97fa640 1 -- 172.21.15.114:0/3274614277 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f39ec0bdf30 con 0x55e00edfee20 2024-09-13T18:32:38.713 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.710+0000 7f39d97fa640 1 -- 172.21.15.114:0/3274614277 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39ec0c2580 con 0x55e00edfee20 2024-09-13T18:32:38.713 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.710+0000 7f39fac0a740 10 client.27742 fetch_fsmap learned FSMap version 34 2024-09-13T18:32:38.713 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.710+0000 7f39fac0a740 1 -- 172.21.15.114:0/3274614277 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55e00ed61330 con 0x55e00edfee20 2024-09-13T18:32:38.713 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.710+0000 7f39d97fa640 1 -- 172.21.15.114:0/3274614277 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f39ec0d7020 con 0x55e00edfee20 2024-09-13T18:32:38.714 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.711+0000 7f39d97fa640 1 -- 172.21.15.114:0/3274614277 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f39ec0b95f0 con 0x55e00edfee20 2024-09-13T18:32:38.714 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.711+0000 7f39fac0a740 10 client.27742 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:32:38.714 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.711+0000 7f39fac0a740 10 client.27742 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:32:38.714 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.711+0000 7f39fac0a740 20 client.27742 populate_metadata read hostname 'smithi114' 2024-09-13T18:32:38.714 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.711+0000 7f39fac0a740 1 --2- 172.21.15.114:0/3274614277 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55e00ee18f30 0x55e00ee1b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:38.714 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.711+0000 7f39fac0a740 4 client.27742 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:32:38.714 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.711+0000 7f39fac0a740 1 -- 172.21.15.114:0/3274614277 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e00ed565d0 con 0x55e00ee18f30 2024-09-13T18:32:38.715 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.713+0000 7f39f5c85640 1 --2- 172.21.15.114:0/3274614277 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55e00ee18f30 0x55e00ee1b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:38.715 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.713+0000 7f39d97fa640 10 client.27742 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:32:38.716 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.713+0000 7f39f5c85640 1 --2- 172.21.15.114:0/3274614277 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55e00ee18f30 0x55e00ee1b350 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a9a31b2980153af server_cookie=7ccdcba562b4569a in_seq=0 out_seq=0 2024-09-13T18:32:38.716 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.714+0000 7f39d97fa640 1 -- 172.21.15.114:0/3274614277 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55e00ed565d0 con 0x55e00ee18f30 2024-09-13T18:32:38.717 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.714+0000 7f39d97fa640 10 client.27742 handle_command_reply: tid=0 2024-09-13T18:32:38.717 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.714+0000 7f39fac0a740 1 client.27742 shutdown 2024-09-13T18:32:38.717 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.714+0000 7f39fac0a740 1 -- 172.21.15.114:0/3274614277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e00edfee20 msgr2=0x55e00ee06210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:38.717 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.714+0000 7f39fac0a740 1 --2- 172.21.15.114:0/3274614277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e00edfee20 0x55e00ee06210 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f39ec0ba090 tx=0x7f39ec0b9c40 comp rx=0 tx=0).stop 2024-09-13T18:32:38.717 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.714+0000 7f39fac0a740 1 -- 172.21.15.114:0/3274614277 shutdown_connections 2024-09-13T18:32:38.717 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.714+0000 7f39fac0a740 1 --2- 172.21.15.114:0/3274614277 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55e00ee18f30 0x55e00ee1b350 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:38.717 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.714+0000 7f39fac0a740 1 --2- 172.21.15.114:0/3274614277 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e00ee096c0 0x55e00ee06750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:38.717 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.714+0000 7f39fac0a740 1 --2- 172.21.15.114:0/3274614277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e00edfee20 0x55e00ee06210 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:38.717 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.714+0000 7f39fac0a740 1 --2- 172.21.15.114:0/3274614277 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e00edfe500 0x55e00ee08f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:38.717 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.714+0000 7f39d97fa640 0 client.27742 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:32:38.717 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.714+0000 7f39fac0a740 1 -- 172.21.15.114:0/3274614277 >> 172.21.15.114:0/3274614277 conn(0x55e00edfaa90 msgr2=0x55e00edf9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:38.718 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.715+0000 7f39fac0a740 1 -- 172.21.15.114:0/3274614277 shutdown_connections 2024-09-13T18:32:38.718 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.715+0000 7f39fac0a740 1 -- 172.21.15.114:0/3274614277 wait complete. 2024-09-13T18:32:38.718 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.715+0000 7f39fac0a740 20 client.27742 trim_cache size 0 max 16384 2024-09-13T18:32:38.744 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.740+0000 7f39fac0a740 1 Processor -- start 2024-09-13T18:32:38.744 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.741+0000 7f39fac0a740 1 -- start start 2024-09-13T18:32:38.744 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.741+0000 7f39fac0a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e00edfe500 0x55e00ee04b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:38.745 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.741+0000 7f39fac0a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e00edfee20 0x55e00ee05060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:38.745 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.741+0000 7f39fac0a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e00ee096c0 0x55e00ee034d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:38.745 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.741+0000 7f39fac0a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e00ee15d80 con 0x55e00edfee20 2024-09-13T18:32:38.745 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.741+0000 7f39fac0a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e00ee077b0 con 0x55e00ee096c0 2024-09-13T18:32:38.745 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.741+0000 7f39fac0a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e00edfa7f0 con 0x55e00edfe500 2024-09-13T18:32:38.745 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.741+0000 7f39f6486640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e00ee096c0 0x55e00ee034d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:38.745 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.741+0000 7f39f5c85640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e00edfe500 0x55e00ee04b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:38.745 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.742+0000 7f39f5c85640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e00edfe500 0x55e00ee04b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:58236/0 (socket says 172.21.15.114:58236) 2024-09-13T18:32:38.745 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.742+0000 7f39f5c85640 1 -- 172.21.15.114:0/2799049203 learned_addr learned my addr 172.21.15.114:0/2799049203 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:38.749 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.743+0000 7f39f5484640 1 --2- 172.21.15.114:0/2799049203 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e00edfee20 0x55e00ee05060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:38.749 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.744+0000 7f39f5484640 1 -- 172.21.15.114:0/2799049203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e00edfe500 msgr2=0x55e00ee04b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:38.749 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.744+0000 7f39f5484640 1 --2- 172.21.15.114:0/2799049203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e00edfe500 0x55e00ee04b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:38.749 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.744+0000 7f39f5484640 1 -- 172.21.15.114:0/2799049203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e00ee096c0 msgr2=0x55e00ee034d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:38.749 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.744+0000 7f39f5484640 1 --2- 172.21.15.114:0/2799049203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e00ee096c0 0x55e00ee034d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:38.749 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.744+0000 7f39f5484640 1 -- 172.21.15.114:0/2799049203 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e00ed03300 con 0x55e00edfee20 2024-09-13T18:32:38.749 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.746+0000 7f39f5484640 1 --2- 172.21.15.114:0/2799049203 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e00edfee20 0x55e00ee05060 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f39ec0b7a90 tx=0x7f39ec0eaa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e94550c7e67c532 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:38.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.747+0000 7f39e4ff9640 1 -- 172.21.15.114:0/2799049203 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ec0c2580 con 0x55e00edfee20 2024-09-13T18:32:38.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.747+0000 7f39e4ff9640 1 -- 172.21.15.114:0/2799049203 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f39ec0d7020 con 0x55e00edfee20 2024-09-13T18:32:38.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.748+0000 7f39fac0a740 1 -- 172.21.15.114:0/2799049203 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e00edfee20 msgr2=0x55e00ee05060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:38.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.748+0000 7f39e4ff9640 1 -- 172.21.15.114:0/2799049203 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ec0c2580 con 0x55e00edfee20 2024-09-13T18:32:38.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.748+0000 7f39fac0a740 1 --2- 172.21.15.114:0/2799049203 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e00edfee20 0x55e00ee05060 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f39ec0b7a90 tx=0x7f39ec0eaa20 comp rx=0 tx=0).stop 2024-09-13T18:32:38.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.748+0000 7f39fac0a740 1 -- 172.21.15.114:0/2799049203 shutdown_connections 2024-09-13T18:32:38.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.748+0000 7f39fac0a740 1 --2- 172.21.15.114:0/2799049203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e00ee096c0 0x55e00ee034d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:38.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.748+0000 7f39fac0a740 1 --2- 172.21.15.114:0/2799049203 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e00edfee20 0x55e00ee05060 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:38.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.748+0000 7f39fac0a740 1 --2- 172.21.15.114:0/2799049203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e00edfe500 0x55e00ee04b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:38.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.748+0000 7f39fac0a740 1 -- 172.21.15.114:0/2799049203 >> 172.21.15.114:0/2799049203 conn(0x55e00edfaa90 msgr2=0x55e00edf5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:38.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.749+0000 7f39fac0a740 1 -- 172.21.15.114:0/2799049203 shutdown_connections 2024-09-13T18:32:38.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.749+0000 7f39fac0a740 1 -- 172.21.15.114:0/2799049203 wait complete. 2024-09-13T18:32:38.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.749+0000 7f39fac0a740 1 -- start start 2024-09-13T18:32:38.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.749+0000 7f39fac0a740 1 Processor -- start 2024-09-13T18:32:38.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.750+0000 7f39fac0a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e00edfe500 0x55e00ee10080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:38.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.750+0000 7f39fac0a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e00edfee20 0x55e00ee105c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:38.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.750+0000 7f39fac0a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e00ee096c0 0x55e00ee0e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:38.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.750+0000 7f39fac0a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e00ee0ebb0 con 0x55e00edfee20 2024-09-13T18:32:38.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.750+0000 7f39fac0a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e00ee0ed20 con 0x55e00edfe500 2024-09-13T18:32:38.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.750+0000 7f39fac0a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e00ee0eee0 con 0x55e00ee096c0 2024-09-13T18:32:38.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.750+0000 7f39fac0a740 10 client.0 _ll_register_callbacks cb 0x7f39e8009b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:32:38.754 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.750+0000 7f39f5484640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e00edfee20 0x55e00ee105c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:38.754 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.750+0000 7f39f5484640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e00edfee20 0x55e00ee105c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43768/0 (socket says 172.21.15.114:43768) 2024-09-13T18:32:38.754 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.751+0000 7f39f5c85640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e00edfe500 0x55e00ee10080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:38.754 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.751+0000 7f39f6486640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e00ee096c0 0x55e00ee0e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:38.754 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.751+0000 7f39f5484640 1 -- 172.21.15.114:0/2827189601 learned_addr learned my addr 172.21.15.114:0/2827189601 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:38.760 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.758+0000 7f39f5484640 1 -- 172.21.15.114:0/2827189601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e00ee096c0 msgr2=0x55e00ee0e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:38.760 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.758+0000 7f39f5484640 1 --2- 172.21.15.114:0/2827189601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e00ee096c0 0x55e00ee0e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:38.761 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.758+0000 7f39f5484640 1 -- 172.21.15.114:0/2827189601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e00edfe500 msgr2=0x55e00ee10080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:38.761 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.758+0000 7f39f5484640 1 --2- 172.21.15.114:0/2827189601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e00edfe500 0x55e00ee10080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:38.761 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.758+0000 7f39f5484640 1 -- 172.21.15.114:0/2827189601 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e00ed61330 con 0x55e00edfee20 2024-09-13T18:32:38.763 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.761+0000 7f39f5484640 1 --2- 172.21.15.114:0/2827189601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e00edfee20 0x55e00ee105c0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f39ec0edd40 tx=0x7f39ec0b9d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7262739e71a144e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:38.763 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.761+0000 7f39d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:32:38.767 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.764+0000 7f39d97fa640 1 -- 172.21.15.114:0/2827189601 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ec0bdf30 con 0x55e00edfee20 2024-09-13T18:32:38.767 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.765+0000 7f39d97fa640 1 -- 172.21.15.114:0/2827189601 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f39ec0d8080 con 0x55e00edfee20 2024-09-13T18:32:38.769 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.767+0000 7f39fac0a740 1 -- 172.21.15.114:0/2827189601 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f39e800fbe0 con 0x55e00edfee20 2024-09-13T18:32:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.773+0000 7f39d97fa640 1 -- 172.21.15.114:0/2827189601 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39ec0ba100 con 0x55e00edfee20 2024-09-13T18:32:38.780 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.778+0000 7f39fac0a740 10 client.18324 fetch_fsmap learned FSMap version 34 2024-09-13T18:32:38.780 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.778+0000 7f39fac0a740 1 -- 172.21.15.114:0/2827189601 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55e00ed288e0 con 0x55e00edfee20 2024-09-13T18:32:38.784 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.781+0000 7f39d97fa640 1 -- 172.21.15.114:0/2827189601 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f39e800fbe0 con 0x55e00edfee20 2024-09-13T18:32:38.784 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.781+0000 7f39d97fa640 1 -- 172.21.15.114:0/2827189601 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f39ec0b95f0 con 0x55e00edfee20 2024-09-13T18:32:38.785 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.782+0000 7f39fac0a740 10 client.18324 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:32:38.785 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.782+0000 7f39fac0a740 10 client.18324 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:32:38.785 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.782+0000 7f39fac0a740 20 client.18324 populate_metadata read hostname 'smithi114' 2024-09-13T18:32:38.785 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.782+0000 7f39fac0a740 1 --2- 172.21.15.114:0/2827189601 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55e00ee70150 0x55e00ee72540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:38.785 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.782+0000 7f39fac0a740 4 client.18324 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T18:32:38.785 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.782+0000 7f39fac0a740 1 -- 172.21.15.114:0/2827189601 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e00ea56300 con 0x55e00ee70150 2024-09-13T18:32:38.785 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.784+0000 7f39f5c85640 1 --2- 172.21.15.114:0/2827189601 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55e00ee70150 0x55e00ee72540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.795+0000 7f39f5c85640 1 --2- 172.21.15.114:0/2827189601 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55e00ee70150 0x55e00ee72540 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11830ccd1f6eeddf server_cookie=c2abf0e1c1fa5ddb in_seq=0 out_seq=0 2024-09-13T18:32:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.795+0000 7f39d97fa640 10 client.18324 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:32:38.806 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.804+0000 7f39d97fa640 1 -- 172.21.15.114:0/2827189601 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e00ea56300 con 0x55e00ee70150 2024-09-13T18:32:38.806 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.804+0000 7f39d97fa640 10 client.18324 handle_command_reply: tid=0 2024-09-13T18:32:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.808+0000 7f39fac0a740 1 client.18324 shutdown 2024-09-13T18:32:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.810+0000 7f39fac0a740 1 -- 172.21.15.114:0/2827189601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e00edfee20 msgr2=0x55e00ee105c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.810+0000 7f39fac0a740 1 --2- 172.21.15.114:0/2827189601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e00edfee20 0x55e00ee105c0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f39ec0edd40 tx=0x7f39ec0b9d70 comp rx=0 tx=0).stop 2024-09-13T18:32:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.811+0000 7f39fac0a740 1 -- 172.21.15.114:0/2827189601 shutdown_connections 2024-09-13T18:32:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.811+0000 7f39fac0a740 1 --2- 172.21.15.114:0/2827189601 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55e00ee70150 0x55e00ee72540 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.811+0000 7f39fac0a740 1 --2- 172.21.15.114:0/2827189601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e00ee096c0 0x55e00ee0e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.811+0000 7f39fac0a740 1 --2- 172.21.15.114:0/2827189601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e00edfee20 0x55e00ee105c0 secure :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f39ec0edd40 tx=0x7f39ec0b9d70 comp rx=0 tx=0).stop 2024-09-13T18:32:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.811+0000 7f39fac0a740 1 --2- 172.21.15.114:0/2827189601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e00edfe500 0x55e00ee10080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.811+0000 7f39fac0a740 1 -- 172.21.15.114:0/2827189601 >> 172.21.15.114:0/2827189601 conn(0x55e00edfaa90 msgr2=0x55e00edf5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.811+0000 7f39d97fa640 0 client.18324 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:32:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.812+0000 7f39fac0a740 1 -- 172.21.15.114:0/2827189601 shutdown_connections 2024-09-13T18:32:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.812+0000 7f39fac0a740 1 -- 172.21.15.114:0/2827189601 wait complete. 2024-09-13T18:32:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.813+0000 7f39fac0a740 20 client.18324 trim_cache size 0 max 16384 2024-09-13T18:32:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-13T18:32:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-13T18:32:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-13T18:32:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-13T18:32:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.815+0000 7f39f7ee9640 1 -- 172.21.15.114:0/2050020932 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f39dc0a14d0 msgr2=0x7f39dc0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.815+0000 7f39f7ee9640 1 --2- 172.21.15.114:0/2050020932 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f39dc0a14d0 0x7f39dc0a3990 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f39e8020000 tx=0x7f39e8018c80 comp rx=0 tx=0).stop 2024-09-13T18:32:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.815+0000 7f39f7ee9640 1 -- 172.21.15.114:0/2050020932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39f0107890 msgr2=0x7f39f0116db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.815+0000 7f39f7ee9640 1 --2- 172.21.15.114:0/2050020932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39f0107890 0x7f39f0116db0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f39ec000c00 tx=0x7f39ec033c80 comp rx=0 tx=0).stop 2024-09-13T18:32:38.820 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.817+0000 7f39f7ee9640 1 -- 172.21.15.114:0/2050020932 shutdown_connections 2024-09-13T18:32:38.820 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.817+0000 7f39f7ee9640 1 --2- 172.21.15.114:0/2050020932 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f39dc0a14d0 0x7f39dc0a3990 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:38.821 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.817+0000 7f39f7ee9640 1 --2- 172.21.15.114:0/2050020932 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39f0119400 0x7f39f010fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:38.821 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.817+0000 7f39f7ee9640 1 --2- 172.21.15.114:0/2050020932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39f0107890 0x7f39f0116db0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:38.821 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.817+0000 7f39f7ee9640 1 --2- 172.21.15.114:0/2050020932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39f01074b0 0x7f39f0116850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:38.821 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.817+0000 7f39f7ee9640 1 -- 172.21.15.114:0/2050020932 >> 172.21.15.114:0/2050020932 conn(0x7f39f006ae30 msgr2=0x7f39f0119d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:38.821 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.818+0000 7f39f7ee9640 1 -- 172.21.15.114:0/2050020932 shutdown_connections 2024-09-13T18:32:38.823 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:38.821+0000 7f39f7ee9640 1 -- 172.21.15.114:0/2050020932 wait complete. 2024-09-13T18:32:40.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:39 smithi114 ceph-mon[30897]: pgmap v316: 129 pgs: 129 active+clean; 20 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 28 MiB/s wr, 631 op/s 2024-09-13T18:32:40.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:39 smithi134 ceph-mon[34283]: pgmap v316: 129 pgs: 129 active+clean; 20 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 28 MiB/s wr, 631 op/s 2024-09-13T18:32:40.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:39 smithi204 ceph-mon[34777]: pgmap v316: 129 pgs: 129 active+clean; 20 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 28 MiB/s wr, 631 op/s 2024-09-13T18:32:42.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:41 smithi114 ceph-mon[30897]: pgmap v317: 129 pgs: 129 active+clean; 20 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 35 MiB/s wr, 797 op/s 2024-09-13T18:32:42.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:41 smithi134 ceph-mon[34283]: pgmap v317: 129 pgs: 129 active+clean; 20 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 35 MiB/s wr, 797 op/s 2024-09-13T18:32:42.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:41 smithi204 ceph-mon[34777]: pgmap v317: 129 pgs: 129 active+clean; 20 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 35 MiB/s wr, 797 op/s 2024-09-13T18:32:44.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:43 smithi114 ceph-mon[30897]: pgmap v318: 129 pgs: 129 active+clean; 21 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 30 MiB/s wr, 674 op/s 2024-09-13T18:32:44.246 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:43 smithi204 ceph-mon[34777]: pgmap v318: 129 pgs: 129 active+clean; 21 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 30 MiB/s wr, 674 op/s 2024-09-13T18:32:44.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:43 smithi134 ceph-mon[34283]: pgmap v318: 129 pgs: 129 active+clean; 21 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 30 MiB/s wr, 674 op/s 2024-09-13T18:32:46.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:45 smithi134 ceph-mon[34283]: pgmap v319: 129 pgs: 129 active+clean; 21 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 28 MiB/s wr, 640 op/s 2024-09-13T18:32:46.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:45 smithi204 ceph-mon[34777]: pgmap v319: 129 pgs: 129 active+clean; 21 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 28 MiB/s wr, 640 op/s 2024-09-13T18:32:46.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:45 smithi114 ceph-mon[30897]: pgmap v319: 129 pgs: 129 active+clean; 21 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 28 MiB/s wr, 640 op/s 2024-09-13T18:32:47.230 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:46 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:32:47.234 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:46 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:32:47.282 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:46 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:32:48.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:47 smithi134 ceph-mon[34283]: pgmap v320: 129 pgs: 129 active+clean; 21 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 34 MiB/s wr, 771 op/s 2024-09-13T18:32:48.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:47 smithi204 ceph-mon[34777]: pgmap v320: 129 pgs: 129 active+clean; 21 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 34 MiB/s wr, 771 op/s 2024-09-13T18:32:48.472 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:47 smithi114 ceph-mon[30897]: pgmap v320: 129 pgs: 129 active+clean; 21 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 34 MiB/s wr, 771 op/s 2024-09-13T18:32:50.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:49 smithi134 ceph-mon[34283]: pgmap v321: 129 pgs: 129 active+clean; 21 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 27 MiB/s wr, 614 op/s 2024-09-13T18:32:50.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:49 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:32:50.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:49 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:32:50.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:49 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:32:50.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:49 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:32:50.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:49 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:32:50.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:49 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:32:50.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:49 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:50.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:49 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:50.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:49 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:50.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:49 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:50.370 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:49 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:50.371 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:49 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:50.371 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:49 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:50.371 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:49 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:50.371 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:49 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:50.371 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:49 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:50.371 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:49 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:50.371 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:49 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:50.371 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:49 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:50.371 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:49 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:50.371 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:49 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:50.371 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:49 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:50.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:49 smithi204 ceph-mon[34777]: pgmap v321: 129 pgs: 129 active+clean; 21 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 27 MiB/s wr, 614 op/s 2024-09-13T18:32:50.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:49 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:32:50.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:49 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:32:50.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:49 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:32:50.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:49 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:32:50.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:49 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:32:50.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:49 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:32:50.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:49 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:50.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:49 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:50.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:49 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:50.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:49 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:50.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:49 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:50.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:49 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:50.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:49 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:50.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:49 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:50.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:49 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:50.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:49 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:50.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:49 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:50.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:49 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:50.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:49 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:50.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:49 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:50.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:49 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:50.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:49 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:50.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:49 smithi114 ceph-mon[30897]: pgmap v321: 129 pgs: 129 active+clean; 21 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 27 MiB/s wr, 614 op/s 2024-09-13T18:32:50.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:49 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:32:50.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:49 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:32:50.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:49 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:32:50.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:49 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:32:50.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:49 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:32:50.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:49 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:32:50.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:49 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:50.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:49 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:50.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:49 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:50.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:49 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:50.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:49 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:50.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:49 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:50.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:49 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:50.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:49 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:50.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:49 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:50.472 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:49 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:50.472 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:49 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:50.472 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:49 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:50.472 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:49 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:50.472 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:49 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:50.472 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:49 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:50.472 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:49 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:51.372 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:50 smithi134 ceph-mon[34283]: Adjusting osd_memory_target on smithi134 to 176.0M 2024-09-13T18:32:51.373 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:50 smithi134 ceph-mon[34283]: Unable to set osd_memory_target on smithi134 to 184585318: error parsing value: Value '184585318' is below minimum 939524096 2024-09-13T18:32:51.373 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:50 smithi134 ceph-mon[34283]: Adjusting osd_memory_target on smithi204 to 176.0M 2024-09-13T18:32:51.373 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:50 smithi134 ceph-mon[34283]: Unable to set osd_memory_target on smithi204 to 184552345: error parsing value: Value '184552345' is below minimum 939524096 2024-09-13T18:32:51.373 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:50 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:51.373 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:50 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:51.373 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:50 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:51.373 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:50 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:51.373 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:50 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:51.373 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:50 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:51.373 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:50 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:51.373 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:50 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:51.374 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:50 smithi134 ceph-mon[34283]: Adjusting osd_memory_target on smithi114 to 176.0M 2024-09-13T18:32:51.374 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:50 smithi134 ceph-mon[34283]: Unable to set osd_memory_target on smithi114 to 184563814: error parsing value: Value '184563814' is below minimum 939524096 2024-09-13T18:32:51.374 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:50 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:32:51.374 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:50 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:32:51.374 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:50 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:32:51.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:50 smithi204 ceph-mon[34777]: Adjusting osd_memory_target on smithi134 to 176.0M 2024-09-13T18:32:51.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:50 smithi204 ceph-mon[34777]: Unable to set osd_memory_target on smithi134 to 184585318: error parsing value: Value '184585318' is below minimum 939524096 2024-09-13T18:32:51.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:50 smithi204 ceph-mon[34777]: Adjusting osd_memory_target on smithi204 to 176.0M 2024-09-13T18:32:51.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:50 smithi204 ceph-mon[34777]: Unable to set osd_memory_target on smithi204 to 184552345: error parsing value: Value '184552345' is below minimum 939524096 2024-09-13T18:32:51.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:50 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:51.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:50 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:51.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:50 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:51.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:50 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:51.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:50 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:51.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:50 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:51.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:50 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:51.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:50 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:51.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:50 smithi204 ceph-mon[34777]: Adjusting osd_memory_target on smithi114 to 176.0M 2024-09-13T18:32:51.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:50 smithi204 ceph-mon[34777]: Unable to set osd_memory_target on smithi114 to 184563814: error parsing value: Value '184563814' is below minimum 939524096 2024-09-13T18:32:51.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:50 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:32:51.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:50 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:32:51.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:50 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:32:51.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:50 smithi114 ceph-mon[30897]: Adjusting osd_memory_target on smithi134 to 176.0M 2024-09-13T18:32:51.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:50 smithi114 ceph-mon[30897]: Unable to set osd_memory_target on smithi134 to 184585318: error parsing value: Value '184585318' is below minimum 939524096 2024-09-13T18:32:51.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:50 smithi114 ceph-mon[30897]: Adjusting osd_memory_target on smithi204 to 176.0M 2024-09-13T18:32:51.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:50 smithi114 ceph-mon[30897]: Unable to set osd_memory_target on smithi204 to 184552345: error parsing value: Value '184552345' is below minimum 939524096 2024-09-13T18:32:51.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:50 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:51.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:50 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:51.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:50 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:51.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:50 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:51.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:50 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:51.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:50 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:51.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:50 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:51.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:50 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-13T18:32:51.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:50 smithi114 ceph-mon[30897]: Adjusting osd_memory_target on smithi114 to 176.0M 2024-09-13T18:32:51.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:50 smithi114 ceph-mon[30897]: Unable to set osd_memory_target on smithi114 to 184563814: error parsing value: Value '184563814' is below minimum 939524096 2024-09-13T18:32:51.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:50 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:32:51.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:50 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:32:51.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:50 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:32:52.042 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.030+0000 7f9d041e9640 1 -- 172.21.15.114:0/791692320 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9cfc119400 msgr2=0x7f9cfc11b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:52.042 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.030+0000 7f9d041e9640 1 --2- 172.21.15.114:0/791692320 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9cfc119400 0x7f9cfc11b7f0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8009900 tx=0x7f9cf802a440 comp rx=0 tx=0).stop 2024-09-13T18:32:52.042 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.030+0000 7f9d041e9640 1 -- 172.21.15.114:0/791692320 shutdown_connections 2024-09-13T18:32:52.042 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.030+0000 7f9d041e9640 1 --2- 172.21.15.114:0/791692320 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9cfc119400 0x7f9cfc11b7f0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:52.042 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.030+0000 7f9d041e9640 1 --2- 172.21.15.114:0/791692320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cfc107890 0x7f9cfc107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:52.042 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.030+0000 7f9d041e9640 1 --2- 172.21.15.114:0/791692320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9cfc1074b0 0x7f9cfc10e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:52.042 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.030+0000 7f9d041e9640 1 -- 172.21.15.114:0/791692320 >> 172.21.15.114:0/791692320 conn(0x7f9cfc06af50 msgr2=0x7f9cfc06b360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:52.042 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.032+0000 7f9d041e9640 1 -- 172.21.15.114:0/791692320 shutdown_connections 2024-09-13T18:32:52.042 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.032+0000 7f9d041e9640 1 -- 172.21.15.114:0/791692320 wait complete. 2024-09-13T18:32:52.042 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.033+0000 7f9d041e9640 1 Processor -- start 2024-09-13T18:32:52.043 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.033+0000 7f9d041e9640 1 -- start start 2024-09-13T18:32:52.043 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.034+0000 7f9d041e9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cfc1074b0 0x7f9cfc116910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:52.043 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.034+0000 7f9d041e9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9cfc107890 0x7f9cfc116e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:52.043 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.034+0000 7f9d041e9640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9cfc119400 0x7f9cfc10fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:52.043 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.034+0000 7f9d041e9640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cfc110350 con 0x7f9cfc119400 2024-09-13T18:32:52.043 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.034+0000 7f9d041e9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cfc1104c0 con 0x7f9cfc1074b0 2024-09-13T18:32:52.043 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.034+0000 7f9d041e9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cfc110680 con 0x7f9cfc107890 2024-09-13T18:32:52.043 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.034+0000 7f9d01784640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9cfc107890 0x7f9cfc116e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:52.043 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.034+0000 7f9d01784640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9cfc107890 0x7f9cfc116e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:51342/0 (socket says 172.21.15.114:51342) 2024-09-13T18:32:52.043 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.034+0000 7f9d01784640 1 -- 172.21.15.114:0/303432086 learned_addr learned my addr 172.21.15.114:0/303432086 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.035+0000 7f9d02786640 1 --2- 172.21.15.114:0/303432086 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9cfc119400 0x7f9cfc10fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.035+0000 7f9d01f85640 1 --2- 172.21.15.114:0/303432086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cfc1074b0 0x7f9cfc116910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.036+0000 7f9d02786640 1 -- 172.21.15.114:0/303432086 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9cfc107890 msgr2=0x7f9cfc116e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.036+0000 7f9d02786640 1 --2- 172.21.15.114:0/303432086 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9cfc107890 0x7f9cfc116e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.036+0000 7f9d02786640 1 -- 172.21.15.114:0/303432086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cfc1074b0 msgr2=0x7f9cfc116910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.036+0000 7f9d02786640 1 --2- 172.21.15.114:0/303432086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cfc1074b0 0x7f9cfc116910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.036+0000 7f9d02786640 1 -- 172.21.15.114:0/303432086 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf8009560 con 0x7f9cfc119400 2024-09-13T18:32:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.037+0000 7f9d02786640 1 --2- 172.21.15.114:0/303432086 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9cfc119400 0x7f9cfc10fbc0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8009230 tx=0x7f9cf8003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.038+0000 7f9cf2ffd640 1 -- 172.21.15.114:0/303432086 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf803b460 con 0x7f9cfc119400 2024-09-13T18:32:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.038+0000 7f9d041e9640 1 -- 172.21.15.114:0/303432086 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cfc1144a0 con 0x7f9cfc119400 2024-09-13T18:32:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.038+0000 7f9d041e9640 1 -- 172.21.15.114:0/303432086 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cfc114920 con 0x7f9cfc119400 2024-09-13T18:32:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.038+0000 7f9cf2ffd640 1 -- 172.21.15.114:0/303432086 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9cf803ba50 con 0x7f9cfc119400 2024-09-13T18:32:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.038+0000 7f9cf2ffd640 1 -- 172.21.15.114:0/303432086 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf8031860 con 0x7f9cfc119400 2024-09-13T18:32:52.055 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.052+0000 7f9d06fe5740 1 Processor -- start 2024-09-13T18:32:52.055 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.052+0000 7f9d06fe5740 1 -- start start 2024-09-13T18:32:52.055 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.052+0000 7f9cf2ffd640 1 -- 172.21.15.114:0/303432086 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f9cf80319c0 con 0x7f9cfc119400 2024-09-13T18:32:52.059 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.053+0000 7f9cf2ffd640 1 --2- 172.21.15.114:0/303432086 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f9ce80a1410 0x7f9ce80a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:52.059 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.053+0000 7f9d06fe5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d2ff93a330 0x55d2ff93a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:52.059 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.053+0000 7f9d06fe5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d2ff93ac50 0x55d2ff944fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:52.059 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.053+0000 7f9d06fe5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2ff9454f0 0x55d2ff9478e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:52.059 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.053+0000 7f9d06fe5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2ff8a5c30 con 0x55d2ff93ac50 2024-09-13T18:32:52.059 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.053+0000 7f9d06fe5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2ff8a33c0 con 0x55d2ff9454f0 2024-09-13T18:32:52.059 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.053+0000 7f9d06fe5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2ff89ecb0 con 0x55d2ff93a330 2024-09-13T18:32:52.059 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.054+0000 7f9cf2ffd640 1 -- 172.21.15.114:0/303432086 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f9cf80ed740 con 0x7f9cfc119400 2024-09-13T18:32:52.059 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.054+0000 7f9d01784640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d2ff93ac50 0x55d2ff944fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:52.059 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.054+0000 7f9d01784640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d2ff93ac50 0x55d2ff944fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44258/0 (socket says 172.21.15.114:44258) 2024-09-13T18:32:52.059 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.054+0000 7f9d01784640 1 -- 172.21.15.114:0/359744494 learned_addr learned my addr 172.21.15.114:0/359744494 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:52.061 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.056+0000 7f9d01784640 1 -- 172.21.15.114:0/359744494 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d2ff93a330 msgr2=0x55d2ff93a710 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:32:52.061 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.056+0000 7f9d01784640 1 --2- 172.21.15.114:0/359744494 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d2ff93a330 0x55d2ff93a710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:52.061 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.056+0000 7f9d01784640 1 -- 172.21.15.114:0/359744494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2ff9454f0 msgr2=0x55d2ff9478e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:32:52.062 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.056+0000 7f9d01784640 1 --2- 172.21.15.114:0/359744494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2ff9454f0 0x55d2ff9478e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:52.062 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.056+0000 7f9d01784640 1 -- 172.21.15.114:0/359744494 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2ff845910 con 0x55d2ff93ac50 2024-09-13T18:32:52.062 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.058+0000 7f9d01784640 1 --2- 172.21.15.114:0/303432086 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f9ce80a1410 0x7f9ce80a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:52.068 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.060+0000 7f9d01784640 1 --2- 172.21.15.114:0/359744494 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d2ff93ac50 0x55d2ff944fb0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4013800 tx=0x7f9cf4013cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5deca5693f48ed7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:52.068 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.061+0000 7f9ce7fff640 1 -- 172.21.15.114:0/359744494 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf4023030 con 0x55d2ff93ac50 2024-09-13T18:32:52.068 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.061+0000 7f9ce7fff640 1 -- 172.21.15.114:0/359744494 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9cf400fde0 con 0x55d2ff93ac50 2024-09-13T18:32:52.068 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.061+0000 7f9ce7fff640 1 -- 172.21.15.114:0/359744494 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf401dc50 con 0x55d2ff93ac50 2024-09-13T18:32:52.068 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.063+0000 7f9d01784640 1 --2- 172.21.15.114:0/303432086 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f9ce80a1410 0x7f9ce80a38d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4023000 tx=0x7f9cf4020000 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:52.068 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.063+0000 7f9d06fe5740 1 -- 172.21.15.114:0/359744494 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d2ff93ac50 msgr2=0x55d2ff944fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:52.068 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.063+0000 7f9d06fe5740 1 --2- 172.21.15.114:0/359744494 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d2ff93ac50 0x55d2ff944fb0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4013800 tx=0x7f9cf4013cc0 comp rx=0 tx=0).stop 2024-09-13T18:32:52.071 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.067+0000 7f9d06fe5740 1 -- 172.21.15.114:0/359744494 shutdown_connections 2024-09-13T18:32:52.071 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.067+0000 7f9d06fe5740 1 --2- 172.21.15.114:0/359744494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2ff9454f0 0x55d2ff9478e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:52.071 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.068+0000 7f9d06fe5740 1 --2- 172.21.15.114:0/359744494 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d2ff93ac50 0x55d2ff944fb0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:52.071 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.068+0000 7f9d06fe5740 1 --2- 172.21.15.114:0/359744494 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d2ff93a330 0x55d2ff93a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:52.071 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.068+0000 7f9d06fe5740 1 -- 172.21.15.114:0/359744494 >> 172.21.15.114:0/359744494 conn(0x55d2ff9368c0 msgr2=0x55d2ff936ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.073+0000 7f9d06fe5740 1 -- 172.21.15.114:0/359744494 shutdown_connections 2024-09-13T18:32:52.077 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.073+0000 7f9d06fe5740 1 -- 172.21.15.114:0/359744494 wait complete. 2024-09-13T18:32:52.077 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.074+0000 7f9d06fe5740 1 -- start start 2024-09-13T18:32:52.077 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.074+0000 7f9d06fe5740 1 Processor -- start 2024-09-13T18:32:52.078 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.075+0000 7f9d06fe5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d2ff93a330 0x55d2ff944f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:52.079 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.075+0000 7f9d06fe5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2ff93ac50 0x55d2ff9421c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:52.079 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.075+0000 7f9d06fe5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d2ff9454f0 0x55d2ff942950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:52.079 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.075+0000 7f9d01784640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2ff93ac50 0x55d2ff9421c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:52.079 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.076+0000 7f9d01784640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2ff93ac50 0x55d2ff9421c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:49702/0 (socket says 172.21.15.114:49702) 2024-09-13T18:32:52.079 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.076+0000 7f9d01784640 1 -- 172.21.15.114:0/1162119104 learned_addr learned my addr 172.21.15.114:0/1162119104 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:52.079 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.076+0000 7f9d01f85640 1 --2- 172.21.15.114:0/1162119104 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d2ff93a330 0x55d2ff944f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:52.080 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.076+0000 7f9d02786640 1 --2- 172.21.15.114:0/1162119104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d2ff9454f0 0x55d2ff942950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:52.080 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.076+0000 7f9d06fe5740 1 -- 172.21.15.114:0/1162119104 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2ff85ce80 con 0x55d2ff93a330 2024-09-13T18:32:52.080 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.076+0000 7f9d06fe5740 1 -- 172.21.15.114:0/1162119104 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2ff85cb20 con 0x55d2ff93ac50 2024-09-13T18:32:52.080 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.076+0000 7f9d06fe5740 1 -- 172.21.15.114:0/1162119104 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2ff822260 con 0x55d2ff9454f0 2024-09-13T18:32:52.080 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.076+0000 7f9d06fe5740 10 client.0 _ll_register_callbacks cb 0x55d2ff857df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:32:52.083 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.080+0000 7f9d01784640 1 -- 172.21.15.114:0/1162119104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d2ff9454f0 msgr2=0x55d2ff942950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:52.083 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.080+0000 7f9d01784640 1 --2- 172.21.15.114:0/1162119104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d2ff9454f0 0x55d2ff942950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:52.083 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.080+0000 7f9d01784640 1 -- 172.21.15.114:0/1162119104 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d2ff93a330 msgr2=0x55d2ff944f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:52.083 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.080+0000 7f9d01784640 1 --2- 172.21.15.114:0/1162119104 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d2ff93a330 0x55d2ff944f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:52.083 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.080+0000 7f9d01784640 1 -- 172.21.15.114:0/1162119104 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cf4013460 con 0x55d2ff93ac50 2024-09-13T18:32:52.085 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.081+0000 7f9d01784640 1 --2- 172.21.15.114:0/1162119104 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2ff93ac50 0x55d2ff9421c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4009540 tx=0x7f9cf401d630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db5fa6ffdc1eb783 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:52.085 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.082+0000 7f9ce57fa640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:32:52.087 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.083+0000 7f9ce57fa640 1 -- 172.21.15.114:0/1162119104 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf4004e10 con 0x55d2ff93ac50 2024-09-13T18:32:52.087 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.084+0000 7f9ce57fa640 1 -- 172.21.15.114:0/1162119104 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f9cf4024070 con 0x55d2ff93ac50 2024-09-13T18:32:52.087 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.084+0000 7f9d06fe5740 1 -- 172.21.15.114:0/1162119104 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d2ff8c7680 con 0x55d2ff93ac50 2024-09-13T18:32:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.085+0000 7f9ce57fa640 1 -- 172.21.15.114:0/1162119104 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9cf4005040 con 0x55d2ff93ac50 2024-09-13T18:32:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.085+0000 7f9d06fe5740 10 client.27821 fetch_fsmap learned FSMap version 34 2024-09-13T18:32:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.085+0000 7f9d06fe5740 1 -- 172.21.15.114:0/1162119104 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55d2ff89d330 con 0x55d2ff93ac50 2024-09-13T18:32:52.092 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.089+0000 7f9ce57fa640 1 -- 172.21.15.114:0/1162119104 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9cf4005630 con 0x55d2ff93ac50 2024-09-13T18:32:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.089+0000 7f9ce57fa640 1 -- 172.21.15.114:0/1162119104 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9cf402a720 con 0x55d2ff93ac50 2024-09-13T18:32:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.089+0000 7f9d06fe5740 10 client.27821 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:32:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.089+0000 7f9d06fe5740 10 client.27821 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:32:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.089+0000 7f9d06fe5740 20 client.27821 populate_metadata read hostname 'smithi114' 2024-09-13T18:32:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.089+0000 7f9d06fe5740 1 --2- 172.21.15.114:0/1162119104 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55d2ff954e50 0x55d2ff957270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.089+0000 7f9d06fe5740 4 client.27821 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:32:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.089+0000 7f9d06fe5740 1 -- 172.21.15.114:0/1162119104 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d2ff8c7680 con 0x55d2ff954e50 2024-09-13T18:32:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.090+0000 7f9d01f85640 1 --2- 172.21.15.114:0/1162119104 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55d2ff954e50 0x55d2ff957270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.090+0000 7f9ce57fa640 10 client.27821 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:32:52.095 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.092+0000 7f9d01f85640 1 --2- 172.21.15.114:0/1162119104 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55d2ff954e50 0x55d2ff957270 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50036e4be2701ea4 server_cookie=8177debfc3a4b233 in_seq=0 out_seq=0 2024-09-13T18:32:52.097 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.094+0000 7f9ce57fa640 1 -- 172.21.15.114:0/1162119104 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55d2ff8c7680 con 0x55d2ff954e50 2024-09-13T18:32:52.098 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.094+0000 7f9ce57fa640 10 client.27821 handle_command_reply: tid=0 2024-09-13T18:32:52.098 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.094+0000 7f9d06fe5740 1 client.27821 shutdown 2024-09-13T18:32:52.098 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.094+0000 7f9d06fe5740 1 -- 172.21.15.114:0/1162119104 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2ff93ac50 msgr2=0x55d2ff9421c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:52.098 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.094+0000 7f9d06fe5740 1 --2- 172.21.15.114:0/1162119104 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2ff93ac50 0x55d2ff9421c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4009540 tx=0x7f9cf401d630 comp rx=0 tx=0).stop 2024-09-13T18:32:52.098 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.094+0000 7f9d06fe5740 1 -- 172.21.15.114:0/1162119104 shutdown_connections 2024-09-13T18:32:52.098 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.094+0000 7f9d06fe5740 1 --2- 172.21.15.114:0/1162119104 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55d2ff954e50 0x55d2ff957270 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:52.098 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.094+0000 7f9d06fe5740 1 --2- 172.21.15.114:0/1162119104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d2ff9454f0 0x55d2ff942950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:52.098 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.094+0000 7f9d06fe5740 1 --2- 172.21.15.114:0/1162119104 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2ff93ac50 0x55d2ff9421c0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:52.098 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.094+0000 7f9d06fe5740 1 --2- 172.21.15.114:0/1162119104 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d2ff93a330 0x55d2ff944f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:52.098 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.094+0000 7f9d06fe5740 1 -- 172.21.15.114:0/1162119104 >> 172.21.15.114:0/1162119104 conn(0x55d2ff9368c0 msgr2=0x55d2ff935b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:52.098 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.094+0000 7f9ce57fa640 0 client.27821 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:32:52.099 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.096+0000 7f9d06fe5740 1 -- 172.21.15.114:0/1162119104 shutdown_connections 2024-09-13T18:32:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.098+0000 7f9d06fe5740 1 -- 172.21.15.114:0/1162119104 wait complete. 2024-09-13T18:32:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.101+0000 7f9d06fe5740 20 client.27821 trim_cache size 0 max 16384 2024-09-13T18:32:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.135+0000 7f9d06fe5740 1 Processor -- start 2024-09-13T18:32:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.138+0000 7f9d06fe5740 1 -- start start 2024-09-13T18:32:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.138+0000 7f9d06fe5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d2ff93a330 0x55d2ff936300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.138+0000 7f9d06fe5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d2ff93ac50 0x55d2ff931a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.138+0000 7f9d06fe5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2ff9454f0 0x55d2ff931f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.138+0000 7f9d06fe5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2ff932490 con 0x55d2ff93a330 2024-09-13T18:32:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.138+0000 7f9d06fe5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2ff942f50 con 0x55d2ff9454f0 2024-09-13T18:32:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.138+0000 7f9d06fe5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2ff93b2a0 con 0x55d2ff93ac50 2024-09-13T18:32:52.143 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.139+0000 7f9d02786640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2ff9454f0 0x55d2ff931f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:52.143 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.139+0000 7f9d02786640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2ff9454f0 0x55d2ff931f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:49704/0 (socket says 172.21.15.114:49704) 2024-09-13T18:32:52.143 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.139+0000 7f9d02786640 1 -- 172.21.15.114:0/1016385539 learned_addr learned my addr 172.21.15.114:0/1016385539 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.141+0000 7f9d02786640 1 -- 172.21.15.114:0/1016385539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d2ff93ac50 msgr2=0x55d2ff931a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.141+0000 7f9d02786640 1 --2- 172.21.15.114:0/1016385539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d2ff93ac50 0x55d2ff931a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.142+0000 7f9d02786640 1 -- 172.21.15.114:0/1016385539 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d2ff93a330 msgr2=0x55d2ff936300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.142+0000 7f9d02786640 1 --2- 172.21.15.114:0/1016385539 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d2ff93a330 0x55d2ff936300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.142+0000 7f9d02786640 1 -- 172.21.15.114:0/1016385539 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2ff83f300 con 0x55d2ff9454f0 2024-09-13T18:32:52.157 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.152+0000 7f9d02786640 1 --2- 172.21.15.114:0/1016385539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2ff9454f0 0x55d2ff931f50 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f9cf80ba3e0 tx=0x7f9cf8009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f7c6b8264fb8f3b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:52.165 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.159+0000 7f9cf0ff9640 1 -- 172.21.15.114:0/1016385539 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf80bab50 con 0x55d2ff9454f0 2024-09-13T18:32:52.165 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.159+0000 7f9cf0ff9640 1 -- 172.21.15.114:0/1016385539 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9cf80ba450 con 0x55d2ff9454f0 2024-09-13T18:32:52.165 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.159+0000 7f9cf0ff9640 1 -- 172.21.15.114:0/1016385539 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf80ead10 con 0x55d2ff9454f0 2024-09-13T18:32:52.165 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.160+0000 7f9d06fe5740 1 -- 172.21.15.114:0/1016385539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2ff9454f0 msgr2=0x55d2ff931f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:52.165 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.160+0000 7f9d06fe5740 1 --2- 172.21.15.114:0/1016385539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2ff9454f0 0x55d2ff931f50 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f9cf80ba3e0 tx=0x7f9cf8009260 comp rx=0 tx=0).stop 2024-09-13T18:32:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.164+0000 7f9d06fe5740 1 -- 172.21.15.114:0/1016385539 shutdown_connections 2024-09-13T18:32:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.164+0000 7f9d06fe5740 1 --2- 172.21.15.114:0/1016385539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2ff9454f0 0x55d2ff931f50 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.164+0000 7f9d06fe5740 1 --2- 172.21.15.114:0/1016385539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d2ff93ac50 0x55d2ff931a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.164+0000 7f9d06fe5740 1 --2- 172.21.15.114:0/1016385539 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d2ff93a330 0x55d2ff936300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.164+0000 7f9d06fe5740 1 -- 172.21.15.114:0/1016385539 >> 172.21.15.114:0/1016385539 conn(0x55d2ff9368c0 msgr2=0x55d2ff93d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.169+0000 7f9d06fe5740 1 -- 172.21.15.114:0/1016385539 shutdown_connections 2024-09-13T18:32:52.175 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.171+0000 7f9d06fe5740 1 -- 172.21.15.114:0/1016385539 wait complete. 2024-09-13T18:32:52.175 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.172+0000 7f9d06fe5740 1 -- start start 2024-09-13T18:32:52.175 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.172+0000 7f9d06fe5740 1 Processor -- start 2024-09-13T18:32:52.175 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.172+0000 7f9d06fe5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d2ff93a330 0x55d2ff948b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:52.176 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.172+0000 7f9d06fe5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2ff93ac50 0x55d2ff9490c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:52.176 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.172+0000 7f9d06fe5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d2ff9454f0 0x55d2ff949600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:52.176 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.172+0000 7f9d06fe5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2ff9ab310 con 0x55d2ff93a330 2024-09-13T18:32:52.176 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.172+0000 7f9d06fe5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2ff9ab450 con 0x55d2ff93ac50 2024-09-13T18:32:52.176 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.172+0000 7f9d06fe5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2ff9ab590 con 0x55d2ff9454f0 2024-09-13T18:32:52.176 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.172+0000 7f9d06fe5740 10 client.0 _ll_register_callbacks cb 0x7f9cf4009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:32:52.176 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.173+0000 7f9d02786640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d2ff9454f0 0x55d2ff949600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:52.177 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.173+0000 7f9d02786640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d2ff9454f0 0x55d2ff949600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:51368/0 (socket says 172.21.15.114:51368) 2024-09-13T18:32:52.177 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.173+0000 7f9d02786640 1 -- 172.21.15.114:0/2393186466 learned_addr learned my addr 172.21.15.114:0/2393186466 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:52.177 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.174+0000 7f9d01f85640 1 --2- 172.21.15.114:0/2393186466 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d2ff93a330 0x55d2ff948b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:52.177 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.174+0000 7f9d01784640 1 --2- 172.21.15.114:0/2393186466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2ff93ac50 0x55d2ff9490c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:52.178 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.174+0000 7f9d02786640 1 -- 172.21.15.114:0/2393186466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2ff93ac50 msgr2=0x55d2ff9490c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:52.178 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.174+0000 7f9d02786640 1 --2- 172.21.15.114:0/2393186466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2ff93ac50 0x55d2ff9490c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:52.178 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.174+0000 7f9d02786640 1 -- 172.21.15.114:0/2393186466 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d2ff93a330 msgr2=0x55d2ff948b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:52.178 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.174+0000 7f9d02786640 1 --2- 172.21.15.114:0/2393186466 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d2ff93a330 0x55d2ff948b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:52.178 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.174+0000 7f9d02786640 1 -- 172.21.15.114:0/2393186466 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cfc114920 con 0x55d2ff9454f0 2024-09-13T18:32:52.178 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.175+0000 7f9d02786640 1 --2- 172.21.15.114:0/2393186466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d2ff9454f0 0x55d2ff949600 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9cf803b040 tx=0x7f9cf8005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe3cbc3811e328c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:52.181 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.176+0000 7f9ce57fa640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:32:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.179+0000 7f9ce57fa640 1 -- 172.21.15.114:0/2393186466 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf80c6070 con 0x55d2ff9454f0 2024-09-13T18:32:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.180+0000 7f9ce57fa640 1 -- 172.21.15.114:0/2393186466 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f9cf80bab50 con 0x55d2ff9454f0 2024-09-13T18:32:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.180+0000 7f9d06fe5740 1 -- 172.21.15.114:0/2393186466 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9cf400fde0 con 0x55d2ff9454f0 2024-09-13T18:32:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.184+0000 7f9ce57fa640 1 -- 172.21.15.114:0/2393186466 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9cf80c28d0 con 0x55d2ff9454f0 2024-09-13T18:32:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.185+0000 7f9d06fe5740 10 client.27757 fetch_fsmap learned FSMap version 34 2024-09-13T18:32:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.185+0000 7f9d06fe5740 1 -- 172.21.15.114:0/2393186466 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55d2ff8648e0 con 0x55d2ff9454f0 2024-09-13T18:32:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.188+0000 7f9ce57fa640 1 -- 172.21.15.114:0/2393186466 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9cf400fde0 con 0x55d2ff9454f0 2024-09-13T18:32:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.188+0000 7f9ce57fa640 1 -- 172.21.15.114:0/2393186466 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9cf80ba6b0 con 0x55d2ff9454f0 2024-09-13T18:32:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.190+0000 7f9d06fe5740 10 client.27757 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:32:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.190+0000 7f9d06fe5740 10 client.27757 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:32:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.190+0000 7f9d06fe5740 20 client.27757 populate_metadata read hostname 'smithi114' 2024-09-13T18:32:52.194 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.190+0000 7f9d06fe5740 1 --2- 172.21.15.114:0/2393186466 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55d2ff9ac7d0 0x55d2ff9aebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:52.194 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.190+0000 7f9d06fe5740 4 client.27757 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T18:32:52.194 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.190+0000 7f9d06fe5740 1 -- 172.21.15.114:0/2393186466 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d2ff8925d0 con 0x55d2ff9ac7d0 2024-09-13T18:32:52.197 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.191+0000 7f9d01f85640 1 --2- 172.21.15.114:0/2393186466 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55d2ff9ac7d0 0x55d2ff9aebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:52.210 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.206+0000 7f9d01f85640 1 --2- 172.21.15.114:0/2393186466 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55d2ff9ac7d0 0x55d2ff9aebc0 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5526ac3967509310 server_cookie=77f86926cba2731 in_seq=0 out_seq=0 2024-09-13T18:32:52.210 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.206+0000 7f9ce57fa640 10 client.27757 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:32:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.226+0000 7f9ce57fa640 1 -- 172.21.15.114:0/2393186466 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d2ff8925d0 con 0x55d2ff9ac7d0 2024-09-13T18:32:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.226+0000 7f9ce57fa640 10 client.27757 handle_command_reply: tid=0 2024-09-13T18:32:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.226+0000 7f9d06fe5740 1 client.27757 shutdown 2024-09-13T18:32:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.230+0000 7f9d06fe5740 1 -- 172.21.15.114:0/2393186466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d2ff9454f0 msgr2=0x55d2ff949600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.230+0000 7f9d06fe5740 1 --2- 172.21.15.114:0/2393186466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d2ff9454f0 0x55d2ff949600 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9cf803b040 tx=0x7f9cf8005c90 comp rx=0 tx=0).stop 2024-09-13T18:32:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.234+0000 7f9d06fe5740 1 -- 172.21.15.114:0/2393186466 shutdown_connections 2024-09-13T18:32:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.234+0000 7f9d06fe5740 1 --2- 172.21.15.114:0/2393186466 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55d2ff9ac7d0 0x55d2ff9aebc0 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.234+0000 7f9d06fe5740 1 --2- 172.21.15.114:0/2393186466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d2ff9454f0 0x55d2ff949600 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.234+0000 7f9d06fe5740 1 --2- 172.21.15.114:0/2393186466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2ff93ac50 0x55d2ff9490c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.234+0000 7f9d06fe5740 1 --2- 172.21.15.114:0/2393186466 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d2ff93a330 0x55d2ff948b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.234+0000 7f9d06fe5740 1 -- 172.21.15.114:0/2393186466 >> 172.21.15.114:0/2393186466 conn(0x55d2ff9368c0 msgr2=0x55d2ff93d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.234+0000 7f9ce57fa640 0 client.27757 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:32:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.234+0000 7f9d06fe5740 1 -- 172.21.15.114:0/2393186466 shutdown_connections 2024-09-13T18:32:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.234+0000 7f9d06fe5740 1 -- 172.21.15.114:0/2393186466 wait complete. 2024-09-13T18:32:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.236+0000 7f9d06fe5740 20 client.27757 trim_cache size 0 max 16384 2024-09-13T18:32:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-13T18:32:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-13T18:32:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-13T18:32:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-13T18:32:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.240+0000 7f9d041e9640 1 -- 172.21.15.114:0/303432086 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f9ce80a1410 msgr2=0x7f9ce80a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.240+0000 7f9d041e9640 1 --2- 172.21.15.114:0/303432086 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f9ce80a1410 0x7f9ce80a38d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4023000 tx=0x7f9cf4020000 comp rx=0 tx=0).stop 2024-09-13T18:32:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.240+0000 7f9d041e9640 1 -- 172.21.15.114:0/303432086 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9cfc119400 msgr2=0x7f9cfc10fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.240+0000 7f9d041e9640 1 --2- 172.21.15.114:0/303432086 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9cfc119400 0x7f9cfc10fbc0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8009230 tx=0x7f9cf8003b90 comp rx=0 tx=0).stop 2024-09-13T18:32:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.244+0000 7f9d041e9640 1 -- 172.21.15.114:0/303432086 shutdown_connections 2024-09-13T18:32:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.244+0000 7f9d041e9640 1 --2- 172.21.15.114:0/303432086 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f9ce80a1410 0x7f9ce80a38d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.244+0000 7f9d041e9640 1 --2- 172.21.15.114:0/303432086 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9cfc119400 0x7f9cfc10fbc0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.244+0000 7f9d041e9640 1 --2- 172.21.15.114:0/303432086 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9cfc107890 0x7f9cfc116e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.244+0000 7f9d041e9640 1 --2- 172.21.15.114:0/303432086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cfc1074b0 0x7f9cfc116910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.244+0000 7f9d041e9640 1 -- 172.21.15.114:0/303432086 >> 172.21.15.114:0/303432086 conn(0x7f9cfc06af50 msgr2=0x7f9cfc119ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.246+0000 7f9d041e9640 1 -- 172.21.15.114:0/303432086 shutdown_connections 2024-09-13T18:32:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:32:52.246+0000 7f9d041e9640 1 -- 172.21.15.114:0/303432086 wait complete. 2024-09-13T18:32:52.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:52 smithi204 ceph-mon[34777]: pgmap v322: 129 pgs: 129 active+clean; 21 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 35 MiB/s wr, 774 op/s 2024-09-13T18:32:52.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:52 smithi114 ceph-mon[30897]: pgmap v322: 129 pgs: 129 active+clean; 21 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 35 MiB/s wr, 774 op/s 2024-09-13T18:32:52.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:52 smithi134 ceph-mon[34283]: pgmap v322: 129 pgs: 129 active+clean; 21 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 35 MiB/s wr, 774 op/s 2024-09-13T18:32:54.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:54 smithi204 ceph-mon[34777]: pgmap v323: 129 pgs: 129 active+clean; 21 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 30 MiB/s wr, 650 op/s 2024-09-13T18:32:54.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:54 smithi114 ceph-mon[30897]: pgmap v323: 129 pgs: 129 active+clean; 21 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 30 MiB/s wr, 650 op/s 2024-09-13T18:32:54.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:54 smithi134 ceph-mon[34283]: pgmap v323: 129 pgs: 129 active+clean; 21 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 30 MiB/s wr, 650 op/s 2024-09-13T18:32:56.191 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T18:32:56.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:56 smithi204 ceph-mon[34777]: pgmap v324: 129 pgs: 129 active+clean; 21 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 28 MiB/s wr, 614 op/s 2024-09-13T18:32:56.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.441+0000 7f0f2d33a640 1 -- 172.21.15.114:0/175685510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f28107890 msgr2=0x7f0f28107d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:56.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.441+0000 7f0f2d33a640 1 --2- 172.21.15.114:0/175685510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f28107890 0x7f0f28107d10 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0f20009900 tx=0x7f0f2002a420 comp rx=0 tx=0).stop 2024-09-13T18:32:56.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.447+0000 7f0f2d33a640 1 -- 172.21.15.114:0/175685510 shutdown_connections 2024-09-13T18:32:56.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.447+0000 7f0f2d33a640 1 --2- 172.21.15.114:0/175685510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f28119400 0x7f0f2811b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:56.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.447+0000 7f0f2d33a640 1 --2- 172.21.15.114:0/175685510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f28107890 0x7f0f28107d10 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:56.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.447+0000 7f0f2d33a640 1 --2- 172.21.15.114:0/175685510 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f281074b0 0x7f0f2810e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:56.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.447+0000 7f0f2d33a640 1 -- 172.21.15.114:0/175685510 >> 172.21.15.114:0/175685510 conn(0x7f0f2806aec0 msgr2=0x7f0f2806b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:56.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.451+0000 7f0f2d33a640 1 -- 172.21.15.114:0/175685510 shutdown_connections 2024-09-13T18:32:56.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.451+0000 7f0f2d33a640 1 -- 172.21.15.114:0/175685510 wait complete. 2024-09-13T18:32:56.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.452+0000 7f0f2d33a640 1 Processor -- start 2024-09-13T18:32:56.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.453+0000 7f0f2d33a640 1 -- start start 2024-09-13T18:32:56.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.454+0000 7f0f2d33a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f281074b0 0x7f0f28116910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:56.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.454+0000 7f0f2d33a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f28107890 0x7f0f28116e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:56.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.454+0000 7f0f2d33a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f28119400 0x7f0f2810fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:56.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.454+0000 7f0f2d33a640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f28110290 con 0x7f0f28119400 2024-09-13T18:32:56.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.454+0000 7f0f2d33a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f28110400 con 0x7f0f281074b0 2024-09-13T18:32:56.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.454+0000 7f0f2d33a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f281105c0 con 0x7f0f28107890 2024-09-13T18:32:56.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.456+0000 7f0f27fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f281074b0 0x7f0f28116910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:56.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.456+0000 7f0f27fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f281074b0 0x7f0f28116910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:47548/0 (socket says 172.21.15.114:47548) 2024-09-13T18:32:56.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.456+0000 7f0f27fff640 1 -- 172.21.15.114:0/1835919856 learned_addr learned my addr 172.21.15.114:0/1835919856 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:56.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.456+0000 7f0f2cb39640 1 --2- 172.21.15.114:0/1835919856 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f28119400 0x7f0f2810fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:56.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.457+0000 7f0f27fff640 1 -- 172.21.15.114:0/1835919856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f28107890 msgr2=0x7f0f28116e50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:32:56.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.457+0000 7f0f27fff640 1 --2- 172.21.15.114:0/1835919856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f28107890 0x7f0f28116e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:56.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.457+0000 7f0f27fff640 1 -- 172.21.15.114:0/1835919856 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f28119400 msgr2=0x7f0f2810fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:56.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.457+0000 7f0f27fff640 1 --2- 172.21.15.114:0/1835919856 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f28119400 0x7f0f2810fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:56.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.457+0000 7f0f27fff640 1 -- 172.21.15.114:0/1835919856 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f20009560 con 0x7f0f281074b0 2024-09-13T18:32:56.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.458+0000 7f0f27fff640 1 --2- 172.21.15.114:0/1835919856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f281074b0 0x7f0f28116910 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0f1000d950 tx=0x7f0f1000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:56.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.459+0000 7f0f257fa640 1 -- 172.21.15.114:0/1835919856 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f10009c30 con 0x7f0f281074b0 2024-09-13T18:32:56.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.459+0000 7f0f2d33a640 1 -- 172.21.15.114:0/1835919856 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f28114430 con 0x7f0f281074b0 2024-09-13T18:32:56.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.459+0000 7f0f2d33a640 1 -- 172.21.15.114:0/1835919856 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f28114940 con 0x7f0f281074b0 2024-09-13T18:32:56.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.459+0000 7f0f257fa640 1 -- 172.21.15.114:0/1835919856 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0f10009d90 con 0x7f0f281074b0 2024-09-13T18:32:56.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.459+0000 7f0f257fa640 1 -- 172.21.15.114:0/1835919856 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1000f620 con 0x7f0f281074b0 2024-09-13T18:32:56.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.465+0000 7f0f257fa640 1 -- 172.21.15.114:0/1835919856 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f0f1000f780 con 0x7f0f281074b0 2024-09-13T18:32:56.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.466+0000 7f0f257fa640 1 --2- 172.21.15.114:0/1835919856 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0f140a1400 0x7f0f140a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:56.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.466+0000 7f0f257fa640 1 -- 172.21.15.114:0/1835919856 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f0f100ccc80 con 0x7f0f281074b0 2024-09-13T18:32:56.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:56 smithi114 ceph-mon[30897]: pgmap v324: 129 pgs: 129 active+clean; 21 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 28 MiB/s wr, 614 op/s 2024-09-13T18:32:56.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.467+0000 7f0f277fe640 1 --2- 172.21.15.114:0/1835919856 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0f140a1400 0x7f0f140a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:56.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.469+0000 7f0f312c4740 1 Processor -- start 2024-09-13T18:32:56.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.469+0000 7f0f312c4740 1 -- start start 2024-09-13T18:32:56.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.470+0000 7f0f312c4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f93d2d4030 0x55f93d2d4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:56.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.470+0000 7f0f312c4740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f93d2d4950 0x55f93d2decb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:56.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.470+0000 7f0f312c4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f93d2df1f0 0x55f93d2e15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:56.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.470+0000 7f0f312c4740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f93d23ef20 con 0x55f93d2d4950 2024-09-13T18:32:56.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.470+0000 7f0f312c4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f93d23c6b0 con 0x55f93d2d4030 2024-09-13T18:32:56.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.470+0000 7f0f312c4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f93d237fa0 con 0x55f93d2df1f0 2024-09-13T18:32:56.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.470+0000 7f0f277fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f93d2d4950 0x55f93d2decb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:56.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.470+0000 7f0f277fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f93d2d4950 0x55f93d2decb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41490/0 (socket says 172.21.15.114:41490) 2024-09-13T18:32:56.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.470+0000 7f0f277fe640 1 -- 172.21.15.114:0/2805211381 learned_addr learned my addr 172.21.15.114:0/2805211381 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:56.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.471+0000 7f0f277fe640 1 -- 172.21.15.114:0/2805211381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f93d2df1f0 msgr2=0x55f93d2e15e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:32:56.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.471+0000 7f0f277fe640 1 --2- 172.21.15.114:0/2805211381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f93d2df1f0 0x55f93d2e15e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:56.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.471+0000 7f0f277fe640 1 -- 172.21.15.114:0/2805211381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f93d2d4030 msgr2=0x55f93d2d4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:56.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.471+0000 7f0f27fff640 1 --2- 172.21.15.114:0/2805211381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f93d2d4030 0x55f93d2d4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:56.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.471+0000 7f0f277fe640 1 --2- 172.21.15.114:0/2805211381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f93d2d4030 0x55f93d2d4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:56.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.471+0000 7f0f277fe640 1 -- 172.21.15.114:0/2805211381 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f93d1f2840 con 0x55f93d2d4950 2024-09-13T18:32:56.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.472+0000 7f0f277fe640 1 --2- 172.21.15.114:0/2805211381 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f93d2d4950 0x55f93d2decb0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f0f200040c0 tx=0x7f0f200040f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=768478c20c17efe4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:56.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.472+0000 7f0f1e7fc640 1 -- 172.21.15.114:0/2805211381 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f200035f0 con 0x55f93d2d4950 2024-09-13T18:32:56.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.473+0000 7f0f1e7fc640 1 -- 172.21.15.114:0/2805211381 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0f20003be0 con 0x55f93d2d4950 2024-09-13T18:32:56.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.474+0000 7f0f312c4740 1 -- 172.21.15.114:0/2805211381 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f93d2d4950 msgr2=0x55f93d2decb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:56.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.474+0000 7f0f312c4740 1 --2- 172.21.15.114:0/2805211381 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f93d2d4950 0x55f93d2decb0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f0f200040c0 tx=0x7f0f200040f0 comp rx=0 tx=0).stop 2024-09-13T18:32:56.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.479+0000 7f0f277fe640 1 --2- 172.21.15.114:0/1835919856 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0f140a1400 0x7f0f140a38c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0f20039000 tx=0x7f0f20043000 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:56.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.480+0000 7f0f312c4740 1 -- 172.21.15.114:0/2805211381 shutdown_connections 2024-09-13T18:32:56.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.480+0000 7f0f312c4740 1 --2- 172.21.15.114:0/2805211381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f93d2df1f0 0x55f93d2e15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:56.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.480+0000 7f0f312c4740 1 --2- 172.21.15.114:0/2805211381 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f93d2d4950 0x55f93d2decb0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:56.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.480+0000 7f0f312c4740 1 --2- 172.21.15.114:0/2805211381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f93d2d4030 0x55f93d2d4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:56.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.480+0000 7f0f312c4740 1 -- 172.21.15.114:0/2805211381 >> 172.21.15.114:0/2805211381 conn(0x55f93d2d05a0 msgr2=0x55f93d2d0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:56.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.485+0000 7f0f312c4740 1 -- 172.21.15.114:0/2805211381 shutdown_connections 2024-09-13T18:32:56.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.487+0000 7f0f312c4740 1 -- 172.21.15.114:0/2805211381 wait complete. 2024-09-13T18:32:56.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.488+0000 7f0f312c4740 1 -- start start 2024-09-13T18:32:56.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.488+0000 7f0f312c4740 1 Processor -- start 2024-09-13T18:32:56.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.488+0000 7f0f312c4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f93d2d4030 0x55f93d2dea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:56.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.488+0000 7f0f312c4740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f93d2d4950 0x55f93d2dbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:56.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.488+0000 7f0f312c4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f93d2df1f0 0x55f93d2dc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:56.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.488+0000 7f0f312c4740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f93d1d92a0 con 0x55f93d2d4950 2024-09-13T18:32:56.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.488+0000 7f0f312c4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f93cf10430 con 0x55f93d2d4030 2024-09-13T18:32:56.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.488+0000 7f0f312c4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f93d1bb680 con 0x55f93d2df1f0 2024-09-13T18:32:56.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.488+0000 7f0f312c4740 10 client.0 _ll_register_callbacks cb 0x55f93d231610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:32:56.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.489+0000 7f0f2cb39640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f93d2df1f0 0x55f93d2dc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:56.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.489+0000 7f0f277fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f93d2d4950 0x55f93d2dbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:56.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.489+0000 7f0f277fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f93d2d4950 0x55f93d2dbd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41494/0 (socket says 172.21.15.114:41494) 2024-09-13T18:32:56.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.489+0000 7f0f277fe640 1 -- 172.21.15.114:0/3188742029 learned_addr learned my addr 172.21.15.114:0/3188742029 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:56.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.491+0000 7f0f277fe640 1 -- 172.21.15.114:0/3188742029 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f93d2df1f0 msgr2=0x55f93d2dc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:56.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.491+0000 7f0f277fe640 1 --2- 172.21.15.114:0/3188742029 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f93d2df1f0 0x55f93d2dc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:56.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.491+0000 7f0f277fe640 1 -- 172.21.15.114:0/3188742029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f93d2d4030 msgr2=0x55f93d2dea30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:32:56.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.491+0000 7f0f277fe640 1 --2- 172.21.15.114:0/3188742029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f93d2d4030 0x55f93d2dea30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:56.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.491+0000 7f0f277fe640 1 -- 172.21.15.114:0/3188742029 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f2811c120 con 0x55f93d2d4950 2024-09-13T18:32:56.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.492+0000 7f0f277fe640 1 --2- 172.21.15.114:0/3188742029 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f93d2d4950 0x55f93d2dbd00 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f0f20009900 tx=0x7f0f20032fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e401b122c978f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:56.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.492+0000 7f0f077fe640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:32:56.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.492+0000 7f0f077fe640 1 -- 172.21.15.114:0/3188742029 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2003fcf0 con 0x55f93d2d4950 2024-09-13T18:32:56.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.493+0000 7f0f312c4740 1 -- 172.21.15.114:0/3188742029 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f93cf2b4e0 con 0x55f93d2d4950 2024-09-13T18:32:56.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.493+0000 7f0f077fe640 1 -- 172.21.15.114:0/3188742029 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f0f200035f0 con 0x55f93d2d4950 2024-09-13T18:32:56.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.493+0000 7f0f077fe640 1 -- 172.21.15.114:0/3188742029 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f2003fe50 con 0x55f93d2d4950 2024-09-13T18:32:56.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.493+0000 7f0f312c4740 10 client.18351 fetch_fsmap learned FSMap version 34 2024-09-13T18:32:56.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.493+0000 7f0f312c4740 1 -- 172.21.15.114:0/3188742029 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55f93d236620 con 0x55f93d2d4950 2024-09-13T18:32:56.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.494+0000 7f0f077fe640 1 -- 172.21.15.114:0/3188742029 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f93cf2b4e0 con 0x55f93d2d4950 2024-09-13T18:32:56.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.494+0000 7f0f077fe640 1 -- 172.21.15.114:0/3188742029 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0f20050040 con 0x55f93d2d4950 2024-09-13T18:32:56.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.494+0000 7f0f312c4740 10 client.18351 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:32:56.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.494+0000 7f0f312c4740 10 client.18351 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:32:56.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.494+0000 7f0f312c4740 20 client.18351 populate_metadata read hostname 'smithi114' 2024-09-13T18:32:56.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.494+0000 7f0f312c4740 1 --2- 172.21.15.114:0/3188742029 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f93d2eee70 0x55f93d2f1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:56.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.494+0000 7f0f312c4740 4 client.18351 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:32:56.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.494+0000 7f0f312c4740 1 -- 172.21.15.114:0/3188742029 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f93cf3c7e0 con 0x55f93d2eee70 2024-09-13T18:32:56.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.502+0000 7f0f27fff640 1 --2- 172.21.15.114:0/3188742029 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f93d2eee70 0x55f93d2f1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:56.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.525+0000 7f0f27fff640 1 --2- 172.21.15.114:0/3188742029 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f93d2eee70 0x55f93d2f1290 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bac4f474c60ef9e server_cookie=a2433df6d027c062 in_seq=0 out_seq=0 2024-09-13T18:32:56.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.526+0000 7f0f077fe640 10 client.18351 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:32:56.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.528+0000 7f0f077fe640 1 -- 172.21.15.114:0/3188742029 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55f93cf3c7e0 con 0x55f93d2eee70 2024-09-13T18:32:56.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.528+0000 7f0f077fe640 10 client.18351 handle_command_reply: tid=0 2024-09-13T18:32:56.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.529+0000 7f0f312c4740 1 client.18351 shutdown 2024-09-13T18:32:56.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.535+0000 7f0f312c4740 1 -- 172.21.15.114:0/3188742029 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f93d2d4950 msgr2=0x55f93d2dbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:56.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.535+0000 7f0f312c4740 1 --2- 172.21.15.114:0/3188742029 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f93d2d4950 0x55f93d2dbd00 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f0f20009900 tx=0x7f0f20032fa0 comp rx=0 tx=0).stop 2024-09-13T18:32:56.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.538+0000 7f0f312c4740 1 -- 172.21.15.114:0/3188742029 shutdown_connections 2024-09-13T18:32:56.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.538+0000 7f0f312c4740 1 --2- 172.21.15.114:0/3188742029 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f93d2eee70 0x55f93d2f1290 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:56.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.538+0000 7f0f312c4740 1 --2- 172.21.15.114:0/3188742029 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f93d2df1f0 0x55f93d2dc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:56.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.538+0000 7f0f312c4740 1 --2- 172.21.15.114:0/3188742029 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f93d2d4950 0x55f93d2dbd00 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:56.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.538+0000 7f0f312c4740 1 --2- 172.21.15.114:0/3188742029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f93d2d4030 0x55f93d2dea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:56.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.538+0000 7f0f312c4740 1 -- 172.21.15.114:0/3188742029 >> 172.21.15.114:0/3188742029 conn(0x55f93d2d05a0 msgr2=0x55f93d2d6300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:56.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.538+0000 7f0f077fe640 0 client.18351 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:32:56.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.539+0000 7f0f312c4740 1 -- 172.21.15.114:0/3188742029 shutdown_connections 2024-09-13T18:32:56.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.543+0000 7f0f312c4740 1 -- 172.21.15.114:0/3188742029 wait complete. 2024-09-13T18:32:56.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.544+0000 7f0f312c4740 20 client.18351 trim_cache size 0 max 16384 2024-09-13T18:32:56.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.579+0000 7f0f312c4740 1 Processor -- start 2024-09-13T18:32:56.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.580+0000 7f0f312c4740 1 -- start start 2024-09-13T18:32:56.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.580+0000 7f0f312c4740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f93d2d4030 0x55f93d2cfc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:56.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.580+0000 7f0f312c4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f93d2d4950 0x55f93d2e6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:56.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.580+0000 7f0f312c4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f93d2df1f0 0x55f93d2e65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:56.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.580+0000 7f0f312c4740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f93d2dbb60 con 0x55f93d2d4030 2024-09-13T18:32:56.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.580+0000 7f0f312c4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f93cf3c7e0 con 0x55f93d2d4950 2024-09-13T18:32:56.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.580+0000 7f0f312c4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f93cf2b4e0 con 0x55f93d2df1f0 2024-09-13T18:32:56.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.580+0000 7f0f2cb39640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f93d2df1f0 0x55f93d2e65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:56.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.580+0000 7f0f2cb39640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f93d2df1f0 0x55f93d2e65d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:39188/0 (socket says 172.21.15.114:39188) 2024-09-13T18:32:56.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.580+0000 7f0f2cb39640 1 -- 172.21.15.114:0/278536862 learned_addr learned my addr 172.21.15.114:0/278536862 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:56.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.581+0000 7f0f2cb39640 1 -- 172.21.15.114:0/278536862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f93d2d4950 msgr2=0x55f93d2e6090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:32:56.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.581+0000 7f0f2cb39640 1 --2- 172.21.15.114:0/278536862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f93d2d4950 0x55f93d2e6090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:56.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.581+0000 7f0f2cb39640 1 -- 172.21.15.114:0/278536862 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f93d2d4030 msgr2=0x55f93d2cfc20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:32:56.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.581+0000 7f0f2cb39640 1 --2- 172.21.15.114:0/278536862 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f93d2d4030 0x55f93d2cfc20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:56.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.581+0000 7f0f2cb39640 1 -- 172.21.15.114:0/278536862 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f93d1d8790 con 0x55f93d2df1f0 2024-09-13T18:32:56.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.581+0000 7f0f2cb39640 1 --2- 172.21.15.114:0/278536862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f93d2df1f0 0x55f93d2e65d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0f1800ad70 tx=0x7f0f180097a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81633a24ad71cc50 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:56.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.583+0000 7f0f1effd640 1 -- 172.21.15.114:0/278536862 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f18013760 con 0x55f93d2df1f0 2024-09-13T18:32:56.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.583+0000 7f0f1effd640 1 -- 172.21.15.114:0/278536862 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0f18010040 con 0x55f93d2df1f0 2024-09-13T18:32:56.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.584+0000 7f0f312c4740 1 -- 172.21.15.114:0/278536862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f93d2df1f0 msgr2=0x55f93d2e65d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:56.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.584+0000 7f0f312c4740 1 --2- 172.21.15.114:0/278536862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f93d2df1f0 0x55f93d2e65d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0f1800ad70 tx=0x7f0f180097a0 comp rx=0 tx=0).stop 2024-09-13T18:32:56.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.587+0000 7f0f312c4740 1 -- 172.21.15.114:0/278536862 shutdown_connections 2024-09-13T18:32:56.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.587+0000 7f0f312c4740 1 --2- 172.21.15.114:0/278536862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f93d2df1f0 0x55f93d2e65d0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:56.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.587+0000 7f0f312c4740 1 --2- 172.21.15.114:0/278536862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f93d2d4950 0x55f93d2e6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:56.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.587+0000 7f0f312c4740 1 --2- 172.21.15.114:0/278536862 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f93d2d4030 0x55f93d2cfc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:56.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.587+0000 7f0f312c4740 1 -- 172.21.15.114:0/278536862 >> 172.21.15.114:0/278536862 conn(0x55f93d2d05a0 msgr2=0x55f93d2f5c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:56.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.594+0000 7f0f312c4740 1 -- 172.21.15.114:0/278536862 shutdown_connections 2024-09-13T18:32:56.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.594+0000 7f0f312c4740 1 -- 172.21.15.114:0/278536862 wait complete. 2024-09-13T18:32:56.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.594+0000 7f0f312c4740 1 -- start start 2024-09-13T18:32:56.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.594+0000 7f0f312c4740 1 Processor -- start 2024-09-13T18:32:56.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.595+0000 7f0f312c4740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f93d2d4030 0x55f93d2e4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:56.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.595+0000 7f0f312c4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f93d2d4950 0x55f93d2e45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:56.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.595+0000 7f0f312c4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f93d2df1f0 0x55f93d2e4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:56.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.595+0000 7f0f312c4740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f93d344550 con 0x55f93d2d4030 2024-09-13T18:32:56.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.595+0000 7f0f312c4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f93d3446c0 con 0x55f93d2df1f0 2024-09-13T18:32:56.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.595+0000 7f0f312c4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f93d344880 con 0x55f93d2d4950 2024-09-13T18:32:56.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.595+0000 7f0f312c4740 10 client.0 _ll_register_callbacks cb 0x7f0f200317d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:32:56.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.595+0000 7f0f27fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f93d2d4030 0x55f93d2e4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:56.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.595+0000 7f0f27fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f93d2d4030 0x55f93d2e4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41498/0 (socket says 172.21.15.114:41498) 2024-09-13T18:32:56.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.595+0000 7f0f27fff640 1 -- 172.21.15.114:0/1238404218 learned_addr learned my addr 172.21.15.114:0/1238404218 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:56.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.595+0000 7f0f2cb39640 1 --2- 172.21.15.114:0/1238404218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f93d2df1f0 0x55f93d2e4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:56.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.596+0000 7f0f277fe640 1 --2- 172.21.15.114:0/1238404218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f93d2d4950 0x55f93d2e45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:56.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.598+0000 7f0f277fe640 1 -- 172.21.15.114:0/1238404218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f93d2df1f0 msgr2=0x55f93d2e4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:56.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.598+0000 7f0f277fe640 1 --2- 172.21.15.114:0/1238404218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f93d2df1f0 0x55f93d2e4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:56.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.598+0000 7f0f277fe640 1 -- 172.21.15.114:0/1238404218 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f93d2d4030 msgr2=0x55f93d2e4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:56.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.598+0000 7f0f277fe640 1 --2- 172.21.15.114:0/1238404218 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f93d2d4030 0x55f93d2e4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:56.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.598+0000 7f0f277fe640 1 -- 172.21.15.114:0/1238404218 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f1800aa20 con 0x55f93d2d4950 2024-09-13T18:32:56.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.598+0000 7f0f277fe640 1 --2- 172.21.15.114:0/1238404218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f93d2d4950 0x55f93d2e45b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x55f93d2d48b0 tx=0x7f0f20004020 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc077dafcf7e8500 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:56.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.598+0000 7f0f077fe640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:32:56.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.599+0000 7f0f077fe640 1 -- 172.21.15.114:0/1238404218 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2003fbb0 con 0x55f93d2d4950 2024-09-13T18:32:56.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.599+0000 7f0f077fe640 1 -- 172.21.15.114:0/1238404218 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f0f20005d10 con 0x55f93d2d4950 2024-09-13T18:32:56.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.603+0000 7f0f312c4740 1 -- 172.21.15.114:0/1238404218 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0f20003be0 con 0x55f93d2d4950 2024-09-13T18:32:56.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.604+0000 7f0f077fe640 1 -- 172.21.15.114:0/1238404218 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f20005890 con 0x55f93d2d4950 2024-09-13T18:32:56.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.610+0000 7f0f312c4740 10 client.27763 fetch_fsmap learned FSMap version 34 2024-09-13T18:32:56.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.610+0000 7f0f312c4740 1 -- 172.21.15.114:0/1238404218 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55f93d1fdea0 con 0x55f93d2d4950 2024-09-13T18:32:56.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.611+0000 7f0f077fe640 1 -- 172.21.15.114:0/1238404218 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0f20003be0 con 0x55f93d2d4950 2024-09-13T18:32:56.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.612+0000 7f0f077fe640 1 -- 172.21.15.114:0/1238404218 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0f20051720 con 0x55f93d2d4950 2024-09-13T18:32:56.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.614+0000 7f0f312c4740 10 client.27763 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:32:56.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.614+0000 7f0f312c4740 10 client.27763 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:32:56.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.614+0000 7f0f312c4740 20 client.27763 populate_metadata read hostname 'smithi114' 2024-09-13T18:32:56.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.614+0000 7f0f312c4740 1 --2- 172.21.15.114:0/1238404218 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f93d345c20 0x55f93d348040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:56.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.614+0000 7f0f312c4740 4 client.27763 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T18:32:56.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.614+0000 7f0f312c4740 1 -- 172.21.15.114:0/1238404218 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f93cf8fe30 con 0x55f93d345c20 2024-09-13T18:32:56.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:56 smithi134 ceph-mon[34283]: pgmap v324: 129 pgs: 129 active+clean; 21 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 28 MiB/s wr, 614 op/s 2024-09-13T18:32:56.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.618+0000 7f0f27fff640 1 --2- 172.21.15.114:0/1238404218 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f93d345c20 0x55f93d348040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:56.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.624+0000 7f0f27fff640 1 --2- 172.21.15.114:0/1238404218 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f93d345c20 0x55f93d348040 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcce1c5c4c1afb38 server_cookie=31509d1201ae5316 in_seq=0 out_seq=0 2024-09-13T18:32:56.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.624+0000 7f0f077fe640 10 client.27763 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:32:56.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.624+0000 7f0f077fe640 1 -- 172.21.15.114:0/1238404218 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f93cf8fe30 con 0x55f93d345c20 2024-09-13T18:32:56.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.624+0000 7f0f077fe640 10 client.27763 handle_command_reply: tid=0 2024-09-13T18:32:56.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.626+0000 7f0f312c4740 1 client.27763 shutdown 2024-09-13T18:32:56.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.632+0000 7f0f312c4740 1 -- 172.21.15.114:0/1238404218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f93d2d4950 msgr2=0x55f93d2e45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:56.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.632+0000 7f0f312c4740 1 --2- 172.21.15.114:0/1238404218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f93d2d4950 0x55f93d2e45b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x55f93d2d48b0 tx=0x7f0f20004020 comp rx=0 tx=0).stop 2024-09-13T18:32:56.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.632+0000 7f0f312c4740 1 -- 172.21.15.114:0/1238404218 shutdown_connections 2024-09-13T18:32:56.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.632+0000 7f0f312c4740 1 --2- 172.21.15.114:0/1238404218 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55f93d345c20 0x55f93d348040 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:56.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.632+0000 7f0f312c4740 1 --2- 172.21.15.114:0/1238404218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f93d2df1f0 0x55f93d2e4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:56.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.632+0000 7f0f312c4740 1 --2- 172.21.15.114:0/1238404218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f93d2d4950 0x55f93d2e45b0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:56.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.632+0000 7f0f312c4740 1 --2- 172.21.15.114:0/1238404218 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f93d2d4030 0x55f93d2e4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:56.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.632+0000 7f0f312c4740 1 -- 172.21.15.114:0/1238404218 >> 172.21.15.114:0/1238404218 conn(0x55f93d2d05a0 msgr2=0x55f93d2dbca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:56.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.632+0000 7f0f077fe640 0 client.27763 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:32:56.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.632+0000 7f0f312c4740 1 -- 172.21.15.114:0/1238404218 shutdown_connections 2024-09-13T18:32:56.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.633+0000 7f0f312c4740 1 -- 172.21.15.114:0/1238404218 wait complete. 2024-09-13T18:32:56.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.634+0000 7f0f312c4740 20 client.27763 trim_cache size 0 max 16384 2024-09-13T18:32:56.642 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:32:56.642 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "no active scrubs running", 2024-09-13T18:32:56.642 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": {} 2024-09-13T18:32:56.642 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:32:56.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.643+0000 7f0f2d33a640 1 -- 172.21.15.114:0/1835919856 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0f140a1400 msgr2=0x7f0f140a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:56.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.643+0000 7f0f2d33a640 1 --2- 172.21.15.114:0/1835919856 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0f140a1400 0x7f0f140a38c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0f20039000 tx=0x7f0f20043000 comp rx=0 tx=0).stop 2024-09-13T18:32:56.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.643+0000 7f0f2d33a640 1 -- 172.21.15.114:0/1835919856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f281074b0 msgr2=0x7f0f28116910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:56.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.643+0000 7f0f2d33a640 1 --2- 172.21.15.114:0/1835919856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f281074b0 0x7f0f28116910 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0f1000d950 tx=0x7f0f1000de10 comp rx=0 tx=0).stop 2024-09-13T18:32:56.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.643+0000 7f0f2d33a640 1 -- 172.21.15.114:0/1835919856 shutdown_connections 2024-09-13T18:32:56.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.643+0000 7f0f2d33a640 1 --2- 172.21.15.114:0/1835919856 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0f140a1400 0x7f0f140a38c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:56.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.643+0000 7f0f2d33a640 1 --2- 172.21.15.114:0/1835919856 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f28119400 0x7f0f2810fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:56.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.643+0000 7f0f2d33a640 1 --2- 172.21.15.114:0/1835919856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f28107890 0x7f0f28116e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:56.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.643+0000 7f0f2d33a640 1 --2- 172.21.15.114:0/1835919856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f281074b0 0x7f0f28116910 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:56.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.643+0000 7f0f2d33a640 1 -- 172.21.15.114:0/1835919856 >> 172.21.15.114:0/1835919856 conn(0x7f0f2806aec0 msgr2=0x7f0f28119a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:56.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.649+0000 7f0f2d33a640 1 -- 172.21.15.114:0/1835919856 shutdown_connections 2024-09-13T18:32:56.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.649+0000 7f0f2d33a640 1 -- 172.21.15.114:0/1835919856 wait complete. 2024-09-13T18:32:56.694 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-13T18:32:56.694 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:32:56.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.936+0000 7f3ff6d53640 1 -- 172.21.15.114:0/954541927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3fe80b7500 msgr2=0x7f3fe80b98f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:56.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.936+0000 7f3ff6d53640 1 --2- 172.21.15.114:0/954541927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3fe80b7500 0x7f3fe80b98f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0063b50 tx=0x7f3ff008c5d0 comp rx=0 tx=0).stop 2024-09-13T18:32:56.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.937+0000 7f3ff6d53640 1 -- 172.21.15.114:0/954541927 shutdown_connections 2024-09-13T18:32:56.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.937+0000 7f3ff6d53640 1 --2- 172.21.15.114:0/954541927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3fe80b7500 0x7f3fe80b98f0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:56.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.937+0000 7f3ff6d53640 1 --2- 172.21.15.114:0/954541927 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3fe80a5100 0x7f3fe80b6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:56.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.937+0000 7f3ff6d53640 1 --2- 172.21.15.114:0/954541927 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fe80a47e0 0x7f3fe80a4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:56.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.937+0000 7f3ff6d53640 1 -- 172.21.15.114:0/954541927 >> 172.21.15.114:0/954541927 conn(0x7f3fe801a0c0 msgr2=0x7f3fe801a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:56.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.943+0000 7f3ff6d53640 1 -- 172.21.15.114:0/954541927 shutdown_connections 2024-09-13T18:32:56.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.945+0000 7f3ff6d53640 1 -- 172.21.15.114:0/954541927 wait complete. 2024-09-13T18:32:56.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.946+0000 7f3ff6d53640 1 Processor -- start 2024-09-13T18:32:56.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.946+0000 7f3ff6d53640 1 -- start start 2024-09-13T18:32:56.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.946+0000 7f3ff6d53640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3fe80a47e0 0x7f3fe8142fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:56.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.946+0000 7f3ff6d53640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fe80a5100 0x7f3fe81434e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:56.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.946+0000 7f3ff6d53640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3fe80b7500 0x7f3fe81478d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:56.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.946+0000 7f3ff6d53640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe8147ed0 con 0x7f3fe80b7500 2024-09-13T18:32:56.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.946+0000 7f3ff6d53640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe8148040 con 0x7f3fe80a5100 2024-09-13T18:32:56.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.946+0000 7f3ff6d53640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe8148200 con 0x7f3fe80a47e0 2024-09-13T18:32:56.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.947+0000 7f3ff6552640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3fe80b7500 0x7f3fe81478d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:56.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.947+0000 7f3ff6552640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3fe80b7500 0x7f3fe81478d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41524/0 (socket says 172.21.15.114:41524) 2024-09-13T18:32:56.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.947+0000 7f3ff6552640 1 -- 172.21.15.114:0/4220713879 learned_addr learned my addr 172.21.15.114:0/4220713879 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:56.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.947+0000 7f3ff5550640 1 --2- 172.21.15.114:0/4220713879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fe80a5100 0x7f3fe81434e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:56.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.952+0000 7f3ff5d51640 1 --2- 172.21.15.114:0/4220713879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3fe80a47e0 0x7f3fe8142fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:56.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.952+0000 7f3ff5550640 1 -- 172.21.15.114:0/4220713879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3fe80a47e0 msgr2=0x7f3fe8142fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:56.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.953+0000 7f3ff5550640 1 --2- 172.21.15.114:0/4220713879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3fe80a47e0 0x7f3fe8142fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:56.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.953+0000 7f3ff5550640 1 -- 172.21.15.114:0/4220713879 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3fe80b7500 msgr2=0x7f3fe81478d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:56.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.953+0000 7f3ff5550640 1 --2- 172.21.15.114:0/4220713879 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3fe80b7500 0x7f3fe81478d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:56.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.954+0000 7f3ff5550640 1 -- 172.21.15.114:0/4220713879 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff004eac0 con 0x7f3fe80a5100 2024-09-13T18:32:56.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.957+0000 7f3ff5550640 1 --2- 172.21.15.114:0/4220713879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fe80a5100 0x7f3fe81434e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3fec00c970 tx=0x7f3fec00ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:56.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.959+0000 7f3fe6ffd640 1 -- 172.21.15.114:0/4220713879 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fec007bb0 con 0x7f3fe80a5100 2024-09-13T18:32:56.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.959+0000 7f3fe6ffd640 1 -- 172.21.15.114:0/4220713879 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3fec007d10 con 0x7f3fe80a5100 2024-09-13T18:32:56.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.959+0000 7f3fe6ffd640 1 -- 172.21.15.114:0/4220713879 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fec018660 con 0x7f3fe80a5100 2024-09-13T18:32:56.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.964+0000 7f3ff6d53640 1 -- 172.21.15.114:0/4220713879 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fe81484e0 con 0x7f3fe80a5100 2024-09-13T18:32:56.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.964+0000 7f3ff6d53640 1 -- 172.21.15.114:0/4220713879 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fe800b970 con 0x7f3fe80a5100 2024-09-13T18:32:56.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.971+0000 7f3ff6d53640 1 -- 172.21.15.114:0/4220713879 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fe80a5b10 con 0x7f3fe80a5100 2024-09-13T18:32:56.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.975+0000 7f3fe6ffd640 1 -- 172.21.15.114:0/4220713879 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f3fec00f450 con 0x7f3fe80a5100 2024-09-13T18:32:56.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.976+0000 7f3fe6ffd640 1 --2- 172.21.15.114:0/4220713879 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f3fdc0a1410 0x7f3fdc0a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:56.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.976+0000 7f3fe6ffd640 1 -- 172.21.15.114:0/4220713879 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f3fec0cd140 con 0x7f3fe80a5100 2024-09-13T18:32:56.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.985+0000 7f3fe6ffd640 1 -- 172.21.15.114:0/4220713879 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f3fec096290 con 0x7f3fe80a5100 2024-09-13T18:32:56.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.986+0000 7f3ff5d51640 1 --2- 172.21.15.114:0/4220713879 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f3fdc0a1410 0x7f3fdc0a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:57.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:56.997+0000 7f3ff5d51640 1 --2- 172.21.15.114:0/4220713879 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f3fdc0a1410 0x7f3fdc0a38d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3fe00058c0 tx=0x7f3fe0005c60 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:57.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.549+0000 7f3ff6d53640 1 -- 172.21.15.114:0/4220713879 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3fe80a4bc0 con 0x7f3fe80a5100 2024-09-13T18:32:57.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.551+0000 7f3fe6ffd640 1 -- 172.21.15.114:0/4220713879 <== mon.2 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+10930 (secure 0 0 0) 0x7f3fec0998b0 con 0x7f3fe80a5100 2024-09-13T18:32:57.561 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:32:57.561 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4162817497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4162817497},{"type":"v1","addr":"172.21.15.114:6835","nonce":4162817497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/2435428478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":2435428478},{"type":"v1","addr":"172.21.15.114:6837","nonce":2435428478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T18:20:01.895117+0000","modified":"2024-09-13T18:27:22.172031+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24482,"mds_2":24475,"mds_3":24458,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/2046516451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":2046516451},{"type":"v1","addr":"172.21.15.114:6839","nonce":2046516451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":10,"addr":"172.21.15.114:6841/1956923446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":1956923446},{"type":"v1","addr":"172.21.15.114:6841","nonce":1956923446}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24391":{"gid":24391,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/507301496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":507301496},{"type":"v1","addr":"172.21.15.204:6833","nonce":507301496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4028657219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4028657219},{"type":"v1","addr":"172.21.15.134:6833","nonce":4028657219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/57029119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":57029119},{"type":"v1","addr":"172.21.15.204:6835","nonce":57029119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3068880939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3068880939},{"type":"v1","addr":"172.21.15.204:6837","nonce":3068880939}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"e","rank":3,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.134:6835/1150003275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1150003275},{"type":"v1","addr":"172.21.15.134:6835","nonce":1150003275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/344883912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":344883912},{"type":"v1","addr":"172.21.15.204:6839","nonce":344883912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6837/891692210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":891692210},{"type":"v1","addr":"172.21.15.134:6837","nonce":891692210}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6839/3466309724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3466309724},{"type":"v1","addr":"172.21.15.134:6839","nonce":3466309724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:32:57.562 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 34 2024-09-13T18:32:57.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.564+0000 7f3fe4ff9640 1 -- 172.21.15.114:0/4220713879 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f3fdc0a1410 msgr2=0x7f3fdc0a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:57.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.564+0000 7f3fe4ff9640 1 --2- 172.21.15.114:0/4220713879 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f3fdc0a1410 0x7f3fdc0a38d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3fe00058c0 tx=0x7f3fe0005c60 comp rx=0 tx=0).stop 2024-09-13T18:32:57.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.564+0000 7f3fe4ff9640 1 -- 172.21.15.114:0/4220713879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fe80a5100 msgr2=0x7f3fe81434e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:57.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.564+0000 7f3fe4ff9640 1 --2- 172.21.15.114:0/4220713879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fe80a5100 0x7f3fe81434e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3fec00c970 tx=0x7f3fec00ce30 comp rx=0 tx=0).stop 2024-09-13T18:32:57.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.568+0000 7f3fe4ff9640 1 -- 172.21.15.114:0/4220713879 shutdown_connections 2024-09-13T18:32:57.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.568+0000 7f3fe4ff9640 1 --2- 172.21.15.114:0/4220713879 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f3fdc0a1410 0x7f3fdc0a38d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:57.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.568+0000 7f3fe4ff9640 1 --2- 172.21.15.114:0/4220713879 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3fe80b7500 0x7f3fe81478d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:57.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.568+0000 7f3fe4ff9640 1 --2- 172.21.15.114:0/4220713879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fe80a5100 0x7f3fe81434e0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:57.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.568+0000 7f3fe4ff9640 1 --2- 172.21.15.114:0/4220713879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3fe80a47e0 0x7f3fe8142fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:57.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.568+0000 7f3fe4ff9640 1 -- 172.21.15.114:0/4220713879 >> 172.21.15.114:0/4220713879 conn(0x7f3fe801a0c0 msgr2=0x7f3fe80b4fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:57.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.572+0000 7f3fe4ff9640 1 -- 172.21.15.114:0/4220713879 shutdown_connections 2024-09-13T18:32:57.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.572+0000 7f3fe4ff9640 1 -- 172.21.15.114:0/4220713879 wait complete. 2024-09-13T18:32:57.609 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-09-13T18:32:57.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.893+0000 7f4121f53640 1 -- 172.21.15.114:0/3642710727 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f411c1074b0 msgr2=0x7f411c10e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:57.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.893+0000 7f4121f53640 1 --2- 172.21.15.114:0/3642710727 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f411c1074b0 0x7f411c10e510 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f4104009860 tx=0x7f410402a380 comp rx=0 tx=0).stop 2024-09-13T18:32:57.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.898+0000 7f4121f53640 1 -- 172.21.15.114:0/3642710727 shutdown_connections 2024-09-13T18:32:57.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.898+0000 7f4121f53640 1 --2- 172.21.15.114:0/3642710727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f411c119400 0x7f411c11b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:57.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.898+0000 7f4121f53640 1 --2- 172.21.15.114:0/3642710727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f411c107890 0x7f411c107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:57.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.898+0000 7f4121f53640 1 --2- 172.21.15.114:0/3642710727 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f411c1074b0 0x7f411c10e510 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:57.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.898+0000 7f4121f53640 1 -- 172.21.15.114:0/3642710727 >> 172.21.15.114:0/3642710727 conn(0x7f411c06af40 msgr2=0x7f411c06b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:57.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.903+0000 7f4121f53640 1 -- 172.21.15.114:0/3642710727 shutdown_connections 2024-09-13T18:32:57.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.908+0000 7f4121f53640 1 -- 172.21.15.114:0/3642710727 wait complete. 2024-09-13T18:32:57.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.908+0000 7f4121f53640 1 Processor -- start 2024-09-13T18:32:57.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.908+0000 7f4121f53640 1 -- start start 2024-09-13T18:32:57.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.909+0000 7f4121f53640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f411c1074b0 0x7f411c114900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:57.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.909+0000 7f4121f53640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f411c107890 0x7f411c114e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:57.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.909+0000 7f4121f53640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f411c119400 0x7f411c1a20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:57.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.909+0000 7f4121f53640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411c1a26e0 con 0x7f411c119400 2024-09-13T18:32:57.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.909+0000 7f4121f53640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411c1a2850 con 0x7f411c1074b0 2024-09-13T18:32:57.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.909+0000 7f4121f53640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411c1a2a10 con 0x7f411c107890 2024-09-13T18:32:57.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.912+0000 7f411bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f411c107890 0x7f411c114e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:57.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.912+0000 7f411bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f411c107890 0x7f411c114e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:39220/0 (socket says 172.21.15.114:39220) 2024-09-13T18:32:57.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.912+0000 7f411bfff640 1 -- 172.21.15.114:0/2698792566 learned_addr learned my addr 172.21.15.114:0/2698792566 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:57.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.912+0000 7f4121752640 1 --2- 172.21.15.114:0/2698792566 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f411c119400 0x7f411c1a20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:57.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.916+0000 7f411bfff640 1 -- 172.21.15.114:0/2698792566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f411c1074b0 msgr2=0x7f411c114900 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:32:57.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.916+0000 7f411bfff640 1 --2- 172.21.15.114:0/2698792566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f411c1074b0 0x7f411c114900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:57.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.916+0000 7f411bfff640 1 -- 172.21.15.114:0/2698792566 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f411c119400 msgr2=0x7f411c1a20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:57.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.916+0000 7f411bfff640 1 --2- 172.21.15.114:0/2698792566 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f411c119400 0x7f411c1a20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:57.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.916+0000 7f411bfff640 1 -- 172.21.15.114:0/2698792566 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4104009490 con 0x7f411c107890 2024-09-13T18:32:57.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.920+0000 7f411bfff640 1 --2- 172.21.15.114:0/2698792566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f411c107890 0x7f411c114e40 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f410c00ca80 tx=0x7f410c00cf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:57.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.922+0000 7f4119ffb640 1 -- 172.21.15.114:0/2698792566 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c007d30 con 0x7f411c107890 2024-09-13T18:32:57.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.922+0000 7f4121f53640 1 -- 172.21.15.114:0/2698792566 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f411c1a2cf0 con 0x7f411c107890 2024-09-13T18:32:57.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.922+0000 7f4121f53640 1 -- 172.21.15.114:0/2698792566 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f411c1cb3a0 con 0x7f411c107890 2024-09-13T18:32:57.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.925+0000 7f4119ffb640 1 -- 172.21.15.114:0/2698792566 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f410c007e90 con 0x7f411c107890 2024-09-13T18:32:57.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.925+0000 7f4119ffb640 1 -- 172.21.15.114:0/2698792566 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c018820 con 0x7f411c107890 2024-09-13T18:32:57.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.927+0000 7f4125ed6740 1 Processor -- start 2024-09-13T18:32:57.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.927+0000 7f4125ed6740 1 -- start start 2024-09-13T18:32:57.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.927+0000 7f4125ed6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f25bd29990 0x55f25bd29d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:57.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.927+0000 7f4125ed6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f25bd2a2b0 0x55f25bd34610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:57.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.927+0000 7f4125ed6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f25bd34b50 0x55f25bd36f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:57.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.927+0000 7f4125ed6740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f25bc93cc0 con 0x55f25bd29990 2024-09-13T18:32:57.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.927+0000 7f4125ed6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f25bc91450 con 0x55f25bd2a2b0 2024-09-13T18:32:57.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.927+0000 7f4125ed6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f25bc8cd40 con 0x55f25bd34b50 2024-09-13T18:32:57.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.929+0000 7f4120f51640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f25bd2a2b0 0x55f25bd34610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:57.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.929+0000 7f4121752640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f25bd29990 0x55f25bd29d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:57.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.929+0000 7f4121752640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f25bd29990 0x55f25bd29d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41544/0 (socket says 172.21.15.114:41544) 2024-09-13T18:32:57.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.929+0000 7f4121752640 1 -- 172.21.15.114:0/38983426 learned_addr learned my addr 172.21.15.114:0/38983426 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:57.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.943+0000 7f4120f51640 1 -- 172.21.15.114:0/38983426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f25bd34b50 msgr2=0x55f25bd36f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:57.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.943+0000 7f4120f51640 1 --2- 172.21.15.114:0/38983426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f25bd34b50 0x55f25bd36f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:57.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.943+0000 7f4120f51640 1 -- 172.21.15.114:0/38983426 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f25bd29990 msgr2=0x55f25bd29d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:57.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.943+0000 7f4120f51640 1 --2- 172.21.15.114:0/38983426 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f25bd29990 0x55f25bd29d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:57.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.943+0000 7f4120f51640 1 -- 172.21.15.114:0/38983426 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f25bc54620 con 0x55f25bd2a2b0 2024-09-13T18:32:57.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.944+0000 7f4120f51640 1 --2- 172.21.15.114:0/38983426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f25bd2a2b0 0x55f25bd34610 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f410402a8f0 tx=0x7f41040303d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3699ca75fb0fa36a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:57.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.945+0000 7f4119ffb640 1 -- 172.21.15.114:0/2698792566 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f410c018980 con 0x7f411c107890 2024-09-13T18:32:57.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.946+0000 7f4112ffd640 1 -- 172.21.15.114:0/38983426 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4104032da0 con 0x55f25bd2a2b0 2024-09-13T18:32:57.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.946+0000 7f4112ffd640 1 -- 172.21.15.114:0/38983426 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f410400b090 con 0x55f25bd2a2b0 2024-09-13T18:32:57.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.946+0000 7f4119ffb640 1 --2- 172.21.15.114:0/2698792566 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f41080a1410 0x7f41080a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:57.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.947+0000 7f4119ffb640 1 -- 172.21.15.114:0/2698792566 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f410c0cf330 con 0x7f411c107890 2024-09-13T18:32:57.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.947+0000 7f4125ed6740 1 -- 172.21.15.114:0/38983426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f25bd2a2b0 msgr2=0x55f25bd34610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:57.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.947+0000 7f4125ed6740 1 --2- 172.21.15.114:0/38983426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f25bd2a2b0 0x55f25bd34610 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f410402a8f0 tx=0x7f41040303d0 comp rx=0 tx=0).stop 2024-09-13T18:32:57.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.948+0000 7f4125ed6740 1 -- 172.21.15.114:0/38983426 shutdown_connections 2024-09-13T18:32:57.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.948+0000 7f4125ed6740 1 --2- 172.21.15.114:0/38983426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f25bd34b50 0x55f25bd36f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:57.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.948+0000 7f4125ed6740 1 --2- 172.21.15.114:0/38983426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f25bd2a2b0 0x55f25bd34610 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:57.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.948+0000 7f4125ed6740 1 --2- 172.21.15.114:0/38983426 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f25bd29990 0x55f25bd29d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:57.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.948+0000 7f4125ed6740 1 -- 172.21.15.114:0/38983426 >> 172.21.15.114:0/38983426 conn(0x55f25bd25f00 msgr2=0x55f25bd262e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:57.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.950+0000 7f4121752640 1 --2- 172.21.15.114:0/2698792566 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f41080a1410 0x7f41080a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:57.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.951+0000 7f4125ed6740 1 -- 172.21.15.114:0/38983426 shutdown_connections 2024-09-13T18:32:57.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.954+0000 7f4125ed6740 1 -- 172.21.15.114:0/38983426 wait complete. 2024-09-13T18:32:57.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.954+0000 7f4125ed6740 1 -- start start 2024-09-13T18:32:57.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.954+0000 7f4125ed6740 1 Processor -- start 2024-09-13T18:32:57.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.955+0000 7f4125ed6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f25bd29990 0x55f25bd33db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:57.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.955+0000 7f4125ed6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f25bd34b50 0x55f25bd342f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:57.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.955+0000 7f4125ed6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f25bd31150 0x55f25bd315d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:57.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.955+0000 7f4125ed6740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f25bc4b180 con 0x55f25bd29990 2024-09-13T18:32:57.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.955+0000 7f4125ed6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f25b965430 con 0x55f25bd31150 2024-09-13T18:32:57.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.955+0000 7f4125ed6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f25bc105e0 con 0x55f25bd34b50 2024-09-13T18:32:57.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.955+0000 7f4125ed6740 10 client.0 _ll_register_callbacks cb 0x55f25bc7caa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:32:57.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.955+0000 7f4121752640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f25bd31150 0x55f25bd315d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:57.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.955+0000 7f4121752640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f25bd31150 0x55f25bd315d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:47638/0 (socket says 172.21.15.114:47638) 2024-09-13T18:32:57.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.955+0000 7f4121752640 1 -- 172.21.15.114:0/2926050272 learned_addr learned my addr 172.21.15.114:0/2926050272 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:57.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.955+0000 7f411bfff640 1 --2- 172.21.15.114:0/2926050272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f25bd34b50 0x55f25bd342f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:57.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.956+0000 7f4121752640 1 -- 172.21.15.114:0/2926050272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f25bd34b50 msgr2=0x55f25bd342f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:57.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.956+0000 7f4121752640 1 --2- 172.21.15.114:0/2926050272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f25bd34b50 0x55f25bd342f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:57.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.956+0000 7f4121752640 1 -- 172.21.15.114:0/2926050272 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f25bd29990 msgr2=0x55f25bd33db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:57.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.956+0000 7f4121752640 1 --2- 172.21.15.114:0/2926050272 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f25bd29990 0x55f25bd33db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:57.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.956+0000 7f4121752640 1 -- 172.21.15.114:0/2926050272 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f411c11c170 con 0x55f25bd31150 2024-09-13T18:32:57.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.956+0000 7f4121752640 1 --2- 172.21.15.114:0/2926050272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f25bd31150 0x55f25bd315d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f4114007a20 tx=0x7f4114009f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4c59d0d00806716 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:57.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.957+0000 7f4121752640 1 --2- 172.21.15.114:0/2698792566 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f41080a1410 0x7f41080a38d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f41140093e0 tx=0x7f411400e040 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:57.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.962+0000 7f40fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:32:57.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.962+0000 7f40fbfff640 1 -- 172.21.15.114:0/2926050272 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4114018d60 con 0x55f25bd31150 2024-09-13T18:32:57.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.962+0000 7f4125ed6740 1 -- 172.21.15.114:0/2926050272 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f25bc7f1f0 con 0x55f25bd31150 2024-09-13T18:32:57.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.962+0000 7f40fbfff640 1 -- 172.21.15.114:0/2926050272 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f4114023400 con 0x55f25bd31150 2024-09-13T18:32:57.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.971+0000 7f40fbfff640 1 -- 172.21.15.114:0/2926050272 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4114018ec0 con 0x55f25bd31150 2024-09-13T18:32:57.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.973+0000 7f4125ed6740 10 client.27848 fetch_fsmap learned FSMap version 34 2024-09-13T18:32:57.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.973+0000 7f4125ed6740 1 -- 172.21.15.114:0/2926050272 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55f25bc8b3c0 con 0x55f25bd31150 2024-09-13T18:32:57.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.973+0000 7f40fbfff640 1 -- 172.21.15.114:0/2926050272 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f25bc7f1f0 con 0x55f25bd31150 2024-09-13T18:32:57.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.974+0000 7f40fbfff640 1 -- 172.21.15.114:0/2926050272 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4114021030 con 0x55f25bd31150 2024-09-13T18:32:57.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.974+0000 7f4125ed6740 10 client.27848 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:32:57.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.974+0000 7f4125ed6740 10 client.27848 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-13T18:32:57.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.974+0000 7f4125ed6740 20 client.27848 populate_metadata read hostname 'smithi114' 2024-09-13T18:32:57.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.974+0000 7f4125ed6740 1 --2- 172.21.15.114:0/2926050272 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55f25bd448b0 0x55f25bd46cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:57.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.974+0000 7f4125ed6740 4 client.27848 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:32:57.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.974+0000 7f4125ed6740 1 -- 172.21.15.114:0/2926050272 --> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f25b9804e0 con 0x55f25bd448b0 2024-09-13T18:32:57.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.976+0000 7f4120f51640 1 --2- 172.21.15.114:0/2926050272 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55f25bd448b0 0x55f25bd46cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:57.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.994+0000 7f4120f51640 1 --2- 172.21.15.114:0/2926050272 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55f25bd448b0 0x55f25bd46cd0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a3b1ca1f4bd4ff0f server_cookie=e7af961e52967e75 in_seq=0 out_seq=0 2024-09-13T18:32:58.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:57.994+0000 7f40fbfff640 10 client.27848 ms_handle_connect on v2:172.21.15.114:6840/1956923446 2024-09-13T18:32:58.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.000+0000 7f40fbfff640 1 -- 172.21.15.114:0/2926050272 <== mds.4 v2:172.21.15.114:6840/1956923446 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55f25b9804e0 con 0x55f25bd448b0 2024-09-13T18:32:58.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.000+0000 7f40fbfff640 10 client.27848 handle_command_reply: tid=0 2024-09-13T18:32:58.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.000+0000 7f4125ed6740 1 client.27848 shutdown 2024-09-13T18:32:58.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.002+0000 7f4125ed6740 1 -- 172.21.15.114:0/2926050272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f25bd31150 msgr2=0x55f25bd315d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:58.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.002+0000 7f4125ed6740 1 --2- 172.21.15.114:0/2926050272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f25bd31150 0x55f25bd315d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f4114007a20 tx=0x7f4114009f00 comp rx=0 tx=0).stop 2024-09-13T18:32:58.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.003+0000 7f4125ed6740 1 -- 172.21.15.114:0/2926050272 shutdown_connections 2024-09-13T18:32:58.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.003+0000 7f4125ed6740 1 --2- 172.21.15.114:0/2926050272 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55f25bd448b0 0x55f25bd46cd0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.003+0000 7f4125ed6740 1 --2- 172.21.15.114:0/2926050272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f25bd31150 0x55f25bd315d0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.003+0000 7f4125ed6740 1 --2- 172.21.15.114:0/2926050272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f25bd34b50 0x55f25bd342f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.003+0000 7f4125ed6740 1 --2- 172.21.15.114:0/2926050272 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f25bd29990 0x55f25bd33db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.003+0000 7f40fbfff640 0 client.27848 ms_handle_reset on v2:172.21.15.114:6840/1956923446 2024-09-13T18:32:58.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.003+0000 7f4125ed6740 1 -- 172.21.15.114:0/2926050272 >> 172.21.15.114:0/2926050272 conn(0x55f25bd25f00 msgr2=0x55f25bd24a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:58.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.005+0000 7f4125ed6740 1 -- 172.21.15.114:0/2926050272 shutdown_connections 2024-09-13T18:32:58.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.007+0000 7f4125ed6740 1 -- 172.21.15.114:0/2926050272 wait complete. 2024-09-13T18:32:58.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.011+0000 7f4125ed6740 20 client.27848 trim_cache size 0 max 16384 2024-09-13T18:32:58.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.042+0000 7f4125ed6740 1 Processor -- start 2024-09-13T18:32:58.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.043+0000 7f4125ed6740 1 -- start start 2024-09-13T18:32:58.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.043+0000 7f4125ed6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f25bd29990 0x55f25bd4b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:58.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.043+0000 7f4125ed6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f25bd31150 0x55f25bd3b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:58.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.043+0000 7f4125ed6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f25bd34b50 0x55f25bd3be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:58.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.043+0000 7f4125ed6740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f25bd430c0 con 0x55f25bd31150 2024-09-13T18:32:58.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.043+0000 7f4125ed6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f25bd21cf0 con 0x55f25bd34b50 2024-09-13T18:32:58.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.043+0000 7f4125ed6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f25bd29810 con 0x55f25bd29990 2024-09-13T18:32:58.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.044+0000 7f411bfff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f25bd31150 0x55f25bd3b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:58.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.044+0000 7f411bfff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f25bd31150 0x55f25bd3b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41548/0 (socket says 172.21.15.114:41548) 2024-09-13T18:32:58.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.044+0000 7f411bfff640 1 -- 172.21.15.114:0/572537715 learned_addr learned my addr 172.21.15.114:0/572537715 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:58.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.044+0000 7f411bfff640 1 -- 172.21.15.114:0/572537715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f25bd29990 msgr2=0x55f25bd4b650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:32:58.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.044+0000 7f411bfff640 1 --2- 172.21.15.114:0/572537715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f25bd29990 0x55f25bd4b650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.044+0000 7f411bfff640 1 -- 172.21.15.114:0/572537715 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f25bd34b50 msgr2=0x55f25bd3be60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:32:58.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.044+0000 7f411bfff640 1 --2- 172.21.15.114:0/572537715 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f25bd34b50 0x55f25bd3be60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.044+0000 7f411bfff640 1 -- 172.21.15.114:0/572537715 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f25bc2d770 con 0x55f25bd31150 2024-09-13T18:32:58.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.045+0000 7f411bfff640 1 --2- 172.21.15.114:0/572537715 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f25bd31150 0x55f25bd3b920 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f410c09bd40 tx=0x7f410c09c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec20b49a7de30881 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:58.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.047+0000 7f41137fe640 1 -- 172.21.15.114:0/572537715 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c0a44c0 con 0x55f25bd31150 2024-09-13T18:32:58.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.047+0000 7f41137fe640 1 -- 172.21.15.114:0/572537715 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f410c098750 con 0x55f25bd31150 2024-09-13T18:32:58.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.047+0000 7f41137fe640 1 -- 172.21.15.114:0/572537715 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c09b790 con 0x55f25bd31150 2024-09-13T18:32:58.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.049+0000 7f4125ed6740 1 -- 172.21.15.114:0/572537715 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f25bd31150 msgr2=0x55f25bd3b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:58.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.049+0000 7f4125ed6740 1 --2- 172.21.15.114:0/572537715 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f25bd31150 0x55f25bd3b920 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f410c09bd40 tx=0x7f410c09c880 comp rx=0 tx=0).stop 2024-09-13T18:32:58.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.054+0000 7f4125ed6740 1 -- 172.21.15.114:0/572537715 shutdown_connections 2024-09-13T18:32:58.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.054+0000 7f4125ed6740 1 --2- 172.21.15.114:0/572537715 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f25bd34b50 0x55f25bd3be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.054+0000 7f4125ed6740 1 --2- 172.21.15.114:0/572537715 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f25bd31150 0x55f25bd3b920 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.054+0000 7f4125ed6740 1 --2- 172.21.15.114:0/572537715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f25bd29990 0x55f25bd4b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.054+0000 7f4125ed6740 1 -- 172.21.15.114:0/572537715 >> 172.21.15.114:0/572537715 conn(0x55f25bd25f00 msgr2=0x55f25bd2c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:58.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.066+0000 7f4125ed6740 1 -- 172.21.15.114:0/572537715 shutdown_connections 2024-09-13T18:32:58.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.070+0000 7f4125ed6740 1 -- 172.21.15.114:0/572537715 wait complete. 2024-09-13T18:32:58.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.070+0000 7f4125ed6740 1 -- start start 2024-09-13T18:32:58.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.070+0000 7f4125ed6740 1 Processor -- start 2024-09-13T18:32:58.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.071+0000 7f4125ed6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f25bd29990 0x55f25bd3a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:58.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.071+0000 7f4125ed6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f25bd31150 0x55f25bd3af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:58.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.071+0000 7f4125ed6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f25bd34b50 0x55f25bd9ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:58.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.071+0000 7f4125ed6740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f25bd3b610 con 0x55f25bd31150 2024-09-13T18:32:58.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.071+0000 7f4125ed6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f25bd9b320 con 0x55f25bd34b50 2024-09-13T18:32:58.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.071+0000 7f4125ed6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f25bd9b490 con 0x55f25bd29990 2024-09-13T18:32:58.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.071+0000 7f4125ed6740 10 client.0 _ll_register_callbacks cb 0x7f4104035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:32:58.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.073+0000 7f4121752640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f25bd34b50 0x55f25bd9ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:58.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.073+0000 7f4121752640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f25bd34b50 0x55f25bd9ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:47646/0 (socket says 172.21.15.114:47646) 2024-09-13T18:32:58.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.073+0000 7f4121752640 1 -- 172.21.15.114:0/118820032 learned_addr learned my addr 172.21.15.114:0/118820032 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:58.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.077+0000 7f4121752640 1 -- 172.21.15.114:0/118820032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f25bd29990 msgr2=0x55f25bd3a9d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:32:58.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.077+0000 7f4121752640 1 --2- 172.21.15.114:0/118820032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f25bd29990 0x55f25bd3a9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.077+0000 7f4121752640 1 -- 172.21.15.114:0/118820032 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f25bd31150 msgr2=0x55f25bd3af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:58.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.077+0000 7f4121752640 1 --2- 172.21.15.114:0/118820032 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f25bd31150 0x55f25bd3af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.077+0000 7f4121752640 1 -- 172.21.15.114:0/118820032 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f411c1cb3a0 con 0x55f25bd34b50 2024-09-13T18:32:58.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.088+0000 7f4121752640 1 --2- 172.21.15.114:0/118820032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f25bd34b50 0x55f25bd9ade0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f41140079f0 tx=0x7f4114007740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3a5cdfd06596bb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:58.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.089+0000 7f40fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:32:58.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.097+0000 7f40fbfff640 1 -- 172.21.15.114:0/118820032 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4114030510 con 0x55f25bd34b50 2024-09-13T18:32:58.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.097+0000 7f4125ed6740 1 -- 172.21.15.114:0/118820032 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f25bcb6b80 con 0x55f25bd34b50 2024-09-13T18:32:58.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.097+0000 7f40fbfff640 1 -- 172.21.15.114:0/118820032 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f4114030670 con 0x55f25bd34b50 2024-09-13T18:32:58.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.112+0000 7f40fbfff640 1 -- 172.21.15.114:0/118820032 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41140099c0 con 0x55f25bd34b50 2024-09-13T18:32:58.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.117+0000 7f4125ed6740 10 client.27851 fetch_fsmap learned FSMap version 34 2024-09-13T18:32:58.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.117+0000 7f4125ed6740 1 -- 172.21.15.114:0/118820032 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55f25bc52e80 con 0x55f25bd34b50 2024-09-13T18:32:58.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.122+0000 7f40fbfff640 1 -- 172.21.15.114:0/118820032 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f25bcb6b80 con 0x55f25bd34b50 2024-09-13T18:32:58.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.122+0000 7f40fbfff640 1 -- 172.21.15.114:0/118820032 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4114039030 con 0x55f25bd34b50 2024-09-13T18:32:58.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.122+0000 7f4125ed6740 10 client.27851 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:32:58.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.122+0000 7f4125ed6740 10 client.27851 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-13T18:32:58.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.122+0000 7f4125ed6740 20 client.27851 populate_metadata read hostname 'smithi114' 2024-09-13T18:32:58.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.122+0000 7f4125ed6740 1 --2- 172.21.15.114:0/118820032 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55f25bd9cbd0 0x55f25bd9eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:58.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.122+0000 7f4125ed6740 4 client.27851 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:32:58.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.122+0000 7f4125ed6740 1 -- 172.21.15.114:0/118820032 --> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f25b9917e0 con 0x55f25bd9cbd0 2024-09-13T18:32:58.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.129+0000 7f4120f51640 1 --2- 172.21.15.114:0/118820032 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55f25bd9cbd0 0x55f25bd9eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:58.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.140+0000 7f4120f51640 1 --2- 172.21.15.114:0/118820032 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55f25bd9cbd0 0x55f25bd9eff0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9039187ce80b6a9d server_cookie=91b863a5fa4d2100 in_seq=0 out_seq=0 2024-09-13T18:32:58.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.141+0000 7f40fbfff640 10 client.27851 ms_handle_connect on v2:172.21.15.114:6840/1956923446 2024-09-13T18:32:58.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.146+0000 7f40fbfff640 1 -- 172.21.15.114:0/118820032 <== mds.4 v2:172.21.15.114:6840/1956923446 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f25b9917e0 con 0x55f25bd9cbd0 2024-09-13T18:32:58.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.146+0000 7f40fbfff640 10 client.27851 handle_command_reply: tid=0 2024-09-13T18:32:58.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.147+0000 7f4125ed6740 1 client.27851 shutdown 2024-09-13T18:32:58.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.157+0000 7f4125ed6740 1 -- 172.21.15.114:0/118820032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f25bd34b50 msgr2=0x55f25bd9ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:58.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.157+0000 7f4125ed6740 1 --2- 172.21.15.114:0/118820032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f25bd34b50 0x55f25bd9ade0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f41140079f0 tx=0x7f4114007740 comp rx=0 tx=0).stop 2024-09-13T18:32:58.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.158+0000 7f4125ed6740 1 -- 172.21.15.114:0/118820032 shutdown_connections 2024-09-13T18:32:58.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.158+0000 7f4125ed6740 1 --2- 172.21.15.114:0/118820032 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55f25bd9cbd0 0x55f25bd9eff0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.158+0000 7f4125ed6740 1 --2- 172.21.15.114:0/118820032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f25bd34b50 0x55f25bd9ade0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.158+0000 7f4125ed6740 1 --2- 172.21.15.114:0/118820032 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f25bd31150 0x55f25bd3af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.158+0000 7f4125ed6740 1 --2- 172.21.15.114:0/118820032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f25bd29990 0x55f25bd3a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.158+0000 7f4125ed6740 1 -- 172.21.15.114:0/118820032 >> 172.21.15.114:0/118820032 conn(0x55f25bd25f00 msgr2=0x55f25bd4b650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:58.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.159+0000 7f40fbfff640 0 client.27851 ms_handle_reset on v2:172.21.15.114:6840/1956923446 2024-09-13T18:32:58.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.163+0000 7f4125ed6740 1 -- 172.21.15.114:0/118820032 shutdown_connections 2024-09-13T18:32:58.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.170+0000 7f4125ed6740 1 -- 172.21.15.114:0/118820032 wait complete. 2024-09-13T18:32:58.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.175+0000 7f4125ed6740 20 client.27851 trim_cache size 0 max 16384 2024-09-13T18:32:58.181 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:32:58.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.181+0000 7f4121f53640 1 -- 172.21.15.114:0/2698792566 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f41080a1410 msgr2=0x7f41080a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:58.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.181+0000 7f4121f53640 1 --2- 172.21.15.114:0/2698792566 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f41080a1410 0x7f41080a38d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f41140093e0 tx=0x7f411400e040 comp rx=0 tx=0).stop 2024-09-13T18:32:58.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.181+0000 7f4121f53640 1 -- 172.21.15.114:0/2698792566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f411c107890 msgr2=0x7f411c114e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:58.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.181+0000 7f4121f53640 1 --2- 172.21.15.114:0/2698792566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f411c107890 0x7f411c114e40 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f410c00ca80 tx=0x7f410c00cf40 comp rx=0 tx=0).stop 2024-09-13T18:32:58.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.189+0000 7f4121f53640 1 -- 172.21.15.114:0/2698792566 shutdown_connections 2024-09-13T18:32:58.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.189+0000 7f4121f53640 1 --2- 172.21.15.114:0/2698792566 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f41080a1410 0x7f41080a38d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.189+0000 7f4121f53640 1 --2- 172.21.15.114:0/2698792566 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f411c119400 0x7f411c1a20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.189+0000 7f4121f53640 1 --2- 172.21.15.114:0/2698792566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f411c107890 0x7f411c114e40 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.189+0000 7f4121f53640 1 --2- 172.21.15.114:0/2698792566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f411c1074b0 0x7f411c114900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.189+0000 7f4121f53640 1 -- 172.21.15.114:0/2698792566 >> 172.21.15.114:0/2698792566 conn(0x7f411c06af40 msgr2=0x7f411c06eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:58.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.192+0000 7f4121f53640 1 -- 172.21.15.114:0/2698792566 shutdown_connections 2024-09-13T18:32:58.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.195+0000 7f4121f53640 1 -- 172.21.15.114:0/2698792566 wait complete. 2024-09-13T18:32:58.238 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-09-13T18:32:58.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:58 smithi204 ceph-mon[34777]: pgmap v325: 129 pgs: 129 active+clean; 21 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 34 MiB/s wr, 751 op/s 2024-09-13T18:32:58.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:32:58 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/4220713879' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:32:58.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:58 smithi114 ceph-mon[30897]: pgmap v325: 129 pgs: 129 active+clean; 21 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 34 MiB/s wr, 751 op/s 2024-09-13T18:32:58.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:32:58 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/4220713879' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:32:58.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.481+0000 7fcaebc9c640 1 -- 172.21.15.114:0/1435048238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcae4119400 msgr2=0x7fcae411b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:58.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.481+0000 7fcaebc9c640 1 --2- 172.21.15.114:0/1435048238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcae4119400 0x7fcae411b7f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fcadc009ca0 tx=0x7fcadc02a8f0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.485+0000 7fcaebc9c640 1 -- 172.21.15.114:0/1435048238 shutdown_connections 2024-09-13T18:32:58.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.485+0000 7fcaebc9c640 1 --2- 172.21.15.114:0/1435048238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcae4119400 0x7fcae411b7f0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.485+0000 7fcaebc9c640 1 --2- 172.21.15.114:0/1435048238 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcae4107920 0x7fcae4107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.485+0000 7fcaebc9c640 1 --2- 172.21.15.114:0/1435048238 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcae4107540 0x7fcae410e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.485+0000 7fcaebc9c640 1 -- 172.21.15.114:0/1435048238 >> 172.21.15.114:0/1435048238 conn(0x7fcae406afe0 msgr2=0x7fcae406b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:58.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.485+0000 7fcaebc9c640 1 -- 172.21.15.114:0/1435048238 shutdown_connections 2024-09-13T18:32:58.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.488+0000 7fcaebc9c640 1 -- 172.21.15.114:0/1435048238 wait complete. 2024-09-13T18:32:58.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.489+0000 7fcaebc9c640 1 Processor -- start 2024-09-13T18:32:58.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.489+0000 7fcaebc9c640 1 -- start start 2024-09-13T18:32:58.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.489+0000 7fcaebc9c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcae4107540 0x7fcae41168a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:58.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.489+0000 7fcaebc9c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcae4107920 0x7fcae4116de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:58.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.489+0000 7fcaebc9c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcae4119400 0x7fcae410fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:58.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.489+0000 7fcaebc9c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae4110200 con 0x7fcae4107920 2024-09-13T18:32:58.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.489+0000 7fcaebc9c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae4110370 con 0x7fcae4119400 2024-09-13T18:32:58.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.489+0000 7fcaebc9c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae4110530 con 0x7fcae4107540 2024-09-13T18:32:58.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.489+0000 7fcae9237640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcae4107920 0x7fcae4116de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:58.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.489+0000 7fcae9237640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcae4107920 0x7fcae4116de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41580/0 (socket says 172.21.15.114:41580) 2024-09-13T18:32:58.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.489+0000 7fcae9237640 1 -- 172.21.15.114:0/4139803403 learned_addr learned my addr 172.21.15.114:0/4139803403 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:58.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.489+0000 7fcaea239640 1 --2- 172.21.15.114:0/4139803403 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcae4119400 0x7fcae410fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:58.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.489+0000 7fcae9a38640 1 --2- 172.21.15.114:0/4139803403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcae4107540 0x7fcae41168a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:58.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.490+0000 7fcaea239640 1 -- 172.21.15.114:0/4139803403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcae4107540 msgr2=0x7fcae41168a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:58.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.490+0000 7fcaea239640 1 --2- 172.21.15.114:0/4139803403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcae4107540 0x7fcae41168a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.490+0000 7fcaea239640 1 -- 172.21.15.114:0/4139803403 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcae4107920 msgr2=0x7fcae4116de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:58.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.490+0000 7fcaea239640 1 --2- 172.21.15.114:0/4139803403 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcae4107920 0x7fcae4116de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.490+0000 7fcaea239640 1 -- 172.21.15.114:0/4139803403 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcadc009900 con 0x7fcae4119400 2024-09-13T18:32:58.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.490+0000 7fcae9237640 1 --2- 172.21.15.114:0/4139803403 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcae4107920 0x7fcae4116de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:32:58.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.490+0000 7fcaea239640 1 --2- 172.21.15.114:0/4139803403 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcae4119400 0x7fcae410fa70 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fcadc006010 tx=0x7fcadc031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:58.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.491+0000 7fcadaffd640 1 -- 172.21.15.114:0/4139803403 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcadc030a00 con 0x7fcae4119400 2024-09-13T18:32:58.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.491+0000 7fcaebc9c640 1 -- 172.21.15.114:0/4139803403 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcae41107b0 con 0x7fcae4119400 2024-09-13T18:32:58.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.491+0000 7fcaebc9c640 1 -- 172.21.15.114:0/4139803403 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcae4114790 con 0x7fcae4119400 2024-09-13T18:32:58.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.491+0000 7fcadaffd640 1 -- 172.21.15.114:0/4139803403 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcadc03f070 con 0x7fcae4119400 2024-09-13T18:32:58.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.492+0000 7fcadaffd640 1 -- 172.21.15.114:0/4139803403 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcadc0466e0 con 0x7fcae4119400 2024-09-13T18:32:58.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.498+0000 7fcadaffd640 1 -- 172.21.15.114:0/4139803403 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fcadc00a040 con 0x7fcae4119400 2024-09-13T18:32:58.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.499+0000 7fcadaffd640 1 --2- 172.21.15.114:0/4139803403 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fcad00a11e0 0x7fcad00a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:58.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.500+0000 7fcadaffd640 1 -- 172.21.15.114:0/4139803403 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fcadc041070 con 0x7fcae4119400 2024-09-13T18:32:58.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.500+0000 7fcae9a38640 1 --2- 172.21.15.114:0/4139803403 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fcad00a11e0 0x7fcad00a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:58.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.500+0000 7fcaee9c5740 1 Processor -- start 2024-09-13T18:32:58.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.502+0000 7fcaee9c5740 1 -- start start 2024-09-13T18:32:58.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.502+0000 7fcaee9c5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578536cc790 0x5578536ccb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:58.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.502+0000 7fcaee9c5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5578536cd0b0 0x5578536d7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:58.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.502+0000 7fcaee9c5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578536d7950 0x5578536d9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:58.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.502+0000 7fcaee9c5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x557853636cc0 con 0x5578536cd0b0 2024-09-13T18:32:58.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.502+0000 7fcaee9c5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557853634450 con 0x5578536d7950 2024-09-13T18:32:58.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.502+0000 7fcaee9c5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55785362fd40 con 0x5578536cc790 2024-09-13T18:32:58.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.502+0000 7fcae9237640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5578536cd0b0 0x5578536d7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:58.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.502+0000 7fcae9237640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5578536cd0b0 0x5578536d7410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41584/0 (socket says 172.21.15.114:41584) 2024-09-13T18:32:58.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.502+0000 7fcae9237640 1 -- 172.21.15.114:0/891148800 learned_addr learned my addr 172.21.15.114:0/891148800 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:58.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.503+0000 7fcaea239640 1 --2- 172.21.15.114:0/891148800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578536d7950 0x5578536d9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:58.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.503+0000 7fcae9237640 1 -- 172.21.15.114:0/891148800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578536cc790 msgr2=0x5578536ccb70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:32:58.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.503+0000 7fcae9237640 1 --2- 172.21.15.114:0/891148800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578536cc790 0x5578536ccb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.503+0000 7fcae9237640 1 -- 172.21.15.114:0/891148800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578536d7950 msgr2=0x5578536d9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:58.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.503+0000 7fcae9237640 1 --2- 172.21.15.114:0/891148800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578536d7950 0x5578536d9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.503+0000 7fcae9237640 1 -- 172.21.15.114:0/891148800 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578535f7620 con 0x5578536cd0b0 2024-09-13T18:32:58.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.503+0000 7fcaea239640 1 --2- 172.21.15.114:0/891148800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578536d7950 0x5578536d9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:32:58.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.503+0000 7fcae9237640 1 --2- 172.21.15.114:0/891148800 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5578536cd0b0 0x5578536d7410 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fcae0010550 tx=0x7fcae0010a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23fb517001528295 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:58.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.503+0000 7fcacffff640 1 -- 172.21.15.114:0/891148800 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae0011820 con 0x5578536cd0b0 2024-09-13T18:32:58.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.503+0000 7fcacffff640 1 -- 172.21.15.114:0/891148800 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcae0015070 con 0x5578536cd0b0 2024-09-13T18:32:58.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.503+0000 7fcacffff640 1 -- 172.21.15.114:0/891148800 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae001bc90 con 0x5578536cd0b0 2024-09-13T18:32:58.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.505+0000 7fcaee9c5740 1 -- 172.21.15.114:0/891148800 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5578536cd0b0 msgr2=0x5578536d7410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:58.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.505+0000 7fcaee9c5740 1 --2- 172.21.15.114:0/891148800 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5578536cd0b0 0x5578536d7410 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fcae0010550 tx=0x7fcae0010a10 comp rx=0 tx=0).stop 2024-09-13T18:32:58.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.505+0000 7fcae9a38640 1 --2- 172.21.15.114:0/4139803403 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fcad00a11e0 0x7fcad00a36a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fcad4006fd0 tx=0x7fcad4008040 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:58.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.510+0000 7fcaee9c5740 1 -- 172.21.15.114:0/891148800 shutdown_connections 2024-09-13T18:32:58.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.510+0000 7fcaee9c5740 1 --2- 172.21.15.114:0/891148800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578536d7950 0x5578536d9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.510+0000 7fcaee9c5740 1 --2- 172.21.15.114:0/891148800 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5578536cd0b0 0x5578536d7410 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.510+0000 7fcaee9c5740 1 --2- 172.21.15.114:0/891148800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578536cc790 0x5578536ccb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.510+0000 7fcaee9c5740 1 -- 172.21.15.114:0/891148800 >> 172.21.15.114:0/891148800 conn(0x5578536c8d00 msgr2=0x5578536c90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:58.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.510+0000 7fcaee9c5740 1 -- 172.21.15.114:0/891148800 shutdown_connections 2024-09-13T18:32:58.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.510+0000 7fcaee9c5740 1 -- 172.21.15.114:0/891148800 wait complete. 2024-09-13T18:32:58.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.511+0000 7fcaee9c5740 1 -- start start 2024-09-13T18:32:58.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.511+0000 7fcaee9c5740 1 Processor -- start 2024-09-13T18:32:58.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.512+0000 7fcaee9c5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578536cc790 0x5578536d6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:58.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.512+0000 7fcaee9c5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5578536cd0b0 0x5578536d4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:58.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.512+0000 7fcaee9c5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578536d7950 0x5578536d47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:58.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.512+0000 7fcae9a38640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578536cc790 0x5578536d6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:58.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.512+0000 7fcae9a38640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578536cc790 0x5578536d6f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:39270/0 (socket says 172.21.15.114:39270) 2024-09-13T18:32:58.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.512+0000 7fcae9a38640 1 -- 172.21.15.114:0/697605863 learned_addr learned my addr 172.21.15.114:0/697605863 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:58.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.513+0000 7fcaee9c5740 1 -- 172.21.15.114:0/697605863 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578535ee180 con 0x5578536cd0b0 2024-09-13T18:32:58.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.513+0000 7fcaee9c5740 1 -- 172.21.15.114:0/697605863 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557853308430 con 0x5578536d7950 2024-09-13T18:32:58.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.513+0000 7fcaee9c5740 1 -- 172.21.15.114:0/697605863 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578535b35e0 con 0x5578536cc790 2024-09-13T18:32:58.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.513+0000 7fcaea239640 1 --2- 172.21.15.114:0/697605863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578536d7950 0x5578536d47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:58.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.513+0000 7fcaea239640 1 -- 172.21.15.114:0/697605863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578536cc790 msgr2=0x5578536d6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:58.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.513+0000 7fcaea239640 1 --2- 172.21.15.114:0/697605863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578536cc790 0x5578536d6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.513+0000 7fcaea239640 1 -- 172.21.15.114:0/697605863 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5578536cd0b0 msgr2=0x5578536d4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:58.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.513+0000 7fcaea239640 1 --2- 172.21.15.114:0/697605863 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5578536cd0b0 0x5578536d4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.513+0000 7fcaea239640 1 -- 172.21.15.114:0/697605863 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcae000bd40 con 0x5578536d7950 2024-09-13T18:32:58.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.514+0000 7fcaea239640 1 --2- 172.21.15.114:0/697605863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578536d7950 0x5578536d47d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fcadc0eb8e0 tx=0x7fcadc0bbae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2480bac06cfbb7e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:58.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.514+0000 7fcaee9c5740 10 client.0 _ll_register_callbacks cb 0x55785361faa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:32:58.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.514+0000 7fcaee9c5740 1 -- 172.21.15.114:0/697605863 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557853659b80 con 0x5578536d7950 2024-09-13T18:32:58.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.515+0000 7fcacd7fa640 10 client.27860 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:32:58.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.515+0000 7fcacd7fa640 1 -- 172.21.15.114:0/697605863 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcadc0eba50 con 0x5578536d7950 2024-09-13T18:32:58.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.515+0000 7fcacd7fa640 1 -- 172.21.15.114:0/697605863 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fcadc0ebbb0 con 0x5578536d7950 2024-09-13T18:32:58.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.515+0000 7fcacd7fa640 1 -- 172.21.15.114:0/697605863 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcadc0c3290 con 0x5578536d7950 2024-09-13T18:32:58.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.519+0000 7fcaee9c5740 10 client.27860 fetch_fsmap learned FSMap version 34 2024-09-13T18:32:58.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.519+0000 7fcaee9c5740 1 -- 172.21.15.114:0/697605863 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({fsmap=34}) v3 -- 0x55785362e3c0 con 0x5578536d7950 2024-09-13T18:32:58.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.523+0000 7fcacd7fa640 1 -- 172.21.15.114:0/697605863 <== mon.2 v2:172.21.15.134:3300/0 4 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcadc0bd070 con 0x5578536d7950 2024-09-13T18:32:58.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.523+0000 7fcaee9c5740 10 client.27860 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:32:58.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.523+0000 7fcaee9c5740 10 client.27860 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-13T18:32:58.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.524+0000 7fcaee9c5740 20 client.27860 populate_metadata read hostname 'smithi114' 2024-09-13T18:32:58.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.524+0000 7fcaee9c5740 1 --2- 172.21.15.114:0/697605863 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x5578536e6b40 0x5578536e9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:58.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.524+0000 7fcaee9c5740 4 client.27860 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:32:58.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.524+0000 7fcaee9c5740 1 -- 172.21.15.114:0/697605863 --> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557853659b80 con 0x5578536e6b40 2024-09-13T18:32:58.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.524+0000 7fcae9a38640 1 --2- 172.21.15.114:0/697605863 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x5578536e6b40 0x5578536e9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:58.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.525+0000 7fcae9a38640 1 --2- 172.21.15.114:0/697605863 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x5578536e6b40 0x5578536e9000 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=53899cb790fa8fd4 server_cookie=e050a5931c0dcdb8 in_seq=0 out_seq=0 2024-09-13T18:32:58.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.525+0000 7fcacd7fa640 10 client.27860 ms_handle_connect on v2:172.21.15.134:6834/1150003275 2024-09-13T18:32:58.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.528+0000 7fcacd7fa640 1 -- 172.21.15.114:0/697605863 <== mds.3 v2:172.21.15.134:6834/1150003275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x557853659b80 con 0x5578536e6b40 2024-09-13T18:32:58.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.528+0000 7fcacd7fa640 10 client.27860 handle_command_reply: tid=0 2024-09-13T18:32:58.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.528+0000 7fcaee9c5740 1 client.27860 shutdown 2024-09-13T18:32:58.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.531+0000 7fcaee9c5740 1 -- 172.21.15.114:0/697605863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578536d7950 msgr2=0x5578536d47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:58.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.531+0000 7fcaee9c5740 1 --2- 172.21.15.114:0/697605863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578536d7950 0x5578536d47d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fcadc0eb8e0 tx=0x7fcadc0bbae0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.540+0000 7fcaee9c5740 1 -- 172.21.15.114:0/697605863 shutdown_connections 2024-09-13T18:32:58.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.540+0000 7fcaee9c5740 1 --2- 172.21.15.114:0/697605863 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x5578536e6b40 0x5578536e9000 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.540+0000 7fcaee9c5740 1 --2- 172.21.15.114:0/697605863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578536d7950 0x5578536d47d0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.540+0000 7fcaee9c5740 1 --2- 172.21.15.114:0/697605863 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5578536cd0b0 0x5578536d4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.540+0000 7fcaee9c5740 1 --2- 172.21.15.114:0/697605863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578536cc790 0x5578536d6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.540+0000 7fcaee9c5740 1 -- 172.21.15.114:0/697605863 >> 172.21.15.114:0/697605863 conn(0x5578536c8d00 msgr2=0x5578536cea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:58.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.540+0000 7fcacd7fa640 0 client.27860 ms_handle_reset on v2:172.21.15.134:6834/1150003275 2024-09-13T18:32:58.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.546+0000 7fcaee9c5740 1 -- 172.21.15.114:0/697605863 shutdown_connections 2024-09-13T18:32:58.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.546+0000 7fcaee9c5740 1 -- 172.21.15.114:0/697605863 wait complete. 2024-09-13T18:32:58.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.550+0000 7fcaee9c5740 20 client.27860 trim_cache size 0 max 16384 2024-09-13T18:32:58.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.589+0000 7fcaee9c5740 1 Processor -- start 2024-09-13T18:32:58.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.589+0000 7fcaee9c5740 1 -- start start 2024-09-13T18:32:58.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.589+0000 7fcaee9c5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5578536cc790 0x5578536c86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:58.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.589+0000 7fcaee9c5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578536cd0b0 0x5578536d4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:58.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.589+0000 7fcaee9c5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578536d7950 0x5578536d4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:58.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.589+0000 7fcaee9c5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578536d5430 con 0x5578536cc790 2024-09-13T18:32:58.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.590+0000 7fcaee9c5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557853659b80 con 0x5578536d7950 2024-09-13T18:32:58.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.590+0000 7fcaee9c5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578536da400 con 0x5578536cd0b0 2024-09-13T18:32:58.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.590+0000 7fcae9237640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578536cd0b0 0x5578536d4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:58.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.590+0000 7fcae9237640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578536cd0b0 0x5578536d4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:39284/0 (socket says 172.21.15.114:39284) 2024-09-13T18:32:58.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.590+0000 7fcae9237640 1 -- 172.21.15.114:0/1967735819 learned_addr learned my addr 172.21.15.114:0/1967735819 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:58.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.590+0000 7fcaea239640 1 --2- 172.21.15.114:0/1967735819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578536d7950 0x5578536d4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:58.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.590+0000 7fcae9a38640 1 --2- 172.21.15.114:0/1967735819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5578536cc790 0x5578536c86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:58.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.590+0000 7fcaea239640 1 -- 172.21.15.114:0/1967735819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578536cd0b0 msgr2=0x5578536d4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:58.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.590+0000 7fcaea239640 1 --2- 172.21.15.114:0/1967735819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578536cd0b0 0x5578536d4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.590+0000 7fcaea239640 1 -- 172.21.15.114:0/1967735819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5578536cc790 msgr2=0x5578536c86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:58.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.590+0000 7fcaea239640 1 --2- 172.21.15.114:0/1967735819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5578536cc790 0x5578536c86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.590+0000 7fcaea239640 1 -- 172.21.15.114:0/1967735819 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578535d0770 con 0x5578536d7950 2024-09-13T18:32:58.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.591+0000 7fcaea239640 1 --2- 172.21.15.114:0/1967735819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578536d7950 0x5578536d4780 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fcadc0a6010 tx=0x7fcadc03d070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6eee06465e059de9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:58.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.595+0000 7fcad8ff9640 1 -- 172.21.15.114:0/1967735819 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcadc0b75e0 con 0x5578536d7950 2024-09-13T18:32:58.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.595+0000 7fcad8ff9640 1 -- 172.21.15.114:0/1967735819 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcadc09f020 con 0x5578536d7950 2024-09-13T18:32:58.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.596+0000 7fcaee9c5740 1 -- 172.21.15.114:0/1967735819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578536d7950 msgr2=0x5578536d4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:58.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.596+0000 7fcaee9c5740 1 --2- 172.21.15.114:0/1967735819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578536d7950 0x5578536d4780 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fcadc0a6010 tx=0x7fcadc03d070 comp rx=0 tx=0).stop 2024-09-13T18:32:58.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.600+0000 7fcaee9c5740 1 -- 172.21.15.114:0/1967735819 shutdown_connections 2024-09-13T18:32:58.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.600+0000 7fcaee9c5740 1 --2- 172.21.15.114:0/1967735819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578536d7950 0x5578536d4780 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.600+0000 7fcaee9c5740 1 --2- 172.21.15.114:0/1967735819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578536cd0b0 0x5578536d4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.600+0000 7fcaee9c5740 1 --2- 172.21.15.114:0/1967735819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5578536cc790 0x5578536c86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.600+0000 7fcaee9c5740 1 -- 172.21.15.114:0/1967735819 >> 172.21.15.114:0/1967735819 conn(0x5578536c8d00 msgr2=0x5578536d3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:58.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.602+0000 7fcaee9c5740 1 -- 172.21.15.114:0/1967735819 shutdown_connections 2024-09-13T18:32:58.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.602+0000 7fcaee9c5740 1 -- 172.21.15.114:0/1967735819 wait complete. 2024-09-13T18:32:58.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.602+0000 7fcaee9c5740 1 -- start start 2024-09-13T18:32:58.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.603+0000 7fcaee9c5740 1 Processor -- start 2024-09-13T18:32:58.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.603+0000 7fcaee9c5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578536cc790 0x5578536de910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:58.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.603+0000 7fcaee9c5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578536cd0b0 0x5578536dac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:58.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.603+0000 7fcaee9c5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5578536d7950 0x5578536db180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:58.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.603+0000 7fcaee9c5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578536db880 con 0x5578536d7950 2024-09-13T18:32:58.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.603+0000 7fcaee9c5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578536db9f0 con 0x5578536cd0b0 2024-09-13T18:32:58.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.603+0000 7fcaee9c5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55785373d120 con 0x5578536cc790 2024-09-13T18:32:58.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.603+0000 7fcaee9c5740 10 client.0 _ll_register_callbacks cb 0x7fcae000bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:32:58.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.604+0000 7fcae9a38640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578536cc790 0x5578536de910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:58.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.604+0000 7fcae9237640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578536cd0b0 0x5578536dac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:58.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.604+0000 7fcae9237640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578536cd0b0 0x5578536dac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:47706/0 (socket says 172.21.15.114:47706) 2024-09-13T18:32:58.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.604+0000 7fcae9237640 1 -- 172.21.15.114:0/1108598882 learned_addr learned my addr 172.21.15.114:0/1108598882 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:58.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.604+0000 7fcaea239640 1 --2- 172.21.15.114:0/1108598882 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5578536d7950 0x5578536db180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:58.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.604+0000 7fcae9237640 1 -- 172.21.15.114:0/1108598882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578536cc790 msgr2=0x5578536de910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:58.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.604+0000 7fcae9237640 1 --2- 172.21.15.114:0/1108598882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578536cc790 0x5578536de910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.604+0000 7fcae9237640 1 -- 172.21.15.114:0/1108598882 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5578536d7950 msgr2=0x5578536db180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:58.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.604+0000 7fcae9237640 1 --2- 172.21.15.114:0/1108598882 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5578536d7950 0x5578536db180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.604+0000 7fcae9237640 1 -- 172.21.15.114:0/1108598882 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55785362e3c0 con 0x5578536cd0b0 2024-09-13T18:32:58.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.605+0000 7fcae9237640 1 --2- 172.21.15.114:0/1108598882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578536cd0b0 0x5578536dac40 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fcae001a000 tx=0x7fcae0004680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44160259f1920874 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:58.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.605+0000 7fcacd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:32:58.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.607+0000 7fcacd7fa640 1 -- 172.21.15.114:0/1108598882 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae0023070 con 0x5578536cd0b0 2024-09-13T18:32:58.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.608+0000 7fcaee9c5740 1 -- 172.21.15.114:0/1108598882 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcae0015070 con 0x5578536cd0b0 2024-09-13T18:32:58.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.608+0000 7fcacd7fa640 1 -- 172.21.15.114:0/1108598882 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fcae0017070 con 0x5578536cd0b0 2024-09-13T18:32:58.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.611+0000 7fcacd7fa640 1 -- 172.21.15.114:0/1108598882 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcae0004a70 con 0x5578536cd0b0 2024-09-13T18:32:58.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.611+0000 7fcaee9c5740 10 client.27866 fetch_fsmap learned FSMap version 34 2024-09-13T18:32:58.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.611+0000 7fcaee9c5740 1 -- 172.21.15.114:0/1108598882 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5578535f5e80 con 0x5578536cd0b0 2024-09-13T18:32:58.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.612+0000 7fcacd7fa640 1 -- 172.21.15.114:0/1108598882 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcae0015070 con 0x5578536cd0b0 2024-09-13T18:32:58.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.612+0000 7fcacd7fa640 1 -- 172.21.15.114:0/1108598882 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcae0026020 con 0x5578536cd0b0 2024-09-13T18:32:58.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.612+0000 7fcaee9c5740 10 client.27866 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:32:58.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.612+0000 7fcaee9c5740 10 client.27866 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-13T18:32:58.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.612+0000 7fcaee9c5740 20 client.27866 populate_metadata read hostname 'smithi114' 2024-09-13T18:32:58.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.612+0000 7fcaee9c5740 1 --2- 172.21.15.114:0/1108598882 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x557853760810 0x557853762c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:58.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.612+0000 7fcaee9c5740 4 client.27866 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:32:58.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.612+0000 7fcaee9c5740 1 -- 172.21.15.114:0/1108598882 --> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5578536221f0 con 0x557853760810 2024-09-13T18:32:58.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:58 smithi134 ceph-mon[34283]: pgmap v325: 129 pgs: 129 active+clean; 21 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 34 MiB/s wr, 751 op/s 2024-09-13T18:32:58.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:32:58 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/4220713879' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:32:58.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.617+0000 7fcae9a38640 1 --2- 172.21.15.114:0/1108598882 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x557853760810 0x557853762c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:58.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.622+0000 7fcae9a38640 1 --2- 172.21.15.114:0/1108598882 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x557853760810 0x557853762c30 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a014f44dff14f84b server_cookie=d20b2ad06e27c20 in_seq=0 out_seq=0 2024-09-13T18:32:58.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.622+0000 7fcacd7fa640 10 client.27866 ms_handle_connect on v2:172.21.15.134:6834/1150003275 2024-09-13T18:32:58.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.622+0000 7fcacd7fa640 1 -- 172.21.15.114:0/1108598882 <== mds.3 v2:172.21.15.134:6834/1150003275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5578536221f0 con 0x557853760810 2024-09-13T18:32:58.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.622+0000 7fcacd7fa640 10 client.27866 handle_command_reply: tid=0 2024-09-13T18:32:58.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.622+0000 7fcaee9c5740 1 client.27866 shutdown 2024-09-13T18:32:58.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.627+0000 7fcaee9c5740 1 -- 172.21.15.114:0/1108598882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578536cd0b0 msgr2=0x5578536dac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:58.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.627+0000 7fcaee9c5740 1 --2- 172.21.15.114:0/1108598882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578536cd0b0 0x5578536dac40 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fcae001a000 tx=0x7fcae0004680 comp rx=0 tx=0).stop 2024-09-13T18:32:58.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.628+0000 7fcaee9c5740 1 -- 172.21.15.114:0/1108598882 shutdown_connections 2024-09-13T18:32:58.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.628+0000 7fcaee9c5740 1 --2- 172.21.15.114:0/1108598882 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x557853760810 0x557853762c30 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.628+0000 7fcaee9c5740 1 --2- 172.21.15.114:0/1108598882 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5578536d7950 0x5578536db180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.628+0000 7fcaee9c5740 1 --2- 172.21.15.114:0/1108598882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578536cd0b0 0x5578536dac40 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.628+0000 7fcaee9c5740 1 --2- 172.21.15.114:0/1108598882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578536cc790 0x5578536de910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.628+0000 7fcaee9c5740 1 -- 172.21.15.114:0/1108598882 >> 172.21.15.114:0/1108598882 conn(0x5578536c8d00 msgr2=0x5578536d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:58.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.630+0000 7fcacd7fa640 0 client.27866 ms_handle_reset on v2:172.21.15.134:6834/1150003275 2024-09-13T18:32:58.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.631+0000 7fcaee9c5740 1 -- 172.21.15.114:0/1108598882 shutdown_connections 2024-09-13T18:32:58.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.635+0000 7fcaee9c5740 1 -- 172.21.15.114:0/1108598882 wait complete. 2024-09-13T18:32:58.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.639+0000 7fcaee9c5740 20 client.27866 trim_cache size 0 max 16384 2024-09-13T18:32:58.643 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:32:58.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.641+0000 7fcaebc9c640 1 -- 172.21.15.114:0/4139803403 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fcad00a11e0 msgr2=0x7fcad00a36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:58.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.641+0000 7fcaebc9c640 1 --2- 172.21.15.114:0/4139803403 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fcad00a11e0 0x7fcad00a36a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fcad4006fd0 tx=0x7fcad4008040 comp rx=0 tx=0).stop 2024-09-13T18:32:58.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.641+0000 7fcaebc9c640 1 -- 172.21.15.114:0/4139803403 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcae4119400 msgr2=0x7fcae410fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:58.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.641+0000 7fcaebc9c640 1 --2- 172.21.15.114:0/4139803403 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcae4119400 0x7fcae410fa70 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fcadc006010 tx=0x7fcadc031a20 comp rx=0 tx=0).stop 2024-09-13T18:32:58.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.641+0000 7fcaebc9c640 1 -- 172.21.15.114:0/4139803403 shutdown_connections 2024-09-13T18:32:58.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.641+0000 7fcaebc9c640 1 --2- 172.21.15.114:0/4139803403 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fcad00a11e0 0x7fcad00a36a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.641+0000 7fcaebc9c640 1 --2- 172.21.15.114:0/4139803403 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcae4119400 0x7fcae410fa70 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.641+0000 7fcaebc9c640 1 --2- 172.21.15.114:0/4139803403 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcae4107920 0x7fcae4116de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.641+0000 7fcaebc9c640 1 --2- 172.21.15.114:0/4139803403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcae4107540 0x7fcae41168a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.641+0000 7fcaebc9c640 1 -- 172.21.15.114:0/4139803403 >> 172.21.15.114:0/4139803403 conn(0x7fcae406afe0 msgr2=0x7fcae4119b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:58.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.644+0000 7fcaebc9c640 1 -- 172.21.15.114:0/4139803403 shutdown_connections 2024-09-13T18:32:58.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.646+0000 7fcaebc9c640 1 -- 172.21.15.114:0/4139803403 wait complete. 2024-09-13T18:32:58.698 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-09-13T18:32:58.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.917+0000 7fc033287640 1 -- 172.21.15.114:0/433505827 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc0240a64b0 msgr2=0x7fc0240a88a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:58.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.917+0000 7fc033287640 1 --2- 172.21.15.114:0/433505827 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc0240a64b0 0x7fc0240a88a0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fc02c063b50 tx=0x7fc02c094aa0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.918+0000 7fc033287640 1 -- 172.21.15.114:0/433505827 shutdown_connections 2024-09-13T18:32:58.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.918+0000 7fc033287640 1 --2- 172.21.15.114:0/433505827 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc0240a64b0 0x7fc0240a88a0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.918+0000 7fc033287640 1 --2- 172.21.15.114:0/433505827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc02400a250 0x7fc0240a5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.918+0000 7fc033287640 1 --2- 172.21.15.114:0/433505827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc024009930 0x7fc024009d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.918+0000 7fc033287640 1 -- 172.21.15.114:0/433505827 >> 172.21.15.114:0/433505827 conn(0x7fc02400e620 msgr2=0x7fc024095b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:58.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.930+0000 7fc033287640 1 -- 172.21.15.114:0/433505827 shutdown_connections 2024-09-13T18:32:58.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.936+0000 7fc033287640 1 -- 172.21.15.114:0/433505827 wait complete. 2024-09-13T18:32:58.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.936+0000 7fc033287640 1 Processor -- start 2024-09-13T18:32:58.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.936+0000 7fc033287640 1 -- start start 2024-09-13T18:32:58.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.937+0000 7fc033287640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc024009930 0x7fc0240a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:58.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.937+0000 7fc033287640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc02400a250 0x7fc0240a5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:58.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.937+0000 7fc033287640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc0240a64b0 0x7fc02409fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:58.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.937+0000 7fc033287640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0240a03a0 con 0x7fc0240a64b0 2024-09-13T18:32:58.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.937+0000 7fc033287640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0240a0510 con 0x7fc02400a250 2024-09-13T18:32:58.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.937+0000 7fc033287640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0240a06d0 con 0x7fc024009930 2024-09-13T18:32:58.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.941+0000 7fc031824640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc0240a64b0 0x7fc02409fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:58.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.941+0000 7fc031824640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc0240a64b0 0x7fc02409fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41612/0 (socket says 172.21.15.114:41612) 2024-09-13T18:32:58.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.941+0000 7fc031824640 1 -- 172.21.15.114:0/1181389611 learned_addr learned my addr 172.21.15.114:0/1181389611 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:58.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.944+0000 7fc031023640 1 --2- 172.21.15.114:0/1181389611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc024009930 0x7fc0240a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:58.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.944+0000 7fc030822640 1 --2- 172.21.15.114:0/1181389611 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc02400a250 0x7fc0240a5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:58.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.947+0000 7fc031824640 1 -- 172.21.15.114:0/1181389611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc024009930 msgr2=0x7fc0240a59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:58.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.947+0000 7fc031824640 1 --2- 172.21.15.114:0/1181389611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc024009930 0x7fc0240a59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.947+0000 7fc031824640 1 -- 172.21.15.114:0/1181389611 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc02400a250 msgr2=0x7fc0240a5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:58.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.947+0000 7fc031824640 1 --2- 172.21.15.114:0/1181389611 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc02400a250 0x7fc0240a5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.947+0000 7fc031824640 1 -- 172.21.15.114:0/1181389611 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc02c04eac0 con 0x7fc0240a64b0 2024-09-13T18:32:58.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.948+0000 7fc031824640 1 --2- 172.21.15.114:0/1181389611 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc0240a64b0 0x7fc02409fc10 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fc02c000b60 tx=0x7fc02c075300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:58.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.949+0000 7fc0227fc640 1 -- 172.21.15.114:0/1181389611 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02c068040 con 0x7fc0240a64b0 2024-09-13T18:32:58.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.949+0000 7fc033287640 1 -- 172.21.15.114:0/1181389611 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0240b1560 con 0x7fc0240a64b0 2024-09-13T18:32:58.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.950+0000 7fc033287640 1 -- 172.21.15.114:0/1181389611 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0240b1a40 con 0x7fc0240a64b0 2024-09-13T18:32:58.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.950+0000 7fc0227fc640 1 -- 172.21.15.114:0/1181389611 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc02c068630 con 0x7fc0240a64b0 2024-09-13T18:32:58.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.950+0000 7fc0227fc640 1 -- 172.21.15.114:0/1181389611 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02c0a3e00 con 0x7fc0240a64b0 2024-09-13T18:32:58.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.957+0000 7fc0227fc640 1 -- 172.21.15.114:0/1181389611 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fc02c096040 con 0x7fc0240a64b0 2024-09-13T18:32:58.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.958+0000 7fc0227fc640 1 --2- 172.21.15.114:0/1181389611 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc0180a1410 0x7fc0180a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:58.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.958+0000 7fc0227fc640 1 -- 172.21.15.114:0/1181389611 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fc02c158840 con 0x7fc0240a64b0 2024-09-13T18:32:58.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.958+0000 7fc036083740 1 Processor -- start 2024-09-13T18:32:58.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.959+0000 7fc036083740 1 -- start start 2024-09-13T18:32:58.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.959+0000 7fc036083740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568cb7b8970 0x5568cb7b8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:58.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.959+0000 7fc036083740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568cb7b9290 0x5568cb7c35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:58.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.959+0000 7fc036083740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568cb7c3b30 0x5568cb7c5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:58.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.959+0000 7fc036083740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568cb722cc0 con 0x5568cb7b9290 2024-09-13T18:32:58.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.959+0000 7fc036083740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568cb720450 con 0x5568cb7b8970 2024-09-13T18:32:58.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.959+0000 7fc036083740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568cb71bd40 con 0x5568cb7c3b30 2024-09-13T18:32:58.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.960+0000 7fc031023640 1 --2- 172.21.15.114:0/1181389611 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc0180a1410 0x7fc0180a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:58.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.960+0000 7fc031023640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568cb7b8970 0x5568cb7b8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:58.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.960+0000 7fc031023640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568cb7b8970 0x5568cb7b8d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:47726/0 (socket says 172.21.15.114:47726) 2024-09-13T18:32:58.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.960+0000 7fc031023640 1 -- 172.21.15.114:0/2706711167 learned_addr learned my addr 172.21.15.114:0/2706711167 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:58.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.960+0000 7fc031023640 1 -- 172.21.15.114:0/2706711167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568cb7c3b30 msgr2=0x5568cb7c5f20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:32:58.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.960+0000 7fc031023640 1 --2- 172.21.15.114:0/2706711167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568cb7c3b30 0x5568cb7c5f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.960+0000 7fc031023640 1 -- 172.21.15.114:0/2706711167 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568cb7b9290 msgr2=0x5568cb7c35f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:32:58.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.960+0000 7fc031023640 1 --2- 172.21.15.114:0/2706711167 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568cb7b9290 0x5568cb7c35f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.960+0000 7fc031023640 1 -- 172.21.15.114:0/2706711167 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568cb6e3620 con 0x5568cb7b8970 2024-09-13T18:32:58.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.961+0000 7fc031023640 1 --2- 172.21.15.114:0/2706711167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568cb7b8970 0x5568cb7b8d50 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc028017620 tx=0x7fc028018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8982c319a9cd4ef server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:58.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.961+0000 7fc031023640 1 --2- 172.21.15.114:0/1181389611 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc0180a1410 0x7fc0180a38d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc0280154f0 tx=0x7fc02800c810 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:58.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.962+0000 7fc0177fe640 1 -- 172.21.15.114:0/2706711167 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc028018680 con 0x5568cb7b8970 2024-09-13T18:32:58.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.962+0000 7fc0177fe640 1 -- 172.21.15.114:0/2706711167 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc028018c70 con 0x5568cb7b8970 2024-09-13T18:32:58.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.962+0000 7fc0177fe640 1 -- 172.21.15.114:0/2706711167 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0280239c0 con 0x5568cb7b8970 2024-09-13T18:32:58.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.963+0000 7fc036083740 1 -- 172.21.15.114:0/2706711167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568cb7b8970 msgr2=0x5568cb7b8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:58.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.963+0000 7fc036083740 1 --2- 172.21.15.114:0/2706711167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568cb7b8970 0x5568cb7b8d50 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc028017620 tx=0x7fc028018000 comp rx=0 tx=0).stop 2024-09-13T18:32:58.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.968+0000 7fc036083740 1 -- 172.21.15.114:0/2706711167 shutdown_connections 2024-09-13T18:32:58.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.968+0000 7fc036083740 1 --2- 172.21.15.114:0/2706711167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568cb7c3b30 0x5568cb7c5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.968+0000 7fc036083740 1 --2- 172.21.15.114:0/2706711167 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568cb7b9290 0x5568cb7c35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.968+0000 7fc036083740 1 --2- 172.21.15.114:0/2706711167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568cb7b8970 0x5568cb7b8d50 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.968+0000 7fc036083740 1 -- 172.21.15.114:0/2706711167 >> 172.21.15.114:0/2706711167 conn(0x5568cb7b4ee0 msgr2=0x5568cb7b52c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:58.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.981+0000 7fc036083740 1 -- 172.21.15.114:0/2706711167 shutdown_connections 2024-09-13T18:32:58.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.988+0000 7fc036083740 1 -- 172.21.15.114:0/2706711167 wait complete. 2024-09-13T18:32:58.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.989+0000 7fc036083740 1 -- start start 2024-09-13T18:32:58.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.989+0000 7fc036083740 1 Processor -- start 2024-09-13T18:32:58.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.989+0000 7fc036083740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568cb7b8970 0x5568cb7c3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:58.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.989+0000 7fc036083740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568cb7b9290 0x5568cb7c35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:58.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.989+0000 7fc036083740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568cb7c3b30 0x5568cb7c0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:58.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.989+0000 7fc036083740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568cb6da180 con 0x5568cb7b9290 2024-09-13T18:32:58.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.989+0000 7fc036083740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568cb3f4430 con 0x5568cb7c3b30 2024-09-13T18:32:58.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.989+0000 7fc036083740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568cb69f5e0 con 0x5568cb7b8970 2024-09-13T18:32:58.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.989+0000 7fc036083740 10 client.0 _ll_register_callbacks cb 0x5568cb70baa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:32:58.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.990+0000 7fc031824640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568cb7c3b30 0x5568cb7c0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:58.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.990+0000 7fc031824640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568cb7c3b30 0x5568cb7c0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:47732/0 (socket says 172.21.15.114:47732) 2024-09-13T18:32:58.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.990+0000 7fc031824640 1 -- 172.21.15.114:0/253984705 learned_addr learned my addr 172.21.15.114:0/253984705 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:58.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.990+0000 7fc031023640 1 --2- 172.21.15.114:0/253984705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568cb7b8970 0x5568cb7c3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:58.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.991+0000 7fc031824640 1 -- 172.21.15.114:0/253984705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568cb7b8970 msgr2=0x5568cb7c3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:58.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.991+0000 7fc031824640 1 --2- 172.21.15.114:0/253984705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568cb7b8970 0x5568cb7c3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.991+0000 7fc031824640 1 -- 172.21.15.114:0/253984705 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568cb7b9290 msgr2=0x5568cb7c35d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:32:58.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.991+0000 7fc031824640 1 --2- 172.21.15.114:0/253984705 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568cb7b9290 0x5568cb7c35d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:58.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.991+0000 7fc031824640 1 -- 172.21.15.114:0/253984705 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0280173e0 con 0x5568cb7c3b30 2024-09-13T18:32:58.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.991+0000 7fc031824640 1 --2- 172.21.15.114:0/253984705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568cb7c3b30 0x5568cb7c0630 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc02c155bf0 tx=0x7fc02c125180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5e4c1df435882f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:59.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.992+0000 7fc014ff9640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:32:59.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.992+0000 7fc014ff9640 1 -- 172.21.15.114:0/253984705 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02c156010 con 0x5568cb7c3b30 2024-09-13T18:32:59.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.992+0000 7fc014ff9640 1 -- 172.21.15.114:0/253984705 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fc02c129150 con 0x5568cb7c3b30 2024-09-13T18:32:59.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.994+0000 7fc036083740 1 -- 172.21.15.114:0/253984705 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5568cb745b80 con 0x5568cb7c3b30 2024-09-13T18:32:59.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:58.998+0000 7fc014ff9640 1 -- 172.21.15.114:0/253984705 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc02c129380 con 0x5568cb7c3b30 2024-09-13T18:32:59.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.002+0000 7fc036083740 10 client.27872 fetch_fsmap learned FSMap version 34 2024-09-13T18:32:59.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.002+0000 7fc036083740 1 -- 172.21.15.114:0/253984705 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5568cb71a3c0 con 0x5568cb7c3b30 2024-09-13T18:32:59.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.004+0000 7fc014ff9640 1 -- 172.21.15.114:0/253984705 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5568cb745b80 con 0x5568cb7c3b30 2024-09-13T18:32:59.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.004+0000 7fc014ff9640 1 -- 172.21.15.114:0/253984705 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc02c12d650 con 0x5568cb7c3b30 2024-09-13T18:32:59.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.005+0000 7fc036083740 10 client.27872 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:32:59.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.005+0000 7fc036083740 10 client.27872 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-13T18:32:59.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.005+0000 7fc036083740 20 client.27872 populate_metadata read hostname 'smithi114' 2024-09-13T18:32:59.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.005+0000 7fc036083740 1 --2- 172.21.15.114:0/253984705 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x5568cb7d31f0 0x5568cb7d5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:59.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.005+0000 7fc036083740 4 client.27872 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:32:59.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.005+0000 7fc036083740 1 -- 172.21.15.114:0/253984705 --> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5568cb70e1f0 con 0x5568cb7d31f0 2024-09-13T18:32:59.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.018+0000 7fc031023640 1 --2- 172.21.15.114:0/253984705 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x5568cb7d31f0 0x5568cb7d5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:59.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.042+0000 7fc031023640 1 --2- 172.21.15.114:0/253984705 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x5568cb7d31f0 0x5568cb7d5610 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d4b369bdcafc3597 server_cookie=5e8553860c495fef in_seq=0 out_seq=0 2024-09-13T18:32:59.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.046+0000 7fc014ff9640 10 client.27872 ms_handle_connect on v2:172.21.15.204:6838/344883912 2024-09-13T18:32:59.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.046+0000 7fc014ff9640 1 -- 172.21.15.114:0/253984705 <== mds.2 v2:172.21.15.204:6838/344883912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5568cb70e1f0 con 0x5568cb7d31f0 2024-09-13T18:32:59.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.046+0000 7fc014ff9640 10 client.27872 handle_command_reply: tid=0 2024-09-13T18:32:59.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.046+0000 7fc036083740 1 client.27872 shutdown 2024-09-13T18:32:59.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.050+0000 7fc036083740 1 -- 172.21.15.114:0/253984705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568cb7c3b30 msgr2=0x5568cb7c0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:59.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.050+0000 7fc036083740 1 --2- 172.21.15.114:0/253984705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568cb7c3b30 0x5568cb7c0630 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc02c155bf0 tx=0x7fc02c125180 comp rx=0 tx=0).stop 2024-09-13T18:32:59.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.051+0000 7fc036083740 1 -- 172.21.15.114:0/253984705 shutdown_connections 2024-09-13T18:32:59.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.051+0000 7fc036083740 1 --2- 172.21.15.114:0/253984705 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x5568cb7d31f0 0x5568cb7d5610 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.051+0000 7fc036083740 1 --2- 172.21.15.114:0/253984705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568cb7c3b30 0x5568cb7c0630 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.051+0000 7fc036083740 1 --2- 172.21.15.114:0/253984705 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568cb7b9290 0x5568cb7c35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.051+0000 7fc036083740 1 --2- 172.21.15.114:0/253984705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568cb7b8970 0x5568cb7c3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.051+0000 7fc036083740 1 -- 172.21.15.114:0/253984705 >> 172.21.15.114:0/253984705 conn(0x5568cb7b4ee0 msgr2=0x5568cb7b4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:59.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.052+0000 7fc014ff9640 0 client.27872 ms_handle_reset on v2:172.21.15.204:6838/344883912 2024-09-13T18:32:59.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.059+0000 7fc036083740 1 -- 172.21.15.114:0/253984705 shutdown_connections 2024-09-13T18:32:59.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.064+0000 7fc036083740 1 -- 172.21.15.114:0/253984705 wait complete. 2024-09-13T18:32:59.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.067+0000 7fc036083740 20 client.27872 trim_cache size 0 max 16384 2024-09-13T18:32:59.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.095+0000 7fc036083740 1 Processor -- start 2024-09-13T18:32:59.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.099+0000 7fc036083740 1 -- start start 2024-09-13T18:32:59.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.099+0000 7fc036083740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568cb7b8970 0x5568cb7d9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:59.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.099+0000 7fc036083740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568cb7b9290 0x5568cb7ca840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:59.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.099+0000 7fc036083740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568cb7c3b30 0x5568cb7cad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:59.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.099+0000 7fc036083740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568cb7af6f0 con 0x5568cb7b9290 2024-09-13T18:32:59.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.099+0000 7fc036083740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568cb7c65e0 con 0x5568cb7b8970 2024-09-13T18:32:59.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.099+0000 7fc036083740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568cb7b9930 con 0x5568cb7c3b30 2024-09-13T18:32:59.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.099+0000 7fc031023640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568cb7b8970 0x5568cb7d9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:59.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.099+0000 7fc031023640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568cb7b8970 0x5568cb7d9f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:47744/0 (socket says 172.21.15.114:47744) 2024-09-13T18:32:59.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.099+0000 7fc031023640 1 -- 172.21.15.114:0/2883909105 learned_addr learned my addr 172.21.15.114:0/2883909105 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:59.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.100+0000 7fc031023640 1 -- 172.21.15.114:0/2883909105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568cb7c3b30 msgr2=0x5568cb7cad80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:32:59.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.100+0000 7fc031023640 1 --2- 172.21.15.114:0/2883909105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568cb7c3b30 0x5568cb7cad80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.100+0000 7fc031023640 1 -- 172.21.15.114:0/2883909105 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568cb7b9290 msgr2=0x5568cb7ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:59.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.100+0000 7fc031023640 1 --2- 172.21.15.114:0/2883909105 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568cb7b9290 0x5568cb7ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.100+0000 7fc031023640 1 -- 172.21.15.114:0/2883909105 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568cb6bc770 con 0x5568cb7b8970 2024-09-13T18:32:59.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.100+0000 7fc031023640 1 --2- 172.21.15.114:0/2883909105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568cb7b8970 0x5568cb7d9f30 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc028017620 tx=0x7fc028008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c68eef1989ae8608 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:59.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.103+0000 7fc017fff640 1 -- 172.21.15.114:0/2883909105 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02801d040 con 0x5568cb7b8970 2024-09-13T18:32:59.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.103+0000 7fc017fff640 1 -- 172.21.15.114:0/2883909105 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc028023450 con 0x5568cb7b8970 2024-09-13T18:32:59.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.103+0000 7fc017fff640 1 -- 172.21.15.114:0/2883909105 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc028027040 con 0x5568cb7b8970 2024-09-13T18:32:59.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.106+0000 7fc036083740 1 -- 172.21.15.114:0/2883909105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568cb7b8970 msgr2=0x5568cb7d9f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:59.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.106+0000 7fc036083740 1 --2- 172.21.15.114:0/2883909105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568cb7b8970 0x5568cb7d9f30 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc028017620 tx=0x7fc028008040 comp rx=0 tx=0).stop 2024-09-13T18:32:59.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.110+0000 7fc036083740 1 -- 172.21.15.114:0/2883909105 shutdown_connections 2024-09-13T18:32:59.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.110+0000 7fc036083740 1 --2- 172.21.15.114:0/2883909105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568cb7c3b30 0x5568cb7cad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.110+0000 7fc036083740 1 --2- 172.21.15.114:0/2883909105 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568cb7b9290 0x5568cb7ca840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.110+0000 7fc036083740 1 --2- 172.21.15.114:0/2883909105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568cb7b8970 0x5568cb7d9f30 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.110+0000 7fc036083740 1 -- 172.21.15.114:0/2883909105 >> 172.21.15.114:0/2883909105 conn(0x5568cb7b4ee0 msgr2=0x5568cb7cf2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:59.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.115+0000 7fc036083740 1 -- 172.21.15.114:0/2883909105 shutdown_connections 2024-09-13T18:32:59.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.117+0000 7fc036083740 1 -- 172.21.15.114:0/2883909105 wait complete. 2024-09-13T18:32:59.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.117+0000 7fc036083740 1 -- start start 2024-09-13T18:32:59.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.117+0000 7fc036083740 1 Processor -- start 2024-09-13T18:32:59.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.118+0000 7fc036083740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568cb7b8970 0x5568cb7ca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:59.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.118+0000 7fc036083740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568cb7b9290 0x5568cb7cad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:59.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.118+0000 7fc036083740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568cb7c3b30 0x5568cb8258f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:59.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.118+0000 7fc036083740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568cb825ff0 con 0x5568cb7b8970 2024-09-13T18:32:59.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.118+0000 7fc036083740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568cb826160 con 0x5568cb7b9290 2024-09-13T18:32:59.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.118+0000 7fc036083740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568cb826320 con 0x5568cb7c3b30 2024-09-13T18:32:59.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.118+0000 7fc036083740 10 client.0 _ll_register_callbacks cb 0x7fc02800cc60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:32:59.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.119+0000 7fc031023640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568cb7b8970 0x5568cb7ca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:59.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.119+0000 7fc031023640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568cb7b8970 0x5568cb7ca7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41642/0 (socket says 172.21.15.114:41642) 2024-09-13T18:32:59.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.119+0000 7fc031023640 1 -- 172.21.15.114:0/1951336129 learned_addr learned my addr 172.21.15.114:0/1951336129 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:59.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.120+0000 7fc031023640 1 -- 172.21.15.114:0/1951336129 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568cb7c3b30 msgr2=0x5568cb8258f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:59.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.120+0000 7fc031023640 1 --2- 172.21.15.114:0/1951336129 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568cb7c3b30 0x5568cb8258f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.120+0000 7fc031023640 1 -- 172.21.15.114:0/1951336129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568cb7b9290 msgr2=0x5568cb7cad10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:32:59.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.120+0000 7fc031023640 1 --2- 172.21.15.114:0/1951336129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568cb7b9290 0x5568cb7cad10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.120+0000 7fc031023640 1 -- 172.21.15.114:0/1951336129 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5568cb6e3620 con 0x5568cb7b8970 2024-09-13T18:32:59.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.121+0000 7fc031023640 1 --2- 172.21.15.114:0/1951336129 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568cb7b8970 0x5568cb7ca7d0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fc028035040 tx=0x7fc02800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c468f674fc7876c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:59.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.127+0000 7fc014ff9640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:32:59.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.127+0000 7fc014ff9640 1 -- 172.21.15.114:0/1951336129 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc028027040 con 0x5568cb7b8970 2024-09-13T18:32:59.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.127+0000 7fc014ff9640 1 -- 172.21.15.114:0/1951336129 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fc028018480 con 0x5568cb7b8970 2024-09-13T18:32:59.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.127+0000 7fc036083740 1 -- 172.21.15.114:0/1951336129 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc028018c70 con 0x5568cb7b8970 2024-09-13T18:32:59.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.130+0000 7fc014ff9640 1 -- 172.21.15.114:0/1951336129 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc028018660 con 0x5568cb7b8970 2024-09-13T18:32:59.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.131+0000 7fc036083740 10 client.18393 fetch_fsmap learned FSMap version 34 2024-09-13T18:32:59.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.131+0000 7fc036083740 1 -- 172.21.15.114:0/1951336129 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5568cb6e1e80 con 0x5568cb7b8970 2024-09-13T18:32:59.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.132+0000 7fc014ff9640 1 -- 172.21.15.114:0/1951336129 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc028018c70 con 0x5568cb7b8970 2024-09-13T18:32:59.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.132+0000 7fc014ff9640 1 -- 172.21.15.114:0/1951336129 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc028043070 con 0x5568cb7b8970 2024-09-13T18:32:59.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.132+0000 7fc036083740 10 client.18393 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:32:59.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.132+0000 7fc036083740 10 client.18393 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-13T18:32:59.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.132+0000 7fc036083740 20 client.18393 populate_metadata read hostname 'smithi114' 2024-09-13T18:32:59.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.132+0000 7fc036083740 1 --2- 172.21.15.114:0/1951336129 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x5568cb83b6c0 0x5568cb83dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:59.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.132+0000 7fc036083740 4 client.18393 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:32:59.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.132+0000 7fc036083740 1 -- 172.21.15.114:0/1951336129 --> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5568cb40f4e0 con 0x5568cb83b6c0 2024-09-13T18:32:59.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.134+0000 7fc030822640 1 --2- 172.21.15.114:0/1951336129 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x5568cb83b6c0 0x5568cb83dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:59.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.136+0000 7fc030822640 1 --2- 172.21.15.114:0/1951336129 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x5568cb83b6c0 0x5568cb83dab0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ac097de5962d927f server_cookie=6122a5994ec2d51b in_seq=0 out_seq=0 2024-09-13T18:32:59.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.136+0000 7fc014ff9640 10 client.18393 ms_handle_connect on v2:172.21.15.204:6838/344883912 2024-09-13T18:32:59.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.136+0000 7fc014ff9640 1 -- 172.21.15.114:0/1951336129 <== mds.2 v2:172.21.15.204:6838/344883912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5568cb40f4e0 con 0x5568cb83b6c0 2024-09-13T18:32:59.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.136+0000 7fc014ff9640 10 client.18393 handle_command_reply: tid=0 2024-09-13T18:32:59.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.136+0000 7fc036083740 1 client.18393 shutdown 2024-09-13T18:32:59.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.139+0000 7fc036083740 1 -- 172.21.15.114:0/1951336129 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568cb7b8970 msgr2=0x5568cb7ca7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:59.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.139+0000 7fc036083740 1 --2- 172.21.15.114:0/1951336129 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568cb7b8970 0x5568cb7ca7d0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fc028035040 tx=0x7fc02800c910 comp rx=0 tx=0).stop 2024-09-13T18:32:59.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.140+0000 7fc036083740 1 -- 172.21.15.114:0/1951336129 shutdown_connections 2024-09-13T18:32:59.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.140+0000 7fc036083740 1 --2- 172.21.15.114:0/1951336129 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x5568cb83b6c0 0x5568cb83dab0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.140+0000 7fc036083740 1 --2- 172.21.15.114:0/1951336129 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568cb7c3b30 0x5568cb8258f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.140+0000 7fc036083740 1 --2- 172.21.15.114:0/1951336129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5568cb7b9290 0x5568cb7cad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.140+0000 7fc036083740 1 --2- 172.21.15.114:0/1951336129 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5568cb7b8970 0x5568cb7ca7d0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.140+0000 7fc036083740 1 -- 172.21.15.114:0/1951336129 >> 172.21.15.114:0/1951336129 conn(0x5568cb7b4ee0 msgr2=0x5568cb7cf2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:59.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.141+0000 7fc014ff9640 0 client.18393 ms_handle_reset on v2:172.21.15.204:6838/344883912 2024-09-13T18:32:59.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.141+0000 7fc036083740 1 -- 172.21.15.114:0/1951336129 shutdown_connections 2024-09-13T18:32:59.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.141+0000 7fc036083740 1 -- 172.21.15.114:0/1951336129 wait complete. 2024-09-13T18:32:59.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.145+0000 7fc036083740 20 client.18393 trim_cache size 0 max 16384 2024-09-13T18:32:59.149 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:32:59.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.149+0000 7fc033287640 1 -- 172.21.15.114:0/1181389611 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc0180a1410 msgr2=0x7fc0180a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:59.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.149+0000 7fc033287640 1 --2- 172.21.15.114:0/1181389611 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc0180a1410 0x7fc0180a38d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc0280154f0 tx=0x7fc02800c810 comp rx=0 tx=0).stop 2024-09-13T18:32:59.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.149+0000 7fc033287640 1 -- 172.21.15.114:0/1181389611 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc0240a64b0 msgr2=0x7fc02409fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:59.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.149+0000 7fc033287640 1 --2- 172.21.15.114:0/1181389611 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc0240a64b0 0x7fc02409fc10 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fc02c000b60 tx=0x7fc02c075300 comp rx=0 tx=0).stop 2024-09-13T18:32:59.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.149+0000 7fc033287640 1 -- 172.21.15.114:0/1181389611 shutdown_connections 2024-09-13T18:32:59.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.149+0000 7fc033287640 1 --2- 172.21.15.114:0/1181389611 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc0180a1410 0x7fc0180a38d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.149+0000 7fc033287640 1 --2- 172.21.15.114:0/1181389611 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc0240a64b0 0x7fc02409fc10 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.149+0000 7fc033287640 1 --2- 172.21.15.114:0/1181389611 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc02400a250 0x7fc0240a5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.149+0000 7fc033287640 1 --2- 172.21.15.114:0/1181389611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc024009930 0x7fc0240a59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.149+0000 7fc033287640 1 -- 172.21.15.114:0/1181389611 >> 172.21.15.114:0/1181389611 conn(0x7fc02400e620 msgr2=0x7fc024095b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:59.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.155+0000 7fc033287640 1 -- 172.21.15.114:0/1181389611 shutdown_connections 2024-09-13T18:32:59.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.155+0000 7fc033287640 1 -- 172.21.15.114:0/1181389611 wait complete. 2024-09-13T18:32:59.183 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-09-13T18:32:59.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.398+0000 7fe20e841640 1 -- 172.21.15.114:0/2739369872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe208119400 msgr2=0x7fe20811b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:59.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.398+0000 7fe20e841640 1 --2- 172.21.15.114:0/2739369872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe208119400 0x7fe20811b7f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe200009be0 tx=0x7fe20002a6d0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.402+0000 7fe20e841640 1 -- 172.21.15.114:0/2739369872 shutdown_connections 2024-09-13T18:32:59.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.402+0000 7fe20e841640 1 --2- 172.21.15.114:0/2739369872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe208119400 0x7fe20811b7f0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.402+0000 7fe20e841640 1 --2- 172.21.15.114:0/2739369872 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe208107920 0x7fe208107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.402+0000 7fe20e841640 1 --2- 172.21.15.114:0/2739369872 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe208107540 0x7fe20810e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.402+0000 7fe20e841640 1 -- 172.21.15.114:0/2739369872 >> 172.21.15.114:0/2739369872 conn(0x7fe20806afe0 msgr2=0x7fe20806b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:59.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.403+0000 7fe20e841640 1 -- 172.21.15.114:0/2739369872 shutdown_connections 2024-09-13T18:32:59.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.405+0000 7fe20e841640 1 -- 172.21.15.114:0/2739369872 wait complete. 2024-09-13T18:32:59.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.406+0000 7fe20e841640 1 Processor -- start 2024-09-13T18:32:59.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.406+0000 7fe20e841640 1 -- start start 2024-09-13T18:32:59.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.406+0000 7fe20e841640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe208107540 0x7fe2081168f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:59.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.406+0000 7fe20e841640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe208107920 0x7fe208116e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:59.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.406+0000 7fe20e841640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe208119400 0x7fe20810fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:59.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.406+0000 7fe20e841640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe208110330 con 0x7fe208119400 2024-09-13T18:32:59.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.406+0000 7fe20e841640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2081104a0 con 0x7fe208107540 2024-09-13T18:32:59.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.406+0000 7fe20e841640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe208110660 con 0x7fe208107920 2024-09-13T18:32:59.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.407+0000 7fe2077fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe208107920 0x7fe208116e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:59.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.407+0000 7fe2077fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe208107920 0x7fe208116e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:39380/0 (socket says 172.21.15.114:39380) 2024-09-13T18:32:59.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.407+0000 7fe2077fe640 1 -- 172.21.15.114:0/928289305 learned_addr learned my addr 172.21.15.114:0/928289305 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:59.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.407+0000 7fe20cdde640 1 --2- 172.21.15.114:0/928289305 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe208119400 0x7fe20810fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:59.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.408+0000 7fe2077fe640 1 -- 172.21.15.114:0/928289305 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe208107540 msgr2=0x7fe2081168f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:59.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.408+0000 7fe2077fe640 1 --2- 172.21.15.114:0/928289305 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe208107540 0x7fe2081168f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.408+0000 7fe2077fe640 1 -- 172.21.15.114:0/928289305 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe208119400 msgr2=0x7fe20810fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:59.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.408+0000 7fe2077fe640 1 --2- 172.21.15.114:0/928289305 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe208119400 0x7fe20810fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.408+0000 7fe2077fe640 1 -- 172.21.15.114:0/928289305 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe200009840 con 0x7fe208107920 2024-09-13T18:32:59.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.408+0000 7fe2077fe640 1 --2- 172.21.15.114:0/928289305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe208107920 0x7fe208116e50 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe1f800d950 tx=0x7fe1f800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:59.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.409+0000 7fe2057fa640 1 -- 172.21.15.114:0/928289305 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f8009c30 con 0x7fe208107920 2024-09-13T18:32:59.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.409+0000 7fe2057fa640 1 -- 172.21.15.114:0/928289305 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe1f8009d90 con 0x7fe208107920 2024-09-13T18:32:59.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.409+0000 7fe2057fa640 1 -- 172.21.15.114:0/928289305 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f800f620 con 0x7fe208107920 2024-09-13T18:32:59.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.411+0000 7fe20e841640 1 -- 172.21.15.114:0/928289305 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe208114480 con 0x7fe208107920 2024-09-13T18:32:59.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.411+0000 7fe20e841640 1 -- 172.21.15.114:0/928289305 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe208114940 con 0x7fe208107920 2024-09-13T18:32:59.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.416+0000 7fe211643740 1 Processor -- start 2024-09-13T18:32:59.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.416+0000 7fe211643740 1 -- start start 2024-09-13T18:32:59.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.416+0000 7fe211643740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572a4b21790 0x5572a4b21b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:59.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.416+0000 7fe211643740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572a4b220b0 0x5572a4b2c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:59.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.416+0000 7fe211643740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5572a4b2c950 0x5572a4b2ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:59.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.416+0000 7fe211643740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572a4a8bcc0 con 0x5572a4b2c950 2024-09-13T18:32:59.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.416+0000 7fe211643740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572a4a89450 con 0x5572a4b220b0 2024-09-13T18:32:59.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.416+0000 7fe211643740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572a4a84d40 con 0x5572a4b21790 2024-09-13T18:32:59.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.417+0000 7fe207fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572a4b220b0 0x5572a4b2c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:59.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.417+0000 7fe207fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572a4b220b0 0x5572a4b2c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:47776/0 (socket says 172.21.15.114:47776) 2024-09-13T18:32:59.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.417+0000 7fe207fff640 1 -- 172.21.15.114:0/503101686 learned_addr learned my addr 172.21.15.114:0/503101686 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:59.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.417+0000 7fe20cdde640 1 --2- 172.21.15.114:0/503101686 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572a4b21790 0x5572a4b21b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:59.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.418+0000 7fe207fff640 1 -- 172.21.15.114:0/503101686 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572a4b21790 msgr2=0x5572a4b21b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:59.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.418+0000 7fe207fff640 1 --2- 172.21.15.114:0/503101686 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572a4b21790 0x5572a4b21b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.418+0000 7fe207fff640 1 -- 172.21.15.114:0/503101686 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5572a4b2c950 msgr2=0x5572a4b2ed40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:32:59.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.418+0000 7fe207fff640 1 --2- 172.21.15.114:0/503101686 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5572a4b2c950 0x5572a4b2ed40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.418+0000 7fe207fff640 1 -- 172.21.15.114:0/503101686 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572a4a4c620 con 0x5572a4b220b0 2024-09-13T18:32:59.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.419+0000 7fe20cdde640 1 --2- 172.21.15.114:0/503101686 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572a4b21790 0x5572a4b21b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:32:59.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.419+0000 7fe2057fa640 1 -- 172.21.15.114:0/928289305 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fe1f8010460 con 0x7fe208107920 2024-09-13T18:32:59.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.420+0000 7fe2057fa640 1 --2- 172.21.15.114:0/928289305 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe1ec0a1390 0x7fe1ec0a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:59.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.420+0000 7fe2057fa640 1 -- 172.21.15.114:0/928289305 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fe1f80ccaf0 con 0x7fe208107920 2024-09-13T18:32:59.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.420+0000 7fe20cdde640 1 --2- 172.21.15.114:0/928289305 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe1ec0a1390 0x7fe1ec0a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:59.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.421+0000 7fe20cdde640 1 --2- 172.21.15.114:0/928289305 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe1ec0a1390 0x7fe1ec0a3850 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe200004af0 tx=0x7fe2000303c0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:59.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.421+0000 7fe207fff640 1 --2- 172.21.15.114:0/503101686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572a4b220b0 0x5572a4b2c410 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe1f400bb10 tx=0x7fe1f400d2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed67ff74d7cc7c7a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:59.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.424+0000 7fe1fe7fc640 1 -- 172.21.15.114:0/503101686 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f400dce0 con 0x5572a4b220b0 2024-09-13T18:32:59.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.424+0000 7fe1fe7fc640 1 -- 172.21.15.114:0/503101686 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe1f400de40 con 0x5572a4b220b0 2024-09-13T18:32:59.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.425+0000 7fe1fe7fc640 1 -- 172.21.15.114:0/503101686 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f4010460 con 0x5572a4b220b0 2024-09-13T18:32:59.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.425+0000 7fe211643740 1 -- 172.21.15.114:0/503101686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572a4b220b0 msgr2=0x5572a4b2c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:59.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.425+0000 7fe211643740 1 --2- 172.21.15.114:0/503101686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572a4b220b0 0x5572a4b2c410 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe1f400bb10 tx=0x7fe1f400d2e0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.425+0000 7fe211643740 1 -- 172.21.15.114:0/503101686 shutdown_connections 2024-09-13T18:32:59.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.425+0000 7fe211643740 1 --2- 172.21.15.114:0/503101686 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5572a4b2c950 0x5572a4b2ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.425+0000 7fe211643740 1 --2- 172.21.15.114:0/503101686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572a4b220b0 0x5572a4b2c410 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.425+0000 7fe211643740 1 --2- 172.21.15.114:0/503101686 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572a4b21790 0x5572a4b21b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.425+0000 7fe211643740 1 -- 172.21.15.114:0/503101686 >> 172.21.15.114:0/503101686 conn(0x5572a4b1dd00 msgr2=0x5572a4b1e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:59.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.425+0000 7fe211643740 1 -- 172.21.15.114:0/503101686 shutdown_connections 2024-09-13T18:32:59.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.425+0000 7fe211643740 1 -- 172.21.15.114:0/503101686 wait complete. 2024-09-13T18:32:59.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.426+0000 7fe211643740 1 -- start start 2024-09-13T18:32:59.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.426+0000 7fe211643740 1 Processor -- start 2024-09-13T18:32:59.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.431+0000 7fe211643740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572a4b21790 0x5572a4b2be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:59.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.431+0000 7fe211643740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572a4b220b0 0x5572a4b2c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:59.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.431+0000 7fe211643740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5572a4b2c950 0x5572a4b293e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:59.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.431+0000 7fe211643740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572a4a43180 con 0x5572a4b2c950 2024-09-13T18:32:59.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.431+0000 7fe211643740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572a475d430 con 0x5572a4b21790 2024-09-13T18:32:59.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.431+0000 7fe211643740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572a4a085e0 con 0x5572a4b220b0 2024-09-13T18:32:59.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.431+0000 7fe211643740 10 client.0 _ll_register_callbacks cb 0x5572a4a74aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:32:59.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.431+0000 7fe207fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572a4b21790 0x5572a4b2be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:59.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.431+0000 7fe207fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572a4b21790 0x5572a4b2be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:47784/0 (socket says 172.21.15.114:47784) 2024-09-13T18:32:59.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.431+0000 7fe2077fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572a4b220b0 0x5572a4b2c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:59.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.431+0000 7fe2077fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572a4b220b0 0x5572a4b2c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:39406/0 (socket says 172.21.15.114:39406) 2024-09-13T18:32:59.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.431+0000 7fe2077fe640 1 -- 172.21.15.114:0/1699280438 learned_addr learned my addr 172.21.15.114:0/1699280438 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:59.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.431+0000 7fe20cdde640 1 --2- 172.21.15.114:0/1699280438 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5572a4b2c950 0x5572a4b293e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:59.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.431+0000 7fe2077fe640 1 -- 172.21.15.114:0/1699280438 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572a4b21790 msgr2=0x5572a4b2be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:59.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.431+0000 7fe2077fe640 1 --2- 172.21.15.114:0/1699280438 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572a4b21790 0x5572a4b2be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.431+0000 7fe2077fe640 1 -- 172.21.15.114:0/1699280438 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5572a4b2c950 msgr2=0x5572a4b293e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:59.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.431+0000 7fe2077fe640 1 --2- 172.21.15.114:0/1699280438 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5572a4b2c950 0x5572a4b293e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.431+0000 7fe2077fe640 1 -- 172.21.15.114:0/1699280438 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1f400b740 con 0x5572a4b220b0 2024-09-13T18:32:59.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.432+0000 7fe1eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:32:59.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.432+0000 7fe2077fe640 1 --2- 172.21.15.114:0/1699280438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572a4b220b0 0x5572a4b2c3b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe1f80ca4a0 tx=0x7fe1f8099f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=719dbb670e7f9d8f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:59.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.432+0000 7fe1eb7fe640 1 -- 172.21.15.114:0/1699280438 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f80a1ce0 con 0x5572a4b220b0 2024-09-13T18:32:59.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.432+0000 7fe211643740 1 -- 172.21.15.114:0/1699280438 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5572a4aaeb80 con 0x5572a4b220b0 2024-09-13T18:32:59.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.432+0000 7fe1eb7fe640 1 -- 172.21.15.114:0/1699280438 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fe1f80a1850 con 0x5572a4b220b0 2024-09-13T18:32:59.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.432+0000 7fe1eb7fe640 1 -- 172.21.15.114:0/1699280438 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1f80a1a10 con 0x5572a4b220b0 2024-09-13T18:32:59.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.432+0000 7fe211643740 10 client.27799 fetch_fsmap learned FSMap version 34 2024-09-13T18:32:59.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.432+0000 7fe211643740 1 -- 172.21.15.114:0/1699280438 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5572a4a833c0 con 0x5572a4b220b0 2024-09-13T18:32:59.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.433+0000 7fe1eb7fe640 1 -- 172.21.15.114:0/1699280438 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5572a4aaeb80 con 0x5572a4b220b0 2024-09-13T18:32:59.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.433+0000 7fe1eb7fe640 1 -- 172.21.15.114:0/1699280438 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe1f80b9190 con 0x5572a4b220b0 2024-09-13T18:32:59.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.433+0000 7fe211643740 10 client.27799 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:32:59.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.433+0000 7fe211643740 10 client.27799 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-13T18:32:59.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.433+0000 7fe211643740 20 client.27799 populate_metadata read hostname 'smithi114' 2024-09-13T18:32:59.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.433+0000 7fe211643740 1 --2- 172.21.15.114:0/1699280438 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x5572a4b3c440 0x5572a4b3e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:59.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.433+0000 7fe211643740 4 client.27799 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:32:59.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.433+0000 7fe211643740 1 -- 172.21.15.114:0/1699280438 --> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5572a4a771f0 con 0x5572a4b3c440 2024-09-13T18:32:59.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.437+0000 7fe207fff640 1 --2- 172.21.15.114:0/1699280438 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x5572a4b3c440 0x5572a4b3e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:59.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.439+0000 7fe1eb7fe640 10 client.27799 ms_handle_connect on v2:172.21.15.134:6836/891692210 2024-09-13T18:32:59.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.439+0000 7fe207fff640 1 --2- 172.21.15.114:0/1699280438 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x5572a4b3c440 0x5572a4b3e860 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e0c207b2420a0822 server_cookie=756aa2bc821b40ac in_seq=0 out_seq=0 2024-09-13T18:32:59.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.440+0000 7fe1eb7fe640 1 -- 172.21.15.114:0/1699280438 <== mds.1 v2:172.21.15.134:6836/891692210 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5572a4a771f0 con 0x5572a4b3c440 2024-09-13T18:32:59.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.440+0000 7fe1eb7fe640 10 client.27799 handle_command_reply: tid=0 2024-09-13T18:32:59.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.440+0000 7fe211643740 1 client.27799 shutdown 2024-09-13T18:32:59.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.441+0000 7fe211643740 1 -- 172.21.15.114:0/1699280438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572a4b220b0 msgr2=0x5572a4b2c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:59.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.441+0000 7fe211643740 1 --2- 172.21.15.114:0/1699280438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572a4b220b0 0x5572a4b2c3b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe1f80ca4a0 tx=0x7fe1f8099f60 comp rx=0 tx=0).stop 2024-09-13T18:32:59.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.441+0000 7fe211643740 1 -- 172.21.15.114:0/1699280438 shutdown_connections 2024-09-13T18:32:59.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.441+0000 7fe211643740 1 --2- 172.21.15.114:0/1699280438 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x5572a4b3c440 0x5572a4b3e860 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.441+0000 7fe1eb7fe640 0 client.27799 ms_handle_reset on v2:172.21.15.134:6836/891692210 2024-09-13T18:32:59.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.441+0000 7fe211643740 1 --2- 172.21.15.114:0/1699280438 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5572a4b2c950 0x5572a4b293e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.441+0000 7fe211643740 1 --2- 172.21.15.114:0/1699280438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572a4b220b0 0x5572a4b2c3b0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.441+0000 7fe211643740 1 --2- 172.21.15.114:0/1699280438 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572a4b21790 0x5572a4b2be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.441+0000 7fe211643740 1 -- 172.21.15.114:0/1699280438 >> 172.21.15.114:0/1699280438 conn(0x5572a4b1dd00 msgr2=0x5572a4b1ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:59.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.441+0000 7fe211643740 1 -- 172.21.15.114:0/1699280438 shutdown_connections 2024-09-13T18:32:59.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.441+0000 7fe211643740 1 -- 172.21.15.114:0/1699280438 wait complete. 2024-09-13T18:32:59.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.442+0000 7fe211643740 20 client.27799 trim_cache size 0 max 16384 2024-09-13T18:32:59.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.474+0000 7fe211643740 1 Processor -- start 2024-09-13T18:32:59.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.474+0000 7fe211643740 1 -- start start 2024-09-13T18:32:59.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.475+0000 7fe211643740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572a4b21790 0x5572a4b33720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:59.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.475+0000 7fe211643740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572a4b220b0 0x5572a4b33c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:59.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.475+0000 7fe207fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572a4b21790 0x5572a4b33720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:59.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.475+0000 7fe207fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572a4b21790 0x5572a4b33720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:47790/0 (socket says 172.21.15.114:47790) 2024-09-13T18:32:59.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.475+0000 7fe207fff640 1 -- 172.21.15.114:0/1262486786 learned_addr learned my addr 172.21.15.114:0/1262486786 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:59.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.475+0000 7fe2077fe640 1 --2- 172.21.15.114:0/1262486786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572a4b220b0 0x5572a4b33c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:59.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.476+0000 7fe211643740 1 --2- 172.21.15.114:0/1262486786 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5572a4b2c950 0x5572a4b286b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:59.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.476+0000 7fe211643740 1 -- 172.21.15.114:0/1262486786 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572a4b1ec60 con 0x5572a4b2c950 2024-09-13T18:32:59.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.476+0000 7fe211643740 1 -- 172.21.15.114:0/1262486786 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572a4b2a2d0 con 0x5572a4b21790 2024-09-13T18:32:59.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.476+0000 7fe211643740 1 -- 172.21.15.114:0/1262486786 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572a4b2ba60 con 0x5572a4b220b0 2024-09-13T18:32:59.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.476+0000 7fe20cdde640 1 --2- 172.21.15.114:0/1262486786 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5572a4b2c950 0x5572a4b286b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:59.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.477+0000 7fe2077fe640 1 -- 172.21.15.114:0/1262486786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572a4b21790 msgr2=0x5572a4b33720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:59.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.477+0000 7fe2077fe640 1 --2- 172.21.15.114:0/1262486786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572a4b21790 0x5572a4b33720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.477+0000 7fe2077fe640 1 -- 172.21.15.114:0/1262486786 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5572a4b2c950 msgr2=0x5572a4b286b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:59.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.477+0000 7fe2077fe640 1 --2- 172.21.15.114:0/1262486786 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5572a4b2c950 0x5572a4b286b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.477+0000 7fe2077fe640 1 -- 172.21.15.114:0/1262486786 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572a4a25770 con 0x5572a4b220b0 2024-09-13T18:32:59.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.477+0000 7fe2077fe640 1 --2- 172.21.15.114:0/1262486786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572a4b220b0 0x5572a4b33c60 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fe1f80cabe0 tx=0x7fe1f8097e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c2bd7ff14359380 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:59.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.478+0000 7fe1feffd640 1 -- 172.21.15.114:0/1262486786 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f80a1a10 con 0x5572a4b220b0 2024-09-13T18:32:59.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.478+0000 7fe1feffd640 1 -- 172.21.15.114:0/1262486786 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe1f8095650 con 0x5572a4b220b0 2024-09-13T18:32:59.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.478+0000 7fe1feffd640 1 -- 172.21.15.114:0/1262486786 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f80b21f0 con 0x5572a4b220b0 2024-09-13T18:32:59.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.479+0000 7fe211643740 1 -- 172.21.15.114:0/1262486786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572a4b220b0 msgr2=0x5572a4b33c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:59.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.479+0000 7fe211643740 1 --2- 172.21.15.114:0/1262486786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572a4b220b0 0x5572a4b33c60 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fe1f80cabe0 tx=0x7fe1f8097e10 comp rx=0 tx=0).stop 2024-09-13T18:32:59.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.479+0000 7fe211643740 1 -- 172.21.15.114:0/1262486786 shutdown_connections 2024-09-13T18:32:59.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.479+0000 7fe211643740 1 --2- 172.21.15.114:0/1262486786 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5572a4b2c950 0x5572a4b286b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.479+0000 7fe211643740 1 --2- 172.21.15.114:0/1262486786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572a4b220b0 0x5572a4b33c60 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.479+0000 7fe211643740 1 --2- 172.21.15.114:0/1262486786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572a4b21790 0x5572a4b33720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.479+0000 7fe211643740 1 -- 172.21.15.114:0/1262486786 >> 172.21.15.114:0/1262486786 conn(0x5572a4b1dd00 msgr2=0x5572a4b43180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:59.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.482+0000 7fe211643740 1 -- 172.21.15.114:0/1262486786 shutdown_connections 2024-09-13T18:32:59.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.482+0000 7fe211643740 1 -- 172.21.15.114:0/1262486786 wait complete. 2024-09-13T18:32:59.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.482+0000 7fe211643740 1 -- start start 2024-09-13T18:32:59.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.482+0000 7fe211643740 1 Processor -- start 2024-09-13T18:32:59.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.483+0000 7fe211643740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5572a4b21790 0x5572a4b31d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:59.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.483+0000 7fe211643740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572a4b220b0 0x5572a4b322c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:59.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.483+0000 7fe211643740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572a4b2c950 0x5572a4b33160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:59.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.483+0000 7fe211643740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572a4b337e0 con 0x5572a4b21790 2024-09-13T18:32:59.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.483+0000 7fe211643740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572a4b33950 con 0x5572a4b2c950 2024-09-13T18:32:59.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.484+0000 7fe211643740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572a4b33b10 con 0x5572a4b220b0 2024-09-13T18:32:59.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.484+0000 7fe211643740 10 client.0 _ll_register_callbacks cb 0x7fe1f400b2a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:32:59.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.484+0000 7fe207fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5572a4b21790 0x5572a4b31d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:59.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.484+0000 7fe207fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5572a4b21790 0x5572a4b31d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41700/0 (socket says 172.21.15.114:41700) 2024-09-13T18:32:59.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.484+0000 7fe207fff640 1 -- 172.21.15.114:0/2976732035 learned_addr learned my addr 172.21.15.114:0/2976732035 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:59.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.484+0000 7fe207fff640 1 -- 172.21.15.114:0/2976732035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572a4b220b0 msgr2=0x5572a4b322c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:32:59.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.484+0000 7fe207fff640 1 --2- 172.21.15.114:0/2976732035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572a4b220b0 0x5572a4b322c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.484+0000 7fe207fff640 1 -- 172.21.15.114:0/2976732035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572a4b2c950 msgr2=0x5572a4b33160 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:32:59.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.484+0000 7fe207fff640 1 --2- 172.21.15.114:0/2976732035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572a4b2c950 0x5572a4b33160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.484+0000 7fe207fff640 1 -- 172.21.15.114:0/2976732035 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5572a4a833c0 con 0x5572a4b21790 2024-09-13T18:32:59.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.485+0000 7fe207fff640 1 --2- 172.21.15.114:0/2976732035 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5572a4b21790 0x5572a4b31d80 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fe1f400dfd0 tx=0x7fe1f4005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7407d2d18b592e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:59.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.485+0000 7fe1eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:32:59.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.485+0000 7fe1eb7fe640 1 -- 172.21.15.114:0/2976732035 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f4020040 con 0x5572a4b21790 2024-09-13T18:32:59.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.485+0000 7fe1eb7fe640 1 -- 172.21.15.114:0/2976732035 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fe1f4004240 con 0x5572a4b21790 2024-09-13T18:32:59.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.486+0000 7fe211643740 1 -- 172.21.15.114:0/2976732035 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe1f400de40 con 0x5572a4b21790 2024-09-13T18:32:59.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.486+0000 7fe1eb7fe640 1 -- 172.21.15.114:0/2976732035 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1f4024a90 con 0x5572a4b21790 2024-09-13T18:32:59.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.487+0000 7fe211643740 10 client.18408 fetch_fsmap learned FSMap version 34 2024-09-13T18:32:59.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.487+0000 7fe211643740 1 -- 172.21.15.114:0/2976732035 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5572a4a4ae80 con 0x5572a4b21790 2024-09-13T18:32:59.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.494+0000 7fe1eb7fe640 1 -- 172.21.15.114:0/2976732035 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5572a4a771f0 con 0x5572a4b21790 2024-09-13T18:32:59.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.494+0000 7fe1eb7fe640 1 -- 172.21.15.114:0/2976732035 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe1f402e070 con 0x5572a4b21790 2024-09-13T18:32:59.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.495+0000 7fe211643740 10 client.18408 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:32:59.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.495+0000 7fe211643740 10 client.18408 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-13T18:32:59.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.495+0000 7fe211643740 20 client.18408 populate_metadata read hostname 'smithi114' 2024-09-13T18:32:59.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.495+0000 7fe211643740 1 --2- 172.21.15.114:0/2976732035 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x5572a4b93a00 0x5572a4b95e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:59.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.495+0000 7fe211643740 4 client.18408 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:32:59.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.495+0000 7fe211643740 1 -- 172.21.15.114:0/2976732035 --> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fe1f400de40 con 0x5572a4b93a00 2024-09-13T18:32:59.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.496+0000 7fe2077fe640 1 --2- 172.21.15.114:0/2976732035 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x5572a4b93a00 0x5572a4b95e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:59.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.506+0000 7fe2077fe640 1 --2- 172.21.15.114:0/2976732035 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x5572a4b93a00 0x5572a4b95e20 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d502c0667b437c1d server_cookie=7b9a6f9841c1129d in_seq=0 out_seq=0 2024-09-13T18:32:59.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.506+0000 7fe1eb7fe640 10 client.18408 ms_handle_connect on v2:172.21.15.134:6836/891692210 2024-09-13T18:32:59.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.509+0000 7fe1eb7fe640 1 -- 172.21.15.114:0/2976732035 <== mds.1 v2:172.21.15.134:6836/891692210 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fe1f400de40 con 0x5572a4b93a00 2024-09-13T18:32:59.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.509+0000 7fe1eb7fe640 10 client.18408 handle_command_reply: tid=0 2024-09-13T18:32:59.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.509+0000 7fe211643740 1 client.18408 shutdown 2024-09-13T18:32:59.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.512+0000 7fe211643740 1 -- 172.21.15.114:0/2976732035 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5572a4b21790 msgr2=0x5572a4b31d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:59.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.512+0000 7fe211643740 1 --2- 172.21.15.114:0/2976732035 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5572a4b21790 0x5572a4b31d80 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fe1f400dfd0 tx=0x7fe1f4005c90 comp rx=0 tx=0).stop 2024-09-13T18:32:59.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.512+0000 7fe211643740 1 -- 172.21.15.114:0/2976732035 shutdown_connections 2024-09-13T18:32:59.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.512+0000 7fe211643740 1 --2- 172.21.15.114:0/2976732035 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x5572a4b93a00 0x5572a4b95e20 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.512+0000 7fe1eb7fe640 0 client.18408 ms_handle_reset on v2:172.21.15.134:6836/891692210 2024-09-13T18:32:59.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.512+0000 7fe211643740 1 --2- 172.21.15.114:0/2976732035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572a4b2c950 0x5572a4b33160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.512+0000 7fe211643740 1 --2- 172.21.15.114:0/2976732035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572a4b220b0 0x5572a4b322c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.512+0000 7fe211643740 1 --2- 172.21.15.114:0/2976732035 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5572a4b21790 0x5572a4b31d80 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.512+0000 7fe211643740 1 -- 172.21.15.114:0/2976732035 >> 172.21.15.114:0/2976732035 conn(0x5572a4b1dd00 msgr2=0x5572a4b27610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:59.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.515+0000 7fe211643740 1 -- 172.21.15.114:0/2976732035 shutdown_connections 2024-09-13T18:32:59.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.518+0000 7fe211643740 1 -- 172.21.15.114:0/2976732035 wait complete. 2024-09-13T18:32:59.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.519+0000 7fe211643740 20 client.18408 trim_cache size 0 max 16384 2024-09-13T18:32:59.523 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:32:59.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.529+0000 7fe20e841640 1 -- 172.21.15.114:0/928289305 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe1ec0a1390 msgr2=0x7fe1ec0a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:59.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.529+0000 7fe20e841640 1 --2- 172.21.15.114:0/928289305 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe1ec0a1390 0x7fe1ec0a3850 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe200004af0 tx=0x7fe2000303c0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.529+0000 7fe20e841640 1 -- 172.21.15.114:0/928289305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe208107920 msgr2=0x7fe208116e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:59.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.529+0000 7fe20e841640 1 --2- 172.21.15.114:0/928289305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe208107920 0x7fe208116e50 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe1f800d950 tx=0x7fe1f800de10 comp rx=0 tx=0).stop 2024-09-13T18:32:59.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.533+0000 7fe20e841640 1 -- 172.21.15.114:0/928289305 shutdown_connections 2024-09-13T18:32:59.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.533+0000 7fe20e841640 1 --2- 172.21.15.114:0/928289305 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe1ec0a1390 0x7fe1ec0a3850 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.533+0000 7fe20e841640 1 --2- 172.21.15.114:0/928289305 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe208119400 0x7fe20810fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.533+0000 7fe20e841640 1 --2- 172.21.15.114:0/928289305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe208107920 0x7fe208116e50 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.533+0000 7fe20e841640 1 --2- 172.21.15.114:0/928289305 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe208107540 0x7fe2081168f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.533+0000 7fe20e841640 1 -- 172.21.15.114:0/928289305 >> 172.21.15.114:0/928289305 conn(0x7fe20806afe0 msgr2=0x7fe208119dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:59.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.537+0000 7fe20e841640 1 -- 172.21.15.114:0/928289305 shutdown_connections 2024-09-13T18:32:59.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.537+0000 7fe20e841640 1 -- 172.21.15.114:0/928289305 wait complete. 2024-09-13T18:32:59.570 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-09-13T18:32:59.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.824+0000 7f4baea41640 1 -- 172.21.15.114:0/3338715934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ba8105010 msgr2=0x7f4ba810cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:59.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.824+0000 7f4baea41640 1 --2- 172.21.15.114:0/3338715934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ba8105010 0x7f4ba810cb60 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4b98009930 tx=0x7f4b9802a440 comp rx=0 tx=0).stop 2024-09-13T18:32:59.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.824+0000 7f4baea41640 1 -- 172.21.15.114:0/3338715934 shutdown_connections 2024-09-13T18:32:59.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.824+0000 7f4baea41640 1 --2- 172.21.15.114:0/3338715934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ba8105010 0x7f4ba810cb60 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.824+0000 7f4baea41640 1 --2- 172.21.15.114:0/3338715934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ba8100870 0x7f4ba81048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.824+0000 7f4baea41640 1 --2- 172.21.15.114:0/3338715934 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ba80ffec0 0x7f4ba81002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.824+0000 7f4baea41640 1 -- 172.21.15.114:0/3338715934 >> 172.21.15.114:0/3338715934 conn(0x7f4ba80fb970 msgr2=0x7f4ba80fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:59.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.825+0000 7f4baea41640 1 -- 172.21.15.114:0/3338715934 shutdown_connections 2024-09-13T18:32:59.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.825+0000 7f4baea41640 1 -- 172.21.15.114:0/3338715934 wait complete. 2024-09-13T18:32:59.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.825+0000 7f4baea41640 1 Processor -- start 2024-09-13T18:32:59.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.826+0000 7f4baea41640 1 -- start start 2024-09-13T18:32:59.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.826+0000 7f4baea41640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ba80ffec0 0x7f4ba8071a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:59.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.826+0000 7f4baea41640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ba8105010 0x7f4ba8071f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:59.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.826+0000 7f4baea41640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ba810ba70 0x7f4ba810bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:59.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.826+0000 7f4baea41640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba810c570 con 0x7f4ba80ffec0 2024-09-13T18:32:59.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.826+0000 7f4baea41640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba810c6e0 con 0x7f4ba810ba70 2024-09-13T18:32:59.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.826+0000 7f4baea41640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba810c8a0 con 0x7f4ba8105010 2024-09-13T18:32:59.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.826+0000 7f4bad23e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ba8105010 0x7f4ba8071f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:59.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.826+0000 7f4bad23e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ba8105010 0x7f4ba8071f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:39444/0 (socket says 172.21.15.114:39444) 2024-09-13T18:32:59.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.826+0000 7f4bad23e640 1 -- 172.21.15.114:0/11467570 learned_addr learned my addr 172.21.15.114:0/11467570 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:59.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.827+0000 7f4bada3f640 1 --2- 172.21.15.114:0/11467570 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ba80ffec0 0x7f4ba8071a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:59.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.827+0000 7f4bad23e640 1 -- 172.21.15.114:0/11467570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ba810ba70 msgr2=0x7f4ba810bf20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:32:59.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.827+0000 7f4bad23e640 1 --2- 172.21.15.114:0/11467570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ba810ba70 0x7f4ba810bf20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.827+0000 7f4bad23e640 1 -- 172.21.15.114:0/11467570 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ba80ffec0 msgr2=0x7f4ba8071a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:59.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.827+0000 7f4bad23e640 1 --2- 172.21.15.114:0/11467570 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ba80ffec0 0x7f4ba8071a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.827+0000 7f4bad23e640 1 -- 172.21.15.114:0/11467570 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b98009560 con 0x7f4ba8105010 2024-09-13T18:32:59.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.827+0000 7f4bada3f640 1 --2- 172.21.15.114:0/11467570 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ba80ffec0 0x7f4ba8071a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:32:59.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.828+0000 7f4bad23e640 1 --2- 172.21.15.114:0/11467570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ba8105010 0x7f4ba8071f40 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4ba402e980 tx=0x7f4ba402ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:59.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.828+0000 7f4b9effd640 1 -- 172.21.15.114:0/11467570 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba4007c10 con 0x7f4ba8105010 2024-09-13T18:32:59.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.828+0000 7f4baea41640 1 -- 172.21.15.114:0/11467570 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ba806d5c0 con 0x7f4ba8105010 2024-09-13T18:32:59.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.828+0000 7f4baea41640 1 -- 172.21.15.114:0/11467570 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ba806db20 con 0x7f4ba8105010 2024-09-13T18:32:59.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.828+0000 7f4b9effd640 1 -- 172.21.15.114:0/11467570 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4ba4007d70 con 0x7f4ba8105010 2024-09-13T18:32:59.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.828+0000 7f4b9effd640 1 -- 172.21.15.114:0/11467570 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba4007c10 con 0x7f4ba8105010 2024-09-13T18:32:59.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.836+0000 7f4b9effd640 1 -- 172.21.15.114:0/11467570 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f4ba403a400 con 0x7f4ba8105010 2024-09-13T18:32:59.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.837+0000 7f4b9effd640 1 --2- 172.21.15.114:0/11467570 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f4b940a1340 0x7f4b940a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:59.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.837+0000 7f4b9effd640 1 -- 172.21.15.114:0/11467570 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f4ba40f0200 con 0x7f4ba8105010 2024-09-13T18:32:59.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.838+0000 7f4bb29c8740 1 Processor -- start 2024-09-13T18:32:59.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.838+0000 7f4bada3f640 1 --2- 172.21.15.114:0/11467570 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f4b940a1340 0x7f4b940a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:59.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.838+0000 7f4bb29c8740 1 -- start start 2024-09-13T18:32:59.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.838+0000 7f4bb29c8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573793a3970 0x5573793a3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:59.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.838+0000 7f4bb29c8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5573793a4290 0x5573793ae5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:59.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.838+0000 7f4bb29c8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573793aeb30 0x5573793b0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:59.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.838+0000 7f4bb29c8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55737930dcc0 con 0x5573793a4290 2024-09-13T18:32:59.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.839+0000 7f4bb29c8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55737930b450 con 0x5573793aeb30 2024-09-13T18:32:59.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.839+0000 7f4bb29c8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557379306d40 con 0x5573793a3970 2024-09-13T18:32:59.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.839+0000 7f4bada3f640 1 --2- 172.21.15.114:0/11467570 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f4b940a1340 0x7f4b940a3800 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4ba0005fd0 tx=0x7f4ba0009500 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:59.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.839+0000 7f4bada3f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573793a3970 0x5573793a3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:59.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.839+0000 7f4bada3f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573793a3970 0x5573793a3d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:39460/0 (socket says 172.21.15.114:39460) 2024-09-13T18:32:59.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.839+0000 7f4bada3f640 1 -- 172.21.15.114:0/460471358 learned_addr learned my addr 172.21.15.114:0/460471358 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:59.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.840+0000 7f4bad23e640 1 --2- 172.21.15.114:0/460471358 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5573793a4290 0x5573793ae5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:59.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.840+0000 7f4bad23e640 1 -- 172.21.15.114:0/460471358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573793a3970 msgr2=0x5573793a3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:59.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.840+0000 7f4bad23e640 1 --2- 172.21.15.114:0/460471358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573793a3970 0x5573793a3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.840+0000 7f4bad23e640 1 -- 172.21.15.114:0/460471358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573793aeb30 msgr2=0x5573793b0f20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:32:59.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.840+0000 7f4bad23e640 1 --2- 172.21.15.114:0/460471358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573793aeb30 0x5573793b0f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.840+0000 7f4bad23e640 1 -- 172.21.15.114:0/460471358 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573792ce620 con 0x5573793a4290 2024-09-13T18:32:59.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.841+0000 7f4bad23e640 1 --2- 172.21.15.114:0/460471358 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5573793a4290 0x5573793ae5f0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f4ba40ed550 tx=0x7f4ba40bd610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea7fc036284cf448 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:59.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.842+0000 7f4b93fff640 1 -- 172.21.15.114:0/460471358 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba4007c10 con 0x5573793a4290 2024-09-13T18:32:59.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.848+0000 7f4b93fff640 1 -- 172.21.15.114:0/460471358 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4ba40c5390 con 0x5573793a4290 2024-09-13T18:32:59.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.849+0000 7f4bb29c8740 1 -- 172.21.15.114:0/460471358 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5573793a4290 msgr2=0x5573793ae5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:59.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.849+0000 7f4bb29c8740 1 --2- 172.21.15.114:0/460471358 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5573793a4290 0x5573793ae5f0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f4ba40ed550 tx=0x7f4ba40bd610 comp rx=0 tx=0).stop 2024-09-13T18:32:59.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.851+0000 7f4bb29c8740 1 -- 172.21.15.114:0/460471358 shutdown_connections 2024-09-13T18:32:59.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.851+0000 7f4bb29c8740 1 --2- 172.21.15.114:0/460471358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573793aeb30 0x5573793b0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.851+0000 7f4bb29c8740 1 --2- 172.21.15.114:0/460471358 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5573793a4290 0x5573793ae5f0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.851+0000 7f4bb29c8740 1 --2- 172.21.15.114:0/460471358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573793a3970 0x5573793a3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.851+0000 7f4bb29c8740 1 -- 172.21.15.114:0/460471358 >> 172.21.15.114:0/460471358 conn(0x55737939fee0 msgr2=0x5573793a02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:59.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.855+0000 7f4bb29c8740 1 -- 172.21.15.114:0/460471358 shutdown_connections 2024-09-13T18:32:59.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.855+0000 7f4bb29c8740 1 -- 172.21.15.114:0/460471358 wait complete. 2024-09-13T18:32:59.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.856+0000 7f4bb29c8740 1 -- start start 2024-09-13T18:32:59.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.856+0000 7f4bb29c8740 1 Processor -- start 2024-09-13T18:32:59.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.857+0000 7f4bb29c8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573793a3970 0x5573793ae0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:59.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.857+0000 7f4bb29c8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573793a4290 0x5573793ae5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:59.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.857+0000 7f4bb29c8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5573793aeb30 0x5573793ab690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:59.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.857+0000 7f4bb29c8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573792c5180 con 0x5573793aeb30 2024-09-13T18:32:59.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.857+0000 7f4bb29c8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557378fdf430 con 0x5573793a3970 2024-09-13T18:32:59.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.857+0000 7f4bb29c8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55737928a5e0 con 0x5573793a4290 2024-09-13T18:32:59.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.857+0000 7f4bb29c8740 10 client.0 _ll_register_callbacks cb 0x5573792f6aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:32:59.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.857+0000 7f4bae240640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5573793aeb30 0x5573793ab690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:59.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.857+0000 7f4bae240640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5573793aeb30 0x5573793ab690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41726/0 (socket says 172.21.15.114:41726) 2024-09-13T18:32:59.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.857+0000 7f4bae240640 1 -- 172.21.15.114:0/2196288636 learned_addr learned my addr 172.21.15.114:0/2196288636 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:59.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.860+0000 7f4bad23e640 1 --2- 172.21.15.114:0/2196288636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573793a4290 0x5573793ae5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:59.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.861+0000 7f4bae240640 1 -- 172.21.15.114:0/2196288636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573793a4290 msgr2=0x5573793ae5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:59.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.861+0000 7f4bae240640 1 --2- 172.21.15.114:0/2196288636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573793a4290 0x5573793ae5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.861+0000 7f4bada3f640 1 --2- 172.21.15.114:0/2196288636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573793a3970 0x5573793ae0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:59.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.861+0000 7f4bae240640 1 -- 172.21.15.114:0/2196288636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573793a3970 msgr2=0x5573793ae0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:59.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.861+0000 7f4bae240640 1 --2- 172.21.15.114:0/2196288636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573793a3970 0x5573793ae0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.861+0000 7f4bae240640 1 -- 172.21.15.114:0/2196288636 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ba806db20 con 0x5573793aeb30 2024-09-13T18:32:59.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.862+0000 7f4bada3f640 1 --2- 172.21.15.114:0/2196288636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573793a3970 0x5573793ae0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:32:59.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.886+0000 7f4bae240640 1 --2- 172.21.15.114:0/2196288636 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5573793aeb30 0x5573793ab690 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f4b98009230 tx=0x7f4b980043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91a610dc94a4162e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:59.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.886+0000 7f4b917fa640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:32:59.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.887+0000 7f4b917fa640 1 -- 172.21.15.114:0/2196288636 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b98004520 con 0x5573793aeb30 2024-09-13T18:32:59.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.887+0000 7f4bb29c8740 1 -- 172.21.15.114:0/2196288636 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557379330b80 con 0x5573793aeb30 2024-09-13T18:32:59.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.887+0000 7f4b917fa640 1 -- 172.21.15.114:0/2196288636 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f4b98003800 con 0x5573793aeb30 2024-09-13T18:32:59.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.891+0000 7f4b917fa640 1 -- 172.21.15.114:0/2196288636 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b980039c0 con 0x5573793aeb30 2024-09-13T18:32:59.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.894+0000 7f4bb29c8740 10 client.18417 fetch_fsmap learned FSMap version 34 2024-09-13T18:32:59.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.894+0000 7f4bb29c8740 1 -- 172.21.15.114:0/2196288636 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5573793053c0 con 0x5573793aeb30 2024-09-13T18:32:59.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.897+0000 7f4b917fa640 1 -- 172.21.15.114:0/2196288636 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557379330b80 con 0x5573793aeb30 2024-09-13T18:32:59.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.897+0000 7f4b917fa640 1 -- 172.21.15.114:0/2196288636 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4b98031400 con 0x5573793aeb30 2024-09-13T18:32:59.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.898+0000 7f4bb29c8740 10 client.18417 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:32:59.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.898+0000 7f4bb29c8740 10 client.18417 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:32:59.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.898+0000 7f4bb29c8740 20 client.18417 populate_metadata read hostname 'smithi114' 2024-09-13T18:32:59.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.898+0000 7f4bb29c8740 1 --2- 172.21.15.114:0/2196288636 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5573793be570 0x5573793c0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:59.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.898+0000 7f4bb29c8740 4 client.18417 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:32:59.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.898+0000 7f4bb29c8740 1 -- 172.21.15.114:0/2196288636 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5573792f91f0 con 0x5573793be570 2024-09-13T18:32:59.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.901+0000 7f4bada3f640 1 --2- 172.21.15.114:0/2196288636 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5573793be570 0x5573793c0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:59.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.909+0000 7f4bada3f640 1 --2- 172.21.15.114:0/2196288636 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5573793be570 0x5573793c0990 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97ae8cbbaa7b497a server_cookie=622153a4eb7107f3 in_seq=0 out_seq=0 2024-09-13T18:32:59.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.910+0000 7f4b917fa640 10 client.18417 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:32:59.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.913+0000 7f4b917fa640 1 -- 172.21.15.114:0/2196288636 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5573792f91f0 con 0x5573793be570 2024-09-13T18:32:59.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.913+0000 7f4b917fa640 10 client.18417 handle_command_reply: tid=0 2024-09-13T18:32:59.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.913+0000 7f4bb29c8740 1 client.18417 shutdown 2024-09-13T18:32:59.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.916+0000 7f4bb29c8740 1 -- 172.21.15.114:0/2196288636 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5573793aeb30 msgr2=0x5573793ab690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:59.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.916+0000 7f4bb29c8740 1 --2- 172.21.15.114:0/2196288636 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5573793aeb30 0x5573793ab690 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f4b98009230 tx=0x7f4b980043b0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.919+0000 7f4bb29c8740 1 -- 172.21.15.114:0/2196288636 shutdown_connections 2024-09-13T18:32:59.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.919+0000 7f4bb29c8740 1 --2- 172.21.15.114:0/2196288636 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5573793be570 0x5573793c0990 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.919+0000 7f4bb29c8740 1 --2- 172.21.15.114:0/2196288636 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5573793aeb30 0x5573793ab690 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.919+0000 7f4bb29c8740 1 --2- 172.21.15.114:0/2196288636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573793a4290 0x5573793ae5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.919+0000 7f4bb29c8740 1 --2- 172.21.15.114:0/2196288636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573793a3970 0x5573793ae0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.919+0000 7f4bb29c8740 1 -- 172.21.15.114:0/2196288636 >> 172.21.15.114:0/2196288636 conn(0x55737939fee0 msgr2=0x55737939f230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:59.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.922+0000 7f4b917fa640 0 client.18417 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:32:59.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.928+0000 7f4bb29c8740 1 -- 172.21.15.114:0/2196288636 shutdown_connections 2024-09-13T18:32:59.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.929+0000 7f4bb29c8740 1 -- 172.21.15.114:0/2196288636 wait complete. 2024-09-13T18:32:59.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.933+0000 7f4bb29c8740 20 client.18417 trim_cache size 0 max 16384 2024-09-13T18:32:59.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.958+0000 7f4bb29c8740 1 Processor -- start 2024-09-13T18:32:59.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.959+0000 7f4bb29c8740 1 -- start start 2024-09-13T18:32:59.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.959+0000 7f4bb29c8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573793a3970 0x5573793a9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:59.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.959+0000 7f4bb29c8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573793a4290 0x5573793aa230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:59.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.959+0000 7f4bb29c8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5573793aeb30 0x5573793aff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:59.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.959+0000 7f4bb29c8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573793aa7a0 con 0x5573793aeb30 2024-09-13T18:32:59.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.959+0000 7f4bb29c8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573793baba0 con 0x5573793a3970 2024-09-13T18:32:59.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.959+0000 7f4bb29c8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573793b15e0 con 0x5573793a4290 2024-09-13T18:32:59.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.960+0000 7f4bad23e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573793a4290 0x5573793aa230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:59.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.960+0000 7f4bad23e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573793a4290 0x5573793aa230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:39478/0 (socket says 172.21.15.114:39478) 2024-09-13T18:32:59.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.960+0000 7f4bad23e640 1 -- 172.21.15.114:0/1942266086 learned_addr learned my addr 172.21.15.114:0/1942266086 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:59.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.960+0000 7f4bae240640 1 --2- 172.21.15.114:0/1942266086 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5573793aeb30 0x5573793aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:59.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.961+0000 7f4bae240640 1 -- 172.21.15.114:0/1942266086 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573793a4290 msgr2=0x5573793aa230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:32:59.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.961+0000 7f4bae240640 1 --2- 172.21.15.114:0/1942266086 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573793a4290 0x5573793aa230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.961+0000 7f4bae240640 1 -- 172.21.15.114:0/1942266086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573793a3970 msgr2=0x5573793a9cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:32:59.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.961+0000 7f4bae240640 1 --2- 172.21.15.114:0/1942266086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573793a3970 0x5573793a9cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.961+0000 7f4bae240640 1 -- 172.21.15.114:0/1942266086 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573792a7770 con 0x5573793aeb30 2024-09-13T18:32:59.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.961+0000 7f4bae240640 1 --2- 172.21.15.114:0/1942266086 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5573793aeb30 0x5573793aff20 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f4b9802a960 tx=0x7f4b9802a440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=949cd00b33cc69cf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:59.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.962+0000 7f4b9cff9640 1 -- 172.21.15.114:0/1942266086 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b980044a0 con 0x5573793aeb30 2024-09-13T18:32:59.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.962+0000 7f4b9cff9640 1 -- 172.21.15.114:0/1942266086 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4b98031a30 con 0x5573793aeb30 2024-09-13T18:32:59.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.962+0000 7f4b9cff9640 1 -- 172.21.15.114:0/1942266086 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b98031600 con 0x5573793aeb30 2024-09-13T18:32:59.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.963+0000 7f4bb29c8740 1 -- 172.21.15.114:0/1942266086 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5573793aeb30 msgr2=0x5573793aff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:32:59.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.963+0000 7f4bb29c8740 1 --2- 172.21.15.114:0/1942266086 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5573793aeb30 0x5573793aff20 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f4b9802a960 tx=0x7f4b9802a440 comp rx=0 tx=0).stop 2024-09-13T18:32:59.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.967+0000 7f4bb29c8740 1 -- 172.21.15.114:0/1942266086 shutdown_connections 2024-09-13T18:32:59.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.967+0000 7f4bb29c8740 1 --2- 172.21.15.114:0/1942266086 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5573793aeb30 0x5573793aff20 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.967+0000 7f4bb29c8740 1 --2- 172.21.15.114:0/1942266086 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573793a4290 0x5573793aa230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.967+0000 7f4bb29c8740 1 --2- 172.21.15.114:0/1942266086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573793a3970 0x5573793a9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.967+0000 7f4bb29c8740 1 -- 172.21.15.114:0/1942266086 >> 172.21.15.114:0/1942266086 conn(0x55737939fee0 msgr2=0x5573793c52f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:32:59.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.973+0000 7f4bb29c8740 1 -- 172.21.15.114:0/1942266086 shutdown_connections 2024-09-13T18:32:59.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.978+0000 7f4bb29c8740 1 -- 172.21.15.114:0/1942266086 wait complete. 2024-09-13T18:32:59.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.978+0000 7f4bb29c8740 1 -- start start 2024-09-13T18:32:59.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.978+0000 7f4bb29c8740 1 Processor -- start 2024-09-13T18:32:59.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.980+0000 7f4bb29c8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5573793a3970 0x5573793b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:59.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.980+0000 7f4bb29c8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573793a4290 0x5573793b1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:59.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.980+0000 7f4bb29c8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573793aeb30 0x5573793b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:32:59.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.980+0000 7f4bb29c8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573793b2930 con 0x5573793a3970 2024-09-13T18:32:59.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.980+0000 7f4bb29c8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573793b2aa0 con 0x5573793a4290 2024-09-13T18:32:59.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.980+0000 7f4bb29c8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573793b2c60 con 0x5573793aeb30 2024-09-13T18:32:59.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.980+0000 7f4bb29c8740 10 client.0 _ll_register_callbacks cb 0x7f4ba40b8c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:32:59.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.980+0000 7f4bada3f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5573793a3970 0x5573793b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:32:59.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.981+0000 7f4bada3f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5573793a3970 0x5573793b59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41740/0 (socket says 172.21.15.114:41740) 2024-09-13T18:32:59.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.981+0000 7f4bada3f640 1 -- 172.21.15.114:0/2934399582 learned_addr learned my addr 172.21.15.114:0/2934399582 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:32:59.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.982+0000 7f4bada3f640 1 -- 172.21.15.114:0/2934399582 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573793aeb30 msgr2=0x5573793b23f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:32:59.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.982+0000 7f4bada3f640 1 --2- 172.21.15.114:0/2934399582 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573793aeb30 0x5573793b23f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.982+0000 7f4bada3f640 1 -- 172.21.15.114:0/2934399582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573793a4290 msgr2=0x5573793b1eb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:32:59.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.982+0000 7f4bada3f640 1 --2- 172.21.15.114:0/2934399582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573793a4290 0x5573793b1eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:32:59.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.982+0000 7f4bada3f640 1 -- 172.21.15.114:0/2934399582 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5573793053c0 con 0x5573793a3970 2024-09-13T18:32:59.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.987+0000 7f4bada3f640 1 --2- 172.21.15.114:0/2934399582 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5573793a3970 0x5573793b59d0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f4ba0010010 tx=0x7f4ba0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3ec56d092c1233a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:32:59.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.987+0000 7f4b917fa640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:32:59.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.988+0000 7f4b917fa640 1 -- 172.21.15.114:0/2934399582 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba0025040 con 0x5573793a3970 2024-09-13T18:32:59.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.988+0000 7f4bb29c8740 1 -- 172.21.15.114:0/2934399582 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4ba40c5390 con 0x5573793a3970 2024-09-13T18:32:59.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.988+0000 7f4b917fa640 1 -- 172.21.15.114:0/2934399582 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f4ba0028530 con 0x5573793a3970 2024-09-13T18:32:59.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.989+0000 7f4b917fa640 1 -- 172.21.15.114:0/2934399582 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ba0028730 con 0x5573793a3970 2024-09-13T18:32:59.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.991+0000 7f4bb29c8740 10 client.18423 fetch_fsmap learned FSMap version 34 2024-09-13T18:32:59.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:32:59.991+0000 7f4bb29c8740 1 -- 172.21.15.114:0/2934399582 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5573792cce80 con 0x5573793a3970 2024-09-13T18:33:00.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:00.000+0000 7f4b917fa640 1 -- 172.21.15.114:0/2934399582 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4ba40c5390 con 0x5573793a3970 2024-09-13T18:33:00.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:00.000+0000 7f4b917fa640 1 -- 172.21.15.114:0/2934399582 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4ba0028050 con 0x5573793a3970 2024-09-13T18:33:00.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:00.000+0000 7f4bb29c8740 10 client.18423 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:33:00.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:00.000+0000 7f4bb29c8740 10 client.18423 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:33:00.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:00.000+0000 7f4bb29c8740 20 client.18423 populate_metadata read hostname 'smithi114' 2024-09-13T18:33:00.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:00.000+0000 7f4bb29c8740 1 --2- 172.21.15.114:0/2934399582 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x557379416040 0x557379418460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:00.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:00.000+0000 7f4bb29c8740 4 client.18423 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:33:00.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:00.000+0000 7f4bb29c8740 1 -- 172.21.15.114:0/2934399582 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557378ffa4e0 con 0x557379416040 2024-09-13T18:33:00.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:00.002+0000 7f4bad23e640 1 --2- 172.21.15.114:0/2934399582 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x557379416040 0x557379418460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:00.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:00.003+0000 7f4bad23e640 1 --2- 172.21.15.114:0/2934399582 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x557379416040 0x557379418460 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d888c468140d6ca9 server_cookie=48faeb72ff0539b3 in_seq=0 out_seq=0 2024-09-13T18:33:00.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:00.003+0000 7f4b917fa640 10 client.18423 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:33:00.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:00.004+0000 7f4b917fa640 1 -- 172.21.15.114:0/2934399582 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557378ffa4e0 con 0x557379416040 2024-09-13T18:33:00.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:00.004+0000 7f4b917fa640 10 client.18423 handle_command_reply: tid=0 2024-09-13T18:33:00.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:00.004+0000 7f4bb29c8740 1 client.18423 shutdown 2024-09-13T18:33:00.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:00.008+0000 7f4bb29c8740 1 -- 172.21.15.114:0/2934399582 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5573793a3970 msgr2=0x5573793b59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:00.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:00.008+0000 7f4bb29c8740 1 --2- 172.21.15.114:0/2934399582 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5573793a3970 0x5573793b59d0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f4ba0010010 tx=0x7f4ba0004ab0 comp rx=0 tx=0).stop 2024-09-13T18:33:00.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:00.010+0000 7f4bb29c8740 1 -- 172.21.15.114:0/2934399582 shutdown_connections 2024-09-13T18:33:00.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:00.010+0000 7f4bb29c8740 1 --2- 172.21.15.114:0/2934399582 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x557379416040 0x557379418460 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:00.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:00.010+0000 7f4bb29c8740 1 --2- 172.21.15.114:0/2934399582 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5573793aeb30 0x5573793b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:00.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:00.010+0000 7f4bb29c8740 1 --2- 172.21.15.114:0/2934399582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573793a4290 0x5573793b1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:00.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:00.010+0000 7f4bb29c8740 1 --2- 172.21.15.114:0/2934399582 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5573793a3970 0x5573793b59d0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:00.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:00.010+0000 7f4bb29c8740 1 -- 172.21.15.114:0/2934399582 >> 172.21.15.114:0/2934399582 conn(0x55737939fee0 msgr2=0x5573793a9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:00.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:00.012+0000 7f4b917fa640 0 client.18423 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:33:00.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:00.016+0000 7f4bb29c8740 1 -- 172.21.15.114:0/2934399582 shutdown_connections 2024-09-13T18:33:00.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:00.019+0000 7f4bb29c8740 1 -- 172.21.15.114:0/2934399582 wait complete. 2024-09-13T18:33:00.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:00.029+0000 7f4bb29c8740 20 client.18423 trim_cache size 0 max 16384 2024-09-13T18:33:00.036 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:33:00.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:00.040+0000 7f4baea41640 1 -- 172.21.15.114:0/11467570 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f4b940a1340 msgr2=0x7f4b940a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:00.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:00.040+0000 7f4baea41640 1 --2- 172.21.15.114:0/11467570 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f4b940a1340 0x7f4b940a3800 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4ba0005fd0 tx=0x7f4ba0009500 comp rx=0 tx=0).stop 2024-09-13T18:33:00.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:00.040+0000 7f4baea41640 1 -- 172.21.15.114:0/11467570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ba8105010 msgr2=0x7f4ba8071f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:00.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:00.040+0000 7f4baea41640 1 --2- 172.21.15.114:0/11467570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ba8105010 0x7f4ba8071f40 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4ba402e980 tx=0x7f4ba402ee40 comp rx=0 tx=0).stop 2024-09-13T18:33:00.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:00.041+0000 7f4baea41640 1 -- 172.21.15.114:0/11467570 shutdown_connections 2024-09-13T18:33:00.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:00.041+0000 7f4baea41640 1 --2- 172.21.15.114:0/11467570 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f4b940a1340 0x7f4b940a3800 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:00.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:00.041+0000 7f4baea41640 1 --2- 172.21.15.114:0/11467570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ba810ba70 0x7f4ba810bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:00.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:00.041+0000 7f4baea41640 1 --2- 172.21.15.114:0/11467570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ba8105010 0x7f4ba8071f40 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:00.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:00.041+0000 7f4baea41640 1 --2- 172.21.15.114:0/11467570 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ba80ffec0 0x7f4ba8071a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:00.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:00.041+0000 7f4baea41640 1 -- 172.21.15.114:0/11467570 >> 172.21.15.114:0/11467570 conn(0x7f4ba80fb970 msgr2=0x7f4ba80fcc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:00.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:00.046+0000 7f4baea41640 1 -- 172.21.15.114:0/11467570 shutdown_connections 2024-09-13T18:33:00.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:00.046+0000 7f4baea41640 1 -- 172.21.15.114:0/11467570 wait complete. 2024-09-13T18:33:00.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:00 smithi204 ceph-mon[34777]: pgmap v326: 129 pgs: 129 active+clean; 21 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 27 MiB/s wr, 589 op/s 2024-09-13T18:33:00.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:00 smithi114 ceph-mon[30897]: pgmap v326: 129 pgs: 129 active+clean; 21 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 27 MiB/s wr, 589 op/s 2024-09-13T18:33:00.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:00 smithi134 ceph-mon[34283]: pgmap v326: 129 pgs: 129 active+clean; 21 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 27 MiB/s wr, 589 op/s 2024-09-13T18:33:01.087 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-13T18:33:01.087 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-09-13T18:33:01.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.293+0000 7fbfa6f8b640 1 -- 172.21.15.114:0/2148699343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfa0119400 msgr2=0x7fbfa011b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:01.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.293+0000 7fbfa6f8b640 1 --2- 172.21.15.114:0/2148699343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfa0119400 0x7fbfa011b7f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c009ca0 tx=0x7fbf9c02a8f0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.300+0000 7fbfa6f8b640 1 -- 172.21.15.114:0/2148699343 shutdown_connections 2024-09-13T18:33:01.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.300+0000 7fbfa6f8b640 1 --2- 172.21.15.114:0/2148699343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfa0119400 0x7fbfa011b7f0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.300+0000 7fbfa6f8b640 1 --2- 172.21.15.114:0/2148699343 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbfa0107890 0x7fbfa0107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.300+0000 7fbfa6f8b640 1 --2- 172.21.15.114:0/2148699343 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbfa01074b0 0x7fbfa010e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.300+0000 7fbfa6f8b640 1 -- 172.21.15.114:0/2148699343 >> 172.21.15.114:0/2148699343 conn(0x7fbfa006af40 msgr2=0x7fbfa006b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:01.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.301+0000 7fbfa6f8b640 1 -- 172.21.15.114:0/2148699343 shutdown_connections 2024-09-13T18:33:01.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.304+0000 7fbfa6f8b640 1 -- 172.21.15.114:0/2148699343 wait complete. 2024-09-13T18:33:01.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.305+0000 7fbfa6f8b640 1 Processor -- start 2024-09-13T18:33:01.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.305+0000 7fbfa6f8b640 1 -- start start 2024-09-13T18:33:01.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.305+0000 7fbfa6f8b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbfa01074b0 0x7fbfa01a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:01.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.305+0000 7fbfa6f8b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfa0107890 0x7fbfa01a5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:01.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.305+0000 7fbfa6f8b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbfa0119400 0x7fbfa01a9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:01.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.305+0000 7fbfa6f8b640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa01a9f60 con 0x7fbfa0119400 2024-09-13T18:33:01.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.305+0000 7fbfa6f8b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa01aa0a0 con 0x7fbfa0107890 2024-09-13T18:33:01.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.305+0000 7fbfa6f8b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa01aa260 con 0x7fbfa01074b0 2024-09-13T18:33:01.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.307+0000 7fbfa5f89640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbfa01074b0 0x7fbfa01a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:01.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.307+0000 7fbfa5f89640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbfa01074b0 0x7fbfa01a5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:39498/0 (socket says 172.21.15.114:39498) 2024-09-13T18:33:01.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.307+0000 7fbfa678a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbfa0119400 0x7fbfa01a9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:01.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.307+0000 7fbfa5f89640 1 -- 172.21.15.114:0/120501049 learned_addr learned my addr 172.21.15.114:0/120501049 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:01.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.308+0000 7fbfa5788640 1 --2- 172.21.15.114:0/120501049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfa0107890 0x7fbfa01a5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:01.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.309+0000 7fbfa5788640 1 -- 172.21.15.114:0/120501049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbfa01074b0 msgr2=0x7fbfa01a5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:01.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.309+0000 7fbfa5788640 1 --2- 172.21.15.114:0/120501049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbfa01074b0 0x7fbfa01a5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.309+0000 7fbfa5788640 1 -- 172.21.15.114:0/120501049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbfa0119400 msgr2=0x7fbfa01a9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:01.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.309+0000 7fbfa5788640 1 --2- 172.21.15.114:0/120501049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbfa0119400 0x7fbfa01a9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.309+0000 7fbfa5788640 1 -- 172.21.15.114:0/120501049 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf9c009900 con 0x7fbfa0107890 2024-09-13T18:33:01.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.312+0000 7fbfa5788640 1 --2- 172.21.15.114:0/120501049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfa0107890 0x7fbfa01a5570 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fbf9000c9a0 tx=0x7fbf9000ce60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:01.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.315+0000 7fbf96ffd640 1 -- 172.21.15.114:0/120501049 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf90007bb0 con 0x7fbfa0107890 2024-09-13T18:33:01.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.315+0000 7fbfa6f8b640 1 -- 172.21.15.114:0/120501049 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfa01aa540 con 0x7fbfa0107890 2024-09-13T18:33:01.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.315+0000 7fbfa6f8b640 1 -- 172.21.15.114:0/120501049 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfa010edc0 con 0x7fbfa0107890 2024-09-13T18:33:01.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.315+0000 7fbf96ffd640 1 -- 172.21.15.114:0/120501049 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbf90007d10 con 0x7fbfa0107890 2024-09-13T18:33:01.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.315+0000 7fbf96ffd640 1 -- 172.21.15.114:0/120501049 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf90004cf0 con 0x7fbfa0107890 2024-09-13T18:33:01.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.320+0000 7fbfaafe8740 1 Processor -- start 2024-09-13T18:33:01.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.320+0000 7fbfaafe8740 1 -- start start 2024-09-13T18:33:01.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.320+0000 7fbfaafe8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5654174be9a0 0x5654174bed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:01.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.321+0000 7fbfaafe8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654174bf2c0 0x5654174c9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:01.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.321+0000 7fbfaafe8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654174c9b60 0x5654174cbf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:01.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.321+0000 7fbfaafe8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x565417429c60 con 0x5654174be9a0 2024-09-13T18:33:01.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.321+0000 7fbfaafe8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654174273f0 con 0x5654174c9b60 2024-09-13T18:33:01.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.321+0000 7fbfaafe8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x565417422ce0 con 0x5654174bf2c0 2024-09-13T18:33:01.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.321+0000 7fbfa678a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5654174be9a0 0x5654174bed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:01.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.321+0000 7fbfa678a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5654174be9a0 0x5654174bed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41762/0 (socket says 172.21.15.114:41762) 2024-09-13T18:33:01.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.321+0000 7fbfa678a640 1 -- 172.21.15.114:0/841003083 learned_addr learned my addr 172.21.15.114:0/841003083 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:01.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.322+0000 7fbfa678a640 1 -- 172.21.15.114:0/841003083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654174bf2c0 msgr2=0x5654174c9620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:33:01.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.322+0000 7fbfa678a640 1 --2- 172.21.15.114:0/841003083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654174bf2c0 0x5654174c9620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.322+0000 7fbfa678a640 1 -- 172.21.15.114:0/841003083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654174c9b60 msgr2=0x5654174cbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:01.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.322+0000 7fbfa678a640 1 --2- 172.21.15.114:0/841003083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654174c9b60 0x5654174cbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.322+0000 7fbfa678a640 1 -- 172.21.15.114:0/841003083 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5654173de970 con 0x5654174be9a0 2024-09-13T18:33:01.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.327+0000 7fbf96ffd640 1 -- 172.21.15.114:0/120501049 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fbf90004e50 con 0x7fbfa0107890 2024-09-13T18:33:01.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.327+0000 7fbfa678a640 1 --2- 172.21.15.114:0/841003083 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5654174be9a0 0x5654174bed80 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c035000 tx=0x7fbf9c031ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80fe5cc0f2ffa100 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:01.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.328+0000 7fbf96ffd640 1 --2- 172.21.15.114:0/120501049 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbf8c0a1410 0x7fbf8c0a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:01.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.328+0000 7fbf96ffd640 1 -- 172.21.15.114:0/120501049 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fbf900cedc0 con 0x7fbfa0107890 2024-09-13T18:33:01.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.332+0000 7fbf8bfff640 1 -- 172.21.15.114:0/841003083 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c040070 con 0x5654174be9a0 2024-09-13T18:33:01.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.332+0000 7fbf8bfff640 1 -- 172.21.15.114:0/841003083 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbf9c031460 con 0x5654174be9a0 2024-09-13T18:33:01.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.332+0000 7fbf8bfff640 1 -- 172.21.15.114:0/841003083 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c004040 con 0x5654174be9a0 2024-09-13T18:33:01.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.333+0000 7fbfaafe8740 1 -- 172.21.15.114:0/841003083 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5654174be9a0 msgr2=0x5654174bed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:01.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.333+0000 7fbfaafe8740 1 --2- 172.21.15.114:0/841003083 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5654174be9a0 0x5654174bed80 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c035000 tx=0x7fbf9c031ac0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.334+0000 7fbfaafe8740 1 -- 172.21.15.114:0/841003083 shutdown_connections 2024-09-13T18:33:01.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.334+0000 7fbfaafe8740 1 --2- 172.21.15.114:0/841003083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654174c9b60 0x5654174cbf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.334+0000 7fbfaafe8740 1 --2- 172.21.15.114:0/841003083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654174bf2c0 0x5654174c9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.334+0000 7fbfaafe8740 1 --2- 172.21.15.114:0/841003083 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5654174be9a0 0x5654174bed80 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.334+0000 7fbfaafe8740 1 -- 172.21.15.114:0/841003083 >> 172.21.15.114:0/841003083 conn(0x5654174baf30 msgr2=0x5654174bb310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:01.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.335+0000 7fbfa5f89640 1 --2- 172.21.15.114:0/120501049 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbf8c0a1410 0x7fbf8c0a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:01.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.338+0000 7fbfaafe8740 1 -- 172.21.15.114:0/841003083 shutdown_connections 2024-09-13T18:33:01.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.338+0000 7fbfaafe8740 1 -- 172.21.15.114:0/841003083 wait complete. 2024-09-13T18:33:01.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.338+0000 7fbfaafe8740 1 -- start start 2024-09-13T18:33:01.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.338+0000 7fbfaafe8740 1 Processor -- start 2024-09-13T18:33:01.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.339+0000 7fbfaafe8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5654174be9a0 0x5654174c9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:01.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.339+0000 7fbfaafe8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654174bf2c0 0x5654174c6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:01.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.339+0000 7fbfaafe8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654174c9b60 0x5654174c6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:01.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.339+0000 7fbfaafe8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654173e0f80 con 0x5654174be9a0 2024-09-13T18:33:01.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.339+0000 7fbfaafe8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654173e0c20 con 0x5654174c9b60 2024-09-13T18:33:01.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.339+0000 7fbfaafe8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654173adab0 con 0x5654174bf2c0 2024-09-13T18:33:01.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.339+0000 7fbfaafe8740 10 client.0 _ll_register_callbacks cb 0x56541741c350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:33:01.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.342+0000 7fbfa5788640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654174bf2c0 0x5654174c6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:01.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.342+0000 7fbfa5788640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654174bf2c0 0x5654174c6780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:39500/0 (socket says 172.21.15.114:39500) 2024-09-13T18:33:01.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.342+0000 7fbfa5788640 1 -- 172.21.15.114:0/4120498770 learned_addr learned my addr 172.21.15.114:0/4120498770 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:01.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.344+0000 7fbfa678a640 1 --2- 172.21.15.114:0/4120498770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654174c9b60 0x5654174c6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:01.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.344+0000 7fbfa5f89640 1 --2- 172.21.15.114:0/4120498770 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5654174be9a0 0x5654174c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:01.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.345+0000 7fbfa5788640 1 -- 172.21.15.114:0/4120498770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654174c9b60 msgr2=0x5654174c6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:01.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.345+0000 7fbfa5788640 1 --2- 172.21.15.114:0/4120498770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654174c9b60 0x5654174c6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.345+0000 7fbfa5788640 1 -- 172.21.15.114:0/4120498770 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5654174be9a0 msgr2=0x5654174c9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:01.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.345+0000 7fbfa5788640 1 --2- 172.21.15.114:0/4120498770 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5654174be9a0 0x5654174c9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.345+0000 7fbfa5788640 1 -- 172.21.15.114:0/4120498770 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfa011c170 con 0x5654174bf2c0 2024-09-13T18:33:01.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.347+0000 7fbfa5f89640 1 --2- 172.21.15.114:0/120501049 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbf8c0a1410 0x7fbf8c0a38d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x5654174baa30 tx=0x7fbf98014950 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:01.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.348+0000 7fbfa5788640 1 --2- 172.21.15.114:0/4120498770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654174bf2c0 0x5654174c6780 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fbf9000cfd0 tx=0x7fbf9009bb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83a39872444eb986 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:01.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.348+0000 7fbf897fa640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:33:01.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.351+0000 7fbf897fa640 1 -- 172.21.15.114:0/4120498770 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf900cc590 con 0x5654174bf2c0 2024-09-13T18:33:01.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.351+0000 7fbf897fa640 1 -- 172.21.15.114:0/4120498770 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fbf900a3f50 con 0x5654174bf2c0 2024-09-13T18:33:01.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.355+0000 7fbfaafe8740 1 -- 172.21.15.114:0/4120498770 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565417116590 con 0x5654174bf2c0 2024-09-13T18:33:01.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.365+0000 7fbf897fa640 1 -- 172.21.15.114:0/4120498770 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf900b4200 con 0x5654174bf2c0 2024-09-13T18:33:01.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.366+0000 7fbfaafe8740 10 client.27826 fetch_fsmap learned FSMap version 34 2024-09-13T18:33:01.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.366+0000 7fbfaafe8740 1 -- 172.21.15.114:0/4120498770 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x565417421360 con 0x5654174bf2c0 2024-09-13T18:33:01.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.368+0000 7fbf897fa640 1 -- 172.21.15.114:0/4120498770 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x565417116590 con 0x5654174bf2c0 2024-09-13T18:33:01.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.368+0000 7fbf897fa640 1 -- 172.21.15.114:0/4120498770 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbf90020070 con 0x5654174bf2c0 2024-09-13T18:33:01.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.368+0000 7fbfaafe8740 10 client.27826 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:33:01.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.368+0000 7fbfaafe8740 10 client.27826 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:33:01.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.368+0000 7fbfaafe8740 20 client.27826 populate_metadata read hostname 'smithi114' 2024-09-13T18:33:01.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.368+0000 7fbfaafe8740 1 --2- 172.21.15.114:0/4120498770 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5654174d9890 0x5654174dbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:01.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.368+0000 7fbfaafe8740 4 client.27826 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:33:01.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.368+0000 7fbfaafe8740 1 -- 172.21.15.114:0/4120498770 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565417127860 con 0x5654174d9890 2024-09-13T18:33:01.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.374+0000 7fbfa5f89640 1 --2- 172.21.15.114:0/4120498770 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5654174d9890 0x5654174dbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:01.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.404+0000 7fbfa5f89640 1 --2- 172.21.15.114:0/4120498770 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5654174d9890 0x5654174dbcb0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22fa3f2eaaabb61e server_cookie=7a02da2bba795a1e in_seq=0 out_seq=0 2024-09-13T18:33:01.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.404+0000 7fbf897fa640 10 client.27826 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:33:01.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.409+0000 7fbf897fa640 1 -- 172.21.15.114:0/4120498770 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x565417127860 con 0x5654174d9890 2024-09-13T18:33:01.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.409+0000 7fbf897fa640 10 client.27826 handle_command_reply: tid=0 2024-09-13T18:33:01.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.409+0000 7fbfaafe8740 1 client.27826 shutdown 2024-09-13T18:33:01.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.416+0000 7fbfaafe8740 1 -- 172.21.15.114:0/4120498770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654174bf2c0 msgr2=0x5654174c6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:01.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.416+0000 7fbfaafe8740 1 --2- 172.21.15.114:0/4120498770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654174bf2c0 0x5654174c6780 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fbf9000cfd0 tx=0x7fbf9009bb80 comp rx=0 tx=0).stop 2024-09-13T18:33:01.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.425+0000 7fbfaafe8740 1 -- 172.21.15.114:0/4120498770 shutdown_connections 2024-09-13T18:33:01.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.425+0000 7fbfaafe8740 1 --2- 172.21.15.114:0/4120498770 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5654174d9890 0x5654174dbcb0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.425+0000 7fbfaafe8740 1 --2- 172.21.15.114:0/4120498770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654174c9b60 0x5654174c6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.425+0000 7fbfaafe8740 1 --2- 172.21.15.114:0/4120498770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654174bf2c0 0x5654174c6780 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.425+0000 7fbfaafe8740 1 --2- 172.21.15.114:0/4120498770 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5654174be9a0 0x5654174c9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.425+0000 7fbfaafe8740 1 -- 172.21.15.114:0/4120498770 >> 172.21.15.114:0/4120498770 conn(0x5654174baf30 msgr2=0x5654174ba160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:01.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.425+0000 7fbf897fa640 0 client.27826 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:33:01.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.429+0000 7fbfaafe8740 1 -- 172.21.15.114:0/4120498770 shutdown_connections 2024-09-13T18:33:01.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.433+0000 7fbfaafe8740 1 -- 172.21.15.114:0/4120498770 wait complete. 2024-09-13T18:33:01.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.438+0000 7fbfaafe8740 20 client.27826 trim_cache size 0 max 16384 2024-09-13T18:33:01.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.477+0000 7fbfaafe8740 1 Processor -- start 2024-09-13T18:33:01.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.477+0000 7fbfaafe8740 1 -- start start 2024-09-13T18:33:01.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.477+0000 7fbfaafe8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654174be9a0 0x5654174c1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:01.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.477+0000 7fbfaafe8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5654174bf2c0 0x5654174c1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:01.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.477+0000 7fbfaafe8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654174c9b60 0x5654174c5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:01.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.477+0000 7fbfaafe8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654174bf0b0 con 0x5654174bf2c0 2024-09-13T18:33:01.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.477+0000 7fbfaafe8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654174cc270 con 0x5654174c9b60 2024-09-13T18:33:01.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.477+0000 7fbfaafe8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654174e0e30 con 0x5654174be9a0 2024-09-13T18:33:01.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.478+0000 7fbfa678a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654174c9b60 0x5654174c5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:01.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.478+0000 7fbfa678a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654174c9b60 0x5654174c5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:47946/0 (socket says 172.21.15.114:47946) 2024-09-13T18:33:01.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.478+0000 7fbfa678a640 1 -- 172.21.15.114:0/2839128670 learned_addr learned my addr 172.21.15.114:0/2839128670 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:01.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.478+0000 7fbfa5788640 1 --2- 172.21.15.114:0/2839128670 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5654174bf2c0 0x5654174c1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:01.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.478+0000 7fbfa678a640 1 -- 172.21.15.114:0/2839128670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654174be9a0 msgr2=0x5654174c1a80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:33:01.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.478+0000 7fbfa678a640 1 --2- 172.21.15.114:0/2839128670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654174be9a0 0x5654174c1a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.478+0000 7fbfa678a640 1 -- 172.21.15.114:0/2839128670 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5654174bf2c0 msgr2=0x5654174c1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:01.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.478+0000 7fbfa678a640 1 --2- 172.21.15.114:0/2839128670 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5654174bf2c0 0x5654174c1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.478+0000 7fbfa678a640 1 -- 172.21.15.114:0/2839128670 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5654173c31c0 con 0x5654174c9b60 2024-09-13T18:33:01.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.479+0000 7fbfa678a640 1 --2- 172.21.15.114:0/2839128670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654174c9b60 0x5654174c5030 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c0096a0 tx=0x7fbf9c009830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73463c21c3148651 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:01.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.479+0000 7fbf94ff9640 1 -- 172.21.15.114:0/2839128670 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c040070 con 0x5654174c9b60 2024-09-13T18:33:01.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.479+0000 7fbf94ff9640 1 -- 172.21.15.114:0/2839128670 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbf9c041040 con 0x5654174c9b60 2024-09-13T18:33:01.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.479+0000 7fbf94ff9640 1 -- 172.21.15.114:0/2839128670 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c0305d0 con 0x5654174c9b60 2024-09-13T18:33:01.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.481+0000 7fbfaafe8740 1 -- 172.21.15.114:0/2839128670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654174c9b60 msgr2=0x5654174c5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:01.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.481+0000 7fbfaafe8740 1 --2- 172.21.15.114:0/2839128670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654174c9b60 0x5654174c5030 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c0096a0 tx=0x7fbf9c009830 comp rx=0 tx=0).stop 2024-09-13T18:33:01.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.482+0000 7fbfaafe8740 1 -- 172.21.15.114:0/2839128670 shutdown_connections 2024-09-13T18:33:01.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.482+0000 7fbfaafe8740 1 --2- 172.21.15.114:0/2839128670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654174c9b60 0x5654174c5030 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.482+0000 7fbfaafe8740 1 --2- 172.21.15.114:0/2839128670 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5654174bf2c0 0x5654174c1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.482+0000 7fbfaafe8740 1 --2- 172.21.15.114:0/2839128670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654174be9a0 0x5654174c1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.482+0000 7fbfaafe8740 1 -- 172.21.15.114:0/2839128670 >> 172.21.15.114:0/2839128670 conn(0x5654174baf30 msgr2=0x5654174bb310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:01.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.488+0000 7fbfaafe8740 1 -- 172.21.15.114:0/2839128670 shutdown_connections 2024-09-13T18:33:01.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.491+0000 7fbfaafe8740 1 -- 172.21.15.114:0/2839128670 wait complete. 2024-09-13T18:33:01.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.491+0000 7fbfaafe8740 1 -- start start 2024-09-13T18:33:01.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.491+0000 7fbfaafe8740 1 Processor -- start 2024-09-13T18:33:01.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.493+0000 7fbfaafe8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5654174be9a0 0x5654174d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:01.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.493+0000 7fbfaafe8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654174bf2c0 0x5654174c3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:01.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.493+0000 7fbfaafe8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654174c9b60 0x5654174c3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:01.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.493+0000 7fbfaafe8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654174c3d80 con 0x5654174be9a0 2024-09-13T18:33:01.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.493+0000 7fbfa5788640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654174bf2c0 0x5654174c3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:01.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.494+0000 7fbfa678a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654174c9b60 0x5654174c3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:01.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.494+0000 7fbfa678a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654174c9b60 0x5654174c3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:47954/0 (socket says 172.21.15.114:47954) 2024-09-13T18:33:01.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.494+0000 7fbfa678a640 1 -- 172.21.15.114:0/293878043 learned_addr learned my addr 172.21.15.114:0/293878043 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:01.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.494+0000 7fbfaafe8740 1 -- 172.21.15.114:0/293878043 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654174c3ef0 con 0x5654174c9b60 2024-09-13T18:33:01.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.494+0000 7fbfaafe8740 1 -- 172.21.15.114:0/293878043 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56541752fd90 con 0x5654174bf2c0 2024-09-13T18:33:01.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.494+0000 7fbfaafe8740 10 client.0 _ll_register_callbacks cb 0x7fbf900058c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:33:01.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.494+0000 7fbfa5788640 1 -- 172.21.15.114:0/293878043 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654174c9b60 msgr2=0x5654174c3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:01.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.494+0000 7fbfa5788640 1 --2- 172.21.15.114:0/293878043 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654174c9b60 0x5654174c3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.494+0000 7fbfa5788640 1 -- 172.21.15.114:0/293878043 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5654174be9a0 msgr2=0x5654174d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:01.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.494+0000 7fbfa5788640 1 --2- 172.21.15.114:0/293878043 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5654174be9a0 0x5654174d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.495+0000 7fbfa5788640 1 -- 172.21.15.114:0/293878043 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5654173de970 con 0x5654174bf2c0 2024-09-13T18:33:01.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.507+0000 7fbfa5788640 1 --2- 172.21.15.114:0/293878043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654174bf2c0 0x5654174c3140 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fbf90098030 tx=0x7fbf900a3cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50265d774cec83db server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:01.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.507+0000 7fbf897fa640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:33:01.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.520+0000 7fbf897fa640 1 -- 172.21.15.114:0/293878043 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf90097c90 con 0x5654174bf2c0 2024-09-13T18:33:01.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.520+0000 7fbf897fa640 1 -- 172.21.15.114:0/293878043 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fbf9009b280 con 0x5654174bf2c0 2024-09-13T18:33:01.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.523+0000 7fbfaafe8740 1 -- 172.21.15.114:0/293878043 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbf9c031460 con 0x5654174bf2c0 2024-09-13T18:33:01.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.525+0000 7fbf897fa640 1 -- 172.21.15.114:0/293878043 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf90007e80 con 0x5654174bf2c0 2024-09-13T18:33:01.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.527+0000 7fbfaafe8740 10 client.27829 fetch_fsmap learned FSMap version 34 2024-09-13T18:33:01.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.527+0000 7fbfaafe8740 1 -- 172.21.15.114:0/293878043 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5654173e8990 con 0x5654174bf2c0 2024-09-13T18:33:01.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.530+0000 7fbf897fa640 1 -- 172.21.15.114:0/293878043 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbf9c031460 con 0x5654174bf2c0 2024-09-13T18:33:01.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.530+0000 7fbf897fa640 1 -- 172.21.15.114:0/293878043 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbf90020040 con 0x5654174bf2c0 2024-09-13T18:33:01.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.530+0000 7fbfaafe8740 10 client.27829 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:33:01.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.530+0000 7fbfaafe8740 10 client.27829 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:33:01.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.530+0000 7fbfaafe8740 20 client.27829 populate_metadata read hostname 'smithi114' 2024-09-13T18:33:01.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.530+0000 7fbfaafe8740 1 --2- 172.21.15.114:0/293878043 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x565417530fe0 0x5654175333d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:01.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.530+0000 7fbfaafe8740 4 client.27829 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-13T18:33:01.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.530+0000 7fbfaafe8740 1 -- 172.21.15.114:0/293878043 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56541717aeb0 con 0x565417530fe0 2024-09-13T18:33:01.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.532+0000 7fbfa5f89640 1 --2- 172.21.15.114:0/293878043 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x565417530fe0 0x5654175333d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:01.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.542+0000 7fbfa5f89640 1 --2- 172.21.15.114:0/293878043 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x565417530fe0 0x5654175333d0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddf5d9a2777284a3 server_cookie=9a70e7dfeb4d10b6 in_seq=0 out_seq=0 2024-09-13T18:33:01.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.542+0000 7fbf897fa640 10 client.27829 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:33:01.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.544+0000 7fbf897fa640 1 -- 172.21.15.114:0/293878043 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56541717aeb0 con 0x565417530fe0 2024-09-13T18:33:01.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.544+0000 7fbf897fa640 10 client.27829 handle_command_reply: tid=0 2024-09-13T18:33:01.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.544+0000 7fbfaafe8740 1 client.27829 shutdown 2024-09-13T18:33:01.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.558+0000 7fbfaafe8740 1 -- 172.21.15.114:0/293878043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654174bf2c0 msgr2=0x5654174c3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:01.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.558+0000 7fbfaafe8740 1 --2- 172.21.15.114:0/293878043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654174bf2c0 0x5654174c3140 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fbf90098030 tx=0x7fbf900a3cf0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.558+0000 7fbfaafe8740 1 -- 172.21.15.114:0/293878043 shutdown_connections 2024-09-13T18:33:01.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.558+0000 7fbfaafe8740 1 --2- 172.21.15.114:0/293878043 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x565417530fe0 0x5654175333d0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.558+0000 7fbfaafe8740 1 --2- 172.21.15.114:0/293878043 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654174c9b60 0x5654174c3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.558+0000 7fbfaafe8740 1 --2- 172.21.15.114:0/293878043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654174bf2c0 0x5654174c3140 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.558+0000 7fbfaafe8740 1 --2- 172.21.15.114:0/293878043 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5654174be9a0 0x5654174d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.558+0000 7fbfaafe8740 1 -- 172.21.15.114:0/293878043 >> 172.21.15.114:0/293878043 conn(0x5654174baf30 msgr2=0x5654174bb310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:01.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.562+0000 7fbf897fa640 0 client.27829 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:33:01.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.562+0000 7fbfaafe8740 1 -- 172.21.15.114:0/293878043 shutdown_connections 2024-09-13T18:33:01.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.562+0000 7fbfaafe8740 1 -- 172.21.15.114:0/293878043 wait complete. 2024-09-13T18:33:01.568 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:33:01.568 INFO:teuthology.orchestra.run.smithi114.stdout: "return_code": 0, 2024-09-13T18:33:01.568 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_tag": "555b5e0e-6a75-44d5-a8d6-df6176f96f2d", 2024-09-13T18:33:01.568 INFO:teuthology.orchestra.run.smithi114.stdout: "mode": "asynchronous" 2024-09-13T18:33:01.568 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:33:01.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.565+0000 7fbfaafe8740 20 client.27829 trim_cache size 0 max 16384 2024-09-13T18:33:01.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.569+0000 7fbfa6f8b640 1 -- 172.21.15.114:0/120501049 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbf8c0a1410 msgr2=0x7fbf8c0a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:01.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.569+0000 7fbfa6f8b640 1 --2- 172.21.15.114:0/120501049 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbf8c0a1410 0x7fbf8c0a38d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x5654174baa30 tx=0x7fbf98014950 comp rx=0 tx=0).stop 2024-09-13T18:33:01.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.569+0000 7fbfa6f8b640 1 -- 172.21.15.114:0/120501049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfa0107890 msgr2=0x7fbfa01a5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:01.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.569+0000 7fbfa6f8b640 1 --2- 172.21.15.114:0/120501049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfa0107890 0x7fbfa01a5570 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fbf9000c9a0 tx=0x7fbf9000ce60 comp rx=0 tx=0).stop 2024-09-13T18:33:01.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.569+0000 7fbfa6f8b640 1 -- 172.21.15.114:0/120501049 shutdown_connections 2024-09-13T18:33:01.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.569+0000 7fbfa6f8b640 1 --2- 172.21.15.114:0/120501049 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbf8c0a1410 0x7fbf8c0a38d0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.569+0000 7fbfa6f8b640 1 --2- 172.21.15.114:0/120501049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbfa0119400 0x7fbfa01a9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.569+0000 7fbfa6f8b640 1 --2- 172.21.15.114:0/120501049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfa0107890 0x7fbfa01a5570 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.569+0000 7fbfa6f8b640 1 --2- 172.21.15.114:0/120501049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbfa01074b0 0x7fbfa01a5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.569+0000 7fbfa6f8b640 1 -- 172.21.15.114:0/120501049 >> 172.21.15.114:0/120501049 conn(0x7fbfa006af40 msgr2=0x7fbfa0119cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:01.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.572+0000 7fbfa6f8b640 1 -- 172.21.15.114:0/120501049 shutdown_connections 2024-09-13T18:33:01.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.572+0000 7fbfa6f8b640 1 -- 172.21.15.114:0/120501049 wait complete. 2024-09-13T18:33:01.597 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T18:33:01.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.814+0000 7f33ab65a640 1 -- 172.21.15.114:0/3750077787 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f33a4119400 msgr2=0x7f33a411b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:01.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.814+0000 7f33ab65a640 1 --2- 172.21.15.114:0/3750077787 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f33a4119400 0x7f33a411b7f0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f3394009900 tx=0x7f339402a440 comp rx=0 tx=0).stop 2024-09-13T18:33:01.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.821+0000 7f33ab65a640 1 -- 172.21.15.114:0/3750077787 shutdown_connections 2024-09-13T18:33:01.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.821+0000 7f33ab65a640 1 --2- 172.21.15.114:0/3750077787 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f33a4119400 0x7f33a411b7f0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.821+0000 7f33ab65a640 1 --2- 172.21.15.114:0/3750077787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33a4107920 0x7f33a4107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.821+0000 7f33ab65a640 1 --2- 172.21.15.114:0/3750077787 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33a4107540 0x7f33a410e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.821+0000 7f33ab65a640 1 -- 172.21.15.114:0/3750077787 >> 172.21.15.114:0/3750077787 conn(0x7f33a406afe0 msgr2=0x7f33a406b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:01.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.827+0000 7f33ab65a640 1 -- 172.21.15.114:0/3750077787 shutdown_connections 2024-09-13T18:33:01.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.832+0000 7f33ab65a640 1 -- 172.21.15.114:0/3750077787 wait complete. 2024-09-13T18:33:01.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.832+0000 7f33ab65a640 1 Processor -- start 2024-09-13T18:33:01.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.832+0000 7f33ab65a640 1 -- start start 2024-09-13T18:33:01.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.833+0000 7f33ab65a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33a4107540 0x7f33a41a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:01.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.833+0000 7f33ab65a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f33a4107920 0x7f33a41a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:01.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.833+0000 7f33ab65a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33a4119400 0x7f33a41a9a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:01.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.833+0000 7f33ab65a640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a41aa060 con 0x7f33a4107920 2024-09-13T18:33:01.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.833+0000 7f33ab65a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a41aa1d0 con 0x7f33a4119400 2024-09-13T18:33:01.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.833+0000 7f33ab65a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a41aa390 con 0x7f33a4107540 2024-09-13T18:33:01.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.833+0000 7f33a9bf7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33a4119400 0x7f33a41a9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:01.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.834+0000 7f33a9bf7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33a4119400 0x7f33a41a9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:47966/0 (socket says 172.21.15.114:47966) 2024-09-13T18:33:01.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.834+0000 7f33a9bf7640 1 -- 172.21.15.114:0/4276416359 learned_addr learned my addr 172.21.15.114:0/4276416359 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:01.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.836+0000 7f33a93f6640 1 --2- 172.21.15.114:0/4276416359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33a4107540 0x7f33a41a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:01.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.836+0000 7f33a8bf5640 1 --2- 172.21.15.114:0/4276416359 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f33a4107920 0x7f33a41a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:01.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.837+0000 7f33a8bf5640 1 -- 172.21.15.114:0/4276416359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33a4107540 msgr2=0x7f33a41a5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:01.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.837+0000 7f33a8bf5640 1 --2- 172.21.15.114:0/4276416359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33a4107540 0x7f33a41a5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.837+0000 7f33a8bf5640 1 -- 172.21.15.114:0/4276416359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33a4119400 msgr2=0x7f33a41a9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:01.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.837+0000 7f33a8bf5640 1 --2- 172.21.15.114:0/4276416359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33a4119400 0x7f33a41a9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.837+0000 7f33a8bf5640 1 -- 172.21.15.114:0/4276416359 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3394009560 con 0x7f33a4107920 2024-09-13T18:33:01.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.840+0000 7f33a8bf5640 1 --2- 172.21.15.114:0/4276416359 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f33a4107920 0x7f33a41a5670 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f33a0009fd0 tx=0x7f33a000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:01.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.842+0000 7f339a7fc640 1 -- 172.21.15.114:0/4276416359 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a0011830 con 0x7f33a4107920 2024-09-13T18:33:01.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.842+0000 7f33ab65a640 1 -- 172.21.15.114:0/4276416359 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33a41aa670 con 0x7f33a4107920 2024-09-13T18:33:01.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.842+0000 7f33ab65a640 1 -- 172.21.15.114:0/4276416359 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33a40788f0 con 0x7f33a4107920 2024-09-13T18:33:01.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.843+0000 7f339a7fc640 1 -- 172.21.15.114:0/4276416359 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f33a0011e20 con 0x7f33a4107920 2024-09-13T18:33:01.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.843+0000 7f339a7fc640 1 -- 172.21.15.114:0/4276416359 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a000f2e0 con 0x7f33a4107920 2024-09-13T18:33:01.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.853+0000 7f339a7fc640 1 -- 172.21.15.114:0/4276416359 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f33a001a400 con 0x7f33a4107920 2024-09-13T18:33:01.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.855+0000 7f339a7fc640 1 --2- 172.21.15.114:0/4276416359 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f33900a1410 0x7f33900a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:01.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.856+0000 7f339a7fc640 1 -- 172.21.15.114:0/4276416359 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f33a00ccdc0 con 0x7f33a4107920 2024-09-13T18:33:01.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.856+0000 7f33a93f6640 1 --2- 172.21.15.114:0/4276416359 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f33900a1410 0x7f33900a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:01.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.860+0000 7f33ae456740 1 Processor -- start 2024-09-13T18:33:01.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.862+0000 7f33a93f6640 1 --2- 172.21.15.114:0/4276416359 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f33900a1410 0x7f33900a38d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f339c009f40 tx=0x7f339c0094d0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:01.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.865+0000 7f33ae456740 1 -- start start 2024-09-13T18:33:01.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.865+0000 7f33ae456740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e63c24e10 0x555e63c251f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:01.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.865+0000 7f33ae456740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e63c25730 0x555e63c2fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:01.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.866+0000 7f33ae456740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e63c2ffd0 0x555e63c323c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:01.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.866+0000 7f33a93f6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e63c24e10 0x555e63c251f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:01.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.866+0000 7f33a93f6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e63c24e10 0x555e63c251f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:47978/0 (socket says 172.21.15.114:47978) 2024-09-13T18:33:01.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.866+0000 7f33a93f6640 1 -- 172.21.15.114:0/2466927298 learned_addr learned my addr 172.21.15.114:0/2466927298 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:01.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.866+0000 7f33ae456740 1 -- 172.21.15.114:0/2466927298 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e63b8ff20 con 0x555e63c2ffd0 2024-09-13T18:33:01.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.866+0000 7f33ae456740 1 -- 172.21.15.114:0/2466927298 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e63b8d6b0 con 0x555e63c24e10 2024-09-13T18:33:01.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.866+0000 7f33ae456740 1 -- 172.21.15.114:0/2466927298 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e63b88fa0 con 0x555e63c25730 2024-09-13T18:33:01.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.867+0000 7f33a8bf5640 1 --2- 172.21.15.114:0/2466927298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e63c25730 0x555e63c2fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:01.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.868+0000 7f33a9bf7640 1 --2- 172.21.15.114:0/2466927298 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e63c2ffd0 0x555e63c323c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:01.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.870+0000 7f33a9bf7640 1 -- 172.21.15.114:0/2466927298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e63c25730 msgr2=0x555e63c2fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:01.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.870+0000 7f33a9bf7640 1 --2- 172.21.15.114:0/2466927298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e63c25730 0x555e63c2fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.870+0000 7f33a9bf7640 1 -- 172.21.15.114:0/2466927298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e63c24e10 msgr2=0x555e63c251f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:01.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.870+0000 7f33a9bf7640 1 --2- 172.21.15.114:0/2466927298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e63c24e10 0x555e63c251f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.870+0000 7f33a9bf7640 1 -- 172.21.15.114:0/2466927298 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e63b43840 con 0x555e63c2ffd0 2024-09-13T18:33:01.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.871+0000 7f33a9bf7640 1 --2- 172.21.15.114:0/2466927298 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e63c2ffd0 0x555e63c323c0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f3394005ab0 tx=0x7f3394004310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be98ace4b89df9e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:01.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.875+0000 7f338f7fe640 1 -- 172.21.15.114:0/2466927298 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3394005200 con 0x555e63c2ffd0 2024-09-13T18:33:01.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.875+0000 7f338f7fe640 1 -- 172.21.15.114:0/2466927298 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3394005360 con 0x555e63c2ffd0 2024-09-13T18:33:01.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.875+0000 7f338f7fe640 1 -- 172.21.15.114:0/2466927298 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3394003920 con 0x555e63c2ffd0 2024-09-13T18:33:01.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.876+0000 7f33ae456740 1 -- 172.21.15.114:0/2466927298 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e63c2ffd0 msgr2=0x555e63c323c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:01.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.877+0000 7f33ae456740 1 --2- 172.21.15.114:0/2466927298 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e63c2ffd0 0x555e63c323c0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f3394005ab0 tx=0x7f3394004310 comp rx=0 tx=0).stop 2024-09-13T18:33:01.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.883+0000 7f33ae456740 1 -- 172.21.15.114:0/2466927298 shutdown_connections 2024-09-13T18:33:01.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.883+0000 7f33ae456740 1 --2- 172.21.15.114:0/2466927298 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e63c2ffd0 0x555e63c323c0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.883+0000 7f33ae456740 1 --2- 172.21.15.114:0/2466927298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e63c25730 0x555e63c2fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.883+0000 7f33ae456740 1 --2- 172.21.15.114:0/2466927298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e63c24e10 0x555e63c251f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.883+0000 7f33ae456740 1 -- 172.21.15.114:0/2466927298 >> 172.21.15.114:0/2466927298 conn(0x555e63c21380 msgr2=0x555e63c21760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:01.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.888+0000 7f33ae456740 1 -- 172.21.15.114:0/2466927298 shutdown_connections 2024-09-13T18:33:01.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.889+0000 7f33ae456740 1 -- 172.21.15.114:0/2466927298 wait complete. 2024-09-13T18:33:01.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.889+0000 7f33ae456740 1 -- start start 2024-09-13T18:33:01.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.889+0000 7f33ae456740 1 Processor -- start 2024-09-13T18:33:01.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.889+0000 7f33ae456740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e63c24e10 0x555e63c2f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:01.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.890+0000 7f33ae456740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e63c25730 0x555e63c2cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:01.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.890+0000 7f33ae456740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e63c2ffd0 0x555e63c2d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:01.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.890+0000 7f33ae456740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e63b2a2a0 con 0x555e63c2ffd0 2024-09-13T18:33:01.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.890+0000 7f33ae456740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e63861430 con 0x555e63c24e10 2024-09-13T18:33:01.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.890+0000 7f33ae456740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e63b0c680 con 0x555e63c25730 2024-09-13T18:33:01.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.890+0000 7f33ae456740 10 client.0 _ll_register_callbacks cb 0x555e63b82610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:33:01.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.890+0000 7f33a93f6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e63c24e10 0x555e63c2f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:01.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.890+0000 7f33a93f6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e63c24e10 0x555e63c2f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:47992/0 (socket says 172.21.15.114:47992) 2024-09-13T18:33:01.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.890+0000 7f33a93f6640 1 -- 172.21.15.114:0/3227523861 learned_addr learned my addr 172.21.15.114:0/3227523861 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:01.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.890+0000 7f33a8bf5640 1 --2- 172.21.15.114:0/3227523861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e63c25730 0x555e63c2cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:01.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.891+0000 7f33a93f6640 1 -- 172.21.15.114:0/3227523861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e63c25730 msgr2=0x555e63c2cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:01.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.891+0000 7f33a93f6640 1 --2- 172.21.15.114:0/3227523861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e63c25730 0x555e63c2cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.891+0000 7f33a93f6640 1 -- 172.21.15.114:0/3227523861 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e63c2ffd0 msgr2=0x555e63c2d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:01.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.891+0000 7f33a93f6640 1 --2- 172.21.15.114:0/3227523861 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e63c2ffd0 0x555e63c2d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.891+0000 7f33a93f6640 1 -- 172.21.15.114:0/3227523861 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33a411c2e0 con 0x555e63c24e10 2024-09-13T18:33:01.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.891+0000 7f33a93f6640 1 --2- 172.21.15.114:0/3227523861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e63c24e10 0x555e63c2f9a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f339c017e60 tx=0x7f339c011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a41f4e45764216 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:01.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.891+0000 7f338cff9640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:33:01.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.891+0000 7f338cff9640 1 -- 172.21.15.114:0/3227523861 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339c01e040 con 0x555e63c24e10 2024-09-13T18:33:01.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.892+0000 7f338cff9640 1 -- 172.21.15.114:0/3227523861 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f339c016b70 con 0x555e63c24e10 2024-09-13T18:33:01.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.892+0000 7f33ae456740 1 -- 172.21.15.114:0/3227523861 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555e6387c4e0 con 0x555e63c24e10 2024-09-13T18:33:01.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.892+0000 7f338cff9640 1 -- 172.21.15.114:0/3227523861 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f339c011070 con 0x555e63c24e10 2024-09-13T18:33:01.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.894+0000 7f33ae456740 10 client.27911 fetch_fsmap learned FSMap version 34 2024-09-13T18:33:01.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.894+0000 7f33ae456740 1 -- 172.21.15.114:0/3227523861 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x555e63b87620 con 0x555e63c24e10 2024-09-13T18:33:01.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.895+0000 7f338cff9640 1 -- 172.21.15.114:0/3227523861 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f339c016d30 con 0x555e63c24e10 2024-09-13T18:33:01.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.895+0000 7f338cff9640 1 -- 172.21.15.114:0/3227523861 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f339c02c070 con 0x555e63c24e10 2024-09-13T18:33:01.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.895+0000 7f33ae456740 10 client.27911 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:33:01.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.895+0000 7f33ae456740 10 client.27911 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:33:01.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.895+0000 7f33ae456740 20 client.27911 populate_metadata read hostname 'smithi114' 2024-09-13T18:33:01.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.895+0000 7f33ae456740 1 --2- 172.21.15.114:0/3227523861 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x555e63c50d40 0x555e63c531e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:01.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.895+0000 7f33ae456740 4 client.27911 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:33:01.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.895+0000 7f33ae456740 1 -- 172.21.15.114:0/3227523861 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555e6387c4e0 con 0x555e63c50d40 2024-09-13T18:33:01.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.900+0000 7f33a8bf5640 1 --2- 172.21.15.114:0/3227523861 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x555e63c50d40 0x555e63c531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:01.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.914+0000 7f33a8bf5640 1 --2- 172.21.15.114:0/3227523861 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x555e63c50d40 0x555e63c531e0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3249bb464425ecc3 server_cookie=8dd435b796d6ca91 in_seq=0 out_seq=0 2024-09-13T18:33:01.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.915+0000 7f338cff9640 10 client.27911 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:33:01.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.919+0000 7f338cff9640 1 -- 172.21.15.114:0/3227523861 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x555e6387c4e0 con 0x555e63c50d40 2024-09-13T18:33:01.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.919+0000 7f338cff9640 10 client.27911 handle_command_reply: tid=0 2024-09-13T18:33:01.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.922+0000 7f33ae456740 1 client.27911 shutdown 2024-09-13T18:33:01.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.928+0000 7f33ae456740 1 -- 172.21.15.114:0/3227523861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e63c24e10 msgr2=0x555e63c2f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:01.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.928+0000 7f33ae456740 1 --2- 172.21.15.114:0/3227523861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e63c24e10 0x555e63c2f9a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f339c017e60 tx=0x7f339c011000 comp rx=0 tx=0).stop 2024-09-13T18:33:01.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.932+0000 7f33ae456740 1 -- 172.21.15.114:0/3227523861 shutdown_connections 2024-09-13T18:33:01.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.932+0000 7f33ae456740 1 --2- 172.21.15.114:0/3227523861 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x555e63c50d40 0x555e63c531e0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.933+0000 7f33ae456740 1 --2- 172.21.15.114:0/3227523861 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e63c2ffd0 0x555e63c2d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.933+0000 7f33ae456740 1 --2- 172.21.15.114:0/3227523861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e63c25730 0x555e63c2cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.933+0000 7f33ae456740 1 --2- 172.21.15.114:0/3227523861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e63c24e10 0x555e63c2f9a0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.933+0000 7f33ae456740 1 -- 172.21.15.114:0/3227523861 >> 172.21.15.114:0/3227523861 conn(0x555e63c21380 msgr2=0x555e63c27090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:01.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.933+0000 7f338cff9640 0 client.27911 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:33:01.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.935+0000 7f33ae456740 1 -- 172.21.15.114:0/3227523861 shutdown_connections 2024-09-13T18:33:01.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.938+0000 7f33ae456740 1 -- 172.21.15.114:0/3227523861 wait complete. 2024-09-13T18:33:01.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.939+0000 7f33ae456740 20 client.27911 trim_cache size 0 max 16384 2024-09-13T18:33:01.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.970+0000 7f33ae456740 1 Processor -- start 2024-09-13T18:33:01.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.971+0000 7f33ae456740 1 -- start start 2024-09-13T18:33:01.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.971+0000 7f33ae456740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e63c24e10 0x555e63c2f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:01.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.971+0000 7f33ae456740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e63c25730 0x555e63c21760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:01.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.972+0000 7f33a93f6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e63c24e10 0x555e63c2f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:01.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.972+0000 7f33a93f6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e63c24e10 0x555e63c2f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:39552/0 (socket says 172.21.15.114:39552) 2024-09-13T18:33:01.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.972+0000 7f33a93f6640 1 -- 172.21.15.114:0/3470863299 learned_addr learned my addr 172.21.15.114:0/3470863299 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:01.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.972+0000 7f33a8bf5640 1 --2- 172.21.15.114:0/3470863299 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e63c25730 0x555e63c21760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:01.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.972+0000 7f33ae456740 1 --2- 172.21.15.114:0/3470863299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e63c2ffd0 0x555e63c21ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:01.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.972+0000 7f33ae456740 1 -- 172.21.15.114:0/3470863299 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e63c4f6f0 con 0x555e63c25730 2024-09-13T18:33:01.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.972+0000 7f33ae456740 1 -- 172.21.15.114:0/3470863299 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e63c4db40 con 0x555e63c2ffd0 2024-09-13T18:33:01.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.972+0000 7f33ae456740 1 -- 172.21.15.114:0/3470863299 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e63c1b710 con 0x555e63c24e10 2024-09-13T18:33:01.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.972+0000 7f33a9bf7640 1 --2- 172.21.15.114:0/3470863299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e63c2ffd0 0x555e63c21ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:01.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.973+0000 7f33a8bf5640 1 -- 172.21.15.114:0/3470863299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e63c24e10 msgr2=0x555e63c2f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:01.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.973+0000 7f33a8bf5640 1 --2- 172.21.15.114:0/3470863299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e63c24e10 0x555e63c2f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.973+0000 7f33a8bf5640 1 -- 172.21.15.114:0/3470863299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e63c2ffd0 msgr2=0x555e63c21ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:01.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.973+0000 7f33a8bf5640 1 --2- 172.21.15.114:0/3470863299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e63c2ffd0 0x555e63c21ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.973+0000 7f33a8bf5640 1 -- 172.21.15.114:0/3470863299 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e63b29790 con 0x555e63c25730 2024-09-13T18:33:01.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.975+0000 7f33a8bf5640 1 --2- 172.21.15.114:0/3470863299 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e63c25730 0x555e63c21760 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f33a00ca1f0 tx=0x7f33a0013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3c5053608a18d52 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:01.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.981+0000 7f338ffff640 1 -- 172.21.15.114:0/3470863299 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a009a2e0 con 0x555e63c25730 2024-09-13T18:33:01.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.981+0000 7f338ffff640 1 -- 172.21.15.114:0/3470863299 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f33a0099be0 con 0x555e63c25730 2024-09-13T18:33:01.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.985+0000 7f338ffff640 1 -- 172.21.15.114:0/3470863299 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a00ca390 con 0x555e63c25730 2024-09-13T18:33:01.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.986+0000 7f33ae456740 1 -- 172.21.15.114:0/3470863299 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e63c25730 msgr2=0x555e63c21760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:01.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.986+0000 7f33ae456740 1 --2- 172.21.15.114:0/3470863299 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e63c25730 0x555e63c21760 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f33a00ca1f0 tx=0x7f33a0013040 comp rx=0 tx=0).stop 2024-09-13T18:33:01.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.991+0000 7f33ae456740 1 -- 172.21.15.114:0/3470863299 shutdown_connections 2024-09-13T18:33:01.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.991+0000 7f33ae456740 1 --2- 172.21.15.114:0/3470863299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e63c2ffd0 0x555e63c21ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.991+0000 7f33ae456740 1 --2- 172.21.15.114:0/3470863299 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e63c25730 0x555e63c21760 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.991+0000 7f33ae456740 1 --2- 172.21.15.114:0/3470863299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e63c24e10 0x555e63c2f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:01.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.991+0000 7f33ae456740 1 -- 172.21.15.114:0/3470863299 >> 172.21.15.114:0/3470863299 conn(0x555e63c21380 msgr2=0x555e63c27f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:01.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.995+0000 7f33ae456740 1 -- 172.21.15.114:0/3470863299 shutdown_connections 2024-09-13T18:33:01.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.995+0000 7f33ae456740 1 -- 172.21.15.114:0/3470863299 wait complete. 2024-09-13T18:33:01.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.995+0000 7f33ae456740 1 -- start start 2024-09-13T18:33:01.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.995+0000 7f33ae456740 1 Processor -- start 2024-09-13T18:33:01.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.996+0000 7f33ae456740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e63c24e10 0x555e63c9f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:01.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.996+0000 7f33ae456740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e63c25730 0x555e63c9fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:01.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.996+0000 7f33ae456740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e63c2ffd0 0x555e63c441f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:01.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.996+0000 7f33ae456740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e63c448f0 con 0x555e63c2ffd0 2024-09-13T18:33:01.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.996+0000 7f33ae456740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e63c44a60 con 0x555e63c25730 2024-09-13T18:33:01.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.996+0000 7f33ae456740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e63c44c20 con 0x555e63c24e10 2024-09-13T18:33:01.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.996+0000 7f33ae456740 10 client.0 _ll_register_callbacks cb 0x7f339c0175b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:33:02.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.997+0000 7f33a93f6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e63c24e10 0x555e63c9f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:02.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.997+0000 7f33a93f6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e63c24e10 0x555e63c9f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:39568/0 (socket says 172.21.15.114:39568) 2024-09-13T18:33:02.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.997+0000 7f33a93f6640 1 -- 172.21.15.114:0/1722666320 learned_addr learned my addr 172.21.15.114:0/1722666320 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:02.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:01.999+0000 7f33a9bf7640 1 --2- 172.21.15.114:0/1722666320 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e63c2ffd0 0x555e63c441f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:02.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:02.000+0000 7f33a8bf5640 1 --2- 172.21.15.114:0/1722666320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e63c25730 0x555e63c9fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:02.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:02.003+0000 7f33a93f6640 1 -- 172.21.15.114:0/1722666320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e63c25730 msgr2=0x555e63c9fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:02.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:02.003+0000 7f33a93f6640 1 --2- 172.21.15.114:0/1722666320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e63c25730 0x555e63c9fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:02.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:02.003+0000 7f33a93f6640 1 -- 172.21.15.114:0/1722666320 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e63c2ffd0 msgr2=0x555e63c441f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:02.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:02.003+0000 7f33a93f6640 1 --2- 172.21.15.114:0/1722666320 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e63c2ffd0 0x555e63c441f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:02.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:02.003+0000 7f33a93f6640 1 -- 172.21.15.114:0/1722666320 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33a40788f0 con 0x555e63c24e10 2024-09-13T18:33:02.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:02.004+0000 7f33a93f6640 1 --2- 172.21.15.114:0/1722666320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e63c24e10 0x555e63c9f580 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f339c02d3a0 tx=0x7f339c003c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ed4b95644563e60 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:02.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:02.005+0000 7f338cff9640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:33:02.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:02.005+0000 7f338cff9640 1 -- 172.21.15.114:0/1722666320 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339c01e030 con 0x555e63c24e10 2024-09-13T18:33:02.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:02.005+0000 7f33ae456740 1 -- 172.21.15.114:0/1722666320 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3394005360 con 0x555e63c24e10 2024-09-13T18:33:02.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:02.006+0000 7f338cff9640 1 -- 172.21.15.114:0/1722666320 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f339c02d610 con 0x555e63c24e10 2024-09-13T18:33:02.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:02.009+0000 7f338cff9640 1 -- 172.21.15.114:0/1722666320 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f339c003e80 con 0x555e63c24e10 2024-09-13T18:33:02.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:02.010+0000 7f33ae456740 10 client.27844 fetch_fsmap learned FSMap version 34 2024-09-13T18:33:02.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:02.010+0000 7f33ae456740 1 -- 172.21.15.114:0/1722666320 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x555e63b4eea0 con 0x555e63c24e10 2024-09-13T18:33:02.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:02.012+0000 7f338cff9640 1 -- 172.21.15.114:0/1722666320 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3394005360 con 0x555e63c24e10 2024-09-13T18:33:02.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:02.012+0000 7f338cff9640 1 -- 172.21.15.114:0/1722666320 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f339c02c040 con 0x555e63c24e10 2024-09-13T18:33:02.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:02.012+0000 7f33ae456740 10 client.27844 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:33:02.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:02.012+0000 7f33ae456740 10 client.27844 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:33:02.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:02.012+0000 7f33ae456740 20 client.27844 populate_metadata read hostname 'smithi114' 2024-09-13T18:33:02.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:02.012+0000 7f33ae456740 1 --2- 172.21.15.114:0/1722666320 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x555e63ca8480 0x555e63caa8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:02.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:02.012+0000 7f33ae456740 4 client.27844 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T18:33:02.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:02.012+0000 7f33ae456740 1 -- 172.21.15.114:0/1722666320 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555e6388d7e0 con 0x555e63ca8480 2024-09-13T18:33:02.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:02.017+0000 7f33a8bf5640 1 --2- 172.21.15.114:0/1722666320 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x555e63ca8480 0x555e63caa8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:02.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:02.023+0000 7f33a8bf5640 1 --2- 172.21.15.114:0/1722666320 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x555e63ca8480 0x555e63caa8a0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=deb5b88eddfbfb45 server_cookie=d2dcba23e73e43 in_seq=0 out_seq=0 2024-09-13T18:33:02.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:02.023+0000 7f338cff9640 10 client.27844 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:33:02.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:02.025+0000 7f338cff9640 1 -- 172.21.15.114:0/1722666320 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x555e6388d7e0 con 0x555e63ca8480 2024-09-13T18:33:02.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:02.025+0000 7f338cff9640 10 client.27844 handle_command_reply: tid=0 2024-09-13T18:33:02.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:02.025+0000 7f33ae456740 1 client.27844 shutdown 2024-09-13T18:33:02.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:02.031+0000 7f33ae456740 1 -- 172.21.15.114:0/1722666320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e63c24e10 msgr2=0x555e63c9f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:02.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:02.031+0000 7f33ae456740 1 --2- 172.21.15.114:0/1722666320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e63c24e10 0x555e63c9f580 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f339c02d3a0 tx=0x7f339c003c60 comp rx=0 tx=0).stop 2024-09-13T18:33:02.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:02.031+0000 7f33ae456740 1 -- 172.21.15.114:0/1722666320 shutdown_connections 2024-09-13T18:33:02.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:02.032+0000 7f33ae456740 1 --2- 172.21.15.114:0/1722666320 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x555e63ca8480 0x555e63caa8a0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:02.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:02.032+0000 7f33ae456740 1 --2- 172.21.15.114:0/1722666320 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e63c2ffd0 0x555e63c441f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:02.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:02.032+0000 7f33ae456740 1 --2- 172.21.15.114:0/1722666320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e63c25730 0x555e63c9fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:02.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:02.032+0000 7f33ae456740 1 --2- 172.21.15.114:0/1722666320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e63c24e10 0x555e63c9f580 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:02.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:02.032+0000 7f33ae456740 1 -- 172.21.15.114:0/1722666320 >> 172.21.15.114:0/1722666320 conn(0x555e63c21380 msgr2=0x555e63c21760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:02.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:02.032+0000 7f338cff9640 0 client.27844 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:33:02.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:02.034+0000 7f33ae456740 1 -- 172.21.15.114:0/1722666320 shutdown_connections 2024-09-13T18:33:02.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:02.035+0000 7f33ae456740 1 -- 172.21.15.114:0/1722666320 wait complete. 2024-09-13T18:33:02.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:02.037+0000 7f33ae456740 20 client.27844 trim_cache size 0 max 16384 2024-09-13T18:33:02.044 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:33:02.044 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-13T18:33:02.044 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": { 2024-09-13T18:33:02.044 INFO:teuthology.orchestra.run.smithi114.stdout: "555b5e0e-6a75-44d5-a8d6-df6176f96f2d": { 2024-09-13T18:33:02.045 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-13T18:33:02.045 INFO:teuthology.orchestra.run.smithi114.stdout: "tag": "555b5e0e-6a75-44d5-a8d6-df6176f96f2d", 2024-09-13T18:33:02.045 INFO:teuthology.orchestra.run.smithi114.stdout: "options": "recursive,force" 2024-09-13T18:33:02.045 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:33:02.045 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:33:02.045 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:33:02.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:02.049+0000 7f33ab65a640 1 -- 172.21.15.114:0/4276416359 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f33900a1410 msgr2=0x7f33900a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:02.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:02.049+0000 7f33ab65a640 1 --2- 172.21.15.114:0/4276416359 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f33900a1410 0x7f33900a38d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f339c009f40 tx=0x7f339c0094d0 comp rx=0 tx=0).stop 2024-09-13T18:33:02.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:02.049+0000 7f33ab65a640 1 -- 172.21.15.114:0/4276416359 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f33a4107920 msgr2=0x7f33a41a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:02.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:02.049+0000 7f33ab65a640 1 --2- 172.21.15.114:0/4276416359 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f33a4107920 0x7f33a41a5670 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f33a0009fd0 tx=0x7f33a000b870 comp rx=0 tx=0).stop 2024-09-13T18:33:02.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:02.050+0000 7f33ab65a640 1 -- 172.21.15.114:0/4276416359 shutdown_connections 2024-09-13T18:33:02.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:02.050+0000 7f33ab65a640 1 --2- 172.21.15.114:0/4276416359 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f33900a1410 0x7f33900a38d0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:02.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:02.050+0000 7f33ab65a640 1 --2- 172.21.15.114:0/4276416359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33a4119400 0x7f33a41a9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:02.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:02.050+0000 7f33ab65a640 1 --2- 172.21.15.114:0/4276416359 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f33a4107920 0x7f33a41a5670 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:02.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:02.050+0000 7f33ab65a640 1 --2- 172.21.15.114:0/4276416359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33a4107540 0x7f33a41a5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:02.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:02.050+0000 7f33ab65a640 1 -- 172.21.15.114:0/4276416359 >> 172.21.15.114:0/4276416359 conn(0x7f33a406afe0 msgr2=0x7f33a4119b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:02.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:02.050+0000 7f33ab65a640 1 -- 172.21.15.114:0/4276416359 shutdown_connections 2024-09-13T18:33:02.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:02.050+0000 7f33ab65a640 1 -- 172.21.15.114:0/4276416359 wait complete. 2024-09-13T18:33:02.080 INFO:tasks.cephfs.filesystem:scrub status for tag:555b5e0e-6a75-44d5-a8d6-df6176f96f2d - {'path': '/', 'tag': '555b5e0e-6a75-44d5-a8d6-df6176f96f2d', 'options': 'recursive,force'} 2024-09-13T18:33:02.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:02 smithi204 ceph-mon[34777]: pgmap v327: 129 pgs: 129 active+clean; 21 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 33 MiB/s wr, 730 op/s 2024-09-13T18:33:02.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:02 smithi204 ceph-mon[34777]: scrub summary: active paths [/] 2024-09-13T18:33:02.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:02 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:33:02.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:02 smithi114 ceph-mon[30897]: pgmap v327: 129 pgs: 129 active+clean; 21 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 33 MiB/s wr, 730 op/s 2024-09-13T18:33:02.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:02 smithi114 ceph-mon[30897]: scrub summary: active paths [/] 2024-09-13T18:33:02.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:02 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:33:02.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:02 smithi134 ceph-mon[34283]: pgmap v327: 129 pgs: 129 active+clean; 21 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 33 MiB/s wr, 730 op/s 2024-09-13T18:33:02.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:02 smithi134 ceph-mon[34283]: scrub summary: active paths [/] 2024-09-13T18:33:02.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:02 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:33:03.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:03 smithi114 ceph-mon[30897]: scrub queued for path: / 2024-09-13T18:33:03.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:03 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:33:03.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:03 smithi114 ceph-mon[30897]: scrub summary: active paths [/] 2024-09-13T18:33:03.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:03 smithi114 ceph-mon[30897]: scrub summary: active paths [/] 2024-09-13T18:33:03.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:03 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:33:03.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:03 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:33:03.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:03 smithi134 ceph-mon[34283]: scrub queued for path: / 2024-09-13T18:33:03.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:03 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:33:03.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:03 smithi134 ceph-mon[34283]: scrub summary: active paths [/] 2024-09-13T18:33:03.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:03 smithi134 ceph-mon[34283]: scrub summary: active paths [/] 2024-09-13T18:33:03.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:03 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:33:03.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:03 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:33:03.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:03 smithi204 ceph-mon[34777]: scrub queued for path: / 2024-09-13T18:33:03.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:03 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:33:03.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:03 smithi204 ceph-mon[34777]: scrub summary: active paths [/] 2024-09-13T18:33:03.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:03 smithi204 ceph-mon[34777]: scrub summary: active paths [/] 2024-09-13T18:33:03.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:03 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:33:03.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:03 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:33:04.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:04 smithi114 ceph-mon[30897]: pgmap v328: 129 pgs: 129 active+clean; 21 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 28 MiB/s wr, 626 op/s 2024-09-13T18:33:04.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:04 smithi134 ceph-mon[34283]: pgmap v328: 129 pgs: 129 active+clean; 21 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 28 MiB/s wr, 626 op/s 2024-09-13T18:33:04.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:04 smithi204 ceph-mon[34777]: pgmap v328: 129 pgs: 129 active+clean; 21 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 28 MiB/s wr, 626 op/s 2024-09-13T18:33:05.433 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.430+0000 7fb9869f4640 1 -- 172.21.15.114:0/1417603478 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb980112d70 msgr2=0x7fb980115160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:05.434 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.430+0000 7fb9869f4640 1 --2- 172.21.15.114:0/1417603478 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb980112d70 0x7fb980115160 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fb970009900 tx=0x7fb97002e850 comp rx=0 tx=0).stop 2024-09-13T18:33:05.436 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.431+0000 7fb9869f4640 1 -- 172.21.15.114:0/1417603478 shutdown_connections 2024-09-13T18:33:05.436 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.431+0000 7fb9869f4640 1 --2- 172.21.15.114:0/1417603478 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb980112d70 0x7fb980115160 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:05.436 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.431+0000 7fb9869f4640 1 --2- 172.21.15.114:0/1417603478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb980107dd0 0x7fb980112830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:05.436 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.431+0000 7fb9869f4640 1 --2- 172.21.15.114:0/1417603478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9801074b0 0x7fb980107890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:05.436 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.431+0000 7fb9869f4640 1 -- 172.21.15.114:0/1417603478 >> 172.21.15.114:0/1417603478 conn(0x7fb98006af50 msgr2=0x7fb98006b360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:05.442 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.434+0000 7fb9869f4640 1 -- 172.21.15.114:0/1417603478 shutdown_connections 2024-09-13T18:33:05.442 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.435+0000 7fb9869f4640 1 -- 172.21.15.114:0/1417603478 wait complete. 2024-09-13T18:33:05.442 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.437+0000 7fb9869f4640 1 Processor -- start 2024-09-13T18:33:05.442 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.437+0000 7fb9869f4640 1 -- start start 2024-09-13T18:33:05.442 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.437+0000 7fb9869f4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9801074b0 0x7fb98019c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:05.442 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.438+0000 7fb9869f4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb980107dd0 0x7fb98019cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:05.442 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.438+0000 7fb97ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9801074b0 0x7fb98019c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:05.442 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.438+0000 7fb97ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9801074b0 0x7fb98019c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:32904/0 (socket says 172.21.15.114:32904) 2024-09-13T18:33:05.442 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.438+0000 7fb97ffff640 1 -- 172.21.15.114:0/489151801 learned_addr learned my addr 172.21.15.114:0/489151801 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:05.442 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.438+0000 7fb97f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb980107dd0 0x7fb98019cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:05.442 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.438+0000 7fb9869f4640 1 --2- 172.21.15.114:0/489151801 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb980112d70 0x7fb9801a0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:05.443 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.438+0000 7fb9869f4640 1 -- 172.21.15.114:0/489151801 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9801a1570 con 0x7fb980112d70 2024-09-13T18:33:05.443 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.438+0000 7fb9869f4640 1 -- 172.21.15.114:0/489151801 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9801a16e0 con 0x7fb9801074b0 2024-09-13T18:33:05.443 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.438+0000 7fb9869f4640 1 -- 172.21.15.114:0/489151801 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9801a18a0 con 0x7fb980107dd0 2024-09-13T18:33:05.443 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.438+0000 7fb984f91640 1 --2- 172.21.15.114:0/489151801 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb980112d70 0x7fb9801a0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:05.444 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.440+0000 7fb97ffff640 1 -- 172.21.15.114:0/489151801 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb980107dd0 msgr2=0x7fb98019cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:05.444 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.440+0000 7fb97ffff640 1 --2- 172.21.15.114:0/489151801 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb980107dd0 0x7fb98019cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:05.444 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.440+0000 7fb97ffff640 1 -- 172.21.15.114:0/489151801 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb980112d70 msgr2=0x7fb9801a0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:05.444 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.440+0000 7fb97ffff640 1 --2- 172.21.15.114:0/489151801 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb980112d70 0x7fb9801a0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:05.444 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.440+0000 7fb97ffff640 1 -- 172.21.15.114:0/489151801 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb970009560 con 0x7fb9801074b0 2024-09-13T18:33:05.450 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.445+0000 7fb97ffff640 1 --2- 172.21.15.114:0/489151801 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9801074b0 0x7fb98019c640 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fb97800ba50 tx=0x7fb97800bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:05.450 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.446+0000 7fb97d7fa640 1 -- 172.21.15.114:0/489151801 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb97800cbe0 con 0x7fb9801074b0 2024-09-13T18:33:05.450 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.446+0000 7fb9869f4640 1 -- 172.21.15.114:0/489151801 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9801a1b80 con 0x7fb9801074b0 2024-09-13T18:33:05.450 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.446+0000 7fb9869f4640 1 -- 172.21.15.114:0/489151801 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9800788f0 con 0x7fb9801074b0 2024-09-13T18:33:05.450 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.446+0000 7fb97d7fa640 1 -- 172.21.15.114:0/489151801 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb97800cd40 con 0x7fb9801074b0 2024-09-13T18:33:05.450 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.446+0000 7fb97d7fa640 1 -- 172.21.15.114:0/489151801 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9780076e0 con 0x7fb9801074b0 2024-09-13T18:33:05.466 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.463+0000 7fb97d7fa640 1 -- 172.21.15.114:0/489151801 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fb978007840 con 0x7fb9801074b0 2024-09-13T18:33:05.466 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.463+0000 7fb9897f1740 1 Processor -- start 2024-09-13T18:33:05.466 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.463+0000 7fb97d7fa640 1 --2- 172.21.15.114:0/489151801 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fb9680a9c20 0x7fb9680ac0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:05.467 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.464+0000 7fb97d7fa640 1 -- 172.21.15.114:0/489151801 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fb9780cc630 con 0x7fb9801074b0 2024-09-13T18:33:05.467 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.464+0000 7fb984f91640 1 --2- 172.21.15.114:0/489151801 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fb9680a9c20 0x7fb9680ac0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:05.468 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.465+0000 7fb984f91640 1 --2- 172.21.15.114:0/489151801 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fb9680a9c20 0x7fb9680ac0e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb970004f50 tx=0x7fb970004da0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:05.470 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.467+0000 7fb9897f1740 1 -- start start 2024-09-13T18:33:05.470 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.467+0000 7fb9897f1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f47f2e330 0x557f47f2e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:05.470 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.467+0000 7fb9897f1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557f47f2ec50 0x557f47f38fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:05.470 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.467+0000 7fb9897f1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f47f394f0 0x557f47f3b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:05.471 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.467+0000 7fb9897f1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f47e99c30 con 0x557f47f2ec50 2024-09-13T18:33:05.471 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.467+0000 7fb9897f1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f47e973c0 con 0x557f47f2e330 2024-09-13T18:33:05.471 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.467+0000 7fb9897f1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f47e92cb0 con 0x557f47f394f0 2024-09-13T18:33:05.471 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.467+0000 7fb97ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f47f2e330 0x557f47f2e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:05.472 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.467+0000 7fb97f7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557f47f2ec50 0x557f47f38fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:05.472 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.467+0000 7fb97ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f47f2e330 0x557f47f2e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:32908/0 (socket says 172.21.15.114:32908) 2024-09-13T18:33:05.472 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.467+0000 7fb97ffff640 1 -- 172.21.15.114:0/3101790956 learned_addr learned my addr 172.21.15.114:0/3101790956 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:05.472 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.468+0000 7fb97f7fe640 1 -- 172.21.15.114:0/3101790956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f47f394f0 msgr2=0x557f47f3b8e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:33:05.472 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.468+0000 7fb97f7fe640 1 --2- 172.21.15.114:0/3101790956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f47f394f0 0x557f47f3b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:05.472 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.468+0000 7fb97f7fe640 1 -- 172.21.15.114:0/3101790956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f47f2e330 msgr2=0x557f47f2e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.468+0000 7fb97f7fe640 1 --2- 172.21.15.114:0/3101790956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f47f2e330 0x557f47f2e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.468+0000 7fb97f7fe640 1 -- 172.21.15.114:0/3101790956 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f47e39910 con 0x557f47f2ec50 2024-09-13T18:33:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.468+0000 7fb97f7fe640 1 --2- 172.21.15.114:0/3101790956 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557f47f2ec50 0x557f47f38fb0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fb974006fd0 tx=0x7fb974009e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d03584eb128012fd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:05.474 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.469+0000 7fb97ffff640 1 --2- 172.21.15.114:0/3101790956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f47f2e330 0x557f47f2e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:33:05.474 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.470+0000 7fb9667fc640 1 -- 172.21.15.114:0/3101790956 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb97400ea80 con 0x557f47f2ec50 2024-09-13T18:33:05.474 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.470+0000 7fb9667fc640 1 -- 172.21.15.114:0/3101790956 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb97400ebe0 con 0x557f47f2ec50 2024-09-13T18:33:05.474 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.471+0000 7fb9897f1740 1 -- 172.21.15.114:0/3101790956 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557f47f2ec50 msgr2=0x557f47f38fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:05.474 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.471+0000 7fb9897f1740 1 --2- 172.21.15.114:0/3101790956 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557f47f2ec50 0x557f47f38fb0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fb974006fd0 tx=0x7fb974009e10 comp rx=0 tx=0).stop 2024-09-13T18:33:05.476 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.472+0000 7fb9897f1740 1 -- 172.21.15.114:0/3101790956 shutdown_connections 2024-09-13T18:33:05.477 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.472+0000 7fb9897f1740 1 --2- 172.21.15.114:0/3101790956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f47f394f0 0x557f47f3b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:05.477 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.472+0000 7fb9897f1740 1 --2- 172.21.15.114:0/3101790956 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557f47f2ec50 0x557f47f38fb0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:05.477 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.472+0000 7fb9897f1740 1 --2- 172.21.15.114:0/3101790956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f47f2e330 0x557f47f2e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:05.477 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.472+0000 7fb9897f1740 1 -- 172.21.15.114:0/3101790956 >> 172.21.15.114:0/3101790956 conn(0x557f47f2a8c0 msgr2=0x557f47f2aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.475+0000 7fb9897f1740 1 -- 172.21.15.114:0/3101790956 shutdown_connections 2024-09-13T18:33:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.475+0000 7fb9897f1740 1 -- 172.21.15.114:0/3101790956 wait complete. 2024-09-13T18:33:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.475+0000 7fb9897f1740 1 -- start start 2024-09-13T18:33:05.479 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.476+0000 7fb9897f1740 1 Processor -- start 2024-09-13T18:33:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.480+0000 7fb9897f1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557f47f2e330 0x557f47f38d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.480+0000 7fb9897f1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f47f2ec50 0x557f47f36040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.480+0000 7fb9897f1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f47f394f0 0x557f47f36580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.480+0000 7fb9897f1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f47e50e80 con 0x557f47f2e330 2024-09-13T18:33:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.480+0000 7fb9897f1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f47e50b20 con 0x557f47f2ec50 2024-09-13T18:33:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.480+0000 7fb9897f1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f47e16260 con 0x557f47f394f0 2024-09-13T18:33:05.486 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.480+0000 7fb9897f1740 10 client.0 _ll_register_callbacks cb 0x557f47e4bdf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:33:05.486 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.480+0000 7fb97f7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f47f2ec50 0x557f47f36040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:05.486 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.480+0000 7fb97f7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f47f2ec50 0x557f47f36040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:32924/0 (socket says 172.21.15.114:32924) 2024-09-13T18:33:05.486 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.480+0000 7fb97f7fe640 1 -- 172.21.15.114:0/764375983 learned_addr learned my addr 172.21.15.114:0/764375983 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:05.486 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.481+0000 7fb984f91640 1 --2- 172.21.15.114:0/764375983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f47f394f0 0x557f47f36580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:05.486 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.482+0000 7fb97f7fe640 1 -- 172.21.15.114:0/764375983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f47f394f0 msgr2=0x557f47f36580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.482+0000 7fb97f7fe640 1 --2- 172.21.15.114:0/764375983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f47f394f0 0x557f47f36580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.482+0000 7fb97f7fe640 1 -- 172.21.15.114:0/764375983 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557f47f2e330 msgr2=0x557f47f38d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.482+0000 7fb97f7fe640 1 --2- 172.21.15.114:0/764375983 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557f47f2e330 0x557f47f38d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.482+0000 7fb97f7fe640 1 -- 172.21.15.114:0/764375983 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9740095e0 con 0x557f47f2ec50 2024-09-13T18:33:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.483+0000 7fb97f7fe640 1 --2- 172.21.15.114:0/764375983 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f47f2ec50 0x557f47f36040 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fb974013cd0 tx=0x7fb974013a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=883b16f31dd55286 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:05.488 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.484+0000 7fb95f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:33:05.488 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.484+0000 7fb95f7fe640 1 -- 172.21.15.114:0/764375983 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb974004240 con 0x557f47f2ec50 2024-09-13T18:33:05.488 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.485+0000 7fb9897f1740 1 -- 172.21.15.114:0/764375983 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557f47ebb680 con 0x557f47f2ec50 2024-09-13T18:33:05.488 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.485+0000 7fb95f7fe640 1 -- 172.21.15.114:0/764375983 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fb9740125d0 con 0x557f47f2ec50 2024-09-13T18:33:05.495 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.492+0000 7fb95f7fe640 1 -- 172.21.15.114:0/764375983 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9740043a0 con 0x557f47f2ec50 2024-09-13T18:33:05.495 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.492+0000 7fb9897f1740 10 client.27923 fetch_fsmap learned FSMap version 34 2024-09-13T18:33:05.495 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.492+0000 7fb9897f1740 1 -- 172.21.15.114:0/764375983 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x557f47e91330 con 0x557f47f2ec50 2024-09-13T18:33:05.496 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.493+0000 7fb95f7fe640 1 -- 172.21.15.114:0/764375983 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557f47ebb680 con 0x557f47f2ec50 2024-09-13T18:33:05.496 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.493+0000 7fb95f7fe640 1 -- 172.21.15.114:0/764375983 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb974011070 con 0x557f47f2ec50 2024-09-13T18:33:05.496 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.493+0000 7fb9897f1740 10 client.27923 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:33:05.496 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.493+0000 7fb9897f1740 10 client.27923 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:33:05.496 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.493+0000 7fb9897f1740 20 client.27923 populate_metadata read hostname 'smithi114' 2024-09-13T18:33:05.496 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.493+0000 7fb9897f1740 1 --2- 172.21.15.114:0/764375983 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x557f47f48de0 0x557f47f4b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:05.497 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.493+0000 7fb9897f1740 4 client.27923 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:33:05.497 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.493+0000 7fb9897f1740 1 -- 172.21.15.114:0/764375983 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557f47e865d0 con 0x557f47f48de0 2024-09-13T18:33:05.498 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.495+0000 7fb97ffff640 1 --2- 172.21.15.114:0/764375983 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x557f47f48de0 0x557f47f4b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:05.503 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.500+0000 7fb97ffff640 1 --2- 172.21.15.114:0/764375983 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x557f47f48de0 0x557f47f4b200 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc629ecdebc92691 server_cookie=71003f5b4a7fdf1b in_seq=0 out_seq=0 2024-09-13T18:33:05.503 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.500+0000 7fb95f7fe640 10 client.27923 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:33:05.503 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.500+0000 7fb95f7fe640 1 -- 172.21.15.114:0/764375983 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x557f47e865d0 con 0x557f47f48de0 2024-09-13T18:33:05.503 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.500+0000 7fb95f7fe640 10 client.27923 handle_command_reply: tid=0 2024-09-13T18:33:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.502+0000 7fb9897f1740 1 client.27923 shutdown 2024-09-13T18:33:05.513 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.507+0000 7fb9897f1740 1 -- 172.21.15.114:0/764375983 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f47f2ec50 msgr2=0x557f47f36040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:05.513 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.507+0000 7fb9897f1740 1 --2- 172.21.15.114:0/764375983 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f47f2ec50 0x557f47f36040 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fb974013cd0 tx=0x7fb974013a20 comp rx=0 tx=0).stop 2024-09-13T18:33:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.511+0000 7fb9897f1740 1 -- 172.21.15.114:0/764375983 shutdown_connections 2024-09-13T18:33:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.511+0000 7fb9897f1740 1 --2- 172.21.15.114:0/764375983 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x557f47f48de0 0x557f47f4b200 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.511+0000 7fb9897f1740 1 --2- 172.21.15.114:0/764375983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f47f394f0 0x557f47f36580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.511+0000 7fb9897f1740 1 --2- 172.21.15.114:0/764375983 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f47f2ec50 0x557f47f36040 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.511+0000 7fb9897f1740 1 --2- 172.21.15.114:0/764375983 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557f47f2e330 0x557f47f38d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.511+0000 7fb9897f1740 1 -- 172.21.15.114:0/764375983 >> 172.21.15.114:0/764375983 conn(0x557f47f2a8c0 msgr2=0x557f47f29b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.511+0000 7fb95f7fe640 0 client.27923 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:33:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.512+0000 7fb9897f1740 1 -- 172.21.15.114:0/764375983 shutdown_connections 2024-09-13T18:33:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.514+0000 7fb9897f1740 1 -- 172.21.15.114:0/764375983 wait complete. 2024-09-13T18:33:05.524 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.521+0000 7fb9897f1740 20 client.27923 trim_cache size 0 max 16384 2024-09-13T18:33:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.545+0000 7fb9897f1740 1 Processor -- start 2024-09-13T18:33:05.549 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.545+0000 7fb9897f1740 1 -- start start 2024-09-13T18:33:05.549 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.545+0000 7fb9897f1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f47f2e330 0x557f47f348f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:05.549 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.546+0000 7fb9897f1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f47f2ec50 0x557f47f34e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:05.550 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.546+0000 7fb9897f1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557f47f394f0 0x557f47f33280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:05.550 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.546+0000 7fb97ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f47f2e330 0x557f47f348f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:05.550 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.546+0000 7fb97ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f47f2e330 0x557f47f348f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:32934/0 (socket says 172.21.15.114:32934) 2024-09-13T18:33:05.550 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.546+0000 7fb97f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f47f2ec50 0x557f47f34e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:05.550 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.546+0000 7fb97f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f47f2ec50 0x557f47f34e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:56342/0 (socket says 172.21.15.114:56342) 2024-09-13T18:33:05.550 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.546+0000 7fb97ffff640 1 -- 172.21.15.114:0/3438554466 learned_addr learned my addr 172.21.15.114:0/3438554466 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:05.550 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.546+0000 7fb9897f1740 1 -- 172.21.15.114:0/3438554466 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f47f35370 con 0x557f47f394f0 2024-09-13T18:33:05.551 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.546+0000 7fb9897f1740 1 -- 172.21.15.114:0/3438554466 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f47f368f0 con 0x557f47f2e330 2024-09-13T18:33:05.551 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.546+0000 7fb9897f1740 1 -- 172.21.15.114:0/3438554466 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f47f2f2a0 con 0x557f47f2ec50 2024-09-13T18:33:05.551 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.547+0000 7fb984f91640 1 --2- 172.21.15.114:0/3438554466 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557f47f394f0 0x557f47f33280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:05.551 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.547+0000 7fb97ffff640 1 -- 172.21.15.114:0/3438554466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f47f2ec50 msgr2=0x557f47f34e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:05.551 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.547+0000 7fb97ffff640 1 --2- 172.21.15.114:0/3438554466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f47f2ec50 0x557f47f34e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:05.551 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.547+0000 7fb97ffff640 1 -- 172.21.15.114:0/3438554466 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557f47f394f0 msgr2=0x557f47f33280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:05.551 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.547+0000 7fb97ffff640 1 --2- 172.21.15.114:0/3438554466 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557f47f394f0 0x557f47f33280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:05.551 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.547+0000 7fb97ffff640 1 -- 172.21.15.114:0/3438554466 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f47e33300 con 0x557f47f2e330 2024-09-13T18:33:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.551+0000 7fb97ffff640 1 --2- 172.21.15.114:0/3438554466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f47f2e330 0x557f47f348f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb9780994a0 tx=0x7fb978095520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a8446fa9a8fb4e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.557+0000 7fb966ffd640 1 -- 172.21.15.114:0/3438554466 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb978095ed0 con 0x557f47f2e330 2024-09-13T18:33:05.561 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.557+0000 7fb966ffd640 1 -- 172.21.15.114:0/3438554466 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb9780cc040 con 0x557f47f2e330 2024-09-13T18:33:05.561 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.557+0000 7fb966ffd640 1 -- 172.21.15.114:0/3438554466 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb978098ef0 con 0x557f47f2e330 2024-09-13T18:33:05.561 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.558+0000 7fb9897f1740 1 -- 172.21.15.114:0/3438554466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f47f2e330 msgr2=0x557f47f348f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:05.561 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.558+0000 7fb9897f1740 1 --2- 172.21.15.114:0/3438554466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f47f2e330 0x557f47f348f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb9780994a0 tx=0x7fb978095520 comp rx=0 tx=0).stop 2024-09-13T18:33:05.561 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.558+0000 7fb9897f1740 1 -- 172.21.15.114:0/3438554466 shutdown_connections 2024-09-13T18:33:05.561 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.558+0000 7fb9897f1740 1 --2- 172.21.15.114:0/3438554466 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557f47f394f0 0x557f47f33280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:05.561 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.558+0000 7fb9897f1740 1 --2- 172.21.15.114:0/3438554466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f47f2ec50 0x557f47f34e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:05.562 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.558+0000 7fb9897f1740 1 --2- 172.21.15.114:0/3438554466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f47f2e330 0x557f47f348f0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:05.562 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.558+0000 7fb9897f1740 1 -- 172.21.15.114:0/3438554466 >> 172.21.15.114:0/3438554466 conn(0x557f47f2a8c0 msgr2=0x557f47f312d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.562+0000 7fb9897f1740 1 -- 172.21.15.114:0/3438554466 shutdown_connections 2024-09-13T18:33:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.562+0000 7fb9897f1740 1 -- 172.21.15.114:0/3438554466 wait complete. 2024-09-13T18:33:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.562+0000 7fb9897f1740 1 -- start start 2024-09-13T18:33:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.562+0000 7fb9897f1740 1 Processor -- start 2024-09-13T18:33:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.563+0000 7fb9897f1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f47f2e330 0x557f47f3f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.563+0000 7fb9897f1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f47f2ec50 0x557f47f3f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.563+0000 7fb9897f1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557f47f394f0 0x557f47f9f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:05.567 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.563+0000 7fb9897f1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f47f3fdd0 con 0x557f47f394f0 2024-09-13T18:33:05.567 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.563+0000 7fb9897f1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f47f3ff40 con 0x557f47f2ec50 2024-09-13T18:33:05.567 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.563+0000 7fb9897f1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f47fb0890 con 0x557f47f2e330 2024-09-13T18:33:05.567 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.563+0000 7fb9897f1740 10 client.0 _ll_register_callbacks cb 0x7fb9740093d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:33:05.567 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.564+0000 7fb984f91640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557f47f394f0 0x557f47f9f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:05.567 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.564+0000 7fb984f91640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557f47f394f0 0x557f47f9f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60632/0 (socket says 172.21.15.114:60632) 2024-09-13T18:33:05.567 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.564+0000 7fb984f91640 1 -- 172.21.15.114:0/1885196811 learned_addr learned my addr 172.21.15.114:0/1885196811 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:05.569 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.566+0000 7fb97f7fe640 1 --2- 172.21.15.114:0/1885196811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f47f2ec50 0x557f47f3f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:05.569 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.566+0000 7fb97ffff640 1 --2- 172.21.15.114:0/1885196811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f47f2e330 0x557f47f3f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:05.570 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.567+0000 7fb97f7fe640 1 -- 172.21.15.114:0/1885196811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f47f2e330 msgr2=0x557f47f3f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:05.570 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.567+0000 7fb97f7fe640 1 --2- 172.21.15.114:0/1885196811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f47f2e330 0x557f47f3f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.567+0000 7fb97f7fe640 1 -- 172.21.15.114:0/1885196811 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557f47f394f0 msgr2=0x557f47f9f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.567+0000 7fb97f7fe640 1 --2- 172.21.15.114:0/1885196811 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557f47f394f0 0x557f47f9f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.567+0000 7fb97f7fe640 1 -- 172.21.15.114:0/1885196811 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9801a1b80 con 0x557f47f2ec50 2024-09-13T18:33:05.572 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.568+0000 7fb97f7fe640 1 --2- 172.21.15.114:0/1885196811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f47f2ec50 0x557f47f3f5f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb974006fd0 tx=0x7fb974009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4a5a9afbdbb877d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:05.572 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.569+0000 7fb95f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:33:05.572 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.569+0000 7fb95f7fe640 1 -- 172.21.15.114:0/1885196811 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb974004010 con 0x557f47f2ec50 2024-09-13T18:33:05.572 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.569+0000 7fb9897f1740 1 -- 172.21.15.114:0/1885196811 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb97400ebe0 con 0x557f47f2ec50 2024-09-13T18:33:05.572 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.569+0000 7fb95f7fe640 1 -- 172.21.15.114:0/1885196811 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fb974011e90 con 0x557f47f2ec50 2024-09-13T18:33:05.576 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.573+0000 7fb95f7fe640 1 -- 172.21.15.114:0/1885196811 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb974009120 con 0x557f47f2ec50 2024-09-13T18:33:05.577 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.573+0000 7fb9897f1740 10 client.27929 fetch_fsmap learned FSMap version 34 2024-09-13T18:33:05.577 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.573+0000 7fb9897f1740 1 -- 172.21.15.114:0/1885196811 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x557f47e588e0 con 0x557f47f2ec50 2024-09-13T18:33:05.581 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.577+0000 7fb95f7fe640 1 -- 172.21.15.114:0/1885196811 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb97400ebe0 con 0x557f47f2ec50 2024-09-13T18:33:05.581 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.577+0000 7fb95f7fe640 1 -- 172.21.15.114:0/1885196811 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb97402d040 con 0x557f47f2ec50 2024-09-13T18:33:05.581 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.578+0000 7fb9897f1740 10 client.27929 fetch_fsmap finished waiting for FSMap version 34 2024-09-13T18:33:05.581 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.578+0000 7fb9897f1740 10 client.27929 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:33:05.581 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.578+0000 7fb9897f1740 20 client.27929 populate_metadata read hostname 'smithi114' 2024-09-13T18:33:05.581 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.578+0000 7fb9897f1740 1 --2- 172.21.15.114:0/1885196811 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x557f47fb1c60 0x557f47fb4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:05.581 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.578+0000 7fb9897f1740 4 client.27929 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T18:33:05.581 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.578+0000 7fb9897f1740 1 -- 172.21.15.114:0/1885196811 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557f47b86300 con 0x557f47fb1c60 2024-09-13T18:33:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.579+0000 7fb97ffff640 1 --2- 172.21.15.114:0/1885196811 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x557f47fb1c60 0x557f47fb4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.582+0000 7fb95f7fe640 10 client.27929 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:33:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.584+0000 7fb97ffff640 1 --2- 172.21.15.114:0/1885196811 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x557f47fb1c60 0x557f47fb4080 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e42ca8f4cf61b05c server_cookie=555d3524df24a9db in_seq=0 out_seq=0 2024-09-13T18:33:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.649+0000 7fb95f7fe640 1 -- 172.21.15.114:0/1885196811 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557f47b86300 con 0x557f47fb1c60 2024-09-13T18:33:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.649+0000 7fb95f7fe640 10 client.27929 handle_command_reply: tid=0 2024-09-13T18:33:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.649+0000 7fb9897f1740 1 client.27929 shutdown 2024-09-13T18:33:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.651+0000 7fb9897f1740 1 -- 172.21.15.114:0/1885196811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f47f2ec50 msgr2=0x557f47f3f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.651+0000 7fb9897f1740 1 --2- 172.21.15.114:0/1885196811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f47f2ec50 0x557f47f3f5f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb974006fd0 tx=0x7fb974009800 comp rx=0 tx=0).stop 2024-09-13T18:33:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.652+0000 7fb9897f1740 1 -- 172.21.15.114:0/1885196811 shutdown_connections 2024-09-13T18:33:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.652+0000 7fb9897f1740 1 --2- 172.21.15.114:0/1885196811 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x557f47fb1c60 0x557f47fb4080 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.652+0000 7fb9897f1740 1 --2- 172.21.15.114:0/1885196811 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557f47f394f0 0x557f47f9f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.652+0000 7fb9897f1740 1 --2- 172.21.15.114:0/1885196811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f47f2ec50 0x557f47f3f5f0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.652+0000 7fb9897f1740 1 --2- 172.21.15.114:0/1885196811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f47f2e330 0x557f47f3f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.652+0000 7fb9897f1740 1 -- 172.21.15.114:0/1885196811 >> 172.21.15.114:0/1885196811 conn(0x557f47f2a8c0 msgr2=0x557f47f312d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.652+0000 7fb95f7fe640 0 client.27929 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:33:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.656+0000 7fb9897f1740 1 -- 172.21.15.114:0/1885196811 shutdown_connections 2024-09-13T18:33:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.656+0000 7fb9897f1740 1 -- 172.21.15.114:0/1885196811 wait complete. 2024-09-13T18:33:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.660+0000 7fb9897f1740 20 client.27929 trim_cache size 0 max 16384 2024-09-13T18:33:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-13T18:33:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-13T18:33:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-13T18:33:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-13T18:33:05.672 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.669+0000 7fb9869f4640 1 -- 172.21.15.114:0/489151801 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fb9680a9c20 msgr2=0x7fb9680ac0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:05.672 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.669+0000 7fb9869f4640 1 --2- 172.21.15.114:0/489151801 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fb9680a9c20 0x7fb9680ac0e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb970004f50 tx=0x7fb970004da0 comp rx=0 tx=0).stop 2024-09-13T18:33:05.672 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.669+0000 7fb9869f4640 1 -- 172.21.15.114:0/489151801 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9801074b0 msgr2=0x7fb98019c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:05.672 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.669+0000 7fb9869f4640 1 --2- 172.21.15.114:0/489151801 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9801074b0 0x7fb98019c640 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fb97800ba50 tx=0x7fb97800bf10 comp rx=0 tx=0).stop 2024-09-13T18:33:05.673 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.670+0000 7fb9869f4640 1 -- 172.21.15.114:0/489151801 shutdown_connections 2024-09-13T18:33:05.674 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.670+0000 7fb9869f4640 1 --2- 172.21.15.114:0/489151801 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fb9680a9c20 0x7fb9680ac0e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:05.674 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.670+0000 7fb9869f4640 1 --2- 172.21.15.114:0/489151801 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb980112d70 0x7fb9801a0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:05.674 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.670+0000 7fb9869f4640 1 --2- 172.21.15.114:0/489151801 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb980107dd0 0x7fb98019cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:05.674 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.670+0000 7fb9869f4640 1 --2- 172.21.15.114:0/489151801 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9801074b0 0x7fb98019c640 secure :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fb97800ba50 tx=0x7fb97800bf10 comp rx=0 tx=0).stop 2024-09-13T18:33:05.674 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.670+0000 7fb9869f4640 1 -- 172.21.15.114:0/489151801 >> 172.21.15.114:0/489151801 conn(0x7fb98006af50 msgr2=0x7fb98006ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:05.677 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.674+0000 7fb9869f4640 1 -- 172.21.15.114:0/489151801 shutdown_connections 2024-09-13T18:33:05.678 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:05.674+0000 7fb9869f4640 1 -- 172.21.15.114:0/489151801 wait complete. 2024-09-13T18:33:06.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:06 smithi114 ceph-mon[30897]: pgmap v329: 129 pgs: 129 active+clean; 21 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 26 MiB/s wr, 595 op/s 2024-09-13T18:33:06.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:06 smithi134 ceph-mon[34283]: pgmap v329: 129 pgs: 129 active+clean; 21 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 26 MiB/s wr, 595 op/s 2024-09-13T18:33:06.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:06 smithi204 ceph-mon[34777]: pgmap v329: 129 pgs: 129 active+clean; 21 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 26 MiB/s wr, 595 op/s 2024-09-13T18:33:08.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:08 smithi114 ceph-mon[30897]: pgmap v330: 129 pgs: 129 active+clean; 21 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 32 MiB/s wr, 732 op/s 2024-09-13T18:33:08.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:08 smithi114 ceph-mon[30897]: osdmap e103: 12 total, 12 up, 12 in 2024-09-13T18:33:08.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:08 smithi134 ceph-mon[34283]: pgmap v330: 129 pgs: 129 active+clean; 21 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 32 MiB/s wr, 732 op/s 2024-09-13T18:33:08.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:08 smithi134 ceph-mon[34283]: osdmap e103: 12 total, 12 up, 12 in 2024-09-13T18:33:08.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:08 smithi204 ceph-mon[34777]: pgmap v330: 129 pgs: 129 active+clean; 21 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 32 MiB/s wr, 732 op/s 2024-09-13T18:33:08.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:08 smithi204 ceph-mon[34777]: osdmap e103: 12 total, 12 up, 12 in 2024-09-13T18:33:10.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:10 smithi134 ceph-mon[34283]: pgmap v332: 129 pgs: 129 active+clean; 21 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 29 MiB/s wr, 652 op/s 2024-09-13T18:33:10.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:10 smithi204 ceph-mon[34777]: pgmap v332: 129 pgs: 129 active+clean; 21 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 29 MiB/s wr, 652 op/s 2024-09-13T18:33:10.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:10 smithi114 ceph-mon[30897]: pgmap v332: 129 pgs: 129 active+clean; 21 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 29 MiB/s wr, 652 op/s 2024-09-13T18:33:12.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:12 smithi134 ceph-mon[34283]: pgmap v333: 129 pgs: 14 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 79 active+clean; 21 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 27 MiB/s wr, 610 op/s 2024-09-13T18:33:12.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:12 smithi134 ceph-mon[34283]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-13T18:33:12.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:12 smithi204 ceph-mon[34777]: pgmap v333: 129 pgs: 14 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 79 active+clean; 21 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 27 MiB/s wr, 610 op/s 2024-09-13T18:33:12.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:12 smithi204 ceph-mon[34777]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-13T18:33:12.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:12 smithi114 ceph-mon[30897]: pgmap v333: 129 pgs: 14 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 79 active+clean; 21 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 27 MiB/s wr, 610 op/s 2024-09-13T18:33:12.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:12 smithi114 ceph-mon[30897]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-13T18:33:14.496 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:14 smithi204 ceph-mon[34777]: pgmap v334: 129 pgs: 20 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 55 active+clean; 20 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 25 MiB/s wr, 553 op/s 2024-09-13T18:33:14.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:14 smithi134 ceph-mon[34283]: pgmap v334: 129 pgs: 20 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 55 active+clean; 20 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 25 MiB/s wr, 553 op/s 2024-09-13T18:33:14.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:14 smithi114 ceph-mon[30897]: pgmap v334: 129 pgs: 20 active+clean+snaptrim, 54 active+clean+snaptrim_wait, 55 active+clean; 20 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 25 MiB/s wr, 553 op/s 2024-09-13T18:33:16.511 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:16 smithi114 ceph-mon[30897]: pgmap v335: 129 pgs: 18 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 61 active+clean; 20 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 23 MiB/s wr, 509 op/s 2024-09-13T18:33:16.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:16 smithi134 ceph-mon[34283]: pgmap v335: 129 pgs: 18 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 61 active+clean; 20 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 23 MiB/s wr, 509 op/s 2024-09-13T18:33:16.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:16 smithi204 ceph-mon[34777]: pgmap v335: 129 pgs: 18 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 61 active+clean; 20 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 23 MiB/s wr, 509 op/s 2024-09-13T18:33:17.521 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:17 smithi114 ceph-mon[30897]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-13T18:33:17.521 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:17 smithi114 ceph-mon[30897]: scrub summary: idle 2024-09-13T18:33:17.521 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:17 smithi114 ceph-mon[30897]: scrub summary: idle 2024-09-13T18:33:17.521 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:17 smithi114 ceph-mon[30897]: scrub summary: idle 2024-09-13T18:33:17.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:17 smithi134 ceph-mon[34283]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-13T18:33:17.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:17 smithi134 ceph-mon[34283]: scrub summary: idle 2024-09-13T18:33:17.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:17 smithi134 ceph-mon[34283]: scrub summary: idle 2024-09-13T18:33:17.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:17 smithi134 ceph-mon[34283]: scrub summary: idle 2024-09-13T18:33:17.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:17 smithi204 ceph-mon[34777]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-13T18:33:17.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:17 smithi204 ceph-mon[34777]: scrub summary: idle 2024-09-13T18:33:17.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:17 smithi204 ceph-mon[34777]: scrub summary: idle 2024-09-13T18:33:17.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:17 smithi204 ceph-mon[34777]: scrub summary: idle 2024-09-13T18:33:18.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:18 smithi134 ceph-mon[34283]: pgmap v336: 129 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 111 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 21 MiB/s wr, 456 op/s 2024-09-13T18:33:18.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:18 smithi204 ceph-mon[34777]: pgmap v336: 129 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 111 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 21 MiB/s wr, 456 op/s 2024-09-13T18:33:18.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:18 smithi114 ceph-mon[30897]: pgmap v336: 129 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 111 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 21 MiB/s wr, 456 op/s 2024-09-13T18:33:18.827 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.824+0000 7f45b8fc3640 1 -- 172.21.15.114:0/1093163563 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45b4103f80 msgr2=0x7f45b410bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:18.827 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.824+0000 7f45b8fc3640 1 --2- 172.21.15.114:0/1093163563 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45b4103f80 0x7f45b410bad0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f45ac009cd0 tx=0x7f45ac02a8e0 comp rx=0 tx=0).stop 2024-09-13T18:33:18.827 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.824+0000 7f45b8fc3640 1 -- 172.21.15.114:0/1093163563 shutdown_connections 2024-09-13T18:33:18.827 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.824+0000 7f45b8fc3640 1 --2- 172.21.15.114:0/1093163563 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45b4103f80 0x7f45b410bad0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:18.827 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.824+0000 7f45b8fc3640 1 --2- 172.21.15.114:0/1093163563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45b40ff7e0 0x7f45b4103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:18.827 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.824+0000 7f45b8fc3640 1 --2- 172.21.15.114:0/1093163563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45b40fee30 0x7f45b40ff210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:18.827 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.824+0000 7f45b8fc3640 1 -- 172.21.15.114:0/1093163563 >> 172.21.15.114:0/1093163563 conn(0x7f45b40fa8c0 msgr2=0x7f45b40fcce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:18.827 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.824+0000 7f45b8fc3640 1 -- 172.21.15.114:0/1093163563 shutdown_connections 2024-09-13T18:33:18.827 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.824+0000 7f45b8fc3640 1 -- 172.21.15.114:0/1093163563 wait complete. 2024-09-13T18:33:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.825+0000 7f45b8fc3640 1 Processor -- start 2024-09-13T18:33:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.825+0000 7f45b8fc3640 1 -- start start 2024-09-13T18:33:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.825+0000 7f45b8fc3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45b40fee30 0x7f45b4199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.825+0000 7f45b8fc3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45b4103f80 0x7f45b4199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.826+0000 7f45b8fc3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45b4193ba0 0x7f45b4194020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.826+0000 7f45b8fc3640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b4194620 con 0x7f45b4103f80 2024-09-13T18:33:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.826+0000 7f45b1d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45b4103f80 0x7f45b4199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.826+0000 7f45b259c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45b40fee30 0x7f45b4199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.826+0000 7f45b8fc3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b4194790 con 0x7f45b4193ba0 2024-09-13T18:33:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.826+0000 7f45b1d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45b4103f80 0x7f45b4199fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59334/0 (socket says 172.21.15.114:59334) 2024-09-13T18:33:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.826+0000 7f45b1d9b640 1 -- 172.21.15.114:0/2075299352 learned_addr learned my addr 172.21.15.114:0/2075299352 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.826+0000 7f45b8fc3640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b4194950 con 0x7f45b40fee30 2024-09-13T18:33:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.826+0000 7f45b2d9d640 1 --2- 172.21.15.114:0/2075299352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45b4193ba0 0x7f45b4194020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.826+0000 7f45b259c640 1 -- 172.21.15.114:0/2075299352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45b4193ba0 msgr2=0x7f45b4194020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.826+0000 7f45b259c640 1 --2- 172.21.15.114:0/2075299352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45b4193ba0 0x7f45b4194020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.826+0000 7f45b259c640 1 -- 172.21.15.114:0/2075299352 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45b4103f80 msgr2=0x7f45b4199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.826+0000 7f45b259c640 1 --2- 172.21.15.114:0/2075299352 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45b4103f80 0x7f45b4199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.826+0000 7f45b259c640 1 -- 172.21.15.114:0/2075299352 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45ac009900 con 0x7f45b40fee30 2024-09-13T18:33:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.826+0000 7f45b259c640 1 --2- 172.21.15.114:0/2075299352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45b40fee30 0x7f45b4199a70 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f45a400d720 tx=0x7f45a400dbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.827+0000 7f45a37fe640 1 -- 172.21.15.114:0/2075299352 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a4004220 con 0x7f45b40fee30 2024-09-13T18:33:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.827+0000 7f45a37fe640 1 -- 172.21.15.114:0/2075299352 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f45a4009e60 con 0x7f45b40fee30 2024-09-13T18:33:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.827+0000 7f45a37fe640 1 -- 172.21.15.114:0/2075299352 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a40047a0 con 0x7f45b40fee30 2024-09-13T18:33:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.827+0000 7f45b8fc3640 1 -- 172.21.15.114:0/2075299352 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45b41a08e0 con 0x7f45b40fee30 2024-09-13T18:33:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.828+0000 7f45b8fc3640 1 -- 172.21.15.114:0/2075299352 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45b41a0d70 con 0x7f45b40fee30 2024-09-13T18:33:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.830+0000 7f45a37fe640 1 -- 172.21.15.114:0/2075299352 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f45a40099d0 con 0x7f45b40fee30 2024-09-13T18:33:18.833 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.830+0000 7f45a37fe640 1 --2- 172.21.15.114:0/2075299352 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f459c0a3410 0x7f459c0a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:18.833 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.831+0000 7f45a37fe640 1 -- 172.21.15.114:0/2075299352 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f45a40ccaa0 con 0x7f45b40fee30 2024-09-13T18:33:18.833 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.831+0000 7f45b1d9b640 1 --2- 172.21.15.114:0/2075299352 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f459c0a3410 0x7f459c0a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:18.833 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.831+0000 7f45bbdbf740 1 Processor -- start 2024-09-13T18:33:18.835 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.831+0000 7f45bbdbf740 1 -- start start 2024-09-13T18:33:18.835 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.831+0000 7f45b1d9b640 1 --2- 172.21.15.114:0/2075299352 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f459c0a3410 0x7f459c0a58d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f45b4194fc0 tx=0x7f45a8005f90 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:18.835 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.831+0000 7f45bbdbf740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5652f78a78e0 0x5652f78a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:18.835 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.832+0000 7f45bbdbf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652f78a6710 0x5652f78a6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:18.835 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.832+0000 7f45bbdbf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652f78b9b90 0x5652f78bbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:18.835 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.832+0000 7f45bbdbf740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652f7811c30 con 0x5652f78a78e0 2024-09-13T18:33:18.835 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.832+0000 7f45bbdbf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652f780f3c0 con 0x5652f78a6710 2024-09-13T18:33:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.832+0000 7f45bbdbf740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652f780acb0 con 0x5652f78b9b90 2024-09-13T18:33:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.832+0000 7f45b259c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5652f78a78e0 0x5652f78a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.832+0000 7f45b259c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5652f78a78e0 0x5652f78a61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59344/0 (socket says 172.21.15.114:59344) 2024-09-13T18:33:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.832+0000 7f45b2d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652f78b9b90 0x5652f78bbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.832+0000 7f45b259c640 1 -- 172.21.15.114:0/2913118234 learned_addr learned my addr 172.21.15.114:0/2913118234 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.832+0000 7f45b1d9b640 1 --2- 172.21.15.114:0/2913118234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652f78a6710 0x5652f78a6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.832+0000 7f45b2d9d640 1 -- 172.21.15.114:0/2913118234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652f78a6710 msgr2=0x5652f78a6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.832+0000 7f45b2d9d640 1 --2- 172.21.15.114:0/2913118234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652f78a6710 0x5652f78a6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.832+0000 7f45b2d9d640 1 -- 172.21.15.114:0/2913118234 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5652f78a78e0 msgr2=0x5652f78a61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.832+0000 7f45b2d9d640 1 --2- 172.21.15.114:0/2913118234 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5652f78a78e0 0x5652f78a61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.832+0000 7f45b2d9d640 1 -- 172.21.15.114:0/2913118234 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652f77b1910 con 0x5652f78b9b90 2024-09-13T18:33:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.832+0000 7f45b1d9b640 1 --2- 172.21.15.114:0/2913118234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652f78a6710 0x5652f78a6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:33:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.832+0000 7f45b2d9d640 1 --2- 172.21.15.114:0/2913118234 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652f78b9b90 0x5652f78bbf80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f45ac02ae00 tx=0x7f45ac001060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9333f61123f23673 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.833+0000 7f45a0ff9640 1 -- 172.21.15.114:0/2913118234 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ac0307a0 con 0x5652f78b9b90 2024-09-13T18:33:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.833+0000 7f45a0ff9640 1 -- 172.21.15.114:0/2913118234 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f45ac040070 con 0x5652f78b9b90 2024-09-13T18:33:18.841 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.833+0000 7f45a0ff9640 1 -- 172.21.15.114:0/2913118234 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ac003c60 con 0x5652f78b9b90 2024-09-13T18:33:18.841 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.833+0000 7f45bbdbf740 1 -- 172.21.15.114:0/2913118234 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652f78b9b90 msgr2=0x5652f78bbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:18.841 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.833+0000 7f45bbdbf740 1 --2- 172.21.15.114:0/2913118234 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652f78b9b90 0x5652f78bbf80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f45ac02ae00 tx=0x7f45ac001060 comp rx=0 tx=0).stop 2024-09-13T18:33:18.842 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.833+0000 7f45bbdbf740 1 -- 172.21.15.114:0/2913118234 shutdown_connections 2024-09-13T18:33:18.842 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.833+0000 7f45bbdbf740 1 --2- 172.21.15.114:0/2913118234 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652f78b9b90 0x5652f78bbf80 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:18.842 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.833+0000 7f45bbdbf740 1 --2- 172.21.15.114:0/2913118234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652f78a6710 0x5652f78a6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:18.842 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.833+0000 7f45bbdbf740 1 --2- 172.21.15.114:0/2913118234 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5652f78a78e0 0x5652f78a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:18.842 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.833+0000 7f45bbdbf740 1 -- 172.21.15.114:0/2913118234 >> 172.21.15.114:0/2913118234 conn(0x5652f78a28c0 msgr2=0x5652f78a2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:18.842 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.834+0000 7f45bbdbf740 1 -- 172.21.15.114:0/2913118234 shutdown_connections 2024-09-13T18:33:18.842 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.834+0000 7f45bbdbf740 1 -- 172.21.15.114:0/2913118234 wait complete. 2024-09-13T18:33:18.842 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.834+0000 7f45bbdbf740 1 -- start start 2024-09-13T18:33:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.834+0000 7f45bbdbf740 1 Processor -- start 2024-09-13T18:33:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.834+0000 7f45bbdbf740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5652f78a6710 0x5652f78ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.834+0000 7f45bbdbf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652f78a78e0 0x5652f78a9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.834+0000 7f45bbdbf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652f78b9b90 0x5652f78aa0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.834+0000 7f45b259c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5652f78a6710 0x5652f78ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.835+0000 7f45b259c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5652f78a6710 0x5652f78ac8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59354/0 (socket says 172.21.15.114:59354) 2024-09-13T18:33:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.835+0000 7f45b259c640 1 -- 172.21.15.114:0/3182866382 learned_addr learned my addr 172.21.15.114:0/3182866382 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.835+0000 7f45bbdbf740 1 -- 172.21.15.114:0/3182866382 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652f77c8e80 con 0x5652f78a6710 2024-09-13T18:33:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.835+0000 7f45bbdbf740 1 -- 172.21.15.114:0/3182866382 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652f77c8b20 con 0x5652f78b9b90 2024-09-13T18:33:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.835+0000 7f45bbdbf740 1 -- 172.21.15.114:0/3182866382 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652f778e260 con 0x5652f78a78e0 2024-09-13T18:33:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.835+0000 7f45b1d9b640 1 --2- 172.21.15.114:0/3182866382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652f78a78e0 0x5652f78a9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.835+0000 7f45b2d9d640 1 --2- 172.21.15.114:0/3182866382 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652f78b9b90 0x5652f78aa0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.835+0000 7f45bbdbf740 10 client.0 _ll_register_callbacks cb 0x5652f77c3df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:33:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.835+0000 7f45b1d9b640 1 -- 172.21.15.114:0/3182866382 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652f78b9b90 msgr2=0x5652f78aa0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.835+0000 7f45b1d9b640 1 --2- 172.21.15.114:0/3182866382 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652f78b9b90 0x5652f78aa0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.835+0000 7f45b1d9b640 1 -- 172.21.15.114:0/3182866382 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5652f78a6710 msgr2=0x5652f78ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.835+0000 7f45b1d9b640 1 --2- 172.21.15.114:0/3182866382 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5652f78a6710 0x5652f78ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.835+0000 7f45b1d9b640 1 -- 172.21.15.114:0/3182866382 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45b410c310 con 0x5652f78a78e0 2024-09-13T18:33:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.835+0000 7f45b1d9b640 1 --2- 172.21.15.114:0/3182866382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652f78a78e0 0x5652f78a9bb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x5652f78a72a0 tx=0x7f45a8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68df64e8213796e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.835+0000 7f4599ffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:33:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.835+0000 7f4599ffb640 1 -- 172.21.15.114:0/3182866382 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a801eb10 con 0x5652f78a78e0 2024-09-13T18:33:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.835+0000 7f4599ffb640 1 -- 172.21.15.114:0/3182866382 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f45a801ec70 con 0x5652f78a78e0 2024-09-13T18:33:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.836+0000 7f45bbdbf740 1 -- 172.21.15.114:0/3182866382 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5652f7833680 con 0x5652f78a78e0 2024-09-13T18:33:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.836+0000 7f4599ffb640 1 -- 172.21.15.114:0/3182866382 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45a8003e50 con 0x5652f78a78e0 2024-09-13T18:33:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.836+0000 7f45bbdbf740 10 client.27865 fetch_fsmap learned FSMap version 36 2024-09-13T18:33:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.836+0000 7f45bbdbf740 1 -- 172.21.15.114:0/3182866382 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5652f7809330 con 0x5652f78a78e0 2024-09-13T18:33:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.836+0000 7f4599ffb640 1 -- 172.21.15.114:0/3182866382 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5652f7833680 con 0x5652f78a78e0 2024-09-13T18:33:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.836+0000 7f4599ffb640 1 -- 172.21.15.114:0/3182866382 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 36) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f45a8028430 con 0x5652f78a78e0 2024-09-13T18:33:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.836+0000 7f45bbdbf740 10 client.27865 fetch_fsmap finished waiting for FSMap version 36 2024-09-13T18:33:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.836+0000 7f45bbdbf740 10 client.27865 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:33:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.836+0000 7f45bbdbf740 20 client.27865 populate_metadata read hostname 'smithi114' 2024-09-13T18:33:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.836+0000 7f45bbdbf740 1 --2- 172.21.15.114:0/3182866382 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5652f78c9780 0x5652f78cbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.836+0000 7f45bbdbf740 4 client.27865 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:33:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.836+0000 7f45bbdbf740 1 -- 172.21.15.114:0/3182866382 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5652f77fe5d0 con 0x5652f78c9780 2024-09-13T18:33:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.836+0000 7f45b259c640 1 --2- 172.21.15.114:0/3182866382 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5652f78c9780 0x5652f78cbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.837+0000 7f4599ffb640 10 client.27865 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:33:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.837+0000 7f45b259c640 1 --2- 172.21.15.114:0/3182866382 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5652f78c9780 0x5652f78cbbc0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9baa10e785d1c4a2 server_cookie=9b19a86ff69e6b03 in_seq=0 out_seq=0 2024-09-13T18:33:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.838+0000 7f4599ffb640 1 -- 172.21.15.114:0/3182866382 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5652f77fe5d0 con 0x5652f78c9780 2024-09-13T18:33:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.838+0000 7f4599ffb640 10 client.27865 handle_command_reply: tid=0 2024-09-13T18:33:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.838+0000 7f45bbdbf740 1 client.27865 shutdown 2024-09-13T18:33:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.838+0000 7f45bbdbf740 1 -- 172.21.15.114:0/3182866382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652f78a78e0 msgr2=0x5652f78a9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.838+0000 7f45bbdbf740 1 --2- 172.21.15.114:0/3182866382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652f78a78e0 0x5652f78a9bb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x5652f78a72a0 tx=0x7f45a8004730 comp rx=0 tx=0).stop 2024-09-13T18:33:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.838+0000 7f45bbdbf740 1 -- 172.21.15.114:0/3182866382 shutdown_connections 2024-09-13T18:33:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.838+0000 7f45bbdbf740 1 --2- 172.21.15.114:0/3182866382 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5652f78c9780 0x5652f78cbbc0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.838+0000 7f45bbdbf740 1 --2- 172.21.15.114:0/3182866382 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652f78b9b90 0x5652f78aa0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.838+0000 7f4599ffb640 0 client.27865 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:33:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.838+0000 7f45bbdbf740 1 --2- 172.21.15.114:0/3182866382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652f78a78e0 0x5652f78a9bb0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.838+0000 7f45bbdbf740 1 --2- 172.21.15.114:0/3182866382 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5652f78a6710 0x5652f78ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.838+0000 7f45bbdbf740 1 -- 172.21.15.114:0/3182866382 >> 172.21.15.114:0/3182866382 conn(0x5652f78a28c0 msgr2=0x5652f78a1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.838+0000 7f45bbdbf740 1 -- 172.21.15.114:0/3182866382 shutdown_connections 2024-09-13T18:33:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.838+0000 7f45bbdbf740 1 -- 172.21.15.114:0/3182866382 wait complete. 2024-09-13T18:33:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.838+0000 7f45bbdbf740 20 client.27865 trim_cache size 0 max 16384 2024-09-13T18:33:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.854+0000 7f45bbdbf740 1 Processor -- start 2024-09-13T18:33:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.855+0000 7f45bbdbf740 1 -- start start 2024-09-13T18:33:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.855+0000 7f45bbdbf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652f78a6710 0x5652f78a2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.855+0000 7f45bbdbf740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5652f78a78e0 0x5652f78baf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.855+0000 7f45bbdbf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652f78b9b90 0x5652f78ac3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.855+0000 7f45bbdbf740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652f78aaa10 con 0x5652f78a78e0 2024-09-13T18:33:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.855+0000 7f45bbdbf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652f789e210 con 0x5652f78b9b90 2024-09-13T18:33:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.855+0000 7f45bbdbf740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652f78c7f90 con 0x5652f78a6710 2024-09-13T18:33:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.855+0000 7f45b1d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5652f78a78e0 0x5652f78baf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.855+0000 7f45b1d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5652f78a78e0 0x5652f78baf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59364/0 (socket says 172.21.15.114:59364) 2024-09-13T18:33:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.855+0000 7f45b1d9b640 1 -- 172.21.15.114:0/2306113284 learned_addr learned my addr 172.21.15.114:0/2306113284 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.855+0000 7f45b259c640 1 --2- 172.21.15.114:0/2306113284 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652f78a6710 0x5652f78a2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.855+0000 7f45b2d9d640 1 --2- 172.21.15.114:0/2306113284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652f78b9b90 0x5652f78ac3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.855+0000 7f45b1d9b640 1 -- 172.21.15.114:0/2306113284 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652f78a6710 msgr2=0x5652f78a2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.855+0000 7f45b1d9b640 1 --2- 172.21.15.114:0/2306113284 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652f78a6710 0x5652f78a2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.856+0000 7f45b1d9b640 1 -- 172.21.15.114:0/2306113284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652f78b9b90 msgr2=0x5652f78ac3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.856+0000 7f45b1d9b640 1 --2- 172.21.15.114:0/2306113284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652f78b9b90 0x5652f78ac3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.856+0000 7f45b1d9b640 1 -- 172.21.15.114:0/2306113284 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652f77ab300 con 0x5652f78a78e0 2024-09-13T18:33:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.856+0000 7f45b1d9b640 1 --2- 172.21.15.114:0/2306113284 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5652f78a78e0 0x5652f78baf80 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f45a8016040 tx=0x7f45a8009a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ec040032456eb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.856+0000 7f45a17fa640 1 -- 172.21.15.114:0/2306113284 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a8038070 con 0x5652f78a78e0 2024-09-13T18:33:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.856+0000 7f45a17fa640 1 -- 172.21.15.114:0/2306113284 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f45a8023070 con 0x5652f78a78e0 2024-09-13T18:33:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.856+0000 7f45a17fa640 1 -- 172.21.15.114:0/2306113284 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a8032470 con 0x5652f78a78e0 2024-09-13T18:33:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.857+0000 7f45bbdbf740 1 -- 172.21.15.114:0/2306113284 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5652f78a78e0 msgr2=0x5652f78baf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.857+0000 7f45bbdbf740 1 --2- 172.21.15.114:0/2306113284 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5652f78a78e0 0x5652f78baf80 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f45a8016040 tx=0x7f45a8009a30 comp rx=0 tx=0).stop 2024-09-13T18:33:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.857+0000 7f45bbdbf740 1 -- 172.21.15.114:0/2306113284 shutdown_connections 2024-09-13T18:33:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.857+0000 7f45bbdbf740 1 --2- 172.21.15.114:0/2306113284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652f78b9b90 0x5652f78ac3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:18.861 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.857+0000 7f45bbdbf740 1 --2- 172.21.15.114:0/2306113284 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5652f78a78e0 0x5652f78baf80 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:18.861 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.857+0000 7f45bbdbf740 1 --2- 172.21.15.114:0/2306113284 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652f78a6710 0x5652f78a2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:18.861 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.857+0000 7f45bbdbf740 1 -- 172.21.15.114:0/2306113284 >> 172.21.15.114:0/2306113284 conn(0x5652f78a28c0 msgr2=0x5652f78b5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:18.861 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.858+0000 7f45bbdbf740 1 -- 172.21.15.114:0/2306113284 shutdown_connections 2024-09-13T18:33:18.861 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.858+0000 7f45bbdbf740 1 -- 172.21.15.114:0/2306113284 wait complete. 2024-09-13T18:33:18.862 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.858+0000 7f45bbdbf740 1 -- start start 2024-09-13T18:33:18.862 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.859+0000 7f45bbdbf740 1 Processor -- start 2024-09-13T18:33:18.862 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.859+0000 7f45bbdbf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652f78a6710 0x5652f78c0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:18.862 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.859+0000 7f45bbdbf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652f78a78e0 0x5652f78b7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:18.862 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.859+0000 7f45bbdbf740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5652f78b9b90 0x5652f78b7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:18.862 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.859+0000 7f45bbdbf740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652f78b8120 con 0x5652f78b9b90 2024-09-13T18:33:18.862 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.859+0000 7f45bbdbf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652f791fca0 con 0x5652f78a6710 2024-09-13T18:33:18.863 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.859+0000 7f45bbdbf740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652f791fde0 con 0x5652f78a78e0 2024-09-13T18:33:18.863 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.859+0000 7f45bbdbf740 10 client.0 _ll_register_callbacks cb 0x7f45ac009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:33:18.863 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.859+0000 7f45b259c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652f78a6710 0x5652f78c0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:18.863 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.859+0000 7f45b259c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652f78a6710 0x5652f78c0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:38658/0 (socket says 172.21.15.114:38658) 2024-09-13T18:33:18.863 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.859+0000 7f45b259c640 1 -- 172.21.15.114:0/3095456441 learned_addr learned my addr 172.21.15.114:0/3095456441 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:18.863 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.859+0000 7f45b1d9b640 1 --2- 172.21.15.114:0/3095456441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652f78a78e0 0x5652f78b7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:18.863 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.859+0000 7f45b2d9d640 1 --2- 172.21.15.114:0/3095456441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5652f78b9b90 0x5652f78b7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:18.864 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.860+0000 7f45b259c640 1 -- 172.21.15.114:0/3095456441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652f78a78e0 msgr2=0x5652f78b7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:18.864 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.860+0000 7f45b259c640 1 --2- 172.21.15.114:0/3095456441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652f78a78e0 0x5652f78b7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.860+0000 7f45b259c640 1 -- 172.21.15.114:0/3095456441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5652f78b9b90 msgr2=0x5652f78b7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.860+0000 7f45b259c640 1 --2- 172.21.15.114:0/3095456441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5652f78b9b90 0x5652f78b7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.860+0000 7f45b259c640 1 -- 172.21.15.114:0/3095456441 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5652f7809330 con 0x5652f78a6710 2024-09-13T18:33:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.860+0000 7f45b259c640 1 --2- 172.21.15.114:0/3095456441 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652f78a6710 0x5652f78c0f80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f45a4020000 tx=0x7f45a4005c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8aeacca1ce982538 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.860+0000 7f4599ffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:33:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.860+0000 7f4599ffb640 1 -- 172.21.15.114:0/3095456441 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a40997b0 con 0x5652f78a6710 2024-09-13T18:33:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.860+0000 7f4599ffb640 1 -- 172.21.15.114:0/3095456441 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f45a409d5e0 con 0x5652f78a6710 2024-09-13T18:33:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.860+0000 7f45bbdbf740 1 -- 172.21.15.114:0/3095456441 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f45ac040070 con 0x5652f78a6710 2024-09-13T18:33:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.861+0000 7f4599ffb640 1 -- 172.21.15.114:0/3095456441 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45a40a1c30 con 0x5652f78a6710 2024-09-13T18:33:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.861+0000 7f45bbdbf740 10 client.27947 fetch_fsmap learned FSMap version 36 2024-09-13T18:33:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.861+0000 7f45bbdbf740 1 -- 172.21.15.114:0/3095456441 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5652f77d08e0 con 0x5652f78a6710 2024-09-13T18:33:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.861+0000 7f4599ffb640 1 -- 172.21.15.114:0/3095456441 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f45ac040070 con 0x5652f78a6710 2024-09-13T18:33:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.861+0000 7f4599ffb640 1 -- 172.21.15.114:0/3095456441 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 36) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f45a40a17a0 con 0x5652f78a6710 2024-09-13T18:33:18.866 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.861+0000 7f45bbdbf740 10 client.27947 fetch_fsmap finished waiting for FSMap version 36 2024-09-13T18:33:18.866 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.861+0000 7f45bbdbf740 10 client.27947 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:33:18.866 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.861+0000 7f45bbdbf740 20 client.27947 populate_metadata read hostname 'smithi114' 2024-09-13T18:33:18.866 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.861+0000 7f45bbdbf740 1 --2- 172.21.15.114:0/3095456441 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5652f78c9780 0x5652f79235b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:18.866 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.862+0000 7f45bbdbf740 4 client.27947 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T18:33:18.866 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.862+0000 7f45bbdbf740 1 -- 172.21.15.114:0/3095456441 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5652f74fe300 con 0x5652f78c9780 2024-09-13T18:33:18.866 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.862+0000 7f45b1d9b640 1 --2- 172.21.15.114:0/3095456441 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5652f78c9780 0x5652f79235b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:18.866 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.862+0000 7f4599ffb640 10 client.27947 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:33:18.866 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.863+0000 7f45b1d9b640 1 --2- 172.21.15.114:0/3095456441 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5652f78c9780 0x5652f79235b0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f6e2f46780c265d server_cookie=e5afb2584d10532c in_seq=0 out_seq=0 2024-09-13T18:33:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.869+0000 7f4599ffb640 1 -- 172.21.15.114:0/3095456441 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5652f74fe300 con 0x5652f78c9780 2024-09-13T18:33:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.869+0000 7f4599ffb640 10 client.27947 handle_command_reply: tid=0 2024-09-13T18:33:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.869+0000 7f45bbdbf740 1 client.27947 shutdown 2024-09-13T18:33:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.870+0000 7f45bbdbf740 1 -- 172.21.15.114:0/3095456441 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652f78a6710 msgr2=0x5652f78c0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.870+0000 7f45bbdbf740 1 --2- 172.21.15.114:0/3095456441 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652f78a6710 0x5652f78c0f80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f45a4020000 tx=0x7f45a4005c10 comp rx=0 tx=0).stop 2024-09-13T18:33:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.870+0000 7f45bbdbf740 1 -- 172.21.15.114:0/3095456441 shutdown_connections 2024-09-13T18:33:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.870+0000 7f45bbdbf740 1 --2- 172.21.15.114:0/3095456441 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x5652f78c9780 0x5652f79235b0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.870+0000 7f4599ffb640 0 client.27947 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:33:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.870+0000 7f45bbdbf740 1 --2- 172.21.15.114:0/3095456441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5652f78b9b90 0x5652f78b7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.870+0000 7f45bbdbf740 1 --2- 172.21.15.114:0/3095456441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652f78a78e0 0x5652f78b7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.870+0000 7f45bbdbf740 1 --2- 172.21.15.114:0/3095456441 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5652f78a6710 0x5652f78c0f80 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.870+0000 7f45bbdbf740 1 -- 172.21.15.114:0/3095456441 >> 172.21.15.114:0/3095456441 conn(0x5652f78a28c0 msgr2=0x5652f78b8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.871+0000 7f45bbdbf740 1 -- 172.21.15.114:0/3095456441 shutdown_connections 2024-09-13T18:33:18.874 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.871+0000 7f45bbdbf740 1 -- 172.21.15.114:0/3095456441 wait complete. 2024-09-13T18:33:18.874 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.871+0000 7f45bbdbf740 20 client.27947 trim_cache size 0 max 16384 2024-09-13T18:33:18.874 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-13T18:33:18.874 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-13T18:33:18.874 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-13T18:33:18.874 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-13T18:33:18.875 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.872+0000 7f45b8fc3640 1 -- 172.21.15.114:0/2075299352 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f459c0a3410 msgr2=0x7f459c0a58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:18.875 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.872+0000 7f45b8fc3640 1 --2- 172.21.15.114:0/2075299352 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f459c0a3410 0x7f459c0a58d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f45b4194fc0 tx=0x7f45a8005f90 comp rx=0 tx=0).stop 2024-09-13T18:33:18.875 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.872+0000 7f45b8fc3640 1 -- 172.21.15.114:0/2075299352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45b40fee30 msgr2=0x7f45b4199a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:18.875 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.872+0000 7f45b8fc3640 1 --2- 172.21.15.114:0/2075299352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45b40fee30 0x7f45b4199a70 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f45a400d720 tx=0x7f45a400dbe0 comp rx=0 tx=0).stop 2024-09-13T18:33:18.875 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.872+0000 7f45b8fc3640 1 -- 172.21.15.114:0/2075299352 shutdown_connections 2024-09-13T18:33:18.875 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.872+0000 7f45b8fc3640 1 --2- 172.21.15.114:0/2075299352 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f459c0a3410 0x7f459c0a58d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:18.875 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.872+0000 7f45b8fc3640 1 --2- 172.21.15.114:0/2075299352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45b4193ba0 0x7f45b4194020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:18.875 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.872+0000 7f45b8fc3640 1 --2- 172.21.15.114:0/2075299352 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45b4103f80 0x7f45b4199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:18.875 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.872+0000 7f45b8fc3640 1 --2- 172.21.15.114:0/2075299352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45b40fee30 0x7f45b4199a70 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:18.875 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.872+0000 7f45b8fc3640 1 -- 172.21.15.114:0/2075299352 >> 172.21.15.114:0/2075299352 conn(0x7f45b40fa8c0 msgr2=0x7f45b40fb380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:18.876 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.872+0000 7f45b8fc3640 1 -- 172.21.15.114:0/2075299352 shutdown_connections 2024-09-13T18:33:18.876 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:18.872+0000 7f45b8fc3640 1 -- 172.21.15.114:0/2075299352 wait complete. 2024-09-13T18:33:20.622 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:20 smithi134 ceph-mon[34283]: pgmap v337: 129 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 118 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 18 MiB/s wr, 399 op/s 2024-09-13T18:33:20.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:20 smithi204 ceph-mon[34777]: pgmap v337: 129 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 118 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 18 MiB/s wr, 399 op/s 2024-09-13T18:33:20.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:20 smithi114 ceph-mon[30897]: pgmap v337: 129 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 118 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 18 MiB/s wr, 399 op/s 2024-09-13T18:33:22.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:22 smithi134 ceph-mon[34283]: pgmap v338: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 24 MiB/s wr, 532 op/s 2024-09-13T18:33:22.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:22 smithi204 ceph-mon[34777]: pgmap v338: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 24 MiB/s wr, 532 op/s 2024-09-13T18:33:22.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:22 smithi114 ceph-mon[30897]: pgmap v338: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 24 MiB/s wr, 532 op/s 2024-09-13T18:33:24.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:24 smithi134 ceph-mon[34283]: pgmap v339: 129 pgs: 129 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 21 MiB/s wr, 458 op/s 2024-09-13T18:33:24.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:24 smithi134 ceph-mon[34283]: osdmap e104: 12 total, 12 up, 12 in 2024-09-13T18:33:24.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:24 smithi204 ceph-mon[34777]: pgmap v339: 129 pgs: 129 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 21 MiB/s wr, 458 op/s 2024-09-13T18:33:24.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:24 smithi204 ceph-mon[34777]: osdmap e104: 12 total, 12 up, 12 in 2024-09-13T18:33:24.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:24 smithi114 ceph-mon[30897]: pgmap v339: 129 pgs: 129 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 21 MiB/s wr, 458 op/s 2024-09-13T18:33:24.722 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:24 smithi114 ceph-mon[30897]: osdmap e104: 12 total, 12 up, 12 in 2024-09-13T18:33:26.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:26 smithi134 ceph-mon[34283]: pgmap v341: 129 pgs: 129 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 23 MiB/s wr, 511 op/s 2024-09-13T18:33:26.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:26 smithi204 ceph-mon[34777]: pgmap v341: 129 pgs: 129 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 23 MiB/s wr, 511 op/s 2024-09-13T18:33:26.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:26 smithi114 ceph-mon[30897]: pgmap v341: 129 pgs: 129 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 23 MiB/s wr, 511 op/s 2024-09-13T18:33:28.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:28 smithi134 ceph-mon[34283]: pgmap v342: 129 pgs: 129 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 27 MiB/s wr, 587 op/s 2024-09-13T18:33:28.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:28 smithi204 ceph-mon[34777]: pgmap v342: 129 pgs: 129 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 27 MiB/s wr, 587 op/s 2024-09-13T18:33:28.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:28 smithi114 ceph-mon[30897]: pgmap v342: 129 pgs: 129 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 27 MiB/s wr, 587 op/s 2024-09-13T18:33:30.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:30 smithi134 ceph-mon[34283]: pgmap v343: 129 pgs: 129 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 27 MiB/s wr, 599 op/s 2024-09-13T18:33:30.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:30 smithi134 ceph-mon[34283]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-13T18:33:30.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:30 smithi204 ceph-mon[34777]: pgmap v343: 129 pgs: 129 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 27 MiB/s wr, 599 op/s 2024-09-13T18:33:30.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:30 smithi204 ceph-mon[34777]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-13T18:33:30.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:30 smithi114 ceph-mon[30897]: pgmap v343: 129 pgs: 129 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 27 MiB/s wr, 599 op/s 2024-09-13T18:33:30.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:30 smithi114 ceph-mon[30897]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-13T18:33:32.073 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.068+0000 7f4289d3e640 1 -- 172.21.15.114:0/3777101878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4284119400 msgr2=0x7f428411b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:32.073 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.068+0000 7f4289d3e640 1 --2- 172.21.15.114:0/3777101878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4284119400 0x7f428411b7f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f427c01ace0 tx=0x7f427c03b990 comp rx=0 tx=0).stop 2024-09-13T18:33:32.073 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.069+0000 7f4289d3e640 1 -- 172.21.15.114:0/3777101878 shutdown_connections 2024-09-13T18:33:32.073 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.069+0000 7f4289d3e640 1 --2- 172.21.15.114:0/3777101878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4284119400 0x7f428411b7f0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.069+0000 7f4289d3e640 1 --2- 172.21.15.114:0/3777101878 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4284107890 0x7f4284107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.069+0000 7f4289d3e640 1 --2- 172.21.15.114:0/3777101878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42841074b0 0x7f428410e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.069+0000 7f4289d3e640 1 -- 172.21.15.114:0/3777101878 >> 172.21.15.114:0/3777101878 conn(0x7f428406ae30 msgr2=0x7f428406b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:32.077 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.074+0000 7f4289d3e640 1 -- 172.21.15.114:0/3777101878 shutdown_connections 2024-09-13T18:33:32.078 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.076+0000 7f4289d3e640 1 -- 172.21.15.114:0/3777101878 wait complete. 2024-09-13T18:33:32.079 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.077+0000 7f4289d3e640 1 Processor -- start 2024-09-13T18:33:32.083 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.077+0000 7f4289d3e640 1 -- start start 2024-09-13T18:33:32.084 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.078+0000 7f4289d3e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42841074b0 0x7f4284114c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:32.084 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.078+0000 7f4289d3e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4284107890 0x7f4284115170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:32.084 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.078+0000 7f4289d3e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4284119400 0x7f42841c2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:32.084 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.078+0000 7f4289d3e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42841c3610 con 0x7f4284119400 2024-09-13T18:33:32.084 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.078+0000 7f4289d3e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42841c3780 con 0x7f42841074b0 2024-09-13T18:33:32.084 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.078+0000 7f4289d3e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42841c3940 con 0x7f4284107890 2024-09-13T18:33:32.084 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.078+0000 7f4282ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4284107890 0x7f4284115170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.078+0000 7f4283fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4284119400 0x7f42841c2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.078+0000 7f4283fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4284119400 0x7f42841c2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35930/0 (socket says 172.21.15.114:35930) 2024-09-13T18:33:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.078+0000 7f4283fff640 1 -- 172.21.15.114:0/4200527746 learned_addr learned my addr 172.21.15.114:0/4200527746 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.078+0000 7f42837fe640 1 --2- 172.21.15.114:0/4200527746 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42841074b0 0x7f4284114c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.079+0000 7f4283fff640 1 -- 172.21.15.114:0/4200527746 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4284107890 msgr2=0x7f4284115170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.079+0000 7f4283fff640 1 --2- 172.21.15.114:0/4200527746 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4284107890 0x7f4284115170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.079+0000 7f4283fff640 1 -- 172.21.15.114:0/4200527746 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42841074b0 msgr2=0x7f4284114c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.079+0000 7f4283fff640 1 --2- 172.21.15.114:0/4200527746 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42841074b0 0x7f4284114c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.079+0000 7f4283fff640 1 -- 172.21.15.114:0/4200527746 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f427c01a910 con 0x7f4284119400 2024-09-13T18:33:32.086 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.080+0000 7f4283fff640 1 --2- 172.21.15.114:0/4200527746 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4284119400 0x7f42841c2fa0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f427c003ba0 tx=0x7f427c003bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:32.086 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T18:33:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.084+0000 7f4280ff9640 1 -- 172.21.15.114:0/4200527746 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427c0406a0 con 0x7f4284119400 2024-09-13T18:33:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.084+0000 7f4280ff9640 1 -- 172.21.15.114:0/4200527746 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f427c040c90 con 0x7f4284119400 2024-09-13T18:33:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.084+0000 7f4280ff9640 1 -- 172.21.15.114:0/4200527746 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427c0406a0 con 0x7f4284119400 2024-09-13T18:33:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.087+0000 7f4289d3e640 1 -- 172.21.15.114:0/4200527746 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42841c3bc0 con 0x7f4284119400 2024-09-13T18:33:32.092 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.087+0000 7f4289d3e640 1 -- 172.21.15.114:0/4200527746 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42841cb3c0 con 0x7f4284119400 2024-09-13T18:33:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.090+0000 7f428ca5e740 1 Processor -- start 2024-09-13T18:33:32.103 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.095+0000 7f4280ff9640 1 -- 172.21.15.114:0/4200527746 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f427c03f050 con 0x7f4284119400 2024-09-13T18:33:32.103 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.096+0000 7f428ca5e740 1 -- start start 2024-09-13T18:33:32.103 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.096+0000 7f428ca5e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563d42c80500 0x563d42c808e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:32.103 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.096+0000 7f4280ff9640 1 --2- 172.21.15.114:0/4200527746 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f426c0a13f0 0x7f426c0a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:32.103 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.097+0000 7f4280ff9640 1 -- 172.21.15.114:0/4200527746 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f427c0edcc0 con 0x7f4284119400 2024-09-13T18:33:32.103 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.097+0000 7f428ca5e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d42c80e20 0x563d42c8b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:32.103 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.097+0000 7f428ca5e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d42c8b6c0 0x563d42c8dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:32.103 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.097+0000 7f428ca5e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d42bebc30 con 0x563d42c80500 2024-09-13T18:33:32.103 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.097+0000 7f428ca5e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d42be93c0 con 0x563d42c80e20 2024-09-13T18:33:32.103 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.097+0000 7f428ca5e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d42be4cb0 con 0x563d42c8b6c0 2024-09-13T18:33:32.103 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.097+0000 7f4283fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d42c8b6c0 0x563d42c8dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.097+0000 7f4283fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d42c8b6c0 0x563d42c8dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:49698/0 (socket says 172.21.15.114:49698) 2024-09-13T18:33:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.097+0000 7f4283fff640 1 -- 172.21.15.114:0/497759844 learned_addr learned my addr 172.21.15.114:0/497759844 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.098+0000 7f4283fff640 1 -- 172.21.15.114:0/497759844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d42c80e20 msgr2=0x563d42c8b180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:33:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.098+0000 7f4283fff640 1 --2- 172.21.15.114:0/497759844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d42c80e20 0x563d42c8b180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.098+0000 7f4283fff640 1 -- 172.21.15.114:0/497759844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563d42c80500 msgr2=0x563d42c808e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.098+0000 7f4283fff640 1 --2- 172.21.15.114:0/497759844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563d42c80500 0x563d42c808e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.098+0000 7f4283fff640 1 -- 172.21.15.114:0/497759844 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d42b8b910 con 0x563d42c8b6c0 2024-09-13T18:33:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.098+0000 7f4283fff640 1 --2- 172.21.15.114:0/497759844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d42c8b6c0 0x563d42c8dab0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f427c0c2920 tx=0x7f427c0ba960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8fad1b611d42faa server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.098+0000 7f4269ffb640 1 -- 172.21.15.114:0/497759844 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427c0406a0 con 0x563d42c8b6c0 2024-09-13T18:33:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.098+0000 7f4282ffd640 1 --2- 172.21.15.114:0/4200527746 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f426c0a13f0 0x7f426c0a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.099+0000 7f4269ffb640 1 -- 172.21.15.114:0/497759844 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f427c0c2e50 con 0x563d42c8b6c0 2024-09-13T18:33:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.099+0000 7f4269ffb640 1 -- 172.21.15.114:0/497759844 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427c0eb290 con 0x563d42c8b6c0 2024-09-13T18:33:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.099+0000 7f428ca5e740 1 -- 172.21.15.114:0/497759844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d42c8b6c0 msgr2=0x563d42c8dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.099+0000 7f428ca5e740 1 --2- 172.21.15.114:0/497759844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d42c8b6c0 0x563d42c8dab0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f427c0c2920 tx=0x7f427c0ba960 comp rx=0 tx=0).stop 2024-09-13T18:33:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.099+0000 7f4282ffd640 1 --2- 172.21.15.114:0/4200527746 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f426c0a13f0 0x7f426c0a38b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4278006fd0 tx=0x7f4278006d60 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.100+0000 7f428ca5e740 1 -- 172.21.15.114:0/497759844 shutdown_connections 2024-09-13T18:33:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.100+0000 7f428ca5e740 1 --2- 172.21.15.114:0/497759844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d42c8b6c0 0x563d42c8dab0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.100+0000 7f428ca5e740 1 --2- 172.21.15.114:0/497759844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d42c80e20 0x563d42c8b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.100+0000 7f428ca5e740 1 --2- 172.21.15.114:0/497759844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563d42c80500 0x563d42c808e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.100+0000 7f428ca5e740 1 -- 172.21.15.114:0/497759844 >> 172.21.15.114:0/497759844 conn(0x563d42c7ca90 msgr2=0x563d42c7ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.100+0000 7f428ca5e740 1 -- 172.21.15.114:0/497759844 shutdown_connections 2024-09-13T18:33:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.100+0000 7f428ca5e740 1 -- 172.21.15.114:0/497759844 wait complete. 2024-09-13T18:33:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.100+0000 7f428ca5e740 1 -- start start 2024-09-13T18:33:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.100+0000 7f428ca5e740 1 Processor -- start 2024-09-13T18:33:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.101+0000 7f428ca5e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d42c80500 0x563d42c8af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.101+0000 7f428ca5e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563d42c80e20 0x563d42c88430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.101+0000 7f428ca5e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d42c8b6c0 0x563d42c88970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.101+0000 7f428ca5e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d42ba2e80 con 0x563d42c80e20 2024-09-13T18:33:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.101+0000 7f428ca5e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d42ba2b20 con 0x563d42c8b6c0 2024-09-13T18:33:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.101+0000 7f428ca5e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d42b68260 con 0x563d42c80500 2024-09-13T18:33:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.101+0000 7f428ca5e740 10 client.0 _ll_register_callbacks cb 0x563d42b9ddf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:33:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.102+0000 7f4282ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563d42c80e20 0x563d42c88430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.102+0000 7f4283fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d42c8b6c0 0x563d42c88970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.102+0000 7f42837fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d42c80500 0x563d42c8af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.102+0000 7f42837fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d42c80500 0x563d42c8af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:49714/0 (socket says 172.21.15.114:49714) 2024-09-13T18:33:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.102+0000 7f4282ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563d42c80e20 0x563d42c88430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35948/0 (socket says 172.21.15.114:35948) 2024-09-13T18:33:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.102+0000 7f42837fe640 1 -- 172.21.15.114:0/3272445090 learned_addr learned my addr 172.21.15.114:0/3272445090 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.103+0000 7f4282ffd640 1 -- 172.21.15.114:0/3272445090 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d42c80500 msgr2=0x563d42c8af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:32.108 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.103+0000 7f4282ffd640 1 --2- 172.21.15.114:0/3272445090 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d42c80500 0x563d42c8af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.108 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.103+0000 7f4282ffd640 1 -- 172.21.15.114:0/3272445090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d42c8b6c0 msgr2=0x563d42c88970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:32.108 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.103+0000 7f4282ffd640 1 --2- 172.21.15.114:0/3272445090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d42c8b6c0 0x563d42c88970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.108 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.103+0000 7f4282ffd640 1 -- 172.21.15.114:0/3272445090 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42841cb3c0 con 0x563d42c80e20 2024-09-13T18:33:32.109 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.104+0000 7f4282ffd640 1 --2- 172.21.15.114:0/3272445090 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563d42c80e20 0x563d42c88430 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f4278006f40 tx=0x7f4278006ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be895a8d5d0d6609 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:32.109 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.104+0000 7f4262ffd640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:33:32.109 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.105+0000 7f4262ffd640 1 -- 172.21.15.114:0/3272445090 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4278012b90 con 0x563d42c80e20 2024-09-13T18:33:32.109 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.105+0000 7f4262ffd640 1 -- 172.21.15.114:0/3272445090 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f4278012cf0 con 0x563d42c80e20 2024-09-13T18:33:32.110 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.105+0000 7f428ca5e740 1 -- 172.21.15.114:0/3272445090 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563d42c0d680 con 0x563d42c80e20 2024-09-13T18:33:32.110 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.105+0000 7f4262ffd640 1 -- 172.21.15.114:0/3272445090 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f427801d440 con 0x563d42c80e20 2024-09-13T18:33:32.110 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.105+0000 7f428ca5e740 10 client.18489 fetch_fsmap learned FSMap version 37 2024-09-13T18:33:32.110 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.105+0000 7f428ca5e740 1 -- 172.21.15.114:0/3272445090 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x563d42be3330 con 0x563d42c80e20 2024-09-13T18:33:32.110 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.106+0000 7f4262ffd640 1 -- 172.21.15.114:0/3272445090 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563d42c0d680 con 0x563d42c80e20 2024-09-13T18:33:32.110 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.106+0000 7f4262ffd640 1 -- 172.21.15.114:0/3272445090 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 37) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f42780254b0 con 0x563d42c80e20 2024-09-13T18:33:32.110 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.106+0000 7f428ca5e740 10 client.18489 fetch_fsmap finished waiting for FSMap version 37 2024-09-13T18:33:32.110 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.106+0000 7f428ca5e740 10 client.18489 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:33:32.110 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.106+0000 7f428ca5e740 20 client.18489 populate_metadata read hostname 'smithi114' 2024-09-13T18:33:32.110 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.106+0000 7f428ca5e740 1 --2- 172.21.15.114:0/3272445090 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x563d42c9ad70 0x563d42c9d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:32.110 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.106+0000 7f428ca5e740 4 client.18489 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:33:32.110 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.106+0000 7f428ca5e740 1 -- 172.21.15.114:0/3272445090 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563d42bd85d0 con 0x563d42c9ad70 2024-09-13T18:33:32.110 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.106+0000 7f42837fe640 1 --2- 172.21.15.114:0/3272445090 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x563d42c9ad70 0x563d42c9d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.107+0000 7f42837fe640 1 --2- 172.21.15.114:0/3272445090 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x563d42c9ad70 0x563d42c9d1b0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=887be9e0dc81d3ef server_cookie=7c28e05fcea3c35a in_seq=0 out_seq=0 2024-09-13T18:33:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.107+0000 7f4262ffd640 10 client.18489 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:33:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.109+0000 7f4262ffd640 1 -- 172.21.15.114:0/3272445090 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x563d42bd85d0 con 0x563d42c9ad70 2024-09-13T18:33:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.109+0000 7f4262ffd640 10 client.18489 handle_command_reply: tid=0 2024-09-13T18:33:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.109+0000 7f428ca5e740 1 client.18489 shutdown 2024-09-13T18:33:32.114 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.109+0000 7f428ca5e740 1 -- 172.21.15.114:0/3272445090 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563d42c80e20 msgr2=0x563d42c88430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:32.114 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.109+0000 7f428ca5e740 1 --2- 172.21.15.114:0/3272445090 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563d42c80e20 0x563d42c88430 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f4278006f40 tx=0x7f4278006ad0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.116 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.113+0000 7f428ca5e740 1 -- 172.21.15.114:0/3272445090 shutdown_connections 2024-09-13T18:33:32.116 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.113+0000 7f428ca5e740 1 --2- 172.21.15.114:0/3272445090 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x563d42c9ad70 0x563d42c9d1b0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.116 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.113+0000 7f428ca5e740 1 --2- 172.21.15.114:0/3272445090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d42c8b6c0 0x563d42c88970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.116 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.113+0000 7f428ca5e740 1 --2- 172.21.15.114:0/3272445090 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563d42c80e20 0x563d42c88430 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.116 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.113+0000 7f428ca5e740 1 --2- 172.21.15.114:0/3272445090 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d42c80500 0x563d42c8af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.116 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.113+0000 7f428ca5e740 1 -- 172.21.15.114:0/3272445090 >> 172.21.15.114:0/3272445090 conn(0x563d42c7ca90 msgr2=0x563d42c7bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:32.116 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.113+0000 7f4262ffd640 0 client.18489 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:33:32.116 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.114+0000 7f428ca5e740 1 -- 172.21.15.114:0/3272445090 shutdown_connections 2024-09-13T18:33:32.116 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.114+0000 7f428ca5e740 1 -- 172.21.15.114:0/3272445090 wait complete. 2024-09-13T18:33:32.128 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.119+0000 7f428ca5e740 20 client.18489 trim_cache size 0 max 16384 2024-09-13T18:33:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.147+0000 7f428ca5e740 1 Processor -- start 2024-09-13T18:33:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.147+0000 7f428ca5e740 1 -- start start 2024-09-13T18:33:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.147+0000 7f428ca5e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d42c80500 0x563d42c7c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.147+0000 7f428ca5e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563d42c8b6c0 0x563d42c83720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.147+0000 7f428ca5e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d42c83c60 0x563d42c7ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.147+0000 7f428ca5e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d42bd85d0 con 0x563d42c8b6c0 2024-09-13T18:33:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.147+0000 7f428ca5e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d42c0d680 con 0x563d42c80500 2024-09-13T18:33:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.147+0000 7f428ca5e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d42c99620 con 0x563d42c83c60 2024-09-13T18:33:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.147+0000 7f42837fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d42c80500 0x563d42c7c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.148+0000 7f42837fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d42c80500 0x563d42c7c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:42306/0 (socket says 172.21.15.114:42306) 2024-09-13T18:33:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.148+0000 7f42837fe640 1 -- 172.21.15.114:0/131796604 learned_addr learned my addr 172.21.15.114:0/131796604 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.148+0000 7f4282ffd640 1 --2- 172.21.15.114:0/131796604 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563d42c8b6c0 0x563d42c83720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:32.151 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.148+0000 7f42837fe640 1 -- 172.21.15.114:0/131796604 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d42c83c60 msgr2=0x563d42c7ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:32.152 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.148+0000 7f42837fe640 1 --2- 172.21.15.114:0/131796604 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d42c83c60 0x563d42c7ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.152 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.148+0000 7f42837fe640 1 -- 172.21.15.114:0/131796604 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563d42c8b6c0 msgr2=0x563d42c83720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:32.152 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.148+0000 7f42837fe640 1 --2- 172.21.15.114:0/131796604 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563d42c8b6c0 0x563d42c83720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.152 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.148+0000 7f42837fe640 1 -- 172.21.15.114:0/131796604 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d42b85300 con 0x563d42c80500 2024-09-13T18:33:32.152 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.149+0000 7f42837fe640 1 --2- 172.21.15.114:0/131796604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d42c80500 0x563d42c7c550 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x563d42c9dad0 tx=0x7f427400ac80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f354c5feb7de779 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:32.152 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.149+0000 7f426a7fc640 1 -- 172.21.15.114:0/131796604 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4274019410 con 0x563d42c80500 2024-09-13T18:33:32.152 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.149+0000 7f426a7fc640 1 -- 172.21.15.114:0/131796604 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f427400c040 con 0x563d42c80500 2024-09-13T18:33:32.152 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.149+0000 7f426a7fc640 1 -- 172.21.15.114:0/131796604 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4274015d70 con 0x563d42c80500 2024-09-13T18:33:32.152 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.150+0000 7f428ca5e740 1 -- 172.21.15.114:0/131796604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d42c80500 msgr2=0x563d42c7c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:32.153 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.150+0000 7f428ca5e740 1 --2- 172.21.15.114:0/131796604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d42c80500 0x563d42c7c550 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x563d42c9dad0 tx=0x7f427400ac80 comp rx=0 tx=0).stop 2024-09-13T18:33:32.156 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.153+0000 7f428ca5e740 1 -- 172.21.15.114:0/131796604 shutdown_connections 2024-09-13T18:33:32.157 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.153+0000 7f428ca5e740 1 --2- 172.21.15.114:0/131796604 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d42c83c60 0x563d42c7ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.157 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.153+0000 7f428ca5e740 1 --2- 172.21.15.114:0/131796604 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563d42c8b6c0 0x563d42c83720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.157 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.153+0000 7f428ca5e740 1 --2- 172.21.15.114:0/131796604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d42c80500 0x563d42c7c550 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.157 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.153+0000 7f428ca5e740 1 -- 172.21.15.114:0/131796604 >> 172.21.15.114:0/131796604 conn(0x563d42c7ca90 msgr2=0x563d42c88430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:32.166 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.162+0000 7f428ca5e740 1 -- 172.21.15.114:0/131796604 shutdown_connections 2024-09-13T18:33:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.165+0000 7f428ca5e740 1 -- 172.21.15.114:0/131796604 wait complete. 2024-09-13T18:33:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.165+0000 7f428ca5e740 1 -- start start 2024-09-13T18:33:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.165+0000 7f428ca5e740 1 Processor -- start 2024-09-13T18:33:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.166+0000 7f428ca5e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d42c80500 0x563d42c8f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.166+0000 7f428ca5e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563d42c83c60 0x563d42c8f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.166+0000 7f428ca5e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d42c8b6c0 0x563d42ceb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.166+0000 7f428ca5e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d42ceb850 con 0x563d42c83c60 2024-09-13T18:33:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.166+0000 7f428ca5e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d42ceb9c0 con 0x563d42c8b6c0 2024-09-13T18:33:32.169 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.166+0000 7f428ca5e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d42cebb80 con 0x563d42c80500 2024-09-13T18:33:32.169 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.166+0000 7f428ca5e740 10 client.0 _ll_register_callbacks cb 0x7f427c0418f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:33:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.168+0000 7f42837fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d42c80500 0x563d42c8f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.168+0000 7f4283fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d42c8b6c0 0x563d42ceb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.168+0000 7f42837fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d42c80500 0x563d42c8f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:49736/0 (socket says 172.21.15.114:49736) 2024-09-13T18:33:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.168+0000 7f42837fe640 1 -- 172.21.15.114:0/4071627434 learned_addr learned my addr 172.21.15.114:0/4071627434 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.169+0000 7f4282ffd640 1 --2- 172.21.15.114:0/4071627434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563d42c83c60 0x563d42c8f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.169+0000 7f42837fe640 1 -- 172.21.15.114:0/4071627434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d42c8b6c0 msgr2=0x563d42ceb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.169+0000 7f42837fe640 1 --2- 172.21.15.114:0/4071627434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d42c8b6c0 0x563d42ceb310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.169+0000 7f42837fe640 1 -- 172.21.15.114:0/4071627434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563d42c83c60 msgr2=0x563d42c8f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.169+0000 7f42837fe640 1 --2- 172.21.15.114:0/4071627434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563d42c83c60 0x563d42c8f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.169+0000 7f42837fe640 1 -- 172.21.15.114:0/4071627434 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f427400a600 con 0x563d42c80500 2024-09-13T18:33:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.170+0000 7f4282ffd640 1 --2- 172.21.15.114:0/4071627434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563d42c83c60 0x563d42c8f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:33:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.170+0000 7f42837fe640 1 --2- 172.21.15.114:0/4071627434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d42c80500 0x563d42c8f390 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4274015a50 tx=0x7f4274015d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff627991b6b0b24d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.170+0000 7f4262ffd640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:33:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.174+0000 7f4262ffd640 1 -- 172.21.15.114:0/4071627434 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4274015610 con 0x563d42c80500 2024-09-13T18:33:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.175+0000 7f4262ffd640 1 -- 172.21.15.114:0/4071627434 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f4274015770 con 0x563d42c80500 2024-09-13T18:33:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.176+0000 7f428ca5e740 1 -- 172.21.15.114:0/4071627434 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f427c0c2e50 con 0x563d42c80500 2024-09-13T18:33:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.180+0000 7f4262ffd640 1 -- 172.21.15.114:0/4071627434 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4274005970 con 0x563d42c80500 2024-09-13T18:33:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.180+0000 7f428ca5e740 10 client.27883 fetch_fsmap learned FSMap version 37 2024-09-13T18:33:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.180+0000 7f428ca5e740 1 -- 172.21.15.114:0/4071627434 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x563d42baa8e0 con 0x563d42c80500 2024-09-13T18:33:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.184+0000 7f4262ffd640 1 -- 172.21.15.114:0/4071627434 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f427c0c2e50 con 0x563d42c80500 2024-09-13T18:33:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.185+0000 7f4262ffd640 1 -- 172.21.15.114:0/4071627434 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 37) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f4274011040 con 0x563d42c80500 2024-09-13T18:33:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.185+0000 7f428ca5e740 10 client.27883 fetch_fsmap finished waiting for FSMap version 37 2024-09-13T18:33:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.185+0000 7f428ca5e740 10 client.27883 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-13T18:33:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.185+0000 7f428ca5e740 20 client.27883 populate_metadata read hostname 'smithi114' 2024-09-13T18:33:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.185+0000 7f428ca5e740 1 --2- 172.21.15.114:0/4071627434 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x563d42cf33f0 0x563d42cf5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.185+0000 7f428ca5e740 4 client.27883 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-13T18:33:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.185+0000 7f428ca5e740 1 -- 172.21.15.114:0/4071627434 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563d428d8300 con 0x563d42cf33f0 2024-09-13T18:33:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.187+0000 7f4282ffd640 1 --2- 172.21.15.114:0/4071627434 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x563d42cf33f0 0x563d42cf5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.201+0000 7f4262ffd640 10 client.27883 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:33:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.201+0000 7f4282ffd640 1 --2- 172.21.15.114:0/4071627434 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x563d42cf33f0 0x563d42cf5810 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=458fe895bf9f09aa server_cookie=e9a2b3dc4b155361 in_seq=0 out_seq=0 2024-09-13T18:33:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.216+0000 7f4262ffd640 1 -- 172.21.15.114:0/4071627434 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563d428d8300 con 0x563d42cf33f0 2024-09-13T18:33:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.216+0000 7f4262ffd640 10 client.27883 handle_command_reply: tid=0 2024-09-13T18:33:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.216+0000 7f428ca5e740 1 client.27883 shutdown 2024-09-13T18:33:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.224+0000 7f428ca5e740 1 -- 172.21.15.114:0/4071627434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d42c80500 msgr2=0x563d42c8f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.224+0000 7f428ca5e740 1 --2- 172.21.15.114:0/4071627434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d42c80500 0x563d42c8f390 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4274015a50 tx=0x7f4274015d00 comp rx=0 tx=0).stop 2024-09-13T18:33:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.224+0000 7f428ca5e740 1 -- 172.21.15.114:0/4071627434 shutdown_connections 2024-09-13T18:33:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.224+0000 7f428ca5e740 1 --2- 172.21.15.114:0/4071627434 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x563d42cf33f0 0x563d42cf5810 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.224+0000 7f428ca5e740 1 --2- 172.21.15.114:0/4071627434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d42c8b6c0 0x563d42ceb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.224+0000 7f428ca5e740 1 --2- 172.21.15.114:0/4071627434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563d42c83c60 0x563d42c8f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.224+0000 7f428ca5e740 1 --2- 172.21.15.114:0/4071627434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d42c80500 0x563d42c8f390 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.224+0000 7f428ca5e740 1 -- 172.21.15.114:0/4071627434 >> 172.21.15.114:0/4071627434 conn(0x563d42c7ca90 msgr2=0x563d42c96bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.224+0000 7f4262ffd640 0 client.27883 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:33:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.224+0000 7f428ca5e740 1 -- 172.21.15.114:0/4071627434 shutdown_connections 2024-09-13T18:33:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.224+0000 7f428ca5e740 1 -- 172.21.15.114:0/4071627434 wait complete. 2024-09-13T18:33:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.226+0000 7f428ca5e740 20 client.27883 trim_cache size 0 max 16384 2024-09-13T18:33:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-13T18:33:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-13T18:33:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-13T18:33:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-13T18:33:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.237+0000 7f4289d3e640 1 -- 172.21.15.114:0/4200527746 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f426c0a13f0 msgr2=0x7f426c0a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.237+0000 7f4289d3e640 1 --2- 172.21.15.114:0/4200527746 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f426c0a13f0 0x7f426c0a38b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4278006fd0 tx=0x7f4278006d60 comp rx=0 tx=0).stop 2024-09-13T18:33:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.237+0000 7f4289d3e640 1 -- 172.21.15.114:0/4200527746 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4284119400 msgr2=0x7f42841c2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.237+0000 7f4289d3e640 1 --2- 172.21.15.114:0/4200527746 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4284119400 0x7f42841c2fa0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f427c003ba0 tx=0x7f427c003bd0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.238+0000 7f4289d3e640 1 -- 172.21.15.114:0/4200527746 shutdown_connections 2024-09-13T18:33:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.238+0000 7f4289d3e640 1 --2- 172.21.15.114:0/4200527746 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f426c0a13f0 0x7f426c0a38b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.238+0000 7f4289d3e640 1 --2- 172.21.15.114:0/4200527746 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4284119400 0x7f42841c2fa0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.238+0000 7f4289d3e640 1 --2- 172.21.15.114:0/4200527746 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4284107890 0x7f4284115170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.238+0000 7f4289d3e640 1 --2- 172.21.15.114:0/4200527746 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42841074b0 0x7f4284114c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.238+0000 7f4289d3e640 1 -- 172.21.15.114:0/4200527746 >> 172.21.15.114:0/4200527746 conn(0x7f428406ae30 msgr2=0x7f428406ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.240+0000 7f4289d3e640 1 -- 172.21.15.114:0/4200527746 shutdown_connections 2024-09-13T18:33:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-13T18:33:32.240+0000 7f4289d3e640 1 -- 172.21.15.114:0/4200527746 wait complete. 2024-09-13T18:33:32.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.359+0000 7f25a842b640 1 -- 172.21.15.114:0/787756686 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25a0107920 msgr2=0x7f25a0107d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:32.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.359+0000 7f25a842b640 1 --2- 172.21.15.114:0/787756686 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25a0107920 0x7f25a0107d60 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f2590009900 tx=0x7f259002a410 comp rx=0 tx=0).stop 2024-09-13T18:33:32.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.360+0000 7f25a842b640 1 -- 172.21.15.114:0/787756686 shutdown_connections 2024-09-13T18:33:32.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.360+0000 7f25a842b640 1 --2- 172.21.15.114:0/787756686 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25a0119400 0x7f25a011b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.360+0000 7f25a842b640 1 --2- 172.21.15.114:0/787756686 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25a0107920 0x7f25a0107d60 secure :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f2590009900 tx=0x7f259002a410 comp rx=0 tx=0).stop 2024-09-13T18:33:32.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.360+0000 7f25a842b640 1 --2- 172.21.15.114:0/787756686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25a0107540 0x7f25a010e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.360+0000 7f25a842b640 1 -- 172.21.15.114:0/787756686 >> 172.21.15.114:0/787756686 conn(0x7f25a006afe0 msgr2=0x7f25a006b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:32.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.365+0000 7f25a842b640 1 -- 172.21.15.114:0/787756686 shutdown_connections 2024-09-13T18:33:32.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.369+0000 7f25a842b640 1 -- 172.21.15.114:0/787756686 wait complete. 2024-09-13T18:33:32.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.369+0000 7f25a842b640 1 Processor -- start 2024-09-13T18:33:32.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.369+0000 7f25a842b640 1 -- start start 2024-09-13T18:33:32.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.370+0000 7f25a842b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25a0107540 0x7f25a0078520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:32.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.370+0000 7f25a842b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25a0119400 0x7f25a0078a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:32.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.370+0000 7f25a842b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25a0073680 0x7f25a0073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:32.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.370+0000 7f25a842b640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a00742c0 con 0x7f25a0073680 2024-09-13T18:33:32.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.370+0000 7f25a842b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a0074430 con 0x7f25a0119400 2024-09-13T18:33:32.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.370+0000 7f25a842b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a0080240 con 0x7f25a0107540 2024-09-13T18:33:32.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.371+0000 7f25a69c8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25a0073680 0x7f25a0073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:32.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.371+0000 7f25a69c8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25a0073680 0x7f25a0073b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36000/0 (socket says 172.21.15.114:36000) 2024-09-13T18:33:32.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.371+0000 7f25a69c8640 1 -- 172.21.15.114:0/4204444660 learned_addr learned my addr 172.21.15.114:0/4204444660 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:32.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.371+0000 7f25a69c8640 1 -- 172.21.15.114:0/4204444660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25a0107540 msgr2=0x7f25a0078520 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:33:32.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.371+0000 7f25a69c8640 1 --2- 172.21.15.114:0/4204444660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25a0107540 0x7f25a0078520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.371+0000 7f25a69c8640 1 -- 172.21.15.114:0/4204444660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25a0119400 msgr2=0x7f25a0078a60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:33:32.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.371+0000 7f25a69c8640 1 --2- 172.21.15.114:0/4204444660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25a0119400 0x7f25a0078a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.371+0000 7f25a69c8640 1 -- 172.21.15.114:0/4204444660 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2590009560 con 0x7f25a0073680 2024-09-13T18:33:32.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.377+0000 7f25a69c8640 1 --2- 172.21.15.114:0/4204444660 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25a0073680 0x7f25a0073b30 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f259800eaa0 tx=0x7f259800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:32.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.383+0000 7f25977fe640 1 -- 172.21.15.114:0/4204444660 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259800cdb0 con 0x7f25a0073680 2024-09-13T18:33:32.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.383+0000 7f25977fe640 1 -- 172.21.15.114:0/4204444660 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2598005020 con 0x7f25a0073680 2024-09-13T18:33:32.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.383+0000 7f25977fe640 1 -- 172.21.15.114:0/4204444660 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25980116d0 con 0x7f25a0073680 2024-09-13T18:33:32.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.384+0000 7f25a842b640 1 -- 172.21.15.114:0/4204444660 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25a0080520 con 0x7f25a0073680 2024-09-13T18:33:32.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.384+0000 7f25a842b640 1 -- 172.21.15.114:0/4204444660 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25a00809e0 con 0x7f25a0073680 2024-09-13T18:33:32.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.391+0000 7f25977fe640 1 -- 172.21.15.114:0/4204444660 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f2598005190 con 0x7f25a0073680 2024-09-13T18:33:32.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.392+0000 7f25977fe640 1 --2- 172.21.15.114:0/4204444660 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f258c0a1410 0x7f258c0a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:32.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.392+0000 7f25977fe640 1 -- 172.21.15.114:0/4204444660 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f25980cce70 con 0x7f25a0073680 2024-09-13T18:33:32.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.394+0000 7f25ab227740 1 Processor -- start 2024-09-13T18:33:32.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.395+0000 7f25ab227740 1 -- start start 2024-09-13T18:33:32.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.396+0000 7f25ab227740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c26534e10 0x555c265351f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:32.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.396+0000 7f25ab227740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c26535730 0x555c2653fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:32.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.396+0000 7f25ab227740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555c2653ffd0 0x555c265423c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:32.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.396+0000 7f25a61c7640 1 --2- 172.21.15.114:0/4204444660 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f258c0a1410 0x7f258c0a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:32.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.396+0000 7f25ab227740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c2649ff20 con 0x555c2653ffd0 2024-09-13T18:33:32.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.396+0000 7f25ab227740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c2649d6b0 con 0x555c26534e10 2024-09-13T18:33:32.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.396+0000 7f25ab227740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c26498fa0 con 0x555c26535730 2024-09-13T18:33:32.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.396+0000 7f25a61c7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c26534e10 0x555c265351f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:32.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.397+0000 7f25a61c7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c26534e10 0x555c265351f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:42340/0 (socket says 172.21.15.114:42340) 2024-09-13T18:33:32.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.397+0000 7f25a61c7640 1 -- 172.21.15.114:0/1465971642 learned_addr learned my addr 172.21.15.114:0/1465971642 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:32.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.399+0000 7f25a59c6640 1 --2- 172.21.15.114:0/1465971642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c26535730 0x555c2653fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:32.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.403+0000 7f25a69c8640 1 --2- 172.21.15.114:0/1465971642 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555c2653ffd0 0x555c265423c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:32.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.403+0000 7f25a61c7640 1 -- 172.21.15.114:0/1465971642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c26535730 msgr2=0x555c2653fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:32.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.403+0000 7f25a61c7640 1 --2- 172.21.15.114:0/1465971642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c26535730 0x555c2653fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.403+0000 7f25a61c7640 1 -- 172.21.15.114:0/1465971642 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555c2653ffd0 msgr2=0x555c265423c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:32.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.403+0000 7f25a61c7640 1 --2- 172.21.15.114:0/1465971642 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555c2653ffd0 0x555c265423c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.403+0000 7f25a61c7640 1 -- 172.21.15.114:0/1465971642 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c26453840 con 0x555c26534e10 2024-09-13T18:33:32.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.412+0000 7f25a61c7640 1 --2- 172.21.15.114:0/1465971642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c26534e10 0x555c265351f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f259c00bcb0 tx=0x7f259c019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aafaaccd32e596d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:32.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.412+0000 7f25a61c7640 1 --2- 172.21.15.114:0/4204444660 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f258c0a1410 0x7f258c0a38d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f259c005d00 tx=0x7f259c00b9e0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:32.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.413+0000 7f2594e05640 1 -- 172.21.15.114:0/1465971642 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259c023070 con 0x555c26534e10 2024-09-13T18:33:32.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.413+0000 7f2594e05640 1 -- 172.21.15.114:0/1465971642 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f259c0054b0 con 0x555c26534e10 2024-09-13T18:33:32.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.413+0000 7f2594e05640 1 -- 172.21.15.114:0/1465971642 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259c019070 con 0x555c26534e10 2024-09-13T18:33:32.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.417+0000 7f25ab227740 1 -- 172.21.15.114:0/1465971642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c26534e10 msgr2=0x555c265351f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:32.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.418+0000 7f25ab227740 1 --2- 172.21.15.114:0/1465971642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c26534e10 0x555c265351f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f259c00bcb0 tx=0x7f259c019000 comp rx=0 tx=0).stop 2024-09-13T18:33:32.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.421+0000 7f25ab227740 1 -- 172.21.15.114:0/1465971642 shutdown_connections 2024-09-13T18:33:32.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.421+0000 7f25ab227740 1 --2- 172.21.15.114:0/1465971642 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555c2653ffd0 0x555c265423c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.421+0000 7f25ab227740 1 --2- 172.21.15.114:0/1465971642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c26535730 0x555c2653fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.421+0000 7f25ab227740 1 --2- 172.21.15.114:0/1465971642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c26534e10 0x555c265351f0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.421+0000 7f25ab227740 1 -- 172.21.15.114:0/1465971642 >> 172.21.15.114:0/1465971642 conn(0x555c26531380 msgr2=0x555c26531760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:32.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.427+0000 7f25ab227740 1 -- 172.21.15.114:0/1465971642 shutdown_connections 2024-09-13T18:33:32.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.429+0000 7f25ab227740 1 -- 172.21.15.114:0/1465971642 wait complete. 2024-09-13T18:33:32.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.429+0000 7f25ab227740 1 -- start start 2024-09-13T18:33:32.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.429+0000 7f25ab227740 1 Processor -- start 2024-09-13T18:33:32.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.430+0000 7f25ab227740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555c26534e10 0x555c2653cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:32.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.430+0000 7f25ab227740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c26535730 0x555c2653d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:32.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.430+0000 7f25ab227740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c2653ffd0 0x555c2653d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:32.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.430+0000 7f25ab227740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c2643a2a0 con 0x555c26534e10 2024-09-13T18:33:32.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.430+0000 7f25ab227740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c26171430 con 0x555c26535730 2024-09-13T18:33:32.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.430+0000 7f25ab227740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c2641c680 con 0x555c2653ffd0 2024-09-13T18:33:32.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.431+0000 7f25ab227740 10 client.0 _ll_register_callbacks cb 0x555c26492610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:33:32.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.431+0000 7f25a59c6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c26535730 0x555c2653d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:32.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.431+0000 7f25a59c6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c26535730 0x555c2653d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:42342/0 (socket says 172.21.15.114:42342) 2024-09-13T18:33:32.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.431+0000 7f25a59c6640 1 -- 172.21.15.114:0/3480180702 learned_addr learned my addr 172.21.15.114:0/3480180702 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:32.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.431+0000 7f25a59c6640 1 -- 172.21.15.114:0/3480180702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c2653ffd0 msgr2=0x555c2653d7a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:33:32.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.431+0000 7f25a59c6640 1 --2- 172.21.15.114:0/3480180702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c2653ffd0 0x555c2653d7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.431+0000 7f25a59c6640 1 -- 172.21.15.114:0/3480180702 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555c26534e10 msgr2=0x555c2653cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:32.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.431+0000 7f25a59c6640 1 --2- 172.21.15.114:0/3480180702 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555c26534e10 0x555c2653cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.431+0000 7f25a59c6640 1 -- 172.21.15.114:0/3480180702 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f259c014e00 con 0x555c26535730 2024-09-13T18:33:32.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.432+0000 7f25a59c6640 1 --2- 172.21.15.114:0/3480180702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c26535730 0x555c2653d260 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f2590009230 tx=0x7f2590004480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=864db70097208a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:32.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.432+0000 7f2589ffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:33:32.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.432+0000 7f2589ffb640 1 -- 172.21.15.114:0/3480180702 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2590005220 con 0x555c26535730 2024-09-13T18:33:32.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.432+0000 7f2589ffb640 1 -- 172.21.15.114:0/3480180702 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f2590005380 con 0x555c26535730 2024-09-13T18:33:32.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.434+0000 7f25ab227740 1 -- 172.21.15.114:0/3480180702 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555c2618c4e0 con 0x555c26535730 2024-09-13T18:33:32.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.436+0000 7f2589ffb640 1 -- 172.21.15.114:0/3480180702 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2590005540 con 0x555c26535730 2024-09-13T18:33:32.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.439+0000 7f25ab227740 10 client.27965 fetch_fsmap learned FSMap version 37 2024-09-13T18:33:32.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.439+0000 7f25ab227740 1 -- 172.21.15.114:0/3480180702 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x555c26497620 con 0x555c26535730 2024-09-13T18:33:32.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.443+0000 7f2589ffb640 1 -- 172.21.15.114:0/3480180702 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x555c2618c4e0 con 0x555c26535730 2024-09-13T18:33:32.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.443+0000 7f2589ffb640 1 -- 172.21.15.114:0/3480180702 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 37) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f2590003890 con 0x555c26535730 2024-09-13T18:33:32.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.443+0000 7f25ab227740 10 client.27965 fetch_fsmap finished waiting for FSMap version 37 2024-09-13T18:33:32.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.443+0000 7f25ab227740 10 client.27965 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:33:32.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.443+0000 7f25ab227740 20 client.27965 populate_metadata read hostname 'smithi114' 2024-09-13T18:33:32.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.443+0000 7f25ab227740 1 --2- 172.21.15.114:0/3480180702 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x555c2654ff30 0x555c26552350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:32.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.443+0000 7f25ab227740 4 client.27965 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:33:32.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.443+0000 7f25ab227740 1 -- 172.21.15.114:0/3480180702 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555c2619d7e0 con 0x555c2654ff30 2024-09-13T18:33:32.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.443+0000 7f25a61c7640 1 --2- 172.21.15.114:0/3480180702 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x555c2654ff30 0x555c26552350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:32.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.444+0000 7f2589ffb640 10 client.27965 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:33:32.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.444+0000 7f25a61c7640 1 --2- 172.21.15.114:0/3480180702 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x555c2654ff30 0x555c26552350 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=853144083610167a server_cookie=b281c526816874ec in_seq=0 out_seq=0 2024-09-13T18:33:32.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.446+0000 7f2589ffb640 1 -- 172.21.15.114:0/3480180702 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x555c2619d7e0 con 0x555c2654ff30 2024-09-13T18:33:32.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.446+0000 7f2589ffb640 10 client.27965 handle_command_reply: tid=0 2024-09-13T18:33:32.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.446+0000 7f25ab227740 1 client.27965 shutdown 2024-09-13T18:33:32.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.450+0000 7f25ab227740 1 -- 172.21.15.114:0/3480180702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c26535730 msgr2=0x555c2653d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:32.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.450+0000 7f25ab227740 1 --2- 172.21.15.114:0/3480180702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c26535730 0x555c2653d260 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f2590009230 tx=0x7f2590004480 comp rx=0 tx=0).stop 2024-09-13T18:33:32.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.451+0000 7f25ab227740 1 -- 172.21.15.114:0/3480180702 shutdown_connections 2024-09-13T18:33:32.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.451+0000 7f25ab227740 1 --2- 172.21.15.114:0/3480180702 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x555c2654ff30 0x555c26552350 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.451+0000 7f25ab227740 1 --2- 172.21.15.114:0/3480180702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c2653ffd0 0x555c2653d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.451+0000 7f25ab227740 1 --2- 172.21.15.114:0/3480180702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c26535730 0x555c2653d260 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.451+0000 7f25ab227740 1 --2- 172.21.15.114:0/3480180702 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555c26534e10 0x555c2653cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.451+0000 7f25ab227740 1 -- 172.21.15.114:0/3480180702 >> 172.21.15.114:0/3480180702 conn(0x555c26531380 msgr2=0x555c265370e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:32.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.452+0000 7f2589ffb640 0 client.27965 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:33:32.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.452+0000 7f25ab227740 1 -- 172.21.15.114:0/3480180702 shutdown_connections 2024-09-13T18:33:32.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.454+0000 7f25ab227740 1 -- 172.21.15.114:0/3480180702 wait complete. 2024-09-13T18:33:32.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.454+0000 7f25ab227740 20 client.27965 trim_cache size 0 max 16384 2024-09-13T18:33:32.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:32 smithi114 ceph-mon[30897]: pgmap v344: 129 pgs: 129 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 28 MiB/s wr, 618 op/s 2024-09-13T18:33:32.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.487+0000 7f25ab227740 1 Processor -- start 2024-09-13T18:33:32.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.487+0000 7f25ab227740 1 -- start start 2024-09-13T18:33:32.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.487+0000 7f25ab227740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c26534e10 0x555c2653d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:32.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.487+0000 7f25ab227740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c26535730 0x555c26546d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:32.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.492+0000 7f25a61c7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c26534e10 0x555c2653d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:32.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.492+0000 7f25a61c7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c26534e10 0x555c2653d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:49784/0 (socket says 172.21.15.114:49784) 2024-09-13T18:33:32.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.492+0000 7f25a61c7640 1 -- 172.21.15.114:0/735319160 learned_addr learned my addr 172.21.15.114:0/735319160 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:32.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.492+0000 7f25ab227740 1 --2- 172.21.15.114:0/735319160 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555c2653ffd0 0x555c26547270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:32.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.492+0000 7f25ab227740 1 -- 172.21.15.114:0/735319160 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c2619d7e0 con 0x555c2653ffd0 2024-09-13T18:33:32.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.492+0000 7f25ab227740 1 -- 172.21.15.114:0/735319160 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c2618c4e0 con 0x555c26535730 2024-09-13T18:33:32.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.492+0000 7f25ab227740 1 -- 172.21.15.114:0/735319160 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c26534b90 con 0x555c26534e10 2024-09-13T18:33:32.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.493+0000 7f25a59c6640 1 --2- 172.21.15.114:0/735319160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c26535730 0x555c26546d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:32.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.493+0000 7f25a69c8640 1 --2- 172.21.15.114:0/735319160 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555c2653ffd0 0x555c26547270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:32.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.494+0000 7f25a61c7640 1 -- 172.21.15.114:0/735319160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c26535730 msgr2=0x555c26546d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:32.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.494+0000 7f25a61c7640 1 --2- 172.21.15.114:0/735319160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c26535730 0x555c26546d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.494+0000 7f25a61c7640 1 -- 172.21.15.114:0/735319160 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555c2653ffd0 msgr2=0x555c26547270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:32.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.494+0000 7f25a61c7640 1 --2- 172.21.15.114:0/735319160 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555c2653ffd0 0x555c26547270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.494+0000 7f25a61c7640 1 -- 172.21.15.114:0/735319160 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c26439790 con 0x555c26534e10 2024-09-13T18:33:32.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.494+0000 7f25a61c7640 1 --2- 172.21.15.114:0/735319160 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c26534e10 0x555c2653d5d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f259c00b5a0 tx=0x7f259c00b510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77fad9278ba3bf3b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:32.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.498+0000 7f2595606640 1 -- 172.21.15.114:0/735319160 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259c023040 con 0x555c26534e10 2024-09-13T18:33:32.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.498+0000 7f2595606640 1 -- 172.21.15.114:0/735319160 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f259c005620 con 0x555c26534e10 2024-09-13T18:33:32.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.499+0000 7f25ab227740 1 -- 172.21.15.114:0/735319160 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c26534e10 msgr2=0x555c2653d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:32.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.499+0000 7f25ab227740 1 --2- 172.21.15.114:0/735319160 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c26534e10 0x555c2653d5d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f259c00b5a0 tx=0x7f259c00b510 comp rx=0 tx=0).stop 2024-09-13T18:33:32.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.499+0000 7f25ab227740 1 -- 172.21.15.114:0/735319160 shutdown_connections 2024-09-13T18:33:32.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.499+0000 7f25ab227740 1 --2- 172.21.15.114:0/735319160 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555c2653ffd0 0x555c26547270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.499+0000 7f25ab227740 1 --2- 172.21.15.114:0/735319160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c26535730 0x555c26546d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.499+0000 7f25ab227740 1 --2- 172.21.15.114:0/735319160 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c26534e10 0x555c2653d5d0 secure :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f259c00b5a0 tx=0x7f259c00b510 comp rx=0 tx=0).stop 2024-09-13T18:33:32.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.499+0000 7f25ab227740 1 -- 172.21.15.114:0/735319160 >> 172.21.15.114:0/735319160 conn(0x555c26531380 msgr2=0x555c26537fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:32.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.508+0000 7f25ab227740 1 -- 172.21.15.114:0/735319160 shutdown_connections 2024-09-13T18:33:32.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.508+0000 7f25ab227740 1 -- 172.21.15.114:0/735319160 wait complete. 2024-09-13T18:33:32.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.508+0000 7f25ab227740 1 -- start start 2024-09-13T18:33:32.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.508+0000 7f25ab227740 1 Processor -- start 2024-09-13T18:33:32.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.509+0000 7f25ab227740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c26535730 0x555c2659e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:32.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.509+0000 7f25ab227740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c2653ffd0 0x555c2659ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:32.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.509+0000 7f25ab227740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555c26543100 0x555c26543560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:32.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.509+0000 7f25a61c7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c26535730 0x555c2659e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:32.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.510+0000 7f25a61c7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c26535730 0x555c2659e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:42366/0 (socket says 172.21.15.114:42366) 2024-09-13T18:33:32.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.510+0000 7f25a61c7640 1 -- 172.21.15.114:0/541011204 learned_addr learned my addr 172.21.15.114:0/541011204 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:32.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.510+0000 7f25ab227740 1 -- 172.21.15.114:0/541011204 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c26543c60 con 0x555c26543100 2024-09-13T18:33:32.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.510+0000 7f25ab227740 1 -- 172.21.15.114:0/541011204 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c26543dd0 con 0x555c26535730 2024-09-13T18:33:32.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.510+0000 7f25ab227740 1 -- 172.21.15.114:0/541011204 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c26543f90 con 0x555c2653ffd0 2024-09-13T18:33:32.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.510+0000 7f25ab227740 10 client.0 _ll_register_callbacks cb 0x7f2590005dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:33:32.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.511+0000 7f25a59c6640 1 --2- 172.21.15.114:0/541011204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c2653ffd0 0x555c2659ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:32.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.511+0000 7f25a61c7640 1 -- 172.21.15.114:0/541011204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c2653ffd0 msgr2=0x555c2659ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:32.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.511+0000 7f25a61c7640 1 --2- 172.21.15.114:0/541011204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c2653ffd0 0x555c2659ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.511+0000 7f25a61c7640 1 -- 172.21.15.114:0/541011204 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555c26543100 msgr2=0x555c26543560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:32.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.511+0000 7f25a61c7640 1 --2- 172.21.15.114:0/541011204 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555c26543100 0x555c26543560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.511+0000 7f25a61c7640 1 -- 172.21.15.114:0/541011204 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555c26453840 con 0x555c26535730 2024-09-13T18:33:32.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.513+0000 7f25a61c7640 1 --2- 172.21.15.114:0/541011204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c26535730 0x555c2659e840 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f259c02e040 tx=0x7f259c0058e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c8b2bb6579b61f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:32.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.518+0000 7f2589ffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:33:32.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.518+0000 7f2589ffb640 1 -- 172.21.15.114:0/541011204 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259c019070 con 0x555c26535730 2024-09-13T18:33:32.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.518+0000 7f2589ffb640 1 -- 172.21.15.114:0/541011204 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f259c02b2e0 con 0x555c26535730 2024-09-13T18:33:32.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.520+0000 7f25ab227740 1 -- 172.21.15.114:0/541011204 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f259c0054b0 con 0x555c26535730 2024-09-13T18:33:32.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.524+0000 7f2589ffb640 1 -- 172.21.15.114:0/541011204 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f259c023040 con 0x555c26535730 2024-09-13T18:33:32.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.526+0000 7f25ab227740 10 client.27971 fetch_fsmap learned FSMap version 37 2024-09-13T18:33:32.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.526+0000 7f25ab227740 1 -- 172.21.15.114:0/541011204 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x555c2645eea0 con 0x555c26535730 2024-09-13T18:33:32.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.528+0000 7f2589ffb640 1 -- 172.21.15.114:0/541011204 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f259c0054b0 con 0x555c26535730 2024-09-13T18:33:32.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.528+0000 7f2589ffb640 1 -- 172.21.15.114:0/541011204 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 37) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f259c00f9e0 con 0x555c26535730 2024-09-13T18:33:32.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.535+0000 7f25ab227740 10 client.27971 fetch_fsmap finished waiting for FSMap version 37 2024-09-13T18:33:32.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.535+0000 7f25ab227740 10 client.27971 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:33:32.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.535+0000 7f25ab227740 20 client.27971 populate_metadata read hostname 'smithi114' 2024-09-13T18:33:32.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.535+0000 7f25ab227740 1 --2- 172.21.15.114:0/541011204 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x555c265a7630 0x555c265a9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:32.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.535+0000 7f25ab227740 4 client.27971 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T18:33:32.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.535+0000 7f25ab227740 1 -- 172.21.15.114:0/541011204 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555c261f0e30 con 0x555c265a7630 2024-09-13T18:33:32.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.535+0000 7f25a59c6640 1 --2- 172.21.15.114:0/541011204 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x555c265a7630 0x555c265a9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:32.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.545+0000 7f25a59c6640 1 --2- 172.21.15.114:0/541011204 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x555c265a7630 0x555c265a9a20 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=412e485806673ae5 server_cookie=28bba97b4df99ba2 in_seq=0 out_seq=0 2024-09-13T18:33:32.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.546+0000 7f2589ffb640 10 client.27971 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:33:32.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.550+0000 7f2589ffb640 1 -- 172.21.15.114:0/541011204 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x555c261f0e30 con 0x555c265a7630 2024-09-13T18:33:32.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.550+0000 7f2589ffb640 10 client.27971 handle_command_reply: tid=0 2024-09-13T18:33:32.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.551+0000 7f25ab227740 1 client.27971 shutdown 2024-09-13T18:33:32.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.553+0000 7f25ab227740 1 -- 172.21.15.114:0/541011204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c26535730 msgr2=0x555c2659e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:32.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.553+0000 7f25ab227740 1 --2- 172.21.15.114:0/541011204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c26535730 0x555c2659e840 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f259c02e040 tx=0x7f259c0058e0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.558+0000 7f25ab227740 1 -- 172.21.15.114:0/541011204 shutdown_connections 2024-09-13T18:33:32.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.558+0000 7f25ab227740 1 --2- 172.21.15.114:0/541011204 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x555c265a7630 0x555c265a9a20 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.558+0000 7f25ab227740 1 --2- 172.21.15.114:0/541011204 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555c26543100 0x555c26543560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.558+0000 7f25ab227740 1 --2- 172.21.15.114:0/541011204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c2653ffd0 0x555c2659ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.558+0000 7f25ab227740 1 --2- 172.21.15.114:0/541011204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c26535730 0x555c2659e840 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.558+0000 7f25ab227740 1 -- 172.21.15.114:0/541011204 >> 172.21.15.114:0/541011204 conn(0x555c26531380 msgr2=0x555c2653b5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:32.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.559+0000 7f2589ffb640 0 client.27971 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:33:32.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.561+0000 7f25ab227740 1 -- 172.21.15.114:0/541011204 shutdown_connections 2024-09-13T18:33:32.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.562+0000 7f25ab227740 1 -- 172.21.15.114:0/541011204 wait complete. 2024-09-13T18:33:32.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.563+0000 7f25ab227740 20 client.27971 trim_cache size 0 max 16384 2024-09-13T18:33:32.565 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:33:32.565 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "no active scrubs running", 2024-09-13T18:33:32.565 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": {} 2024-09-13T18:33:32.565 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:33:32.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.565+0000 7f25a842b640 1 -- 172.21.15.114:0/4204444660 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f258c0a1410 msgr2=0x7f258c0a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:32.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.565+0000 7f25a842b640 1 --2- 172.21.15.114:0/4204444660 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f258c0a1410 0x7f258c0a38d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f259c005d00 tx=0x7f259c00b9e0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.565+0000 7f25a842b640 1 -- 172.21.15.114:0/4204444660 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25a0073680 msgr2=0x7f25a0073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:32.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.565+0000 7f25a842b640 1 --2- 172.21.15.114:0/4204444660 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25a0073680 0x7f25a0073b30 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f259800eaa0 tx=0x7f259800ef60 comp rx=0 tx=0).stop 2024-09-13T18:33:32.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.565+0000 7f25a842b640 1 -- 172.21.15.114:0/4204444660 shutdown_connections 2024-09-13T18:33:32.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.565+0000 7f25a842b640 1 --2- 172.21.15.114:0/4204444660 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f258c0a1410 0x7f258c0a38d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.565+0000 7f25a842b640 1 --2- 172.21.15.114:0/4204444660 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25a0073680 0x7f25a0073b30 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.565+0000 7f25a842b640 1 --2- 172.21.15.114:0/4204444660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25a0119400 0x7f25a0078a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.565+0000 7f25a842b640 1 --2- 172.21.15.114:0/4204444660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25a0107540 0x7f25a0078520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.565+0000 7f25a842b640 1 -- 172.21.15.114:0/4204444660 >> 172.21.15.114:0/4204444660 conn(0x7f25a006afe0 msgr2=0x7f25a0119d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:32.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.566+0000 7f25a842b640 1 -- 172.21.15.114:0/4204444660 shutdown_connections 2024-09-13T18:33:32.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.566+0000 7f25a842b640 1 -- 172.21.15.114:0/4204444660 wait complete. 2024-09-13T18:33:32.609 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-13T18:33:32.609 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:33:32.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:32 smithi134 ceph-mon[34283]: pgmap v344: 129 pgs: 129 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 28 MiB/s wr, 618 op/s 2024-09-13T18:33:32.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:32 smithi204 ceph-mon[34777]: pgmap v344: 129 pgs: 129 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 28 MiB/s wr, 618 op/s 2024-09-13T18:33:32.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.876+0000 7f0ff759e640 1 -- 172.21.15.114:0/1812240458 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ff8119400 msgr2=0x7f0ff811b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:32.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.876+0000 7f0ff759e640 1 --2- 172.21.15.114:0/1812240458 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ff8119400 0x7f0ff811b7f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0009cd0 tx=0x7f0ff002a930 comp rx=0 tx=0).stop 2024-09-13T18:33:32.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.876+0000 7f0ff759e640 1 -- 172.21.15.114:0/1812240458 shutdown_connections 2024-09-13T18:33:32.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.876+0000 7f0ff759e640 1 --2- 172.21.15.114:0/1812240458 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ff8119400 0x7f0ff811b7f0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.876+0000 7f0ff759e640 1 --2- 172.21.15.114:0/1812240458 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ff8107890 0x7f0ff8107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.876+0000 7f0ff759e640 1 --2- 172.21.15.114:0/1812240458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ff81074b0 0x7f0ff810e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.876+0000 7f0ff759e640 1 -- 172.21.15.114:0/1812240458 >> 172.21.15.114:0/1812240458 conn(0x7f0ff806ae10 msgr2=0x7f0ff806b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:32.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.883+0000 7f0ff759e640 1 -- 172.21.15.114:0/1812240458 shutdown_connections 2024-09-13T18:33:32.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.885+0000 7f0ff759e640 1 -- 172.21.15.114:0/1812240458 wait complete. 2024-09-13T18:33:32.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.886+0000 7f0ff759e640 1 Processor -- start 2024-09-13T18:33:32.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.886+0000 7f0ff759e640 1 -- start start 2024-09-13T18:33:32.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.886+0000 7f0ff759e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ff81074b0 0x7f0ff8118810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:32.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.886+0000 7f0ff759e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ff8107890 0x7f0ff8118d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:32.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.886+0000 7f0ff759e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ff8119400 0x7f0ff8112950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:32.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.886+0000 7f0ff759e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff8113130 con 0x7f0ff81074b0 2024-09-13T18:33:32.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.886+0000 7f0ff759e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff81132a0 con 0x7f0ff8107890 2024-09-13T18:33:32.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.886+0000 7f0ff759e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff8113460 con 0x7f0ff8119400 2024-09-13T18:33:32.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.887+0000 7f0ff6d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ff8119400 0x7f0ff8112950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:32.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.887+0000 7f0ff6d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ff8119400 0x7f0ff8112950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:49816/0 (socket says 172.21.15.114:49816) 2024-09-13T18:33:32.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.887+0000 7f0ff6d9d640 1 -- 172.21.15.114:0/3925352619 learned_addr learned my addr 172.21.15.114:0/3925352619 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:32.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.887+0000 7f0ff659c640 1 --2- 172.21.15.114:0/3925352619 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ff81074b0 0x7f0ff8118810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:32.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.887+0000 7f0ff6d9d640 1 -- 172.21.15.114:0/3925352619 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ff8107890 msgr2=0x7f0ff8118d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:32.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.887+0000 7f0ff6d9d640 1 --2- 172.21.15.114:0/3925352619 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ff8107890 0x7f0ff8118d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.887+0000 7f0ff6d9d640 1 -- 172.21.15.114:0/3925352619 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ff81074b0 msgr2=0x7f0ff8118810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:32.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.887+0000 7f0ff6d9d640 1 --2- 172.21.15.114:0/3925352619 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ff81074b0 0x7f0ff8118810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:32.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.887+0000 7f0ff6d9d640 1 -- 172.21.15.114:0/3925352619 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff0009900 con 0x7f0ff8119400 2024-09-13T18:33:32.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.888+0000 7f0ff6d9d640 1 --2- 172.21.15.114:0/3925352619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ff8119400 0x7f0ff8112950 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0005e30 tx=0x7f0ff0003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:32.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.888+0000 7f0fe77fe640 1 -- 172.21.15.114:0/3925352619 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff0031930 con 0x7f0ff8119400 2024-09-13T18:33:32.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.888+0000 7f0fe77fe640 1 -- 172.21.15.114:0/3925352619 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0ff0040070 con 0x7f0ff8119400 2024-09-13T18:33:32.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.888+0000 7f0fe77fe640 1 -- 172.21.15.114:0/3925352619 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff002f9d0 con 0x7f0ff8119400 2024-09-13T18:33:32.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.891+0000 7f0ff759e640 1 -- 172.21.15.114:0/3925352619 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ff8113680 con 0x7f0ff8119400 2024-09-13T18:33:32.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.891+0000 7f0ff759e640 1 -- 172.21.15.114:0/3925352619 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ff81ba380 con 0x7f0ff8119400 2024-09-13T18:33:32.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.893+0000 7f0fe57fa640 1 -- 172.21.15.114:0/3925352619 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ff8071aa0 con 0x7f0ff8119400 2024-09-13T18:33:32.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.897+0000 7f0fe77fe640 1 -- 172.21.15.114:0/3925352619 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f0ff0042030 con 0x7f0ff8119400 2024-09-13T18:33:32.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.898+0000 7f0fe77fe640 1 --2- 172.21.15.114:0/3925352619 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0fe00a1340 0x7f0fe00a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:32.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.899+0000 7f0fe77fe640 1 -- 172.21.15.114:0/3925352619 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f0ff00ee370 con 0x7f0ff8119400 2024-09-13T18:33:32.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.904+0000 7f0fe77fe640 1 -- 172.21.15.114:0/3925352619 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f0ff00b74d0 con 0x7f0ff8119400 2024-09-13T18:33:32.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.909+0000 7f0ff659c640 1 --2- 172.21.15.114:0/3925352619 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0fe00a1340 0x7f0fe00a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:32.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:32.923+0000 7f0ff659c640 1 --2- 172.21.15.114:0/3925352619 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0fe00a1340 0x7f0fe00a3800 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0fe8009b60 tx=0x7f0fe8005f90 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:33.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.404+0000 7f0fe57fa640 1 -- 172.21.15.114:0/3925352619 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ff8113d90 con 0x7f0ff8119400 2024-09-13T18:33:33.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.409+0000 7f0fe77fe640 1 -- 172.21.15.114:0/3925352619 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+10938 (secure 0 0 0) 0x7f0ff00baaf0 con 0x7f0ff8119400 2024-09-13T18:33:33.414 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:33:33.415 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4162817497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4162817497},{"type":"v1","addr":"172.21.15.114:6835","nonce":4162817497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/2435428478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":2435428478},{"type":"v1","addr":"172.21.15.114:6837","nonce":2435428478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T18:20:01.895117+0000","modified":"2024-09-13T18:33:29.306186+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24482,"mds_2":24475,"mds_3":24458,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/2046516451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":2046516451},{"type":"v1","addr":"172.21.15.114:6839","nonce":2046516451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":10,"addr":"172.21.15.114:6841/1956923446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":1956923446},{"type":"v1","addr":"172.21.15.114:6841","nonce":1956923446}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24391":{"gid":24391,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/507301496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":507301496},{"type":"v1","addr":"172.21.15.204:6833","nonce":507301496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4028657219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4028657219},{"type":"v1","addr":"172.21.15.134:6833","nonce":4028657219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/57029119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":57029119},{"type":"v1","addr":"172.21.15.204:6835","nonce":57029119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3068880939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3068880939},{"type":"v1","addr":"172.21.15.204:6837","nonce":3068880939}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"e","rank":3,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.134:6835/1150003275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1150003275},{"type":"v1","addr":"172.21.15.134:6835","nonce":1150003275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/344883912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":344883912},{"type":"v1","addr":"172.21.15.204:6839","nonce":344883912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6837/891692210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":891692210},{"type":"v1","addr":"172.21.15.134:6837","nonce":891692210}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6839/3466309724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3466309724},{"type":"v1","addr":"172.21.15.134:6839","nonce":3466309724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:33:33.415 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 37 2024-09-13T18:33:33.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.427+0000 7f0fe57fa640 1 -- 172.21.15.114:0/3925352619 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0fe00a1340 msgr2=0x7f0fe00a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:33.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.427+0000 7f0fe57fa640 1 --2- 172.21.15.114:0/3925352619 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0fe00a1340 0x7f0fe00a3800 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0fe8009b60 tx=0x7f0fe8005f90 comp rx=0 tx=0).stop 2024-09-13T18:33:33.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.427+0000 7f0fe57fa640 1 -- 172.21.15.114:0/3925352619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ff8119400 msgr2=0x7f0ff8112950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:33.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.427+0000 7f0fe57fa640 1 --2- 172.21.15.114:0/3925352619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ff8119400 0x7f0ff8112950 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0005e30 tx=0x7f0ff0003ba0 comp rx=0 tx=0).stop 2024-09-13T18:33:33.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.431+0000 7f0fe57fa640 1 -- 172.21.15.114:0/3925352619 shutdown_connections 2024-09-13T18:33:33.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.431+0000 7f0fe57fa640 1 --2- 172.21.15.114:0/3925352619 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0fe00a1340 0x7f0fe00a3800 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:33.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.431+0000 7f0fe57fa640 1 --2- 172.21.15.114:0/3925352619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ff8119400 0x7f0ff8112950 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:33.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.431+0000 7f0fe57fa640 1 --2- 172.21.15.114:0/3925352619 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ff8107890 0x7f0ff8118d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:33.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.431+0000 7f0fe57fa640 1 --2- 172.21.15.114:0/3925352619 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ff81074b0 0x7f0ff8118810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:33.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.431+0000 7f0fe57fa640 1 -- 172.21.15.114:0/3925352619 >> 172.21.15.114:0/3925352619 conn(0x7f0ff806ae10 msgr2=0x7f0ff806fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:33.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.431+0000 7f0fe57fa640 1 -- 172.21.15.114:0/3925352619 shutdown_connections 2024-09-13T18:33:33.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.431+0000 7f0fe57fa640 1 -- 172.21.15.114:0/3925352619 wait complete. 2024-09-13T18:33:33.466 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-09-13T18:33:33.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.711+0000 7f0f8b59e640 1 -- 172.21.15.114:0/3674412686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f840aa5b0 msgr2=0x7f0f840aa990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:33.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.711+0000 7f0f8b59e640 1 --2- 172.21.15.114:0/3674412686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f840aa5b0 0x7f0f840aa990 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c009900 tx=0x7f0f7c02a4b0 comp rx=0 tx=0).stop 2024-09-13T18:33:33.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.711+0000 7f0f8b59e640 1 -- 172.21.15.114:0/3674412686 shutdown_connections 2024-09-13T18:33:33.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.711+0000 7f0f8b59e640 1 --2- 172.21.15.114:0/3674412686 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f840a6b10 0x7f0f840b2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:33.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.711+0000 7f0f8b59e640 1 --2- 172.21.15.114:0/3674412686 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f840a6190 0x7f0f840a65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:33.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.711+0000 7f0f8b59e640 1 --2- 172.21.15.114:0/3674412686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f840aa5b0 0x7f0f840aa990 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:33.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.711+0000 7f0f8b59e640 1 -- 172.21.15.114:0/3674412686 >> 172.21.15.114:0/3674412686 conn(0x7f0f8401a0e0 msgr2=0x7f0f8401a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:33.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.717+0000 7f0f8b59e640 1 -- 172.21.15.114:0/3674412686 shutdown_connections 2024-09-13T18:33:33.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.717+0000 7f0f8b59e640 1 -- 172.21.15.114:0/3674412686 wait complete. 2024-09-13T18:33:33.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.718+0000 7f0f8b59e640 1 Processor -- start 2024-09-13T18:33:33.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.718+0000 7f0f8b59e640 1 -- start start 2024-09-13T18:33:33.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.718+0000 7f0f8b59e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f840a6190 0x7f0f840b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:33.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.718+0000 7f0f8b59e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f840a6b10 0x7f0f840b1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:33.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.718+0000 7f0f8b59e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f840aa5b0 0x7f0f8414edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:33.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.718+0000 7f0f8b59e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8414f390 con 0x7f0f840a6b10 2024-09-13T18:33:33.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.718+0000 7f0f8b59e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f840b7860 con 0x7f0f840aa5b0 2024-09-13T18:33:33.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.718+0000 7f0f8b59e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f840b7a20 con 0x7f0f840a6190 2024-09-13T18:33:33.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.719+0000 7f0f8ad9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f840aa5b0 0x7f0f8414edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:33.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.719+0000 7f0f8ad9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f840aa5b0 0x7f0f8414edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:40504/0 (socket says 172.21.15.114:40504) 2024-09-13T18:33:33.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.719+0000 7f0f8ad9d640 1 -- 172.21.15.114:0/2514464801 learned_addr learned my addr 172.21.15.114:0/2514464801 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:33.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.719+0000 7f0f89d9b640 1 --2- 172.21.15.114:0/2514464801 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f840a6b10 0x7f0f840b1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:33.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.722+0000 7f0f8ad9d640 1 -- 172.21.15.114:0/2514464801 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f840a6190 msgr2=0x7f0f840b0c00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:33:33.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.722+0000 7f0f8ad9d640 1 --2- 172.21.15.114:0/2514464801 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f840a6190 0x7f0f840b0c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:33.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.722+0000 7f0f8ad9d640 1 -- 172.21.15.114:0/2514464801 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f840a6b10 msgr2=0x7f0f840b1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:33.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.722+0000 7f0f8ad9d640 1 --2- 172.21.15.114:0/2514464801 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f840a6b10 0x7f0f840b1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:33.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.722+0000 7f0f8ad9d640 1 -- 172.21.15.114:0/2514464801 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f7c009560 con 0x7f0f840aa5b0 2024-09-13T18:33:33.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.723+0000 7f0f8ad9d640 1 --2- 172.21.15.114:0/2514464801 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f840aa5b0 0x7f0f8414edc0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c0639a0 tx=0x7f0f8c070d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:33.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.728+0000 7f0f7b7fe640 1 -- 172.21.15.114:0/2514464801 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c066cd0 con 0x7f0f840aa5b0 2024-09-13T18:33:33.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.728+0000 7f0f7b7fe640 1 -- 172.21.15.114:0/2514464801 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0f8c066e30 con 0x7f0f840aa5b0 2024-09-13T18:33:33.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.728+0000 7f0f7b7fe640 1 -- 172.21.15.114:0/2514464801 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c073460 con 0x7f0f840aa5b0 2024-09-13T18:33:33.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.731+0000 7f0f8b59e640 1 -- 172.21.15.114:0/2514464801 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f840b7d00 con 0x7f0f840aa5b0 2024-09-13T18:33:33.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.731+0000 7f0f8b59e640 1 -- 172.21.15.114:0/2514464801 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f840b8190 con 0x7f0f840aa5b0 2024-09-13T18:33:33.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.739+0000 7f0f7b7fe640 1 -- 172.21.15.114:0/2514464801 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f0f8c06e810 con 0x7f0f840aa5b0 2024-09-13T18:33:33.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.742+0000 7f0f93a68740 1 Processor -- start 2024-09-13T18:33:33.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.743+0000 7f0f93a68740 1 -- start start 2024-09-13T18:33:33.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.743+0000 7f0f93a68740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0e159d970 0x55c0e159dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:33.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.744+0000 7f0f93a68740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0e159e290 0x55c0e15a85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:33.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.744+0000 7f0f93a68740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0e15a8b30 0x55c0e15aaf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:33.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.744+0000 7f0f93a68740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0e1507cc0 con 0x55c0e15a8b30 2024-09-13T18:33:33.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.744+0000 7f0f93a68740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0e1505450 con 0x55c0e159d970 2024-09-13T18:33:33.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.744+0000 7f0f93a68740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0e1500d40 con 0x55c0e159e290 2024-09-13T18:33:33.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.745+0000 7f0f89d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0e159d970 0x55c0e159dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:33.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.745+0000 7f0f8a59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0e159e290 0x55c0e15a85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:33.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.745+0000 7f0f8a59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0e159e290 0x55c0e15a85f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:59532/0 (socket says 172.21.15.114:59532) 2024-09-13T18:33:33.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.745+0000 7f0f8a59c640 1 -- 172.21.15.114:0/4205761037 learned_addr learned my addr 172.21.15.114:0/4205761037 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:33.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.747+0000 7f0f89d9b640 1 --2- 172.21.15.114:0/4205761037 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0e15a8b30 0x55c0e15aaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:33.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.748+0000 7f0f89d9b640 1 -- 172.21.15.114:0/4205761037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0e159e290 msgr2=0x55c0e15a85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:33.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.748+0000 7f0f89d9b640 1 --2- 172.21.15.114:0/4205761037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0e159e290 0x55c0e15a85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:33.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.748+0000 7f0f89d9b640 1 -- 172.21.15.114:0/4205761037 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0e15a8b30 msgr2=0x55c0e15aaf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:33.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.748+0000 7f0f89d9b640 1 --2- 172.21.15.114:0/4205761037 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0e15a8b30 0x55c0e15aaf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:33.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.748+0000 7f0f89d9b640 1 -- 172.21.15.114:0/4205761037 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c0e14c8620 con 0x55c0e159d970 2024-09-13T18:33:33.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.750+0000 7f0f89d9b640 1 --2- 172.21.15.114:0/4205761037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0e159d970 0x55c0e159dd50 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f0f80006fd0 tx=0x7f0f80009480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=caf8c4a9788ef62f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:33.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.751+0000 7f0f7b7fe640 1 --2- 172.21.15.114:0/2514464801 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0f740a13e0 0x7f0f740a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:33.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.751+0000 7f0f7b7fe640 1 -- 172.21.15.114:0/2514464801 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f0f8c12f0f0 con 0x7f0f840aa5b0 2024-09-13T18:33:33.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.752+0000 7f0f8a59c640 1 --2- 172.21.15.114:0/2514464801 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0f740a13e0 0x7f0f740a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:33.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.753+0000 7f0f8a59c640 1 --2- 172.21.15.114:0/2514464801 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0f740a13e0 0x7f0f740a38a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c000c00 tx=0x7f0f7c005740 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:33.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.754+0000 7f0f78ff9640 1 -- 172.21.15.114:0/4205761037 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f80017470 con 0x55c0e159d970 2024-09-13T18:33:33.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.754+0000 7f0f78ff9640 1 -- 172.21.15.114:0/4205761037 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0f80017a60 con 0x55c0e159d970 2024-09-13T18:33:33.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.755+0000 7f0f93a68740 1 -- 172.21.15.114:0/4205761037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0e159d970 msgr2=0x55c0e159dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:33.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.755+0000 7f0f93a68740 1 --2- 172.21.15.114:0/4205761037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0e159d970 0x55c0e159dd50 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f0f80006fd0 tx=0x7f0f80009480 comp rx=0 tx=0).stop 2024-09-13T18:33:33.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.762+0000 7f0f93a68740 1 -- 172.21.15.114:0/4205761037 shutdown_connections 2024-09-13T18:33:33.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.762+0000 7f0f93a68740 1 --2- 172.21.15.114:0/4205761037 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0e15a8b30 0x55c0e15aaf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:33.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.762+0000 7f0f93a68740 1 --2- 172.21.15.114:0/4205761037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0e159e290 0x55c0e15a85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:33.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.762+0000 7f0f93a68740 1 --2- 172.21.15.114:0/4205761037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0e159d970 0x55c0e159dd50 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:33.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.762+0000 7f0f93a68740 1 -- 172.21.15.114:0/4205761037 >> 172.21.15.114:0/4205761037 conn(0x55c0e1599ee0 msgr2=0x55c0e159a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:33.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.778+0000 7f0f93a68740 1 -- 172.21.15.114:0/4205761037 shutdown_connections 2024-09-13T18:33:33.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.780+0000 7f0f93a68740 1 -- 172.21.15.114:0/4205761037 wait complete. 2024-09-13T18:33:33.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.780+0000 7f0f93a68740 1 -- start start 2024-09-13T18:33:33.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.780+0000 7f0f93a68740 1 Processor -- start 2024-09-13T18:33:33.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.780+0000 7f0f93a68740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0e159d970 0x55c0e15a80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:33.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.780+0000 7f0f93a68740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0e159e290 0x55c0e15a85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:33.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.780+0000 7f0f93a68740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0e15a8b30 0x55c0e15a56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:33.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.780+0000 7f0f93a68740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0e14bf180 con 0x55c0e159d970 2024-09-13T18:33:33.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.780+0000 7f0f93a68740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0e11d9430 con 0x55c0e159e290 2024-09-13T18:33:33.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.780+0000 7f0f93a68740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0e14845e0 con 0x55c0e15a8b30 2024-09-13T18:33:33.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.780+0000 7f0f93a68740 10 client.0 _ll_register_callbacks cb 0x55c0e14f0aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:33:33.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.781+0000 7f0f8a59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0e159d970 0x55c0e15a80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:33.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.781+0000 7f0f8a59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0e159d970 0x55c0e15a80b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:52598/0 (socket says 172.21.15.114:52598) 2024-09-13T18:33:33.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.781+0000 7f0f8a59c640 1 -- 172.21.15.114:0/2418160051 learned_addr learned my addr 172.21.15.114:0/2418160051 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:33.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.781+0000 7f0f89d9b640 1 --2- 172.21.15.114:0/2418160051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0e159e290 0x55c0e15a85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:33.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.782+0000 7f0f89d9b640 1 -- 172.21.15.114:0/2418160051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0e15a8b30 msgr2=0x55c0e15a56e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:33:33.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.782+0000 7f0f89d9b640 1 --2- 172.21.15.114:0/2418160051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0e15a8b30 0x55c0e15a56e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:33.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.782+0000 7f0f89d9b640 1 -- 172.21.15.114:0/2418160051 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0e159d970 msgr2=0x55c0e15a80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:33.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.782+0000 7f0f89d9b640 1 --2- 172.21.15.114:0/2418160051 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0e159d970 0x55c0e15a80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:33.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.782+0000 7f0f89d9b640 1 -- 172.21.15.114:0/2418160051 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f80009c20 con 0x55c0e159e290 2024-09-13T18:33:33.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.783+0000 7f0f89d9b640 1 --2- 172.21.15.114:0/2418160051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0e159e290 0x55c0e15a85f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f0f80006fd0 tx=0x7f0f80018e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=485e8aee4c6d43a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:33.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.783+0000 7f0f71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:33:33.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.783+0000 7f0f71ffb640 1 -- 172.21.15.114:0/2418160051 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f80017470 con 0x55c0e159e290 2024-09-13T18:33:33.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.783+0000 7f0f71ffb640 1 -- 172.21.15.114:0/2418160051 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f0f800175d0 con 0x55c0e159e290 2024-09-13T18:33:33.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.783+0000 7f0f93a68740 1 -- 172.21.15.114:0/2418160051 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c0e152ab80 con 0x55c0e159e290 2024-09-13T18:33:33.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.784+0000 7f0f71ffb640 1 -- 172.21.15.114:0/2418160051 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f800177d0 con 0x55c0e159e290 2024-09-13T18:33:33.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.784+0000 7f0f93a68740 10 client.27986 fetch_fsmap learned FSMap version 37 2024-09-13T18:33:33.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.784+0000 7f0f93a68740 1 -- 172.21.15.114:0/2418160051 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55c0e14ff3c0 con 0x55c0e159e290 2024-09-13T18:33:33.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.785+0000 7f0f71ffb640 1 -- 172.21.15.114:0/2418160051 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0f80031080 con 0x55c0e159e290 2024-09-13T18:33:33.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.786+0000 7f0f71ffb640 1 -- 172.21.15.114:0/2418160051 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 37) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f0f80026070 con 0x55c0e159e290 2024-09-13T18:33:33.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.787+0000 7f0f93a68740 10 client.27986 fetch_fsmap finished waiting for FSMap version 37 2024-09-13T18:33:33.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.787+0000 7f0f93a68740 10 client.27986 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-13T18:33:33.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.788+0000 7f0f93a68740 20 client.27986 populate_metadata read hostname 'smithi114' 2024-09-13T18:33:33.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.788+0000 7f0f93a68740 1 --2- 172.21.15.114:0/2418160051 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55c0e15b84a0 0x55c0e15ba8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:33.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.788+0000 7f0f93a68740 4 client.27986 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:33:33.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.788+0000 7f0f93a68740 1 -- 172.21.15.114:0/2418160051 --> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c0e14f31f0 con 0x55c0e15b84a0 2024-09-13T18:33:33.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.790+0000 7f0f8a59c640 1 --2- 172.21.15.114:0/2418160051 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55c0e15b84a0 0x55c0e15ba8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:33.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.790+0000 7f0f8a59c640 1 --2- 172.21.15.114:0/2418160051 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55c0e15b84a0 0x55c0e15ba8c0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=49bc950450c57163 server_cookie=e48841cbc911e3e0 in_seq=0 out_seq=0 2024-09-13T18:33:33.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.791+0000 7f0f71ffb640 10 client.27986 ms_handle_connect on v2:172.21.15.114:6840/1956923446 2024-09-13T18:33:33.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.792+0000 7f0f71ffb640 1 -- 172.21.15.114:0/2418160051 <== mds.4 v2:172.21.15.114:6840/1956923446 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55c0e14f31f0 con 0x55c0e15b84a0 2024-09-13T18:33:33.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.792+0000 7f0f71ffb640 10 client.27986 handle_command_reply: tid=0 2024-09-13T18:33:33.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.794+0000 7f0f93a68740 1 client.27986 shutdown 2024-09-13T18:33:33.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.797+0000 7f0f93a68740 1 -- 172.21.15.114:0/2418160051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0e159e290 msgr2=0x55c0e15a85f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:33.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.797+0000 7f0f93a68740 1 --2- 172.21.15.114:0/2418160051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0e159e290 0x55c0e15a85f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f0f80006fd0 tx=0x7f0f80018e10 comp rx=0 tx=0).stop 2024-09-13T18:33:33.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.803+0000 7f0f93a68740 1 -- 172.21.15.114:0/2418160051 shutdown_connections 2024-09-13T18:33:33.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.803+0000 7f0f93a68740 1 --2- 172.21.15.114:0/2418160051 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55c0e15b84a0 0x55c0e15ba8c0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:33.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.803+0000 7f0f93a68740 1 --2- 172.21.15.114:0/2418160051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0e15a8b30 0x55c0e15a56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:33.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.803+0000 7f0f93a68740 1 --2- 172.21.15.114:0/2418160051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0e159e290 0x55c0e15a85f0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:33.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.803+0000 7f0f93a68740 1 --2- 172.21.15.114:0/2418160051 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0e159d970 0x55c0e15a80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:33.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.804+0000 7f0f93a68740 1 -- 172.21.15.114:0/2418160051 >> 172.21.15.114:0/2418160051 conn(0x55c0e1599ee0 msgr2=0x55c0e1599230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:33.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.804+0000 7f0f71ffb640 0 client.27986 ms_handle_reset on v2:172.21.15.114:6840/1956923446 2024-09-13T18:33:33.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.805+0000 7f0f93a68740 1 -- 172.21.15.114:0/2418160051 shutdown_connections 2024-09-13T18:33:33.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.807+0000 7f0f93a68740 1 -- 172.21.15.114:0/2418160051 wait complete. 2024-09-13T18:33:33.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.810+0000 7f0f93a68740 20 client.27986 trim_cache size 0 max 16384 2024-09-13T18:33:33.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.833+0000 7f0f93a68740 1 Processor -- start 2024-09-13T18:33:33.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.833+0000 7f0f93a68740 1 -- start start 2024-09-13T18:33:33.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.833+0000 7f0f93a68740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0e159d970 0x55c0e1599980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:33.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.833+0000 7f0f93a68740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0e159e290 0x55c0e15af900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:33.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.833+0000 7f0f93a68740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0e15a8b30 0x55c0e15afe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:33.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.833+0000 7f0f93a68740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0e159f0f0 con 0x55c0e159d970 2024-09-13T18:33:33.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.833+0000 7f0f93a68740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0e15ab2a0 con 0x55c0e15a8b30 2024-09-13T18:33:33.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.833+0000 7f0f93a68740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0e159d7a0 con 0x55c0e159e290 2024-09-13T18:33:33.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.833+0000 7f0f8ad9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0e15a8b30 0x55c0e15afe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:33.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.833+0000 7f0f8ad9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0e15a8b30 0x55c0e15afe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:40534/0 (socket says 172.21.15.114:40534) 2024-09-13T18:33:33.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.833+0000 7f0f8ad9d640 1 -- 172.21.15.114:0/281965946 learned_addr learned my addr 172.21.15.114:0/281965946 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:33.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.833+0000 7f0f89d9b640 1 --2- 172.21.15.114:0/281965946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0e159e290 0x55c0e15af900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:33.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.834+0000 7f0f8ad9d640 1 -- 172.21.15.114:0/281965946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0e159e290 msgr2=0x55c0e15af900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:33.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.834+0000 7f0f8ad9d640 1 --2- 172.21.15.114:0/281965946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0e159e290 0x55c0e15af900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:33.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.834+0000 7f0f8ad9d640 1 -- 172.21.15.114:0/281965946 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0e159d970 msgr2=0x55c0e1599980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:33.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.834+0000 7f0f8ad9d640 1 --2- 172.21.15.114:0/281965946 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0e159d970 0x55c0e1599980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:33.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.834+0000 7f0f8ad9d640 1 -- 172.21.15.114:0/281965946 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c0e14a1770 con 0x55c0e15a8b30 2024-09-13T18:33:33.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.834+0000 7f0f8ad9d640 1 --2- 172.21.15.114:0/281965946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0e15a8b30 0x55c0e15afe40 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c02c9a0 tx=0x7f0f8c0fbbd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2956e4316a1f355 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:33.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.836+0000 7f0f797fa640 1 -- 172.21.15.114:0/281965946 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c0fc500 con 0x55c0e15a8b30 2024-09-13T18:33:33.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.836+0000 7f0f797fa640 1 -- 172.21.15.114:0/281965946 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0f8c0fbe00 con 0x55c0e15a8b30 2024-09-13T18:33:33.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.837+0000 7f0f93a68740 1 -- 172.21.15.114:0/281965946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0e15a8b30 msgr2=0x55c0e15afe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:33.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.837+0000 7f0f93a68740 1 --2- 172.21.15.114:0/281965946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0e15a8b30 0x55c0e15afe40 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c02c9a0 tx=0x7f0f8c0fbbd0 comp rx=0 tx=0).stop 2024-09-13T18:33:33.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.841+0000 7f0f93a68740 1 -- 172.21.15.114:0/281965946 shutdown_connections 2024-09-13T18:33:33.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.841+0000 7f0f93a68740 1 --2- 172.21.15.114:0/281965946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0e15a8b30 0x55c0e15afe40 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:33.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.841+0000 7f0f93a68740 1 --2- 172.21.15.114:0/281965946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0e159e290 0x55c0e15af900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:33.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.841+0000 7f0f93a68740 1 --2- 172.21.15.114:0/281965946 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0e159d970 0x55c0e1599980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:33.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.841+0000 7f0f93a68740 1 -- 172.21.15.114:0/281965946 >> 172.21.15.114:0/281965946 conn(0x55c0e1599ee0 msgr2=0x55c0e15a3d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:33.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.845+0000 7f0f93a68740 1 -- 172.21.15.114:0/281965946 shutdown_connections 2024-09-13T18:33:33.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.847+0000 7f0f93a68740 1 -- 172.21.15.114:0/281965946 wait complete. 2024-09-13T18:33:33.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.848+0000 7f0f93a68740 1 -- start start 2024-09-13T18:33:33.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.848+0000 7f0f93a68740 1 Processor -- start 2024-09-13T18:33:33.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.848+0000 7f0f93a68740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0e159d970 0x55c0e15ac550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:33.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.848+0000 7f0f93a68740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0e159e290 0x55c0e15aca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:33.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.848+0000 7f0f93a68740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0e15a8b30 0x55c0e15acfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:33.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.848+0000 7f0f93a68740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0e160ea70 con 0x55c0e159e290 2024-09-13T18:33:33.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.848+0000 7f0f93a68740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0e160ebe0 con 0x55c0e159d970 2024-09-13T18:33:33.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.848+0000 7f0f93a68740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0e160eda0 con 0x55c0e15a8b30 2024-09-13T18:33:33.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.848+0000 7f0f93a68740 10 client.0 _ll_register_callbacks cb 0x7f0f800097b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:33:33.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.849+0000 7f0f8ad9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0e15a8b30 0x55c0e15acfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:33.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.849+0000 7f0f8ad9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0e15a8b30 0x55c0e15acfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:59554/0 (socket says 172.21.15.114:59554) 2024-09-13T18:33:33.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.849+0000 7f0f8ad9d640 1 -- 172.21.15.114:0/1926041211 learned_addr learned my addr 172.21.15.114:0/1926041211 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:33.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.850+0000 7f0f8ad9d640 1 -- 172.21.15.114:0/1926041211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0e159d970 msgr2=0x55c0e15ac550 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:33:33.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.850+0000 7f0f8ad9d640 1 --2- 172.21.15.114:0/1926041211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0e159d970 0x55c0e15ac550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:33.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.850+0000 7f0f8ad9d640 1 -- 172.21.15.114:0/1926041211 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0e159e290 msgr2=0x55c0e15aca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:33.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.850+0000 7f0f8ad9d640 1 --2- 172.21.15.114:0/1926041211 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0e159e290 0x55c0e15aca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:33.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.850+0000 7f0f8ad9d640 1 -- 172.21.15.114:0/1926041211 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f840b8190 con 0x55c0e15a8b30 2024-09-13T18:33:33.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.851+0000 7f0f8ad9d640 1 --2- 172.21.15.114:0/1926041211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0e15a8b30 0x55c0e15acfd0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c0fbd90 tx=0x7f0f8c0662e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd78936178484e39 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:33.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.851+0000 7f0f71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:33:33.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.851+0000 7f0f71ffb640 1 -- 172.21.15.114:0/1926041211 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c103ff0 con 0x55c0e15a8b30 2024-09-13T18:33:33.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.852+0000 7f0f93a68740 1 -- 172.21.15.114:0/1926041211 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0f80017a60 con 0x55c0e15a8b30 2024-09-13T18:33:33.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.852+0000 7f0f71ffb640 1 -- 172.21.15.114:0/1926041211 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f0f8c104280 con 0x55c0e15a8b30 2024-09-13T18:33:33.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.852+0000 7f0f71ffb640 1 -- 172.21.15.114:0/1926041211 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f8c0fa1a0 con 0x55c0e15a8b30 2024-09-13T18:33:33.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.856+0000 7f0f93a68740 10 client.27907 fetch_fsmap learned FSMap version 37 2024-09-13T18:33:33.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.856+0000 7f0f93a68740 1 -- 172.21.15.114:0/1926041211 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55c0e14c6e80 con 0x55c0e15a8b30 2024-09-13T18:33:33.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.860+0000 7f0f71ffb640 1 -- 172.21.15.114:0/1926041211 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0f80017a60 con 0x55c0e15a8b30 2024-09-13T18:33:33.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.861+0000 7f0f71ffb640 1 -- 172.21.15.114:0/1926041211 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 37) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f0f8c12cb90 con 0x55c0e15a8b30 2024-09-13T18:33:33.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.861+0000 7f0f93a68740 10 client.27907 fetch_fsmap finished waiting for FSMap version 37 2024-09-13T18:33:33.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.861+0000 7f0f93a68740 10 client.27907 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-13T18:33:33.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.861+0000 7f0f93a68740 20 client.27907 populate_metadata read hostname 'smithi114' 2024-09-13T18:33:33.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.861+0000 7f0f93a68740 1 --2- 172.21.15.114:0/1926041211 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55c0e1610150 0x55c0e1612570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:33.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.861+0000 7f0f93a68740 4 client.27907 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:33:33.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.861+0000 7f0f93a68740 1 -- 172.21.15.114:0/1926041211 --> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c0e11f44e0 con 0x55c0e1610150 2024-09-13T18:33:33.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.863+0000 7f0f8a59c640 1 --2- 172.21.15.114:0/1926041211 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55c0e1610150 0x55c0e1612570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:33.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.864+0000 7f0f8a59c640 1 --2- 172.21.15.114:0/1926041211 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55c0e1610150 0x55c0e1612570 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b3fb9904c60a9041 server_cookie=cafff9dfaadef9d2 in_seq=0 out_seq=0 2024-09-13T18:33:33.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.864+0000 7f0f71ffb640 10 client.27907 ms_handle_connect on v2:172.21.15.114:6840/1956923446 2024-09-13T18:33:33.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.864+0000 7f0f71ffb640 1 -- 172.21.15.114:0/1926041211 <== mds.4 v2:172.21.15.114:6840/1956923446 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c0e11f44e0 con 0x55c0e1610150 2024-09-13T18:33:33.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.864+0000 7f0f71ffb640 10 client.27907 handle_command_reply: tid=0 2024-09-13T18:33:33.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.864+0000 7f0f93a68740 1 client.27907 shutdown 2024-09-13T18:33:33.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.867+0000 7f0f93a68740 1 -- 172.21.15.114:0/1926041211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0e15a8b30 msgr2=0x55c0e15acfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:33.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.867+0000 7f0f93a68740 1 --2- 172.21.15.114:0/1926041211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0e15a8b30 0x55c0e15acfd0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c0fbd90 tx=0x7f0f8c0662e0 comp rx=0 tx=0).stop 2024-09-13T18:33:33.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.872+0000 7f0f93a68740 1 -- 172.21.15.114:0/1926041211 shutdown_connections 2024-09-13T18:33:33.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.872+0000 7f0f93a68740 1 --2- 172.21.15.114:0/1926041211 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x55c0e1610150 0x55c0e1612570 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:33.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.872+0000 7f0f93a68740 1 --2- 172.21.15.114:0/1926041211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c0e15a8b30 0x55c0e15acfd0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:33.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.872+0000 7f0f93a68740 1 --2- 172.21.15.114:0/1926041211 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0e159e290 0x55c0e15aca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:33.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.872+0000 7f0f93a68740 1 --2- 172.21.15.114:0/1926041211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0e159d970 0x55c0e15ac550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:33.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.872+0000 7f0f93a68740 1 -- 172.21.15.114:0/1926041211 >> 172.21.15.114:0/1926041211 conn(0x55c0e1599ee0 msgr2=0x55c0e15a04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:33.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.874+0000 7f0f71ffb640 0 client.27907 ms_handle_reset on v2:172.21.15.114:6840/1956923446 2024-09-13T18:33:33.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.877+0000 7f0f93a68740 1 -- 172.21.15.114:0/1926041211 shutdown_connections 2024-09-13T18:33:33.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.877+0000 7f0f93a68740 1 -- 172.21.15.114:0/1926041211 wait complete. 2024-09-13T18:33:33.881 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:33:33.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.879+0000 7f0f93a68740 20 client.27907 trim_cache size 0 max 16384 2024-09-13T18:33:33.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.882+0000 7f0f8b59e640 1 -- 172.21.15.114:0/2514464801 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0f740a13e0 msgr2=0x7f0f740a38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:33.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.882+0000 7f0f8b59e640 1 --2- 172.21.15.114:0/2514464801 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0f740a13e0 0x7f0f740a38a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c000c00 tx=0x7f0f7c005740 comp rx=0 tx=0).stop 2024-09-13T18:33:33.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.883+0000 7f0f8b59e640 1 -- 172.21.15.114:0/2514464801 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f840aa5b0 msgr2=0x7f0f8414edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:33.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.883+0000 7f0f8b59e640 1 --2- 172.21.15.114:0/2514464801 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f840aa5b0 0x7f0f8414edc0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c0639a0 tx=0x7f0f8c070d90 comp rx=0 tx=0).stop 2024-09-13T18:33:33.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.884+0000 7f0f8b59e640 1 -- 172.21.15.114:0/2514464801 shutdown_connections 2024-09-13T18:33:33.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.884+0000 7f0f8b59e640 1 --2- 172.21.15.114:0/2514464801 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0f740a13e0 0x7f0f740a38a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:33.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.884+0000 7f0f8b59e640 1 --2- 172.21.15.114:0/2514464801 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f840aa5b0 0x7f0f8414edc0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:33.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.884+0000 7f0f8b59e640 1 --2- 172.21.15.114:0/2514464801 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f840a6b10 0x7f0f840b1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:33.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.884+0000 7f0f8b59e640 1 --2- 172.21.15.114:0/2514464801 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f840a6190 0x7f0f840b0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:33.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.884+0000 7f0f8b59e640 1 -- 172.21.15.114:0/2514464801 >> 172.21.15.114:0/2514464801 conn(0x7f0f8401a0e0 msgr2=0x7f0f840a4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:33.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.885+0000 7f0f8b59e640 1 -- 172.21.15.114:0/2514464801 shutdown_connections 2024-09-13T18:33:33.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:33.886+0000 7f0f8b59e640 1 -- 172.21.15.114:0/2514464801 wait complete. 2024-09-13T18:33:33.917 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-09-13T18:33:34.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.156+0000 7ff0dec34640 1 -- 172.21.15.114:0/4041325084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0d81074b0 msgr2=0x7ff0d810e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:34.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.156+0000 7ff0dec34640 1 --2- 172.21.15.114:0/4041325084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0d81074b0 0x7ff0d810e510 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff0d001acb0 tx=0x7ff0d003b900 comp rx=0 tx=0).stop 2024-09-13T18:33:34.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.156+0000 7ff0dec34640 1 -- 172.21.15.114:0/4041325084 shutdown_connections 2024-09-13T18:33:34.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.156+0000 7ff0dec34640 1 --2- 172.21.15.114:0/4041325084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0d8119400 0x7ff0d811b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.156+0000 7ff0dec34640 1 --2- 172.21.15.114:0/4041325084 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0d8107890 0x7ff0d8107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.156+0000 7ff0dec34640 1 --2- 172.21.15.114:0/4041325084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0d81074b0 0x7ff0d810e510 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.156+0000 7ff0dec34640 1 -- 172.21.15.114:0/4041325084 >> 172.21.15.114:0/4041325084 conn(0x7ff0d806af40 msgr2=0x7ff0d806b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:34.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.163+0000 7ff0dec34640 1 -- 172.21.15.114:0/4041325084 shutdown_connections 2024-09-13T18:33:34.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.168+0000 7ff0dec34640 1 -- 172.21.15.114:0/4041325084 wait complete. 2024-09-13T18:33:34.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.169+0000 7ff0dec34640 1 Processor -- start 2024-09-13T18:33:34.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.169+0000 7ff0dec34640 1 -- start start 2024-09-13T18:33:34.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.169+0000 7ff0dec34640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0d81074b0 0x7ff0d81186e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:34.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.169+0000 7ff0dec34640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0d8107890 0x7ff0d8118c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:34.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.169+0000 7ff0dec34640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0d8119400 0x7ff0d81148c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:34.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.169+0000 7ff0dec34640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d8114f60 con 0x7ff0d8119400 2024-09-13T18:33:34.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.169+0000 7ff0dec34640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d81150d0 con 0x7ff0d8107890 2024-09-13T18:33:34.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.169+0000 7ff0dec34640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d8115290 con 0x7ff0d81074b0 2024-09-13T18:33:34.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.170+0000 7ff0de433640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0d8119400 0x7ff0d81148c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:34.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.170+0000 7ff0de433640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0d8119400 0x7ff0d81148c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:52630/0 (socket says 172.21.15.114:52630) 2024-09-13T18:33:34.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.170+0000 7ff0de433640 1 -- 172.21.15.114:0/216905659 learned_addr learned my addr 172.21.15.114:0/216905659 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:34.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.170+0000 7ff0ddc32640 1 --2- 172.21.15.114:0/216905659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0d81074b0 0x7ff0d81186e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:34.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.173+0000 7ff0dd431640 1 --2- 172.21.15.114:0/216905659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0d8107890 0x7ff0d8118c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:34.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.175+0000 7ff0de433640 1 -- 172.21.15.114:0/216905659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0d81074b0 msgr2=0x7ff0d81186e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:34.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.175+0000 7ff0de433640 1 --2- 172.21.15.114:0/216905659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0d81074b0 0x7ff0d81186e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.175+0000 7ff0de433640 1 -- 172.21.15.114:0/216905659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0d8107890 msgr2=0x7ff0d8118c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:34.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.175+0000 7ff0de433640 1 --2- 172.21.15.114:0/216905659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0d8107890 0x7ff0d8118c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.175+0000 7ff0de433640 1 -- 172.21.15.114:0/216905659 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d001a910 con 0x7ff0d8119400 2024-09-13T18:33:34.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.181+0000 7ff0de433640 1 --2- 172.21.15.114:0/216905659 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0d8119400 0x7ff0d81148c0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7ff0d400e970 tx=0x7ff0d400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:34.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.182+0000 7ff0ceffd640 1 -- 172.21.15.114:0/216905659 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d4009720 con 0x7ff0d8119400 2024-09-13T18:33:34.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.182+0000 7ff0ceffd640 1 -- 172.21.15.114:0/216905659 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff0d40054b0 con 0x7ff0d8119400 2024-09-13T18:33:34.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.182+0000 7ff0dec34640 1 -- 172.21.15.114:0/216905659 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0d8115510 con 0x7ff0d8119400 2024-09-13T18:33:34.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.182+0000 7ff0dec34640 1 -- 172.21.15.114:0/216905659 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0d81a9500 con 0x7ff0d8119400 2024-09-13T18:33:34.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.182+0000 7ff0ceffd640 1 -- 172.21.15.114:0/216905659 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d4004e10 con 0x7ff0d8119400 2024-09-13T18:33:34.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.186+0000 7ff0e2bba740 1 Processor -- start 2024-09-13T18:33:34.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.190+0000 7ff0e2bba740 1 -- start start 2024-09-13T18:33:34.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.190+0000 7ff0e2bba740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561cf7b95970 0x561cf7b95d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:34.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.190+0000 7ff0e2bba740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cf7b96290 0x561cf7ba05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:34.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.190+0000 7ff0e2bba740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cf7ba0b30 0x561cf7ba2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:34.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.190+0000 7ff0e2bba740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cf7affcc0 con 0x561cf7b95970 2024-09-13T18:33:34.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.190+0000 7ff0e2bba740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cf7afd450 con 0x561cf7ba0b30 2024-09-13T18:33:34.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.190+0000 7ff0e2bba740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cf7af8d40 con 0x561cf7b96290 2024-09-13T18:33:34.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.192+0000 7ff0ceffd640 1 -- 172.21.15.114:0/216905659 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7ff0d40104f0 con 0x7ff0d8119400 2024-09-13T18:33:34.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.192+0000 7ff0dd431640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cf7ba0b30 0x561cf7ba2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:34.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.192+0000 7ff0dd431640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561cf7b95970 0x561cf7b95d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:34.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.192+0000 7ff0dd431640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561cf7b95970 0x561cf7b95d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:52646/0 (socket says 172.21.15.114:52646) 2024-09-13T18:33:34.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.192+0000 7ff0dd431640 1 -- 172.21.15.114:0/3452840688 learned_addr learned my addr 172.21.15.114:0/3452840688 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:34.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.193+0000 7ff0dd431640 1 -- 172.21.15.114:0/3452840688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cf7b96290 msgr2=0x561cf7ba05f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:33:34.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.193+0000 7ff0dd431640 1 --2- 172.21.15.114:0/3452840688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cf7b96290 0x561cf7ba05f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.193+0000 7ff0dd431640 1 -- 172.21.15.114:0/3452840688 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561cf7b95970 msgr2=0x561cf7b95d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:34.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.193+0000 7ff0dd431640 1 --2- 172.21.15.114:0/3452840688 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561cf7b95970 0x561cf7b95d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.193+0000 7ff0dd431640 1 -- 172.21.15.114:0/3452840688 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561cf7ac0620 con 0x561cf7ba0b30 2024-09-13T18:33:34.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.194+0000 7ff0ceffd640 1 --2- 172.21.15.114:0/216905659 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff0c40a1410 0x7ff0c40a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:34.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.194+0000 7ff0ceffd640 1 -- 172.21.15.114:0/216905659 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ff0d4014070 con 0x7ff0d8119400 2024-09-13T18:33:34.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.194+0000 7ff0dd431640 1 --2- 172.21.15.114:0/3452840688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cf7ba0b30 0x561cf7ba2f20 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff0c80166b0 tx=0x7ff0c8006ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78633ef595e99732 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:34.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.195+0000 7ff0c3fff640 1 -- 172.21.15.114:0/3452840688 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c8018680 con 0x561cf7ba0b30 2024-09-13T18:33:34.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.195+0000 7ff0c3fff640 1 -- 172.21.15.114:0/3452840688 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff0c8018c70 con 0x561cf7ba0b30 2024-09-13T18:33:34.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.195+0000 7ff0c3fff640 1 -- 172.21.15.114:0/3452840688 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c8017a20 con 0x561cf7ba0b30 2024-09-13T18:33:34.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.198+0000 7ff0e2bba740 1 -- 172.21.15.114:0/3452840688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cf7ba0b30 msgr2=0x561cf7ba2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:34.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.198+0000 7ff0e2bba740 1 --2- 172.21.15.114:0/3452840688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cf7ba0b30 0x561cf7ba2f20 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff0c80166b0 tx=0x7ff0c8006ee0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.202+0000 7ff0e2bba740 1 -- 172.21.15.114:0/3452840688 shutdown_connections 2024-09-13T18:33:34.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.202+0000 7ff0e2bba740 1 --2- 172.21.15.114:0/3452840688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cf7ba0b30 0x561cf7ba2f20 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.202+0000 7ff0e2bba740 1 --2- 172.21.15.114:0/3452840688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cf7b96290 0x561cf7ba05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.202+0000 7ff0e2bba740 1 --2- 172.21.15.114:0/3452840688 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561cf7b95970 0x561cf7b95d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.202+0000 7ff0e2bba740 1 -- 172.21.15.114:0/3452840688 >> 172.21.15.114:0/3452840688 conn(0x561cf7b91ee0 msgr2=0x561cf7b922c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:34.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.203+0000 7ff0ddc32640 1 --2- 172.21.15.114:0/216905659 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff0c40a1410 0x7ff0c40a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:34.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.206+0000 7ff0e2bba740 1 -- 172.21.15.114:0/3452840688 shutdown_connections 2024-09-13T18:33:34.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.211+0000 7ff0e2bba740 1 -- 172.21.15.114:0/3452840688 wait complete. 2024-09-13T18:33:34.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.211+0000 7ff0e2bba740 1 -- start start 2024-09-13T18:33:34.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.211+0000 7ff0e2bba740 1 Processor -- start 2024-09-13T18:33:34.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.213+0000 7ff0ddc32640 1 --2- 172.21.15.114:0/216905659 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff0c40a1410 0x7ff0c40a38d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0000f80 tx=0x7ff0d004c040 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:34.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.213+0000 7ff0e2bba740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cf7b95970 0x561cf7ba00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:34.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.213+0000 7ff0e2bba740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cf7b96290 0x561cf7b9d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:34.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.213+0000 7ff0e2bba740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561cf7ba0b30 0x561cf7b9d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:34.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.213+0000 7ff0e2bba740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cf7ab7180 con 0x561cf7ba0b30 2024-09-13T18:33:34.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.213+0000 7ff0e2bba740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cf77d1430 con 0x561cf7b95970 2024-09-13T18:33:34.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.213+0000 7ff0e2bba740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cf7a7c5e0 con 0x561cf7b96290 2024-09-13T18:33:34.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.213+0000 7ff0e2bba740 10 client.0 _ll_register_callbacks cb 0x561cf7ae8aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:33:34.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.214+0000 7ff0ddc32640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cf7b95970 0x561cf7ba00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:34.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.214+0000 7ff0ddc32640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cf7b95970 0x561cf7ba00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:40584/0 (socket says 172.21.15.114:40584) 2024-09-13T18:33:34.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.214+0000 7ff0ddc32640 1 -- 172.21.15.114:0/547872349 learned_addr learned my addr 172.21.15.114:0/547872349 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:34.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.215+0000 7ff0dd431640 1 --2- 172.21.15.114:0/547872349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cf7b96290 0x561cf7b9d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:34.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.216+0000 7ff0dd431640 1 -- 172.21.15.114:0/547872349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cf7b95970 msgr2=0x561cf7ba00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:34.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.216+0000 7ff0dd431640 1 --2- 172.21.15.114:0/547872349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cf7b95970 0x561cf7ba00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.216+0000 7ff0dd431640 1 -- 172.21.15.114:0/547872349 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561cf7ba0b30 msgr2=0x561cf7b9d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:34.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.216+0000 7ff0dd431640 1 --2- 172.21.15.114:0/547872349 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561cf7ba0b30 0x561cf7b9d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.216+0000 7ff0dd431640 1 -- 172.21.15.114:0/547872349 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0c8016390 con 0x561cf7b96290 2024-09-13T18:33:34.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.217+0000 7ff0dd431640 1 --2- 172.21.15.114:0/547872349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cf7b96290 0x561cf7b9d410 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8001eb0 tx=0x7ff0c80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55489f24d06fd12 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:34.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.217+0000 7ff0c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:33:34.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.218+0000 7ff0c17fa640 1 -- 172.21.15.114:0/547872349 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c80177a0 con 0x561cf7b96290 2024-09-13T18:33:34.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.218+0000 7ff0e2bba740 1 -- 172.21.15.114:0/547872349 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561cf7b22b80 con 0x561cf7b96290 2024-09-13T18:33:34.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.218+0000 7ff0c17fa640 1 -- 172.21.15.114:0/547872349 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ff0c8017900 con 0x561cf7b96290 2024-09-13T18:33:34.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.218+0000 7ff0c17fa640 1 -- 172.21.15.114:0/547872349 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0c8017ac0 con 0x561cf7b96290 2024-09-13T18:33:34.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.218+0000 7ff0e2bba740 10 client.27916 fetch_fsmap learned FSMap version 37 2024-09-13T18:33:34.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.218+0000 7ff0e2bba740 1 -- 172.21.15.114:0/547872349 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x561cf7af73c0 con 0x561cf7b96290 2024-09-13T18:33:34.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.221+0000 7ff0c17fa640 1 -- 172.21.15.114:0/547872349 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561cf7b22b80 con 0x561cf7b96290 2024-09-13T18:33:34.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.221+0000 7ff0c17fa640 1 -- 172.21.15.114:0/547872349 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 37) v1 ==== 7490+0+0 (secure 0 0 0) 0x7ff0c8012070 con 0x561cf7b96290 2024-09-13T18:33:34.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.221+0000 7ff0e2bba740 10 client.27916 fetch_fsmap finished waiting for FSMap version 37 2024-09-13T18:33:34.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.221+0000 7ff0e2bba740 10 client.27916 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-13T18:33:34.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.221+0000 7ff0e2bba740 20 client.27916 populate_metadata read hostname 'smithi114' 2024-09-13T18:33:34.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.221+0000 7ff0e2bba740 1 --2- 172.21.15.114:0/547872349 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x561cf7bb0440 0x561cf7bb2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:34.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.221+0000 7ff0e2bba740 4 client.27916 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:33:34.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.221+0000 7ff0e2bba740 1 -- 172.21.15.114:0/547872349 --> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561cf7aeb1f0 con 0x561cf7bb0440 2024-09-13T18:33:34.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.224+0000 7ff0ddc32640 1 --2- 172.21.15.114:0/547872349 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x561cf7bb0440 0x561cf7bb2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:34.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.236+0000 7ff0ddc32640 1 --2- 172.21.15.114:0/547872349 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x561cf7bb0440 0x561cf7bb2860 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7c30f8dcf582ce84 server_cookie=5a88badaa32577c5 in_seq=0 out_seq=0 2024-09-13T18:33:34.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.237+0000 7ff0c17fa640 10 client.27916 ms_handle_connect on v2:172.21.15.134:6834/1150003275 2024-09-13T18:33:34.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.238+0000 7ff0c17fa640 1 -- 172.21.15.114:0/547872349 <== mds.3 v2:172.21.15.134:6834/1150003275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x561cf7aeb1f0 con 0x561cf7bb0440 2024-09-13T18:33:34.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.238+0000 7ff0c17fa640 10 client.27916 handle_command_reply: tid=0 2024-09-13T18:33:34.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.238+0000 7ff0e2bba740 1 client.27916 shutdown 2024-09-13T18:33:34.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.249+0000 7ff0e2bba740 1 -- 172.21.15.114:0/547872349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cf7b96290 msgr2=0x561cf7b9d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:34.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.249+0000 7ff0e2bba740 1 --2- 172.21.15.114:0/547872349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cf7b96290 0x561cf7b9d410 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8001eb0 tx=0x7ff0c80046c0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.250+0000 7ff0e2bba740 1 -- 172.21.15.114:0/547872349 shutdown_connections 2024-09-13T18:33:34.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.250+0000 7ff0e2bba740 1 --2- 172.21.15.114:0/547872349 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x561cf7bb0440 0x561cf7bb2860 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.250+0000 7ff0e2bba740 1 --2- 172.21.15.114:0/547872349 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561cf7ba0b30 0x561cf7b9d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.250+0000 7ff0e2bba740 1 --2- 172.21.15.114:0/547872349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cf7b96290 0x561cf7b9d410 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.250+0000 7ff0e2bba740 1 --2- 172.21.15.114:0/547872349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cf7b95970 0x561cf7ba00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.250+0000 7ff0e2bba740 1 -- 172.21.15.114:0/547872349 >> 172.21.15.114:0/547872349 conn(0x561cf7b91ee0 msgr2=0x561cf7b91230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:34.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.250+0000 7ff0c17fa640 0 client.27916 ms_handle_reset on v2:172.21.15.134:6834/1150003275 2024-09-13T18:33:34.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.256+0000 7ff0e2bba740 1 -- 172.21.15.114:0/547872349 shutdown_connections 2024-09-13T18:33:34.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.257+0000 7ff0e2bba740 1 -- 172.21.15.114:0/547872349 wait complete. 2024-09-13T18:33:34.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.259+0000 7ff0e2bba740 20 client.27916 trim_cache size 0 max 16384 2024-09-13T18:33:34.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.300+0000 7ff0e2bba740 1 Processor -- start 2024-09-13T18:33:34.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.300+0000 7ff0e2bba740 1 -- start start 2024-09-13T18:33:34.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.300+0000 7ff0e2bba740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cf7b95970 0x561cf7ba7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:34.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.301+0000 7ff0e2bba740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cf7b96290 0x561cf7ba7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:34.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.301+0000 7ff0e2bba740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561cf7ba0b30 0x561cf7b9bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:34.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.301+0000 7ff0ddc32640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cf7b95970 0x561cf7ba7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:34.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.301+0000 7ff0ddc32640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cf7b95970 0x561cf7ba7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:59594/0 (socket says 172.21.15.114:59594) 2024-09-13T18:33:34.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.301+0000 7ff0dd431640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cf7b96290 0x561cf7ba7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:34.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.301+0000 7ff0dd431640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cf7b96290 0x561cf7ba7dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:40592/0 (socket says 172.21.15.114:40592) 2024-09-13T18:33:34.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.301+0000 7ff0dd431640 1 -- 172.21.15.114:0/69340892 learned_addr learned my addr 172.21.15.114:0/69340892 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:34.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.301+0000 7ff0e2bba740 1 -- 172.21.15.114:0/69340892 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cf7b96b10 con 0x561cf7ba0b30 2024-09-13T18:33:34.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.301+0000 7ff0e2bba740 1 -- 172.21.15.114:0/69340892 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cf7b95760 con 0x561cf7b96290 2024-09-13T18:33:34.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.301+0000 7ff0e2bba740 1 -- 172.21.15.114:0/69340892 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cf7b9b940 con 0x561cf7b95970 2024-09-13T18:33:34.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.305+0000 7ff0de433640 1 --2- 172.21.15.114:0/69340892 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561cf7ba0b30 0x561cf7b9bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:34.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.307+0000 7ff0de433640 1 -- 172.21.15.114:0/69340892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cf7b95970 msgr2=0x561cf7ba7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:34.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.307+0000 7ff0de433640 1 --2- 172.21.15.114:0/69340892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cf7b95970 0x561cf7ba7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.307+0000 7ff0de433640 1 -- 172.21.15.114:0/69340892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cf7b96290 msgr2=0x561cf7ba7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:34.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.307+0000 7ff0de433640 1 --2- 172.21.15.114:0/69340892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cf7b96290 0x561cf7ba7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.307+0000 7ff0de433640 1 -- 172.21.15.114:0/69340892 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561cf7a99770 con 0x561cf7ba0b30 2024-09-13T18:33:34.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.308+0000 7ff0de433640 1 --2- 172.21.15.114:0/69340892 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561cf7ba0b30 0x561cf7b9bcc0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7ff0d40ca710 tx=0x7ff0d4096540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7515e8360e609dd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:34.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.308+0000 7ff0ccff9640 1 -- 172.21.15.114:0/69340892 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d409df00 con 0x561cf7ba0b30 2024-09-13T18:33:34.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.309+0000 7ff0ccff9640 1 -- 172.21.15.114:0/69340892 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff0d40a2550 con 0x561cf7ba0b30 2024-09-13T18:33:34.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.309+0000 7ff0ccff9640 1 -- 172.21.15.114:0/69340892 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d40ca990 con 0x561cf7ba0b30 2024-09-13T18:33:34.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.312+0000 7ff0e2bba740 1 -- 172.21.15.114:0/69340892 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561cf7ba0b30 msgr2=0x561cf7b9bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:34.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.312+0000 7ff0e2bba740 1 --2- 172.21.15.114:0/69340892 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561cf7ba0b30 0x561cf7b9bcc0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7ff0d40ca710 tx=0x7ff0d4096540 comp rx=0 tx=0).stop 2024-09-13T18:33:34.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.312+0000 7ff0e2bba740 1 -- 172.21.15.114:0/69340892 shutdown_connections 2024-09-13T18:33:34.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.312+0000 7ff0e2bba740 1 --2- 172.21.15.114:0/69340892 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561cf7ba0b30 0x561cf7b9bcc0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.312+0000 7ff0e2bba740 1 --2- 172.21.15.114:0/69340892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cf7b96290 0x561cf7ba7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.312+0000 7ff0e2bba740 1 --2- 172.21.15.114:0/69340892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cf7b95970 0x561cf7ba7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.312+0000 7ff0e2bba740 1 -- 172.21.15.114:0/69340892 >> 172.21.15.114:0/69340892 conn(0x561cf7b91ee0 msgr2=0x561cf7b922c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:34.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.316+0000 7ff0e2bba740 1 -- 172.21.15.114:0/69340892 shutdown_connections 2024-09-13T18:33:34.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.316+0000 7ff0e2bba740 1 -- 172.21.15.114:0/69340892 wait complete. 2024-09-13T18:33:34.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.316+0000 7ff0e2bba740 1 -- start start 2024-09-13T18:33:34.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.316+0000 7ff0e2bba740 1 Processor -- start 2024-09-13T18:33:34.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.317+0000 7ff0e2bba740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561cf7b95970 0x561cf7ba7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:34.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.317+0000 7ff0e2bba740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cf7b96290 0x561cf7ba3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:34.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.317+0000 7ff0e2bba740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cf7ba0b30 0x561cf7ba43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:34.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.317+0000 7ff0dd431640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cf7b96290 0x561cf7ba3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:34.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.317+0000 7ff0dd431640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cf7b96290 0x561cf7ba3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:59606/0 (socket says 172.21.15.114:59606) 2024-09-13T18:33:34.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.317+0000 7ff0de433640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cf7ba0b30 0x561cf7ba43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:34.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.317+0000 7ff0de433640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cf7ba0b30 0x561cf7ba43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:40600/0 (socket says 172.21.15.114:40600) 2024-09-13T18:33:34.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.318+0000 7ff0dd431640 1 -- 172.21.15.114:0/1137608765 learned_addr learned my addr 172.21.15.114:0/1137608765 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:34.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.318+0000 7ff0e2bba740 1 -- 172.21.15.114:0/1137608765 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cf7ba48e0 con 0x561cf7b95970 2024-09-13T18:33:34.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.318+0000 7ff0e2bba740 1 -- 172.21.15.114:0/1137608765 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cf7ba4a50 con 0x561cf7ba0b30 2024-09-13T18:33:34.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.318+0000 7ff0e2bba740 1 -- 172.21.15.114:0/1137608765 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cf7ba4c10 con 0x561cf7b96290 2024-09-13T18:33:34.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.318+0000 7ff0e2bba740 10 client.0 _ll_register_callbacks cb 0x7ff0c800c820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:33:34.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.318+0000 7ff0ddc32640 1 --2- 172.21.15.114:0/1137608765 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561cf7b95970 0x561cf7ba7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:34.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.319+0000 7ff0dd431640 1 -- 172.21.15.114:0/1137608765 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cf7ba0b30 msgr2=0x561cf7ba43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:34.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.319+0000 7ff0dd431640 1 --2- 172.21.15.114:0/1137608765 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cf7ba0b30 0x561cf7ba43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.319+0000 7ff0dd431640 1 -- 172.21.15.114:0/1137608765 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561cf7b95970 msgr2=0x561cf7ba7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:34.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.319+0000 7ff0dd431640 1 --2- 172.21.15.114:0/1137608765 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561cf7b95970 0x561cf7ba7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.319+0000 7ff0dd431640 1 -- 172.21.15.114:0/1137608765 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0d81a9500 con 0x561cf7b96290 2024-09-13T18:33:34.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.319+0000 7ff0dd431640 1 --2- 172.21.15.114:0/1137608765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cf7b96290 0x561cf7ba3e60 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8006aa0 tx=0x7ff0c800c1b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=237c31ac4f52076a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:34.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.319+0000 7ff0c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:33:34.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.320+0000 7ff0c17fa640 1 -- 172.21.15.114:0/1137608765 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c8023440 con 0x561cf7b96290 2024-09-13T18:33:34.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.320+0000 7ff0e2bba740 1 -- 172.21.15.114:0/1137608765 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff0c8018c70 con 0x561cf7b96290 2024-09-13T18:33:34.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.320+0000 7ff0c17fa640 1 -- 172.21.15.114:0/1137608765 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ff0c8000e40 con 0x561cf7b96290 2024-09-13T18:33:34.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.320+0000 7ff0c17fa640 1 -- 172.21.15.114:0/1137608765 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0c802c440 con 0x561cf7b96290 2024-09-13T18:33:34.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.321+0000 7ff0e2bba740 10 client.27922 fetch_fsmap learned FSMap version 37 2024-09-13T18:33:34.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.321+0000 7ff0e2bba740 1 -- 172.21.15.114:0/1137608765 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x561cf7abee80 con 0x561cf7b96290 2024-09-13T18:33:34.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.322+0000 7ff0c17fa640 1 -- 172.21.15.114:0/1137608765 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff0c8018c70 con 0x561cf7b96290 2024-09-13T18:33:34.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.322+0000 7ff0c17fa640 1 -- 172.21.15.114:0/1137608765 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 37) v1 ==== 7490+0+0 (secure 0 0 0) 0x7ff0c803f070 con 0x561cf7b96290 2024-09-13T18:33:34.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.322+0000 7ff0e2bba740 10 client.27922 fetch_fsmap finished waiting for FSMap version 37 2024-09-13T18:33:34.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.322+0000 7ff0e2bba740 10 client.27922 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-13T18:33:34.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.322+0000 7ff0e2bba740 20 client.27922 populate_metadata read hostname 'smithi114' 2024-09-13T18:33:34.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.322+0000 7ff0e2bba740 1 --2- 172.21.15.114:0/1137608765 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x561cf7c07ad0 0x561cf7c09ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:34.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.322+0000 7ff0e2bba740 4 client.27922 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:33:34.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.322+0000 7ff0e2bba740 1 -- 172.21.15.114:0/1137608765 --> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561cf77ec4e0 con 0x561cf7c07ad0 2024-09-13T18:33:34.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.323+0000 7ff0ddc32640 1 --2- 172.21.15.114:0/1137608765 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x561cf7c07ad0 0x561cf7c09ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:34.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.324+0000 7ff0ddc32640 1 --2- 172.21.15.114:0/1137608765 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x561cf7c07ad0 0x561cf7c09ef0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ab2e246d17fd0b49 server_cookie=13d65736ea96b0e6 in_seq=0 out_seq=0 2024-09-13T18:33:34.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.324+0000 7ff0c17fa640 10 client.27922 ms_handle_connect on v2:172.21.15.134:6834/1150003275 2024-09-13T18:33:34.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.324+0000 7ff0c17fa640 1 -- 172.21.15.114:0/1137608765 <== mds.3 v2:172.21.15.134:6834/1150003275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561cf77ec4e0 con 0x561cf7c07ad0 2024-09-13T18:33:34.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.324+0000 7ff0c17fa640 10 client.27922 handle_command_reply: tid=0 2024-09-13T18:33:34.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.324+0000 7ff0e2bba740 1 client.27922 shutdown 2024-09-13T18:33:34.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.327+0000 7ff0e2bba740 1 -- 172.21.15.114:0/1137608765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cf7b96290 msgr2=0x561cf7ba3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:34.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.327+0000 7ff0e2bba740 1 --2- 172.21.15.114:0/1137608765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cf7b96290 0x561cf7ba3e60 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8006aa0 tx=0x7ff0c800c1b0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.327+0000 7ff0e2bba740 1 -- 172.21.15.114:0/1137608765 shutdown_connections 2024-09-13T18:33:34.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.327+0000 7ff0e2bba740 1 --2- 172.21.15.114:0/1137608765 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x561cf7c07ad0 0x561cf7c09ef0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.327+0000 7ff0e2bba740 1 --2- 172.21.15.114:0/1137608765 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561cf7ba0b30 0x561cf7ba43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.327+0000 7ff0e2bba740 1 --2- 172.21.15.114:0/1137608765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561cf7b96290 0x561cf7ba3e60 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.327+0000 7ff0e2bba740 1 --2- 172.21.15.114:0/1137608765 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561cf7b95970 0x561cf7ba7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.327+0000 7ff0e2bba740 1 -- 172.21.15.114:0/1137608765 >> 172.21.15.114:0/1137608765 conn(0x561cf7b91ee0 msgr2=0x561cf7b9bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:34.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.327+0000 7ff0c17fa640 0 client.27922 ms_handle_reset on v2:172.21.15.134:6834/1150003275 2024-09-13T18:33:34.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.327+0000 7ff0e2bba740 1 -- 172.21.15.114:0/1137608765 shutdown_connections 2024-09-13T18:33:34.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.327+0000 7ff0e2bba740 1 -- 172.21.15.114:0/1137608765 wait complete. 2024-09-13T18:33:34.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.328+0000 7ff0e2bba740 20 client.27922 trim_cache size 0 max 16384 2024-09-13T18:33:34.331 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:33:34.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.328+0000 7ff0dec34640 1 -- 172.21.15.114:0/216905659 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff0c40a1410 msgr2=0x7ff0c40a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:34.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.328+0000 7ff0dec34640 1 --2- 172.21.15.114:0/216905659 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff0c40a1410 0x7ff0c40a38d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0000f80 tx=0x7ff0d004c040 comp rx=0 tx=0).stop 2024-09-13T18:33:34.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.329+0000 7ff0dec34640 1 -- 172.21.15.114:0/216905659 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0d8119400 msgr2=0x7ff0d81148c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:34.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.329+0000 7ff0dec34640 1 --2- 172.21.15.114:0/216905659 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0d8119400 0x7ff0d81148c0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7ff0d400e970 tx=0x7ff0d400ed70 comp rx=0 tx=0).stop 2024-09-13T18:33:34.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.329+0000 7ff0dec34640 1 -- 172.21.15.114:0/216905659 shutdown_connections 2024-09-13T18:33:34.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.329+0000 7ff0dec34640 1 --2- 172.21.15.114:0/216905659 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff0c40a1410 0x7ff0c40a38d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.329+0000 7ff0dec34640 1 --2- 172.21.15.114:0/216905659 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0d8119400 0x7ff0d81148c0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.329+0000 7ff0dec34640 1 --2- 172.21.15.114:0/216905659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0d8107890 0x7ff0d8118c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.329+0000 7ff0dec34640 1 --2- 172.21.15.114:0/216905659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0d81074b0 0x7ff0d81186e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.329+0000 7ff0dec34640 1 -- 172.21.15.114:0/216905659 >> 172.21.15.114:0/216905659 conn(0x7ff0d806af40 msgr2=0x7ff0d806ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:34.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.333+0000 7ff0dec34640 1 -- 172.21.15.114:0/216905659 shutdown_connections 2024-09-13T18:33:34.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.334+0000 7ff0dec34640 1 -- 172.21.15.114:0/216905659 wait complete. 2024-09-13T18:33:34.368 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-09-13T18:33:34.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:34 smithi134 ceph-mon[34283]: pgmap v345: 129 pgs: 129 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 30 MiB/s wr, 647 op/s 2024-09-13T18:33:34.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:34 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/3925352619' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:33:34.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.618+0000 7fa59bfff640 1 -- 172.21.15.114:0/3161041588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa59c107dd0 msgr2=0x7fa59c112830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:34.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.618+0000 7fa59bfff640 1 --2- 172.21.15.114:0/3161041588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa59c107dd0 0x7fa59c112830 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa588009890 tx=0x7fa58802a390 comp rx=0 tx=0).stop 2024-09-13T18:33:34.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.619+0000 7fa59bfff640 1 -- 172.21.15.114:0/3161041588 shutdown_connections 2024-09-13T18:33:34.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.619+0000 7fa59bfff640 1 --2- 172.21.15.114:0/3161041588 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa59c112d70 0x7fa59c115160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.619+0000 7fa59bfff640 1 --2- 172.21.15.114:0/3161041588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa59c107dd0 0x7fa59c112830 secure :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa588009890 tx=0x7fa58802a390 comp rx=0 tx=0).stop 2024-09-13T18:33:34.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.619+0000 7fa59bfff640 1 --2- 172.21.15.114:0/3161041588 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa59c1074b0 0x7fa59c107890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.619+0000 7fa59bfff640 1 -- 172.21.15.114:0/3161041588 >> 172.21.15.114:0/3161041588 conn(0x7fa59c06af40 msgr2=0x7fa59c06b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:34.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.620+0000 7fa59bfff640 1 -- 172.21.15.114:0/3161041588 shutdown_connections 2024-09-13T18:33:34.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.622+0000 7fa59bfff640 1 -- 172.21.15.114:0/3161041588 wait complete. 2024-09-13T18:33:34.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.623+0000 7fa59bfff640 1 Processor -- start 2024-09-13T18:33:34.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.624+0000 7fa59bfff640 1 -- start start 2024-09-13T18:33:34.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.624+0000 7fa59bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa59c1074b0 0x7fa59c1a4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:34.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.624+0000 7fa59bfff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa59c112d70 0x7fa59c1a4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:34.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.624+0000 7fa59bfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa59c19ea20 0x7fa59c19eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:34.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.624+0000 7fa59bfff640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59c19f4a0 con 0x7fa59c112d70 2024-09-13T18:33:34.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.625+0000 7fa59bfff640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59c19f610 con 0x7fa59c19ea20 2024-09-13T18:33:34.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.625+0000 7fa59bfff640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59c19f7d0 con 0x7fa59c1074b0 2024-09-13T18:33:34.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.625+0000 7fa59b7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa59c19ea20 0x7fa59c19eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:34.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.625+0000 7fa59b7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa59c19ea20 0x7fa59c19eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:40624/0 (socket says 172.21.15.114:40624) 2024-09-13T18:33:34.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.625+0000 7fa59b7fe640 1 -- 172.21.15.114:0/3845132950 learned_addr learned my addr 172.21.15.114:0/3845132950 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:34.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.626+0000 7fa59a7fc640 1 --2- 172.21.15.114:0/3845132950 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa59c112d70 0x7fa59c1a4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:34.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.626+0000 7fa59b7fe640 1 -- 172.21.15.114:0/3845132950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa59c1074b0 msgr2=0x7fa59c1a4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:34.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.626+0000 7fa59affd640 1 --2- 172.21.15.114:0/3845132950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa59c1074b0 0x7fa59c1a4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:34.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.626+0000 7fa59b7fe640 1 --2- 172.21.15.114:0/3845132950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa59c1074b0 0x7fa59c1a4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.626+0000 7fa59b7fe640 1 -- 172.21.15.114:0/3845132950 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa59c112d70 msgr2=0x7fa59c1a4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:34.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.626+0000 7fa59b7fe640 1 --2- 172.21.15.114:0/3845132950 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa59c112d70 0x7fa59c1a4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.626+0000 7fa59b7fe640 1 -- 172.21.15.114:0/3845132950 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa588009490 con 0x7fa59c19ea20 2024-09-13T18:33:34.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.627+0000 7fa59affd640 1 --2- 172.21.15.114:0/3845132950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa59c1074b0 0x7fa59c1a4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:33:34.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.629+0000 7fa59a7fc640 1 --2- 172.21.15.114:0/3845132950 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa59c112d70 0x7fa59c1a4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:33:34.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.630+0000 7fa59b7fe640 1 --2- 172.21.15.114:0/3845132950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa59c19ea20 0x7fa59c19eea0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa59400eda0 tx=0x7fa59400c6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:34.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.631+0000 7fa593fff640 1 -- 172.21.15.114:0/3845132950 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa594003b70 con 0x7fa59c19ea20 2024-09-13T18:33:34.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.631+0000 7fa593fff640 1 -- 172.21.15.114:0/3845132950 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa594009860 con 0x7fa59c19ea20 2024-09-13T18:33:34.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.631+0000 7fa593fff640 1 -- 172.21.15.114:0/3845132950 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5940177b0 con 0x7fa59c19ea20 2024-09-13T18:33:34.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.632+0000 7fa59bfff640 1 -- 172.21.15.114:0/3845132950 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa59c108620 con 0x7fa59c19ea20 2024-09-13T18:33:34.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.632+0000 7fa59bfff640 1 -- 172.21.15.114:0/3845132950 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa59c108ab0 con 0x7fa59c19ea20 2024-09-13T18:33:34.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.660+0000 7fa5a4016740 1 Processor -- start 2024-09-13T18:33:34.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.660+0000 7fa5a4016740 1 -- start start 2024-09-13T18:33:34.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.661+0000 7fa5a4016740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577ffbcc180 0x5577ffbcc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:34.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.661+0000 7fa5a4016740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577ffbccaa0 0x5577ffbd6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:34.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.661+0000 7fa5a4016740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5577ffbd7340 0x5577ffbd9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:34.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.661+0000 7fa5a4016740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ffb2dcc0 con 0x5577ffbcc180 2024-09-13T18:33:34.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.661+0000 7fa5a4016740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ffb2b450 con 0x5577ffbd7340 2024-09-13T18:33:34.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.661+0000 7fa5a4016740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ffb26d40 con 0x5577ffbccaa0 2024-09-13T18:33:34.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.661+0000 7fa593fff640 1 -- 172.21.15.114:0/3845132950 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fa59400a040 con 0x7fa59c19ea20 2024-09-13T18:33:34.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.662+0000 7fa593fff640 1 --2- 172.21.15.114:0/3845132950 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa5800a1410 0x7fa5800a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:34.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.662+0000 7fa59b7fe640 1 --2- 172.21.15.114:0/3845132950 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa5800a1410 0x7fa5800a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:34.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.662+0000 7fa59a7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5577ffbd7340 0x5577ffbd9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:34.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.662+0000 7fa59a7fc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577ffbcc180 0x5577ffbcc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:34.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.662+0000 7fa59a7fc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577ffbcc180 0x5577ffbcc560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:52714/0 (socket says 172.21.15.114:52714) 2024-09-13T18:33:34.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.662+0000 7fa59a7fc640 1 -- 172.21.15.114:0/3972472594 learned_addr learned my addr 172.21.15.114:0/3972472594 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:34.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.662+0000 7fa59affd640 1 --2- 172.21.15.114:0/3972472594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577ffbccaa0 0x5577ffbd6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:34.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.664+0000 7fa593fff640 1 -- 172.21.15.114:0/3845132950 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fa5940cd330 con 0x7fa59c19ea20 2024-09-13T18:33:34.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.665+0000 7fa59affd640 1 -- 172.21.15.114:0/3972472594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5577ffbd7340 msgr2=0x5577ffbd9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:34.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.665+0000 7fa59affd640 1 --2- 172.21.15.114:0/3972472594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5577ffbd7340 0x5577ffbd9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.665+0000 7fa59affd640 1 -- 172.21.15.114:0/3972472594 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577ffbcc180 msgr2=0x5577ffbcc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:34.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.665+0000 7fa59affd640 1 --2- 172.21.15.114:0/3972472594 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577ffbcc180 0x5577ffbcc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.665+0000 7fa59affd640 1 -- 172.21.15.114:0/3972472594 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577ffaee620 con 0x5577ffbccaa0 2024-09-13T18:33:34.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.669+0000 7fa59affd640 1 --2- 172.21.15.114:0/3972472594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577ffbccaa0 0x5577ffbd6e00 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa58c00a940 tx=0x7fa58c00ae00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8a97c8c9bbcb0b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:34.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.670+0000 7fa5917fa640 1 -- 172.21.15.114:0/3972472594 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c00cc00 con 0x5577ffbccaa0 2024-09-13T18:33:34.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.670+0000 7fa5917fa640 1 -- 172.21.15.114:0/3972472594 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa58c00cd60 con 0x5577ffbccaa0 2024-09-13T18:33:34.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.671+0000 7fa5a4016740 1 -- 172.21.15.114:0/3972472594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577ffbccaa0 msgr2=0x5577ffbd6e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:34.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.671+0000 7fa5a4016740 1 --2- 172.21.15.114:0/3972472594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577ffbccaa0 0x5577ffbd6e00 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa58c00a940 tx=0x7fa58c00ae00 comp rx=0 tx=0).stop 2024-09-13T18:33:34.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.679+0000 7fa5a4016740 1 -- 172.21.15.114:0/3972472594 shutdown_connections 2024-09-13T18:33:34.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.679+0000 7fa5a4016740 1 --2- 172.21.15.114:0/3972472594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5577ffbd7340 0x5577ffbd9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.679+0000 7fa5a4016740 1 --2- 172.21.15.114:0/3972472594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577ffbccaa0 0x5577ffbd6e00 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.679+0000 7fa5a4016740 1 --2- 172.21.15.114:0/3972472594 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577ffbcc180 0x5577ffbcc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.679+0000 7fa5a4016740 1 -- 172.21.15.114:0/3972472594 >> 172.21.15.114:0/3972472594 conn(0x5577ffbc86f0 msgr2=0x5577ffbc8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:34.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.681+0000 7fa59b7fe640 1 --2- 172.21.15.114:0/3845132950 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa5800a1410 0x7fa5800a38d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa594003fd0 tx=0x7fa5940c4690 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:34.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.682+0000 7fa5a4016740 1 -- 172.21.15.114:0/3972472594 shutdown_connections 2024-09-13T18:33:34.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.686+0000 7fa5a4016740 1 -- 172.21.15.114:0/3972472594 wait complete. 2024-09-13T18:33:34.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.686+0000 7fa5a4016740 1 -- start start 2024-09-13T18:33:34.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.686+0000 7fa5a4016740 1 Processor -- start 2024-09-13T18:33:34.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.687+0000 7fa5a4016740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577ffbcc180 0x5577ffbc78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:34.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.687+0000 7fa5a4016740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5577ffbccaa0 0x5577ffbc4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:34.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.687+0000 7fa5a4016740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577ffbd7340 0x5577ffbc5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:34.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.687+0000 7fa5a4016740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ffae5180 con 0x5577ffbd7340 2024-09-13T18:33:34.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.687+0000 7fa5a4016740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ff7ff430 con 0x5577ffbccaa0 2024-09-13T18:33:34.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.687+0000 7fa5a4016740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ffaaa5e0 con 0x5577ffbcc180 2024-09-13T18:33:34.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.687+0000 7fa5a4016740 10 client.0 _ll_register_callbacks cb 0x5577ffb16aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:33:34.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.689+0000 7fa59a7fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577ffbcc180 0x5577ffbc78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:34.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.689+0000 7fa59a7fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577ffbcc180 0x5577ffbc78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:59652/0 (socket says 172.21.15.114:59652) 2024-09-13T18:33:34.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.689+0000 7fa59a7fc640 1 -- 172.21.15.114:0/500461525 learned_addr learned my addr 172.21.15.114:0/500461525 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:34.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.689+0000 7fa59a7fc640 1 --2- 172.21.15.114:0/500461525 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577ffbd7340 0x5577ffbc5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:34.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:34 smithi204 ceph-mon[34777]: pgmap v345: 129 pgs: 129 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 30 MiB/s wr, 647 op/s 2024-09-13T18:33:34.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:34 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/3925352619' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:33:34.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.690+0000 7fa59a7fc640 1 -- 172.21.15.114:0/500461525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5577ffbccaa0 msgr2=0x5577ffbc4c20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:33:34.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.690+0000 7fa59a7fc640 1 --2- 172.21.15.114:0/500461525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5577ffbccaa0 0x5577ffbc4c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.690+0000 7fa59a7fc640 1 -- 172.21.15.114:0/500461525 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577ffbd7340 msgr2=0x5577ffbc5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:34.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.690+0000 7fa59a7fc640 1 --2- 172.21.15.114:0/500461525 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577ffbd7340 0x5577ffbc5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.690+0000 7fa59a7fc640 1 -- 172.21.15.114:0/500461525 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa58c00a570 con 0x5577ffbcc180 2024-09-13T18:33:34.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.691+0000 7fa59a7fc640 1 --2- 172.21.15.114:0/500461525 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577ffbcc180 0x5577ffbc78b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa588009890 tx=0x7fa58802c040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c2553cd1cae7b8e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:34.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.693+0000 7fa57e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:33:34.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.696+0000 7fa57e7fc640 1 -- 172.21.15.114:0/500461525 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa588057070 con 0x5577ffbcc180 2024-09-13T18:33:34.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.696+0000 7fa57e7fc640 1 -- 172.21.15.114:0/500461525 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fa588004d00 con 0x5577ffbcc180 2024-09-13T18:33:34.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.696+0000 7fa5a4016740 1 -- 172.21.15.114:0/500461525 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5577ffb50b80 con 0x5577ffbcc180 2024-09-13T18:33:34.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.699+0000 7fa57e7fc640 1 -- 172.21.15.114:0/500461525 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5880055f0 con 0x5577ffbcc180 2024-09-13T18:33:34.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.700+0000 7fa5a4016740 10 client.27931 fetch_fsmap learned FSMap version 37 2024-09-13T18:33:34.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.700+0000 7fa5a4016740 1 -- 172.21.15.114:0/500461525 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x5577ffb253c0 con 0x5577ffbcc180 2024-09-13T18:33:34.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.703+0000 7fa57e7fc640 1 -- 172.21.15.114:0/500461525 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa588062030 con 0x5577ffbcc180 2024-09-13T18:33:34.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.704+0000 7fa57e7fc640 1 -- 172.21.15.114:0/500461525 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 37) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fa58805b020 con 0x5577ffbcc180 2024-09-13T18:33:34.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.705+0000 7fa5a4016740 10 client.27931 fetch_fsmap finished waiting for FSMap version 37 2024-09-13T18:33:34.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.705+0000 7fa5a4016740 10 client.27931 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-13T18:33:34.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.705+0000 7fa5a4016740 20 client.27931 populate_metadata read hostname 'smithi114' 2024-09-13T18:33:34.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.705+0000 7fa5a4016740 1 --2- 172.21.15.114:0/500461525 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x5577ffbde510 0x5577ffbe0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:34.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.705+0000 7fa5a4016740 4 client.27931 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:33:34.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.705+0000 7fa5a4016740 1 -- 172.21.15.114:0/500461525 --> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5577ffb191f0 con 0x5577ffbde510 2024-09-13T18:33:34.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.706+0000 7fa59affd640 1 --2- 172.21.15.114:0/500461525 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x5577ffbde510 0x5577ffbe0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:34.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.709+0000 7fa59affd640 1 --2- 172.21.15.114:0/500461525 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x5577ffbde510 0x5577ffbe0930 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8a26d0a9cc0166bf server_cookie=4c7de3f5080b98e1 in_seq=0 out_seq=0 2024-09-13T18:33:34.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.710+0000 7fa57e7fc640 10 client.27931 ms_handle_connect on v2:172.21.15.204:6838/344883912 2024-09-13T18:33:34.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.710+0000 7fa57e7fc640 1 -- 172.21.15.114:0/500461525 <== mds.2 v2:172.21.15.204:6838/344883912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x5577ffb191f0 con 0x5577ffbde510 2024-09-13T18:33:34.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.710+0000 7fa57e7fc640 10 client.27931 handle_command_reply: tid=0 2024-09-13T18:33:34.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.712+0000 7fa5a4016740 1 client.27931 shutdown 2024-09-13T18:33:34.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:34 smithi114 ceph-mon[30897]: pgmap v345: 129 pgs: 129 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 30 MiB/s wr, 647 op/s 2024-09-13T18:33:34.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:34 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/3925352619' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:33:34.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.720+0000 7fa5a4016740 1 -- 172.21.15.114:0/500461525 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577ffbcc180 msgr2=0x5577ffbc78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:34.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.720+0000 7fa5a4016740 1 --2- 172.21.15.114:0/500461525 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577ffbcc180 0x5577ffbc78b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa588009890 tx=0x7fa58802c040 comp rx=0 tx=0).stop 2024-09-13T18:33:34.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.723+0000 7fa5a4016740 1 -- 172.21.15.114:0/500461525 shutdown_connections 2024-09-13T18:33:34.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.723+0000 7fa5a4016740 1 --2- 172.21.15.114:0/500461525 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x5577ffbde510 0x5577ffbe0930 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.723+0000 7fa5a4016740 1 --2- 172.21.15.114:0/500461525 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577ffbd7340 0x5577ffbc5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.723+0000 7fa5a4016740 1 --2- 172.21.15.114:0/500461525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5577ffbccaa0 0x5577ffbc4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.723+0000 7fa5a4016740 1 --2- 172.21.15.114:0/500461525 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577ffbcc180 0x5577ffbc78b0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.723+0000 7fa5a4016740 1 -- 172.21.15.114:0/500461525 >> 172.21.15.114:0/500461525 conn(0x5577ffbc86f0 msgr2=0x5577ffbcdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:34.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.725+0000 7fa57e7fc640 0 client.27931 ms_handle_reset on v2:172.21.15.204:6838/344883912 2024-09-13T18:33:34.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.726+0000 7fa5a4016740 1 -- 172.21.15.114:0/500461525 shutdown_connections 2024-09-13T18:33:34.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.726+0000 7fa5a4016740 1 -- 172.21.15.114:0/500461525 wait complete. 2024-09-13T18:33:34.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.730+0000 7fa5a4016740 20 client.27931 trim_cache size 0 max 16384 2024-09-13T18:33:34.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.752+0000 7fa5a4016740 1 Processor -- start 2024-09-13T18:33:34.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.753+0000 7fa5a4016740 1 -- start start 2024-09-13T18:33:34.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.753+0000 7fa5a4016740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5577ffbcc180 0x5577ffbd2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:34.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.753+0000 7fa5a4016740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577ffbccaa0 0x5577ffbd3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:34.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.753+0000 7fa5a4016740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577ffbd7340 0x5577ffbd57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:34.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.753+0000 7fa5a4016740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ffbbb650 con 0x5577ffbccaa0 2024-09-13T18:33:34.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.753+0000 7fa5a4016740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ffbe5ab0 con 0x5577ffbcc180 2024-09-13T18:33:34.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.754+0000 7fa5a4016740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ffbe0ba0 con 0x5577ffbd7340 2024-09-13T18:33:34.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.754+0000 7fa59a7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5577ffbcc180 0x5577ffbd2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:34.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.754+0000 7fa59a7fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577ffbd7340 0x5577ffbd57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:34.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.754+0000 7fa59a7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5577ffbcc180 0x5577ffbd2d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:40632/0 (socket says 172.21.15.114:40632) 2024-09-13T18:33:34.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.754+0000 7fa59a7fc640 1 -- 172.21.15.114:0/2717357446 learned_addr learned my addr 172.21.15.114:0/2717357446 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:34.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.755+0000 7fa59a7fc640 1 -- 172.21.15.114:0/2717357446 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577ffbd7340 msgr2=0x5577ffbd57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:34.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.755+0000 7fa59a7fc640 1 --2- 172.21.15.114:0/2717357446 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577ffbd7340 0x5577ffbd57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.755+0000 7fa59a7fc640 1 -- 172.21.15.114:0/2717357446 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577ffbccaa0 msgr2=0x5577ffbd3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:34.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.757+0000 7fa59affd640 1 --2- 172.21.15.114:0/2717357446 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577ffbccaa0 0x5577ffbd3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:34.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.761+0000 7fa59a7fc640 1 --2- 172.21.15.114:0/2717357446 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577ffbccaa0 0x5577ffbd3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.761+0000 7fa59a7fc640 1 -- 172.21.15.114:0/2717357446 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577ffac7770 con 0x5577ffbcc180 2024-09-13T18:33:34.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.761+0000 7fa59affd640 1 --2- 172.21.15.114:0/2717357446 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577ffbccaa0 0x5577ffbd3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:33:34.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.761+0000 7fa59a7fc640 1 --2- 172.21.15.114:0/2717357446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5577ffbcc180 0x5577ffbd2d20 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa58802a980 tx=0x7fa588000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a1bb380d8de9b60 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:34.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.763+0000 7fa591ffb640 1 -- 172.21.15.114:0/2717357446 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa588057030 con 0x5577ffbcc180 2024-09-13T18:33:34.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.763+0000 7fa591ffb640 1 -- 172.21.15.114:0/2717357446 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa588062040 con 0x5577ffbcc180 2024-09-13T18:33:34.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.764+0000 7fa5a4016740 1 -- 172.21.15.114:0/2717357446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5577ffbcc180 msgr2=0x5577ffbd2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:34.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.764+0000 7fa5a4016740 1 --2- 172.21.15.114:0/2717357446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5577ffbcc180 0x5577ffbd2d20 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa58802a980 tx=0x7fa588000bc0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.764+0000 7fa5a4016740 1 -- 172.21.15.114:0/2717357446 shutdown_connections 2024-09-13T18:33:34.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.764+0000 7fa5a4016740 1 --2- 172.21.15.114:0/2717357446 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577ffbd7340 0x5577ffbd57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.764+0000 7fa5a4016740 1 --2- 172.21.15.114:0/2717357446 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577ffbccaa0 0x5577ffbd3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.764+0000 7fa5a4016740 1 --2- 172.21.15.114:0/2717357446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5577ffbcc180 0x5577ffbd2d20 secure :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa58802a980 tx=0x7fa588000bc0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.764+0000 7fa5a4016740 1 -- 172.21.15.114:0/2717357446 >> 172.21.15.114:0/2717357446 conn(0x5577ffbc86f0 msgr2=0x5577ffbc8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:34.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.771+0000 7fa5a4016740 1 -- 172.21.15.114:0/2717357446 shutdown_connections 2024-09-13T18:33:34.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.772+0000 7fa5a4016740 1 -- 172.21.15.114:0/2717357446 wait complete. 2024-09-13T18:33:34.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.772+0000 7fa5a4016740 1 -- start start 2024-09-13T18:33:34.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.772+0000 7fa5a4016740 1 Processor -- start 2024-09-13T18:33:34.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.773+0000 7fa5a4016740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5577ffbccaa0 0x5577ffbd0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:34.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.773+0000 7fa5a4016740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577ffbd7340 0x5577ffbd12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:34.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.773+0000 7fa5a4016740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577ffc31a00 0x5577ffc33df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:34.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.773+0000 7fa5a4016740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ffbd19c0 con 0x5577ffbd7340 2024-09-13T18:33:34.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.773+0000 7fa5a4016740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ffc34330 con 0x5577ffbccaa0 2024-09-13T18:33:34.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.773+0000 7fa5a4016740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ffc34470 con 0x5577ffc31a00 2024-09-13T18:33:34.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.773+0000 7fa5a4016740 10 client.0 _ll_register_callbacks cb 0x7fa58c00a150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:33:34.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.773+0000 7fa59a7fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577ffc31a00 0x5577ffc33df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:34.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.773+0000 7fa59a7fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577ffc31a00 0x5577ffc33df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:59678/0 (socket says 172.21.15.114:59678) 2024-09-13T18:33:34.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.773+0000 7fa59a7fc640 1 -- 172.21.15.114:0/509139682 learned_addr learned my addr 172.21.15.114:0/509139682 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:34.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.773+0000 7fa59a7fc640 1 --2- 172.21.15.114:0/509139682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5577ffbccaa0 0x5577ffbd0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:34.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.777+0000 7fa59a7fc640 1 -- 172.21.15.114:0/509139682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5577ffbccaa0 msgr2=0x5577ffbd0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:34.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.777+0000 7fa59a7fc640 1 --2- 172.21.15.114:0/509139682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5577ffbccaa0 0x5577ffbd0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.777+0000 7fa59a7fc640 1 -- 172.21.15.114:0/509139682 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577ffbd7340 msgr2=0x5577ffbd12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:34.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.777+0000 7fa59a7fc640 1 --2- 172.21.15.114:0/509139682 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577ffbd7340 0x5577ffbd12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.777+0000 7fa59a7fc640 1 -- 172.21.15.114:0/509139682 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5577ffb253c0 con 0x5577ffc31a00 2024-09-13T18:33:34.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.783+0000 7fa59a7fc640 1 --2- 172.21.15.114:0/509139682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577ffc31a00 0x5577ffc33df0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa588009780 tx=0x7fa588055040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=361bd44a61ae37d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:34.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.783+0000 7fa57e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:33:34.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.787+0000 7fa57e7fc640 1 -- 172.21.15.114:0/509139682 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa588004020 con 0x5577ffc31a00 2024-09-13T18:33:34.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.788+0000 7fa57e7fc640 1 -- 172.21.15.114:0/509139682 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fa588004180 con 0x5577ffc31a00 2024-09-13T18:33:34.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.788+0000 7fa5a4016740 1 -- 172.21.15.114:0/509139682 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5577ff81a4e0 con 0x5577ffc31a00 2024-09-13T18:33:34.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.795+0000 7fa57e7fc640 1 -- 172.21.15.114:0/509139682 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa588004340 con 0x5577ffc31a00 2024-09-13T18:33:34.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.796+0000 7fa5a4016740 10 client.27934 fetch_fsmap learned FSMap version 37 2024-09-13T18:33:34.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.796+0000 7fa5a4016740 1 -- 172.21.15.114:0/509139682 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x5577ffaece80 con 0x5577ffc31a00 2024-09-13T18:33:34.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.800+0000 7fa57e7fc640 1 -- 172.21.15.114:0/509139682 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa588076020 con 0x5577ffc31a00 2024-09-13T18:33:34.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.800+0000 7fa57e7fc640 1 -- 172.21.15.114:0/509139682 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 37) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fa58805b070 con 0x5577ffc31a00 2024-09-13T18:33:34.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.801+0000 7fa5a4016740 10 client.27934 fetch_fsmap finished waiting for FSMap version 37 2024-09-13T18:33:34.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.801+0000 7fa5a4016740 10 client.27934 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-13T18:33:34.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.801+0000 7fa5a4016740 20 client.27934 populate_metadata read hostname 'smithi114' 2024-09-13T18:33:34.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.801+0000 7fa5a4016740 1 --2- 172.21.15.114:0/509139682 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x5577ffc577a0 0x5577ffc59b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:34.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.801+0000 7fa5a4016740 4 client.27934 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:33:34.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.801+0000 7fa5a4016740 1 -- 172.21.15.114:0/509139682 --> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5577ff82b7e0 con 0x5577ffc577a0 2024-09-13T18:33:34.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.802+0000 7fa59affd640 1 --2- 172.21.15.114:0/509139682 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x5577ffc577a0 0x5577ffc59b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:34.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.804+0000 7fa59affd640 1 --2- 172.21.15.114:0/509139682 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x5577ffc577a0 0x5577ffc59b90 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e329792d6302c927 server_cookie=7b3d5e9c0728f3c0 in_seq=0 out_seq=0 2024-09-13T18:33:34.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.804+0000 7fa57e7fc640 10 client.27934 ms_handle_connect on v2:172.21.15.204:6838/344883912 2024-09-13T18:33:34.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.804+0000 7fa57e7fc640 1 -- 172.21.15.114:0/509139682 <== mds.2 v2:172.21.15.204:6838/344883912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5577ff82b7e0 con 0x5577ffc577a0 2024-09-13T18:33:34.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.804+0000 7fa57e7fc640 10 client.27934 handle_command_reply: tid=0 2024-09-13T18:33:34.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.804+0000 7fa5a4016740 1 client.27934 shutdown 2024-09-13T18:33:34.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.805+0000 7fa5a4016740 1 -- 172.21.15.114:0/509139682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577ffc31a00 msgr2=0x5577ffc33df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:34.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.805+0000 7fa5a4016740 1 --2- 172.21.15.114:0/509139682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577ffc31a00 0x5577ffc33df0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa588009780 tx=0x7fa588055040 comp rx=0 tx=0).stop 2024-09-13T18:33:34.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.807+0000 7fa5a4016740 1 -- 172.21.15.114:0/509139682 shutdown_connections 2024-09-13T18:33:34.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.807+0000 7fa5a4016740 1 --2- 172.21.15.114:0/509139682 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x5577ffc577a0 0x5577ffc59b90 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.807+0000 7fa5a4016740 1 --2- 172.21.15.114:0/509139682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577ffc31a00 0x5577ffc33df0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.807+0000 7fa5a4016740 1 --2- 172.21.15.114:0/509139682 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577ffbd7340 0x5577ffbd12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.807+0000 7fa5a4016740 1 --2- 172.21.15.114:0/509139682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5577ffbccaa0 0x5577ffbd0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.807+0000 7fa5a4016740 1 -- 172.21.15.114:0/509139682 >> 172.21.15.114:0/509139682 conn(0x5577ffbc86f0 msgr2=0x5577ffbbfcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:34.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.808+0000 7fa57e7fc640 0 client.27934 ms_handle_reset on v2:172.21.15.204:6838/344883912 2024-09-13T18:33:34.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.810+0000 7fa5a4016740 1 -- 172.21.15.114:0/509139682 shutdown_connections 2024-09-13T18:33:34.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.810+0000 7fa5a4016740 1 -- 172.21.15.114:0/509139682 wait complete. 2024-09-13T18:33:34.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.811+0000 7fa5a4016740 20 client.27934 trim_cache size 0 max 16384 2024-09-13T18:33:34.814 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:33:34.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.817+0000 7fa59bfff640 1 -- 172.21.15.114:0/3845132950 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa5800a1410 msgr2=0x7fa5800a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:34.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.817+0000 7fa59bfff640 1 --2- 172.21.15.114:0/3845132950 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa5800a1410 0x7fa5800a38d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa594003fd0 tx=0x7fa5940c4690 comp rx=0 tx=0).stop 2024-09-13T18:33:34.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.817+0000 7fa59bfff640 1 -- 172.21.15.114:0/3845132950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa59c19ea20 msgr2=0x7fa59c19eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:34.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.817+0000 7fa59bfff640 1 --2- 172.21.15.114:0/3845132950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa59c19ea20 0x7fa59c19eea0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa59400eda0 tx=0x7fa59400c6e0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.818+0000 7fa59bfff640 1 -- 172.21.15.114:0/3845132950 shutdown_connections 2024-09-13T18:33:34.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.818+0000 7fa59bfff640 1 --2- 172.21.15.114:0/3845132950 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fa5800a1410 0x7fa5800a38d0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.818+0000 7fa59bfff640 1 --2- 172.21.15.114:0/3845132950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa59c19ea20 0x7fa59c19eea0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.818+0000 7fa59bfff640 1 --2- 172.21.15.114:0/3845132950 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa59c112d70 0x7fa59c1a4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.818+0000 7fa59bfff640 1 --2- 172.21.15.114:0/3845132950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa59c1074b0 0x7fa59c1a4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:34.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.818+0000 7fa59bfff640 1 -- 172.21.15.114:0/3845132950 >> 172.21.15.114:0/3845132950 conn(0x7fa59c06af40 msgr2=0x7fa59c06ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:34.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.823+0000 7fa59bfff640 1 -- 172.21.15.114:0/3845132950 shutdown_connections 2024-09-13T18:33:34.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:34.825+0000 7fa59bfff640 1 -- 172.21.15.114:0/3845132950 wait complete. 2024-09-13T18:33:34.862 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-09-13T18:33:35.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.146+0000 7f951af91640 1 -- 172.21.15.114:0/1794029642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9514107890 msgr2=0x7f9514107cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:35.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.146+0000 7f951af91640 1 --2- 172.21.15.114:0/1794029642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9514107890 0x7f9514107cd0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f9504009930 tx=0x7f950402a450 comp rx=0 tx=0).stop 2024-09-13T18:33:35.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.152+0000 7f951af91640 1 -- 172.21.15.114:0/1794029642 shutdown_connections 2024-09-13T18:33:35.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.152+0000 7f951af91640 1 --2- 172.21.15.114:0/1794029642 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9514119400 0x7f951411b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.152+0000 7f951af91640 1 --2- 172.21.15.114:0/1794029642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9514107890 0x7f9514107cd0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.152+0000 7f951af91640 1 --2- 172.21.15.114:0/1794029642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95141074b0 0x7f951410e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.152+0000 7f951af91640 1 -- 172.21.15.114:0/1794029642 >> 172.21.15.114:0/1794029642 conn(0x7f951406af70 msgr2=0x7f951406b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:35.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.155+0000 7f951af91640 1 -- 172.21.15.114:0/1794029642 shutdown_connections 2024-09-13T18:33:35.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.156+0000 7f951af91640 1 -- 172.21.15.114:0/1794029642 wait complete. 2024-09-13T18:33:35.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.156+0000 7f951af91640 1 Processor -- start 2024-09-13T18:33:35.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.157+0000 7f951af91640 1 -- start start 2024-09-13T18:33:35.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.157+0000 7f951af91640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95141074b0 0x7f9514116980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:35.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.157+0000 7f951af91640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9514107890 0x7f951410f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:35.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.157+0000 7f951af91640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9514119400 0x7f951410ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:35.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.157+0000 7f951af91640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9514110460 con 0x7f95141074b0 2024-09-13T18:33:35.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.157+0000 7f951af91640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95141105d0 con 0x7f9514107890 2024-09-13T18:33:35.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.157+0000 7f951af91640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9514110790 con 0x7f9514119400 2024-09-13T18:33:35.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.158+0000 7f9518d2d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95141074b0 0x7f9514116980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:35.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.158+0000 7f9518d2d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95141074b0 0x7f9514116980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:52770/0 (socket says 172.21.15.114:52770) 2024-09-13T18:33:35.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.158+0000 7f9518d2d640 1 -- 172.21.15.114:0/2918841682 learned_addr learned my addr 172.21.15.114:0/2918841682 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:35.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.159+0000 7f9513fff640 1 --2- 172.21.15.114:0/2918841682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9514107890 0x7f951410f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:35.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.160+0000 7f951952e640 1 --2- 172.21.15.114:0/2918841682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9514119400 0x7f951410ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:35.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.162+0000 7f9513fff640 1 -- 172.21.15.114:0/2918841682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9514119400 msgr2=0x7f951410ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:35.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.162+0000 7f9513fff640 1 --2- 172.21.15.114:0/2918841682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9514119400 0x7f951410ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.162+0000 7f9513fff640 1 -- 172.21.15.114:0/2918841682 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95141074b0 msgr2=0x7f9514116980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:35.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.162+0000 7f9513fff640 1 --2- 172.21.15.114:0/2918841682 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95141074b0 0x7f9514116980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.162+0000 7f9513fff640 1 -- 172.21.15.114:0/2918841682 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9504009560 con 0x7f9514107890 2024-09-13T18:33:35.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.162+0000 7f9518d2d640 1 --2- 172.21.15.114:0/2918841682 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95141074b0 0x7f9514116980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:33:35.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.164+0000 7f9513fff640 1 --2- 172.21.15.114:0/2918841682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9514107890 0x7f951410f9e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f9504033040 tx=0x7f9504005580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:35.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.164+0000 7f9511ffb640 1 -- 172.21.15.114:0/2918841682 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9504031850 con 0x7f9514107890 2024-09-13T18:33:35.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.164+0000 7f951af91640 1 -- 172.21.15.114:0/2918841682 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95141144a0 con 0x7f9514107890 2024-09-13T18:33:35.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.164+0000 7f951af91640 1 -- 172.21.15.114:0/2918841682 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9514114920 con 0x7f9514107890 2024-09-13T18:33:35.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.165+0000 7f9511ffb640 1 -- 172.21.15.114:0/2918841682 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9504031e40 con 0x7f9514107890 2024-09-13T18:33:35.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.165+0000 7f9511ffb640 1 -- 172.21.15.114:0/2918841682 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950403bd10 con 0x7f9514107890 2024-09-13T18:33:35.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.170+0000 7f9511ffb640 1 -- 172.21.15.114:0/2918841682 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f95040319b0 con 0x7f9514107890 2024-09-13T18:33:35.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.171+0000 7f9511ffb640 1 --2- 172.21.15.114:0/2918841682 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f94fc0a1390 0x7f94fc0a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:35.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.171+0000 7f951dcb1740 1 Processor -- start 2024-09-13T18:33:35.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.171+0000 7f951dcb1740 1 -- start start 2024-09-13T18:33:35.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.171+0000 7f9518d2d640 1 --2- 172.21.15.114:0/2918841682 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f94fc0a1390 0x7f94fc0a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:35.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.171+0000 7f951dcb1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c52e6c3970 0x55c52e6c3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:35.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.171+0000 7f951dcb1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c52e6c4290 0x55c52e6ce5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:35.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.171+0000 7f951dcb1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c52e6ceb30 0x55c52e6d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:35.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.171+0000 7f951dcb1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c52e62dcc0 con 0x55c52e6c4290 2024-09-13T18:33:35.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.171+0000 7f951dcb1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c52e62b450 con 0x55c52e6ceb30 2024-09-13T18:33:35.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.171+0000 7f951dcb1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c52e626d40 con 0x55c52e6c3970 2024-09-13T18:33:35.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.171+0000 7f9511ffb640 1 -- 172.21.15.114:0/2918841682 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f95040edd20 con 0x7f9514107890 2024-09-13T18:33:35.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.171+0000 7f9513fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c52e6c4290 0x55c52e6ce5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:35.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.171+0000 7f9513fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c52e6c4290 0x55c52e6ce5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:52784/0 (socket says 172.21.15.114:52784) 2024-09-13T18:33:35.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.171+0000 7f9513fff640 1 -- 172.21.15.114:0/2502331399 learned_addr learned my addr 172.21.15.114:0/2502331399 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:35.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.171+0000 7f9518d2d640 1 --2- 172.21.15.114:0/2502331399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c52e6c3970 0x55c52e6c3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:35.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.171+0000 7f951952e640 1 --2- 172.21.15.114:0/2502331399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c52e6ceb30 0x55c52e6d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:35.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.172+0000 7f951952e640 1 -- 172.21.15.114:0/2502331399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c52e6c3970 msgr2=0x55c52e6c3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:35.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.172+0000 7f951952e640 1 --2- 172.21.15.114:0/2502331399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c52e6c3970 0x55c52e6c3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.172+0000 7f9518d2d640 1 --2- 172.21.15.114:0/2918841682 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f94fc0a1390 0x7f94fc0a3850 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9508005fd0 tx=0x7f95080137d0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:35.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.172+0000 7f951952e640 1 -- 172.21.15.114:0/2502331399 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c52e6c4290 msgr2=0x55c52e6ce5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:35.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.172+0000 7f951952e640 1 --2- 172.21.15.114:0/2502331399 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c52e6c4290 0x55c52e6ce5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.172+0000 7f951952e640 1 -- 172.21.15.114:0/2502331399 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c52e5ee620 con 0x55c52e6ceb30 2024-09-13T18:33:35.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.172+0000 7f951952e640 1 --2- 172.21.15.114:0/2502331399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c52e6ceb30 0x55c52e6d0f20 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f950c009f70 tx=0x7f950c009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa2ea32e2f01d6f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:35.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.172+0000 7f94faffd640 1 -- 172.21.15.114:0/2502331399 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950c010040 con 0x55c52e6ceb30 2024-09-13T18:33:35.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.172+0000 7f94faffd640 1 -- 172.21.15.114:0/2502331399 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f950c013ce0 con 0x55c52e6ceb30 2024-09-13T18:33:35.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.172+0000 7f94faffd640 1 -- 172.21.15.114:0/2502331399 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950c014d80 con 0x55c52e6ceb30 2024-09-13T18:33:35.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.173+0000 7f951dcb1740 1 -- 172.21.15.114:0/2502331399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c52e6ceb30 msgr2=0x55c52e6d0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:35.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.173+0000 7f951dcb1740 1 --2- 172.21.15.114:0/2502331399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c52e6ceb30 0x55c52e6d0f20 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f950c009f70 tx=0x7f950c009650 comp rx=0 tx=0).stop 2024-09-13T18:33:35.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.174+0000 7f951dcb1740 1 -- 172.21.15.114:0/2502331399 shutdown_connections 2024-09-13T18:33:35.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.174+0000 7f951dcb1740 1 --2- 172.21.15.114:0/2502331399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c52e6ceb30 0x55c52e6d0f20 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.174+0000 7f951dcb1740 1 --2- 172.21.15.114:0/2502331399 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c52e6c4290 0x55c52e6ce5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.174+0000 7f951dcb1740 1 --2- 172.21.15.114:0/2502331399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c52e6c3970 0x55c52e6c3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.174+0000 7f951dcb1740 1 -- 172.21.15.114:0/2502331399 >> 172.21.15.114:0/2502331399 conn(0x55c52e6bfee0 msgr2=0x55c52e6c02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:35.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.174+0000 7f951dcb1740 1 -- 172.21.15.114:0/2502331399 shutdown_connections 2024-09-13T18:33:35.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.174+0000 7f951dcb1740 1 -- 172.21.15.114:0/2502331399 wait complete. 2024-09-13T18:33:35.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.174+0000 7f951dcb1740 1 -- start start 2024-09-13T18:33:35.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.174+0000 7f951dcb1740 1 Processor -- start 2024-09-13T18:33:35.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.175+0000 7f951dcb1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c52e6c3970 0x55c52e6ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:35.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.175+0000 7f951dcb1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c52e6c4290 0x55c52e6ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:35.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.175+0000 7f951dcb1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c52e6ceb30 0x55c52e6cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:35.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.176+0000 7f951dcb1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c52e5e5180 con 0x55c52e6c4290 2024-09-13T18:33:35.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.176+0000 7f9518d2d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c52e6c3970 0x55c52e6ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:35.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.176+0000 7f951dcb1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c52e2ff430 con 0x55c52e6ceb30 2024-09-13T18:33:35.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.176+0000 7f951dcb1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c52e5aa5e0 con 0x55c52e6c3970 2024-09-13T18:33:35.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.176+0000 7f951dcb1740 10 client.0 _ll_register_callbacks cb 0x55c52e616aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:33:35.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.176+0000 7f9513fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c52e6c4290 0x55c52e6ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:35.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.176+0000 7f9513fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c52e6c4290 0x55c52e6ce5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:52790/0 (socket says 172.21.15.114:52790) 2024-09-13T18:33:35.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.176+0000 7f9513fff640 1 -- 172.21.15.114:0/4059727844 learned_addr learned my addr 172.21.15.114:0/4059727844 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:35.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.176+0000 7f951952e640 1 --2- 172.21.15.114:0/4059727844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c52e6ceb30 0x55c52e6cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:35.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.176+0000 7f9513fff640 1 -- 172.21.15.114:0/4059727844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c52e6c3970 msgr2=0x55c52e6ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:35.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.176+0000 7f9513fff640 1 --2- 172.21.15.114:0/4059727844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c52e6c3970 0x55c52e6ce0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.176+0000 7f9513fff640 1 -- 172.21.15.114:0/4059727844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c52e6ceb30 msgr2=0x55c52e6cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:35.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.176+0000 7f9513fff640 1 --2- 172.21.15.114:0/4059727844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c52e6ceb30 0x55c52e6cb630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.176+0000 7f9513fff640 1 -- 172.21.15.114:0/4059727844 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f950c009ba0 con 0x55c52e6c4290 2024-09-13T18:33:35.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.177+0000 7f9513fff640 1 --2- 172.21.15.114:0/4059727844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c52e6c4290 0x55c52e6ce5e0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f950403b8e0 tx=0x7f95040ba8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=215352d4c5795468 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:35.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.177+0000 7f94f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:33:35.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.177+0000 7f94f3fff640 1 -- 172.21.15.114:0/4059727844 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950403b630 con 0x55c52e6c4290 2024-09-13T18:33:35.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.177+0000 7f951dcb1740 1 -- 172.21.15.114:0/4059727844 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c52e650b80 con 0x55c52e6c4290 2024-09-13T18:33:35.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.177+0000 7f94f3fff640 1 -- 172.21.15.114:0/4059727844 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f9504031c60 con 0x55c52e6c4290 2024-09-13T18:33:35.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.177+0000 7f94f3fff640 1 -- 172.21.15.114:0/4059727844 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9504032b40 con 0x55c52e6c4290 2024-09-13T18:33:35.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.177+0000 7f951dcb1740 10 client.18552 fetch_fsmap learned FSMap version 37 2024-09-13T18:33:35.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.177+0000 7f951dcb1740 1 -- 172.21.15.114:0/4059727844 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55c52e6253c0 con 0x55c52e6c4290 2024-09-13T18:33:35.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.178+0000 7f94f3fff640 1 -- 172.21.15.114:0/4059727844 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f95040d7020 con 0x55c52e6c4290 2024-09-13T18:33:35.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.178+0000 7f94f3fff640 1 -- 172.21.15.114:0/4059727844 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 37) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f950404d020 con 0x55c52e6c4290 2024-09-13T18:33:35.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.179+0000 7f951dcb1740 10 client.18552 fetch_fsmap finished waiting for FSMap version 37 2024-09-13T18:33:35.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.179+0000 7f951dcb1740 10 client.18552 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-13T18:33:35.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.179+0000 7f951dcb1740 20 client.18552 populate_metadata read hostname 'smithi114' 2024-09-13T18:33:35.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.179+0000 7f951dcb1740 1 --2- 172.21.15.114:0/4059727844 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55c52e6de230 0x55c52e6e0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:35.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.179+0000 7f951dcb1740 4 client.18552 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:33:35.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.179+0000 7f951dcb1740 1 -- 172.21.15.114:0/4059727844 --> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c52e650b80 con 0x55c52e6de230 2024-09-13T18:33:35.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.180+0000 7f9518d2d640 1 --2- 172.21.15.114:0/4059727844 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55c52e6de230 0x55c52e6e0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:35.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.180+0000 7f9518d2d640 1 --2- 172.21.15.114:0/4059727844 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55c52e6de230 0x55c52e6e0670 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=de8b161e0089336f server_cookie=8e16efc7cf57051b in_seq=0 out_seq=0 2024-09-13T18:33:35.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.180+0000 7f94f3fff640 10 client.18552 ms_handle_connect on v2:172.21.15.134:6836/891692210 2024-09-13T18:33:35.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.181+0000 7f94f3fff640 1 -- 172.21.15.114:0/4059727844 <== mds.1 v2:172.21.15.134:6836/891692210 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55c52e650b80 con 0x55c52e6de230 2024-09-13T18:33:35.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.181+0000 7f94f3fff640 10 client.18552 handle_command_reply: tid=0 2024-09-13T18:33:35.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.181+0000 7f951dcb1740 1 client.18552 shutdown 2024-09-13T18:33:35.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.181+0000 7f951dcb1740 1 -- 172.21.15.114:0/4059727844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c52e6c4290 msgr2=0x55c52e6ce5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:35.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.181+0000 7f951dcb1740 1 --2- 172.21.15.114:0/4059727844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c52e6c4290 0x55c52e6ce5e0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f950403b8e0 tx=0x7f95040ba8c0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.182+0000 7f951dcb1740 1 -- 172.21.15.114:0/4059727844 shutdown_connections 2024-09-13T18:33:35.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.182+0000 7f951dcb1740 1 --2- 172.21.15.114:0/4059727844 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55c52e6de230 0x55c52e6e0670 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.183+0000 7f94f3fff640 0 client.18552 ms_handle_reset on v2:172.21.15.134:6836/891692210 2024-09-13T18:33:35.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.183+0000 7f951dcb1740 1 --2- 172.21.15.114:0/4059727844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c52e6ceb30 0x55c52e6cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.183+0000 7f951dcb1740 1 --2- 172.21.15.114:0/4059727844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c52e6c4290 0x55c52e6ce5e0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.183+0000 7f951dcb1740 1 --2- 172.21.15.114:0/4059727844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c52e6c3970 0x55c52e6ce0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.183+0000 7f951dcb1740 1 -- 172.21.15.114:0/4059727844 >> 172.21.15.114:0/4059727844 conn(0x55c52e6bfee0 msgr2=0x55c52e6bf230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:35.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.186+0000 7f951dcb1740 1 -- 172.21.15.114:0/4059727844 shutdown_connections 2024-09-13T18:33:35.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.186+0000 7f951dcb1740 1 -- 172.21.15.114:0/4059727844 wait complete. 2024-09-13T18:33:35.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.187+0000 7f951dcb1740 20 client.18552 trim_cache size 0 max 16384 2024-09-13T18:33:35.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.225+0000 7f951dcb1740 1 Processor -- start 2024-09-13T18:33:35.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.227+0000 7f951dcb1740 1 -- start start 2024-09-13T18:33:35.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.228+0000 7f951dcb1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c52e6c3970 0x55c52e6cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:35.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.228+0000 7f951dcb1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c52e6c4290 0x55c52e6d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:35.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.228+0000 7f9518d2d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c52e6c3970 0x55c52e6cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:35.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.228+0000 7f9518d2d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c52e6c3970 0x55c52e6cff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:40668/0 (socket says 172.21.15.114:40668) 2024-09-13T18:33:35.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.228+0000 7f9518d2d640 1 -- 172.21.15.114:0/1170217564 learned_addr learned my addr 172.21.15.114:0/1170217564 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:35.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.228+0000 7f9513fff640 1 --2- 172.21.15.114:0/1170217564 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c52e6c4290 0x55c52e6d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:35.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.228+0000 7f951dcb1740 1 --2- 172.21.15.114:0/1170217564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c52e6ceb30 0x55c52e6c8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:35.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.228+0000 7f951dcb1740 1 -- 172.21.15.114:0/1170217564 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c52e650b80 con 0x55c52e6c4290 2024-09-13T18:33:35.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.228+0000 7f951dcb1740 1 -- 172.21.15.114:0/1170217564 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c52e6c0bc0 con 0x55c52e6c3970 2024-09-13T18:33:35.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.228+0000 7f951dcb1740 1 -- 172.21.15.114:0/1170217564 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c52e6dac00 con 0x55c52e6ceb30 2024-09-13T18:33:35.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.229+0000 7f951952e640 1 --2- 172.21.15.114:0/1170217564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c52e6ceb30 0x55c52e6c8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:35.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.229+0000 7f9513fff640 1 -- 172.21.15.114:0/1170217564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c52e6ceb30 msgr2=0x55c52e6c8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:35.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.229+0000 7f9513fff640 1 --2- 172.21.15.114:0/1170217564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c52e6ceb30 0x55c52e6c8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.229+0000 7f9513fff640 1 -- 172.21.15.114:0/1170217564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c52e6c3970 msgr2=0x55c52e6cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:35.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.229+0000 7f9513fff640 1 --2- 172.21.15.114:0/1170217564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c52e6c3970 0x55c52e6cff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.229+0000 7f9513fff640 1 -- 172.21.15.114:0/1170217564 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c52e5c7770 con 0x55c52e6c4290 2024-09-13T18:33:35.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.229+0000 7f9513fff640 1 --2- 172.21.15.114:0/1170217564 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c52e6c4290 0x55c52e6d0460 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x55c52e6cc990 tx=0x7f95040ba930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fef88c42e3c33d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:35.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.230+0000 7f94fb7fe640 1 -- 172.21.15.114:0/1170217564 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9504032b40 con 0x55c52e6c4290 2024-09-13T18:33:35.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.230+0000 7f94fb7fe640 1 -- 172.21.15.114:0/1170217564 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f95040d7020 con 0x55c52e6c4290 2024-09-13T18:33:35.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.230+0000 7f94fb7fe640 1 -- 172.21.15.114:0/1170217564 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95040bb130 con 0x55c52e6c4290 2024-09-13T18:33:35.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.231+0000 7f951dcb1740 1 -- 172.21.15.114:0/1170217564 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c52e6c4290 msgr2=0x55c52e6d0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:35.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.231+0000 7f951dcb1740 1 --2- 172.21.15.114:0/1170217564 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c52e6c4290 0x55c52e6d0460 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x55c52e6cc990 tx=0x7f95040ba930 comp rx=0 tx=0).stop 2024-09-13T18:33:35.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.231+0000 7f951dcb1740 1 -- 172.21.15.114:0/1170217564 shutdown_connections 2024-09-13T18:33:35.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.232+0000 7f951dcb1740 1 --2- 172.21.15.114:0/1170217564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c52e6ceb30 0x55c52e6c8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.232+0000 7f951dcb1740 1 --2- 172.21.15.114:0/1170217564 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c52e6c4290 0x55c52e6d0460 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.232+0000 7f951dcb1740 1 --2- 172.21.15.114:0/1170217564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c52e6c3970 0x55c52e6cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.232+0000 7f951dcb1740 1 -- 172.21.15.114:0/1170217564 >> 172.21.15.114:0/1170217564 conn(0x55c52e6bfee0 msgr2=0x55c52e6e4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:35.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.232+0000 7f951dcb1740 1 -- 172.21.15.114:0/1170217564 shutdown_connections 2024-09-13T18:33:35.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.232+0000 7f951dcb1740 1 -- 172.21.15.114:0/1170217564 wait complete. 2024-09-13T18:33:35.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.232+0000 7f951dcb1740 1 -- start start 2024-09-13T18:33:35.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.233+0000 7f951dcb1740 1 Processor -- start 2024-09-13T18:33:35.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.233+0000 7f951dcb1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c52e6c3970 0x55c52e6d2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:35.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.233+0000 7f951dcb1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c52e6c4290 0x55c52e6d2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:35.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.234+0000 7f951dcb1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c52e6ceb30 0x55c52e6d30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:35.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.234+0000 7f951dcb1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c52e734870 con 0x55c52e6c4290 2024-09-13T18:33:35.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.234+0000 7f951dcb1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c52e7349e0 con 0x55c52e6c3970 2024-09-13T18:33:35.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.234+0000 7f951dcb1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c52e734ba0 con 0x55c52e6ceb30 2024-09-13T18:33:35.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.234+0000 7f951dcb1740 10 client.0 _ll_register_callbacks cb 0x7f950c009990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:33:35.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.234+0000 7f9518d2d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c52e6c3970 0x55c52e6d2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:35.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.234+0000 7f9518d2d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c52e6c3970 0x55c52e6d2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:40684/0 (socket says 172.21.15.114:40684) 2024-09-13T18:33:35.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.234+0000 7f9518d2d640 1 -- 172.21.15.114:0/1469005553 learned_addr learned my addr 172.21.15.114:0/1469005553 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:35.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.234+0000 7f9513fff640 1 --2- 172.21.15.114:0/1469005553 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c52e6c4290 0x55c52e6d2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:35.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.234+0000 7f951952e640 1 --2- 172.21.15.114:0/1469005553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c52e6ceb30 0x55c52e6d30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:35.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.234+0000 7f9518d2d640 1 -- 172.21.15.114:0/1469005553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c52e6ceb30 msgr2=0x55c52e6d30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:35.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.234+0000 7f9518d2d640 1 --2- 172.21.15.114:0/1469005553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c52e6ceb30 0x55c52e6d30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.234+0000 7f9518d2d640 1 -- 172.21.15.114:0/1469005553 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c52e6c4290 msgr2=0x55c52e6d2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:35.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.234+0000 7f9518d2d640 1 --2- 172.21.15.114:0/1469005553 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c52e6c4290 0x55c52e6d2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.234+0000 7f9518d2d640 1 -- 172.21.15.114:0/1469005553 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c52e6253c0 con 0x55c52e6c3970 2024-09-13T18:33:35.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.235+0000 7f9513fff640 1 --2- 172.21.15.114:0/1469005553 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c52e6c4290 0x55c52e6d2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:33:35.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.235+0000 7f951952e640 1 --2- 172.21.15.114:0/1469005553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c52e6ceb30 0x55c52e6d30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:33:35.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.235+0000 7f9518d2d640 1 --2- 172.21.15.114:0/1469005553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c52e6c3970 0x55c52e6d2640 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f95080058c0 tx=0x7f9508015040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63dc793e2fc77499 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:35.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.235+0000 7f94f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:33:35.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.239+0000 7f94f3fff640 1 -- 172.21.15.114:0/1469005553 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9508024040 con 0x55c52e6c3970 2024-09-13T18:33:35.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.239+0000 7f94f3fff640 1 -- 172.21.15.114:0/1469005553 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f9508004220 con 0x55c52e6c3970 2024-09-13T18:33:35.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.243+0000 7f951dcb1740 1 -- 172.21.15.114:0/1469005553 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f950c013ce0 con 0x55c52e6c3970 2024-09-13T18:33:35.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.247+0000 7f94f3fff640 1 -- 172.21.15.114:0/1469005553 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95080043e0 con 0x55c52e6c3970 2024-09-13T18:33:35.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.251+0000 7f951dcb1740 10 client.28037 fetch_fsmap learned FSMap version 37 2024-09-13T18:33:35.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.251+0000 7f951dcb1740 1 -- 172.21.15.114:0/1469005553 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55c52e5ece80 con 0x55c52e6c3970 2024-09-13T18:33:35.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.256+0000 7f94f3fff640 1 -- 172.21.15.114:0/1469005553 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9508039030 con 0x55c52e6c3970 2024-09-13T18:33:35.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.256+0000 7f94f3fff640 1 -- 172.21.15.114:0/1469005553 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 37) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f9508032020 con 0x55c52e6c3970 2024-09-13T18:33:35.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.256+0000 7f951dcb1740 10 client.28037 fetch_fsmap finished waiting for FSMap version 37 2024-09-13T18:33:35.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.256+0000 7f951dcb1740 10 client.28037 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-13T18:33:35.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.256+0000 7f951dcb1740 20 client.28037 populate_metadata read hostname 'smithi114' 2024-09-13T18:33:35.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.256+0000 7f951dcb1740 1 --2- 172.21.15.114:0/1469005553 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55c52e6de230 0x55c52e738420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:35.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.256+0000 7f951dcb1740 4 client.28037 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:33:35.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.256+0000 7f951dcb1740 1 -- 172.21.15.114:0/1469005553 --> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c52e6191f0 con 0x55c52e6de230 2024-09-13T18:33:35.264 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.262+0000 7f9513fff640 1 --2- 172.21.15.114:0/1469005553 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55c52e6de230 0x55c52e738420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:35.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.278+0000 7f9513fff640 1 --2- 172.21.15.114:0/1469005553 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55c52e6de230 0x55c52e738420 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=13cdda069012af58 server_cookie=cb71db7e1240de2 in_seq=0 out_seq=0 2024-09-13T18:33:35.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.278+0000 7f94f3fff640 10 client.28037 ms_handle_connect on v2:172.21.15.134:6836/891692210 2024-09-13T18:33:35.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.283+0000 7f94f3fff640 1 -- 172.21.15.114:0/1469005553 <== mds.1 v2:172.21.15.134:6836/891692210 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c52e6191f0 con 0x55c52e6de230 2024-09-13T18:33:35.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.283+0000 7f94f3fff640 10 client.28037 handle_command_reply: tid=0 2024-09-13T18:33:35.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.283+0000 7f951dcb1740 1 client.28037 shutdown 2024-09-13T18:33:35.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.287+0000 7f951dcb1740 1 -- 172.21.15.114:0/1469005553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c52e6c3970 msgr2=0x55c52e6d2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:35.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.287+0000 7f951dcb1740 1 --2- 172.21.15.114:0/1469005553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c52e6c3970 0x55c52e6d2640 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f95080058c0 tx=0x7f9508015040 comp rx=0 tx=0).stop 2024-09-13T18:33:35.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.287+0000 7f951dcb1740 1 -- 172.21.15.114:0/1469005553 shutdown_connections 2024-09-13T18:33:35.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.287+0000 7f951dcb1740 1 --2- 172.21.15.114:0/1469005553 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55c52e6de230 0x55c52e738420 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.287+0000 7f94f3fff640 0 client.28037 ms_handle_reset on v2:172.21.15.134:6836/891692210 2024-09-13T18:33:35.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.287+0000 7f951dcb1740 1 --2- 172.21.15.114:0/1469005553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c52e6ceb30 0x55c52e6d30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.287+0000 7f951dcb1740 1 --2- 172.21.15.114:0/1469005553 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c52e6c4290 0x55c52e6d2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.287+0000 7f951dcb1740 1 --2- 172.21.15.114:0/1469005553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c52e6c3970 0x55c52e6d2640 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.287+0000 7f951dcb1740 1 -- 172.21.15.114:0/1469005553 >> 172.21.15.114:0/1469005553 conn(0x55c52e6bfee0 msgr2=0x55c52e6c53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:35.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.293+0000 7f951dcb1740 1 -- 172.21.15.114:0/1469005553 shutdown_connections 2024-09-13T18:33:35.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.293+0000 7f951dcb1740 1 -- 172.21.15.114:0/1469005553 wait complete. 2024-09-13T18:33:35.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.294+0000 7f951dcb1740 20 client.28037 trim_cache size 0 max 16384 2024-09-13T18:33:35.298 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:33:35.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.301+0000 7f951af91640 1 -- 172.21.15.114:0/2918841682 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f94fc0a1390 msgr2=0x7f94fc0a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:35.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.301+0000 7f951af91640 1 --2- 172.21.15.114:0/2918841682 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f94fc0a1390 0x7f94fc0a3850 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9508005fd0 tx=0x7f95080137d0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.301+0000 7f951af91640 1 -- 172.21.15.114:0/2918841682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9514107890 msgr2=0x7f951410f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:35.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.301+0000 7f951af91640 1 --2- 172.21.15.114:0/2918841682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9514107890 0x7f951410f9e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f9504033040 tx=0x7f9504005580 comp rx=0 tx=0).stop 2024-09-13T18:33:35.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.303+0000 7f951af91640 1 -- 172.21.15.114:0/2918841682 shutdown_connections 2024-09-13T18:33:35.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.303+0000 7f951af91640 1 --2- 172.21.15.114:0/2918841682 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f94fc0a1390 0x7f94fc0a3850 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.303+0000 7f951af91640 1 --2- 172.21.15.114:0/2918841682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9514119400 0x7f951410ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.304+0000 7f951af91640 1 --2- 172.21.15.114:0/2918841682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9514107890 0x7f951410f9e0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.304+0000 7f951af91640 1 --2- 172.21.15.114:0/2918841682 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95141074b0 0x7f9514116980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.304+0000 7f951af91640 1 -- 172.21.15.114:0/2918841682 >> 172.21.15.114:0/2918841682 conn(0x7f951406af70 msgr2=0x7f9514119b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:35.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.309+0000 7f951af91640 1 -- 172.21.15.114:0/2918841682 shutdown_connections 2024-09-13T18:33:35.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.311+0000 7f951af91640 1 -- 172.21.15.114:0/2918841682 wait complete. 2024-09-13T18:33:35.349 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-09-13T18:33:35.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.584+0000 7fd9bcbc7640 1 -- 172.21.15.114:0/1352018250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9b8107720 msgr2=0x7fd9b8107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:35.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.584+0000 7fd9bcbc7640 1 --2- 172.21.15.114:0/1352018250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9b8107720 0x7fd9b8107b00 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd9b00091e0 tx=0x7fd9b002cb90 comp rx=0 tx=0).stop 2024-09-13T18:33:35.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.591+0000 7fd9bcbc7640 1 -- 172.21.15.114:0/1352018250 shutdown_connections 2024-09-13T18:33:35.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.591+0000 7fd9bcbc7640 1 --2- 172.21.15.114:0/1352018250 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9b8072b10 0x7fd9b80794d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.591+0000 7fd9bcbc7640 1 --2- 172.21.15.114:0/1352018250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9b8108040 0x7fd9b80723e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.591+0000 7fd9bcbc7640 1 --2- 172.21.15.114:0/1352018250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9b8107720 0x7fd9b8107b00 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.591+0000 7fd9bcbc7640 1 -- 172.21.15.114:0/1352018250 >> 172.21.15.114:0/1352018250 conn(0x7fd9b806afe0 msgr2=0x7fd9b806b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:35.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.596+0000 7fd9bcbc7640 1 -- 172.21.15.114:0/1352018250 shutdown_connections 2024-09-13T18:33:35.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.597+0000 7fd9bcbc7640 1 -- 172.21.15.114:0/1352018250 wait complete. 2024-09-13T18:33:35.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.598+0000 7fd9bcbc7640 1 Processor -- start 2024-09-13T18:33:35.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.599+0000 7fd9bcbc7640 1 -- start start 2024-09-13T18:33:35.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.599+0000 7fd9bcbc7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9b8072b10 0x7fd9b8078ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:35.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.599+0000 7fd9bcbc7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9b8107720 0x7fd9b80793e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:35.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.599+0000 7fd9bcbc7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9b8108040 0x7fd9b8073fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:35.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.599+0000 7fd9bcbc7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b8074730 con 0x7fd9b8072b10 2024-09-13T18:33:35.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.599+0000 7fd9bcbc7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b80748a0 con 0x7fd9b8108040 2024-09-13T18:33:35.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.599+0000 7fd9bcbc7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b8074a60 con 0x7fd9b8107720 2024-09-13T18:33:35.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.603+0000 7fd9b5d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9b8107720 0x7fd9b80793e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:35.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.603+0000 7fd9b6d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9b8108040 0x7fd9b8073fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:35.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.603+0000 7fd9b6d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9b8108040 0x7fd9b8073fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:40692/0 (socket says 172.21.15.114:40692) 2024-09-13T18:33:35.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.603+0000 7fd9b6d9d640 1 -- 172.21.15.114:0/1870851355 learned_addr learned my addr 172.21.15.114:0/1870851355 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:35.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.604+0000 7fd9b659c640 1 --2- 172.21.15.114:0/1870851355 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9b8072b10 0x7fd9b8078ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:35.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.606+0000 7fd9b6d9d640 1 -- 172.21.15.114:0/1870851355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9b8107720 msgr2=0x7fd9b80793e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:35.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.606+0000 7fd9b6d9d640 1 --2- 172.21.15.114:0/1870851355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9b8107720 0x7fd9b80793e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.606+0000 7fd9b6d9d640 1 -- 172.21.15.114:0/1870851355 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9b8072b10 msgr2=0x7fd9b8078ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:35.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.606+0000 7fd9b6d9d640 1 --2- 172.21.15.114:0/1870851355 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9b8072b10 0x7fd9b8078ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.606+0000 7fd9b6d9d640 1 -- 172.21.15.114:0/1870851355 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b0009900 con 0x7fd9b8108040 2024-09-13T18:33:35.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.606+0000 7fd9b6d9d640 1 --2- 172.21.15.114:0/1870851355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9b8108040 0x7fd9b8073fa0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fd9a800e970 tx=0x7fd9a800ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:35.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.610+0000 7fd9a77fe640 1 -- 172.21.15.114:0/1870851355 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a800cc60 con 0x7fd9b8108040 2024-09-13T18:33:35.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.611+0000 7fd9a77fe640 1 -- 172.21.15.114:0/1870851355 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd9a800cdc0 con 0x7fd9b8108040 2024-09-13T18:33:35.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.611+0000 7fd9a77fe640 1 -- 172.21.15.114:0/1870851355 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a8004d40 con 0x7fd9b8108040 2024-09-13T18:33:35.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.612+0000 7fd9bcbc7640 1 -- 172.21.15.114:0/1870851355 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9b8074d40 con 0x7fd9b8108040 2024-09-13T18:33:35.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.612+0000 7fd9bcbc7640 1 -- 172.21.15.114:0/1870851355 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9b8135880 con 0x7fd9b8108040 2024-09-13T18:33:35.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.617+0000 7fd9bf9c1740 1 Processor -- start 2024-09-13T18:33:35.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.623+0000 7fd9a77fe640 1 -- 172.21.15.114:0/1870851355 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fd9a8004ea0 con 0x7fd9b8108040 2024-09-13T18:33:35.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.626+0000 7fd9bf9c1740 1 -- start start 2024-09-13T18:33:35.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.627+0000 7fd9bf9c1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c82f2a790 0x564c82f2ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:35.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.627+0000 7fd9bf9c1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564c82f2b0b0 0x564c82f35410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:35.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.627+0000 7fd9bf9c1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c82f35950 0x564c82f37d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:35.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.627+0000 7fd9bf9c1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c82e94cc0 con 0x564c82f2b0b0 2024-09-13T18:33:35.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.627+0000 7fd9bf9c1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c82e92450 con 0x564c82f2a790 2024-09-13T18:33:35.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.627+0000 7fd9bf9c1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c82e8dd40 con 0x564c82f35950 2024-09-13T18:33:35.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.627+0000 7fd9a77fe640 1 --2- 172.21.15.114:0/1870851355 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fd9a00a1410 0x7fd9a00a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:35.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.628+0000 7fd9a77fe640 1 -- 172.21.15.114:0/1870851355 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fd9a8014070 con 0x7fd9b8108040 2024-09-13T18:33:35.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.628+0000 7fd9b5d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564c82f2b0b0 0x564c82f35410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:35.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.629+0000 7fd9b5d9b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564c82f2b0b0 0x564c82f35410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:52850/0 (socket says 172.21.15.114:52850) 2024-09-13T18:33:35.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.629+0000 7fd9b5d9b640 1 -- 172.21.15.114:0/2583725025 learned_addr learned my addr 172.21.15.114:0/2583725025 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:35.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.629+0000 7fd9b5d9b640 1 --2- 172.21.15.114:0/1870851355 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fd9a00a1410 0x7fd9a00a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:35.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.629+0000 7fd9b6d9d640 1 --2- 172.21.15.114:0/2583725025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c82f35950 0x564c82f37d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:35.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.631+0000 7fd9b6d9d640 1 -- 172.21.15.114:0/2583725025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c82f2a790 msgr2=0x564c82f2ab70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:33:35.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.631+0000 7fd9b6d9d640 1 --2- 172.21.15.114:0/2583725025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c82f2a790 0x564c82f2ab70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.631+0000 7fd9b6d9d640 1 -- 172.21.15.114:0/2583725025 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564c82f2b0b0 msgr2=0x564c82f35410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:35.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.631+0000 7fd9b6d9d640 1 --2- 172.21.15.114:0/2583725025 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564c82f2b0b0 0x564c82f35410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.631+0000 7fd9b6d9d640 1 -- 172.21.15.114:0/2583725025 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c82e55620 con 0x564c82f35950 2024-09-13T18:33:35.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.631+0000 7fd9b5d9b640 1 --2- 172.21.15.114:0/1870851355 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fd9a00a1410 0x7fd9a00a38d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac00a520 tx=0x7fd9ac00a7e0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:35.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.635+0000 7fd9b6d9d640 1 --2- 172.21.15.114:0/2583725025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c82f35950 0x564c82f37d40 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd9a80ca790 tx=0x7fd9a8099d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa62153c113baa4f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:35.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.635+0000 7fd9a4ff9640 1 -- 172.21.15.114:0/2583725025 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a809a850 con 0x564c82f35950 2024-09-13T18:33:35.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.635+0000 7fd9a4ff9640 1 -- 172.21.15.114:0/2583725025 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd9a809a150 con 0x564c82f35950 2024-09-13T18:33:35.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.635+0000 7fd9a4ff9640 1 -- 172.21.15.114:0/2583725025 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a8096860 con 0x564c82f35950 2024-09-13T18:33:35.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.636+0000 7fd9bf9c1740 1 -- 172.21.15.114:0/2583725025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c82f35950 msgr2=0x564c82f37d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:35.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.636+0000 7fd9bf9c1740 1 --2- 172.21.15.114:0/2583725025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c82f35950 0x564c82f37d40 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd9a80ca790 tx=0x7fd9a8099d50 comp rx=0 tx=0).stop 2024-09-13T18:33:35.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.638+0000 7fd9bf9c1740 1 -- 172.21.15.114:0/2583725025 shutdown_connections 2024-09-13T18:33:35.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.638+0000 7fd9bf9c1740 1 --2- 172.21.15.114:0/2583725025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c82f35950 0x564c82f37d40 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.639+0000 7fd9bf9c1740 1 --2- 172.21.15.114:0/2583725025 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564c82f2b0b0 0x564c82f35410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.639+0000 7fd9bf9c1740 1 --2- 172.21.15.114:0/2583725025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c82f2a790 0x564c82f2ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.639+0000 7fd9bf9c1740 1 -- 172.21.15.114:0/2583725025 >> 172.21.15.114:0/2583725025 conn(0x564c82f26d00 msgr2=0x564c82f270e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:35.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.644+0000 7fd9bf9c1740 1 -- 172.21.15.114:0/2583725025 shutdown_connections 2024-09-13T18:33:35.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.644+0000 7fd9bf9c1740 1 -- 172.21.15.114:0/2583725025 wait complete. 2024-09-13T18:33:35.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.645+0000 7fd9bf9c1740 1 -- start start 2024-09-13T18:33:35.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.645+0000 7fd9bf9c1740 1 Processor -- start 2024-09-13T18:33:35.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.645+0000 7fd9bf9c1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c82f2a790 0x564c82f34f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:35.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.646+0000 7fd9bf9c1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564c82f2b0b0 0x564c82f322a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:35.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.646+0000 7fd9b659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c82f2a790 0x564c82f34f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:35.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.646+0000 7fd9b659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c82f2a790 0x564c82f34f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:40718/0 (socket says 172.21.15.114:40718) 2024-09-13T18:33:35.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.646+0000 7fd9b659c640 1 -- 172.21.15.114:0/3054671069 learned_addr learned my addr 172.21.15.114:0/3054671069 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:35.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.646+0000 7fd9bf9c1740 1 --2- 172.21.15.114:0/3054671069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c82f35950 0x564c82f327e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:35.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.646+0000 7fd9bf9c1740 1 -- 172.21.15.114:0/3054671069 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c82e4c180 con 0x564c82f2b0b0 2024-09-13T18:33:35.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.646+0000 7fd9bf9c1740 1 -- 172.21.15.114:0/3054671069 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c82b66430 con 0x564c82f2a790 2024-09-13T18:33:35.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.646+0000 7fd9bf9c1740 1 -- 172.21.15.114:0/3054671069 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c82e115e0 con 0x564c82f35950 2024-09-13T18:33:35.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.646+0000 7fd9b6d9d640 1 --2- 172.21.15.114:0/3054671069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c82f35950 0x564c82f327e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:35.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.647+0000 7fd9b659c640 1 -- 172.21.15.114:0/3054671069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c82f35950 msgr2=0x564c82f327e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:35.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.647+0000 7fd9b659c640 1 --2- 172.21.15.114:0/3054671069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c82f35950 0x564c82f327e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.647+0000 7fd9b659c640 1 -- 172.21.15.114:0/3054671069 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564c82f2b0b0 msgr2=0x564c82f322a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:35.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.647+0000 7fd9b659c640 1 --2- 172.21.15.114:0/3054671069 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564c82f2b0b0 0x564c82f322a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.647+0000 7fd9b659c640 1 -- 172.21.15.114:0/3054671069 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9b8135880 con 0x564c82f2a790 2024-09-13T18:33:35.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.647+0000 7fd9bf9c1740 10 client.0 _ll_register_callbacks cb 0x564c82e7daa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:33:35.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.647+0000 7fd9b659c640 1 --2- 172.21.15.114:0/3054671069 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c82f2a790 0x564c82f34f30 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fd9b000f2e0 tx=0x7fd9b000f310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bf5f80413057424 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:35.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.648+0000 7fd9bf9c1740 1 -- 172.21.15.114:0/3054671069 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564c82eb7b80 con 0x564c82f2a790 2024-09-13T18:33:35.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.648+0000 7fd99dffb640 10 client.28043 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:33:35.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.648+0000 7fd99dffb640 1 -- 172.21.15.114:0/3054671069 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b0003d80 con 0x564c82f2a790 2024-09-13T18:33:35.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.648+0000 7fd99dffb640 1 -- 172.21.15.114:0/3054671069 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fd9b0032070 con 0x564c82f2a790 2024-09-13T18:33:35.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.648+0000 7fd99dffb640 1 -- 172.21.15.114:0/3054671069 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9b000f4b0 con 0x564c82f2a790 2024-09-13T18:33:35.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.648+0000 7fd9bf9c1740 10 client.28043 fetch_fsmap learned FSMap version 37 2024-09-13T18:33:35.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.648+0000 7fd9bf9c1740 1 -- 172.21.15.114:0/3054671069 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({fsmap=37}) v3 -- 0x564c82e8c3c0 con 0x564c82f2a790 2024-09-13T18:33:35.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.653+0000 7fd99dffb640 1 -- 172.21.15.114:0/3054671069 <== mon.2 v2:172.21.15.134:3300/0 4 ==== fsmap(e 37) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fd9b003e040 con 0x564c82f2a790 2024-09-13T18:33:35.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.653+0000 7fd9bf9c1740 10 client.28043 fetch_fsmap finished waiting for FSMap version 37 2024-09-13T18:33:35.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.653+0000 7fd9bf9c1740 10 client.28043 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:33:35.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.653+0000 7fd9bf9c1740 20 client.28043 populate_metadata read hostname 'smithi114' 2024-09-13T18:33:35.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.653+0000 7fd9bf9c1740 1 --2- 172.21.15.114:0/3054671069 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x564c82f451c0 0x564c82f475e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:35.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.653+0000 7fd9bf9c1740 4 client.28043 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:33:35.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.653+0000 7fd9bf9c1740 1 -- 172.21.15.114:0/3054671069 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564c82eb7b80 con 0x564c82f451c0 2024-09-13T18:33:35.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.659+0000 7fd9b5d9b640 1 --2- 172.21.15.114:0/3054671069 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x564c82f451c0 0x564c82f475e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:35.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.660+0000 7fd9b5d9b640 1 --2- 172.21.15.114:0/3054671069 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x564c82f451c0 0x564c82f475e0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=792dbb59ee18fa8f server_cookie=635d2c83e6b7d6ad in_seq=0 out_seq=0 2024-09-13T18:33:35.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.660+0000 7fd99dffb640 10 client.28043 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:33:35.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.661+0000 7fd99dffb640 1 -- 172.21.15.114:0/3054671069 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x564c82eb7b80 con 0x564c82f451c0 2024-09-13T18:33:35.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.661+0000 7fd99dffb640 10 client.28043 handle_command_reply: tid=0 2024-09-13T18:33:35.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.662+0000 7fd9bf9c1740 1 client.28043 shutdown 2024-09-13T18:33:35.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.662+0000 7fd9bf9c1740 1 -- 172.21.15.114:0/3054671069 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c82f2a790 msgr2=0x564c82f34f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:35.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.662+0000 7fd9bf9c1740 1 --2- 172.21.15.114:0/3054671069 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c82f2a790 0x564c82f34f30 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fd9b000f2e0 tx=0x7fd9b000f310 comp rx=0 tx=0).stop 2024-09-13T18:33:35.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.662+0000 7fd9bf9c1740 1 -- 172.21.15.114:0/3054671069 shutdown_connections 2024-09-13T18:33:35.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.662+0000 7fd9bf9c1740 1 --2- 172.21.15.114:0/3054671069 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x564c82f451c0 0x564c82f475e0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.662+0000 7fd9bf9c1740 1 --2- 172.21.15.114:0/3054671069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c82f35950 0x564c82f327e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.662+0000 7fd9bf9c1740 1 --2- 172.21.15.114:0/3054671069 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564c82f2b0b0 0x564c82f322a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.662+0000 7fd9bf9c1740 1 --2- 172.21.15.114:0/3054671069 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c82f2a790 0x564c82f34f30 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.662+0000 7fd9bf9c1740 1 -- 172.21.15.114:0/3054671069 >> 172.21.15.114:0/3054671069 conn(0x564c82f26d00 msgr2=0x564c82f25eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:35.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.662+0000 7fd99dffb640 0 client.28043 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:33:35.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.662+0000 7fd9bf9c1740 1 -- 172.21.15.114:0/3054671069 shutdown_connections 2024-09-13T18:33:35.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.662+0000 7fd9bf9c1740 1 -- 172.21.15.114:0/3054671069 wait complete. 2024-09-13T18:33:35.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.668+0000 7fd9bf9c1740 20 client.28043 trim_cache size 0 max 16384 2024-09-13T18:33:35.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.695+0000 7fd9bf9c1740 1 Processor -- start 2024-09-13T18:33:35.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.699+0000 7fd9bf9c1740 1 -- start start 2024-09-13T18:33:35.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.699+0000 7fd9bf9c1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c82f2a790 0x564c82f34a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:35.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.699+0000 7fd9bf9c1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564c82f2b0b0 0x564c82f3c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:35.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.700+0000 7fd9b659c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c82f2a790 0x564c82f34a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:35.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.700+0000 7fd9b659c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c82f2a790 0x564c82f34a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:59760/0 (socket says 172.21.15.114:59760) 2024-09-13T18:33:35.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.700+0000 7fd9b659c640 1 -- 172.21.15.114:0/3808708569 learned_addr learned my addr 172.21.15.114:0/3808708569 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:35.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.700+0000 7fd9bf9c1740 1 --2- 172.21.15.114:0/3808708569 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c82f35950 0x564c82f3cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:35.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.700+0000 7fd9bf9c1740 1 -- 172.21.15.114:0/3808708569 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c82f27c60 con 0x564c82f2b0b0 2024-09-13T18:33:35.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.700+0000 7fd9bf9c1740 1 -- 172.21.15.114:0/3808708569 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c82f333c0 con 0x564c82f35950 2024-09-13T18:33:35.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.700+0000 7fd9bf9c1740 1 -- 172.21.15.114:0/3808708569 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c82f27df0 con 0x564c82f2a790 2024-09-13T18:33:35.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.701+0000 7fd9b5d9b640 1 --2- 172.21.15.114:0/3808708569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564c82f2b0b0 0x564c82f3c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:35.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.701+0000 7fd9b5d9b640 1 -- 172.21.15.114:0/3808708569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c82f2a790 msgr2=0x564c82f34a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:35.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.701+0000 7fd9b5d9b640 1 --2- 172.21.15.114:0/3808708569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c82f2a790 0x564c82f34a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.701+0000 7fd9b5d9b640 1 -- 172.21.15.114:0/3808708569 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c82f35950 msgr2=0x564c82f3cb50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:33:35.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.701+0000 7fd9b5d9b640 1 --2- 172.21.15.114:0/3808708569 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c82f35950 0x564c82f3cb50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.701+0000 7fd9b5d9b640 1 -- 172.21.15.114:0/3808708569 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c82e2e770 con 0x564c82f2b0b0 2024-09-13T18:33:35.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.702+0000 7fd9b5d9b640 1 --2- 172.21.15.114:0/3808708569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564c82f2b0b0 0x564c82f3c610 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x564c82f33870 tx=0x7fd9ac014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9faa37d04200baf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:35.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.702+0000 7fd9a57fa640 1 -- 172.21.15.114:0/3808708569 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac022040 con 0x564c82f2b0b0 2024-09-13T18:33:35.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.703+0000 7fd9a57fa640 1 -- 172.21.15.114:0/3808708569 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd9ac024040 con 0x564c82f2b0b0 2024-09-13T18:33:35.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.704+0000 7fd9bf9c1740 1 -- 172.21.15.114:0/3808708569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564c82f2b0b0 msgr2=0x564c82f3c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:35.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.704+0000 7fd9bf9c1740 1 --2- 172.21.15.114:0/3808708569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564c82f2b0b0 0x564c82f3c610 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x564c82f33870 tx=0x7fd9ac014000 comp rx=0 tx=0).stop 2024-09-13T18:33:35.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.704+0000 7fd9bf9c1740 1 -- 172.21.15.114:0/3808708569 shutdown_connections 2024-09-13T18:33:35.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.704+0000 7fd9bf9c1740 1 --2- 172.21.15.114:0/3808708569 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c82f35950 0x564c82f3cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.704+0000 7fd9bf9c1740 1 --2- 172.21.15.114:0/3808708569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564c82f2b0b0 0x564c82f3c610 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.704+0000 7fd9bf9c1740 1 --2- 172.21.15.114:0/3808708569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c82f2a790 0x564c82f34a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.705+0000 7fd9bf9c1740 1 -- 172.21.15.114:0/3808708569 >> 172.21.15.114:0/3808708569 conn(0x564c82f26d00 msgr2=0x564c82f30b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:35.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.705+0000 7fd9bf9c1740 1 -- 172.21.15.114:0/3808708569 shutdown_connections 2024-09-13T18:33:35.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.705+0000 7fd9bf9c1740 1 -- 172.21.15.114:0/3808708569 wait complete. 2024-09-13T18:33:35.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.705+0000 7fd9bf9c1740 1 -- start start 2024-09-13T18:33:35.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.706+0000 7fd9bf9c1740 1 Processor -- start 2024-09-13T18:33:35.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.706+0000 7fd9bf9c1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c82f2a790 0x564c82f2eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:35.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.706+0000 7fd9bf9c1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564c82f2b0b0 0x564c82f2f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:35.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.707+0000 7fd9b659c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c82f2a790 0x564c82f2eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:35.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.707+0000 7fd9b659c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c82f2a790 0x564c82f2eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:59762/0 (socket says 172.21.15.114:59762) 2024-09-13T18:33:35.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.707+0000 7fd9b659c640 1 -- 172.21.15.114:0/1964986833 learned_addr learned my addr 172.21.15.114:0/1964986833 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:35.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.707+0000 7fd9bf9c1740 1 --2- 172.21.15.114:0/1964986833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c82f35950 0x564c82f2f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:35.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.707+0000 7fd9bf9c1740 1 -- 172.21.15.114:0/1964986833 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c82f3cf40 con 0x564c82f2b0b0 2024-09-13T18:33:35.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.707+0000 7fd9bf9c1740 1 -- 172.21.15.114:0/1964986833 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c82f9b700 con 0x564c82f35950 2024-09-13T18:33:35.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.707+0000 7fd9b6d9d640 1 --2- 172.21.15.114:0/1964986833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c82f35950 0x564c82f2f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:35.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.707+0000 7fd9bf9c1740 1 -- 172.21.15.114:0/1964986833 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c82f9b840 con 0x564c82f2a790 2024-09-13T18:33:35.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.708+0000 7fd9bf9c1740 10 client.0 _ll_register_callbacks cb 0x7fd9a80046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:33:35.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.708+0000 7fd9b659c640 1 -- 172.21.15.114:0/1964986833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c82f35950 msgr2=0x564c82f2f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:35.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.708+0000 7fd9b659c640 1 --2- 172.21.15.114:0/1964986833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c82f35950 0x564c82f2f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.708+0000 7fd9b659c640 1 -- 172.21.15.114:0/1964986833 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564c82f2b0b0 msgr2=0x564c82f2f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:35.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.708+0000 7fd9b659c640 1 --2- 172.21.15.114:0/1964986833 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564c82f2b0b0 0x564c82f2f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.708+0000 7fd9b659c640 1 -- 172.21.15.114:0/1964986833 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9ac019040 con 0x564c82f2a790 2024-09-13T18:33:35.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.708+0000 7fd9b6d9d640 1 --2- 172.21.15.114:0/1964986833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c82f35950 0x564c82f2f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:33:35.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.708+0000 7fd9b659c640 1 --2- 172.21.15.114:0/1964986833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c82f2a790 0x564c82f2eda0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0003d90 tx=0x7fd9b002cb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b29e46169fb9eda server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:35.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.710+0000 7fd99dffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:33:35.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.710+0000 7fd99dffb640 1 -- 172.21.15.114:0/1964986833 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b0036a10 con 0x564c82f2a790 2024-09-13T18:33:35.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.710+0000 7fd99dffb640 1 -- 172.21.15.114:0/1964986833 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fd9b0032070 con 0x564c82f2a790 2024-09-13T18:33:35.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.710+0000 7fd9bf9c1740 1 -- 172.21.15.114:0/1964986833 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564c82e801f0 con 0x564c82f2a790 2024-09-13T18:33:35.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.712+0000 7fd99dffb640 1 -- 172.21.15.114:0/1964986833 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9b0036420 con 0x564c82f2a790 2024-09-13T18:33:35.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.717+0000 7fd9bf9c1740 10 client.27970 fetch_fsmap learned FSMap version 37 2024-09-13T18:33:35.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.717+0000 7fd9bf9c1740 1 -- 172.21.15.114:0/1964986833 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x564c82e53e80 con 0x564c82f2a790 2024-09-13T18:33:35.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.719+0000 7fd99dffb640 1 -- 172.21.15.114:0/1964986833 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564c82e801f0 con 0x564c82f2a790 2024-09-13T18:33:35.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.719+0000 7fd99dffb640 1 -- 172.21.15.114:0/1964986833 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 37) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fd9b005d070 con 0x564c82f2a790 2024-09-13T18:33:35.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.722+0000 7fd9bf9c1740 10 client.27970 fetch_fsmap finished waiting for FSMap version 37 2024-09-13T18:33:35.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.722+0000 7fd9bf9c1740 10 client.27970 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:33:35.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.722+0000 7fd9bf9c1740 20 client.27970 populate_metadata read hostname 'smithi114' 2024-09-13T18:33:35.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.722+0000 7fd9bf9c1740 1 --2- 172.21.15.114:0/1964986833 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x564c82f9ce00 0x564c82f9f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:35.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.722+0000 7fd9bf9c1740 4 client.27970 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:33:35.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.722+0000 7fd9bf9c1740 1 -- 172.21.15.114:0/1964986833 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564c82b814e0 con 0x564c82f9ce00 2024-09-13T18:33:35.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.722+0000 7fd9b5d9b640 1 --2- 172.21.15.114:0/1964986833 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x564c82f9ce00 0x564c82f9f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:35.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.733+0000 7fd9b5d9b640 1 --2- 172.21.15.114:0/1964986833 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x564c82f9ce00 0x564c82f9f1f0 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a7c782559774ec1 server_cookie=32c33c5d1453c140 in_seq=0 out_seq=0 2024-09-13T18:33:35.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.733+0000 7fd99dffb640 10 client.27970 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:33:35.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.734+0000 7fd99dffb640 1 -- 172.21.15.114:0/1964986833 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564c82b814e0 con 0x564c82f9ce00 2024-09-13T18:33:35.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.734+0000 7fd99dffb640 10 client.27970 handle_command_reply: tid=0 2024-09-13T18:33:35.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.738+0000 7fd9bf9c1740 1 client.27970 shutdown 2024-09-13T18:33:35.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.745+0000 7fd9bf9c1740 1 -- 172.21.15.114:0/1964986833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c82f2a790 msgr2=0x564c82f2eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:35.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.745+0000 7fd9bf9c1740 1 --2- 172.21.15.114:0/1964986833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c82f2a790 0x564c82f2eda0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0003d90 tx=0x7fd9b002cb90 comp rx=0 tx=0).stop 2024-09-13T18:33:35.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.747+0000 7fd9bf9c1740 1 -- 172.21.15.114:0/1964986833 shutdown_connections 2024-09-13T18:33:35.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.747+0000 7fd9bf9c1740 1 --2- 172.21.15.114:0/1964986833 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x564c82f9ce00 0x564c82f9f1f0 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.747+0000 7fd9bf9c1740 1 --2- 172.21.15.114:0/1964986833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c82f35950 0x564c82f2f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.747+0000 7fd9bf9c1740 1 --2- 172.21.15.114:0/1964986833 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564c82f2b0b0 0x564c82f2f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.747+0000 7fd9bf9c1740 1 --2- 172.21.15.114:0/1964986833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c82f2a790 0x564c82f2eda0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.747+0000 7fd9bf9c1740 1 -- 172.21.15.114:0/1964986833 >> 172.21.15.114:0/1964986833 conn(0x564c82f26d00 msgr2=0x564c82f2d360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:35.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.749+0000 7fd99dffb640 0 client.27970 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:33:35.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.749+0000 7fd9bf9c1740 1 -- 172.21.15.114:0/1964986833 shutdown_connections 2024-09-13T18:33:35.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.750+0000 7fd9bf9c1740 1 -- 172.21.15.114:0/1964986833 wait complete. 2024-09-13T18:33:35.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.754+0000 7fd9bf9c1740 20 client.27970 trim_cache size 0 max 16384 2024-09-13T18:33:35.756 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:33:35.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.761+0000 7fd9bcbc7640 1 -- 172.21.15.114:0/1870851355 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fd9a00a1410 msgr2=0x7fd9a00a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:35.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.761+0000 7fd9bcbc7640 1 --2- 172.21.15.114:0/1870851355 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fd9a00a1410 0x7fd9a00a38d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac00a520 tx=0x7fd9ac00a7e0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.761+0000 7fd9bcbc7640 1 -- 172.21.15.114:0/1870851355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9b8108040 msgr2=0x7fd9b8073fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:35.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.761+0000 7fd9bcbc7640 1 --2- 172.21.15.114:0/1870851355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9b8108040 0x7fd9b8073fa0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fd9a800e970 tx=0x7fd9a800ee30 comp rx=0 tx=0).stop 2024-09-13T18:33:35.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.764+0000 7fd9bcbc7640 1 -- 172.21.15.114:0/1870851355 shutdown_connections 2024-09-13T18:33:35.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.764+0000 7fd9bcbc7640 1 --2- 172.21.15.114:0/1870851355 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fd9a00a1410 0x7fd9a00a38d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.764+0000 7fd9bcbc7640 1 --2- 172.21.15.114:0/1870851355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9b8108040 0x7fd9b8073fa0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.764+0000 7fd9bcbc7640 1 --2- 172.21.15.114:0/1870851355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9b8107720 0x7fd9b80793e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.764+0000 7fd9bcbc7640 1 --2- 172.21.15.114:0/1870851355 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9b8072b10 0x7fd9b8078ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:35.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.764+0000 7fd9bcbc7640 1 -- 172.21.15.114:0/1870851355 >> 172.21.15.114:0/1870851355 conn(0x7fd9b806afe0 msgr2=0x7fd9b80709f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:35.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.772+0000 7fd9bcbc7640 1 -- 172.21.15.114:0/1870851355 shutdown_connections 2024-09-13T18:33:35.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:35.775+0000 7fd9bcbc7640 1 -- 172.21.15.114:0/1870851355 wait complete. 2024-09-13T18:33:36.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:36 smithi134 ceph-mon[34283]: pgmap v346: 129 pgs: 129 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 28 MiB/s wr, 614 op/s 2024-09-13T18:33:36.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:36 smithi204 ceph-mon[34777]: pgmap v346: 129 pgs: 129 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 28 MiB/s wr, 614 op/s 2024-09-13T18:33:36.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:36 smithi114 ceph-mon[30897]: pgmap v346: 129 pgs: 129 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 28 MiB/s wr, 614 op/s 2024-09-13T18:33:36.814 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-13T18:33:36.814 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-09-13T18:33:37.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.060+0000 7f8c1d441640 1 -- 172.21.15.114:0/707125107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c18107890 msgr2=0x7f8c18107cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:37.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.060+0000 7f8c1d441640 1 --2- 172.21.15.114:0/707125107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c18107890 0x7f8c18107cd0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8c10009ca0 tx=0x7f8c1002a8b0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.062+0000 7f8c1d441640 1 -- 172.21.15.114:0/707125107 shutdown_connections 2024-09-13T18:33:37.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.062+0000 7f8c1d441640 1 --2- 172.21.15.114:0/707125107 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c18119400 0x7f8c1811b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.062+0000 7f8c1d441640 1 --2- 172.21.15.114:0/707125107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c18107890 0x7f8c18107cd0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.062+0000 7f8c1d441640 1 --2- 172.21.15.114:0/707125107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c181074b0 0x7f8c1810e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.062+0000 7f8c1d441640 1 -- 172.21.15.114:0/707125107 >> 172.21.15.114:0/707125107 conn(0x7f8c1806af40 msgr2=0x7f8c1806b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:37.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.066+0000 7f8c1d441640 1 -- 172.21.15.114:0/707125107 shutdown_connections 2024-09-13T18:33:37.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.071+0000 7f8c1d441640 1 -- 172.21.15.114:0/707125107 wait complete. 2024-09-13T18:33:37.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.071+0000 7f8c1d441640 1 Processor -- start 2024-09-13T18:33:37.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.077+0000 7f8c1d441640 1 -- start start 2024-09-13T18:33:37.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.077+0000 7f8c1d441640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c181074b0 0x7f8c181188c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:37.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.077+0000 7f8c1d441640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c18107890 0x7f8c18118e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:37.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.077+0000 7f8c1d441640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c18119400 0x7f8c18111b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:37.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.077+0000 7f8c1d441640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c181122b0 con 0x7f8c18119400 2024-09-13T18:33:37.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.077+0000 7f8c1d441640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c18112420 con 0x7f8c181074b0 2024-09-13T18:33:37.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.077+0000 7f8c1d441640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c181125e0 con 0x7f8c18107890 2024-09-13T18:33:37.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.078+0000 7f8c177fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c18107890 0x7f8c18118e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:37.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.079+0000 7f8c177fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c18107890 0x7f8c18118e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:59768/0 (socket says 172.21.15.114:59768) 2024-09-13T18:33:37.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.079+0000 7f8c177fe640 1 -- 172.21.15.114:0/1183965772 learned_addr learned my addr 172.21.15.114:0/1183965772 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:37.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.079+0000 7f8c1cc40640 1 --2- 172.21.15.114:0/1183965772 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c18119400 0x7f8c18111b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:37.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.081+0000 7f8c177fe640 1 -- 172.21.15.114:0/1183965772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c181074b0 msgr2=0x7f8c181188c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:33:37.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.081+0000 7f8c177fe640 1 --2- 172.21.15.114:0/1183965772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c181074b0 0x7f8c181188c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.081+0000 7f8c177fe640 1 -- 172.21.15.114:0/1183965772 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c18119400 msgr2=0x7f8c18111b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:37.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.081+0000 7f8c177fe640 1 --2- 172.21.15.114:0/1183965772 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c18119400 0x7f8c18111b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.081+0000 7f8c177fe640 1 -- 172.21.15.114:0/1183965772 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c10009900 con 0x7f8c18107890 2024-09-13T18:33:37.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.087+0000 7f8c177fe640 1 --2- 172.21.15.114:0/1183965772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c18107890 0x7f8c18118e20 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f8c1002aea0 tx=0x7f8c10009870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:37.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.090+0000 7f8c157fa640 1 -- 172.21.15.114:0/1183965772 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c10004040 con 0x7f8c18107890 2024-09-13T18:33:37.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.090+0000 7f8c1d441640 1 -- 172.21.15.114:0/1183965772 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c18116450 con 0x7f8c18107890 2024-09-13T18:33:37.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.090+0000 7f8c1d441640 1 -- 172.21.15.114:0/1183965772 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c181169b0 con 0x7f8c18107890 2024-09-13T18:33:37.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.094+0000 7f8c213c9740 1 Processor -- start 2024-09-13T18:33:37.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.094+0000 7f8c157fa640 1 -- 172.21.15.114:0/1183965772 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8c10004630 con 0x7f8c18107890 2024-09-13T18:33:37.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.094+0000 7f8c157fa640 1 -- 172.21.15.114:0/1183965772 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c10045b90 con 0x7f8c18107890 2024-09-13T18:33:37.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.095+0000 7f8c213c9740 1 -- start start 2024-09-13T18:33:37.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.095+0000 7f8c213c9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559e3fc5e9a0 0x559e3fc5ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:37.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.095+0000 7f8c213c9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e3fc5f2c0 0x559e3fc69620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:37.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.095+0000 7f8c213c9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e3fc69b60 0x559e3fc6bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:37.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.095+0000 7f8c213c9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e3fbc9c60 con 0x559e3fc5e9a0 2024-09-13T18:33:37.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.095+0000 7f8c213c9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e3fbc73f0 con 0x559e3fc69b60 2024-09-13T18:33:37.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.095+0000 7f8c213c9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e3fbc2ce0 con 0x559e3fc5f2c0 2024-09-13T18:33:37.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.097+0000 7f8c177fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e3fc69b60 0x559e3fc6bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:37.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.097+0000 7f8c157fa640 1 -- 172.21.15.114:0/1183965772 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f8c10042070 con 0x7f8c18107890 2024-09-13T18:33:37.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.098+0000 7f8c157fa640 1 --2- 172.21.15.114:0/1183965772 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f8c000a1340 0x7f8c000a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:37.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.099+0000 7f8c157fa640 1 -- 172.21.15.114:0/1183965772 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f8c100ee3b0 con 0x7f8c18107890 2024-09-13T18:33:37.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.099+0000 7f8c177fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e3fc69b60 0x559e3fc6bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:40744/0 (socket says 172.21.15.114:40744) 2024-09-13T18:33:37.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.099+0000 7f8c177fe640 1 -- 172.21.15.114:0/1501493090 learned_addr learned my addr 172.21.15.114:0/1501493090 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:37.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.100+0000 7f8c177fe640 1 -- 172.21.15.114:0/1501493090 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e3fc5f2c0 msgr2=0x559e3fc69620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:33:37.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.100+0000 7f8c177fe640 1 --2- 172.21.15.114:0/1501493090 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e3fc5f2c0 0x559e3fc69620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.100+0000 7f8c177fe640 1 -- 172.21.15.114:0/1501493090 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559e3fc5e9a0 msgr2=0x559e3fc5ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:37.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.100+0000 7f8c177fe640 1 --2- 172.21.15.114:0/1501493090 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559e3fc5e9a0 0x559e3fc5ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.100+0000 7f8c177fe640 1 -- 172.21.15.114:0/1501493090 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e3fb7e970 con 0x559e3fc69b60 2024-09-13T18:33:37.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.101+0000 7f8c1cc40640 1 --2- 172.21.15.114:0/1183965772 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f8c000a1340 0x7f8c000a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:37.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.103+0000 7f8c177fe640 1 --2- 172.21.15.114:0/1501493090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e3fc69b60 0x559e3fc6bf50 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f8c10000f80 tx=0x7f8c10003f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f2a8b44679102fb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:37.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.103+0000 7f8bfe7fc640 1 -- 172.21.15.114:0/1501493090 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c100b5ea0 con 0x559e3fc69b60 2024-09-13T18:33:37.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.103+0000 7f8bfe7fc640 1 -- 172.21.15.114:0/1501493090 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8c100d1020 con 0x559e3fc69b60 2024-09-13T18:33:37.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.104+0000 7f8c213c9740 1 -- 172.21.15.114:0/1501493090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e3fc69b60 msgr2=0x559e3fc6bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:37.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.104+0000 7f8c213c9740 1 --2- 172.21.15.114:0/1501493090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e3fc69b60 0x559e3fc6bf50 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f8c10000f80 tx=0x7f8c10003f40 comp rx=0 tx=0).stop 2024-09-13T18:33:37.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.105+0000 7f8c213c9740 1 -- 172.21.15.114:0/1501493090 shutdown_connections 2024-09-13T18:33:37.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.105+0000 7f8c213c9740 1 --2- 172.21.15.114:0/1501493090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e3fc69b60 0x559e3fc6bf50 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.105+0000 7f8c213c9740 1 --2- 172.21.15.114:0/1501493090 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e3fc5f2c0 0x559e3fc69620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.105+0000 7f8c213c9740 1 --2- 172.21.15.114:0/1501493090 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559e3fc5e9a0 0x559e3fc5ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.105+0000 7f8c213c9740 1 -- 172.21.15.114:0/1501493090 >> 172.21.15.114:0/1501493090 conn(0x559e3fc5af30 msgr2=0x559e3fc5b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:37.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.110+0000 7f8c213c9740 1 -- 172.21.15.114:0/1501493090 shutdown_connections 2024-09-13T18:33:37.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.111+0000 7f8c1cc40640 1 --2- 172.21.15.114:0/1183965772 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f8c000a1340 0x7f8c000a3800 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c00a950 tx=0x7f8c0c006f10 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:37.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.111+0000 7f8c213c9740 1 -- 172.21.15.114:0/1501493090 wait complete. 2024-09-13T18:33:37.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.111+0000 7f8c213c9740 1 -- start start 2024-09-13T18:33:37.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.112+0000 7f8c213c9740 1 Processor -- start 2024-09-13T18:33:37.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.114+0000 7f8c213c9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e3fc5e9a0 0x559e3fc69460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:37.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.114+0000 7f8c213c9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559e3fc5f2c0 0x559e3fc66700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:37.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.114+0000 7f8c213c9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e3fc69b60 0x559e3fc66c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:37.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.114+0000 7f8c213c9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e3fb80f80 con 0x559e3fc5f2c0 2024-09-13T18:33:37.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.114+0000 7f8c213c9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e3fb80c20 con 0x559e3fc69b60 2024-09-13T18:33:37.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.114+0000 7f8c213c9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e3fb4dab0 con 0x559e3fc5e9a0 2024-09-13T18:33:37.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.114+0000 7f8c213c9740 10 client.0 _ll_register_callbacks cb 0x559e3fbbc350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:33:37.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.114+0000 7f8c177fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559e3fc5f2c0 0x559e3fc66700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:37.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.114+0000 7f8c177fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559e3fc5f2c0 0x559e3fc66700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:52912/0 (socket says 172.21.15.114:52912) 2024-09-13T18:33:37.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.114+0000 7f8c177fe640 1 -- 172.21.15.114:0/4049048603 learned_addr learned my addr 172.21.15.114:0/4049048603 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:37.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.115+0000 7f8c17fff640 1 --2- 172.21.15.114:0/4049048603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e3fc5e9a0 0x559e3fc69460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:37.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.115+0000 7f8c1cc40640 1 --2- 172.21.15.114:0/4049048603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e3fc69b60 0x559e3fc66c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:37.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.116+0000 7f8c177fe640 1 -- 172.21.15.114:0/4049048603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e3fc5e9a0 msgr2=0x559e3fc69460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:37.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.116+0000 7f8c177fe640 1 --2- 172.21.15.114:0/4049048603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e3fc5e9a0 0x559e3fc69460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.116+0000 7f8c177fe640 1 -- 172.21.15.114:0/4049048603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e3fc69b60 msgr2=0x559e3fc66c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:37.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.116+0000 7f8c177fe640 1 --2- 172.21.15.114:0/4049048603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e3fc69b60 0x559e3fc66c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.116+0000 7f8c177fe640 1 -- 172.21.15.114:0/4049048603 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c181169b0 con 0x559e3fc5f2c0 2024-09-13T18:33:37.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.116+0000 7f8c177fe640 1 --2- 172.21.15.114:0/4049048603 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559e3fc5f2c0 0x559e3fc66700 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f8c100bb160 tx=0x7f8c100ebba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bcad25c1247eb77 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:37.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.121+0000 7f8bf77fe640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:33:37.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.121+0000 7f8bf77fe640 1 -- 172.21.15.114:0/4049048603 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c100bb1d0 con 0x559e3fc5f2c0 2024-09-13T18:33:37.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.121+0000 7f8bf77fe640 1 -- 172.21.15.114:0/4049048603 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f8c100d2070 con 0x559e3fc5f2c0 2024-09-13T18:33:37.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.121+0000 7f8c213c9740 1 -- 172.21.15.114:0/4049048603 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559e3f8c7860 con 0x559e3fc5f2c0 2024-09-13T18:33:37.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.121+0000 7f8bf77fe640 1 -- 172.21.15.114:0/4049048603 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c100bf000 con 0x559e3fc5f2c0 2024-09-13T18:33:37.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.122+0000 7f8c213c9740 10 client.18576 fetch_fsmap learned FSMap version 37 2024-09-13T18:33:37.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.122+0000 7f8c213c9740 1 -- 172.21.15.114:0/4049048603 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x559e3fbc1360 con 0x559e3fc5f2c0 2024-09-13T18:33:37.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.122+0000 7f8bf77fe640 1 -- 172.21.15.114:0/4049048603 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559e3f8c7860 con 0x559e3fc5f2c0 2024-09-13T18:33:37.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.122+0000 7f8bf77fe640 1 -- 172.21.15.114:0/4049048603 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 37) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f8c100c31c0 con 0x559e3fc5f2c0 2024-09-13T18:33:37.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.122+0000 7f8c213c9740 10 client.18576 fetch_fsmap finished waiting for FSMap version 37 2024-09-13T18:33:37.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.122+0000 7f8c213c9740 10 client.18576 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:33:37.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.122+0000 7f8c213c9740 20 client.18576 populate_metadata read hostname 'smithi114' 2024-09-13T18:33:37.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.122+0000 7f8c213c9740 1 --2- 172.21.15.114:0/4049048603 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559e3fc793f0 0x559e3fc7b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:37.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.122+0000 7f8c213c9740 4 client.18576 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:33:37.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.122+0000 7f8c213c9740 1 -- 172.21.15.114:0/4049048603 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559e3f91aeb0 con 0x559e3fc793f0 2024-09-13T18:33:37.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.124+0000 7f8c17fff640 1 --2- 172.21.15.114:0/4049048603 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559e3fc793f0 0x559e3fc7b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:37.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.128+0000 7f8c17fff640 1 --2- 172.21.15.114:0/4049048603 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559e3fc793f0 0x559e3fc7b810 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78f0b7502364f312 server_cookie=819a989d12cac4de in_seq=0 out_seq=0 2024-09-13T18:33:37.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.129+0000 7f8bf77fe640 10 client.18576 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:33:37.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.130+0000 7f8bf77fe640 1 -- 172.21.15.114:0/4049048603 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x559e3f91aeb0 con 0x559e3fc793f0 2024-09-13T18:33:37.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.130+0000 7f8bf77fe640 10 client.18576 handle_command_reply: tid=0 2024-09-13T18:33:37.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.130+0000 7f8c213c9740 1 client.18576 shutdown 2024-09-13T18:33:37.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.141+0000 7f8c213c9740 1 -- 172.21.15.114:0/4049048603 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559e3fc5f2c0 msgr2=0x559e3fc66700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:37.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.141+0000 7f8c213c9740 1 --2- 172.21.15.114:0/4049048603 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559e3fc5f2c0 0x559e3fc66700 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f8c100bb160 tx=0x7f8c100ebba0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.145+0000 7f8c213c9740 1 -- 172.21.15.114:0/4049048603 shutdown_connections 2024-09-13T18:33:37.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.145+0000 7f8c213c9740 1 --2- 172.21.15.114:0/4049048603 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559e3fc793f0 0x559e3fc7b810 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.145+0000 7f8c213c9740 1 --2- 172.21.15.114:0/4049048603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e3fc69b60 0x559e3fc66c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.145+0000 7f8c213c9740 1 --2- 172.21.15.114:0/4049048603 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559e3fc5f2c0 0x559e3fc66700 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.145+0000 7f8c213c9740 1 --2- 172.21.15.114:0/4049048603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e3fc5e9a0 0x559e3fc69460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.145+0000 7f8c213c9740 1 -- 172.21.15.114:0/4049048603 >> 172.21.15.114:0/4049048603 conn(0x559e3fc5af30 msgr2=0x559e3fc60bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:37.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.145+0000 7f8bf77fe640 0 client.18576 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:33:37.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.147+0000 7f8c213c9740 1 -- 172.21.15.114:0/4049048603 shutdown_connections 2024-09-13T18:33:37.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.147+0000 7f8c213c9740 1 -- 172.21.15.114:0/4049048603 wait complete. 2024-09-13T18:33:37.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.155+0000 7f8c213c9740 20 client.18576 trim_cache size 0 max 16384 2024-09-13T18:33:37.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.195+0000 7f8c213c9740 1 Processor -- start 2024-09-13T18:33:37.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.196+0000 7f8c213c9740 1 -- start start 2024-09-13T18:33:37.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.197+0000 7f8c213c9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e3fc5e9a0 0x559e3fc64fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:37.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.197+0000 7f8c213c9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559e3fc5f2c0 0x559e3fc654f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:37.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.197+0000 7f8c213c9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e3fc69b60 0x559e3fc63940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:37.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.197+0000 7f8c213c9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e3fc5f890 con 0x559e3fc5f2c0 2024-09-13T18:33:37.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.197+0000 7f8c1cc40640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e3fc69b60 0x559e3fc63940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:37.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.198+0000 7f8c1cc40640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e3fc69b60 0x559e3fc63940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:40752/0 (socket says 172.21.15.114:40752) 2024-09-13T18:33:37.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.198+0000 7f8c1cc40640 1 -- 172.21.15.114:0/1973718617 learned_addr learned my addr 172.21.15.114:0/1973718617 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:37.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.197+0000 7f8c213c9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e3fc5e820 con 0x559e3fc69b60 2024-09-13T18:33:37.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.198+0000 7f8c213c9740 1 -- 172.21.15.114:0/1973718617 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e3fc755e0 con 0x559e3fc5e9a0 2024-09-13T18:33:37.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.198+0000 7f8c17fff640 1 --2- 172.21.15.114:0/1973718617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e3fc5e9a0 0x559e3fc64fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:37.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.198+0000 7f8c17fff640 1 -- 172.21.15.114:0/1973718617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e3fc69b60 msgr2=0x559e3fc63940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:37.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.199+0000 7f8c17fff640 1 --2- 172.21.15.114:0/1973718617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e3fc69b60 0x559e3fc63940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.199+0000 7f8c17fff640 1 -- 172.21.15.114:0/1973718617 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559e3fc5f2c0 msgr2=0x559e3fc654f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:37.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.199+0000 7f8c17fff640 1 --2- 172.21.15.114:0/1973718617 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559e3fc5f2c0 0x559e3fc654f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.199+0000 7f8c17fff640 1 -- 172.21.15.114:0/1973718617 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e3fb631c0 con 0x559e3fc5e9a0 2024-09-13T18:33:37.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.199+0000 7f8c17fff640 1 --2- 172.21.15.114:0/1973718617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e3fc5e9a0 0x559e3fc64fb0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x559e3fc7bdf0 tx=0x7f8c08005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b29908ec6fd05e40 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:37.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.202+0000 7f8bfeffd640 1 -- 172.21.15.114:0/1973718617 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c08012980 con 0x559e3fc5e9a0 2024-09-13T18:33:37.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.202+0000 7f8bfeffd640 1 -- 172.21.15.114:0/1973718617 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8c08019320 con 0x559e3fc5e9a0 2024-09-13T18:33:37.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.203+0000 7f8c213c9740 1 -- 172.21.15.114:0/1973718617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e3fc5e9a0 msgr2=0x559e3fc64fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:37.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.203+0000 7f8c213c9740 1 --2- 172.21.15.114:0/1973718617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e3fc5e9a0 0x559e3fc64fb0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x559e3fc7bdf0 tx=0x7f8c08005f90 comp rx=0 tx=0).stop 2024-09-13T18:33:37.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.207+0000 7f8c213c9740 1 -- 172.21.15.114:0/1973718617 shutdown_connections 2024-09-13T18:33:37.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.207+0000 7f8c213c9740 1 --2- 172.21.15.114:0/1973718617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e3fc69b60 0x559e3fc63940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.207+0000 7f8c213c9740 1 --2- 172.21.15.114:0/1973718617 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559e3fc5f2c0 0x559e3fc654f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.207+0000 7f8c213c9740 1 --2- 172.21.15.114:0/1973718617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e3fc5e9a0 0x559e3fc64fb0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.207+0000 7f8c213c9740 1 -- 172.21.15.114:0/1973718617 >> 172.21.15.114:0/1973718617 conn(0x559e3fc5af30 msgr2=0x559e3fc61a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:37.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.211+0000 7f8c213c9740 1 -- 172.21.15.114:0/1973718617 shutdown_connections 2024-09-13T18:33:37.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.211+0000 7f8c213c9740 1 -- 172.21.15.114:0/1973718617 wait complete. 2024-09-13T18:33:37.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.212+0000 7f8c213c9740 1 -- start start 2024-09-13T18:33:37.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.212+0000 7f8c213c9740 1 Processor -- start 2024-09-13T18:33:37.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.212+0000 7f8c213c9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559e3fc5e9a0 0x559e3fc6f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:37.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.212+0000 7f8c213c9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e3fc5f2c0 0x559e3fc6f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:37.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.212+0000 7f8c213c9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e3fc69b60 0x559e3fc6fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:37.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.212+0000 7f8c213c9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e3fccfac0 con 0x559e3fc5e9a0 2024-09-13T18:33:37.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.212+0000 7f8c213c9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e3fccfc30 con 0x559e3fc5f2c0 2024-09-13T18:33:37.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.212+0000 7f8c213c9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e3fccfdf0 con 0x559e3fc69b60 2024-09-13T18:33:37.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.212+0000 7f8c213c9740 10 client.0 _ll_register_callbacks cb 0x7f8c100b7b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:33:37.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.213+0000 7f8c177fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e3fc5f2c0 0x559e3fc6f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:37.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.213+0000 7f8c177fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e3fc5f2c0 0x559e3fc6f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:40760/0 (socket says 172.21.15.114:40760) 2024-09-13T18:33:37.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.213+0000 7f8c177fe640 1 -- 172.21.15.114:0/2745958637 learned_addr learned my addr 172.21.15.114:0/2745958637 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:37.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.213+0000 7f8c1cc40640 1 --2- 172.21.15.114:0/2745958637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e3fc69b60 0x559e3fc6fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:37.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.213+0000 7f8c17fff640 1 --2- 172.21.15.114:0/2745958637 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559e3fc5e9a0 0x559e3fc6f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:37.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.213+0000 7f8c177fe640 1 -- 172.21.15.114:0/2745958637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e3fc69b60 msgr2=0x559e3fc6fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:37.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.213+0000 7f8c177fe640 1 --2- 172.21.15.114:0/2745958637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e3fc69b60 0x559e3fc6fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.213+0000 7f8c177fe640 1 -- 172.21.15.114:0/2745958637 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559e3fc5e9a0 msgr2=0x559e3fc6f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:37.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.213+0000 7f8c177fe640 1 --2- 172.21.15.114:0/2745958637 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559e3fc5e9a0 0x559e3fc6f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.213+0000 7f8c177fe640 1 -- 172.21.15.114:0/2745958637 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c08009900 con 0x559e3fc5f2c0 2024-09-13T18:33:37.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.214+0000 7f8c177fe640 1 --2- 172.21.15.114:0/2745958637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e3fc5f2c0 0x559e3fc6f950 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8c100097b0 tx=0x7f8c100d9630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9aee3f50b3472ae1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:37.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.214+0000 7f8c17fff640 1 --2- 172.21.15.114:0/2745958637 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559e3fc5e9a0 0x559e3fc6f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:33:37.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.214+0000 7f8bf77fe640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:33:37.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.214+0000 7f8bf77fe640 1 -- 172.21.15.114:0/2745958637 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c100b5ea0 con 0x559e3fc5f2c0 2024-09-13T18:33:37.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.214+0000 7f8c213c9740 1 -- 172.21.15.114:0/2745958637 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559e3f8b6590 con 0x559e3fc5f2c0 2024-09-13T18:33:37.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.214+0000 7f8bf77fe640 1 -- 172.21.15.114:0/2745958637 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f8c100bb1d0 con 0x559e3fc5f2c0 2024-09-13T18:33:37.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.215+0000 7f8bf77fe640 1 -- 172.21.15.114:0/2745958637 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c10009540 con 0x559e3fc5f2c0 2024-09-13T18:33:37.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.217+0000 7f8c213c9740 10 client.28061 fetch_fsmap learned FSMap version 37 2024-09-13T18:33:37.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.217+0000 7f8c213c9740 1 -- 172.21.15.114:0/2745958637 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x559e3fb88990 con 0x559e3fc5f2c0 2024-09-13T18:33:37.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.221+0000 7f8bf77fe640 1 -- 172.21.15.114:0/2745958637 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559e3f8b6590 con 0x559e3fc5f2c0 2024-09-13T18:33:37.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.221+0000 7f8bf77fe640 1 -- 172.21.15.114:0/2745958637 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 37) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f8c100bb430 con 0x559e3fc5f2c0 2024-09-13T18:33:37.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.225+0000 7f8c213c9740 10 client.28061 fetch_fsmap finished waiting for FSMap version 37 2024-09-13T18:33:37.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.225+0000 7f8c213c9740 10 client.28061 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:33:37.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.225+0000 7f8c213c9740 20 client.28061 populate_metadata read hostname 'smithi114' 2024-09-13T18:33:37.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.225+0000 7f8c213c9740 1 --2- 172.21.15.114:0/2745958637 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559e3fcd10a0 0x559e3fcd3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:37.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.225+0000 7f8c213c9740 4 client.28061 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-13T18:33:37.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.225+0000 7f8c213c9740 1 -- 172.21.15.114:0/2745958637 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x559e3fcd3b60 con 0x559e3fcd10a0 2024-09-13T18:33:37.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.228+0000 7f8c17fff640 1 --2- 172.21.15.114:0/2745958637 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559e3fcd10a0 0x559e3fcd3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:37.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.229+0000 7f8c17fff640 1 --2- 172.21.15.114:0/2745958637 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559e3fcd10a0 0x559e3fcd3490 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d098b882729383ae server_cookie=31613512844284cc in_seq=0 out_seq=0 2024-09-13T18:33:37.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.229+0000 7f8bf77fe640 10 client.28061 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:33:37.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.230+0000 7f8bf77fe640 1 -- 172.21.15.114:0/2745958637 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x559e3fcd3b60 con 0x559e3fcd10a0 2024-09-13T18:33:37.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.230+0000 7f8bf77fe640 10 client.28061 handle_command_reply: tid=0 2024-09-13T18:33:37.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.231+0000 7f8c213c9740 1 client.28061 shutdown 2024-09-13T18:33:37.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.238+0000 7f8c213c9740 1 -- 172.21.15.114:0/2745958637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e3fc5f2c0 msgr2=0x559e3fc6f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:37.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.238+0000 7f8c213c9740 1 --2- 172.21.15.114:0/2745958637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e3fc5f2c0 0x559e3fc6f950 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8c100097b0 tx=0x7f8c100d9630 comp rx=0 tx=0).stop 2024-09-13T18:33:37.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.239+0000 7f8c213c9740 1 -- 172.21.15.114:0/2745958637 shutdown_connections 2024-09-13T18:33:37.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.239+0000 7f8c213c9740 1 --2- 172.21.15.114:0/2745958637 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559e3fcd10a0 0x559e3fcd3490 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.239+0000 7f8c213c9740 1 --2- 172.21.15.114:0/2745958637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e3fc69b60 0x559e3fc6fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.239+0000 7f8c213c9740 1 --2- 172.21.15.114:0/2745958637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e3fc5f2c0 0x559e3fc6f950 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.239+0000 7f8c213c9740 1 --2- 172.21.15.114:0/2745958637 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559e3fc5e9a0 0x559e3fc6f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.239+0000 7f8c213c9740 1 -- 172.21.15.114:0/2745958637 >> 172.21.15.114:0/2745958637 conn(0x559e3fc5af30 msgr2=0x559e3fc61a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:37.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.239+0000 7f8bf77fe640 0 client.28061 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:33:37.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.243+0000 7f8c213c9740 1 -- 172.21.15.114:0/2745958637 shutdown_connections 2024-09-13T18:33:37.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.243+0000 7f8c213c9740 1 -- 172.21.15.114:0/2745958637 wait complete. 2024-09-13T18:33:37.249 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:33:37.250 INFO:teuthology.orchestra.run.smithi114.stdout: "return_code": 0, 2024-09-13T18:33:37.250 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_tag": "d4eae42d-d6e7-478f-be69-6c4995a4bb2d", 2024-09-13T18:33:37.250 INFO:teuthology.orchestra.run.smithi114.stdout: "mode": "asynchronous" 2024-09-13T18:33:37.250 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:33:37.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.246+0000 7f8c213c9740 20 client.28061 trim_cache size 0 max 16384 2024-09-13T18:33:37.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.247+0000 7f8c1d441640 1 -- 172.21.15.114:0/1183965772 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f8c000a1340 msgr2=0x7f8c000a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:37.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.247+0000 7f8c1d441640 1 --2- 172.21.15.114:0/1183965772 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f8c000a1340 0x7f8c000a3800 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c00a950 tx=0x7f8c0c006f10 comp rx=0 tx=0).stop 2024-09-13T18:33:37.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.247+0000 7f8c1d441640 1 -- 172.21.15.114:0/1183965772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c18107890 msgr2=0x7f8c18118e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:37.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.247+0000 7f8c1d441640 1 --2- 172.21.15.114:0/1183965772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c18107890 0x7f8c18118e20 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f8c1002aea0 tx=0x7f8c10009870 comp rx=0 tx=0).stop 2024-09-13T18:33:37.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.248+0000 7f8c1d441640 1 -- 172.21.15.114:0/1183965772 shutdown_connections 2024-09-13T18:33:37.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.248+0000 7f8c1d441640 1 --2- 172.21.15.114:0/1183965772 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f8c000a1340 0x7f8c000a3800 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.248+0000 7f8c1d441640 1 --2- 172.21.15.114:0/1183965772 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c18119400 0x7f8c18111b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.248+0000 7f8c1d441640 1 --2- 172.21.15.114:0/1183965772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c18107890 0x7f8c18118e20 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.248+0000 7f8c1d441640 1 --2- 172.21.15.114:0/1183965772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c181074b0 0x7f8c181188c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.248+0000 7f8c1d441640 1 -- 172.21.15.114:0/1183965772 >> 172.21.15.114:0/1183965772 conn(0x7f8c1806af40 msgr2=0x7f8c18070120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:37.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.248+0000 7f8c1d441640 1 -- 172.21.15.114:0/1183965772 shutdown_connections 2024-09-13T18:33:37.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.250+0000 7f8c1d441640 1 -- 172.21.15.114:0/1183965772 wait complete. 2024-09-13T18:33:37.283 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T18:33:37.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.529+0000 7fbf09255640 1 -- 172.21.15.114:0/3679556277 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbefc0a47f0 msgr2=0x7fbefc0a4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:37.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.529+0000 7fbf09255640 1 --2- 172.21.15.114:0/3679556277 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbefc0a47f0 0x7fbefc0a4bd0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fbef0009860 tx=0x7fbef002a410 comp rx=0 tx=0).stop 2024-09-13T18:33:37.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.529+0000 7fbf09255640 1 -- 172.21.15.114:0/3679556277 shutdown_connections 2024-09-13T18:33:37.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.529+0000 7fbf09255640 1 --2- 172.21.15.114:0/3679556277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbefc0b7510 0x7fbefc0b9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.529+0000 7fbf09255640 1 --2- 172.21.15.114:0/3679556277 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbefc0a5110 0x7fbefc0b6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.529+0000 7fbf09255640 1 --2- 172.21.15.114:0/3679556277 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbefc0a47f0 0x7fbefc0a4bd0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.529+0000 7fbf09255640 1 -- 172.21.15.114:0/3679556277 >> 172.21.15.114:0/3679556277 conn(0x7fbefc01a0e0 msgr2=0x7fbefc01a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:37.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.530+0000 7fbf09255640 1 -- 172.21.15.114:0/3679556277 shutdown_connections 2024-09-13T18:33:37.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.530+0000 7fbf09255640 1 -- 172.21.15.114:0/3679556277 wait complete. 2024-09-13T18:33:37.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.531+0000 7fbf09255640 1 Processor -- start 2024-09-13T18:33:37.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.531+0000 7fbf09255640 1 -- start start 2024-09-13T18:33:37.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.532+0000 7fbf09255640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbefc0a47f0 0x7fbefc143090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:37.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.532+0000 7fbf09255640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbefc0a5110 0x7fbefc1435d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:37.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.532+0000 7fbf09255640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbefc0b7510 0x7fbefc1479c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:37.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.532+0000 7fbf03fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbefc0a47f0 0x7fbefc143090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:37.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.532+0000 7fbf03fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbefc0a47f0 0x7fbefc143090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:40774/0 (socket says 172.21.15.114:40774) 2024-09-13T18:33:37.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.532+0000 7fbf03fff640 1 -- 172.21.15.114:0/2041238967 learned_addr learned my addr 172.21.15.114:0/2041238967 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:37.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.532+0000 7fbf09255640 1 -- 172.21.15.114:0/2041238967 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbefc147fc0 con 0x7fbefc0a5110 2024-09-13T18:33:37.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.532+0000 7fbf09255640 1 -- 172.21.15.114:0/2041238967 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbefc148130 con 0x7fbefc0a47f0 2024-09-13T18:33:37.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.532+0000 7fbf09255640 1 -- 172.21.15.114:0/2041238967 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbefc1482f0 con 0x7fbefc0b7510 2024-09-13T18:33:37.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.532+0000 7fbf08a54640 1 --2- 172.21.15.114:0/2041238967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbefc0b7510 0x7fbefc1479c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:37.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.534+0000 7fbf08a54640 1 -- 172.21.15.114:0/2041238967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbefc0a47f0 msgr2=0x7fbefc143090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:37.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.534+0000 7fbf08a54640 1 --2- 172.21.15.114:0/2041238967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbefc0a47f0 0x7fbefc143090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.534+0000 7fbf08a54640 1 -- 172.21.15.114:0/2041238967 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbefc0a5110 msgr2=0x7fbefc1435d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:37.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.534+0000 7fbf08a54640 1 --2- 172.21.15.114:0/2041238967 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbefc0a5110 0x7fbefc1435d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.534+0000 7fbf08a54640 1 -- 172.21.15.114:0/2041238967 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef0009490 con 0x7fbefc0b7510 2024-09-13T18:33:37.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.537+0000 7fbf08a54640 1 --2- 172.21.15.114:0/2041238967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbefc0b7510 0x7fbefc1479c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fbf0404e950 tx=0x7fbf04071d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:37.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.539+0000 7fbf017fa640 1 -- 172.21.15.114:0/2041238967 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0406fcf0 con 0x7fbefc0b7510 2024-09-13T18:33:37.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.539+0000 7fbf09255640 1 -- 172.21.15.114:0/2041238967 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbefc1485d0 con 0x7fbefc0b7510 2024-09-13T18:33:37.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.539+0000 7fbf09255640 1 -- 172.21.15.114:0/2041238967 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbefc148820 con 0x7fbefc0b7510 2024-09-13T18:33:37.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.539+0000 7fbf017fa640 1 -- 172.21.15.114:0/2041238967 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbf0406fe50 con 0x7fbefc0b7510 2024-09-13T18:33:37.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.539+0000 7fbf017fa640 1 -- 172.21.15.114:0/2041238967 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0407b660 con 0x7fbefc0b7510 2024-09-13T18:33:37.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.550+0000 7fbf0d1dc740 1 Processor -- start 2024-09-13T18:33:37.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.552+0000 7fbf017fa640 1 -- 172.21.15.114:0/2041238967 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fbf04073430 con 0x7fbefc0b7510 2024-09-13T18:33:37.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.553+0000 7fbf017fa640 1 --2- 172.21.15.114:0/2041238967 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbef40a1410 0x7fbef40a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:37.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.553+0000 7fbf0d1dc740 1 -- start start 2024-09-13T18:33:37.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.554+0000 7fbf0d1dc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7e1bb3fe0 0x55d7e1bb43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:37.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.554+0000 7fbf0d1dc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d7e1bb4900 0x55d7e1bbec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:37.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.554+0000 7fbf0d1dc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7e1bbf1a0 0x55d7e1bc1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:37.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.554+0000 7fbf0d1dc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7e1b1ef20 con 0x55d7e1bb4900 2024-09-13T18:33:37.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.554+0000 7fbf0d1dc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7e1b1c6b0 con 0x55d7e1bb3fe0 2024-09-13T18:33:37.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.554+0000 7fbf0d1dc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7e1b17fa0 con 0x55d7e1bbf1a0 2024-09-13T18:33:37.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.554+0000 7fbf017fa640 1 -- 172.21.15.114:0/2041238967 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fbf0412f340 con 0x7fbefc0b7510 2024-09-13T18:33:37.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.555+0000 7fbf037fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d7e1bb4900 0x55d7e1bbec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:37.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.555+0000 7fbf037fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d7e1bb4900 0x55d7e1bbec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:52958/0 (socket says 172.21.15.114:52958) 2024-09-13T18:33:37.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.555+0000 7fbf037fe640 1 -- 172.21.15.114:0/4100837939 learned_addr learned my addr 172.21.15.114:0/4100837939 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:37.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.555+0000 7fbf037fe640 1 --2- 172.21.15.114:0/2041238967 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbef40a1410 0x7fbef40a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:37.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.556+0000 7fbf037fe640 1 -- 172.21.15.114:0/4100837939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7e1bbf1a0 msgr2=0x55d7e1bc1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:37.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.556+0000 7fbf037fe640 1 --2- 172.21.15.114:0/4100837939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7e1bbf1a0 0x55d7e1bc1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.556+0000 7fbf037fe640 1 -- 172.21.15.114:0/4100837939 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7e1bb3fe0 msgr2=0x55d7e1bb43c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:33:37.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.556+0000 7fbf037fe640 1 --2- 172.21.15.114:0/4100837939 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7e1bb3fe0 0x55d7e1bb43c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.556+0000 7fbf037fe640 1 -- 172.21.15.114:0/4100837939 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7e1ad2840 con 0x55d7e1bb4900 2024-09-13T18:33:37.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.557+0000 7fbf037fe640 1 --2- 172.21.15.114:0/2041238967 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbef40a1410 0x7fbef40a38d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbef8005d00 tx=0x7fbef800d660 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:37.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.557+0000 7fbf037fe640 1 --2- 172.21.15.114:0/4100837939 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d7e1bb4900 0x55d7e1bbec60 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fbef8012630 tx=0x7fbef8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fb5b95be86373d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:37.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.558+0000 7fbeee7fc640 1 -- 172.21.15.114:0/4100837939 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef801c8e0 con 0x55d7e1bb4900 2024-09-13T18:33:37.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.558+0000 7fbeee7fc640 1 -- 172.21.15.114:0/4100837939 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbef8018040 con 0x55d7e1bb4900 2024-09-13T18:33:37.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.559+0000 7fbf0d1dc740 1 -- 172.21.15.114:0/4100837939 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d7e1bb4900 msgr2=0x55d7e1bbec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:37.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.559+0000 7fbf0d1dc740 1 --2- 172.21.15.114:0/4100837939 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d7e1bb4900 0x55d7e1bbec60 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fbef8012630 tx=0x7fbef8019000 comp rx=0 tx=0).stop 2024-09-13T18:33:37.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.561+0000 7fbf0d1dc740 1 -- 172.21.15.114:0/4100837939 shutdown_connections 2024-09-13T18:33:37.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.561+0000 7fbf0d1dc740 1 --2- 172.21.15.114:0/4100837939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7e1bbf1a0 0x55d7e1bc1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.561+0000 7fbf0d1dc740 1 --2- 172.21.15.114:0/4100837939 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d7e1bb4900 0x55d7e1bbec60 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.561+0000 7fbf0d1dc740 1 --2- 172.21.15.114:0/4100837939 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7e1bb3fe0 0x55d7e1bb43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.561+0000 7fbf0d1dc740 1 -- 172.21.15.114:0/4100837939 >> 172.21.15.114:0/4100837939 conn(0x55d7e1bb0570 msgr2=0x55d7e1bb0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:37.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.567+0000 7fbf0d1dc740 1 -- 172.21.15.114:0/4100837939 shutdown_connections 2024-09-13T18:33:37.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.570+0000 7fbf0d1dc740 1 -- 172.21.15.114:0/4100837939 wait complete. 2024-09-13T18:33:37.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.570+0000 7fbf0d1dc740 1 -- start start 2024-09-13T18:33:37.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.571+0000 7fbf0d1dc740 1 Processor -- start 2024-09-13T18:33:37.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.572+0000 7fbf0d1dc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d7e1bb3fe0 0x55d7e1bbe9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:37.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.572+0000 7fbf0d1dc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7e1bb4900 0x55d7e1bbbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:37.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.572+0000 7fbf0d1dc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7e1bbf1a0 0x55d7e1bbc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:37.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.572+0000 7fbf0d1dc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7e1ab92a0 con 0x55d7e1bb3fe0 2024-09-13T18:33:37.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.572+0000 7fbf0d1dc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7e17f0430 con 0x55d7e1bbf1a0 2024-09-13T18:33:37.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.572+0000 7fbf0d1dc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7e1a9b680 con 0x55d7e1bb4900 2024-09-13T18:33:37.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.572+0000 7fbf0d1dc740 10 client.0 _ll_register_callbacks cb 0x55d7e1b11610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:33:37.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.572+0000 7fbf037fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7e1bb4900 0x55d7e1bbbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:37.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.572+0000 7fbf037fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7e1bb4900 0x55d7e1bbbcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:59830/0 (socket says 172.21.15.114:59830) 2024-09-13T18:33:37.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.572+0000 7fbf037fe640 1 -- 172.21.15.114:0/1934657398 learned_addr learned my addr 172.21.15.114:0/1934657398 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:37.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.572+0000 7fbf08a54640 1 --2- 172.21.15.114:0/1934657398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7e1bbf1a0 0x55d7e1bbc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:37.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.572+0000 7fbf037fe640 1 -- 172.21.15.114:0/1934657398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7e1bbf1a0 msgr2=0x55d7e1bbc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:37.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.572+0000 7fbf037fe640 1 --2- 172.21.15.114:0/1934657398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7e1bbf1a0 0x55d7e1bbc200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.572+0000 7fbf037fe640 1 -- 172.21.15.114:0/1934657398 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d7e1bb3fe0 msgr2=0x55d7e1bbe9f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:33:37.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.572+0000 7fbf037fe640 1 --2- 172.21.15.114:0/1934657398 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d7e1bb3fe0 0x55d7e1bbe9f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.572+0000 7fbf037fe640 1 -- 172.21.15.114:0/1934657398 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbef8012e10 con 0x55d7e1bb4900 2024-09-13T18:33:37.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.573+0000 7fbf037fe640 1 --2- 172.21.15.114:0/1934657398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7e1bb4900 0x55d7e1bbbcc0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fbef801f5f0 tx=0x7fbef801c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9000e22a13ff3d65 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:37.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.573+0000 7fbee37fe640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:33:37.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.573+0000 7fbee37fe640 1 -- 172.21.15.114:0/1934657398 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef800d930 con 0x55d7e1bb4900 2024-09-13T18:33:37.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.573+0000 7fbf0d1dc740 1 -- 172.21.15.114:0/1934657398 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d7e180b4e0 con 0x55d7e1bb4900 2024-09-13T18:33:37.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.573+0000 7fbee37fe640 1 -- 172.21.15.114:0/1934657398 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fbef800fe70 con 0x55d7e1bb4900 2024-09-13T18:33:37.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.574+0000 7fbee37fe640 1 -- 172.21.15.114:0/1934657398 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbef8036940 con 0x55d7e1bb4900 2024-09-13T18:33:37.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.577+0000 7fbf0d1dc740 10 client.27994 fetch_fsmap learned FSMap version 37 2024-09-13T18:33:37.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.577+0000 7fbf0d1dc740 1 -- 172.21.15.114:0/1934657398 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55d7e1b16620 con 0x55d7e1bb4900 2024-09-13T18:33:37.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.582+0000 7fbee37fe640 1 -- 172.21.15.114:0/1934657398 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d7e180b4e0 con 0x55d7e1bb4900 2024-09-13T18:33:37.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.582+0000 7fbee37fe640 1 -- 172.21.15.114:0/1934657398 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 37) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fbef8028630 con 0x55d7e1bb4900 2024-09-13T18:33:37.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.585+0000 7fbf0d1dc740 10 client.27994 fetch_fsmap finished waiting for FSMap version 37 2024-09-13T18:33:37.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.585+0000 7fbf0d1dc740 10 client.27994 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:33:37.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.585+0000 7fbf0d1dc740 20 client.27994 populate_metadata read hostname 'smithi114' 2024-09-13T18:33:37.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.585+0000 7fbf0d1dc740 1 --2- 172.21.15.114:0/1934657398 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55d7e1bced90 0x55d7e1bd11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:37.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.585+0000 7fbf0d1dc740 4 client.27994 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:33:37.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.585+0000 7fbf0d1dc740 1 -- 172.21.15.114:0/1934657398 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d7e181c7e0 con 0x55d7e1bced90 2024-09-13T18:33:37.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.585+0000 7fbf03fff640 1 --2- 172.21.15.114:0/1934657398 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55d7e1bced90 0x55d7e1bd11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:37.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.609+0000 7fbf03fff640 1 --2- 172.21.15.114:0/1934657398 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55d7e1bced90 0x55d7e1bd11b0 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d76cf4be86bfa807 server_cookie=914f5afa1c809adc in_seq=0 out_seq=0 2024-09-13T18:33:37.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.609+0000 7fbee37fe640 10 client.27994 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:33:37.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.613+0000 7fbee37fe640 1 -- 172.21.15.114:0/1934657398 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55d7e181c7e0 con 0x55d7e1bced90 2024-09-13T18:33:37.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.613+0000 7fbee37fe640 10 client.27994 handle_command_reply: tid=0 2024-09-13T18:33:37.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.613+0000 7fbf0d1dc740 1 client.27994 shutdown 2024-09-13T18:33:37.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:37 smithi134 ceph-mon[34283]: scrub summary: active paths [/] 2024-09-13T18:33:37.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:37 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:33:37.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.621+0000 7fbf0d1dc740 1 -- 172.21.15.114:0/1934657398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7e1bb4900 msgr2=0x55d7e1bbbcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:37.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.621+0000 7fbf0d1dc740 1 --2- 172.21.15.114:0/1934657398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7e1bb4900 0x55d7e1bbbcc0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fbef801f5f0 tx=0x7fbef801c910 comp rx=0 tx=0).stop 2024-09-13T18:33:37.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.621+0000 7fbf0d1dc740 1 -- 172.21.15.114:0/1934657398 shutdown_connections 2024-09-13T18:33:37.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.621+0000 7fbf0d1dc740 1 --2- 172.21.15.114:0/1934657398 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55d7e1bced90 0x55d7e1bd11b0 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.621+0000 7fbf0d1dc740 1 --2- 172.21.15.114:0/1934657398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7e1bbf1a0 0x55d7e1bbc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.621+0000 7fbf0d1dc740 1 --2- 172.21.15.114:0/1934657398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7e1bb4900 0x55d7e1bbbcc0 secure :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fbef801f5f0 tx=0x7fbef801c910 comp rx=0 tx=0).stop 2024-09-13T18:33:37.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.621+0000 7fbf0d1dc740 1 --2- 172.21.15.114:0/1934657398 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d7e1bb3fe0 0x55d7e1bbe9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.621+0000 7fbf0d1dc740 1 -- 172.21.15.114:0/1934657398 >> 172.21.15.114:0/1934657398 conn(0x55d7e1bb0570 msgr2=0x55d7e1baf760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:37.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.622+0000 7fbee37fe640 0 client.27994 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:33:37.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.625+0000 7fbf0d1dc740 1 -- 172.21.15.114:0/1934657398 shutdown_connections 2024-09-13T18:33:37.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.626+0000 7fbf0d1dc740 1 -- 172.21.15.114:0/1934657398 wait complete. 2024-09-13T18:33:37.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.628+0000 7fbf0d1dc740 20 client.27994 trim_cache size 0 max 16384 2024-09-13T18:33:37.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.667+0000 7fbf0d1dc740 1 Processor -- start 2024-09-13T18:33:37.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.667+0000 7fbf0d1dc740 1 -- start start 2024-09-13T18:33:37.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.668+0000 7fbf0d1dc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d7e1bb3fe0 0x55d7e1bb8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:37.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.668+0000 7fbf0d1dc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7e1bbf1a0 0x55d7e1bb9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:37.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.668+0000 7fbf0d1dc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7e1bb76d0 0x55d7e1bba570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:37.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.668+0000 7fbf0d1dc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7e1bbc800 con 0x55d7e1bb3fe0 2024-09-13T18:33:37.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.668+0000 7fbf0d1dc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7e1bbcde0 con 0x55d7e1bb76d0 2024-09-13T18:33:37.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.668+0000 7fbf0d1dc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7e1babee0 con 0x55d7e1bbf1a0 2024-09-13T18:33:37.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.668+0000 7fbf08a54640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7e1bb76d0 0x55d7e1bba570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:37.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.668+0000 7fbf08a54640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7e1bb76d0 0x55d7e1bba570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:40790/0 (socket says 172.21.15.114:40790) 2024-09-13T18:33:37.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.668+0000 7fbf08a54640 1 -- 172.21.15.114:0/3714986608 learned_addr learned my addr 172.21.15.114:0/3714986608 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:37.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.668+0000 7fbf037fe640 1 --2- 172.21.15.114:0/3714986608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7e1bbf1a0 0x55d7e1bb9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:37.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.669+0000 7fbf03fff640 1 --2- 172.21.15.114:0/3714986608 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d7e1bb3fe0 0x55d7e1bb8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:37.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.669+0000 7fbf08a54640 1 -- 172.21.15.114:0/3714986608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7e1bbf1a0 msgr2=0x55d7e1bb9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:37.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.669+0000 7fbf08a54640 1 --2- 172.21.15.114:0/3714986608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7e1bbf1a0 0x55d7e1bb9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.669+0000 7fbf08a54640 1 -- 172.21.15.114:0/3714986608 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d7e1bb3fe0 msgr2=0x55d7e1bb8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:37.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.669+0000 7fbf08a54640 1 --2- 172.21.15.114:0/3714986608 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d7e1bb3fe0 0x55d7e1bb8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.669+0000 7fbf08a54640 1 -- 172.21.15.114:0/3714986608 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7e1ab8790 con 0x55d7e1bb76d0 2024-09-13T18:33:37.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.670+0000 7fbf08a54640 1 --2- 172.21.15.114:0/3714986608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7e1bb76d0 0x55d7e1bba570 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fbf04063cf0 tx=0x7fbf040662e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2cba734c570e6da server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:37.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.670+0000 7fbeeeffd640 1 -- 172.21.15.114:0/3714986608 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf040fc050 con 0x55d7e1bb76d0 2024-09-13T18:33:37.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.670+0000 7fbeeeffd640 1 -- 172.21.15.114:0/3714986608 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbf040ffe80 con 0x55d7e1bb76d0 2024-09-13T18:33:37.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.670+0000 7fbeeeffd640 1 -- 172.21.15.114:0/3714986608 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf040fb540 con 0x55d7e1bb76d0 2024-09-13T18:33:37.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.671+0000 7fbf0d1dc740 1 -- 172.21.15.114:0/3714986608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7e1bb76d0 msgr2=0x55d7e1bba570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:37.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.671+0000 7fbf0d1dc740 1 --2- 172.21.15.114:0/3714986608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7e1bb76d0 0x55d7e1bba570 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fbf04063cf0 tx=0x7fbf040662e0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.671+0000 7fbf0d1dc740 1 -- 172.21.15.114:0/3714986608 shutdown_connections 2024-09-13T18:33:37.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.671+0000 7fbf0d1dc740 1 --2- 172.21.15.114:0/3714986608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7e1bb76d0 0x55d7e1bba570 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.671+0000 7fbf0d1dc740 1 --2- 172.21.15.114:0/3714986608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7e1bbf1a0 0x55d7e1bb9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.671+0000 7fbf0d1dc740 1 --2- 172.21.15.114:0/3714986608 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d7e1bb3fe0 0x55d7e1bb8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.671+0000 7fbf0d1dc740 1 -- 172.21.15.114:0/3714986608 >> 172.21.15.114:0/3714986608 conn(0x55d7e1bb0570 msgr2=0x55d7e1bb6fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:37.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.672+0000 7fbf0d1dc740 1 -- 172.21.15.114:0/3714986608 shutdown_connections 2024-09-13T18:33:37.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.672+0000 7fbf0d1dc740 1 -- 172.21.15.114:0/3714986608 wait complete. 2024-09-13T18:33:37.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.673+0000 7fbf0d1dc740 1 -- start start 2024-09-13T18:33:37.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.673+0000 7fbf0d1dc740 1 Processor -- start 2024-09-13T18:33:37.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.674+0000 7fbf0d1dc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d7e1bb3fe0 0x55d7e1bc4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:37.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.674+0000 7fbf0d1dc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7e1bbf1a0 0x55d7e1bc4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:37.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.674+0000 7fbf0d1dc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7e1bc5510 0x55d7e1c252a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:37.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.674+0000 7fbf0d1dc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7e1bc67b0 con 0x55d7e1bb3fe0 2024-09-13T18:33:37.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.674+0000 7fbf0d1dc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7e1bc6920 con 0x55d7e1bbf1a0 2024-09-13T18:33:37.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.674+0000 7fbf0d1dc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7e1c25830 con 0x55d7e1bc5510 2024-09-13T18:33:37.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.674+0000 7fbf0d1dc740 10 client.0 _ll_register_callbacks cb 0x7fbef8004680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:33:37.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.675+0000 7fbf08a54640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7e1bc5510 0x55d7e1c252a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:37.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.675+0000 7fbf03fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d7e1bb3fe0 0x55d7e1bc4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:37.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.675+0000 7fbf03fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d7e1bb3fe0 0x55d7e1bc4a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:52972/0 (socket says 172.21.15.114:52972) 2024-09-13T18:33:37.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.675+0000 7fbf03fff640 1 -- 172.21.15.114:0/4091444546 learned_addr learned my addr 172.21.15.114:0/4091444546 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:37.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.675+0000 7fbf08a54640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7e1bc5510 0x55d7e1c252a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:59848/0 (socket says 172.21.15.114:59848) 2024-09-13T18:33:37.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.675+0000 7fbf03fff640 1 -- 172.21.15.114:0/4091444546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7e1bc5510 msgr2=0x55d7e1c252a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:33:37.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.675+0000 7fbf03fff640 1 --2- 172.21.15.114:0/4091444546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7e1bc5510 0x55d7e1c252a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.675+0000 7fbf03fff640 1 -- 172.21.15.114:0/4091444546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7e1bbf1a0 msgr2=0x55d7e1bc4fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:33:37.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.675+0000 7fbf03fff640 1 --2- 172.21.15.114:0/4091444546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7e1bbf1a0 0x55d7e1bc4fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.675+0000 7fbf03fff640 1 -- 172.21.15.114:0/4091444546 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbefc148820 con 0x55d7e1bb3fe0 2024-09-13T18:33:37.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.679+0000 7fbf03fff640 1 --2- 172.21.15.114:0/4091444546 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d7e1bb3fe0 0x55d7e1bc4a90 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fbef002a980 tx=0x7fbef0005240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=375cf0810ff04460 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:37.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.680+0000 7fbee37fe640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:33:37.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.688+0000 7fbee37fe640 1 -- 172.21.15.114:0/4091444546 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef0042040 con 0x55d7e1bb3fe0 2024-09-13T18:33:37.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.688+0000 7fbf0d1dc740 1 -- 172.21.15.114:0/4091444546 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbef8018040 con 0x55d7e1bb3fe0 2024-09-13T18:33:37.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.688+0000 7fbee37fe640 1 -- 172.21.15.114:0/4091444546 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fbef0044b70 con 0x55d7e1bb3fe0 2024-09-13T18:33:37.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.690+0000 7fbee37fe640 1 -- 172.21.15.114:0/4091444546 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbef0005680 con 0x55d7e1bb3fe0 2024-09-13T18:33:37.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.690+0000 7fbf0d1dc740 10 client.18591 fetch_fsmap learned FSMap version 37 2024-09-13T18:33:37.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.690+0000 7fbf0d1dc740 1 -- 172.21.15.114:0/4091444546 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55d7e1addea0 con 0x55d7e1bb3fe0 2024-09-13T18:33:37.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:37 smithi204 ceph-mon[34777]: scrub summary: active paths [/] 2024-09-13T18:33:37.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:37 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:33:37.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.692+0000 7fbee37fe640 1 -- 172.21.15.114:0/4091444546 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbef8018040 con 0x55d7e1bb3fe0 2024-09-13T18:33:37.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.692+0000 7fbee37fe640 1 -- 172.21.15.114:0/4091444546 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 37) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fbef0038070 con 0x55d7e1bb3fe0 2024-09-13T18:33:37.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.692+0000 7fbf0d1dc740 10 client.18591 fetch_fsmap finished waiting for FSMap version 37 2024-09-13T18:33:37.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.692+0000 7fbf0d1dc740 10 client.18591 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:33:37.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.692+0000 7fbf0d1dc740 20 client.18591 populate_metadata read hostname 'smithi114' 2024-09-13T18:33:37.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.692+0000 7fbf0d1dc740 1 --2- 172.21.15.114:0/4091444546 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55d7e1c27030 0x55d7e1c29450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:37.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.692+0000 7fbf0d1dc740 4 client.18591 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T18:33:37.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.692+0000 7fbf0d1dc740 1 -- 172.21.15.114:0/4091444546 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d7e186fe30 con 0x55d7e1c27030 2024-09-13T18:33:37.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.695+0000 7fbf037fe640 1 --2- 172.21.15.114:0/4091444546 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55d7e1c27030 0x55d7e1c29450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:37.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.697+0000 7fbf037fe640 1 --2- 172.21.15.114:0/4091444546 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55d7e1c27030 0x55d7e1c29450 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=140c17ec7bbd4614 server_cookie=668facd12afae7c in_seq=0 out_seq=0 2024-09-13T18:33:37.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.698+0000 7fbee37fe640 10 client.18591 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:33:37.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.698+0000 7fbee37fe640 1 -- 172.21.15.114:0/4091444546 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55d7e186fe30 con 0x55d7e1c27030 2024-09-13T18:33:37.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.698+0000 7fbee37fe640 10 client.18591 handle_command_reply: tid=0 2024-09-13T18:33:37.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.698+0000 7fbf0d1dc740 1 client.18591 shutdown 2024-09-13T18:33:37.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.699+0000 7fbf0d1dc740 1 -- 172.21.15.114:0/4091444546 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d7e1bb3fe0 msgr2=0x55d7e1bc4a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:37.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.699+0000 7fbf0d1dc740 1 --2- 172.21.15.114:0/4091444546 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d7e1bb3fe0 0x55d7e1bc4a90 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fbef002a980 tx=0x7fbef0005240 comp rx=0 tx=0).stop 2024-09-13T18:33:37.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.702+0000 7fbf0d1dc740 1 -- 172.21.15.114:0/4091444546 shutdown_connections 2024-09-13T18:33:37.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.702+0000 7fbf0d1dc740 1 --2- 172.21.15.114:0/4091444546 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55d7e1c27030 0x55d7e1c29450 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.702+0000 7fbf0d1dc740 1 --2- 172.21.15.114:0/4091444546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7e1bc5510 0x55d7e1c252a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.702+0000 7fbf0d1dc740 1 --2- 172.21.15.114:0/4091444546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7e1bbf1a0 0x55d7e1bc4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.702+0000 7fbf0d1dc740 1 --2- 172.21.15.114:0/4091444546 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d7e1bb3fe0 0x55d7e1bc4a90 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.702+0000 7fbf0d1dc740 1 -- 172.21.15.114:0/4091444546 >> 172.21.15.114:0/4091444546 conn(0x55d7e1bb0570 msgr2=0x55d7e1bba570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:37.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.702+0000 7fbee37fe640 0 client.18591 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:33:37.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.705+0000 7fbf0d1dc740 1 -- 172.21.15.114:0/4091444546 shutdown_connections 2024-09-13T18:33:37.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.706+0000 7fbf0d1dc740 1 -- 172.21.15.114:0/4091444546 wait complete. 2024-09-13T18:33:37.711 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:33:37.711 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-13T18:33:37.711 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": { 2024-09-13T18:33:37.711 INFO:teuthology.orchestra.run.smithi114.stdout: "d4eae42d-d6e7-478f-be69-6c4995a4bb2d": { 2024-09-13T18:33:37.711 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-13T18:33:37.711 INFO:teuthology.orchestra.run.smithi114.stdout: "tag": "d4eae42d-d6e7-478f-be69-6c4995a4bb2d", 2024-09-13T18:33:37.711 INFO:teuthology.orchestra.run.smithi114.stdout: "options": "recursive,force" 2024-09-13T18:33:37.711 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:33:37.711 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:33:37.711 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:33:37.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.707+0000 7fbf0d1dc740 20 client.18591 trim_cache size 0 max 16384 2024-09-13T18:33:37.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.711+0000 7fbf09255640 1 -- 172.21.15.114:0/2041238967 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbef40a1410 msgr2=0x7fbef40a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:37.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.711+0000 7fbf09255640 1 --2- 172.21.15.114:0/2041238967 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbef40a1410 0x7fbef40a38d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbef8005d00 tx=0x7fbef800d660 comp rx=0 tx=0).stop 2024-09-13T18:33:37.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.711+0000 7fbf09255640 1 -- 172.21.15.114:0/2041238967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbefc0b7510 msgr2=0x7fbefc1479c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:37.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.711+0000 7fbf09255640 1 --2- 172.21.15.114:0/2041238967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbefc0b7510 0x7fbefc1479c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fbf0404e950 tx=0x7fbf04071d80 comp rx=0 tx=0).stop 2024-09-13T18:33:37.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.712+0000 7fbf09255640 1 -- 172.21.15.114:0/2041238967 shutdown_connections 2024-09-13T18:33:37.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.712+0000 7fbf09255640 1 --2- 172.21.15.114:0/2041238967 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbef40a1410 0x7fbef40a38d0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.712+0000 7fbf09255640 1 --2- 172.21.15.114:0/2041238967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbefc0b7510 0x7fbefc1479c0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.712+0000 7fbf09255640 1 --2- 172.21.15.114:0/2041238967 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbefc0a5110 0x7fbefc1435d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.712+0000 7fbf09255640 1 --2- 172.21.15.114:0/2041238967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbefc0a47f0 0x7fbefc143090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:37.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.712+0000 7fbf09255640 1 -- 172.21.15.114:0/2041238967 >> 172.21.15.114:0/2041238967 conn(0x7fbefc01a0e0 msgr2=0x7fbefc0b5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:37.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:37 smithi114 ceph-mon[30897]: scrub summary: active paths [/] 2024-09-13T18:33:37.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:37 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:33:37.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.717+0000 7fbf09255640 1 -- 172.21.15.114:0/2041238967 shutdown_connections 2024-09-13T18:33:37.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:37.719+0000 7fbf09255640 1 -- 172.21.15.114:0/2041238967 wait complete. 2024-09-13T18:33:37.753 INFO:tasks.cephfs.filesystem:scrub status for tag:d4eae42d-d6e7-478f-be69-6c4995a4bb2d - {'path': '/', 'tag': 'd4eae42d-d6e7-478f-be69-6c4995a4bb2d', 'options': 'recursive,force'} 2024-09-13T18:33:38.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:38 smithi134 ceph-mon[34283]: pgmap v347: 129 pgs: 129 active+clean; 20 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 35 MiB/s wr, 749 op/s 2024-09-13T18:33:38.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:38 smithi134 ceph-mon[34283]: scrub queued for path: / 2024-09-13T18:33:38.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:38 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:33:38.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:38 smithi134 ceph-mon[34283]: scrub summary: active paths [/] 2024-09-13T18:33:38.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:38 smithi134 ceph-mon[34283]: scrub summary: active paths [/] 2024-09-13T18:33:38.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:38 smithi134 ceph-mon[34283]: scrub summary: active paths [/] 2024-09-13T18:33:38.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:38 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:33:38.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:38 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:33:38.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:38 smithi134 ceph-mon[34283]: scrub summary: idle+waiting paths [/] 2024-09-13T18:33:38.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:38 smithi204 ceph-mon[34777]: pgmap v347: 129 pgs: 129 active+clean; 20 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 35 MiB/s wr, 749 op/s 2024-09-13T18:33:38.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:38 smithi204 ceph-mon[34777]: scrub queued for path: / 2024-09-13T18:33:38.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:38 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:33:38.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:38 smithi204 ceph-mon[34777]: scrub summary: active paths [/] 2024-09-13T18:33:38.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:38 smithi204 ceph-mon[34777]: scrub summary: active paths [/] 2024-09-13T18:33:38.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:38 smithi204 ceph-mon[34777]: scrub summary: active paths [/] 2024-09-13T18:33:38.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:38 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:33:38.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:38 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:33:38.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:38 smithi204 ceph-mon[34777]: scrub summary: idle+waiting paths [/] 2024-09-13T18:33:38.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:38 smithi114 ceph-mon[30897]: pgmap v347: 129 pgs: 129 active+clean; 20 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 35 MiB/s wr, 749 op/s 2024-09-13T18:33:38.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:38 smithi114 ceph-mon[30897]: scrub queued for path: / 2024-09-13T18:33:38.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:38 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:33:38.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:38 smithi114 ceph-mon[30897]: scrub summary: active paths [/] 2024-09-13T18:33:38.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:38 smithi114 ceph-mon[30897]: scrub summary: active paths [/] 2024-09-13T18:33:38.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:38 smithi114 ceph-mon[30897]: scrub summary: active paths [/] 2024-09-13T18:33:38.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:38 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:33:38.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:38 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:33:38.721 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:38 smithi114 ceph-mon[30897]: scrub summary: idle+waiting paths [/] 2024-09-13T18:33:39.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:39 smithi204 ceph-mon[34777]: pgmap v348: 129 pgs: 129 active+clean; 20 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 27 MiB/s wr, 574 op/s 2024-09-13T18:33:39.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:39 smithi114 ceph-mon[30897]: pgmap v348: 129 pgs: 129 active+clean; 20 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 27 MiB/s wr, 574 op/s 2024-09-13T18:33:39.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:39 smithi134 ceph-mon[34283]: pgmap v348: 129 pgs: 129 active+clean; 20 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 27 MiB/s wr, 574 op/s 2024-09-13T18:33:41.213 INFO:tasks.workunit.client.0.smithi114.stdout:stopping iogen 2024-09-13T18:33:41.228 INFO:tasks.workunit.client.0.smithi114.stdout:OK 2024-09-13T18:33:41.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:41 smithi134 ceph-mon[34283]: pgmap v349: 129 pgs: 129 active+clean; 20 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 33 MiB/s wr, 716 op/s 2024-09-13T18:33:41.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:41 smithi204 ceph-mon[34777]: pgmap v349: 129 pgs: 129 active+clean; 20 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 33 MiB/s wr, 716 op/s 2024-09-13T18:33:41.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:41 smithi114 ceph-mon[30897]: pgmap v349: 129 pgs: 129 active+clean; 20 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 33 MiB/s wr, 716 op/s 2024-09-13T18:33:43.595 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:43 smithi114 ceph-mon[30897]: pgmap v350: 129 pgs: 129 active+clean; 20 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 29 MiB/s wr, 630 op/s 2024-09-13T18:33:43.596 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-13T18:33:43.597 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-13T18:33:43.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:43 smithi204 ceph-mon[34777]: pgmap v350: 129 pgs: 129 active+clean; 20 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 29 MiB/s wr, 630 op/s 2024-09-13T18:33:43.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:43 smithi134 ceph-mon[34283]: pgmap v350: 129 pgs: 129 active+clean; 20 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 29 MiB/s wr, 630 op/s 2024-09-13T18:33:44.143 INFO:tasks.workunit:Stopping ['suites/iogen.sh'] on client.0... 2024-09-13T18:33:44.143 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-13T18:33:44.460 DEBUG:teuthology.parallel:result is None 2024-09-13T18:33:44.461 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-13T18:33:44.495 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-13T18:33:44.495 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-09-13T18:33:44.506 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi114.front.sepia.ceph.com 2024-09-13T18:33:44.543 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-09-13T18:33:44.553 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-09-13T18:33:44.554 INFO:teuthology.task.exec:Executing custom commands... 2024-09-13T18:33:44.554 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi114.front.sepia.ceph.com 2024-09-13T18:33:44.554 DEBUG:teuthology.orchestra.run.smithi114:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-09-13T18:33:44.635 DEBUG:teuthology.orchestra.run.smithi114:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-09-13T18:33:44.695 DEBUG:teuthology.orchestra.run.smithi114:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-13T18:33:44.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:44.874+0000 7fc87c75e640 1 -- 172.21.15.114:0/1632928837 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc87410cc80 msgr2=0x7fc874113620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:44.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:44.874+0000 7fc87c75e640 1 --2- 172.21.15.114:0/1632928837 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc87410cc80 0x7fc874113620 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fc86800af00 tx=0x7fc86802e620 comp rx=0 tx=0).stop 2024-09-13T18:33:44.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:44.874+0000 7fc87c75e640 1 -- 172.21.15.114:0/1632928837 shutdown_connections 2024-09-13T18:33:44.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:44.874+0000 7fc87c75e640 1 --2- 172.21.15.114:0/1632928837 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc87410cc80 0x7fc874113620 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:44.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:44.874+0000 7fc87c75e640 1 --2- 172.21.15.114:0/1632928837 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8741080d0 0x7fc87410c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:44.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:44.874+0000 7fc87c75e640 1 --2- 172.21.15.114:0/1632928837 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc874107720 0x7fc874107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:44.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:44.874+0000 7fc87c75e640 1 -- 172.21.15.114:0/1632928837 >> 172.21.15.114:0/1632928837 conn(0x7fc87406b3c0 msgr2=0x7fc87406b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:44.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:44.876+0000 7fc87c75e640 1 -- 172.21.15.114:0/1632928837 shutdown_connections 2024-09-13T18:33:44.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:44.876+0000 7fc87c75e640 1 -- 172.21.15.114:0/1632928837 wait complete. 2024-09-13T18:33:44.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:44.876+0000 7fc87c75e640 1 Processor -- start 2024-09-13T18:33:44.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:44.877+0000 7fc87c75e640 1 -- start start 2024-09-13T18:33:44.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:44.877+0000 7fc87c75e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc874107720 0x7fc8741a4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:44.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:44.877+0000 7fc87c75e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc87410cc80 0x7fc8741a5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:44.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:44.877+0000 7fc87c75e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8741a9810 0x7fc8741a9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:44.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:44.877+0000 7fc87c75e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8741aa200 con 0x7fc874107720 2024-09-13T18:33:44.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:44.877+0000 7fc87c75e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8741aa370 con 0x7fc8741a9810 2024-09-13T18:33:44.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:44.877+0000 7fc87c75e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8741aa4e0 con 0x7fc87410cc80 2024-09-13T18:33:44.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:44.877+0000 7fc87a4fa640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc874107720 0x7fc8741a4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:44.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:44.877+0000 7fc87a4fa640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc874107720 0x7fc8741a4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37462/0 (socket says 172.21.15.114:37462) 2024-09-13T18:33:44.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:44.877+0000 7fc87a4fa640 1 -- 172.21.15.114:0/959694721 learned_addr learned my addr 172.21.15.114:0/959694721 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:44.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:44.878+0000 7fc87a4fa640 1 -- 172.21.15.114:0/959694721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc87410cc80 msgr2=0x7fc8741a5420 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:33:44.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:44.878+0000 7fc87a4fa640 1 --2- 172.21.15.114:0/959694721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc87410cc80 0x7fc8741a5420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:44.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:44.878+0000 7fc87a4fa640 1 -- 172.21.15.114:0/959694721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8741a9810 msgr2=0x7fc8741a9cc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:33:44.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:44.878+0000 7fc87a4fa640 1 --2- 172.21.15.114:0/959694721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8741a9810 0x7fc8741a9cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:44.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:44.878+0000 7fc87a4fa640 1 -- 172.21.15.114:0/959694721 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8680081d0 con 0x7fc874107720 2024-09-13T18:33:44.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:44.878+0000 7fc87a4fa640 1 --2- 172.21.15.114:0/959694721 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc874107720 0x7fc8741a4ee0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fc86400ba50 tx=0x7fc86400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:44.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:44.879+0000 7fc8737fe640 1 -- 172.21.15.114:0/959694721 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86400cbe0 con 0x7fc874107720 2024-09-13T18:33:44.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:44.879+0000 7fc8737fe640 1 -- 172.21.15.114:0/959694721 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc86400cd40 con 0x7fc874107720 2024-09-13T18:33:44.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:44.879+0000 7fc8737fe640 1 -- 172.21.15.114:0/959694721 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8640076a0 con 0x7fc874107720 2024-09-13T18:33:44.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:44.879+0000 7fc87c75e640 1 -- 172.21.15.114:0/959694721 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc874115680 con 0x7fc874107720 2024-09-13T18:33:44.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:44.879+0000 7fc87c75e640 1 -- 172.21.15.114:0/959694721 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc874115a80 con 0x7fc874107720 2024-09-13T18:33:44.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:44.894+0000 7fc8737fe640 1 -- 172.21.15.114:0/959694721 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fc864007800 con 0x7fc874107720 2024-09-13T18:33:44.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:44.894+0000 7fc87c75e640 1 -- 172.21.15.114:0/959694721 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc860005250 con 0x7fc874107720 2024-09-13T18:33:44.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:44.895+0000 7fc8737fe640 1 --2- 172.21.15.114:0/959694721 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc85c0a1410 0x7fc85c0a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:44.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:44.895+0000 7fc8737fe640 1 -- 172.21.15.114:0/959694721 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fc8640cc660 con 0x7fc874107720 2024-09-13T18:33:44.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:44.899+0000 7fc879cf9640 1 --2- 172.21.15.114:0/959694721 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc85c0a1410 0x7fc85c0a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:44.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:44.903+0000 7fc879cf9640 1 --2- 172.21.15.114:0/959694721 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc85c0a1410 0x7fc85c0a38d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc8741a6480 tx=0x7fc86c005b80 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:44.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:44.904+0000 7fc8737fe640 1 -- 172.21.15.114:0/959694721 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fc8640957c0 con 0x7fc874107720 2024-09-13T18:33:45.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:45.283+0000 7fc87c75e640 1 -- 172.21.15.114:0/959694721 --> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7fc860000bc0 con 0x7fc85c0a1410 2024-09-13T18:33:45.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:45.383+0000 7fc8737fe640 1 -- 172.21.15.114:0/959694721 <== mgr.15555 v2:172.21.15.114:6800/2440721366 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+350 (secure 0 0 0) 0x7fc860000bc0 con 0x7fc85c0a1410 2024-09-13T18:33:45.386 INFO:teuthology.orchestra.run.smithi114.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-13T00:00:00", "created": "2024-09-13T18:22:36", "first": "2024-09-13T18:23:00", "last": "2024-09-13T18:33:00", "last_pruned": "2024-09-13T18:33:00", "created_count": 11, "pruned_count": 5, "active": true} 2024-09-13T18:33:45.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:45.386+0000 7fc87c75e640 1 -- 172.21.15.114:0/959694721 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc85c0a1410 msgr2=0x7fc85c0a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:45.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:45.386+0000 7fc87c75e640 1 --2- 172.21.15.114:0/959694721 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc85c0a1410 0x7fc85c0a38d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc8741a6480 tx=0x7fc86c005b80 comp rx=0 tx=0).stop 2024-09-13T18:33:45.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:45.386+0000 7fc87c75e640 1 -- 172.21.15.114:0/959694721 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc874107720 msgr2=0x7fc8741a4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:45.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:45.387+0000 7fc87c75e640 1 --2- 172.21.15.114:0/959694721 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc874107720 0x7fc8741a4ee0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fc86400ba50 tx=0x7fc86400bf10 comp rx=0 tx=0).stop 2024-09-13T18:33:45.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:45.388+0000 7fc87c75e640 1 -- 172.21.15.114:0/959694721 shutdown_connections 2024-09-13T18:33:45.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:45.388+0000 7fc87c75e640 1 --2- 172.21.15.114:0/959694721 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc85c0a1410 0x7fc85c0a38d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:45.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:45.388+0000 7fc87c75e640 1 --2- 172.21.15.114:0/959694721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8741a9810 0x7fc8741a9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:45.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:45.388+0000 7fc87c75e640 1 --2- 172.21.15.114:0/959694721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc87410cc80 0x7fc8741a5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:45.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:45.388+0000 7fc87c75e640 1 --2- 172.21.15.114:0/959694721 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc874107720 0x7fc8741a4ee0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:45.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:45.388+0000 7fc87c75e640 1 -- 172.21.15.114:0/959694721 >> 172.21.15.114:0/959694721 conn(0x7fc87406b3c0 msgr2=0x7fc87410a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:45.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:45.391+0000 7fc87c75e640 1 -- 172.21.15.114:0/959694721 shutdown_connections 2024-09-13T18:33:45.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:45.391+0000 7fc87c75e640 1 -- 172.21.15.114:0/959694721 wait complete. 2024-09-13T18:33:45.418 DEBUG:teuthology.orchestra.run.smithi114:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-13T18:33:45.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:45.602+0000 7f4d86ec9640 1 -- 172.21.15.114:0/2983067756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d801080d0 msgr2=0x7f4d800722d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:45.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:45.602+0000 7f4d86ec9640 1 --2- 172.21.15.114:0/2983067756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d801080d0 0x7f4d800722d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c009900 tx=0x7f4d7c02a440 comp rx=0 tx=0).stop 2024-09-13T18:33:45.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:45.602+0000 7f4d86ec9640 1 -- 172.21.15.114:0/2983067756 shutdown_connections 2024-09-13T18:33:45.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:45.602+0000 7f4d86ec9640 1 --2- 172.21.15.114:0/2983067756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d800729e0 0x7f4d800793c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:45.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:45.602+0000 7f4d86ec9640 1 --2- 172.21.15.114:0/2983067756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d801080d0 0x7f4d800722d0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:45.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:45.602+0000 7f4d86ec9640 1 --2- 172.21.15.114:0/2983067756 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d80107720 0x7f4d80107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:45.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:45.602+0000 7f4d86ec9640 1 -- 172.21.15.114:0/2983067756 >> 172.21.15.114:0/2983067756 conn(0x7f4d8006af50 msgr2=0x7f4d8006b360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:45.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:45.603+0000 7f4d86ec9640 1 -- 172.21.15.114:0/2983067756 shutdown_connections 2024-09-13T18:33:45.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:45.604+0000 7f4d86ec9640 1 -- 172.21.15.114:0/2983067756 wait complete. 2024-09-13T18:33:45.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:45.604+0000 7f4d86ec9640 1 Processor -- start 2024-09-13T18:33:45.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:45.606+0000 7f4d86ec9640 1 -- start start 2024-09-13T18:33:45.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:45.606+0000 7f4d86ec9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d800729e0 0x7f4d80078f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:45.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:45.606+0000 7f4d86ec9640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d80107720 0x7f4d80074020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:45.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:45.606+0000 7f4d86ec9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d80074560 0x7f4d80074a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:45.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:45.606+0000 7f4d86ec9640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d80079790 con 0x7f4d80107720 2024-09-13T18:33:45.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:45.606+0000 7f4d86ec9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d80135550 con 0x7f4d80074560 2024-09-13T18:33:45.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:45.606+0000 7f4d86ec9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d80135710 con 0x7f4d800729e0 2024-09-13T18:33:45.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:45.607+0000 7f4d85466640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d80074560 0x7f4d80074a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:45.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:45.607+0000 7f4d85466640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d80074560 0x7f4d80074a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:46848/0 (socket says 172.21.15.114:46848) 2024-09-13T18:33:45.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:45.607+0000 7f4d85466640 1 -- 172.21.15.114:0/839202510 learned_addr learned my addr 172.21.15.114:0/839202510 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:33:45.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:45.607+0000 7f4d85466640 1 -- 172.21.15.114:0/839202510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d800729e0 msgr2=0x7f4d80078f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:45.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:45.607+0000 7f4d85466640 1 --2- 172.21.15.114:0/839202510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d800729e0 0x7f4d80078f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:45.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:45.607+0000 7f4d85466640 1 -- 172.21.15.114:0/839202510 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d80107720 msgr2=0x7f4d80074020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:45.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:45.607+0000 7f4d85466640 1 --2- 172.21.15.114:0/839202510 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d80107720 0x7f4d80074020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:45.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:45.607+0000 7f4d85466640 1 -- 172.21.15.114:0/839202510 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d7c009560 con 0x7f4d80074560 2024-09-13T18:33:45.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:45.608+0000 7f4d85466640 1 --2- 172.21.15.114:0/839202510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d80074560 0x7f4d80074a10 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4d7800b970 tx=0x7f4d7800be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:45.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:45.610+0000 7f4d75ffb640 1 -- 172.21.15.114:0/839202510 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7800cb40 con 0x7f4d80074560 2024-09-13T18:33:45.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:45.610+0000 7f4d75ffb640 1 -- 172.21.15.114:0/839202510 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4d7800cca0 con 0x7f4d80074560 2024-09-13T18:33:45.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:45.610+0000 7f4d75ffb640 1 -- 172.21.15.114:0/839202510 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d780076a0 con 0x7f4d80074560 2024-09-13T18:33:45.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:45.610+0000 7f4d86ec9640 1 -- 172.21.15.114:0/839202510 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d801359f0 con 0x7f4d80074560 2024-09-13T18:33:45.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:45.611+0000 7f4d86ec9640 1 -- 172.21.15.114:0/839202510 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d80135eb0 con 0x7f4d80074560 2024-09-13T18:33:45.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:45.619+0000 7f4d75ffb640 1 -- 172.21.15.114:0/839202510 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f4d78014440 con 0x7f4d80074560 2024-09-13T18:33:45.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:45.619+0000 7f4d75ffb640 1 --2- 172.21.15.114:0/839202510 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f4d680a1400 0x7f4d680a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:33:45.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:45.620+0000 7f4d75ffb640 1 -- 172.21.15.114:0/839202510 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f4d780cc810 con 0x7f4d80074560 2024-09-13T18:33:45.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:45.620+0000 7f4d86ec9640 1 -- 172.21.15.114:0/839202510 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d80071bf0 con 0x7f4d80074560 2024-09-13T18:33:45.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:45.621+0000 7f4d84c65640 1 --2- 172.21.15.114:0/839202510 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f4d680a1400 0x7f4d680a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:33:45.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:45.628+0000 7f4d84c65640 1 --2- 172.21.15.114:0/839202510 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f4d680a1400 0x7f4d680a38c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4d700096f0 tx=0x7f4d70006c40 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:33:45.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:45.632+0000 7f4d75ffb640 1 -- 172.21.15.114:0/839202510 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f4d78095970 con 0x7f4d80074560 2024-09-13T18:33:45.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:45 smithi204 ceph-mon[34777]: pgmap v351: 129 pgs: 129 active+clean; 20 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 26 MiB/s wr, 558 op/s 2024-09-13T18:33:45.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:45 smithi204 ceph-mon[34777]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-13T18:33:45.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:45 smithi204 ceph-mon[34777]: from='client.18594 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:33:45.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:45 smithi114 ceph-mon[30897]: pgmap v351: 129 pgs: 129 active+clean; 20 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 26 MiB/s wr, 558 op/s 2024-09-13T18:33:45.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:45 smithi114 ceph-mon[30897]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-13T18:33:45.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:45 smithi114 ceph-mon[30897]: from='client.18594 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:33:45.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:45 smithi134 ceph-mon[34283]: pgmap v351: 129 pgs: 129 active+clean; 20 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 26 MiB/s wr, 558 op/s 2024-09-13T18:33:45.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:45 smithi134 ceph-mon[34283]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-13T18:33:45.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:45 smithi134 ceph-mon[34283]: from='client.18594 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:33:45.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:45.988+0000 7f4d86ec9640 1 -- 172.21.15.114:0/839202510 --> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f4d80075570 con 0x7f4d680a1400 2024-09-13T18:33:46.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:46.111+0000 7f4d75ffb640 1 -- 172.21.15.114:0/839202510 <== mgr.15555 v2:172.21.15.114:6800/2440721366 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f4d80075570 con 0x7f4d680a1400 2024-09-13T18:33:46.115 INFO:teuthology.orchestra.run.smithi114.stdout:/ 1m 6m3h 2024-09-13T18:33:46.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:46.114+0000 7f4d86ec9640 1 -- 172.21.15.114:0/839202510 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f4d680a1400 msgr2=0x7f4d680a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:46.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:46.114+0000 7f4d86ec9640 1 --2- 172.21.15.114:0/839202510 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f4d680a1400 0x7f4d680a38c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4d700096f0 tx=0x7f4d70006c40 comp rx=0 tx=0).stop 2024-09-13T18:33:46.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:46.114+0000 7f4d86ec9640 1 -- 172.21.15.114:0/839202510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d80074560 msgr2=0x7f4d80074a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:33:46.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:46.114+0000 7f4d86ec9640 1 --2- 172.21.15.114:0/839202510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d80074560 0x7f4d80074a10 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4d7800b970 tx=0x7f4d7800be30 comp rx=0 tx=0).stop 2024-09-13T18:33:46.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:46.117+0000 7f4d86ec9640 1 -- 172.21.15.114:0/839202510 shutdown_connections 2024-09-13T18:33:46.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:46.117+0000 7f4d86ec9640 1 --2- 172.21.15.114:0/839202510 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f4d680a1400 0x7f4d680a38c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:46.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:46.117+0000 7f4d86ec9640 1 --2- 172.21.15.114:0/839202510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d80074560 0x7f4d80074a10 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:46.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:46.117+0000 7f4d86ec9640 1 --2- 172.21.15.114:0/839202510 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d80107720 0x7f4d80074020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:46.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:46.117+0000 7f4d86ec9640 1 --2- 172.21.15.114:0/839202510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d800729e0 0x7f4d80078f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:33:46.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:46.117+0000 7f4d86ec9640 1 -- 172.21.15.114:0/839202510 >> 172.21.15.114:0/839202510 conn(0x7f4d8006af50 msgr2=0x7f4d8006ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:33:46.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:46.117+0000 7f4d86ec9640 1 -- 172.21.15.114:0/839202510 shutdown_connections 2024-09-13T18:33:46.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:33:46.117+0000 7f4d86ec9640 1 -- 172.21.15.114:0/839202510 wait complete. 2024-09-13T18:33:46.137 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-13T18:33:46.148 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-13T18:33:46.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:46 smithi204 ceph-mon[34777]: from='client.28079 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:33:46.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:46 smithi114 ceph-mon[30897]: from='client.28079 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:33:46.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:46 smithi134 ceph-mon[34283]: from='client.28079 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-13T18:33:47.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:47 smithi204 ceph-mon[34777]: pgmap v352: 129 pgs: 129 active+clean; 20 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 32 MiB/s wr, 714 op/s 2024-09-13T18:33:47.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:47 smithi114 ceph-mon[30897]: pgmap v352: 129 pgs: 129 active+clean; 20 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 32 MiB/s wr, 714 op/s 2024-09-13T18:33:47.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:47 smithi134 ceph-mon[34283]: pgmap v352: 129 pgs: 129 active+clean; 20 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 32 MiB/s wr, 714 op/s 2024-09-13T18:33:49.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:49 smithi204 ceph-mon[34777]: pgmap v353: 129 pgs: 129 active+clean; 20 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 23 MiB/s wr, 521 op/s 2024-09-13T18:33:49.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:49 smithi204 ceph-mon[34777]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-13T18:33:49.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:49 smithi114 ceph-mon[30897]: pgmap v353: 129 pgs: 129 active+clean; 20 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 23 MiB/s wr, 521 op/s 2024-09-13T18:33:49.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:49 smithi114 ceph-mon[30897]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-13T18:33:49.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:49 smithi134 ceph-mon[34283]: pgmap v353: 129 pgs: 129 active+clean; 20 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 23 MiB/s wr, 521 op/s 2024-09-13T18:33:49.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:49 smithi134 ceph-mon[34283]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-13T18:33:50.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:50 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:33:50.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:50 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:33:50.770 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:50 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:33:51.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:51 smithi134 ceph-mon[34283]: pgmap v354: 129 pgs: 129 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 21 MiB/s wr, 626 op/s 2024-09-13T18:33:51.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:51 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:33:51.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:51 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:33:51.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:51 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:33:51.870 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:51 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:33:51.870 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:51 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:33:51.870 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:51 smithi134 ceph-mon[34283]: scrub summary: idle 2024-09-13T18:33:51.870 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:51 smithi134 ceph-mon[34283]: scrub summary: idle 2024-09-13T18:33:51.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:51 smithi204 ceph-mon[34777]: pgmap v354: 129 pgs: 129 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 21 MiB/s wr, 626 op/s 2024-09-13T18:33:51.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:51 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:33:51.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:51 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:33:51.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:51 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:33:51.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:51 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:33:51.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:51 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:33:51.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:51 smithi204 ceph-mon[34777]: scrub summary: idle 2024-09-13T18:33:51.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:51 smithi204 ceph-mon[34777]: scrub summary: idle 2024-09-13T18:33:51.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:51 smithi114 ceph-mon[30897]: pgmap v354: 129 pgs: 129 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 21 MiB/s wr, 626 op/s 2024-09-13T18:33:51.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:51 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:33:51.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:51 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:33:51.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:51 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:33:51.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:51 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:33:51.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:51 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:33:51.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:51 smithi114 ceph-mon[30897]: scrub summary: idle 2024-09-13T18:33:51.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:51 smithi114 ceph-mon[30897]: scrub summary: idle 2024-09-13T18:33:52.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:52 smithi134 ceph-mon[34283]: scrub summary: idle 2024-09-13T18:33:52.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:52 smithi134 ceph-mon[34283]: scrub summary: idle 2024-09-13T18:33:52.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:52 smithi204 ceph-mon[34777]: scrub summary: idle 2024-09-13T18:33:52.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:52 smithi204 ceph-mon[34777]: scrub summary: idle 2024-09-13T18:33:52.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:52 smithi114 ceph-mon[30897]: scrub summary: idle 2024-09-13T18:33:52.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:52 smithi114 ceph-mon[30897]: scrub summary: idle 2024-09-13T18:33:53.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:53 smithi134 ceph-mon[34283]: pgmap v355: 129 pgs: 129 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 13 MiB/s wr, 501 op/s 2024-09-13T18:33:53.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:53 smithi204 ceph-mon[34777]: pgmap v355: 129 pgs: 129 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 13 MiB/s wr, 501 op/s 2024-09-13T18:33:53.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:53 smithi114 ceph-mon[30897]: pgmap v355: 129 pgs: 129 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 13 MiB/s wr, 501 op/s 2024-09-13T18:33:55.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:55 smithi134 ceph-mon[34283]: pgmap v356: 129 pgs: 129 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.6 MiB/s wr, 413 op/s 2024-09-13T18:33:55.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:55 smithi204 ceph-mon[34777]: pgmap v356: 129 pgs: 129 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.6 MiB/s wr, 413 op/s 2024-09-13T18:33:55.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:55 smithi114 ceph-mon[30897]: pgmap v356: 129 pgs: 129 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.6 MiB/s wr, 413 op/s 2024-09-13T18:33:57.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:57 smithi134 ceph-mon[34283]: pgmap v357: 129 pgs: 129 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.6 MiB/s wr, 398 op/s 2024-09-13T18:33:57.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:57 smithi204 ceph-mon[34777]: pgmap v357: 129 pgs: 129 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.6 MiB/s wr, 398 op/s 2024-09-13T18:33:57.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:57 smithi114 ceph-mon[30897]: pgmap v357: 129 pgs: 129 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.6 MiB/s wr, 398 op/s 2024-09-13T18:33:59.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:33:59 smithi134 ceph-mon[34283]: pgmap v358: 129 pgs: 129 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 422 KiB/s wr, 223 op/s 2024-09-13T18:33:59.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:33:59 smithi204 ceph-mon[34777]: pgmap v358: 129 pgs: 129 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 422 KiB/s wr, 223 op/s 2024-09-13T18:33:59.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:33:59 smithi114 ceph-mon[30897]: pgmap v358: 129 pgs: 129 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 422 KiB/s wr, 223 op/s 2024-09-13T18:34:01.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:01 smithi134 ceph-mon[34283]: pgmap v359: 129 pgs: 129 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.7 KiB/s wr, 202 op/s 2024-09-13T18:34:01.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:01 smithi204 ceph-mon[34777]: pgmap v359: 129 pgs: 129 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.7 KiB/s wr, 202 op/s 2024-09-13T18:34:01.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:01 smithi114 ceph-mon[30897]: pgmap v359: 129 pgs: 129 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.7 KiB/s wr, 202 op/s 2024-09-13T18:34:03.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:03 smithi134 ceph-mon[34283]: pgmap v360: 129 pgs: 129 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 26 KiB/s wr, 81 op/s 2024-09-13T18:34:03.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:03 smithi204 ceph-mon[34777]: pgmap v360: 129 pgs: 129 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 26 KiB/s wr, 81 op/s 2024-09-13T18:34:03.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:03 smithi114 ceph-mon[30897]: pgmap v360: 129 pgs: 129 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 26 KiB/s wr, 81 op/s 2024-09-13T18:34:05.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:05 smithi134 ceph-mon[34283]: pgmap v361: 129 pgs: 129 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 27 KiB/s wr, 31 op/s 2024-09-13T18:34:05.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:05 smithi204 ceph-mon[34777]: pgmap v361: 129 pgs: 129 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 27 KiB/s wr, 31 op/s 2024-09-13T18:34:05.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:05 smithi114 ceph-mon[30897]: pgmap v361: 129 pgs: 129 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 27 KiB/s wr, 31 op/s 2024-09-13T18:34:07.754 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-09-13T18:34:07.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:07 smithi204 ceph-mon[34777]: pgmap v362: 129 pgs: 129 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 50 KiB/s wr, 32 op/s 2024-09-13T18:34:07.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:07 smithi204 ceph-mon[34777]: osdmap e105: 12 total, 12 up, 12 in 2024-09-13T18:34:07.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.963+0000 7fbeec945640 1 -- 172.21.15.114:0/4149512735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbee8119400 msgr2=0x7fbee811b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:07.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.963+0000 7fbeec945640 1 --2- 172.21.15.114:0/4149512735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbee8119400 0x7fbee811b7f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fbee0009ca0 tx=0x7fbee002a8f0 comp rx=0 tx=0).stop 2024-09-13T18:34:07.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.967+0000 7fbeec945640 1 -- 172.21.15.114:0/4149512735 shutdown_connections 2024-09-13T18:34:07.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:07 smithi114 ceph-mon[30897]: pgmap v362: 129 pgs: 129 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 50 KiB/s wr, 32 op/s 2024-09-13T18:34:07.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:07 smithi114 ceph-mon[30897]: osdmap e105: 12 total, 12 up, 12 in 2024-09-13T18:34:07.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.967+0000 7fbeec945640 1 --2- 172.21.15.114:0/4149512735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbee8119400 0x7fbee811b7f0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:07.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.967+0000 7fbeec945640 1 --2- 172.21.15.114:0/4149512735 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbee8107920 0x7fbee8107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:07.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.967+0000 7fbeec945640 1 --2- 172.21.15.114:0/4149512735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbee8107540 0x7fbee810e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:07.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.967+0000 7fbeec945640 1 -- 172.21.15.114:0/4149512735 >> 172.21.15.114:0/4149512735 conn(0x7fbee806afe0 msgr2=0x7fbee806b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:07.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.973+0000 7fbeec945640 1 -- 172.21.15.114:0/4149512735 shutdown_connections 2024-09-13T18:34:07.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.975+0000 7fbeec945640 1 -- 172.21.15.114:0/4149512735 wait complete. 2024-09-13T18:34:07.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.977+0000 7fbeec945640 1 Processor -- start 2024-09-13T18:34:07.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.978+0000 7fbeec945640 1 -- start start 2024-09-13T18:34:07.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.978+0000 7fbeec945640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbee8107540 0x7fbee81169d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:07.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.978+0000 7fbeec945640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbee8107920 0x7fbee810faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:07.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.978+0000 7fbeec945640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbee8119400 0x7fbee810ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:07.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.978+0000 7fbeec945640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee8110520 con 0x7fbee8107920 2024-09-13T18:34:07.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.978+0000 7fbeec945640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee8110690 con 0x7fbee8119400 2024-09-13T18:34:07.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.978+0000 7fbeec945640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee8110850 con 0x7fbee8107540 2024-09-13T18:34:07.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.978+0000 7fbee6d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbee8119400 0x7fbee810ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:07.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.978+0000 7fbee6d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbee8119400 0x7fbee810ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:59106/0 (socket says 172.21.15.114:59106) 2024-09-13T18:34:07.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.978+0000 7fbee6d9d640 1 -- 172.21.15.114:0/2275575128 learned_addr learned my addr 172.21.15.114:0/2275575128 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:34:07.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.979+0000 7fbee6d9d640 1 -- 172.21.15.114:0/2275575128 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbee8107540 msgr2=0x7fbee81169d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:34:07.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.979+0000 7fbee6d9d640 1 --2- 172.21.15.114:0/2275575128 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbee8107540 0x7fbee81169d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:07.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.979+0000 7fbee6d9d640 1 -- 172.21.15.114:0/2275575128 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbee8107920 msgr2=0x7fbee810faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:07.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.979+0000 7fbee6d9d640 1 --2- 172.21.15.114:0/2275575128 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbee8107920 0x7fbee810faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:07.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.979+0000 7fbee6d9d640 1 -- 172.21.15.114:0/2275575128 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee0009900 con 0x7fbee8119400 2024-09-13T18:34:07.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.979+0000 7fbee6d9d640 1 --2- 172.21.15.114:0/2275575128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbee8119400 0x7fbee810ffe0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fbee0030820 tx=0x7fbee0030850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:07.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.983+0000 7fbedffff640 1 -- 172.21.15.114:0/2275575128 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee0030d80 con 0x7fbee8119400 2024-09-13T18:34:07.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.983+0000 7fbedffff640 1 -- 172.21.15.114:0/2275575128 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbee003f070 con 0x7fbee8119400 2024-09-13T18:34:07.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.983+0000 7fbeec945640 1 -- 172.21.15.114:0/2275575128 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbee81145c0 con 0x7fbee8119400 2024-09-13T18:34:07.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.983+0000 7fbeec945640 1 -- 172.21.15.114:0/2275575128 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbee8114ad0 con 0x7fbee8119400 2024-09-13T18:34:07.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.986+0000 7fbedffff640 1 -- 172.21.15.114:0/2275575128 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee002f6c0 con 0x7fbee8119400 2024-09-13T18:34:07.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.989+0000 7fbeef667740 1 Processor -- start 2024-09-13T18:34:07.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.989+0000 7fbeef667740 1 -- start start 2024-09-13T18:34:07.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.989+0000 7fbedffff640 1 -- 172.21.15.114:0/2275575128 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fbee000a040 con 0x7fbee8119400 2024-09-13T18:34:07.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.989+0000 7fbeef667740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb74de2e10 0x55fb74de31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:07.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.989+0000 7fbeef667740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb74de3730 0x55fb74deda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:07.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.989+0000 7fbeef667740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb74dedfd0 0x55fb74df03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:07.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.989+0000 7fbeef667740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb74d4df20 con 0x55fb74de3730 2024-09-13T18:34:07.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.989+0000 7fbeef667740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb74d4b6b0 con 0x55fb74de2e10 2024-09-13T18:34:07.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.989+0000 7fbeef667740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb74d46fa0 con 0x55fb74dedfd0 2024-09-13T18:34:07.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.990+0000 7fbeddd9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb74de2e10 0x55fb74de31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:07.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.990+0000 7fbeddd9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb74de2e10 0x55fb74de31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:59114/0 (socket says 172.21.15.114:59114) 2024-09-13T18:34:07.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.990+0000 7fbeddd9b640 1 -- 172.21.15.114:0/2576738384 learned_addr learned my addr 172.21.15.114:0/2576738384 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:34:07.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.990+0000 7fbeddd9b640 1 --2- 172.21.15.114:0/2576738384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb74dedfd0 0x55fb74df03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:07.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.990+0000 7fbee659c640 1 --2- 172.21.15.114:0/2576738384 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb74de3730 0x55fb74deda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:07.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.990+0000 7fbeddd9b640 1 -- 172.21.15.114:0/2576738384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb74dedfd0 msgr2=0x55fb74df03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:07.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.990+0000 7fbeddd9b640 1 --2- 172.21.15.114:0/2576738384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb74dedfd0 0x55fb74df03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:07.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.990+0000 7fbeddd9b640 1 -- 172.21.15.114:0/2576738384 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb74de3730 msgr2=0x55fb74deda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:07.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.990+0000 7fbeddd9b640 1 --2- 172.21.15.114:0/2576738384 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb74de3730 0x55fb74deda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:07.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.990+0000 7fbeddd9b640 1 -- 172.21.15.114:0/2576738384 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb74d01840 con 0x55fb74de2e10 2024-09-13T18:34:07.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.991+0000 7fbeddd9b640 1 --2- 172.21.15.114:0/2576738384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb74de2e10 0x55fb74de31f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fbed8009460 tx=0x7fbed80095f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdd1b524ae542230 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:07.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.992+0000 7fbedffff640 1 --2- 172.21.15.114:0/2275575128 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbec80a1410 0x7fbec80a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:08.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.993+0000 7fbedffff640 1 -- 172.21.15.114:0/2275575128 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fbee0041070 con 0x7fbee8119400 2024-09-13T18:34:08.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.993+0000 7fbedcd99640 1 -- 172.21.15.114:0/2576738384 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed8013070 con 0x55fb74de2e10 2024-09-13T18:34:08.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.993+0000 7fbedcd99640 1 -- 172.21.15.114:0/2576738384 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbed8019030 con 0x55fb74de2e10 2024-09-13T18:34:08.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.993+0000 7fbedcd99640 1 -- 172.21.15.114:0/2576738384 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed80055b0 con 0x55fb74de2e10 2024-09-13T18:34:08.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.997+0000 7fbeef667740 1 -- 172.21.15.114:0/2576738384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb74de2e10 msgr2=0x55fb74de31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:08.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.997+0000 7fbeef667740 1 --2- 172.21.15.114:0/2576738384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb74de2e10 0x55fb74de31f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fbed8009460 tx=0x7fbed80095f0 comp rx=0 tx=0).stop 2024-09-13T18:34:08.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.997+0000 7fbeef667740 1 -- 172.21.15.114:0/2576738384 shutdown_connections 2024-09-13T18:34:08.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.997+0000 7fbeef667740 1 --2- 172.21.15.114:0/2576738384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb74dedfd0 0x55fb74df03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:08.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.997+0000 7fbeef667740 1 --2- 172.21.15.114:0/2576738384 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb74de3730 0x55fb74deda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:08.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.997+0000 7fbeef667740 1 --2- 172.21.15.114:0/2576738384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb74de2e10 0x55fb74de31f0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:08.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.998+0000 7fbeef667740 1 -- 172.21.15.114:0/2576738384 >> 172.21.15.114:0/2576738384 conn(0x55fb74ddf380 msgr2=0x55fb74ddf760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:08.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.998+0000 7fbeef667740 1 -- 172.21.15.114:0/2576738384 shutdown_connections 2024-09-13T18:34:08.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:07.998+0000 7fbee659c640 1 --2- 172.21.15.114:0/2275575128 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbec80a1410 0x7fbec80a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:08.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.001+0000 7fbeef667740 1 -- 172.21.15.114:0/2576738384 wait complete. 2024-09-13T18:34:08.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.001+0000 7fbee659c640 1 --2- 172.21.15.114:0/2275575128 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbec80a1410 0x7fbec80a38d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fbed00058c0 tx=0x7fbed0005d00 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:08.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.002+0000 7fbeef667740 1 -- start start 2024-09-13T18:34:08.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.002+0000 7fbeef667740 1 Processor -- start 2024-09-13T18:34:08.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.006+0000 7fbeef667740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb74de2e10 0x55fb74ded930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:08.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.006+0000 7fbeef667740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb74de3730 0x55fb74deac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:08.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.006+0000 7fbeef667740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb74dedfd0 0x55fb74deb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:08.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.006+0000 7fbeef667740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb74ce82a0 con 0x55fb74de2e10 2024-09-13T18:34:08.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.006+0000 7fbeef667740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb74a1f430 con 0x55fb74dedfd0 2024-09-13T18:34:08.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.006+0000 7fbeef667740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb74cca680 con 0x55fb74de3730 2024-09-13T18:34:08.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.006+0000 7fbeef667740 10 client.0 _ll_register_callbacks cb 0x55fb74d40610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:34:08.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.006+0000 7fbee659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb74de2e10 0x55fb74ded930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:08.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.006+0000 7fbee659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb74de2e10 0x55fb74ded930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:54894/0 (socket says 172.21.15.114:54894) 2024-09-13T18:34:08.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.006+0000 7fbee659c640 1 -- 172.21.15.114:0/2788670797 learned_addr learned my addr 172.21.15.114:0/2788670797 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:34:08.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.006+0000 7fbeddd9b640 1 --2- 172.21.15.114:0/2788670797 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb74de3730 0x55fb74deac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:08.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.006+0000 7fbee6d9d640 1 --2- 172.21.15.114:0/2788670797 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb74dedfd0 0x55fb74deb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:08.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.007+0000 7fbee6d9d640 1 -- 172.21.15.114:0/2788670797 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb74de3730 msgr2=0x55fb74deac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:08.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.007+0000 7fbee6d9d640 1 --2- 172.21.15.114:0/2788670797 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb74de3730 0x55fb74deac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:08.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.007+0000 7fbee6d9d640 1 -- 172.21.15.114:0/2788670797 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb74de2e10 msgr2=0x55fb74ded930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:08.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.007+0000 7fbee6d9d640 1 --2- 172.21.15.114:0/2788670797 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb74de2e10 0x55fb74ded930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:08.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.007+0000 7fbee6d9d640 1 -- 172.21.15.114:0/2788670797 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbed80134a0 con 0x55fb74dedfd0 2024-09-13T18:34:08.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.007+0000 7fbee6d9d640 1 --2- 172.21.15.114:0/2788670797 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb74dedfd0 0x55fb74deb1a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fbee00ec750 tx=0x7fbee00bc810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77ff29a942158ad4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:08.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.010+0000 7fbed5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:34:08.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.010+0000 7fbed5ffb640 1 -- 172.21.15.114:0/2788670797 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee00ecac0 con 0x55fb74dedfd0 2024-09-13T18:34:08.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.010+0000 7fbeef667740 1 -- 172.21.15.114:0/2788670797 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fb74a3a4e0 con 0x55fb74dedfd0 2024-09-13T18:34:08.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.010+0000 7fbed5ffb640 1 -- 172.21.15.114:0/2788670797 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fbee00bc110 con 0x55fb74dedfd0 2024-09-13T18:34:08.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.016+0000 7fbed5ffb640 1 -- 172.21.15.114:0/2788670797 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbee004d560 con 0x55fb74dedfd0 2024-09-13T18:34:08.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.016+0000 7fbeef667740 10 client.28091 fetch_fsmap learned FSMap version 39 2024-09-13T18:34:08.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.016+0000 7fbeef667740 1 -- 172.21.15.114:0/2788670797 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55fb74d45620 con 0x55fb74dedfd0 2024-09-13T18:34:08.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.018+0000 7fbed5ffb640 1 -- 172.21.15.114:0/2788670797 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbee00d8030 con 0x55fb74dedfd0 2024-09-13T18:34:08.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.018+0000 7fbed5ffb640 1 -- 172.21.15.114:0/2788670797 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 39) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fbee00b6d20 con 0x55fb74dedfd0 2024-09-13T18:34:08.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.020+0000 7fbeef667740 10 client.28091 fetch_fsmap finished waiting for FSMap version 39 2024-09-13T18:34:08.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.020+0000 7fbeef667740 10 client.28091 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:34:08.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.020+0000 7fbeef667740 20 client.28091 populate_metadata read hostname 'smithi114' 2024-09-13T18:34:08.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.020+0000 7fbeef667740 1 --2- 172.21.15.114:0/2788670797 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55fb74e1fbb0 0x55fb74e21fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:08.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.020+0000 7fbeef667740 4 client.28091 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:34:08.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.020+0000 7fbeef667740 1 -- 172.21.15.114:0/2788670797 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fb74a4b7e0 con 0x55fb74e1fbb0 2024-09-13T18:34:08.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.025+0000 7fbee659c640 1 --2- 172.21.15.114:0/2788670797 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55fb74e1fbb0 0x55fb74e21fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:08.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.040+0000 7fbee659c640 1 --2- 172.21.15.114:0/2788670797 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55fb74e1fbb0 0x55fb74e21fd0 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dc0ac1371c2466 server_cookie=553d1907677970b3 in_seq=0 out_seq=0 2024-09-13T18:34:08.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.041+0000 7fbed5ffb640 10 client.28091 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:34:08.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.041+0000 7fbed5ffb640 1 -- 172.21.15.114:0/2788670797 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55fb74a4b7e0 con 0x55fb74e1fbb0 2024-09-13T18:34:08.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.041+0000 7fbed5ffb640 10 client.28091 handle_command_reply: tid=0 2024-09-13T18:34:08.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.042+0000 7fbeef667740 1 client.28091 shutdown 2024-09-13T18:34:08.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.045+0000 7fbeef667740 1 -- 172.21.15.114:0/2788670797 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb74dedfd0 msgr2=0x55fb74deb1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:08.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.045+0000 7fbeef667740 1 --2- 172.21.15.114:0/2788670797 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb74dedfd0 0x55fb74deb1a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fbee00ec750 tx=0x7fbee00bc810 comp rx=0 tx=0).stop 2024-09-13T18:34:08.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.048+0000 7fbeef667740 1 -- 172.21.15.114:0/2788670797 shutdown_connections 2024-09-13T18:34:08.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.048+0000 7fbeef667740 1 --2- 172.21.15.114:0/2788670797 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55fb74e1fbb0 0x55fb74e21fd0 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:08.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.048+0000 7fbeef667740 1 --2- 172.21.15.114:0/2788670797 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb74dedfd0 0x55fb74deb1a0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:08.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.048+0000 7fbeef667740 1 --2- 172.21.15.114:0/2788670797 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb74de3730 0x55fb74deac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:08.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.048+0000 7fbeef667740 1 --2- 172.21.15.114:0/2788670797 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb74de2e10 0x55fb74ded930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:08.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.048+0000 7fbeef667740 1 -- 172.21.15.114:0/2788670797 >> 172.21.15.114:0/2788670797 conn(0x55fb74ddf380 msgr2=0x55fb74de5100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:08.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.049+0000 7fbed5ffb640 0 client.28091 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:34:08.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.053+0000 7fbeef667740 1 -- 172.21.15.114:0/2788670797 shutdown_connections 2024-09-13T18:34:08.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.056+0000 7fbeef667740 1 -- 172.21.15.114:0/2788670797 wait complete. 2024-09-13T18:34:08.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.062+0000 7fbeef667740 20 client.28091 trim_cache size 0 max 16384 2024-09-13T18:34:08.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.092+0000 7fbeef667740 1 Processor -- start 2024-09-13T18:34:08.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.097+0000 7fbeef667740 1 -- start start 2024-09-13T18:34:08.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.097+0000 7fbeef667740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb74de2e10 0x55fb74ded490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:08.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.097+0000 7fbeef667740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb74de3730 0x55fb74e16dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:08.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.097+0000 7fbee659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb74de2e10 0x55fb74ded490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:08.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.097+0000 7fbee659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb74de2e10 0x55fb74ded490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:54896/0 (socket says 172.21.15.114:54896) 2024-09-13T18:34:08.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.097+0000 7fbee659c640 1 -- 172.21.15.114:0/1613633303 learned_addr learned my addr 172.21.15.114:0/1613633303 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:34:08.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.097+0000 7fbeef667740 1 --2- 172.21.15.114:0/1613633303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb74dedfd0 0x55fb74e17310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:08.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.097+0000 7fbeef667740 1 -- 172.21.15.114:0/1613633303 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb74de9130 con 0x55fb74de2e10 2024-09-13T18:34:08.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.097+0000 7fbeef667740 1 -- 172.21.15.114:0/1613633303 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb74de3520 con 0x55fb74de3730 2024-09-13T18:34:08.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.097+0000 7fbeef667740 1 -- 172.21.15.114:0/1613633303 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb74deb4b0 con 0x55fb74dedfd0 2024-09-13T18:34:08.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.097+0000 7fbeddd9b640 1 --2- 172.21.15.114:0/1613633303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb74de3730 0x55fb74e16dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:08.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.099+0000 7fbee6d9d640 1 --2- 172.21.15.114:0/1613633303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb74dedfd0 0x55fb74e17310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:08.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.102+0000 7fbeddd9b640 1 -- 172.21.15.114:0/1613633303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb74dedfd0 msgr2=0x55fb74e17310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:08.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.102+0000 7fbeddd9b640 1 --2- 172.21.15.114:0/1613633303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb74dedfd0 0x55fb74e17310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:08.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.102+0000 7fbeddd9b640 1 -- 172.21.15.114:0/1613633303 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb74de2e10 msgr2=0x55fb74ded490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:08.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.102+0000 7fbeddd9b640 1 --2- 172.21.15.114:0/1613633303 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb74de2e10 0x55fb74ded490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:08.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.102+0000 7fbeddd9b640 1 -- 172.21.15.114:0/1613633303 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb74ce7790 con 0x55fb74de3730 2024-09-13T18:34:08.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.106+0000 7fbeddd9b640 1 --2- 172.21.15.114:0/1613633303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb74de3730 0x55fb74e16dd0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x55fb74dea470 tx=0x7fbed80093d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cae317e2daecb9c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:08.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.110+0000 7fbedd59a640 1 -- 172.21.15.114:0/1613633303 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed80132b0 con 0x55fb74de3730 2024-09-13T18:34:08.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.110+0000 7fbedd59a640 1 -- 172.21.15.114:0/1613633303 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbed8004030 con 0x55fb74de3730 2024-09-13T18:34:08.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.110+0000 7fbedd59a640 1 -- 172.21.15.114:0/1613633303 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed800e980 con 0x55fb74de3730 2024-09-13T18:34:08.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.112+0000 7fbeef667740 1 -- 172.21.15.114:0/1613633303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb74de3730 msgr2=0x55fb74e16dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:08.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.112+0000 7fbeef667740 1 --2- 172.21.15.114:0/1613633303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb74de3730 0x55fb74e16dd0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x55fb74dea470 tx=0x7fbed80093d0 comp rx=0 tx=0).stop 2024-09-13T18:34:08.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.115+0000 7fbeef667740 1 -- 172.21.15.114:0/1613633303 shutdown_connections 2024-09-13T18:34:08.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.115+0000 7fbeef667740 1 --2- 172.21.15.114:0/1613633303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb74dedfd0 0x55fb74e17310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:08.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.115+0000 7fbeef667740 1 --2- 172.21.15.114:0/1613633303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb74de3730 0x55fb74e16dd0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:08.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.115+0000 7fbeef667740 1 --2- 172.21.15.114:0/1613633303 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb74de2e10 0x55fb74ded490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:08.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.115+0000 7fbeef667740 1 -- 172.21.15.114:0/1613633303 >> 172.21.15.114:0/1613633303 conn(0x55fb74ddf380 msgr2=0x55fb74ddf760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:08.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.115+0000 7fbeef667740 1 -- 172.21.15.114:0/1613633303 shutdown_connections 2024-09-13T18:34:08.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.115+0000 7fbeef667740 1 -- 172.21.15.114:0/1613633303 wait complete. 2024-09-13T18:34:08.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.116+0000 7fbeef667740 1 -- start start 2024-09-13T18:34:08.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.116+0000 7fbeef667740 1 Processor -- start 2024-09-13T18:34:08.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.116+0000 7fbeef667740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb74de2e10 0x55fb74e131c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:08.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.116+0000 7fbeef667740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb74de3730 0x55fb74e13700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:08.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.116+0000 7fbeef667740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb74dedfd0 0x55fb74e6f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:08.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.116+0000 7fbeef667740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb74e6f5b0 con 0x55fb74de2e10 2024-09-13T18:34:08.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.116+0000 7fbeef667740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb74e6f720 con 0x55fb74dedfd0 2024-09-13T18:34:08.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.116+0000 7fbeef667740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb74e6f8e0 con 0x55fb74de3730 2024-09-13T18:34:08.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.116+0000 7fbeef667740 10 client.0 _ll_register_callbacks cb 0x7fbee0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:34:08.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.116+0000 7fbee659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb74de2e10 0x55fb74e131c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:08.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.117+0000 7fbee659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb74de2e10 0x55fb74e131c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:54906/0 (socket says 172.21.15.114:54906) 2024-09-13T18:34:08.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.117+0000 7fbee659c640 1 -- 172.21.15.114:0/514271152 learned_addr learned my addr 172.21.15.114:0/514271152 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:34:08.120 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:07 smithi134 ceph-mon[34283]: pgmap v362: 129 pgs: 129 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 50 KiB/s wr, 32 op/s 2024-09-13T18:34:08.120 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:07 smithi134 ceph-mon[34283]: osdmap e105: 12 total, 12 up, 12 in 2024-09-13T18:34:08.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.117+0000 7fbee6d9d640 1 --2- 172.21.15.114:0/514271152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb74dedfd0 0x55fb74e6f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:08.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.117+0000 7fbee6d9d640 1 -- 172.21.15.114:0/514271152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb74de3730 msgr2=0x55fb74e13700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:08.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.117+0000 7fbee6d9d640 1 --2- 172.21.15.114:0/514271152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb74de3730 0x55fb74e13700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:08.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.117+0000 7fbee6d9d640 1 -- 172.21.15.114:0/514271152 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb74de2e10 msgr2=0x55fb74e131c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:08.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.117+0000 7fbee6d9d640 1 --2- 172.21.15.114:0/514271152 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb74de2e10 0x55fb74e131c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:08.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.117+0000 7fbee6d9d640 1 -- 172.21.15.114:0/514271152 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fb74d01840 con 0x55fb74dedfd0 2024-09-13T18:34:08.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.119+0000 7fbee6d9d640 1 --2- 172.21.15.114:0/514271152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb74dedfd0 0x55fb74e6f070 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fbee00bbe10 tx=0x7fbee0009830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d23448c984667b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:08.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.119+0000 7fbed5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:34:08.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.122+0000 7fbed5ffb640 1 -- 172.21.15.114:0/514271152 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee004d560 con 0x55fb74dedfd0 2024-09-13T18:34:08.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.122+0000 7fbed5ffb640 1 -- 172.21.15.114:0/514271152 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fbee00bc110 con 0x55fb74dedfd0 2024-09-13T18:34:08.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.123+0000 7fbeef667740 1 -- 172.21.15.114:0/514271152 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbed8019030 con 0x55fb74dedfd0 2024-09-13T18:34:08.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.123+0000 7fbed5ffb640 1 -- 172.21.15.114:0/514271152 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbee00bb8b0 con 0x55fb74dedfd0 2024-09-13T18:34:08.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.128+0000 7fbeef667740 10 client.28097 fetch_fsmap learned FSMap version 39 2024-09-13T18:34:08.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.128+0000 7fbeef667740 1 -- 172.21.15.114:0/514271152 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55fb74d0cea0 con 0x55fb74dedfd0 2024-09-13T18:34:08.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.129+0000 7fbed5ffb640 1 -- 172.21.15.114:0/514271152 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbed8019030 con 0x55fb74dedfd0 2024-09-13T18:34:08.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.129+0000 7fbed5ffb640 1 -- 172.21.15.114:0/514271152 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 39) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fbee00b6d20 con 0x55fb74dedfd0 2024-09-13T18:34:08.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.129+0000 7fbeef667740 10 client.28097 fetch_fsmap finished waiting for FSMap version 39 2024-09-13T18:34:08.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.129+0000 7fbeef667740 10 client.28097 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:34:08.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.129+0000 7fbeef667740 20 client.28097 populate_metadata read hostname 'smithi114' 2024-09-13T18:34:08.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.129+0000 7fbeef667740 1 --2- 172.21.15.114:0/514271152 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55fb74e76c70 0x55fb74e79060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:08.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.129+0000 7fbeef667740 4 client.28097 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-13T18:34:08.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.129+0000 7fbeef667740 1 -- 172.21.15.114:0/514271152 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55fb74a9ee30 con 0x55fb74e76c70 2024-09-13T18:34:08.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.137+0000 7fbee659c640 1 --2- 172.21.15.114:0/514271152 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55fb74e76c70 0x55fb74e79060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:08.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.142+0000 7fbee659c640 1 --2- 172.21.15.114:0/514271152 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55fb74e76c70 0x55fb74e79060 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7369b8abf6112459 server_cookie=c3cf3e4326c7a1b5 in_seq=0 out_seq=0 2024-09-13T18:34:08.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.142+0000 7fbed5ffb640 10 client.28097 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:34:08.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.142+0000 7fbed5ffb640 1 -- 172.21.15.114:0/514271152 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55fb74a9ee30 con 0x55fb74e76c70 2024-09-13T18:34:08.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.142+0000 7fbed5ffb640 10 client.28097 handle_command_reply: tid=0 2024-09-13T18:34:08.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.142+0000 7fbeef667740 1 client.28097 shutdown 2024-09-13T18:34:08.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.147+0000 7fbeef667740 1 -- 172.21.15.114:0/514271152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb74dedfd0 msgr2=0x55fb74e6f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:08.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.147+0000 7fbeef667740 1 --2- 172.21.15.114:0/514271152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb74dedfd0 0x55fb74e6f070 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fbee00bbe10 tx=0x7fbee0009830 comp rx=0 tx=0).stop 2024-09-13T18:34:08.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.153+0000 7fbeef667740 1 -- 172.21.15.114:0/514271152 shutdown_connections 2024-09-13T18:34:08.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.153+0000 7fbeef667740 1 --2- 172.21.15.114:0/514271152 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x55fb74e76c70 0x55fb74e79060 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:08.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.153+0000 7fbeef667740 1 --2- 172.21.15.114:0/514271152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb74dedfd0 0x55fb74e6f070 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:08.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.153+0000 7fbeef667740 1 --2- 172.21.15.114:0/514271152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb74de3730 0x55fb74e13700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:08.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.153+0000 7fbeef667740 1 --2- 172.21.15.114:0/514271152 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb74de2e10 0x55fb74e131c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:08.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.153+0000 7fbeef667740 1 -- 172.21.15.114:0/514271152 >> 172.21.15.114:0/514271152 conn(0x55fb74ddf380 msgr2=0x55fb74de5ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:08.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.153+0000 7fbed5ffb640 0 client.28097 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:34:08.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.155+0000 7fbeef667740 1 -- 172.21.15.114:0/514271152 shutdown_connections 2024-09-13T18:34:08.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.157+0000 7fbeef667740 1 -- 172.21.15.114:0/514271152 wait complete. 2024-09-13T18:34:08.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.159+0000 7fbeef667740 20 client.28097 trim_cache size 0 max 16384 2024-09-13T18:34:08.162 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:34:08.162 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "no active scrubs running", 2024-09-13T18:34:08.162 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": {} 2024-09-13T18:34:08.162 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:34:08.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.161+0000 7fbeec945640 1 -- 172.21.15.114:0/2275575128 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbec80a1410 msgr2=0x7fbec80a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:08.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.161+0000 7fbeec945640 1 --2- 172.21.15.114:0/2275575128 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbec80a1410 0x7fbec80a38d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fbed00058c0 tx=0x7fbed0005d00 comp rx=0 tx=0).stop 2024-09-13T18:34:08.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.162+0000 7fbeec945640 1 -- 172.21.15.114:0/2275575128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbee8119400 msgr2=0x7fbee810ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:08.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.162+0000 7fbeec945640 1 --2- 172.21.15.114:0/2275575128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbee8119400 0x7fbee810ffe0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fbee0030820 tx=0x7fbee0030850 comp rx=0 tx=0).stop 2024-09-13T18:34:08.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.162+0000 7fbeec945640 1 -- 172.21.15.114:0/2275575128 shutdown_connections 2024-09-13T18:34:08.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.162+0000 7fbeec945640 1 --2- 172.21.15.114:0/2275575128 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbec80a1410 0x7fbec80a38d0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:08.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.162+0000 7fbeec945640 1 --2- 172.21.15.114:0/2275575128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbee8119400 0x7fbee810ffe0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:08.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.162+0000 7fbeec945640 1 --2- 172.21.15.114:0/2275575128 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbee8107920 0x7fbee810faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:08.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.162+0000 7fbeec945640 1 --2- 172.21.15.114:0/2275575128 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbee8107540 0x7fbee81169d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:08.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.162+0000 7fbeec945640 1 -- 172.21.15.114:0/2275575128 >> 172.21.15.114:0/2275575128 conn(0x7fbee806afe0 msgr2=0x7fbee8119d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:08.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.164+0000 7fbeec945640 1 -- 172.21.15.114:0/2275575128 shutdown_connections 2024-09-13T18:34:08.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.167+0000 7fbeec945640 1 -- 172.21.15.114:0/2275575128 wait complete. 2024-09-13T18:34:08.189 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-13T18:34:08.189 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:34:08.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.377+0000 7fc204461640 1 -- 172.21.15.114:0/3309017561 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc1fc107890 msgr2=0x7fc1fc107cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:08.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.377+0000 7fc204461640 1 --2- 172.21.15.114:0/3309017561 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc1fc107890 0x7fc1fc107cf0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec009900 tx=0x7fc1ec02a440 comp rx=0 tx=0).stop 2024-09-13T18:34:08.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.383+0000 7fc204461640 1 -- 172.21.15.114:0/3309017561 shutdown_connections 2024-09-13T18:34:08.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.383+0000 7fc204461640 1 --2- 172.21.15.114:0/3309017561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1fc119400 0x7fc1fc11b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:08.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.383+0000 7fc204461640 1 --2- 172.21.15.114:0/3309017561 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc1fc107890 0x7fc1fc107cf0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:08.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.383+0000 7fc204461640 1 --2- 172.21.15.114:0/3309017561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1fc1074b0 0x7fc1fc10e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:08.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.383+0000 7fc204461640 1 -- 172.21.15.114:0/3309017561 >> 172.21.15.114:0/3309017561 conn(0x7fc1fc06aff0 msgr2=0x7fc1fc06b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:08.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.388+0000 7fc204461640 1 -- 172.21.15.114:0/3309017561 shutdown_connections 2024-09-13T18:34:08.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.389+0000 7fc204461640 1 -- 172.21.15.114:0/3309017561 wait complete. 2024-09-13T18:34:08.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.390+0000 7fc204461640 1 Processor -- start 2024-09-13T18:34:08.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.390+0000 7fc204461640 1 -- start start 2024-09-13T18:34:08.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.391+0000 7fc204461640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc1fc1074b0 0x7fc1fc1168d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:08.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.391+0000 7fc204461640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1fc107890 0x7fc1fc116e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:08.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.391+0000 7fc204461640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1fc119400 0x7fc1fc10fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:08.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.391+0000 7fc204461640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1fc110310 con 0x7fc1fc1074b0 2024-09-13T18:34:08.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.391+0000 7fc204461640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1fc110480 con 0x7fc1fc119400 2024-09-13T18:34:08.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.391+0000 7fc204461640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1fc110640 con 0x7fc1fc107890 2024-09-13T18:34:08.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.391+0000 7fc2019fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1fc107890 0x7fc1fc116e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:08.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.392+0000 7fc2019fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1fc107890 0x7fc1fc116e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:54764/0 (socket says 172.21.15.114:54764) 2024-09-13T18:34:08.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.392+0000 7fc2019fc640 1 -- 172.21.15.114:0/2279794526 learned_addr learned my addr 172.21.15.114:0/2279794526 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:34:08.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.392+0000 7fc2029fe640 1 --2- 172.21.15.114:0/2279794526 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1fc119400 0x7fc1fc10fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:08.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.393+0000 7fc2019fc640 1 -- 172.21.15.114:0/2279794526 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1fc119400 msgr2=0x7fc1fc10fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:08.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.393+0000 7fc2019fc640 1 --2- 172.21.15.114:0/2279794526 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1fc119400 0x7fc1fc10fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:08.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.393+0000 7fc2019fc640 1 -- 172.21.15.114:0/2279794526 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc1fc1074b0 msgr2=0x7fc1fc1168d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:08.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.393+0000 7fc2019fc640 1 --2- 172.21.15.114:0/2279794526 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc1fc1074b0 0x7fc1fc1168d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:08.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.393+0000 7fc2019fc640 1 -- 172.21.15.114:0/2279794526 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1ec009560 con 0x7fc1fc107890 2024-09-13T18:34:08.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.394+0000 7fc2019fc640 1 --2- 172.21.15.114:0/2279794526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1fc107890 0x7fc1fc116e30 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec006010 tx=0x7fc1ec0055a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:08.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.395+0000 7fc1f37fe640 1 -- 172.21.15.114:0/2279794526 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ec004b50 con 0x7fc1fc107890 2024-09-13T18:34:08.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.395+0000 7fc204461640 1 -- 172.21.15.114:0/2279794526 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1fc114460 con 0x7fc1fc107890 2024-09-13T18:34:08.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.396+0000 7fc204461640 1 -- 172.21.15.114:0/2279794526 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1fc114910 con 0x7fc1fc107890 2024-09-13T18:34:08.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.396+0000 7fc1f37fe640 1 -- 172.21.15.114:0/2279794526 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc1ec004cb0 con 0x7fc1fc107890 2024-09-13T18:34:08.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.396+0000 7fc1f37fe640 1 -- 172.21.15.114:0/2279794526 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ec03c640 con 0x7fc1fc107890 2024-09-13T18:34:08.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.400+0000 7fc204461640 1 -- 172.21.15.114:0/2279794526 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1e0005250 con 0x7fc1fc107890 2024-09-13T18:34:08.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.408+0000 7fc1f37fe640 1 -- 172.21.15.114:0/2279794526 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fc1ec039040 con 0x7fc1fc107890 2024-09-13T18:34:08.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.409+0000 7fc1f37fe640 1 --2- 172.21.15.114:0/2279794526 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc1e80a1410 0x7fc1e80a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:08.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.409+0000 7fc2021fd640 1 --2- 172.21.15.114:0/2279794526 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc1e80a1410 0x7fc1e80a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:08.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.410+0000 7fc1f37fe640 1 -- 172.21.15.114:0/2279794526 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fc1ec0ee870 con 0x7fc1fc107890 2024-09-13T18:34:08.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.410+0000 7fc2021fd640 1 --2- 172.21.15.114:0/2279794526 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc1e80a1410 0x7fc1e80a38d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc1f80096f0 tx=0x7fc1f8006c40 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:08.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.413+0000 7fc1f37fe640 1 -- 172.21.15.114:0/2279794526 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fc1ec0b7a00 con 0x7fc1fc107890 2024-09-13T18:34:08.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.804+0000 7fc204461640 1 -- 172.21.15.114:0/2279794526 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc1e00057d0 con 0x7fc1fc107890 2024-09-13T18:34:08.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.809+0000 7fc1f37fe640 1 -- 172.21.15.114:0/2279794526 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+10945 (secure 0 0 0) 0x7fc1ec035020 con 0x7fc1fc107890 2024-09-13T18:34:08.811 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:34:08.812 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4162817497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4162817497},{"type":"v1","addr":"172.21.15.114:6835","nonce":4162817497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/2435428478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":2435428478},{"type":"v1","addr":"172.21.15.114:6837","nonce":2435428478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T18:20:01.895117+0000","modified":"2024-09-13T18:33:48.420534+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24482,"mds_2":24475,"mds_3":24458,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/2046516451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":2046516451},{"type":"v1","addr":"172.21.15.114:6839","nonce":2046516451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":10,"addr":"172.21.15.114:6841/1956923446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":1956923446},{"type":"v1","addr":"172.21.15.114:6841","nonce":1956923446}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24391":{"gid":24391,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/507301496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":507301496},{"type":"v1","addr":"172.21.15.204:6833","nonce":507301496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4028657219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4028657219},{"type":"v1","addr":"172.21.15.134:6833","nonce":4028657219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/57029119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":57029119},{"type":"v1","addr":"172.21.15.204:6835","nonce":57029119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3068880939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3068880939},{"type":"v1","addr":"172.21.15.204:6837","nonce":3068880939}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"e","rank":3,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.134:6835/1150003275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1150003275},{"type":"v1","addr":"172.21.15.134:6835","nonce":1150003275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/344883912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":344883912},{"type":"v1","addr":"172.21.15.204:6839","nonce":344883912}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6837/891692210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":891692210},{"type":"v1","addr":"172.21.15.134:6837","nonce":891692210}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6839/3466309724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3466309724},{"type":"v1","addr":"172.21.15.134:6839","nonce":3466309724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:34:08.812 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 39 2024-09-13T18:34:08.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.814+0000 7fc204461640 1 -- 172.21.15.114:0/2279794526 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc1e80a1410 msgr2=0x7fc1e80a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:08.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.814+0000 7fc204461640 1 --2- 172.21.15.114:0/2279794526 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc1e80a1410 0x7fc1e80a38d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc1f80096f0 tx=0x7fc1f8006c40 comp rx=0 tx=0).stop 2024-09-13T18:34:08.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.814+0000 7fc204461640 1 -- 172.21.15.114:0/2279794526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1fc107890 msgr2=0x7fc1fc116e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:08.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.814+0000 7fc204461640 1 --2- 172.21.15.114:0/2279794526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1fc107890 0x7fc1fc116e30 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec006010 tx=0x7fc1ec0055a0 comp rx=0 tx=0).stop 2024-09-13T18:34:08.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.815+0000 7fc204461640 1 -- 172.21.15.114:0/2279794526 shutdown_connections 2024-09-13T18:34:08.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.815+0000 7fc204461640 1 --2- 172.21.15.114:0/2279794526 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc1e80a1410 0x7fc1e80a38d0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:08.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.815+0000 7fc204461640 1 --2- 172.21.15.114:0/2279794526 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1fc119400 0x7fc1fc10fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:08.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.815+0000 7fc204461640 1 --2- 172.21.15.114:0/2279794526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1fc107890 0x7fc1fc116e30 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:08.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.815+0000 7fc204461640 1 --2- 172.21.15.114:0/2279794526 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc1fc1074b0 0x7fc1fc1168d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:08.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.815+0000 7fc204461640 1 -- 172.21.15.114:0/2279794526 >> 172.21.15.114:0/2279794526 conn(0x7fc1fc06aff0 msgr2=0x7fc1fc119dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:08.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.815+0000 7fc204461640 1 -- 172.21.15.114:0/2279794526 shutdown_connections 2024-09-13T18:34:08.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:08.817+0000 7fc204461640 1 -- 172.21.15.114:0/2279794526 wait complete. 2024-09-13T18:34:08.840 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-09-13T18:34:09.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.027+0000 7f6203261640 1 -- 172.21.15.114:0/2651075832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61fc0fee60 msgr2=0x7f61fc0ff240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:09.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.027+0000 7f6203261640 1 --2- 172.21.15.114:0/2651075832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61fc0fee60 0x7f61fc0ff240 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f61f0009860 tx=0x7f61f002a340 comp rx=0 tx=0).stop 2024-09-13T18:34:09.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.027+0000 7f6203261640 1 -- 172.21.15.114:0/2651075832 shutdown_connections 2024-09-13T18:34:09.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.027+0000 7f6203261640 1 --2- 172.21.15.114:0/2651075832 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f61fc1040c0 0x7f61fc10bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.027+0000 7f6203261640 1 --2- 172.21.15.114:0/2651075832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61fc0ff810 0x7f61fc103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.027+0000 7f6203261640 1 --2- 172.21.15.114:0/2651075832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61fc0fee60 0x7f61fc0ff240 secure :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f61f0009860 tx=0x7f61f002a340 comp rx=0 tx=0).stop 2024-09-13T18:34:09.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.027+0000 7f6203261640 1 -- 172.21.15.114:0/2651075832 >> 172.21.15.114:0/2651075832 conn(0x7f61fc0fa8d0 msgr2=0x7f61fc0fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:09.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.033+0000 7f6203261640 1 -- 172.21.15.114:0/2651075832 shutdown_connections 2024-09-13T18:34:09.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.034+0000 7f6203261640 1 -- 172.21.15.114:0/2651075832 wait complete. 2024-09-13T18:34:09.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.035+0000 7f6203261640 1 Processor -- start 2024-09-13T18:34:09.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.039+0000 7f6203261640 1 -- start start 2024-09-13T18:34:09.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.039+0000 7f6203261640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61fc1040c0 0x7f61fc072160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.039+0000 7f6203261640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61fc06c160 0x7f61fc06c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.039+0000 7f6203261640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f61fc06cb00 0x7f61fc1aa440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.039+0000 7f6203261640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61fc0727f0 con 0x7f61fc06cb00 2024-09-13T18:34:09.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.039+0000 7f6203261640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61fc1aa980 con 0x7f61fc1040c0 2024-09-13T18:34:09.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.039+0000 7f6203261640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61fc1aaac0 con 0x7f61fc06c160 2024-09-13T18:34:09.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.041+0000 7f61fbfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61fc06c160 0x7f61fc06c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.041+0000 7f61fbfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61fc06c160 0x7f61fc06c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:54780/0 (socket says 172.21.15.114:54780) 2024-09-13T18:34:09.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.041+0000 7f61fbfff640 1 -- 172.21.15.114:0/1911619591 learned_addr learned my addr 172.21.15.114:0/1911619591 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:34:09.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.041+0000 7f62017fe640 1 --2- 172.21.15.114:0/1911619591 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f61fc06cb00 0x7f61fc1aa440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.042+0000 7f61fbfff640 1 -- 172.21.15.114:0/1911619591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61fc1040c0 msgr2=0x7f61fc072160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:34:09.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.042+0000 7f61fbfff640 1 --2- 172.21.15.114:0/1911619591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61fc1040c0 0x7f61fc072160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.042+0000 7f61fbfff640 1 -- 172.21.15.114:0/1911619591 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f61fc06cb00 msgr2=0x7f61fc1aa440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:09.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.042+0000 7f61fbfff640 1 --2- 172.21.15.114:0/1911619591 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f61fc06cb00 0x7f61fc1aa440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.042+0000 7f61fbfff640 1 -- 172.21.15.114:0/1911619591 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f0009490 con 0x7f61fc06c160 2024-09-13T18:34:09.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.043+0000 7f61fbfff640 1 --2- 172.21.15.114:0/1911619591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61fc06c160 0x7f61fc06c5c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f61e8009680 tx=0x7f61e8009b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:09.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.048+0000 7f61f9ffb640 1 -- 172.21.15.114:0/1911619591 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e80079a0 con 0x7f61fc06c160 2024-09-13T18:34:09.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.048+0000 7f61f9ffb640 1 -- 172.21.15.114:0/1911619591 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f61e8015420 con 0x7f61fc06c160 2024-09-13T18:34:09.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.049+0000 7f61f9ffb640 1 -- 172.21.15.114:0/1911619591 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e80145e0 con 0x7f61fc06c160 2024-09-13T18:34:09.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.049+0000 7f6203261640 1 -- 172.21.15.114:0/1911619591 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61fc068220 con 0x7f61fc06c160 2024-09-13T18:34:09.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.050+0000 7f6203261640 1 -- 172.21.15.114:0/1911619591 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61fc0686e0 con 0x7f61fc06c160 2024-09-13T18:34:09.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.054+0000 7f620605d740 1 Processor -- start 2024-09-13T18:34:09.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.054+0000 7f620605d740 1 -- start start 2024-09-13T18:34:09.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.054+0000 7f620605d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f3ee52fa0 0x563f3ee53380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.054+0000 7f620605d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f3ee538c0 0x563f3ee5dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.054+0000 7f620605d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f3ee5e160 0x563f3ee60550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.054+0000 7f620605d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f3edb4cc0 con 0x563f3ee52fa0 2024-09-13T18:34:09.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.054+0000 7f620605d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f3edb2450 con 0x563f3ee538c0 2024-09-13T18:34:09.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.054+0000 7f620605d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f3edadd40 con 0x563f3ee5e160 2024-09-13T18:34:09.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.056+0000 7f61f9ffb640 1 -- 172.21.15.114:0/1911619591 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f61e800d020 con 0x7f61fc06c160 2024-09-13T18:34:09.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.056+0000 7f6200ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f3ee538c0 0x563f3ee5dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.057+0000 7f6200ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f3ee538c0 0x563f3ee5dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:59148/0 (socket says 172.21.15.114:59148) 2024-09-13T18:34:09.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.057+0000 7f6200ffd640 1 -- 172.21.15.114:0/2190099105 learned_addr learned my addr 172.21.15.114:0/2190099105 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:34:09.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.058+0000 7f6200ffd640 1 -- 172.21.15.114:0/2190099105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f3ee5e160 msgr2=0x563f3ee60550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:09.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.058+0000 7f62017fe640 1 --2- 172.21.15.114:0/2190099105 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f3ee52fa0 0x563f3ee53380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.059+0000 7f61f9ffb640 1 --2- 172.21.15.114:0/1911619591 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f61ec0a1410 0x7f61ec0a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.059+0000 7f61fbfff640 1 --2- 172.21.15.114:0/2190099105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f3ee5e160 0x563f3ee60550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.060+0000 7f61f9ffb640 1 -- 172.21.15.114:0/1911619591 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f61e80cc670 con 0x7f61fc06c160 2024-09-13T18:34:09.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.060+0000 7f6200ffd640 1 --2- 172.21.15.114:0/2190099105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f3ee5e160 0x563f3ee60550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.061+0000 7f6200ffd640 1 -- 172.21.15.114:0/2190099105 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f3ee52fa0 msgr2=0x563f3ee53380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:09.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.061+0000 7f6200ffd640 1 --2- 172.21.15.114:0/2190099105 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f3ee52fa0 0x563f3ee53380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.061+0000 7f6200ffd640 1 -- 172.21.15.114:0/2190099105 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f3ed75620 con 0x563f3ee538c0 2024-09-13T18:34:09.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.061+0000 7f62017fe640 1 --2- 172.21.15.114:0/2190099105 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f3ee52fa0 0x563f3ee53380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:34:09.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.061+0000 7f6200ffd640 1 --2- 172.21.15.114:0/2190099105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f3ee538c0 0x563f3ee5dc20 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f61f002a8b0 tx=0x7f61f00303b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=431438453ca4de59 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:09.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.061+0000 7f61fbfff640 1 --2- 172.21.15.114:0/2190099105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f3ee5e160 0x563f3ee60550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:34:09.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.066+0000 7f62017fe640 1 --2- 172.21.15.114:0/1911619591 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f61ec0a1410 0x7f61ec0a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.066+0000 7f61e6ffd640 1 -- 172.21.15.114:0/2190099105 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f0032760 con 0x563f3ee538c0 2024-09-13T18:34:09.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.066+0000 7f61e6ffd640 1 -- 172.21.15.114:0/2190099105 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f61f0004030 con 0x563f3ee538c0 2024-09-13T18:34:09.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.067+0000 7f61e6ffd640 1 -- 172.21.15.114:0/2190099105 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f00055b0 con 0x563f3ee538c0 2024-09-13T18:34:09.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.070+0000 7f620605d740 1 -- 172.21.15.114:0/2190099105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f3ee538c0 msgr2=0x563f3ee5dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:09.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.070+0000 7f620605d740 1 --2- 172.21.15.114:0/2190099105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f3ee538c0 0x563f3ee5dc20 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f61f002a8b0 tx=0x7f61f00303b0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.072+0000 7f620605d740 1 -- 172.21.15.114:0/2190099105 shutdown_connections 2024-09-13T18:34:09.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.072+0000 7f620605d740 1 --2- 172.21.15.114:0/2190099105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f3ee5e160 0x563f3ee60550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.072+0000 7f620605d740 1 --2- 172.21.15.114:0/2190099105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f3ee538c0 0x563f3ee5dc20 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.072+0000 7f620605d740 1 --2- 172.21.15.114:0/2190099105 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f3ee52fa0 0x563f3ee53380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.072+0000 7f620605d740 1 -- 172.21.15.114:0/2190099105 >> 172.21.15.114:0/2190099105 conn(0x563f3ee4f510 msgr2=0x563f3ee4f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:09.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.072+0000 7f62017fe640 1 --2- 172.21.15.114:0/1911619591 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f61ec0a1410 0x7f61ec0a38d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f61f4010fd0 tx=0x7f61f4004a40 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:09.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.073+0000 7f620605d740 1 -- 172.21.15.114:0/2190099105 shutdown_connections 2024-09-13T18:34:09.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.073+0000 7f620605d740 1 -- 172.21.15.114:0/2190099105 wait complete. 2024-09-13T18:34:09.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.074+0000 7f620605d740 1 -- start start 2024-09-13T18:34:09.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.074+0000 7f620605d740 1 Processor -- start 2024-09-13T18:34:09.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.074+0000 7f620605d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f3ee52fa0 0x563f3ee5d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.074+0000 7f620605d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f3ee538c0 0x563f3ee5dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.074+0000 7f620605d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f3ee5e160 0x563f3ee5ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.074+0000 7f620605d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f3ed6c180 con 0x563f3ee5e160 2024-09-13T18:34:09.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.074+0000 7f620605d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f3ea86430 con 0x563f3ee52fa0 2024-09-13T18:34:09.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.074+0000 7f620605d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f3ed315e0 con 0x563f3ee538c0 2024-09-13T18:34:09.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.075+0000 7f620605d740 10 client.0 _ll_register_callbacks cb 0x563f3ed9daa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:34:09.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.076+0000 7f6200ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f3ee52fa0 0x563f3ee5d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.076+0000 7f6200ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f3ee52fa0 0x563f3ee5d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:59164/0 (socket says 172.21.15.114:59164) 2024-09-13T18:34:09.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.076+0000 7f6200ffd640 1 -- 172.21.15.114:0/2081501601 learned_addr learned my addr 172.21.15.114:0/2081501601 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:34:09.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.076+0000 7f61fbfff640 1 --2- 172.21.15.114:0/2081501601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f3ee538c0 0x563f3ee5dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.076+0000 7f62017fe640 1 --2- 172.21.15.114:0/2081501601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f3ee5e160 0x563f3ee5ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.076+0000 7f6200ffd640 1 -- 172.21.15.114:0/2081501601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f3ee538c0 msgr2=0x563f3ee5dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:09.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.076+0000 7f6200ffd640 1 --2- 172.21.15.114:0/2081501601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f3ee538c0 0x563f3ee5dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.076+0000 7f6200ffd640 1 -- 172.21.15.114:0/2081501601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f3ee5e160 msgr2=0x563f3ee5ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:09.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.076+0000 7f6200ffd640 1 --2- 172.21.15.114:0/2081501601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f3ee5e160 0x563f3ee5ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.076+0000 7f6200ffd640 1 -- 172.21.15.114:0/2081501601 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61fc10c450 con 0x563f3ee52fa0 2024-09-13T18:34:09.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.077+0000 7f6200ffd640 1 --2- 172.21.15.114:0/2081501601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f3ee52fa0 0x563f3ee5d690 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f61f0035040 tx=0x7f61f00043d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6316f2a862cfca72 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:09.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.077+0000 7f61dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:34:09.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.080+0000 7f61dbfff640 1 -- 172.21.15.114:0/2081501601 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f0032e50 con 0x563f3ee52fa0 2024-09-13T18:34:09.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.080+0000 7f620605d740 1 -- 172.21.15.114:0/2081501601 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563f3edd7b80 con 0x563f3ee52fa0 2024-09-13T18:34:09.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.080+0000 7f61dbfff640 1 -- 172.21.15.114:0/2081501601 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f61f00044c0 con 0x563f3ee52fa0 2024-09-13T18:34:09.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.083+0000 7f61dbfff640 1 -- 172.21.15.114:0/2081501601 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61f00055b0 con 0x563f3ee52fa0 2024-09-13T18:34:09.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.084+0000 7f620605d740 10 client.28109 fetch_fsmap learned FSMap version 39 2024-09-13T18:34:09.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.084+0000 7f620605d740 1 -- 172.21.15.114:0/2081501601 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x563f3edac3c0 con 0x563f3ee52fa0 2024-09-13T18:34:09.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.087+0000 7f61dbfff640 1 -- 172.21.15.114:0/2081501601 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563f3edd7b80 con 0x563f3ee52fa0 2024-09-13T18:34:09.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.087+0000 7f61dbfff640 1 -- 172.21.15.114:0/2081501601 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 39) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f61f0004860 con 0x563f3ee52fa0 2024-09-13T18:34:09.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.088+0000 7f620605d740 10 client.28109 fetch_fsmap finished waiting for FSMap version 39 2024-09-13T18:34:09.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.088+0000 7f620605d740 10 client.28109 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-13T18:34:09.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.088+0000 7f620605d740 20 client.28109 populate_metadata read hostname 'smithi114' 2024-09-13T18:34:09.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.088+0000 7f620605d740 1 --2- 172.21.15.114:0/2081501601 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x563f3ee8f6f0 0x563f3ee91b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.088+0000 7f620605d740 4 client.28109 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:34:09.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.088+0000 7f620605d740 1 -- 172.21.15.114:0/2081501601 --> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563f3eda01f0 con 0x563f3ee8f6f0 2024-09-13T18:34:09.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.093+0000 7f61fbfff640 1 --2- 172.21.15.114:0/2081501601 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x563f3ee8f6f0 0x563f3ee91b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.097+0000 7f61fbfff640 1 --2- 172.21.15.114:0/2081501601 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x563f3ee8f6f0 0x563f3ee91b10 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e4cb7628404a5c62 server_cookie=47bba85ac36649d6 in_seq=0 out_seq=0 2024-09-13T18:34:09.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.097+0000 7f61dbfff640 10 client.28109 ms_handle_connect on v2:172.21.15.114:6840/1956923446 2024-09-13T18:34:09.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.097+0000 7f61dbfff640 1 -- 172.21.15.114:0/2081501601 <== mds.4 v2:172.21.15.114:6840/1956923446 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x563f3eda01f0 con 0x563f3ee8f6f0 2024-09-13T18:34:09.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.097+0000 7f61dbfff640 10 client.28109 handle_command_reply: tid=0 2024-09-13T18:34:09.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.098+0000 7f620605d740 1 client.28109 shutdown 2024-09-13T18:34:09.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.102+0000 7f620605d740 1 -- 172.21.15.114:0/2081501601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f3ee52fa0 msgr2=0x563f3ee5d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:09.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.102+0000 7f620605d740 1 --2- 172.21.15.114:0/2081501601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f3ee52fa0 0x563f3ee5d690 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f61f0035040 tx=0x7f61f00043d0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.103+0000 7f620605d740 1 -- 172.21.15.114:0/2081501601 shutdown_connections 2024-09-13T18:34:09.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.103+0000 7f620605d740 1 --2- 172.21.15.114:0/2081501601 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x563f3ee8f6f0 0x563f3ee91b10 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.103+0000 7f620605d740 1 --2- 172.21.15.114:0/2081501601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f3ee5e160 0x563f3ee5ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.103+0000 7f620605d740 1 --2- 172.21.15.114:0/2081501601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f3ee538c0 0x563f3ee5dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.103+0000 7f620605d740 1 --2- 172.21.15.114:0/2081501601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f3ee52fa0 0x563f3ee5d690 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.103+0000 7f620605d740 1 -- 172.21.15.114:0/2081501601 >> 172.21.15.114:0/2081501601 conn(0x563f3ee4f510 msgr2=0x563f3ee45eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:09.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.104+0000 7f61dbfff640 0 client.28109 ms_handle_reset on v2:172.21.15.114:6840/1956923446 2024-09-13T18:34:09.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.107+0000 7f620605d740 1 -- 172.21.15.114:0/2081501601 shutdown_connections 2024-09-13T18:34:09.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.107+0000 7f620605d740 1 -- 172.21.15.114:0/2081501601 wait complete. 2024-09-13T18:34:09.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.108+0000 7f620605d740 20 client.28109 trim_cache size 0 max 16384 2024-09-13T18:34:09.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.128+0000 7f620605d740 1 Processor -- start 2024-09-13T18:34:09.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.128+0000 7f620605d740 1 -- start start 2024-09-13T18:34:09.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.128+0000 7f620605d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f3ee52fa0 0x563f3ee96470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.129+0000 7f620605d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f3ee538c0 0x563f3ee64e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.129+0000 7f6200ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f3ee52fa0 0x563f3ee96470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.129+0000 7f6200ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f3ee52fa0 0x563f3ee96470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:59178/0 (socket says 172.21.15.114:59178) 2024-09-13T18:34:09.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.129+0000 7f6200ffd640 1 -- 172.21.15.114:0/777113895 learned_addr learned my addr 172.21.15.114:0/777113895 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:34:09.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.129+0000 7f620605d740 1 --2- 172.21.15.114:0/777113895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f3ee5e160 0x563f3ee65340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.129+0000 7f620605d740 1 -- 172.21.15.114:0/777113895 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f3ee50200 con 0x563f3ee538c0 2024-09-13T18:34:09.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.129+0000 7f620605d740 1 -- 172.21.15.114:0/777113895 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f3ee428e0 con 0x563f3ee52fa0 2024-09-13T18:34:09.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.129+0000 7f620605d740 1 -- 172.21.15.114:0/777113895 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f3ee5bd50 con 0x563f3ee5e160 2024-09-13T18:34:09.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.129+0000 7f62017fe640 1 --2- 172.21.15.114:0/777113895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f3ee5e160 0x563f3ee65340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.130+0000 7f61fbfff640 1 --2- 172.21.15.114:0/777113895 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f3ee538c0 0x563f3ee64e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.130+0000 7f62017fe640 1 -- 172.21.15.114:0/777113895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f3ee52fa0 msgr2=0x563f3ee96470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:09.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.130+0000 7f62017fe640 1 --2- 172.21.15.114:0/777113895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f3ee52fa0 0x563f3ee96470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.130+0000 7f62017fe640 1 -- 172.21.15.114:0/777113895 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f3ee538c0 msgr2=0x563f3ee64e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:09.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.130+0000 7f62017fe640 1 --2- 172.21.15.114:0/777113895 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f3ee538c0 0x563f3ee64e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.130+0000 7f62017fe640 1 -- 172.21.15.114:0/777113895 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f3ed4e770 con 0x563f3ee5e160 2024-09-13T18:34:09.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.131+0000 7f6200ffd640 1 --2- 172.21.15.114:0/777113895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f3ee52fa0 0x563f3ee96470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:34:09.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.131+0000 7f62017fe640 1 --2- 172.21.15.114:0/777113895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f3ee5e160 0x563f3ee65340 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f61f403f5c0 tx=0x7f61f4014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e7b2c1d572e39f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:09.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.132+0000 7f61e77fe640 1 -- 172.21.15.114:0/777113895 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f4044070 con 0x563f3ee5e160 2024-09-13T18:34:09.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.132+0000 7f61e77fe640 1 -- 172.21.15.114:0/777113895 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f61f401a880 con 0x563f3ee5e160 2024-09-13T18:34:09.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.133+0000 7f61e77fe640 1 -- 172.21.15.114:0/777113895 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f400e8c0 con 0x563f3ee5e160 2024-09-13T18:34:09.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.133+0000 7f620605d740 1 -- 172.21.15.114:0/777113895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f3ee5e160 msgr2=0x563f3ee65340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:09.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.133+0000 7f620605d740 1 --2- 172.21.15.114:0/777113895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f3ee5e160 0x563f3ee65340 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f61f403f5c0 tx=0x7f61f4014000 comp rx=0 tx=0).stop 2024-09-13T18:34:09.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.133+0000 7f620605d740 1 -- 172.21.15.114:0/777113895 shutdown_connections 2024-09-13T18:34:09.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.133+0000 7f620605d740 1 --2- 172.21.15.114:0/777113895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f3ee5e160 0x563f3ee65340 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.133+0000 7f620605d740 1 --2- 172.21.15.114:0/777113895 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f3ee538c0 0x563f3ee64e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.133+0000 7f620605d740 1 --2- 172.21.15.114:0/777113895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f3ee52fa0 0x563f3ee96470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.133+0000 7f620605d740 1 -- 172.21.15.114:0/777113895 >> 172.21.15.114:0/777113895 conn(0x563f3ee4f510 msgr2=0x563f3ee698b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:09.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.137+0000 7f620605d740 1 -- 172.21.15.114:0/777113895 shutdown_connections 2024-09-13T18:34:09.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.137+0000 7f620605d740 1 -- 172.21.15.114:0/777113895 wait complete. 2024-09-13T18:34:09.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.137+0000 7f620605d740 1 -- start start 2024-09-13T18:34:09.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.137+0000 7f620605d740 1 Processor -- start 2024-09-13T18:34:09.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.138+0000 7f620605d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f3ee52fa0 0x563f3ee4e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.138+0000 7f620605d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f3ee538c0 0x563f3ee45d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.138+0000 7f620605d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f3ee5e160 0x563f3ee464e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.138+0000 7f620605d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f3ee46be0 con 0x563f3ee52fa0 2024-09-13T18:34:09.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.138+0000 7f620605d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f3ee4a420 con 0x563f3ee5e160 2024-09-13T18:34:09.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.138+0000 7f620605d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f3ee4a5e0 con 0x563f3ee538c0 2024-09-13T18:34:09.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.138+0000 7f61fbfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f3ee538c0 0x563f3ee45d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.138+0000 7f61fbfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f3ee538c0 0x563f3ee45d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:54802/0 (socket says 172.21.15.114:54802) 2024-09-13T18:34:09.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.138+0000 7f61fbfff640 1 -- 172.21.15.114:0/3130298874 learned_addr learned my addr 172.21.15.114:0/3130298874 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:34:09.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.138+0000 7f62017fe640 1 --2- 172.21.15.114:0/3130298874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f3ee5e160 0x563f3ee464e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.138+0000 7f620605d740 10 client.0 _ll_register_callbacks cb 0x7f61f0035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:34:09.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.139+0000 7f6200ffd640 1 --2- 172.21.15.114:0/3130298874 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f3ee52fa0 0x563f3ee4e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.139+0000 7f62017fe640 1 -- 172.21.15.114:0/3130298874 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f3ee538c0 msgr2=0x563f3ee45d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:09.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.139+0000 7f62017fe640 1 --2- 172.21.15.114:0/3130298874 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f3ee538c0 0x563f3ee45d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.139+0000 7f62017fe640 1 -- 172.21.15.114:0/3130298874 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f3ee52fa0 msgr2=0x563f3ee4e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:09.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.139+0000 7f62017fe640 1 --2- 172.21.15.114:0/3130298874 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f3ee52fa0 0x563f3ee4e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.139+0000 7f62017fe640 1 -- 172.21.15.114:0/3130298874 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61f4015030 con 0x563f3ee5e160 2024-09-13T18:34:09.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.141+0000 7f62017fe640 1 --2- 172.21.15.114:0/3130298874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f3ee5e160 0x563f3ee464e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f61f4049000 tx=0x7f61f400e770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2c96f58c239094 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:09.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.142+0000 7f61dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:34:09.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.142+0000 7f61dbfff640 1 -- 172.21.15.114:0/3130298874 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f4044040 con 0x563f3ee5e160 2024-09-13T18:34:09.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.142+0000 7f61dbfff640 1 -- 172.21.15.114:0/3130298874 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f61f400eda0 con 0x563f3ee5e160 2024-09-13T18:34:09.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.144+0000 7f620605d740 1 -- 172.21.15.114:0/3130298874 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f61f0004030 con 0x563f3ee5e160 2024-09-13T18:34:09.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.149+0000 7f61dbfff640 1 -- 172.21.15.114:0/3130298874 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61f4051440 con 0x563f3ee5e160 2024-09-13T18:34:09.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.156+0000 7f620605d740 10 client.28112 fetch_fsmap learned FSMap version 39 2024-09-13T18:34:09.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.156+0000 7f620605d740 1 -- 172.21.15.114:0/3130298874 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x563f3ed73e80 con 0x563f3ee5e160 2024-09-13T18:34:09.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.161+0000 7f61dbfff640 1 -- 172.21.15.114:0/3130298874 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f61f0004030 con 0x563f3ee5e160 2024-09-13T18:34:09.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.162+0000 7f61dbfff640 1 -- 172.21.15.114:0/3130298874 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 39) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f61f4059070 con 0x563f3ee5e160 2024-09-13T18:34:09.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.162+0000 7f620605d740 10 client.28112 fetch_fsmap finished waiting for FSMap version 39 2024-09-13T18:34:09.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.162+0000 7f620605d740 10 client.28112 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-13T18:34:09.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.162+0000 7f620605d740 20 client.28112 populate_metadata read hostname 'smithi114' 2024-09-13T18:34:09.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.162+0000 7f620605d740 1 --2- 172.21.15.114:0/3130298874 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x563f3eede200 0x563f3eee0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.162+0000 7f620605d740 4 client.28112 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:34:09.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.162+0000 7f620605d740 1 -- 172.21.15.114:0/3130298874 --> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563f3eaa14e0 con 0x563f3eede200 2024-09-13T18:34:09.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.162+0000 7f6200ffd640 1 --2- 172.21.15.114:0/3130298874 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x563f3eede200 0x563f3eee0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.163+0000 7f6200ffd640 1 --2- 172.21.15.114:0/3130298874 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x563f3eede200 0x563f3eee0620 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1c3f2723fb491b08 server_cookie=a38309d1d1e9bff8 in_seq=0 out_seq=0 2024-09-13T18:34:09.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.165+0000 7f61dbfff640 10 client.28112 ms_handle_connect on v2:172.21.15.114:6840/1956923446 2024-09-13T18:34:09.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.165+0000 7f61dbfff640 1 -- 172.21.15.114:0/3130298874 <== mds.4 v2:172.21.15.114:6840/1956923446 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563f3eaa14e0 con 0x563f3eede200 2024-09-13T18:34:09.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.165+0000 7f61dbfff640 10 client.28112 handle_command_reply: tid=0 2024-09-13T18:34:09.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.166+0000 7f620605d740 1 client.28112 shutdown 2024-09-13T18:34:09.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.170+0000 7f620605d740 1 -- 172.21.15.114:0/3130298874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f3ee5e160 msgr2=0x563f3ee464e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:09.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.170+0000 7f620605d740 1 --2- 172.21.15.114:0/3130298874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f3ee5e160 0x563f3ee464e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f61f4049000 tx=0x7f61f400e770 comp rx=0 tx=0).stop 2024-09-13T18:34:09.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.175+0000 7f620605d740 1 -- 172.21.15.114:0/3130298874 shutdown_connections 2024-09-13T18:34:09.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.175+0000 7f620605d740 1 --2- 172.21.15.114:0/3130298874 >> [v2:172.21.15.114:6840/1956923446,v1:172.21.15.114:6841/1956923446] conn(0x563f3eede200 0x563f3eee0620 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.175+0000 7f620605d740 1 --2- 172.21.15.114:0/3130298874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f3ee5e160 0x563f3ee464e0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.175+0000 7f620605d740 1 --2- 172.21.15.114:0/3130298874 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f3ee538c0 0x563f3ee45d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.175+0000 7f620605d740 1 --2- 172.21.15.114:0/3130298874 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f3ee52fa0 0x563f3ee4e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.176+0000 7f61dbfff640 0 client.28112 ms_handle_reset on v2:172.21.15.114:6840/1956923446 2024-09-13T18:34:09.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.176+0000 7f620605d740 1 -- 172.21.15.114:0/3130298874 >> 172.21.15.114:0/3130298874 conn(0x563f3ee4f510 msgr2=0x563f3ee698b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:09.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.179+0000 7f620605d740 1 -- 172.21.15.114:0/3130298874 shutdown_connections 2024-09-13T18:34:09.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.181+0000 7f620605d740 1 -- 172.21.15.114:0/3130298874 wait complete. 2024-09-13T18:34:09.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.185+0000 7f620605d740 20 client.28112 trim_cache size 0 max 16384 2024-09-13T18:34:09.188 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:34:09.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.187+0000 7f6203261640 1 -- 172.21.15.114:0/1911619591 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f61ec0a1410 msgr2=0x7f61ec0a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:09.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.187+0000 7f6203261640 1 --2- 172.21.15.114:0/1911619591 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f61ec0a1410 0x7f61ec0a38d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f61f4010fd0 tx=0x7f61f4004a40 comp rx=0 tx=0).stop 2024-09-13T18:34:09.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.187+0000 7f6203261640 1 -- 172.21.15.114:0/1911619591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61fc06c160 msgr2=0x7f61fc06c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:09.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.187+0000 7f6203261640 1 --2- 172.21.15.114:0/1911619591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61fc06c160 0x7f61fc06c5c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f61e8009680 tx=0x7f61e8009b40 comp rx=0 tx=0).stop 2024-09-13T18:34:09.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.188+0000 7f6203261640 1 -- 172.21.15.114:0/1911619591 shutdown_connections 2024-09-13T18:34:09.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.188+0000 7f6203261640 1 --2- 172.21.15.114:0/1911619591 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f61ec0a1410 0x7f61ec0a38d0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.188+0000 7f6203261640 1 --2- 172.21.15.114:0/1911619591 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f61fc06cb00 0x7f61fc1aa440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.188+0000 7f6203261640 1 --2- 172.21.15.114:0/1911619591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61fc06c160 0x7f61fc06c5c0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.188+0000 7f6203261640 1 --2- 172.21.15.114:0/1911619591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61fc1040c0 0x7f61fc072160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.188+0000 7f6203261640 1 -- 172.21.15.114:0/1911619591 >> 172.21.15.114:0/1911619591 conn(0x7f61fc0fa8d0 msgr2=0x7f61fc103150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:09.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.188+0000 7f6203261640 1 -- 172.21.15.114:0/1911619591 shutdown_connections 2024-09-13T18:34:09.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.188+0000 7f6203261640 1 -- 172.21.15.114:0/1911619591 wait complete. 2024-09-13T18:34:09.208 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-09-13T18:34:09.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.414+0000 7f23246c9640 1 -- 172.21.15.114:0/2245275548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f231c0feed0 msgr2=0x7f231c0ff2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:09.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.414+0000 7f23246c9640 1 --2- 172.21.15.114:0/2245275548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f231c0feed0 0x7f231c0ff2b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f2318009900 tx=0x7f231802a420 comp rx=0 tx=0).stop 2024-09-13T18:34:09.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.421+0000 7f23246c9640 1 -- 172.21.15.114:0/2245275548 shutdown_connections 2024-09-13T18:34:09.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.421+0000 7f23246c9640 1 --2- 172.21.15.114:0/2245275548 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f231c104020 0x7f231c10bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.421+0000 7f23246c9640 1 --2- 172.21.15.114:0/2245275548 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f231c0ff880 0x7f231c1038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.421+0000 7f23246c9640 1 --2- 172.21.15.114:0/2245275548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f231c0feed0 0x7f231c0ff2b0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.421+0000 7f23246c9640 1 -- 172.21.15.114:0/2245275548 >> 172.21.15.114:0/2245275548 conn(0x7f231c0fa9a0 msgr2=0x7f231c0fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:09.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.426+0000 7f23246c9640 1 -- 172.21.15.114:0/2245275548 shutdown_connections 2024-09-13T18:34:09.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.430+0000 7f23246c9640 1 -- 172.21.15.114:0/2245275548 wait complete. 2024-09-13T18:34:09.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.431+0000 7f23246c9640 1 Processor -- start 2024-09-13T18:34:09.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.431+0000 7f23246c9640 1 -- start start 2024-09-13T18:34:09.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.432+0000 7f23246c9640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f231c0feed0 0x7f231c071a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.432+0000 7f23246c9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f231c104020 0x7f231c071f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.432+0000 7f23246c9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f231c068510 0x7f231c0689c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.432+0000 7f23246c9640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f231c069090 con 0x7f231c0feed0 2024-09-13T18:34:09.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.432+0000 7f23246c9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f231c069200 con 0x7f231c104020 2024-09-13T18:34:09.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.432+0000 7f23246c9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f231c0693c0 con 0x7f231c068510 2024-09-13T18:34:09.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.436+0000 7f2322c66640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f231c068510 0x7f231c0689c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.436+0000 7f2322c66640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f231c068510 0x7f231c0689c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:54820/0 (socket says 172.21.15.114:54820) 2024-09-13T18:34:09.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.436+0000 7f2322c66640 1 -- 172.21.15.114:0/3379994387 learned_addr learned my addr 172.21.15.114:0/3379994387 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:34:09.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.436+0000 7f2321c64640 1 --2- 172.21.15.114:0/3379994387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f231c104020 0x7f231c071f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.439+0000 7f2322465640 1 --2- 172.21.15.114:0/3379994387 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f231c0feed0 0x7f231c071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.440+0000 7f2322c66640 1 -- 172.21.15.114:0/3379994387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f231c104020 msgr2=0x7f231c071f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:09.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.440+0000 7f2322c66640 1 --2- 172.21.15.114:0/3379994387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f231c104020 0x7f231c071f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.440+0000 7f2322c66640 1 -- 172.21.15.114:0/3379994387 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f231c0feed0 msgr2=0x7f231c071a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:09.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.440+0000 7f2322c66640 1 --2- 172.21.15.114:0/3379994387 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f231c0feed0 0x7f231c071a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.440+0000 7f2322c66640 1 -- 172.21.15.114:0/3379994387 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2318009560 con 0x7f231c068510 2024-09-13T18:34:09.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.440+0000 7f2321c64640 1 --2- 172.21.15.114:0/3379994387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f231c104020 0x7f231c071f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:34:09.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.441+0000 7f2322c66640 1 --2- 172.21.15.114:0/3379994387 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f231c068510 0x7f231c0689c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f231400b970 tx=0x7f231400bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:09.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.441+0000 7f23137fe640 1 -- 172.21.15.114:0/3379994387 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231400caa0 con 0x7f231c068510 2024-09-13T18:34:09.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.442+0000 7f23137fe640 1 -- 172.21.15.114:0/3379994387 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f231400cc00 con 0x7f231c068510 2024-09-13T18:34:09.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.442+0000 7f23137fe640 1 -- 172.21.15.114:0/3379994387 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23140075d0 con 0x7f231c068510 2024-09-13T18:34:09.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.444+0000 7f23246c9640 1 -- 172.21.15.114:0/3379994387 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f231c06f740 con 0x7f231c068510 2024-09-13T18:34:09.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.444+0000 7f23246c9640 1 -- 172.21.15.114:0/3379994387 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f231c06fc00 con 0x7f231c068510 2024-09-13T18:34:09.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.449+0000 7f23137fe640 1 -- 172.21.15.114:0/3379994387 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f2314007730 con 0x7f231c068510 2024-09-13T18:34:09.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.452+0000 7f23137fe640 1 --2- 172.21.15.114:0/3379994387 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f23080a1410 0x7f23080a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.453+0000 7f23137fe640 1 -- 172.21.15.114:0/3379994387 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f23140cc880 con 0x7f231c068510 2024-09-13T18:34:09.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.453+0000 7f23273e6740 1 Processor -- start 2024-09-13T18:34:09.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.454+0000 7f23273e6740 1 -- start start 2024-09-13T18:34:09.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.454+0000 7f23273e6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564598174790 0x564598174b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.454+0000 7f23273e6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645981750b0 0x56459817f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.454+0000 7f23273e6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56459817f950 0x564598181d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.455+0000 7f2321c64640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645981750b0 0x56459817f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.455+0000 7f2322465640 1 --2- 172.21.15.114:0/3379994387 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f23080a1410 0x7f23080a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.455+0000 7f2321c64640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645981750b0 0x56459817f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:59214/0 (socket says 172.21.15.114:59214) 2024-09-13T18:34:09.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.455+0000 7f2321c64640 1 -- 172.21.15.114:0/1899219600 learned_addr learned my addr 172.21.15.114:0/1899219600 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:34:09.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.455+0000 7f23273e6740 1 -- 172.21.15.114:0/1899219600 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645980decc0 con 0x564598174790 2024-09-13T18:34:09.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.455+0000 7f2322c66640 1 --2- 172.21.15.114:0/1899219600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56459817f950 0x564598181d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.455+0000 7f23273e6740 1 -- 172.21.15.114:0/1899219600 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645980dc450 con 0x5645981750b0 2024-09-13T18:34:09.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.455+0000 7f23273e6740 1 -- 172.21.15.114:0/1899219600 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645980d7d40 con 0x56459817f950 2024-09-13T18:34:09.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.455+0000 7f2322465640 1 --2- 172.21.15.114:0/1899219600 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564598174790 0x564598174b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.456+0000 7f2321c64640 1 -- 172.21.15.114:0/1899219600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56459817f950 msgr2=0x564598181d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:09.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.456+0000 7f2321c64640 1 --2- 172.21.15.114:0/1899219600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56459817f950 0x564598181d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.456+0000 7f2321c64640 1 -- 172.21.15.114:0/1899219600 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564598174790 msgr2=0x564598174b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:09.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.456+0000 7f2321c64640 1 --2- 172.21.15.114:0/1899219600 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564598174790 0x564598174b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.456+0000 7f2321c64640 1 -- 172.21.15.114:0/1899219600 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56459809f620 con 0x5645981750b0 2024-09-13T18:34:09.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.456+0000 7f2322465640 1 --2- 172.21.15.114:0/3379994387 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f23080a1410 0x7f23080a38d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2318000c00 tx=0x7f2318003720 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:09.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.456+0000 7f2321c64640 1 --2- 172.21.15.114:0/1899219600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645981750b0 0x56459817f410 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f230c009de0 tx=0x7f230c00e5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47839d1e057314a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:09.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.457+0000 7f2310ff9640 1 -- 172.21.15.114:0/1899219600 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230c013470 con 0x5645981750b0 2024-09-13T18:34:09.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.458+0000 7f2310ff9640 1 -- 172.21.15.114:0/1899219600 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f230c013a60 con 0x5645981750b0 2024-09-13T18:34:09.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.458+0000 7f2310ff9640 1 -- 172.21.15.114:0/1899219600 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230c012a30 con 0x5645981750b0 2024-09-13T18:34:09.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.459+0000 7f23273e6740 1 -- 172.21.15.114:0/1899219600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645981750b0 msgr2=0x56459817f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:09.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.459+0000 7f23273e6740 1 --2- 172.21.15.114:0/1899219600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645981750b0 0x56459817f410 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f230c009de0 tx=0x7f230c00e5a0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.461+0000 7f23273e6740 1 -- 172.21.15.114:0/1899219600 shutdown_connections 2024-09-13T18:34:09.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.461+0000 7f23273e6740 1 --2- 172.21.15.114:0/1899219600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56459817f950 0x564598181d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.461+0000 7f23273e6740 1 --2- 172.21.15.114:0/1899219600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645981750b0 0x56459817f410 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.461+0000 7f23273e6740 1 --2- 172.21.15.114:0/1899219600 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564598174790 0x564598174b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.461+0000 7f23273e6740 1 -- 172.21.15.114:0/1899219600 >> 172.21.15.114:0/1899219600 conn(0x564598170d00 msgr2=0x5645981710e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:09.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.461+0000 7f23273e6740 1 -- 172.21.15.114:0/1899219600 shutdown_connections 2024-09-13T18:34:09.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.462+0000 7f23273e6740 1 -- 172.21.15.114:0/1899219600 wait complete. 2024-09-13T18:34:09.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.463+0000 7f23273e6740 1 -- start start 2024-09-13T18:34:09.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.463+0000 7f23273e6740 1 Processor -- start 2024-09-13T18:34:09.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.466+0000 7f23273e6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564598174790 0x56459817ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.466+0000 7f23273e6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645981750b0 0x56459817c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.467+0000 7f2322465640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564598174790 0x56459817ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.467+0000 7f2322465640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564598174790 0x56459817ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:54846/0 (socket says 172.21.15.114:54846) 2024-09-13T18:34:09.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.467+0000 7f2322465640 1 -- 172.21.15.114:0/571040563 learned_addr learned my addr 172.21.15.114:0/571040563 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:34:09.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.467+0000 7f2321c64640 1 --2- 172.21.15.114:0/571040563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645981750b0 0x56459817c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.467+0000 7f23273e6740 1 --2- 172.21.15.114:0/571040563 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56459817f950 0x56459817c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.467+0000 7f23273e6740 1 -- 172.21.15.114:0/571040563 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x564598096180 con 0x56459817f950 2024-09-13T18:34:09.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.467+0000 7f23273e6740 1 -- 172.21.15.114:0/571040563 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564597db0430 con 0x5645981750b0 2024-09-13T18:34:09.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.467+0000 7f23273e6740 1 -- 172.21.15.114:0/571040563 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56459805b5e0 con 0x564598174790 2024-09-13T18:34:09.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.467+0000 7f23273e6740 10 client.0 _ll_register_callbacks cb 0x5645980c7aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:34:09.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.468+0000 7f2322465640 1 -- 172.21.15.114:0/571040563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645981750b0 msgr2=0x56459817c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:09.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.468+0000 7f2322465640 1 --2- 172.21.15.114:0/571040563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645981750b0 0x56459817c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.468+0000 7f2322465640 1 -- 172.21.15.114:0/571040563 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56459817f950 msgr2=0x56459817c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:09.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.468+0000 7f2322465640 1 --2- 172.21.15.114:0/571040563 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56459817f950 0x56459817c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.468+0000 7f2322465640 1 -- 172.21.15.114:0/571040563 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f230c009a40 con 0x564598174790 2024-09-13T18:34:09.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.468+0000 7f2322465640 1 --2- 172.21.15.114:0/571040563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564598174790 0x56459817ef70 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2318005740 tx=0x7f231802a420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bdd7c58c4a99617 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:09.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.468+0000 7f23273e6740 1 -- 172.21.15.114:0/571040563 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564598101b80 con 0x564598174790 2024-09-13T18:34:09.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.470+0000 7f2305ffb640 10 client.28039 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:34:09.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.474+0000 7f2305ffb640 1 -- 172.21.15.114:0/571040563 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23180406e0 con 0x564598174790 2024-09-13T18:34:09.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.474+0000 7f2305ffb640 1 -- 172.21.15.114:0/571040563 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f2318040840 con 0x564598174790 2024-09-13T18:34:09.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.474+0000 7f2305ffb640 1 -- 172.21.15.114:0/571040563 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2318040a20 con 0x564598174790 2024-09-13T18:34:09.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.475+0000 7f23273e6740 10 client.28039 fetch_fsmap learned FSMap version 39 2024-09-13T18:34:09.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.475+0000 7f23273e6740 1 -- 172.21.15.114:0/571040563 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({fsmap=39}) v3 -- 0x5645980d63c0 con 0x564598174790 2024-09-13T18:34:09.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.479+0000 7f2305ffb640 1 -- 172.21.15.114:0/571040563 <== mon.1 v2:172.21.15.204:3300/0 4 ==== fsmap(e 39) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f2318051020 con 0x564598174790 2024-09-13T18:34:09.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.479+0000 7f23273e6740 10 client.28039 fetch_fsmap finished waiting for FSMap version 39 2024-09-13T18:34:09.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.479+0000 7f23273e6740 10 client.28039 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-13T18:34:09.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.479+0000 7f23273e6740 20 client.28039 populate_metadata read hostname 'smithi114' 2024-09-13T18:34:09.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.479+0000 7f23273e6740 1 --2- 172.21.15.114:0/571040563 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x56459818f290 0x564598191750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.480+0000 7f23273e6740 4 client.28039 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:34:09.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.480+0000 7f23273e6740 1 -- 172.21.15.114:0/571040563 --> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564598101b80 con 0x56459818f290 2024-09-13T18:34:09.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.482+0000 7f2321c64640 1 --2- 172.21.15.114:0/571040563 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x56459818f290 0x564598191750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.485+0000 7f2305ffb640 10 client.28039 ms_handle_connect on v2:172.21.15.134:6834/1150003275 2024-09-13T18:34:09.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.485+0000 7f2321c64640 1 --2- 172.21.15.114:0/571040563 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x56459818f290 0x564598191750 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b1089fc11640b392 server_cookie=8af2f0aafb636c80 in_seq=0 out_seq=0 2024-09-13T18:34:09.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.490+0000 7f2305ffb640 1 -- 172.21.15.114:0/571040563 <== mds.3 v2:172.21.15.134:6834/1150003275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x564598101b80 con 0x56459818f290 2024-09-13T18:34:09.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.490+0000 7f2305ffb640 10 client.28039 handle_command_reply: tid=0 2024-09-13T18:34:09.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.490+0000 7f23273e6740 1 client.28039 shutdown 2024-09-13T18:34:09.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.492+0000 7f23273e6740 1 -- 172.21.15.114:0/571040563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564598174790 msgr2=0x56459817ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:09.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.492+0000 7f23273e6740 1 --2- 172.21.15.114:0/571040563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564598174790 0x56459817ef70 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2318005740 tx=0x7f231802a420 comp rx=0 tx=0).stop 2024-09-13T18:34:09.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.494+0000 7f23273e6740 1 -- 172.21.15.114:0/571040563 shutdown_connections 2024-09-13T18:34:09.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.494+0000 7f23273e6740 1 --2- 172.21.15.114:0/571040563 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x56459818f290 0x564598191750 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.494+0000 7f23273e6740 1 --2- 172.21.15.114:0/571040563 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56459817f950 0x56459817c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.494+0000 7f23273e6740 1 --2- 172.21.15.114:0/571040563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645981750b0 0x56459817c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.494+0000 7f23273e6740 1 --2- 172.21.15.114:0/571040563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564598174790 0x56459817ef70 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.494+0000 7f23273e6740 1 -- 172.21.15.114:0/571040563 >> 172.21.15.114:0/571040563 conn(0x564598170d00 msgr2=0x564598176a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:09.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.494+0000 7f2305ffb640 0 client.28039 ms_handle_reset on v2:172.21.15.134:6834/1150003275 2024-09-13T18:34:09.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.498+0000 7f23273e6740 1 -- 172.21.15.114:0/571040563 shutdown_connections 2024-09-13T18:34:09.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.502+0000 7f23273e6740 1 -- 172.21.15.114:0/571040563 wait complete. 2024-09-13T18:34:09.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.503+0000 7f23273e6740 20 client.28039 trim_cache size 0 max 16384 2024-09-13T18:34:09.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.530+0000 7f23273e6740 1 Processor -- start 2024-09-13T18:34:09.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.530+0000 7f23273e6740 1 -- start start 2024-09-13T18:34:09.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.531+0000 7f23273e6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564598174790 0x5645981710e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.531+0000 7f23273e6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645981750b0 0x564598171620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.531+0000 7f2322465640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564598174790 0x5645981710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.531+0000 7f2322465640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564598174790 0x5645981710e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:54848/0 (socket says 172.21.15.114:54848) 2024-09-13T18:34:09.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.531+0000 7f23273e6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56459817f950 0x56459817c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.531+0000 7f2321c64640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645981750b0 0x564598171620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.531+0000 7f2321c64640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645981750b0 0x564598171620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:59230/0 (socket says 172.21.15.114:59230) 2024-09-13T18:34:09.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.532+0000 7f2322465640 1 -- 172.21.15.114:0/1796247342 learned_addr learned my addr 172.21.15.114:0/1796247342 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:34:09.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.532+0000 7f2322c66640 1 --2- 172.21.15.114:0/1796247342 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56459817f950 0x56459817c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.532+0000 7f23273e6740 1 -- 172.21.15.114:0/1796247342 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x564598175750 con 0x56459817f950 2024-09-13T18:34:09.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.532+0000 7f23273e6740 1 -- 172.21.15.114:0/1796247342 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564598191ae0 con 0x5645981750b0 2024-09-13T18:34:09.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.532+0000 7f23273e6740 1 -- 172.21.15.114:0/1796247342 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56459818b920 con 0x564598174790 2024-09-13T18:34:09.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.535+0000 7f2321c64640 1 -- 172.21.15.114:0/1796247342 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564598174790 msgr2=0x5645981710e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:09.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.535+0000 7f2321c64640 1 --2- 172.21.15.114:0/1796247342 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564598174790 0x5645981710e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.535+0000 7f2321c64640 1 -- 172.21.15.114:0/1796247342 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56459817f950 msgr2=0x56459817c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:09.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.535+0000 7f2321c64640 1 --2- 172.21.15.114:0/1796247342 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56459817f950 0x56459817c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.535+0000 7f2321c64640 1 -- 172.21.15.114:0/1796247342 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564598078770 con 0x5645981750b0 2024-09-13T18:34:09.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.535+0000 7f2321c64640 1 --2- 172.21.15.114:0/1796247342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645981750b0 0x564598171620 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f230c012410 tx=0x7f230c008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=582bf1d779768fd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:09.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.544+0000 7f23117fa640 1 -- 172.21.15.114:0/1796247342 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230c021040 con 0x5645981750b0 2024-09-13T18:34:09.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.544+0000 7f23117fa640 1 -- 172.21.15.114:0/1796247342 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f230c023040 con 0x5645981750b0 2024-09-13T18:34:09.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.544+0000 7f23117fa640 1 -- 172.21.15.114:0/1796247342 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230c005db0 con 0x5645981750b0 2024-09-13T18:34:09.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.545+0000 7f23273e6740 1 -- 172.21.15.114:0/1796247342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645981750b0 msgr2=0x564598171620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:09.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.545+0000 7f23273e6740 1 --2- 172.21.15.114:0/1796247342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645981750b0 0x564598171620 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f230c012410 tx=0x7f230c008040 comp rx=0 tx=0).stop 2024-09-13T18:34:09.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.547+0000 7f23273e6740 1 -- 172.21.15.114:0/1796247342 shutdown_connections 2024-09-13T18:34:09.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.547+0000 7f23273e6740 1 --2- 172.21.15.114:0/1796247342 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56459817f950 0x56459817c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.547+0000 7f23273e6740 1 --2- 172.21.15.114:0/1796247342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645981750b0 0x564598171620 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.547+0000 7f23273e6740 1 --2- 172.21.15.114:0/1796247342 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564598174790 0x5645981710e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.547+0000 7f23273e6740 1 -- 172.21.15.114:0/1796247342 >> 172.21.15.114:0/1796247342 conn(0x564598170d00 msgr2=0x5645981960f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:09.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.555+0000 7f23273e6740 1 -- 172.21.15.114:0/1796247342 shutdown_connections 2024-09-13T18:34:09.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.559+0000 7f23273e6740 1 -- 172.21.15.114:0/1796247342 wait complete. 2024-09-13T18:34:09.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.559+0000 7f23273e6740 1 -- start start 2024-09-13T18:34:09.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.559+0000 7f23273e6740 1 Processor -- start 2024-09-13T18:34:09.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.560+0000 7f23273e6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564598174790 0x564598178fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.560+0000 7f23273e6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645981750b0 0x564598179510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.560+0000 7f23273e6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56459817f950 0x564598179a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.560+0000 7f23273e6740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645981e5870 con 0x5645981750b0 2024-09-13T18:34:09.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.560+0000 7f23273e6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645981e59b0 con 0x56459817f950 2024-09-13T18:34:09.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.560+0000 7f2322465640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564598174790 0x564598178fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.560+0000 7f23273e6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645981e5af0 con 0x564598174790 2024-09-13T18:34:09.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.560+0000 7f2322465640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564598174790 0x564598178fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:54856/0 (socket says 172.21.15.114:54856) 2024-09-13T18:34:09.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.560+0000 7f2322465640 1 -- 172.21.15.114:0/950166462 learned_addr learned my addr 172.21.15.114:0/950166462 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:34:09.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.560+0000 7f23273e6740 10 client.0 _ll_register_callbacks cb 0x7f230c009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:34:09.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.561+0000 7f2322c66640 1 --2- 172.21.15.114:0/950166462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56459817f950 0x564598179a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.562+0000 7f2321c64640 1 --2- 172.21.15.114:0/950166462 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645981750b0 0x564598179510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.562+0000 7f2322465640 1 -- 172.21.15.114:0/950166462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56459817f950 msgr2=0x564598179a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:09.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.562+0000 7f2322465640 1 --2- 172.21.15.114:0/950166462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56459817f950 0x564598179a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.562+0000 7f2322465640 1 -- 172.21.15.114:0/950166462 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645981750b0 msgr2=0x564598179510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:09.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.562+0000 7f2322465640 1 --2- 172.21.15.114:0/950166462 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645981750b0 0x564598179510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.562+0000 7f2322465640 1 -- 172.21.15.114:0/950166462 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56459809f620 con 0x564598174790 2024-09-13T18:34:09.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.567+0000 7f2322465640 1 --2- 172.21.15.114:0/950166462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564598174790 0x564598178fd0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f2318005740 tx=0x7f2318004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5722f9be95ea0c8b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:09.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.569+0000 7f2305ffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:34:09.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.571+0000 7f2305ffb640 1 -- 172.21.15.114:0/950166462 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2318009400 con 0x564598174790 2024-09-13T18:34:09.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.571+0000 7f2305ffb640 1 -- 172.21.15.114:0/950166462 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f2318062070 con 0x564598174790 2024-09-13T18:34:09.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.573+0000 7f23273e6740 1 -- 172.21.15.114:0/950166462 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f230c013a60 con 0x564598174790 2024-09-13T18:34:09.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.579+0000 7f2305ffb640 1 -- 172.21.15.114:0/950166462 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23180039b0 con 0x564598174790 2024-09-13T18:34:09.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.581+0000 7f23273e6740 10 client.28045 fetch_fsmap learned FSMap version 39 2024-09-13T18:34:09.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.581+0000 7f23273e6740 1 -- 172.21.15.114:0/950166462 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x56459809de80 con 0x564598174790 2024-09-13T18:34:09.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.584+0000 7f2305ffb640 1 -- 172.21.15.114:0/950166462 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f230c013a60 con 0x564598174790 2024-09-13T18:34:09.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.585+0000 7f2305ffb640 1 -- 172.21.15.114:0/950166462 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 39) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f2318051040 con 0x564598174790 2024-09-13T18:34:09.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.586+0000 7f23273e6740 10 client.28045 fetch_fsmap finished waiting for FSMap version 39 2024-09-13T18:34:09.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.586+0000 7f23273e6740 10 client.28045 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-13T18:34:09.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.586+0000 7f23273e6740 20 client.28045 populate_metadata read hostname 'smithi114' 2024-09-13T18:34:09.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.586+0000 7f23273e6740 1 --2- 172.21.15.114:0/950166462 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x5645981e6d10 0x5645981e9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.586+0000 7f23273e6740 4 client.28045 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:34:09.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.586+0000 7f23273e6740 1 -- 172.21.15.114:0/950166462 --> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5645980ca1f0 con 0x5645981e6d10 2024-09-13T18:34:09.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.590+0000 7f2321c64640 1 --2- 172.21.15.114:0/950166462 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x5645981e6d10 0x5645981e9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.593+0000 7f2321c64640 1 --2- 172.21.15.114:0/950166462 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x5645981e6d10 0x5645981e9100 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=38d81ea406278503 server_cookie=275e596329042b4c in_seq=0 out_seq=0 2024-09-13T18:34:09.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.594+0000 7f2305ffb640 10 client.28045 ms_handle_connect on v2:172.21.15.134:6834/1150003275 2024-09-13T18:34:09.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.597+0000 7f2305ffb640 1 -- 172.21.15.114:0/950166462 <== mds.3 v2:172.21.15.134:6834/1150003275 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5645980ca1f0 con 0x5645981e6d10 2024-09-13T18:34:09.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.597+0000 7f2305ffb640 10 client.28045 handle_command_reply: tid=0 2024-09-13T18:34:09.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.600+0000 7f23273e6740 1 client.28045 shutdown 2024-09-13T18:34:09.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.602+0000 7f23273e6740 1 -- 172.21.15.114:0/950166462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564598174790 msgr2=0x564598178fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:09.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.602+0000 7f23273e6740 1 --2- 172.21.15.114:0/950166462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564598174790 0x564598178fd0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f2318005740 tx=0x7f2318004040 comp rx=0 tx=0).stop 2024-09-13T18:34:09.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.603+0000 7f23273e6740 1 -- 172.21.15.114:0/950166462 shutdown_connections 2024-09-13T18:34:09.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.603+0000 7f23273e6740 1 --2- 172.21.15.114:0/950166462 >> [v2:172.21.15.134:6834/1150003275,v1:172.21.15.134:6835/1150003275] conn(0x5645981e6d10 0x5645981e9100 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.603+0000 7f23273e6740 1 --2- 172.21.15.114:0/950166462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56459817f950 0x564598179a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.603+0000 7f23273e6740 1 --2- 172.21.15.114:0/950166462 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645981750b0 0x564598179510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.603+0000 7f23273e6740 1 --2- 172.21.15.114:0/950166462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564598174790 0x564598178fd0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.603+0000 7f2305ffb640 0 client.28045 ms_handle_reset on v2:172.21.15.134:6834/1150003275 2024-09-13T18:34:09.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.603+0000 7f23273e6740 1 -- 172.21.15.114:0/950166462 >> 172.21.15.114:0/950166462 conn(0x564598170d00 msgr2=0x5645981710e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:09.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.604+0000 7f23273e6740 1 -- 172.21.15.114:0/950166462 shutdown_connections 2024-09-13T18:34:09.611 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:34:09.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.605+0000 7f23273e6740 1 -- 172.21.15.114:0/950166462 wait complete. 2024-09-13T18:34:09.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.606+0000 7f23273e6740 20 client.28045 trim_cache size 0 max 16384 2024-09-13T18:34:09.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.613+0000 7f23246c9640 1 -- 172.21.15.114:0/3379994387 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f23080a1410 msgr2=0x7f23080a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:09.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.613+0000 7f23246c9640 1 --2- 172.21.15.114:0/3379994387 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f23080a1410 0x7f23080a38d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2318000c00 tx=0x7f2318003720 comp rx=0 tx=0).stop 2024-09-13T18:34:09.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.613+0000 7f23246c9640 1 -- 172.21.15.114:0/3379994387 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f231c068510 msgr2=0x7f231c0689c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:09.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.613+0000 7f23246c9640 1 --2- 172.21.15.114:0/3379994387 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f231c068510 0x7f231c0689c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f231400b970 tx=0x7f231400bd70 comp rx=0 tx=0).stop 2024-09-13T18:34:09.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.618+0000 7f23246c9640 1 -- 172.21.15.114:0/3379994387 shutdown_connections 2024-09-13T18:34:09.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.618+0000 7f23246c9640 1 --2- 172.21.15.114:0/3379994387 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f23080a1410 0x7f23080a38d0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.618+0000 7f23246c9640 1 --2- 172.21.15.114:0/3379994387 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f231c068510 0x7f231c0689c0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.618+0000 7f23246c9640 1 --2- 172.21.15.114:0/3379994387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f231c104020 0x7f231c071f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.618+0000 7f23246c9640 1 --2- 172.21.15.114:0/3379994387 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f231c0feed0 0x7f231c071a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.618+0000 7f23246c9640 1 -- 172.21.15.114:0/3379994387 >> 172.21.15.114:0/3379994387 conn(0x7f231c0fa9a0 msgr2=0x7f231c0fbe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:09.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.621+0000 7f23246c9640 1 -- 172.21.15.114:0/3379994387 shutdown_connections 2024-09-13T18:34:09.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.622+0000 7f23246c9640 1 -- 172.21.15.114:0/3379994387 wait complete. 2024-09-13T18:34:09.646 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-09-13T18:34:09.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.840+0000 7ff96d7c7640 1 -- 172.21.15.114:0/2833084121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9680fef90 msgr2=0x7ff968102fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:09.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.840+0000 7ff96d7c7640 1 --2- 172.21.15.114:0/2833084121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9680fef90 0x7ff968102fd0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff958010910 tx=0x7ff95802e2f0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.844+0000 7ff96d7c7640 1 -- 172.21.15.114:0/2833084121 shutdown_connections 2024-09-13T18:34:09.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.844+0000 7ff96d7c7640 1 --2- 172.21.15.114:0/2833084121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff968103730 0x7ff96810b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.844+0000 7ff96d7c7640 1 --2- 172.21.15.114:0/2833084121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9680fef90 0x7ff968102fd0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.844+0000 7ff96d7c7640 1 --2- 172.21.15.114:0/2833084121 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9680fe5e0 0x7ff9680fe9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.844+0000 7ff96d7c7640 1 -- 172.21.15.114:0/2833084121 >> 172.21.15.114:0/2833084121 conn(0x7ff9680753f0 msgr2=0x7ff9680fc740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:09.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.846+0000 7ff96d7c7640 1 -- 172.21.15.114:0/2833084121 shutdown_connections 2024-09-13T18:34:09.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.847+0000 7ff96d7c7640 1 -- 172.21.15.114:0/2833084121 wait complete. 2024-09-13T18:34:09.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.848+0000 7ff96d7c7640 1 Processor -- start 2024-09-13T18:34:09.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.849+0000 7ff96d7c7640 1 -- start start 2024-09-13T18:34:09.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.849+0000 7ff96d7c7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9680fe5e0 0x7ff9681a2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.849+0000 7ff96d7c7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff968103730 0x7ff9681a2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.849+0000 7ff96d7c7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff96819c630 0x7ff96819cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.849+0000 7ff96d7c7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96819d0b0 con 0x7ff96819c630 2024-09-13T18:34:09.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.849+0000 7ff96d7c7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96819d220 con 0x7ff9680fe5e0 2024-09-13T18:34:09.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.850+0000 7ff96d7c7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96819d3e0 con 0x7ff968103730 2024-09-13T18:34:09.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.850+0000 7ff9677fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff96819c630 0x7ff96819cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.850+0000 7ff9677fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff96819c630 0x7ff96819cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55076/0 (socket says 172.21.15.114:55076) 2024-09-13T18:34:09.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.850+0000 7ff9677fe640 1 -- 172.21.15.114:0/2166800105 learned_addr learned my addr 172.21.15.114:0/2166800105 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:34:09.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.851+0000 7ff9677fe640 1 -- 172.21.15.114:0/2166800105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff968103730 msgr2=0x7ff9681a2a90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:34:09.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.851+0000 7ff9677fe640 1 --2- 172.21.15.114:0/2166800105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff968103730 0x7ff9681a2a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.851+0000 7ff9677fe640 1 -- 172.21.15.114:0/2166800105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9680fe5e0 msgr2=0x7ff9681a2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:09.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.851+0000 7ff9677fe640 1 --2- 172.21.15.114:0/2166800105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9680fe5e0 0x7ff9681a2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.851+0000 7ff9677fe640 1 -- 172.21.15.114:0/2166800105 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff958010510 con 0x7ff96819c630 2024-09-13T18:34:09.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.852+0000 7ff9677fe640 1 --2- 172.21.15.114:0/2166800105 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff96819c630 0x7ff96819cab0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7ff960004410 tx=0x7ff9600048d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:09.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.852+0000 7ff95ffff640 1 -- 172.21.15.114:0/2166800105 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff960018070 con 0x7ff96819c630 2024-09-13T18:34:09.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.852+0000 7ff96d7c7640 1 -- 172.21.15.114:0/2166800105 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9681a93c0 con 0x7ff96819c630 2024-09-13T18:34:09.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.852+0000 7ff96d7c7640 1 -- 172.21.15.114:0/2166800105 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9681a98a0 con 0x7ff96819c630 2024-09-13T18:34:09.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.853+0000 7ff95ffff640 1 -- 172.21.15.114:0/2166800105 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff96000bd10 con 0x7ff96819c630 2024-09-13T18:34:09.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.853+0000 7ff95ffff640 1 -- 172.21.15.114:0/2166800105 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff960010040 con 0x7ff96819c630 2024-09-13T18:34:09.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.859+0000 7ff95ffff640 1 -- 172.21.15.114:0/2166800105 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7ff960005020 con 0x7ff96819c630 2024-09-13T18:34:09.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.860+0000 7ff95ffff640 1 --2- 172.21.15.114:0/2166800105 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff9540a1410 0x7ff9540a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.860+0000 7ff95ffff640 1 -- 172.21.15.114:0/2166800105 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9541+0+0 (secure 0 0 0) 0x7ff9600cce20 con 0x7ff96819c630 2024-09-13T18:34:09.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.861+0000 7ff9705c1740 1 Processor -- start 2024-09-13T18:34:09.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.861+0000 7ff9705c1740 1 -- start start 2024-09-13T18:34:09.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.862+0000 7ff9705c1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e80d68d790 0x55e80d68db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.862+0000 7ff9705c1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e80d68e0b0 0x55e80d698410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.862+0000 7ff9705c1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e80d698950 0x55e80d69ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.862+0000 7ff9667fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e80d68e0b0 0x55e80d698410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.862+0000 7ff9667fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e80d68e0b0 0x55e80d698410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:54892/0 (socket says 172.21.15.114:54892) 2024-09-13T18:34:09.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.863+0000 7ff9667fc640 1 -- 172.21.15.114:0/2935007385 learned_addr learned my addr 172.21.15.114:0/2935007385 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:34:09.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.863+0000 7ff9705c1740 1 -- 172.21.15.114:0/2935007385 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e80d5f7cc0 con 0x55e80d698950 2024-09-13T18:34:09.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.863+0000 7ff9705c1740 1 -- 172.21.15.114:0/2935007385 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e80d5f5450 con 0x55e80d68d790 2024-09-13T18:34:09.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.863+0000 7ff9705c1740 1 -- 172.21.15.114:0/2935007385 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e80d5f0d40 con 0x55e80d68e0b0 2024-09-13T18:34:09.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.863+0000 7ff9677fe640 1 --2- 172.21.15.114:0/2935007385 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e80d698950 0x55e80d69ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.864+0000 7ff9677fe640 1 -- 172.21.15.114:0/2935007385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e80d68e0b0 msgr2=0x55e80d698410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:09.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.864+0000 7ff9677fe640 1 --2- 172.21.15.114:0/2935007385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e80d68e0b0 0x55e80d698410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.864+0000 7ff9677fe640 1 -- 172.21.15.114:0/2935007385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e80d68d790 msgr2=0x55e80d68db70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:34:09.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.864+0000 7ff9677fe640 1 --2- 172.21.15.114:0/2935007385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e80d68d790 0x55e80d68db70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.864+0000 7ff9677fe640 1 -- 172.21.15.114:0/2935007385 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e80d5b8620 con 0x55e80d698950 2024-09-13T18:34:09.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.864+0000 7ff9677fe640 1 --2- 172.21.15.114:0/2935007385 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e80d698950 0x55e80d69ad40 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7ff9600ca110 tx=0x7ff9600998c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b45c47b55b616129 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:09.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.865+0000 7ff95d7fa640 1 -- 172.21.15.114:0/2935007385 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff960099a30 con 0x55e80d698950 2024-09-13T18:34:09.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.865+0000 7ff95d7fa640 1 -- 172.21.15.114:0/2935007385 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff96009d860 con 0x55e80d698950 2024-09-13T18:34:09.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.865+0000 7ff95d7fa640 1 -- 172.21.15.114:0/2935007385 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9600ca390 con 0x55e80d698950 2024-09-13T18:34:09.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.865+0000 7ff9667fc640 1 --2- 172.21.15.114:0/2935007385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e80d68e0b0 0x55e80d698410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:34:09.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.866+0000 7ff9705c1740 1 -- 172.21.15.114:0/2935007385 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e80d698950 msgr2=0x55e80d69ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:09.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.866+0000 7ff9705c1740 1 --2- 172.21.15.114:0/2935007385 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e80d698950 0x55e80d69ad40 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7ff9600ca110 tx=0x7ff9600998c0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.867+0000 7ff9705c1740 1 -- 172.21.15.114:0/2935007385 shutdown_connections 2024-09-13T18:34:09.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.867+0000 7ff9705c1740 1 --2- 172.21.15.114:0/2935007385 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e80d698950 0x55e80d69ad40 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.867+0000 7ff9705c1740 1 --2- 172.21.15.114:0/2935007385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e80d68e0b0 0x55e80d698410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.867+0000 7ff9705c1740 1 --2- 172.21.15.114:0/2935007385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e80d68d790 0x55e80d68db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.867+0000 7ff9705c1740 1 -- 172.21.15.114:0/2935007385 >> 172.21.15.114:0/2935007385 conn(0x55e80d689d00 msgr2=0x55e80d68a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:09.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.872+0000 7ff966ffd640 1 --2- 172.21.15.114:0/2166800105 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff9540a1410 0x7ff9540a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.874+0000 7ff9705c1740 1 -- 172.21.15.114:0/2935007385 shutdown_connections 2024-09-13T18:34:09.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.876+0000 7ff9705c1740 1 -- 172.21.15.114:0/2935007385 wait complete. 2024-09-13T18:34:09.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.876+0000 7ff9705c1740 1 -- start start 2024-09-13T18:34:09.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.876+0000 7ff9705c1740 1 Processor -- start 2024-09-13T18:34:09.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.876+0000 7ff9705c1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e80d68d790 0x55e80d698030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.876+0000 7ff9705c1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e80d68e0b0 0x55e80d696b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.877+0000 7ff9705c1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e80d698950 0x55e80d697040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.877+0000 7ff9705c1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e80d5af180 con 0x55e80d68e0b0 2024-09-13T18:34:09.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.877+0000 7ff9705c1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e80d2c9430 con 0x55e80d698950 2024-09-13T18:34:09.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.877+0000 7ff9705c1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e80d5745e0 con 0x55e80d68d790 2024-09-13T18:34:09.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.877+0000 7ff9677fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e80d698950 0x55e80d697040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.877+0000 7ff9677fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e80d698950 0x55e80d697040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:59274/0 (socket says 172.21.15.114:59274) 2024-09-13T18:34:09.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.877+0000 7ff9677fe640 1 -- 172.21.15.114:0/2741297868 learned_addr learned my addr 172.21.15.114:0/2741297868 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:34:09.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.877+0000 7ff9705c1740 10 client.0 _ll_register_callbacks cb 0x55e80d5e0aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:34:09.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.879+0000 7ff9667fc640 1 --2- 172.21.15.114:0/2741297868 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e80d68e0b0 0x55e80d696b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.881+0000 7ff9677fe640 1 -- 172.21.15.114:0/2741297868 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e80d68d790 msgr2=0x55e80d698030 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:34:09.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.881+0000 7ff9677fe640 1 --2- 172.21.15.114:0/2741297868 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e80d68d790 0x55e80d698030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.881+0000 7ff9677fe640 1 -- 172.21.15.114:0/2741297868 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e80d68e0b0 msgr2=0x55e80d696b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:09.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.881+0000 7ff9677fe640 1 --2- 172.21.15.114:0/2741297868 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e80d68e0b0 0x55e80d696b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.881+0000 7ff9677fe640 1 -- 172.21.15.114:0/2741297868 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9681a98a0 con 0x55e80d698950 2024-09-13T18:34:09.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.882+0000 7ff966ffd640 1 --2- 172.21.15.114:0/2166800105 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff9540a1410 0x7ff9540a38d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff95000a8b0 tx=0x7ff950005c10 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:09.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.883+0000 7ff9677fe640 1 --2- 172.21.15.114:0/2741297868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e80d698950 0x55e80d697040 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff9600043e0 tx=0x7ff9600a1ff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd02d8305776c2b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:09.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.883+0000 7ff94e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:34:09.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.884+0000 7ff94e7fc640 1 -- 172.21.15.114:0/2741297868 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9600b2420 con 0x55e80d698950 2024-09-13T18:34:09.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.884+0000 7ff9705c1740 1 -- 172.21.15.114:0/2741297868 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e80d61ab80 con 0x55e80d698950 2024-09-13T18:34:09.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.884+0000 7ff94e7fc640 1 -- 172.21.15.114:0/2741297868 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9541+0+0 (secure 0 0 0) 0x7ff9600a1a20 con 0x55e80d698950 2024-09-13T18:34:09.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.884+0000 7ff94e7fc640 1 -- 172.21.15.114:0/2741297868 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff960099ab0 con 0x55e80d698950 2024-09-13T18:34:09.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.889+0000 7ff9705c1740 10 client.28136 fetch_fsmap learned FSMap version 39 2024-09-13T18:34:09.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.889+0000 7ff9705c1740 1 -- 172.21.15.114:0/2741297868 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55e80d5ef3c0 con 0x55e80d698950 2024-09-13T18:34:09.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.891+0000 7ff94e7fc640 1 -- 172.21.15.114:0/2741297868 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff9600ca590 con 0x55e80d698950 2024-09-13T18:34:09.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.892+0000 7ff94e7fc640 1 -- 172.21.15.114:0/2741297868 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 39) v1 ==== 7506+0+0 (secure 0 0 0) 0x7ff960095820 con 0x55e80d698950 2024-09-13T18:34:09.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.892+0000 7ff9705c1740 10 client.28136 fetch_fsmap finished waiting for FSMap version 39 2024-09-13T18:34:09.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.892+0000 7ff9705c1740 10 client.28136 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-13T18:34:09.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.892+0000 7ff9705c1740 20 client.28136 populate_metadata read hostname 'smithi114' 2024-09-13T18:34:09.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.892+0000 7ff9705c1740 1 --2- 172.21.15.114:0/2741297868 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55e80d6a7f80 0x55e80d6aa3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.892+0000 7ff9705c1740 4 client.28136 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:34:09.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.892+0000 7ff9705c1740 1 -- 172.21.15.114:0/2741297868 --> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e80d61ab80 con 0x55e80d6a7f80 2024-09-13T18:34:09.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.897+0000 7ff966ffd640 1 --2- 172.21.15.114:0/2741297868 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55e80d6a7f80 0x55e80d6aa3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.900+0000 7ff966ffd640 1 --2- 172.21.15.114:0/2741297868 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55e80d6a7f80 0x55e80d6aa3a0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=298231201ec64120 server_cookie=abcc5269b73e8c68 in_seq=0 out_seq=0 2024-09-13T18:34:09.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.901+0000 7ff94e7fc640 10 client.28136 ms_handle_connect on v2:172.21.15.204:6838/344883912 2024-09-13T18:34:09.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.905+0000 7ff94e7fc640 1 -- 172.21.15.114:0/2741297868 <== mds.2 v2:172.21.15.204:6838/344883912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55e80d61ab80 con 0x55e80d6a7f80 2024-09-13T18:34:09.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.905+0000 7ff94e7fc640 10 client.28136 handle_command_reply: tid=0 2024-09-13T18:34:09.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.905+0000 7ff9705c1740 1 client.28136 shutdown 2024-09-13T18:34:09.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.909+0000 7ff9705c1740 1 -- 172.21.15.114:0/2741297868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e80d698950 msgr2=0x55e80d697040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:09.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.909+0000 7ff9705c1740 1 --2- 172.21.15.114:0/2741297868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e80d698950 0x55e80d697040 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff9600043e0 tx=0x7ff9600a1ff0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.918+0000 7ff9705c1740 1 -- 172.21.15.114:0/2741297868 shutdown_connections 2024-09-13T18:34:09.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.918+0000 7ff9705c1740 1 --2- 172.21.15.114:0/2741297868 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55e80d6a7f80 0x55e80d6aa3a0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.918+0000 7ff9705c1740 1 --2- 172.21.15.114:0/2741297868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e80d698950 0x55e80d697040 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.918+0000 7ff9705c1740 1 --2- 172.21.15.114:0/2741297868 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e80d68e0b0 0x55e80d696b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.918+0000 7ff9705c1740 1 --2- 172.21.15.114:0/2741297868 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e80d68d790 0x55e80d698030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.918+0000 7ff9705c1740 1 -- 172.21.15.114:0/2741297868 >> 172.21.15.114:0/2741297868 conn(0x55e80d689d00 msgr2=0x55e80d688eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:09.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.919+0000 7ff94e7fc640 0 client.28136 ms_handle_reset on v2:172.21.15.204:6838/344883912 2024-09-13T18:34:09.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.924+0000 7ff9705c1740 1 -- 172.21.15.114:0/2741297868 shutdown_connections 2024-09-13T18:34:09.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.927+0000 7ff9705c1740 1 -- 172.21.15.114:0/2741297868 wait complete. 2024-09-13T18:34:09.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.927+0000 7ff9705c1740 20 client.28136 trim_cache size 0 max 16384 2024-09-13T18:34:09.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:09 smithi204 ceph-mon[34777]: pgmap v364: 129 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 124 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 56 KiB/s wr, 31 op/s 2024-09-13T18:34:09.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:09 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/2279794526' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:34:09.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.951+0000 7ff9705c1740 1 Processor -- start 2024-09-13T18:34:09.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.951+0000 7ff9705c1740 1 -- start start 2024-09-13T18:34:09.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.952+0000 7ff9705c1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e80d68d790 0x55e80d6aed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.952+0000 7ff9705c1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e80d68e0b0 0x55e80d6a4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.952+0000 7ff9705c1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e80d698950 0x55e80d6a4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.952+0000 7ff966ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e80d68d790 0x55e80d6aed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.952+0000 7ff966ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e80d68d790 0x55e80d6aed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:54918/0 (socket says 172.21.15.114:54918) 2024-09-13T18:34:09.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.952+0000 7ff966ffd640 1 -- 172.21.15.114:0/3110883110 learned_addr learned my addr 172.21.15.114:0/3110883110 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:34:09.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.952+0000 7ff9705c1740 1 -- 172.21.15.114:0/3110883110 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e80d69b400 con 0x55e80d68e0b0 2024-09-13T18:34:09.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.952+0000 7ff9705c1740 1 -- 172.21.15.114:0/3110883110 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e80d68d510 con 0x55e80d698950 2024-09-13T18:34:09.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.952+0000 7ff9705c1740 1 -- 172.21.15.114:0/3110883110 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e80d693800 con 0x55e80d68d790 2024-09-13T18:34:09.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.953+0000 7ff966ffd640 1 -- 172.21.15.114:0/3110883110 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e80d698950 msgr2=0x55e80d6a4670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:34:09.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.953+0000 7ff966ffd640 1 --2- 172.21.15.114:0/3110883110 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e80d698950 0x55e80d6a4670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.953+0000 7ff966ffd640 1 -- 172.21.15.114:0/3110883110 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e80d68e0b0 msgr2=0x55e80d6a4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:09.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.953+0000 7ff966ffd640 1 --2- 172.21.15.114:0/3110883110 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e80d68e0b0 0x55e80d6a4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.953+0000 7ff966ffd640 1 -- 172.21.15.114:0/3110883110 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e80d591770 con 0x55e80d68d790 2024-09-13T18:34:09.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.953+0000 7ff966ffd640 1 --2- 172.21.15.114:0/3110883110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e80d68d790 0x55e80d6aed00 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff95000ccf0 tx=0x7ff95000f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bdd84ca28fdc956 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:09.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.957+0000 7ff95dffb640 1 -- 172.21.15.114:0/3110883110 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95000cd20 con 0x55e80d68d790 2024-09-13T18:34:09.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.957+0000 7ff95dffb640 1 -- 172.21.15.114:0/3110883110 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff950021e90 con 0x55e80d68d790 2024-09-13T18:34:09.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.957+0000 7ff95dffb640 1 -- 172.21.15.114:0/3110883110 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95000f910 con 0x55e80d68d790 2024-09-13T18:34:09.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.958+0000 7ff9705c1740 1 -- 172.21.15.114:0/3110883110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e80d68d790 msgr2=0x55e80d6aed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:09.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.958+0000 7ff9705c1740 1 --2- 172.21.15.114:0/3110883110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e80d68d790 0x55e80d6aed00 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff95000ccf0 tx=0x7ff95000f000 comp rx=0 tx=0).stop 2024-09-13T18:34:09.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.961+0000 7ff9705c1740 1 -- 172.21.15.114:0/3110883110 shutdown_connections 2024-09-13T18:34:09.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.961+0000 7ff9705c1740 1 --2- 172.21.15.114:0/3110883110 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e80d698950 0x55e80d6a4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.961+0000 7ff9705c1740 1 --2- 172.21.15.114:0/3110883110 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e80d68e0b0 0x55e80d6a4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.961+0000 7ff9705c1740 1 --2- 172.21.15.114:0/3110883110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e80d68d790 0x55e80d6aed00 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.961+0000 7ff9705c1740 1 -- 172.21.15.114:0/3110883110 >> 172.21.15.114:0/3110883110 conn(0x55e80d689d00 msgr2=0x55e80d6a9f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:09.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:09 smithi114 ceph-mon[30897]: pgmap v364: 129 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 124 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 56 KiB/s wr, 31 op/s 2024-09-13T18:34:09.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:09 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2279794526' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:34:09.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.968+0000 7ff9705c1740 1 -- 172.21.15.114:0/3110883110 shutdown_connections 2024-09-13T18:34:09.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.968+0000 7ff9705c1740 1 -- 172.21.15.114:0/3110883110 wait complete. 2024-09-13T18:34:09.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.968+0000 7ff9705c1740 1 -- start start 2024-09-13T18:34:09.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.968+0000 7ff9705c1740 1 Processor -- start 2024-09-13T18:34:09.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.969+0000 7ff9705c1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e80d68d790 0x55e80d69faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.969+0000 7ff9705c1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e80d68e0b0 0x55e80d69bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.969+0000 7ff9705c1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e80d698950 0x55e80d69c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.969+0000 7ff9705c1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e80d69cc30 con 0x55e80d68d790 2024-09-13T18:34:09.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.969+0000 7ff9705c1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e80d69cda0 con 0x55e80d698950 2024-09-13T18:34:09.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.969+0000 7ff9705c1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e80d70f490 con 0x55e80d68e0b0 2024-09-13T18:34:09.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.969+0000 7ff9705c1740 10 client.0 _ll_register_callbacks cb 0x7ff960005380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:34:09.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.969+0000 7ff966ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e80d68d790 0x55e80d69faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.969+0000 7ff966ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e80d68d790 0x55e80d69faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55100/0 (socket says 172.21.15.114:55100) 2024-09-13T18:34:09.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.969+0000 7ff966ffd640 1 -- 172.21.15.114:0/3926836257 learned_addr learned my addr 172.21.15.114:0/3926836257 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:34:09.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.969+0000 7ff9677fe640 1 --2- 172.21.15.114:0/3926836257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e80d698950 0x55e80d69c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.969+0000 7ff9667fc640 1 --2- 172.21.15.114:0/3926836257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e80d68e0b0 0x55e80d69bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.970+0000 7ff9677fe640 1 -- 172.21.15.114:0/3926836257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e80d68e0b0 msgr2=0x55e80d69bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:09.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.970+0000 7ff9677fe640 1 --2- 172.21.15.114:0/3926836257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e80d68e0b0 0x55e80d69bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.970+0000 7ff9677fe640 1 -- 172.21.15.114:0/3926836257 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e80d68d790 msgr2=0x55e80d69faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:09.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.970+0000 7ff9677fe640 1 --2- 172.21.15.114:0/3926836257 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e80d68d790 0x55e80d69faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:09.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.970+0000 7ff9677fe640 1 -- 172.21.15.114:0/3926836257 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff95000c950 con 0x55e80d698950 2024-09-13T18:34:09.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.970+0000 7ff9677fe640 1 --2- 172.21.15.114:0/3926836257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e80d698950 0x55e80d69c530 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff960099730 tx=0x7ff9600963c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71fbedf8dd9af224 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:09.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.971+0000 7ff94e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:34:09.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.971+0000 7ff94e7fc640 1 -- 172.21.15.114:0/3926836257 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9600b2420 con 0x55e80d698950 2024-09-13T18:34:09.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.971+0000 7ff9705c1740 1 -- 172.21.15.114:0/3926836257 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff96009d860 con 0x55e80d698950 2024-09-13T18:34:09.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.971+0000 7ff94e7fc640 1 -- 172.21.15.114:0/3926836257 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9541+0+0 (secure 0 0 0) 0x7ff960098a60 con 0x55e80d698950 2024-09-13T18:34:09.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.975+0000 7ff94e7fc640 1 -- 172.21.15.114:0/3926836257 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff960095a20 con 0x55e80d698950 2024-09-13T18:34:09.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.982+0000 7ff9705c1740 10 client.28139 fetch_fsmap learned FSMap version 39 2024-09-13T18:34:09.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.982+0000 7ff9705c1740 1 -- 172.21.15.114:0/3926836257 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55e80d5b6e80 con 0x55e80d698950 2024-09-13T18:34:09.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.983+0000 7ff94e7fc640 1 -- 172.21.15.114:0/3926836257 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff96009d860 con 0x55e80d698950 2024-09-13T18:34:09.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.983+0000 7ff94e7fc640 1 -- 172.21.15.114:0/3926836257 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 39) v1 ==== 7506+0+0 (secure 0 0 0) 0x7ff9600ca390 con 0x55e80d698950 2024-09-13T18:34:09.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.983+0000 7ff9705c1740 10 client.28139 fetch_fsmap finished waiting for FSMap version 39 2024-09-13T18:34:09.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.983+0000 7ff9705c1740 10 client.28139 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-13T18:34:09.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.983+0000 7ff9705c1740 20 client.28139 populate_metadata read hostname 'smithi114' 2024-09-13T18:34:09.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.983+0000 7ff9705c1740 1 --2- 172.21.15.114:0/3926836257 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55e80d710750 0x55e80d712b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:09.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.983+0000 7ff9705c1740 4 client.28139 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:34:09.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.983+0000 7ff9705c1740 1 -- 172.21.15.114:0/3926836257 --> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e80d5e31f0 con 0x55e80d710750 2024-09-13T18:34:09.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.984+0000 7ff966ffd640 1 --2- 172.21.15.114:0/3926836257 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55e80d710750 0x55e80d712b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:09.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.985+0000 7ff966ffd640 1 --2- 172.21.15.114:0/3926836257 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55e80d710750 0x55e80d712b40 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8c1369496c7f66f9 server_cookie=a5a9b4bb3a8ca62c in_seq=0 out_seq=0 2024-09-13T18:34:09.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.987+0000 7ff94e7fc640 10 client.28139 ms_handle_connect on v2:172.21.15.204:6838/344883912 2024-09-13T18:34:09.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.989+0000 7ff94e7fc640 1 -- 172.21.15.114:0/3926836257 <== mds.2 v2:172.21.15.204:6838/344883912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e80d5e31f0 con 0x55e80d710750 2024-09-13T18:34:09.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.989+0000 7ff94e7fc640 10 client.28139 handle_command_reply: tid=0 2024-09-13T18:34:09.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.989+0000 7ff9705c1740 1 client.28139 shutdown 2024-09-13T18:34:10.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.998+0000 7ff9705c1740 1 -- 172.21.15.114:0/3926836257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e80d698950 msgr2=0x55e80d69c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:10.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.998+0000 7ff9705c1740 1 --2- 172.21.15.114:0/3926836257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e80d698950 0x55e80d69c530 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff960099730 tx=0x7ff9600963c0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.999+0000 7ff9705c1740 1 -- 172.21.15.114:0/3926836257 shutdown_connections 2024-09-13T18:34:10.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.999+0000 7ff9705c1740 1 --2- 172.21.15.114:0/3926836257 >> [v2:172.21.15.204:6838/344883912,v1:172.21.15.204:6839/344883912] conn(0x55e80d710750 0x55e80d712b40 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.999+0000 7ff9705c1740 1 --2- 172.21.15.114:0/3926836257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e80d698950 0x55e80d69c530 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:09.999+0000 7ff9705c1740 1 --2- 172.21.15.114:0/3926836257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e80d68e0b0 0x55e80d69bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.000+0000 7ff9705c1740 1 --2- 172.21.15.114:0/3926836257 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e80d68d790 0x55e80d69faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.000+0000 7ff9705c1740 1 -- 172.21.15.114:0/3926836257 >> 172.21.15.114:0/3926836257 conn(0x55e80d689d00 msgr2=0x55e80d6aed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:10.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.001+0000 7ff94e7fc640 0 client.28139 ms_handle_reset on v2:172.21.15.204:6838/344883912 2024-09-13T18:34:10.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.002+0000 7ff9705c1740 1 -- 172.21.15.114:0/3926836257 shutdown_connections 2024-09-13T18:34:10.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.003+0000 7ff9705c1740 1 -- 172.21.15.114:0/3926836257 wait complete. 2024-09-13T18:34:10.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.010+0000 7ff9705c1740 20 client.28139 trim_cache size 0 max 16384 2024-09-13T18:34:10.012 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:34:10.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.011+0000 7ff96d7c7640 1 -- 172.21.15.114:0/2166800105 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff9540a1410 msgr2=0x7ff9540a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:10.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.011+0000 7ff96d7c7640 1 --2- 172.21.15.114:0/2166800105 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff9540a1410 0x7ff9540a38d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff95000a8b0 tx=0x7ff950005c10 comp rx=0 tx=0).stop 2024-09-13T18:34:10.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.011+0000 7ff96d7c7640 1 -- 172.21.15.114:0/2166800105 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff96819c630 msgr2=0x7ff96819cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:10.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.011+0000 7ff96d7c7640 1 --2- 172.21.15.114:0/2166800105 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff96819c630 0x7ff96819cab0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7ff960004410 tx=0x7ff9600048d0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.015+0000 7ff96d7c7640 1 -- 172.21.15.114:0/2166800105 shutdown_connections 2024-09-13T18:34:10.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.015+0000 7ff96d7c7640 1 --2- 172.21.15.114:0/2166800105 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff9540a1410 0x7ff9540a38d0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.015+0000 7ff96d7c7640 1 --2- 172.21.15.114:0/2166800105 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff96819c630 0x7ff96819cab0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.015+0000 7ff96d7c7640 1 --2- 172.21.15.114:0/2166800105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff968103730 0x7ff9681a2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.015+0000 7ff96d7c7640 1 --2- 172.21.15.114:0/2166800105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9680fe5e0 0x7ff9681a2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.015+0000 7ff96d7c7640 1 -- 172.21.15.114:0/2166800105 >> 172.21.15.114:0/2166800105 conn(0x7ff9680753f0 msgr2=0x7ff968101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:10.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.016+0000 7ff96d7c7640 1 -- 172.21.15.114:0/2166800105 shutdown_connections 2024-09-13T18:34:10.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.016+0000 7ff96d7c7640 1 -- 172.21.15.114:0/2166800105 wait complete. 2024-09-13T18:34:10.040 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-09-13T18:34:10.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:09 smithi134 ceph-mon[34283]: pgmap v364: 129 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 124 active+clean; 19 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 56 KiB/s wr, 31 op/s 2024-09-13T18:34:10.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:09 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/2279794526' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:34:10.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.228+0000 7f9d5a59c640 1 -- 172.21.15.114:0/2497779937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d540fec80 msgr2=0x7f9d54102960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:10.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.228+0000 7f9d5a59c640 1 --2- 172.21.15.114:0/2497779937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d540fec80 0x7f9d54102960 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f9d44009ca0 tx=0x7f9d4402a8f0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.230+0000 7f9d5a59c640 1 -- 172.21.15.114:0/2497779937 shutdown_connections 2024-09-13T18:34:10.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.230+0000 7f9d5a59c640 1 --2- 172.21.15.114:0/2497779937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d540fec80 0x7f9d54102960 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.230+0000 7f9d5a59c640 1 --2- 172.21.15.114:0/2497779937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d540fe220 0x7f9d540fe660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.230+0000 7f9d5a59c640 1 --2- 172.21.15.114:0/2497779937 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9d54104e60 0x7f9d54105240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.230+0000 7f9d5a59c640 1 -- 172.21.15.114:0/2497779937 >> 172.21.15.114:0/2497779937 conn(0x7f9d540753f0 msgr2=0x7f9d540fc740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:10.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.234+0000 7f9d5a59c640 1 -- 172.21.15.114:0/2497779937 shutdown_connections 2024-09-13T18:34:10.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.235+0000 7f9d5a59c640 1 -- 172.21.15.114:0/2497779937 wait complete. 2024-09-13T18:34:10.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.236+0000 7f9d5a59c640 1 Processor -- start 2024-09-13T18:34:10.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.236+0000 7f9d5a59c640 1 -- start start 2024-09-13T18:34:10.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.237+0000 7f9d5a59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9d540fe220 0x7f9d54113640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:10.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.237+0000 7f9d5a59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d540fec80 0x7f9d54113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:10.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.237+0000 7f9d5a59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d54104e60 0x7f9d5410d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:10.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.237+0000 7f9d5a59c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5410df60 con 0x7f9d540fe220 2024-09-13T18:34:10.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.237+0000 7f9d5a59c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5410e0d0 con 0x7f9d540fec80 2024-09-13T18:34:10.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.237+0000 7f9d5a59c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5410e290 con 0x7f9d54104e60 2024-09-13T18:34:10.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.237+0000 7f9d53fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9d540fe220 0x7f9d54113640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:10.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.237+0000 7f9d53fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9d540fe220 0x7f9d54113640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55114/0 (socket says 172.21.15.114:55114) 2024-09-13T18:34:10.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.237+0000 7f9d53fff640 1 -- 172.21.15.114:0/1626753778 learned_addr learned my addr 172.21.15.114:0/1626753778 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:34:10.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.238+0000 7f9d53fff640 1 -- 172.21.15.114:0/1626753778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d54104e60 msgr2=0x7f9d5410d7d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:34:10.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.238+0000 7f9d53fff640 1 --2- 172.21.15.114:0/1626753778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d54104e60 0x7f9d5410d7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.238+0000 7f9d53fff640 1 -- 172.21.15.114:0/1626753778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d540fec80 msgr2=0x7f9d54113b80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:34:10.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.238+0000 7f9d53fff640 1 --2- 172.21.15.114:0/1626753778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d540fec80 0x7f9d54113b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.238+0000 7f9d53fff640 1 -- 172.21.15.114:0/1626753778 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d44009900 con 0x7f9d540fe220 2024-09-13T18:34:10.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.238+0000 7f9d53fff640 1 --2- 172.21.15.114:0/1626753778 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9d540fe220 0x7f9d54113640 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c00b450 tx=0x7f9d4c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:10.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.246+0000 7f9d517fa640 1 -- 172.21.15.114:0/1626753778 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c011890 con 0x7f9d540fe220 2024-09-13T18:34:10.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.246+0000 7f9d5a59c640 1 -- 172.21.15.114:0/1626753778 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d541b1c70 con 0x7f9d540fe220 2024-09-13T18:34:10.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.246+0000 7f9d5a59c640 1 -- 172.21.15.114:0/1626753778 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d541b2090 con 0x7f9d540fe220 2024-09-13T18:34:10.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.247+0000 7f9d517fa640 1 -- 172.21.15.114:0/1626753778 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9d4c011e80 con 0x7f9d540fe220 2024-09-13T18:34:10.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.247+0000 7f9d517fa640 1 -- 172.21.15.114:0/1626753778 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c00f340 con 0x7f9d540fe220 2024-09-13T18:34:10.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.252+0000 7f9d5d2c2740 1 Processor -- start 2024-09-13T18:34:10.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.252+0000 7f9d5d2c2740 1 -- start start 2024-09-13T18:34:10.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.252+0000 7f9d5d2c2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d25b4f4790 0x55d25b4f4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:10.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.254+0000 7f9d5d2c2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d25b4f50b0 0x55d25b4ff410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:10.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.254+0000 7f9d58b39640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d25b4f4790 0x55d25b4f4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:10.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.256+0000 7f9d58b39640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d25b4f4790 0x55d25b4f4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:54954/0 (socket says 172.21.15.114:54954) 2024-09-13T18:34:10.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.257+0000 7f9d5d2c2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d25b4ff950 0x55d25b501d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:10.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.257+0000 7f9d5d2c2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d25b45ecc0 con 0x55d25b4ff950 2024-09-13T18:34:10.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.257+0000 7f9d5d2c2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d25b45c450 con 0x55d25b4f50b0 2024-09-13T18:34:10.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.257+0000 7f9d5d2c2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d25b457d40 con 0x55d25b4f4790 2024-09-13T18:34:10.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.260+0000 7f9d517fa640 1 -- 172.21.15.114:0/1626753778 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f9d4c0103e0 con 0x7f9d540fe220 2024-09-13T18:34:10.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.261+0000 7f9d58b39640 1 -- 172.21.15.114:0/4208336503 learned_addr learned my addr 172.21.15.114:0/4208336503 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:34:10.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.261+0000 7f9d517fa640 1 --2- 172.21.15.114:0/1626753778 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f9d400a1410 0x7f9d400a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:10.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.261+0000 7f9d517fa640 1 -- 172.21.15.114:0/1626753778 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f9d4c0ccda0 con 0x7f9d540fe220 2024-09-13T18:34:10.264 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.262+0000 7f9d537fe640 1 --2- 172.21.15.114:0/4208336503 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d25b4ff950 0x55d25b501d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:10.264 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.262+0000 7f9d53fff640 1 --2- 172.21.15.114:0/4208336503 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d25b4f50b0 0x55d25b4ff410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:10.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.262+0000 7f9d537fe640 1 -- 172.21.15.114:0/4208336503 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d25b4f4790 msgr2=0x55d25b4f4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:10.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.266+0000 7f9d537fe640 1 --2- 172.21.15.114:0/4208336503 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d25b4f4790 0x55d25b4f4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.266+0000 7f9d537fe640 1 -- 172.21.15.114:0/4208336503 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d25b4f50b0 msgr2=0x55d25b4ff410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:10.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.266+0000 7f9d537fe640 1 --2- 172.21.15.114:0/4208336503 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d25b4f50b0 0x55d25b4ff410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.266+0000 7f9d537fe640 1 -- 172.21.15.114:0/4208336503 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d25b41f620 con 0x55d25b4ff950 2024-09-13T18:34:10.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.268+0000 7f9d537fe640 1 --2- 172.21.15.114:0/4208336503 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d25b4ff950 0x55d25b501d40 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f9d540fd970 tx=0x7f9d3c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=536925bf6ab74403 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:10.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.268+0000 7f9d53fff640 1 --2- 172.21.15.114:0/4208336503 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d25b4f50b0 0x55d25b4ff410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:34:10.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.268+0000 7f9d4a7fc640 1 -- 172.21.15.114:0/4208336503 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c00b890 con 0x55d25b4ff950 2024-09-13T18:34:10.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.268+0000 7f9d4a7fc640 1 -- 172.21.15.114:0/4208336503 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9d3c00be80 con 0x55d25b4ff950 2024-09-13T18:34:10.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.268+0000 7f9d4a7fc640 1 -- 172.21.15.114:0/4208336503 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c010380 con 0x55d25b4ff950 2024-09-13T18:34:10.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.268+0000 7f9d58b39640 1 --2- 172.21.15.114:0/4208336503 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d25b4f4790 0x55d25b4f4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:34:10.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.269+0000 7f9d58b39640 1 --2- 172.21.15.114:0/1626753778 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f9d400a1410 0x7f9d400a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:10.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.270+0000 7f9d58b39640 1 --2- 172.21.15.114:0/1626753778 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f9d400a1410 0x7f9d400a38d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9d44006010 tx=0x7f9d4402a8f0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:10.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.270+0000 7f9d5d2c2740 1 -- 172.21.15.114:0/4208336503 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d25b4ff950 msgr2=0x55d25b501d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:10.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.270+0000 7f9d5d2c2740 1 --2- 172.21.15.114:0/4208336503 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d25b4ff950 0x55d25b501d40 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f9d540fd970 tx=0x7f9d3c009b20 comp rx=0 tx=0).stop 2024-09-13T18:34:10.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.270+0000 7f9d5d2c2740 1 -- 172.21.15.114:0/4208336503 shutdown_connections 2024-09-13T18:34:10.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.270+0000 7f9d5d2c2740 1 --2- 172.21.15.114:0/4208336503 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d25b4ff950 0x55d25b501d40 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.270+0000 7f9d5d2c2740 1 --2- 172.21.15.114:0/4208336503 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d25b4f50b0 0x55d25b4ff410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.270+0000 7f9d5d2c2740 1 --2- 172.21.15.114:0/4208336503 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d25b4f4790 0x55d25b4f4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.270+0000 7f9d5d2c2740 1 -- 172.21.15.114:0/4208336503 >> 172.21.15.114:0/4208336503 conn(0x55d25b4f0d00 msgr2=0x55d25b4f10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:10.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.275+0000 7f9d5d2c2740 1 -- 172.21.15.114:0/4208336503 shutdown_connections 2024-09-13T18:34:10.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.277+0000 7f9d5d2c2740 1 -- 172.21.15.114:0/4208336503 wait complete. 2024-09-13T18:34:10.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.277+0000 7f9d5d2c2740 1 -- start start 2024-09-13T18:34:10.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.277+0000 7f9d5d2c2740 1 Processor -- start 2024-09-13T18:34:10.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.278+0000 7f9d5d2c2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d25b4f4790 0x55d25b4fef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:10.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.278+0000 7f9d5d2c2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d25b4f50b0 0x55d25b4fc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:10.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.278+0000 7f9d5d2c2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d25b4ff950 0x55d25b4fc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:10.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.278+0000 7f9d5d2c2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d25b416180 con 0x55d25b4f4790 2024-09-13T18:34:10.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.278+0000 7f9d5d2c2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d25b130430 con 0x55d25b4ff950 2024-09-13T18:34:10.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.278+0000 7f9d5d2c2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d25b3db5e0 con 0x55d25b4f50b0 2024-09-13T18:34:10.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.278+0000 7f9d5d2c2740 10 client.0 _ll_register_callbacks cb 0x55d25b447aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:34:10.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.281+0000 7f9d58b39640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d25b4ff950 0x55d25b4fc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:10.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.282+0000 7f9d58b39640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d25b4ff950 0x55d25b4fc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:59328/0 (socket says 172.21.15.114:59328) 2024-09-13T18:34:10.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.282+0000 7f9d58b39640 1 -- 172.21.15.114:0/2597578698 learned_addr learned my addr 172.21.15.114:0/2597578698 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:34:10.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.282+0000 7f9d537fe640 1 --2- 172.21.15.114:0/2597578698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d25b4f50b0 0x55d25b4fc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:10.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.282+0000 7f9d58b39640 1 -- 172.21.15.114:0/2597578698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d25b4f50b0 msgr2=0x55d25b4fc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:10.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.282+0000 7f9d58b39640 1 --2- 172.21.15.114:0/2597578698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d25b4f50b0 0x55d25b4fc2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.282+0000 7f9d58b39640 1 -- 172.21.15.114:0/2597578698 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d25b4f4790 msgr2=0x55d25b4fef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:10.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.282+0000 7f9d58b39640 1 --2- 172.21.15.114:0/2597578698 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d25b4f4790 0x55d25b4fef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.282+0000 7f9d58b39640 1 -- 172.21.15.114:0/2597578698 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d3c0092f0 con 0x55d25b4ff950 2024-09-13T18:34:10.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.282+0000 7f9d58b39640 1 --2- 172.21.15.114:0/2597578698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d25b4ff950 0x55d25b4fc7e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f9d44009ca0 tx=0x7f9d440095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=649bdcc9d70a9e30 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:10.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.283+0000 7f9d337fe640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-13T18:34:10.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.283+0000 7f9d337fe640 1 -- 172.21.15.114:0/2597578698 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d44004040 con 0x55d25b4ff950 2024-09-13T18:34:10.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.283+0000 7f9d337fe640 1 -- 172.21.15.114:0/2597578698 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f9d440041a0 con 0x55d25b4ff950 2024-09-13T18:34:10.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.284+0000 7f9d5d2c2740 1 -- 172.21.15.114:0/2597578698 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d25b44a1f0 con 0x55d25b4ff950 2024-09-13T18:34:10.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.285+0000 7f9d337fe640 1 -- 172.21.15.114:0/2597578698 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d44004360 con 0x55d25b4ff950 2024-09-13T18:34:10.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.285+0000 7f9d5d2c2740 10 client.28145 fetch_fsmap learned FSMap version 39 2024-09-13T18:34:10.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.285+0000 7f9d5d2c2740 1 -- 172.21.15.114:0/2597578698 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55d25b4563c0 con 0x55d25b4ff950 2024-09-13T18:34:10.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.286+0000 7f9d337fe640 1 -- 172.21.15.114:0/2597578698 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9d44059030 con 0x55d25b4ff950 2024-09-13T18:34:10.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.286+0000 7f9d337fe640 1 -- 172.21.15.114:0/2597578698 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 39) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f9d44044040 con 0x55d25b4ff950 2024-09-13T18:34:10.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.287+0000 7f9d5d2c2740 10 client.28145 fetch_fsmap finished waiting for FSMap version 39 2024-09-13T18:34:10.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.287+0000 7f9d5d2c2740 10 client.28145 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-13T18:34:10.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.287+0000 7f9d5d2c2740 20 client.28145 populate_metadata read hostname 'smithi114' 2024-09-13T18:34:10.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.287+0000 7f9d5d2c2740 1 --2- 172.21.15.114:0/2597578698 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55d25b50f370 0x55d25b5117b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:10.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.287+0000 7f9d5d2c2740 4 client.28145 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:34:10.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.287+0000 7f9d5d2c2740 1 -- 172.21.15.114:0/2597578698 --> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d25b14b4e0 con 0x55d25b50f370 2024-09-13T18:34:10.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.290+0000 7f9d53fff640 1 --2- 172.21.15.114:0/2597578698 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55d25b50f370 0x55d25b5117b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:10.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.302+0000 7f9d53fff640 1 --2- 172.21.15.114:0/2597578698 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55d25b50f370 0x55d25b5117b0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ba4692827561ed7c server_cookie=6cdd891f55367985 in_seq=0 out_seq=0 2024-09-13T18:34:10.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.302+0000 7f9d337fe640 10 client.28145 ms_handle_connect on v2:172.21.15.134:6836/891692210 2024-09-13T18:34:10.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.303+0000 7f9d337fe640 1 -- 172.21.15.114:0/2597578698 <== mds.1 v2:172.21.15.134:6836/891692210 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55d25b14b4e0 con 0x55d25b50f370 2024-09-13T18:34:10.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.303+0000 7f9d337fe640 10 client.28145 handle_command_reply: tid=0 2024-09-13T18:34:10.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.303+0000 7f9d5d2c2740 1 client.28145 shutdown 2024-09-13T18:34:10.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.314+0000 7f9d5d2c2740 1 -- 172.21.15.114:0/2597578698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d25b4ff950 msgr2=0x55d25b4fc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:10.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.314+0000 7f9d5d2c2740 1 --2- 172.21.15.114:0/2597578698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d25b4ff950 0x55d25b4fc7e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f9d44009ca0 tx=0x7f9d440095c0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.316+0000 7f9d5d2c2740 1 -- 172.21.15.114:0/2597578698 shutdown_connections 2024-09-13T18:34:10.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.316+0000 7f9d5d2c2740 1 --2- 172.21.15.114:0/2597578698 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55d25b50f370 0x55d25b5117b0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.316+0000 7f9d5d2c2740 1 --2- 172.21.15.114:0/2597578698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d25b4ff950 0x55d25b4fc7e0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.316+0000 7f9d5d2c2740 1 --2- 172.21.15.114:0/2597578698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d25b4f50b0 0x55d25b4fc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.316+0000 7f9d5d2c2740 1 --2- 172.21.15.114:0/2597578698 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d25b4f4790 0x55d25b4fef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.316+0000 7f9d5d2c2740 1 -- 172.21.15.114:0/2597578698 >> 172.21.15.114:0/2597578698 conn(0x55d25b4f0d00 msgr2=0x55d25b4efeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:10.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.316+0000 7f9d337fe640 0 client.28145 ms_handle_reset on v2:172.21.15.134:6836/891692210 2024-09-13T18:34:10.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.316+0000 7f9d5d2c2740 1 -- 172.21.15.114:0/2597578698 shutdown_connections 2024-09-13T18:34:10.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.318+0000 7f9d5d2c2740 1 -- 172.21.15.114:0/2597578698 wait complete. 2024-09-13T18:34:10.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.319+0000 7f9d5d2c2740 20 client.28145 trim_cache size 0 max 16384 2024-09-13T18:34:10.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.339+0000 7f9d5d2c2740 1 Processor -- start 2024-09-13T18:34:10.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.340+0000 7f9d5d2c2740 1 -- start start 2024-09-13T18:34:10.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.340+0000 7f9d5d2c2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d25b4f4790 0x55d25b4fe9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:10.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.340+0000 7f9d5d2c2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d25b4f50b0 0x55d25b4feef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:10.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.340+0000 7f9d5d2c2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d25b4ff950 0x55d25b4fab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:10.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.340+0000 7f9d5d2c2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d25b4ff4f0 con 0x55d25b4f50b0 2024-09-13T18:34:10.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.340+0000 7f9d5d2c2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d25b4f1e90 con 0x55d25b4f4790 2024-09-13T18:34:10.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.340+0000 7f9d5d2c2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d25b502400 con 0x55d25b4ff950 2024-09-13T18:34:10.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.340+0000 7f9d537fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d25b4f50b0 0x55d25b4feef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:10.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.340+0000 7f9d537fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d25b4f50b0 0x55d25b4feef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55136/0 (socket says 172.21.15.114:55136) 2024-09-13T18:34:10.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.340+0000 7f9d537fe640 1 -- 172.21.15.114:0/353881159 learned_addr learned my addr 172.21.15.114:0/353881159 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:34:10.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.340+0000 7f9d53fff640 1 --2- 172.21.15.114:0/353881159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d25b4f4790 0x55d25b4fe9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:10.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.341+0000 7f9d58b39640 1 --2- 172.21.15.114:0/353881159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d25b4ff950 0x55d25b4fab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:10.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.344+0000 7f9d53fff640 1 -- 172.21.15.114:0/353881159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d25b4ff950 msgr2=0x55d25b4fab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:10.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.344+0000 7f9d53fff640 1 --2- 172.21.15.114:0/353881159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d25b4ff950 0x55d25b4fab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.344+0000 7f9d53fff640 1 -- 172.21.15.114:0/353881159 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d25b4f50b0 msgr2=0x55d25b4feef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:10.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.344+0000 7f9d53fff640 1 --2- 172.21.15.114:0/353881159 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d25b4f50b0 0x55d25b4feef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.344+0000 7f9d53fff640 1 -- 172.21.15.114:0/353881159 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d25b3f8770 con 0x55d25b4f4790 2024-09-13T18:34:10.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.346+0000 7f9d53fff640 1 --2- 172.21.15.114:0/353881159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d25b4f4790 0x55d25b4fe9b0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c0ca1a0 tx=0x7f9d4c010850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cf1572ebd625b0e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:10.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.347+0000 7f9d4affd640 1 -- 172.21.15.114:0/353881159 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c0a1fe0 con 0x55d25b4f4790 2024-09-13T18:34:10.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.347+0000 7f9d4affd640 1 -- 172.21.15.114:0/353881159 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9d4c097f00 con 0x55d25b4f4790 2024-09-13T18:34:10.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.347+0000 7f9d4affd640 1 -- 172.21.15.114:0/353881159 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c0a1b50 con 0x55d25b4f4790 2024-09-13T18:34:10.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.348+0000 7f9d5d2c2740 1 -- 172.21.15.114:0/353881159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d25b4f4790 msgr2=0x55d25b4fe9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:10.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.348+0000 7f9d5d2c2740 1 --2- 172.21.15.114:0/353881159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d25b4f4790 0x55d25b4fe9b0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c0ca1a0 tx=0x7f9d4c010850 comp rx=0 tx=0).stop 2024-09-13T18:34:10.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.353+0000 7f9d5d2c2740 1 -- 172.21.15.114:0/353881159 shutdown_connections 2024-09-13T18:34:10.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.353+0000 7f9d5d2c2740 1 --2- 172.21.15.114:0/353881159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d25b4ff950 0x55d25b4fab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.353+0000 7f9d5d2c2740 1 --2- 172.21.15.114:0/353881159 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d25b4f50b0 0x55d25b4feef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.353+0000 7f9d5d2c2740 1 --2- 172.21.15.114:0/353881159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d25b4f4790 0x55d25b4fe9b0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.353+0000 7f9d5d2c2740 1 -- 172.21.15.114:0/353881159 >> 172.21.15.114:0/353881159 conn(0x55d25b4f0d00 msgr2=0x55d25b4f10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:10.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.357+0000 7f9d5d2c2740 1 -- 172.21.15.114:0/353881159 shutdown_connections 2024-09-13T18:34:10.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.357+0000 7f9d5d2c2740 1 -- 172.21.15.114:0/353881159 wait complete. 2024-09-13T18:34:10.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.358+0000 7f9d5d2c2740 1 -- start start 2024-09-13T18:34:10.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.358+0000 7f9d5d2c2740 1 Processor -- start 2024-09-13T18:34:10.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.359+0000 7f9d5d2c2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d25b4f4790 0x55d25b5068a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:10.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.359+0000 7f9d5d2c2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d25b4f50b0 0x55d25b4f8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:10.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.359+0000 7f9d53fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d25b4f4790 0x55d25b5068a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:10.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.359+0000 7f9d53fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d25b4f4790 0x55d25b5068a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:54986/0 (socket says 172.21.15.114:54986) 2024-09-13T18:34:10.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.359+0000 7f9d53fff640 1 -- 172.21.15.114:0/3455072625 learned_addr learned my addr 172.21.15.114:0/3455072625 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:34:10.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.359+0000 7f9d537fe640 1 --2- 172.21.15.114:0/3455072625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d25b4f50b0 0x55d25b4f8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:10.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.360+0000 7f9d5d2c2740 1 --2- 172.21.15.114:0/3455072625 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d25b4ff950 0x55d25b4f9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:10.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.360+0000 7f9d5d2c2740 1 -- 172.21.15.114:0/3455072625 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d25b4f9570 con 0x55d25b4ff950 2024-09-13T18:34:10.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.360+0000 7f9d5d2c2740 1 -- 172.21.15.114:0/3455072625 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d25b4f96e0 con 0x55d25b4f50b0 2024-09-13T18:34:10.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.360+0000 7f9d5d2c2740 1 -- 172.21.15.114:0/3455072625 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d25b4f98a0 con 0x55d25b4f4790 2024-09-13T18:34:10.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.360+0000 7f9d5d2c2740 10 client.0 _ll_register_callbacks cb 0x7f9d3c006e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:34:10.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.360+0000 7f9d53fff640 1 -- 172.21.15.114:0/3455072625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d25b4f50b0 msgr2=0x55d25b4f8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:10.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.360+0000 7f9d53fff640 1 --2- 172.21.15.114:0/3455072625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d25b4f50b0 0x55d25b4f8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.360+0000 7f9d53fff640 1 -- 172.21.15.114:0/3455072625 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d25b4ff950 msgr2=0x55d25b4f9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:10.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.360+0000 7f9d53fff640 1 --2- 172.21.15.114:0/3455072625 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d25b4ff950 0x55d25b4f9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.360+0000 7f9d53fff640 1 -- 172.21.15.114:0/3455072625 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d541b2090 con 0x55d25b4f4790 2024-09-13T18:34:10.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.361+0000 7f9d53fff640 1 --2- 172.21.15.114:0/3455072625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d25b4f4790 0x55d25b5068a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c0965d0 tx=0x7f9d4c0964a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=feed9ef149095df0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:10.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.361+0000 7f9d337fe640 10 client.28063 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:34:10.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.361+0000 7f9d5d2c2740 1 -- 172.21.15.114:0/3455072625 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d25b481b80 con 0x55d25b4f4790 2024-09-13T18:34:10.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.362+0000 7f9d337fe640 1 -- 172.21.15.114:0/3455072625 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c0a1d50 con 0x55d25b4f4790 2024-09-13T18:34:10.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.362+0000 7f9d337fe640 1 -- 172.21.15.114:0/3455072625 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f9d4c0a1fe0 con 0x55d25b4f4790 2024-09-13T18:34:10.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.362+0000 7f9d337fe640 1 -- 172.21.15.114:0/3455072625 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d4c0ca520 con 0x55d25b4f4790 2024-09-13T18:34:10.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.366+0000 7f9d5d2c2740 10 client.28063 fetch_fsmap learned FSMap version 39 2024-09-13T18:34:10.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.366+0000 7f9d5d2c2740 1 -- 172.21.15.114:0/3455072625 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({fsmap=39}) v3 -- 0x55d25b41de80 con 0x55d25b4f4790 2024-09-13T18:34:10.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.367+0000 7f9d337fe640 1 -- 172.21.15.114:0/3455072625 <== mon.1 v2:172.21.15.204:3300/0 4 ==== fsmap(e 39) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f9d4c0b2260 con 0x55d25b4f4790 2024-09-13T18:34:10.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.368+0000 7f9d5d2c2740 10 client.28063 fetch_fsmap finished waiting for FSMap version 39 2024-09-13T18:34:10.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.368+0000 7f9d5d2c2740 10 client.28063 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-13T18:34:10.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.368+0000 7f9d5d2c2740 20 client.28063 populate_metadata read hostname 'smithi114' 2024-09-13T18:34:10.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.368+0000 7f9d5d2c2740 1 --2- 172.21.15.114:0/3455072625 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55d25b5667b0 0x55d25b568ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:10.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.368+0000 7f9d5d2c2740 4 client.28063 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:34:10.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.368+0000 7f9d5d2c2740 1 -- 172.21.15.114:0/3455072625 --> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d25b481b80 con 0x55d25b5667b0 2024-09-13T18:34:10.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.372+0000 7f9d537fe640 1 --2- 172.21.15.114:0/3455072625 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55d25b5667b0 0x55d25b568ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:10.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.376+0000 7f9d537fe640 1 --2- 172.21.15.114:0/3455072625 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55d25b5667b0 0x55d25b568ba0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dcf748d3e9c2fc6d server_cookie=a708eafb58d51614 in_seq=0 out_seq=0 2024-09-13T18:34:10.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.377+0000 7f9d337fe640 10 client.28063 ms_handle_connect on v2:172.21.15.134:6836/891692210 2024-09-13T18:34:10.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.380+0000 7f9d337fe640 1 -- 172.21.15.114:0/3455072625 <== mds.1 v2:172.21.15.134:6836/891692210 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d25b481b80 con 0x55d25b5667b0 2024-09-13T18:34:10.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.380+0000 7f9d337fe640 10 client.28063 handle_command_reply: tid=0 2024-09-13T18:34:10.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.381+0000 7f9d5d2c2740 1 client.28063 shutdown 2024-09-13T18:34:10.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.383+0000 7f9d5d2c2740 1 -- 172.21.15.114:0/3455072625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d25b4f4790 msgr2=0x55d25b5068a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:10.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.383+0000 7f9d5d2c2740 1 --2- 172.21.15.114:0/3455072625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d25b4f4790 0x55d25b5068a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c0965d0 tx=0x7f9d4c0964a0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.384+0000 7f9d5d2c2740 1 -- 172.21.15.114:0/3455072625 shutdown_connections 2024-09-13T18:34:10.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.384+0000 7f9d5d2c2740 1 --2- 172.21.15.114:0/3455072625 >> [v2:172.21.15.134:6836/891692210,v1:172.21.15.134:6837/891692210] conn(0x55d25b5667b0 0x55d25b568ba0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.384+0000 7f9d5d2c2740 1 --2- 172.21.15.114:0/3455072625 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d25b4ff950 0x55d25b4f9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.384+0000 7f9d5d2c2740 1 --2- 172.21.15.114:0/3455072625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d25b4f50b0 0x55d25b4f8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.384+0000 7f9d5d2c2740 1 --2- 172.21.15.114:0/3455072625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d25b4f4790 0x55d25b5068a0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.384+0000 7f9d5d2c2740 1 -- 172.21.15.114:0/3455072625 >> 172.21.15.114:0/3455072625 conn(0x55d25b4f0d00 msgr2=0x55d25b4faa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:10.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.384+0000 7f9d337fe640 0 client.28063 ms_handle_reset on v2:172.21.15.134:6836/891692210 2024-09-13T18:34:10.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.384+0000 7f9d5d2c2740 1 -- 172.21.15.114:0/3455072625 shutdown_connections 2024-09-13T18:34:10.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.384+0000 7f9d5d2c2740 1 -- 172.21.15.114:0/3455072625 wait complete. 2024-09-13T18:34:10.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.385+0000 7f9d5d2c2740 20 client.28063 trim_cache size 0 max 16384 2024-09-13T18:34:10.388 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:34:10.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.386+0000 7f9d5a59c640 1 -- 172.21.15.114:0/1626753778 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f9d400a1410 msgr2=0x7f9d400a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:10.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.386+0000 7f9d5a59c640 1 --2- 172.21.15.114:0/1626753778 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f9d400a1410 0x7f9d400a38d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9d44006010 tx=0x7f9d4402a8f0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.386+0000 7f9d5a59c640 1 -- 172.21.15.114:0/1626753778 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9d540fe220 msgr2=0x7f9d54113640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:10.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.386+0000 7f9d5a59c640 1 --2- 172.21.15.114:0/1626753778 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9d540fe220 0x7f9d54113640 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c00b450 tx=0x7f9d4c00b910 comp rx=0 tx=0).stop 2024-09-13T18:34:10.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.386+0000 7f9d5a59c640 1 -- 172.21.15.114:0/1626753778 shutdown_connections 2024-09-13T18:34:10.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.386+0000 7f9d5a59c640 1 --2- 172.21.15.114:0/1626753778 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f9d400a1410 0x7f9d400a38d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.386+0000 7f9d5a59c640 1 --2- 172.21.15.114:0/1626753778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d54104e60 0x7f9d5410d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.386+0000 7f9d5a59c640 1 --2- 172.21.15.114:0/1626753778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d540fec80 0x7f9d54113b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.386+0000 7f9d5a59c640 1 --2- 172.21.15.114:0/1626753778 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9d540fe220 0x7f9d54113640 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.386+0000 7f9d5a59c640 1 -- 172.21.15.114:0/1626753778 >> 172.21.15.114:0/1626753778 conn(0x7f9d540753f0 msgr2=0x7f9d541007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:10.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.387+0000 7f9d5a59c640 1 -- 172.21.15.114:0/1626753778 shutdown_connections 2024-09-13T18:34:10.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.387+0000 7f9d5a59c640 1 -- 172.21.15.114:0/1626753778 wait complete. 2024-09-13T18:34:10.414 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-09-13T18:34:10.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.591+0000 7f2ea575e640 1 -- 172.21.15.114:0/3566131187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ea0104160 msgr2=0x7f2ea01045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:10.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.591+0000 7f2ea575e640 1 --2- 172.21.15.114:0/3566131187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ea0104160 0x7f2ea01045f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f2e98009ca0 tx=0x7f2e9802a940 comp rx=0 tx=0).stop 2024-09-13T18:34:10.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.595+0000 7f2ea575e640 1 -- 172.21.15.114:0/3566131187 shutdown_connections 2024-09-13T18:34:10.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.595+0000 7f2ea575e640 1 --2- 172.21.15.114:0/3566131187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ea0104160 0x7f2ea01045f0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.595+0000 7f2ea575e640 1 --2- 172.21.15.114:0/3566131187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ea00ff650 0x7f2ea0103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.595+0000 7f2ea575e640 1 --2- 172.21.15.114:0/3566131187 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ea00fed30 0x7f2ea00ff110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.595+0000 7f2ea575e640 1 -- 172.21.15.114:0/3566131187 >> 172.21.15.114:0/3566131187 conn(0x7f2ea0075c60 msgr2=0x7f2ea00fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:10.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.599+0000 7f2ea575e640 1 -- 172.21.15.114:0/3566131187 shutdown_connections 2024-09-13T18:34:10.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.600+0000 7f2ea575e640 1 -- 172.21.15.114:0/3566131187 wait complete. 2024-09-13T18:34:10.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.600+0000 7f2ea575e640 1 Processor -- start 2024-09-13T18:34:10.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.601+0000 7f2ea575e640 1 -- start start 2024-09-13T18:34:10.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.601+0000 7f2ea575e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ea00fed30 0x7f2ea006fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:10.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.601+0000 7f2ea575e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ea00ff650 0x7f2ea0070360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:10.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.601+0000 7f2ea575e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ea0104160 0x7f2ea0071670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:10.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.601+0000 7f2ea575e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea0071ce0 con 0x7f2ea00fed30 2024-09-13T18:34:10.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.601+0000 7f2ea575e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea0071e50 con 0x7f2ea00ff650 2024-09-13T18:34:10.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.601+0000 7f2ea575e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea0072010 con 0x7f2ea0104160 2024-09-13T18:34:10.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.601+0000 7f2e9effd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ea00fed30 0x7f2ea006fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:10.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.601+0000 7f2e9effd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ea00fed30 0x7f2ea006fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55172/0 (socket says 172.21.15.114:55172) 2024-09-13T18:34:10.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.601+0000 7f2e9effd640 1 -- 172.21.15.114:0/592867244 learned_addr learned my addr 172.21.15.114:0/592867244 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:34:10.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.602+0000 7f2e9effd640 1 -- 172.21.15.114:0/592867244 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ea0104160 msgr2=0x7f2ea0071670 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:34:10.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.602+0000 7f2e9effd640 1 --2- 172.21.15.114:0/592867244 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ea0104160 0x7f2ea0071670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.602+0000 7f2e9effd640 1 -- 172.21.15.114:0/592867244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ea00ff650 msgr2=0x7f2ea0070360 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:34:10.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.602+0000 7f2e9effd640 1 --2- 172.21.15.114:0/592867244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ea00ff650 0x7f2ea0070360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.602+0000 7f2e9effd640 1 -- 172.21.15.114:0/592867244 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e98009900 con 0x7f2ea00fed30 2024-09-13T18:34:10.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.602+0000 7f2e9effd640 1 --2- 172.21.15.114:0/592867244 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ea00fed30 0x7f2ea006fe00 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f2e9000b4a0 tx=0x7f2e9000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:10.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.606+0000 7f2e9cff9640 1 -- 172.21.15.114:0/592867244 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e90010610 con 0x7f2ea00fed30 2024-09-13T18:34:10.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.606+0000 7f2e9cff9640 1 -- 172.21.15.114:0/592867244 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2e90010c00 con 0x7f2ea00fed30 2024-09-13T18:34:10.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.607+0000 7f2e9cff9640 1 -- 172.21.15.114:0/592867244 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e90017be0 con 0x7f2ea00fed30 2024-09-13T18:34:10.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.607+0000 7f2ea575e640 1 -- 172.21.15.114:0/592867244 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ea00722f0 con 0x7f2ea00fed30 2024-09-13T18:34:10.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.607+0000 7f2ea575e640 1 -- 172.21.15.114:0/592867244 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ea0111550 con 0x7f2ea00fed30 2024-09-13T18:34:10.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.611+0000 7f2ea8486740 1 Processor -- start 2024-09-13T18:34:10.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.611+0000 7f2ea8486740 1 -- start start 2024-09-13T18:34:10.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.612+0000 7f2ea8486740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d0d18e790 0x559d0d18eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:10.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.613+0000 7f2ea8486740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559d0d18f0b0 0x559d0d199410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:10.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.613+0000 7f2ea8486740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d0d199950 0x559d0d19bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:10.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.613+0000 7f2ea8486740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d0d0f8cc0 con 0x559d0d18f0b0 2024-09-13T18:34:10.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.613+0000 7f2ea8486740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d0d0f6450 con 0x559d0d199950 2024-09-13T18:34:10.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.613+0000 7f2ea8486740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d0d0f1d40 con 0x559d0d18e790 2024-09-13T18:34:10.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.614+0000 7f2e9cff9640 1 -- 172.21.15.114:0/592867244 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f2e90017d40 con 0x7f2ea00fed30 2024-09-13T18:34:10.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.614+0000 7f2e97fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d0d199950 0x559d0d19bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:10.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.614+0000 7f2e97fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d0d199950 0x559d0d19bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:59386/0 (socket says 172.21.15.114:59386) 2024-09-13T18:34:10.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.614+0000 7f2e97fff640 1 -- 172.21.15.114:0/1436443276 learned_addr learned my addr 172.21.15.114:0/1436443276 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:34:10.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.614+0000 7f2e97fff640 1 -- 172.21.15.114:0/1436443276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d0d18e790 msgr2=0x559d0d18eb70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:34:10.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.614+0000 7f2e97fff640 1 --2- 172.21.15.114:0/1436443276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d0d18e790 0x559d0d18eb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.614+0000 7f2e97fff640 1 -- 172.21.15.114:0/1436443276 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559d0d18f0b0 msgr2=0x559d0d199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:10.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.614+0000 7f2e97fff640 1 --2- 172.21.15.114:0/1436443276 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559d0d18f0b0 0x559d0d199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.614+0000 7f2e97fff640 1 -- 172.21.15.114:0/1436443276 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d0d0b9620 con 0x559d0d199950 2024-09-13T18:34:10.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.614+0000 7f2e97fff640 1 --2- 172.21.15.114:0/1436443276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d0d199950 0x559d0d19bd40 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f2e88009740 tx=0x7f2e88009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83c0598c036ed928 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:10.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.615+0000 7f2e9cff9640 1 --2- 172.21.15.114:0/592867244 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2e8c0a1260 0x7f2e8c0a3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:10.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.616+0000 7f2e957fa640 1 -- 172.21.15.114:0/1436443276 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8800bad0 con 0x559d0d199950 2024-09-13T18:34:10.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.616+0000 7f2e9f7fe640 1 --2- 172.21.15.114:0/592867244 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2e8c0a1260 0x7f2e8c0a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:10.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.616+0000 7f2e957fa640 1 -- 172.21.15.114:0/1436443276 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2e88009e50 con 0x559d0d199950 2024-09-13T18:34:10.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.616+0000 7f2e957fa640 1 -- 172.21.15.114:0/1436443276 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e88010380 con 0x559d0d199950 2024-09-13T18:34:10.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.616+0000 7f2e9cff9640 1 -- 172.21.15.114:0/592867244 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f2e900cc8c0 con 0x7f2ea00fed30 2024-09-13T18:34:10.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.617+0000 7f2ea8486740 1 -- 172.21.15.114:0/1436443276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d0d199950 msgr2=0x559d0d19bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:10.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.617+0000 7f2ea8486740 1 --2- 172.21.15.114:0/1436443276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d0d199950 0x559d0d19bd40 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f2e88009740 tx=0x7f2e88009c00 comp rx=0 tx=0).stop 2024-09-13T18:34:10.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.618+0000 7f2e9f7fe640 1 --2- 172.21.15.114:0/592867244 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2e8c0a1260 0x7f2e8c0a3720 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2e98006010 tx=0x7f2e98009890 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:10.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.618+0000 7f2ea8486740 1 -- 172.21.15.114:0/1436443276 shutdown_connections 2024-09-13T18:34:10.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.618+0000 7f2ea8486740 1 --2- 172.21.15.114:0/1436443276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d0d199950 0x559d0d19bd40 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.618+0000 7f2ea8486740 1 --2- 172.21.15.114:0/1436443276 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559d0d18f0b0 0x559d0d199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.618+0000 7f2ea8486740 1 --2- 172.21.15.114:0/1436443276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d0d18e790 0x559d0d18eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.618+0000 7f2ea8486740 1 -- 172.21.15.114:0/1436443276 >> 172.21.15.114:0/1436443276 conn(0x559d0d18ad00 msgr2=0x559d0d18b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:10.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.624+0000 7f2ea8486740 1 -- 172.21.15.114:0/1436443276 shutdown_connections 2024-09-13T18:34:10.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.626+0000 7f2ea8486740 1 -- 172.21.15.114:0/1436443276 wait complete. 2024-09-13T18:34:10.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.626+0000 7f2ea8486740 1 -- start start 2024-09-13T18:34:10.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.627+0000 7f2ea8486740 1 Processor -- start 2024-09-13T18:34:10.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.627+0000 7f2ea8486740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d0d18e790 0x559d0d198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:10.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.627+0000 7f2ea8486740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d0d18f0b0 0x559d0d196210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:10.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.627+0000 7f2ea8486740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559d0d199950 0x559d0d196750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:10.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.627+0000 7f2ea8486740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d0d0b0180 con 0x559d0d199950 2024-09-13T18:34:10.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.627+0000 7f2ea8486740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d0cdca430 con 0x559d0d18f0b0 2024-09-13T18:34:10.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.627+0000 7f2ea8486740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d0d0755e0 con 0x559d0d18e790 2024-09-13T18:34:10.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.627+0000 7f2ea8486740 10 client.0 _ll_register_callbacks cb 0x559d0d0e1aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:34:10.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.628+0000 7f2e9effd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d0d18e790 0x559d0d198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:10.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.628+0000 7f2e9effd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d0d18e790 0x559d0d198ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:55028/0 (socket says 172.21.15.114:55028) 2024-09-13T18:34:10.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.628+0000 7f2e9effd640 1 -- 172.21.15.114:0/3910172208 learned_addr learned my addr 172.21.15.114:0/3910172208 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:34:10.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.628+0000 7f2e97fff640 1 --2- 172.21.15.114:0/3910172208 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d0d18f0b0 0x559d0d196210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:10.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.629+0000 7f2e9f7fe640 1 --2- 172.21.15.114:0/3910172208 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559d0d199950 0x559d0d196750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:10.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.630+0000 7f2e9f7fe640 1 -- 172.21.15.114:0/3910172208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d0d18e790 msgr2=0x559d0d198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:10.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.630+0000 7f2e9f7fe640 1 --2- 172.21.15.114:0/3910172208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d0d18e790 0x559d0d198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.630+0000 7f2e9f7fe640 1 -- 172.21.15.114:0/3910172208 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d0d18f0b0 msgr2=0x559d0d196210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:10.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.630+0000 7f2e9f7fe640 1 --2- 172.21.15.114:0/3910172208 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d0d18f0b0 0x559d0d196210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.630+0000 7f2e9f7fe640 1 -- 172.21.15.114:0/3910172208 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e88009370 con 0x559d0d199950 2024-09-13T18:34:10.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.630+0000 7f2e9f7fe640 1 --2- 172.21.15.114:0/3910172208 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559d0d199950 0x559d0d196750 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f2e98009ca0 tx=0x7f2e980095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeefa3dcb222fa76 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:10.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.630+0000 7f2e867fc640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-13T18:34:10.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.632+0000 7f2e867fc640 1 -- 172.21.15.114:0/3910172208 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e98030ac0 con 0x559d0d199950 2024-09-13T18:34:10.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.632+0000 7f2ea8486740 1 -- 172.21.15.114:0/3910172208 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559d0d11bb80 con 0x559d0d199950 2024-09-13T18:34:10.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.632+0000 7f2e867fc640 1 -- 172.21.15.114:0/3910172208 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f2e98004040 con 0x559d0d199950 2024-09-13T18:34:10.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.637+0000 7f2e867fc640 1 -- 172.21.15.114:0/3910172208 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e98004200 con 0x559d0d199950 2024-09-13T18:34:10.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.637+0000 7f2ea8486740 10 client.18666 fetch_fsmap learned FSMap version 39 2024-09-13T18:34:10.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.637+0000 7f2ea8486740 1 -- 172.21.15.114:0/3910172208 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x559d0d0f03c0 con 0x559d0d199950 2024-09-13T18:34:10.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.638+0000 7f2e867fc640 1 -- 172.21.15.114:0/3910172208 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559d0d11bb80 con 0x559d0d199950 2024-09-13T18:34:10.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.638+0000 7f2e867fc640 1 -- 172.21.15.114:0/3910172208 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 39) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f2e9804d020 con 0x559d0d199950 2024-09-13T18:34:10.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.638+0000 7f2ea8486740 10 client.18666 fetch_fsmap finished waiting for FSMap version 39 2024-09-13T18:34:10.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.638+0000 7f2ea8486740 10 client.18666 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:34:10.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.638+0000 7f2ea8486740 20 client.18666 populate_metadata read hostname 'smithi114' 2024-09-13T18:34:10.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.638+0000 7f2ea8486740 1 --2- 172.21.15.114:0/3910172208 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559d0d1a9450 0x559d0d1ab870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:10.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.638+0000 7f2ea8486740 4 client.18666 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:34:10.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.638+0000 7f2ea8486740 1 -- 172.21.15.114:0/3910172208 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559d0d0e41f0 con 0x559d0d1a9450 2024-09-13T18:34:10.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.639+0000 7f2e9effd640 1 --2- 172.21.15.114:0/3910172208 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559d0d1a9450 0x559d0d1ab870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:10.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.645+0000 7f2e9effd640 1 --2- 172.21.15.114:0/3910172208 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559d0d1a9450 0x559d0d1ab870 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e13b057dcee287f1 server_cookie=8dd3fb5c78e95b5 in_seq=0 out_seq=0 2024-09-13T18:34:10.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.646+0000 7f2e867fc640 10 client.18666 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:34:10.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.648+0000 7f2e867fc640 1 -- 172.21.15.114:0/3910172208 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x559d0d0e41f0 con 0x559d0d1a9450 2024-09-13T18:34:10.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.648+0000 7f2e867fc640 10 client.18666 handle_command_reply: tid=0 2024-09-13T18:34:10.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.648+0000 7f2ea8486740 1 client.18666 shutdown 2024-09-13T18:34:10.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.650+0000 7f2ea8486740 1 -- 172.21.15.114:0/3910172208 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559d0d199950 msgr2=0x559d0d196750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:10.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.650+0000 7f2ea8486740 1 --2- 172.21.15.114:0/3910172208 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559d0d199950 0x559d0d196750 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f2e98009ca0 tx=0x7f2e980095c0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.652+0000 7f2ea8486740 1 -- 172.21.15.114:0/3910172208 shutdown_connections 2024-09-13T18:34:10.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.652+0000 7f2ea8486740 1 --2- 172.21.15.114:0/3910172208 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559d0d1a9450 0x559d0d1ab870 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.652+0000 7f2ea8486740 1 --2- 172.21.15.114:0/3910172208 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559d0d199950 0x559d0d196750 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.652+0000 7f2ea8486740 1 --2- 172.21.15.114:0/3910172208 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d0d18f0b0 0x559d0d196210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.652+0000 7f2ea8486740 1 --2- 172.21.15.114:0/3910172208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d0d18e790 0x559d0d198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.652+0000 7f2ea8486740 1 -- 172.21.15.114:0/3910172208 >> 172.21.15.114:0/3910172208 conn(0x559d0d18ad00 msgr2=0x559d0d190a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:10.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.654+0000 7f2e867fc640 0 client.18666 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:34:10.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.658+0000 7f2ea8486740 1 -- 172.21.15.114:0/3910172208 shutdown_connections 2024-09-13T18:34:10.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.661+0000 7f2ea8486740 1 -- 172.21.15.114:0/3910172208 wait complete. 2024-09-13T18:34:10.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.663+0000 7f2ea8486740 20 client.18666 trim_cache size 0 max 16384 2024-09-13T18:34:10.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.684+0000 7f2ea8486740 1 Processor -- start 2024-09-13T18:34:10.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.684+0000 7f2ea8486740 1 -- start start 2024-09-13T18:34:10.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.684+0000 7f2ea8486740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d0d18e790 0x559d0d18a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:10.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.684+0000 7f2ea8486740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559d0d18f0b0 0x559d0d194ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:10.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.684+0000 7f2ea8486740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d0d199950 0x559d0d195000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:10.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.684+0000 7f2ea8486740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d0d1853e0 con 0x559d0d18f0b0 2024-09-13T18:34:10.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.684+0000 7f2ea8486740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d0d1994d0 con 0x559d0d18e790 2024-09-13T18:34:10.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.684+0000 7f2ea8486740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d0d199770 con 0x559d0d199950 2024-09-13T18:34:10.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.685+0000 7f2e97fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559d0d18f0b0 0x559d0d194ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:10.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.685+0000 7f2e9f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d0d199950 0x559d0d195000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:10.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.685+0000 7f2e97fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559d0d18f0b0 0x559d0d194ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55188/0 (socket says 172.21.15.114:55188) 2024-09-13T18:34:10.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.685+0000 7f2e97fff640 1 -- 172.21.15.114:0/3714770157 learned_addr learned my addr 172.21.15.114:0/3714770157 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:34:10.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.685+0000 7f2e9effd640 1 --2- 172.21.15.114:0/3714770157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d0d18e790 0x559d0d18a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:10.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.688+0000 7f2e97fff640 1 -- 172.21.15.114:0/3714770157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d0d199950 msgr2=0x559d0d195000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:10.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.688+0000 7f2e97fff640 1 --2- 172.21.15.114:0/3714770157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d0d199950 0x559d0d195000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.688+0000 7f2e97fff640 1 -- 172.21.15.114:0/3714770157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d0d18e790 msgr2=0x559d0d18a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:10.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.688+0000 7f2e97fff640 1 --2- 172.21.15.114:0/3714770157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d0d18e790 0x559d0d18a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.688+0000 7f2e97fff640 1 -- 172.21.15.114:0/3714770157 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d0d092770 con 0x559d0d18f0b0 2024-09-13T18:34:10.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.693+0000 7f2e97fff640 1 --2- 172.21.15.114:0/3714770157 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559d0d18f0b0 0x559d0d194ac0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x559d0d191330 tx=0x7f2e8800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2c04ce272f19a0d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:10.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.693+0000 7f2e95ffb640 1 -- 172.21.15.114:0/3714770157 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8800bc40 con 0x559d0d18f0b0 2024-09-13T18:34:10.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.693+0000 7f2e95ffb640 1 -- 172.21.15.114:0/3714770157 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2e8801f070 con 0x559d0d18f0b0 2024-09-13T18:34:10.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.693+0000 7f2e95ffb640 1 -- 172.21.15.114:0/3714770157 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8801b920 con 0x559d0d18f0b0 2024-09-13T18:34:10.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.694+0000 7f2ea8486740 1 -- 172.21.15.114:0/3714770157 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559d0d18f0b0 msgr2=0x559d0d194ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:10.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.694+0000 7f2ea8486740 1 --2- 172.21.15.114:0/3714770157 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559d0d18f0b0 0x559d0d194ac0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x559d0d191330 tx=0x7f2e8800b8d0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.697+0000 7f2ea8486740 1 -- 172.21.15.114:0/3714770157 shutdown_connections 2024-09-13T18:34:10.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.697+0000 7f2ea8486740 1 --2- 172.21.15.114:0/3714770157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d0d199950 0x559d0d195000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.697+0000 7f2ea8486740 1 --2- 172.21.15.114:0/3714770157 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559d0d18f0b0 0x559d0d194ac0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.697+0000 7f2ea8486740 1 --2- 172.21.15.114:0/3714770157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d0d18e790 0x559d0d18a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.697+0000 7f2ea8486740 1 -- 172.21.15.114:0/3714770157 >> 172.21.15.114:0/3714770157 conn(0x559d0d18ad00 msgr2=0x559d0d1b0190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:10.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.699+0000 7f2ea8486740 1 -- 172.21.15.114:0/3714770157 shutdown_connections 2024-09-13T18:34:10.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.699+0000 7f2ea8486740 1 -- 172.21.15.114:0/3714770157 wait complete. 2024-09-13T18:34:10.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.700+0000 7f2ea8486740 1 -- start start 2024-09-13T18:34:10.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.700+0000 7f2ea8486740 1 Processor -- start 2024-09-13T18:34:10.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.700+0000 7f2ea8486740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559d0d18e790 0x559d0d1a04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:10.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.700+0000 7f2ea8486740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d0d18f0b0 0x559d0d1a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:10.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.700+0000 7f2ea8486740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d0d199950 0x559d0d19e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:10.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.700+0000 7f2ea8486740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d0d19f0c0 con 0x559d0d18e790 2024-09-13T18:34:10.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.700+0000 7f2ea8486740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d0d19f230 con 0x559d0d18f0b0 2024-09-13T18:34:10.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.700+0000 7f2ea8486740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d0d19f3f0 con 0x559d0d199950 2024-09-13T18:34:10.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.700+0000 7f2ea8486740 10 client.0 _ll_register_callbacks cb 0x7f2e88006f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:34:10.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.701+0000 7f2e9f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d0d199950 0x559d0d19e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:10.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.701+0000 7f2e9f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d0d199950 0x559d0d19e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:55042/0 (socket says 172.21.15.114:55042) 2024-09-13T18:34:10.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.701+0000 7f2e9f7fe640 1 -- 172.21.15.114:0/3666439230 learned_addr learned my addr 172.21.15.114:0/3666439230 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:34:10.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.701+0000 7f2e97fff640 1 --2- 172.21.15.114:0/3666439230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d0d18f0b0 0x559d0d1a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:10.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.701+0000 7f2e9f7fe640 1 -- 172.21.15.114:0/3666439230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d0d18f0b0 msgr2=0x559d0d1a0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:10.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.701+0000 7f2e9f7fe640 1 --2- 172.21.15.114:0/3666439230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d0d18f0b0 0x559d0d1a0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.701+0000 7f2e9f7fe640 1 -- 172.21.15.114:0/3666439230 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559d0d18e790 msgr2=0x559d0d1a04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:34:10.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.701+0000 7f2e9f7fe640 1 --2- 172.21.15.114:0/3666439230 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559d0d18e790 0x559d0d1a04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.701+0000 7f2e9f7fe640 1 -- 172.21.15.114:0/3666439230 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559d0d0b9620 con 0x559d0d199950 2024-09-13T18:34:10.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.702+0000 7f2e9f7fe640 1 --2- 172.21.15.114:0/3666439230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d0d199950 0x559d0d19e9c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2e980049d0 tx=0x7f2e9803a6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65e619b833ae47f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:10.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.702+0000 7f2e867fc640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-13T18:34:10.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.703+0000 7f2e867fc640 1 -- 172.21.15.114:0/3666439230 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e98004450 con 0x559d0d199950 2024-09-13T18:34:10.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.703+0000 7f2e867fc640 1 -- 172.21.15.114:0/3666439230 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f2e9803acb0 con 0x559d0d199950 2024-09-13T18:34:10.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.703+0000 7f2ea8486740 1 -- 172.21.15.114:0/3666439230 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2e88009e50 con 0x559d0d199950 2024-09-13T18:34:10.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.703+0000 7f2e867fc640 1 -- 172.21.15.114:0/3666439230 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e9802afc0 con 0x559d0d199950 2024-09-13T18:34:10.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.704+0000 7f2ea8486740 10 client.28075 fetch_fsmap learned FSMap version 39 2024-09-13T18:34:10.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.704+0000 7f2ea8486740 1 -- 172.21.15.114:0/3666439230 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x559d0d0b7e80 con 0x559d0d199950 2024-09-13T18:34:10.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.706+0000 7f2e867fc640 1 -- 172.21.15.114:0/3666439230 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2e88009e50 con 0x559d0d199950 2024-09-13T18:34:10.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.706+0000 7f2e867fc640 1 -- 172.21.15.114:0/3666439230 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 39) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f2e9804d040 con 0x559d0d199950 2024-09-13T18:34:10.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.707+0000 7f2ea8486740 10 client.28075 fetch_fsmap finished waiting for FSMap version 39 2024-09-13T18:34:10.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.707+0000 7f2ea8486740 10 client.28075 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-13T18:34:10.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.707+0000 7f2ea8486740 20 client.28075 populate_metadata read hostname 'smithi114' 2024-09-13T18:34:10.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.707+0000 7f2ea8486740 1 --2- 172.21.15.114:0/3666439230 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559d0d200650 0x559d0d202a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:10.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.707+0000 7f2ea8486740 4 client.28075 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-13T18:34:10.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.707+0000 7f2ea8486740 1 -- 172.21.15.114:0/3666439230 --> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559d0cde54e0 con 0x559d0d200650 2024-09-13T18:34:10.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.708+0000 7f2e9effd640 1 --2- 172.21.15.114:0/3666439230 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559d0d200650 0x559d0d202a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:10.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.709+0000 7f2e9effd640 1 --2- 172.21.15.114:0/3666439230 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559d0d200650 0x559d0d202a40 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d9026b32f7a1ee6 server_cookie=bc25f1524d77c788 in_seq=0 out_seq=0 2024-09-13T18:34:10.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.709+0000 7f2e867fc640 10 client.28075 ms_handle_connect on v2:172.21.15.134:6838/3466309724 2024-09-13T18:34:10.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.709+0000 7f2e867fc640 1 -- 172.21.15.114:0/3666439230 <== mds.0 v2:172.21.15.134:6838/3466309724 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559d0cde54e0 con 0x559d0d200650 2024-09-13T18:34:10.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.709+0000 7f2e867fc640 10 client.28075 handle_command_reply: tid=0 2024-09-13T18:34:10.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.709+0000 7f2ea8486740 1 client.28075 shutdown 2024-09-13T18:34:10.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.710+0000 7f2ea8486740 1 -- 172.21.15.114:0/3666439230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d0d199950 msgr2=0x559d0d19e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:10.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.710+0000 7f2ea8486740 1 --2- 172.21.15.114:0/3666439230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d0d199950 0x559d0d19e9c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2e980049d0 tx=0x7f2e9803a6e0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.710+0000 7f2ea8486740 1 -- 172.21.15.114:0/3666439230 shutdown_connections 2024-09-13T18:34:10.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.710+0000 7f2ea8486740 1 --2- 172.21.15.114:0/3666439230 >> [v2:172.21.15.134:6838/3466309724,v1:172.21.15.134:6839/3466309724] conn(0x559d0d200650 0x559d0d202a40 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.710+0000 7f2ea8486740 1 --2- 172.21.15.114:0/3666439230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d0d199950 0x559d0d19e9c0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.710+0000 7f2ea8486740 1 --2- 172.21.15.114:0/3666439230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d0d18f0b0 0x559d0d1a0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.710+0000 7f2ea8486740 1 --2- 172.21.15.114:0/3666439230 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559d0d18e790 0x559d0d1a04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.710+0000 7f2ea8486740 1 -- 172.21.15.114:0/3666439230 >> 172.21.15.114:0/3666439230 conn(0x559d0d18ad00 msgr2=0x559d0d1949b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:10.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.710+0000 7f2e867fc640 0 client.28075 ms_handle_reset on v2:172.21.15.134:6838/3466309724 2024-09-13T18:34:10.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.710+0000 7f2ea8486740 1 -- 172.21.15.114:0/3666439230 shutdown_connections 2024-09-13T18:34:10.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.710+0000 7f2ea8486740 1 -- 172.21.15.114:0/3666439230 wait complete. 2024-09-13T18:34:10.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.710+0000 7f2ea8486740 20 client.28075 trim_cache size 0 max 16384 2024-09-13T18:34:10.714 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-13T18:34:10.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.711+0000 7f2ea575e640 1 -- 172.21.15.114:0/592867244 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2e8c0a1260 msgr2=0x7f2e8c0a3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:10.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.711+0000 7f2ea575e640 1 --2- 172.21.15.114:0/592867244 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2e8c0a1260 0x7f2e8c0a3720 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2e98006010 tx=0x7f2e98009890 comp rx=0 tx=0).stop 2024-09-13T18:34:10.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.711+0000 7f2ea575e640 1 -- 172.21.15.114:0/592867244 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ea00fed30 msgr2=0x7f2ea006fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:10.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.711+0000 7f2ea575e640 1 --2- 172.21.15.114:0/592867244 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ea00fed30 0x7f2ea006fe00 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f2e9000b4a0 tx=0x7f2e9000b960 comp rx=0 tx=0).stop 2024-09-13T18:34:10.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.714+0000 7f2ea575e640 1 -- 172.21.15.114:0/592867244 shutdown_connections 2024-09-13T18:34:10.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.714+0000 7f2ea575e640 1 --2- 172.21.15.114:0/592867244 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f2e8c0a1260 0x7f2e8c0a3720 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.714+0000 7f2ea575e640 1 --2- 172.21.15.114:0/592867244 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ea0104160 0x7f2ea0071670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.714+0000 7f2ea575e640 1 --2- 172.21.15.114:0/592867244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ea00ff650 0x7f2ea0070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.714+0000 7f2ea575e640 1 --2- 172.21.15.114:0/592867244 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ea00fed30 0x7f2ea006fe00 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:10.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.714+0000 7f2ea575e640 1 -- 172.21.15.114:0/592867244 >> 172.21.15.114:0/592867244 conn(0x7f2ea0075c60 msgr2=0x7f2ea0106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:10.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.715+0000 7f2ea575e640 1 -- 172.21.15.114:0/592867244 shutdown_connections 2024-09-13T18:34:10.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:10.715+0000 7f2ea575e640 1 -- 172.21.15.114:0/592867244 wait complete. 2024-09-13T18:34:11.735 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-09-13T18:34:11.736 INFO:tasks.fwd_scrub:done joining 2024-09-13T18:34:11.736 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-13T18:34:11.749 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:34:11.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:11.891+0000 7fbe60096640 1 -- 172.21.15.114:0/2996822247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe580fe630 msgr2=0x7fbe580fea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:11.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:11.892+0000 7fbe60096640 1 --2- 172.21.15.114:0/2996822247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe580fe630 0x7fbe580fea10 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fbe54009900 tx=0x7fbe5402a3f0 comp rx=0 tx=0).stop 2024-09-13T18:34:11.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:11.892+0000 7fbe60096640 1 -- 172.21.15.114:0/2996822247 shutdown_connections 2024-09-13T18:34:11.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:11.892+0000 7fbe60096640 1 --2- 172.21.15.114:0/2996822247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe58103730 0x7fbe5810b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:11.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:11.892+0000 7fbe60096640 1 --2- 172.21.15.114:0/2996822247 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe580fefe0 0x7fbe58102fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:11.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:11.892+0000 7fbe60096640 1 --2- 172.21.15.114:0/2996822247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe580fe630 0x7fbe580fea10 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:11.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:11.892+0000 7fbe60096640 1 -- 172.21.15.114:0/2996822247 >> 172.21.15.114:0/2996822247 conn(0x7fbe580753f0 msgr2=0x7fbe580fc740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:11.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:11.892+0000 7fbe60096640 1 -- 172.21.15.114:0/2996822247 shutdown_connections 2024-09-13T18:34:11.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:11.892+0000 7fbe60096640 1 -- 172.21.15.114:0/2996822247 wait complete. 2024-09-13T18:34:11.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:11.893+0000 7fbe60096640 1 Processor -- start 2024-09-13T18:34:11.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:11.893+0000 7fbe60096640 1 -- start start 2024-09-13T18:34:11.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:11.893+0000 7fbe60096640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe580fe630 0x7fbe58198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:11.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:11.893+0000 7fbe60096640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe58103730 0x7fbe581988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:11.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:11.893+0000 7fbe60096640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe5819ccb0 0x7fbe5819d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:11.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:11.893+0000 7fbe60096640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5819d6a0 con 0x7fbe58103730 2024-09-13T18:34:11.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:11.893+0000 7fbe60096640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5819d810 con 0x7fbe5819ccb0 2024-09-13T18:34:11.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:11.893+0000 7fbe60096640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5819d980 con 0x7fbe580fe630 2024-09-13T18:34:11.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:11.893+0000 7fbe5e633640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe5819ccb0 0x7fbe5819d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:11.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:11.893+0000 7fbe5e633640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe5819ccb0 0x7fbe5819d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:59444/0 (socket says 172.21.15.114:59444) 2024-09-13T18:34:11.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:11.893+0000 7fbe5e633640 1 -- 172.21.15.114:0/3908739727 learned_addr learned my addr 172.21.15.114:0/3908739727 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:34:11.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:11.893+0000 7fbe5d631640 1 --2- 172.21.15.114:0/3908739727 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe58103730 0x7fbe581988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:11.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:11.893+0000 7fbe5de32640 1 --2- 172.21.15.114:0/3908739727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe580fe630 0x7fbe58198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:11.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:11.894+0000 7fbe5e633640 1 -- 172.21.15.114:0/3908739727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe580fe630 msgr2=0x7fbe58198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:11.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:11.894+0000 7fbe5e633640 1 --2- 172.21.15.114:0/3908739727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe580fe630 0x7fbe58198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:11.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:11.894+0000 7fbe5e633640 1 -- 172.21.15.114:0/3908739727 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe58103730 msgr2=0x7fbe581988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:11.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:11.894+0000 7fbe5e633640 1 --2- 172.21.15.114:0/3908739727 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe58103730 0x7fbe581988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:11.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:11.894+0000 7fbe5e633640 1 -- 172.21.15.114:0/3908739727 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe50009840 con 0x7fbe5819ccb0 2024-09-13T18:34:11.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:11.894+0000 7fbe5e633640 1 --2- 172.21.15.114:0/3908739727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe5819ccb0 0x7fbe5819d160 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fbe5000efd0 tx=0x7fbe5000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:11.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:11.894+0000 7fbe4effd640 1 -- 172.21.15.114:0/3908739727 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe50009b40 con 0x7fbe5819ccb0 2024-09-13T18:34:11.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:11.894+0000 7fbe60096640 1 -- 172.21.15.114:0/3908739727 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe54009560 con 0x7fbe5819ccb0 2024-09-13T18:34:11.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:11.894+0000 7fbe60096640 1 -- 172.21.15.114:0/3908739727 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe581a5280 con 0x7fbe5819ccb0 2024-09-13T18:34:11.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:11.894+0000 7fbe4effd640 1 -- 172.21.15.114:0/3908739727 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbe50004020 con 0x7fbe5819ccb0 2024-09-13T18:34:11.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:11.894+0000 7fbe4effd640 1 -- 172.21.15.114:0/3908739727 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe50010740 con 0x7fbe5819ccb0 2024-09-13T18:34:11.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:11.895+0000 7fbe60096640 1 -- 172.21.15.114:0/3908739727 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe3c005250 con 0x7fbe5819ccb0 2024-09-13T18:34:11.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:11.898+0000 7fbe4effd640 1 -- 172.21.15.114:0/3908739727 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fbe50003b70 con 0x7fbe5819ccb0 2024-09-13T18:34:11.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:11.899+0000 7fbe4effd640 1 --2- 172.21.15.114:0/3908739727 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbe440a1300 0x7fbe440a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:11.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:11.899+0000 7fbe4effd640 1 -- 172.21.15.114:0/3908739727 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fbe50014070 con 0x7fbe5819ccb0 2024-09-13T18:34:11.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:11.899+0000 7fbe5de32640 1 --2- 172.21.15.114:0/3908739727 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbe440a1300 0x7fbe440a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:11.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:11.902+0000 7fbe5de32640 1 --2- 172.21.15.114:0/3908739727 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbe440a1300 0x7fbe440a37c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbe54000c00 tx=0x7fbe540058d0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:11.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:11.905+0000 7fbe4effd640 1 -- 172.21.15.114:0/3908739727 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fbe500965c0 con 0x7fbe5819ccb0 2024-09-13T18:34:11.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:11 smithi204 ceph-mon[34777]: pgmap v365: 129 pgs: 12 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 89 active+clean; 18 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-09-13T18:34:11.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:11 smithi114 ceph-mon[30897]: pgmap v365: 129 pgs: 12 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 89 active+clean; 18 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-09-13T18:34:12.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.105+0000 7fbe60096640 1 -- 172.21.15.114:0/3908739727 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbe3c0050c0 con 0x7fbe5819ccb0 2024-09-13T18:34:12.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.108+0000 7fbe4effd640 1 -- 172.21.15.114:0/3908739727 <== mon.2 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) v1 ==== 74+0+25400 (secure 0 0 0) 0x7fbe50099be0 con 0x7fbe5819ccb0 2024-09-13T18:34:12.110 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:34:12.111 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":105,"fsid":"97e7edf2-71fb-11ef-bceb-c7b262605968","created":"2024-09-13T18:12:36.360625+0000","modified":"2024-09-13T18:34:06.616269+0000","last_up_change":"2024-09-13T18:19:12.350370+0000","last_in_change":"2024-09-13T18:18:52.321596+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:15:40.129659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:20:00.024407+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:20:01.085549+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":105,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T18:20:04.334216+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":105,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9914f01a-9f6f-4878-b955-83570199c354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6803","nonce":3499525826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6805","nonce":3499525826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6809","nonce":3499525826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6807","nonce":3499525826}]},"public_addr":"172.21.15.114:6803/3499525826","cluster_addr":"172.21.15.114:6805/3499525826","heartbeat_back_addr":"172.21.15.114:6809/3499525826","heartbeat_front_addr":"172.21.15.114:6807/3499525826","state":["exists","up"]},{"osd":1,"uuid":"ef65e012-7469-4c20-b354-17022bca63c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6801","nonce":4264264866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6803","nonce":4264264866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6807","nonce":4264264866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6805","nonce":4264264866}]},"public_addr":"172.21.15.134:6801/4264264866","cluster_addr":"172.21.15.134:6803/4264264866","heartbeat_back_addr":"172.21.15.134:6807/4264264866","heartbeat_front_addr":"172.21.15.134:6805/4264264866","state":["exists","up"]},{"osd":2,"uuid":"578213fd-715f-4be7-91fa-254f3d0e3022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6801","nonce":2288807707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6803","nonce":2288807707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6807","nonce":2288807707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6805","nonce":2288807707}]},"public_addr":"172.21.15.204:6801/2288807707","cluster_addr":"172.21.15.204:6803/2288807707","heartbeat_back_addr":"172.21.15.204:6807/2288807707","heartbeat_front_addr":"172.21.15.204:6805/2288807707","state":["exists","up"]},{"osd":3,"uuid":"b64aab3f-1e52-4828-8397-74af5e05f2cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6811","nonce":2904204254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6813","nonce":2904204254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6817","nonce":2904204254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6815","nonce":2904204254}]},"public_addr":"172.21.15.114:6811/2904204254","cluster_addr":"172.21.15.114:6813/2904204254","heartbeat_back_addr":"172.21.15.114:6817/2904204254","heartbeat_front_addr":"172.21.15.114:6815/2904204254","state":["exists","up"]},{"osd":4,"uuid":"f38943d6-39c0-42e1-b427-cb6bb90ac38a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6809","nonce":1773945333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6811","nonce":1773945333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6815","nonce":1773945333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6813","nonce":1773945333}]},"public_addr":"172.21.15.134:6809/1773945333","cluster_addr":"172.21.15.134:6811/1773945333","heartbeat_back_addr":"172.21.15.134:6815/1773945333","heartbeat_front_addr":"172.21.15.134:6813/1773945333","state":["exists","up"]},{"osd":5,"uuid":"bb38f361-8e00-4178-9752-5ddbf3af20e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6809","nonce":2371778055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6811","nonce":2371778055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6815","nonce":2371778055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6813","nonce":2371778055}]},"public_addr":"172.21.15.204:6809/2371778055","cluster_addr":"172.21.15.204:6811/2371778055","heartbeat_back_addr":"172.21.15.204:6815/2371778055","heartbeat_front_addr":"172.21.15.204:6813/2371778055","state":["exists","up"]},{"osd":6,"uuid":"93d40a03-e7ff-4a3f-9efa-5c5403c9983d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6819","nonce":1484325044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6821","nonce":1484325044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6825","nonce":1484325044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6823","nonce":1484325044}]},"public_addr":"172.21.15.114:6819/1484325044","cluster_addr":"172.21.15.114:6821/1484325044","heartbeat_back_addr":"172.21.15.114:6825/1484325044","heartbeat_front_addr":"172.21.15.114:6823/1484325044","state":["exists","up"]},{"osd":7,"uuid":"982f629a-1b95-4be1-8e75-5ddce1ad77b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6817","nonce":799246614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6819","nonce":799246614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6823","nonce":799246614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6821","nonce":799246614}]},"public_addr":"172.21.15.134:6817/799246614","cluster_addr":"172.21.15.134:6819/799246614","heartbeat_back_addr":"172.21.15.134:6823/799246614","heartbeat_front_addr":"172.21.15.134:6821/799246614","state":["exists","up"]},{"osd":8,"uuid":"409beb9b-8227-4dad-a4a0-f0e5ce70dc5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6817","nonce":858331805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6819","nonce":858331805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6823","nonce":858331805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6821","nonce":858331805}]},"public_addr":"172.21.15.204:6817/858331805","cluster_addr":"172.21.15.204:6819/858331805","heartbeat_back_addr":"172.21.15.204:6823/858331805","heartbeat_front_addr":"172.21.15.204:6821/858331805","state":["exists","up"]},{"osd":9,"uuid":"a345a15c-b1fc-4877-b445-418eebf0d0ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6827","nonce":2172392854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6829","nonce":2172392854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6833","nonce":2172392854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6831","nonce":2172392854}]},"public_addr":"172.21.15.114:6827/2172392854","cluster_addr":"172.21.15.114:6829/2172392854","heartbeat_back_addr":"172.21.15.114:6833/2172392854","heartbeat_front_addr":"172.21.15.114:6831/2172392854","state":["exists","up"]},{"osd":10,"uuid":"464d995f-d90b-4caa-addb-1f7562bd74b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6825","nonce":3073513468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6827","nonce":3073513468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6831","nonce":3073513468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6829","nonce":3073513468}]},"public_addr":"172.21.15.134:6825/3073513468","cluster_addr":"172.21.15.134:6827/3073513468","heartbeat_back_addr":"172.21.15.134:6831/3073513468","heartbeat_front_addr":"172.21.15.134:6829/3073513468","state":["exists","up"]},{"osd":11,"uuid":"71dd794d-326c-477f-8d02-ee32bff1350d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6825","nonce":3239521765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6827","nonce":3239521765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6831","nonce":3239521765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6829","nonce":3239521765}]},"public_addr":"172.21.15.204:6825/3239521765","cluster_addr":"172.21.15.204:6827/3239521765","heartbeat_back_addr":"172.21.15.204:6831/3239521765","heartbeat_front_addr":"172.21.15.204:6829/3239521765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:14:54.542687+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:15:15.746055+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:15:38.371131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:01.383932+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:24.796237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:48.422214+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:11.284323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:34.570365+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:58.232855+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:18:21.609596+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:18:45.565859+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:19:10.017969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:6801/3396185541":"2024-09-14T18:22:28.057623+0000","172.21.15.114:6800/830738794":"2024-09-14T18:13:03.521116+0000","172.21.15.114:6801/830738794":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/1684954293":"2024-09-14T18:22:28.057623+0000","172.21.15.114:0/3544951322":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/722571999":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/2949422849":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2034742251":"2024-09-14T18:13:03.521116+0000","172.21.15.114:6800/4281866858":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/3407523320":"2024-09-14T18:22:28.057623+0000","172.21.15.114:6800/3396185541":"2024-09-14T18:22:28.057623+0000","172.21.15.114:0/4257828906":"2024-09-14T18:13:25.823027+0000","172.21.15.114:6801/4281866858":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2828364892":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2714091073":"2024-09-14T18:22:28.057623+0000","172.21.15.114:0/842237781":"2024-09-14T18:22:28.057623+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]},{"pool":4,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]},{"pool":4,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:34:12.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.110+0000 7fbe60096640 1 -- 172.21.15.114:0/3908739727 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbe440a1300 msgr2=0x7fbe440a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:12.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.110+0000 7fbe60096640 1 --2- 172.21.15.114:0/3908739727 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbe440a1300 0x7fbe440a37c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbe54000c00 tx=0x7fbe540058d0 comp rx=0 tx=0).stop 2024-09-13T18:34:12.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.110+0000 7fbe60096640 1 -- 172.21.15.114:0/3908739727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe5819ccb0 msgr2=0x7fbe5819d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:12.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.110+0000 7fbe60096640 1 --2- 172.21.15.114:0/3908739727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe5819ccb0 0x7fbe5819d160 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fbe5000efd0 tx=0x7fbe5000c6a0 comp rx=0 tx=0).stop 2024-09-13T18:34:12.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.110+0000 7fbe60096640 1 -- 172.21.15.114:0/3908739727 shutdown_connections 2024-09-13T18:34:12.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.110+0000 7fbe60096640 1 --2- 172.21.15.114:0/3908739727 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbe440a1300 0x7fbe440a37c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:12.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.110+0000 7fbe60096640 1 --2- 172.21.15.114:0/3908739727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe5819ccb0 0x7fbe5819d160 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:12.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.110+0000 7fbe60096640 1 --2- 172.21.15.114:0/3908739727 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe58103730 0x7fbe581988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:12.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.110+0000 7fbe60096640 1 --2- 172.21.15.114:0/3908739727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe580fe630 0x7fbe58198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:12.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.110+0000 7fbe60096640 1 -- 172.21.15.114:0/3908739727 >> 172.21.15.114:0/3908739727 conn(0x7fbe580753f0 msgr2=0x7fbe58100fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:12.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.110+0000 7fbe60096640 1 -- 172.21.15.114:0/3908739727 shutdown_connections 2024-09-13T18:34:12.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.110+0000 7fbe60096640 1 -- 172.21.15.114:0/3908739727 wait complete. 2024-09-13T18:34:12.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:11 smithi134 ceph-mon[34283]: pgmap v365: 129 pgs: 12 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 89 active+clean; 18 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-09-13T18:34:12.123 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T18:15:40.129659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T18:20:00.024407+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T18:20:01.085549+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 19, 'snap_epoch': 105, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T18:20:04.334216+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 19, 'snap_epoch': 105, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T18:34:12.124 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T18:34:12.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.229+0000 7ff22d8b7640 1 -- 172.21.15.114:0/2474320978 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff2280fee60 msgr2=0x7ff2280ff240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:12.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.229+0000 7ff22d8b7640 1 --2- 172.21.15.114:0/2474320978 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff2280fee60 0x7ff2280ff240 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7ff218009930 tx=0x7ff21802a420 comp rx=0 tx=0).stop 2024-09-13T18:34:12.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.229+0000 7ff22d8b7640 1 -- 172.21.15.114:0/2474320978 shutdown_connections 2024-09-13T18:34:12.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.229+0000 7ff22d8b7640 1 --2- 172.21.15.114:0/2474320978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff228103fb0 0x7ff22810bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:12.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.229+0000 7ff22d8b7640 1 --2- 172.21.15.114:0/2474320978 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2280ff810 0x7ff228103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:12.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.229+0000 7ff22d8b7640 1 --2- 172.21.15.114:0/2474320978 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff2280fee60 0x7ff2280ff240 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:12.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.229+0000 7ff22d8b7640 1 -- 172.21.15.114:0/2474320978 >> 172.21.15.114:0/2474320978 conn(0x7ff228075c60 msgr2=0x7ff2280fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:12.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.229+0000 7ff22d8b7640 1 -- 172.21.15.114:0/2474320978 shutdown_connections 2024-09-13T18:34:12.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.229+0000 7ff22d8b7640 1 -- 172.21.15.114:0/2474320978 wait complete. 2024-09-13T18:34:12.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.230+0000 7ff22d8b7640 1 Processor -- start 2024-09-13T18:34:12.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.230+0000 7ff22d8b7640 1 -- start start 2024-09-13T18:34:12.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.230+0000 7ff22d8b7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff2280fee60 0x7ff228196210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:12.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.230+0000 7ff22d8b7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff228103fb0 0x7ff228196750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:12.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.230+0000 7ff22d8b7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff22819ab40 0x7ff22819aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:12.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.230+0000 7ff22d8b7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff22819b530 con 0x7ff2280fee60 2024-09-13T18:34:12.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.230+0000 7ff22d8b7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff22819b6a0 con 0x7ff22819ab40 2024-09-13T18:34:12.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.230+0000 7ff22d8b7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff22819b810 con 0x7ff228103fb0 2024-09-13T18:34:12.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.230+0000 7ff226ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff2280fee60 0x7ff228196210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:12.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.230+0000 7ff226ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff2280fee60 0x7ff228196210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55246/0 (socket says 172.21.15.114:55246) 2024-09-13T18:34:12.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.230+0000 7ff226ffd640 1 -- 172.21.15.114:0/3138633016 learned_addr learned my addr 172.21.15.114:0/3138633016 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:34:12.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.230+0000 7ff2267fc640 1 --2- 172.21.15.114:0/3138633016 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff228103fb0 0x7ff228196750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:12.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.230+0000 7ff2277fe640 1 --2- 172.21.15.114:0/3138633016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff22819ab40 0x7ff22819aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:12.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.231+0000 7ff226ffd640 1 -- 172.21.15.114:0/3138633016 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff228103fb0 msgr2=0x7ff228196750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:12.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.231+0000 7ff226ffd640 1 --2- 172.21.15.114:0/3138633016 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff228103fb0 0x7ff228196750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:12.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.231+0000 7ff226ffd640 1 -- 172.21.15.114:0/3138633016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff22819ab40 msgr2=0x7ff22819aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:12.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.231+0000 7ff226ffd640 1 --2- 172.21.15.114:0/3138633016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff22819ab40 0x7ff22819aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:12.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.231+0000 7ff226ffd640 1 -- 172.21.15.114:0/3138633016 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff218009560 con 0x7ff2280fee60 2024-09-13T18:34:12.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.231+0000 7ff226ffd640 1 --2- 172.21.15.114:0/3138633016 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff2280fee60 0x7ff228196210 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7ff21802a940 tx=0x7ff218005610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:12.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.231+0000 7ff22c8b5640 1 -- 172.21.15.114:0/3138633016 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff218004c50 con 0x7ff2280fee60 2024-09-13T18:34:12.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.231+0000 7ff22c8b5640 1 -- 172.21.15.114:0/3138633016 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff218004db0 con 0x7ff2280fee60 2024-09-13T18:34:12.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.231+0000 7ff22c8b5640 1 -- 172.21.15.114:0/3138633016 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21803c9c0 con 0x7ff2280fee60 2024-09-13T18:34:12.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.231+0000 7ff22d8b7640 1 -- 172.21.15.114:0/3138633016 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2281a2df0 con 0x7ff2280fee60 2024-09-13T18:34:12.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.231+0000 7ff22d8b7640 1 -- 172.21.15.114:0/3138633016 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2281a31c0 con 0x7ff2280fee60 2024-09-13T18:34:12.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.232+0000 7ff22d8b7640 1 -- 172.21.15.114:0/3138633016 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff228100330 con 0x7ff2280fee60 2024-09-13T18:34:12.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.233+0000 7ff22c8b5640 1 -- 172.21.15.114:0/3138633016 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7ff218003ca0 con 0x7ff2280fee60 2024-09-13T18:34:12.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.234+0000 7ff22c8b5640 1 --2- 172.21.15.114:0/3138633016 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff2100a1200 0x7ff2100a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:12.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.234+0000 7ff2267fc640 1 --2- 172.21.15.114:0/3138633016 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff2100a1200 0x7ff2100a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:12.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.234+0000 7ff22c8b5640 1 -- 172.21.15.114:0/3138633016 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9541+0+0 (secure 0 0 0) 0x7ff2180ee2f0 con 0x7ff2280fee60 2024-09-13T18:34:12.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.235+0000 7ff2267fc640 1 --2- 172.21.15.114:0/3138633016 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff2100a1200 0x7ff2100a36c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff2281977b0 tx=0x7ff21c005f90 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:12.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.238+0000 7ff22c8b5640 1 -- 172.21.15.114:0/3138633016 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7ff2180b73f0 con 0x7ff2280fee60 2024-09-13T18:34:12.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.406+0000 7ff22d8b7640 1 -- 172.21.15.114:0/3138633016 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff2280ff8d0 con 0x7ff2280fee60 2024-09-13T18:34:12.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.407+0000 7ff22c8b5640 1 -- 172.21.15.114:0/3138633016 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v105) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff21804a020 con 0x7ff2280fee60 2024-09-13T18:34:12.409 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 1 2024-09-13T18:34:12.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.409+0000 7ff22d8b7640 1 -- 172.21.15.114:0/3138633016 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff2100a1200 msgr2=0x7ff2100a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:12.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.409+0000 7ff22d8b7640 1 --2- 172.21.15.114:0/3138633016 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff2100a1200 0x7ff2100a36c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff2281977b0 tx=0x7ff21c005f90 comp rx=0 tx=0).stop 2024-09-13T18:34:12.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.409+0000 7ff22d8b7640 1 -- 172.21.15.114:0/3138633016 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff2280fee60 msgr2=0x7ff228196210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:12.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.409+0000 7ff22d8b7640 1 --2- 172.21.15.114:0/3138633016 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff2280fee60 0x7ff228196210 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7ff21802a940 tx=0x7ff218005610 comp rx=0 tx=0).stop 2024-09-13T18:34:12.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.409+0000 7ff22d8b7640 1 -- 172.21.15.114:0/3138633016 shutdown_connections 2024-09-13T18:34:12.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.409+0000 7ff22d8b7640 1 --2- 172.21.15.114:0/3138633016 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff2100a1200 0x7ff2100a36c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:12.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.409+0000 7ff22d8b7640 1 --2- 172.21.15.114:0/3138633016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff22819ab40 0x7ff22819aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:12.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.409+0000 7ff22d8b7640 1 --2- 172.21.15.114:0/3138633016 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff228103fb0 0x7ff228196750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:12.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.409+0000 7ff22d8b7640 1 --2- 172.21.15.114:0/3138633016 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff2280fee60 0x7ff228196210 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:12.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.409+0000 7ff22d8b7640 1 -- 172.21.15.114:0/3138633016 >> 172.21.15.114:0/3138633016 conn(0x7ff228075c60 msgr2=0x7ff228067b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:12.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.409+0000 7ff22d8b7640 1 -- 172.21.15.114:0/3138633016 shutdown_connections 2024-09-13T18:34:12.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.409+0000 7ff22d8b7640 1 -- 172.21.15.114:0/3138633016 wait complete. 2024-09-13T18:34:12.422 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T18:34:12.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.522+0000 7f288639c640 1 -- 172.21.15.114:0/2131288137 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f28800ff810 msgr2=0x7f2880103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:12.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.522+0000 7f288639c640 1 --2- 172.21.15.114:0/2131288137 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f28800ff810 0x7f2880103850 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f2874009930 tx=0x7f287402a420 comp rx=0 tx=0).stop 2024-09-13T18:34:12.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.522+0000 7f288639c640 1 -- 172.21.15.114:0/2131288137 shutdown_connections 2024-09-13T18:34:12.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.522+0000 7f288639c640 1 --2- 172.21.15.114:0/2131288137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2880103fb0 0x7f288010bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:12.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.522+0000 7f288639c640 1 --2- 172.21.15.114:0/2131288137 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f28800ff810 0x7f2880103850 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:12.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.522+0000 7f288639c640 1 --2- 172.21.15.114:0/2131288137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28800fee60 0x7f28800ff240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:12.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.522+0000 7f288639c640 1 -- 172.21.15.114:0/2131288137 >> 172.21.15.114:0/2131288137 conn(0x7f2880075c60 msgr2=0x7f28800fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:12.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.522+0000 7f288639c640 1 -- 172.21.15.114:0/2131288137 shutdown_connections 2024-09-13T18:34:12.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.522+0000 7f288639c640 1 -- 172.21.15.114:0/2131288137 wait complete. 2024-09-13T18:34:12.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.522+0000 7f288639c640 1 Processor -- start 2024-09-13T18:34:12.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.522+0000 7f288639c640 1 -- start start 2024-09-13T18:34:12.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.523+0000 7f288639c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f28800fee60 0x7f2880196ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:12.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.523+0000 7f288639c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2880103fb0 0x7f2880197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:12.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.523+0000 7f288639c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2880197a70 0x7f28801a2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:12.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.523+0000 7f288639c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28801a2710 con 0x7f28800fee60 2024-09-13T18:34:12.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.523+0000 7f288639c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28801a2880 con 0x7f2880197a70 2024-09-13T18:34:12.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.523+0000 7f288639c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28801a2a40 con 0x7f2880103fb0 2024-09-13T18:34:12.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.523+0000 7f287ffff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f28800fee60 0x7f2880196ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:12.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.523+0000 7f287ffff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f28800fee60 0x7f2880196ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55264/0 (socket says 172.21.15.114:55264) 2024-09-13T18:34:12.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.523+0000 7f287ffff640 1 -- 172.21.15.114:0/2392849303 learned_addr learned my addr 172.21.15.114:0/2392849303 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:34:12.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.523+0000 7f287f7fe640 1 --2- 172.21.15.114:0/2392849303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2880103fb0 0x7f2880197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:12.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.523+0000 7f287ffff640 1 -- 172.21.15.114:0/2392849303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2880103fb0 msgr2=0x7f2880197530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:12.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.523+0000 7f287ffff640 1 --2- 172.21.15.114:0/2392849303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2880103fb0 0x7f2880197530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:12.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.523+0000 7f2884939640 1 --2- 172.21.15.114:0/2392849303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2880197a70 0x7f28801a2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:12.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.523+0000 7f287ffff640 1 -- 172.21.15.114:0/2392849303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2880197a70 msgr2=0x7f28801a2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:12.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.523+0000 7f287ffff640 1 --2- 172.21.15.114:0/2392849303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2880197a70 0x7f28801a2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:12.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.523+0000 7f287ffff640 1 -- 172.21.15.114:0/2392849303 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2874009560 con 0x7f28800fee60 2024-09-13T18:34:12.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.523+0000 7f287f7fe640 1 --2- 172.21.15.114:0/2392849303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2880103fb0 0x7f2880197530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:34:12.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.523+0000 7f287ffff640 1 --2- 172.21.15.114:0/2392849303 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f28800fee60 0x7f2880196ff0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f287000d950 tx=0x7f287000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:12.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.523+0000 7f287d7fa640 1 -- 172.21.15.114:0/2392849303 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2870009a30 con 0x7f28800fee60 2024-09-13T18:34:12.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.523+0000 7f287d7fa640 1 -- 172.21.15.114:0/2392849303 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f28700044b0 con 0x7f28800fee60 2024-09-13T18:34:12.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.523+0000 7f288639c640 1 -- 172.21.15.114:0/2392849303 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28801a2d20 con 0x7f28800fee60 2024-09-13T18:34:12.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.523+0000 7f287d7fa640 1 -- 172.21.15.114:0/2392849303 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28700047a0 con 0x7f28800fee60 2024-09-13T18:34:12.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.523+0000 7f288639c640 1 -- 172.21.15.114:0/2392849303 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28801a31f0 con 0x7f28800fee60 2024-09-13T18:34:12.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.524+0000 7f288639c640 1 -- 172.21.15.114:0/2392849303 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2880100330 con 0x7f28800fee60 2024-09-13T18:34:12.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.525+0000 7f287d7fa640 1 -- 172.21.15.114:0/2392849303 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f2870004020 con 0x7f28800fee60 2024-09-13T18:34:12.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.526+0000 7f287d7fa640 1 --2- 172.21.15.114:0/2392849303 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f28680a1280 0x7f28680a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:12.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.526+0000 7f287f7fe640 1 --2- 172.21.15.114:0/2392849303 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f28680a1280 0x7f28680a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:12.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.526+0000 7f287d7fa640 1 -- 172.21.15.114:0/2392849303 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f28700ccb30 con 0x7f28800fee60 2024-09-13T18:34:12.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.526+0000 7f287f7fe640 1 --2- 172.21.15.114:0/2392849303 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f28680a1280 0x7f28680a3740 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2874009900 tx=0x7f2874005a20 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:12.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.530+0000 7f287d7fa640 1 -- 172.21.15.114:0/2392849303 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f2870095b80 con 0x7f28800fee60 2024-09-13T18:34:12.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.695+0000 7f288639c640 1 -- 172.21.15.114:0/2392849303 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f28800ff810 con 0x7f28800fee60 2024-09-13T18:34:12.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.695+0000 7f287d7fa640 1 -- 172.21.15.114:0/2392849303 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v105) v1 ==== 104+0+11 (secure 0 0 0) 0x7f28700991a0 con 0x7f28800fee60 2024-09-13T18:34:12.698 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2024-09-13T18:34:12.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.697+0000 7f288639c640 1 -- 172.21.15.114:0/2392849303 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f28680a1280 msgr2=0x7f28680a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:12.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.697+0000 7f288639c640 1 --2- 172.21.15.114:0/2392849303 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f28680a1280 0x7f28680a3740 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2874009900 tx=0x7f2874005a20 comp rx=0 tx=0).stop 2024-09-13T18:34:12.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.697+0000 7f288639c640 1 -- 172.21.15.114:0/2392849303 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f28800fee60 msgr2=0x7f2880196ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:12.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.697+0000 7f288639c640 1 --2- 172.21.15.114:0/2392849303 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f28800fee60 0x7f2880196ff0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f287000d950 tx=0x7f287000de10 comp rx=0 tx=0).stop 2024-09-13T18:34:12.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.697+0000 7f288639c640 1 -- 172.21.15.114:0/2392849303 shutdown_connections 2024-09-13T18:34:12.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.697+0000 7f288639c640 1 --2- 172.21.15.114:0/2392849303 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f28680a1280 0x7f28680a3740 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:12.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.697+0000 7f288639c640 1 --2- 172.21.15.114:0/2392849303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2880197a70 0x7f28801a2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:12.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.697+0000 7f288639c640 1 --2- 172.21.15.114:0/2392849303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2880103fb0 0x7f2880197530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:12.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.697+0000 7f288639c640 1 --2- 172.21.15.114:0/2392849303 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f28800fee60 0x7f2880196ff0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:12.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.697+0000 7f288639c640 1 -- 172.21.15.114:0/2392849303 >> 172.21.15.114:0/2392849303 conn(0x7f2880075c60 msgr2=0x7f2880102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:12.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.697+0000 7f288639c640 1 -- 172.21.15.114:0/2392849303 shutdown_connections 2024-09-13T18:34:12.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.697+0000 7f288639c640 1 -- 172.21.15.114:0/2392849303 wait complete. 2024-09-13T18:34:12.709 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:12 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/3908739727' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:34:12.709 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:12 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/3138633016' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T18:34:12.710 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T18:34:12.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.810+0000 7f03e7765640 1 -- 172.21.15.114:0/3572862115 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f03e0101800 msgr2=0x7f03e0109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:12.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.810+0000 7f03e7765640 1 --2- 172.21.15.114:0/3572862115 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f03e0101800 0x7f03e0109ce0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f03d0009900 tx=0x7f03d002a410 comp rx=0 tx=0).stop 2024-09-13T18:34:12.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.810+0000 7f03e7765640 1 -- 172.21.15.114:0/3572862115 shutdown_connections 2024-09-13T18:34:12.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.810+0000 7f03e7765640 1 --2- 172.21.15.114:0/3572862115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03e010a350 0x7f03e010c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:12.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.810+0000 7f03e7765640 1 --2- 172.21.15.114:0/3572862115 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f03e0101800 0x7f03e0109ce0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:12.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.810+0000 7f03e7765640 1 --2- 172.21.15.114:0/3572862115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03e0100e50 0x7f03e0101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:12.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.810+0000 7f03e7765640 1 -- 172.21.15.114:0/3572862115 >> 172.21.15.114:0/3572862115 conn(0x7f03e00fa930 msgr2=0x7f03e00fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:12.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.810+0000 7f03e7765640 1 -- 172.21.15.114:0/3572862115 shutdown_connections 2024-09-13T18:34:12.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.810+0000 7f03e7765640 1 -- 172.21.15.114:0/3572862115 wait complete. 2024-09-13T18:34:12.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.811+0000 7f03e7765640 1 Processor -- start 2024-09-13T18:34:12.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.811+0000 7f03e7765640 1 -- start start 2024-09-13T18:34:12.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.811+0000 7f03e7765640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03e0100e50 0x7f03e01961b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:12.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.811+0000 7f03e7765640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03e0101800 0x7f03e01966f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:12.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.811+0000 7f03e7765640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f03e010a350 0x7f03e019aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:12.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.811+0000 7f03e7765640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e019b0e0 con 0x7f03e010a350 2024-09-13T18:34:12.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.811+0000 7f03e7765640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e019b250 con 0x7f03e0100e50 2024-09-13T18:34:12.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.811+0000 7f03e7765640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e019b410 con 0x7f03e0101800 2024-09-13T18:34:12.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.811+0000 7f03e5d02640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f03e010a350 0x7f03e019aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:12.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.811+0000 7f03e5d02640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f03e010a350 0x7f03e019aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55288/0 (socket says 172.21.15.114:55288) 2024-09-13T18:34:12.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.811+0000 7f03e5d02640 1 -- 172.21.15.114:0/111222938 learned_addr learned my addr 172.21.15.114:0/111222938 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:34:12.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.811+0000 7f03e4d00640 1 --2- 172.21.15.114:0/111222938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03e0101800 0x7f03e01966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:12.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.811+0000 7f03e5501640 1 --2- 172.21.15.114:0/111222938 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03e0100e50 0x7f03e01961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:12.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.811+0000 7f03e5d02640 1 -- 172.21.15.114:0/111222938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03e0101800 msgr2=0x7f03e01966f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:12.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.811+0000 7f03e5d02640 1 --2- 172.21.15.114:0/111222938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03e0101800 0x7f03e01966f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:12.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.811+0000 7f03e5d02640 1 -- 172.21.15.114:0/111222938 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03e0100e50 msgr2=0x7f03e01961b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:12.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.811+0000 7f03e5d02640 1 --2- 172.21.15.114:0/111222938 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03e0100e50 0x7f03e01961b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:12.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.811+0000 7f03e5d02640 1 -- 172.21.15.114:0/111222938 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d0009560 con 0x7f03e010a350 2024-09-13T18:34:12.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.811+0000 7f03e5501640 1 --2- 172.21.15.114:0/111222938 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03e0100e50 0x7f03e01961b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:34:12.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.812+0000 7f03e5d02640 1 --2- 172.21.15.114:0/111222938 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f03e010a350 0x7f03e019aae0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f03d8009fd0 tx=0x7f03d800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:12.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.812+0000 7f03d67fc640 1 -- 172.21.15.114:0/111222938 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d800bea0 con 0x7f03e010a350 2024-09-13T18:34:12.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.812+0000 7f03e7765640 1 -- 172.21.15.114:0/111222938 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03e019b6f0 con 0x7f03e010a350 2024-09-13T18:34:12.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.812+0000 7f03d67fc640 1 -- 172.21.15.114:0/111222938 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f03d8009940 con 0x7f03e010a350 2024-09-13T18:34:12.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.812+0000 7f03d67fc640 1 -- 172.21.15.114:0/111222938 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d8010480 con 0x7f03e010a350 2024-09-13T18:34:12.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.812+0000 7f03e7765640 1 -- 172.21.15.114:0/111222938 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03e01a2f70 con 0x7f03e010a350 2024-09-13T18:34:12.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.813+0000 7f03e7765640 1 -- 172.21.15.114:0/111222938 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03e0068e50 con 0x7f03e010a350 2024-09-13T18:34:12.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.814+0000 7f03d67fc640 1 -- 172.21.15.114:0/111222938 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f03d8004a40 con 0x7f03e010a350 2024-09-13T18:34:12.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.814+0000 7f03d67fc640 1 --2- 172.21.15.114:0/111222938 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f03cc0a1200 0x7f03cc0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:12.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.815+0000 7f03e5501640 1 --2- 172.21.15.114:0/111222938 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f03cc0a1200 0x7f03cc0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:12.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.815+0000 7f03d67fc640 1 -- 172.21.15.114:0/111222938 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f03d80ccd50 con 0x7f03e010a350 2024-09-13T18:34:12.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.815+0000 7f03e5501640 1 --2- 172.21.15.114:0/111222938 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f03cc0a1200 0x7f03cc0a36c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f03dc009c20 tx=0x7f03dc005f90 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:12.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.818+0000 7f03d67fc640 1 -- 172.21.15.114:0/111222938 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f03d8095da0 con 0x7f03e010a350 2024-09-13T18:34:12.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:12 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/3908739727' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:34:12.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:12 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/3138633016' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T18:34:12.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.979+0000 7f03e7765640 1 -- 172.21.15.114:0/111222938 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f03e0197500 con 0x7f03e010a350 2024-09-13T18:34:12.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.980+0000 7f03d67fc640 1 -- 172.21.15.114:0/111222938 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v105) v1 ==== 100+0+11 (secure 0 0 0) 0x7f03d8017020 con 0x7f03e010a350 2024-09-13T18:34:12.982 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 64 2024-09-13T18:34:12.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.982+0000 7f03e7765640 1 -- 172.21.15.114:0/111222938 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f03cc0a1200 msgr2=0x7f03cc0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:12.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.982+0000 7f03e7765640 1 --2- 172.21.15.114:0/111222938 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f03cc0a1200 0x7f03cc0a36c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f03dc009c20 tx=0x7f03dc005f90 comp rx=0 tx=0).stop 2024-09-13T18:34:12.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.982+0000 7f03e7765640 1 -- 172.21.15.114:0/111222938 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f03e010a350 msgr2=0x7f03e019aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:12.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.982+0000 7f03e7765640 1 --2- 172.21.15.114:0/111222938 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f03e010a350 0x7f03e019aae0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f03d8009fd0 tx=0x7f03d800b260 comp rx=0 tx=0).stop 2024-09-13T18:34:12.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.982+0000 7f03e7765640 1 -- 172.21.15.114:0/111222938 shutdown_connections 2024-09-13T18:34:12.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.982+0000 7f03e7765640 1 --2- 172.21.15.114:0/111222938 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f03cc0a1200 0x7f03cc0a36c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:12.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.982+0000 7f03e7765640 1 --2- 172.21.15.114:0/111222938 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f03e010a350 0x7f03e019aae0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:12.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.982+0000 7f03e7765640 1 --2- 172.21.15.114:0/111222938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03e0101800 0x7f03e01966f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:12.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.982+0000 7f03e7765640 1 --2- 172.21.15.114:0/111222938 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03e0100e50 0x7f03e01961b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:12.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.982+0000 7f03e7765640 1 -- 172.21.15.114:0/111222938 >> 172.21.15.114:0/111222938 conn(0x7f03e00fa930 msgr2=0x7f03e00fc290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:12.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.982+0000 7f03e7765640 1 -- 172.21.15.114:0/111222938 shutdown_connections 2024-09-13T18:34:12.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:12.982+0000 7f03e7765640 1 -- 172.21.15.114:0/111222938 wait complete. 2024-09-13T18:34:12.995 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T18:34:13.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.093+0000 7ff543419640 1 -- 172.21.15.114:0/2686791789 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff53c100f60 msgr2=0x7ff53c101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:13.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.093+0000 7ff543419640 1 --2- 172.21.15.114:0/2686791789 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff53c100f60 0x7ff53c101340 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7ff538009900 tx=0x7ff53802a480 comp rx=0 tx=0).stop 2024-09-13T18:34:13.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.093+0000 7ff543419640 1 -- 172.21.15.114:0/2686791789 shutdown_connections 2024-09-13T18:34:13.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.093+0000 7ff543419640 1 --2- 172.21.15.114:0/2686791789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff53c10a2f0 0x7ff53c10c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:13.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.093+0000 7ff543419640 1 --2- 172.21.15.114:0/2686791789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff53c101880 0x7ff53c109d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:13.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.093+0000 7ff543419640 1 --2- 172.21.15.114:0/2686791789 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff53c100f60 0x7ff53c101340 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:13.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.093+0000 7ff543419640 1 -- 172.21.15.114:0/2686791789 >> 172.21.15.114:0/2686791789 conn(0x7ff53c0fa890 msgr2=0x7ff53c0fccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:13.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.093+0000 7ff543419640 1 -- 172.21.15.114:0/2686791789 shutdown_connections 2024-09-13T18:34:13.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.093+0000 7ff543419640 1 -- 172.21.15.114:0/2686791789 wait complete. 2024-09-13T18:34:13.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.094+0000 7ff543419640 1 Processor -- start 2024-09-13T18:34:13.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.094+0000 7ff543419640 1 -- start start 2024-09-13T18:34:13.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.094+0000 7ff543419640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff53c100f60 0x7ff53c0febb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:13.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.094+0000 7ff543419640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff53c101880 0x7ff53c0ff0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:13.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.094+0000 7ff543419640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff53c10a2f0 0x7ff53c0ff630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:13.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.094+0000 7ff543419640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53c1a4540 con 0x7ff53c100f60 2024-09-13T18:34:13.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.094+0000 7ff543419640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53c1a46b0 con 0x7ff53c10a2f0 2024-09-13T18:34:13.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.094+0000 7ff543419640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53c1a4870 con 0x7ff53c101880 2024-09-13T18:34:13.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.094+0000 7ff5411b5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff53c100f60 0x7ff53c0febb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:13.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.094+0000 7ff5411b5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff53c100f60 0x7ff53c0febb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55314/0 (socket says 172.21.15.114:55314) 2024-09-13T18:34:13.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.094+0000 7ff5411b5640 1 -- 172.21.15.114:0/2333576398 learned_addr learned my addr 172.21.15.114:0/2333576398 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:34:13.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.094+0000 7ff5419b6640 1 --2- 172.21.15.114:0/2333576398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff53c10a2f0 0x7ff53c0ff630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:13.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.095+0000 7ff5409b4640 1 --2- 172.21.15.114:0/2333576398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff53c101880 0x7ff53c0ff0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:13.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.095+0000 7ff5411b5640 1 -- 172.21.15.114:0/2333576398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff53c101880 msgr2=0x7ff53c0ff0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:13.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.095+0000 7ff5411b5640 1 --2- 172.21.15.114:0/2333576398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff53c101880 0x7ff53c0ff0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:13.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.095+0000 7ff5411b5640 1 -- 172.21.15.114:0/2333576398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff53c10a2f0 msgr2=0x7ff53c0ff630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:13.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.095+0000 7ff5411b5640 1 --2- 172.21.15.114:0/2333576398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff53c10a2f0 0x7ff53c0ff630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:13.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.095+0000 7ff5411b5640 1 -- 172.21.15.114:0/2333576398 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff538009560 con 0x7ff53c100f60 2024-09-13T18:34:13.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.095+0000 7ff5409b4640 1 --2- 172.21.15.114:0/2333576398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff53c101880 0x7ff53c0ff0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:34:13.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.095+0000 7ff5419b6640 1 --2- 172.21.15.114:0/2333576398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff53c10a2f0 0x7ff53c0ff630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:34:13.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.095+0000 7ff5411b5640 1 --2- 172.21.15.114:0/2333576398 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff53c100f60 0x7ff53c0febb0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7ff5380057c0 tx=0x7ff538032be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:13.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.095+0000 7ff5327fc640 1 -- 172.21.15.114:0/2333576398 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff538005000 con 0x7ff53c100f60 2024-09-13T18:34:13.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.095+0000 7ff543419640 1 -- 172.21.15.114:0/2333576398 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff53c1a4af0 con 0x7ff53c100f60 2024-09-13T18:34:13.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.095+0000 7ff5327fc640 1 -- 172.21.15.114:0/2333576398 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff5380046c0 con 0x7ff53c100f60 2024-09-13T18:34:13.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.095+0000 7ff5327fc640 1 -- 172.21.15.114:0/2333576398 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff538003850 con 0x7ff53c100f60 2024-09-13T18:34:13.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.095+0000 7ff543419640 1 -- 172.21.15.114:0/2333576398 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff53c1a4f70 con 0x7ff53c100f60 2024-09-13T18:34:13.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.096+0000 7ff543419640 1 -- 172.21.15.114:0/2333576398 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff53c068db0 con 0x7ff53c100f60 2024-09-13T18:34:13.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.097+0000 7ff5327fc640 1 -- 172.21.15.114:0/2333576398 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7ff538005160 con 0x7ff53c100f60 2024-09-13T18:34:13.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.098+0000 7ff5327fc640 1 --2- 172.21.15.114:0/2333576398 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff5280a1200 0x7ff5280a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:13.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.098+0000 7ff5409b4640 1 --2- 172.21.15.114:0/2333576398 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff5280a1200 0x7ff5280a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:13.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.098+0000 7ff5327fc640 1 -- 172.21.15.114:0/2333576398 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9541+0+0 (secure 0 0 0) 0x7ff5380ee390 con 0x7ff53c100f60 2024-09-13T18:34:13.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.098+0000 7ff5409b4640 1 --2- 172.21.15.114:0/2333576398 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff5280a1200 0x7ff5280a36c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff53c100330 tx=0x7ff52c005f90 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:13.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.101+0000 7ff5327fc640 1 -- 172.21.15.114:0/2333576398 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7ff5380b7380 con 0x7ff53c100f60 2024-09-13T18:34:13.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:12 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/3908739727' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:34:13.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:12 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/3138633016' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T18:34:13.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.263+0000 7ff543419640 1 -- 172.21.15.114:0/2333576398 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff53c1053c0 con 0x7ff53c100f60 2024-09-13T18:34:13.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.263+0000 7ff5327fc640 1 -- 172.21.15.114:0/2333576398 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v105) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff53804a020 con 0x7ff53c100f60 2024-09-13T18:34:13.265 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2024-09-13T18:34:13.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.265+0000 7ff543419640 1 -- 172.21.15.114:0/2333576398 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff5280a1200 msgr2=0x7ff5280a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:13.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.265+0000 7ff543419640 1 --2- 172.21.15.114:0/2333576398 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff5280a1200 0x7ff5280a36c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff53c100330 tx=0x7ff52c005f90 comp rx=0 tx=0).stop 2024-09-13T18:34:13.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.265+0000 7ff543419640 1 -- 172.21.15.114:0/2333576398 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff53c100f60 msgr2=0x7ff53c0febb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:13.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.265+0000 7ff543419640 1 --2- 172.21.15.114:0/2333576398 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff53c100f60 0x7ff53c0febb0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7ff5380057c0 tx=0x7ff538032be0 comp rx=0 tx=0).stop 2024-09-13T18:34:13.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.265+0000 7ff543419640 1 -- 172.21.15.114:0/2333576398 shutdown_connections 2024-09-13T18:34:13.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.265+0000 7ff543419640 1 --2- 172.21.15.114:0/2333576398 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff5280a1200 0x7ff5280a36c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:13.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.265+0000 7ff543419640 1 --2- 172.21.15.114:0/2333576398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff53c10a2f0 0x7ff53c0ff630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:13.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.265+0000 7ff543419640 1 --2- 172.21.15.114:0/2333576398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff53c101880 0x7ff53c0ff0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:13.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.265+0000 7ff543419640 1 --2- 172.21.15.114:0/2333576398 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff53c100f60 0x7ff53c0febb0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:13.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.265+0000 7ff543419640 1 -- 172.21.15.114:0/2333576398 >> 172.21.15.114:0/2333576398 conn(0x7ff53c0fa890 msgr2=0x7ff53c0fc260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:13.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.265+0000 7ff543419640 1 -- 172.21.15.114:0/2333576398 shutdown_connections 2024-09-13T18:34:13.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.265+0000 7ff543419640 1 -- 172.21.15.114:0/2333576398 wait complete. 2024-09-13T18:34:13.278 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-13T18:34:13.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.375+0000 7f7d2d5c7640 1 -- 172.21.15.114:0/454236401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7d28101020 msgr2=0x7f7d28101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:13.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.375+0000 7f7d2d5c7640 1 --2- 172.21.15.114:0/454236401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7d28101020 0x7f7d28101400 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f7d18009900 tx=0x7f7d1802a480 comp rx=0 tx=0).stop 2024-09-13T18:34:13.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.375+0000 7f7d2d5c7640 1 -- 172.21.15.114:0/454236401 shutdown_connections 2024-09-13T18:34:13.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.375+0000 7f7d2d5c7640 1 --2- 172.21.15.114:0/454236401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d2810a3f0 0x7f7d2810c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:13.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.375+0000 7f7d2d5c7640 1 --2- 172.21.15.114:0/454236401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d28101940 0x7f7d28109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:13.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.375+0000 7f7d2d5c7640 1 --2- 172.21.15.114:0/454236401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7d28101020 0x7f7d28101400 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:13.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.375+0000 7f7d2d5c7640 1 -- 172.21.15.114:0/454236401 >> 172.21.15.114:0/454236401 conn(0x7f7d280fa970 msgr2=0x7f7d280fcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:13.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.375+0000 7f7d2d5c7640 1 -- 172.21.15.114:0/454236401 shutdown_connections 2024-09-13T18:34:13.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.375+0000 7f7d2d5c7640 1 -- 172.21.15.114:0/454236401 wait complete. 2024-09-13T18:34:13.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.376+0000 7f7d2d5c7640 1 Processor -- start 2024-09-13T18:34:13.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.376+0000 7f7d2d5c7640 1 -- start start 2024-09-13T18:34:13.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.376+0000 7f7d2d5c7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d28101020 0x7f7d28196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:13.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.376+0000 7f7d2d5c7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7d28101940 0x7f7d281967b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:13.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.376+0000 7f7d2d5c7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d2810a3f0 0x7f7d2819aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:13.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.376+0000 7f7d2d5c7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2819b1a0 con 0x7f7d28101940 2024-09-13T18:34:13.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.376+0000 7f7d2d5c7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2819b310 con 0x7f7d28101020 2024-09-13T18:34:13.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.376+0000 7f7d2d5c7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2819b4d0 con 0x7f7d2810a3f0 2024-09-13T18:34:13.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.376+0000 7f7d1ffff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7d28101940 0x7f7d281967b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:13.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.376+0000 7f7d1ffff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7d28101940 0x7f7d281967b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:56582/0 (socket says 172.21.15.114:56582) 2024-09-13T18:34:13.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.376+0000 7f7d1ffff640 1 -- 172.21.15.114:0/1780221248 learned_addr learned my addr 172.21.15.114:0/1780221248 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:34:13.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.376+0000 7f7d26ffd640 1 --2- 172.21.15.114:0/1780221248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d28101020 0x7f7d28196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:13.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.376+0000 7f7d277fe640 1 --2- 172.21.15.114:0/1780221248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d2810a3f0 0x7f7d2819aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:13.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.376+0000 7f7d1ffff640 1 -- 172.21.15.114:0/1780221248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d2810a3f0 msgr2=0x7f7d2819aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:13.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.376+0000 7f7d1ffff640 1 --2- 172.21.15.114:0/1780221248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d2810a3f0 0x7f7d2819aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:13.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.376+0000 7f7d1ffff640 1 -- 172.21.15.114:0/1780221248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d28101020 msgr2=0x7f7d28196270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:13.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.376+0000 7f7d1ffff640 1 --2- 172.21.15.114:0/1780221248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d28101020 0x7f7d28196270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:13.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.376+0000 7f7d1ffff640 1 -- 172.21.15.114:0/1780221248 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d18009560 con 0x7f7d28101940 2024-09-13T18:34:13.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.376+0000 7f7d277fe640 1 --2- 172.21.15.114:0/1780221248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d2810a3f0 0x7f7d2819aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:34:13.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.376+0000 7f7d26ffd640 1 --2- 172.21.15.114:0/1780221248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d28101020 0x7f7d28196270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:34:13.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.376+0000 7f7d1ffff640 1 --2- 172.21.15.114:0/1780221248 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7d28101940 0x7f7d281967b0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f7d1000d950 tx=0x7f7d1000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:13.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.377+0000 7f7d24ff9640 1 -- 172.21.15.114:0/1780221248 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d10009c30 con 0x7f7d28101940 2024-09-13T18:34:13.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.377+0000 7f7d24ff9640 1 -- 172.21.15.114:0/1780221248 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7d10009d90 con 0x7f7d28101940 2024-09-13T18:34:13.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.377+0000 7f7d2d5c7640 1 -- 172.21.15.114:0/1780221248 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d2819b7b0 con 0x7f7d28101940 2024-09-13T18:34:13.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.377+0000 7f7d24ff9640 1 -- 172.21.15.114:0/1780221248 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1000f620 con 0x7f7d28101940 2024-09-13T18:34:13.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.377+0000 7f7d2d5c7640 1 -- 172.21.15.114:0/1780221248 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d281a3030 con 0x7f7d28101940 2024-09-13T18:34:13.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.378+0000 7f7d2d5c7640 1 -- 172.21.15.114:0/1780221248 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d28068dd0 con 0x7f7d28101940 2024-09-13T18:34:13.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.379+0000 7f7d24ff9640 1 -- 172.21.15.114:0/1780221248 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f7d100109f0 con 0x7f7d28101940 2024-09-13T18:34:13.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.380+0000 7f7d24ff9640 1 --2- 172.21.15.114:0/1780221248 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7d140a1280 0x7f7d140a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:13.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.380+0000 7f7d26ffd640 1 --2- 172.21.15.114:0/1780221248 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7d140a1280 0x7f7d140a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:13.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.380+0000 7f7d24ff9640 1 -- 172.21.15.114:0/1780221248 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f7d100ccd10 con 0x7f7d28101940 2024-09-13T18:34:13.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.380+0000 7f7d26ffd640 1 --2- 172.21.15.114:0/1780221248 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7d140a1280 0x7f7d140a3740 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7d18000c00 tx=0x7f7d18005a20 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:13.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.383+0000 7f7d24ff9640 1 -- 172.21.15.114:0/1780221248 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f7d10095de0 con 0x7f7d28101940 2024-09-13T18:34:13.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.590+0000 7f7d2d5c7640 1 -- 172.21.15.114:0/1780221248 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f7d28101400 con 0x7f7d28101940 2024-09-13T18:34:13.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.598+0000 7f7d24ff9640 1 -- 172.21.15.114:0/1780221248 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) v1 ==== 81+0+733951 (secure 0 0 0) 0x7f7d10020020 con 0x7f7d28101940 2024-09-13T18:34:13.610 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:34:13.610 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:34:13.611 INFO:teuthology.orchestra.run.smithi114.stdout: "epoch": 26, 2024-09-13T18:34:13.611 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.611 INFO:teuthology.orchestra.run.smithi114.stdout: "active_gid": 15555, 2024-09-13T18:34:13.611 INFO:teuthology.orchestra.run.smithi114.stdout: "active_name": "x", 2024-09-13T18:34:13.611 INFO:teuthology.orchestra.run.smithi114.stdout: "active_addrs": { 2024-09-13T18:34:13.611 INFO:teuthology.orchestra.run.smithi114.stdout: "addrvec": [ 2024-09-13T18:34:13.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:13.611 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "v2", 2024-09-13T18:34:13.611 INFO:teuthology.orchestra.run.smithi114.stdout: "addr": "172.21.15.114:6800", 2024-09-13T18:34:13.611 INFO:teuthology.orchestra.run.smithi114.stdout: "nonce": 2440721366 2024-09-13T18:34:13.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:13.611 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "v1", 2024-09-13T18:34:13.611 INFO:teuthology.orchestra.run.smithi114.stdout: "addr": "172.21.15.114:6801", 2024-09-13T18:34:13.612 INFO:teuthology.orchestra.run.smithi114.stdout: "nonce": 2440721366 2024-09-13T18:34:13.612 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.612 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-13T18:34:13.612 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.612 INFO:teuthology.orchestra.run.smithi114.stdout: "active_addr": "172.21.15.114:6801/2440721366", 2024-09-13T18:34:13.612 INFO:teuthology.orchestra.run.smithi114.stdout: "active_change": "2024-09-13T18:22:28.057879+0000", 2024-09-13T18:34:13.612 INFO:teuthology.orchestra.run.smithi114.stdout: "active_mgr_features": 4540138322906710015, 2024-09-13T18:34:13.612 INFO:teuthology.orchestra.run.smithi114.stdout: "available": true, 2024-09-13T18:34:13.612 INFO:teuthology.orchestra.run.smithi114.stdout: "standbys": [ 2024-09-13T18:34:13.612 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:13.612 INFO:teuthology.orchestra.run.smithi114.stdout: "gid": 25076, 2024-09-13T18:34:13.612 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "y", 2024-09-13T18:34:13.612 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr_features": 4540138322906710015, 2024-09-13T18:34:13.613 INFO:teuthology.orchestra.run.smithi114.stdout: "available_modules": [ 2024-09-13T18:34:13.613 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:13.613 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "alerts", 2024-09-13T18:34:13.613 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:13.613 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:13.613 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:13.613 INFO:teuthology.orchestra.run.smithi114.stdout: "interval": { 2024-09-13T18:34:13.613 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "interval", 2024-09-13T18:34:13.613 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:13.613 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.613 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.613 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "60", 2024-09-13T18:34:13.614 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.614 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.614 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.614 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "How frequently to reexamine health status", 2024-09-13T18:34:13.614 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.614 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.614 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.614 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:13.614 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:13.614 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.614 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.615 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.615 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.615 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.615 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.615 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.615 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.615 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.615 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.615 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.615 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.615 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.615 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.615 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.616 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.616 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.616 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.616 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:13.616 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:13.616 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.616 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.616 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.616 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.616 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.616 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.616 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.617 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.617 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.617 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.617 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.617 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:13.617 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:13.617 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.617 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.617 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.617 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:13.617 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.617 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.618 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.618 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.618 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.618 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.618 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.618 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.618 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.618 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.618 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.618 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.618 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.618 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.618 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.619 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:13.619 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:13.619 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.619 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.619 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.619 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.619 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.619 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.619 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.619 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.619 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.619 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.619 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.620 INFO:teuthology.orchestra.run.smithi114.stdout: "smtp_destination": { 2024-09-13T18:34:13.620 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "smtp_destination", 2024-09-13T18:34:13.620 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.620 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.620 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.620 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.620 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.620 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.620 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.620 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Email address to send alerts to", 2024-09-13T18:34:13.620 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.620 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.620 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.621 INFO:teuthology.orchestra.run.smithi114.stdout: "smtp_from_name": { 2024-09-13T18:34:13.621 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "smtp_from_name", 2024-09-13T18:34:13.621 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.621 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.621 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.621 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "Ceph", 2024-09-13T18:34:13.621 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.621 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.621 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.621 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Email From: name", 2024-09-13T18:34:13.621 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.621 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.621 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.622 INFO:teuthology.orchestra.run.smithi114.stdout: "smtp_host": { 2024-09-13T18:34:13.622 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "smtp_host", 2024-09-13T18:34:13.622 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.622 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.622 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.622 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.622 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.622 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.622 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.622 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "SMTP server", 2024-09-13T18:34:13.622 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.622 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.622 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.623 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.623 INFO:teuthology.orchestra.run.smithi114.stdout: "smtp_password": { 2024-09-13T18:34:13.623 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "smtp_password", 2024-09-13T18:34:13.623 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.623 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.623 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.623 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.623 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.623 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.623 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.623 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Password to authenticate with", 2024-09-13T18:34:13.623 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.624 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.624 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.624 INFO:teuthology.orchestra.run.smithi114.stdout: "smtp_port": { 2024-09-13T18:34:13.624 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "smtp_port", 2024-09-13T18:34:13.624 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:13.624 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.624 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.624 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "465", 2024-09-13T18:34:13.624 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.624 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.624 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.624 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "SMTP port", 2024-09-13T18:34:13.624 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.625 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.625 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.625 INFO:teuthology.orchestra.run.smithi114.stdout: "smtp_sender": { 2024-09-13T18:34:13.625 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "smtp_sender", 2024-09-13T18:34:13.625 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.625 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.625 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.625 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.625 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.625 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.625 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.625 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "SMTP envelope sender", 2024-09-13T18:34:13.626 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.626 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.626 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.626 INFO:teuthology.orchestra.run.smithi114.stdout: "smtp_ssl": { 2024-09-13T18:34:13.626 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "smtp_ssl", 2024-09-13T18:34:13.626 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.626 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.626 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.626 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:13.626 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.626 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.626 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.627 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-13T18:34:13.627 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.627 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.627 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.627 INFO:teuthology.orchestra.run.smithi114.stdout: "smtp_user": { 2024-09-13T18:34:13.627 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "smtp_user", 2024-09-13T18:34:13.627 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.627 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.627 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.627 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.627 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.627 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.627 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.628 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "User to authenticate as", 2024-09-13T18:34:13.628 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.628 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.628 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.628 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.628 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:13.628 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "balancer", 2024-09-13T18:34:13.628 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:13.628 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:13.628 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:13.629 INFO:teuthology.orchestra.run.smithi114.stdout: "active": { 2024-09-13T18:34:13.629 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "active", 2024-09-13T18:34:13.629 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.629 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.629 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.629 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:13.629 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.629 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.629 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.629 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "automatically balance PGs across cluster", 2024-09-13T18:34:13.629 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.629 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.629 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.630 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.630 INFO:teuthology.orchestra.run.smithi114.stdout: "begin_time": { 2024-09-13T18:34:13.630 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "begin_time", 2024-09-13T18:34:13.630 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.630 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.630 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.630 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0000", 2024-09-13T18:34:13.630 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.630 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.630 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.630 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "beginning time of day to automatically balance", 2024-09-13T18:34:13.630 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-13T18:34:13.630 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.631 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.631 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.631 INFO:teuthology.orchestra.run.smithi114.stdout: "begin_weekday": { 2024-09-13T18:34:13.631 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "begin_weekday", 2024-09-13T18:34:13.631 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "uint", 2024-09-13T18:34:13.631 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.631 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.631 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-13T18:34:13.631 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "0", 2024-09-13T18:34:13.631 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "6", 2024-09-13T18:34:13.631 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.631 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-13T18:34:13.631 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-13T18:34:13.632 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.632 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.632 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.632 INFO:teuthology.orchestra.run.smithi114.stdout: "crush_compat_max_iterations": { 2024-09-13T18:34:13.632 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "crush_compat_max_iterations", 2024-09-13T18:34:13.632 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "uint", 2024-09-13T18:34:13.632 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.632 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.632 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "25", 2024-09-13T18:34:13.632 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "1", 2024-09-13T18:34:13.632 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "250", 2024-09-13T18:34:13.632 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.633 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-13T18:34:13.633 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.633 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.633 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.633 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.633 INFO:teuthology.orchestra.run.smithi114.stdout: "crush_compat_metrics": { 2024-09-13T18:34:13.633 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "crush_compat_metrics", 2024-09-13T18:34:13.633 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.633 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.633 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.633 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "pgs,objects,bytes", 2024-09-13T18:34:13.633 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.633 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.634 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.634 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-13T18:34:13.634 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-13T18:34:13.634 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.634 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.634 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.634 INFO:teuthology.orchestra.run.smithi114.stdout: "crush_compat_step": { 2024-09-13T18:34:13.634 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "crush_compat_step", 2024-09-13T18:34:13.634 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "float", 2024-09-13T18:34:13.634 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.634 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.634 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0.5", 2024-09-13T18:34:13.634 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "0.001", 2024-09-13T18:34:13.634 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "0.999", 2024-09-13T18:34:13.635 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.635 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "aggressiveness of optimization", 2024-09-13T18:34:13.635 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-13T18:34:13.635 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.635 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.635 INFO:teuthology.orchestra.run.smithi114.stdout: "end_time": { 2024-09-13T18:34:13.635 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "end_time", 2024-09-13T18:34:13.635 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.635 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.635 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.635 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "2359", 2024-09-13T18:34:13.635 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.636 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.636 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.636 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "ending time of day to automatically balance", 2024-09-13T18:34:13.636 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-13T18:34:13.636 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.636 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.636 INFO:teuthology.orchestra.run.smithi114.stdout: "end_weekday": { 2024-09-13T18:34:13.636 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "end_weekday", 2024-09-13T18:34:13.636 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "uint", 2024-09-13T18:34:13.636 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.636 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.636 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-13T18:34:13.637 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "0", 2024-09-13T18:34:13.637 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "6", 2024-09-13T18:34:13.637 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.637 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-13T18:34:13.637 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-13T18:34:13.637 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.637 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.637 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:13.637 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:13.637 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.637 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.637 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.637 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.638 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.638 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.638 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.638 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.638 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.638 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.638 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.638 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.638 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.638 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.638 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.638 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.638 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.638 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.639 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:13.639 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:13.639 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.639 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.639 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.639 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.639 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.639 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.639 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.639 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.639 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.640 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.640 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.640 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:13.640 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:13.640 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.640 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.640 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.640 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:13.640 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.640 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.640 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.640 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.640 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.641 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.641 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.641 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.641 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.641 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.641 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.641 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.641 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.641 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.641 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.641 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:13.641 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:13.641 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.642 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.642 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.642 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.642 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.642 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.642 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.642 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.642 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.642 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.642 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.642 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.642 INFO:teuthology.orchestra.run.smithi114.stdout: "min_score": { 2024-09-13T18:34:13.642 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "min_score", 2024-09-13T18:34:13.643 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "float", 2024-09-13T18:34:13.643 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.643 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.643 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-13T18:34:13.643 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.643 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.643 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.643 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-13T18:34:13.643 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.643 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.643 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.643 INFO:teuthology.orchestra.run.smithi114.stdout: "mode": { 2024-09-13T18:34:13.643 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "mode", 2024-09-13T18:34:13.644 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.644 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.644 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.644 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "upmap", 2024-09-13T18:34:13.644 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.644 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.644 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.644 INFO:teuthology.orchestra.run.smithi114.stdout: "crush-compat", 2024-09-13T18:34:13.644 INFO:teuthology.orchestra.run.smithi114.stdout: "none", 2024-09-13T18:34:13.644 INFO:teuthology.orchestra.run.smithi114.stdout: "upmap" 2024-09-13T18:34:13.644 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.644 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Balancer mode", 2024-09-13T18:34:13.644 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.645 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.645 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.645 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.645 INFO:teuthology.orchestra.run.smithi114.stdout: "pool_ids": { 2024-09-13T18:34:13.645 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "pool_ids", 2024-09-13T18:34:13.645 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.645 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.645 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.645 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.645 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.645 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.645 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.645 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-13T18:34:13.645 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.646 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.646 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.646 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.646 INFO:teuthology.orchestra.run.smithi114.stdout: "sleep_interval": { 2024-09-13T18:34:13.646 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sleep_interval", 2024-09-13T18:34:13.646 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:13.646 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.646 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.646 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "60", 2024-09-13T18:34:13.646 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.646 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.646 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.646 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-13T18:34:13.646 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.646 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.647 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.647 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.647 INFO:teuthology.orchestra.run.smithi114.stdout: "upmap_max_deviation": { 2024-09-13T18:34:13.647 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "upmap_max_deviation", 2024-09-13T18:34:13.647 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:13.647 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.647 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.647 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "5", 2024-09-13T18:34:13.647 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "1", 2024-09-13T18:34:13.647 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.647 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.647 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-13T18:34:13.647 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-13T18:34:13.647 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.648 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.648 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.648 INFO:teuthology.orchestra.run.smithi114.stdout: "upmap_max_optimizations": { 2024-09-13T18:34:13.648 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "upmap_max_optimizations", 2024-09-13T18:34:13.648 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "uint", 2024-09-13T18:34:13.648 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.648 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.648 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "10", 2024-09-13T18:34:13.648 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.648 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.648 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.648 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-13T18:34:13.648 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.649 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.649 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.649 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.649 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.649 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.649 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:13.649 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "cephadm", 2024-09-13T18:34:13.649 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:13.649 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:13.649 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:13.649 INFO:teuthology.orchestra.run.smithi114.stdout: "agent_down_multiplier": { 2024-09-13T18:34:13.649 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "agent_down_multiplier", 2024-09-13T18:34:13.649 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "float", 2024-09-13T18:34:13.649 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.650 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.650 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "3.0", 2024-09-13T18:34:13.650 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.650 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.650 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.650 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-13T18:34:13.650 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.650 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.650 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.650 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.650 INFO:teuthology.orchestra.run.smithi114.stdout: "agent_refresh_rate": { 2024-09-13T18:34:13.650 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "agent_refresh_rate", 2024-09-13T18:34:13.650 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:13.651 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.651 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.651 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "20", 2024-09-13T18:34:13.651 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.651 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.651 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.651 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-13T18:34:13.651 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.651 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.651 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.651 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.651 INFO:teuthology.orchestra.run.smithi114.stdout: "agent_starting_port": { 2024-09-13T18:34:13.651 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "agent_starting_port", 2024-09-13T18:34:13.651 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:13.651 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.652 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.652 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "4721", 2024-09-13T18:34:13.652 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.652 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.652 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.652 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-13T18:34:13.652 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.652 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.652 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.652 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.652 INFO:teuthology.orchestra.run.smithi114.stdout: "allow_ptrace": { 2024-09-13T18:34:13.653 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "allow_ptrace", 2024-09-13T18:34:13.653 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.653 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.653 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.653 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.653 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.653 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.653 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.653 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-13T18:34:13.653 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-13T18:34:13.653 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.653 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.653 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.653 INFO:teuthology.orchestra.run.smithi114.stdout: "autotune_interval": { 2024-09-13T18:34:13.654 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "autotune_interval", 2024-09-13T18:34:13.654 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:13.654 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.654 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.654 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "600", 2024-09-13T18:34:13.654 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.654 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.654 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.654 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-13T18:34:13.654 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.654 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.654 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.654 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.654 INFO:teuthology.orchestra.run.smithi114.stdout: "autotune_memory_target_ratio": { 2024-09-13T18:34:13.655 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "autotune_memory_target_ratio", 2024-09-13T18:34:13.655 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "float", 2024-09-13T18:34:13.655 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.655 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.655 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0.7", 2024-09-13T18:34:13.655 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.655 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.655 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.655 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-13T18:34:13.655 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.655 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.655 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.656 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.656 INFO:teuthology.orchestra.run.smithi114.stdout: "cgroups_split": { 2024-09-13T18:34:13.656 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "cgroups_split", 2024-09-13T18:34:13.656 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.656 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.656 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.656 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:13.656 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.656 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.656 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.656 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-13T18:34:13.656 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.657 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.657 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.657 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.657 INFO:teuthology.orchestra.run.smithi114.stdout: "config_checks_enabled": { 2024-09-13T18:34:13.657 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "config_checks_enabled", 2024-09-13T18:34:13.657 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.657 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.657 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.657 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.657 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.657 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.657 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.657 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-13T18:34:13.657 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.658 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.658 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.658 INFO:teuthology.orchestra.run.smithi114.stdout: "config_dashboard": { 2024-09-13T18:34:13.658 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "config_dashboard", 2024-09-13T18:34:13.658 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.658 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.658 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.658 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:13.658 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.658 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.658 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.658 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-13T18:34:13.659 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.659 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.659 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.659 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.659 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_alertmanager": { 2024-09-13T18:34:13.659 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_alertmanager", 2024-09-13T18:34:13.659 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.659 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.659 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.659 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-13T18:34:13.659 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.659 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.659 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.659 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Prometheus container image", 2024-09-13T18:34:13.660 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.660 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.660 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.660 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.660 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_base": { 2024-09-13T18:34:13.660 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_base", 2024-09-13T18:34:13.660 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.660 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.660 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.660 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-13T18:34:13.660 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.660 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.660 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.661 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Container image name, without the tag", 2024-09-13T18:34:13.661 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.661 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.661 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.661 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_elasticsearch": { 2024-09-13T18:34:13.661 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_elasticsearch", 2024-09-13T18:34:13.661 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.661 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.662 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.662 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-13T18:34:13.662 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.662 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.662 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.662 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "elasticsearch container image", 2024-09-13T18:34:13.662 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.662 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.662 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.662 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.662 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_grafana": { 2024-09-13T18:34:13.662 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_grafana", 2024-09-13T18:34:13.662 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.663 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.663 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.663 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-09-13T18:34:13.663 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.663 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.663 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.663 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Prometheus container image", 2024-09-13T18:34:13.663 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.663 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.664 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.664 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.664 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_haproxy": { 2024-09-13T18:34:13.664 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_haproxy", 2024-09-13T18:34:13.664 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.664 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.664 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.664 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-13T18:34:13.664 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.664 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.664 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.664 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "HAproxy container image", 2024-09-13T18:34:13.664 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.665 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.665 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.665 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.665 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_jaeger_agent": { 2024-09-13T18:34:13.665 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_jaeger_agent", 2024-09-13T18:34:13.665 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.665 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.665 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.665 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-13T18:34:13.665 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.665 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.666 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.666 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Jaeger agent container image", 2024-09-13T18:34:13.666 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.666 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.666 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.666 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.666 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_jaeger_collector": { 2024-09-13T18:34:13.666 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_jaeger_collector", 2024-09-13T18:34:13.666 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.666 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.666 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.666 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-13T18:34:13.666 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.667 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.667 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.667 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Jaeger collector container image", 2024-09-13T18:34:13.667 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.667 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.667 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.667 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.667 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_jaeger_query": { 2024-09-13T18:34:13.667 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_jaeger_query", 2024-09-13T18:34:13.667 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.667 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.667 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.668 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-13T18:34:13.668 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.668 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.668 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.668 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Jaeger query container image", 2024-09-13T18:34:13.668 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.668 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.668 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.668 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.668 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_keepalived": { 2024-09-13T18:34:13.668 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_keepalived", 2024-09-13T18:34:13.668 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.668 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.669 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.669 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-13T18:34:13.669 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.669 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.669 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.669 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Keepalived container image", 2024-09-13T18:34:13.669 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.669 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.669 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.669 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.669 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_loki": { 2024-09-13T18:34:13.669 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_loki", 2024-09-13T18:34:13.669 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.670 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.670 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.670 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-13T18:34:13.670 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.670 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.670 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.670 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Loki container image", 2024-09-13T18:34:13.670 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.670 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.670 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.670 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.670 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_node_exporter": { 2024-09-13T18:34:13.670 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_node_exporter", 2024-09-13T18:34:13.670 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.671 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.671 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.671 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-13T18:34:13.671 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.671 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.671 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.671 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Prometheus container image", 2024-09-13T18:34:13.671 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.671 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.671 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.671 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.671 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_nvmeof": { 2024-09-13T18:34:13.671 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_nvmeof", 2024-09-13T18:34:13.672 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.672 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.672 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.672 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-09-13T18:34:13.672 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.672 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.672 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.672 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Nvme-of container image", 2024-09-13T18:34:13.672 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.672 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.672 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.672 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.672 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_prometheus": { 2024-09-13T18:34:13.672 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_prometheus", 2024-09-13T18:34:13.673 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.673 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.673 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.673 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-13T18:34:13.673 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.673 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.673 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.673 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Prometheus container image", 2024-09-13T18:34:13.673 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.673 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.673 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.673 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.673 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_promtail": { 2024-09-13T18:34:13.673 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_promtail", 2024-09-13T18:34:13.674 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.674 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.674 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.674 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-13T18:34:13.674 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.674 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.674 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.674 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Promtail container image", 2024-09-13T18:34:13.674 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.674 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.674 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.674 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.674 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_snmp_gateway": { 2024-09-13T18:34:13.674 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_snmp_gateway", 2024-09-13T18:34:13.675 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.675 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.675 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.675 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-13T18:34:13.675 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.675 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.675 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.675 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "SNMP Gateway container image", 2024-09-13T18:34:13.675 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.675 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.675 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.675 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.675 INFO:teuthology.orchestra.run.smithi114.stdout: "container_init": { 2024-09-13T18:34:13.675 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_init", 2024-09-13T18:34:13.676 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.676 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.676 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.676 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:13.676 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.676 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.676 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.676 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Run podman/docker with `--init`", 2024-09-13T18:34:13.676 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.676 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.676 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.676 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.676 INFO:teuthology.orchestra.run.smithi114.stdout: "daemon_cache_timeout": { 2024-09-13T18:34:13.676 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "daemon_cache_timeout", 2024-09-13T18:34:13.677 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:13.677 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.677 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.677 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "600", 2024-09-13T18:34:13.677 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.677 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.677 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.677 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-13T18:34:13.677 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.677 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.677 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.677 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.677 INFO:teuthology.orchestra.run.smithi114.stdout: "default_cephadm_command_timeout": { 2024-09-13T18:34:13.677 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "default_cephadm_command_timeout", 2024-09-13T18:34:13.678 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:13.678 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.678 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.678 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "900", 2024-09-13T18:34:13.678 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.678 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.678 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.678 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-13T18:34:13.678 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.678 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.678 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.678 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.679 INFO:teuthology.orchestra.run.smithi114.stdout: "default_registry": { 2024-09-13T18:34:13.679 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "default_registry", 2024-09-13T18:34:13.679 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.679 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.679 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.679 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "docker.io", 2024-09-13T18:34:13.679 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.679 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.679 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.679 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-13T18:34:13.679 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.679 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.679 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.679 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.680 INFO:teuthology.orchestra.run.smithi114.stdout: "device_cache_timeout": { 2024-09-13T18:34:13.680 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "device_cache_timeout", 2024-09-13T18:34:13.680 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:13.680 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.680 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.680 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "1800", 2024-09-13T18:34:13.680 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.680 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.680 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.680 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "seconds to cache device inventory", 2024-09-13T18:34:13.680 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.680 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.680 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.680 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.681 INFO:teuthology.orchestra.run.smithi114.stdout: "device_enhanced_scan": { 2024-09-13T18:34:13.681 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "device_enhanced_scan", 2024-09-13T18:34:13.681 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.681 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.681 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.681 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.681 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.681 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.681 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.681 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-13T18:34:13.681 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.681 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.681 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.682 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.682 INFO:teuthology.orchestra.run.smithi114.stdout: "facts_cache_timeout": { 2024-09-13T18:34:13.682 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "facts_cache_timeout", 2024-09-13T18:34:13.682 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:13.682 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.682 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.682 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "60", 2024-09-13T18:34:13.682 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.682 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.682 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.682 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "seconds to cache host facts data", 2024-09-13T18:34:13.682 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.682 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.683 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.683 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.683 INFO:teuthology.orchestra.run.smithi114.stdout: "host_check_interval": { 2024-09-13T18:34:13.683 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "host_check_interval", 2024-09-13T18:34:13.683 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:13.683 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.683 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.683 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "600", 2024-09-13T18:34:13.683 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.684 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.684 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.684 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "how frequently to perform a host check", 2024-09-13T18:34:13.684 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.684 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.684 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.684 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.684 INFO:teuthology.orchestra.run.smithi114.stdout: "hw_monitoring": { 2024-09-13T18:34:13.684 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "hw_monitoring", 2024-09-13T18:34:13.684 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.684 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.684 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.684 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.685 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.685 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.685 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.685 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-13T18:34:13.685 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.685 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.685 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.685 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.685 INFO:teuthology.orchestra.run.smithi114.stdout: "inventory_list_all": { 2024-09-13T18:34:13.685 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "inventory_list_all", 2024-09-13T18:34:13.685 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.685 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.685 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.686 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.686 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.686 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.686 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.686 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-13T18:34:13.686 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.686 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.686 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.686 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.686 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:13.686 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:13.686 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.686 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.686 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.687 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.687 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.687 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.687 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.687 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.687 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.687 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.687 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.687 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.687 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.687 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.687 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.687 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.687 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.687 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.688 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.688 INFO:teuthology.orchestra.run.smithi114.stdout: "log_refresh_metadata": { 2024-09-13T18:34:13.688 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_refresh_metadata", 2024-09-13T18:34:13.688 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.688 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.688 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.688 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.688 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.688 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.688 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.688 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-13T18:34:13.688 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.688 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.689 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.689 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.689 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:13.689 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:13.689 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.689 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.689 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.689 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:13.689 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.689 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.689 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.689 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-13T18:34:13.689 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.689 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.690 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.690 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.690 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:13.690 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:13.690 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.690 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.690 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.690 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:13.690 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.690 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.690 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.690 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.690 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.690 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.690 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.691 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.691 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.691 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.691 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.691 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.691 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.691 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.691 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.691 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:13.691 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:13.691 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.691 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.691 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.692 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.692 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.692 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.692 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.692 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.692 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.692 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.692 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.692 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.692 INFO:teuthology.orchestra.run.smithi114.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-13T18:34:13.692 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-13T18:34:13.692 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.692 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.692 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.693 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.693 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.693 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.693 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.693 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-13T18:34:13.693 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.693 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.693 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.693 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.693 INFO:teuthology.orchestra.run.smithi114.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-13T18:34:13.693 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-13T18:34:13.693 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.694 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.694 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.694 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "*", 2024-09-13T18:34:13.694 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.694 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.694 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.694 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-13T18:34:13.694 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.694 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.694 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.694 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.694 INFO:teuthology.orchestra.run.smithi114.stdout: "max_count_per_host": { 2024-09-13T18:34:13.694 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "max_count_per_host", 2024-09-13T18:34:13.695 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:13.695 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.695 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.695 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "10", 2024-09-13T18:34:13.695 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.695 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.695 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.695 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "max number of daemons per service per host", 2024-09-13T18:34:13.695 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.695 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.695 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.695 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.695 INFO:teuthology.orchestra.run.smithi114.stdout: "max_osd_draining_count": { 2024-09-13T18:34:13.695 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "max_osd_draining_count", 2024-09-13T18:34:13.695 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:13.696 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.696 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.696 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "10", 2024-09-13T18:34:13.696 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.696 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.696 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.696 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-13T18:34:13.696 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.696 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.696 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.696 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.696 INFO:teuthology.orchestra.run.smithi114.stdout: "migration_current": { 2024-09-13T18:34:13.696 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "migration_current", 2024-09-13T18:34:13.696 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:13.697 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.697 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.697 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.697 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.697 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.697 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.697 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "internal - do not modify", 2024-09-13T18:34:13.697 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.697 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.697 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.697 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.697 INFO:teuthology.orchestra.run.smithi114.stdout: "mode": { 2024-09-13T18:34:13.697 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "mode", 2024-09-13T18:34:13.697 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.698 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.698 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.698 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "root", 2024-09-13T18:34:13.698 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.698 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.698 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.698 INFO:teuthology.orchestra.run.smithi114.stdout: "cephadm-package", 2024-09-13T18:34:13.698 INFO:teuthology.orchestra.run.smithi114.stdout: "root" 2024-09-13T18:34:13.698 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.698 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "mode for remote execution of cephadm", 2024-09-13T18:34:13.698 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.698 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.698 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.698 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.698 INFO:teuthology.orchestra.run.smithi114.stdout: "oob_default_addr": { 2024-09-13T18:34:13.699 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "oob_default_addr", 2024-09-13T18:34:13.699 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.699 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.699 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.699 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "169.254.1.1", 2024-09-13T18:34:13.699 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.699 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.699 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.699 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-13T18:34:13.699 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.699 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.699 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.699 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.699 INFO:teuthology.orchestra.run.smithi114.stdout: "prometheus_alerts_path": { 2024-09-13T18:34:13.700 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "prometheus_alerts_path", 2024-09-13T18:34:13.700 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.700 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.700 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.700 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-13T18:34:13.700 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.700 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.700 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.700 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-13T18:34:13.700 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.700 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.700 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.700 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.701 INFO:teuthology.orchestra.run.smithi114.stdout: "registry_insecure": { 2024-09-13T18:34:13.701 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "registry_insecure", 2024-09-13T18:34:13.701 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.701 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.701 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.701 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.701 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.701 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.701 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.701 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-13T18:34:13.701 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.701 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.701 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.701 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.702 INFO:teuthology.orchestra.run.smithi114.stdout: "registry_password": { 2024-09-13T18:34:13.702 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "registry_password", 2024-09-13T18:34:13.702 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.704 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.704 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.705 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.705 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.705 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.705 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.705 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-13T18:34:13.705 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.705 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.705 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.705 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.705 INFO:teuthology.orchestra.run.smithi114.stdout: "registry_url": { 2024-09-13T18:34:13.705 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "registry_url", 2024-09-13T18:34:13.705 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.706 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.706 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.706 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.706 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.706 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.706 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.706 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-13T18:34:13.706 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.706 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.706 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.706 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.706 INFO:teuthology.orchestra.run.smithi114.stdout: "registry_username": { 2024-09-13T18:34:13.706 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "registry_username", 2024-09-13T18:34:13.706 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.706 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.707 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.707 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.707 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.707 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.707 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.707 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-13T18:34:13.707 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.707 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.707 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.707 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.707 INFO:teuthology.orchestra.run.smithi114.stdout: "secure_monitoring_stack": { 2024-09-13T18:34:13.708 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "secure_monitoring_stack", 2024-09-13T18:34:13.708 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.708 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.708 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.708 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.708 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.708 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.708 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.708 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-13T18:34:13.708 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.708 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.708 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.708 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.709 INFO:teuthology.orchestra.run.smithi114.stdout: "service_discovery_port": { 2024-09-13T18:34:13.709 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "service_discovery_port", 2024-09-13T18:34:13.709 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:13.709 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.709 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.709 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "8765", 2024-09-13T18:34:13.709 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.709 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.709 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.709 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "cephadm service discovery port", 2024-09-13T18:34:13.709 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.709 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.709 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.710 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.710 INFO:teuthology.orchestra.run.smithi114.stdout: "ssh_config_file": { 2024-09-13T18:34:13.710 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ssh_config_file", 2024-09-13T18:34:13.710 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.710 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.710 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.710 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.710 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.710 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.710 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.710 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-13T18:34:13.710 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.710 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.710 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.710 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.711 INFO:teuthology.orchestra.run.smithi114.stdout: "use_agent": { 2024-09-13T18:34:13.711 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "use_agent", 2024-09-13T18:34:13.711 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.711 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.711 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.711 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.711 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.711 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.711 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.711 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-13T18:34:13.711 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.711 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.711 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.712 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.712 INFO:teuthology.orchestra.run.smithi114.stdout: "use_repo_digest": { 2024-09-13T18:34:13.712 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "use_repo_digest", 2024-09-13T18:34:13.712 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.712 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.712 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.712 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:13.712 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.712 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.712 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.712 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-13T18:34:13.712 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.712 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.713 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.713 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.713 INFO:teuthology.orchestra.run.smithi114.stdout: "warn_on_failed_host_check": { 2024-09-13T18:34:13.713 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "warn_on_failed_host_check", 2024-09-13T18:34:13.713 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.713 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.713 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.713 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:13.713 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.713 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.713 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.713 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "raise a health warning if the host check fails", 2024-09-13T18:34:13.713 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.713 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.713 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.714 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.714 INFO:teuthology.orchestra.run.smithi114.stdout: "warn_on_stray_daemons": { 2024-09-13T18:34:13.714 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "warn_on_stray_daemons", 2024-09-13T18:34:13.714 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.714 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.714 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.714 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:13.714 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.714 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.714 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.714 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-13T18:34:13.714 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.714 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.715 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.715 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.715 INFO:teuthology.orchestra.run.smithi114.stdout: "warn_on_stray_hosts": { 2024-09-13T18:34:13.715 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "warn_on_stray_hosts", 2024-09-13T18:34:13.715 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.715 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.715 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.715 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:13.715 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.715 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.715 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.715 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-13T18:34:13.715 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.715 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.716 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.716 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.716 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.716 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.716 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:13.716 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "crash", 2024-09-13T18:34:13.716 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:13.716 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:13.716 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:13.716 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:13.716 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:13.716 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.716 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.716 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.716 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.717 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.717 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.717 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.717 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.717 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.717 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.717 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.717 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.717 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.717 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.717 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.717 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.717 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.718 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.718 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.718 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:13.718 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:13.718 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.718 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.718 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.718 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.718 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.718 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.718 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.718 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.718 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.718 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.719 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.719 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.719 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:13.719 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:13.719 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.719 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.719 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.719 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:13.719 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.719 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.719 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.719 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.719 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.720 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.720 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.720 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.720 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.720 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.720 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.720 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.720 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.720 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.720 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.720 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:13.721 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:13.721 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.721 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.721 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.721 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.721 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.721 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.721 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.721 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.721 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.721 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.721 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.721 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.721 INFO:teuthology.orchestra.run.smithi114.stdout: "retain_interval": { 2024-09-13T18:34:13.722 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "retain_interval", 2024-09-13T18:34:13.722 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:13.722 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.722 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.722 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "31536000", 2024-09-13T18:34:13.722 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.722 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.722 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.722 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-13T18:34:13.722 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.722 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.722 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.722 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.723 INFO:teuthology.orchestra.run.smithi114.stdout: "warn_recent_interval": { 2024-09-13T18:34:13.723 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "warn_recent_interval", 2024-09-13T18:34:13.723 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:13.723 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.723 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.723 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "1209600", 2024-09-13T18:34:13.723 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.723 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.723 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.723 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-13T18:34:13.723 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.723 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.724 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.724 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.724 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.724 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.724 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:13.724 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "dashboard", 2024-09-13T18:34:13.724 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:13.724 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:13.724 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:13.724 INFO:teuthology.orchestra.run.smithi114.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-13T18:34:13.724 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-13T18:34:13.724 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:13.724 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.724 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.725 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "10", 2024-09-13T18:34:13.725 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.725 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.725 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.725 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.725 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.725 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.725 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.725 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.725 INFO:teuthology.orchestra.run.smithi114.stdout: "ALERTMANAGER_API_HOST": { 2024-09-13T18:34:13.725 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-13T18:34:13.725 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.725 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.726 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.726 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.726 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.726 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.726 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.726 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.726 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.726 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.726 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.726 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.726 INFO:teuthology.orchestra.run.smithi114.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-13T18:34:13.726 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-13T18:34:13.726 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.727 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.727 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.727 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:13.727 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.727 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.727 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.727 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.727 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.727 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.727 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.727 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.727 INFO:teuthology.orchestra.run.smithi114.stdout: "AUDIT_API_ENABLED": { 2024-09-13T18:34:13.727 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "AUDIT_API_ENABLED", 2024-09-13T18:34:13.727 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.728 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.728 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.728 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.728 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.728 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.728 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.728 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.728 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.728 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.728 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.728 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.728 INFO:teuthology.orchestra.run.smithi114.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-13T18:34:13.729 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-13T18:34:13.729 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.729 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.729 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.729 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:13.729 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.729 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.729 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.729 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.729 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.729 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.729 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.729 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.730 INFO:teuthology.orchestra.run.smithi114.stdout: "ENABLE_BROWSABLE_API": { 2024-09-13T18:34:13.730 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-13T18:34:13.730 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.730 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.730 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.730 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:13.730 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.730 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.730 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.730 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.730 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.730 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.730 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.730 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.731 INFO:teuthology.orchestra.run.smithi114.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-13T18:34:13.731 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-13T18:34:13.731 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.731 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.731 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.731 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:13.731 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.731 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.731 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.731 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.731 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.731 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.731 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.731 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.732 INFO:teuthology.orchestra.run.smithi114.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-13T18:34:13.732 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-13T18:34:13.732 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.732 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.732 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.732 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:13.732 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.732 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.732 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.732 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.732 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.732 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.732 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.733 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.733 INFO:teuthology.orchestra.run.smithi114.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-13T18:34:13.733 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-13T18:34:13.733 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.733 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.733 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.733 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:13.733 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.733 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.733 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.733 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.733 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.733 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.733 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.734 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.734 INFO:teuthology.orchestra.run.smithi114.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-13T18:34:13.734 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-13T18:34:13.734 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.734 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.734 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.734 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:13.734 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.734 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.734 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.734 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.735 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.735 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.735 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.735 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.735 INFO:teuthology.orchestra.run.smithi114.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-13T18:34:13.735 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-13T18:34:13.735 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.735 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.735 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.735 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:13.735 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.735 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.735 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.736 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.736 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.736 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.736 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.736 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.736 INFO:teuthology.orchestra.run.smithi114.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-13T18:34:13.736 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-13T18:34:13.736 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.736 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.736 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.736 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:13.736 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.736 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.737 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.737 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.737 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.737 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.737 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.737 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.737 INFO:teuthology.orchestra.run.smithi114.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-13T18:34:13.737 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-13T18:34:13.737 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.737 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.737 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.737 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:13.737 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.738 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.738 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.738 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.738 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.738 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.738 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.738 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.738 INFO:teuthology.orchestra.run.smithi114.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-13T18:34:13.738 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-13T18:34:13.738 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.738 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.738 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.738 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.738 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.739 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.739 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.739 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.739 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.739 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.739 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.739 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.739 INFO:teuthology.orchestra.run.smithi114.stdout: "GRAFANA_API_PASSWORD": { 2024-09-13T18:34:13.739 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-13T18:34:13.739 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.739 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.739 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.739 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "admin", 2024-09-13T18:34:13.739 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.740 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.740 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.740 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.740 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.740 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.740 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.740 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.740 INFO:teuthology.orchestra.run.smithi114.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-13T18:34:13.740 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-13T18:34:13.740 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.740 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.740 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.740 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:13.740 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.741 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.741 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.741 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.741 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.741 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.741 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.741 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.741 INFO:teuthology.orchestra.run.smithi114.stdout: "GRAFANA_API_URL": { 2024-09-13T18:34:13.741 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "GRAFANA_API_URL", 2024-09-13T18:34:13.741 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.741 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.741 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.741 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.741 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.742 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.742 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.742 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.742 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.742 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.742 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.742 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.742 INFO:teuthology.orchestra.run.smithi114.stdout: "GRAFANA_API_USERNAME": { 2024-09-13T18:34:13.742 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-13T18:34:13.742 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.742 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.742 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.742 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "admin", 2024-09-13T18:34:13.742 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.742 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.743 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.743 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.743 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.743 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.743 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.743 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.743 INFO:teuthology.orchestra.run.smithi114.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-13T18:34:13.743 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-13T18:34:13.743 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.743 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.743 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.743 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.743 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.743 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.744 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.744 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.744 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.744 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.744 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.744 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.744 INFO:teuthology.orchestra.run.smithi114.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-13T18:34:13.744 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-13T18:34:13.744 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.744 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.744 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.744 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.744 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.744 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.745 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.745 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.745 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.745 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.745 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.745 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.745 INFO:teuthology.orchestra.run.smithi114.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-13T18:34:13.745 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-13T18:34:13.745 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.745 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.745 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.745 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:13.745 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.745 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.746 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.746 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.746 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.746 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.746 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.746 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.746 INFO:teuthology.orchestra.run.smithi114.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-13T18:34:13.746 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-13T18:34:13.746 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.746 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.746 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.746 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.746 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.746 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.746 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.747 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.747 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.747 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.747 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.747 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.747 INFO:teuthology.orchestra.run.smithi114.stdout: "PROMETHEUS_API_HOST": { 2024-09-13T18:34:13.747 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-13T18:34:13.747 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.747 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.747 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.747 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.747 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.747 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.747 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.748 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.748 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.748 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.748 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.748 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.748 INFO:teuthology.orchestra.run.smithi114.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-13T18:34:13.748 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-13T18:34:13.748 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.748 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.748 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.748 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:13.748 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.748 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.749 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.749 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.749 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.749 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.749 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.749 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.749 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-13T18:34:13.749 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-13T18:34:13.749 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.749 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.749 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.749 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.749 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.750 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.750 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.750 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.750 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.750 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.750 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.750 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.750 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-13T18:34:13.750 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-13T18:34:13.750 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.750 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.750 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.750 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.750 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.751 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.751 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.751 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.751 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.751 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.751 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.751 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.751 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-13T18:34:13.751 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-13T18:34:13.751 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.751 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.751 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.751 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:13.751 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.752 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.752 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.752 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.752 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.752 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.752 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.752 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.752 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-13T18:34:13.752 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-13T18:34:13.752 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.752 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.752 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.752 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:13.752 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.753 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.753 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.753 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.753 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.753 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.753 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.753 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.753 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-13T18:34:13.753 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-13T18:34:13.753 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.753 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.753 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.753 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.753 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.754 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.754 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.754 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.754 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.754 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.754 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.754 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.754 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-13T18:34:13.754 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-13T18:34:13.754 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.754 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.754 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.754 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.754 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.755 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.755 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.755 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.755 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.755 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.755 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.755 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.755 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-13T18:34:13.755 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-13T18:34:13.755 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.755 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.755 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.755 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.755 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.756 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.756 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.756 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.756 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.756 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.756 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.756 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.756 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_ENABLED": { 2024-09-13T18:34:13.756 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-13T18:34:13.756 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.756 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.756 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.756 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:13.757 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.757 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.757 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.757 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.757 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.757 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.757 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.757 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.757 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-13T18:34:13.757 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-13T18:34:13.757 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.757 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.757 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.757 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-13T18:34:13.758 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.758 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.758 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.758 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.758 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.758 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.758 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.758 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.758 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-13T18:34:13.758 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-13T18:34:13.758 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:13.758 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.758 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.758 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "10", 2024-09-13T18:34:13.759 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.759 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.759 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.759 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.759 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.759 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.759 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.759 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.759 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-13T18:34:13.759 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-13T18:34:13.759 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:13.759 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.759 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.759 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "8", 2024-09-13T18:34:13.760 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.760 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.760 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.760 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.760 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.760 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.760 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.760 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.760 INFO:teuthology.orchestra.run.smithi114.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-13T18:34:13.760 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-13T18:34:13.760 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:13.760 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.760 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.760 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "45", 2024-09-13T18:34:13.761 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.761 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.761 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.761 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.761 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.761 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.761 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.761 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.761 INFO:teuthology.orchestra.run.smithi114.stdout: "RGW_API_ACCESS_KEY": { 2024-09-13T18:34:13.761 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-13T18:34:13.761 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.761 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.761 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.762 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.762 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.762 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.762 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.762 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.762 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.762 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.762 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.762 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.762 INFO:teuthology.orchestra.run.smithi114.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-13T18:34:13.762 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-13T18:34:13.762 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.762 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.762 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.763 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "admin", 2024-09-13T18:34:13.763 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.763 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.763 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.763 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.763 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.763 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.763 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.763 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.763 INFO:teuthology.orchestra.run.smithi114.stdout: "RGW_API_SECRET_KEY": { 2024-09-13T18:34:13.763 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-13T18:34:13.763 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.763 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.763 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.764 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.764 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.764 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.764 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.764 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.764 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.764 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.764 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.764 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.764 INFO:teuthology.orchestra.run.smithi114.stdout: "RGW_API_SSL_VERIFY": { 2024-09-13T18:34:13.764 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-13T18:34:13.764 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.764 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.765 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.765 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:13.765 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.765 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.765 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.765 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.765 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.765 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.765 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.765 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.765 INFO:teuthology.orchestra.run.smithi114.stdout: "UNSAFE_TLS_v1_2": { 2024-09-13T18:34:13.765 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-13T18:34:13.765 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.766 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.766 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.766 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.766 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.766 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.766 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.766 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.766 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.766 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.766 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.766 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.766 INFO:teuthology.orchestra.run.smithi114.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-13T18:34:13.766 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-13T18:34:13.766 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:13.767 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.767 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.767 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-13T18:34:13.767 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.767 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.767 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.767 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.767 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.767 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.767 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.767 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.767 INFO:teuthology.orchestra.run.smithi114.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-13T18:34:13.767 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-13T18:34:13.767 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:13.768 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.768 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.768 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "10", 2024-09-13T18:34:13.768 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.768 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.768 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.768 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.768 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.768 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.768 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.768 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.768 INFO:teuthology.orchestra.run.smithi114.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-13T18:34:13.768 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-13T18:34:13.769 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:13.769 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.769 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.769 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "5", 2024-09-13T18:34:13.769 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.769 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.769 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.769 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.769 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.769 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.769 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.769 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.769 INFO:teuthology.orchestra.run.smithi114.stdout: "cross_origin_url": { 2024-09-13T18:34:13.770 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "cross_origin_url", 2024-09-13T18:34:13.770 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.770 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.770 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.770 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.770 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.770 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.770 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.770 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.770 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.770 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.770 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.770 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.770 INFO:teuthology.orchestra.run.smithi114.stdout: "crt_file": { 2024-09-13T18:34:13.771 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "crt_file", 2024-09-13T18:34:13.771 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.771 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.771 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.771 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.771 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.771 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.771 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.771 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.771 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.771 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.771 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.771 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.772 INFO:teuthology.orchestra.run.smithi114.stdout: "debug": { 2024-09-13T18:34:13.772 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "debug", 2024-09-13T18:34:13.772 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.772 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.772 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.772 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.772 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.772 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.772 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.772 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Enable/disable debug options", 2024-09-13T18:34:13.772 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.772 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.772 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.772 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.773 INFO:teuthology.orchestra.run.smithi114.stdout: "jwt_token_ttl": { 2024-09-13T18:34:13.773 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "jwt_token_ttl", 2024-09-13T18:34:13.773 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:13.773 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.773 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.773 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "28800", 2024-09-13T18:34:13.773 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.773 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.773 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.773 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.773 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.773 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.773 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.773 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.774 INFO:teuthology.orchestra.run.smithi114.stdout: "key_file": { 2024-09-13T18:34:13.774 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "key_file", 2024-09-13T18:34:13.774 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.774 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.774 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.774 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.774 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.774 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.774 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.774 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.774 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.774 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.775 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.775 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.775 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:13.775 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:13.775 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.775 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.775 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.775 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.775 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.775 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.775 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.775 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.775 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.776 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.776 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.776 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.776 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.776 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.776 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.776 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.776 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.776 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.776 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.776 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:13.776 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:13.777 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.777 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.777 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.777 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.777 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.777 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.777 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.777 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.777 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.777 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.777 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.777 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.777 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:13.777 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:13.778 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.778 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.778 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.778 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:13.778 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.778 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.778 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.778 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.778 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.778 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.778 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.778 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.778 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.778 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.778 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.779 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.779 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.779 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.779 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.779 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:13.779 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:13.779 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.779 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.779 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.779 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.779 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.779 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.779 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.780 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.780 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.780 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.780 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.780 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.780 INFO:teuthology.orchestra.run.smithi114.stdout: "motd": { 2024-09-13T18:34:13.780 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "motd", 2024-09-13T18:34:13.780 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.780 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.780 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.780 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.780 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.780 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.781 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.781 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "The message of the day", 2024-09-13T18:34:13.781 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.781 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.781 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.781 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.781 INFO:teuthology.orchestra.run.smithi114.stdout: "redirect_resolve_ip_addr": { 2024-09-13T18:34:13.781 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "redirect_resolve_ip_addr", 2024-09-13T18:34:13.781 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.781 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.781 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.781 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.781 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.782 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.782 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.782 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.782 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.782 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.782 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.782 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.782 INFO:teuthology.orchestra.run.smithi114.stdout: "server_addr": { 2024-09-13T18:34:13.782 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "server_addr", 2024-09-13T18:34:13.782 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.782 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.782 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.782 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "::", 2024-09-13T18:34:13.782 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.782 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.783 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.783 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.783 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.783 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.783 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.783 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.783 INFO:teuthology.orchestra.run.smithi114.stdout: "server_port": { 2024-09-13T18:34:13.783 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "server_port", 2024-09-13T18:34:13.783 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:13.783 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.783 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.783 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "8080", 2024-09-13T18:34:13.783 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.783 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.784 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.784 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.784 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.784 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.784 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.784 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.784 INFO:teuthology.orchestra.run.smithi114.stdout: "ssl": { 2024-09-13T18:34:13.784 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ssl", 2024-09-13T18:34:13.784 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.784 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.784 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.784 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:13.784 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.784 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.785 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.785 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.785 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.785 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.785 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.785 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.785 INFO:teuthology.orchestra.run.smithi114.stdout: "ssl_server_port": { 2024-09-13T18:34:13.785 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ssl_server_port", 2024-09-13T18:34:13.785 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:13.785 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.785 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.785 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "8443", 2024-09-13T18:34:13.785 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.785 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.786 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.786 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.786 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.786 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.786 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.786 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.786 INFO:teuthology.orchestra.run.smithi114.stdout: "standby_behaviour": { 2024-09-13T18:34:13.786 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "standby_behaviour", 2024-09-13T18:34:13.786 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.786 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.786 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.786 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "redirect", 2024-09-13T18:34:13.786 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.786 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.787 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.787 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.787 INFO:teuthology.orchestra.run.smithi114.stdout: "redirect" 2024-09-13T18:34:13.787 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.787 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.787 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.787 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.787 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.787 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.787 INFO:teuthology.orchestra.run.smithi114.stdout: "standby_error_status_code": { 2024-09-13T18:34:13.787 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "standby_error_status_code", 2024-09-13T18:34:13.787 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:13.787 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.787 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.788 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "500", 2024-09-13T18:34:13.788 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "400", 2024-09-13T18:34:13.788 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "599", 2024-09-13T18:34:13.788 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.788 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.788 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.788 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.788 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.788 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.788 INFO:teuthology.orchestra.run.smithi114.stdout: "url_prefix": { 2024-09-13T18:34:13.788 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "url_prefix", 2024-09-13T18:34:13.788 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.788 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.789 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.789 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.789 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.789 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.789 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.789 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.789 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.789 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.789 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.789 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.789 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.789 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.790 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:13.790 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "devicehealth", 2024-09-13T18:34:13.790 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:13.790 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:13.790 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:13.790 INFO:teuthology.orchestra.run.smithi114.stdout: "enable_monitoring": { 2024-09-13T18:34:13.790 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "enable_monitoring", 2024-09-13T18:34:13.790 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.790 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.790 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.790 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:13.790 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.790 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.790 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.791 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "monitor device health metrics", 2024-09-13T18:34:13.791 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.791 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.791 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.791 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.791 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:13.791 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:13.791 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.791 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.791 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.791 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.791 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.791 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.791 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.791 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.792 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.792 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.792 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.792 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.792 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.792 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.792 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.792 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.792 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.792 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.792 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.792 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:13.792 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:13.792 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.793 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.793 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.793 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.793 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.793 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.793 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.793 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.793 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.793 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.793 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.793 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.793 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:13.793 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:13.794 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.794 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.794 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.794 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:13.794 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.794 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.794 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.794 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.794 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.794 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.794 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.794 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.794 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.794 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.795 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.795 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.795 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.795 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.795 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.795 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:13.795 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:13.795 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.795 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.795 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.795 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.795 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.795 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.796 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.796 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.796 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.796 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.796 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.796 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.796 INFO:teuthology.orchestra.run.smithi114.stdout: "mark_out_threshold": { 2024-09-13T18:34:13.796 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "mark_out_threshold", 2024-09-13T18:34:13.796 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:13.796 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.796 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.796 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "2419200", 2024-09-13T18:34:13.796 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.796 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.797 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.797 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-13T18:34:13.797 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.797 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.797 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.797 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.797 INFO:teuthology.orchestra.run.smithi114.stdout: "pool_name": { 2024-09-13T18:34:13.797 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "pool_name", 2024-09-13T18:34:13.797 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.797 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.797 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.797 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "device_health_metrics", 2024-09-13T18:34:13.797 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.797 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.798 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.798 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-13T18:34:13.798 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.798 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.798 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.798 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.798 INFO:teuthology.orchestra.run.smithi114.stdout: "retention_period": { 2024-09-13T18:34:13.798 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "retention_period", 2024-09-13T18:34:13.798 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:13.798 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.798 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.798 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "15552000", 2024-09-13T18:34:13.798 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.798 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.798 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.799 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "how long to retain device health metrics", 2024-09-13T18:34:13.799 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.799 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.799 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.799 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.799 INFO:teuthology.orchestra.run.smithi114.stdout: "scrape_frequency": { 2024-09-13T18:34:13.799 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "scrape_frequency", 2024-09-13T18:34:13.799 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:13.799 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.799 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.799 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "86400", 2024-09-13T18:34:13.799 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.799 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.799 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.799 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-13T18:34:13.800 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.800 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.800 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.800 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.800 INFO:teuthology.orchestra.run.smithi114.stdout: "self_heal": { 2024-09-13T18:34:13.800 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "self_heal", 2024-09-13T18:34:13.800 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.800 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.800 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.800 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:13.800 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.800 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.800 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.800 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-13T18:34:13.801 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.801 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.801 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.801 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.801 INFO:teuthology.orchestra.run.smithi114.stdout: "sleep_interval": { 2024-09-13T18:34:13.801 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sleep_interval", 2024-09-13T18:34:13.801 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:13.801 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.801 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.801 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "600", 2024-09-13T18:34:13.801 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.801 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.801 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.801 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "how frequently to wake up and check device health", 2024-09-13T18:34:13.801 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.802 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.802 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.802 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.802 INFO:teuthology.orchestra.run.smithi114.stdout: "warn_threshold": { 2024-09-13T18:34:13.802 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "warn_threshold", 2024-09-13T18:34:13.802 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:13.802 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.802 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.802 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "7257600", 2024-09-13T18:34:13.802 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.802 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.802 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.802 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-13T18:34:13.803 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.803 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.803 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.803 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.803 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.803 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.803 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:13.803 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "diskprediction_local", 2024-09-13T18:34:13.803 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:13.803 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:13.803 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:13.803 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:13.803 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:13.803 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.804 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.804 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.804 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.804 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.804 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.804 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.804 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.804 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.804 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.804 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.804 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.804 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.804 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.804 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.805 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.805 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.805 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.805 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.805 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:13.805 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:13.805 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.805 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.805 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.805 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.805 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.805 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.805 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.805 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.806 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.806 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.806 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.806 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.806 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:13.806 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:13.806 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.806 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.806 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.806 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:13.806 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.806 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.806 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.806 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.807 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.807 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.807 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.807 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.807 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.807 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.807 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.807 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.807 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.807 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.807 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.807 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:13.807 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:13.807 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.808 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.808 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.808 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.808 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.808 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.808 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.808 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.808 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.808 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.808 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.808 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.808 INFO:teuthology.orchestra.run.smithi114.stdout: "predict_interval": { 2024-09-13T18:34:13.808 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "predict_interval", 2024-09-13T18:34:13.808 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.809 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.809 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.809 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "86400", 2024-09-13T18:34:13.809 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.809 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.809 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.809 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.809 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.809 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.809 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.809 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.809 INFO:teuthology.orchestra.run.smithi114.stdout: "predictor_model": { 2024-09-13T18:34:13.809 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "predictor_model", 2024-09-13T18:34:13.809 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.810 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.810 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.810 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "prophetstor", 2024-09-13T18:34:13.810 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.810 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.810 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.810 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.810 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.810 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.810 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.810 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.810 INFO:teuthology.orchestra.run.smithi114.stdout: "sleep_interval": { 2024-09-13T18:34:13.810 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sleep_interval", 2024-09-13T18:34:13.810 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.811 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.811 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.811 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "600", 2024-09-13T18:34:13.811 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.811 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.811 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.811 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.811 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.811 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.811 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.811 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.811 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.811 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.811 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:13.812 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "influx", 2024-09-13T18:34:13.812 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": false, 2024-09-13T18:34:13.812 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "influxdb python module not found", 2024-09-13T18:34:13.812 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:13.812 INFO:teuthology.orchestra.run.smithi114.stdout: "batch_size": { 2024-09-13T18:34:13.812 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "batch_size", 2024-09-13T18:34:13.812 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:13.812 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.812 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.812 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "5000", 2024-09-13T18:34:13.812 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.812 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.812 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.812 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-13T18:34:13.813 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.813 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.813 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.813 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.813 INFO:teuthology.orchestra.run.smithi114.stdout: "database": { 2024-09-13T18:34:13.813 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "database", 2024-09-13T18:34:13.813 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.813 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.813 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.813 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "ceph", 2024-09-13T18:34:13.813 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.813 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.813 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.813 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-13T18:34:13.814 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.814 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.814 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.814 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.814 INFO:teuthology.orchestra.run.smithi114.stdout: "hostname": { 2024-09-13T18:34:13.814 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "hostname", 2024-09-13T18:34:13.814 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.814 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.814 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.814 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.814 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.814 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.814 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.814 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "InfluxDB server hostname", 2024-09-13T18:34:13.814 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.815 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.815 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.815 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.815 INFO:teuthology.orchestra.run.smithi114.stdout: "interval": { 2024-09-13T18:34:13.815 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "interval", 2024-09-13T18:34:13.815 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:13.815 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.815 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.815 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "30", 2024-09-13T18:34:13.815 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "5", 2024-09-13T18:34:13.815 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.815 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.816 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-13T18:34:13.816 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.816 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.816 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.816 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.816 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:13.816 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:13.816 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.816 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.816 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.816 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.816 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.816 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.817 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.817 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.817 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.817 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.817 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.817 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.817 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.817 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.817 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.817 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.817 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.817 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.817 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.817 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:13.817 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:13.818 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.818 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.818 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.818 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.818 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.818 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.818 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.818 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.818 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.818 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.818 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.818 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.818 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:13.818 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:13.819 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.819 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.819 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.819 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:13.819 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.819 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.819 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.819 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.819 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.819 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.819 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.819 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.819 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.819 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.820 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.820 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.820 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.820 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.820 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.820 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:13.820 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:13.820 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.820 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.820 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.820 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.820 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.820 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.820 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.821 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.821 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.821 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.821 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.821 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.821 INFO:teuthology.orchestra.run.smithi114.stdout: "password": { 2024-09-13T18:34:13.821 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "password", 2024-09-13T18:34:13.821 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.821 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.821 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.821 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.821 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.821 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.821 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.822 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "password of InfluxDB server user", 2024-09-13T18:34:13.822 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.822 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.822 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.822 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.822 INFO:teuthology.orchestra.run.smithi114.stdout: "port": { 2024-09-13T18:34:13.822 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "port", 2024-09-13T18:34:13.822 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:13.822 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.822 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.822 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "8086", 2024-09-13T18:34:13.822 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.822 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.822 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.822 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "InfluxDB server port", 2024-09-13T18:34:13.823 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.823 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.823 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.823 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.823 INFO:teuthology.orchestra.run.smithi114.stdout: "ssl": { 2024-09-13T18:34:13.823 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ssl", 2024-09-13T18:34:13.823 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.823 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.823 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.823 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "false", 2024-09-13T18:34:13.823 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.823 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.823 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.823 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-13T18:34:13.824 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.824 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.824 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.824 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.824 INFO:teuthology.orchestra.run.smithi114.stdout: "threads": { 2024-09-13T18:34:13.824 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "threads", 2024-09-13T18:34:13.824 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:13.824 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.824 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.824 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "5", 2024-09-13T18:34:13.824 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "1", 2024-09-13T18:34:13.824 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "32", 2024-09-13T18:34:13.824 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.824 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-13T18:34:13.824 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.825 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.825 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.825 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.825 INFO:teuthology.orchestra.run.smithi114.stdout: "username": { 2024-09-13T18:34:13.825 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "username", 2024-09-13T18:34:13.825 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.825 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.825 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.825 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.825 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.825 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.825 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.825 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "username of InfluxDB server user", 2024-09-13T18:34:13.825 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.826 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.826 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.826 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.826 INFO:teuthology.orchestra.run.smithi114.stdout: "verify_ssl": { 2024-09-13T18:34:13.826 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "verify_ssl", 2024-09-13T18:34:13.826 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.826 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.826 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.826 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "true", 2024-09-13T18:34:13.826 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.826 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.826 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.826 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-13T18:34:13.826 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.827 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.827 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.827 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.827 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.827 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.827 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:13.827 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "insights", 2024-09-13T18:34:13.827 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:13.827 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:13.827 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:13.827 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:13.827 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:13.827 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.827 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.828 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.828 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.828 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.828 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.828 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.828 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.828 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.828 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.828 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.828 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.828 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.828 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.828 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.828 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.829 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.829 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.829 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.829 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:13.829 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:13.829 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.829 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.829 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.829 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.829 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.829 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.829 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.829 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.830 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.830 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.830 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.830 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.830 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:13.830 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:13.830 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.830 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.830 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.830 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:13.830 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.830 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.830 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.831 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.831 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.831 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.831 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.831 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.831 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.831 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.831 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.831 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.831 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.831 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.831 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:13.831 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:13.832 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.832 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.832 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.832 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.832 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.832 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.832 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.832 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.832 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.832 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.832 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.832 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.832 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.833 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.833 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:13.833 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "iostat", 2024-09-13T18:34:13.833 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:13.833 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:13.833 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:13.833 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:13.833 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:13.833 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.833 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.833 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.833 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.833 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.833 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.834 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.834 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.834 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.834 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.834 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.834 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.834 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.834 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.834 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.834 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.834 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.834 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.834 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.835 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:13.835 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:13.835 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.835 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.835 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.835 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.835 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.835 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.835 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.835 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.835 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.835 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.835 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.835 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.836 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:13.836 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:13.836 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.836 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.836 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.836 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:13.836 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.836 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.836 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.836 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.836 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.836 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.836 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.836 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.837 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.837 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.837 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.837 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.837 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.837 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.837 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.837 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:13.837 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:13.837 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.837 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.837 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.837 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.837 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.838 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.838 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.838 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.838 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.838 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.838 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.838 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.838 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.838 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.838 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:13.838 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "k8sevents", 2024-09-13T18:34:13.838 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:13.838 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:13.838 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:13.839 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_event_retention_days": { 2024-09-13T18:34:13.839 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ceph_event_retention_days", 2024-09-13T18:34:13.839 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:13.839 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.839 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.839 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "7", 2024-09-13T18:34:13.839 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.839 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.839 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.839 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-13T18:34:13.839 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.839 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.839 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.839 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.840 INFO:teuthology.orchestra.run.smithi114.stdout: "config_check_secs": { 2024-09-13T18:34:13.840 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "config_check_secs", 2024-09-13T18:34:13.840 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:13.840 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.840 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.840 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "10", 2024-09-13T18:34:13.840 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "10", 2024-09-13T18:34:13.840 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.840 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.840 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-13T18:34:13.840 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.840 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.840 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.840 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.840 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:13.841 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:13.841 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.841 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.841 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.841 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.841 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.841 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.841 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.841 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.841 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.841 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.841 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.841 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.841 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.842 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.842 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.842 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.842 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.842 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.842 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.842 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:13.842 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:13.842 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.842 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.842 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.842 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.842 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.842 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.843 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.843 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.843 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.843 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.843 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.843 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.843 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:13.843 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:13.843 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.843 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.843 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.843 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:13.844 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.844 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.844 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.844 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.844 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.844 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.844 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.844 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.844 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.844 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.844 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.844 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.844 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.845 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.845 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.845 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:13.845 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:13.845 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.845 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.845 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.845 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.845 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.845 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.845 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.845 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.845 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.845 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.846 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.846 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.846 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.846 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.846 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:13.846 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "localpool", 2024-09-13T18:34:13.846 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:13.846 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:13.846 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:13.846 INFO:teuthology.orchestra.run.smithi114.stdout: "failure_domain": { 2024-09-13T18:34:13.846 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "failure_domain", 2024-09-13T18:34:13.846 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.846 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.846 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.847 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "host", 2024-09-13T18:34:13.847 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.847 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.847 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.847 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "failure domain for any created local pool", 2024-09-13T18:34:13.847 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-13T18:34:13.847 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.847 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.847 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.847 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:13.847 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:13.847 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.847 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.847 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.848 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.848 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.848 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.848 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.848 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.848 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.848 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.848 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.848 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.848 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.848 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.848 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.848 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.849 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.849 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.849 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.849 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:13.849 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:13.849 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.849 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.849 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.849 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.849 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.849 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.849 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.849 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.849 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.850 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.850 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.850 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.850 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:13.850 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:13.850 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.850 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.850 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.850 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:13.850 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.850 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.850 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.850 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.850 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.851 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.851 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.851 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.851 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.851 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.851 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.851 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.851 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.851 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.851 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.851 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:13.851 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:13.851 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.852 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.852 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.852 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.852 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.852 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.852 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.852 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.852 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.852 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.852 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.852 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.852 INFO:teuthology.orchestra.run.smithi114.stdout: "min_size": { 2024-09-13T18:34:13.852 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "min_size", 2024-09-13T18:34:13.852 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:13.853 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.853 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.853 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.853 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.853 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.853 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.853 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "default min_size for any created local pool", 2024-09-13T18:34:13.853 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-13T18:34:13.853 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.853 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.853 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.853 INFO:teuthology.orchestra.run.smithi114.stdout: "num_rep": { 2024-09-13T18:34:13.853 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "num_rep", 2024-09-13T18:34:13.854 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:13.854 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.854 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.854 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "3", 2024-09-13T18:34:13.854 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.854 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.854 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.854 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "default replica count for any created local pool", 2024-09-13T18:34:13.854 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.854 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.854 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.854 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.854 INFO:teuthology.orchestra.run.smithi114.stdout: "pg_num": { 2024-09-13T18:34:13.855 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "pg_num", 2024-09-13T18:34:13.855 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:13.855 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.855 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.855 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "128", 2024-09-13T18:34:13.855 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.855 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.855 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.855 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "default pg_num for any created local pool", 2024-09-13T18:34:13.855 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.855 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.855 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.855 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.856 INFO:teuthology.orchestra.run.smithi114.stdout: "prefix": { 2024-09-13T18:34:13.856 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "prefix", 2024-09-13T18:34:13.856 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.856 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.856 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.856 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.856 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.856 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.856 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.856 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "name prefix for any created local pool", 2024-09-13T18:34:13.856 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.856 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.856 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.856 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.857 INFO:teuthology.orchestra.run.smithi114.stdout: "subtree": { 2024-09-13T18:34:13.857 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "subtree", 2024-09-13T18:34:13.857 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.857 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.857 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.857 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "rack", 2024-09-13T18:34:13.857 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.857 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.857 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.857 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-13T18:34:13.857 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-13T18:34:13.857 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.857 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.858 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.858 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.858 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.858 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:13.858 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "mds_autoscaler", 2024-09-13T18:34:13.858 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:13.858 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:13.858 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:13.858 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:13.858 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:13.858 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.858 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.858 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.858 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.859 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.859 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.859 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.859 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.859 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.859 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.859 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.859 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.859 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.859 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.859 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.859 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.859 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.859 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.860 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:13.860 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:13.860 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.860 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.860 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.860 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.860 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.860 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.860 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.860 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.860 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.860 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.860 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.860 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.861 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:13.861 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:13.861 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.861 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.861 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.861 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:13.861 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.861 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.861 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.861 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.861 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.861 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.861 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.861 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.862 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.862 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.862 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.862 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.862 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.862 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.862 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.862 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:13.862 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:13.862 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.862 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.862 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.862 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.862 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.863 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.863 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.863 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.863 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.863 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.863 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.863 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.863 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.863 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.863 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:13.863 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "mirroring", 2024-09-13T18:34:13.863 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:13.863 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:13.863 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:13.864 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:13.864 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:13.864 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.864 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.864 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.864 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.864 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.864 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.864 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.864 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.864 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.864 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.864 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.864 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.865 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.865 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.865 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.865 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.865 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.865 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.865 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.865 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:13.865 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:13.865 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.865 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.865 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.865 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.865 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.866 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.866 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.866 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.866 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.866 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.866 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.866 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.866 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:13.866 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:13.866 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.866 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.866 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.866 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:13.866 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.866 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.867 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.867 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.867 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.867 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.867 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.867 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.867 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.867 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.867 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.867 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.867 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.867 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.867 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.867 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:13.867 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:13.868 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.868 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.868 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.868 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.868 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.868 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.868 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.868 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.868 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.868 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.868 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.868 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.868 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.869 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.869 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:13.869 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "nfs", 2024-09-13T18:34:13.869 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:13.869 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:13.869 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:13.869 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:13.869 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:13.869 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.869 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.869 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.869 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.869 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.869 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.869 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.870 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.870 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.870 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.870 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.870 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.870 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.870 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.870 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.870 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.870 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.870 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.870 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.870 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:13.870 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:13.871 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.871 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.871 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.871 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.871 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.871 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.871 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.871 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.871 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.871 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.871 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.871 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.871 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:13.872 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:13.872 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.872 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.872 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.872 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:13.872 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.872 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.872 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.872 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.872 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.872 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.872 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.872 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.872 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.873 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.873 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.873 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.873 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.873 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.873 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.873 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:13.873 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:13.873 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.873 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.873 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.873 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.873 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.873 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.874 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.874 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.874 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.874 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.874 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.874 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.874 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.874 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.874 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:13.874 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "orchestrator", 2024-09-13T18:34:13.874 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:13.874 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:13.874 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:13.874 INFO:teuthology.orchestra.run.smithi114.stdout: "fail_fs": { 2024-09-13T18:34:13.875 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "fail_fs", 2024-09-13T18:34:13.875 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.875 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.875 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.875 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.875 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.875 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.875 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.875 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-13T18:34:13.875 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.875 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.875 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.875 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.875 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:13.876 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:13.876 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.876 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.876 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.876 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.876 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.876 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.876 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.876 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.876 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.876 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.876 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.876 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.876 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.877 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.877 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.877 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.877 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.877 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.877 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.877 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:13.877 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:13.877 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.877 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.877 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.877 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.877 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.878 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.878 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.878 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.878 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.878 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.878 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.878 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.878 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:13.878 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:13.878 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.878 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.878 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.878 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:13.879 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.879 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.879 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.879 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.879 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.879 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.879 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.879 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.879 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.879 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.879 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.879 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.879 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.879 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.879 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.880 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:13.880 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:13.880 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.880 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.880 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.880 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.880 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.880 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.880 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.880 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.880 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.880 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.880 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.881 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.881 INFO:teuthology.orchestra.run.smithi114.stdout: "orchestrator": { 2024-09-13T18:34:13.881 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "orchestrator", 2024-09-13T18:34:13.881 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.881 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.881 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.881 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.881 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.881 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.881 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.881 INFO:teuthology.orchestra.run.smithi114.stdout: "cephadm", 2024-09-13T18:34:13.881 INFO:teuthology.orchestra.run.smithi114.stdout: "rook", 2024-09-13T18:34:13.881 INFO:teuthology.orchestra.run.smithi114.stdout: "test_orchestrator" 2024-09-13T18:34:13.882 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.882 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Orchestrator backend", 2024-09-13T18:34:13.882 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.882 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.882 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.882 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.882 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.882 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.882 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:13.882 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "osd_perf_query", 2024-09-13T18:34:13.882 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:13.882 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:13.882 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:13.882 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:13.882 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:13.883 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.883 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.883 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.883 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.883 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.883 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.883 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.883 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.883 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.883 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.883 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.883 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.883 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.884 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.884 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.884 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.884 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.884 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.884 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.884 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:13.884 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:13.884 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.884 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.884 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.884 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.884 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.885 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.885 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.885 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.885 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.885 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.885 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.885 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.885 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:13.885 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:13.885 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.885 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.885 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.885 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:13.886 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.886 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.886 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.886 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.886 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.886 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.886 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.886 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.886 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.886 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.886 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.886 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.886 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.886 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.887 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.887 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:13.887 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:13.887 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.887 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.887 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.887 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.887 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.887 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.887 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.887 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.887 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.887 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.887 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.887 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.888 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.888 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.888 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:13.888 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "osd_support", 2024-09-13T18:34:13.888 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:13.888 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:13.888 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:13.888 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:13.888 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:13.888 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.888 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.888 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.888 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.888 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.888 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.889 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.889 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.889 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.889 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.889 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.889 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.889 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.889 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.889 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.889 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.889 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.889 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.889 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.889 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:13.890 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:13.890 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.890 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.890 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.890 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.890 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.890 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.890 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.890 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.890 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.890 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.890 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.890 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.890 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:13.891 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:13.891 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.891 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.891 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.891 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:13.891 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.891 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.891 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.891 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.891 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.891 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.891 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.891 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.891 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.892 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.892 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.892 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.892 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.892 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.892 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.892 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:13.892 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:13.892 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.892 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.892 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.892 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.892 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.893 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.893 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.893 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.893 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.893 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.893 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.893 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.893 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.893 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.893 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:13.893 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "pg_autoscaler", 2024-09-13T18:34:13.893 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:13.893 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:13.893 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:13.894 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:13.894 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:13.894 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.894 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.894 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.894 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.894 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.894 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.894 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.894 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.894 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.894 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.894 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.894 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.895 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.895 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.895 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.895 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.895 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.895 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.895 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.895 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:13.895 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:13.895 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.895 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.895 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.895 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.895 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.896 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.896 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.896 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.896 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.896 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.896 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.896 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.896 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:13.896 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:13.896 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.896 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.896 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.896 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:13.896 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.897 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.897 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.897 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.897 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.897 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.897 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.897 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.897 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.897 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.897 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.897 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.897 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.897 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.897 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.897 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:13.898 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:13.898 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.898 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.898 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.898 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.898 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.898 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.898 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.898 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.898 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.898 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.898 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.898 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.898 INFO:teuthology.orchestra.run.smithi114.stdout: "sleep_interval": { 2024-09-13T18:34:13.899 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sleep_interval", 2024-09-13T18:34:13.899 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:13.899 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.899 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.899 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "60", 2024-09-13T18:34:13.899 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.899 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.899 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.899 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.899 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.899 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.899 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.899 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.900 INFO:teuthology.orchestra.run.smithi114.stdout: "threshold": { 2024-09-13T18:34:13.900 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "threshold", 2024-09-13T18:34:13.900 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "float", 2024-09-13T18:34:13.900 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.900 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.900 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "3.0", 2024-09-13T18:34:13.900 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "1.0", 2024-09-13T18:34:13.900 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.900 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.900 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "scaling threshold", 2024-09-13T18:34:13.900 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-13T18:34:13.900 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.900 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.900 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.901 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.901 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.901 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:13.901 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "progress", 2024-09-13T18:34:13.901 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:13.901 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:13.901 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:13.901 INFO:teuthology.orchestra.run.smithi114.stdout: "allow_pg_recovery_event": { 2024-09-13T18:34:13.901 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "allow_pg_recovery_event", 2024-09-13T18:34:13.901 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.901 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.901 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.901 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.901 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.902 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.902 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.902 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-13T18:34:13.902 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.902 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.902 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.902 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.902 INFO:teuthology.orchestra.run.smithi114.stdout: "enabled": { 2024-09-13T18:34:13.902 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "enabled", 2024-09-13T18:34:13.902 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.902 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.902 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.902 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:13.903 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.903 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.903 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.903 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.903 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.903 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.903 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.903 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.903 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:13.903 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:13.903 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.903 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.903 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.903 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.904 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.904 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.904 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.904 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.904 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.904 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.904 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.904 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.904 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.904 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.904 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.904 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.904 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.904 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.905 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.905 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:13.905 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:13.905 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.905 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.905 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.905 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.905 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.905 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.905 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.905 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.905 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.905 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.905 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.906 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.906 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:13.906 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:13.906 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.906 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.906 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.906 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:13.906 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.906 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.906 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.906 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.906 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.906 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.908 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.908 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.909 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.909 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.909 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.909 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.909 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.909 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.909 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.909 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:13.909 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:13.909 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.909 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.909 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.909 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.910 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.910 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.910 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.910 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.910 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.910 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.910 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.910 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.910 INFO:teuthology.orchestra.run.smithi114.stdout: "max_completed_events": { 2024-09-13T18:34:13.910 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "max_completed_events", 2024-09-13T18:34:13.910 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:13.910 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.910 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.910 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "50", 2024-09-13T18:34:13.911 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.911 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.911 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.911 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "number of past completed events to remember", 2024-09-13T18:34:13.911 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.911 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.911 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.911 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.911 INFO:teuthology.orchestra.run.smithi114.stdout: "sleep_interval": { 2024-09-13T18:34:13.911 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sleep_interval", 2024-09-13T18:34:13.911 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:13.911 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.911 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.912 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "5", 2024-09-13T18:34:13.912 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.912 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.912 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.912 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "how long the module is going to sleep", 2024-09-13T18:34:13.912 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.912 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.912 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.912 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.912 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.912 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.912 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:13.912 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "prometheus", 2024-09-13T18:34:13.912 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:13.913 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:13.913 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:13.913 INFO:teuthology.orchestra.run.smithi114.stdout: "cache": { 2024-09-13T18:34:13.913 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "cache", 2024-09-13T18:34:13.913 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.913 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.913 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.913 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:13.913 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.913 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.913 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.913 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.913 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.914 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.914 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.914 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.914 INFO:teuthology.orchestra.run.smithi114.stdout: "exclude_perf_counters": { 2024-09-13T18:34:13.914 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "exclude_perf_counters", 2024-09-13T18:34:13.914 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.914 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.914 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.914 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:13.914 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.914 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.914 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.914 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-13T18:34:13.915 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-13T18:34:13.915 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.915 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.915 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.915 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:13.915 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:13.915 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.915 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.915 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.915 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.915 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.915 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.916 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.916 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.916 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.916 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.916 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.916 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.916 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.916 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.916 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.916 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.916 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.916 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.916 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.916 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:13.917 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:13.917 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.917 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.917 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.917 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.917 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.917 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.917 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.917 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.917 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.917 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.917 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.917 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.918 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:13.918 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:13.918 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.918 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.918 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.918 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:13.918 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.918 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.918 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.918 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.918 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.918 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.918 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.918 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.918 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.919 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.919 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.919 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.919 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.919 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.919 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.919 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:13.919 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:13.919 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.919 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.919 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.919 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.919 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.919 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.920 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.920 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.920 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.920 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.920 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.920 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.920 INFO:teuthology.orchestra.run.smithi114.stdout: "rbd_stats_pools": { 2024-09-13T18:34:13.920 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rbd_stats_pools", 2024-09-13T18:34:13.920 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.920 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.920 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.920 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.921 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.921 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.921 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.921 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.921 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.921 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.921 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.921 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.921 INFO:teuthology.orchestra.run.smithi114.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-13T18:34:13.921 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-13T18:34:13.921 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:13.921 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.921 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.921 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "300", 2024-09-13T18:34:13.922 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.922 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.922 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.922 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.922 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.922 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.922 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.922 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.922 INFO:teuthology.orchestra.run.smithi114.stdout: "scrape_interval": { 2024-09-13T18:34:13.922 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "scrape_interval", 2024-09-13T18:34:13.922 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "float", 2024-09-13T18:34:13.922 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.922 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.923 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "15.0", 2024-09-13T18:34:13.923 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.923 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.923 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.923 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.923 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.923 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.923 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.923 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.923 INFO:teuthology.orchestra.run.smithi114.stdout: "server_addr": { 2024-09-13T18:34:13.923 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "server_addr", 2024-09-13T18:34:13.923 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.923 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.923 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.924 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "::", 2024-09-13T18:34:13.924 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.924 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.924 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.924 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-13T18:34:13.924 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.924 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.924 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.924 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.924 INFO:teuthology.orchestra.run.smithi114.stdout: "server_port": { 2024-09-13T18:34:13.924 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "server_port", 2024-09-13T18:34:13.924 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:13.924 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.924 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.925 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "9283", 2024-09-13T18:34:13.925 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.925 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.925 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.925 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-13T18:34:13.925 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.925 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.925 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.925 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.925 INFO:teuthology.orchestra.run.smithi114.stdout: "stale_cache_strategy": { 2024-09-13T18:34:13.925 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "stale_cache_strategy", 2024-09-13T18:34:13.925 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.926 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.926 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.926 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "log", 2024-09-13T18:34:13.926 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.926 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.926 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.926 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.926 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.926 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.926 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.926 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.926 INFO:teuthology.orchestra.run.smithi114.stdout: "standby_behaviour": { 2024-09-13T18:34:13.926 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "standby_behaviour", 2024-09-13T18:34:13.926 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.926 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.927 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.927 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "default", 2024-09-13T18:34:13.927 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.927 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.927 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.927 INFO:teuthology.orchestra.run.smithi114.stdout: "default", 2024-09-13T18:34:13.927 INFO:teuthology.orchestra.run.smithi114.stdout: "error" 2024-09-13T18:34:13.927 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.927 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.927 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.927 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.927 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.927 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.928 INFO:teuthology.orchestra.run.smithi114.stdout: "standby_error_status_code": { 2024-09-13T18:34:13.928 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "standby_error_status_code", 2024-09-13T18:34:13.928 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:13.928 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.928 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.928 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "500", 2024-09-13T18:34:13.928 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "400", 2024-09-13T18:34:13.928 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "599", 2024-09-13T18:34:13.928 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.928 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.928 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.928 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.928 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.928 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.929 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.929 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.929 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:13.929 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rbd_support", 2024-09-13T18:34:13.929 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:13.929 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:13.929 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:13.929 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:13.929 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:13.929 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.929 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.929 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.929 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.930 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.930 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.930 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.930 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.930 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.930 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.930 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.930 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.930 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.930 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.930 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.930 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.930 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.930 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.931 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.931 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:13.931 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:13.931 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.931 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.931 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.931 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.931 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.931 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.931 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.931 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.931 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.931 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.931 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.931 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.932 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:13.932 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:13.932 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.932 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.932 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.932 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:13.932 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.932 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.932 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.932 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.932 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.932 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.932 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.932 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.933 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.933 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.933 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.933 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.933 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.933 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.933 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.933 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:13.933 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:13.933 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.933 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.933 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.933 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.933 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.934 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.934 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.934 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.934 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.934 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.934 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.934 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.934 INFO:teuthology.orchestra.run.smithi114.stdout: "max_concurrent_snap_create": { 2024-09-13T18:34:13.934 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "max_concurrent_snap_create", 2024-09-13T18:34:13.934 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:13.934 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.934 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.934 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "10", 2024-09-13T18:34:13.935 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.935 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.935 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.935 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.935 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.935 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.935 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.935 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.935 INFO:teuthology.orchestra.run.smithi114.stdout: "mirror_snapshot_schedule": { 2024-09-13T18:34:13.935 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "mirror_snapshot_schedule", 2024-09-13T18:34:13.935 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.935 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.935 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.936 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.936 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.936 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.936 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.936 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.936 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.936 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.936 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.936 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.936 INFO:teuthology.orchestra.run.smithi114.stdout: "trash_purge_schedule": { 2024-09-13T18:34:13.936 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "trash_purge_schedule", 2024-09-13T18:34:13.936 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.936 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.936 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.937 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.937 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.937 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.937 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.937 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.937 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.937 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.937 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.937 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.937 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.937 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.937 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:13.937 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "restful", 2024-09-13T18:34:13.937 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:13.938 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:13.938 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:13.938 INFO:teuthology.orchestra.run.smithi114.stdout: "enable_auth": { 2024-09-13T18:34:13.938 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "enable_auth", 2024-09-13T18:34:13.938 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.938 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.938 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.938 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:13.938 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.938 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.938 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.938 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.939 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.939 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.939 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.939 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.939 INFO:teuthology.orchestra.run.smithi114.stdout: "key_file": { 2024-09-13T18:34:13.939 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "key_file", 2024-09-13T18:34:13.939 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.939 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.939 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.939 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.939 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.939 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.939 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.939 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.939 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.940 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.940 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.940 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.940 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:13.940 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:13.940 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.940 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.940 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.940 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.940 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.940 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.940 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.940 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.941 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.941 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.941 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.941 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.941 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.941 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.941 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.941 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.941 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.941 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.941 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.941 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:13.941 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:13.941 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.942 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.942 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.942 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.942 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.942 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.942 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.942 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.942 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.942 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.942 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.942 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.942 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:13.942 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:13.943 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.943 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.943 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.943 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:13.943 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.943 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.943 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.943 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.943 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.943 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.944 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.944 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.944 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.944 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.944 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.944 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.944 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.944 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.944 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.944 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:13.944 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:13.944 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.945 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.945 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.945 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.945 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.945 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.945 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.945 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.945 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.945 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.945 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.945 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.945 INFO:teuthology.orchestra.run.smithi114.stdout: "server_addr": { 2024-09-13T18:34:13.945 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "server_addr", 2024-09-13T18:34:13.945 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.946 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.946 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.946 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.946 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.946 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.946 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.946 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.946 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.946 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.946 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.946 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.946 INFO:teuthology.orchestra.run.smithi114.stdout: "server_port": { 2024-09-13T18:34:13.946 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "server_port", 2024-09-13T18:34:13.947 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.947 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.947 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.947 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.947 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.947 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.947 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.947 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.947 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.947 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.947 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.947 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.947 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.948 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.948 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:13.948 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rgw", 2024-09-13T18:34:13.948 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:13.948 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:13.948 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:13.948 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:13.948 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:13.948 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.948 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.948 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.948 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.949 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.949 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.949 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.949 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.949 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.949 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.949 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.949 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.949 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.949 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.949 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.949 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.949 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.950 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.950 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.950 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:13.950 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:13.950 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.950 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.950 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.950 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.950 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.950 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.950 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.950 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.950 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.951 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.951 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.951 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.951 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:13.951 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:13.951 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.951 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.951 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.951 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:13.951 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.951 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.951 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.951 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.952 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.952 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.952 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.952 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.952 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.952 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.952 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.952 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.952 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.952 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.952 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.952 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:13.952 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:13.953 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.953 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.953 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.953 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.953 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.953 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.953 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.953 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.953 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.953 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.953 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.953 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.954 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.954 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.954 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:13.954 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rook", 2024-09-13T18:34:13.954 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:13.954 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:13.954 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:13.954 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:13.954 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:13.954 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.954 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.954 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.954 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.954 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.955 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.955 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.955 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.955 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.955 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.955 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.955 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.955 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.955 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.955 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.955 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.955 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.955 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.955 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.955 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:13.956 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:13.956 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.956 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.956 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.956 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.956 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.956 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.956 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.956 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.956 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.956 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.956 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.956 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.957 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:13.957 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:13.957 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.957 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.957 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.957 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:13.957 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.957 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.957 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.957 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.957 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.957 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.957 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.957 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.957 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.958 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.958 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.958 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.958 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.958 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.958 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.958 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:13.958 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:13.958 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.958 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.958 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.958 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.959 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.959 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.959 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.959 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.959 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.959 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.959 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.959 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.959 INFO:teuthology.orchestra.run.smithi114.stdout: "storage_class": { 2024-09-13T18:34:13.959 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "storage_class", 2024-09-13T18:34:13.959 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.959 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.959 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.959 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "local", 2024-09-13T18:34:13.960 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.960 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.960 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.960 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-13T18:34:13.960 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.960 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.960 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.960 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.960 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.960 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.960 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:13.960 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "selftest", 2024-09-13T18:34:13.960 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:13.961 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:13.961 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:13.961 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:13.961 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:13.961 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.961 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.961 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.961 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.961 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.961 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.961 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.961 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.961 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.961 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.962 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.962 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.962 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.962 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.962 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.962 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.962 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.962 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.962 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.962 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:13.962 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:13.962 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.962 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.962 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.963 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.963 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.963 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.963 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.963 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.963 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.963 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.963 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.963 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.963 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:13.963 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:13.963 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.963 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.963 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.964 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:13.964 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.964 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.964 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.964 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.964 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.964 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.964 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.964 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.964 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.964 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.964 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.964 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.965 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.965 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.965 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.965 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:13.965 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:13.965 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.965 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.965 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.965 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.965 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.965 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.965 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.965 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.965 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.966 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.966 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.966 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.966 INFO:teuthology.orchestra.run.smithi114.stdout: "roption1": { 2024-09-13T18:34:13.966 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "roption1", 2024-09-13T18:34:13.966 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.966 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.966 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.966 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.966 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.966 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.966 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.966 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.966 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.967 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.967 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.967 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.967 INFO:teuthology.orchestra.run.smithi114.stdout: "roption2": { 2024-09-13T18:34:13.967 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "roption2", 2024-09-13T18:34:13.967 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.967 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.967 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.967 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "xyz", 2024-09-13T18:34:13.967 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.967 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.967 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.967 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.967 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.968 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.968 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.968 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.968 INFO:teuthology.orchestra.run.smithi114.stdout: "rwoption1": { 2024-09-13T18:34:13.968 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rwoption1", 2024-09-13T18:34:13.968 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.968 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.968 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.968 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.968 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.968 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.968 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.968 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.969 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.969 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.969 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.969 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.969 INFO:teuthology.orchestra.run.smithi114.stdout: "rwoption2": { 2024-09-13T18:34:13.969 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rwoption2", 2024-09-13T18:34:13.969 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:13.969 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.969 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.969 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.969 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.969 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.969 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.969 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.970 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.970 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.970 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.970 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.970 INFO:teuthology.orchestra.run.smithi114.stdout: "rwoption3": { 2024-09-13T18:34:13.970 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rwoption3", 2024-09-13T18:34:13.970 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "float", 2024-09-13T18:34:13.970 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.970 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.970 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.970 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.970 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.970 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.970 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.970 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.971 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.971 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.971 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.971 INFO:teuthology.orchestra.run.smithi114.stdout: "rwoption4": { 2024-09-13T18:34:13.971 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rwoption4", 2024-09-13T18:34:13.971 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.971 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.971 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.971 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.971 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.971 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.971 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.971 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.971 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.972 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.972 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.972 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.972 INFO:teuthology.orchestra.run.smithi114.stdout: "rwoption5": { 2024-09-13T18:34:13.972 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rwoption5", 2024-09-13T18:34:13.972 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.972 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.972 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.972 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.972 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.972 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.972 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.972 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.973 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.973 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.973 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.973 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.973 INFO:teuthology.orchestra.run.smithi114.stdout: "rwoption6": { 2024-09-13T18:34:13.973 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rwoption6", 2024-09-13T18:34:13.973 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.973 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.973 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.973 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:13.973 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.973 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.973 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.973 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.974 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.974 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.974 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.974 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.974 INFO:teuthology.orchestra.run.smithi114.stdout: "rwoption7": { 2024-09-13T18:34:13.974 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rwoption7", 2024-09-13T18:34:13.974 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:13.974 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.974 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.974 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.974 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "1", 2024-09-13T18:34:13.974 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "42", 2024-09-13T18:34:13.974 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.974 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.974 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.975 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.975 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.975 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.975 INFO:teuthology.orchestra.run.smithi114.stdout: "testkey": { 2024-09-13T18:34:13.975 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "testkey", 2024-09-13T18:34:13.975 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.975 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.975 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.975 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.975 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.975 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.975 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.975 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.976 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.976 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.976 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.976 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.976 INFO:teuthology.orchestra.run.smithi114.stdout: "testlkey": { 2024-09-13T18:34:13.976 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "testlkey", 2024-09-13T18:34:13.976 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.976 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.976 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.976 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.976 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.976 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.976 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.976 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.977 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.977 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.977 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.977 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.977 INFO:teuthology.orchestra.run.smithi114.stdout: "testnewline": { 2024-09-13T18:34:13.977 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "testnewline", 2024-09-13T18:34:13.977 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.977 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.977 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.977 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.977 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.977 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.977 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.977 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.978 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.978 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.978 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.978 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.978 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.978 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.978 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:13.978 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "snap_schedule", 2024-09-13T18:34:13.978 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:13.978 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:13.978 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:13.978 INFO:teuthology.orchestra.run.smithi114.stdout: "allow_m_granularity": { 2024-09-13T18:34:13.978 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "allow_m_granularity", 2024-09-13T18:34:13.978 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.979 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.979 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.979 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.979 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.979 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.979 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.979 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "allow minute scheduled snapshots", 2024-09-13T18:34:13.979 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.979 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.979 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.979 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.979 INFO:teuthology.orchestra.run.smithi114.stdout: "dump_on_update": { 2024-09-13T18:34:13.979 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "dump_on_update", 2024-09-13T18:34:13.979 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.980 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.980 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.980 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.980 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.980 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.980 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.980 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "dump database to debug log on update", 2024-09-13T18:34:13.980 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.980 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.980 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.980 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.980 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:13.980 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:13.980 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.981 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.981 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.981 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.981 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.981 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.981 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.981 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.981 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.981 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.981 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.981 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.981 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.981 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.982 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.982 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.982 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.982 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.982 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.982 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:13.982 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:13.982 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.982 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.982 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.982 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.982 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.982 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.982 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.983 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.983 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.983 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.983 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.983 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.983 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:13.983 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:13.983 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.983 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.983 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.983 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:13.983 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.983 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.983 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.983 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.984 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.984 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.984 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.984 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.984 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.984 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.984 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.984 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.984 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.984 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.984 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.984 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:13.984 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:13.984 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.985 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.985 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.985 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.985 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.985 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.985 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.985 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.985 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.985 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.985 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.985 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.985 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.985 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.985 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:13.986 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "stats", 2024-09-13T18:34:13.986 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:13.986 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:13.986 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:13.986 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:13.986 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:13.986 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.986 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.986 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.986 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.986 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.986 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.986 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.987 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.987 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.987 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.987 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.987 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.987 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.987 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.987 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.987 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.987 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.987 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.987 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.987 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:13.987 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:13.988 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.988 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.988 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.988 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.988 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.988 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.988 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.988 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.988 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.988 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.988 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.988 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.988 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:13.988 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:13.989 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.989 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.989 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.989 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:13.989 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.989 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.989 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.989 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.989 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.989 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.989 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.989 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.989 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.989 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.990 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.990 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.990 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.990 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.990 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.990 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:13.990 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:13.990 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.990 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.990 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.990 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.990 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.990 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.990 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.991 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.991 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.991 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.991 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.991 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.991 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.991 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.991 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:13.991 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "status", 2024-09-13T18:34:13.991 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:13.991 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:13.991 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:13.991 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:13.991 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:13.991 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.992 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.992 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.992 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.992 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.992 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.992 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.992 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.992 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.992 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.992 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.992 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.992 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.992 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.992 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.993 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.993 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.993 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.993 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.993 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:13.993 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:13.993 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.993 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.993 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.993 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.993 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.993 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.993 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.994 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.994 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.994 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.994 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.994 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.994 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:13.994 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:13.994 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.994 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.994 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.994 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:13.994 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.994 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.995 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.995 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.995 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:13.995 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:13.995 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:13.995 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:13.995 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:13.995 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:13.995 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.995 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.995 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.995 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.995 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.995 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:13.995 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:13.995 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:13.996 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.996 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.996 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:13.996 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.996 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.996 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.996 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.996 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.996 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.996 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.996 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.996 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:13.996 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.996 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:13.996 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "telegraf", 2024-09-13T18:34:13.997 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:13.997 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:13.997 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:13.997 INFO:teuthology.orchestra.run.smithi114.stdout: "address": { 2024-09-13T18:34:13.997 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "address", 2024-09-13T18:34:13.997 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.997 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.997 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.997 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-13T18:34:13.997 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.997 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.997 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.997 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.997 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.998 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.998 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.998 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.998 INFO:teuthology.orchestra.run.smithi114.stdout: "interval": { 2024-09-13T18:34:13.998 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "interval", 2024-09-13T18:34:13.998 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:13.998 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.998 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:13.998 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "15", 2024-09-13T18:34:13.998 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.998 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.998 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:13.998 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:13.998 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:13.999 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:13.999 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:13.999 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:13.999 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:13.999 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:13.999 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:13.999 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:13.999 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:13.999 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:13.999 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:13.999 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:13.999 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:13.999 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:13.999 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.000 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.000 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.000 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.000 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.000 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.000 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.000 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.000 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.000 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.000 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.000 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.001 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.001 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.001 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.001 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.001 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.001 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.001 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.001 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.001 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.001 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.001 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.001 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.001 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.002 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.002 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.002 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.002 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.002 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.002 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.002 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.002 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.002 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.002 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.002 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.002 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.002 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.002 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.003 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.003 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.003 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.003 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.003 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.003 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.003 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.003 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.003 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.003 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.003 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.003 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.003 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.003 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.004 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.004 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.004 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.004 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.004 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.004 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.004 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.004 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.004 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.004 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.004 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "telemetry", 2024-09-13T18:34:14.004 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.004 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.004 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.005 INFO:teuthology.orchestra.run.smithi114.stdout: "channel_basic": { 2024-09-13T18:34:14.005 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "channel_basic", 2024-09-13T18:34:14.005 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.005 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.005 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.005 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.005 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.005 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.005 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.005 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-13T18:34:14.005 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.005 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.005 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.005 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.006 INFO:teuthology.orchestra.run.smithi114.stdout: "channel_crash": { 2024-09-13T18:34:14.006 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "channel_crash", 2024-09-13T18:34:14.006 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.006 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.006 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.006 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.006 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.006 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.006 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.006 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-13T18:34:14.006 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.006 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.006 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.006 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.007 INFO:teuthology.orchestra.run.smithi114.stdout: "channel_device": { 2024-09-13T18:34:14.007 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "channel_device", 2024-09-13T18:34:14.007 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.007 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.007 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.007 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.007 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.007 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.007 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.007 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-13T18:34:14.007 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.007 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.007 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.007 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.007 INFO:teuthology.orchestra.run.smithi114.stdout: "channel_ident": { 2024-09-13T18:34:14.008 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "channel_ident", 2024-09-13T18:34:14.008 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.008 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.008 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.008 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.008 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.008 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.008 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.008 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-13T18:34:14.008 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.008 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.008 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.008 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.008 INFO:teuthology.orchestra.run.smithi114.stdout: "channel_perf": { 2024-09-13T18:34:14.009 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "channel_perf", 2024-09-13T18:34:14.009 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.009 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.009 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.009 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.009 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.009 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.009 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.009 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-13T18:34:14.009 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.009 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.009 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.010 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.010 INFO:teuthology.orchestra.run.smithi114.stdout: "contact": { 2024-09-13T18:34:14.010 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "contact", 2024-09-13T18:34:14.010 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.010 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.010 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.010 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.010 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.010 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.010 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.010 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.010 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.010 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.010 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.011 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.011 INFO:teuthology.orchestra.run.smithi114.stdout: "description": { 2024-09-13T18:34:14.011 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "description", 2024-09-13T18:34:14.011 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.011 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.011 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.011 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.011 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.011 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.011 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.011 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.011 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.011 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.011 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.012 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.012 INFO:teuthology.orchestra.run.smithi114.stdout: "device_url": { 2024-09-13T18:34:14.012 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "device_url", 2024-09-13T18:34:14.012 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.012 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.012 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.012 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-13T18:34:14.012 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.012 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.012 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.012 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.012 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.012 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.013 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.013 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.013 INFO:teuthology.orchestra.run.smithi114.stdout: "enabled": { 2024-09-13T18:34:14.013 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "enabled", 2024-09-13T18:34:14.013 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.013 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.013 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.013 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.013 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.013 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.013 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.013 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.013 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.013 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.014 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.014 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.014 INFO:teuthology.orchestra.run.smithi114.stdout: "interval": { 2024-09-13T18:34:14.014 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "interval", 2024-09-13T18:34:14.014 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.014 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.014 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.014 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "24", 2024-09-13T18:34:14.014 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "8", 2024-09-13T18:34:14.014 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.014 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.014 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.014 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.014 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.015 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.015 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.015 INFO:teuthology.orchestra.run.smithi114.stdout: "last_opt_revision": { 2024-09-13T18:34:14.015 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "last_opt_revision", 2024-09-13T18:34:14.015 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.015 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.015 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.015 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "1", 2024-09-13T18:34:14.015 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.015 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.015 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.015 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.015 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.015 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.016 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.016 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.016 INFO:teuthology.orchestra.run.smithi114.stdout: "leaderboard": { 2024-09-13T18:34:14.016 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "leaderboard", 2024-09-13T18:34:14.016 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.016 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.016 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.016 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.016 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.016 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.016 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.016 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.016 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.016 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.017 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.017 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.017 INFO:teuthology.orchestra.run.smithi114.stdout: "leaderboard_description": { 2024-09-13T18:34:14.017 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "leaderboard_description", 2024-09-13T18:34:14.017 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.017 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.017 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.017 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.017 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.017 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.017 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.017 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.017 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.017 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.018 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.018 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.018 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.018 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.018 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.018 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.018 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.018 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.018 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.018 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.018 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.018 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.018 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.018 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.019 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.019 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.019 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.019 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.019 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.019 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.019 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.019 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.019 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.019 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.019 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.019 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.019 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.019 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.019 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.020 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.020 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.020 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.020 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.020 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.020 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.020 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.020 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.020 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.020 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.020 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.020 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.020 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.021 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.021 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.021 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.021 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.021 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.021 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.021 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.021 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.021 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.021 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.021 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.021 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.021 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.021 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.022 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.022 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.022 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.022 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.022 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.022 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.022 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.022 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.022 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.022 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.022 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.022 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.022 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.022 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.022 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.023 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.023 INFO:teuthology.orchestra.run.smithi114.stdout: "organization": { 2024-09-13T18:34:14.023 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "organization", 2024-09-13T18:34:14.023 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.023 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.023 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.023 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.023 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.023 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.023 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.023 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.023 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.023 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.023 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.024 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.024 INFO:teuthology.orchestra.run.smithi114.stdout: "proxy": { 2024-09-13T18:34:14.024 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "proxy", 2024-09-13T18:34:14.024 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.024 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.024 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.024 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.024 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.024 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.024 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.024 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.024 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.024 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.024 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.024 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.025 INFO:teuthology.orchestra.run.smithi114.stdout: "url": { 2024-09-13T18:34:14.025 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "url", 2024-09-13T18:34:14.025 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.025 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.025 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.025 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-13T18:34:14.025 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.025 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.025 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.025 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.025 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.025 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.025 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.025 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.026 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.026 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.026 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.026 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "test_orchestrator", 2024-09-13T18:34:14.026 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.026 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.026 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.026 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.026 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.026 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.026 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.026 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.026 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.026 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.026 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.027 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.027 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.027 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.027 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.027 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.027 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.027 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.027 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.027 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.027 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.027 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.027 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.027 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.028 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.028 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.028 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.028 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.028 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.028 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.028 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.028 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.028 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.028 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.028 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.028 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.028 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.029 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.029 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.029 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.029 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.029 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.029 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.029 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.029 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.029 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.029 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.029 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.029 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.029 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.029 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.030 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.030 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.030 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.030 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.030 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.030 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.030 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.030 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.030 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.030 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.030 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.030 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.030 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.030 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.030 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.031 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.031 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.031 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.031 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.031 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.031 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.031 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.031 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.031 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.031 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.031 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "volumes", 2024-09-13T18:34:14.031 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.031 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.031 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.032 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.032 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.032 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.032 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.032 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.032 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.032 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.032 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.032 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.032 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.032 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.032 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.032 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.032 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.033 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.033 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.033 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.033 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.033 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.033 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.033 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.033 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.033 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.033 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.033 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.033 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.033 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.034 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.034 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.034 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.034 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.034 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.034 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.034 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.034 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.034 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.034 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.034 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.034 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.034 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.035 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.035 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.035 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.035 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.035 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.035 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.035 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.035 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.035 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.035 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.035 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.035 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.035 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.036 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.036 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.036 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.036 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.036 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.036 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.036 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.036 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.036 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.036 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.036 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.036 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.037 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.037 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.037 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.037 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.037 INFO:teuthology.orchestra.run.smithi114.stdout: "max_concurrent_clones": { 2024-09-13T18:34:14.037 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "max_concurrent_clones", 2024-09-13T18:34:14.037 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.037 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.037 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.037 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "4", 2024-09-13T18:34:14.037 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.037 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.037 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.038 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-13T18:34:14.038 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.038 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.038 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.038 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.038 INFO:teuthology.orchestra.run.smithi114.stdout: "snapshot_clone_delay": { 2024-09-13T18:34:14.038 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "snapshot_clone_delay", 2024-09-13T18:34:14.038 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.038 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.038 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.038 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-13T18:34:14.038 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.038 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.038 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.039 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-13T18:34:14.039 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.039 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.039 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.039 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.039 INFO:teuthology.orchestra.run.smithi114.stdout: "snapshot_clone_no_wait": { 2024-09-13T18:34:14.039 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "snapshot_clone_no_wait", 2024-09-13T18:34:14.039 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.039 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.039 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.039 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.039 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.039 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.039 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.040 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-13T18:34:14.040 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.040 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.040 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.040 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.040 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.040 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.040 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.040 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "zabbix", 2024-09-13T18:34:14.040 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.040 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.040 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.040 INFO:teuthology.orchestra.run.smithi114.stdout: "discovery_interval": { 2024-09-13T18:34:14.041 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "discovery_interval", 2024-09-13T18:34:14.041 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "uint", 2024-09-13T18:34:14.041 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.041 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.041 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "100", 2024-09-13T18:34:14.041 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.041 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.041 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.041 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.041 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.041 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.041 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.041 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.042 INFO:teuthology.orchestra.run.smithi114.stdout: "identifier": { 2024-09-13T18:34:14.042 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "identifier", 2024-09-13T18:34:14.042 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.042 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.042 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.042 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.042 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.042 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.042 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.042 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.042 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.042 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.042 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.043 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.043 INFO:teuthology.orchestra.run.smithi114.stdout: "interval": { 2024-09-13T18:34:14.043 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "interval", 2024-09-13T18:34:14.043 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:14.043 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.043 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.043 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "60", 2024-09-13T18:34:14.043 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.043 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.043 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.043 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.043 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.043 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.043 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.044 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.044 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.044 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.044 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.044 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.044 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.044 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.044 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.044 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.044 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.044 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.044 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.044 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.045 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.045 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.045 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.045 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.045 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.045 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.045 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.045 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.045 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.045 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.045 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.045 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.045 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.045 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.046 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.046 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.046 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.046 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.046 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.046 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.046 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.046 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.046 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.046 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.046 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.046 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.046 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.047 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.047 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.047 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.047 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.047 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.047 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.047 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.047 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.047 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.047 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.047 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.047 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.047 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.047 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.048 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.048 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.048 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.048 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.048 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.048 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.048 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.048 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.048 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.048 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.048 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.048 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.048 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.049 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.049 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.049 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.049 INFO:teuthology.orchestra.run.smithi114.stdout: "zabbix_host": { 2024-09-13T18:34:14.049 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "zabbix_host", 2024-09-13T18:34:14.049 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.049 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.049 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.049 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.049 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.049 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.049 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.049 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.049 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.050 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.050 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.050 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.050 INFO:teuthology.orchestra.run.smithi114.stdout: "zabbix_port": { 2024-09-13T18:34:14.050 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "zabbix_port", 2024-09-13T18:34:14.050 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.050 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.050 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.050 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "10051", 2024-09-13T18:34:14.050 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.050 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.050 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.050 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.050 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.051 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.051 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.051 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.051 INFO:teuthology.orchestra.run.smithi114.stdout: "zabbix_sender": { 2024-09-13T18:34:14.051 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "zabbix_sender", 2024-09-13T18:34:14.051 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.051 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.051 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.051 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-13T18:34:14.051 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.051 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.051 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.051 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.051 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.052 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.052 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.052 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.052 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.052 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.052 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-13T18:34:14.052 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.052 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.052 INFO:teuthology.orchestra.run.smithi114.stdout: "gid": 25102, 2024-09-13T18:34:14.052 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "z", 2024-09-13T18:34:14.052 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr_features": 4540138322906710015, 2024-09-13T18:34:14.052 INFO:teuthology.orchestra.run.smithi114.stdout: "available_modules": [ 2024-09-13T18:34:14.052 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.052 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "alerts", 2024-09-13T18:34:14.053 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.053 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.053 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.053 INFO:teuthology.orchestra.run.smithi114.stdout: "interval": { 2024-09-13T18:34:14.053 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "interval", 2024-09-13T18:34:14.053 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:14.053 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.053 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.053 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "60", 2024-09-13T18:34:14.053 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.053 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.053 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.053 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "How frequently to reexamine health status", 2024-09-13T18:34:14.053 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.054 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.054 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.054 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.054 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.054 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.054 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.054 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.054 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.054 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.054 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.054 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.054 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.054 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.054 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.055 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.055 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.055 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.055 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.055 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.055 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.055 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.055 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.055 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.055 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.055 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.055 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.055 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.055 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.056 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.056 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.056 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.056 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.056 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.056 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.056 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.056 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.056 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.056 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.056 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.056 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.057 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.057 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.057 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.057 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.057 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.057 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.057 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.057 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.057 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.057 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.057 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.057 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.057 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.057 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.058 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.058 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.058 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.058 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.058 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.058 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.058 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.058 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.058 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.058 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.058 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.058 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.058 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.058 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.059 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.059 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.059 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.059 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.059 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.059 INFO:teuthology.orchestra.run.smithi114.stdout: "smtp_destination": { 2024-09-13T18:34:14.059 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "smtp_destination", 2024-09-13T18:34:14.059 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.059 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.059 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.059 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.059 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.059 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.059 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.060 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Email address to send alerts to", 2024-09-13T18:34:14.060 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.060 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.060 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.060 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.060 INFO:teuthology.orchestra.run.smithi114.stdout: "smtp_from_name": { 2024-09-13T18:34:14.060 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "smtp_from_name", 2024-09-13T18:34:14.060 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.060 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.060 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.060 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "Ceph", 2024-09-13T18:34:14.060 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.060 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.061 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.061 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Email From: name", 2024-09-13T18:34:14.061 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.061 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.061 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.061 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.061 INFO:teuthology.orchestra.run.smithi114.stdout: "smtp_host": { 2024-09-13T18:34:14.061 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "smtp_host", 2024-09-13T18:34:14.061 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.061 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.061 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.061 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.061 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.062 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.062 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.062 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "SMTP server", 2024-09-13T18:34:14.062 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.062 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.062 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.062 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.062 INFO:teuthology.orchestra.run.smithi114.stdout: "smtp_password": { 2024-09-13T18:34:14.062 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "smtp_password", 2024-09-13T18:34:14.062 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.062 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.062 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.062 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.063 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.063 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.063 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.063 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Password to authenticate with", 2024-09-13T18:34:14.063 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.063 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.063 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.063 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.063 INFO:teuthology.orchestra.run.smithi114.stdout: "smtp_port": { 2024-09-13T18:34:14.063 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "smtp_port", 2024-09-13T18:34:14.063 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.063 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.063 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.063 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "465", 2024-09-13T18:34:14.064 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.064 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.064 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.064 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "SMTP port", 2024-09-13T18:34:14.064 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.064 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.064 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.064 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.064 INFO:teuthology.orchestra.run.smithi114.stdout: "smtp_sender": { 2024-09-13T18:34:14.064 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "smtp_sender", 2024-09-13T18:34:14.064 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.064 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.064 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.064 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.065 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.065 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.065 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.065 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "SMTP envelope sender", 2024-09-13T18:34:14.065 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.065 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.065 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.065 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.065 INFO:teuthology.orchestra.run.smithi114.stdout: "smtp_ssl": { 2024-09-13T18:34:14.065 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "smtp_ssl", 2024-09-13T18:34:14.065 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.065 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.065 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.065 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.066 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.066 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.066 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.066 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-13T18:34:14.066 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.066 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.066 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.066 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.066 INFO:teuthology.orchestra.run.smithi114.stdout: "smtp_user": { 2024-09-13T18:34:14.066 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "smtp_user", 2024-09-13T18:34:14.066 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.066 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.066 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.066 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.067 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.067 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.067 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.067 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "User to authenticate as", 2024-09-13T18:34:14.067 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.067 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.067 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.067 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.067 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.067 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.067 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.067 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "balancer", 2024-09-13T18:34:14.068 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.068 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.068 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.068 INFO:teuthology.orchestra.run.smithi114.stdout: "active": { 2024-09-13T18:34:14.068 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "active", 2024-09-13T18:34:14.068 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.068 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.068 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.068 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.068 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.068 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.068 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.068 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "automatically balance PGs across cluster", 2024-09-13T18:34:14.069 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.069 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.069 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.069 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.069 INFO:teuthology.orchestra.run.smithi114.stdout: "begin_time": { 2024-09-13T18:34:14.069 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "begin_time", 2024-09-13T18:34:14.069 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.069 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.069 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.069 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0000", 2024-09-13T18:34:14.069 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.069 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.069 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.069 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "beginning time of day to automatically balance", 2024-09-13T18:34:14.069 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-13T18:34:14.070 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.070 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.070 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.070 INFO:teuthology.orchestra.run.smithi114.stdout: "begin_weekday": { 2024-09-13T18:34:14.070 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "begin_weekday", 2024-09-13T18:34:14.070 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "uint", 2024-09-13T18:34:14.070 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.070 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.070 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-13T18:34:14.070 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "0", 2024-09-13T18:34:14.070 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "6", 2024-09-13T18:34:14.070 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.070 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-13T18:34:14.070 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-13T18:34:14.071 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.071 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.071 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.071 INFO:teuthology.orchestra.run.smithi114.stdout: "crush_compat_max_iterations": { 2024-09-13T18:34:14.071 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "crush_compat_max_iterations", 2024-09-13T18:34:14.071 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "uint", 2024-09-13T18:34:14.071 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.071 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.071 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "25", 2024-09-13T18:34:14.071 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "1", 2024-09-13T18:34:14.071 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "250", 2024-09-13T18:34:14.071 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.071 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-13T18:34:14.072 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.072 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.072 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.072 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.072 INFO:teuthology.orchestra.run.smithi114.stdout: "crush_compat_metrics": { 2024-09-13T18:34:14.072 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "crush_compat_metrics", 2024-09-13T18:34:14.072 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.072 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.072 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.072 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "pgs,objects,bytes", 2024-09-13T18:34:14.072 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.072 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.072 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.072 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-13T18:34:14.073 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-13T18:34:14.073 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.073 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.073 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.073 INFO:teuthology.orchestra.run.smithi114.stdout: "crush_compat_step": { 2024-09-13T18:34:14.073 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "crush_compat_step", 2024-09-13T18:34:14.073 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "float", 2024-09-13T18:34:14.073 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.073 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.073 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0.5", 2024-09-13T18:34:14.073 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "0.001", 2024-09-13T18:34:14.073 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "0.999", 2024-09-13T18:34:14.073 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.074 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "aggressiveness of optimization", 2024-09-13T18:34:14.074 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-13T18:34:14.074 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.074 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.074 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.074 INFO:teuthology.orchestra.run.smithi114.stdout: "end_time": { 2024-09-13T18:34:14.074 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "end_time", 2024-09-13T18:34:14.074 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.074 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.074 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.074 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "2359", 2024-09-13T18:34:14.074 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.074 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.075 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.075 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "ending time of day to automatically balance", 2024-09-13T18:34:14.075 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-13T18:34:14.075 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.075 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.075 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.075 INFO:teuthology.orchestra.run.smithi114.stdout: "end_weekday": { 2024-09-13T18:34:14.075 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "end_weekday", 2024-09-13T18:34:14.075 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "uint", 2024-09-13T18:34:14.075 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.075 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.075 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-13T18:34:14.076 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "0", 2024-09-13T18:34:14.076 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "6", 2024-09-13T18:34:14.076 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.076 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-13T18:34:14.076 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-13T18:34:14.076 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.076 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.076 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.076 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.076 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.076 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.076 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.076 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.076 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.077 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.077 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.077 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.077 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.077 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.077 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.077 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.077 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.077 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.077 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.077 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.077 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.077 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.078 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.078 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.078 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.078 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.078 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.078 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.078 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.078 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.078 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.078 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.078 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.078 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.078 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.079 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.079 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.079 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.079 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.079 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.079 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.079 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.079 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.079 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.079 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.079 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.079 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.079 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.079 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.079 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.080 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.080 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.080 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.080 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.080 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.080 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.080 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.080 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.080 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.080 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.080 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.080 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.080 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.081 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.081 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.081 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.081 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.081 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.081 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.081 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.081 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.081 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.081 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.081 INFO:teuthology.orchestra.run.smithi114.stdout: "min_score": { 2024-09-13T18:34:14.081 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "min_score", 2024-09-13T18:34:14.081 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "float", 2024-09-13T18:34:14.081 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.082 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.082 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-13T18:34:14.082 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.082 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.082 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.082 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-13T18:34:14.082 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.082 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.082 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.082 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.082 INFO:teuthology.orchestra.run.smithi114.stdout: "mode": { 2024-09-13T18:34:14.082 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "mode", 2024-09-13T18:34:14.082 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.082 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.083 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.083 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "upmap", 2024-09-13T18:34:14.083 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.083 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.083 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.083 INFO:teuthology.orchestra.run.smithi114.stdout: "crush-compat", 2024-09-13T18:34:14.083 INFO:teuthology.orchestra.run.smithi114.stdout: "none", 2024-09-13T18:34:14.083 INFO:teuthology.orchestra.run.smithi114.stdout: "upmap" 2024-09-13T18:34:14.083 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.083 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Balancer mode", 2024-09-13T18:34:14.083 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.083 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.083 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.084 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.084 INFO:teuthology.orchestra.run.smithi114.stdout: "pool_ids": { 2024-09-13T18:34:14.084 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "pool_ids", 2024-09-13T18:34:14.084 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.084 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.084 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.084 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.084 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.084 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.084 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.084 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-13T18:34:14.084 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.084 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.084 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.085 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.085 INFO:teuthology.orchestra.run.smithi114.stdout: "sleep_interval": { 2024-09-13T18:34:14.085 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sleep_interval", 2024-09-13T18:34:14.085 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:14.085 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.085 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.085 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "60", 2024-09-13T18:34:14.085 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.085 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.085 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.085 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-13T18:34:14.085 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.085 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.086 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.086 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.086 INFO:teuthology.orchestra.run.smithi114.stdout: "upmap_max_deviation": { 2024-09-13T18:34:14.086 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "upmap_max_deviation", 2024-09-13T18:34:14.086 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.086 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.086 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.086 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "5", 2024-09-13T18:34:14.086 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "1", 2024-09-13T18:34:14.086 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.086 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.086 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-13T18:34:14.086 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-13T18:34:14.087 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.087 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.087 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.087 INFO:teuthology.orchestra.run.smithi114.stdout: "upmap_max_optimizations": { 2024-09-13T18:34:14.087 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "upmap_max_optimizations", 2024-09-13T18:34:14.087 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "uint", 2024-09-13T18:34:14.087 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.087 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.087 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "10", 2024-09-13T18:34:14.087 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.087 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.087 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.087 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-13T18:34:14.087 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.088 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.088 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.088 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.088 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.088 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.088 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.088 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "cephadm", 2024-09-13T18:34:14.088 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.088 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.088 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.088 INFO:teuthology.orchestra.run.smithi114.stdout: "agent_down_multiplier": { 2024-09-13T18:34:14.088 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "agent_down_multiplier", 2024-09-13T18:34:14.088 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "float", 2024-09-13T18:34:14.089 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.089 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.089 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "3.0", 2024-09-13T18:34:14.089 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.089 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.089 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.089 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-13T18:34:14.089 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.089 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.089 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.089 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.089 INFO:teuthology.orchestra.run.smithi114.stdout: "agent_refresh_rate": { 2024-09-13T18:34:14.089 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "agent_refresh_rate", 2024-09-13T18:34:14.090 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:14.090 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.090 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.090 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "20", 2024-09-13T18:34:14.090 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.090 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.090 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.090 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-13T18:34:14.090 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.090 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.090 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.090 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.090 INFO:teuthology.orchestra.run.smithi114.stdout: "agent_starting_port": { 2024-09-13T18:34:14.090 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "agent_starting_port", 2024-09-13T18:34:14.091 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.091 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.091 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.091 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "4721", 2024-09-13T18:34:14.091 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.091 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.091 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.091 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-13T18:34:14.091 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.091 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.091 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.091 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.091 INFO:teuthology.orchestra.run.smithi114.stdout: "allow_ptrace": { 2024-09-13T18:34:14.091 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "allow_ptrace", 2024-09-13T18:34:14.092 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.092 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.092 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.092 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.092 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.092 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.092 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.092 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-13T18:34:14.092 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-13T18:34:14.092 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.092 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.092 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.092 INFO:teuthology.orchestra.run.smithi114.stdout: "autotune_interval": { 2024-09-13T18:34:14.093 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "autotune_interval", 2024-09-13T18:34:14.093 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:14.093 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.093 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.093 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "600", 2024-09-13T18:34:14.093 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.093 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.093 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.093 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-13T18:34:14.093 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.093 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.093 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.094 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.094 INFO:teuthology.orchestra.run.smithi114.stdout: "autotune_memory_target_ratio": { 2024-09-13T18:34:14.094 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "autotune_memory_target_ratio", 2024-09-13T18:34:14.094 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "float", 2024-09-13T18:34:14.094 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.094 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.094 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0.7", 2024-09-13T18:34:14.094 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.094 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.094 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.094 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-13T18:34:14.094 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.094 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.095 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.095 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.095 INFO:teuthology.orchestra.run.smithi114.stdout: "cgroups_split": { 2024-09-13T18:34:14.095 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "cgroups_split", 2024-09-13T18:34:14.095 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.095 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.095 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.095 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.095 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.095 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.095 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.095 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-13T18:34:14.095 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.095 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.096 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.096 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.096 INFO:teuthology.orchestra.run.smithi114.stdout: "config_checks_enabled": { 2024-09-13T18:34:14.096 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "config_checks_enabled", 2024-09-13T18:34:14.096 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.096 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.096 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.096 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.096 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.096 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.096 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.097 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-13T18:34:14.097 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.097 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.097 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.097 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.097 INFO:teuthology.orchestra.run.smithi114.stdout: "config_dashboard": { 2024-09-13T18:34:14.097 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "config_dashboard", 2024-09-13T18:34:14.097 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.097 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.097 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.097 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.097 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.097 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.097 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.098 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-13T18:34:14.098 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.098 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.098 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.098 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.098 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_alertmanager": { 2024-09-13T18:34:14.098 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_alertmanager", 2024-09-13T18:34:14.098 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.098 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.098 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.098 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-13T18:34:14.098 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.098 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.098 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.099 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Prometheus container image", 2024-09-13T18:34:14.099 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.099 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.099 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.099 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.099 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_base": { 2024-09-13T18:34:14.099 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_base", 2024-09-13T18:34:14.099 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.099 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.099 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.099 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-13T18:34:14.099 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.099 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.100 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.100 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Container image name, without the tag", 2024-09-13T18:34:14.100 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.100 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.100 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.100 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.100 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_elasticsearch": { 2024-09-13T18:34:14.100 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_elasticsearch", 2024-09-13T18:34:14.100 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.100 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.100 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.100 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-13T18:34:14.100 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.100 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.101 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.101 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "elasticsearch container image", 2024-09-13T18:34:14.101 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.101 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.101 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.101 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.101 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_grafana": { 2024-09-13T18:34:14.101 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_grafana", 2024-09-13T18:34:14.101 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.101 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.101 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.101 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-09-13T18:34:14.101 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.101 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.102 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.102 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Prometheus container image", 2024-09-13T18:34:14.102 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.102 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.102 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.102 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.102 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_haproxy": { 2024-09-13T18:34:14.102 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_haproxy", 2024-09-13T18:34:14.102 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.102 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.102 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.102 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-13T18:34:14.102 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.103 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.103 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.103 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "HAproxy container image", 2024-09-13T18:34:14.103 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.103 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.103 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.103 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.103 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_jaeger_agent": { 2024-09-13T18:34:14.103 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_jaeger_agent", 2024-09-13T18:34:14.103 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.103 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.103 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.103 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-13T18:34:14.104 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.104 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.104 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.104 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Jaeger agent container image", 2024-09-13T18:34:14.104 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.104 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.104 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.104 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.104 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_jaeger_collector": { 2024-09-13T18:34:14.104 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_jaeger_collector", 2024-09-13T18:34:14.104 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.104 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.104 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.105 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-13T18:34:14.105 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.105 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.105 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.105 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Jaeger collector container image", 2024-09-13T18:34:14.105 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.105 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.105 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.105 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.105 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_jaeger_query": { 2024-09-13T18:34:14.105 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_jaeger_query", 2024-09-13T18:34:14.105 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.105 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.105 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.106 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-13T18:34:14.106 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.106 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.106 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.106 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Jaeger query container image", 2024-09-13T18:34:14.106 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.106 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.106 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.106 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.106 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_keepalived": { 2024-09-13T18:34:14.106 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_keepalived", 2024-09-13T18:34:14.106 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.106 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.106 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.106 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-13T18:34:14.107 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.107 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.107 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.107 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Keepalived container image", 2024-09-13T18:34:14.107 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.107 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.107 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.107 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.107 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_loki": { 2024-09-13T18:34:14.107 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_loki", 2024-09-13T18:34:14.107 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.107 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.107 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.107 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-13T18:34:14.108 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.108 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.108 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.108 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Loki container image", 2024-09-13T18:34:14.108 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.108 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.108 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.108 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.108 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_node_exporter": { 2024-09-13T18:34:14.108 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_node_exporter", 2024-09-13T18:34:14.108 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.108 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.108 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.109 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-13T18:34:14.109 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.109 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.109 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.109 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Prometheus container image", 2024-09-13T18:34:14.109 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.109 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.109 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.109 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.109 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_nvmeof": { 2024-09-13T18:34:14.109 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_nvmeof", 2024-09-13T18:34:14.109 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.109 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.110 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.112 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-09-13T18:34:14.113 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.113 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.113 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.113 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Nvme-of container image", 2024-09-13T18:34:14.113 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.113 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.113 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.113 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.113 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_prometheus": { 2024-09-13T18:34:14.113 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_prometheus", 2024-09-13T18:34:14.113 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.113 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.114 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.114 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-13T18:34:14.114 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.114 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.114 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.114 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Prometheus container image", 2024-09-13T18:34:14.114 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.114 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.114 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.114 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.114 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_promtail": { 2024-09-13T18:34:14.114 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_promtail", 2024-09-13T18:34:14.114 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.114 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.115 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.115 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-13T18:34:14.115 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.115 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.115 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.115 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Promtail container image", 2024-09-13T18:34:14.115 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.115 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.115 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.115 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.115 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_snmp_gateway": { 2024-09-13T18:34:14.115 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_snmp_gateway", 2024-09-13T18:34:14.116 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.116 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.116 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.116 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-13T18:34:14.116 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.116 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.116 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.116 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "SNMP Gateway container image", 2024-09-13T18:34:14.116 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.116 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.116 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.116 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.116 INFO:teuthology.orchestra.run.smithi114.stdout: "container_init": { 2024-09-13T18:34:14.117 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_init", 2024-09-13T18:34:14.117 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.117 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.117 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.117 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.117 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.117 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.117 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.117 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Run podman/docker with `--init`", 2024-09-13T18:34:14.117 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.117 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.117 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.117 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.118 INFO:teuthology.orchestra.run.smithi114.stdout: "daemon_cache_timeout": { 2024-09-13T18:34:14.118 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "daemon_cache_timeout", 2024-09-13T18:34:14.118 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:14.118 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.118 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.118 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "600", 2024-09-13T18:34:14.118 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.118 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.118 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.118 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-13T18:34:14.118 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.118 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.118 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.119 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.119 INFO:teuthology.orchestra.run.smithi114.stdout: "default_cephadm_command_timeout": { 2024-09-13T18:34:14.119 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "default_cephadm_command_timeout", 2024-09-13T18:34:14.119 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.119 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.119 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.119 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "900", 2024-09-13T18:34:14.119 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.119 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.119 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.119 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-13T18:34:14.119 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.119 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.120 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.120 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.120 INFO:teuthology.orchestra.run.smithi114.stdout: "default_registry": { 2024-09-13T18:34:14.120 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "default_registry", 2024-09-13T18:34:14.120 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.120 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.120 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.120 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "docker.io", 2024-09-13T18:34:14.120 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.120 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.120 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.120 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-13T18:34:14.120 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.121 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.121 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.121 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.121 INFO:teuthology.orchestra.run.smithi114.stdout: "device_cache_timeout": { 2024-09-13T18:34:14.121 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "device_cache_timeout", 2024-09-13T18:34:14.121 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:14.121 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.121 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.121 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "1800", 2024-09-13T18:34:14.121 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.121 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.121 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.121 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "seconds to cache device inventory", 2024-09-13T18:34:14.121 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.122 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.122 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.122 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.122 INFO:teuthology.orchestra.run.smithi114.stdout: "device_enhanced_scan": { 2024-09-13T18:34:14.122 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "device_enhanced_scan", 2024-09-13T18:34:14.122 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.122 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.122 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.122 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.122 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.122 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.122 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.122 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-13T18:34:14.122 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.123 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.123 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.123 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.123 INFO:teuthology.orchestra.run.smithi114.stdout: "facts_cache_timeout": { 2024-09-13T18:34:14.123 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "facts_cache_timeout", 2024-09-13T18:34:14.123 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:14.123 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.123 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.123 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "60", 2024-09-13T18:34:14.123 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.123 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.123 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.123 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "seconds to cache host facts data", 2024-09-13T18:34:14.124 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.124 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.124 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.124 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.124 INFO:teuthology.orchestra.run.smithi114.stdout: "host_check_interval": { 2024-09-13T18:34:14.124 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "host_check_interval", 2024-09-13T18:34:14.124 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:14.124 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.124 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.124 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "600", 2024-09-13T18:34:14.124 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.124 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.124 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.125 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "how frequently to perform a host check", 2024-09-13T18:34:14.125 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.125 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.125 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.125 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.125 INFO:teuthology.orchestra.run.smithi114.stdout: "hw_monitoring": { 2024-09-13T18:34:14.125 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "hw_monitoring", 2024-09-13T18:34:14.125 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.125 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.125 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.125 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.125 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.125 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.126 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.126 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-13T18:34:14.126 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.126 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.126 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.126 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.126 INFO:teuthology.orchestra.run.smithi114.stdout: "inventory_list_all": { 2024-09-13T18:34:14.126 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "inventory_list_all", 2024-09-13T18:34:14.126 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.126 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.126 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.126 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.126 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.126 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.127 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.127 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-13T18:34:14.127 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.127 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.127 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.127 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.127 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.127 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.127 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.127 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.127 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.127 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.127 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.127 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.128 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.128 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.128 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.128 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.128 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.128 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.128 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.128 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.128 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.128 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.128 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.128 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.128 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.129 INFO:teuthology.orchestra.run.smithi114.stdout: "log_refresh_metadata": { 2024-09-13T18:34:14.129 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_refresh_metadata", 2024-09-13T18:34:14.129 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.129 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.129 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.129 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.129 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.129 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.129 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.129 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-13T18:34:14.129 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.129 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.129 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.130 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.130 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.130 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.130 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.130 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.130 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.130 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.130 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.130 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.130 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.130 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-13T18:34:14.130 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.130 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.130 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.131 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.131 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.131 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.131 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.131 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.131 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.131 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.131 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.131 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.131 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.131 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.131 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.131 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.131 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.131 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.132 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.132 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.132 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.132 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.132 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.132 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.132 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.132 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.132 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.132 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.132 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.132 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.132 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.132 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.133 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.133 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.133 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.133 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.133 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.133 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.133 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.133 INFO:teuthology.orchestra.run.smithi114.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-13T18:34:14.133 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-13T18:34:14.133 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.133 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.133 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.134 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.134 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.134 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.134 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.134 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-13T18:34:14.134 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.134 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.134 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.134 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.134 INFO:teuthology.orchestra.run.smithi114.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-13T18:34:14.134 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-13T18:34:14.134 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.134 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.135 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.135 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "*", 2024-09-13T18:34:14.135 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.135 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.135 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.135 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-13T18:34:14.135 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.135 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.135 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.135 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.135 INFO:teuthology.orchestra.run.smithi114.stdout: "max_count_per_host": { 2024-09-13T18:34:14.135 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "max_count_per_host", 2024-09-13T18:34:14.135 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.135 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.136 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.136 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "10", 2024-09-13T18:34:14.136 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.136 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.136 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.136 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "max number of daemons per service per host", 2024-09-13T18:34:14.136 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.136 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.136 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.136 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.136 INFO:teuthology.orchestra.run.smithi114.stdout: "max_osd_draining_count": { 2024-09-13T18:34:14.136 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "max_osd_draining_count", 2024-09-13T18:34:14.137 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.137 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.137 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.137 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "10", 2024-09-13T18:34:14.137 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.137 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.137 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.137 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-13T18:34:14.137 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.137 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.137 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.137 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.137 INFO:teuthology.orchestra.run.smithi114.stdout: "migration_current": { 2024-09-13T18:34:14.138 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "migration_current", 2024-09-13T18:34:14.138 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.138 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.138 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.138 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.138 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.138 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.138 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.138 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "internal - do not modify", 2024-09-13T18:34:14.138 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.138 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.138 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.138 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.138 INFO:teuthology.orchestra.run.smithi114.stdout: "mode": { 2024-09-13T18:34:14.139 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "mode", 2024-09-13T18:34:14.139 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.139 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.139 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.139 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "root", 2024-09-13T18:34:14.139 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.139 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.139 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.139 INFO:teuthology.orchestra.run.smithi114.stdout: "cephadm-package", 2024-09-13T18:34:14.139 INFO:teuthology.orchestra.run.smithi114.stdout: "root" 2024-09-13T18:34:14.139 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.139 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "mode for remote execution of cephadm", 2024-09-13T18:34:14.139 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.139 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.140 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.140 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.140 INFO:teuthology.orchestra.run.smithi114.stdout: "oob_default_addr": { 2024-09-13T18:34:14.140 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "oob_default_addr", 2024-09-13T18:34:14.140 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.140 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.140 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.140 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "169.254.1.1", 2024-09-13T18:34:14.140 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.140 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.140 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.140 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-13T18:34:14.140 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.141 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.141 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.141 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.141 INFO:teuthology.orchestra.run.smithi114.stdout: "prometheus_alerts_path": { 2024-09-13T18:34:14.141 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "prometheus_alerts_path", 2024-09-13T18:34:14.141 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.141 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.141 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.141 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-13T18:34:14.141 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.141 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.141 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.141 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-13T18:34:14.141 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.142 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.142 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.142 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.142 INFO:teuthology.orchestra.run.smithi114.stdout: "registry_insecure": { 2024-09-13T18:34:14.142 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "registry_insecure", 2024-09-13T18:34:14.142 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.142 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.142 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.142 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.142 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.142 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.142 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.142 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-13T18:34:14.142 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.143 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.143 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.143 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.143 INFO:teuthology.orchestra.run.smithi114.stdout: "registry_password": { 2024-09-13T18:34:14.143 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "registry_password", 2024-09-13T18:34:14.143 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.143 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.143 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.143 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.143 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.143 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.143 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.143 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-13T18:34:14.144 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.144 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.144 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.144 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.144 INFO:teuthology.orchestra.run.smithi114.stdout: "registry_url": { 2024-09-13T18:34:14.144 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "registry_url", 2024-09-13T18:34:14.144 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.144 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.144 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.144 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.144 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.144 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.144 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.144 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-13T18:34:14.145 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.145 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.145 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.145 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.145 INFO:teuthology.orchestra.run.smithi114.stdout: "registry_username": { 2024-09-13T18:34:14.145 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "registry_username", 2024-09-13T18:34:14.145 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.145 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.145 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.145 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.145 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.145 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.145 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.146 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-13T18:34:14.146 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.146 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.146 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.146 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.146 INFO:teuthology.orchestra.run.smithi114.stdout: "secure_monitoring_stack": { 2024-09-13T18:34:14.146 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "secure_monitoring_stack", 2024-09-13T18:34:14.146 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.146 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.146 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.146 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.146 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.146 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.146 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.146 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-13T18:34:14.147 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.147 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.147 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.147 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.147 INFO:teuthology.orchestra.run.smithi114.stdout: "service_discovery_port": { 2024-09-13T18:34:14.147 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "service_discovery_port", 2024-09-13T18:34:14.147 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.147 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.147 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.147 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "8765", 2024-09-13T18:34:14.147 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.147 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.147 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.148 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "cephadm service discovery port", 2024-09-13T18:34:14.148 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.148 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.148 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.148 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.148 INFO:teuthology.orchestra.run.smithi114.stdout: "ssh_config_file": { 2024-09-13T18:34:14.148 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ssh_config_file", 2024-09-13T18:34:14.148 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.148 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.148 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.148 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.148 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.148 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.148 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.149 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-13T18:34:14.149 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.149 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.149 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.149 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.149 INFO:teuthology.orchestra.run.smithi114.stdout: "use_agent": { 2024-09-13T18:34:14.149 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "use_agent", 2024-09-13T18:34:14.149 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.149 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.149 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.149 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.149 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.149 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.149 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.150 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-13T18:34:14.150 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.150 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.150 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.150 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.150 INFO:teuthology.orchestra.run.smithi114.stdout: "use_repo_digest": { 2024-09-13T18:34:14.150 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "use_repo_digest", 2024-09-13T18:34:14.150 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.150 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.150 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.150 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.150 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.150 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.150 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.151 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-13T18:34:14.151 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.151 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.151 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.151 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.151 INFO:teuthology.orchestra.run.smithi114.stdout: "warn_on_failed_host_check": { 2024-09-13T18:34:14.151 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "warn_on_failed_host_check", 2024-09-13T18:34:14.151 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.151 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.151 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.151 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.151 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.151 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.151 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.151 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "raise a health warning if the host check fails", 2024-09-13T18:34:14.152 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.152 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.152 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.152 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.152 INFO:teuthology.orchestra.run.smithi114.stdout: "warn_on_stray_daemons": { 2024-09-13T18:34:14.152 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "warn_on_stray_daemons", 2024-09-13T18:34:14.152 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.152 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.152 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.152 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.152 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.152 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.152 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.152 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-13T18:34:14.152 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.153 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.153 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.153 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.153 INFO:teuthology.orchestra.run.smithi114.stdout: "warn_on_stray_hosts": { 2024-09-13T18:34:14.153 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "warn_on_stray_hosts", 2024-09-13T18:34:14.153 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.153 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.153 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.153 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.153 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.153 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.153 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.153 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-13T18:34:14.153 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.154 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.154 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.154 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.154 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.154 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.154 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.154 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "crash", 2024-09-13T18:34:14.154 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.154 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.154 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.154 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.154 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.154 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.155 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.155 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.155 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.155 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.155 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.155 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.155 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.155 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.155 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.155 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.155 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.155 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.155 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.156 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.156 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.156 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.156 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.156 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.156 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.156 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.156 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.156 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.156 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.156 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.156 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.156 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.156 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.157 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.157 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.157 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.157 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.157 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.157 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.157 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.157 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.157 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.157 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.157 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.157 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.157 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.158 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.158 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.158 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.158 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.158 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.158 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.158 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.158 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.158 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.158 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.158 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.158 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.158 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.159 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.159 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.159 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.159 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.159 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.159 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.159 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.159 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.159 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.159 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.159 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.159 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.159 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.159 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.160 INFO:teuthology.orchestra.run.smithi114.stdout: "retain_interval": { 2024-09-13T18:34:14.160 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "retain_interval", 2024-09-13T18:34:14.160 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:14.160 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.160 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.160 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "31536000", 2024-09-13T18:34:14.160 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.160 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.160 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.160 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-13T18:34:14.160 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.160 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.160 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.160 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.161 INFO:teuthology.orchestra.run.smithi114.stdout: "warn_recent_interval": { 2024-09-13T18:34:14.161 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "warn_recent_interval", 2024-09-13T18:34:14.161 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:14.161 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.161 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.161 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "1209600", 2024-09-13T18:34:14.161 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.161 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.161 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.161 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-13T18:34:14.161 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.161 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.161 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.162 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.162 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.162 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.162 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.162 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "dashboard", 2024-09-13T18:34:14.162 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.162 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.162 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.162 INFO:teuthology.orchestra.run.smithi114.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-13T18:34:14.162 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-13T18:34:14.162 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.162 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.162 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.162 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "10", 2024-09-13T18:34:14.163 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.163 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.163 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.163 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.163 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.163 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.163 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.163 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.163 INFO:teuthology.orchestra.run.smithi114.stdout: "ALERTMANAGER_API_HOST": { 2024-09-13T18:34:14.163 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-13T18:34:14.163 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.163 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.163 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.164 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.164 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.164 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.164 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.164 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.164 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.164 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.164 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.164 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.164 INFO:teuthology.orchestra.run.smithi114.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-13T18:34:14.164 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-13T18:34:14.164 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.164 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.165 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.165 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.165 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.165 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.165 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.165 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.165 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.165 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.165 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.165 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.165 INFO:teuthology.orchestra.run.smithi114.stdout: "AUDIT_API_ENABLED": { 2024-09-13T18:34:14.165 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "AUDIT_API_ENABLED", 2024-09-13T18:34:14.165 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.165 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.166 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.166 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.166 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.166 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.166 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.166 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.166 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.166 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.166 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.166 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.166 INFO:teuthology.orchestra.run.smithi114.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-13T18:34:14.166 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-13T18:34:14.166 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.166 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.167 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.167 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.167 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.167 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.167 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.167 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.167 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.167 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.167 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.167 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.167 INFO:teuthology.orchestra.run.smithi114.stdout: "ENABLE_BROWSABLE_API": { 2024-09-13T18:34:14.167 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-13T18:34:14.167 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.167 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.168 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.168 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.168 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.168 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.168 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.168 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.168 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.168 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.168 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.168 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.168 INFO:teuthology.orchestra.run.smithi114.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-13T18:34:14.168 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-13T18:34:14.168 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.169 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.169 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.169 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.169 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.169 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.169 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.169 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.169 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.169 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.169 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.169 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.169 INFO:teuthology.orchestra.run.smithi114.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-13T18:34:14.169 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-13T18:34:14.170 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.170 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.170 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.170 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.170 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.170 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.170 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.170 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.170 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.170 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.170 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.170 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.170 INFO:teuthology.orchestra.run.smithi114.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-13T18:34:14.170 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-13T18:34:14.171 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.171 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.171 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.171 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.171 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.171 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.171 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.171 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.171 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.171 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.171 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.171 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.171 INFO:teuthology.orchestra.run.smithi114.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-13T18:34:14.172 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-13T18:34:14.172 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.172 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.172 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.172 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.172 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.172 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.172 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.172 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.172 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.172 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.172 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.172 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.173 INFO:teuthology.orchestra.run.smithi114.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-13T18:34:14.173 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-13T18:34:14.173 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.173 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.173 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.173 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.173 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.173 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.173 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.173 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.173 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.173 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.173 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.174 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.174 INFO:teuthology.orchestra.run.smithi114.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-13T18:34:14.174 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-13T18:34:14.174 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.174 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.174 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.174 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.174 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.174 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.174 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.174 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.174 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.174 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.175 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.175 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.175 INFO:teuthology.orchestra.run.smithi114.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-13T18:34:14.175 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-13T18:34:14.175 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.175 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.175 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.175 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.175 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.175 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.175 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.175 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.175 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.176 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.176 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.176 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.176 INFO:teuthology.orchestra.run.smithi114.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-13T18:34:14.176 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-13T18:34:14.176 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.176 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.176 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.176 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.176 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.176 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.176 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.176 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.176 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.177 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.177 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.177 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.177 INFO:teuthology.orchestra.run.smithi114.stdout: "GRAFANA_API_PASSWORD": { 2024-09-13T18:34:14.177 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-13T18:34:14.177 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.177 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.177 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.177 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "admin", 2024-09-13T18:34:14.177 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.177 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.177 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.177 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.178 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.178 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.178 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.178 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.178 INFO:teuthology.orchestra.run.smithi114.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-13T18:34:14.178 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-13T18:34:14.178 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.178 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.178 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.178 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.178 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.178 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.178 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.179 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.179 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.179 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.179 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.179 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.179 INFO:teuthology.orchestra.run.smithi114.stdout: "GRAFANA_API_URL": { 2024-09-13T18:34:14.179 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "GRAFANA_API_URL", 2024-09-13T18:34:14.179 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.179 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.179 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.179 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.179 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.179 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.179 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.180 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.180 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.180 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.180 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.180 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.180 INFO:teuthology.orchestra.run.smithi114.stdout: "GRAFANA_API_USERNAME": { 2024-09-13T18:34:14.180 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-13T18:34:14.180 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.180 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.180 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.180 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "admin", 2024-09-13T18:34:14.180 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.180 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.181 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.181 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.181 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.181 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.181 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.181 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.181 INFO:teuthology.orchestra.run.smithi114.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-13T18:34:14.181 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-13T18:34:14.181 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.181 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.181 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.181 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.181 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.182 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.182 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.182 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.182 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.182 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.182 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.182 INFO:teuthology.orchestra.run.smithi114.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-13T18:34:14.182 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-13T18:34:14.182 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.182 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.182 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.182 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.183 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.183 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.183 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.183 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.183 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.183 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.183 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.183 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.183 INFO:teuthology.orchestra.run.smithi114.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-13T18:34:14.183 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-13T18:34:14.183 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.183 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.183 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.183 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.184 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.184 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.184 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.184 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.184 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.184 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.184 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.184 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.184 INFO:teuthology.orchestra.run.smithi114.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-13T18:34:14.184 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-13T18:34:14.184 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.184 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.184 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.184 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.185 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.185 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.185 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.185 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.185 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.185 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.185 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.185 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.185 INFO:teuthology.orchestra.run.smithi114.stdout: "PROMETHEUS_API_HOST": { 2024-09-13T18:34:14.185 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-13T18:34:14.185 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.185 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.185 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.186 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.186 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.186 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.186 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.186 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.186 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.186 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.186 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.186 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.186 INFO:teuthology.orchestra.run.smithi114.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-13T18:34:14.186 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-13T18:34:14.186 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.186 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.186 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.187 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.187 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.187 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.187 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.187 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.187 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.187 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.187 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.187 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.187 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-13T18:34:14.187 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-13T18:34:14.187 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.187 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.187 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.188 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.188 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.188 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.188 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.188 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.188 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.188 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.188 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.188 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.188 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-13T18:34:14.188 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-13T18:34:14.188 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.189 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.189 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.189 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.189 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.189 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.189 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.189 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.189 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.189 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.189 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.189 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.189 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-13T18:34:14.189 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-13T18:34:14.190 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.190 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.190 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.190 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.190 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.190 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.190 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.190 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.190 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.190 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.190 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.190 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.190 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-13T18:34:14.190 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-13T18:34:14.191 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.191 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.191 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.191 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.191 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.191 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.191 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.191 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.191 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.191 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.191 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.191 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.192 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-13T18:34:14.192 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-13T18:34:14.192 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.192 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.192 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.192 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.192 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.192 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.192 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.192 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.192 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.192 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.192 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.193 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.193 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-13T18:34:14.193 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-13T18:34:14.193 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.193 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.193 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.193 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.193 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.193 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.193 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.193 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.193 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.193 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.193 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.194 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.194 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-13T18:34:14.194 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-13T18:34:14.194 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.194 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.194 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.194 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.194 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.194 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.194 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.194 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.194 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.194 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.194 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.195 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.195 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_ENABLED": { 2024-09-13T18:34:14.195 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-13T18:34:14.195 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.195 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.195 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.195 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.195 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.195 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.195 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.195 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.195 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.195 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.196 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.196 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.196 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-13T18:34:14.196 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-13T18:34:14.196 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.196 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.196 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.196 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-13T18:34:14.196 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.196 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.196 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.196 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.196 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.196 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.197 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.197 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.197 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-13T18:34:14.197 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-13T18:34:14.197 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.197 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.197 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.197 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "10", 2024-09-13T18:34:14.197 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.197 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.197 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.197 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.197 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.197 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.198 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.198 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.198 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-13T18:34:14.198 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-13T18:34:14.198 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.198 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.198 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.198 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "8", 2024-09-13T18:34:14.198 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.198 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.198 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.198 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.198 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.198 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.199 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.199 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.199 INFO:teuthology.orchestra.run.smithi114.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-13T18:34:14.199 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-13T18:34:14.199 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.199 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.199 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.199 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "45", 2024-09-13T18:34:14.199 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.199 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.199 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.199 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.199 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.200 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.200 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.200 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.200 INFO:teuthology.orchestra.run.smithi114.stdout: "RGW_API_ACCESS_KEY": { 2024-09-13T18:34:14.200 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-13T18:34:14.200 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.200 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.200 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.200 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.200 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.200 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.200 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.200 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.200 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.201 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.201 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.201 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.201 INFO:teuthology.orchestra.run.smithi114.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-13T18:34:14.201 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-13T18:34:14.201 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.201 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.201 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.201 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "admin", 2024-09-13T18:34:14.201 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.201 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.201 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.202 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.202 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.202 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.202 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.202 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.202 INFO:teuthology.orchestra.run.smithi114.stdout: "RGW_API_SECRET_KEY": { 2024-09-13T18:34:14.202 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-13T18:34:14.202 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.202 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.202 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.202 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.202 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.202 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.202 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.202 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.203 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.203 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.203 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.203 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.203 INFO:teuthology.orchestra.run.smithi114.stdout: "RGW_API_SSL_VERIFY": { 2024-09-13T18:34:14.203 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-13T18:34:14.203 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.203 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.203 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.203 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.203 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.203 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.203 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.203 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.204 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.204 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.204 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.204 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.204 INFO:teuthology.orchestra.run.smithi114.stdout: "UNSAFE_TLS_v1_2": { 2024-09-13T18:34:14.204 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-13T18:34:14.204 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.204 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.204 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.204 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.204 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.204 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.204 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.205 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.205 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.205 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.205 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.205 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.205 INFO:teuthology.orchestra.run.smithi114.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-13T18:34:14.205 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-13T18:34:14.205 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.205 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.205 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.205 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-13T18:34:14.205 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.205 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.205 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.206 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.206 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.206 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.206 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.206 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.206 INFO:teuthology.orchestra.run.smithi114.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-13T18:34:14.206 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-13T18:34:14.206 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.206 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.206 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.206 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "10", 2024-09-13T18:34:14.206 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.207 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.207 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.207 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.207 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.207 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.207 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.207 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.207 INFO:teuthology.orchestra.run.smithi114.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-13T18:34:14.207 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-13T18:34:14.207 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.207 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.207 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.207 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "5", 2024-09-13T18:34:14.207 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.208 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.208 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.208 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.208 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.208 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.208 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.208 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.208 INFO:teuthology.orchestra.run.smithi114.stdout: "cross_origin_url": { 2024-09-13T18:34:14.208 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "cross_origin_url", 2024-09-13T18:34:14.208 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.208 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.208 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.209 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.209 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.209 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.209 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.209 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.209 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.209 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.209 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.209 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.209 INFO:teuthology.orchestra.run.smithi114.stdout: "crt_file": { 2024-09-13T18:34:14.209 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "crt_file", 2024-09-13T18:34:14.209 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.209 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.209 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.210 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.210 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.210 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.210 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.210 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.210 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.210 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.210 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.210 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.210 INFO:teuthology.orchestra.run.smithi114.stdout: "debug": { 2024-09-13T18:34:14.210 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "debug", 2024-09-13T18:34:14.210 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.210 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.210 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.211 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.211 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.211 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.211 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.211 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Enable/disable debug options", 2024-09-13T18:34:14.211 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.211 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.211 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.211 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.211 INFO:teuthology.orchestra.run.smithi114.stdout: "jwt_token_ttl": { 2024-09-13T18:34:14.211 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "jwt_token_ttl", 2024-09-13T18:34:14.211 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.211 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.211 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.212 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "28800", 2024-09-13T18:34:14.212 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.212 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.212 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.212 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.212 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.212 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.212 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.212 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.212 INFO:teuthology.orchestra.run.smithi114.stdout: "key_file": { 2024-09-13T18:34:14.212 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "key_file", 2024-09-13T18:34:14.212 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.212 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.213 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.213 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.213 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.213 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.213 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.213 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.213 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.213 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.213 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.213 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.213 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.213 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.213 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.213 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.214 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.214 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.214 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.214 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.214 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.214 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.214 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.214 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.214 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.214 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.214 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.214 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.214 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.214 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.215 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.215 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.215 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.215 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.215 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.215 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.215 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.215 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.215 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.215 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.215 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.215 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.215 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.215 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.216 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.216 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.216 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.216 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.216 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.216 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.216 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.216 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.216 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.216 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.216 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.216 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.216 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.216 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.217 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.217 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.217 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.217 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.217 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.217 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.217 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.217 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.217 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.217 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.217 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.217 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.217 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.217 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.218 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.218 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.218 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.218 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.218 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.218 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.218 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.218 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.218 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.218 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.218 INFO:teuthology.orchestra.run.smithi114.stdout: "motd": { 2024-09-13T18:34:14.218 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "motd", 2024-09-13T18:34:14.218 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.219 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.219 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.219 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.219 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.219 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.219 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.219 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "The message of the day", 2024-09-13T18:34:14.219 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.219 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.219 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.219 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.219 INFO:teuthology.orchestra.run.smithi114.stdout: "redirect_resolve_ip_addr": { 2024-09-13T18:34:14.220 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "redirect_resolve_ip_addr", 2024-09-13T18:34:14.220 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.220 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.220 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.220 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.220 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.220 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.220 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.220 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.220 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.220 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.220 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.220 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.221 INFO:teuthology.orchestra.run.smithi114.stdout: "server_addr": { 2024-09-13T18:34:14.221 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "server_addr", 2024-09-13T18:34:14.221 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.221 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.221 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.221 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "::", 2024-09-13T18:34:14.221 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.221 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.221 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.221 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.221 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.221 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.221 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.221 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.222 INFO:teuthology.orchestra.run.smithi114.stdout: "server_port": { 2024-09-13T18:34:14.222 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "server_port", 2024-09-13T18:34:14.222 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.222 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.222 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.222 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "8080", 2024-09-13T18:34:14.222 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.222 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.222 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.222 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.222 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.222 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.222 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.223 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.223 INFO:teuthology.orchestra.run.smithi114.stdout: "ssl": { 2024-09-13T18:34:14.223 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ssl", 2024-09-13T18:34:14.223 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.223 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.223 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.223 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.223 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.223 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.223 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.223 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.223 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.223 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.223 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.224 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.224 INFO:teuthology.orchestra.run.smithi114.stdout: "ssl_server_port": { 2024-09-13T18:34:14.224 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ssl_server_port", 2024-09-13T18:34:14.224 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.224 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.224 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.224 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "8443", 2024-09-13T18:34:14.224 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.224 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.224 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.224 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.224 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.224 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.225 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.225 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.225 INFO:teuthology.orchestra.run.smithi114.stdout: "standby_behaviour": { 2024-09-13T18:34:14.225 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "standby_behaviour", 2024-09-13T18:34:14.225 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.225 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.225 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.225 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "redirect", 2024-09-13T18:34:14.225 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.225 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.225 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.225 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.225 INFO:teuthology.orchestra.run.smithi114.stdout: "redirect" 2024-09-13T18:34:14.225 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.226 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.226 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.226 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.226 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.226 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.226 INFO:teuthology.orchestra.run.smithi114.stdout: "standby_error_status_code": { 2024-09-13T18:34:14.226 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "standby_error_status_code", 2024-09-13T18:34:14.226 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.226 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.226 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.226 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "500", 2024-09-13T18:34:14.226 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "400", 2024-09-13T18:34:14.226 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "599", 2024-09-13T18:34:14.226 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.227 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.227 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.227 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.227 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.227 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.227 INFO:teuthology.orchestra.run.smithi114.stdout: "url_prefix": { 2024-09-13T18:34:14.227 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "url_prefix", 2024-09-13T18:34:14.227 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.227 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.227 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.227 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.227 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.227 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.227 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.228 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.228 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.228 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.228 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.228 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.228 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.228 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.228 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.228 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "devicehealth", 2024-09-13T18:34:14.228 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.228 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.228 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.228 INFO:teuthology.orchestra.run.smithi114.stdout: "enable_monitoring": { 2024-09-13T18:34:14.228 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "enable_monitoring", 2024-09-13T18:34:14.229 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.229 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.229 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.229 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.229 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.229 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.229 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.229 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "monitor device health metrics", 2024-09-13T18:34:14.229 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.229 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.229 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.229 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.229 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.229 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.230 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.230 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.230 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.230 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.230 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.230 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.230 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.230 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.230 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.230 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.230 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.230 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.230 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.230 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.231 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.231 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.231 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.231 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.231 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.231 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.231 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.231 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.231 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.231 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.231 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.231 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.231 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.231 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.231 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.232 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.232 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.232 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.232 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.232 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.232 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.232 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.232 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.232 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.232 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.232 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.232 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.233 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.233 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.233 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.233 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.233 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.233 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.233 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.233 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.233 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.233 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.233 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.233 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.233 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.233 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.234 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.234 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.234 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.234 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.234 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.234 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.234 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.234 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.234 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.234 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.234 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.234 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.234 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.234 INFO:teuthology.orchestra.run.smithi114.stdout: "mark_out_threshold": { 2024-09-13T18:34:14.235 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "mark_out_threshold", 2024-09-13T18:34:14.235 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:14.235 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.235 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.235 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "2419200", 2024-09-13T18:34:14.235 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.235 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.235 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.235 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-13T18:34:14.235 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.235 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.235 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.235 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.235 INFO:teuthology.orchestra.run.smithi114.stdout: "pool_name": { 2024-09-13T18:34:14.236 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "pool_name", 2024-09-13T18:34:14.236 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.236 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.236 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.236 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "device_health_metrics", 2024-09-13T18:34:14.236 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.236 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.236 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.236 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-13T18:34:14.236 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.236 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.236 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.236 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.236 INFO:teuthology.orchestra.run.smithi114.stdout: "retention_period": { 2024-09-13T18:34:14.237 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "retention_period", 2024-09-13T18:34:14.237 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:14.237 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.237 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.237 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "15552000", 2024-09-13T18:34:14.237 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.237 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.237 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.237 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "how long to retain device health metrics", 2024-09-13T18:34:14.237 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.237 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.237 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.238 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.238 INFO:teuthology.orchestra.run.smithi114.stdout: "scrape_frequency": { 2024-09-13T18:34:14.238 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "scrape_frequency", 2024-09-13T18:34:14.238 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:14.238 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.238 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.238 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "86400", 2024-09-13T18:34:14.238 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.238 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.238 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.238 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-13T18:34:14.238 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.238 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.238 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.239 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.239 INFO:teuthology.orchestra.run.smithi114.stdout: "self_heal": { 2024-09-13T18:34:14.239 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "self_heal", 2024-09-13T18:34:14.239 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.239 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.239 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.239 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.239 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.239 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.239 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.239 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-13T18:34:14.239 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.239 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.239 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.240 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.240 INFO:teuthology.orchestra.run.smithi114.stdout: "sleep_interval": { 2024-09-13T18:34:14.240 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sleep_interval", 2024-09-13T18:34:14.240 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:14.240 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.240 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.240 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "600", 2024-09-13T18:34:14.240 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.240 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.240 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.240 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "how frequently to wake up and check device health", 2024-09-13T18:34:14.240 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.240 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.240 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.241 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.241 INFO:teuthology.orchestra.run.smithi114.stdout: "warn_threshold": { 2024-09-13T18:34:14.241 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "warn_threshold", 2024-09-13T18:34:14.241 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:14.241 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.241 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.241 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "7257600", 2024-09-13T18:34:14.241 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.241 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.241 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.241 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-13T18:34:14.241 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.241 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.241 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.242 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.242 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.242 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.242 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.242 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "diskprediction_local", 2024-09-13T18:34:14.242 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.242 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.242 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.242 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.242 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.242 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.242 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.242 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.242 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.243 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.243 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.243 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.243 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.243 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.243 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.243 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.243 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.243 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.243 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.243 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.243 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.243 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.243 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.243 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.244 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.244 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.244 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.244 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.244 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.244 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.244 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.244 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.244 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.244 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.244 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.244 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.244 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.244 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.245 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.245 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.245 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.245 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.245 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.245 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.245 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.245 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.245 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.245 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.245 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.245 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.245 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.246 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.246 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.246 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.246 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.246 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.246 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.246 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.246 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.246 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.246 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.246 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.246 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.246 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.247 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.247 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.247 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.247 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.247 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.247 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.247 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.247 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.247 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.247 INFO:teuthology.orchestra.run.smithi114.stdout: "predict_interval": { 2024-09-13T18:34:14.247 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "predict_interval", 2024-09-13T18:34:14.247 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.247 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.248 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.248 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "86400", 2024-09-13T18:34:14.248 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.248 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.248 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.248 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.248 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.248 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.248 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.248 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.248 INFO:teuthology.orchestra.run.smithi114.stdout: "predictor_model": { 2024-09-13T18:34:14.248 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "predictor_model", 2024-09-13T18:34:14.248 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.249 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.249 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.249 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "prophetstor", 2024-09-13T18:34:14.249 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.249 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.249 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.249 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.249 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.249 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.249 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.249 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.249 INFO:teuthology.orchestra.run.smithi114.stdout: "sleep_interval": { 2024-09-13T18:34:14.249 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sleep_interval", 2024-09-13T18:34:14.249 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.250 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.250 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.250 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "600", 2024-09-13T18:34:14.250 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.250 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.250 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.250 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.250 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.250 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.250 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.250 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.250 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.250 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.251 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.251 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "influx", 2024-09-13T18:34:14.251 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": false, 2024-09-13T18:34:14.251 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "influxdb python module not found", 2024-09-13T18:34:14.251 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.251 INFO:teuthology.orchestra.run.smithi114.stdout: "batch_size": { 2024-09-13T18:34:14.251 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "batch_size", 2024-09-13T18:34:14.251 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.251 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.251 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.251 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "5000", 2024-09-13T18:34:14.251 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.251 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.252 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.252 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-13T18:34:14.252 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.252 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.252 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.252 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.252 INFO:teuthology.orchestra.run.smithi114.stdout: "database": { 2024-09-13T18:34:14.252 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "database", 2024-09-13T18:34:14.252 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.252 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.252 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.252 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "ceph", 2024-09-13T18:34:14.252 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.253 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.253 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.253 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-13T18:34:14.253 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.253 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.253 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.253 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.253 INFO:teuthology.orchestra.run.smithi114.stdout: "hostname": { 2024-09-13T18:34:14.253 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "hostname", 2024-09-13T18:34:14.253 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.253 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.253 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.254 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.254 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.254 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.254 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.254 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "InfluxDB server hostname", 2024-09-13T18:34:14.254 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.254 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.254 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.254 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.254 INFO:teuthology.orchestra.run.smithi114.stdout: "interval": { 2024-09-13T18:34:14.254 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "interval", 2024-09-13T18:34:14.254 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:14.254 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.254 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.255 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "30", 2024-09-13T18:34:14.255 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "5", 2024-09-13T18:34:14.255 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.255 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.255 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-13T18:34:14.255 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.255 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.255 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.255 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.255 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.255 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.255 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.255 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.255 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.256 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.256 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.256 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.256 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.256 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.256 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.256 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.256 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.256 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.256 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.256 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.256 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.256 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.256 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.257 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.257 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.257 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.257 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.257 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.257 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.257 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.257 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.257 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.257 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.257 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.257 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.257 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.257 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.258 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.258 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.258 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.258 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.258 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.258 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.258 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.258 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.258 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.258 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.258 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.258 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.258 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.258 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.259 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.259 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.259 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.259 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.259 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.259 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.259 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.259 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.259 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.259 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.259 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.259 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.259 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.260 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.260 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.260 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.260 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.260 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.260 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.260 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.260 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.260 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.260 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.260 INFO:teuthology.orchestra.run.smithi114.stdout: "password": { 2024-09-13T18:34:14.260 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "password", 2024-09-13T18:34:14.260 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.261 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.261 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.261 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.261 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.261 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.261 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.261 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "password of InfluxDB server user", 2024-09-13T18:34:14.261 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.261 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.261 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.261 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.261 INFO:teuthology.orchestra.run.smithi114.stdout: "port": { 2024-09-13T18:34:14.261 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "port", 2024-09-13T18:34:14.262 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.262 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.262 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.262 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "8086", 2024-09-13T18:34:14.262 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.262 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.262 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.262 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "InfluxDB server port", 2024-09-13T18:34:14.262 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.262 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.262 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.262 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.262 INFO:teuthology.orchestra.run.smithi114.stdout: "ssl": { 2024-09-13T18:34:14.262 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ssl", 2024-09-13T18:34:14.263 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.263 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.263 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.263 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "false", 2024-09-13T18:34:14.263 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.263 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.263 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.263 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-13T18:34:14.263 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.263 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.263 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.263 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.263 INFO:teuthology.orchestra.run.smithi114.stdout: "threads": { 2024-09-13T18:34:14.263 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "threads", 2024-09-13T18:34:14.264 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.264 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.264 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.264 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "5", 2024-09-13T18:34:14.264 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "1", 2024-09-13T18:34:14.264 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "32", 2024-09-13T18:34:14.264 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.264 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-13T18:34:14.264 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.264 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.264 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.264 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.264 INFO:teuthology.orchestra.run.smithi114.stdout: "username": { 2024-09-13T18:34:14.265 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "username", 2024-09-13T18:34:14.265 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.265 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.265 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.265 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.265 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.265 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.265 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.265 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "username of InfluxDB server user", 2024-09-13T18:34:14.265 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.265 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.265 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.265 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.266 INFO:teuthology.orchestra.run.smithi114.stdout: "verify_ssl": { 2024-09-13T18:34:14.266 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "verify_ssl", 2024-09-13T18:34:14.266 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.266 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.266 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.266 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "true", 2024-09-13T18:34:14.266 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.266 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.266 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.266 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-13T18:34:14.266 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.266 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.266 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.266 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.267 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.267 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.267 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.267 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "insights", 2024-09-13T18:34:14.267 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.267 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.267 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.267 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.267 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.267 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.267 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.267 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.267 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.267 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.267 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.268 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.268 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.268 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.268 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.268 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.268 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.268 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.268 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.268 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.268 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.268 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.268 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.268 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.268 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.269 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.269 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.269 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.269 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.269 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.269 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.269 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.269 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.269 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.269 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.269 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.269 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.269 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.269 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.270 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.270 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.270 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.270 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.270 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.270 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.270 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.270 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.270 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.270 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.270 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.270 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.270 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.270 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.271 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.271 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.271 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.271 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.271 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.271 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.271 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.271 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.271 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.271 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.271 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.271 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.271 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.271 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.271 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.272 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.272 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.272 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.272 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.272 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.272 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.272 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.272 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.272 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "iostat", 2024-09-13T18:34:14.272 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.272 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.272 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.272 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.272 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.273 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.273 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.273 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.273 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.273 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.273 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.273 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.273 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.273 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.273 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.273 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.273 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.273 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.274 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.274 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.274 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.274 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.274 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.274 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.274 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.274 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.274 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.274 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.274 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.274 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.274 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.274 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.274 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.275 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.275 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.275 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.275 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.275 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.275 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.275 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.275 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.275 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.275 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.275 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.275 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.275 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.275 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.276 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.276 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.276 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.276 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.276 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.276 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.276 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.276 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.276 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.276 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.276 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.276 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.276 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.276 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.277 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.277 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.277 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.277 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.277 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.277 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.277 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.277 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.277 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.277 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.277 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.277 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.277 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.277 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.278 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.278 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "k8sevents", 2024-09-13T18:34:14.278 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.278 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.278 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.278 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_event_retention_days": { 2024-09-13T18:34:14.278 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ceph_event_retention_days", 2024-09-13T18:34:14.278 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.278 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.278 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.278 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "7", 2024-09-13T18:34:14.278 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.278 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.278 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.279 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-13T18:34:14.279 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.279 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.279 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.279 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.279 INFO:teuthology.orchestra.run.smithi114.stdout: "config_check_secs": { 2024-09-13T18:34:14.279 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "config_check_secs", 2024-09-13T18:34:14.279 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.279 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.279 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.279 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "10", 2024-09-13T18:34:14.279 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "10", 2024-09-13T18:34:14.279 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.280 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.280 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-13T18:34:14.280 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.280 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.280 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.280 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.280 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.280 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.280 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.280 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.280 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.280 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.280 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.280 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.281 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.281 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.281 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.281 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.281 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.281 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.281 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.281 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.281 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.281 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.281 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.281 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.281 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.281 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.282 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.282 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.282 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.282 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.282 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.282 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.282 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.282 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.282 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.282 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.282 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.282 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.282 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.282 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.283 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.283 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.283 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.283 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.283 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.283 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.283 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.283 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.283 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.283 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.283 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.283 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.283 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.283 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.284 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.284 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.284 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.284 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.284 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.284 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.284 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.284 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.284 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.284 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.284 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.284 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.284 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.284 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.285 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.285 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.285 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.285 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.285 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.285 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.285 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.285 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.285 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.285 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "localpool", 2024-09-13T18:34:14.285 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.285 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.285 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.285 INFO:teuthology.orchestra.run.smithi114.stdout: "failure_domain": { 2024-09-13T18:34:14.286 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "failure_domain", 2024-09-13T18:34:14.286 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.286 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.286 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.286 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "host", 2024-09-13T18:34:14.286 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.286 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.286 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.286 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "failure domain for any created local pool", 2024-09-13T18:34:14.286 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-13T18:34:14.286 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.286 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.286 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.286 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.286 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.287 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.287 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.287 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.287 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.287 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.287 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.287 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.287 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.287 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.287 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.287 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.287 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.288 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.288 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.288 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.288 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.288 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.288 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.288 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.288 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.288 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.288 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.288 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.288 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.288 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.288 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.289 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.289 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.289 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.289 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.289 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.289 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.289 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.289 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.289 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.289 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.289 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.289 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.289 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.290 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.290 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.290 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.290 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.290 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.290 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.290 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.290 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.290 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.290 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.290 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.290 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.290 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.290 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.290 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.291 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.291 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.291 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.291 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.291 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.291 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.291 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.291 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.291 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.291 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.291 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.291 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.291 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.291 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.292 INFO:teuthology.orchestra.run.smithi114.stdout: "min_size": { 2024-09-13T18:34:14.292 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "min_size", 2024-09-13T18:34:14.292 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.292 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.292 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.292 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.292 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.292 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.292 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.292 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "default min_size for any created local pool", 2024-09-13T18:34:14.292 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-13T18:34:14.292 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.292 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.293 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.293 INFO:teuthology.orchestra.run.smithi114.stdout: "num_rep": { 2024-09-13T18:34:14.293 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "num_rep", 2024-09-13T18:34:14.293 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.293 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.293 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.293 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "3", 2024-09-13T18:34:14.293 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.293 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.293 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.293 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "default replica count for any created local pool", 2024-09-13T18:34:14.293 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.293 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.293 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.294 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.294 INFO:teuthology.orchestra.run.smithi114.stdout: "pg_num": { 2024-09-13T18:34:14.294 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "pg_num", 2024-09-13T18:34:14.294 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.294 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.294 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.294 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "128", 2024-09-13T18:34:14.294 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.294 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.294 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.294 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "default pg_num for any created local pool", 2024-09-13T18:34:14.294 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.294 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.295 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.295 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.295 INFO:teuthology.orchestra.run.smithi114.stdout: "prefix": { 2024-09-13T18:34:14.295 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "prefix", 2024-09-13T18:34:14.295 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.295 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.295 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.295 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.295 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.295 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.295 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.295 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "name prefix for any created local pool", 2024-09-13T18:34:14.295 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.295 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.296 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.296 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.296 INFO:teuthology.orchestra.run.smithi114.stdout: "subtree": { 2024-09-13T18:34:14.296 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "subtree", 2024-09-13T18:34:14.296 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.296 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.296 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.296 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "rack", 2024-09-13T18:34:14.296 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.296 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.296 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.296 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-13T18:34:14.296 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-13T18:34:14.296 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.297 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.297 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.297 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.297 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.297 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.297 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "mds_autoscaler", 2024-09-13T18:34:14.297 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.297 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.297 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.297 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.297 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.297 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.297 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.297 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.298 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.298 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.298 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.298 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.298 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.298 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.298 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.298 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.298 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.298 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.298 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.298 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.298 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.298 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.298 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.298 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.299 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.299 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.299 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.299 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.299 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.299 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.299 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.299 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.299 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.299 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.299 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.299 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.299 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.299 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.300 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.300 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.300 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.300 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.300 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.300 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.300 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.300 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.300 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.300 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.300 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.300 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.300 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.300 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.301 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.301 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.301 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.301 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.301 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.301 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.301 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.301 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.301 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.302 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.302 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.302 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.302 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.302 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.302 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.302 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.302 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.302 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.302 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.302 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.302 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.302 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.303 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.303 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.303 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "mirroring", 2024-09-13T18:34:14.303 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.303 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.303 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.303 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.303 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.303 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.303 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.303 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.303 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.303 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.304 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.304 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.304 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.304 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.304 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.304 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.304 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.304 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.304 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.304 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.304 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.304 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.304 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.304 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.305 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.305 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.305 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.305 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.305 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.305 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.305 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.305 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.305 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.305 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.305 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.305 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.305 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.305 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.306 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.306 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.306 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.306 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.306 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.306 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.306 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.306 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.306 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.306 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.306 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.306 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.306 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.307 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.307 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.307 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.307 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.307 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.307 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.307 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.307 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.307 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.307 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.307 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.307 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.307 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.308 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.308 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.308 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.308 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.308 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.308 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.308 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.308 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.308 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.308 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.308 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.308 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.309 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "nfs", 2024-09-13T18:34:14.309 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.309 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.309 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.309 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.309 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.309 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.309 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.309 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.309 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.309 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.309 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.309 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.309 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.309 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.310 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.310 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.310 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.310 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.310 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.310 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.310 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.310 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.310 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.310 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.310 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.310 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.310 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.310 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.311 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.311 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.311 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.311 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.311 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.311 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.311 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.311 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.311 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.311 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.311 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.311 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.311 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.311 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.312 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.312 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.312 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.312 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.312 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.312 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.312 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.312 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.312 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.312 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.312 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.312 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.312 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.312 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.313 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.313 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.313 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.313 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.316 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.317 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.317 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.317 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.317 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.317 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.317 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.317 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.317 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.317 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.317 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.317 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.317 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.317 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.317 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.318 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.318 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "orchestrator", 2024-09-13T18:34:14.318 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.318 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.318 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.318 INFO:teuthology.orchestra.run.smithi114.stdout: "fail_fs": { 2024-09-13T18:34:14.318 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "fail_fs", 2024-09-13T18:34:14.318 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.318 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.318 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.318 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.318 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.318 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.319 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.319 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-13T18:34:14.319 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.319 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.319 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.319 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.319 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.319 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.319 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.319 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.319 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.319 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.319 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.319 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.320 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.320 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.320 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.320 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.320 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.320 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.320 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.320 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.320 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.320 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.320 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.320 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.320 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.321 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.321 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.321 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.321 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.321 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.321 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.321 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.321 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.321 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.321 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.321 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.321 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.321 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.321 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.322 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.322 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.322 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.322 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.322 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.322 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.322 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.322 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.322 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.322 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.322 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.322 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.322 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.323 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.323 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.323 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.323 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.323 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.323 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.323 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.323 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.323 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.323 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.323 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.323 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.323 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.324 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.324 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.324 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.324 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.324 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.324 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.324 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.324 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.324 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.324 INFO:teuthology.orchestra.run.smithi114.stdout: "orchestrator": { 2024-09-13T18:34:14.324 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "orchestrator", 2024-09-13T18:34:14.324 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.324 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.325 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.325 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.325 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.325 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.325 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.325 INFO:teuthology.orchestra.run.smithi114.stdout: "cephadm", 2024-09-13T18:34:14.325 INFO:teuthology.orchestra.run.smithi114.stdout: "rook", 2024-09-13T18:34:14.325 INFO:teuthology.orchestra.run.smithi114.stdout: "test_orchestrator" 2024-09-13T18:34:14.325 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.325 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Orchestrator backend", 2024-09-13T18:34:14.325 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.325 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.325 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.325 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.326 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.326 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.326 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.326 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "osd_perf_query", 2024-09-13T18:34:14.326 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.326 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.326 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.326 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.326 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.326 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.326 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.326 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.326 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.326 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.327 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.327 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.327 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.327 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.327 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.327 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.327 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.327 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.327 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.327 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.327 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.327 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.327 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.327 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.328 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.328 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.328 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.328 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.328 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.328 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.328 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.328 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.328 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.328 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.328 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.328 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.328 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.328 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.329 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.329 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.329 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.329 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.329 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.329 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.329 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.329 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.329 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.329 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.329 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.329 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.329 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.329 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.330 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.330 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.330 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.330 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.330 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.330 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.330 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.330 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.330 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.330 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.330 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.330 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.330 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.331 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.331 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.331 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.331 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.331 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.331 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.331 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.331 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.331 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.331 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.331 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.331 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "osd_support", 2024-09-13T18:34:14.331 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.331 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.332 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.332 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.332 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.332 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.332 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.332 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.332 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.332 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.332 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.332 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.332 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.332 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.332 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.332 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.332 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.333 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.333 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.333 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.333 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.333 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.333 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.333 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.333 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.333 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.333 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.333 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.333 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.333 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.333 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.334 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.334 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.334 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.334 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.334 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.334 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.334 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.334 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.334 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.334 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.334 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.334 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.334 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.334 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.334 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.335 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.335 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.335 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.335 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.335 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.335 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.335 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.335 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.335 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.335 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.335 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.335 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.335 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.335 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.335 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.335 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.336 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.336 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.336 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.336 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.336 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.336 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.336 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.336 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.336 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.336 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.336 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.336 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.336 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.337 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.337 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "pg_autoscaler", 2024-09-13T18:34:14.337 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.337 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.337 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.337 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.337 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.337 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.337 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.337 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.337 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.337 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.337 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.338 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.338 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.338 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.338 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.338 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.338 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.338 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.338 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.338 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.338 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.338 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.338 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.338 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.338 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.338 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.339 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.339 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.339 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.339 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.339 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.339 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.339 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.339 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.339 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.339 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.339 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.339 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.339 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.340 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.340 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.340 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.340 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.340 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.340 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.340 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.340 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.340 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.340 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.340 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.340 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.340 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.341 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.341 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.341 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.341 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.341 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.341 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.341 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.341 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.341 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.341 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.341 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.341 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.341 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.341 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.342 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.342 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.342 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.342 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.342 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.342 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.342 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.342 INFO:teuthology.orchestra.run.smithi114.stdout: "sleep_interval": { 2024-09-13T18:34:14.342 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sleep_interval", 2024-09-13T18:34:14.342 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:14.342 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.342 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.342 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "60", 2024-09-13T18:34:14.342 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.343 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.343 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.343 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.343 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.343 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.343 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.343 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.343 INFO:teuthology.orchestra.run.smithi114.stdout: "threshold": { 2024-09-13T18:34:14.343 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "threshold", 2024-09-13T18:34:14.343 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "float", 2024-09-13T18:34:14.343 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.343 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.343 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "3.0", 2024-09-13T18:34:14.343 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "1.0", 2024-09-13T18:34:14.344 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.344 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.344 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "scaling threshold", 2024-09-13T18:34:14.344 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-13T18:34:14.344 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.344 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.344 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.344 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.344 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.344 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.344 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "progress", 2024-09-13T18:34:14.344 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.344 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.344 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.345 INFO:teuthology.orchestra.run.smithi114.stdout: "allow_pg_recovery_event": { 2024-09-13T18:34:14.345 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "allow_pg_recovery_event", 2024-09-13T18:34:14.345 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.345 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.345 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.345 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.345 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.345 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.345 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.345 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-13T18:34:14.345 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.345 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.345 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.346 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.346 INFO:teuthology.orchestra.run.smithi114.stdout: "enabled": { 2024-09-13T18:34:14.346 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "enabled", 2024-09-13T18:34:14.346 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.346 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.346 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.346 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.346 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.346 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.346 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.346 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.346 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.346 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.346 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.346 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.347 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.347 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.347 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.347 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.347 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.347 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.347 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.347 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.347 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.347 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.347 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.347 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.347 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.348 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.348 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.348 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.348 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.348 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.348 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.348 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.348 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.348 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.348 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.348 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.348 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.348 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.349 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.349 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.349 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.349 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.349 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.349 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.349 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.349 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.349 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.349 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.349 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.349 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.349 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.349 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.350 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.350 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.350 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.350 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.350 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.350 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.350 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.350 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.350 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.350 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.350 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.350 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.350 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.350 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.351 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.351 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.351 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.351 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.351 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.351 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.351 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.351 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.351 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.351 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.351 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.351 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.351 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.352 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.352 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.352 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.352 INFO:teuthology.orchestra.run.smithi114.stdout: "max_completed_events": { 2024-09-13T18:34:14.352 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "max_completed_events", 2024-09-13T18:34:14.352 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.352 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.352 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.352 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "50", 2024-09-13T18:34:14.352 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.352 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.352 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.352 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "number of past completed events to remember", 2024-09-13T18:34:14.352 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.353 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.353 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.353 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.353 INFO:teuthology.orchestra.run.smithi114.stdout: "sleep_interval": { 2024-09-13T18:34:14.353 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sleep_interval", 2024-09-13T18:34:14.353 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:14.353 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.353 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.353 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "5", 2024-09-13T18:34:14.353 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.353 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.353 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.353 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "how long the module is going to sleep", 2024-09-13T18:34:14.354 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.354 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.354 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.354 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.354 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.354 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.354 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.354 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "prometheus", 2024-09-13T18:34:14.354 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.354 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.354 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.354 INFO:teuthology.orchestra.run.smithi114.stdout: "cache": { 2024-09-13T18:34:14.354 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "cache", 2024-09-13T18:34:14.354 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.354 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.355 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.355 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.355 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.355 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.355 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.355 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.355 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.355 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.355 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.355 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.355 INFO:teuthology.orchestra.run.smithi114.stdout: "exclude_perf_counters": { 2024-09-13T18:34:14.355 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "exclude_perf_counters", 2024-09-13T18:34:14.355 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.355 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.356 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.356 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.356 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.356 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.356 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.356 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-13T18:34:14.356 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-13T18:34:14.356 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.356 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.356 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.356 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.356 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.356 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.357 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.357 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.357 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.357 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.357 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.357 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.357 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.357 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.357 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.357 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.357 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.357 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.357 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.357 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.358 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.358 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.358 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.358 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.358 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.358 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.358 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.358 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.358 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.358 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.358 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.358 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.358 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.359 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.359 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.359 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.359 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.359 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.359 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.359 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.359 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.359 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.359 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.359 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.359 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.359 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.359 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.360 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.360 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.360 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.360 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.360 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.360 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.360 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.360 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.360 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.360 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.360 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.361 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.361 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.361 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.361 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.361 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.361 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.361 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.361 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.361 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.361 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.361 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.362 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.362 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.362 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.362 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.362 INFO:teuthology.orchestra.run.smithi114.stdout: "rbd_stats_pools": { 2024-09-13T18:34:14.362 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rbd_stats_pools", 2024-09-13T18:34:14.362 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.362 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.362 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.362 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.362 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.362 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.362 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.363 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.363 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.363 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.363 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.363 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.363 INFO:teuthology.orchestra.run.smithi114.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-13T18:34:14.363 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-13T18:34:14.363 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.363 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.363 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.363 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "300", 2024-09-13T18:34:14.364 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.364 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.364 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.364 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.364 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.364 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.364 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.364 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.364 INFO:teuthology.orchestra.run.smithi114.stdout: "scrape_interval": { 2024-09-13T18:34:14.364 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "scrape_interval", 2024-09-13T18:34:14.364 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "float", 2024-09-13T18:34:14.365 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.365 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.365 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "15.0", 2024-09-13T18:34:14.365 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.365 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.365 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.365 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.365 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.365 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.365 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.365 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.365 INFO:teuthology.orchestra.run.smithi114.stdout: "server_addr": { 2024-09-13T18:34:14.366 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "server_addr", 2024-09-13T18:34:14.366 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.366 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.366 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.366 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "::", 2024-09-13T18:34:14.366 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.366 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.366 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.366 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-13T18:34:14.366 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.366 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.366 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.366 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.366 INFO:teuthology.orchestra.run.smithi114.stdout: "server_port": { 2024-09-13T18:34:14.367 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "server_port", 2024-09-13T18:34:14.367 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.367 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.367 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.367 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "9283", 2024-09-13T18:34:14.367 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.367 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.367 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.367 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-13T18:34:14.367 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.367 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.367 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.367 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.368 INFO:teuthology.orchestra.run.smithi114.stdout: "stale_cache_strategy": { 2024-09-13T18:34:14.368 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "stale_cache_strategy", 2024-09-13T18:34:14.368 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.368 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.368 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.368 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "log", 2024-09-13T18:34:14.368 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.368 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.368 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.368 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.368 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.368 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.369 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.369 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.369 INFO:teuthology.orchestra.run.smithi114.stdout: "standby_behaviour": { 2024-09-13T18:34:14.369 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "standby_behaviour", 2024-09-13T18:34:14.369 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.369 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.369 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.369 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "default", 2024-09-13T18:34:14.369 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.369 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.369 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.369 INFO:teuthology.orchestra.run.smithi114.stdout: "default", 2024-09-13T18:34:14.369 INFO:teuthology.orchestra.run.smithi114.stdout: "error" 2024-09-13T18:34:14.369 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.370 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.370 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.370 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.370 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.370 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.370 INFO:teuthology.orchestra.run.smithi114.stdout: "standby_error_status_code": { 2024-09-13T18:34:14.370 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "standby_error_status_code", 2024-09-13T18:34:14.370 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.370 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.370 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.370 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "500", 2024-09-13T18:34:14.370 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "400", 2024-09-13T18:34:14.370 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "599", 2024-09-13T18:34:14.370 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.371 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.371 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.371 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.371 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.371 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.371 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.371 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.371 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.371 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rbd_support", 2024-09-13T18:34:14.371 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.371 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.371 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.371 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.371 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.372 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.372 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.372 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.372 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.372 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.372 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.372 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.372 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.372 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.372 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.372 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.372 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.372 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.372 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.373 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.373 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.373 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.373 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.373 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.373 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.373 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.373 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.373 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.373 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.373 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.373 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.373 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.373 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.374 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.374 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.374 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.374 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.374 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.374 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.374 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.374 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.374 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.374 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.374 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.374 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.374 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.375 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.375 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.375 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.375 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.375 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.375 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.375 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.375 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.375 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.375 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.375 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.375 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.375 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.375 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.376 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.376 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.376 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.376 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.376 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.376 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.376 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.376 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.376 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.376 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.376 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.376 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.376 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.377 INFO:teuthology.orchestra.run.smithi114.stdout: "max_concurrent_snap_create": { 2024-09-13T18:34:14.377 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "max_concurrent_snap_create", 2024-09-13T18:34:14.377 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.377 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.377 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.377 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "10", 2024-09-13T18:34:14.377 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.377 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.377 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.377 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.377 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.377 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.378 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.378 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.378 INFO:teuthology.orchestra.run.smithi114.stdout: "mirror_snapshot_schedule": { 2024-09-13T18:34:14.378 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "mirror_snapshot_schedule", 2024-09-13T18:34:14.378 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.378 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.378 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.378 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.378 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.378 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.378 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.378 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.378 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.378 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.379 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.379 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.379 INFO:teuthology.orchestra.run.smithi114.stdout: "trash_purge_schedule": { 2024-09-13T18:34:14.379 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "trash_purge_schedule", 2024-09-13T18:34:14.379 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.379 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.379 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.379 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.379 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.379 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.379 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.379 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.379 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.380 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.380 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.380 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.380 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.380 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.380 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.380 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "restful", 2024-09-13T18:34:14.380 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.380 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.380 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.380 INFO:teuthology.orchestra.run.smithi114.stdout: "enable_auth": { 2024-09-13T18:34:14.380 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "enable_auth", 2024-09-13T18:34:14.380 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.381 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.381 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.381 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.381 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.381 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.381 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.381 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.381 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.381 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.381 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.381 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.381 INFO:teuthology.orchestra.run.smithi114.stdout: "key_file": { 2024-09-13T18:34:14.382 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "key_file", 2024-09-13T18:34:14.382 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.382 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.382 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.382 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.382 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.382 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.382 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.382 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.382 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.382 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.382 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.382 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.382 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.383 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.383 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.383 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.383 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.383 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.383 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.383 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.383 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.383 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.383 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.383 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.383 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.383 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.383 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.384 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.384 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.384 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.384 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.384 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.384 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.384 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.384 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.384 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.384 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.384 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.384 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.384 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.385 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.385 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.385 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.385 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.385 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.385 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.385 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.385 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.385 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.385 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.385 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.385 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.385 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.386 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.386 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.386 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.386 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.386 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.386 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.386 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.386 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.386 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.386 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.386 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.386 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.386 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.386 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.387 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.387 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.387 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.387 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.387 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.387 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.387 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.387 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.387 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.387 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.387 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.387 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.387 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.387 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.388 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.388 INFO:teuthology.orchestra.run.smithi114.stdout: "server_addr": { 2024-09-13T18:34:14.388 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "server_addr", 2024-09-13T18:34:14.388 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.388 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.388 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.388 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.388 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.388 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.388 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.388 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.388 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.388 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.388 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.389 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.389 INFO:teuthology.orchestra.run.smithi114.stdout: "server_port": { 2024-09-13T18:34:14.389 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "server_port", 2024-09-13T18:34:14.389 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.389 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.389 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.389 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.389 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.389 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.389 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.389 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.389 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.389 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.389 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.390 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.390 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.390 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.390 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.390 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rgw", 2024-09-13T18:34:14.390 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.390 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.390 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.390 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.390 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.390 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.390 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.390 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.391 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.391 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.391 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.391 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.391 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.391 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.391 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.391 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.391 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.391 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.391 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.391 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.391 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.391 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.392 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.392 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.392 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.392 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.392 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.392 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.392 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.392 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.392 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.392 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.392 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.392 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.392 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.393 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.393 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.393 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.393 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.393 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.393 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.393 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.393 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.393 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.393 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.393 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.393 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.393 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.393 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.394 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.394 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.394 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.394 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.394 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.394 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.394 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.394 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.394 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.394 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.394 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.394 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.394 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.394 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.394 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.395 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.395 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.395 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.395 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.395 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.395 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.395 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.395 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.395 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.395 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.395 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.395 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.395 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rook", 2024-09-13T18:34:14.396 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.396 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.396 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.396 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.396 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.396 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.396 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.396 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.396 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.396 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.396 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.396 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.396 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.396 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.397 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.397 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.397 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.397 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.397 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.397 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.397 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.397 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.397 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.397 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.397 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.397 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.397 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.397 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.398 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.398 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.398 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.398 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.398 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.398 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.398 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.398 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.398 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.398 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.398 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.398 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.398 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.398 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.399 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.399 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.399 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.399 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.399 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.399 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.399 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.399 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.399 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.399 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.399 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.399 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.399 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.400 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.400 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.400 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.400 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.400 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.400 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.400 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.400 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.400 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.400 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.400 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.400 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.400 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.401 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.401 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.401 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.401 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.401 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.401 INFO:teuthology.orchestra.run.smithi114.stdout: "storage_class": { 2024-09-13T18:34:14.401 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "storage_class", 2024-09-13T18:34:14.401 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.401 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.401 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.401 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "local", 2024-09-13T18:34:14.401 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.401 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.401 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.402 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-13T18:34:14.402 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.402 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.402 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.402 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.402 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.402 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.402 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.402 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "selftest", 2024-09-13T18:34:14.402 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.402 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.402 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.402 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.402 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.403 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.403 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.403 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.403 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.403 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.403 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.403 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.403 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.403 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.403 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.403 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.403 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.403 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.403 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.403 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.404 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.404 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.404 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.404 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.404 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.404 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.404 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.404 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.404 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.404 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.404 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.404 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.404 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.405 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.405 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.405 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.405 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.405 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.405 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.405 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.405 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.405 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.405 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.405 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.405 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.405 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.406 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.406 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.406 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.406 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.406 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.406 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.406 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.406 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.406 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.406 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.406 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.407 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.407 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.407 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.407 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.407 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.407 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.407 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.407 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.407 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.407 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.407 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.407 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.407 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.407 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.408 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.408 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.408 INFO:teuthology.orchestra.run.smithi114.stdout: "roption1": { 2024-09-13T18:34:14.408 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "roption1", 2024-09-13T18:34:14.408 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.408 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.408 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.408 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.408 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.408 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.408 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.408 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.408 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.408 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.409 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.409 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.409 INFO:teuthology.orchestra.run.smithi114.stdout: "roption2": { 2024-09-13T18:34:14.409 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "roption2", 2024-09-13T18:34:14.409 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.409 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.409 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.409 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "xyz", 2024-09-13T18:34:14.409 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.409 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.409 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.409 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.409 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.410 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.410 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.410 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.410 INFO:teuthology.orchestra.run.smithi114.stdout: "rwoption1": { 2024-09-13T18:34:14.410 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rwoption1", 2024-09-13T18:34:14.410 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.410 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.410 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.410 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.410 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.410 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.410 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.410 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.410 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.411 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.411 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.411 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.411 INFO:teuthology.orchestra.run.smithi114.stdout: "rwoption2": { 2024-09-13T18:34:14.411 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rwoption2", 2024-09-13T18:34:14.411 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.411 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.411 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.411 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.411 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.411 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.411 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.411 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.412 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.412 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.412 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.412 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.412 INFO:teuthology.orchestra.run.smithi114.stdout: "rwoption3": { 2024-09-13T18:34:14.412 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rwoption3", 2024-09-13T18:34:14.412 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "float", 2024-09-13T18:34:14.412 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.412 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.412 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.412 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.412 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.412 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.412 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.413 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.413 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.413 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.413 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.413 INFO:teuthology.orchestra.run.smithi114.stdout: "rwoption4": { 2024-09-13T18:34:14.413 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rwoption4", 2024-09-13T18:34:14.413 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.413 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.413 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.413 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.413 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.413 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.413 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.413 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.414 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.414 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.414 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.414 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.414 INFO:teuthology.orchestra.run.smithi114.stdout: "rwoption5": { 2024-09-13T18:34:14.414 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rwoption5", 2024-09-13T18:34:14.414 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.414 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.414 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.414 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.414 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.414 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.414 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.415 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.415 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.415 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.415 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.415 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.415 INFO:teuthology.orchestra.run.smithi114.stdout: "rwoption6": { 2024-09-13T18:34:14.415 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rwoption6", 2024-09-13T18:34:14.415 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.415 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.415 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.415 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.415 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.415 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.415 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.416 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.416 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.416 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.416 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.416 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.416 INFO:teuthology.orchestra.run.smithi114.stdout: "rwoption7": { 2024-09-13T18:34:14.416 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rwoption7", 2024-09-13T18:34:14.416 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.416 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.416 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.416 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.416 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "1", 2024-09-13T18:34:14.416 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "42", 2024-09-13T18:34:14.416 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.417 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.417 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.417 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.417 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.417 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.417 INFO:teuthology.orchestra.run.smithi114.stdout: "testkey": { 2024-09-13T18:34:14.417 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "testkey", 2024-09-13T18:34:14.417 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.417 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.417 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.417 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.417 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.417 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.417 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.417 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.418 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.418 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.418 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.418 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.418 INFO:teuthology.orchestra.run.smithi114.stdout: "testlkey": { 2024-09-13T18:34:14.418 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "testlkey", 2024-09-13T18:34:14.418 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.418 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.418 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.418 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.418 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.418 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.418 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.418 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.419 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.419 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.419 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.419 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.419 INFO:teuthology.orchestra.run.smithi114.stdout: "testnewline": { 2024-09-13T18:34:14.419 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "testnewline", 2024-09-13T18:34:14.419 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.419 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.419 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.419 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.419 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.419 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.419 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.420 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.420 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.420 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.420 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.420 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.420 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.420 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.420 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.420 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "snap_schedule", 2024-09-13T18:34:14.420 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.420 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.420 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.420 INFO:teuthology.orchestra.run.smithi114.stdout: "allow_m_granularity": { 2024-09-13T18:34:14.421 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "allow_m_granularity", 2024-09-13T18:34:14.421 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.421 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.421 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.421 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.421 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.421 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.421 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.421 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "allow minute scheduled snapshots", 2024-09-13T18:34:14.421 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.421 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.421 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.421 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.421 INFO:teuthology.orchestra.run.smithi114.stdout: "dump_on_update": { 2024-09-13T18:34:14.422 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "dump_on_update", 2024-09-13T18:34:14.422 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.422 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.422 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.422 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.422 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.422 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.422 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.422 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "dump database to debug log on update", 2024-09-13T18:34:14.422 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.422 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.422 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.422 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.423 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.423 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.423 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.423 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.423 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.423 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.423 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.423 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.423 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.423 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.423 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.423 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.423 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.423 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.423 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.424 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.424 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.424 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.424 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.424 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.424 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.424 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.424 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.424 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.424 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.424 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.424 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.424 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.425 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.425 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.425 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.425 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.425 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.425 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.425 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.425 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.425 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.425 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.425 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.425 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.425 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.425 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.425 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.426 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.426 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.426 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.426 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.426 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.426 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.426 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.426 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.426 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.426 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.426 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.426 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.426 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.426 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.427 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.427 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.427 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.427 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.427 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.427 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.427 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.427 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.427 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.427 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.427 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.427 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.427 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.427 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.428 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.428 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.428 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "stats", 2024-09-13T18:34:14.428 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.428 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.428 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.428 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.428 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.428 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.428 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.428 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.428 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.428 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.428 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.429 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.429 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.429 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.429 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.429 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.429 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.429 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.429 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.429 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.429 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.429 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.429 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.429 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.429 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.430 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.430 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.430 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.430 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.430 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.430 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.430 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.430 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.430 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.430 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.430 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.430 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.430 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.430 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.431 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.431 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.431 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.431 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.431 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.431 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.431 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.431 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.431 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.431 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.431 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.431 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.431 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.432 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.432 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.432 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.432 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.432 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.432 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.432 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.432 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.432 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.432 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.432 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.432 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.432 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.433 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.433 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.433 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.433 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.433 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.433 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.433 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.433 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.433 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.433 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.433 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.433 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "status", 2024-09-13T18:34:14.433 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.433 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.434 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.434 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.434 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.434 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.434 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.434 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.434 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.434 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.434 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.434 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.434 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.434 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.434 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.435 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.435 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.435 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.435 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.435 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.435 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.435 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.435 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.435 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.435 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.435 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.435 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.435 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.435 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.435 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.436 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.436 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.436 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.436 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.436 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.436 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.436 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.436 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.436 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.436 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.436 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.436 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.436 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.437 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.437 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.437 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.437 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.437 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.437 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.437 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.437 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.437 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.437 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.437 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.437 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.437 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.437 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.438 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.438 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.438 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.438 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.438 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.438 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.438 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.438 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.438 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.438 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.438 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.438 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.438 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.438 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.439 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.439 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.439 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.439 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.439 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.439 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "telegraf", 2024-09-13T18:34:14.439 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.439 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.439 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.439 INFO:teuthology.orchestra.run.smithi114.stdout: "address": { 2024-09-13T18:34:14.439 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "address", 2024-09-13T18:34:14.439 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.439 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.439 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.440 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-13T18:34:14.440 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.440 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.440 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.440 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.440 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.440 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.440 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.440 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.440 INFO:teuthology.orchestra.run.smithi114.stdout: "interval": { 2024-09-13T18:34:14.440 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "interval", 2024-09-13T18:34:14.440 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:14.440 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.441 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.441 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "15", 2024-09-13T18:34:14.441 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.441 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.441 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.441 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.441 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.441 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.441 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.441 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.441 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.441 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.441 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.441 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.441 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.442 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.442 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.442 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.442 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.442 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.442 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.442 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.442 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.442 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.442 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.442 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.442 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.442 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.443 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.443 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.443 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.443 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.443 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.443 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.443 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.443 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.443 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.443 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.443 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.443 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.443 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.444 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.444 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.444 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.444 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.444 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.444 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.444 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.444 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.444 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.444 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.444 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.444 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.444 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.444 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.445 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.445 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.445 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.445 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.445 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.445 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.445 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.445 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.445 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.445 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.445 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.445 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.445 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.445 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.446 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.446 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.446 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.446 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.446 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.446 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.446 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.446 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.446 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.446 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.446 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.446 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.446 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.446 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.447 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "telemetry", 2024-09-13T18:34:14.447 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.447 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.447 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.447 INFO:teuthology.orchestra.run.smithi114.stdout: "channel_basic": { 2024-09-13T18:34:14.447 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "channel_basic", 2024-09-13T18:34:14.447 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.447 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.447 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.447 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.447 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.447 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.447 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.447 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-13T18:34:14.448 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.448 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.448 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.448 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.448 INFO:teuthology.orchestra.run.smithi114.stdout: "channel_crash": { 2024-09-13T18:34:14.448 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "channel_crash", 2024-09-13T18:34:14.448 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.448 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.448 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.448 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.448 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.448 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.448 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.448 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-13T18:34:14.449 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.449 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.449 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.449 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.449 INFO:teuthology.orchestra.run.smithi114.stdout: "channel_device": { 2024-09-13T18:34:14.449 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "channel_device", 2024-09-13T18:34:14.449 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.449 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.449 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.449 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.449 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.449 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.449 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.449 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-13T18:34:14.450 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.450 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.450 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.450 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.450 INFO:teuthology.orchestra.run.smithi114.stdout: "channel_ident": { 2024-09-13T18:34:14.450 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "channel_ident", 2024-09-13T18:34:14.450 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.450 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.450 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.450 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.450 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.450 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.450 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.450 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-13T18:34:14.450 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.451 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.451 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.451 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.451 INFO:teuthology.orchestra.run.smithi114.stdout: "channel_perf": { 2024-09-13T18:34:14.451 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "channel_perf", 2024-09-13T18:34:14.451 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.451 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.451 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.451 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.451 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.451 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.451 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.451 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-13T18:34:14.452 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.452 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.452 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.452 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.452 INFO:teuthology.orchestra.run.smithi114.stdout: "contact": { 2024-09-13T18:34:14.452 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "contact", 2024-09-13T18:34:14.452 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.452 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.452 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.452 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.452 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.452 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.452 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.452 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.453 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.453 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.453 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.453 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.453 INFO:teuthology.orchestra.run.smithi114.stdout: "description": { 2024-09-13T18:34:14.453 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "description", 2024-09-13T18:34:14.453 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.453 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.453 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.453 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.453 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.453 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.453 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.453 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.454 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.454 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.454 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.454 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.454 INFO:teuthology.orchestra.run.smithi114.stdout: "device_url": { 2024-09-13T18:34:14.454 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "device_url", 2024-09-13T18:34:14.454 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.454 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.454 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.454 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-13T18:34:14.454 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.454 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.454 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.454 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.455 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.455 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.455 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.455 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.455 INFO:teuthology.orchestra.run.smithi114.stdout: "enabled": { 2024-09-13T18:34:14.455 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "enabled", 2024-09-13T18:34:14.455 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.455 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.455 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.455 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.455 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.455 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.455 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.455 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.456 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.456 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.456 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.456 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.456 INFO:teuthology.orchestra.run.smithi114.stdout: "interval": { 2024-09-13T18:34:14.456 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "interval", 2024-09-13T18:34:14.456 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.456 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.456 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.456 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "24", 2024-09-13T18:34:14.456 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "8", 2024-09-13T18:34:14.456 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.456 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.456 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.457 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.457 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.457 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.457 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.457 INFO:teuthology.orchestra.run.smithi114.stdout: "last_opt_revision": { 2024-09-13T18:34:14.457 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "last_opt_revision", 2024-09-13T18:34:14.457 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.457 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.457 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.457 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "1", 2024-09-13T18:34:14.457 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.457 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.457 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.457 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.458 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.458 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.458 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.458 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.458 INFO:teuthology.orchestra.run.smithi114.stdout: "leaderboard": { 2024-09-13T18:34:14.458 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "leaderboard", 2024-09-13T18:34:14.458 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.458 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.458 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.458 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.458 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.458 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.458 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.458 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.458 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.459 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.459 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.459 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.459 INFO:teuthology.orchestra.run.smithi114.stdout: "leaderboard_description": { 2024-09-13T18:34:14.459 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "leaderboard_description", 2024-09-13T18:34:14.459 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.459 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.459 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.459 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.459 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.459 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.459 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.459 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.459 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.459 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.460 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.460 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.460 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.460 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.460 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.460 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.460 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.460 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.460 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.460 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.460 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.460 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.460 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.461 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.461 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.461 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.461 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.461 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.461 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.461 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.461 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.461 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.461 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.461 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.461 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.461 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.462 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.462 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.462 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.462 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.462 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.462 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.462 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.462 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.462 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.462 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.462 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.462 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.462 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.462 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.462 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.463 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.463 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.463 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.463 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.463 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.463 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.463 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.463 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.463 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.463 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.463 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.463 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.463 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.463 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.464 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.464 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.464 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.464 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.464 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.464 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.464 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.464 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.464 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.464 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.464 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.464 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.464 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.464 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.465 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.465 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.465 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.465 INFO:teuthology.orchestra.run.smithi114.stdout: "organization": { 2024-09-13T18:34:14.465 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "organization", 2024-09-13T18:34:14.465 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.465 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.465 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.465 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.465 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.465 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.465 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.465 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.465 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.466 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.466 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.466 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.466 INFO:teuthology.orchestra.run.smithi114.stdout: "proxy": { 2024-09-13T18:34:14.466 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "proxy", 2024-09-13T18:34:14.466 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.466 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.466 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.466 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.466 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.466 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.466 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.466 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.466 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.466 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.467 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.467 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.467 INFO:teuthology.orchestra.run.smithi114.stdout: "url": { 2024-09-13T18:34:14.467 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "url", 2024-09-13T18:34:14.467 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.467 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.467 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.467 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-13T18:34:14.467 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.467 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.467 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.467 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.467 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.467 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.468 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.468 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.468 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.468 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.468 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.468 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "test_orchestrator", 2024-09-13T18:34:14.468 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.468 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.468 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.468 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.468 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.468 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.468 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.468 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.469 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.469 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.469 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.469 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.469 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.469 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.469 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.469 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.469 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.469 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.469 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.469 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.469 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.470 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.470 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.470 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.470 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.470 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.470 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.470 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.470 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.470 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.470 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.470 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.470 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.470 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.470 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.471 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.471 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.471 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.471 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.471 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.471 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.471 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.471 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.471 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.471 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.471 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.471 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.471 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.471 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.472 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.472 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.472 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.472 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.472 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.472 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.472 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.472 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.472 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.472 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.472 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.472 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.472 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.472 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.473 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.473 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.473 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.473 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.473 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.473 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.473 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.473 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.473 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.473 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.473 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.473 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.473 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.473 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "volumes", 2024-09-13T18:34:14.474 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.474 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.474 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.474 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.474 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.474 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.474 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.474 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.474 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.474 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.474 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.474 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.474 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.475 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.475 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.475 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.475 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.475 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.475 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.475 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.475 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.475 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.475 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.475 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.475 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.475 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.476 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.476 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.476 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.476 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.476 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.476 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.476 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.476 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.476 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.476 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.476 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.476 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.476 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.476 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.477 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.477 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.477 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.477 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.477 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.477 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.477 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.477 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.477 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.477 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.477 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.477 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.477 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.478 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.478 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.478 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.478 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.478 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.478 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.478 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.478 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.478 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.478 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.478 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.478 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.478 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.479 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.479 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.479 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.479 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.479 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.479 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.479 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.479 INFO:teuthology.orchestra.run.smithi114.stdout: "max_concurrent_clones": { 2024-09-13T18:34:14.479 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "max_concurrent_clones", 2024-09-13T18:34:14.479 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.479 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.479 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.480 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "4", 2024-09-13T18:34:14.480 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.480 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.480 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.480 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-13T18:34:14.480 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.480 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.480 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.480 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.480 INFO:teuthology.orchestra.run.smithi114.stdout: "snapshot_clone_delay": { 2024-09-13T18:34:14.480 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "snapshot_clone_delay", 2024-09-13T18:34:14.480 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.480 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.481 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.481 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-13T18:34:14.481 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.481 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.481 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.481 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-13T18:34:14.481 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.481 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.481 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.481 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.481 INFO:teuthology.orchestra.run.smithi114.stdout: "snapshot_clone_no_wait": { 2024-09-13T18:34:14.481 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "snapshot_clone_no_wait", 2024-09-13T18:34:14.482 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.482 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.482 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.482 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.482 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.482 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.482 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.482 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-13T18:34:14.482 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.482 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.482 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.482 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.483 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.483 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.483 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.483 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "zabbix", 2024-09-13T18:34:14.483 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.483 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.483 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.483 INFO:teuthology.orchestra.run.smithi114.stdout: "discovery_interval": { 2024-09-13T18:34:14.483 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "discovery_interval", 2024-09-13T18:34:14.483 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "uint", 2024-09-13T18:34:14.483 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.483 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.483 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "100", 2024-09-13T18:34:14.484 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.484 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.484 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.484 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.484 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.484 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.484 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.484 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.484 INFO:teuthology.orchestra.run.smithi114.stdout: "identifier": { 2024-09-13T18:34:14.484 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "identifier", 2024-09-13T18:34:14.484 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.485 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.485 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.485 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.485 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.485 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.485 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.485 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.485 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.485 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.485 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.485 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.485 INFO:teuthology.orchestra.run.smithi114.stdout: "interval": { 2024-09-13T18:34:14.485 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "interval", 2024-09-13T18:34:14.486 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:14.486 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.486 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.486 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "60", 2024-09-13T18:34:14.486 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.486 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.486 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.486 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.486 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.486 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.486 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.486 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.486 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.487 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.487 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.487 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.487 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.487 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.487 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.487 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.487 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.487 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.487 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.487 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.487 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.487 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.488 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.488 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.488 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.488 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.488 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.488 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.488 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.488 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.488 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.488 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.488 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.489 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.489 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.489 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.489 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.489 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.489 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.489 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.489 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.489 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.489 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.489 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.489 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.489 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.490 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.490 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.490 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.490 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.490 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.490 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.490 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.490 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.490 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.490 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.490 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.490 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.491 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.491 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.491 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.491 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.491 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.491 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.491 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.491 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.491 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.491 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.491 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.491 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.492 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.492 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.492 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.492 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.492 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.492 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.492 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.492 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.492 INFO:teuthology.orchestra.run.smithi114.stdout: "zabbix_host": { 2024-09-13T18:34:14.492 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "zabbix_host", 2024-09-13T18:34:14.492 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.492 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.493 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.493 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.493 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.493 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.493 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.493 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.493 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.493 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.493 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.493 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.493 INFO:teuthology.orchestra.run.smithi114.stdout: "zabbix_port": { 2024-09-13T18:34:14.493 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "zabbix_port", 2024-09-13T18:34:14.493 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.494 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.494 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.494 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "10051", 2024-09-13T18:34:14.494 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.494 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.494 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.494 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.494 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.494 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.494 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.494 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.494 INFO:teuthology.orchestra.run.smithi114.stdout: "zabbix_sender": { 2024-09-13T18:34:14.494 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "zabbix_sender", 2024-09-13T18:34:14.494 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.495 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.495 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.495 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-13T18:34:14.495 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.495 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.495 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.495 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.495 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.495 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.495 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.495 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.495 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.496 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.496 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-13T18:34:14.496 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.496 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.496 INFO:teuthology.orchestra.run.smithi114.stdout: "modules": [ 2024-09-13T18:34:14.496 INFO:teuthology.orchestra.run.smithi114.stdout: "cephadm", 2024-09-13T18:34:14.496 INFO:teuthology.orchestra.run.smithi114.stdout: "dashboard", 2024-09-13T18:34:14.496 INFO:teuthology.orchestra.run.smithi114.stdout: "iostat", 2024-09-13T18:34:14.496 INFO:teuthology.orchestra.run.smithi114.stdout: "nfs", 2024-09-13T18:34:14.496 INFO:teuthology.orchestra.run.smithi114.stdout: "restful", 2024-09-13T18:34:14.496 INFO:teuthology.orchestra.run.smithi114.stdout: "snap_schedule" 2024-09-13T18:34:14.496 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.496 INFO:teuthology.orchestra.run.smithi114.stdout: "available_modules": [ 2024-09-13T18:34:14.497 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.497 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "alerts", 2024-09-13T18:34:14.497 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.497 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.497 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.497 INFO:teuthology.orchestra.run.smithi114.stdout: "interval": { 2024-09-13T18:34:14.497 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "interval", 2024-09-13T18:34:14.497 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:14.497 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.497 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.497 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "60", 2024-09-13T18:34:14.497 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.497 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.498 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.498 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "How frequently to reexamine health status", 2024-09-13T18:34:14.498 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.498 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.498 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.498 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.498 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.498 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.498 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.498 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.498 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.498 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.498 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.499 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.499 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.499 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.499 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.499 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.499 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.499 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.499 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.499 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.499 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.499 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.499 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.499 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.499 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.500 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.500 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.500 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.500 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.500 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.500 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.500 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.500 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.500 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.500 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.500 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.500 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.501 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.501 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.501 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.501 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.501 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.501 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.501 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.501 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.501 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.501 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.501 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.501 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.501 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.502 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.502 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.502 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.502 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.502 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.502 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.502 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.502 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.502 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.502 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.502 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.502 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.502 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.503 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.503 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.503 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.503 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.503 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.503 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.503 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.503 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.503 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.503 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.503 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.503 INFO:teuthology.orchestra.run.smithi114.stdout: "smtp_destination": { 2024-09-13T18:34:14.503 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "smtp_destination", 2024-09-13T18:34:14.504 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.504 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.504 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.504 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.504 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.504 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.504 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.504 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Email address to send alerts to", 2024-09-13T18:34:14.504 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.504 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.504 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.504 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.504 INFO:teuthology.orchestra.run.smithi114.stdout: "smtp_from_name": { 2024-09-13T18:34:14.505 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "smtp_from_name", 2024-09-13T18:34:14.505 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.505 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.505 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.505 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "Ceph", 2024-09-13T18:34:14.505 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.505 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.505 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.505 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Email From: name", 2024-09-13T18:34:14.505 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.505 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.505 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.506 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.506 INFO:teuthology.orchestra.run.smithi114.stdout: "smtp_host": { 2024-09-13T18:34:14.506 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "smtp_host", 2024-09-13T18:34:14.506 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.506 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.506 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.506 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.506 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.506 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.506 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.506 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "SMTP server", 2024-09-13T18:34:14.506 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.506 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.506 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.507 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.507 INFO:teuthology.orchestra.run.smithi114.stdout: "smtp_password": { 2024-09-13T18:34:14.507 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "smtp_password", 2024-09-13T18:34:14.507 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.507 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.507 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.507 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.507 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.507 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.507 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.507 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Password to authenticate with", 2024-09-13T18:34:14.507 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.507 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.507 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.508 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.508 INFO:teuthology.orchestra.run.smithi114.stdout: "smtp_port": { 2024-09-13T18:34:14.508 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "smtp_port", 2024-09-13T18:34:14.508 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.508 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.508 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.508 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "465", 2024-09-13T18:34:14.508 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.508 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.508 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.508 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "SMTP port", 2024-09-13T18:34:14.508 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.508 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.509 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.509 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.509 INFO:teuthology.orchestra.run.smithi114.stdout: "smtp_sender": { 2024-09-13T18:34:14.509 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "smtp_sender", 2024-09-13T18:34:14.509 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.509 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.509 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.509 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.509 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.509 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.509 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.509 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "SMTP envelope sender", 2024-09-13T18:34:14.509 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.509 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.510 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.510 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.510 INFO:teuthology.orchestra.run.smithi114.stdout: "smtp_ssl": { 2024-09-13T18:34:14.510 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "smtp_ssl", 2024-09-13T18:34:14.510 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.510 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.510 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.510 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.510 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.510 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.510 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.510 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-13T18:34:14.510 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.510 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.511 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.511 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.511 INFO:teuthology.orchestra.run.smithi114.stdout: "smtp_user": { 2024-09-13T18:34:14.511 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "smtp_user", 2024-09-13T18:34:14.511 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.511 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.511 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.511 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.511 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.511 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.511 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.511 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "User to authenticate as", 2024-09-13T18:34:14.511 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.511 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.512 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.512 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.512 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.512 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.512 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.512 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "balancer", 2024-09-13T18:34:14.512 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.512 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.512 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.512 INFO:teuthology.orchestra.run.smithi114.stdout: "active": { 2024-09-13T18:34:14.512 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "active", 2024-09-13T18:34:14.512 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.512 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.512 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.513 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.513 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.513 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.513 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.513 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "automatically balance PGs across cluster", 2024-09-13T18:34:14.513 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.513 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.513 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.513 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.513 INFO:teuthology.orchestra.run.smithi114.stdout: "begin_time": { 2024-09-13T18:34:14.513 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "begin_time", 2024-09-13T18:34:14.513 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.513 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.514 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.514 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0000", 2024-09-13T18:34:14.514 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.514 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.514 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.514 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "beginning time of day to automatically balance", 2024-09-13T18:34:14.514 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-13T18:34:14.514 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.514 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.514 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.514 INFO:teuthology.orchestra.run.smithi114.stdout: "begin_weekday": { 2024-09-13T18:34:14.514 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "begin_weekday", 2024-09-13T18:34:14.514 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "uint", 2024-09-13T18:34:14.514 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.515 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.515 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-13T18:34:14.515 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "0", 2024-09-13T18:34:14.515 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "6", 2024-09-13T18:34:14.515 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.515 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-13T18:34:14.515 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-13T18:34:14.515 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.515 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.515 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.515 INFO:teuthology.orchestra.run.smithi114.stdout: "crush_compat_max_iterations": { 2024-09-13T18:34:14.515 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "crush_compat_max_iterations", 2024-09-13T18:34:14.515 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "uint", 2024-09-13T18:34:14.515 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.516 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.516 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "25", 2024-09-13T18:34:14.516 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "1", 2024-09-13T18:34:14.516 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "250", 2024-09-13T18:34:14.516 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.516 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-13T18:34:14.516 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.516 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.516 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.516 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.516 INFO:teuthology.orchestra.run.smithi114.stdout: "crush_compat_metrics": { 2024-09-13T18:34:14.516 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "crush_compat_metrics", 2024-09-13T18:34:14.516 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.516 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.517 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.517 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "pgs,objects,bytes", 2024-09-13T18:34:14.517 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.517 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.517 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.517 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-13T18:34:14.517 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-13T18:34:14.517 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.517 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.517 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.517 INFO:teuthology.orchestra.run.smithi114.stdout: "crush_compat_step": { 2024-09-13T18:34:14.517 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "crush_compat_step", 2024-09-13T18:34:14.517 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "float", 2024-09-13T18:34:14.517 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.518 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.518 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0.5", 2024-09-13T18:34:14.518 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "0.001", 2024-09-13T18:34:14.518 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "0.999", 2024-09-13T18:34:14.518 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.518 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "aggressiveness of optimization", 2024-09-13T18:34:14.518 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-13T18:34:14.518 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.518 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.518 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.518 INFO:teuthology.orchestra.run.smithi114.stdout: "end_time": { 2024-09-13T18:34:14.518 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "end_time", 2024-09-13T18:34:14.518 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.518 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.519 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.519 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "2359", 2024-09-13T18:34:14.519 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.519 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.519 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.519 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "ending time of day to automatically balance", 2024-09-13T18:34:14.519 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-13T18:34:14.519 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.519 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.519 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.519 INFO:teuthology.orchestra.run.smithi114.stdout: "end_weekday": { 2024-09-13T18:34:14.519 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "end_weekday", 2024-09-13T18:34:14.519 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "uint", 2024-09-13T18:34:14.520 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.520 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.520 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-13T18:34:14.520 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "0", 2024-09-13T18:34:14.520 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "6", 2024-09-13T18:34:14.520 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.520 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-13T18:34:14.520 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-13T18:34:14.520 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.520 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.520 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.520 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.520 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.520 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.521 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.521 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.521 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.521 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.521 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.521 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.521 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.521 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.521 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.524 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.524 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.525 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.525 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.525 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.525 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.525 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.525 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.525 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.525 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.525 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.525 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.525 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.525 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.525 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.525 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.525 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.526 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.526 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.526 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.526 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.526 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.526 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.526 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.526 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.526 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.526 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.526 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.526 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.526 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.527 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.527 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.527 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.527 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.527 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.527 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.527 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.527 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.527 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.527 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.527 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.527 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.527 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.527 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.528 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.528 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.528 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.528 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.528 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.528 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.528 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.528 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.528 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.528 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.528 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.528 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.528 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.529 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.529 INFO:teuthology.orchestra.run.smithi114.stdout: "min_score": { 2024-09-13T18:34:14.529 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "min_score", 2024-09-13T18:34:14.529 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "float", 2024-09-13T18:34:14.529 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.529 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.529 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-13T18:34:14.529 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.529 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.529 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.529 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-13T18:34:14.529 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.529 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.529 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.530 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.530 INFO:teuthology.orchestra.run.smithi114.stdout: "mode": { 2024-09-13T18:34:14.530 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "mode", 2024-09-13T18:34:14.530 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.530 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.530 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.530 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "upmap", 2024-09-13T18:34:14.530 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.530 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.530 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.530 INFO:teuthology.orchestra.run.smithi114.stdout: "crush-compat", 2024-09-13T18:34:14.530 INFO:teuthology.orchestra.run.smithi114.stdout: "none", 2024-09-13T18:34:14.530 INFO:teuthology.orchestra.run.smithi114.stdout: "upmap" 2024-09-13T18:34:14.530 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.531 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Balancer mode", 2024-09-13T18:34:14.531 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.531 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.531 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.531 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.531 INFO:teuthology.orchestra.run.smithi114.stdout: "pool_ids": { 2024-09-13T18:34:14.531 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "pool_ids", 2024-09-13T18:34:14.531 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.531 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.531 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.531 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.531 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.531 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.532 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.532 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-13T18:34:14.532 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.532 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.532 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.532 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.532 INFO:teuthology.orchestra.run.smithi114.stdout: "sleep_interval": { 2024-09-13T18:34:14.532 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sleep_interval", 2024-09-13T18:34:14.532 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:14.532 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.532 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.532 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "60", 2024-09-13T18:34:14.532 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.533 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.533 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.533 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-13T18:34:14.533 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.533 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.533 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.533 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.533 INFO:teuthology.orchestra.run.smithi114.stdout: "upmap_max_deviation": { 2024-09-13T18:34:14.533 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "upmap_max_deviation", 2024-09-13T18:34:14.533 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.533 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.533 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.533 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "5", 2024-09-13T18:34:14.534 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "1", 2024-09-13T18:34:14.534 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.534 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.534 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-13T18:34:14.534 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-13T18:34:14.534 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.534 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.534 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.534 INFO:teuthology.orchestra.run.smithi114.stdout: "upmap_max_optimizations": { 2024-09-13T18:34:14.534 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "upmap_max_optimizations", 2024-09-13T18:34:14.534 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "uint", 2024-09-13T18:34:14.534 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.535 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.535 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "10", 2024-09-13T18:34:14.535 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.535 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.535 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.535 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-13T18:34:14.535 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.535 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.535 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.535 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.535 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.535 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.536 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.536 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "cephadm", 2024-09-13T18:34:14.536 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.536 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.536 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.536 INFO:teuthology.orchestra.run.smithi114.stdout: "agent_down_multiplier": { 2024-09-13T18:34:14.536 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "agent_down_multiplier", 2024-09-13T18:34:14.536 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "float", 2024-09-13T18:34:14.536 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.536 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.536 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "3.0", 2024-09-13T18:34:14.536 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.537 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.537 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.537 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-13T18:34:14.537 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.537 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.537 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.537 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.537 INFO:teuthology.orchestra.run.smithi114.stdout: "agent_refresh_rate": { 2024-09-13T18:34:14.537 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "agent_refresh_rate", 2024-09-13T18:34:14.537 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:14.537 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.537 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.537 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "20", 2024-09-13T18:34:14.537 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.538 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.538 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.538 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-13T18:34:14.538 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.538 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.538 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.538 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.538 INFO:teuthology.orchestra.run.smithi114.stdout: "agent_starting_port": { 2024-09-13T18:34:14.538 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "agent_starting_port", 2024-09-13T18:34:14.538 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.538 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.538 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.538 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "4721", 2024-09-13T18:34:14.538 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.539 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.539 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.539 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-13T18:34:14.539 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.539 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.539 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.539 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.539 INFO:teuthology.orchestra.run.smithi114.stdout: "allow_ptrace": { 2024-09-13T18:34:14.539 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "allow_ptrace", 2024-09-13T18:34:14.539 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.539 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.539 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.539 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.539 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.540 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.540 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.540 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-13T18:34:14.540 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-13T18:34:14.540 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.540 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.540 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.540 INFO:teuthology.orchestra.run.smithi114.stdout: "autotune_interval": { 2024-09-13T18:34:14.540 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "autotune_interval", 2024-09-13T18:34:14.540 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:14.540 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.540 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.540 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "600", 2024-09-13T18:34:14.540 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.541 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.541 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.541 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-13T18:34:14.541 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.541 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.541 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.541 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.541 INFO:teuthology.orchestra.run.smithi114.stdout: "autotune_memory_target_ratio": { 2024-09-13T18:34:14.541 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "autotune_memory_target_ratio", 2024-09-13T18:34:14.541 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "float", 2024-09-13T18:34:14.541 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.541 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.541 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0.7", 2024-09-13T18:34:14.541 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.542 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.542 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.542 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-13T18:34:14.542 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.542 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.542 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.542 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.542 INFO:teuthology.orchestra.run.smithi114.stdout: "cgroups_split": { 2024-09-13T18:34:14.542 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "cgroups_split", 2024-09-13T18:34:14.542 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.542 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.542 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.543 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.543 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.543 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.543 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.543 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-13T18:34:14.543 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.543 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.543 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.543 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.543 INFO:teuthology.orchestra.run.smithi114.stdout: "config_checks_enabled": { 2024-09-13T18:34:14.543 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "config_checks_enabled", 2024-09-13T18:34:14.543 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.543 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.543 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.544 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.544 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.544 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.544 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.544 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-13T18:34:14.544 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.544 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.544 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.544 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.544 INFO:teuthology.orchestra.run.smithi114.stdout: "config_dashboard": { 2024-09-13T18:34:14.544 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "config_dashboard", 2024-09-13T18:34:14.544 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.544 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.544 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.545 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.545 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.545 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.545 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.545 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-13T18:34:14.545 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.545 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.545 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.545 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.545 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_alertmanager": { 2024-09-13T18:34:14.545 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_alertmanager", 2024-09-13T18:34:14.545 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.545 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.545 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.546 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-13T18:34:14.546 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.546 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.546 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.546 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Prometheus container image", 2024-09-13T18:34:14.546 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.546 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.546 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.546 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.546 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_base": { 2024-09-13T18:34:14.546 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_base", 2024-09-13T18:34:14.546 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.546 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.547 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.547 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-13T18:34:14.547 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.547 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.547 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.547 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Container image name, without the tag", 2024-09-13T18:34:14.547 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.547 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.547 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.547 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.547 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_elasticsearch": { 2024-09-13T18:34:14.547 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_elasticsearch", 2024-09-13T18:34:14.547 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.547 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.548 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.548 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-13T18:34:14.548 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.548 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.548 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.548 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "elasticsearch container image", 2024-09-13T18:34:14.548 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.548 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.548 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.548 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.548 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_grafana": { 2024-09-13T18:34:14.548 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_grafana", 2024-09-13T18:34:14.548 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.549 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.549 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.549 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-09-13T18:34:14.549 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.549 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.549 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.549 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Prometheus container image", 2024-09-13T18:34:14.549 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.549 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.549 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.549 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.549 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_haproxy": { 2024-09-13T18:34:14.549 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_haproxy", 2024-09-13T18:34:14.549 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.550 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.550 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.550 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-13T18:34:14.550 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.550 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.550 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.550 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "HAproxy container image", 2024-09-13T18:34:14.550 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.550 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.550 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.550 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.550 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_jaeger_agent": { 2024-09-13T18:34:14.550 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_jaeger_agent", 2024-09-13T18:34:14.550 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.551 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.551 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.551 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-13T18:34:14.551 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.551 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.551 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.551 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Jaeger agent container image", 2024-09-13T18:34:14.551 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.551 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.551 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.551 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.551 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_jaeger_collector": { 2024-09-13T18:34:14.551 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_jaeger_collector", 2024-09-13T18:34:14.551 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.552 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.552 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.552 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-13T18:34:14.552 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.552 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.552 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.552 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Jaeger collector container image", 2024-09-13T18:34:14.552 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.552 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.552 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.552 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.552 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_jaeger_query": { 2024-09-13T18:34:14.553 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_jaeger_query", 2024-09-13T18:34:14.553 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.553 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.553 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.553 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-13T18:34:14.553 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.553 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.553 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.553 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Jaeger query container image", 2024-09-13T18:34:14.553 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.553 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.554 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.554 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.554 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_keepalived": { 2024-09-13T18:34:14.554 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_keepalived", 2024-09-13T18:34:14.554 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.554 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.554 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.554 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-13T18:34:14.554 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.555 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.555 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.555 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Keepalived container image", 2024-09-13T18:34:14.555 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.555 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.555 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.555 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.555 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_loki": { 2024-09-13T18:34:14.555 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_loki", 2024-09-13T18:34:14.555 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.555 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.556 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.556 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-13T18:34:14.556 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.556 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.556 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.556 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Loki container image", 2024-09-13T18:34:14.556 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.556 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.556 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.556 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.556 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_node_exporter": { 2024-09-13T18:34:14.557 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_node_exporter", 2024-09-13T18:34:14.557 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.557 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.557 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.557 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-13T18:34:14.557 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.557 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.557 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.557 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Prometheus container image", 2024-09-13T18:34:14.557 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.557 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.557 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.557 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.557 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_nvmeof": { 2024-09-13T18:34:14.558 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_nvmeof", 2024-09-13T18:34:14.558 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.558 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.558 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.558 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-09-13T18:34:14.558 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.558 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.558 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.558 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Nvme-of container image", 2024-09-13T18:34:14.558 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.558 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.558 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.558 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.559 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_prometheus": { 2024-09-13T18:34:14.559 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_prometheus", 2024-09-13T18:34:14.559 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.559 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.559 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.559 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-13T18:34:14.559 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.559 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.559 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.559 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Prometheus container image", 2024-09-13T18:34:14.559 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.559 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.559 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.560 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.560 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_promtail": { 2024-09-13T18:34:14.560 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_promtail", 2024-09-13T18:34:14.560 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.560 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.560 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.560 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-13T18:34:14.560 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.560 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.560 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.560 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Promtail container image", 2024-09-13T18:34:14.560 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.561 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.561 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.561 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.561 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_snmp_gateway": { 2024-09-13T18:34:14.561 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_snmp_gateway", 2024-09-13T18:34:14.561 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.561 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.561 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.561 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-13T18:34:14.561 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.561 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.561 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.561 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "SNMP Gateway container image", 2024-09-13T18:34:14.561 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.562 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.562 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.562 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.562 INFO:teuthology.orchestra.run.smithi114.stdout: "container_init": { 2024-09-13T18:34:14.562 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_init", 2024-09-13T18:34:14.562 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.562 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.562 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.562 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.562 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.562 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.562 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.562 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Run podman/docker with `--init`", 2024-09-13T18:34:14.563 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.563 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.563 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.563 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.563 INFO:teuthology.orchestra.run.smithi114.stdout: "daemon_cache_timeout": { 2024-09-13T18:34:14.563 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "daemon_cache_timeout", 2024-09-13T18:34:14.563 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:14.563 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.563 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.563 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "600", 2024-09-13T18:34:14.563 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.563 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.563 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.563 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-13T18:34:14.564 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.564 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.564 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.564 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.564 INFO:teuthology.orchestra.run.smithi114.stdout: "default_cephadm_command_timeout": { 2024-09-13T18:34:14.564 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "default_cephadm_command_timeout", 2024-09-13T18:34:14.564 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.564 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.564 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.564 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "900", 2024-09-13T18:34:14.564 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.564 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.565 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.565 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-13T18:34:14.565 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.565 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.565 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.565 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.565 INFO:teuthology.orchestra.run.smithi114.stdout: "default_registry": { 2024-09-13T18:34:14.565 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "default_registry", 2024-09-13T18:34:14.565 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.565 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.565 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.565 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "docker.io", 2024-09-13T18:34:14.565 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.565 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.566 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.566 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-13T18:34:14.566 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.566 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.566 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.566 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.566 INFO:teuthology.orchestra.run.smithi114.stdout: "device_cache_timeout": { 2024-09-13T18:34:14.566 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "device_cache_timeout", 2024-09-13T18:34:14.566 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:14.566 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.566 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.566 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "1800", 2024-09-13T18:34:14.566 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.567 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.567 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.567 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "seconds to cache device inventory", 2024-09-13T18:34:14.567 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.567 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.567 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.567 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.567 INFO:teuthology.orchestra.run.smithi114.stdout: "device_enhanced_scan": { 2024-09-13T18:34:14.567 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "device_enhanced_scan", 2024-09-13T18:34:14.567 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.567 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.567 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.567 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.567 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.568 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.568 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.568 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-13T18:34:14.568 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.568 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.568 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.568 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.568 INFO:teuthology.orchestra.run.smithi114.stdout: "facts_cache_timeout": { 2024-09-13T18:34:14.568 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "facts_cache_timeout", 2024-09-13T18:34:14.568 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:14.568 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.568 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.568 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "60", 2024-09-13T18:34:14.568 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.569 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.569 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.569 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "seconds to cache host facts data", 2024-09-13T18:34:14.569 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.569 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.569 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.569 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.569 INFO:teuthology.orchestra.run.smithi114.stdout: "host_check_interval": { 2024-09-13T18:34:14.569 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "host_check_interval", 2024-09-13T18:34:14.569 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:14.569 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.569 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.569 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "600", 2024-09-13T18:34:14.569 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.569 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.570 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.570 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "how frequently to perform a host check", 2024-09-13T18:34:14.570 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.570 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.570 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.570 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.570 INFO:teuthology.orchestra.run.smithi114.stdout: "hw_monitoring": { 2024-09-13T18:34:14.570 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "hw_monitoring", 2024-09-13T18:34:14.570 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.570 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.570 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.570 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.570 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.570 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.571 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.571 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-13T18:34:14.571 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.571 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.571 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.571 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.571 INFO:teuthology.orchestra.run.smithi114.stdout: "inventory_list_all": { 2024-09-13T18:34:14.571 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "inventory_list_all", 2024-09-13T18:34:14.571 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.571 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.571 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.571 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.571 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.571 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.571 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.572 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-13T18:34:14.572 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.572 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.572 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.572 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.572 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.572 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.572 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.572 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.572 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.572 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.572 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.572 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.573 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.573 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.573 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.573 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.573 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.573 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.573 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.573 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.573 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.573 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.573 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.573 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.573 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.574 INFO:teuthology.orchestra.run.smithi114.stdout: "log_refresh_metadata": { 2024-09-13T18:34:14.574 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_refresh_metadata", 2024-09-13T18:34:14.574 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.574 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.574 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.574 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.574 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.574 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.574 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.574 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-13T18:34:14.574 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.574 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.574 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.575 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.575 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.575 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.575 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.575 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.575 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.575 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.575 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.575 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.575 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.575 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-13T18:34:14.575 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.575 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.576 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.576 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.576 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.576 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.576 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.576 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.576 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.576 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.576 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.576 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.576 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.576 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.576 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.577 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.577 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.577 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.577 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.577 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.577 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.577 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.577 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.577 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.577 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.577 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.577 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.577 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.577 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.578 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.578 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.578 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.578 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.578 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.578 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.578 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.578 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.578 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.578 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.578 INFO:teuthology.orchestra.run.smithi114.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-13T18:34:14.578 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-13T18:34:14.578 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.579 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.579 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.579 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.579 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.579 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.579 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.579 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-13T18:34:14.579 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.579 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.579 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.579 INFO:teuthology.orchestra.run.smithi114.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-13T18:34:14.579 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-13T18:34:14.579 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.580 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.580 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.580 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "*", 2024-09-13T18:34:14.580 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.580 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.580 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.580 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-13T18:34:14.580 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.580 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.580 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.580 INFO:teuthology.orchestra.run.smithi114.stdout: "max_count_per_host": { 2024-09-13T18:34:14.580 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "max_count_per_host", 2024-09-13T18:34:14.581 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.581 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.581 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.581 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "10", 2024-09-13T18:34:14.581 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.581 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.581 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.581 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "max number of daemons per service per host", 2024-09-13T18:34:14.581 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.581 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.581 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.581 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.581 INFO:teuthology.orchestra.run.smithi114.stdout: "max_osd_draining_count": { 2024-09-13T18:34:14.581 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "max_osd_draining_count", 2024-09-13T18:34:14.582 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.582 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.582 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.582 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "10", 2024-09-13T18:34:14.582 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.582 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.582 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.582 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-13T18:34:14.582 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.582 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.582 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.582 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.582 INFO:teuthology.orchestra.run.smithi114.stdout: "migration_current": { 2024-09-13T18:34:14.582 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "migration_current", 2024-09-13T18:34:14.583 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.583 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.583 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.583 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.583 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.583 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.583 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.583 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "internal - do not modify", 2024-09-13T18:34:14.583 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.583 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.583 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.583 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.583 INFO:teuthology.orchestra.run.smithi114.stdout: "mode": { 2024-09-13T18:34:14.583 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "mode", 2024-09-13T18:34:14.584 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.584 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.584 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.584 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "root", 2024-09-13T18:34:14.584 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.584 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.584 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.584 INFO:teuthology.orchestra.run.smithi114.stdout: "cephadm-package", 2024-09-13T18:34:14.584 INFO:teuthology.orchestra.run.smithi114.stdout: "root" 2024-09-13T18:34:14.584 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.584 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "mode for remote execution of cephadm", 2024-09-13T18:34:14.584 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.584 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.584 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.585 INFO:teuthology.orchestra.run.smithi114.stdout: "oob_default_addr": { 2024-09-13T18:34:14.585 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "oob_default_addr", 2024-09-13T18:34:14.585 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.585 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.585 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.585 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "169.254.1.1", 2024-09-13T18:34:14.585 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.585 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.585 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.585 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-13T18:34:14.585 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.585 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.585 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.586 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.586 INFO:teuthology.orchestra.run.smithi114.stdout: "prometheus_alerts_path": { 2024-09-13T18:34:14.586 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "prometheus_alerts_path", 2024-09-13T18:34:14.586 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.586 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.586 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.586 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-13T18:34:14.586 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.586 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.586 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.586 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-13T18:34:14.586 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.586 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.586 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.587 INFO:teuthology.orchestra.run.smithi114.stdout: "registry_insecure": { 2024-09-13T18:34:14.587 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "registry_insecure", 2024-09-13T18:34:14.587 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.587 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.587 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.587 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.587 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.587 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.587 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.587 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-13T18:34:14.587 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.587 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.587 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.588 INFO:teuthology.orchestra.run.smithi114.stdout: "registry_password": { 2024-09-13T18:34:14.588 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "registry_password", 2024-09-13T18:34:14.588 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.588 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.588 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.588 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.588 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.588 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.588 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.588 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-13T18:34:14.588 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.589 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.589 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.589 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.589 INFO:teuthology.orchestra.run.smithi114.stdout: "registry_url": { 2024-09-13T18:34:14.589 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "registry_url", 2024-09-13T18:34:14.589 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.589 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.589 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.589 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.589 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.589 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.589 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.589 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-13T18:34:14.590 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.590 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.590 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.590 INFO:teuthology.orchestra.run.smithi114.stdout: "registry_username": { 2024-09-13T18:34:14.590 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "registry_username", 2024-09-13T18:34:14.590 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.590 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.590 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.590 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.590 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.590 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.590 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.590 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-13T18:34:14.591 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.591 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.591 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.591 INFO:teuthology.orchestra.run.smithi114.stdout: "secure_monitoring_stack": { 2024-09-13T18:34:14.591 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "secure_monitoring_stack", 2024-09-13T18:34:14.591 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.591 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.591 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.591 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.591 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.591 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.591 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.591 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-13T18:34:14.591 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.592 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.592 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.592 INFO:teuthology.orchestra.run.smithi114.stdout: "service_discovery_port": { 2024-09-13T18:34:14.592 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "service_discovery_port", 2024-09-13T18:34:14.592 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.592 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.592 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.592 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "8765", 2024-09-13T18:34:14.592 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.592 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.592 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.592 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "cephadm service discovery port", 2024-09-13T18:34:14.592 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.593 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.593 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.593 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.593 INFO:teuthology.orchestra.run.smithi114.stdout: "ssh_config_file": { 2024-09-13T18:34:14.593 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ssh_config_file", 2024-09-13T18:34:14.593 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.593 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.593 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.593 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.593 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.593 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.593 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.593 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-13T18:34:14.594 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.594 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.594 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.594 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.594 INFO:teuthology.orchestra.run.smithi114.stdout: "use_agent": { 2024-09-13T18:34:14.594 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "use_agent", 2024-09-13T18:34:14.594 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.594 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.594 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.594 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.594 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.594 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.594 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.595 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-13T18:34:14.595 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.595 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.595 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.595 INFO:teuthology.orchestra.run.smithi114.stdout: "use_repo_digest": { 2024-09-13T18:34:14.595 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "use_repo_digest", 2024-09-13T18:34:14.595 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.595 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.595 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.595 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.595 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.595 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.596 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.596 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-13T18:34:14.596 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.596 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.596 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.596 INFO:teuthology.orchestra.run.smithi114.stdout: "warn_on_failed_host_check": { 2024-09-13T18:34:14.596 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "warn_on_failed_host_check", 2024-09-13T18:34:14.596 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.596 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.596 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.596 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.596 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.596 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.597 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.597 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "raise a health warning if the host check fails", 2024-09-13T18:34:14.597 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.597 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.597 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.597 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.597 INFO:teuthology.orchestra.run.smithi114.stdout: "warn_on_stray_daemons": { 2024-09-13T18:34:14.597 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "warn_on_stray_daemons", 2024-09-13T18:34:14.597 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.597 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.597 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.597 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.597 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.597 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.598 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.598 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-13T18:34:14.598 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.598 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.598 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.598 INFO:teuthology.orchestra.run.smithi114.stdout: "warn_on_stray_hosts": { 2024-09-13T18:34:14.598 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "warn_on_stray_hosts", 2024-09-13T18:34:14.598 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.598 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.598 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.598 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.598 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.598 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.599 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.599 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-13T18:34:14.599 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.599 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.599 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.599 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.599 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.599 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.599 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.599 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "crash", 2024-09-13T18:34:14.599 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.599 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.599 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.600 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.600 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.600 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.600 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.600 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.600 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.600 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.600 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.600 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.600 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.600 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.600 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.600 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.600 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.601 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.601 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.601 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.601 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.601 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.601 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.601 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.601 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.601 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.601 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.601 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.602 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.602 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.602 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.602 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.602 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.602 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.602 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.602 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.602 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.602 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.602 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.602 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.602 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.602 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.603 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.603 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.603 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.603 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.603 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.603 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.603 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.603 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.603 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.603 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.603 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.603 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.603 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.604 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.604 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.604 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.604 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.604 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.604 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.604 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.604 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.604 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.604 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.604 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.604 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.604 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.605 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.605 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.605 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.605 INFO:teuthology.orchestra.run.smithi114.stdout: "retain_interval": { 2024-09-13T18:34:14.605 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "retain_interval", 2024-09-13T18:34:14.605 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:14.605 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.605 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.605 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "31536000", 2024-09-13T18:34:14.605 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.605 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.605 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.606 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-13T18:34:14.606 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.606 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.606 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.606 INFO:teuthology.orchestra.run.smithi114.stdout: "warn_recent_interval": { 2024-09-13T18:34:14.606 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "warn_recent_interval", 2024-09-13T18:34:14.606 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:14.606 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.606 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.606 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "1209600", 2024-09-13T18:34:14.607 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.607 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.607 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.607 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-13T18:34:14.607 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.607 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.607 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.607 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.607 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.607 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "dashboard", 2024-09-13T18:34:14.608 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.608 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.608 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.608 INFO:teuthology.orchestra.run.smithi114.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-13T18:34:14.608 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-13T18:34:14.608 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.608 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.608 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.608 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "10", 2024-09-13T18:34:14.608 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.608 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.608 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.608 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.609 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.609 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.609 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.609 INFO:teuthology.orchestra.run.smithi114.stdout: "ALERTMANAGER_API_HOST": { 2024-09-13T18:34:14.609 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-13T18:34:14.609 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.609 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.609 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.609 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.609 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.609 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.610 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.610 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.610 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.610 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.610 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.610 INFO:teuthology.orchestra.run.smithi114.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-13T18:34:14.610 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-13T18:34:14.610 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.610 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.610 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.610 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.611 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.611 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.611 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.611 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.611 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.611 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.611 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.611 INFO:teuthology.orchestra.run.smithi114.stdout: "AUDIT_API_ENABLED": { 2024-09-13T18:34:14.611 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "AUDIT_API_ENABLED", 2024-09-13T18:34:14.611 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.611 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.611 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.611 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.612 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.612 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.612 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.612 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.612 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.612 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.612 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.612 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.612 INFO:teuthology.orchestra.run.smithi114.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-13T18:34:14.612 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-13T18:34:14.612 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.612 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.612 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.612 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.613 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.613 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.613 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.613 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.613 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.613 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.613 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.613 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.613 INFO:teuthology.orchestra.run.smithi114.stdout: "ENABLE_BROWSABLE_API": { 2024-09-13T18:34:14.613 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-13T18:34:14.613 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.613 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.613 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.613 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.614 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.614 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.614 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.614 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.614 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.614 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.614 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.614 INFO:teuthology.orchestra.run.smithi114.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-13T18:34:14.614 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-13T18:34:14.614 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.614 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.615 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.615 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.615 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.615 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.615 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.615 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.615 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.615 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.615 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.615 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.615 INFO:teuthology.orchestra.run.smithi114.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-13T18:34:14.615 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-13T18:34:14.615 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.615 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.616 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.616 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.616 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.616 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.616 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.616 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.616 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.616 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.616 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.616 INFO:teuthology.orchestra.run.smithi114.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-13T18:34:14.616 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-13T18:34:14.616 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.616 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.617 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.617 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.617 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.617 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.617 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.617 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.617 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.617 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.617 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.617 INFO:teuthology.orchestra.run.smithi114.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-13T18:34:14.617 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-13T18:34:14.617 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.618 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.618 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.618 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.618 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.618 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.618 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.618 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.618 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.618 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.618 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.618 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.618 INFO:teuthology.orchestra.run.smithi114.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-13T18:34:14.618 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-13T18:34:14.618 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.619 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.619 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.619 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.619 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.619 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.619 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.619 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.619 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.619 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.619 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.619 INFO:teuthology.orchestra.run.smithi114.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-13T18:34:14.619 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-13T18:34:14.619 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.620 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.620 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.620 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.620 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.620 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.620 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.620 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.620 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.620 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.620 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.620 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.620 INFO:teuthology.orchestra.run.smithi114.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-13T18:34:14.621 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-13T18:34:14.621 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.621 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.621 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.621 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.621 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.621 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.621 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.621 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.621 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.621 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.621 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.622 INFO:teuthology.orchestra.run.smithi114.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-13T18:34:14.622 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-13T18:34:14.622 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.622 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.622 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.622 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.622 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.622 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.622 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.622 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.622 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.622 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.623 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.623 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.623 INFO:teuthology.orchestra.run.smithi114.stdout: "GRAFANA_API_PASSWORD": { 2024-09-13T18:34:14.623 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-13T18:34:14.623 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.623 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.623 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.623 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "admin", 2024-09-13T18:34:14.623 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.623 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.623 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.623 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.623 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.623 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.624 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.624 INFO:teuthology.orchestra.run.smithi114.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-13T18:34:14.624 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-13T18:34:14.624 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.624 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.624 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.624 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.624 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.624 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.624 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.624 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.625 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.625 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.625 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.625 INFO:teuthology.orchestra.run.smithi114.stdout: "GRAFANA_API_URL": { 2024-09-13T18:34:14.625 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "GRAFANA_API_URL", 2024-09-13T18:34:14.625 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.625 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.625 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.625 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.625 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.625 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.625 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.626 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.626 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.626 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.626 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.626 INFO:teuthology.orchestra.run.smithi114.stdout: "GRAFANA_API_USERNAME": { 2024-09-13T18:34:14.626 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-13T18:34:14.626 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.626 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.626 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.626 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "admin", 2024-09-13T18:34:14.626 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.626 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.626 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.627 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.627 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.627 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.627 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.627 INFO:teuthology.orchestra.run.smithi114.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-13T18:34:14.627 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-13T18:34:14.627 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.627 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.627 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.627 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.627 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.627 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.627 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.628 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.628 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.628 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.628 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.628 INFO:teuthology.orchestra.run.smithi114.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-13T18:34:14.628 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-13T18:34:14.628 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.628 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.628 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.628 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.628 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.628 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.628 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.629 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.629 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.629 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.629 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.629 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.629 INFO:teuthology.orchestra.run.smithi114.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-13T18:34:14.629 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-13T18:34:14.629 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.629 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.629 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.629 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.629 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.629 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.629 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.630 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.630 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.630 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.630 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.630 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.630 INFO:teuthology.orchestra.run.smithi114.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-13T18:34:14.630 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-13T18:34:14.630 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.630 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.630 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.630 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.630 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.630 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.630 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.630 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.631 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.631 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.631 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.631 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.631 INFO:teuthology.orchestra.run.smithi114.stdout: "PROMETHEUS_API_HOST": { 2024-09-13T18:34:14.631 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-13T18:34:14.631 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.631 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.631 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.631 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.631 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.631 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.631 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.631 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.632 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.632 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.632 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.632 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.632 INFO:teuthology.orchestra.run.smithi114.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-13T18:34:14.632 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-13T18:34:14.632 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.632 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.632 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.632 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.632 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.632 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.632 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.632 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.633 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.633 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.633 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.633 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.633 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-13T18:34:14.633 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-13T18:34:14.633 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.633 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.633 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.633 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.633 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.633 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.633 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.633 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.634 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.634 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.634 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.634 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.634 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-13T18:34:14.634 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-13T18:34:14.634 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.634 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.634 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.634 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.634 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.634 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.634 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.635 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.635 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.635 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.635 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.635 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-13T18:34:14.635 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-13T18:34:14.635 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.635 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.635 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.635 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.635 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.635 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.636 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.636 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.636 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.636 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.636 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.636 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-13T18:34:14.636 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-13T18:34:14.636 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.636 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.636 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.636 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.636 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.636 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.637 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.637 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.637 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.637 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.637 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.637 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-13T18:34:14.637 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-13T18:34:14.637 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.637 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.637 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.637 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.637 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.638 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.638 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.638 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.638 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.638 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.638 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.638 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.638 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-13T18:34:14.638 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-13T18:34:14.638 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.638 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.638 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.638 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.639 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.639 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.639 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.639 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.639 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.639 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.639 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.639 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-13T18:34:14.639 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-13T18:34:14.639 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.639 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.639 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.640 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.640 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.640 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.640 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.640 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.640 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.640 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.640 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.640 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_ENABLED": { 2024-09-13T18:34:14.640 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-13T18:34:14.640 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.641 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.641 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.641 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.641 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.641 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.641 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.641 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.641 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.641 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.641 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.641 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.642 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-13T18:34:14.642 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-13T18:34:14.642 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.642 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.642 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.642 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-13T18:34:14.642 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.642 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.642 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.642 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.642 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.643 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.643 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.643 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-13T18:34:14.643 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-13T18:34:14.643 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.643 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.643 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.643 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "10", 2024-09-13T18:34:14.643 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.643 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.644 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.644 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.644 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.644 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.644 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.644 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-13T18:34:14.644 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-13T18:34:14.644 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.644 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.644 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.644 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "8", 2024-09-13T18:34:14.645 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.645 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.645 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.645 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.645 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.645 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.645 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.645 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.645 INFO:teuthology.orchestra.run.smithi114.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-13T18:34:14.645 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-13T18:34:14.645 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.645 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.645 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.646 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "45", 2024-09-13T18:34:14.646 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.646 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.646 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.646 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.646 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.646 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.646 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.646 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.646 INFO:teuthology.orchestra.run.smithi114.stdout: "RGW_API_ACCESS_KEY": { 2024-09-13T18:34:14.646 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-13T18:34:14.646 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.646 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.647 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.647 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.647 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.647 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.647 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.647 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.647 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.647 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.647 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.647 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.647 INFO:teuthology.orchestra.run.smithi114.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-13T18:34:14.647 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-13T18:34:14.647 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.647 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.648 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.648 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "admin", 2024-09-13T18:34:14.648 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.648 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.648 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.648 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.648 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.648 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.648 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.648 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.648 INFO:teuthology.orchestra.run.smithi114.stdout: "RGW_API_SECRET_KEY": { 2024-09-13T18:34:14.648 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-13T18:34:14.648 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.649 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.649 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.649 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.649 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.649 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.649 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.649 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.649 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.649 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.649 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.649 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.649 INFO:teuthology.orchestra.run.smithi114.stdout: "RGW_API_SSL_VERIFY": { 2024-09-13T18:34:14.649 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-13T18:34:14.649 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.650 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.650 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.650 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.650 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.650 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.650 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.650 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.650 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.650 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.650 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.650 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.651 INFO:teuthology.orchestra.run.smithi114.stdout: "UNSAFE_TLS_v1_2": { 2024-09-13T18:34:14.651 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-13T18:34:14.651 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.651 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.651 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.651 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.651 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.651 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.651 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.651 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.651 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.651 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.651 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.652 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.652 INFO:teuthology.orchestra.run.smithi114.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-13T18:34:14.652 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-13T18:34:14.652 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.652 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.652 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.652 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-13T18:34:14.652 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.652 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.652 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.652 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.652 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.652 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.653 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.653 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.653 INFO:teuthology.orchestra.run.smithi114.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-13T18:34:14.653 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-13T18:34:14.653 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.653 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.653 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.653 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "10", 2024-09-13T18:34:14.653 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.653 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.653 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.654 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.654 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.654 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.654 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.654 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.654 INFO:teuthology.orchestra.run.smithi114.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-13T18:34:14.654 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-13T18:34:14.654 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.654 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.654 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.654 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "5", 2024-09-13T18:34:14.655 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.655 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.655 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.655 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.655 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.655 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.655 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.655 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.655 INFO:teuthology.orchestra.run.smithi114.stdout: "cross_origin_url": { 2024-09-13T18:34:14.655 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "cross_origin_url", 2024-09-13T18:34:14.655 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.656 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.656 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.656 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.656 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.656 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.656 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.656 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.656 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.656 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.656 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.656 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.657 INFO:teuthology.orchestra.run.smithi114.stdout: "crt_file": { 2024-09-13T18:34:14.657 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "crt_file", 2024-09-13T18:34:14.657 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.657 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.657 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.657 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.657 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.657 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.657 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.657 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.657 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.657 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.657 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.658 INFO:teuthology.orchestra.run.smithi114.stdout: "debug": { 2024-09-13T18:34:14.658 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "debug", 2024-09-13T18:34:14.658 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.658 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.658 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.658 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.658 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.658 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.658 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.658 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Enable/disable debug options", 2024-09-13T18:34:14.658 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.658 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.658 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.659 INFO:teuthology.orchestra.run.smithi114.stdout: "jwt_token_ttl": { 2024-09-13T18:34:14.659 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "jwt_token_ttl", 2024-09-13T18:34:14.659 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.659 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.659 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.659 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "28800", 2024-09-13T18:34:14.659 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.659 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.659 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.659 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.659 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.659 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.659 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.659 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.660 INFO:teuthology.orchestra.run.smithi114.stdout: "key_file": { 2024-09-13T18:34:14.660 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "key_file", 2024-09-13T18:34:14.660 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.660 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.660 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.660 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.660 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.660 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.660 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.660 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.660 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.660 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.661 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.661 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.661 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.661 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.661 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.661 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.661 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.661 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.661 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.661 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.661 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.661 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.661 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.662 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.662 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.662 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.662 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.662 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.662 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.662 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.662 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.662 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.662 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.662 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.662 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.662 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.662 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.663 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.663 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.663 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.663 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.663 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.663 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.663 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.663 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.663 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.663 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.663 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.663 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.663 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.664 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.664 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.664 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.664 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.664 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.664 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.664 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.664 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.664 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.664 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.664 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.665 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.665 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.665 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.665 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.665 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.665 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.665 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.665 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.665 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.665 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.665 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.665 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.665 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.665 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.665 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.666 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.666 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.666 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.666 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.666 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.666 INFO:teuthology.orchestra.run.smithi114.stdout: "motd": { 2024-09-13T18:34:14.666 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "motd", 2024-09-13T18:34:14.666 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.666 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.666 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.666 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.666 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.666 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.667 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.667 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "The message of the day", 2024-09-13T18:34:14.667 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.667 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.667 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.667 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.667 INFO:teuthology.orchestra.run.smithi114.stdout: "redirect_resolve_ip_addr": { 2024-09-13T18:34:14.667 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "redirect_resolve_ip_addr", 2024-09-13T18:34:14.667 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.667 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.667 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.667 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.667 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.667 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.668 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.668 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.668 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.668 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.668 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.668 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.668 INFO:teuthology.orchestra.run.smithi114.stdout: "server_addr": { 2024-09-13T18:34:14.668 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "server_addr", 2024-09-13T18:34:14.668 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.668 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.668 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.668 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "::", 2024-09-13T18:34:14.668 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.668 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.669 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.669 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.669 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.669 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.669 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.669 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.669 INFO:teuthology.orchestra.run.smithi114.stdout: "server_port": { 2024-09-13T18:34:14.669 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "server_port", 2024-09-13T18:34:14.669 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.669 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.669 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.669 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "8080", 2024-09-13T18:34:14.669 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.669 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.670 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.670 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.670 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.670 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.670 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.670 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.670 INFO:teuthology.orchestra.run.smithi114.stdout: "ssl": { 2024-09-13T18:34:14.670 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ssl", 2024-09-13T18:34:14.670 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.670 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.670 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.670 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.670 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.670 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.671 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.671 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.671 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.671 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.671 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.671 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.671 INFO:teuthology.orchestra.run.smithi114.stdout: "ssl_server_port": { 2024-09-13T18:34:14.671 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ssl_server_port", 2024-09-13T18:34:14.671 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.671 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.671 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.671 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "8443", 2024-09-13T18:34:14.671 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.671 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.672 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.672 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.672 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.672 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.672 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.672 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.672 INFO:teuthology.orchestra.run.smithi114.stdout: "standby_behaviour": { 2024-09-13T18:34:14.672 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "standby_behaviour", 2024-09-13T18:34:14.672 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.672 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.672 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.672 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "redirect", 2024-09-13T18:34:14.672 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.672 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.673 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.673 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.673 INFO:teuthology.orchestra.run.smithi114.stdout: "redirect" 2024-09-13T18:34:14.673 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.673 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.673 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.673 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.673 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.673 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.673 INFO:teuthology.orchestra.run.smithi114.stdout: "standby_error_status_code": { 2024-09-13T18:34:14.673 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "standby_error_status_code", 2024-09-13T18:34:14.673 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.673 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.673 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.674 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "500", 2024-09-13T18:34:14.674 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "400", 2024-09-13T18:34:14.674 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "599", 2024-09-13T18:34:14.674 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.674 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.674 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.674 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.674 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.674 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.674 INFO:teuthology.orchestra.run.smithi114.stdout: "url_prefix": { 2024-09-13T18:34:14.674 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "url_prefix", 2024-09-13T18:34:14.674 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.674 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.674 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.675 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.675 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.675 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.675 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.675 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.675 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.675 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.675 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.675 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.675 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.675 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.675 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.675 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "devicehealth", 2024-09-13T18:34:14.675 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.676 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.676 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.676 INFO:teuthology.orchestra.run.smithi114.stdout: "enable_monitoring": { 2024-09-13T18:34:14.676 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "enable_monitoring", 2024-09-13T18:34:14.676 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.676 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.676 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.676 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.676 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.676 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.676 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.676 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "monitor device health metrics", 2024-09-13T18:34:14.676 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.676 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.677 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.677 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.677 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.677 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.677 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.677 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.677 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.677 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.677 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.677 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.677 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.677 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.677 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.678 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.678 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.678 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.678 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.678 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.678 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.678 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.678 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.678 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.678 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.678 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.678 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.678 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.679 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.679 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.679 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.679 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.679 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.679 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.679 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.679 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.679 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.679 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.679 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.679 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.679 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.679 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.680 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.680 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.680 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.680 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.680 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.680 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.680 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.680 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.680 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.680 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.680 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.680 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.681 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.681 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.681 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.681 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.681 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.681 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.681 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.681 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.681 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.681 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.681 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.681 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.682 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.682 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.682 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.682 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.682 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.682 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.682 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.682 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.682 INFO:teuthology.orchestra.run.smithi114.stdout: "mark_out_threshold": { 2024-09-13T18:34:14.682 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "mark_out_threshold", 2024-09-13T18:34:14.682 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:14.683 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.683 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.683 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "2419200", 2024-09-13T18:34:14.683 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.683 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.683 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.683 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-13T18:34:14.683 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.683 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.683 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.683 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.683 INFO:teuthology.orchestra.run.smithi114.stdout: "pool_name": { 2024-09-13T18:34:14.684 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "pool_name", 2024-09-13T18:34:14.684 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.684 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.684 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.684 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "device_health_metrics", 2024-09-13T18:34:14.684 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.684 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.684 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.684 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-13T18:34:14.684 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.684 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.684 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.684 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.685 INFO:teuthology.orchestra.run.smithi114.stdout: "retention_period": { 2024-09-13T18:34:14.685 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "retention_period", 2024-09-13T18:34:14.685 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:14.685 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.685 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.685 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "15552000", 2024-09-13T18:34:14.685 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.685 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.685 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.685 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "how long to retain device health metrics", 2024-09-13T18:34:14.685 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.685 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.685 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.685 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.686 INFO:teuthology.orchestra.run.smithi114.stdout: "scrape_frequency": { 2024-09-13T18:34:14.686 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "scrape_frequency", 2024-09-13T18:34:14.686 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:14.686 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.686 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.686 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "86400", 2024-09-13T18:34:14.686 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.686 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.686 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.686 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-13T18:34:14.686 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.686 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.686 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.686 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.687 INFO:teuthology.orchestra.run.smithi114.stdout: "self_heal": { 2024-09-13T18:34:14.687 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "self_heal", 2024-09-13T18:34:14.687 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.687 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.687 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.687 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.687 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.687 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.687 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.687 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-13T18:34:14.687 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.687 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.687 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.687 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.688 INFO:teuthology.orchestra.run.smithi114.stdout: "sleep_interval": { 2024-09-13T18:34:14.688 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sleep_interval", 2024-09-13T18:34:14.688 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:14.688 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.688 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.688 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "600", 2024-09-13T18:34:14.688 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.688 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.688 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.688 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "how frequently to wake up and check device health", 2024-09-13T18:34:14.688 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.688 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.688 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.689 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.689 INFO:teuthology.orchestra.run.smithi114.stdout: "warn_threshold": { 2024-09-13T18:34:14.689 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "warn_threshold", 2024-09-13T18:34:14.689 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:14.689 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.689 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.689 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "7257600", 2024-09-13T18:34:14.689 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.689 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.689 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.689 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-13T18:34:14.689 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.689 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.689 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.689 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.690 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.690 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.690 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.690 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "diskprediction_local", 2024-09-13T18:34:14.690 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.690 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.690 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.690 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.690 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.690 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.690 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.690 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.690 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.690 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.691 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.691 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.691 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.691 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.691 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.691 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.691 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.691 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.691 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.691 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.691 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.691 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.691 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.692 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.692 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.692 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.692 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.692 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.692 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.692 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.692 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.692 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.692 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.692 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.692 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.692 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.692 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.693 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.693 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.693 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.693 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.693 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.693 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.693 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.693 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.693 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.693 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.693 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.693 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.693 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.694 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.694 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.694 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.694 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.694 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.694 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.694 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.694 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.694 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.694 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.694 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.694 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.694 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.694 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.695 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.695 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.695 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.695 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.695 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.695 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.695 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.695 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.695 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.695 INFO:teuthology.orchestra.run.smithi114.stdout: "predict_interval": { 2024-09-13T18:34:14.695 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "predict_interval", 2024-09-13T18:34:14.695 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.695 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.696 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.696 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "86400", 2024-09-13T18:34:14.696 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.696 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.696 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.696 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.696 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.696 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.696 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.696 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.696 INFO:teuthology.orchestra.run.smithi114.stdout: "predictor_model": { 2024-09-13T18:34:14.696 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "predictor_model", 2024-09-13T18:34:14.696 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.697 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.697 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.697 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "prophetstor", 2024-09-13T18:34:14.697 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.697 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.697 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.697 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.697 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.697 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.697 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.697 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.697 INFO:teuthology.orchestra.run.smithi114.stdout: "sleep_interval": { 2024-09-13T18:34:14.697 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sleep_interval", 2024-09-13T18:34:14.697 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.698 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.698 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.698 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "600", 2024-09-13T18:34:14.698 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.698 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.698 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.698 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.698 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.698 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.698 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.698 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.698 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.698 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.698 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.699 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "influx", 2024-09-13T18:34:14.699 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": false, 2024-09-13T18:34:14.699 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "influxdb python module not found", 2024-09-13T18:34:14.699 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.699 INFO:teuthology.orchestra.run.smithi114.stdout: "batch_size": { 2024-09-13T18:34:14.699 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "batch_size", 2024-09-13T18:34:14.699 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.699 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.699 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.699 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "5000", 2024-09-13T18:34:14.699 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.699 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.699 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.699 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-13T18:34:14.700 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.700 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.700 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.700 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.700 INFO:teuthology.orchestra.run.smithi114.stdout: "database": { 2024-09-13T18:34:14.700 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "database", 2024-09-13T18:34:14.700 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.700 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.700 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.700 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "ceph", 2024-09-13T18:34:14.700 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.700 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.701 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.701 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-13T18:34:14.701 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.701 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.701 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.701 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.701 INFO:teuthology.orchestra.run.smithi114.stdout: "hostname": { 2024-09-13T18:34:14.701 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "hostname", 2024-09-13T18:34:14.701 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.701 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.701 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.701 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.701 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.701 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.702 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.702 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "InfluxDB server hostname", 2024-09-13T18:34:14.702 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.702 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.702 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.702 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.702 INFO:teuthology.orchestra.run.smithi114.stdout: "interval": { 2024-09-13T18:34:14.702 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "interval", 2024-09-13T18:34:14.702 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:14.702 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.702 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.702 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "30", 2024-09-13T18:34:14.702 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "5", 2024-09-13T18:34:14.702 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.703 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.703 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-13T18:34:14.703 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.703 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.703 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.703 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.703 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.703 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.703 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.703 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.703 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.703 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.703 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.703 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.704 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.704 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.704 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.704 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.704 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.704 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.704 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.704 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.704 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.704 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.704 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.704 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.704 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.704 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.705 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.705 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.705 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.705 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.705 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.705 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.705 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.705 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.705 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.705 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.705 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.705 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.706 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.706 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.706 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.706 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.706 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.706 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.706 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.706 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.706 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.706 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.706 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.706 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.706 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.706 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.707 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.707 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.707 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.707 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.707 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.707 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.707 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.707 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.707 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.707 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.707 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.707 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.707 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.707 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.707 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.708 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.708 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.708 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.708 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.708 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.708 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.708 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.708 INFO:teuthology.orchestra.run.smithi114.stdout: "password": { 2024-09-13T18:34:14.708 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "password", 2024-09-13T18:34:14.709 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.709 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.709 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.709 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.709 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.709 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.709 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.709 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "password of InfluxDB server user", 2024-09-13T18:34:14.709 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.709 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.709 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.709 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.709 INFO:teuthology.orchestra.run.smithi114.stdout: "port": { 2024-09-13T18:34:14.709 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "port", 2024-09-13T18:34:14.710 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.710 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.710 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.710 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "8086", 2024-09-13T18:34:14.710 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.710 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.710 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.710 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "InfluxDB server port", 2024-09-13T18:34:14.710 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.710 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.710 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.710 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.710 INFO:teuthology.orchestra.run.smithi114.stdout: "ssl": { 2024-09-13T18:34:14.711 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ssl", 2024-09-13T18:34:14.711 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.711 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.711 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.711 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "false", 2024-09-13T18:34:14.711 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.711 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.711 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.711 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-13T18:34:14.711 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.711 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.711 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.711 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.712 INFO:teuthology.orchestra.run.smithi114.stdout: "threads": { 2024-09-13T18:34:14.712 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "threads", 2024-09-13T18:34:14.712 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.712 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.712 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.712 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "5", 2024-09-13T18:34:14.712 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "1", 2024-09-13T18:34:14.712 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "32", 2024-09-13T18:34:14.712 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.712 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-13T18:34:14.712 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.712 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.712 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.713 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.713 INFO:teuthology.orchestra.run.smithi114.stdout: "username": { 2024-09-13T18:34:14.713 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "username", 2024-09-13T18:34:14.713 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.713 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.713 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.713 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.713 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.713 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.713 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.713 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "username of InfluxDB server user", 2024-09-13T18:34:14.713 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.714 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.714 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.714 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.714 INFO:teuthology.orchestra.run.smithi114.stdout: "verify_ssl": { 2024-09-13T18:34:14.714 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "verify_ssl", 2024-09-13T18:34:14.714 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.714 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.714 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.714 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "true", 2024-09-13T18:34:14.714 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.714 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.715 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.715 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-13T18:34:14.715 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.715 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.715 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.715 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.715 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.715 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.715 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.715 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "insights", 2024-09-13T18:34:14.715 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.715 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.716 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.716 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.716 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.716 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.716 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.716 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.716 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.716 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.716 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.716 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.716 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.716 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.716 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.717 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.717 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.717 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.717 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.717 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.717 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.717 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.717 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.717 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.717 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.717 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.717 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.718 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.718 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.718 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.718 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.718 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.718 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.718 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.718 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.718 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.718 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.718 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.719 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.719 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.719 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.719 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.719 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.719 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.719 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.719 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.719 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.719 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.719 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.719 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.719 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.719 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.720 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.720 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.720 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.720 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.720 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.720 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.720 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.720 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.720 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.720 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.720 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.720 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.720 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.720 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.721 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.721 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.721 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.721 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.721 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.721 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.721 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.721 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.721 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.721 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.721 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "iostat", 2024-09-13T18:34:14.721 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.721 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.721 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.722 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.722 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.722 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.722 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.722 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.722 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.722 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.722 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.722 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.722 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.722 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.722 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.722 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.722 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.723 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.723 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.723 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.723 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.723 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.723 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.723 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.723 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.723 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.723 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.723 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.723 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.723 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.723 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.724 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.724 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.724 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.724 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.724 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.724 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.724 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.724 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.724 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.724 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.724 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.724 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.724 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.725 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.725 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.725 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.725 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.725 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.729 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.729 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.729 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.729 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.729 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.729 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.729 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.729 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.729 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.729 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.729 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.729 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.730 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.730 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.730 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.730 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.730 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.730 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.730 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.730 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.730 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.730 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.730 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.730 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.730 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.731 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.731 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.731 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "k8sevents", 2024-09-13T18:34:14.731 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.731 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.731 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.731 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_event_retention_days": { 2024-09-13T18:34:14.731 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ceph_event_retention_days", 2024-09-13T18:34:14.731 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.731 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.731 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.731 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "7", 2024-09-13T18:34:14.731 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.731 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.732 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.732 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-13T18:34:14.732 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.732 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.732 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.732 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.732 INFO:teuthology.orchestra.run.smithi114.stdout: "config_check_secs": { 2024-09-13T18:34:14.732 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "config_check_secs", 2024-09-13T18:34:14.732 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.732 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.732 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.732 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "10", 2024-09-13T18:34:14.732 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "10", 2024-09-13T18:34:14.733 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.733 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.733 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-13T18:34:14.733 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.733 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.733 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.733 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.733 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.733 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.733 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.733 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.733 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.733 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.734 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.734 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.734 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.734 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.734 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.734 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.734 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.734 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.734 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.734 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.734 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.734 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.734 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.734 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.735 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.735 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.735 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.735 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.735 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.735 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.735 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.735 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.735 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.735 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.735 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.735 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.735 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.736 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.736 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.736 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.736 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.736 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.736 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.736 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.736 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.736 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.736 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.736 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.736 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.736 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.737 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.737 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.737 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.737 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.737 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.737 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.737 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.737 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.737 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.737 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.737 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.737 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.737 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.737 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.738 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.738 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.738 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.738 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.738 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.738 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.738 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.738 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.738 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.738 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.738 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.739 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.739 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.739 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "localpool", 2024-09-13T18:34:14.739 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.739 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.739 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.739 INFO:teuthology.orchestra.run.smithi114.stdout: "failure_domain": { 2024-09-13T18:34:14.739 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "failure_domain", 2024-09-13T18:34:14.739 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.739 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.739 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.739 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "host", 2024-09-13T18:34:14.739 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.739 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.740 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.740 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "failure domain for any created local pool", 2024-09-13T18:34:14.740 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-13T18:34:14.740 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.740 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.740 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.740 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.740 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.740 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.740 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.740 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.740 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.740 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.741 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.741 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.741 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.741 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.741 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.741 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.741 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.741 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.741 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.741 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.741 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.741 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.741 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.741 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.742 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.742 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.742 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.742 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.742 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.742 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.742 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.742 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.742 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.742 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.742 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.742 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.742 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.743 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.743 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.743 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.743 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.743 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.743 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.743 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.743 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.743 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.743 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.743 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.743 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.743 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.744 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.744 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.744 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.744 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.744 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.744 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.744 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.744 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.744 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.744 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.744 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.744 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.744 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.744 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.745 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.745 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.745 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.745 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.745 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.745 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.745 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.745 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.745 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.745 INFO:teuthology.orchestra.run.smithi114.stdout: "min_size": { 2024-09-13T18:34:14.745 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "min_size", 2024-09-13T18:34:14.745 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.745 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.746 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.746 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.746 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.746 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.746 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.746 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "default min_size for any created local pool", 2024-09-13T18:34:14.746 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-13T18:34:14.746 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.746 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.746 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.746 INFO:teuthology.orchestra.run.smithi114.stdout: "num_rep": { 2024-09-13T18:34:14.746 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "num_rep", 2024-09-13T18:34:14.747 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.747 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.747 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.747 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "3", 2024-09-13T18:34:14.747 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.747 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.747 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.747 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "default replica count for any created local pool", 2024-09-13T18:34:14.747 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.747 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.747 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.747 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.747 INFO:teuthology.orchestra.run.smithi114.stdout: "pg_num": { 2024-09-13T18:34:14.748 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "pg_num", 2024-09-13T18:34:14.748 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.748 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.748 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.748 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "128", 2024-09-13T18:34:14.748 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.748 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.748 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.748 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "default pg_num for any created local pool", 2024-09-13T18:34:14.748 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.748 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.748 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.748 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.749 INFO:teuthology.orchestra.run.smithi114.stdout: "prefix": { 2024-09-13T18:34:14.749 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "prefix", 2024-09-13T18:34:14.749 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.749 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.749 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.749 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.749 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.749 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.749 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.749 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "name prefix for any created local pool", 2024-09-13T18:34:14.749 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.749 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.749 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.749 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.750 INFO:teuthology.orchestra.run.smithi114.stdout: "subtree": { 2024-09-13T18:34:14.750 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "subtree", 2024-09-13T18:34:14.750 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.750 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.750 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.750 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "rack", 2024-09-13T18:34:14.750 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.750 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.750 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.750 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-13T18:34:14.750 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-13T18:34:14.750 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.750 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.750 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.751 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.751 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.751 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.751 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "mds_autoscaler", 2024-09-13T18:34:14.751 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.751 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.751 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.751 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.751 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.751 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.751 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.751 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.751 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.751 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.752 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.752 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.752 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.752 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.752 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.752 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.752 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.752 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.752 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.752 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.752 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.752 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.752 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.753 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.753 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.753 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.753 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.753 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.753 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.753 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.753 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.753 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.753 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.753 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.753 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.753 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.754 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.754 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.754 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.754 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.754 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.754 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.754 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.754 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.754 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.754 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.754 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.754 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.754 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.754 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.755 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.755 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.755 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.755 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.755 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.755 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.755 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.755 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.755 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.755 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.755 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.755 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.755 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.755 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.756 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.756 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.756 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.756 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.756 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.756 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.756 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.756 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.756 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.756 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.756 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.756 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.756 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "mirroring", 2024-09-13T18:34:14.756 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.757 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.757 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.757 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.757 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.757 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.757 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.757 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.757 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.757 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.757 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.757 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.757 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.757 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.757 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.758 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.758 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.758 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.758 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.758 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.758 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.758 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.758 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.758 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.758 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.758 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.758 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.758 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.758 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.759 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.759 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.759 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.759 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.759 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.759 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.759 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.759 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.759 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.759 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.759 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.759 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.759 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.759 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.759 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.760 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.760 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.760 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.760 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.760 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.760 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.760 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.760 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.760 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.760 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.760 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.760 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.761 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.761 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.761 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.761 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.761 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.761 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.761 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.761 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.761 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.761 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.761 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.761 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.761 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.761 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.762 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.762 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.762 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.762 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.762 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.762 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.762 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "nfs", 2024-09-13T18:34:14.762 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.762 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.762 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.762 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.762 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.762 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.762 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.763 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.763 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.763 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.763 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.763 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.763 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.763 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.763 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.763 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.763 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.763 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.763 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.763 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.763 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.763 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.764 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.764 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.764 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.764 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.764 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.764 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.764 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.764 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.764 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.764 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.764 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.764 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.765 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.765 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.765 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.765 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.765 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.765 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.765 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.765 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.765 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.765 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.765 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.765 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.765 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.766 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.766 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.766 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.766 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.766 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.766 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.766 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.766 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.766 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.766 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.766 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.766 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.766 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.766 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.767 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.767 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.767 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.767 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.767 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.767 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.767 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.767 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.767 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.767 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.767 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.767 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.767 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.767 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.768 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.768 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "orchestrator", 2024-09-13T18:34:14.768 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.768 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.768 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.768 INFO:teuthology.orchestra.run.smithi114.stdout: "fail_fs": { 2024-09-13T18:34:14.768 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "fail_fs", 2024-09-13T18:34:14.768 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.768 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.768 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.768 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.768 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.768 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.769 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.769 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-13T18:34:14.769 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.769 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.769 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.769 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.769 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.769 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.769 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.769 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.769 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.769 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.769 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.770 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.770 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.770 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.770 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.770 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.770 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.770 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.770 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.770 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.770 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.770 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.770 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.770 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.770 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.771 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.771 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.771 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.771 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.771 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.771 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.771 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.771 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.771 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.771 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.771 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.771 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.771 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.772 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.772 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.772 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.772 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.772 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.772 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.772 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.772 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.772 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.772 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.772 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.772 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.772 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.772 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.773 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.773 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.773 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.773 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.773 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.773 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.773 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.773 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.773 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.773 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.773 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.773 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.773 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.774 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.774 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.774 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.774 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.774 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.774 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.774 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.774 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.774 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.774 INFO:teuthology.orchestra.run.smithi114.stdout: "orchestrator": { 2024-09-13T18:34:14.774 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "orchestrator", 2024-09-13T18:34:14.774 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.774 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.774 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.775 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.775 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.775 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.775 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.775 INFO:teuthology.orchestra.run.smithi114.stdout: "cephadm", 2024-09-13T18:34:14.775 INFO:teuthology.orchestra.run.smithi114.stdout: "rook", 2024-09-13T18:34:14.775 INFO:teuthology.orchestra.run.smithi114.stdout: "test_orchestrator" 2024-09-13T18:34:14.775 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.775 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Orchestrator backend", 2024-09-13T18:34:14.775 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.775 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.775 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.775 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.776 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.776 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.776 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.776 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "osd_perf_query", 2024-09-13T18:34:14.776 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.776 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.776 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.776 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.776 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.776 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.776 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.777 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.777 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.777 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.777 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.777 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.777 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.777 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.777 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.777 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.777 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.777 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.777 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.777 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.778 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.778 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.778 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.778 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.778 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.778 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.778 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.778 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.778 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.778 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.778 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.778 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.779 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.779 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.779 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.779 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.779 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.779 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.779 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.779 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.779 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.779 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.779 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.780 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.780 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.780 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.780 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.780 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.780 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.780 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.780 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.780 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.780 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.780 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.780 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.780 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.781 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.781 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.781 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.781 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.781 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.781 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.781 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.781 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.781 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.781 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.781 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.781 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.782 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.782 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.782 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.782 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.782 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.782 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.782 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.782 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.782 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "osd_support", 2024-09-13T18:34:14.782 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.782 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.783 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.783 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.783 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.783 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.783 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.783 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.783 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.783 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.783 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.783 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.783 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.783 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.784 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.784 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.784 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.784 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.784 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.784 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.784 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.784 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.784 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.784 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.784 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.784 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.784 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.785 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.785 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.785 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.785 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.785 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.785 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.785 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.785 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.785 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.785 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.785 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.785 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.785 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.786 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.786 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.786 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.786 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.786 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.786 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.786 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.786 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.786 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.786 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.786 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.786 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.786 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.787 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.787 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.787 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.787 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.787 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.787 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.787 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.787 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.787 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.787 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.787 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.787 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.787 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.787 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.788 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.788 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.788 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.788 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.788 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.788 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.788 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.788 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.788 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.788 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "pg_autoscaler", 2024-09-13T18:34:14.788 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.788 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.788 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.789 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.789 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.789 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.789 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.789 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.789 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.789 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.789 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.789 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.789 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.789 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.789 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.789 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.789 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.790 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.790 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.790 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.790 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.790 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.790 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.790 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.790 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.790 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.790 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.790 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.790 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.790 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.790 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.791 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.791 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.791 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.791 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.791 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.791 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.791 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.791 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.791 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.791 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.791 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.791 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.791 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.791 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.792 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.792 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.792 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.792 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.792 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.792 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.792 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.792 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.792 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.792 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.792 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.792 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.793 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.793 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.793 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.793 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.793 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.793 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.793 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.793 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.793 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.793 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.793 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.793 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.794 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.794 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.794 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.794 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.794 INFO:teuthology.orchestra.run.smithi114.stdout: "sleep_interval": { 2024-09-13T18:34:14.794 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sleep_interval", 2024-09-13T18:34:14.794 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:14.794 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.794 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.794 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "60", 2024-09-13T18:34:14.794 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.795 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.795 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.795 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.795 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.795 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.795 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.795 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.795 INFO:teuthology.orchestra.run.smithi114.stdout: "threshold": { 2024-09-13T18:34:14.795 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "threshold", 2024-09-13T18:34:14.795 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "float", 2024-09-13T18:34:14.795 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.796 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.796 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "3.0", 2024-09-13T18:34:14.796 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "1.0", 2024-09-13T18:34:14.796 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.796 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.796 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "scaling threshold", 2024-09-13T18:34:14.796 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-13T18:34:14.796 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.796 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.796 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.796 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.797 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.797 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.797 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "progress", 2024-09-13T18:34:14.797 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.797 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.797 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.797 INFO:teuthology.orchestra.run.smithi114.stdout: "allow_pg_recovery_event": { 2024-09-13T18:34:14.797 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "allow_pg_recovery_event", 2024-09-13T18:34:14.797 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.797 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.797 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.798 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.798 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.798 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.798 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.798 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-13T18:34:14.798 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.798 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.798 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.798 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.798 INFO:teuthology.orchestra.run.smithi114.stdout: "enabled": { 2024-09-13T18:34:14.798 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "enabled", 2024-09-13T18:34:14.798 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.798 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.799 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.799 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.799 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.799 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.799 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.799 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.799 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.799 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.799 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.799 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.799 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.799 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.799 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.799 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.800 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.800 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.800 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.800 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.800 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.800 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.800 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.800 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.800 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.800 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.800 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.800 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.800 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.800 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.801 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.801 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.801 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.801 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.801 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.801 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.801 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.801 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.801 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.801 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.801 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.801 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.801 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.801 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.802 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.802 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.802 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.802 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.802 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.802 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.802 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.802 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.802 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.802 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.802 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.802 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.802 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.802 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.803 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.803 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.803 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.803 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.803 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.803 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.803 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.803 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.803 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.803 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.803 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.803 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.803 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.804 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.804 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.804 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.804 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.804 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.804 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.804 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.804 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.804 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.804 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.804 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.804 INFO:teuthology.orchestra.run.smithi114.stdout: "max_completed_events": { 2024-09-13T18:34:14.804 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "max_completed_events", 2024-09-13T18:34:14.804 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.805 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.805 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.805 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "50", 2024-09-13T18:34:14.805 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.805 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.805 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.805 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "number of past completed events to remember", 2024-09-13T18:34:14.805 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.805 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.805 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.805 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.805 INFO:teuthology.orchestra.run.smithi114.stdout: "sleep_interval": { 2024-09-13T18:34:14.805 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sleep_interval", 2024-09-13T18:34:14.805 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:14.806 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.806 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.806 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "5", 2024-09-13T18:34:14.806 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.806 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.806 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.806 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "how long the module is going to sleep", 2024-09-13T18:34:14.806 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.806 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.806 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.806 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.806 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.806 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.806 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.807 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "prometheus", 2024-09-13T18:34:14.807 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.807 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.807 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.807 INFO:teuthology.orchestra.run.smithi114.stdout: "cache": { 2024-09-13T18:34:14.807 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "cache", 2024-09-13T18:34:14.807 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.807 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.807 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.807 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.807 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.807 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.807 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.807 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.808 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.808 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.808 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.808 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.808 INFO:teuthology.orchestra.run.smithi114.stdout: "exclude_perf_counters": { 2024-09-13T18:34:14.808 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "exclude_perf_counters", 2024-09-13T18:34:14.808 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.808 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.808 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.808 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.808 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.808 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.808 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.808 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-13T18:34:14.809 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-13T18:34:14.809 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.809 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.809 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.809 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.809 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.809 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.809 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.809 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.809 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.809 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.809 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.809 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.810 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.810 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.810 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.810 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.810 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.810 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.810 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.810 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.810 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.810 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.810 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.810 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.810 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.811 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.811 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.811 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.811 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.811 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.811 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.811 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.811 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.811 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.811 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.811 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.811 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.811 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.811 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.812 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.812 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.812 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.812 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.812 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.812 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.812 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.812 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.812 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.812 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.812 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.812 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.812 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.812 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.813 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.813 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.813 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.813 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.813 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.813 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.813 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.813 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.813 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.813 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.813 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.813 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.813 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.813 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.814 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.814 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.814 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.814 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.814 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.814 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.814 INFO:teuthology.orchestra.run.smithi114.stdout: "rbd_stats_pools": { 2024-09-13T18:34:14.814 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rbd_stats_pools", 2024-09-13T18:34:14.814 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.814 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.814 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.814 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.814 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.814 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.815 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.815 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.815 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.815 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.815 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.815 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.815 INFO:teuthology.orchestra.run.smithi114.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-13T18:34:14.815 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-13T18:34:14.815 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.815 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.815 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.815 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "300", 2024-09-13T18:34:14.815 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.816 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.816 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.816 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.816 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.816 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.816 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.816 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.816 INFO:teuthology.orchestra.run.smithi114.stdout: "scrape_interval": { 2024-09-13T18:34:14.816 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "scrape_interval", 2024-09-13T18:34:14.816 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "float", 2024-09-13T18:34:14.816 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.816 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.816 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "15.0", 2024-09-13T18:34:14.817 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.817 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.817 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.817 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.817 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.817 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.817 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.817 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.817 INFO:teuthology.orchestra.run.smithi114.stdout: "server_addr": { 2024-09-13T18:34:14.817 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "server_addr", 2024-09-13T18:34:14.817 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.817 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.817 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.818 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "::", 2024-09-13T18:34:14.818 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.818 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.818 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.818 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-13T18:34:14.818 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.818 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.818 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.818 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.818 INFO:teuthology.orchestra.run.smithi114.stdout: "server_port": { 2024-09-13T18:34:14.818 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "server_port", 2024-09-13T18:34:14.818 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.818 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.818 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.819 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "9283", 2024-09-13T18:34:14.819 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.819 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.819 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.819 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-13T18:34:14.819 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.819 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.819 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.819 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.819 INFO:teuthology.orchestra.run.smithi114.stdout: "stale_cache_strategy": { 2024-09-13T18:34:14.819 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "stale_cache_strategy", 2024-09-13T18:34:14.819 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.819 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.819 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.820 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "log", 2024-09-13T18:34:14.820 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.820 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.820 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.820 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.820 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.820 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.820 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.820 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.820 INFO:teuthology.orchestra.run.smithi114.stdout: "standby_behaviour": { 2024-09-13T18:34:14.820 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "standby_behaviour", 2024-09-13T18:34:14.820 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.820 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.821 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.821 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "default", 2024-09-13T18:34:14.821 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.821 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.821 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.821 INFO:teuthology.orchestra.run.smithi114.stdout: "default", 2024-09-13T18:34:14.821 INFO:teuthology.orchestra.run.smithi114.stdout: "error" 2024-09-13T18:34:14.821 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.821 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.821 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.821 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.821 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.821 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.821 INFO:teuthology.orchestra.run.smithi114.stdout: "standby_error_status_code": { 2024-09-13T18:34:14.822 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "standby_error_status_code", 2024-09-13T18:34:14.822 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.822 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.822 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.822 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "500", 2024-09-13T18:34:14.822 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "400", 2024-09-13T18:34:14.822 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "599", 2024-09-13T18:34:14.822 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.822 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.822 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.822 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.822 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.822 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.822 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.823 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.823 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.823 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rbd_support", 2024-09-13T18:34:14.823 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.823 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.823 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.823 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.823 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.823 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.823 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.823 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.823 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.823 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.823 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.824 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.824 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.824 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.824 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.824 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.824 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.824 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.824 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.824 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.824 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.824 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.824 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.824 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.825 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.825 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.825 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.825 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.825 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.825 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.825 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.825 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.825 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.825 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.825 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.825 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.825 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.826 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.826 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.826 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.826 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.826 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.826 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.826 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.826 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.826 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.826 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.826 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.826 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.827 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.827 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.827 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.827 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.827 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.827 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.827 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.827 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.827 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.827 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.827 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.827 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.828 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.828 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.828 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.828 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.828 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.828 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.828 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.828 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.828 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.828 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.828 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.828 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.829 INFO:teuthology.orchestra.run.smithi114.stdout: "max_concurrent_snap_create": { 2024-09-13T18:34:14.829 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "max_concurrent_snap_create", 2024-09-13T18:34:14.829 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.829 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.829 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.829 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "10", 2024-09-13T18:34:14.829 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.829 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.829 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.829 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.829 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.829 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.829 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.830 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.830 INFO:teuthology.orchestra.run.smithi114.stdout: "mirror_snapshot_schedule": { 2024-09-13T18:34:14.830 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "mirror_snapshot_schedule", 2024-09-13T18:34:14.830 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.830 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.830 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.830 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.830 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.830 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.830 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.830 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.831 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.831 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.831 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.831 INFO:teuthology.orchestra.run.smithi114.stdout: "trash_purge_schedule": { 2024-09-13T18:34:14.831 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "trash_purge_schedule", 2024-09-13T18:34:14.831 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.831 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.831 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.831 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.831 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.831 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.832 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.832 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.832 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.832 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.832 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.832 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.832 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.832 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.832 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.832 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "restful", 2024-09-13T18:34:14.832 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.832 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.833 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.833 INFO:teuthology.orchestra.run.smithi114.stdout: "enable_auth": { 2024-09-13T18:34:14.833 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "enable_auth", 2024-09-13T18:34:14.833 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.833 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.833 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.833 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.833 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.833 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.833 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.833 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.833 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.833 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.833 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.834 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.834 INFO:teuthology.orchestra.run.smithi114.stdout: "key_file": { 2024-09-13T18:34:14.834 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "key_file", 2024-09-13T18:34:14.834 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.834 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.834 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.834 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.834 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.834 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.834 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.834 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.834 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.834 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.834 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.835 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.835 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.835 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.835 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.835 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.835 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.835 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.835 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.835 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.835 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.835 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.835 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.835 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.835 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.836 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.836 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.836 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.836 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.836 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.836 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.836 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.836 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.836 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.836 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.836 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.836 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.836 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.837 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.837 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.837 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.837 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.837 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.837 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.837 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.837 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.837 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.837 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.837 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.837 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.837 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.837 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.838 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.838 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.838 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.838 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.838 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.838 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.838 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.838 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.838 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.838 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.838 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.838 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.838 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.839 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.839 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.839 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.839 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.839 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.839 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.839 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.839 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.839 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.839 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.839 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.839 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.839 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.840 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.840 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.840 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.840 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.840 INFO:teuthology.orchestra.run.smithi114.stdout: "server_addr": { 2024-09-13T18:34:14.840 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "server_addr", 2024-09-13T18:34:14.840 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.840 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.840 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.840 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.840 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.840 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.840 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.840 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.841 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.841 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.841 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.841 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.841 INFO:teuthology.orchestra.run.smithi114.stdout: "server_port": { 2024-09-13T18:34:14.841 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "server_port", 2024-09-13T18:34:14.841 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.841 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.841 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.841 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.841 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.841 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.842 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.842 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.842 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.842 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.842 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.842 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.842 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.842 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.842 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.842 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rgw", 2024-09-13T18:34:14.842 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.842 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.843 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.843 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.843 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.843 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.843 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.843 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.843 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.843 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.843 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.843 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.843 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.843 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.843 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.844 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.844 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.844 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.844 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.844 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.844 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.844 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.844 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.844 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.844 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.844 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.844 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.844 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.845 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.845 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.845 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.845 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.845 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.845 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.845 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.845 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.845 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.845 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.845 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.845 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.846 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.846 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.846 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.846 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.846 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.846 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.846 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.846 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.846 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.846 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.846 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.846 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.847 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.847 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.847 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.847 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.847 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.847 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.847 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.847 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.847 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.847 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.847 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.848 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.848 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.848 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.848 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.848 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.848 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.848 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.848 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.848 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.848 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.849 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.849 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.849 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.849 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rook", 2024-09-13T18:34:14.849 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.849 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.849 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.849 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.849 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.849 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.850 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.850 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.850 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.850 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.850 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.850 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.850 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.850 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.850 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.850 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.850 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.850 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.851 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.851 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.851 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.851 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.851 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.851 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.851 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.851 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.851 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.851 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.851 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.851 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.851 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.852 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.852 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.852 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.852 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.852 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.852 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.852 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.852 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.852 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.852 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.852 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.852 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.852 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.852 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.853 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.853 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.853 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.853 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.853 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.853 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.853 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.853 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.853 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.853 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.853 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.853 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.853 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.853 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.854 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.854 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.854 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.854 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.854 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.854 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.854 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.854 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.854 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.854 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.854 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.854 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.854 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.854 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.854 INFO:teuthology.orchestra.run.smithi114.stdout: "storage_class": { 2024-09-13T18:34:14.855 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "storage_class", 2024-09-13T18:34:14.855 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.855 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.855 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.855 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "local", 2024-09-13T18:34:14.855 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.855 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.855 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.855 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-13T18:34:14.855 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.855 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.855 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.855 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.855 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.855 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.856 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.856 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "selftest", 2024-09-13T18:34:14.856 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.856 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.856 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.856 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.856 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.856 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.856 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.856 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.856 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.856 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.856 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.857 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.857 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.857 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.857 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.857 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.857 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.857 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.857 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.857 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.857 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.857 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.857 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.857 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.858 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.858 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.858 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.858 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.858 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.858 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.858 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.858 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.858 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.858 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.858 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.858 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.858 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.858 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.859 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.859 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.859 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.859 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.859 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.859 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.859 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.859 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.859 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.859 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.859 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.859 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.859 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.860 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.860 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.860 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.860 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.860 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.860 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.860 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.860 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.860 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.860 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.860 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.860 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.860 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.860 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.861 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.861 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.861 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.861 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.861 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.861 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.861 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.861 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.861 INFO:teuthology.orchestra.run.smithi114.stdout: "roption1": { 2024-09-13T18:34:14.861 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "roption1", 2024-09-13T18:34:14.861 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.861 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.861 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.862 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.862 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.862 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.862 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.862 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.862 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.862 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.862 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.862 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.862 INFO:teuthology.orchestra.run.smithi114.stdout: "roption2": { 2024-09-13T18:34:14.862 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "roption2", 2024-09-13T18:34:14.862 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.862 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.862 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.863 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "xyz", 2024-09-13T18:34:14.863 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.863 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.863 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.863 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.863 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.863 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.863 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.863 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.863 INFO:teuthology.orchestra.run.smithi114.stdout: "rwoption1": { 2024-09-13T18:34:14.863 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rwoption1", 2024-09-13T18:34:14.863 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.863 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.863 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.863 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.864 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.864 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.864 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.864 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.864 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.864 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.864 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.864 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.864 INFO:teuthology.orchestra.run.smithi114.stdout: "rwoption2": { 2024-09-13T18:34:14.864 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rwoption2", 2024-09-13T18:34:14.864 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.864 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.864 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.865 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.865 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.865 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.865 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.865 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.865 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.865 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.865 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.865 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.865 INFO:teuthology.orchestra.run.smithi114.stdout: "rwoption3": { 2024-09-13T18:34:14.865 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rwoption3", 2024-09-13T18:34:14.865 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "float", 2024-09-13T18:34:14.865 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.865 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.866 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.866 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.866 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.866 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.866 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.866 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.866 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.866 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.866 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.866 INFO:teuthology.orchestra.run.smithi114.stdout: "rwoption4": { 2024-09-13T18:34:14.866 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rwoption4", 2024-09-13T18:34:14.866 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.866 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.867 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.867 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.867 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.867 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.867 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.867 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.867 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.867 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.867 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.867 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.867 INFO:teuthology.orchestra.run.smithi114.stdout: "rwoption5": { 2024-09-13T18:34:14.867 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rwoption5", 2024-09-13T18:34:14.867 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.867 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.867 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.868 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.868 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.868 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.868 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.868 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.868 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.868 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.868 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.868 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.868 INFO:teuthology.orchestra.run.smithi114.stdout: "rwoption6": { 2024-09-13T18:34:14.868 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rwoption6", 2024-09-13T18:34:14.868 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.868 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.869 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.869 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.869 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.869 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.869 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.869 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.869 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.869 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.869 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.869 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.869 INFO:teuthology.orchestra.run.smithi114.stdout: "rwoption7": { 2024-09-13T18:34:14.869 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rwoption7", 2024-09-13T18:34:14.869 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.870 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.870 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.870 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.870 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "1", 2024-09-13T18:34:14.870 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "42", 2024-09-13T18:34:14.870 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.870 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.870 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.870 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.870 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.870 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.870 INFO:teuthology.orchestra.run.smithi114.stdout: "testkey": { 2024-09-13T18:34:14.870 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "testkey", 2024-09-13T18:34:14.870 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.871 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.871 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.871 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.871 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.871 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.871 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.871 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.871 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.871 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.871 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.871 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.871 INFO:teuthology.orchestra.run.smithi114.stdout: "testlkey": { 2024-09-13T18:34:14.871 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "testlkey", 2024-09-13T18:34:14.872 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.872 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.872 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.872 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.872 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.872 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.872 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.872 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.872 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.872 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.872 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.873 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.873 INFO:teuthology.orchestra.run.smithi114.stdout: "testnewline": { 2024-09-13T18:34:14.873 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "testnewline", 2024-09-13T18:34:14.873 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.873 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.873 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.873 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.873 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.873 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.873 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.874 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.874 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.874 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.874 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.874 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.874 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.874 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.874 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.874 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "snap_schedule", 2024-09-13T18:34:14.874 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.875 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.875 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.875 INFO:teuthology.orchestra.run.smithi114.stdout: "allow_m_granularity": { 2024-09-13T18:34:14.875 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "allow_m_granularity", 2024-09-13T18:34:14.875 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.875 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.875 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.875 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.875 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.875 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.875 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.875 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "allow minute scheduled snapshots", 2024-09-13T18:34:14.876 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.876 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.876 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.876 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.876 INFO:teuthology.orchestra.run.smithi114.stdout: "dump_on_update": { 2024-09-13T18:34:14.876 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "dump_on_update", 2024-09-13T18:34:14.876 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.876 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.876 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.876 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.877 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.877 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.877 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.877 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "dump database to debug log on update", 2024-09-13T18:34:14.877 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.877 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.877 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.877 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.877 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.877 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.878 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.878 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.878 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.878 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.878 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.878 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.878 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.878 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.878 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.878 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.878 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.878 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.879 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.879 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.879 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.879 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.879 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.879 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.879 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.879 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.879 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.879 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.879 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.880 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.880 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.880 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.880 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.880 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.880 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.880 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.880 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.880 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.880 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.881 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.881 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.881 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.881 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.881 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.881 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.881 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.881 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.881 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.881 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.881 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.881 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.882 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.882 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.882 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.882 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.882 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.882 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.882 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.882 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.882 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.882 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.882 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.882 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.882 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.882 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.883 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.883 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.883 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.883 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.883 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.883 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.883 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.883 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.883 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.883 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.883 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.883 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.883 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "stats", 2024-09-13T18:34:14.883 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.884 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.884 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.884 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.884 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.884 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.884 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.884 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.884 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.884 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.884 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.884 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.884 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.884 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.885 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.885 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.885 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.885 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.885 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.885 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.885 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.885 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.885 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.885 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.885 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.885 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.885 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.885 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.886 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.886 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.886 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.886 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.886 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.886 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.886 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.886 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.886 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.886 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.886 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.886 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.886 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.886 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.887 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.887 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.887 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.887 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.887 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.887 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.887 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.887 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.887 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.887 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.887 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.887 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.887 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.888 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.888 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.888 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.888 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.888 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.888 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.888 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.888 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.888 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.888 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.889 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.889 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.889 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.889 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.889 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.889 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.889 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.889 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.889 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.889 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.889 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.889 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "status", 2024-09-13T18:34:14.890 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.890 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.890 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.890 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.890 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.890 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.890 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.890 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.890 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.890 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.890 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.890 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.890 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.890 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.891 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.891 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.891 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.891 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.891 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.891 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.891 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.891 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.891 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.891 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.891 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.891 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.891 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.891 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.892 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.892 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.892 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.892 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.892 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.892 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.892 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.892 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.892 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.892 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.892 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.892 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.892 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.892 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.893 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.893 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.893 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.893 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.893 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.893 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.893 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.893 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.893 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.893 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.893 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.893 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.893 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.893 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.894 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.894 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.894 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.894 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.894 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.894 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.894 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.894 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.894 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.894 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.894 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.894 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.895 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.895 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.895 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.895 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.895 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.895 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.895 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.895 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.895 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "telegraf", 2024-09-13T18:34:14.895 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.895 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.895 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.895 INFO:teuthology.orchestra.run.smithi114.stdout: "address": { 2024-09-13T18:34:14.896 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "address", 2024-09-13T18:34:14.896 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.896 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.896 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.896 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-13T18:34:14.896 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.896 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.896 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.896 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.896 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.896 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.896 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.896 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.896 INFO:teuthology.orchestra.run.smithi114.stdout: "interval": { 2024-09-13T18:34:14.897 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "interval", 2024-09-13T18:34:14.897 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:14.897 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.897 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.897 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "15", 2024-09-13T18:34:14.897 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.897 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.897 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.897 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.897 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.897 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.897 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.897 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.897 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.898 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.898 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.898 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.898 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.898 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.898 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.898 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.898 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.898 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.898 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.898 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.898 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.898 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.898 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.899 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.899 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.899 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.899 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.899 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.899 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.899 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.899 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.899 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.899 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.899 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.899 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.899 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.899 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.900 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.900 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.900 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.900 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.900 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.900 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.900 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.900 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.900 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.900 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.900 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.900 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.900 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.900 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.901 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.901 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.901 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.901 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.901 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.901 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.901 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.901 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.901 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.901 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.901 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.901 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.901 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.901 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.902 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.902 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.902 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.902 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.902 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.902 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.902 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.902 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.902 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.902 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.902 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.902 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.902 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.902 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.903 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.903 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.903 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "telemetry", 2024-09-13T18:34:14.903 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.903 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.903 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.903 INFO:teuthology.orchestra.run.smithi114.stdout: "channel_basic": { 2024-09-13T18:34:14.903 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "channel_basic", 2024-09-13T18:34:14.903 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.903 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.903 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.903 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.903 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.903 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.904 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.904 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-13T18:34:14.904 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.904 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.904 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.904 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.904 INFO:teuthology.orchestra.run.smithi114.stdout: "channel_crash": { 2024-09-13T18:34:14.904 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "channel_crash", 2024-09-13T18:34:14.904 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.904 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.904 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.904 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.904 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.904 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.905 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.905 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-13T18:34:14.905 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.905 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.905 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.905 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.905 INFO:teuthology.orchestra.run.smithi114.stdout: "channel_device": { 2024-09-13T18:34:14.905 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "channel_device", 2024-09-13T18:34:14.905 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.905 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.905 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.905 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.905 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.905 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.905 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.906 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-13T18:34:14.906 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.906 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.906 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.906 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.906 INFO:teuthology.orchestra.run.smithi114.stdout: "channel_ident": { 2024-09-13T18:34:14.906 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "channel_ident", 2024-09-13T18:34:14.906 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.906 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.906 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.906 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.906 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.906 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.906 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.907 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-13T18:34:14.907 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.907 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.907 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.907 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.907 INFO:teuthology.orchestra.run.smithi114.stdout: "channel_perf": { 2024-09-13T18:34:14.907 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "channel_perf", 2024-09-13T18:34:14.907 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.907 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.907 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.907 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.907 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.907 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.907 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.907 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-13T18:34:14.908 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.908 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.908 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.908 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.908 INFO:teuthology.orchestra.run.smithi114.stdout: "contact": { 2024-09-13T18:34:14.908 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "contact", 2024-09-13T18:34:14.908 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.908 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.908 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.908 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.908 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.908 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.908 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.909 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.909 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.909 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.909 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.909 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.909 INFO:teuthology.orchestra.run.smithi114.stdout: "description": { 2024-09-13T18:34:14.909 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "description", 2024-09-13T18:34:14.909 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.909 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.909 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.909 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.909 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.909 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.909 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.910 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.910 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.910 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.910 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.910 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.910 INFO:teuthology.orchestra.run.smithi114.stdout: "device_url": { 2024-09-13T18:34:14.910 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "device_url", 2024-09-13T18:34:14.910 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.910 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.910 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.910 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-13T18:34:14.910 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.910 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.910 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.910 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.911 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.911 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.911 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.911 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.911 INFO:teuthology.orchestra.run.smithi114.stdout: "enabled": { 2024-09-13T18:34:14.911 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "enabled", 2024-09-13T18:34:14.911 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.911 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.911 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.911 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.911 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.911 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.911 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.912 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.912 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.912 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.912 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.912 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.912 INFO:teuthology.orchestra.run.smithi114.stdout: "interval": { 2024-09-13T18:34:14.912 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "interval", 2024-09-13T18:34:14.912 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.912 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.912 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.912 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "24", 2024-09-13T18:34:14.912 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "8", 2024-09-13T18:34:14.912 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.912 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.913 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.913 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.913 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.913 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.913 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.913 INFO:teuthology.orchestra.run.smithi114.stdout: "last_opt_revision": { 2024-09-13T18:34:14.913 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "last_opt_revision", 2024-09-13T18:34:14.913 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.913 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.913 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.913 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "1", 2024-09-13T18:34:14.913 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.913 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.913 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.914 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.914 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.914 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.914 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.914 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.914 INFO:teuthology.orchestra.run.smithi114.stdout: "leaderboard": { 2024-09-13T18:34:14.914 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "leaderboard", 2024-09-13T18:34:14.914 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.914 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.914 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.914 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.914 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.914 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.914 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.915 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.915 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.915 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.915 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.915 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.915 INFO:teuthology.orchestra.run.smithi114.stdout: "leaderboard_description": { 2024-09-13T18:34:14.915 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "leaderboard_description", 2024-09-13T18:34:14.915 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.915 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.915 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.915 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.915 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.915 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.915 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.916 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.916 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.916 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.916 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.916 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.916 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.916 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.916 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.916 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.916 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.916 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.916 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.916 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.916 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.917 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.917 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.917 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.917 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.917 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.917 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.917 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.917 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.917 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.917 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.917 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.917 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.917 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.917 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.918 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.918 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.918 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.918 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.918 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.918 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.918 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.918 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.918 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.918 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.918 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.918 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.918 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.918 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.919 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.919 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.919 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.919 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.919 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.919 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.919 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.919 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.919 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.919 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.919 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.919 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.919 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.920 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.920 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.920 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.920 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.920 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.920 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.920 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.920 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.920 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.920 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.920 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.920 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.920 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.920 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.921 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.921 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.921 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.921 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.921 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.921 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.921 INFO:teuthology.orchestra.run.smithi114.stdout: "organization": { 2024-09-13T18:34:14.921 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "organization", 2024-09-13T18:34:14.921 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.921 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.921 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.921 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.921 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.922 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.922 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.922 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.922 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.922 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.922 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.922 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.922 INFO:teuthology.orchestra.run.smithi114.stdout: "proxy": { 2024-09-13T18:34:14.922 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "proxy", 2024-09-13T18:34:14.922 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.922 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.922 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.922 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.922 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.923 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.923 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.923 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.923 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.923 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.923 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.923 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.923 INFO:teuthology.orchestra.run.smithi114.stdout: "url": { 2024-09-13T18:34:14.923 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "url", 2024-09-13T18:34:14.923 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.923 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.923 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.923 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-13T18:34:14.923 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.924 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.924 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.924 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.924 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.924 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.924 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.924 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.924 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.924 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.924 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.924 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "test_orchestrator", 2024-09-13T18:34:14.924 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.924 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.925 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.925 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.925 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.925 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.925 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.925 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.925 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.925 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.925 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.925 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.925 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.925 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.925 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.925 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.926 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.926 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.926 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.926 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.926 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.926 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.926 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.926 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.926 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.926 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.926 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.926 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.926 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.926 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.927 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.927 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.927 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.927 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.927 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.927 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.927 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.927 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.927 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.927 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.927 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.927 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.927 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.927 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.928 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.928 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.928 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.928 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.928 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.928 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.928 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.928 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.928 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.928 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.929 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.929 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.933 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.933 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.933 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.933 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.933 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.933 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.933 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.933 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.933 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.933 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.933 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.933 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.933 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.934 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.934 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.934 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.934 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.934 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.934 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.934 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.934 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "volumes", 2024-09-13T18:34:14.934 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.934 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.934 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.934 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.935 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.935 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.935 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.935 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.935 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.935 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.935 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.935 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.935 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.935 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.935 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.935 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.935 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.936 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.936 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.936 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.936 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.936 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.936 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.936 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.936 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.936 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.936 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.936 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.936 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.937 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.937 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.937 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.937 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.937 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.937 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.937 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.937 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.937 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.937 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.937 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.937 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.937 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.938 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.938 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.938 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.938 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.938 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.938 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.938 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.938 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.938 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.938 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.938 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.938 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.938 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.938 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.939 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.939 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.939 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.939 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.939 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.939 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.939 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.939 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.939 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.939 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.939 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.939 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.939 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.940 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.940 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.940 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.940 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.940 INFO:teuthology.orchestra.run.smithi114.stdout: "max_concurrent_clones": { 2024-09-13T18:34:14.940 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "max_concurrent_clones", 2024-09-13T18:34:14.940 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.940 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.940 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.940 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "4", 2024-09-13T18:34:14.940 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.940 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.940 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.941 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-13T18:34:14.941 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.941 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.941 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.941 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.941 INFO:teuthology.orchestra.run.smithi114.stdout: "snapshot_clone_delay": { 2024-09-13T18:34:14.941 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "snapshot_clone_delay", 2024-09-13T18:34:14.941 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.941 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.941 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.941 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-13T18:34:14.941 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.942 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.942 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.942 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-13T18:34:14.942 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.942 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.942 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.942 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.942 INFO:teuthology.orchestra.run.smithi114.stdout: "snapshot_clone_no_wait": { 2024-09-13T18:34:14.942 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "snapshot_clone_no_wait", 2024-09-13T18:34:14.942 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.942 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.942 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.942 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-13T18:34:14.943 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.943 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.943 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.943 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-13T18:34:14.943 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.943 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.943 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.943 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.943 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.943 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.943 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.943 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "zabbix", 2024-09-13T18:34:14.943 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-13T18:34:14.944 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-13T18:34:14.944 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-13T18:34:14.944 INFO:teuthology.orchestra.run.smithi114.stdout: "discovery_interval": { 2024-09-13T18:34:14.944 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "discovery_interval", 2024-09-13T18:34:14.944 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "uint", 2024-09-13T18:34:14.944 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.944 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.944 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "100", 2024-09-13T18:34:14.944 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.944 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.944 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.944 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.945 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.945 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.945 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.945 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.945 INFO:teuthology.orchestra.run.smithi114.stdout: "identifier": { 2024-09-13T18:34:14.945 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "identifier", 2024-09-13T18:34:14.945 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.945 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.945 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.945 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.945 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.945 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.945 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.946 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.946 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.946 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.946 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.946 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.946 INFO:teuthology.orchestra.run.smithi114.stdout: "interval": { 2024-09-13T18:34:14.946 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "interval", 2024-09-13T18:34:14.946 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-13T18:34:14.946 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.946 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.946 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "60", 2024-09-13T18:34:14.946 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.947 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.947 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.947 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.947 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.947 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.947 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.947 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.947 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-13T18:34:14.947 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-13T18:34:14.947 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.947 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.947 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.947 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.948 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.948 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.948 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.948 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.948 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.948 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.948 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.948 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.948 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.948 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.948 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.948 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.949 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.949 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.949 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.949 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-13T18:34:14.949 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-13T18:34:14.949 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.949 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.949 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.949 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.949 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.949 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.949 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.949 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.950 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.950 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.950 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.950 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.950 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-13T18:34:14.950 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-13T18:34:14.950 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.950 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.950 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.950 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-13T18:34:14.950 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.950 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.951 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-13T18:34:14.951 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-13T18:34:14.951 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-13T18:34:14.951 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-13T18:34:14.951 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-13T18:34:14.951 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-13T18:34:14.951 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-13T18:34:14.951 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.951 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.951 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.951 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.951 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.952 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.952 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-13T18:34:14.952 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-13T18:34:14.952 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-13T18:34:14.952 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.952 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-13T18:34:14.952 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-13T18:34:14.952 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.952 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.952 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.952 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.952 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.953 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.953 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.953 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.953 INFO:teuthology.orchestra.run.smithi114.stdout: "zabbix_host": { 2024-09-13T18:34:14.953 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "zabbix_host", 2024-09-13T18:34:14.953 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.953 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.953 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.953 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-13T18:34:14.953 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.953 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.953 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.954 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.954 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.954 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.954 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.954 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.954 INFO:teuthology.orchestra.run.smithi114.stdout: "zabbix_port": { 2024-09-13T18:34:14.954 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "zabbix_port", 2024-09-13T18:34:14.954 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-13T18:34:14.954 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.954 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.954 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "10051", 2024-09-13T18:34:14.955 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.955 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.955 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.955 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.955 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.955 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.955 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.955 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.955 INFO:teuthology.orchestra.run.smithi114.stdout: "zabbix_sender": { 2024-09-13T18:34:14.955 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "zabbix_sender", 2024-09-13T18:34:14.955 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-13T18:34:14.955 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-13T18:34:14.955 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-13T18:34:14.956 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-13T18:34:14.956 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-13T18:34:14.956 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-13T18:34:14.956 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-13T18:34:14.956 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-13T18:34:14.956 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-13T18:34:14.956 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-13T18:34:14.956 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-13T18:34:14.956 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.956 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.956 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.956 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.956 INFO:teuthology.orchestra.run.smithi114.stdout: "services": { 2024-09-13T18:34:14.957 INFO:teuthology.orchestra.run.smithi114.stdout: "dashboard": "https://172.21.15.114:8443/" 2024-09-13T18:34:14.957 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.957 INFO:teuthology.orchestra.run.smithi114.stdout: "always_on_modules": { 2024-09-13T18:34:14.957 INFO:teuthology.orchestra.run.smithi114.stdout: "octopus": [ 2024-09-13T18:34:14.957 INFO:teuthology.orchestra.run.smithi114.stdout: "balancer", 2024-09-13T18:34:14.957 INFO:teuthology.orchestra.run.smithi114.stdout: "crash", 2024-09-13T18:34:14.957 INFO:teuthology.orchestra.run.smithi114.stdout: "devicehealth", 2024-09-13T18:34:14.957 INFO:teuthology.orchestra.run.smithi114.stdout: "orchestrator", 2024-09-13T18:34:14.957 INFO:teuthology.orchestra.run.smithi114.stdout: "pg_autoscaler", 2024-09-13T18:34:14.957 INFO:teuthology.orchestra.run.smithi114.stdout: "progress", 2024-09-13T18:34:14.957 INFO:teuthology.orchestra.run.smithi114.stdout: "rbd_support", 2024-09-13T18:34:14.957 INFO:teuthology.orchestra.run.smithi114.stdout: "status", 2024-09-13T18:34:14.957 INFO:teuthology.orchestra.run.smithi114.stdout: "telemetry", 2024-09-13T18:34:14.957 INFO:teuthology.orchestra.run.smithi114.stdout: "volumes" 2024-09-13T18:34:14.958 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.958 INFO:teuthology.orchestra.run.smithi114.stdout: "pacific": [ 2024-09-13T18:34:14.958 INFO:teuthology.orchestra.run.smithi114.stdout: "balancer", 2024-09-13T18:34:14.958 INFO:teuthology.orchestra.run.smithi114.stdout: "crash", 2024-09-13T18:34:14.958 INFO:teuthology.orchestra.run.smithi114.stdout: "devicehealth", 2024-09-13T18:34:14.958 INFO:teuthology.orchestra.run.smithi114.stdout: "orchestrator", 2024-09-13T18:34:14.958 INFO:teuthology.orchestra.run.smithi114.stdout: "pg_autoscaler", 2024-09-13T18:34:14.958 INFO:teuthology.orchestra.run.smithi114.stdout: "progress", 2024-09-13T18:34:14.958 INFO:teuthology.orchestra.run.smithi114.stdout: "rbd_support", 2024-09-13T18:34:14.958 INFO:teuthology.orchestra.run.smithi114.stdout: "status", 2024-09-13T18:34:14.958 INFO:teuthology.orchestra.run.smithi114.stdout: "telemetry", 2024-09-13T18:34:14.958 INFO:teuthology.orchestra.run.smithi114.stdout: "volumes" 2024-09-13T18:34:14.958 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.958 INFO:teuthology.orchestra.run.smithi114.stdout: "quincy": [ 2024-09-13T18:34:14.959 INFO:teuthology.orchestra.run.smithi114.stdout: "balancer", 2024-09-13T18:34:14.959 INFO:teuthology.orchestra.run.smithi114.stdout: "crash", 2024-09-13T18:34:14.959 INFO:teuthology.orchestra.run.smithi114.stdout: "devicehealth", 2024-09-13T18:34:14.959 INFO:teuthology.orchestra.run.smithi114.stdout: "orchestrator", 2024-09-13T18:34:14.959 INFO:teuthology.orchestra.run.smithi114.stdout: "pg_autoscaler", 2024-09-13T18:34:14.959 INFO:teuthology.orchestra.run.smithi114.stdout: "progress", 2024-09-13T18:34:14.959 INFO:teuthology.orchestra.run.smithi114.stdout: "rbd_support", 2024-09-13T18:34:14.959 INFO:teuthology.orchestra.run.smithi114.stdout: "status", 2024-09-13T18:34:14.959 INFO:teuthology.orchestra.run.smithi114.stdout: "telemetry", 2024-09-13T18:34:14.959 INFO:teuthology.orchestra.run.smithi114.stdout: "volumes" 2024-09-13T18:34:14.959 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-13T18:34:14.959 INFO:teuthology.orchestra.run.smithi114.stdout: "reef": [ 2024-09-13T18:34:14.959 INFO:teuthology.orchestra.run.smithi114.stdout: "balancer", 2024-09-13T18:34:14.959 INFO:teuthology.orchestra.run.smithi114.stdout: "crash", 2024-09-13T18:34:14.960 INFO:teuthology.orchestra.run.smithi114.stdout: "devicehealth", 2024-09-13T18:34:14.960 INFO:teuthology.orchestra.run.smithi114.stdout: "orchestrator", 2024-09-13T18:34:14.960 INFO:teuthology.orchestra.run.smithi114.stdout: "pg_autoscaler", 2024-09-13T18:34:14.960 INFO:teuthology.orchestra.run.smithi114.stdout: "progress", 2024-09-13T18:34:14.960 INFO:teuthology.orchestra.run.smithi114.stdout: "rbd_support", 2024-09-13T18:34:14.960 INFO:teuthology.orchestra.run.smithi114.stdout: "status", 2024-09-13T18:34:14.960 INFO:teuthology.orchestra.run.smithi114.stdout: "telemetry", 2024-09-13T18:34:14.960 INFO:teuthology.orchestra.run.smithi114.stdout: "volumes" 2024-09-13T18:34:14.960 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-13T18:34:14.960 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.960 INFO:teuthology.orchestra.run.smithi114.stdout: "last_failure_osd_epoch": 94, 2024-09-13T18:34:14.960 INFO:teuthology.orchestra.run.smithi114.stdout: "active_clients": [ 2024-09-13T18:34:14.960 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.961 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "devicehealth", 2024-09-13T18:34:14.961 INFO:teuthology.orchestra.run.smithi114.stdout: "addrvec": [ 2024-09-13T18:34:14.961 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.961 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "v2", 2024-09-13T18:34:14.961 INFO:teuthology.orchestra.run.smithi114.stdout: "addr": "172.21.15.114:0", 2024-09-13T18:34:14.961 INFO:teuthology.orchestra.run.smithi114.stdout: "nonce": 3591469708 2024-09-13T18:34:14.961 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.961 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-13T18:34:14.961 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.961 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.961 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "libcephsqlite", 2024-09-13T18:34:14.961 INFO:teuthology.orchestra.run.smithi114.stdout: "addrvec": [ 2024-09-13T18:34:14.961 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.961 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "v2", 2024-09-13T18:34:14.962 INFO:teuthology.orchestra.run.smithi114.stdout: "addr": "172.21.15.114:0", 2024-09-13T18:34:14.962 INFO:teuthology.orchestra.run.smithi114.stdout: "nonce": 101903528 2024-09-13T18:34:14.962 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.962 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-13T18:34:14.962 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.962 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.962 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rbd_support", 2024-09-13T18:34:14.962 INFO:teuthology.orchestra.run.smithi114.stdout: "addrvec": [ 2024-09-13T18:34:14.962 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.962 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "v2", 2024-09-13T18:34:14.962 INFO:teuthology.orchestra.run.smithi114.stdout: "addr": "172.21.15.114:0", 2024-09-13T18:34:14.962 INFO:teuthology.orchestra.run.smithi114.stdout: "nonce": 3711651372 2024-09-13T18:34:14.962 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.962 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-13T18:34:14.963 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.963 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.963 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "snap_schedule", 2024-09-13T18:34:14.963 INFO:teuthology.orchestra.run.smithi114.stdout: "addrvec": [ 2024-09-13T18:34:14.963 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.963 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "v2", 2024-09-13T18:34:14.963 INFO:teuthology.orchestra.run.smithi114.stdout: "addr": "172.21.15.114:0", 2024-09-13T18:34:14.963 INFO:teuthology.orchestra.run.smithi114.stdout: "nonce": 3401005996 2024-09-13T18:34:14.963 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.963 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-13T18:34:14.963 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.963 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.963 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "snap_schedule", 2024-09-13T18:34:14.963 INFO:teuthology.orchestra.run.smithi114.stdout: "addrvec": [ 2024-09-13T18:34:14.964 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.964 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "v2", 2024-09-13T18:34:14.964 INFO:teuthology.orchestra.run.smithi114.stdout: "addr": "172.21.15.114:0", 2024-09-13T18:34:14.964 INFO:teuthology.orchestra.run.smithi114.stdout: "nonce": 3779544845 2024-09-13T18:34:14.964 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.964 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-13T18:34:14.964 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:14.964 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.964 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "volumes", 2024-09-13T18:34:14.964 INFO:teuthology.orchestra.run.smithi114.stdout: "addrvec": [ 2024-09-13T18:34:14.964 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-13T18:34:14.964 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "v2", 2024-09-13T18:34:14.964 INFO:teuthology.orchestra.run.smithi114.stdout: "addr": "172.21.15.114:0", 2024-09-13T18:34:14.964 INFO:teuthology.orchestra.run.smithi114.stdout: "nonce": 4195337509 2024-09-13T18:34:14.965 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.965 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-13T18:34:14.965 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:14.965 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-13T18:34:14.965 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:34:14.966 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:13 smithi134 ceph-mon[34283]: pgmap v366: 129 pgs: 11 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 93 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 29 KiB/s wr, 13 op/s 2024-09-13T18:34:14.966 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:13 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/2392849303' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T18:34:14.967 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:13 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/111222938' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T18:34:14.967 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:13 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/2333576398' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T18:34:14.967 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:13 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/1780221248' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-13T18:34:14.967 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:13 smithi204 ceph-mon[34777]: pgmap v366: 129 pgs: 11 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 93 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 29 KiB/s wr, 13 op/s 2024-09-13T18:34:14.967 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:13 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/2392849303' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T18:34:14.967 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:13 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/111222938' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T18:34:14.967 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:13 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/2333576398' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T18:34:14.967 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:13 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/1780221248' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-13T18:34:14.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.603+0000 7f7d2d5c7640 1 -- 172.21.15.114:0/1780221248 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7d140a1280 msgr2=0x7f7d140a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:14.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.603+0000 7f7d2d5c7640 1 --2- 172.21.15.114:0/1780221248 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7d140a1280 0x7f7d140a3740 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7d18000c00 tx=0x7f7d18005a20 comp rx=0 tx=0).stop 2024-09-13T18:34:14.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.603+0000 7f7d2d5c7640 1 -- 172.21.15.114:0/1780221248 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7d28101940 msgr2=0x7f7d281967b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:14.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.603+0000 7f7d2d5c7640 1 --2- 172.21.15.114:0/1780221248 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7d28101940 0x7f7d281967b0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f7d1000d950 tx=0x7f7d1000de10 comp rx=0 tx=0).stop 2024-09-13T18:34:14.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.603+0000 7f7d2d5c7640 1 -- 172.21.15.114:0/1780221248 shutdown_connections 2024-09-13T18:34:14.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.603+0000 7f7d2d5c7640 1 --2- 172.21.15.114:0/1780221248 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f7d140a1280 0x7f7d140a3740 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:14.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.603+0000 7f7d2d5c7640 1 --2- 172.21.15.114:0/1780221248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d2810a3f0 0x7f7d2819aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:14.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.603+0000 7f7d2d5c7640 1 --2- 172.21.15.114:0/1780221248 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7d28101940 0x7f7d281967b0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:14.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.603+0000 7f7d2d5c7640 1 --2- 172.21.15.114:0/1780221248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d28101020 0x7f7d28196270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:14.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.603+0000 7f7d2d5c7640 1 -- 172.21.15.114:0/1780221248 >> 172.21.15.114:0/1780221248 conn(0x7f7d280fa970 msgr2=0x7f7d280fc360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:14.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.603+0000 7f7d2d5c7640 1 -- 172.21.15.114:0/1780221248 shutdown_connections 2024-09-13T18:34:14.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:13.603+0000 7f7d2d5c7640 1 -- 172.21.15.114:0/1780221248 wait complete. 2024-09-13T18:34:14.975 DEBUG:tasks.check_counter:Getting stats from a 2024-09-13T18:34:14.975 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-13T18:34:14.976 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:13 smithi114 ceph-mon[30897]: pgmap v366: 129 pgs: 11 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 93 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 29 KiB/s wr, 13 op/s 2024-09-13T18:34:14.976 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:13 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2392849303' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T18:34:14.976 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:13 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/111222938' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T18:34:14.976 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:13 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2333576398' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T18:34:14.977 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:13 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/1780221248' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-13T18:34:15.190 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:34:15.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:15 smithi204 ceph-mon[34777]: pgmap v367: 129 pgs: 9 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 95 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-09-13T18:34:15.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:15 smithi114 ceph-mon[30897]: pgmap v367: 129 pgs: 9 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 95 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-09-13T18:34:16.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:15 smithi134 ceph-mon[34283]: pgmap v367: 129 pgs: 9 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 95 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-09-13T18:34:17.384 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:34:17.384 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-0": { 2024-09-13T18:34:17.384 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 1448, 2024-09-13T18:34:17.384 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 1602, 2024-09-13T18:34:17.384 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 2071420, 2024-09-13T18:34:17.384 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 1414728, 2024-09-13T18:34:17.385 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 10, 2024-09-13T18:34:17.385 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 4, 2024-09-13T18:34:17.385 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.210853127, 2024-09-13T18:34:17.385 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.068807805, 2024-09-13T18:34:17.385 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.128673735, 2024-09-13T18:34:17.385 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.028228187, 2024-09-13T18:34:17.385 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T18:34:17.385 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 1601, 2024-09-13T18:34:17.385 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.208791263, 2024-09-13T18:34:17.385 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000130413 2024-09-13T18:34:17.385 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.385 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-09-13T18:34:17.385 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 1528, 2024-09-13T18:34:17.386 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000331112, 2024-09-13T18:34:17.386 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000216 2024-09-13T18:34:17.386 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.386 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 1669376, 2024-09-13T18:34:17.386 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 477264 2024-09-13T18:34:17.386 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.386 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-1": { 2024-09-13T18:34:17.386 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 3197, 2024-09-13T18:34:17.386 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 4444, 2024-09-13T18:34:17.386 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 1045416, 2024-09-13T18:34:17.386 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 2586668, 2024-09-13T18:34:17.386 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 56, 2024-09-13T18:34:17.386 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 6, 2024-09-13T18:34:17.387 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.546273704, 2024-09-13T18:34:17.387 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.262251522, 2024-09-13T18:34:17.387 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.278340416, 2024-09-13T18:34:17.387 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.022298794, 2024-09-13T18:34:17.387 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T18:34:17.387 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 4442, 2024-09-13T18:34:17.387 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.617573891, 2024-09-13T18:34:17.387 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000139030 2024-09-13T18:34:17.387 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.387 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-09-13T18:34:17.387 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 5656, 2024-09-13T18:34:17.387 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.001746474, 2024-09-13T18:34:17.387 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000308 2024-09-13T18:34:17.388 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.388 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 4048, 2024-09-13T18:34:17.388 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 128752 2024-09-13T18:34:17.388 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.388 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-2": { 2024-09-13T18:34:17.388 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 3300, 2024-09-13T18:34:17.388 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 4657, 2024-09-13T18:34:17.388 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 847137, 2024-09-13T18:34:17.388 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 1401256, 2024-09-13T18:34:17.388 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 12, 2024-09-13T18:34:17.388 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 7, 2024-09-13T18:34:17.388 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.334429406, 2024-09-13T18:34:17.388 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.118881384, 2024-09-13T18:34:17.388 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.203680991, 2024-09-13T18:34:17.389 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.027342615, 2024-09-13T18:34:17.389 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T18:34:17.389 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 4655, 2024-09-13T18:34:17.389 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.884925330, 2024-09-13T18:34:17.389 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000190102 2024-09-13T18:34:17.389 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.389 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-09-13T18:34:17.389 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 6559, 2024-09-13T18:34:17.389 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.001543109, 2024-09-13T18:34:17.389 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000235 2024-09-13T18:34:17.389 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.389 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 928, 2024-09-13T18:34:17.389 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 976 2024-09-13T18:34:17.389 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.390 INFO:teuthology.orchestra.run.smithi114.stdout: "cct": { 2024-09-13T18:34:17.390 INFO:teuthology.orchestra.run.smithi114.stdout: "total_workers": 1, 2024-09-13T18:34:17.390 INFO:teuthology.orchestra.run.smithi114.stdout: "unhealthy_workers": 0 2024-09-13T18:34:17.390 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.390 INFO:teuthology.orchestra.run.smithi114.stdout: "finisher-MDSRank": { 2024-09-13T18:34:17.390 INFO:teuthology.orchestra.run.smithi114.stdout: "queue_len": 0, 2024-09-13T18:34:17.390 INFO:teuthology.orchestra.run.smithi114.stdout: "complete_latency": { 2024-09-13T18:34:17.390 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 700, 2024-09-13T18:34:17.390 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.260803565, 2024-09-13T18:34:17.390 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000372576 2024-09-13T18:34:17.390 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:17.390 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.391 INFO:teuthology.orchestra.run.smithi114.stdout: "finisher-PurgeQueue": { 2024-09-13T18:34:17.391 INFO:teuthology.orchestra.run.smithi114.stdout: "queue_len": 0, 2024-09-13T18:34:17.391 INFO:teuthology.orchestra.run.smithi114.stdout: "complete_latency": { 2024-09-13T18:34:17.391 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 990, 2024-09-13T18:34:17.391 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.040733160, 2024-09-13T18:34:17.391 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000041144 2024-09-13T18:34:17.391 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:17.391 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.391 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": { 2024-09-13T18:34:17.391 INFO:teuthology.orchestra.run.smithi114.stdout: "request": 949, 2024-09-13T18:34:17.391 INFO:teuthology.orchestra.run.smithi114.stdout: "reply": 949, 2024-09-13T18:34:17.391 INFO:teuthology.orchestra.run.smithi114.stdout: "reply_latency": { 2024-09-13T18:34:17.391 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 949, 2024-09-13T18:34:17.392 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 6.964556001, 2024-09-13T18:34:17.392 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.007338836 2024-09-13T18:34:17.392 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.392 INFO:teuthology.orchestra.run.smithi114.stdout: "slow_reply": 0, 2024-09-13T18:34:17.392 INFO:teuthology.orchestra.run.smithi114.stdout: "forward": 0, 2024-09-13T18:34:17.392 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_fetch_complete": 0, 2024-09-13T18:34:17.392 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_fetch_keys": 0, 2024-09-13T18:34:17.392 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_commit": 11, 2024-09-13T18:34:17.392 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_split": 0, 2024-09-13T18:34:17.392 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_merge": 0, 2024-09-13T18:34:17.392 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes": 13, 2024-09-13T18:34:17.392 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_top": 1, 2024-09-13T18:34:17.393 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_bottom": 2, 2024-09-13T18:34:17.393 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_pin_tail": 10, 2024-09-13T18:34:17.393 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_pinned": 11, 2024-09-13T18:34:17.393 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_expired": 56, 2024-09-13T18:34:17.393 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_with_caps": 0, 2024-09-13T18:34:17.393 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": 0, 2024-09-13T18:34:17.393 INFO:teuthology.orchestra.run.smithi114.stdout: "subtrees": 4, 2024-09-13T18:34:17.393 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse": 1906, 2024-09-13T18:34:17.393 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_hit": 1900, 2024-09-13T18:34:17.393 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_forward": 0, 2024-09-13T18:34:17.393 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_discover": 4, 2024-09-13T18:34:17.393 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_dir_fetch": 0, 2024-09-13T18:34:17.393 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_remote_ino": 0, 2024-09-13T18:34:17.394 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_lock": 0, 2024-09-13T18:34:17.394 INFO:teuthology.orchestra.run.smithi114.stdout: "load_cent": 2000, 2024-09-13T18:34:17.394 INFO:teuthology.orchestra.run.smithi114.stdout: "q": 2, 2024-09-13T18:34:17.394 INFO:teuthology.orchestra.run.smithi114.stdout: "exported": 2, 2024-09-13T18:34:17.394 INFO:teuthology.orchestra.run.smithi114.stdout: "exported_inodes": 43, 2024-09-13T18:34:17.394 INFO:teuthology.orchestra.run.smithi114.stdout: "imported": 2, 2024-09-13T18:34:17.394 INFO:teuthology.orchestra.run.smithi114.stdout: "imported_inodes": 40, 2024-09-13T18:34:17.394 INFO:teuthology.orchestra.run.smithi114.stdout: "openino_dir_fetch": 0, 2024-09-13T18:34:17.394 INFO:teuthology.orchestra.run.smithi114.stdout: "openino_backtrace_fetch": 4, 2024-09-13T18:34:17.395 INFO:teuthology.orchestra.run.smithi114.stdout: "openino_peer_discover": 0, 2024-09-13T18:34:17.395 INFO:teuthology.orchestra.run.smithi114.stdout: "root_rfiles": 22, 2024-09-13T18:34:17.395 INFO:teuthology.orchestra.run.smithi114.stdout: "root_rbytes": 10738143812, 2024-09-13T18:34:17.395 INFO:teuthology.orchestra.run.smithi114.stdout: "root_rsnaps": 6, 2024-09-13T18:34:17.395 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_backtrace_fetch": 4, 2024-09-13T18:34:17.395 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_set_tag": 0, 2024-09-13T18:34:17.395 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_backtrace_repaired": 0, 2024-09-13T18:34:17.395 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_inotable_repaired": 0, 2024-09-13T18:34:17.395 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_dir_inodes": 0, 2024-09-13T18:34:17.395 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_dir_base_inodes": 0, 2024-09-13T18:34:17.395 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_dirfrag_rstats": 0, 2024-09-13T18:34:17.395 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_file_inodes": 4, 2024-09-13T18:34:17.395 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_inode_file_caps": 2, 2024-09-13T18:34:17.396 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_revoke": 1368, 2024-09-13T18:34:17.396 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_grant": 1013, 2024-09-13T18:34:17.396 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_trunc": 0, 2024-09-13T18:34:17.396 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-13T18:34:17.396 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-13T18:34:17.396 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_caps": 1421, 2024-09-13T18:34:17.396 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_caps_dirty": 310, 2024-09-13T18:34:17.396 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_cap_release": 2, 2024-09-13T18:34:17.396 INFO:teuthology.orchestra.run.smithi114.stdout: "process_request_cap_release": 6 2024-09-13T18:34:17.396 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.396 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_cache": { 2024-09-13T18:34:17.396 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_update": 0, 2024-09-13T18:34:17.396 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_update_receipt": 12, 2024-09-13T18:34:17.396 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_try_discover": 8, 2024-09-13T18:34:17.397 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_send_discover": 9, 2024-09-13T18:34:17.397 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_handle_discover": 0, 2024-09-13T18:34:17.397 INFO:teuthology.orchestra.run.smithi114.stdout: "num_strays": 0, 2024-09-13T18:34:17.397 INFO:teuthology.orchestra.run.smithi114.stdout: "num_strays_delayed": 0, 2024-09-13T18:34:17.397 INFO:teuthology.orchestra.run.smithi114.stdout: "num_strays_enqueuing": 0, 2024-09-13T18:34:17.397 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_created": 3, 2024-09-13T18:34:17.402 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_enqueued": 3, 2024-09-13T18:34:17.402 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_reintegrated": 0, 2024-09-13T18:34:17.402 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_migrated": 0, 2024-09-13T18:34:17.402 INFO:teuthology.orchestra.run.smithi114.stdout: "num_recovering_processing": 0, 2024-09-13T18:34:17.402 INFO:teuthology.orchestra.run.smithi114.stdout: "num_recovering_enqueued": 0, 2024-09-13T18:34:17.402 INFO:teuthology.orchestra.run.smithi114.stdout: "num_recovering_prioritized": 0, 2024-09-13T18:34:17.402 INFO:teuthology.orchestra.run.smithi114.stdout: "recovery_started": 0, 2024-09-13T18:34:17.402 INFO:teuthology.orchestra.run.smithi114.stdout: "recovery_completed": 0, 2024-09-13T18:34:17.402 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_enqueue_scrub": 0, 2024-09-13T18:34:17.402 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_exportdir": 3, 2024-09-13T18:34:17.402 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_flush": 0, 2024-09-13T18:34:17.403 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_fragmentdir": 0, 2024-09-13T18:34:17.403 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_fragstats": 0, 2024-09-13T18:34:17.403 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_inodestats": 0 2024-09-13T18:34:17.403 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.403 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_log": { 2024-09-13T18:34:17.403 INFO:teuthology.orchestra.run.smithi114.stdout: "evadd": 343, 2024-09-13T18:34:17.403 INFO:teuthology.orchestra.run.smithi114.stdout: "evex": 0, 2024-09-13T18:34:17.403 INFO:teuthology.orchestra.run.smithi114.stdout: "evtrm": 0, 2024-09-13T18:34:17.403 INFO:teuthology.orchestra.run.smithi114.stdout: "ev": 343, 2024-09-13T18:34:17.403 INFO:teuthology.orchestra.run.smithi114.stdout: "evexg": 0, 2024-09-13T18:34:17.403 INFO:teuthology.orchestra.run.smithi114.stdout: "evexd": 0, 2024-09-13T18:34:17.403 INFO:teuthology.orchestra.run.smithi114.stdout: "segadd": 1, 2024-09-13T18:34:17.403 INFO:teuthology.orchestra.run.smithi114.stdout: "segex": 0, 2024-09-13T18:34:17.403 INFO:teuthology.orchestra.run.smithi114.stdout: "segtrm": 0, 2024-09-13T18:34:17.404 INFO:teuthology.orchestra.run.smithi114.stdout: "seg": 1, 2024-09-13T18:34:17.404 INFO:teuthology.orchestra.run.smithi114.stdout: "segexg": 0, 2024-09-13T18:34:17.404 INFO:teuthology.orchestra.run.smithi114.stdout: "segexd": 0, 2024-09-13T18:34:17.404 INFO:teuthology.orchestra.run.smithi114.stdout: "expos": 4194304, 2024-09-13T18:34:17.404 INFO:teuthology.orchestra.run.smithi114.stdout: "wrpos": 4623826, 2024-09-13T18:34:17.404 INFO:teuthology.orchestra.run.smithi114.stdout: "rdpos": 0, 2024-09-13T18:34:17.404 INFO:teuthology.orchestra.run.smithi114.stdout: "jlat": { 2024-09-13T18:34:17.404 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 324, 2024-09-13T18:34:17.404 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 5.212055123, 2024-09-13T18:34:17.404 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.016086589 2024-09-13T18:34:17.404 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.404 INFO:teuthology.orchestra.run.smithi114.stdout: "replayed": 0 2024-09-13T18:34:17.405 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.405 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_mem": { 2024-09-13T18:34:17.405 INFO:teuthology.orchestra.run.smithi114.stdout: "ino": 18, 2024-09-13T18:34:17.405 INFO:teuthology.orchestra.run.smithi114.stdout: "ino+": 69, 2024-09-13T18:34:17.405 INFO:teuthology.orchestra.run.smithi114.stdout: "ino-": 51, 2024-09-13T18:34:17.405 INFO:teuthology.orchestra.run.smithi114.stdout: "dir": 17, 2024-09-13T18:34:17.405 INFO:teuthology.orchestra.run.smithi114.stdout: "dir+": 34, 2024-09-13T18:34:17.405 INFO:teuthology.orchestra.run.smithi114.stdout: "dir-": 17, 2024-09-13T18:34:17.405 INFO:teuthology.orchestra.run.smithi114.stdout: "dn": 13, 2024-09-13T18:34:17.405 INFO:teuthology.orchestra.run.smithi114.stdout: "dn+": 72, 2024-09-13T18:34:17.405 INFO:teuthology.orchestra.run.smithi114.stdout: "dn-": 59, 2024-09-13T18:34:17.405 INFO:teuthology.orchestra.run.smithi114.stdout: "cap": 0, 2024-09-13T18:34:17.405 INFO:teuthology.orchestra.run.smithi114.stdout: "cap+": 7, 2024-09-13T18:34:17.405 INFO:teuthology.orchestra.run.smithi114.stdout: "cap-": 7, 2024-09-13T18:34:17.406 INFO:teuthology.orchestra.run.smithi114.stdout: "rss": 51736, 2024-09-13T18:34:17.406 INFO:teuthology.orchestra.run.smithi114.stdout: "heap": 198932 2024-09-13T18:34:17.406 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.406 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_server": { 2024-09-13T18:34:17.406 INFO:teuthology.orchestra.run.smithi114.stdout: "dispatch_client_request": 1775, 2024-09-13T18:34:17.406 INFO:teuthology.orchestra.run.smithi114.stdout: "dispatch_server_request": 120, 2024-09-13T18:34:17.406 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_request": 949, 2024-09-13T18:34:17.406 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_session": 20, 2024-09-13T18:34:17.406 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_peer_request": 252, 2024-09-13T18:34:17.406 INFO:teuthology.orchestra.run.smithi114.stdout: "req_create_latency": { 2024-09-13T18:34:17.406 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:17.406 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:17.406 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:17.406 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.407 INFO:teuthology.orchestra.run.smithi114.stdout: "req_getattr_latency": { 2024-09-13T18:34:17.407 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 442, 2024-09-13T18:34:17.407 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 1.048267852, 2024-09-13T18:34:17.407 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.002371646 2024-09-13T18:34:17.407 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.407 INFO:teuthology.orchestra.run.smithi114.stdout: "req_getfilelock_latency": { 2024-09-13T18:34:17.407 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:17.407 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:17.407 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:17.407 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.407 INFO:teuthology.orchestra.run.smithi114.stdout: "req_link_latency": { 2024-09-13T18:34:17.407 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:17.407 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:17.408 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:17.408 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.408 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookup_latency": { 2024-09-13T18:34:17.408 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 503, 2024-09-13T18:34:17.408 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 5.903172716, 2024-09-13T18:34:17.408 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.011735929 2024-09-13T18:34:17.408 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.408 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookuphash_latency": { 2024-09-13T18:34:17.408 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:17.408 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:17.408 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:17.408 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.408 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupino_latency": { 2024-09-13T18:34:17.408 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:17.409 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:17.409 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:17.409 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.409 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupname_latency": { 2024-09-13T18:34:17.409 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:17.409 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:17.409 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:17.409 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.409 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupparent_latency": { 2024-09-13T18:34:17.409 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:17.409 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:17.409 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:17.409 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.410 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupsnap_latency": { 2024-09-13T18:34:17.410 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:17.410 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:17.410 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:17.410 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.410 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lssnap_latency": { 2024-09-13T18:34:17.410 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:17.410 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:17.410 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:17.410 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.410 INFO:teuthology.orchestra.run.smithi114.stdout: "req_mkdir_latency": { 2024-09-13T18:34:17.410 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:17.411 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:17.411 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:17.411 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.411 INFO:teuthology.orchestra.run.smithi114.stdout: "req_mknod_latency": { 2024-09-13T18:34:17.411 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:17.411 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:17.411 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:17.411 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.412 INFO:teuthology.orchestra.run.smithi114.stdout: "req_mksnap_latency": { 2024-09-13T18:34:17.412 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:17.412 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:17.412 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:17.412 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.412 INFO:teuthology.orchestra.run.smithi114.stdout: "req_open_latency": { 2024-09-13T18:34:17.412 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:17.412 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:17.412 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:17.412 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.412 INFO:teuthology.orchestra.run.smithi114.stdout: "req_readdir_latency": { 2024-09-13T18:34:17.412 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 1, 2024-09-13T18:34:17.412 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000398079, 2024-09-13T18:34:17.413 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000398079 2024-09-13T18:34:17.413 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.413 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rename_latency": { 2024-09-13T18:34:17.413 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:17.413 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:17.413 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:17.413 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.413 INFO:teuthology.orchestra.run.smithi114.stdout: "req_renamesnap_latency": { 2024-09-13T18:34:17.413 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:17.413 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:17.413 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:17.413 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.413 INFO:teuthology.orchestra.run.smithi114.stdout: "req_snapdiff_latency": { 2024-09-13T18:34:17.413 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:17.414 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:17.414 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:17.414 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.414 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rmdir_latency": { 2024-09-13T18:34:17.414 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:17.414 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:17.414 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:17.414 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.414 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rmsnap_latency": { 2024-09-13T18:34:17.414 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:17.414 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:17.414 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:17.414 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.414 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rmxattr_latency": { 2024-09-13T18:34:17.415 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:17.415 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:17.415 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:17.415 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.415 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setattr_latency": { 2024-09-13T18:34:17.415 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:17.415 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:17.415 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:17.415 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.415 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setdirlayout_latency": { 2024-09-13T18:34:17.415 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:17.415 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:17.415 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:17.415 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.416 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setfilelock_latency": { 2024-09-13T18:34:17.416 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:17.416 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:17.416 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:17.416 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.416 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setlayout_latency": { 2024-09-13T18:34:17.416 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:17.416 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:17.416 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:17.416 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.416 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setxattr_latency": { 2024-09-13T18:34:17.416 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:17.416 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:17.416 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:17.417 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.417 INFO:teuthology.orchestra.run.smithi114.stdout: "req_symlink_latency": { 2024-09-13T18:34:17.417 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:17.417 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:17.417 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:17.417 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.417 INFO:teuthology.orchestra.run.smithi114.stdout: "req_unlink_latency": { 2024-09-13T18:34:17.417 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 3, 2024-09-13T18:34:17.417 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.012717354, 2024-09-13T18:34:17.417 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.004239118 2024-09-13T18:34:17.417 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.417 INFO:teuthology.orchestra.run.smithi114.stdout: "cap_revoke_eviction": 0, 2024-09-13T18:34:17.417 INFO:teuthology.orchestra.run.smithi114.stdout: "cap_acquisition_throttle": 0, 2024-09-13T18:34:17.417 INFO:teuthology.orchestra.run.smithi114.stdout: "req_getvxattr_latency": { 2024-09-13T18:34:17.418 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:17.418 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:17.418 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:17.418 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:17.418 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.418 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_sessions": { 2024-09-13T18:34:17.418 INFO:teuthology.orchestra.run.smithi114.stdout: "session_count": 1, 2024-09-13T18:34:17.418 INFO:teuthology.orchestra.run.smithi114.stdout: "session_add": 3, 2024-09-13T18:34:17.418 INFO:teuthology.orchestra.run.smithi114.stdout: "session_remove": 2, 2024-09-13T18:34:17.418 INFO:teuthology.orchestra.run.smithi114.stdout: "sessions_open": 1, 2024-09-13T18:34:17.418 INFO:teuthology.orchestra.run.smithi114.stdout: "sessions_stale": 0, 2024-09-13T18:34:17.418 INFO:teuthology.orchestra.run.smithi114.stdout: "total_load": 803, 2024-09-13T18:34:17.418 INFO:teuthology.orchestra.run.smithi114.stdout: "average_load": 803, 2024-09-13T18:34:17.418 INFO:teuthology.orchestra.run.smithi114.stdout: "avg_session_uptime": 706, 2024-09-13T18:34:17.419 INFO:teuthology.orchestra.run.smithi114.stdout: "mdthresh_evicted": 0 2024-09-13T18:34:17.419 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.419 INFO:teuthology.orchestra.run.smithi114.stdout: "mempool": { 2024-09-13T18:34:17.419 INFO:teuthology.orchestra.run.smithi114.stdout: "bloom_filter_bytes": 0, 2024-09-13T18:34:17.419 INFO:teuthology.orchestra.run.smithi114.stdout: "bloom_filter_items": 0, 2024-09-13T18:34:17.419 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_alloc_bytes": 0, 2024-09-13T18:34:17.419 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_alloc_items": 0, 2024-09-13T18:34:17.419 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_data_bytes": 0, 2024-09-13T18:34:17.419 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_data_items": 0, 2024-09-13T18:34:17.419 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-13T18:34:17.419 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_onode_items": 0, 2024-09-13T18:34:17.419 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-13T18:34:17.419 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_meta_items": 0, 2024-09-13T18:34:17.420 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_other_bytes": 0, 2024-09-13T18:34:17.420 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_other_items": 0, 2024-09-13T18:34:17.420 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-13T18:34:17.420 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_buffer_items": 0, 2024-09-13T18:34:17.420 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_extent_bytes": 0, 2024-09-13T18:34:17.420 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_extent_items": 0, 2024-09-13T18:34:17.420 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_blob_bytes": 0, 2024-09-13T18:34:17.420 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_blob_items": 0, 2024-09-13T18:34:17.420 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-13T18:34:17.420 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_shared_blob_items": 0, 2024-09-13T18:34:17.420 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-13T18:34:17.420 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_inline_bl_items": 0, 2024-09-13T18:34:17.420 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_fsck_bytes": 0, 2024-09-13T18:34:17.420 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_fsck_items": 0, 2024-09-13T18:34:17.421 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_txc_bytes": 0, 2024-09-13T18:34:17.421 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_txc_items": 0, 2024-09-13T18:34:17.421 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-13T18:34:17.421 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_deferred_items": 0, 2024-09-13T18:34:17.421 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_bytes": 0, 2024-09-13T18:34:17.421 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_items": 0, 2024-09-13T18:34:17.421 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_bytes": 0, 2024-09-13T18:34:17.421 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_items": 0, 2024-09-13T18:34:17.421 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_reader_bytes": 0, 2024-09-13T18:34:17.421 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_reader_items": 0, 2024-09-13T18:34:17.421 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_writer_bytes": 0, 2024-09-13T18:34:17.421 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_writer_items": 0, 2024-09-13T18:34:17.421 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_anon_bytes": 305873, 2024-09-13T18:34:17.421 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_anon_items": 121, 2024-09-13T18:34:17.422 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_meta_bytes": 0, 2024-09-13T18:34:17.422 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_meta_items": 0, 2024-09-13T18:34:17.422 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_bytes": 0, 2024-09-13T18:34:17.422 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_items": 0, 2024-09-13T18:34:17.422 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_mapbl_bytes": 0, 2024-09-13T18:34:17.422 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_mapbl_items": 0, 2024-09-13T18:34:17.422 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_pglog_bytes": 0, 2024-09-13T18:34:17.422 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_pglog_items": 0, 2024-09-13T18:34:17.422 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_bytes": 9944, 2024-09-13T18:34:17.422 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_items": 202, 2024-09-13T18:34:17.422 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_mapping_bytes": 0, 2024-09-13T18:34:17.422 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_mapping_items": 0, 2024-09-13T18:34:17.422 INFO:teuthology.orchestra.run.smithi114.stdout: "pgmap_bytes": 0, 2024-09-13T18:34:17.422 INFO:teuthology.orchestra.run.smithi114.stdout: "pgmap_items": 0, 2024-09-13T18:34:17.423 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_co_bytes": 91560, 2024-09-13T18:34:17.423 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_co_items": 401, 2024-09-13T18:34:17.423 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_1_bytes": 0, 2024-09-13T18:34:17.423 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_1_items": 0, 2024-09-13T18:34:17.423 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_2_bytes": 0, 2024-09-13T18:34:17.423 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_2_items": 0 2024-09-13T18:34:17.423 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.423 INFO:teuthology.orchestra.run.smithi114.stdout: "objecter": { 2024-09-13T18:34:17.423 INFO:teuthology.orchestra.run.smithi114.stdout: "op_active": 0, 2024-09-13T18:34:17.423 INFO:teuthology.orchestra.run.smithi114.stdout: "op_laggy": 0, 2024-09-13T18:34:17.423 INFO:teuthology.orchestra.run.smithi114.stdout: "op_send": 1412, 2024-09-13T18:34:17.423 INFO:teuthology.orchestra.run.smithi114.stdout: "op_send_bytes": 442062, 2024-09-13T18:34:17.423 INFO:teuthology.orchestra.run.smithi114.stdout: "op_resend": 33, 2024-09-13T18:34:17.424 INFO:teuthology.orchestra.run.smithi114.stdout: "op_reply": 1412, 2024-09-13T18:34:17.424 INFO:teuthology.orchestra.run.smithi114.stdout: "op_latency": { 2024-09-13T18:34:17.424 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 1412, 2024-09-13T18:34:17.424 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 45.729994621, 2024-09-13T18:34:17.424 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.032386681 2024-09-13T18:34:17.424 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.424 INFO:teuthology.orchestra.run.smithi114.stdout: "op_inflight": 0, 2024-09-13T18:34:17.424 INFO:teuthology.orchestra.run.smithi114.stdout: "oplen_avg": { 2024-09-13T18:34:17.424 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 1412, 2024-09-13T18:34:17.424 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 1437 2024-09-13T18:34:17.424 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.424 INFO:teuthology.orchestra.run.smithi114.stdout: "op": 1412, 2024-09-13T18:34:17.424 INFO:teuthology.orchestra.run.smithi114.stdout: "op_r": 6, 2024-09-13T18:34:17.424 INFO:teuthology.orchestra.run.smithi114.stdout: "op_w": 1406, 2024-09-13T18:34:17.425 INFO:teuthology.orchestra.run.smithi114.stdout: "op_rmw": 0, 2024-09-13T18:34:17.425 INFO:teuthology.orchestra.run.smithi114.stdout: "op_pg": 0, 2024-09-13T18:34:17.425 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_stat": 0, 2024-09-13T18:34:17.425 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_create": 10, 2024-09-13T18:34:17.425 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_read": 2, 2024-09-13T18:34:17.425 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_write": 326, 2024-09-13T18:34:17.425 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_writefull": 12, 2024-09-13T18:34:17.425 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_writesame": 0, 2024-09-13T18:34:17.425 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_append": 0, 2024-09-13T18:34:17.425 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_zero": 0, 2024-09-13T18:34:17.425 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_truncate": 0, 2024-09-13T18:34:17.425 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_delete": 1042, 2024-09-13T18:34:17.425 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_mapext": 0, 2024-09-13T18:34:17.425 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_sparse_read": 0, 2024-09-13T18:34:17.426 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_clonerange": 0, 2024-09-13T18:34:17.426 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_getxattr": 4, 2024-09-13T18:34:17.426 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_setxattr": 20, 2024-09-13T18:34:17.426 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_cmpxattr": 0, 2024-09-13T18:34:17.426 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_rmxattr": 0, 2024-09-13T18:34:17.426 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_resetxattrs": 0, 2024-09-13T18:34:17.426 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_call": 0, 2024-09-13T18:34:17.426 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_watch": 0, 2024-09-13T18:34:17.426 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_notify": 0, 2024-09-13T18:34:17.426 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_src_cmpxattr": 0, 2024-09-13T18:34:17.426 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_pgls": 0, 2024-09-13T18:34:17.426 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_pgls_filter": 0, 2024-09-13T18:34:17.426 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_other": 0, 2024-09-13T18:34:17.426 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_active": 0, 2024-09-13T18:34:17.426 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_send": 0, 2024-09-13T18:34:17.427 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_resend": 0, 2024-09-13T18:34:17.427 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_ping": 0, 2024-09-13T18:34:17.427 INFO:teuthology.orchestra.run.smithi114.stdout: "poolop_active": 0, 2024-09-13T18:34:17.427 INFO:teuthology.orchestra.run.smithi114.stdout: "poolop_send": 0, 2024-09-13T18:34:17.427 INFO:teuthology.orchestra.run.smithi114.stdout: "poolop_resend": 0, 2024-09-13T18:34:17.427 INFO:teuthology.orchestra.run.smithi114.stdout: "poolstat_active": 0, 2024-09-13T18:34:17.427 INFO:teuthology.orchestra.run.smithi114.stdout: "poolstat_send": 0, 2024-09-13T18:34:17.427 INFO:teuthology.orchestra.run.smithi114.stdout: "poolstat_resend": 0, 2024-09-13T18:34:17.427 INFO:teuthology.orchestra.run.smithi114.stdout: "statfs_active": 0, 2024-09-13T18:34:17.427 INFO:teuthology.orchestra.run.smithi114.stdout: "statfs_send": 0, 2024-09-13T18:34:17.427 INFO:teuthology.orchestra.run.smithi114.stdout: "statfs_resend": 0, 2024-09-13T18:34:17.427 INFO:teuthology.orchestra.run.smithi114.stdout: "command_active": 0, 2024-09-13T18:34:17.427 INFO:teuthology.orchestra.run.smithi114.stdout: "command_send": 0, 2024-09-13T18:34:17.427 INFO:teuthology.orchestra.run.smithi114.stdout: "command_resend": 0, 2024-09-13T18:34:17.428 INFO:teuthology.orchestra.run.smithi114.stdout: "map_epoch": 105, 2024-09-13T18:34:17.428 INFO:teuthology.orchestra.run.smithi114.stdout: "map_full": 0, 2024-09-13T18:34:17.428 INFO:teuthology.orchestra.run.smithi114.stdout: "map_inc": 14, 2024-09-13T18:34:17.428 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_sessions": 12, 2024-09-13T18:34:17.428 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_session_open": 12, 2024-09-13T18:34:17.428 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_session_close": 0, 2024-09-13T18:34:17.428 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_laggy": 0, 2024-09-13T18:34:17.428 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_wr": 19, 2024-09-13T18:34:17.428 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_rd": 0, 2024-09-13T18:34:17.428 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_del": 2 2024-09-13T18:34:17.428 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.428 INFO:teuthology.orchestra.run.smithi114.stdout: "oft": { 2024-09-13T18:34:17.428 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_objs": 1, 2024-09-13T18:34:17.429 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_kv_pairs": 0, 2024-09-13T18:34:17.429 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_updates": 0, 2024-09-13T18:34:17.429 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_removes": 0 2024-09-13T18:34:17.429 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.429 INFO:teuthology.orchestra.run.smithi114.stdout: "osd-slow-ops": { 2024-09-13T18:34:17.429 INFO:teuthology.orchestra.run.smithi114.stdout: "slow_ops_count": 0 2024-09-13T18:34:17.429 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.429 INFO:teuthology.orchestra.run.smithi114.stdout: "purge_queue": { 2024-09-13T18:34:17.429 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing_ops": 0, 2024-09-13T18:34:17.429 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing_ops_high_water": 516, 2024-09-13T18:34:17.429 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing": 0, 2024-09-13T18:34:17.429 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing_high_water": 2, 2024-09-13T18:34:17.429 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executed_ops": 1030, 2024-09-13T18:34:17.429 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executed": 3, 2024-09-13T18:34:17.430 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_item_in_journal": 0 2024-09-13T18:34:17.430 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.430 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-13T18:34:17.430 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-09-13T18:34:17.430 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 104857600, 2024-09-13T18:34:17.430 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-09-13T18:34:17.430 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 7941, 2024-09-13T18:34:17.430 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 3230466, 2024-09-13T18:34:17.430 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-09-13T18:34:17.430 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 7941, 2024-09-13T18:34:17.430 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 0, 2024-09-13T18:34:17.430 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 0, 2024-09-13T18:34:17.430 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 7941, 2024-09-13T18:34:17.431 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 3230466, 2024-09-13T18:34:17.431 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-09-13T18:34:17.431 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:17.431 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:17.431 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:17.431 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:17.431 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.431 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-objecter_bytes": { 2024-09-13T18:34:17.431 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-09-13T18:34:17.431 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 104857600, 2024-09-13T18:34:17.431 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-09-13T18:34:17.431 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 0, 2024-09-13T18:34:17.432 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 0, 2024-09-13T18:34:17.432 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-09-13T18:34:17.432 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 0, 2024-09-13T18:34:17.432 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 1412, 2024-09-13T18:34:17.432 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 442533, 2024-09-13T18:34:17.432 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 1412, 2024-09-13T18:34:17.432 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 442533, 2024-09-13T18:34:17.432 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-09-13T18:34:17.432 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:17.432 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:17.432 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:17.432 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:17.432 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.433 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-objecter_ops": { 2024-09-13T18:34:17.433 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-09-13T18:34:17.433 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 1024, 2024-09-13T18:34:17.433 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-09-13T18:34:17.433 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 0, 2024-09-13T18:34:17.433 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 0, 2024-09-13T18:34:17.433 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-09-13T18:34:17.433 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 0, 2024-09-13T18:34:17.433 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 1412, 2024-09-13T18:34:17.433 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 1412, 2024-09-13T18:34:17.433 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 1412, 2024-09-13T18:34:17.433 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 1412, 2024-09-13T18:34:17.433 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-09-13T18:34:17.433 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:17.433 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:17.434 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:17.434 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:17.434 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.434 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-write_buf_throttle": { 2024-09-13T18:34:17.434 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-09-13T18:34:17.434 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 3758096384, 2024-09-13T18:34:17.434 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-09-13T18:34:17.434 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 3, 2024-09-13T18:34:17.434 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 471, 2024-09-13T18:34:17.434 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-09-13T18:34:17.434 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 3, 2024-09-13T18:34:17.434 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 0, 2024-09-13T18:34:17.434 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 0, 2024-09-13T18:34:17.434 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 2, 2024-09-13T18:34:17.435 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 471, 2024-09-13T18:34:17.435 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-09-13T18:34:17.435 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:17.435 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:17.435 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:17.435 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:17.435 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:17.435 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-write_buf_throttle-0x559c570b4500": { 2024-09-13T18:34:17.435 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-09-13T18:34:17.435 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 3758096384, 2024-09-13T18:34:17.435 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-09-13T18:34:17.435 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 343, 2024-09-13T18:34:17.435 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 429522, 2024-09-13T18:34:17.435 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-09-13T18:34:17.436 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 343, 2024-09-13T18:34:17.436 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 0, 2024-09-13T18:34:17.436 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 0, 2024-09-13T18:34:17.436 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 324, 2024-09-13T18:34:17.436 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 429522, 2024-09-13T18:34:17.436 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-09-13T18:34:17.436 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:17.436 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:17.436 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:17.436 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:17.436 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:17.436 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:34:17.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:17 smithi204 ceph-mon[34777]: pgmap v368: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2024-09-13T18:34:17.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:17 smithi114 ceph-mon[30897]: pgmap v368: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2024-09-13T18:34:18.119 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:17 smithi134 ceph-mon[34283]: pgmap v368: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2024-09-13T18:34:18.172 INFO:tasks.check_counter:Daemon mds.a mds.exported=2 2024-09-13T18:34:18.172 INFO:tasks.check_counter:Daemon mds.a mds.imported=2 2024-09-13T18:34:18.172 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-09-13T18:34:18.172 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=12 2024-09-13T18:34:18.172 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=6 2024-09-13T18:34:18.172 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-09-13T18:34:18.172 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-09-13T18:34:18.172 DEBUG:tasks.check_counter:Getting stats from d 2024-09-13T18:34:18.173 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-13T18:34:18.372 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:34:19.727 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:19 smithi114 ceph-mon[30897]: pgmap v369: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 261 B/s wr, 9 op/s 2024-09-13T18:34:19.727 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:19 smithi114 ceph-mon[30897]: osdmap e106: 12 total, 12 up, 12 in 2024-09-13T18:34:19.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:19 smithi134 ceph-mon[34283]: pgmap v369: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 261 B/s wr, 9 op/s 2024-09-13T18:34:19.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:19 smithi134 ceph-mon[34283]: osdmap e106: 12 total, 12 up, 12 in 2024-09-13T18:34:19.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:19 smithi204 ceph-mon[34777]: pgmap v369: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 261 B/s wr, 9 op/s 2024-09-13T18:34:19.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:19 smithi204 ceph-mon[34777]: osdmap e106: 12 total, 12 up, 12 in 2024-09-13T18:34:21.006 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:34:21.006 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-0": { 2024-09-13T18:34:21.006 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 0, 2024-09-13T18:34:21.006 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 2, 2024-09-13T18:34:21.006 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 0, 2024-09-13T18:34:21.007 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 0, 2024-09-13T18:34:21.007 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 4, 2024-09-13T18:34:21.007 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 0, 2024-09-13T18:34:21.007 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.001084086, 2024-09-13T18:34:21.007 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.000000000, 2024-09-13T18:34:21.007 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.000217974, 2024-09-13T18:34:21.007 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-13T18:34:21.007 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T18:34:21.007 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:21.007 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:21.007 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:21.007 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:21.007 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-09-13T18:34:21.007 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:21.007 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:21.008 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:21.008 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:21.008 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-13T18:34:21.008 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 0 2024-09-13T18:34:21.008 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:21.008 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-1": { 2024-09-13T18:34:21.008 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 248, 2024-09-13T18:34:21.008 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 237, 2024-09-13T18:34:21.008 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 1566272, 2024-09-13T18:34:21.008 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 135312, 2024-09-13T18:34:21.008 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 2, 2024-09-13T18:34:21.008 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 1, 2024-09-13T18:34:21.009 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.052286053, 2024-09-13T18:34:21.009 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.015911761, 2024-09-13T18:34:21.009 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.048502575, 2024-09-13T18:34:21.009 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-13T18:34:21.009 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T18:34:21.009 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 236, 2024-09-13T18:34:21.009 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.014884055, 2024-09-13T18:34:21.009 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000063068 2024-09-13T18:34:21.009 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:21.009 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-09-13T18:34:21.009 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:21.009 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:21.009 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:21.009 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:21.010 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 1566272, 2024-09-13T18:34:21.010 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 135312 2024-09-13T18:34:21.010 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:21.010 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-2": { 2024-09-13T18:34:21.010 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 6, 2024-09-13T18:34:21.010 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 184, 2024-09-13T18:34:21.010 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 3280, 2024-09-13T18:34:21.010 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 184816, 2024-09-13T18:34:21.010 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 5, 2024-09-13T18:34:21.010 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 3, 2024-09-13T18:34:21.010 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.016686310, 2024-09-13T18:34:21.010 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.010243536, 2024-09-13T18:34:21.010 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.003813519, 2024-09-13T18:34:21.010 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-13T18:34:21.011 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T18:34:21.011 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 182, 2024-09-13T18:34:21.011 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.020481334, 2024-09-13T18:34:21.011 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000112534 2024-09-13T18:34:21.011 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:21.011 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-09-13T18:34:21.011 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:21.011 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:21.011 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:21.011 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:21.011 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 3280, 2024-09-13T18:34:21.011 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 184816 2024-09-13T18:34:21.011 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:21.011 INFO:teuthology.orchestra.run.smithi114.stdout: "cct": { 2024-09-13T18:34:21.012 INFO:teuthology.orchestra.run.smithi114.stdout: "total_workers": 0, 2024-09-13T18:34:21.012 INFO:teuthology.orchestra.run.smithi114.stdout: "unhealthy_workers": 0 2024-09-13T18:34:21.012 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:21.012 INFO:teuthology.orchestra.run.smithi114.stdout: "mempool": { 2024-09-13T18:34:21.012 INFO:teuthology.orchestra.run.smithi114.stdout: "bloom_filter_bytes": 0, 2024-09-13T18:34:21.012 INFO:teuthology.orchestra.run.smithi114.stdout: "bloom_filter_items": 0, 2024-09-13T18:34:21.012 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_alloc_bytes": 0, 2024-09-13T18:34:21.012 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_alloc_items": 0, 2024-09-13T18:34:21.012 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_data_bytes": 0, 2024-09-13T18:34:21.012 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_data_items": 0, 2024-09-13T18:34:21.012 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-13T18:34:21.012 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_onode_items": 0, 2024-09-13T18:34:21.012 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-13T18:34:21.012 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_meta_items": 0, 2024-09-13T18:34:21.013 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_other_bytes": 0, 2024-09-13T18:34:21.013 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_other_items": 0, 2024-09-13T18:34:21.013 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-13T18:34:21.013 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_buffer_items": 0, 2024-09-13T18:34:21.013 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_extent_bytes": 0, 2024-09-13T18:34:21.013 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_extent_items": 0, 2024-09-13T18:34:21.013 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_blob_bytes": 0, 2024-09-13T18:34:21.013 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_blob_items": 0, 2024-09-13T18:34:21.013 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-13T18:34:21.013 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_shared_blob_items": 0, 2024-09-13T18:34:21.013 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-13T18:34:21.013 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_inline_bl_items": 0, 2024-09-13T18:34:21.013 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_fsck_bytes": 0, 2024-09-13T18:34:21.013 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_fsck_items": 0, 2024-09-13T18:34:21.014 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_txc_bytes": 0, 2024-09-13T18:34:21.014 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_txc_items": 0, 2024-09-13T18:34:21.014 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-13T18:34:21.014 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_deferred_items": 0, 2024-09-13T18:34:21.014 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_bytes": 0, 2024-09-13T18:34:21.014 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_items": 0, 2024-09-13T18:34:21.014 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_bytes": 0, 2024-09-13T18:34:21.014 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_items": 0, 2024-09-13T18:34:21.014 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_reader_bytes": 0, 2024-09-13T18:34:21.014 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_reader_items": 0, 2024-09-13T18:34:21.014 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_writer_bytes": 0, 2024-09-13T18:34:21.014 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_writer_items": 0, 2024-09-13T18:34:21.014 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_anon_bytes": 148994, 2024-09-13T18:34:21.014 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_anon_items": 31, 2024-09-13T18:34:21.015 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_meta_bytes": 0, 2024-09-13T18:34:21.015 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_meta_items": 0, 2024-09-13T18:34:21.015 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_bytes": 0, 2024-09-13T18:34:21.015 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_items": 0, 2024-09-13T18:34:21.015 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_mapbl_bytes": 0, 2024-09-13T18:34:21.015 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_mapbl_items": 0, 2024-09-13T18:34:21.015 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_pglog_bytes": 0, 2024-09-13T18:34:21.015 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_pglog_items": 0, 2024-09-13T18:34:21.015 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_bytes": 0, 2024-09-13T18:34:21.015 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_items": 0, 2024-09-13T18:34:21.015 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_mapping_bytes": 0, 2024-09-13T18:34:21.015 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_mapping_items": 0, 2024-09-13T18:34:21.015 INFO:teuthology.orchestra.run.smithi114.stdout: "pgmap_bytes": 0, 2024-09-13T18:34:21.015 INFO:teuthology.orchestra.run.smithi114.stdout: "pgmap_items": 0, 2024-09-13T18:34:21.016 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_co_bytes": 928, 2024-09-13T18:34:21.016 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_co_items": 2, 2024-09-13T18:34:21.016 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_1_bytes": 0, 2024-09-13T18:34:21.016 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_1_items": 0, 2024-09-13T18:34:21.016 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_2_bytes": 0, 2024-09-13T18:34:21.016 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_2_items": 0 2024-09-13T18:34:21.016 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:21.016 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-13T18:34:21.016 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-09-13T18:34:21.016 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 104857600, 2024-09-13T18:34:21.016 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-09-13T18:34:21.016 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 250, 2024-09-13T18:34:21.016 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 1531235, 2024-09-13T18:34:21.016 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-09-13T18:34:21.017 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 250, 2024-09-13T18:34:21.017 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 0, 2024-09-13T18:34:21.017 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 0, 2024-09-13T18:34:21.017 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 250, 2024-09-13T18:34:21.017 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 1531235, 2024-09-13T18:34:21.017 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-09-13T18:34:21.017 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:21.017 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:21.017 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:21.017 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:21.017 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:21.017 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:34:21.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:21 smithi134 ceph-mon[34283]: pgmap v371: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-13T18:34:21.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:21 smithi204 ceph-mon[34777]: pgmap v371: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-13T18:34:21.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:21 smithi114 ceph-mon[30897]: pgmap v371: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-13T18:34:23.249 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-13T18:34:23.249 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-13T18:34:23.249 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-09-13T18:34:23.249 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-09-13T18:34:23.249 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-09-13T18:34:23.249 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-09-13T18:34:23.249 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-09-13T18:34:23.250 DEBUG:tasks.check_counter:Getting stats from g 2024-09-13T18:34:23.250 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-13T18:34:23.456 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:34:23.729 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:23 smithi114 ceph-mon[30897]: pgmap v372: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-13T18:34:23.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:23 smithi134 ceph-mon[34283]: pgmap v372: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-13T18:34:23.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:23 smithi204 ceph-mon[34777]: pgmap v372: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-13T18:34:25.165 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:34:25.165 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-0": { 2024-09-13T18:34:25.165 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 1105, 2024-09-13T18:34:25.165 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 1084, 2024-09-13T18:34:25.165 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 1933024, 2024-09-13T18:34:25.165 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 395138, 2024-09-13T18:34:25.165 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 5, 2024-09-13T18:34:25.165 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 1, 2024-09-13T18:34:25.166 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.155309197, 2024-09-13T18:34:25.166 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.053040592, 2024-09-13T18:34:25.166 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.075939191, 2024-09-13T18:34:25.166 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.033000910, 2024-09-13T18:34:25.166 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T18:34:25.166 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 1083, 2024-09-13T18:34:25.166 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.104694134, 2024-09-13T18:34:25.166 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000096670 2024-09-13T18:34:25.166 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.166 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-09-13T18:34:25.166 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:25.167 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:25.167 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:25.167 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.167 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 1654144, 2024-09-13T18:34:25.167 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 137008 2024-09-13T18:34:25.167 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.167 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-1": { 2024-09-13T18:34:25.167 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 1585, 2024-09-13T18:34:25.167 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 1763, 2024-09-13T18:34:25.167 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 832604, 2024-09-13T18:34:25.167 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 958930, 2024-09-13T18:34:25.167 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 7, 2024-09-13T18:34:25.167 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 5, 2024-09-13T18:34:25.168 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.125635967, 2024-09-13T18:34:25.168 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.038941135, 2024-09-13T18:34:25.168 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.030728609, 2024-09-13T18:34:25.168 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.047715942, 2024-09-13T18:34:25.168 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T18:34:25.168 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 1760, 2024-09-13T18:34:25.168 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.139365857, 2024-09-13T18:34:25.168 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000079185 2024-09-13T18:34:25.168 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.168 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-09-13T18:34:25.168 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:25.168 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:25.168 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:25.169 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.169 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-13T18:34:25.169 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 466560 2024-09-13T18:34:25.169 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.169 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-2": { 2024-09-13T18:34:25.169 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 8, 2024-09-13T18:34:25.169 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 8, 2024-09-13T18:34:25.169 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 4589, 2024-09-13T18:34:25.169 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 1704, 2024-09-13T18:34:25.169 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 5, 2024-09-13T18:34:25.169 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 4, 2024-09-13T18:34:25.169 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.003050542, 2024-09-13T18:34:25.170 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.000090826, 2024-09-13T18:34:25.170 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.002842126, 2024-09-13T18:34:25.170 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.000053115, 2024-09-13T18:34:25.170 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T18:34:25.170 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 7, 2024-09-13T18:34:25.170 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.005739696, 2024-09-13T18:34:25.170 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000819956 2024-09-13T18:34:25.170 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.171 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-09-13T18:34:25.172 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:25.172 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:25.172 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:25.172 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.172 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 2960, 2024-09-13T18:34:25.172 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 448 2024-09-13T18:34:25.172 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.172 INFO:teuthology.orchestra.run.smithi114.stdout: "cct": { 2024-09-13T18:34:25.172 INFO:teuthology.orchestra.run.smithi114.stdout: "total_workers": 1, 2024-09-13T18:34:25.172 INFO:teuthology.orchestra.run.smithi114.stdout: "unhealthy_workers": 0 2024-09-13T18:34:25.172 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.172 INFO:teuthology.orchestra.run.smithi114.stdout: "finisher-MDSRank": { 2024-09-13T18:34:25.172 INFO:teuthology.orchestra.run.smithi114.stdout: "queue_len": 0, 2024-09-13T18:34:25.173 INFO:teuthology.orchestra.run.smithi114.stdout: "complete_latency": { 2024-09-13T18:34:25.173 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 3872, 2024-09-13T18:34:25.173 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.107553532, 2024-09-13T18:34:25.173 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000027777 2024-09-13T18:34:25.173 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:25.173 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.173 INFO:teuthology.orchestra.run.smithi114.stdout: "finisher-PurgeQueue": { 2024-09-13T18:34:25.173 INFO:teuthology.orchestra.run.smithi114.stdout: "queue_len": 0, 2024-09-13T18:34:25.173 INFO:teuthology.orchestra.run.smithi114.stdout: "complete_latency": { 2024-09-13T18:34:25.173 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:25.173 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:25.173 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:25.173 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:25.174 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.174 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": { 2024-09-13T18:34:25.174 INFO:teuthology.orchestra.run.smithi114.stdout: "request": 0, 2024-09-13T18:34:25.174 INFO:teuthology.orchestra.run.smithi114.stdout: "reply": 0, 2024-09-13T18:34:25.174 INFO:teuthology.orchestra.run.smithi114.stdout: "reply_latency": { 2024-09-13T18:34:25.174 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:25.174 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:25.174 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:25.174 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.174 INFO:teuthology.orchestra.run.smithi114.stdout: "slow_reply": 0, 2024-09-13T18:34:25.174 INFO:teuthology.orchestra.run.smithi114.stdout: "forward": 0, 2024-09-13T18:34:25.174 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_fetch_complete": 0, 2024-09-13T18:34:25.174 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_fetch_keys": 0, 2024-09-13T18:34:25.175 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_commit": 0, 2024-09-13T18:34:25.175 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_split": 0, 2024-09-13T18:34:25.175 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_merge": 0, 2024-09-13T18:34:25.175 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes": 4, 2024-09-13T18:34:25.175 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_top": 0, 2024-09-13T18:34:25.175 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_bottom": 4, 2024-09-13T18:34:25.175 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_pin_tail": 0, 2024-09-13T18:34:25.175 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_pinned": 4, 2024-09-13T18:34:25.175 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_expired": 0, 2024-09-13T18:34:25.175 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_with_caps": 0, 2024-09-13T18:34:25.175 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": 0, 2024-09-13T18:34:25.175 INFO:teuthology.orchestra.run.smithi114.stdout: "subtrees": 1, 2024-09-13T18:34:25.175 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse": 0, 2024-09-13T18:34:25.175 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_hit": 0, 2024-09-13T18:34:25.176 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_forward": 0, 2024-09-13T18:34:25.176 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_discover": 0, 2024-09-13T18:34:25.176 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_dir_fetch": 0, 2024-09-13T18:34:25.176 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_remote_ino": 0, 2024-09-13T18:34:25.176 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_lock": 0, 2024-09-13T18:34:25.176 INFO:teuthology.orchestra.run.smithi114.stdout: "load_cent": 0, 2024-09-13T18:34:25.176 INFO:teuthology.orchestra.run.smithi114.stdout: "q": 0, 2024-09-13T18:34:25.176 INFO:teuthology.orchestra.run.smithi114.stdout: "exported": 0, 2024-09-13T18:34:25.176 INFO:teuthology.orchestra.run.smithi114.stdout: "exported_inodes": 0, 2024-09-13T18:34:25.176 INFO:teuthology.orchestra.run.smithi114.stdout: "imported": 0, 2024-09-13T18:34:25.176 INFO:teuthology.orchestra.run.smithi114.stdout: "imported_inodes": 0, 2024-09-13T18:34:25.176 INFO:teuthology.orchestra.run.smithi114.stdout: "openino_dir_fetch": 0, 2024-09-13T18:34:25.177 INFO:teuthology.orchestra.run.smithi114.stdout: "openino_backtrace_fetch": 0, 2024-09-13T18:34:25.177 INFO:teuthology.orchestra.run.smithi114.stdout: "openino_peer_discover": 0, 2024-09-13T18:34:25.177 INFO:teuthology.orchestra.run.smithi114.stdout: "root_rfiles": 0, 2024-09-13T18:34:25.177 INFO:teuthology.orchestra.run.smithi114.stdout: "root_rbytes": 0, 2024-09-13T18:34:25.177 INFO:teuthology.orchestra.run.smithi114.stdout: "root_rsnaps": 0, 2024-09-13T18:34:25.177 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_backtrace_fetch": 0, 2024-09-13T18:34:25.177 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_set_tag": 0, 2024-09-13T18:34:25.177 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_backtrace_repaired": 0, 2024-09-13T18:34:25.177 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_inotable_repaired": 0, 2024-09-13T18:34:25.177 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_dir_inodes": 0, 2024-09-13T18:34:25.177 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_dir_base_inodes": 0, 2024-09-13T18:34:25.177 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_dirfrag_rstats": 0, 2024-09-13T18:34:25.177 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_file_inodes": 0, 2024-09-13T18:34:25.178 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_inode_file_caps": 0, 2024-09-13T18:34:25.178 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_revoke": 0, 2024-09-13T18:34:25.178 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_grant": 0, 2024-09-13T18:34:25.178 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_trunc": 0, 2024-09-13T18:34:25.178 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-13T18:34:25.178 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-13T18:34:25.178 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_caps": 0, 2024-09-13T18:34:25.178 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_caps_dirty": 0, 2024-09-13T18:34:25.178 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_cap_release": 0, 2024-09-13T18:34:25.178 INFO:teuthology.orchestra.run.smithi114.stdout: "process_request_cap_release": 0 2024-09-13T18:34:25.178 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.178 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_cache": { 2024-09-13T18:34:25.178 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_update": 0, 2024-09-13T18:34:25.178 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_update_receipt": 0, 2024-09-13T18:34:25.179 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_try_discover": 0, 2024-09-13T18:34:25.179 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_send_discover": 0, 2024-09-13T18:34:25.179 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_handle_discover": 0, 2024-09-13T18:34:25.179 INFO:teuthology.orchestra.run.smithi114.stdout: "num_strays": 0, 2024-09-13T18:34:25.179 INFO:teuthology.orchestra.run.smithi114.stdout: "num_strays_delayed": 0, 2024-09-13T18:34:25.179 INFO:teuthology.orchestra.run.smithi114.stdout: "num_strays_enqueuing": 0, 2024-09-13T18:34:25.179 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_created": 0, 2024-09-13T18:34:25.179 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_enqueued": 0, 2024-09-13T18:34:25.179 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_reintegrated": 0, 2024-09-13T18:34:25.179 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_migrated": 0, 2024-09-13T18:34:25.179 INFO:teuthology.orchestra.run.smithi114.stdout: "num_recovering_processing": 0, 2024-09-13T18:34:25.179 INFO:teuthology.orchestra.run.smithi114.stdout: "num_recovering_enqueued": 0, 2024-09-13T18:34:25.179 INFO:teuthology.orchestra.run.smithi114.stdout: "num_recovering_prioritized": 0, 2024-09-13T18:34:25.179 INFO:teuthology.orchestra.run.smithi114.stdout: "recovery_started": 0, 2024-09-13T18:34:25.180 INFO:teuthology.orchestra.run.smithi114.stdout: "recovery_completed": 0, 2024-09-13T18:34:25.180 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_enqueue_scrub": 0, 2024-09-13T18:34:25.180 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_exportdir": 0, 2024-09-13T18:34:25.180 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_flush": 0, 2024-09-13T18:34:25.180 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_fragmentdir": 0, 2024-09-13T18:34:25.180 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_fragstats": 0, 2024-09-13T18:34:25.180 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_inodestats": 0 2024-09-13T18:34:25.180 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.180 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_log": { 2024-09-13T18:34:25.180 INFO:teuthology.orchestra.run.smithi114.stdout: "evadd": 0, 2024-09-13T18:34:25.180 INFO:teuthology.orchestra.run.smithi114.stdout: "evex": 0, 2024-09-13T18:34:25.180 INFO:teuthology.orchestra.run.smithi114.stdout: "evtrm": 0, 2024-09-13T18:34:25.180 INFO:teuthology.orchestra.run.smithi114.stdout: "ev": 343, 2024-09-13T18:34:25.180 INFO:teuthology.orchestra.run.smithi114.stdout: "evexg": 0, 2024-09-13T18:34:25.181 INFO:teuthology.orchestra.run.smithi114.stdout: "evexd": 0, 2024-09-13T18:34:25.181 INFO:teuthology.orchestra.run.smithi114.stdout: "segadd": 0, 2024-09-13T18:34:25.181 INFO:teuthology.orchestra.run.smithi114.stdout: "segex": 0, 2024-09-13T18:34:25.181 INFO:teuthology.orchestra.run.smithi114.stdout: "segtrm": 0, 2024-09-13T18:34:25.181 INFO:teuthology.orchestra.run.smithi114.stdout: "seg": 1, 2024-09-13T18:34:25.181 INFO:teuthology.orchestra.run.smithi114.stdout: "segexg": 0, 2024-09-13T18:34:25.181 INFO:teuthology.orchestra.run.smithi114.stdout: "segexd": 0, 2024-09-13T18:34:25.181 INFO:teuthology.orchestra.run.smithi114.stdout: "expos": 4194304, 2024-09-13T18:34:25.181 INFO:teuthology.orchestra.run.smithi114.stdout: "wrpos": 4623826, 2024-09-13T18:34:25.181 INFO:teuthology.orchestra.run.smithi114.stdout: "rdpos": 4621221, 2024-09-13T18:34:25.181 INFO:teuthology.orchestra.run.smithi114.stdout: "jlat": { 2024-09-13T18:34:25.181 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:25.181 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:25.182 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:25.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.182 INFO:teuthology.orchestra.run.smithi114.stdout: "replayed": 343 2024-09-13T18:34:25.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.182 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_mem": { 2024-09-13T18:34:25.182 INFO:teuthology.orchestra.run.smithi114.stdout: "ino": 4, 2024-09-13T18:34:25.182 INFO:teuthology.orchestra.run.smithi114.stdout: "ino+": 56, 2024-09-13T18:34:25.182 INFO:teuthology.orchestra.run.smithi114.stdout: "ino-": 52, 2024-09-13T18:34:25.182 INFO:teuthology.orchestra.run.smithi114.stdout: "dir": 2, 2024-09-13T18:34:25.182 INFO:teuthology.orchestra.run.smithi114.stdout: "dir+": 13, 2024-09-13T18:34:25.182 INFO:teuthology.orchestra.run.smithi114.stdout: "dir-": 11, 2024-09-13T18:34:25.182 INFO:teuthology.orchestra.run.smithi114.stdout: "dn": 4, 2024-09-13T18:34:25.182 INFO:teuthology.orchestra.run.smithi114.stdout: "dn+": 56, 2024-09-13T18:34:25.182 INFO:teuthology.orchestra.run.smithi114.stdout: "dn-": 52, 2024-09-13T18:34:25.183 INFO:teuthology.orchestra.run.smithi114.stdout: "cap": 0, 2024-09-13T18:34:25.183 INFO:teuthology.orchestra.run.smithi114.stdout: "cap+": 0, 2024-09-13T18:34:25.183 INFO:teuthology.orchestra.run.smithi114.stdout: "cap-": 0, 2024-09-13T18:34:25.183 INFO:teuthology.orchestra.run.smithi114.stdout: "rss": 48500, 2024-09-13T18:34:25.183 INFO:teuthology.orchestra.run.smithi114.stdout: "heap": 207124 2024-09-13T18:34:25.183 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.183 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_server": { 2024-09-13T18:34:25.183 INFO:teuthology.orchestra.run.smithi114.stdout: "dispatch_client_request": 0, 2024-09-13T18:34:25.183 INFO:teuthology.orchestra.run.smithi114.stdout: "dispatch_server_request": 0, 2024-09-13T18:34:25.183 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_request": 0, 2024-09-13T18:34:25.183 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_session": 0, 2024-09-13T18:34:25.183 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_peer_request": 0, 2024-09-13T18:34:25.183 INFO:teuthology.orchestra.run.smithi114.stdout: "req_create_latency": { 2024-09-13T18:34:25.183 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:25.184 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:25.184 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:25.184 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.184 INFO:teuthology.orchestra.run.smithi114.stdout: "req_getattr_latency": { 2024-09-13T18:34:25.184 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:25.184 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:25.184 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:25.184 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.184 INFO:teuthology.orchestra.run.smithi114.stdout: "req_getfilelock_latency": { 2024-09-13T18:34:25.184 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:25.184 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:25.184 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:25.184 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.185 INFO:teuthology.orchestra.run.smithi114.stdout: "req_link_latency": { 2024-09-13T18:34:25.185 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:25.185 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:25.185 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:25.185 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.185 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookup_latency": { 2024-09-13T18:34:25.185 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:25.185 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:25.185 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:25.185 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.185 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookuphash_latency": { 2024-09-13T18:34:25.185 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:25.185 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:25.185 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:25.186 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.186 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupino_latency": { 2024-09-13T18:34:25.186 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:25.187 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:25.187 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:25.187 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.187 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupname_latency": { 2024-09-13T18:34:25.187 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:25.188 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:25.188 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:25.188 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.188 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupparent_latency": { 2024-09-13T18:34:25.188 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:25.188 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:25.188 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:25.188 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.188 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupsnap_latency": { 2024-09-13T18:34:25.188 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:25.188 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:25.188 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:25.189 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.189 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lssnap_latency": { 2024-09-13T18:34:25.189 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:25.189 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:25.189 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:25.189 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.189 INFO:teuthology.orchestra.run.smithi114.stdout: "req_mkdir_latency": { 2024-09-13T18:34:25.189 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:25.189 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:25.189 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:25.189 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.189 INFO:teuthology.orchestra.run.smithi114.stdout: "req_mknod_latency": { 2024-09-13T18:34:25.190 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:25.190 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:25.190 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:25.190 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.190 INFO:teuthology.orchestra.run.smithi114.stdout: "req_mksnap_latency": { 2024-09-13T18:34:25.191 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:25.191 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:25.191 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:25.191 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.191 INFO:teuthology.orchestra.run.smithi114.stdout: "req_open_latency": { 2024-09-13T18:34:25.191 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:25.191 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:25.191 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:25.191 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.191 INFO:teuthology.orchestra.run.smithi114.stdout: "req_readdir_latency": { 2024-09-13T18:34:25.191 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:25.191 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:25.191 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:25.191 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.192 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rename_latency": { 2024-09-13T18:34:25.192 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:25.192 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:25.192 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:25.192 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.192 INFO:teuthology.orchestra.run.smithi114.stdout: "req_renamesnap_latency": { 2024-09-13T18:34:25.192 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:25.192 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:25.192 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:25.192 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.192 INFO:teuthology.orchestra.run.smithi114.stdout: "req_snapdiff_latency": { 2024-09-13T18:34:25.192 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:25.192 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:25.192 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:25.193 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.193 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rmdir_latency": { 2024-09-13T18:34:25.193 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:25.193 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:25.193 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:25.193 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.193 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rmsnap_latency": { 2024-09-13T18:34:25.193 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:25.193 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:25.193 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:25.193 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.193 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rmxattr_latency": { 2024-09-13T18:34:25.193 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:25.193 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:25.194 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:25.194 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.194 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setattr_latency": { 2024-09-13T18:34:25.194 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:25.194 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:25.194 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:25.194 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.194 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setdirlayout_latency": { 2024-09-13T18:34:25.194 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:25.194 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:25.194 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:25.194 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.194 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setfilelock_latency": { 2024-09-13T18:34:25.195 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:25.195 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:25.195 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:25.195 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.195 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setlayout_latency": { 2024-09-13T18:34:25.195 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:25.195 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:25.195 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:25.195 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.195 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setxattr_latency": { 2024-09-13T18:34:25.195 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:25.195 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:25.195 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:25.196 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.196 INFO:teuthology.orchestra.run.smithi114.stdout: "req_symlink_latency": { 2024-09-13T18:34:25.196 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:25.196 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:25.196 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:25.196 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.196 INFO:teuthology.orchestra.run.smithi114.stdout: "req_unlink_latency": { 2024-09-13T18:34:25.196 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:25.196 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:25.196 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:25.196 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.196 INFO:teuthology.orchestra.run.smithi114.stdout: "cap_revoke_eviction": 0, 2024-09-13T18:34:25.196 INFO:teuthology.orchestra.run.smithi114.stdout: "cap_acquisition_throttle": 0, 2024-09-13T18:34:25.196 INFO:teuthology.orchestra.run.smithi114.stdout: "req_getvxattr_latency": { 2024-09-13T18:34:25.196 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:25.197 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:25.197 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:25.197 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:25.197 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.197 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_sessions": { 2024-09-13T18:34:25.197 INFO:teuthology.orchestra.run.smithi114.stdout: "session_count": 1, 2024-09-13T18:34:25.197 INFO:teuthology.orchestra.run.smithi114.stdout: "session_add": 3, 2024-09-13T18:34:25.197 INFO:teuthology.orchestra.run.smithi114.stdout: "session_remove": 2, 2024-09-13T18:34:25.197 INFO:teuthology.orchestra.run.smithi114.stdout: "sessions_open": 3, 2024-09-13T18:34:25.197 INFO:teuthology.orchestra.run.smithi114.stdout: "sessions_stale": 0, 2024-09-13T18:34:25.197 INFO:teuthology.orchestra.run.smithi114.stdout: "total_load": 0, 2024-09-13T18:34:25.197 INFO:teuthology.orchestra.run.smithi114.stdout: "average_load": 0, 2024-09-13T18:34:25.197 INFO:teuthology.orchestra.run.smithi114.stdout: "avg_session_uptime": 3886, 2024-09-13T18:34:25.198 INFO:teuthology.orchestra.run.smithi114.stdout: "mdthresh_evicted": 0 2024-09-13T18:34:25.198 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.198 INFO:teuthology.orchestra.run.smithi114.stdout: "mempool": { 2024-09-13T18:34:25.198 INFO:teuthology.orchestra.run.smithi114.stdout: "bloom_filter_bytes": 0, 2024-09-13T18:34:25.198 INFO:teuthology.orchestra.run.smithi114.stdout: "bloom_filter_items": 0, 2024-09-13T18:34:25.198 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_alloc_bytes": 0, 2024-09-13T18:34:25.198 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_alloc_items": 0, 2024-09-13T18:34:25.198 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_data_bytes": 0, 2024-09-13T18:34:25.198 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_data_items": 0, 2024-09-13T18:34:25.198 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-13T18:34:25.198 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_onode_items": 0, 2024-09-13T18:34:25.198 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-13T18:34:25.198 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_meta_items": 0, 2024-09-13T18:34:25.198 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_other_bytes": 0, 2024-09-13T18:34:25.199 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_other_items": 0, 2024-09-13T18:34:25.199 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-13T18:34:25.199 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_buffer_items": 0, 2024-09-13T18:34:25.199 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_extent_bytes": 0, 2024-09-13T18:34:25.199 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_extent_items": 0, 2024-09-13T18:34:25.199 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_blob_bytes": 0, 2024-09-13T18:34:25.199 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_blob_items": 0, 2024-09-13T18:34:25.199 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-13T18:34:25.199 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_shared_blob_items": 0, 2024-09-13T18:34:25.199 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-13T18:34:25.199 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_inline_bl_items": 0, 2024-09-13T18:34:25.199 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_fsck_bytes": 0, 2024-09-13T18:34:25.199 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_fsck_items": 0, 2024-09-13T18:34:25.199 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_txc_bytes": 0, 2024-09-13T18:34:25.199 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_txc_items": 0, 2024-09-13T18:34:25.200 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-13T18:34:25.200 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_deferred_items": 0, 2024-09-13T18:34:25.200 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_bytes": 0, 2024-09-13T18:34:25.200 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_items": 0, 2024-09-13T18:34:25.200 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_bytes": 0, 2024-09-13T18:34:25.200 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_items": 0, 2024-09-13T18:34:25.200 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_reader_bytes": 0, 2024-09-13T18:34:25.200 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_reader_items": 0, 2024-09-13T18:34:25.200 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_writer_bytes": 0, 2024-09-13T18:34:25.200 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_writer_items": 0, 2024-09-13T18:34:25.200 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_anon_bytes": 197513, 2024-09-13T18:34:25.200 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_anon_items": 56, 2024-09-13T18:34:25.200 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_meta_bytes": 0, 2024-09-13T18:34:25.201 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_meta_items": 0, 2024-09-13T18:34:25.201 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_bytes": 0, 2024-09-13T18:34:25.201 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_items": 0, 2024-09-13T18:34:25.201 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_mapbl_bytes": 0, 2024-09-13T18:34:25.201 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_mapbl_items": 0, 2024-09-13T18:34:25.201 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_pglog_bytes": 0, 2024-09-13T18:34:25.201 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_pglog_items": 0, 2024-09-13T18:34:25.201 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_bytes": 9704, 2024-09-13T18:34:25.201 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_items": 196, 2024-09-13T18:34:25.201 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_mapping_bytes": 0, 2024-09-13T18:34:25.201 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_mapping_items": 0, 2024-09-13T18:34:25.201 INFO:teuthology.orchestra.run.smithi114.stdout: "pgmap_bytes": 0, 2024-09-13T18:34:25.201 INFO:teuthology.orchestra.run.smithi114.stdout: "pgmap_items": 0, 2024-09-13T18:34:25.201 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_co_bytes": 16352, 2024-09-13T18:34:25.201 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_co_items": 48, 2024-09-13T18:34:25.202 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_1_bytes": 0, 2024-09-13T18:34:25.202 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_1_items": 0, 2024-09-13T18:34:25.202 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_2_bytes": 0, 2024-09-13T18:34:25.202 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_2_items": 0 2024-09-13T18:34:25.202 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.202 INFO:teuthology.orchestra.run.smithi114.stdout: "objecter": { 2024-09-13T18:34:25.202 INFO:teuthology.orchestra.run.smithi114.stdout: "op_active": 0, 2024-09-13T18:34:25.202 INFO:teuthology.orchestra.run.smithi114.stdout: "op_laggy": 0, 2024-09-13T18:34:25.202 INFO:teuthology.orchestra.run.smithi114.stdout: "op_send": 2417, 2024-09-13T18:34:25.202 INFO:teuthology.orchestra.run.smithi114.stdout: "op_send_bytes": 21, 2024-09-13T18:34:25.202 INFO:teuthology.orchestra.run.smithi114.stdout: "op_resend": 3, 2024-09-13T18:34:25.202 INFO:teuthology.orchestra.run.smithi114.stdout: "op_reply": 2417, 2024-09-13T18:34:25.202 INFO:teuthology.orchestra.run.smithi114.stdout: "op_latency": { 2024-09-13T18:34:25.203 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 2417, 2024-09-13T18:34:25.203 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 6.632852786, 2024-09-13T18:34:25.203 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.002744250 2024-09-13T18:34:25.203 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.203 INFO:teuthology.orchestra.run.smithi114.stdout: "op_inflight": 0, 2024-09-13T18:34:25.203 INFO:teuthology.orchestra.run.smithi114.stdout: "oplen_avg": { 2024-09-13T18:34:25.203 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 2417, 2024-09-13T18:34:25.203 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 2418 2024-09-13T18:34:25.203 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.203 INFO:teuthology.orchestra.run.smithi114.stdout: "op": 2417, 2024-09-13T18:34:25.203 INFO:teuthology.orchestra.run.smithi114.stdout: "op_r": 2417, 2024-09-13T18:34:25.203 INFO:teuthology.orchestra.run.smithi114.stdout: "op_w": 0, 2024-09-13T18:34:25.203 INFO:teuthology.orchestra.run.smithi114.stdout: "op_rmw": 0, 2024-09-13T18:34:25.204 INFO:teuthology.orchestra.run.smithi114.stdout: "op_pg": 0, 2024-09-13T18:34:25.204 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_stat": 1542, 2024-09-13T18:34:25.204 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_create": 0, 2024-09-13T18:34:25.204 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_read": 873, 2024-09-13T18:34:25.204 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_write": 0, 2024-09-13T18:34:25.204 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_writefull": 0, 2024-09-13T18:34:25.204 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_writesame": 0, 2024-09-13T18:34:25.204 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_append": 0, 2024-09-13T18:34:25.204 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_zero": 0, 2024-09-13T18:34:25.204 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_truncate": 0, 2024-09-13T18:34:25.204 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_delete": 0, 2024-09-13T18:34:25.204 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_mapext": 0, 2024-09-13T18:34:25.204 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_sparse_read": 0, 2024-09-13T18:34:25.204 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_clonerange": 0, 2024-09-13T18:34:25.205 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_getxattr": 1, 2024-09-13T18:34:25.205 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_setxattr": 0, 2024-09-13T18:34:25.205 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_cmpxattr": 0, 2024-09-13T18:34:25.205 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_rmxattr": 0, 2024-09-13T18:34:25.205 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_resetxattrs": 0, 2024-09-13T18:34:25.205 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_call": 0, 2024-09-13T18:34:25.205 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_watch": 0, 2024-09-13T18:34:25.205 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_notify": 0, 2024-09-13T18:34:25.205 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_src_cmpxattr": 0, 2024-09-13T18:34:25.205 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_pgls": 0, 2024-09-13T18:34:25.205 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_pgls_filter": 0, 2024-09-13T18:34:25.205 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_other": 0, 2024-09-13T18:34:25.205 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_active": 0, 2024-09-13T18:34:25.205 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_send": 0, 2024-09-13T18:34:25.206 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_resend": 0, 2024-09-13T18:34:25.206 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_ping": 0, 2024-09-13T18:34:25.206 INFO:teuthology.orchestra.run.smithi114.stdout: "poolop_active": 0, 2024-09-13T18:34:25.206 INFO:teuthology.orchestra.run.smithi114.stdout: "poolop_send": 0, 2024-09-13T18:34:25.206 INFO:teuthology.orchestra.run.smithi114.stdout: "poolop_resend": 0, 2024-09-13T18:34:25.206 INFO:teuthology.orchestra.run.smithi114.stdout: "poolstat_active": 0, 2024-09-13T18:34:25.206 INFO:teuthology.orchestra.run.smithi114.stdout: "poolstat_send": 0, 2024-09-13T18:34:25.206 INFO:teuthology.orchestra.run.smithi114.stdout: "poolstat_resend": 0, 2024-09-13T18:34:25.206 INFO:teuthology.orchestra.run.smithi114.stdout: "statfs_active": 0, 2024-09-13T18:34:25.206 INFO:teuthology.orchestra.run.smithi114.stdout: "statfs_send": 0, 2024-09-13T18:34:25.206 INFO:teuthology.orchestra.run.smithi114.stdout: "statfs_resend": 0, 2024-09-13T18:34:25.206 INFO:teuthology.orchestra.run.smithi114.stdout: "command_active": 0, 2024-09-13T18:34:25.206 INFO:teuthology.orchestra.run.smithi114.stdout: "command_send": 0, 2024-09-13T18:34:25.206 INFO:teuthology.orchestra.run.smithi114.stdout: "command_resend": 0, 2024-09-13T18:34:25.207 INFO:teuthology.orchestra.run.smithi114.stdout: "map_epoch": 106, 2024-09-13T18:34:25.207 INFO:teuthology.orchestra.run.smithi114.stdout: "map_full": 0, 2024-09-13T18:34:25.207 INFO:teuthology.orchestra.run.smithi114.stdout: "map_inc": 15, 2024-09-13T18:34:25.207 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_sessions": 6, 2024-09-13T18:34:25.207 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_session_open": 6, 2024-09-13T18:34:25.207 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_session_close": 0, 2024-09-13T18:34:25.207 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_laggy": 0, 2024-09-13T18:34:25.207 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_wr": 0, 2024-09-13T18:34:25.207 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_rd": 2, 2024-09-13T18:34:25.207 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_del": 0 2024-09-13T18:34:25.207 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.207 INFO:teuthology.orchestra.run.smithi114.stdout: "oft": { 2024-09-13T18:34:25.207 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_objs": 0, 2024-09-13T18:34:25.207 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_kv_pairs": 0, 2024-09-13T18:34:25.208 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_updates": 0, 2024-09-13T18:34:25.208 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_removes": 0 2024-09-13T18:34:25.208 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.208 INFO:teuthology.orchestra.run.smithi114.stdout: "osd-slow-ops": { 2024-09-13T18:34:25.208 INFO:teuthology.orchestra.run.smithi114.stdout: "slow_ops_count": 0 2024-09-13T18:34:25.208 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.208 INFO:teuthology.orchestra.run.smithi114.stdout: "purge_queue": { 2024-09-13T18:34:25.208 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing_ops": 0, 2024-09-13T18:34:25.208 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing_ops_high_water": 0, 2024-09-13T18:34:25.208 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing": 0, 2024-09-13T18:34:25.208 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing_high_water": 0, 2024-09-13T18:34:25.208 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executed_ops": 0, 2024-09-13T18:34:25.208 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executed": 0, 2024-09-13T18:34:25.208 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_item_in_journal": 0 2024-09-13T18:34:25.209 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.209 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-13T18:34:25.209 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-09-13T18:34:25.209 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 104857600, 2024-09-13T18:34:25.209 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-09-13T18:34:25.209 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 2694, 2024-09-13T18:34:25.209 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 2510680, 2024-09-13T18:34:25.209 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-09-13T18:34:25.209 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 2694, 2024-09-13T18:34:25.209 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 0, 2024-09-13T18:34:25.209 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 0, 2024-09-13T18:34:25.209 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 2694, 2024-09-13T18:34:25.209 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 2510680, 2024-09-13T18:34:25.209 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-09-13T18:34:25.210 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:25.210 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:25.210 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:25.210 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:25.210 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.210 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-objecter_bytes": { 2024-09-13T18:34:25.210 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-09-13T18:34:25.210 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 104857600, 2024-09-13T18:34:25.210 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-09-13T18:34:25.210 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 0, 2024-09-13T18:34:25.210 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 0, 2024-09-13T18:34:25.210 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-09-13T18:34:25.210 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 0, 2024-09-13T18:34:25.210 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 2417, 2024-09-13T18:34:25.210 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 429527, 2024-09-13T18:34:25.211 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 2417, 2024-09-13T18:34:25.211 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 429527, 2024-09-13T18:34:25.211 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-09-13T18:34:25.211 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:25.211 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:25.211 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:25.211 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:25.211 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.211 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-objecter_ops": { 2024-09-13T18:34:25.211 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-09-13T18:34:25.211 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 1024, 2024-09-13T18:34:25.211 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-09-13T18:34:25.211 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 0, 2024-09-13T18:34:25.211 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 0, 2024-09-13T18:34:25.212 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-09-13T18:34:25.212 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 0, 2024-09-13T18:34:25.212 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 2417, 2024-09-13T18:34:25.212 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 2417, 2024-09-13T18:34:25.212 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 2417, 2024-09-13T18:34:25.212 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 2417, 2024-09-13T18:34:25.212 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-09-13T18:34:25.212 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:25.212 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:25.212 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:25.212 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:25.212 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.212 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-write_buf_throttle": { 2024-09-13T18:34:25.212 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-09-13T18:34:25.213 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 3758096384, 2024-09-13T18:34:25.213 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-09-13T18:34:25.213 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 0, 2024-09-13T18:34:25.213 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 0, 2024-09-13T18:34:25.213 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-09-13T18:34:25.213 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 0, 2024-09-13T18:34:25.213 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 0, 2024-09-13T18:34:25.213 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 0, 2024-09-13T18:34:25.213 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 0, 2024-09-13T18:34:25.213 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 0, 2024-09-13T18:34:25.213 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-09-13T18:34:25.213 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:25.213 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:25.213 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:25.214 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:25.214 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:25.214 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-write_buf_throttle-0x55701e47c820": { 2024-09-13T18:34:25.214 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-09-13T18:34:25.214 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 3758096384, 2024-09-13T18:34:25.214 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-09-13T18:34:25.214 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 0, 2024-09-13T18:34:25.214 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 0, 2024-09-13T18:34:25.214 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-09-13T18:34:25.214 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 0, 2024-09-13T18:34:25.214 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 0, 2024-09-13T18:34:25.214 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 0, 2024-09-13T18:34:25.214 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 0, 2024-09-13T18:34:25.214 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 0, 2024-09-13T18:34:25.214 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-09-13T18:34:25.215 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:25.215 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:25.215 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:25.215 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:25.215 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:25.215 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:34:25.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:25 smithi134 ceph-mon[34283]: pgmap v373: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-13T18:34:25.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:25 smithi204 ceph-mon[34777]: pgmap v373: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-13T18:34:25.964 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:25 smithi114 ceph-mon[30897]: pgmap v373: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-13T18:34:25.989 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-09-13T18:34:25.989 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-09-13T18:34:25.989 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-09-13T18:34:25.989 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-09-13T18:34:25.989 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-09-13T18:34:25.989 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-09-13T18:34:25.989 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-09-13T18:34:25.989 DEBUG:tasks.check_counter:Getting stats from j 2024-09-13T18:34:25.990 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-13T18:34:26.192 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.a/config 2024-09-13T18:34:27.557 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:27 smithi114 ceph-mon[30897]: pgmap v374: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-13T18:34:27.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:27 smithi134 ceph-mon[34283]: pgmap v374: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-13T18:34:27.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:27 smithi204 ceph-mon[34777]: pgmap v374: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-13T18:34:27.949 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-13T18:34:27.949 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-0": { 2024-09-13T18:34:27.949 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 4, 2024-09-13T18:34:27.949 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 4, 2024-09-13T18:34:27.949 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 2960, 2024-09-13T18:34:27.949 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 448, 2024-09-13T18:34:27.949 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 4, 2024-09-13T18:34:27.949 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 0, 2024-09-13T18:34:27.949 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.003088990, 2024-09-13T18:34:27.950 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.000040850, 2024-09-13T18:34:27.950 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.002228300, 2024-09-13T18:34:27.950 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-13T18:34:27.950 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T18:34:27.950 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 3, 2024-09-13T18:34:27.950 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.003910479, 2024-09-13T18:34:27.950 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.001303493 2024-09-13T18:34:27.950 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:27.950 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-09-13T18:34:27.950 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:27.950 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:27.950 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:27.950 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:27.951 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 2960, 2024-09-13T18:34:27.951 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 448 2024-09-13T18:34:27.951 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:27.951 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-1": { 2024-09-13T18:34:27.951 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 2, 2024-09-13T18:34:27.951 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 184, 2024-09-13T18:34:27.951 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 320, 2024-09-13T18:34:27.951 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 184720, 2024-09-13T18:34:27.951 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 5, 2024-09-13T18:34:27.951 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 3, 2024-09-13T18:34:27.951 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.014395800, 2024-09-13T18:34:27.951 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.009755909, 2024-09-13T18:34:27.951 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.001833342, 2024-09-13T18:34:27.951 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-13T18:34:27.951 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T18:34:27.952 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 181, 2024-09-13T18:34:27.952 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.019239159, 2024-09-13T18:34:27.952 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000106293 2024-09-13T18:34:27.952 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:27.952 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-09-13T18:34:27.952 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:27.952 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:27.952 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:27.952 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:27.952 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-13T18:34:27.952 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 184720 2024-09-13T18:34:27.952 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:27.952 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-2": { 2024-09-13T18:34:27.953 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 250, 2024-09-13T18:34:27.953 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 239, 2024-09-13T18:34:27.953 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 1566816, 2024-09-13T18:34:27.953 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 136496, 2024-09-13T18:34:27.953 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 2, 2024-09-13T18:34:27.953 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 1, 2024-09-13T18:34:27.953 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.046511873, 2024-09-13T18:34:27.953 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.016065478, 2024-09-13T18:34:27.953 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.043777973, 2024-09-13T18:34:27.953 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-13T18:34:27.953 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T18:34:27.953 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 238, 2024-09-13T18:34:27.953 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.013425504, 2024-09-13T18:34:27.953 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000056409 2024-09-13T18:34:27.954 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:27.954 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-09-13T18:34:27.954 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:27.954 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:27.954 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:27.954 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:27.954 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 1566816, 2024-09-13T18:34:27.954 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 136496 2024-09-13T18:34:27.954 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:27.954 INFO:teuthology.orchestra.run.smithi114.stdout: "cct": { 2024-09-13T18:34:27.954 INFO:teuthology.orchestra.run.smithi114.stdout: "total_workers": 0, 2024-09-13T18:34:27.954 INFO:teuthology.orchestra.run.smithi114.stdout: "unhealthy_workers": 0 2024-09-13T18:34:27.954 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:27.954 INFO:teuthology.orchestra.run.smithi114.stdout: "mempool": { 2024-09-13T18:34:27.955 INFO:teuthology.orchestra.run.smithi114.stdout: "bloom_filter_bytes": 0, 2024-09-13T18:34:27.955 INFO:teuthology.orchestra.run.smithi114.stdout: "bloom_filter_items": 0, 2024-09-13T18:34:27.955 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_alloc_bytes": 0, 2024-09-13T18:34:27.955 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_alloc_items": 0, 2024-09-13T18:34:27.955 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_data_bytes": 0, 2024-09-13T18:34:27.955 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_data_items": 0, 2024-09-13T18:34:27.955 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-13T18:34:27.955 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_onode_items": 0, 2024-09-13T18:34:27.955 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-13T18:34:27.955 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_meta_items": 0, 2024-09-13T18:34:27.955 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_other_bytes": 0, 2024-09-13T18:34:27.955 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_other_items": 0, 2024-09-13T18:34:27.955 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-13T18:34:27.955 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_buffer_items": 0, 2024-09-13T18:34:27.956 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_extent_bytes": 0, 2024-09-13T18:34:27.956 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_extent_items": 0, 2024-09-13T18:34:27.956 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_blob_bytes": 0, 2024-09-13T18:34:27.956 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_blob_items": 0, 2024-09-13T18:34:27.956 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-13T18:34:27.956 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_shared_blob_items": 0, 2024-09-13T18:34:27.956 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-13T18:34:27.956 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_inline_bl_items": 0, 2024-09-13T18:34:27.956 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_fsck_bytes": 0, 2024-09-13T18:34:27.956 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_fsck_items": 0, 2024-09-13T18:34:27.956 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_txc_bytes": 0, 2024-09-13T18:34:27.956 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_txc_items": 0, 2024-09-13T18:34:27.956 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-13T18:34:27.957 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_deferred_items": 0, 2024-09-13T18:34:27.957 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_bytes": 0, 2024-09-13T18:34:27.957 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_items": 0, 2024-09-13T18:34:27.957 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_bytes": 0, 2024-09-13T18:34:27.957 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_items": 0, 2024-09-13T18:34:27.957 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_reader_bytes": 0, 2024-09-13T18:34:27.957 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_reader_items": 0, 2024-09-13T18:34:27.957 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_writer_bytes": 0, 2024-09-13T18:34:27.957 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_writer_items": 0, 2024-09-13T18:34:27.957 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_anon_bytes": 148994, 2024-09-13T18:34:27.957 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_anon_items": 31, 2024-09-13T18:34:27.957 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_meta_bytes": 0, 2024-09-13T18:34:27.957 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_meta_items": 0, 2024-09-13T18:34:27.957 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_bytes": 0, 2024-09-13T18:34:27.957 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_items": 0, 2024-09-13T18:34:27.958 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_mapbl_bytes": 0, 2024-09-13T18:34:27.958 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_mapbl_items": 0, 2024-09-13T18:34:27.958 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_pglog_bytes": 0, 2024-09-13T18:34:27.958 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_pglog_items": 0, 2024-09-13T18:34:27.958 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_bytes": 0, 2024-09-13T18:34:27.958 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_items": 0, 2024-09-13T18:34:27.958 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_mapping_bytes": 0, 2024-09-13T18:34:27.958 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_mapping_items": 0, 2024-09-13T18:34:27.958 INFO:teuthology.orchestra.run.smithi114.stdout: "pgmap_bytes": 0, 2024-09-13T18:34:27.958 INFO:teuthology.orchestra.run.smithi114.stdout: "pgmap_items": 0, 2024-09-13T18:34:27.958 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_co_bytes": 928, 2024-09-13T18:34:27.958 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_co_items": 2, 2024-09-13T18:34:27.958 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_1_bytes": 0, 2024-09-13T18:34:27.958 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_1_items": 0, 2024-09-13T18:34:27.959 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_2_bytes": 0, 2024-09-13T18:34:27.959 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_2_items": 0 2024-09-13T18:34:27.959 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-13T18:34:27.959 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-13T18:34:27.959 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-09-13T18:34:27.959 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 104857600, 2024-09-13T18:34:27.959 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-09-13T18:34:27.959 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 252, 2024-09-13T18:34:27.959 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 1531495, 2024-09-13T18:34:27.959 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-09-13T18:34:27.959 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 252, 2024-09-13T18:34:27.959 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 0, 2024-09-13T18:34:27.959 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 0, 2024-09-13T18:34:27.959 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 252, 2024-09-13T18:34:27.959 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 1531495, 2024-09-13T18:34:27.960 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-09-13T18:34:27.960 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-13T18:34:27.960 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-13T18:34:27.960 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-13T18:34:27.960 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:27.960 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-13T18:34:27.960 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-13T18:34:28.713 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-13T18:34:28.713 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-13T18:34:28.713 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-09-13T18:34:28.713 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-09-13T18:34:28.713 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-09-13T18:34:28.713 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-09-13T18:34:28.713 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-09-13T18:34:28.713 DEBUG:tasks.check_counter:Getting stats from b 2024-09-13T18:34:28.713 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-13T18:34:29.029 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T18:34:29.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:29 smithi134 ceph-mon[34283]: pgmap v375: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-13T18:34:29.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:29 smithi204 ceph-mon[34777]: pgmap v375: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-13T18:34:29.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:29 smithi114 ceph-mon[30897]: pgmap v375: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-13T18:34:30.915 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-13T18:34:30.915 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-0": { 2024-09-13T18:34:30.915 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 2932, 2024-09-13T18:34:30.915 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 2349, 2024-09-13T18:34:30.915 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 1143675, 2024-09-13T18:34:30.916 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 3084842, 2024-09-13T18:34:30.916 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 17, 2024-09-13T18:34:30.916 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 5, 2024-09-13T18:34:30.916 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.280167048, 2024-09-13T18:34:30.916 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.074634343, 2024-09-13T18:34:30.916 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.125909785, 2024-09-13T18:34:30.916 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.073783105, 2024-09-13T18:34:30.916 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T18:34:30.916 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2347, 2024-09-13T18:34:30.916 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 1.149790028, 2024-09-13T18:34:30.916 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000489897 2024-09-13T18:34:30.916 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.917 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-13T18:34:30.917 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 3502, 2024-09-13T18:34:30.917 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000660535, 2024-09-13T18:34:30.917 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000188 2024-09-13T18:34:30.917 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.917 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 57744, 2024-09-13T18:34:30.917 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 375632 2024-09-13T18:34:30.917 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.917 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-1": { 2024-09-13T18:34:30.917 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 2385, 2024-09-13T18:34:30.917 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 1667, 2024-09-13T18:34:30.917 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 2539223, 2024-09-13T18:34:30.917 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 865867, 2024-09-13T18:34:30.918 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 18, 2024-09-13T18:34:30.918 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 8, 2024-09-13T18:34:30.918 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.225692546, 2024-09-13T18:34:30.918 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.062364751, 2024-09-13T18:34:30.918 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.144334749, 2024-09-13T18:34:30.918 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.047359929, 2024-09-13T18:34:30.918 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T18:34:30.918 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1665, 2024-09-13T18:34:30.918 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.597336149, 2024-09-13T18:34:30.918 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000358760 2024-09-13T18:34:30.918 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.918 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-13T18:34:30.918 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2345, 2024-09-13T18:34:30.918 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000422174, 2024-09-13T18:34:30.919 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000180 2024-09-13T18:34:30.919 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.919 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 2076096, 2024-09-13T18:34:30.919 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 279600 2024-09-13T18:34:30.919 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.919 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-2": { 2024-09-13T18:34:30.919 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 2585, 2024-09-13T18:34:30.919 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 2307, 2024-09-13T18:34:30.919 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 1087966, 2024-09-13T18:34:30.919 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 3469648, 2024-09-13T18:34:30.919 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 266, 2024-09-13T18:34:30.919 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 5, 2024-09-13T18:34:30.919 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.296582879, 2024-09-13T18:34:30.919 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.089872346, 2024-09-13T18:34:30.920 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.201467211, 2024-09-13T18:34:30.920 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.022013423, 2024-09-13T18:34:30.920 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T18:34:30.920 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2306, 2024-09-13T18:34:30.920 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.329912885, 2024-09-13T18:34:30.920 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000143067 2024-09-13T18:34:30.920 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.920 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-13T18:34:30.920 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 3717, 2024-09-13T18:34:30.920 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000659691, 2024-09-13T18:34:30.920 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000177 2024-09-13T18:34:30.920 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.920 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 85232, 2024-09-13T18:34:30.921 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 139552 2024-09-13T18:34:30.921 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.921 INFO:teuthology.orchestra.run.smithi134.stdout: "cct": { 2024-09-13T18:34:30.921 INFO:teuthology.orchestra.run.smithi134.stdout: "total_workers": 1, 2024-09-13T18:34:30.921 INFO:teuthology.orchestra.run.smithi134.stdout: "unhealthy_workers": 0 2024-09-13T18:34:30.921 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.921 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-MDSRank": { 2024-09-13T18:34:30.921 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-09-13T18:34:30.921 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-09-13T18:34:30.921 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1375, 2024-09-13T18:34:30.921 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.233844158, 2024-09-13T18:34:30.921 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000170068 2024-09-13T18:34:30.921 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-13T18:34:30.921 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.922 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-PurgeQueue": { 2024-09-13T18:34:30.922 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-09-13T18:34:30.922 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-09-13T18:34:30.922 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2, 2024-09-13T18:34:30.922 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000012031, 2024-09-13T18:34:30.922 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000006015 2024-09-13T18:34:30.922 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-13T18:34:30.922 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.922 INFO:teuthology.orchestra.run.smithi134.stdout: "mds": { 2024-09-13T18:34:30.922 INFO:teuthology.orchestra.run.smithi134.stdout: "request": 398, 2024-09-13T18:34:30.922 INFO:teuthology.orchestra.run.smithi134.stdout: "reply": 242, 2024-09-13T18:34:30.922 INFO:teuthology.orchestra.run.smithi134.stdout: "reply_latency": { 2024-09-13T18:34:30.922 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 242, 2024-09-13T18:34:30.922 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 58.494638372, 2024-09-13T18:34:30.923 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.241713381 2024-09-13T18:34:30.923 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.923 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_reply": 0, 2024-09-13T18:34:30.923 INFO:teuthology.orchestra.run.smithi134.stdout: "forward": 156, 2024-09-13T18:34:30.923 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_complete": 4, 2024-09-13T18:34:30.923 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_keys": 1, 2024-09-13T18:34:30.923 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_commit": 152, 2024-09-13T18:34:30.923 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_split": 0, 2024-09-13T18:34:30.923 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_merge": 0, 2024-09-13T18:34:30.923 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes": 28, 2024-09-13T18:34:30.923 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_top": 7, 2024-09-13T18:34:30.923 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_bottom": 6, 2024-09-13T18:34:30.923 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pin_tail": 15, 2024-09-13T18:34:30.923 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pinned": 18, 2024-09-13T18:34:30.924 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_expired": 46, 2024-09-13T18:34:30.924 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_with_caps": 3, 2024-09-13T18:34:30.924 INFO:teuthology.orchestra.run.smithi134.stdout: "caps": 3, 2024-09-13T18:34:30.924 INFO:teuthology.orchestra.run.smithi134.stdout: "subtrees": 5, 2024-09-13T18:34:30.924 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse": 490, 2024-09-13T18:34:30.924 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_hit": 320, 2024-09-13T18:34:30.924 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_forward": 156, 2024-09-13T18:34:30.924 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_discover": 0, 2024-09-13T18:34:30.924 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_dir_fetch": 0, 2024-09-13T18:34:30.924 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_remote_ino": 0, 2024-09-13T18:34:30.924 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_lock": 0, 2024-09-13T18:34:30.924 INFO:teuthology.orchestra.run.smithi134.stdout: "load_cent": 0, 2024-09-13T18:34:30.924 INFO:teuthology.orchestra.run.smithi134.stdout: "q": 0, 2024-09-13T18:34:30.925 INFO:teuthology.orchestra.run.smithi134.stdout: "exported": 3, 2024-09-13T18:34:30.925 INFO:teuthology.orchestra.run.smithi134.stdout: "exported_inodes": 32, 2024-09-13T18:34:30.925 INFO:teuthology.orchestra.run.smithi134.stdout: "imported": 1, 2024-09-13T18:34:30.925 INFO:teuthology.orchestra.run.smithi134.stdout: "imported_inodes": 2, 2024-09-13T18:34:30.925 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_dir_fetch": 0, 2024-09-13T18:34:30.925 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_backtrace_fetch": 271, 2024-09-13T18:34:30.925 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_peer_discover": 0, 2024-09-13T18:34:30.925 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rfiles": 4, 2024-09-13T18:34:30.925 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rbytes": 234, 2024-09-13T18:34:30.925 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rsnaps": 6, 2024-09-13T18:34:30.925 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_fetch": 271, 2024-09-13T18:34:30.925 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_set_tag": 0, 2024-09-13T18:34:30.925 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_repaired": 0, 2024-09-13T18:34:30.925 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_inotable_repaired": 0, 2024-09-13T18:34:30.926 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_inodes": 153, 2024-09-13T18:34:30.926 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_base_inodes": 20, 2024-09-13T18:34:30.926 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dirfrag_rstats": 133, 2024-09-13T18:34:30.926 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_file_inodes": 118, 2024-09-13T18:34:30.926 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_inode_file_caps": 0, 2024-09-13T18:34:30.926 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_revoke": 44, 2024-09-13T18:34:30.926 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_grant": 103, 2024-09-13T18:34:30.926 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_trunc": 0, 2024-09-13T18:34:30.926 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-13T18:34:30.926 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-13T18:34:30.926 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps": 168, 2024-09-13T18:34:30.926 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps_dirty": 118, 2024-09-13T18:34:30.926 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_cap_release": 3, 2024-09-13T18:34:30.926 INFO:teuthology.orchestra.run.smithi134.stdout: "process_request_cap_release": 40 2024-09-13T18:34:30.927 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.927 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_cache": { 2024-09-13T18:34:30.927 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update": 16, 2024-09-13T18:34:30.927 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update_receipt": 0, 2024-09-13T18:34:30.927 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_try_discover": 0, 2024-09-13T18:34:30.927 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_send_discover": 0, 2024-09-13T18:34:30.927 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_handle_discover": 35, 2024-09-13T18:34:30.927 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays": 0, 2024-09-13T18:34:30.927 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_delayed": 0, 2024-09-13T18:34:30.927 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_enqueuing": 0, 2024-09-13T18:34:30.927 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_created": 0, 2024-09-13T18:34:30.927 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_enqueued": 0, 2024-09-13T18:34:30.927 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_reintegrated": 0, 2024-09-13T18:34:30.928 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_migrated": 0, 2024-09-13T18:34:30.928 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_processing": 0, 2024-09-13T18:34:30.928 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_enqueued": 0, 2024-09-13T18:34:30.928 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_prioritized": 0, 2024-09-13T18:34:30.928 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_started": 0, 2024-09-13T18:34:30.928 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_completed": 0, 2024-09-13T18:34:30.928 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_enqueue_scrub": 20, 2024-09-13T18:34:30.928 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_exportdir": 3, 2024-09-13T18:34:30.928 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_flush": 0, 2024-09-13T18:34:30.928 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragmentdir": 0, 2024-09-13T18:34:30.929 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragstats": 0, 2024-09-13T18:34:30.929 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_inodestats": 0 2024-09-13T18:34:30.929 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.929 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_log": { 2024-09-13T18:34:30.929 INFO:teuthology.orchestra.run.smithi134.stdout: "evadd": 353, 2024-09-13T18:34:30.929 INFO:teuthology.orchestra.run.smithi134.stdout: "evex": 337, 2024-09-13T18:34:30.929 INFO:teuthology.orchestra.run.smithi134.stdout: "evtrm": 337, 2024-09-13T18:34:30.929 INFO:teuthology.orchestra.run.smithi134.stdout: "ev": 16, 2024-09-13T18:34:30.929 INFO:teuthology.orchestra.run.smithi134.stdout: "evexg": 0, 2024-09-13T18:34:30.929 INFO:teuthology.orchestra.run.smithi134.stdout: "evexd": 1, 2024-09-13T18:34:30.929 INFO:teuthology.orchestra.run.smithi134.stdout: "segadd": 50, 2024-09-13T18:34:30.929 INFO:teuthology.orchestra.run.smithi134.stdout: "segex": 49, 2024-09-13T18:34:30.929 INFO:teuthology.orchestra.run.smithi134.stdout: "segtrm": 49, 2024-09-13T18:34:30.930 INFO:teuthology.orchestra.run.smithi134.stdout: "seg": 2, 2024-09-13T18:34:30.930 INFO:teuthology.orchestra.run.smithi134.stdout: "segexg": 0, 2024-09-13T18:34:30.930 INFO:teuthology.orchestra.run.smithi134.stdout: "segexd": 1, 2024-09-13T18:34:30.930 INFO:teuthology.orchestra.run.smithi134.stdout: "expos": 5861362, 2024-09-13T18:34:30.930 INFO:teuthology.orchestra.run.smithi134.stdout: "wrpos": 5963471, 2024-09-13T18:34:30.930 INFO:teuthology.orchestra.run.smithi134.stdout: "rdpos": 0, 2024-09-13T18:34:30.930 INFO:teuthology.orchestra.run.smithi134.stdout: "jlat": { 2024-09-13T18:34:30.930 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 279, 2024-09-13T18:34:30.930 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.784434750, 2024-09-13T18:34:30.930 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.002811594 2024-09-13T18:34:30.931 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.931 INFO:teuthology.orchestra.run.smithi134.stdout: "replayed": 0 2024-09-13T18:34:30.931 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.931 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_mem": { 2024-09-13T18:34:30.931 INFO:teuthology.orchestra.run.smithi134.stdout: "ino": 33, 2024-09-13T18:34:30.931 INFO:teuthology.orchestra.run.smithi134.stdout: "ino+": 86, 2024-09-13T18:34:30.931 INFO:teuthology.orchestra.run.smithi134.stdout: "ino-": 53, 2024-09-13T18:34:30.931 INFO:teuthology.orchestra.run.smithi134.stdout: "dir": 32, 2024-09-13T18:34:30.931 INFO:teuthology.orchestra.run.smithi134.stdout: "dir+": 37, 2024-09-13T18:34:30.931 INFO:teuthology.orchestra.run.smithi134.stdout: "dir-": 5, 2024-09-13T18:34:30.931 INFO:teuthology.orchestra.run.smithi134.stdout: "dn": 29, 2024-09-13T18:34:30.932 INFO:teuthology.orchestra.run.smithi134.stdout: "dn+": 75, 2024-09-13T18:34:30.932 INFO:teuthology.orchestra.run.smithi134.stdout: "dn-": 46, 2024-09-13T18:34:30.932 INFO:teuthology.orchestra.run.smithi134.stdout: "cap": 3, 2024-09-13T18:34:30.932 INFO:teuthology.orchestra.run.smithi134.stdout: "cap+": 45, 2024-09-13T18:34:30.932 INFO:teuthology.orchestra.run.smithi134.stdout: "cap-": 42, 2024-09-13T18:34:30.932 INFO:teuthology.orchestra.run.smithi134.stdout: "rss": 55428, 2024-09-13T18:34:30.932 INFO:teuthology.orchestra.run.smithi134.stdout: "heap": 207124 2024-09-13T18:34:30.932 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.932 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_server": { 2024-09-13T18:34:30.932 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_client_request": 469, 2024-09-13T18:34:30.933 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_server_request": 4, 2024-09-13T18:34:30.933 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_request": 398, 2024-09-13T18:34:30.933 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_session": 40, 2024-09-13T18:34:30.933 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_peer_request": 16, 2024-09-13T18:34:30.933 INFO:teuthology.orchestra.run.smithi134.stdout: "req_create_latency": { 2024-09-13T18:34:30.933 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 22, 2024-09-13T18:34:30.933 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.015098666, 2024-09-13T18:34:30.933 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000686303 2024-09-13T18:34:30.933 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.933 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getattr_latency": { 2024-09-13T18:34:30.934 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 129, 2024-09-13T18:34:30.934 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.022141218, 2024-09-13T18:34:30.934 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000171637 2024-09-13T18:34:30.934 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.934 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getfilelock_latency": { 2024-09-13T18:34:30.934 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:30.934 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:30.934 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:30.934 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.934 INFO:teuthology.orchestra.run.smithi134.stdout: "req_link_latency": { 2024-09-13T18:34:30.934 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:30.935 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:30.935 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:30.935 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.935 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookup_latency": { 2024-09-13T18:34:30.935 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 37, 2024-09-13T18:34:30.935 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 58.294560757, 2024-09-13T18:34:30.935 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 1.575528669 2024-09-13T18:34:30.935 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.935 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookuphash_latency": { 2024-09-13T18:34:30.935 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:30.936 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:30.936 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:30.936 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.936 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupino_latency": { 2024-09-13T18:34:30.936 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:30.936 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:30.936 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:30.936 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.936 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupname_latency": { 2024-09-13T18:34:30.936 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:30.936 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:30.937 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:30.937 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.937 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupparent_latency": { 2024-09-13T18:34:30.937 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:30.937 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:30.937 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:30.937 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.937 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupsnap_latency": { 2024-09-13T18:34:30.937 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:30.937 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:30.938 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:30.938 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.938 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lssnap_latency": { 2024-09-13T18:34:30.938 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 12, 2024-09-13T18:34:30.938 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.001819565, 2024-09-13T18:34:30.938 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000151630 2024-09-13T18:34:30.938 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.938 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mkdir_latency": { 2024-09-13T18:34:30.938 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 10, 2024-09-13T18:34:30.939 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.016059068, 2024-09-13T18:34:30.939 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.001605906 2024-09-13T18:34:30.939 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.939 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mknod_latency": { 2024-09-13T18:34:30.939 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:30.940 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:30.940 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:30.940 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.940 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mksnap_latency": { 2024-09-13T18:34:30.940 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 12, 2024-09-13T18:34:30.940 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.089751327, 2024-09-13T18:34:30.940 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.007479277 2024-09-13T18:34:30.940 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.940 INFO:teuthology.orchestra.run.smithi134.stdout: "req_open_latency": { 2024-09-13T18:34:30.940 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1, 2024-09-13T18:34:30.941 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000399446, 2024-09-13T18:34:30.941 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000399446 2024-09-13T18:34:30.941 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.941 INFO:teuthology.orchestra.run.smithi134.stdout: "req_readdir_latency": { 2024-09-13T18:34:30.941 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1, 2024-09-13T18:34:30.941 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.005979613, 2024-09-13T18:34:30.941 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.005979613 2024-09-13T18:34:30.941 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.941 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rename_latency": { 2024-09-13T18:34:30.941 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2, 2024-09-13T18:34:30.941 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.002617764, 2024-09-13T18:34:30.942 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.001308882 2024-09-13T18:34:30.942 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.942 INFO:teuthology.orchestra.run.smithi134.stdout: "req_renamesnap_latency": { 2024-09-13T18:34:30.942 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:30.942 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:30.942 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:30.942 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.942 INFO:teuthology.orchestra.run.smithi134.stdout: "req_snapdiff_latency": { 2024-09-13T18:34:30.942 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:30.942 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:30.942 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:30.942 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.943 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmdir_latency": { 2024-09-13T18:34:30.943 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:30.943 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:30.943 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:30.943 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.943 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmsnap_latency": { 2024-09-13T18:34:30.943 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 6, 2024-09-13T18:34:30.943 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.026202054, 2024-09-13T18:34:30.943 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.004367009 2024-09-13T18:34:30.943 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.943 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmxattr_latency": { 2024-09-13T18:34:30.943 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:30.943 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:30.943 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:30.944 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.944 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setattr_latency": { 2024-09-13T18:34:30.944 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2, 2024-09-13T18:34:30.944 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.001057302, 2024-09-13T18:34:30.944 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000528651 2024-09-13T18:34:30.944 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.944 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setdirlayout_latency": { 2024-09-13T18:34:30.944 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:30.944 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:30.944 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:30.944 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.944 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setfilelock_latency": { 2024-09-13T18:34:30.944 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:30.945 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:30.945 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:30.945 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.945 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setlayout_latency": { 2024-09-13T18:34:30.945 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:30.945 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:30.945 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:30.945 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.945 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setxattr_latency": { 2024-09-13T18:34:30.945 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 8, 2024-09-13T18:34:30.945 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.018951592, 2024-09-13T18:34:30.945 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.002368949 2024-09-13T18:34:30.945 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.945 INFO:teuthology.orchestra.run.smithi134.stdout: "req_symlink_latency": { 2024-09-13T18:34:30.946 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:30.946 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:30.946 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:30.946 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.946 INFO:teuthology.orchestra.run.smithi134.stdout: "req_unlink_latency": { 2024-09-13T18:34:30.946 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:30.946 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:30.946 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:30.946 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.946 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_revoke_eviction": 0, 2024-09-13T18:34:30.946 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_acquisition_throttle": 0, 2024-09-13T18:34:30.946 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getvxattr_latency": { 2024-09-13T18:34:30.946 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:30.946 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:30.947 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:30.947 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-13T18:34:30.947 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.947 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_sessions": { 2024-09-13T18:34:30.947 INFO:teuthology.orchestra.run.smithi134.stdout: "session_count": 2, 2024-09-13T18:34:30.947 INFO:teuthology.orchestra.run.smithi134.stdout: "session_add": 6, 2024-09-13T18:34:30.947 INFO:teuthology.orchestra.run.smithi134.stdout: "session_remove": 4, 2024-09-13T18:34:30.947 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_open": 2, 2024-09-13T18:34:30.947 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_stale": 0, 2024-09-13T18:34:30.947 INFO:teuthology.orchestra.run.smithi134.stdout: "total_load": 33, 2024-09-13T18:34:30.947 INFO:teuthology.orchestra.run.smithi134.stdout: "average_load": 16, 2024-09-13T18:34:30.947 INFO:teuthology.orchestra.run.smithi134.stdout: "avg_session_uptime": 702, 2024-09-13T18:34:30.947 INFO:teuthology.orchestra.run.smithi134.stdout: "mdthresh_evicted": 0 2024-09-13T18:34:30.947 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.948 INFO:teuthology.orchestra.run.smithi134.stdout: "mempool": { 2024-09-13T18:34:30.948 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_bytes": 0, 2024-09-13T18:34:30.948 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_items": 0, 2024-09-13T18:34:30.948 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_bytes": 0, 2024-09-13T18:34:30.948 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_items": 0, 2024-09-13T18:34:30.948 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_bytes": 0, 2024-09-13T18:34:30.948 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_items": 0, 2024-09-13T18:34:30.948 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-13T18:34:30.948 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_items": 0, 2024-09-13T18:34:30.948 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-13T18:34:30.948 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_items": 0, 2024-09-13T18:34:30.948 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_bytes": 0, 2024-09-13T18:34:30.948 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_items": 0, 2024-09-13T18:34:30.948 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-13T18:34:30.949 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_items": 0, 2024-09-13T18:34:30.949 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_bytes": 0, 2024-09-13T18:34:30.949 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_items": 0, 2024-09-13T18:34:30.949 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_bytes": 0, 2024-09-13T18:34:30.949 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_items": 0, 2024-09-13T18:34:30.949 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-13T18:34:30.949 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_items": 0, 2024-09-13T18:34:30.949 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-13T18:34:30.949 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_items": 0, 2024-09-13T18:34:30.949 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_bytes": 0, 2024-09-13T18:34:30.949 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_items": 0, 2024-09-13T18:34:30.949 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_bytes": 0, 2024-09-13T18:34:30.949 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_items": 0, 2024-09-13T18:34:30.949 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-13T18:34:30.950 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_items": 0, 2024-09-13T18:34:30.950 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_bytes": 0, 2024-09-13T18:34:30.950 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_items": 0, 2024-09-13T18:34:30.950 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_bytes": 0, 2024-09-13T18:34:30.950 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_items": 0, 2024-09-13T18:34:30.950 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_bytes": 0, 2024-09-13T18:34:30.950 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_items": 0, 2024-09-13T18:34:30.950 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_bytes": 0, 2024-09-13T18:34:30.950 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_items": 0, 2024-09-13T18:34:30.950 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_bytes": 346133, 2024-09-13T18:34:30.950 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_items": 136, 2024-09-13T18:34:30.950 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_bytes": 352, 2024-09-13T18:34:30.950 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_items": 4, 2024-09-13T18:34:30.951 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_bytes": 0, 2024-09-13T18:34:30.951 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_items": 0, 2024-09-13T18:34:30.951 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_bytes": 0, 2024-09-13T18:34:30.951 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_items": 0, 2024-09-13T18:34:30.951 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_bytes": 0, 2024-09-13T18:34:30.951 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_items": 0, 2024-09-13T18:34:30.951 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_bytes": 9704, 2024-09-13T18:34:30.951 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_items": 196, 2024-09-13T18:34:30.951 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_bytes": 0, 2024-09-13T18:34:30.951 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_items": 0, 2024-09-13T18:34:30.951 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_bytes": 0, 2024-09-13T18:34:30.951 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_items": 0, 2024-09-13T18:34:30.951 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_bytes": 164522, 2024-09-13T18:34:30.951 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_items": 804, 2024-09-13T18:34:30.952 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_bytes": 0, 2024-09-13T18:34:30.952 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_items": 0, 2024-09-13T18:34:30.952 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_bytes": 0, 2024-09-13T18:34:30.952 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_items": 0 2024-09-13T18:34:30.952 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.952 INFO:teuthology.orchestra.run.smithi134.stdout: "objecter": { 2024-09-13T18:34:30.952 INFO:teuthology.orchestra.run.smithi134.stdout: "op_active": 0, 2024-09-13T18:34:30.952 INFO:teuthology.orchestra.run.smithi134.stdout: "op_laggy": 0, 2024-09-13T18:34:30.952 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send": 1556, 2024-09-13T18:34:30.952 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send_bytes": 2265140, 2024-09-13T18:34:30.952 INFO:teuthology.orchestra.run.smithi134.stdout: "op_resend": 37, 2024-09-13T18:34:30.952 INFO:teuthology.orchestra.run.smithi134.stdout: "op_reply": 1556, 2024-09-13T18:34:30.953 INFO:teuthology.orchestra.run.smithi134.stdout: "op_latency": { 2024-09-13T18:34:30.953 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1556, 2024-09-13T18:34:30.953 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 21.615465166, 2024-09-13T18:34:30.953 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.013891687 2024-09-13T18:34:30.953 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.953 INFO:teuthology.orchestra.run.smithi134.stdout: "op_inflight": 0, 2024-09-13T18:34:30.953 INFO:teuthology.orchestra.run.smithi134.stdout: "oplen_avg": { 2024-09-13T18:34:30.953 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1556, 2024-09-13T18:34:30.953 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 1962 2024-09-13T18:34:30.953 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.953 INFO:teuthology.orchestra.run.smithi134.stdout: "op": 1556, 2024-09-13T18:34:30.953 INFO:teuthology.orchestra.run.smithi134.stdout: "op_r": 316, 2024-09-13T18:34:30.953 INFO:teuthology.orchestra.run.smithi134.stdout: "op_w": 1240, 2024-09-13T18:34:30.954 INFO:teuthology.orchestra.run.smithi134.stdout: "op_rmw": 0, 2024-09-13T18:34:30.954 INFO:teuthology.orchestra.run.smithi134.stdout: "op_pg": 0, 2024-09-13T18:34:30.954 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_stat": 130, 2024-09-13T18:34:30.954 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_create": 66, 2024-09-13T18:34:30.954 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_read": 20, 2024-09-13T18:34:30.954 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_write": 279, 2024-09-13T18:34:30.954 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writefull": 156, 2024-09-13T18:34:30.954 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writesame": 0, 2024-09-13T18:34:30.954 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_append": 0, 2024-09-13T18:34:30.954 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_zero": 0, 2024-09-13T18:34:30.954 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_truncate": 0, 2024-09-13T18:34:30.954 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_delete": 506, 2024-09-13T18:34:30.954 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_mapext": 0, 2024-09-13T18:34:30.955 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_sparse_read": 0, 2024-09-13T18:34:30.955 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_clonerange": 0, 2024-09-13T18:34:30.955 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_getxattr": 296, 2024-09-13T18:34:30.955 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_setxattr": 110, 2024-09-13T18:34:30.955 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_cmpxattr": 0, 2024-09-13T18:34:30.955 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_rmxattr": 0, 2024-09-13T18:34:30.955 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_resetxattrs": 0, 2024-09-13T18:34:30.955 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_call": 0, 2024-09-13T18:34:30.955 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_watch": 0, 2024-09-13T18:34:30.955 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_notify": 0, 2024-09-13T18:34:30.955 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_src_cmpxattr": 0, 2024-09-13T18:34:30.955 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls": 0, 2024-09-13T18:34:30.955 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls_filter": 0, 2024-09-13T18:34:30.955 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_other": 0, 2024-09-13T18:34:30.956 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_active": 0, 2024-09-13T18:34:30.956 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_send": 0, 2024-09-13T18:34:30.956 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_resend": 0, 2024-09-13T18:34:30.956 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_ping": 0, 2024-09-13T18:34:30.956 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_active": 0, 2024-09-13T18:34:30.956 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_send": 0, 2024-09-13T18:34:30.956 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_resend": 0, 2024-09-13T18:34:30.956 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_active": 0, 2024-09-13T18:34:30.956 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_send": 0, 2024-09-13T18:34:30.956 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_resend": 0, 2024-09-13T18:34:30.956 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_active": 0, 2024-09-13T18:34:30.956 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_send": 0, 2024-09-13T18:34:30.956 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_resend": 0, 2024-09-13T18:34:30.957 INFO:teuthology.orchestra.run.smithi134.stdout: "command_active": 0, 2024-09-13T18:34:30.957 INFO:teuthology.orchestra.run.smithi134.stdout: "command_send": 0, 2024-09-13T18:34:30.957 INFO:teuthology.orchestra.run.smithi134.stdout: "command_resend": 0, 2024-09-13T18:34:30.957 INFO:teuthology.orchestra.run.smithi134.stdout: "map_epoch": 106, 2024-09-13T18:34:30.957 INFO:teuthology.orchestra.run.smithi134.stdout: "map_full": 0, 2024-09-13T18:34:30.957 INFO:teuthology.orchestra.run.smithi134.stdout: "map_inc": 22, 2024-09-13T18:34:30.957 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_sessions": 12, 2024-09-13T18:34:30.957 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_open": 12, 2024-09-13T18:34:30.957 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_close": 0, 2024-09-13T18:34:30.957 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_laggy": 0, 2024-09-13T18:34:30.957 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_wr": 377, 2024-09-13T18:34:30.957 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_rd": 10, 2024-09-13T18:34:30.957 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_del": 12 2024-09-13T18:34:30.957 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.958 INFO:teuthology.orchestra.run.smithi134.stdout: "oft": { 2024-09-13T18:34:30.958 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_objs": 1, 2024-09-13T18:34:30.958 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_kv_pairs": 0, 2024-09-13T18:34:30.958 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_updates": 0, 2024-09-13T18:34:30.958 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_removes": 0 2024-09-13T18:34:30.958 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.958 INFO:teuthology.orchestra.run.smithi134.stdout: "osd-slow-ops": { 2024-09-13T18:34:30.958 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_ops_count": 5 2024-09-13T18:34:30.958 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.958 INFO:teuthology.orchestra.run.smithi134.stdout: "purge_queue": { 2024-09-13T18:34:30.958 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops": 0, 2024-09-13T18:34:30.958 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops_high_water": 0, 2024-09-13T18:34:30.958 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing": 0, 2024-09-13T18:34:30.958 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_high_water": 0, 2024-09-13T18:34:30.959 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed_ops": 0, 2024-09-13T18:34:30.959 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed": 0, 2024-09-13T18:34:30.959 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_item_in_journal": 0 2024-09-13T18:34:30.959 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.959 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-13T18:34:30.959 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-13T18:34:30.959 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-09-13T18:34:30.959 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-13T18:34:30.959 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 7898, 2024-09-13T18:34:30.959 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 3985290, 2024-09-13T18:34:30.959 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-13T18:34:30.959 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 7898, 2024-09-13T18:34:30.959 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-13T18:34:30.959 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-13T18:34:30.960 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 7898, 2024-09-13T18:34:30.960 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 3985290, 2024-09-13T18:34:30.960 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-13T18:34:30.960 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:30.960 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:30.960 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:30.960 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-13T18:34:30.960 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.960 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_bytes": { 2024-09-13T18:34:30.960 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-13T18:34:30.960 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-09-13T18:34:30.960 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-13T18:34:30.960 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-13T18:34:30.960 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-13T18:34:30.961 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-13T18:34:30.961 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-13T18:34:30.961 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 1556, 2024-09-13T18:34:30.961 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 2265072, 2024-09-13T18:34:30.961 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 1556, 2024-09-13T18:34:30.961 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 2265072, 2024-09-13T18:34:30.961 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-13T18:34:30.961 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:30.961 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:30.961 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:30.961 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-13T18:34:30.961 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.961 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_ops": { 2024-09-13T18:34:30.962 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-13T18:34:30.962 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 1024, 2024-09-13T18:34:30.962 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-13T18:34:30.962 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-13T18:34:30.962 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-13T18:34:30.962 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-13T18:34:30.962 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-13T18:34:30.962 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 1556, 2024-09-13T18:34:30.962 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 1556, 2024-09-13T18:34:30.962 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 1556, 2024-09-13T18:34:30.962 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 1556, 2024-09-13T18:34:30.963 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-13T18:34:30.963 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:30.963 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:30.963 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:30.963 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-13T18:34:30.963 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.963 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle": { 2024-09-13T18:34:30.963 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-13T18:34:30.963 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-09-13T18:34:30.963 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-13T18:34:30.963 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-13T18:34:30.963 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-13T18:34:30.963 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-13T18:34:30.963 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-13T18:34:30.963 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-13T18:34:30.964 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-13T18:34:30.964 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 0, 2024-09-13T18:34:30.964 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 0, 2024-09-13T18:34:30.964 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-13T18:34:30.964 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:30.964 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:30.964 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:30.964 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-13T18:34:30.964 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:30.964 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle-0x55d96b3ec500": { 2024-09-13T18:34:30.965 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-13T18:34:30.965 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-09-13T18:34:30.965 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-13T18:34:30.965 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 353, 2024-09-13T18:34:30.965 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 1769167, 2024-09-13T18:34:30.965 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-13T18:34:30.965 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 353, 2024-09-13T18:34:30.965 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-13T18:34:30.965 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-13T18:34:30.965 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 279, 2024-09-13T18:34:30.965 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 1769167, 2024-09-13T18:34:30.965 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-13T18:34:30.965 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:30.966 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:30.966 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:30.966 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-13T18:34:30.966 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-13T18:34:30.966 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-13T18:34:31.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:31 smithi134 ceph-mon[34283]: pgmap v376: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s 2024-09-13T18:34:31.676 INFO:tasks.check_counter:Daemon mds.b mds.exported=3 2024-09-13T18:34:31.677 INFO:tasks.check_counter:Daemon mds.b mds.imported=1 2024-09-13T18:34:31.677 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=16 2024-09-13T18:34:31.677 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-09-13T18:34:31.677 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-09-13T18:34:31.677 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=12 2024-09-13T18:34:31.677 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=6 2024-09-13T18:34:31.677 DEBUG:tasks.check_counter:Getting stats from e 2024-09-13T18:34:31.678 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-13T18:34:31.879 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T18:34:31.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:31 smithi204 ceph-mon[34777]: pgmap v376: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s 2024-09-13T18:34:31.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:31 smithi114 ceph-mon[30897]: pgmap v376: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s 2024-09-13T18:34:33.783 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:33 smithi134 ceph-mon[34283]: pgmap v377: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-13T18:34:33.884 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-13T18:34:33.884 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-0": { 2024-09-13T18:34:33.884 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 786, 2024-09-13T18:34:33.884 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 794, 2024-09-13T18:34:33.884 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 268658, 2024-09-13T18:34:33.884 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 696845, 2024-09-13T18:34:33.885 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 50, 2024-09-13T18:34:33.885 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 2, 2024-09-13T18:34:33.885 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.093102101, 2024-09-13T18:34:33.885 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.033887206, 2024-09-13T18:34:33.885 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.054601555, 2024-09-13T18:34:33.885 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.004296594, 2024-09-13T18:34:33.885 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T18:34:33.885 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 792, 2024-09-13T18:34:33.885 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.054627615, 2024-09-13T18:34:33.885 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000068974 2024-09-13T18:34:33.885 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.885 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-13T18:34:33.885 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1472, 2024-09-13T18:34:33.886 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000296336, 2024-09-13T18:34:33.886 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000201 2024-09-13T18:34:33.886 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.886 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-13T18:34:33.886 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 0 2024-09-13T18:34:33.886 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.886 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-1": { 2024-09-13T18:34:33.886 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 447, 2024-09-13T18:34:33.886 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 564, 2024-09-13T18:34:33.886 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 222369, 2024-09-13T18:34:33.886 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 675182, 2024-09-13T18:34:33.886 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 16, 2024-09-13T18:34:33.886 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 5, 2024-09-13T18:34:33.887 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.045020890, 2024-09-13T18:34:33.887 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.016508197, 2024-09-13T18:34:33.887 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.030740692, 2024-09-13T18:34:33.887 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.000296874, 2024-09-13T18:34:33.887 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T18:34:33.887 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 564, 2024-09-13T18:34:33.887 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.053438825, 2024-09-13T18:34:33.887 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000094749 2024-09-13T18:34:33.887 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.887 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-13T18:34:33.887 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 755, 2024-09-13T18:34:33.887 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000179341, 2024-09-13T18:34:33.887 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000237 2024-09-13T18:34:33.888 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.888 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 3280, 2024-09-13T18:34:33.888 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 469296 2024-09-13T18:34:33.888 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.888 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-2": { 2024-09-13T18:34:33.888 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 497, 2024-09-13T18:34:33.888 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 421, 2024-09-13T18:34:33.888 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 1802926, 2024-09-13T18:34:33.888 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 240336, 2024-09-13T18:34:33.888 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 7, 2024-09-13T18:34:33.888 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 6, 2024-09-13T18:34:33.888 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.057782995, 2024-09-13T18:34:33.888 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.020246699, 2024-09-13T18:34:33.889 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.055261129, 2024-09-13T18:34:33.889 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.000204121, 2024-09-13T18:34:33.889 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T18:34:33.889 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 418, 2024-09-13T18:34:33.889 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.032273610, 2024-09-13T18:34:33.889 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000077209 2024-09-13T18:34:33.889 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.889 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-13T18:34:33.889 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 273, 2024-09-13T18:34:33.889 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000118980, 2024-09-13T18:34:33.889 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000435 2024-09-13T18:34:33.889 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.889 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 1679424, 2024-09-13T18:34:33.889 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 145360 2024-09-13T18:34:33.890 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.890 INFO:teuthology.orchestra.run.smithi134.stdout: "cct": { 2024-09-13T18:34:33.890 INFO:teuthology.orchestra.run.smithi134.stdout: "total_workers": 1, 2024-09-13T18:34:33.890 INFO:teuthology.orchestra.run.smithi134.stdout: "unhealthy_workers": 0 2024-09-13T18:34:33.890 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.890 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-MDSRank": { 2024-09-13T18:34:33.890 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-09-13T18:34:33.890 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-09-13T18:34:33.890 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 42, 2024-09-13T18:34:33.890 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.001697136, 2024-09-13T18:34:33.890 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000040408 2024-09-13T18:34:33.890 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-13T18:34:33.890 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.891 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-PurgeQueue": { 2024-09-13T18:34:33.891 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-09-13T18:34:33.891 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-09-13T18:34:33.891 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2, 2024-09-13T18:34:33.891 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000014220, 2024-09-13T18:34:33.891 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000007110 2024-09-13T18:34:33.891 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-13T18:34:33.891 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.891 INFO:teuthology.orchestra.run.smithi134.stdout: "mds": { 2024-09-13T18:34:33.891 INFO:teuthology.orchestra.run.smithi134.stdout: "request": 142, 2024-09-13T18:34:33.891 INFO:teuthology.orchestra.run.smithi134.stdout: "reply": 0, 2024-09-13T18:34:33.891 INFO:teuthology.orchestra.run.smithi134.stdout: "reply_latency": { 2024-09-13T18:34:33.891 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:33.891 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:33.892 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:33.892 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.892 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_reply": 0, 2024-09-13T18:34:33.892 INFO:teuthology.orchestra.run.smithi134.stdout: "forward": 142, 2024-09-13T18:34:33.892 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_complete": 0, 2024-09-13T18:34:33.892 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_keys": 0, 2024-09-13T18:34:33.892 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_commit": 11, 2024-09-13T18:34:33.892 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_split": 0, 2024-09-13T18:34:33.892 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_merge": 0, 2024-09-13T18:34:33.892 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes": 17, 2024-09-13T18:34:33.892 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_top": 0, 2024-09-13T18:34:33.892 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_bottom": 17, 2024-09-13T18:34:33.892 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pin_tail": 0, 2024-09-13T18:34:33.892 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pinned": 13, 2024-09-13T18:34:33.893 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_expired": 8, 2024-09-13T18:34:33.893 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_with_caps": 0, 2024-09-13T18:34:33.893 INFO:teuthology.orchestra.run.smithi134.stdout: "caps": 0, 2024-09-13T18:34:33.893 INFO:teuthology.orchestra.run.smithi134.stdout: "subtrees": 4, 2024-09-13T18:34:33.893 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse": 153, 2024-09-13T18:34:33.893 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_hit": 7, 2024-09-13T18:34:33.893 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_forward": 142, 2024-09-13T18:34:33.893 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_discover": 4, 2024-09-13T18:34:33.893 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_dir_fetch": 0, 2024-09-13T18:34:33.893 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_remote_ino": 0, 2024-09-13T18:34:33.893 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_lock": 0, 2024-09-13T18:34:33.893 INFO:teuthology.orchestra.run.smithi134.stdout: "load_cent": 0, 2024-09-13T18:34:33.893 INFO:teuthology.orchestra.run.smithi134.stdout: "q": 0, 2024-09-13T18:34:33.894 INFO:teuthology.orchestra.run.smithi134.stdout: "exported": 0, 2024-09-13T18:34:33.894 INFO:teuthology.orchestra.run.smithi134.stdout: "exported_inodes": 0, 2024-09-13T18:34:33.894 INFO:teuthology.orchestra.run.smithi134.stdout: "imported": 0, 2024-09-13T18:34:33.894 INFO:teuthology.orchestra.run.smithi134.stdout: "imported_inodes": 0, 2024-09-13T18:34:33.894 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_dir_fetch": 0, 2024-09-13T18:34:33.894 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_backtrace_fetch": 0, 2024-09-13T18:34:33.894 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_peer_discover": 0, 2024-09-13T18:34:33.894 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rfiles": 22, 2024-09-13T18:34:33.894 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rbytes": 10738143812, 2024-09-13T18:34:33.894 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rsnaps": 6, 2024-09-13T18:34:33.894 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_fetch": 0, 2024-09-13T18:34:33.894 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_set_tag": 0, 2024-09-13T18:34:33.894 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_repaired": 0, 2024-09-13T18:34:33.895 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_inotable_repaired": 0, 2024-09-13T18:34:33.895 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_inodes": 0, 2024-09-13T18:34:33.895 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_base_inodes": 0, 2024-09-13T18:34:33.895 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dirfrag_rstats": 0, 2024-09-13T18:34:33.895 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_file_inodes": 0, 2024-09-13T18:34:33.895 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_inode_file_caps": 0, 2024-09-13T18:34:33.895 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_revoke": 0, 2024-09-13T18:34:33.895 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_grant": 0, 2024-09-13T18:34:33.895 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_trunc": 0, 2024-09-13T18:34:33.895 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-13T18:34:33.895 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-13T18:34:33.895 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps": 0, 2024-09-13T18:34:33.895 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps_dirty": 0, 2024-09-13T18:34:33.895 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_cap_release": 0, 2024-09-13T18:34:33.896 INFO:teuthology.orchestra.run.smithi134.stdout: "process_request_cap_release": 0 2024-09-13T18:34:33.896 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.896 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_cache": { 2024-09-13T18:34:33.896 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update": 0, 2024-09-13T18:34:33.896 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update_receipt": 14, 2024-09-13T18:34:33.896 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_try_discover": 9, 2024-09-13T18:34:33.896 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_send_discover": 10, 2024-09-13T18:34:33.896 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_handle_discover": 0, 2024-09-13T18:34:33.896 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays": 0, 2024-09-13T18:34:33.896 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_delayed": 0, 2024-09-13T18:34:33.896 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_enqueuing": 0, 2024-09-13T18:34:33.896 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_created": 0, 2024-09-13T18:34:33.896 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_enqueued": 0, 2024-09-13T18:34:33.896 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_reintegrated": 0, 2024-09-13T18:34:33.897 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_migrated": 0, 2024-09-13T18:34:33.897 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_processing": 0, 2024-09-13T18:34:33.897 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_enqueued": 0, 2024-09-13T18:34:33.897 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_prioritized": 0, 2024-09-13T18:34:33.897 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_started": 0, 2024-09-13T18:34:33.897 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_completed": 0, 2024-09-13T18:34:33.897 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_enqueue_scrub": 0, 2024-09-13T18:34:33.897 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_exportdir": 0, 2024-09-13T18:34:33.897 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_flush": 0, 2024-09-13T18:34:33.897 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragmentdir": 0, 2024-09-13T18:34:33.897 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragstats": 0, 2024-09-13T18:34:33.897 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_inodestats": 0 2024-09-13T18:34:33.897 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.898 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_log": { 2024-09-13T18:34:33.898 INFO:teuthology.orchestra.run.smithi134.stdout: "evadd": 2, 2024-09-13T18:34:33.898 INFO:teuthology.orchestra.run.smithi134.stdout: "evex": 0, 2024-09-13T18:34:33.898 INFO:teuthology.orchestra.run.smithi134.stdout: "evtrm": 0, 2024-09-13T18:34:33.898 INFO:teuthology.orchestra.run.smithi134.stdout: "ev": 2, 2024-09-13T18:34:33.898 INFO:teuthology.orchestra.run.smithi134.stdout: "evexg": 0, 2024-09-13T18:34:33.898 INFO:teuthology.orchestra.run.smithi134.stdout: "evexd": 0, 2024-09-13T18:34:33.898 INFO:teuthology.orchestra.run.smithi134.stdout: "segadd": 1, 2024-09-13T18:34:33.898 INFO:teuthology.orchestra.run.smithi134.stdout: "segex": 0, 2024-09-13T18:34:33.898 INFO:teuthology.orchestra.run.smithi134.stdout: "segtrm": 0, 2024-09-13T18:34:33.898 INFO:teuthology.orchestra.run.smithi134.stdout: "seg": 1, 2024-09-13T18:34:33.898 INFO:teuthology.orchestra.run.smithi134.stdout: "segexg": 0, 2024-09-13T18:34:33.898 INFO:teuthology.orchestra.run.smithi134.stdout: "segexd": 0, 2024-09-13T18:34:33.898 INFO:teuthology.orchestra.run.smithi134.stdout: "expos": 4194304, 2024-09-13T18:34:33.898 INFO:teuthology.orchestra.run.smithi134.stdout: "wrpos": 4195338, 2024-09-13T18:34:33.899 INFO:teuthology.orchestra.run.smithi134.stdout: "rdpos": 0, 2024-09-13T18:34:33.899 INFO:teuthology.orchestra.run.smithi134.stdout: "jlat": { 2024-09-13T18:34:33.899 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2, 2024-09-13T18:34:33.899 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.003202379, 2024-09-13T18:34:33.899 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.001601189 2024-09-13T18:34:33.899 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.899 INFO:teuthology.orchestra.run.smithi134.stdout: "replayed": 0 2024-09-13T18:34:33.899 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.899 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_mem": { 2024-09-13T18:34:33.899 INFO:teuthology.orchestra.run.smithi134.stdout: "ino": 22, 2024-09-13T18:34:33.899 INFO:teuthology.orchestra.run.smithi134.stdout: "ino+": 26, 2024-09-13T18:34:33.899 INFO:teuthology.orchestra.run.smithi134.stdout: "ino-": 4, 2024-09-13T18:34:33.899 INFO:teuthology.orchestra.run.smithi134.stdout: "dir": 20, 2024-09-13T18:34:33.900 INFO:teuthology.orchestra.run.smithi134.stdout: "dir+": 34, 2024-09-13T18:34:33.900 INFO:teuthology.orchestra.run.smithi134.stdout: "dir-": 14, 2024-09-13T18:34:33.900 INFO:teuthology.orchestra.run.smithi134.stdout: "dn": 17, 2024-09-13T18:34:33.900 INFO:teuthology.orchestra.run.smithi134.stdout: "dn+": 25, 2024-09-13T18:34:33.900 INFO:teuthology.orchestra.run.smithi134.stdout: "dn-": 8, 2024-09-13T18:34:33.900 INFO:teuthology.orchestra.run.smithi134.stdout: "cap": 0, 2024-09-13T18:34:33.900 INFO:teuthology.orchestra.run.smithi134.stdout: "cap+": 0, 2024-09-13T18:34:33.900 INFO:teuthology.orchestra.run.smithi134.stdout: "cap-": 0, 2024-09-13T18:34:33.900 INFO:teuthology.orchestra.run.smithi134.stdout: "rss": 49744, 2024-09-13T18:34:33.900 INFO:teuthology.orchestra.run.smithi134.stdout: "heap": 198932 2024-09-13T18:34:33.900 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.900 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_server": { 2024-09-13T18:34:33.900 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_client_request": 142, 2024-09-13T18:34:33.900 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_server_request": 2, 2024-09-13T18:34:33.901 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_request": 142, 2024-09-13T18:34:33.901 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_session": 10, 2024-09-13T18:34:33.901 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_peer_request": 4, 2024-09-13T18:34:33.901 INFO:teuthology.orchestra.run.smithi134.stdout: "req_create_latency": { 2024-09-13T18:34:33.901 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:33.901 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:33.901 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:33.901 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.901 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getattr_latency": { 2024-09-13T18:34:33.901 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:33.901 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:33.901 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:33.901 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.901 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getfilelock_latency": { 2024-09-13T18:34:33.902 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:33.902 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:33.902 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:33.902 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.902 INFO:teuthology.orchestra.run.smithi134.stdout: "req_link_latency": { 2024-09-13T18:34:33.902 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:33.902 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:33.902 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:33.902 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.902 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookup_latency": { 2024-09-13T18:34:33.902 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:33.902 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:33.902 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:33.902 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.903 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookuphash_latency": { 2024-09-13T18:34:33.903 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:33.903 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:33.903 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:33.903 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.903 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupino_latency": { 2024-09-13T18:34:33.903 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:33.903 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:33.903 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:33.903 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.903 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupname_latency": { 2024-09-13T18:34:33.903 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:33.903 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:33.903 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:33.904 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.904 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupparent_latency": { 2024-09-13T18:34:33.904 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:33.904 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:33.904 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:33.904 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.904 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupsnap_latency": { 2024-09-13T18:34:33.904 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:33.904 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:33.904 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:33.904 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.904 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lssnap_latency": { 2024-09-13T18:34:33.904 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:33.904 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:33.905 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:33.905 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.905 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mkdir_latency": { 2024-09-13T18:34:33.905 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:33.905 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:33.905 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:33.905 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.906 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mknod_latency": { 2024-09-13T18:34:33.906 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:33.906 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:33.906 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:33.906 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.906 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mksnap_latency": { 2024-09-13T18:34:33.906 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:33.906 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:33.907 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:33.907 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.907 INFO:teuthology.orchestra.run.smithi134.stdout: "req_open_latency": { 2024-09-13T18:34:33.907 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:33.907 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:33.907 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:33.907 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.907 INFO:teuthology.orchestra.run.smithi134.stdout: "req_readdir_latency": { 2024-09-13T18:34:33.907 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:33.907 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:33.907 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:33.907 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.907 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rename_latency": { 2024-09-13T18:34:33.907 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:33.908 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:33.908 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:33.908 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.908 INFO:teuthology.orchestra.run.smithi134.stdout: "req_renamesnap_latency": { 2024-09-13T18:34:33.908 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:33.908 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:33.908 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:33.908 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.908 INFO:teuthology.orchestra.run.smithi134.stdout: "req_snapdiff_latency": { 2024-09-13T18:34:33.908 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:33.908 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:33.908 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:33.908 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.909 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmdir_latency": { 2024-09-13T18:34:33.909 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:33.909 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:33.909 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:33.909 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.909 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmsnap_latency": { 2024-09-13T18:34:33.909 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:33.909 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:33.909 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:33.909 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.909 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmxattr_latency": { 2024-09-13T18:34:33.909 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:33.909 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:33.909 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:33.910 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.910 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setattr_latency": { 2024-09-13T18:34:33.910 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:33.910 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:33.910 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:33.910 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.910 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setdirlayout_latency": { 2024-09-13T18:34:33.910 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:33.910 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:33.910 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:33.910 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.910 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setfilelock_latency": { 2024-09-13T18:34:33.910 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:33.910 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:33.911 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:33.911 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.911 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setlayout_latency": { 2024-09-13T18:34:33.911 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:33.911 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:33.911 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:33.911 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.911 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setxattr_latency": { 2024-09-13T18:34:33.911 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:33.911 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:33.911 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:33.911 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.911 INFO:teuthology.orchestra.run.smithi134.stdout: "req_symlink_latency": { 2024-09-13T18:34:33.911 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:33.912 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:33.912 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:33.912 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.912 INFO:teuthology.orchestra.run.smithi134.stdout: "req_unlink_latency": { 2024-09-13T18:34:33.912 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:33.912 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:33.912 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:33.912 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.912 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_revoke_eviction": 0, 2024-09-13T18:34:33.912 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_acquisition_throttle": 0, 2024-09-13T18:34:33.912 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getvxattr_latency": { 2024-09-13T18:34:33.912 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:33.912 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:33.913 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:33.913 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-13T18:34:33.913 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.913 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_sessions": { 2024-09-13T18:34:33.913 INFO:teuthology.orchestra.run.smithi134.stdout: "session_count": 1, 2024-09-13T18:34:33.913 INFO:teuthology.orchestra.run.smithi134.stdout: "session_add": 1, 2024-09-13T18:34:33.913 INFO:teuthology.orchestra.run.smithi134.stdout: "session_remove": 0, 2024-09-13T18:34:33.913 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_open": 1, 2024-09-13T18:34:33.913 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_stale": 0, 2024-09-13T18:34:33.913 INFO:teuthology.orchestra.run.smithi134.stdout: "total_load": 0, 2024-09-13T18:34:33.913 INFO:teuthology.orchestra.run.smithi134.stdout: "average_load": 0, 2024-09-13T18:34:33.913 INFO:teuthology.orchestra.run.smithi134.stdout: "avg_session_uptime": 725, 2024-09-13T18:34:33.913 INFO:teuthology.orchestra.run.smithi134.stdout: "mdthresh_evicted": 0 2024-09-13T18:34:33.913 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.914 INFO:teuthology.orchestra.run.smithi134.stdout: "mempool": { 2024-09-13T18:34:33.914 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_bytes": 0, 2024-09-13T18:34:33.914 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_items": 0, 2024-09-13T18:34:33.914 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_bytes": 0, 2024-09-13T18:34:33.914 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_items": 0, 2024-09-13T18:34:33.914 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_bytes": 0, 2024-09-13T18:34:33.914 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_items": 0, 2024-09-13T18:34:33.914 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-13T18:34:33.914 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_items": 0, 2024-09-13T18:34:33.914 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-13T18:34:33.914 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_items": 0, 2024-09-13T18:34:33.914 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_bytes": 0, 2024-09-13T18:34:33.914 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_items": 0, 2024-09-13T18:34:33.914 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-13T18:34:33.915 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_items": 0, 2024-09-13T18:34:33.915 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_bytes": 0, 2024-09-13T18:34:33.915 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_items": 0, 2024-09-13T18:34:33.915 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_bytes": 0, 2024-09-13T18:34:33.915 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_items": 0, 2024-09-13T18:34:33.915 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-13T18:34:33.915 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_items": 0, 2024-09-13T18:34:33.915 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-13T18:34:33.915 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_items": 0, 2024-09-13T18:34:33.915 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_bytes": 0, 2024-09-13T18:34:33.915 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_items": 0, 2024-09-13T18:34:33.915 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_bytes": 0, 2024-09-13T18:34:33.915 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_items": 0, 2024-09-13T18:34:33.915 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-13T18:34:33.916 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_items": 0, 2024-09-13T18:34:33.916 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_bytes": 0, 2024-09-13T18:34:33.916 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_items": 0, 2024-09-13T18:34:33.916 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_bytes": 0, 2024-09-13T18:34:33.916 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_items": 0, 2024-09-13T18:34:33.916 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_bytes": 0, 2024-09-13T18:34:33.916 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_items": 0, 2024-09-13T18:34:33.916 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_bytes": 0, 2024-09-13T18:34:33.916 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_items": 0, 2024-09-13T18:34:33.916 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_bytes": 282147, 2024-09-13T18:34:33.916 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_items": 109, 2024-09-13T18:34:33.916 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_bytes": 0, 2024-09-13T18:34:33.916 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_items": 0, 2024-09-13T18:34:33.916 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_bytes": 0, 2024-09-13T18:34:33.917 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_items": 0, 2024-09-13T18:34:33.917 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_bytes": 0, 2024-09-13T18:34:33.917 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_items": 0, 2024-09-13T18:34:33.917 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_bytes": 0, 2024-09-13T18:34:33.917 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_items": 0, 2024-09-13T18:34:33.917 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_bytes": 9704, 2024-09-13T18:34:33.917 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_items": 196, 2024-09-13T18:34:33.917 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_bytes": 0, 2024-09-13T18:34:33.917 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_items": 0, 2024-09-13T18:34:33.917 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_bytes": 0, 2024-09-13T18:34:33.917 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_items": 0, 2024-09-13T18:34:33.917 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_bytes": 88653, 2024-09-13T18:34:33.917 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_items": 345, 2024-09-13T18:34:33.917 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_bytes": 0, 2024-09-13T18:34:33.918 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_items": 0, 2024-09-13T18:34:33.918 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_bytes": 0, 2024-09-13T18:34:33.918 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_items": 0 2024-09-13T18:34:33.918 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.918 INFO:teuthology.orchestra.run.smithi134.stdout: "objecter": { 2024-09-13T18:34:33.918 INFO:teuthology.orchestra.run.smithi134.stdout: "op_active": 0, 2024-09-13T18:34:33.918 INFO:teuthology.orchestra.run.smithi134.stdout: "op_laggy": 0, 2024-09-13T18:34:33.918 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send": 37, 2024-09-13T18:34:33.918 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send_bytes": 11246, 2024-09-13T18:34:33.918 INFO:teuthology.orchestra.run.smithi134.stdout: "op_resend": 33, 2024-09-13T18:34:33.919 INFO:teuthology.orchestra.run.smithi134.stdout: "op_reply": 37, 2024-09-13T18:34:33.919 INFO:teuthology.orchestra.run.smithi134.stdout: "op_latency": { 2024-09-13T18:34:33.919 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 37, 2024-09-13T18:34:33.919 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.205994915, 2024-09-13T18:34:33.919 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.005567430 2024-09-13T18:34:33.919 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.919 INFO:teuthology.orchestra.run.smithi134.stdout: "op_inflight": 0, 2024-09-13T18:34:33.919 INFO:teuthology.orchestra.run.smithi134.stdout: "oplen_avg": { 2024-09-13T18:34:33.919 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 37, 2024-09-13T18:34:33.919 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 59 2024-09-13T18:34:33.919 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.919 INFO:teuthology.orchestra.run.smithi134.stdout: "op": 37, 2024-09-13T18:34:33.919 INFO:teuthology.orchestra.run.smithi134.stdout: "op_r": 0, 2024-09-13T18:34:33.920 INFO:teuthology.orchestra.run.smithi134.stdout: "op_w": 37, 2024-09-13T18:34:33.920 INFO:teuthology.orchestra.run.smithi134.stdout: "op_rmw": 0, 2024-09-13T18:34:33.920 INFO:teuthology.orchestra.run.smithi134.stdout: "op_pg": 0, 2024-09-13T18:34:33.920 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_stat": 0, 2024-09-13T18:34:33.920 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_create": 10, 2024-09-13T18:34:33.920 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_read": 0, 2024-09-13T18:34:33.920 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_write": 2, 2024-09-13T18:34:33.920 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writefull": 6, 2024-09-13T18:34:33.920 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writesame": 0, 2024-09-13T18:34:33.920 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_append": 0, 2024-09-13T18:34:33.920 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_zero": 0, 2024-09-13T18:34:33.920 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_truncate": 0, 2024-09-13T18:34:33.920 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_delete": 6, 2024-09-13T18:34:33.921 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_mapext": 0, 2024-09-13T18:34:33.921 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_sparse_read": 0, 2024-09-13T18:34:33.921 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_clonerange": 0, 2024-09-13T18:34:33.921 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_getxattr": 0, 2024-09-13T18:34:33.921 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_setxattr": 20, 2024-09-13T18:34:33.921 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_cmpxattr": 0, 2024-09-13T18:34:33.921 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_rmxattr": 0, 2024-09-13T18:34:33.921 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_resetxattrs": 0, 2024-09-13T18:34:33.921 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_call": 0, 2024-09-13T18:34:33.921 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_watch": 0, 2024-09-13T18:34:33.921 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_notify": 0, 2024-09-13T18:34:33.921 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_src_cmpxattr": 0, 2024-09-13T18:34:33.921 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls": 0, 2024-09-13T18:34:33.921 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls_filter": 0, 2024-09-13T18:34:33.922 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_other": 0, 2024-09-13T18:34:33.922 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_active": 0, 2024-09-13T18:34:33.922 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_send": 0, 2024-09-13T18:34:33.922 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_resend": 0, 2024-09-13T18:34:33.922 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_ping": 0, 2024-09-13T18:34:33.922 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_active": 0, 2024-09-13T18:34:33.922 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_send": 0, 2024-09-13T18:34:33.922 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_resend": 0, 2024-09-13T18:34:33.922 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_active": 0, 2024-09-13T18:34:33.922 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_send": 0, 2024-09-13T18:34:33.922 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_resend": 0, 2024-09-13T18:34:33.922 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_active": 0, 2024-09-13T18:34:33.922 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_send": 0, 2024-09-13T18:34:33.923 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_resend": 0, 2024-09-13T18:34:33.923 INFO:teuthology.orchestra.run.smithi134.stdout: "command_active": 0, 2024-09-13T18:34:33.923 INFO:teuthology.orchestra.run.smithi134.stdout: "command_send": 0, 2024-09-13T18:34:33.923 INFO:teuthology.orchestra.run.smithi134.stdout: "command_resend": 0, 2024-09-13T18:34:33.923 INFO:teuthology.orchestra.run.smithi134.stdout: "map_epoch": 106, 2024-09-13T18:34:33.923 INFO:teuthology.orchestra.run.smithi134.stdout: "map_full": 0, 2024-09-13T18:34:33.923 INFO:teuthology.orchestra.run.smithi134.stdout: "map_inc": 15, 2024-09-13T18:34:33.923 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_sessions": 9, 2024-09-13T18:34:33.923 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_open": 9, 2024-09-13T18:34:33.923 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_close": 0, 2024-09-13T18:34:33.923 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_laggy": 0, 2024-09-13T18:34:33.923 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_wr": 14, 2024-09-13T18:34:33.923 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_rd": 0, 2024-09-13T18:34:33.923 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_del": 1 2024-09-13T18:34:33.924 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.924 INFO:teuthology.orchestra.run.smithi134.stdout: "oft": { 2024-09-13T18:34:33.924 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_objs": 1, 2024-09-13T18:34:33.924 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_kv_pairs": 0, 2024-09-13T18:34:33.924 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_updates": 0, 2024-09-13T18:34:33.924 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_removes": 0 2024-09-13T18:34:33.924 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.924 INFO:teuthology.orchestra.run.smithi134.stdout: "osd-slow-ops": { 2024-09-13T18:34:33.924 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_ops_count": 0 2024-09-13T18:34:33.924 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.924 INFO:teuthology.orchestra.run.smithi134.stdout: "purge_queue": { 2024-09-13T18:34:33.924 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops": 0, 2024-09-13T18:34:33.924 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops_high_water": 0, 2024-09-13T18:34:33.925 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing": 0, 2024-09-13T18:34:33.925 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_high_water": 0, 2024-09-13T18:34:33.925 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed_ops": 0, 2024-09-13T18:34:33.925 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed": 0, 2024-09-13T18:34:33.925 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_item_in_journal": 0 2024-09-13T18:34:33.925 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.925 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-13T18:34:33.925 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-13T18:34:33.925 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-09-13T18:34:33.925 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-13T18:34:33.925 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 1726, 2024-09-13T18:34:33.925 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 2119670, 2024-09-13T18:34:33.925 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-13T18:34:33.925 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 1726, 2024-09-13T18:34:33.926 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-13T18:34:33.926 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-13T18:34:33.926 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 1726, 2024-09-13T18:34:33.926 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 2119670, 2024-09-13T18:34:33.926 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-13T18:34:33.926 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:33.926 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:33.926 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:33.926 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-13T18:34:33.926 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.926 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_bytes": { 2024-09-13T18:34:33.926 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-13T18:34:33.926 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-09-13T18:34:33.926 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-13T18:34:33.927 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-13T18:34:33.927 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-13T18:34:33.927 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-13T18:34:33.927 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-13T18:34:33.927 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 37, 2024-09-13T18:34:33.927 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 11246, 2024-09-13T18:34:33.927 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 37, 2024-09-13T18:34:33.927 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 11246, 2024-09-13T18:34:33.927 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-13T18:34:33.927 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:33.927 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:33.927 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:33.927 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-13T18:34:33.927 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.928 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_ops": { 2024-09-13T18:34:33.928 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-13T18:34:33.928 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 1024, 2024-09-13T18:34:33.928 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-13T18:34:33.928 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-13T18:34:33.928 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-13T18:34:33.928 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-13T18:34:33.928 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-13T18:34:33.928 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 37, 2024-09-13T18:34:33.928 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 37, 2024-09-13T18:34:33.928 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 37, 2024-09-13T18:34:33.928 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 37, 2024-09-13T18:34:33.928 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-13T18:34:33.929 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:33.929 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:33.929 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:33.929 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-13T18:34:33.929 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.929 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle": { 2024-09-13T18:34:33.929 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-13T18:34:33.929 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-09-13T18:34:33.929 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-13T18:34:33.929 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-13T18:34:33.929 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-13T18:34:33.929 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-13T18:34:33.929 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-13T18:34:33.929 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-13T18:34:33.930 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-13T18:34:33.930 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 0, 2024-09-13T18:34:33.930 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 0, 2024-09-13T18:34:33.930 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-13T18:34:33.930 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:33.930 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:33.930 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:33.930 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-13T18:34:33.930 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:33.930 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle-0x5617c549e500": { 2024-09-13T18:34:33.930 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-13T18:34:33.930 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-09-13T18:34:33.930 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-13T18:34:33.931 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 2, 2024-09-13T18:34:33.931 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 1034, 2024-09-13T18:34:33.931 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-13T18:34:33.931 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 2, 2024-09-13T18:34:33.931 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-13T18:34:33.931 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-13T18:34:33.931 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 2, 2024-09-13T18:34:33.931 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 1034, 2024-09-13T18:34:33.931 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-13T18:34:33.931 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:33.931 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:33.931 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:33.931 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-13T18:34:33.931 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-13T18:34:33.931 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-13T18:34:33.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:33 smithi204 ceph-mon[34777]: pgmap v377: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-13T18:34:33.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:33 smithi114 ceph-mon[30897]: pgmap v377: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-13T18:34:34.890 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-13T18:34:34.890 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-13T18:34:34.890 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-09-13T18:34:34.891 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=14 2024-09-13T18:34:34.891 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-09-13T18:34:34.891 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-09-13T18:34:34.891 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-09-13T18:34:34.891 DEBUG:tasks.check_counter:Getting stats from h 2024-09-13T18:34:34.891 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-13T18:34:35.103 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T18:34:35.772 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:35 smithi114 ceph-mon[30897]: pgmap v378: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-13T18:34:35.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:35 smithi134 ceph-mon[34283]: pgmap v378: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-13T18:34:35.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:35 smithi204 ceph-mon[34777]: pgmap v378: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-13T18:34:36.853 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-13T18:34:36.853 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-0": { 2024-09-13T18:34:36.853 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 4796, 2024-09-13T18:34:36.853 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 3592, 2024-09-13T18:34:36.853 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 1388655, 2024-09-13T18:34:36.853 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 2871451, 2024-09-13T18:34:36.853 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 14, 2024-09-13T18:34:36.854 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 8, 2024-09-13T18:34:36.854 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.279337774, 2024-09-13T18:34:36.854 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.092992293, 2024-09-13T18:34:36.854 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.204531765, 2024-09-13T18:34:36.854 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.024111304, 2024-09-13T18:34:36.854 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T18:34:36.854 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 3588, 2024-09-13T18:34:36.854 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.204698967, 2024-09-13T18:34:36.854 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000057050 2024-09-13T18:34:36.854 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.854 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-13T18:34:36.854 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 6853, 2024-09-13T18:34:36.854 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.001076463, 2024-09-13T18:34:36.855 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000157 2024-09-13T18:34:36.855 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.855 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-13T18:34:36.855 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 343760 2024-09-13T18:34:36.855 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.855 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-1": { 2024-09-13T18:34:36.855 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 1756, 2024-09-13T18:34:36.855 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 1945, 2024-09-13T18:34:36.855 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 706196, 2024-09-13T18:34:36.855 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 1145432, 2024-09-13T18:34:36.855 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 14, 2024-09-13T18:34:36.855 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 5, 2024-09-13T18:34:36.855 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.150506384, 2024-09-13T18:34:36.855 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.061877450, 2024-09-13T18:34:36.856 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.078986536, 2024-09-13T18:34:36.856 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.017205019, 2024-09-13T18:34:36.856 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T18:34:36.856 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1947, 2024-09-13T18:34:36.856 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.127952568, 2024-09-13T18:34:36.856 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000065717 2024-09-13T18:34:36.856 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.856 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-13T18:34:36.856 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2669, 2024-09-13T18:34:36.856 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000538111, 2024-09-13T18:34:36.856 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000201 2024-09-13T18:34:36.856 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.857 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 4048, 2024-09-13T18:34:36.857 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 130688 2024-09-13T18:34:36.857 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.857 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-2": { 2024-09-13T18:34:36.857 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 3303, 2024-09-13T18:34:36.857 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 4141, 2024-09-13T18:34:36.857 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 2642395, 2024-09-13T18:34:36.857 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 2063062, 2024-09-13T18:34:36.857 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 50, 2024-09-13T18:34:36.857 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 7, 2024-09-13T18:34:36.857 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.282949042, 2024-09-13T18:34:36.857 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.095057270, 2024-09-13T18:34:36.857 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.186713226, 2024-09-13T18:34:36.858 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.030875825, 2024-09-13T18:34:36.858 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T18:34:36.858 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 4138, 2024-09-13T18:34:36.858 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.327848897, 2024-09-13T18:34:36.858 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000079228 2024-09-13T18:34:36.858 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.858 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-13T18:34:36.858 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 3783, 2024-09-13T18:34:36.858 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000858699, 2024-09-13T18:34:36.858 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000226 2024-09-13T18:34:36.858 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.858 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 1698304, 2024-09-13T18:34:36.858 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 171536 2024-09-13T18:34:36.859 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.859 INFO:teuthology.orchestra.run.smithi134.stdout: "cct": { 2024-09-13T18:34:36.859 INFO:teuthology.orchestra.run.smithi134.stdout: "total_workers": 1, 2024-09-13T18:34:36.859 INFO:teuthology.orchestra.run.smithi134.stdout: "unhealthy_workers": 0 2024-09-13T18:34:36.859 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.859 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-MDSRank": { 2024-09-13T18:34:36.859 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-09-13T18:34:36.859 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-09-13T18:34:36.859 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 540, 2024-09-13T18:34:36.859 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.167783475, 2024-09-13T18:34:36.859 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000310710 2024-09-13T18:34:36.859 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-13T18:34:36.859 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.859 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-PurgeQueue": { 2024-09-13T18:34:36.860 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-09-13T18:34:36.860 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-09-13T18:34:36.860 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1502, 2024-09-13T18:34:36.860 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.058038734, 2024-09-13T18:34:36.860 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000038640 2024-09-13T18:34:36.860 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-13T18:34:36.860 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.860 INFO:teuthology.orchestra.run.smithi134.stdout: "mds": { 2024-09-13T18:34:36.860 INFO:teuthology.orchestra.run.smithi134.stdout: "request": 295, 2024-09-13T18:34:36.860 INFO:teuthology.orchestra.run.smithi134.stdout: "reply": 291, 2024-09-13T18:34:36.860 INFO:teuthology.orchestra.run.smithi134.stdout: "reply_latency": { 2024-09-13T18:34:36.860 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 291, 2024-09-13T18:34:36.860 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 2.356288477, 2024-09-13T18:34:36.861 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.008097211 2024-09-13T18:34:36.861 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.861 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_reply": 0, 2024-09-13T18:34:36.861 INFO:teuthology.orchestra.run.smithi134.stdout: "forward": 4, 2024-09-13T18:34:36.861 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_complete": 1, 2024-09-13T18:34:36.861 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_keys": 0, 2024-09-13T18:34:36.861 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_commit": 19, 2024-09-13T18:34:36.861 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_split": 1, 2024-09-13T18:34:36.861 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_merge": 0, 2024-09-13T18:34:36.861 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes": 142, 2024-09-13T18:34:36.861 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_top": 57, 2024-09-13T18:34:36.861 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_bottom": 70, 2024-09-13T18:34:36.861 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pin_tail": 15, 2024-09-13T18:34:36.862 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pinned": 59, 2024-09-13T18:34:36.862 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_expired": 52, 2024-09-13T18:34:36.862 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_with_caps": 2, 2024-09-13T18:34:36.862 INFO:teuthology.orchestra.run.smithi134.stdout: "caps": 3, 2024-09-13T18:34:36.862 INFO:teuthology.orchestra.run.smithi134.stdout: "subtrees": 4, 2024-09-13T18:34:36.862 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse": 580, 2024-09-13T18:34:36.862 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_hit": 561, 2024-09-13T18:34:36.862 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_forward": 1, 2024-09-13T18:34:36.862 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_discover": 4, 2024-09-13T18:34:36.862 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_dir_fetch": 0, 2024-09-13T18:34:36.862 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_remote_ino": 0, 2024-09-13T18:34:36.862 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_lock": 0, 2024-09-13T18:34:36.862 INFO:teuthology.orchestra.run.smithi134.stdout: "load_cent": 0, 2024-09-13T18:34:36.862 INFO:teuthology.orchestra.run.smithi134.stdout: "q": 0, 2024-09-13T18:34:36.863 INFO:teuthology.orchestra.run.smithi134.stdout: "exported": 3, 2024-09-13T18:34:36.863 INFO:teuthology.orchestra.run.smithi134.stdout: "exported_inodes": 44, 2024-09-13T18:34:36.863 INFO:teuthology.orchestra.run.smithi134.stdout: "imported": 4, 2024-09-13T18:34:36.863 INFO:teuthology.orchestra.run.smithi134.stdout: "imported_inodes": 73, 2024-09-13T18:34:36.863 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_dir_fetch": 0, 2024-09-13T18:34:36.863 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_backtrace_fetch": 300, 2024-09-13T18:34:36.863 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_peer_discover": 0, 2024-09-13T18:34:36.863 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rfiles": 22, 2024-09-13T18:34:36.863 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rbytes": 10738143812, 2024-09-13T18:34:36.863 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rsnaps": 6, 2024-09-13T18:34:36.863 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_fetch": 300, 2024-09-13T18:34:36.863 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_set_tag": 0, 2024-09-13T18:34:36.863 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_repaired": 0, 2024-09-13T18:34:36.863 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_inotable_repaired": 0, 2024-09-13T18:34:36.864 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_inodes": 30, 2024-09-13T18:34:36.864 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_base_inodes": 0, 2024-09-13T18:34:36.864 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dirfrag_rstats": 30, 2024-09-13T18:34:36.864 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_file_inodes": 270, 2024-09-13T18:34:36.864 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_inode_file_caps": 2, 2024-09-13T18:34:36.864 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_revoke": 1123, 2024-09-13T18:34:36.864 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_grant": 1077, 2024-09-13T18:34:36.864 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_trunc": 0, 2024-09-13T18:34:36.864 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-13T18:34:36.864 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-13T18:34:36.864 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps": 1600, 2024-09-13T18:34:36.864 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps_dirty": 147, 2024-09-13T18:34:36.864 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_cap_release": 2, 2024-09-13T18:34:36.864 INFO:teuthology.orchestra.run.smithi134.stdout: "process_request_cap_release": 34 2024-09-13T18:34:36.865 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.865 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_cache": { 2024-09-13T18:34:36.865 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update": 3, 2024-09-13T18:34:36.865 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update_receipt": 14, 2024-09-13T18:34:36.865 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_try_discover": 9, 2024-09-13T18:34:36.865 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_send_discover": 10, 2024-09-13T18:34:36.865 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_handle_discover": 4, 2024-09-13T18:34:36.865 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays": 2, 2024-09-13T18:34:36.865 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_delayed": 0, 2024-09-13T18:34:36.865 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_enqueuing": 0, 2024-09-13T18:34:36.865 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_created": 17, 2024-09-13T18:34:36.865 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_enqueued": 15, 2024-09-13T18:34:36.865 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_reintegrated": 0, 2024-09-13T18:34:36.865 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_migrated": 0, 2024-09-13T18:34:36.866 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_processing": 0, 2024-09-13T18:34:36.866 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_enqueued": 0, 2024-09-13T18:34:36.866 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_prioritized": 0, 2024-09-13T18:34:36.866 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_started": 0, 2024-09-13T18:34:36.866 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_completed": 0, 2024-09-13T18:34:36.866 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_enqueue_scrub": 0, 2024-09-13T18:34:36.866 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_exportdir": 3, 2024-09-13T18:34:36.866 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_flush": 0, 2024-09-13T18:34:36.866 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragmentdir": 1, 2024-09-13T18:34:36.866 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragstats": 0, 2024-09-13T18:34:36.866 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_inodestats": 0 2024-09-13T18:34:36.866 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.866 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_log": { 2024-09-13T18:34:36.866 INFO:teuthology.orchestra.run.smithi134.stdout: "evadd": 276, 2024-09-13T18:34:36.867 INFO:teuthology.orchestra.run.smithi134.stdout: "evex": 0, 2024-09-13T18:34:36.867 INFO:teuthology.orchestra.run.smithi134.stdout: "evtrm": 0, 2024-09-13T18:34:36.867 INFO:teuthology.orchestra.run.smithi134.stdout: "ev": 276, 2024-09-13T18:34:36.867 INFO:teuthology.orchestra.run.smithi134.stdout: "evexg": 0, 2024-09-13T18:34:36.867 INFO:teuthology.orchestra.run.smithi134.stdout: "evexd": 0, 2024-09-13T18:34:36.867 INFO:teuthology.orchestra.run.smithi134.stdout: "segadd": 1, 2024-09-13T18:34:36.867 INFO:teuthology.orchestra.run.smithi134.stdout: "segex": 0, 2024-09-13T18:34:36.867 INFO:teuthology.orchestra.run.smithi134.stdout: "segtrm": 0, 2024-09-13T18:34:36.867 INFO:teuthology.orchestra.run.smithi134.stdout: "seg": 1, 2024-09-13T18:34:36.867 INFO:teuthology.orchestra.run.smithi134.stdout: "segexg": 0, 2024-09-13T18:34:36.867 INFO:teuthology.orchestra.run.smithi134.stdout: "segexd": 0, 2024-09-13T18:34:36.867 INFO:teuthology.orchestra.run.smithi134.stdout: "expos": 4194304, 2024-09-13T18:34:36.867 INFO:teuthology.orchestra.run.smithi134.stdout: "wrpos": 5735562, 2024-09-13T18:34:36.868 INFO:teuthology.orchestra.run.smithi134.stdout: "rdpos": 0, 2024-09-13T18:34:36.868 INFO:teuthology.orchestra.run.smithi134.stdout: "jlat": { 2024-09-13T18:34:36.868 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 125, 2024-09-13T18:34:36.868 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.423894442, 2024-09-13T18:34:36.868 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.003391155 2024-09-13T18:34:36.868 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.868 INFO:teuthology.orchestra.run.smithi134.stdout: "replayed": 0 2024-09-13T18:34:36.868 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.868 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_mem": { 2024-09-13T18:34:36.868 INFO:teuthology.orchestra.run.smithi134.stdout: "ino": 128, 2024-09-13T18:34:36.868 INFO:teuthology.orchestra.run.smithi134.stdout: "ino+": 189, 2024-09-13T18:34:36.868 INFO:teuthology.orchestra.run.smithi134.stdout: "ino-": 61, 2024-09-13T18:34:36.868 INFO:teuthology.orchestra.run.smithi134.stdout: "dir": 29, 2024-09-13T18:34:36.868 INFO:teuthology.orchestra.run.smithi134.stdout: "dir+": 34, 2024-09-13T18:34:36.869 INFO:teuthology.orchestra.run.smithi134.stdout: "dir-": 5, 2024-09-13T18:34:36.869 INFO:teuthology.orchestra.run.smithi134.stdout: "dn": 142, 2024-09-13T18:34:36.869 INFO:teuthology.orchestra.run.smithi134.stdout: "dn+": 209, 2024-09-13T18:34:36.869 INFO:teuthology.orchestra.run.smithi134.stdout: "dn-": 67, 2024-09-13T18:34:36.869 INFO:teuthology.orchestra.run.smithi134.stdout: "cap": 3, 2024-09-13T18:34:36.869 INFO:teuthology.orchestra.run.smithi134.stdout: "cap+": 26, 2024-09-13T18:34:36.869 INFO:teuthology.orchestra.run.smithi134.stdout: "cap-": 23, 2024-09-13T18:34:36.869 INFO:teuthology.orchestra.run.smithi134.stdout: "rss": 56584, 2024-09-13T18:34:36.869 INFO:teuthology.orchestra.run.smithi134.stdout: "heap": 198932 2024-09-13T18:34:36.869 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.869 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_server": { 2024-09-13T18:34:36.869 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_client_request": 781, 2024-09-13T18:34:36.869 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_server_request": 24, 2024-09-13T18:34:36.869 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_request": 295, 2024-09-13T18:34:36.870 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_session": 25, 2024-09-13T18:34:36.870 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_peer_request": 177, 2024-09-13T18:34:36.870 INFO:teuthology.orchestra.run.smithi134.stdout: "req_create_latency": { 2024-09-13T18:34:36.870 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:36.870 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:36.870 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:36.870 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.870 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getattr_latency": { 2024-09-13T18:34:36.870 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 59, 2024-09-13T18:34:36.870 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.070205801, 2024-09-13T18:34:36.870 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.001189928 2024-09-13T18:34:36.870 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.870 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getfilelock_latency": { 2024-09-13T18:34:36.870 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:36.870 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:36.871 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:36.871 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.871 INFO:teuthology.orchestra.run.smithi134.stdout: "req_link_latency": { 2024-09-13T18:34:36.871 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:36.871 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:36.871 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:36.871 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.871 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookup_latency": { 2024-09-13T18:34:36.871 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 205, 2024-09-13T18:34:36.871 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 2.242917938, 2024-09-13T18:34:36.871 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.010941063 2024-09-13T18:34:36.871 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.871 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookuphash_latency": { 2024-09-13T18:34:36.871 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:36.872 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:36.872 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:36.872 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.872 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupino_latency": { 2024-09-13T18:34:36.872 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:36.872 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:36.872 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:36.872 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.872 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupname_latency": { 2024-09-13T18:34:36.872 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:36.872 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:36.872 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:36.872 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.873 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupparent_latency": { 2024-09-13T18:34:36.873 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:36.873 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:36.873 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:36.873 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.873 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupsnap_latency": { 2024-09-13T18:34:36.873 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:36.873 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:36.873 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:36.873 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.873 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lssnap_latency": { 2024-09-13T18:34:36.873 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:36.873 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:36.873 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:36.874 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.874 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mkdir_latency": { 2024-09-13T18:34:36.874 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:36.874 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:36.874 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:36.874 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.874 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mknod_latency": { 2024-09-13T18:34:36.874 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:36.874 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:36.874 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:36.874 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.874 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mksnap_latency": { 2024-09-13T18:34:36.874 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:36.875 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:36.875 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:36.875 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.875 INFO:teuthology.orchestra.run.smithi134.stdout: "req_open_latency": { 2024-09-13T18:34:36.875 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:36.875 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:36.875 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:36.875 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.875 INFO:teuthology.orchestra.run.smithi134.stdout: "req_readdir_latency": { 2024-09-13T18:34:36.875 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 10, 2024-09-13T18:34:36.876 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.002613930, 2024-09-13T18:34:36.876 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000261393 2024-09-13T18:34:36.876 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.876 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rename_latency": { 2024-09-13T18:34:36.876 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:36.876 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:36.876 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:36.876 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.876 INFO:teuthology.orchestra.run.smithi134.stdout: "req_renamesnap_latency": { 2024-09-13T18:34:36.876 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:36.876 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:36.876 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:36.876 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.877 INFO:teuthology.orchestra.run.smithi134.stdout: "req_snapdiff_latency": { 2024-09-13T18:34:36.877 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:36.877 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:36.877 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:36.877 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.877 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmdir_latency": { 2024-09-13T18:34:36.877 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2, 2024-09-13T18:34:36.877 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.013381099, 2024-09-13T18:34:36.877 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.006690549 2024-09-13T18:34:36.877 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.877 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmsnap_latency": { 2024-09-13T18:34:36.877 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:36.877 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:36.877 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:36.878 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.878 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmxattr_latency": { 2024-09-13T18:34:36.878 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:36.878 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:36.878 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:36.878 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.878 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setattr_latency": { 2024-09-13T18:34:36.878 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:36.878 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:36.878 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:36.878 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.878 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setdirlayout_latency": { 2024-09-13T18:34:36.878 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:36.878 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:36.879 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:36.879 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.879 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setfilelock_latency": { 2024-09-13T18:34:36.879 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:36.879 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:36.879 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:36.879 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.879 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setlayout_latency": { 2024-09-13T18:34:36.879 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:36.879 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:36.879 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:36.879 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.879 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setxattr_latency": { 2024-09-13T18:34:36.879 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:36.880 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:36.880 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:36.880 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.880 INFO:teuthology.orchestra.run.smithi134.stdout: "req_symlink_latency": { 2024-09-13T18:34:36.880 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:36.880 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:36.880 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:36.880 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.880 INFO:teuthology.orchestra.run.smithi134.stdout: "req_unlink_latency": { 2024-09-13T18:34:36.880 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 15, 2024-09-13T18:34:36.880 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.027169709, 2024-09-13T18:34:36.880 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.001811313 2024-09-13T18:34:36.880 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.881 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_revoke_eviction": 0, 2024-09-13T18:34:36.881 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_acquisition_throttle": 0, 2024-09-13T18:34:36.881 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getvxattr_latency": { 2024-09-13T18:34:36.881 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:36.881 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:36.881 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:36.881 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-13T18:34:36.881 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.881 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_sessions": { 2024-09-13T18:34:36.881 INFO:teuthology.orchestra.run.smithi134.stdout: "session_count": 2, 2024-09-13T18:34:36.881 INFO:teuthology.orchestra.run.smithi134.stdout: "session_add": 3, 2024-09-13T18:34:36.881 INFO:teuthology.orchestra.run.smithi134.stdout: "session_remove": 1, 2024-09-13T18:34:36.881 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_open": 2, 2024-09-13T18:34:36.881 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_stale": 0, 2024-09-13T18:34:36.882 INFO:teuthology.orchestra.run.smithi134.stdout: "total_load": 261, 2024-09-13T18:34:36.882 INFO:teuthology.orchestra.run.smithi134.stdout: "average_load": 130, 2024-09-13T18:34:36.882 INFO:teuthology.orchestra.run.smithi134.stdout: "avg_session_uptime": 711, 2024-09-13T18:34:36.882 INFO:teuthology.orchestra.run.smithi134.stdout: "mdthresh_evicted": 0 2024-09-13T18:34:36.882 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.882 INFO:teuthology.orchestra.run.smithi134.stdout: "mempool": { 2024-09-13T18:34:36.882 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_bytes": 0, 2024-09-13T18:34:36.882 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_items": 0, 2024-09-13T18:34:36.882 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_bytes": 0, 2024-09-13T18:34:36.882 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_items": 0, 2024-09-13T18:34:36.882 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_bytes": 0, 2024-09-13T18:34:36.882 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_items": 0, 2024-09-13T18:34:36.882 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-13T18:34:36.882 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_items": 0, 2024-09-13T18:34:36.883 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-13T18:34:36.883 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_items": 0, 2024-09-13T18:34:36.883 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_bytes": 0, 2024-09-13T18:34:36.883 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_items": 0, 2024-09-13T18:34:36.883 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-13T18:34:36.883 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_items": 0, 2024-09-13T18:34:36.883 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_bytes": 0, 2024-09-13T18:34:36.883 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_items": 0, 2024-09-13T18:34:36.883 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_bytes": 0, 2024-09-13T18:34:36.883 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_items": 0, 2024-09-13T18:34:36.883 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-13T18:34:36.883 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_items": 0, 2024-09-13T18:34:36.883 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-13T18:34:36.883 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_items": 0, 2024-09-13T18:34:36.884 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_bytes": 0, 2024-09-13T18:34:36.884 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_items": 0, 2024-09-13T18:34:36.884 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_bytes": 0, 2024-09-13T18:34:36.884 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_items": 0, 2024-09-13T18:34:36.884 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-13T18:34:36.884 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_items": 0, 2024-09-13T18:34:36.884 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_bytes": 0, 2024-09-13T18:34:36.884 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_items": 0, 2024-09-13T18:34:36.884 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_bytes": 0, 2024-09-13T18:34:36.884 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_items": 0, 2024-09-13T18:34:36.884 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_bytes": 0, 2024-09-13T18:34:36.884 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_items": 0, 2024-09-13T18:34:36.884 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_bytes": 0, 2024-09-13T18:34:36.884 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_items": 0, 2024-09-13T18:34:36.885 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_bytes": 323206, 2024-09-13T18:34:36.885 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_items": 128, 2024-09-13T18:34:36.885 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_bytes": 0, 2024-09-13T18:34:36.885 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_items": 0, 2024-09-13T18:34:36.885 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_bytes": 0, 2024-09-13T18:34:36.885 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_items": 0, 2024-09-13T18:34:36.885 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_bytes": 0, 2024-09-13T18:34:36.885 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_items": 0, 2024-09-13T18:34:36.885 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_bytes": 0, 2024-09-13T18:34:36.885 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_items": 0, 2024-09-13T18:34:36.885 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_bytes": 9704, 2024-09-13T18:34:36.885 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_items": 196, 2024-09-13T18:34:36.885 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_bytes": 0, 2024-09-13T18:34:36.885 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_items": 0, 2024-09-13T18:34:36.886 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_bytes": 0, 2024-09-13T18:34:36.886 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_items": 0, 2024-09-13T18:34:36.886 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_bytes": 425232, 2024-09-13T18:34:36.886 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_items": 2194, 2024-09-13T18:34:36.886 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_bytes": 0, 2024-09-13T18:34:36.886 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_items": 0, 2024-09-13T18:34:36.886 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_bytes": 0, 2024-09-13T18:34:36.886 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_items": 0 2024-09-13T18:34:36.886 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.886 INFO:teuthology.orchestra.run.smithi134.stdout: "objecter": { 2024-09-13T18:34:36.886 INFO:teuthology.orchestra.run.smithi134.stdout: "op_active": 0, 2024-09-13T18:34:36.886 INFO:teuthology.orchestra.run.smithi134.stdout: "op_laggy": 0, 2024-09-13T18:34:36.886 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send": 2080, 2024-09-13T18:34:36.886 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send_bytes": 1618167, 2024-09-13T18:34:36.886 INFO:teuthology.orchestra.run.smithi134.stdout: "op_resend": 27, 2024-09-13T18:34:36.887 INFO:teuthology.orchestra.run.smithi134.stdout: "op_reply": 2080, 2024-09-13T18:34:36.887 INFO:teuthology.orchestra.run.smithi134.stdout: "op_latency": { 2024-09-13T18:34:36.887 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2080, 2024-09-13T18:34:36.887 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 53.262755824, 2024-09-13T18:34:36.887 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.025607094 2024-09-13T18:34:36.887 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.887 INFO:teuthology.orchestra.run.smithi134.stdout: "op_inflight": 0, 2024-09-13T18:34:36.887 INFO:teuthology.orchestra.run.smithi134.stdout: "oplen_avg": { 2024-09-13T18:34:36.887 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2080, 2024-09-13T18:34:36.887 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 2121 2024-09-13T18:34:36.887 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.887 INFO:teuthology.orchestra.run.smithi134.stdout: "op": 2080, 2024-09-13T18:34:36.887 INFO:teuthology.orchestra.run.smithi134.stdout: "op_r": 304, 2024-09-13T18:34:36.887 INFO:teuthology.orchestra.run.smithi134.stdout: "op_w": 1776, 2024-09-13T18:34:36.888 INFO:teuthology.orchestra.run.smithi134.stdout: "op_rmw": 0, 2024-09-13T18:34:36.888 INFO:teuthology.orchestra.run.smithi134.stdout: "op_pg": 0, 2024-09-13T18:34:36.888 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_stat": 0, 2024-09-13T18:34:36.888 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_create": 10, 2024-09-13T18:34:36.888 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_read": 3, 2024-09-13T18:34:36.888 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_write": 128, 2024-09-13T18:34:36.888 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writefull": 31, 2024-09-13T18:34:36.888 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writesame": 0, 2024-09-13T18:34:36.888 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_append": 0, 2024-09-13T18:34:36.888 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_zero": 0, 2024-09-13T18:34:36.888 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_truncate": 1, 2024-09-13T18:34:36.888 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_delete": 1578, 2024-09-13T18:34:36.888 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_mapext": 0, 2024-09-13T18:34:36.889 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_sparse_read": 0, 2024-09-13T18:34:36.889 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_clonerange": 0, 2024-09-13T18:34:36.889 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_getxattr": 300, 2024-09-13T18:34:36.889 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_setxattr": 20, 2024-09-13T18:34:36.889 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_cmpxattr": 0, 2024-09-13T18:34:36.889 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_rmxattr": 0, 2024-09-13T18:34:36.889 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_resetxattrs": 0, 2024-09-13T18:34:36.889 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_call": 0, 2024-09-13T18:34:36.889 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_watch": 0, 2024-09-13T18:34:36.889 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_notify": 0, 2024-09-13T18:34:36.889 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_src_cmpxattr": 0, 2024-09-13T18:34:36.889 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls": 0, 2024-09-13T18:34:36.889 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls_filter": 0, 2024-09-13T18:34:36.889 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_other": 0, 2024-09-13T18:34:36.889 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_active": 0, 2024-09-13T18:34:36.890 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_send": 0, 2024-09-13T18:34:36.890 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_resend": 0, 2024-09-13T18:34:36.890 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_ping": 0, 2024-09-13T18:34:36.890 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_active": 0, 2024-09-13T18:34:36.890 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_send": 0, 2024-09-13T18:34:36.890 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_resend": 0, 2024-09-13T18:34:36.890 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_active": 0, 2024-09-13T18:34:36.890 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_send": 0, 2024-09-13T18:34:36.890 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_resend": 0, 2024-09-13T18:34:36.890 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_active": 0, 2024-09-13T18:34:36.890 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_send": 0, 2024-09-13T18:34:36.890 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_resend": 0, 2024-09-13T18:34:36.890 INFO:teuthology.orchestra.run.smithi134.stdout: "command_active": 0, 2024-09-13T18:34:36.890 INFO:teuthology.orchestra.run.smithi134.stdout: "command_send": 0, 2024-09-13T18:34:36.891 INFO:teuthology.orchestra.run.smithi134.stdout: "command_resend": 0, 2024-09-13T18:34:36.891 INFO:teuthology.orchestra.run.smithi134.stdout: "map_epoch": 106, 2024-09-13T18:34:36.891 INFO:teuthology.orchestra.run.smithi134.stdout: "map_full": 0, 2024-09-13T18:34:36.891 INFO:teuthology.orchestra.run.smithi134.stdout: "map_inc": 17, 2024-09-13T18:34:36.891 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_sessions": 12, 2024-09-13T18:34:36.891 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_open": 12, 2024-09-13T18:34:36.891 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_close": 0, 2024-09-13T18:34:36.891 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_laggy": 0, 2024-09-13T18:34:36.891 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_wr": 43, 2024-09-13T18:34:36.891 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_rd": 2, 2024-09-13T18:34:36.891 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_del": 5 2024-09-13T18:34:36.891 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.891 INFO:teuthology.orchestra.run.smithi134.stdout: "oft": { 2024-09-13T18:34:36.891 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_objs": 1, 2024-09-13T18:34:36.892 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_kv_pairs": 0, 2024-09-13T18:34:36.892 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_updates": 0, 2024-09-13T18:34:36.892 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_removes": 0 2024-09-13T18:34:36.892 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.892 INFO:teuthology.orchestra.run.smithi134.stdout: "osd-slow-ops": { 2024-09-13T18:34:36.892 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_ops_count": 1 2024-09-13T18:34:36.892 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.892 INFO:teuthology.orchestra.run.smithi134.stdout: "purge_queue": { 2024-09-13T18:34:36.892 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops": 0, 2024-09-13T18:34:36.892 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops_high_water": 522, 2024-09-13T18:34:36.892 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing": 0, 2024-09-13T18:34:36.892 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_high_water": 5, 2024-09-13T18:34:36.892 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed_ops": 1566, 2024-09-13T18:34:36.892 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed": 15, 2024-09-13T18:34:36.893 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_item_in_journal": 0 2024-09-13T18:34:36.893 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.893 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-13T18:34:36.893 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-13T18:34:36.893 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-09-13T18:34:36.893 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-13T18:34:36.893 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 9851, 2024-09-13T18:34:36.893 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 3829503, 2024-09-13T18:34:36.893 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-13T18:34:36.893 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 9851, 2024-09-13T18:34:36.893 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-13T18:34:36.893 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-13T18:34:36.893 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 9851, 2024-09-13T18:34:36.893 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 3829503, 2024-09-13T18:34:36.894 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-13T18:34:36.894 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:36.894 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:36.894 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:36.894 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-13T18:34:36.894 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.894 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_bytes": { 2024-09-13T18:34:36.894 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-13T18:34:36.894 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-09-13T18:34:36.894 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-13T18:34:36.894 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-13T18:34:36.894 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-13T18:34:36.894 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-13T18:34:36.894 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-13T18:34:36.895 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 2080, 2024-09-13T18:34:36.895 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 1620506, 2024-09-13T18:34:36.895 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 2080, 2024-09-13T18:34:36.895 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 1620506, 2024-09-13T18:34:36.895 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-13T18:34:36.895 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:36.895 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:36.896 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:36.896 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-13T18:34:36.896 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.896 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_ops": { 2024-09-13T18:34:36.896 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-13T18:34:36.896 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 1024, 2024-09-13T18:34:36.896 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-13T18:34:36.896 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-13T18:34:36.896 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-13T18:34:36.896 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-13T18:34:36.896 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-13T18:34:36.896 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 2080, 2024-09-13T18:34:36.896 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 2080, 2024-09-13T18:34:36.896 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 2080, 2024-09-13T18:34:36.897 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 2080, 2024-09-13T18:34:36.897 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-13T18:34:36.897 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:36.897 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:36.897 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:36.897 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-13T18:34:36.897 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.897 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle": { 2024-09-13T18:34:36.897 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-13T18:34:36.897 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-09-13T18:34:36.897 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-13T18:34:36.897 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 15, 2024-09-13T18:34:36.897 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 2355, 2024-09-13T18:34:36.897 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-13T18:34:36.898 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 15, 2024-09-13T18:34:36.898 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-13T18:34:36.898 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-13T18:34:36.898 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 3, 2024-09-13T18:34:36.898 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 2355, 2024-09-13T18:34:36.898 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-13T18:34:36.898 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:36.898 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:36.898 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:36.898 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-13T18:34:36.898 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:36.898 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle-0x5653fff58500": { 2024-09-13T18:34:36.898 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-13T18:34:36.898 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-09-13T18:34:36.899 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-13T18:34:36.899 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 276, 2024-09-13T18:34:36.899 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 1541258, 2024-09-13T18:34:36.899 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-13T18:34:36.899 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 276, 2024-09-13T18:34:36.899 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-13T18:34:36.899 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-13T18:34:36.899 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 125, 2024-09-13T18:34:36.899 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 1541258, 2024-09-13T18:34:36.899 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-13T18:34:36.899 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:36.899 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:36.899 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:36.899 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-13T18:34:36.900 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-13T18:34:36.900 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-13T18:34:37.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:37 smithi134 ceph-mon[34283]: pgmap v379: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-13T18:34:37.663 INFO:tasks.check_counter:Daemon mds.h mds.exported=3 2024-09-13T18:34:37.663 INFO:tasks.check_counter:Daemon mds.h mds.imported=4 2024-09-13T18:34:37.663 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=3 2024-09-13T18:34:37.664 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=14 2024-09-13T18:34:37.664 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-09-13T18:34:37.664 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-09-13T18:34:37.664 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-09-13T18:34:37.664 DEBUG:tasks.check_counter:Getting stats from k 2024-09-13T18:34:37.664 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-13T18:34:37.863 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.b/config 2024-09-13T18:34:37.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:37 smithi204 ceph-mon[34777]: pgmap v379: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-13T18:34:37.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:37 smithi114 ceph-mon[30897]: pgmap v379: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-13T18:34:39.585 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:39 smithi134 ceph-mon[34283]: pgmap v380: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-13T18:34:39.586 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-13T18:34:39.586 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-0": { 2024-09-13T18:34:39.586 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 289, 2024-09-13T18:34:39.586 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 265, 2024-09-13T18:34:39.586 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 1674290, 2024-09-13T18:34:39.587 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 142989, 2024-09-13T18:34:39.587 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 5, 2024-09-13T18:34:39.587 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 1, 2024-09-13T18:34:39.587 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.046490303, 2024-09-13T18:34:39.587 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.016017553, 2024-09-13T18:34:39.587 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.046145922, 2024-09-13T18:34:39.587 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.000018872, 2024-09-13T18:34:39.587 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T18:34:39.587 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 264, 2024-09-13T18:34:39.587 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.014002731, 2024-09-13T18:34:39.587 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000053040 2024-09-13T18:34:39.587 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.587 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-13T18:34:39.588 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:39.588 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:39.588 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:39.588 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.588 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 1673456, 2024-09-13T18:34:39.588 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 142672 2024-09-13T18:34:39.588 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.588 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-1": { 2024-09-13T18:34:39.588 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 732, 2024-09-13T18:34:39.588 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 917, 2024-09-13T18:34:39.588 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 179922, 2024-09-13T18:34:39.588 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 699579, 2024-09-13T18:34:39.588 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 7, 2024-09-13T18:34:39.589 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 5, 2024-09-13T18:34:39.589 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.073678146, 2024-09-13T18:34:39.589 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.021746850, 2024-09-13T18:34:39.589 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.016702776, 2024-09-13T18:34:39.589 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.027392793, 2024-09-13T18:34:39.589 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T18:34:39.589 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 914, 2024-09-13T18:34:39.589 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.058658403, 2024-09-13T18:34:39.589 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000064177 2024-09-13T18:34:39.589 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.589 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-13T18:34:39.589 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:39.589 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:39.590 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:39.590 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.590 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-13T18:34:39.590 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 472544 2024-09-13T18:34:39.590 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.590 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-2": { 2024-09-13T18:34:39.590 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 1743, 2024-09-13T18:34:39.590 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 1743, 2024-09-13T18:34:39.590 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 762893, 2024-09-13T18:34:39.590 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 541335, 2024-09-13T18:34:39.590 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 4, 2024-09-13T18:34:39.590 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 3, 2024-09-13T18:34:39.590 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.166755254, 2024-09-13T18:34:39.590 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.049804029, 2024-09-13T18:34:39.591 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.042486086, 2024-09-13T18:34:39.591 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.062274728, 2024-09-13T18:34:39.591 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T18:34:39.591 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1742, 2024-09-13T18:34:39.591 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.092348609, 2024-09-13T18:34:39.591 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000053012 2024-09-13T18:34:39.591 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.591 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-13T18:34:39.591 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:39.591 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:39.591 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:39.591 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.591 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 2960, 2024-09-13T18:34:39.591 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 448 2024-09-13T18:34:39.592 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.592 INFO:teuthology.orchestra.run.smithi134.stdout: "cct": { 2024-09-13T18:34:39.592 INFO:teuthology.orchestra.run.smithi134.stdout: "total_workers": 1, 2024-09-13T18:34:39.592 INFO:teuthology.orchestra.run.smithi134.stdout: "unhealthy_workers": 0 2024-09-13T18:34:39.592 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.592 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-MDSRank": { 2024-09-13T18:34:39.592 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-09-13T18:34:39.592 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-09-13T18:34:39.592 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 4171, 2024-09-13T18:34:39.592 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.116798008, 2024-09-13T18:34:39.592 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000028002 2024-09-13T18:34:39.592 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-13T18:34:39.592 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.592 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-PurgeQueue": { 2024-09-13T18:34:39.593 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-09-13T18:34:39.593 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-09-13T18:34:39.593 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:39.593 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:39.593 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:39.593 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-13T18:34:39.593 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.593 INFO:teuthology.orchestra.run.smithi134.stdout: "mds": { 2024-09-13T18:34:39.593 INFO:teuthology.orchestra.run.smithi134.stdout: "request": 0, 2024-09-13T18:34:39.593 INFO:teuthology.orchestra.run.smithi134.stdout: "reply": 0, 2024-09-13T18:34:39.593 INFO:teuthology.orchestra.run.smithi134.stdout: "reply_latency": { 2024-09-13T18:34:39.593 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:39.593 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:39.594 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:39.594 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.594 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_reply": 0, 2024-09-13T18:34:39.594 INFO:teuthology.orchestra.run.smithi134.stdout: "forward": 0, 2024-09-13T18:34:39.594 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_complete": 0, 2024-09-13T18:34:39.594 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_keys": 0, 2024-09-13T18:34:39.594 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_commit": 0, 2024-09-13T18:34:39.594 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_split": 0, 2024-09-13T18:34:39.594 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_merge": 0, 2024-09-13T18:34:39.594 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes": 9, 2024-09-13T18:34:39.594 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_top": 0, 2024-09-13T18:34:39.594 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_bottom": 9, 2024-09-13T18:34:39.594 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pin_tail": 0, 2024-09-13T18:34:39.594 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pinned": 9, 2024-09-13T18:34:39.595 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_expired": 0, 2024-09-13T18:34:39.595 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_with_caps": 0, 2024-09-13T18:34:39.595 INFO:teuthology.orchestra.run.smithi134.stdout: "caps": 0, 2024-09-13T18:34:39.595 INFO:teuthology.orchestra.run.smithi134.stdout: "subtrees": 1, 2024-09-13T18:34:39.595 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse": 0, 2024-09-13T18:34:39.595 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_hit": 0, 2024-09-13T18:34:39.595 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_forward": 0, 2024-09-13T18:34:39.595 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_discover": 0, 2024-09-13T18:34:39.595 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_dir_fetch": 0, 2024-09-13T18:34:39.595 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_remote_ino": 0, 2024-09-13T18:34:39.595 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_lock": 0, 2024-09-13T18:34:39.595 INFO:teuthology.orchestra.run.smithi134.stdout: "load_cent": 0, 2024-09-13T18:34:39.595 INFO:teuthology.orchestra.run.smithi134.stdout: "q": 0, 2024-09-13T18:34:39.595 INFO:teuthology.orchestra.run.smithi134.stdout: "exported": 0, 2024-09-13T18:34:39.596 INFO:teuthology.orchestra.run.smithi134.stdout: "exported_inodes": 0, 2024-09-13T18:34:39.596 INFO:teuthology.orchestra.run.smithi134.stdout: "imported": 0, 2024-09-13T18:34:39.596 INFO:teuthology.orchestra.run.smithi134.stdout: "imported_inodes": 0, 2024-09-13T18:34:39.596 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_dir_fetch": 0, 2024-09-13T18:34:39.596 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_backtrace_fetch": 0, 2024-09-13T18:34:39.596 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_peer_discover": 0, 2024-09-13T18:34:39.596 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rfiles": 0, 2024-09-13T18:34:39.596 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rbytes": 0, 2024-09-13T18:34:39.596 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rsnaps": 0, 2024-09-13T18:34:39.596 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_fetch": 0, 2024-09-13T18:34:39.596 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_set_tag": 0, 2024-09-13T18:34:39.596 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_repaired": 0, 2024-09-13T18:34:39.596 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_inotable_repaired": 0, 2024-09-13T18:34:39.596 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_inodes": 0, 2024-09-13T18:34:39.597 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_base_inodes": 0, 2024-09-13T18:34:39.597 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dirfrag_rstats": 0, 2024-09-13T18:34:39.597 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_file_inodes": 0, 2024-09-13T18:34:39.597 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_inode_file_caps": 0, 2024-09-13T18:34:39.597 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_revoke": 0, 2024-09-13T18:34:39.597 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_grant": 0, 2024-09-13T18:34:39.597 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_trunc": 0, 2024-09-13T18:34:39.597 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-13T18:34:39.597 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-13T18:34:39.597 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps": 0, 2024-09-13T18:34:39.597 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps_dirty": 0, 2024-09-13T18:34:39.597 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_cap_release": 0, 2024-09-13T18:34:39.597 INFO:teuthology.orchestra.run.smithi134.stdout: "process_request_cap_release": 0 2024-09-13T18:34:39.597 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.598 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_cache": { 2024-09-13T18:34:39.598 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update": 0, 2024-09-13T18:34:39.598 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update_receipt": 0, 2024-09-13T18:34:39.598 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_try_discover": 0, 2024-09-13T18:34:39.598 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_send_discover": 0, 2024-09-13T18:34:39.598 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_handle_discover": 0, 2024-09-13T18:34:39.598 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays": 0, 2024-09-13T18:34:39.598 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_delayed": 0, 2024-09-13T18:34:39.598 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_enqueuing": 0, 2024-09-13T18:34:39.598 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_created": 0, 2024-09-13T18:34:39.598 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_enqueued": 0, 2024-09-13T18:34:39.598 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_reintegrated": 0, 2024-09-13T18:34:39.598 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_migrated": 0, 2024-09-13T18:34:39.598 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_processing": 0, 2024-09-13T18:34:39.598 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_enqueued": 0, 2024-09-13T18:34:39.599 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_prioritized": 0, 2024-09-13T18:34:39.599 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_started": 0, 2024-09-13T18:34:39.599 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_completed": 0, 2024-09-13T18:34:39.599 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_enqueue_scrub": 0, 2024-09-13T18:34:39.599 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_exportdir": 0, 2024-09-13T18:34:39.599 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_flush": 0, 2024-09-13T18:34:39.599 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragmentdir": 0, 2024-09-13T18:34:39.599 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragstats": 0, 2024-09-13T18:34:39.599 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_inodestats": 0 2024-09-13T18:34:39.599 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.599 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_log": { 2024-09-13T18:34:39.599 INFO:teuthology.orchestra.run.smithi134.stdout: "evadd": 0, 2024-09-13T18:34:39.599 INFO:teuthology.orchestra.run.smithi134.stdout: "evex": 0, 2024-09-13T18:34:39.599 INFO:teuthology.orchestra.run.smithi134.stdout: "evtrm": 0, 2024-09-13T18:34:39.600 INFO:teuthology.orchestra.run.smithi134.stdout: "ev": 57, 2024-09-13T18:34:39.600 INFO:teuthology.orchestra.run.smithi134.stdout: "evexg": 0, 2024-09-13T18:34:39.600 INFO:teuthology.orchestra.run.smithi134.stdout: "evexd": 0, 2024-09-13T18:34:39.600 INFO:teuthology.orchestra.run.smithi134.stdout: "segadd": 0, 2024-09-13T18:34:39.600 INFO:teuthology.orchestra.run.smithi134.stdout: "segex": 0, 2024-09-13T18:34:39.600 INFO:teuthology.orchestra.run.smithi134.stdout: "segtrm": 0, 2024-09-13T18:34:39.600 INFO:teuthology.orchestra.run.smithi134.stdout: "seg": 1, 2024-09-13T18:34:39.600 INFO:teuthology.orchestra.run.smithi134.stdout: "segexg": 0, 2024-09-13T18:34:39.600 INFO:teuthology.orchestra.run.smithi134.stdout: "segexd": 0, 2024-09-13T18:34:39.600 INFO:teuthology.orchestra.run.smithi134.stdout: "expos": 4194304, 2024-09-13T18:34:39.600 INFO:teuthology.orchestra.run.smithi134.stdout: "wrpos": 4436562, 2024-09-13T18:34:39.600 INFO:teuthology.orchestra.run.smithi134.stdout: "rdpos": 4433957, 2024-09-13T18:34:39.600 INFO:teuthology.orchestra.run.smithi134.stdout: "jlat": { 2024-09-13T18:34:39.600 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:39.601 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:39.601 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:39.601 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.601 INFO:teuthology.orchestra.run.smithi134.stdout: "replayed": 57 2024-09-13T18:34:39.601 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.601 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_mem": { 2024-09-13T18:34:39.601 INFO:teuthology.orchestra.run.smithi134.stdout: "ino": 7, 2024-09-13T18:34:39.601 INFO:teuthology.orchestra.run.smithi134.stdout: "ino+": 11, 2024-09-13T18:34:39.601 INFO:teuthology.orchestra.run.smithi134.stdout: "ino-": 4, 2024-09-13T18:34:39.601 INFO:teuthology.orchestra.run.smithi134.stdout: "dir": 4, 2024-09-13T18:34:39.601 INFO:teuthology.orchestra.run.smithi134.stdout: "dir+": 9, 2024-09-13T18:34:39.601 INFO:teuthology.orchestra.run.smithi134.stdout: "dir-": 5, 2024-09-13T18:34:39.601 INFO:teuthology.orchestra.run.smithi134.stdout: "dn": 9, 2024-09-13T18:34:39.601 INFO:teuthology.orchestra.run.smithi134.stdout: "dn+": 15, 2024-09-13T18:34:39.602 INFO:teuthology.orchestra.run.smithi134.stdout: "dn-": 6, 2024-09-13T18:34:39.602 INFO:teuthology.orchestra.run.smithi134.stdout: "cap": 0, 2024-09-13T18:34:39.602 INFO:teuthology.orchestra.run.smithi134.stdout: "cap+": 0, 2024-09-13T18:34:39.602 INFO:teuthology.orchestra.run.smithi134.stdout: "cap-": 0, 2024-09-13T18:34:39.602 INFO:teuthology.orchestra.run.smithi134.stdout: "rss": 47940, 2024-09-13T18:34:39.602 INFO:teuthology.orchestra.run.smithi134.stdout: "heap": 207124 2024-09-13T18:34:39.602 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.602 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_server": { 2024-09-13T18:34:39.602 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_client_request": 0, 2024-09-13T18:34:39.602 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_server_request": 0, 2024-09-13T18:34:39.602 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_request": 0, 2024-09-13T18:34:39.602 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_session": 0, 2024-09-13T18:34:39.602 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_peer_request": 0, 2024-09-13T18:34:39.602 INFO:teuthology.orchestra.run.smithi134.stdout: "req_create_latency": { 2024-09-13T18:34:39.603 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:39.603 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:39.603 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:39.603 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.603 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getattr_latency": { 2024-09-13T18:34:39.603 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:39.603 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:39.603 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:39.603 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.603 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getfilelock_latency": { 2024-09-13T18:34:39.603 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:39.603 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:39.603 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:39.603 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.603 INFO:teuthology.orchestra.run.smithi134.stdout: "req_link_latency": { 2024-09-13T18:34:39.604 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:39.604 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:39.604 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:39.604 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.604 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookup_latency": { 2024-09-13T18:34:39.604 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:39.604 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:39.604 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:39.604 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.604 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookuphash_latency": { 2024-09-13T18:34:39.604 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:39.604 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:39.604 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:39.604 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.605 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupino_latency": { 2024-09-13T18:34:39.605 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:39.605 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:39.605 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:39.605 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.605 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupname_latency": { 2024-09-13T18:34:39.605 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:39.605 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:39.605 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:39.605 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.605 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupparent_latency": { 2024-09-13T18:34:39.605 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:39.605 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:39.605 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:39.606 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.606 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupsnap_latency": { 2024-09-13T18:34:39.606 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:39.606 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:39.606 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:39.606 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.606 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lssnap_latency": { 2024-09-13T18:34:39.606 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:39.606 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:39.606 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:39.606 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.606 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mkdir_latency": { 2024-09-13T18:34:39.606 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:39.606 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:39.607 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:39.607 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.607 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mknod_latency": { 2024-09-13T18:34:39.607 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:39.607 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:39.607 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:39.608 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.608 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mksnap_latency": { 2024-09-13T18:34:39.608 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:39.608 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:39.608 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:39.608 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.608 INFO:teuthology.orchestra.run.smithi134.stdout: "req_open_latency": { 2024-09-13T18:34:39.608 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:39.608 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:39.608 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:39.608 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.608 INFO:teuthology.orchestra.run.smithi134.stdout: "req_readdir_latency": { 2024-09-13T18:34:39.608 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:39.608 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:39.609 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:39.609 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.609 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rename_latency": { 2024-09-13T18:34:39.609 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:39.609 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:39.609 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:39.609 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.609 INFO:teuthology.orchestra.run.smithi134.stdout: "req_renamesnap_latency": { 2024-09-13T18:34:39.609 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:39.609 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:39.609 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:39.609 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.609 INFO:teuthology.orchestra.run.smithi134.stdout: "req_snapdiff_latency": { 2024-09-13T18:34:39.609 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:39.610 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:39.610 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:39.610 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.610 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmdir_latency": { 2024-09-13T18:34:39.610 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:39.610 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:39.610 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:39.610 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.610 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmsnap_latency": { 2024-09-13T18:34:39.610 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:39.610 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:39.610 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:39.610 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.610 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmxattr_latency": { 2024-09-13T18:34:39.611 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:39.611 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:39.611 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:39.611 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.611 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setattr_latency": { 2024-09-13T18:34:39.611 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:39.611 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:39.611 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:39.611 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.611 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setdirlayout_latency": { 2024-09-13T18:34:39.611 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:39.612 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:39.612 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:39.612 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.612 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setfilelock_latency": { 2024-09-13T18:34:39.612 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:39.612 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:39.612 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:39.612 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.612 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setlayout_latency": { 2024-09-13T18:34:39.612 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:39.612 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:39.612 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:39.612 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.613 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setxattr_latency": { 2024-09-13T18:34:39.613 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:39.613 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:39.613 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:39.613 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.613 INFO:teuthology.orchestra.run.smithi134.stdout: "req_symlink_latency": { 2024-09-13T18:34:39.613 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:39.613 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:39.613 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:39.613 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.613 INFO:teuthology.orchestra.run.smithi134.stdout: "req_unlink_latency": { 2024-09-13T18:34:39.613 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:39.613 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:39.614 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:39.614 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.614 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_revoke_eviction": 0, 2024-09-13T18:34:39.614 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_acquisition_throttle": 0, 2024-09-13T18:34:39.614 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getvxattr_latency": { 2024-09-13T18:34:39.614 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:39.614 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:39.614 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:39.614 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-13T18:34:39.614 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.614 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_sessions": { 2024-09-13T18:34:39.614 INFO:teuthology.orchestra.run.smithi134.stdout: "session_count": 1, 2024-09-13T18:34:39.615 INFO:teuthology.orchestra.run.smithi134.stdout: "session_add": 3, 2024-09-13T18:34:39.615 INFO:teuthology.orchestra.run.smithi134.stdout: "session_remove": 2, 2024-09-13T18:34:39.615 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_open": 3, 2024-09-13T18:34:39.615 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_stale": 0, 2024-09-13T18:34:39.615 INFO:teuthology.orchestra.run.smithi134.stdout: "total_load": 0, 2024-09-13T18:34:39.615 INFO:teuthology.orchestra.run.smithi134.stdout: "average_load": 0, 2024-09-13T18:34:39.615 INFO:teuthology.orchestra.run.smithi134.stdout: "avg_session_uptime": 3888, 2024-09-13T18:34:39.615 INFO:teuthology.orchestra.run.smithi134.stdout: "mdthresh_evicted": 0 2024-09-13T18:34:39.615 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.615 INFO:teuthology.orchestra.run.smithi134.stdout: "mempool": { 2024-09-13T18:34:39.615 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_bytes": 0, 2024-09-13T18:34:39.615 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_items": 0, 2024-09-13T18:34:39.615 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_bytes": 0, 2024-09-13T18:34:39.616 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_items": 0, 2024-09-13T18:34:39.616 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_bytes": 0, 2024-09-13T18:34:39.616 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_items": 0, 2024-09-13T18:34:39.616 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-13T18:34:39.616 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_items": 0, 2024-09-13T18:34:39.616 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-13T18:34:39.616 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_items": 0, 2024-09-13T18:34:39.616 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_bytes": 0, 2024-09-13T18:34:39.616 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_items": 0, 2024-09-13T18:34:39.616 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-13T18:34:39.616 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_items": 0, 2024-09-13T18:34:39.616 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_bytes": 0, 2024-09-13T18:34:39.616 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_items": 0, 2024-09-13T18:34:39.617 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_bytes": 0, 2024-09-13T18:34:39.617 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_items": 0, 2024-09-13T18:34:39.617 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-13T18:34:39.617 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_items": 0, 2024-09-13T18:34:39.617 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-13T18:34:39.617 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_items": 0, 2024-09-13T18:34:39.617 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_bytes": 0, 2024-09-13T18:34:39.617 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_items": 0, 2024-09-13T18:34:39.617 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_bytes": 0, 2024-09-13T18:34:39.617 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_items": 0, 2024-09-13T18:34:39.617 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-13T18:34:39.617 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_items": 0, 2024-09-13T18:34:39.617 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_bytes": 0, 2024-09-13T18:34:39.617 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_items": 0, 2024-09-13T18:34:39.618 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_bytes": 0, 2024-09-13T18:34:39.618 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_items": 0, 2024-09-13T18:34:39.618 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_bytes": 0, 2024-09-13T18:34:39.618 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_items": 0, 2024-09-13T18:34:39.618 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_bytes": 0, 2024-09-13T18:34:39.618 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_items": 0, 2024-09-13T18:34:39.618 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_bytes": 189393, 2024-09-13T18:34:39.618 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_items": 51, 2024-09-13T18:34:39.618 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_bytes": 0, 2024-09-13T18:34:39.618 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_items": 0, 2024-09-13T18:34:39.618 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_bytes": 0, 2024-09-13T18:34:39.618 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_items": 0, 2024-09-13T18:34:39.618 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_bytes": 0, 2024-09-13T18:34:39.618 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_items": 0, 2024-09-13T18:34:39.619 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_bytes": 0, 2024-09-13T18:34:39.619 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_items": 0, 2024-09-13T18:34:39.619 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_bytes": 9704, 2024-09-13T18:34:39.619 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_items": 196, 2024-09-13T18:34:39.619 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_bytes": 0, 2024-09-13T18:34:39.619 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_items": 0, 2024-09-13T18:34:39.619 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_bytes": 0, 2024-09-13T18:34:39.619 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_items": 0, 2024-09-13T18:34:39.619 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_bytes": 41754, 2024-09-13T18:34:39.619 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_items": 236, 2024-09-13T18:34:39.619 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_bytes": 0, 2024-09-13T18:34:39.619 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_items": 0, 2024-09-13T18:34:39.619 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_bytes": 0, 2024-09-13T18:34:39.620 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_items": 0 2024-09-13T18:34:39.620 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.620 INFO:teuthology.orchestra.run.smithi134.stdout: "objecter": { 2024-09-13T18:34:39.620 INFO:teuthology.orchestra.run.smithi134.stdout: "op_active": 0, 2024-09-13T18:34:39.620 INFO:teuthology.orchestra.run.smithi134.stdout: "op_laggy": 0, 2024-09-13T18:34:39.620 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send": 2470, 2024-09-13T18:34:39.620 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send_bytes": 21, 2024-09-13T18:34:39.620 INFO:teuthology.orchestra.run.smithi134.stdout: "op_resend": 3, 2024-09-13T18:34:39.620 INFO:teuthology.orchestra.run.smithi134.stdout: "op_reply": 2470, 2024-09-13T18:34:39.620 INFO:teuthology.orchestra.run.smithi134.stdout: "op_latency": { 2024-09-13T18:34:39.620 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2470, 2024-09-13T18:34:39.620 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 4.670886038, 2024-09-13T18:34:39.620 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.001891046 2024-09-13T18:34:39.620 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.621 INFO:teuthology.orchestra.run.smithi134.stdout: "op_inflight": 0, 2024-09-13T18:34:39.621 INFO:teuthology.orchestra.run.smithi134.stdout: "oplen_avg": { 2024-09-13T18:34:39.621 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2470, 2024-09-13T18:34:39.621 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 2471 2024-09-13T18:34:39.621 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.621 INFO:teuthology.orchestra.run.smithi134.stdout: "op": 2470, 2024-09-13T18:34:39.621 INFO:teuthology.orchestra.run.smithi134.stdout: "op_r": 2470, 2024-09-13T18:34:39.621 INFO:teuthology.orchestra.run.smithi134.stdout: "op_w": 0, 2024-09-13T18:34:39.621 INFO:teuthology.orchestra.run.smithi134.stdout: "op_rmw": 0, 2024-09-13T18:34:39.621 INFO:teuthology.orchestra.run.smithi134.stdout: "op_pg": 0, 2024-09-13T18:34:39.621 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_stat": 1575, 2024-09-13T18:34:39.621 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_create": 0, 2024-09-13T18:34:39.621 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_read": 893, 2024-09-13T18:34:39.621 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_write": 0, 2024-09-13T18:34:39.622 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writefull": 0, 2024-09-13T18:34:39.622 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writesame": 0, 2024-09-13T18:34:39.622 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_append": 0, 2024-09-13T18:34:39.622 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_zero": 0, 2024-09-13T18:34:39.622 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_truncate": 0, 2024-09-13T18:34:39.622 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_delete": 0, 2024-09-13T18:34:39.622 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_mapext": 0, 2024-09-13T18:34:39.622 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_sparse_read": 0, 2024-09-13T18:34:39.622 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_clonerange": 0, 2024-09-13T18:34:39.622 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_getxattr": 1, 2024-09-13T18:34:39.622 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_setxattr": 0, 2024-09-13T18:34:39.622 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_cmpxattr": 0, 2024-09-13T18:34:39.622 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_rmxattr": 0, 2024-09-13T18:34:39.622 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_resetxattrs": 0, 2024-09-13T18:34:39.623 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_call": 0, 2024-09-13T18:34:39.623 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_watch": 0, 2024-09-13T18:34:39.623 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_notify": 0, 2024-09-13T18:34:39.623 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_src_cmpxattr": 0, 2024-09-13T18:34:39.623 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls": 0, 2024-09-13T18:34:39.623 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls_filter": 0, 2024-09-13T18:34:39.623 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_other": 0, 2024-09-13T18:34:39.623 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_active": 0, 2024-09-13T18:34:39.623 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_send": 0, 2024-09-13T18:34:39.623 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_resend": 0, 2024-09-13T18:34:39.623 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_ping": 0, 2024-09-13T18:34:39.623 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_active": 0, 2024-09-13T18:34:39.623 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_send": 0, 2024-09-13T18:34:39.623 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_resend": 0, 2024-09-13T18:34:39.624 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_active": 0, 2024-09-13T18:34:39.624 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_send": 0, 2024-09-13T18:34:39.624 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_resend": 0, 2024-09-13T18:34:39.624 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_active": 0, 2024-09-13T18:34:39.624 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_send": 0, 2024-09-13T18:34:39.624 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_resend": 0, 2024-09-13T18:34:39.624 INFO:teuthology.orchestra.run.smithi134.stdout: "command_active": 0, 2024-09-13T18:34:39.624 INFO:teuthology.orchestra.run.smithi134.stdout: "command_send": 0, 2024-09-13T18:34:39.624 INFO:teuthology.orchestra.run.smithi134.stdout: "command_resend": 0, 2024-09-13T18:34:39.624 INFO:teuthology.orchestra.run.smithi134.stdout: "map_epoch": 106, 2024-09-13T18:34:39.624 INFO:teuthology.orchestra.run.smithi134.stdout: "map_full": 0, 2024-09-13T18:34:39.624 INFO:teuthology.orchestra.run.smithi134.stdout: "map_inc": 15, 2024-09-13T18:34:39.624 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_sessions": 5, 2024-09-13T18:34:39.624 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_open": 5, 2024-09-13T18:34:39.625 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_close": 0, 2024-09-13T18:34:39.625 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_laggy": 0, 2024-09-13T18:34:39.625 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_wr": 0, 2024-09-13T18:34:39.625 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_rd": 2, 2024-09-13T18:34:39.625 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_del": 0 2024-09-13T18:34:39.625 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.625 INFO:teuthology.orchestra.run.smithi134.stdout: "oft": { 2024-09-13T18:34:39.625 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_objs": 0, 2024-09-13T18:34:39.625 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_kv_pairs": 0, 2024-09-13T18:34:39.625 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_updates": 0, 2024-09-13T18:34:39.625 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_removes": 0 2024-09-13T18:34:39.625 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.625 INFO:teuthology.orchestra.run.smithi134.stdout: "osd-slow-ops": { 2024-09-13T18:34:39.625 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_ops_count": 0 2024-09-13T18:34:39.625 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.626 INFO:teuthology.orchestra.run.smithi134.stdout: "purge_queue": { 2024-09-13T18:34:39.626 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops": 0, 2024-09-13T18:34:39.626 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops_high_water": 0, 2024-09-13T18:34:39.626 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing": 0, 2024-09-13T18:34:39.626 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_high_water": 0, 2024-09-13T18:34:39.626 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed_ops": 0, 2024-09-13T18:34:39.626 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed": 0, 2024-09-13T18:34:39.626 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_item_in_journal": 0 2024-09-13T18:34:39.626 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.626 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-13T18:34:39.626 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-13T18:34:39.626 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-09-13T18:34:39.626 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-13T18:34:39.626 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 2760, 2024-09-13T18:34:39.627 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 2350994, 2024-09-13T18:34:39.627 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-13T18:34:39.627 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 2760, 2024-09-13T18:34:39.627 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-13T18:34:39.627 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-13T18:34:39.627 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 2760, 2024-09-13T18:34:39.627 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 2350994, 2024-09-13T18:34:39.627 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-13T18:34:39.627 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:39.627 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:39.627 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:39.627 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-13T18:34:39.627 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.628 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_bytes": { 2024-09-13T18:34:39.628 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-13T18:34:39.628 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-09-13T18:34:39.628 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-13T18:34:39.628 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-13T18:34:39.628 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-13T18:34:39.628 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-13T18:34:39.628 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-13T18:34:39.628 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 2470, 2024-09-13T18:34:39.628 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 242263, 2024-09-13T18:34:39.628 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 2470, 2024-09-13T18:34:39.628 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 242263, 2024-09-13T18:34:39.628 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-13T18:34:39.628 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:39.629 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:39.629 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:39.629 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-13T18:34:39.629 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.629 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_ops": { 2024-09-13T18:34:39.629 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-13T18:34:39.629 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 1024, 2024-09-13T18:34:39.629 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-13T18:34:39.629 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-13T18:34:39.629 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-13T18:34:39.629 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-13T18:34:39.629 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-13T18:34:39.629 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 2470, 2024-09-13T18:34:39.629 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 2470, 2024-09-13T18:34:39.630 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 2470, 2024-09-13T18:34:39.630 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 2470, 2024-09-13T18:34:39.630 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-13T18:34:39.630 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:39.630 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:39.630 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:39.630 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-13T18:34:39.630 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.630 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle": { 2024-09-13T18:34:39.630 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-13T18:34:39.630 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-09-13T18:34:39.630 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-13T18:34:39.630 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-13T18:34:39.630 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-13T18:34:39.631 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-13T18:34:39.631 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-13T18:34:39.631 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-13T18:34:39.631 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-13T18:34:39.631 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 0, 2024-09-13T18:34:39.631 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 0, 2024-09-13T18:34:39.631 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-13T18:34:39.631 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:39.631 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:39.631 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:39.631 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-13T18:34:39.631 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-13T18:34:39.631 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle-0x561efb4488c0": { 2024-09-13T18:34:39.631 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-13T18:34:39.631 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-09-13T18:34:39.632 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-13T18:34:39.632 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-13T18:34:39.632 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-13T18:34:39.632 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-13T18:34:39.632 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-13T18:34:39.632 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-13T18:34:39.632 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-13T18:34:39.632 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 0, 2024-09-13T18:34:39.632 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 0, 2024-09-13T18:34:39.632 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-13T18:34:39.632 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-13T18:34:39.632 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-13T18:34:39.632 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-13T18:34:39.632 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-13T18:34:39.633 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-13T18:34:39.633 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-13T18:34:39.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:39 smithi204 ceph-mon[34777]: pgmap v380: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-13T18:34:39.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:39 smithi114 ceph-mon[30897]: pgmap v380: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-13T18:34:40.489 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-09-13T18:34:40.490 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-09-13T18:34:40.490 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-09-13T18:34:40.490 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-09-13T18:34:40.490 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-09-13T18:34:40.490 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-09-13T18:34:40.490 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-09-13T18:34:40.490 DEBUG:tasks.check_counter:Getting stats from c 2024-09-13T18:34:40.490 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-13T18:34:40.929 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T18:34:41.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:41 smithi204 ceph-mon[34777]: pgmap v381: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-13T18:34:41.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:41 smithi134 ceph-mon[34283]: pgmap v381: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-13T18:34:41.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:41 smithi114 ceph-mon[30897]: pgmap v381: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-13T18:34:42.619 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-09-13T18:34:42.619 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-0": { 2024-09-13T18:34:42.619 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 286, 2024-09-13T18:34:42.619 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 262, 2024-09-13T18:34:42.619 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 1672172, 2024-09-13T18:34:42.619 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 141031, 2024-09-13T18:34:42.619 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 5, 2024-09-13T18:34:42.619 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 1, 2024-09-13T18:34:42.619 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.042968457, 2024-09-13T18:34:42.620 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.014887851, 2024-09-13T18:34:42.620 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.040082399, 2024-09-13T18:34:42.620 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.000037740, 2024-09-13T18:34:42.620 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T18:34:42.620 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 261, 2024-09-13T18:34:42.620 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.014366557, 2024-09-13T18:34:42.620 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000055044 2024-09-13T18:34:42.620 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.620 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-09-13T18:34:42.620 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:42.620 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:42.620 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:42.620 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.621 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 1671904, 2024-09-13T18:34:42.621 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 140720 2024-09-13T18:34:42.621 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.621 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-1": { 2024-09-13T18:34:42.621 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 1834, 2024-09-13T18:34:42.621 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 2016, 2024-09-13T18:34:42.621 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 2311227, 2024-09-13T18:34:42.621 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 1040245, 2024-09-13T18:34:42.621 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 7, 2024-09-13T18:34:42.621 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 5, 2024-09-13T18:34:42.621 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.168785175, 2024-09-13T18:34:42.622 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.051204249, 2024-09-13T18:34:42.622 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.046771293, 2024-09-13T18:34:42.622 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.057185988, 2024-09-13T18:34:42.622 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T18:34:42.622 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2013, 2024-09-13T18:34:42.623 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.099250522, 2024-09-13T18:34:42.623 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000049304 2024-09-13T18:34:42.623 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.623 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-09-13T18:34:42.623 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:42.623 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:42.623 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:42.623 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.624 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-13T18:34:42.624 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 470432 2024-09-13T18:34:42.624 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.624 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-2": { 2024-09-13T18:34:42.624 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 862, 2024-09-13T18:34:42.624 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 862, 2024-09-13T18:34:42.624 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 214720, 2024-09-13T18:34:42.624 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 267302, 2024-09-13T18:34:42.625 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 5, 2024-09-13T18:34:42.625 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 4, 2024-09-13T18:34:42.625 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.076038887, 2024-09-13T18:34:42.625 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.023710225, 2024-09-13T18:34:42.625 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.018484489, 2024-09-13T18:34:42.625 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.027415074, 2024-09-13T18:34:42.625 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T18:34:42.625 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 861, 2024-09-13T18:34:42.625 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.045924311, 2024-09-13T18:34:42.625 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000053338 2024-09-13T18:34:42.625 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.625 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-09-13T18:34:42.626 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:42.626 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:42.626 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:42.626 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.626 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 2960, 2024-09-13T18:34:42.626 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 448 2024-09-13T18:34:42.626 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.626 INFO:teuthology.orchestra.run.smithi204.stdout: "cct": { 2024-09-13T18:34:42.626 INFO:teuthology.orchestra.run.smithi204.stdout: "total_workers": 1, 2024-09-13T18:34:42.626 INFO:teuthology.orchestra.run.smithi204.stdout: "unhealthy_workers": 0 2024-09-13T18:34:42.626 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.626 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-MDSRank": { 2024-09-13T18:34:42.626 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-09-13T18:34:42.626 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-09-13T18:34:42.627 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 4421, 2024-09-13T18:34:42.627 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.118845105, 2024-09-13T18:34:42.627 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000026881 2024-09-13T18:34:42.627 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-13T18:34:42.627 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.627 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-PurgeQueue": { 2024-09-13T18:34:42.627 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-09-13T18:34:42.627 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-09-13T18:34:42.627 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:42.627 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:42.627 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:42.627 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-13T18:34:42.627 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.627 INFO:teuthology.orchestra.run.smithi204.stdout: "mds": { 2024-09-13T18:34:42.628 INFO:teuthology.orchestra.run.smithi204.stdout: "request": 0, 2024-09-13T18:34:42.628 INFO:teuthology.orchestra.run.smithi204.stdout: "reply": 0, 2024-09-13T18:34:42.628 INFO:teuthology.orchestra.run.smithi204.stdout: "reply_latency": { 2024-09-13T18:34:42.628 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:42.628 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:42.628 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:42.628 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.628 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_reply": 0, 2024-09-13T18:34:42.628 INFO:teuthology.orchestra.run.smithi204.stdout: "forward": 0, 2024-09-13T18:34:42.628 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_complete": 0, 2024-09-13T18:34:42.628 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_keys": 0, 2024-09-13T18:34:42.628 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_commit": 0, 2024-09-13T18:34:42.629 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_split": 0, 2024-09-13T18:34:42.629 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_merge": 0, 2024-09-13T18:34:42.629 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes": 14, 2024-09-13T18:34:42.629 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_top": 5, 2024-09-13T18:34:42.629 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_bottom": 9, 2024-09-13T18:34:42.629 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pin_tail": 0, 2024-09-13T18:34:42.629 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pinned": 7, 2024-09-13T18:34:42.629 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_expired": 0, 2024-09-13T18:34:42.630 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_with_caps": 0, 2024-09-13T18:34:42.630 INFO:teuthology.orchestra.run.smithi204.stdout: "caps": 0, 2024-09-13T18:34:42.630 INFO:teuthology.orchestra.run.smithi204.stdout: "subtrees": 2, 2024-09-13T18:34:42.630 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse": 0, 2024-09-13T18:34:42.630 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_hit": 0, 2024-09-13T18:34:42.631 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_forward": 0, 2024-09-13T18:34:42.631 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_discover": 0, 2024-09-13T18:34:42.631 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_dir_fetch": 0, 2024-09-13T18:34:42.631 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_remote_ino": 0, 2024-09-13T18:34:42.631 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_lock": 0, 2024-09-13T18:34:42.632 INFO:teuthology.orchestra.run.smithi204.stdout: "load_cent": 0, 2024-09-13T18:34:42.632 INFO:teuthology.orchestra.run.smithi204.stdout: "q": 0, 2024-09-13T18:34:42.632 INFO:teuthology.orchestra.run.smithi204.stdout: "exported": 0, 2024-09-13T18:34:42.632 INFO:teuthology.orchestra.run.smithi204.stdout: "exported_inodes": 0, 2024-09-13T18:34:42.632 INFO:teuthology.orchestra.run.smithi204.stdout: "imported": 0, 2024-09-13T18:34:42.633 INFO:teuthology.orchestra.run.smithi204.stdout: "imported_inodes": 0, 2024-09-13T18:34:42.633 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_dir_fetch": 0, 2024-09-13T18:34:42.633 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_backtrace_fetch": 0, 2024-09-13T18:34:42.633 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_peer_discover": 0, 2024-09-13T18:34:42.633 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rfiles": 4, 2024-09-13T18:34:42.633 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rbytes": 234, 2024-09-13T18:34:42.633 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rsnaps": 6, 2024-09-13T18:34:42.633 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_fetch": 0, 2024-09-13T18:34:42.634 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_set_tag": 0, 2024-09-13T18:34:42.634 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_repaired": 0, 2024-09-13T18:34:42.634 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_inotable_repaired": 0, 2024-09-13T18:34:42.634 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_inodes": 0, 2024-09-13T18:34:42.634 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_base_inodes": 0, 2024-09-13T18:34:42.634 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dirfrag_rstats": 0, 2024-09-13T18:34:42.634 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_file_inodes": 0, 2024-09-13T18:34:42.634 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_inode_file_caps": 0, 2024-09-13T18:34:42.634 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_revoke": 0, 2024-09-13T18:34:42.634 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_grant": 0, 2024-09-13T18:34:42.634 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_trunc": 0, 2024-09-13T18:34:42.634 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-13T18:34:42.634 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-13T18:34:42.634 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps": 0, 2024-09-13T18:34:42.635 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps_dirty": 0, 2024-09-13T18:34:42.635 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_cap_release": 0, 2024-09-13T18:34:42.635 INFO:teuthology.orchestra.run.smithi204.stdout: "process_request_cap_release": 0 2024-09-13T18:34:42.635 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.635 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_cache": { 2024-09-13T18:34:42.635 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update": 0, 2024-09-13T18:34:42.635 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update_receipt": 0, 2024-09-13T18:34:42.635 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_try_discover": 0, 2024-09-13T18:34:42.635 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_send_discover": 0, 2024-09-13T18:34:42.635 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_handle_discover": 0, 2024-09-13T18:34:42.635 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays": 0, 2024-09-13T18:34:42.635 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_delayed": 0, 2024-09-13T18:34:42.635 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_enqueuing": 0, 2024-09-13T18:34:42.636 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_created": 0, 2024-09-13T18:34:42.636 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_enqueued": 0, 2024-09-13T18:34:42.636 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_reintegrated": 0, 2024-09-13T18:34:42.636 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_migrated": 0, 2024-09-13T18:34:42.636 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_processing": 0, 2024-09-13T18:34:42.636 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_enqueued": 0, 2024-09-13T18:34:42.636 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_prioritized": 0, 2024-09-13T18:34:42.636 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_started": 0, 2024-09-13T18:34:42.636 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_completed": 0, 2024-09-13T18:34:42.637 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_enqueue_scrub": 0, 2024-09-13T18:34:42.637 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_exportdir": 0, 2024-09-13T18:34:42.637 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_flush": 0, 2024-09-13T18:34:42.637 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragmentdir": 0, 2024-09-13T18:34:42.637 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragstats": 0, 2024-09-13T18:34:42.637 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_inodestats": 0 2024-09-13T18:34:42.637 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.637 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_log": { 2024-09-13T18:34:42.638 INFO:teuthology.orchestra.run.smithi204.stdout: "evadd": 0, 2024-09-13T18:34:42.638 INFO:teuthology.orchestra.run.smithi204.stdout: "evex": 0, 2024-09-13T18:34:42.638 INFO:teuthology.orchestra.run.smithi204.stdout: "evtrm": 0, 2024-09-13T18:34:42.638 INFO:teuthology.orchestra.run.smithi204.stdout: "ev": 353, 2024-09-13T18:34:42.638 INFO:teuthology.orchestra.run.smithi204.stdout: "evexg": 0, 2024-09-13T18:34:42.638 INFO:teuthology.orchestra.run.smithi204.stdout: "evexd": 0, 2024-09-13T18:34:42.638 INFO:teuthology.orchestra.run.smithi204.stdout: "segadd": 0, 2024-09-13T18:34:42.638 INFO:teuthology.orchestra.run.smithi204.stdout: "segex": 0, 2024-09-13T18:34:42.638 INFO:teuthology.orchestra.run.smithi204.stdout: "segtrm": 0, 2024-09-13T18:34:42.638 INFO:teuthology.orchestra.run.smithi204.stdout: "seg": 2, 2024-09-13T18:34:42.638 INFO:teuthology.orchestra.run.smithi204.stdout: "segexg": 0, 2024-09-13T18:34:42.638 INFO:teuthology.orchestra.run.smithi204.stdout: "segexd": 0, 2024-09-13T18:34:42.638 INFO:teuthology.orchestra.run.smithi204.stdout: "expos": 5861362, 2024-09-13T18:34:42.638 INFO:teuthology.orchestra.run.smithi204.stdout: "wrpos": 5963471, 2024-09-13T18:34:42.639 INFO:teuthology.orchestra.run.smithi204.stdout: "rdpos": 5963335, 2024-09-13T18:34:42.639 INFO:teuthology.orchestra.run.smithi204.stdout: "jlat": { 2024-09-13T18:34:42.639 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:42.639 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:42.639 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:42.639 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.639 INFO:teuthology.orchestra.run.smithi204.stdout: "replayed": 353 2024-09-13T18:34:42.639 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.639 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_mem": { 2024-09-13T18:34:42.639 INFO:teuthology.orchestra.run.smithi204.stdout: "ino": 13, 2024-09-13T18:34:42.639 INFO:teuthology.orchestra.run.smithi204.stdout: "ino+": 45, 2024-09-13T18:34:42.639 INFO:teuthology.orchestra.run.smithi204.stdout: "ino-": 32, 2024-09-13T18:34:42.639 INFO:teuthology.orchestra.run.smithi204.stdout: "dir": 8, 2024-09-13T18:34:42.640 INFO:teuthology.orchestra.run.smithi204.stdout: "dir+": 12, 2024-09-13T18:34:42.640 INFO:teuthology.orchestra.run.smithi204.stdout: "dir-": 4, 2024-09-13T18:34:42.640 INFO:teuthology.orchestra.run.smithi204.stdout: "dn": 14, 2024-09-13T18:34:42.640 INFO:teuthology.orchestra.run.smithi204.stdout: "dn+": 46, 2024-09-13T18:34:42.640 INFO:teuthology.orchestra.run.smithi204.stdout: "dn-": 32, 2024-09-13T18:34:42.640 INFO:teuthology.orchestra.run.smithi204.stdout: "cap": 0, 2024-09-13T18:34:42.640 INFO:teuthology.orchestra.run.smithi204.stdout: "cap+": 0, 2024-09-13T18:34:42.640 INFO:teuthology.orchestra.run.smithi204.stdout: "cap-": 0, 2024-09-13T18:34:42.640 INFO:teuthology.orchestra.run.smithi204.stdout: "rss": 48928, 2024-09-13T18:34:42.640 INFO:teuthology.orchestra.run.smithi204.stdout: "heap": 207124 2024-09-13T18:34:42.640 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.640 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_server": { 2024-09-13T18:34:42.641 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_client_request": 0, 2024-09-13T18:34:42.641 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_server_request": 0, 2024-09-13T18:34:42.641 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_request": 0, 2024-09-13T18:34:42.641 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_session": 0, 2024-09-13T18:34:42.641 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_peer_request": 0, 2024-09-13T18:34:42.641 INFO:teuthology.orchestra.run.smithi204.stdout: "req_create_latency": { 2024-09-13T18:34:42.641 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:42.641 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:42.641 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:42.641 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.641 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getattr_latency": { 2024-09-13T18:34:42.641 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:42.641 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:42.642 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:42.642 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.642 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getfilelock_latency": { 2024-09-13T18:34:42.642 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:42.642 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:42.642 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:42.642 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.642 INFO:teuthology.orchestra.run.smithi204.stdout: "req_link_latency": { 2024-09-13T18:34:42.642 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:42.642 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:42.642 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:42.642 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.642 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookup_latency": { 2024-09-13T18:34:42.642 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:42.643 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:42.643 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:42.643 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.643 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookuphash_latency": { 2024-09-13T18:34:42.643 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:42.643 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:42.643 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:42.643 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.643 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupino_latency": { 2024-09-13T18:34:42.643 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:42.643 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:42.643 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:42.643 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.643 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupname_latency": { 2024-09-13T18:34:42.644 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:42.644 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:42.644 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:42.644 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.644 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupparent_latency": { 2024-09-13T18:34:42.644 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:42.644 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:42.644 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:42.644 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.644 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupsnap_latency": { 2024-09-13T18:34:42.644 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:42.644 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:42.644 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:42.645 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.645 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lssnap_latency": { 2024-09-13T18:34:42.645 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:42.645 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:42.645 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:42.645 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.645 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mkdir_latency": { 2024-09-13T18:34:42.645 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:42.645 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:42.645 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:42.645 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.646 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mknod_latency": { 2024-09-13T18:34:42.646 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:42.646 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:42.646 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:42.646 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.646 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mksnap_latency": { 2024-09-13T18:34:42.646 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:42.647 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:42.647 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:42.647 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.647 INFO:teuthology.orchestra.run.smithi204.stdout: "req_open_latency": { 2024-09-13T18:34:42.647 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:42.647 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:42.647 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:42.647 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.647 INFO:teuthology.orchestra.run.smithi204.stdout: "req_readdir_latency": { 2024-09-13T18:34:42.647 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:42.648 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:42.648 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:42.648 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.648 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rename_latency": { 2024-09-13T18:34:42.648 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:42.648 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:42.648 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:42.648 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.648 INFO:teuthology.orchestra.run.smithi204.stdout: "req_renamesnap_latency": { 2024-09-13T18:34:42.648 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:42.648 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:42.648 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:42.648 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.649 INFO:teuthology.orchestra.run.smithi204.stdout: "req_snapdiff_latency": { 2024-09-13T18:34:42.649 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:42.649 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:42.649 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:42.649 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.649 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmdir_latency": { 2024-09-13T18:34:42.649 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:42.649 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:42.649 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:42.649 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.649 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmsnap_latency": { 2024-09-13T18:34:42.649 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:42.649 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:42.650 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:42.650 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.650 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmxattr_latency": { 2024-09-13T18:34:42.650 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:42.650 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:42.650 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:42.650 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.650 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setattr_latency": { 2024-09-13T18:34:42.650 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:42.650 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:42.650 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:42.650 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.651 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setdirlayout_latency": { 2024-09-13T18:34:42.651 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:42.651 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:42.651 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:42.651 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.651 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setfilelock_latency": { 2024-09-13T18:34:42.651 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:42.651 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:42.651 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:42.651 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.651 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setlayout_latency": { 2024-09-13T18:34:42.651 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:42.651 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:42.652 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:42.652 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.652 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setxattr_latency": { 2024-09-13T18:34:42.652 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:42.652 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:42.652 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:42.652 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.652 INFO:teuthology.orchestra.run.smithi204.stdout: "req_symlink_latency": { 2024-09-13T18:34:42.652 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:42.652 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:42.652 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:42.652 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.652 INFO:teuthology.orchestra.run.smithi204.stdout: "req_unlink_latency": { 2024-09-13T18:34:42.653 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:42.653 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:42.653 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:42.653 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.653 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_revoke_eviction": 0, 2024-09-13T18:34:42.653 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_acquisition_throttle": 0, 2024-09-13T18:34:42.653 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getvxattr_latency": { 2024-09-13T18:34:42.653 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:42.653 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:42.653 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:42.653 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-13T18:34:42.653 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.653 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_sessions": { 2024-09-13T18:34:42.654 INFO:teuthology.orchestra.run.smithi204.stdout: "session_count": 2, 2024-09-13T18:34:42.654 INFO:teuthology.orchestra.run.smithi204.stdout: "session_add": 6, 2024-09-13T18:34:42.654 INFO:teuthology.orchestra.run.smithi204.stdout: "session_remove": 4, 2024-09-13T18:34:42.654 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_open": 4, 2024-09-13T18:34:42.654 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_stale": 0, 2024-09-13T18:34:42.654 INFO:teuthology.orchestra.run.smithi204.stdout: "total_load": 0, 2024-09-13T18:34:42.654 INFO:teuthology.orchestra.run.smithi204.stdout: "average_load": 0, 2024-09-13T18:34:42.654 INFO:teuthology.orchestra.run.smithi204.stdout: "avg_session_uptime": 2815, 2024-09-13T18:34:42.654 INFO:teuthology.orchestra.run.smithi204.stdout: "mdthresh_evicted": 0 2024-09-13T18:34:42.654 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.655 INFO:teuthology.orchestra.run.smithi204.stdout: "mempool": { 2024-09-13T18:34:42.655 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_bytes": 0, 2024-09-13T18:34:42.655 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_items": 0, 2024-09-13T18:34:42.655 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_bytes": 0, 2024-09-13T18:34:42.655 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_items": 0, 2024-09-13T18:34:42.655 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_bytes": 0, 2024-09-13T18:34:42.655 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_items": 0, 2024-09-13T18:34:42.655 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-13T18:34:42.655 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_items": 0, 2024-09-13T18:34:42.655 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-13T18:34:42.655 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_items": 0, 2024-09-13T18:34:42.655 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_bytes": 0, 2024-09-13T18:34:42.656 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_items": 0, 2024-09-13T18:34:42.656 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-13T18:34:42.656 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_items": 0, 2024-09-13T18:34:42.656 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_bytes": 0, 2024-09-13T18:34:42.656 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_items": 0, 2024-09-13T18:34:42.656 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_bytes": 0, 2024-09-13T18:34:42.656 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_items": 0, 2024-09-13T18:34:42.656 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-13T18:34:42.656 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_items": 0, 2024-09-13T18:34:42.656 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-13T18:34:42.656 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_items": 0, 2024-09-13T18:34:42.656 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_bytes": 0, 2024-09-13T18:34:42.656 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_items": 0, 2024-09-13T18:34:42.656 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_bytes": 0, 2024-09-13T18:34:42.657 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_items": 0, 2024-09-13T18:34:42.657 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-13T18:34:42.657 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_items": 0, 2024-09-13T18:34:42.657 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_bytes": 0, 2024-09-13T18:34:42.657 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_items": 0, 2024-09-13T18:34:42.657 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_bytes": 0, 2024-09-13T18:34:42.657 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_items": 0, 2024-09-13T18:34:42.657 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_bytes": 0, 2024-09-13T18:34:42.657 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_items": 0, 2024-09-13T18:34:42.657 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_bytes": 0, 2024-09-13T18:34:42.657 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_items": 0, 2024-09-13T18:34:42.657 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_bytes": 197473, 2024-09-13T18:34:42.657 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_items": 55, 2024-09-13T18:34:42.658 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_bytes": 0, 2024-09-13T18:34:42.658 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_items": 0, 2024-09-13T18:34:42.658 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_bytes": 0, 2024-09-13T18:34:42.658 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_items": 0, 2024-09-13T18:34:42.658 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_bytes": 0, 2024-09-13T18:34:42.658 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_items": 0, 2024-09-13T18:34:42.658 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_bytes": 0, 2024-09-13T18:34:42.658 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_items": 0, 2024-09-13T18:34:42.658 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_bytes": 9704, 2024-09-13T18:34:42.658 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_items": 196, 2024-09-13T18:34:42.658 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_bytes": 0, 2024-09-13T18:34:42.658 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_items": 0, 2024-09-13T18:34:42.658 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_bytes": 0, 2024-09-13T18:34:42.659 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_items": 0, 2024-09-13T18:34:42.659 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_bytes": 71498, 2024-09-13T18:34:42.659 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_items": 268, 2024-09-13T18:34:42.659 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_bytes": 0, 2024-09-13T18:34:42.659 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_items": 0, 2024-09-13T18:34:42.659 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_bytes": 0, 2024-09-13T18:34:42.659 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_items": 0 2024-09-13T18:34:42.659 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.659 INFO:teuthology.orchestra.run.smithi204.stdout: "objecter": { 2024-09-13T18:34:42.659 INFO:teuthology.orchestra.run.smithi204.stdout: "op_active": 0, 2024-09-13T18:34:42.659 INFO:teuthology.orchestra.run.smithi204.stdout: "op_laggy": 0, 2024-09-13T18:34:42.659 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send": 2691, 2024-09-13T18:34:42.659 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send_bytes": 26, 2024-09-13T18:34:42.659 INFO:teuthology.orchestra.run.smithi204.stdout: "op_resend": 4, 2024-09-13T18:34:42.660 INFO:teuthology.orchestra.run.smithi204.stdout: "op_reply": 2691, 2024-09-13T18:34:42.660 INFO:teuthology.orchestra.run.smithi204.stdout: "op_latency": { 2024-09-13T18:34:42.660 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2691, 2024-09-13T18:34:42.660 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 1.765975313, 2024-09-13T18:34:42.660 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000656252 2024-09-13T18:34:42.660 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.660 INFO:teuthology.orchestra.run.smithi204.stdout: "op_inflight": 0, 2024-09-13T18:34:42.660 INFO:teuthology.orchestra.run.smithi204.stdout: "oplen_avg": { 2024-09-13T18:34:42.660 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2691, 2024-09-13T18:34:42.660 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 2692 2024-09-13T18:34:42.660 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.660 INFO:teuthology.orchestra.run.smithi204.stdout: "op": 2691, 2024-09-13T18:34:42.660 INFO:teuthology.orchestra.run.smithi204.stdout: "op_r": 2691, 2024-09-13T18:34:42.660 INFO:teuthology.orchestra.run.smithi204.stdout: "op_w": 0, 2024-09-13T18:34:42.660 INFO:teuthology.orchestra.run.smithi204.stdout: "op_rmw": 0, 2024-09-13T18:34:42.661 INFO:teuthology.orchestra.run.smithi204.stdout: "op_pg": 0, 2024-09-13T18:34:42.661 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_stat": 1706, 2024-09-13T18:34:42.661 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_create": 0, 2024-09-13T18:34:42.661 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_read": 982, 2024-09-13T18:34:42.661 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_write": 0, 2024-09-13T18:34:42.661 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writefull": 0, 2024-09-13T18:34:42.661 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writesame": 0, 2024-09-13T18:34:42.661 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_append": 0, 2024-09-13T18:34:42.661 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_zero": 0, 2024-09-13T18:34:42.661 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_truncate": 0, 2024-09-13T18:34:42.661 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_delete": 0, 2024-09-13T18:34:42.662 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_mapext": 0, 2024-09-13T18:34:42.662 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_sparse_read": 0, 2024-09-13T18:34:42.662 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_clonerange": 0, 2024-09-13T18:34:42.662 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_getxattr": 2, 2024-09-13T18:34:42.662 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_setxattr": 0, 2024-09-13T18:34:42.662 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_cmpxattr": 0, 2024-09-13T18:34:42.662 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_rmxattr": 0, 2024-09-13T18:34:42.662 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_resetxattrs": 0, 2024-09-13T18:34:42.662 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_call": 0, 2024-09-13T18:34:42.662 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_watch": 0, 2024-09-13T18:34:42.662 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_notify": 0, 2024-09-13T18:34:42.662 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_src_cmpxattr": 0, 2024-09-13T18:34:42.663 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls": 0, 2024-09-13T18:34:42.663 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls_filter": 0, 2024-09-13T18:34:42.663 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_other": 0, 2024-09-13T18:34:42.663 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_active": 0, 2024-09-13T18:34:42.663 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_send": 0, 2024-09-13T18:34:42.663 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_resend": 0, 2024-09-13T18:34:42.664 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_ping": 0, 2024-09-13T18:34:42.664 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_active": 0, 2024-09-13T18:34:42.664 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_send": 0, 2024-09-13T18:34:42.664 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_resend": 0, 2024-09-13T18:34:42.664 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_active": 0, 2024-09-13T18:34:42.664 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_send": 0, 2024-09-13T18:34:42.664 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_resend": 0, 2024-09-13T18:34:42.665 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_active": 0, 2024-09-13T18:34:42.665 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_send": 0, 2024-09-13T18:34:42.665 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_resend": 0, 2024-09-13T18:34:42.665 INFO:teuthology.orchestra.run.smithi204.stdout: "command_active": 0, 2024-09-13T18:34:42.665 INFO:teuthology.orchestra.run.smithi204.stdout: "command_send": 0, 2024-09-13T18:34:42.666 INFO:teuthology.orchestra.run.smithi204.stdout: "command_resend": 0, 2024-09-13T18:34:42.666 INFO:teuthology.orchestra.run.smithi204.stdout: "map_epoch": 106, 2024-09-13T18:34:42.666 INFO:teuthology.orchestra.run.smithi204.stdout: "map_full": 0, 2024-09-13T18:34:42.666 INFO:teuthology.orchestra.run.smithi204.stdout: "map_inc": 15, 2024-09-13T18:34:42.666 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_sessions": 6, 2024-09-13T18:34:42.666 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_open": 6, 2024-09-13T18:34:42.667 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_close": 0, 2024-09-13T18:34:42.667 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_laggy": 0, 2024-09-13T18:34:42.667 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_wr": 0, 2024-09-13T18:34:42.667 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_rd": 2, 2024-09-13T18:34:42.667 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_del": 0 2024-09-13T18:34:42.668 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.668 INFO:teuthology.orchestra.run.smithi204.stdout: "oft": { 2024-09-13T18:34:42.668 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_objs": 0, 2024-09-13T18:34:42.668 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_kv_pairs": 0, 2024-09-13T18:34:42.668 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_updates": 0, 2024-09-13T18:34:42.668 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_removes": 0 2024-09-13T18:34:42.669 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.669 INFO:teuthology.orchestra.run.smithi204.stdout: "osd-slow-ops": { 2024-09-13T18:34:42.669 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_ops_count": 0 2024-09-13T18:34:42.669 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.669 INFO:teuthology.orchestra.run.smithi204.stdout: "purge_queue": { 2024-09-13T18:34:42.670 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops": 0, 2024-09-13T18:34:42.670 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops_high_water": 0, 2024-09-13T18:34:42.670 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing": 0, 2024-09-13T18:34:42.670 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_high_water": 0, 2024-09-13T18:34:42.670 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed_ops": 0, 2024-09-13T18:34:42.670 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed": 0, 2024-09-13T18:34:42.671 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_item_in_journal": 0 2024-09-13T18:34:42.671 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.671 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-13T18:34:42.671 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-13T18:34:42.671 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-09-13T18:34:42.672 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-13T18:34:42.672 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 2978, 2024-09-13T18:34:42.672 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 3912537, 2024-09-13T18:34:42.672 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-13T18:34:42.672 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 2978, 2024-09-13T18:34:42.672 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-09-13T18:34:42.673 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-09-13T18:34:42.673 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2978, 2024-09-13T18:34:42.673 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 3912537, 2024-09-13T18:34:42.673 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-13T18:34:42.673 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:42.673 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:42.673 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:42.673 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-13T18:34:42.673 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.673 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_bytes": { 2024-09-13T18:34:42.673 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-13T18:34:42.673 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-09-13T18:34:42.673 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-13T18:34:42.673 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-09-13T18:34:42.674 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-09-13T18:34:42.674 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-13T18:34:42.674 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-09-13T18:34:42.674 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 2691, 2024-09-13T18:34:42.674 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 1769177, 2024-09-13T18:34:42.674 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2691, 2024-09-13T18:34:42.674 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 1769177, 2024-09-13T18:34:42.674 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-13T18:34:42.674 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:42.674 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:42.675 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:42.675 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-13T18:34:42.675 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.675 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_ops": { 2024-09-13T18:34:42.675 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-13T18:34:42.675 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 1024, 2024-09-13T18:34:42.675 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-13T18:34:42.675 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-09-13T18:34:42.675 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-09-13T18:34:42.675 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-13T18:34:42.675 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-09-13T18:34:42.675 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 2691, 2024-09-13T18:34:42.675 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 2691, 2024-09-13T18:34:42.676 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2691, 2024-09-13T18:34:42.676 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 2691, 2024-09-13T18:34:42.676 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-13T18:34:42.676 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:42.676 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:42.676 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:42.676 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-13T18:34:42.676 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.676 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle": { 2024-09-13T18:34:42.676 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-13T18:34:42.676 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-09-13T18:34:42.676 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-13T18:34:42.676 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-09-13T18:34:42.676 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-09-13T18:34:42.677 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-13T18:34:42.677 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-09-13T18:34:42.677 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-09-13T18:34:42.677 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-09-13T18:34:42.677 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 0, 2024-09-13T18:34:42.677 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 0, 2024-09-13T18:34:42.677 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-13T18:34:42.677 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:42.677 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:42.677 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:42.677 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-13T18:34:42.677 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:42.677 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle-0x55b0d86c6780": { 2024-09-13T18:34:42.677 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-13T18:34:42.677 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-09-13T18:34:42.678 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-13T18:34:42.678 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-09-13T18:34:42.678 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-09-13T18:34:42.678 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-13T18:34:42.678 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-09-13T18:34:42.678 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-09-13T18:34:42.678 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-09-13T18:34:42.678 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 0, 2024-09-13T18:34:42.678 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 0, 2024-09-13T18:34:42.678 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-13T18:34:42.678 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:42.678 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:42.678 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:42.678 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-13T18:34:42.678 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-13T18:34:42.679 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-09-13T18:34:43.392 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-09-13T18:34:43.392 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-09-13T18:34:43.392 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-09-13T18:34:43.392 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-09-13T18:34:43.392 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-09-13T18:34:43.392 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-09-13T18:34:43.392 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-09-13T18:34:43.393 DEBUG:tasks.check_counter:Getting stats from f 2024-09-13T18:34:43.393 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-13T18:34:43.634 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T18:34:43.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:43 smithi204 ceph-mon[34777]: pgmap v382: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-13T18:34:43.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:43 smithi134 ceph-mon[34283]: pgmap v382: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-13T18:34:43.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:43 smithi114 ceph-mon[30897]: pgmap v382: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-13T18:34:45.439 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-09-13T18:34:45.439 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-0": { 2024-09-13T18:34:45.439 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 2509, 2024-09-13T18:34:45.439 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 2509, 2024-09-13T18:34:45.439 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 2251807, 2024-09-13T18:34:45.440 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 779510, 2024-09-13T18:34:45.440 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 6, 2024-09-13T18:34:45.440 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 2, 2024-09-13T18:34:45.440 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.201183940, 2024-09-13T18:34:45.440 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.055060931, 2024-09-13T18:34:45.440 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.051926913, 2024-09-13T18:34:45.440 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.081243118, 2024-09-13T18:34:45.440 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T18:34:45.440 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2508, 2024-09-13T18:34:45.440 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.093622942, 2024-09-13T18:34:45.440 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000037329 2024-09-13T18:34:45.440 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.440 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-09-13T18:34:45.441 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:45.441 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:45.441 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:45.441 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.441 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 2960, 2024-09-13T18:34:45.441 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 448 2024-09-13T18:34:45.441 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.441 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-1": { 2024-09-13T18:34:45.441 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 289, 2024-09-13T18:34:45.441 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 265, 2024-09-13T18:34:45.441 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 1673740, 2024-09-13T18:34:45.441 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 142764, 2024-09-13T18:34:45.441 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 3, 2024-09-13T18:34:45.441 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 2, 2024-09-13T18:34:45.442 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.048419078, 2024-09-13T18:34:45.442 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.016230841, 2024-09-13T18:34:45.444 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.047674076, 2024-09-13T18:34:45.444 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.000070484, 2024-09-13T18:34:45.444 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T18:34:45.444 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 264, 2024-09-13T18:34:45.444 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.016876391, 2024-09-13T18:34:45.444 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000063925 2024-09-13T18:34:45.444 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.444 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-09-13T18:34:45.445 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:45.445 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:45.445 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:45.445 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.445 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 1673184, 2024-09-13T18:34:45.445 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 142080 2024-09-13T18:34:45.445 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.445 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-2": { 2024-09-13T18:34:45.445 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 3, 2024-09-13T18:34:45.445 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 187, 2024-09-13T18:34:45.445 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 588, 2024-09-13T18:34:45.445 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 474263, 2024-09-13T18:34:45.445 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 6, 2024-09-13T18:34:45.445 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 4, 2024-09-13T18:34:45.446 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.011121606, 2024-09-13T18:34:45.446 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.006756856, 2024-09-13T18:34:45.446 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.002525582, 2024-09-13T18:34:45.446 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.000018954, 2024-09-13T18:34:45.446 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T18:34:45.446 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 184, 2024-09-13T18:34:45.446 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.019220829, 2024-09-13T18:34:45.446 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000104461 2024-09-13T18:34:45.446 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.446 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-09-13T18:34:45.446 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:45.446 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:45.446 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:45.446 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.447 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-13T18:34:45.447 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 473952 2024-09-13T18:34:45.447 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.447 INFO:teuthology.orchestra.run.smithi204.stdout: "cct": { 2024-09-13T18:34:45.447 INFO:teuthology.orchestra.run.smithi204.stdout: "total_workers": 1, 2024-09-13T18:34:45.447 INFO:teuthology.orchestra.run.smithi204.stdout: "unhealthy_workers": 0 2024-09-13T18:34:45.447 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.447 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-MDSRank": { 2024-09-13T18:34:45.447 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-09-13T18:34:45.447 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-09-13T18:34:45.447 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 4240, 2024-09-13T18:34:45.447 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.100875248, 2024-09-13T18:34:45.448 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000023791 2024-09-13T18:34:45.448 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-13T18:34:45.448 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.448 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-PurgeQueue": { 2024-09-13T18:34:45.448 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-09-13T18:34:45.448 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-09-13T18:34:45.448 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:45.448 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:45.448 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:45.448 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-13T18:34:45.448 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.448 INFO:teuthology.orchestra.run.smithi204.stdout: "mds": { 2024-09-13T18:34:45.448 INFO:teuthology.orchestra.run.smithi204.stdout: "request": 0, 2024-09-13T18:34:45.448 INFO:teuthology.orchestra.run.smithi204.stdout: "reply": 0, 2024-09-13T18:34:45.449 INFO:teuthology.orchestra.run.smithi204.stdout: "reply_latency": { 2024-09-13T18:34:45.449 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:45.449 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:45.449 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:45.449 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.449 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_reply": 0, 2024-09-13T18:34:45.449 INFO:teuthology.orchestra.run.smithi204.stdout: "forward": 0, 2024-09-13T18:34:45.449 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_complete": 0, 2024-09-13T18:34:45.449 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_keys": 0, 2024-09-13T18:34:45.449 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_commit": 0, 2024-09-13T18:34:45.449 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_split": 0, 2024-09-13T18:34:45.449 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_merge": 0, 2024-09-13T18:34:45.449 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes": 144, 2024-09-13T18:34:45.450 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_top": 25, 2024-09-13T18:34:45.450 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_bottom": 119, 2024-09-13T18:34:45.450 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pin_tail": 0, 2024-09-13T18:34:45.450 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pinned": 109, 2024-09-13T18:34:45.450 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_expired": 0, 2024-09-13T18:34:45.450 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_with_caps": 0, 2024-09-13T18:34:45.450 INFO:teuthology.orchestra.run.smithi204.stdout: "caps": 0, 2024-09-13T18:34:45.450 INFO:teuthology.orchestra.run.smithi204.stdout: "subtrees": 1, 2024-09-13T18:34:45.450 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse": 0, 2024-09-13T18:34:45.450 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_hit": 0, 2024-09-13T18:34:45.450 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_forward": 0, 2024-09-13T18:34:45.450 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_discover": 0, 2024-09-13T18:34:45.450 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_dir_fetch": 0, 2024-09-13T18:34:45.450 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_remote_ino": 0, 2024-09-13T18:34:45.451 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_lock": 0, 2024-09-13T18:34:45.451 INFO:teuthology.orchestra.run.smithi204.stdout: "load_cent": 0, 2024-09-13T18:34:45.451 INFO:teuthology.orchestra.run.smithi204.stdout: "q": 0, 2024-09-13T18:34:45.451 INFO:teuthology.orchestra.run.smithi204.stdout: "exported": 0, 2024-09-13T18:34:45.451 INFO:teuthology.orchestra.run.smithi204.stdout: "exported_inodes": 0, 2024-09-13T18:34:45.451 INFO:teuthology.orchestra.run.smithi204.stdout: "imported": 0, 2024-09-13T18:34:45.451 INFO:teuthology.orchestra.run.smithi204.stdout: "imported_inodes": 0, 2024-09-13T18:34:45.451 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_dir_fetch": 0, 2024-09-13T18:34:45.451 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_backtrace_fetch": 0, 2024-09-13T18:34:45.451 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_peer_discover": 0, 2024-09-13T18:34:45.451 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rfiles": 0, 2024-09-13T18:34:45.451 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rbytes": 0, 2024-09-13T18:34:45.451 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rsnaps": 0, 2024-09-13T18:34:45.451 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_fetch": 0, 2024-09-13T18:34:45.452 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_set_tag": 0, 2024-09-13T18:34:45.452 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_repaired": 0, 2024-09-13T18:34:45.452 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_inotable_repaired": 0, 2024-09-13T18:34:45.452 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_inodes": 0, 2024-09-13T18:34:45.452 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_base_inodes": 0, 2024-09-13T18:34:45.452 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dirfrag_rstats": 0, 2024-09-13T18:34:45.452 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_file_inodes": 0, 2024-09-13T18:34:45.452 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_inode_file_caps": 0, 2024-09-13T18:34:45.452 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_revoke": 0, 2024-09-13T18:34:45.452 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_grant": 0, 2024-09-13T18:34:45.452 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_trunc": 0, 2024-09-13T18:34:45.452 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-13T18:34:45.452 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-13T18:34:45.452 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps": 0, 2024-09-13T18:34:45.453 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps_dirty": 0, 2024-09-13T18:34:45.453 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_cap_release": 0, 2024-09-13T18:34:45.453 INFO:teuthology.orchestra.run.smithi204.stdout: "process_request_cap_release": 0 2024-09-13T18:34:45.453 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.453 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_cache": { 2024-09-13T18:34:45.453 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update": 0, 2024-09-13T18:34:45.453 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update_receipt": 0, 2024-09-13T18:34:45.453 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_try_discover": 0, 2024-09-13T18:34:45.453 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_send_discover": 0, 2024-09-13T18:34:45.453 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_handle_discover": 0, 2024-09-13T18:34:45.453 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays": 0, 2024-09-13T18:34:45.453 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_delayed": 0, 2024-09-13T18:34:45.453 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_enqueuing": 0, 2024-09-13T18:34:45.453 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_created": 0, 2024-09-13T18:34:45.454 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_enqueued": 0, 2024-09-13T18:34:45.454 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_reintegrated": 0, 2024-09-13T18:34:45.454 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_migrated": 0, 2024-09-13T18:34:45.454 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_processing": 0, 2024-09-13T18:34:45.454 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_enqueued": 0, 2024-09-13T18:34:45.454 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_prioritized": 0, 2024-09-13T18:34:45.454 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_started": 0, 2024-09-13T18:34:45.454 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_completed": 0, 2024-09-13T18:34:45.454 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_enqueue_scrub": 0, 2024-09-13T18:34:45.454 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_exportdir": 0, 2024-09-13T18:34:45.454 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_flush": 0, 2024-09-13T18:34:45.454 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragmentdir": 0, 2024-09-13T18:34:45.454 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragstats": 0, 2024-09-13T18:34:45.454 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_inodestats": 0 2024-09-13T18:34:45.454 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.455 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_log": { 2024-09-13T18:34:45.455 INFO:teuthology.orchestra.run.smithi204.stdout: "evadd": 0, 2024-09-13T18:34:45.455 INFO:teuthology.orchestra.run.smithi204.stdout: "evex": 0, 2024-09-13T18:34:45.455 INFO:teuthology.orchestra.run.smithi204.stdout: "evtrm": 0, 2024-09-13T18:34:45.455 INFO:teuthology.orchestra.run.smithi204.stdout: "ev": 276, 2024-09-13T18:34:45.455 INFO:teuthology.orchestra.run.smithi204.stdout: "evexg": 0, 2024-09-13T18:34:45.455 INFO:teuthology.orchestra.run.smithi204.stdout: "evexd": 0, 2024-09-13T18:34:45.455 INFO:teuthology.orchestra.run.smithi204.stdout: "segadd": 0, 2024-09-13T18:34:45.455 INFO:teuthology.orchestra.run.smithi204.stdout: "segex": 0, 2024-09-13T18:34:45.455 INFO:teuthology.orchestra.run.smithi204.stdout: "segtrm": 0, 2024-09-13T18:34:45.456 INFO:teuthology.orchestra.run.smithi204.stdout: "seg": 1, 2024-09-13T18:34:45.456 INFO:teuthology.orchestra.run.smithi204.stdout: "segexg": 0, 2024-09-13T18:34:45.456 INFO:teuthology.orchestra.run.smithi204.stdout: "segexd": 0, 2024-09-13T18:34:45.456 INFO:teuthology.orchestra.run.smithi204.stdout: "expos": 4194304, 2024-09-13T18:34:45.456 INFO:teuthology.orchestra.run.smithi204.stdout: "wrpos": 5735562, 2024-09-13T18:34:45.456 INFO:teuthology.orchestra.run.smithi204.stdout: "rdpos": 5732484, 2024-09-13T18:34:45.456 INFO:teuthology.orchestra.run.smithi204.stdout: "jlat": { 2024-09-13T18:34:45.456 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:45.456 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:45.456 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:45.456 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.456 INFO:teuthology.orchestra.run.smithi204.stdout: "replayed": 276 2024-09-13T18:34:45.456 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.456 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_mem": { 2024-09-13T18:34:45.457 INFO:teuthology.orchestra.run.smithi204.stdout: "ino": 113, 2024-09-13T18:34:45.457 INFO:teuthology.orchestra.run.smithi204.stdout: "ino+": 177, 2024-09-13T18:34:45.457 INFO:teuthology.orchestra.run.smithi204.stdout: "ino-": 64, 2024-09-13T18:34:45.457 INFO:teuthology.orchestra.run.smithi204.stdout: "dir": 11, 2024-09-13T18:34:45.457 INFO:teuthology.orchestra.run.smithi204.stdout: "dir+": 21, 2024-09-13T18:34:45.457 INFO:teuthology.orchestra.run.smithi204.stdout: "dir-": 10, 2024-09-13T18:34:45.457 INFO:teuthology.orchestra.run.smithi204.stdout: "dn": 144, 2024-09-13T18:34:45.457 INFO:teuthology.orchestra.run.smithi204.stdout: "dn+": 196, 2024-09-13T18:34:45.457 INFO:teuthology.orchestra.run.smithi204.stdout: "dn-": 52, 2024-09-13T18:34:45.457 INFO:teuthology.orchestra.run.smithi204.stdout: "cap": 0, 2024-09-13T18:34:45.457 INFO:teuthology.orchestra.run.smithi204.stdout: "cap+": 0, 2024-09-13T18:34:45.457 INFO:teuthology.orchestra.run.smithi204.stdout: "cap-": 0, 2024-09-13T18:34:45.457 INFO:teuthology.orchestra.run.smithi204.stdout: "rss": 52024, 2024-09-13T18:34:45.457 INFO:teuthology.orchestra.run.smithi204.stdout: "heap": 207124 2024-09-13T18:34:45.458 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.458 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_server": { 2024-09-13T18:34:45.458 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_client_request": 0, 2024-09-13T18:34:45.458 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_server_request": 0, 2024-09-13T18:34:45.458 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_request": 0, 2024-09-13T18:34:45.458 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_session": 0, 2024-09-13T18:34:45.458 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_peer_request": 0, 2024-09-13T18:34:45.458 INFO:teuthology.orchestra.run.smithi204.stdout: "req_create_latency": { 2024-09-13T18:34:45.458 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:45.458 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:45.458 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:45.458 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.458 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getattr_latency": { 2024-09-13T18:34:45.459 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:45.459 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:45.459 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:45.459 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.459 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getfilelock_latency": { 2024-09-13T18:34:45.459 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:45.459 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:45.459 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:45.459 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.459 INFO:teuthology.orchestra.run.smithi204.stdout: "req_link_latency": { 2024-09-13T18:34:45.459 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:45.459 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:45.459 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:45.459 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.459 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookup_latency": { 2024-09-13T18:34:45.460 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:45.460 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:45.460 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:45.460 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.460 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookuphash_latency": { 2024-09-13T18:34:45.460 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:45.460 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:45.460 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:45.460 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.460 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupino_latency": { 2024-09-13T18:34:45.460 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:45.460 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:45.460 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:45.460 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.461 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupname_latency": { 2024-09-13T18:34:45.461 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:45.461 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:45.461 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:45.461 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.461 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupparent_latency": { 2024-09-13T18:34:45.461 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:45.461 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:45.461 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:45.461 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.461 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupsnap_latency": { 2024-09-13T18:34:45.461 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:45.461 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:45.461 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:45.462 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.462 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lssnap_latency": { 2024-09-13T18:34:45.462 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:45.462 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:45.462 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:45.462 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.462 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mkdir_latency": { 2024-09-13T18:34:45.462 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:45.462 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:45.462 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:45.462 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.463 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mknod_latency": { 2024-09-13T18:34:45.463 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:45.463 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:45.463 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:45.463 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.463 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mksnap_latency": { 2024-09-13T18:34:45.463 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:45.463 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:45.464 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:45.464 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.464 INFO:teuthology.orchestra.run.smithi204.stdout: "req_open_latency": { 2024-09-13T18:34:45.464 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:45.464 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:45.464 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:45.464 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.464 INFO:teuthology.orchestra.run.smithi204.stdout: "req_readdir_latency": { 2024-09-13T18:34:45.464 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:45.464 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:45.464 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:45.464 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.464 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rename_latency": { 2024-09-13T18:34:45.465 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:45.465 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:45.465 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:45.465 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.465 INFO:teuthology.orchestra.run.smithi204.stdout: "req_renamesnap_latency": { 2024-09-13T18:34:45.465 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:45.465 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:45.465 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:45.465 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.465 INFO:teuthology.orchestra.run.smithi204.stdout: "req_snapdiff_latency": { 2024-09-13T18:34:45.465 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:45.465 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:45.465 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:45.465 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.466 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmdir_latency": { 2024-09-13T18:34:45.466 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:45.466 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:45.466 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:45.466 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.466 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmsnap_latency": { 2024-09-13T18:34:45.466 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:45.466 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:45.466 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:45.466 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.466 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmxattr_latency": { 2024-09-13T18:34:45.466 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:45.466 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:45.466 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:45.467 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.467 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setattr_latency": { 2024-09-13T18:34:45.467 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:45.467 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:45.467 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:45.467 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.467 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setdirlayout_latency": { 2024-09-13T18:34:45.467 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:45.467 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:45.467 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:45.467 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.467 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setfilelock_latency": { 2024-09-13T18:34:45.467 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:45.468 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:45.468 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:45.468 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.468 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setlayout_latency": { 2024-09-13T18:34:45.468 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:45.468 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:45.468 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:45.468 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.468 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setxattr_latency": { 2024-09-13T18:34:45.468 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:45.468 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:45.468 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:45.468 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.468 INFO:teuthology.orchestra.run.smithi204.stdout: "req_symlink_latency": { 2024-09-13T18:34:45.469 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:45.469 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:45.469 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:45.469 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.469 INFO:teuthology.orchestra.run.smithi204.stdout: "req_unlink_latency": { 2024-09-13T18:34:45.469 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:45.469 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:45.469 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:45.469 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.469 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_revoke_eviction": 0, 2024-09-13T18:34:45.469 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_acquisition_throttle": 0, 2024-09-13T18:34:45.469 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getvxattr_latency": { 2024-09-13T18:34:45.469 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:45.469 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:45.470 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:45.470 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-13T18:34:45.470 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.470 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_sessions": { 2024-09-13T18:34:45.470 INFO:teuthology.orchestra.run.smithi204.stdout: "session_count": 2, 2024-09-13T18:34:45.470 INFO:teuthology.orchestra.run.smithi204.stdout: "session_add": 3, 2024-09-13T18:34:45.470 INFO:teuthology.orchestra.run.smithi204.stdout: "session_remove": 1, 2024-09-13T18:34:45.470 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_open": 3, 2024-09-13T18:34:45.470 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_stale": 0, 2024-09-13T18:34:45.470 INFO:teuthology.orchestra.run.smithi204.stdout: "total_load": 0, 2024-09-13T18:34:45.470 INFO:teuthology.orchestra.run.smithi204.stdout: "average_load": 0, 2024-09-13T18:34:45.470 INFO:teuthology.orchestra.run.smithi204.stdout: "avg_session_uptime": 2301, 2024-09-13T18:34:45.470 INFO:teuthology.orchestra.run.smithi204.stdout: "mdthresh_evicted": 0 2024-09-13T18:34:45.470 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.470 INFO:teuthology.orchestra.run.smithi204.stdout: "mempool": { 2024-09-13T18:34:45.471 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_bytes": 0, 2024-09-13T18:34:45.471 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_items": 0, 2024-09-13T18:34:45.471 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_bytes": 0, 2024-09-13T18:34:45.471 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_items": 0, 2024-09-13T18:34:45.471 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_bytes": 0, 2024-09-13T18:34:45.471 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_items": 0, 2024-09-13T18:34:45.471 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-13T18:34:45.471 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_items": 0, 2024-09-13T18:34:45.471 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-13T18:34:45.471 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_items": 0, 2024-09-13T18:34:45.471 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_bytes": 0, 2024-09-13T18:34:45.471 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_items": 0, 2024-09-13T18:34:45.471 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-13T18:34:45.471 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_items": 0, 2024-09-13T18:34:45.472 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_bytes": 0, 2024-09-13T18:34:45.472 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_items": 0, 2024-09-13T18:34:45.472 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_bytes": 0, 2024-09-13T18:34:45.472 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_items": 0, 2024-09-13T18:34:45.472 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-13T18:34:45.472 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_items": 0, 2024-09-13T18:34:45.472 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-13T18:34:45.472 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_items": 0, 2024-09-13T18:34:45.472 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_bytes": 0, 2024-09-13T18:34:45.472 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_items": 0, 2024-09-13T18:34:45.472 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_bytes": 0, 2024-09-13T18:34:45.472 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_items": 0, 2024-09-13T18:34:45.472 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-13T18:34:45.473 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_items": 0, 2024-09-13T18:34:45.473 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_bytes": 0, 2024-09-13T18:34:45.473 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_items": 0, 2024-09-13T18:34:45.473 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_bytes": 0, 2024-09-13T18:34:45.473 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_items": 0, 2024-09-13T18:34:45.473 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_bytes": 0, 2024-09-13T18:34:45.473 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_items": 0, 2024-09-13T18:34:45.473 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_bytes": 0, 2024-09-13T18:34:45.473 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_items": 0, 2024-09-13T18:34:45.473 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_bytes": 181353, 2024-09-13T18:34:45.473 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_items": 48, 2024-09-13T18:34:45.473 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_bytes": 0, 2024-09-13T18:34:45.473 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_items": 0, 2024-09-13T18:34:45.473 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_bytes": 0, 2024-09-13T18:34:45.474 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_items": 0, 2024-09-13T18:34:45.474 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_bytes": 0, 2024-09-13T18:34:45.474 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_items": 0, 2024-09-13T18:34:45.474 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_bytes": 0, 2024-09-13T18:34:45.474 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_items": 0, 2024-09-13T18:34:45.474 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_bytes": 9704, 2024-09-13T18:34:45.474 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_items": 196, 2024-09-13T18:34:45.474 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_bytes": 0, 2024-09-13T18:34:45.474 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_items": 0, 2024-09-13T18:34:45.474 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_bytes": 0, 2024-09-13T18:34:45.474 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_items": 0, 2024-09-13T18:34:45.474 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_bytes": 360378, 2024-09-13T18:34:45.474 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_items": 1201, 2024-09-13T18:34:45.474 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_bytes": 0, 2024-09-13T18:34:45.474 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_items": 0, 2024-09-13T18:34:45.475 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_bytes": 0, 2024-09-13T18:34:45.475 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_items": 0 2024-09-13T18:34:45.475 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.475 INFO:teuthology.orchestra.run.smithi204.stdout: "objecter": { 2024-09-13T18:34:45.475 INFO:teuthology.orchestra.run.smithi204.stdout: "op_active": 0, 2024-09-13T18:34:45.475 INFO:teuthology.orchestra.run.smithi204.stdout: "op_laggy": 0, 2024-09-13T18:34:45.475 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send": 2508, 2024-09-13T18:34:45.475 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send_bytes": 21, 2024-09-13T18:34:45.475 INFO:teuthology.orchestra.run.smithi204.stdout: "op_resend": 3, 2024-09-13T18:34:45.475 INFO:teuthology.orchestra.run.smithi204.stdout: "op_reply": 2508, 2024-09-13T18:34:45.475 INFO:teuthology.orchestra.run.smithi204.stdout: "op_latency": { 2024-09-13T18:34:45.475 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2508, 2024-09-13T18:34:45.475 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 2.564964373, 2024-09-13T18:34:45.475 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.001022713 2024-09-13T18:34:45.475 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.476 INFO:teuthology.orchestra.run.smithi204.stdout: "op_inflight": 0, 2024-09-13T18:34:45.476 INFO:teuthology.orchestra.run.smithi204.stdout: "oplen_avg": { 2024-09-13T18:34:45.476 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2508, 2024-09-13T18:34:45.476 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 2509 2024-09-13T18:34:45.476 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.476 INFO:teuthology.orchestra.run.smithi204.stdout: "op": 2508, 2024-09-13T18:34:45.476 INFO:teuthology.orchestra.run.smithi204.stdout: "op_r": 2508, 2024-09-13T18:34:45.476 INFO:teuthology.orchestra.run.smithi204.stdout: "op_w": 0, 2024-09-13T18:34:45.476 INFO:teuthology.orchestra.run.smithi204.stdout: "op_rmw": 0, 2024-09-13T18:34:45.476 INFO:teuthology.orchestra.run.smithi204.stdout: "op_pg": 0, 2024-09-13T18:34:45.476 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_stat": 1593, 2024-09-13T18:34:45.476 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_create": 0, 2024-09-13T18:34:45.476 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_read": 913, 2024-09-13T18:34:45.476 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_write": 0, 2024-09-13T18:34:45.477 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writefull": 0, 2024-09-13T18:34:45.477 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writesame": 0, 2024-09-13T18:34:45.477 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_append": 0, 2024-09-13T18:34:45.477 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_zero": 0, 2024-09-13T18:34:45.477 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_truncate": 0, 2024-09-13T18:34:45.477 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_delete": 0, 2024-09-13T18:34:45.477 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_mapext": 0, 2024-09-13T18:34:45.477 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_sparse_read": 0, 2024-09-13T18:34:45.477 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_clonerange": 0, 2024-09-13T18:34:45.477 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_getxattr": 1, 2024-09-13T18:34:45.477 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_setxattr": 0, 2024-09-13T18:34:45.477 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_cmpxattr": 0, 2024-09-13T18:34:45.477 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_rmxattr": 0, 2024-09-13T18:34:45.477 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_resetxattrs": 0, 2024-09-13T18:34:45.478 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_call": 0, 2024-09-13T18:34:45.478 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_watch": 0, 2024-09-13T18:34:45.478 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_notify": 0, 2024-09-13T18:34:45.478 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_src_cmpxattr": 0, 2024-09-13T18:34:45.478 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls": 0, 2024-09-13T18:34:45.478 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls_filter": 0, 2024-09-13T18:34:45.478 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_other": 0, 2024-09-13T18:34:45.478 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_active": 0, 2024-09-13T18:34:45.478 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_send": 0, 2024-09-13T18:34:45.478 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_resend": 0, 2024-09-13T18:34:45.478 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_ping": 0, 2024-09-13T18:34:45.478 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_active": 0, 2024-09-13T18:34:45.478 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_send": 0, 2024-09-13T18:34:45.478 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_resend": 0, 2024-09-13T18:34:45.478 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_active": 0, 2024-09-13T18:34:45.479 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_send": 0, 2024-09-13T18:34:45.479 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_resend": 0, 2024-09-13T18:34:45.479 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_active": 0, 2024-09-13T18:34:45.479 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_send": 0, 2024-09-13T18:34:45.479 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_resend": 0, 2024-09-13T18:34:45.479 INFO:teuthology.orchestra.run.smithi204.stdout: "command_active": 0, 2024-09-13T18:34:45.479 INFO:teuthology.orchestra.run.smithi204.stdout: "command_send": 0, 2024-09-13T18:34:45.479 INFO:teuthology.orchestra.run.smithi204.stdout: "command_resend": 0, 2024-09-13T18:34:45.479 INFO:teuthology.orchestra.run.smithi204.stdout: "map_epoch": 106, 2024-09-13T18:34:45.479 INFO:teuthology.orchestra.run.smithi204.stdout: "map_full": 0, 2024-09-13T18:34:45.479 INFO:teuthology.orchestra.run.smithi204.stdout: "map_inc": 15, 2024-09-13T18:34:45.479 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_sessions": 4, 2024-09-13T18:34:45.479 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_open": 4, 2024-09-13T18:34:45.480 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_close": 0, 2024-09-13T18:34:45.480 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_laggy": 0, 2024-09-13T18:34:45.480 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_wr": 0, 2024-09-13T18:34:45.480 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_rd": 2, 2024-09-13T18:34:45.480 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_del": 0 2024-09-13T18:34:45.480 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.480 INFO:teuthology.orchestra.run.smithi204.stdout: "oft": { 2024-09-13T18:34:45.480 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_objs": 0, 2024-09-13T18:34:45.480 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_kv_pairs": 0, 2024-09-13T18:34:45.481 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_updates": 0, 2024-09-13T18:34:45.481 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_removes": 0 2024-09-13T18:34:45.481 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.481 INFO:teuthology.orchestra.run.smithi204.stdout: "osd-slow-ops": { 2024-09-13T18:34:45.481 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_ops_count": 0 2024-09-13T18:34:45.481 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.481 INFO:teuthology.orchestra.run.smithi204.stdout: "purge_queue": { 2024-09-13T18:34:45.481 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops": 0, 2024-09-13T18:34:45.481 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops_high_water": 0, 2024-09-13T18:34:45.481 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing": 0, 2024-09-13T18:34:45.482 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_high_water": 0, 2024-09-13T18:34:45.482 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed_ops": 0, 2024-09-13T18:34:45.482 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed": 0, 2024-09-13T18:34:45.482 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_item_in_journal": 0 2024-09-13T18:34:45.482 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.482 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-13T18:34:45.482 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-13T18:34:45.482 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-09-13T18:34:45.482 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-13T18:34:45.483 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 2797, 2024-09-13T18:34:45.483 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 3656746, 2024-09-13T18:34:45.483 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-13T18:34:45.483 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 2797, 2024-09-13T18:34:45.483 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-09-13T18:34:45.483 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-09-13T18:34:45.483 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2797, 2024-09-13T18:34:45.483 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 3656746, 2024-09-13T18:34:45.484 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-13T18:34:45.484 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:45.484 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:45.484 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:45.484 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-13T18:34:45.484 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.484 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_bytes": { 2024-09-13T18:34:45.484 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-13T18:34:45.484 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-09-13T18:34:45.484 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-13T18:34:45.484 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-09-13T18:34:45.484 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-09-13T18:34:45.484 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-13T18:34:45.484 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-09-13T18:34:45.485 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 2508, 2024-09-13T18:34:45.485 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 1541263, 2024-09-13T18:34:45.485 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2508, 2024-09-13T18:34:45.485 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 1541263, 2024-09-13T18:34:45.485 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-13T18:34:45.485 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:45.485 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:45.485 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:45.485 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-13T18:34:45.485 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.485 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_ops": { 2024-09-13T18:34:45.485 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-13T18:34:45.485 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 1024, 2024-09-13T18:34:45.486 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-13T18:34:45.486 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-09-13T18:34:45.486 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-09-13T18:34:45.486 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-13T18:34:45.486 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-09-13T18:34:45.486 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 2508, 2024-09-13T18:34:45.486 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 2508, 2024-09-13T18:34:45.486 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2508, 2024-09-13T18:34:45.486 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 2508, 2024-09-13T18:34:45.486 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-13T18:34:45.486 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:45.486 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:45.486 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:45.486 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-13T18:34:45.486 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.487 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle": { 2024-09-13T18:34:45.487 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-13T18:34:45.487 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-09-13T18:34:45.487 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-13T18:34:45.487 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-09-13T18:34:45.487 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-09-13T18:34:45.487 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-13T18:34:45.487 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-09-13T18:34:45.487 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-09-13T18:34:45.487 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-09-13T18:34:45.487 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 0, 2024-09-13T18:34:45.487 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 0, 2024-09-13T18:34:45.487 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-13T18:34:45.487 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:45.488 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:45.488 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:45.488 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-13T18:34:45.488 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:45.488 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle-0x555fd61668c0": { 2024-09-13T18:34:45.488 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-13T18:34:45.488 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-09-13T18:34:45.488 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-13T18:34:45.488 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-09-13T18:34:45.489 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-09-13T18:34:45.489 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-13T18:34:45.489 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-09-13T18:34:45.489 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-09-13T18:34:45.489 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-09-13T18:34:45.489 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 0, 2024-09-13T18:34:45.489 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 0, 2024-09-13T18:34:45.489 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-13T18:34:45.489 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:45.489 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:45.489 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:45.489 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-13T18:34:45.489 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-13T18:34:45.489 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-09-13T18:34:45.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:45 smithi134 ceph-mon[34283]: pgmap v383: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-13T18:34:45.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:45 smithi204 ceph-mon[34777]: pgmap v383: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-13T18:34:45.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:45 smithi114 ceph-mon[30897]: pgmap v383: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-13T18:34:46.481 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-13T18:34:46.482 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-13T18:34:46.482 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-09-13T18:34:46.482 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-09-13T18:34:46.482 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-09-13T18:34:46.482 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-09-13T18:34:46.482 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-09-13T18:34:46.482 DEBUG:tasks.check_counter:Getting stats from i 2024-09-13T18:34:46.482 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-13T18:34:46.701 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T18:34:47.857 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:47 smithi204 ceph-mon[34777]: pgmap v384: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-13T18:34:47.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:47 smithi134 ceph-mon[34283]: pgmap v384: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-13T18:34:47.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:47 smithi114 ceph-mon[30897]: pgmap v384: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-13T18:34:48.442 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-09-13T18:34:48.442 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-0": { 2024-09-13T18:34:48.442 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 1346, 2024-09-13T18:34:48.442 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 1690, 2024-09-13T18:34:48.443 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 534693, 2024-09-13T18:34:48.443 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 1276148, 2024-09-13T18:34:48.443 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 11, 2024-09-13T18:34:48.443 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 3, 2024-09-13T18:34:48.443 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.121869027, 2024-09-13T18:34:48.443 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.046171336, 2024-09-13T18:34:48.443 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.064220285, 2024-09-13T18:34:48.443 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.013027546, 2024-09-13T18:34:48.443 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T18:34:48.443 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 1694, 2024-09-13T18:34:48.443 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.126061292, 2024-09-13T18:34:48.443 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000074416 2024-09-13T18:34:48.444 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.444 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-09-13T18:34:48.444 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2598, 2024-09-13T18:34:48.444 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000463074, 2024-09-13T18:34:48.444 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000178 2024-09-13T18:34:48.444 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.444 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-13T18:34:48.444 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 347248 2024-09-13T18:34:48.444 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.444 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-1": { 2024-09-13T18:34:48.444 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 294, 2024-09-13T18:34:48.444 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 376, 2024-09-13T18:34:48.444 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 85652, 2024-09-13T18:34:48.444 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 636129, 2024-09-13T18:34:48.445 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 55, 2024-09-13T18:34:48.445 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 9, 2024-09-13T18:34:48.445 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.047384649, 2024-09-13T18:34:48.445 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.009807172, 2024-09-13T18:34:48.445 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.035935527, 2024-09-13T18:34:48.445 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.001216634, 2024-09-13T18:34:48.445 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T18:34:48.445 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 369, 2024-09-13T18:34:48.445 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.024379562, 2024-09-13T18:34:48.445 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000066069 2024-09-13T18:34:48.445 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.445 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-09-13T18:34:48.446 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 471, 2024-09-13T18:34:48.446 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000099258, 2024-09-13T18:34:48.446 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000210 2024-09-13T18:34:48.446 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.446 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 2016, 2024-09-13T18:34:48.446 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 130368 2024-09-13T18:34:48.446 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.446 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-2": { 2024-09-13T18:34:48.446 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 501, 2024-09-13T18:34:48.446 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 557, 2024-09-13T18:34:48.446 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 1778346, 2024-09-13T18:34:48.446 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 280213, 2024-09-13T18:34:48.446 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 9, 2024-09-13T18:34:48.446 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 6, 2024-09-13T18:34:48.447 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.063110708, 2024-09-13T18:34:48.447 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.022651615, 2024-09-13T18:34:48.447 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.059073065, 2024-09-13T18:34:48.447 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.000279979, 2024-09-13T18:34:48.447 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T18:34:48.447 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 555, 2024-09-13T18:34:48.447 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.051342580, 2024-09-13T18:34:48.447 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000092509 2024-09-13T18:34:48.447 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.447 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-09-13T18:34:48.447 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 326, 2024-09-13T18:34:48.447 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000064868, 2024-09-13T18:34:48.447 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000198 2024-09-13T18:34:48.447 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.448 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 1697664, 2024-09-13T18:34:48.448 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 162432 2024-09-13T18:34:48.448 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.448 INFO:teuthology.orchestra.run.smithi204.stdout: "cct": { 2024-09-13T18:34:48.448 INFO:teuthology.orchestra.run.smithi204.stdout: "total_workers": 1, 2024-09-13T18:34:48.448 INFO:teuthology.orchestra.run.smithi204.stdout: "unhealthy_workers": 0 2024-09-13T18:34:48.448 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.448 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-MDSRank": { 2024-09-13T18:34:48.448 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-09-13T18:34:48.448 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-09-13T18:34:48.448 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 195, 2024-09-13T18:34:48.448 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.024634752, 2024-09-13T18:34:48.448 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000126332 2024-09-13T18:34:48.449 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-13T18:34:48.449 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.449 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-PurgeQueue": { 2024-09-13T18:34:48.449 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-09-13T18:34:48.449 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-09-13T18:34:48.449 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2, 2024-09-13T18:34:48.449 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000017413, 2024-09-13T18:34:48.449 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000008706 2024-09-13T18:34:48.449 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-13T18:34:48.449 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.449 INFO:teuthology.orchestra.run.smithi204.stdout: "mds": { 2024-09-13T18:34:48.449 INFO:teuthology.orchestra.run.smithi204.stdout: "request": 181, 2024-09-13T18:34:48.449 INFO:teuthology.orchestra.run.smithi204.stdout: "reply": 36, 2024-09-13T18:34:48.449 INFO:teuthology.orchestra.run.smithi204.stdout: "reply_latency": { 2024-09-13T18:34:48.450 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 36, 2024-09-13T18:34:48.450 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.323736166, 2024-09-13T18:34:48.450 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.008992671 2024-09-13T18:34:48.450 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.450 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_reply": 0, 2024-09-13T18:34:48.450 INFO:teuthology.orchestra.run.smithi204.stdout: "forward": 145, 2024-09-13T18:34:48.450 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_complete": 0, 2024-09-13T18:34:48.450 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_keys": 0, 2024-09-13T18:34:48.450 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_commit": 11, 2024-09-13T18:34:48.450 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_split": 0, 2024-09-13T18:34:48.450 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_merge": 0, 2024-09-13T18:34:48.450 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes": 23, 2024-09-13T18:34:48.450 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_top": 0, 2024-09-13T18:34:48.451 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_bottom": 7, 2024-09-13T18:34:48.451 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pin_tail": 16, 2024-09-13T18:34:48.451 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pinned": 22, 2024-09-13T18:34:48.451 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_expired": 6, 2024-09-13T18:34:48.451 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_with_caps": 2, 2024-09-13T18:34:48.451 INFO:teuthology.orchestra.run.smithi204.stdout: "caps": 2, 2024-09-13T18:34:48.451 INFO:teuthology.orchestra.run.smithi204.stdout: "subtrees": 3, 2024-09-13T18:34:48.451 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse": 216, 2024-09-13T18:34:48.451 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_hit": 62, 2024-09-13T18:34:48.451 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_forward": 144, 2024-09-13T18:34:48.451 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_discover": 4, 2024-09-13T18:34:48.451 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_dir_fetch": 0, 2024-09-13T18:34:48.451 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_remote_ino": 0, 2024-09-13T18:34:48.451 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_lock": 0, 2024-09-13T18:34:48.452 INFO:teuthology.orchestra.run.smithi204.stdout: "load_cent": 2000, 2024-09-13T18:34:48.452 INFO:teuthology.orchestra.run.smithi204.stdout: "q": 2, 2024-09-13T18:34:48.452 INFO:teuthology.orchestra.run.smithi204.stdout: "exported": 1, 2024-09-13T18:34:48.452 INFO:teuthology.orchestra.run.smithi204.stdout: "exported_inodes": 2, 2024-09-13T18:34:48.452 INFO:teuthology.orchestra.run.smithi204.stdout: "imported": 2, 2024-09-13T18:34:48.452 INFO:teuthology.orchestra.run.smithi204.stdout: "imported_inodes": 6, 2024-09-13T18:34:48.452 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_dir_fetch": 0, 2024-09-13T18:34:48.452 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_backtrace_fetch": 30, 2024-09-13T18:34:48.452 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_peer_discover": 0, 2024-09-13T18:34:48.452 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rfiles": 22, 2024-09-13T18:34:48.452 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rbytes": 10738143812, 2024-09-13T18:34:48.452 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rsnaps": 6, 2024-09-13T18:34:48.452 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_fetch": 30, 2024-09-13T18:34:48.453 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_set_tag": 0, 2024-09-13T18:34:48.453 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_repaired": 0, 2024-09-13T18:34:48.453 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_inotable_repaired": 0, 2024-09-13T18:34:48.453 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_inodes": 30, 2024-09-13T18:34:48.453 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_base_inodes": 0, 2024-09-13T18:34:48.453 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dirfrag_rstats": 30, 2024-09-13T18:34:48.453 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_file_inodes": 0, 2024-09-13T18:34:48.453 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_inode_file_caps": 0, 2024-09-13T18:34:48.453 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_revoke": 11, 2024-09-13T18:34:48.453 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_grant": 5, 2024-09-13T18:34:48.453 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_trunc": 0, 2024-09-13T18:34:48.453 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-13T18:34:48.453 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-13T18:34:48.453 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps": 12, 2024-09-13T18:34:48.454 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps_dirty": 0, 2024-09-13T18:34:48.454 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_cap_release": 1, 2024-09-13T18:34:48.454 INFO:teuthology.orchestra.run.smithi204.stdout: "process_request_cap_release": 3 2024-09-13T18:34:48.454 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.454 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_cache": { 2024-09-13T18:34:48.454 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update": 0, 2024-09-13T18:34:48.454 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update_receipt": 14, 2024-09-13T18:34:48.454 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_try_discover": 9, 2024-09-13T18:34:48.454 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_send_discover": 10, 2024-09-13T18:34:48.454 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_handle_discover": 0, 2024-09-13T18:34:48.454 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays": 2, 2024-09-13T18:34:48.454 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_delayed": 0, 2024-09-13T18:34:48.454 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_enqueuing": 0, 2024-09-13T18:34:48.454 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_created": 2, 2024-09-13T18:34:48.455 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_enqueued": 0, 2024-09-13T18:34:48.455 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_reintegrated": 0, 2024-09-13T18:34:48.455 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_migrated": 0, 2024-09-13T18:34:48.455 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_processing": 0, 2024-09-13T18:34:48.455 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_enqueued": 0, 2024-09-13T18:34:48.455 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_prioritized": 0, 2024-09-13T18:34:48.455 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_started": 0, 2024-09-13T18:34:48.455 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_completed": 0, 2024-09-13T18:34:48.455 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_enqueue_scrub": 0, 2024-09-13T18:34:48.455 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_exportdir": 1, 2024-09-13T18:34:48.455 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_flush": 0, 2024-09-13T18:34:48.455 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragmentdir": 0, 2024-09-13T18:34:48.455 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragstats": 0, 2024-09-13T18:34:48.455 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_inodestats": 0 2024-09-13T18:34:48.456 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.456 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_log": { 2024-09-13T18:34:48.456 INFO:teuthology.orchestra.run.smithi204.stdout: "evadd": 57, 2024-09-13T18:34:48.456 INFO:teuthology.orchestra.run.smithi204.stdout: "evex": 0, 2024-09-13T18:34:48.456 INFO:teuthology.orchestra.run.smithi204.stdout: "evtrm": 0, 2024-09-13T18:34:48.456 INFO:teuthology.orchestra.run.smithi204.stdout: "ev": 57, 2024-09-13T18:34:48.456 INFO:teuthology.orchestra.run.smithi204.stdout: "evexg": 0, 2024-09-13T18:34:48.456 INFO:teuthology.orchestra.run.smithi204.stdout: "evexd": 0, 2024-09-13T18:34:48.456 INFO:teuthology.orchestra.run.smithi204.stdout: "segadd": 1, 2024-09-13T18:34:48.456 INFO:teuthology.orchestra.run.smithi204.stdout: "segex": 0, 2024-09-13T18:34:48.456 INFO:teuthology.orchestra.run.smithi204.stdout: "segtrm": 0, 2024-09-13T18:34:48.456 INFO:teuthology.orchestra.run.smithi204.stdout: "seg": 1, 2024-09-13T18:34:48.456 INFO:teuthology.orchestra.run.smithi204.stdout: "segexg": 0, 2024-09-13T18:34:48.456 INFO:teuthology.orchestra.run.smithi204.stdout: "segexd": 0, 2024-09-13T18:34:48.457 INFO:teuthology.orchestra.run.smithi204.stdout: "expos": 4194304, 2024-09-13T18:34:48.457 INFO:teuthology.orchestra.run.smithi204.stdout: "wrpos": 4436562, 2024-09-13T18:34:48.457 INFO:teuthology.orchestra.run.smithi204.stdout: "rdpos": 0, 2024-09-13T18:34:48.457 INFO:teuthology.orchestra.run.smithi204.stdout: "jlat": { 2024-09-13T18:34:48.457 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 52, 2024-09-13T18:34:48.457 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.170721363, 2024-09-13T18:34:48.457 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.003283103 2024-09-13T18:34:48.457 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.457 INFO:teuthology.orchestra.run.smithi204.stdout: "replayed": 0 2024-09-13T18:34:48.457 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.457 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_mem": { 2024-09-13T18:34:48.457 INFO:teuthology.orchestra.run.smithi204.stdout: "ino": 22, 2024-09-13T18:34:48.457 INFO:teuthology.orchestra.run.smithi204.stdout: "ino+": 25, 2024-09-13T18:34:48.457 INFO:teuthology.orchestra.run.smithi204.stdout: "ino-": 3, 2024-09-13T18:34:48.458 INFO:teuthology.orchestra.run.smithi204.stdout: "dir": 20, 2024-09-13T18:34:48.458 INFO:teuthology.orchestra.run.smithi204.stdout: "dir+": 32, 2024-09-13T18:34:48.458 INFO:teuthology.orchestra.run.smithi204.stdout: "dir-": 12, 2024-09-13T18:34:48.458 INFO:teuthology.orchestra.run.smithi204.stdout: "dn": 23, 2024-09-13T18:34:48.458 INFO:teuthology.orchestra.run.smithi204.stdout: "dn+": 29, 2024-09-13T18:34:48.458 INFO:teuthology.orchestra.run.smithi204.stdout: "dn-": 6, 2024-09-13T18:34:48.458 INFO:teuthology.orchestra.run.smithi204.stdout: "cap": 2, 2024-09-13T18:34:48.458 INFO:teuthology.orchestra.run.smithi204.stdout: "cap+": 9, 2024-09-13T18:34:48.458 INFO:teuthology.orchestra.run.smithi204.stdout: "cap-": 7, 2024-09-13T18:34:48.458 INFO:teuthology.orchestra.run.smithi204.stdout: "rss": 50828, 2024-09-13T18:34:48.458 INFO:teuthology.orchestra.run.smithi204.stdout: "heap": 198932 2024-09-13T18:34:48.458 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.458 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_server": { 2024-09-13T18:34:48.458 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_client_request": 209, 2024-09-13T18:34:48.458 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_server_request": 6, 2024-09-13T18:34:48.459 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_request": 181, 2024-09-13T18:34:48.459 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_session": 18, 2024-09-13T18:34:48.459 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_peer_request": 27, 2024-09-13T18:34:48.459 INFO:teuthology.orchestra.run.smithi204.stdout: "req_create_latency": { 2024-09-13T18:34:48.459 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:48.459 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:48.459 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:48.459 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.459 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getattr_latency": { 2024-09-13T18:34:48.460 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 10, 2024-09-13T18:34:48.460 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.001906530, 2024-09-13T18:34:48.460 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000190653 2024-09-13T18:34:48.460 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.460 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getfilelock_latency": { 2024-09-13T18:34:48.460 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:48.460 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:48.460 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:48.460 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.460 INFO:teuthology.orchestra.run.smithi204.stdout: "req_link_latency": { 2024-09-13T18:34:48.460 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:48.460 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:48.460 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:48.461 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.461 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookup_latency": { 2024-09-13T18:34:48.461 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 24, 2024-09-13T18:34:48.461 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.309549176, 2024-09-13T18:34:48.461 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.012897882 2024-09-13T18:34:48.461 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.461 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookuphash_latency": { 2024-09-13T18:34:48.461 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:48.461 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:48.461 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:48.461 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.461 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupino_latency": { 2024-09-13T18:34:48.461 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:48.461 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:48.462 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:48.462 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.462 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupname_latency": { 2024-09-13T18:34:48.462 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:48.462 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:48.462 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:48.462 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.462 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupparent_latency": { 2024-09-13T18:34:48.462 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:48.462 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:48.462 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:48.462 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.462 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupsnap_latency": { 2024-09-13T18:34:48.462 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:48.463 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:48.463 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:48.463 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.463 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lssnap_latency": { 2024-09-13T18:34:48.463 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:48.463 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:48.463 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:48.463 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.463 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mkdir_latency": { 2024-09-13T18:34:48.463 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:48.463 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:48.464 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:48.464 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.464 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mknod_latency": { 2024-09-13T18:34:48.464 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:48.464 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:48.464 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:48.464 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.464 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mksnap_latency": { 2024-09-13T18:34:48.465 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:48.465 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:48.465 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:48.465 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.465 INFO:teuthology.orchestra.run.smithi204.stdout: "req_open_latency": { 2024-09-13T18:34:48.465 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:48.465 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:48.465 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:48.465 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.465 INFO:teuthology.orchestra.run.smithi204.stdout: "req_readdir_latency": { 2024-09-13T18:34:48.466 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:48.466 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:48.466 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:48.466 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.466 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rename_latency": { 2024-09-13T18:34:48.466 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:48.466 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:48.466 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:48.466 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.466 INFO:teuthology.orchestra.run.smithi204.stdout: "req_renamesnap_latency": { 2024-09-13T18:34:48.466 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:48.466 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:48.467 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:48.467 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.467 INFO:teuthology.orchestra.run.smithi204.stdout: "req_snapdiff_latency": { 2024-09-13T18:34:48.467 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:48.467 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:48.467 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:48.467 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.467 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmdir_latency": { 2024-09-13T18:34:48.467 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2, 2024-09-13T18:34:48.467 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.012280460, 2024-09-13T18:34:48.467 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.006140230 2024-09-13T18:34:48.467 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.467 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmsnap_latency": { 2024-09-13T18:34:48.467 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:48.468 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:48.468 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:48.468 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.468 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmxattr_latency": { 2024-09-13T18:34:48.468 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:48.468 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:48.468 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:48.468 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.468 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setattr_latency": { 2024-09-13T18:34:48.468 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:48.468 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:48.468 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:48.468 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.469 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setdirlayout_latency": { 2024-09-13T18:34:48.469 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:48.469 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:48.469 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:48.469 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.469 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setfilelock_latency": { 2024-09-13T18:34:48.469 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:48.469 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:48.469 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:48.469 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.469 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setlayout_latency": { 2024-09-13T18:34:48.469 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:48.469 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:48.470 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:48.470 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.470 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setxattr_latency": { 2024-09-13T18:34:48.470 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:48.470 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:48.470 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:48.470 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.470 INFO:teuthology.orchestra.run.smithi204.stdout: "req_symlink_latency": { 2024-09-13T18:34:48.470 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:48.470 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:48.470 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:48.470 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.470 INFO:teuthology.orchestra.run.smithi204.stdout: "req_unlink_latency": { 2024-09-13T18:34:48.471 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:48.471 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:48.471 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:48.471 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.471 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_revoke_eviction": 0, 2024-09-13T18:34:48.471 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_acquisition_throttle": 0, 2024-09-13T18:34:48.471 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getvxattr_latency": { 2024-09-13T18:34:48.471 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:48.471 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:48.471 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:48.471 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-13T18:34:48.471 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.471 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_sessions": { 2024-09-13T18:34:48.471 INFO:teuthology.orchestra.run.smithi204.stdout: "session_count": 1, 2024-09-13T18:34:48.472 INFO:teuthology.orchestra.run.smithi204.stdout: "session_add": 3, 2024-09-13T18:34:48.472 INFO:teuthology.orchestra.run.smithi204.stdout: "session_remove": 2, 2024-09-13T18:34:48.472 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_open": 1, 2024-09-13T18:34:48.472 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_stale": 0, 2024-09-13T18:34:48.472 INFO:teuthology.orchestra.run.smithi204.stdout: "total_load": 21, 2024-09-13T18:34:48.472 INFO:teuthology.orchestra.run.smithi204.stdout: "average_load": 21, 2024-09-13T18:34:48.472 INFO:teuthology.orchestra.run.smithi204.stdout: "avg_session_uptime": 741, 2024-09-13T18:34:48.472 INFO:teuthology.orchestra.run.smithi204.stdout: "mdthresh_evicted": 0 2024-09-13T18:34:48.472 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.472 INFO:teuthology.orchestra.run.smithi204.stdout: "mempool": { 2024-09-13T18:34:48.472 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_bytes": 0, 2024-09-13T18:34:48.472 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_items": 0, 2024-09-13T18:34:48.472 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_bytes": 0, 2024-09-13T18:34:48.473 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_items": 0, 2024-09-13T18:34:48.473 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_bytes": 0, 2024-09-13T18:34:48.473 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_items": 0, 2024-09-13T18:34:48.473 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-13T18:34:48.473 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_items": 0, 2024-09-13T18:34:48.473 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-13T18:34:48.473 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_items": 0, 2024-09-13T18:34:48.473 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_bytes": 0, 2024-09-13T18:34:48.473 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_items": 0, 2024-09-13T18:34:48.474 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-13T18:34:48.474 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_items": 0, 2024-09-13T18:34:48.474 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_bytes": 0, 2024-09-13T18:34:48.474 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_items": 0, 2024-09-13T18:34:48.474 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_bytes": 0, 2024-09-13T18:34:48.474 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_items": 0, 2024-09-13T18:34:48.474 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-13T18:34:48.474 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_items": 0, 2024-09-13T18:34:48.474 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-13T18:34:48.474 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_items": 0, 2024-09-13T18:34:48.474 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_bytes": 0, 2024-09-13T18:34:48.474 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_items": 0, 2024-09-13T18:34:48.474 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_bytes": 0, 2024-09-13T18:34:48.475 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_items": 0, 2024-09-13T18:34:48.475 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-13T18:34:48.475 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_items": 0, 2024-09-13T18:34:48.475 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_bytes": 0, 2024-09-13T18:34:48.475 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_items": 0, 2024-09-13T18:34:48.475 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_bytes": 0, 2024-09-13T18:34:48.475 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_items": 0, 2024-09-13T18:34:48.475 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_bytes": 0, 2024-09-13T18:34:48.475 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_items": 0, 2024-09-13T18:34:48.475 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_bytes": 0, 2024-09-13T18:34:48.475 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_items": 0, 2024-09-13T18:34:48.475 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_bytes": 297124, 2024-09-13T18:34:48.475 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_items": 108, 2024-09-13T18:34:48.475 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_bytes": 0, 2024-09-13T18:34:48.475 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_items": 0, 2024-09-13T18:34:48.476 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_bytes": 0, 2024-09-13T18:34:48.476 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_items": 0, 2024-09-13T18:34:48.476 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_bytes": 0, 2024-09-13T18:34:48.476 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_items": 0, 2024-09-13T18:34:48.476 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_bytes": 0, 2024-09-13T18:34:48.476 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_items": 0, 2024-09-13T18:34:48.476 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_bytes": 9704, 2024-09-13T18:34:48.476 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_items": 196, 2024-09-13T18:34:48.476 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_bytes": 0, 2024-09-13T18:34:48.476 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_items": 0, 2024-09-13T18:34:48.476 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_bytes": 0, 2024-09-13T18:34:48.476 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_items": 0, 2024-09-13T18:34:48.476 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_bytes": 104703, 2024-09-13T18:34:48.477 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_items": 648, 2024-09-13T18:34:48.477 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_bytes": 0, 2024-09-13T18:34:48.477 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_items": 0, 2024-09-13T18:34:48.477 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_bytes": 0, 2024-09-13T18:34:48.477 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_items": 0 2024-09-13T18:34:48.477 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.477 INFO:teuthology.orchestra.run.smithi204.stdout: "objecter": { 2024-09-13T18:34:48.477 INFO:teuthology.orchestra.run.smithi204.stdout: "op_active": 0, 2024-09-13T18:34:48.477 INFO:teuthology.orchestra.run.smithi204.stdout: "op_laggy": 0, 2024-09-13T18:34:48.477 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send": 141, 2024-09-13T18:34:48.477 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send_bytes": 255189, 2024-09-13T18:34:48.477 INFO:teuthology.orchestra.run.smithi204.stdout: "op_resend": 33, 2024-09-13T18:34:48.477 INFO:teuthology.orchestra.run.smithi204.stdout: "op_reply": 141, 2024-09-13T18:34:48.477 INFO:teuthology.orchestra.run.smithi204.stdout: "op_latency": { 2024-09-13T18:34:48.478 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 141, 2024-09-13T18:34:48.478 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 5.975915347, 2024-09-13T18:34:48.478 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.042382378 2024-09-13T18:34:48.478 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.478 INFO:teuthology.orchestra.run.smithi204.stdout: "op_inflight": 0, 2024-09-13T18:34:48.478 INFO:teuthology.orchestra.run.smithi204.stdout: "oplen_avg": { 2024-09-13T18:34:48.478 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 141, 2024-09-13T18:34:48.478 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 168 2024-09-13T18:34:48.478 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.478 INFO:teuthology.orchestra.run.smithi204.stdout: "op": 141, 2024-09-13T18:34:48.478 INFO:teuthology.orchestra.run.smithi204.stdout: "op_r": 30, 2024-09-13T18:34:48.478 INFO:teuthology.orchestra.run.smithi204.stdout: "op_w": 111, 2024-09-13T18:34:48.478 INFO:teuthology.orchestra.run.smithi204.stdout: "op_rmw": 0, 2024-09-13T18:34:48.478 INFO:teuthology.orchestra.run.smithi204.stdout: "op_pg": 0, 2024-09-13T18:34:48.479 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_stat": 0, 2024-09-13T18:34:48.479 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_create": 10, 2024-09-13T18:34:48.479 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_read": 0, 2024-09-13T18:34:48.479 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_write": 52, 2024-09-13T18:34:48.479 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writefull": 26, 2024-09-13T18:34:48.479 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writesame": 0, 2024-09-13T18:34:48.479 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_append": 0, 2024-09-13T18:34:48.479 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_zero": 0, 2024-09-13T18:34:48.479 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_truncate": 0, 2024-09-13T18:34:48.479 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_delete": 6, 2024-09-13T18:34:48.479 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_mapext": 0, 2024-09-13T18:34:48.479 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_sparse_read": 0, 2024-09-13T18:34:48.479 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_clonerange": 0, 2024-09-13T18:34:48.479 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_getxattr": 30, 2024-09-13T18:34:48.480 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_setxattr": 20, 2024-09-13T18:34:48.480 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_cmpxattr": 0, 2024-09-13T18:34:48.480 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_rmxattr": 0, 2024-09-13T18:34:48.480 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_resetxattrs": 0, 2024-09-13T18:34:48.480 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_call": 0, 2024-09-13T18:34:48.480 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_watch": 0, 2024-09-13T18:34:48.480 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_notify": 0, 2024-09-13T18:34:48.480 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_src_cmpxattr": 0, 2024-09-13T18:34:48.480 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls": 0, 2024-09-13T18:34:48.480 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls_filter": 0, 2024-09-13T18:34:48.480 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_other": 0, 2024-09-13T18:34:48.480 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_active": 0, 2024-09-13T18:34:48.481 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_send": 0, 2024-09-13T18:34:48.481 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_resend": 0, 2024-09-13T18:34:48.481 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_ping": 0, 2024-09-13T18:34:48.481 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_active": 0, 2024-09-13T18:34:48.481 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_send": 0, 2024-09-13T18:34:48.481 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_resend": 0, 2024-09-13T18:34:48.481 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_active": 0, 2024-09-13T18:34:48.481 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_send": 0, 2024-09-13T18:34:48.481 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_resend": 0, 2024-09-13T18:34:48.481 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_active": 0, 2024-09-13T18:34:48.481 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_send": 0, 2024-09-13T18:34:48.481 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_resend": 0, 2024-09-13T18:34:48.481 INFO:teuthology.orchestra.run.smithi204.stdout: "command_active": 0, 2024-09-13T18:34:48.481 INFO:teuthology.orchestra.run.smithi204.stdout: "command_send": 0, 2024-09-13T18:34:48.482 INFO:teuthology.orchestra.run.smithi204.stdout: "command_resend": 0, 2024-09-13T18:34:48.482 INFO:teuthology.orchestra.run.smithi204.stdout: "map_epoch": 106, 2024-09-13T18:34:48.482 INFO:teuthology.orchestra.run.smithi204.stdout: "map_full": 0, 2024-09-13T18:34:48.482 INFO:teuthology.orchestra.run.smithi204.stdout: "map_inc": 16, 2024-09-13T18:34:48.482 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_sessions": 9, 2024-09-13T18:34:48.482 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_open": 9, 2024-09-13T18:34:48.482 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_close": 0, 2024-09-13T18:34:48.482 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_laggy": 0, 2024-09-13T18:34:48.482 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_wr": 22, 2024-09-13T18:34:48.482 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_rd": 0, 2024-09-13T18:34:48.482 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_del": 2 2024-09-13T18:34:48.482 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.482 INFO:teuthology.orchestra.run.smithi204.stdout: "oft": { 2024-09-13T18:34:48.482 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_objs": 1, 2024-09-13T18:34:48.483 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_kv_pairs": 0, 2024-09-13T18:34:48.483 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_updates": 0, 2024-09-13T18:34:48.483 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_removes": 0 2024-09-13T18:34:48.483 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.483 INFO:teuthology.orchestra.run.smithi204.stdout: "osd-slow-ops": { 2024-09-13T18:34:48.483 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_ops_count": 0 2024-09-13T18:34:48.483 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.483 INFO:teuthology.orchestra.run.smithi204.stdout: "purge_queue": { 2024-09-13T18:34:48.483 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops": 0, 2024-09-13T18:34:48.483 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops_high_water": 0, 2024-09-13T18:34:48.483 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing": 0, 2024-09-13T18:34:48.483 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_high_water": 0, 2024-09-13T18:34:48.484 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed_ops": 0, 2024-09-13T18:34:48.484 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed": 0, 2024-09-13T18:34:48.484 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_item_in_journal": 0 2024-09-13T18:34:48.484 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.484 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-13T18:34:48.484 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-13T18:34:48.484 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-09-13T18:34:48.484 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-13T18:34:48.484 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 2137, 2024-09-13T18:34:48.484 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 2185280, 2024-09-13T18:34:48.484 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-13T18:34:48.484 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 2137, 2024-09-13T18:34:48.484 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-09-13T18:34:48.485 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-09-13T18:34:48.485 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2137, 2024-09-13T18:34:48.485 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 2185280, 2024-09-13T18:34:48.485 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-13T18:34:48.485 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:48.485 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:48.485 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:48.485 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-13T18:34:48.485 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.485 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_bytes": { 2024-09-13T18:34:48.485 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-13T18:34:48.485 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-09-13T18:34:48.485 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-13T18:34:48.485 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-09-13T18:34:48.486 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-09-13T18:34:48.486 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-13T18:34:48.486 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-09-13T18:34:48.486 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 141, 2024-09-13T18:34:48.486 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 255189, 2024-09-13T18:34:48.486 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 141, 2024-09-13T18:34:48.486 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 255189, 2024-09-13T18:34:48.486 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-13T18:34:48.486 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:48.486 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:48.486 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:48.486 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-13T18:34:48.486 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.486 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_ops": { 2024-09-13T18:34:48.487 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-13T18:34:48.487 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 1024, 2024-09-13T18:34:48.487 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-13T18:34:48.487 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-09-13T18:34:48.487 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-09-13T18:34:48.487 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-13T18:34:48.487 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-09-13T18:34:48.487 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 141, 2024-09-13T18:34:48.487 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 141, 2024-09-13T18:34:48.487 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 141, 2024-09-13T18:34:48.487 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 141, 2024-09-13T18:34:48.487 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-13T18:34:48.487 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:48.487 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:48.488 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:48.488 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-13T18:34:48.488 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.488 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle": { 2024-09-13T18:34:48.488 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-13T18:34:48.488 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-09-13T18:34:48.488 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-13T18:34:48.488 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-09-13T18:34:48.488 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-09-13T18:34:48.488 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-13T18:34:48.488 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-09-13T18:34:48.488 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-09-13T18:34:48.488 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-09-13T18:34:48.488 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 0, 2024-09-13T18:34:48.489 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 0, 2024-09-13T18:34:48.489 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-13T18:34:48.489 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:48.489 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:48.489 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:48.489 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-13T18:34:48.489 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:48.489 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle-0x5611c1a28500": { 2024-09-13T18:34:48.489 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-13T18:34:48.489 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-09-13T18:34:48.489 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-13T18:34:48.489 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 57, 2024-09-13T18:34:48.489 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 242258, 2024-09-13T18:34:48.489 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-13T18:34:48.490 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 57, 2024-09-13T18:34:48.490 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-09-13T18:34:48.490 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-09-13T18:34:48.490 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 52, 2024-09-13T18:34:48.490 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 242258, 2024-09-13T18:34:48.490 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-13T18:34:48.490 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:48.490 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:48.490 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:48.490 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-13T18:34:48.490 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-13T18:34:48.490 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-09-13T18:34:49.244 INFO:tasks.check_counter:Daemon mds.i mds.exported=1 2024-09-13T18:34:49.244 INFO:tasks.check_counter:Daemon mds.i mds.imported=2 2024-09-13T18:34:49.244 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-09-13T18:34:49.244 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=14 2024-09-13T18:34:49.244 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-09-13T18:34:49.244 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-09-13T18:34:49.245 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-09-13T18:34:49.245 DEBUG:tasks.check_counter:Getting stats from l 2024-09-13T18:34:49.245 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 shell --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-13T18:34:49.444 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/mon.c/config 2024-09-13T18:34:49.784 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:49 smithi204 ceph-mon[34777]: pgmap v385: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-13T18:34:49.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:49 smithi134 ceph-mon[34283]: pgmap v385: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-13T18:34:49.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:49 smithi114 ceph-mon[30897]: pgmap v385: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-13T18:34:51.069 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-09-13T18:34:51.069 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-0": { 2024-09-13T18:34:51.069 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 1725, 2024-09-13T18:34:51.069 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 1727, 2024-09-13T18:34:51.069 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 516685, 2024-09-13T18:34:51.070 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 536476, 2024-09-13T18:34:51.070 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 6, 2024-09-13T18:34:51.070 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 2, 2024-09-13T18:34:51.070 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.148674661, 2024-09-13T18:34:51.070 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.049654060, 2024-09-13T18:34:51.070 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.039645990, 2024-09-13T18:34:51.070 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.046946480, 2024-09-13T18:34:51.070 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T18:34:51.070 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 1725, 2024-09-13T18:34:51.071 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.063369200, 2024-09-13T18:34:51.071 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000036735 2024-09-13T18:34:51.071 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.071 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-09-13T18:34:51.071 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:51.071 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:51.071 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:51.071 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.071 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-13T18:34:51.071 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 0 2024-09-13T18:34:51.072 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.072 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-1": { 2024-09-13T18:34:51.072 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 751, 2024-09-13T18:34:51.072 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 937, 2024-09-13T18:34:51.072 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 187202, 2024-09-13T18:34:51.072 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 708451, 2024-09-13T18:34:51.072 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 7, 2024-09-13T18:34:51.072 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 5, 2024-09-13T18:34:51.072 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.074699013, 2024-09-13T18:34:51.073 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.026999849, 2024-09-13T18:34:51.073 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.017143495, 2024-09-13T18:34:51.073 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.024165432, 2024-09-13T18:34:51.073 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T18:34:51.073 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 935, 2024-09-13T18:34:51.073 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.064014155, 2024-09-13T18:34:51.073 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000068464 2024-09-13T18:34:51.073 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.073 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-09-13T18:34:51.074 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:51.074 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:51.074 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:51.074 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.074 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 3280, 2024-09-13T18:34:51.074 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 476688 2024-09-13T18:34:51.074 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.074 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-2": { 2024-09-13T18:34:51.074 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 293, 2024-09-13T18:34:51.074 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 269, 2024-09-13T18:34:51.074 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 1674812, 2024-09-13T18:34:51.074 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 145351, 2024-09-13T18:34:51.074 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 3, 2024-09-13T18:34:51.074 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 2, 2024-09-13T18:34:51.075 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.043856317, 2024-09-13T18:34:51.075 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.014420299, 2024-09-13T18:34:51.075 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.044258389, 2024-09-13T18:34:51.075 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.000026426, 2024-09-13T18:34:51.075 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-09-13T18:34:51.075 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 268, 2024-09-13T18:34:51.075 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.013277860, 2024-09-13T18:34:51.075 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000049544 2024-09-13T18:34:51.075 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.075 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-09-13T18:34:51.075 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:51.075 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:51.075 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:51.075 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.076 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 1674544, 2024-09-13T18:34:51.076 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 145040 2024-09-13T18:34:51.076 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.076 INFO:teuthology.orchestra.run.smithi204.stdout: "cct": { 2024-09-13T18:34:51.076 INFO:teuthology.orchestra.run.smithi204.stdout: "total_workers": 1, 2024-09-13T18:34:51.076 INFO:teuthology.orchestra.run.smithi204.stdout: "unhealthy_workers": 0 2024-09-13T18:34:51.076 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.076 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-MDSRank": { 2024-09-13T18:34:51.076 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-09-13T18:34:51.076 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-09-13T18:34:51.076 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 4081, 2024-09-13T18:34:51.076 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.102533274, 2024-09-13T18:34:51.076 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000025124 2024-09-13T18:34:51.077 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-13T18:34:51.077 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.077 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-PurgeQueue": { 2024-09-13T18:34:51.077 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-09-13T18:34:51.077 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-09-13T18:34:51.077 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:51.077 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:51.077 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:51.077 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-13T18:34:51.077 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.077 INFO:teuthology.orchestra.run.smithi204.stdout: "mds": { 2024-09-13T18:34:51.077 INFO:teuthology.orchestra.run.smithi204.stdout: "request": 0, 2024-09-13T18:34:51.077 INFO:teuthology.orchestra.run.smithi204.stdout: "reply": 0, 2024-09-13T18:34:51.077 INFO:teuthology.orchestra.run.smithi204.stdout: "reply_latency": { 2024-09-13T18:34:51.078 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:51.078 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:51.078 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:51.078 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.078 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_reply": 0, 2024-09-13T18:34:51.078 INFO:teuthology.orchestra.run.smithi204.stdout: "forward": 0, 2024-09-13T18:34:51.078 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_complete": 0, 2024-09-13T18:34:51.078 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_keys": 0, 2024-09-13T18:34:51.078 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_commit": 0, 2024-09-13T18:34:51.078 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_split": 0, 2024-09-13T18:34:51.078 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_merge": 0, 2024-09-13T18:34:51.078 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes": 0, 2024-09-13T18:34:51.078 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_top": 0, 2024-09-13T18:34:51.078 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_bottom": 0, 2024-09-13T18:34:51.079 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pin_tail": 0, 2024-09-13T18:34:51.079 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pinned": 0, 2024-09-13T18:34:51.079 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_expired": 0, 2024-09-13T18:34:51.079 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_with_caps": 0, 2024-09-13T18:34:51.079 INFO:teuthology.orchestra.run.smithi204.stdout: "caps": 0, 2024-09-13T18:34:51.079 INFO:teuthology.orchestra.run.smithi204.stdout: "subtrees": 1, 2024-09-13T18:34:51.079 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse": 0, 2024-09-13T18:34:51.079 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_hit": 0, 2024-09-13T18:34:51.079 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_forward": 0, 2024-09-13T18:34:51.079 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_discover": 0, 2024-09-13T18:34:51.079 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_dir_fetch": 0, 2024-09-13T18:34:51.079 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_remote_ino": 0, 2024-09-13T18:34:51.079 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_lock": 0, 2024-09-13T18:34:51.079 INFO:teuthology.orchestra.run.smithi204.stdout: "load_cent": 0, 2024-09-13T18:34:51.080 INFO:teuthology.orchestra.run.smithi204.stdout: "q": 0, 2024-09-13T18:34:51.080 INFO:teuthology.orchestra.run.smithi204.stdout: "exported": 0, 2024-09-13T18:34:51.080 INFO:teuthology.orchestra.run.smithi204.stdout: "exported_inodes": 0, 2024-09-13T18:34:51.080 INFO:teuthology.orchestra.run.smithi204.stdout: "imported": 0, 2024-09-13T18:34:51.080 INFO:teuthology.orchestra.run.smithi204.stdout: "imported_inodes": 0, 2024-09-13T18:34:51.080 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_dir_fetch": 0, 2024-09-13T18:34:51.080 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_backtrace_fetch": 0, 2024-09-13T18:34:51.080 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_peer_discover": 0, 2024-09-13T18:34:51.080 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rfiles": 0, 2024-09-13T18:34:51.080 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rbytes": 0, 2024-09-13T18:34:51.080 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rsnaps": 0, 2024-09-13T18:34:51.080 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_fetch": 0, 2024-09-13T18:34:51.080 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_set_tag": 0, 2024-09-13T18:34:51.080 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_repaired": 0, 2024-09-13T18:34:51.081 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_inotable_repaired": 0, 2024-09-13T18:34:51.081 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_inodes": 0, 2024-09-13T18:34:51.081 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_base_inodes": 0, 2024-09-13T18:34:51.081 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dirfrag_rstats": 0, 2024-09-13T18:34:51.081 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_file_inodes": 0, 2024-09-13T18:34:51.081 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_inode_file_caps": 0, 2024-09-13T18:34:51.081 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_revoke": 0, 2024-09-13T18:34:51.081 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_grant": 0, 2024-09-13T18:34:51.082 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_trunc": 0, 2024-09-13T18:34:51.082 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-13T18:34:51.084 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-13T18:34:51.084 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps": 0, 2024-09-13T18:34:51.084 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps_dirty": 0, 2024-09-13T18:34:51.084 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_cap_release": 0, 2024-09-13T18:34:51.084 INFO:teuthology.orchestra.run.smithi204.stdout: "process_request_cap_release": 0 2024-09-13T18:34:51.084 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.084 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_cache": { 2024-09-13T18:34:51.084 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update": 0, 2024-09-13T18:34:51.084 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update_receipt": 0, 2024-09-13T18:34:51.084 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_try_discover": 0, 2024-09-13T18:34:51.085 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_send_discover": 0, 2024-09-13T18:34:51.085 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_handle_discover": 0, 2024-09-13T18:34:51.085 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays": 0, 2024-09-13T18:34:51.085 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_delayed": 0, 2024-09-13T18:34:51.085 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_enqueuing": 0, 2024-09-13T18:34:51.085 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_created": 0, 2024-09-13T18:34:51.085 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_enqueued": 0, 2024-09-13T18:34:51.085 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_reintegrated": 0, 2024-09-13T18:34:51.085 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_migrated": 0, 2024-09-13T18:34:51.085 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_processing": 0, 2024-09-13T18:34:51.085 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_enqueued": 0, 2024-09-13T18:34:51.085 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_prioritized": 0, 2024-09-13T18:34:51.085 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_started": 0, 2024-09-13T18:34:51.086 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_completed": 0, 2024-09-13T18:34:51.086 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_enqueue_scrub": 0, 2024-09-13T18:34:51.086 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_exportdir": 0, 2024-09-13T18:34:51.086 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_flush": 0, 2024-09-13T18:34:51.086 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragmentdir": 0, 2024-09-13T18:34:51.086 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragstats": 0, 2024-09-13T18:34:51.086 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_inodestats": 0 2024-09-13T18:34:51.086 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.086 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_log": { 2024-09-13T18:34:51.086 INFO:teuthology.orchestra.run.smithi204.stdout: "evadd": 0, 2024-09-13T18:34:51.086 INFO:teuthology.orchestra.run.smithi204.stdout: "evex": 0, 2024-09-13T18:34:51.086 INFO:teuthology.orchestra.run.smithi204.stdout: "evtrm": 0, 2024-09-13T18:34:51.086 INFO:teuthology.orchestra.run.smithi204.stdout: "ev": 2, 2024-09-13T18:34:51.087 INFO:teuthology.orchestra.run.smithi204.stdout: "evexg": 0, 2024-09-13T18:34:51.087 INFO:teuthology.orchestra.run.smithi204.stdout: "evexd": 0, 2024-09-13T18:34:51.087 INFO:teuthology.orchestra.run.smithi204.stdout: "segadd": 0, 2024-09-13T18:34:51.087 INFO:teuthology.orchestra.run.smithi204.stdout: "segex": 0, 2024-09-13T18:34:51.087 INFO:teuthology.orchestra.run.smithi204.stdout: "segtrm": 0, 2024-09-13T18:34:51.087 INFO:teuthology.orchestra.run.smithi204.stdout: "seg": 1, 2024-09-13T18:34:51.087 INFO:teuthology.orchestra.run.smithi204.stdout: "segexg": 0, 2024-09-13T18:34:51.087 INFO:teuthology.orchestra.run.smithi204.stdout: "segexd": 0, 2024-09-13T18:34:51.087 INFO:teuthology.orchestra.run.smithi204.stdout: "expos": 4194304, 2024-09-13T18:34:51.088 INFO:teuthology.orchestra.run.smithi204.stdout: "wrpos": 4195338, 2024-09-13T18:34:51.088 INFO:teuthology.orchestra.run.smithi204.stdout: "rdpos": 4194844, 2024-09-13T18:34:51.088 INFO:teuthology.orchestra.run.smithi204.stdout: "jlat": { 2024-09-13T18:34:51.088 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:51.088 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:51.088 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:51.088 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.088 INFO:teuthology.orchestra.run.smithi204.stdout: "replayed": 2 2024-09-13T18:34:51.088 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.088 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_mem": { 2024-09-13T18:34:51.088 INFO:teuthology.orchestra.run.smithi204.stdout: "ino": 3, 2024-09-13T18:34:51.088 INFO:teuthology.orchestra.run.smithi204.stdout: "ino+": 3, 2024-09-13T18:34:51.088 INFO:teuthology.orchestra.run.smithi204.stdout: "ino-": 0, 2024-09-13T18:34:51.089 INFO:teuthology.orchestra.run.smithi204.stdout: "dir": 1, 2024-09-13T18:34:51.089 INFO:teuthology.orchestra.run.smithi204.stdout: "dir+": 1, 2024-09-13T18:34:51.089 INFO:teuthology.orchestra.run.smithi204.stdout: "dir-": 0, 2024-09-13T18:34:51.089 INFO:teuthology.orchestra.run.smithi204.stdout: "dn": 0, 2024-09-13T18:34:51.089 INFO:teuthology.orchestra.run.smithi204.stdout: "dn+": 0, 2024-09-13T18:34:51.089 INFO:teuthology.orchestra.run.smithi204.stdout: "dn-": 0, 2024-09-13T18:34:51.089 INFO:teuthology.orchestra.run.smithi204.stdout: "cap": 0, 2024-09-13T18:34:51.089 INFO:teuthology.orchestra.run.smithi204.stdout: "cap+": 0, 2024-09-13T18:34:51.089 INFO:teuthology.orchestra.run.smithi204.stdout: "cap-": 0, 2024-09-13T18:34:51.089 INFO:teuthology.orchestra.run.smithi204.stdout: "rss": 48404, 2024-09-13T18:34:51.089 INFO:teuthology.orchestra.run.smithi204.stdout: "heap": 207124 2024-09-13T18:34:51.089 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.089 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_server": { 2024-09-13T18:34:51.089 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_client_request": 0, 2024-09-13T18:34:51.090 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_server_request": 0, 2024-09-13T18:34:51.090 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_request": 0, 2024-09-13T18:34:51.090 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_session": 0, 2024-09-13T18:34:51.090 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_peer_request": 0, 2024-09-13T18:34:51.090 INFO:teuthology.orchestra.run.smithi204.stdout: "req_create_latency": { 2024-09-13T18:34:51.090 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:51.090 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:51.090 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:51.090 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.090 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getattr_latency": { 2024-09-13T18:34:51.090 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:51.090 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:51.090 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:51.090 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.091 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getfilelock_latency": { 2024-09-13T18:34:51.091 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:51.091 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:51.091 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:51.091 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.091 INFO:teuthology.orchestra.run.smithi204.stdout: "req_link_latency": { 2024-09-13T18:34:51.091 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:51.091 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:51.091 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:51.091 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.091 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookup_latency": { 2024-09-13T18:34:51.091 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:51.091 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:51.092 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:51.092 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.092 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookuphash_latency": { 2024-09-13T18:34:51.092 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:51.092 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:51.092 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:51.092 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.092 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupino_latency": { 2024-09-13T18:34:51.092 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:51.092 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:51.092 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:51.092 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.092 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupname_latency": { 2024-09-13T18:34:51.093 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:51.093 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:51.093 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:51.093 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.093 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupparent_latency": { 2024-09-13T18:34:51.093 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:51.093 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:51.093 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:51.093 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.093 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupsnap_latency": { 2024-09-13T18:34:51.093 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:51.093 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:51.093 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:51.094 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.094 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lssnap_latency": { 2024-09-13T18:34:51.094 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:51.094 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:51.094 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:51.094 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.094 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mkdir_latency": { 2024-09-13T18:34:51.094 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:51.094 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:51.094 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:51.094 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.094 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mknod_latency": { 2024-09-13T18:34:51.095 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:51.095 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:51.095 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:51.095 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.096 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mksnap_latency": { 2024-09-13T18:34:51.096 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:51.096 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:51.096 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:51.096 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.096 INFO:teuthology.orchestra.run.smithi204.stdout: "req_open_latency": { 2024-09-13T18:34:51.096 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:51.096 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:51.096 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:51.096 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.096 INFO:teuthology.orchestra.run.smithi204.stdout: "req_readdir_latency": { 2024-09-13T18:34:51.096 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:51.096 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:51.097 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:51.097 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.097 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rename_latency": { 2024-09-13T18:34:51.097 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:51.097 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:51.097 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:51.097 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.097 INFO:teuthology.orchestra.run.smithi204.stdout: "req_renamesnap_latency": { 2024-09-13T18:34:51.097 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:51.097 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:51.097 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:51.097 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.098 INFO:teuthology.orchestra.run.smithi204.stdout: "req_snapdiff_latency": { 2024-09-13T18:34:51.098 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:51.098 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:51.098 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:51.098 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.098 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmdir_latency": { 2024-09-13T18:34:51.098 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:51.098 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:51.098 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:51.098 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.098 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmsnap_latency": { 2024-09-13T18:34:51.098 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:51.099 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:51.099 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:51.099 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.099 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmxattr_latency": { 2024-09-13T18:34:51.099 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:51.099 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:51.099 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:51.099 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.099 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setattr_latency": { 2024-09-13T18:34:51.099 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:51.099 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:51.099 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:51.099 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.100 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setdirlayout_latency": { 2024-09-13T18:34:51.100 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:51.100 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:51.100 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:51.100 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.100 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setfilelock_latency": { 2024-09-13T18:34:51.100 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:51.100 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:51.100 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:51.100 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.100 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setlayout_latency": { 2024-09-13T18:34:51.100 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:51.101 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:51.101 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:51.101 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.101 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setxattr_latency": { 2024-09-13T18:34:51.101 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:51.101 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:51.101 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:51.101 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.101 INFO:teuthology.orchestra.run.smithi204.stdout: "req_symlink_latency": { 2024-09-13T18:34:51.101 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:51.101 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:51.101 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:51.101 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.101 INFO:teuthology.orchestra.run.smithi204.stdout: "req_unlink_latency": { 2024-09-13T18:34:51.102 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:51.102 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:51.102 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:51.102 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.102 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_revoke_eviction": 0, 2024-09-13T18:34:51.102 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_acquisition_throttle": 0, 2024-09-13T18:34:51.102 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getvxattr_latency": { 2024-09-13T18:34:51.102 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:51.102 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:51.102 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:51.102 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-13T18:34:51.102 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.103 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_sessions": { 2024-09-13T18:34:51.103 INFO:teuthology.orchestra.run.smithi204.stdout: "session_count": 1, 2024-09-13T18:34:51.103 INFO:teuthology.orchestra.run.smithi204.stdout: "session_add": 1, 2024-09-13T18:34:51.103 INFO:teuthology.orchestra.run.smithi204.stdout: "session_remove": 0, 2024-09-13T18:34:51.103 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_open": 1, 2024-09-13T18:34:51.103 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_stale": 0, 2024-09-13T18:34:51.103 INFO:teuthology.orchestra.run.smithi204.stdout: "total_load": 0, 2024-09-13T18:34:51.103 INFO:teuthology.orchestra.run.smithi204.stdout: "average_load": 0, 2024-09-13T18:34:51.103 INFO:teuthology.orchestra.run.smithi204.stdout: "avg_session_uptime": 1781, 2024-09-13T18:34:51.103 INFO:teuthology.orchestra.run.smithi204.stdout: "mdthresh_evicted": 0 2024-09-13T18:34:51.103 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.103 INFO:teuthology.orchestra.run.smithi204.stdout: "mempool": { 2024-09-13T18:34:51.103 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_bytes": 0, 2024-09-13T18:34:51.104 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_items": 0, 2024-09-13T18:34:51.104 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_bytes": 0, 2024-09-13T18:34:51.104 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_items": 0, 2024-09-13T18:34:51.104 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_bytes": 0, 2024-09-13T18:34:51.104 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_items": 0, 2024-09-13T18:34:51.104 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-13T18:34:51.104 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_items": 0, 2024-09-13T18:34:51.104 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-13T18:34:51.104 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_items": 0, 2024-09-13T18:34:51.104 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_bytes": 0, 2024-09-13T18:34:51.104 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_items": 0, 2024-09-13T18:34:51.104 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-13T18:34:51.105 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_items": 0, 2024-09-13T18:34:51.105 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_bytes": 0, 2024-09-13T18:34:51.105 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_items": 0, 2024-09-13T18:34:51.105 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_bytes": 0, 2024-09-13T18:34:51.105 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_items": 0, 2024-09-13T18:34:51.105 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-13T18:34:51.105 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_items": 0, 2024-09-13T18:34:51.105 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-13T18:34:51.105 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_items": 0, 2024-09-13T18:34:51.105 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_bytes": 0, 2024-09-13T18:34:51.105 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_items": 0, 2024-09-13T18:34:51.105 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_bytes": 0, 2024-09-13T18:34:51.105 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_items": 0, 2024-09-13T18:34:51.105 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-13T18:34:51.106 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_items": 0, 2024-09-13T18:34:51.106 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_bytes": 0, 2024-09-13T18:34:51.106 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_items": 0, 2024-09-13T18:34:51.106 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_bytes": 0, 2024-09-13T18:34:51.106 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_items": 0, 2024-09-13T18:34:51.106 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_bytes": 0, 2024-09-13T18:34:51.106 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_items": 0, 2024-09-13T18:34:51.106 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_bytes": 0, 2024-09-13T18:34:51.106 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_items": 0, 2024-09-13T18:34:51.106 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_bytes": 190161, 2024-09-13T18:34:51.106 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_items": 52, 2024-09-13T18:34:51.106 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_bytes": 0, 2024-09-13T18:34:51.106 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_items": 0, 2024-09-13T18:34:51.107 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_bytes": 0, 2024-09-13T18:34:51.107 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_items": 0, 2024-09-13T18:34:51.107 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_bytes": 0, 2024-09-13T18:34:51.107 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_items": 0, 2024-09-13T18:34:51.107 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_bytes": 0, 2024-09-13T18:34:51.107 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_items": 0, 2024-09-13T18:34:51.107 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_bytes": 9704, 2024-09-13T18:34:51.107 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_items": 196, 2024-09-13T18:34:51.107 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_bytes": 0, 2024-09-13T18:34:51.107 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_items": 0, 2024-09-13T18:34:51.107 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_bytes": 0, 2024-09-13T18:34:51.107 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_items": 0, 2024-09-13T18:34:51.107 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_bytes": 8368, 2024-09-13T18:34:51.108 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_items": 13, 2024-09-13T18:34:51.108 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_bytes": 0, 2024-09-13T18:34:51.108 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_items": 0, 2024-09-13T18:34:51.108 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_bytes": 0, 2024-09-13T18:34:51.108 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_items": 0 2024-09-13T18:34:51.108 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.108 INFO:teuthology.orchestra.run.smithi204.stdout: "objecter": { 2024-09-13T18:34:51.108 INFO:teuthology.orchestra.run.smithi204.stdout: "op_active": 0, 2024-09-13T18:34:51.108 INFO:teuthology.orchestra.run.smithi204.stdout: "op_laggy": 0, 2024-09-13T18:34:51.108 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send": 2471, 2024-09-13T18:34:51.108 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send_bytes": 21, 2024-09-13T18:34:51.108 INFO:teuthology.orchestra.run.smithi204.stdout: "op_resend": 3, 2024-09-13T18:34:51.108 INFO:teuthology.orchestra.run.smithi204.stdout: "op_reply": 2471, 2024-09-13T18:34:51.108 INFO:teuthology.orchestra.run.smithi204.stdout: "op_latency": { 2024-09-13T18:34:51.109 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2471, 2024-09-13T18:34:51.109 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 1.919973277, 2024-09-13T18:34:51.109 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000777002 2024-09-13T18:34:51.109 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.109 INFO:teuthology.orchestra.run.smithi204.stdout: "op_inflight": 0, 2024-09-13T18:34:51.109 INFO:teuthology.orchestra.run.smithi204.stdout: "oplen_avg": { 2024-09-13T18:34:51.109 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2471, 2024-09-13T18:34:51.109 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 2472 2024-09-13T18:34:51.109 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.109 INFO:teuthology.orchestra.run.smithi204.stdout: "op": 2471, 2024-09-13T18:34:51.109 INFO:teuthology.orchestra.run.smithi204.stdout: "op_r": 2471, 2024-09-13T18:34:51.109 INFO:teuthology.orchestra.run.smithi204.stdout: "op_w": 0, 2024-09-13T18:34:51.109 INFO:teuthology.orchestra.run.smithi204.stdout: "op_rmw": 0, 2024-09-13T18:34:51.110 INFO:teuthology.orchestra.run.smithi204.stdout: "op_pg": 0, 2024-09-13T18:34:51.110 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_stat": 1603, 2024-09-13T18:34:51.110 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_create": 0, 2024-09-13T18:34:51.110 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_read": 866, 2024-09-13T18:34:51.110 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_write": 0, 2024-09-13T18:34:51.110 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writefull": 0, 2024-09-13T18:34:51.110 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writesame": 0, 2024-09-13T18:34:51.110 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_append": 0, 2024-09-13T18:34:51.110 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_zero": 0, 2024-09-13T18:34:51.110 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_truncate": 0, 2024-09-13T18:34:51.110 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_delete": 0, 2024-09-13T18:34:51.110 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_mapext": 0, 2024-09-13T18:34:51.110 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_sparse_read": 0, 2024-09-13T18:34:51.110 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_clonerange": 0, 2024-09-13T18:34:51.111 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_getxattr": 1, 2024-09-13T18:34:51.111 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_setxattr": 0, 2024-09-13T18:34:51.111 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_cmpxattr": 0, 2024-09-13T18:34:51.111 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_rmxattr": 0, 2024-09-13T18:34:51.111 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_resetxattrs": 0, 2024-09-13T18:34:51.111 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_call": 0, 2024-09-13T18:34:51.111 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_watch": 0, 2024-09-13T18:34:51.111 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_notify": 0, 2024-09-13T18:34:51.111 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_src_cmpxattr": 0, 2024-09-13T18:34:51.111 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls": 0, 2024-09-13T18:34:51.111 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls_filter": 0, 2024-09-13T18:34:51.111 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_other": 0, 2024-09-13T18:34:51.111 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_active": 0, 2024-09-13T18:34:51.111 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_send": 0, 2024-09-13T18:34:51.112 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_resend": 0, 2024-09-13T18:34:51.112 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_ping": 0, 2024-09-13T18:34:51.112 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_active": 0, 2024-09-13T18:34:51.112 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_send": 0, 2024-09-13T18:34:51.112 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_resend": 0, 2024-09-13T18:34:51.112 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_active": 0, 2024-09-13T18:34:51.112 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_send": 0, 2024-09-13T18:34:51.112 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_resend": 0, 2024-09-13T18:34:51.112 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_active": 0, 2024-09-13T18:34:51.112 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_send": 0, 2024-09-13T18:34:51.112 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_resend": 0, 2024-09-13T18:34:51.112 INFO:teuthology.orchestra.run.smithi204.stdout: "command_active": 0, 2024-09-13T18:34:51.113 INFO:teuthology.orchestra.run.smithi204.stdout: "command_send": 0, 2024-09-13T18:34:51.113 INFO:teuthology.orchestra.run.smithi204.stdout: "command_resend": 0, 2024-09-13T18:34:51.113 INFO:teuthology.orchestra.run.smithi204.stdout: "map_epoch": 106, 2024-09-13T18:34:51.113 INFO:teuthology.orchestra.run.smithi204.stdout: "map_full": 0, 2024-09-13T18:34:51.113 INFO:teuthology.orchestra.run.smithi204.stdout: "map_inc": 15, 2024-09-13T18:34:51.113 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_sessions": 5, 2024-09-13T18:34:51.113 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_open": 5, 2024-09-13T18:34:51.113 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_close": 0, 2024-09-13T18:34:51.113 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_laggy": 0, 2024-09-13T18:34:51.113 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_wr": 0, 2024-09-13T18:34:51.113 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_rd": 2, 2024-09-13T18:34:51.113 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_del": 0 2024-09-13T18:34:51.113 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.114 INFO:teuthology.orchestra.run.smithi204.stdout: "oft": { 2024-09-13T18:34:51.114 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_objs": 0, 2024-09-13T18:34:51.114 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_kv_pairs": 0, 2024-09-13T18:34:51.114 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_updates": 0, 2024-09-13T18:34:51.114 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_removes": 0 2024-09-13T18:34:51.114 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.114 INFO:teuthology.orchestra.run.smithi204.stdout: "osd-slow-ops": { 2024-09-13T18:34:51.114 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_ops_count": 0 2024-09-13T18:34:51.114 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.114 INFO:teuthology.orchestra.run.smithi204.stdout: "purge_queue": { 2024-09-13T18:34:51.114 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops": 0, 2024-09-13T18:34:51.114 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops_high_water": 0, 2024-09-13T18:34:51.114 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing": 0, 2024-09-13T18:34:51.115 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_high_water": 0, 2024-09-13T18:34:51.115 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed_ops": 0, 2024-09-13T18:34:51.115 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed": 0, 2024-09-13T18:34:51.115 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_item_in_journal": 0 2024-09-13T18:34:51.115 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.115 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-13T18:34:51.115 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-13T18:34:51.115 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-09-13T18:34:51.115 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-13T18:34:51.115 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 2765, 2024-09-13T18:34:51.115 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 2111930, 2024-09-13T18:34:51.115 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-13T18:34:51.116 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 2765, 2024-09-13T18:34:51.116 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-09-13T18:34:51.116 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-09-13T18:34:51.116 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2765, 2024-09-13T18:34:51.116 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 2111930, 2024-09-13T18:34:51.116 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-13T18:34:51.116 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:51.116 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:51.116 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:51.116 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-13T18:34:51.116 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.116 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_bytes": { 2024-09-13T18:34:51.116 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-13T18:34:51.117 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-09-13T18:34:51.117 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-13T18:34:51.117 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-09-13T18:34:51.117 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-09-13T18:34:51.117 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-13T18:34:51.117 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-09-13T18:34:51.117 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 2471, 2024-09-13T18:34:51.117 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 1039, 2024-09-13T18:34:51.117 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2471, 2024-09-13T18:34:51.117 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 1039, 2024-09-13T18:34:51.117 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-13T18:34:51.117 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:51.117 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:51.118 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:51.118 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-13T18:34:51.118 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.118 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_ops": { 2024-09-13T18:34:51.118 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-13T18:34:51.118 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 1024, 2024-09-13T18:34:51.118 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-13T18:34:51.118 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-09-13T18:34:51.118 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-09-13T18:34:51.118 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-13T18:34:51.118 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-09-13T18:34:51.118 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 2471, 2024-09-13T18:34:51.118 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 2471, 2024-09-13T18:34:51.119 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2471, 2024-09-13T18:34:51.119 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 2471, 2024-09-13T18:34:51.119 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-13T18:34:51.119 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:51.119 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:51.119 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:51.119 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-13T18:34:51.119 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.119 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle": { 2024-09-13T18:34:51.119 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-13T18:34:51.119 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-09-13T18:34:51.119 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-13T18:34:51.119 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-09-13T18:34:51.120 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-09-13T18:34:51.120 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-13T18:34:51.120 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-09-13T18:34:51.120 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-09-13T18:34:51.120 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-09-13T18:34:51.120 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 0, 2024-09-13T18:34:51.120 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 0, 2024-09-13T18:34:51.120 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-13T18:34:51.120 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:51.120 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:51.120 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:51.120 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-13T18:34:51.120 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-13T18:34:51.120 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle-0x55fabbb1e820": { 2024-09-13T18:34:51.121 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-13T18:34:51.121 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-09-13T18:34:51.121 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-13T18:34:51.121 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-09-13T18:34:51.121 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-09-13T18:34:51.121 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-13T18:34:51.121 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-09-13T18:34:51.121 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-09-13T18:34:51.121 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-09-13T18:34:51.121 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 0, 2024-09-13T18:34:51.121 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 0, 2024-09-13T18:34:51.121 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-13T18:34:51.121 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-13T18:34:51.122 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-13T18:34:51.122 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-13T18:34:51.122 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-13T18:34:51.122 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-13T18:34:51.122 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-09-13T18:34:51.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:51 smithi134 ceph-mon[34283]: pgmap v386: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-13T18:34:51.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:51 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:34:51.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:51 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:34:51.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:51 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:34:51.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:51 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:34:51.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:51 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:34:51.884 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:51 smithi204 ceph-mon[34777]: pgmap v386: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-13T18:34:51.884 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:51 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:34:51.884 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:51 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:34:51.884 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:51 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:34:51.884 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:51 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:34:51.884 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:51 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:34:51.905 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-13T18:34:51.905 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-09-13T18:34:51.905 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-09-13T18:34:51.905 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-09-13T18:34:51.905 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-09-13T18:34:51.905 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-09-13T18:34:51.905 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-09-13T18:34:51.905 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-09-13T18:34:51.919 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-09-13T18:34:51.919 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-13T18:34:51.919 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:34:51.950 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-13T18:34:51.950 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:34:51.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:51 smithi114 ceph-mon[30897]: pgmap v386: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-13T18:34:51.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:51 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-13T18:34:51.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:51 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:34:51.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:51 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:34:51.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:51 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:34:51.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:51 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:34:51.987 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:34:52.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.130+0000 7f9e5f518640 1 -- 172.21.15.114:0/1935547529 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e58103fd0 msgr2=0x7f9e5810bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:52.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.130+0000 7f9e5f518640 1 --2- 172.21.15.114:0/1935547529 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e58103fd0 0x7f9e5810bb20 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f9e50009cd0 tx=0x7f9e5002a8e0 comp rx=0 tx=0).stop 2024-09-13T18:34:52.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.130+0000 7f9e5f518640 1 -- 172.21.15.114:0/1935547529 shutdown_connections 2024-09-13T18:34:52.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.130+0000 7f9e5f518640 1 --2- 172.21.15.114:0/1935547529 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e58103fd0 0x7f9e5810bb20 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:52.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.130+0000 7f9e5f518640 1 --2- 172.21.15.114:0/1935547529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e580ff830 0x7f9e58103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:52.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.130+0000 7f9e5f518640 1 --2- 172.21.15.114:0/1935547529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e580fee80 0x7f9e580ff260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:52.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.130+0000 7f9e5f518640 1 -- 172.21.15.114:0/1935547529 >> 172.21.15.114:0/1935547529 conn(0x7f9e58075c60 msgr2=0x7f9e580fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:52.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.131+0000 7f9e5f518640 1 -- 172.21.15.114:0/1935547529 shutdown_connections 2024-09-13T18:34:52.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.131+0000 7f9e5f518640 1 -- 172.21.15.114:0/1935547529 wait complete. 2024-09-13T18:34:52.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.131+0000 7f9e5f518640 1 Processor -- start 2024-09-13T18:34:52.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.131+0000 7f9e5f518640 1 -- start start 2024-09-13T18:34:52.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.131+0000 7f9e5f518640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e580fee80 0x7f9e58196160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:52.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.131+0000 7f9e5f518640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e58103fd0 0x7f9e581966a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:52.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.131+0000 7f9e5f518640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e5819aa90 0x7f9e5819af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:52.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.131+0000 7f9e5f518640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5819b480 con 0x7f9e580fee80 2024-09-13T18:34:52.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.131+0000 7f9e5f518640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5819b5f0 con 0x7f9e58103fd0 2024-09-13T18:34:52.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.131+0000 7f9e5f518640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5819b760 con 0x7f9e5819aa90 2024-09-13T18:34:52.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.132+0000 7f9e5d2b4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e580fee80 0x7f9e58196160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:52.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.132+0000 7f9e5d2b4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e580fee80 0x7f9e58196160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:56114/0 (socket says 172.21.15.114:56114) 2024-09-13T18:34:52.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.132+0000 7f9e5d2b4640 1 -- 172.21.15.114:0/1541447020 learned_addr learned my addr 172.21.15.114:0/1541447020 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:34:52.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.132+0000 7f9e5cab3640 1 --2- 172.21.15.114:0/1541447020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e58103fd0 0x7f9e581966a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:52.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.132+0000 7f9e5dab5640 1 --2- 172.21.15.114:0/1541447020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e5819aa90 0x7f9e5819af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:52.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.132+0000 7f9e5d2b4640 1 -- 172.21.15.114:0/1541447020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e5819aa90 msgr2=0x7f9e5819af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:52.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.132+0000 7f9e5d2b4640 1 --2- 172.21.15.114:0/1541447020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e5819aa90 0x7f9e5819af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:52.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.132+0000 7f9e5d2b4640 1 -- 172.21.15.114:0/1541447020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e58103fd0 msgr2=0x7f9e581966a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:52.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.132+0000 7f9e5d2b4640 1 --2- 172.21.15.114:0/1541447020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e58103fd0 0x7f9e581966a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:52.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.132+0000 7f9e5d2b4640 1 -- 172.21.15.114:0/1541447020 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e50009900 con 0x7f9e580fee80 2024-09-13T18:34:52.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.132+0000 7f9e5d2b4640 1 --2- 172.21.15.114:0/1541447020 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e580fee80 0x7f9e58196160 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f9e5400e9f0 tx=0x7f9e5400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:52.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.132+0000 7f9e4e7fc640 1 -- 172.21.15.114:0/1541447020 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e54009700 con 0x7f9e580fee80 2024-09-13T18:34:52.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.132+0000 7f9e4e7fc640 1 -- 172.21.15.114:0/1541447020 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9e5400cd80 con 0x7f9e580fee80 2024-09-13T18:34:52.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.132+0000 7f9e5f518640 1 -- 172.21.15.114:0/1541447020 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e581a2d40 con 0x7f9e580fee80 2024-09-13T18:34:52.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.132+0000 7f9e4e7fc640 1 -- 172.21.15.114:0/1541447020 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e540187e0 con 0x7f9e580fee80 2024-09-13T18:34:52.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.132+0000 7f9e5f518640 1 -- 172.21.15.114:0/1541447020 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e581a31e0 con 0x7f9e580fee80 2024-09-13T18:34:52.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.133+0000 7f9e5f518640 1 -- 172.21.15.114:0/1541447020 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e58100350 con 0x7f9e580fee80 2024-09-13T18:34:52.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.134+0000 7f9e4e7fc640 1 -- 172.21.15.114:0/1541447020 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f9e54010430 con 0x7f9e580fee80 2024-09-13T18:34:52.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.135+0000 7f9e4e7fc640 1 --2- 172.21.15.114:0/1541447020 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f9e440a1280 0x7f9e440a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:52.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.135+0000 7f9e5cab3640 1 --2- 172.21.15.114:0/1541447020 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f9e440a1280 0x7f9e440a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:52.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.135+0000 7f9e4e7fc640 1 -- 172.21.15.114:0/1541447020 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f9e54014070 con 0x7f9e580fee80 2024-09-13T18:34:52.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.135+0000 7f9e5cab3640 1 --2- 172.21.15.114:0/1541447020 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f9e440a1280 0x7f9e440a3740 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9e58197700 tx=0x7f9e4800b390 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:52.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.139+0000 7f9e4e7fc640 1 -- 172.21.15.114:0/1541447020 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f9e54052e20 con 0x7f9e580fee80 2024-09-13T18:34:52.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.304+0000 7f9e5f518640 1 -- 172.21.15.114:0/1541447020 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e580ff260 con 0x7f9e580fee80 2024-09-13T18:34:52.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.305+0000 7f9e4e7fc640 1 -- 172.21.15.114:0/1541447020 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+25265 (secure 0 0 0) 0x7f9e540964a0 con 0x7f9e580fee80 2024-09-13T18:34:52.308 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:34:52.308 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":106,"fsid":"97e7edf2-71fb-11ef-bceb-c7b262605968","created":"2024-09-13T18:12:36.360625+0000","modified":"2024-09-13T18:34:18.467182+0000","last_up_change":"2024-09-13T18:19:12.350370+0000","last_in_change":"2024-09-13T18:18:52.321596+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:15:40.129659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:20:00.024407+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:20:01.085549+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":105,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T18:20:04.334216+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":105,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9914f01a-9f6f-4878-b955-83570199c354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6803","nonce":3499525826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6805","nonce":3499525826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6809","nonce":3499525826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6807","nonce":3499525826}]},"public_addr":"172.21.15.114:6803/3499525826","cluster_addr":"172.21.15.114:6805/3499525826","heartbeat_back_addr":"172.21.15.114:6809/3499525826","heartbeat_front_addr":"172.21.15.114:6807/3499525826","state":["exists","up"]},{"osd":1,"uuid":"ef65e012-7469-4c20-b354-17022bca63c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6801","nonce":4264264866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6803","nonce":4264264866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6807","nonce":4264264866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6805","nonce":4264264866}]},"public_addr":"172.21.15.134:6801/4264264866","cluster_addr":"172.21.15.134:6803/4264264866","heartbeat_back_addr":"172.21.15.134:6807/4264264866","heartbeat_front_addr":"172.21.15.134:6805/4264264866","state":["exists","up"]},{"osd":2,"uuid":"578213fd-715f-4be7-91fa-254f3d0e3022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6801","nonce":2288807707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6803","nonce":2288807707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6807","nonce":2288807707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6805","nonce":2288807707}]},"public_addr":"172.21.15.204:6801/2288807707","cluster_addr":"172.21.15.204:6803/2288807707","heartbeat_back_addr":"172.21.15.204:6807/2288807707","heartbeat_front_addr":"172.21.15.204:6805/2288807707","state":["exists","up"]},{"osd":3,"uuid":"b64aab3f-1e52-4828-8397-74af5e05f2cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6811","nonce":2904204254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6813","nonce":2904204254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6817","nonce":2904204254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6815","nonce":2904204254}]},"public_addr":"172.21.15.114:6811/2904204254","cluster_addr":"172.21.15.114:6813/2904204254","heartbeat_back_addr":"172.21.15.114:6817/2904204254","heartbeat_front_addr":"172.21.15.114:6815/2904204254","state":["exists","up"]},{"osd":4,"uuid":"f38943d6-39c0-42e1-b427-cb6bb90ac38a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6809","nonce":1773945333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6811","nonce":1773945333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6815","nonce":1773945333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6813","nonce":1773945333}]},"public_addr":"172.21.15.134:6809/1773945333","cluster_addr":"172.21.15.134:6811/1773945333","heartbeat_back_addr":"172.21.15.134:6815/1773945333","heartbeat_front_addr":"172.21.15.134:6813/1773945333","state":["exists","up"]},{"osd":5,"uuid":"bb38f361-8e00-4178-9752-5ddbf3af20e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6809","nonce":2371778055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6811","nonce":2371778055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6815","nonce":2371778055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6813","nonce":2371778055}]},"public_addr":"172.21.15.204:6809/2371778055","cluster_addr":"172.21.15.204:6811/2371778055","heartbeat_back_addr":"172.21.15.204:6815/2371778055","heartbeat_front_addr":"172.21.15.204:6813/2371778055","state":["exists","up"]},{"osd":6,"uuid":"93d40a03-e7ff-4a3f-9efa-5c5403c9983d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6819","nonce":1484325044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6821","nonce":1484325044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6825","nonce":1484325044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6823","nonce":1484325044}]},"public_addr":"172.21.15.114:6819/1484325044","cluster_addr":"172.21.15.114:6821/1484325044","heartbeat_back_addr":"172.21.15.114:6825/1484325044","heartbeat_front_addr":"172.21.15.114:6823/1484325044","state":["exists","up"]},{"osd":7,"uuid":"982f629a-1b95-4be1-8e75-5ddce1ad77b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6817","nonce":799246614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6819","nonce":799246614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6823","nonce":799246614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6821","nonce":799246614}]},"public_addr":"172.21.15.134:6817/799246614","cluster_addr":"172.21.15.134:6819/799246614","heartbeat_back_addr":"172.21.15.134:6823/799246614","heartbeat_front_addr":"172.21.15.134:6821/799246614","state":["exists","up"]},{"osd":8,"uuid":"409beb9b-8227-4dad-a4a0-f0e5ce70dc5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6817","nonce":858331805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6819","nonce":858331805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6823","nonce":858331805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6821","nonce":858331805}]},"public_addr":"172.21.15.204:6817/858331805","cluster_addr":"172.21.15.204:6819/858331805","heartbeat_back_addr":"172.21.15.204:6823/858331805","heartbeat_front_addr":"172.21.15.204:6821/858331805","state":["exists","up"]},{"osd":9,"uuid":"a345a15c-b1fc-4877-b445-418eebf0d0ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6827","nonce":2172392854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6829","nonce":2172392854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6833","nonce":2172392854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6831","nonce":2172392854}]},"public_addr":"172.21.15.114:6827/2172392854","cluster_addr":"172.21.15.114:6829/2172392854","heartbeat_back_addr":"172.21.15.114:6833/2172392854","heartbeat_front_addr":"172.21.15.114:6831/2172392854","state":["exists","up"]},{"osd":10,"uuid":"464d995f-d90b-4caa-addb-1f7562bd74b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6825","nonce":3073513468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6827","nonce":3073513468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6831","nonce":3073513468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6829","nonce":3073513468}]},"public_addr":"172.21.15.134:6825/3073513468","cluster_addr":"172.21.15.134:6827/3073513468","heartbeat_back_addr":"172.21.15.134:6831/3073513468","heartbeat_front_addr":"172.21.15.134:6829/3073513468","state":["exists","up"]},{"osd":11,"uuid":"71dd794d-326c-477f-8d02-ee32bff1350d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6825","nonce":3239521765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6827","nonce":3239521765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6831","nonce":3239521765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6829","nonce":3239521765}]},"public_addr":"172.21.15.204:6825/3239521765","cluster_addr":"172.21.15.204:6827/3239521765","heartbeat_back_addr":"172.21.15.204:6831/3239521765","heartbeat_front_addr":"172.21.15.204:6829/3239521765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:14:54.542687+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:15:15.746055+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:15:38.371131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:01.383932+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:24.796237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:48.422214+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:11.284323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:34.570365+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:58.232855+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:18:21.609596+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:18:45.565859+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:19:10.017969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:6801/3396185541":"2024-09-14T18:22:28.057623+0000","172.21.15.114:6800/830738794":"2024-09-14T18:13:03.521116+0000","172.21.15.114:6801/830738794":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/1684954293":"2024-09-14T18:22:28.057623+0000","172.21.15.114:0/3544951322":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/722571999":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/2949422849":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2034742251":"2024-09-14T18:13:03.521116+0000","172.21.15.114:6800/4281866858":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/3407523320":"2024-09-14T18:22:28.057623+0000","172.21.15.114:6800/3396185541":"2024-09-14T18:22:28.057623+0000","172.21.15.114:0/4257828906":"2024-09-14T18:13:25.823027+0000","172.21.15.114:6801/4281866858":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2828364892":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2714091073":"2024-09-14T18:22:28.057623+0000","172.21.15.114:0/842237781":"2024-09-14T18:22:28.057623+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]},{"pool":4,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:34:52.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.307+0000 7f9e5f518640 1 -- 172.21.15.114:0/1541447020 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f9e440a1280 msgr2=0x7f9e440a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:52.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.307+0000 7f9e5f518640 1 --2- 172.21.15.114:0/1541447020 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f9e440a1280 0x7f9e440a3740 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9e58197700 tx=0x7f9e4800b390 comp rx=0 tx=0).stop 2024-09-13T18:34:52.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.307+0000 7f9e5f518640 1 -- 172.21.15.114:0/1541447020 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e580fee80 msgr2=0x7f9e58196160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:52.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.307+0000 7f9e5f518640 1 --2- 172.21.15.114:0/1541447020 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e580fee80 0x7f9e58196160 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f9e5400e9f0 tx=0x7f9e5400eeb0 comp rx=0 tx=0).stop 2024-09-13T18:34:52.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.307+0000 7f9e5f518640 1 -- 172.21.15.114:0/1541447020 shutdown_connections 2024-09-13T18:34:52.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.307+0000 7f9e5f518640 1 --2- 172.21.15.114:0/1541447020 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f9e440a1280 0x7f9e440a3740 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:52.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.307+0000 7f9e5f518640 1 --2- 172.21.15.114:0/1541447020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e5819aa90 0x7f9e5819af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:52.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.307+0000 7f9e5f518640 1 --2- 172.21.15.114:0/1541447020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e58103fd0 0x7f9e581966a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:52.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.307+0000 7f9e5f518640 1 --2- 172.21.15.114:0/1541447020 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e580fee80 0x7f9e58196160 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:52.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.307+0000 7f9e5f518640 1 -- 172.21.15.114:0/1541447020 >> 172.21.15.114:0/1541447020 conn(0x7f9e58075c60 msgr2=0x7f9e58067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:52.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.307+0000 7f9e5f518640 1 -- 172.21.15.114:0/1541447020 shutdown_connections 2024-09-13T18:34:52.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.308+0000 7f9e5f518640 1 -- 172.21.15.114:0/1541447020 wait complete. 2024-09-13T18:34:52.321 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T18:15:40.129659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T18:20:00.024407+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T18:20:01.085549+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 19, 'snap_epoch': 105, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T18:20:04.334216+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 19, 'snap_epoch': 105, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T18:34:52.322 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T18:34:52.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.421+0000 7fbc594d7640 1 -- 172.21.15.114:0/3127580039 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc54100640 msgr2=0x7fbc54104a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:52.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.421+0000 7fbc594d7640 1 --2- 172.21.15.114:0/3127580039 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc54100640 0x7fbc54104a70 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c009930 tx=0x7fbc3c02a420 comp rx=0 tx=0).stop 2024-09-13T18:34:52.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.421+0000 7fbc594d7640 1 -- 172.21.15.114:0/3127580039 shutdown_connections 2024-09-13T18:34:52.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.421+0000 7fbc594d7640 1 --2- 172.21.15.114:0/3127580039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc541051d0 0x7fbc5410bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:52.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.421+0000 7fbc594d7640 1 --2- 172.21.15.114:0/3127580039 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc54100640 0x7fbc54104a70 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:52.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.421+0000 7fbc594d7640 1 --2- 172.21.15.114:0/3127580039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc540ffc90 0x7fbc54100070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:52.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.421+0000 7fbc594d7640 1 -- 172.21.15.114:0/3127580039 >> 172.21.15.114:0/3127580039 conn(0x7fbc54074110 msgr2=0x7fbc54074520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:52.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.421+0000 7fbc594d7640 1 -- 172.21.15.114:0/3127580039 shutdown_connections 2024-09-13T18:34:52.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.421+0000 7fbc594d7640 1 -- 172.21.15.114:0/3127580039 wait complete. 2024-09-13T18:34:52.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.421+0000 7fbc594d7640 1 Processor -- start 2024-09-13T18:34:52.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.421+0000 7fbc594d7640 1 -- start start 2024-09-13T18:34:52.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.422+0000 7fbc594d7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc540ffc90 0x7fbc54195230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:52.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.422+0000 7fbc594d7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc541051d0 0x7fbc54195770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:52.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.422+0000 7fbc594d7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc54199b60 0x7fbc5419a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:52.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.422+0000 7fbc594d7640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5419a550 con 0x7fbc54199b60 2024-09-13T18:34:52.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.422+0000 7fbc594d7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5419a6c0 con 0x7fbc541051d0 2024-09-13T18:34:52.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.422+0000 7fbc594d7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5419a830 con 0x7fbc540ffc90 2024-09-13T18:34:52.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.422+0000 7fbc537fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc54199b60 0x7fbc5419a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:52.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.422+0000 7fbc537fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc54199b60 0x7fbc5419a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:56136/0 (socket says 172.21.15.114:56136) 2024-09-13T18:34:52.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.422+0000 7fbc537fe640 1 -- 172.21.15.114:0/2178637226 learned_addr learned my addr 172.21.15.114:0/2178637226 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:34:52.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.422+0000 7fbc4a7fc640 1 --2- 172.21.15.114:0/2178637226 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc541051d0 0x7fbc54195770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:52.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.422+0000 7fbc52ffd640 1 --2- 172.21.15.114:0/2178637226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc540ffc90 0x7fbc54195230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:52.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.422+0000 7fbc537fe640 1 -- 172.21.15.114:0/2178637226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc540ffc90 msgr2=0x7fbc54195230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:52.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.422+0000 7fbc537fe640 1 --2- 172.21.15.114:0/2178637226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc540ffc90 0x7fbc54195230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:52.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.422+0000 7fbc537fe640 1 -- 172.21.15.114:0/2178637226 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc541051d0 msgr2=0x7fbc54195770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:52.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.422+0000 7fbc537fe640 1 --2- 172.21.15.114:0/2178637226 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc541051d0 0x7fbc54195770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:52.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.422+0000 7fbc537fe640 1 -- 172.21.15.114:0/2178637226 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc3c009560 con 0x7fbc54199b60 2024-09-13T18:34:52.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.422+0000 7fbc537fe640 1 --2- 172.21.15.114:0/2178637226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc54199b60 0x7fbc5419a010 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c00eaa0 tx=0x7fbc4c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:52.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.422+0000 7fbc50ff9640 1 -- 172.21.15.114:0/2178637226 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c00cdb0 con 0x7fbc54199b60 2024-09-13T18:34:52.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.422+0000 7fbc594d7640 1 -- 172.21.15.114:0/2178637226 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc541a1e10 con 0x7fbc54199b60 2024-09-13T18:34:52.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.422+0000 7fbc50ff9640 1 -- 172.21.15.114:0/2178637226 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbc4c005020 con 0x7fbc54199b60 2024-09-13T18:34:52.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.422+0000 7fbc50ff9640 1 -- 172.21.15.114:0/2178637226 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c0116d0 con 0x7fbc54199b60 2024-09-13T18:34:52.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.422+0000 7fbc594d7640 1 -- 172.21.15.114:0/2178637226 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc541a2290 con 0x7fbc54199b60 2024-09-13T18:34:52.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.423+0000 7fbc594d7640 1 -- 172.21.15.114:0/2178637226 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc54068030 con 0x7fbc54199b60 2024-09-13T18:34:52.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.424+0000 7fbc50ff9640 1 -- 172.21.15.114:0/2178637226 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fbc4c004a40 con 0x7fbc54199b60 2024-09-13T18:34:52.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.425+0000 7fbc50ff9640 1 --2- 172.21.15.114:0/2178637226 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbc400a1200 0x7fbc400a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:52.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.425+0000 7fbc52ffd640 1 --2- 172.21.15.114:0/2178637226 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbc400a1200 0x7fbc400a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:52.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.425+0000 7fbc50ff9640 1 -- 172.21.15.114:0/2178637226 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fbc4c0cd130 con 0x7fbc54199b60 2024-09-13T18:34:52.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.426+0000 7fbc52ffd640 1 --2- 172.21.15.114:0/2178637226 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbc400a1200 0x7fbc400a36c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fbc44009bd0 tx=0x7fbc44005f90 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:52.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.429+0000 7fbc50ff9640 1 -- 172.21.15.114:0/2178637226 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fbc4c096290 con 0x7fbc54199b60 2024-09-13T18:34:52.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.593+0000 7fbc594d7640 1 -- 172.21.15.114:0/2178637226 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbc54100070 con 0x7fbc54199b60 2024-09-13T18:34:52.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.594+0000 7fbc50ff9640 1 -- 172.21.15.114:0/2178637226 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbc4c01f0a0 con 0x7fbc54199b60 2024-09-13T18:34:52.596 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 1 2024-09-13T18:34:52.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.596+0000 7fbc594d7640 1 -- 172.21.15.114:0/2178637226 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbc400a1200 msgr2=0x7fbc400a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:52.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.596+0000 7fbc594d7640 1 --2- 172.21.15.114:0/2178637226 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbc400a1200 0x7fbc400a36c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fbc44009bd0 tx=0x7fbc44005f90 comp rx=0 tx=0).stop 2024-09-13T18:34:52.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.596+0000 7fbc594d7640 1 -- 172.21.15.114:0/2178637226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc54199b60 msgr2=0x7fbc5419a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:52.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.596+0000 7fbc594d7640 1 --2- 172.21.15.114:0/2178637226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc54199b60 0x7fbc5419a010 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c00eaa0 tx=0x7fbc4c00ef60 comp rx=0 tx=0).stop 2024-09-13T18:34:52.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.596+0000 7fbc594d7640 1 -- 172.21.15.114:0/2178637226 shutdown_connections 2024-09-13T18:34:52.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.596+0000 7fbc594d7640 1 --2- 172.21.15.114:0/2178637226 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fbc400a1200 0x7fbc400a36c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:52.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.596+0000 7fbc594d7640 1 --2- 172.21.15.114:0/2178637226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc54199b60 0x7fbc5419a010 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:52.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.596+0000 7fbc594d7640 1 --2- 172.21.15.114:0/2178637226 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc541051d0 0x7fbc54195770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:52.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.596+0000 7fbc594d7640 1 --2- 172.21.15.114:0/2178637226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc540ffc90 0x7fbc54195230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:52.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.596+0000 7fbc594d7640 1 -- 172.21.15.114:0/2178637226 >> 172.21.15.114:0/2178637226 conn(0x7fbc54074110 msgr2=0x7fbc54102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:52.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.596+0000 7fbc594d7640 1 -- 172.21.15.114:0/2178637226 shutdown_connections 2024-09-13T18:34:52.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.596+0000 7fbc594d7640 1 -- 172.21.15.114:0/2178637226 wait complete. 2024-09-13T18:34:52.609 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:52 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:34:52.609 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:52 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:34:52.609 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:52 smithi114 ceph-mon[30897]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:34:52.609 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:52 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/1541447020' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:34:52.610 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T18:34:52.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.709+0000 7f817605a640 1 -- 172.21.15.114:0/3944638828 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8170100900 msgr2=0x7f8170104940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:52.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.709+0000 7f817605a640 1 --2- 172.21.15.114:0/3944638828 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8170100900 0x7f8170104940 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f8164009930 tx=0x7f816402a420 comp rx=0 tx=0).stop 2024-09-13T18:34:52.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.709+0000 7f817605a640 1 -- 172.21.15.114:0/3944638828 shutdown_connections 2024-09-13T18:34:52.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.709+0000 7f817605a640 1 --2- 172.21.15.114:0/3944638828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81701050a0 0x7f817010cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:52.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.709+0000 7f817605a640 1 --2- 172.21.15.114:0/3944638828 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8170100900 0x7f8170104940 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:52.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.709+0000 7f817605a640 1 --2- 172.21.15.114:0/3944638828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81700fff50 0x7f8170100330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:52.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.709+0000 7f817605a640 1 -- 172.21.15.114:0/3944638828 >> 172.21.15.114:0/3944638828 conn(0x7f81700fb9e0 msgr2=0x7f81700fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:52.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.709+0000 7f817605a640 1 -- 172.21.15.114:0/3944638828 shutdown_connections 2024-09-13T18:34:52.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.709+0000 7f817605a640 1 -- 172.21.15.114:0/3944638828 wait complete. 2024-09-13T18:34:52.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.710+0000 7f817605a640 1 Processor -- start 2024-09-13T18:34:52.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.710+0000 7f817605a640 1 -- start start 2024-09-13T18:34:52.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.710+0000 7f817605a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81700fff50 0x7f81701962e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:52.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.710+0000 7f817605a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81701050a0 0x7f8170196820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:52.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.710+0000 7f817605a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f817019abc0 0x7f817019b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:52.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.710+0000 7f817605a640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f817019b5b0 con 0x7f817019abc0 2024-09-13T18:34:52.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.710+0000 7f817605a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f817019b720 con 0x7f81701050a0 2024-09-13T18:34:52.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.710+0000 7f817605a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f817019b890 con 0x7f81700fff50 2024-09-13T18:34:52.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.710+0000 7f816ffff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f817019abc0 0x7f817019b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:52.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.710+0000 7f816ffff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f817019abc0 0x7f817019b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:56148/0 (socket says 172.21.15.114:56148) 2024-09-13T18:34:52.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.710+0000 7f816ffff640 1 -- 172.21.15.114:0/659626006 learned_addr learned my addr 172.21.15.114:0/659626006 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:34:52.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.711+0000 7f816effd640 1 --2- 172.21.15.114:0/659626006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81701050a0 0x7f8170196820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:52.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.711+0000 7f816f7fe640 1 --2- 172.21.15.114:0/659626006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81700fff50 0x7f81701962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:52.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.711+0000 7f816ffff640 1 -- 172.21.15.114:0/659626006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81700fff50 msgr2=0x7f81701962e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:52.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.711+0000 7f816ffff640 1 --2- 172.21.15.114:0/659626006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81700fff50 0x7f81701962e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:52.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.711+0000 7f816ffff640 1 -- 172.21.15.114:0/659626006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81701050a0 msgr2=0x7f8170196820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:52.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.711+0000 7f816ffff640 1 --2- 172.21.15.114:0/659626006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81701050a0 0x7f8170196820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:52.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.711+0000 7f816ffff640 1 -- 172.21.15.114:0/659626006 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8164009560 con 0x7f817019abc0 2024-09-13T18:34:52.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.711+0000 7f816ffff640 1 --2- 172.21.15.114:0/659626006 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f817019abc0 0x7f817019b070 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f816800eb10 tx=0x7f816800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:52.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.711+0000 7f816cff9640 1 -- 172.21.15.114:0/659626006 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8168004b20 con 0x7f817019abc0 2024-09-13T18:34:52.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.711+0000 7f816cff9640 1 -- 172.21.15.114:0/659626006 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8168004c80 con 0x7f817019abc0 2024-09-13T18:34:52.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.711+0000 7f817605a640 1 -- 172.21.15.114:0/659626006 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81701a2e70 con 0x7f817019abc0 2024-09-13T18:34:52.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.711+0000 7f816cff9640 1 -- 172.21.15.114:0/659626006 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8168005a10 con 0x7f817019abc0 2024-09-13T18:34:52.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.711+0000 7f817605a640 1 -- 172.21.15.114:0/659626006 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81701a3240 con 0x7f817019abc0 2024-09-13T18:34:52.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.712+0000 7f817605a640 1 -- 172.21.15.114:0/659626006 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8170101420 con 0x7f817019abc0 2024-09-13T18:34:52.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.713+0000 7f816cff9640 1 -- 172.21.15.114:0/659626006 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f8168004fd0 con 0x7f817019abc0 2024-09-13T18:34:52.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.714+0000 7f816cff9640 1 --2- 172.21.15.114:0/659626006 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f81580a1200 0x7f81580a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:52.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.714+0000 7f816f7fe640 1 --2- 172.21.15.114:0/659626006 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f81580a1200 0x7f81580a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:52.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.714+0000 7f816cff9640 1 -- 172.21.15.114:0/659626006 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f8168014070 con 0x7f817019abc0 2024-09-13T18:34:52.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.715+0000 7f816f7fe640 1 --2- 172.21.15.114:0/659626006 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f81580a1200 0x7f81580a36c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8160005fd0 tx=0x7f8160005850 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:52.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.717+0000 7f816cff9640 1 -- 172.21.15.114:0/659626006 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f81680968c0 con 0x7f817019abc0 2024-09-13T18:34:52.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:52 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:34:52.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:52 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:34:52.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:52 smithi134 ceph-mon[34283]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:34:52.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:52 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/1541447020' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:34:52.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.878+0000 7f817605a640 1 -- 172.21.15.114:0/659626006 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f81701975e0 con 0x7f817019abc0 2024-09-13T18:34:52.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.879+0000 7f816cff9640 1 -- 172.21.15.114:0/659626006 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v106) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8168099ee0 con 0x7f817019abc0 2024-09-13T18:34:52.881 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2024-09-13T18:34:52.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.881+0000 7f817605a640 1 -- 172.21.15.114:0/659626006 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f81580a1200 msgr2=0x7f81580a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:52.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.881+0000 7f817605a640 1 --2- 172.21.15.114:0/659626006 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f81580a1200 0x7f81580a36c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8160005fd0 tx=0x7f8160005850 comp rx=0 tx=0).stop 2024-09-13T18:34:52.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.881+0000 7f817605a640 1 -- 172.21.15.114:0/659626006 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f817019abc0 msgr2=0x7f817019b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:52.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.881+0000 7f817605a640 1 --2- 172.21.15.114:0/659626006 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f817019abc0 0x7f817019b070 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f816800eb10 tx=0x7f816800c2d0 comp rx=0 tx=0).stop 2024-09-13T18:34:52.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.881+0000 7f817605a640 1 -- 172.21.15.114:0/659626006 shutdown_connections 2024-09-13T18:34:52.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.881+0000 7f817605a640 1 --2- 172.21.15.114:0/659626006 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f81580a1200 0x7f81580a36c0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:52.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.881+0000 7f817605a640 1 --2- 172.21.15.114:0/659626006 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f817019abc0 0x7f817019b070 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:52.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.881+0000 7f817605a640 1 --2- 172.21.15.114:0/659626006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81701050a0 0x7f8170196820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:52.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.881+0000 7f817605a640 1 --2- 172.21.15.114:0/659626006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81700fff50 0x7f81701962e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:52.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.881+0000 7f817605a640 1 -- 172.21.15.114:0/659626006 >> 172.21.15.114:0/659626006 conn(0x7f81700fb9e0 msgr2=0x7f81700fcbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:52.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.881+0000 7f817605a640 1 -- 172.21.15.114:0/659626006 shutdown_connections 2024-09-13T18:34:52.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.881+0000 7f817605a640 1 -- 172.21.15.114:0/659626006 wait complete. 2024-09-13T18:34:52.894 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T18:34:52.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:52 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-13T18:34:52.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:52 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-13T18:34:52.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:52 smithi204 ceph-mon[34777]: from='mgr.15555 ' entity='mgr.x' 2024-09-13T18:34:52.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:52 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/1541447020' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:34:53.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.995+0000 7ff802d25640 1 -- 172.21.15.114:0/4053700595 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff7fc0fee60 msgr2=0x7ff7fc0ff240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:53.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.995+0000 7ff802d25640 1 --2- 172.21.15.114:0/4053700595 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff7fc0fee60 0x7ff7fc0ff240 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec009930 tx=0x7ff7ec02a420 comp rx=0 tx=0).stop 2024-09-13T18:34:53.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.995+0000 7ff802d25640 1 -- 172.21.15.114:0/4053700595 shutdown_connections 2024-09-13T18:34:53.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.995+0000 7ff802d25640 1 --2- 172.21.15.114:0/4053700595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7fc103fb0 0x7ff7fc10bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:53.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.995+0000 7ff802d25640 1 --2- 172.21.15.114:0/4053700595 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7fc0ff810 0x7ff7fc103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:53.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.995+0000 7ff802d25640 1 --2- 172.21.15.114:0/4053700595 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff7fc0fee60 0x7ff7fc0ff240 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:53.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.995+0000 7ff802d25640 1 -- 172.21.15.114:0/4053700595 >> 172.21.15.114:0/4053700595 conn(0x7ff7fc075c60 msgr2=0x7ff7fc0fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:53.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.995+0000 7ff802d25640 1 -- 172.21.15.114:0/4053700595 shutdown_connections 2024-09-13T18:34:53.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.995+0000 7ff802d25640 1 -- 172.21.15.114:0/4053700595 wait complete. 2024-09-13T18:34:53.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.995+0000 7ff802d25640 1 Processor -- start 2024-09-13T18:34:53.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.995+0000 7ff802d25640 1 -- start start 2024-09-13T18:34:53.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.995+0000 7ff802d25640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7fc0fee60 0x7ff7fc0fb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:53.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.995+0000 7ff802d25640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7fc103fb0 0x7ff7fc0fb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:53.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.995+0000 7ff802d25640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff7fc0fbb20 0x7ff7fc1a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:53.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.995+0000 7ff802d25640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7fc1a4980 con 0x7ff7fc0fbb20 2024-09-13T18:34:53.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.995+0000 7ff802d25640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7fc1a4af0 con 0x7ff7fc103fb0 2024-09-13T18:34:53.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.995+0000 7ff802d25640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7fc1a4cb0 con 0x7ff7fc0fee60 2024-09-13T18:34:53.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.996+0000 7ff8012c2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff7fc0fbb20 0x7ff7fc1a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:53.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.996+0000 7ff8012c2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff7fc0fbb20 0x7ff7fc1a43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:56156/0 (socket says 172.21.15.114:56156) 2024-09-13T18:34:53.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.996+0000 7ff8012c2640 1 -- 172.21.15.114:0/2313887324 learned_addr learned my addr 172.21.15.114:0/2313887324 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:34:53.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.996+0000 7ff7fbfff640 1 --2- 172.21.15.114:0/2313887324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7fc103fb0 0x7ff7fc0fb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:53.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.996+0000 7ff800ac1640 1 --2- 172.21.15.114:0/2313887324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7fc0fee60 0x7ff7fc0fb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:53.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.996+0000 7ff8012c2640 1 -- 172.21.15.114:0/2313887324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7fc0fee60 msgr2=0x7ff7fc0fb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:53.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.996+0000 7ff8012c2640 1 --2- 172.21.15.114:0/2313887324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7fc0fee60 0x7ff7fc0fb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:53.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.996+0000 7ff8012c2640 1 -- 172.21.15.114:0/2313887324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7fc103fb0 msgr2=0x7ff7fc0fb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:53.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.996+0000 7ff8012c2640 1 --2- 172.21.15.114:0/2313887324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7fc103fb0 0x7ff7fc0fb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:53.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.996+0000 7ff8012c2640 1 -- 172.21.15.114:0/2313887324 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7ec009560 con 0x7ff7fc0fbb20 2024-09-13T18:34:53.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.996+0000 7ff800ac1640 1 --2- 172.21.15.114:0/2313887324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7fc0fee60 0x7ff7fc0fb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:34:53.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.996+0000 7ff8012c2640 1 --2- 172.21.15.114:0/2313887324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff7fc0fbb20 0x7ff7fc1a43b0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7ff7f400cab0 tx=0x7ff7f400cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:53.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.996+0000 7ff7f9ffb640 1 -- 172.21.15.114:0/2313887324 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f400adc0 con 0x7ff7fc0fbb20 2024-09-13T18:34:53.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.996+0000 7ff7f9ffb640 1 -- 172.21.15.114:0/2313887324 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff7f400f8e0 con 0x7ff7fc0fbb20 2024-09-13T18:34:53.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.996+0000 7ff802d25640 1 -- 172.21.15.114:0/2313887324 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7fc1a4f90 con 0x7ff7fc0fbb20 2024-09-13T18:34:53.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.996+0000 7ff7f9ffb640 1 -- 172.21.15.114:0/2313887324 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f4018830 con 0x7ff7fc0fbb20 2024-09-13T18:34:53.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.996+0000 7ff802d25640 1 -- 172.21.15.114:0/2313887324 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7fc1a5410 con 0x7ff7fc0fbb20 2024-09-13T18:34:53.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.997+0000 7ff802d25640 1 -- 172.21.15.114:0/2313887324 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7fc100330 con 0x7ff7fc0fbb20 2024-09-13T18:34:53.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.999+0000 7ff7f9ffb640 1 -- 172.21.15.114:0/2313887324 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7ff7f400fd30 con 0x7ff7fc0fbb20 2024-09-13T18:34:53.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.999+0000 7ff7f9ffb640 1 --2- 172.21.15.114:0/2313887324 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff7e40a1230 0x7ff7e40a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:53.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.999+0000 7ff7f9ffb640 1 -- 172.21.15.114:0/2313887324 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ff7f40cd2e0 con 0x7ff7fc0fbb20 2024-09-13T18:34:53.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:52.999+0000 7ff800ac1640 1 --2- 172.21.15.114:0/2313887324 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff7e40a1230 0x7ff7e40a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:53.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.000+0000 7ff800ac1640 1 --2- 172.21.15.114:0/2313887324 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff7e40a1230 0x7ff7e40a36f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec0057c0 tx=0x7ff7ec000bc0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:53.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.003+0000 7ff7f9ffb640 1 -- 172.21.15.114:0/2313887324 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7ff7f4096390 con 0x7ff7fc0fbb20 2024-09-13T18:34:53.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.165+0000 7ff802d25640 1 -- 172.21.15.114:0/2313887324 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff7fc0ff240 con 0x7ff7fc0fbb20 2024-09-13T18:34:53.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.166+0000 7ff7f9ffb640 1 -- 172.21.15.114:0/2313887324 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v106) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff7f40999b0 con 0x7ff7fc0fbb20 2024-09-13T18:34:53.168 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 64 2024-09-13T18:34:53.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.168+0000 7ff802d25640 1 -- 172.21.15.114:0/2313887324 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff7e40a1230 msgr2=0x7ff7e40a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:53.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.168+0000 7ff802d25640 1 --2- 172.21.15.114:0/2313887324 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff7e40a1230 0x7ff7e40a36f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec0057c0 tx=0x7ff7ec000bc0 comp rx=0 tx=0).stop 2024-09-13T18:34:53.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.168+0000 7ff802d25640 1 -- 172.21.15.114:0/2313887324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff7fc0fbb20 msgr2=0x7ff7fc1a43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:53.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.168+0000 7ff802d25640 1 --2- 172.21.15.114:0/2313887324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff7fc0fbb20 0x7ff7fc1a43b0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7ff7f400cab0 tx=0x7ff7f400cf70 comp rx=0 tx=0).stop 2024-09-13T18:34:53.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.168+0000 7ff802d25640 1 -- 172.21.15.114:0/2313887324 shutdown_connections 2024-09-13T18:34:53.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.168+0000 7ff802d25640 1 --2- 172.21.15.114:0/2313887324 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7ff7e40a1230 0x7ff7e40a36f0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:53.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.168+0000 7ff802d25640 1 --2- 172.21.15.114:0/2313887324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff7fc0fbb20 0x7ff7fc1a43b0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:53.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.168+0000 7ff802d25640 1 --2- 172.21.15.114:0/2313887324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7fc103fb0 0x7ff7fc0fb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:53.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.168+0000 7ff802d25640 1 --2- 172.21.15.114:0/2313887324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7fc0fee60 0x7ff7fc0fb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:53.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.168+0000 7ff802d25640 1 -- 172.21.15.114:0/2313887324 >> 172.21.15.114:0/2313887324 conn(0x7ff7fc075c60 msgr2=0x7ff7fc067800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:53.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.168+0000 7ff802d25640 1 -- 172.21.15.114:0/2313887324 shutdown_connections 2024-09-13T18:34:53.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.168+0000 7ff802d25640 1 -- 172.21.15.114:0/2313887324 wait complete. 2024-09-13T18:34:53.183 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T18:34:53.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.282+0000 7f330b5e2640 1 -- 172.21.15.114:0/2677238691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3304100ee0 msgr2=0x7f33041012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:53.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.282+0000 7f330b5e2640 1 --2- 172.21.15.114:0/2677238691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3304100ee0 0x7f33041012c0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f3300009900 tx=0x7f330002a480 comp rx=0 tx=0).stop 2024-09-13T18:34:53.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.282+0000 7f330b5e2640 1 -- 172.21.15.114:0/2677238691 shutdown_connections 2024-09-13T18:34:53.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.282+0000 7f330b5e2640 1 --2- 172.21.15.114:0/2677238691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f330410a330 0x7f330410c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:53.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.282+0000 7f330b5e2640 1 --2- 172.21.15.114:0/2677238691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3304101800 0x7f3304109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:53.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.282+0000 7f330b5e2640 1 --2- 172.21.15.114:0/2677238691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3304100ee0 0x7f33041012c0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:53.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.282+0000 7f330b5e2640 1 -- 172.21.15.114:0/2677238691 >> 172.21.15.114:0/2677238691 conn(0x7f33040fa8d0 msgr2=0x7f33040fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:53.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.282+0000 7f330b5e2640 1 -- 172.21.15.114:0/2677238691 shutdown_connections 2024-09-13T18:34:53.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.282+0000 7f330b5e2640 1 -- 172.21.15.114:0/2677238691 wait complete. 2024-09-13T18:34:53.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.283+0000 7f330b5e2640 1 Processor -- start 2024-09-13T18:34:53.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.283+0000 7f330b5e2640 1 -- start start 2024-09-13T18:34:53.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.283+0000 7f330b5e2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3304100ee0 0x7f33040feb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:53.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.283+0000 7f330b5e2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3304101800 0x7f33040ff070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:53.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.283+0000 7f330b5e2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f330410a330 0x7f33040ff5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:53.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.283+0000 7f330b5e2640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33041a4490 con 0x7f3304101800 2024-09-13T18:34:53.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.283+0000 7f330b5e2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33041a4600 con 0x7f3304100ee0 2024-09-13T18:34:53.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.283+0000 7f330b5e2640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33041a47c0 con 0x7f330410a330 2024-09-13T18:34:53.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.283+0000 7f3308b7d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3304101800 0x7f33040ff070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:53.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.283+0000 7f330937e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3304100ee0 0x7f33040feb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:53.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.283+0000 7f3308b7d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3304101800 0x7f33040ff070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57198/0 (socket says 172.21.15.114:57198) 2024-09-13T18:34:53.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.283+0000 7f3308b7d640 1 -- 172.21.15.114:0/2421480011 learned_addr learned my addr 172.21.15.114:0/2421480011 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:34:53.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.283+0000 7f3309b7f640 1 --2- 172.21.15.114:0/2421480011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f330410a330 0x7f33040ff5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:53.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.283+0000 7f3308b7d640 1 -- 172.21.15.114:0/2421480011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f330410a330 msgr2=0x7f33040ff5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:53.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.283+0000 7f3308b7d640 1 --2- 172.21.15.114:0/2421480011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f330410a330 0x7f33040ff5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:53.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.283+0000 7f3308b7d640 1 -- 172.21.15.114:0/2421480011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3304100ee0 msgr2=0x7f33040feb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:53.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.283+0000 7f3308b7d640 1 --2- 172.21.15.114:0/2421480011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3304100ee0 0x7f33040feb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:53.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.284+0000 7f3308b7d640 1 -- 172.21.15.114:0/2421480011 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3300009560 con 0x7f3304101800 2024-09-13T18:34:53.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.284+0000 7f3308b7d640 1 --2- 172.21.15.114:0/2421480011 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3304101800 0x7f33040ff070 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f32f400c970 tx=0x7f32f400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:53.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.284+0000 7f32fa7fc640 1 -- 172.21.15.114:0/2421480011 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f4007bb0 con 0x7f3304101800 2024-09-13T18:34:53.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.284+0000 7f32fa7fc640 1 -- 172.21.15.114:0/2421480011 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f32f4007d10 con 0x7f3304101800 2024-09-13T18:34:53.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.284+0000 7f330b5e2640 1 -- 172.21.15.114:0/2421480011 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33041a4aa0 con 0x7f3304101800 2024-09-13T18:34:53.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.284+0000 7f32fa7fc640 1 -- 172.21.15.114:0/2421480011 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f4018660 con 0x7f3304101800 2024-09-13T18:34:53.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.284+0000 7f330b5e2640 1 -- 172.21.15.114:0/2421480011 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33041a4fb0 con 0x7f3304101800 2024-09-13T18:34:53.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.285+0000 7f330b5e2640 1 -- 172.21.15.114:0/2421480011 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3304068e50 con 0x7f3304101800 2024-09-13T18:34:53.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.287+0000 7f32fa7fc640 1 -- 172.21.15.114:0/2421480011 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f32f400f450 con 0x7f3304101800 2024-09-13T18:34:53.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.287+0000 7f32fa7fc640 1 --2- 172.21.15.114:0/2421480011 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f32f00a1230 0x7f32f00a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:53.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.288+0000 7f330937e640 1 --2- 172.21.15.114:0/2421480011 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f32f00a1230 0x7f32f00a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:53.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.288+0000 7f32fa7fc640 1 -- 172.21.15.114:0/2421480011 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f32f40cd380 con 0x7f3304101800 2024-09-13T18:34:53.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.288+0000 7f330937e640 1 --2- 172.21.15.114:0/2421480011 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f32f00a1230 0x7f32f00a36f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3300006010 tx=0x7f3300005a20 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:53.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.291+0000 7f32fa7fc640 1 -- 172.21.15.114:0/2421480011 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f32f40964c0 con 0x7f3304101800 2024-09-13T18:34:53.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.454+0000 7f330b5e2640 1 -- 172.21.15.114:0/2421480011 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f33041012c0 con 0x7f3304101800 2024-09-13T18:34:53.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.454+0000 7f32fa7fc640 1 -- 172.21.15.114:0/2421480011 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v106) v1 ==== 103+0+11 (secure 0 0 0) 0x7f32f4099ae0 con 0x7f3304101800 2024-09-13T18:34:53.457 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2024-09-13T18:34:53.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.456+0000 7f330b5e2640 1 -- 172.21.15.114:0/2421480011 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f32f00a1230 msgr2=0x7f32f00a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:53.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.456+0000 7f330b5e2640 1 --2- 172.21.15.114:0/2421480011 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f32f00a1230 0x7f32f00a36f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3300006010 tx=0x7f3300005a20 comp rx=0 tx=0).stop 2024-09-13T18:34:53.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.456+0000 7f330b5e2640 1 -- 172.21.15.114:0/2421480011 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3304101800 msgr2=0x7f33040ff070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:53.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.456+0000 7f330b5e2640 1 --2- 172.21.15.114:0/2421480011 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3304101800 0x7f33040ff070 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f32f400c970 tx=0x7f32f400ce30 comp rx=0 tx=0).stop 2024-09-13T18:34:53.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.456+0000 7f330b5e2640 1 -- 172.21.15.114:0/2421480011 shutdown_connections 2024-09-13T18:34:53.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.456+0000 7f330b5e2640 1 --2- 172.21.15.114:0/2421480011 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f32f00a1230 0x7f32f00a36f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:53.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.456+0000 7f330b5e2640 1 --2- 172.21.15.114:0/2421480011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f330410a330 0x7f33040ff5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:53.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.456+0000 7f330b5e2640 1 --2- 172.21.15.114:0/2421480011 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3304101800 0x7f33040ff070 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:53.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.456+0000 7f330b5e2640 1 --2- 172.21.15.114:0/2421480011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3304100ee0 0x7f33040feb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:53.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.456+0000 7f330b5e2640 1 -- 172.21.15.114:0/2421480011 >> 172.21.15.114:0/2421480011 conn(0x7f33040fa8d0 msgr2=0x7f33040fc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:53.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.456+0000 7f330b5e2640 1 -- 172.21.15.114:0/2421480011 shutdown_connections 2024-09-13T18:34:53.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.456+0000 7f330b5e2640 1 -- 172.21.15.114:0/2421480011 wait complete. 2024-09-13T18:34:53.470 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T18:34:53.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.569+0000 7fe238296640 1 -- 172.21.15.114:0/2892211132 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe2300fff80 msgr2=0x7fe230100360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:53.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.569+0000 7fe238296640 1 --2- 172.21.15.114:0/2892211132 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe2300fff80 0x7fe230100360 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fe228009cd0 tx=0x7fe22802a8e0 comp rx=0 tx=0).stop 2024-09-13T18:34:53.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.569+0000 7fe238296640 1 -- 172.21.15.114:0/2892211132 shutdown_connections 2024-09-13T18:34:53.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.569+0000 7fe238296640 1 --2- 172.21.15.114:0/2892211132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2301050d0 0x7fe23010cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:53.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.569+0000 7fe238296640 1 --2- 172.21.15.114:0/2892211132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe230100930 0x7fe230104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:53.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.569+0000 7fe238296640 1 --2- 172.21.15.114:0/2892211132 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe2300fff80 0x7fe230100360 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:53.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.569+0000 7fe238296640 1 -- 172.21.15.114:0/2892211132 >> 172.21.15.114:0/2892211132 conn(0x7fe2300fb9f0 msgr2=0x7fe2300fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:53.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.569+0000 7fe238296640 1 -- 172.21.15.114:0/2892211132 shutdown_connections 2024-09-13T18:34:53.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.569+0000 7fe238296640 1 -- 172.21.15.114:0/2892211132 wait complete. 2024-09-13T18:34:53.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.570+0000 7fe238296640 1 Processor -- start 2024-09-13T18:34:53.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.570+0000 7fe238296640 1 -- start start 2024-09-13T18:34:53.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.570+0000 7fe238296640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe2300fff80 0x7fe230196200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:53.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.570+0000 7fe238296640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2301050d0 0x7fe230196740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:53.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.570+0000 7fe238296640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe23019ab30 0x7fe23019afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:53.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.570+0000 7fe238296640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23019b520 con 0x7fe2300fff80 2024-09-13T18:34:53.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.570+0000 7fe238296640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23019b690 con 0x7fe23019ab30 2024-09-13T18:34:53.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.570+0000 7fe238296640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23019b800 con 0x7fe2301050d0 2024-09-13T18:34:53.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.570+0000 7fe236032640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe2300fff80 0x7fe230196200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:53.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.570+0000 7fe236032640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe2300fff80 0x7fe230196200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57220/0 (socket says 172.21.15.114:57220) 2024-09-13T18:34:53.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.570+0000 7fe236032640 1 -- 172.21.15.114:0/4116386735 learned_addr learned my addr 172.21.15.114:0/4116386735 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:34:53.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.571+0000 7fe236032640 1 -- 172.21.15.114:0/4116386735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2301050d0 msgr2=0x7fe230196740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:53.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.571+0000 7fe235831640 1 --2- 172.21.15.114:0/4116386735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2301050d0 0x7fe230196740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:53.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.571+0000 7fe236833640 1 --2- 172.21.15.114:0/4116386735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe23019ab30 0x7fe23019afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:53.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.571+0000 7fe236032640 1 --2- 172.21.15.114:0/4116386735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2301050d0 0x7fe230196740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:53.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.571+0000 7fe236032640 1 -- 172.21.15.114:0/4116386735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe23019ab30 msgr2=0x7fe23019afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:53.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.571+0000 7fe236032640 1 --2- 172.21.15.114:0/4116386735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe23019ab30 0x7fe23019afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:53.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.571+0000 7fe236032640 1 -- 172.21.15.114:0/4116386735 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe228009900 con 0x7fe2300fff80 2024-09-13T18:34:53.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.571+0000 7fe236032640 1 --2- 172.21.15.114:0/4116386735 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe2300fff80 0x7fe230196200 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fe228004af0 tx=0x7fe228030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:53.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.571+0000 7fe2277fe640 1 -- 172.21.15.114:0/4116386735 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe228031c00 con 0x7fe2300fff80 2024-09-13T18:34:53.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.571+0000 7fe2277fe640 1 -- 172.21.15.114:0/4116386735 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe228003e00 con 0x7fe2300fff80 2024-09-13T18:34:53.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.571+0000 7fe2277fe640 1 -- 172.21.15.114:0/4116386735 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe228045410 con 0x7fe2300fff80 2024-09-13T18:34:53.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.571+0000 7fe238296640 1 -- 172.21.15.114:0/4116386735 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2301a2de0 con 0x7fe2300fff80 2024-09-13T18:34:53.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.571+0000 7fe238296640 1 -- 172.21.15.114:0/4116386735 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2301a3200 con 0x7fe2300fff80 2024-09-13T18:34:53.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.572+0000 7fe238296640 1 -- 172.21.15.114:0/4116386735 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe230101450 con 0x7fe2300fff80 2024-09-13T18:34:53.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.573+0000 7fe2277fe640 1 -- 172.21.15.114:0/4116386735 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fe22800a040 con 0x7fe2300fff80 2024-09-13T18:34:53.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.574+0000 7fe2277fe640 1 --2- 172.21.15.114:0/4116386735 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe21c0a1280 0x7fe21c0a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:53.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.574+0000 7fe235831640 1 --2- 172.21.15.114:0/4116386735 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe21c0a1280 0x7fe21c0a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:53.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.574+0000 7fe2277fe640 1 -- 172.21.15.114:0/4116386735 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fe2280ee440 con 0x7fe2300fff80 2024-09-13T18:34:53.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.574+0000 7fe235831640 1 --2- 172.21.15.114:0/4116386735 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe21c0a1280 0x7fe21c0a3740 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe2301977a0 tx=0x7fe220005c60 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:53.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.577+0000 7fe2277fe640 1 -- 172.21.15.114:0/4116386735 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fe2280b74f0 con 0x7fe2300fff80 2024-09-13T18:34:53.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:53 smithi114 ceph-mon[30897]: pgmap v387: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-13T18:34:53.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:53 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2178637226' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T18:34:53.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:53 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/659626006' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T18:34:53.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:53 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2313887324' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T18:34:53.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:53 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2421480011' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T18:34:53.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.752+0000 7fe238296640 1 -- 172.21.15.114:0/4116386735 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe2301975b0 con 0x7fe2300fff80 2024-09-13T18:34:53.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.752+0000 7fe2277fe640 1 -- 172.21.15.114:0/4116386735 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v106) v1 ==== 81+0+958 (secure 0 0 0) 0x7fe2280bab10 con 0x7fe2300fff80 2024-09-13T18:34:53.755 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6801/3396185541 2024-09-14T18:22:28.057623+0000 2024-09-13T18:34:53.755 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6800/830738794 2024-09-14T18:13:03.521116+0000 2024-09-13T18:34:53.755 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6801/830738794 2024-09-14T18:13:03.521116+0000 2024-09-13T18:34:53.755 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/1684954293 2024-09-14T18:22:28.057623+0000 2024-09-13T18:34:53.755 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/3544951322 2024-09-14T18:13:03.521116+0000 2024-09-13T18:34:53.755 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/722571999 2024-09-14T18:13:03.521116+0000 2024-09-13T18:34:53.755 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/2949422849 2024-09-14T18:13:25.823027+0000 2024-09-13T18:34:53.755 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/2034742251 2024-09-14T18:13:03.521116+0000 2024-09-13T18:34:53.755 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6800/4281866858 2024-09-14T18:13:25.823027+0000 2024-09-13T18:34:53.755 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/3407523320 2024-09-14T18:22:28.057623+0000 2024-09-13T18:34:53.755 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6800/3396185541 2024-09-14T18:22:28.057623+0000 2024-09-13T18:34:53.756 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/4257828906 2024-09-14T18:13:25.823027+0000 2024-09-13T18:34:53.756 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6801/4281866858 2024-09-14T18:13:25.823027+0000 2024-09-13T18:34:53.756 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/2828364892 2024-09-14T18:13:25.823027+0000 2024-09-13T18:34:53.756 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/2714091073 2024-09-14T18:22:28.057623+0000 2024-09-13T18:34:53.756 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/842237781 2024-09-14T18:22:28.057623+0000 2024-09-13T18:34:53.757 INFO:teuthology.orchestra.run.smithi114.stderr:listed 16 entries 2024-09-13T18:34:53.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.754+0000 7fe238296640 1 -- 172.21.15.114:0/4116386735 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe21c0a1280 msgr2=0x7fe21c0a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:53.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.754+0000 7fe238296640 1 --2- 172.21.15.114:0/4116386735 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe21c0a1280 0x7fe21c0a3740 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe2301977a0 tx=0x7fe220005c60 comp rx=0 tx=0).stop 2024-09-13T18:34:53.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.754+0000 7fe238296640 1 -- 172.21.15.114:0/4116386735 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe2300fff80 msgr2=0x7fe230196200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:53.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.754+0000 7fe238296640 1 --2- 172.21.15.114:0/4116386735 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe2300fff80 0x7fe230196200 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fe228004af0 tx=0x7fe228030ec0 comp rx=0 tx=0).stop 2024-09-13T18:34:53.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.754+0000 7fe238296640 1 -- 172.21.15.114:0/4116386735 shutdown_connections 2024-09-13T18:34:53.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.754+0000 7fe238296640 1 --2- 172.21.15.114:0/4116386735 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe21c0a1280 0x7fe21c0a3740 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:53.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.754+0000 7fe238296640 1 --2- 172.21.15.114:0/4116386735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe23019ab30 0x7fe23019afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:53.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.754+0000 7fe238296640 1 --2- 172.21.15.114:0/4116386735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2301050d0 0x7fe230196740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:53.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.754+0000 7fe238296640 1 --2- 172.21.15.114:0/4116386735 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe2300fff80 0x7fe230196200 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:53.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.754+0000 7fe238296640 1 -- 172.21.15.114:0/4116386735 >> 172.21.15.114:0/4116386735 conn(0x7fe2300fb9f0 msgr2=0x7fe2300fce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:53.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.754+0000 7fe238296640 1 -- 172.21.15.114:0/4116386735 shutdown_connections 2024-09-13T18:34:53.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.754+0000 7fe238296640 1 -- 172.21.15.114:0/4116386735 wait complete. 2024-09-13T18:34:53.767 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-13T18:34:53.767 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:34:53.784 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:34:53.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:53 smithi134 ceph-mon[34283]: pgmap v387: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-13T18:34:53.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:53 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/2178637226' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T18:34:53.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:53 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/659626006' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T18:34:53.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:53 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/2313887324' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T18:34:53.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:53 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/2421480011' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T18:34:53.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.921+0000 7f0dd6f25640 1 -- 172.21.15.114:0/4142812752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0dd00ff860 msgr2=0x7f0dd01038a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:53.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.921+0000 7f0dd6f25640 1 --2- 172.21.15.114:0/4142812752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0dd00ff860 0x7f0dd01038a0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f0db8009930 tx=0x7f0db802a420 comp rx=0 tx=0).stop 2024-09-13T18:34:53.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.921+0000 7f0dd6f25640 1 -- 172.21.15.114:0/4142812752 shutdown_connections 2024-09-13T18:34:53.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.921+0000 7f0dd6f25640 1 --2- 172.21.15.114:0/4142812752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dd0104000 0x7f0dd010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:53.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.921+0000 7f0dd6f25640 1 --2- 172.21.15.114:0/4142812752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0dd00ff860 0x7f0dd01038a0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:53.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.921+0000 7f0dd6f25640 1 --2- 172.21.15.114:0/4142812752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0dd00feeb0 0x7f0dd00ff290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:53.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.921+0000 7f0dd6f25640 1 -- 172.21.15.114:0/4142812752 >> 172.21.15.114:0/4142812752 conn(0x7f0dd00fa900 msgr2=0x7f0dd00fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:53.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.921+0000 7f0dd6f25640 1 -- 172.21.15.114:0/4142812752 shutdown_connections 2024-09-13T18:34:53.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.921+0000 7f0dd6f25640 1 -- 172.21.15.114:0/4142812752 wait complete. 2024-09-13T18:34:53.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.921+0000 7f0dd6f25640 1 Processor -- start 2024-09-13T18:34:53.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.922+0000 7f0dd6f25640 1 -- start start 2024-09-13T18:34:53.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.922+0000 7f0dd6f25640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0dd00feeb0 0x7f0dd0100940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:53.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.922+0000 7f0dd6f25640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0dd0104000 0x7f0dd0100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:53.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.922+0000 7f0dd6f25640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dd01013c0 0x7f0dd01a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:53.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.922+0000 7f0dd6f25640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd01a4a60 con 0x7f0dd0104000 2024-09-13T18:34:53.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.922+0000 7f0dd6f25640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd01a4bd0 con 0x7f0dd01013c0 2024-09-13T18:34:53.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.922+0000 7f0dd6f25640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd01a4d90 con 0x7f0dd00feeb0 2024-09-13T18:34:53.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.922+0000 7f0dcffff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0dd0104000 0x7f0dd0100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:53.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.922+0000 7f0dcffff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0dd0104000 0x7f0dd0100e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57226/0 (socket says 172.21.15.114:57226) 2024-09-13T18:34:53.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.922+0000 7f0dcffff640 1 -- 172.21.15.114:0/2055128411 learned_addr learned my addr 172.21.15.114:0/2055128411 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:34:53.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.922+0000 7f0dd4cc1640 1 --2- 172.21.15.114:0/2055128411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0dd00feeb0 0x7f0dd0100940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:53.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.922+0000 7f0dd54c2640 1 --2- 172.21.15.114:0/2055128411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dd01013c0 0x7f0dd01a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:53.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.922+0000 7f0dcffff640 1 -- 172.21.15.114:0/2055128411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0dd00feeb0 msgr2=0x7f0dd0100940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:53.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.922+0000 7f0dcffff640 1 --2- 172.21.15.114:0/2055128411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0dd00feeb0 0x7f0dd0100940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:53.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.922+0000 7f0dcffff640 1 -- 172.21.15.114:0/2055128411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dd01013c0 msgr2=0x7f0dd01a4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:53.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.922+0000 7f0dcffff640 1 --2- 172.21.15.114:0/2055128411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dd01013c0 0x7f0dd01a4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:53.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.922+0000 7f0dcffff640 1 -- 172.21.15.114:0/2055128411 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db8009560 con 0x7f0dd0104000 2024-09-13T18:34:53.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.922+0000 7f0dcffff640 1 --2- 172.21.15.114:0/2055128411 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0dd0104000 0x7f0dd0100e80 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f0db8005ab0 tx=0x7f0db8032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:53.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.922+0000 7f0dcdffb640 1 -- 172.21.15.114:0/2055128411 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db8032e10 con 0x7f0dd0104000 2024-09-13T18:34:53.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.923+0000 7f0dcdffb640 1 -- 172.21.15.114:0/2055128411 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0db80046c0 con 0x7f0dd0104000 2024-09-13T18:34:53.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.923+0000 7f0dcdffb640 1 -- 172.21.15.114:0/2055128411 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db8003850 con 0x7f0dd0104000 2024-09-13T18:34:53.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.923+0000 7f0dd6f25640 1 -- 172.21.15.114:0/2055128411 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dd01a5010 con 0x7f0dd0104000 2024-09-13T18:34:53.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.923+0000 7f0dd6f25640 1 -- 172.21.15.114:0/2055128411 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dd01a54c0 con 0x7f0dd0104000 2024-09-13T18:34:53.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.924+0000 7f0dd6f25640 1 -- 172.21.15.114:0/2055128411 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dd0100380 con 0x7f0dd0104000 2024-09-13T18:34:53.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.926+0000 7f0dcdffb640 1 -- 172.21.15.114:0/2055128411 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f0db8004830 con 0x7f0dd0104000 2024-09-13T18:34:53.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.926+0000 7f0dcdffb640 1 --2- 172.21.15.114:0/2055128411 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0dbc0a1280 0x7f0dbc0a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:53.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.927+0000 7f0dd4cc1640 1 --2- 172.21.15.114:0/2055128411 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0dbc0a1280 0x7f0dbc0a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:53.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.927+0000 7f0dcdffb640 1 -- 172.21.15.114:0/2055128411 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f0db80ee070 con 0x7f0dd0104000 2024-09-13T18:34:53.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.927+0000 7f0dd4cc1640 1 --2- 172.21.15.114:0/2055128411 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0dbc0a1280 0x7f0dbc0a3740 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f0dc0005fd0 tx=0x7f0dc0009680 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:53.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:53.929+0000 7f0dcdffb640 1 -- 172.21.15.114:0/2055128411 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f0db80b70c0 con 0x7f0dd0104000 2024-09-13T18:34:53.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:53 smithi204 ceph-mon[34777]: pgmap v387: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-13T18:34:53.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:53 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/2178637226' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T18:34:53.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:53 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/659626006' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T18:34:53.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:53 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/2313887324' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T18:34:53.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:53 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/2421480011' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T18:34:54.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.094+0000 7f0dd6f25640 1 -- 172.21.15.114:0/2055128411 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0dd00ff290 con 0x7f0dd0104000 2024-09-13T18:34:54.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.095+0000 7f0dcdffb640 1 -- 172.21.15.114:0/2055128411 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+25265 (secure 0 0 0) 0x7f0db804a090 con 0x7f0dd0104000 2024-09-13T18:34:54.098 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:34:54.099 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":106,"fsid":"97e7edf2-71fb-11ef-bceb-c7b262605968","created":"2024-09-13T18:12:36.360625+0000","modified":"2024-09-13T18:34:18.467182+0000","last_up_change":"2024-09-13T18:19:12.350370+0000","last_in_change":"2024-09-13T18:18:52.321596+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:15:40.129659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:20:00.024407+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:20:01.085549+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":105,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T18:20:04.334216+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":105,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9914f01a-9f6f-4878-b955-83570199c354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6803","nonce":3499525826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6805","nonce":3499525826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6809","nonce":3499525826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6807","nonce":3499525826}]},"public_addr":"172.21.15.114:6803/3499525826","cluster_addr":"172.21.15.114:6805/3499525826","heartbeat_back_addr":"172.21.15.114:6809/3499525826","heartbeat_front_addr":"172.21.15.114:6807/3499525826","state":["exists","up"]},{"osd":1,"uuid":"ef65e012-7469-4c20-b354-17022bca63c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6801","nonce":4264264866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6803","nonce":4264264866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6807","nonce":4264264866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6805","nonce":4264264866}]},"public_addr":"172.21.15.134:6801/4264264866","cluster_addr":"172.21.15.134:6803/4264264866","heartbeat_back_addr":"172.21.15.134:6807/4264264866","heartbeat_front_addr":"172.21.15.134:6805/4264264866","state":["exists","up"]},{"osd":2,"uuid":"578213fd-715f-4be7-91fa-254f3d0e3022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6801","nonce":2288807707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6803","nonce":2288807707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6807","nonce":2288807707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6805","nonce":2288807707}]},"public_addr":"172.21.15.204:6801/2288807707","cluster_addr":"172.21.15.204:6803/2288807707","heartbeat_back_addr":"172.21.15.204:6807/2288807707","heartbeat_front_addr":"172.21.15.204:6805/2288807707","state":["exists","up"]},{"osd":3,"uuid":"b64aab3f-1e52-4828-8397-74af5e05f2cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6811","nonce":2904204254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6813","nonce":2904204254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6817","nonce":2904204254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6815","nonce":2904204254}]},"public_addr":"172.21.15.114:6811/2904204254","cluster_addr":"172.21.15.114:6813/2904204254","heartbeat_back_addr":"172.21.15.114:6817/2904204254","heartbeat_front_addr":"172.21.15.114:6815/2904204254","state":["exists","up"]},{"osd":4,"uuid":"f38943d6-39c0-42e1-b427-cb6bb90ac38a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6809","nonce":1773945333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6811","nonce":1773945333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6815","nonce":1773945333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6813","nonce":1773945333}]},"public_addr":"172.21.15.134:6809/1773945333","cluster_addr":"172.21.15.134:6811/1773945333","heartbeat_back_addr":"172.21.15.134:6815/1773945333","heartbeat_front_addr":"172.21.15.134:6813/1773945333","state":["exists","up"]},{"osd":5,"uuid":"bb38f361-8e00-4178-9752-5ddbf3af20e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6809","nonce":2371778055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6811","nonce":2371778055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6815","nonce":2371778055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6813","nonce":2371778055}]},"public_addr":"172.21.15.204:6809/2371778055","cluster_addr":"172.21.15.204:6811/2371778055","heartbeat_back_addr":"172.21.15.204:6815/2371778055","heartbeat_front_addr":"172.21.15.204:6813/2371778055","state":["exists","up"]},{"osd":6,"uuid":"93d40a03-e7ff-4a3f-9efa-5c5403c9983d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6819","nonce":1484325044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6821","nonce":1484325044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6825","nonce":1484325044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6823","nonce":1484325044}]},"public_addr":"172.21.15.114:6819/1484325044","cluster_addr":"172.21.15.114:6821/1484325044","heartbeat_back_addr":"172.21.15.114:6825/1484325044","heartbeat_front_addr":"172.21.15.114:6823/1484325044","state":["exists","up"]},{"osd":7,"uuid":"982f629a-1b95-4be1-8e75-5ddce1ad77b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6817","nonce":799246614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6819","nonce":799246614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6823","nonce":799246614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6821","nonce":799246614}]},"public_addr":"172.21.15.134:6817/799246614","cluster_addr":"172.21.15.134:6819/799246614","heartbeat_back_addr":"172.21.15.134:6823/799246614","heartbeat_front_addr":"172.21.15.134:6821/799246614","state":["exists","up"]},{"osd":8,"uuid":"409beb9b-8227-4dad-a4a0-f0e5ce70dc5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6817","nonce":858331805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6819","nonce":858331805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6823","nonce":858331805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6821","nonce":858331805}]},"public_addr":"172.21.15.204:6817/858331805","cluster_addr":"172.21.15.204:6819/858331805","heartbeat_back_addr":"172.21.15.204:6823/858331805","heartbeat_front_addr":"172.21.15.204:6821/858331805","state":["exists","up"]},{"osd":9,"uuid":"a345a15c-b1fc-4877-b445-418eebf0d0ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6827","nonce":2172392854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6829","nonce":2172392854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6833","nonce":2172392854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6831","nonce":2172392854}]},"public_addr":"172.21.15.114:6827/2172392854","cluster_addr":"172.21.15.114:6829/2172392854","heartbeat_back_addr":"172.21.15.114:6833/2172392854","heartbeat_front_addr":"172.21.15.114:6831/2172392854","state":["exists","up"]},{"osd":10,"uuid":"464d995f-d90b-4caa-addb-1f7562bd74b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6825","nonce":3073513468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6827","nonce":3073513468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6831","nonce":3073513468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6829","nonce":3073513468}]},"public_addr":"172.21.15.134:6825/3073513468","cluster_addr":"172.21.15.134:6827/3073513468","heartbeat_back_addr":"172.21.15.134:6831/3073513468","heartbeat_front_addr":"172.21.15.134:6829/3073513468","state":["exists","up"]},{"osd":11,"uuid":"71dd794d-326c-477f-8d02-ee32bff1350d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6825","nonce":3239521765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6827","nonce":3239521765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6831","nonce":3239521765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6829","nonce":3239521765}]},"public_addr":"172.21.15.204:6825/3239521765","cluster_addr":"172.21.15.204:6827/3239521765","heartbeat_back_addr":"172.21.15.204:6831/3239521765","heartbeat_front_addr":"172.21.15.204:6829/3239521765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:14:54.542687+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:15:15.746055+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:15:38.371131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:01.383932+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:24.796237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:48.422214+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:11.284323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:34.570365+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:58.232855+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:18:21.609596+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:18:45.565859+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:19:10.017969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:6801/3396185541":"2024-09-14T18:22:28.057623+0000","172.21.15.114:6800/830738794":"2024-09-14T18:13:03.521116+0000","172.21.15.114:6801/830738794":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/1684954293":"2024-09-14T18:22:28.057623+0000","172.21.15.114:0/3544951322":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/722571999":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/2949422849":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2034742251":"2024-09-14T18:13:03.521116+0000","172.21.15.114:6800/4281866858":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/3407523320":"2024-09-14T18:22:28.057623+0000","172.21.15.114:6800/3396185541":"2024-09-14T18:22:28.057623+0000","172.21.15.114:0/4257828906":"2024-09-14T18:13:25.823027+0000","172.21.15.114:6801/4281866858":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2828364892":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2714091073":"2024-09-14T18:22:28.057623+0000","172.21.15.114:0/842237781":"2024-09-14T18:22:28.057623+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]},{"pool":4,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:34:54.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.098+0000 7f0dd6f25640 1 -- 172.21.15.114:0/2055128411 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0dbc0a1280 msgr2=0x7f0dbc0a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:54.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.098+0000 7f0dd6f25640 1 --2- 172.21.15.114:0/2055128411 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0dbc0a1280 0x7f0dbc0a3740 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f0dc0005fd0 tx=0x7f0dc0009680 comp rx=0 tx=0).stop 2024-09-13T18:34:54.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.098+0000 7f0dd6f25640 1 -- 172.21.15.114:0/2055128411 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0dd0104000 msgr2=0x7f0dd0100e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:54.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.098+0000 7f0dd6f25640 1 --2- 172.21.15.114:0/2055128411 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0dd0104000 0x7f0dd0100e80 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f0db8005ab0 tx=0x7f0db8032c20 comp rx=0 tx=0).stop 2024-09-13T18:34:54.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.098+0000 7f0dd6f25640 1 -- 172.21.15.114:0/2055128411 shutdown_connections 2024-09-13T18:34:54.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.098+0000 7f0dd6f25640 1 --2- 172.21.15.114:0/2055128411 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f0dbc0a1280 0x7f0dbc0a3740 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.098+0000 7f0dd6f25640 1 --2- 172.21.15.114:0/2055128411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dd01013c0 0x7f0dd01a4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.098+0000 7f0dd6f25640 1 --2- 172.21.15.114:0/2055128411 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0dd0104000 0x7f0dd0100e80 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.098+0000 7f0dd6f25640 1 --2- 172.21.15.114:0/2055128411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0dd00feeb0 0x7f0dd0100940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.098+0000 7f0dd6f25640 1 -- 172.21.15.114:0/2055128411 >> 172.21.15.114:0/2055128411 conn(0x7f0dd00fa900 msgr2=0x7f0dd0068950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:54.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.098+0000 7f0dd6f25640 1 -- 172.21.15.114:0/2055128411 shutdown_connections 2024-09-13T18:34:54.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.098+0000 7f0dd6f25640 1 -- 172.21.15.114:0/2055128411 wait complete. 2024-09-13T18:34:54.111 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T18:15:40.129659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T18:20:00.024407+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T18:20:01.085549+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 19, 'snap_epoch': 105, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T18:20:04.334216+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 19, 'snap_epoch': 105, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T18:34:54.112 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T18:34:54.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.212+0000 7f82f5619640 1 -- 172.21.15.114:0/1727629620 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82f00ff810 msgr2=0x7f82f0103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:54.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.212+0000 7f82f5619640 1 --2- 172.21.15.114:0/1727629620 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82f00ff810 0x7f82f0103850 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f82d8009930 tx=0x7f82d802a420 comp rx=0 tx=0).stop 2024-09-13T18:34:54.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.212+0000 7f82f5619640 1 -- 172.21.15.114:0/1727629620 shutdown_connections 2024-09-13T18:34:54.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.212+0000 7f82f5619640 1 --2- 172.21.15.114:0/1727629620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82f0103fb0 0x7f82f010bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.212+0000 7f82f5619640 1 --2- 172.21.15.114:0/1727629620 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82f00ff810 0x7f82f0103850 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.212+0000 7f82f5619640 1 --2- 172.21.15.114:0/1727629620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82f00fee60 0x7f82f00ff240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.212+0000 7f82f5619640 1 -- 172.21.15.114:0/1727629620 >> 172.21.15.114:0/1727629620 conn(0x7f82f0075c60 msgr2=0x7f82f00fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:54.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.212+0000 7f82f5619640 1 -- 172.21.15.114:0/1727629620 shutdown_connections 2024-09-13T18:34:54.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.212+0000 7f82f5619640 1 -- 172.21.15.114:0/1727629620 wait complete. 2024-09-13T18:34:54.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.213+0000 7f82f5619640 1 Processor -- start 2024-09-13T18:34:54.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.213+0000 7f82f5619640 1 -- start start 2024-09-13T18:34:54.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.213+0000 7f82f5619640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82f00fee60 0x7f82f0197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:54.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.213+0000 7f82f5619640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82f0103fb0 0x7f82f0197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:54.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.213+0000 7f82f5619640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82f0197b90 0x7f82f01a2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:54.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.213+0000 7f82f5619640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f01a2830 con 0x7f82f0103fb0 2024-09-13T18:34:54.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.213+0000 7f82f5619640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f01a29a0 con 0x7f82f00fee60 2024-09-13T18:34:54.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.213+0000 7f82f5619640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f01a2b60 con 0x7f82f0197b90 2024-09-13T18:34:54.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.214+0000 7f82e67fc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82f0103fb0 0x7f82f0197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:54.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.214+0000 7f82e67fc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82f0103fb0 0x7f82f0197650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57248/0 (socket says 172.21.15.114:57248) 2024-09-13T18:34:54.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.214+0000 7f82e67fc640 1 -- 172.21.15.114:0/2309207024 learned_addr learned my addr 172.21.15.114:0/2309207024 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:34:54.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.214+0000 7f82eeffd640 1 --2- 172.21.15.114:0/2309207024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82f00fee60 0x7f82f0197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:54.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.214+0000 7f82ef7fe640 1 --2- 172.21.15.114:0/2309207024 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82f0197b90 0x7f82f01a2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:54.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.214+0000 7f82e67fc640 1 -- 172.21.15.114:0/2309207024 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82f0197b90 msgr2=0x7f82f01a2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:54.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.214+0000 7f82e67fc640 1 --2- 172.21.15.114:0/2309207024 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82f0197b90 0x7f82f01a2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.214+0000 7f82e67fc640 1 -- 172.21.15.114:0/2309207024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82f00fee60 msgr2=0x7f82f0197110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:54.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.214+0000 7f82e67fc640 1 --2- 172.21.15.114:0/2309207024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82f00fee60 0x7f82f0197110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.214+0000 7f82e67fc640 1 -- 172.21.15.114:0/2309207024 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d8009560 con 0x7f82f0103fb0 2024-09-13T18:34:54.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.214+0000 7f82e67fc640 1 --2- 172.21.15.114:0/2309207024 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82f0103fb0 0x7f82f0197650 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f82d8005ab0 tx=0x7f82d8032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:54.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.214+0000 7f82ecff9640 1 -- 172.21.15.114:0/2309207024 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d80044d0 con 0x7f82f0103fb0 2024-09-13T18:34:54.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.214+0000 7f82ecff9640 1 -- 172.21.15.114:0/2309207024 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f82d8032e10 con 0x7f82f0103fb0 2024-09-13T18:34:54.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.214+0000 7f82ecff9640 1 -- 172.21.15.114:0/2309207024 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d8003850 con 0x7f82f0103fb0 2024-09-13T18:34:54.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.214+0000 7f82f5619640 1 -- 172.21.15.114:0/2309207024 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82f01a2de0 con 0x7f82f0103fb0 2024-09-13T18:34:54.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.214+0000 7f82f5619640 1 -- 172.21.15.114:0/2309207024 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82f01a31e0 con 0x7f82f0103fb0 2024-09-13T18:34:54.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.215+0000 7f82f5619640 1 -- 172.21.15.114:0/2309207024 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82f0100330 con 0x7f82f0103fb0 2024-09-13T18:34:54.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.216+0000 7f82ecff9640 1 -- 172.21.15.114:0/2309207024 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f82d8004850 con 0x7f82f0103fb0 2024-09-13T18:34:54.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.217+0000 7f82ecff9640 1 --2- 172.21.15.114:0/2309207024 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f82dc0a1200 0x7f82dc0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:54.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.217+0000 7f82eeffd640 1 --2- 172.21.15.114:0/2309207024 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f82dc0a1200 0x7f82dc0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:54.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.217+0000 7f82ecff9640 1 -- 172.21.15.114:0/2309207024 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f82d80edfd0 con 0x7f82f0103fb0 2024-09-13T18:34:54.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.218+0000 7f82eeffd640 1 --2- 172.21.15.114:0/2309207024 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f82dc0a1200 0x7f82dc0a36c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f82e0009bd0 tx=0x7f82e0005f90 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:54.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.220+0000 7f82ecff9640 1 -- 172.21.15.114:0/2309207024 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f82d80b7080 con 0x7f82f0103fb0 2024-09-13T18:34:54.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.383+0000 7f82f5619640 1 -- 172.21.15.114:0/2309207024 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f82f0107260 con 0x7f82f0103fb0 2024-09-13T18:34:54.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.384+0000 7f82ecff9640 1 -- 172.21.15.114:0/2309207024 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) v1 ==== 93+0+10 (secure 0 0 0) 0x7f82d8004e60 con 0x7f82f0103fb0 2024-09-13T18:34:54.386 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 1 2024-09-13T18:34:54.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.386+0000 7f82f5619640 1 -- 172.21.15.114:0/2309207024 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f82dc0a1200 msgr2=0x7f82dc0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:54.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.386+0000 7f82f5619640 1 --2- 172.21.15.114:0/2309207024 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f82dc0a1200 0x7f82dc0a36c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f82e0009bd0 tx=0x7f82e0005f90 comp rx=0 tx=0).stop 2024-09-13T18:34:54.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.386+0000 7f82f5619640 1 -- 172.21.15.114:0/2309207024 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82f0103fb0 msgr2=0x7f82f0197650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:54.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.386+0000 7f82f5619640 1 --2- 172.21.15.114:0/2309207024 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82f0103fb0 0x7f82f0197650 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f82d8005ab0 tx=0x7f82d8032c20 comp rx=0 tx=0).stop 2024-09-13T18:34:54.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.386+0000 7f82f5619640 1 -- 172.21.15.114:0/2309207024 shutdown_connections 2024-09-13T18:34:54.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.386+0000 7f82f5619640 1 --2- 172.21.15.114:0/2309207024 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f82dc0a1200 0x7f82dc0a36c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.386+0000 7f82f5619640 1 --2- 172.21.15.114:0/2309207024 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82f0197b90 0x7f82f01a2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.386+0000 7f82f5619640 1 --2- 172.21.15.114:0/2309207024 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82f0103fb0 0x7f82f0197650 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.386+0000 7f82f5619640 1 --2- 172.21.15.114:0/2309207024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82f00fee60 0x7f82f0197110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.386+0000 7f82f5619640 1 -- 172.21.15.114:0/2309207024 >> 172.21.15.114:0/2309207024 conn(0x7f82f0075c60 msgr2=0x7f82f0102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:54.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.386+0000 7f82f5619640 1 -- 172.21.15.114:0/2309207024 shutdown_connections 2024-09-13T18:34:54.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.386+0000 7f82f5619640 1 -- 172.21.15.114:0/2309207024 wait complete. 2024-09-13T18:34:54.399 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T18:34:54.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.499+0000 7f6e08557640 1 -- 172.21.15.114:0/2654838604 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e00100900 msgr2=0x7f6e00104940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:54.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.499+0000 7f6e08557640 1 --2- 172.21.15.114:0/2654838604 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e00100900 0x7f6e00104940 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f6df0009930 tx=0x7f6df002a420 comp rx=0 tx=0).stop 2024-09-13T18:34:54.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.499+0000 7f6e08557640 1 -- 172.21.15.114:0/2654838604 shutdown_connections 2024-09-13T18:34:54.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.499+0000 7f6e08557640 1 --2- 172.21.15.114:0/2654838604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e001050a0 0x7f6e0010cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.499+0000 7f6e08557640 1 --2- 172.21.15.114:0/2654838604 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e00100900 0x7f6e00104940 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.499+0000 7f6e08557640 1 --2- 172.21.15.114:0/2654838604 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e000fff50 0x7f6e00100330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.499+0000 7f6e08557640 1 -- 172.21.15.114:0/2654838604 >> 172.21.15.114:0/2654838604 conn(0x7f6e000fb9e0 msgr2=0x7f6e000fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:54.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.499+0000 7f6e08557640 1 -- 172.21.15.114:0/2654838604 shutdown_connections 2024-09-13T18:34:54.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.499+0000 7f6e08557640 1 -- 172.21.15.114:0/2654838604 wait complete. 2024-09-13T18:34:54.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.500+0000 7f6e08557640 1 Processor -- start 2024-09-13T18:34:54.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.500+0000 7f6e08557640 1 -- start start 2024-09-13T18:34:54.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.500+0000 7f6e08557640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e000fff50 0x7f6e00196200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:54.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.500+0000 7f6e08557640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e001050a0 0x7f6e00196740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:54.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.500+0000 7f6e08557640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e0019aac0 0x7f6e0019af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:54.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.500+0000 7f6e08557640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0019b4b0 con 0x7f6e0019aac0 2024-09-13T18:34:54.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.500+0000 7f6e08557640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0019b620 con 0x7f6e001050a0 2024-09-13T18:34:54.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.500+0000 7f6e08557640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0019b790 con 0x7f6e000fff50 2024-09-13T18:34:54.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.500+0000 7f6e06af4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e0019aac0 0x7f6e0019af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:54.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.500+0000 7f6e06af4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e0019aac0 0x7f6e0019af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57260/0 (socket says 172.21.15.114:57260) 2024-09-13T18:34:54.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.500+0000 7f6e06af4640 1 -- 172.21.15.114:0/467293090 learned_addr learned my addr 172.21.15.114:0/467293090 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:34:54.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.500+0000 7f6e05af2640 1 --2- 172.21.15.114:0/467293090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e001050a0 0x7f6e00196740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:54.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.500+0000 7f6e06af4640 1 -- 172.21.15.114:0/467293090 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e000fff50 msgr2=0x7f6e00196200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:54.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.500+0000 7f6e06af4640 1 --2- 172.21.15.114:0/467293090 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e000fff50 0x7f6e00196200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.500+0000 7f6e06af4640 1 -- 172.21.15.114:0/467293090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e001050a0 msgr2=0x7f6e00196740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:54.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.500+0000 7f6e06af4640 1 --2- 172.21.15.114:0/467293090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e001050a0 0x7f6e00196740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.500+0000 7f6e06af4640 1 -- 172.21.15.114:0/467293090 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df0009560 con 0x7f6e0019aac0 2024-09-13T18:34:54.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.501+0000 7f6e06af4640 1 --2- 172.21.15.114:0/467293090 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e0019aac0 0x7f6e0019af70 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f6df8009fd0 tx=0x7f6df800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:54.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.501+0000 7f6df77fe640 1 -- 172.21.15.114:0/467293090 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df800bea0 con 0x7f6e0019aac0 2024-09-13T18:34:54.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.501+0000 7f6df77fe640 1 -- 172.21.15.114:0/467293090 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6df8009940 con 0x7f6e0019aac0 2024-09-13T18:34:54.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.501+0000 7f6e08557640 1 -- 172.21.15.114:0/467293090 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e001a2d70 con 0x7f6e0019aac0 2024-09-13T18:34:54.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.501+0000 7f6df77fe640 1 -- 172.21.15.114:0/467293090 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df8010480 con 0x7f6e0019aac0 2024-09-13T18:34:54.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.501+0000 7f6e08557640 1 -- 172.21.15.114:0/467293090 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e001a31a0 con 0x7f6e0019aac0 2024-09-13T18:34:54.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.502+0000 7f6e08557640 1 -- 172.21.15.114:0/467293090 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e00101420 con 0x7f6e0019aac0 2024-09-13T18:34:54.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.503+0000 7f6df77fe640 1 -- 172.21.15.114:0/467293090 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f6df8003970 con 0x7f6e0019aac0 2024-09-13T18:34:54.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.504+0000 7f6df77fe640 1 --2- 172.21.15.114:0/467293090 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f6dec0a1200 0x7f6dec0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:54.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.504+0000 7f6e062f3640 1 --2- 172.21.15.114:0/467293090 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f6dec0a1200 0x7f6dec0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:54.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.504+0000 7f6df77fe640 1 -- 172.21.15.114:0/467293090 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f6df80ccdb0 con 0x7f6e0019aac0 2024-09-13T18:34:54.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.504+0000 7f6e062f3640 1 --2- 172.21.15.114:0/467293090 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f6dec0a1200 0x7f6dec0a36c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc005fd0 tx=0x7f6dfc005850 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:54.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.508+0000 7f6df77fe640 1 -- 172.21.15.114:0/467293090 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f6df8095e60 con 0x7f6e0019aac0 2024-09-13T18:34:54.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.675+0000 7f6e08557640 1 -- 172.21.15.114:0/467293090 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6e001082b0 con 0x7f6e0019aac0 2024-09-13T18:34:54.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.675+0000 7f6df77fe640 1 -- 172.21.15.114:0/467293090 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v106) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6df8017020 con 0x7f6e0019aac0 2024-09-13T18:34:54.677 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2024-09-13T18:34:54.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.677+0000 7f6e08557640 1 -- 172.21.15.114:0/467293090 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f6dec0a1200 msgr2=0x7f6dec0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:54.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.677+0000 7f6e08557640 1 --2- 172.21.15.114:0/467293090 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f6dec0a1200 0x7f6dec0a36c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc005fd0 tx=0x7f6dfc005850 comp rx=0 tx=0).stop 2024-09-13T18:34:54.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.677+0000 7f6e08557640 1 -- 172.21.15.114:0/467293090 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e0019aac0 msgr2=0x7f6e0019af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:54.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.677+0000 7f6e08557640 1 --2- 172.21.15.114:0/467293090 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e0019aac0 0x7f6e0019af70 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f6df8009fd0 tx=0x7f6df800b260 comp rx=0 tx=0).stop 2024-09-13T18:34:54.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.677+0000 7f6e08557640 1 -- 172.21.15.114:0/467293090 shutdown_connections 2024-09-13T18:34:54.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.677+0000 7f6e08557640 1 --2- 172.21.15.114:0/467293090 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f6dec0a1200 0x7f6dec0a36c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.677+0000 7f6e08557640 1 --2- 172.21.15.114:0/467293090 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e0019aac0 0x7f6e0019af70 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.677+0000 7f6e08557640 1 --2- 172.21.15.114:0/467293090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e001050a0 0x7f6e00196740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.677+0000 7f6e08557640 1 --2- 172.21.15.114:0/467293090 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e000fff50 0x7f6e00196200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.677+0000 7f6e08557640 1 -- 172.21.15.114:0/467293090 >> 172.21.15.114:0/467293090 conn(0x7f6e000fb9e0 msgr2=0x7f6e000fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:54.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.677+0000 7f6e08557640 1 -- 172.21.15.114:0/467293090 shutdown_connections 2024-09-13T18:34:54.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.677+0000 7f6e08557640 1 -- 172.21.15.114:0/467293090 wait complete. 2024-09-13T18:34:54.689 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:54 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/4116386735' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-13T18:34:54.689 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:54 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2055128411' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:34:54.689 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:54 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2309207024' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T18:34:54.690 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T18:34:54.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.789+0000 7fe375c96640 1 -- 172.21.15.114:0/3136296759 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe370103840 msgr2=0x7fe37010b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:54.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.789+0000 7fe375c96640 1 --2- 172.21.15.114:0/3136296759 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe370103840 0x7fe37010b3e0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fe368009900 tx=0x7fe36802a410 comp rx=0 tx=0).stop 2024-09-13T18:34:54.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.789+0000 7fe375c96640 1 -- 172.21.15.114:0/3136296759 shutdown_connections 2024-09-13T18:34:54.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.789+0000 7fe375c96640 1 --2- 172.21.15.114:0/3136296759 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe370103840 0x7fe37010b3e0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.789+0000 7fe375c96640 1 --2- 172.21.15.114:0/3136296759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3700ff530 0x7fe370103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.789+0000 7fe375c96640 1 --2- 172.21.15.114:0/3136296759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3700feb80 0x7fe3700fef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.789+0000 7fe375c96640 1 -- 172.21.15.114:0/3136296759 >> 172.21.15.114:0/3136296759 conn(0x7fe370074110 msgr2=0x7fe370074520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:54.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.789+0000 7fe375c96640 1 -- 172.21.15.114:0/3136296759 shutdown_connections 2024-09-13T18:34:54.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.789+0000 7fe375c96640 1 -- 172.21.15.114:0/3136296759 wait complete. 2024-09-13T18:34:54.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.790+0000 7fe375c96640 1 Processor -- start 2024-09-13T18:34:54.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.790+0000 7fe375c96640 1 -- start start 2024-09-13T18:34:54.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.790+0000 7fe375c96640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3700feb80 0x7fe370197140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:54.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.790+0000 7fe375c96640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe370103840 0x7fe370197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:54.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.790+0000 7fe375c96640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe370197c10 0x7fe3701a21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:54.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.790+0000 7fe375c96640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3701a2860 con 0x7fe370197c10 2024-09-13T18:34:54.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.790+0000 7fe375c96640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3701a29d0 con 0x7fe370103840 2024-09-13T18:34:54.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.790+0000 7fe375c96640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3701a2b90 con 0x7fe3700feb80 2024-09-13T18:34:54.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.790+0000 7fe36ffff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe370197c10 0x7fe3701a21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:54.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.790+0000 7fe36ffff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe370197c10 0x7fe3701a21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57286/0 (socket says 172.21.15.114:57286) 2024-09-13T18:34:54.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.790+0000 7fe36ffff640 1 -- 172.21.15.114:0/82873329 learned_addr learned my addr 172.21.15.114:0/82873329 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:34:54.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.791+0000 7fe36f7fe640 1 --2- 172.21.15.114:0/82873329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3700feb80 0x7fe370197140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:54.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.791+0000 7fe36ffff640 1 -- 172.21.15.114:0/82873329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3700feb80 msgr2=0x7fe370197140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:54.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.791+0000 7fe36ffff640 1 --2- 172.21.15.114:0/82873329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3700feb80 0x7fe370197140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.791+0000 7fe36ffff640 1 -- 172.21.15.114:0/82873329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe370103840 msgr2=0x7fe370197680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:54.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.791+0000 7fe36ffff640 1 --2- 172.21.15.114:0/82873329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe370103840 0x7fe370197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.791+0000 7fe36ffff640 1 -- 172.21.15.114:0/82873329 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe368009560 con 0x7fe370197c10 2024-09-13T18:34:54.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.791+0000 7fe36ffff640 1 --2- 172.21.15.114:0/82873329 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe370197c10 0x7fe3701a21c0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fe36802aa00 tx=0x7fe368032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:54.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.791+0000 7fe36cff9640 1 -- 172.21.15.114:0/82873329 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe368005d70 con 0x7fe370197c10 2024-09-13T18:34:54.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.791+0000 7fe36cff9640 1 -- 172.21.15.114:0/82873329 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe368032e10 con 0x7fe370197c10 2024-09-13T18:34:54.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.791+0000 7fe375c96640 1 -- 172.21.15.114:0/82873329 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3701a2e10 con 0x7fe370197c10 2024-09-13T18:34:54.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.791+0000 7fe36cff9640 1 -- 172.21.15.114:0/82873329 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe368003850 con 0x7fe370197c10 2024-09-13T18:34:54.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.791+0000 7fe375c96640 1 -- 172.21.15.114:0/82873329 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3701a32c0 con 0x7fe370197c10 2024-09-13T18:34:54.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.792+0000 7fe375c96640 1 -- 172.21.15.114:0/82873329 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3700ffbf0 con 0x7fe370197c10 2024-09-13T18:34:54.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.794+0000 7fe36cff9640 1 -- 172.21.15.114:0/82873329 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fe368005210 con 0x7fe370197c10 2024-09-13T18:34:54.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.794+0000 7fe36cff9640 1 --2- 172.21.15.114:0/82873329 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe3580a1200 0x7fe3580a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:54.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.795+0000 7fe36cff9640 1 -- 172.21.15.114:0/82873329 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fe3680ee070 con 0x7fe370197c10 2024-09-13T18:34:54.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.795+0000 7fe36f7fe640 1 --2- 172.21.15.114:0/82873329 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe3580a1200 0x7fe3580a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:54.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.795+0000 7fe36f7fe640 1 --2- 172.21.15.114:0/82873329 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe3580a1200 0x7fe3580a36c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe360005c40 tx=0x7fe36000a590 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:54.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.797+0000 7fe36cff9640 1 -- 172.21.15.114:0/82873329 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fe3680b7120 con 0x7fe370197c10 2024-09-13T18:34:54.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:54 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/4116386735' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-13T18:34:54.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:54 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/2055128411' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:34:54.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:54 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/2309207024' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T18:34:54.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:54 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/4116386735' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-13T18:34:54.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:54 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/2055128411' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:34:54.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:54 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/2309207024' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-13T18:34:54.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.961+0000 7fe375c96640 1 -- 172.21.15.114:0/82873329 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe370106a30 con 0x7fe370197c10 2024-09-13T18:34:54.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.962+0000 7fe36cff9640 1 -- 172.21.15.114:0/82873329 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v106) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe36804a0a0 con 0x7fe370197c10 2024-09-13T18:34:54.964 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 64 2024-09-13T18:34:54.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.964+0000 7fe375c96640 1 -- 172.21.15.114:0/82873329 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe3580a1200 msgr2=0x7fe3580a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:54.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.964+0000 7fe375c96640 1 --2- 172.21.15.114:0/82873329 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe3580a1200 0x7fe3580a36c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe360005c40 tx=0x7fe36000a590 comp rx=0 tx=0).stop 2024-09-13T18:34:54.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.964+0000 7fe375c96640 1 -- 172.21.15.114:0/82873329 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe370197c10 msgr2=0x7fe3701a21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:54.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.964+0000 7fe375c96640 1 --2- 172.21.15.114:0/82873329 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe370197c10 0x7fe3701a21c0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fe36802aa00 tx=0x7fe368032c20 comp rx=0 tx=0).stop 2024-09-13T18:34:54.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.964+0000 7fe375c96640 1 -- 172.21.15.114:0/82873329 shutdown_connections 2024-09-13T18:34:54.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.964+0000 7fe375c96640 1 --2- 172.21.15.114:0/82873329 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fe3580a1200 0x7fe3580a36c0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.964+0000 7fe375c96640 1 --2- 172.21.15.114:0/82873329 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe370197c10 0x7fe3701a21c0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.964+0000 7fe375c96640 1 --2- 172.21.15.114:0/82873329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe370103840 0x7fe370197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.964+0000 7fe375c96640 1 --2- 172.21.15.114:0/82873329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3700feb80 0x7fe370197140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:54.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.964+0000 7fe375c96640 1 -- 172.21.15.114:0/82873329 >> 172.21.15.114:0/82873329 conn(0x7fe370074110 msgr2=0x7fe3700fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:54.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.964+0000 7fe375c96640 1 -- 172.21.15.114:0/82873329 shutdown_connections 2024-09-13T18:34:54.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:54.964+0000 7fe375c96640 1 -- 172.21.15.114:0/82873329 wait complete. 2024-09-13T18:34:54.977 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T18:34:55.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.077+0000 7f39eace9640 1 -- 172.21.15.114:0/2893464930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39e4103fb0 msgr2=0x7f39e410bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:55.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.077+0000 7f39eace9640 1 --2- 172.21.15.114:0/2893464930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39e4103fb0 0x7f39e410bb00 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f39dc009930 tx=0x7f39dc02a420 comp rx=0 tx=0).stop 2024-09-13T18:34:55.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.077+0000 7f39eace9640 1 -- 172.21.15.114:0/2893464930 shutdown_connections 2024-09-13T18:34:55.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.077+0000 7f39eace9640 1 --2- 172.21.15.114:0/2893464930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39e4103fb0 0x7f39e410bb00 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:55.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.077+0000 7f39eace9640 1 --2- 172.21.15.114:0/2893464930 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39e40ff810 0x7f39e4103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:55.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.077+0000 7f39eace9640 1 --2- 172.21.15.114:0/2893464930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39e40fee60 0x7f39e40ff240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:55.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.077+0000 7f39eace9640 1 -- 172.21.15.114:0/2893464930 >> 172.21.15.114:0/2893464930 conn(0x7f39e4075c60 msgr2=0x7f39e40fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:55.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.077+0000 7f39eace9640 1 -- 172.21.15.114:0/2893464930 shutdown_connections 2024-09-13T18:34:55.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.077+0000 7f39eace9640 1 -- 172.21.15.114:0/2893464930 wait complete. 2024-09-13T18:34:55.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.077+0000 7f39eace9640 1 Processor -- start 2024-09-13T18:34:55.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.077+0000 7f39eace9640 1 -- start start 2024-09-13T18:34:55.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.077+0000 7f39eace9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39e40fee60 0x7f39e4197080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:55.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.077+0000 7f39eace9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39e4103fb0 0x7f39e41975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:55.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.077+0000 7f39eace9640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39e4197b00 0x7f39e41a2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:55.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.077+0000 7f39eace9640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e41a27a0 con 0x7f39e4197b00 2024-09-13T18:34:55.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.077+0000 7f39eace9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e41a2910 con 0x7f39e40fee60 2024-09-13T18:34:55.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.077+0000 7f39eace9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e41a2ad0 con 0x7f39e4103fb0 2024-09-13T18:34:55.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.078+0000 7f39e9286640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39e4197b00 0x7f39e41a2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:55.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.078+0000 7f39e9286640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39e4197b00 0x7f39e41a2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57296/0 (socket says 172.21.15.114:57296) 2024-09-13T18:34:55.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.078+0000 7f39e9286640 1 -- 172.21.15.114:0/3013868771 learned_addr learned my addr 172.21.15.114:0/3013868771 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:34:55.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.078+0000 7f39e8a85640 1 --2- 172.21.15.114:0/3013868771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39e40fee60 0x7f39e4197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:55.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.078+0000 7f39e3fff640 1 --2- 172.21.15.114:0/3013868771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39e4103fb0 0x7f39e41975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:55.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.078+0000 7f39e9286640 1 -- 172.21.15.114:0/3013868771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39e4103fb0 msgr2=0x7f39e41975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:55.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.078+0000 7f39e9286640 1 --2- 172.21.15.114:0/3013868771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39e4103fb0 0x7f39e41975c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:55.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.078+0000 7f39e9286640 1 -- 172.21.15.114:0/3013868771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39e40fee60 msgr2=0x7f39e4197080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:55.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.078+0000 7f39e9286640 1 --2- 172.21.15.114:0/3013868771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39e40fee60 0x7f39e4197080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:55.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.078+0000 7f39e9286640 1 -- 172.21.15.114:0/3013868771 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39dc009560 con 0x7f39e4197b00 2024-09-13T18:34:55.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.078+0000 7f39e3fff640 1 --2- 172.21.15.114:0/3013868771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39e4103fb0 0x7f39e41975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:34:55.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.078+0000 7f39e9286640 1 --2- 172.21.15.114:0/3013868771 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39e4197b00 0x7f39e41a2100 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f39dc02aa10 tx=0x7f39dc0055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:55.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.078+0000 7f39e1ffb640 1 -- 172.21.15.114:0/3013868771 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39dc004bd0 con 0x7f39e4197b00 2024-09-13T18:34:55.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.078+0000 7f39e1ffb640 1 -- 172.21.15.114:0/3013868771 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f39dc004d30 con 0x7f39e4197b00 2024-09-13T18:34:55.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.078+0000 7f39eace9640 1 -- 172.21.15.114:0/3013868771 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39e41a2d50 con 0x7f39e4197b00 2024-09-13T18:34:55.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.078+0000 7f39e1ffb640 1 -- 172.21.15.114:0/3013868771 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39dc03c8d0 con 0x7f39e4197b00 2024-09-13T18:34:55.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.078+0000 7f39eace9640 1 -- 172.21.15.114:0/3013868771 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39e41a3250 con 0x7f39e4197b00 2024-09-13T18:34:55.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.079+0000 7f39eace9640 1 -- 172.21.15.114:0/3013868771 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39e4100330 con 0x7f39e4197b00 2024-09-13T18:34:55.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.080+0000 7f39e1ffb640 1 -- 172.21.15.114:0/3013868771 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f39dc03ca30 con 0x7f39e4197b00 2024-09-13T18:34:55.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.081+0000 7f39e1ffb640 1 --2- 172.21.15.114:0/3013868771 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f39d00a1280 0x7f39d00a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:55.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.081+0000 7f39e8a85640 1 --2- 172.21.15.114:0/3013868771 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f39d00a1280 0x7f39d00a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:55.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.081+0000 7f39e1ffb640 1 -- 172.21.15.114:0/3013868771 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f39dc0ee200 con 0x7f39e4197b00 2024-09-13T18:34:55.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.081+0000 7f39e8a85640 1 --2- 172.21.15.114:0/3013868771 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f39d00a1280 0x7f39d00a3740 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f39cc005fd0 tx=0x7f39cc009680 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:55.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.084+0000 7f39e1ffb640 1 -- 172.21.15.114:0/3013868771 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f39dc0b72b0 con 0x7f39e4197b00 2024-09-13T18:34:55.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.247+0000 7f39eace9640 1 -- 172.21.15.114:0/3013868771 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f39e4107160 con 0x7f39e4197b00 2024-09-13T18:34:55.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.247+0000 7f39e1ffb640 1 -- 172.21.15.114:0/3013868771 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v106) v1 ==== 103+0+11 (secure 0 0 0) 0x7f39dc0400a0 con 0x7f39e4197b00 2024-09-13T18:34:55.249 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2024-09-13T18:34:55.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.249+0000 7f39eace9640 1 -- 172.21.15.114:0/3013868771 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f39d00a1280 msgr2=0x7f39d00a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:55.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.249+0000 7f39eace9640 1 --2- 172.21.15.114:0/3013868771 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f39d00a1280 0x7f39d00a3740 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f39cc005fd0 tx=0x7f39cc009680 comp rx=0 tx=0).stop 2024-09-13T18:34:55.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.249+0000 7f39eace9640 1 -- 172.21.15.114:0/3013868771 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39e4197b00 msgr2=0x7f39e41a2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:55.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.249+0000 7f39eace9640 1 --2- 172.21.15.114:0/3013868771 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39e4197b00 0x7f39e41a2100 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f39dc02aa10 tx=0x7f39dc0055a0 comp rx=0 tx=0).stop 2024-09-13T18:34:55.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.249+0000 7f39eace9640 1 -- 172.21.15.114:0/3013868771 shutdown_connections 2024-09-13T18:34:55.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.249+0000 7f39eace9640 1 --2- 172.21.15.114:0/3013868771 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f39d00a1280 0x7f39d00a3740 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:55.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.249+0000 7f39eace9640 1 --2- 172.21.15.114:0/3013868771 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39e4197b00 0x7f39e41a2100 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:55.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.249+0000 7f39eace9640 1 --2- 172.21.15.114:0/3013868771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39e4103fb0 0x7f39e41975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:55.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.249+0000 7f39eace9640 1 --2- 172.21.15.114:0/3013868771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39e40fee60 0x7f39e4197080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:55.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.249+0000 7f39eace9640 1 -- 172.21.15.114:0/3013868771 >> 172.21.15.114:0/3013868771 conn(0x7f39e4075c60 msgr2=0x7f39e4102300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:55.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.249+0000 7f39eace9640 1 -- 172.21.15.114:0/3013868771 shutdown_connections 2024-09-13T18:34:55.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.249+0000 7f39eace9640 1 -- 172.21.15.114:0/3013868771 wait complete. 2024-09-13T18:34:55.263 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T18:34:55.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.360+0000 7f423aeaa640 1 -- 172.21.15.114:0/1258669831 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42340feef0 msgr2=0x7f42340ff2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:55.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.360+0000 7f423aeaa640 1 --2- 172.21.15.114:0/1258669831 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42340feef0 0x7f42340ff2d0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f422c009900 tx=0x7f422c02a3f0 comp rx=0 tx=0).stop 2024-09-13T18:34:55.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.360+0000 7f423aeaa640 1 -- 172.21.15.114:0/1258669831 shutdown_connections 2024-09-13T18:34:55.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.360+0000 7f423aeaa640 1 --2- 172.21.15.114:0/1258669831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4234104040 0x7f423410bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:55.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.360+0000 7f423aeaa640 1 --2- 172.21.15.114:0/1258669831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42340ff8a0 0x7f42341038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:55.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.360+0000 7f423aeaa640 1 --2- 172.21.15.114:0/1258669831 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42340feef0 0x7f42340ff2d0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:55.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.360+0000 7f423aeaa640 1 -- 172.21.15.114:0/1258669831 >> 172.21.15.114:0/1258669831 conn(0x7f42340fa940 msgr2=0x7f42340fcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:55.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.360+0000 7f423aeaa640 1 -- 172.21.15.114:0/1258669831 shutdown_connections 2024-09-13T18:34:55.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.360+0000 7f423aeaa640 1 -- 172.21.15.114:0/1258669831 wait complete. 2024-09-13T18:34:55.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.361+0000 7f423aeaa640 1 Processor -- start 2024-09-13T18:34:55.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.361+0000 7f423aeaa640 1 -- start start 2024-09-13T18:34:55.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.361+0000 7f423aeaa640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42340feef0 0x7f42341961f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:55.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.361+0000 7f423aeaa640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4234104040 0x7f4234196730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:55.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.361+0000 7f423aeaa640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f423419ab20 0x7f423419afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:55.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.361+0000 7f423aeaa640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423419b510 con 0x7f42340feef0 2024-09-13T18:34:55.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.361+0000 7f423aeaa640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423419b680 con 0x7f4234104040 2024-09-13T18:34:55.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.361+0000 7f423aeaa640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423419b7f0 con 0x7f423419ab20 2024-09-13T18:34:55.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.361+0000 7f4238c46640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42340feef0 0x7f42341961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:55.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.361+0000 7f4238c46640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42340feef0 0x7f42341961f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57308/0 (socket says 172.21.15.114:57308) 2024-09-13T18:34:55.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.361+0000 7f4238c46640 1 -- 172.21.15.114:0/1310775571 learned_addr learned my addr 172.21.15.114:0/1310775571 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:34:55.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.361+0000 7f4233fff640 1 --2- 172.21.15.114:0/1310775571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4234104040 0x7f4234196730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:55.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.361+0000 7f4239447640 1 --2- 172.21.15.114:0/1310775571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f423419ab20 0x7f423419afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:55.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.361+0000 7f4238c46640 1 -- 172.21.15.114:0/1310775571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f423419ab20 msgr2=0x7f423419afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:55.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.361+0000 7f4238c46640 1 --2- 172.21.15.114:0/1310775571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f423419ab20 0x7f423419afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:55.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.361+0000 7f4238c46640 1 -- 172.21.15.114:0/1310775571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4234104040 msgr2=0x7f4234196730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:55.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.361+0000 7f4238c46640 1 --2- 172.21.15.114:0/1310775571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4234104040 0x7f4234196730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:55.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.361+0000 7f4238c46640 1 -- 172.21.15.114:0/1310775571 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f422c009560 con 0x7f42340feef0 2024-09-13T18:34:55.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.361+0000 7f4238c46640 1 --2- 172.21.15.114:0/1310775571 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42340feef0 0x7f42341961f0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f422c006010 tx=0x7f422c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:55.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.361+0000 7f4231ffb640 1 -- 172.21.15.114:0/1310775571 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422c004cc0 con 0x7f42340feef0 2024-09-13T18:34:55.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.362+0000 7f423aeaa640 1 -- 172.21.15.114:0/1310775571 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42341a2dd0 con 0x7f42340feef0 2024-09-13T18:34:55.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.362+0000 7f4231ffb640 1 -- 172.21.15.114:0/1310775571 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f422c003a80 con 0x7f42340feef0 2024-09-13T18:34:55.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.362+0000 7f4231ffb640 1 -- 172.21.15.114:0/1310775571 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422c03cad0 con 0x7f42340feef0 2024-09-13T18:34:55.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.362+0000 7f423aeaa640 1 -- 172.21.15.114:0/1310775571 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42341a31f0 con 0x7f42340feef0 2024-09-13T18:34:55.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.363+0000 7f423aeaa640 1 -- 172.21.15.114:0/1310775571 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42341003c0 con 0x7f42340feef0 2024-09-13T18:34:55.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.364+0000 7f4231ffb640 1 -- 172.21.15.114:0/1310775571 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f422c003bf0 con 0x7f42340feef0 2024-09-13T18:34:55.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.364+0000 7f4231ffb640 1 --2- 172.21.15.114:0/1310775571 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f42200a1200 0x7f42200a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:34:55.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.365+0000 7f4231ffb640 1 -- 172.21.15.114:0/1310775571 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f422c0edfa0 con 0x7f42340feef0 2024-09-13T18:34:55.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.365+0000 7f4233fff640 1 --2- 172.21.15.114:0/1310775571 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f42200a1200 0x7f42200a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:34:55.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.365+0000 7f4233fff640 1 --2- 172.21.15.114:0/1310775571 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f42200a1200 0x7f42200a36c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4234197790 tx=0x7f42240075b0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:34:55.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.369+0000 7f4231ffb640 1 -- 172.21.15.114:0/1310775571 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f422c0b7100 con 0x7f42340feef0 2024-09-13T18:34:55.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.545+0000 7f423aeaa640 1 -- 172.21.15.114:0/1310775571 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4234197540 con 0x7f42340feef0 2024-09-13T18:34:55.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.546+0000 7f4231ffb640 1 -- 172.21.15.114:0/1310775571 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v106) v1 ==== 81+0+958 (secure 0 0 0) 0x7f422c0403d0 con 0x7f42340feef0 2024-09-13T18:34:55.548 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6801/3396185541 2024-09-14T18:22:28.057623+0000 2024-09-13T18:34:55.549 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6800/830738794 2024-09-14T18:13:03.521116+0000 2024-09-13T18:34:55.549 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6801/830738794 2024-09-14T18:13:03.521116+0000 2024-09-13T18:34:55.549 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/1684954293 2024-09-14T18:22:28.057623+0000 2024-09-13T18:34:55.549 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/3544951322 2024-09-14T18:13:03.521116+0000 2024-09-13T18:34:55.549 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/722571999 2024-09-14T18:13:03.521116+0000 2024-09-13T18:34:55.549 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/2949422849 2024-09-14T18:13:25.823027+0000 2024-09-13T18:34:55.549 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/2034742251 2024-09-14T18:13:03.521116+0000 2024-09-13T18:34:55.549 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6800/4281866858 2024-09-14T18:13:25.823027+0000 2024-09-13T18:34:55.549 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/3407523320 2024-09-14T18:22:28.057623+0000 2024-09-13T18:34:55.549 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6800/3396185541 2024-09-14T18:22:28.057623+0000 2024-09-13T18:34:55.549 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/4257828906 2024-09-14T18:13:25.823027+0000 2024-09-13T18:34:55.549 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6801/4281866858 2024-09-14T18:13:25.823027+0000 2024-09-13T18:34:55.549 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/2828364892 2024-09-14T18:13:25.823027+0000 2024-09-13T18:34:55.550 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/2714091073 2024-09-14T18:22:28.057623+0000 2024-09-13T18:34:55.550 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/842237781 2024-09-14T18:22:28.057623+0000 2024-09-13T18:34:55.550 INFO:teuthology.orchestra.run.smithi114.stderr:listed 16 entries 2024-09-13T18:34:55.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.548+0000 7f423aeaa640 1 -- 172.21.15.114:0/1310775571 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f42200a1200 msgr2=0x7f42200a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:55.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.548+0000 7f423aeaa640 1 --2- 172.21.15.114:0/1310775571 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f42200a1200 0x7f42200a36c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4234197790 tx=0x7f42240075b0 comp rx=0 tx=0).stop 2024-09-13T18:34:55.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.548+0000 7f423aeaa640 1 -- 172.21.15.114:0/1310775571 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42340feef0 msgr2=0x7f42341961f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:34:55.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.548+0000 7f423aeaa640 1 --2- 172.21.15.114:0/1310775571 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42340feef0 0x7f42341961f0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f422c006010 tx=0x7f422c0046c0 comp rx=0 tx=0).stop 2024-09-13T18:34:55.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.548+0000 7f423aeaa640 1 -- 172.21.15.114:0/1310775571 shutdown_connections 2024-09-13T18:34:55.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.548+0000 7f423aeaa640 1 --2- 172.21.15.114:0/1310775571 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f42200a1200 0x7f42200a36c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:55.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.548+0000 7f423aeaa640 1 --2- 172.21.15.114:0/1310775571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f423419ab20 0x7f423419afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:55.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.548+0000 7f423aeaa640 1 --2- 172.21.15.114:0/1310775571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4234104040 0x7f4234196730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:55.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.548+0000 7f423aeaa640 1 --2- 172.21.15.114:0/1310775571 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42340feef0 0x7f42341961f0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:34:55.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.548+0000 7f423aeaa640 1 -- 172.21.15.114:0/1310775571 >> 172.21.15.114:0/1310775571 conn(0x7f42340fa940 msgr2=0x7f42340fbd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:34:55.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.548+0000 7f423aeaa640 1 -- 172.21.15.114:0/1310775571 shutdown_connections 2024-09-13T18:34:55.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:34:55.548+0000 7f423aeaa640 1 -- 172.21.15.114:0/1310775571 wait complete. 2024-09-13T18:34:55.561 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi114.front.sepia.ceph.com... 2024-09-13T18:34:55.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:34:55.561 DEBUG:teuthology.orchestra.run.smithi114:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-13T18:34:55.635 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi114.stderr:ceph-fuse[83676]: fuse finished with error 0 and tester_r 0 2024-09-13T18:34:55.636 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:34:55.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:55 smithi134 ceph-mon[34283]: pgmap v388: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-13T18:34:55.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:55 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/467293090' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T18:34:55.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:55 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/82873329' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T18:34:55.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:55 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/3013868771' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T18:34:55.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:55 smithi204 ceph-mon[34777]: pgmap v388: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-13T18:34:55.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:55 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/467293090' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T18:34:55.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:55 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/82873329' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T18:34:55.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:55 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/3013868771' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T18:34:55.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:55 smithi114 ceph-mon[30897]: pgmap v388: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-13T18:34:55.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:55 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/467293090' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-13T18:34:55.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:55 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/82873329' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-13T18:34:55.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:55 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/3013868771' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-13T18:34:56.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:56 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/1310775571' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-13T18:34:56.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:56 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/1310775571' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-13T18:34:56.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:56 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/1310775571' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-13T18:34:57.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:57 smithi134 ceph-mon[34283]: pgmap v389: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-13T18:34:57.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:57 smithi204 ceph-mon[34777]: pgmap v389: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-13T18:34:57.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:57 smithi114 ceph-mon[30897]: pgmap v389: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-13T18:34:59.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:34:59 smithi134 ceph-mon[34283]: pgmap v390: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-13T18:34:59.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:34:59 smithi204 ceph-mon[34777]: pgmap v390: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-13T18:34:59.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:34:59 smithi114 ceph-mon[30897]: pgmap v390: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-13T18:35:01.738 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi114.front.sepia.ceph.com 2024-09-13T18:35:01.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:35:01.738 DEBUG:teuthology.orchestra.run.smithi114:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-13T18:35:01.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:35:01.781 DEBUG:teuthology.orchestra.run.smithi114:> ip netns list 2024-09-13T18:35:01.865 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-13T18:35:01.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:35:01.865 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T18:35:01.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:01 smithi134 ceph-mon[34283]: pgmap v391: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-13T18:35:01.931 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:01 smithi114 ceph-mon[30897]: pgmap v391: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-13T18:35:01.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:35:01.936 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link delete ceph-brx 2024-09-13T18:35:01.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:01 smithi204 ceph-mon[34777]: pgmap v391: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-13T18:35:02.029 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-13T18:35:02.040 INFO:tasks.cephadm:Teardown begin 2024-09-13T18:35:02.040 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-13T18:35:02.044 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:35:02.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.148+0000 7f99d8fb5640 1 -- 172.21.15.114:0/1516644629 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99d40fe880 msgr2=0x7f99d40fece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:02.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.148+0000 7f99d8fb5640 1 --2- 172.21.15.114:0/1516644629 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99d40fe880 0x7f99d40fece0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f99c800aa40 tx=0x7f99c802b750 comp rx=0 tx=0).stop 2024-09-13T18:35:02.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.148+0000 7f99d8fb5640 1 -- 172.21.15.114:0/1516644629 shutdown_connections 2024-09-13T18:35:02.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.148+0000 7f99d8fb5640 1 --2- 172.21.15.114:0/1516644629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99d40ff220 0x7f99d410b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:02.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.148+0000 7f99d8fb5640 1 --2- 172.21.15.114:0/1516644629 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99d40fe880 0x7f99d40fece0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:02.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.148+0000 7f99d8fb5640 1 --2- 172.21.15.114:0/1516644629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99d4104450 0x7f99d4104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:02.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.148+0000 7f99d8fb5640 1 -- 172.21.15.114:0/1516644629 >> 172.21.15.114:0/1516644629 conn(0x7f99d40741b0 msgr2=0x7f99d40745c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:02.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.148+0000 7f99d8fb5640 1 -- 172.21.15.114:0/1516644629 shutdown_connections 2024-09-13T18:35:02.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.148+0000 7f99d8fb5640 1 -- 172.21.15.114:0/1516644629 wait complete. 2024-09-13T18:35:02.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.149+0000 7f99d8fb5640 1 Processor -- start 2024-09-13T18:35:02.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.149+0000 7f99d8fb5640 1 -- start start 2024-09-13T18:35:02.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.149+0000 7f99d8fb5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99d40fe880 0x7f99d4196220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:02.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.149+0000 7f99d8fb5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99d40ff220 0x7f99d4196760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:02.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.149+0000 7f99d8fb5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99d4104450 0x7f99d419ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:02.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.149+0000 7f99d8fb5640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d419b150 con 0x7f99d40fe880 2024-09-13T18:35:02.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.149+0000 7f99d8fb5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d419b2c0 con 0x7f99d4104450 2024-09-13T18:35:02.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.149+0000 7f99d8fb5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d419b480 con 0x7f99d40ff220 2024-09-13T18:35:02.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.149+0000 7f99d259c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99d40fe880 0x7f99d4196220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:02.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.149+0000 7f99d259c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99d40fe880 0x7f99d4196220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57320/0 (socket says 172.21.15.114:57320) 2024-09-13T18:35:02.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.149+0000 7f99d259c640 1 -- 172.21.15.114:0/2722783128 learned_addr learned my addr 172.21.15.114:0/2722783128 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:35:02.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.149+0000 7f99d1d9b640 1 --2- 172.21.15.114:0/2722783128 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99d40ff220 0x7f99d4196760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:02.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.149+0000 7f99d2d9d640 1 --2- 172.21.15.114:0/2722783128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99d4104450 0x7f99d419ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:02.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.149+0000 7f99d259c640 1 -- 172.21.15.114:0/2722783128 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99d40ff220 msgr2=0x7f99d4196760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:02.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.149+0000 7f99d259c640 1 --2- 172.21.15.114:0/2722783128 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99d40ff220 0x7f99d4196760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:02.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.149+0000 7f99d259c640 1 -- 172.21.15.114:0/2722783128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99d4104450 msgr2=0x7f99d419ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:02.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.149+0000 7f99d259c640 1 --2- 172.21.15.114:0/2722783128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99d4104450 0x7f99d419ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:02.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.149+0000 7f99d259c640 1 -- 172.21.15.114:0/2722783128 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c800a670 con 0x7f99d40fe880 2024-09-13T18:35:02.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.149+0000 7f99d2d9d640 1 --2- 172.21.15.114:0/2722783128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99d4104450 0x7f99d419ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:35:02.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.149+0000 7f99d259c640 1 --2- 172.21.15.114:0/2722783128 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99d40fe880 0x7f99d4196220 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f99c400e9f0 tx=0x7f99c400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:02.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.150+0000 7f99c37fe640 1 -- 172.21.15.114:0/2722783128 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c4009720 con 0x7f99d40fe880 2024-09-13T18:35:02.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.150+0000 7f99c37fe640 1 -- 172.21.15.114:0/2722783128 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f99c400cd80 con 0x7f99d40fe880 2024-09-13T18:35:02.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.150+0000 7f99d8fb5640 1 -- 172.21.15.114:0/2722783128 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99d419b760 con 0x7f99d40fe880 2024-09-13T18:35:02.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.150+0000 7f99c37fe640 1 -- 172.21.15.114:0/2722783128 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c4010430 con 0x7f99d40fe880 2024-09-13T18:35:02.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.150+0000 7f99d8fb5640 1 -- 172.21.15.114:0/2722783128 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99d41a2fe0 con 0x7f99d40fe880 2024-09-13T18:35:02.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.151+0000 7f99d8fb5640 1 -- 172.21.15.114:0/2722783128 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99d40ffb10 con 0x7f99d40fe880 2024-09-13T18:35:02.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.152+0000 7f99c37fe640 1 -- 172.21.15.114:0/2722783128 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f99c4004fb0 con 0x7f99d40fe880 2024-09-13T18:35:02.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.153+0000 7f99c37fe640 1 --2- 172.21.15.114:0/2722783128 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f99bc0a1200 0x7f99bc0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:02.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.153+0000 7f99d1d9b640 1 --2- 172.21.15.114:0/2722783128 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f99bc0a1200 0x7f99bc0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:02.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.153+0000 7f99c37fe640 1 -- 172.21.15.114:0/2722783128 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f99c4014070 con 0x7f99d40fe880 2024-09-13T18:35:02.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.153+0000 7f99d1d9b640 1 --2- 172.21.15.114:0/2722783128 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f99bc0a1200 0x7f99bc0a36c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f99c80057c0 tx=0x7f99c8005a20 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:02.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.157+0000 7f99c37fe640 1 -- 172.21.15.114:0/2722783128 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f99c40964f0 con 0x7f99d40fe880 2024-09-13T18:35:02.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.359+0000 7f99d8fb5640 1 -- 172.21.15.114:0/2722783128 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f99d4197570 con 0x7f99d40fe880 2024-09-13T18:35:02.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.360+0000 7f99c37fe640 1 -- 172.21.15.114:0/2722783128 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+10945 (secure 0 0 0) 0x7f99c4099b10 con 0x7f99d40fe880 2024-09-13T18:35:02.363 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:35:02.363 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4162817497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4162817497},{"type":"v1","addr":"172.21.15.114:6835","nonce":4162817497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/2435428478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":2435428478},{"type":"v1","addr":"172.21.15.114:6837","nonce":2435428478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T18:20:01.895117+0000","modified":"2024-09-13T18:33:48.420534+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24482,"mds_2":24475,"mds_3":24458,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/2046516451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":2046516451},{"type":"v1","addr":"172.21.15.114:6839","nonce":2046516451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":10,"addr":"172.21.15.114:6841/1956923446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":1956923446},{"type":"v1","addr":"172.21.15.114:6841","nonce":1956923446}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24391":{"gid":24391,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/507301496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":507301496},{"type":"v1","addr":"172.21.15.204:6833","nonce":507301496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4028657219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4028657219},{"type":"v1","addr":"172.21.15.134:6833","nonce":4028657219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/57029119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":57029119},{"type":"v1","addr":"172.21.15.204:6835","nonce":57029119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3068880939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3068880939},{"type":"v1","addr":"172.21.15.204:6837","nonce":3068880939}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"e","rank":3,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.134:6835/1150003275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1150003275},{"type":"v1","addr":"172.21.15.134:6835","nonce":1150003275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/344883912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":344883912},{"type":"v1","addr":"172.21.15.204:6839","nonce":344883912}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6837/891692210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":891692210},{"type":"v1","addr":"172.21.15.134:6837","nonce":891692210}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6839/3466309724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3466309724},{"type":"v1","addr":"172.21.15.134:6839","nonce":3466309724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:35:02.363 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 39 2024-09-13T18:35:02.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.362+0000 7f99d8fb5640 1 -- 172.21.15.114:0/2722783128 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f99bc0a1200 msgr2=0x7f99bc0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:02.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.362+0000 7f99d8fb5640 1 --2- 172.21.15.114:0/2722783128 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f99bc0a1200 0x7f99bc0a36c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f99c80057c0 tx=0x7f99c8005a20 comp rx=0 tx=0).stop 2024-09-13T18:35:02.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.362+0000 7f99d8fb5640 1 -- 172.21.15.114:0/2722783128 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99d40fe880 msgr2=0x7f99d4196220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:02.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.362+0000 7f99d8fb5640 1 --2- 172.21.15.114:0/2722783128 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99d40fe880 0x7f99d4196220 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f99c400e9f0 tx=0x7f99c400edf0 comp rx=0 tx=0).stop 2024-09-13T18:35:02.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.362+0000 7f99d8fb5640 1 -- 172.21.15.114:0/2722783128 shutdown_connections 2024-09-13T18:35:02.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.362+0000 7f99d8fb5640 1 --2- 172.21.15.114:0/2722783128 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f99bc0a1200 0x7f99bc0a36c0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:02.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.362+0000 7f99d8fb5640 1 --2- 172.21.15.114:0/2722783128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99d4104450 0x7f99d419ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:02.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.362+0000 7f99d8fb5640 1 --2- 172.21.15.114:0/2722783128 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99d40ff220 0x7f99d4196760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:02.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.362+0000 7f99d8fb5640 1 --2- 172.21.15.114:0/2722783128 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99d40fe880 0x7f99d4196220 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:02.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.362+0000 7f99d8fb5640 1 -- 172.21.15.114:0/2722783128 >> 172.21.15.114:0/2722783128 conn(0x7f99d40741b0 msgr2=0x7f99d4100440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:02.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.362+0000 7f99d8fb5640 1 -- 172.21.15.114:0/2722783128 shutdown_connections 2024-09-13T18:35:02.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.362+0000 7f99d8fb5640 1 -- 172.21.15.114:0/2722783128 wait complete. 2024-09-13T18:35:02.376 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:35:02.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.473+0000 7f5faf0e9640 1 -- 172.21.15.114:0/2495276062 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5fa81005b0 msgr2=0x7f5fa8104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:02.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.473+0000 7f5faf0e9640 1 --2- 172.21.15.114:0/2495276062 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5fa81005b0 0x7f5fa8104a50 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f5f90009930 tx=0x7f5f9002a420 comp rx=0 tx=0).stop 2024-09-13T18:35:02.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.473+0000 7f5faf0e9640 1 -- 172.21.15.114:0/2495276062 shutdown_connections 2024-09-13T18:35:02.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.473+0000 7f5faf0e9640 1 --2- 172.21.15.114:0/2495276062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5fa81051b0 0x7f5fa810bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:02.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.473+0000 7f5faf0e9640 1 --2- 172.21.15.114:0/2495276062 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5fa81005b0 0x7f5fa8104a50 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:02.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.473+0000 7f5faf0e9640 1 --2- 172.21.15.114:0/2495276062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fa80ffc00 0x7f5fa80fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:02.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.473+0000 7f5faf0e9640 1 -- 172.21.15.114:0/2495276062 >> 172.21.15.114:0/2495276062 conn(0x7f5fa80741b0 msgr2=0x7f5fa80745c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:02.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.473+0000 7f5faf0e9640 1 -- 172.21.15.114:0/2495276062 shutdown_connections 2024-09-13T18:35:02.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.473+0000 7f5faf0e9640 1 -- 172.21.15.114:0/2495276062 wait complete. 2024-09-13T18:35:02.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.473+0000 7f5faf0e9640 1 Processor -- start 2024-09-13T18:35:02.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.474+0000 7f5faf0e9640 1 -- start start 2024-09-13T18:35:02.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.474+0000 7f5faf0e9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fa80ffc00 0x7f5fa8195150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:02.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.474+0000 7f5faf0e9640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5fa81051b0 0x7f5fa8195690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:02.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.474+0000 7f5faf0e9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5fa8199a30 0x7f5fa8199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:02.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.474+0000 7f5faf0e9640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa819a420 con 0x7f5fa81051b0 2024-09-13T18:35:02.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.474+0000 7f5faf0e9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa819a590 con 0x7f5fa80ffc00 2024-09-13T18:35:02.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.474+0000 7f5faf0e9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa819a700 con 0x7f5fa8199a30 2024-09-13T18:35:02.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.474+0000 7f5fa7fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5fa81051b0 0x7f5fa8195690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:02.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.474+0000 7f5fa7fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5fa81051b0 0x7f5fa8195690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57346/0 (socket says 172.21.15.114:57346) 2024-09-13T18:35:02.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.474+0000 7f5fa7fff640 1 -- 172.21.15.114:0/3181293365 learned_addr learned my addr 172.21.15.114:0/3181293365 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:35:02.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.474+0000 7f5fad686640 1 --2- 172.21.15.114:0/3181293365 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5fa8199a30 0x7f5fa8199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:02.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.474+0000 7f5face85640 1 --2- 172.21.15.114:0/3181293365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fa80ffc00 0x7f5fa8195150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:02.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.474+0000 7f5fa7fff640 1 -- 172.21.15.114:0/3181293365 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5fa8199a30 msgr2=0x7f5fa8199ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:02.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.474+0000 7f5fa7fff640 1 --2- 172.21.15.114:0/3181293365 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5fa8199a30 0x7f5fa8199ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:02.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.474+0000 7f5fa7fff640 1 -- 172.21.15.114:0/3181293365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fa80ffc00 msgr2=0x7f5fa8195150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:02.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.474+0000 7f5fa7fff640 1 --2- 172.21.15.114:0/3181293365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fa80ffc00 0x7f5fa8195150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:02.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.474+0000 7f5fa7fff640 1 -- 172.21.15.114:0/3181293365 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f90009560 con 0x7f5fa81051b0 2024-09-13T18:35:02.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.474+0000 7f5fa7fff640 1 --2- 172.21.15.114:0/3181293365 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5fa81051b0 0x7f5fa8195690 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f5f90005ab0 tx=0x7f5f90032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:02.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.475+0000 7f5fa5ffb640 1 -- 172.21.15.114:0/3181293365 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f900044d0 con 0x7f5fa81051b0 2024-09-13T18:35:02.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.475+0000 7f5fa5ffb640 1 -- 172.21.15.114:0/3181293365 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5f90032e10 con 0x7f5fa81051b0 2024-09-13T18:35:02.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.475+0000 7f5fa5ffb640 1 -- 172.21.15.114:0/3181293365 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f90003850 con 0x7f5fa81051b0 2024-09-13T18:35:02.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.475+0000 7f5faf0e9640 1 -- 172.21.15.114:0/3181293365 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fa81a1ce0 con 0x7f5fa81051b0 2024-09-13T18:35:02.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.475+0000 7f5faf0e9640 1 -- 172.21.15.114:0/3181293365 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fa81a20b0 con 0x7f5fa81051b0 2024-09-13T18:35:02.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.476+0000 7f5faf0e9640 1 -- 172.21.15.114:0/3181293365 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fa8068060 con 0x7f5fa81051b0 2024-09-13T18:35:02.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.477+0000 7f5fa5ffb640 1 -- 172.21.15.114:0/3181293365 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f5f90004850 con 0x7f5fa81051b0 2024-09-13T18:35:02.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.477+0000 7f5fa5ffb640 1 --2- 172.21.15.114:0/3181293365 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f5f940a1200 0x7f5f940a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:02.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.478+0000 7f5fa5ffb640 1 -- 172.21.15.114:0/3181293365 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f5f900edf50 con 0x7f5fa81051b0 2024-09-13T18:35:02.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.478+0000 7f5face85640 1 --2- 172.21.15.114:0/3181293365 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f5f940a1200 0x7f5f940a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:02.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.478+0000 7f5face85640 1 --2- 172.21.15.114:0/3181293365 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f5f940a1200 0x7f5f940a36c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5f9800bbe0 tx=0x7f5f9800b390 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:02.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.481+0000 7f5fa5ffb640 1 -- 172.21.15.114:0/3181293365 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f5f900b70b0 con 0x7f5fa81051b0 2024-09-13T18:35:02.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.680+0000 7f5faf0e9640 1 -- 172.21.15.114:0/3181293365 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5fa80fffe0 con 0x7f5fa81051b0 2024-09-13T18:35:02.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.681+0000 7f5fa5ffb640 1 -- 172.21.15.114:0/3181293365 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+10945 (secure 0 0 0) 0x7f5f90004e60 con 0x7f5fa81051b0 2024-09-13T18:35:02.683 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:35:02.683 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/4162817497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":4162817497},{"type":"v1","addr":"172.21.15.114:6835","nonce":4162817497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/2435428478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":2435428478},{"type":"v1","addr":"172.21.15.114:6837","nonce":2435428478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-13T18:20:01.895117+0000","modified":"2024-09-13T18:33:48.420534+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24482,"mds_2":24475,"mds_3":24458,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/2046516451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":2046516451},{"type":"v1","addr":"172.21.15.114:6839","nonce":2046516451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":10,"addr":"172.21.15.114:6841/1956923446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":1956923446},{"type":"v1","addr":"172.21.15.114:6841","nonce":1956923446}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24391":{"gid":24391,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/507301496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":507301496},{"type":"v1","addr":"172.21.15.204:6833","nonce":507301496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4028657219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4028657219},{"type":"v1","addr":"172.21.15.134:6833","nonce":4028657219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24445":{"gid":24445,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/57029119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":57029119},{"type":"v1","addr":"172.21.15.204:6835","nonce":57029119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/3068880939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":3068880939},{"type":"v1","addr":"172.21.15.204:6837","nonce":3068880939}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"e","rank":3,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.134:6835/1150003275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1150003275},{"type":"v1","addr":"172.21.15.134:6835","nonce":1150003275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.204:6839/344883912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":344883912},{"type":"v1","addr":"172.21.15.204:6839","nonce":344883912}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6837/891692210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":891692210},{"type":"v1","addr":"172.21.15.134:6837","nonce":891692210}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6839/3466309724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3466309724},{"type":"v1","addr":"172.21.15.134:6839","nonce":3466309724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:35:02.684 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 39 2024-09-13T18:35:02.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.683+0000 7f5faf0e9640 1 -- 172.21.15.114:0/3181293365 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f5f940a1200 msgr2=0x7f5f940a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:02.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.683+0000 7f5faf0e9640 1 --2- 172.21.15.114:0/3181293365 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f5f940a1200 0x7f5f940a36c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5f9800bbe0 tx=0x7f5f9800b390 comp rx=0 tx=0).stop 2024-09-13T18:35:02.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.683+0000 7f5faf0e9640 1 -- 172.21.15.114:0/3181293365 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5fa81051b0 msgr2=0x7f5fa8195690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:02.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.683+0000 7f5faf0e9640 1 --2- 172.21.15.114:0/3181293365 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5fa81051b0 0x7f5fa8195690 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f5f90005ab0 tx=0x7f5f90032c20 comp rx=0 tx=0).stop 2024-09-13T18:35:02.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.683+0000 7f5faf0e9640 1 -- 172.21.15.114:0/3181293365 shutdown_connections 2024-09-13T18:35:02.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.683+0000 7f5faf0e9640 1 --2- 172.21.15.114:0/3181293365 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f5f940a1200 0x7f5f940a36c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:02.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.683+0000 7f5faf0e9640 1 --2- 172.21.15.114:0/3181293365 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5fa8199a30 0x7f5fa8199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:02.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.683+0000 7f5faf0e9640 1 --2- 172.21.15.114:0/3181293365 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5fa81051b0 0x7f5fa8195690 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:02.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.683+0000 7f5faf0e9640 1 --2- 172.21.15.114:0/3181293365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fa80ffc00 0x7f5fa8195150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:02.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.683+0000 7f5faf0e9640 1 -- 172.21.15.114:0/3181293365 >> 172.21.15.114:0/3181293365 conn(0x7f5fa80741b0 msgr2=0x7f5fa8102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:02.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.683+0000 7f5faf0e9640 1 -- 172.21.15.114:0/3181293365 shutdown_connections 2024-09-13T18:35:02.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.683+0000 7f5faf0e9640 1 -- 172.21.15.114:0/3181293365 wait complete. 2024-09-13T18:35:02.697 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:35:02.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.793+0000 7f42b0e6c640 1 -- 172.21.15.114:0/1663116591 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42ac0ff830 msgr2=0x7f42ac103870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:02.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.793+0000 7f42b0e6c640 1 --2- 172.21.15.114:0/1663116591 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42ac0ff830 0x7f42ac103870 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f4294009930 tx=0x7f429402a420 comp rx=0 tx=0).stop 2024-09-13T18:35:02.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.793+0000 7f42b0e6c640 1 -- 172.21.15.114:0/1663116591 shutdown_connections 2024-09-13T18:35:02.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.793+0000 7f42b0e6c640 1 --2- 172.21.15.114:0/1663116591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42ac103fd0 0x7f42ac10bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:02.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.793+0000 7f42b0e6c640 1 --2- 172.21.15.114:0/1663116591 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42ac0ff830 0x7f42ac103870 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:02.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.793+0000 7f42b0e6c640 1 --2- 172.21.15.114:0/1663116591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42ac0fee80 0x7f42ac0ff260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:02.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.793+0000 7f42b0e6c640 1 -- 172.21.15.114:0/1663116591 >> 172.21.15.114:0/1663116591 conn(0x7f42ac075c60 msgr2=0x7f42ac0fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:02.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.793+0000 7f42b0e6c640 1 -- 172.21.15.114:0/1663116591 shutdown_connections 2024-09-13T18:35:02.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.793+0000 7f42b0e6c640 1 -- 172.21.15.114:0/1663116591 wait complete. 2024-09-13T18:35:02.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.793+0000 7f42b0e6c640 1 Processor -- start 2024-09-13T18:35:02.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.793+0000 7f42b0e6c640 1 -- start start 2024-09-13T18:35:02.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.794+0000 7f42b0e6c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42ac0fee80 0x7f42ac196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:02.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.794+0000 7f42b0e6c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42ac103fd0 0x7f42ac1967c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:02.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.794+0000 7f42b0e6c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42ac19abb0 0x7f42ac19b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:02.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.794+0000 7f42b0e6c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ac19b5a0 con 0x7f42ac19abb0 2024-09-13T18:35:02.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.794+0000 7f42b0e6c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ac19b710 con 0x7f42ac103fd0 2024-09-13T18:35:02.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.794+0000 7f42b0e6c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ac19b880 con 0x7f42ac0fee80 2024-09-13T18:35:02.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.794+0000 7f42aad9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42ac19abb0 0x7f42ac19b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:02.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.794+0000 7f42aad9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42ac19abb0 0x7f42ac19b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57360/0 (socket says 172.21.15.114:57360) 2024-09-13T18:35:02.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.794+0000 7f42aad9d640 1 -- 172.21.15.114:0/170187350 learned_addr learned my addr 172.21.15.114:0/170187350 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:35:02.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.794+0000 7f42aa59c640 1 --2- 172.21.15.114:0/170187350 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42ac0fee80 0x7f42ac196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:02.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.794+0000 7f42a9d9b640 1 --2- 172.21.15.114:0/170187350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42ac103fd0 0x7f42ac1967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:02.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.794+0000 7f42aad9d640 1 -- 172.21.15.114:0/170187350 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42ac0fee80 msgr2=0x7f42ac196280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:02.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.794+0000 7f42aad9d640 1 --2- 172.21.15.114:0/170187350 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42ac0fee80 0x7f42ac196280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:02.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.794+0000 7f42aad9d640 1 -- 172.21.15.114:0/170187350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42ac103fd0 msgr2=0x7f42ac1967c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:02.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.794+0000 7f42aad9d640 1 --2- 172.21.15.114:0/170187350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42ac103fd0 0x7f42ac1967c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:02.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.794+0000 7f42aad9d640 1 -- 172.21.15.114:0/170187350 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4294009560 con 0x7f42ac19abb0 2024-09-13T18:35:02.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.794+0000 7f42aad9d640 1 --2- 172.21.15.114:0/170187350 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42ac19abb0 0x7f42ac19b060 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f429c009fd0 tx=0x7f429c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:02.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.794+0000 7f42a37fe640 1 -- 172.21.15.114:0/170187350 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429c011830 con 0x7f42ac19abb0 2024-09-13T18:35:02.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.794+0000 7f42a37fe640 1 -- 172.21.15.114:0/170187350 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f429c011e20 con 0x7f42ac19abb0 2024-09-13T18:35:02.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.794+0000 7f42b0e6c640 1 -- 172.21.15.114:0/170187350 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42ac1a2e60 con 0x7f42ac19abb0 2024-09-13T18:35:02.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.795+0000 7f42a37fe640 1 -- 172.21.15.114:0/170187350 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429c010e50 con 0x7f42ac19abb0 2024-09-13T18:35:02.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.795+0000 7f42b0e6c640 1 -- 172.21.15.114:0/170187350 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42ac1a32e0 con 0x7f42ac19abb0 2024-09-13T18:35:02.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.796+0000 7f42b0e6c640 1 -- 172.21.15.114:0/170187350 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42ac100350 con 0x7f42ac19abb0 2024-09-13T18:35:02.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.797+0000 7f42a37fe640 1 -- 172.21.15.114:0/170187350 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f429c011990 con 0x7f42ac19abb0 2024-09-13T18:35:02.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.798+0000 7f42a37fe640 1 --2- 172.21.15.114:0/170187350 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f42980a1280 0x7f42980a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:02.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.798+0000 7f42aa59c640 1 --2- 172.21.15.114:0/170187350 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f42980a1280 0x7f42980a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:02.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.798+0000 7f42a37fe640 1 -- 172.21.15.114:0/170187350 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f429c0cc6f0 con 0x7f42ac19abb0 2024-09-13T18:35:02.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.798+0000 7f42aa59c640 1 --2- 172.21.15.114:0/170187350 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f42980a1280 0x7f42980a3740 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f42a4005c40 tx=0x7f42a400a570 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:02.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.802+0000 7f42a37fe640 1 -- 172.21.15.114:0/170187350 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f429c0958b0 con 0x7f42ac19abb0 2024-09-13T18:35:02.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:02 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/2722783128' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:35:02.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:02 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/2722783128' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:35:02.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.967+0000 7f42b0e6c640 1 -- 172.21.15.114:0/170187350 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f42ac0ff260 con 0x7f42ac19abb0 2024-09-13T18:35:02.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:02 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2722783128' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:35:02.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.969+0000 7f42a37fe640 1 -- 172.21.15.114:0/170187350 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+25265 (secure 0 0 0) 0x7f429c098ed0 con 0x7f42ac19abb0 2024-09-13T18:35:02.971 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-13T18:35:02.971 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":106,"fsid":"97e7edf2-71fb-11ef-bceb-c7b262605968","created":"2024-09-13T18:12:36.360625+0000","modified":"2024-09-13T18:34:18.467182+0000","last_up_change":"2024-09-13T18:19:12.350370+0000","last_in_change":"2024-09-13T18:18:52.321596+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:15:40.129659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:20:00.024407+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:20:01.085549+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":105,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T18:20:04.334216+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":105,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9914f01a-9f6f-4878-b955-83570199c354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6803","nonce":3499525826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6805","nonce":3499525826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6809","nonce":3499525826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3499525826},{"type":"v1","addr":"172.21.15.114:6807","nonce":3499525826}]},"public_addr":"172.21.15.114:6803/3499525826","cluster_addr":"172.21.15.114:6805/3499525826","heartbeat_back_addr":"172.21.15.114:6809/3499525826","heartbeat_front_addr":"172.21.15.114:6807/3499525826","state":["exists","up"]},{"osd":1,"uuid":"ef65e012-7469-4c20-b354-17022bca63c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6801","nonce":4264264866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6803","nonce":4264264866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6807","nonce":4264264866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4264264866},{"type":"v1","addr":"172.21.15.134:6805","nonce":4264264866}]},"public_addr":"172.21.15.134:6801/4264264866","cluster_addr":"172.21.15.134:6803/4264264866","heartbeat_back_addr":"172.21.15.134:6807/4264264866","heartbeat_front_addr":"172.21.15.134:6805/4264264866","state":["exists","up"]},{"osd":2,"uuid":"578213fd-715f-4be7-91fa-254f3d0e3022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6801","nonce":2288807707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6803","nonce":2288807707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6807","nonce":2288807707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":2288807707},{"type":"v1","addr":"172.21.15.204:6805","nonce":2288807707}]},"public_addr":"172.21.15.204:6801/2288807707","cluster_addr":"172.21.15.204:6803/2288807707","heartbeat_back_addr":"172.21.15.204:6807/2288807707","heartbeat_front_addr":"172.21.15.204:6805/2288807707","state":["exists","up"]},{"osd":3,"uuid":"b64aab3f-1e52-4828-8397-74af5e05f2cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6811","nonce":2904204254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6813","nonce":2904204254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6817","nonce":2904204254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2904204254},{"type":"v1","addr":"172.21.15.114:6815","nonce":2904204254}]},"public_addr":"172.21.15.114:6811/2904204254","cluster_addr":"172.21.15.114:6813/2904204254","heartbeat_back_addr":"172.21.15.114:6817/2904204254","heartbeat_front_addr":"172.21.15.114:6815/2904204254","state":["exists","up"]},{"osd":4,"uuid":"f38943d6-39c0-42e1-b427-cb6bb90ac38a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6809","nonce":1773945333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6811","nonce":1773945333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6815","nonce":1773945333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1773945333},{"type":"v1","addr":"172.21.15.134:6813","nonce":1773945333}]},"public_addr":"172.21.15.134:6809/1773945333","cluster_addr":"172.21.15.134:6811/1773945333","heartbeat_back_addr":"172.21.15.134:6815/1773945333","heartbeat_front_addr":"172.21.15.134:6813/1773945333","state":["exists","up"]},{"osd":5,"uuid":"bb38f361-8e00-4178-9752-5ddbf3af20e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6809","nonce":2371778055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6811","nonce":2371778055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6815","nonce":2371778055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2371778055},{"type":"v1","addr":"172.21.15.204:6813","nonce":2371778055}]},"public_addr":"172.21.15.204:6809/2371778055","cluster_addr":"172.21.15.204:6811/2371778055","heartbeat_back_addr":"172.21.15.204:6815/2371778055","heartbeat_front_addr":"172.21.15.204:6813/2371778055","state":["exists","up"]},{"osd":6,"uuid":"93d40a03-e7ff-4a3f-9efa-5c5403c9983d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6819","nonce":1484325044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6821","nonce":1484325044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6825","nonce":1484325044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1484325044},{"type":"v1","addr":"172.21.15.114:6823","nonce":1484325044}]},"public_addr":"172.21.15.114:6819/1484325044","cluster_addr":"172.21.15.114:6821/1484325044","heartbeat_back_addr":"172.21.15.114:6825/1484325044","heartbeat_front_addr":"172.21.15.114:6823/1484325044","state":["exists","up"]},{"osd":7,"uuid":"982f629a-1b95-4be1-8e75-5ddce1ad77b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6817","nonce":799246614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6819","nonce":799246614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6823","nonce":799246614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":799246614},{"type":"v1","addr":"172.21.15.134:6821","nonce":799246614}]},"public_addr":"172.21.15.134:6817/799246614","cluster_addr":"172.21.15.134:6819/799246614","heartbeat_back_addr":"172.21.15.134:6823/799246614","heartbeat_front_addr":"172.21.15.134:6821/799246614","state":["exists","up"]},{"osd":8,"uuid":"409beb9b-8227-4dad-a4a0-f0e5ce70dc5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6817","nonce":858331805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6819","nonce":858331805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6823","nonce":858331805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":858331805},{"type":"v1","addr":"172.21.15.204:6821","nonce":858331805}]},"public_addr":"172.21.15.204:6817/858331805","cluster_addr":"172.21.15.204:6819/858331805","heartbeat_back_addr":"172.21.15.204:6823/858331805","heartbeat_front_addr":"172.21.15.204:6821/858331805","state":["exists","up"]},{"osd":9,"uuid":"a345a15c-b1fc-4877-b445-418eebf0d0ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6827","nonce":2172392854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6829","nonce":2172392854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6833","nonce":2172392854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2172392854},{"type":"v1","addr":"172.21.15.114:6831","nonce":2172392854}]},"public_addr":"172.21.15.114:6827/2172392854","cluster_addr":"172.21.15.114:6829/2172392854","heartbeat_back_addr":"172.21.15.114:6833/2172392854","heartbeat_front_addr":"172.21.15.114:6831/2172392854","state":["exists","up"]},{"osd":10,"uuid":"464d995f-d90b-4caa-addb-1f7562bd74b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6825","nonce":3073513468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6827","nonce":3073513468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6831","nonce":3073513468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3073513468},{"type":"v1","addr":"172.21.15.134:6829","nonce":3073513468}]},"public_addr":"172.21.15.134:6825/3073513468","cluster_addr":"172.21.15.134:6827/3073513468","heartbeat_back_addr":"172.21.15.134:6831/3073513468","heartbeat_front_addr":"172.21.15.134:6829/3073513468","state":["exists","up"]},{"osd":11,"uuid":"71dd794d-326c-477f-8d02-ee32bff1350d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6825","nonce":3239521765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6827","nonce":3239521765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6831","nonce":3239521765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3239521765},{"type":"v1","addr":"172.21.15.204:6829","nonce":3239521765}]},"public_addr":"172.21.15.204:6825/3239521765","cluster_addr":"172.21.15.204:6827/3239521765","heartbeat_back_addr":"172.21.15.204:6831/3239521765","heartbeat_front_addr":"172.21.15.204:6829/3239521765","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:14:54.542687+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:15:15.746055+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:15:38.371131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:01.383932+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:24.796237+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:16:48.422214+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:11.284323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:34.570365+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:17:58.232855+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:18:21.609596+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:18:45.565859+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:19:10.017969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:6801/3396185541":"2024-09-14T18:22:28.057623+0000","172.21.15.114:6800/830738794":"2024-09-14T18:13:03.521116+0000","172.21.15.114:6801/830738794":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/1684954293":"2024-09-14T18:22:28.057623+0000","172.21.15.114:0/3544951322":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/722571999":"2024-09-14T18:13:03.521116+0000","172.21.15.114:0/2949422849":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2034742251":"2024-09-14T18:13:03.521116+0000","172.21.15.114:6800/4281866858":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/3407523320":"2024-09-14T18:22:28.057623+0000","172.21.15.114:6800/3396185541":"2024-09-14T18:22:28.057623+0000","172.21.15.114:0/4257828906":"2024-09-14T18:13:25.823027+0000","172.21.15.114:6801/4281866858":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2828364892":"2024-09-14T18:13:25.823027+0000","172.21.15.114:0/2714091073":"2024-09-14T18:22:28.057623+0000","172.21.15.114:0/842237781":"2024-09-14T18:22:28.057623+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]},{"pool":4,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:35:02.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.971+0000 7f42b0e6c640 1 -- 172.21.15.114:0/170187350 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f42980a1280 msgr2=0x7f42980a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:02.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.971+0000 7f42b0e6c640 1 --2- 172.21.15.114:0/170187350 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f42980a1280 0x7f42980a3740 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f42a4005c40 tx=0x7f42a400a570 comp rx=0 tx=0).stop 2024-09-13T18:35:02.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.971+0000 7f42b0e6c640 1 -- 172.21.15.114:0/170187350 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42ac19abb0 msgr2=0x7f42ac19b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:02.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.971+0000 7f42b0e6c640 1 --2- 172.21.15.114:0/170187350 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42ac19abb0 0x7f42ac19b060 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f429c009fd0 tx=0x7f429c00b8e0 comp rx=0 tx=0).stop 2024-09-13T18:35:02.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.971+0000 7f42b0e6c640 1 -- 172.21.15.114:0/170187350 shutdown_connections 2024-09-13T18:35:02.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.971+0000 7f42b0e6c640 1 --2- 172.21.15.114:0/170187350 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f42980a1280 0x7f42980a3740 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:02.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.971+0000 7f42b0e6c640 1 --2- 172.21.15.114:0/170187350 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42ac19abb0 0x7f42ac19b060 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:02.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.971+0000 7f42b0e6c640 1 --2- 172.21.15.114:0/170187350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42ac103fd0 0x7f42ac1967c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:02.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.971+0000 7f42b0e6c640 1 --2- 172.21.15.114:0/170187350 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42ac0fee80 0x7f42ac196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:02.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.971+0000 7f42b0e6c640 1 -- 172.21.15.114:0/170187350 >> 172.21.15.114:0/170187350 conn(0x7f42ac075c60 msgr2=0x7f42ac0677a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:02.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.971+0000 7f42b0e6c640 1 -- 172.21.15.114:0/170187350 shutdown_connections 2024-09-13T18:35:02.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:02.971+0000 7f42b0e6c640 1 -- 172.21.15.114:0/170187350 wait complete. 2024-09-13T18:35:02.985 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-13T18:35:03.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.085+0000 7f51df357640 1 -- 172.21.15.114:0/3484774894 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51d8103fa0 msgr2=0x7f51d810baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:03.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.085+0000 7f51df357640 1 --2- 172.21.15.114:0/3484774894 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51d8103fa0 0x7f51d810baf0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f51d0009cd0 tx=0x7f51d002a8e0 comp rx=0 tx=0).stop 2024-09-13T18:35:03.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.085+0000 7f51df357640 1 -- 172.21.15.114:0/3484774894 shutdown_connections 2024-09-13T18:35:03.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.085+0000 7f51df357640 1 --2- 172.21.15.114:0/3484774894 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51d8103fa0 0x7f51d810baf0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:03.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.085+0000 7f51df357640 1 --2- 172.21.15.114:0/3484774894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51d80ff850 0x7f51d8103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:03.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.085+0000 7f51df357640 1 --2- 172.21.15.114:0/3484774894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51d80feea0 0x7f51d80ff280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:03.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.085+0000 7f51df357640 1 -- 172.21.15.114:0/3484774894 >> 172.21.15.114:0/3484774894 conn(0x7f51d8075c60 msgr2=0x7f51d80fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:03.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.085+0000 7f51df357640 1 -- 172.21.15.114:0/3484774894 shutdown_connections 2024-09-13T18:35:03.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.085+0000 7f51df357640 1 -- 172.21.15.114:0/3484774894 wait complete. 2024-09-13T18:35:03.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.085+0000 7f51df357640 1 Processor -- start 2024-09-13T18:35:03.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.085+0000 7f51df357640 1 -- start start 2024-09-13T18:35:03.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.086+0000 7f51df357640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51d80feea0 0x7f51d8196160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:03.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.086+0000 7f51df357640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51d8103fa0 0x7f51d81966a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:03.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.086+0000 7f51df357640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51d819aa90 0x7f51d819af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:03.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.086+0000 7f51df357640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d819b480 con 0x7f51d80feea0 2024-09-13T18:35:03.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.086+0000 7f51df357640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d819b5f0 con 0x7f51d8103fa0 2024-09-13T18:35:03.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.086+0000 7f51df357640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d819b760 con 0x7f51d819aa90 2024-09-13T18:35:03.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.086+0000 7f51dd0f3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51d80feea0 0x7f51d8196160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:03.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.086+0000 7f51dd0f3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51d80feea0 0x7f51d8196160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57386/0 (socket says 172.21.15.114:57386) 2024-09-13T18:35:03.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.086+0000 7f51dd0f3640 1 -- 172.21.15.114:0/2325856669 learned_addr learned my addr 172.21.15.114:0/2325856669 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:35:03.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.086+0000 7f51dc8f2640 1 --2- 172.21.15.114:0/2325856669 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51d8103fa0 0x7f51d81966a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:03.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.086+0000 7f51dd8f4640 1 --2- 172.21.15.114:0/2325856669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51d819aa90 0x7f51d819af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:03.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.086+0000 7f51dd0f3640 1 -- 172.21.15.114:0/2325856669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51d819aa90 msgr2=0x7f51d819af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:03.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.086+0000 7f51dd0f3640 1 --2- 172.21.15.114:0/2325856669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51d819aa90 0x7f51d819af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:03.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.086+0000 7f51dd0f3640 1 -- 172.21.15.114:0/2325856669 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51d8103fa0 msgr2=0x7f51d81966a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:03.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.086+0000 7f51dd0f3640 1 --2- 172.21.15.114:0/2325856669 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51d8103fa0 0x7f51d81966a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:03.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.086+0000 7f51dd0f3640 1 -- 172.21.15.114:0/2325856669 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d0009900 con 0x7f51d80feea0 2024-09-13T18:35:03.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.086+0000 7f51dc8f2640 1 --2- 172.21.15.114:0/2325856669 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51d8103fa0 0x7f51d81966a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:35:03.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.086+0000 7f51dd0f3640 1 --2- 172.21.15.114:0/2325856669 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51d80feea0 0x7f51d8196160 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f51cc00e9f0 tx=0x7f51cc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:03.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.086+0000 7f51ca7fc640 1 -- 172.21.15.114:0/2325856669 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51cc009720 con 0x7f51d80feea0 2024-09-13T18:35:03.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.086+0000 7f51ca7fc640 1 -- 172.21.15.114:0/2325856669 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f51cc00cd80 con 0x7f51d80feea0 2024-09-13T18:35:03.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.087+0000 7f51df357640 1 -- 172.21.15.114:0/2325856669 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51d81a2d40 con 0x7f51d80feea0 2024-09-13T18:35:03.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.087+0000 7f51ca7fc640 1 -- 172.21.15.114:0/2325856669 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51cc010430 con 0x7f51d80feea0 2024-09-13T18:35:03.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.087+0000 7f51df357640 1 -- 172.21.15.114:0/2325856669 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51d81a31a0 con 0x7f51d80feea0 2024-09-13T18:35:03.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.087+0000 7f51df357640 1 -- 172.21.15.114:0/2325856669 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51d8100320 con 0x7f51d80feea0 2024-09-13T18:35:03.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.090+0000 7f51ca7fc640 1 -- 172.21.15.114:0/2325856669 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f51cc004f90 con 0x7f51d80feea0 2024-09-13T18:35:03.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.090+0000 7f51ca7fc640 1 --2- 172.21.15.114:0/2325856669 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f51c40a1200 0x7f51c40a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:03.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.090+0000 7f51ca7fc640 1 -- 172.21.15.114:0/2325856669 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f51cc014070 con 0x7f51d80feea0 2024-09-13T18:35:03.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.090+0000 7f51dc8f2640 1 --2- 172.21.15.114:0/2325856669 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f51c40a1200 0x7f51c40a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:03.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.091+0000 7f51dc8f2640 1 --2- 172.21.15.114:0/2325856669 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f51c40a1200 0x7f51c40a36c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f51d8197700 tx=0x7f51d4005f90 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:03.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.092+0000 7f51ca7fc640 1 -- 172.21.15.114:0/2325856669 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f51cc096720 con 0x7f51d80feea0 2024-09-13T18:35:03.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.295+0000 7f51df357640 1 -- 172.21.15.114:0/2325856669 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f51d80ff850 con 0x7f51d80feea0 2024-09-13T18:35:03.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.319+0000 7f51ca7fc640 1 -- 172.21.15.114:0/2325856669 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v40) v1 ==== 161+0+0 (secure 0 0 0) 0x7f51cc099d40 con 0x7f51d80feea0 2024-09-13T18:35:03.322 INFO:teuthology.orchestra.run.smithi114.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-13T18:35:03.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.323+0000 7f51df357640 1 -- 172.21.15.114:0/2325856669 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f51c40a1200 msgr2=0x7f51c40a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:03.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.323+0000 7f51df357640 1 --2- 172.21.15.114:0/2325856669 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f51c40a1200 0x7f51c40a36c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f51d8197700 tx=0x7f51d4005f90 comp rx=0 tx=0).stop 2024-09-13T18:35:03.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.323+0000 7f51df357640 1 -- 172.21.15.114:0/2325856669 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51d80feea0 msgr2=0x7f51d8196160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:03.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.323+0000 7f51df357640 1 --2- 172.21.15.114:0/2325856669 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51d80feea0 0x7f51d8196160 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f51cc00e9f0 tx=0x7f51cc00edf0 comp rx=0 tx=0).stop 2024-09-13T18:35:03.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.323+0000 7f51df357640 1 -- 172.21.15.114:0/2325856669 shutdown_connections 2024-09-13T18:35:03.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.323+0000 7f51df357640 1 --2- 172.21.15.114:0/2325856669 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f51c40a1200 0x7f51c40a36c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:03.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.323+0000 7f51df357640 1 --2- 172.21.15.114:0/2325856669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51d819aa90 0x7f51d819af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:03.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.323+0000 7f51df357640 1 --2- 172.21.15.114:0/2325856669 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51d8103fa0 0x7f51d81966a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:03.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.323+0000 7f51df357640 1 --2- 172.21.15.114:0/2325856669 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51d80feea0 0x7f51d8196160 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:03.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.323+0000 7f51df357640 1 -- 172.21.15.114:0/2325856669 >> 172.21.15.114:0/2325856669 conn(0x7f51d8075c60 msgr2=0x7f51d8067740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:03.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.323+0000 7f51df357640 1 -- 172.21.15.114:0/2325856669 shutdown_connections 2024-09-13T18:35:03.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.323+0000 7f51df357640 1 -- 172.21.15.114:0/2325856669 wait complete. 2024-09-13T18:35:03.338 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-13T18:35:03.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.446+0000 7f5ebb2b1640 1 -- 172.21.15.114:0/3122422970 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5eb4101860 msgr2=0x7f5eb4109d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:03.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.446+0000 7f5ebb2b1640 1 --2- 172.21.15.114:0/3122422970 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5eb4101860 0x7f5eb4109d40 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4009900 tx=0x7f5ea402a480 comp rx=0 tx=0).stop 2024-09-13T18:35:03.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.446+0000 7f5ebb2b1640 1 -- 172.21.15.114:0/3122422970 shutdown_connections 2024-09-13T18:35:03.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.446+0000 7f5ebb2b1640 1 --2- 172.21.15.114:0/3122422970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5eb410a310 0x7f5eb410c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:03.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.446+0000 7f5ebb2b1640 1 --2- 172.21.15.114:0/3122422970 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5eb4101860 0x7f5eb4109d40 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:03.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.446+0000 7f5ebb2b1640 1 --2- 172.21.15.114:0/3122422970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5eb4100f40 0x7f5eb4101320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:03.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.446+0000 7f5ebb2b1640 1 -- 172.21.15.114:0/3122422970 >> 172.21.15.114:0/3122422970 conn(0x7f5eb4075c60 msgr2=0x7f5eb40fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:03.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.446+0000 7f5ebb2b1640 1 -- 172.21.15.114:0/3122422970 shutdown_connections 2024-09-13T18:35:03.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.446+0000 7f5ebb2b1640 1 -- 172.21.15.114:0/3122422970 wait complete. 2024-09-13T18:35:03.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.447+0000 7f5ebb2b1640 1 Processor -- start 2024-09-13T18:35:03.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.447+0000 7f5ebb2b1640 1 -- start start 2024-09-13T18:35:03.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.447+0000 7f5ebb2b1640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5eb4100f40 0x7f5eb41961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:03.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.447+0000 7f5ebb2b1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5eb4101860 0x7f5eb4196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:03.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.447+0000 7f5ebb2b1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5eb410a310 0x7f5eb419ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:03.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.447+0000 7f5ebb2b1640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb419b110 con 0x7f5eb4100f40 2024-09-13T18:35:03.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.447+0000 7f5ebb2b1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb419b280 con 0x7f5eb4101860 2024-09-13T18:35:03.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.447+0000 7f5ebb2b1640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb419b440 con 0x7f5eb410a310 2024-09-13T18:35:03.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.447+0000 7f5eb904d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5eb4100f40 0x7f5eb41961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:03.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.447+0000 7f5eb904d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5eb4100f40 0x7f5eb41961e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35650/0 (socket says 172.21.15.114:35650) 2024-09-13T18:35:03.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.447+0000 7f5eb904d640 1 -- 172.21.15.114:0/1642421111 learned_addr learned my addr 172.21.15.114:0/1642421111 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:35:03.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.447+0000 7f5eb984e640 1 --2- 172.21.15.114:0/1642421111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5eb410a310 0x7f5eb419ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:03.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.447+0000 7f5eb884c640 1 --2- 172.21.15.114:0/1642421111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5eb4101860 0x7f5eb4196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:03.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.447+0000 7f5eb904d640 1 -- 172.21.15.114:0/1642421111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5eb410a310 msgr2=0x7f5eb419ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:03.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.447+0000 7f5eb904d640 1 --2- 172.21.15.114:0/1642421111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5eb410a310 0x7f5eb419ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:03.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.447+0000 7f5eb904d640 1 -- 172.21.15.114:0/1642421111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5eb4101860 msgr2=0x7f5eb4196720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:03.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.447+0000 7f5eb904d640 1 --2- 172.21.15.114:0/1642421111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5eb4101860 0x7f5eb4196720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:03.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.447+0000 7f5eb904d640 1 -- 172.21.15.114:0/1642421111 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea4009560 con 0x7f5eb4100f40 2024-09-13T18:35:03.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.447+0000 7f5eb984e640 1 --2- 172.21.15.114:0/1642421111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5eb410a310 0x7f5eb419ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:35:03.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.447+0000 7f5eb904d640 1 --2- 172.21.15.114:0/1642421111 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5eb4100f40 0x7f5eb41961e0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f5eb000e9f0 tx=0x7f5eb000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:03.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.448+0000 7f5eaa7fc640 1 -- 172.21.15.114:0/1642421111 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0009720 con 0x7f5eb4100f40 2024-09-13T18:35:03.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.448+0000 7f5eaa7fc640 1 -- 172.21.15.114:0/1642421111 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5eb000cd80 con 0x7f5eb4100f40 2024-09-13T18:35:03.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.448+0000 7f5ebb2b1640 1 -- 172.21.15.114:0/1642421111 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eb419b720 con 0x7f5eb4100f40 2024-09-13T18:35:03.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.448+0000 7f5eaa7fc640 1 -- 172.21.15.114:0/1642421111 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0010430 con 0x7f5eb4100f40 2024-09-13T18:35:03.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.448+0000 7f5ebb2b1640 1 -- 172.21.15.114:0/1642421111 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eb41a2fa0 con 0x7f5eb4100f40 2024-09-13T18:35:03.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.449+0000 7f5ebb2b1640 1 -- 172.21.15.114:0/1642421111 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eb4067e00 con 0x7f5eb4100f40 2024-09-13T18:35:03.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.451+0000 7f5eaa7fc640 1 -- 172.21.15.114:0/1642421111 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f5eb0004f90 con 0x7f5eb4100f40 2024-09-13T18:35:03.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.452+0000 7f5eaa7fc640 1 --2- 172.21.15.114:0/1642421111 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f5ea00a1280 0x7f5ea00a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:03.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.452+0000 7f5eb884c640 1 --2- 172.21.15.114:0/1642421111 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f5ea00a1280 0x7f5ea00a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:03.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.452+0000 7f5eaa7fc640 1 -- 172.21.15.114:0/1642421111 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9795+0+0 (secure 0 0 0) 0x7f5eb0014070 con 0x7f5eb4100f40 2024-09-13T18:35:03.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.452+0000 7f5eb884c640 1 --2- 172.21.15.114:0/1642421111 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f5ea00a1280 0x7f5ea00a3740 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5ea402aa70 tx=0x7f5ea4005a20 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:03.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.454+0000 7f5eaa7fc640 1 -- 172.21.15.114:0/1642421111 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f5eb0096610 con 0x7f5eb4100f40 2024-09-13T18:35:03.559 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:03 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mon-a[30893]: 2024-09-13T18:35:03.309+0000 7faedbcb3640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-13T18:35:03.559 INFO:journalctl@ceph.mds.g.smithi114.stdout:Sep 13 18:35:03 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-g[70858]: ignoring --setuser ceph since I am not root 2024-09-13T18:35:03.559 INFO:journalctl@ceph.mds.g.smithi114.stdout:Sep 13 18:35:03 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-g[70858]: ignoring --setgroup ceph since I am not root 2024-09-13T18:35:03.559 INFO:journalctl@ceph.mds.g.smithi114.stdout:Sep 13 18:35:03 smithi114 ceph-mds[70862]: -- 172.21.15.114:0/3140672288 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b9afecad00 con 0x55b9afe9f000 2024-09-13T18:35:03.559 INFO:journalctl@ceph.mds.g.smithi114.stdout:Sep 13 18:35:03 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-g[70858]: starting mds.g at 2024-09-13T18:35:03.560 INFO:journalctl@ceph.mds.a.smithi114.stdout:Sep 13 18:35:03 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-a[71215]: ignoring --setuser ceph since I am not root 2024-09-13T18:35:03.560 INFO:journalctl@ceph.mds.a.smithi114.stdout:Sep 13 18:35:03 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-a[71215]: ignoring --setgroup ceph since I am not root 2024-09-13T18:35:03.560 INFO:journalctl@ceph.mds.a.smithi114.stdout:Sep 13 18:35:03 smithi114 ceph-mds[71233]: -- 172.21.15.114:0/829815912 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558e02a0ed00 con 0x558e029e3400 2024-09-13T18:35:03.560 INFO:journalctl@ceph.mds.a.smithi114.stdout:Sep 13 18:35:03 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-a[71215]: starting mds.a at 2024-09-13T18:35:03.560 INFO:journalctl@ceph.mds.a.smithi114.stdout:Sep 13 18:35:03 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-a[71215]: 2024-09-13T18:35:03.382+0000 7f3af6ebf640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6838/0: (98) Address already in use 2024-09-13T18:35:03.560 INFO:journalctl@ceph.mds.a.smithi114.stdout:Sep 13 18:35:03 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-a[71215]: 2024-09-13T18:35:03.382+0000 7f3af6ebf640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6840/0: (98) Address already in use 2024-09-13T18:35:03.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:03 smithi134 ceph-mon[34283]: pgmap v392: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-13T18:35:03.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:03 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/3181293365' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:35:03.619 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:03 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/170187350' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:35:03.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:03 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/2325856669' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-13T18:35:03.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:03 smithi134 ceph-mon[34283]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-13T18:35:03.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:03 smithi134 ceph-mon[34283]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-13T18:35:03.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:03 smithi134 ceph-mon[34283]: osdmap e107: 12 total, 12 up, 12 in 2024-09-13T18:35:03.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:03 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/2325856669' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-13T18:35:03.620 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:03 smithi134 ceph-mon[34283]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-13T18:35:03.620 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 13 18:35:03 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-e[63289]: ignoring --setuser ceph since I am not root 2024-09-13T18:35:03.621 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 13 18:35:03 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-e[63289]: ignoring --setgroup ceph since I am not root 2024-09-13T18:35:03.621 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 13 18:35:03 smithi134 ceph-mds[63293]: -- 172.21.15.134:0/2497538695 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c893b60d00 con 0x55c893b34800 2024-09-13T18:35:03.621 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 13 18:35:03 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-e[63289]: starting mds.e at 2024-09-13T18:35:03.621 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 13 18:35:03 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-b[63915]: ignoring --setuser ceph since I am not root 2024-09-13T18:35:03.621 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 13 18:35:03 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-b[63915]: ignoring --setgroup ceph since I am not root 2024-09-13T18:35:03.621 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 13 18:35:03 smithi134 ceph-mds[63920]: -- 172.21.15.134:0/376329167 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56106c848d00 con 0x56106c81d000 2024-09-13T18:35:03.621 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 13 18:35:03 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-b[63915]: starting mds.b at 2024-09-13T18:35:03.622 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 13 18:35:03 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-h[63594]: ignoring --setuser ceph since I am not root 2024-09-13T18:35:03.622 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 13 18:35:03 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-h[63594]: ignoring --setgroup ceph since I am not root 2024-09-13T18:35:03.622 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 13 18:35:03 smithi134 ceph-mds[63598]: -- 172.21.15.134:0/2874533698 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556b449d6d00 con 0x556b449ab000 2024-09-13T18:35:03.622 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 13 18:35:03 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-h[63594]: starting mds.h at 2024-09-13T18:35:03.622 INFO:journalctl@ceph.mds.k.smithi134.stdout:Sep 13 18:35:03 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-k[62972]: ignoring --setuser ceph since I am not root 2024-09-13T18:35:03.622 INFO:journalctl@ceph.mds.k.smithi134.stdout:Sep 13 18:35:03 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-k[62972]: ignoring --setgroup ceph since I am not root 2024-09-13T18:35:03.623 INFO:journalctl@ceph.mds.k.smithi134.stdout:Sep 13 18:35:03 smithi134 ceph-mds[62976]: -- 172.21.15.134:0/4222221270 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560ee2d68d00 con 0x560ee2d3c800 2024-09-13T18:35:03.623 INFO:journalctl@ceph.mds.k.smithi134.stdout:Sep 13 18:35:03 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-k[62972]: starting mds.k at 2024-09-13T18:35:03.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:03.662+0000 7f5ebb2b1640 1 -- 172.21.15.114:0/1642421111 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f5eb4197530 con 0x7f5eb4100f40 2024-09-13T18:35:03.694 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:03 smithi204 ceph-mon[34777]: pgmap v392: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-13T18:35:03.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:03 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/3181293365' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:35:03.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:03 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/170187350' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:35:03.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:03 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/2325856669' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-13T18:35:03.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:03 smithi204 ceph-mon[34777]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-13T18:35:03.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:03 smithi204 ceph-mon[34777]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-13T18:35:03.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:03 smithi204 ceph-mon[34777]: osdmap e107: 12 total, 12 up, 12 in 2024-09-13T18:35:03.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:03 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/2325856669' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-13T18:35:03.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:03 smithi204 ceph-mon[34777]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-13T18:35:03.695 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 13 18:35:03 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-c[64760]: ignoring --setuser ceph since I am not root 2024-09-13T18:35:03.696 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 13 18:35:03 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-c[64760]: ignoring --setgroup ceph since I am not root 2024-09-13T18:35:03.696 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 13 18:35:03 smithi204 ceph-mds[64764]: -- 172.21.15.204:0/2032062422 <== mon.0 v2:172.21.15.114:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563620808d00 con 0x5636207dd000 2024-09-13T18:35:03.696 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 13 18:35:03 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-c[64760]: starting mds.c at 2024-09-13T18:35:03.696 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 13 18:35:03 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-i[65062]: ignoring --setuser ceph since I am not root 2024-09-13T18:35:03.696 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 13 18:35:03 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-i[65062]: ignoring --setgroup ceph since I am not root 2024-09-13T18:35:03.696 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 13 18:35:03 smithi204 ceph-mds[65068]: -- 172.21.15.204:0/1431752462 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555e32536d00 con 0x555e3250b000 2024-09-13T18:35:03.696 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 13 18:35:03 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-i[65062]: starting mds.i at 2024-09-13T18:35:03.697 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 13 18:35:03 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-l[64136]: ignoring --setuser ceph since I am not root 2024-09-13T18:35:03.697 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 13 18:35:03 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-l[64136]: ignoring --setgroup ceph since I am not root 2024-09-13T18:35:03.697 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 13 18:35:03 smithi204 ceph-mds[64140]: -- 172.21.15.204:0/3184558573 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5646a6f28d00 con 0x5646a6efd400 2024-09-13T18:35:03.697 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 13 18:35:03 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-l[64136]: starting mds.l at 2024-09-13T18:35:03.697 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 13 18:35:03 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-f[64457]: ignoring --setuser ceph since I am not root 2024-09-13T18:35:03.697 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 13 18:35:03 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-f[64457]: ignoring --setgroup ceph since I am not root 2024-09-13T18:35:03.697 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 13 18:35:03 smithi204 ceph-mds[64461]: -- 172.21.15.204:0/1359843929 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a0cf1fad00 con 0x55a0cf1cf400 2024-09-13T18:35:03.697 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 13 18:35:03 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-f[64457]: starting mds.f at 2024-09-13T18:35:03.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:03 smithi114 ceph-mon[30897]: pgmap v392: 129 pgs: 129 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-13T18:35:03.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:03 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/3181293365' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-13T18:35:03.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:03 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/170187350' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-13T18:35:03.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:03 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2325856669' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-13T18:35:03.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:03 smithi114 ceph-mon[30897]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-13T18:35:03.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:03 smithi114 ceph-mon[30897]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-13T18:35:03.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:03 smithi114 ceph-mon[30897]: osdmap e107: 12 total, 12 up, 12 in 2024-09-13T18:35:03.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:03 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/2325856669' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-13T18:35:03.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:03 smithi114 ceph-mon[30897]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-13T18:35:04.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:04.333+0000 7f5eaa7fc640 1 -- 172.21.15.114:0/1642421111 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v41) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5eb0099c30 con 0x7f5eb4100f40 2024-09-13T18:35:04.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:04.337+0000 7f5e97fff640 1 -- 172.21.15.114:0/1642421111 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f5ea00a1280 msgr2=0x7f5ea00a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:04.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:04.337+0000 7f5e97fff640 1 --2- 172.21.15.114:0/1642421111 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f5ea00a1280 0x7f5ea00a3740 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5ea402aa70 tx=0x7f5ea4005a20 comp rx=0 tx=0).stop 2024-09-13T18:35:04.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:04.337+0000 7f5e97fff640 1 -- 172.21.15.114:0/1642421111 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5eb4100f40 msgr2=0x7f5eb41961e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:04.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:04.337+0000 7f5e97fff640 1 --2- 172.21.15.114:0/1642421111 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5eb4100f40 0x7f5eb41961e0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f5eb000e9f0 tx=0x7f5eb000edf0 comp rx=0 tx=0).stop 2024-09-13T18:35:04.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:04.337+0000 7f5e97fff640 1 -- 172.21.15.114:0/1642421111 shutdown_connections 2024-09-13T18:35:04.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:04.337+0000 7f5e97fff640 1 --2- 172.21.15.114:0/1642421111 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f5ea00a1280 0x7f5ea00a3740 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:04.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:04.337+0000 7f5e97fff640 1 --2- 172.21.15.114:0/1642421111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5eb410a310 0x7f5eb419ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:04.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:04.337+0000 7f5e97fff640 1 --2- 172.21.15.114:0/1642421111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5eb4101860 0x7f5eb4196720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:04.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:04.337+0000 7f5e97fff640 1 --2- 172.21.15.114:0/1642421111 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5eb4100f40 0x7f5eb41961e0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:04.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:04.337+0000 7f5e97fff640 1 -- 172.21.15.114:0/1642421111 >> 172.21.15.114:0/1642421111 conn(0x7f5eb4075c60 msgr2=0x7f5eb40fb680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:04.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:04.337+0000 7f5e97fff640 1 -- 172.21.15.114:0/1642421111 shutdown_connections 2024-09-13T18:35:04.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:04.337+0000 7f5e97fff640 1 -- 172.21.15.114:0/1642421111 wait complete. 2024-09-13T18:35:04.359 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-13T18:35:04.564 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:04 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/1642421111' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-13T18:35:04.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:04 smithi114 ceph-mon[30897]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-13T18:35:04.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:04 smithi114 ceph-mon[30897]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-13T18:35:04.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:04 smithi114 ceph-mon[30897]: Cluster is now healthy 2024-09-13T18:35:04.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:04 smithi114 ceph-mon[30897]: osdmap e108: 12 total, 12 up, 12 in 2024-09-13T18:35:04.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:04 smithi114 ceph-mon[30897]: mds.? [v2:172.21.15.134:6832/3857752284,v1:172.21.15.134:6833/3857752284] up:boot 2024-09-13T18:35:04.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:04 smithi114 ceph-mon[30897]: mds.? [v2:172.21.15.204:6832/1683779721,v1:172.21.15.204:6834/1683779721] up:boot 2024-09-13T18:35:04.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:04 smithi114 ceph-mon[30897]: mds.? [v2:172.21.15.204:6833/2188204749,v1:172.21.15.204:6835/2188204749] up:boot 2024-09-13T18:35:04.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:04 smithi114 ceph-mon[30897]: mds.? [v2:172.21.15.134:6834/4136848612,v1:172.21.15.134:6835/4136848612] up:boot 2024-09-13T18:35:04.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:04 smithi114 ceph-mon[30897]: mds.? [v2:172.21.15.204:6837/1145374109,v1:172.21.15.204:6839/1145374109] up:boot 2024-09-13T18:35:04.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:04 smithi114 ceph-mon[30897]: mds.? [v2:172.21.15.204:6836/2805809772,v1:172.21.15.204:6838/2805809772] up:boot 2024-09-13T18:35:04.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:04 smithi114 ceph-mon[30897]: mds.? [v2:172.21.15.134:6836/2448428108,v1:172.21.15.134:6837/2448428108] up:boot 2024-09-13T18:35:04.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:04 smithi114 ceph-mon[30897]: mds.? [v2:172.21.15.134:6838/4080994235,v1:172.21.15.134:6839/4080994235] up:boot 2024-09-13T18:35:04.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:04 smithi114 ceph-mon[30897]: mds.? [v2:172.21.15.114:6838/1705134098,v1:172.21.15.114:6840/1705134098] up:boot 2024-09-13T18:35:04.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:04 smithi114 ceph-mon[30897]: mds.? [v2:172.21.15.114:6839/1437154786,v1:172.21.15.114:6841/1437154786] up:boot 2024-09-13T18:35:04.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:04 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/1642421111' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-13T18:35:04.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:04 smithi114 ceph-mon[30897]: fsmap 12 up:standby 2024-09-13T18:35:04.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:04 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-13T18:35:04.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:04 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-13T18:35:04.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:04 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-13T18:35:04.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:04 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-13T18:35:04.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:04 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-13T18:35:04.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:04 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-13T18:35:04.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:04 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-13T18:35:04.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:04 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-13T18:35:04.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:04 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-13T18:35:04.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:04 smithi114 ceph-mon[30897]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-13T18:35:04.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:04.577+0000 7f6852a4f640 1 -- 172.21.15.114:0/3449075591 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68440a6600 msgr2=0x7f68440a89f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:04.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:04.577+0000 7f6852a4f640 1 --2- 172.21.15.114:0/3449075591 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68440a6600 0x7f68440a89f0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f684c0660a0 tx=0x7f684c094910 comp rx=0 tx=0).stop 2024-09-13T18:35:04.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:04.580+0000 7f6852a4f640 1 -- 172.21.15.114:0/3449075591 shutdown_connections 2024-09-13T18:35:04.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:04.580+0000 7f6852a4f640 1 --2- 172.21.15.114:0/3449075591 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68440a6600 0x7f68440a89f0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:04.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:04.580+0000 7f6852a4f640 1 --2- 172.21.15.114:0/3449075591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6844006de0 0x7f6844007240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:04.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:04.580+0000 7f6852a4f640 1 --2- 172.21.15.114:0/3449075591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6844008340 0x7f68440068a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:04.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:04.580+0000 7f6852a4f640 1 -- 172.21.15.114:0/3449075591 >> 172.21.15.114:0/3449075591 conn(0x7f684400c690 msgr2=0x7f684400caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:04.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:04.583+0000 7f6852a4f640 1 -- 172.21.15.114:0/3449075591 shutdown_connections 2024-09-13T18:35:04.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:04.586+0000 7f6852a4f640 1 -- 172.21.15.114:0/3449075591 wait complete. 2024-09-13T18:35:04.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:04.587+0000 7f6852a4f640 1 Processor -- start 2024-09-13T18:35:04.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:04.589+0000 7f6852a4f640 1 -- start start 2024-09-13T18:35:04.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:04.589+0000 7f6852a4f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6844006de0 0x7f68440a2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:04.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:04.589+0000 7f6852a4f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6844008340 0x7f684409bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:04.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:04.589+0000 7f6852a4f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68440a6600 0x7f684409c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:04.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:04.590+0000 7f6852a4f640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684409c6b0 con 0x7f68440a6600 2024-09-13T18:35:04.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:04.590+0000 7f6852a4f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684409c820 con 0x7f6844008340 2024-09-13T18:35:04.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:04.590+0000 7f6852a4f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684409c9e0 con 0x7f6844006de0 2024-09-13T18:35:04.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:04.590+0000 7f6851a4d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6844006de0 0x7f68440a2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:04.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:04.590+0000 7f6851a4d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6844006de0 0x7f68440a2bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.114:56766/0 (socket says 172.21.15.114:56766) 2024-09-13T18:35:04.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:04.590+0000 7f6851a4d640 1 -- 172.21.15.114:0/1824519331 learned_addr learned my addr 172.21.15.114:0/1824519331 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:35:04.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:04.590+0000 7f685124c640 1 --2- 172.21.15.114:0/1824519331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6844008340 0x7f684409bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:04.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:04.590+0000 7f685224e640 1 --2- 172.21.15.114:0/1824519331 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68440a6600 0x7f684409c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:04.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:04.591+0000 7f685124c640 1 -- 172.21.15.114:0/1824519331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6844006de0 msgr2=0x7f68440a2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:04.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:04.591+0000 7f685124c640 1 --2- 172.21.15.114:0/1824519331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6844006de0 0x7f68440a2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:04.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:04.591+0000 7f685124c640 1 -- 172.21.15.114:0/1824519331 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68440a6600 msgr2=0x7f684409c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:04.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:04.591+0000 7f685124c640 1 --2- 172.21.15.114:0/1824519331 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68440a6600 0x7f684409c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:04.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:04.591+0000 7f685124c640 1 -- 172.21.15.114:0/1824519331 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f684c04ea80 con 0x7f6844008340 2024-09-13T18:35:04.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:04.592+0000 7f685124c640 1 --2- 172.21.15.114:0/1824519331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6844008340 0x7f684409bc30 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f684000c970 tx=0x7f684000ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:04.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:04.592+0000 7f683effd640 1 -- 172.21.15.114:0/1824519331 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6840007bb0 con 0x7f6844008340 2024-09-13T18:35:04.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:04.592+0000 7f683effd640 1 -- 172.21.15.114:0/1824519331 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6840007d10 con 0x7f6844008340 2024-09-13T18:35:04.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:04.592+0000 7f683effd640 1 -- 172.21.15.114:0/1824519331 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6840007bb0 con 0x7f6844008340 2024-09-13T18:35:04.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:04.595+0000 7f6852a4f640 1 -- 172.21.15.114:0/1824519331 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68440a07a0 con 0x7f6844008340 2024-09-13T18:35:04.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:04.595+0000 7f6852a4f640 1 -- 172.21.15.114:0/1824519331 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68440a0c60 con 0x7f6844008340 2024-09-13T18:35:04.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:04.598+0000 7f6852a4f640 1 -- 172.21.15.114:0/1824519331 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6844007be0 con 0x7f6844008340 2024-09-13T18:35:04.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:04.601+0000 7f683effd640 1 -- 172.21.15.114:0/1824519331 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f684000f450 con 0x7f6844008340 2024-09-13T18:35:04.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:04.603+0000 7f683effd640 1 --2- 172.21.15.114:0/1824519331 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f68380a1300 0x7f68380a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:04.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:04.603+0000 7f683effd640 1 -- 172.21.15.114:0/1824519331 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9971+0+0 (secure 0 0 0) 0x7f68400ccb50 con 0x7f6844008340 2024-09-13T18:35:04.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:04.608+0000 7f6851a4d640 1 --2- 172.21.15.114:0/1824519331 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f68380a1300 0x7f68380a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:04.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:04.611+0000 7f6851a4d640 1 --2- 172.21.15.114:0/1824519331 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f68380a1300 0x7f68380a37c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6848005ed0 tx=0x7f6848005e60 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:04.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:04.613+0000 7f683effd640 1 -- 172.21.15.114:0/1824519331 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f6840095990 con 0x7f6844008340 2024-09-13T18:35:04.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:04 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/1642421111' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-13T18:35:04.870 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:04 smithi134 ceph-mon[34283]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-13T18:35:04.870 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:04 smithi134 ceph-mon[34283]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-13T18:35:04.870 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:04 smithi134 ceph-mon[34283]: Cluster is now healthy 2024-09-13T18:35:04.870 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:04 smithi134 ceph-mon[34283]: osdmap e108: 12 total, 12 up, 12 in 2024-09-13T18:35:04.870 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:04 smithi134 ceph-mon[34283]: mds.? [v2:172.21.15.134:6832/3857752284,v1:172.21.15.134:6833/3857752284] up:boot 2024-09-13T18:35:04.870 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:04 smithi134 ceph-mon[34283]: mds.? [v2:172.21.15.204:6832/1683779721,v1:172.21.15.204:6834/1683779721] up:boot 2024-09-13T18:35:04.870 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:04 smithi134 ceph-mon[34283]: mds.? [v2:172.21.15.204:6833/2188204749,v1:172.21.15.204:6835/2188204749] up:boot 2024-09-13T18:35:04.870 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:04 smithi134 ceph-mon[34283]: mds.? [v2:172.21.15.134:6834/4136848612,v1:172.21.15.134:6835/4136848612] up:boot 2024-09-13T18:35:04.870 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:04 smithi134 ceph-mon[34283]: mds.? [v2:172.21.15.204:6837/1145374109,v1:172.21.15.204:6839/1145374109] up:boot 2024-09-13T18:35:04.870 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:04 smithi134 ceph-mon[34283]: mds.? [v2:172.21.15.204:6836/2805809772,v1:172.21.15.204:6838/2805809772] up:boot 2024-09-13T18:35:04.870 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:04 smithi134 ceph-mon[34283]: mds.? [v2:172.21.15.134:6836/2448428108,v1:172.21.15.134:6837/2448428108] up:boot 2024-09-13T18:35:04.870 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:04 smithi134 ceph-mon[34283]: mds.? [v2:172.21.15.134:6838/4080994235,v1:172.21.15.134:6839/4080994235] up:boot 2024-09-13T18:35:04.871 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:04 smithi134 ceph-mon[34283]: mds.? [v2:172.21.15.114:6838/1705134098,v1:172.21.15.114:6840/1705134098] up:boot 2024-09-13T18:35:04.871 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:04 smithi134 ceph-mon[34283]: mds.? [v2:172.21.15.114:6839/1437154786,v1:172.21.15.114:6841/1437154786] up:boot 2024-09-13T18:35:04.871 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:04 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/1642421111' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-13T18:35:04.871 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:04 smithi134 ceph-mon[34283]: fsmap 12 up:standby 2024-09-13T18:35:04.871 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:04 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-13T18:35:04.871 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:04 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-13T18:35:04.871 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:04 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-13T18:35:04.871 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:04 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-13T18:35:04.871 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:04 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-13T18:35:04.871 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:04 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-13T18:35:04.871 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:04 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-13T18:35:04.872 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:04 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-13T18:35:04.872 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:04 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-13T18:35:04.872 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:04 smithi134 ceph-mon[34283]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-13T18:35:04.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:04 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/1642421111' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-13T18:35:04.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:04 smithi204 ceph-mon[34777]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-13T18:35:04.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:04 smithi204 ceph-mon[34777]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-13T18:35:04.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:04 smithi204 ceph-mon[34777]: Cluster is now healthy 2024-09-13T18:35:04.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:04 smithi204 ceph-mon[34777]: osdmap e108: 12 total, 12 up, 12 in 2024-09-13T18:35:04.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:04 smithi204 ceph-mon[34777]: mds.? [v2:172.21.15.134:6832/3857752284,v1:172.21.15.134:6833/3857752284] up:boot 2024-09-13T18:35:04.946 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:04 smithi204 ceph-mon[34777]: mds.? [v2:172.21.15.204:6832/1683779721,v1:172.21.15.204:6834/1683779721] up:boot 2024-09-13T18:35:04.946 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:04 smithi204 ceph-mon[34777]: mds.? [v2:172.21.15.204:6833/2188204749,v1:172.21.15.204:6835/2188204749] up:boot 2024-09-13T18:35:04.946 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:04 smithi204 ceph-mon[34777]: mds.? [v2:172.21.15.134:6834/4136848612,v1:172.21.15.134:6835/4136848612] up:boot 2024-09-13T18:35:04.946 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:04 smithi204 ceph-mon[34777]: mds.? [v2:172.21.15.204:6837/1145374109,v1:172.21.15.204:6839/1145374109] up:boot 2024-09-13T18:35:04.946 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:04 smithi204 ceph-mon[34777]: mds.? [v2:172.21.15.204:6836/2805809772,v1:172.21.15.204:6838/2805809772] up:boot 2024-09-13T18:35:04.946 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:04 smithi204 ceph-mon[34777]: mds.? [v2:172.21.15.134:6836/2448428108,v1:172.21.15.134:6837/2448428108] up:boot 2024-09-13T18:35:04.946 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:04 smithi204 ceph-mon[34777]: mds.? [v2:172.21.15.134:6838/4080994235,v1:172.21.15.134:6839/4080994235] up:boot 2024-09-13T18:35:04.946 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:04 smithi204 ceph-mon[34777]: mds.? [v2:172.21.15.114:6838/1705134098,v1:172.21.15.114:6840/1705134098] up:boot 2024-09-13T18:35:04.946 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:04 smithi204 ceph-mon[34777]: mds.? [v2:172.21.15.114:6839/1437154786,v1:172.21.15.114:6841/1437154786] up:boot 2024-09-13T18:35:04.946 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:04 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/1642421111' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-13T18:35:04.946 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:04 smithi204 ceph-mon[34777]: fsmap 12 up:standby 2024-09-13T18:35:04.946 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:04 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-13T18:35:04.946 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:04 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-13T18:35:04.947 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:04 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-13T18:35:04.947 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:04 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-13T18:35:04.947 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:04 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-13T18:35:04.947 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:04 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-13T18:35:04.947 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:04 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-13T18:35:04.947 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:04 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-13T18:35:04.947 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:04 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-13T18:35:04.947 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:04 smithi204 ceph-mon[34777]: from='mgr.15555 172.21.15.114:0/832974581' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-13T18:35:04.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:04.947+0000 7f6852a4f640 1 -- 172.21.15.114:0/1824519331 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6844007240 con 0x7f6844008340 2024-09-13T18:35:05.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:05.564+0000 7f683effd640 1 -- 172.21.15.114:0/1824519331 <== mon.2 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v109) v1 ==== 181+0+0 (secure 0 0 0) 0x7f684001d070 con 0x7f6844008340 2024-09-13T18:35:05.567 INFO:teuthology.orchestra.run.smithi114.stderr:pool 'cephfs_metadata' removed 2024-09-13T18:35:05.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:05.568+0000 7f6852a4f640 1 -- 172.21.15.114:0/1824519331 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f68380a1300 msgr2=0x7f68380a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:05.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:05.568+0000 7f6852a4f640 1 --2- 172.21.15.114:0/1824519331 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f68380a1300 0x7f68380a37c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6848005ed0 tx=0x7f6848005e60 comp rx=0 tx=0).stop 2024-09-13T18:35:05.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:05.568+0000 7f6852a4f640 1 -- 172.21.15.114:0/1824519331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6844008340 msgr2=0x7f684409bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:05.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:05.568+0000 7f6852a4f640 1 --2- 172.21.15.114:0/1824519331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6844008340 0x7f684409bc30 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f684000c970 tx=0x7f684000ce30 comp rx=0 tx=0).stop 2024-09-13T18:35:05.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:05.568+0000 7f6852a4f640 1 -- 172.21.15.114:0/1824519331 shutdown_connections 2024-09-13T18:35:05.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:05.568+0000 7f6852a4f640 1 --2- 172.21.15.114:0/1824519331 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f68380a1300 0x7f68380a37c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:05.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:05.568+0000 7f6852a4f640 1 --2- 172.21.15.114:0/1824519331 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68440a6600 0x7f684409c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:05.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:05.568+0000 7f6852a4f640 1 --2- 172.21.15.114:0/1824519331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6844008340 0x7f684409bc30 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:05.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:05.568+0000 7f6852a4f640 1 --2- 172.21.15.114:0/1824519331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6844006de0 0x7f68440a2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:05.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:05.568+0000 7f6852a4f640 1 -- 172.21.15.114:0/1824519331 >> 172.21.15.114:0/1824519331 conn(0x7f684400c690 msgr2=0x7f68440929a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:05.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:05.572+0000 7f6852a4f640 1 -- 172.21.15.114:0/1824519331 shutdown_connections 2024-09-13T18:35:05.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:05.572+0000 7f6852a4f640 1 -- 172.21.15.114:0/1824519331 wait complete. 2024-09-13T18:35:05.593 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-13T18:35:05.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:05.791+0000 7fc40bf84640 1 -- 172.21.15.114:0/3017001632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4040ff810 msgr2=0x7fc404103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:05.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:05.791+0000 7fc40bf84640 1 --2- 172.21.15.114:0/3017001632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4040ff810 0x7fc404103850 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4009860 tx=0x7fc3f402a360 comp rx=0 tx=0).stop 2024-09-13T18:35:05.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:05.791+0000 7fc40bf84640 1 -- 172.21.15.114:0/3017001632 shutdown_connections 2024-09-13T18:35:05.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:05.791+0000 7fc40bf84640 1 --2- 172.21.15.114:0/3017001632 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc404103fb0 0x7fc40410bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:05.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:05.791+0000 7fc40bf84640 1 --2- 172.21.15.114:0/3017001632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4040ff810 0x7fc404103850 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:05.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:05.791+0000 7fc40bf84640 1 --2- 172.21.15.114:0/3017001632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4040fee60 0x7fc4040ff240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:05.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:05.791+0000 7fc40bf84640 1 -- 172.21.15.114:0/3017001632 >> 172.21.15.114:0/3017001632 conn(0x7fc404075c60 msgr2=0x7fc4040fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:05.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:05.800+0000 7fc40bf84640 1 -- 172.21.15.114:0/3017001632 shutdown_connections 2024-09-13T18:35:05.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:05.801+0000 7fc40bf84640 1 -- 172.21.15.114:0/3017001632 wait complete. 2024-09-13T18:35:05.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:05.801+0000 7fc40bf84640 1 Processor -- start 2024-09-13T18:35:05.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:05.801+0000 7fc40bf84640 1 -- start start 2024-09-13T18:35:05.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:05.802+0000 7fc40bf84640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4040fee60 0x7fc404071c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:05.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:05.802+0000 7fc40bf84640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc404103fb0 0x7fc40406ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:05.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:05.802+0000 7fc40bf84640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc40406b280 0x7fc40406b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:05.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:05.802+0000 7fc40bf84640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4040724c0 con 0x7fc404103fb0 2024-09-13T18:35:05.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:05.802+0000 7fc40bf84640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc40406f800 con 0x7fc4040fee60 2024-09-13T18:35:05.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:05.802+0000 7fc40bf84640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc40406f9c0 con 0x7fc40406b280 2024-09-13T18:35:05.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:05.802+0000 7fc40a521640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc40406b280 0x7fc40406b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:05.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:05.803+0000 7fc40951f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc404103fb0 0x7fc40406ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:05.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:05.803+0000 7fc40951f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc404103fb0 0x7fc40406ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35682/0 (socket says 172.21.15.114:35682) 2024-09-13T18:35:05.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:05.803+0000 7fc40951f640 1 -- 172.21.15.114:0/259020716 learned_addr learned my addr 172.21.15.114:0/259020716 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:35:05.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:05.803+0000 7fc40951f640 1 -- 172.21.15.114:0/259020716 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc40406b280 msgr2=0x7fc40406b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:05.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:05.803+0000 7fc40951f640 1 --2- 172.21.15.114:0/259020716 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc40406b280 0x7fc40406b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:05.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:05.803+0000 7fc40951f640 1 -- 172.21.15.114:0/259020716 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4040fee60 msgr2=0x7fc404071c70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:35:05.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:05.803+0000 7fc40951f640 1 --2- 172.21.15.114:0/259020716 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4040fee60 0x7fc404071c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:05.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:05.803+0000 7fc40951f640 1 -- 172.21.15.114:0/259020716 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f4009490 con 0x7fc404103fb0 2024-09-13T18:35:05.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:05.804+0000 7fc40951f640 1 --2- 172.21.15.114:0/259020716 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc404103fb0 0x7fc40406ad40 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4035040 tx=0x7fc3f4033dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:05.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:05.804+0000 7fc3faffd640 1 -- 172.21.15.114:0/259020716 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f4004510 con 0x7fc404103fb0 2024-09-13T18:35:05.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:05.804+0000 7fc3faffd640 1 -- 172.21.15.114:0/259020716 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc3f40307d0 con 0x7fc404103fb0 2024-09-13T18:35:05.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:05.804+0000 7fc3faffd640 1 -- 172.21.15.114:0/259020716 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f4003d40 con 0x7fc404103fb0 2024-09-13T18:35:05.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:05.804+0000 7fc40bf84640 1 -- 172.21.15.114:0/259020716 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc40406fc40 con 0x7fc404103fb0 2024-09-13T18:35:05.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:05.804+0000 7fc40bf84640 1 -- 172.21.15.114:0/259020716 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4040700a0 con 0x7fc404103fb0 2024-09-13T18:35:05.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:05.809+0000 7fc40bf84640 1 -- 172.21.15.114:0/259020716 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3e8005250 con 0x7fc404103fb0 2024-09-13T18:35:05.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:05.812+0000 7fc3faffd640 1 -- 172.21.15.114:0/259020716 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7fc3f4039040 con 0x7fc404103fb0 2024-09-13T18:35:05.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:05.813+0000 7fc3faffd640 1 --2- 172.21.15.114:0/259020716 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc3f00a5830 0x7fc3f00a7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:05.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:05.813+0000 7fc3faffd640 1 -- 172.21.15.114:0/259020716 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9329+0+0 (secure 0 0 0) 0x7fc3f40ed370 con 0x7fc404103fb0 2024-09-13T18:35:05.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:05.815+0000 7fc409d20640 1 --2- 172.21.15.114:0/259020716 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc3f00a5830 0x7fc3f00a7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:05.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:05.816+0000 7fc409d20640 1 --2- 172.21.15.114:0/259020716 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc3f00a5830 0x7fc3f00a7cf0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc005c40 tx=0x7fc3fc005bd0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:05.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:05.822+0000 7fc3faffd640 1 -- 172.21.15.114:0/259020716 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fc3f40b6540 con 0x7fc404103fb0 2024-09-13T18:35:05.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:05 smithi134 ceph-mon[34283]: pgmap v395: 129 pgs: 4 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 117 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 35 KiB/s wr, 33 op/s 2024-09-13T18:35:05.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:05 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/1824519331' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-13T18:35:05.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:05 smithi134 ceph-mon[34283]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-13T18:35:05.873 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:05 smithi114 ceph-mon[30897]: pgmap v395: 129 pgs: 4 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 117 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 35 KiB/s wr, 33 op/s 2024-09-13T18:35:05.873 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:05 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/1824519331' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-13T18:35:05.873 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:05 smithi114 ceph-mon[30897]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-13T18:35:05.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:05 smithi204 ceph-mon[34777]: pgmap v395: 129 pgs: 4 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 117 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 35 KiB/s wr, 33 op/s 2024-09-13T18:35:05.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:05 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/1824519331' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-13T18:35:05.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:05 smithi204 ceph-mon[34777]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-13T18:35:06.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:06.158+0000 7fc40bf84640 1 -- 172.21.15.114:0/259020716 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc3e80050c0 con 0x7fc404103fb0 2024-09-13T18:35:06.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:06.584+0000 7fc3faffd640 1 -- 172.21.15.114:0/259020716 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v110) v1 ==== 169+0+0 (secure 0 0 0) 0x7fc3f403f0d0 con 0x7fc404103fb0 2024-09-13T18:35:06.587 INFO:teuthology.orchestra.run.smithi114.stderr:pool 'cephfs_data' removed 2024-09-13T18:35:06.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:06.588+0000 7fc40bf84640 1 -- 172.21.15.114:0/259020716 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc3f00a5830 msgr2=0x7fc3f00a7cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:06.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:06.588+0000 7fc40bf84640 1 --2- 172.21.15.114:0/259020716 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc3f00a5830 0x7fc3f00a7cf0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc005c40 tx=0x7fc3fc005bd0 comp rx=0 tx=0).stop 2024-09-13T18:35:06.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:06.588+0000 7fc40bf84640 1 -- 172.21.15.114:0/259020716 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc404103fb0 msgr2=0x7fc40406ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:06.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:06.588+0000 7fc40bf84640 1 --2- 172.21.15.114:0/259020716 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc404103fb0 0x7fc40406ad40 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4035040 tx=0x7fc3f4033dc0 comp rx=0 tx=0).stop 2024-09-13T18:35:06.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:06.588+0000 7fc40bf84640 1 -- 172.21.15.114:0/259020716 shutdown_connections 2024-09-13T18:35:06.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:06.588+0000 7fc40bf84640 1 --2- 172.21.15.114:0/259020716 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7fc3f00a5830 0x7fc3f00a7cf0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:06.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:06.588+0000 7fc40bf84640 1 --2- 172.21.15.114:0/259020716 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc40406b280 0x7fc40406b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:06.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:06.588+0000 7fc40bf84640 1 --2- 172.21.15.114:0/259020716 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc404103fb0 0x7fc40406ad40 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:06.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:06.588+0000 7fc40bf84640 1 --2- 172.21.15.114:0/259020716 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4040fee60 0x7fc404071c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:06.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:06.588+0000 7fc40bf84640 1 -- 172.21.15.114:0/259020716 >> 172.21.15.114:0/259020716 conn(0x7fc404075c60 msgr2=0x7fc404102400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:06.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:06.588+0000 7fc40bf84640 1 -- 172.21.15.114:0/259020716 shutdown_connections 2024-09-13T18:35:06.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:06.588+0000 7fc40bf84640 1 -- 172.21.15.114:0/259020716 wait complete. 2024-09-13T18:35:06.612 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-13T18:35:06.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:06.816+0000 7f0530e44640 1 -- 172.21.15.114:0/3336793569 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c040770 con 0x7f052c10c3e0 2024-09-13T18:35:06.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:06.817+0000 7f05340aa640 1 -- 172.21.15.114:0/3336793569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f052c10c3e0 msgr2=0x7f052c10c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:06.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:06.817+0000 7f05340aa640 1 --2- 172.21.15.114:0/3336793569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f052c10c3e0 0x7f052c10c7c0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f051c01a940 tx=0x7f051c03b500 comp rx=0 tx=0).stop 2024-09-13T18:35:06.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:06.817+0000 7f05340aa640 1 -- 172.21.15.114:0/3336793569 shutdown_connections 2024-09-13T18:35:06.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:06.817+0000 7f05340aa640 1 --2- 172.21.15.114:0/3336793569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f052c106ca0 0x7f052c113180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:06.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:06.817+0000 7f05340aa640 1 --2- 172.21.15.114:0/3336793569 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f052c106320 0x7f052c106760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:06.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:06.817+0000 7f05340aa640 1 --2- 172.21.15.114:0/3336793569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f052c10c3e0 0x7f052c10c7c0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:06.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:06.817+0000 7f05340aa640 1 -- 172.21.15.114:0/3336793569 >> 172.21.15.114:0/3336793569 conn(0x7f052c075c60 msgr2=0x7f052c078080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:06.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:06.821+0000 7f05340aa640 1 -- 172.21.15.114:0/3336793569 shutdown_connections 2024-09-13T18:35:06.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:06.822+0000 7f05340aa640 1 -- 172.21.15.114:0/3336793569 wait complete. 2024-09-13T18:35:06.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:06.822+0000 7f05340aa640 1 Processor -- start 2024-09-13T18:35:06.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:06.822+0000 7f05340aa640 1 -- start start 2024-09-13T18:35:06.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:06.823+0000 7f05340aa640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f052c106320 0x7f052c112930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:06.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:06.823+0000 7f05340aa640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f052c106ca0 0x7f052c112e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:06.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:06.823+0000 7f05340aa640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f052c10c3e0 0x7f052c071f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:06.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:06.823+0000 7f05340aa640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052c10ea50 con 0x7f052c106320 2024-09-13T18:35:06.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:06.823+0000 7f05340aa640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052c10eb90 con 0x7f052c106ca0 2024-09-13T18:35:06.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:06.823+0000 7f05340aa640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052c10ed50 con 0x7f052c10c3e0 2024-09-13T18:35:06.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:06.823+0000 7f0531645640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f052c106ca0 0x7f052c112e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:06.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:06.823+0000 7f0531645640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f052c106ca0 0x7f052c112e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.114:55322/0 (socket says 172.21.15.114:55322) 2024-09-13T18:35:06.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:06.823+0000 7f0531645640 1 -- 172.21.15.114:0/529368930 learned_addr learned my addr 172.21.15.114:0/529368930 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-13T18:35:06.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:06.824+0000 7f0531645640 1 -- 172.21.15.114:0/529368930 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f052c10c3e0 msgr2=0x7f052c071f20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:35:06.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:06.824+0000 7f0531645640 1 --2- 172.21.15.114:0/529368930 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f052c10c3e0 0x7f052c071f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:06.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:06.824+0000 7f0531645640 1 -- 172.21.15.114:0/529368930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f052c106320 msgr2=0x7f052c112930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:06.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:06.824+0000 7f0531645640 1 --2- 172.21.15.114:0/529368930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f052c106320 0x7f052c112930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:06.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:06.824+0000 7f0531645640 1 -- 172.21.15.114:0/529368930 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f051c01a570 con 0x7f052c106ca0 2024-09-13T18:35:06.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:06.828+0000 7f0531645640 1 --2- 172.21.15.114:0/529368930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f052c106ca0 0x7f052c112e70 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f052800d950 tx=0x7f052800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:06.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:06.835+0000 7f0522ffd640 1 -- 172.21.15.114:0/529368930 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0528009c30 con 0x7f052c106ca0 2024-09-13T18:35:06.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:06.835+0000 7f0522ffd640 1 -- 172.21.15.114:0/529368930 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0528009d90 con 0x7f052c106ca0 2024-09-13T18:35:06.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:06.835+0000 7f0522ffd640 1 -- 172.21.15.114:0/529368930 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052800f620 con 0x7f052c106ca0 2024-09-13T18:35:06.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:06.837+0000 7f05340aa640 1 -- 172.21.15.114:0/529368930 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f052c10f030 con 0x7f052c106ca0 2024-09-13T18:35:06.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:06.837+0000 7f05340aa640 1 -- 172.21.15.114:0/529368930 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f052c10f4c0 con 0x7f052c106ca0 2024-09-13T18:35:06.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:06.839+0000 7f05340aa640 1 -- 172.21.15.114:0/529368930 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f052c107a10 con 0x7f052c106ca0 2024-09-13T18:35:06.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:06.849+0000 7f0522ffd640 1 -- 172.21.15.114:0/529368930 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) v1 ==== 136005+0+0 (secure 0 0 0) 0x7f05280109f0 con 0x7f052c106ca0 2024-09-13T18:35:06.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:06.850+0000 7f0522ffd640 1 --2- 172.21.15.114:0/529368930 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f05180b23e0 0x7f05180b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:35:06.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:06.850+0000 7f0522ffd640 1 -- 172.21.15.114:0/529368930 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8770+0+0 (secure 0 0 0) 0x7f05280cc7b0 con 0x7f052c106ca0 2024-09-13T18:35:06.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:06.854+0000 7f0531e46640 1 --2- 172.21.15.114:0/529368930 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f05180b23e0 0x7f05180b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:35:06.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:06.855+0000 7f0522ffd640 1 -- 172.21.15.114:0/529368930 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f0528095ba0 con 0x7f052c106ca0 2024-09-13T18:35:06.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:06 smithi134 ceph-mon[34283]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-13T18:35:06.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:06 smithi134 ceph-mon[34283]: osdmap e109: 12 total, 12 up, 12 in 2024-09-13T18:35:06.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:06 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/259020716' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-13T18:35:06.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:06.866+0000 7f0531e46640 1 --2- 172.21.15.114:0/529368930 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f05180b23e0 0x7f05180b48a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f051c03ba20 tx=0x7f051c000bc0 comp rx=0 tx=0).ready entity=mgr.15555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:35:06.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:06 smithi204 ceph-mon[34777]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-13T18:35:06.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:06 smithi204 ceph-mon[34777]: osdmap e109: 12 total, 12 up, 12 in 2024-09-13T18:35:06.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:06 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/259020716' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-13T18:35:06.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:06 smithi114 ceph-mon[30897]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-13T18:35:06.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:06 smithi114 ceph-mon[30897]: osdmap e109: 12 total, 12 up, 12 in 2024-09-13T18:35:06.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:06 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/259020716' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-13T18:35:07.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:07.183+0000 7f05340aa640 1 -- 172.21.15.114:0/529368930 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f052c106760 con 0x7f052c106ca0 2024-09-13T18:35:07.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:07.597+0000 7f0522ffd640 1 -- 172.21.15.114:0/529368930 <== mon.2 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v111) v1 ==== 178+0+0 (secure 0 0 0) 0x7f0528017020 con 0x7f052c106ca0 2024-09-13T18:35:07.599 INFO:teuthology.orchestra.run.smithi114.stderr:pool 'cephfs_data_ec' removed 2024-09-13T18:35:07.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:07.600+0000 7f05340aa640 1 -- 172.21.15.114:0/529368930 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f05180b23e0 msgr2=0x7f05180b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:07.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:07.600+0000 7f05340aa640 1 --2- 172.21.15.114:0/529368930 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f05180b23e0 0x7f05180b48a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f051c03ba20 tx=0x7f051c000bc0 comp rx=0 tx=0).stop 2024-09-13T18:35:07.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:07.600+0000 7f05340aa640 1 -- 172.21.15.114:0/529368930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f052c106ca0 msgr2=0x7f052c112e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:35:07.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:07.600+0000 7f05340aa640 1 --2- 172.21.15.114:0/529368930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f052c106ca0 0x7f052c112e70 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f052800d950 tx=0x7f052800de10 comp rx=0 tx=0).stop 2024-09-13T18:35:07.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:07.600+0000 7f05340aa640 1 -- 172.21.15.114:0/529368930 shutdown_connections 2024-09-13T18:35:07.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:07.600+0000 7f05340aa640 1 --2- 172.21.15.114:0/529368930 >> [v2:172.21.15.114:6800/2440721366,v1:172.21.15.114:6801/2440721366] conn(0x7f05180b23e0 0x7f05180b48a0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:07.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:07.600+0000 7f05340aa640 1 --2- 172.21.15.114:0/529368930 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f052c10c3e0 0x7f052c071f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:07.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:07.600+0000 7f05340aa640 1 --2- 172.21.15.114:0/529368930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f052c106ca0 0x7f052c112e70 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:07.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:07.600+0000 7f05340aa640 1 --2- 172.21.15.114:0/529368930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f052c106320 0x7f052c112930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:35:07.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:07.600+0000 7f05340aa640 1 -- 172.21.15.114:0/529368930 >> 172.21.15.114:0/529368930 conn(0x7f052c075c60 msgr2=0x7f052c0776e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:35:07.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:07.600+0000 7f05340aa640 1 -- 172.21.15.114:0/529368930 shutdown_connections 2024-09-13T18:35:07.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-13T18:35:07.600+0000 7f05340aa640 1 -- 172.21.15.114:0/529368930 wait complete. 2024-09-13T18:35:07.620 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-13T18:35:07.677 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-13T18:35:07.723 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-13T18:35:07.783 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-13T18:35:07.784 DEBUG:teuthology.orchestra.run.smithi114:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-13T18:35:07.816 DEBUG:teuthology.orchestra.run.smithi134:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-13T18:35:07.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:07 smithi134 ceph-mon[34283]: pgmap v397: 97 pgs: 19 active+clean+snaptrim, 52 active+clean+snaptrim_wait, 26 active+clean; 16 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:35:07.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:07 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/259020716' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-13T18:35:07.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:07 smithi134 ceph-mon[34283]: osdmap e110: 12 total, 12 up, 12 in 2024-09-13T18:35:07.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:07 smithi134 ceph-mon[34283]: from='client.? 172.21.15.114:0/529368930' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-13T18:35:07.869 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:07 smithi134 ceph-mon[34283]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-13T18:35:07.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:07 smithi204 ceph-mon[34777]: pgmap v397: 97 pgs: 19 active+clean+snaptrim, 52 active+clean+snaptrim_wait, 26 active+clean; 16 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:35:07.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:07 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/259020716' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-13T18:35:07.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:07 smithi204 ceph-mon[34777]: osdmap e110: 12 total, 12 up, 12 in 2024-09-13T18:35:07.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:07 smithi204 ceph-mon[34777]: from='client.? 172.21.15.114:0/529368930' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-13T18:35:07.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:07 smithi204 ceph-mon[34777]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-13T18:35:07.961 DEBUG:teuthology.orchestra.run.smithi204:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-13T18:35:07.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:07 smithi114 ceph-mon[30897]: pgmap v397: 97 pgs: 19 active+clean+snaptrim, 52 active+clean+snaptrim_wait, 26 active+clean; 16 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-13T18:35:07.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:07 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/259020716' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-13T18:35:07.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:07 smithi114 ceph-mon[30897]: osdmap e110: 12 total, 12 up, 12 in 2024-09-13T18:35:07.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:07 smithi114 ceph-mon[30897]: from='client.? 172.21.15.114:0/529368930' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-13T18:35:07.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:07 smithi114 ceph-mon[30897]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-13T18:35:08.065 INFO:tasks.cephadm:Stopping all daemons... 2024-09-13T18:35:08.065 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-13T18:35:08.066 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mon.a 2024-09-13T18:35:08.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:08 smithi114 systemd[1]: Stopping Ceph mon.a for 97e7edf2-71fb-11ef-bceb-c7b262605968... 2024-09-13T18:35:08.870 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:08 smithi134 ceph-mon[34283]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-13T18:35:08.870 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:08 smithi134 ceph-mon[34283]: osdmap e111: 12 total, 12 up, 12 in 2024-09-13T18:35:08.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:08 smithi204 ceph-mon[34777]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-13T18:35:08.945 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:08 smithi204 ceph-mon[34777]: osdmap e111: 12 total, 12 up, 12 in 2024-09-13T18:35:08.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:08 smithi114 ceph-mon[30897]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-13T18:35:08.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:08 smithi114 ceph-mon[30897]: osdmap e111: 12 total, 12 up, 12 in 2024-09-13T18:35:08.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:08 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mon-a[30893]: 2024-09-13T18:35:08.645+0000 7faee14be640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T18:35:08.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:08 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mon-a[30893]: 2024-09-13T18:35:08.645+0000 7faee14be640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-13T18:35:08.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:08 smithi114 podman[110559]: 2024-09-13 18:35:08.780988449 +0000 UTC m=+0.400121174 container died cc359b54bfa5ebeb9d577333f6cb9bd13d4d122c686bca9cc0880d2710b4e368 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mon-a, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-13T18:35:09.707 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:09 smithi114 podman[110559]: 2024-09-13 18:35:09.361597192 +0000 UTC m=+0.980729915 container cleanup cc359b54bfa5ebeb9d577333f6cb9bd13d4d122c686bca9cc0880d2710b4e368 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-13T18:35:09.707 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:09 smithi114 bash[110559]: ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mon-a 2024-09-13T18:35:09.945 DEBUG:teuthology.orchestra.run.smithi114:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mon.a.service' 2024-09-13T18:35:09.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:09 smithi114 podman[110570]: 2024-09-13 18:35:09.70510643 +0000 UTC m=+0.921648180 container remove cc359b54bfa5ebeb9d577333f6cb9bd13d4d122c686bca9cc0880d2710b4e368 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-13T18:35:09.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:09 smithi114 systemd[1]: ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-13T18:35:09.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:09 smithi114 systemd[1]: Stopped Ceph mon.a for 97e7edf2-71fb-11ef-bceb-c7b262605968. 2024-09-13T18:35:09.971 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 13 18:35:09 smithi114 systemd[1]: ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mon.a.service: Consumed 22.894s CPU time. 2024-09-13T18:35:10.013 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T18:35:10.014 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-13T18:35:10.014 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-13T18:35:10.014 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mon.b 2024-09-13T18:35:10.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:10 smithi134 systemd[1]: Stopping Ceph mon.b for 97e7edf2-71fb-11ef-bceb-c7b262605968... 2024-09-13T18:35:12.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:11 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mon-b[34279]: 2024-09-13T18:35:11.969+0000 7f949287d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T18:35:12.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:11 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mon-b[34279]: 2024-09-13T18:35:11.969+0000 7f949287d640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-13T18:35:12.369 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:12 smithi134 podman[69638]: 2024-09-13 18:35:12.184189704 +0000 UTC m=+1.739908594 container died ba2043c577c27758566f2427edc1c77e6dfc8196941be0adfffb0d4563a22d83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mon-b, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-13T18:35:16.191 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:15 smithi134 podman[69638]: 2024-09-13 18:35:15.887804392 +0000 UTC m=+5.443523274 container cleanup ba2043c577c27758566f2427edc1c77e6dfc8196941be0adfffb0d4563a22d83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mon-b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-13T18:35:16.191 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:15 smithi134 bash[69638]: ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mon-b 2024-09-13T18:35:16.441 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 13 18:35:16 smithi134 podman[69650]: 2024-09-13 18:35:16.189233897 +0000 UTC m=+4.001661152 container remove ba2043c577c27758566f2427edc1c77e6dfc8196941be0adfffb0d4563a22d83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mon-b, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-13T18:35:16.445 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mon.b.service' 2024-09-13T18:35:16.492 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T18:35:16.493 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-13T18:35:16.493 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-13T18:35:16.493 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mon.c 2024-09-13T18:35:16.856 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:16 smithi204 systemd[1]: Stopping Ceph mon.c for 97e7edf2-71fb-11ef-bceb-c7b262605968... 2024-09-13T18:35:17.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:16 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mon-c[34752]: 2024-09-13T18:35:16.852+0000 7fc835ae2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T18:35:17.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:16 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mon-c[34752]: 2024-09-13T18:35:16.852+0000 7fc835ae2640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-13T18:35:17.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:16 smithi204 podman[70905]: 2024-09-13 18:35:16.924503998 +0000 UTC m=+0.201923303 container died ce3d7d936a949f59620f5e0ac63781549c06310e8326c99f7857708a26b51387 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mon-c, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T18:35:17.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:17 smithi204 podman[70905]: 2024-09-13 18:35:17.361253578 +0000 UTC m=+0.638672880 container cleanup ce3d7d936a949f59620f5e0ac63781549c06310e8326c99f7857708a26b51387 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mon-c, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-13T18:35:17.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:17 smithi204 bash[70905]: ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mon-c 2024-09-13T18:35:17.944 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 13 18:35:17 smithi204 podman[70919]: 2024-09-13 18:35:17.655260872 +0000 UTC m=+0.724121798 container remove ce3d7d936a949f59620f5e0ac63781549c06310e8326c99f7857708a26b51387 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mon-c, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-13T18:35:17.964 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mon.c.service' 2024-09-13T18:35:18.040 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T18:35:18.041 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-13T18:35:18.041 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-13T18:35:18.041 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mgr.x 2024-09-13T18:35:18.470 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:35:18 smithi114 systemd[1]: Stopping Ceph mgr.x for 97e7edf2-71fb-11ef-bceb-c7b262605968... 2024-09-13T18:35:20.472 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:35:20 smithi114 podman[110673]: 2024-09-13 18:35:20.15913825 +0000 UTC m=+1.918679326 container died 254a2146bf84fe453b2ded5f75cabb1b97370b7d4fdb9f869001b71d0888a226 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-13T18:35:25.970 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:35:25 smithi114 podman[110673]: 2024-09-13 18:35:25.519597419 +0000 UTC m=+7.279138487 container cleanup 254a2146bf84fe453b2ded5f75cabb1b97370b7d4fdb9f869001b71d0888a226 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-13T18:35:25.970 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 13 18:35:25 smithi114 bash[110673]: ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-x 2024-09-13T18:35:27.797 DEBUG:teuthology.orchestra.run.smithi114:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mgr.x.service' 2024-09-13T18:35:27.834 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T18:35:27.834 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-13T18:35:27.834 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-13T18:35:27.835 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mgr.y 2024-09-13T18:35:28.119 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:35:27 smithi134 systemd[1]: Stopping Ceph mgr.y for 97e7edf2-71fb-11ef-bceb-c7b262605968... 2024-09-13T18:35:29.369 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:35:28 smithi134 podman[69761]: 2024-09-13 18:35:28.941188598 +0000 UTC m=+0.910415700 container died 2a4536a41e9315cedf95afec38051ff4fc8abde24038b769de9cd0222ee4aa4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-13T18:35:30.369 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:35:30 smithi134 podman[69761]: 2024-09-13 18:35:30.110213405 +0000 UTC m=+2.079440569 container cleanup 2a4536a41e9315cedf95afec38051ff4fc8abde24038b769de9cd0222ee4aa4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-13T18:35:30.369 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:35:30 smithi134 bash[69761]: ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y 2024-09-13T18:35:30.630 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:35:30 smithi134 podman[69776]: 2024-09-13 18:35:30.496166691 +0000 UTC m=+1.478387187 container remove 2a4536a41e9315cedf95afec38051ff4fc8abde24038b769de9cd0222ee4aa4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-13T18:35:30.630 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 13 18:35:30 smithi134 systemd[1]: ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-09-13T18:35:30.635 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mgr.y.service' 2024-09-13T18:35:30.672 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T18:35:30.673 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-13T18:35:30.673 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-13T18:35:30.673 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mgr.z 2024-09-13T18:35:31.031 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:35:30 smithi204 systemd[1]: Stopping Ceph mgr.z for 97e7edf2-71fb-11ef-bceb-c7b262605968... 2024-09-13T18:35:31.328 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:35:31 smithi204 podman[71041]: 2024-09-13 18:35:31.029390831 +0000 UTC m=+0.147673098 container died 130ab02f9a47adf7781653f394871faf0546b2c75d91c63fa44287cbcdf8acc1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-13T18:35:31.647 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:35:31 smithi204 podman[71041]: 2024-09-13 18:35:31.392567546 +0000 UTC m=+0.510849857 container cleanup 130ab02f9a47adf7781653f394871faf0546b2c75d91c63fa44287cbcdf8acc1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True) 2024-09-13T18:35:31.647 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:35:31 smithi204 bash[71041]: ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z 2024-09-13T18:35:31.875 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mgr.z.service' 2024-09-13T18:35:31.897 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:35:31 smithi204 podman[71055]: 2024-09-13 18:35:31.64491 +0000 UTC m=+0.600191884 container remove 130ab02f9a47adf7781653f394871faf0546b2c75d91c63fa44287cbcdf8acc1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True) 2024-09-13T18:35:31.900 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:35:31 smithi204 systemd[1]: ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-09-13T18:35:31.900 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:35:31 smithi204 systemd[1]: ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mgr.z.service: Failed with result 'exit-code'. 2024-09-13T18:35:31.900 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:35:31 smithi204 systemd[1]: Stopped Ceph mgr.z for 97e7edf2-71fb-11ef-bceb-c7b262605968. 2024-09-13T18:35:31.900 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 13 18:35:31 smithi204 systemd[1]: ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mgr.z.service: Consumed 11.453s CPU time. 2024-09-13T18:35:31.912 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T18:35:31.912 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-13T18:35:31.913 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-13T18:35:31.913 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@osd.0 2024-09-13T18:35:32.220 INFO:journalctl@ceph.osd.0.smithi114.stdout:Sep 13 18:35:31 smithi114 systemd[1]: Stopping Ceph osd.0 for 97e7edf2-71fb-11ef-bceb-c7b262605968... 2024-09-13T18:35:32.720 INFO:journalctl@ceph.osd.0.smithi114.stdout:Sep 13 18:35:32 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-0[44930]: 2024-09-13T18:35:32.233+0000 7fde2ebfa640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T18:35:32.720 INFO:journalctl@ceph.osd.0.smithi114.stdout:Sep 13 18:35:32 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-0[44930]: 2024-09-13T18:35:32.233+0000 7fde2ebfa640 -1 osd.0 111 *** Got signal Terminated *** 2024-09-13T18:35:32.720 INFO:journalctl@ceph.osd.0.smithi114.stdout:Sep 13 18:35:32 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-0[44930]: 2024-09-13T18:35:32.233+0000 7fde2ebfa640 -1 osd.0 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T18:35:38.126 INFO:journalctl@ceph.osd.0.smithi114.stdout:Sep 13 18:35:37 smithi114 podman[110798]: 2024-09-13 18:35:37.846779525 +0000 UTC m=+5.744001892 container died 9364668bed481db34a85deffc4f16beb21f288e46eb575f56200b34d2cde48dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083) 2024-09-13T18:35:38.470 INFO:journalctl@ceph.osd.0.smithi114.stdout:Sep 13 18:35:38 smithi114 podman[110798]: 2024-09-13 18:35:38.267398685 +0000 UTC m=+6.164620977 container cleanup 9364668bed481db34a85deffc4f16beb21f288e46eb575f56200b34d2cde48dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-13T18:35:38.470 INFO:journalctl@ceph.osd.0.smithi114.stdout:Sep 13 18:35:38 smithi114 bash[110798]: ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-0 2024-09-13T18:35:39.220 INFO:journalctl@ceph.osd.0.smithi114.stdout:Sep 13 18:35:38 smithi114 podman[110816]: 2024-09-13 18:35:38.819202833 +0000 UTC m=+0.963890424 container remove 9364668bed481db34a85deffc4f16beb21f288e46eb575f56200b34d2cde48dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083) 2024-09-13T18:35:39.507 INFO:journalctl@ceph.osd.0.smithi114.stdout:Sep 13 18:35:39 smithi114 podman[110903]: 2024-09-13 18:35:39.12821678 +0000 UTC m=+0.015930886 image pull 8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256 2024-09-13T18:35:39.508 INFO:journalctl@ceph.osd.0.smithi114.stdout:Sep 13 18:35:39 smithi114 podman[110903]: 2024-09-13 18:35:39.253329125 +0000 UTC m=+0.141043234 container create 4ad31b2c189477922ce7cb8f34db5c0bf4658ad6902c0e34dbd4d624e312a7f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083) 2024-09-13T18:35:39.508 INFO:journalctl@ceph.osd.0.smithi114.stdout:Sep 13 18:35:39 smithi114 podman[110903]: 2024-09-13 18:35:39.505956907 +0000 UTC m=+0.393671011 container init 4ad31b2c189477922ce7cb8f34db5c0bf4658ad6902c0e34dbd4d624e312a7f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, maintainer=Guillaume Abrioux ) 2024-09-13T18:35:39.886 INFO:journalctl@ceph.osd.0.smithi114.stdout:Sep 13 18:35:39 smithi114 podman[110903]: 2024-09-13 18:35:39.509289267 +0000 UTC m=+0.397003373 container start 4ad31b2c189477922ce7cb8f34db5c0bf4658ad6902c0e34dbd4d624e312a7f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-13T18:35:39.886 INFO:journalctl@ceph.osd.0.smithi114.stdout:Sep 13 18:35:39 smithi114 podman[110903]: 2024-09-13 18:35:39.625052279 +0000 UTC m=+0.512766396 container attach 4ad31b2c189477922ce7cb8f34db5c0bf4658ad6902c0e34dbd4d624e312a7f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-13T18:35:40.220 INFO:journalctl@ceph.osd.0.smithi114.stdout:Sep 13 18:35:39 smithi114 podman[110903]: 2024-09-13 18:35:39.88482051 +0000 UTC m=+0.772534618 container died 4ad31b2c189477922ce7cb8f34db5c0bf4658ad6902c0e34dbd4d624e312a7f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-13T18:35:40.559 DEBUG:teuthology.orchestra.run.smithi114:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@osd.0.service' 2024-09-13T18:35:40.599 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T18:35:40.599 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-13T18:35:40.599 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-13T18:35:40.600 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@osd.1 2024-09-13T18:35:40.917 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 13 18:35:40 smithi134 systemd[1]: Stopping Ceph osd.1 for 97e7edf2-71fb-11ef-bceb-c7b262605968... 2024-09-13T18:35:41.369 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 13 18:35:40 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-1[38780]: 2024-09-13T18:35:40.914+0000 7fa92121f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T18:35:41.369 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 13 18:35:40 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-1[38780]: 2024-09-13T18:35:40.914+0000 7fa92121f640 -1 osd.1 111 *** Got signal Terminated *** 2024-09-13T18:35:41.369 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 13 18:35:40 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-1[38780]: 2024-09-13T18:35:40.914+0000 7fa92121f640 -1 osd.1 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T18:35:46.820 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 13 18:35:46 smithi134 podman[69900]: 2024-09-13 18:35:46.532971286 +0000 UTC m=+5.739755454 container died c0b5b27a010a9c69ec02fb7c5479af5a40b780f90d4a1b5f51f7d280b76320e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-1, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-13T18:35:47.119 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 13 18:35:46 smithi134 podman[69900]: 2024-09-13 18:35:46.93128954 +0000 UTC m=+6.138073705 container cleanup c0b5b27a010a9c69ec02fb7c5479af5a40b780f90d4a1b5f51f7d280b76320e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-1, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-13T18:35:47.119 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 13 18:35:46 smithi134 bash[69900]: ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-1 2024-09-13T18:35:47.500 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 13 18:35:47 smithi134 podman[69920]: 2024-09-13 18:35:47.174385055 +0000 UTC m=+0.635107444 container remove c0b5b27a010a9c69ec02fb7c5479af5a40b780f90d4a1b5f51f7d280b76320e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, io.buildah.version=1.37.2) 2024-09-13T18:35:47.855 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 13 18:35:47 smithi134 podman[70008]: 2024-09-13 18:35:47.400469337 +0000 UTC m=+0.015758419 image pull 8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256 2024-09-13T18:35:47.855 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 13 18:35:47 smithi134 podman[70008]: 2024-09-13 18:35:47.541974222 +0000 UTC m=+0.157263299 container create b9efb17b5cdb9f79a1f99170c947712cd362a327d006776050b6216acd4c3342 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.schema-version=1.0) 2024-09-13T18:35:48.119 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 13 18:35:47 smithi134 podman[70008]: 2024-09-13 18:35:47.852905118 +0000 UTC m=+0.468194200 container init b9efb17b5cdb9f79a1f99170c947712cd362a327d006776050b6216acd4c3342 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-13T18:35:48.120 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 13 18:35:47 smithi134 podman[70008]: 2024-09-13 18:35:47.856978795 +0000 UTC m=+0.472267874 container start b9efb17b5cdb9f79a1f99170c947712cd362a327d006776050b6216acd4c3342 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=) 2024-09-13T18:35:48.120 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 13 18:35:48 smithi134 podman[70008]: 2024-09-13 18:35:48.00334029 +0000 UTC m=+0.618629368 container attach b9efb17b5cdb9f79a1f99170c947712cd362a327d006776050b6216acd4c3342 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-13T18:35:48.120 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 13 18:35:48 smithi134 podman[70008]: 2024-09-13 18:35:48.097545628 +0000 UTC m=+0.712834708 container died b9efb17b5cdb9f79a1f99170c947712cd362a327d006776050b6216acd4c3342 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-1-deactivate, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-13T18:35:48.826 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@osd.1.service' 2024-09-13T18:35:48.865 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T18:35:48.865 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-13T18:35:48.866 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-13T18:35:48.866 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@osd.2 2024-09-13T18:35:49.194 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 13 18:35:48 smithi204 systemd[1]: Stopping Ceph osd.2 for 97e7edf2-71fb-11ef-bceb-c7b262605968... 2024-09-13T18:35:49.453 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 13 18:35:49 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-2[39815]: 2024-09-13T18:35:49.262+0000 7f7efd442640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T18:35:49.453 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 13 18:35:49 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-2[39815]: 2024-09-13T18:35:49.262+0000 7f7efd442640 -1 osd.2 111 *** Got signal Terminated *** 2024-09-13T18:35:49.453 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 13 18:35:49 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-2[39815]: 2024-09-13T18:35:49.262+0000 7f7efd442640 -1 osd.2 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T18:35:55.325 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 13 18:35:55 smithi204 podman[71163]: 2024-09-13 18:35:55.05008024 +0000 UTC m=+5.989993059 container died f862f9bc03c9b46a02b62d3c0d872d2332bf6a5f7c87805949e8f9d74d934a54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-13T18:35:55.694 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 13 18:35:55 smithi204 podman[71163]: 2024-09-13 18:35:55.422386792 +0000 UTC m=+6.362299601 container cleanup f862f9bc03c9b46a02b62d3c0d872d2332bf6a5f7c87805949e8f9d74d934a54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-2, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-13T18:35:55.695 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 13 18:35:55 smithi204 bash[71163]: ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-2 2024-09-13T18:35:56.093 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 13 18:35:55 smithi204 podman[71191]: 2024-09-13 18:35:55.766144904 +0000 UTC m=+0.713223919 container remove f862f9bc03c9b46a02b62d3c0d872d2332bf6a5f7c87805949e8f9d74d934a54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-13T18:35:56.389 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 13 18:35:56 smithi204 podman[71280]: 2024-09-13 18:35:55.993995493 +0000 UTC m=+0.016513704 image pull 8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256 2024-09-13T18:35:56.389 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 13 18:35:56 smithi204 podman[71280]: 2024-09-13 18:35:56.125504488 +0000 UTC m=+0.148022702 container create 1357ed83c0e96c4962fbdd18976188368c1560fce6b38d14d47943d8ac95eebc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-13T18:35:56.389 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 13 18:35:56 smithi204 podman[71280]: 2024-09-13 18:35:56.386537747 +0000 UTC m=+0.409055958 container init 1357ed83c0e96c4962fbdd18976188368c1560fce6b38d14d47943d8ac95eebc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-2-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T18:35:56.694 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 13 18:35:56 smithi204 podman[71280]: 2024-09-13 18:35:56.390096468 +0000 UTC m=+0.412614682 container start 1357ed83c0e96c4962fbdd18976188368c1560fce6b38d14d47943d8ac95eebc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-13T18:35:56.695 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 13 18:35:56 smithi204 podman[71280]: 2024-09-13 18:35:56.495002882 +0000 UTC m=+0.517521094 container attach 1357ed83c0e96c4962fbdd18976188368c1560fce6b38d14d47943d8ac95eebc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-2-deactivate, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-13T18:35:56.695 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 13 18:35:56 smithi204 podman[71280]: 2024-09-13 18:35:56.629050437 +0000 UTC m=+0.651568650 container died 1357ed83c0e96c4962fbdd18976188368c1560fce6b38d14d47943d8ac95eebc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-13T18:35:57.254 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@osd.2.service' 2024-09-13T18:35:57.295 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T18:35:57.296 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-13T18:35:57.296 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-13T18:35:57.296 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@osd.3 2024-09-13T18:35:57.601 INFO:journalctl@ceph.osd.3.smithi114.stdout:Sep 13 18:35:57 smithi114 systemd[1]: Stopping Ceph osd.3 for 97e7edf2-71fb-11ef-bceb-c7b262605968... 2024-09-13T18:35:57.970 INFO:journalctl@ceph.osd.3.smithi114.stdout:Sep 13 18:35:57 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-3[51352]: 2024-09-13T18:35:57.597+0000 7ff230e11640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T18:35:57.970 INFO:journalctl@ceph.osd.3.smithi114.stdout:Sep 13 18:35:57 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-3[51352]: 2024-09-13T18:35:57.597+0000 7ff230e11640 -1 osd.3 111 *** Got signal Terminated *** 2024-09-13T18:35:57.970 INFO:journalctl@ceph.osd.3.smithi114.stdout:Sep 13 18:35:57 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-3[51352]: 2024-09-13T18:35:57.597+0000 7ff230e11640 -1 osd.3 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T18:35:58.619 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:35:58 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:35:58.163+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:35:58.695 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 13 18:35:58 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-5[46242]: 2024-09-13T18:35:58.325+0000 7f0e16251640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.267471+0000 front 2024-09-13T18:35:35.267456+0000 (oldest deadline 2024-09-13T18:35:58.167332+0000) 2024-09-13T18:35:58.970 INFO:journalctl@ceph.osd.3.smithi114.stdout:Sep 13 18:35:58 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-3[51352]: 2024-09-13T18:35:58.666+0000 7ff22d425640 -1 osd.3 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:32.273043+0000 front 2024-09-13T18:35:32.272954+0000 (oldest deadline 2024-09-13T18:35:58.173124+0000) 2024-09-13T18:35:59.619 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:35:59 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:35:59.148+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:35:59.694 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 13 18:35:59 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-5[46242]: 2024-09-13T18:35:59.304+0000 7f0e16251640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.267471+0000 front 2024-09-13T18:35:35.267456+0000 (oldest deadline 2024-09-13T18:35:58.167332+0000) 2024-09-13T18:35:59.970 INFO:journalctl@ceph.osd.3.smithi114.stdout:Sep 13 18:35:59 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-3[51352]: 2024-09-13T18:35:59.624+0000 7ff22d425640 -1 osd.3 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:32.273043+0000 front 2024-09-13T18:35:32.272954+0000 (oldest deadline 2024-09-13T18:35:58.173124+0000) 2024-09-13T18:35:59.970 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:35:59 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:35:59.719+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.588828+0000 front 2024-09-13T18:35:34.588857+0000 (oldest deadline 2024-09-13T18:35:59.288937+0000) 2024-09-13T18:36:00.619 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:00 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:00.127+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:00.694 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 13 18:36:00 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-5[46242]: 2024-09-13T18:36:00.333+0000 7f0e16251640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.267471+0000 front 2024-09-13T18:35:35.267456+0000 (oldest deadline 2024-09-13T18:35:58.167332+0000) 2024-09-13T18:36:00.970 INFO:journalctl@ceph.osd.3.smithi114.stdout:Sep 13 18:36:00 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-3[51352]: 2024-09-13T18:36:00.671+0000 7ff22d425640 -1 osd.3 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:32.273043+0000 front 2024-09-13T18:35:32.272954+0000 (oldest deadline 2024-09-13T18:35:58.173124+0000) 2024-09-13T18:36:00.970 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:00 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:00.749+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.588828+0000 front 2024-09-13T18:35:34.588857+0000 (oldest deadline 2024-09-13T18:35:59.288937+0000) 2024-09-13T18:36:01.619 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:01 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:01.120+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:01.621 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 13 18:36:01 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-5[46242]: 2024-09-13T18:36:01.348+0000 7f0e16251640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.267471+0000 front 2024-09-13T18:35:35.267456+0000 (oldest deadline 2024-09-13T18:35:58.167332+0000) 2024-09-13T18:36:01.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:01 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:01.317+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:01.944 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:01 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:01.618+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.701894+0000 front 2024-09-13T18:35:35.701959+0000 (oldest deadline 2024-09-13T18:36:01.502502+0000) 2024-09-13T18:36:01.970 INFO:journalctl@ceph.osd.3.smithi114.stdout:Sep 13 18:36:01 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-3[51352]: 2024-09-13T18:36:01.701+0000 7ff22d425640 -1 osd.3 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:32.273043+0000 front 2024-09-13T18:35:32.272954+0000 (oldest deadline 2024-09-13T18:35:58.173124+0000) 2024-09-13T18:36:01.970 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:01 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:01.747+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.588828+0000 front 2024-09-13T18:35:34.588857+0000 (oldest deadline 2024-09-13T18:35:59.288937+0000) 2024-09-13T18:36:02.583 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:02 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:02.268+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:02.583 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 13 18:36:02 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-5[46242]: 2024-09-13T18:36:02.361+0000 7f0e16251640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.267471+0000 front 2024-09-13T18:35:35.267456+0000 (oldest deadline 2024-09-13T18:35:58.167332+0000) 2024-09-13T18:36:02.619 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:02 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:02.135+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:02.944 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:02 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:02.581+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.701894+0000 front 2024-09-13T18:35:35.701959+0000 (oldest deadline 2024-09-13T18:36:01.502502+0000) 2024-09-13T18:36:03.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:02 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:02.772+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.588828+0000 front 2024-09-13T18:35:34.588857+0000 (oldest deadline 2024-09-13T18:35:59.288937+0000) 2024-09-13T18:36:03.580 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 13 18:36:03 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-5[46242]: 2024-09-13T18:36:03.356+0000 7f0e16251640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.267471+0000 front 2024-09-13T18:35:35.267456+0000 (oldest deadline 2024-09-13T18:35:58.167332+0000) 2024-09-13T18:36:03.581 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:03 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:03.236+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:03.601 INFO:journalctl@ceph.osd.3.smithi114.stdout:Sep 13 18:36:03 smithi114 podman[111017]: 2024-09-13 18:36:03.298944175 +0000 UTC m=+5.806484213 container died a5a5d5b5f3481c815592895ced5e05020865d5f60f579844dfebecd34379c83a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-3, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-13T18:36:03.619 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:03 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:03.164+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:03.944 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:03 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:03.578+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.701894+0000 front 2024-09-13T18:35:35.701959+0000 (oldest deadline 2024-09-13T18:36:01.502502+0000) 2024-09-13T18:36:03.970 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:03 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:03.810+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.588828+0000 front 2024-09-13T18:35:34.588857+0000 (oldest deadline 2024-09-13T18:35:59.288937+0000) 2024-09-13T18:36:03.970 INFO:journalctl@ceph.osd.3.smithi114.stdout:Sep 13 18:36:03 smithi114 podman[111017]: 2024-09-13 18:36:03.698556433 +0000 UTC m=+6.206096471 container cleanup a5a5d5b5f3481c815592895ced5e05020865d5f60f579844dfebecd34379c83a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-3, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-13T18:36:03.970 INFO:journalctl@ceph.osd.3.smithi114.stdout:Sep 13 18:36:03 smithi114 bash[111017]: ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-3 2024-09-13T18:36:04.312 INFO:journalctl@ceph.osd.3.smithi114.stdout:Sep 13 18:36:03 smithi114 podman[111036]: 2024-09-13 18:36:03.983532798 +0000 UTC m=+0.676728977 container remove a5a5d5b5f3481c815592895ced5e05020865d5f60f579844dfebecd34379c83a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-3, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-13T18:36:04.549 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 13 18:36:04 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-5[46242]: 2024-09-13T18:36:04.347+0000 7f0e16251640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.267471+0000 front 2024-09-13T18:35:35.267456+0000 (oldest deadline 2024-09-13T18:35:58.167332+0000) 2024-09-13T18:36:04.550 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:04 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:04.248+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:04.597 INFO:journalctl@ceph.osd.3.smithi114.stdout:Sep 13 18:36:04 smithi114 podman[111126]: 2024-09-13 18:36:04.212361242 +0000 UTC m=+0.015756704 image pull 8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256 2024-09-13T18:36:04.597 INFO:journalctl@ceph.osd.3.smithi114.stdout:Sep 13 18:36:04 smithi114 podman[111126]: 2024-09-13 18:36:04.334470245 +0000 UTC m=+0.137865714 container create 2b42c36732ad2f45c78bfb73638ef2a77ce0442f1db8e2b4047cecd831c034b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.schema-version=1.0) 2024-09-13T18:36:04.619 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:04 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:04.173+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:04.944 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:04 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:04.547+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.701894+0000 front 2024-09-13T18:35:35.701959+0000 (oldest deadline 2024-09-13T18:36:01.502502+0000) 2024-09-13T18:36:04.970 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:04 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:04.829+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.588828+0000 front 2024-09-13T18:35:34.588857+0000 (oldest deadline 2024-09-13T18:35:59.288937+0000) 2024-09-13T18:36:04.970 INFO:journalctl@ceph.osd.3.smithi114.stdout:Sep 13 18:36:04 smithi114 podman[111126]: 2024-09-13 18:36:04.595384549 +0000 UTC m=+0.398780011 container init 2b42c36732ad2f45c78bfb73638ef2a77ce0442f1db8e2b4047cecd831c034b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083) 2024-09-13T18:36:04.970 INFO:journalctl@ceph.osd.3.smithi114.stdout:Sep 13 18:36:04 smithi114 podman[111126]: 2024-09-13 18:36:04.599082466 +0000 UTC m=+0.402477929 container start 2b42c36732ad2f45c78bfb73638ef2a77ce0442f1db8e2b4047cecd831c034b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-13T18:36:04.970 INFO:journalctl@ceph.osd.3.smithi114.stdout:Sep 13 18:36:04 smithi114 podman[111126]: 2024-09-13 18:36:04.662206043 +0000 UTC m=+0.465601517 container attach 2b42c36732ad2f45c78bfb73638ef2a77ce0442f1db8e2b4047cecd831c034b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True) 2024-09-13T18:36:04.971 INFO:journalctl@ceph.osd.3.smithi114.stdout:Sep 13 18:36:04 smithi114 podman[111126]: 2024-09-13 18:36:04.783002326 +0000 UTC m=+0.586397789 container died 2b42c36732ad2f45c78bfb73638ef2a77ce0442f1db8e2b4047cecd831c034b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-3-deactivate, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-13T18:36:05.419 DEBUG:teuthology.orchestra.run.smithi114:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@osd.3.service' 2024-09-13T18:36:05.459 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T18:36:05.459 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-13T18:36:05.459 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-13T18:36:05.460 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@osd.4 2024-09-13T18:36:05.487 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:05 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:05.211+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:05.565 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 13 18:36:05 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-5[46242]: 2024-09-13T18:36:05.300+0000 7f0e16251640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.267471+0000 front 2024-09-13T18:35:35.267456+0000 (oldest deadline 2024-09-13T18:35:58.167332+0000) 2024-09-13T18:36:05.565 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:05 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:05.257+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:05.804 INFO:journalctl@ceph.osd.4.smithi134.stdout:Sep 13 18:36:05 smithi134 systemd[1]: Stopping Ceph osd.4 for 97e7edf2-71fb-11ef-bceb-c7b262605968... 2024-09-13T18:36:05.944 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:05 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:05.563+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.701894+0000 front 2024-09-13T18:35:35.701959+0000 (oldest deadline 2024-09-13T18:36:01.502502+0000) 2024-09-13T18:36:06.119 INFO:journalctl@ceph.osd.4.smithi134.stdout:Sep 13 18:36:05 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-4[45342]: 2024-09-13T18:36:05.801+0000 7fc0c0ab4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T18:36:06.119 INFO:journalctl@ceph.osd.4.smithi134.stdout:Sep 13 18:36:05 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-4[45342]: 2024-09-13T18:36:05.801+0000 7fc0c0ab4640 -1 osd.4 111 *** Got signal Terminated *** 2024-09-13T18:36:06.119 INFO:journalctl@ceph.osd.4.smithi134.stdout:Sep 13 18:36:05 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-4[45342]: 2024-09-13T18:36:05.801+0000 7fc0c0ab4640 -1 osd.4 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T18:36:06.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:05 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:05.844+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.588828+0000 front 2024-09-13T18:35:34.588857+0000 (oldest deadline 2024-09-13T18:35:59.288937+0000) 2024-09-13T18:36:06.619 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:06 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:06.220+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:06.694 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:06 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:06.548+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.701894+0000 front 2024-09-13T18:35:35.701959+0000 (oldest deadline 2024-09-13T18:36:01.502502+0000) 2024-09-13T18:36:06.695 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 13 18:36:06 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-5[46242]: 2024-09-13T18:36:06.310+0000 7f0e16251640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.267471+0000 front 2024-09-13T18:35:35.267456+0000 (oldest deadline 2024-09-13T18:35:58.167332+0000) 2024-09-13T18:36:06.695 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:06 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:06.302+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:07.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:06 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:06.866+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.588828+0000 front 2024-09-13T18:35:34.588857+0000 (oldest deadline 2024-09-13T18:35:59.288937+0000) 2024-09-13T18:36:07.580 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 13 18:36:07 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-5[46242]: 2024-09-13T18:36:07.283+0000 7f0e16251640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.267471+0000 front 2024-09-13T18:35:35.267456+0000 (oldest deadline 2024-09-13T18:35:58.167332+0000) 2024-09-13T18:36:07.581 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:07 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:07.264+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:07.619 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:07 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:07.248+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:07.944 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:07 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:07.578+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.701894+0000 front 2024-09-13T18:35:35.701959+0000 (oldest deadline 2024-09-13T18:36:01.502502+0000) 2024-09-13T18:36:07.944 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:07 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:07.578+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:41.503145+0000 front 2024-09-13T18:35:41.503165+0000 (oldest deadline 2024-09-13T18:36:07.403215+0000) 2024-09-13T18:36:08.119 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:07 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:07.691+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:45.793799+0000 front 2024-09-13T18:35:45.793857+0000 (oldest deadline 2024-09-13T18:36:07.493982+0000) 2024-09-13T18:36:08.121 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:07 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:07.842+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.588828+0000 front 2024-09-13T18:35:34.588857+0000 (oldest deadline 2024-09-13T18:35:59.288937+0000) 2024-09-13T18:36:08.121 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 13 18:36:07 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-6[58262]: 2024-09-13T18:36:07.922+0000 7f9067057640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:42.386572+0000 front 2024-09-13T18:35:42.386641+0000 (oldest deadline 2024-09-13T18:36:07.686627+0000) 2024-09-13T18:36:08.612 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 13 18:36:08 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-5[46242]: 2024-09-13T18:36:08.304+0000 7f0e16251640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.267471+0000 front 2024-09-13T18:35:35.267456+0000 (oldest deadline 2024-09-13T18:35:58.167332+0000) 2024-09-13T18:36:08.612 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:08 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:08.305+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:08.619 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:08 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:08.228+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:08.619 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:08 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:08.228+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.824009+0000 front 2024-09-13T18:35:43.824043+0000 (oldest deadline 2024-09-13T18:36:07.324119+0000) 2024-09-13T18:36:08.944 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:08 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:08.609+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.701894+0000 front 2024-09-13T18:35:35.701959+0000 (oldest deadline 2024-09-13T18:36:01.502502+0000) 2024-09-13T18:36:08.944 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:08 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:08.609+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:41.503145+0000 front 2024-09-13T18:35:41.503165+0000 (oldest deadline 2024-09-13T18:36:07.403215+0000) 2024-09-13T18:36:09.119 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:08 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:08.677+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:45.793799+0000 front 2024-09-13T18:35:45.793857+0000 (oldest deadline 2024-09-13T18:36:07.493982+0000) 2024-09-13T18:36:09.137 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 13 18:36:08 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-6[58262]: 2024-09-13T18:36:08.892+0000 7f9067057640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:42.386572+0000 front 2024-09-13T18:35:42.386641+0000 (oldest deadline 2024-09-13T18:36:07.686627+0000) 2024-09-13T18:36:09.137 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:08 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:08.853+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.588828+0000 front 2024-09-13T18:35:34.588857+0000 (oldest deadline 2024-09-13T18:35:59.288937+0000) 2024-09-13T18:36:09.619 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:09 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:09.259+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:09.619 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:09 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:09.259+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.824009+0000 front 2024-09-13T18:35:43.824043+0000 (oldest deadline 2024-09-13T18:36:07.324119+0000) 2024-09-13T18:36:09.694 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 13 18:36:09 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-5[46242]: 2024-09-13T18:36:09.345+0000 7f0e16251640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.267471+0000 front 2024-09-13T18:35:35.267456+0000 (oldest deadline 2024-09-13T18:35:58.167332+0000) 2024-09-13T18:36:09.695 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:09 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:09.581+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.701894+0000 front 2024-09-13T18:35:35.701959+0000 (oldest deadline 2024-09-13T18:36:01.502502+0000) 2024-09-13T18:36:09.695 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:09 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:09.581+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:41.503145+0000 front 2024-09-13T18:35:41.503165+0000 (oldest deadline 2024-09-13T18:36:07.403215+0000) 2024-09-13T18:36:09.695 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:09 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:09.349+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:10.119 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:09 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:09.659+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:45.793799+0000 front 2024-09-13T18:35:45.793857+0000 (oldest deadline 2024-09-13T18:36:07.493982+0000) 2024-09-13T18:36:10.220 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 13 18:36:09 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-6[58262]: 2024-09-13T18:36:09.933+0000 7f9067057640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:42.386572+0000 front 2024-09-13T18:35:42.386641+0000 (oldest deadline 2024-09-13T18:36:07.686627+0000) 2024-09-13T18:36:10.221 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:09 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:09.844+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.588828+0000 front 2024-09-13T18:35:34.588857+0000 (oldest deadline 2024-09-13T18:35:59.288937+0000) 2024-09-13T18:36:10.592 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 13 18:36:10 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-5[46242]: 2024-09-13T18:36:10.319+0000 7f0e16251640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.267471+0000 front 2024-09-13T18:35:35.267456+0000 (oldest deadline 2024-09-13T18:35:58.167332+0000) 2024-09-13T18:36:10.593 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:10 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:10.352+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:10.619 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:10 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:10.292+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:10.619 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:10 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:10.292+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.824009+0000 front 2024-09-13T18:35:43.824043+0000 (oldest deadline 2024-09-13T18:36:07.324119+0000) 2024-09-13T18:36:10.944 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:10 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:10.590+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.701894+0000 front 2024-09-13T18:35:35.701959+0000 (oldest deadline 2024-09-13T18:36:01.502502+0000) 2024-09-13T18:36:10.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:10 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:10.590+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:41.503145+0000 front 2024-09-13T18:35:41.503165+0000 (oldest deadline 2024-09-13T18:36:07.403215+0000) 2024-09-13T18:36:11.119 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:10 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:10.685+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:45.793799+0000 front 2024-09-13T18:35:45.793857+0000 (oldest deadline 2024-09-13T18:36:07.493982+0000) 2024-09-13T18:36:11.220 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 13 18:36:10 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-6[58262]: 2024-09-13T18:36:10.970+0000 7f9067057640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:42.386572+0000 front 2024-09-13T18:35:42.386641+0000 (oldest deadline 2024-09-13T18:36:07.686627+0000) 2024-09-13T18:36:11.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:10 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:10.846+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.588828+0000 front 2024-09-13T18:35:34.588857+0000 (oldest deadline 2024-09-13T18:35:59.288937+0000) 2024-09-13T18:36:11.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:10 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:10.846+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.790933+0000 front 2024-09-13T18:35:43.791007+0000 (oldest deadline 2024-09-13T18:36:10.791344+0000) 2024-09-13T18:36:11.595 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:11 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:11.315+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:11.595 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 13 18:36:11 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-5[46242]: 2024-09-13T18:36:11.351+0000 7f0e16251640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.267471+0000 front 2024-09-13T18:35:35.267456+0000 (oldest deadline 2024-09-13T18:35:58.167332+0000) 2024-09-13T18:36:11.596 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 13 18:36:11 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-5[46242]: 2024-09-13T18:36:11.351+0000 7f0e16251640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:45.169068+0000 front 2024-09-13T18:35:45.168939+0000 (oldest deadline 2024-09-13T18:36:11.069051+0000) 2024-09-13T18:36:11.598 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:11 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:11.281+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:11.598 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:11 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:11.281+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.824009+0000 front 2024-09-13T18:35:43.824043+0000 (oldest deadline 2024-09-13T18:36:07.324119+0000) 2024-09-13T18:36:11.865 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:11 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:11.733+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:45.793799+0000 front 2024-09-13T18:35:45.793857+0000 (oldest deadline 2024-09-13T18:36:07.493982+0000) 2024-09-13T18:36:11.865 INFO:journalctl@ceph.osd.4.smithi134.stdout:Sep 13 18:36:11 smithi134 podman[70124]: 2024-09-13 18:36:11.596728009 +0000 UTC m=+5.938330355 container died 497774642f4512d56db2feee0011b6704652bff5e0e70504c5115f2cd17eae92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-4, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-09-13T18:36:11.944 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:11 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:11.592+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.701894+0000 front 2024-09-13T18:35:35.701959+0000 (oldest deadline 2024-09-13T18:36:01.502502+0000) 2024-09-13T18:36:11.944 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:11 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:11.592+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:41.503145+0000 front 2024-09-13T18:35:41.503165+0000 (oldest deadline 2024-09-13T18:36:07.403215+0000) 2024-09-13T18:36:12.119 INFO:journalctl@ceph.osd.4.smithi134.stdout:Sep 13 18:36:11 smithi134 podman[70124]: 2024-09-13 18:36:11.968773552 +0000 UTC m=+6.310375899 container cleanup 497774642f4512d56db2feee0011b6704652bff5e0e70504c5115f2cd17eae92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-13T18:36:12.119 INFO:journalctl@ceph.osd.4.smithi134.stdout:Sep 13 18:36:11 smithi134 bash[70124]: ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-4 2024-09-13T18:36:12.220 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 13 18:36:11 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-6[58262]: 2024-09-13T18:36:11.969+0000 7f9067057640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:42.386572+0000 front 2024-09-13T18:35:42.386641+0000 (oldest deadline 2024-09-13T18:36:07.686627+0000) 2024-09-13T18:36:12.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:11 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:11.835+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.588828+0000 front 2024-09-13T18:35:34.588857+0000 (oldest deadline 2024-09-13T18:35:59.288937+0000) 2024-09-13T18:36:12.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:11 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:11.835+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.790933+0000 front 2024-09-13T18:35:43.791007+0000 (oldest deadline 2024-09-13T18:36:10.791344+0000) 2024-09-13T18:36:12.544 INFO:journalctl@ceph.osd.4.smithi134.stdout:Sep 13 18:36:12 smithi134 podman[70141]: 2024-09-13 18:36:12.136787278 +0000 UTC m=+0.532364884 container remove 497774642f4512d56db2feee0011b6704652bff5e0e70504c5115f2cd17eae92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-4, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.schema-version=1.0) 2024-09-13T18:36:12.544 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:12 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:12.246+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:12.544 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:12 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:12.246+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.824009+0000 front 2024-09-13T18:35:43.824043+0000 (oldest deadline 2024-09-13T18:36:07.324119+0000) 2024-09-13T18:36:12.569 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:12 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:12.304+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:12.570 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 13 18:36:12 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-5[46242]: 2024-09-13T18:36:12.378+0000 7f0e16251640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.267471+0000 front 2024-09-13T18:35:35.267456+0000 (oldest deadline 2024-09-13T18:35:58.167332+0000) 2024-09-13T18:36:12.570 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 13 18:36:12 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-5[46242]: 2024-09-13T18:36:12.378+0000 7f0e16251640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:45.169068+0000 front 2024-09-13T18:35:45.168939+0000 (oldest deadline 2024-09-13T18:36:11.069051+0000) 2024-09-13T18:36:12.800 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:12 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:12.781+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:45.793799+0000 front 2024-09-13T18:35:45.793857+0000 (oldest deadline 2024-09-13T18:36:07.493982+0000) 2024-09-13T18:36:12.800 INFO:journalctl@ceph.osd.4.smithi134.stdout:Sep 13 18:36:12 smithi134 podman[70234]: 2024-09-13 18:36:12.445047392 +0000 UTC m=+0.016071146 image pull 8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256 2024-09-13T18:36:12.800 INFO:journalctl@ceph.osd.4.smithi134.stdout:Sep 13 18:36:12 smithi134 podman[70234]: 2024-09-13 18:36:12.545831614 +0000 UTC m=+0.116855367 container create 66ff6485efd76ce926bf9abc5ee2aefe912ba5d33af1bff8c8a7a4a5e2be5f5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-13T18:36:12.801 INFO:journalctl@ceph.osd.4.smithi134.stdout:Sep 13 18:36:12 smithi134 podman[70234]: 2024-09-13 18:36:12.798472604 +0000 UTC m=+0.369496353 container init 66ff6485efd76ce926bf9abc5ee2aefe912ba5d33af1bff8c8a7a4a5e2be5f5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-13T18:36:12.944 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:12 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:12.566+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.701894+0000 front 2024-09-13T18:35:35.701959+0000 (oldest deadline 2024-09-13T18:36:01.502502+0000) 2024-09-13T18:36:12.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:12 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:12.566+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:41.503145+0000 front 2024-09-13T18:35:41.503165+0000 (oldest deadline 2024-09-13T18:36:07.403215+0000) 2024-09-13T18:36:13.119 INFO:journalctl@ceph.osd.4.smithi134.stdout:Sep 13 18:36:12 smithi134 podman[70234]: 2024-09-13 18:36:12.802062619 +0000 UTC m=+0.373086377 container start 66ff6485efd76ce926bf9abc5ee2aefe912ba5d33af1bff8c8a7a4a5e2be5f5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-4-deactivate, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-13T18:36:13.120 INFO:journalctl@ceph.osd.4.smithi134.stdout:Sep 13 18:36:12 smithi134 podman[70234]: 2024-09-13 18:36:12.927127594 +0000 UTC m=+0.498151349 container attach 66ff6485efd76ce926bf9abc5ee2aefe912ba5d33af1bff8c8a7a4a5e2be5f5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, CEPH_POINT_RELEASE=) 2024-09-13T18:36:13.120 INFO:journalctl@ceph.osd.4.smithi134.stdout:Sep 13 18:36:12 smithi134 podman[70234]: 2024-09-13 18:36:12.987874385 +0000 UTC m=+0.558898132 container died 66ff6485efd76ce926bf9abc5ee2aefe912ba5d33af1bff8c8a7a4a5e2be5f5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-4-deactivate, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-13T18:36:13.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:12 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:12.811+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.588828+0000 front 2024-09-13T18:35:34.588857+0000 (oldest deadline 2024-09-13T18:35:59.288937+0000) 2024-09-13T18:36:13.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:12 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:12.811+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.790933+0000 front 2024-09-13T18:35:43.791007+0000 (oldest deadline 2024-09-13T18:36:10.791344+0000) 2024-09-13T18:36:13.220 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 13 18:36:12 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-6[58262]: 2024-09-13T18:36:12.957+0000 7f9067057640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:42.386572+0000 front 2024-09-13T18:35:42.386641+0000 (oldest deadline 2024-09-13T18:36:07.686627+0000) 2024-09-13T18:36:13.619 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:13 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:13.200+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:13.619 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:13 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:13.200+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.824009+0000 front 2024-09-13T18:35:43.824043+0000 (oldest deadline 2024-09-13T18:36:07.324119+0000) 2024-09-13T18:36:13.663 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@osd.4.service' 2024-09-13T18:36:13.694 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:13 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:13.348+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:13.695 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:13 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:13.538+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.701894+0000 front 2024-09-13T18:35:35.701959+0000 (oldest deadline 2024-09-13T18:36:01.502502+0000) 2024-09-13T18:36:13.695 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:13 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:13.538+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:41.503145+0000 front 2024-09-13T18:35:41.503165+0000 (oldest deadline 2024-09-13T18:36:07.403215+0000) 2024-09-13T18:36:13.695 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 13 18:36:13 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-5[46242]: 2024-09-13T18:36:13.357+0000 7f0e16251640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.267471+0000 front 2024-09-13T18:35:35.267456+0000 (oldest deadline 2024-09-13T18:35:58.167332+0000) 2024-09-13T18:36:13.695 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 13 18:36:13 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-5[46242]: 2024-09-13T18:36:13.357+0000 7f0e16251640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:45.169068+0000 front 2024-09-13T18:35:45.168939+0000 (oldest deadline 2024-09-13T18:36:11.069051+0000) 2024-09-13T18:36:13.743 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T18:36:13.743 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-13T18:36:13.743 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-13T18:36:13.744 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@osd.5 2024-09-13T18:36:14.071 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 13 18:36:13 smithi204 systemd[1]: Stopping Ceph osd.5 for 97e7edf2-71fb-11ef-bceb-c7b262605968... 2024-09-13T18:36:14.119 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:13 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:13.807+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:45.793799+0000 front 2024-09-13T18:35:45.793857+0000 (oldest deadline 2024-09-13T18:36:07.493982+0000) 2024-09-13T18:36:14.220 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 13 18:36:13 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-6[58262]: 2024-09-13T18:36:13.948+0000 7f9067057640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:42.386572+0000 front 2024-09-13T18:35:42.386641+0000 (oldest deadline 2024-09-13T18:36:07.686627+0000) 2024-09-13T18:36:14.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:13 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:13.832+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.588828+0000 front 2024-09-13T18:35:34.588857+0000 (oldest deadline 2024-09-13T18:35:59.288937+0000) 2024-09-13T18:36:14.221 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:13 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:13.832+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.790933+0000 front 2024-09-13T18:35:43.791007+0000 (oldest deadline 2024-09-13T18:36:10.791344+0000) 2024-09-13T18:36:14.342 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 13 18:36:14 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-5[46242]: 2024-09-13T18:36:14.068+0000 7f0e19c3d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T18:36:14.342 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 13 18:36:14 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-5[46242]: 2024-09-13T18:36:14.068+0000 7f0e19c3d640 -1 osd.5 111 *** Got signal Terminated *** 2024-09-13T18:36:14.342 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 13 18:36:14 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-5[46242]: 2024-09-13T18:36:14.068+0000 7f0e19c3d640 -1 osd.5 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T18:36:14.342 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 13 18:36:14 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-5[46242]: 2024-09-13T18:36:14.340+0000 7f0e16251640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.267471+0000 front 2024-09-13T18:35:35.267456+0000 (oldest deadline 2024-09-13T18:35:58.167332+0000) 2024-09-13T18:36:14.342 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 13 18:36:14 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-5[46242]: 2024-09-13T18:36:14.340+0000 7f0e16251640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:45.169068+0000 front 2024-09-13T18:35:45.168939+0000 (oldest deadline 2024-09-13T18:36:11.069051+0000) 2024-09-13T18:36:14.619 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:14 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:14.227+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:14.619 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:14 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:14.227+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.824009+0000 front 2024-09-13T18:35:43.824043+0000 (oldest deadline 2024-09-13T18:36:07.324119+0000) 2024-09-13T18:36:14.694 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:14 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:14.368+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:14.695 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:14 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:14.528+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.701894+0000 front 2024-09-13T18:35:35.701959+0000 (oldest deadline 2024-09-13T18:36:01.502502+0000) 2024-09-13T18:36:14.695 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:14 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:14.528+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:41.503145+0000 front 2024-09-13T18:35:41.503165+0000 (oldest deadline 2024-09-13T18:36:07.403215+0000) 2024-09-13T18:36:15.119 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:14 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:14.825+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:45.793799+0000 front 2024-09-13T18:35:45.793857+0000 (oldest deadline 2024-09-13T18:36:07.493982+0000) 2024-09-13T18:36:15.220 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 13 18:36:14 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-6[58262]: 2024-09-13T18:36:14.960+0000 7f9067057640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:42.386572+0000 front 2024-09-13T18:35:42.386641+0000 (oldest deadline 2024-09-13T18:36:07.686627+0000) 2024-09-13T18:36:15.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:14 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:14.828+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.588828+0000 front 2024-09-13T18:35:34.588857+0000 (oldest deadline 2024-09-13T18:35:59.288937+0000) 2024-09-13T18:36:15.221 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:14 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:14.828+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.790933+0000 front 2024-09-13T18:35:43.791007+0000 (oldest deadline 2024-09-13T18:36:10.791344+0000) 2024-09-13T18:36:15.619 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:15 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:15.194+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:15.619 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:15 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:15.194+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.824009+0000 front 2024-09-13T18:35:43.824043+0000 (oldest deadline 2024-09-13T18:36:07.324119+0000) 2024-09-13T18:36:15.694 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:15 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:15.380+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:15.695 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:15 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:15.530+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.701894+0000 front 2024-09-13T18:35:35.701959+0000 (oldest deadline 2024-09-13T18:36:01.502502+0000) 2024-09-13T18:36:15.695 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:15 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:15.530+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:41.503145+0000 front 2024-09-13T18:35:41.503165+0000 (oldest deadline 2024-09-13T18:36:07.403215+0000) 2024-09-13T18:36:15.695 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 13 18:36:15 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-5[46242]: 2024-09-13T18:36:15.325+0000 7f0e16251640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.267471+0000 front 2024-09-13T18:35:35.267456+0000 (oldest deadline 2024-09-13T18:35:58.167332+0000) 2024-09-13T18:36:15.695 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 13 18:36:15 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-5[46242]: 2024-09-13T18:36:15.325+0000 7f0e16251640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:45.169068+0000 front 2024-09-13T18:35:45.168939+0000 (oldest deadline 2024-09-13T18:36:11.069051+0000) 2024-09-13T18:36:16.119 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:15 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:15.866+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:45.793799+0000 front 2024-09-13T18:35:45.793857+0000 (oldest deadline 2024-09-13T18:36:07.493982+0000) 2024-09-13T18:36:16.119 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:15 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:15.866+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.194892+0000 front 2024-09-13T18:35:52.194937+0000 (oldest deadline 2024-09-13T18:36:15.695008+0000) 2024-09-13T18:36:16.220 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 13 18:36:15 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-6[58262]: 2024-09-13T18:36:15.947+0000 7f9067057640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:42.386572+0000 front 2024-09-13T18:35:42.386641+0000 (oldest deadline 2024-09-13T18:36:07.686627+0000) 2024-09-13T18:36:16.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:15 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:15.872+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.588828+0000 front 2024-09-13T18:35:34.588857+0000 (oldest deadline 2024-09-13T18:35:59.288937+0000) 2024-09-13T18:36:16.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:15 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:15.872+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.790933+0000 front 2024-09-13T18:35:43.791007+0000 (oldest deadline 2024-09-13T18:36:10.791344+0000) 2024-09-13T18:36:16.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:15 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:15.872+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:50.791852+0000 front 2024-09-13T18:35:50.791998+0000 (oldest deadline 2024-09-13T18:36:15.491911+0000) 2024-09-13T18:36:16.619 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:16 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:16.147+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:16.619 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:16 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:16.147+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.824009+0000 front 2024-09-13T18:35:43.824043+0000 (oldest deadline 2024-09-13T18:36:07.324119+0000) 2024-09-13T18:36:16.619 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:16 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:16.147+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.025152+0000 front 2024-09-13T18:35:52.025201+0000 (oldest deadline 2024-09-13T18:36:15.525034+0000) 2024-09-13T18:36:16.694 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 13 18:36:16 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-5[46242]: 2024-09-13T18:36:16.352+0000 7f0e16251640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.267471+0000 front 2024-09-13T18:35:35.267456+0000 (oldest deadline 2024-09-13T18:35:58.167332+0000) 2024-09-13T18:36:16.695 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 13 18:36:16 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-5[46242]: 2024-09-13T18:36:16.352+0000 7f0e16251640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:45.169068+0000 front 2024-09-13T18:35:45.168939+0000 (oldest deadline 2024-09-13T18:36:11.069051+0000) 2024-09-13T18:36:16.695 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:16 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:16.331+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:16.695 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:16 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:16.331+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.187910+0000 front 2024-09-13T18:35:52.187860+0000 (oldest deadline 2024-09-13T18:36:16.287954+0000) 2024-09-13T18:36:16.695 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:16 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:16.503+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.701894+0000 front 2024-09-13T18:35:35.701959+0000 (oldest deadline 2024-09-13T18:36:01.502502+0000) 2024-09-13T18:36:16.695 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:16 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:16.503+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:41.503145+0000 front 2024-09-13T18:35:41.503165+0000 (oldest deadline 2024-09-13T18:36:07.403215+0000) 2024-09-13T18:36:16.695 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:16 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:16.503+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:51.504425+0000 front 2024-09-13T18:35:51.504551+0000 (oldest deadline 2024-09-13T18:36:15.604567+0000) 2024-09-13T18:36:17.182 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:16 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:16.894+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:45.793799+0000 front 2024-09-13T18:35:45.793857+0000 (oldest deadline 2024-09-13T18:36:07.493982+0000) 2024-09-13T18:36:17.183 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:16 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:16.894+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.194892+0000 front 2024-09-13T18:35:52.194937+0000 (oldest deadline 2024-09-13T18:36:15.695008+0000) 2024-09-13T18:36:17.220 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 13 18:36:16 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-6[58262]: 2024-09-13T18:36:16.973+0000 7f9067057640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:42.386572+0000 front 2024-09-13T18:35:42.386641+0000 (oldest deadline 2024-09-13T18:36:07.686627+0000) 2024-09-13T18:36:17.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:16 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:16.907+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.588828+0000 front 2024-09-13T18:35:34.588857+0000 (oldest deadline 2024-09-13T18:35:59.288937+0000) 2024-09-13T18:36:17.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:16 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:16.907+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.790933+0000 front 2024-09-13T18:35:43.791007+0000 (oldest deadline 2024-09-13T18:36:10.791344+0000) 2024-09-13T18:36:17.221 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:16 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:16.907+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:50.791852+0000 front 2024-09-13T18:35:50.791998+0000 (oldest deadline 2024-09-13T18:36:15.491911+0000) 2024-09-13T18:36:17.619 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:17 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:17.180+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:17.619 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:17 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:17.180+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.824009+0000 front 2024-09-13T18:35:43.824043+0000 (oldest deadline 2024-09-13T18:36:07.324119+0000) 2024-09-13T18:36:17.619 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:17 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:17.180+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.025152+0000 front 2024-09-13T18:35:52.025201+0000 (oldest deadline 2024-09-13T18:36:15.525034+0000) 2024-09-13T18:36:17.694 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:17 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:17.514+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.701894+0000 front 2024-09-13T18:35:35.701959+0000 (oldest deadline 2024-09-13T18:36:01.502502+0000) 2024-09-13T18:36:17.694 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:17 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:17.514+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:41.503145+0000 front 2024-09-13T18:35:41.503165+0000 (oldest deadline 2024-09-13T18:36:07.403215+0000) 2024-09-13T18:36:17.695 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:17 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:17.514+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:51.504425+0000 front 2024-09-13T18:35:51.504551+0000 (oldest deadline 2024-09-13T18:36:15.604567+0000) 2024-09-13T18:36:17.695 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:17 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:17.305+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:17.695 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:17 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:17.305+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.187910+0000 front 2024-09-13T18:35:52.187860+0000 (oldest deadline 2024-09-13T18:36:16.287954+0000) 2024-09-13T18:36:17.695 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 13 18:36:17 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-5[46242]: 2024-09-13T18:36:17.385+0000 7f0e16251640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.267471+0000 front 2024-09-13T18:35:35.267456+0000 (oldest deadline 2024-09-13T18:35:58.167332+0000) 2024-09-13T18:36:17.695 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 13 18:36:17 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-5[46242]: 2024-09-13T18:36:17.385+0000 7f0e16251640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:45.169068+0000 front 2024-09-13T18:35:45.168939+0000 (oldest deadline 2024-09-13T18:36:11.069051+0000) 2024-09-13T18:36:17.695 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 13 18:36:17 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-5[46242]: 2024-09-13T18:36:17.385+0000 7f0e16251640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:53.369939+0000 front 2024-09-13T18:35:53.370057+0000 (oldest deadline 2024-09-13T18:36:17.370364+0000) 2024-09-13T18:36:18.220 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 13 18:36:17 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-6[58262]: 2024-09-13T18:36:17.988+0000 7f9067057640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:42.386572+0000 front 2024-09-13T18:35:42.386641+0000 (oldest deadline 2024-09-13T18:36:07.686627+0000) 2024-09-13T18:36:18.221 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:17 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:17.937+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.588828+0000 front 2024-09-13T18:35:34.588857+0000 (oldest deadline 2024-09-13T18:35:59.288937+0000) 2024-09-13T18:36:18.221 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:17 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:17.937+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.790933+0000 front 2024-09-13T18:35:43.791007+0000 (oldest deadline 2024-09-13T18:36:10.791344+0000) 2024-09-13T18:36:18.221 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:17 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:17.937+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:50.791852+0000 front 2024-09-13T18:35:50.791998+0000 (oldest deadline 2024-09-13T18:36:15.491911+0000) 2024-09-13T18:36:18.369 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:17 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:17.905+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:45.793799+0000 front 2024-09-13T18:35:45.793857+0000 (oldest deadline 2024-09-13T18:36:07.493982+0000) 2024-09-13T18:36:18.369 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:17 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:17.905+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.194892+0000 front 2024-09-13T18:35:52.194937+0000 (oldest deadline 2024-09-13T18:36:15.695008+0000) 2024-09-13T18:36:18.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:18 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:18.152+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:18.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:18 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:18.152+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.824009+0000 front 2024-09-13T18:35:43.824043+0000 (oldest deadline 2024-09-13T18:36:07.324119+0000) 2024-09-13T18:36:18.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:18 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:18.152+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.025152+0000 front 2024-09-13T18:35:52.025201+0000 (oldest deadline 2024-09-13T18:36:15.525034+0000) 2024-09-13T18:36:18.694 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:18 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:18.493+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.701894+0000 front 2024-09-13T18:35:35.701959+0000 (oldest deadline 2024-09-13T18:36:01.502502+0000) 2024-09-13T18:36:18.695 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:18 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:18.493+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:41.503145+0000 front 2024-09-13T18:35:41.503165+0000 (oldest deadline 2024-09-13T18:36:07.403215+0000) 2024-09-13T18:36:18.695 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:18 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:18.493+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:51.504425+0000 front 2024-09-13T18:35:51.504551+0000 (oldest deadline 2024-09-13T18:36:15.604567+0000) 2024-09-13T18:36:18.695 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 13 18:36:18 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-5[46242]: 2024-09-13T18:36:18.411+0000 7f0e16251640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.267471+0000 front 2024-09-13T18:35:35.267456+0000 (oldest deadline 2024-09-13T18:35:58.167332+0000) 2024-09-13T18:36:18.695 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 13 18:36:18 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-5[46242]: 2024-09-13T18:36:18.411+0000 7f0e16251640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:45.169068+0000 front 2024-09-13T18:35:45.168939+0000 (oldest deadline 2024-09-13T18:36:11.069051+0000) 2024-09-13T18:36:18.695 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 13 18:36:18 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-5[46242]: 2024-09-13T18:36:18.411+0000 7f0e16251640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:53.369939+0000 front 2024-09-13T18:35:53.370057+0000 (oldest deadline 2024-09-13T18:36:17.370364+0000) 2024-09-13T18:36:18.695 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:18 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:18.304+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:18.696 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:18 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:18.304+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.187910+0000 front 2024-09-13T18:35:52.187860+0000 (oldest deadline 2024-09-13T18:36:16.287954+0000) 2024-09-13T18:36:19.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:18 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:18.921+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.588828+0000 front 2024-09-13T18:35:34.588857+0000 (oldest deadline 2024-09-13T18:35:59.288937+0000) 2024-09-13T18:36:19.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:18 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:18.921+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.790933+0000 front 2024-09-13T18:35:43.791007+0000 (oldest deadline 2024-09-13T18:36:10.791344+0000) 2024-09-13T18:36:19.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:18 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:18.921+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:50.791852+0000 front 2024-09-13T18:35:50.791998+0000 (oldest deadline 2024-09-13T18:36:15.491911+0000) 2024-09-13T18:36:19.221 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 13 18:36:18 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-6[58262]: 2024-09-13T18:36:18.985+0000 7f9067057640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:42.386572+0000 front 2024-09-13T18:35:42.386641+0000 (oldest deadline 2024-09-13T18:36:07.686627+0000) 2024-09-13T18:36:19.221 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 13 18:36:18 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-6[58262]: 2024-09-13T18:36:18.985+0000 7f9067057640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:51.088149+0000 front 2024-09-13T18:35:51.088198+0000 (oldest deadline 2024-09-13T18:36:18.088642+0000) 2024-09-13T18:36:19.369 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:18 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:18.940+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:45.793799+0000 front 2024-09-13T18:35:45.793857+0000 (oldest deadline 2024-09-13T18:36:07.493982+0000) 2024-09-13T18:36:19.370 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:18 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:18.940+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.194892+0000 front 2024-09-13T18:35:52.194937+0000 (oldest deadline 2024-09-13T18:36:15.695008+0000) 2024-09-13T18:36:19.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:19 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:19.137+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:19.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:19 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:19.137+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.824009+0000 front 2024-09-13T18:35:43.824043+0000 (oldest deadline 2024-09-13T18:36:07.324119+0000) 2024-09-13T18:36:19.371 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:19 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:19.137+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.025152+0000 front 2024-09-13T18:35:52.025201+0000 (oldest deadline 2024-09-13T18:36:15.525034+0000) 2024-09-13T18:36:19.596 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:19 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:19.280+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:19.596 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:19 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:19.280+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.187910+0000 front 2024-09-13T18:35:52.187860+0000 (oldest deadline 2024-09-13T18:36:16.287954+0000) 2024-09-13T18:36:19.596 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:19 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:19.456+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.701894+0000 front 2024-09-13T18:35:35.701959+0000 (oldest deadline 2024-09-13T18:36:01.502502+0000) 2024-09-13T18:36:19.597 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:19 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:19.456+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:41.503145+0000 front 2024-09-13T18:35:41.503165+0000 (oldest deadline 2024-09-13T18:36:07.403215+0000) 2024-09-13T18:36:19.597 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:19 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:19.456+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:51.504425+0000 front 2024-09-13T18:35:51.504551+0000 (oldest deadline 2024-09-13T18:36:15.604567+0000) 2024-09-13T18:36:20.195 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 13 18:36:20 smithi204 podman[71383]: 2024-09-13 18:36:20.096034039 +0000 UTC m=+6.152403843 container died d279a8bb0f21143a677872ace754d9c55c206b9e3ab6f54263d2a8562f1d2ab3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-5, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-13T18:36:20.220 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 13 18:36:19 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-6[58262]: 2024-09-13T18:36:19.944+0000 7f9067057640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:42.386572+0000 front 2024-09-13T18:35:42.386641+0000 (oldest deadline 2024-09-13T18:36:07.686627+0000) 2024-09-13T18:36:20.220 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 13 18:36:19 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-6[58262]: 2024-09-13T18:36:19.944+0000 7f9067057640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:51.088149+0000 front 2024-09-13T18:35:51.088198+0000 (oldest deadline 2024-09-13T18:36:18.088642+0000) 2024-09-13T18:36:20.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:19 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:19.938+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.588828+0000 front 2024-09-13T18:35:34.588857+0000 (oldest deadline 2024-09-13T18:35:59.288937+0000) 2024-09-13T18:36:20.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:19 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:19.938+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.790933+0000 front 2024-09-13T18:35:43.791007+0000 (oldest deadline 2024-09-13T18:36:10.791344+0000) 2024-09-13T18:36:20.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:19 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:19.938+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:50.791852+0000 front 2024-09-13T18:35:50.791998+0000 (oldest deadline 2024-09-13T18:36:15.491911+0000) 2024-09-13T18:36:20.369 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:19 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:19.964+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:45.793799+0000 front 2024-09-13T18:35:45.793857+0000 (oldest deadline 2024-09-13T18:36:07.493982+0000) 2024-09-13T18:36:20.369 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:19 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:19.964+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.194892+0000 front 2024-09-13T18:35:52.194937+0000 (oldest deadline 2024-09-13T18:36:15.695008+0000) 2024-09-13T18:36:20.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:20 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:20.156+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:20.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:20 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:20.156+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.824009+0000 front 2024-09-13T18:35:43.824043+0000 (oldest deadline 2024-09-13T18:36:07.324119+0000) 2024-09-13T18:36:20.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:20 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:20.156+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.025152+0000 front 2024-09-13T18:35:52.025201+0000 (oldest deadline 2024-09-13T18:36:15.525034+0000) 2024-09-13T18:36:20.694 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 13 18:36:20 smithi204 podman[71383]: 2024-09-13 18:36:20.485667664 +0000 UTC m=+6.542037459 container cleanup d279a8bb0f21143a677872ace754d9c55c206b9e3ab6f54263d2a8562f1d2ab3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-5, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083) 2024-09-13T18:36:20.694 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 13 18:36:20 smithi204 bash[71383]: ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-5 2024-09-13T18:36:20.695 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:20 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:20.328+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:20.695 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:20 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:20.328+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.187910+0000 front 2024-09-13T18:35:52.187860+0000 (oldest deadline 2024-09-13T18:36:16.287954+0000) 2024-09-13T18:36:20.695 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:20 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:20.474+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.701894+0000 front 2024-09-13T18:35:35.701959+0000 (oldest deadline 2024-09-13T18:36:01.502502+0000) 2024-09-13T18:36:20.695 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:20 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:20.474+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:41.503145+0000 front 2024-09-13T18:35:41.503165+0000 (oldest deadline 2024-09-13T18:36:07.403215+0000) 2024-09-13T18:36:20.695 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:20 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:20.474+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:51.504425+0000 front 2024-09-13T18:35:51.504551+0000 (oldest deadline 2024-09-13T18:36:15.604567+0000) 2024-09-13T18:36:21.164 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 13 18:36:20 smithi204 podman[71407]: 2024-09-13 18:36:20.820762927 +0000 UTC m=+0.722613856 container remove d279a8bb0f21143a677872ace754d9c55c206b9e3ab6f54263d2a8562f1d2ab3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T18:36:21.220 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 13 18:36:20 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-6[58262]: 2024-09-13T18:36:20.955+0000 7f9067057640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:42.386572+0000 front 2024-09-13T18:35:42.386641+0000 (oldest deadline 2024-09-13T18:36:07.686627+0000) 2024-09-13T18:36:21.220 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 13 18:36:20 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-6[58262]: 2024-09-13T18:36:20.955+0000 7f9067057640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:51.088149+0000 front 2024-09-13T18:35:51.088198+0000 (oldest deadline 2024-09-13T18:36:18.088642+0000) 2024-09-13T18:36:21.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:20 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:20.916+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.588828+0000 front 2024-09-13T18:35:34.588857+0000 (oldest deadline 2024-09-13T18:35:59.288937+0000) 2024-09-13T18:36:21.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:20 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:20.916+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.790933+0000 front 2024-09-13T18:35:43.791007+0000 (oldest deadline 2024-09-13T18:36:10.791344+0000) 2024-09-13T18:36:21.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:20 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:20.916+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:50.791852+0000 front 2024-09-13T18:35:50.791998+0000 (oldest deadline 2024-09-13T18:36:15.491911+0000) 2024-09-13T18:36:21.369 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:20 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:20.918+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:45.793799+0000 front 2024-09-13T18:35:45.793857+0000 (oldest deadline 2024-09-13T18:36:07.493982+0000) 2024-09-13T18:36:21.369 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:20 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:20.918+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.194892+0000 front 2024-09-13T18:35:52.194937+0000 (oldest deadline 2024-09-13T18:36:15.695008+0000) 2024-09-13T18:36:21.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:21 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:21.152+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:21.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:21 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:21.152+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.824009+0000 front 2024-09-13T18:35:43.824043+0000 (oldest deadline 2024-09-13T18:36:07.324119+0000) 2024-09-13T18:36:21.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:21 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:21.152+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.025152+0000 front 2024-09-13T18:35:52.025201+0000 (oldest deadline 2024-09-13T18:36:15.525034+0000) 2024-09-13T18:36:21.444 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 13 18:36:21 smithi204 podman[71497]: 2024-09-13 18:36:21.065173446 +0000 UTC m=+0.016213518 image pull 8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256 2024-09-13T18:36:21.445 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 13 18:36:21 smithi204 podman[71497]: 2024-09-13 18:36:21.188746556 +0000 UTC m=+0.139786663 container create 3030df0434ba248cd0441bb44ad1ec2600a9f6df04d3df4dd5d90aa290578075 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-13T18:36:21.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:21 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:21.281+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:21.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:21 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:21.281+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.187910+0000 front 2024-09-13T18:35:52.187860+0000 (oldest deadline 2024-09-13T18:36:16.287954+0000) 2024-09-13T18:36:21.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:21 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:21.480+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.701894+0000 front 2024-09-13T18:35:35.701959+0000 (oldest deadline 2024-09-13T18:36:01.502502+0000) 2024-09-13T18:36:21.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:21 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:21.480+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:41.503145+0000 front 2024-09-13T18:35:41.503165+0000 (oldest deadline 2024-09-13T18:36:07.403215+0000) 2024-09-13T18:36:21.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:21 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:21.480+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:51.504425+0000 front 2024-09-13T18:35:51.504551+0000 (oldest deadline 2024-09-13T18:36:15.604567+0000) 2024-09-13T18:36:21.945 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 13 18:36:21 smithi204 podman[71497]: 2024-09-13 18:36:21.449486924 +0000 UTC m=+0.400527002 container init 3030df0434ba248cd0441bb44ad1ec2600a9f6df04d3df4dd5d90aa290578075 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083) 2024-09-13T18:36:21.945 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 13 18:36:21 smithi204 podman[71497]: 2024-09-13 18:36:21.452802474 +0000 UTC m=+0.403842540 container start 3030df0434ba248cd0441bb44ad1ec2600a9f6df04d3df4dd5d90aa290578075 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-13T18:36:21.945 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 13 18:36:21 smithi204 podman[71497]: 2024-09-13 18:36:21.558045846 +0000 UTC m=+0.509085922 container attach 3030df0434ba248cd0441bb44ad1ec2600a9f6df04d3df4dd5d90aa290578075 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-13T18:36:21.945 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 13 18:36:21 smithi204 podman[71497]: 2024-09-13 18:36:21.637158028 +0000 UTC m=+0.588198099 container died 3030df0434ba248cd0441bb44ad1ec2600a9f6df04d3df4dd5d90aa290578075 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-13T18:36:22.182 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:21 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:21.915+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:45.793799+0000 front 2024-09-13T18:35:45.793857+0000 (oldest deadline 2024-09-13T18:36:07.493982+0000) 2024-09-13T18:36:22.182 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:21 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:21.915+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.194892+0000 front 2024-09-13T18:35:52.194937+0000 (oldest deadline 2024-09-13T18:36:15.695008+0000) 2024-09-13T18:36:22.220 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 13 18:36:21 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-6[58262]: 2024-09-13T18:36:21.967+0000 7f9067057640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:42.386572+0000 front 2024-09-13T18:35:42.386641+0000 (oldest deadline 2024-09-13T18:36:07.686627+0000) 2024-09-13T18:36:22.220 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 13 18:36:21 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-6[58262]: 2024-09-13T18:36:21.967+0000 7f9067057640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:51.088149+0000 front 2024-09-13T18:35:51.088198+0000 (oldest deadline 2024-09-13T18:36:18.088642+0000) 2024-09-13T18:36:22.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:21 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:21.915+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.588828+0000 front 2024-09-13T18:35:34.588857+0000 (oldest deadline 2024-09-13T18:35:59.288937+0000) 2024-09-13T18:36:22.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:21 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:21.915+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.790933+0000 front 2024-09-13T18:35:43.791007+0000 (oldest deadline 2024-09-13T18:36:10.791344+0000) 2024-09-13T18:36:22.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:21 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:21.915+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:50.791852+0000 front 2024-09-13T18:35:50.791998+0000 (oldest deadline 2024-09-13T18:36:15.491911+0000) 2024-09-13T18:36:22.263 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@osd.5.service' 2024-09-13T18:36:22.288 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:22 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:22.244+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:22.288 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:22 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:22.244+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.187910+0000 front 2024-09-13T18:35:52.187860+0000 (oldest deadline 2024-09-13T18:36:16.287954+0000) 2024-09-13T18:36:22.288 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 13 18:36:22 smithi204 podman[71521]: 2024-09-13 18:36:22.237016656 +0000 UTC m=+0.589477857 container remove 3030df0434ba248cd0441bb44ad1ec2600a9f6df04d3df4dd5d90aa290578075 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-5-deactivate, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-13T18:36:22.289 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 13 18:36:22 smithi204 systemd[1]: ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@osd.5.service: Deactivated successfully. 2024-09-13T18:36:22.289 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 13 18:36:22 smithi204 systemd[1]: Stopped Ceph osd.5 for 97e7edf2-71fb-11ef-bceb-c7b262605968. 2024-09-13T18:36:22.289 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 13 18:36:22 smithi204 systemd[1]: ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@osd.5.service: Consumed 17min 6.231s CPU time. 2024-09-13T18:36:22.305 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T18:36:22.305 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-13T18:36:22.306 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-13T18:36:22.306 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@osd.6 2024-09-13T18:36:22.587 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 13 18:36:22 smithi114 systemd[1]: Stopping Ceph osd.6 for 97e7edf2-71fb-11ef-bceb-c7b262605968... 2024-09-13T18:36:22.619 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:22 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:22.179+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:22.619 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:22 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:22.179+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.824009+0000 front 2024-09-13T18:35:43.824043+0000 (oldest deadline 2024-09-13T18:36:07.324119+0000) 2024-09-13T18:36:22.619 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:22 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:22.179+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.025152+0000 front 2024-09-13T18:35:52.025201+0000 (oldest deadline 2024-09-13T18:36:15.525034+0000) 2024-09-13T18:36:22.694 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:22 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:22.466+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.701894+0000 front 2024-09-13T18:35:35.701959+0000 (oldest deadline 2024-09-13T18:36:01.502502+0000) 2024-09-13T18:36:22.694 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:22 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:22.466+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:41.503145+0000 front 2024-09-13T18:35:41.503165+0000 (oldest deadline 2024-09-13T18:36:07.403215+0000) 2024-09-13T18:36:22.694 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:22 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:22.466+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:51.504425+0000 front 2024-09-13T18:35:51.504551+0000 (oldest deadline 2024-09-13T18:36:15.604567+0000) 2024-09-13T18:36:22.958 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 13 18:36:22 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-6[58262]: 2024-09-13T18:36:22.585+0000 7f906ea4b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T18:36:22.959 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 13 18:36:22 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-6[58262]: 2024-09-13T18:36:22.585+0000 7f906ea4b640 -1 osd.6 111 *** Got signal Terminated *** 2024-09-13T18:36:22.959 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 13 18:36:22 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-6[58262]: 2024-09-13T18:36:22.585+0000 7f906ea4b640 -1 osd.6 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T18:36:23.220 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 13 18:36:22 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-6[58262]: 2024-09-13T18:36:22.961+0000 7f9067057640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:42.386572+0000 front 2024-09-13T18:35:42.386641+0000 (oldest deadline 2024-09-13T18:36:07.686627+0000) 2024-09-13T18:36:23.220 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 13 18:36:22 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-6[58262]: 2024-09-13T18:36:22.961+0000 7f9067057640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:51.088149+0000 front 2024-09-13T18:35:51.088198+0000 (oldest deadline 2024-09-13T18:36:18.088642+0000) 2024-09-13T18:36:23.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:22 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:22.956+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.588828+0000 front 2024-09-13T18:35:34.588857+0000 (oldest deadline 2024-09-13T18:35:59.288937+0000) 2024-09-13T18:36:23.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:22 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:22.956+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.790933+0000 front 2024-09-13T18:35:43.791007+0000 (oldest deadline 2024-09-13T18:36:10.791344+0000) 2024-09-13T18:36:23.221 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:22 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:22.956+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:50.791852+0000 front 2024-09-13T18:35:50.791998+0000 (oldest deadline 2024-09-13T18:36:15.491911+0000) 2024-09-13T18:36:23.369 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:22 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:22.951+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:45.793799+0000 front 2024-09-13T18:35:45.793857+0000 (oldest deadline 2024-09-13T18:36:07.493982+0000) 2024-09-13T18:36:23.369 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:22 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:22.951+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.194892+0000 front 2024-09-13T18:35:52.194937+0000 (oldest deadline 2024-09-13T18:36:15.695008+0000) 2024-09-13T18:36:23.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:23 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:23.183+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:23.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:23 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:23.183+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.824009+0000 front 2024-09-13T18:35:43.824043+0000 (oldest deadline 2024-09-13T18:36:07.324119+0000) 2024-09-13T18:36:23.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:23 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:23.183+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.025152+0000 front 2024-09-13T18:35:52.025201+0000 (oldest deadline 2024-09-13T18:36:15.525034+0000) 2024-09-13T18:36:23.490 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:23 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:23.208+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:23.490 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:23 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:23.208+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.187910+0000 front 2024-09-13T18:35:52.187860+0000 (oldest deadline 2024-09-13T18:36:16.287954+0000) 2024-09-13T18:36:23.944 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:23 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:23.488+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.701894+0000 front 2024-09-13T18:35:35.701959+0000 (oldest deadline 2024-09-13T18:36:01.502502+0000) 2024-09-13T18:36:23.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:23 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:23.488+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:41.503145+0000 front 2024-09-13T18:35:41.503165+0000 (oldest deadline 2024-09-13T18:36:07.403215+0000) 2024-09-13T18:36:23.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:23 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:23.488+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:51.504425+0000 front 2024-09-13T18:35:51.504551+0000 (oldest deadline 2024-09-13T18:36:15.604567+0000) 2024-09-13T18:36:24.220 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 13 18:36:23 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-6[58262]: 2024-09-13T18:36:23.986+0000 7f9067057640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:42.386572+0000 front 2024-09-13T18:35:42.386641+0000 (oldest deadline 2024-09-13T18:36:07.686627+0000) 2024-09-13T18:36:24.220 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 13 18:36:23 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-6[58262]: 2024-09-13T18:36:23.986+0000 7f9067057640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:51.088149+0000 front 2024-09-13T18:35:51.088198+0000 (oldest deadline 2024-09-13T18:36:18.088642+0000) 2024-09-13T18:36:24.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:23 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:23.951+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.588828+0000 front 2024-09-13T18:35:34.588857+0000 (oldest deadline 2024-09-13T18:35:59.288937+0000) 2024-09-13T18:36:24.221 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:23 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:23.951+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.790933+0000 front 2024-09-13T18:35:43.791007+0000 (oldest deadline 2024-09-13T18:36:10.791344+0000) 2024-09-13T18:36:24.221 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:23 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:23.951+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:50.791852+0000 front 2024-09-13T18:35:50.791998+0000 (oldest deadline 2024-09-13T18:36:15.491911+0000) 2024-09-13T18:36:24.369 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:23 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:23.975+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:45.793799+0000 front 2024-09-13T18:35:45.793857+0000 (oldest deadline 2024-09-13T18:36:07.493982+0000) 2024-09-13T18:36:24.369 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:23 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:23.975+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.194892+0000 front 2024-09-13T18:35:52.194937+0000 (oldest deadline 2024-09-13T18:36:15.695008+0000) 2024-09-13T18:36:24.369 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:23 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:23.975+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:35:59.796269+0000 front 2024-09-13T18:35:59.796314+0000 (oldest deadline 2024-09-13T18:36:23.796996+0000) 2024-09-13T18:36:24.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:24 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:24.161+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:24.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:24 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:24.161+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.824009+0000 front 2024-09-13T18:35:43.824043+0000 (oldest deadline 2024-09-13T18:36:07.324119+0000) 2024-09-13T18:36:24.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:24 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:24.161+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.025152+0000 front 2024-09-13T18:35:52.025201+0000 (oldest deadline 2024-09-13T18:36:15.525034+0000) 2024-09-13T18:36:24.531 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:24 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:24.232+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:24.531 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:24 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:24.232+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.187910+0000 front 2024-09-13T18:35:52.187860+0000 (oldest deadline 2024-09-13T18:36:16.287954+0000) 2024-09-13T18:36:24.944 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:24 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:24.529+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.701894+0000 front 2024-09-13T18:35:35.701959+0000 (oldest deadline 2024-09-13T18:36:01.502502+0000) 2024-09-13T18:36:24.944 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:24 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:24.529+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:41.503145+0000 front 2024-09-13T18:35:41.503165+0000 (oldest deadline 2024-09-13T18:36:07.403215+0000) 2024-09-13T18:36:24.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:24 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:24.529+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:51.504425+0000 front 2024-09-13T18:35:51.504551+0000 (oldest deadline 2024-09-13T18:36:15.604567+0000) 2024-09-13T18:36:25.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:24 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:24.902+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.588828+0000 front 2024-09-13T18:35:34.588857+0000 (oldest deadline 2024-09-13T18:35:59.288937+0000) 2024-09-13T18:36:25.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:24 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:24.902+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.790933+0000 front 2024-09-13T18:35:43.791007+0000 (oldest deadline 2024-09-13T18:36:10.791344+0000) 2024-09-13T18:36:25.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:24 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:24.902+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:50.791852+0000 front 2024-09-13T18:35:50.791998+0000 (oldest deadline 2024-09-13T18:36:15.491911+0000) 2024-09-13T18:36:25.220 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 13 18:36:25 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-6[58262]: 2024-09-13T18:36:25.032+0000 7f9067057640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:42.386572+0000 front 2024-09-13T18:35:42.386641+0000 (oldest deadline 2024-09-13T18:36:07.686627+0000) 2024-09-13T18:36:25.220 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 13 18:36:25 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-6[58262]: 2024-09-13T18:36:25.032+0000 7f9067057640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:51.088149+0000 front 2024-09-13T18:35:51.088198+0000 (oldest deadline 2024-09-13T18:36:18.088642+0000) 2024-09-13T18:36:25.369 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:25 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:25.022+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:45.793799+0000 front 2024-09-13T18:35:45.793857+0000 (oldest deadline 2024-09-13T18:36:07.493982+0000) 2024-09-13T18:36:25.369 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:25 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:25.022+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.194892+0000 front 2024-09-13T18:35:52.194937+0000 (oldest deadline 2024-09-13T18:36:15.695008+0000) 2024-09-13T18:36:25.369 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:25 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:25.022+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:35:59.796269+0000 front 2024-09-13T18:35:59.796314+0000 (oldest deadline 2024-09-13T18:36:23.796996+0000) 2024-09-13T18:36:25.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:25 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:25.130+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:25.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:25 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:25.130+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.824009+0000 front 2024-09-13T18:35:43.824043+0000 (oldest deadline 2024-09-13T18:36:07.324119+0000) 2024-09-13T18:36:25.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:25 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:25.130+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.025152+0000 front 2024-09-13T18:35:52.025201+0000 (oldest deadline 2024-09-13T18:36:15.525034+0000) 2024-09-13T18:36:25.562 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:25 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:25.278+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:25.563 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:25 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:25.278+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.187910+0000 front 2024-09-13T18:35:52.187860+0000 (oldest deadline 2024-09-13T18:36:16.287954+0000) 2024-09-13T18:36:25.563 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:25 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:25.278+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.589086+0000 front 2024-09-13T18:36:01.588873+0000 (oldest deadline 2024-09-13T18:36:24.389897+0000) 2024-09-13T18:36:25.944 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:25 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:25.560+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.701894+0000 front 2024-09-13T18:35:35.701959+0000 (oldest deadline 2024-09-13T18:36:01.502502+0000) 2024-09-13T18:36:25.944 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:25 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:25.560+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:41.503145+0000 front 2024-09-13T18:35:41.503165+0000 (oldest deadline 2024-09-13T18:36:07.403215+0000) 2024-09-13T18:36:25.944 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:25 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:25.560+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:51.504425+0000 front 2024-09-13T18:35:51.504551+0000 (oldest deadline 2024-09-13T18:36:15.604567+0000) 2024-09-13T18:36:25.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:25 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:25.560+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:00.806201+0000 front 2024-09-13T18:36:00.806372+0000 (oldest deadline 2024-09-13T18:36:25.506104+0000) 2024-09-13T18:36:26.220 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 13 18:36:25 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-6[58262]: 2024-09-13T18:36:25.986+0000 7f9067057640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:42.386572+0000 front 2024-09-13T18:35:42.386641+0000 (oldest deadline 2024-09-13T18:36:07.686627+0000) 2024-09-13T18:36:26.220 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 13 18:36:25 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-6[58262]: 2024-09-13T18:36:25.986+0000 7f9067057640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:51.088149+0000 front 2024-09-13T18:35:51.088198+0000 (oldest deadline 2024-09-13T18:36:18.088642+0000) 2024-09-13T18:36:26.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:25 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:25.896+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.588828+0000 front 2024-09-13T18:35:34.588857+0000 (oldest deadline 2024-09-13T18:35:59.288937+0000) 2024-09-13T18:36:26.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:25 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:25.896+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.790933+0000 front 2024-09-13T18:35:43.791007+0000 (oldest deadline 2024-09-13T18:36:10.791344+0000) 2024-09-13T18:36:26.221 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:25 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:25.896+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:50.791852+0000 front 2024-09-13T18:35:50.791998+0000 (oldest deadline 2024-09-13T18:36:15.491911+0000) 2024-09-13T18:36:26.369 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:26 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:26.036+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:45.793799+0000 front 2024-09-13T18:35:45.793857+0000 (oldest deadline 2024-09-13T18:36:07.493982+0000) 2024-09-13T18:36:26.369 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:26 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:26.036+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.194892+0000 front 2024-09-13T18:35:52.194937+0000 (oldest deadline 2024-09-13T18:36:15.695008+0000) 2024-09-13T18:36:26.369 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:26 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:26.036+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:35:59.796269+0000 front 2024-09-13T18:35:59.796314+0000 (oldest deadline 2024-09-13T18:36:23.796996+0000) 2024-09-13T18:36:26.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:26 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:26.126+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:26.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:26 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:26.126+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.824009+0000 front 2024-09-13T18:35:43.824043+0000 (oldest deadline 2024-09-13T18:36:07.324119+0000) 2024-09-13T18:36:26.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:26 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:26.126+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.025152+0000 front 2024-09-13T18:35:52.025201+0000 (oldest deadline 2024-09-13T18:36:15.525034+0000) 2024-09-13T18:36:26.557 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:26 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:26.300+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:26.558 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:26 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:26.300+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.187910+0000 front 2024-09-13T18:35:52.187860+0000 (oldest deadline 2024-09-13T18:36:16.287954+0000) 2024-09-13T18:36:26.558 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:26 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:26.300+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.589086+0000 front 2024-09-13T18:36:01.588873+0000 (oldest deadline 2024-09-13T18:36:24.389897+0000) 2024-09-13T18:36:26.944 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:26 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:26.555+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.701894+0000 front 2024-09-13T18:35:35.701959+0000 (oldest deadline 2024-09-13T18:36:01.502502+0000) 2024-09-13T18:36:26.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:26 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:26.555+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:41.503145+0000 front 2024-09-13T18:35:41.503165+0000 (oldest deadline 2024-09-13T18:36:07.403215+0000) 2024-09-13T18:36:26.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:26 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:26.555+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:51.504425+0000 front 2024-09-13T18:35:51.504551+0000 (oldest deadline 2024-09-13T18:36:15.604567+0000) 2024-09-13T18:36:26.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:26 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:26.555+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:00.806201+0000 front 2024-09-13T18:36:00.806372+0000 (oldest deadline 2024-09-13T18:36:25.506104+0000) 2024-09-13T18:36:27.220 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 13 18:36:27 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-6[58262]: 2024-09-13T18:36:27.004+0000 7f9067057640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:42.386572+0000 front 2024-09-13T18:35:42.386641+0000 (oldest deadline 2024-09-13T18:36:07.686627+0000) 2024-09-13T18:36:27.220 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 13 18:36:27 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-6[58262]: 2024-09-13T18:36:27.004+0000 7f9067057640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:51.088149+0000 front 2024-09-13T18:35:51.088198+0000 (oldest deadline 2024-09-13T18:36:18.088642+0000) 2024-09-13T18:36:27.220 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 13 18:36:27 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-6[58262]: 2024-09-13T18:36:27.004+0000 7f9067057640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:35:58.088915+0000 front 2024-09-13T18:35:58.088931+0000 (oldest deadline 2024-09-13T18:36:26.889452+0000) 2024-09-13T18:36:27.221 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:26 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:26.869+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.588828+0000 front 2024-09-13T18:35:34.588857+0000 (oldest deadline 2024-09-13T18:35:59.288937+0000) 2024-09-13T18:36:27.221 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:26 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:26.869+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.790933+0000 front 2024-09-13T18:35:43.791007+0000 (oldest deadline 2024-09-13T18:36:10.791344+0000) 2024-09-13T18:36:27.221 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:26 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:26.869+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:50.791852+0000 front 2024-09-13T18:35:50.791998+0000 (oldest deadline 2024-09-13T18:36:15.491911+0000) 2024-09-13T18:36:27.221 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:26 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:26.869+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.293330+0000 front 2024-09-13T18:36:01.293344+0000 (oldest deadline 2024-09-13T18:36:25.993496+0000) 2024-09-13T18:36:27.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:27 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:27.096+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:27.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:27 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:27.096+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.824009+0000 front 2024-09-13T18:35:43.824043+0000 (oldest deadline 2024-09-13T18:36:07.324119+0000) 2024-09-13T18:36:27.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:27 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:27.096+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.025152+0000 front 2024-09-13T18:35:52.025201+0000 (oldest deadline 2024-09-13T18:36:15.525034+0000) 2024-09-13T18:36:27.370 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:27 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:27.045+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:45.793799+0000 front 2024-09-13T18:35:45.793857+0000 (oldest deadline 2024-09-13T18:36:07.493982+0000) 2024-09-13T18:36:27.370 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:27 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:27.045+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.194892+0000 front 2024-09-13T18:35:52.194937+0000 (oldest deadline 2024-09-13T18:36:15.695008+0000) 2024-09-13T18:36:27.370 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:27 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:27.045+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:35:59.796269+0000 front 2024-09-13T18:35:59.796314+0000 (oldest deadline 2024-09-13T18:36:23.796996+0000) 2024-09-13T18:36:27.557 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:27 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:27.271+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:27.557 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:27 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:27.271+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.187910+0000 front 2024-09-13T18:35:52.187860+0000 (oldest deadline 2024-09-13T18:36:16.287954+0000) 2024-09-13T18:36:27.557 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:27 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:27.271+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.589086+0000 front 2024-09-13T18:36:01.588873+0000 (oldest deadline 2024-09-13T18:36:24.389897+0000) 2024-09-13T18:36:27.944 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:27 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:27.555+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.701894+0000 front 2024-09-13T18:35:35.701959+0000 (oldest deadline 2024-09-13T18:36:01.502502+0000) 2024-09-13T18:36:27.944 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:27 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:27.555+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:41.503145+0000 front 2024-09-13T18:35:41.503165+0000 (oldest deadline 2024-09-13T18:36:07.403215+0000) 2024-09-13T18:36:27.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:27 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:27.555+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:51.504425+0000 front 2024-09-13T18:35:51.504551+0000 (oldest deadline 2024-09-13T18:36:15.604567+0000) 2024-09-13T18:36:27.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:27 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:27.555+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:00.806201+0000 front 2024-09-13T18:36:00.806372+0000 (oldest deadline 2024-09-13T18:36:25.506104+0000) 2024-09-13T18:36:28.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:27 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:27.857+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.588828+0000 front 2024-09-13T18:35:34.588857+0000 (oldest deadline 2024-09-13T18:35:59.288937+0000) 2024-09-13T18:36:28.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:27 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:27.857+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.790933+0000 front 2024-09-13T18:35:43.791007+0000 (oldest deadline 2024-09-13T18:36:10.791344+0000) 2024-09-13T18:36:28.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:27 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:27.857+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:50.791852+0000 front 2024-09-13T18:35:50.791998+0000 (oldest deadline 2024-09-13T18:36:15.491911+0000) 2024-09-13T18:36:28.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:27 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:27.857+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.293330+0000 front 2024-09-13T18:36:01.293344+0000 (oldest deadline 2024-09-13T18:36:25.993496+0000) 2024-09-13T18:36:28.270 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:28 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:28.013+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:45.793799+0000 front 2024-09-13T18:35:45.793857+0000 (oldest deadline 2024-09-13T18:36:07.493982+0000) 2024-09-13T18:36:28.271 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:28 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:28.013+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.194892+0000 front 2024-09-13T18:35:52.194937+0000 (oldest deadline 2024-09-13T18:36:15.695008+0000) 2024-09-13T18:36:28.271 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:28 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:28.013+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:35:59.796269+0000 front 2024-09-13T18:35:59.796314+0000 (oldest deadline 2024-09-13T18:36:23.796996+0000) 2024-09-13T18:36:28.271 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:28 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:28.144+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:28.271 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:28 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:28.144+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.824009+0000 front 2024-09-13T18:35:43.824043+0000 (oldest deadline 2024-09-13T18:36:07.324119+0000) 2024-09-13T18:36:28.271 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:28 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:28.144+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.025152+0000 front 2024-09-13T18:35:52.025201+0000 (oldest deadline 2024-09-13T18:36:15.525034+0000) 2024-09-13T18:36:28.271 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:28 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:28.144+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.326799+0000 front 2024-09-13T18:36:01.326736+0000 (oldest deadline 2024-09-13T18:36:27.226720+0000) 2024-09-13T18:36:28.533 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:28 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:28.256+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:28.533 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:28 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:28.256+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.187910+0000 front 2024-09-13T18:35:52.187860+0000 (oldest deadline 2024-09-13T18:36:16.287954+0000) 2024-09-13T18:36:28.534 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:28 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:28.256+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.589086+0000 front 2024-09-13T18:36:01.588873+0000 (oldest deadline 2024-09-13T18:36:24.389897+0000) 2024-09-13T18:36:28.588 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 13 18:36:28 smithi114 podman[111241]: 2024-09-13 18:36:28.291190656 +0000 UTC m=+5.790378529 container died ac5aa81b2a31da89318deafafa6682ee7cd21f5119a11f53d73c37779813b905 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-6, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-13T18:36:28.865 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 13 18:36:28 smithi114 podman[111241]: 2024-09-13 18:36:28.685156064 +0000 UTC m=+6.184343948 container cleanup ac5aa81b2a31da89318deafafa6682ee7cd21f5119a11f53d73c37779813b905 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-6, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-13T18:36:28.865 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 13 18:36:28 smithi114 bash[111241]: ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-6 2024-09-13T18:36:28.944 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:28 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:28.531+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.701894+0000 front 2024-09-13T18:35:35.701959+0000 (oldest deadline 2024-09-13T18:36:01.502502+0000) 2024-09-13T18:36:28.944 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:28 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:28.531+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:41.503145+0000 front 2024-09-13T18:35:41.503165+0000 (oldest deadline 2024-09-13T18:36:07.403215+0000) 2024-09-13T18:36:28.944 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:28 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:28.531+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:51.504425+0000 front 2024-09-13T18:35:51.504551+0000 (oldest deadline 2024-09-13T18:36:15.604567+0000) 2024-09-13T18:36:28.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:28 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:28.531+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:00.806201+0000 front 2024-09-13T18:36:00.806372+0000 (oldest deadline 2024-09-13T18:36:25.506104+0000) 2024-09-13T18:36:29.220 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 13 18:36:28 smithi114 podman[111257]: 2024-09-13 18:36:28.97844121 +0000 UTC m=+0.682181084 container remove ac5aa81b2a31da89318deafafa6682ee7cd21f5119a11f53d73c37779813b905 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-6, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-13T18:36:29.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:28 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:28.862+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.588828+0000 front 2024-09-13T18:35:34.588857+0000 (oldest deadline 2024-09-13T18:35:59.288937+0000) 2024-09-13T18:36:29.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:28 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:28.862+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.790933+0000 front 2024-09-13T18:35:43.791007+0000 (oldest deadline 2024-09-13T18:36:10.791344+0000) 2024-09-13T18:36:29.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:28 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:28.862+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:50.791852+0000 front 2024-09-13T18:35:50.791998+0000 (oldest deadline 2024-09-13T18:36:15.491911+0000) 2024-09-13T18:36:29.221 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:28 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:28.862+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.293330+0000 front 2024-09-13T18:36:01.293344+0000 (oldest deadline 2024-09-13T18:36:25.993496+0000) 2024-09-13T18:36:29.369 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:29 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:29.010+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:45.793799+0000 front 2024-09-13T18:35:45.793857+0000 (oldest deadline 2024-09-13T18:36:07.493982+0000) 2024-09-13T18:36:29.369 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:29 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:29.010+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.194892+0000 front 2024-09-13T18:35:52.194937+0000 (oldest deadline 2024-09-13T18:36:15.695008+0000) 2024-09-13T18:36:29.369 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:29 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:29.010+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:35:59.796269+0000 front 2024-09-13T18:35:59.796314+0000 (oldest deadline 2024-09-13T18:36:23.796996+0000) 2024-09-13T18:36:29.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:29 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:29.118+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:29.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:29 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:29.118+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.824009+0000 front 2024-09-13T18:35:43.824043+0000 (oldest deadline 2024-09-13T18:36:07.324119+0000) 2024-09-13T18:36:29.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:29 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:29.118+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.025152+0000 front 2024-09-13T18:35:52.025201+0000 (oldest deadline 2024-09-13T18:36:15.525034+0000) 2024-09-13T18:36:29.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:29 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:29.118+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.326799+0000 front 2024-09-13T18:36:01.326736+0000 (oldest deadline 2024-09-13T18:36:27.226720+0000) 2024-09-13T18:36:29.567 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 13 18:36:29 smithi114 podman[111346]: 2024-09-13 18:36:29.191825703 +0000 UTC m=+0.015975649 image pull 8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256 2024-09-13T18:36:29.567 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 13 18:36:29 smithi114 podman[111346]: 2024-09-13 18:36:29.321093443 +0000 UTC m=+0.145243421 container create 8939af6a878778ae2c074ef648a66cbf895e3bb3b891c8e1a634e89699c6345a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-6-deactivate, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-13T18:36:29.567 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 13 18:36:29 smithi114 podman[111346]: 2024-09-13 18:36:29.565158038 +0000 UTC m=+0.389307976 container init 8939af6a878778ae2c074ef648a66cbf895e3bb3b891c8e1a634e89699c6345a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True) 2024-09-13T18:36:29.579 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:29 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:29.244+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:29.579 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:29 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:29.244+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.187910+0000 front 2024-09-13T18:35:52.187860+0000 (oldest deadline 2024-09-13T18:36:16.287954+0000) 2024-09-13T18:36:29.579 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:29 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:29.244+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.589086+0000 front 2024-09-13T18:36:01.588873+0000 (oldest deadline 2024-09-13T18:36:24.389897+0000) 2024-09-13T18:36:29.853 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 13 18:36:29 smithi114 podman[111346]: 2024-09-13 18:36:29.569070716 +0000 UTC m=+0.393220655 container start 8939af6a878778ae2c074ef648a66cbf895e3bb3b891c8e1a634e89699c6345a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-13T18:36:29.853 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 13 18:36:29 smithi114 podman[111346]: 2024-09-13 18:36:29.648430425 +0000 UTC m=+0.472580373 container attach 8939af6a878778ae2c074ef648a66cbf895e3bb3b891c8e1a634e89699c6345a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-6-deactivate, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-13T18:36:29.854 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 13 18:36:29 smithi114 podman[111346]: 2024-09-13 18:36:29.749675251 +0000 UTC m=+0.573825193 container died 8939af6a878778ae2c074ef648a66cbf895e3bb3b891c8e1a634e89699c6345a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T18:36:29.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:29 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:29.576+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.701894+0000 front 2024-09-13T18:35:35.701959+0000 (oldest deadline 2024-09-13T18:36:01.502502+0000) 2024-09-13T18:36:29.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:29 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:29.576+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:41.503145+0000 front 2024-09-13T18:35:41.503165+0000 (oldest deadline 2024-09-13T18:36:07.403215+0000) 2024-09-13T18:36:29.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:29 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:29.576+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:51.504425+0000 front 2024-09-13T18:35:51.504551+0000 (oldest deadline 2024-09-13T18:36:15.604567+0000) 2024-09-13T18:36:29.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:29 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:29.576+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:00.806201+0000 front 2024-09-13T18:36:00.806372+0000 (oldest deadline 2024-09-13T18:36:25.506104+0000) 2024-09-13T18:36:30.112 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:29 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:29.850+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.588828+0000 front 2024-09-13T18:35:34.588857+0000 (oldest deadline 2024-09-13T18:35:59.288937+0000) 2024-09-13T18:36:30.112 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:29 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:29.850+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.790933+0000 front 2024-09-13T18:35:43.791007+0000 (oldest deadline 2024-09-13T18:36:10.791344+0000) 2024-09-13T18:36:30.112 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:29 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:29.850+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:50.791852+0000 front 2024-09-13T18:35:50.791998+0000 (oldest deadline 2024-09-13T18:36:15.491911+0000) 2024-09-13T18:36:30.112 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:29 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:29.850+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.293330+0000 front 2024-09-13T18:36:01.293344+0000 (oldest deadline 2024-09-13T18:36:25.993496+0000) 2024-09-13T18:36:30.369 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:30 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:30.033+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:45.793799+0000 front 2024-09-13T18:35:45.793857+0000 (oldest deadline 2024-09-13T18:36:07.493982+0000) 2024-09-13T18:36:30.369 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:30 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:30.033+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.194892+0000 front 2024-09-13T18:35:52.194937+0000 (oldest deadline 2024-09-13T18:36:15.695008+0000) 2024-09-13T18:36:30.370 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:30 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:30.033+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:35:59.796269+0000 front 2024-09-13T18:35:59.796314+0000 (oldest deadline 2024-09-13T18:36:23.796996+0000) 2024-09-13T18:36:30.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:30 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:30.142+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:30.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:30 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:30.142+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.824009+0000 front 2024-09-13T18:35:43.824043+0000 (oldest deadline 2024-09-13T18:36:07.324119+0000) 2024-09-13T18:36:30.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:30 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:30.142+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.025152+0000 front 2024-09-13T18:35:52.025201+0000 (oldest deadline 2024-09-13T18:36:15.525034+0000) 2024-09-13T18:36:30.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:30 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:30.142+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.326799+0000 front 2024-09-13T18:36:01.326736+0000 (oldest deadline 2024-09-13T18:36:27.226720+0000) 2024-09-13T18:36:30.437 DEBUG:teuthology.orchestra.run.smithi114:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@osd.6.service' 2024-09-13T18:36:30.476 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T18:36:30.476 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-13T18:36:30.476 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-13T18:36:30.477 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@osd.7 2024-09-13T18:36:30.539 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:30 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:30.201+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:30.539 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:30 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:30.201+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.187910+0000 front 2024-09-13T18:35:52.187860+0000 (oldest deadline 2024-09-13T18:36:16.287954+0000) 2024-09-13T18:36:30.539 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:30 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:30.201+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.589086+0000 front 2024-09-13T18:36:01.588873+0000 (oldest deadline 2024-09-13T18:36:24.389897+0000) 2024-09-13T18:36:30.807 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:30 smithi134 systemd[1]: Stopping Ceph osd.7 for 97e7edf2-71fb-11ef-bceb-c7b262605968... 2024-09-13T18:36:30.941 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:30 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:30.536+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.701894+0000 front 2024-09-13T18:35:35.701959+0000 (oldest deadline 2024-09-13T18:36:01.502502+0000) 2024-09-13T18:36:30.942 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:30 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:30.536+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:41.503145+0000 front 2024-09-13T18:35:41.503165+0000 (oldest deadline 2024-09-13T18:36:07.403215+0000) 2024-09-13T18:36:30.942 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:30 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:30.536+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:51.504425+0000 front 2024-09-13T18:35:51.504551+0000 (oldest deadline 2024-09-13T18:36:15.604567+0000) 2024-09-13T18:36:30.942 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:30 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:30.536+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:00.806201+0000 front 2024-09-13T18:36:00.806372+0000 (oldest deadline 2024-09-13T18:36:25.506104+0000) 2024-09-13T18:36:31.065 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:30 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:30.804+0000 7f789d7d1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T18:36:31.065 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:30 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:30.804+0000 7f789d7d1640 -1 osd.7 111 *** Got signal Terminated *** 2024-09-13T18:36:31.065 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:30 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:30.804+0000 7f789d7d1640 -1 osd.7 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T18:36:31.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:30 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:30.881+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.588828+0000 front 2024-09-13T18:35:34.588857+0000 (oldest deadline 2024-09-13T18:35:59.288937+0000) 2024-09-13T18:36:31.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:30 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:30.881+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.790933+0000 front 2024-09-13T18:35:43.791007+0000 (oldest deadline 2024-09-13T18:36:10.791344+0000) 2024-09-13T18:36:31.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:30 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:30.881+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:50.791852+0000 front 2024-09-13T18:35:50.791998+0000 (oldest deadline 2024-09-13T18:36:15.491911+0000) 2024-09-13T18:36:31.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:30 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:30.881+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.293330+0000 front 2024-09-13T18:36:01.293344+0000 (oldest deadline 2024-09-13T18:36:25.993496+0000) 2024-09-13T18:36:31.369 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:31 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:31.062+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:45.793799+0000 front 2024-09-13T18:35:45.793857+0000 (oldest deadline 2024-09-13T18:36:07.493982+0000) 2024-09-13T18:36:31.369 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:31 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:31.062+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.194892+0000 front 2024-09-13T18:35:52.194937+0000 (oldest deadline 2024-09-13T18:36:15.695008+0000) 2024-09-13T18:36:31.369 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:31 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:31.062+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:35:59.796269+0000 front 2024-09-13T18:35:59.796314+0000 (oldest deadline 2024-09-13T18:36:23.796996+0000) 2024-09-13T18:36:31.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:31 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:31.152+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:31.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:31 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:31.152+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.824009+0000 front 2024-09-13T18:35:43.824043+0000 (oldest deadline 2024-09-13T18:36:07.324119+0000) 2024-09-13T18:36:31.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:31 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:31.152+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.025152+0000 front 2024-09-13T18:35:52.025201+0000 (oldest deadline 2024-09-13T18:36:15.525034+0000) 2024-09-13T18:36:31.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:31 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:31.152+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.326799+0000 front 2024-09-13T18:36:01.326736+0000 (oldest deadline 2024-09-13T18:36:27.226720+0000) 2024-09-13T18:36:31.561 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:31 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:31.226+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:31.561 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:31 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:31.226+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.187910+0000 front 2024-09-13T18:35:52.187860+0000 (oldest deadline 2024-09-13T18:36:16.287954+0000) 2024-09-13T18:36:31.561 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:31 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:31.226+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.589086+0000 front 2024-09-13T18:36:01.588873+0000 (oldest deadline 2024-09-13T18:36:24.389897+0000) 2024-09-13T18:36:31.944 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:31 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:31.558+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.701894+0000 front 2024-09-13T18:35:35.701959+0000 (oldest deadline 2024-09-13T18:36:01.502502+0000) 2024-09-13T18:36:31.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:31 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:31.558+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:41.503145+0000 front 2024-09-13T18:35:41.503165+0000 (oldest deadline 2024-09-13T18:36:07.403215+0000) 2024-09-13T18:36:31.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:31 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:31.558+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:51.504425+0000 front 2024-09-13T18:35:51.504551+0000 (oldest deadline 2024-09-13T18:36:15.604567+0000) 2024-09-13T18:36:31.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:31 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:31.558+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:00.806201+0000 front 2024-09-13T18:36:00.806372+0000 (oldest deadline 2024-09-13T18:36:25.506104+0000) 2024-09-13T18:36:32.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:31 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:31.904+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.588828+0000 front 2024-09-13T18:35:34.588857+0000 (oldest deadline 2024-09-13T18:35:59.288937+0000) 2024-09-13T18:36:32.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:31 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:31.904+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.790933+0000 front 2024-09-13T18:35:43.791007+0000 (oldest deadline 2024-09-13T18:36:10.791344+0000) 2024-09-13T18:36:32.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:31 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:31.904+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:50.791852+0000 front 2024-09-13T18:35:50.791998+0000 (oldest deadline 2024-09-13T18:36:15.491911+0000) 2024-09-13T18:36:32.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:31 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:31.904+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.293330+0000 front 2024-09-13T18:36:01.293344+0000 (oldest deadline 2024-09-13T18:36:25.993496+0000) 2024-09-13T18:36:32.369 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:32 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:32.052+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:45.793799+0000 front 2024-09-13T18:35:45.793857+0000 (oldest deadline 2024-09-13T18:36:07.493982+0000) 2024-09-13T18:36:32.369 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:32 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:32.052+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.194892+0000 front 2024-09-13T18:35:52.194937+0000 (oldest deadline 2024-09-13T18:36:15.695008+0000) 2024-09-13T18:36:32.369 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:32 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:32.052+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:35:59.796269+0000 front 2024-09-13T18:35:59.796314+0000 (oldest deadline 2024-09-13T18:36:23.796996+0000) 2024-09-13T18:36:32.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:32 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:32.142+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:32.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:32 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:32.142+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.824009+0000 front 2024-09-13T18:35:43.824043+0000 (oldest deadline 2024-09-13T18:36:07.324119+0000) 2024-09-13T18:36:32.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:32 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:32.142+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.025152+0000 front 2024-09-13T18:35:52.025201+0000 (oldest deadline 2024-09-13T18:36:15.525034+0000) 2024-09-13T18:36:32.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:32 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:32.142+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.326799+0000 front 2024-09-13T18:36:01.326736+0000 (oldest deadline 2024-09-13T18:36:27.226720+0000) 2024-09-13T18:36:32.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:32 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:32.142+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:07.227160+0000 front 2024-09-13T18:36:07.227093+0000 (oldest deadline 2024-09-13T18:36:31.927195+0000) 2024-09-13T18:36:32.444 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:32 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:32.189+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:32.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:32 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:32.189+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.187910+0000 front 2024-09-13T18:35:52.187860+0000 (oldest deadline 2024-09-13T18:36:16.287954+0000) 2024-09-13T18:36:32.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:32 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:32.189+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.589086+0000 front 2024-09-13T18:36:01.588873+0000 (oldest deadline 2024-09-13T18:36:24.389897+0000) 2024-09-13T18:36:32.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:32 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:32.535+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.701894+0000 front 2024-09-13T18:35:35.701959+0000 (oldest deadline 2024-09-13T18:36:01.502502+0000) 2024-09-13T18:36:32.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:32 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:32.535+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:41.503145+0000 front 2024-09-13T18:35:41.503165+0000 (oldest deadline 2024-09-13T18:36:07.403215+0000) 2024-09-13T18:36:32.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:32 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:32.535+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:51.504425+0000 front 2024-09-13T18:35:51.504551+0000 (oldest deadline 2024-09-13T18:36:15.604567+0000) 2024-09-13T18:36:32.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:32 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:32.535+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:00.806201+0000 front 2024-09-13T18:36:00.806372+0000 (oldest deadline 2024-09-13T18:36:25.506104+0000) 2024-09-13T18:36:33.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:32 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:32.860+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.588828+0000 front 2024-09-13T18:35:34.588857+0000 (oldest deadline 2024-09-13T18:35:59.288937+0000) 2024-09-13T18:36:33.221 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:32 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:32.860+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.790933+0000 front 2024-09-13T18:35:43.791007+0000 (oldest deadline 2024-09-13T18:36:10.791344+0000) 2024-09-13T18:36:33.221 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:32 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:32.860+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:50.791852+0000 front 2024-09-13T18:35:50.791998+0000 (oldest deadline 2024-09-13T18:36:15.491911+0000) 2024-09-13T18:36:33.221 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:32 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:32.860+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.293330+0000 front 2024-09-13T18:36:01.293344+0000 (oldest deadline 2024-09-13T18:36:25.993496+0000) 2024-09-13T18:36:33.369 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:33 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:33.056+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:45.793799+0000 front 2024-09-13T18:35:45.793857+0000 (oldest deadline 2024-09-13T18:36:07.493982+0000) 2024-09-13T18:36:33.369 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:33 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:33.056+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.194892+0000 front 2024-09-13T18:35:52.194937+0000 (oldest deadline 2024-09-13T18:36:15.695008+0000) 2024-09-13T18:36:33.369 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:33 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:33.056+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:35:59.796269+0000 front 2024-09-13T18:35:59.796314+0000 (oldest deadline 2024-09-13T18:36:23.796996+0000) 2024-09-13T18:36:33.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:33 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:33.113+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:33.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:33 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:33.113+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.824009+0000 front 2024-09-13T18:35:43.824043+0000 (oldest deadline 2024-09-13T18:36:07.324119+0000) 2024-09-13T18:36:33.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:33 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:33.113+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.025152+0000 front 2024-09-13T18:35:52.025201+0000 (oldest deadline 2024-09-13T18:36:15.525034+0000) 2024-09-13T18:36:33.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:33 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:33.113+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.326799+0000 front 2024-09-13T18:36:01.326736+0000 (oldest deadline 2024-09-13T18:36:27.226720+0000) 2024-09-13T18:36:33.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:33 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:33.113+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:07.227160+0000 front 2024-09-13T18:36:07.227093+0000 (oldest deadline 2024-09-13T18:36:31.927195+0000) 2024-09-13T18:36:33.444 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:33 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:33.161+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:33.444 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:33 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:33.161+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.187910+0000 front 2024-09-13T18:35:52.187860+0000 (oldest deadline 2024-09-13T18:36:16.287954+0000) 2024-09-13T18:36:33.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:33 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:33.161+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.589086+0000 front 2024-09-13T18:36:01.588873+0000 (oldest deadline 2024-09-13T18:36:24.389897+0000) 2024-09-13T18:36:33.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:33 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:33.555+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.701894+0000 front 2024-09-13T18:35:35.701959+0000 (oldest deadline 2024-09-13T18:36:01.502502+0000) 2024-09-13T18:36:33.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:33 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:33.555+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:41.503145+0000 front 2024-09-13T18:35:41.503165+0000 (oldest deadline 2024-09-13T18:36:07.403215+0000) 2024-09-13T18:36:33.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:33 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:33.555+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:51.504425+0000 front 2024-09-13T18:35:51.504551+0000 (oldest deadline 2024-09-13T18:36:15.604567+0000) 2024-09-13T18:36:33.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:33 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:33.555+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:00.806201+0000 front 2024-09-13T18:36:00.806372+0000 (oldest deadline 2024-09-13T18:36:25.506104+0000) 2024-09-13T18:36:34.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:33 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:33.839+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.588828+0000 front 2024-09-13T18:35:34.588857+0000 (oldest deadline 2024-09-13T18:35:59.288937+0000) 2024-09-13T18:36:34.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:33 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:33.839+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.790933+0000 front 2024-09-13T18:35:43.791007+0000 (oldest deadline 2024-09-13T18:36:10.791344+0000) 2024-09-13T18:36:34.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:33 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:33.839+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:50.791852+0000 front 2024-09-13T18:35:50.791998+0000 (oldest deadline 2024-09-13T18:36:15.491911+0000) 2024-09-13T18:36:34.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:33 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:33.839+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.293330+0000 front 2024-09-13T18:36:01.293344+0000 (oldest deadline 2024-09-13T18:36:25.993496+0000) 2024-09-13T18:36:34.369 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:34 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:34.037+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:45.793799+0000 front 2024-09-13T18:35:45.793857+0000 (oldest deadline 2024-09-13T18:36:07.493982+0000) 2024-09-13T18:36:34.369 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:34 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:34.037+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.194892+0000 front 2024-09-13T18:35:52.194937+0000 (oldest deadline 2024-09-13T18:36:15.695008+0000) 2024-09-13T18:36:34.369 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:34 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:34.037+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:35:59.796269+0000 front 2024-09-13T18:35:59.796314+0000 (oldest deadline 2024-09-13T18:36:23.796996+0000) 2024-09-13T18:36:34.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:34 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:34.069+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:34.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:34 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:34.069+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.824009+0000 front 2024-09-13T18:35:43.824043+0000 (oldest deadline 2024-09-13T18:36:07.324119+0000) 2024-09-13T18:36:34.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:34 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:34.069+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.025152+0000 front 2024-09-13T18:35:52.025201+0000 (oldest deadline 2024-09-13T18:36:15.525034+0000) 2024-09-13T18:36:34.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:34 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:34.069+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.326799+0000 front 2024-09-13T18:36:01.326736+0000 (oldest deadline 2024-09-13T18:36:27.226720+0000) 2024-09-13T18:36:34.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:34 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:34.069+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:07.227160+0000 front 2024-09-13T18:36:07.227093+0000 (oldest deadline 2024-09-13T18:36:31.927195+0000) 2024-09-13T18:36:34.593 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:34 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:34.200+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:34.593 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:34 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:34.200+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.187910+0000 front 2024-09-13T18:35:52.187860+0000 (oldest deadline 2024-09-13T18:36:16.287954+0000) 2024-09-13T18:36:34.594 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:34 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:34.200+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.589086+0000 front 2024-09-13T18:36:01.588873+0000 (oldest deadline 2024-09-13T18:36:24.389897+0000) 2024-09-13T18:36:34.944 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:34 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:34.589+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.701894+0000 front 2024-09-13T18:35:35.701959+0000 (oldest deadline 2024-09-13T18:36:01.502502+0000) 2024-09-13T18:36:34.944 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:34 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:34.589+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:41.503145+0000 front 2024-09-13T18:35:41.503165+0000 (oldest deadline 2024-09-13T18:36:07.403215+0000) 2024-09-13T18:36:34.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:34 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:34.589+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:51.504425+0000 front 2024-09-13T18:35:51.504551+0000 (oldest deadline 2024-09-13T18:36:15.604567+0000) 2024-09-13T18:36:34.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:34 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:34.589+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:00.806201+0000 front 2024-09-13T18:36:00.806372+0000 (oldest deadline 2024-09-13T18:36:25.506104+0000) 2024-09-13T18:36:35.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:34 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:34.879+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.588828+0000 front 2024-09-13T18:35:34.588857+0000 (oldest deadline 2024-09-13T18:35:59.288937+0000) 2024-09-13T18:36:35.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:34 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:34.879+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.790933+0000 front 2024-09-13T18:35:43.791007+0000 (oldest deadline 2024-09-13T18:36:10.791344+0000) 2024-09-13T18:36:35.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:34 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:34.879+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:50.791852+0000 front 2024-09-13T18:35:50.791998+0000 (oldest deadline 2024-09-13T18:36:15.491911+0000) 2024-09-13T18:36:35.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:34 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:34.879+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.293330+0000 front 2024-09-13T18:36:01.293344+0000 (oldest deadline 2024-09-13T18:36:25.993496+0000) 2024-09-13T18:36:35.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:34 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:34.879+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:08.294646+0000 front 2024-09-13T18:36:08.294716+0000 (oldest deadline 2024-09-13T18:36:34.194541+0000) 2024-09-13T18:36:35.369 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:35 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:35.045+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:45.793799+0000 front 2024-09-13T18:35:45.793857+0000 (oldest deadline 2024-09-13T18:36:07.493982+0000) 2024-09-13T18:36:35.369 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:35 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:35.045+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.194892+0000 front 2024-09-13T18:35:52.194937+0000 (oldest deadline 2024-09-13T18:36:15.695008+0000) 2024-09-13T18:36:35.369 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:35 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7[51808]: 2024-09-13T18:36:35.045+0000 7f7895ddd640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:35:59.796269+0000 front 2024-09-13T18:35:59.796314+0000 (oldest deadline 2024-09-13T18:36:23.796996+0000) 2024-09-13T18:36:35.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:35 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:35.091+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:35.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:35 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:35.091+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.824009+0000 front 2024-09-13T18:35:43.824043+0000 (oldest deadline 2024-09-13T18:36:07.324119+0000) 2024-09-13T18:36:35.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:35 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:35.091+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.025152+0000 front 2024-09-13T18:35:52.025201+0000 (oldest deadline 2024-09-13T18:36:15.525034+0000) 2024-09-13T18:36:35.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:35 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:35.091+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.326799+0000 front 2024-09-13T18:36:01.326736+0000 (oldest deadline 2024-09-13T18:36:27.226720+0000) 2024-09-13T18:36:35.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:35 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:35.091+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:07.227160+0000 front 2024-09-13T18:36:07.227093+0000 (oldest deadline 2024-09-13T18:36:31.927195+0000) 2024-09-13T18:36:35.444 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:35 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:35.177+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:35.444 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:35 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:35.177+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.187910+0000 front 2024-09-13T18:35:52.187860+0000 (oldest deadline 2024-09-13T18:36:16.287954+0000) 2024-09-13T18:36:35.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:35 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:35.177+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.589086+0000 front 2024-09-13T18:36:01.588873+0000 (oldest deadline 2024-09-13T18:36:24.389897+0000) 2024-09-13T18:36:35.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:35 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:35.177+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:10.091830+0000 front 2024-09-13T18:36:10.091887+0000 (oldest deadline 2024-09-13T18:36:34.791715+0000) 2024-09-13T18:36:35.944 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:35 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:35.546+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.701894+0000 front 2024-09-13T18:35:35.701959+0000 (oldest deadline 2024-09-13T18:36:01.502502+0000) 2024-09-13T18:36:35.944 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:35 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:35.546+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:41.503145+0000 front 2024-09-13T18:35:41.503165+0000 (oldest deadline 2024-09-13T18:36:07.403215+0000) 2024-09-13T18:36:35.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:35 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:35.546+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:51.504425+0000 front 2024-09-13T18:35:51.504551+0000 (oldest deadline 2024-09-13T18:36:15.604567+0000) 2024-09-13T18:36:35.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:35 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:35.546+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:00.806201+0000 front 2024-09-13T18:36:00.806372+0000 (oldest deadline 2024-09-13T18:36:25.506104+0000) 2024-09-13T18:36:36.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:35 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:35.926+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.588828+0000 front 2024-09-13T18:35:34.588857+0000 (oldest deadline 2024-09-13T18:35:59.288937+0000) 2024-09-13T18:36:36.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:35 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:35.926+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.790933+0000 front 2024-09-13T18:35:43.791007+0000 (oldest deadline 2024-09-13T18:36:10.791344+0000) 2024-09-13T18:36:36.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:35 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:35.926+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:50.791852+0000 front 2024-09-13T18:35:50.791998+0000 (oldest deadline 2024-09-13T18:36:15.491911+0000) 2024-09-13T18:36:36.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:35 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:35.926+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.293330+0000 front 2024-09-13T18:36:01.293344+0000 (oldest deadline 2024-09-13T18:36:25.993496+0000) 2024-09-13T18:36:36.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:35 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:35.926+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:08.294646+0000 front 2024-09-13T18:36:08.294716+0000 (oldest deadline 2024-09-13T18:36:34.194541+0000) 2024-09-13T18:36:36.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:36 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:36.097+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:36.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:36 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:36.097+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.824009+0000 front 2024-09-13T18:35:43.824043+0000 (oldest deadline 2024-09-13T18:36:07.324119+0000) 2024-09-13T18:36:36.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:36 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:36.097+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.025152+0000 front 2024-09-13T18:35:52.025201+0000 (oldest deadline 2024-09-13T18:36:15.525034+0000) 2024-09-13T18:36:36.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:36 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:36.097+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.326799+0000 front 2024-09-13T18:36:01.326736+0000 (oldest deadline 2024-09-13T18:36:27.226720+0000) 2024-09-13T18:36:36.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:36 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:36.097+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:07.227160+0000 front 2024-09-13T18:36:07.227093+0000 (oldest deadline 2024-09-13T18:36:31.927195+0000) 2024-09-13T18:36:36.444 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:36 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:36.176+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:36.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:36 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:36.176+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.187910+0000 front 2024-09-13T18:35:52.187860+0000 (oldest deadline 2024-09-13T18:36:16.287954+0000) 2024-09-13T18:36:36.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:36 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:36.176+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.589086+0000 front 2024-09-13T18:36:01.588873+0000 (oldest deadline 2024-09-13T18:36:24.389897+0000) 2024-09-13T18:36:36.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:36 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:36.176+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:10.091830+0000 front 2024-09-13T18:36:10.091887+0000 (oldest deadline 2024-09-13T18:36:34.791715+0000) 2024-09-13T18:36:36.807 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:36 smithi134 podman[70347]: 2024-09-13 18:36:36.532871661 +0000 UTC m=+5.860224487 container died a7c8dc269fa311331b950c398a531ca518252ede1ac1d2ec8a398114ee551a80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-13T18:36:36.944 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:36 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:36.587+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.701894+0000 front 2024-09-13T18:35:35.701959+0000 (oldest deadline 2024-09-13T18:36:01.502502+0000) 2024-09-13T18:36:36.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:36 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:36.587+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:41.503145+0000 front 2024-09-13T18:35:41.503165+0000 (oldest deadline 2024-09-13T18:36:07.403215+0000) 2024-09-13T18:36:36.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:36 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:36.587+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:51.504425+0000 front 2024-09-13T18:35:51.504551+0000 (oldest deadline 2024-09-13T18:36:15.604567+0000) 2024-09-13T18:36:36.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:36 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:36.587+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:00.806201+0000 front 2024-09-13T18:36:00.806372+0000 (oldest deadline 2024-09-13T18:36:25.506104+0000) 2024-09-13T18:36:37.119 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:36 smithi134 podman[70347]: 2024-09-13 18:36:36.8966517 +0000 UTC m=+6.224004525 container cleanup a7c8dc269fa311331b950c398a531ca518252ede1ac1d2ec8a398114ee551a80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-13T18:36:37.119 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:36 smithi134 bash[70347]: ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7 2024-09-13T18:36:37.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:36 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:36.935+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.588828+0000 front 2024-09-13T18:35:34.588857+0000 (oldest deadline 2024-09-13T18:35:59.288937+0000) 2024-09-13T18:36:37.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:36 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:36.935+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.790933+0000 front 2024-09-13T18:35:43.791007+0000 (oldest deadline 2024-09-13T18:36:10.791344+0000) 2024-09-13T18:36:37.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:36 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:36.935+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:50.791852+0000 front 2024-09-13T18:35:50.791998+0000 (oldest deadline 2024-09-13T18:36:15.491911+0000) 2024-09-13T18:36:37.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:36 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:36.935+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.293330+0000 front 2024-09-13T18:36:01.293344+0000 (oldest deadline 2024-09-13T18:36:25.993496+0000) 2024-09-13T18:36:37.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:36 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:36.935+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:08.294646+0000 front 2024-09-13T18:36:08.294716+0000 (oldest deadline 2024-09-13T18:36:34.194541+0000) 2024-09-13T18:36:37.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:37 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:37.154+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:37.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:37 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:37.154+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.187910+0000 front 2024-09-13T18:35:52.187860+0000 (oldest deadline 2024-09-13T18:36:16.287954+0000) 2024-09-13T18:36:37.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:37 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:37.154+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.589086+0000 front 2024-09-13T18:36:01.588873+0000 (oldest deadline 2024-09-13T18:36:24.389897+0000) 2024-09-13T18:36:37.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:37 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:37.154+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:10.091830+0000 front 2024-09-13T18:36:10.091887+0000 (oldest deadline 2024-09-13T18:36:34.791715+0000) 2024-09-13T18:36:37.527 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:37 smithi134 podman[70363]: 2024-09-13 18:36:37.21519004 +0000 UTC m=+0.675893647 container remove a7c8dc269fa311331b950c398a531ca518252ede1ac1d2ec8a398114ee551a80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-13T18:36:37.528 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:37 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:37.119+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:37.528 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:37 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:37.119+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.824009+0000 front 2024-09-13T18:35:43.824043+0000 (oldest deadline 2024-09-13T18:36:07.324119+0000) 2024-09-13T18:36:37.528 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:37 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:37.119+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.025152+0000 front 2024-09-13T18:35:52.025201+0000 (oldest deadline 2024-09-13T18:36:15.525034+0000) 2024-09-13T18:36:37.528 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:37 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:37.119+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.326799+0000 front 2024-09-13T18:36:01.326736+0000 (oldest deadline 2024-09-13T18:36:27.226720+0000) 2024-09-13T18:36:37.528 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:37 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:37.119+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:07.227160+0000 front 2024-09-13T18:36:07.227093+0000 (oldest deadline 2024-09-13T18:36:31.927195+0000) 2024-09-13T18:36:37.821 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:37 smithi134 podman[70453]: 2024-09-13 18:36:37.428542488 +0000 UTC m=+0.016376268 image pull 8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256 2024-09-13T18:36:37.821 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:37 smithi134 podman[70453]: 2024-09-13 18:36:37.557944675 +0000 UTC m=+0.145778445 container create ef97c8bdf4a2ef3d70b2a0afdffe59327023e7772dad4615b7e91d589005565d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.license=GPLv2) 2024-09-13T18:36:37.821 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:37 smithi134 podman[70453]: 2024-09-13 18:36:37.818943996 +0000 UTC m=+0.406777770 container init ef97c8bdf4a2ef3d70b2a0afdffe59327023e7772dad4615b7e91d589005565d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T18:36:37.944 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:37 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:37.569+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.701894+0000 front 2024-09-13T18:35:35.701959+0000 (oldest deadline 2024-09-13T18:36:01.502502+0000) 2024-09-13T18:36:37.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:37 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:37.569+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:41.503145+0000 front 2024-09-13T18:35:41.503165+0000 (oldest deadline 2024-09-13T18:36:07.403215+0000) 2024-09-13T18:36:37.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:37 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:37.569+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:51.504425+0000 front 2024-09-13T18:35:51.504551+0000 (oldest deadline 2024-09-13T18:36:15.604567+0000) 2024-09-13T18:36:37.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:37 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:37.569+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:00.806201+0000 front 2024-09-13T18:36:00.806372+0000 (oldest deadline 2024-09-13T18:36:25.506104+0000) 2024-09-13T18:36:38.119 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:37 smithi134 podman[70453]: 2024-09-13 18:36:37.822539243 +0000 UTC m=+0.410373016 container start ef97c8bdf4a2ef3d70b2a0afdffe59327023e7772dad4615b7e91d589005565d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T18:36:38.119 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:37 smithi134 podman[70453]: 2024-09-13 18:36:37.910502956 +0000 UTC m=+0.498336728 container attach ef97c8bdf4a2ef3d70b2a0afdffe59327023e7772dad4615b7e91d589005565d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7-deactivate, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-13T18:36:38.119 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 13 18:36:38 smithi134 podman[70453]: 2024-09-13 18:36:38.008058076 +0000 UTC m=+0.595891843 container died ef97c8bdf4a2ef3d70b2a0afdffe59327023e7772dad4615b7e91d589005565d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-13T18:36:38.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:37 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:37.962+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.588828+0000 front 2024-09-13T18:35:34.588857+0000 (oldest deadline 2024-09-13T18:35:59.288937+0000) 2024-09-13T18:36:38.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:37 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:37.962+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.790933+0000 front 2024-09-13T18:35:43.791007+0000 (oldest deadline 2024-09-13T18:36:10.791344+0000) 2024-09-13T18:36:38.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:37 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:37.962+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:50.791852+0000 front 2024-09-13T18:35:50.791998+0000 (oldest deadline 2024-09-13T18:36:15.491911+0000) 2024-09-13T18:36:38.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:37 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:37.962+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.293330+0000 front 2024-09-13T18:36:01.293344+0000 (oldest deadline 2024-09-13T18:36:25.993496+0000) 2024-09-13T18:36:38.221 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:37 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:37.962+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:08.294646+0000 front 2024-09-13T18:36:08.294716+0000 (oldest deadline 2024-09-13T18:36:34.194541+0000) 2024-09-13T18:36:38.444 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:38 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:38.128+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:38.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:38 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:38.128+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.187910+0000 front 2024-09-13T18:35:52.187860+0000 (oldest deadline 2024-09-13T18:36:16.287954+0000) 2024-09-13T18:36:38.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:38 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:38.128+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.589086+0000 front 2024-09-13T18:36:01.588873+0000 (oldest deadline 2024-09-13T18:36:24.389897+0000) 2024-09-13T18:36:38.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:38 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:38.128+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:10.091830+0000 front 2024-09-13T18:36:10.091887+0000 (oldest deadline 2024-09-13T18:36:34.791715+0000) 2024-09-13T18:36:38.619 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:38 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:38.121+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:38.619 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:38 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:38.121+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.824009+0000 front 2024-09-13T18:35:43.824043+0000 (oldest deadline 2024-09-13T18:36:07.324119+0000) 2024-09-13T18:36:38.619 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:38 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:38.121+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.025152+0000 front 2024-09-13T18:35:52.025201+0000 (oldest deadline 2024-09-13T18:36:15.525034+0000) 2024-09-13T18:36:38.619 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:38 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:38.121+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.326799+0000 front 2024-09-13T18:36:01.326736+0000 (oldest deadline 2024-09-13T18:36:27.226720+0000) 2024-09-13T18:36:38.619 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:38 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:38.121+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:07.227160+0000 front 2024-09-13T18:36:07.227093+0000 (oldest deadline 2024-09-13T18:36:31.927195+0000) 2024-09-13T18:36:38.645 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@osd.7.service' 2024-09-13T18:36:38.686 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T18:36:38.687 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-13T18:36:38.687 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-13T18:36:38.687 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@osd.8 2024-09-13T18:36:38.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:38 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:38.581+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.701894+0000 front 2024-09-13T18:35:35.701959+0000 (oldest deadline 2024-09-13T18:36:01.502502+0000) 2024-09-13T18:36:38.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:38 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:38.581+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:41.503145+0000 front 2024-09-13T18:35:41.503165+0000 (oldest deadline 2024-09-13T18:36:07.403215+0000) 2024-09-13T18:36:38.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:38 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:38.581+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:51.504425+0000 front 2024-09-13T18:35:51.504551+0000 (oldest deadline 2024-09-13T18:36:15.604567+0000) 2024-09-13T18:36:38.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:38 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:38.581+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:00.806201+0000 front 2024-09-13T18:36:00.806372+0000 (oldest deadline 2024-09-13T18:36:25.506104+0000) 2024-09-13T18:36:38.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:38 smithi204 systemd[1]: Stopping Ceph osd.8 for 97e7edf2-71fb-11ef-bceb-c7b262605968... 2024-09-13T18:36:39.079 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:39 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:39.076+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:39.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:38 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:38.948+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.588828+0000 front 2024-09-13T18:35:34.588857+0000 (oldest deadline 2024-09-13T18:35:59.288937+0000) 2024-09-13T18:36:39.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:38 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:38.948+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.790933+0000 front 2024-09-13T18:35:43.791007+0000 (oldest deadline 2024-09-13T18:36:10.791344+0000) 2024-09-13T18:36:39.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:38 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:38.948+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:50.791852+0000 front 2024-09-13T18:35:50.791998+0000 (oldest deadline 2024-09-13T18:36:15.491911+0000) 2024-09-13T18:36:39.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:38 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:38.948+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.293330+0000 front 2024-09-13T18:36:01.293344+0000 (oldest deadline 2024-09-13T18:36:25.993496+0000) 2024-09-13T18:36:39.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:38 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:38.948+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:08.294646+0000 front 2024-09-13T18:36:08.294716+0000 (oldest deadline 2024-09-13T18:36:34.194541+0000) 2024-09-13T18:36:39.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:39 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:39.076+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.824009+0000 front 2024-09-13T18:35:43.824043+0000 (oldest deadline 2024-09-13T18:36:07.324119+0000) 2024-09-13T18:36:39.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:39 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:39.076+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.025152+0000 front 2024-09-13T18:35:52.025201+0000 (oldest deadline 2024-09-13T18:36:15.525034+0000) 2024-09-13T18:36:39.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:39 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:39.076+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.326799+0000 front 2024-09-13T18:36:01.326736+0000 (oldest deadline 2024-09-13T18:36:27.226720+0000) 2024-09-13T18:36:39.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:39 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:39.076+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:07.227160+0000 front 2024-09-13T18:36:07.227093+0000 (oldest deadline 2024-09-13T18:36:31.927195+0000) 2024-09-13T18:36:39.445 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:39 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:39.007+0000 7fd48809b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T18:36:39.445 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:39 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:39.007+0000 7fd48809b640 -1 osd.8 111 *** Got signal Terminated *** 2024-09-13T18:36:39.445 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:39 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:39.007+0000 7fd48809b640 -1 osd.8 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T18:36:39.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:39 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:39.171+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:39.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:39 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:39.171+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.187910+0000 front 2024-09-13T18:35:52.187860+0000 (oldest deadline 2024-09-13T18:36:16.287954+0000) 2024-09-13T18:36:39.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:39 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:39.171+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.589086+0000 front 2024-09-13T18:36:01.588873+0000 (oldest deadline 2024-09-13T18:36:24.389897+0000) 2024-09-13T18:36:39.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:39 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:39.171+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:10.091830+0000 front 2024-09-13T18:36:10.091887+0000 (oldest deadline 2024-09-13T18:36:34.791715+0000) 2024-09-13T18:36:39.944 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:39 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:39.590+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.701894+0000 front 2024-09-13T18:35:35.701959+0000 (oldest deadline 2024-09-13T18:36:01.502502+0000) 2024-09-13T18:36:39.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:39 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:39.590+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:41.503145+0000 front 2024-09-13T18:35:41.503165+0000 (oldest deadline 2024-09-13T18:36:07.403215+0000) 2024-09-13T18:36:39.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:39 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:39.590+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:51.504425+0000 front 2024-09-13T18:35:51.504551+0000 (oldest deadline 2024-09-13T18:36:15.604567+0000) 2024-09-13T18:36:39.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:39 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:39.590+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:00.806201+0000 front 2024-09-13T18:36:00.806372+0000 (oldest deadline 2024-09-13T18:36:25.506104+0000) 2024-09-13T18:36:40.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:39 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:39.909+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.588828+0000 front 2024-09-13T18:35:34.588857+0000 (oldest deadline 2024-09-13T18:35:59.288937+0000) 2024-09-13T18:36:40.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:39 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:39.909+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.790933+0000 front 2024-09-13T18:35:43.791007+0000 (oldest deadline 2024-09-13T18:36:10.791344+0000) 2024-09-13T18:36:40.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:39 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:39.909+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:50.791852+0000 front 2024-09-13T18:35:50.791998+0000 (oldest deadline 2024-09-13T18:36:15.491911+0000) 2024-09-13T18:36:40.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:39 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:39.909+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.293330+0000 front 2024-09-13T18:36:01.293344+0000 (oldest deadline 2024-09-13T18:36:25.993496+0000) 2024-09-13T18:36:40.221 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:39 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:39.909+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:08.294646+0000 front 2024-09-13T18:36:08.294716+0000 (oldest deadline 2024-09-13T18:36:34.194541+0000) 2024-09-13T18:36:40.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:40 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:40.080+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:40.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:40 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:40.080+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.824009+0000 front 2024-09-13T18:35:43.824043+0000 (oldest deadline 2024-09-13T18:36:07.324119+0000) 2024-09-13T18:36:40.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:40 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:40.080+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.025152+0000 front 2024-09-13T18:35:52.025201+0000 (oldest deadline 2024-09-13T18:36:15.525034+0000) 2024-09-13T18:36:40.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:40 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:40.080+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.326799+0000 front 2024-09-13T18:36:01.326736+0000 (oldest deadline 2024-09-13T18:36:27.226720+0000) 2024-09-13T18:36:40.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:40 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:40.080+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:07.227160+0000 front 2024-09-13T18:36:07.227093+0000 (oldest deadline 2024-09-13T18:36:31.927195+0000) 2024-09-13T18:36:40.625 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:40 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:40.194+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:40.625 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:40 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:40.194+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.187910+0000 front 2024-09-13T18:35:52.187860+0000 (oldest deadline 2024-09-13T18:36:16.287954+0000) 2024-09-13T18:36:40.625 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:40 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:40.194+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.589086+0000 front 2024-09-13T18:36:01.588873+0000 (oldest deadline 2024-09-13T18:36:24.389897+0000) 2024-09-13T18:36:40.625 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:40 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:40.194+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:10.091830+0000 front 2024-09-13T18:36:10.091887+0000 (oldest deadline 2024-09-13T18:36:34.791715+0000) 2024-09-13T18:36:40.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:40 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:40.621+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.701894+0000 front 2024-09-13T18:35:35.701959+0000 (oldest deadline 2024-09-13T18:36:01.502502+0000) 2024-09-13T18:36:40.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:40 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:40.621+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:41.503145+0000 front 2024-09-13T18:35:41.503165+0000 (oldest deadline 2024-09-13T18:36:07.403215+0000) 2024-09-13T18:36:40.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:40 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:40.621+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:51.504425+0000 front 2024-09-13T18:35:51.504551+0000 (oldest deadline 2024-09-13T18:36:15.604567+0000) 2024-09-13T18:36:40.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:40 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:40.621+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:00.806201+0000 front 2024-09-13T18:36:00.806372+0000 (oldest deadline 2024-09-13T18:36:25.506104+0000) 2024-09-13T18:36:41.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:40 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:40.898+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.588828+0000 front 2024-09-13T18:35:34.588857+0000 (oldest deadline 2024-09-13T18:35:59.288937+0000) 2024-09-13T18:36:41.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:40 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:40.898+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.790933+0000 front 2024-09-13T18:35:43.791007+0000 (oldest deadline 2024-09-13T18:36:10.791344+0000) 2024-09-13T18:36:41.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:40 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:40.898+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:50.791852+0000 front 2024-09-13T18:35:50.791998+0000 (oldest deadline 2024-09-13T18:36:15.491911+0000) 2024-09-13T18:36:41.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:40 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:40.898+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.293330+0000 front 2024-09-13T18:36:01.293344+0000 (oldest deadline 2024-09-13T18:36:25.993496+0000) 2024-09-13T18:36:41.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:40 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:40.898+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:08.294646+0000 front 2024-09-13T18:36:08.294716+0000 (oldest deadline 2024-09-13T18:36:34.194541+0000) 2024-09-13T18:36:41.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:41 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:41.064+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:41.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:41 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:41.064+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.824009+0000 front 2024-09-13T18:35:43.824043+0000 (oldest deadline 2024-09-13T18:36:07.324119+0000) 2024-09-13T18:36:41.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:41 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:41.064+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.025152+0000 front 2024-09-13T18:35:52.025201+0000 (oldest deadline 2024-09-13T18:36:15.525034+0000) 2024-09-13T18:36:41.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:41 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:41.064+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.326799+0000 front 2024-09-13T18:36:01.326736+0000 (oldest deadline 2024-09-13T18:36:27.226720+0000) 2024-09-13T18:36:41.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:41 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:41.064+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:07.227160+0000 front 2024-09-13T18:36:07.227093+0000 (oldest deadline 2024-09-13T18:36:31.927195+0000) 2024-09-13T18:36:41.444 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:41 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:41.177+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:41.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:41 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:41.177+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.187910+0000 front 2024-09-13T18:35:52.187860+0000 (oldest deadline 2024-09-13T18:36:16.287954+0000) 2024-09-13T18:36:41.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:41 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:41.177+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.589086+0000 front 2024-09-13T18:36:01.588873+0000 (oldest deadline 2024-09-13T18:36:24.389897+0000) 2024-09-13T18:36:41.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:41 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:41.177+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:10.091830+0000 front 2024-09-13T18:36:10.091887+0000 (oldest deadline 2024-09-13T18:36:34.791715+0000) 2024-09-13T18:36:41.944 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:41 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:41.576+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.701894+0000 front 2024-09-13T18:35:35.701959+0000 (oldest deadline 2024-09-13T18:36:01.502502+0000) 2024-09-13T18:36:41.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:41 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:41.576+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:41.503145+0000 front 2024-09-13T18:35:41.503165+0000 (oldest deadline 2024-09-13T18:36:07.403215+0000) 2024-09-13T18:36:41.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:41 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:41.576+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:51.504425+0000 front 2024-09-13T18:35:51.504551+0000 (oldest deadline 2024-09-13T18:36:15.604567+0000) 2024-09-13T18:36:41.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:41 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:41.576+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:00.806201+0000 front 2024-09-13T18:36:00.806372+0000 (oldest deadline 2024-09-13T18:36:25.506104+0000) 2024-09-13T18:36:42.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:41 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:41.910+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.588828+0000 front 2024-09-13T18:35:34.588857+0000 (oldest deadline 2024-09-13T18:35:59.288937+0000) 2024-09-13T18:36:42.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:41 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:41.910+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.790933+0000 front 2024-09-13T18:35:43.791007+0000 (oldest deadline 2024-09-13T18:36:10.791344+0000) 2024-09-13T18:36:42.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:41 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:41.910+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:50.791852+0000 front 2024-09-13T18:35:50.791998+0000 (oldest deadline 2024-09-13T18:36:15.491911+0000) 2024-09-13T18:36:42.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:41 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:41.910+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.293330+0000 front 2024-09-13T18:36:01.293344+0000 (oldest deadline 2024-09-13T18:36:25.993496+0000) 2024-09-13T18:36:42.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:41 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:41.910+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:08.294646+0000 front 2024-09-13T18:36:08.294716+0000 (oldest deadline 2024-09-13T18:36:34.194541+0000) 2024-09-13T18:36:42.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:42 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:42.056+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:42.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:42 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:42.056+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.824009+0000 front 2024-09-13T18:35:43.824043+0000 (oldest deadline 2024-09-13T18:36:07.324119+0000) 2024-09-13T18:36:42.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:42 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:42.056+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.025152+0000 front 2024-09-13T18:35:52.025201+0000 (oldest deadline 2024-09-13T18:36:15.525034+0000) 2024-09-13T18:36:42.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:42 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:42.056+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.326799+0000 front 2024-09-13T18:36:01.326736+0000 (oldest deadline 2024-09-13T18:36:27.226720+0000) 2024-09-13T18:36:42.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:42 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:42.056+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:07.227160+0000 front 2024-09-13T18:36:07.227093+0000 (oldest deadline 2024-09-13T18:36:31.927195+0000) 2024-09-13T18:36:42.444 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:42 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:42.181+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:42.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:42 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:42.181+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.187910+0000 front 2024-09-13T18:35:52.187860+0000 (oldest deadline 2024-09-13T18:36:16.287954+0000) 2024-09-13T18:36:42.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:42 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:42.181+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.589086+0000 front 2024-09-13T18:36:01.588873+0000 (oldest deadline 2024-09-13T18:36:24.389897+0000) 2024-09-13T18:36:42.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:42 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:42.181+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:10.091830+0000 front 2024-09-13T18:36:10.091887+0000 (oldest deadline 2024-09-13T18:36:34.791715+0000) 2024-09-13T18:36:42.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:42 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:42.614+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.701894+0000 front 2024-09-13T18:35:35.701959+0000 (oldest deadline 2024-09-13T18:36:01.502502+0000) 2024-09-13T18:36:42.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:42 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:42.614+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:41.503145+0000 front 2024-09-13T18:35:41.503165+0000 (oldest deadline 2024-09-13T18:36:07.403215+0000) 2024-09-13T18:36:42.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:42 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:42.614+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:51.504425+0000 front 2024-09-13T18:35:51.504551+0000 (oldest deadline 2024-09-13T18:36:15.604567+0000) 2024-09-13T18:36:42.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:42 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:42.614+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:00.806201+0000 front 2024-09-13T18:36:00.806372+0000 (oldest deadline 2024-09-13T18:36:25.506104+0000) 2024-09-13T18:36:42.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:42 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:42.614+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-13T18:36:18.908185+0000 front 2024-09-13T18:36:18.908202+0000 (oldest deadline 2024-09-13T18:36:41.808287+0000) 2024-09-13T18:36:43.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:42 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:42.934+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.588828+0000 front 2024-09-13T18:35:34.588857+0000 (oldest deadline 2024-09-13T18:35:59.288937+0000) 2024-09-13T18:36:43.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:42 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:42.934+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.790933+0000 front 2024-09-13T18:35:43.791007+0000 (oldest deadline 2024-09-13T18:36:10.791344+0000) 2024-09-13T18:36:43.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:42 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:42.934+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:50.791852+0000 front 2024-09-13T18:35:50.791998+0000 (oldest deadline 2024-09-13T18:36:15.491911+0000) 2024-09-13T18:36:43.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:42 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:42.934+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.293330+0000 front 2024-09-13T18:36:01.293344+0000 (oldest deadline 2024-09-13T18:36:25.993496+0000) 2024-09-13T18:36:43.221 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:42 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:42.934+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:08.294646+0000 front 2024-09-13T18:36:08.294716+0000 (oldest deadline 2024-09-13T18:36:34.194541+0000) 2024-09-13T18:36:43.221 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:42 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:42.934+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-13T18:36:17.095551+0000 front 2024-09-13T18:36:17.095653+0000 (oldest deadline 2024-09-13T18:36:42.395537+0000) 2024-09-13T18:36:43.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:43 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:43.083+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:43.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:43 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:43.083+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.824009+0000 front 2024-09-13T18:35:43.824043+0000 (oldest deadline 2024-09-13T18:36:07.324119+0000) 2024-09-13T18:36:43.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:43 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:43.083+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.025152+0000 front 2024-09-13T18:35:52.025201+0000 (oldest deadline 2024-09-13T18:36:15.525034+0000) 2024-09-13T18:36:43.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:43 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:43.083+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.326799+0000 front 2024-09-13T18:36:01.326736+0000 (oldest deadline 2024-09-13T18:36:27.226720+0000) 2024-09-13T18:36:43.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:43 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:43.083+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:07.227160+0000 front 2024-09-13T18:36:07.227093+0000 (oldest deadline 2024-09-13T18:36:31.927195+0000) 2024-09-13T18:36:43.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:43 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:43.083+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-13T18:36:17.728557+0000 front 2024-09-13T18:36:17.728465+0000 (oldest deadline 2024-09-13T18:36:43.028465+0000) 2024-09-13T18:36:43.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:43 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:43.149+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:43.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:43 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:43.149+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.187910+0000 front 2024-09-13T18:35:52.187860+0000 (oldest deadline 2024-09-13T18:36:16.287954+0000) 2024-09-13T18:36:43.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:43 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:43.149+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.589086+0000 front 2024-09-13T18:36:01.588873+0000 (oldest deadline 2024-09-13T18:36:24.389897+0000) 2024-09-13T18:36:43.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:43 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:43.149+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:10.091830+0000 front 2024-09-13T18:36:10.091887+0000 (oldest deadline 2024-09-13T18:36:34.791715+0000) 2024-09-13T18:36:43.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:43 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:43.598+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:35.701894+0000 front 2024-09-13T18:35:35.701959+0000 (oldest deadline 2024-09-13T18:36:01.502502+0000) 2024-09-13T18:36:43.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:43 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:43.598+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:41.503145+0000 front 2024-09-13T18:35:41.503165+0000 (oldest deadline 2024-09-13T18:36:07.403215+0000) 2024-09-13T18:36:43.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:43 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:43.598+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:51.504425+0000 front 2024-09-13T18:35:51.504551+0000 (oldest deadline 2024-09-13T18:36:15.604567+0000) 2024-09-13T18:36:43.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:43 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:43.598+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:00.806201+0000 front 2024-09-13T18:36:00.806372+0000 (oldest deadline 2024-09-13T18:36:25.506104+0000) 2024-09-13T18:36:43.945 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:43 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8[52699]: 2024-09-13T18:36:43.598+0000 7fd4806a7640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-13T18:36:18.908185+0000 front 2024-09-13T18:36:18.908202+0000 (oldest deadline 2024-09-13T18:36:41.808287+0000) 2024-09-13T18:36:44.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:43 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:43.925+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.588828+0000 front 2024-09-13T18:35:34.588857+0000 (oldest deadline 2024-09-13T18:35:59.288937+0000) 2024-09-13T18:36:44.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:43 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:43.925+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.790933+0000 front 2024-09-13T18:35:43.791007+0000 (oldest deadline 2024-09-13T18:36:10.791344+0000) 2024-09-13T18:36:44.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:43 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:43.925+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:50.791852+0000 front 2024-09-13T18:35:50.791998+0000 (oldest deadline 2024-09-13T18:36:15.491911+0000) 2024-09-13T18:36:44.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:43 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:43.925+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.293330+0000 front 2024-09-13T18:36:01.293344+0000 (oldest deadline 2024-09-13T18:36:25.993496+0000) 2024-09-13T18:36:44.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:43 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:43.925+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:08.294646+0000 front 2024-09-13T18:36:08.294716+0000 (oldest deadline 2024-09-13T18:36:34.194541+0000) 2024-09-13T18:36:44.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:43 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:43.925+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-13T18:36:17.095551+0000 front 2024-09-13T18:36:17.095653+0000 (oldest deadline 2024-09-13T18:36:42.395537+0000) 2024-09-13T18:36:44.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:44 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:44.065+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:44.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:44 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:44.065+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.824009+0000 front 2024-09-13T18:35:43.824043+0000 (oldest deadline 2024-09-13T18:36:07.324119+0000) 2024-09-13T18:36:44.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:44 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:44.065+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.025152+0000 front 2024-09-13T18:35:52.025201+0000 (oldest deadline 2024-09-13T18:36:15.525034+0000) 2024-09-13T18:36:44.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:44 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:44.065+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.326799+0000 front 2024-09-13T18:36:01.326736+0000 (oldest deadline 2024-09-13T18:36:27.226720+0000) 2024-09-13T18:36:44.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:44 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:44.065+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:07.227160+0000 front 2024-09-13T18:36:07.227093+0000 (oldest deadline 2024-09-13T18:36:31.927195+0000) 2024-09-13T18:36:44.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:44 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:44.065+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-13T18:36:17.728557+0000 front 2024-09-13T18:36:17.728465+0000 (oldest deadline 2024-09-13T18:36:43.028465+0000) 2024-09-13T18:36:44.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:44 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:44.184+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:44.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:44 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:44.184+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.187910+0000 front 2024-09-13T18:35:52.187860+0000 (oldest deadline 2024-09-13T18:36:16.287954+0000) 2024-09-13T18:36:44.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:44 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:44.184+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.589086+0000 front 2024-09-13T18:36:01.588873+0000 (oldest deadline 2024-09-13T18:36:24.389897+0000) 2024-09-13T18:36:44.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:44 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:44.184+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:10.091830+0000 front 2024-09-13T18:36:10.091887+0000 (oldest deadline 2024-09-13T18:36:34.791715+0000) 2024-09-13T18:36:44.870 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:44 smithi204 podman[71602]: 2024-09-13 18:36:44.565371872 +0000 UTC m=+5.684285774 container died 12a7be3c1e86a9b5107935ebeb31d7554c029e5ddd1f939eca0666ec8570ffe0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-13T18:36:45.177 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:44 smithi204 podman[71602]: 2024-09-13 18:36:44.975997614 +0000 UTC m=+6.094911514 container cleanup 12a7be3c1e86a9b5107935ebeb31d7554c029e5ddd1f939eca0666ec8570ffe0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-13T18:36:45.177 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:44 smithi204 bash[71602]: ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8 2024-09-13T18:36:45.222 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:44 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:44.945+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.588828+0000 front 2024-09-13T18:35:34.588857+0000 (oldest deadline 2024-09-13T18:35:59.288937+0000) 2024-09-13T18:36:45.222 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:44 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:44.945+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.790933+0000 front 2024-09-13T18:35:43.791007+0000 (oldest deadline 2024-09-13T18:36:10.791344+0000) 2024-09-13T18:36:45.222 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:44 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:44.945+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:50.791852+0000 front 2024-09-13T18:35:50.791998+0000 (oldest deadline 2024-09-13T18:36:15.491911+0000) 2024-09-13T18:36:45.222 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:44 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:44.945+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.293330+0000 front 2024-09-13T18:36:01.293344+0000 (oldest deadline 2024-09-13T18:36:25.993496+0000) 2024-09-13T18:36:45.222 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:44 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:44.945+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:08.294646+0000 front 2024-09-13T18:36:08.294716+0000 (oldest deadline 2024-09-13T18:36:34.194541+0000) 2024-09-13T18:36:45.222 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:44 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:44.945+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-13T18:36:17.095551+0000 front 2024-09-13T18:36:17.095653+0000 (oldest deadline 2024-09-13T18:36:42.395537+0000) 2024-09-13T18:36:45.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:45 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:45.034+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:45.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:45 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:45.034+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.824009+0000 front 2024-09-13T18:35:43.824043+0000 (oldest deadline 2024-09-13T18:36:07.324119+0000) 2024-09-13T18:36:45.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:45 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:45.034+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.025152+0000 front 2024-09-13T18:35:52.025201+0000 (oldest deadline 2024-09-13T18:36:15.525034+0000) 2024-09-13T18:36:45.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:45 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:45.034+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.326799+0000 front 2024-09-13T18:36:01.326736+0000 (oldest deadline 2024-09-13T18:36:27.226720+0000) 2024-09-13T18:36:45.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:45 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:45.034+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:07.227160+0000 front 2024-09-13T18:36:07.227093+0000 (oldest deadline 2024-09-13T18:36:31.927195+0000) 2024-09-13T18:36:45.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:45 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:45.034+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-13T18:36:17.728557+0000 front 2024-09-13T18:36:17.728465+0000 (oldest deadline 2024-09-13T18:36:43.028465+0000) 2024-09-13T18:36:45.444 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:45 smithi204 podman[71619]: 2024-09-13 18:36:45.244053684 +0000 UTC m=+0.680587425 container remove 12a7be3c1e86a9b5107935ebeb31d7554c029e5ddd1f939eca0666ec8570ffe0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-13T18:36:45.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:45 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:45.172+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:45.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:45 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:45.172+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.187910+0000 front 2024-09-13T18:35:52.187860+0000 (oldest deadline 2024-09-13T18:36:16.287954+0000) 2024-09-13T18:36:45.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:45 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:45.172+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.589086+0000 front 2024-09-13T18:36:01.588873+0000 (oldest deadline 2024-09-13T18:36:24.389897+0000) 2024-09-13T18:36:45.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:45 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:45.172+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:10.091830+0000 front 2024-09-13T18:36:10.091887+0000 (oldest deadline 2024-09-13T18:36:34.791715+0000) 2024-09-13T18:36:45.874 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:45 smithi204 podman[71706]: 2024-09-13 18:36:45.487139468 +0000 UTC m=+0.016443889 image pull 8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256 2024-09-13T18:36:45.874 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:45 smithi204 podman[71706]: 2024-09-13 18:36:45.620211739 +0000 UTC m=+0.149516147 container create 5b0b9982acd06c79f268a0de508acb5690b0e2eba16c47546618a5f2503b515e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8-deactivate, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-13T18:36:46.174 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:45 smithi204 podman[71706]: 2024-09-13 18:36:45.872635611 +0000 UTC m=+0.401940020 container init 5b0b9982acd06c79f268a0de508acb5690b0e2eba16c47546618a5f2503b515e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8-deactivate, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-13T18:36:46.174 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:45 smithi204 podman[71706]: 2024-09-13 18:36:45.875881055 +0000 UTC m=+0.405185466 container start 5b0b9982acd06c79f268a0de508acb5690b0e2eba16c47546618a5f2503b515e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T18:36:46.174 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:45 smithi204 podman[71706]: 2024-09-13 18:36:45.972719996 +0000 UTC m=+0.502024427 container attach 5b0b9982acd06c79f268a0de508acb5690b0e2eba16c47546618a5f2503b515e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-13T18:36:46.174 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 13 18:36:46 smithi204 podman[71706]: 2024-09-13 18:36:46.070218131 +0000 UTC m=+0.599522543 container died 5b0b9982acd06c79f268a0de508acb5690b0e2eba16c47546618a5f2503b515e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-8-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-13T18:36:46.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:45 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:45.938+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.588828+0000 front 2024-09-13T18:35:34.588857+0000 (oldest deadline 2024-09-13T18:35:59.288937+0000) 2024-09-13T18:36:46.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:45 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:45.938+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.790933+0000 front 2024-09-13T18:35:43.791007+0000 (oldest deadline 2024-09-13T18:36:10.791344+0000) 2024-09-13T18:36:46.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:45 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:45.938+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:50.791852+0000 front 2024-09-13T18:35:50.791998+0000 (oldest deadline 2024-09-13T18:36:15.491911+0000) 2024-09-13T18:36:46.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:45 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:45.938+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.293330+0000 front 2024-09-13T18:36:01.293344+0000 (oldest deadline 2024-09-13T18:36:25.993496+0000) 2024-09-13T18:36:46.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:45 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:45.938+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:08.294646+0000 front 2024-09-13T18:36:08.294716+0000 (oldest deadline 2024-09-13T18:36:34.194541+0000) 2024-09-13T18:36:46.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:45 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:45.938+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-13T18:36:17.095551+0000 front 2024-09-13T18:36:17.095653+0000 (oldest deadline 2024-09-13T18:36:42.395537+0000) 2024-09-13T18:36:46.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:46 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:46.062+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:46.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:46 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:46.062+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.824009+0000 front 2024-09-13T18:35:43.824043+0000 (oldest deadline 2024-09-13T18:36:07.324119+0000) 2024-09-13T18:36:46.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:46 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:46.062+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.025152+0000 front 2024-09-13T18:35:52.025201+0000 (oldest deadline 2024-09-13T18:36:15.525034+0000) 2024-09-13T18:36:46.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:46 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:46.062+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.326799+0000 front 2024-09-13T18:36:01.326736+0000 (oldest deadline 2024-09-13T18:36:27.226720+0000) 2024-09-13T18:36:46.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:46 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:46.062+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:07.227160+0000 front 2024-09-13T18:36:07.227093+0000 (oldest deadline 2024-09-13T18:36:31.927195+0000) 2024-09-13T18:36:46.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:46 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:46.062+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-13T18:36:17.728557+0000 front 2024-09-13T18:36:17.728465+0000 (oldest deadline 2024-09-13T18:36:43.028465+0000) 2024-09-13T18:36:46.444 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:46 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:46.169+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:46.444 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:46 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:46.169+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.187910+0000 front 2024-09-13T18:35:52.187860+0000 (oldest deadline 2024-09-13T18:36:16.287954+0000) 2024-09-13T18:36:46.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:46 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:46.169+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.589086+0000 front 2024-09-13T18:36:01.588873+0000 (oldest deadline 2024-09-13T18:36:24.389897+0000) 2024-09-13T18:36:46.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:46 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:46.169+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:10.091830+0000 front 2024-09-13T18:36:10.091887+0000 (oldest deadline 2024-09-13T18:36:34.791715+0000) 2024-09-13T18:36:46.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:46 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:46.169+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-13T18:36:17.692751+0000 front 2024-09-13T18:36:17.692781+0000 (oldest deadline 2024-09-13T18:36:45.793624+0000) 2024-09-13T18:36:46.663 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@osd.8.service' 2024-09-13T18:36:46.702 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T18:36:46.702 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-13T18:36:46.703 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-13T18:36:46.703 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@osd.9 2024-09-13T18:36:47.015 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:46 smithi114 systemd[1]: Stopping Ceph osd.9 for 97e7edf2-71fb-11ef-bceb-c7b262605968... 2024-09-13T18:36:47.015 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:46 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:46.921+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.588828+0000 front 2024-09-13T18:35:34.588857+0000 (oldest deadline 2024-09-13T18:35:59.288937+0000) 2024-09-13T18:36:47.016 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:46 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:46.921+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.790933+0000 front 2024-09-13T18:35:43.791007+0000 (oldest deadline 2024-09-13T18:36:10.791344+0000) 2024-09-13T18:36:47.016 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:46 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:46.921+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:50.791852+0000 front 2024-09-13T18:35:50.791998+0000 (oldest deadline 2024-09-13T18:36:15.491911+0000) 2024-09-13T18:36:47.016 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:46 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:46.921+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.293330+0000 front 2024-09-13T18:36:01.293344+0000 (oldest deadline 2024-09-13T18:36:25.993496+0000) 2024-09-13T18:36:47.016 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:46 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:46.921+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:08.294646+0000 front 2024-09-13T18:36:08.294716+0000 (oldest deadline 2024-09-13T18:36:34.194541+0000) 2024-09-13T18:36:47.016 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:46 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:46.921+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-13T18:36:17.095551+0000 front 2024-09-13T18:36:17.095653+0000 (oldest deadline 2024-09-13T18:36:42.395537+0000) 2024-09-13T18:36:47.016 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:47 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:47.012+0000 7fe9bcbab640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T18:36:47.016 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:47 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:47.012+0000 7fe9bcbab640 -1 osd.9 111 *** Got signal Terminated *** 2024-09-13T18:36:47.016 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:47 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:47.012+0000 7fe9bcbab640 -1 osd.9 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T18:36:47.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:47 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:47.075+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:47.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:47 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:47.075+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.824009+0000 front 2024-09-13T18:35:43.824043+0000 (oldest deadline 2024-09-13T18:36:07.324119+0000) 2024-09-13T18:36:47.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:47 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:47.075+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.025152+0000 front 2024-09-13T18:35:52.025201+0000 (oldest deadline 2024-09-13T18:36:15.525034+0000) 2024-09-13T18:36:47.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:47 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:47.075+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.326799+0000 front 2024-09-13T18:36:01.326736+0000 (oldest deadline 2024-09-13T18:36:27.226720+0000) 2024-09-13T18:36:47.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:47 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:47.075+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:07.227160+0000 front 2024-09-13T18:36:07.227093+0000 (oldest deadline 2024-09-13T18:36:31.927195+0000) 2024-09-13T18:36:47.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:47 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:47.075+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-13T18:36:17.728557+0000 front 2024-09-13T18:36:17.728465+0000 (oldest deadline 2024-09-13T18:36:43.028465+0000) 2024-09-13T18:36:47.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:47 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:47.166+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:47.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:47 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:47.166+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.187910+0000 front 2024-09-13T18:35:52.187860+0000 (oldest deadline 2024-09-13T18:36:16.287954+0000) 2024-09-13T18:36:47.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:47 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:47.166+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.589086+0000 front 2024-09-13T18:36:01.588873+0000 (oldest deadline 2024-09-13T18:36:24.389897+0000) 2024-09-13T18:36:47.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:47 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:47.166+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:10.091830+0000 front 2024-09-13T18:36:10.091887+0000 (oldest deadline 2024-09-13T18:36:34.791715+0000) 2024-09-13T18:36:47.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:47 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:47.166+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-13T18:36:17.692751+0000 front 2024-09-13T18:36:17.692781+0000 (oldest deadline 2024-09-13T18:36:45.793624+0000) 2024-09-13T18:36:48.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:47 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:47.956+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.588828+0000 front 2024-09-13T18:35:34.588857+0000 (oldest deadline 2024-09-13T18:35:59.288937+0000) 2024-09-13T18:36:48.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:47 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:47.956+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.790933+0000 front 2024-09-13T18:35:43.791007+0000 (oldest deadline 2024-09-13T18:36:10.791344+0000) 2024-09-13T18:36:48.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:47 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:47.956+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:50.791852+0000 front 2024-09-13T18:35:50.791998+0000 (oldest deadline 2024-09-13T18:36:15.491911+0000) 2024-09-13T18:36:48.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:47 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:47.956+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.293330+0000 front 2024-09-13T18:36:01.293344+0000 (oldest deadline 2024-09-13T18:36:25.993496+0000) 2024-09-13T18:36:48.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:47 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:47.956+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:08.294646+0000 front 2024-09-13T18:36:08.294716+0000 (oldest deadline 2024-09-13T18:36:34.194541+0000) 2024-09-13T18:36:48.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:47 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:47.956+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-13T18:36:17.095551+0000 front 2024-09-13T18:36:17.095653+0000 (oldest deadline 2024-09-13T18:36:42.395537+0000) 2024-09-13T18:36:48.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:48 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:48.082+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:48.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:48 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:48.082+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.824009+0000 front 2024-09-13T18:35:43.824043+0000 (oldest deadline 2024-09-13T18:36:07.324119+0000) 2024-09-13T18:36:48.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:48 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:48.082+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.025152+0000 front 2024-09-13T18:35:52.025201+0000 (oldest deadline 2024-09-13T18:36:15.525034+0000) 2024-09-13T18:36:48.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:48 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:48.082+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.326799+0000 front 2024-09-13T18:36:01.326736+0000 (oldest deadline 2024-09-13T18:36:27.226720+0000) 2024-09-13T18:36:48.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:48 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:48.082+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:07.227160+0000 front 2024-09-13T18:36:07.227093+0000 (oldest deadline 2024-09-13T18:36:31.927195+0000) 2024-09-13T18:36:48.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:48 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:48.082+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-13T18:36:17.728557+0000 front 2024-09-13T18:36:17.728465+0000 (oldest deadline 2024-09-13T18:36:43.028465+0000) 2024-09-13T18:36:48.444 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:48 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:48.171+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:48.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:48 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:48.171+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.187910+0000 front 2024-09-13T18:35:52.187860+0000 (oldest deadline 2024-09-13T18:36:16.287954+0000) 2024-09-13T18:36:48.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:48 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:48.171+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.589086+0000 front 2024-09-13T18:36:01.588873+0000 (oldest deadline 2024-09-13T18:36:24.389897+0000) 2024-09-13T18:36:48.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:48 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:48.171+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:10.091830+0000 front 2024-09-13T18:36:10.091887+0000 (oldest deadline 2024-09-13T18:36:34.791715+0000) 2024-09-13T18:36:48.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:48 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:48.171+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-13T18:36:17.692751+0000 front 2024-09-13T18:36:17.692781+0000 (oldest deadline 2024-09-13T18:36:45.793624+0000) 2024-09-13T18:36:49.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:48 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:48.920+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.588828+0000 front 2024-09-13T18:35:34.588857+0000 (oldest deadline 2024-09-13T18:35:59.288937+0000) 2024-09-13T18:36:49.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:48 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:48.920+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.790933+0000 front 2024-09-13T18:35:43.791007+0000 (oldest deadline 2024-09-13T18:36:10.791344+0000) 2024-09-13T18:36:49.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:48 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:48.920+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:50.791852+0000 front 2024-09-13T18:35:50.791998+0000 (oldest deadline 2024-09-13T18:36:15.491911+0000) 2024-09-13T18:36:49.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:48 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:48.920+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.293330+0000 front 2024-09-13T18:36:01.293344+0000 (oldest deadline 2024-09-13T18:36:25.993496+0000) 2024-09-13T18:36:49.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:48 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:48.920+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:08.294646+0000 front 2024-09-13T18:36:08.294716+0000 (oldest deadline 2024-09-13T18:36:34.194541+0000) 2024-09-13T18:36:49.221 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:48 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:48.920+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-13T18:36:17.095551+0000 front 2024-09-13T18:36:17.095653+0000 (oldest deadline 2024-09-13T18:36:42.395537+0000) 2024-09-13T18:36:49.444 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:49 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:49.177+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:49.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:49 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:49.177+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.187910+0000 front 2024-09-13T18:35:52.187860+0000 (oldest deadline 2024-09-13T18:36:16.287954+0000) 2024-09-13T18:36:49.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:49 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:49.177+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.589086+0000 front 2024-09-13T18:36:01.588873+0000 (oldest deadline 2024-09-13T18:36:24.389897+0000) 2024-09-13T18:36:49.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:49 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:49.177+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:10.091830+0000 front 2024-09-13T18:36:10.091887+0000 (oldest deadline 2024-09-13T18:36:34.791715+0000) 2024-09-13T18:36:49.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:49 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:49.177+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-13T18:36:17.692751+0000 front 2024-09-13T18:36:17.692781+0000 (oldest deadline 2024-09-13T18:36:45.793624+0000) 2024-09-13T18:36:49.619 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:49 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:49.130+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:49.619 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:49 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:49.130+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.824009+0000 front 2024-09-13T18:35:43.824043+0000 (oldest deadline 2024-09-13T18:36:07.324119+0000) 2024-09-13T18:36:49.619 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:49 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:49.130+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.025152+0000 front 2024-09-13T18:35:52.025201+0000 (oldest deadline 2024-09-13T18:36:15.525034+0000) 2024-09-13T18:36:49.619 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:49 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:49.130+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.326799+0000 front 2024-09-13T18:36:01.326736+0000 (oldest deadline 2024-09-13T18:36:27.226720+0000) 2024-09-13T18:36:49.619 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:49 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:49.130+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:07.227160+0000 front 2024-09-13T18:36:07.227093+0000 (oldest deadline 2024-09-13T18:36:31.927195+0000) 2024-09-13T18:36:49.620 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:49 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:49.130+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-13T18:36:17.728557+0000 front 2024-09-13T18:36:17.728465+0000 (oldest deadline 2024-09-13T18:36:43.028465+0000) 2024-09-13T18:36:49.620 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:49 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:49.130+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-13T18:36:23.029066+0000 front 2024-09-13T18:36:23.029087+0000 (oldest deadline 2024-09-13T18:36:48.928951+0000) 2024-09-13T18:36:50.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:49 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:49.956+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.588828+0000 front 2024-09-13T18:35:34.588857+0000 (oldest deadline 2024-09-13T18:35:59.288937+0000) 2024-09-13T18:36:50.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:49 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:49.956+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.790933+0000 front 2024-09-13T18:35:43.791007+0000 (oldest deadline 2024-09-13T18:36:10.791344+0000) 2024-09-13T18:36:50.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:49 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:49.956+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:50.791852+0000 front 2024-09-13T18:35:50.791998+0000 (oldest deadline 2024-09-13T18:36:15.491911+0000) 2024-09-13T18:36:50.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:49 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:49.956+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.293330+0000 front 2024-09-13T18:36:01.293344+0000 (oldest deadline 2024-09-13T18:36:25.993496+0000) 2024-09-13T18:36:50.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:49 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:49.956+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:08.294646+0000 front 2024-09-13T18:36:08.294716+0000 (oldest deadline 2024-09-13T18:36:34.194541+0000) 2024-09-13T18:36:50.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:49 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:49.956+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-13T18:36:17.095551+0000 front 2024-09-13T18:36:17.095653+0000 (oldest deadline 2024-09-13T18:36:42.395537+0000) 2024-09-13T18:36:50.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:50 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:50.088+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:50.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:50 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:50.088+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.824009+0000 front 2024-09-13T18:35:43.824043+0000 (oldest deadline 2024-09-13T18:36:07.324119+0000) 2024-09-13T18:36:50.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:50 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:50.088+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.025152+0000 front 2024-09-13T18:35:52.025201+0000 (oldest deadline 2024-09-13T18:36:15.525034+0000) 2024-09-13T18:36:50.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:50 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:50.088+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.326799+0000 front 2024-09-13T18:36:01.326736+0000 (oldest deadline 2024-09-13T18:36:27.226720+0000) 2024-09-13T18:36:50.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:50 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:50.088+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:07.227160+0000 front 2024-09-13T18:36:07.227093+0000 (oldest deadline 2024-09-13T18:36:31.927195+0000) 2024-09-13T18:36:50.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:50 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:50.088+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-13T18:36:17.728557+0000 front 2024-09-13T18:36:17.728465+0000 (oldest deadline 2024-09-13T18:36:43.028465+0000) 2024-09-13T18:36:50.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:50 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:50.088+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-13T18:36:23.029066+0000 front 2024-09-13T18:36:23.029087+0000 (oldest deadline 2024-09-13T18:36:48.928951+0000) 2024-09-13T18:36:50.695 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:50 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:50.194+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:50.695 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:50 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:50.194+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.187910+0000 front 2024-09-13T18:35:52.187860+0000 (oldest deadline 2024-09-13T18:36:16.287954+0000) 2024-09-13T18:36:50.695 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:50 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:50.194+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.589086+0000 front 2024-09-13T18:36:01.588873+0000 (oldest deadline 2024-09-13T18:36:24.389897+0000) 2024-09-13T18:36:50.695 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:50 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:50.194+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:10.091830+0000 front 2024-09-13T18:36:10.091887+0000 (oldest deadline 2024-09-13T18:36:34.791715+0000) 2024-09-13T18:36:50.695 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:50 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:50.194+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-13T18:36:17.692751+0000 front 2024-09-13T18:36:17.692781+0000 (oldest deadline 2024-09-13T18:36:45.793624+0000) 2024-09-13T18:36:51.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:51 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:51.101+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:51.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:51 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:51.101+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.824009+0000 front 2024-09-13T18:35:43.824043+0000 (oldest deadline 2024-09-13T18:36:07.324119+0000) 2024-09-13T18:36:51.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:51 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:51.101+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.025152+0000 front 2024-09-13T18:35:52.025201+0000 (oldest deadline 2024-09-13T18:36:15.525034+0000) 2024-09-13T18:36:51.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:51 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:51.101+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.326799+0000 front 2024-09-13T18:36:01.326736+0000 (oldest deadline 2024-09-13T18:36:27.226720+0000) 2024-09-13T18:36:51.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:51 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:51.101+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:07.227160+0000 front 2024-09-13T18:36:07.227093+0000 (oldest deadline 2024-09-13T18:36:31.927195+0000) 2024-09-13T18:36:51.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:51 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:51.101+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-13T18:36:17.728557+0000 front 2024-09-13T18:36:17.728465+0000 (oldest deadline 2024-09-13T18:36:43.028465+0000) 2024-09-13T18:36:51.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:51 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:51.101+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-13T18:36:23.029066+0000 front 2024-09-13T18:36:23.029087+0000 (oldest deadline 2024-09-13T18:36:48.928951+0000) 2024-09-13T18:36:51.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:51 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:51.150+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:51.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:51 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:51.150+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.187910+0000 front 2024-09-13T18:35:52.187860+0000 (oldest deadline 2024-09-13T18:36:16.287954+0000) 2024-09-13T18:36:51.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:51 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:51.150+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.589086+0000 front 2024-09-13T18:36:01.588873+0000 (oldest deadline 2024-09-13T18:36:24.389897+0000) 2024-09-13T18:36:51.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:51 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:51.150+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:10.091830+0000 front 2024-09-13T18:36:10.091887+0000 (oldest deadline 2024-09-13T18:36:34.791715+0000) 2024-09-13T18:36:51.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:51 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:51.150+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-13T18:36:17.692751+0000 front 2024-09-13T18:36:17.692781+0000 (oldest deadline 2024-09-13T18:36:45.793624+0000) 2024-09-13T18:36:51.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:51 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:51.150+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-13T18:36:27.494541+0000 front 2024-09-13T18:36:27.494598+0000 (oldest deadline 2024-09-13T18:36:50.894740+0000) 2024-09-13T18:36:51.470 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:51 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:51.005+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.588828+0000 front 2024-09-13T18:35:34.588857+0000 (oldest deadline 2024-09-13T18:35:59.288937+0000) 2024-09-13T18:36:51.470 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:51 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:51.005+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.790933+0000 front 2024-09-13T18:35:43.791007+0000 (oldest deadline 2024-09-13T18:36:10.791344+0000) 2024-09-13T18:36:51.470 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:51 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:51.005+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:50.791852+0000 front 2024-09-13T18:35:50.791998+0000 (oldest deadline 2024-09-13T18:36:15.491911+0000) 2024-09-13T18:36:51.470 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:51 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:51.005+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.293330+0000 front 2024-09-13T18:36:01.293344+0000 (oldest deadline 2024-09-13T18:36:25.993496+0000) 2024-09-13T18:36:51.470 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:51 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:51.005+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:08.294646+0000 front 2024-09-13T18:36:08.294716+0000 (oldest deadline 2024-09-13T18:36:34.194541+0000) 2024-09-13T18:36:51.470 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:51 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:51.005+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-13T18:36:17.095551+0000 front 2024-09-13T18:36:17.095653+0000 (oldest deadline 2024-09-13T18:36:42.395537+0000) 2024-09-13T18:36:52.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:52 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:52.058+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:52.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:52 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:52.058+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.824009+0000 front 2024-09-13T18:35:43.824043+0000 (oldest deadline 2024-09-13T18:36:07.324119+0000) 2024-09-13T18:36:52.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:52 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:52.058+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.025152+0000 front 2024-09-13T18:35:52.025201+0000 (oldest deadline 2024-09-13T18:36:15.525034+0000) 2024-09-13T18:36:52.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:52 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:52.058+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.326799+0000 front 2024-09-13T18:36:01.326736+0000 (oldest deadline 2024-09-13T18:36:27.226720+0000) 2024-09-13T18:36:52.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:52 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:52.058+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:07.227160+0000 front 2024-09-13T18:36:07.227093+0000 (oldest deadline 2024-09-13T18:36:31.927195+0000) 2024-09-13T18:36:52.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:52 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:52.058+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-13T18:36:17.728557+0000 front 2024-09-13T18:36:17.728465+0000 (oldest deadline 2024-09-13T18:36:43.028465+0000) 2024-09-13T18:36:52.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:52 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:52.058+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-13T18:36:23.029066+0000 front 2024-09-13T18:36:23.029087+0000 (oldest deadline 2024-09-13T18:36:48.928951+0000) 2024-09-13T18:36:52.444 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:52 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:52.143+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:52.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:52 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:52.143+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.187910+0000 front 2024-09-13T18:35:52.187860+0000 (oldest deadline 2024-09-13T18:36:16.287954+0000) 2024-09-13T18:36:52.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:52 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:52.143+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.589086+0000 front 2024-09-13T18:36:01.588873+0000 (oldest deadline 2024-09-13T18:36:24.389897+0000) 2024-09-13T18:36:52.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:52 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:52.143+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:10.091830+0000 front 2024-09-13T18:36:10.091887+0000 (oldest deadline 2024-09-13T18:36:34.791715+0000) 2024-09-13T18:36:52.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:52 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:52.143+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-13T18:36:17.692751+0000 front 2024-09-13T18:36:17.692781+0000 (oldest deadline 2024-09-13T18:36:45.793624+0000) 2024-09-13T18:36:52.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:52 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:52.143+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-13T18:36:27.494541+0000 front 2024-09-13T18:36:27.494598+0000 (oldest deadline 2024-09-13T18:36:50.894740+0000) 2024-09-13T18:36:52.470 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:52 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:52.002+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.588828+0000 front 2024-09-13T18:35:34.588857+0000 (oldest deadline 2024-09-13T18:35:59.288937+0000) 2024-09-13T18:36:52.470 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:52 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:52.002+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.790933+0000 front 2024-09-13T18:35:43.791007+0000 (oldest deadline 2024-09-13T18:36:10.791344+0000) 2024-09-13T18:36:52.470 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:52 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:52.002+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:50.791852+0000 front 2024-09-13T18:35:50.791998+0000 (oldest deadline 2024-09-13T18:36:15.491911+0000) 2024-09-13T18:36:52.470 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:52 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:52.002+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.293330+0000 front 2024-09-13T18:36:01.293344+0000 (oldest deadline 2024-09-13T18:36:25.993496+0000) 2024-09-13T18:36:52.470 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:52 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:52.002+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:08.294646+0000 front 2024-09-13T18:36:08.294716+0000 (oldest deadline 2024-09-13T18:36:34.194541+0000) 2024-09-13T18:36:52.470 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:52 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9[64763]: 2024-09-13T18:36:52.002+0000 7fe9b91bf640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-13T18:36:17.095551+0000 front 2024-09-13T18:36:17.095653+0000 (oldest deadline 2024-09-13T18:36:42.395537+0000) 2024-09-13T18:36:53.114 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:52 smithi114 podman[111460]: 2024-09-13 18:36:52.828709159 +0000 UTC m=+5.930626302 container died 3907e244d6580f9841a73238d7fe8affa9e198b1b3ce6efb69b5b08d47353930 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2) 2024-09-13T18:36:53.114 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:53 smithi114 podman[111460]: 2024-09-13 18:36:53.112694578 +0000 UTC m=+6.214611722 container cleanup 3907e244d6580f9841a73238d7fe8affa9e198b1b3ce6efb69b5b08d47353930 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083) 2024-09-13T18:36:53.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:53 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:53.048+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:53.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:53 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:53.048+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.824009+0000 front 2024-09-13T18:35:43.824043+0000 (oldest deadline 2024-09-13T18:36:07.324119+0000) 2024-09-13T18:36:53.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:53 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:53.048+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.025152+0000 front 2024-09-13T18:35:52.025201+0000 (oldest deadline 2024-09-13T18:36:15.525034+0000) 2024-09-13T18:36:53.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:53 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:53.048+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.326799+0000 front 2024-09-13T18:36:01.326736+0000 (oldest deadline 2024-09-13T18:36:27.226720+0000) 2024-09-13T18:36:53.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:53 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:53.048+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:07.227160+0000 front 2024-09-13T18:36:07.227093+0000 (oldest deadline 2024-09-13T18:36:31.927195+0000) 2024-09-13T18:36:53.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:53 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:53.048+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-13T18:36:17.728557+0000 front 2024-09-13T18:36:17.728465+0000 (oldest deadline 2024-09-13T18:36:43.028465+0000) 2024-09-13T18:36:53.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:53 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:53.048+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-13T18:36:23.029066+0000 front 2024-09-13T18:36:23.029087+0000 (oldest deadline 2024-09-13T18:36:48.928951+0000) 2024-09-13T18:36:53.444 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:53 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:53.107+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:53.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:53 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:53.107+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.187910+0000 front 2024-09-13T18:35:52.187860+0000 (oldest deadline 2024-09-13T18:36:16.287954+0000) 2024-09-13T18:36:53.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:53 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:53.107+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.589086+0000 front 2024-09-13T18:36:01.588873+0000 (oldest deadline 2024-09-13T18:36:24.389897+0000) 2024-09-13T18:36:53.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:53 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:53.108+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:10.091830+0000 front 2024-09-13T18:36:10.091887+0000 (oldest deadline 2024-09-13T18:36:34.791715+0000) 2024-09-13T18:36:53.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:53 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:53.108+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-13T18:36:17.692751+0000 front 2024-09-13T18:36:17.692781+0000 (oldest deadline 2024-09-13T18:36:45.793624+0000) 2024-09-13T18:36:53.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:53 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:53.108+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-13T18:36:27.494541+0000 front 2024-09-13T18:36:27.494598+0000 (oldest deadline 2024-09-13T18:36:50.894740+0000) 2024-09-13T18:36:53.470 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:53 smithi114 bash[111460]: ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9 2024-09-13T18:36:53.470 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:53 smithi114 podman[111475]: 2024-09-13 18:36:53.338859405 +0000 UTC m=+0.507980810 container remove 3907e244d6580f9841a73238d7fe8affa9e198b1b3ce6efb69b5b08d47353930 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-13T18:36:53.919 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:53 smithi114 podman[111567]: 2024-09-13 18:36:53.550375584 +0000 UTC m=+0.016045214 image pull 8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256 2024-09-13T18:36:53.920 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:53 smithi114 podman[111567]: 2024-09-13 18:36:53.689889716 +0000 UTC m=+0.155559340 container create 1691765c9e6eed18447a8775969181b46fb1080408a8559a8d2b1e54a985ee6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True) 2024-09-13T18:36:54.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:53 smithi114 podman[111567]: 2024-09-13 18:36:53.917772983 +0000 UTC m=+0.383442616 container init 1691765c9e6eed18447a8775969181b46fb1080408a8559a8d2b1e54a985ee6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T18:36:54.220 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:53 smithi114 podman[111567]: 2024-09-13 18:36:53.92145976 +0000 UTC m=+0.387129389 container start 1691765c9e6eed18447a8775969181b46fb1080408a8559a8d2b1e54a985ee6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.license=GPLv2) 2024-09-13T18:36:54.221 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:53 smithi114 podman[111567]: 2024-09-13 18:36:53.992399846 +0000 UTC m=+0.458069490 container attach 1691765c9e6eed18447a8775969181b46fb1080408a8559a8d2b1e54a985ee6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-13T18:36:54.221 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 13 18:36:54 smithi114 podman[111567]: 2024-09-13 18:36:54.105316819 +0000 UTC m=+0.570986444 container died 1691765c9e6eed18447a8775969181b46fb1080408a8559a8d2b1e54a985ee6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-13T18:36:54.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:54 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:54.075+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:54.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:54 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:54.075+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.824009+0000 front 2024-09-13T18:35:43.824043+0000 (oldest deadline 2024-09-13T18:36:07.324119+0000) 2024-09-13T18:36:54.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:54 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:54.075+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.025152+0000 front 2024-09-13T18:35:52.025201+0000 (oldest deadline 2024-09-13T18:36:15.525034+0000) 2024-09-13T18:36:54.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:54 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:54.075+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.326799+0000 front 2024-09-13T18:36:01.326736+0000 (oldest deadline 2024-09-13T18:36:27.226720+0000) 2024-09-13T18:36:54.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:54 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:54.075+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:07.227160+0000 front 2024-09-13T18:36:07.227093+0000 (oldest deadline 2024-09-13T18:36:31.927195+0000) 2024-09-13T18:36:54.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:54 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:54.075+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-13T18:36:17.728557+0000 front 2024-09-13T18:36:17.728465+0000 (oldest deadline 2024-09-13T18:36:43.028465+0000) 2024-09-13T18:36:54.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:54 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:54.075+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-13T18:36:23.029066+0000 front 2024-09-13T18:36:23.029087+0000 (oldest deadline 2024-09-13T18:36:48.928951+0000) 2024-09-13T18:36:54.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:54 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:54.080+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:54.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:54 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:54.080+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.187910+0000 front 2024-09-13T18:35:52.187860+0000 (oldest deadline 2024-09-13T18:36:16.287954+0000) 2024-09-13T18:36:54.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:54 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:54.080+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.589086+0000 front 2024-09-13T18:36:01.588873+0000 (oldest deadline 2024-09-13T18:36:24.389897+0000) 2024-09-13T18:36:54.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:54 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:54.080+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:10.091830+0000 front 2024-09-13T18:36:10.091887+0000 (oldest deadline 2024-09-13T18:36:34.791715+0000) 2024-09-13T18:36:54.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:54 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:54.080+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-13T18:36:17.692751+0000 front 2024-09-13T18:36:17.692781+0000 (oldest deadline 2024-09-13T18:36:45.793624+0000) 2024-09-13T18:36:54.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:54 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:54.080+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-13T18:36:27.494541+0000 front 2024-09-13T18:36:27.494598+0000 (oldest deadline 2024-09-13T18:36:50.894740+0000) 2024-09-13T18:36:54.725 DEBUG:teuthology.orchestra.run.smithi114:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@osd.9.service' 2024-09-13T18:36:54.764 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T18:36:54.764 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-13T18:36:54.764 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-13T18:36:54.765 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@osd.10 2024-09-13T18:36:55.055 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:54 smithi134 systemd[1]: Stopping Ceph osd.10 for 97e7edf2-71fb-11ef-bceb-c7b262605968... 2024-09-13T18:36:55.055 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:55 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:55.052+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:55.056 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:55 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:55.052+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.824009+0000 front 2024-09-13T18:35:43.824043+0000 (oldest deadline 2024-09-13T18:36:07.324119+0000) 2024-09-13T18:36:55.056 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:55 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:55.052+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.025152+0000 front 2024-09-13T18:35:52.025201+0000 (oldest deadline 2024-09-13T18:36:15.525034+0000) 2024-09-13T18:36:55.056 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:55 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:55.052+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.326799+0000 front 2024-09-13T18:36:01.326736+0000 (oldest deadline 2024-09-13T18:36:27.226720+0000) 2024-09-13T18:36:55.056 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:55 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:55.052+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:07.227160+0000 front 2024-09-13T18:36:07.227093+0000 (oldest deadline 2024-09-13T18:36:31.927195+0000) 2024-09-13T18:36:55.056 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:55 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:55.052+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-13T18:36:17.728557+0000 front 2024-09-13T18:36:17.728465+0000 (oldest deadline 2024-09-13T18:36:43.028465+0000) 2024-09-13T18:36:55.056 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:55 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:55.052+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-13T18:36:23.029066+0000 front 2024-09-13T18:36:23.029087+0000 (oldest deadline 2024-09-13T18:36:48.928951+0000) 2024-09-13T18:36:55.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:55 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:55.086+0000 7fe0b58c9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T18:36:55.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:55 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:55.086+0000 7fe0b58c9640 -1 osd.10 111 *** Got signal Terminated *** 2024-09-13T18:36:55.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:55 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:55.086+0000 7fe0b58c9640 -1 osd.10 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T18:36:55.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:55 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:55.034+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:55.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:55 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:55.034+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.187910+0000 front 2024-09-13T18:35:52.187860+0000 (oldest deadline 2024-09-13T18:36:16.287954+0000) 2024-09-13T18:36:55.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:55 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:55.034+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.589086+0000 front 2024-09-13T18:36:01.588873+0000 (oldest deadline 2024-09-13T18:36:24.389897+0000) 2024-09-13T18:36:55.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:55 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:55.034+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:10.091830+0000 front 2024-09-13T18:36:10.091887+0000 (oldest deadline 2024-09-13T18:36:34.791715+0000) 2024-09-13T18:36:55.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:55 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:55.034+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-13T18:36:17.692751+0000 front 2024-09-13T18:36:17.692781+0000 (oldest deadline 2024-09-13T18:36:45.793624+0000) 2024-09-13T18:36:55.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:55 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:55.034+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-13T18:36:27.494541+0000 front 2024-09-13T18:36:27.494598+0000 (oldest deadline 2024-09-13T18:36:50.894740+0000) 2024-09-13T18:36:56.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:56 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:56.070+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:56.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:56 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:56.070+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.824009+0000 front 2024-09-13T18:35:43.824043+0000 (oldest deadline 2024-09-13T18:36:07.324119+0000) 2024-09-13T18:36:56.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:56 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:56.070+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.025152+0000 front 2024-09-13T18:35:52.025201+0000 (oldest deadline 2024-09-13T18:36:15.525034+0000) 2024-09-13T18:36:56.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:56 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:56.070+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.326799+0000 front 2024-09-13T18:36:01.326736+0000 (oldest deadline 2024-09-13T18:36:27.226720+0000) 2024-09-13T18:36:56.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:56 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:56.070+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:07.227160+0000 front 2024-09-13T18:36:07.227093+0000 (oldest deadline 2024-09-13T18:36:31.927195+0000) 2024-09-13T18:36:56.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:56 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:56.070+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-13T18:36:17.728557+0000 front 2024-09-13T18:36:17.728465+0000 (oldest deadline 2024-09-13T18:36:43.028465+0000) 2024-09-13T18:36:56.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:56 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:56.070+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-13T18:36:23.029066+0000 front 2024-09-13T18:36:23.029087+0000 (oldest deadline 2024-09-13T18:36:48.928951+0000) 2024-09-13T18:36:56.444 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:56 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:56.051+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:56.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:56 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:56.051+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.187910+0000 front 2024-09-13T18:35:52.187860+0000 (oldest deadline 2024-09-13T18:36:16.287954+0000) 2024-09-13T18:36:56.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:56 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:56.051+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.589086+0000 front 2024-09-13T18:36:01.588873+0000 (oldest deadline 2024-09-13T18:36:24.389897+0000) 2024-09-13T18:36:56.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:56 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:56.051+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:10.091830+0000 front 2024-09-13T18:36:10.091887+0000 (oldest deadline 2024-09-13T18:36:34.791715+0000) 2024-09-13T18:36:56.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:56 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:56.051+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-13T18:36:17.692751+0000 front 2024-09-13T18:36:17.692781+0000 (oldest deadline 2024-09-13T18:36:45.793624+0000) 2024-09-13T18:36:56.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:56 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:56.051+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-13T18:36:27.494541+0000 front 2024-09-13T18:36:27.494598+0000 (oldest deadline 2024-09-13T18:36:50.894740+0000) 2024-09-13T18:36:57.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:57 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:57.095+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:57.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:57 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:57.095+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.824009+0000 front 2024-09-13T18:35:43.824043+0000 (oldest deadline 2024-09-13T18:36:07.324119+0000) 2024-09-13T18:36:57.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:57 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:57.095+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.025152+0000 front 2024-09-13T18:35:52.025201+0000 (oldest deadline 2024-09-13T18:36:15.525034+0000) 2024-09-13T18:36:57.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:57 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:57.095+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.326799+0000 front 2024-09-13T18:36:01.326736+0000 (oldest deadline 2024-09-13T18:36:27.226720+0000) 2024-09-13T18:36:57.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:57 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:57.095+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:07.227160+0000 front 2024-09-13T18:36:07.227093+0000 (oldest deadline 2024-09-13T18:36:31.927195+0000) 2024-09-13T18:36:57.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:57 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:57.095+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-13T18:36:17.728557+0000 front 2024-09-13T18:36:17.728465+0000 (oldest deadline 2024-09-13T18:36:43.028465+0000) 2024-09-13T18:36:57.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:57 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:57.095+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-13T18:36:23.029066+0000 front 2024-09-13T18:36:23.029087+0000 (oldest deadline 2024-09-13T18:36:48.928951+0000) 2024-09-13T18:36:57.444 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:57 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:57.031+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:57.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:57 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:57.031+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.187910+0000 front 2024-09-13T18:35:52.187860+0000 (oldest deadline 2024-09-13T18:36:16.287954+0000) 2024-09-13T18:36:57.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:57 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:57.031+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.589086+0000 front 2024-09-13T18:36:01.588873+0000 (oldest deadline 2024-09-13T18:36:24.389897+0000) 2024-09-13T18:36:57.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:57 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:57.031+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:10.091830+0000 front 2024-09-13T18:36:10.091887+0000 (oldest deadline 2024-09-13T18:36:34.791715+0000) 2024-09-13T18:36:57.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:57 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:57.031+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-13T18:36:17.692751+0000 front 2024-09-13T18:36:17.692781+0000 (oldest deadline 2024-09-13T18:36:45.793624+0000) 2024-09-13T18:36:57.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:57 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:57.031+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-13T18:36:27.494541+0000 front 2024-09-13T18:36:27.494598+0000 (oldest deadline 2024-09-13T18:36:50.894740+0000) 2024-09-13T18:36:58.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:58 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:58.081+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:58.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:58 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:58.081+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.824009+0000 front 2024-09-13T18:35:43.824043+0000 (oldest deadline 2024-09-13T18:36:07.324119+0000) 2024-09-13T18:36:58.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:58 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:58.081+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.025152+0000 front 2024-09-13T18:35:52.025201+0000 (oldest deadline 2024-09-13T18:36:15.525034+0000) 2024-09-13T18:36:58.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:58 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:58.081+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.326799+0000 front 2024-09-13T18:36:01.326736+0000 (oldest deadline 2024-09-13T18:36:27.226720+0000) 2024-09-13T18:36:58.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:58 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:58.081+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:07.227160+0000 front 2024-09-13T18:36:07.227093+0000 (oldest deadline 2024-09-13T18:36:31.927195+0000) 2024-09-13T18:36:58.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:58 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:58.081+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-13T18:36:17.728557+0000 front 2024-09-13T18:36:17.728465+0000 (oldest deadline 2024-09-13T18:36:43.028465+0000) 2024-09-13T18:36:58.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:58 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:58.081+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-13T18:36:23.029066+0000 front 2024-09-13T18:36:23.029087+0000 (oldest deadline 2024-09-13T18:36:48.928951+0000) 2024-09-13T18:36:58.444 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:57 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:57.987+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:58.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:57 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:57.987+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.187910+0000 front 2024-09-13T18:35:52.187860+0000 (oldest deadline 2024-09-13T18:36:16.287954+0000) 2024-09-13T18:36:58.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:57 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:57.987+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.589086+0000 front 2024-09-13T18:36:01.588873+0000 (oldest deadline 2024-09-13T18:36:24.389897+0000) 2024-09-13T18:36:58.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:57 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:57.987+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:10.091830+0000 front 2024-09-13T18:36:10.091887+0000 (oldest deadline 2024-09-13T18:36:34.791715+0000) 2024-09-13T18:36:58.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:57 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:57.987+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-13T18:36:17.692751+0000 front 2024-09-13T18:36:17.692781+0000 (oldest deadline 2024-09-13T18:36:45.793624+0000) 2024-09-13T18:36:58.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:57 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:57.987+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-13T18:36:27.494541+0000 front 2024-09-13T18:36:27.494598+0000 (oldest deadline 2024-09-13T18:36:50.894740+0000) 2024-09-13T18:36:59.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:59 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:59.088+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:36:59.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:59 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:59.088+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.824009+0000 front 2024-09-13T18:35:43.824043+0000 (oldest deadline 2024-09-13T18:36:07.324119+0000) 2024-09-13T18:36:59.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:59 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:59.088+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.025152+0000 front 2024-09-13T18:35:52.025201+0000 (oldest deadline 2024-09-13T18:36:15.525034+0000) 2024-09-13T18:36:59.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:59 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:59.088+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.326799+0000 front 2024-09-13T18:36:01.326736+0000 (oldest deadline 2024-09-13T18:36:27.226720+0000) 2024-09-13T18:36:59.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:59 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:59.088+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:07.227160+0000 front 2024-09-13T18:36:07.227093+0000 (oldest deadline 2024-09-13T18:36:31.927195+0000) 2024-09-13T18:36:59.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:59 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:59.088+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-13T18:36:17.728557+0000 front 2024-09-13T18:36:17.728465+0000 (oldest deadline 2024-09-13T18:36:43.028465+0000) 2024-09-13T18:36:59.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:36:59 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:36:59.088+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-13T18:36:23.029066+0000 front 2024-09-13T18:36:23.029087+0000 (oldest deadline 2024-09-13T18:36:48.928951+0000) 2024-09-13T18:36:59.398 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:58 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:58.969+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:36:59.399 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:58 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:58.969+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.187910+0000 front 2024-09-13T18:35:52.187860+0000 (oldest deadline 2024-09-13T18:36:16.287954+0000) 2024-09-13T18:36:59.399 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:58 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:58.969+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.589086+0000 front 2024-09-13T18:36:01.588873+0000 (oldest deadline 2024-09-13T18:36:24.389897+0000) 2024-09-13T18:36:59.399 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:58 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:58.969+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:10.091830+0000 front 2024-09-13T18:36:10.091887+0000 (oldest deadline 2024-09-13T18:36:34.791715+0000) 2024-09-13T18:36:59.399 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:58 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:58.969+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-13T18:36:17.692751+0000 front 2024-09-13T18:36:17.692781+0000 (oldest deadline 2024-09-13T18:36:45.793624+0000) 2024-09-13T18:36:59.399 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:58 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:58.969+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-13T18:36:27.494541+0000 front 2024-09-13T18:36:27.494598+0000 (oldest deadline 2024-09-13T18:36:50.894740+0000) 2024-09-13T18:37:00.194 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:59 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:59.922+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:37:00.195 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:59 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:59.922+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.187910+0000 front 2024-09-13T18:35:52.187860+0000 (oldest deadline 2024-09-13T18:36:16.287954+0000) 2024-09-13T18:37:00.195 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:59 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:59.922+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.589086+0000 front 2024-09-13T18:36:01.588873+0000 (oldest deadline 2024-09-13T18:36:24.389897+0000) 2024-09-13T18:37:00.195 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:59 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:59.922+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:10.091830+0000 front 2024-09-13T18:36:10.091887+0000 (oldest deadline 2024-09-13T18:36:34.791715+0000) 2024-09-13T18:37:00.195 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:59 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:59.922+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-13T18:36:17.692751+0000 front 2024-09-13T18:36:17.692781+0000 (oldest deadline 2024-09-13T18:36:45.793624+0000) 2024-09-13T18:37:00.195 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:36:59 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:36:59.922+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-13T18:36:27.494541+0000 front 2024-09-13T18:36:27.494598+0000 (oldest deadline 2024-09-13T18:36:50.894740+0000) 2024-09-13T18:37:00.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:37:00 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:37:00.078+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:34.423305+0000 front 2024-09-13T18:35:34.423211+0000 (oldest deadline 2024-09-13T18:35:57.923280+0000) 2024-09-13T18:37:00.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:37:00 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:37:00.078+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-13T18:35:43.824009+0000 front 2024-09-13T18:35:43.824043+0000 (oldest deadline 2024-09-13T18:36:07.324119+0000) 2024-09-13T18:37:00.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:37:00 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:37:00.078+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.025152+0000 front 2024-09-13T18:35:52.025201+0000 (oldest deadline 2024-09-13T18:36:15.525034+0000) 2024-09-13T18:37:00.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:37:00 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:37:00.078+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.326799+0000 front 2024-09-13T18:36:01.326736+0000 (oldest deadline 2024-09-13T18:36:27.226720+0000) 2024-09-13T18:37:00.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:37:00 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:37:00.078+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:07.227160+0000 front 2024-09-13T18:36:07.227093+0000 (oldest deadline 2024-09-13T18:36:31.927195+0000) 2024-09-13T18:37:00.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:37:00 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:37:00.078+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-13T18:36:17.728557+0000 front 2024-09-13T18:36:17.728465+0000 (oldest deadline 2024-09-13T18:36:43.028465+0000) 2024-09-13T18:37:00.370 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:37:00 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10[58739]: 2024-09-13T18:37:00.078+0000 7fe0b1edd640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-13T18:36:23.029066+0000 front 2024-09-13T18:36:23.029087+0000 (oldest deadline 2024-09-13T18:36:48.928951+0000) 2024-09-13T18:37:01.003 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:37:00 smithi134 podman[70557]: 2024-09-13 18:37:00.731887845 +0000 UTC m=+5.766000454 container died 9ac9b7f8e79c16d62eda21df609ed7d2f7d0fa93b6e3e7b87c9d2f9f7b719394 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-13T18:37:01.194 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:00 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:00.920+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:37:01.195 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:00 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:00.920+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.187910+0000 front 2024-09-13T18:35:52.187860+0000 (oldest deadline 2024-09-13T18:36:16.287954+0000) 2024-09-13T18:37:01.195 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:00 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:00.920+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.589086+0000 front 2024-09-13T18:36:01.588873+0000 (oldest deadline 2024-09-13T18:36:24.389897+0000) 2024-09-13T18:37:01.195 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:00 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:00.920+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:10.091830+0000 front 2024-09-13T18:36:10.091887+0000 (oldest deadline 2024-09-13T18:36:34.791715+0000) 2024-09-13T18:37:01.195 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:00 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:00.920+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-13T18:36:17.692751+0000 front 2024-09-13T18:36:17.692781+0000 (oldest deadline 2024-09-13T18:36:45.793624+0000) 2024-09-13T18:37:01.195 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:00 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:00.920+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-13T18:36:27.494541+0000 front 2024-09-13T18:36:27.494598+0000 (oldest deadline 2024-09-13T18:36:50.894740+0000) 2024-09-13T18:37:01.195 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:00 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:00.920+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2024-09-13T18:36:35.496161+0000 front 2024-09-13T18:36:35.496236+0000 (oldest deadline 2024-09-13T18:37:00.796017+0000) 2024-09-13T18:37:01.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:37:01 smithi134 podman[70557]: 2024-09-13 18:37:01.099773651 +0000 UTC m=+6.133886259 container cleanup 9ac9b7f8e79c16d62eda21df609ed7d2f7d0fa93b6e3e7b87c9d2f9f7b719394 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-13T18:37:01.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:37:01 smithi134 bash[70557]: ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10 2024-09-13T18:37:01.712 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:37:01 smithi134 podman[70574]: 2024-09-13 18:37:01.384228733 +0000 UTC m=+0.650630811 container remove 9ac9b7f8e79c16d62eda21df609ed7d2f7d0fa93b6e3e7b87c9d2f9f7b719394 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-13T18:37:02.040 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:37:01 smithi134 podman[70658]: 2024-09-13 18:37:01.710493323 +0000 UTC m=+0.100254400 container create 2508af16b0b3a4d9ea460e7fb39c62a5ee4bea96ef02d6e0be006ddd52a65595 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-13T18:37:02.040 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:37:01 smithi134 podman[70658]: 2024-09-13 18:37:01.626282662 +0000 UTC m=+0.016043680 image pull 8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256 2024-09-13T18:37:02.040 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:37:01 smithi134 podman[70658]: 2024-09-13 18:37:01.954720333 +0000 UTC m=+0.344481351 container init 2508af16b0b3a4d9ea460e7fb39c62a5ee4bea96ef02d6e0be006ddd52a65595 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True) 2024-09-13T18:37:02.040 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:37:01 smithi134 podman[70658]: 2024-09-13 18:37:01.958044291 +0000 UTC m=+0.347805311 container start 2508af16b0b3a4d9ea460e7fb39c62a5ee4bea96ef02d6e0be006ddd52a65595 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-13T18:37:02.040 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:37:02 smithi134 podman[70658]: 2024-09-13 18:37:02.038096256 +0000 UTC m=+0.427857281 container attach 2508af16b0b3a4d9ea460e7fb39c62a5ee4bea96ef02d6e0be006ddd52a65595 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-13T18:37:02.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 13 18:37:02 smithi134 podman[70658]: 2024-09-13 18:37:02.144018931 +0000 UTC m=+0.533779950 container died 2508af16b0b3a4d9ea460e7fb39c62a5ee4bea96ef02d6e0be006ddd52a65595 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-10-deactivate, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-13T18:37:02.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:01 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:01.950+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:37:02.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:01 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:01.950+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.187910+0000 front 2024-09-13T18:35:52.187860+0000 (oldest deadline 2024-09-13T18:36:16.287954+0000) 2024-09-13T18:37:02.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:01 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:01.950+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.589086+0000 front 2024-09-13T18:36:01.588873+0000 (oldest deadline 2024-09-13T18:36:24.389897+0000) 2024-09-13T18:37:02.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:01 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:01.950+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:10.091830+0000 front 2024-09-13T18:36:10.091887+0000 (oldest deadline 2024-09-13T18:36:34.791715+0000) 2024-09-13T18:37:02.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:01 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:01.950+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-13T18:36:17.692751+0000 front 2024-09-13T18:36:17.692781+0000 (oldest deadline 2024-09-13T18:36:45.793624+0000) 2024-09-13T18:37:02.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:01 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:01.950+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-13T18:36:27.494541+0000 front 2024-09-13T18:36:27.494598+0000 (oldest deadline 2024-09-13T18:36:50.894740+0000) 2024-09-13T18:37:02.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:01 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:01.950+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2024-09-13T18:36:35.496161+0000 front 2024-09-13T18:36:35.496236+0000 (oldest deadline 2024-09-13T18:37:00.796017+0000) 2024-09-13T18:37:02.745 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@osd.10.service' 2024-09-13T18:37:02.783 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T18:37:02.783 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-13T18:37:02.783 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-13T18:37:02.783 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@osd.11 2024-09-13T18:37:03.091 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:02 smithi204 systemd[1]: Stopping Ceph osd.11 for 97e7edf2-71fb-11ef-bceb-c7b262605968... 2024-09-13T18:37:03.091 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:02 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:02.910+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:37:03.091 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:02 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:02.910+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.187910+0000 front 2024-09-13T18:35:52.187860+0000 (oldest deadline 2024-09-13T18:36:16.287954+0000) 2024-09-13T18:37:03.091 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:02 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:02.910+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.589086+0000 front 2024-09-13T18:36:01.588873+0000 (oldest deadline 2024-09-13T18:36:24.389897+0000) 2024-09-13T18:37:03.091 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:02 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:02.910+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:10.091830+0000 front 2024-09-13T18:36:10.091887+0000 (oldest deadline 2024-09-13T18:36:34.791715+0000) 2024-09-13T18:37:03.091 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:02 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:02.910+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-13T18:36:17.692751+0000 front 2024-09-13T18:36:17.692781+0000 (oldest deadline 2024-09-13T18:36:45.793624+0000) 2024-09-13T18:37:03.091 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:02 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:02.910+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-13T18:36:27.494541+0000 front 2024-09-13T18:36:27.494598+0000 (oldest deadline 2024-09-13T18:36:50.894740+0000) 2024-09-13T18:37:03.091 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:02 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:02.910+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2024-09-13T18:36:35.496161+0000 front 2024-09-13T18:36:35.496236+0000 (oldest deadline 2024-09-13T18:37:00.796017+0000) 2024-09-13T18:37:03.444 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:03 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:03.088+0000 7f430246a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T18:37:03.444 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:03 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:03.088+0000 7f430246a640 -1 osd.11 111 *** Got signal Terminated *** 2024-09-13T18:37:03.444 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:03 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:03.088+0000 7f430246a640 -1 osd.11 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-13T18:37:04.194 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:03 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:03.931+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:37:04.195 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:03 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:03.931+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.187910+0000 front 2024-09-13T18:35:52.187860+0000 (oldest deadline 2024-09-13T18:36:16.287954+0000) 2024-09-13T18:37:04.195 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:03 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:03.931+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.589086+0000 front 2024-09-13T18:36:01.588873+0000 (oldest deadline 2024-09-13T18:36:24.389897+0000) 2024-09-13T18:37:04.195 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:03 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:03.931+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:10.091830+0000 front 2024-09-13T18:36:10.091887+0000 (oldest deadline 2024-09-13T18:36:34.791715+0000) 2024-09-13T18:37:04.195 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:03 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:03.931+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-13T18:36:17.692751+0000 front 2024-09-13T18:36:17.692781+0000 (oldest deadline 2024-09-13T18:36:45.793624+0000) 2024-09-13T18:37:04.195 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:03 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:03.931+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-13T18:36:27.494541+0000 front 2024-09-13T18:36:27.494598+0000 (oldest deadline 2024-09-13T18:36:50.894740+0000) 2024-09-13T18:37:04.195 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:03 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:03.931+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2024-09-13T18:36:35.496161+0000 front 2024-09-13T18:36:35.496236+0000 (oldest deadline 2024-09-13T18:37:00.796017+0000) 2024-09-13T18:37:05.444 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:04 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:04.974+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:37:05.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:04 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:04.974+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.187910+0000 front 2024-09-13T18:35:52.187860+0000 (oldest deadline 2024-09-13T18:36:16.287954+0000) 2024-09-13T18:37:05.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:04 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:04.974+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.589086+0000 front 2024-09-13T18:36:01.588873+0000 (oldest deadline 2024-09-13T18:36:24.389897+0000) 2024-09-13T18:37:05.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:04 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:04.974+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:10.091830+0000 front 2024-09-13T18:36:10.091887+0000 (oldest deadline 2024-09-13T18:36:34.791715+0000) 2024-09-13T18:37:05.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:04 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:04.974+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-13T18:36:17.692751+0000 front 2024-09-13T18:36:17.692781+0000 (oldest deadline 2024-09-13T18:36:45.793624+0000) 2024-09-13T18:37:05.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:04 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:04.974+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-13T18:36:27.494541+0000 front 2024-09-13T18:36:27.494598+0000 (oldest deadline 2024-09-13T18:36:50.894740+0000) 2024-09-13T18:37:05.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:04 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:04.974+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2024-09-13T18:36:35.496161+0000 front 2024-09-13T18:36:35.496236+0000 (oldest deadline 2024-09-13T18:37:00.796017+0000) 2024-09-13T18:37:06.444 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:06 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:06.020+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:37:06.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:06 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:06.020+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.187910+0000 front 2024-09-13T18:35:52.187860+0000 (oldest deadline 2024-09-13T18:36:16.287954+0000) 2024-09-13T18:37:06.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:06 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:06.020+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.589086+0000 front 2024-09-13T18:36:01.588873+0000 (oldest deadline 2024-09-13T18:36:24.389897+0000) 2024-09-13T18:37:06.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:06 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:06.020+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:10.091830+0000 front 2024-09-13T18:36:10.091887+0000 (oldest deadline 2024-09-13T18:36:34.791715+0000) 2024-09-13T18:37:06.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:06 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:06.020+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-13T18:36:17.692751+0000 front 2024-09-13T18:36:17.692781+0000 (oldest deadline 2024-09-13T18:36:45.793624+0000) 2024-09-13T18:37:06.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:06 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:06.020+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-13T18:36:27.494541+0000 front 2024-09-13T18:36:27.494598+0000 (oldest deadline 2024-09-13T18:36:50.894740+0000) 2024-09-13T18:37:06.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:06 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:06.020+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2024-09-13T18:36:35.496161+0000 front 2024-09-13T18:36:35.496236+0000 (oldest deadline 2024-09-13T18:37:00.796017+0000) 2024-09-13T18:37:07.444 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:07 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:07.036+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:37:07.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:07 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:07.036+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.187910+0000 front 2024-09-13T18:35:52.187860+0000 (oldest deadline 2024-09-13T18:36:16.287954+0000) 2024-09-13T18:37:07.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:07 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:07.036+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.589086+0000 front 2024-09-13T18:36:01.588873+0000 (oldest deadline 2024-09-13T18:36:24.389897+0000) 2024-09-13T18:37:07.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:07 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:07.036+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:10.091830+0000 front 2024-09-13T18:36:10.091887+0000 (oldest deadline 2024-09-13T18:36:34.791715+0000) 2024-09-13T18:37:07.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:07 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:07.036+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-13T18:36:17.692751+0000 front 2024-09-13T18:36:17.692781+0000 (oldest deadline 2024-09-13T18:36:45.793624+0000) 2024-09-13T18:37:07.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:07 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:07.036+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-13T18:36:27.494541+0000 front 2024-09-13T18:36:27.494598+0000 (oldest deadline 2024-09-13T18:36:50.894740+0000) 2024-09-13T18:37:07.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:07 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:07.036+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2024-09-13T18:36:35.496161+0000 front 2024-09-13T18:36:35.496236+0000 (oldest deadline 2024-09-13T18:37:00.796017+0000) 2024-09-13T18:37:08.444 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:08 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:08.000+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-13T18:35:36.584960+0000 front 2024-09-13T18:35:36.584983+0000 (oldest deadline 2024-09-13T18:36:01.285153+0000) 2024-09-13T18:37:08.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:08 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:08.000+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-13T18:35:52.187910+0000 front 2024-09-13T18:35:52.187860+0000 (oldest deadline 2024-09-13T18:36:16.287954+0000) 2024-09-13T18:37:08.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:08 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:08.000+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-13T18:36:01.589086+0000 front 2024-09-13T18:36:01.588873+0000 (oldest deadline 2024-09-13T18:36:24.389897+0000) 2024-09-13T18:37:08.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:08 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:08.000+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-13T18:36:10.091830+0000 front 2024-09-13T18:36:10.091887+0000 (oldest deadline 2024-09-13T18:36:34.791715+0000) 2024-09-13T18:37:08.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:08 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:08.000+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-13T18:36:17.692751+0000 front 2024-09-13T18:36:17.692781+0000 (oldest deadline 2024-09-13T18:36:45.793624+0000) 2024-09-13T18:37:08.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:08 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:08.000+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-13T18:36:27.494541+0000 front 2024-09-13T18:36:27.494598+0000 (oldest deadline 2024-09-13T18:36:50.894740+0000) 2024-09-13T18:37:08.445 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:08 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11[59516]: 2024-09-13T18:37:08.000+0000 7f42faa76640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2024-09-13T18:36:35.496161+0000 front 2024-09-13T18:36:35.496236+0000 (oldest deadline 2024-09-13T18:37:00.796017+0000) 2024-09-13T18:37:08.916 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:08 smithi204 podman[71820]: 2024-09-13 18:37:08.649856484 +0000 UTC m=+5.665620732 container died 8236824a6098f52bc6a9d01e1b4f4b392c8fbcbd871df54a520a6add7208d892 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, maintainer=Guillaume Abrioux ) 2024-09-13T18:37:09.194 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:09 smithi204 podman[71820]: 2024-09-13 18:37:09.012800176 +0000 UTC m=+6.028564463 container cleanup 8236824a6098f52bc6a9d01e1b4f4b392c8fbcbd871df54a520a6add7208d892 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-13T18:37:09.195 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:09 smithi204 bash[71820]: ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11 2024-09-13T18:37:09.623 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:09 smithi204 podman[71836]: 2024-09-13 18:37:09.288739182 +0000 UTC m=+0.632989975 container remove 8236824a6098f52bc6a9d01e1b4f4b392c8fbcbd871df54a520a6add7208d892 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-13T18:37:09.935 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:09 smithi204 podman[71926]: 2024-09-13 18:37:09.52422904 +0000 UTC m=+0.015820189 image pull 8a91f241d7d4e1d6e26ee4f6ee67f99a623a7da71bd993082f39318b3b5efc61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256 2024-09-13T18:37:09.935 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:09 smithi204 podman[71926]: 2024-09-13 18:37:09.622821864 +0000 UTC m=+0.114413058 container create 2951d0c263b7b4267043253dbdcc37287b462a53606aef0c8648b5f6df9aa850 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-13T18:37:09.935 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:09 smithi204 podman[71926]: 2024-09-13 18:37:09.85011207 +0000 UTC m=+0.341703212 container init 2951d0c263b7b4267043253dbdcc37287b462a53606aef0c8648b5f6df9aa850 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-13T18:37:09.935 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:09 smithi204 podman[71926]: 2024-09-13 18:37:09.853546138 +0000 UTC m=+0.345137280 container start 2951d0c263b7b4267043253dbdcc37287b462a53606aef0c8648b5f6df9aa850 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-13T18:37:09.936 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:09 smithi204 podman[71926]: 2024-09-13 18:37:09.933322587 +0000 UTC m=+0.424913727 container attach 2951d0c263b7b4267043253dbdcc37287b462a53606aef0c8648b5f6df9aa850 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T18:37:10.194 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 13 18:37:10 smithi204 podman[71926]: 2024-09-13 18:37:10.052277402 +0000 UTC m=+0.543868549 container died 2951d0c263b7b4267043253dbdcc37287b462a53606aef0c8648b5f6df9aa850 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-13T18:37:10.607 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@osd.11.service' 2024-09-13T18:37:10.644 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T18:37:10.645 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-13T18:37:10.645 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-13T18:37:10.645 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mds.a 2024-09-13T18:37:10.949 INFO:journalctl@ceph.mds.a.smithi114.stdout:Sep 13 18:37:10 smithi114 systemd[1]: Stopping Ceph mds.a for 97e7edf2-71fb-11ef-bceb-c7b262605968... 2024-09-13T18:37:11.220 INFO:journalctl@ceph.mds.a.smithi114.stdout:Sep 13 18:37:10 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-a[71215]: 2024-09-13T18:37:10.947+0000 7f3af4ebb640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T18:37:11.220 INFO:journalctl@ceph.mds.a.smithi114.stdout:Sep 13 18:37:10 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-a[71215]: 2024-09-13T18:37:10.947+0000 7f3af4ebb640 -1 mds.a *** got signal Terminated *** 2024-09-13T18:37:15.220 INFO:journalctl@ceph.mds.a.smithi114.stdout:Sep 13 18:37:14 smithi114 podman[111667]: 2024-09-13 18:37:14.955441846 +0000 UTC m=+4.113383466 container died c8f4947b120ffba2412be0786d1097022fb9a1a6e996ca9be9c3fc1cec61307c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-13T18:37:15.600 INFO:journalctl@ceph.mds.a.smithi114.stdout:Sep 13 18:37:15 smithi114 podman[111667]: 2024-09-13 18:37:15.338709732 +0000 UTC m=+4.496651341 container cleanup c8f4947b120ffba2412be0786d1097022fb9a1a6e996ca9be9c3fc1cec61307c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-a, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-13T18:37:15.600 INFO:journalctl@ceph.mds.a.smithi114.stdout:Sep 13 18:37:15 smithi114 bash[111667]: ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-a 2024-09-13T18:37:15.774 DEBUG:teuthology.orchestra.run.smithi114:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mds.a.service' 2024-09-13T18:37:15.810 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T18:37:15.810 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-13T18:37:15.810 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-13T18:37:15.810 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mds.d 2024-09-13T18:37:16.147 INFO:journalctl@ceph.mds.d.smithi114.stdout:Sep 13 18:37:15 smithi114 systemd[1]: Stopping Ceph mds.d for 97e7edf2-71fb-11ef-bceb-c7b262605968... 2024-09-13T18:37:16.470 INFO:journalctl@ceph.mds.d.smithi114.stdout:Sep 13 18:37:16 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-d[70266]: 2024-09-13T18:37:16.144+0000 7fbcf4264640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T18:37:16.470 INFO:journalctl@ceph.mds.d.smithi114.stdout:Sep 13 18:37:16 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-d[70266]: 2024-09-13T18:37:16.144+0000 7fbcf4264640 -1 mds.d *** got signal Terminated *** 2024-09-13T18:37:19.242 INFO:journalctl@ceph.mds.d.smithi114.stdout:Sep 13 18:37:18 smithi114 podman[111801]: 2024-09-13 18:37:18.976975334 +0000 UTC m=+2.946713769 container died d50a04dd9db45efb7b47f511f3d0e4d5cc8dbf5579fefb3f954b6a5c4a03b01d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-d, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-13T18:37:19.520 INFO:journalctl@ceph.mds.d.smithi114.stdout:Sep 13 18:37:19 smithi114 podman[111801]: 2024-09-13 18:37:19.309435926 +0000 UTC m=+3.279174391 container cleanup d50a04dd9db45efb7b47f511f3d0e4d5cc8dbf5579fefb3f954b6a5c4a03b01d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-d, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-13T18:37:19.521 INFO:journalctl@ceph.mds.d.smithi114.stdout:Sep 13 18:37:19 smithi114 bash[111801]: ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-d 2024-09-13T18:37:19.746 DEBUG:teuthology.orchestra.run.smithi114:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mds.d.service' 2024-09-13T18:37:19.780 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T18:37:19.780 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-13T18:37:19.781 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-13T18:37:19.781 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mds.g 2024-09-13T18:37:20.090 INFO:journalctl@ceph.mds.g.smithi114.stdout:Sep 13 18:37:19 smithi114 systemd[1]: Stopping Ceph mds.g for 97e7edf2-71fb-11ef-bceb-c7b262605968... 2024-09-13T18:37:20.470 INFO:journalctl@ceph.mds.g.smithi114.stdout:Sep 13 18:37:20 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-g[70858]: 2024-09-13T18:37:20.087+0000 7efdd1d58640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T18:37:20.470 INFO:journalctl@ceph.mds.g.smithi114.stdout:Sep 13 18:37:20 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-g[70858]: 2024-09-13T18:37:20.087+0000 7efdd1d58640 -1 mds.g *** got signal Terminated *** 2024-09-13T18:37:23.720 INFO:journalctl@ceph.mds.g.smithi114.stdout:Sep 13 18:37:23 smithi114 podman[111923]: 2024-09-13 18:37:23.400534101 +0000 UTC m=+3.406788833 container died 15d44722d0352257ac825a8476f4550b182366d907fd53a6c26099ea2a4dd971 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-g, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-13T18:37:24.115 INFO:journalctl@ceph.mds.g.smithi114.stdout:Sep 13 18:37:23 smithi114 podman[111923]: 2024-09-13 18:37:23.721791125 +0000 UTC m=+3.728045859 container cleanup 15d44722d0352257ac825a8476f4550b182366d907fd53a6c26099ea2a4dd971 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-g, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-13T18:37:24.115 INFO:journalctl@ceph.mds.g.smithi114.stdout:Sep 13 18:37:23 smithi114 bash[111923]: ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-g 2024-09-13T18:37:24.115 INFO:journalctl@ceph.mds.g.smithi114.stdout:Sep 13 18:37:23 smithi114 podman[111935]: 2024-09-13 18:37:23.906589536 +0000 UTC m=+0.502778375 container remove 15d44722d0352257ac825a8476f4550b182366d907fd53a6c26099ea2a4dd971 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-g, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-13T18:37:24.128 DEBUG:teuthology.orchestra.run.smithi114:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mds.g.service' 2024-09-13T18:37:24.161 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T18:37:24.162 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-13T18:37:24.162 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-13T18:37:24.162 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mds.j 2024-09-13T18:37:24.469 INFO:journalctl@ceph.mds.j.smithi114.stdout:Sep 13 18:37:24 smithi114 systemd[1]: Stopping Ceph mds.j for 97e7edf2-71fb-11ef-bceb-c7b262605968... 2024-09-13T18:37:24.720 INFO:journalctl@ceph.mds.j.smithi114.stdout:Sep 13 18:37:24 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-j[69922]: 2024-09-13T18:37:24.467+0000 7fb55ab00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T18:37:24.720 INFO:journalctl@ceph.mds.j.smithi114.stdout:Sep 13 18:37:24 smithi114 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-j[69922]: 2024-09-13T18:37:24.467+0000 7fb55ab00640 -1 mds.j *** got signal Terminated *** 2024-09-13T18:37:28.720 INFO:journalctl@ceph.mds.j.smithi114.stdout:Sep 13 18:37:28 smithi114 podman[112043]: 2024-09-13 18:37:28.428819497 +0000 UTC m=+4.056844535 container died 4687664a4f360d8730d0b856f8dcb0ce2fdf89aaa5c1972491ef918dd1fed490 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-j, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-13T18:37:29.153 INFO:journalctl@ceph.mds.j.smithi114.stdout:Sep 13 18:37:28 smithi114 podman[112043]: 2024-09-13 18:37:28.875338248 +0000 UTC m=+4.503363294 container cleanup 4687664a4f360d8730d0b856f8dcb0ce2fdf89aaa5c1972491ef918dd1fed490 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-j, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083) 2024-09-13T18:37:29.153 INFO:journalctl@ceph.mds.j.smithi114.stdout:Sep 13 18:37:28 smithi114 bash[112043]: ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-j 2024-09-13T18:37:29.286 DEBUG:teuthology.orchestra.run.smithi114:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mds.j.service' 2024-09-13T18:37:29.318 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T18:37:29.319 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-13T18:37:29.319 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-13T18:37:29.319 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mds.b 2024-09-13T18:37:29.619 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 13 18:37:29 smithi134 systemd[1]: Stopping Ceph mds.b for 97e7edf2-71fb-11ef-bceb-c7b262605968... 2024-09-13T18:37:29.871 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 13 18:37:29 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-b[63915]: 2024-09-13T18:37:29.616+0000 7f1059514640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T18:37:29.871 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 13 18:37:29 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-b[63915]: 2024-09-13T18:37:29.616+0000 7f1059514640 -1 mds.b *** got signal Terminated *** 2024-09-13T18:37:31.645 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 13 18:37:31 smithi134 podman[70760]: 2024-09-13 18:37:31.387538671 +0000 UTC m=+1.880000926 container died 90dfb0348c834e5dac0df30a466f40ecabb783e9263d332ba0808114dc4990ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-13T18:37:32.028 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 13 18:37:31 smithi134 podman[70760]: 2024-09-13 18:37:31.741578187 +0000 UTC m=+2.234040442 container cleanup 90dfb0348c834e5dac0df30a466f40ecabb783e9263d332ba0808114dc4990ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-13T18:37:32.028 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 13 18:37:31 smithi134 bash[70760]: ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-b 2024-09-13T18:37:32.173 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mds.b.service' 2024-09-13T18:37:32.209 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T18:37:32.209 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-13T18:37:32.209 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-13T18:37:32.209 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mds.e 2024-09-13T18:37:32.619 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 13 18:37:32 smithi134 systemd[1]: Stopping Ceph mds.e for 97e7edf2-71fb-11ef-bceb-c7b262605968... 2024-09-13T18:37:32.619 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 13 18:37:32 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-e[63289]: 2024-09-13T18:37:32.528+0000 7fac25b88640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T18:37:32.619 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 13 18:37:32 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-e[63289]: 2024-09-13T18:37:32.528+0000 7fac25b88640 -1 mds.e *** got signal Terminated *** 2024-09-13T18:37:35.689 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 13 18:37:35 smithi134 podman[70892]: 2024-09-13 18:37:35.377007123 +0000 UTC m=+2.952388340 container died b4bc60a2e73a3a2152ec93200f3558c89a41607330006d53a2a267405432ff36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-e, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-13T18:37:35.689 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 13 18:37:35 smithi134 podman[70892]: 2024-09-13 18:37:35.687008198 +0000 UTC m=+3.262389407 container cleanup b4bc60a2e73a3a2152ec93200f3558c89a41607330006d53a2a267405432ff36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-e, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-13T18:37:36.094 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 13 18:37:35 smithi134 bash[70892]: ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-e 2024-09-13T18:37:36.094 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 13 18:37:35 smithi134 podman[70904]: 2024-09-13 18:37:35.897083419 +0000 UTC m=+0.509666226 container remove b4bc60a2e73a3a2152ec93200f3558c89a41607330006d53a2a267405432ff36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-e, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-13T18:37:36.117 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mds.e.service' 2024-09-13T18:37:36.151 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T18:37:36.151 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-13T18:37:36.151 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-13T18:37:36.152 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mds.h 2024-09-13T18:37:36.369 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 13 18:37:36 smithi134 systemd[1]: Stopping Ceph mds.h for 97e7edf2-71fb-11ef-bceb-c7b262605968... 2024-09-13T18:37:36.869 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 13 18:37:36 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-h[63594]: 2024-09-13T18:37:36.474+0000 7f48b25ca640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T18:37:37.267 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 13 18:37:36 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-h[63594]: 2024-09-13T18:37:36.474+0000 7f48b25ca640 -1 mds.h *** got signal Terminated *** 2024-09-13T18:37:39.685 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 13 18:37:39 smithi134 podman[71015]: 2024-09-13 18:37:39.393138506 +0000 UTC m=+3.022219901 container died a14a3375eea7e014f462bc28e778dd55377d4e3ab5b6473213c6962dbd7cdc1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-h, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-13T18:37:40.036 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 13 18:37:39 smithi134 podman[71015]: 2024-09-13 18:37:39.766171857 +0000 UTC m=+3.395253243 container cleanup a14a3375eea7e014f462bc28e778dd55377d4e3ab5b6473213c6962dbd7cdc1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-h, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-13T18:37:40.036 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 13 18:37:39 smithi134 bash[71015]: ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-h 2024-09-13T18:37:40.186 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mds.h.service' 2024-09-13T18:37:40.220 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T18:37:40.221 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-13T18:37:40.221 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-13T18:37:40.221 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mds.k 2024-09-13T18:37:40.614 INFO:journalctl@ceph.mds.k.smithi134.stdout:Sep 13 18:37:40 smithi134 systemd[1]: Stopping Ceph mds.k for 97e7edf2-71fb-11ef-bceb-c7b262605968... 2024-09-13T18:37:40.869 INFO:journalctl@ceph.mds.k.smithi134.stdout:Sep 13 18:37:40 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-k[62972]: 2024-09-13T18:37:40.610+0000 7ff1dd832640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T18:37:40.869 INFO:journalctl@ceph.mds.k.smithi134.stdout:Sep 13 18:37:40 smithi134 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-k[62972]: 2024-09-13T18:37:40.610+0000 7ff1dd832640 -1 mds.k *** got signal Terminated *** 2024-09-13T18:37:43.657 INFO:journalctl@ceph.mds.k.smithi134.stdout:Sep 13 18:37:43 smithi134 podman[71135]: 2024-09-13 18:37:43.396839666 +0000 UTC m=+2.957516254 container died b3714723861eca8d91445079a991555a6fa19dbdef6af06b2784b8dea24ba285 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-k, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-13T18:37:44.022 INFO:journalctl@ceph.mds.k.smithi134.stdout:Sep 13 18:37:43 smithi134 podman[71135]: 2024-09-13 18:37:43.744569332 +0000 UTC m=+3.305245908 container cleanup b3714723861eca8d91445079a991555a6fa19dbdef6af06b2784b8dea24ba285 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-k, ceph=True, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-13T18:37:44.022 INFO:journalctl@ceph.mds.k.smithi134.stdout:Sep 13 18:37:43 smithi134 bash[71135]: ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-k 2024-09-13T18:37:44.176 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mds.k.service' 2024-09-13T18:37:44.209 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T18:37:44.209 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-13T18:37:44.210 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-13T18:37:44.210 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mds.c 2024-09-13T18:37:44.548 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 13 18:37:44 smithi204 systemd[1]: Stopping Ceph mds.c for 97e7edf2-71fb-11ef-bceb-c7b262605968... 2024-09-13T18:37:44.944 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 13 18:37:44 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-c[64760]: 2024-09-13T18:37:44.545+0000 7fedce455640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T18:37:44.944 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 13 18:37:44 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-c[64760]: 2024-09-13T18:37:44.545+0000 7fedce455640 -1 mds.c *** got signal Terminated *** 2024-09-13T18:37:47.752 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 13 18:37:47 smithi204 podman[72029]: 2024-09-13 18:37:47.460266564 +0000 UTC m=+3.023425648 container died 9a125d62aba44f255ccf980327a9f39d806369fd92a803eea120457f7e46ca1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-c, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-13T18:37:48.106 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 13 18:37:47 smithi204 podman[72029]: 2024-09-13 18:37:47.836316912 +0000 UTC m=+3.399475989 container cleanup 9a125d62aba44f255ccf980327a9f39d806369fd92a803eea120457f7e46ca1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-c, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-13T18:37:48.106 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 13 18:37:47 smithi204 bash[72029]: ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-c 2024-09-13T18:37:48.227 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mds.c.service' 2024-09-13T18:37:48.263 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T18:37:48.263 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-13T18:37:48.263 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-13T18:37:48.263 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mds.f 2024-09-13T18:37:48.444 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 13 18:37:48 smithi204 systemd[1]: Stopping Ceph mds.f for 97e7edf2-71fb-11ef-bceb-c7b262605968... 2024-09-13T18:37:48.944 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 13 18:37:48 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-f[64457]: 2024-09-13T18:37:48.582+0000 7fa0aa340640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T18:37:48.944 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 13 18:37:48 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-f[64457]: 2024-09-13T18:37:48.582+0000 7fa0aa340640 -1 mds.f *** got signal Terminated *** 2024-09-13T18:37:51.693 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 13 18:37:51 smithi204 podman[72160]: 2024-09-13 18:37:51.397453206 +0000 UTC m=+2.916780987 container died 047c0dcb239a088987f2da965ca414ada76ec41e4ef1e2dee16c5946b9095fea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-f, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-09-13T18:37:51.944 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 13 18:37:51 smithi204 podman[72160]: 2024-09-13 18:37:51.774020961 +0000 UTC m=+3.293348738 container cleanup 047c0dcb239a088987f2da965ca414ada76ec41e4ef1e2dee16c5946b9095fea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-f, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-13T18:37:51.944 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 13 18:37:51 smithi204 bash[72160]: ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-f 2024-09-13T18:37:52.187 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mds.f.service' 2024-09-13T18:37:52.221 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T18:37:52.221 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-13T18:37:52.221 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-13T18:37:52.221 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mds.i 2024-09-13T18:37:52.444 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 13 18:37:52 smithi204 systemd[1]: Stopping Ceph mds.i for 97e7edf2-71fb-11ef-bceb-c7b262605968... 2024-09-13T18:37:52.944 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 13 18:37:52 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-i[65062]: 2024-09-13T18:37:52.542+0000 7f858b634640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T18:37:52.944 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 13 18:37:52 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-i[65062]: 2024-09-13T18:37:52.542+0000 7f858b634640 -1 mds.i *** got signal Terminated *** 2024-09-13T18:37:55.647 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 13 18:37:55 smithi204 podman[72281]: 2024-09-13 18:37:55.381603253 +0000 UTC m=+2.941709339 container died aecc8a6696be291f6a46b1184f3c2186c4e73a9bce18e359fda5f33b9459d07f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-13T18:37:55.944 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 13 18:37:55 smithi204 podman[72281]: 2024-09-13 18:37:55.744247786 +0000 UTC m=+3.304353865 container cleanup aecc8a6696be291f6a46b1184f3c2186c4e73a9bce18e359fda5f33b9459d07f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-i, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-13T18:37:55.944 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 13 18:37:55 smithi204 bash[72281]: ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-i 2024-09-13T18:37:56.165 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mds.i.service' 2024-09-13T18:37:56.199 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T18:37:56.200 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-13T18:37:56.200 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-13T18:37:56.200 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mds.l 2024-09-13T18:37:56.444 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 13 18:37:56 smithi204 systemd[1]: Stopping Ceph mds.l for 97e7edf2-71fb-11ef-bceb-c7b262605968... 2024-09-13T18:37:56.944 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 13 18:37:56 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-l[64136]: 2024-09-13T18:37:56.539+0000 7fe18f908640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-13T18:37:56.944 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 13 18:37:56 smithi204 ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-l[64136]: 2024-09-13T18:37:56.539+0000 7fe18f908640 -1 mds.l *** got signal Terminated *** 2024-09-13T18:37:59.694 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 13 18:37:59 smithi204 podman[72402]: 2024-09-13 18:37:59.398051549 +0000 UTC m=+2.986290042 container died 11004bae139144d39017446d0ecaab4ee5f777648e31522391fd08903a5d7703 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-l, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-13T18:38:00.017 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 13 18:37:59 smithi204 podman[72402]: 2024-09-13 18:37:59.789942163 +0000 UTC m=+3.378180656 container cleanup 11004bae139144d39017446d0ecaab4ee5f777648e31522391fd08903a5d7703 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:71670858e25918b847e38c5069e48a496647b62c9399b70b7ae1aa13108cb256, name=ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-l, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240910.100730-reef-6c6b083, io.buildah.version=1.37.2) 2024-09-13T18:38:00.018 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 13 18:37:59 smithi204 bash[72402]: ceph-97e7edf2-71fb-11ef-bceb-c7b262605968-mds-l 2024-09-13T18:38:00.140 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-97e7edf2-71fb-11ef-bceb-c7b262605968@mds.l.service' 2024-09-13T18:38:00.172 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-13T18:38:00.172 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-13T18:38:00.172 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-13T18:38:00.325 INFO:teuthology.orchestra.run.smithi114.stdout:Deleting cluster with fsid: 97e7edf2-71fb-11ef-bceb-c7b262605968 2024-09-13T18:38:02.661 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-13T18:38:02.816 INFO:teuthology.orchestra.run.smithi134.stdout:Deleting cluster with fsid: 97e7edf2-71fb-11ef-bceb-c7b262605968 2024-09-13T18:38:05.126 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-13T18:38:05.279 INFO:teuthology.orchestra.run.smithi204.stdout:Deleting cluster with fsid: 97e7edf2-71fb-11ef-bceb-c7b262605968 2024-09-13T18:38:07.547 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-13T18:38:07.580 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-13T18:38:07.613 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-13T18:38:07.641 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-13T18:38:07.643 DEBUG:teuthology.misc:Transferring archived files from smithi114:/var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903964/remote/smithi114/crash 2024-09-13T18:38:07.644 DEBUG:teuthology.orchestra.run.smithi114:> sudo tar c -f - -C /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/crash -- . 2024-09-13T18:38:07.675 INFO:teuthology.orchestra.run.smithi114.stderr:tar: /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-13T18:38:07.676 INFO:teuthology.orchestra.run.smithi114.stderr:tar: Error is not recoverable: exiting now 2024-09-13T18:38:07.677 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903964/remote/smithi134/crash 2024-09-13T18:38:07.678 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/crash -- . 2024-09-13T18:38:07.751 INFO:teuthology.orchestra.run.smithi134.stderr:tar: /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-13T18:38:07.751 INFO:teuthology.orchestra.run.smithi134.stderr:tar: Error is not recoverable: exiting now 2024-09-13T18:38:07.754 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903964/remote/smithi204/crash 2024-09-13T18:38:07.754 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/crash -- . 2024-09-13T18:38:07.796 INFO:teuthology.orchestra.run.smithi204.stderr:tar: /var/lib/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-13T18:38:07.796 INFO:teuthology.orchestra.run.smithi204.stderr:tar: Error is not recoverable: exiting now 2024-09-13T18:38:07.798 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-13T18:38:07.798 DEBUG:teuthology.orchestra.run.smithi114:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-13T18:38:07.830 INFO:teuthology.orchestra.run.smithi114.stderr:grep: /var/log/ceph/97e7edf2-71fb-11ef-bceb-c7b262605968/ceph.log: No such file or directory 2024-09-13T18:38:07.834 INFO:tasks.cephadm:Compressing logs... 2024-09-13T18:38:07.834 DEBUG:teuthology.orchestra.run.smithi114:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T18:38:07.877 DEBUG:teuthology.orchestra.run.smithi134:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T18:38:07.879 DEBUG:teuthology.orchestra.run.smithi204:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T18:38:07.908 INFO:teuthology.orchestra.run.smithi204.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-13T18:38:07.910 INFO:teuthology.orchestra.run.smithi114.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-13T18:38:07.912 INFO:teuthology.orchestra.run.smithi134.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-13T18:47:11.424 INFO:tasks.cephadm:Archiving logs... 2024-09-13T18:47:11.444 DEBUG:teuthology.misc:Transferring archived files from smithi114:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903964/remote/smithi114/log 2024-09-13T18:47:11.445 DEBUG:teuthology.orchestra.run.smithi114:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T18:48:20.515 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903964/remote/smithi134/log 2024-09-13T18:48:20.517 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T18:49:06.294 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903964/remote/smithi204/log 2024-09-13T18:49:06.295 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T18:49:55.884 INFO:tasks.cephadm:Removing cluster... 2024-09-13T18:49:55.885 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 --force 2024-09-13T18:49:56.037 INFO:teuthology.orchestra.run.smithi114.stdout:Deleting cluster with fsid: 97e7edf2-71fb-11ef-bceb-c7b262605968 2024-09-13T18:49:56.745 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 --force 2024-09-13T18:49:56.898 INFO:teuthology.orchestra.run.smithi134.stdout:Deleting cluster with fsid: 97e7edf2-71fb-11ef-bceb-c7b262605968 2024-09-13T18:49:57.565 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 97e7edf2-71fb-11ef-bceb-c7b262605968 --force 2024-09-13T18:49:57.718 INFO:teuthology.orchestra.run.smithi204.stdout:Deleting cluster with fsid: 97e7edf2-71fb-11ef-bceb-c7b262605968 2024-09-13T18:49:58.361 INFO:tasks.cephadm:Removing cephadm ... 2024-09-13T18:49:58.361 DEBUG:teuthology.orchestra.run.smithi114:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-13T18:49:58.379 DEBUG:teuthology.orchestra.run.smithi134:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-13T18:49:58.398 DEBUG:teuthology.orchestra.run.smithi204:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-13T18:49:58.413 INFO:tasks.cephadm:Teardown complete 2024-09-13T18:49:58.414 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-13T18:49:58.424 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-13T18:49:58.425 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T18:49:58.427 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T18:49:58.441 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T18:49:58.585 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-13T18:49:58.595 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-13T18:49:58.596 DEBUG:teuthology.orchestra.run.smithi114:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T18:49:58.599 DEBUG:teuthology.orchestra.run.smithi134:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T18:49:58.601 DEBUG:teuthology.orchestra.run.smithi204:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T18:49:58.611 INFO:teuthology.orchestra.run.smithi114.stderr:bash: line 1: ntpq: command not found 2024-09-13T18:49:58.615 INFO:teuthology.orchestra.run.smithi114.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-13T18:49:58.615 INFO:teuthology.orchestra.run.smithi114.stdout:=============================================================================== 2024-09-13T18:49:58.615 INFO:teuthology.orchestra.run.smithi114.stdout:^* hv01.front.sepia.ceph.com 3 6 377 0 -1200us[-1177us] +/- 36ms 2024-09-13T18:49:58.615 INFO:teuthology.orchestra.run.smithi114.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 64 +3154us[+3176us] +/- 57ms 2024-09-13T18:49:58.616 INFO:teuthology.orchestra.run.smithi114.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 1 -1414us[-1391us] +/- 70ms 2024-09-13T18:49:58.616 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T18:49:58.616 INFO:teuthology.orchestra.run.smithi204.stderr:bash: line 1: ntpq: command not found 2024-09-13T18:49:58.618 INFO:teuthology.orchestra.run.smithi134.stderr:bash: line 1: ntpq: command not found 2024-09-13T18:49:58.631 INFO:teuthology.orchestra.run.smithi204.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-13T18:49:58.632 INFO:teuthology.orchestra.run.smithi204.stdout:=============================================================================== 2024-09-13T18:49:58.632 INFO:teuthology.orchestra.run.smithi204.stdout:^* hv01.front.sepia.ceph.com 3 6 377 63 -1470us[-1075us] +/- 66ms 2024-09-13T18:49:58.632 INFO:teuthology.orchestra.run.smithi204.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 2 +2928us[+2928us] +/- 58ms 2024-09-13T18:49:58.632 INFO:teuthology.orchestra.run.smithi204.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 1 -1637us[-1637us] +/- 70ms 2024-09-13T18:49:58.632 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T18:49:58.637 INFO:teuthology.orchestra.run.smithi134.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-13T18:49:58.638 INFO:teuthology.orchestra.run.smithi134.stdout:=============================================================================== 2024-09-13T18:49:58.638 INFO:teuthology.orchestra.run.smithi134.stdout:^* hv01.front.sepia.ceph.com 3 6 377 2 -1174us[-1434us] +/- 36ms 2024-09-13T18:49:58.638 INFO:teuthology.orchestra.run.smithi134.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 4 +3189us[+2929us] +/- 58ms 2024-09-13T18:49:58.638 INFO:teuthology.orchestra.run.smithi134.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 2 -1393us[-1653us] +/- 70ms 2024-09-13T18:49:58.638 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T18:49:58.639 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-13T18:49:58.649 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-13T18:49:58.650 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-13T18:49:58.661 DEBUG:teuthology.orchestra.run.smithi114:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-13T18:49:58.786 DEBUG:teuthology.orchestra.run.smithi134:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-13T18:49:58.907 DEBUG:teuthology.orchestra.run.smithi204:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-13T18:49:58.998 DEBUG:teuthology.orchestra.run.smithi114:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-13T18:49:59.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:49:59.027 DEBUG:teuthology.orchestra.run.smithi134:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-13T18:49:59.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:49:59.059 DEBUG:teuthology.orchestra.run.smithi204:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-13T18:49:59.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:49:59.087 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-13T18:49:59.099 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-13T18:49:59.114 INFO:teuthology.task.internal:Duration was 2633.374952 seconds 2024-09-13T18:49:59.114 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-13T18:49:59.124 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-13T18:49:59.125 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T18:49:59.127 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T18:49:59.129 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T18:49:59.185 INFO:teuthology.orchestra.run.smithi204.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-13T18:49:59.202 INFO:teuthology.orchestra.run.smithi134.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-13T18:49:59.218 INFO:teuthology.orchestra.run.smithi114.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-13T18:49:59.676 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-13T18:49:59.676 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi114.front.sepia.ceph.com 2024-09-13T18:49:59.676 DEBUG:teuthology.orchestra.run.smithi114:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T18:49:59.696 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi134.front.sepia.ceph.com 2024-09-13T18:49:59.697 DEBUG:teuthology.orchestra.run.smithi134:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T18:49:59.732 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi204.front.sepia.ceph.com 2024-09-13T18:49:59.732 DEBUG:teuthology.orchestra.run.smithi204:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T18:49:59.762 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-13T18:49:59.763 DEBUG:teuthology.orchestra.run.smithi114:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T18:49:59.765 DEBUG:teuthology.orchestra.run.smithi134:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T18:49:59.775 DEBUG:teuthology.orchestra.run.smithi204:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T18:49:59.830 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-13T18:49:59.830 DEBUG:teuthology.orchestra.run.smithi114:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T18:49:59.833 DEBUG:teuthology.orchestra.run.smithi134:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T18:49:59.845 DEBUG:teuthology.orchestra.run.smithi204:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T18:50:00.282 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-13T18:50:00.294 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-13T18:50:00.294 DEBUG:teuthology.orchestra.run.smithi114:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T18:50:00.325 DEBUG:teuthology.orchestra.run.smithi134:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T18:50:00.362 DEBUG:teuthology.orchestra.run.smithi204:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T18:50:00.425 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-13T18:50:00.436 DEBUG:teuthology.orchestra.run.smithi114:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T18:50:00.438 DEBUG:teuthology.orchestra.run.smithi134:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T18:50:00.441 DEBUG:teuthology.orchestra.run.smithi204:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T18:50:00.465 INFO:teuthology.orchestra.run.smithi114.stdout:kernel.core_pattern = core 2024-09-13T18:50:00.469 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern = core 2024-09-13T18:50:00.473 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern = core 2024-09-13T18:50:00.497 DEBUG:teuthology.orchestra.run.smithi114:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T18:50:00.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:50:00.551 DEBUG:teuthology.orchestra.run.smithi134:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T18:50:00.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:50:00.566 DEBUG:teuthology.orchestra.run.smithi204:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T18:50:00.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:50:00.581 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-13T18:50:00.593 INFO:teuthology.task.internal:Transferring archived files... 2024-09-13T18:50:00.594 DEBUG:teuthology.misc:Transferring archived files from smithi114:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903964/remote/smithi114 2024-09-13T18:50:00.594 DEBUG:teuthology.orchestra.run.smithi114:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T18:50:00.637 DEBUG:teuthology.misc:Transferring archived files from smithi134:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903964/remote/smithi134 2024-09-13T18:50:00.638 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T18:50:00.674 DEBUG:teuthology.misc:Transferring archived files from smithi204:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903964/remote/smithi204 2024-09-13T18:50:00.674 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T18:50:00.711 INFO:teuthology.task.internal:Removing archive directory... 2024-09-13T18:50:00.711 DEBUG:teuthology.orchestra.run.smithi114:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T18:50:00.714 DEBUG:teuthology.orchestra.run.smithi134:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T18:50:00.716 DEBUG:teuthology.orchestra.run.smithi204:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T18:50:00.760 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-13T18:50:00.774 INFO:teuthology.task.internal:Not uploading archives. 2024-09-13T18:50:00.774 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-13T18:50:00.785 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-13T18:50:00.785 DEBUG:teuthology.orchestra.run.smithi114:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T18:50:00.788 DEBUG:teuthology.orchestra.run.smithi134:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T18:50:00.790 DEBUG:teuthology.orchestra.run.smithi204:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T18:50:00.801 INFO:teuthology.orchestra.run.smithi114.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 18:50 /home/ubuntu/cephtest 2024-09-13T18:50:00.802 INFO:teuthology.orchestra.run.smithi134.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 18:50 /home/ubuntu/cephtest 2024-09-13T18:50:00.815 INFO:teuthology.orchestra.run.smithi204.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 18:50 /home/ubuntu/cephtest 2024-09-13T18:50:00.817 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-13T18:50:00.830 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-13T18:50:00.857 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/iogen}} duration: 2633.374952316284 flavor: default owner: scheduled_rishabh@teuthology success: true 2024-09-13T18:50:00.858 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-13T18:50:00.951 INFO:teuthology.run:pass